2024-09-10T07:45:37.093 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T07:45:37.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T07:45:37.215 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898886 branch: wip-rishabh-testing-20240909.165402-squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: null first_in_suite: false job_id: '7898886' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi002.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGZUNuh+fNUKRn9MUQA2wdxMILMgOOmQ9TwxzWl6bN93g/yOIt3nqYHJotbCbog7m1DyHcLanqAbjcaiynfXuwQ= smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHydZgG0Q8TUCSI30pTkj1ypbIQsOt4MySVMrl1eZOnyCo8qFsSdjcL4IgsL1bDxsnzbIqMx/dChVoA5ri7xp7M= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFTBdbqeOwMTrFMQ+BO6LgUrNSdubDZ5MNb585YtbRt2cIVfA/qex5yjJPhIhkEZAo381dy6lVlMCxFlhpNlBbk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T07:45:37.215 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T07:45:37.216 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T07:45:37.216 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T07:45:37.218 INFO:teuthology.task.internal:Checking packages... 2024-09-10T07:45:37.238 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T07:45:37.238 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T07:45:37.238 INFO:teuthology.packaging:ref: None 2024-09-10T07:45:37.238 INFO:teuthology.packaging:tag: None 2024-09-10T07:45:37.238 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T07:45:37.238 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:45:37.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T07:45:37.378 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T07:45:37.379 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T07:45:37.385 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T07:45:37.386 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T07:45:37.392 INFO:teuthology.task.internal:Saving configuration 2024-09-10T07:45:37.412 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T07:45:37.419 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T07:45:37.444 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898886', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 07:40:23.783165', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGZUNuh+fNUKRn9MUQA2wdxMILMgOOmQ9TwxzWl6bN93g/yOIt3nqYHJotbCbog7m1DyHcLanqAbjcaiynfXuwQ='} 2024-09-10T07:45:37.465 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898886', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 07:40:23.784900', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHydZgG0Q8TUCSI30pTkj1ypbIQsOt4MySVMrl1eZOnyCo8qFsSdjcL4IgsL1bDxsnzbIqMx/dChVoA5ri7xp7M='} 2024-09-10T07:45:37.488 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898886', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 07:40:23.781220', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFTBdbqeOwMTrFMQ+BO6LgUrNSdubDZ5MNb585YtbRt2cIVfA/qex5yjJPhIhkEZAo381dy6lVlMCxFlhpNlBbk='} 2024-09-10T07:45:37.488 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T07:45:37.495 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T07:45:37.495 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T07:45:37.495 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T07:45:37.495 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T07:45:37.582 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc919087be0>, signals=[15]) 2024-09-10T07:45:37.582 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T07:45:37.589 INFO:teuthology.task.internal:Opening connections... 2024-09-10T07:45:37.589 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi002.front.sepia.ceph.com 2024-09-10T07:45:37.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:45:37.666 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-09-10T07:45:37.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:45:37.741 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-09-10T07:45:37.741 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:45:37.817 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T07:45:37.824 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2024-09-10T07:45:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2024-09-10T07:45:37.843 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2024-09-10T07:45:37.900 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2024-09-10T07:45:37.900 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9" 2024-09-10T07:45:37.900 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2024-09-10T07:45:37.900 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2024-09-10T07:45:37.901 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9" 2024-09-10T07:45:37.901 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9" 2024-09-10T07:45:37.901 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T07:45:37.901 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2024-09-10T07:45:37.901 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon" 2024-09-10T07:45:37.901 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T07:45:37.901 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2024-09-10T07:45:37.901 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T07:45:37.901 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T07:45:37.901 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T07:45:37.902 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2024-09-10T07:45:37.918 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-09-10T07:45:37.937 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-09-10T07:45:37.937 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-09-10T07:45:37.995 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-09-10T07:45:37.995 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-09-10T07:45:37.995 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-09-10T07:45:37.995 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-09-10T07:45:37.995 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-09-10T07:45:37.995 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-09-10T07:45:37.995 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T07:45:37.996 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-09-10T07:45:37.996 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-09-10T07:45:37.996 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T07:45:37.996 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-09-10T07:45:37.996 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T07:45:37.996 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T07:45:37.996 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T07:45:37.996 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-09-10T07:45:38.019 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-09-10T07:45:38.037 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-09-10T07:45:38.038 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-09-10T07:45:38.096 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2024-09-10T07:45:38.096 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9" 2024-09-10T07:45:38.097 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2024-09-10T07:45:38.097 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2024-09-10T07:45:38.097 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9" 2024-09-10T07:45:38.097 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9" 2024-09-10T07:45:38.097 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T07:45:38.097 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2024-09-10T07:45:38.097 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon" 2024-09-10T07:45:38.097 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T07:45:38.097 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2024-09-10T07:45:38.097 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T07:45:38.097 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T07:45:38.097 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T07:45:38.098 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-09-10T07:45:38.118 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T07:45:38.129 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T07:45:38.134 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T07:45:38.135 DEBUG:teuthology.orchestra.run.smithi002:> test '!' -e /home/ubuntu/cephtest 2024-09-10T07:45:38.137 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-09-10T07:45:38.139 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-09-10T07:45:38.156 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T07:45:38.163 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T07:45:38.163 DEBUG:teuthology.orchestra.run.smithi002:> test -z $(ls -A /var/lib/ceph) 2024-09-10T07:45:38.194 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-09-10T07:45:38.196 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-09-10T07:45:38.302 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T07:45:38.368 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T07:45:38.383 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-10T07:45:38.383 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-10T07:45:38.383 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-10T07:45:38.384 DEBUG:teuthology.orchestra.run.smithi002:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T07:45:38.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:45:38.402 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-10T07:45:38.654 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T07:45:38.655 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T07:45:38.655 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-09-10T07:45:38.672 INFO:teuthology.orchestra.run.smithi002.stdout:5.14.0-503.el9.x86_64 2024-09-10T07:45:38.673 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T07:45:38.673 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T07:45:38.674 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum upgrade -y linux-firmware 2024-09-10T07:45:42.503 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.2 MB 00:02 2024-09-10T07:45:46.762 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 20 MB 00:03 2024-09-10T07:45:54.732 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.5 MB 00:03 2024-09-10T07:45:57.038 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-10T07:45:58.067 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-10T07:46:03.690 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-09-10T07:46:05.869 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-10T07:46:05.870 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-09-10T07:46:05.870 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-10T07:46:05.969 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T07:46:05.970 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T07:46:06.123 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T07:46:06.124 DEBUG:teuthology.orchestra.run.smithi002:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T07:46:06.127 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi002.front.sepia.ceph.com to complete... 2024-09-10T07:46:12.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10 07:46:12 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T07:46:12.438 INFO:teuthology.orchestra.run.smithi002.stderr:FINISHED --2024-09-10 07:46:12-- 2024-09-10T07:46:12.438 INFO:teuthology.orchestra.run.smithi002.stderr:Total wall clock time: 6.3s 2024-09-10T07:46:12.438 INFO:teuthology.orchestra.run.smithi002.stderr:Downloaded: 1 files, 343M in 6.0s (57.1 MB/s) 2024-09-10T07:46:12.441 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T07:46:12.441 DEBUG:teuthology.orchestra.run.smithi002:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T07:46:14.651 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying... ######################################## 2024-09-10T07:46:14.804 INFO:teuthology.orchestra.run.smithi002.stdout:Preparing... ######################################## 2024-09-10T07:46:14.920 INFO:teuthology.orchestra.run.smithi002.stdout:Updating / installing... 2024-09-10T07:46:23.249 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T07:47:03.146 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi002.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T07:47:03.147 DEBUG:teuthology.orchestra.run.smithi002:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T07:47:03.233 INFO:teuthology.orchestra.run.smithi002.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:47:03.233 INFO:teuthology.orchestra.run.smithi002.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:47:03.234 INFO:teuthology.orchestra.run.smithi002.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:47:03.234 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:47:03.234 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T07:47:03.234 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T07:47:03.234 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T07:47:03.234 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T07:47:03.234 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T07:47:03.234 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T07:47:03.234 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T07:47:03.234 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T07:47:03.234 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T07:47:03.234 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T07:47:03.235 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T07:47:03.235 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T07:47:03.235 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T07:47:03.235 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T07:47:03.235 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T07:47:03.235 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T07:47:03.235 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T07:47:03.235 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T07:47:03.235 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T07:47:03.235 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T07:47:03.235 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T07:47:03.235 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T07:47:03.235 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T07:47:03.236 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T07:47:03.236 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T07:47:03.236 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T07:47:03.236 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T07:47:03.236 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T07:47:03.236 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T07:47:03.236 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T07:47:03.236 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T07:47:03.236 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T07:47:03.236 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T07:47:03.236 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T07:47:03.236 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T07:47:03.237 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T07:47:03.237 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T07:47:03.237 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T07:47:03.237 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T07:47:03.237 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T07:47:03.237 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T07:47:03.237 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T07:47:03.237 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T07:47:03.237 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T07:47:03.237 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T07:47:03.237 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T07:47:03.237 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T07:47:03.238 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T07:47:03.238 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T07:47:03.238 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T07:47:03.238 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T07:47:03.238 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T07:47:03.238 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T07:47:03.238 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T07:47:03.238 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T07:47:03.238 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T07:47:03.238 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T07:47:03.238 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T07:47:03.238 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T07:47:03.238 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T07:47:03.239 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T07:47:03.239 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T07:47:03.239 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T07:47:03.239 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T07:47:03.239 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T07:47:03.239 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T07:47:03.239 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T07:47:03.239 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T07:47:03.239 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T07:47:03.239 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T07:47:03.239 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T07:47:03.239 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T07:47:03.239 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T07:47:03.240 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T07:47:03.240 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T07:47:03.240 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T07:47:03.240 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T07:47:03.240 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T07:47:03.240 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T07:47:03.240 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T07:47:03.240 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T07:47:03.240 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T07:47:03.240 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T07:47:03.240 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T07:47:03.240 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T07:47:03.240 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T07:47:03.241 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T07:47:03.241 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T07:47:03.241 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T07:47:03.241 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T07:47:03.241 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T07:47:03.241 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T07:47:03.241 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T07:47:03.241 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T07:47:03.241 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T07:47:03.241 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T07:47:03.241 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T07:47:03.241 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T07:47:03.241 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T07:47:03.242 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T07:47:03.242 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T07:47:03.242 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T07:47:03.242 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T07:47:03.242 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T07:47:03.242 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T07:47:03.242 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T07:47:03.242 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T07:47:03.242 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T07:47:03.242 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T07:47:03.242 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T07:47:03.242 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T07:47:03.242 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T07:47:03.243 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T07:47:03.243 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T07:47:03.243 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T07:47:03.243 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T07:47:03.243 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T07:47:03.243 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T07:47:03.243 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T07:47:03.243 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T07:47:03.243 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T07:47:03.243 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T07:47:03.243 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T07:47:03.243 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T07:47:03.244 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T07:47:03.244 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T07:47:03.244 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T07:47:03.244 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T07:47:03.244 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T07:47:03.244 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T07:47:03.244 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T07:47:03.244 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T07:47:03.244 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T07:47:03.244 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T07:47:03.244 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T07:47:03.244 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T07:47:03.244 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T07:47:03.245 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T07:47:03.246 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T07:47:03.246 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T07:47:03.246 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T07:47:03.246 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T07:47:03.246 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T07:47:03.246 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T07:47:03.246 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T07:47:03.246 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T07:47:03.246 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T07:47:03.246 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T07:47:03.246 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T07:47:03.246 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T07:47:03.246 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T07:47:03.247 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T07:47:03.247 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T07:47:03.247 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T07:47:03.247 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T07:47:03.247 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T07:47:03.247 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T07:47:03.247 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T07:47:03.247 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T07:47:03.247 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T07:47:03.247 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T07:47:03.247 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T07:47:03.247 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T07:47:03.247 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T07:47:03.248 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T07:47:03.248 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T07:47:03.248 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T07:47:03.248 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T07:47:03.248 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T07:47:03.248 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T07:47:03.248 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T07:47:03.248 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T07:47:03.248 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T07:47:03.248 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T07:47:03.248 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T07:47:03.248 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T07:47:03.248 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T07:47:03.249 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T07:47:03.249 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T07:47:03.249 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T07:47:03.249 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T07:47:03.249 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T07:47:03.249 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T07:47:03.249 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T07:47:03.249 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T07:47:03.249 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T07:47:03.249 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T07:47:03.249 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T07:47:03.249 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T07:47:03.249 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T07:47:03.250 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T07:47:03.250 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T07:47:03.250 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T07:47:03.250 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T07:47:03.250 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T07:47:03.250 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T07:47:03.250 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T07:47:03.250 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T07:47:03.250 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T07:47:03.251 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T07:47:03.251 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T07:47:03.251 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T07:47:03.251 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T07:47:03.251 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T07:47:03.251 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T07:47:03.251 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T07:47:03.251 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T07:47:03.251 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T07:47:03.251 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T07:47:03.251 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T07:47:03.251 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T07:47:03.251 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T07:47:03.252 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T07:47:03.252 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T07:47:03.252 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T07:47:03.252 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T07:47:03.252 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T07:47:03.252 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T07:47:03.252 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T07:47:03.252 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T07:47:03.252 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T07:47:03.252 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T07:47:03.252 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T07:47:03.252 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T07:47:03.253 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T07:47:03.253 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T07:47:03.253 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T07:47:03.253 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T07:47:03.253 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T07:47:03.253 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T07:47:03.253 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T07:47:03.253 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T07:47:03.253 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T07:47:03.253 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T07:47:03.253 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T07:47:03.253 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T07:47:03.254 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T07:47:03.254 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T07:47:03.254 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T07:47:03.254 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T07:47:03.254 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T07:47:03.254 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T07:47:03.254 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T07:47:03.254 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T07:47:03.254 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T07:47:03.254 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T07:47:03.254 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T07:47:03.254 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T07:47:03.254 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T07:47:03.255 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T07:47:03.255 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T07:47:03.255 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T07:47:03.255 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T07:47:03.255 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T07:47:03.255 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T07:47:03.255 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T07:47:03.255 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T07:47:03.255 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T07:47:03.255 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T07:47:03.255 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T07:47:03.255 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T07:47:03.256 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T07:47:03.257 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T07:47:03.258 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T07:47:03.258 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T07:47:03.258 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T07:47:03.258 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T07:47:03.258 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T07:47:03.258 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T07:47:03.258 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T07:47:03.258 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T07:47:03.258 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T07:47:03.258 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T07:47:03.258 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T07:47:03.258 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T07:47:03.258 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T07:47:03.259 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T07:47:03.259 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T07:47:03.259 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T07:47:03.259 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T07:47:03.259 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T07:47:03.259 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T07:47:03.259 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T07:47:03.259 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T07:47:03.259 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T07:47:03.259 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T07:47:03.259 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T07:47:03.259 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T07:47:03.259 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T07:47:03.260 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T07:47:03.261 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T07:47:03.261 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T07:47:03.261 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T07:47:03.261 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T07:47:03.261 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T07:47:03.261 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T07:47:03.261 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T07:47:03.261 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T07:47:03.261 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T07:47:03.261 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T07:47:03.261 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T07:47:03.261 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T07:47:03.262 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T07:47:03.262 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T07:47:03.262 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T07:47:03.262 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T07:47:03.262 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T07:47:03.262 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T07:47:03.262 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T07:47:03.262 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T07:47:03.262 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T07:47:03.262 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T07:47:03.262 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T07:47:03.262 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T07:47:03.262 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T07:47:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T07:47:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T07:47:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T07:47:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T07:47:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T07:47:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T07:47:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T07:47:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T07:47:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T07:47:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T07:47:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T07:47:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T07:47:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T07:47:03.264 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T07:47:03.264 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T07:47:03.264 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T07:47:03.264 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T07:47:03.264 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T07:47:03.264 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T07:47:03.264 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T07:47:03.264 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T07:47:03.264 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T07:47:03.264 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T07:47:03.264 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T07:47:03.264 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T07:47:03.265 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T07:47:03.265 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T07:47:03.265 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T07:47:03.265 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T07:47:03.265 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T07:47:03.265 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T07:47:03.265 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T07:47:03.265 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T07:47:03.265 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T07:47:03.265 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T07:47:03.265 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T07:47:03.265 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T07:47:03.265 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T07:47:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T07:47:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T07:47:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T07:47:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T07:47:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T07:47:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T07:47:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T07:47:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T07:47:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T07:47:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T07:47:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T07:47:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T07:47:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T07:47:03.267 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T07:47:03.268 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T07:47:03.268 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T07:47:03.268 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T07:47:03.268 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T07:47:03.268 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T07:47:03.268 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T07:47:03.268 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T07:47:03.268 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T07:47:03.268 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T07:47:03.268 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T07:47:03.268 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T07:47:03.268 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T07:47:03.269 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T07:47:03.269 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T07:47:03.269 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T07:47:03.269 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T07:47:03.269 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T07:47:03.269 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T07:47:03.269 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T07:47:03.269 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T07:47:03.269 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T07:47:03.269 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T07:47:03.269 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T07:47:03.269 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T07:47:03.269 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T07:47:03.270 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T07:47:03.270 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T07:47:03.270 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T07:47:03.270 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T07:47:03.270 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T07:47:03.270 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T07:47:03.270 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T07:47:03.270 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T07:47:03.270 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T07:47:03.270 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T07:47:03.270 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T07:47:03.270 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T07:47:03.270 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T07:47:03.271 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T07:47:03.271 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T07:47:03.271 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T07:47:03.271 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T07:47:03.271 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T07:47:03.271 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T07:47:03.271 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T07:47:03.271 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T07:47:03.271 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T07:47:03.271 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T07:47:03.271 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T07:47:03.271 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T07:47:03.271 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T07:47:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T07:47:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T07:47:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T07:47:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T07:47:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T07:47:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T07:47:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T07:47:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T07:47:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T07:47:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T07:47:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T07:47:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T07:47:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T07:47:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T07:47:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T07:47:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T07:47:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T07:47:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T07:47:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T07:47:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T07:47:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T07:47:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T07:47:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T07:47:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T07:47:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T07:47:03.274 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T07:47:03.275 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T07:47:03.275 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T07:47:03.275 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T07:47:03.275 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T07:47:03.275 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T07:47:03.275 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T07:47:03.275 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T07:47:03.275 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T07:47:03.275 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T07:47:03.275 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T07:47:03.275 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T07:47:03.275 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T07:47:03.276 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T07:47:03.276 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T07:47:03.276 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T07:47:03.276 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T07:47:03.276 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T07:47:03.276 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T07:47:03.276 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T07:47:03.276 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T07:47:03.276 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T07:47:03.276 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T07:47:03.276 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T07:47:03.276 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T07:47:03.276 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T07:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T07:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T07:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T07:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T07:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T07:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T07:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T07:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T07:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T07:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T07:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T07:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T07:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T07:47:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T07:47:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T07:47:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T07:47:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T07:47:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T07:47:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T07:47:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T07:47:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T07:47:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T07:47:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T07:47:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T07:47:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T07:47:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T07:47:03.279 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T07:47:03.279 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T07:47:03.279 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T07:47:03.279 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T07:47:03.279 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T07:47:03.279 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T07:47:03.279 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T07:47:03.279 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T07:47:03.279 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T07:47:03.279 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T07:47:03.279 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T07:47:03.279 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T07:47:03.279 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T07:47:03.280 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T07:47:03.280 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T07:47:03.280 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T07:47:03.280 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T07:47:03.280 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T07:47:03.280 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T07:47:03.280 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T07:47:03.280 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T07:47:03.280 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T07:47:03.280 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T07:47:03.280 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T07:47:03.280 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T07:47:03.281 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T07:47:03.281 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T07:47:03.281 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T07:47:03.281 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T07:47:03.281 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T07:47:03.281 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T07:47:03.281 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T07:47:03.281 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T07:47:03.281 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T07:47:03.281 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T07:47:03.281 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T07:47:03.281 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T07:47:03.281 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T07:47:03.282 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T07:47:03.282 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T07:47:03.282 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T07:47:03.282 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T07:47:03.282 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T07:47:03.282 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T07:47:03.282 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T07:47:03.282 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T07:47:03.282 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T07:47:03.282 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T07:47:03.282 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T07:47:03.282 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T07:47:03.282 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T07:47:03.283 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T07:47:03.283 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T07:47:03.283 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T07:47:03.283 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T07:47:03.283 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T07:47:03.283 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T07:47:03.283 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T07:47:03.283 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T07:47:03.283 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T07:47:03.283 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T07:47:03.283 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T07:47:03.283 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T07:47:03.283 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T07:47:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T07:47:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T07:47:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T07:47:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T07:47:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T07:47:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T07:47:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T07:47:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T07:47:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T07:47:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T07:47:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T07:47:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T07:47:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T07:47:03.285 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T07:47:03.285 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T07:47:03.285 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T07:47:03.285 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T07:47:03.285 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T07:47:03.285 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T07:47:03.285 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T07:47:03.285 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T07:47:03.285 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T07:47:03.285 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T07:47:03.285 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T07:47:03.285 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T07:47:03.286 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T07:47:03.286 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T07:47:03.286 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T07:47:03.286 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T07:47:03.286 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T07:47:03.286 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T07:47:03.286 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T07:47:03.286 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T07:47:03.286 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T07:47:03.286 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T07:47:03.286 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T07:47:03.286 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T07:47:03.286 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T07:47:03.287 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T07:47:03.287 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T07:47:03.287 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T07:47:03.287 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T07:47:03.287 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T07:47:03.287 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T07:47:03.287 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T07:47:03.287 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T07:47:03.287 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T07:47:03.287 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T07:47:03.287 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T07:47:03.287 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T07:47:03.287 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T07:47:03.288 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T07:47:03.288 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T07:47:03.288 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T07:47:03.288 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T07:47:03.288 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T07:47:03.288 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T07:47:03.288 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T07:47:03.288 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T07:47:03.288 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T07:47:03.288 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T07:47:03.288 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T07:47:03.288 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T07:47:03.288 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T07:47:03.289 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T07:47:03.289 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T07:47:03.289 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T07:47:03.289 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T07:47:03.289 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T07:47:03.289 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T07:47:03.289 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T07:47:03.289 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T07:47:03.289 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T07:47:03.289 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T07:47:03.289 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T07:47:03.289 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T07:47:03.289 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T07:47:03.290 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T07:47:03.290 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T07:47:03.290 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T07:47:03.290 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T07:47:03.290 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T07:47:03.290 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T07:47:03.290 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T07:47:03.290 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T07:47:03.290 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T07:47:03.290 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T07:47:03.290 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T07:47:03.290 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T07:47:03.290 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T07:47:03.291 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T07:47:03.291 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T07:47:03.291 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T07:47:03.291 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T07:47:03.291 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T07:47:03.291 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T07:47:03.291 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T07:47:03.291 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T07:47:03.291 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T07:47:03.291 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T07:47:03.291 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T07:47:03.291 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T07:47:03.291 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T07:47:03.292 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T07:47:03.292 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T07:47:03.292 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T07:47:03.292 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T07:47:03.292 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T07:47:03.292 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T07:47:03.292 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T07:47:03.292 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T07:47:03.292 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T07:47:03.292 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T07:47:03.292 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T07:47:03.292 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T07:47:03.292 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T07:47:03.293 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T07:47:03.293 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T07:47:03.293 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T07:47:03.293 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T07:47:03.293 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T07:47:03.293 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T07:47:03.293 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T07:47:03.293 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T07:47:03.293 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T07:47:03.293 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T07:47:03.293 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T07:47:03.293 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T07:47:03.293 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T07:47:03.294 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T07:47:03.294 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T07:47:03.294 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T07:47:03.294 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T07:47:03.294 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T07:47:03.294 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T07:47:03.294 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T07:47:03.294 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T07:47:03.294 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T07:47:03.294 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T07:47:03.294 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T07:47:03.294 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T07:47:03.294 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T07:47:03.295 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T07:47:03.295 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T07:47:03.295 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T07:47:03.295 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T07:47:03.295 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T07:47:03.295 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T07:47:03.295 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T07:47:03.295 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T07:47:03.295 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T07:47:03.295 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T07:47:03.295 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T07:47:03.295 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T07:47:03.295 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T07:47:03.296 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T07:47:03.296 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T07:47:03.296 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T07:47:03.296 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T07:47:03.296 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T07:47:03.296 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T07:47:03.296 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T07:47:03.296 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T07:47:03.296 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T07:47:03.296 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T07:47:03.296 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T07:47:03.296 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T07:47:03.296 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T07:47:03.297 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T07:47:03.297 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T07:47:03.297 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T07:47:03.297 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T07:47:03.297 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T07:47:03.297 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T07:47:03.297 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T07:47:03.297 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T07:47:03.297 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T07:47:03.297 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T07:47:03.297 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T07:47:03.297 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T07:47:03.298 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T07:47:03.299 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T07:47:03.299 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T07:47:03.299 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T07:47:03.299 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T07:47:03.299 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T07:47:03.299 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T07:47:03.299 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T07:47:03.299 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T07:47:03.299 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T07:47:03.299 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T07:47:03.299 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T07:47:03.299 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T07:47:03.299 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T07:47:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T07:47:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T07:47:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T07:47:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T07:47:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T07:47:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T07:47:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T07:47:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T07:47:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T07:47:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T07:47:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T07:47:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T07:47:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T07:47:03.301 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T07:47:03.301 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T07:47:03.301 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T07:47:03.301 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T07:47:03.301 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T07:47:03.301 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T07:47:03.301 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T07:47:03.301 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T07:47:03.301 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T07:47:03.301 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T07:47:03.301 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T07:47:03.301 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T07:47:03.301 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T07:47:03.302 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T07:47:03.302 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T07:47:03.302 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T07:47:03.302 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T07:47:03.302 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T07:47:03.302 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T07:47:03.302 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T07:47:03.302 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T07:47:03.302 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T07:47:03.302 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T07:47:03.302 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T07:47:03.302 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T07:47:03.302 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T07:47:03.303 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T07:47:03.303 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T07:47:03.303 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T07:47:03.303 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T07:47:03.303 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T07:47:03.303 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T07:47:03.303 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T07:47:03.303 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T07:47:03.303 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T07:47:03.303 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T07:47:03.303 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T07:47:03.303 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T07:47:03.304 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T07:47:03.304 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T07:47:03.304 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T07:47:03.304 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T07:47:03.304 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T07:47:03.304 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T07:47:03.304 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T07:47:03.304 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T07:47:03.304 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T07:47:03.304 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T07:47:03.304 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T07:47:03.304 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T07:47:03.304 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T07:47:03.305 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T07:47:03.305 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T07:47:03.305 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T07:47:03.305 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T07:47:03.305 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T07:47:03.305 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T07:47:03.305 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T07:47:03.305 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T07:47:03.305 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T07:47:03.305 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T07:47:03.305 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T07:47:03.305 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T07:47:03.305 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T07:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T07:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T07:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T07:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T07:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T07:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T07:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T07:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T07:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T07:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T07:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T07:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T07:47:03.306 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T07:47:03.307 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T07:47:03.307 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T07:47:03.307 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T07:47:03.307 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T07:47:03.307 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T07:47:03.307 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T07:47:03.307 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T07:47:03.307 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T07:47:03.307 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T07:47:03.307 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T07:47:03.307 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T07:47:03.307 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T07:47:03.307 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T07:47:03.308 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T07:47:03.308 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T07:47:03.308 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T07:47:03.308 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T07:47:03.308 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T07:47:03.308 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T07:47:03.308 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T07:47:03.308 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T07:47:03.308 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T07:47:03.308 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T07:47:03.308 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T07:47:03.308 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T07:47:03.308 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T07:47:03.309 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T07:47:03.309 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T07:47:03.309 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T07:47:03.309 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T07:47:03.309 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T07:47:03.309 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T07:47:03.309 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T07:47:03.309 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T07:47:03.309 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T07:47:03.309 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T07:47:03.309 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T07:47:03.309 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T07:47:03.309 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T07:47:03.310 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T07:47:03.311 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T07:47:03.311 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T07:47:03.311 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T07:47:03.311 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T07:47:03.311 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T07:47:03.311 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T07:47:03.311 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T07:47:03.311 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T07:47:03.311 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T07:47:03.311 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T07:47:03.311 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T07:47:03.311 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T07:47:03.311 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T07:47:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T07:47:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T07:47:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T07:47:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T07:47:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T07:47:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T07:47:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T07:47:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T07:47:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T07:47:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T07:47:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T07:47:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T07:47:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T07:47:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T07:47:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T07:47:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T07:47:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T07:47:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T07:47:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T07:47:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T07:47:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T07:47:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T07:47:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T07:47:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T07:47:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T07:47:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T07:47:03.314 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T07:47:03.315 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T07:47:03.315 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T07:47:03.315 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T07:47:03.315 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T07:47:03.315 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T07:47:03.315 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T07:47:03.315 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T07:47:03.315 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T07:47:03.315 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T07:47:03.315 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T07:47:03.315 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T07:47:03.315 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T07:47:03.315 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T07:47:03.316 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T07:47:03.316 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T07:47:03.316 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T07:47:03.316 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T07:47:03.316 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T07:47:03.316 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T07:47:03.316 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T07:47:03.316 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T07:47:03.316 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T07:47:03.316 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T07:47:03.316 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T07:47:03.316 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T07:47:03.316 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T07:47:03.317 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T07:47:03.317 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T07:47:03.317 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T07:47:03.317 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T07:47:03.317 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T07:47:03.317 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T07:47:03.317 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T07:47:03.317 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T07:47:03.317 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T07:47:03.317 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T07:47:03.317 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T07:47:03.317 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T07:47:03.317 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T07:47:03.318 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T07:47:03.318 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T07:47:03.318 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T07:47:03.318 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T07:47:03.318 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T07:47:03.318 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T07:47:03.318 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T07:47:03.318 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T07:47:03.318 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T07:47:03.318 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T07:47:03.318 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T07:47:03.318 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T07:47:03.318 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T07:47:03.319 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T07:47:03.319 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T07:47:03.319 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T07:47:03.319 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T07:47:03.319 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T07:47:03.319 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T07:47:03.319 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T07:47:03.319 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T07:47:03.319 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T07:47:03.319 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T07:47:03.319 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T07:47:03.319 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T07:47:03.319 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T07:47:03.320 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T07:47:03.320 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T07:47:03.320 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T07:47:03.320 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T07:47:03.320 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T07:47:03.320 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T07:47:03.320 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T07:47:03.320 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T07:47:03.320 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T07:47:03.320 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T07:47:03.320 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T07:47:03.320 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T07:47:03.320 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T07:47:03.321 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T07:47:03.321 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T07:47:03.321 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T07:47:03.321 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T07:47:03.321 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T07:47:03.321 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T07:47:03.321 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T07:47:03.321 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T07:47:03.321 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T07:47:03.321 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T07:47:03.321 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T07:47:03.321 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T07:47:03.321 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T07:47:03.322 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T07:47:03.323 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T07:47:03.323 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T07:47:03.323 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T07:47:03.323 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T07:47:03.323 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T07:47:03.323 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T07:47:03.323 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T07:47:03.323 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T07:47:03.323 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T07:47:03.323 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T07:47:03.323 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T07:47:03.323 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T07:47:03.323 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T07:47:03.324 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T07:47:03.324 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T07:47:03.324 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T07:47:03.324 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T07:47:03.324 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T07:47:03.324 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T07:47:03.324 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T07:47:03.324 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T07:47:03.324 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T07:47:03.324 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T07:47:03.324 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T07:47:03.324 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T07:47:03.324 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T07:47:03.325 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T07:47:03.326 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T07:47:03.327 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T07:47:03.327 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T07:47:03.327 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T07:47:03.327 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T07:47:03.327 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T07:47:03.327 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T07:47:03.327 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T07:47:03.327 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T07:47:03.327 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T07:47:03.327 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T07:47:03.327 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T07:47:03.327 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T07:47:03.327 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T07:47:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T07:47:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T07:47:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T07:47:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T07:47:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T07:47:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T07:47:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T07:47:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T07:47:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T07:47:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T07:47:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T07:47:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T07:47:03.328 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T07:47:03.329 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T07:47:03.329 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T07:47:03.329 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T07:47:03.329 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T07:47:03.329 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T07:47:03.329 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T07:47:03.329 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T07:47:03.329 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T07:47:03.329 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T07:47:03.329 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T07:47:03.329 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T07:47:03.329 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T07:47:03.330 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T07:47:03.331 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T07:47:03.331 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T07:47:03.331 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T07:47:03.331 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T07:47:03.331 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T07:47:03.331 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T07:47:03.331 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T07:47:03.331 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T07:47:03.331 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T07:47:03.331 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T07:47:03.331 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T07:47:03.331 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T07:47:03.331 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T07:47:03.332 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T07:47:03.332 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T07:47:03.332 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T07:47:03.332 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T07:47:03.332 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T07:47:03.332 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T07:47:03.332 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T07:47:03.332 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T07:47:03.332 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T07:47:03.332 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T07:47:03.332 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T07:47:03.332 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T07:47:03.332 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T07:47:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T07:47:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T07:47:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T07:47:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T07:47:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T07:47:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T07:47:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T07:47:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T07:47:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T07:47:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T07:47:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T07:47:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T07:47:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T07:47:03.334 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T07:47:03.335 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T07:47:03.336 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T07:47:03.336 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T07:47:03.336 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T07:47:03.336 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T07:47:03.336 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T07:47:03.336 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T07:47:03.336 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T07:47:03.336 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T07:47:03.336 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T07:47:03.336 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T07:47:03.336 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T07:47:03.336 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T07:47:03.336 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T07:47:03.337 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T07:47:03.338 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T07:47:03.338 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T07:47:03.338 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T07:47:03.338 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T07:47:03.338 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T07:47:03.338 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T07:47:03.338 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T07:47:03.338 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T07:47:03.338 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T07:47:03.338 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T07:47:03.338 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T07:47:03.338 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T07:47:03.339 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T07:47:03.340 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T07:47:03.340 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T07:47:03.340 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T07:47:03.340 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T07:47:03.340 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T07:47:03.340 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T07:47:03.340 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T07:47:03.340 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T07:47:03.340 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T07:47:03.340 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T07:47:03.340 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T07:47:03.340 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T07:47:03.340 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T07:47:03.341 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T07:47:03.341 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T07:47:03.341 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T07:47:03.341 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T07:47:03.341 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T07:47:03.341 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T07:47:03.341 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T07:47:03.341 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T07:47:03.341 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T07:47:03.341 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T07:47:03.341 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T07:47:03.341 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T07:47:03.341 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T07:47:03.342 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T07:47:03.342 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T07:47:03.342 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T07:47:03.342 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T07:47:03.342 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T07:47:03.342 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T07:47:03.342 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T07:47:03.342 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T07:47:03.342 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T07:47:03.342 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T07:47:03.342 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T07:47:03.342 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T07:47:03.342 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T07:47:03.343 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T07:47:03.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T07:47:03.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T07:47:03.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T07:47:03.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T07:47:03.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T07:47:03.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T07:47:03.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T07:47:03.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T07:47:03.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T07:47:03.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T07:47:03.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T07:47:03.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T07:47:03.344 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T07:47:03.345 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T07:47:03.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T07:47:03.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T07:47:03.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T07:47:03.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T07:47:03.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T07:47:03.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T07:47:03.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T07:47:03.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T07:47:03.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T07:47:03.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T07:47:03.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T07:47:03.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T07:47:03.346 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T07:47:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T07:47:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T07:47:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T07:47:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T07:47:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T07:47:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T07:47:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T07:47:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T07:47:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T07:47:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T07:47:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T07:47:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T07:47:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T07:47:03.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T07:47:03.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T07:47:03.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T07:47:03.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T07:47:03.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T07:47:03.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T07:47:03.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T07:47:03.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T07:47:03.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T07:47:03.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T07:47:03.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T07:47:03.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T07:47:03.348 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T07:47:03.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T07:47:03.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T07:47:03.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T07:47:03.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T07:47:03.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T07:47:03.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T07:47:03.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T07:47:03.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T07:47:03.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T07:47:03.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T07:47:03.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T07:47:03.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T07:47:03.349 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T07:47:03.350 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T07:47:03.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T07:47:03.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T07:47:03.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T07:47:03.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T07:47:03.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T07:47:03.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T07:47:03.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T07:47:03.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T07:47:03.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T07:47:03.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T07:47:03.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T07:47:03.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T07:47:03.351 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T07:47:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T07:47:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T07:47:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T07:47:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T07:47:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T07:47:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T07:47:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T07:47:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T07:47:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T07:47:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T07:47:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T07:47:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T07:47:03.352 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T07:47:03.353 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T07:47:03.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T07:47:03.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T07:47:03.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T07:47:03.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T07:47:03.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T07:47:03.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T07:47:03.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T07:47:03.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T07:47:03.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T07:47:03.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T07:47:03.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T07:47:03.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T07:47:03.354 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T07:47:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T07:47:03.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T07:47:03.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T07:47:03.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T07:47:03.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T07:47:03.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T07:47:03.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T07:47:03.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T07:47:03.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T07:47:03.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T07:47:03.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T07:47:03.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T07:47:03.356 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T07:47:03.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T07:47:03.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T07:47:03.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T07:47:03.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T07:47:03.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T07:47:03.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T07:47:03.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T07:47:03.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T07:47:03.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T07:47:03.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T07:47:03.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T07:47:03.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T07:47:03.357 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T07:47:03.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T07:47:03.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T07:47:03.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T07:47:03.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T07:47:03.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T07:47:03.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T07:47:03.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T07:47:03.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T07:47:03.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T07:47:03.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T07:47:03.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T07:47:03.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T07:47:03.358 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T07:47:03.359 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T07:47:03.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T07:47:03.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T07:47:03.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T07:47:03.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T07:47:03.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T07:47:03.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T07:47:03.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T07:47:03.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T07:47:03.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T07:47:03.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T07:47:03.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T07:47:03.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T07:47:03.360 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T07:47:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T07:47:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T07:47:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T07:47:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T07:47:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T07:47:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T07:47:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T07:47:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T07:47:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T07:47:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T07:47:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T07:47:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T07:47:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T07:47:03.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T07:47:03.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T07:47:03.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T07:47:03.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T07:47:03.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T07:47:03.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T07:47:03.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T07:47:03.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T07:47:03.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T07:47:03.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T07:47:03.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T07:47:03.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T07:47:03.362 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T07:47:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T07:47:03.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T07:47:03.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T07:47:03.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T07:47:03.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T07:47:03.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T07:47:03.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T07:47:03.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T07:47:03.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T07:47:03.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T07:47:03.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T07:47:03.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T07:47:03.364 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T07:47:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T07:47:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T07:47:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T07:47:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T07:47:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T07:47:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T07:47:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T07:47:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T07:47:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T07:47:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T07:47:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T07:47:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T07:47:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T07:47:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T07:47:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T07:47:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T07:47:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T07:47:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T07:47:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T07:47:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T07:47:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T07:47:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T07:47:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T07:47:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T07:47:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T07:47:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T07:47:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T07:47:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T07:47:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T07:47:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T07:47:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T07:47:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T07:47:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T07:47:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T07:47:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T07:47:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T07:47:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T07:47:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T07:47:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T07:47:03.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T07:47:03.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T07:47:03.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T07:47:03.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T07:47:03.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T07:47:03.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T07:47:03.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T07:47:03.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T07:47:03.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T07:47:03.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T07:47:03.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T07:47:03.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T07:47:03.368 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T07:47:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T07:47:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T07:47:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T07:47:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T07:47:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T07:47:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T07:47:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T07:47:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T07:47:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T07:47:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T07:47:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T07:47:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T07:47:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T07:47:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T07:47:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T07:47:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T07:47:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T07:47:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T07:47:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T07:47:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T07:47:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T07:47:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T07:47:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T07:47:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T07:47:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T07:47:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T07:47:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T07:47:03.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T07:47:03.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T07:47:03.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T07:47:03.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T07:47:03.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T07:47:03.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T07:47:03.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T07:47:03.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T07:47:03.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T07:47:03.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T07:47:03.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T07:47:03.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T07:47:03.372 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T07:47:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T07:47:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T07:47:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T07:47:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T07:47:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T07:47:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T07:47:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T07:47:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T07:47:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T07:47:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T07:47:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T07:47:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T07:47:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T07:47:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T07:47:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T07:47:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T07:47:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T07:47:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T07:47:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T07:47:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T07:47:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T07:47:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T07:47:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T07:47:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T07:47:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T07:47:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T07:47:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T07:47:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T07:47:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T07:47:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T07:47:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T07:47:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T07:47:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T07:47:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T07:47:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T07:47:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T07:47:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T07:47:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T07:47:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T07:47:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T07:47:03.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T07:47:03.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T07:47:03.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T07:47:03.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T07:47:03.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T07:47:03.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T07:47:03.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T07:47:03.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T07:47:03.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T07:47:03.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T07:47:03.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T07:47:03.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T07:47:03.378 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T07:47:03.379 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T07:47:03.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T07:47:03.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T07:47:03.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T07:47:03.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T07:47:03.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T07:47:03.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T07:47:03.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T07:47:03.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T07:47:03.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T07:47:03.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T07:47:03.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T07:47:03.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T07:47:03.380 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T07:47:03.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T07:47:03.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T07:47:03.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T07:47:03.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T07:47:03.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T07:47:03.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T07:47:03.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T07:47:03.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T07:47:03.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T07:47:03.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T07:47:03.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T07:47:03.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T07:47:03.381 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T07:47:03.382 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T07:47:03.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T07:47:03.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T07:47:03.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T07:47:03.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T07:47:03.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T07:47:03.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T07:47:03.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T07:47:03.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T07:47:03.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T07:47:03.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T07:47:03.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T07:47:03.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T07:47:03.383 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T07:47:03.384 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T07:47:03.384 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T07:47:03.384 INFO:teuthology.orchestra.run.smithi002.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T07:47:03.388 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:47:03.388 DEBUG:teuthology.orchestra.run.smithi002:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T07:47:03.413 INFO:teuthology.orchestra.run.smithi002.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T07:47:03.413 INFO:teuthology.orchestra.run.smithi002.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T07:47:03.413 INFO:teuthology.orchestra.run.smithi002.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T07:47:03.413 INFO:teuthology.orchestra.run.smithi002.stdout:fi 2024-09-10T07:47:03.413 INFO:teuthology.orchestra.run.smithi002.stdout:for file in vmlinuz System.map config; do 2024-09-10T07:47:03.413 INFO:teuthology.orchestra.run.smithi002.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T07:47:03.413 INFO:teuthology.orchestra.run.smithi002.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T07:47:03.413 INFO:teuthology.orchestra.run.smithi002.stdout: fi 2024-09-10T07:47:03.414 INFO:teuthology.orchestra.run.smithi002.stdout:done 2024-09-10T07:47:03.414 INFO:teuthology.orchestra.run.smithi002.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T07:47:03.414 INFO:teuthology.orchestra.run.smithi002.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T07:47:03.414 INFO:teuthology.orchestra.run.smithi002.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T07:47:03.414 INFO:teuthology.orchestra.run.smithi002.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T07:47:03.414 INFO:teuthology.orchestra.run.smithi002.stdout:fi 2024-09-10T07:47:03.414 INFO:teuthology.orchestra.run.smithi002.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T07:47:03.414 INFO:teuthology.orchestra.run.smithi002.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:47:03.414 INFO:teuthology.orchestra.run.smithi002.stdout:fi 2024-09-10T07:47:03.414 INFO:teuthology.orchestra.run.smithi002.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T07:47:03.414 INFO:teuthology.orchestra.run.smithi002.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T07:47:03.414 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:47:03.414 INFO:teuthology.orchestra.run.smithi002.stdout:fi 2024-09-10T07:47:03.415 DEBUG:teuthology.orchestra.run.smithi002:> sudo rpm -qi grub2-tools 2024-09-10T07:47:03.500 INFO:teuthology.orchestra.run.smithi002.stdout:Name : grub2-tools 2024-09-10T07:47:03.500 INFO:teuthology.orchestra.run.smithi002.stdout:Epoch : 1 2024-09-10T07:47:03.500 INFO:teuthology.orchestra.run.smithi002.stdout:Version : 2.06 2024-09-10T07:47:03.500 INFO:teuthology.orchestra.run.smithi002.stdout:Release : 82.el9 2024-09-10T07:47:03.500 INFO:teuthology.orchestra.run.smithi002.stdout:Architecture: x86_64 2024-09-10T07:47:03.501 INFO:teuthology.orchestra.run.smithi002.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T07:47:03.501 INFO:teuthology.orchestra.run.smithi002.stdout:Group : Unspecified 2024-09-10T07:47:03.501 INFO:teuthology.orchestra.run.smithi002.stdout:Size : 8274670 2024-09-10T07:47:03.501 INFO:teuthology.orchestra.run.smithi002.stdout:License : GPLv3+ 2024-09-10T07:47:03.501 INFO:teuthology.orchestra.run.smithi002.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T07:47:03.501 INFO:teuthology.orchestra.run.smithi002.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T07:47:03.501 INFO:teuthology.orchestra.run.smithi002.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T07:47:03.501 INFO:teuthology.orchestra.run.smithi002.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T07:47:03.501 INFO:teuthology.orchestra.run.smithi002.stdout:Packager : builder@centos.org 2024-09-10T07:47:03.501 INFO:teuthology.orchestra.run.smithi002.stdout:Vendor : CentOS 2024-09-10T07:47:03.501 INFO:teuthology.orchestra.run.smithi002.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T07:47:03.501 INFO:teuthology.orchestra.run.smithi002.stdout:Summary : Support tools for GRUB. 2024-09-10T07:47:03.501 INFO:teuthology.orchestra.run.smithi002.stdout:Description : 2024-09-10T07:47:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:47:03.502 INFO:teuthology.orchestra.run.smithi002.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T07:47:03.502 INFO:teuthology.orchestra.run.smithi002.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T07:47:03.502 INFO:teuthology.orchestra.run.smithi002.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T07:47:03.502 INFO:teuthology.orchestra.run.smithi002.stdout:hardware devices. 2024-09-10T07:47:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:47:03.502 INFO:teuthology.orchestra.run.smithi002.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T07:47:03.503 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T07:47:03.503 INFO:teuthology.task.kernel:Updating grub on smithi002 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:47:03.503 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T07:47:04.135 INFO:teuthology.orchestra.run.smithi002.stderr:Generating grub configuration file ... 2024-09-10T07:47:05.527 INFO:teuthology.orchestra.run.smithi002.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T07:47:05.552 INFO:teuthology.orchestra.run.smithi002.stderr:done 2024-09-10T07:47:05.554 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2024-09-10T07:47:05.569 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.EOgxcaSpg2 2024-09-10T07:47:05.570 DEBUG:teuthology.orchestra.run.smithi002:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EOgxcaSpg2 2024-09-10T07:47:05.635 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.EOgxcaSpg2 2024-09-10T07:47:05.774 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.EOgxcaSpg2 is 6KB 2024-09-10T07:47:05.822 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.EOgxcaSpg2 2024-09-10T07:47:05.837 DEBUG:teuthology.orchestra.run.smithi002:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T07:47:05.899 INFO:teuthology.orchestra.run.smithi002.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T07:47:05.900 INFO:teuthology.orchestra.run.smithi002.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T07:47:05.900 INFO:teuthology.orchestra.run.smithi002.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T07:47:05.900 INFO:teuthology.orchestra.run.smithi002.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T07:47:05.901 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:47:06.077 DEBUG:teuthology.orchestra.run.smithi002:> sudo shutdown -r now 2024-09-10T07:47:36.081 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T07:47:36.082 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-10T07:47:36.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:47:54.557 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-09-10T07:48:03.558 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-10T07:48:03.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:49:03.562 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T07:49:15.563 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-10T07:49:15.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:49:15.942 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-09-10T07:49:16.426 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-10T07:49:16.426 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T07:49:16.426 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-09-10T07:49:16.461 INFO:teuthology.orchestra.run.smithi002.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:49:16.461 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T07:49:16.461 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T07:49:16.462 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T07:49:17.463 DEBUG:teuthology.parallel:result is None 2024-09-10T07:49:17.463 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T07:49:17.472 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T07:49:17.472 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T07:49:17.477 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T07:49:17.479 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T07:49:17.511 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T07:49:17.517 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T07:49:17.524 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T07:49:17.525 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T07:49:17.555 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T07:49:17.558 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T07:49:17.675 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T07:49:17.682 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T07:49:17.682 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T07:49:17.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:49:17.700 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T07:49:17.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:49:17.717 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T07:49:17.721 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T07:49:17.743 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T07:49:17.799 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:49:17.811 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:49:17.818 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:49:17.825 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:49:17.832 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:49:17.857 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T07:49:17.860 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T07:49:17.868 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T07:49:17.868 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T07:49:17.872 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T07:49:17.874 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T07:49:17.924 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T07:49:17.933 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T07:49:17.933 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T07:49:17.938 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T07:49:17.942 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T07:49:17.968 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:49:18.070 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:49:18.152 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:49:18.211 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:49:18.286 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:49:18.286 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T07:49:18.340 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:49:18.379 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:49:18.438 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:49:18.523 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:49:18.566 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T07:49:18.566 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T07:49:18.633 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:49:18.674 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T07:49:18.733 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:49:18.816 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T07:49:18.864 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-10T07:49:18.864 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T07:49:18.932 DEBUG:teuthology.orchestra.run.smithi002:> sudo service rsyslog restart 2024-09-10T07:49:18.936 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-09-10T07:49:18.939 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-09-10T07:49:18.978 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T07:49:19.020 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T07:49:19.021 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T07:49:19.457 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T07:49:19.466 INFO:teuthology.task.internal:Starting timer... 2024-09-10T07:49:19.466 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T07:49:19.482 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T07:49:19.490 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-10T07:49:19.491 DEBUG:teuthology.orchestra.run.smithi002:> sudo service auditd rotate 2024-09-10T07:49:19.579 INFO:teuthology.orchestra.run.smithi002.stdout:Rotating logs: 2024-09-10T07:49:19.583 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-09-10T07:49:19.645 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-09-10T07:49:19.648 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2024-09-10T07:49:19.717 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2024-09-10T07:49:19.719 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T07:49:19.719 DEBUG:teuthology.orchestra.run.smithi002:> /usr/sbin/getenforce 2024-09-10T07:49:19.757 INFO:teuthology.orchestra.run.smithi002.stdout:Permissive 2024-09-10T07:49:19.758 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-09-10T07:49:19.779 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-09-10T07:49:19.779 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2024-09-10T07:49:19.806 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2024-09-10T07:49:19.806 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi002.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive'} 2024-09-10T07:49:19.806 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T07:49:19.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:49:19.865 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T07:49:19.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:49:19.897 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T07:49:19.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T07:49:19.926 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T07:49:19.926 DEBUG:teuthology.orchestra.run.smithi002:> sudo /usr/sbin/setenforce permissive 2024-09-10T07:49:19.990 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-09-10T07:49:20.024 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2024-09-10T07:49:20.061 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T07:49:20.070 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T07:49:20.073 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T07:49:20.203 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T07:49:20.215 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T07:49:20.216 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi002.front.sepia.ceph.com,smithi107.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T07:54:00.092 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi002.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2024-09-10T07:54:00.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-10T07:54:00.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:54:00.189 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-09-10T07:54:00.315 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-10T07:54:00.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-10T07:54:00.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:54:00.391 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-10T07:54:00.471 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-10T07:54:00.472 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-10T07:54:00.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T07:54:00.551 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-09-10T07:54:00.635 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-10T07:54:00.635 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T07:54:00.646 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T07:54:00.647 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T07:54:00.647 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T07:54:00.651 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T07:54:00.651 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T07:54:00.654 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T07:54:00.654 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T07:54:00.680 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T07:54:00.694 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T07:54:00.708 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T07:54:00.709 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T07:54:00.718 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-09-10T07:54:00.724 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T07:54:00.729 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-09-10T07:54:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T07:54:00.743 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T07:54:00.750 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2024-09-10T07:54:00.757 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T07:54:00.762 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2024-09-10T07:54:00.780 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T07:54:00.790 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: ntpd: command not found 2024-09-10T07:54:00.792 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T07:54:00.814 INFO:teuthology.orchestra.run.smithi002.stdout:506 Cannot talk to daemon 2024-09-10T07:54:00.818 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-10T07:54:00.821 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T07:54:00.821 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-10T07:54:00.821 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:00.821 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:00.821 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:00.821 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:00.844 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T07:54:00.856 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-09-10T07:54:00.860 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T07:54:00.860 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-09-10T07:54:00.860 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:00.860 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:00.860 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:00.860 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:00.873 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T07:54:00.966 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-09-10T07:54:00.973 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T07:54:00.973 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-09-10T07:54:00.973 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:00.973 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:00.973 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:00.973 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T07:54:00.974 INFO:teuthology.run_tasks:Running task install... 2024-09-10T07:54:00.983 DEBUG:teuthology.task.install:project ceph 2024-09-10T07:54:00.983 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T07:54:00.983 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T07:54:00.983 INFO:teuthology.task.install:Using flavor: default 2024-09-10T07:54:00.991 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T07:54:00.991 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T07:54:00.991 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T07:54:00.992 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:54:00.994 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T07:54:00.995 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:54:00.995 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T07:54:00.996 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:54:01.161 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T07:54:01.161 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T07:54:01.169 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T07:54:01.169 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T07:54:01.202 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T07:54:01.202 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T07:54:01.274 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T07:54:01.275 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:54:01.275 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T07:54:01.283 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T07:54:01.284 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-10T07:54:01.284 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T07:54:01.310 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T07:54:01.311 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T07:54:01.313 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T07:54:01.313 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T07:54:01.313 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T07:54:01.332 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T07:54:01.332 DEBUG:teuthology.orchestra.run.smithi002:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T07:54:01.340 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T07:54:01.340 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T07:54:01.377 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T07:54:01.407 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T07:54:01.429 DEBUG:teuthology.orchestra.run.smithi002:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T07:54:01.454 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T07:54:01.483 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T07:54:01.508 DEBUG:teuthology.orchestra.run.smithi002:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T07:54:01.519 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2024-09-10T07:54:01.520 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2024-09-10T07:54:01.547 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-09-10T07:54:01.549 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-09-10T07:54:01.564 INFO:teuthology.orchestra.run.smithi002.stdout:check_obsoletes = 1 2024-09-10T07:54:01.566 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum clean all 2024-09-10T07:54:01.746 INFO:teuthology.orchestra.run.smithi155.stdout:45 files removed 2024-09-10T07:54:01.772 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T07:54:01.773 INFO:teuthology.orchestra.run.smithi107.stdout:45 files removed 2024-09-10T07:54:01.799 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T07:54:01.855 INFO:teuthology.orchestra.run.smithi002.stdout:45 files removed 2024-09-10T07:54:01.889 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T07:54:02.317 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 316 kB/s | 84 kB 00:00 2024-09-10T07:54:02.333 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 320 kB/s | 84 kB 00:00 2024-09-10T07:54:02.573 INFO:teuthology.orchestra.run.smithi002.stdout:ceph packages for x86_64 304 kB/s | 84 kB 00:00 2024-09-10T07:54:02.601 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T07:54:02.617 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T07:54:02.857 INFO:teuthology.orchestra.run.smithi002.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T07:54:02.876 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T07:54:02.893 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T07:54:03.124 INFO:teuthology.orchestra.run.smithi002.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-10T07:54:03.992 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.2 MB 00:01 2024-09-10T07:54:04.299 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 7.2 MB/s | 8.2 MB 00:01 2024-09-10T07:54:05.835 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.2 MB 00:02 2024-09-10T07:54:07.136 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 20 MB 00:02 2024-09-10T07:54:07.561 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-10T07:54:12.579 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.5 MB 00:01 2024-09-10T07:54:12.994 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.5 MB 00:02 2024-09-10T07:54:14.629 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 19 kB 00:00 2024-09-10T07:54:14.694 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 19 kB 00:00 2024-09-10T07:54:15.531 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-10T07:54:15.617 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-10T07:54:20.240 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 20 MB 00:14 2024-09-10T07:54:20.904 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T07:54:21.139 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T07:54:23.172 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:54:23.172 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:54:23.183 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T07:54:23.184 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T07:54:23.185 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T07:54:23.186 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T07:54:23.188 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T07:54:23.190 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T07:54:23.191 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T07:54:23.252 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T07:54:23.265 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-09-10T07:54:23.265 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-10T07:54:23.265 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-09-10T07:54:23.265 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T07:54:23.265 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T07:54:23.265 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T07:54:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T07:54:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T07:54:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T07:54:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T07:54:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T07:54:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T07:54:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T07:54:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T07:54:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T07:54:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T07:54:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T07:54:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T07:54:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T07:54:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T07:54:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T07:54:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T07:54:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T07:54:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T07:54:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T07:54:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T07:54:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T07:54:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T07:54:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T07:54:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T07:54:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T07:54:23.268 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T07:54:23.268 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T07:54:23.268 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T07:54:23.268 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T07:54:23.268 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T07:54:23.268 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T07:54:23.268 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T07:54:23.268 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T07:54:23.268 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T07:54:23.268 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T07:54:23.268 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-09-10T07:54:23.268 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T07:54:23.268 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T07:54:23.269 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T07:54:23.269 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T07:54:23.269 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T07:54:23.269 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T07:54:23.269 INFO:teuthology.orchestra.run.smithi107.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T07:54:23.269 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T07:54:23.269 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-10T07:54:23.269 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T07:54:23.269 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T07:54:23.269 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T07:54:23.269 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T07:54:23.269 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T07:54:23.269 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T07:54:23.270 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T07:54:23.270 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T07:54:23.270 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T07:54:23.270 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T07:54:23.270 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T07:54:23.270 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T07:54:23.270 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T07:54:23.270 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T07:54:23.270 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T07:54:23.270 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T07:54:23.270 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T07:54:23.270 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T07:54:23.270 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T07:54:23.271 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T07:54:23.271 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T07:54:23.271 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T07:54:23.271 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T07:54:23.271 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T07:54:23.271 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T07:54:23.271 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T07:54:23.271 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T07:54:23.271 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T07:54:23.271 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T07:54:23.271 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T07:54:23.271 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T07:54:23.272 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T07:54:23.272 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T07:54:23.272 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T07:54:23.272 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T07:54:23.272 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T07:54:23.272 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T07:54:23.272 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T07:54:23.272 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T07:54:23.272 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T07:54:23.272 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T07:54:23.272 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T07:54:23.272 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T07:54:23.272 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T07:54:23.273 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T07:54:23.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T07:54:23.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T07:54:23.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T07:54:23.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T07:54:23.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T07:54:23.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T07:54:23.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T07:54:23.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T07:54:23.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T07:54:23.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T07:54:23.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T07:54:23.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T07:54:23.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T07:54:23.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T07:54:23.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T07:54:23.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T07:54:23.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T07:54:23.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T07:54:23.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T07:54:23.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T07:54:23.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T07:54:23.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T07:54:23.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T07:54:23.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T07:54:23.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T07:54:23.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T07:54:23.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T07:54:23.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T07:54:23.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T07:54:23.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T07:54:23.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T07:54:23.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T07:54:23.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T07:54:23.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T07:54:23.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T07:54:23.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T07:54:23.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T07:54:23.277 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T07:54:23.277 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T07:54:23.277 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T07:54:23.277 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T07:54:23.277 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T07:54:23.277 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T07:54:23.277 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T07:54:23.277 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T07:54:23.277 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T07:54:23.277 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T07:54:23.277 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T07:54:23.277 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T07:54:23.277 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T07:54:23.278 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T07:54:23.278 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T07:54:23.278 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-09-10T07:54:23.278 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T07:54:23.278 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:54:23.278 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T07:54:23.278 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-09-10T07:54:23.278 INFO:teuthology.orchestra.run.smithi107.stdout:Install 148 Packages 2024-09-10T07:54:23.278 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 8 Packages 2024-09-10T07:54:23.278 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:54:23.279 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 226 M 2024-09-10T07:54:23.279 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T07:54:23.433 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:54:23.433 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:54:23.444 INFO:teuthology.orchestra.run.smithi155.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T07:54:23.445 INFO:teuthology.orchestra.run.smithi155.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T07:54:23.446 INFO:teuthology.orchestra.run.smithi155.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T07:54:23.447 INFO:teuthology.orchestra.run.smithi155.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T07:54:23.449 INFO:teuthology.orchestra.run.smithi155.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T07:54:23.451 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T07:54:23.452 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T07:54:23.515 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-09-10T07:54:23.532 INFO:teuthology.orchestra.run.smithi155.stdout:====================================================================================== 2024-09-10T07:54:23.532 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-09-10T07:54:23.532 INFO:teuthology.orchestra.run.smithi155.stdout:====================================================================================== 2024-09-10T07:54:23.532 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-09-10T07:54:23.532 INFO:teuthology.orchestra.run.smithi155.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T07:54:23.532 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T07:54:23.532 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T07:54:23.532 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T07:54:23.533 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T07:54:23.534 INFO:teuthology.orchestra.run.smithi155.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T07:54:23.534 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T07:54:23.534 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T07:54:23.534 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T07:54:23.534 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T07:54:23.534 INFO:teuthology.orchestra.run.smithi155.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T07:54:23.534 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T07:54:23.534 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T07:54:23.534 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T07:54:23.534 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T07:54:23.534 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T07:54:23.534 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T07:54:23.535 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T07:54:23.536 INFO:teuthology.orchestra.run.smithi155.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T07:54:23.536 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T07:54:23.536 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-09-10T07:54:23.536 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T07:54:23.536 INFO:teuthology.orchestra.run.smithi155.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T07:54:23.536 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T07:54:23.536 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T07:54:23.536 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T07:54:23.536 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T07:54:23.536 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T07:54:23.536 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T07:54:23.536 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T07:54:23.537 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T07:54:23.537 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T07:54:23.537 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T07:54:23.537 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T07:54:23.537 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T07:54:23.537 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T07:54:23.537 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T07:54:23.537 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T07:54:23.537 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T07:54:23.537 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T07:54:23.537 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T07:54:23.538 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T07:54:23.538 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T07:54:23.538 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T07:54:23.538 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T07:54:23.538 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T07:54:23.538 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T07:54:23.538 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T07:54:23.538 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T07:54:23.538 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T07:54:23.538 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T07:54:23.538 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T07:54:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T07:54:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T07:54:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T07:54:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T07:54:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T07:54:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T07:54:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T07:54:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T07:54:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T07:54:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T07:54:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T07:54:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T07:54:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T07:54:23.540 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T07:54:23.540 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T07:54:23.540 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T07:54:23.540 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T07:54:23.540 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T07:54:23.540 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T07:54:23.540 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T07:54:23.540 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T07:54:23.540 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T07:54:23.540 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T07:54:23.540 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T07:54:23.541 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T07:54:23.541 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T07:54:23.541 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T07:54:23.541 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T07:54:23.541 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T07:54:23.541 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T07:54:23.541 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T07:54:23.541 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T07:54:23.541 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T07:54:23.541 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T07:54:23.541 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T07:54:23.541 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T07:54:23.542 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T07:54:23.542 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T07:54:23.542 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T07:54:23.542 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T07:54:23.542 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T07:54:23.542 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T07:54:23.542 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T07:54:23.542 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T07:54:23.542 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T07:54:23.542 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T07:54:23.542 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T07:54:23.542 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T07:54:23.542 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T07:54:23.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T07:54:23.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T07:54:23.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T07:54:23.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T07:54:23.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T07:54:23.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T07:54:23.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T07:54:23.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T07:54:23.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T07:54:23.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T07:54:23.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T07:54:23.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T07:54:23.543 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T07:54:23.544 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T07:54:23.545 INFO:teuthology.orchestra.run.smithi155.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T07:54:23.545 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T07:54:23.545 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T07:54:23.545 INFO:teuthology.orchestra.run.smithi155.stdout:Installing weak dependencies: 2024-09-10T07:54:23.545 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T07:54:23.545 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:54:23.545 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-09-10T07:54:23.545 INFO:teuthology.orchestra.run.smithi155.stdout:====================================================================================== 2024-09-10T07:54:23.545 INFO:teuthology.orchestra.run.smithi155.stdout:Install 148 Packages 2024-09-10T07:54:23.545 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 8 Packages 2024-09-10T07:54:23.545 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:54:23.546 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 226 M 2024-09-10T07:54:23.546 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-09-10T07:54:23.647 INFO:teuthology.orchestra.run.smithi107.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 47 kB/s | 6.5 kB 00:00 2024-09-10T07:54:23.855 INFO:teuthology.orchestra.run.smithi107.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 4.2 MB/s | 897 kB 00:00 2024-09-10T07:54:23.947 INFO:teuthology.orchestra.run.smithi107.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.6 MB/s | 145 kB 00:00 2024-09-10T07:54:24.022 INFO:teuthology.orchestra.run.smithi107.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-10T07:54:24.033 INFO:teuthology.orchestra.run.smithi155.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 29 kB/s | 6.5 kB 00:00 2024-09-10T07:54:24.123 INFO:teuthology.orchestra.run.smithi107.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 15 MB/s | 1.5 MB 00:00 2024-09-10T07:54:24.289 INFO:teuthology.orchestra.run.smithi107.stdout:(6/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 7.0 MB/s | 2.4 MB 00:00 2024-09-10T07:54:24.373 INFO:teuthology.orchestra.run.smithi107.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 19 MB/s | 4.7 MB 00:00 2024-09-10T07:54:24.665 INFO:teuthology.orchestra.run.smithi107.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 18 MB/s | 21 MB 00:01 2024-09-10T07:54:24.666 INFO:teuthology.orchestra.run.smithi155.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 1.4 MB/s | 897 kB 00:00 2024-09-10T07:54:24.715 INFO:teuthology.orchestra.run.smithi107.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 504 kB/s | 25 kB 00:00 2024-09-10T07:54:24.842 INFO:teuthology.orchestra.run.smithi155.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 830 kB/s | 145 kB 00:00 2024-09-10T07:54:24.915 INFO:teuthology.orchestra.run.smithi107.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 20 MB/s | 11 MB 00:00 2024-09-10T07:54:24.966 INFO:teuthology.orchestra.run.smithi107.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 4.5 MB/s | 228 kB 00:00 2024-09-10T07:54:25.033 INFO:teuthology.orchestra.run.smithi107.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 406 kB/s | 27 kB 00:00 2024-09-10T07:54:25.099 INFO:teuthology.orchestra.run.smithi107.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 478 kB/s | 32 kB 00:00 2024-09-10T07:54:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 8.8 MB/s | 751 kB 00:00 2024-09-10T07:54:25.250 INFO:teuthology.orchestra.run.smithi107.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.4 MB/s | 164 kB 00:00 2024-09-10T07:54:25.317 INFO:teuthology.orchestra.run.smithi107.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 1.9 MB/s | 127 kB 00:00 2024-09-10T07:54:25.392 INFO:teuthology.orchestra.run.smithi107.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 6.6 MB/s | 506 kB 00:00 2024-09-10T07:54:25.709 INFO:teuthology.orchestra.run.smithi107.stdout:(18/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 17 MB/s | 5.3 MB 00:00 2024-09-10T07:54:25.900 INFO:teuthology.orchestra.run.smithi155.stdout:(4/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 2.3 MB/s | 2.4 MB 00:01 2024-09-10T07:54:25.957 INFO:teuthology.orchestra.run.smithi107.stdout:(19/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 10 MB/s | 17 MB 00:01 2024-09-10T07:54:26.001 INFO:teuthology.orchestra.run.smithi107.stdout:(20/156): python3-ceph-argparse-19.1.1-305.g1cf 154 kB/s | 45 kB 00:00 2024-09-10T07:54:26.026 INFO:teuthology.orchestra.run.smithi107.stdout:(21/156): python3-ceph-common-19.1.1-305.g1cf81 1.9 MB/s | 136 kB 00:00 2024-09-10T07:54:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 3.1 MB/s | 161 kB 00:00 2024-09-10T07:54:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 4.2 MB/s | 320 kB 00:00 2024-09-10T07:54:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 3.9 MB/s | 301 kB 00:00 2024-09-10T07:54:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.6 MB/s | 99 kB 00:00 2024-09-10T07:54:26.188 INFO:teuthology.orchestra.run.smithi107.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-10T07:54:26.248 INFO:teuthology.orchestra.run.smithi107.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.8 MB/s | 170 kB 00:00 2024-09-10T07:54:26.297 INFO:teuthology.orchestra.run.smithi107.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 636 kB/s | 32 kB 00:00 2024-09-10T07:54:26.300 INFO:teuthology.orchestra.run.smithi155.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 3.8 MB/s | 1.5 MB 00:00 2024-09-10T07:54:26.341 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-10T07:54:26.356 INFO:teuthology.orchestra.run.smithi107.stdout:(29/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.4 MB/s | 146 kB 00:00 2024-09-10T07:54:26.440 INFO:teuthology.orchestra.run.smithi107.stdout:(30/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 11 MB/s | 3.1 MB 00:00 2024-09-10T07:54:26.573 INFO:teuthology.orchestra.run.smithi107.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 16 MB/s | 3.6 MB 00:00 2024-09-10T07:54:26.640 INFO:teuthology.orchestra.run.smithi107.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 3.7 MB/s | 249 kB 00:00 2024-09-10T07:54:26.699 INFO:teuthology.orchestra.run.smithi107.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 847 kB/s | 49 kB 00:00 2024-09-10T07:54:26.867 INFO:teuthology.orchestra.run.smithi155.stdout:(6/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 1.8 MB/s | 5.4 MB 00:03 2024-09-10T07:54:27.100 INFO:teuthology.orchestra.run.smithi107.stdout:(34/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 20 MB/s | 49 MB 00:02 2024-09-10T07:54:27.124 INFO:teuthology.orchestra.run.smithi107.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 39 kB/s | 16 kB 00:00 2024-09-10T07:54:27.150 INFO:teuthology.orchestra.run.smithi107.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 5.6 MB/s | 285 kB 00:00 2024-09-10T07:54:27.192 INFO:teuthology.orchestra.run.smithi107.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 11 MB/s | 766 kB 00:00 2024-09-10T07:54:27.217 INFO:teuthology.orchestra.run.smithi107.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 273 kB/s | 18 kB 00:00 2024-09-10T07:54:27.325 INFO:teuthology.orchestra.run.smithi107.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 8.3 MB/s | 7.4 MB 00:00 2024-09-10T07:54:27.401 INFO:teuthology.orchestra.run.smithi155.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 4.3 MB/s | 4.7 MB 00:01 2024-09-10T07:54:27.942 INFO:teuthology.orchestra.run.smithi107.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 106 kB/s | 80 kB 00:00 2024-09-10T07:54:28.109 INFO:teuthology.orchestra.run.smithi107.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 244 kB/s | 191 kB 00:00 2024-09-10T07:54:28.143 INFO:teuthology.orchestra.run.smithi107.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 866 kB/s | 801 kB 00:00 2024-09-10T07:54:28.701 INFO:teuthology.orchestra.run.smithi107.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 334 kB/s | 253 kB 00:00 2024-09-10T07:54:28.733 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-10T07:54:28.810 INFO:teuthology.orchestra.run.smithi107.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 160 kB/s | 106 kB 00:00 2024-09-10T07:54:28.917 INFO:teuthology.orchestra.run.smithi155.stdout:(8/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9. 7.0 MB/s | 11 MB 00:01 2024-09-10T07:54:28.951 INFO:teuthology.orchestra.run.smithi155.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 758 kB/s | 25 kB 00:00 2024-09-10T07:54:29.052 INFO:teuthology.orchestra.run.smithi107.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-10T07:54:29.430 INFO:teuthology.orchestra.run.smithi107.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 186 kB/s | 135 kB 00:00 2024-09-10T07:54:29.526 INFO:teuthology.orchestra.run.smithi155.stdout:(10/156): ceph-common-19.1.1-305.g1cf819b7.el9. 3.7 MB/s | 21 MB 00:05 2024-09-10T07:54:29.527 INFO:teuthology.orchestra.run.smithi107.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 174 kB/s | 125 kB 00:00 2024-09-10T07:54:29.593 INFO:teuthology.orchestra.run.smithi155.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 3.4 MB/s | 228 kB 00:00 2024-09-10T07:54:29.635 INFO:teuthology.orchestra.run.smithi155.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 653 kB/s | 27 kB 00:00 2024-09-10T07:54:29.661 INFO:teuthology.orchestra.run.smithi107.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-09-10T07:54:29.685 INFO:teuthology.orchestra.run.smithi155.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 632 kB/s | 32 kB 00:00 2024-09-10T07:54:29.711 INFO:teuthology.orchestra.run.smithi107.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-10T07:54:29.753 INFO:teuthology.orchestra.run.smithi107.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 521 kB/s | 22 kB 00:00 2024-09-10T07:54:29.778 INFO:teuthology.orchestra.run.smithi107.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 943 kB 00:00 2024-09-10T07:54:29.804 INFO:teuthology.orchestra.run.smithi107.stdout:(52/156): python3-urllib3-1.26.5-6.el9.noarch.r 289 kB/s | 215 kB 00:00 2024-09-10T07:54:29.827 INFO:teuthology.orchestra.run.smithi155.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 5.2 MB/s | 751 kB 00:00 2024-09-10T07:54:29.829 INFO:teuthology.orchestra.run.smithi107.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 313 kB 00:00 2024-09-10T07:54:29.846 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-10T07:54:29.854 INFO:teuthology.orchestra.run.smithi107.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 431 kB/s | 33 kB 00:00 2024-09-10T07:54:29.888 INFO:teuthology.orchestra.run.smithi107.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 337 kB/s | 11 kB 00:00 2024-09-10T07:54:29.913 INFO:teuthology.orchestra.run.smithi107.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-09-10T07:54:29.939 INFO:teuthology.orchestra.run.smithi107.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-10T07:54:29.986 INFO:teuthology.orchestra.run.smithi155.stdout:(15/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 5.4 MB/s | 17 MB 00:03 2024-09-10T07:54:29.989 INFO:teuthology.orchestra.run.smithi107.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-09-10T07:54:30.006 INFO:teuthology.orchestra.run.smithi107.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 491 kB/s | 45 kB 00:00 2024-09-10T07:54:30.011 INFO:teuthology.orchestra.run.smithi155.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 891 kB/s | 164 kB 00:00 2024-09-10T07:54:30.031 INFO:teuthology.orchestra.run.smithi107.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-09-10T07:54:30.036 INFO:teuthology.orchestra.run.smithi155.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 2.5 MB/s | 127 kB 00:00 2024-09-10T07:54:30.056 INFO:teuthology.orchestra.run.smithi107.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 486 kB/s | 24 kB 00:00 2024-09-10T07:54:30.082 INFO:teuthology.orchestra.run.smithi107.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-10T07:54:30.103 INFO:teuthology.orchestra.run.smithi155.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 5.4 MB/s | 506 kB 00:00 2024-09-10T07:54:30.107 INFO:teuthology.orchestra.run.smithi107.stdout:(63/156): libtool-2.4.6-46.el9.x86_64.rpm 4.8 MB/s | 581 kB 00:00 2024-09-10T07:54:30.132 INFO:teuthology.orchestra.run.smithi107.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T07:54:30.154 INFO:teuthology.orchestra.run.smithi155.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 905 kB/s | 45 kB 00:00 2024-09-10T07:54:30.157 INFO:teuthology.orchestra.run.smithi107.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-10T07:54:30.183 INFO:teuthology.orchestra.run.smithi107.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-10T07:54:30.204 INFO:teuthology.orchestra.run.smithi155.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 2.7 MB/s | 136 kB 00:00 2024-09-10T07:54:30.254 INFO:teuthology.orchestra.run.smithi155.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 3.2 MB/s | 161 kB 00:00 2024-09-10T07:54:30.275 INFO:teuthology.orchestra.run.smithi107.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-10T07:54:30.317 INFO:teuthology.orchestra.run.smithi107.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-10T07:54:30.321 INFO:teuthology.orchestra.run.smithi155.stdout:(22/156): python3-rados-19.1.1-305.g1cf819b7.el 4.7 MB/s | 320 kB 00:00 2024-09-10T07:54:30.350 INFO:teuthology.orchestra.run.smithi107.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-10T07:54:30.388 INFO:teuthology.orchestra.run.smithi155.stdout:(23/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 4.4 MB/s | 301 kB 00:00 2024-09-10T07:54:30.417 INFO:teuthology.orchestra.run.smithi107.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 17 MB/s | 4.9 MB 00:00 2024-09-10T07:54:30.438 INFO:teuthology.orchestra.run.smithi155.stdout:(24/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T07:54:30.442 INFO:teuthology.orchestra.run.smithi107.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-10T07:54:30.485 INFO:teuthology.orchestra.run.smithi107.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-10T07:54:30.489 INFO:teuthology.orchestra.run.smithi155.stdout:(25/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T07:54:30.510 INFO:teuthology.orchestra.run.smithi107.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-10T07:54:30.552 INFO:teuthology.orchestra.run.smithi107.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T07:54:30.627 INFO:teuthology.orchestra.run.smithi107.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-09-10T07:54:30.652 INFO:teuthology.orchestra.run.smithi107.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-10T07:54:30.677 INFO:teuthology.orchestra.run.smithi107.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-10T07:54:30.703 INFO:teuthology.orchestra.run.smithi107.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-10T07:54:30.728 INFO:teuthology.orchestra.run.smithi107.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-10T07:54:30.753 INFO:teuthology.orchestra.run.smithi107.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-10T07:54:30.779 INFO:teuthology.orchestra.run.smithi107.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-10T07:54:30.781 INFO:teuthology.orchestra.run.smithi155.stdout:(26/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 7.2 MB/s | 5.3 MB 00:00 2024-09-10T07:54:30.840 INFO:teuthology.orchestra.run.smithi155.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.9 MB/s | 170 kB 00:00 2024-09-10T07:54:30.863 INFO:teuthology.orchestra.run.smithi107.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-10T07:54:30.888 INFO:teuthology.orchestra.run.smithi107.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.2 MB/s | 250 kB 00:00 2024-09-10T07:54:30.890 INFO:teuthology.orchestra.run.smithi155.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 638 kB/s | 32 kB 00:00 2024-09-10T07:54:30.913 INFO:teuthology.orchestra.run.smithi107.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-10T07:54:30.939 INFO:teuthology.orchestra.run.smithi107.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T07:54:30.964 INFO:teuthology.orchestra.run.smithi107.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-10T07:54:30.965 INFO:teuthology.orchestra.run.smithi155.stdout:(29/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 6.5 MB/s | 3.1 MB 00:00 2024-09-10T07:54:30.989 INFO:teuthology.orchestra.run.smithi107.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-10T07:54:30.990 INFO:teuthology.orchestra.run.smithi155.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 1.4 MB/s | 146 kB 00:00 2024-09-10T07:54:31.019 INFO:teuthology.orchestra.run.smithi107.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.8 MB/s | 159 kB 00:00 2024-09-10T07:54:31.044 INFO:teuthology.orchestra.run.smithi107.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.0 MB/s | 279 kB 00:00 2024-09-10T07:54:31.069 INFO:teuthology.orchestra.run.smithi107.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T07:54:31.094 INFO:teuthology.orchestra.run.smithi107.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T07:54:31.178 INFO:teuthology.orchestra.run.smithi107.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-10T07:54:31.203 INFO:teuthology.orchestra.run.smithi107.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-10T07:54:31.254 INFO:teuthology.orchestra.run.smithi107.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 590 kB/s | 44 kB 00:00 2024-09-10T07:54:31.279 INFO:teuthology.orchestra.run.smithi107.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-10T07:54:31.304 INFO:teuthology.orchestra.run.smithi107.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-10T07:54:31.329 INFO:teuthology.orchestra.run.smithi107.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-10T07:54:31.355 INFO:teuthology.orchestra.run.smithi107.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-10T07:54:31.380 INFO:teuthology.orchestra.run.smithi107.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T07:54:31.406 INFO:teuthology.orchestra.run.smithi107.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T07:54:31.416 INFO:teuthology.orchestra.run.smithi155.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 7.9 MB/s | 3.6 MB 00:00 2024-09-10T07:54:31.431 INFO:teuthology.orchestra.run.smithi107.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T07:54:31.483 INFO:teuthology.orchestra.run.smithi155.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 3.7 MB/s | 249 kB 00:00 2024-09-10T07:54:31.519 INFO:teuthology.orchestra.run.smithi107.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 394 kB/s | 45 kB 00:00 2024-09-10T07:54:31.533 INFO:teuthology.orchestra.run.smithi155.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 992 kB/s | 49 kB 00:00 2024-09-10T07:54:31.575 INFO:teuthology.orchestra.run.smithi155.stdout:(34/156): ceph-prometheus-alerts-19.1.1-305.g1c 396 kB/s | 16 kB 00:00 2024-09-10T07:54:31.598 INFO:teuthology.orchestra.run.smithi107.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-09-10T07:54:31.642 INFO:teuthology.orchestra.run.smithi155.stdout:(35/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 4.2 MB/s | 285 kB 00:00 2024-09-10T07:54:31.657 INFO:teuthology.orchestra.run.smithi107.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-10T07:54:31.767 INFO:teuthology.orchestra.run.smithi155.stdout:(36/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 6.0 MB/s | 766 kB 00:00 2024-09-10T07:54:31.809 INFO:teuthology.orchestra.run.smithi155.stdout:(37/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 439 kB/s | 18 kB 00:00 2024-09-10T07:54:31.824 INFO:teuthology.orchestra.run.smithi107.stdout:(105/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-10T07:54:31.874 INFO:teuthology.orchestra.run.smithi107.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-09-10T07:54:31.899 INFO:teuthology.orchestra.run.smithi107.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-09-10T07:54:31.910 INFO:teuthology.orchestra.run.smithi155.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 8.0 MB/s | 7.4 MB 00:00 2024-09-10T07:54:31.933 INFO:teuthology.orchestra.run.smithi107.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-09-10T07:54:31.958 INFO:teuthology.orchestra.run.smithi107.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 506 kB/s | 26 kB 00:00 2024-09-10T07:54:31.983 INFO:teuthology.orchestra.run.smithi107.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 1.0 MB/s | 49 kB 00:00 2024-09-10T07:54:32.009 INFO:teuthology.orchestra.run.smithi107.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T07:54:32.026 INFO:teuthology.orchestra.run.smithi107.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-09-10T07:54:32.026 INFO:teuthology.orchestra.run.smithi155.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 369 kB/s | 80 kB 00:00 2024-09-10T07:54:32.068 INFO:teuthology.orchestra.run.smithi107.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-10T07:54:32.101 INFO:teuthology.orchestra.run.smithi107.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-10T07:54:32.110 INFO:teuthology.orchestra.run.smithi155.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-10T07:54:32.126 INFO:teuthology.orchestra.run.smithi107.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 744 kB/s | 43 kB 00:00 2024-09-10T07:54:32.135 INFO:teuthology.orchestra.run.smithi155.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-10T07:54:32.152 INFO:teuthology.orchestra.run.smithi107.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-10T07:54:32.177 INFO:teuthology.orchestra.run.smithi107.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-10T07:54:32.178 INFO:teuthology.orchestra.run.smithi155.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-10T07:54:32.202 INFO:teuthology.orchestra.run.smithi107.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-10T07:54:32.219 INFO:teuthology.orchestra.run.smithi155.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-09-10T07:54:32.244 INFO:teuthology.orchestra.run.smithi107.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-10T07:54:32.245 INFO:teuthology.orchestra.run.smithi155.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-10T07:54:32.270 INFO:teuthology.orchestra.run.smithi107.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-10T07:54:32.270 INFO:teuthology.orchestra.run.smithi155.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-10T07:54:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T07:54:32.295 INFO:teuthology.orchestra.run.smithi155.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T07:54:32.321 INFO:teuthology.orchestra.run.smithi155.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-10T07:54:32.328 INFO:teuthology.orchestra.run.smithi107.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-10T07:54:32.354 INFO:teuthology.orchestra.run.smithi107.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-10T07:54:32.362 INFO:teuthology.orchestra.run.smithi155.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-09-10T07:54:32.371 INFO:teuthology.orchestra.run.smithi107.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-10T07:54:32.387 INFO:teuthology.orchestra.run.smithi107.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-10T07:54:32.404 INFO:teuthology.orchestra.run.smithi155.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-09-10T07:54:32.405 INFO:teuthology.orchestra.run.smithi107.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-10T07:54:32.422 INFO:teuthology.orchestra.run.smithi107.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-10T07:54:32.430 INFO:teuthology.orchestra.run.smithi155.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-10T07:54:32.447 INFO:teuthology.orchestra.run.smithi155.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 518 kB/s | 22 kB 00:00 2024-09-10T07:54:32.480 INFO:teuthology.orchestra.run.smithi155.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-10T07:54:32.488 INFO:teuthology.orchestra.run.smithi107.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-09-10T07:54:32.497 INFO:teuthology.orchestra.run.smithi155.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-10T07:54:32.522 INFO:teuthology.orchestra.run.smithi107.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-09-10T07:54:32.531 INFO:teuthology.orchestra.run.smithi155.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 550 kB/s | 18 kB 00:00 2024-09-10T07:54:32.548 INFO:teuthology.orchestra.run.smithi107.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 786 kB/s | 46 kB 00:00 2024-09-10T07:54:32.573 INFO:teuthology.orchestra.run.smithi107.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-10T07:54:32.581 INFO:teuthology.orchestra.run.smithi155.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-09-10T07:54:32.598 INFO:teuthology.orchestra.run.smithi107.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-10T07:54:32.607 INFO:teuthology.orchestra.run.smithi155.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 147 kB/s | 11 kB 00:00 2024-09-10T07:54:32.623 INFO:teuthology.orchestra.run.smithi107.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-10T07:54:32.640 INFO:teuthology.orchestra.run.smithi155.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-10T07:54:32.649 INFO:teuthology.orchestra.run.smithi107.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-10T07:54:32.666 INFO:teuthology.orchestra.run.smithi155.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-10T07:54:32.674 INFO:teuthology.orchestra.run.smithi107.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-10T07:54:32.699 INFO:teuthology.orchestra.run.smithi107.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-10T07:54:32.708 INFO:teuthology.orchestra.run.smithi155.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-10T07:54:32.725 INFO:teuthology.orchestra.run.smithi107.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-10T07:54:32.741 INFO:teuthology.orchestra.run.smithi155.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-10T07:54:32.750 INFO:teuthology.orchestra.run.smithi107.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T07:54:32.767 INFO:teuthology.orchestra.run.smithi155.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-10T07:54:32.775 INFO:teuthology.orchestra.run.smithi107.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-10T07:54:32.792 INFO:teuthology.orchestra.run.smithi155.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-10T07:54:32.801 INFO:teuthology.orchestra.run.smithi107.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-10T07:54:32.817 INFO:teuthology.orchestra.run.smithi155.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-10T07:54:32.842 INFO:teuthology.orchestra.run.smithi155.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T07:54:32.868 INFO:teuthology.orchestra.run.smithi155.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-10T07:54:32.868 INFO:teuthology.orchestra.run.smithi107.stdout:(141/156): libarrow-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 4.4 MB 00:00 2024-09-10T07:54:32.893 INFO:teuthology.orchestra.run.smithi107.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 635 kB/s | 74 kB 00:00 2024-09-10T07:54:32.919 INFO:teuthology.orchestra.run.smithi107.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T07:54:32.952 INFO:teuthology.orchestra.run.smithi107.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-09-10T07:54:32.968 INFO:teuthology.orchestra.run.smithi155.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 39 MB/s | 4.9 MB 00:00 2024-09-10T07:54:32.977 INFO:teuthology.orchestra.run.smithi107.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-09-10T07:54:33.003 INFO:teuthology.orchestra.run.smithi107.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T07:54:33.028 INFO:teuthology.orchestra.run.smithi107.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-10T07:54:33.035 INFO:teuthology.orchestra.run.smithi155.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-09-10T07:54:33.077 INFO:teuthology.orchestra.run.smithi155.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-10T07:54:33.078 INFO:teuthology.orchestra.run.smithi107.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T07:54:33.119 INFO:teuthology.orchestra.run.smithi155.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-10T07:54:33.161 INFO:teuthology.orchestra.run.smithi155.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-10T07:54:33.228 INFO:teuthology.orchestra.run.smithi155.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-10T07:54:33.278 INFO:teuthology.orchestra.run.smithi107.stdout:(149/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 13 MB/s | 3.2 MB 00:00 2024-09-10T07:54:33.320 INFO:teuthology.orchestra.run.smithi155.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-09-10T07:54:33.345 INFO:teuthology.orchestra.run.smithi155.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 750 kB/s | 87 kB 00:00 2024-09-10T07:54:33.354 INFO:teuthology.orchestra.run.smithi107.stdout:(150/156): librados2-19.1.1-305.g1cf819b7.el9.x 9.8 MB/s | 3.4 MB 00:00 2024-09-10T07:54:33.391 INFO:teuthology.orchestra.run.smithi155.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.3 MB/s | 61 kB 00:00 2024-09-10T07:54:33.475 INFO:teuthology.orchestra.run.smithi155.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 39 MB/s | 6.0 MB 00:00 2024-09-10T07:54:33.500 INFO:teuthology.orchestra.run.smithi155.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T07:54:33.525 INFO:teuthology.orchestra.run.smithi155.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-10T07:54:33.551 INFO:teuthology.orchestra.run.smithi155.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-10T07:54:33.576 INFO:teuthology.orchestra.run.smithi155.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-10T07:54:33.601 INFO:teuthology.orchestra.run.smithi155.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-10T07:54:33.627 INFO:teuthology.orchestra.run.smithi155.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-10T07:54:33.677 INFO:teuthology.orchestra.run.smithi155.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-10T07:54:33.744 INFO:teuthology.orchestra.run.smithi155.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-09-10T07:54:33.769 INFO:teuthology.orchestra.run.smithi155.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-10T07:54:33.786 INFO:teuthology.orchestra.run.smithi155.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-09-10T07:54:33.811 INFO:teuthology.orchestra.run.smithi155.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-10T07:54:33.853 INFO:teuthology.orchestra.run.smithi155.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-09-10T07:54:33.878 INFO:teuthology.orchestra.run.smithi155.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-10T07:54:33.904 INFO:teuthology.orchestra.run.smithi155.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-10T07:54:33.929 INFO:teuthology.orchestra.run.smithi155.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T07:54:33.954 INFO:teuthology.orchestra.run.smithi155.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T07:54:33.970 INFO:teuthology.orchestra.run.smithi107.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 66 kB/s | 41 kB 00:00 2024-09-10T07:54:33.997 INFO:teuthology.orchestra.run.smithi155.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-10T07:54:34.039 INFO:teuthology.orchestra.run.smithi155.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-10T07:54:34.089 INFO:teuthology.orchestra.run.smithi155.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-09-10T07:54:34.104 INFO:teuthology.orchestra.run.smithi107.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.3 MB/s | 2.3 MB 00:01 2024-09-10T07:54:34.139 INFO:teuthology.orchestra.run.smithi155.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-09-10T07:54:34.288 INFO:teuthology.orchestra.run.smithi107.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 1.8 MB/s | 1.8 MB 00:01 2024-09-10T07:54:34.306 INFO:teuthology.orchestra.run.smithi155.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-09-10T07:54:34.331 INFO:teuthology.orchestra.run.smithi155.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-09-10T07:54:34.357 INFO:teuthology.orchestra.run.smithi155.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-10T07:54:34.382 INFO:teuthology.orchestra.run.smithi155.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T07:54:34.407 INFO:teuthology.orchestra.run.smithi155.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T07:54:34.433 INFO:teuthology.orchestra.run.smithi155.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T07:54:34.458 INFO:teuthology.orchestra.run.smithi155.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 888 kB/s | 45 kB 00:00 2024-09-10T07:54:34.771 INFO:teuthology.orchestra.run.smithi107.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 311 kB/s | 249 kB 00:00 2024-09-10T07:54:34.942 INFO:teuthology.orchestra.run.smithi155.stdout:(103/156): ceph-test-19.1.1-305.g1cf819b7.el9.x 8.1 MB/s | 49 MB 00:05 2024-09-10T07:54:34.992 INFO:teuthology.orchestra.run.smithi155.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T07:54:35.017 INFO:teuthology.orchestra.run.smithi155.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 38 kB/s | 22 kB 00:00 2024-09-10T07:54:35.051 INFO:teuthology.orchestra.run.smithi155.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-09-10T07:54:35.076 INFO:teuthology.orchestra.run.smithi155.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-09-10T07:54:35.079 INFO:teuthology.orchestra.run.smithi107.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-10T07:54:35.101 INFO:teuthology.orchestra.run.smithi155.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T07:54:35.127 INFO:teuthology.orchestra.run.smithi155.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T07:54:35.160 INFO:teuthology.orchestra.run.smithi155.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-09-10T07:54:35.194 INFO:teuthology.orchestra.run.smithi155.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-10T07:54:35.228 INFO:teuthology.orchestra.run.smithi155.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-10T07:54:35.270 INFO:teuthology.orchestra.run.smithi155.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-10T07:54:35.303 INFO:teuthology.orchestra.run.smithi155.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-10T07:54:35.305 INFO:teuthology.orchestra.run.smithi107.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.4 MB/s | 2.4 MB 00:01 2024-09-10T07:54:35.311 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T07:54:35.311 INFO:teuthology.orchestra.run.smithi107.stdout:Total 19 MB/s | 226 MB 00:12 2024-09-10T07:54:35.328 INFO:teuthology.orchestra.run.smithi155.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 745 kB/s | 43 kB 00:00 2024-09-10T07:54:35.354 INFO:teuthology.orchestra.run.smithi155.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-09-10T07:54:35.379 INFO:teuthology.orchestra.run.smithi155.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-10T07:54:35.404 INFO:teuthology.orchestra.run.smithi155.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T07:54:35.438 INFO:teuthology.orchestra.run.smithi155.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T07:54:35.463 INFO:teuthology.orchestra.run.smithi155.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-10T07:54:35.489 INFO:teuthology.orchestra.run.smithi155.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T07:54:35.522 INFO:teuthology.orchestra.run.smithi155.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T07:54:35.547 INFO:teuthology.orchestra.run.smithi155.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-10T07:54:35.573 INFO:teuthology.orchestra.run.smithi155.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 712 kB/s | 18 kB 00:00 2024-09-10T07:54:35.598 INFO:teuthology.orchestra.run.smithi155.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-10T07:54:35.623 INFO:teuthology.orchestra.run.smithi155.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 779 kB/s | 19 kB 00:00 2024-09-10T07:54:35.648 INFO:teuthology.orchestra.run.smithi155.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 794 kB/s | 20 kB 00:00 2024-09-10T07:54:35.707 INFO:teuthology.orchestra.run.smithi155.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-09-10T07:54:35.732 INFO:teuthology.orchestra.run.smithi155.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T07:54:35.774 INFO:teuthology.orchestra.run.smithi155.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-09-10T07:54:35.808 INFO:teuthology.orchestra.run.smithi155.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-09-10T07:54:35.833 INFO:teuthology.orchestra.run.smithi155.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 982 kB/s | 58 kB 00:00 2024-09-10T07:54:35.859 INFO:teuthology.orchestra.run.smithi155.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-10T07:54:35.918 INFO:teuthology.orchestra.run.smithi155.stdout:(134/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-09-10T07:54:35.943 INFO:teuthology.orchestra.run.smithi155.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-09-10T07:54:35.968 INFO:teuthology.orchestra.run.smithi155.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-10T07:54:35.993 INFO:teuthology.orchestra.run.smithi155.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-10T07:54:36.018 INFO:teuthology.orchestra.run.smithi155.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-09-10T07:54:36.044 INFO:teuthology.orchestra.run.smithi155.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-10T07:54:36.069 INFO:teuthology.orchestra.run.smithi155.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-10T07:54:36.095 INFO:teuthology.orchestra.run.smithi155.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T07:54:36.120 INFO:teuthology.orchestra.run.smithi155.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-10T07:54:36.145 INFO:teuthology.orchestra.run.smithi155.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T07:54:36.170 INFO:teuthology.orchestra.run.smithi155.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 806 kB/s | 20 kB 00:00 2024-09-10T07:54:36.196 INFO:teuthology.orchestra.run.smithi155.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-10T07:54:36.221 INFO:teuthology.orchestra.run.smithi155.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T07:54:36.246 INFO:teuthology.orchestra.run.smithi155.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-10T07:54:36.296 INFO:teuthology.orchestra.run.smithi155.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T07:54:36.372 INFO:teuthology.orchestra.run.smithi155.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 31 MB/s | 2.3 MB 00:00 2024-09-10T07:54:36.447 INFO:teuthology.orchestra.run.smithi155.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-09-10T07:54:36.468 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-10T07:54:36.489 INFO:teuthology.orchestra.run.smithi155.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 978 kB/s | 41 kB 00:00 2024-09-10T07:54:36.539 INFO:teuthology.orchestra.run.smithi155.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-10T07:54:36.548 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T07:54:36.598 INFO:teuthology.orchestra.run.smithi155.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 23 MB/s | 1.4 MB 00:00 2024-09-10T07:54:36.673 INFO:teuthology.orchestra.run.smithi155.stdout:(154/156): librados2-19.1.1-305.g1cf819b7.el9.x 7.6 MB/s | 3.4 MB 00:00 2024-09-10T07:54:36.700 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T07:54:36.700 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T07:54:36.723 INFO:teuthology.orchestra.run.smithi155.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-09-10T07:54:36.815 INFO:teuthology.orchestra.run.smithi155.stdout:(156/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 5.6 MB/s | 3.2 MB 00:00 2024-09-10T07:54:36.821 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-09-10T07:54:36.821 INFO:teuthology.orchestra.run.smithi155.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-10T07:54:38.061 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-09-10T07:54:38.212 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-09-10T07:54:38.212 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-09-10T07:54:38.567 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T07:54:38.567 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T07:54:39.081 INFO:teuthology.orchestra.run.smithi002.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:54:39.082 INFO:teuthology.orchestra.run.smithi002.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T07:54:39.093 INFO:teuthology.orchestra.run.smithi002.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T07:54:39.094 INFO:teuthology.orchestra.run.smithi002.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T07:54:39.095 INFO:teuthology.orchestra.run.smithi002.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T07:54:39.096 INFO:teuthology.orchestra.run.smithi002.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T07:54:39.098 INFO:teuthology.orchestra.run.smithi002.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T07:54:39.100 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T07:54:39.101 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T07:54:39.170 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-10T07:54:39.185 INFO:teuthology.orchestra.run.smithi002.stdout:====================================================================================== 2024-09-10T07:54:39.185 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-09-10T07:54:39.185 INFO:teuthology.orchestra.run.smithi002.stdout:====================================================================================== 2024-09-10T07:54:39.185 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-10T07:54:39.185 INFO:teuthology.orchestra.run.smithi002.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T07:54:39.186 INFO:teuthology.orchestra.run.smithi002.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T07:54:39.186 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T07:54:39.186 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T07:54:39.186 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T07:54:39.186 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T07:54:39.186 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T07:54:39.186 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T07:54:39.187 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T07:54:39.187 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T07:54:39.187 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T07:54:39.187 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T07:54:39.187 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T07:54:39.187 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T07:54:39.187 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T07:54:39.187 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T07:54:39.188 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T07:54:39.188 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T07:54:39.188 INFO:teuthology.orchestra.run.smithi002.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T07:54:39.188 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T07:54:39.189 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T07:54:39.189 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T07:54:39.189 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T07:54:39.189 INFO:teuthology.orchestra.run.smithi002.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T07:54:39.189 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T07:54:39.189 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T07:54:39.189 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T07:54:39.189 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T07:54:39.189 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T07:54:39.190 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T07:54:39.190 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T07:54:39.190 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T07:54:39.190 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T07:54:39.190 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T07:54:39.190 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T07:54:39.190 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T07:54:39.190 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T07:54:39.190 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrading: 2024-09-10T07:54:39.190 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T07:54:39.190 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T07:54:39.190 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T07:54:39.191 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T07:54:39.192 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T07:54:39.192 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T07:54:39.192 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T07:54:39.192 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T07:54:39.192 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T07:54:39.192 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T07:54:39.192 INFO:teuthology.orchestra.run.smithi002.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T07:54:39.192 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T07:54:39.192 INFO:teuthology.orchestra.run.smithi002.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T07:54:39.192 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T07:54:39.192 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T07:54:39.192 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T07:54:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T07:54:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T07:54:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T07:54:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T07:54:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T07:54:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T07:54:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T07:54:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T07:54:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T07:54:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T07:54:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T07:54:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T07:54:39.194 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T07:54:39.195 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T07:54:39.196 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T07:54:39.197 INFO:teuthology.orchestra.run.smithi002.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T07:54:39.197 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T07:54:39.197 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T07:54:39.197 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T07:54:39.197 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T07:54:39.197 INFO:teuthology.orchestra.run.smithi002.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T07:54:39.197 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T07:54:39.197 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T07:54:39.197 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T07:54:39.197 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T07:54:39.197 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T07:54:39.197 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T07:54:39.197 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T07:54:39.198 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T07:54:39.199 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T07:54:39.199 INFO:teuthology.orchestra.run.smithi002.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T07:54:39.199 INFO:teuthology.orchestra.run.smithi002.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T07:54:39.199 INFO:teuthology.orchestra.run.smithi002.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T07:54:39.199 INFO:teuthology.orchestra.run.smithi002.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T07:54:39.199 INFO:teuthology.orchestra.run.smithi002.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T07:54:39.199 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T07:54:39.199 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T07:54:39.199 INFO:teuthology.orchestra.run.smithi002.stdout:Installing weak dependencies: 2024-09-10T07:54:39.199 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T07:54:39.200 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:54:39.200 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-10T07:54:39.200 INFO:teuthology.orchestra.run.smithi002.stdout:====================================================================================== 2024-09-10T07:54:39.200 INFO:teuthology.orchestra.run.smithi002.stdout:Install 148 Packages 2024-09-10T07:54:39.200 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrade 8 Packages 2024-09-10T07:54:39.200 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:54:39.200 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 226 M 2024-09-10T07:54:39.200 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-10T07:54:39.561 INFO:teuthology.orchestra.run.smithi002.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 53 kB/s | 6.5 kB 00:00 2024-09-10T07:54:39.770 INFO:teuthology.orchestra.run.smithi002.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 4.2 MB/s | 897 kB 00:00 2024-09-10T07:54:39.822 INFO:teuthology.orchestra.run.smithi002.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 2.7 MB/s | 145 kB 00:00 2024-09-10T07:54:40.022 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-09-10T07:54:40.023 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-09-10T07:54:40.081 INFO:teuthology.orchestra.run.smithi002.stdout:(4/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 9.2 MB/s | 2.4 MB 00:00 2024-09-10T07:54:40.198 INFO:teuthology.orchestra.run.smithi002.stdout:(5/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 7.1 MB/s | 5.4 MB 00:00 2024-09-10T07:54:40.281 INFO:teuthology.orchestra.run.smithi002.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 7.6 MB/s | 1.5 MB 00:00 2024-09-10T07:54:40.590 INFO:teuthology.orchestra.run.smithi002.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-10T07:54:40.999 INFO:teuthology.orchestra.run.smithi002.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 14 MB/s | 21 MB 00:01 2024-09-10T07:54:41.040 INFO:teuthology.orchestra.run.smithi002.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 611 kB/s | 25 kB 00:00 2024-09-10T07:54:41.404 INFO:teuthology.orchestra.run.smithi002.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 13 MB/s | 11 MB 00:00 2024-09-10T07:54:41.625 INFO:teuthology.orchestra.run.smithi002.stdout:(11/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 13 MB/s | 17 MB 00:01 2024-09-10T07:54:41.658 INFO:teuthology.orchestra.run.smithi002.stdout:(12/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 902 kB/s | 228 kB 00:00 2024-09-10T07:54:41.675 INFO:teuthology.orchestra.run.smithi002.stdout:(13/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 542 kB/s | 27 kB 00:00 2024-09-10T07:54:41.700 INFO:teuthology.orchestra.run.smithi002.stdout:(14/156): libcephfs-devel-19.1.1-305.g1cf819b7. 758 kB/s | 32 kB 00:00 2024-09-10T07:54:41.750 INFO:teuthology.orchestra.run.smithi002.stdout:(15/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 9.8 MB/s | 751 kB 00:00 2024-09-10T07:54:41.801 INFO:teuthology.orchestra.run.smithi002.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 1.6 MB/s | 164 kB 00:00 2024-09-10T07:54:41.826 INFO:teuthology.orchestra.run.smithi002.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 1.6 MB/s | 127 kB 00:00 2024-09-10T07:54:41.868 INFO:teuthology.orchestra.run.smithi002.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 7.4 MB/s | 506 kB 00:00 2024-09-10T07:54:41.918 INFO:teuthology.orchestra.run.smithi002.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 904 kB/s | 45 kB 00:00 2024-09-10T07:54:41.985 INFO:teuthology.orchestra.run.smithi002.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 2.0 MB/s | 136 kB 00:00 2024-09-10T07:54:42.060 INFO:teuthology.orchestra.run.smithi002.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.1 MB/s | 161 kB 00:00 2024-09-10T07:54:42.176 INFO:teuthology.orchestra.run.smithi002.stdout:(22/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-10T07:54:42.211 INFO:teuthology.orchestra.run.smithi002.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 2.1 MB/s | 320 kB 00:00 2024-09-10T07:54:42.236 INFO:teuthology.orchestra.run.smithi002.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 4.9 MB/s | 301 kB 00:00 2024-09-10T07:54:42.261 INFO:teuthology.orchestra.run.smithi002.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T07:54:42.287 INFO:teuthology.orchestra.run.smithi002.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T07:54:42.345 INFO:teuthology.orchestra.run.smithi002.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.9 MB/s | 170 kB 00:00 2024-09-10T07:54:42.421 INFO:teuthology.orchestra.run.smithi002.stdout:(28/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 20 MB/s | 3.1 MB 00:00 2024-09-10T07:54:42.438 INFO:teuthology.orchestra.run.smithi002.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 346 kB/s | 32 kB 00:00 2024-09-10T07:54:42.471 INFO:teuthology.orchestra.run.smithi002.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.9 MB/s | 146 kB 00:00 2024-09-10T07:54:42.689 INFO:teuthology.orchestra.run.smithi002.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 14 MB/s | 3.6 MB 00:00 2024-09-10T07:54:42.764 INFO:teuthology.orchestra.run.smithi002.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 3.2 MB/s | 249 kB 00:00 2024-09-10T07:54:42.898 INFO:teuthology.orchestra.run.smithi002.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-10T07:54:42.940 INFO:teuthology.orchestra.run.smithi002.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 281 kB/s | 49 kB 00:00 2024-09-10T07:54:42.973 INFO:teuthology.orchestra.run.smithi002.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 219 kB/s | 16 kB 00:00 2024-09-10T07:54:42.998 INFO:teuthology.orchestra.run.smithi002.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 4.8 MB/s | 285 kB 00:00 2024-09-10T07:54:43.032 INFO:teuthology.orchestra.run.smithi002.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 13 MB/s | 766 kB 00:00 2024-09-10T07:54:43.057 INFO:teuthology.orchestra.run.smithi002.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 311 kB/s | 18 kB 00:00 2024-09-10T07:54:43.158 INFO:teuthology.orchestra.run.smithi002.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 638 kB/s | 80 kB 00:00 2024-09-10T07:54:43.224 INFO:teuthology.orchestra.run.smithi002.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-10T07:54:43.266 INFO:teuthology.orchestra.run.smithi002.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.8 MB/s | 801 kB 00:00 2024-09-10T07:54:43.292 INFO:teuthology.orchestra.run.smithi002.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-10T07:54:43.350 INFO:teuthology.orchestra.run.smithi002.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-10T07:54:43.384 INFO:teuthology.orchestra.run.smithi002.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-09-10T07:54:43.926 INFO:teuthology.orchestra.run.smithi002.stdout:(45/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 17 MB/s | 49 MB 00:02 2024-09-10T07:54:43.943 INFO:teuthology.orchestra.run.smithi002.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 228 kB/s | 135 kB 00:00 2024-09-10T07:54:43.968 INFO:teuthology.orchestra.run.smithi002.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 214 kB/s | 125 kB 00:00 2024-09-10T07:54:44.002 INFO:teuthology.orchestra.run.smithi002.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-10T07:54:44.710 INFO:teuthology.orchestra.run.smithi002.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 151 kB/s | 107 kB 00:00 2024-09-10T07:54:44.784 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T07:54:44.844 INFO:teuthology.orchestra.run.smithi002.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 765 kB/s | 689 kB 00:00 2024-09-10T07:54:44.883 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T07:54:44.902 INFO:teuthology.orchestra.run.smithi002.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-10T07:54:44.989 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T07:54:45.186 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T07:54:45.226 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T07:54:45.251 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T07:54:45.296 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T07:54:45.303 INFO:teuthology.orchestra.run.smithi002.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 37 kB/s | 22 kB 00:00 2024-09-10T07:54:45.528 INFO:teuthology.orchestra.run.smithi002.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 52 kB/s | 33 kB 00:00 2024-09-10T07:54:45.623 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T07:54:45.651 INFO:teuthology.orchestra.run.smithi002.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 389 kB/s | 313 kB 00:00 2024-09-10T07:54:45.735 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:54:45.856 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:54:45.878 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:54:45.969 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:54:45.999 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:54:46.078 INFO:teuthology.orchestra.run.smithi002.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 34 kB/s | 18 kB 00:00 2024-09-10T07:54:46.120 INFO:teuthology.orchestra.run.smithi002.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 24 kB/s | 11 kB 00:00 2024-09-10T07:54:46.148 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T07:54:46.196 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T07:54:46.224 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:54:46.324 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-09-10T07:54:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:54:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:54:46.421 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:54:46.427 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T07:54:46.443 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:54:46.463 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:54:46.504 INFO:teuthology.orchestra.run.smithi002.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-09-10T07:54:46.560 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T07:54:46.645 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:54:46.737 INFO:teuthology.orchestra.run.smithi002.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-09-10T07:54:46.789 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T07:54:46.792 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:54:46.830 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T07:54:46.846 INFO:teuthology.orchestra.run.smithi002.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 209 kB/s | 160 kB 00:00 2024-09-10T07:54:46.863 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T07:54:46.908 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T07:54:46.909 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T07:54:46.993 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:54:47.077 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:54:47.246 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T07:54:47.300 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:54:47.355 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:54:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 44 kB/s | 24 kB 00:00 2024-09-10T07:54:47.411 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T07:54:47.438 INFO:teuthology.orchestra.run.smithi002.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 709 kB/s | 662 kB 00:00 2024-09-10T07:54:47.477 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:54:47.504 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T07:54:47.506 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:54:47.541 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T07:54:47.575 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T07:54:47.606 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:54:47.630 INFO:teuthology.orchestra.run.smithi002.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 660 kB/s | 581 kB 00:00 2024-09-10T07:54:47.636 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:54:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T07:54:47.693 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T07:54:47.725 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T07:54:47.777 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T07:54:47.787 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T07:54:47.810 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T07:54:47.832 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T07:54:47.861 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:54:47.862 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T07:54:47.932 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T07:54:47.964 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:54:47.984 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T07:54:47.995 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:54:48.014 INFO:teuthology.orchestra.run.smithi002.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 78 kB/s | 47 kB 00:00 2024-09-10T07:54:48.023 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T07:54:48.058 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:54:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T07:54:48.079 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:54:48.105 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:54:48.136 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T07:54:48.170 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T07:54:48.214 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T07:54:48.243 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T07:54:48.265 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:54:48.281 INFO:teuthology.orchestra.run.smithi002.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 347 kB/s | 292 kB 00:00 2024-09-10T07:54:48.313 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T07:54:48.357 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T07:54:48.373 INFO:teuthology.orchestra.run.smithi002.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 258 kB/s | 188 kB 00:00 2024-09-10T07:54:48.397 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:54:48.432 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T07:54:48.512 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T07:54:48.571 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T07:54:48.590 INFO:teuthology.orchestra.run.smithi002.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 67 kB/s | 38 kB 00:00 2024-09-10T07:54:48.625 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:54:48.626 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T07:54:48.705 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:54:48.930 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:54:49.038 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T07:54:49.191 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T07:54:49.191 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T07:54:49.191 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T07:54:49.220 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T07:54:49.239 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T07:54:49.277 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T07:54:49.294 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T07:54:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T07:54:49.347 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T07:54:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T07:54:49.389 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T07:54:49.390 INFO:teuthology.orchestra.run.smithi002.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.5 MB/s | 4.9 MB 00:01 2024-09-10T07:54:49.402 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T07:54:49.430 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T07:54:49.462 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T07:54:49.490 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T07:54:49.527 INFO:teuthology.orchestra.run.smithi002.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-09-10T07:54:49.559 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T07:54:49.560 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T07:54:49.621 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T07:54:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T07:54:49.659 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T07:54:49.674 INFO:teuthology.orchestra.run.smithi002.stdout:(69/156): postgresql-13.16-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-09-10T07:54:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T07:54:49.695 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T07:54:49.750 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T07:54:49.771 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T07:54:49.782 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T07:54:49.806 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T07:54:49.818 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T07:54:49.849 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T07:54:49.865 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T07:54:49.878 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T07:54:49.901 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T07:54:49.944 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T07:54:49.949 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T07:54:49.987 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T07:54:49.993 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T07:54:50.069 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T07:54:50.124 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T07:54:50.172 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T07:54:50.209 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T07:54:50.213 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T07:54:50.263 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T07:54:50.315 INFO:teuthology.orchestra.run.smithi002.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 174 kB/s | 137 kB 00:00 2024-09-10T07:54:50.317 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T07:54:50.349 INFO:teuthology.orchestra.run.smithi002.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 949 kB/s | 910 kB 00:00 2024-09-10T07:54:50.355 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T07:54:50.395 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T07:54:50.588 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T07:54:50.613 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T07:54:50.690 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T07:54:50.804 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T07:54:50.817 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T07:54:50.849 INFO:teuthology.orchestra.run.smithi002.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 4.9 MB/s | 5.8 MB 00:01 2024-09-10T07:54:50.857 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T07:54:50.878 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T07:54:50.885 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T07:54:50.907 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T07:54:50.920 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T07:54:50.950 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T07:54:50.959 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T07:54:50.988 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T07:54:50.991 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T07:54:51.021 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T07:54:51.032 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T07:54:51.041 INFO:teuthology.orchestra.run.smithi002.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 126 kB/s | 87 kB 00:00 2024-09-10T07:54:51.055 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T07:54:51.089 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T07:54:51.130 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T07:54:51.165 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T07:54:51.215 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T07:54:51.233 INFO:teuthology.orchestra.run.smithi002.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-09-10T07:54:51.251 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T07:54:51.275 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T07:54:51.303 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T07:54:51.329 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T07:54:51.334 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T07:54:51.404 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T07:54:51.434 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T07:54:51.436 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T07:54:51.470 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T07:54:51.494 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T07:54:51.518 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T07:54:51.553 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T07:54:51.605 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T07:54:51.647 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T07:54:51.666 INFO:teuthology.orchestra.run.smithi002.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 98 kB/s | 61 kB 00:00 2024-09-10T07:54:51.793 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T07:54:51.832 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T07:54:51.874 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T07:54:51.975 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T07:54:52.000 INFO:teuthology.orchestra.run.smithi002.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-10T07:54:52.007 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T07:54:52.047 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T07:54:52.109 INFO:teuthology.orchestra.run.smithi002.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 280 kB/s | 245 kB 00:00 2024-09-10T07:54:52.165 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T07:54:52.205 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T07:54:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T07:54:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T07:54:52.343 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T07:54:52.382 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T07:54:52.425 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T07:54:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T07:54:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T07:54:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 298 kB/s | 248 kB 00:00 2024-09-10T07:54:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T07:54:52.567 INFO:teuthology.orchestra.run.smithi002.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 45 kB/s | 26 kB 00:00 2024-09-10T07:54:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T07:54:52.601 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T07:54:52.628 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T07:54:52.629 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T07:54:52.663 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T07:54:52.676 INFO:teuthology.orchestra.run.smithi002.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 34 kB/s | 19 kB 00:00 2024-09-10T07:54:52.702 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T07:54:52.734 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T07:54:52.763 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T07:54:52.798 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T07:54:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T07:54:52.832 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T07:54:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T07:54:52.873 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T07:54:52.908 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T07:54:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T07:54:52.995 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T07:54:53.047 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T07:54:53.086 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T07:54:53.190 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T07:54:53.193 INFO:teuthology.orchestra.run.smithi002.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-09-10T07:54:53.246 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T07:54:53.251 INFO:teuthology.orchestra.run.smithi002.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 229 kB/s | 172 kB 00:00 2024-09-10T07:54:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T07:54:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T07:54:53.802 INFO:teuthology.orchestra.run.smithi002.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-10T07:54:53.960 INFO:teuthology.orchestra.run.smithi002.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 327 kB/s | 250 kB 00:00 2024-09-10T07:54:53.973 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T07:54:54.002 INFO:teuthology.orchestra.run.smithi002.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 296 kB/s | 222 kB 00:00 2024-09-10T07:54:54.027 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T07:54:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T07:54:54.161 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T07:54:54.225 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T07:54:54.269 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T07:54:54.356 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T07:54:54.389 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T07:54:54.453 INFO:teuthology.orchestra.run.smithi002.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 119 kB/s | 77 kB 00:00 2024-09-10T07:54:54.553 INFO:teuthology.orchestra.run.smithi002.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 71 kB/s | 42 kB 00:00 2024-09-10T07:54:54.584 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T07:54:54.611 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T07:54:54.689 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T07:54:54.820 INFO:teuthology.orchestra.run.smithi002.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 330 kB/s | 269 kB 00:00 2024-09-10T07:54:54.915 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T07:54:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T07:54:55.034 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T07:54:55.067 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T07:54:55.195 INFO:teuthology.orchestra.run.smithi002.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 214 kB/s | 159 kB 00:00 2024-09-10T07:54:55.341 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T07:54:55.376 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T07:54:55.413 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:54:55.462 INFO:teuthology.orchestra.run.smithi002.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 307 kB/s | 279 kB 00:00 2024-09-10T07:54:55.469 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T07:54:55.487 INFO:teuthology.orchestra.run.smithi002.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 76 kB/s | 51 kB 00:00 2024-09-10T07:54:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:54:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T07:54:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T07:54:55.921 INFO:teuthology.orchestra.run.smithi002.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 74 kB/s | 54 kB 00:00 2024-09-10T07:54:56.051 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T07:54:56.063 INFO:teuthology.orchestra.run.smithi002.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 72 kB/s | 42 kB 00:00 2024-09-10T07:54:56.150 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:54:56.222 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:54:56.252 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T07:54:56.497 INFO:teuthology.orchestra.run.smithi002.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 77 kB/s | 44 kB 00:00 2024-09-10T07:54:56.703 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T07:54:56.793 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T07:54:56.821 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T07:54:56.863 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T07:54:56.941 INFO:teuthology.orchestra.run.smithi002.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-10T07:54:56.972 INFO:teuthology.orchestra.run.smithi002.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 243 kB/s | 221 kB 00:00 2024-09-10T07:54:57.176 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T07:54:57.217 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:54:57.222 INFO:teuthology.orchestra.run.smithi002.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 49 kB/s | 36 kB 00:00 2024-09-10T07:54:57.308 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:54:57.339 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T07:54:57.375 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T07:54:57.531 INFO:teuthology.orchestra.run.smithi002.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-09-10T07:54:57.740 INFO:teuthology.orchestra.run.smithi002.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 396 kB/s | 304 kB 00:00 2024-09-10T07:54:57.840 INFO:teuthology.orchestra.run.smithi002.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 94 kB/s | 58 kB 00:00 2024-09-10T07:54:57.856 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T07:54:57.887 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:54:57.942 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:54:57.972 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T07:54:58.198 INFO:teuthology.orchestra.run.smithi002.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 144 kB/s | 96 kB 00:00 2024-09-10T07:54:58.290 INFO:teuthology.orchestra.run.smithi002.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 245 kB/s | 22 kB 00:00 2024-09-10T07:54:58.399 INFO:teuthology.orchestra.run.smithi002.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 97 kB/s | 64 kB 00:00 2024-09-10T07:54:58.441 INFO:teuthology.orchestra.run.smithi002.stdout:(104/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 75 kB/s | 45 kB 00:00 2024-09-10T07:54:58.508 INFO:teuthology.orchestra.run.smithi002.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.0 MB/s | 548 kB 00:00 2024-09-10T07:54:58.549 INFO:teuthology.orchestra.run.smithi002.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 473 kB/s | 19 kB 00:00 2024-09-10T07:54:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:54:58.592 INFO:teuthology.orchestra.run.smithi002.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-09-10T07:54:58.617 INFO:teuthology.orchestra.run.smithi002.stdout:(108/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.8 MB/s | 309 kB 00:00 2024-09-10T07:54:58.659 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:54:58.675 INFO:teuthology.orchestra.run.smithi002.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.0 MB/s | 49 kB 00:00 2024-09-10T07:54:58.717 INFO:teuthology.orchestra.run.smithi002.stdout:(110/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 208 kB/s | 26 kB 00:00 2024-09-10T07:54:58.742 INFO:teuthology.orchestra.run.smithi002.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-10T07:54:58.776 INFO:teuthology.orchestra.run.smithi002.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-10T07:54:58.818 INFO:teuthology.orchestra.run.smithi002.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-10T07:54:58.882 INFO:teuthology.orchestra.run.smithi002.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 688 kB/s | 43 kB 00:00 2024-09-10T07:54:58.927 INFO:teuthology.orchestra.run.smithi002.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-09-10T07:54:58.969 INFO:teuthology.orchestra.run.smithi002.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 370 kB/s | 32 kB 00:00 2024-09-10T07:54:59.011 INFO:teuthology.orchestra.run.smithi002.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 169 kB/s | 14 kB 00:00 2024-09-10T07:54:59.044 INFO:teuthology.orchestra.run.smithi002.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-09-10T07:54:59.078 INFO:teuthology.orchestra.run.smithi002.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-10T07:54:59.104 INFO:teuthology.orchestra.run.smithi002.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-10T07:54:59.137 INFO:teuthology.orchestra.run.smithi002.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-10T07:54:59.171 INFO:teuthology.orchestra.run.smithi002.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T07:54:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:54:59.221 INFO:teuthology.orchestra.run.smithi002.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 215 kB/s | 11 kB 00:00 2024-09-10T07:54:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:54:59.288 INFO:teuthology.orchestra.run.smithi002.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-09-10T07:54:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:54:59.336 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T07:54:59.336 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T07:54:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:54:59.339 INFO:teuthology.orchestra.run.smithi002.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 465 kB/s | 23 kB 00:00 2024-09-10T07:54:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:54:59.389 INFO:teuthology.orchestra.run.smithi002.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 392 kB/s | 19 kB 00:00 2024-09-10T07:54:59.439 INFO:teuthology.orchestra.run.smithi002.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-09-10T07:54:59.506 INFO:teuthology.orchestra.run.smithi002.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-09-10T07:54:59.548 INFO:teuthology.orchestra.run.smithi002.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-09-10T07:54:59.573 INFO:teuthology.orchestra.run.smithi002.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T07:54:59.615 INFO:teuthology.orchestra.run.smithi002.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-09-10T07:54:59.682 INFO:teuthology.orchestra.run.smithi002.stdout:(132/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-09-10T07:54:59.724 INFO:teuthology.orchestra.run.smithi002.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 395 kB/s | 16 kB 00:00 2024-09-10T07:54:59.749 INFO:teuthology.orchestra.run.smithi002.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.6 MB/s | 90 kB 00:00 2024-09-10T07:54:59.774 INFO:teuthology.orchestra.run.smithi002.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-10T07:54:59.816 INFO:teuthology.orchestra.run.smithi002.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 424 kB/s | 28 kB 00:00 2024-09-10T07:54:59.868 INFO:teuthology.orchestra.run.smithi002.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-09-10T07:54:59.909 INFO:teuthology.orchestra.run.smithi002.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-10T07:54:59.951 INFO:teuthology.orchestra.run.smithi002.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 861 kB/s | 36 kB 00:00 2024-09-10T07:54:59.976 INFO:teuthology.orchestra.run.smithi002.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-10T07:55:00.002 INFO:teuthology.orchestra.run.smithi002.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-10T07:55:00.069 INFO:teuthology.orchestra.run.smithi002.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-10T07:55:00.144 INFO:teuthology.orchestra.run.smithi002.stdout:(143/156): libarrow-9.0.0-11.el9.x86_64.rpm 2.8 MB/s | 4.4 MB 00:01 2024-09-10T07:55:00.169 INFO:teuthology.orchestra.run.smithi002.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-09-10T07:55:00.211 INFO:teuthology.orchestra.run.smithi002.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 484 kB/s | 20 kB 00:00 2024-09-10T07:55:00.237 INFO:teuthology.orchestra.run.smithi002.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-10T07:55:00.270 INFO:teuthology.orchestra.run.smithi002.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.1 MB/s | 427 kB 00:00 2024-09-10T07:55:00.278 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:55:00.310 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:55:00.320 INFO:teuthology.orchestra.run.smithi002.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-10T07:55:00.512 INFO:teuthology.orchestra.run.smithi002.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-09-10T07:55:00.571 INFO:teuthology.orchestra.run.smithi002.stdout:(150/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-10T07:55:00.655 INFO:teuthology.orchestra.run.smithi002.stdout:(151/156): librados2-19.1.1-305.g1cf819b7.el9.x 8.2 MB/s | 3.4 MB 00:00 2024-09-10T07:55:00.697 INFO:teuthology.orchestra.run.smithi002.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 326 kB/s | 41 kB 00:00 2024-09-10T07:55:00.739 INFO:teuthology.orchestra.run.smithi002.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 8.0 MB/s | 1.8 MB 00:00 2024-09-10T07:55:00.840 INFO:teuthology.orchestra.run.smithi002.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-09-10T07:55:00.858 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:55:00.864 INFO:teuthology.orchestra.run.smithi002.stdout:(155/156): NetworkManager-tui-1.49.90-1.el9.x86 1.2 MB/s | 249 kB 00:00 2024-09-10T07:55:00.906 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:55:00.907 INFO:teuthology.orchestra.run.smithi002.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 6.5 MB/s | 1.4 MB 00:00 2024-09-10T07:55:00.913 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-10T07:55:00.913 INFO:teuthology.orchestra.run.smithi002.stdout:Total 10 MB/s | 226 MB 00:21 2024-09-10T07:55:00.997 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:55:00.997 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T07:55:00.997 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T07:55:00.998 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:55:01.024 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:55:03.063 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-10T07:55:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-10T07:55:03.263 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-10T07:55:05.808 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-10T07:55:05.808 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-10T07:55:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:55:11.095 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-09-10T07:55:11.095 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-09-10T07:55:11.095 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-09-10T07:55:11.096 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-09-10T07:55:11.096 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-09-10T07:55:11.096 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-09-10T07:55:11.096 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-09-10T07:55:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:55:11.277 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:55:11.293 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:55:11.294 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:11.294 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T07:55:11.294 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:55:12.700 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:55:12.701 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2024-09-10T07:55:12.701 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2024-09-10T07:55:12.701 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2024-09-10T07:55:12.701 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2024-09-10T07:55:12.701 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2024-09-10T07:55:12.701 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /root 2024-09-10T07:55:12.701 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2024-09-10T07:55:12.701 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:55:12.772 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:55:12.798 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:55:12.799 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:12.799 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T07:55:12.799 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:55:12.799 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:55:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:55:12.869 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-10T07:55:12.871 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:55:12.888 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:55:12.888 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:12.888 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T07:55:12.888 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:55:13.011 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T07:55:13.022 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:55:13.046 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:55:13.046 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:13.046 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T07:55:13.046 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:55:13.046 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:55:13.046 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:55:13.068 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T07:55:13.145 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T07:55:13.189 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T07:55:13.249 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T07:55:13.311 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T07:55:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:55:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:55:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T07:55:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:55:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:55:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:55:13.611 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T07:55:13.668 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T07:55:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T07:55:13.723 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T07:55:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:55:13.907 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T07:55:13.943 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:55:13.966 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T07:55:13.980 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:55:14.042 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T07:55:14.110 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:55:14.131 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T07:55:14.146 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:55:14.264 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T07:55:14.316 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T07:55:14.348 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T07:55:14.376 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T07:55:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T07:55:14.399 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:55:14.426 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:55:14.426 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:14.426 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T07:55:14.426 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:55:14.426 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:55:14.426 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:55:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T07:55:14.484 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:55:14.612 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:55:14.643 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:55:14.658 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:55:14.682 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:55:14.682 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:14.682 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T07:55:14.682 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:55:14.682 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:55:14.682 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:55:14.718 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T07:55:14.744 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:55:14.779 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:55:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T07:55:14.808 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:55:14.839 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T07:55:14.893 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T07:55:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:55:15.118 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:55:15.131 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:55:15.141 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:55:15.141 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:15.141 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T07:55:15.141 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:55:15.141 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:55:15.142 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:55:15.247 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T07:55:15.304 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T07:55:15.354 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T07:55:15.360 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T07:55:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T07:55:15.546 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T07:55:15.593 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T07:55:15.652 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:55:15.663 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T07:55:15.744 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:55:15.751 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T07:55:15.885 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T07:55:15.936 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T07:55:15.968 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T07:55:15.974 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:55:15.996 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T07:55:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T07:55:16.024 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:55:16.166 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T07:55:16.291 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T07:55:16.357 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T07:55:16.405 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T07:55:16.436 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:55:16.461 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:55:16.488 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T07:55:16.492 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T07:55:16.541 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T07:55:16.555 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T07:55:16.578 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T07:55:16.584 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:55:16.634 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T07:55:16.672 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T07:55:16.705 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T07:55:16.822 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T07:55:16.884 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T07:55:16.917 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T07:55:16.964 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T07:55:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:55:17.016 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:55:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T07:55:17.088 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T07:55:17.118 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:55:17.157 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T07:55:17.184 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T07:55:17.277 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T07:55:17.280 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T07:55:17.310 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:55:17.334 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:55:17.335 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:17.335 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T07:55:17.335 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:55:17.335 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:55:17.335 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:55:17.338 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T07:55:17.386 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:55:17.398 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:55:17.431 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T07:55:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T07:55:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T07:55:17.608 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T07:55:17.628 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:55:17.661 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:55:17.683 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T07:55:18.363 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T07:55:18.465 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T07:55:18.478 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T07:55:18.497 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:55:18.518 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T07:55:18.521 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:55:18.521 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:18.521 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T07:55:18.521 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:55:18.521 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:55:18.521 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:55:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T07:55:18.564 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:55:18.587 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:55:18.587 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:18.587 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T07:55:18.587 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:55:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T07:55:18.624 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:55:18.632 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:55:18.653 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:55:18.655 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:55:18.655 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:18.655 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T07:55:18.656 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:55:18.656 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:55:18.656 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:55:18.676 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T07:55:18.757 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:55:18.881 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T07:55:18.924 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T07:55:18.937 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T07:55:18.955 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:55:18.966 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:55:18.980 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:55:18.980 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:18.980 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T07:55:18.980 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:55:18.980 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:55:18.980 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:55:18.989 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:55:18.989 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:18.989 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T07:55:18.989 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:55:18.989 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:55:18.990 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:55:19.007 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T07:55:19.031 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:55:19.043 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:55:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T07:55:19.141 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T07:55:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T07:55:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T07:55:19.197 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T07:55:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T07:55:19.281 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T07:55:19.328 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T07:55:19.378 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T07:55:19.578 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T07:55:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T07:55:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T07:55:19.804 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T07:55:19.856 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T07:55:19.897 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T07:55:19.927 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T07:55:19.943 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T07:55:20.066 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T07:55:20.128 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T07:55:20.167 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:55:20.191 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:55:20.191 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:20.191 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T07:55:20.191 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:55:20.191 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:55:20.192 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:55:20.235 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:55:20.236 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T07:55:20.258 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:55:20.258 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:20.258 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T07:55:20.258 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:55:20.284 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T07:55:20.310 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:55:20.321 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T07:55:20.334 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:55:20.334 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:20.334 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T07:55:20.334 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:55:20.334 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:55:20.334 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:55:20.358 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T07:55:20.405 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T07:55:20.444 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T07:55:20.473 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T07:55:20.508 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T07:55:20.541 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T07:55:20.609 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T07:55:20.636 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:55:20.650 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T07:55:20.660 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:55:20.660 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:20.660 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T07:55:20.660 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:55:20.660 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:55:20.660 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:55:20.753 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T07:55:20.810 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T07:55:20.845 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T07:55:20.972 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T07:55:21.043 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T07:55:21.997 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T07:55:22.069 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T07:55:22.102 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T07:55:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T07:55:22.319 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T07:55:22.374 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T07:55:22.480 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T07:55:22.538 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T07:55:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T07:55:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T07:55:22.917 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T07:55:23.536 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T07:55:23.578 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T07:55:23.619 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T07:55:23.651 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T07:55:23.678 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T07:55:23.782 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T07:55:23.857 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T07:55:23.900 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T07:55:23.930 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T07:55:23.935 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T07:55:23.960 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T07:55:23.998 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T07:55:24.618 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:55:24.663 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:55:24.683 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:55:25.207 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T07:55:25.248 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T07:55:25.289 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T07:55:25.321 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T07:55:25.349 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T07:55:25.530 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T07:55:25.552 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T07:55:25.579 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T07:55:25.594 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T07:55:25.609 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T07:55:25.643 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T07:55:25.647 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T07:55:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T07:55:25.689 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T07:55:25.694 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T07:55:25.725 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T07:55:25.758 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T07:55:25.765 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T07:55:25.796 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T07:55:25.822 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T07:55:25.848 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T07:55:25.848 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:55:25.940 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:55:25.941 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:55:26.033 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:55:26.058 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T07:55:26.063 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:55:26.063 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:55:26.115 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T07:55:26.161 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:55:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:55:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:55:26.274 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T07:55:26.309 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T07:55:26.319 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:55:26.327 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:55:26.367 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:55:26.380 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T07:55:26.383 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:55:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:55:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T07:55:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:55:27.064 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T07:55:27.090 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T07:55:27.245 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T07:55:27.339 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T07:55:27.380 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T07:55:27.463 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T07:55:27.492 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T07:55:27.517 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T07:55:27.552 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T07:55:27.552 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:55:27.644 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:55:27.645 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:55:27.737 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:55:27.762 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T07:55:27.767 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:55:27.768 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:55:27.947 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:55:27.948 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:55:28.031 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:55:28.094 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:55:29.780 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:55:29.844 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:55:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T07:55:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:55:30.859 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T07:55:30.859 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T07:55:30.859 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T07:55:30.859 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:55:30.908 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:55:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:55:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T07:55:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T07:55:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T07:55:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T07:55:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T07:55:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T07:55:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T07:55:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:55:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:55:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:55:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T07:55:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T07:55:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:55:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T07:55:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T07:55:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T07:55:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T07:55:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T07:55:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T07:55:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T07:55:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T07:55:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T07:55:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T07:55:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T07:55:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T07:55:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T07:55:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T07:55:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T07:55:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T07:55:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T07:55:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T07:55:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T07:55:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T07:55:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T07:55:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T07:55:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T07:55:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T07:55:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T07:55:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T07:55:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T07:55:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T07:55:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T07:55:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T07:55:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T07:55:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T07:55:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T07:55:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T07:55:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T07:55:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T07:55:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T07:55:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T07:55:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T07:55:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T07:55:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T07:55:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T07:55:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T07:55:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T07:55:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T07:55:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T07:55:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T07:55:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T07:55:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T07:55:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T07:55:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T07:55:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T07:55:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T07:55:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T07:55:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T07:55:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T07:55:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T07:55:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T07:55:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T07:55:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T07:55:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T07:55:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T07:55:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T07:55:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T07:55:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T07:55:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T07:55:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T07:55:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T07:55:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T07:55:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T07:55:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T07:55:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T07:55:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T07:55:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T07:55:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T07:55:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T07:55:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T07:55:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T07:55:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T07:55:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T07:55:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T07:55:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T07:55:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T07:55:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T07:55:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T07:55:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T07:55:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T07:55:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T07:55:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T07:55:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T07:55:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T07:55:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T07:55:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T07:55:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T07:55:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T07:55:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T07:55:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T07:55:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T07:55:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T07:55:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T07:55:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T07:55:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T07:55:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T07:55:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T07:55:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T07:55:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T07:55:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T07:55:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T07:55:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T07:55:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T07:55:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T07:55:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T07:55:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T07:55:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T07:55:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T07:55:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T07:55:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T07:55:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T07:55:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T07:55:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T07:55:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T07:55:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T07:55:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T07:55:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T07:55:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T07:55:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T07:55:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T07:55:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T07:55:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T07:55:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T07:55:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T07:55:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T07:55:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T07:55:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T07:55:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T07:55:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T07:55:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T07:55:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T07:55:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T07:55:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T07:55:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T07:55:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T07:55:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T07:55:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T07:55:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T07:55:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T07:55:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T07:55:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T07:55:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T07:55:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T07:55:33.732 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:55:33.732 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T07:55:33.732 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T07:55:33.732 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T07:55:33.733 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T07:55:33.733 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T07:55:33.733 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T07:55:33.733 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T07:55:33.733 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:55:33.733 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:55:33.733 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:55:33.733 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T07:55:33.733 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T07:55:33.733 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:55:33.733 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T07:55:33.733 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T07:55:33.733 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T07:55:33.734 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T07:55:33.734 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T07:55:33.734 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T07:55:33.734 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T07:55:33.734 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T07:55:33.734 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T07:55:33.734 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T07:55:33.734 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T07:55:33.734 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T07:55:33.734 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T07:55:33.734 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T07:55:33.734 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T07:55:33.746 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T07:55:33.746 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T07:55:33.746 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T07:55:33.746 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T07:55:33.746 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T07:55:33.747 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T07:55:33.747 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T07:55:33.747 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T07:55:33.747 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T07:55:33.747 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T07:55:33.747 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T07:55:33.747 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T07:55:33.747 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T07:55:33.747 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T07:55:33.747 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T07:55:33.747 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T07:55:33.747 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T07:55:33.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T07:55:33.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T07:55:33.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T07:55:33.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T07:55:33.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T07:55:33.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T07:55:33.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T07:55:33.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T07:55:33.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T07:55:33.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T07:55:33.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T07:55:33.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T07:55:33.748 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T07:55:33.749 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T07:55:33.749 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T07:55:33.749 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T07:55:33.749 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T07:55:33.749 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T07:55:33.749 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T07:55:33.749 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T07:55:33.749 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T07:55:33.749 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T07:55:33.749 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T07:55:33.749 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T07:55:33.749 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T07:55:33.749 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T07:55:33.750 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T07:55:33.750 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T07:55:33.750 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T07:55:33.750 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T07:55:33.750 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T07:55:33.750 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T07:55:33.750 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T07:55:33.750 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T07:55:33.750 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T07:55:33.750 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T07:55:33.750 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T07:55:33.750 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T07:55:33.750 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T07:55:33.751 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T07:55:33.752 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T07:55:33.752 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T07:55:33.752 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T07:55:33.752 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T07:55:33.752 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T07:55:33.752 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T07:55:33.752 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T07:55:33.752 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T07:55:33.752 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T07:55:33.752 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T07:55:33.752 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T07:55:33.752 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T07:55:33.752 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T07:55:33.753 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T07:55:33.753 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T07:55:33.753 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T07:55:33.753 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T07:55:33.753 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T07:55:33.753 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T07:55:33.753 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T07:55:33.753 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T07:55:33.753 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T07:55:33.753 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T07:55:33.753 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T07:55:33.753 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T07:55:33.754 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T07:55:33.755 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T07:55:33.755 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T07:55:33.755 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T07:55:33.755 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T07:55:33.755 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T07:55:33.755 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T07:55:33.755 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T07:55:33.755 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T07:55:33.755 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T07:55:33.755 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T07:55:33.755 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T07:55:33.755 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T07:55:33.755 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T07:55:33.756 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T07:55:33.756 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T07:55:33.756 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T07:55:33.756 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T07:55:33.756 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T07:55:33.756 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T07:55:33.756 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T07:55:33.756 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T07:55:33.756 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T07:55:33.756 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T07:55:33.756 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T07:55:33.756 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T07:55:33.756 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T07:55:34.820 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:55:34.820 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:55:34.821 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-09-10T07:55:34.821 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T07:55:34.821 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T07:55:34.821 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T07:55:34.821 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T07:55:34.821 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.821 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.821 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T07:55:34.821 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T07:55:34.821 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T07:55:34.821 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T07:55:34.821 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T07:55:34.822 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T07:55:34.822 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T07:55:34.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:34.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:34.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:34.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:34.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:34.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:34.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:34.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:34.823 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:34.823 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.823 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T07:55:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T07:55:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T07:55:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T07:55:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T07:55:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T07:55:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T07:55:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T07:55:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T07:55:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T07:55:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T07:55:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T07:55:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T07:55:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T07:55:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T07:55:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T07:55:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T07:55:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T07:55:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T07:55:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T07:55:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T07:55:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T07:55:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T07:55:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T07:55:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T07:55:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T07:55:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T07:55:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T07:55:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T07:55:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T07:55:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T07:55:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T07:55:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T07:55:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T07:55:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T07:55:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T07:55:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T07:55:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T07:55:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T07:55:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T07:55:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T07:55:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T07:55:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T07:55:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T07:55:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T07:55:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T07:55:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T07:55:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T07:55:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T07:55:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T07:55:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T07:55:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T07:55:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T07:55:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T07:55:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T07:55:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T07:55:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T07:55:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T07:55:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T07:55:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T07:55:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T07:55:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T07:55:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T07:55:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T07:55:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T07:55:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T07:55:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T07:55:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T07:55:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T07:55:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T07:55:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T07:55:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T07:55:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T07:55:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T07:55:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T07:55:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T07:55:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T07:55:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T07:55:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T07:55:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T07:55:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T07:55:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T07:55:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:55:34.834 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T07:55:35.241 DEBUG:teuthology.parallel:result is None 2024-09-10T07:55:36.659 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:55:36.660 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:55:36.660 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2024-09-10T07:55:36.660 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T07:55:36.660 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T07:55:36.660 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T07:55:36.660 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T07:55:36.660 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.660 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.660 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T07:55:36.660 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T07:55:36.660 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-09-10T07:55:36.661 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T07:55:36.661 INFO:teuthology.orchestra.run.smithi155.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T07:55:36.661 INFO:teuthology.orchestra.run.smithi155.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T07:55:36.661 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T07:55:36.661 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.661 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.661 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.661 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.661 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:36.661 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.661 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.661 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:36.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:36.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:36.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:36.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:36.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:36.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:36.663 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:36.663 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.663 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.663 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:55:36.663 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T07:55:36.663 INFO:teuthology.orchestra.run.smithi155.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T07:55:36.663 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T07:55:36.663 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T07:55:36.663 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T07:55:36.663 INFO:teuthology.orchestra.run.smithi155.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T07:55:36.663 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T07:55:36.663 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T07:55:36.663 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T07:55:36.664 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T07:55:36.664 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T07:55:36.664 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.664 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.664 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.664 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T07:55:36.664 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T07:55:36.664 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T07:55:36.664 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T07:55:36.664 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T07:55:36.664 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.664 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.664 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T07:55:36.665 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.665 INFO:teuthology.orchestra.run.smithi155.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T07:55:36.665 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T07:55:36.665 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T07:55:36.665 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T07:55:36.666 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T07:55:36.666 INFO:teuthology.orchestra.run.smithi155.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T07:55:36.666 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T07:55:36.666 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T07:55:36.667 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T07:55:36.667 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T07:55:36.667 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T07:55:36.667 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T07:55:36.667 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T07:55:36.667 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T07:55:36.668 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T07:55:36.668 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T07:55:36.668 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T07:55:36.668 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T07:55:36.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T07:55:36.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T07:55:36.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T07:55:36.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T07:55:36.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T07:55:36.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.669 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.669 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T07:55:36.669 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T07:55:36.669 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T07:55:36.669 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T07:55:36.669 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T07:55:36.669 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T07:55:36.669 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T07:55:36.669 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T07:55:36.669 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T07:55:36.669 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T07:55:36.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T07:55:36.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T07:55:36.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T07:55:36.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T07:55:36.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T07:55:36.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T07:55:36.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T07:55:36.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T07:55:36.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T07:55:36.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T07:55:36.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T07:55:36.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T07:55:36.670 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T07:55:36.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T07:55:36.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T07:55:36.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T07:55:36.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T07:55:36.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T07:55:36.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T07:55:36.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T07:55:36.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T07:55:36.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T07:55:36.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T07:55:36.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T07:55:36.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T07:55:36.671 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T07:55:36.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T07:55:36.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T07:55:36.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T07:55:36.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T07:55:36.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T07:55:36.672 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T07:55:36.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T07:55:36.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T07:55:36.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T07:55:36.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T07:55:36.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T07:55:36.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T07:55:36.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T07:55:36.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T07:55:36.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T07:55:36.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T07:55:36.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T07:55:36.673 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T07:55:36.674 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T07:55:36.674 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T07:55:36.674 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T07:55:36.674 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T07:55:36.674 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.674 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.674 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:55:36.674 INFO:teuthology.orchestra.run.smithi155.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T07:55:36.674 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T07:55:36.674 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T07:55:36.674 INFO:teuthology.orchestra.run.smithi155.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T07:55:36.674 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T07:55:36.675 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T07:55:36.675 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T07:55:36.676 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:55:36.676 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-09-10T07:55:37.103 DEBUG:teuthology.parallel:result is None 2024-09-10T07:55:44.268 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T07:55:44.268 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /sys 2024-09-10T07:55:44.268 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /proc 2024-09-10T07:55:44.268 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /mnt 2024-09-10T07:55:44.269 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /var/tmp 2024-09-10T07:55:44.269 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /home 2024-09-10T07:55:44.269 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /root 2024-09-10T07:55:44.269 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /tmp 2024-09-10T07:55:44.269 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:55:44.500 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:55:44.536 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T07:55:44.536 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:44.536 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T07:55:44.536 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:55:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:55:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T07:55:46.243 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:46.243 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T07:55:46.243 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:55:46.243 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T07:55:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:55:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:55:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T07:55:46.553 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:46.553 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T07:55:46.553 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:55:46.553 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T07:55:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:55:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:55:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T07:55:47.120 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:47.120 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T07:55:47.120 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:55:47.121 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T07:55:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:55:47.252 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T07:55:47.314 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T07:55:47.387 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T07:55:47.642 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T07:55:47.698 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T07:55:47.788 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T07:55:47.907 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T07:55:48.086 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T07:55:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T07:55:48.204 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T07:55:48.229 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T07:55:48.938 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:55:48.978 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T07:55:49.020 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T07:55:49.124 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T07:55:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:55:50.457 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T07:55:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:55:51.673 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T07:55:51.702 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:55:51.848 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T07:55:52.032 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T07:55:52.071 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:55:52.123 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T07:55:52.123 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:52.123 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T07:55:52.123 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:55:52.123 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T07:55:52.123 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:55:52.180 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:55:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T07:55:52.326 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T07:55:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T07:55:53.481 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T07:55:53.516 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:55:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T07:55:53.568 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:53.568 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T07:55:53.568 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:55:53.568 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T07:55:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:55:53.617 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:55:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T07:55:53.667 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:53.667 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T07:55:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:55:53.726 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:55:53.777 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T07:55:53.777 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:53.778 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T07:55:53.778 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:55:53.778 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T07:55:53.778 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:55:54.128 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:55:54.177 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T07:55:54.178 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T07:55:54.178 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T07:55:54.178 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:55:54.178 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T07:55:54.178 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:55:59.457 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T07:55:59.508 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T07:55:59.560 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T07:55:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T07:55:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T07:55:59.882 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T07:55:59.929 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T07:55:59.958 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T07:55:59.990 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T07:56:00.033 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T07:56:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:56:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:56:01.090 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T07:56:02.465 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T07:56:02.587 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T07:56:02.648 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T07:56:02.755 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T07:56:02.805 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T07:56:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T07:56:02.893 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T07:56:02.893 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:56:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T07:56:02.992 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:56:03.093 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T07:56:03.117 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T07:56:03.133 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:56:03.133 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:56:03.394 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T07:56:03.394 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:56:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T07:56:03.564 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:56:03.815 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:56:03.815 INFO:teuthology.orchestra.run.smithi002.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T07:56:03.815 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:56:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T07:56:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T07:56:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T07:56:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T07:56:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T07:56:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T07:56:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T07:56:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T07:56:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T07:56:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T07:56:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T07:56:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T07:56:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T07:56:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T07:56:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T07:56:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T07:56:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T07:56:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T07:56:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T07:56:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T07:56:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T07:56:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T07:56:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T07:56:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T07:56:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T07:56:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T07:56:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T07:56:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T07:56:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T07:56:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T07:56:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T07:56:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T07:56:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T07:56:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T07:56:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T07:56:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T07:56:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T07:56:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T07:56:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T07:56:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T07:56:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T07:56:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T07:56:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T07:56:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T07:56:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T07:56:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T07:56:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T07:56:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T07:56:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T07:56:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T07:56:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T07:56:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T07:56:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T07:56:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T07:56:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T07:56:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T07:56:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T07:56:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T07:56:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T07:56:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T07:56:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T07:56:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T07:56:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T07:56:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T07:56:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T07:56:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T07:56:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T07:56:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T07:56:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T07:56:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T07:56:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T07:56:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T07:56:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T07:56:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T07:56:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T07:56:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T07:56:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T07:56:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T07:56:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T07:56:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T07:56:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T07:56:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T07:56:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T07:56:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T07:56:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T07:56:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T07:56:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T07:56:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T07:56:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T07:56:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T07:56:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T07:56:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T07:56:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T07:56:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T07:56:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T07:56:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T07:56:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T07:56:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T07:56:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T07:56:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T07:56:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T07:56:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T07:56:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T07:56:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T07:56:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T07:56:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T07:56:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T07:56:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T07:56:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T07:56:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T07:56:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T07:56:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T07:56:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T07:56:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T07:56:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T07:56:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T07:56:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T07:56:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T07:56:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T07:56:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T07:56:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T07:56:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T07:56:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T07:56:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T07:56:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T07:56:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T07:56:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T07:56:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T07:56:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T07:56:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T07:56:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T07:56:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T07:56:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T07:56:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T07:56:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T07:56:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T07:56:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T07:56:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T07:56:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T07:56:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T07:56:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T07:56:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T07:56:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T07:56:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T07:56:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T07:56:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T07:56:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T07:56:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T07:56:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T07:56:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T07:56:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T07:56:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T07:56:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T07:56:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T07:56:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T07:56:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T07:56:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T07:56:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T07:56:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T07:56:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T07:56:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T07:56:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T07:56:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T07:56:11.735 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T07:56:11.735 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:56:11.735 INFO:teuthology.orchestra.run.smithi002.stdout:Upgraded: 2024-09-10T07:56:11.735 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T07:56:11.736 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T07:56:11.736 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T07:56:11.736 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T07:56:11.736 INFO:teuthology.orchestra.run.smithi002.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.736 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.736 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T07:56:11.736 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T07:56:11.736 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-10T07:56:11.736 INFO:teuthology.orchestra.run.smithi002.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T07:56:11.736 INFO:teuthology.orchestra.run.smithi002.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T07:56:11.736 INFO:teuthology.orchestra.run.smithi002.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T07:56:11.737 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T07:56:11.737 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.737 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.737 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.737 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.737 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:56:11.737 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.737 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.737 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.737 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:56:11.737 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:56:11.738 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:56:11.738 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:56:11.738 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:56:11.738 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.738 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.738 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:56:11.738 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.738 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.738 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.738 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:56:11.738 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:56:11.739 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.739 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.739 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T07:56:11.739 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T07:56:11.739 INFO:teuthology.orchestra.run.smithi002.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T07:56:11.739 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T07:56:11.739 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T07:56:11.739 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T07:56:11.739 INFO:teuthology.orchestra.run.smithi002.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T07:56:11.739 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T07:56:11.739 INFO:teuthology.orchestra.run.smithi002.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T07:56:11.739 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T07:56:11.739 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T07:56:11.740 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T07:56:11.740 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.740 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.740 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.740 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T07:56:11.740 INFO:teuthology.orchestra.run.smithi002.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T07:56:11.740 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T07:56:11.740 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T07:56:11.740 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T07:56:11.740 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.740 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.741 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T07:56:11.741 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.741 INFO:teuthology.orchestra.run.smithi002.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T07:56:11.741 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T07:56:11.741 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T07:56:11.741 INFO:teuthology.orchestra.run.smithi002.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T07:56:11.741 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T07:56:11.741 INFO:teuthology.orchestra.run.smithi002.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T07:56:11.741 INFO:teuthology.orchestra.run.smithi002.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T07:56:11.741 INFO:teuthology.orchestra.run.smithi002.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T07:56:11.741 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T07:56:11.742 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T07:56:11.742 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T07:56:11.742 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T07:56:11.742 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T07:56:11.742 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T07:56:11.742 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T07:56:11.742 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T07:56:11.742 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T07:56:11.742 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T07:56:11.742 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T07:56:11.742 INFO:teuthology.orchestra.run.smithi002.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T07:56:11.743 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T07:56:11.743 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T07:56:11.743 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T07:56:11.743 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.743 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.743 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.743 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T07:56:11.743 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T07:56:11.743 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T07:56:11.743 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T07:56:11.743 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T07:56:11.743 INFO:teuthology.orchestra.run.smithi002.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T07:56:11.744 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T07:56:11.744 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T07:56:11.744 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T07:56:11.744 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T07:56:11.744 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T07:56:11.744 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T07:56:11.744 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T07:56:11.744 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T07:56:11.744 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T07:56:11.744 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T07:56:11.744 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T07:56:11.745 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T07:56:11.745 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T07:56:11.745 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T07:56:11.745 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T07:56:11.745 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T07:56:11.745 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T07:56:11.745 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T07:56:11.745 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T07:56:11.745 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T07:56:11.745 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T07:56:11.745 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T07:56:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T07:56:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T07:56:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T07:56:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T07:56:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T07:56:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T07:56:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T07:56:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T07:56:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T07:56:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T07:56:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T07:56:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T07:56:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T07:56:11.747 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T07:56:11.748 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T07:56:11.749 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T07:56:11.749 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T07:56:11.749 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:56:11.749 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-10T07:56:12.495 DEBUG:teuthology.parallel:result is None 2024-09-10T07:56:12.496 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:56:12.644 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T07:56:12.692 INFO:teuthology.orchestra.run.smithi002.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T07:56:12.693 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T07:56:12.693 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T07:56:12.695 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:56:12.840 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T07:56:12.867 INFO:teuthology.orchestra.run.smithi107.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T07:56:12.867 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T07:56:12.867 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T07:56:12.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:56:13.013 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T07:56:13.040 INFO:teuthology.orchestra.run.smithi155.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T07:56:13.041 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T07:56:13.041 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T07:56:13.043 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T07:56:13.043 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:56:13.043 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T07:56:13.107 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T07:56:13.107 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T07:56:13.139 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-10T07:56:13.139 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T07:56:13.172 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T07:56:13.172 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:56:13.172 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T07:56:13.228 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T07:56:13.329 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T07:56:13.329 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T07:56:13.359 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T07:56:13.431 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-10T07:56:13.431 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T07:56:13.463 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T07:56:13.531 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T07:56:13.531 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:56:13.531 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T07:56:13.590 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T07:56:13.685 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T07:56:13.686 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T07:56:13.710 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T07:56:13.775 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-10T07:56:13.775 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T07:56:13.800 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T07:56:13.866 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T07:56:13.866 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:56:13.866 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T07:56:13.921 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T07:56:14.018 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T07:56:14.018 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T07:56:14.043 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T07:56:14.109 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-10T07:56:14.109 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T07:56:14.134 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T07:56:14.199 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T07:56:14.305 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-10T07:56:14.305 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:56:14.306 INFO:tasks.cephadm:Cluster fsid is 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:56:14.306 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T07:56:14.306 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.2', 'mon.b': '172.21.15.107', 'mon.c': '172.21.15.155'} 2024-09-10T07:56:14.307 INFO:tasks.cephadm:First mon is mon.a on smithi002 2024-09-10T07:56:14.307 INFO:tasks.cephadm:First mgr is x 2024-09-10T07:56:14.307 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T07:56:14.307 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname $(hostname -s) 2024-09-10T07:56:14.365 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-09-10T07:56:14.390 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname $(hostname -s) 2024-09-10T07:56:14.415 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T07:56:14.416 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:56:14.559 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82914/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:14:38.445020', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82912/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:05:03.996701', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-10T07:56:14.706 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240909.165402-squid, sha1 1cf819b7212a6048e92fc78c4eff963344f6d56b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:56:14.707 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T07:56:14.707 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T07:56:14.707 DEBUG:teuthology.orchestra.run.smithi002:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T07:56:15.132 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 07:56 /home/ubuntu/cephtest/cephadm 2024-09-10T07:56:15.132 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T07:56:15.515 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 07:56 /home/ubuntu/cephtest/cephadm 2024-09-10T07:56:15.516 DEBUG:teuthology.orchestra.run.smithi155:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T07:56:15.900 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 07:56 /home/ubuntu/cephtest/cephadm 2024-09-10T07:56:15.901 DEBUG:teuthology.orchestra.run.smithi002:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T07:56:15.939 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T07:56:15.958 DEBUG:teuthology.orchestra.run.smithi155:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T07:56:15.993 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b on all hosts... 2024-09-10T07:56:15.993 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T07:56:15.997 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T07:56:16.003 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T07:56:17.493 INFO:teuthology.orchestra.run.smithi155.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T07:56:17.512 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T07:56:18.357 INFO:teuthology.orchestra.run.smithi002.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T07:56:48.947 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T07:56:48.947 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T07:56:48.947 INFO:teuthology.orchestra.run.smithi002.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T07:56:48.947 INFO:teuthology.orchestra.run.smithi002.stdout: "repo_digests": [ 2024-09-10T07:56:48.948 INFO:teuthology.orchestra.run.smithi002.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T07:56:48.948 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-10T07:56:48.948 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T07:56:49.752 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-10T07:56:49.753 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T07:56:49.753 INFO:teuthology.orchestra.run.smithi155.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T07:56:49.753 INFO:teuthology.orchestra.run.smithi155.stdout: "repo_digests": [ 2024-09-10T07:56:49.753 INFO:teuthology.orchestra.run.smithi155.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T07:56:49.753 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-10T07:56:49.753 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-10T07:56:49.802 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-10T07:56:49.803 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T07:56:49.803 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T07:56:49.803 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-09-10T07:56:49.803 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T07:56:49.803 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-10T07:56:49.803 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-10T07:56:49.827 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph 2024-09-10T07:56:49.893 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-09-10T07:56:49.920 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph 2024-09-10T07:56:49.948 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 777 /etc/ceph 2024-09-10T07:56:50.006 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-09-10T07:56:50.030 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /etc/ceph 2024-09-10T07:56:50.055 INFO:tasks.cephadm:Writing seed config... 2024-09-10T07:56:50.056 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-10T07:56:50.056 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-10T07:56:50.056 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-10T07:56:50.056 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-10T07:56:50.057 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-10T07:56:50.057 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-10T07:56:50.057 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-10T07:56:50.057 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-10T07:56:50.057 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-10T07:56:50.057 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-10T07:56:50.057 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-10T07:56:50.057 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-10T07:56:50.057 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-10T07:56:50.057 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-10T07:56:50.057 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-10T07:56:50.057 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-10T07:56:50.057 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T07:56:50.058 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-10T07:56:50.059 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-10T07:56:50.059 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T07:56:50.059 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-10T07:56:50.059 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T07:56:50.059 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-10T07:56:50.059 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-10T07:56:50.059 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T07:56:50.059 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T07:56:50.059 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-10T07:56:50.059 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T07:56:50.059 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T07:56:50.059 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T07:56:50.059 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T07:56:50.060 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T07:56:50.060 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-10T07:56:50.060 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T07:56:50.060 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-10T07:56:50.060 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:56:50.061 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T07:56:50.096 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 26ec64b0-6f4a-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-10T07:56:50.096 DEBUG:teuthology.orchestra.run.smithi002:mon.a> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T07:56:50.140 DEBUG:teuthology.orchestra.run.smithi002:mgr.x> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T07:56:50.185 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T07:56:50.186 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b -v bootstrap --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.2 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T07:56:50.450 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-10T07:56:50.450 INFO:teuthology.orchestra.run.smithi002.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b', '-v', 'bootstrap', '--fsid', '26ec64b0-6f4a-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.2', '--skip-admin-label'] 2024-09-10T07:56:50.450 INFO:teuthology.orchestra.run.smithi002.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T07:56:50.451 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying podman|docker is present... 2024-09-10T07:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 5.2.2 2024-09-10T07:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying lvm2 is present... 2024-09-10T07:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying time synchronization is in place... 2024-09-10T07:56:50.511 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T07:56:50.511 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T07:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T07:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-09-10T07:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2024-09-10T07:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2024-09-10T07:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2024-09-10T07:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout:Repeating the final host check... 2024-09-10T07:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 5.2.2 2024-09-10T07:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T07:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl is present 2024-09-10T07:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout:lvcreate is present 2024-09-10T07:56:50.606 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T07:56:50.606 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T07:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T07:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-09-10T07:56:50.632 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2024-09-10T07:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2024-09-10T07:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2024-09-10T07:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout:Host looks OK 2024-09-10T07:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout:Cluster fsid: 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout:Acquiring lock 140301302866944 on /run/cephadm/26ec64b0-6f4a-11ef-bcea-c7b262605968.lock 2024-09-10T07:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout:Lock 140301302866944 acquired on /run/cephadm/26ec64b0-6f4a-11ef-bcea-c7b262605968.lock 2024-09-10T07:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 3300 ... 2024-09-10T07:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 6789 ... 2024-09-10T07:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout:Base mon IP(s) is [172.21.15.2:3300, 172.21.15.2:6789], mon addrv is [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2024-09-10T07:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.2 metric 100 2024-09-10T07:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.2 metric 100 2024-09-10T07:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T07:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-10T07:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1794sec hoplimit 64 pref medium 2024-09-10T07:56:50.665 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T07:56:50.665 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T07:56:50.665 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T07:56:50.665 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-10T07:56:50.666 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1920/64 scope link 2024-09-10T07:56:50.666 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T07:56:50.666 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2024-09-10T07:56:50.666 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2024-09-10T07:56:50.666 INFO:teuthology.orchestra.run.smithi002.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T07:56:50.667 INFO:teuthology.orchestra.run.smithi002.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T07:56:50.668 INFO:teuthology.orchestra.run.smithi002.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T07:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T07:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T07:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T07:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:3965ca83761b6e221cd35b61f2ebac0e3b1f9d68a1ce39c1ada69153a1d73116 2024-09-10T07:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T07:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying config sha256:41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T07:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T07:56:52.285 INFO:teuthology.orchestra.run.smithi002.stdout:ceph: stdout ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T07:56:52.285 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph version: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T07:56:52.285 INFO:teuthology.orchestra.run.smithi002.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T07:57:02.963 INFO:teuthology.orchestra.run.smithi002.stdout:stat: stdout 167 167 2024-09-10T07:57:02.964 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial keys... 2024-09-10T07:57:04.066 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQDP+99morogFhAA23NdCWdmXnyKUpqiApb3jg== 2024-09-10T07:57:05.097 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQDQ+99mVkVfHBAAk8bdXprK0TMCI7fpKODTrg== 2024-09-10T07:57:06.268 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQDR+99m74mkJxAAN71aar2v4jv5OiteTHz9ig== 2024-09-10T07:57:06.269 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial monmap... 2024-09-10T07:57:07.359 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T07:57:07.359 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T07:57:07.359 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:57:07.359 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T07:57:07.359 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool for a [v2:172.21.15.2:3300,v1:172.21.15.2:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T07:57:07.359 INFO:teuthology.orchestra.run.smithi002.stdout:setting min_mon_release = quincy 2024-09-10T07:57:07.359 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: set fsid to 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:57:07.360 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T07:57:07.360 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:57:07.360 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mon... 2024-09-10T07:57:09.159 INFO:teuthology.orchestra.run.smithi002.stdout:create mon.a on 2024-09-10T07:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T07:57:09.702 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T07:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968.target. 2024-09-10T07:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968.target. 2024-09-10T07:57:10.312 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.a 2024-09-10T07:57:10.312 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T07:57:10.587 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968.target.wants/ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@.service. 2024-09-10T07:57:11.201 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:11 smithi002 podman[30188]: 2024-09-10 07:57:11.19616288 +0000 UTC m=+0.369880581 container init 4cf63af4d00c192b9d219b6496c001219032fb954f1fca5642e9521268087354 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T07:57:11.201 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:11 smithi002 podman[30188]: 2024-09-10 07:57:11.202817377 +0000 UTC m=+0.376535068 container start 4cf63af4d00c192b9d219b6496c001219032fb954f1fca5642e9521268087354 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T07:57:11.330 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T07:57:11.330 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-09-10T07:57:11.343 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T07:57:11.343 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-09-10T07:57:11.343 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-09-10T07:57:11.343 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T07:57:11.344 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon to start... 2024-09-10T07:57:11.344 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon... 2024-09-10T07:57:11.485 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:11 smithi002 ceph-mon[30206]: mkfs 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:57:11.485 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:11 smithi002 ceph-mon[30206]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T07:57:11.485 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:11 smithi002 bash[30188]: 4cf63af4d00c192b9d219b6496c001219032fb954f1fca5642e9521268087354 2024-09-10T07:57:11.485 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:11 smithi002 systemd[1]: Started Ceph mon.a for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T07:57:12.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:12 smithi002 ceph-mon[30206]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T07:57:12.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:12 smithi002 ceph-mon[30206]: monmap epoch 1 2024-09-10T07:57:12.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:12 smithi002 ceph-mon[30206]: fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:57:12.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:12 smithi002 ceph-mon[30206]: last_changed 2024-09-10T07:57:06.640971+0000 2024-09-10T07:57:12.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:12 smithi002 ceph-mon[30206]: created 2024-09-10T07:57:06.640971+0000 2024-09-10T07:57:12.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:12 smithi002 ceph-mon[30206]: min_mon_release 19 (squid) 2024-09-10T07:57:12.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:12 smithi002 ceph-mon[30206]: election_strategy: 1 2024-09-10T07:57:12.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:12 smithi002 ceph-mon[30206]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-10T07:57:12.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:12 smithi002 ceph-mon[30206]: fsmap 2024-09-10T07:57:12.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:12 smithi002 ceph-mon[30206]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T07:57:12.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:12 smithi002 ceph-mon[30206]: mgrmap e1: no daemons active 2024-09-10T07:57:12.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:12 smithi002 ceph-mon[30206]: from='client.? 172.21.15.2:0/1634957936' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T07:57:12.623 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T07:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout id: 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T07:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-09-10T07:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout services: 2024-09-10T07:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.675633s) 2024-09-10T07:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T07:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T07:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-09-10T07:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout data: 2024-09-10T07:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T07:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T07:57:12.625 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T07:57:12.625 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T07:57:12.625 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-09-10T07:57:12.625 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.856+0000 7f37ed800640 1 Processor -- start 2024-09-10T07:57:12.625 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.857+0000 7f37ed800640 1 -- start start 2024-09-10T07:57:12.625 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.857+0000 7f37ed800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 0x7f37e8107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:12.625 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.857+0000 7f37ed800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8107ee0 con 0x7f37e8107510 2024-09-10T07:57:12.625 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.858+0000 7f37e7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 0x7f37e8107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:12.625 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.858+0000 7f37e7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 0x7f37e8107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52542/0 (socket says 172.21.15.2:52542) 2024-09-10T07:57:12.625 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.858+0000 7f37e7e00640 1 -- 172.21.15.2:0/631703530 learned_addr learned my addr 172.21.15.2:0/631703530 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:12.625 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.858+0000 7f37e7e00640 1 -- 172.21.15.2:0/631703530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e8108710 con 0x7f37e8107510 2024-09-10T07:57:12.625 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.861+0000 7f37e7e00640 1 --2- 172.21.15.2:0/631703530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 0x7f37e8107910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009820 tx=0x7f37d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a461fe2ca966f42b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:12.626 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.861+0000 7f37e6a00640 1 -- 172.21.15.2:0/631703530 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37d8037070 con 0x7f37e8107510 2024-09-10T07:57:12.626 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.861+0000 7f37e6a00640 1 -- 172.21.15.2:0/631703530 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d8032440 con 0x7f37e8107510 2024-09-10T07:57:12.626 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.861+0000 7f37e6a00640 1 -- 172.21.15.2:0/631703530 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37d8032740 con 0x7f37e8107510 2024-09-10T07:57:12.626 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.862+0000 7f37ed800640 1 -- 172.21.15.2:0/631703530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 msgr2=0x7f37e8107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:12.626 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.862+0000 7f37ed800640 1 --2- 172.21.15.2:0/631703530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 0x7f37e8107910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009820 tx=0x7f37d802a270 comp rx=0 tx=0).stop 2024-09-10T07:57:12.626 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.862+0000 7f37ed800640 1 -- 172.21.15.2:0/631703530 shutdown_connections 2024-09-10T07:57:12.626 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.862+0000 7f37ed800640 1 --2- 172.21.15.2:0/631703530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 0x7f37e8107910 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:12.626 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.862+0000 7f37ed800640 1 -- 172.21.15.2:0/631703530 >> 172.21.15.2:0/631703530 conn(0x7f37e8102cc0 msgr2=0x7f37e81050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:12.627 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.862+0000 7f37ed800640 1 -- 172.21.15.2:0/631703530 shutdown_connections 2024-09-10T07:57:12.627 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.862+0000 7f37ed800640 1 -- 172.21.15.2:0/631703530 wait complete. 2024-09-10T07:57:12.627 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.863+0000 7f37ed800640 1 Processor -- start 2024-09-10T07:57:12.627 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.863+0000 7f37ed800640 1 -- start start 2024-09-10T07:57:12.627 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.864+0000 7f37ed800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 0x7f37e819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:12.627 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.864+0000 7f37ed800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8108cb0 con 0x7f37e8107510 2024-09-10T07:57:12.627 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.864+0000 7f37e7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 0x7f37e819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:12.627 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.864+0000 7f37e7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 0x7f37e819ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52552/0 (socket says 172.21.15.2:52552) 2024-09-10T07:57:12.627 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.864+0000 7f37e7e00640 1 -- 172.21.15.2:0/1634957936 learned_addr learned my addr 172.21.15.2:0/1634957936 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:12.627 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.864+0000 7f37e7e00640 1 -- 172.21.15.2:0/1634957936 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e819d410 con 0x7f37e8107510 2024-09-10T07:57:12.627 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.864+0000 7f37e7e00640 1 --2- 172.21.15.2:0/1634957936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 0x7f37e819ced0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f37d80090b0 tx=0x7f37d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:12.627 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.865+0000 7f37e4c00640 1 -- 172.21.15.2:0/1634957936 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37d8041070 con 0x7f37e8107510 2024-09-10T07:57:12.628 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.865+0000 7f37e4c00640 1 -- 172.21.15.2:0/1634957936 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d803c7a0 con 0x7f37e8107510 2024-09-10T07:57:12.628 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.865+0000 7f37e4c00640 1 -- 172.21.15.2:0/1634957936 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37d8037040 con 0x7f37e8107510 2024-09-10T07:57:12.628 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.865+0000 7f37ed800640 1 -- 172.21.15.2:0/1634957936 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e819d6a0 con 0x7f37e8107510 2024-09-10T07:57:12.628 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.865+0000 7f37ed800640 1 -- 172.21.15.2:0/1634957936 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e81a0390 con 0x7f37e8107510 2024-09-10T07:57:12.628 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.865+0000 7f37e4c00640 1 -- 172.21.15.2:0/1634957936 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f37d8044070 con 0x7f37e8107510 2024-09-10T07:57:12.628 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.865+0000 7f37e4c00640 1 -- 172.21.15.2:0/1634957936 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f37d803d3f0 con 0x7f37e8107510 2024-09-10T07:57:12.628 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.866+0000 7f37ed800640 1 -- 172.21.15.2:0/1634957936 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e8109300 con 0x7f37e8107510 2024-09-10T07:57:12.628 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.869+0000 7f37e4c00640 1 -- 172.21.15.2:0/1634957936 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f37d8046450 con 0x7f37e8107510 2024-09-10T07:57:12.628 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.934+0000 7f37ed800640 1 -- 172.21.15.2:0/1634957936 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f37e802d910 con 0x7f37e8107510 2024-09-10T07:57:12.628 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.935+0000 7f37e4c00640 1 -- 172.21.15.2:0/1634957936 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f37d8046630 con 0x7f37e8107510 2024-09-10T07:57:12.628 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.935+0000 7f37ed800640 1 -- 172.21.15.2:0/1634957936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 msgr2=0x7f37e819ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:12.629 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.935+0000 7f37ed800640 1 --2- 172.21.15.2:0/1634957936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 0x7f37e819ced0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f37d80090b0 tx=0x7f37d80046c0 comp rx=0 tx=0).stop 2024-09-10T07:57:12.629 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.936+0000 7f37ed800640 1 -- 172.21.15.2:0/1634957936 shutdown_connections 2024-09-10T07:57:12.629 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.936+0000 7f37ed800640 1 --2- 172.21.15.2:0/1634957936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37e8107510 0x7f37e819ced0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:12.629 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.936+0000 7f37ed800640 1 -- 172.21.15.2:0/1634957936 >> 172.21.15.2:0/1634957936 conn(0x7f37e8102cc0 msgr2=0x7f37e818efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:12.629 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.936+0000 7f37ed800640 1 -- 172.21.15.2:0/1634957936 shutdown_connections 2024-09-10T07:57:12.629 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:11.936+0000 7f37ed800640 1 -- 172.21.15.2:0/1634957936 wait complete. 2024-09-10T07:57:12.629 INFO:teuthology.orchestra.run.smithi002.stdout:mon is available 2024-09-10T07:57:12.629 INFO:teuthology.orchestra.run.smithi002.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T07:57:13.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:13 smithi002 ceph-mon[30206]: from='client.? 172.21.15.2:0/4069370469' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T07:57:13.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:13 smithi002 ceph-mon[30206]: from='client.? 172.21.15.2:0/4069370469' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T07:57:13.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-09-10T07:57:13.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T07:57:13.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:57:13.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2024-09-10T07:57:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T07:57:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T07:57:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T07:57:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T07:57:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-09-10T07:57:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T07:57:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T07:57:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-09-10T07:57:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T07:57:13.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T07:57:13.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T07:57:13.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T07:57:13.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T07:57:13.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T07:57:13.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.113+0000 7fefb3e00640 1 Processor -- start 2024-09-10T07:57:13.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.113+0000 7fefb3e00640 1 -- start start 2024-09-10T07:57:13.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.113+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 0x7fefb4109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:13.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.114+0000 7fefb3e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410a2b0 con 0x7fefb41098e0 2024-09-10T07:57:13.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.114+0000 7fefb2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 0x7fefb4109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:13.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.114+0000 7fefb2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 0x7fefb4109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52556/0 (socket says 172.21.15.2:52556) 2024-09-10T07:57:13.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.114+0000 7fefb2a00640 1 -- 172.21.15.2:0/954064060 learned_addr learned my addr 172.21.15.2:0/954064060 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:13.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.114+0000 7fefb2a00640 1 -- 172.21.15.2:0/954064060 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb410a430 con 0x7fefb41098e0 2024-09-10T07:57:13.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.115+0000 7fefb2a00640 1 --2- 172.21.15.2:0/954064060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 0x7fefb4109ce0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009e80 tx=0x7fefa402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12a0af879c7bc5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:13.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.116+0000 7fefb2000640 1 -- 172.21.15.2:0/954064060 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefa4036070 con 0x7fefb41098e0 2024-09-10T07:57:13.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.116+0000 7fefb2000640 1 -- 172.21.15.2:0/954064060 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa4031400 con 0x7fefb41098e0 2024-09-10T07:57:13.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.116+0000 7fefb2000640 1 -- 172.21.15.2:0/954064060 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefa4031700 con 0x7fefb41098e0 2024-09-10T07:57:13.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.116+0000 7fefb3e00640 1 -- 172.21.15.2:0/954064060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 msgr2=0x7fefb4109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:13.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.116+0000 7fefb3e00640 1 --2- 172.21.15.2:0/954064060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 0x7fefb4109ce0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009e80 tx=0x7fefa402a7a0 comp rx=0 tx=0).stop 2024-09-10T07:57:13.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.116+0000 7fefb3e00640 1 -- 172.21.15.2:0/954064060 shutdown_connections 2024-09-10T07:57:13.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.116+0000 7fefb3e00640 1 --2- 172.21.15.2:0/954064060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 0x7fefb4109ce0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:13.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.116+0000 7fefb3e00640 1 -- 172.21.15.2:0/954064060 >> 172.21.15.2:0/954064060 conn(0x7fefb407ccf0 msgr2=0x7fefb407d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:13.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.116+0000 7fefb3e00640 1 -- 172.21.15.2:0/954064060 shutdown_connections 2024-09-10T07:57:13.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.116+0000 7fefb3e00640 1 -- 172.21.15.2:0/954064060 wait complete. 2024-09-10T07:57:13.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.117+0000 7fefb3e00640 1 Processor -- start 2024-09-10T07:57:13.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.117+0000 7fefb3e00640 1 -- start start 2024-09-10T07:57:13.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.118+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 0x7fefb419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:13.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.118+0000 7fefb3e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410ac70 con 0x7fefb41098e0 2024-09-10T07:57:13.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.118+0000 7fefb2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 0x7fefb419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:13.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.118+0000 7fefb2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 0x7fefb419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52566/0 (socket says 172.21.15.2:52566) 2024-09-10T07:57:13.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.118+0000 7fefb2a00640 1 -- 172.21.15.2:0/4069370469 learned_addr learned my addr 172.21.15.2:0/4069370469 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:13.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.119+0000 7fefb2a00640 1 -- 172.21.15.2:0/4069370469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb419d3f0 con 0x7fefb41098e0 2024-09-10T07:57:13.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.119+0000 7fefb2a00640 1 --2- 172.21.15.2:0/4069370469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 0x7fefb419ceb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fefa4034040 tx=0x7fefa4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:13.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.119+0000 7fefabe00640 1 -- 172.21.15.2:0/4069370469 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefa4041070 con 0x7fefb41098e0 2024-09-10T07:57:13.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.119+0000 7fefabe00640 1 -- 172.21.15.2:0/4069370469 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa403c650 con 0x7fefb41098e0 2024-09-10T07:57:13.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.119+0000 7fefabe00640 1 -- 172.21.15.2:0/4069370469 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefa4036040 con 0x7fefb41098e0 2024-09-10T07:57:13.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.119+0000 7fefb3e00640 1 -- 172.21.15.2:0/4069370469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb419d680 con 0x7fefb41098e0 2024-09-10T07:57:13.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.119+0000 7fefb3e00640 1 -- 172.21.15.2:0/4069370469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb41a0370 con 0x7fefb41098e0 2024-09-10T07:57:13.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.120+0000 7fefabe00640 1 -- 172.21.15.2:0/4069370469 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fefa4044070 con 0x7fefb41098e0 2024-09-10T07:57:13.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.120+0000 7fefabe00640 1 -- 172.21.15.2:0/4069370469 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fefa4046c10 con 0x7fefb41098e0 2024-09-10T07:57:13.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.120+0000 7fefb3e00640 1 -- 172.21.15.2:0/4069370469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb4109d60 con 0x7fefb41098e0 2024-09-10T07:57:13.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.123+0000 7fefabe00640 1 -- 172.21.15.2:0/4069370469 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fefa403c950 con 0x7fefb41098e0 2024-09-10T07:57:13.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.187+0000 7fefb3e00640 1 -- 172.21.15.2:0/4069370469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fefb402d910 con 0x7fefb41098e0 2024-09-10T07:57:13.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.191+0000 7fefabe00640 1 -- 172.21.15.2:0/4069370469 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7fefa403cb30 con 0x7fefb41098e0 2024-09-10T07:57:13.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.192+0000 7fefabe00640 1 -- 172.21.15.2:0/4069370469 <== mon.0 v2:172.21.15.2:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fefa4031400 con 0x7fefb41098e0 2024-09-10T07:57:13.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.193+0000 7fefb3e00640 1 -- 172.21.15.2:0/4069370469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 msgr2=0x7fefb419ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:13.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.193+0000 7fefb3e00640 1 --2- 172.21.15.2:0/4069370469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 0x7fefb419ceb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fefa4034040 tx=0x7fefa4004a40 comp rx=0 tx=0).stop 2024-09-10T07:57:13.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.193+0000 7fefb3e00640 1 -- 172.21.15.2:0/4069370469 shutdown_connections 2024-09-10T07:57:13.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.193+0000 7fefb3e00640 1 --2- 172.21.15.2:0/4069370469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb41098e0 0x7fefb419ceb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:13.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.193+0000 7fefb3e00640 1 -- 172.21.15.2:0/4069370469 >> 172.21.15.2:0/4069370469 conn(0x7fefb407ccf0 msgr2=0x7fefb4193ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:13.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.193+0000 7fefb3e00640 1 -- 172.21.15.2:0/4069370469 shutdown_connections 2024-09-10T07:57:13.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:13.193+0000 7fefb3e00640 1 -- 172.21.15.2:0/4069370469 wait complete. 2024-09-10T07:57:13.856 INFO:teuthology.orchestra.run.smithi002.stdout:Generating new minimal ceph.conf... 2024-09-10T07:57:14.725 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:14 smithi002 ceph-mon[30206]: from='client.? 172.21.15.2:0/2581500240' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:57:15.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.327+0000 7fbfbc400640 1 Processor -- start 2024-09-10T07:57:15.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.327+0000 7fbfbc400640 1 -- start start 2024-09-10T07:57:15.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.328+0000 7fbfbc400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 0x7fbfb4109d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:15.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.328+0000 7fbfbc400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb410a350 con 0x7fbfb4109980 2024-09-10T07:57:15.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.328+0000 7fbfb9a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 0x7fbfb4109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:15.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.328+0000 7fbfb9a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 0x7fbfb4109d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52570/0 (socket says 172.21.15.2:52570) 2024-09-10T07:57:15.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.328+0000 7fbfb9a00640 1 -- 172.21.15.2:0/3012121810 learned_addr learned my addr 172.21.15.2:0/3012121810 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:15.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.328+0000 7fbfb9a00640 1 -- 172.21.15.2:0/3012121810 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb410aaa0 con 0x7fbfb4109980 2024-09-10T07:57:15.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.329+0000 7fbfb9a00640 1 --2- 172.21.15.2:0/3012121810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 0x7fbfb4109d80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009820 tx=0x7fbfa802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d50cc04d1ea2b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:15.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.330+0000 7fbfb3e00640 1 -- 172.21.15.2:0/3012121810 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfa8037070 con 0x7fbfb4109980 2024-09-10T07:57:15.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.330+0000 7fbfb3e00640 1 -- 172.21.15.2:0/3012121810 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbfa8032900 con 0x7fbfb4109980 2024-09-10T07:57:15.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.330+0000 7fbfb3e00640 1 -- 172.21.15.2:0/3012121810 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfa8030850 con 0x7fbfb4109980 2024-09-10T07:57:15.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.330+0000 7fbfbc400640 1 -- 172.21.15.2:0/3012121810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 msgr2=0x7fbfb4109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:15.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.330+0000 7fbfbc400640 1 --2- 172.21.15.2:0/3012121810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 0x7fbfb4109d80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009820 tx=0x7fbfa802a270 comp rx=0 tx=0).stop 2024-09-10T07:57:15.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.330+0000 7fbfbc400640 1 -- 172.21.15.2:0/3012121810 shutdown_connections 2024-09-10T07:57:15.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.330+0000 7fbfbc400640 1 --2- 172.21.15.2:0/3012121810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 0x7fbfb4109d80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:15.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.330+0000 7fbfbc400640 1 -- 172.21.15.2:0/3012121810 >> 172.21.15.2:0/3012121810 conn(0x7fbfb407ced0 msgr2=0x7fbfb407d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:15.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.331+0000 7fbfbc400640 1 -- 172.21.15.2:0/3012121810 shutdown_connections 2024-09-10T07:57:15.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.331+0000 7fbfbc400640 1 -- 172.21.15.2:0/3012121810 wait complete. 2024-09-10T07:57:15.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.332+0000 7fbfbc400640 1 Processor -- start 2024-09-10T07:57:15.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.332+0000 7fbfbc400640 1 -- start start 2024-09-10T07:57:15.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.332+0000 7fbfbc400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 0x7fbfb419d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:15.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.332+0000 7fbfbc400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb410de50 con 0x7fbfb4109980 2024-09-10T07:57:15.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.332+0000 7fbfb9a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 0x7fbfb419d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:15.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.332+0000 7fbfb9a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 0x7fbfb419d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52574/0 (socket says 172.21.15.2:52574) 2024-09-10T07:57:15.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.332+0000 7fbfb9a00640 1 -- 172.21.15.2:0/2581500240 learned_addr learned my addr 172.21.15.2:0/2581500240 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:15.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.333+0000 7fbfb9a00640 1 -- 172.21.15.2:0/2581500240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb419dde0 con 0x7fbfb4109980 2024-09-10T07:57:15.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.333+0000 7fbfb9a00640 1 --2- 172.21.15.2:0/2581500240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 0x7fbfb419d8a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80097f0 tx=0x7fbfa8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:15.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.333+0000 7fbfb2000640 1 -- 172.21.15.2:0/2581500240 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfa8037040 con 0x7fbfb4109980 2024-09-10T07:57:15.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.333+0000 7fbfb2000640 1 -- 172.21.15.2:0/2581500240 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbfa8040070 con 0x7fbfb4109980 2024-09-10T07:57:15.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.333+0000 7fbfb2000640 1 -- 172.21.15.2:0/2581500240 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfa803a390 con 0x7fbfb4109980 2024-09-10T07:57:15.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.333+0000 7fbfbc400640 1 -- 172.21.15.2:0/2581500240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb419e070 con 0x7fbfb4109980 2024-09-10T07:57:15.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.333+0000 7fbfbc400640 1 -- 172.21.15.2:0/2581500240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb41a0d60 con 0x7fbfb4109980 2024-09-10T07:57:15.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.334+0000 7fbfb2000640 1 -- 172.21.15.2:0/2581500240 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbfa803a5e0 con 0x7fbfb4109980 2024-09-10T07:57:15.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.334+0000 7fbfb2000640 1 -- 172.21.15.2:0/2581500240 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbfa8045950 con 0x7fbfb4109980 2024-09-10T07:57:15.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.335+0000 7fbfbc400640 1 -- 172.21.15.2:0/2581500240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb410dfd0 con 0x7fbfb4109980 2024-09-10T07:57:15.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.337+0000 7fbfb2000640 1 -- 172.21.15.2:0/2581500240 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbfa8045460 con 0x7fbfb4109980 2024-09-10T07:57:15.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.401+0000 7fbfbc400640 1 -- 172.21.15.2:0/2581500240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fbfb41a0f30 con 0x7fbfb4109980 2024-09-10T07:57:15.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.401+0000 7fbfb2000640 1 -- 172.21.15.2:0/2581500240 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7fbfa803b430 con 0x7fbfb4109980 2024-09-10T07:57:15.034 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.402+0000 7fbfbc400640 1 -- 172.21.15.2:0/2581500240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 msgr2=0x7fbfb419d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:15.035 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.402+0000 7fbfbc400640 1 --2- 172.21.15.2:0/2581500240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 0x7fbfb419d8a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80097f0 tx=0x7fbfa8030d80 comp rx=0 tx=0).stop 2024-09-10T07:57:15.035 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.402+0000 7fbfbc400640 1 -- 172.21.15.2:0/2581500240 shutdown_connections 2024-09-10T07:57:15.035 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.402+0000 7fbfbc400640 1 --2- 172.21.15.2:0/2581500240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb4109980 0x7fbfb419d8a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:15.035 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.402+0000 7fbfbc400640 1 -- 172.21.15.2:0/2581500240 >> 172.21.15.2:0/2581500240 conn(0x7fbfb407ced0 msgr2=0x7fbfb4107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:15.035 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.403+0000 7fbfbc400640 1 -- 172.21.15.2:0/2581500240 shutdown_connections 2024-09-10T07:57:15.035 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:14.403+0000 7fbfbc400640 1 -- 172.21.15.2:0/2581500240 wait complete. 2024-09-10T07:57:15.035 INFO:teuthology.orchestra.run.smithi002.stdout:Restarting the monitor... 2024-09-10T07:57:15.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:15 smithi002 systemd[1]: Stopping Ceph mon.a for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T07:57:15.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:15 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a[30202]: 2024-09-10T07:57:15.403+0000 7f2b0b200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T07:57:15.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:15 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a[30202]: 2024-09-10T07:57:15.403+0000 7f2b0b200640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T07:57:15.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:15 smithi002 podman[30442]: 2024-09-10 07:57:15.640481434 +0000 UTC m=+0.355472555 container died 4cf63af4d00c192b9d219b6496c001219032fb954f1fca5642e9521268087354 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T07:57:16.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:16 smithi002 podman[30442]: 2024-09-10 07:57:16.100642976 +0000 UTC m=+0.815634093 container cleanup 4cf63af4d00c192b9d219b6496c001219032fb954f1fca5642e9521268087354 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T07:57:16.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:16 smithi002 bash[30442]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a 2024-09-10T07:57:16.518 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:16 smithi002 podman[30454]: 2024-09-10 07:57:16.259676072 +0000 UTC m=+0.608387279 container remove 4cf63af4d00c192b9d219b6496c001219032fb954f1fca5642e9521268087354 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-10T07:57:16.842 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:16 smithi002 systemd[1]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T07:57:16.843 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:16 smithi002 systemd[1]: Stopped Ceph mon.a for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T07:57:16.843 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:16 smithi002 systemd[1]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.a.service: Consumed 1.217s CPU time. 2024-09-10T07:57:16.843 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:16 smithi002 systemd[1]: Starting Ceph mon.a for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T07:57:17.118 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:16 smithi002 podman[30545]: 2024-09-10 07:57:16.748591286 +0000 UTC m=+0.026897033 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:57:17.119 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:16 smithi002 podman[30545]: 2024-09-10 07:57:16.868842298 +0000 UTC m=+0.147148023 container create f3a068f1b9648de30831622ded23e5d2cd06caa8a39b35a11b79cbb97588267c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:57:17.119 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 podman[30545]: 2024-09-10 07:57:17.079811185 +0000 UTC m=+0.358116927 container init f3a068f1b9648de30831622ded23e5d2cd06caa8a39b35a11b79cbb97588267c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T07:57:17.119 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 podman[30545]: 2024-09-10 07:57:17.08622986 +0000 UTC m=+0.364535587 container start f3a068f1b9648de30831622ded23e5d2cd06caa8a39b35a11b79cbb97588267c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T07:57:17.176 INFO:teuthology.orchestra.run.smithi002.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T07:57:17.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T07:57:17.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T07:57:17.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: pidfile_write: ignore empty --pid-file 2024-09-10T07:57:17.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: load: jerasure load: lrc 2024-09-10T07:57:17.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: RocksDB version: 7.9.2 2024-09-10T07:57:17.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Git sha 0 2024-09-10T07:57:17.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T07:57:17.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: DB SUMMARY 2024-09-10T07:57:17.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: DB Session ID: F04N6S6NOHVFGP1MDOAR 2024-09-10T07:57:17.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: CURRENT file: CURRENT 2024-09-10T07:57:17.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T07:57:17.409 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105952 ; 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.error_if_exists: 0 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.create_if_missing: 0 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.paranoid_checks: 1 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.env: 0x55971a30b940 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.info_log: 0x559723eab6a0 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.statistics: (nil) 2024-09-10T07:57:17.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.use_fsync: 0 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_log_file_size: 0 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.allow_fallocate: 1 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.use_direct_reads: 0 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.db_log_dir: 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.wal_dir: 2024-09-10T07:57:17.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T07:57:17.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T07:57:17.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T07:57:17.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T07:57:17.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T07:57:17.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T07:57:17.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T07:57:17.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T07:57:17.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.write_buffer_manager: 0x559723ebadc0 2024-09-10T07:57:17.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T07:57:17.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T07:57:17.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T07:57:17.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T07:57:17.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T07:57:17.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T07:57:17.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T07:57:17.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T07:57:17.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.unordered_write: 0 2024-09-10T07:57:17.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T07:57:17.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T07:57:17.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T07:57:17.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T07:57:17.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.row_cache: None 2024-09-10T07:57:17.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.wal_filter: None 2024-09-10T07:57:17.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T07:57:17.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T07:57:17.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.two_write_queues: 0 2024-09-10T07:57:17.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T07:57:17.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.wal_compression: 0 2024-09-10T07:57:17.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.atomic_flush: 0 2024-09-10T07:57:17.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T07:57:17.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T07:57:17.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T07:57:17.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.log_readahead_size: 0 2024-09-10T07:57:17.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T07:57:17.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T07:57:17.415 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T07:57:17.415 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T07:57:17.415 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T07:57:17.415 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T07:57:17.415 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T07:57:17.416 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_background_jobs: 2 2024-09-10T07:57:17.416 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_background_compactions: -1 2024-09-10T07:57:17.416 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_subcompactions: 1 2024-09-10T07:57:17.417 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T07:57:17.417 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T07:57:17.417 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T07:57:17.417 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T07:57:17.417 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T07:57:17.417 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T07:57:17.417 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T07:57:17.417 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T07:57:17.417 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_open_files: -1 2024-09-10T07:57:17.417 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T07:57:17.417 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T07:57:17.417 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T07:57:17.417 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T07:57:17.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_background_flushes: -1 2024-09-10T07:57:17.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Compression algorithms supported: 2024-09-10T07:57:17.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: kZSTD supported: 0 2024-09-10T07:57:17.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: kXpressCompression supported: 0 2024-09-10T07:57:17.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: kBZip2Compression supported: 0 2024-09-10T07:57:17.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T07:57:17.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: kLZ4Compression supported: 1 2024-09-10T07:57:17.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: kZlibCompression supported: 1 2024-09-10T07:57:17.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T07:57:17.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: kSnappyCompression supported: 1 2024-09-10T07:57:17.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T07:57:17.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T07:57:17.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T07:57:17.419 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T07:57:17.419 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T07:57:17.419 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.merge_operator: 2024-09-10T07:57:17.419 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compaction_filter: None 2024-09-10T07:57:17.419 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T07:57:17.419 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T07:57:17.419 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T07:57:17.419 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T07:57:17.419 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559723eab300) 2024-09-10T07:57:17.419 INFO:journalctl@ceph.mon.a.smithi002.stdout: cache_index_and_filter_blocks: 1 2024-09-10T07:57:17.420 INFO:journalctl@ceph.mon.a.smithi002.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T07:57:17.420 INFO:journalctl@ceph.mon.a.smithi002.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T07:57:17.420 INFO:journalctl@ceph.mon.a.smithi002.stdout: pin_top_level_index_and_filter: 1 2024-09-10T07:57:17.420 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_type: 0 2024-09-10T07:57:17.420 INFO:journalctl@ceph.mon.a.smithi002.stdout: data_block_index_type: 0 2024-09-10T07:57:17.420 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_shortening: 1 2024-09-10T07:57:17.420 INFO:journalctl@ceph.mon.a.smithi002.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T07:57:17.420 INFO:journalctl@ceph.mon.a.smithi002.stdout: checksum: 4 2024-09-10T07:57:17.420 INFO:journalctl@ceph.mon.a.smithi002.stdout: no_block_cache: 0 2024-09-10T07:57:17.420 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache: 0x559723e9f1f0 2024-09-10T07:57:17.420 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_name: BinnedLRUCache 2024-09-10T07:57:17.421 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_options: 2024-09-10T07:57:17.421 INFO:journalctl@ceph.mon.a.smithi002.stdout: capacity : 536870912 2024-09-10T07:57:17.421 INFO:journalctl@ceph.mon.a.smithi002.stdout: num_shard_bits : 4 2024-09-10T07:57:17.421 INFO:journalctl@ceph.mon.a.smithi002.stdout: strict_capacity_limit : 0 2024-09-10T07:57:17.421 INFO:journalctl@ceph.mon.a.smithi002.stdout: high_pri_pool_ratio: 0.000 2024-09-10T07:57:17.421 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_compressed: (nil) 2024-09-10T07:57:17.421 INFO:journalctl@ceph.mon.a.smithi002.stdout: persistent_cache: (nil) 2024-09-10T07:57:17.421 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_size: 4096 2024-09-10T07:57:17.421 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_size_deviation: 10 2024-09-10T07:57:17.421 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_restart_interval: 16 2024-09-10T07:57:17.421 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_block_restart_interval: 1 2024-09-10T07:57:17.421 INFO:journalctl@ceph.mon.a.smithi002.stdout: metadata_block_size: 4096 2024-09-10T07:57:17.421 INFO:journalctl@ceph.mon.a.smithi002.stdout: partition_filters: 0 2024-09-10T07:57:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout: use_delta_encoding: 1 2024-09-10T07:57:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout: filter_policy: bloomfilter 2024-09-10T07:57:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout: whole_key_filtering: 1 2024-09-10T07:57:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout: verify_compression: 0 2024-09-10T07:57:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout: read_amp_bytes_per_bit: 0 2024-09-10T07:57:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout: format_version: 5 2024-09-10T07:57:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout: enable_index_compression: 1 2024-09-10T07:57:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_align: 0 2024-09-10T07:57:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout: max_auto_readahead_size: 262144 2024-09-10T07:57:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout: prepopulate_block_cache: 0 2024-09-10T07:57:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout: initial_auto_readahead_size: 8192 2024-09-10T07:57:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T07:57:17.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compression: NoCompression 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.num_levels: 7 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T07:57:17.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T07:57:17.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T07:57:17.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T07:57:17.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T07:57:17.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T07:57:17.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T07:57:17.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T07:57:17.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T07:57:17.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T07:57:17.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T07:57:17.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T07:57:17.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T07:57:17.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T07:57:17.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T07:57:17.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T07:57:17.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T07:57:17.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T07:57:17.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T07:57:17.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T07:57:17.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T07:57:17.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T07:57:17.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.inplace_update_support: 0 2024-09-10T07:57:17.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T07:57:17.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T07:57:17.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T07:57:17.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T07:57:17.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.bloom_locality: 0 2024-09-10T07:57:17.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.max_successive_merges: 0 2024-09-10T07:57:17.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.ttl: 2592000 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.enable_blob_files: false 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.min_blob_size: 0 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T07:57:17.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T07:57:17.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T07:57:17.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T07:57:17.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T07:57:17.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T07:57:17.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T07:57:17.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6afca57a-069a-485e-b730-17a2e3a009b6 2024-09-10T07:57:17.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725955037130050, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T07:57:17.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T07:57:17.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725955037131683, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100967, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98852, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14453, "raw_average_key_size": 50, "raw_value_size": 91024, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725955037, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6afca57a-069a-485e-b730-17a2e3a009b6", "db_session_id": "F04N6S6NOHVFGP1MDOAR", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T07:57:17.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725955037131878, "job": 1, "event": "recovery_finished"} 2024-09-10T07:57:17.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T07:57:17.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T07:57:17.430 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559723ecee00 2024-09-10T07:57:17.430 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: DB pointer 0x559723fca000 2024-09-10T07:57:17.430 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T07:57:17.430 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T07:57:17.430 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** DB Stats ** 2024-09-10T07:57:17.430 INFO:journalctl@ceph.mon.a.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T07:57:17.430 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T07:57:17.430 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T07:57:17.430 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T07:57:17.430 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T07:57:17.430 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T07:57:17.430 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T07:57:17.430 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** Compaction Stats [default] ** 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: L0 2/0 100.46 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 98.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: Sum 2/0 100.46 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 98.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 98.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** Compaction Stats [default] ** 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 98.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T07:57:17.431 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative compaction: 0.00 GB write, 13.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval compaction: 0.00 GB write, 13.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout: Block cache BinnedLRUCache@0x559723e9f1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T07:57:17.432 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: starting mon.a rank 0 at public addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] at bind addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:57:17.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: mon.a@-1(???) e1 preinit fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:57:17.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: mon.a@-1(???).mds e1 new map 2024-09-10T07:57:17.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: mon.a@-1(???).mds e1 print_map 2024-09-10T07:57:17.433 INFO:journalctl@ceph.mon.a.smithi002.stdout: e1 2024-09-10T07:57:17.433 INFO:journalctl@ceph.mon.a.smithi002.stdout: btime 2024-09-10T07:57:11:261417+0000 2024-09-10T07:57:17.433 INFO:journalctl@ceph.mon.a.smithi002.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T07:57:17.433 INFO:journalctl@ceph.mon.a.smithi002.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T07:57:17.433 INFO:journalctl@ceph.mon.a.smithi002.stdout: legacy client fscid: -1 2024-09-10T07:57:17.433 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-09-10T07:57:17.433 INFO:journalctl@ceph.mon.a.smithi002.stdout: No filesystems configured 2024-09-10T07:57:17.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T07:57:17.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T07:57:17.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 bash[30545]: f3a068f1b9648de30831622ded23e5d2cd06caa8a39b35a11b79cbb97588267c 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 systemd[1]: Started Ceph mon.a for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: monmap epoch 1 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: last_changed 2024-09-10T07:57:06.640971+0000 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: created 2024-09-10T07:57:06.640971+0000 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: min_mon_release 19 (squid) 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: election_strategy: 1 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: fsmap 2024-09-10T07:57:17.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T07:57:17.435 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:17 smithi002 ceph-mon[30563]: mgrmap e1: no daemons active 2024-09-10T07:57:18.548 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.680+0000 7f52cfa00640 1 Processor -- start 2024-09-10T07:57:18.548 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.680+0000 7f52cfa00640 1 -- start start 2024-09-10T07:57:18.548 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.681+0000 7f52cfa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 0x7f52c807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:18.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.681+0000 7f52cfa00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52c807c8f0 con 0x7f52c807dec0 2024-09-10T07:57:18.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.681+0000 7f52cd000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 0x7f52c807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:18.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.681+0000 7f52cd000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 0x7f52c807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56228/0 (socket says 172.21.15.2:56228) 2024-09-10T07:57:18.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.681+0000 7f52cd000640 1 -- 172.21.15.2:0/1375462481 learned_addr learned my addr 172.21.15.2:0/1375462481 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:18.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.682+0000 7f52cd000640 1 -- 172.21.15.2:0/1375462481 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c807ca70 con 0x7f52c807dec0 2024-09-10T07:57:18.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.684+0000 7f52cd000640 1 --2- 172.21.15.2:0/1375462481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 0x7f52c807c320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009820 tx=0x7f52bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cd8ddf8b07c7c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:18.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.685+0000 7f52c7400640 1 -- 172.21.15.2:0/1375462481 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52bc037070 con 0x7f52c807dec0 2024-09-10T07:57:18.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.685+0000 7f52c7400640 1 -- 172.21.15.2:0/1375462481 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52bc032900 con 0x7f52c807dec0 2024-09-10T07:57:18.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.685+0000 7f52c7400640 1 -- 172.21.15.2:0/1375462481 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52bc030850 con 0x7f52c807dec0 2024-09-10T07:57:18.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.686+0000 7f52cfa00640 1 -- 172.21.15.2:0/1375462481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 msgr2=0x7f52c807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:18.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.686+0000 7f52cfa00640 1 --2- 172.21.15.2:0/1375462481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 0x7f52c807c320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009820 tx=0x7f52bc02a270 comp rx=0 tx=0).stop 2024-09-10T07:57:18.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.686+0000 7f52cfa00640 1 -- 172.21.15.2:0/1375462481 shutdown_connections 2024-09-10T07:57:18.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.686+0000 7f52cfa00640 1 --2- 172.21.15.2:0/1375462481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 0x7f52c807c320 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:18.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.686+0000 7f52cfa00640 1 -- 172.21.15.2:0/1375462481 >> 172.21.15.2:0/1375462481 conn(0x7f52c8102e40 msgr2=0x7f52c8105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:18.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.686+0000 7f52cfa00640 1 -- 172.21.15.2:0/1375462481 shutdown_connections 2024-09-10T07:57:18.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.686+0000 7f52cfa00640 1 -- 172.21.15.2:0/1375462481 wait complete. 2024-09-10T07:57:18.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.687+0000 7f52cfa00640 1 Processor -- start 2024-09-10T07:57:18.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.687+0000 7f52cfa00640 1 -- start start 2024-09-10T07:57:18.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.687+0000 7f52cfa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 0x7f52c819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:18.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.687+0000 7f52cfa00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52c810dd20 con 0x7f52c807dec0 2024-09-10T07:57:18.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.687+0000 7f52cd000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 0x7f52c819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:18.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.687+0000 7f52cd000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 0x7f52c819d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56236/0 (socket says 172.21.15.2:56236) 2024-09-10T07:57:18.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.687+0000 7f52cd000640 1 -- 172.21.15.2:0/3986521660 learned_addr learned my addr 172.21.15.2:0/3986521660 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.688+0000 7f52cd000640 1 -- 172.21.15.2:0/3986521660 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c819ddf0 con 0x7f52c807dec0 2024-09-10T07:57:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.688+0000 7f52cd000640 1 --2- 172.21.15.2:0/3986521660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 0x7f52c819d8b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f52bc032ce0 tx=0x7f52bc032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.688+0000 7f52c5600640 1 -- 172.21.15.2:0/3986521660 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52bc042070 con 0x7f52c807dec0 2024-09-10T07:57:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.688+0000 7f52c5600640 1 -- 172.21.15.2:0/3986521660 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52bc0308c0 con 0x7f52c807dec0 2024-09-10T07:57:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.688+0000 7f52cfa00640 1 -- 172.21.15.2:0/3986521660 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c819e080 con 0x7f52c807dec0 2024-09-10T07:57:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.688+0000 7f52c5600640 1 -- 172.21.15.2:0/3986521660 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52bc037040 con 0x7f52c807dec0 2024-09-10T07:57:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.689+0000 7f52cfa00640 1 -- 172.21.15.2:0/3986521660 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c81a0d70 con 0x7f52c807dec0 2024-09-10T07:57:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.689+0000 7f52c5600640 1 -- 172.21.15.2:0/3986521660 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f52bc045070 con 0x7f52c807dec0 2024-09-10T07:57:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.689+0000 7f52c5600640 1 -- 172.21.15.2:0/3986521660 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f52bc047bd0 con 0x7f52c807dec0 2024-09-10T07:57:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.690+0000 7f52cfa00640 1 -- 172.21.15.2:0/3986521660 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c807c3a0 con 0x7f52c807dec0 2024-09-10T07:57:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.693+0000 7f52c5600640 1 -- 172.21.15.2:0/3986521660 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f52bc03c830 con 0x7f52c807dec0 2024-09-10T07:57:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.756+0000 7f52cfa00640 1 -- 172.21.15.2:0/3986521660 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f52c81a0f40 con 0x7f52c807dec0 2024-09-10T07:57:18.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.758+0000 7f52c5600640 1 -- 172.21.15.2:0/3986521660 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f52bc03ca10 con 0x7f52c807dec0 2024-09-10T07:57:18.552 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.759+0000 7f52cfa00640 1 -- 172.21.15.2:0/3986521660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 msgr2=0x7f52c819d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:18.552 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.759+0000 7f52cfa00640 1 --2- 172.21.15.2:0/3986521660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 0x7f52c819d8b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f52bc032ce0 tx=0x7f52bc032d10 comp rx=0 tx=0).stop 2024-09-10T07:57:18.552 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.759+0000 7f52cfa00640 1 -- 172.21.15.2:0/3986521660 shutdown_connections 2024-09-10T07:57:18.552 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.759+0000 7f52cfa00640 1 --2- 172.21.15.2:0/3986521660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52c807dec0 0x7f52c819d8b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:18.552 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.759+0000 7f52cfa00640 1 -- 172.21.15.2:0/3986521660 >> 172.21.15.2:0/3986521660 conn(0x7f52c8102e40 msgr2=0x7f52c81039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:18.552 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.760+0000 7f52cfa00640 1 -- 172.21.15.2:0/3986521660 shutdown_connections 2024-09-10T07:57:18.552 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:17.760+0000 7f52cfa00640 1 -- 172.21.15.2:0/3986521660 wait complete. 2024-09-10T07:57:18.552 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T07:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T07:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mgr... 2024-09-10T07:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T07:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T07:57:18.942 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.x 2024-09-10T07:57:18.942 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T07:57:18.956 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:18 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3986521660' entity='client.admin' 2024-09-10T07:57:19.181 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968.target.wants/ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@.service. 2024-09-10T07:57:19.208 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:19 smithi002 systemd[1]: Starting Ceph mgr.x for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T07:57:19.770 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:19 smithi002 podman[30785]: 2024-09-10 07:57:19.513217604 +0000 UTC m=+0.117500065 container create 3688b0cf814b44ebfc5d4e47d74c7628ee68eb1207efa12ffc495bf5b64c94ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T07:57:19.770 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:19 smithi002 podman[30785]: 2024-09-10 07:57:19.423498783 +0000 UTC m=+0.027781247 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T07:57:19.770 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:19 smithi002 podman[30785]: 2024-09-10 07:57:19.757238461 +0000 UTC m=+0.361520939 container init 3688b0cf814b44ebfc5d4e47d74c7628ee68eb1207efa12ffc495bf5b64c94ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:57:19.770 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:19 smithi002 podman[30785]: 2024-09-10 07:57:19.763459726 +0000 UTC m=+0.367742189 container start 3688b0cf814b44ebfc5d4e47d74c7628ee68eb1207efa12ffc495bf5b64c94ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:57:19.910 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T07:57:19.910 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-09-10T07:57:19.925 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T07:57:19.926 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-09-10T07:57:19.926 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-09-10T07:57:19.926 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T07:57:19.945 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T07:57:19.945 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-09-10T07:57:19.960 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T07:57:19.961 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-09-10T07:57:19.961 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-09-10T07:57:19.961 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T07:57:19.962 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr to start... 2024-09-10T07:57:19.962 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr... 2024-09-10T07:57:20.227 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:19 smithi002 bash[30785]: 3688b0cf814b44ebfc5d4e47d74c7628ee68eb1207efa12ffc495bf5b64c94ac 2024-09-10T07:57:20.227 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:19 smithi002 systemd[1]: Started Ceph mgr.x for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T07:57:20.227 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:19 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:19.942+0000 7f80bb9da040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T07:57:20.227 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:20.028+0000 7f80bb9da040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T07:57:20.486 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:20.229+0000 7f80bb9da040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T07:57:20.486 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:20.314+0000 7f80bb9da040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T07:57:20.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:20 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1199218626' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T07:57:20.770 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:20.498+0000 7f80bb9da040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:57:20.770 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:20.674+0000 7f80bb9da040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T07:57:21.229 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:21 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:21.067+0000 7f80bb9da040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T07:57:21.229 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:21 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:21.141+0000 7f80bb9da040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T07:57:21.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-09-10T07:57:21.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:57:21.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "26ec64b0-6f4a-11ef-bcea-c7b262605968", 2024-09-10T07:57:21.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T07:57:21.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T07:57:21.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T07:57:21.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T07:57:21.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:21.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T07:57:21.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T07:57:21.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-09-10T07:57:21.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:57:21.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T07:57:21.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T07:57:21.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:57:21.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-10T07:57:21.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T07:57:21.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:57:21.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T07:57:21.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T07:57:21.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:21.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T07:57:21.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:57:21.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T07:57:21.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T07:57:21.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T07:57:21.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T07:57:21.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T07:57:21.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T07:57:21.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:21.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T07:57:21.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T07:57:21.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T07:57:21.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T07:57:21.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T07:57:21.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T07:57:21.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T07:57:21.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T07:57:21.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T07:57:21.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:21.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T07:57:21.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:57:21.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T07:57:11:261417+0000", 2024-09-10T07:57:21.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T07:57:21.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T07:57:21.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:21.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T07:57:21.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T07:57:21.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T07:57:21.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T07:57:21.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T07:57:21.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T07:57:21.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T07:57:21.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:57:21.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T07:57:21.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:21.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T07:57:21.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:57:21.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T07:57:11.262177+0000", 2024-09-10T07:57:21.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T07:57:21.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:21.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T07:57:21.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:57:21.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.486+0000 7f7bf6000640 1 Processor -- start 2024-09-10T07:57:21.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.486+0000 7f7bf6000640 1 -- start start 2024-09-10T07:57:21.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.486+0000 7f7bf6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 0x7f7bf0109d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:21.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.486+0000 7f7bf6000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf010a350 con 0x7f7bf0109980 2024-09-10T07:57:21.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.487+0000 7f7bef400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 0x7f7bf0109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:21.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.487+0000 7f7bef400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 0x7f7bf0109d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56264/0 (socket says 172.21.15.2:56264) 2024-09-10T07:57:21.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.487+0000 7f7bef400640 1 -- 172.21.15.2:0/2802361059 learned_addr learned my addr 172.21.15.2:0/2802361059 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:21.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.487+0000 7f7bef400640 1 -- 172.21.15.2:0/2802361059 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf010aaa0 con 0x7f7bf0109980 2024-09-10T07:57:21.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.488+0000 7f7bef400640 1 --2- 172.21.15.2:0/2802361059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 0x7f7bf0109d80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009820 tx=0x7f7be002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf6439163c46283 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:21.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.489+0000 7f7bee000640 1 -- 172.21.15.2:0/2802361059 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7be0037070 con 0x7f7bf0109980 2024-09-10T07:57:21.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.489+0000 7f7bee000640 1 -- 172.21.15.2:0/2802361059 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7be0032900 con 0x7f7bf0109980 2024-09-10T07:57:21.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.489+0000 7f7bee000640 1 -- 172.21.15.2:0/2802361059 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7be0030850 con 0x7f7bf0109980 2024-09-10T07:57:21.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.490+0000 7f7bf6000640 1 -- 172.21.15.2:0/2802361059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 msgr2=0x7f7bf0109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:21.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.490+0000 7f7bf6000640 1 --2- 172.21.15.2:0/2802361059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 0x7f7bf0109d80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009820 tx=0x7f7be002a270 comp rx=0 tx=0).stop 2024-09-10T07:57:21.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.490+0000 7f7bf6000640 1 -- 172.21.15.2:0/2802361059 shutdown_connections 2024-09-10T07:57:21.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.490+0000 7f7bf6000640 1 --2- 172.21.15.2:0/2802361059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 0x7f7bf0109d80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:21.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.490+0000 7f7bf6000640 1 -- 172.21.15.2:0/2802361059 >> 172.21.15.2:0/2802361059 conn(0x7f7bf007ced0 msgr2=0x7f7bf007d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:21.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.490+0000 7f7bf6000640 1 -- 172.21.15.2:0/2802361059 shutdown_connections 2024-09-10T07:57:21.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.490+0000 7f7bf6000640 1 -- 172.21.15.2:0/2802361059 wait complete. 2024-09-10T07:57:21.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.492+0000 7f7bf6000640 1 Processor -- start 2024-09-10T07:57:21.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.492+0000 7f7bf6000640 1 -- start start 2024-09-10T07:57:21.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.492+0000 7f7bf6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 0x7f7bf019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:21.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.492+0000 7f7bf6000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf010de50 con 0x7f7bf0109980 2024-09-10T07:57:21.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.492+0000 7f7bef400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 0x7f7bf019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:21.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.493+0000 7f7bef400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 0x7f7bf019d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56268/0 (socket says 172.21.15.2:56268) 2024-09-10T07:57:21.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.493+0000 7f7bef400640 1 -- 172.21.15.2:0/1199218626 learned_addr learned my addr 172.21.15.2:0/1199218626 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:21.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.493+0000 7f7bef400640 1 -- 172.21.15.2:0/1199218626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf019de00 con 0x7f7bf0109980 2024-09-10T07:57:21.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.493+0000 7f7bef400640 1 --2- 172.21.15.2:0/1199218626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 0x7f7bf019d8c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7be00097f0 tx=0x7f7be0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:21.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.493+0000 7f7be7e00640 1 -- 172.21.15.2:0/1199218626 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7be0037040 con 0x7f7bf0109980 2024-09-10T07:57:21.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.493+0000 7f7be7e00640 1 -- 172.21.15.2:0/1199218626 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7be0040070 con 0x7f7bf0109980 2024-09-10T07:57:21.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.493+0000 7f7be7e00640 1 -- 172.21.15.2:0/1199218626 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7be003a390 con 0x7f7bf0109980 2024-09-10T07:57:21.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.493+0000 7f7bf6000640 1 -- 172.21.15.2:0/1199218626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf019e090 con 0x7f7bf0109980 2024-09-10T07:57:21.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.494+0000 7f7bf6000640 1 -- 172.21.15.2:0/1199218626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf01a0d80 con 0x7f7bf0109980 2024-09-10T07:57:21.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.494+0000 7f7be7e00640 1 -- 172.21.15.2:0/1199218626 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7be003a5e0 con 0x7f7bf0109980 2024-09-10T07:57:21.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.494+0000 7f7be7e00640 1 -- 172.21.15.2:0/1199218626 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7be0045950 con 0x7f7bf0109980 2024-09-10T07:57:21.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.495+0000 7f7bf6000640 1 -- 172.21.15.2:0/1199218626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf0109e00 con 0x7f7bf0109980 2024-09-10T07:57:21.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.497+0000 7f7be7e00640 1 -- 172.21.15.2:0/1199218626 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7be0045460 con 0x7f7bf0109980 2024-09-10T07:57:21.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.563+0000 7f7bf6000640 1 -- 172.21.15.2:0/1199218626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7bf01a0f50 con 0x7f7bf0109980 2024-09-10T07:57:21.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.563+0000 7f7be7e00640 1 -- 172.21.15.2:0/1199218626 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f7be003b430 con 0x7f7bf0109980 2024-09-10T07:57:21.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.565+0000 7f7bf6000640 1 -- 172.21.15.2:0/1199218626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 msgr2=0x7f7bf019d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:21.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.565+0000 7f7bf6000640 1 --2- 172.21.15.2:0/1199218626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 0x7f7bf019d8c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7be00097f0 tx=0x7f7be0030d80 comp rx=0 tx=0).stop 2024-09-10T07:57:21.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.565+0000 7f7bf6000640 1 -- 172.21.15.2:0/1199218626 shutdown_connections 2024-09-10T07:57:21.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.565+0000 7f7bf6000640 1 --2- 172.21.15.2:0/1199218626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf0109980 0x7f7bf019d8c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:21.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.565+0000 7f7bf6000640 1 -- 172.21.15.2:0/1199218626 >> 172.21.15.2:0/1199218626 conn(0x7f7bf007ced0 msgr2=0x7f7bf0107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:21.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.565+0000 7f7bf6000640 1 -- 172.21.15.2:0/1199218626 shutdown_connections 2024-09-10T07:57:21.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:20.565+0000 7f7bf6000640 1 -- 172.21.15.2:0/1199218626 wait complete. 2024-09-10T07:57:21.255 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (1/15)... 2024-09-10T07:57:21.519 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:21 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:21.419+0000 7f80bb9da040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T07:57:23.019 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:22 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:22.606+0000 7f80bb9da040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T07:57:23.020 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:22 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:22.689+0000 7f80bb9da040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T07:57:23.949 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:23 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:23.662+0000 7f80bb9da040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T07:57:23.950 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:23 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:23.806+0000 7f80bb9da040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T07:57:24.238 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:24 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2699551638' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T07:57:24.239 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:23 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:23.981+0000 7f80bb9da040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:57:24.239 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:24.059+0000 7f80bb9da040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T07:57:24.239 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:24.238+0000 7f80bb9da040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T07:57:24.519 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:24.323+0000 7f80bb9da040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T07:57:24.520 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:24.398+0000 7f80bb9da040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T07:57:24.735 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-09-10T07:57:24.735 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:57:24.735 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "26ec64b0-6f4a-11ef-bcea-c7b262605968", 2024-09-10T07:57:24.735 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T07:57:24.735 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T07:57:24.735 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T07:57:24.735 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T07:57:24.736 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:24.736 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T07:57:24.736 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T07:57:24.736 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-09-10T07:57:24.736 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:57:24.736 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T07:57:24.736 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T07:57:24.736 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:57:24.736 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-10T07:57:24.736 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T07:57:24.736 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:57:24.736 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T07:57:24.736 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T07:57:24.737 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:24.737 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T07:57:24.737 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:57:24.737 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T07:57:24.737 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T07:57:24.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T07:57:24.744 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:24.745 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T07:57:11:261417+0000", 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T07:57:24.746 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:24.747 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T07:57:24.747 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:57:24.747 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T07:57:11.262177+0000", 2024-09-10T07:57:24.747 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T07:57:24.747 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:24.747 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T07:57:24.747 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:57:24.747 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.950+0000 7ff987e00640 1 Processor -- start 2024-09-10T07:57:24.747 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.950+0000 7ff987e00640 1 -- start start 2024-09-10T07:57:24.747 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.950+0000 7ff987e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 0x7ff98807c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:24.747 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.950+0000 7ff987e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff98807c930 con 0x7ff98807deb0 2024-09-10T07:57:24.747 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.950+0000 7ff986a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 0x7ff98807c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:24.747 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.950+0000 7ff986a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 0x7ff98807c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56270/0 (socket says 172.21.15.2:56270) 2024-09-10T07:57:24.748 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.950+0000 7ff986a00640 1 -- 172.21.15.2:0/2101207179 learned_addr learned my addr 172.21.15.2:0/2101207179 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:24.748 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.951+0000 7ff986a00640 1 -- 172.21.15.2:0/2101207179 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98807cab0 con 0x7ff98807deb0 2024-09-10T07:57:24.748 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.952+0000 7ff986a00640 1 --2- 172.21.15.2:0/2101207179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 0x7ff98807c360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff978009820 tx=0x7ff97802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db6c8d0bf996ff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:24.748 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.952+0000 7ff985600640 1 -- 172.21.15.2:0/2101207179 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff978037070 con 0x7ff98807deb0 2024-09-10T07:57:24.748 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.952+0000 7ff985600640 1 -- 172.21.15.2:0/2101207179 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff978032900 con 0x7ff98807deb0 2024-09-10T07:57:24.748 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.952+0000 7ff985600640 1 -- 172.21.15.2:0/2101207179 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff978030850 con 0x7ff98807deb0 2024-09-10T07:57:24.748 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.953+0000 7ff987e00640 1 -- 172.21.15.2:0/2101207179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 msgr2=0x7ff98807c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:24.748 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.953+0000 7ff987e00640 1 --2- 172.21.15.2:0/2101207179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 0x7ff98807c360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff978009820 tx=0x7ff97802a270 comp rx=0 tx=0).stop 2024-09-10T07:57:24.748 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.953+0000 7ff987e00640 1 -- 172.21.15.2:0/2101207179 shutdown_connections 2024-09-10T07:57:24.748 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.953+0000 7ff987e00640 1 --2- 172.21.15.2:0/2101207179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 0x7ff98807c360 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:24.748 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.953+0000 7ff987e00640 1 -- 172.21.15.2:0/2101207179 >> 172.21.15.2:0/2101207179 conn(0x7ff988102d10 msgr2=0x7ff988105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:24.748 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.953+0000 7ff987e00640 1 -- 172.21.15.2:0/2101207179 shutdown_connections 2024-09-10T07:57:24.749 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.953+0000 7ff987e00640 1 -- 172.21.15.2:0/2101207179 wait complete. 2024-09-10T07:57:24.749 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.954+0000 7ff987e00640 1 Processor -- start 2024-09-10T07:57:24.749 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.954+0000 7ff987e00640 1 -- start start 2024-09-10T07:57:24.749 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.955+0000 7ff987e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 0x7ff9881a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:24.749 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.955+0000 7ff987e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff988109740 con 0x7ff98807deb0 2024-09-10T07:57:24.749 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.955+0000 7ff986a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 0x7ff9881a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:24.749 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.955+0000 7ff986a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 0x7ff9881a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56278/0 (socket says 172.21.15.2:56278) 2024-09-10T07:57:24.749 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.955+0000 7ff986a00640 1 -- 172.21.15.2:0/2699551638 learned_addr learned my addr 172.21.15.2:0/2699551638 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:24.749 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.955+0000 7ff986a00640 1 -- 172.21.15.2:0/2699551638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9881a2080 con 0x7ff98807deb0 2024-09-10T07:57:24.749 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.956+0000 7ff986a00640 1 --2- 172.21.15.2:0/2699551638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 0x7ff9881a1b40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff9780097f0 tx=0x7ff978030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:24.749 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.956+0000 7ff97f400640 1 -- 172.21.15.2:0/2699551638 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff978042070 con 0x7ff98807deb0 2024-09-10T07:57:24.749 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.956+0000 7ff97f400640 1 -- 172.21.15.2:0/2699551638 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff978032ca0 con 0x7ff98807deb0 2024-09-10T07:57:24.749 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.956+0000 7ff987e00640 1 -- 172.21.15.2:0/2699551638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9881a2310 con 0x7ff98807deb0 2024-09-10T07:57:24.750 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.956+0000 7ff97f400640 1 -- 172.21.15.2:0/2699551638 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff978037040 con 0x7ff98807deb0 2024-09-10T07:57:24.750 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.956+0000 7ff987e00640 1 -- 172.21.15.2:0/2699551638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9881a5000 con 0x7ff98807deb0 2024-09-10T07:57:24.750 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.956+0000 7ff97f400640 1 -- 172.21.15.2:0/2699551638 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff978045070 con 0x7ff98807deb0 2024-09-10T07:57:24.750 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.957+0000 7ff97f400640 1 -- 172.21.15.2:0/2699551638 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff9780463b0 con 0x7ff98807deb0 2024-09-10T07:57:24.750 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.957+0000 7ff97cc00640 1 -- 172.21.15.2:0/2699551638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff98807c3e0 con 0x7ff98807deb0 2024-09-10T07:57:24.750 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:23.960+0000 7ff97f400640 1 -- 172.21.15.2:0/2699551638 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff97803db10 con 0x7ff98807deb0 2024-09-10T07:57:24.750 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:24.026+0000 7ff97cc00640 1 -- 172.21.15.2:0/2699551638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff988062df0 con 0x7ff98807deb0 2024-09-10T07:57:24.750 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:24.026+0000 7ff97f400640 1 -- 172.21.15.2:0/2699551638 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7ff978047440 con 0x7ff98807deb0 2024-09-10T07:57:24.750 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:24.027+0000 7ff97cc00640 1 -- 172.21.15.2:0/2699551638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 msgr2=0x7ff9881a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:24.750 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:24.027+0000 7ff97cc00640 1 --2- 172.21.15.2:0/2699551638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 0x7ff9881a1b40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff9780097f0 tx=0x7ff978030ec0 comp rx=0 tx=0).stop 2024-09-10T07:57:24.750 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:24.027+0000 7ff97cc00640 1 -- 172.21.15.2:0/2699551638 shutdown_connections 2024-09-10T07:57:24.750 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:24.027+0000 7ff97cc00640 1 --2- 172.21.15.2:0/2699551638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff98807deb0 0x7ff9881a1b40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:24.750 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:24.027+0000 7ff97cc00640 1 -- 172.21.15.2:0/2699551638 >> 172.21.15.2:0/2699551638 conn(0x7ff988102d10 msgr2=0x7ff988105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:24.751 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:24.028+0000 7ff97cc00640 1 -- 172.21.15.2:0/2699551638 shutdown_connections 2024-09-10T07:57:24.751 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:24.028+0000 7ff97cc00640 1 -- 172.21.15.2:0/2699551638 wait complete. 2024-09-10T07:57:24.751 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (2/15)... 2024-09-10T07:57:24.850 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:24.586+0000 7f80bb9da040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T07:57:24.850 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:24.667+0000 7f80bb9da040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T07:57:24.850 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:24.750+0000 7f80bb9da040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T07:57:24.851 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:24.850+0000 7f80bb9da040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T07:57:25.269 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:24.923+0000 7f80bb9da040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T07:57:25.270 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:24.996+0000 7f80bb9da040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T07:57:26.270 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:25 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:25.945+0000 7f80bb9da040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T07:57:26.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:26 smithi002 ceph-mon[30563]: Activating manager daemon x 2024-09-10T07:57:26.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:26 smithi002 ceph-mon[30563]: mgrmap e2: x(active, starting, since 0.00402898s) 2024-09-10T07:57:26.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:26 smithi002 ceph-mon[30563]: from='mgr.14100 172.21.15.2:0/979222190' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T07:57:26.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:26 smithi002 ceph-mon[30563]: from='mgr.14100 172.21.15.2:0/979222190' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T07:57:26.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:26 smithi002 ceph-mon[30563]: from='mgr.14100 172.21.15.2:0/979222190' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T07:57:26.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:26 smithi002 ceph-mon[30563]: from='mgr.14100 172.21.15.2:0/979222190' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:57:26.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:26 smithi002 ceph-mon[30563]: from='mgr.14100 172.21.15.2:0/979222190' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T07:57:26.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:26 smithi002 ceph-mon[30563]: Manager daemon x is now available 2024-09-10T07:57:26.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:26 smithi002 ceph-mon[30563]: from='mgr.14100 172.21.15.2:0/979222190' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T07:57:26.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:26 smithi002 ceph-mon[30563]: from='mgr.14100 172.21.15.2:0/979222190' entity='mgr.x' 2024-09-10T07:57:26.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:26 smithi002 ceph-mon[30563]: from='mgr.14100 172.21.15.2:0/979222190' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T07:57:26.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:26 smithi002 ceph-mon[30563]: from='mgr.14100 172.21.15.2:0/979222190' entity='mgr.x' 2024-09-10T07:57:26.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:26 smithi002 ceph-mon[30563]: from='mgr.14100 172.21.15.2:0/979222190' entity='mgr.x' 2024-09-10T07:57:28.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:27 smithi002 ceph-mon[30563]: mgrmap e3: x(active, since 1.01067s) 2024-09-10T07:57:28.142 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:27 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3142441164' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T07:57:28.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-09-10T07:57:28.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:57:28.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "26ec64b0-6f4a-11ef-bcea-c7b262605968", 2024-09-10T07:57:28.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T07:57:28.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T07:57:28.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T07:57:28.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T07:57:28.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:28.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T07:57:28.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T07:57:28.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-09-10T07:57:28.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:57:28.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T07:57:28.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T07:57:28.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:57:28.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-10T07:57:28.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T07:57:28.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:57:28.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T07:57:28.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T07:57:28.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:28.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T07:57:28.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:57:28.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T07:57:28.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T07:57:28.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T07:57:28.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T07:57:28.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T07:57:28.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T07:57:28.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:28.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T07:57:28.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T07:57:28.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T07:57:28.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T07:57:28.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T07:57:28.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T07:57:28.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T07:57:28.159 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T07:57:28.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T07:57:28.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:28.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T07:57:28.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:57:28.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T07:57:11:261417+0000", 2024-09-10T07:57:28.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T07:57:28.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T07:57:28.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:28.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T07:57:28.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T07:57:28.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T07:57:28.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T07:57:28.160 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T07:57:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T07:57:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T07:57:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-09-10T07:57:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T07:57:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T07:57:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T07:57:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T07:57:11.262177+0000", 2024-09-10T07:57:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T07:57:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-09-10T07:57:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T07:57:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:57:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.224+0000 7f5a37e00640 1 Processor -- start 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.224+0000 7f5a37e00640 1 -- start start 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.224+0000 7f5a37e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 0x7f5a30109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.224+0000 7f5a37e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3010a2b0 con 0x7f5a301098e0 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.224+0000 7f5a35400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 0x7f5a30109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.225+0000 7f5a35400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 0x7f5a30109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43088/0 (socket says 172.21.15.2:43088) 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.225+0000 7f5a35400640 1 -- 172.21.15.2:0/3246542466 learned_addr learned my addr 172.21.15.2:0/3246542466 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.225+0000 7f5a35400640 1 -- 172.21.15.2:0/3246542466 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3010aae0 con 0x7f5a301098e0 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.226+0000 7f5a35400640 1 --2- 172.21.15.2:0/3246542466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 0x7f5a30109ce0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009820 tx=0x7f5a2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264bdc25e3957deb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.226+0000 7f5a2fe00640 1 -- 172.21.15.2:0/3246542466 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a24037070 con 0x7f5a301098e0 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.226+0000 7f5a2fe00640 1 -- 172.21.15.2:0/3246542466 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5a24032900 con 0x7f5a301098e0 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.226+0000 7f5a2fe00640 1 -- 172.21.15.2:0/3246542466 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a24030850 con 0x7f5a301098e0 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.227+0000 7f5a37e00640 1 -- 172.21.15.2:0/3246542466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 msgr2=0x7f5a30109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.227+0000 7f5a37e00640 1 --2- 172.21.15.2:0/3246542466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 0x7f5a30109ce0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009820 tx=0x7f5a2402a270 comp rx=0 tx=0).stop 2024-09-10T07:57:28.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.227+0000 7f5a37e00640 1 -- 172.21.15.2:0/3246542466 shutdown_connections 2024-09-10T07:57:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.227+0000 7f5a37e00640 1 --2- 172.21.15.2:0/3246542466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 0x7f5a30109ce0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.227+0000 7f5a37e00640 1 -- 172.21.15.2:0/3246542466 >> 172.21.15.2:0/3246542466 conn(0x7f5a3007ceb0 msgr2=0x7f5a3007d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.227+0000 7f5a37e00640 1 -- 172.21.15.2:0/3246542466 shutdown_connections 2024-09-10T07:57:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.227+0000 7f5a37e00640 1 -- 172.21.15.2:0/3246542466 wait complete. 2024-09-10T07:57:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.228+0000 7f5a37e00640 1 Processor -- start 2024-09-10T07:57:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.228+0000 7f5a37e00640 1 -- start start 2024-09-10T07:57:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.229+0000 7f5a37e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 0x7f5a3019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.229+0000 7f5a37e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3010de90 con 0x7f5a301098e0 2024-09-10T07:57:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.229+0000 7f5a35400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 0x7f5a3019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.229+0000 7f5a35400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 0x7f5a3019d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43100/0 (socket says 172.21.15.2:43100) 2024-09-10T07:57:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.229+0000 7f5a35400640 1 -- 172.21.15.2:0/3142441164 learned_addr learned my addr 172.21.15.2:0/3142441164 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.229+0000 7f5a35400640 1 -- 172.21.15.2:0/3142441164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3019dec0 con 0x7f5a301098e0 2024-09-10T07:57:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.229+0000 7f5a35400640 1 --2- 172.21.15.2:0/3142441164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 0x7f5a3019d980 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009940 tx=0x7f5a24032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.230+0000 7f5a2e000640 1 -- 172.21.15.2:0/3142441164 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a24042070 con 0x7f5a301098e0 2024-09-10T07:57:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.230+0000 7f5a2e000640 1 -- 172.21.15.2:0/3142441164 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5a240308c0 con 0x7f5a301098e0 2024-09-10T07:57:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.230+0000 7f5a2e000640 1 -- 172.21.15.2:0/3142441164 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a24037040 con 0x7f5a301098e0 2024-09-10T07:57:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.230+0000 7f5a37e00640 1 -- 172.21.15.2:0/3142441164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3019e150 con 0x7f5a301098e0 2024-09-10T07:57:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.230+0000 7f5a37e00640 1 -- 172.21.15.2:0/3142441164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a301a0e40 con 0x7f5a301098e0 2024-09-10T07:57:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.231+0000 7f5a2e000640 1 -- 172.21.15.2:0/3142441164 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f5a24045070 con 0x7f5a301098e0 2024-09-10T07:57:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.231+0000 7f5a37e00640 1 -- 172.21.15.2:0/3142441164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a3010e010 con 0x7f5a301098e0 2024-09-10T07:57:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.231+0000 7f5a2e000640 1 --2- 172.21.15.2:0/3142441164 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f5a18038e60 0x7f5a1803b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.232+0000 7f5a2e000640 1 -- 172.21.15.2:0/3142441164 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5a2406bdc0 con 0x7f5a301098e0 2024-09-10T07:57:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.232+0000 7f5a34a00640 1 --2- 172.21.15.2:0/3142441164 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f5a18038e60 0x7f5a1803b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.232+0000 7f5a34a00640 1 --2- 172.21.15.2:0/3142441164 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f5a18038e60 0x7f5a1803b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0098c0 tx=0x7f5a1c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.238+0000 7f5a2e000640 1 -- 172.21.15.2:0/3142441164 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a2404b170 con 0x7f5a301098e0 2024-09-10T07:57:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.494+0000 7f5a37e00640 1 -- 172.21.15.2:0/3142441164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5a301a1010 con 0x7f5a301098e0 2024-09-10T07:57:28.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.495+0000 7f5a2e000640 1 -- 172.21.15.2:0/3142441164 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f5a2402fd90 con 0x7f5a301098e0 2024-09-10T07:57:28.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.497+0000 7f5a37e00640 1 -- 172.21.15.2:0/3142441164 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f5a18038e60 msgr2=0x7f5a1803b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:28.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.497+0000 7f5a37e00640 1 --2- 172.21.15.2:0/3142441164 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f5a18038e60 0x7f5a1803b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0098c0 tx=0x7f5a1c006db0 comp rx=0 tx=0).stop 2024-09-10T07:57:28.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.497+0000 7f5a37e00640 1 -- 172.21.15.2:0/3142441164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 msgr2=0x7f5a3019d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:28.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.497+0000 7f5a37e00640 1 --2- 172.21.15.2:0/3142441164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 0x7f5a3019d980 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009940 tx=0x7f5a24032ce0 comp rx=0 tx=0).stop 2024-09-10T07:57:28.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.497+0000 7f5a37e00640 1 -- 172.21.15.2:0/3142441164 shutdown_connections 2024-09-10T07:57:28.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.497+0000 7f5a37e00640 1 --2- 172.21.15.2:0/3142441164 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f5a18038e60 0x7f5a1803b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:28.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.497+0000 7f5a37e00640 1 --2- 172.21.15.2:0/3142441164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a301098e0 0x7f5a3019d980 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:28.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.497+0000 7f5a37e00640 1 -- 172.21.15.2:0/3142441164 >> 172.21.15.2:0/3142441164 conn(0x7f5a3007ceb0 msgr2=0x7f5a301075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:28.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.498+0000 7f5a37e00640 1 -- 172.21.15.2:0/3142441164 shutdown_connections 2024-09-10T07:57:28.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:27.498+0000 7f5a37e00640 1 -- 172.21.15.2:0/3142441164 wait complete. 2024-09-10T07:57:28.165 INFO:teuthology.orchestra.run.smithi002.stdout:mgr is available 2024-09-10T07:57:29.105 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:28 smithi002 ceph-mon[30563]: mgrmap e4: x(active, since 2s) 2024-09-10T07:57:29.106 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:28 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/7060518' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T07:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-09-10T07:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T07:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2024-09-10T07:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T07:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T07:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T07:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T07:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-09-10T07:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T07:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T07:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-09-10T07:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T07:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T07:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T07:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T07:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T07:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T07:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.593+0000 7f041f400640 1 Processor -- start 2024-09-10T07:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.593+0000 7f041f400640 1 -- start start 2024-09-10T07:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.594+0000 7f041f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 0x7f0418107ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.594+0000 7f041f400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0418108080 con 0x7f04181076b0 2024-09-10T07:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.594+0000 7f041ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 0x7f0418107ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.594+0000 7f041ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 0x7f0418107ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43116/0 (socket says 172.21.15.2:43116) 2024-09-10T07:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.594+0000 7f041ca00640 1 -- 172.21.15.2:0/2976825545 learned_addr learned my addr 172.21.15.2:0/2976825545 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.594+0000 7f041ca00640 1 -- 172.21.15.2:0/2976825545 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181088b0 con 0x7f04181076b0 2024-09-10T07:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.595+0000 7f041ca00640 1 --2- 172.21.15.2:0/2976825545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 0x7f0418107ab0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0408009820 tx=0x7f040802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1c36517bf5d1ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.596+0000 7f0417e00640 1 -- 172.21.15.2:0/2976825545 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0408037070 con 0x7f04181076b0 2024-09-10T07:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.596+0000 7f0417e00640 1 -- 172.21.15.2:0/2976825545 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0408032900 con 0x7f04181076b0 2024-09-10T07:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.596+0000 7f0417e00640 1 -- 172.21.15.2:0/2976825545 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0408030850 con 0x7f04181076b0 2024-09-10T07:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.596+0000 7f041f400640 1 -- 172.21.15.2:0/2976825545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 msgr2=0x7f0418107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.596+0000 7f041f400640 1 --2- 172.21.15.2:0/2976825545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 0x7f0418107ab0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0408009820 tx=0x7f040802a270 comp rx=0 tx=0).stop 2024-09-10T07:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.596+0000 7f041f400640 1 -- 172.21.15.2:0/2976825545 shutdown_connections 2024-09-10T07:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.596+0000 7f041f400640 1 --2- 172.21.15.2:0/2976825545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 0x7f0418107ab0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.596+0000 7f041f400640 1 -- 172.21.15.2:0/2976825545 >> 172.21.15.2:0/2976825545 conn(0x7f0418102e40 msgr2=0x7f0418105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.597+0000 7f041f400640 1 -- 172.21.15.2:0/2976825545 shutdown_connections 2024-09-10T07:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.597+0000 7f041f400640 1 -- 172.21.15.2:0/2976825545 wait complete. 2024-09-10T07:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.597+0000 7f041f400640 1 Processor -- start 2024-09-10T07:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.598+0000 7f041f400640 1 -- start start 2024-09-10T07:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.598+0000 7f041f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 0x7f041819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.598+0000 7f041f400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f041810bc60 con 0x7f04181076b0 2024-09-10T07:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.598+0000 7f041ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 0x7f041819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.598+0000 7f041ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 0x7f041819b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43118/0 (socket says 172.21.15.2:43118) 2024-09-10T07:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.598+0000 7f041ca00640 1 -- 172.21.15.2:0/7060518 learned_addr learned my addr 172.21.15.2:0/7060518 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.598+0000 7f041ca00640 1 -- 172.21.15.2:0/7060518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041819bc80 con 0x7f04181076b0 2024-09-10T07:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.599+0000 7f041ca00640 1 --2- 172.21.15.2:0/7060518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 0x7f041819b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f04080097f0 tx=0x7f0408030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.599+0000 7f0416000640 1 -- 172.21.15.2:0/7060518 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0408042070 con 0x7f04181076b0 2024-09-10T07:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.599+0000 7f0416000640 1 -- 172.21.15.2:0/7060518 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0408032ca0 con 0x7f04181076b0 2024-09-10T07:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.599+0000 7f0416000640 1 -- 172.21.15.2:0/7060518 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0408037040 con 0x7f04181076b0 2024-09-10T07:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.599+0000 7f041f400640 1 -- 172.21.15.2:0/7060518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041819bf10 con 0x7f04181076b0 2024-09-10T07:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.599+0000 7f041f400640 1 -- 172.21.15.2:0/7060518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041819ec00 con 0x7f04181076b0 2024-09-10T07:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.600+0000 7f0416000640 1 -- 172.21.15.2:0/7060518 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f0408045070 con 0x7f04181076b0 2024-09-10T07:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.600+0000 7f041f400640 1 -- 172.21.15.2:0/7060518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f041810bde0 con 0x7f04181076b0 2024-09-10T07:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.601+0000 7f0416000640 1 --2- 172.21.15.2:0/7060518 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f03fc038f70 0x7f03fc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.601+0000 7f0416000640 1 -- 172.21.15.2:0/7060518 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f040806be50 con 0x7f04181076b0 2024-09-10T07:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.601+0000 7f040fe00640 1 --2- 172.21.15.2:0/7060518 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f03fc038f70 0x7f03fc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.601+0000 7f040fe00640 1 --2- 172.21.15.2:0/7060518 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f03fc038f70 0x7f03fc03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f04000098c0 tx=0x7f0400006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.606+0000 7f0416000640 1 -- 172.21.15.2:0/7060518 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f040804b170 con 0x7f04181076b0 2024-09-10T07:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.790+0000 7f041f400640 1 -- 172.21.15.2:0/7060518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0418107b30 con 0x7f04181076b0 2024-09-10T07:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.791+0000 7f0416000640 1 -- 172.21.15.2:0/7060518 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+427 (secure 0 0 0) 0x7f040806b870 con 0x7f04181076b0 2024-09-10T07:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.793+0000 7f041f400640 1 -- 172.21.15.2:0/7060518 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f03fc038f70 msgr2=0x7f03fc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.793+0000 7f041f400640 1 --2- 172.21.15.2:0/7060518 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f03fc038f70 0x7f03fc03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f04000098c0 tx=0x7f0400006db0 comp rx=0 tx=0).stop 2024-09-10T07:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.793+0000 7f041f400640 1 -- 172.21.15.2:0/7060518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 msgr2=0x7f041819b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.793+0000 7f041f400640 1 --2- 172.21.15.2:0/7060518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 0x7f041819b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f04080097f0 tx=0x7f0408030ec0 comp rx=0 tx=0).stop 2024-09-10T07:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.793+0000 7f041f400640 1 -- 172.21.15.2:0/7060518 shutdown_connections 2024-09-10T07:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.793+0000 7f041f400640 1 --2- 172.21.15.2:0/7060518 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f03fc038f70 0x7f03fc03b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.793+0000 7f041f400640 1 --2- 172.21.15.2:0/7060518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04181076b0 0x7f041819b740 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.793+0000 7f041f400640 1 -- 172.21.15.2:0/7060518 >> 172.21.15.2:0/7060518 conn(0x7f0418102e40 msgr2=0x7f0418103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.793+0000 7f041f400640 1 -- 172.21.15.2:0/7060518 shutdown_connections 2024-09-10T07:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:28.794+0000 7f041f400640 1 -- 172.21.15.2:0/7060518 wait complete. 2024-09-10T07:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling cephadm module... 2024-09-10T07:57:30.507 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:30 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2465997078' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T07:57:30.508 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:30 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: ignoring --setuser ceph since I am not root 2024-09-10T07:57:30.508 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:30 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: ignoring --setgroup ceph since I am not root 2024-09-10T07:57:30.508 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:30 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:30.508+0000 7f1e7bc90040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T07:57:30.769 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:30 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:30.584+0000 7f1e7bc90040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T07:57:30.988 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.987+0000 7fb794c00640 1 Processor -- start 2024-09-10T07:57:30.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.987+0000 7fb794c00640 1 -- start start 2024-09-10T07:57:30.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.987+0000 7fb794c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 0x7fb790109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:30.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.987+0000 7fb794c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb79010a2b0 con 0x7fb7901098e0 2024-09-10T07:57:30.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.988+0000 7fb78de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 0x7fb790109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:30.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.988+0000 7fb78de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 0x7fb790109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43126/0 (socket says 172.21.15.2:43126) 2024-09-10T07:57:30.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.988+0000 7fb78de00640 1 -- 172.21.15.2:0/3279107265 learned_addr learned my addr 172.21.15.2:0/3279107265 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:30.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.988+0000 7fb78de00640 1 -- 172.21.15.2:0/3279107265 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79010aae0 con 0x7fb7901098e0 2024-09-10T07:57:30.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.989+0000 7fb78de00640 1 --2- 172.21.15.2:0/3279107265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 0x7fb790109ce0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb780009e80 tx=0x7fb78002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3226648aa48ca646 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:30.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.989+0000 7fb78d400640 1 -- 172.21.15.2:0/3279107265 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb780036070 con 0x7fb7901098e0 2024-09-10T07:57:30.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.989+0000 7fb78d400640 1 -- 172.21.15.2:0/3279107265 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb7800318c0 con 0x7fb7901098e0 2024-09-10T07:57:30.989 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.989+0000 7fb78d400640 1 -- 172.21.15.2:0/3279107265 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb780007ab0 con 0x7fb7901098e0 2024-09-10T07:57:30.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.990+0000 7fb794c00640 1 -- 172.21.15.2:0/3279107265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 msgr2=0x7fb790109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:30.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.990+0000 7fb794c00640 1 --2- 172.21.15.2:0/3279107265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 0x7fb790109ce0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb780009e80 tx=0x7fb78002a7a0 comp rx=0 tx=0).stop 2024-09-10T07:57:30.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.990+0000 7fb794c00640 1 -- 172.21.15.2:0/3279107265 shutdown_connections 2024-09-10T07:57:30.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.990+0000 7fb794c00640 1 --2- 172.21.15.2:0/3279107265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 0x7fb790109ce0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:30.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.990+0000 7fb794c00640 1 -- 172.21.15.2:0/3279107265 >> 172.21.15.2:0/3279107265 conn(0x7fb79007ceb0 msgr2=0x7fb79007d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:30.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.990+0000 7fb794c00640 1 -- 172.21.15.2:0/3279107265 shutdown_connections 2024-09-10T07:57:30.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.990+0000 7fb794c00640 1 -- 172.21.15.2:0/3279107265 wait complete. 2024-09-10T07:57:30.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.991+0000 7fb794c00640 1 Processor -- start 2024-09-10T07:57:30.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.991+0000 7fb794c00640 1 -- start start 2024-09-10T07:57:30.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.991+0000 7fb794c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 0x7fb79019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:30.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.991+0000 7fb794c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb79010de90 con 0x7fb7901098e0 2024-09-10T07:57:30.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.991+0000 7fb78de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 0x7fb79019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.992+0000 7fb78de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 0x7fb79019d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43138/0 (socket says 172.21.15.2:43138) 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.992+0000 7fb78de00640 1 -- 172.21.15.2:0/2465997078 learned_addr learned my addr 172.21.15.2:0/2465997078 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.992+0000 7fb78de00640 1 -- 172.21.15.2:0/2465997078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79019dec0 con 0x7fb7901098e0 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.992+0000 7fb78de00640 1 --2- 172.21.15.2:0/2465997078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 0x7fb79019d980 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb78002acc0 tx=0x7fb780031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.992+0000 7fb787400640 1 -- 172.21.15.2:0/2465997078 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb780036050 con 0x7fb7901098e0 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.992+0000 7fb787400640 1 -- 172.21.15.2:0/2465997078 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb780037040 con 0x7fb7901098e0 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.992+0000 7fb787400640 1 -- 172.21.15.2:0/2465997078 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb78003d8c0 con 0x7fb7901098e0 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.992+0000 7fb794c00640 1 -- 172.21.15.2:0/2465997078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79019e150 con 0x7fb7901098e0 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.993+0000 7fb794c00640 1 -- 172.21.15.2:0/2465997078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7901a0e40 con 0x7fb7901098e0 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.994+0000 7fb787400640 1 -- 172.21.15.2:0/2465997078 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fb780044020 con 0x7fb7901098e0 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.994+0000 7fb794c00640 1 -- 172.21.15.2:0/2465997078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb79010e010 con 0x7fb7901098e0 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.994+0000 7fb787400640 1 --2- 172.21.15.2:0/2465997078 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fb774038f70 0x7fb77403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.994+0000 7fb787400640 1 -- 172.21.15.2:0/2465997078 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb780042020 con 0x7fb7901098e0 2024-09-10T07:57:30.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.994+0000 7fb785400640 1 --2- 172.21.15.2:0/2465997078 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fb774038f70 0x7fb77403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:30.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.995+0000 7fb785400640 1 --2- 172.21.15.2:0/2465997078 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fb774038f70 0x7fb77403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb7780098c0 tx=0x7fb778006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:30.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:29.999+0000 7fb787400640 1 -- 172.21.15.2:0/2465997078 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb78004b020 con 0x7fb7901098e0 2024-09-10T07:57:30.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:30.231+0000 7fb794c00640 1 -- 172.21.15.2:0/2465997078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fb7901a1010 con 0x7fb7901098e0 2024-09-10T07:57:30.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:30.283+0000 7fb787400640 1 -- 172.21.15.2:0/2465997078 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fb78003bb30 con 0x7fb7901098e0 2024-09-10T07:57:30.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:30.284+0000 7fb787400640 1 -- 172.21.15.2:0/2465997078 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fb78003b610 con 0x7fb7901098e0 2024-09-10T07:57:30.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:30.286+0000 7fb794c00640 1 -- 172.21.15.2:0/2465997078 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fb774038f70 msgr2=0x7fb77403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:30.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:30.286+0000 7fb794c00640 1 --2- 172.21.15.2:0/2465997078 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fb774038f70 0x7fb77403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb7780098c0 tx=0x7fb778006db0 comp rx=0 tx=0).stop 2024-09-10T07:57:30.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:30.286+0000 7fb794c00640 1 -- 172.21.15.2:0/2465997078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 msgr2=0x7fb79019d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:30.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:30.286+0000 7fb794c00640 1 --2- 172.21.15.2:0/2465997078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 0x7fb79019d980 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb78002acc0 tx=0x7fb780031ca0 comp rx=0 tx=0).stop 2024-09-10T07:57:30.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:30.286+0000 7fb794c00640 1 -- 172.21.15.2:0/2465997078 shutdown_connections 2024-09-10T07:57:30.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:30.286+0000 7fb794c00640 1 --2- 172.21.15.2:0/2465997078 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fb774038f70 0x7fb77403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:30.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:30.286+0000 7fb794c00640 1 --2- 172.21.15.2:0/2465997078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7901098e0 0x7fb79019d980 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:30.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:30.286+0000 7fb794c00640 1 -- 172.21.15.2:0/2465997078 >> 172.21.15.2:0/2465997078 conn(0x7fb79007ceb0 msgr2=0x7fb7901075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:30.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:30.287+0000 7fb794c00640 1 -- 172.21.15.2:0/2465997078 shutdown_connections 2024-09-10T07:57:30.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:30.287+0000 7fb794c00640 1 -- 172.21.15.2:0/2465997078 wait complete. 2024-09-10T07:57:31.049 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:30 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:30.786+0000 7f1e7bc90040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T07:57:31.049 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:30 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:30.869+0000 7f1e7bc90040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T07:57:31.049 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:31.049+0000 7f1e7bc90040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:57:31.309 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:31 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2465997078' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T07:57:31.310 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:31 smithi002 ceph-mon[30563]: mgrmap e5: x(active, since 4s) 2024-09-10T07:57:31.310 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:31.220+0000 7f1e7bc90040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T07:57:31.909 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:31.613+0000 7f1e7bc90040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T07:57:31.910 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:31.687+0000 7f1e7bc90040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T07:57:31.910 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:31.910+0000 7f1e7bc90040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T07:57:32.510 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:32 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2618152232' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T07:57:32.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:57:32.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T07:57:32.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T07:57:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T07:57:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T07:57:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:57:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.540+0000 7fbdf8000640 1 Processor -- start 2024-09-10T07:57:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.540+0000 7fbdf8000640 1 -- start start 2024-09-10T07:57:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.540+0000 7fbdf8000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 0x7fbdf0109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.540+0000 7fbdf8000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf010a2b0 con 0x7fbdf01098e0 2024-09-10T07:57:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.541+0000 7fbdf5600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 0x7fbdf0109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.541+0000 7fbdf5600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 0x7fbdf0109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43152/0 (socket says 172.21.15.2:43152) 2024-09-10T07:57:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.541+0000 7fbdf5600640 1 -- 172.21.15.2:0/1529380106 learned_addr learned my addr 172.21.15.2:0/1529380106 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.541+0000 7fbdf5600640 1 -- 172.21.15.2:0/1529380106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf010aae0 con 0x7fbdf01098e0 2024-09-10T07:57:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.542+0000 7fbdf5600640 1 --2- 172.21.15.2:0/1529380106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 0x7fbdf0109ce0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009820 tx=0x7fbde402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a490fe2e43fdb94a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:32.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.542+0000 7fbdefe00640 1 -- 172.21.15.2:0/1529380106 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde4037070 con 0x7fbdf01098e0 2024-09-10T07:57:32.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.542+0000 7fbdefe00640 1 -- 172.21.15.2:0/1529380106 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbde4032900 con 0x7fbdf01098e0 2024-09-10T07:57:32.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.542+0000 7fbdefe00640 1 -- 172.21.15.2:0/1529380106 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde4030850 con 0x7fbdf01098e0 2024-09-10T07:57:32.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.543+0000 7fbdf8000640 1 -- 172.21.15.2:0/1529380106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 msgr2=0x7fbdf0109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:32.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.543+0000 7fbdf8000640 1 --2- 172.21.15.2:0/1529380106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 0x7fbdf0109ce0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009820 tx=0x7fbde402a270 comp rx=0 tx=0).stop 2024-09-10T07:57:32.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.543+0000 7fbdf8000640 1 -- 172.21.15.2:0/1529380106 shutdown_connections 2024-09-10T07:57:32.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.543+0000 7fbdf8000640 1 --2- 172.21.15.2:0/1529380106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 0x7fbdf0109ce0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:32.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.543+0000 7fbdf8000640 1 -- 172.21.15.2:0/1529380106 >> 172.21.15.2:0/1529380106 conn(0x7fbdf007ceb0 msgr2=0x7fbdf007d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:32.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.543+0000 7fbdf8000640 1 -- 172.21.15.2:0/1529380106 shutdown_connections 2024-09-10T07:57:32.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.543+0000 7fbdf8000640 1 -- 172.21.15.2:0/1529380106 wait complete. 2024-09-10T07:57:32.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.544+0000 7fbdf8000640 1 Processor -- start 2024-09-10T07:57:32.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.544+0000 7fbdf8000640 1 -- start start 2024-09-10T07:57:32.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.545+0000 7fbdf8000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 0x7fbdf019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:32.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.545+0000 7fbdf8000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf010de90 con 0x7fbdf01098e0 2024-09-10T07:57:32.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.545+0000 7fbdf5600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 0x7fbdf019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:32.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.545+0000 7fbdf5600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 0x7fbdf019d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43158/0 (socket says 172.21.15.2:43158) 2024-09-10T07:57:32.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.545+0000 7fbdf5600640 1 -- 172.21.15.2:0/2618152232 learned_addr learned my addr 172.21.15.2:0/2618152232 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:32.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.545+0000 7fbdf5600640 1 -- 172.21.15.2:0/2618152232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf019dec0 con 0x7fbdf01098e0 2024-09-10T07:57:32.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.546+0000 7fbdf5600640 1 --2- 172.21.15.2:0/2618152232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 0x7fbdf019d980 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbde40097f0 tx=0x7fbde4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:32.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.546+0000 7fbdee000640 1 -- 172.21.15.2:0/2618152232 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde4037040 con 0x7fbdf01098e0 2024-09-10T07:57:32.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.546+0000 7fbdee000640 1 -- 172.21.15.2:0/2618152232 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbde4040070 con 0x7fbdf01098e0 2024-09-10T07:57:32.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.546+0000 7fbdf8000640 1 -- 172.21.15.2:0/2618152232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf019e150 con 0x7fbdf01098e0 2024-09-10T07:57:32.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.546+0000 7fbdee000640 1 -- 172.21.15.2:0/2618152232 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde403a390 con 0x7fbdf01098e0 2024-09-10T07:57:32.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.546+0000 7fbdf8000640 1 -- 172.21.15.2:0/2618152232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf01a0e40 con 0x7fbdf01098e0 2024-09-10T07:57:32.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.547+0000 7fbdee000640 1 -- 172.21.15.2:0/2618152232 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fbde403b430 con 0x7fbdf01098e0 2024-09-10T07:57:32.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.547+0000 7fbdf8000640 1 -- 172.21.15.2:0/2618152232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf0109d60 con 0x7fbdf01098e0 2024-09-10T07:57:32.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.548+0000 7fbdee000640 1 --2- 172.21.15.2:0/2618152232 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fbdd8038f70 0x7fbdd803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:32.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.548+0000 7fbdee000640 1 -- 172.21.15.2:0/2618152232 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbde406c120 con 0x7fbdf01098e0 2024-09-10T07:57:32.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.548+0000 7fbdf4c00640 1 -- 172.21.15.2:0/2618152232 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fbdd8038f70 msgr2=0x7fbdd803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/712393349 2024-09-10T07:57:32.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.548+0000 7fbdf4c00640 1 --2- 172.21.15.2:0/2618152232 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fbdd8038f70 0x7fbdd803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:57:32.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.557+0000 7fbdee000640 1 -- 172.21.15.2:0/2618152232 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbde406be80 con 0x7fbdf01098e0 2024-09-10T07:57:32.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.749+0000 7fbdf4c00640 1 -- 172.21.15.2:0/2618152232 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fbdd8038f70 msgr2=0x7fbdd803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/712393349 2024-09-10T07:57:32.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.749+0000 7fbdf4c00640 1 --2- 172.21.15.2:0/2618152232 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fbdd8038f70 0x7fbdd803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T07:57:32.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.789+0000 7fbdf8000640 1 -- 172.21.15.2:0/2618152232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fbdf01a1010 con 0x7fbdf01098e0 2024-09-10T07:57:32.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.789+0000 7fbdee000640 1 -- 172.21.15.2:0/2618152232 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fbde4032900 con 0x7fbdf01098e0 2024-09-10T07:57:32.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.791+0000 7fbdf8000640 1 -- 172.21.15.2:0/2618152232 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fbdd8038f70 msgr2=0x7fbdd803b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:57:32.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.791+0000 7fbdf8000640 1 --2- 172.21.15.2:0/2618152232 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fbdd8038f70 0x7fbdd803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:32.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.791+0000 7fbdf8000640 1 -- 172.21.15.2:0/2618152232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 msgr2=0x7fbdf019d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:32.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.791+0000 7fbdf8000640 1 --2- 172.21.15.2:0/2618152232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 0x7fbdf019d980 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbde40097f0 tx=0x7fbde4030d80 comp rx=0 tx=0).stop 2024-09-10T07:57:32.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.792+0000 7fbdf8000640 1 -- 172.21.15.2:0/2618152232 shutdown_connections 2024-09-10T07:57:32.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.792+0000 7fbdf8000640 1 --2- 172.21.15.2:0/2618152232 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7fbdd8038f70 0x7fbdd803b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:32.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.792+0000 7fbdf8000640 1 --2- 172.21.15.2:0/2618152232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdf01098e0 0x7fbdf019d980 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:32.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.792+0000 7fbdf8000640 1 -- 172.21.15.2:0/2618152232 >> 172.21.15.2:0/2618152232 conn(0x7fbdf007ceb0 msgr2=0x7fbdf01075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:32.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.792+0000 7fbdf8000640 1 -- 172.21.15.2:0/2618152232 shutdown_connections 2024-09-10T07:57:32.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:31.792+0000 7fbdf8000640 1 -- 172.21.15.2:0/2618152232 wait complete. 2024-09-10T07:57:32.526 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2024-09-10T07:57:32.526 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 5... 2024-09-10T07:57:33.519 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:33 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:33.069+0000 7f1e7bc90040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T07:57:33.520 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:33 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:33.152+0000 7f1e7bc90040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T07:57:34.405 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:34 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:34.095+0000 7f1e7bc90040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T07:57:34.406 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:34 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:34.237+0000 7f1e7bc90040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T07:57:34.406 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:34 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:34.406+0000 7f1e7bc90040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:57:34.657 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:34 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:34.481+0000 7f1e7bc90040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T07:57:34.657 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:34 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:34.657+0000 7f1e7bc90040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T07:57:35.002 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:34 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:34.742+0000 7f1e7bc90040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T07:57:35.002 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:34 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:34.816+0000 7f1e7bc90040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T07:57:35.002 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:35 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:35.002+0000 7f1e7bc90040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T07:57:35.263 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:35 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:35.082+0000 7f1e7bc90040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T07:57:35.264 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:35 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:35.164+0000 7f1e7bc90040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T07:57:35.264 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:35 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:35.264+0000 7f1e7bc90040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T07:57:35.519 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:35 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:35.336+0000 7f1e7bc90040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T07:57:35.520 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:35 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:35.408+0000 7f1e7bc90040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T07:57:36.769 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:36.356+0000 7f1e7bc90040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T07:57:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: Active manager daemon x restarted 2024-09-10T07:57:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: Activating manager daemon x 2024-09-10T07:57:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T07:57:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: mgrmap e6: x(active, starting, since 0.00406042s) 2024-09-10T07:57:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:57:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T07:57:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T07:57:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T07:57:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T07:57:36.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: Manager daemon x is now available 2024-09-10T07:57:36.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: Found migration_current of "None". Setting to last migration. 2024-09-10T07:57:36.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' 2024-09-10T07:57:36.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' 2024-09-10T07:57:36.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:57:36.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:36 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:57:37.716 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:37 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T07:57:37.716 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:37 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T07:57:37.716 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:37 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' 2024-09-10T07:57:37.716 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:37 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' 2024-09-10T07:57:37.716 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:37 smithi002 ceph-mon[30563]: mgrmap e7: x(active, since 1.00666s) 2024-09-10T07:57:37.716 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:37 smithi002 ceph-mon[30563]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T07:57:37.717 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:37 smithi002 ceph-mon[30563]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T07:57:38.369 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:57:38.369 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T07:57:38.369 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T07:57:38.370 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:57:38.370 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.040+0000 7f8cab200640 1 Processor -- start 2024-09-10T07:57:38.370 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.041+0000 7f8cab200640 1 -- start start 2024-09-10T07:57:38.370 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.041+0000 7f8cab200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 0x7f8ca4109d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:38.370 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.041+0000 7f8cab200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410a300 con 0x7f8ca4109930 2024-09-10T07:57:38.370 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.041+0000 7f8ca3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 0x7f8ca4109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:38.370 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.041+0000 7f8ca3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 0x7f8ca4109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43160/0 (socket says 172.21.15.2:43160) 2024-09-10T07:57:38.370 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.041+0000 7f8ca3e00640 1 -- 172.21.15.2:0/3848145051 learned_addr learned my addr 172.21.15.2:0/3848145051 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:38.371 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.042+0000 7f8ca3e00640 1 -- 172.21.15.2:0/3848145051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca410ab00 con 0x7f8ca4109930 2024-09-10T07:57:38.371 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.043+0000 7f8ca3e00640 1 --2- 172.21.15.2:0/3848145051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 0x7f8ca4109d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009820 tx=0x7f8c9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac37a10e3f9b1ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:38.371 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.043+0000 7f8ca2a00640 1 -- 172.21.15.2:0/3848145051 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c94037070 con 0x7f8ca4109930 2024-09-10T07:57:38.371 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.043+0000 7f8ca2a00640 1 -- 172.21.15.2:0/3848145051 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c94032900 con 0x7f8ca4109930 2024-09-10T07:57:38.371 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.043+0000 7f8ca2a00640 1 -- 172.21.15.2:0/3848145051 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c94030850 con 0x7f8ca4109930 2024-09-10T07:57:38.371 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.044+0000 7f8cab200640 1 -- 172.21.15.2:0/3848145051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 msgr2=0x7f8ca4109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:38.371 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.044+0000 7f8cab200640 1 --2- 172.21.15.2:0/3848145051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 0x7f8ca4109d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009820 tx=0x7f8c9402a270 comp rx=0 tx=0).stop 2024-09-10T07:57:38.371 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.044+0000 7f8cab200640 1 -- 172.21.15.2:0/3848145051 shutdown_connections 2024-09-10T07:57:38.371 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.044+0000 7f8cab200640 1 --2- 172.21.15.2:0/3848145051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 0x7f8ca4109d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:38.371 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.044+0000 7f8cab200640 1 -- 172.21.15.2:0/3848145051 >> 172.21.15.2:0/3848145051 conn(0x7f8ca407cf30 msgr2=0x7f8ca407d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:38.371 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.044+0000 7f8cab200640 1 -- 172.21.15.2:0/3848145051 shutdown_connections 2024-09-10T07:57:38.371 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.044+0000 7f8cab200640 1 -- 172.21.15.2:0/3848145051 wait complete. 2024-09-10T07:57:38.372 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.045+0000 7f8cab200640 1 Processor -- start 2024-09-10T07:57:38.372 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.046+0000 7f8cab200640 1 -- start start 2024-09-10T07:57:38.372 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.046+0000 7f8cab200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 0x7f8ca419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:38.372 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.046+0000 7f8cab200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410deb0 con 0x7f8ca4109930 2024-09-10T07:57:38.372 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.046+0000 7f8ca3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 0x7f8ca419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:38.372 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.046+0000 7f8ca3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 0x7f8ca419d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43172/0 (socket says 172.21.15.2:43172) 2024-09-10T07:57:38.372 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.046+0000 7f8ca3e00640 1 -- 172.21.15.2:0/3822516036 learned_addr learned my addr 172.21.15.2:0/3822516036 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:38.372 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.047+0000 7f8ca3e00640 1 -- 172.21.15.2:0/3822516036 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca419de70 con 0x7f8ca4109930 2024-09-10T07:57:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.047+0000 7f8ca3e00640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 0x7f8ca419d930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8c94030460 tx=0x7f8c94030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.048+0000 7f8ca0c00640 1 -- 172.21.15.2:0/3822516036 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c94042070 con 0x7f8ca4109930 2024-09-10T07:57:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.048+0000 7f8ca0c00640 1 -- 172.21.15.2:0/3822516036 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c94032be0 con 0x7f8ca4109930 2024-09-10T07:57:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.048+0000 7f8cab200640 1 -- 172.21.15.2:0/3822516036 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca419e100 con 0x7f8ca4109930 2024-09-10T07:57:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.048+0000 7f8ca0c00640 1 -- 172.21.15.2:0/3822516036 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c94037040 con 0x7f8ca4109930 2024-09-10T07:57:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.048+0000 7f8cab200640 1 -- 172.21.15.2:0/3822516036 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca41a0df0 con 0x7f8ca4109930 2024-09-10T07:57:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.049+0000 7f8ca0c00640 1 -- 172.21.15.2:0/3822516036 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f8c94045070 con 0x7f8ca4109930 2024-09-10T07:57:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.049+0000 7f8ca0c00640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 0x7f8c8803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.049+0000 7f8ca0c00640 1 -- 172.21.15.2:0/3822516036 --> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8c8803bb40 con 0x7f8c88038f70 2024-09-10T07:57:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.049+0000 7f8ca0c00640 1 -- 172.21.15.2:0/3822516036 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8c9406be70 con 0x7f8ca4109930 2024-09-10T07:57:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.049+0000 7f8ca3400640 1 -- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 msgr2=0x7f8c8803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/712393349 2024-09-10T07:57:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.049+0000 7f8ca3400640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 0x7f8c8803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:57:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.250+0000 7f8ca3400640 1 -- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 msgr2=0x7f8c8803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/712393349 2024-09-10T07:57:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.250+0000 7f8ca3400640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 0x7f8c8803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T07:57:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.651+0000 7f8ca3400640 1 -- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 msgr2=0x7f8c8803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/712393349 2024-09-10T07:57:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:33.651+0000 7f8ca3400640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 0x7f8c8803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T07:57:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:34.452+0000 7f8ca3400640 1 -- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 msgr2=0x7f8c8803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/712393349 2024-09-10T07:57:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:34.452+0000 7f8ca3400640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 0x7f8c8803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T07:57:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:36.053+0000 7f8ca3400640 1 -- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 msgr2=0x7f8c8803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/712393349 2024-09-10T07:57:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:36.053+0000 7f8ca3400640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 0x7f8c8803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T07:57:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:36.360+0000 7f8ca0c00640 1 -- 172.21.15.2:0/3822516036 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f8c9403d430 con 0x7f8ca4109930 2024-09-10T07:57:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:36.361+0000 7f8ca0c00640 1 -- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 msgr2=0x7f8c8803b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:57:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:36.361+0000 7f8ca0c00640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 0x7f8c8803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.362+0000 7f8ca0c00640 1 -- 172.21.15.2:0/3822516036 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f8c94065d00 con 0x7f8ca4109930 2024-09-10T07:57:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.362+0000 7f8ca0c00640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f8c8803c8a0 0x7f8c8803ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.362+0000 7f8ca0c00640 1 -- 172.21.15.2:0/3822516036 --> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8c9400af30 con 0x7f8c8803c8a0 2024-09-10T07:57:38.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.363+0000 7f8ca3400640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f8c8803c8a0 0x7f8c8803ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:38.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.365+0000 7f8ca3400640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f8c8803c8a0 0x7f8c8803ec90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8c90003980 tx=0x7f8c90009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:38.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.366+0000 7f8ca0c00640 1 -- 172.21.15.2:0/3822516036 <== mgr.14118 v2:172.21.15.2:6800/3590511604 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f8c9400af30 con 0x7f8c8803c8a0 2024-09-10T07:57:38.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.373+0000 7f8cab200640 1 -- 172.21.15.2:0/3822516036 --> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8ca4109d30 con 0x7f8c8803c8a0 2024-09-10T07:57:38.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.373+0000 7f8ca0c00640 1 -- 172.21.15.2:0/3822516036 <== mgr.14118 v2:172.21.15.2:6800/3590511604 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f8ca4109d30 con 0x7f8c8803c8a0 2024-09-10T07:57:38.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.374+0000 7f8cab200640 1 -- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f8c8803c8a0 msgr2=0x7f8c8803ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:38.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.374+0000 7f8cab200640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f8c8803c8a0 0x7f8c8803ec90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8c90003980 tx=0x7f8c90009290 comp rx=0 tx=0).stop 2024-09-10T07:57:38.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.374+0000 7f8cab200640 1 -- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 msgr2=0x7f8ca419d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:38.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.374+0000 7f8cab200640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 0x7f8ca419d930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8c94030460 tx=0x7f8c94030d80 comp rx=0 tx=0).stop 2024-09-10T07:57:38.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.374+0000 7f8cab200640 1 -- 172.21.15.2:0/3822516036 shutdown_connections 2024-09-10T07:57:38.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.374+0000 7f8cab200640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f8c8803c8a0 0x7f8c8803ec90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:38.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.374+0000 7f8cab200640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:6800/712393349,v1:172.21.15.2:6801/712393349] conn(0x7f8c88038f70 0x7f8c8803b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:38.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.374+0000 7f8cab200640 1 --2- 172.21.15.2:0/3822516036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ca4109930 0x7f8ca419d930 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:38.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.374+0000 7f8cab200640 1 -- 172.21.15.2:0/3822516036 >> 172.21.15.2:0/3822516036 conn(0x7f8ca407cf30 msgr2=0x7f8ca41074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:38.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.374+0000 7f8cab200640 1 -- 172.21.15.2:0/3822516036 shutdown_connections 2024-09-10T07:57:38.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:37.374+0000 7f8cab200640 1 -- 172.21.15.2:0/3822516036 wait complete. 2024-09-10T07:57:38.376 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 5 is available 2024-09-10T07:57:38.376 INFO:teuthology.orchestra.run.smithi002.stdout:Setting orchestrator backend to cephadm... 2024-09-10T07:57:39.020 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:38.771+0000 7f1ddb400640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:07:57:38] ENGINE Error in HTTPServer.serve 2024-09-10T07:57:39.020 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: Traceback (most recent call last): 2024-09-10T07:57:39.020 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T07:57:39.021 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: self._connections.run(self.expiration_interval) 2024-09-10T07:57:39.021 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T07:57:39.021 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: self._run(expiration_interval) 2024-09-10T07:57:39.021 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T07:57:39.021 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T07:57:39.021 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T07:57:39.021 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T07:57:39.021 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T07:57:39.021 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: s = self.context.wrap_socket( 2024-09-10T07:57:39.021 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T07:57:39.021 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: return self.sslsocket_class._create( 2024-09-10T07:57:39.021 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T07:57:39.022 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: self.do_handshake() 2024-09-10T07:57:39.022 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T07:57:39.022 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: self._sslobj.do_handshake() 2024-09-10T07:57:39.022 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T07:57:39.022 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:39.705 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:39 smithi002 ceph-mon[30563]: mgrmap e8: x(active, since 2s) 2024-09-10T07:57:39.706 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:39 smithi002 ceph-mon[30563]: [10/Sep/2024:07:57:38] ENGINE Bus STARTING 2024-09-10T07:57:39.706 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:39 smithi002 ceph-mon[30563]: [10/Sep/2024:07:57:38] ENGINE Serving on https://172.21.15.2:7150 2024-09-10T07:57:39.706 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:39 smithi002 ceph-mon[30563]: [10/Sep/2024:07:57:38] ENGINE Error in HTTPServer.serve 2024-09-10T07:57:39.706 INFO:journalctl@ceph.mon.a.smithi002.stdout: Traceback (most recent call last): 2024-09-10T07:57:39.706 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T07:57:39.706 INFO:journalctl@ceph.mon.a.smithi002.stdout: self._connections.run(self.expiration_interval) 2024-09-10T07:57:39.706 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T07:57:39.706 INFO:journalctl@ceph.mon.a.smithi002.stdout: self._run(expiration_interval) 2024-09-10T07:57:39.706 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T07:57:39.706 INFO:journalctl@ceph.mon.a.smithi002.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T07:57:39.706 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi002.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi002.stdout: s = self.context.wrap_socket( 2024-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi002.stdout: return self.sslsocket_class._create( 2024-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi002.stdout: self.do_handshake() 2024-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi002.stdout: self._sslobj.do_handshake() 2024-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi002.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:39 smithi002 ceph-mon[30563]: [10/Sep/2024:07:57:38] ENGINE Serving on http://172.21.15.2:8765 2024-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:39 smithi002 ceph-mon[30563]: [10/Sep/2024:07:57:38] ENGINE Bus STARTED 2024-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:39 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:57:39.708 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:39 smithi002 ceph-mon[30563]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:57:39.708 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:39 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' 2024-09-10T07:57:39.708 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:39 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:57:39.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.829+0000 7fcf9fe00640 1 Processor -- start 2024-09-10T07:57:39.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.830+0000 7fcf9fe00640 1 -- start start 2024-09-10T07:57:39.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.830+0000 7fcf9fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 0x7fcfa0109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:39.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.830+0000 7fcf9fe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa010a2b0 con 0x7fcfa01098e0 2024-09-10T07:57:39.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.830+0000 7fcf9ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 0x7fcfa0109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:39.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.830+0000 7fcf9ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 0x7fcfa0109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42238/0 (socket says 172.21.15.2:42238) 2024-09-10T07:57:39.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.830+0000 7fcf9ea00640 1 -- 172.21.15.2:0/2731713862 learned_addr learned my addr 172.21.15.2:0/2731713862 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:39.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.831+0000 7fcf9ea00640 1 -- 172.21.15.2:0/2731713862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa010a430 con 0x7fcfa01098e0 2024-09-10T07:57:39.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.832+0000 7fcf9ea00640 1 --2- 172.21.15.2:0/2731713862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 0x7fcfa0109ce0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009820 tx=0x7fcf9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af61634807f74be6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:39.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.832+0000 7fcf9e000640 1 -- 172.21.15.2:0/2731713862 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf90037070 con 0x7fcfa01098e0 2024-09-10T07:57:39.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.832+0000 7fcf9e000640 1 -- 172.21.15.2:0/2731713862 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcf90032900 con 0x7fcfa01098e0 2024-09-10T07:57:39.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.832+0000 7fcf9e000640 1 -- 172.21.15.2:0/2731713862 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf90030850 con 0x7fcfa01098e0 2024-09-10T07:57:39.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.833+0000 7fcf9fe00640 1 -- 172.21.15.2:0/2731713862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 msgr2=0x7fcfa0109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:39.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.833+0000 7fcf9fe00640 1 --2- 172.21.15.2:0/2731713862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 0x7fcfa0109ce0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009820 tx=0x7fcf9002a270 comp rx=0 tx=0).stop 2024-09-10T07:57:39.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.833+0000 7fcf9fe00640 1 -- 172.21.15.2:0/2731713862 shutdown_connections 2024-09-10T07:57:39.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.833+0000 7fcf9fe00640 1 --2- 172.21.15.2:0/2731713862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 0x7fcfa0109ce0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:39.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.833+0000 7fcf9fe00640 1 -- 172.21.15.2:0/2731713862 >> 172.21.15.2:0/2731713862 conn(0x7fcfa007cd90 msgr2=0x7fcfa007d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:39.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.833+0000 7fcf9fe00640 1 -- 172.21.15.2:0/2731713862 shutdown_connections 2024-09-10T07:57:39.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.833+0000 7fcf9fe00640 1 -- 172.21.15.2:0/2731713862 wait complete. 2024-09-10T07:57:39.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.834+0000 7fcf9fe00640 1 Processor -- start 2024-09-10T07:57:39.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.834+0000 7fcf9fe00640 1 -- start start 2024-09-10T07:57:39.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.834+0000 7fcf9fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 0x7fcfa019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:39.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.834+0000 7fcf9fe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa010db60 con 0x7fcfa01098e0 2024-09-10T07:57:39.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.835+0000 7fcf9ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 0x7fcfa019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:39.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.835+0000 7fcf9ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 0x7fcfa019d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42240/0 (socket says 172.21.15.2:42240) 2024-09-10T07:57:39.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.835+0000 7fcf9ea00640 1 -- 172.21.15.2:0/660386874 learned_addr learned my addr 172.21.15.2:0/660386874 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:39.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.835+0000 7fcf9ea00640 1 -- 172.21.15.2:0/660386874 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa019dc80 con 0x7fcfa01098e0 2024-09-10T07:57:39.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.835+0000 7fcf9ea00640 1 --2- 172.21.15.2:0/660386874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 0x7fcfa019d740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcf900097f0 tx=0x7fcf90030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:39.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.836+0000 7fcf97400640 1 -- 172.21.15.2:0/660386874 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf90042070 con 0x7fcfa01098e0 2024-09-10T07:57:39.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.836+0000 7fcf97400640 1 -- 172.21.15.2:0/660386874 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcf90032ca0 con 0x7fcfa01098e0 2024-09-10T07:57:39.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.836+0000 7fcf9fe00640 1 -- 172.21.15.2:0/660386874 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa019df10 con 0x7fcfa01098e0 2024-09-10T07:57:39.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.836+0000 7fcf97400640 1 -- 172.21.15.2:0/660386874 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf90037040 con 0x7fcfa01098e0 2024-09-10T07:57:39.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.836+0000 7fcf9fe00640 1 -- 172.21.15.2:0/660386874 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa01a0c00 con 0x7fcfa01098e0 2024-09-10T07:57:39.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.837+0000 7fcf97400640 1 -- 172.21.15.2:0/660386874 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcf90045070 con 0x7fcfa01098e0 2024-09-10T07:57:39.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.837+0000 7fcf9fe00640 1 -- 172.21.15.2:0/660386874 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa0109d60 con 0x7fcfa01098e0 2024-09-10T07:57:39.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.837+0000 7fcf97400640 1 --2- 172.21.15.2:0/660386874 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fcf84038fc0 0x7fcf8403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:39.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.837+0000 7fcf97400640 1 -- 172.21.15.2:0/660386874 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcf9006bfc0 con 0x7fcfa01098e0 2024-09-10T07:57:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.838+0000 7fcf97e00640 1 --2- 172.21.15.2:0/660386874 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fcf84038fc0 0x7fcf8403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.838+0000 7fcf97e00640 1 --2- 172.21.15.2:0/660386874 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fcf84038fc0 0x7fcf8403b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcf880098c0 tx=0x7fcf88006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:38.843+0000 7fcf97400640 1 -- 172.21.15.2:0/660386874 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf9006b910 con 0x7fcfa01098e0 2024-09-10T07:57:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:39.042+0000 7fcf9fe00640 1 -- 172.21.15.2:0/660386874 --> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fcfa0107ce0 con 0x7fcf84038fc0 2024-09-10T07:57:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:39.047+0000 7fcf97400640 1 -- 172.21.15.2:0/660386874 <== mgr.14118 v2:172.21.15.2:6800/3590511604 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcfa0107ce0 con 0x7fcf84038fc0 2024-09-10T07:57:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:39.050+0000 7fcf9fe00640 1 -- 172.21.15.2:0/660386874 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fcf84038fc0 msgr2=0x7fcf8403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:39.050+0000 7fcf9fe00640 1 --2- 172.21.15.2:0/660386874 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fcf84038fc0 0x7fcf8403b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcf880098c0 tx=0x7fcf88006db0 comp rx=0 tx=0).stop 2024-09-10T07:57:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:39.050+0000 7fcf9fe00640 1 -- 172.21.15.2:0/660386874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 msgr2=0x7fcfa019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:39.050+0000 7fcf9fe00640 1 --2- 172.21.15.2:0/660386874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 0x7fcfa019d740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcf900097f0 tx=0x7fcf90030ec0 comp rx=0 tx=0).stop 2024-09-10T07:57:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:39.050+0000 7fcf9fe00640 1 -- 172.21.15.2:0/660386874 shutdown_connections 2024-09-10T07:57:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:39.050+0000 7fcf9fe00640 1 --2- 172.21.15.2:0/660386874 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fcf84038fc0 0x7fcf8403b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:39.050+0000 7fcf9fe00640 1 --2- 172.21.15.2:0/660386874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfa01098e0 0x7fcfa019d740 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:39.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:39.050+0000 7fcf9fe00640 1 -- 172.21.15.2:0/660386874 >> 172.21.15.2:0/660386874 conn(0x7fcfa007cd90 msgr2=0x7fcfa01075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:39.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:39.050+0000 7fcf9fe00640 1 -- 172.21.15.2:0/660386874 shutdown_connections 2024-09-10T07:57:39.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:39.050+0000 7fcf9fe00640 1 -- 172.21.15.2:0/660386874 wait complete. 2024-09-10T07:57:41.090 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T07:57:41.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.264+0000 7f9a04c00640 1 Processor -- start 2024-09-10T07:57:41.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.264+0000 7f9a04c00640 1 -- start start 2024-09-10T07:57:41.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.264+0000 7f9a04c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 0x7f9a0007c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:41.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.264+0000 7f9a04c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0007c930 con 0x7f9a0007deb0 2024-09-10T07:57:41.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.264+0000 7f99ff400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 0x7f9a0007c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:41.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.264+0000 7f99ff400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 0x7f9a0007c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42246/0 (socket says 172.21.15.2:42246) 2024-09-10T07:57:41.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.264+0000 7f99ff400640 1 -- 172.21.15.2:0/1300945867 learned_addr learned my addr 172.21.15.2:0/1300945867 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:41.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.265+0000 7f99ff400640 1 -- 172.21.15.2:0/1300945867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0007cab0 con 0x7f9a0007deb0 2024-09-10T07:57:41.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.266+0000 7f99ff400640 1 --2- 172.21.15.2:0/1300945867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 0x7f9a0007c360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009820 tx=0x7f99f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3bdfe982e5a0c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:41.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.266+0000 7f99fea00640 1 -- 172.21.15.2:0/1300945867 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f0037070 con 0x7f9a0007deb0 2024-09-10T07:57:41.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.266+0000 7f99fea00640 1 -- 172.21.15.2:0/1300945867 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f99f0032900 con 0x7f9a0007deb0 2024-09-10T07:57:41.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.266+0000 7f99fea00640 1 -- 172.21.15.2:0/1300945867 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f0030850 con 0x7f9a0007deb0 2024-09-10T07:57:41.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.267+0000 7f9a04c00640 1 -- 172.21.15.2:0/1300945867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 msgr2=0x7f9a0007c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:41.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.267+0000 7f9a04c00640 1 --2- 172.21.15.2:0/1300945867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 0x7f9a0007c360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009820 tx=0x7f99f002a270 comp rx=0 tx=0).stop 2024-09-10T07:57:41.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.267+0000 7f9a04c00640 1 -- 172.21.15.2:0/1300945867 shutdown_connections 2024-09-10T07:57:41.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.267+0000 7f9a04c00640 1 --2- 172.21.15.2:0/1300945867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 0x7f9a0007c360 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:41.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.267+0000 7f9a04c00640 1 -- 172.21.15.2:0/1300945867 >> 172.21.15.2:0/1300945867 conn(0x7f9a00102d10 msgr2=0x7f9a00105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:41.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.267+0000 7f9a04c00640 1 -- 172.21.15.2:0/1300945867 shutdown_connections 2024-09-10T07:57:41.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.267+0000 7f9a04c00640 1 -- 172.21.15.2:0/1300945867 wait complete. 2024-09-10T07:57:41.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.268+0000 7f9a04c00640 1 Processor -- start 2024-09-10T07:57:41.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.268+0000 7f9a04c00640 1 -- start start 2024-09-10T07:57:41.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.268+0000 7f9a04c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 0x7f9a001a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:41.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.268+0000 7f9a04c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00109740 con 0x7f9a0007deb0 2024-09-10T07:57:41.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.268+0000 7f99ff400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 0x7f9a001a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:41.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.268+0000 7f99ff400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 0x7f9a001a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42256/0 (socket says 172.21.15.2:42256) 2024-09-10T07:57:41.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.268+0000 7f99ff400640 1 -- 172.21.15.2:0/3658441529 learned_addr learned my addr 172.21.15.2:0/3658441529 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:41.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.269+0000 7f99ff400640 1 -- 172.21.15.2:0/3658441529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a001a2080 con 0x7f9a0007deb0 2024-09-10T07:57:41.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.269+0000 7f99ff400640 1 --2- 172.21.15.2:0/3658441529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 0x7f9a001a1b40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99f00097f0 tx=0x7f99f0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:41.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.269+0000 7f99fcc00640 1 -- 172.21.15.2:0/3658441529 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f0042070 con 0x7f9a0007deb0 2024-09-10T07:57:41.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.269+0000 7f99fcc00640 1 -- 172.21.15.2:0/3658441529 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f99f0032ca0 con 0x7f9a0007deb0 2024-09-10T07:57:41.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.269+0000 7f9a04c00640 1 -- 172.21.15.2:0/3658441529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a001a2310 con 0x7f9a0007deb0 2024-09-10T07:57:41.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.269+0000 7f99fcc00640 1 -- 172.21.15.2:0/3658441529 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f0037040 con 0x7f9a0007deb0 2024-09-10T07:57:41.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.270+0000 7f9a04c00640 1 -- 172.21.15.2:0/3658441529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a001a5000 con 0x7f9a0007deb0 2024-09-10T07:57:41.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.271+0000 7f9a04c00640 1 -- 172.21.15.2:0/3658441529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a0007c3e0 con 0x7f9a0007deb0 2024-09-10T07:57:41.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.271+0000 7f99fcc00640 1 -- 172.21.15.2:0/3658441529 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f99f0045070 con 0x7f9a0007deb0 2024-09-10T07:57:41.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.271+0000 7f99fcc00640 1 --2- 172.21.15.2:0/3658441529 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f99e4038fc0 0x7f99e403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:41.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.271+0000 7f99fcc00640 1 -- 172.21.15.2:0/3658441529 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f99f006bfc0 con 0x7f9a0007deb0 2024-09-10T07:57:41.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.271+0000 7f99f7e00640 1 --2- 172.21.15.2:0/3658441529 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f99e4038fc0 0x7f99e403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:41.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.272+0000 7f99f7e00640 1 --2- 172.21.15.2:0/3658441529 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f99e4038fc0 0x7f99e403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f99e80098c0 tx=0x7f99e8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:41.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.276+0000 7f99fcc00640 1 -- 172.21.15.2:0/3658441529 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99f0045360 con 0x7f9a0007deb0 2024-09-10T07:57:41.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.462+0000 7f9a04c00640 1 -- 172.21.15.2:0/3658441529 --> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9a00062630 con 0x7f99e4038fc0 2024-09-10T07:57:41.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.462+0000 7f99fcc00640 1 -- 172.21.15.2:0/3658441529 <== mgr.14118 v2:172.21.15.2:6800/3590511604 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9a00062630 con 0x7f99e4038fc0 2024-09-10T07:57:41.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.464+0000 7f9a04c00640 1 -- 172.21.15.2:0/3658441529 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f99e4038fc0 msgr2=0x7f99e403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:41.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.464+0000 7f9a04c00640 1 --2- 172.21.15.2:0/3658441529 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f99e4038fc0 0x7f99e403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f99e80098c0 tx=0x7f99e8006db0 comp rx=0 tx=0).stop 2024-09-10T07:57:41.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.464+0000 7f9a04c00640 1 -- 172.21.15.2:0/3658441529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 msgr2=0x7f9a001a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:41.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.464+0000 7f9a04c00640 1 --2- 172.21.15.2:0/3658441529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 0x7f9a001a1b40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99f00097f0 tx=0x7f99f0030ec0 comp rx=0 tx=0).stop 2024-09-10T07:57:41.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.465+0000 7f9a04c00640 1 -- 172.21.15.2:0/3658441529 shutdown_connections 2024-09-10T07:57:41.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.465+0000 7f9a04c00640 1 --2- 172.21.15.2:0/3658441529 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f99e4038fc0 0x7f99e403b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:41.096 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.465+0000 7f9a04c00640 1 --2- 172.21.15.2:0/3658441529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a0007deb0 0x7f9a001a1b40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:41.096 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.465+0000 7f9a04c00640 1 -- 172.21.15.2:0/3658441529 >> 172.21.15.2:0/3658441529 conn(0x7f9a00102d10 msgr2=0x7f9a00105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:41.096 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.465+0000 7f9a04c00640 1 -- 172.21.15.2:0/3658441529 shutdown_connections 2024-09-10T07:57:41.096 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:40.465+0000 7f9a04c00640 1 -- 172.21.15.2:0/3658441529 wait complete. 2024-09-10T07:57:41.096 INFO:teuthology.orchestra.run.smithi002.stdout:Generating ssh key... 2024-09-10T07:57:41.584 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:41 smithi002 ceph-mon[30563]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:57:42.272 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: Generating public/private rsa key pair. 2024-09-10T07:57:42.273 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: Your identification has been saved in /tmp/tmpss6labsr/key 2024-09-10T07:57:42.273 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: Your public key has been saved in /tmp/tmpss6labsr/key.pub 2024-09-10T07:57:42.273 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: The key fingerprint is: 2024-09-10T07:57:42.273 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: SHA256:+i0Y9k8+hHYdCllizpeAVypo7fRPSaQVjuoZLpd2at4 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:57:42.273 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: The key's randomart image is: 2024-09-10T07:57:42.273 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: +---[RSA 3072]----+ 2024-09-10T07:57:42.273 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: | . .=. | 2024-09-10T07:57:42.273 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: | o. =B. | 2024-09-10T07:57:42.273 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: | o +=+=o. | 2024-09-10T07:57:42.274 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: | . o +=.o.. | 2024-09-10T07:57:42.274 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: | + S+oo . | 2024-09-10T07:57:42.274 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: | oo=oo+ . | 2024-09-10T07:57:42.274 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: | ..O=.oo | 2024-09-10T07:57:42.274 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: | +o=o+. | 2024-09-10T07:57:42.274 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: | oo Eo+. | 2024-09-10T07:57:42.274 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: +----[SHA256]-----+ 2024-09-10T07:57:42.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.572+0000 7f83f6000640 1 Processor -- start 2024-09-10T07:57:42.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.572+0000 7f83f6000640 1 -- start start 2024-09-10T07:57:42.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.573+0000 7f83f6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 0x7f83f0107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:42.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.573+0000 7f83f6000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83f0108110 con 0x7f83f0107740 2024-09-10T07:57:42.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.573+0000 7f83ef400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 0x7f83f0107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:42.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.573+0000 7f83ef400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 0x7f83f0107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42270/0 (socket says 172.21.15.2:42270) 2024-09-10T07:57:42.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.573+0000 7f83ef400640 1 -- 172.21.15.2:0/1837171879 learned_addr learned my addr 172.21.15.2:0/1837171879 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:42.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.573+0000 7f83ef400640 1 -- 172.21.15.2:0/1837171879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f01088a0 con 0x7f83f0107740 2024-09-10T07:57:42.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.574+0000 7f83ef400640 1 --2- 172.21.15.2:0/1837171879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 0x7f83f0107b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009820 tx=0x7f83e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130ef50629b53aef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:42.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.575+0000 7f83eea00640 1 -- 172.21.15.2:0/1837171879 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83e0037070 con 0x7f83f0107740 2024-09-10T07:57:42.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.575+0000 7f83eea00640 1 -- 172.21.15.2:0/1837171879 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f83e0032900 con 0x7f83f0107740 2024-09-10T07:57:42.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.575+0000 7f83eea00640 1 -- 172.21.15.2:0/1837171879 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83e0030850 con 0x7f83f0107740 2024-09-10T07:57:42.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.575+0000 7f83f6000640 1 -- 172.21.15.2:0/1837171879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 msgr2=0x7f83f0107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:42.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.575+0000 7f83f6000640 1 --2- 172.21.15.2:0/1837171879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 0x7f83f0107b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009820 tx=0x7f83e002a270 comp rx=0 tx=0).stop 2024-09-10T07:57:42.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.575+0000 7f83f6000640 1 -- 172.21.15.2:0/1837171879 shutdown_connections 2024-09-10T07:57:42.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.575+0000 7f83f6000640 1 --2- 172.21.15.2:0/1837171879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 0x7f83f0107b40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:42.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.575+0000 7f83f6000640 1 -- 172.21.15.2:0/1837171879 >> 172.21.15.2:0/1837171879 conn(0x7f83f0102ed0 msgr2=0x7f83f0105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:42.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.575+0000 7f83f6000640 1 -- 172.21.15.2:0/1837171879 shutdown_connections 2024-09-10T07:57:42.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.576+0000 7f83f6000640 1 -- 172.21.15.2:0/1837171879 wait complete. 2024-09-10T07:57:42.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.576+0000 7f83f6000640 1 Processor -- start 2024-09-10T07:57:42.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.577+0000 7f83f6000640 1 -- start start 2024-09-10T07:57:42.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.577+0000 7f83f6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 0x7f83f007db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:42.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.577+0000 7f83f6000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83f010bc50 con 0x7f83f0107740 2024-09-10T07:57:42.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.577+0000 7f83ef400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 0x7f83f007db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:42.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.577+0000 7f83ef400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 0x7f83f007db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42272/0 (socket says 172.21.15.2:42272) 2024-09-10T07:57:42.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.577+0000 7f83ef400640 1 -- 172.21.15.2:0/580276593 learned_addr learned my addr 172.21.15.2:0/580276593 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:42.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.577+0000 7f83ef400640 1 -- 172.21.15.2:0/580276593 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f007e0d0 con 0x7f83f0107740 2024-09-10T07:57:42.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.578+0000 7f83ef400640 1 --2- 172.21.15.2:0/580276593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 0x7f83f007db90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f83e00097f0 tx=0x7f83e0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:42.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.578+0000 7f83ecc00640 1 -- 172.21.15.2:0/580276593 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83e0042070 con 0x7f83f0107740 2024-09-10T07:57:42.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.578+0000 7f83ecc00640 1 -- 172.21.15.2:0/580276593 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f83e0032ca0 con 0x7f83f0107740 2024-09-10T07:57:42.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.578+0000 7f83f6000640 1 -- 172.21.15.2:0/580276593 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f007c2c0 con 0x7f83f0107740 2024-09-10T07:57:42.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.578+0000 7f83ecc00640 1 -- 172.21.15.2:0/580276593 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83e0037040 con 0x7f83f0107740 2024-09-10T07:57:42.673 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.578+0000 7f83f6000640 1 -- 172.21.15.2:0/580276593 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f007c800 con 0x7f83f0107740 2024-09-10T07:57:42.673 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.579+0000 7f83ecc00640 1 -- 172.21.15.2:0/580276593 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f83e0045070 con 0x7f83f0107740 2024-09-10T07:57:42.673 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.579+0000 7f83f6000640 1 -- 172.21.15.2:0/580276593 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f0107bc0 con 0x7f83f0107740 2024-09-10T07:57:42.673 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.580+0000 7f83ecc00640 1 --2- 172.21.15.2:0/580276593 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f83d4038fc0 0x7f83d403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:42.673 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.580+0000 7f83ecc00640 1 -- 172.21.15.2:0/580276593 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f83e006c130 con 0x7f83f0107740 2024-09-10T07:57:42.673 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.580+0000 7f83e7e00640 1 --2- 172.21.15.2:0/580276593 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f83d4038fc0 0x7f83d403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:42.673 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.581+0000 7f83e7e00640 1 --2- 172.21.15.2:0/580276593 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f83d4038fc0 0x7f83d403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f83d80098c0 tx=0x7f83d8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:42.673 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.585+0000 7f83ecc00640 1 -- 172.21.15.2:0/580276593 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83e0045360 con 0x7f83f0107740 2024-09-10T07:57:42.673 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:41.772+0000 7f83f6000640 1 -- 172.21.15.2:0/580276593 --> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f83f0104200 con 0x7f83d4038fc0 2024-09-10T07:57:42.673 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:42.019+0000 7f83ecc00640 1 -- 172.21.15.2:0/580276593 <== mgr.14118 v2:172.21.15.2:6800/3590511604 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f83f0104200 con 0x7f83d4038fc0 2024-09-10T07:57:42.673 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:42.022+0000 7f83f6000640 1 -- 172.21.15.2:0/580276593 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f83d4038fc0 msgr2=0x7f83d403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:42.673 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:42.022+0000 7f83f6000640 1 --2- 172.21.15.2:0/580276593 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f83d4038fc0 0x7f83d403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f83d80098c0 tx=0x7f83d8006db0 comp rx=0 tx=0).stop 2024-09-10T07:57:42.674 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:42.022+0000 7f83f6000640 1 -- 172.21.15.2:0/580276593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 msgr2=0x7f83f007db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:42.674 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:42.022+0000 7f83f6000640 1 --2- 172.21.15.2:0/580276593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 0x7f83f007db90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f83e00097f0 tx=0x7f83e0030ec0 comp rx=0 tx=0).stop 2024-09-10T07:57:42.674 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:42.022+0000 7f83f6000640 1 -- 172.21.15.2:0/580276593 shutdown_connections 2024-09-10T07:57:42.674 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:42.022+0000 7f83f6000640 1 --2- 172.21.15.2:0/580276593 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f83d4038fc0 0x7f83d403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:42.674 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:42.022+0000 7f83f6000640 1 --2- 172.21.15.2:0/580276593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83f0107740 0x7f83f007db90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:42.674 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:42.022+0000 7f83f6000640 1 -- 172.21.15.2:0/580276593 >> 172.21.15.2:0/580276593 conn(0x7f83f0102ed0 msgr2=0x7f83f0103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:42.674 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:42.022+0000 7f83f6000640 1 -- 172.21.15.2:0/580276593 shutdown_connections 2024-09-10T07:57:42.674 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:42.022+0000 7f83f6000640 1 -- 172.21.15.2:0/580276593 wait complete. 2024-09-10T07:57:43.279 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:43 smithi002 ceph-mon[30563]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:57:43.279 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:43 smithi002 ceph-mon[30563]: Generating ssh key... 2024-09-10T07:57:43.279 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:43 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' 2024-09-10T07:57:43.279 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:43 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' 2024-09-10T07:57:43.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp3MSJJbXjGKLV/1XS92rApG62zlBXKbNJcfq12m5Ksfh7SmEd2+xAS3tSKmSbpuTdAzqUTL54hgXydbc9ea+zSX8h8weaOcilwjqxrV7Xe3ZxS+4235t8TvOL09p+m0AmDqTgpQOt1+hARY3P68HUNsyMsXii1JU1og/LqwIDAX9Oslk+MjxxrdZTxj57PuFqpqYUiZoDjoD7LyPFkmiHJggJRjBTyo00vWqPTF3chXYWdHFolbAuP99XIaLo/mG14iBxzAnX0ewI6+rBkIqdTtRUwxmyPJh/aj7F7pcjs4y9jns57wZkU7wZ+cmoFpoXGDMGNWtxECy2UFrnuSzpbYo7tpvjVB+9uuRDhrLoa+BUNLGBt1nYXjlovvaTphsBKT+t3wDDDmP5hgVL3UAGQa07A+zsn7d5OLO4KkyjvDfIvudi3b2QKEZJGz0YUi5OMeIaGA1FxM5N3ztAIRu5Qd5Ur1sip8qdzxA5OgQhFlTwQBUY6oaxLP2yMPKANGU= ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:57:43.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.161+0000 7f2e48a00640 1 Processor -- start 2024-09-10T07:57:43.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.161+0000 7f2e48a00640 1 -- start start 2024-09-10T07:57:43.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.161+0000 7f2e48a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 0x7f2e44109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:43.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.161+0000 7f2e48a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4410a2b0 con 0x7f2e441098e0 2024-09-10T07:57:43.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.161+0000 7f2e41e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 0x7f2e44109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:43.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.162+0000 7f2e41e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 0x7f2e44109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42276/0 (socket says 172.21.15.2:42276) 2024-09-10T07:57:43.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.162+0000 7f2e41e00640 1 -- 172.21.15.2:0/3035232312 learned_addr learned my addr 172.21.15.2:0/3035232312 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:43.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.162+0000 7f2e41e00640 1 -- 172.21.15.2:0/3035232312 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4410aae0 con 0x7f2e441098e0 2024-09-10T07:57:43.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.163+0000 7f2e41e00640 1 --2- 172.21.15.2:0/3035232312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 0x7f2e44109ce0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009820 tx=0x7f2e3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9641e99b299a7f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.163+0000 7f2e41400640 1 -- 172.21.15.2:0/3035232312 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e34037070 con 0x7f2e441098e0 2024-09-10T07:57:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.163+0000 7f2e41400640 1 -- 172.21.15.2:0/3035232312 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2e34032900 con 0x7f2e441098e0 2024-09-10T07:57:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.163+0000 7f2e41400640 1 -- 172.21.15.2:0/3035232312 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e34030850 con 0x7f2e441098e0 2024-09-10T07:57:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.164+0000 7f2e48a00640 1 -- 172.21.15.2:0/3035232312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 msgr2=0x7f2e44109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.164+0000 7f2e48a00640 1 --2- 172.21.15.2:0/3035232312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 0x7f2e44109ce0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009820 tx=0x7f2e3402a270 comp rx=0 tx=0).stop 2024-09-10T07:57:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.164+0000 7f2e48a00640 1 -- 172.21.15.2:0/3035232312 shutdown_connections 2024-09-10T07:57:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.164+0000 7f2e48a00640 1 --2- 172.21.15.2:0/3035232312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 0x7f2e44109ce0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.164+0000 7f2e48a00640 1 -- 172.21.15.2:0/3035232312 >> 172.21.15.2:0/3035232312 conn(0x7f2e4407ceb0 msgr2=0x7f2e4407d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.164+0000 7f2e48a00640 1 -- 172.21.15.2:0/3035232312 shutdown_connections 2024-09-10T07:57:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.164+0000 7f2e48a00640 1 -- 172.21.15.2:0/3035232312 wait complete. 2024-09-10T07:57:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.165+0000 7f2e48a00640 1 Processor -- start 2024-09-10T07:57:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.165+0000 7f2e48a00640 1 -- start start 2024-09-10T07:57:43.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.165+0000 7f2e48a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 0x7f2e4419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:43.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.165+0000 7f2e48a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4410de90 con 0x7f2e441098e0 2024-09-10T07:57:43.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.165+0000 7f2e41e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 0x7f2e4419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:43.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.166+0000 7f2e41e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 0x7f2e4419d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42282/0 (socket says 172.21.15.2:42282) 2024-09-10T07:57:43.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.166+0000 7f2e41e00640 1 -- 172.21.15.2:0/1672287718 learned_addr learned my addr 172.21.15.2:0/1672287718 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:43.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.166+0000 7f2e41e00640 1 -- 172.21.15.2:0/1672287718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4419dec0 con 0x7f2e441098e0 2024-09-10T07:57:43.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.166+0000 7f2e41e00640 1 --2- 172.21.15.2:0/1672287718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 0x7f2e4419d980 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2e340097f0 tx=0x7f2e34030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:43.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.166+0000 7f2e3aa00640 1 -- 172.21.15.2:0/1672287718 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e34037040 con 0x7f2e441098e0 2024-09-10T07:57:43.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.166+0000 7f2e3aa00640 1 -- 172.21.15.2:0/1672287718 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2e34040070 con 0x7f2e441098e0 2024-09-10T07:57:43.995 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.166+0000 7f2e3aa00640 1 -- 172.21.15.2:0/1672287718 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e3403a390 con 0x7f2e441098e0 2024-09-10T07:57:43.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.166+0000 7f2e48a00640 1 -- 172.21.15.2:0/1672287718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4419e150 con 0x7f2e441098e0 2024-09-10T07:57:43.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.167+0000 7f2e48a00640 1 -- 172.21.15.2:0/1672287718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441a0e40 con 0x7f2e441098e0 2024-09-10T07:57:43.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.168+0000 7f2e3aa00640 1 -- 172.21.15.2:0/1672287718 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2e3403b430 con 0x7f2e441098e0 2024-09-10T07:57:43.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.168+0000 7f2e48a00640 1 -- 172.21.15.2:0/1672287718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e44109d60 con 0x7f2e441098e0 2024-09-10T07:57:43.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.168+0000 7f2e3aa00640 1 --2- 172.21.15.2:0/1672287718 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f2e28038f70 0x7f2e2803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:43.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.168+0000 7f2e3aa00640 1 -- 172.21.15.2:0/1672287718 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2e3406c570 con 0x7f2e441098e0 2024-09-10T07:57:43.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.168+0000 7f2e3be00640 1 --2- 172.21.15.2:0/1672287718 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f2e28038f70 0x7f2e2803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:43.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.169+0000 7f2e3be00640 1 --2- 172.21.15.2:0/1672287718 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f2e28038f70 0x7f2e2803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0098c0 tx=0x7f2e2c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:43.996 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.174+0000 7f2e3aa00640 1 -- 172.21.15.2:0/1672287718 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e3406bec0 con 0x7f2e441098e0 2024-09-10T07:57:43.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.363+0000 7f2e48a00640 1 -- 172.21.15.2:0/1672287718 --> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f2e44107cb0 con 0x7f2e28038f70 2024-09-10T07:57:43.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.364+0000 7f2e3aa00640 1 -- 172.21.15.2:0/1672287718 <== mgr.14118 v2:172.21.15.2:6800/3590511604 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f2e44107cb0 con 0x7f2e28038f70 2024-09-10T07:57:43.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.366+0000 7f2e48a00640 1 -- 172.21.15.2:0/1672287718 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f2e28038f70 msgr2=0x7f2e2803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:43.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.366+0000 7f2e48a00640 1 --2- 172.21.15.2:0/1672287718 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f2e28038f70 0x7f2e2803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0098c0 tx=0x7f2e2c006db0 comp rx=0 tx=0).stop 2024-09-10T07:57:43.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.366+0000 7f2e48a00640 1 -- 172.21.15.2:0/1672287718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 msgr2=0x7f2e4419d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:43.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.366+0000 7f2e48a00640 1 --2- 172.21.15.2:0/1672287718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 0x7f2e4419d980 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2e340097f0 tx=0x7f2e34030d80 comp rx=0 tx=0).stop 2024-09-10T07:57:43.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.366+0000 7f2e48a00640 1 -- 172.21.15.2:0/1672287718 shutdown_connections 2024-09-10T07:57:43.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.366+0000 7f2e48a00640 1 --2- 172.21.15.2:0/1672287718 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f2e28038f70 0x7f2e2803b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:43.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.366+0000 7f2e48a00640 1 --2- 172.21.15.2:0/1672287718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e441098e0 0x7f2e4419d980 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:43.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.366+0000 7f2e48a00640 1 -- 172.21.15.2:0/1672287718 >> 172.21.15.2:0/1672287718 conn(0x7f2e4407ceb0 msgr2=0x7f2e441075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:43.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.367+0000 7f2e48a00640 1 -- 172.21.15.2:0/1672287718 shutdown_connections 2024-09-10T07:57:43.997 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:43.367+0000 7f2e48a00640 1 -- 172.21.15.2:0/1672287718 wait complete. 2024-09-10T07:57:43.997 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T07:57:43.998 INFO:teuthology.orchestra.run.smithi002.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T07:57:44.000 INFO:teuthology.orchestra.run.smithi002.stdout:Adding host smithi002... 2024-09-10T07:57:44.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:44 smithi002 ceph-mon[30563]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:57:45.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:45 smithi002 ceph-mon[30563]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:57:46.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:46 smithi002 ceph-mon[30563]: Deploying cephadm binary to smithi002 2024-09-10T07:57:49.919 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Added host 'smithi002' with addr '172.21.15.2' 2024-09-10T07:57:49.919 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.472+0000 7fe20d000640 1 Processor -- start 2024-09-10T07:57:49.919 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.472+0000 7fe20d000640 1 -- start start 2024-09-10T07:57:49.919 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.472+0000 7fe20d000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 0x7fe208109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:49.919 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.472+0000 7fe20d000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe20810a2b0 con 0x7fe2081098e0 2024-09-10T07:57:49.919 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.472+0000 7fe205e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 0x7fe208109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:49.919 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.473+0000 7fe205e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 0x7fe208109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42288/0 (socket says 172.21.15.2:42288) 2024-09-10T07:57:49.920 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.473+0000 7fe205e00640 1 -- 172.21.15.2:0/125081663 learned_addr learned my addr 172.21.15.2:0/125081663 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:49.920 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.473+0000 7fe205e00640 1 -- 172.21.15.2:0/125081663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20810aae0 con 0x7fe2081098e0 2024-09-10T07:57:49.920 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.474+0000 7fe205e00640 1 --2- 172.21.15.2:0/125081663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 0x7fe208109ce0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009820 tx=0x7fe1f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b075ece7da53f542 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:49.920 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.474+0000 7fe204a00640 1 -- 172.21.15.2:0/125081663 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f8037070 con 0x7fe2081098e0 2024-09-10T07:57:49.920 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.474+0000 7fe204a00640 1 -- 172.21.15.2:0/125081663 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1f8032900 con 0x7fe2081098e0 2024-09-10T07:57:49.920 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.474+0000 7fe204a00640 1 -- 172.21.15.2:0/125081663 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f8030850 con 0x7fe2081098e0 2024-09-10T07:57:49.920 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.475+0000 7fe20d000640 1 -- 172.21.15.2:0/125081663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 msgr2=0x7fe208109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:49.920 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.475+0000 7fe20d000640 1 --2- 172.21.15.2:0/125081663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 0x7fe208109ce0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009820 tx=0x7fe1f802a270 comp rx=0 tx=0).stop 2024-09-10T07:57:49.920 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.475+0000 7fe20d000640 1 -- 172.21.15.2:0/125081663 shutdown_connections 2024-09-10T07:57:49.920 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.475+0000 7fe20d000640 1 --2- 172.21.15.2:0/125081663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 0x7fe208109ce0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:49.920 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.475+0000 7fe20d000640 1 -- 172.21.15.2:0/125081663 >> 172.21.15.2:0/125081663 conn(0x7fe20807ceb0 msgr2=0x7fe20807d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:49.920 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.475+0000 7fe20d000640 1 -- 172.21.15.2:0/125081663 shutdown_connections 2024-09-10T07:57:49.921 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.475+0000 7fe20d000640 1 -- 172.21.15.2:0/125081663 wait complete. 2024-09-10T07:57:49.921 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.476+0000 7fe20d000640 1 Processor -- start 2024-09-10T07:57:49.921 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.476+0000 7fe20d000640 1 -- start start 2024-09-10T07:57:49.921 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.476+0000 7fe20d000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 0x7fe20819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:49.921 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.476+0000 7fe20d000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe20810de90 con 0x7fe2081098e0 2024-09-10T07:57:49.921 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.477+0000 7fe205e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 0x7fe20819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:49.921 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.477+0000 7fe205e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 0x7fe20819d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42304/0 (socket says 172.21.15.2:42304) 2024-09-10T07:57:49.921 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.477+0000 7fe205e00640 1 -- 172.21.15.2:0/2108604475 learned_addr learned my addr 172.21.15.2:0/2108604475 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:49.921 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.477+0000 7fe205e00640 1 -- 172.21.15.2:0/2108604475 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20819dec0 con 0x7fe2081098e0 2024-09-10T07:57:49.921 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.477+0000 7fe205e00640 1 --2- 172.21.15.2:0/2108604475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 0x7fe20819d980 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80097f0 tx=0x7fe1f8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:49.921 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.477+0000 7fe1fea00640 1 -- 172.21.15.2:0/2108604475 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f8037040 con 0x7fe2081098e0 2024-09-10T07:57:49.921 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.478+0000 7fe1fea00640 1 -- 172.21.15.2:0/2108604475 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1f8040070 con 0x7fe2081098e0 2024-09-10T07:57:49.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.478+0000 7fe20d000640 1 -- 172.21.15.2:0/2108604475 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20819e150 con 0x7fe2081098e0 2024-09-10T07:57:49.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.478+0000 7fe1fea00640 1 -- 172.21.15.2:0/2108604475 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f803a390 con 0x7fe2081098e0 2024-09-10T07:57:49.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.478+0000 7fe20d000640 1 -- 172.21.15.2:0/2108604475 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2081a0e40 con 0x7fe2081098e0 2024-09-10T07:57:49.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.479+0000 7fe20d000640 1 -- 172.21.15.2:0/2108604475 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe208109d60 con 0x7fe2081098e0 2024-09-10T07:57:49.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.479+0000 7fe1fea00640 1 -- 172.21.15.2:0/2108604475 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe1f803b430 con 0x7fe2081098e0 2024-09-10T07:57:49.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.479+0000 7fe1fea00640 1 --2- 172.21.15.2:0/2108604475 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fe1ec038fc0 0x7fe1ec03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:49.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.479+0000 7fe1fea00640 1 -- 172.21.15.2:0/2108604475 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe1f806c570 con 0x7fe2081098e0 2024-09-10T07:57:49.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.480+0000 7fe205400640 1 --2- 172.21.15.2:0/2108604475 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fe1ec038fc0 0x7fe1ec03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:49.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.480+0000 7fe205400640 1 --2- 172.21.15.2:0/2108604475 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fe1ec038fc0 0x7fe1ec03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40098c0 tx=0x7fe1f4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:49.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.484+0000 7fe1fea00640 1 -- 172.21.15.2:0/2108604475 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1f8032900 con 0x7fe2081098e0 2024-09-10T07:57:49.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:44.672+0000 7fe20d000640 1 -- 172.21.15.2:0/2108604475 --> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}) -- 0x7fe208107cb0 con 0x7fe1ec038fc0 2024-09-10T07:57:49.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:49.192+0000 7fe1fea00640 1 -- 172.21.15.2:0/2108604475 <== mgr.14118 v2:172.21.15.2:6800/3590511604 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fe208107cb0 con 0x7fe1ec038fc0 2024-09-10T07:57:49.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:49.195+0000 7fe20d000640 1 -- 172.21.15.2:0/2108604475 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fe1ec038fc0 msgr2=0x7fe1ec03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:49.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:49.195+0000 7fe20d000640 1 --2- 172.21.15.2:0/2108604475 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fe1ec038fc0 0x7fe1ec03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40098c0 tx=0x7fe1f4006db0 comp rx=0 tx=0).stop 2024-09-10T07:57:49.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:49.195+0000 7fe20d000640 1 -- 172.21.15.2:0/2108604475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 msgr2=0x7fe20819d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:49.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:49.195+0000 7fe20d000640 1 --2- 172.21.15.2:0/2108604475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 0x7fe20819d980 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80097f0 tx=0x7fe1f8030d80 comp rx=0 tx=0).stop 2024-09-10T07:57:49.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:49.195+0000 7fe20d000640 1 -- 172.21.15.2:0/2108604475 shutdown_connections 2024-09-10T07:57:49.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:49.195+0000 7fe20d000640 1 --2- 172.21.15.2:0/2108604475 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fe1ec038fc0 0x7fe1ec03b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:49.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:49.195+0000 7fe20d000640 1 --2- 172.21.15.2:0/2108604475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2081098e0 0x7fe20819d980 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:49.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:49.195+0000 7fe20d000640 1 -- 172.21.15.2:0/2108604475 >> 172.21.15.2:0/2108604475 conn(0x7fe20807ceb0 msgr2=0x7fe2081075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:49.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:49.195+0000 7fe20d000640 1 -- 172.21.15.2:0/2108604475 shutdown_connections 2024-09-10T07:57:49.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:49.195+0000 7fe20d000640 1 -- 172.21.15.2:0/2108604475 wait complete. 2024-09-10T07:57:49.923 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying unmanaged mon service... 2024-09-10T07:57:50.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:50 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' 2024-09-10T07:57:50.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:50 smithi002 ceph-mon[30563]: Added host smithi002 2024-09-10T07:57:50.413 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:50 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:57:51.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T07:57:51.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.400+0000 7ff6f8800640 1 Processor -- start 2024-09-10T07:57:51.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.401+0000 7ff6f8800640 1 -- start start 2024-09-10T07:57:51.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.401+0000 7ff6f8800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 0x7ff6f0109d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:51.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.401+0000 7ff6f8800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f010a350 con 0x7ff6f0109980 2024-09-10T07:57:51.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.401+0000 7ff6f5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 0x7ff6f0109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:51.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.401+0000 7ff6f5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 0x7ff6f0109d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58074/0 (socket says 172.21.15.2:58074) 2024-09-10T07:57:51.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.401+0000 7ff6f5e00640 1 -- 172.21.15.2:0/3181477378 learned_addr learned my addr 172.21.15.2:0/3181477378 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:51.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.402+0000 7ff6f5e00640 1 -- 172.21.15.2:0/3181477378 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f010aaa0 con 0x7ff6f0109980 2024-09-10T07:57:51.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.403+0000 7ff6f5e00640 1 --2- 172.21.15.2:0/3181477378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 0x7ff6f0109d80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009820 tx=0x7ff6e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1246eaaa1295278a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:51.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.403+0000 7ff6f4a00640 1 -- 172.21.15.2:0/3181477378 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6e4037070 con 0x7ff6f0109980 2024-09-10T07:57:51.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.403+0000 7ff6f4a00640 1 -- 172.21.15.2:0/3181477378 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff6e4032900 con 0x7ff6f0109980 2024-09-10T07:57:51.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.403+0000 7ff6f4a00640 1 -- 172.21.15.2:0/3181477378 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6e4030850 con 0x7ff6f0109980 2024-09-10T07:57:51.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.404+0000 7ff6f8800640 1 -- 172.21.15.2:0/3181477378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 msgr2=0x7ff6f0109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:51.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.404+0000 7ff6f8800640 1 --2- 172.21.15.2:0/3181477378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 0x7ff6f0109d80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009820 tx=0x7ff6e402a270 comp rx=0 tx=0).stop 2024-09-10T07:57:51.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.404+0000 7ff6f8800640 1 -- 172.21.15.2:0/3181477378 shutdown_connections 2024-09-10T07:57:51.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.404+0000 7ff6f8800640 1 --2- 172.21.15.2:0/3181477378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 0x7ff6f0109d80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:51.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.404+0000 7ff6f8800640 1 -- 172.21.15.2:0/3181477378 >> 172.21.15.2:0/3181477378 conn(0x7ff6f007ced0 msgr2=0x7ff6f007d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:51.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.404+0000 7ff6f8800640 1 -- 172.21.15.2:0/3181477378 shutdown_connections 2024-09-10T07:57:51.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.404+0000 7ff6f8800640 1 -- 172.21.15.2:0/3181477378 wait complete. 2024-09-10T07:57:51.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.405+0000 7ff6f8800640 1 Processor -- start 2024-09-10T07:57:51.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.405+0000 7ff6f8800640 1 -- start start 2024-09-10T07:57:51.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.405+0000 7ff6f8800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 0x7ff6f019d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:51.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.405+0000 7ff6f8800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f010de50 con 0x7ff6f0109980 2024-09-10T07:57:51.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.406+0000 7ff6f5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 0x7ff6f019d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:51.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.406+0000 7ff6f5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 0x7ff6f019d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58088/0 (socket says 172.21.15.2:58088) 2024-09-10T07:57:51.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.406+0000 7ff6f5e00640 1 -- 172.21.15.2:0/3610832709 learned_addr learned my addr 172.21.15.2:0/3610832709 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:51.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.406+0000 7ff6f5e00640 1 -- 172.21.15.2:0/3610832709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f019dde0 con 0x7ff6f0109980 2024-09-10T07:57:51.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.406+0000 7ff6f5e00640 1 --2- 172.21.15.2:0/3610832709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 0x7ff6f019d8a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40097f0 tx=0x7ff6e4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:51.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.407+0000 7ff6eaa00640 1 -- 172.21.15.2:0/3610832709 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6e4037040 con 0x7ff6f0109980 2024-09-10T07:57:51.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.407+0000 7ff6eaa00640 1 -- 172.21.15.2:0/3610832709 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff6e4040070 con 0x7ff6f0109980 2024-09-10T07:57:51.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.407+0000 7ff6eaa00640 1 -- 172.21.15.2:0/3610832709 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6e403a390 con 0x7ff6f0109980 2024-09-10T07:57:51.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.407+0000 7ff6f8800640 1 -- 172.21.15.2:0/3610832709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f019e070 con 0x7ff6f0109980 2024-09-10T07:57:51.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.407+0000 7ff6f8800640 1 -- 172.21.15.2:0/3610832709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f01a0d60 con 0x7ff6f0109980 2024-09-10T07:57:51.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.408+0000 7ff6eaa00640 1 -- 172.21.15.2:0/3610832709 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff6e403b430 con 0x7ff6f0109980 2024-09-10T07:57:51.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.408+0000 7ff6f8800640 1 -- 172.21.15.2:0/3610832709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f0109e00 con 0x7ff6f0109980 2024-09-10T07:57:51.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.408+0000 7ff6eaa00640 1 --2- 172.21.15.2:0/3610832709 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7ff6d8038fc0 0x7ff6d803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:51.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.409+0000 7ff6eaa00640 1 -- 172.21.15.2:0/3610832709 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff6e406c4e0 con 0x7ff6f0109980 2024-09-10T07:57:51.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.409+0000 7ff6f5400640 1 --2- 172.21.15.2:0/3610832709 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7ff6d8038fc0 0x7ff6d803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:51.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.410+0000 7ff6f5400640 1 --2- 172.21.15.2:0/3610832709 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7ff6d8038fc0 0x7ff6d803b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00098c0 tx=0x7ff6e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:51.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.414+0000 7ff6eaa00640 1 -- 172.21.15.2:0/3610832709 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff6e406be30 con 0x7ff6f0109980 2024-09-10T07:57:51.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.617+0000 7ff6f8800640 1 -- 172.21.15.2:0/3610832709 --> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff6f0107c50 con 0x7ff6d8038fc0 2024-09-10T07:57:51.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.622+0000 7ff6eaa00640 1 -- 172.21.15.2:0/3610832709 <== mgr.14118 v2:172.21.15.2:6800/3590511604 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff6f0107c50 con 0x7ff6d8038fc0 2024-09-10T07:57:51.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.624+0000 7ff6f8800640 1 -- 172.21.15.2:0/3610832709 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7ff6d8038fc0 msgr2=0x7ff6d803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:51.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.624+0000 7ff6f8800640 1 --2- 172.21.15.2:0/3610832709 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7ff6d8038fc0 0x7ff6d803b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00098c0 tx=0x7ff6e0006db0 comp rx=0 tx=0).stop 2024-09-10T07:57:51.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.624+0000 7ff6f8800640 1 -- 172.21.15.2:0/3610832709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 msgr2=0x7ff6f019d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:51.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.624+0000 7ff6f8800640 1 --2- 172.21.15.2:0/3610832709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 0x7ff6f019d8a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40097f0 tx=0x7ff6e4030d80 comp rx=0 tx=0).stop 2024-09-10T07:57:51.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.624+0000 7ff6f8800640 1 -- 172.21.15.2:0/3610832709 shutdown_connections 2024-09-10T07:57:51.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.624+0000 7ff6f8800640 1 --2- 172.21.15.2:0/3610832709 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7ff6d8038fc0 0x7ff6d803b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:51.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.624+0000 7ff6f8800640 1 --2- 172.21.15.2:0/3610832709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6f0109980 0x7ff6f019d8a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:51.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.624+0000 7ff6f8800640 1 -- 172.21.15.2:0/3610832709 >> 172.21.15.2:0/3610832709 conn(0x7ff6f007ced0 msgr2=0x7ff6f0107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:51.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.625+0000 7ff6f8800640 1 -- 172.21.15.2:0/3610832709 shutdown_connections 2024-09-10T07:57:51.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:50.625+0000 7ff6f8800640 1 -- 172.21.15.2:0/3610832709 wait complete. 2024-09-10T07:57:51.508 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying unmanaged mgr service... 2024-09-10T07:57:51.767 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:51 smithi002 ceph-mon[30563]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:57:51.768 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:51 smithi002 ceph-mon[30563]: Saving service mon spec with placement count:5 2024-09-10T07:57:51.768 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:51 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' 2024-09-10T07:57:52.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T07:57:52.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.965+0000 7fb0f5800640 1 Processor -- start 2024-09-10T07:57:52.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.965+0000 7fb0f5800640 1 -- start start 2024-09-10T07:57:52.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.965+0000 7fb0f5800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 0x7fb0f0109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:52.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.965+0000 7fb0f5800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f010a2b0 con 0x7fb0f01098e0 2024-09-10T07:57:52.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.965+0000 7fb0efe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 0x7fb0f0109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:52.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.965+0000 7fb0efe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 0x7fb0f0109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58092/0 (socket says 172.21.15.2:58092) 2024-09-10T07:57:52.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.965+0000 7fb0efe00640 1 -- 172.21.15.2:0/2453152414 learned_addr learned my addr 172.21.15.2:0/2453152414 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:52.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.966+0000 7fb0efe00640 1 -- 172.21.15.2:0/2453152414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f010a430 con 0x7fb0f01098e0 2024-09-10T07:57:52.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.967+0000 7fb0efe00640 1 --2- 172.21.15.2:0/2453152414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 0x7fb0f0109ce0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009e80 tx=0x7fb0e002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3585c0846a2e2e00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:52.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.967+0000 7fb0ef400640 1 -- 172.21.15.2:0/2453152414 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0e0036070 con 0x7fb0f01098e0 2024-09-10T07:57:52.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.967+0000 7fb0ef400640 1 -- 172.21.15.2:0/2453152414 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb0e00318c0 con 0x7fb0f01098e0 2024-09-10T07:57:52.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.967+0000 7fb0ef400640 1 -- 172.21.15.2:0/2453152414 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0e0007ab0 con 0x7fb0f01098e0 2024-09-10T07:57:52.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.968+0000 7fb0f5800640 1 -- 172.21.15.2:0/2453152414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 msgr2=0x7fb0f0109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:52.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.968+0000 7fb0f5800640 1 --2- 172.21.15.2:0/2453152414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 0x7fb0f0109ce0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009e80 tx=0x7fb0e002a7a0 comp rx=0 tx=0).stop 2024-09-10T07:57:52.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.968+0000 7fb0f5800640 1 -- 172.21.15.2:0/2453152414 shutdown_connections 2024-09-10T07:57:52.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.968+0000 7fb0f5800640 1 --2- 172.21.15.2:0/2453152414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 0x7fb0f0109ce0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:52.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.968+0000 7fb0f5800640 1 -- 172.21.15.2:0/2453152414 >> 172.21.15.2:0/2453152414 conn(0x7fb0f007cd90 msgr2=0x7fb0f007d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:52.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.968+0000 7fb0f5800640 1 -- 172.21.15.2:0/2453152414 shutdown_connections 2024-09-10T07:57:52.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.968+0000 7fb0f5800640 1 -- 172.21.15.2:0/2453152414 wait complete. 2024-09-10T07:57:52.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.969+0000 7fb0f5800640 1 Processor -- start 2024-09-10T07:57:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.969+0000 7fb0f5800640 1 -- start start 2024-09-10T07:57:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.969+0000 7fb0f5800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 0x7fb0f019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.969+0000 7fb0f5800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f010db60 con 0x7fb0f01098e0 2024-09-10T07:57:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.969+0000 7fb0efe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 0x7fb0f019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.969+0000 7fb0efe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 0x7fb0f019d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58098/0 (socket says 172.21.15.2:58098) 2024-09-10T07:57:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.969+0000 7fb0efe00640 1 -- 172.21.15.2:0/3999333957 learned_addr learned my addr 172.21.15.2:0/3999333957 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.970+0000 7fb0efe00640 1 -- 172.21.15.2:0/3999333957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f019dc80 con 0x7fb0f01098e0 2024-09-10T07:57:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.970+0000 7fb0efe00640 1 --2- 172.21.15.2:0/3999333957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 0x7fb0f019d740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb0e002acc0 tx=0x7fb0e0031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.970+0000 7fb0ed600640 1 -- 172.21.15.2:0/3999333957 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0e0036050 con 0x7fb0f01098e0 2024-09-10T07:57:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.970+0000 7fb0ed600640 1 -- 172.21.15.2:0/3999333957 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb0e0037040 con 0x7fb0f01098e0 2024-09-10T07:57:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.970+0000 7fb0ed600640 1 -- 172.21.15.2:0/3999333957 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0e003d920 con 0x7fb0f01098e0 2024-09-10T07:57:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.970+0000 7fb0f5800640 1 -- 172.21.15.2:0/3999333957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f019df10 con 0x7fb0f01098e0 2024-09-10T07:57:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.971+0000 7fb0f5800640 1 -- 172.21.15.2:0/3999333957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f01a0c00 con 0x7fb0f01098e0 2024-09-10T07:57:52.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.971+0000 7fb0ed600640 1 -- 172.21.15.2:0/3999333957 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb0e0044020 con 0x7fb0f01098e0 2024-09-10T07:57:52.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.972+0000 7fb0f5800640 1 -- 172.21.15.2:0/3999333957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f0109d60 con 0x7fb0f01098e0 2024-09-10T07:57:52.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.972+0000 7fb0ed600640 1 --2- 172.21.15.2:0/3999333957 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fb0d4038f70 0x7fb0d403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:52.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.972+0000 7fb0ed600640 1 -- 172.21.15.2:0/3999333957 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb0e0042020 con 0x7fb0f01098e0 2024-09-10T07:57:52.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.972+0000 7fb0e7400640 1 --2- 172.21.15.2:0/3999333957 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fb0d4038f70 0x7fb0d403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:52.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.973+0000 7fb0e7400640 1 --2- 172.21.15.2:0/3999333957 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fb0d4038f70 0x7fb0d403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80098c0 tx=0x7fb0d8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:52.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:51.977+0000 7fb0ed600640 1 -- 172.21.15.2:0/3999333957 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0e004b020 con 0x7fb0f01098e0 2024-09-10T07:57:52.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:52.187+0000 7fb0f5800640 1 -- 172.21.15.2:0/3999333957 --> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb0f0107ce0 con 0x7fb0d4038f70 2024-09-10T07:57:52.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:52.192+0000 7fb0ed600640 1 -- 172.21.15.2:0/3999333957 <== mgr.14118 v2:172.21.15.2:6800/3590511604 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb0f0107ce0 con 0x7fb0d4038f70 2024-09-10T07:57:52.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:52.194+0000 7fb0f5800640 1 -- 172.21.15.2:0/3999333957 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fb0d4038f70 msgr2=0x7fb0d403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:52.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:52.194+0000 7fb0f5800640 1 --2- 172.21.15.2:0/3999333957 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fb0d4038f70 0x7fb0d403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80098c0 tx=0x7fb0d8006db0 comp rx=0 tx=0).stop 2024-09-10T07:57:52.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:52.194+0000 7fb0f5800640 1 -- 172.21.15.2:0/3999333957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 msgr2=0x7fb0f019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:52.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:52.194+0000 7fb0f5800640 1 --2- 172.21.15.2:0/3999333957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 0x7fb0f019d740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb0e002acc0 tx=0x7fb0e0031ca0 comp rx=0 tx=0).stop 2024-09-10T07:57:52.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:52.194+0000 7fb0f5800640 1 -- 172.21.15.2:0/3999333957 shutdown_connections 2024-09-10T07:57:52.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:52.194+0000 7fb0f5800640 1 --2- 172.21.15.2:0/3999333957 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fb0d4038f70 0x7fb0d403b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:52.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:52.194+0000 7fb0f5800640 1 --2- 172.21.15.2:0/3999333957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f01098e0 0x7fb0f019d740 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:52.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:52.194+0000 7fb0f5800640 1 -- 172.21.15.2:0/3999333957 >> 172.21.15.2:0/3999333957 conn(0x7fb0f007cd90 msgr2=0x7fb0f01075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:52.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:52.194+0000 7fb0f5800640 1 -- 172.21.15.2:0/3999333957 shutdown_connections 2024-09-10T07:57:52.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:52.194+0000 7fb0f5800640 1 -- 172.21.15.2:0/3999333957 wait complete. 2024-09-10T07:57:53.239 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:53 smithi002 ceph-mon[30563]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:57:53.239 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:53 smithi002 ceph-mon[30563]: Saving service mgr spec with placement count:2 2024-09-10T07:57:53.239 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:53 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' 2024-09-10T07:57:54.715 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.645+0000 7f1249200640 1 Processor -- start 2024-09-10T07:57:54.715 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.645+0000 7f1249200640 1 -- start start 2024-09-10T07:57:54.715 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.646+0000 7f1249200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 0x7f1244107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:54.715 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.646+0000 7f1249200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1244108110 con 0x7f1244107740 2024-09-10T07:57:54.715 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.646+0000 7f1241e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 0x7f1244107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:54.715 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.646+0000 7f1241e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 0x7f1244107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58100/0 (socket says 172.21.15.2:58100) 2024-09-10T07:57:54.715 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.646+0000 7f1241e00640 1 -- 172.21.15.2:0/2208095058 learned_addr learned my addr 172.21.15.2:0/2208095058 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:54.715 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.647+0000 7f1241e00640 1 -- 172.21.15.2:0/2208095058 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12441088a0 con 0x7f1244107740 2024-09-10T07:57:54.715 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.648+0000 7f1241e00640 1 --2- 172.21.15.2:0/2208095058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 0x7f1244107b40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1234009820 tx=0x7f123402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84baec7d850ee878 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:54.715 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.648+0000 7f1241400640 1 -- 172.21.15.2:0/2208095058 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1234037070 con 0x7f1244107740 2024-09-10T07:57:54.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.648+0000 7f1241400640 1 -- 172.21.15.2:0/2208095058 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1234032900 con 0x7f1244107740 2024-09-10T07:57:54.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.648+0000 7f1241400640 1 -- 172.21.15.2:0/2208095058 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1234030850 con 0x7f1244107740 2024-09-10T07:57:54.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.648+0000 7f1249200640 1 -- 172.21.15.2:0/2208095058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 msgr2=0x7f1244107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:54.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.648+0000 7f1249200640 1 --2- 172.21.15.2:0/2208095058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 0x7f1244107b40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1234009820 tx=0x7f123402a270 comp rx=0 tx=0).stop 2024-09-10T07:57:54.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.649+0000 7f1249200640 1 -- 172.21.15.2:0/2208095058 shutdown_connections 2024-09-10T07:57:54.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.649+0000 7f1249200640 1 --2- 172.21.15.2:0/2208095058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 0x7f1244107b40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:54.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.649+0000 7f1249200640 1 -- 172.21.15.2:0/2208095058 >> 172.21.15.2:0/2208095058 conn(0x7f1244102ed0 msgr2=0x7f1244105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:54.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.649+0000 7f1249200640 1 -- 172.21.15.2:0/2208095058 shutdown_connections 2024-09-10T07:57:54.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.649+0000 7f1249200640 1 -- 172.21.15.2:0/2208095058 wait complete. 2024-09-10T07:57:54.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.650+0000 7f1249200640 1 Processor -- start 2024-09-10T07:57:54.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.650+0000 7f1249200640 1 -- start start 2024-09-10T07:57:54.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.650+0000 7f1249200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 0x7f124419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:54.716 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.650+0000 7f1249200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f124410bc50 con 0x7f1244107740 2024-09-10T07:57:54.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.650+0000 7f1241e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 0x7f124419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:54.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.650+0000 7f1241e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 0x7f124419d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58108/0 (socket says 172.21.15.2:58108) 2024-09-10T07:57:54.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.650+0000 7f1241e00640 1 -- 172.21.15.2:0/3109799701 learned_addr learned my addr 172.21.15.2:0/3109799701 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:54.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.651+0000 7f1241e00640 1 -- 172.21.15.2:0/3109799701 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124419de80 con 0x7f1244107740 2024-09-10T07:57:54.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.651+0000 7f1241e00640 1 --2- 172.21.15.2:0/3109799701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 0x7f124419d940 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1234009940 tx=0x7f12340326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:54.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.651+0000 7f123b400640 1 -- 172.21.15.2:0/3109799701 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1234042070 con 0x7f1244107740 2024-09-10T07:57:54.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.651+0000 7f123b400640 1 -- 172.21.15.2:0/3109799701 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1234032be0 con 0x7f1244107740 2024-09-10T07:57:54.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.651+0000 7f123b400640 1 -- 172.21.15.2:0/3109799701 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1234037040 con 0x7f1244107740 2024-09-10T07:57:54.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.651+0000 7f1249200640 1 -- 172.21.15.2:0/3109799701 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124419e110 con 0x7f1244107740 2024-09-10T07:57:54.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.651+0000 7f1249200640 1 -- 172.21.15.2:0/3109799701 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12441a0e00 con 0x7f1244107740 2024-09-10T07:57:54.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.652+0000 7f123b400640 1 -- 172.21.15.2:0/3109799701 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1234045070 con 0x7f1244107740 2024-09-10T07:57:54.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.652+0000 7f1249200640 1 -- 172.21.15.2:0/3109799701 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f124410bdd0 con 0x7f1244107740 2024-09-10T07:57:54.717 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.653+0000 7f123b400640 1 --2- 172.21.15.2:0/3109799701 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f1228038fc0 0x7f122803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:54.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.653+0000 7f123b400640 1 -- 172.21.15.2:0/3109799701 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f123406bfa0 con 0x7f1244107740 2024-09-10T07:57:54.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.653+0000 7f1239400640 1 --2- 172.21.15.2:0/3109799701 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f1228038fc0 0x7f122803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:54.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.654+0000 7f1239400640 1 --2- 172.21.15.2:0/3109799701 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f1228038fc0 0x7f122803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f122c0098c0 tx=0x7f122c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:54.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.658+0000 7f123b400640 1 -- 172.21.15.2:0/3109799701 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f123404b170 con 0x7f1244107740 2024-09-10T07:57:54.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.845+0000 7f1249200640 1 -- 172.21.15.2:0/3109799701 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f1244107bc0 con 0x7f1244107740 2024-09-10T07:57:54.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.848+0000 7f123b400640 1 -- 172.21.15.2:0/3109799701 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f123406b8f0 con 0x7f1244107740 2024-09-10T07:57:54.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.850+0000 7f1249200640 1 -- 172.21.15.2:0/3109799701 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f1228038fc0 msgr2=0x7f122803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:54.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.850+0000 7f1249200640 1 --2- 172.21.15.2:0/3109799701 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f1228038fc0 0x7f122803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f122c0098c0 tx=0x7f122c006db0 comp rx=0 tx=0).stop 2024-09-10T07:57:54.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.850+0000 7f1249200640 1 -- 172.21.15.2:0/3109799701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 msgr2=0x7f124419d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:54.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.850+0000 7f1249200640 1 --2- 172.21.15.2:0/3109799701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 0x7f124419d940 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1234009940 tx=0x7f12340326e0 comp rx=0 tx=0).stop 2024-09-10T07:57:54.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.851+0000 7f1249200640 1 -- 172.21.15.2:0/3109799701 shutdown_connections 2024-09-10T07:57:54.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.851+0000 7f1249200640 1 --2- 172.21.15.2:0/3109799701 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f1228038fc0 0x7f122803b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:54.718 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.851+0000 7f1249200640 1 --2- 172.21.15.2:0/3109799701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1244107740 0x7f124419d940 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:54.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.851+0000 7f1249200640 1 -- 172.21.15.2:0/3109799701 >> 172.21.15.2:0/3109799701 conn(0x7f1244102ed0 msgr2=0x7f1244103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:54.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.851+0000 7f1249200640 1 -- 172.21.15.2:0/3109799701 shutdown_connections 2024-09-10T07:57:54.719 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:53.851+0000 7f1249200640 1 -- 172.21.15.2:0/3109799701 wait complete. 2024-09-10T07:57:54.983 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:54 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3109799701' entity='client.admin' 2024-09-10T07:57:54.983 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:54 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' 2024-09-10T07:57:56.002 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.211+0000 7ff4d8e00640 1 Processor -- start 2024-09-10T07:57:56.002 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.211+0000 7ff4d8e00640 1 -- start start 2024-09-10T07:57:56.002 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.211+0000 7ff4d8e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 0x7ff4d4109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:56.002 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.211+0000 7ff4d8e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d410a2b0 con 0x7ff4d41098e0 2024-09-10T07:57:56.002 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.211+0000 7ff4d1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 0x7ff4d4109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:56.002 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.212+0000 7ff4d1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 0x7ff4d4109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58116/0 (socket says 172.21.15.2:58116) 2024-09-10T07:57:56.002 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.212+0000 7ff4d1e00640 1 -- 172.21.15.2:0/2858412891 learned_addr learned my addr 172.21.15.2:0/2858412891 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:56.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.212+0000 7ff4d1e00640 1 -- 172.21.15.2:0/2858412891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d410aae0 con 0x7ff4d41098e0 2024-09-10T07:57:56.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.213+0000 7ff4d1e00640 1 --2- 172.21.15.2:0/2858412891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 0x7ff4d4109ce0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009820 tx=0x7ff4c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa0e01b25315504 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:56.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.213+0000 7ff4d1400640 1 -- 172.21.15.2:0/2858412891 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4c4037070 con 0x7ff4d41098e0 2024-09-10T07:57:56.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.213+0000 7ff4d1400640 1 -- 172.21.15.2:0/2858412891 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4c4032920 con 0x7ff4d41098e0 2024-09-10T07:57:56.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.213+0000 7ff4d1400640 1 -- 172.21.15.2:0/2858412891 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4c4030810 con 0x7ff4d41098e0 2024-09-10T07:57:56.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.214+0000 7ff4d8e00640 1 -- 172.21.15.2:0/2858412891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 msgr2=0x7ff4d4109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:56.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.214+0000 7ff4d8e00640 1 --2- 172.21.15.2:0/2858412891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 0x7ff4d4109ce0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009820 tx=0x7ff4c402a270 comp rx=0 tx=0).stop 2024-09-10T07:57:56.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.214+0000 7ff4d8e00640 1 -- 172.21.15.2:0/2858412891 shutdown_connections 2024-09-10T07:57:56.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.214+0000 7ff4d8e00640 1 --2- 172.21.15.2:0/2858412891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 0x7ff4d4109ce0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:56.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.214+0000 7ff4d8e00640 1 -- 172.21.15.2:0/2858412891 >> 172.21.15.2:0/2858412891 conn(0x7ff4d407ceb0 msgr2=0x7ff4d407d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:56.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.214+0000 7ff4d8e00640 1 -- 172.21.15.2:0/2858412891 shutdown_connections 2024-09-10T07:57:56.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.214+0000 7ff4d8e00640 1 -- 172.21.15.2:0/2858412891 wait complete. 2024-09-10T07:57:56.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.215+0000 7ff4d8e00640 1 Processor -- start 2024-09-10T07:57:56.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.215+0000 7ff4d8e00640 1 -- start start 2024-09-10T07:57:56.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.216+0000 7ff4d8e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 0x7ff4d419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:56.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.216+0000 7ff4d8e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d410de90 con 0x7ff4d41098e0 2024-09-10T07:57:56.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.216+0000 7ff4d1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 0x7ff4d419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:56.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.216+0000 7ff4d1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 0x7ff4d419d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58130/0 (socket says 172.21.15.2:58130) 2024-09-10T07:57:56.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.216+0000 7ff4d1e00640 1 -- 172.21.15.2:0/3013408170 learned_addr learned my addr 172.21.15.2:0/3013408170 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:56.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.216+0000 7ff4d1e00640 1 -- 172.21.15.2:0/3013408170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d419df00 con 0x7ff4d41098e0 2024-09-10T07:57:56.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.217+0000 7ff4d1e00640 1 --2- 172.21.15.2:0/3013408170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 0x7ff4d419d9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40097f0 tx=0x7ff4c40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:56.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.217+0000 7ff4caa00640 1 -- 172.21.15.2:0/3013408170 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4c4037040 con 0x7ff4d41098e0 2024-09-10T07:57:56.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.217+0000 7ff4caa00640 1 -- 172.21.15.2:0/3013408170 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4c4030cc0 con 0x7ff4d41098e0 2024-09-10T07:57:56.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.217+0000 7ff4d8e00640 1 -- 172.21.15.2:0/3013408170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d419e190 con 0x7ff4d41098e0 2024-09-10T07:57:56.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.217+0000 7ff4caa00640 1 -- 172.21.15.2:0/3013408170 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4c403a640 con 0x7ff4d41098e0 2024-09-10T07:57:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.217+0000 7ff4d8e00640 1 -- 172.21.15.2:0/3013408170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d41a0e80 con 0x7ff4d41098e0 2024-09-10T07:57:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.218+0000 7ff4caa00640 1 -- 172.21.15.2:0/3013408170 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff4c403b3f0 con 0x7ff4d41098e0 2024-09-10T07:57:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.218+0000 7ff4d8e00640 1 -- 172.21.15.2:0/3013408170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d410e010 con 0x7ff4d41098e0 2024-09-10T07:57:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.219+0000 7ff4caa00640 1 --2- 172.21.15.2:0/3013408170 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7ff4b8038f70 0x7ff4b803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.219+0000 7ff4caa00640 1 -- 172.21.15.2:0/3013408170 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff4c406c5a0 con 0x7ff4d41098e0 2024-09-10T07:57:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.219+0000 7ff4cbe00640 1 --2- 172.21.15.2:0/3013408170 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7ff4b8038f70 0x7ff4b803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.220+0000 7ff4cbe00640 1 --2- 172.21.15.2:0/3013408170 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7ff4b8038f70 0x7ff4b803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0098c0 tx=0x7ff4bc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.224+0000 7ff4caa00640 1 -- 172.21.15.2:0/3013408170 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4c40328c0 con 0x7ff4d41098e0 2024-09-10T07:57:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.408+0000 7ff4d8e00640 1 -- 172.21.15.2:0/3013408170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7ff4d41a1050 con 0x7ff4d41098e0 2024-09-10T07:57:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.410+0000 7ff4caa00640 1 -- 172.21.15.2:0/3013408170 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7ff4c4042dc0 con 0x7ff4d41098e0 2024-09-10T07:57:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.413+0000 7ff4d8e00640 1 -- 172.21.15.2:0/3013408170 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7ff4b8038f70 msgr2=0x7ff4b803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.413+0000 7ff4d8e00640 1 --2- 172.21.15.2:0/3013408170 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7ff4b8038f70 0x7ff4b803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0098c0 tx=0x7ff4bc006db0 comp rx=0 tx=0).stop 2024-09-10T07:57:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.413+0000 7ff4d8e00640 1 -- 172.21.15.2:0/3013408170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 msgr2=0x7ff4d419d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:56.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.413+0000 7ff4d8e00640 1 --2- 172.21.15.2:0/3013408170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 0x7ff4d419d9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40097f0 tx=0x7ff4c40327b0 comp rx=0 tx=0).stop 2024-09-10T07:57:56.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.413+0000 7ff4d8e00640 1 -- 172.21.15.2:0/3013408170 shutdown_connections 2024-09-10T07:57:56.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.413+0000 7ff4d8e00640 1 --2- 172.21.15.2:0/3013408170 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7ff4b8038f70 0x7ff4b803b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:56.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.413+0000 7ff4d8e00640 1 --2- 172.21.15.2:0/3013408170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d41098e0 0x7ff4d419d9c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:56.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.413+0000 7ff4d8e00640 1 -- 172.21.15.2:0/3013408170 >> 172.21.15.2:0/3013408170 conn(0x7ff4d407ceb0 msgr2=0x7ff4d41075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:56.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.413+0000 7ff4d8e00640 1 -- 172.21.15.2:0/3013408170 shutdown_connections 2024-09-10T07:57:56.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:55.413+0000 7ff4d8e00640 1 -- 172.21.15.2:0/3013408170 wait complete. 2024-09-10T07:57:56.006 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling the dashboard module... 2024-09-10T07:57:56.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:56 smithi002 ceph-mon[30563]: from='mgr.14118 172.21.15.2:0/3508535874' entity='mgr.x' 2024-09-10T07:57:56.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:56 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3013408170' entity='client.admin' 2024-09-10T07:57:57.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:57 smithi002 ceph-mon[30563]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:57:57.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:57 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/816065296' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T07:57:57.520 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:57 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: ignoring --setuser ceph since I am not root 2024-09-10T07:57:57.520 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:57 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: ignoring --setgroup ceph since I am not root 2024-09-10T07:57:57.889 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:57 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:57.619+0000 7f63eb1da040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T07:57:57.889 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:57 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:57.694+0000 7f63eb1da040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T07:57:57.889 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:57 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:57.889+0000 7f63eb1da040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T07:57:58.151 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:57 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:57.972+0000 7f63eb1da040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T07:57:58.151 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:58 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:58.151+0000 7f63eb1da040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:57:58.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.647+0000 7fd378800640 1 Processor -- start 2024-09-10T07:57:58.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.647+0000 7fd378800640 1 -- start start 2024-09-10T07:57:58.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.647+0000 7fd378800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 0x7fd370109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:58.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.647+0000 7fd378800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd37010a2b0 con 0x7fd3701098e0 2024-09-10T07:57:58.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.647+0000 7fd375e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 0x7fd370109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:58.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.648+0000 7fd375e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 0x7fd370109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58132/0 (socket says 172.21.15.2:58132) 2024-09-10T07:57:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.648+0000 7fd375e00640 1 -- 172.21.15.2:0/319817156 learned_addr learned my addr 172.21.15.2:0/319817156 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.648+0000 7fd375e00640 1 -- 172.21.15.2:0/319817156 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37010aae0 con 0x7fd3701098e0 2024-09-10T07:57:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.649+0000 7fd375e00640 1 --2- 172.21.15.2:0/319817156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 0x7fd370109ce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd364009820 tx=0x7fd36402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91767ec2af208c79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.649+0000 7fd374a00640 1 -- 172.21.15.2:0/319817156 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd364037070 con 0x7fd3701098e0 2024-09-10T07:57:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.649+0000 7fd374a00640 1 -- 172.21.15.2:0/319817156 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd364032920 con 0x7fd3701098e0 2024-09-10T07:57:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.649+0000 7fd374a00640 1 -- 172.21.15.2:0/319817156 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd364030810 con 0x7fd3701098e0 2024-09-10T07:57:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.650+0000 7fd378800640 1 -- 172.21.15.2:0/319817156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 msgr2=0x7fd370109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.650+0000 7fd378800640 1 --2- 172.21.15.2:0/319817156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 0x7fd370109ce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd364009820 tx=0x7fd36402a270 comp rx=0 tx=0).stop 2024-09-10T07:57:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.650+0000 7fd378800640 1 -- 172.21.15.2:0/319817156 shutdown_connections 2024-09-10T07:57:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.650+0000 7fd378800640 1 --2- 172.21.15.2:0/319817156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 0x7fd370109ce0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.650+0000 7fd378800640 1 -- 172.21.15.2:0/319817156 >> 172.21.15.2:0/319817156 conn(0x7fd37007ceb0 msgr2=0x7fd37007d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.650+0000 7fd378800640 1 -- 172.21.15.2:0/319817156 shutdown_connections 2024-09-10T07:57:58.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.650+0000 7fd378800640 1 -- 172.21.15.2:0/319817156 wait complete. 2024-09-10T07:57:58.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.651+0000 7fd378800640 1 Processor -- start 2024-09-10T07:57:58.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.651+0000 7fd378800640 1 -- start start 2024-09-10T07:57:58.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.651+0000 7fd378800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 0x7fd37019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:58.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.651+0000 7fd378800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd37010de90 con 0x7fd3701098e0 2024-09-10T07:57:58.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.651+0000 7fd375e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 0x7fd37019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:58.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.651+0000 7fd375e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 0x7fd37019d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58144/0 (socket says 172.21.15.2:58144) 2024-09-10T07:57:58.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.651+0000 7fd375e00640 1 -- 172.21.15.2:0/816065296 learned_addr learned my addr 172.21.15.2:0/816065296 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:58.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.652+0000 7fd375e00640 1 -- 172.21.15.2:0/816065296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37019df00 con 0x7fd3701098e0 2024-09-10T07:57:58.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.652+0000 7fd375e00640 1 --2- 172.21.15.2:0/816065296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 0x7fd37019d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd3640097f0 tx=0x7fd3640327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:58.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.652+0000 7fd36aa00640 1 -- 172.21.15.2:0/816065296 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd364037040 con 0x7fd3701098e0 2024-09-10T07:57:58.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.652+0000 7fd36aa00640 1 -- 172.21.15.2:0/816065296 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd364030cc0 con 0x7fd3701098e0 2024-09-10T07:57:58.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.652+0000 7fd36aa00640 1 -- 172.21.15.2:0/816065296 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd36403a640 con 0x7fd3701098e0 2024-09-10T07:57:58.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.652+0000 7fd378800640 1 -- 172.21.15.2:0/816065296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37019e190 con 0x7fd3701098e0 2024-09-10T07:57:58.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.652+0000 7fd378800640 1 -- 172.21.15.2:0/816065296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3701a0e80 con 0x7fd3701098e0 2024-09-10T07:57:58.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.653+0000 7fd378800640 1 -- 172.21.15.2:0/816065296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd37010e010 con 0x7fd3701098e0 2024-09-10T07:57:58.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.653+0000 7fd36aa00640 1 -- 172.21.15.2:0/816065296 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd36403b3f0 con 0x7fd3701098e0 2024-09-10T07:57:58.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.654+0000 7fd36aa00640 1 --2- 172.21.15.2:0/816065296 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fd358038fc0 0x7fd35803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:58.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.654+0000 7fd36aa00640 1 -- 172.21.15.2:0/816065296 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd36406c570 con 0x7fd3701098e0 2024-09-10T07:57:58.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.654+0000 7fd375400640 1 --2- 172.21.15.2:0/816065296 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fd358038fc0 0x7fd35803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:58.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.655+0000 7fd375400640 1 --2- 172.21.15.2:0/816065296 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fd358038fc0 0x7fd35803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd3600098c0 tx=0x7fd360006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:58.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.659+0000 7fd36aa00640 1 -- 172.21.15.2:0/816065296 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd364042e10 con 0x7fd3701098e0 2024-09-10T07:57:58.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:56.911+0000 7fd378800640 1 -- 172.21.15.2:0/816065296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fd370109d60 con 0x7fd3701098e0 2024-09-10T07:57:58.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:57.371+0000 7fd36aa00640 1 -- 172.21.15.2:0/816065296 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fd3640328c0 con 0x7fd3701098e0 2024-09-10T07:57:58.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:57.372+0000 7fd36aa00640 1 -- 172.21.15.2:0/816065296 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fd364049ca0 con 0x7fd3701098e0 2024-09-10T07:57:58.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:57.375+0000 7fd378800640 1 -- 172.21.15.2:0/816065296 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fd358038fc0 msgr2=0x7fd35803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:58.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:57.375+0000 7fd378800640 1 --2- 172.21.15.2:0/816065296 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fd358038fc0 0x7fd35803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd3600098c0 tx=0x7fd360006db0 comp rx=0 tx=0).stop 2024-09-10T07:57:58.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:57.375+0000 7fd378800640 1 -- 172.21.15.2:0/816065296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 msgr2=0x7fd37019d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:58.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:57.375+0000 7fd378800640 1 --2- 172.21.15.2:0/816065296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 0x7fd37019d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd3640097f0 tx=0x7fd3640327b0 comp rx=0 tx=0).stop 2024-09-10T07:57:58.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:57.375+0000 7fd378800640 1 -- 172.21.15.2:0/816065296 shutdown_connections 2024-09-10T07:57:58.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:57.375+0000 7fd378800640 1 --2- 172.21.15.2:0/816065296 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7fd358038fc0 0x7fd35803b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:58.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:57.375+0000 7fd378800640 1 --2- 172.21.15.2:0/816065296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3701098e0 0x7fd37019d9c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:58.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:57.375+0000 7fd378800640 1 -- 172.21.15.2:0/816065296 >> 172.21.15.2:0/816065296 conn(0x7fd37007ceb0 msgr2=0x7fd3701075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:58.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:57.375+0000 7fd378800640 1 -- 172.21.15.2:0/816065296 shutdown_connections 2024-09-10T07:57:58.275 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:57.375+0000 7fd378800640 1 -- 172.21.15.2:0/816065296 wait complete. 2024-09-10T07:57:58.409 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:58 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:58.327+0000 7f63eb1da040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T07:57:58.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:58 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/816065296' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T07:57:58.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:58 smithi002 ceph-mon[30563]: mgrmap e9: x(active, since 21s) 2024-09-10T07:57:59.008 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:58 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:58.722+0000 7f63eb1da040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T07:57:59.009 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:58 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:58.798+0000 7f63eb1da040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T07:57:59.269 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:57:59 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:57:59.022+0000 7f63eb1da040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T07:57:59.709 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:57:59 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/643736149' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T07:57:59.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:57:59.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T07:57:59.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T07:57:59.720 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T07:57:59.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T07:57:59.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:57:59.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.799+0000 7f9845000640 1 Processor -- start 2024-09-10T07:57:59.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.799+0000 7f9845000640 1 -- start start 2024-09-10T07:57:59.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.800+0000 7f9845000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 0x7f9840109d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:59.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.800+0000 7f9845000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f984010a350 con 0x7f9840109980 2024-09-10T07:57:59.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.800+0000 7f983de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 0x7f9840109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:59.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.800+0000 7f983de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 0x7f9840109d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57230/0 (socket says 172.21.15.2:57230) 2024-09-10T07:57:59.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.800+0000 7f983de00640 1 -- 172.21.15.2:0/1536492245 learned_addr learned my addr 172.21.15.2:0/1536492245 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:59.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.800+0000 7f983de00640 1 -- 172.21.15.2:0/1536492245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984010aaa0 con 0x7f9840109980 2024-09-10T07:57:59.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.801+0000 7f983de00640 1 --2- 172.21.15.2:0/1536492245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 0x7f9840109d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9830009820 tx=0x7f983002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217b8ee6e7d7e7c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:59.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.802+0000 7f983d400640 1 -- 172.21.15.2:0/1536492245 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830037070 con 0x7f9840109980 2024-09-10T07:57:59.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.802+0000 7f983d400640 1 -- 172.21.15.2:0/1536492245 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9830032920 con 0x7f9840109980 2024-09-10T07:57:59.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.802+0000 7f983d400640 1 -- 172.21.15.2:0/1536492245 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830030810 con 0x7f9840109980 2024-09-10T07:57:59.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.803+0000 7f9845000640 1 -- 172.21.15.2:0/1536492245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 msgr2=0x7f9840109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:59.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.803+0000 7f9845000640 1 --2- 172.21.15.2:0/1536492245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 0x7f9840109d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9830009820 tx=0x7f983002a270 comp rx=0 tx=0).stop 2024-09-10T07:57:59.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.803+0000 7f9845000640 1 -- 172.21.15.2:0/1536492245 shutdown_connections 2024-09-10T07:57:59.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.803+0000 7f9845000640 1 --2- 172.21.15.2:0/1536492245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 0x7f9840109d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:59.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.803+0000 7f9845000640 1 -- 172.21.15.2:0/1536492245 >> 172.21.15.2:0/1536492245 conn(0x7f984007ced0 msgr2=0x7f984007d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:59.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.803+0000 7f9845000640 1 -- 172.21.15.2:0/1536492245 shutdown_connections 2024-09-10T07:57:59.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.803+0000 7f9845000640 1 -- 172.21.15.2:0/1536492245 wait complete. 2024-09-10T07:57:59.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.804+0000 7f9845000640 1 Processor -- start 2024-09-10T07:57:59.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.804+0000 7f9845000640 1 -- start start 2024-09-10T07:57:59.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.804+0000 7f9845000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 0x7f984019d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:59.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.804+0000 7f9845000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f984010de50 con 0x7f9840109980 2024-09-10T07:57:59.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.805+0000 7f983de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 0x7f984019d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:57:59.723 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.805+0000 7f983de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 0x7f984019d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57238/0 (socket says 172.21.15.2:57238) 2024-09-10T07:57:59.723 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.805+0000 7f983de00640 1 -- 172.21.15.2:0/643736149 learned_addr learned my addr 172.21.15.2:0/643736149 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:57:59.723 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.805+0000 7f983de00640 1 -- 172.21.15.2:0/643736149 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984019dde0 con 0x7f9840109980 2024-09-10T07:57:59.723 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.805+0000 7f983de00640 1 --2- 172.21.15.2:0/643736149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 0x7f984019d8a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f98300097f0 tx=0x7f98300327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:57:59.723 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.805+0000 7f9837400640 1 -- 172.21.15.2:0/643736149 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830037040 con 0x7f9840109980 2024-09-10T07:57:59.723 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.806+0000 7f9837400640 1 -- 172.21.15.2:0/643736149 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9830030cc0 con 0x7f9840109980 2024-09-10T07:57:59.723 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.806+0000 7f9837400640 1 -- 172.21.15.2:0/643736149 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f983003a640 con 0x7f9840109980 2024-09-10T07:57:59.723 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.806+0000 7f9845000640 1 -- 172.21.15.2:0/643736149 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984019e070 con 0x7f9840109980 2024-09-10T07:57:59.723 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.806+0000 7f9845000640 1 -- 172.21.15.2:0/643736149 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98401a0d60 con 0x7f9840109980 2024-09-10T07:57:59.723 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.807+0000 7f9845000640 1 -- 172.21.15.2:0/643736149 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984010dfd0 con 0x7f9840109980 2024-09-10T07:57:59.723 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.807+0000 7f9837400640 1 -- 172.21.15.2:0/643736149 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f983003b3f0 con 0x7f9840109980 2024-09-10T07:57:59.723 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.807+0000 7f9837400640 1 --2- 172.21.15.2:0/643736149 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f9824039020 0x7f982403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:57:59.724 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.807+0000 7f9837400640 1 -- 172.21.15.2:0/643736149 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f983006c5c0 con 0x7f9840109980 2024-09-10T07:57:59.724 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.807+0000 7f9835400640 1 -- 172.21.15.2:0/643736149 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f9824039020 msgr2=0x7f982403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3590511604 2024-09-10T07:57:59.724 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.808+0000 7f9835400640 1 --2- 172.21.15.2:0/643736149 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f9824039020 0x7f982403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:57:59.724 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:58.812+0000 7f9837400640 1 -- 172.21.15.2:0/643736149 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98300328c0 con 0x7f9840109980 2024-09-10T07:57:59.724 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.009+0000 7f9835400640 1 -- 172.21.15.2:0/643736149 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f9824039020 msgr2=0x7f982403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3590511604 2024-09-10T07:57:59.724 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.009+0000 7f9835400640 1 --2- 172.21.15.2:0/643736149 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f9824039020 0x7f982403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T07:57:59.724 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.049+0000 7f9845000640 1 -- 172.21.15.2:0/643736149 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f98401a0f30 con 0x7f9840109980 2024-09-10T07:57:59.724 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.049+0000 7f9837400640 1 -- 172.21.15.2:0/643736149 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f9830042dc0 con 0x7f9840109980 2024-09-10T07:57:59.724 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.052+0000 7f9845000640 1 -- 172.21.15.2:0/643736149 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f9824039020 msgr2=0x7f982403b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:57:59.724 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.052+0000 7f9845000640 1 --2- 172.21.15.2:0/643736149 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f9824039020 0x7f982403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:59.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.052+0000 7f9845000640 1 -- 172.21.15.2:0/643736149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 msgr2=0x7f984019d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:57:59.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.052+0000 7f9845000640 1 --2- 172.21.15.2:0/643736149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 0x7f984019d8a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f98300097f0 tx=0x7f98300327b0 comp rx=0 tx=0).stop 2024-09-10T07:57:59.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.052+0000 7f9845000640 1 -- 172.21.15.2:0/643736149 shutdown_connections 2024-09-10T07:57:59.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.052+0000 7f9845000640 1 --2- 172.21.15.2:0/643736149 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f9824039020 0x7f982403b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:59.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.052+0000 7f9845000640 1 --2- 172.21.15.2:0/643736149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9840109980 0x7f984019d8a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:57:59.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.052+0000 7f9845000640 1 -- 172.21.15.2:0/643736149 >> 172.21.15.2:0/643736149 conn(0x7f984007ced0 msgr2=0x7f9840107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:57:59.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.052+0000 7f9845000640 1 -- 172.21.15.2:0/643736149 shutdown_connections 2024-09-10T07:57:59.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:57:59.052+0000 7f9845000640 1 -- 172.21.15.2:0/643736149 wait complete. 2024-09-10T07:57:59.725 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2024-09-10T07:57:59.725 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 9... 2024-09-10T07:58:00.273 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:00 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:00.198+0000 7f63eb1da040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T07:58:00.770 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:00 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:00.286+0000 7f63eb1da040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T07:58:01.519 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:01 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:01.237+0000 7f63eb1da040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T07:58:01.520 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:01 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:01.378+0000 7f63eb1da040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T07:58:01.835 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:01 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:01.548+0000 7f63eb1da040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:58:01.835 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:01 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:01.623+0000 7f63eb1da040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T07:58:01.835 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:01 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:01.835+0000 7f63eb1da040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T07:58:02.182 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:01 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:01.920+0000 7f63eb1da040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T07:58:02.182 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:01 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:01.994+0000 7f63eb1da040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T07:58:02.183 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:02 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:02.182+0000 7f63eb1da040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T07:58:02.445 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:02 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:02.263+0000 7f63eb1da040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T07:58:02.445 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:02 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:02.344+0000 7f63eb1da040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T07:58:02.445 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:02 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:02.445+0000 7f63eb1da040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T07:58:02.769 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:02 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:02.518+0000 7f63eb1da040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T07:58:02.770 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:02 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:02.591+0000 7f63eb1da040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T07:58:03.817 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:03 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:03.545+0000 7f63eb1da040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T07:58:03.817 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:03 smithi002 ceph-mon[30563]: Active manager daemon x restarted 2024-09-10T07:58:03.817 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:03 smithi002 ceph-mon[30563]: Activating manager daemon x 2024-09-10T07:58:03.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:03 smithi002 ceph-mon[30563]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T07:58:03.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:03 smithi002 ceph-mon[30563]: mgrmap e10: x(active, starting, since 0.00366419s) 2024-09-10T07:58:03.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:58:03.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T07:58:03.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T07:58:03.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T07:58:03.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T07:58:03.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:03 smithi002 ceph-mon[30563]: Manager daemon x is now available 2024-09-10T07:58:03.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:58:04.833 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T07:58:04.833 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T07:58:04.833 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:04 smithi002 ceph-mon[30563]: mgrmap e11: x(active, since 1.00786s) 2024-09-10T07:58:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-mon[30563]: [10/Sep/2024:07:58:05] ENGINE Bus STARTING 2024-09-10T07:58:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-mon[30563]: [10/Sep/2024:07:58:05] ENGINE Serving on http://172.21.15.2:8765 2024-09-10T07:58:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-mon[30563]: [10/Sep/2024:07:58:05] ENGINE Serving on https://172.21.15.2:7150 2024-09-10T07:58:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-mon[30563]: [10/Sep/2024:07:58:05] ENGINE Bus STARTED 2024-09-10T07:58:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-mon[30563]: [10/Sep/2024:07:58:05] ENGINE Error in HTTPServer.serve 2024-09-10T07:58:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout: Traceback (most recent call last): 2024-09-10T07:58:05.771 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T07:58:05.771 INFO:journalctl@ceph.mon.a.smithi002.stdout: self._connections.run(self.expiration_interval) 2024-09-10T07:58:05.771 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T07:58:05.771 INFO:journalctl@ceph.mon.a.smithi002.stdout: self._run(expiration_interval) 2024-09-10T07:58:05.771 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T07:58:05.771 INFO:journalctl@ceph.mon.a.smithi002.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T07:58:05.771 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T07:58:05.771 INFO:journalctl@ceph.mon.a.smithi002.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T07:58:05.771 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T07:58:05.771 INFO:journalctl@ceph.mon.a.smithi002.stdout: s = self.context.wrap_socket( 2024-09-10T07:58:05.771 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T07:58:05.772 INFO:journalctl@ceph.mon.a.smithi002.stdout: return self.sslsocket_class._create( 2024-09-10T07:58:05.772 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T07:58:05.772 INFO:journalctl@ceph.mon.a.smithi002.stdout: self.do_handshake() 2024-09-10T07:58:05.772 INFO:journalctl@ceph.mon.a.smithi002.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T07:58:05.772 INFO:journalctl@ceph.mon.a.smithi002.stdout: self._sslobj.do_handshake() 2024-09-10T07:58:05.772 INFO:journalctl@ceph.mon.a.smithi002.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T07:58:05.772 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:05.483+0000 7f6340c00640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:07:58:05] ENGINE Error in HTTPServer.serve 2024-09-10T07:58:05.772 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: Traceback (most recent call last): 2024-09-10T07:58:05.772 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T07:58:05.773 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: self._connections.run(self.expiration_interval) 2024-09-10T07:58:05.773 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T07:58:05.773 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: self._run(expiration_interval) 2024-09-10T07:58:05.773 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T07:58:05.773 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T07:58:05.773 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T07:58:05.773 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T07:58:05.773 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T07:58:05.773 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: s = self.context.wrap_socket( 2024-09-10T07:58:05.773 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T07:58:05.773 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: return self.sslsocket_class._create( 2024-09-10T07:58:05.774 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T07:58:05.774 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: self.do_handshake() 2024-09-10T07:58:05.774 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T07:58:05.774 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: self._sslobj.do_handshake() 2024-09-10T07:58:05.774 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T07:58:05.774 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:05.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-09-10T07:58:05.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T07:58:05.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T07:58:05.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-09-10T07:58:05.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.273+0000 7f5da9000640 1 Processor -- start 2024-09-10T07:58:05.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.273+0000 7f5da9000640 1 -- start start 2024-09-10T07:58:05.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.273+0000 7f5da9000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 0x7f5da4107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:05.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.273+0000 7f5da9000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5da41080f0 con 0x7f5da4107720 2024-09-10T07:58:05.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.274+0000 7f5da1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 0x7f5da4107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:05.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.274+0000 7f5da1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 0x7f5da4107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57240/0 (socket says 172.21.15.2:57240) 2024-09-10T07:58:05.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.274+0000 7f5da1e00640 1 -- 172.21.15.2:0/3523113070 learned_addr learned my addr 172.21.15.2:0/3523113070 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:05.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.274+0000 7f5da1e00640 1 -- 172.21.15.2:0/3523113070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da41088d0 con 0x7f5da4107720 2024-09-10T07:58:05.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.275+0000 7f5da1e00640 1 --2- 172.21.15.2:0/3523113070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 0x7f5da4107b20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009820 tx=0x7f5d9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c87b2408f71dd3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:05.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.276+0000 7f5da0a00640 1 -- 172.21.15.2:0/3523113070 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d94037070 con 0x7f5da4107720 2024-09-10T07:58:05.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.276+0000 7f5da0a00640 1 -- 172.21.15.2:0/3523113070 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d94032920 con 0x7f5da4107720 2024-09-10T07:58:05.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.276+0000 7f5da0a00640 1 -- 172.21.15.2:0/3523113070 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d94030810 con 0x7f5da4107720 2024-09-10T07:58:05.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.276+0000 7f5da9000640 1 -- 172.21.15.2:0/3523113070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 msgr2=0x7f5da4107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:05.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.276+0000 7f5da9000640 1 --2- 172.21.15.2:0/3523113070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 0x7f5da4107b20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009820 tx=0x7f5d9402a270 comp rx=0 tx=0).stop 2024-09-10T07:58:05.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.276+0000 7f5da9000640 1 -- 172.21.15.2:0/3523113070 shutdown_connections 2024-09-10T07:58:05.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.276+0000 7f5da9000640 1 --2- 172.21.15.2:0/3523113070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 0x7f5da4107b20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:05.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.276+0000 7f5da9000640 1 -- 172.21.15.2:0/3523113070 >> 172.21.15.2:0/3523113070 conn(0x7f5da4102e90 msgr2=0x7f5da41052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:05.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.277+0000 7f5da9000640 1 -- 172.21.15.2:0/3523113070 shutdown_connections 2024-09-10T07:58:05.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.277+0000 7f5da9000640 1 -- 172.21.15.2:0/3523113070 wait complete. 2024-09-10T07:58:05.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.277+0000 7f5da9000640 1 Processor -- start 2024-09-10T07:58:05.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.278+0000 7f5da9000640 1 -- start start 2024-09-10T07:58:05.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.278+0000 7f5da9000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 0x7f5da419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:05.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.278+0000 7f5da9000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5da410bc80 con 0x7f5da4107720 2024-09-10T07:58:05.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.278+0000 7f5da1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 0x7f5da419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:05.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.278+0000 7f5da1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 0x7f5da419b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57252/0 (socket says 172.21.15.2:57252) 2024-09-10T07:58:05.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.278+0000 7f5da1e00640 1 -- 172.21.15.2:0/3883526914 learned_addr learned my addr 172.21.15.2:0/3883526914 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:05.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.279+0000 7f5da1e00640 1 -- 172.21.15.2:0/3883526914 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da419bca0 con 0x7f5da4107720 2024-09-10T07:58:05.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.279+0000 7f5da1e00640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 0x7f5da419b760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d940097f0 tx=0x7f5d940327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:05.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.279+0000 7f5d9aa00640 1 -- 172.21.15.2:0/3883526914 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d94037040 con 0x7f5da4107720 2024-09-10T07:58:05.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.279+0000 7f5d9aa00640 1 -- 172.21.15.2:0/3883526914 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d94030c40 con 0x7f5da4107720 2024-09-10T07:58:05.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.279+0000 7f5da9000640 1 -- 172.21.15.2:0/3883526914 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da419bf30 con 0x7f5da4107720 2024-09-10T07:58:05.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.279+0000 7f5d9aa00640 1 -- 172.21.15.2:0/3883526914 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d9403a5b0 con 0x7f5da4107720 2024-09-10T07:58:05.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.280+0000 7f5da9000640 1 -- 172.21.15.2:0/3883526914 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da419ec20 con 0x7f5da4107720 2024-09-10T07:58:05.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.281+0000 7f5d9aa00640 1 -- 172.21.15.2:0/3883526914 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f5d9403a7a0 con 0x7f5da4107720 2024-09-10T07:58:05.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.282+0000 7f5d9aa00640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 0x7f5d8803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:05.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.282+0000 7f5d9aa00640 1 -- 172.21.15.2:0/3883526914 --> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5d8803bbf0 con 0x7f5d88039020 2024-09-10T07:58:05.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.282+0000 7f5d9aa00640 1 -- 172.21.15.2:0/3883526914 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5d9406c4c0 con 0x7f5da4107720 2024-09-10T07:58:05.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.282+0000 7f5da1400640 1 -- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 msgr2=0x7f5d8803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3590511604 2024-09-10T07:58:05.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.282+0000 7f5da1400640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 0x7f5d8803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:58:05.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.483+0000 7f5da1400640 1 -- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 msgr2=0x7f5d8803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3590511604 2024-09-10T07:58:05.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.483+0000 7f5da1400640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 0x7f5d8803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T07:58:05.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.884+0000 7f5da1400640 1 -- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 msgr2=0x7f5d8803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3590511604 2024-09-10T07:58:05.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:00.884+0000 7f5da1400640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 0x7f5d8803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T07:58:05.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:01.685+0000 7f5da1400640 1 -- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 msgr2=0x7f5d8803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3590511604 2024-09-10T07:58:05.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:01.685+0000 7f5da1400640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 0x7f5d8803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T07:58:05.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:03.286+0000 7f5da1400640 1 -- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 msgr2=0x7f5d8803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3590511604 2024-09-10T07:58:05.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:03.286+0000 7f5da1400640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 0x7f5d8803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T07:58:05.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:03.550+0000 7f5d9aa00640 1 -- 172.21.15.2:0/3883526914 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f5d9403b3f0 con 0x7f5da4107720 2024-09-10T07:58:05.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:03.550+0000 7f5d9aa00640 1 -- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 msgr2=0x7f5d8803b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:58:05.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:03.550+0000 7f5d9aa00640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 0x7f5d8803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:05.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.552+0000 7f5d9aa00640 1 -- 172.21.15.2:0/3883526914 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f5d9406dc80 con 0x7f5da4107720 2024-09-10T07:58:05.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.553+0000 7f5d9aa00640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5d8803cae0 0x7f5d8803eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:05.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.553+0000 7f5d9aa00640 1 -- 172.21.15.2:0/3883526914 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5d8803bbf0 con 0x7f5d8803cae0 2024-09-10T07:58:05.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.553+0000 7f5da1400640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5d8803cae0 0x7f5d8803eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:05.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.554+0000 7f5da1400640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5d8803cae0 0x7f5d8803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5d90003980 tx=0x7f5d900092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:05.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.555+0000 7f5d9aa00640 1 -- 172.21.15.2:0/3883526914 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5d8803bbf0 con 0x7f5d8803cae0 2024-09-10T07:58:05.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.569+0000 7f5da9000640 1 -- 172.21.15.2:0/3883526914 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5da410be00 con 0x7f5d8803cae0 2024-09-10T07:58:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.569+0000 7f5d9aa00640 1 -- 172.21.15.2:0/3883526914 <== mgr.14150 v2:172.21.15.2:6800/3751597725 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f5da410be00 con 0x7f5d8803cae0 2024-09-10T07:58:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.569+0000 7f5da9000640 1 -- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5d8803cae0 msgr2=0x7f5d8803eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.569+0000 7f5da9000640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5d8803cae0 0x7f5d8803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5d90003980 tx=0x7f5d900092b0 comp rx=0 tx=0).stop 2024-09-10T07:58:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.569+0000 7f5da9000640 1 -- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 msgr2=0x7f5da419b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.569+0000 7f5da9000640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 0x7f5da419b760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d940097f0 tx=0x7f5d940327b0 comp rx=0 tx=0).stop 2024-09-10T07:58:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.570+0000 7f5da9000640 1 -- 172.21.15.2:0/3883526914 shutdown_connections 2024-09-10T07:58:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.570+0000 7f5da9000640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5d8803cae0 0x7f5d8803eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.570+0000 7f5da9000640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:6800/3590511604,v1:172.21.15.2:6801/3590511604] conn(0x7f5d88039020 0x7f5d8803b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.570+0000 7f5da9000640 1 --2- 172.21.15.2:0/3883526914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5da4107720 0x7f5da419b760 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.570+0000 7f5da9000640 1 -- 172.21.15.2:0/3883526914 >> 172.21.15.2:0/3883526914 conn(0x7f5da4102e90 msgr2=0x7f5da4103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.570+0000 7f5da9000640 1 -- 172.21.15.2:0/3883526914 shutdown_connections 2024-09-10T07:58:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:04.570+0000 7f5da9000640 1 -- 172.21.15.2:0/3883526914 wait complete. 2024-09-10T07:58:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 9 is available 2024-09-10T07:58:05.819 INFO:teuthology.orchestra.run.smithi002.stdout:Generating a dashboard self-signed certificate... 2024-09-10T07:58:06.692 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:06 smithi002 ceph-mon[30563]: mgrmap e12: x(active, since 2s) 2024-09-10T07:58:06.692 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:06 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:07.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T07:58:07.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.342+0000 7f5926a00640 1 Processor -- start 2024-09-10T07:58:07.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.342+0000 7f5926a00640 1 -- start start 2024-09-10T07:58:07.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.342+0000 7f5926a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 0x7f5920109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:07.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.342+0000 7f5926a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f592010a2b0 con 0x7f59201098e0 2024-09-10T07:58:07.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.343+0000 7f591fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 0x7f5920109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:07.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.343+0000 7f591fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 0x7f5920109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57328/0 (socket says 172.21.15.2:57328) 2024-09-10T07:58:07.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.343+0000 7f591fe00640 1 -- 172.21.15.2:0/3406825962 learned_addr learned my addr 172.21.15.2:0/3406825962 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:07.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.343+0000 7f591fe00640 1 -- 172.21.15.2:0/3406825962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592010aab0 con 0x7f59201098e0 2024-09-10T07:58:07.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.344+0000 7f591fe00640 1 --2- 172.21.15.2:0/3406825962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 0x7f5920109ce0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5910009820 tx=0x7f591002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a347281c1c1aa0c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:07.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.344+0000 7f591f400640 1 -- 172.21.15.2:0/3406825962 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5910037070 con 0x7f59201098e0 2024-09-10T07:58:07.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.344+0000 7f591f400640 1 -- 172.21.15.2:0/3406825962 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5910032920 con 0x7f59201098e0 2024-09-10T07:58:07.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.344+0000 7f591f400640 1 -- 172.21.15.2:0/3406825962 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5910030810 con 0x7f59201098e0 2024-09-10T07:58:07.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.345+0000 7f5926a00640 1 -- 172.21.15.2:0/3406825962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 msgr2=0x7f5920109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:07.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.345+0000 7f5926a00640 1 --2- 172.21.15.2:0/3406825962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 0x7f5920109ce0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5910009820 tx=0x7f591002a270 comp rx=0 tx=0).stop 2024-09-10T07:58:07.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.345+0000 7f5926a00640 1 -- 172.21.15.2:0/3406825962 shutdown_connections 2024-09-10T07:58:07.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.345+0000 7f5926a00640 1 --2- 172.21.15.2:0/3406825962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 0x7f5920109ce0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:07.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.345+0000 7f5926a00640 1 -- 172.21.15.2:0/3406825962 >> 172.21.15.2:0/3406825962 conn(0x7f592007ce90 msgr2=0x7f592007d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:07.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.345+0000 7f5926a00640 1 -- 172.21.15.2:0/3406825962 shutdown_connections 2024-09-10T07:58:07.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.345+0000 7f5926a00640 1 -- 172.21.15.2:0/3406825962 wait complete. 2024-09-10T07:58:07.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.346+0000 7f5926a00640 1 Processor -- start 2024-09-10T07:58:07.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.347+0000 7f5926a00640 1 -- start start 2024-09-10T07:58:07.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.347+0000 7f5926a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 0x7f592019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:07.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.347+0000 7f5926a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f592010de60 con 0x7f59201098e0 2024-09-10T07:58:07.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.347+0000 7f591fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 0x7f592019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:07.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.347+0000 7f591fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 0x7f592019d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57342/0 (socket says 172.21.15.2:57342) 2024-09-10T07:58:07.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.347+0000 7f591fe00640 1 -- 172.21.15.2:0/886047587 learned_addr learned my addr 172.21.15.2:0/886047587 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:07.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.348+0000 7f591fe00640 1 -- 172.21.15.2:0/886047587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592019dea0 con 0x7f59201098e0 2024-09-10T07:58:07.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.348+0000 7f591fe00640 1 --2- 172.21.15.2:0/886047587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 0x7f592019d960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f59100097f0 tx=0x7f59100327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:07.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.348+0000 7f591d600640 1 -- 172.21.15.2:0/886047587 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5910037040 con 0x7f59201098e0 2024-09-10T07:58:07.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.348+0000 7f5926a00640 1 -- 172.21.15.2:0/886047587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592019e130 con 0x7f59201098e0 2024-09-10T07:58:07.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.348+0000 7f591d600640 1 -- 172.21.15.2:0/886047587 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5910030c40 con 0x7f59201098e0 2024-09-10T07:58:07.378 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.348+0000 7f591d600640 1 -- 172.21.15.2:0/886047587 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f591003a5b0 con 0x7f59201098e0 2024-09-10T07:58:07.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.348+0000 7f5926a00640 1 -- 172.21.15.2:0/886047587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59201a0e20 con 0x7f59201098e0 2024-09-10T07:58:07.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.349+0000 7f591d600640 1 -- 172.21.15.2:0/886047587 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f591003a7f0 con 0x7f59201098e0 2024-09-10T07:58:07.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.350+0000 7f591d600640 1 --2- 172.21.15.2:0/886047587 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5904038fc0 0x7f590403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:07.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.350+0000 7f591d600640 1 -- 172.21.15.2:0/886047587 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f591006c5d0 con 0x7f59201098e0 2024-09-10T07:58:07.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.350+0000 7f5926a00640 1 -- 172.21.15.2:0/886047587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f592010dfe0 con 0x7f59201098e0 2024-09-10T07:58:07.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.350+0000 7f5917e00640 1 --2- 172.21.15.2:0/886047587 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5904038fc0 0x7f590403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:07.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.351+0000 7f5917e00640 1 --2- 172.21.15.2:0/886047587 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5904038fc0 0x7f590403b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59080098c0 tx=0x7f5908006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:07.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.356+0000 7f591d600640 1 -- 172.21.15.2:0/886047587 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5910030df0 con 0x7f59201098e0 2024-09-10T07:58:07.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.551+0000 7f5926a00640 1 -- 172.21.15.2:0/886047587 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f5920107d90 con 0x7f5904038fc0 2024-09-10T07:58:07.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.692+0000 7f591d600640 1 -- 172.21.15.2:0/886047587 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f5920107d90 con 0x7f5904038fc0 2024-09-10T07:58:07.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.694+0000 7f5926a00640 1 -- 172.21.15.2:0/886047587 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5904038fc0 msgr2=0x7f590403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:07.379 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.694+0000 7f5926a00640 1 --2- 172.21.15.2:0/886047587 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5904038fc0 0x7f590403b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59080098c0 tx=0x7f5908006db0 comp rx=0 tx=0).stop 2024-09-10T07:58:07.380 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.694+0000 7f5926a00640 1 -- 172.21.15.2:0/886047587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 msgr2=0x7f592019d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:07.380 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.694+0000 7f5926a00640 1 --2- 172.21.15.2:0/886047587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 0x7f592019d960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f59100097f0 tx=0x7f59100327b0 comp rx=0 tx=0).stop 2024-09-10T07:58:07.380 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.695+0000 7f5926a00640 1 -- 172.21.15.2:0/886047587 shutdown_connections 2024-09-10T07:58:07.380 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.695+0000 7f5926a00640 1 --2- 172.21.15.2:0/886047587 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5904038fc0 0x7f590403b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:07.380 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.695+0000 7f5926a00640 1 --2- 172.21.15.2:0/886047587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59201098e0 0x7f592019d960 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:07.380 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.695+0000 7f5926a00640 1 -- 172.21.15.2:0/886047587 >> 172.21.15.2:0/886047587 conn(0x7f592007ce90 msgr2=0x7f5920107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:07.380 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.695+0000 7f5926a00640 1 -- 172.21.15.2:0/886047587 shutdown_connections 2024-09-10T07:58:07.380 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:06.695+0000 7f5926a00640 1 -- 172.21.15.2:0/886047587 wait complete. 2024-09-10T07:58:07.380 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial admin user... 2024-09-10T07:58:07.877 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:07 smithi002 ceph-mon[30563]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:58:07.877 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:07 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:07.878 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:07 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:07.878 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:07 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:09.331 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$iMyIOg0fX64tQgf/6ArIeu3vrn0edioadEv8tY/C7g1zM9SULIzgq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725955088, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T07:58:09.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.886+0000 7ffab1e00640 1 Processor -- start 2024-09-10T07:58:09.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.886+0000 7ffab1e00640 1 -- start start 2024-09-10T07:58:09.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.887+0000 7ffab1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 0x7ffaac109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:09.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.887+0000 7ffab1e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac10a2b0 con 0x7ffaac1098e0 2024-09-10T07:58:09.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.887+0000 7ffab0a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 0x7ffaac109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:09.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.887+0000 7ffab0a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 0x7ffaac109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42620/0 (socket says 172.21.15.2:42620) 2024-09-10T07:58:09.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.887+0000 7ffab0a00640 1 -- 172.21.15.2:0/4013835944 learned_addr learned my addr 172.21.15.2:0/4013835944 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:09.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.888+0000 7ffab0a00640 1 -- 172.21.15.2:0/4013835944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaac10a430 con 0x7ffaac1098e0 2024-09-10T07:58:09.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.889+0000 7ffab0a00640 1 --2- 172.21.15.2:0/4013835944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 0x7ffaac109ce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009820 tx=0x7ffa9c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33dc8772d70fb8bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:09.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.889+0000 7ffaabe00640 1 -- 172.21.15.2:0/4013835944 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa9c037070 con 0x7ffaac1098e0 2024-09-10T07:58:09.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.889+0000 7ffaabe00640 1 -- 172.21.15.2:0/4013835944 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa9c032920 con 0x7ffaac1098e0 2024-09-10T07:58:09.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.889+0000 7ffaabe00640 1 -- 172.21.15.2:0/4013835944 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa9c030810 con 0x7ffaac1098e0 2024-09-10T07:58:09.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.890+0000 7ffab1e00640 1 -- 172.21.15.2:0/4013835944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 msgr2=0x7ffaac109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:09.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.890+0000 7ffab1e00640 1 --2- 172.21.15.2:0/4013835944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 0x7ffaac109ce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009820 tx=0x7ffa9c02a270 comp rx=0 tx=0).stop 2024-09-10T07:58:09.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.890+0000 7ffab1e00640 1 -- 172.21.15.2:0/4013835944 shutdown_connections 2024-09-10T07:58:09.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.890+0000 7ffab1e00640 1 --2- 172.21.15.2:0/4013835944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 0x7ffaac109ce0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:09.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.890+0000 7ffab1e00640 1 -- 172.21.15.2:0/4013835944 >> 172.21.15.2:0/4013835944 conn(0x7ffaac07ccf0 msgr2=0x7ffaac07d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:09.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.890+0000 7ffab1e00640 1 -- 172.21.15.2:0/4013835944 shutdown_connections 2024-09-10T07:58:09.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.890+0000 7ffab1e00640 1 -- 172.21.15.2:0/4013835944 wait complete. 2024-09-10T07:58:09.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.891+0000 7ffab1e00640 1 Processor -- start 2024-09-10T07:58:09.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.891+0000 7ffab1e00640 1 -- start start 2024-09-10T07:58:09.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.891+0000 7ffab1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 0x7ffaac19d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:09.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.891+0000 7ffab1e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac10db60 con 0x7ffaac1098e0 2024-09-10T07:58:09.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.892+0000 7ffab0a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 0x7ffaac19d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:09.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.892+0000 7ffab0a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 0x7ffaac19d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42628/0 (socket says 172.21.15.2:42628) 2024-09-10T07:58:09.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.892+0000 7ffab0a00640 1 -- 172.21.15.2:0/3466983920 learned_addr learned my addr 172.21.15.2:0/3466983920 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:09.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.892+0000 7ffab0a00640 1 -- 172.21.15.2:0/3466983920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaac19dca0 con 0x7ffaac1098e0 2024-09-10T07:58:09.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.893+0000 7ffab0a00640 1 --2- 172.21.15.2:0/3466983920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 0x7ffaac19d760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0097f0 tx=0x7ffa9c030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:09.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.893+0000 7ffaaa000640 1 -- 172.21.15.2:0/3466983920 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa9c037040 con 0x7ffaac1098e0 2024-09-10T07:58:09.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.893+0000 7ffab1e00640 1 -- 172.21.15.2:0/3466983920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaac19df30 con 0x7ffaac1098e0 2024-09-10T07:58:09.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.893+0000 7ffaaa000640 1 -- 172.21.15.2:0/3466983920 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa9c040070 con 0x7ffaac1098e0 2024-09-10T07:58:09.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.893+0000 7ffaaa000640 1 -- 172.21.15.2:0/3466983920 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa9c03a6d0 con 0x7ffaac1098e0 2024-09-10T07:58:09.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.893+0000 7ffab1e00640 1 -- 172.21.15.2:0/3466983920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaac1a0c20 con 0x7ffaac1098e0 2024-09-10T07:58:09.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.894+0000 7ffaaa000640 1 -- 172.21.15.2:0/3466983920 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ffa9c03b420 con 0x7ffaac1098e0 2024-09-10T07:58:09.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.894+0000 7ffaa2a00640 1 -- 172.21.15.2:0/3466983920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa84005290 con 0x7ffaac1098e0 2024-09-10T07:58:09.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.895+0000 7ffaaa000640 1 --2- 172.21.15.2:0/3466983920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa90038fc0 0x7ffa9003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:09.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.895+0000 7ffaaa000640 1 -- 172.21.15.2:0/3466983920 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffa9c06c760 con 0x7ffaac1098e0 2024-09-10T07:58:09.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.895+0000 7ffaa3e00640 1 --2- 172.21.15.2:0/3466983920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa90038fc0 0x7ffa9003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:09.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.896+0000 7ffaa3e00640 1 --2- 172.21.15.2:0/3466983920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa90038fc0 0x7ffa9003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa940098c0 tx=0x7ffa94006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:09.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:07.900+0000 7ffaaa000640 1 -- 172.21.15.2:0/3466983920 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa9c038280 con 0x7ffaac1098e0 2024-09-10T07:58:09.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:08.106+0000 7ffaa2a00640 1 -- 172.21.15.2:0/3466983920 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ffa840023f0 con 0x7ffa90038fc0 2024-09-10T07:58:09.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:08.356+0000 7ffaaa000640 1 -- 172.21.15.2:0/3466983920 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ffa840023f0 con 0x7ffa90038fc0 2024-09-10T07:58:09.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:08.358+0000 7ffaa2a00640 1 -- 172.21.15.2:0/3466983920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa90038fc0 msgr2=0x7ffa9003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:09.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:08.358+0000 7ffaa2a00640 1 --2- 172.21.15.2:0/3466983920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa90038fc0 0x7ffa9003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa940098c0 tx=0x7ffa94006db0 comp rx=0 tx=0).stop 2024-09-10T07:58:09.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:08.358+0000 7ffaa2a00640 1 -- 172.21.15.2:0/3466983920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 msgr2=0x7ffaac19d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:09.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:08.358+0000 7ffaa2a00640 1 --2- 172.21.15.2:0/3466983920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 0x7ffaac19d760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0097f0 tx=0x7ffa9c030d40 comp rx=0 tx=0).stop 2024-09-10T07:58:09.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:08.358+0000 7ffaa2a00640 1 -- 172.21.15.2:0/3466983920 shutdown_connections 2024-09-10T07:58:09.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:08.358+0000 7ffaa2a00640 1 --2- 172.21.15.2:0/3466983920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa90038fc0 0x7ffa9003b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:09.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:08.358+0000 7ffaa2a00640 1 --2- 172.21.15.2:0/3466983920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffaac1098e0 0x7ffaac19d760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:09.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:08.358+0000 7ffaa2a00640 1 -- 172.21.15.2:0/3466983920 >> 172.21.15.2:0/3466983920 conn(0x7ffaac07ccf0 msgr2=0x7ffaac107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:09.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:08.359+0000 7ffaa2a00640 1 -- 172.21.15.2:0/3466983920 shutdown_connections 2024-09-10T07:58:09.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:08.359+0000 7ffaa2a00640 1 -- 172.21.15.2:0/3466983920 wait complete. 2024-09-10T07:58:09.344 INFO:teuthology.orchestra.run.smithi002.stdout:Fetching dashboard port number... 2024-09-10T07:58:09.600 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:09 smithi002 ceph-mon[30563]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:58:09.600 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:10.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:10 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3909862743' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T07:58:10.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T07:58:10.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.949+0000 7f645ee00640 1 Processor -- start 2024-09-10T07:58:10.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.949+0000 7f645ee00640 1 -- start start 2024-09-10T07:58:10.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.950+0000 7f645ee00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 0x7f645807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:10.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.950+0000 7f645ee00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f645807c8f0 con 0x7f645807dec0 2024-09-10T07:58:10.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.950+0000 7f6457e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 0x7f645807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:10.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.950+0000 7f6457e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 0x7f645807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42634/0 (socket says 172.21.15.2:42634) 2024-09-10T07:58:10.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.950+0000 7f6457e00640 1 -- 172.21.15.2:0/4249112821 learned_addr learned my addr 172.21.15.2:0/4249112821 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:10.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.951+0000 7f6457e00640 1 -- 172.21.15.2:0/4249112821 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645807ca70 con 0x7f645807dec0 2024-09-10T07:58:10.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.952+0000 7f6457e00640 1 --2- 172.21.15.2:0/4249112821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 0x7f645807c320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6448009820 tx=0x7f644802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179999a26c077ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:10.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.952+0000 7f6457400640 1 -- 172.21.15.2:0/4249112821 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6448037070 con 0x7f645807dec0 2024-09-10T07:58:10.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.952+0000 7f6457400640 1 -- 172.21.15.2:0/4249112821 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6448032920 con 0x7f645807dec0 2024-09-10T07:58:10.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.952+0000 7f6457400640 1 -- 172.21.15.2:0/4249112821 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6448030810 con 0x7f645807dec0 2024-09-10T07:58:10.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.952+0000 7f645ee00640 1 -- 172.21.15.2:0/4249112821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 msgr2=0x7f645807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:10.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.952+0000 7f645ee00640 1 --2- 172.21.15.2:0/4249112821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 0x7f645807c320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6448009820 tx=0x7f644802a270 comp rx=0 tx=0).stop 2024-09-10T07:58:10.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.953+0000 7f645ee00640 1 -- 172.21.15.2:0/4249112821 shutdown_connections 2024-09-10T07:58:10.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.953+0000 7f645ee00640 1 --2- 172.21.15.2:0/4249112821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 0x7f645807c320 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:10.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.953+0000 7f645ee00640 1 -- 172.21.15.2:0/4249112821 >> 172.21.15.2:0/4249112821 conn(0x7f6458102e40 msgr2=0x7f6458105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:10.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.953+0000 7f645ee00640 1 -- 172.21.15.2:0/4249112821 shutdown_connections 2024-09-10T07:58:10.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.953+0000 7f645ee00640 1 -- 172.21.15.2:0/4249112821 wait complete. 2024-09-10T07:58:10.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.954+0000 7f645ee00640 1 Processor -- start 2024-09-10T07:58:10.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.954+0000 7f645ee00640 1 -- start start 2024-09-10T07:58:10.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.954+0000 7f645ee00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 0x7f645819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:10.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.954+0000 7f645ee00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f645810dd80 con 0x7f645807dec0 2024-09-10T07:58:10.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.954+0000 7f6457e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 0x7f645819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:10.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.954+0000 7f6457e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 0x7f645819d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42650/0 (socket says 172.21.15.2:42650) 2024-09-10T07:58:10.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.954+0000 7f6457e00640 1 -- 172.21.15.2:0/3909862743 learned_addr learned my addr 172.21.15.2:0/3909862743 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:10.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.955+0000 7f6457e00640 1 -- 172.21.15.2:0/3909862743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645819de90 con 0x7f645807dec0 2024-09-10T07:58:10.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.955+0000 7f6457e00640 1 --2- 172.21.15.2:0/3909862743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 0x7f645819d950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f64480097f0 tx=0x7f64480327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:10.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.955+0000 7f6455600640 1 -- 172.21.15.2:0/3909862743 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6448037040 con 0x7f645807dec0 2024-09-10T07:58:10.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.955+0000 7f6455600640 1 -- 172.21.15.2:0/3909862743 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6448030c40 con 0x7f645807dec0 2024-09-10T07:58:10.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.955+0000 7f6455600640 1 -- 172.21.15.2:0/3909862743 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f644803a5d0 con 0x7f645807dec0 2024-09-10T07:58:10.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.955+0000 7f645ee00640 1 -- 172.21.15.2:0/3909862743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645819e120 con 0x7f645807dec0 2024-09-10T07:58:10.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.955+0000 7f645ee00640 1 -- 172.21.15.2:0/3909862743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64581a0e10 con 0x7f645807dec0 2024-09-10T07:58:10.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.956+0000 7f6455600640 1 -- 172.21.15.2:0/3909862743 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f644803b3f0 con 0x7f645807dec0 2024-09-10T07:58:10.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.956+0000 7f645ee00640 1 -- 172.21.15.2:0/3909862743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645807c3a0 con 0x7f645807dec0 2024-09-10T07:58:10.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.957+0000 7f6455600640 1 --2- 172.21.15.2:0/3909862743 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f643c038fc0 0x7f643c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:10.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.957+0000 7f6455600640 1 -- 172.21.15.2:0/3909862743 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f644806c650 con 0x7f645807dec0 2024-09-10T07:58:10.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.957+0000 7f644fe00640 1 --2- 172.21.15.2:0/3909862743 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f643c038fc0 0x7f643c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:10.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.958+0000 7f644fe00640 1 --2- 172.21.15.2:0/3909862743 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f643c038fc0 0x7f643c03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f64400098c0 tx=0x7f6440006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:10.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:09.962+0000 7f6455600640 1 -- 172.21.15.2:0/3909862743 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64480313a0 con 0x7f645807dec0 2024-09-10T07:58:10.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:10.150+0000 7f645ee00640 1 -- 172.21.15.2:0/3909862743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f64581a0fe0 con 0x7f645807dec0 2024-09-10T07:58:10.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:10.151+0000 7f6455600640 1 -- 172.21.15.2:0/3909862743 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f6448038250 con 0x7f645807dec0 2024-09-10T07:58:10.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:10.153+0000 7f645ee00640 1 -- 172.21.15.2:0/3909862743 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f643c038fc0 msgr2=0x7f643c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:10.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:10.153+0000 7f645ee00640 1 --2- 172.21.15.2:0/3909862743 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f643c038fc0 0x7f643c03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f64400098c0 tx=0x7f6440006db0 comp rx=0 tx=0).stop 2024-09-10T07:58:10.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:10.153+0000 7f645ee00640 1 -- 172.21.15.2:0/3909862743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 msgr2=0x7f645819d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:10.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:10.153+0000 7f645ee00640 1 --2- 172.21.15.2:0/3909862743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 0x7f645819d950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f64480097f0 tx=0x7f64480327b0 comp rx=0 tx=0).stop 2024-09-10T07:58:10.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:10.153+0000 7f645ee00640 1 -- 172.21.15.2:0/3909862743 shutdown_connections 2024-09-10T07:58:10.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:10.153+0000 7f645ee00640 1 --2- 172.21.15.2:0/3909862743 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f643c038fc0 0x7f643c03b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:10.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:10.153+0000 7f645ee00640 1 --2- 172.21.15.2:0/3909862743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f645807dec0 0x7f645819d950 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:10.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:10.153+0000 7f645ee00640 1 -- 172.21.15.2:0/3909862743 >> 172.21.15.2:0/3909862743 conn(0x7f6458102e40 msgr2=0x7f6458103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:10.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:10.153+0000 7f645ee00640 1 -- 172.21.15.2:0/3909862743 shutdown_connections 2024-09-10T07:58:10.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:10.153+0000 7f645ee00640 1 -- 172.21.15.2:0/3909862743 wait complete. 2024-09-10T07:58:10.966 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T07:58:10.966 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-09-10T07:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T07:58:10.981 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-09-10T07:58:10.981 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-09-10T07:58:10.981 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T07:58:10.983 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph Dashboard is now available at: 2024-09-10T07:58:10.983 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:58:10.983 INFO:teuthology.orchestra.run.smithi002.stdout: URL: https://smithi002.front.sepia.ceph.com:8443/ 2024-09-10T07:58:10.983 INFO:teuthology.orchestra.run.smithi002.stdout: User: admin 2024-09-10T07:58:10.984 INFO:teuthology.orchestra.run.smithi002.stdout: Password: a7enfirajt 2024-09-10T07:58:10.984 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:58:10.984 INFO:teuthology.orchestra.run.smithi002.stdout:Saving cluster configuration to /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config directory 2024-09-10T07:58:11.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:11 smithi002 ceph-mon[30563]: mgrmap e13: x(active, since 6s) 2024-09-10T07:58:12.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.486+0000 7f9fc6000640 1 Processor -- start 2024-09-10T07:58:12.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.486+0000 7f9fc6000640 1 -- start start 2024-09-10T07:58:12.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.486+0000 7f9fc6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 0x7f9fc0109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:12.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.486+0000 7f9fc6000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc010a2b0 con 0x7f9fc01098e0 2024-09-10T07:58:12.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.486+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 0x7f9fc0109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:12.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.487+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 0x7f9fc0109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42658/0 (socket says 172.21.15.2:42658) 2024-09-10T07:58:12.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.487+0000 7f9fbf400640 1 -- 172.21.15.2:0/1007594985 learned_addr learned my addr 172.21.15.2:0/1007594985 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:12.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.487+0000 7f9fbf400640 1 -- 172.21.15.2:0/1007594985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc010aae0 con 0x7f9fc01098e0 2024-09-10T07:58:12.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.488+0000 7f9fbf400640 1 --2- 172.21.15.2:0/1007594985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 0x7f9fc0109ce0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009820 tx=0x7f9fb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f683cfb6b58c241 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:12.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.488+0000 7f9fbea00640 1 -- 172.21.15.2:0/1007594985 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fb0037070 con 0x7f9fc01098e0 2024-09-10T07:58:12.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.488+0000 7f9fbea00640 1 -- 172.21.15.2:0/1007594985 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb0032920 con 0x7f9fc01098e0 2024-09-10T07:58:12.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.488+0000 7f9fbea00640 1 -- 172.21.15.2:0/1007594985 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fb0030810 con 0x7f9fc01098e0 2024-09-10T07:58:12.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.489+0000 7f9fc6000640 1 -- 172.21.15.2:0/1007594985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 msgr2=0x7f9fc0109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:12.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.489+0000 7f9fc6000640 1 --2- 172.21.15.2:0/1007594985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 0x7f9fc0109ce0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009820 tx=0x7f9fb002a270 comp rx=0 tx=0).stop 2024-09-10T07:58:12.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.489+0000 7f9fc6000640 1 -- 172.21.15.2:0/1007594985 shutdown_connections 2024-09-10T07:58:12.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.489+0000 7f9fc6000640 1 --2- 172.21.15.2:0/1007594985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 0x7f9fc0109ce0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:12.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.489+0000 7f9fc6000640 1 -- 172.21.15.2:0/1007594985 >> 172.21.15.2:0/1007594985 conn(0x7f9fc007ceb0 msgr2=0x7f9fc007d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:12.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.489+0000 7f9fc6000640 1 -- 172.21.15.2:0/1007594985 shutdown_connections 2024-09-10T07:58:12.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.489+0000 7f9fc6000640 1 -- 172.21.15.2:0/1007594985 wait complete. 2024-09-10T07:58:12.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.490+0000 7f9fc6000640 1 Processor -- start 2024-09-10T07:58:12.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.490+0000 7f9fc6000640 1 -- start start 2024-09-10T07:58:12.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.490+0000 7f9fc6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 0x7f9fc019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:12.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.490+0000 7f9fc6000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc010de90 con 0x7f9fc01098e0 2024-09-10T07:58:12.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.491+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 0x7f9fc019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:12.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.491+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 0x7f9fc019d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42674/0 (socket says 172.21.15.2:42674) 2024-09-10T07:58:12.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.491+0000 7f9fbf400640 1 -- 172.21.15.2:0/1609308653 learned_addr learned my addr 172.21.15.2:0/1609308653 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:12.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.491+0000 7f9fbf400640 1 -- 172.21.15.2:0/1609308653 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc019df00 con 0x7f9fc01098e0 2024-09-10T07:58:12.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.491+0000 7f9fbf400640 1 --2- 172.21.15.2:0/1609308653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 0x7f9fc019d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00097f0 tx=0x7f9fb00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:12.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.491+0000 7f9fbcc00640 1 -- 172.21.15.2:0/1609308653 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fb0037040 con 0x7f9fc01098e0 2024-09-10T07:58:12.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.491+0000 7f9fbcc00640 1 -- 172.21.15.2:0/1609308653 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb0030cc0 con 0x7f9fc01098e0 2024-09-10T07:58:12.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.491+0000 7f9fbcc00640 1 -- 172.21.15.2:0/1609308653 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fb003a640 con 0x7f9fc01098e0 2024-09-10T07:58:12.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.491+0000 7f9fc6000640 1 -- 172.21.15.2:0/1609308653 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc019e190 con 0x7f9fc01098e0 2024-09-10T07:58:12.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.492+0000 7f9fc6000640 1 -- 172.21.15.2:0/1609308653 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc01a0e80 con 0x7f9fc01098e0 2024-09-10T07:58:12.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.493+0000 7f9fbcc00640 1 -- 172.21.15.2:0/1609308653 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f9fb003b3f0 con 0x7f9fc01098e0 2024-09-10T07:58:12.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.493+0000 7f9fc6000640 1 -- 172.21.15.2:0/1609308653 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc010e010 con 0x7f9fc01098e0 2024-09-10T07:58:12.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.493+0000 7f9fbcc00640 1 --2- 172.21.15.2:0/1609308653 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9fa40390c0 0x7f9fa403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:12.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.493+0000 7f9fbcc00640 1 -- 172.21.15.2:0/1609308653 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9fb006c7a0 con 0x7f9fc01098e0 2024-09-10T07:58:12.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.493+0000 7f9fb7e00640 1 --2- 172.21.15.2:0/1609308653 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9fa40390c0 0x7f9fa403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:12.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.494+0000 7f9fb7e00640 1 --2- 172.21.15.2:0/1609308653 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9fa40390c0 0x7f9fa403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80098c0 tx=0x7f9fa8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:12.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.498+0000 7f9fbcc00640 1 -- 172.21.15.2:0/1609308653 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9fb00328c0 con 0x7f9fc01098e0 2024-09-10T07:58:12.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.785+0000 7f9fc6000640 1 -- 172.21.15.2:0/1609308653 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f9fc01a1050 con 0x7f9fc01098e0 2024-09-10T07:58:12.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.788+0000 7f9fbcc00640 1 -- 172.21.15.2:0/1609308653 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7f9fb0041890 con 0x7f9fc01098e0 2024-09-10T07:58:12.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T07:58:12.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.790+0000 7f9fc6000640 1 -- 172.21.15.2:0/1609308653 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9fa40390c0 msgr2=0x7f9fa403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:12.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.790+0000 7f9fc6000640 1 --2- 172.21.15.2:0/1609308653 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9fa40390c0 0x7f9fa403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80098c0 tx=0x7f9fa8006db0 comp rx=0 tx=0).stop 2024-09-10T07:58:12.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.790+0000 7f9fc6000640 1 -- 172.21.15.2:0/1609308653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 msgr2=0x7f9fc019d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:12.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.790+0000 7f9fc6000640 1 --2- 172.21.15.2:0/1609308653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 0x7f9fc019d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00097f0 tx=0x7f9fb00327b0 comp rx=0 tx=0).stop 2024-09-10T07:58:12.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.791+0000 7f9fc6000640 1 -- 172.21.15.2:0/1609308653 shutdown_connections 2024-09-10T07:58:12.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.791+0000 7f9fc6000640 1 --2- 172.21.15.2:0/1609308653 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9fa40390c0 0x7f9fa403b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:12.398 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.791+0000 7f9fc6000640 1 --2- 172.21.15.2:0/1609308653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9fc01098e0 0x7f9fc019d9c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:12.398 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.791+0000 7f9fc6000640 1 -- 172.21.15.2:0/1609308653 >> 172.21.15.2:0/1609308653 conn(0x7f9fc007ceb0 msgr2=0x7f9fc01075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:12.398 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.791+0000 7f9fc6000640 1 -- 172.21.15.2:0/1609308653 shutdown_connections 2024-09-10T07:58:12.398 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-09-10T07:58:11.791+0000 7f9fc6000640 1 -- 172.21.15.2:0/1609308653 wait complete. 2024-09-10T07:58:12.398 INFO:teuthology.orchestra.run.smithi002.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T07:58:12.398 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:58:12.398 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T07:58:12.398 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:58:12.398 INFO:teuthology.orchestra.run.smithi002.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T07:58:12.398 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:58:12.398 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T07:58:12.398 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:58:12.398 INFO:teuthology.orchestra.run.smithi002.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T07:58:12.399 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:58:12.399 INFO:teuthology.orchestra.run.smithi002.stdout: ceph telemetry on 2024-09-10T07:58:12.399 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:58:12.399 INFO:teuthology.orchestra.run.smithi002.stdout:For more information see: 2024-09-10T07:58:12.399 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:58:12.399 INFO:teuthology.orchestra.run.smithi002.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T07:58:12.399 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:58:12.399 INFO:teuthology.orchestra.run.smithi002.stdout:Bootstrap complete. 2024-09-10T07:58:12.450 INFO:tasks.cephadm:Fetching config... 2024-09-10T07:58:12.451 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:58:12.451 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T07:58:12.487 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T07:58:12.487 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:58:12.488 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T07:58:12.563 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T07:58:12.563 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:58:12.563 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T07:58:12.661 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T07:58:12.661 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:58:12.662 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T07:58:12.736 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T07:58:12.736 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp3MSJJbXjGKLV/1XS92rApG62zlBXKbNJcfq12m5Ksfh7SmEd2+xAS3tSKmSbpuTdAzqUTL54hgXydbc9ea+zSX8h8weaOcilwjqxrV7Xe3ZxS+4235t8TvOL09p+m0AmDqTgpQOt1+hARY3P68HUNsyMsXii1JU1og/LqwIDAX9Oslk+MjxxrdZTxj57PuFqpqYUiZoDjoD7LyPFkmiHJggJRjBTyo00vWqPTF3chXYWdHFolbAuP99XIaLo/mG14iBxzAnX0ewI6+rBkIqdTtRUwxmyPJh/aj7F7pcjs4y9jns57wZkU7wZ+cmoFpoXGDMGNWtxECy2UFrnuSzpbYo7tpvjVB+9uuRDhrLoa+BUNLGBt1nYXjlovvaTphsBKT+t3wDDDmP5hgVL3UAGQa07A+zsn7d5OLO4KkyjvDfIvudi3b2QKEZJGz0YUi5OMeIaGA1FxM5N3ztAIRu5Qd5Ur1sip8qdzxA5OgQhFlTwQBUY6oaxLP2yMPKANGU= ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T07:58:12.865 INFO:teuthology.orchestra.run.smithi002.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp3MSJJbXjGKLV/1XS92rApG62zlBXKbNJcfq12m5Ksfh7SmEd2+xAS3tSKmSbpuTdAzqUTL54hgXydbc9ea+zSX8h8weaOcilwjqxrV7Xe3ZxS+4235t8TvOL09p+m0AmDqTgpQOt1+hARY3P68HUNsyMsXii1JU1og/LqwIDAX9Oslk+MjxxrdZTxj57PuFqpqYUiZoDjoD7LyPFkmiHJggJRjBTyo00vWqPTF3chXYWdHFolbAuP99XIaLo/mG14iBxzAnX0ewI6+rBkIqdTtRUwxmyPJh/aj7F7pcjs4y9jns57wZkU7wZ+cmoFpoXGDMGNWtxECy2UFrnuSzpbYo7tpvjVB+9uuRDhrLoa+BUNLGBt1nYXjlovvaTphsBKT+t3wDDDmP5hgVL3UAGQa07A+zsn7d5OLO4KkyjvDfIvudi3b2QKEZJGz0YUi5OMeIaGA1FxM5N3ztAIRu5Qd5Ur1sip8qdzxA5OgQhFlTwQBUY6oaxLP2yMPKANGU= ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:12.890 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp3MSJJbXjGKLV/1XS92rApG62zlBXKbNJcfq12m5Ksfh7SmEd2+xAS3tSKmSbpuTdAzqUTL54hgXydbc9ea+zSX8h8weaOcilwjqxrV7Xe3ZxS+4235t8TvOL09p+m0AmDqTgpQOt1+hARY3P68HUNsyMsXii1JU1og/LqwIDAX9Oslk+MjxxrdZTxj57PuFqpqYUiZoDjoD7LyPFkmiHJggJRjBTyo00vWqPTF3chXYWdHFolbAuP99XIaLo/mG14iBxzAnX0ewI6+rBkIqdTtRUwxmyPJh/aj7F7pcjs4y9jns57wZkU7wZ+cmoFpoXGDMGNWtxECy2UFrnuSzpbYo7tpvjVB+9uuRDhrLoa+BUNLGBt1nYXjlovvaTphsBKT+t3wDDDmP5hgVL3UAGQa07A+zsn7d5OLO4KkyjvDfIvudi3b2QKEZJGz0YUi5OMeIaGA1FxM5N3ztAIRu5Qd5Ur1sip8qdzxA5OgQhFlTwQBUY6oaxLP2yMPKANGU= ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T07:58:12.932 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp3MSJJbXjGKLV/1XS92rApG62zlBXKbNJcfq12m5Ksfh7SmEd2+xAS3tSKmSbpuTdAzqUTL54hgXydbc9ea+zSX8h8weaOcilwjqxrV7Xe3ZxS+4235t8TvOL09p+m0AmDqTgpQOt1+hARY3P68HUNsyMsXii1JU1og/LqwIDAX9Oslk+MjxxrdZTxj57PuFqpqYUiZoDjoD7LyPFkmiHJggJRjBTyo00vWqPTF3chXYWdHFolbAuP99XIaLo/mG14iBxzAnX0ewI6+rBkIqdTtRUwxmyPJh/aj7F7pcjs4y9jns57wZkU7wZ+cmoFpoXGDMGNWtxECy2UFrnuSzpbYo7tpvjVB+9uuRDhrLoa+BUNLGBt1nYXjlovvaTphsBKT+t3wDDDmP5hgVL3UAGQa07A+zsn7d5OLO4KkyjvDfIvudi3b2QKEZJGz0YUi5OMeIaGA1FxM5N3ztAIRu5Qd5Ur1sip8qdzxA5OgQhFlTwQBUY6oaxLP2yMPKANGU= ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:12.944 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp3MSJJbXjGKLV/1XS92rApG62zlBXKbNJcfq12m5Ksfh7SmEd2+xAS3tSKmSbpuTdAzqUTL54hgXydbc9ea+zSX8h8weaOcilwjqxrV7Xe3ZxS+4235t8TvOL09p+m0AmDqTgpQOt1+hARY3P68HUNsyMsXii1JU1og/LqwIDAX9Oslk+MjxxrdZTxj57PuFqpqYUiZoDjoD7LyPFkmiHJggJRjBTyo00vWqPTF3chXYWdHFolbAuP99XIaLo/mG14iBxzAnX0ewI6+rBkIqdTtRUwxmyPJh/aj7F7pcjs4y9jns57wZkU7wZ+cmoFpoXGDMGNWtxECy2UFrnuSzpbYo7tpvjVB+9uuRDhrLoa+BUNLGBt1nYXjlovvaTphsBKT+t3wDDDmP5hgVL3UAGQa07A+zsn7d5OLO4KkyjvDfIvudi3b2QKEZJGz0YUi5OMeIaGA1FxM5N3ztAIRu5Qd5Ur1sip8qdzxA5OgQhFlTwQBUY6oaxLP2yMPKANGU= ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T07:58:12.970 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:12 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1609308653' entity='client.admin' 2024-09-10T07:58:12.985 INFO:teuthology.orchestra.run.smithi155.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp3MSJJbXjGKLV/1XS92rApG62zlBXKbNJcfq12m5Ksfh7SmEd2+xAS3tSKmSbpuTdAzqUTL54hgXydbc9ea+zSX8h8weaOcilwjqxrV7Xe3ZxS+4235t8TvOL09p+m0AmDqTgpQOt1+hARY3P68HUNsyMsXii1JU1og/LqwIDAX9Oslk+MjxxrdZTxj57PuFqpqYUiZoDjoD7LyPFkmiHJggJRjBTyo00vWqPTF3chXYWdHFolbAuP99XIaLo/mG14iBxzAnX0ewI6+rBkIqdTtRUwxmyPJh/aj7F7pcjs4y9jns57wZkU7wZ+cmoFpoXGDMGNWtxECy2UFrnuSzpbYo7tpvjVB+9uuRDhrLoa+BUNLGBt1nYXjlovvaTphsBKT+t3wDDDmP5hgVL3UAGQa07A+zsn7d5OLO4KkyjvDfIvudi3b2QKEZJGz0YUi5OMeIaGA1FxM5N3ztAIRu5Qd5Ur1sip8qdzxA5OgQhFlTwQBUY6oaxLP2yMPKANGU= ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:12.998 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T07:58:13.373 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:58:13.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.970+0000 7f24b3200640 1 -- 172.21.15.2:0/3680271279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24b4101420 msgr2=0x7f24b4101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:13.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.970+0000 7f24b3200640 1 --2- 172.21.15.2:0/3680271279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24b4101420 0x7f24b4101820 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009880 tx=0x7f24a402a5d0 comp rx=0 tx=0).stop 2024-09-10T07:58:13.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.970+0000 7f24b3200640 1 -- 172.21.15.2:0/3680271279 shutdown_connections 2024-09-10T07:58:13.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.970+0000 7f24b3200640 1 --2- 172.21.15.2:0/3680271279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24b4101420 0x7f24b4101820 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:13.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.970+0000 7f24b3200640 1 -- 172.21.15.2:0/3680271279 >> 172.21.15.2:0/3680271279 conn(0x7f24b40fcbd0 msgr2=0x7f24b40feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:13.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.971+0000 7f24b3200640 1 -- 172.21.15.2:0/3680271279 shutdown_connections 2024-09-10T07:58:13.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.971+0000 7f24b3200640 1 -- 172.21.15.2:0/3680271279 wait complete. 2024-09-10T07:58:13.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.971+0000 7f24b3200640 1 Processor -- start 2024-09-10T07:58:13.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.972+0000 7f24b3200640 1 -- start start 2024-09-10T07:58:13.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.972+0000 7f24b3200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24b4101420 0x7f24b4075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:13.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.972+0000 7f24b3200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f24b4107530 con 0x7f24b4101420 2024-09-10T07:58:13.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.972+0000 7f24b1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24b4101420 0x7f24b4075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:13.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.972+0000 7f24b1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24b4101420 0x7f24b4075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42682/0 (socket says 172.21.15.2:42682) 2024-09-10T07:58:13.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.972+0000 7f24b1e00640 1 -- 172.21.15.2:0/2672940190 learned_addr learned my addr 172.21.15.2:0/2672940190 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:13.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.973+0000 7f24b1e00640 1 -- 172.21.15.2:0/2672940190 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b40763f0 con 0x7f24b4101420 2024-09-10T07:58:13.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.973+0000 7f24b1e00640 1 --2- 172.21.15.2:0/2672940190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24b4101420 0x7f24b4075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f24a4004f60 tx=0x7f24a40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:13.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.973+0000 7f24aaa00640 1 -- 172.21.15.2:0/2672940190 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24a403f070 con 0x7f24b4101420 2024-09-10T07:58:13.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.973+0000 7f24aaa00640 1 -- 172.21.15.2:0/2672940190 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a4004cc0 con 0x7f24b4101420 2024-09-10T07:58:13.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.973+0000 7f24b3200640 1 -- 172.21.15.2:0/2672940190 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b40745e0 con 0x7f24b4101420 2024-09-10T07:58:13.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.973+0000 7f24aaa00640 1 -- 172.21.15.2:0/2672940190 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24a4035030 con 0x7f24b4101420 2024-09-10T07:58:13.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.973+0000 7f24b3200640 1 -- 172.21.15.2:0/2672940190 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b4074b20 con 0x7f24b4101420 2024-09-10T07:58:13.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.975+0000 7f24b3200640 1 -- 172.21.15.2:0/2672940190 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b4107cb0 con 0x7f24b4101420 2024-09-10T07:58:13.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.975+0000 7f24aaa00640 1 -- 172.21.15.2:0/2672940190 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f24a4042070 con 0x7f24b4101420 2024-09-10T07:58:13.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.975+0000 7f24aaa00640 1 --2- 172.21.15.2:0/2672940190 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f24980390c0 0x7f249803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:13.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.976+0000 7f24aaa00640 1 -- 172.21.15.2:0/2672940190 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f24a406cd40 con 0x7f24b4101420 2024-09-10T07:58:13.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.976+0000 7f24b1400640 1 --2- 172.21.15.2:0/2672940190 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f24980390c0 0x7f249803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:13.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.977+0000 7f24b1400640 1 --2- 172.21.15.2:0/2672940190 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f24980390c0 0x7f249803b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f24a00098c0 tx=0x7f24a0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:13.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:13.981+0000 7f24aaa00640 1 -- 172.21.15.2:0/2672940190 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24a403a3c0 con 0x7f24b4101420 2024-09-10T07:58:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:14.169+0000 7f24b3200640 1 -- 172.21.15.2:0/2672940190 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f24b418fff0 con 0x7f24b4101420 2024-09-10T07:58:14.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:14.172+0000 7f24aaa00640 1 -- 172.21.15.2:0/2672940190 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f24a4003dd0 con 0x7f24b4101420 2024-09-10T07:58:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:14.175+0000 7f24b3200640 1 -- 172.21.15.2:0/2672940190 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f24980390c0 msgr2=0x7f249803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:14.175+0000 7f24b3200640 1 --2- 172.21.15.2:0/2672940190 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f24980390c0 0x7f249803b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f24a00098c0 tx=0x7f24a0006db0 comp rx=0 tx=0).stop 2024-09-10T07:58:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:14.175+0000 7f24b3200640 1 -- 172.21.15.2:0/2672940190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24b4101420 msgr2=0x7f24b4075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:14.175+0000 7f24b3200640 1 --2- 172.21.15.2:0/2672940190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24b4101420 0x7f24b4075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f24a4004f60 tx=0x7f24a40048f0 comp rx=0 tx=0).stop 2024-09-10T07:58:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:14.175+0000 7f24b3200640 1 -- 172.21.15.2:0/2672940190 shutdown_connections 2024-09-10T07:58:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:14.175+0000 7f24b3200640 1 --2- 172.21.15.2:0/2672940190 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f24980390c0 0x7f249803b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:14.175+0000 7f24b3200640 1 --2- 172.21.15.2:0/2672940190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24b4101420 0x7f24b4075eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:14.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:14.175+0000 7f24b3200640 1 -- 172.21.15.2:0/2672940190 >> 172.21.15.2:0/2672940190 conn(0x7f24b40fcbd0 msgr2=0x7f24b40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:14.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:14.175+0000 7f24b3200640 1 -- 172.21.15.2:0/2672940190 shutdown_connections 2024-09-10T07:58:14.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:14.176+0000 7f24b3200640 1 -- 172.21.15.2:0/2672940190 wait complete. 2024-09-10T07:58:15.286 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T07:58:15.286 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T07:58:15.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:15 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2672940190' entity='client.admin' 2024-09-10T07:58:15.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:15 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:15.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:15 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:15.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:15 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi002", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:58:15.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:15 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:15.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:15 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:58:15.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:15 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:15.655 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:58:16.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.151+0000 7fce5f200640 1 -- 172.21.15.2:0/406649013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce60103640 msgr2=0x7fce60103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:16.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.151+0000 7fce5f200640 1 --2- 172.21.15.2:0/406649013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce60103640 0x7fce60103a40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fce50009880 tx=0x7fce5002a5d0 comp rx=0 tx=0).stop 2024-09-10T07:58:16.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.151+0000 7fce5f200640 1 -- 172.21.15.2:0/406649013 shutdown_connections 2024-09-10T07:58:16.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.151+0000 7fce5f200640 1 --2- 172.21.15.2:0/406649013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce60103640 0x7fce60103a40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:16.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.151+0000 7fce5f200640 1 -- 172.21.15.2:0/406649013 >> 172.21.15.2:0/406649013 conn(0x7fce600fedd0 msgr2=0x7fce60101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:16.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.151+0000 7fce5f200640 1 -- 172.21.15.2:0/406649013 shutdown_connections 2024-09-10T07:58:16.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.151+0000 7fce5f200640 1 -- 172.21.15.2:0/406649013 wait complete. 2024-09-10T07:58:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.152+0000 7fce5f200640 1 Processor -- start 2024-09-10T07:58:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.152+0000 7fce5f200640 1 -- start start 2024-09-10T07:58:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.152+0000 7fce5f200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce60103640 0x7fce60198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.152+0000 7fce5f200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fce60109750 con 0x7fce60103640 2024-09-10T07:58:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.152+0000 7fce5de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce60103640 0x7fce60198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.153+0000 7fce5de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce60103640 0x7fce60198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42696/0 (socket says 172.21.15.2:42696) 2024-09-10T07:58:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.153+0000 7fce5de00640 1 -- 172.21.15.2:0/4101012472 learned_addr learned my addr 172.21.15.2:0/4101012472 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.153+0000 7fce5de00640 1 -- 172.21.15.2:0/4101012472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce60199340 con 0x7fce60103640 2024-09-10T07:58:16.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.153+0000 7fce5de00640 1 --2- 172.21.15.2:0/4101012472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce60103640 0x7fce60198e00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fce50004f60 tx=0x7fce500048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:16.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.153+0000 7fce56a00640 1 -- 172.21.15.2:0/4101012472 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce5003f070 con 0x7fce60103640 2024-09-10T07:58:16.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.153+0000 7fce56a00640 1 -- 172.21.15.2:0/4101012472 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce50004d40 con 0x7fce60103640 2024-09-10T07:58:16.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.153+0000 7fce56a00640 1 -- 172.21.15.2:0/4101012472 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce50035030 con 0x7fce60103640 2024-09-10T07:58:16.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.153+0000 7fce5f200640 1 -- 172.21.15.2:0/4101012472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce601995d0 con 0x7fce60103640 2024-09-10T07:58:16.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.154+0000 7fce5f200640 1 -- 172.21.15.2:0/4101012472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6019c2c0 con 0x7fce60103640 2024-09-10T07:58:16.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.155+0000 7fce56a00640 1 -- 172.21.15.2:0/4101012472 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fce50042070 con 0x7fce60103640 2024-09-10T07:58:16.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.155+0000 7fce5f200640 1 -- 172.21.15.2:0/4101012472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce60109ed0 con 0x7fce60103640 2024-09-10T07:58:16.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.155+0000 7fce56a00640 1 --2- 172.21.15.2:0/4101012472 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce440390c0 0x7fce4403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:16.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.155+0000 7fce56a00640 1 -- 172.21.15.2:0/4101012472 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fce5006cca0 con 0x7fce60103640 2024-09-10T07:58:16.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.156+0000 7fce5d400640 1 --2- 172.21.15.2:0/4101012472 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce440390c0 0x7fce4403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:16.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.156+0000 7fce5d400640 1 --2- 172.21.15.2:0/4101012472 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce440390c0 0x7fce4403b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0098c0 tx=0x7fce4c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:16.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.161+0000 7fce56a00640 1 -- 172.21.15.2:0/4101012472 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce5004a260 con 0x7fce60103640 2024-09-10T07:58:16.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.349+0000 7fce5f200640 1 -- 172.21.15.2:0/4101012472 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fce6002d830 con 0x7fce440390c0 2024-09-10T07:58:16.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.353+0000 7fce56a00640 1 -- 172.21.15.2:0/4101012472 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fce6002d830 con 0x7fce440390c0 2024-09-10T07:58:16.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.355+0000 7fce5f200640 1 -- 172.21.15.2:0/4101012472 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce440390c0 msgr2=0x7fce4403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:16.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.355+0000 7fce5f200640 1 --2- 172.21.15.2:0/4101012472 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce440390c0 0x7fce4403b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0098c0 tx=0x7fce4c006db0 comp rx=0 tx=0).stop 2024-09-10T07:58:16.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.355+0000 7fce5f200640 1 -- 172.21.15.2:0/4101012472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce60103640 msgr2=0x7fce60198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:16.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.355+0000 7fce5f200640 1 --2- 172.21.15.2:0/4101012472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce60103640 0x7fce60198e00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fce50004f60 tx=0x7fce500048f0 comp rx=0 tx=0).stop 2024-09-10T07:58:16.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.355+0000 7fce5f200640 1 -- 172.21.15.2:0/4101012472 shutdown_connections 2024-09-10T07:58:16.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.355+0000 7fce5f200640 1 --2- 172.21.15.2:0/4101012472 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce440390c0 0x7fce4403b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:16.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.355+0000 7fce5f200640 1 --2- 172.21.15.2:0/4101012472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce60103640 0x7fce60198e00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:16.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.355+0000 7fce5f200640 1 -- 172.21.15.2:0/4101012472 >> 172.21.15.2:0/4101012472 conn(0x7fce600fedd0 msgr2=0x7fce600ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:16.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.355+0000 7fce5f200640 1 -- 172.21.15.2:0/4101012472 shutdown_connections 2024-09-10T07:58:16.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:16.355+0000 7fce5f200640 1 -- 172.21.15.2:0/4101012472 wait complete. 2024-09-10T07:58:17.138 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi107 2024-09-10T07:58:17.139 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T07:58:17.139 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/etc/ceph/ceph.conf 2024-09-10T07:58:17.158 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T07:58:17.158 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:58:17.216 INFO:tasks.cephadm:Adding host smithi107 to orchestrator... 2024-09-10T07:58:17.217 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch host add smithi107 2024-09-10T07:58:17.597 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:58:17.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:17 smithi002 ceph-mon[30563]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:58:17.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:17 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:17.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:17 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:58:17.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:17 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:17.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:17 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:58:17.737 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:17 smithi002 ceph-mon[30563]: Updating smithi002:/etc/ceph/ceph.conf 2024-09-10T07:58:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.238+0000 7f9f25c00640 1 -- 172.21.15.2:0/593108061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f20107660 msgr2=0x7f9f20107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.238+0000 7f9f25c00640 1 --2- 172.21.15.2:0/593108061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f20107660 0x7f9f20107a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009880 tx=0x7f9f1002a5d0 comp rx=0 tx=0).stop 2024-09-10T07:58:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.238+0000 7f9f25c00640 1 -- 172.21.15.2:0/593108061 shutdown_connections 2024-09-10T07:58:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.238+0000 7f9f25c00640 1 --2- 172.21.15.2:0/593108061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f20107660 0x7f9f20107a40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.239+0000 7f9f25c00640 1 -- 172.21.15.2:0/593108061 >> 172.21.15.2:0/593108061 conn(0x7f9f200fd290 msgr2=0x7f9f200ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.239+0000 7f9f25c00640 1 -- 172.21.15.2:0/593108061 shutdown_connections 2024-09-10T07:58:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.239+0000 7f9f25c00640 1 -- 172.21.15.2:0/593108061 wait complete. 2024-09-10T07:58:18.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.240+0000 7f9f25c00640 1 Processor -- start 2024-09-10T07:58:18.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.240+0000 7f9f25c00640 1 -- start start 2024-09-10T07:58:18.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.241+0000 7f9f25c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f20107660 0x7f9f20197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:18.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.241+0000 7f9f25c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2010bc90 con 0x7f9f20107660 2024-09-10T07:58:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.241+0000 7f9f1ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f20107660 0x7f9f20197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.241+0000 7f9f1ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f20107660 0x7f9f20197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57268/0 (socket says 172.21.15.2:57268) 2024-09-10T07:58:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.241+0000 7f9f1ea00640 1 -- 172.21.15.2:0/2184346882 learned_addr learned my addr 172.21.15.2:0/2184346882 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.242+0000 7f9f1ea00640 1 -- 172.21.15.2:0/2184346882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f20197660 con 0x7f9f20107660 2024-09-10T07:58:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.242+0000 7f9f1ea00640 1 --2- 172.21.15.2:0/2184346882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f20107660 0x7f9f20197120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009230 tx=0x7f9f10003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.242+0000 7f9f17400640 1 -- 172.21.15.2:0/2184346882 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f1003f070 con 0x7f9f20107660 2024-09-10T07:58:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.242+0000 7f9f17400640 1 -- 172.21.15.2:0/2184346882 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f1003bd40 con 0x7f9f20107660 2024-09-10T07:58:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.242+0000 7f9f25c00640 1 -- 172.21.15.2:0/2184346882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f201978f0 con 0x7f9f20107660 2024-09-10T07:58:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.242+0000 7f9f17400640 1 -- 172.21.15.2:0/2184346882 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f10035030 con 0x7f9f20107660 2024-09-10T07:58:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.243+0000 7f9f25c00640 1 -- 172.21.15.2:0/2184346882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2019b9f0 con 0x7f9f20107660 2024-09-10T07:58:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.244+0000 7f9f17400640 1 -- 172.21.15.2:0/2184346882 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f9f10042070 con 0x7f9f20107660 2024-09-10T07:58:18.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.244+0000 7f9f25c00640 1 -- 172.21.15.2:0/2184346882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f20102cd0 con 0x7f9f20107660 2024-09-10T07:58:18.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.244+0000 7f9f17400640 1 --2- 172.21.15.2:0/2184346882 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9f040390c0 0x7f9f0403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:18.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.244+0000 7f9f17400640 1 -- 172.21.15.2:0/2184346882 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9f1006cd20 con 0x7f9f20107660 2024-09-10T07:58:18.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.245+0000 7f9f1e000640 1 --2- 172.21.15.2:0/2184346882 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9f040390c0 0x7f9f0403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.245+0000 7f9f1e000640 1 --2- 172.21.15.2:0/2184346882 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9f040390c0 0x7f9f0403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0098c0 tx=0x7f9f0c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:18.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.254+0000 7f9f17400640 1 -- 172.21.15.2:0/2184346882 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f10030d60 con 0x7f9f20107660 2024-09-10T07:58:18.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:18.448+0000 7f9f25c00640 1 -- 172.21.15.2:0/2184346882 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "target": ["mon-mgr", ""]}) -- 0x7f9f20198480 con 0x7f9f040390c0 2024-09-10T07:58:18.715 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:18 smithi002 ceph-mon[30563]: Updating smithi002:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:18.715 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:18 smithi002 ceph-mon[30563]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:58:18.715 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:18 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:18.715 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:18 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:18.715 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:18 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:19.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:19 smithi002 ceph-mon[30563]: Updating smithi002:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T07:58:19.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:19 smithi002 ceph-mon[30563]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:58:21.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:20 smithi002 ceph-mon[30563]: Deploying cephadm binary to smithi107 2024-09-10T07:58:22.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:22.367+0000 7f9f17400640 1 -- 172.21.15.2:0/2184346882 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9f20198480 con 0x7f9f040390c0 2024-09-10T07:58:22.367 INFO:teuthology.orchestra.run.smithi002.stdout:Added host 'smithi107' with addr '172.21.15.107' 2024-09-10T07:58:22.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:22.370+0000 7f9f25c00640 1 -- 172.21.15.2:0/2184346882 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9f040390c0 msgr2=0x7f9f0403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:22.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:22.370+0000 7f9f25c00640 1 --2- 172.21.15.2:0/2184346882 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9f040390c0 0x7f9f0403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0098c0 tx=0x7f9f0c006db0 comp rx=0 tx=0).stop 2024-09-10T07:58:22.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:22.370+0000 7f9f25c00640 1 -- 172.21.15.2:0/2184346882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f20107660 msgr2=0x7f9f20197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:22.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:22.370+0000 7f9f25c00640 1 --2- 172.21.15.2:0/2184346882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f20107660 0x7f9f20197120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009230 tx=0x7f9f10003d80 comp rx=0 tx=0).stop 2024-09-10T07:58:22.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:22.370+0000 7f9f25c00640 1 -- 172.21.15.2:0/2184346882 shutdown_connections 2024-09-10T07:58:22.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:22.370+0000 7f9f25c00640 1 --2- 172.21.15.2:0/2184346882 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9f040390c0 0x7f9f0403b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:22.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:22.370+0000 7f9f25c00640 1 --2- 172.21.15.2:0/2184346882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f20107660 0x7f9f20197120 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:22.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:22.370+0000 7f9f25c00640 1 -- 172.21.15.2:0/2184346882 >> 172.21.15.2:0/2184346882 conn(0x7f9f200fd290 msgr2=0x7f9f200fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:22.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:22.370+0000 7f9f25c00640 1 -- 172.21.15.2:0/2184346882 shutdown_connections 2024-09-10T07:58:22.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:22.370+0000 7f9f25c00640 1 -- 172.21.15.2:0/2184346882 wait complete. 2024-09-10T07:58:23.014 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T07:58:23.382 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:58:23.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:23 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:23.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:23 smithi002 ceph-mon[30563]: Added host smithi107 2024-09-10T07:58:23.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:23 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:58:23.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:23 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:23.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:23 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:23.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.949+0000 7fdddb400640 1 -- 172.21.15.2:0/600459386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd4107660 msgr2=0x7fddd4107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:23.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.949+0000 7fdddb400640 1 --2- 172.21.15.2:0/600459386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd4107660 0x7fddd4107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009880 tx=0x7fddc802a5d0 comp rx=0 tx=0).stop 2024-09-10T07:58:23.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.949+0000 7fdddb400640 1 -- 172.21.15.2:0/600459386 shutdown_connections 2024-09-10T07:58:23.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.949+0000 7fdddb400640 1 --2- 172.21.15.2:0/600459386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd4107660 0x7fddd4107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:23.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.949+0000 7fdddb400640 1 -- 172.21.15.2:0/600459386 >> 172.21.15.2:0/600459386 conn(0x7fddd40fd290 msgr2=0x7fddd40ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:23.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.950+0000 7fdddb400640 1 -- 172.21.15.2:0/600459386 shutdown_connections 2024-09-10T07:58:23.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.950+0000 7fdddb400640 1 -- 172.21.15.2:0/600459386 wait complete. 2024-09-10T07:58:23.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.951+0000 7fdddb400640 1 Processor -- start 2024-09-10T07:58:23.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.951+0000 7fdddb400640 1 -- start start 2024-09-10T07:58:23.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.951+0000 7fdddb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd4107660 0x7fddd4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:23.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.951+0000 7fdddb400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fddd410bc90 con 0x7fddd4107660 2024-09-10T07:58:23.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.951+0000 7fddd8a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd4107660 0x7fddd4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:23.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.951+0000 7fddd8a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd4107660 0x7fddd4199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57294/0 (socket says 172.21.15.2:57294) 2024-09-10T07:58:23.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.951+0000 7fddd8a00640 1 -- 172.21.15.2:0/3198638229 learned_addr learned my addr 172.21.15.2:0/3198638229 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:23.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.952+0000 7fddd8a00640 1 -- 172.21.15.2:0/3198638229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd4199880 con 0x7fddd4107660 2024-09-10T07:58:23.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.952+0000 7fddd8a00640 1 --2- 172.21.15.2:0/3198638229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd4107660 0x7fddd4199340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009230 tx=0x7fddc8003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:23.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.952+0000 7fddd1600640 1 -- 172.21.15.2:0/3198638229 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddc803f070 con 0x7fddd4107660 2024-09-10T07:58:23.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.952+0000 7fddd1600640 1 -- 172.21.15.2:0/3198638229 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc803bd40 con 0x7fddd4107660 2024-09-10T07:58:23.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.952+0000 7fddd1600640 1 -- 172.21.15.2:0/3198638229 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddc8035030 con 0x7fddd4107660 2024-09-10T07:58:23.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.952+0000 7fdddb400640 1 -- 172.21.15.2:0/3198638229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd4199b10 con 0x7fddd4107660 2024-09-10T07:58:23.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.952+0000 7fdddb400640 1 -- 172.21.15.2:0/3198638229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd419dc10 con 0x7fddd4107660 2024-09-10T07:58:23.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.954+0000 7fdddb400640 1 -- 172.21.15.2:0/3198638229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd4102cd0 con 0x7fddd4107660 2024-09-10T07:58:23.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.954+0000 7fddd1600640 1 -- 172.21.15.2:0/3198638229 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fddc8042070 con 0x7fddd4107660 2024-09-10T07:58:23.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.954+0000 7fddd1600640 1 --2- 172.21.15.2:0/3198638229 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fddbc0390c0 0x7fddbc03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:23.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.954+0000 7fddd1600640 1 -- 172.21.15.2:0/3198638229 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fddc806ccd0 con 0x7fddd4107660 2024-09-10T07:58:23.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.954+0000 7fddd3e00640 1 --2- 172.21.15.2:0/3198638229 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fddbc0390c0 0x7fddbc03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:23.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.955+0000 7fddd3e00640 1 --2- 172.21.15.2:0/3198638229 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fddbc0390c0 0x7fddbc03b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fddc00098c0 tx=0x7fddc0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:23.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:23.959+0000 7fddd1600640 1 -- 172.21.15.2:0/3198638229 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fddc80412c0 con 0x7fddd4107660 2024-09-10T07:58:24.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:24.149+0000 7fdddb400640 1 -- 172.21.15.2:0/3198638229 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fddd419a6a0 con 0x7fddbc0390c0 2024-09-10T07:58:24.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:24.151+0000 7fddd1600640 1 -- 172.21.15.2:0/3198638229 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fddd419a6a0 con 0x7fddbc0390c0 2024-09-10T07:58:24.151 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:58:24.151 INFO:teuthology.orchestra.run.smithi002.stdout:[{"addr": "172.21.15.2", "hostname": "smithi002", "labels": [], "status": ""}, {"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}] 2024-09-10T07:58:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:24.153+0000 7fdddb400640 1 -- 172.21.15.2:0/3198638229 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fddbc0390c0 msgr2=0x7fddbc03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:24.153+0000 7fdddb400640 1 --2- 172.21.15.2:0/3198638229 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fddbc0390c0 0x7fddbc03b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fddc00098c0 tx=0x7fddc0006db0 comp rx=0 tx=0).stop 2024-09-10T07:58:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:24.153+0000 7fdddb400640 1 -- 172.21.15.2:0/3198638229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd4107660 msgr2=0x7fddd4199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:24.153+0000 7fdddb400640 1 --2- 172.21.15.2:0/3198638229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd4107660 0x7fddd4199340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009230 tx=0x7fddc8003d80 comp rx=0 tx=0).stop 2024-09-10T07:58:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:24.153+0000 7fdddb400640 1 -- 172.21.15.2:0/3198638229 shutdown_connections 2024-09-10T07:58:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:24.153+0000 7fdddb400640 1 --2- 172.21.15.2:0/3198638229 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fddbc0390c0 0x7fddbc03b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:24.153+0000 7fdddb400640 1 --2- 172.21.15.2:0/3198638229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd4107660 0x7fddd4199340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:24.153+0000 7fdddb400640 1 -- 172.21.15.2:0/3198638229 >> 172.21.15.2:0/3198638229 conn(0x7fddd40fd290 msgr2=0x7fddd4075e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:24.153+0000 7fdddb400640 1 -- 172.21.15.2:0/3198638229 shutdown_connections 2024-09-10T07:58:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:24.153+0000 7fdddb400640 1 -- 172.21.15.2:0/3198638229 wait complete. 2024-09-10T07:58:24.864 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:24 smithi002 ceph-mon[30563]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:24.864 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:24 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:24.864 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:24 smithi002 ceph-mon[30563]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:58:24.901 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi155 2024-09-10T07:58:24.901 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-10T07:58:24.901 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.conf 2024-09-10T07:58:24.924 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-10T07:58:24.925 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:58:24.987 INFO:tasks.cephadm:Adding host smithi155 to orchestrator... 2024-09-10T07:58:24.987 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch host add smithi155 2024-09-10T07:58:25.355 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:58:25.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.856+0000 7fc888400640 1 -- 172.21.15.2:0/2893897143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc880109770 msgr2=0x7fc880109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:25.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.856+0000 7fc888400640 1 --2- 172.21.15.2:0/2893897143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc880109770 0x7fc880109b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc874009c20 tx=0x7fc87402aa90 comp rx=0 tx=0).stop 2024-09-10T07:58:25.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.856+0000 7fc888400640 1 -- 172.21.15.2:0/2893897143 shutdown_connections 2024-09-10T07:58:25.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.856+0000 7fc888400640 1 --2- 172.21.15.2:0/2893897143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc880109770 0x7fc880109b50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:25.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.856+0000 7fc888400640 1 -- 172.21.15.2:0/2893897143 >> 172.21.15.2:0/2893897143 conn(0x7fc8800ff450 msgr2=0x7fc880101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:25.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.856+0000 7fc888400640 1 -- 172.21.15.2:0/2893897143 shutdown_connections 2024-09-10T07:58:25.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.856+0000 7fc888400640 1 -- 172.21.15.2:0/2893897143 wait complete. 2024-09-10T07:58:25.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.857+0000 7fc888400640 1 Processor -- start 2024-09-10T07:58:25.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.857+0000 7fc888400640 1 -- start start 2024-09-10T07:58:25.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.857+0000 7fc888400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc880109770 0x7fc8801993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:25.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.857+0000 7fc888400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc88010de30 con 0x7fc880109770 2024-09-10T07:58:25.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.857+0000 7fc885a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc880109770 0x7fc8801993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:25.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.857+0000 7fc885a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc880109770 0x7fc8801993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57314/0 (socket says 172.21.15.2:57314) 2024-09-10T07:58:25.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.857+0000 7fc885a00640 1 -- 172.21.15.2:0/1817982124 learned_addr learned my addr 172.21.15.2:0/1817982124 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:25.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.858+0000 7fc885a00640 1 -- 172.21.15.2:0/1817982124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8801998e0 con 0x7fc880109770 2024-09-10T07:58:25.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.858+0000 7fc885a00640 1 --2- 172.21.15.2:0/1817982124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc880109770 0x7fc8801993a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc874009d40 tx=0x7fc87402fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:25.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.858+0000 7fc87e000640 1 -- 172.21.15.2:0/1817982124 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc874047070 con 0x7fc880109770 2024-09-10T07:58:25.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.858+0000 7fc87e000640 1 -- 172.21.15.2:0/1817982124 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc874003e40 con 0x7fc880109770 2024-09-10T07:58:25.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.858+0000 7fc888400640 1 -- 172.21.15.2:0/1817982124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc880199b70 con 0x7fc880109770 2024-09-10T07:58:25.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.858+0000 7fc87e000640 1 -- 172.21.15.2:0/1817982124 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc874033050 con 0x7fc880109770 2024-09-10T07:58:25.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.858+0000 7fc888400640 1 -- 172.21.15.2:0/1817982124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc880199fd0 con 0x7fc880109770 2024-09-10T07:58:25.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.859+0000 7fc87e000640 1 -- 172.21.15.2:0/1817982124 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fc87404a070 con 0x7fc880109770 2024-09-10T07:58:25.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.860+0000 7fc888400640 1 -- 172.21.15.2:0/1817982124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc880104eb0 con 0x7fc880109770 2024-09-10T07:58:25.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.860+0000 7fc87e000640 1 --2- 172.21.15.2:0/1817982124 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc868039060 0x7fc86803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:25.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.860+0000 7fc87e000640 1 -- 172.21.15.2:0/1817982124 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc87403e030 con 0x7fc880109770 2024-09-10T07:58:25.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.860+0000 7fc885000640 1 --2- 172.21.15.2:0/1817982124 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc868039060 0x7fc86803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:25.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.861+0000 7fc885000640 1 --2- 172.21.15.2:0/1817982124 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc868039060 0x7fc86803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0098c0 tx=0x7fc86c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:25.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:25.865+0000 7fc87e000640 1 -- 172.21.15.2:0/1817982124 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc874004180 con 0x7fc880109770 2024-09-10T07:58:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:26.055+0000 7fc888400640 1 -- 172.21.15.2:0/1817982124 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}) -- 0x7fc88019a700 con 0x7fc868039060 2024-09-10T07:58:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:27 smithi002 ceph-mon[30563]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:27 smithi002 ceph-mon[30563]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:58:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:58:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:58:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:27 smithi002 ceph-mon[30563]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-10T07:58:27.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:27.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:27.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:28.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:28 smithi002 ceph-mon[30563]: Updating smithi107:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:28.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:28 smithi002 ceph-mon[30563]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:58:28.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:28 smithi002 ceph-mon[30563]: Updating smithi107:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T07:58:28.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:28 smithi002 ceph-mon[30563]: Deploying cephadm binary to smithi155 2024-09-10T07:58:29.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:29 smithi002 ceph-mon[30563]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:29.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:29.961+0000 7fc87e000640 1 -- 172.21.15.2:0/1817982124 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc88019a700 con 0x7fc868039060 2024-09-10T07:58:29.960 INFO:teuthology.orchestra.run.smithi002.stdout:Added host 'smithi155' with addr '172.21.15.155' 2024-09-10T07:58:29.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:29.963+0000 7fc888400640 1 -- 172.21.15.2:0/1817982124 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc868039060 msgr2=0x7fc86803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:29.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:29.963+0000 7fc888400640 1 --2- 172.21.15.2:0/1817982124 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc868039060 0x7fc86803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0098c0 tx=0x7fc86c006db0 comp rx=0 tx=0).stop 2024-09-10T07:58:29.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:29.963+0000 7fc888400640 1 -- 172.21.15.2:0/1817982124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc880109770 msgr2=0x7fc8801993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:29.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:29.963+0000 7fc888400640 1 --2- 172.21.15.2:0/1817982124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc880109770 0x7fc8801993a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc874009d40 tx=0x7fc87402fc10 comp rx=0 tx=0).stop 2024-09-10T07:58:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:29.964+0000 7fc888400640 1 -- 172.21.15.2:0/1817982124 shutdown_connections 2024-09-10T07:58:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:29.964+0000 7fc888400640 1 --2- 172.21.15.2:0/1817982124 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc868039060 0x7fc86803b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:29.964+0000 7fc888400640 1 --2- 172.21.15.2:0/1817982124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc880109770 0x7fc8801993a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:29.964+0000 7fc888400640 1 -- 172.21.15.2:0/1817982124 >> 172.21.15.2:0/1817982124 conn(0x7fc8800ff450 msgr2=0x7fc880107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:29.964+0000 7fc888400640 1 -- 172.21.15.2:0/1817982124 shutdown_connections 2024-09-10T07:58:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:29.964+0000 7fc888400640 1 -- 172.21.15.2:0/1817982124 wait complete. 2024-09-10T07:58:30.846 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T07:58:31.214 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:58:31.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:30 smithi002 ceph-mon[30563]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:30 smithi002 ceph-mon[30563]: Added host smithi155 2024-09-10T07:58:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:58:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:31.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.696+0000 7fb77ec00640 1 -- 172.21.15.2:0/2369016654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb778101890 msgr2=0x7fb778101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:31.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.696+0000 7fb77ec00640 1 --2- 172.21.15.2:0/2369016654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb778101890 0x7fb778101c70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009880 tx=0x7fb76c02a5d0 comp rx=0 tx=0).stop 2024-09-10T07:58:31.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.696+0000 7fb77ec00640 1 -- 172.21.15.2:0/2369016654 shutdown_connections 2024-09-10T07:58:31.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.696+0000 7fb77ec00640 1 --2- 172.21.15.2:0/2369016654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb778101890 0x7fb778101c70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:31.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.696+0000 7fb77ec00640 1 -- 172.21.15.2:0/2369016654 >> 172.21.15.2:0/2369016654 conn(0x7fb778077120 msgr2=0x7fb778077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:31.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.696+0000 7fb77ec00640 1 -- 172.21.15.2:0/2369016654 shutdown_connections 2024-09-10T07:58:31.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.696+0000 7fb77ec00640 1 -- 172.21.15.2:0/2369016654 wait complete. 2024-09-10T07:58:31.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.697+0000 7fb77ec00640 1 Processor -- start 2024-09-10T07:58:31.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.697+0000 7fb77ec00640 1 -- start start 2024-09-10T07:58:31.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.698+0000 7fb77ec00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb778101890 0x7fb778196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:31.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.698+0000 7fb77ec00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb77810ba40 con 0x7fb778101890 2024-09-10T07:58:31.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.698+0000 7fb77d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb778101890 0x7fb778196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:31.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.698+0000 7fb77d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb778101890 0x7fb778196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58272/0 (socket says 172.21.15.2:58272) 2024-09-10T07:58:31.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.698+0000 7fb77d800640 1 -- 172.21.15.2:0/1855970443 learned_addr learned my addr 172.21.15.2:0/1855970443 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:31.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.698+0000 7fb77d800640 1 -- 172.21.15.2:0/1855970443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7781974b0 con 0x7fb778101890 2024-09-10T07:58:31.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.699+0000 7fb77d800640 1 --2- 172.21.15.2:0/1855970443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb778101890 0x7fb778196f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009230 tx=0x7fb76c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:31.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.699+0000 7fb776000640 1 -- 172.21.15.2:0/1855970443 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c03f070 con 0x7fb778101890 2024-09-10T07:58:31.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.699+0000 7fb776000640 1 -- 172.21.15.2:0/1855970443 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb76c03bc80 con 0x7fb778101890 2024-09-10T07:58:31.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.699+0000 7fb77ec00640 1 -- 172.21.15.2:0/1855970443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7781977a0 con 0x7fb778101890 2024-09-10T07:58:31.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.699+0000 7fb776000640 1 -- 172.21.15.2:0/1855970443 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c035030 con 0x7fb778101890 2024-09-10T07:58:31.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.699+0000 7fb77ec00640 1 -- 172.21.15.2:0/1855970443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77819b840 con 0x7fb778101890 2024-09-10T07:58:31.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.700+0000 7fb776000640 1 -- 172.21.15.2:0/1855970443 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fb76c042070 con 0x7fb778101890 2024-09-10T07:58:31.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.700+0000 7fb77ec00640 1 -- 172.21.15.2:0/1855970443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb778102af0 con 0x7fb778101890 2024-09-10T07:58:31.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.700+0000 7fb776000640 1 --2- 172.21.15.2:0/1855970443 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7600390c0 0x7fb76003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:31.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.700+0000 7fb776000640 1 -- 172.21.15.2:0/1855970443 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb76c06cd60 con 0x7fb778101890 2024-09-10T07:58:31.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.701+0000 7fb77ce00640 1 --2- 172.21.15.2:0/1855970443 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7600390c0 0x7fb76003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:31.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.701+0000 7fb77ce00640 1 --2- 172.21.15.2:0/1855970443 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7600390c0 0x7fb76003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb7640098c0 tx=0x7fb764006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:31.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.706+0000 7fb776000640 1 -- 172.21.15.2:0/1855970443 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb76c0372d0 con 0x7fb778101890 2024-09-10T07:58:31.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.898+0000 7fb77ec00640 1 -- 172.21.15.2:0/1855970443 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb778101cf0 con 0x7fb7600390c0 2024-09-10T07:58:31.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.900+0000 7fb776000640 1 -- 172.21.15.2:0/1855970443 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fb778101cf0 con 0x7fb7600390c0 2024-09-10T07:58:31.899 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:58:31.899 INFO:teuthology.orchestra.run.smithi002.stdout:[{"addr": "172.21.15.2", "hostname": "smithi002", "labels": [], "status": ""}, {"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}] 2024-09-10T07:58:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.903+0000 7fb77ec00640 1 -- 172.21.15.2:0/1855970443 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7600390c0 msgr2=0x7fb76003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.903+0000 7fb77ec00640 1 --2- 172.21.15.2:0/1855970443 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7600390c0 0x7fb76003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb7640098c0 tx=0x7fb764006db0 comp rx=0 tx=0).stop 2024-09-10T07:58:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.903+0000 7fb77ec00640 1 -- 172.21.15.2:0/1855970443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb778101890 msgr2=0x7fb778196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.903+0000 7fb77ec00640 1 --2- 172.21.15.2:0/1855970443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb778101890 0x7fb778196f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009230 tx=0x7fb76c003d40 comp rx=0 tx=0).stop 2024-09-10T07:58:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.903+0000 7fb77ec00640 1 -- 172.21.15.2:0/1855970443 shutdown_connections 2024-09-10T07:58:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.903+0000 7fb77ec00640 1 --2- 172.21.15.2:0/1855970443 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7600390c0 0x7fb76003b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.903+0000 7fb77ec00640 1 --2- 172.21.15.2:0/1855970443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb778101890 0x7fb778196f70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.903+0000 7fb77ec00640 1 -- 172.21.15.2:0/1855970443 >> 172.21.15.2:0/1855970443 conn(0x7fb778077120 msgr2=0x7fb778102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.903+0000 7fb77ec00640 1 -- 172.21.15.2:0/1855970443 shutdown_connections 2024-09-10T07:58:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:31.903+0000 7fb77ec00640 1 -- 172.21.15.2:0/1855970443 wait complete. 2024-09-10T07:58:32.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:32 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:32.573 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T07:58:32.574 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T07:58:32.938 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:58:33.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:33 smithi002 ceph-mon[30563]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:33.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:33 smithi002 ceph-mon[30563]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:58:33.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.428+0000 7ff24f000640 1 -- 172.21.15.2:0/866846976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248107580 msgr2=0x7ff248107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:33.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.428+0000 7ff24f000640 1 --2- 172.21.15.2:0/866846976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248107580 0x7ff248107960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009880 tx=0x7ff23c02a5d0 comp rx=0 tx=0).stop 2024-09-10T07:58:33.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.428+0000 7ff24f000640 1 -- 172.21.15.2:0/866846976 shutdown_connections 2024-09-10T07:58:33.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.428+0000 7ff24f000640 1 --2- 172.21.15.2:0/866846976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248107580 0x7ff248107960 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:33.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.428+0000 7ff24f000640 1 -- 172.21.15.2:0/866846976 >> 172.21.15.2:0/866846976 conn(0x7ff248078260 msgr2=0x7ff2480ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:33.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.428+0000 7ff24f000640 1 -- 172.21.15.2:0/866846976 shutdown_connections 2024-09-10T07:58:33.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.429+0000 7ff24f000640 1 -- 172.21.15.2:0/866846976 wait complete. 2024-09-10T07:58:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.429+0000 7ff24f000640 1 Processor -- start 2024-09-10T07:58:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.430+0000 7ff24f000640 1 -- start start 2024-09-10T07:58:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.430+0000 7ff24f000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248107580 0x7ff2481991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.430+0000 7ff24f000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff24810bbb0 con 0x7ff248107580 2024-09-10T07:58:33.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.430+0000 7ff24dc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248107580 0x7ff2481991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:33.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.430+0000 7ff24dc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248107580 0x7ff2481991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58288/0 (socket says 172.21.15.2:58288) 2024-09-10T07:58:33.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.430+0000 7ff24dc00640 1 -- 172.21.15.2:0/2754387642 learned_addr learned my addr 172.21.15.2:0/2754387642 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:33.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.431+0000 7ff24dc00640 1 -- 172.21.15.2:0/2754387642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff248199730 con 0x7ff248107580 2024-09-10T07:58:33.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.431+0000 7ff24dc00640 1 --2- 172.21.15.2:0/2754387642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248107580 0x7ff2481991f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009230 tx=0x7ff23c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.431+0000 7ff246000640 1 -- 172.21.15.2:0/2754387642 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff23c03f070 con 0x7ff248107580 2024-09-10T07:58:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.431+0000 7ff246000640 1 -- 172.21.15.2:0/2754387642 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff23c03bd40 con 0x7ff248107580 2024-09-10T07:58:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.431+0000 7ff246000640 1 -- 172.21.15.2:0/2754387642 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff23c035030 con 0x7ff248107580 2024-09-10T07:58:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.431+0000 7ff24f000640 1 -- 172.21.15.2:0/2754387642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2481999c0 con 0x7ff248107580 2024-09-10T07:58:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.432+0000 7ff24f000640 1 -- 172.21.15.2:0/2754387642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24819dac0 con 0x7ff248107580 2024-09-10T07:58:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.433+0000 7ff246000640 1 -- 172.21.15.2:0/2754387642 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7ff23c042070 con 0x7ff248107580 2024-09-10T07:58:33.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.433+0000 7ff24f000640 1 -- 172.21.15.2:0/2754387642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff248102ae0 con 0x7ff248107580 2024-09-10T07:58:33.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.433+0000 7ff246000640 1 --2- 172.21.15.2:0/2754387642 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff2300390c0 0x7ff23003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:33.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.433+0000 7ff246000640 1 -- 172.21.15.2:0/2754387642 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff23c06ccd0 con 0x7ff248107580 2024-09-10T07:58:33.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.433+0000 7ff24d200640 1 --2- 172.21.15.2:0/2754387642 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff2300390c0 0x7ff23003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:33.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.434+0000 7ff24d200640 1 --2- 172.21.15.2:0/2754387642 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff2300390c0 0x7ff23003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff2340098c0 tx=0x7ff234006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:33.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.438+0000 7ff246000640 1 -- 172.21.15.2:0/2754387642 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff23c0412c0 con 0x7ff248107580 2024-09-10T07:58:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:33.632+0000 7ff24f000640 1 -- 172.21.15.2:0/2754387642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff2481079e0 con 0x7ff248107580 2024-09-10T07:58:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:34.320+0000 7ff246000640 1 -- 172.21.15.2:0/2754387642 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7ff23c03b860 con 0x7ff248107580 2024-09-10T07:58:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:adjusted tunables profile to default 2024-09-10T07:58:34.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:34.323+0000 7ff24f000640 1 -- 172.21.15.2:0/2754387642 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff2300390c0 msgr2=0x7ff23003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:34.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:34.324+0000 7ff24f000640 1 --2- 172.21.15.2:0/2754387642 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff2300390c0 0x7ff23003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff2340098c0 tx=0x7ff234006db0 comp rx=0 tx=0).stop 2024-09-10T07:58:34.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:34.324+0000 7ff24f000640 1 -- 172.21.15.2:0/2754387642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248107580 msgr2=0x7ff2481991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:34.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:34.324+0000 7ff24f000640 1 --2- 172.21.15.2:0/2754387642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248107580 0x7ff2481991f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009230 tx=0x7ff23c003d80 comp rx=0 tx=0).stop 2024-09-10T07:58:34.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:34.324+0000 7ff24f000640 1 -- 172.21.15.2:0/2754387642 shutdown_connections 2024-09-10T07:58:34.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:34.324+0000 7ff24f000640 1 --2- 172.21.15.2:0/2754387642 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff2300390c0 0x7ff23003b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:34.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:34.324+0000 7ff24f000640 1 --2- 172.21.15.2:0/2754387642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248107580 0x7ff2481991f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:34.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:34.324+0000 7ff24f000640 1 -- 172.21.15.2:0/2754387642 >> 172.21.15.2:0/2754387642 conn(0x7ff248078260 msgr2=0x7ff248075e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:34.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:34.324+0000 7ff24f000640 1 -- 172.21.15.2:0/2754387642 shutdown_connections 2024-09-10T07:58:34.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:34.324+0000 7ff24f000640 1 -- 172.21.15.2:0/2754387642 wait complete. 2024-09-10T07:58:34.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:34 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2754387642' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T07:58:34.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:34.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:34.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:34.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:34.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:58:34.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:34.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:58:35.063 INFO:tasks.cephadm:Adding mon.a on smithi002 2024-09-10T07:58:35.063 INFO:tasks.cephadm:Adding mon.b on smithi107 2024-09-10T07:58:35.063 INFO:tasks.cephadm:Adding mon.c on smithi155 2024-09-10T07:58:35.064 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi002:172.21.15.2=a;smithi107:172.21.15.107=b;smithi155:172.21.15.155=c' 2024-09-10T07:58:35.301 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:35.701 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.701+0000 7f346203f640 1 -- 172.21.15.155:0/3752297538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c106540 msgr2=0x7f345c106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:35.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.701+0000 7f346203f640 1 --2- 172.21.15.155:0/3752297538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c106540 0x7f345c106920 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f344c009810 tx=0x7f344c02a540 comp rx=0 tx=0).stop 2024-09-10T07:58:35.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.701+0000 7f346203f640 1 -- 172.21.15.155:0/3752297538 shutdown_connections 2024-09-10T07:58:35.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.701+0000 7f346203f640 1 --2- 172.21.15.155:0/3752297538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c106540 0x7f345c106920 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:35.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.701+0000 7f346203f640 1 -- 172.21.15.155:0/3752297538 >> 172.21.15.155:0/3752297538 conn(0x7f345c077310 msgr2=0x7f345c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:35.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.702+0000 7f346203f640 1 -- 172.21.15.155:0/3752297538 shutdown_connections 2024-09-10T07:58:35.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.702+0000 7f346203f640 1 -- 172.21.15.155:0/3752297538 wait complete. 2024-09-10T07:58:35.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.702+0000 7f346203f640 1 Processor -- start 2024-09-10T07:58:35.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.702+0000 7f346203f640 1 -- start start 2024-09-10T07:58:35.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.703+0000 7f346203f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c106540 0x7f345c196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:35.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.703+0000 7f346203f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f345c10ab70 con 0x7f345c106540 2024-09-10T07:58:35.703 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.703+0000 7f345b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c106540 0x7f345c196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:35.703 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.703+0000 7f345b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c106540 0x7f345c196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.155:60198/0 (socket says 172.21.15.155:60198) 2024-09-10T07:58:35.703 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.703+0000 7f345b7fe640 1 -- 172.21.15.155:0/1799677419 learned_addr learned my addr 172.21.15.155:0/1799677419 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-10T07:58:35.704 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.703+0000 7f345b7fe640 1 -- 172.21.15.155:0/1799677419 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c1965a0 con 0x7f345c106540 2024-09-10T07:58:35.704 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.704+0000 7f345b7fe640 1 --2- 172.21.15.155:0/1799677419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c106540 0x7f345c196060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f344c0045e0 tx=0x7f344c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:35.704 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.704+0000 7f34597fa640 1 -- 172.21.15.155:0/1799677419 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f344c030e70 con 0x7f345c106540 2024-09-10T07:58:35.704 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.704+0000 7f346203f640 1 -- 172.21.15.155:0/1799677419 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345c196830 con 0x7f345c106540 2024-09-10T07:58:35.704 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.704+0000 7f34597fa640 1 -- 172.21.15.155:0/1799677419 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344c0318e0 con 0x7f345c106540 2024-09-10T07:58:35.704 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.704+0000 7f34597fa640 1 -- 172.21.15.155:0/1799677419 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f344c0049c0 con 0x7f345c106540 2024-09-10T07:58:35.704 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.704+0000 7f346203f640 1 -- 172.21.15.155:0/1799677419 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345c196c90 con 0x7f345c106540 2024-09-10T07:58:35.705 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.705+0000 7f34597fa640 1 -- 172.21.15.155:0/1799677419 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f344c039040 con 0x7f345c106540 2024-09-10T07:58:35.705 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.705+0000 7f346203f640 1 -- 172.21.15.155:0/1799677419 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f345c101bb0 con 0x7f345c106540 2024-09-10T07:58:35.705 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.705+0000 7f34597fa640 1 --2- 172.21.15.155:0/1799677419 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3448039060 0x7f344803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:35.705 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.705+0000 7f34597fa640 1 -- 172.21.15.155:0/1799677419 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f344c06d360 con 0x7f345c106540 2024-09-10T07:58:35.705 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.705+0000 7f3453fff640 1 --2- 172.21.15.155:0/1799677419 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3448039060 0x7f344803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:35.705 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.706+0000 7f3453fff640 1 --2- 172.21.15.155:0/1799677419 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3448039060 0x7f344803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f34440098c0 tx=0x7f3444006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:35.709 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.710+0000 7f34597fa640 1 -- 172.21.15.155:0/1799677419 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f344c040030 con 0x7f345c106540 2024-09-10T07:58:35.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:35 smithi002 ceph-mon[30563]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:35.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:35 smithi002 ceph-mon[30563]: Updating smithi155:/etc/ceph/ceph.conf 2024-09-10T07:58:35.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:35 smithi002 ceph-mon[30563]: Updating smithi155:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:35.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:35 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2754387642' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T07:58:35.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:35 smithi002 ceph-mon[30563]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:58:35.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:35 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:35.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:35 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:35.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:35 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:35.894 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.895+0000 7f346203f640 1 -- 172.21.15.155:0/1799677419 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi107:172.21.15.107=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}) -- 0x7f345c0ff340 con 0x7f3448039060 2024-09-10T07:58:35.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.902+0000 7f34597fa640 1 -- 172.21.15.155:0/1799677419 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f345c0ff340 con 0x7f3448039060 2024-09-10T07:58:35.901 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mon update... 2024-09-10T07:58:35.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.904+0000 7f346203f640 1 -- 172.21.15.155:0/1799677419 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3448039060 msgr2=0x7f344803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:35.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.904+0000 7f346203f640 1 --2- 172.21.15.155:0/1799677419 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3448039060 0x7f344803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f34440098c0 tx=0x7f3444006db0 comp rx=0 tx=0).stop 2024-09-10T07:58:35.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.904+0000 7f346203f640 1 -- 172.21.15.155:0/1799677419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c106540 msgr2=0x7f345c196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:35.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.904+0000 7f346203f640 1 --2- 172.21.15.155:0/1799677419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c106540 0x7f345c196060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f344c0045e0 tx=0x7f344c004610 comp rx=0 tx=0).stop 2024-09-10T07:58:35.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.904+0000 7f346203f640 1 -- 172.21.15.155:0/1799677419 shutdown_connections 2024-09-10T07:58:35.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.904+0000 7f346203f640 1 --2- 172.21.15.155:0/1799677419 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3448039060 0x7f344803b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:35.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.904+0000 7f346203f640 1 --2- 172.21.15.155:0/1799677419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c106540 0x7f345c196060 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:35.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.904+0000 7f346203f640 1 -- 172.21.15.155:0/1799677419 >> 172.21.15.155:0/1799677419 conn(0x7f345c077310 msgr2=0x7f345c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:35.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.904+0000 7f346203f640 1 -- 172.21.15.155:0/1799677419 shutdown_connections 2024-09-10T07:58:35.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:35.904+0000 7f346203f640 1 -- 172.21.15.155:0/1799677419 wait complete. 2024-09-10T07:58:36.515 DEBUG:teuthology.orchestra.run.smithi107:mon.b> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T07:58:36.518 DEBUG:teuthology.orchestra.run.smithi155:mon.c> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T07:58:36.520 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T07:58:36.520 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T07:58:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:36 smithi002 ceph-mon[30563]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:58:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:36 smithi002 ceph-mon[30563]: Updating smithi155:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T07:58:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:36 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:36 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:58:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:36 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:36 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:58:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:36 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:36 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:58:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:36 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:37.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:37 smithi002 ceph-mon[30563]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:37.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:37 smithi002 ceph-mon[30563]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi107:172.21.15.107=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:58:37.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:37 smithi002 ceph-mon[30563]: Saving service mon spec with placement smithi002:172.21.15.2=a;smithi107:172.21.15.107=b;smithi155:172.21.15.155=c;count:3 2024-09-10T07:58:37.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:37 smithi002 ceph-mon[30563]: Deploying daemon mon.c on smithi155 2024-09-10T07:58:38.051 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:38.467 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.456+0000 7fe1699b3640 1 -- 172.21.15.155:0/757836829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe164107640 msgr2=0x7fe164107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:38.467 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.456+0000 7fe1699b3640 1 --2- 172.21.15.155:0/757836829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe164107640 0x7fe164107a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe154009880 tx=0x7fe15402a5d0 comp rx=0 tx=0).stop 2024-09-10T07:58:38.467 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.456+0000 7fe1699b3640 1 -- 172.21.15.155:0/757836829 shutdown_connections 2024-09-10T07:58:38.467 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.456+0000 7fe1699b3640 1 --2- 172.21.15.155:0/757836829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe164107640 0x7fe164107a20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:38.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.456+0000 7fe1699b3640 1 -- 172.21.15.155:0/757836829 >> 172.21.15.155:0/757836829 conn(0x7fe1640fd230 msgr2=0x7fe1640ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:38.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.456+0000 7fe1699b3640 1 -- 172.21.15.155:0/757836829 shutdown_connections 2024-09-10T07:58:38.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.456+0000 7fe1699b3640 1 -- 172.21.15.155:0/757836829 wait complete. 2024-09-10T07:58:38.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.457+0000 7fe1699b3640 1 Processor -- start 2024-09-10T07:58:38.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.457+0000 7fe1699b3640 1 -- start start 2024-09-10T07:58:38.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.457+0000 7fe1699b3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe164107640 0x7fe164197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:38.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.457+0000 7fe1699b3640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe16410bc70 con 0x7fe164107640 2024-09-10T07:58:38.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.457+0000 7fe162ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe164107640 0x7fe164197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:38.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.457+0000 7fe162ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe164107640 0x7fe164197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.155:60226/0 (socket says 172.21.15.155:60226) 2024-09-10T07:58:38.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.457+0000 7fe162ffd640 1 -- 172.21.15.155:0/3895911945 learned_addr learned my addr 172.21.15.155:0/3895911945 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-10T07:58:38.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.458+0000 7fe162ffd640 1 -- 172.21.15.155:0/3895911945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe164197650 con 0x7fe164107640 2024-09-10T07:58:38.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.458+0000 7fe162ffd640 1 --2- 172.21.15.155:0/3895911945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe164107640 0x7fe164197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe154009230 tx=0x7fe154003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:38.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.458+0000 7fe1689b1640 1 -- 172.21.15.155:0/3895911945 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe15403f070 con 0x7fe164107640 2024-09-10T07:58:38.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.458+0000 7fe1699b3640 1 -- 172.21.15.155:0/3895911945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1641978e0 con 0x7fe164107640 2024-09-10T07:58:38.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.458+0000 7fe1689b1640 1 -- 172.21.15.155:0/3895911945 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15403bc80 con 0x7fe164107640 2024-09-10T07:58:38.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.458+0000 7fe1699b3640 1 -- 172.21.15.155:0/3895911945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16419b9e0 con 0x7fe164107640 2024-09-10T07:58:38.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.458+0000 7fe1689b1640 1 -- 172.21.15.155:0/3895911945 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe154049060 con 0x7fe164107640 2024-09-10T07:58:38.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.459+0000 7fe1699b3640 1 -- 172.21.15.155:0/3895911945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe164102cb0 con 0x7fe164107640 2024-09-10T07:58:38.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.459+0000 7fe1689b1640 1 -- 172.21.15.155:0/3895911945 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fe154042070 con 0x7fe164107640 2024-09-10T07:58:38.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.460+0000 7fe1689b1640 1 --2- 172.21.15.155:0/3895911945 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe150039060 0x7fe15003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:38.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.460+0000 7fe1689b1640 1 -- 172.21.15.155:0/3895911945 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe15406cd80 con 0x7fe164107640 2024-09-10T07:58:38.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.460+0000 7fe15bfff640 1 --2- 172.21.15.155:0/3895911945 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe150039060 0x7fe15003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:38.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.460+0000 7fe15bfff640 1 --2- 172.21.15.155:0/3895911945 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe150039060 0x7fe15003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0098c0 tx=0x7fe14c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:38.469 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.465+0000 7fe1689b1640 1 -- 172.21.15.155:0/3895911945 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe154042360 con 0x7fe164107640 2024-09-10T07:58:38.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.699+0000 7fe1699b3640 1 -- 172.21.15.155:0/3895911945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe164198470 con 0x7fe164107640 2024-09-10T07:58:38.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.699+0000 7fe1689b1640 1 -- 172.21.15.155:0/3895911945 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7fe154031570 con 0x7fe164107640 2024-09-10T07:58:38.699 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:58:38.699 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","modified":"2024-09-10T07:57:06.640971Z","created":"2024-09-10T07:57:06.640971Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T07:58:38.699 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-09-10T07:58:38.701 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.702+0000 7fe1699b3640 1 -- 172.21.15.155:0/3895911945 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe150039060 msgr2=0x7fe15003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:38.701 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.702+0000 7fe1699b3640 1 --2- 172.21.15.155:0/3895911945 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe150039060 0x7fe15003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0098c0 tx=0x7fe14c006db0 comp rx=0 tx=0).stop 2024-09-10T07:58:38.701 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.702+0000 7fe1699b3640 1 -- 172.21.15.155:0/3895911945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe164107640 msgr2=0x7fe164197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:38.701 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.702+0000 7fe1699b3640 1 --2- 172.21.15.155:0/3895911945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe164107640 0x7fe164197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe154009230 tx=0x7fe154003d40 comp rx=0 tx=0).stop 2024-09-10T07:58:38.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.702+0000 7fe1699b3640 1 -- 172.21.15.155:0/3895911945 shutdown_connections 2024-09-10T07:58:38.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.702+0000 7fe1699b3640 1 --2- 172.21.15.155:0/3895911945 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe150039060 0x7fe15003b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:38.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.702+0000 7fe1699b3640 1 --2- 172.21.15.155:0/3895911945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe164107640 0x7fe164197110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:38.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.702+0000 7fe1699b3640 1 -- 172.21.15.155:0/3895911945 >> 172.21.15.155:0/3895911945 conn(0x7fe1640fd230 msgr2=0x7fe1640feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:38.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.702+0000 7fe1699b3640 1 -- 172.21.15.155:0/3895911945 shutdown_connections 2024-09-10T07:58:38.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:38.702+0000 7fe1699b3640 1 -- 172.21.15.155:0/3895911945 wait complete. 2024-09-10T07:58:39.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:39 smithi002 ceph-mon[30563]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:39.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:39 smithi002 ceph-mon[30563]: from='client.? 172.21.15.155:0/3895911945' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:58:40.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:40 smithi155 ceph-mon[35405]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T07:58:40.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:40 smithi155 bash[35386]: 39a0f8f0def57a27a48c921336a08062e5c47cbb611fb105886830eabd773dd3 2024-09-10T07:58:40.440 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T07:58:40.441 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T07:58:40.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:40 smithi155 systemd[1]: Started Ceph mon.c for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T07:58:40.687 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:58:45.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.517+0000 7ff633bdb640 1 -- 172.21.15.155:0/3341243270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff614003530 msgr2=0x7ff6140059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:45.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.517+0000 7ff633bdb640 1 --2- 172.21.15.155:0/3341243270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff614003530 0x7ff6140059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff62c1016f0 tx=0x7ff62402c490 comp rx=0 tx=0).stop 2024-09-10T07:58:45.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.517+0000 7ff633bdb640 1 -- 172.21.15.155:0/3341243270 shutdown_connections 2024-09-10T07:58:45.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.517+0000 7ff633bdb640 1 --2- 172.21.15.155:0/3341243270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff614003530 0x7ff6140059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:45.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.517+0000 7ff633bdb640 1 -- 172.21.15.155:0/3341243270 >> 172.21.15.155:0/3341243270 conn(0x7ff62c0fc290 msgr2=0x7ff62c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:45.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.517+0000 7ff633bdb640 1 -- 172.21.15.155:0/3341243270 shutdown_connections 2024-09-10T07:58:45.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.517+0000 7ff633bdb640 1 -- 172.21.15.155:0/3341243270 wait complete. 2024-09-10T07:58:45.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.518+0000 7ff633bdb640 1 Processor -- start 2024-09-10T07:58:45.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.518+0000 7ff633bdb640 1 -- start start 2024-09-10T07:58:45.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.518+0000 7ff633bdb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff614003530 0x7ff62c102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:45.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.518+0000 7ff633bdb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c102d80 0x7ff62c1031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:45.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.518+0000 7ff633bdb640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1086c0 con 0x7ff614003530 2024-09-10T07:58:45.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.518+0000 7ff633bdb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c108540 con 0x7ff62c102d80 2024-09-10T07:58:45.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.518+0000 7ff631176640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c102d80 0x7ff62c1031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:45.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.518+0000 7ff631176640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c102d80 0x7ff62c1031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:44246/0 (socket says 172.21.15.155:44246) 2024-09-10T07:58:45.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.518+0000 7ff631176640 1 -- 172.21.15.155:0/534031625 learned_addr learned my addr 172.21.15.155:0/534031625 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-10T07:58:45.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.518+0000 7ff631176640 1 -- 172.21.15.155:0/534031625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c102d80 msgr2=0x7ff62c1031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T07:58:45.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.518+0000 7ff631176640 1 -- 172.21.15.155:0/534031625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c102d80 msgr2=0x7ff62c1031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T07:58:45.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.518+0000 7ff631176640 1 --2- 172.21.15.155:0/534031625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c102d80 0x7ff62c1031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T07:58:45.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.519+0000 7ff631176640 1 --2- 172.21.15.155:0/534031625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c102d80 0x7ff62c1031e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:58:45.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.519+0000 7ff631977640 1 --2- 172.21.15.155:0/534031625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff614003530 0x7ff62c102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:45.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.519+0000 7ff631977640 1 -- 172.21.15.155:0/534031625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c102d80 msgr2=0x7ff62c1031e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:58:45.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.519+0000 7ff631977640 1 --2- 172.21.15.155:0/534031625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c102d80 0x7ff62c1031e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:45.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.519+0000 7ff631977640 1 -- 172.21.15.155:0/534031625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62c104730 con 0x7ff614003530 2024-09-10T07:58:45.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.519+0000 7ff631977640 1 --2- 172.21.15.155:0/534031625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff614003530 0x7ff62c102840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff624036040 tx=0x7ff624033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:45.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.520+0000 7ff62affd640 1 -- 172.21.15.155:0/534031625 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff624032920 con 0x7ff614003530 2024-09-10T07:58:45.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.520+0000 7ff62affd640 1 -- 172.21.15.155:0/534031625 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff624041070 con 0x7ff614003530 2024-09-10T07:58:45.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.520+0000 7ff633bdb640 1 -- 172.21.15.155:0/534031625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62c1a34b0 con 0x7ff614003530 2024-09-10T07:58:45.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.520+0000 7ff62affd640 1 -- 172.21.15.155:0/534031625 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff624004970 con 0x7ff614003530 2024-09-10T07:58:45.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.520+0000 7ff633bdb640 1 -- 172.21.15.155:0/534031625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff62c1a3a40 con 0x7ff614003530 2024-09-10T07:58:45.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.521+0000 7ff62affd640 1 -- 172.21.15.155:0/534031625 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7ff624004c20 con 0x7ff614003530 2024-09-10T07:58:45.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.521+0000 7ff62affd640 1 --2- 172.21.15.155:0/534031625 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff618038e50 0x7ff61803b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:45.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.521+0000 7ff62affd640 1 -- 172.21.15.155:0/534031625 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff62406e350 con 0x7ff614003530 2024-09-10T07:58:45.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.521+0000 7ff633bdb640 1 -- 172.21.15.155:0/534031625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff62c101cf0 con 0x7ff614003530 2024-09-10T07:58:45.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.521+0000 7ff631176640 1 --2- 172.21.15.155:0/534031625 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff618038e50 0x7ff61803b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:45.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.522+0000 7ff631176640 1 --2- 172.21.15.155:0/534031625 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff618038e50 0x7ff61803b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff620009400 tx=0x7ff620007040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:45.526 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.527+0000 7ff62affd640 1 -- 172.21.15.155:0/534031625 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff62403a550 con 0x7ff614003530 2024-09-10T07:58:45.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:58:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: mon.a calling monitor election 2024-09-10T07:58:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: mon.c calling monitor election 2024-09-10T07:58:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:45.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T07:58:45.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: monmap epoch 2 2024-09-10T07:58:45.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:45.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: last_changed 2024-09-10T07:58:40.297874+0000 2024-09-10T07:58:45.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: created 2024-09-10T07:57:06.640971+0000 2024-09-10T07:58:45.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: min_mon_release 19 (squid) 2024-09-10T07:58:45.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: election_strategy: 1 2024-09-10T07:58:45.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-10T07:58:45.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-09-10T07:58:45.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: fsmap 2024-09-10T07:58:45.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:58:45.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: mgrmap e13: x(active, since 41s) 2024-09-10T07:58:45.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: overall HEALTH_OK 2024-09-10T07:58:45.659 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:45.659 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:45.659 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:45.659 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:58:45.659 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:45.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.760+0000 7ff633bdb640 1 -- 172.21.15.155:0/534031625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff62c1a3cf0 con 0x7ff614003530 2024-09-10T07:58:45.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.761+0000 7ff62affd640 1 -- 172.21.15.155:0/534031625 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7ff62403a2f0 con 0x7ff614003530 2024-09-10T07:58:45.760 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:58:45.760 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":2,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","modified":"2024-09-10T07:58:40.297874Z","created":"2024-09-10T07:57:06.640971Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T07:58:45.761 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 2 2024-09-10T07:58:45.762 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.763+0000 7ff633bdb640 1 -- 172.21.15.155:0/534031625 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff618038e50 msgr2=0x7ff61803b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:45.762 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.763+0000 7ff633bdb640 1 --2- 172.21.15.155:0/534031625 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff618038e50 0x7ff61803b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff620009400 tx=0x7ff620007040 comp rx=0 tx=0).stop 2024-09-10T07:58:45.762 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.763+0000 7ff633bdb640 1 -- 172.21.15.155:0/534031625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff614003530 msgr2=0x7ff62c102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:45.762 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.763+0000 7ff633bdb640 1 --2- 172.21.15.155:0/534031625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff614003530 0x7ff62c102840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff624036040 tx=0x7ff624033680 comp rx=0 tx=0).stop 2024-09-10T07:58:45.763 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.763+0000 7ff633bdb640 1 -- 172.21.15.155:0/534031625 shutdown_connections 2024-09-10T07:58:45.763 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.763+0000 7ff633bdb640 1 --2- 172.21.15.155:0/534031625 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff618038e50 0x7ff61803b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:45.763 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.763+0000 7ff633bdb640 1 --2- 172.21.15.155:0/534031625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c102d80 0x7ff62c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:45.763 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.763+0000 7ff633bdb640 1 --2- 172.21.15.155:0/534031625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff614003530 0x7ff62c102840 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:45.763 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.763+0000 7ff633bdb640 1 -- 172.21.15.155:0/534031625 >> 172.21.15.155:0/534031625 conn(0x7ff62c0fc290 msgr2=0x7ff62c104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:45.763 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.763+0000 7ff633bdb640 1 -- 172.21.15.155:0/534031625 shutdown_connections 2024-09-10T07:58:45.763 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:45.763+0000 7ff633bdb640 1 -- 172.21.15.155:0/534031625 wait complete. 2024-09-10T07:58:45.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:45.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:58:45.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: mon.a calling monitor election 2024-09-10T07:58:45.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:45.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:45.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:45.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:45.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: mon.c calling monitor election 2024-09-10T07:58:45.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:45.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:45.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:45.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:45.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T07:58:45.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: monmap epoch 2 2024-09-10T07:58:45.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: last_changed 2024-09-10T07:58:40.297874+0000 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: created 2024-09-10T07:57:06.640971+0000 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: min_mon_release 19 (squid) 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: election_strategy: 1 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: fsmap 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: mgrmap e13: x(active, since 41s) 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: overall HEALTH_OK 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:45.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:58:45.773 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:46.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:46 smithi155 ceph-mon[35405]: Deploying daemon mon.b on smithi107 2024-09-10T07:58:46.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:46 smithi155 ceph-mon[35405]: from='client.? 172.21.15.155:0/534031625' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:58:46.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:46.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:46 smithi002 ceph-mon[30563]: Deploying daemon mon.b on smithi107 2024-09-10T07:58:46.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:46 smithi002 ceph-mon[30563]: from='client.? 172.21.15.155:0/534031625' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:58:46.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:47.296 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T07:58:47.296 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T07:58:47.541 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:58:47.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:47 smithi155 ceph-mon[35405]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:47.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:47 smithi002 ceph-mon[30563]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:47.672 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:47 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:47.297+0000 7f6388000640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T07:58:47.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.950+0000 7f4c8a699640 1 -- 172.21.15.155:0/3998506827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c84107660 msgr2=0x7f4c84107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:47.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.950+0000 7f4c8a699640 1 --2- 172.21.15.155:0/3998506827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c84107660 0x7f4c84107a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009880 tx=0x7f4c7402a5d0 comp rx=0 tx=0).stop 2024-09-10T07:58:47.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.950+0000 7f4c8a699640 1 -- 172.21.15.155:0/3998506827 shutdown_connections 2024-09-10T07:58:47.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.950+0000 7f4c8a699640 1 --2- 172.21.15.155:0/3998506827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c84107660 0x7f4c84107a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:47.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.950+0000 7f4c8a699640 1 -- 172.21.15.155:0/3998506827 >> 172.21.15.155:0/3998506827 conn(0x7f4c840fd320 msgr2=0x7f4c840ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:47.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.950+0000 7f4c8a699640 1 -- 172.21.15.155:0/3998506827 shutdown_connections 2024-09-10T07:58:47.954 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.950+0000 7f4c8a699640 1 -- 172.21.15.155:0/3998506827 wait complete. 2024-09-10T07:58:47.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.951+0000 7f4c8a699640 1 Processor -- start 2024-09-10T07:58:47.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.951+0000 7f4c8a699640 1 -- start start 2024-09-10T07:58:47.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.951+0000 7f4c8a699640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c84107660 0x7f4c841973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:47.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.951+0000 7f4c8a699640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c84197900 0x7f4c8419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:47.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.951+0000 7f4c8a699640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8410bd20 con 0x7f4c84197900 2024-09-10T07:58:47.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.951+0000 7f4c8a699640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8410bba0 con 0x7f4c84107660 2024-09-10T07:58:47.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.951+0000 7f4c83fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c84107660 0x7f4c841973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:47.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.951+0000 7f4c83fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c84107660 0x7f4c841973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:44260/0 (socket says 172.21.15.155:44260) 2024-09-10T07:58:47.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.951+0000 7f4c83fff640 1 -- 172.21.15.155:0/2774449442 learned_addr learned my addr 172.21.15.155:0/2774449442 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-10T07:58:47.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.952+0000 7f4c83fff640 1 -- 172.21.15.155:0/2774449442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c84107660 msgr2=0x7f4c841973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T07:58:47.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.952+0000 7f4c83fff640 1 -- 172.21.15.155:0/2774449442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c84107660 msgr2=0x7f4c841973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T07:58:47.955 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.952+0000 7f4c83fff640 1 --2- 172.21.15.155:0/2774449442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c84107660 0x7f4c841973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T07:58:47.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.952+0000 7f4c83fff640 1 --2- 172.21.15.155:0/2774449442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c84107660 0x7f4c841973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:58:47.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.952+0000 7f4c837fe640 1 --2- 172.21.15.155:0/2774449442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c84197900 0x7f4c8419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:47.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.952+0000 7f4c837fe640 1 -- 172.21.15.155:0/2774449442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c84107660 msgr2=0x7f4c841973c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:58:47.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.952+0000 7f4c837fe640 1 --2- 172.21.15.155:0/2774449442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c84107660 0x7f4c841973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:47.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.952+0000 7f4c837fe640 1 -- 172.21.15.155:0/2774449442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c84197f20 con 0x7f4c84197900 2024-09-10T07:58:47.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.952+0000 7f4c837fe640 1 --2- 172.21.15.155:0/2774449442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c84197900 0x7f4c8419bc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800b3d0 tx=0x7f4c7800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:47.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.952+0000 7f4c817fa640 1 -- 172.21.15.155:0/2774449442 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4c78011630 con 0x7f4c84197900 2024-09-10T07:58:47.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.952+0000 7f4c8a699640 1 -- 172.21.15.155:0/2774449442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8419c1d0 con 0x7f4c84197900 2024-09-10T07:58:47.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.952+0000 7f4c817fa640 1 -- 172.21.15.155:0/2774449442 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c78011cb0 con 0x7f4c84197900 2024-09-10T07:58:47.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.952+0000 7f4c817fa640 1 -- 172.21.15.155:0/2774449442 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4c78010df0 con 0x7f4c84197900 2024-09-10T07:58:47.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.953+0000 7f4c8a699640 1 -- 172.21.15.155:0/2774449442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8419c710 con 0x7f4c84197900 2024-09-10T07:58:47.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.953+0000 7f4c817fa640 1 -- 172.21.15.155:0/2774449442 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f4c78018020 con 0x7f4c84197900 2024-09-10T07:58:47.957 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.953+0000 7f4c8a699640 1 -- 172.21.15.155:0/2774449442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c84102da0 con 0x7f4c84197900 2024-09-10T07:58:47.957 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.954+0000 7f4c817fa640 1 --2- 172.21.15.155:0/2774449442 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4c6c038e00 0x7f4c6c03b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:47.957 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.954+0000 7f4c817fa640 1 -- 172.21.15.155:0/2774449442 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4c7804bc30 con 0x7f4c84197900 2024-09-10T07:58:47.957 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.954+0000 7f4c83fff640 1 --2- 172.21.15.155:0/2774449442 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4c6c038e00 0x7f4c6c03b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:47.957 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.955+0000 7f4c83fff640 1 --2- 172.21.15.155:0/2774449442 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4c6c038e00 0x7f4c6c03b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4c740096c0 tx=0x7f4c74005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:47.958 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:47.959+0000 7f4c817fa640 1 -- 172.21.15.155:0/2774449442 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c7801f020 con 0x7f4c84197900 2024-09-10T07:58:48.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:48.191+0000 7f4c8a699640 1 -- 172.21.15.155:0/2774449442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4c8419ca20 con 0x7f4c84197900 2024-09-10T07:58:48.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:48.192+0000 7f4c817fa640 1 -- 172.21.15.155:0/2774449442 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f4c780172c0 con 0x7f4c84197900 2024-09-10T07:58:48.192 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:58:48.192 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":2,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","modified":"2024-09-10T07:58:40.297874Z","created":"2024-09-10T07:57:06.640971Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T07:58:48.193 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 2 2024-09-10T07:58:48.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:48.194+0000 7f4c8a699640 1 -- 172.21.15.155:0/2774449442 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4c6c038e00 msgr2=0x7f4c6c03b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:48.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:48.194+0000 7f4c8a699640 1 --2- 172.21.15.155:0/2774449442 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4c6c038e00 0x7f4c6c03b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4c740096c0 tx=0x7f4c74005a40 comp rx=0 tx=0).stop 2024-09-10T07:58:48.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:48.194+0000 7f4c8a699640 1 -- 172.21.15.155:0/2774449442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c84197900 msgr2=0x7f4c8419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:48.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:48.194+0000 7f4c8a699640 1 --2- 172.21.15.155:0/2774449442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c84197900 0x7f4c8419bc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800b3d0 tx=0x7f4c7800b890 comp rx=0 tx=0).stop 2024-09-10T07:58:48.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:48.194+0000 7f4c8a699640 1 -- 172.21.15.155:0/2774449442 shutdown_connections 2024-09-10T07:58:48.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:48.194+0000 7f4c8a699640 1 --2- 172.21.15.155:0/2774449442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c84197900 0x7f4c8419bc90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:48.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:48.194+0000 7f4c8a699640 1 --2- 172.21.15.155:0/2774449442 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4c6c038e00 0x7f4c6c03b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:48.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:48.194+0000 7f4c8a699640 1 --2- 172.21.15.155:0/2774449442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c84107660 0x7f4c841973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:48.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:48.194+0000 7f4c8a699640 1 -- 172.21.15.155:0/2774449442 >> 172.21.15.155:0/2774449442 conn(0x7f4c840fd320 msgr2=0x7f4c840fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:48.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:48.194+0000 7f4c8a699640 1 -- 172.21.15.155:0/2774449442 shutdown_connections 2024-09-10T07:58:48.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:48.194+0000 7f4c8a699640 1 -- 172.21.15.155:0/2774449442 wait complete. 2024-09-10T07:58:48.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:48 smithi155 ceph-mon[35405]: from='client.? 172.21.15.155:0/2774449442' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:58:48.693 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:48 smithi002 ceph-mon[30563]: from='client.? 172.21.15.155:0/2774449442' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:58:49.032 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(???) e0 preinit fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:49.032 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T07:58:49.032 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T07:58:49.032 INFO:journalctl@ceph.mon.b.smithi107.stdout: e1 2024-09-10T07:58:49.032 INFO:journalctl@ceph.mon.b.smithi107.stdout: btime 2024-09-10T07:57:11:261417+0000 2024-09-10T07:58:49.032 INFO:journalctl@ceph.mon.b.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T07:58:49.032 INFO:journalctl@ceph.mon.b.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T07:58:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout: legacy client fscid: -1 2024-09-10T07:58:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout: 2024-09-10T07:58:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout: No filesystems configured 2024-09-10T07:58:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T07:58:49.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T07:58:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T07:58:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T07:58:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T07:58:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T07:58:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T07:58:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T07:58:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T07:58:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T07:58:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T07:58:49.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2754387642' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T07:58:49.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:49.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:49.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:49.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:49.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:58:49.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:49.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:58:49.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:49.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: Updating smithi155:/etc/ceph/ceph.conf 2024-09-10T07:58:49.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: Updating smithi155:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:49.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2754387642' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T07:58:49.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:58:49.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:49.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:49.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:49.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:58:49.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: Updating smithi155:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T07:58:49.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:49.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:58:49.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:49.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:58:49.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:49.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:58:49.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:49.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:49.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi107:172.21.15.107=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:58:49.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: Saving service mon spec with placement smithi002:172.21.15.2=a;smithi107:172.21.15.107=b;smithi155:172.21.15.155=c;count:3 2024-09-10T07:58:49.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: Deploying daemon mon.c on smithi155 2024-09-10T07:58:49.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:49.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='client.? 172.21.15.155:0/3895911945' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:58:49.421 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:49.421 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:58:49.421 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.a calling monitor election 2024-09-10T07:58:49.421 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:49.421 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:49.421 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:49.421 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:49.421 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.c calling monitor election 2024-09-10T07:58:49.421 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:49.421 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:49.421 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:49.421 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:49.421 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: monmap epoch 2 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: last_changed 2024-09-10T07:58:40.297874+0000 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: created 2024-09-10T07:57:06.640971+0000 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: min_mon_release 19 (squid) 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: election_strategy: 1 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: fsmap 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mgrmap e13: x(active, since 41s) 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: overall HEALTH_OK 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:49.422 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:49.423 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:58:49.423 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:49.423 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: Deploying daemon mon.b on smithi107 2024-09-10T07:58:49.423 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='client.? 172.21.15.155:0/534031625' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:58:49.423 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:49.423 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:49.423 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: from='client.? 172.21.15.155:0/2774449442' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:58:49.423 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 ceph-mon[35048]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T07:58:49.423 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 bash[35029]: e8e08e82260ca00f8f86eb37a0c222fef9af7c61472ee21a7c4019d05909f44c 2024-09-10T07:58:49.423 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:49 smithi107 systemd[1]: Started Ceph mon.b for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T07:58:49.780 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T07:58:49.781 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T07:58:50.030 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:58:54.068 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.068+0000 7f674f6fa640 1 -- 172.21.15.155:0/2140008836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6748106630 msgr2=0x7f6734005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:54.068 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.068+0000 7f674f6fa640 1 --2- 172.21.15.155:0/2140008836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6748106630 0x7f6734005580 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6744014d10 tx=0x7f674402c570 comp rx=0 tx=0).stop 2024-09-10T07:58:54.068 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.068+0000 7f674f6fa640 1 -- 172.21.15.155:0/2140008836 shutdown_connections 2024-09-10T07:58:54.068 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.068+0000 7f674f6fa640 1 --2- 172.21.15.155:0/2140008836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6748106630 0x7f6734005580 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:54.068 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.068+0000 7f674f6fa640 1 -- 172.21.15.155:0/2140008836 >> 172.21.15.155:0/2140008836 conn(0x7f67480fc220 msgr2=0x7f67480fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:54.069 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.068+0000 7f674f6fa640 1 -- 172.21.15.155:0/2140008836 shutdown_connections 2024-09-10T07:58:54.069 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.068+0000 7f674f6fa640 1 -- 172.21.15.155:0/2140008836 wait complete. 2024-09-10T07:58:54.069 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.069+0000 7f674f6fa640 1 Processor -- start 2024-09-10T07:58:54.069 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.069+0000 7f674f6fa640 1 -- start start 2024-09-10T07:58:54.071 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.069+0000 7f674f6fa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748106630 0x7f67481972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:54.071 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.069+0000 7f674f6fa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6748197810 0x7f674819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:54.071 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.069+0000 7f674f6fa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f674819c0e0 0x7f674819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:54.071 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.069+0000 7f674f6fa640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6748108670 con 0x7f6748197810 2024-09-10T07:58:54.071 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.069+0000 7f674f6fa640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f67481084f0 con 0x7f6748106630 2024-09-10T07:58:54.071 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.069+0000 7f674f6fa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f67481087f0 con 0x7f674819c0e0 2024-09-10T07:58:54.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.069+0000 7f674dc97640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f674819c0e0 0x7f674819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:54.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674d496640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748106630 0x7f67481972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:54.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674dc97640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f674819c0e0 0x7f674819c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:53994/0 (socket says 172.21.15.155:53994) 2024-09-10T07:58:54.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674dc97640 1 -- 172.21.15.155:0/1534032351 learned_addr learned my addr 172.21.15.155:0/1534032351 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-10T07:58:54.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674d496640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748106630 0x7f67481972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.155:42004/0 (socket says 172.21.15.155:42004) 2024-09-10T07:58:54.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674cc95640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6748197810 0x7f674819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:54.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674dc97640 1 -- 172.21.15.155:0/1534032351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f674819c0e0 msgr2=0x7f674819c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-10T07:58:54.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674dc97640 1 -- 172.21.15.155:0/1534032351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f674819c0e0 msgr2=0x7f674819c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T07:58:54.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674dc97640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f674819c0e0 0x7f674819c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T07:58:54.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674d496640 1 -- 172.21.15.155:0/1534032351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748106630 msgr2=0x7f67481972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T07:58:54.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674d496640 1 -- 172.21.15.155:0/1534032351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748106630 msgr2=0x7f67481972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T07:58:54.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674d496640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748106630 0x7f67481972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T07:58:54.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674d496640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748106630 0x7f67481972d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:58:54.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674dc97640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f674819c0e0 0x7f674819c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:58:54.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674cc95640 1 -- 172.21.15.155:0/1534032351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f674819c0e0 msgr2=0x7f674819c590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:58:54.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674cc95640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f674819c0e0 0x7f674819c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:54.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674cc95640 1 -- 172.21.15.155:0/1534032351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748106630 msgr2=0x7f67481972d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:58:54.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674cc95640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748106630 0x7f67481972d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:54.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674cc95640 1 -- 172.21.15.155:0/1534032351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67481a3e50 con 0x7f6748197810 2024-09-10T07:58:54.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f674cc95640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6748197810 0x7f674819bba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f673800b450 tx=0x7f673800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:54.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.070+0000 7f673e7fc640 1 -- 172.21.15.155:0/1534032351 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6738004230 con 0x7f6748197810 2024-09-10T07:58:54.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.071+0000 7f673e7fc640 1 -- 172.21.15.155:0/1534032351 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67380043d0 con 0x7f6748197810 2024-09-10T07:58:54.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.071+0000 7f674f6fa640 1 -- 172.21.15.155:0/1534032351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67481a4080 con 0x7f6748197810 2024-09-10T07:58:54.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.071+0000 7f673e7fc640 1 -- 172.21.15.155:0/1534032351 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6738003cb0 con 0x7f6748197810 2024-09-10T07:58:54.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.071+0000 7f674f6fa640 1 -- 172.21.15.155:0/1534032351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67481a4660 con 0x7f6748197810 2024-09-10T07:58:54.074 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.072+0000 7f673e7fc640 1 -- 172.21.15.155:0/1534032351 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f673800f050 con 0x7f6748197810 2024-09-10T07:58:54.074 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.072+0000 7f673e7fc640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f672c0394f0 0x7f672c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:54.074 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.072+0000 7f673e7fc640 1 -- 172.21.15.155:0/1534032351 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f673804be30 con 0x7f6748197810 2024-09-10T07:58:54.074 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.072+0000 7f674f6fa640 1 -- 172.21.15.155:0/1534032351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67481097a0 con 0x7f6748197810 2024-09-10T07:58:54.074 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.072+0000 7f674d496640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f672c0394f0 0x7f672c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:54.074 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.073+0000 7f674d496640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f672c0394f0 0x7f672c03b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6744005e40 tx=0x7f67440141f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:54.076 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.077+0000 7f673e7fc640 1 -- 172.21.15.155:0/1534032351 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f673800d040 con 0x7f6748197810 2024-09-10T07:58:54.317 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.318+0000 7f674f6fa640 1 -- 172.21.15.155:0/1534032351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f67481a48f0 con 0x7f6748197810 2024-09-10T07:58:54.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.319+0000 7f673e7fc640 1 -- 172.21.15.155:0/1534032351 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f6738011d00 con 0x7f6748197810 2024-09-10T07:58:54.318 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:58:54.318 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":3,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","modified":"2024-09-10T07:58:49.038672Z","created":"2024-09-10T07:57:06.640971Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T07:58:54.319 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 3 2024-09-10T07:58:54.319 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:58:54.319 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: mon.c calling monitor election 2024-09-10T07:58:54.319 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: mon.a calling monitor election 2024-09-10T07:58:54.319 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:54.319 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:54.319 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:54.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:54.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:54.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:54.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:54.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:54.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:54.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T07:58:54.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: monmap epoch 3 2024-09-10T07:58:54.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:54.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: last_changed 2024-09-10T07:58:49.038672+0000 2024-09-10T07:58:54.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: created 2024-09-10T07:57:06.640971+0000 2024-09-10T07:58:54.320 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: min_mon_release 19 (squid) 2024-09-10T07:58:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: election_strategy: 1 2024-09-10T07:58:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-10T07:58:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-09-10T07:58:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-09-10T07:58:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: fsmap 2024-09-10T07:58:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:58:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: mgrmap e13: x(active, since 50s) 2024-09-10T07:58:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: overall HEALTH_OK 2024-09-10T07:58:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:54.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:54.322 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.321+0000 7f674f6fa640 1 -- 172.21.15.155:0/1534032351 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f672c0394f0 msgr2=0x7f672c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:54.322 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.321+0000 7f674f6fa640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f672c0394f0 0x7f672c03b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6744005e40 tx=0x7f67440141f0 comp rx=0 tx=0).stop 2024-09-10T07:58:54.322 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.321+0000 7f674f6fa640 1 -- 172.21.15.155:0/1534032351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6748197810 msgr2=0x7f674819bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:54.322 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.321+0000 7f674f6fa640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6748197810 0x7f674819bba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f673800b450 tx=0x7f673800b910 comp rx=0 tx=0).stop 2024-09-10T07:58:54.322 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.321+0000 7f674f6fa640 1 -- 172.21.15.155:0/1534032351 shutdown_connections 2024-09-10T07:58:54.323 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.321+0000 7f674f6fa640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f672c0394f0 0x7f672c03b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:54.323 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.321+0000 7f674f6fa640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f674819c0e0 0x7f674819c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:54.323 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.321+0000 7f674f6fa640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6748197810 0x7f674819bba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:54.323 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.321+0000 7f674f6fa640 1 --2- 172.21.15.155:0/1534032351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748106630 0x7f67481972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:54.323 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.321+0000 7f674f6fa640 1 -- 172.21.15.155:0/1534032351 >> 172.21.15.155:0/1534032351 conn(0x7f67480fc220 msgr2=0x7f674806a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:54.323 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.321+0000 7f674f6fa640 1 -- 172.21.15.155:0/1534032351 shutdown_connections 2024-09-10T07:58:54.323 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:54.321+0000 7f674f6fa640 1 -- 172.21.15.155:0/1534032351 wait complete. 2024-09-10T07:58:54.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:58:54.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: mon.c calling monitor election 2024-09-10T07:58:54.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: mon.a calling monitor election 2024-09-10T07:58:54.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:54.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:54.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:54.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:54.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:54.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:54.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:54.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:54.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:54.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T07:58:54.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: monmap epoch 3 2024-09-10T07:58:54.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:54.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: last_changed 2024-09-10T07:58:49.038672+0000 2024-09-10T07:58:54.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: created 2024-09-10T07:57:06.640971+0000 2024-09-10T07:58:54.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: min_mon_release 19 (squid) 2024-09-10T07:58:54.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: election_strategy: 1 2024-09-10T07:58:54.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-10T07:58:54.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-09-10T07:58:54.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-09-10T07:58:54.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: fsmap 2024-09-10T07:58:54.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:58:54.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: mgrmap e13: x(active, since 50s) 2024-09-10T07:58:54.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: overall HEALTH_OK 2024-09-10T07:58:54.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:54.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:54.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:54.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:54.881 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T07:58:54.881 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T07:58:55.261 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:58:55.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:55 smithi002 ceph-mon[30563]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:55.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:58:55.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:55 smithi002 ceph-mon[30563]: from='client.? 172.21.15.155:0/1534032351' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:58:55.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:55.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:55 smithi155 ceph-mon[35405]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:55.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:58:55.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:55 smithi155 ceph-mon[35405]: from='client.? 172.21.15.155:0/1534032351' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:58:55.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:55.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.954+0000 7f4741e00640 1 -- 172.21.15.2:0/751050576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f473c109770 msgr2=0x7f473c109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:55.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.954+0000 7f4741e00640 1 --2- 172.21.15.2:0/751050576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f473c109770 0x7f473c109b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f472c009880 tx=0x7f472c02a5d0 comp rx=0 tx=0).stop 2024-09-10T07:58:55.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.954+0000 7f4741e00640 1 -- 172.21.15.2:0/751050576 shutdown_connections 2024-09-10T07:58:55.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.954+0000 7f4741e00640 1 --2- 172.21.15.2:0/751050576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f473c109770 0x7f473c109b50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:55.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.954+0000 7f4741e00640 1 -- 172.21.15.2:0/751050576 >> 172.21.15.2:0/751050576 conn(0x7f473c0ff450 msgr2=0x7f473c101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:55.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.954+0000 7f4741e00640 1 -- 172.21.15.2:0/751050576 shutdown_connections 2024-09-10T07:58:55.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.954+0000 7f4741e00640 1 -- 172.21.15.2:0/751050576 wait complete. 2024-09-10T07:58:55.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.955+0000 7f4741e00640 1 Processor -- start 2024-09-10T07:58:55.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.956+0000 7f4741e00640 1 -- start start 2024-09-10T07:58:55.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.956+0000 7f4741e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f473c109770 0x7f473c199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:55.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.956+0000 7f4741e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f473c199ad0 0x7f473c19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:55.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.956+0000 7f4741e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f473c19e3a0 0x7f473c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:55.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.956+0000 7f4741e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f473c10de30 con 0x7f473c19e3a0 2024-09-10T07:58:55.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.956+0000 7f4741e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f473c10dcb0 con 0x7f473c109770 2024-09-10T07:58:55.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.956+0000 7f4741e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f473c10dfb0 con 0x7f473c199ad0 2024-09-10T07:58:55.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f473c109770 0x7f473c199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:55.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f473c19e3a0 0x7f473c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:55.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f473c109770 0x7f473c199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:33938/0 (socket says 172.21.15.2:33938) 2024-09-10T07:58:55.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473aa00640 1 -- 172.21.15.2:0/2587000685 learned_addr learned my addr 172.21.15.2:0/2587000685 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:58:55.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473a000640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f473c199ad0 0x7f473c19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:55.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473aa00640 1 -- 172.21.15.2:0/2587000685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f473c109770 msgr2=0x7f473c199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T07:58:55.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473aa00640 1 -- 172.21.15.2:0/2587000685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f473c109770 msgr2=0x7f473c199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T07:58:55.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473aa00640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f473c109770 0x7f473c199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T07:58:55.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473aa00640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f473c109770 0x7f473c199590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T07:58:55.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473a000640 1 -- 172.21.15.2:0/2587000685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f473c109770 msgr2=0x7f473c199590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T07:58:55.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473a000640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f473c109770 0x7f473c199590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:55.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473a000640 1 -- 172.21.15.2:0/2587000685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f473c19e3a0 msgr2=0x7f473c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:55.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473a000640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f473c19e3a0 0x7f473c19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:55.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473a000640 1 -- 172.21.15.2:0/2587000685 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c1a6110 con 0x7f473c199ad0 2024-09-10T07:58:55.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.957+0000 7f473b400640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f473c19e3a0 0x7f473c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T07:58:55.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.958+0000 7f473a000640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f473c199ad0 0x7f473c19de60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f472800b450 tx=0x7f472800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:55.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.958+0000 7f4740a00640 1 -- 172.21.15.2:0/2587000685 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728004230 con 0x7f473c199ad0 2024-09-10T07:58:55.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.958+0000 7f4740a00640 1 -- 172.21.15.2:0/2587000685 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47280043d0 con 0x7f473c199ad0 2024-09-10T07:58:55.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.958+0000 7f4740a00640 1 -- 172.21.15.2:0/2587000685 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728003cb0 con 0x7f473c199ad0 2024-09-10T07:58:55.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.958+0000 7f4741e00640 1 -- 172.21.15.2:0/2587000685 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c1a6340 con 0x7f473c199ad0 2024-09-10T07:58:55.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.958+0000 7f4741e00640 1 -- 172.21.15.2:0/2587000685 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c1a68a0 con 0x7f473c199ad0 2024-09-10T07:58:55.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.959+0000 7f4741e00640 1 -- 172.21.15.2:0/2587000685 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473c109bd0 con 0x7f473c199ad0 2024-09-10T07:58:55.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.959+0000 7f4740a00640 1 -- 172.21.15.2:0/2587000685 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f472800f050 con 0x7f473c199ad0 2024-09-10T07:58:55.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.960+0000 7f4740a00640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47200394f0 0x7f472003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:55.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.960+0000 7f4740a00640 1 -- 172.21.15.2:0/2587000685 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f472804bd60 con 0x7f473c199ad0 2024-09-10T07:58:55.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.960+0000 7f473aa00640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47200394f0 0x7f472003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:55.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.961+0000 7f473aa00640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47200394f0 0x7f472003b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f472c005ab0 tx=0x7f472c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:55.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:55.966+0000 7f4740a00640 1 -- 172.21.15.2:0/2587000685 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f472800d040 con 0x7f473c199ad0 2024-09-10T07:58:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.157+0000 7f4741e00640 1 -- 172.21.15.2:0/2587000685 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f473c1a6c90 con 0x7f473c199ad0 2024-09-10T07:58:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.158+0000 7f4740a00640 1 -- 172.21.15.2:0/2587000685 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f4728011d00 con 0x7f473c199ad0 2024-09-10T07:58:56.158 INFO:teuthology.orchestra.run.smithi002.stdout:# minimal ceph.conf for 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:56.158 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2024-09-10T07:58:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: mon_host = [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] 2024-09-10T07:58:56.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.160+0000 7f4741e00640 1 -- 172.21.15.2:0/2587000685 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47200394f0 msgr2=0x7f472003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:56.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.160+0000 7f4741e00640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47200394f0 0x7f472003b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f472c005ab0 tx=0x7f472c005a20 comp rx=0 tx=0).stop 2024-09-10T07:58:56.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.160+0000 7f4741e00640 1 -- 172.21.15.2:0/2587000685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f473c199ad0 msgr2=0x7f473c19de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:56.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.160+0000 7f4741e00640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f473c199ad0 0x7f473c19de60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f472800b450 tx=0x7f472800b910 comp rx=0 tx=0).stop 2024-09-10T07:58:56.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.160+0000 7f4741e00640 1 -- 172.21.15.2:0/2587000685 shutdown_connections 2024-09-10T07:58:56.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.160+0000 7f4741e00640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f473c19e3a0 0x7f473c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:56.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.160+0000 7f4741e00640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47200394f0 0x7f472003b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:56.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.160+0000 7f4741e00640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f473c199ad0 0x7f473c19de60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:56.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.160+0000 7f4741e00640 1 --2- 172.21.15.2:0/2587000685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f473c109770 0x7f473c199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:56.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.160+0000 7f4741e00640 1 -- 172.21.15.2:0/2587000685 >> 172.21.15.2:0/2587000685 conn(0x7f473c0ff450 msgr2=0x7f473c107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:56.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.160+0000 7f4741e00640 1 -- 172.21.15.2:0/2587000685 shutdown_connections 2024-09-10T07:58:56.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:58:56.160+0000 7f4741e00640 1 -- 172.21.15.2:0/2587000685 wait complete. 2024-09-10T07:58:56.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T07:58:56.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: mon.c calling monitor election 2024-09-10T07:58:56.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: mon.a calling monitor election 2024-09-10T07:58:56.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:56.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T07:58:56.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:56.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:56.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:56.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:56.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:56.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:56.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:56.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: monmap epoch 3 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: last_changed 2024-09-10T07:58:49.038672+0000 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: created 2024-09-10T07:57:06.640971+0000 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: min_mon_release 19 (squid) 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: election_strategy: 1 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: fsmap 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: mgrmap e13: x(active, since 50s) 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: overall HEALTH_OK 2024-09-10T07:58:56.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:56.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:58:56.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='client.? 172.21.15.155:0/1534032351' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T07:58:56.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:56.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: mon.b calling monitor election 2024-09-10T07:58:56.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:56.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: Updating smithi002:/etc/ceph/ceph.conf 2024-09-10T07:58:56.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-10T07:58:56.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: Updating smithi155:/etc/ceph/ceph.conf 2024-09-10T07:58:56.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: Updating smithi155:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:56.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: Updating smithi107:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:56.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: mon.b calling monitor election 2024-09-10T07:58:56.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: mon.c calling monitor election 2024-09-10T07:58:56.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: mon.a calling monitor election 2024-09-10T07:58:56.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T07:58:56.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: monmap epoch 3 2024-09-10T07:58:56.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:56.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: last_changed 2024-09-10T07:58:49.038672+0000 2024-09-10T07:58:56.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: created 2024-09-10T07:57:06.640971+0000 2024-09-10T07:58:56.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: min_mon_release 19 (squid) 2024-09-10T07:58:56.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: election_strategy: 1 2024-09-10T07:58:56.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-10T07:58:56.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-09-10T07:58:56.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-09-10T07:58:56.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: fsmap 2024-09-10T07:58:56.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:58:56.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: mgrmap e13: x(active, since 52s) 2024-09-10T07:58:56.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: overall HEALTH_OK 2024-09-10T07:58:56.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2587000685' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:56.909 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:56 smithi155 ceph-mon[35405]: Updating smithi002:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:56.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: mon.b calling monitor election 2024-09-10T07:58:56.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:56.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: Updating smithi002:/etc/ceph/ceph.conf 2024-09-10T07:58:56.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: Updating smithi155:/etc/ceph/ceph.conf 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: Updating smithi155:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: Updating smithi107:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: mon.b calling monitor election 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: mon.c calling monitor election 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: mon.a calling monitor election 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: monmap epoch 3 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: last_changed 2024-09-10T07:58:49.038672+0000 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: created 2024-09-10T07:57:06.640971+0000 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: min_mon_release 19 (squid) 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: election_strategy: 1 2024-09-10T07:58:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-10T07:58:56.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-09-10T07:58:56.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-09-10T07:58:56.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: fsmap 2024-09-10T07:58:56.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:58:56.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: mgrmap e13: x(active, since 52s) 2024-09-10T07:58:56.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: overall HEALTH_OK 2024-09-10T07:58:56.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2587000685' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:56.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:56 smithi107 ceph-mon[35048]: Updating smithi002:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:56.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: mon.b calling monitor election 2024-09-10T07:58:56.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:56.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: Updating smithi002:/etc/ceph/ceph.conf 2024-09-10T07:58:56.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-10T07:58:56.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: Updating smithi155:/etc/ceph/ceph.conf 2024-09-10T07:58:56.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: Updating smithi155:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:56.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: Updating smithi107:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:56.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: mon.b calling monitor election 2024-09-10T07:58:56.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: mon.c calling monitor election 2024-09-10T07:58:56.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: mon.a calling monitor election 2024-09-10T07:58:56.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T07:58:56.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: monmap epoch 3 2024-09-10T07:58:56.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T07:58:56.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: last_changed 2024-09-10T07:58:49.038672+0000 2024-09-10T07:58:56.926 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: created 2024-09-10T07:57:06.640971+0000 2024-09-10T07:58:56.926 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: min_mon_release 19 (squid) 2024-09-10T07:58:56.926 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: election_strategy: 1 2024-09-10T07:58:56.926 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-10T07:58:56.926 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: 1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-09-10T07:58:56.926 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-09-10T07:58:56.926 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: fsmap 2024-09-10T07:58:56.926 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T07:58:56.926 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: mgrmap e13: x(active, since 52s) 2024-09-10T07:58:56.926 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: overall HEALTH_OK 2024-09-10T07:58:56.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:56.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2587000685' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:56.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:56 smithi002 ceph-mon[30563]: Updating smithi002:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T07:58:56.965 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T07:58:56.965 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:58:56.965 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T07:58:57.023 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:58:57.023 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:58:57.119 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T07:58:57.120 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T07:58:57.149 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T07:58:57.149 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:58:57.214 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-10T07:58:57.214 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T07:58:57.243 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-10T07:58:57.243 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T07:58:57.309 INFO:tasks.cephadm:Adding mgr.x on smithi002 2024-09-10T07:58:57.309 INFO:tasks.cephadm:Adding mgr.y on smithi107 2024-09-10T07:58:57.309 INFO:tasks.cephadm:Adding mgr.z on smithi155 2024-09-10T07:58:57.310 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi002=x;smithi107=y;smithi155=z' 2024-09-10T07:58:57.584 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:58:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:57 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:57 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:57 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:57 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:57 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:57 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:57 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:57 smithi155 ceph-mon[35405]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T07:58:57.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:57 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:58:57.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:57 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:58:57.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:57 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:57.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:57 smithi155 ceph-mon[35405]: Reconfiguring daemon mon.a on smithi002 2024-09-10T07:58:57.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:57 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:57.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:57 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:57 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:57 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:57 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:57 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:57 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:57 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:57 smithi107 ceph-mon[35048]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T07:58:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:57 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:58:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:57 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:58:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:57 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:57 smithi107 ceph-mon[35048]: Reconfiguring daemon mon.a on smithi002 2024-09-10T07:58:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:57 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:58.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:57 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:58.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:57 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:58.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:57 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:58.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:57 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:58.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:57 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:58.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:57 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:58.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:57 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:58.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:57 smithi002 ceph-mon[30563]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T07:58:58.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:57 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:58:58.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:57 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:58:58.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:57 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:58.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:57 smithi002 ceph-mon[30563]: Reconfiguring daemon mon.a on smithi002 2024-09-10T07:58:58.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:57 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T07:58:58.032 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.027+0000 7fa10c75c640 1 -- 172.21.15.155:0/1748426501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1041005f0 msgr2=0x7fa1041009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:58.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.027+0000 7fa10c75c640 1 --2- 172.21.15.155:0/1748426501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1041005f0 0x7fa1041009f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa100009880 tx=0x7fa10002a5d0 comp rx=0 tx=0).stop 2024-09-10T07:58:58.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.027+0000 7fa10c75c640 1 -- 172.21.15.155:0/1748426501 shutdown_connections 2024-09-10T07:58:58.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.027+0000 7fa10c75c640 1 --2- 172.21.15.155:0/1748426501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1041005f0 0x7fa1041009f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:58.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.027+0000 7fa10c75c640 1 -- 172.21.15.155:0/1748426501 >> 172.21.15.155:0/1748426501 conn(0x7fa1040fbda0 msgr2=0x7fa1040fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:58.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.027+0000 7fa10c75c640 1 -- 172.21.15.155:0/1748426501 shutdown_connections 2024-09-10T07:58:58.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.027+0000 7fa10c75c640 1 -- 172.21.15.155:0/1748426501 wait complete. 2024-09-10T07:58:58.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.028+0000 7fa10c75c640 1 Processor -- start 2024-09-10T07:58:58.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.028+0000 7fa10c75c640 1 -- start start 2024-09-10T07:58:58.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.028+0000 7fa10c75c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1041005f0 0x7fa104197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:58.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.028+0000 7fa10c75c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa104197560 0x7fa10419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:58.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.028+0000 7fa10c75c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1041979e0 0x7fa10419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:58.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.028+0000 7fa10c75c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa1041066e0 con 0x7fa104197560 2024-09-10T07:58:58.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.028+0000 7fa10c75c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa104106560 con 0x7fa1041005f0 2024-09-10T07:58:58.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.028+0000 7fa10c75c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa104106860 con 0x7fa1041979e0 2024-09-10T07:58:58.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.028+0000 7fa10acf9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1041979e0 0x7fa10419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:58.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.028+0000 7fa10acf9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1041979e0 0x7fa10419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:54010/0 (socket says 172.21.15.155:54010) 2024-09-10T07:58:58.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.028+0000 7fa10acf9640 1 -- 172.21.15.155:0/763695911 learned_addr learned my addr 172.21.15.155:0/763695911 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-10T07:58:58.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa109cf7640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa104197560 0x7fa10419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:58.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa10a4f8640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1041005f0 0x7fa104197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:58.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa10acf9640 1 -- 172.21.15.155:0/763695911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1041005f0 msgr2=0x7fa104197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:58.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa10acf9640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1041005f0 0x7fa104197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:58.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa10acf9640 1 -- 172.21.15.155:0/763695911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa104197560 msgr2=0x7fa10419c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:58.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa10acf9640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa104197560 0x7fa10419c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:58.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa10acf9640 1 -- 172.21.15.155:0/763695911 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10419f420 con 0x7fa1041979e0 2024-09-10T07:58:58.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa109cf7640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa104197560 0x7fa10419c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:58:58.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa10acf9640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1041979e0 0x7fa10419eb60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00ba50 tx=0x7fa0fc00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:58.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa0fb7fe640 1 -- 172.21.15.155:0/763695911 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc019070 con 0x7fa1041979e0 2024-09-10T07:58:58.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa0fb7fe640 1 -- 172.21.15.155:0/763695911 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0fc00cbe0 con 0x7fa1041979e0 2024-09-10T07:58:58.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa10c75c640 1 -- 172.21.15.155:0/763695911 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10419f710 con 0x7fa1041979e0 2024-09-10T07:58:58.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa0fb7fe640 1 -- 172.21.15.155:0/763695911 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0076a0 con 0x7fa1041979e0 2024-09-10T07:58:58.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.029+0000 7fa10c75c640 1 -- 172.21.15.155:0/763695911 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10419fcd0 con 0x7fa1041979e0 2024-09-10T07:58:58.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.030+0000 7fa0fb7fe640 1 -- 172.21.15.155:0/763695911 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fa0fc01c070 con 0x7fa1041979e0 2024-09-10T07:58:58.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.030+0000 7fa0fb7fe640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa0f00394f0 0x7fa0f003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:58:58.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.030+0000 7fa10c75c640 1 -- 172.21.15.155:0/763695911 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa10419ff90 con 0x7fa1041979e0 2024-09-10T07:58:58.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.030+0000 7fa0fb7fe640 1 -- 172.21.15.155:0/763695911 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa0fc04bc10 con 0x7fa1041979e0 2024-09-10T07:58:58.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.031+0000 7fa10a4f8640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa0f00394f0 0x7fa0f003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:58:58.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.031+0000 7fa10a4f8640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa0f00394f0 0x7fa0f003b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa100009230 tx=0x7fa1000315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:58:58.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.035+0000 7fa0fb7fe640 1 -- 172.21.15.155:0/763695911 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0fc0112a0 con 0x7fa1041979e0 2024-09-10T07:58:58.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.211+0000 7fa10c75c640 1 -- 172.21.15.155:0/763695911 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi107=y;smithi155=z", "target": ["mon-mgr", ""]}) -- 0x7fa104062630 con 0x7fa0f00394f0 2024-09-10T07:58:58.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.217+0000 7fa0fb7fe640 1 -- 172.21.15.155:0/763695911 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa104062630 con 0x7fa0f00394f0 2024-09-10T07:58:58.217 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mgr update... 2024-09-10T07:58:58.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.219+0000 7fa10c75c640 1 -- 172.21.15.155:0/763695911 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa0f00394f0 msgr2=0x7fa0f003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:58.219 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.219+0000 7fa10c75c640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa0f00394f0 0x7fa0f003b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa100009230 tx=0x7fa1000315d0 comp rx=0 tx=0).stop 2024-09-10T07:58:58.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.219+0000 7fa10c75c640 1 -- 172.21.15.155:0/763695911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1041979e0 msgr2=0x7fa10419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:58:58.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.219+0000 7fa10c75c640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1041979e0 0x7fa10419eb60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00ba50 tx=0x7fa0fc00bf10 comp rx=0 tx=0).stop 2024-09-10T07:58:58.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.219+0000 7fa10c75c640 1 -- 172.21.15.155:0/763695911 shutdown_connections 2024-09-10T07:58:58.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.219+0000 7fa10c75c640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa0f00394f0 0x7fa0f003b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:58.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.219+0000 7fa10c75c640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1041979e0 0x7fa10419eb60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:58.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.219+0000 7fa10c75c640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa104197560 0x7fa10419c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:58.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.219+0000 7fa10c75c640 1 --2- 172.21.15.155:0/763695911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1041005f0 0x7fa104197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:58:58.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.219+0000 7fa10c75c640 1 -- 172.21.15.155:0/763695911 >> 172.21.15.155:0/763695911 conn(0x7fa1040fbda0 msgr2=0x7fa10406acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:58:58.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.219+0000 7fa10c75c640 1 -- 172.21.15.155:0/763695911 shutdown_connections 2024-09-10T07:58:58.220 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:58:58.219+0000 7fa10c75c640 1 -- 172.21.15.155:0/763695911 wait complete. 2024-09-10T07:58:58.423 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 07:58:58 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T07:58:58.038+0000 7f6388000640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T07:58:58.817 DEBUG:teuthology.orchestra.run.smithi107:mgr.y> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T07:58:58.820 DEBUG:teuthology.orchestra.run.smithi155:mgr.z> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T07:58:58.822 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T07:58:58.822 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T07:58:58.822 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T07:58:58.859 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T07:58:58.860 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_1 2024-09-10T07:58:58.937 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T07:58:58.937 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:58:58.937 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 638 Links: 1 2024-09-10T07:58:58.937 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:58:58.937 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:58:58.938 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-10 07:58:13.675565321 +0000 2024-09-10T07:58:58.938 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-10 07:53:36.206078266 +0000 2024-09-10T07:58:58.938 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-10 07:53:36.206078266 +0000 2024-09-10T07:58:58.938 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-10 07:53:36.206078266 +0000 2024-09-10T07:58:58.938 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T07:58:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-10T07:58:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-10T07:58:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000185874 s, 2.8 MB/s 2024-09-10T07:58:59.034 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T07:58:59.073 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_2 2024-09-10T07:58:59.149 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T07:58:59.149 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:58:59.149 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 658 Links: 1 2024-09-10T07:58:59.149 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:58:59.149 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:58:59.149 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-10 07:58:13.722564248 +0000 2024-09-10T07:58:59.149 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-10 07:53:36.688067374 +0000 2024-09-10T07:58:59.149 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-10 07:53:36.688067374 +0000 2024-09-10T07:58:59.150 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-10 07:53:36.688067374 +0000 2024-09-10T07:58:59.150 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T07:58:59.246 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-10T07:58:59.246 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-10T07:58:59.246 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000189375 s, 2.7 MB/s 2024-09-10T07:58:59.249 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T07:58:59.286 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_3 2024-09-10T07:58:59.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:59 smithi002 ceph-mon[30563]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:59.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:59 smithi002 ceph-mon[30563]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi107=y;smithi155=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:58:59.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:59 smithi002 ceph-mon[30563]: Saving service mgr spec with placement smithi002=x;smithi107=y;smithi155=z;count:3 2024-09-10T07:58:59.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:59.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:59.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:59.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:59 smithi002 ceph-mon[30563]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T07:58:59.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:58:59.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:58:59.335 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:59.335 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:59 smithi002 ceph-mon[30563]: Reconfiguring daemon mgr.x on smithi002 2024-09-10T07:58:59.335 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:58:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:59.363 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T07:58:59.363 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:58:59.363 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 678 Links: 1 2024-09-10T07:58:59.363 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:58:59.363 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:58:59.364 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-10 07:58:13.765563266 +0000 2024-09-10T07:58:59.364 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-10 07:53:37.182056211 +0000 2024-09-10T07:58:59.364 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-10 07:53:37.182056211 +0000 2024-09-10T07:58:59.364 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-10 07:53:37.182056211 +0000 2024-09-10T07:58:59.364 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T07:58:59.457 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-10T07:58:59.457 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-10T07:58:59.458 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000183902 s, 2.8 MB/s 2024-09-10T07:58:59.460 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T07:58:59.497 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_4 2024-09-10T07:58:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T07:58:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:58:59.574 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 696 Links: 1 2024-09-10T07:58:59.574 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:58:59.574 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:58:59.574 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-10 07:58:13.806562330 +0000 2024-09-10T07:58:59.574 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-10 07:53:37.633046020 +0000 2024-09-10T07:58:59.574 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-10 07:53:37.633046020 +0000 2024-09-10T07:58:59.574 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-10 07:53:37.633046020 +0000 2024-09-10T07:58:59.575 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T07:58:59.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:59 smithi155 ceph-mon[35405]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:59.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:59 smithi155 ceph-mon[35405]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi107=y;smithi155=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:58:59.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:59 smithi155 ceph-mon[35405]: Saving service mgr spec with placement smithi002=x;smithi107=y;smithi155=z;count:3 2024-09-10T07:58:59.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:59.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:59.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:59.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:59 smithi155 ceph-mon[35405]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T07:58:59.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:58:59.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:58:59.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:59.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:59 smithi155 ceph-mon[35405]: Reconfiguring daemon mgr.x on smithi002 2024-09-10T07:58:59.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:58:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:59.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:59 smithi107 ceph-mon[35048]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:58:59.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:59 smithi107 ceph-mon[35048]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi107=y;smithi155=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:58:59.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:59 smithi107 ceph-mon[35048]: Saving service mgr spec with placement smithi002=x;smithi107=y;smithi155=z;count:3 2024-09-10T07:58:59.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:59 smithi107 ceph-mon[35048]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T07:58:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:58:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:58:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:58:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:59 smithi107 ceph-mon[35048]: Reconfiguring daemon mgr.x on smithi002 2024-09-10T07:58:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:58:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:58:59.667 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-10T07:58:59.667 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-10T07:58:59.668 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000182305 s, 2.8 MB/s 2024-09-10T07:58:59.670 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T07:58:59.707 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T07:58:59.707 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T07:58:59.727 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T07:58:59.727 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-09-10T07:58:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T07:58:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:58:59.786 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-09-10T07:58:59.786 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:58:59.786 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:58:59.786 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-10 07:58:48.475602207 +0000 2024-09-10T07:58:59.786 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-10 07:52:58.749917492 +0000 2024-09-10T07:58:59.786 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-10 07:52:58.749917492 +0000 2024-09-10T07:58:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-10 07:52:58.749917492 +0000 2024-09-10T07:58:59.787 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T07:58:59.852 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-10T07:58:59.853 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-10T07:58:59.853 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000249444 s, 2.1 MB/s 2024-09-10T07:58:59.854 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T07:58:59.912 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-09-10T07:58:59.968 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T07:58:59.969 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:58:59.969 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-09-10T07:58:59.969 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:58:59.969 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:58:59.969 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-10 07:58:48.476602190 +0000 2024-09-10T07:58:59.969 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-10 07:52:59.091912074 +0000 2024-09-10T07:58:59.969 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-10 07:52:59.091912074 +0000 2024-09-10T07:58:59.969 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-10 07:52:59.091912074 +0000 2024-09-10T07:58:59.969 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T07:59:00.034 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-10T07:59:00.034 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-10T07:59:00.034 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000241791 s, 2.1 MB/s 2024-09-10T07:59:00.035 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T07:59:00.093 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-09-10T07:59:00.149 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T07:59:00.149 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:59:00.149 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-10T07:59:00.149 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:59:00.149 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:59:00.149 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-10 07:58:48.476602190 +0000 2024-09-10T07:59:00.149 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-10 07:52:59.406907083 +0000 2024-09-10T07:59:00.150 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-10 07:52:59.406907083 +0000 2024-09-10T07:59:00.150 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-10 07:52:59.406907083 +0000 2024-09-10T07:59:00.150 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T07:59:00.212 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-10T07:59:00.212 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-10T07:59:00.212 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000190738 s, 2.7 MB/s 2024-09-10T07:59:00.214 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T07:59:00.269 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-09-10T07:59:00.326 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T07:59:00.327 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:59:00.327 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-09-10T07:59:00.327 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:59:00.327 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:59:00.327 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-10 07:58:48.477602174 +0000 2024-09-10T07:59:00.327 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-10 07:52:59.725902028 +0000 2024-09-10T07:59:00.327 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-10 07:52:59.725902028 +0000 2024-09-10T07:59:00.327 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-10 07:52:59.725902028 +0000 2024-09-10T07:59:00.327 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T07:59:00.391 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-10T07:59:00.391 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-10T07:59:00.391 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000191808 s, 2.7 MB/s 2024-09-10T07:59:00.392 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T07:59:00.449 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-10T07:59:00.449 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T07:59:00.469 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T07:59:00.469 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-09-10T07:59:00.530 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T07:59:00.530 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:59:00.530 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-10T07:59:00.530 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:59:00.530 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:59:00.530 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-09-10 07:58:39.758382127 +0000 2024-09-10T07:59:00.530 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-09-10 07:52:58.863221831 +0000 2024-09-10T07:59:00.530 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-09-10 07:52:58.863221831 +0000 2024-09-10T07:59:00.530 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-09-10 07:52:58.863221831 +0000 2024-09-10T07:59:00.531 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T07:59:00.598 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-09-10T07:59:00.598 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-09-10T07:59:00.598 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000259049 s, 2.0 MB/s 2024-09-10T07:59:00.599 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T07:59:00.659 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-09-10T07:59:00.716 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T07:59:00.716 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:59:00.717 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T07:59:00.717 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:59:00.717 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:59:00.717 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-09-10 07:58:39.759382106 +0000 2024-09-10T07:59:00.717 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-09-10 07:52:59.194215027 +0000 2024-09-10T07:59:00.717 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-09-10 07:52:59.194215027 +0000 2024-09-10T07:59:00.717 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-09-10 07:52:59.194215027 +0000 2024-09-10T07:59:00.717 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T07:59:00.783 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-09-10T07:59:00.783 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-09-10T07:59:00.783 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000248102 s, 2.1 MB/s 2024-09-10T07:59:00.784 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T07:59:00.843 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-09-10T07:59:00.899 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T07:59:00.899 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:59:00.900 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-10T07:59:00.900 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:59:00.900 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:59:00.900 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-09-10 07:58:39.759382106 +0000 2024-09-10T07:59:00.900 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-09-10 07:52:59.526208203 +0000 2024-09-10T07:59:00.900 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-09-10 07:52:59.526208203 +0000 2024-09-10T07:59:00.900 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-09-10 07:52:59.526208203 +0000 2024-09-10T07:59:00.900 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T07:59:00.965 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-09-10T07:59:00.965 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-09-10T07:59:00.965 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000219903 s, 2.3 MB/s 2024-09-10T07:59:00.966 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T07:59:01.024 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-09-10T07:59:01.080 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T07:59:01.080 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T07:59:01.080 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-10T07:59:01.080 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T07:59:01.080 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T07:59:01.080 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-09-10 07:58:39.760382086 +0000 2024-09-10T07:59:01.080 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-09-10 07:52:59.852201502 +0000 2024-09-10T07:59:01.080 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-09-10 07:52:59.852201502 +0000 2024-09-10T07:59:01.080 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-09-10 07:52:59.852201502 +0000 2024-09-10T07:59:01.081 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T07:59:01.144 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-09-10T07:59:01.144 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-09-10T07:59:01.144 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000185145 s, 2.8 MB/s 2024-09-10T07:59:01.145 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T07:59:01.202 INFO:tasks.cephadm:Deploying osd.0 on smithi002 with /dev/vg_nvme/lv_4... 2024-09-10T07:59:01.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T07:59:01.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:01 smithi107 ceph-mon[35048]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:01.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:01.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:01.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:01 smithi107 ceph-mon[35048]: Reconfiguring mon.b (monmap changed)... 2024-09-10T07:59:01.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:59:01.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:59:01.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:01.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:01 smithi107 ceph-mon[35048]: Reconfiguring daemon mon.b on smithi107 2024-09-10T07:59:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:01 smithi002 ceph-mon[30563]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:01 smithi002 ceph-mon[30563]: Reconfiguring mon.b (monmap changed)... 2024-09-10T07:59:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:59:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:59:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:01 smithi002 ceph-mon[30563]: Reconfiguring daemon mon.b on smithi107 2024-09-10T07:59:01.579 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:59:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:01 smithi155 ceph-mon[35405]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:01 smithi155 ceph-mon[35405]: Reconfiguring mon.b (monmap changed)... 2024-09-10T07:59:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:59:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:59:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:01 smithi155 ceph-mon[35405]: Reconfiguring daemon mon.b on smithi107 2024-09-10T07:59:02.725 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:02 smithi002 ceph-mon[30563]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:02.726 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:02 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:02.726 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:02 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:02.726 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:02 smithi002 ceph-mon[30563]: Reconfiguring mon.c (monmap changed)... 2024-09-10T07:59:02.726 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:02 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:59:02.726 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:02 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:59:02.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:02 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:02.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:02 smithi002 ceph-mon[30563]: Reconfiguring daemon mon.c on smithi155 2024-09-10T07:59:02.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:02 smithi155 ceph-mon[35405]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:02.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:02 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:02.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:02 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:02.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:02 smithi155 ceph-mon[35405]: Reconfiguring mon.c (monmap changed)... 2024-09-10T07:59:02.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:02 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:59:02.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:02 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:59:02.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:02 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:02.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:02 smithi155 ceph-mon[35405]: Reconfiguring daemon mon.c on smithi155 2024-09-10T07:59:02.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:02 smithi107 ceph-mon[35048]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:02.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:02 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:02.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:02 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:02.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:02 smithi107 ceph-mon[35048]: Reconfiguring mon.c (monmap changed)... 2024-09-10T07:59:02.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:02 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T07:59:02.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:02 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T07:59:02.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:02 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:02.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:02 smithi107 ceph-mon[35048]: Reconfiguring daemon mon.c on smithi155 2024-09-10T07:59:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:04.022 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:04.022 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:59:04.022 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T07:59:04.022 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:59:04.022 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:04.022 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:03 smithi002 ceph-mon[30563]: Deploying daemon mgr.z on smithi155 2024-09-10T07:59:04.067 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:04.068 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:04.068 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:04.068 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:04.068 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:04.068 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:04.068 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:59:04.068 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T07:59:04.068 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:59:04.068 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:04.068 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:03 smithi155 ceph-mon[35405]: Deploying daemon mgr.z on smithi155 2024-09-10T07:59:04.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:04.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:59:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T07:59:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:59:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:03 smithi107 ceph-mon[35048]: Deploying daemon mgr.z on smithi155 2024-09-10T07:59:04.600 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T07:59:04.633 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi002:vg_nvme/lv_4 2024-09-10T07:59:05.013 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T07:59:05.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:04 smithi002 ceph-mon[30563]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:05.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:04 smithi155 ceph-mon[35405]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:05.156 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:05.026+0000 7ff96d8f3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T07:59:05.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:04 smithi107 ceph-mon[35048]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:05.525 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:05.175+0000 7ff96d8f3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T07:59:05.525 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:05.240+0000 7ff96d8f3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T07:59:05.525 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:05.387+0000 7ff96d8f3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:59:05.525 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:05.525+0000 7ff96d8f3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T07:59:05.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.564+0000 7f2cf5000640 1 -- 172.21.15.2:0/3292838146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf0105920 msgr2=0x7f2cf010c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:59:05.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.564+0000 7f2cf5000640 1 --2- 172.21.15.2:0/3292838146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf0105920 0x7f2cf010c240 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009be0 tx=0x7f2ce802a6c0 comp rx=0 tx=0).stop 2024-09-10T07:59:05.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.564+0000 7f2cf5000640 1 -- 172.21.15.2:0/3292838146 shutdown_connections 2024-09-10T07:59:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.564+0000 7f2cf5000640 1 --2- 172.21.15.2:0/3292838146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf0105920 0x7f2cf010c240 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.564+0000 7f2cf5000640 1 --2- 172.21.15.2:0/3292838146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cf0104f60 0x7f2cf01053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.564+0000 7f2cf5000640 1 --2- 172.21.15.2:0/3292838146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cf0103e00 0x7f2cf0104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.564+0000 7f2cf5000640 1 -- 172.21.15.2:0/3292838146 >> 172.21.15.2:0/3292838146 conn(0x7f2cf00ff5b0 msgr2=0x7f2cf01019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:59:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.564+0000 7f2cf5000640 1 -- 172.21.15.2:0/3292838146 shutdown_connections 2024-09-10T07:59:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.564+0000 7f2cf5000640 1 -- 172.21.15.2:0/3292838146 wait complete. 2024-09-10T07:59:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.565+0000 7f2cf5000640 1 Processor -- start 2024-09-10T07:59:05.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.565+0000 7f2cf5000640 1 -- start start 2024-09-10T07:59:05.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2cf5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cf0103e00 0x7f2cf019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:59:05.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2cf5000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf0104f60 0x7f2cf019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:59:05.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2cf5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cf0105920 0x7f2cf01a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:59:05.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2cf5000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf010fab0 con 0x7f2cf0105920 2024-09-10T07:59:05.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2cf5000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf010f930 con 0x7f2cf0103e00 2024-09-10T07:59:05.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2cf5000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf010fc30 con 0x7f2cf0104f60 2024-09-10T07:59:05.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2ce7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf0104f60 0x7f2cf019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:59:05.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2cee800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cf0105920 0x7f2cf01a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:59:05.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2ce7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf0104f60 0x7f2cf019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:52782/0 (socket says 172.21.15.2:52782) 2024-09-10T07:59:05.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2cede00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cf0103e00 0x7f2cf019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:59:05.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2ce7e00640 1 -- 172.21.15.2:0/4150586224 learned_addr learned my addr 172.21.15.2:0/4150586224 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T07:59:05.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2ce7e00640 1 -- 172.21.15.2:0/4150586224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cf0103e00 msgr2=0x7f2cf019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:59:05.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2ce7e00640 1 --2- 172.21.15.2:0/4150586224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cf0103e00 0x7f2cf019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:05.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2ce7e00640 1 -- 172.21.15.2:0/4150586224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cf0105920 msgr2=0x7f2cf01a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:59:05.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2ce7e00640 1 --2- 172.21.15.2:0/4150586224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cf0105920 0x7f2cf01a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:05.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.566+0000 7f2ce7e00640 1 -- 172.21.15.2:0/4150586224 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf01a1c80 con 0x7f2cf0104f60 2024-09-10T07:59:05.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.567+0000 7f2ce7e00640 1 --2- 172.21.15.2:0/4150586224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf0104f60 0x7f2cf019a590 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800d950 tx=0x7f2cd800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:59:05.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.567+0000 7f2ce6a00640 1 -- 172.21.15.2:0/4150586224 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8017070 con 0x7f2cf0104f60 2024-09-10T07:59:05.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.567+0000 7f2ce6a00640 1 -- 172.21.15.2:0/4150586224 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd8009c30 con 0x7f2cf0104f60 2024-09-10T07:59:05.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.567+0000 7f2ce6a00640 1 -- 172.21.15.2:0/4150586224 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800f600 con 0x7f2cf0104f60 2024-09-10T07:59:05.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.567+0000 7f2cf5000640 1 -- 172.21.15.2:0/4150586224 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf01a1f70 con 0x7f2cf0104f60 2024-09-10T07:59:05.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.567+0000 7f2cf5000640 1 -- 172.21.15.2:0/4150586224 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf01a23d0 con 0x7f2cf0104f60 2024-09-10T07:59:05.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.568+0000 7f2ce6a00640 1 -- 172.21.15.2:0/4150586224 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f2cd801a070 con 0x7f2cf0104f60 2024-09-10T07:59:05.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.568+0000 7f2cf5000640 1 -- 172.21.15.2:0/4150586224 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf010b140 con 0x7f2cf0104f60 2024-09-10T07:59:05.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.569+0000 7f2ce6a00640 1 --2- 172.21.15.2:0/4150586224 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2cd40392c0 0x7f2cd403b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:59:05.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.569+0000 7f2ce6a00640 1 -- 172.21.15.2:0/4150586224 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2cd804bea0 con 0x7f2cf0104f60 2024-09-10T07:59:05.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.569+0000 7f2cede00640 1 --2- 172.21.15.2:0/4150586224 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2cd40392c0 0x7f2cd403b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:59:05.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.570+0000 7f2cede00640 1 --2- 172.21.15.2:0/4150586224 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2cd40392c0 0x7f2cd403b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009530 tx=0x7f2ce0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:59:05.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.574+0000 7f2ce6a00640 1 -- 172.21.15.2:0/4150586224 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2cd804f030 con 0x7f2cf0104f60 2024-09-10T07:59:05.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:05.765+0000 7f2cf5000640 1 -- 172.21.15.2:0/4150586224 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2cf0062630 con 0x7f2cd40392c0 2024-09-10T07:59:06.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:59:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T07:59:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:59:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:05 smithi002 ceph-mon[30563]: Deploying daemon mgr.y on smithi107 2024-09-10T07:59:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:59:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:59:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:06.156 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:05.841+0000 7ff96d8f3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T07:59:06.156 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:05.901+0000 7ff96d8f3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T07:59:06.156 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:06 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:06.087+0000 7ff96d8f3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T07:59:06.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:06.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:06.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:06.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:59:06.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T07:59:06.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:59:06.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:06.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-mon[35405]: Deploying daemon mgr.y on smithi107 2024-09-10T07:59:06.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:59:06.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:59:06.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:06.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:06.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:06.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:06.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T07:59:06.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T07:59:06.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T07:59:06.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:06.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:05 smithi107 ceph-mon[35048]: Deploying daemon mgr.y on smithi107 2024-09-10T07:59:06.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:59:06.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:59:06.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:06.916 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:06 smithi107 systemd[1]: Starting Ceph mgr.y for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T07:59:07.238 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:07 smithi107 ceph-mon[35048]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:07.238 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:07 smithi107 ceph-mon[35048]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:59:07.238 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:06 smithi107 podman[36403]: 2024-09-10 07:59:06.988020946 +0000 UTC m=+0.092583342 container create eeb53500d4322bee12cf4f83f39123d154e370402e69266d26f4a86fc25c5c43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T07:59:07.239 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:07 smithi107 podman[36403]: 2024-09-10 07:59:06.911366411 +0000 UTC m=+0.015928765 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T07:59:07.239 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:07 smithi107 podman[36403]: 2024-09-10 07:59:07.240023272 +0000 UTC m=+0.344585618 container init eeb53500d4322bee12cf4f83f39123d154e370402e69266d26f4a86fc25c5c43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T07:59:07.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:07 smithi002 ceph-mon[30563]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:07.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:07 smithi002 ceph-mon[30563]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:59:07.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:07 smithi155 ceph-mon[35405]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:07.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:07 smithi155 ceph-mon[35405]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:59:07.406 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:06 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:06.970+0000 7ff96d8f3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T07:59:07.406 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:07 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:07.039+0000 7ff96d8f3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T07:59:07.574 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:07 smithi107 podman[36403]: 2024-09-10 07:59:07.242807488 +0000 UTC m=+0.347369839 container start eeb53500d4322bee12cf4f83f39123d154e370402e69266d26f4a86fc25c5c43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T07:59:07.574 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:07 smithi107 ceph-mgr[36423]: -- 172.21.15.107:0/3004296699 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bc989530e0 con 0x55bc98931000 2024-09-10T07:59:07.575 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:07 smithi107 bash[36403]: eeb53500d4322bee12cf4f83f39123d154e370402e69266d26f4a86fc25c5c43 2024-09-10T07:59:07.575 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:07 smithi107 systemd[1]: Started Ceph mgr.y for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T07:59:07.575 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:07 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:07.361+0000 7f7b81198040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T07:59:07.575 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:07 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:07.423+0000 7f7b81198040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T07:59:07.575 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:07 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:07.575+0000 7f7b81198040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T07:59:07.916 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:07 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:07.645+0000 7f7b81198040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T07:59:07.916 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:07 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:07.799+0000 7f7b81198040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:59:08.067 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:07 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:07.779+0000 7ff96d8f3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T07:59:08.067 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:07 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:07.915+0000 7ff96d8f3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T07:59:08.067 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:08.068+0000 7ff96d8f3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:59:08.285 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:07 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:07.957+0000 7f7b81198040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T07:59:08.285 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:08 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:08.285+0000 7f7b81198040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T07:59:08.359 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:08.130+0000 7ff96d8f3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T07:59:08.359 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:08.292+0000 7ff96d8f3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T07:59:08.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:08.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:08.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:08.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:08.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:08.657 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:08.361+0000 7ff96d8f3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T07:59:08.657 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:08.424+0000 7ff96d8f3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T07:59:08.657 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:08.592+0000 7ff96d8f3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T07:59:08.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:08.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:08.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:08.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:08.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:08.666 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:08 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:08.347+0000 7f7b81198040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T07:59:08.666 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:08 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:08.533+0000 7f7b81198040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T07:59:08.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:08.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:08.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:08.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:08.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:08.936 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:08.661+0000 7ff96d8f3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T07:59:08.937 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:08.728+0000 7ff96d8f3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T07:59:08.937 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:08.815+0000 7ff96d8f3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T07:59:08.937 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:08.874+0000 7ff96d8f3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T07:59:08.937 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:08.937+0000 7ff96d8f3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T07:59:09.729 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:09.729 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.729 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.729 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.729 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.729 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.729 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:09.729 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:09.729 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.730 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:09.730 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:09.730 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:09.730 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.730 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3580643792' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41eb8073-0ba0-4e03-a60a-e7b27476ba0d"}]: dispatch 2024-09-10T07:59:09.730 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41eb8073-0ba0-4e03-a60a-e7b27476ba0d"}]: dispatch 2024-09-10T07:59:09.730 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41eb8073-0ba0-4e03-a60a-e7b27476ba0d"}]': finished 2024-09-10T07:59:09.730 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T07:59:09.730 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:09.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:09.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:09.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:09.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3580643792' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41eb8073-0ba0-4e03-a60a-e7b27476ba0d"}]: dispatch 2024-09-10T07:59:09.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41eb8073-0ba0-4e03-a60a-e7b27476ba0d"}]: dispatch 2024-09-10T07:59:09.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41eb8073-0ba0-4e03-a60a-e7b27476ba0d"}]': finished 2024-09-10T07:59:09.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T07:59:09.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:09.916 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:09.431+0000 7f7b81198040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T07:59:09.916 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:09.497+0000 7f7b81198040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T07:59:09.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:09.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3580643792' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41eb8073-0ba0-4e03-a60a-e7b27476ba0d"}]: dispatch 2024-09-10T07:59:09.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41eb8073-0ba0-4e03-a60a-e7b27476ba0d"}]: dispatch 2024-09-10T07:59:09.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41eb8073-0ba0-4e03-a60a-e7b27476ba0d"}]': finished 2024-09-10T07:59:09.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T07:59:09.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:10.156 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 07:59:09 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z[37216]: 2024-09-10T07:59:09.729+0000 7ff96d8f3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T07:59:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:10.427+0000 7f2ce6a00640 1 -- 172.21.15.2:0/4150586224 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mgrmap(e 14) ==== 92059+0+0 (secure 0 0 0) 0x7f2cd800f7a0 con 0x7f2cf0104f60 2024-09-10T07:59:10.456 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:10.202+0000 7f7b81198040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T07:59:10.456 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:10.315+0000 7f7b81198040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T07:59:10.456 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:10.456+0000 7f7b81198040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T07:59:10.456 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/4072803995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:59:10.456 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-mon[35048]: Standby manager daemon z started 2024-09-10T07:59:10.457 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-mon[35048]: from='mgr.? 172.21.15.155:0/2828793275' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T07:59:10.457 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-mon[35048]: from='mgr.? 172.21.15.155:0/2828793275' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T07:59:10.457 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-mon[35048]: from='mgr.? 172.21.15.155:0/2828793275' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T07:59:10.457 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-mon[35048]: from='mgr.? 172.21.15.155:0/2828793275' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T07:59:10.729 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:10.517+0000 7f7b81198040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T07:59:10.729 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:10.663+0000 7f7b81198040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T07:59:10.729 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:10.729+0000 7f7b81198040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T07:59:10.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:10 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/4072803995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:59:10.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:10 smithi002 ceph-mon[30563]: Standby manager daemon z started 2024-09-10T07:59:10.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:10 smithi002 ceph-mon[30563]: from='mgr.? 172.21.15.155:0/2828793275' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T07:59:10.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:10 smithi002 ceph-mon[30563]: from='mgr.? 172.21.15.155:0/2828793275' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T07:59:10.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:10 smithi002 ceph-mon[30563]: from='mgr.? 172.21.15.155:0/2828793275' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T07:59:10.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:10 smithi002 ceph-mon[30563]: from='mgr.? 172.21.15.155:0/2828793275' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T07:59:10.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:10 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/4072803995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:59:10.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:10 smithi155 ceph-mon[35405]: Standby manager daemon z started 2024-09-10T07:59:10.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:10 smithi155 ceph-mon[35405]: from='mgr.? 172.21.15.155:0/2828793275' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T07:59:10.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:10 smithi155 ceph-mon[35405]: from='mgr.? 172.21.15.155:0/2828793275' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T07:59:10.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:10 smithi155 ceph-mon[35405]: from='mgr.? 172.21.15.155:0/2828793275' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T07:59:10.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:10 smithi155 ceph-mon[35405]: from='mgr.? 172.21.15.155:0/2828793275' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T07:59:11.007 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:10.788+0000 7f7b81198040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T07:59:11.007 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:10 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:10.943+0000 7f7b81198040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T07:59:11.007 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:11 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:11.007+0000 7f7b81198040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T07:59:11.270 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:11 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:11.073+0000 7f7b81198040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T07:59:11.270 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:11 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:11.154+0000 7f7b81198040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T07:59:11.270 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:11 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:11.212+0000 7f7b81198040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T07:59:11.270 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:11 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:11.270+0000 7f7b81198040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T07:59:11.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:11 smithi107 ceph-mon[35048]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:11.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:11 smithi107 ceph-mon[35048]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-10T07:59:11.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:11 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T07:59:11.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:11 smithi002 ceph-mon[30563]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:11.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:11 smithi002 ceph-mon[30563]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-10T07:59:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:11 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T07:59:11.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:11 smithi155 ceph-mon[35405]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:11.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:11 smithi155 ceph-mon[35405]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-10T07:59:11.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:11 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T07:59:12.416 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 07:59:12 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y[36419]: 2024-09-10T07:59:12.047+0000 7f7b81198040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T07:59:12.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:12.436+0000 7f2ce6a00640 1 -- 172.21.15.2:0/4150586224 <== mon.1 v2:172.21.15.155:3300/0 8 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f2cd80107e0 con 0x7f2cf0104f60 2024-09-10T07:59:12.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:12 smithi002 ceph-mon[30563]: Standby manager daemon y started 2024-09-10T07:59:12.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:12 smithi002 ceph-mon[30563]: from='mgr.? 172.21.15.107:0/3567650110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T07:59:12.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:12 smithi002 ceph-mon[30563]: from='mgr.? 172.21.15.107:0/3567650110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T07:59:12.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:12 smithi002 ceph-mon[30563]: from='mgr.? 172.21.15.107:0/3567650110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T07:59:12.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:12 smithi002 ceph-mon[30563]: from='mgr.? 172.21.15.107:0/3567650110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T07:59:12.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:12 smithi155 ceph-mon[35405]: Standby manager daemon y started 2024-09-10T07:59:12.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:12 smithi155 ceph-mon[35405]: from='mgr.? 172.21.15.107:0/3567650110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T07:59:12.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:12 smithi155 ceph-mon[35405]: from='mgr.? 172.21.15.107:0/3567650110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T07:59:12.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:12 smithi155 ceph-mon[35405]: from='mgr.? 172.21.15.107:0/3567650110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T07:59:12.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:12 smithi155 ceph-mon[35405]: from='mgr.? 172.21.15.107:0/3567650110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T07:59:12.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:12 smithi107 ceph-mon[35048]: Standby manager daemon y started 2024-09-10T07:59:12.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:12 smithi107 ceph-mon[35048]: from='mgr.? 172.21.15.107:0/3567650110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T07:59:12.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:12 smithi107 ceph-mon[35048]: from='mgr.? 172.21.15.107:0/3567650110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T07:59:12.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:12 smithi107 ceph-mon[35048]: from='mgr.? 172.21.15.107:0/3567650110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T07:59:12.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:12 smithi107 ceph-mon[35048]: from='mgr.? 172.21.15.107:0/3567650110' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T07:59:13.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:13 smithi002 ceph-mon[30563]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:13.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:13 smithi002 ceph-mon[30563]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-10T07:59:13.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:13 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T07:59:13.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:13 smithi155 ceph-mon[35405]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:13.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:13 smithi155 ceph-mon[35405]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-10T07:59:13.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:13 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T07:59:13.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:13 smithi107 ceph-mon[35048]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:13.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:13 smithi107 ceph-mon[35048]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-10T07:59:13.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:13 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T07:59:15.658 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:15 smithi002 ceph-mon[30563]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:15.708 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:15 smithi155 ceph-mon[35405]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:15.718 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:15 smithi107 ceph-mon[35048]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:17.702 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:17 smithi002 ceph-mon[30563]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:17.702 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:17 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T07:59:17.702 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:17 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:17 smithi155 ceph-mon[35405]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:17 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T07:59:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:17 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:17.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:17 smithi107 ceph-mon[35048]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:17.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:17 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T07:59:17.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:17 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:18.720 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:18 smithi002 ceph-mon[30563]: Deploying daemon osd.0 on smithi002 2024-09-10T07:59:18.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:18 smithi155 ceph-mon[35405]: Deploying daemon osd.0 on smithi002 2024-09-10T07:59:18.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:18 smithi107 ceph-mon[35048]: Deploying daemon osd.0 on smithi002 2024-09-10T07:59:19.645 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:19 smithi002 ceph-mon[30563]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:19 smithi155 ceph-mon[35405]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:19.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:19 smithi107 ceph-mon[35048]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:21.722 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:21 smithi002 ceph-mon[30563]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:21 smithi155 ceph-mon[35405]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:21.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:21 smithi107 ceph-mon[35048]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:23 smithi002 ceph-mon[30563]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:23 smithi155 ceph-mon[35405]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:23.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:23 smithi107 ceph-mon[35048]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:25 smithi002 ceph-mon[30563]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:25 smithi155 ceph-mon[35405]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:25.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:25 smithi107 ceph-mon[35048]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:25.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:25.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:25.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:27.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:27 smithi002 ceph-mon[30563]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:27.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:27 smithi155 ceph-mon[35405]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:27.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:27 smithi107 ceph-mon[35048]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:28.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:28 smithi155 ceph-mon[35405]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:28.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:28 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:28.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:28 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:28.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:28 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:28.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:28 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:28.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:28 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:28.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:28 smithi107 ceph-mon[35048]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:28.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:28 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:28.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:28 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:28.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:28 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:28.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:28 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:28.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:28 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:28.985 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:28 smithi002 ceph-mon[30563]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:28.985 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:28 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:28.985 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:28 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:28.985 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:28 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:28.985 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:28 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:28.985 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:28 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:30.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:30.084+0000 7f2ce6a00640 1 -- 172.21.15.2:0/4150586224 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2cf0062630 con 0x7f2cd40392c0 2024-09-10T07:59:30.084 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 0 on host 'smithi002' 2024-09-10T07:59:30.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:30.086+0000 7f2cf5000640 1 -- 172.21.15.2:0/4150586224 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2cd40392c0 msgr2=0x7f2cd403b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:59:30.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:30.086+0000 7f2cf5000640 1 --2- 172.21.15.2:0/4150586224 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2cd40392c0 0x7f2cd403b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009530 tx=0x7f2ce0009260 comp rx=0 tx=0).stop 2024-09-10T07:59:30.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:30.086+0000 7f2cf5000640 1 -- 172.21.15.2:0/4150586224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf0104f60 msgr2=0x7f2cf019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:59:30.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:30.086+0000 7f2cf5000640 1 --2- 172.21.15.2:0/4150586224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf0104f60 0x7f2cf019a590 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800d950 tx=0x7f2cd800de10 comp rx=0 tx=0).stop 2024-09-10T07:59:30.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:30.086+0000 7f2cf5000640 1 -- 172.21.15.2:0/4150586224 shutdown_connections 2024-09-10T07:59:30.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:30.086+0000 7f2cf5000640 1 --2- 172.21.15.2:0/4150586224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2cf0105920 0x7f2cf01a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:30.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:30.086+0000 7f2cf5000640 1 --2- 172.21.15.2:0/4150586224 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2cd40392c0 0x7f2cd403b780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:30.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:30.086+0000 7f2cf5000640 1 --2- 172.21.15.2:0/4150586224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf0104f60 0x7f2cf019a590 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:30.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:30.086+0000 7f2cf5000640 1 --2- 172.21.15.2:0/4150586224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cf0103e00 0x7f2cf019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:30.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:30.086+0000 7f2cf5000640 1 -- 172.21.15.2:0/4150586224 >> 172.21.15.2:0/4150586224 conn(0x7f2cf00ff5b0 msgr2=0x7f2cf0108310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:59:30.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:30.086+0000 7f2cf5000640 1 -- 172.21.15.2:0/4150586224 shutdown_connections 2024-09-10T07:59:30.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T07:59:30.086+0000 7f2cf5000640 1 -- 172.21.15.2:0/4150586224 wait complete. 2024-09-10T07:59:30.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:30 smithi002 ceph-mon[30563]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:30.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:30.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:30.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:30.835 DEBUG:teuthology.orchestra.run.smithi002:osd.0> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T07:59:30.840 INFO:tasks.cephadm:Deploying osd.1 on smithi107 with /dev/vg_nvme/lv_4... 2024-09-10T07:59:30.840 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T07:59:30.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:30 smithi155 ceph-mon[35405]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:30.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:30.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:30.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:30.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:30 smithi107 ceph-mon[35048]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:30.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:30.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:30.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:31.092 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:59:31.379 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 07:59:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0[41774]: 2024-09-10T07:59:31.143+0000 7f345cc44640 -1 osd.0 0 log_to_monitors true 2024-09-10T07:59:31.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:31 smithi107 ceph-mon[35048]: from='osd.0 [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T07:59:31.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:31 smithi107 ceph-mon[35048]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T07:59:32.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:31 smithi002 ceph-mon[30563]: from='osd.0 [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T07:59:32.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:31 smithi002 ceph-mon[30563]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T07:59:32.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:31 smithi155 ceph-mon[35405]: from='osd.0 [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T07:59:32.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:31 smithi155 ceph-mon[35405]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T07:59:32.806 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:32 smithi107 ceph-mon[35048]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:32.807 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:32 smithi107 ceph-mon[35048]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T07:59:32.807 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:32 smithi107 ceph-mon[35048]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T07:59:32.807 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:32 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:32.807 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:32 smithi107 ceph-mon[35048]: from='osd.0 [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T07:59:32.807 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:32 smithi107 ceph-mon[35048]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T07:59:33.020 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 07:59:32 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0[41774]: 2024-09-10T07:59:32.674+0000 7f3457400640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T07:59:33.020 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 07:59:32 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0[41774]: 2024-09-10T07:59:32.689+0000 7f3452400640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:59:33.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:32 smithi002 ceph-mon[30563]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:33.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:32 smithi002 ceph-mon[30563]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T07:59:33.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:32 smithi002 ceph-mon[30563]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T07:59:33.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:32 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:33.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:32 smithi002 ceph-mon[30563]: from='osd.0 [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T07:59:33.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:32 smithi002 ceph-mon[30563]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T07:59:33.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:32 smithi155 ceph-mon[35405]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:33.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:32 smithi155 ceph-mon[35405]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T07:59:33.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:32 smithi155 ceph-mon[35405]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T07:59:33.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:32 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:33.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:32 smithi155 ceph-mon[35405]: from='osd.0 [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T07:59:33.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:32 smithi155 ceph-mon[35405]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T07:59:33.579 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T07:59:33.605 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-09-10T07:59:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:33 smithi002 ceph-mon[30563]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-09-10T07:59:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:33 smithi002 ceph-mon[30563]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T07:59:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:33 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:33 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:33.855 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T07:59:33.885 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:33 smithi107 ceph-mon[35048]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-09-10T07:59:33.886 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:33 smithi107 ceph-mon[35048]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T07:59:33.886 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:33 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:33.886 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:33 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:34.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:33 smithi155 ceph-mon[35405]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-09-10T07:59:34.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:33 smithi155 ceph-mon[35405]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T07:59:34.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:33 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:34.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:33 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:34.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.304+0000 7f7620670640 1 -- 172.21.15.107:0/2773902203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7618101c20 msgr2=0x7f7618102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:59:34.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.304+0000 7f7620670640 1 --2- 172.21.15.107:0/2773902203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7618101c20 0x7f7618102020 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7614009900 tx=0x7f761402a410 comp rx=0 tx=0).stop 2024-09-10T07:59:34.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.304+0000 7f7620670640 1 -- 172.21.15.107:0/2773902203 shutdown_connections 2024-09-10T07:59:34.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 --2- 172.21.15.107:0/2773902203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76181037e0 0x7f761810a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:34.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 --2- 172.21.15.107:0/2773902203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7618102e20 0x7f76181032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:34.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 --2- 172.21.15.107:0/2773902203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7618101c20 0x7f7618102020 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:34.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 -- 172.21.15.107:0/2773902203 >> 172.21.15.107:0/2773902203 conn(0x7f76180fd3b0 msgr2=0x7f76180ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:59:34.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 -- 172.21.15.107:0/2773902203 shutdown_connections 2024-09-10T07:59:34.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 -- 172.21.15.107:0/2773902203 wait complete. 2024-09-10T07:59:34.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 Processor -- start 2024-09-10T07:59:34.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 -- start start 2024-09-10T07:59:34.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7618101c20 0x7f7618197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:59:34.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7618102e20 0x7f76181984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:59:34.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76181037e0 0x7f761819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:59:34.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f761810d950 con 0x7f7618101c20 2024-09-10T07:59:34.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f761810d7d0 con 0x7f7618102e20 2024-09-10T07:59:34.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f7620670640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f761810dad0 con 0x7f76181037e0 2024-09-10T07:59:34.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f761dc0b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7618102e20 0x7f76181984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:59:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f761dc0b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7618102e20 0x7f76181984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48530/0 (socket says 172.21.15.107:48530) 2024-09-10T07:59:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.305+0000 7f761dc0b640 1 -- 172.21.15.107:0/2699071168 learned_addr learned my addr 172.21.15.107:0/2699071168 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T07:59:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.306+0000 7f761e40c640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7618101c20 0x7f7618197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:59:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.306+0000 7f761ec0d640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76181037e0 0x7f761819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:59:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.306+0000 7f761dc0b640 1 -- 172.21.15.107:0/2699071168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76181037e0 msgr2=0x7f761819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:59:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.306+0000 7f761dc0b640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76181037e0 0x7f761819f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.306+0000 7f761dc0b640 1 -- 172.21.15.107:0/2699071168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7618101c20 msgr2=0x7f7618197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:59:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.306+0000 7f761dc0b640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7618101c20 0x7f7618197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.306+0000 7f761dc0b640 1 -- 172.21.15.107:0/2699071168 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761819fbb0 con 0x7f7618102e20 2024-09-10T07:59:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.306+0000 7f761dc0b640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7618102e20 0x7f76181984c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f760800d950 tx=0x7f760800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:59:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.306+0000 7f760f7fe640 1 -- 172.21.15.107:0/2699071168 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608017070 con 0x7f7618102e20 2024-09-10T07:59:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.306+0000 7f760f7fe640 1 -- 172.21.15.107:0/2699071168 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7608009c30 con 0x7f7618102e20 2024-09-10T07:59:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.306+0000 7f760f7fe640 1 -- 172.21.15.107:0/2699071168 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760800f6b0 con 0x7f7618102e20 2024-09-10T07:59:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.306+0000 7f7620670640 1 -- 172.21.15.107:0/2699071168 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761819fea0 con 0x7f7618102e20 2024-09-10T07:59:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.306+0000 7f7620670640 1 -- 172.21.15.107:0/2699071168 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76181a0420 con 0x7f7618102e20 2024-09-10T07:59:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.307+0000 7f7620670640 1 -- 172.21.15.107:0/2699071168 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7618108fd0 con 0x7f7618102e20 2024-09-10T07:59:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.309+0000 7f760f7fe640 1 -- 172.21.15.107:0/2699071168 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f76080109d0 con 0x7f7618102e20 2024-09-10T07:59:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.309+0000 7f760f7fe640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76040a2ba0 0x7f76040a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:59:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.309+0000 7f760f7fe640 1 -- 172.21.15.107:0/2699071168 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f76080ccf60 con 0x7f7618102e20 2024-09-10T07:59:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.309+0000 7f761e40c640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76040a2ba0 0x7f76040a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:59:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.310+0000 7f761e40c640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76040a2ba0 0x7f76040a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f761402aa00 tx=0x7f76140315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:59:34.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.313+0000 7f760f7fe640 1 -- 172.21.15.107:0/2699071168 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7608009de0 con 0x7f7618102e20 2024-09-10T07:59:34.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:34.494+0000 7f7620670640 1 -- 172.21.15.107:0/2699071168 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7618062630 con 0x7f76040a2ba0 2024-09-10T07:59:34.871 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: purged_snaps scrub starts 2024-09-10T07:59:34.871 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: purged_snaps scrub ok 2024-09-10T07:59:34.871 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:34.871 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:34.871 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: osd.0 [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139] boot 2024-09-10T07:59:34.871 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T07:59:34.871 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:34.871 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: Detected new or changed devices on smithi002 2024-09-10T07:59:34.872 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:34.872 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:34.872 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:59:34.872 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi002 to 17094M 2024-09-10T07:59:34.872 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:34.872 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:34.872 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:34.872 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:34.872 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:59:34.872 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:59:34.872 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:59:34.872 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:35.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: purged_snaps scrub starts 2024-09-10T07:59:35.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: purged_snaps scrub ok 2024-09-10T07:59:35.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:35.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:35.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: osd.0 [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139] boot 2024-09-10T07:59:35.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T07:59:35.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:35.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: Detected new or changed devices on smithi002 2024-09-10T07:59:35.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:35.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:35.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:59:35.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi002 to 17094M 2024-09-10T07:59:35.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:35.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:35.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:35.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:35.022 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:59:35.022 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:59:35.022 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:59:35.022 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:35.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: purged_snaps scrub starts 2024-09-10T07:59:35.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: purged_snaps scrub ok 2024-09-10T07:59:35.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T07:59:35.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:35.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: osd.0 [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139] boot 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: Detected new or changed devices on smithi002 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi002 to 17094M 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T07:59:35.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T07:59:35.158 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:35.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:35 smithi107 ceph-mon[35048]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T07:59:36.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:35 smithi002 ceph-mon[30563]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T07:59:36.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:35 smithi155 ceph-mon[35405]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T07:59:36.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:36 smithi107 ceph-mon[35048]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:37.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:36 smithi002 ceph-mon[30563]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:37.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:36 smithi155 ceph-mon[35405]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:38.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:37 smithi002 ceph-mon[30563]: from='client.? 172.21.15.107:0/1081811420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6"}]: dispatch 2024-09-10T07:59:38.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:37 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6"}]: dispatch 2024-09-10T07:59:38.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:37 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6"}]': finished 2024-09-10T07:59:38.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:37 smithi002 ceph-mon[30563]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T07:59:38.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:37 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:59:38.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:37 smithi002 ceph-mon[30563]: from='client.? 172.21.15.107:0/2475552607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:59:38.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:37 smithi155 ceph-mon[35405]: from='client.? 172.21.15.107:0/1081811420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6"}]: dispatch 2024-09-10T07:59:38.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:37 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6"}]: dispatch 2024-09-10T07:59:38.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:37 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6"}]': finished 2024-09-10T07:59:38.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:37 smithi155 ceph-mon[35405]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T07:59:38.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:37 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:59:38.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:37 smithi155 ceph-mon[35405]: from='client.? 172.21.15.107:0/2475552607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:59:38.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:37 smithi107 ceph-mon[35048]: from='client.? 172.21.15.107:0/1081811420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6"}]: dispatch 2024-09-10T07:59:38.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:37 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6"}]: dispatch 2024-09-10T07:59:38.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:37 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6"}]': finished 2024-09-10T07:59:38.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:37 smithi107 ceph-mon[35048]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T07:59:38.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:37 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:59:38.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:37 smithi107 ceph-mon[35048]: from='client.? 172.21.15.107:0/2475552607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T07:59:39.075 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:38 smithi002 ceph-mon[30563]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:39.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:38 smithi155 ceph-mon[35405]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:39.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:38 smithi107 ceph-mon[35048]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:41.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:40 smithi155 ceph-mon[35405]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:41.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:40 smithi107 ceph-mon[35048]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:41.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:40 smithi002 ceph-mon[30563]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:43.086 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:42 smithi107 ceph-mon[35048]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:43.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:42 smithi155 ceph-mon[35405]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:42 smithi002 ceph-mon[30563]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:45.027 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:44 smithi107 ceph-mon[35048]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:45.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:44 smithi155 ceph-mon[35405]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:45.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:44 smithi002 ceph-mon[30563]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:46.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T07:59:46.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:46.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:45 smithi155 ceph-mon[35405]: Deploying daemon osd.1 on smithi107 2024-09-10T07:59:46.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:45 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T07:59:46.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:45 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:46.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:45 smithi107 ceph-mon[35048]: Deploying daemon osd.1 on smithi107 2024-09-10T07:59:46.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T07:59:46.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:46.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:45 smithi002 ceph-mon[30563]: Deploying daemon osd.1 on smithi107 2024-09-10T07:59:46.908 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:46 smithi107 ceph-mon[35048]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:47.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:46 smithi155 ceph-mon[35405]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:47.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:46 smithi002 ceph-mon[30563]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:49.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:48 smithi155 ceph-mon[35405]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:49.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:48 smithi107 ceph-mon[35048]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:49.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:48 smithi002 ceph-mon[30563]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:51.106 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:50 smithi107 ceph-mon[35048]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:51.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:50 smithi155 ceph-mon[35405]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:51.272 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:50 smithi002 ceph-mon[30563]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:52.077 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:51 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:52.077 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:51 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:52.077 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:51 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:52.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:51 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:52.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:51 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:52.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:51 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:52.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:51 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:52.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:51 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:52.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:51 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:53.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:53 smithi002 ceph-mon[30563]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:53.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:53 smithi107 ceph-mon[35048]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:53.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:53 smithi155 ceph-mon[35405]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:54.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:54 smithi155 ceph-mon[35405]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:54.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:54.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:54.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:54.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:54.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:54.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:54 smithi107 ceph-mon[35048]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:54.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:54.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:54.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:54.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:54.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:55.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:54 smithi002 ceph-mon[30563]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:55.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:55.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:55.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:55.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:55.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:55.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:55.329+0000 7f760f7fe640 1 -- 172.21.15.107:0/2699071168 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7618062630 con 0x7f76040a2ba0 2024-09-10T07:59:55.329 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 1 on host 'smithi107' 2024-09-10T07:59:55.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:55.331+0000 7f7620670640 1 -- 172.21.15.107:0/2699071168 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76040a2ba0 msgr2=0x7f76040a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:59:55.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:55.331+0000 7f7620670640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76040a2ba0 0x7f76040a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f761402aa00 tx=0x7f76140315b0 comp rx=0 tx=0).stop 2024-09-10T07:59:55.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:55.331+0000 7f7620670640 1 -- 172.21.15.107:0/2699071168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7618102e20 msgr2=0x7f76181984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:59:55.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:55.331+0000 7f7620670640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7618102e20 0x7f76181984c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f760800d950 tx=0x7f760800de10 comp rx=0 tx=0).stop 2024-09-10T07:59:55.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:55.331+0000 7f7620670640 1 -- 172.21.15.107:0/2699071168 shutdown_connections 2024-09-10T07:59:55.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:55.331+0000 7f7620670640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76040a2ba0 0x7f76040a5060 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:55.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:55.331+0000 7f7620670640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76181037e0 0x7f761819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:55.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:55.331+0000 7f7620670640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7618102e20 0x7f76181984c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:55.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:55.331+0000 7f7620670640 1 --2- 172.21.15.107:0/2699071168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7618101c20 0x7f7618197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:55.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:55.331+0000 7f7620670640 1 -- 172.21.15.107:0/2699071168 >> 172.21.15.107:0/2699071168 conn(0x7f76180fd3b0 msgr2=0x7f76181061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:59:55.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:55.331+0000 7f7620670640 1 -- 172.21.15.107:0/2699071168 shutdown_connections 2024-09-10T07:59:55.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T07:59:55.331+0000 7f7620670640 1 -- 172.21.15.107:0/2699071168 wait complete. 2024-09-10T07:59:55.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:55.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:55.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:55.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:55.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:55.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:55.993 DEBUG:teuthology.orchestra.run.smithi107:osd.1> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T07:59:55.995 INFO:tasks.cephadm:Deploying osd.2 on smithi155 with /dev/vg_nvme/lv_4... 2024-09-10T07:59:55.996 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T07:59:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T07:59:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:56.246 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:59:57.371 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:57 smithi107 ceph-mon[35048]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:57.372 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:57 smithi107 ceph-mon[35048]: from='osd.1 [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T07:59:57.372 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:57 smithi107 ceph-mon[35048]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T07:59:57.372 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 07:59:57 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1[39509]: 2024-09-10T07:59:57.253+0000 7f397f115640 -1 osd.1 0 log_to_monitors true 2024-09-10T07:59:57.419 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:57 smithi155 ceph-mon[35405]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:57.420 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:57 smithi155 ceph-mon[35405]: from='osd.1 [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T07:59:57.420 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:57 smithi155 ceph-mon[35405]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T07:59:57.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:57 smithi002 ceph-mon[30563]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:57.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:57 smithi002 ceph-mon[30563]: from='osd.1 [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T07:59:57.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:57 smithi002 ceph-mon[30563]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T07:59:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T07:59:58.522 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_4 2024-09-10T07:59:58.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:58 smithi155 ceph-mon[35405]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T07:59:58.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:58 smithi155 ceph-mon[35405]: from='osd.1 [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T07:59:58.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:58 smithi155 ceph-mon[35405]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T07:59:58.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:58 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:59:58.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:58 smithi155 ceph-mon[35405]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T07:59:58.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:58 smithi107 ceph-mon[35048]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T07:59:58.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:58 smithi107 ceph-mon[35048]: from='osd.1 [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T07:59:58.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:58 smithi107 ceph-mon[35048]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T07:59:58.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:58 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:59:58.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:58 smithi107 ceph-mon[35048]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T07:59:58.667 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 07:59:58 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1[39509]: 2024-09-10T07:59:58.339+0000 7f397b098640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T07:59:58.667 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 07:59:58 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1[39509]: 2024-09-10T07:59:58.343+0000 7f3976ec0640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T07:59:58.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:58 smithi002 ceph-mon[30563]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T07:59:58.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:58 smithi002 ceph-mon[30563]: from='osd.1 [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T07:59:58.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:58 smithi002 ceph-mon[30563]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T07:59:58.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:58 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:59:58.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:58 smithi002 ceph-mon[30563]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T07:59:58.770 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T07:59:59.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.200+0000 7fe5965da640 1 -- 172.21.15.155:0/1197645396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5901026c0 msgr2=0x7fe590108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:59:59.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.200+0000 7fe5965da640 1 --2- 172.21.15.155:0/1197645396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5901026c0 0x7fe590108f50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe588009ca0 tx=0x7fe58802a8c0 comp rx=0 tx=0).stop 2024-09-10T07:59:59.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.200+0000 7fe5965da640 1 -- 172.21.15.155:0/1197645396 shutdown_connections 2024-09-10T07:59:59.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.200+0000 7fe5965da640 1 --2- 172.21.15.155:0/1197645396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5901026c0 0x7fe590108f50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:59.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.200+0000 7fe5965da640 1 --2- 172.21.15.155:0/1197645396 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe590101d00 0x7fe590102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:59.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.200+0000 7fe5965da640 1 --2- 172.21.15.155:0/1197645396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe590100b00 0x7fe590100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:59.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.200+0000 7fe5965da640 1 -- 172.21.15.155:0/1197645396 >> 172.21.15.155:0/1197645396 conn(0x7fe5900fc270 msgr2=0x7fe5900fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T07:59:59.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.200+0000 7fe5965da640 1 -- 172.21.15.155:0/1197645396 shutdown_connections 2024-09-10T07:59:59.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.200+0000 7fe5965da640 1 -- 172.21.15.155:0/1197645396 wait complete. 2024-09-10T07:59:59.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.201+0000 7fe5965da640 1 Processor -- start 2024-09-10T07:59:59.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.201+0000 7fe5965da640 1 -- start start 2024-09-10T07:59:59.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.201+0000 7fe5965da640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe590100b00 0x7fe59006b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:59:59.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.201+0000 7fe5965da640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe590101d00 0x7fe59019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:59:59.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.201+0000 7fe5965da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5901026c0 0x7fe59019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:59:59.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.201+0000 7fe5965da640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe59010c800 con 0x7fe5901026c0 2024-09-10T07:59:59.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.201+0000 7fe5965da640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe59010c680 con 0x7fe590101d00 2024-09-10T07:59:59.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.201+0000 7fe5965da640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe59010c980 con 0x7fe590100b00 2024-09-10T07:59:59.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.201+0000 7fe58ffff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe590100b00 0x7fe59006b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:59:59.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.201+0000 7fe58ffff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe590100b00 0x7fe59006b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:51356/0 (socket says 172.21.15.155:51356) 2024-09-10T07:59:59.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.201+0000 7fe58ffff640 1 -- 172.21.15.155:0/3366356931 learned_addr learned my addr 172.21.15.155:0/3366356931 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-10T07:59:59.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe58f7fe640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe590101d00 0x7fe59019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:59:59.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe594b77640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5901026c0 0x7fe59019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:59:59.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe58ffff640 1 -- 172.21.15.155:0/3366356931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe590101d00 msgr2=0x7fe59019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:59:59.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe58ffff640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe590101d00 0x7fe59019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:59.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe58ffff640 1 -- 172.21.15.155:0/3366356931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5901026c0 msgr2=0x7fe59019f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T07:59:59.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe58ffff640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5901026c0 0x7fe59019f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T07:59:59.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe58ffff640 1 -- 172.21.15.155:0/3366356931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59019fba0 con 0x7fe590100b00 2024-09-10T07:59:59.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe594b77640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5901026c0 0x7fe59019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T07:59:59.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe58f7fe640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe590101d00 0x7fe59019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T07:59:59.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe58ffff640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe590100b00 0x7fe59006b330 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe58000c970 tx=0x7fe58000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:59:59.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe58d7fa640 1 -- 172.21.15.155:0/3366356931 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580004220 con 0x7fe590100b00 2024-09-10T07:59:59.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe5965da640 1 -- 172.21.15.155:0/3366356931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59019fe90 con 0x7fe590100b00 2024-09-10T07:59:59.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe58d7fa640 1 -- 172.21.15.155:0/3366356931 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5800043c0 con 0x7fe590100b00 2024-09-10T07:59:59.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe58d7fa640 1 -- 172.21.15.155:0/3366356931 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580004a00 con 0x7fe590100b00 2024-09-10T07:59:59.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.202+0000 7fe5965da640 1 -- 172.21.15.155:0/3366356931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5901a0370 con 0x7fe590100b00 2024-09-10T07:59:59.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.203+0000 7fe5965da640 1 -- 172.21.15.155:0/3366356931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe590107fa0 con 0x7fe590100b00 2024-09-10T07:59:59.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.205+0000 7fe58d7fa640 1 -- 172.21.15.155:0/3366356931 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fe5800079b0 con 0x7fe590100b00 2024-09-10T07:59:59.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.206+0000 7fe58d7fa640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5780a2af0 0x7fe5780a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T07:59:59.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.206+0000 7fe58d7fa640 1 -- 172.21.15.155:0/3366356931 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fe5800cdc80 con 0x7fe590100b00 2024-09-10T07:59:59.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.206+0000 7fe58f7fe640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5780a2af0 0x7fe5780a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T07:59:59.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.207+0000 7fe58f7fe640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5780a2af0 0x7fe5780a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe59006b0b0 tx=0x7fe584008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T07:59:59.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.209+0000 7fe58d7fa640 1 -- 172.21.15.155:0/3366356931 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe580014030 con 0x7fe590100b00 2024-09-10T07:59:59.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T07:59:59.391+0000 7fe5965da640 1 -- 172.21.15.155:0/3366356931 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe590062630 con 0x7fe5780a2af0 2024-09-10T07:59:59.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:59 smithi155 ceph-mon[35405]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:59.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:59 smithi155 ceph-mon[35405]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T07:59:59.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:59 smithi155 ceph-mon[35405]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T07:59:59.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:59:59.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:59:59.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:59.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:59.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:59:59.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:59.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:59.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:59.391 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 07:59:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:59.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:59 smithi002 ceph-mon[30563]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:59.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:59 smithi002 ceph-mon[30563]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T07:59:59.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:59 smithi002 ceph-mon[30563]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T07:59:59.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:59:59.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:59:59.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:59.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:59.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:59:59.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:59.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:59.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:59.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 07:59:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:59.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:59 smithi107 ceph-mon[35048]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T07:59:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:59 smithi107 ceph-mon[35048]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T07:59:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:59 smithi107 ceph-mon[35048]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T07:59:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:59:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T07:59:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T07:59:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T07:59:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T07:59:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T07:59:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 07:59:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:00.620 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:00 smithi155 ceph-mon[35405]: purged_snaps scrub starts 2024-09-10T08:00:00.620 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:00 smithi155 ceph-mon[35405]: purged_snaps scrub ok 2024-09-10T08:00:00.620 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:00 smithi155 ceph-mon[35405]: Detected new or changed devices on smithi107 2024-09-10T08:00:00.621 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:00 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi107 to 17087M 2024-09-10T08:00:00.621 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:00 smithi155 ceph-mon[35405]: osd.1 [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441] boot 2024-09-10T08:00:00.621 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:00 smithi155 ceph-mon[35405]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T08:00:00.621 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:00 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:00:00.621 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:00 smithi155 ceph-mon[35405]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:00:00.621 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:00 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:00:00.621 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:00 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:00:00.621 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:00 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:00.621 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:00 smithi155 ceph-mon[35405]: overall HEALTH_OK 2024-09-10T08:00:00.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:00 smithi107 ceph-mon[35048]: purged_snaps scrub starts 2024-09-10T08:00:00.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:00 smithi107 ceph-mon[35048]: purged_snaps scrub ok 2024-09-10T08:00:00.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:00 smithi107 ceph-mon[35048]: Detected new or changed devices on smithi107 2024-09-10T08:00:00.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:00 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi107 to 17087M 2024-09-10T08:00:00.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:00 smithi107 ceph-mon[35048]: osd.1 [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441] boot 2024-09-10T08:00:00.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:00 smithi107 ceph-mon[35048]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T08:00:00.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:00 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:00:00.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:00 smithi107 ceph-mon[35048]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:00:00.668 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:00 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:00:00.668 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:00 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:00:00.668 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:00 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:00.668 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:00 smithi107 ceph-mon[35048]: overall HEALTH_OK 2024-09-10T08:00:00.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:00 smithi002 ceph-mon[30563]: purged_snaps scrub starts 2024-09-10T08:00:00.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:00 smithi002 ceph-mon[30563]: purged_snaps scrub ok 2024-09-10T08:00:00.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:00 smithi002 ceph-mon[30563]: Detected new or changed devices on smithi107 2024-09-10T08:00:00.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:00 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi107 to 17087M 2024-09-10T08:00:00.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:00 smithi002 ceph-mon[30563]: osd.1 [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441] boot 2024-09-10T08:00:00.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:00 smithi002 ceph-mon[30563]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T08:00:00.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:00 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:00:00.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:00 smithi002 ceph-mon[30563]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:00:00.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:00 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:00:00.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:00 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:00:00.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:00 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:00.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:00 smithi002 ceph-mon[30563]: overall HEALTH_OK 2024-09-10T08:00:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:01 smithi155 ceph-mon[35405]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:00:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:01 smithi155 ceph-mon[35405]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T08:00:01.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:01 smithi107 ceph-mon[35048]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:00:01.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:01 smithi107 ceph-mon[35048]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T08:00:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:01 smithi002 ceph-mon[30563]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:00:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:01 smithi002 ceph-mon[30563]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T08:00:02.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:02 smithi155 ceph-mon[35405]: from='client.? 172.21.15.155:0/1369323742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1c5e964-c621-4441-a87f-17c96c05de10"}]: dispatch 2024-09-10T08:00:02.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:02 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1c5e964-c621-4441-a87f-17c96c05de10"}]: dispatch 2024-09-10T08:00:02.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:02 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1c5e964-c621-4441-a87f-17c96c05de10"}]': finished 2024-09-10T08:00:02.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:02 smithi155 ceph-mon[35405]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T08:00:02.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:02 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:02.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:02 smithi155 ceph-mon[35405]: from='client.? 172.21.15.155:0/2850037195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:00:02.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:02 smithi107 ceph-mon[35048]: from='client.? 172.21.15.155:0/1369323742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1c5e964-c621-4441-a87f-17c96c05de10"}]: dispatch 2024-09-10T08:00:02.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:02 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1c5e964-c621-4441-a87f-17c96c05de10"}]: dispatch 2024-09-10T08:00:02.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:02 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1c5e964-c621-4441-a87f-17c96c05de10"}]': finished 2024-09-10T08:00:02.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:02 smithi107 ceph-mon[35048]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T08:00:02.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:02 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:02.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:02 smithi107 ceph-mon[35048]: from='client.? 172.21.15.155:0/2850037195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:00:02.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:02 smithi002 ceph-mon[30563]: from='client.? 172.21.15.155:0/1369323742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1c5e964-c621-4441-a87f-17c96c05de10"}]: dispatch 2024-09-10T08:00:02.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:02 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1c5e964-c621-4441-a87f-17c96c05de10"}]: dispatch 2024-09-10T08:00:02.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:02 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1c5e964-c621-4441-a87f-17c96c05de10"}]': finished 2024-09-10T08:00:02.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:02 smithi002 ceph-mon[30563]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T08:00:02.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:02 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:02.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:02 smithi002 ceph-mon[30563]: from='client.? 172.21.15.155:0/2850037195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:00:03.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:03 smithi155 ceph-mon[35405]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:03.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:03 smithi107 ceph-mon[35048]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:03.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:03 smithi002 ceph-mon[30563]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:05.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:05 smithi155 ceph-mon[35405]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:05.669 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:05 smithi107 ceph-mon[35048]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:05 smithi002 ceph-mon[30563]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:07.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:07 smithi155 ceph-mon[35405]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:07.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:07 smithi107 ceph-mon[35048]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:07.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:07 smithi002 ceph-mon[30563]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:09.506 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:09 smithi155 ceph-mon[35405]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:09.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:09 smithi107 ceph-mon[35048]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:09.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:09 smithi002 ceph-mon[30563]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:10.502 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:10 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T08:00:10.503 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:10 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:10.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:10 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T08:00:10.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:10 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:10.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:10 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T08:00:10.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:10 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:11.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:11 smithi155 ceph-mon[35405]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:11.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:11 smithi155 ceph-mon[35405]: Deploying daemon osd.2 on smithi155 2024-09-10T08:00:11.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:11 smithi107 ceph-mon[35048]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:11.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:11 smithi107 ceph-mon[35048]: Deploying daemon osd.2 on smithi155 2024-09-10T08:00:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:11 smithi002 ceph-mon[30563]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:11 smithi002 ceph-mon[30563]: Deploying daemon osd.2 on smithi155 2024-09-10T08:00:13.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:13 smithi155 ceph-mon[35405]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:13.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:13 smithi107 ceph-mon[35048]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:13.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:13 smithi002 ceph-mon[30563]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:15.502 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:15 smithi155 ceph-mon[35405]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:15.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:15 smithi107 ceph-mon[35048]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:15.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:15 smithi002 ceph-mon[30563]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:16.456 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:16 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:00:16.456 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:16 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:16.456 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:16 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:16.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:16 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:00:16.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:16 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:16.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:16 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:16.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:16 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:00:16.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:16 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:16.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:16 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:17.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:17 smithi155 ceph-mon[35405]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:17.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:17 smithi107 ceph-mon[35048]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:17.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:17 smithi002 ceph-mon[30563]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:19.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:19 smithi002 ceph-mon[30563]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:19.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:19.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:19.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:19.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:00:19.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:19.607 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:19 smithi155 ceph-mon[35405]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:19.607 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:19.608 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:19.608 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:19.608 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:00:19.608 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:19.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:19 smithi107 ceph-mon[35048]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:19.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:19.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:19.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:19.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:00:19.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:20.237 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:00:20.237+0000 7fe58d7fa640 1 -- 172.21.15.155:0/3366356931 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe590062630 con 0x7fe5780a2af0 2024-09-10T08:00:20.237 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 2 on host 'smithi155' 2024-09-10T08:00:20.239 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:00:20.239+0000 7fe5965da640 1 -- 172.21.15.155:0/3366356931 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5780a2af0 msgr2=0x7fe5780a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:00:20.239 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:00:20.239+0000 7fe5965da640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5780a2af0 0x7fe5780a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe59006b0b0 tx=0x7fe584008040 comp rx=0 tx=0).stop 2024-09-10T08:00:20.239 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:00:20.239+0000 7fe5965da640 1 -- 172.21.15.155:0/3366356931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe590100b00 msgr2=0x7fe59006b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:00:20.239 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:00:20.239+0000 7fe5965da640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe590100b00 0x7fe59006b330 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe58000c970 tx=0x7fe58000ce30 comp rx=0 tx=0).stop 2024-09-10T08:00:20.239 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:00:20.239+0000 7fe5965da640 1 -- 172.21.15.155:0/3366356931 shutdown_connections 2024-09-10T08:00:20.239 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:00:20.239+0000 7fe5965da640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5901026c0 0x7fe59019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:20.239 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:00:20.239+0000 7fe5965da640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe590101d00 0x7fe59019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:20.239 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:00:20.239+0000 7fe5965da640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5780a2af0 0x7fe5780a4fb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:20.239 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:00:20.239+0000 7fe5965da640 1 --2- 172.21.15.155:0/3366356931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe590100b00 0x7fe59006b330 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:20.239 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:00:20.239+0000 7fe5965da640 1 -- 172.21.15.155:0/3366356931 >> 172.21.15.155:0/3366356931 conn(0x7fe5900fc270 msgr2=0x7fe590105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:00:20.240 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:00:20.239+0000 7fe5965da640 1 -- 172.21.15.155:0/3366356931 shutdown_connections 2024-09-10T08:00:20.240 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:00:20.239+0000 7fe5965da640 1 -- 172.21.15.155:0/3366356931 wait complete. 2024-09-10T08:00:20.901 DEBUG:teuthology.orchestra.run.smithi155:osd.2> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T08:00:20.904 INFO:tasks.cephadm:Deploying osd.3 on smithi002 with /dev/vg_nvme/lv_3... 2024-09-10T08:00:20.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T08:00:21.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:21 smithi002 ceph-mon[30563]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:21.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:00:21.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:21.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:21.285 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:00:21.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:21 smithi155 ceph-mon[35405]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:21.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:00:21.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:21.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:21.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:21 smithi107 ceph-mon[35048]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:21.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:00:21.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:21.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:22.656 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:00:22 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2[40370]: 2024-09-10T08:00:22.326+0000 7fbd7ff68640 -1 osd.2 0 log_to_monitors true 2024-09-10T08:00:23.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:23 smithi002 ceph-mon[30563]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:23.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:23 smithi002 ceph-mon[30563]: from='osd.2 [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:00:23.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:23 smithi002 ceph-mon[30563]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:00:23.311 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:23 smithi155 ceph-mon[35405]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:23.311 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:23 smithi155 ceph-mon[35405]: from='osd.2 [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:00:23.311 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:23 smithi155 ceph-mon[35405]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:00:23.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:23 smithi107 ceph-mon[35048]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:23.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:23 smithi107 ceph-mon[35048]: from='osd.2 [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:00:23.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:23 smithi107 ceph-mon[35048]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:00:24.307 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:00:24.340 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi002:vg_nvme/lv_3 2024-09-10T08:00:24.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:24 smithi002 ceph-mon[30563]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T08:00:24.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:24 smithi002 ceph-mon[30563]: from='osd.2 [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:00:24.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:24 smithi002 ceph-mon[30563]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T08:00:24.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:24 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:24.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:24 smithi002 ceph-mon[30563]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:00:24.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:24 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:24.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:24 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:24.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:24 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:00:24.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:24 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:24.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:24 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:24.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:24 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:00:24.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:24 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:24.656 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2[40370]: 2024-09-10T08:00:24.230+0000 7fbd7beeb640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T08:00:24.656 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2[40370]: 2024-09-10T08:00:24.234+0000 7fbd77d13640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:00:24.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-mon[35405]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T08:00:24.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-mon[35405]: from='osd.2 [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:00:24.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-mon[35405]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T08:00:24.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:24.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-mon[35405]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:00:24.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:24.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:24.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:00:24.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:24.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:24.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:00:24.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:24 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:24.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:24 smithi107 ceph-mon[35048]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T08:00:24.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:24 smithi107 ceph-mon[35048]: from='osd.2 [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:00:24.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:24 smithi107 ceph-mon[35048]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T08:00:24.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:24 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:24.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:24 smithi107 ceph-mon[35048]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:00:24.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:24 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:24.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:24 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:24.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:24 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:00:24.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:24 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:24.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:24 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:24.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:24 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:00:24.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:24 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:24.709 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:00:25.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:25 smithi002 ceph-mon[30563]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:25.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:25 smithi002 ceph-mon[30563]: Detected new or changed devices on smithi155 2024-09-10T08:00:25.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:25 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi155 to 17088M 2024-09-10T08:00:25.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:25 smithi002 ceph-mon[30563]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-10T08:00:25.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:25 smithi002 ceph-mon[30563]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T08:00:25.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:25.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:25.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:25.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.406+0000 7fae01200640 1 -- 172.21.15.2:0/68020747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc074c50 msgr2=0x7fadfc0750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:00:25.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.406+0000 7fae01200640 1 --2- 172.21.15.2:0/68020747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc074c50 0x7fadfc0750b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fade4009900 tx=0x7fade402a410 comp rx=0 tx=0).stop 2024-09-10T08:00:25.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.406+0000 7fae01200640 1 -- 172.21.15.2:0/68020747 shutdown_connections 2024-09-10T08:00:25.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.406+0000 7fae01200640 1 --2- 172.21.15.2:0/68020747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadfc10e1e0 0x7fadfc1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:25.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.406+0000 7fae01200640 1 --2- 172.21.15.2:0/68020747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc074c50 0x7fadfc0750b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:25.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.406+0000 7fae01200640 1 --2- 172.21.15.2:0/68020747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadfc076220 0x7fadfc074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:25.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.406+0000 7fae01200640 1 -- 172.21.15.2:0/68020747 >> 172.21.15.2:0/68020747 conn(0x7fadfc0fd390 msgr2=0x7fadfc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:00:25.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.406+0000 7fae01200640 1 -- 172.21.15.2:0/68020747 shutdown_connections 2024-09-10T08:00:25.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.406+0000 7fae01200640 1 -- 172.21.15.2:0/68020747 wait complete. 2024-09-10T08:00:25.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.407+0000 7fae01200640 1 Processor -- start 2024-09-10T08:00:25.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.407+0000 7fae01200640 1 -- start start 2024-09-10T08:00:25.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.407+0000 7fae01200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadfc074c50 0x7fadfc19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:00:25.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.407+0000 7fae01200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadfc076220 0x7fadfc19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:00:25.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.407+0000 7fae01200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc10e1e0 0x7fadfc1a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:00:25.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fae01200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc113ef0 con 0x7fadfc10e1e0 2024-09-10T08:00:25.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fae01200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc113d70 con 0x7fadfc076220 2024-09-10T08:00:25.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fae01200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc114070 con 0x7fadfc074c50 2024-09-10T08:00:25.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fadf9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadfc074c50 0x7fadfc19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:00:25.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fadf3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadfc076220 0x7fadfc19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:00:25.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fadf9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadfc074c50 0x7fadfc19e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55028/0 (socket says 172.21.15.2:55028) 2024-09-10T08:00:25.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fadf9e00640 1 -- 172.21.15.2:0/296846076 learned_addr learned my addr 172.21.15.2:0/296846076 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:00:25.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fadfa800640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc10e1e0 0x7fadfc1a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:00:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fadf9e00640 1 -- 172.21.15.2:0/296846076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadfc076220 msgr2=0x7fadfc19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:00:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fadf9e00640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadfc076220 0x7fadfc19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:25.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fadf9e00640 1 -- 172.21.15.2:0/296846076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc10e1e0 msgr2=0x7fadfc1a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:00:25.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fadf9e00640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc10e1e0 0x7fadfc1a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:25.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fadf9e00640 1 -- 172.21.15.2:0/296846076 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadfc1a6130 con 0x7fadfc074c50 2024-09-10T08:00:25.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fadf3e00640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadfc076220 0x7fadfc19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:00:25.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.408+0000 7fadfa800640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc10e1e0 0x7fadfc1a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:00:25.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.409+0000 7fadf9e00640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadfc074c50 0x7fadfc19e500 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fadec00e9a0 tx=0x7fadec00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:00:25.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.409+0000 7fadf2a00640 1 -- 172.21.15.2:0/296846076 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec00cd80 con 0x7fadfc074c50 2024-09-10T08:00:25.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.409+0000 7fae01200640 1 -- 172.21.15.2:0/296846076 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadfc1a6420 con 0x7fadfc074c50 2024-09-10T08:00:25.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.409+0000 7fadf2a00640 1 -- 172.21.15.2:0/296846076 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadec0035f0 con 0x7fadfc074c50 2024-09-10T08:00:25.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.409+0000 7fadf2a00640 1 -- 172.21.15.2:0/296846076 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec010670 con 0x7fadfc074c50 2024-09-10T08:00:25.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.409+0000 7fae01200640 1 -- 172.21.15.2:0/296846076 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadfc1a6930 con 0x7fadfc074c50 2024-09-10T08:00:25.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.410+0000 7fae01200640 1 -- 172.21.15.2:0/296846076 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadfc1043b0 con 0x7fadfc074c50 2024-09-10T08:00:25.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.412+0000 7fadf2a00640 1 -- 172.21.15.2:0/296846076 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fadec0037a0 con 0x7fadfc074c50 2024-09-10T08:00:25.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.413+0000 7fadf2a00640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fade00a2ba0 0x7fade00a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:00:25.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.414+0000 7fadf2a00640 1 -- 172.21.15.2:0/296846076 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7fadec0cdc90 con 0x7fadfc074c50 2024-09-10T08:00:25.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.414+0000 7fadf3e00640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fade00a2ba0 0x7fade00a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:00:25.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.414+0000 7fadf3e00640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fade00a2ba0 0x7fade00a5060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fade4031440 tx=0x7fade40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:00:25.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.418+0000 7fadf2a00640 1 -- 172.21.15.2:0/296846076 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fadec09c180 con 0x7fadfc074c50 2024-09-10T08:00:25.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:25.610+0000 7fae01200640 1 -- 172.21.15.2:0/296846076 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fadfc062630 con 0x7fade00a2ba0 2024-09-10T08:00:25.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:25 smithi155 ceph-mon[35405]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:25.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:25 smithi155 ceph-mon[35405]: Detected new or changed devices on smithi155 2024-09-10T08:00:25.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:25 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi155 to 17088M 2024-09-10T08:00:25.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:25 smithi155 ceph-mon[35405]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-10T08:00:25.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:25 smithi155 ceph-mon[35405]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T08:00:25.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:25.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:25.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:25.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:25 smithi107 ceph-mon[35048]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:25.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:25 smithi107 ceph-mon[35048]: Detected new or changed devices on smithi155 2024-09-10T08:00:25.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:25 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi155 to 17088M 2024-09-10T08:00:25.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:25 smithi107 ceph-mon[35048]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-10T08:00:25.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:25 smithi107 ceph-mon[35048]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T08:00:25.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:25.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:25.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:26.460 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:26 smithi002 ceph-mon[30563]: purged_snaps scrub starts 2024-09-10T08:00:26.460 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:26 smithi002 ceph-mon[30563]: purged_snaps scrub ok 2024-09-10T08:00:26.460 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:26 smithi002 ceph-mon[30563]: osd.2 [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707] boot 2024-09-10T08:00:26.460 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:26 smithi002 ceph-mon[30563]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T08:00:26.460 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:26 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:26.460 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:26 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:00:26.461 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:26 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:00:26.461 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:26 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:26.461 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:26 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:00:26.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:26 smithi155 ceph-mon[35405]: purged_snaps scrub starts 2024-09-10T08:00:26.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:26 smithi155 ceph-mon[35405]: purged_snaps scrub ok 2024-09-10T08:00:26.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:26 smithi155 ceph-mon[35405]: osd.2 [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707] boot 2024-09-10T08:00:26.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:26 smithi155 ceph-mon[35405]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T08:00:26.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:26 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:26.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:26 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:00:26.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:26 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:00:26.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:26 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:26.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:26 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:00:26.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:26 smithi107 ceph-mon[35048]: purged_snaps scrub starts 2024-09-10T08:00:26.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:26 smithi107 ceph-mon[35048]: purged_snaps scrub ok 2024-09-10T08:00:26.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:26 smithi107 ceph-mon[35048]: osd.2 [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707] boot 2024-09-10T08:00:26.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:26 smithi107 ceph-mon[35048]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T08:00:26.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:26 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:00:26.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:26 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:00:26.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:26 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:00:26.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:26 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:26.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:26 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:00:27.509 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:27 smithi155 ceph-mon[35405]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:27.509 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:27 smithi155 ceph-mon[35405]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:00:27.510 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:27 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T08:00:27.510 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:27 smithi155 ceph-mon[35405]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T08:00:27.510 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:27 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:00:27.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:27 smithi002 ceph-mon[30563]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:27.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:27 smithi002 ceph-mon[30563]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:00:27.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T08:00:27.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:27 smithi002 ceph-mon[30563]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T08:00:27.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:00:27.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:27 smithi107 ceph-mon[35048]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:00:27.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:27 smithi107 ceph-mon[35048]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:00:27.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:27 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T08:00:27.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:27 smithi107 ceph-mon[35048]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T08:00:27.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:27 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:00:28.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:28 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:00:28.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:28 smithi002 ceph-mon[30563]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T08:00:28.520 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:00:28 smithi002 sudo[46306]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:00:28.520 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:00:28 smithi002 sudo[46306]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:00:28.520 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:00:28 smithi002 sudo[46306]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:00:28.520 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:00:28 smithi002 sudo[46306]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:00:28.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:28 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:00:28.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:28 smithi155 ceph-mon[35405]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T08:00:28.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:28 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:00:28.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:28 smithi107 ceph-mon[35048]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T08:00:29.019 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:00:28 smithi002 sudo[46344]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:00:29.020 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:00:28 smithi002 sudo[46344]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:00:29.020 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:00:28 smithi002 sudo[46344]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:00:29.020 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:00:28 smithi002 sudo[46344]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:00:29.303 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:00:29 smithi107 sudo[43215]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:00:29.303 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:00:29 smithi107 sudo[43215]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:00:29.303 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:00:29 smithi107 sudo[43215]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:00:29.303 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:00:29 smithi107 sudo[43215]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:00:29.303 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:29 smithi107 ceph-mon[35048]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:29.303 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:29 smithi107 ceph-mon[35048]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T08:00:29.303 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:29 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3755813699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "400fc872-8c48-40e3-8ecb-b34778d142a9"}]: dispatch 2024-09-10T08:00:29.303 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:29 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "400fc872-8c48-40e3-8ecb-b34778d142a9"}]: dispatch 2024-09-10T08:00:29.303 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:29 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "400fc872-8c48-40e3-8ecb-b34778d142a9"}]': finished 2024-09-10T08:00:29.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:29 smithi107 ceph-mon[35048]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T08:00:29.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:29.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:29.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:29 smithi002 ceph-mon[30563]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:29.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:29 smithi002 ceph-mon[30563]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T08:00:29.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:29 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3755813699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "400fc872-8c48-40e3-8ecb-b34778d142a9"}]: dispatch 2024-09-10T08:00:29.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:29 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "400fc872-8c48-40e3-8ecb-b34778d142a9"}]: dispatch 2024-09-10T08:00:29.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:29 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "400fc872-8c48-40e3-8ecb-b34778d142a9"}]': finished 2024-09-10T08:00:29.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:29 smithi002 ceph-mon[30563]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T08:00:29.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:29.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:29.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:29 smithi155 ceph-mon[35405]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:29.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:29 smithi155 ceph-mon[35405]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T08:00:29.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:29 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3755813699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "400fc872-8c48-40e3-8ecb-b34778d142a9"}]: dispatch 2024-09-10T08:00:29.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:29 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "400fc872-8c48-40e3-8ecb-b34778d142a9"}]: dispatch 2024-09-10T08:00:29.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:29 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "400fc872-8c48-40e3-8ecb-b34778d142a9"}]': finished 2024-09-10T08:00:29.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:29 smithi155 ceph-mon[35405]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T08:00:29.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:29.574 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:29.666 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:00:29 smithi107 sudo[43218]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:00:29.666 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:00:29 smithi107 sudo[43218]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:00:29.666 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:00:29 smithi107 sudo[43218]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:00:29.666 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:00:29 smithi107 sudo[43218]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:00:29.829 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:00:29 smithi155 sudo[44001]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:00:29.829 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:00:29 smithi155 sudo[44001]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:00:29.830 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:00:29 smithi155 sudo[44001]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:00:29.830 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:00:29 smithi155 sudo[44001]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:00:30.156 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:00:29 smithi155 sudo[44004]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:00:30.156 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:00:29 smithi155 sudo[44004]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:00:30.156 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:00:29 smithi155 sudo[44004]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:00:30.156 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:00:29 smithi155 sudo[44004]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:00:30.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:30 smithi002 sudo[46607]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:00:30.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:30 smithi002 sudo[46607]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:00:30.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:30 smithi002 sudo[46607]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:00:30.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:30 smithi002 sudo[46607]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:00:31.246 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:30 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1597130800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:00:31.246 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:31.246 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:31.246 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:30 smithi155 ceph-mon[35405]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:00:31.247 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 sudo[44007]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:00:31.247 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 sudo[44007]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:00:31.247 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 sudo[44007]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:00:31.247 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 sudo[44007]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:00:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:30 smithi002 sudo[46798]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:00:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:30 smithi002 sudo[46798]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:00:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:30 smithi002 sudo[46798]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:00:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:30 smithi002 sudo[46798]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:00:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:30 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1597130800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:00:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:30 smithi002 ceph-mon[30563]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:00:31.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:30 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1597130800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:00:31.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:31.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:31.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:30 smithi107 ceph-mon[35048]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:00:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:31.493+0000 7fadf2a00640 1 -- 172.21.15.2:0/296846076 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fadec0980f0 con 0x7fadfc074c50 2024-09-10T08:00:31.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 sudo[44010]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:00:31.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 sudo[44010]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:00:31.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 sudo[44010]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:00:31.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 sudo[44010]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:00:31.982 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 sudo[43221]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:00:31.982 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 sudo[43221]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:00:31.982 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 sudo[43221]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:00:31.982 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 sudo[43221]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:00:32.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:32.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:00:32.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:00:32.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:00:32.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:00:32.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:00:32.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T08:00:32.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:00:32.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:00:32.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:00:32.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:00:32.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:00:32.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:00:32.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:31 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:00:32.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:32.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:00:32.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:00:32.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:00:32.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:00:32.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:00:32.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T08:00:32.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:00:32.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:00:32.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:00:32.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:00:32.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:00:32.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:00:32.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:31 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:00:32.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 sudo[43224]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:00:32.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 sudo[43224]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:00:32.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 sudo[43224]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:00:32.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 sudo[43224]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:00:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:00:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:00:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:00:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:00:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:00:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T08:00:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:00:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:00:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:00:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:00:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:00:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:00:32.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:31 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:00:33.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:32 smithi002 ceph-mon[30563]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:00:33.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:32 smithi002 ceph-mon[30563]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:33.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:32 smithi002 ceph-mon[30563]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:00:33.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:32 smithi155 ceph-mon[35405]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:00:33.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:32 smithi155 ceph-mon[35405]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:33.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:32 smithi155 ceph-mon[35405]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:00:33.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:32 smithi107 ceph-mon[35048]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:00:33.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:32 smithi107 ceph-mon[35048]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:33.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:32 smithi107 ceph-mon[35048]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:00:35.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:34 smithi002 ceph-mon[30563]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:35.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:34 smithi155 ceph-mon[35405]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:35.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:34 smithi107 ceph-mon[35048]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:37.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:36 smithi002 ceph-mon[30563]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-10T08:00:37.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:36 smithi155 ceph-mon[35405]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-10T08:00:37.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:36 smithi107 ceph-mon[35048]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-10T08:00:39.089 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:39 smithi002 ceph-mon[30563]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T08:00:39.089 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T08:00:39.089 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:39.089 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:39 smithi002 ceph-mon[30563]: Deploying daemon osd.3 on smithi002 2024-09-10T08:00:39.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:39 smithi155 ceph-mon[35405]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T08:00:39.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T08:00:39.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:39.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:39 smithi155 ceph-mon[35405]: Deploying daemon osd.3 on smithi002 2024-09-10T08:00:39.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:39 smithi107 ceph-mon[35048]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T08:00:39.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T08:00:39.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:39.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:39 smithi107 ceph-mon[35048]: Deploying daemon osd.3 on smithi002 2024-09-10T08:00:41.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:41 smithi002 ceph-mon[30563]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T08:00:41.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:41 smithi155 ceph-mon[35405]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T08:00:41.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:41 smithi107 ceph-mon[35048]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T08:00:43.094 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:43 smithi002 ceph-mon[30563]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T08:00:43.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:43 smithi155 ceph-mon[35405]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T08:00:43.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:43 smithi107 ceph-mon[35048]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T08:00:45.173 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:45 smithi002 ceph-mon[30563]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:00:45.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:45 smithi155 ceph-mon[35405]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:00:45.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:45 smithi107 ceph-mon[35048]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:00:47.196 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:47 smithi002 ceph-mon[30563]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:00:47.196 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:47 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:00:47.196 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:47 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:47.197 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:47 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:47.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:47 smithi155 ceph-mon[35405]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:00:47.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:47 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:00:47.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:47 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:47.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:47 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:47.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:47 smithi107 ceph-mon[35048]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:00:47.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:47 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:00:47.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:47 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:47.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:47 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:49.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:49 smithi002 ceph-mon[30563]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:49.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:49 smithi155 ceph-mon[35405]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:49.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:49 smithi107 ceph-mon[35048]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:50 smithi002 ceph-mon[30563]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:00:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:51.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:50 smithi155 ceph-mon[35405]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:51.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:51.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:51.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:51.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:00:51.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:51.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:50 smithi107 ceph-mon[35048]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:51.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:51.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:51.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:51.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:00:51.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:51.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:51.174+0000 7fadf2a00640 1 -- 172.21.15.2:0/296846076 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fadfc062630 con 0x7fade00a2ba0 2024-09-10T08:00:51.174 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 3 on host 'smithi002' 2024-09-10T08:00:51.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:51.176+0000 7fae01200640 1 -- 172.21.15.2:0/296846076 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fade00a2ba0 msgr2=0x7fade00a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:00:51.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:51.176+0000 7fae01200640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fade00a2ba0 0x7fade00a5060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fade4031440 tx=0x7fade40315b0 comp rx=0 tx=0).stop 2024-09-10T08:00:51.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:51.176+0000 7fae01200640 1 -- 172.21.15.2:0/296846076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadfc074c50 msgr2=0x7fadfc19e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:00:51.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:51.176+0000 7fae01200640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadfc074c50 0x7fadfc19e500 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fadec00e9a0 tx=0x7fadec00ee60 comp rx=0 tx=0).stop 2024-09-10T08:00:51.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:51.177+0000 7fae01200640 1 -- 172.21.15.2:0/296846076 shutdown_connections 2024-09-10T08:00:51.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:51.177+0000 7fae01200640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc10e1e0 0x7fadfc1a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:51.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:51.177+0000 7fae01200640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadfc076220 0x7fadfc19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:51.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:51.177+0000 7fae01200640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fade00a2ba0 0x7fade00a5060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:51.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:51.177+0000 7fae01200640 1 --2- 172.21.15.2:0/296846076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadfc074c50 0x7fadfc19e500 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:51.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:51.177+0000 7fae01200640 1 -- 172.21.15.2:0/296846076 >> 172.21.15.2:0/296846076 conn(0x7fadfc0fd390 msgr2=0x7fadfc101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:00:51.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:51.177+0000 7fae01200640 1 -- 172.21.15.2:0/296846076 shutdown_connections 2024-09-10T08:00:51.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:00:51.177+0000 7fae01200640 1 -- 172.21.15.2:0/296846076 wait complete. 2024-09-10T08:00:51.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:51 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:00:51.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:51 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:51.908 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:51 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:51.942 DEBUG:teuthology.orchestra.run.smithi002:osd.3> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T08:00:51.946 INFO:tasks.cephadm:Deploying osd.4 on smithi107 with /dev/vg_nvme/lv_3... 2024-09-10T08:00:51.946 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T08:00:51.974 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:51 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:00:51.974 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:51 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:51.975 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:51 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:52.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:51 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:00:52.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:51 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:52.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:51 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:52.200 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:00:52.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:52 smithi107 ceph-mon[35048]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:52.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:52 smithi107 ceph-mon[35048]: from='osd.3 [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:00:52.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:52 smithi107 ceph-mon[35048]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:00:52.956 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:52 smithi002 ceph-mon[30563]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:52.957 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:52 smithi002 ceph-mon[30563]: from='osd.3 [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:00:52.957 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:52 smithi002 ceph-mon[30563]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:00:53.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:52 smithi155 ceph-mon[35405]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:53.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:52 smithi155 ceph-mon[35405]: from='osd.3 [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:00:53.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:52 smithi155 ceph-mon[35405]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:00:53.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:53 smithi002 ceph-mon[30563]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T08:00:53.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:53 smithi002 ceph-mon[30563]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T08:00:53.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:53 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:53.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:53 smithi002 ceph-mon[30563]: from='osd.3 [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:00:53.861 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:53 smithi002 ceph-mon[30563]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:00:53.861 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:00:53 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3[48390]: 2024-09-10T08:00:53.700+0000 7f09a9400640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T08:00:53.861 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:00:53 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3[48390]: 2024-09-10T08:00:53.706+0000 7f09a4400640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:00:53.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:53 smithi107 ceph-mon[35048]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T08:00:53.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:53 smithi107 ceph-mon[35048]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T08:00:53.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:53 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:53.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:53 smithi107 ceph-mon[35048]: from='osd.3 [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:00:53.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:53 smithi107 ceph-mon[35048]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:00:54.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:53 smithi155 ceph-mon[35405]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T08:00:54.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:53 smithi155 ceph-mon[35405]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T08:00:54.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:53 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:54.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:53 smithi155 ceph-mon[35405]: from='osd.3 [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:00:54.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:53 smithi155 ceph-mon[35405]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:00:54.706 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T08:00:54.732 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-09-10T08:00:55.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:54 smithi002 ceph-mon[30563]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:55.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:54 smithi002 ceph-mon[30563]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-09-10T08:00:55.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:54 smithi002 ceph-mon[30563]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T08:00:55.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:55.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:55.022 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:00:55.142 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:54 smithi107 ceph-mon[35048]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:55.143 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:54 smithi107 ceph-mon[35048]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-09-10T08:00:55.143 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:54 smithi107 ceph-mon[35048]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T08:00:55.143 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:55.143 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:55.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:54 smithi155 ceph-mon[35405]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:55.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:54 smithi155 ceph-mon[35405]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-09-10T08:00:55.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:54 smithi155 ceph-mon[35405]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T08:00:55.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:55.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:55.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.472+0000 7f6514aa9640 1 -- 172.21.15.107:0/2286829955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65101037c0 msgr2=0x7f651010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:00:55.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.472+0000 7f6514aa9640 1 --2- 172.21.15.107:0/2286829955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65101037c0 0x7f651010a050 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6508009ca0 tx=0x7f650802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:00:55.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.472+0000 7f6514aa9640 1 -- 172.21.15.107:0/2286829955 shutdown_connections 2024-09-10T08:00:55.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.472+0000 7f6514aa9640 1 --2- 172.21.15.107:0/2286829955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65101037c0 0x7f651010a050 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:55.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.472+0000 7f6514aa9640 1 --2- 172.21.15.107:0/2286829955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6510102e00 0x7f6510103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:55.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.472+0000 7f6514aa9640 1 --2- 172.21.15.107:0/2286829955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510101c00 0x7f6510102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:55.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.472+0000 7f6514aa9640 1 -- 172.21.15.107:0/2286829955 >> 172.21.15.107:0/2286829955 conn(0x7f65100fd390 msgr2=0x7f65100ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:00:55.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.472+0000 7f6514aa9640 1 -- 172.21.15.107:0/2286829955 shutdown_connections 2024-09-10T08:00:55.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.472+0000 7f6514aa9640 1 -- 172.21.15.107:0/2286829955 wait complete. 2024-09-10T08:00:55.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.473+0000 7f6514aa9640 1 Processor -- start 2024-09-10T08:00:55.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.473+0000 7f6514aa9640 1 -- start start 2024-09-10T08:00:55.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.473+0000 7f6514aa9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6510101c00 0x7f6510197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:00:55.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.473+0000 7f6514aa9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510102e00 0x7f6510198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:00:55.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.473+0000 7f6514aa9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65101037c0 0x7f651019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:00:55.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.473+0000 7f6514aa9640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f651010d8e0 con 0x7f6510102e00 2024-09-10T08:00:55.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.473+0000 7f6514aa9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f651010d760 con 0x7f6510101c00 2024-09-10T08:00:55.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.473+0000 7f6514aa9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f651010da60 con 0x7f65101037c0 2024-09-10T08:00:55.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.473+0000 7f650e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6510101c00 0x7f6510197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:00:55.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.473+0000 7f650e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6510101c00 0x7f6510197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53110/0 (socket says 172.21.15.107:53110) 2024-09-10T08:00:55.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.473+0000 7f650e59c640 1 -- 172.21.15.107:0/3476665451 learned_addr learned my addr 172.21.15.107:0/3476665451 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T08:00:55.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f650dd9b640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510102e00 0x7f6510198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:00:55.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f650ed9d640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65101037c0 0x7f651019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:00:55.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f650e59c640 1 -- 172.21.15.107:0/3476665451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65101037c0 msgr2=0x7f651019f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:00:55.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f650e59c640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65101037c0 0x7f651019f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:55.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f650e59c640 1 -- 172.21.15.107:0/3476665451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510102e00 msgr2=0x7f6510198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:00:55.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f650e59c640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510102e00 0x7f6510198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:00:55.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f650e59c640 1 -- 172.21.15.107:0/3476665451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651019fb60 con 0x7f6510101c00 2024-09-10T08:00:55.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f650dd9b640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510102e00 0x7f6510198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:00:55.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f650e59c640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6510101c00 0x7f6510197f30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f650000c970 tx=0x7f650000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:00:55.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f64ff7fe640 1 -- 172.21.15.107:0/3476665451 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500004220 con 0x7f6510101c00 2024-09-10T08:00:55.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f64ff7fe640 1 -- 172.21.15.107:0/3476665451 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65000043c0 con 0x7f6510101c00 2024-09-10T08:00:55.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f6514aa9640 1 -- 172.21.15.107:0/3476665451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651019fe50 con 0x7f6510101c00 2024-09-10T08:00:55.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f64ff7fe640 1 -- 172.21.15.107:0/3476665451 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650000f450 con 0x7f6510101c00 2024-09-10T08:00:55.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.474+0000 7f6514aa9640 1 -- 172.21.15.107:0/3476665451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65101a0320 con 0x7f6510101c00 2024-09-10T08:00:55.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.475+0000 7f6514aa9640 1 -- 172.21.15.107:0/3476665451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6510108fd0 con 0x7f6510101c00 2024-09-10T08:00:55.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.476+0000 7f64ff7fe640 1 -- 172.21.15.107:0/3476665451 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f65000079b0 con 0x7f6510101c00 2024-09-10T08:00:55.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.477+0000 7f64ff7fe640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f64f80a2bb0 0x7f64f80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:00:55.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.477+0000 7f64ff7fe640 1 -- 172.21.15.107:0/3476665451 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7f65000cdc30 con 0x7f6510101c00 2024-09-10T08:00:55.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.477+0000 7f650dd9b640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f64f80a2bb0 0x7f64f80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:00:55.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.478+0000 7f650dd9b640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f64f80a2bb0 0x7f64f80a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6510199450 tx=0x7f6504008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:00:55.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.481+0000 7f64ff7fe640 1 -- 172.21.15.107:0/3476665451 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6500014030 con 0x7f6510101c00 2024-09-10T08:00:55.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:00:55.661+0000 7f6514aa9640 1 -- 172.21.15.107:0/3476665451 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6510062630 con 0x7f64f80a2bb0 2024-09-10T08:00:55.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: purged_snaps scrub starts 2024-09-10T08:00:55.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: purged_snaps scrub ok 2024-09-10T08:00:55.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: osd.3 [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144] boot 2024-09-10T08:00:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T08:00:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: Detected new or changed devices on smithi002 2024-09-10T08:00:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:00:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:00:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi002 to 8547M 2024-09-10T08:00:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:00:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:00:55.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:00:55.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:55.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:55 smithi107 ceph-mon[35048]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T08:00:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: purged_snaps scrub starts 2024-09-10T08:00:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: purged_snaps scrub ok 2024-09-10T08:00:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: osd.3 [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144] boot 2024-09-10T08:00:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T08:00:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: Detected new or changed devices on smithi002 2024-09-10T08:00:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:00:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:00:56.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi002 to 8547M 2024-09-10T08:00:56.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:56.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:56.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:00:56.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:56.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:00:56.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:00:56.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:56.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:55 smithi002 ceph-mon[30563]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T08:00:56.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: purged_snaps scrub starts 2024-09-10T08:00:56.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: purged_snaps scrub ok 2024-09-10T08:00:56.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: osd.3 [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144] boot 2024-09-10T08:00:56.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T08:00:56.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:00:56.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: Detected new or changed devices on smithi002 2024-09-10T08:00:56.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:56.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:56.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:00:56.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:00:56.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi002 to 8547M 2024-09-10T08:00:56.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:56.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:56.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:00:56.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:00:56.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:00:56.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:00:56.158 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:00:56.158 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:55 smithi155 ceph-mon[35405]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T08:00:57.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:57 smithi107 ceph-mon[35048]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:57.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:57 smithi107 ceph-mon[35048]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:00:57.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:57 smithi107 ceph-mon[35048]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T08:00:57.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:57 smithi107 ceph-mon[35048]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T08:00:57.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:57 smithi002 ceph-mon[30563]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:57.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:57 smithi002 ceph-mon[30563]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:00:57.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:57 smithi002 ceph-mon[30563]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T08:00:57.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:57 smithi002 ceph-mon[30563]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T08:00:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:57 smithi155 ceph-mon[35405]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:00:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:57 smithi155 ceph-mon[35405]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:00:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:57 smithi155 ceph-mon[35405]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T08:00:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:57 smithi155 ceph-mon[35405]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T08:00:58.591 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:58 smithi107 ceph-mon[35048]: from='client.? 172.21.15.107:0/1777903249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "269f71aa-0f17-4260-ad01-452fdd1b997f"}]: dispatch 2024-09-10T08:00:58.591 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:58 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "269f71aa-0f17-4260-ad01-452fdd1b997f"}]: dispatch 2024-09-10T08:00:58.591 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:58 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "269f71aa-0f17-4260-ad01-452fdd1b997f"}]': finished 2024-09-10T08:00:58.591 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:58 smithi107 ceph-mon[35048]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T08:00:58.591 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:58 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:00:58.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:58 smithi002 ceph-mon[30563]: from='client.? 172.21.15.107:0/1777903249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "269f71aa-0f17-4260-ad01-452fdd1b997f"}]: dispatch 2024-09-10T08:00:58.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:58 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "269f71aa-0f17-4260-ad01-452fdd1b997f"}]: dispatch 2024-09-10T08:00:58.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:58 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "269f71aa-0f17-4260-ad01-452fdd1b997f"}]': finished 2024-09-10T08:00:58.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:58 smithi002 ceph-mon[30563]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T08:00:58.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:58 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:00:58.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:58 smithi155 ceph-mon[35405]: from='client.? 172.21.15.107:0/1777903249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "269f71aa-0f17-4260-ad01-452fdd1b997f"}]: dispatch 2024-09-10T08:00:58.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:58 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "269f71aa-0f17-4260-ad01-452fdd1b997f"}]: dispatch 2024-09-10T08:00:58.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:58 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "269f71aa-0f17-4260-ad01-452fdd1b997f"}]': finished 2024-09-10T08:00:58.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:58 smithi155 ceph-mon[35405]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T08:00:58.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:58 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:00:59.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:59 smithi002 ceph-mon[30563]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:00:59.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:00:59 smithi002 ceph-mon[30563]: from='client.? 172.21.15.107:0/2137064960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:00:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:59 smithi155 ceph-mon[35405]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:00:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:00:59 smithi155 ceph-mon[35405]: from='client.? 172.21.15.107:0/2137064960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:00:59.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:59 smithi107 ceph-mon[35048]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:00:59.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:00:59 smithi107 ceph-mon[35048]: from='client.? 172.21.15.107:0/2137064960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:01:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:01 smithi002 ceph-mon[30563]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:01.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:01 smithi155 ceph-mon[35405]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:01.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:01 smithi107 ceph-mon[35048]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:03.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:03 smithi002 ceph-mon[30563]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-10T08:01:03.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:03 smithi155 ceph-mon[35405]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-10T08:01:03.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:03 smithi107 ceph-mon[35048]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-10T08:01:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:05 smithi002 ceph-mon[30563]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-10T08:01:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:05 smithi107 ceph-mon[35048]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-10T08:01:05.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:05 smithi155 ceph-mon[35405]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-10T08:01:06.854 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:06 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T08:01:06.854 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:06 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:06.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:06 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T08:01:06.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:06 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:07.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:06 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T08:01:07.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:06 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:07.794 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:07 smithi107 ceph-mon[35048]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T08:01:07.794 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:07 smithi107 ceph-mon[35048]: Deploying daemon osd.4 on smithi107 2024-09-10T08:01:07.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:07 smithi155 ceph-mon[35405]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T08:01:07.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:07 smithi155 ceph-mon[35405]: Deploying daemon osd.4 on smithi107 2024-09-10T08:01:08.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:07 smithi002 ceph-mon[30563]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T08:01:08.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:07 smithi002 ceph-mon[30563]: Deploying daemon osd.4 on smithi107 2024-09-10T08:01:09.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:09 smithi107 ceph-mon[35048]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:01:09.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:09 smithi155 ceph-mon[35405]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:01:10.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:09 smithi002 ceph-mon[30563]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:01:11.723 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:11 smithi107 ceph-mon[35048]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T08:01:11.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:11 smithi155 ceph-mon[35405]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T08:01:12.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:11 smithi002 ceph-mon[30563]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T08:01:12.899 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:12 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:01:12.899 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:12 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:12.900 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:12 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:12.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:12 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:01:12.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:12 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:12.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:12 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:13.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:12 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:01:13.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:12 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:13.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:12 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:13.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:13 smithi155 ceph-mon[35405]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:01:13.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:13 smithi107 ceph-mon[35048]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:01:14.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:13 smithi002 ceph-mon[30563]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:01:14.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:14 smithi155 ceph-mon[35405]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:14.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:14 smithi107 ceph-mon[35048]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:15.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:14 smithi002 ceph-mon[30563]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:16.560 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:16 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:16.560 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:16 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:16.560 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:16 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:16.560 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:16 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:01:16.561 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:16 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:16.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:01:16.602+0000 7f64ff7fe640 1 -- 172.21.15.107:0/3476665451 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6510062630 con 0x7f64f80a2bb0 2024-09-10T08:01:16.602 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 4 on host 'smithi107' 2024-09-10T08:01:16.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:01:16.605+0000 7f6514aa9640 1 -- 172.21.15.107:0/3476665451 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f64f80a2bb0 msgr2=0x7f64f80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:01:16.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:01:16.605+0000 7f6514aa9640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f64f80a2bb0 0x7f64f80a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6510199450 tx=0x7f6504008040 comp rx=0 tx=0).stop 2024-09-10T08:01:16.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:01:16.605+0000 7f6514aa9640 1 -- 172.21.15.107:0/3476665451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6510101c00 msgr2=0x7f6510197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:01:16.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:01:16.605+0000 7f6514aa9640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6510101c00 0x7f6510197f30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f650000c970 tx=0x7f650000ce30 comp rx=0 tx=0).stop 2024-09-10T08:01:16.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:01:16.605+0000 7f6514aa9640 1 -- 172.21.15.107:0/3476665451 shutdown_connections 2024-09-10T08:01:16.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:01:16.605+0000 7f6514aa9640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f64f80a2bb0 0x7f64f80a5070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:16.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:01:16.605+0000 7f6514aa9640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65101037c0 0x7f651019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:16.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:01:16.605+0000 7f6514aa9640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510102e00 0x7f6510198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:16.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:01:16.605+0000 7f6514aa9640 1 --2- 172.21.15.107:0/3476665451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6510101c00 0x7f6510197f30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:16.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:01:16.605+0000 7f6514aa9640 1 -- 172.21.15.107:0/3476665451 >> 172.21.15.107:0/3476665451 conn(0x7f65100fd390 msgr2=0x7f65101061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:01:16.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:01:16.605+0000 7f6514aa9640 1 -- 172.21.15.107:0/3476665451 shutdown_connections 2024-09-10T08:01:16.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:01:16.605+0000 7f6514aa9640 1 -- 172.21.15.107:0/3476665451 wait complete. 2024-09-10T08:01:16.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:16 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:16.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:16 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:16.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:16 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:16.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:16 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:01:16.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:16 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:16 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:16.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:16 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:16.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:16 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:16.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:16 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:01:16.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:16 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:17.208 DEBUG:teuthology.orchestra.run.smithi107:osd.4> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T08:01:17.211 INFO:tasks.cephadm:Deploying osd.5 on smithi155 with /dev/vg_nvme/lv_3... 2024-09-10T08:01:17.211 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T08:01:17.467 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:01:17.558 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:17 smithi155 ceph-mon[35405]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:17.558 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:17 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:01:17.558 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:17 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:17.558 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:17 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:17.571 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:17 smithi107 ceph-mon[35048]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:17.571 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:17 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:01:17.572 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:17 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:17.572 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:17 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:17.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:17 smithi002 ceph-mon[30563]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:17.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:17 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:01:17.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:17 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:17.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:17 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:18.782 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:01:18 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4[45998]: 2024-09-10T08:01:18.422+0000 7ff49110c640 -1 osd.4 0 log_to_monitors true 2024-09-10T08:01:19.434 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:19 smithi155 ceph-mon[35405]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:19.434 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:19 smithi155 ceph-mon[35405]: from='osd.4 [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:01:19.434 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:19 smithi155 ceph-mon[35405]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:01:19.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:19 smithi107 ceph-mon[35048]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:19.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:19 smithi107 ceph-mon[35048]: from='osd.4 [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:01:19.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:19 smithi107 ceph-mon[35048]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:01:19.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:19 smithi002 ceph-mon[30563]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:19.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:19 smithi002 ceph-mon[30563]: from='osd.4 [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:01:19.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:19 smithi002 ceph-mon[30563]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:01:19.882 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T08:01:19.911 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_3 2024-09-10T08:01:20.161 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:01:20.417 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4[45998]: 2024-09-10T08:01:20.319+0000 7ff48d08f640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T08:01:20.417 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4[45998]: 2024-09-10T08:01:20.326+0000 7ff488eb7640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:01:20.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T08:01:20.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T08:01:20.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: from='osd.4 [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:01:20.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:20.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:01:20.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:20.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:20.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:01:20.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:01:20.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:20.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:20.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:01:20.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:20.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:20 smithi107 ceph-mon[35048]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T08:01:20.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T08:01:20.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T08:01:20.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: from='osd.4 [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:01:20.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:20.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:01:20.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:20.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:20.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:01:20.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:01:20.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:20.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:20.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:01:20.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:20.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:20 smithi002 ceph-mon[30563]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T08:01:20.532 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T08:01:20.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T08:01:20.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: from='osd.4 [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:01:20.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:20.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:01:20.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:20.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:20.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:01:20.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:01:20.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:20.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:20.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:01:20.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:20.534 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:20 smithi155 ceph-mon[35405]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T08:01:20.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.616+0000 7fccf52b3640 1 -- 172.21.15.155:0/756041740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf01026d0 msgr2=0x7fccf0108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:01:20.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.616+0000 7fccf52b3640 1 --2- 172.21.15.155:0/756041740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf01026d0 0x7fccf0108f60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009900 tx=0x7fcce002a410 comp rx=0 tx=0).stop 2024-09-10T08:01:20.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.616+0000 7fccf52b3640 1 -- 172.21.15.155:0/756041740 shutdown_connections 2024-09-10T08:01:20.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.616+0000 7fccf52b3640 1 --2- 172.21.15.155:0/756041740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf01026d0 0x7fccf0108f60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:20.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.616+0000 7fccf52b3640 1 --2- 172.21.15.155:0/756041740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0101d10 0x7fccf0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:20.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.616+0000 7fccf52b3640 1 --2- 172.21.15.155:0/756041740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf0100b10 0x7fccf0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:20.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.616+0000 7fccf52b3640 1 -- 172.21.15.155:0/756041740 >> 172.21.15.155:0/756041740 conn(0x7fccf00fc2a0 msgr2=0x7fccf00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:01:20.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.616+0000 7fccf52b3640 1 -- 172.21.15.155:0/756041740 shutdown_connections 2024-09-10T08:01:20.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.616+0000 7fccf52b3640 1 -- 172.21.15.155:0/756041740 wait complete. 2024-09-10T08:01:20.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.617+0000 7fccf52b3640 1 Processor -- start 2024-09-10T08:01:20.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.617+0000 7fccf52b3640 1 -- start start 2024-09-10T08:01:20.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.617+0000 7fccf52b3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0100b10 0x7fccf0197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:01:20.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.617+0000 7fccf52b3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf0101d10 0x7fccf0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:01:20.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.617+0000 7fccf52b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf01026d0 0x7fccf019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:01:20.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.617+0000 7fccf52b3640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fccf010d920 con 0x7fccf0100b10 2024-09-10T08:01:20.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.617+0000 7fccf52b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fccf010d7a0 con 0x7fccf01026d0 2024-09-10T08:01:20.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.617+0000 7fccf52b3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fccf010daa0 con 0x7fccf0101d10 2024-09-10T08:01:20.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.617+0000 7fccee7fc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf0101d10 0x7fccf0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:01:20.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.617+0000 7fccee7fc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf0101d10 0x7fccf0198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:55746/0 (socket says 172.21.15.155:55746) 2024-09-10T08:01:20.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.617+0000 7fccee7fc640 1 -- 172.21.15.155:0/2173171010 learned_addr learned my addr 172.21.15.155:0/2173171010 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-10T08:01:20.619 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.617+0000 7fccef7fe640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf01026d0 0x7fccf019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:01:20.620 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.618+0000 7fcceeffd640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0100b10 0x7fccf0197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:01:20.620 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.618+0000 7fccee7fc640 1 -- 172.21.15.155:0/2173171010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf01026d0 msgr2=0x7fccf019f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:01:20.620 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.618+0000 7fccee7fc640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf01026d0 0x7fccf019f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:20.620 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.618+0000 7fccee7fc640 1 -- 172.21.15.155:0/2173171010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0100b10 msgr2=0x7fccf0197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:01:20.620 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.618+0000 7fccee7fc640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0100b10 0x7fccf0197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:20.620 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.618+0000 7fccee7fc640 1 -- 172.21.15.155:0/2173171010 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf019fb70 con 0x7fccf0101d10 2024-09-10T08:01:20.620 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.618+0000 7fcceeffd640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0100b10 0x7fccf0197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:01:20.620 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.618+0000 7fccee7fc640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf0101d10 0x7fccf0198480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fccd800c890 tx=0x7fccd800cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:01:20.620 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.618+0000 7fcce7fff640 1 -- 172.21.15.155:0/2173171010 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8004220 con 0x7fccf0101d10 2024-09-10T08:01:20.620 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.618+0000 7fcce7fff640 1 -- 172.21.15.155:0/2173171010 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd80043c0 con 0x7fccf0101d10 2024-09-10T08:01:20.620 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.618+0000 7fccf52b3640 1 -- 172.21.15.155:0/2173171010 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf019fe60 con 0x7fccf0101d10 2024-09-10T08:01:20.621 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.618+0000 7fcce7fff640 1 -- 172.21.15.155:0/2173171010 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd80049e0 con 0x7fccf0101d10 2024-09-10T08:01:20.621 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.618+0000 7fccf52b3640 1 -- 172.21.15.155:0/2173171010 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf01a03e0 con 0x7fccf0101d10 2024-09-10T08:01:20.621 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.619+0000 7fccf52b3640 1 -- 172.21.15.155:0/2173171010 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf0107f10 con 0x7fccf0101d10 2024-09-10T08:01:20.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.620+0000 7fcce7fff640 1 -- 172.21.15.155:0/2173171010 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fccd80079b0 con 0x7fccf0101d10 2024-09-10T08:01:20.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.621+0000 7fcce7fff640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fccdc0a2c30 0x7fccdc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:01:20.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.621+0000 7fcce7fff640 1 -- 172.21.15.155:0/2173171010 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7fccd80cde50 con 0x7fccf0101d10 2024-09-10T08:01:20.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.621+0000 7fcceeffd640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fccdc0a2c30 0x7fccdc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:01:20.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.622+0000 7fcceeffd640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fccdc0a2c30 0x7fccdc0a50f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fccf0101b70 tx=0x7fcce80073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:01:20.624 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.625+0000 7fcce7fff640 1 -- 172.21.15.155:0/2173171010 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fccd8014030 con 0x7fccf0101d10 2024-09-10T08:01:20.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:20.798+0000 7fccf52b3640 1 -- 172.21.15.155:0/2173171010 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fccf0062630 con 0x7fccdc0a2c30 2024-09-10T08:01:21.511 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:21 smithi155 ceph-mon[35405]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:21.511 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:21 smithi155 ceph-mon[35405]: Detected new or changed devices on smithi107 2024-09-10T08:01:21.511 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:21 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi107 to 8543M 2024-09-10T08:01:21.511 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:21 smithi155 ceph-mon[35405]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T08:01:21.511 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:21.511 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:21.511 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:01:21.512 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:01:21.512 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:21.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:21 smithi107 ceph-mon[35048]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:21.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:21 smithi107 ceph-mon[35048]: Detected new or changed devices on smithi107 2024-09-10T08:01:21.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:21 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi107 to 8543M 2024-09-10T08:01:21.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:21 smithi107 ceph-mon[35048]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T08:01:21.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:21.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:21.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:01:21.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:01:21.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:21 smithi002 ceph-mon[30563]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:21 smithi002 ceph-mon[30563]: Detected new or changed devices on smithi107 2024-09-10T08:01:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:21 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi107 to 8543M 2024-09-10T08:01:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:21 smithi002 ceph-mon[30563]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T08:01:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:01:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:01:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:22.490 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:22 smithi155 ceph-mon[35405]: purged_snaps scrub starts 2024-09-10T08:01:22.490 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:22 smithi155 ceph-mon[35405]: purged_snaps scrub ok 2024-09-10T08:01:22.491 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:22 smithi155 ceph-mon[35405]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:01:22.491 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:22 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:22.491 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:22 smithi155 ceph-mon[35405]: osd.4 [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971] boot 2024-09-10T08:01:22.491 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:22 smithi155 ceph-mon[35405]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T08:01:22.491 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:22 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:22.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:22 smithi107 ceph-mon[35048]: purged_snaps scrub starts 2024-09-10T08:01:22.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:22 smithi107 ceph-mon[35048]: purged_snaps scrub ok 2024-09-10T08:01:22.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:22 smithi107 ceph-mon[35048]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:01:22.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:22 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:22.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:22 smithi107 ceph-mon[35048]: osd.4 [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971] boot 2024-09-10T08:01:22.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:22 smithi107 ceph-mon[35048]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T08:01:22.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:22 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:22.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:22 smithi002 ceph-mon[30563]: purged_snaps scrub starts 2024-09-10T08:01:22.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:22 smithi002 ceph-mon[30563]: purged_snaps scrub ok 2024-09-10T08:01:22.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:22 smithi002 ceph-mon[30563]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:01:22.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:22 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:22.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:22 smithi002 ceph-mon[30563]: osd.4 [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971] boot 2024-09-10T08:01:22.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:22 smithi002 ceph-mon[30563]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T08:01:22.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:22 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:01:23.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:23 smithi155 ceph-mon[35405]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:23.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:23 smithi155 ceph-mon[35405]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T08:01:23.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:23 smithi155 ceph-mon[35405]: from='client.? 172.21.15.155:0/1873081370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f0af26-b1a3-428b-a75f-f54956e40bf0"}]: dispatch 2024-09-10T08:01:23.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:23 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f0af26-b1a3-428b-a75f-f54956e40bf0"}]: dispatch 2024-09-10T08:01:23.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:23 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7f0af26-b1a3-428b-a75f-f54956e40bf0"}]': finished 2024-09-10T08:01:23.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:23 smithi155 ceph-mon[35405]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T08:01:23.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:23 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:23.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:23 smithi107 ceph-mon[35048]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:23.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:23 smithi107 ceph-mon[35048]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T08:01:23.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:23 smithi107 ceph-mon[35048]: from='client.? 172.21.15.155:0/1873081370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f0af26-b1a3-428b-a75f-f54956e40bf0"}]: dispatch 2024-09-10T08:01:23.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:23 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f0af26-b1a3-428b-a75f-f54956e40bf0"}]: dispatch 2024-09-10T08:01:23.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:23 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7f0af26-b1a3-428b-a75f-f54956e40bf0"}]': finished 2024-09-10T08:01:23.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:23 smithi107 ceph-mon[35048]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T08:01:23.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:23 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:23 smithi002 ceph-mon[30563]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-10T08:01:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:23 smithi002 ceph-mon[30563]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T08:01:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:23 smithi002 ceph-mon[30563]: from='client.? 172.21.15.155:0/1873081370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f0af26-b1a3-428b-a75f-f54956e40bf0"}]: dispatch 2024-09-10T08:01:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:23 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f0af26-b1a3-428b-a75f-f54956e40bf0"}]: dispatch 2024-09-10T08:01:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:23 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7f0af26-b1a3-428b-a75f-f54956e40bf0"}]': finished 2024-09-10T08:01:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:23 smithi002 ceph-mon[30563]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T08:01:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:23 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:24.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:24 smithi155 ceph-mon[35405]: from='client.? 172.21.15.155:0/3373802757' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:01:24.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:24 smithi107 ceph-mon[35048]: from='client.? 172.21.15.155:0/3373802757' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:01:24.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:24 smithi002 ceph-mon[30563]: from='client.? 172.21.15.155:0/3373802757' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:01:25.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:25 smithi155 ceph-mon[35405]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:25.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:25 smithi107 ceph-mon[35048]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:25 smithi002 ceph-mon[30563]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:27.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:27 smithi155 ceph-mon[35405]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:27.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:27 smithi107 ceph-mon[35048]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:27 smithi002 ceph-mon[30563]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:29.533 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:29 smithi155 ceph-mon[35405]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:29.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:29 smithi107 ceph-mon[35048]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:29.678 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:29 smithi002 ceph-mon[30563]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:31.586 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:31 smithi155 ceph-mon[35405]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:31.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:31 smithi107 ceph-mon[35048]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:31.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:31 smithi002 ceph-mon[30563]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:32.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:32 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T08:01:32.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:32 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:32.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:32 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T08:01:32.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:32 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:32.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:32 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T08:01:32.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:32 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:33.636 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:33 smithi155 ceph-mon[35405]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:33.637 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:33 smithi155 ceph-mon[35405]: Deploying daemon osd.5 on smithi155 2024-09-10T08:01:33.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:33 smithi107 ceph-mon[35048]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:33.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:33 smithi107 ceph-mon[35048]: Deploying daemon osd.5 on smithi155 2024-09-10T08:01:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:33 smithi002 ceph-mon[30563]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:33 smithi002 ceph-mon[30563]: Deploying daemon osd.5 on smithi155 2024-09-10T08:01:35.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:35 smithi155 ceph-mon[35405]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:35.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:35 smithi107 ceph-mon[35048]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:35.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:35 smithi002 ceph-mon[30563]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:37.618 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:37 smithi155 ceph-mon[35405]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:37.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:37 smithi107 ceph-mon[35048]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:37.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:37 smithi002 ceph-mon[30563]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:38.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:01:38.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:38.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:38.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:01:38.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:38.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:38.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:01:38.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:38.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:39.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:39 smithi155 ceph-mon[35405]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:39.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:39 smithi107 ceph-mon[35048]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:39.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:39 smithi002 ceph-mon[30563]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:41.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:41 smithi155 ceph-mon[35405]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:41.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:41 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:41.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:41 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:41.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:41 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:41.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:41 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:01:41.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:41 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:41.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:41 smithi107 ceph-mon[35048]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:41.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:41 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:41.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:41 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:41.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:41 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:41.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:41 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:01:41.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:41 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:41 smithi002 ceph-mon[30563]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:41 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:41 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:41 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:41 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:01:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:41 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:42.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:42.017+0000 7fcce7fff640 1 -- 172.21.15.155:0/2173171010 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fccf0062630 con 0x7fccdc0a2c30 2024-09-10T08:01:42.018 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 5 on host 'smithi155' 2024-09-10T08:01:42.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:42.020+0000 7fccf52b3640 1 -- 172.21.15.155:0/2173171010 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fccdc0a2c30 msgr2=0x7fccdc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:01:42.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:42.020+0000 7fccf52b3640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fccdc0a2c30 0x7fccdc0a50f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fccf0101b70 tx=0x7fcce80073c0 comp rx=0 tx=0).stop 2024-09-10T08:01:42.019 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:42.020+0000 7fccf52b3640 1 -- 172.21.15.155:0/2173171010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf0101d10 msgr2=0x7fccf0198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:01:42.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:42.020+0000 7fccf52b3640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf0101d10 0x7fccf0198480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fccd800c890 tx=0x7fccd800cd50 comp rx=0 tx=0).stop 2024-09-10T08:01:42.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:42.020+0000 7fccf52b3640 1 -- 172.21.15.155:0/2173171010 shutdown_connections 2024-09-10T08:01:42.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:42.020+0000 7fccf52b3640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf01026d0 0x7fccf019f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:42.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:42.020+0000 7fccf52b3640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fccdc0a2c30 0x7fccdc0a50f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:42.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:42.020+0000 7fccf52b3640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf0101d10 0x7fccf0198480 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:42.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:42.020+0000 7fccf52b3640 1 --2- 172.21.15.155:0/2173171010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0100b10 0x7fccf0197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:42.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:42.020+0000 7fccf52b3640 1 -- 172.21.15.155:0/2173171010 >> 172.21.15.155:0/2173171010 conn(0x7fccf00fc2a0 msgr2=0x7fccf01050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:01:42.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:42.020+0000 7fccf52b3640 1 -- 172.21.15.155:0/2173171010 shutdown_connections 2024-09-10T08:01:42.020 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:01:42.020+0000 7fccf52b3640 1 -- 172.21.15.155:0/2173171010 wait complete. 2024-09-10T08:01:42.385 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:01:42.385 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:42.385 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:42.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:01:42.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:42.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:42.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:01:42.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:42.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:42.839 DEBUG:teuthology.orchestra.run.smithi155:osd.5> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T08:01:42.844 INFO:tasks.cephadm:Deploying osd.6 on smithi002 with /dev/vg_nvme/lv_2... 2024-09-10T08:01:42.844 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T08:01:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:01:43.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:43 smithi155 ceph-mon[35405]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:43.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:43 smithi107 ceph-mon[35048]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:43.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:43 smithi002 ceph-mon[30563]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:44.106 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:01:43 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:01:43.778+0000 7fd6c22a3640 -1 osd.5 0 log_to_monitors true 2024-09-10T08:01:44.610 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:44 smithi002 ceph-mon[30563]: from='osd.5 [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:01:44.611 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:44 smithi002 ceph-mon[30563]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:01:44.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:44 smithi155 ceph-mon[35405]: from='osd.5 [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:01:44.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:44 smithi155 ceph-mon[35405]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:01:44.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:44 smithi107 ceph-mon[35048]: from='osd.5 [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:01:44.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:44 smithi107 ceph-mon[35048]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:01:45.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:45 smithi155 ceph-mon[35405]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:45.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:45 smithi155 ceph-mon[35405]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T08:01:45.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:45 smithi155 ceph-mon[35405]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T08:01:45.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:45 smithi155 ceph-mon[35405]: from='osd.5 [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:01:45.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:45.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:45 smithi155 ceph-mon[35405]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:01:45.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:45 smithi002 ceph-mon[30563]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:45.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:45 smithi002 ceph-mon[30563]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T08:01:45.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:45 smithi002 ceph-mon[30563]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T08:01:45.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:45 smithi002 ceph-mon[30563]: from='osd.5 [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:01:45.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:45.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:45 smithi002 ceph-mon[30563]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:01:45.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:45 smithi107 ceph-mon[35048]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:45.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:45 smithi107 ceph-mon[35048]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T08:01:45.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:45 smithi107 ceph-mon[35048]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T08:01:45.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:45 smithi107 ceph-mon[35048]: from='osd.5 [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:01:45.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:45 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:45.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:45 smithi107 ceph-mon[35048]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:01:46.156 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:01:45 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:01:45.833+0000 7fd6be226640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T08:01:46.156 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:01:45 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:01:45.839+0000 7fd6b6046640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:01:46.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-10T08:01:46.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T08:01:46.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:46.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:46.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: Detected new or changed devices on smithi155 2024-09-10T08:01:46.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:46.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:46.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:01:46.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:01:46.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:46.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:46.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:01:46.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:46.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:46 smithi002 ceph-mon[30563]: from='osd.5 ' entity='osd.5' 2024-09-10T08:01:46.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-10T08:01:46.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T08:01:46.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: Detected new or changed devices on smithi155 2024-09-10T08:01:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:01:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:01:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:01:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:46 smithi155 ceph-mon[35405]: from='osd.5 ' entity='osd.5' 2024-09-10T08:01:46.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-10T08:01:46.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T08:01:46.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: Detected new or changed devices on smithi155 2024-09-10T08:01:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:01:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:01:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:01:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:01:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:46 smithi107 ceph-mon[35048]: from='osd.5 ' entity='osd.5' 2024-09-10T08:01:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:01:46.697 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi002:vg_nvme/lv_2 2024-09-10T08:01:47.077 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:01:47.596 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:47 smithi002 ceph-mon[30563]: purged_snaps scrub starts 2024-09-10T08:01:47.596 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:47 smithi002 ceph-mon[30563]: purged_snaps scrub ok 2024-09-10T08:01:47.596 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:47 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi155 to 8544M 2024-09-10T08:01:47.596 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:47 smithi002 ceph-mon[30563]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:47.596 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:47 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:47.596 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:47 smithi002 ceph-mon[30563]: osd.5 [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362] boot 2024-09-10T08:01:47.596 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:47 smithi002 ceph-mon[30563]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T08:01:47.596 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:47 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:47.596 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:47 smithi002 ceph-mon[30563]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T08:01:47.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:47 smithi155 ceph-mon[35405]: purged_snaps scrub starts 2024-09-10T08:01:47.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:47 smithi155 ceph-mon[35405]: purged_snaps scrub ok 2024-09-10T08:01:47.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:47 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi155 to 8544M 2024-09-10T08:01:47.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:47 smithi155 ceph-mon[35405]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:47.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:47 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:47.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:47 smithi155 ceph-mon[35405]: osd.5 [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362] boot 2024-09-10T08:01:47.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:47 smithi155 ceph-mon[35405]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T08:01:47.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:47 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:47.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:47 smithi155 ceph-mon[35405]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T08:01:47.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:47 smithi107 ceph-mon[35048]: purged_snaps scrub starts 2024-09-10T08:01:47.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:47 smithi107 ceph-mon[35048]: purged_snaps scrub ok 2024-09-10T08:01:47.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:47 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi155 to 8544M 2024-09-10T08:01:47.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:47 smithi107 ceph-mon[35048]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:01:47.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:47 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:47.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:47 smithi107 ceph-mon[35048]: osd.5 [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362] boot 2024-09-10T08:01:47.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:47 smithi107 ceph-mon[35048]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T08:01:47.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:47 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:01:47.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:47 smithi107 ceph-mon[35048]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T08:01:47.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.705+0000 7fd16fc00640 1 -- 172.21.15.2:0/2922072769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd168103e00 msgr2=0x7fd168104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:01:47.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.705+0000 7fd16fc00640 1 --2- 172.21.15.2:0/2922072769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd168103e00 0x7fd168104200 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd158009900 tx=0x7fd15802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:01:47.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.705+0000 7fd16fc00640 1 -- 172.21.15.2:0/2922072769 shutdown_connections 2024-09-10T08:01:47.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.705+0000 7fd16fc00640 1 --2- 172.21.15.2:0/2922072769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd168105920 0x7fd16810c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:47.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.705+0000 7fd16fc00640 1 --2- 172.21.15.2:0/2922072769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd168104f60 0x7fd1681053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:47.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.705+0000 7fd16fc00640 1 --2- 172.21.15.2:0/2922072769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd168103e00 0x7fd168104200 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:47.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.705+0000 7fd16fc00640 1 -- 172.21.15.2:0/2922072769 >> 172.21.15.2:0/2922072769 conn(0x7fd1680ff5b0 msgr2=0x7fd1681019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:01:47.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.705+0000 7fd16fc00640 1 -- 172.21.15.2:0/2922072769 shutdown_connections 2024-09-10T08:01:47.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.705+0000 7fd16fc00640 1 -- 172.21.15.2:0/2922072769 wait complete. 2024-09-10T08:01:47.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.706+0000 7fd16fc00640 1 Processor -- start 2024-09-10T08:01:47.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.706+0000 7fd16fc00640 1 -- start start 2024-09-10T08:01:47.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.706+0000 7fd16fc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd168103e00 0x7fd16819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:01:47.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.706+0000 7fd16fc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd168104f60 0x7fd16819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:01:47.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.706+0000 7fd16fc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd168105920 0x7fd1681a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:01:47.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.706+0000 7fd16fc00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd16810fae0 con 0x7fd168104f60 2024-09-10T08:01:47.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.706+0000 7fd16fc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd16810f960 con 0x7fd168105920 2024-09-10T08:01:47.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.706+0000 7fd16fc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd16810fc60 con 0x7fd168103e00 2024-09-10T08:01:47.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd15fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd168104f60 0x7fd16819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:01:47.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd16d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd168103e00 0x7fd16819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:01:47.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd15fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd168104f60 0x7fd16819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52000/0 (socket says 172.21.15.2:52000) 2024-09-10T08:01:47.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd16d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd168103e00 0x7fd16819a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:46088/0 (socket says 172.21.15.2:46088) 2024-09-10T08:01:47.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd15fe00640 1 -- 172.21.15.2:0/113248248 learned_addr learned my addr 172.21.15.2:0/113248248 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:01:47.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd16dc00640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd168105920 0x7fd1681a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:01:47.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd15fe00640 1 -- 172.21.15.2:0/113248248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd168103e00 msgr2=0x7fd16819a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:01:47.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd15fe00640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd168103e00 0x7fd16819a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:47.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd15fe00640 1 -- 172.21.15.2:0/113248248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd168105920 msgr2=0x7fd1681a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:01:47.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd15fe00640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd168105920 0x7fd1681a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:01:47.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd15fe00640 1 -- 172.21.15.2:0/113248248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1681a1cd0 con 0x7fd168104f60 2024-09-10T08:01:47.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd16d200640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd168103e00 0x7fd16819a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:01:47.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd16dc00640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd168105920 0x7fd1681a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:01:47.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.707+0000 7fd15fe00640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd168104f60 0x7fd16819a5e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd15000e9c0 tx=0x7fd15000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:01:47.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.708+0000 7fd166000640 1 -- 172.21.15.2:0/113248248 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15000cd80 con 0x7fd168104f60 2024-09-10T08:01:47.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.708+0000 7fd166000640 1 -- 172.21.15.2:0/113248248 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1500035f0 con 0x7fd168104f60 2024-09-10T08:01:47.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.708+0000 7fd166000640 1 -- 172.21.15.2:0/113248248 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150010690 con 0x7fd168104f60 2024-09-10T08:01:47.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.708+0000 7fd16fc00640 1 -- 172.21.15.2:0/113248248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1681a1fc0 con 0x7fd168104f60 2024-09-10T08:01:47.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.708+0000 7fd16fc00640 1 -- 172.21.15.2:0/113248248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1681a24c0 con 0x7fd168104f60 2024-09-10T08:01:47.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.709+0000 7fd16fc00640 1 -- 172.21.15.2:0/113248248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd16810b1a0 con 0x7fd168104f60 2024-09-10T08:01:47.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.711+0000 7fd166000640 1 -- 172.21.15.2:0/113248248 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd1500037a0 con 0x7fd168104f60 2024-09-10T08:01:47.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.712+0000 7fd166000640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd14c0a2c30 0x7fd14c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:01:47.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.712+0000 7fd166000640 1 -- 172.21.15.2:0/113248248 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4562+0+0 (secure 0 0 0) 0x7fd1500ce540 con 0x7fd168104f60 2024-09-10T08:01:47.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.712+0000 7fd16d200640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd14c0a2c30 0x7fd14c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:01:47.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.713+0000 7fd16d200640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd14c0a2c30 0x7fd14c0a50f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd158031440 tx=0x7fd1580315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:01:47.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.717+0000 7fd166000640 1 -- 172.21.15.2:0/113248248 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd15009c2a0 con 0x7fd168104f60 2024-09-10T08:01:47.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:01:47.912+0000 7fd16fc00640 1 -- 172.21.15.2:0/113248248 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd168062630 con 0x7fd14c0a2c30 2024-09-10T08:01:48.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:48 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:01:48.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:48 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:01:48.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:48 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:48.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:48 smithi155 ceph-mon[35405]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T08:01:48.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:48 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:01:48.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:48 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:01:48.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:48 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:48.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:48 smithi107 ceph-mon[35048]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T08:01:48.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:48 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:01:48.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:48 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:01:48.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:48 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:01:48.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:48 smithi002 ceph-mon[30563]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T08:01:49.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:49 smithi107 ceph-mon[35048]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:01:49.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:49 smithi107 ceph-mon[35048]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:01:49.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:49 smithi107 ceph-mon[35048]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T08:01:49.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:49 smithi002 ceph-mon[30563]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:01:49.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:49 smithi002 ceph-mon[30563]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:01:49.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:49 smithi002 ceph-mon[30563]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T08:01:49.801 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:49 smithi155 ceph-mon[35405]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:01:49.801 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:49 smithi155 ceph-mon[35405]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:01:49.801 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:49 smithi155 ceph-mon[35405]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T08:01:51.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:51 smithi002 ceph-mon[30563]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:01:51.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:51 smithi002 ceph-mon[30563]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T08:01:51.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:51 smithi155 ceph-mon[35405]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:01:51.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:51 smithi155 ceph-mon[35405]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T08:01:51.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:51 smithi107 ceph-mon[35048]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:01:51.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:51 smithi107 ceph-mon[35048]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T08:01:52.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:52 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2789402769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0dc96e4-e62c-44ce-9902-7c168af07b9e"}]: dispatch 2024-09-10T08:01:52.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:52 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2789402769' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0dc96e4-e62c-44ce-9902-7c168af07b9e"}]': finished 2024-09-10T08:01:52.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:52 smithi002 ceph-mon[30563]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T08:01:52.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:52 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:01:52.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:52 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/4216563912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:01:52.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:52 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2789402769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0dc96e4-e62c-44ce-9902-7c168af07b9e"}]: dispatch 2024-09-10T08:01:52.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:52 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2789402769' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0dc96e4-e62c-44ce-9902-7c168af07b9e"}]': finished 2024-09-10T08:01:52.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:52 smithi155 ceph-mon[35405]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T08:01:52.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:52 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:01:52.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:52 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/4216563912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:01:52.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:52 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2789402769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0dc96e4-e62c-44ce-9902-7c168af07b9e"}]: dispatch 2024-09-10T08:01:52.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:52 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2789402769' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0dc96e4-e62c-44ce-9902-7c168af07b9e"}]': finished 2024-09-10T08:01:52.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:52 smithi107 ceph-mon[35048]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T08:01:52.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:52 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:01:52.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:52 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/4216563912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:01:53.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:53 smithi155 ceph-mon[35405]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T08:01:53.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:53 smithi107 ceph-mon[35048]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T08:01:53.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:53 smithi002 ceph-mon[30563]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T08:01:55.605 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:55 smithi002 ceph-mon[30563]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-10T08:01:55.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:55 smithi155 ceph-mon[35405]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-10T08:01:55.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:55 smithi107 ceph-mon[35048]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-10T08:01:57.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:57 smithi002 ceph-mon[30563]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-10T08:01:57.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:57 smithi155 ceph-mon[35405]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-10T08:01:57.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:57 smithi107 ceph-mon[35048]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-10T08:01:59.654 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:01:59 smithi002 ceph-mon[30563]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:01:59.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:01:59 smithi155 ceph-mon[35405]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:01:59.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:01:59 smithi107 ceph-mon[35048]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:02:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:01 smithi155 ceph-mon[35405]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T08:02:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T08:02:01.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:01.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:01 smithi155 ceph-mon[35405]: Deploying daemon osd.6 on smithi002 2024-09-10T08:02:01.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:01 smithi107 ceph-mon[35048]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T08:02:01.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T08:02:01.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:01.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:01 smithi107 ceph-mon[35048]: Deploying daemon osd.6 on smithi002 2024-09-10T08:02:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:01 smithi002 ceph-mon[30563]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T08:02:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T08:02:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:01 smithi002 ceph-mon[30563]: Deploying daemon osd.6 on smithi002 2024-09-10T08:02:03.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:03 smithi155 ceph-mon[35405]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:03.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:03 smithi107 ceph-mon[35048]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:03.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:03 smithi002 ceph-mon[30563]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:05.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:05 smithi002 ceph-mon[30563]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:05.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:05 smithi155 ceph-mon[35405]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:05.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:05 smithi107 ceph-mon[35048]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:07.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:07 smithi002 ceph-mon[30563]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:07.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:07 smithi155 ceph-mon[35405]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:07.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:07 smithi107 ceph-mon[35048]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:09.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:09 smithi155 ceph-mon[35405]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:09.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:02:09.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:09.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:09.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:09 smithi107 ceph-mon[35048]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:09.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:02:09.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:09.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:09 smithi002 ceph-mon[30563]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:02:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:11.437 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:11 smithi002 ceph-mon[30563]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:11.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:11 smithi155 ceph-mon[35405]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:11.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:11 smithi107 ceph-mon[35048]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:12.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:12 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:12.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:12 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:12.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:12 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:12.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:12 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:02:12.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:12 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:12.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:12 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:12.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:12 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:12.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:12 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:12.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:12 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:02:12.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:12 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:12.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:12 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:12.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:12 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:12.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:12 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:12.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:12 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:02:12.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:12 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:13.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:02:13.494+0000 7fd166000640 1 -- 172.21.15.2:0/113248248 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd168062630 con 0x7fd14c0a2c30 2024-09-10T08:02:13.494 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 6 on host 'smithi002' 2024-09-10T08:02:13.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:02:13.496+0000 7fd16fc00640 1 -- 172.21.15.2:0/113248248 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd14c0a2c30 msgr2=0x7fd14c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:02:13.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:02:13.496+0000 7fd16fc00640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd14c0a2c30 0x7fd14c0a50f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd158031440 tx=0x7fd1580315b0 comp rx=0 tx=0).stop 2024-09-10T08:02:13.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:02:13.496+0000 7fd16fc00640 1 -- 172.21.15.2:0/113248248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd168104f60 msgr2=0x7fd16819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:02:13.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:02:13.496+0000 7fd16fc00640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd168104f60 0x7fd16819a5e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd15000e9c0 tx=0x7fd15000ee80 comp rx=0 tx=0).stop 2024-09-10T08:02:13.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:02:13.497+0000 7fd16fc00640 1 -- 172.21.15.2:0/113248248 shutdown_connections 2024-09-10T08:02:13.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:02:13.497+0000 7fd16fc00640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd168105920 0x7fd1681a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:13.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:02:13.497+0000 7fd16fc00640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd168104f60 0x7fd16819a5e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:13.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:02:13.497+0000 7fd16fc00640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd14c0a2c30 0x7fd14c0a50f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:13.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:02:13.497+0000 7fd16fc00640 1 --2- 172.21.15.2:0/113248248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd168103e00 0x7fd16819a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:13.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:02:13.497+0000 7fd16fc00640 1 -- 172.21.15.2:0/113248248 >> 172.21.15.2:0/113248248 conn(0x7fd1680ff5b0 msgr2=0x7fd168108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:02:13.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:02:13.497+0000 7fd16fc00640 1 -- 172.21.15.2:0/113248248 shutdown_connections 2024-09-10T08:02:13.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:02:13.497+0000 7fd16fc00640 1 -- 172.21.15.2:0/113248248 wait complete. 2024-09-10T08:02:13.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:13 smithi155 ceph-mon[35405]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:13.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:13 smithi107 ceph-mon[35048]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:13.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:13 smithi002 ceph-mon[30563]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:14.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:14 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:02:14.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:14 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:14.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:14 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:14.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:14 smithi002 ceph-mon[30563]: from='osd.6 [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T08:02:14.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:14 smithi002 ceph-mon[30563]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T08:02:14.597 DEBUG:teuthology.orchestra.run.smithi002:osd.6> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T08:02:14.602 INFO:tasks.cephadm:Deploying osd.7 on smithi107 with /dev/vg_nvme/lv_2... 2024-09-10T08:02:14.602 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T08:02:14.630 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:14 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:02:14.630 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:14 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:14.630 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:14 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:14.630 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:14 smithi107 ceph-mon[35048]: from='osd.6 [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T08:02:14.630 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:14 smithi107 ceph-mon[35048]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T08:02:14.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:14 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:02:14.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:14 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:14.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:14 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:14.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:14 smithi155 ceph-mon[35405]: from='osd.6 [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T08:02:14.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:14 smithi155 ceph-mon[35405]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T08:02:14.855 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:02:15.539 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:15 smithi002 ceph-mon[30563]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:15.539 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:15 smithi002 ceph-mon[30563]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T08:02:15.539 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:15 smithi002 ceph-mon[30563]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T08:02:15.539 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:15 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:02:15.539 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:15 smithi002 ceph-mon[30563]: from='osd.6 [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:02:15.539 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:15 smithi002 ceph-mon[30563]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:02:15.540 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:02:15 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:02:15.529+0000 7f7039200640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T08:02:15.540 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:02:15 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:02:15.540+0000 7f7034200640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:02:15.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:15 smithi107 ceph-mon[35048]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:15.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:15 smithi107 ceph-mon[35048]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T08:02:15.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:15 smithi107 ceph-mon[35048]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T08:02:15.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:15 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:02:15.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:15 smithi107 ceph-mon[35048]: from='osd.6 [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:02:15.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:15 smithi107 ceph-mon[35048]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:02:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:15 smithi155 ceph-mon[35405]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:15 smithi155 ceph-mon[35405]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T08:02:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:15 smithi155 ceph-mon[35405]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T08:02:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:15 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:02:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:15 smithi155 ceph-mon[35405]: from='osd.6 [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:02:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:15 smithi155 ceph-mon[35405]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:02:16.664 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:16 smithi107 ceph-mon[35048]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-09-10T08:02:16.664 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:16 smithi107 ceph-mon[35048]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T08:02:16.664 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:16 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:02:16.664 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:16 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:02:16.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:16 smithi002 ceph-mon[30563]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-09-10T08:02:16.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:16 smithi002 ceph-mon[30563]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T08:02:16.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:16 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:02:16.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:16 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:02:16.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:16 smithi155 ceph-mon[35405]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-09-10T08:02:16.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:16 smithi155 ceph-mon[35405]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T08:02:16.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:16 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:02:16.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:16 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:02:17.357 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T08:02:17.386 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-09-10T08:02:17.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:17 smithi107 ceph-mon[35048]: purged_snaps scrub starts 2024-09-10T08:02:17.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:17 smithi107 ceph-mon[35048]: purged_snaps scrub ok 2024-09-10T08:02:17.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:17 smithi107 ceph-mon[35048]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:17.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:17 smithi107 ceph-mon[35048]: osd.6 [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508] boot 2024-09-10T08:02:17.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:17 smithi107 ceph-mon[35048]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T08:02:17.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:17 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:02:17.678 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:02:17.717 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:17 smithi002 ceph-mon[30563]: purged_snaps scrub starts 2024-09-10T08:02:17.717 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:17 smithi002 ceph-mon[30563]: purged_snaps scrub ok 2024-09-10T08:02:17.717 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:17 smithi002 ceph-mon[30563]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:17.717 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:17 smithi002 ceph-mon[30563]: osd.6 [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508] boot 2024-09-10T08:02:17.717 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:17 smithi002 ceph-mon[30563]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T08:02:17.717 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:17 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:02:17.807 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:17 smithi155 ceph-mon[35405]: purged_snaps scrub starts 2024-09-10T08:02:17.808 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:17 smithi155 ceph-mon[35405]: purged_snaps scrub ok 2024-09-10T08:02:17.808 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:17 smithi155 ceph-mon[35405]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:02:17.808 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:17 smithi155 ceph-mon[35405]: osd.6 [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508] boot 2024-09-10T08:02:17.808 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:17 smithi155 ceph-mon[35405]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T08:02:17.808 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:17 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:02:18.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.152+0000 7f326c70e640 1 -- 172.21.15.107:0/3543260087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3264107960 msgr2=0x7f3264109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:02:18.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.152+0000 7f326c70e640 1 --2- 172.21.15.107:0/3543260087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3264107960 0x7f3264109d50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f325c009ca0 tx=0x7f325c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:02:18.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.152+0000 7f326c70e640 1 -- 172.21.15.107:0/3543260087 shutdown_connections 2024-09-10T08:02:18.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.152+0000 7f326c70e640 1 --2- 172.21.15.107:0/3543260087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3264107960 0x7f3264109d50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:18.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.152+0000 7f326c70e640 1 --2- 172.21.15.107:0/3543260087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3264105030 0x7f3264107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:18.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.152+0000 7f326c70e640 1 --2- 172.21.15.107:0/3543260087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3264102700 0x7f3264104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:18.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.152+0000 7f326c70e640 1 -- 172.21.15.107:0/3543260087 >> 172.21.15.107:0/3543260087 conn(0x7f32640fc280 msgr2=0x7f32640fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:02:18.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.152+0000 7f326c70e640 1 -- 172.21.15.107:0/3543260087 shutdown_connections 2024-09-10T08:02:18.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.152+0000 7f326c70e640 1 -- 172.21.15.107:0/3543260087 wait complete. 2024-09-10T08:02:18.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326c70e640 1 Processor -- start 2024-09-10T08:02:18.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326c70e640 1 -- start start 2024-09-10T08:02:18.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326c70e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3264102700 0x7f3264102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:02:18.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326c70e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3264105030 0x7f32641006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:02:18.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326c70e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3264107960 0x7f3264100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:02:18.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326c70e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f326410d860 con 0x7f3264107960 2024-09-10T08:02:18.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326c70e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f326410d6e0 con 0x7f3264102700 2024-09-10T08:02:18.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326c70e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f326410d9e0 con 0x7f3264105030 2024-09-10T08:02:18.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326a4aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3264102700 0x7f3264102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:02:18.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326a4aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3264102700 0x7f3264102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47026/0 (socket says 172.21.15.107:47026) 2024-09-10T08:02:18.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326a4aa640 1 -- 172.21.15.107:0/792849268 learned_addr learned my addr 172.21.15.107:0/792849268 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T08:02:18.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326acab640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3264107960 0x7f3264100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:02:18.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f3269ca9640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3264105030 0x7f32641006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:02:18.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326a4aa640 1 -- 172.21.15.107:0/792849268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3264105030 msgr2=0x7f32641006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:02:18.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326a4aa640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3264105030 0x7f32641006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:18.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326a4aa640 1 -- 172.21.15.107:0/792849268 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3264107960 msgr2=0x7f3264100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:02:18.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326a4aa640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3264107960 0x7f3264100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:18.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f326a4aa640 1 -- 172.21.15.107:0/792849268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32641014b0 con 0x7f3264102700 2024-09-10T08:02:18.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.153+0000 7f3269ca9640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3264105030 0x7f32641006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:02:18.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.154+0000 7f326a4aa640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3264102700 0x7f3264102060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f326000c920 tx=0x7f326000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:02:18.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.154+0000 7f325b7fe640 1 -- 172.21.15.107:0/792849268 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260007c70 con 0x7f3264102700 2024-09-10T08:02:18.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.154+0000 7f325b7fe640 1 -- 172.21.15.107:0/792849268 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3260007e10 con 0x7f3264102700 2024-09-10T08:02:18.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.154+0000 7f326c70e640 1 -- 172.21.15.107:0/792849268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32641a1fc0 con 0x7f3264102700 2024-09-10T08:02:18.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.154+0000 7f325b7fe640 1 -- 172.21.15.107:0/792849268 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260018890 con 0x7f3264102700 2024-09-10T08:02:18.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.154+0000 7f326c70e640 1 -- 172.21.15.107:0/792849268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32641a23b0 con 0x7f3264102700 2024-09-10T08:02:18.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.155+0000 7f326c70e640 1 -- 172.21.15.107:0/792849268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32640fdd70 con 0x7f3264102700 2024-09-10T08:02:18.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.156+0000 7f325b7fe640 1 -- 172.21.15.107:0/792849268 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3260018a70 con 0x7f3264102700 2024-09-10T08:02:18.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.156+0000 7f325b7fe640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f32500a2b80 0x7f32500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:02:18.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.157+0000 7f325b7fe640 1 -- 172.21.15.107:0/792849268 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4965+0+0 (secure 0 0 0) 0x7f32600ce480 con 0x7f3264102700 2024-09-10T08:02:18.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.157+0000 7f3269ca9640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f32500a2b80 0x7f32500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:02:18.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.157+0000 7f3269ca9640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f32500a2b80 0x7f32500a5040 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3264101de0 tx=0x7f3254005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:02:18.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.165+0000 7f325b7fe640 1 -- 172.21.15.107:0/792849268 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3260014030 con 0x7f3264102700 2024-09-10T08:02:18.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:18.346+0000 7f326c70e640 1 -- 172.21.15.107:0/792849268 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3264062630 con 0x7f32500a2b80 2024-09-10T08:02:18.661 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:18 smithi107 ceph-mon[35048]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T08:02:18.661 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:18 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:02:18.661 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:18 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:02:18.661 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:18 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:18.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:18 smithi155 ceph-mon[35405]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T08:02:18.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:18 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:02:18.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:18 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:02:18.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:18 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:19.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:18 smithi002 ceph-mon[30563]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T08:02:19.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:18 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:02:19.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:18 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:02:19.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:18 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:19.780 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:19 smithi107 ceph-mon[35048]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:19.781 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:19 smithi107 ceph-mon[35048]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:02:19.781 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:19.781 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:19.781 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:19.781 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:19.781 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:19.781 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:19.781 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:19.781 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:02:19.781 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:19 smithi155 ceph-mon[35405]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:19 smithi155 ceph-mon[35405]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:02:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:19.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:19.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:19.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:02:19.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:19 smithi002 ceph-mon[30563]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:19 smithi002 ceph-mon[30563]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:02:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:02:20.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:20.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:20 smithi155 ceph-mon[35405]: Detected new or changed devices on smithi002 2024-09-10T08:02:20.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:20 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi002 to 5698M 2024-09-10T08:02:20.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:20 smithi107 ceph-mon[35048]: Detected new or changed devices on smithi002 2024-09-10T08:02:20.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:20 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi002 to 5698M 2024-09-10T08:02:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:20 smithi002 ceph-mon[30563]: Detected new or changed devices on smithi002 2024-09-10T08:02:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:20 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi002 to 5698M 2024-09-10T08:02:21.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:21 smithi107 ceph-mon[35048]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:21.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:21 smithi107 ceph-mon[35048]: from='client.? 172.21.15.107:0/3554266501' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8feffc44-c4f3-4e5d-a59f-8ab00b949179"}]: dispatch 2024-09-10T08:02:21.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:21 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8feffc44-c4f3-4e5d-a59f-8ab00b949179"}]: dispatch 2024-09-10T08:02:21.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:21 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8feffc44-c4f3-4e5d-a59f-8ab00b949179"}]': finished 2024-09-10T08:02:21.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:21 smithi107 ceph-mon[35048]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T08:02:21.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:21.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:21 smithi107 ceph-mon[35048]: from='client.? 172.21.15.107:0/3930899527' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:02:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:21 smithi155 ceph-mon[35405]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:21 smithi155 ceph-mon[35405]: from='client.? 172.21.15.107:0/3554266501' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8feffc44-c4f3-4e5d-a59f-8ab00b949179"}]: dispatch 2024-09-10T08:02:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:21 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8feffc44-c4f3-4e5d-a59f-8ab00b949179"}]: dispatch 2024-09-10T08:02:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:21 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8feffc44-c4f3-4e5d-a59f-8ab00b949179"}]': finished 2024-09-10T08:02:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:21 smithi155 ceph-mon[35405]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T08:02:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:21 smithi155 ceph-mon[35405]: from='client.? 172.21.15.107:0/3930899527' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:02:22.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:21 smithi002 ceph-mon[30563]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:22.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:21 smithi002 ceph-mon[30563]: from='client.? 172.21.15.107:0/3554266501' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8feffc44-c4f3-4e5d-a59f-8ab00b949179"}]: dispatch 2024-09-10T08:02:22.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:21 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8feffc44-c4f3-4e5d-a59f-8ab00b949179"}]: dispatch 2024-09-10T08:02:22.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:21 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8feffc44-c4f3-4e5d-a59f-8ab00b949179"}]': finished 2024-09-10T08:02:22.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:21 smithi002 ceph-mon[30563]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T08:02:22.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:22.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:21 smithi002 ceph-mon[30563]: from='client.? 172.21.15.107:0/3930899527' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:02:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:23 smithi155 ceph-mon[35405]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:23.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:23 smithi107 ceph-mon[35048]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:24.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:23 smithi002 ceph-mon[30563]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:25.660 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:25 smithi107 ceph-mon[35048]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:25 smithi155 ceph-mon[35405]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:26.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:25 smithi002 ceph-mon[30563]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:27.754 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:27 smithi107 ceph-mon[35048]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:27.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:27 smithi002 ceph-mon[30563]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:27.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:27 smithi155 ceph-mon[35405]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:29.756 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:29 smithi107 ceph-mon[35048]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:29.756 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T08:02:29.756 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:29 smithi155 ceph-mon[35405]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T08:02:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:30.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:29 smithi002 ceph-mon[30563]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:30.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T08:02:30.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:30.816 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:30 smithi107 ceph-mon[35048]: Deploying daemon osd.7 on smithi107 2024-09-10T08:02:30.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:30 smithi155 ceph-mon[35405]: Deploying daemon osd.7 on smithi107 2024-09-10T08:02:31.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:30 smithi002 ceph-mon[30563]: Deploying daemon osd.7 on smithi107 2024-09-10T08:02:31.874 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:31 smithi107 ceph-mon[35048]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:31.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:31 smithi155 ceph-mon[35405]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:32.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:31 smithi002 ceph-mon[30563]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:32.845 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:32 smithi107 ceph-mon[35048]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:32.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:32 smithi155 ceph-mon[35405]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:33.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:32 smithi002 ceph-mon[30563]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:34.820 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:34 smithi107 ceph-mon[35048]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:34.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:34 smithi155 ceph-mon[35405]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:35.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:34 smithi002 ceph-mon[30563]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:35.816 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:35 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:02:35.816 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:35 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:35.816 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:35 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:35 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:02:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:35 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:35 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:36.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:35 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:02:36.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:35 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:36.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:35 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:37.184 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:37 smithi107 ceph-mon[35048]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:37.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:37 smithi155 ceph-mon[35405]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:37.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:37 smithi002 ceph-mon[30563]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:38.832 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 7 on host 'smithi107' 2024-09-10T08:02:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:38.833+0000 7f325b7fe640 1 -- 172.21.15.107:0/792849268 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3264062630 con 0x7f32500a2b80 2024-09-10T08:02:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:38.835+0000 7f326c70e640 1 -- 172.21.15.107:0/792849268 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f32500a2b80 msgr2=0x7f32500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:02:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:38.835+0000 7f326c70e640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f32500a2b80 0x7f32500a5040 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3264101de0 tx=0x7f3254005e50 comp rx=0 tx=0).stop 2024-09-10T08:02:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:38.835+0000 7f326c70e640 1 -- 172.21.15.107:0/792849268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3264102700 msgr2=0x7f3264102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:02:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:38.835+0000 7f326c70e640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3264102700 0x7f3264102060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f326000c920 tx=0x7f326000cde0 comp rx=0 tx=0).stop 2024-09-10T08:02:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:38.835+0000 7f326c70e640 1 -- 172.21.15.107:0/792849268 shutdown_connections 2024-09-10T08:02:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:38.835+0000 7f326c70e640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3264107960 0x7f3264100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:38.835+0000 7f326c70e640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f32500a2b80 0x7f32500a5040 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:38.835+0000 7f326c70e640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3264105030 0x7f32641006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:38.835+0000 7f326c70e640 1 --2- 172.21.15.107:0/792849268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3264102700 0x7f3264102060 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:38.835+0000 7f326c70e640 1 -- 172.21.15.107:0/792849268 >> 172.21.15.107:0/792849268 conn(0x7f32640fc280 msgr2=0x7f3264102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:02:38.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:38.835+0000 7f326c70e640 1 -- 172.21.15.107:0/792849268 shutdown_connections 2024-09-10T08:02:38.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:02:38.835+0000 7f326c70e640 1 -- 172.21.15.107:0/792849268 wait complete. 2024-09-10T08:02:39.152 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:38 smithi107 ceph-mon[35048]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:39.152 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.152 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.152 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:39.152 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:02:39.152 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.152 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:02:39.152 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.152 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:38 smithi155 ceph-mon[35405]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:39.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:39.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:02:39.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:02:39.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:38 smithi002 ceph-mon[30563]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:02:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:02:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:39.488 DEBUG:teuthology.orchestra.run.smithi107:osd.7> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T08:02:39.491 INFO:tasks.cephadm:Deploying osd.8 on smithi155 with /dev/vg_nvme/lv_2... 2024-09-10T08:02:39.491 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T08:02:39.746 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:02:41.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:40 smithi155 ceph-mon[35405]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:41.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:40 smithi107 ceph-mon[35048]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:41.166 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:02:41 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:02:41.124+0000 7f05bc50e640 -1 osd.7 0 log_to_monitors true 2024-09-10T08:02:41.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:40 smithi002 ceph-mon[30563]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:42.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:41 smithi155 ceph-mon[35405]: from='osd.7 [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:02:42.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:41 smithi155 ceph-mon[35405]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:02:42.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:41 smithi107 ceph-mon[35048]: from='osd.7 [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:02:42.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:41 smithi107 ceph-mon[35048]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:02:42.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:41 smithi002 ceph-mon[30563]: from='osd.7 [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:02:42.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:41 smithi002 ceph-mon[30563]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:02:42.375 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T08:02:42.401 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_2 2024-09-10T08:02:42.670 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:02:42.951 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:42.951 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T08:02:42.951 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T08:02:42.951 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:42.951 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: from='osd.7 [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:02:42.952 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:02:42.952 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: Detected new or changed devices on smithi107 2024-09-10T08:02:42.952 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:42.952 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:42.952 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:42.952 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:42.952 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:42.952 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi107 to 5695M 2024-09-10T08:02:42.952 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:42.952 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:42.952 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:02:42.952 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:43.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.179+0000 7f04a4c0a640 1 -- 172.21.15.155:0/3790017797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a0103c20 msgr2=0x7f04a00ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:02:43.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.179+0000 7f04a4c0a640 1 --2- 172.21.15.155:0/3790017797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a0103c20 0x7f04a00ffcd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0494009900 tx=0x7f049402a410 comp rx=0 tx=0).stop 2024-09-10T08:02:43.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.179+0000 7f04a4c0a640 1 -- 172.21.15.155:0/3790017797 shutdown_connections 2024-09-10T08:02:43.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.179+0000 7f04a4c0a640 1 --2- 172.21.15.155:0/3790017797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a0100390 0x7f04a010a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:43.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.179+0000 7f04a4c0a640 1 --2- 172.21.15.155:0/3790017797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a0103c20 0x7f04a00ffcd0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:43.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.179+0000 7f04a4c0a640 1 --2- 172.21.15.155:0/3790017797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04a0103250 0x7f04a0103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:43.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.179+0000 7f04a4c0a640 1 -- 172.21.15.155:0/3790017797 >> 172.21.15.155:0/3790017797 conn(0x7f04a00fba80 msgr2=0x7f04a00fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:02:43.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.179+0000 7f04a4c0a640 1 -- 172.21.15.155:0/3790017797 shutdown_connections 2024-09-10T08:02:43.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.179+0000 7f04a4c0a640 1 -- 172.21.15.155:0/3790017797 wait complete. 2024-09-10T08:02:43.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.179+0000 7f04a4c0a640 1 Processor -- start 2024-09-10T08:02:43.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.179+0000 7f04a4c0a640 1 -- start start 2024-09-10T08:02:43.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f04a4c0a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a0100390 0x7f04a0102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:02:43.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f04a4c0a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04a0103250 0x7f04a0100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:02:43.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f04a4c0a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a0103c20 0x7f04a01014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:02:43.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f04a4c0a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f04a010d870 con 0x7f04a0103c20 2024-09-10T08:02:43.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f04a4c0a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04a010d6f0 con 0x7f04a0103250 2024-09-10T08:02:43.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f04a4c0a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04a010d9f0 con 0x7f04a0100390 2024-09-10T08:02:43.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f049e59c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a0100390 0x7f04a0102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:02:43.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f049e59c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a0100390 0x7f04a0102950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:46620/0 (socket says 172.21.15.155:46620) 2024-09-10T08:02:43.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f049e59c640 1 -- 172.21.15.155:0/773693423 learned_addr learned my addr 172.21.15.155:0/773693423 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-10T08:02:43.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f049dd9b640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04a0103250 0x7f04a0100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:02:43.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f049ed9d640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a0103c20 0x7f04a01014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:02:43.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f049e59c640 1 -- 172.21.15.155:0/773693423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04a0103250 msgr2=0x7f04a0100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:02:43.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f049e59c640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04a0103250 0x7f04a0100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:43.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f049e59c640 1 -- 172.21.15.155:0/773693423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a0103c20 msgr2=0x7f04a01014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:02:43.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f049e59c640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a0103c20 0x7f04a01014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:02:43.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f049e59c640 1 -- 172.21.15.155:0/773693423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a0101da0 con 0x7f04a0100390 2024-09-10T08:02:43.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f049e59c640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a0100390 0x7f04a0102950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f049000e9c0 tx=0x7f049000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:02:43.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f048f7fe640 1 -- 172.21.15.155:0/773693423 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049000cd80 con 0x7f04a0100390 2024-09-10T08:02:43.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f04a4c0a640 1 -- 172.21.15.155:0/773693423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a01a20c0 con 0x7f04a0100390 2024-09-10T08:02:43.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f048f7fe640 1 -- 172.21.15.155:0/773693423 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04900035f0 con 0x7f04a0100390 2024-09-10T08:02:43.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f048f7fe640 1 -- 172.21.15.155:0/773693423 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490010690 con 0x7f04a0100390 2024-09-10T08:02:43.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.180+0000 7f04a4c0a640 1 -- 172.21.15.155:0/773693423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a01a2580 con 0x7f04a0100390 2024-09-10T08:02:43.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.182+0000 7f04a4c0a640 1 -- 172.21.15.155:0/773693423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a0109240 con 0x7f04a0100390 2024-09-10T08:02:43.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.183+0000 7f048f7fe640 1 -- 172.21.15.155:0/773693423 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0490004030 con 0x7f04a0100390 2024-09-10T08:02:43.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.183+0000 7f048f7fe640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04880a2b80 0x7f04880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:02:43.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.184+0000 7f048f7fe640 1 -- 172.21.15.155:0/773693423 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5117+0+0 (secure 0 0 0) 0x7f04900ce810 con 0x7f04a0100390 2024-09-10T08:02:43.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.184+0000 7f049dd9b640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04880a2b80 0x7f04880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:02:43.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.184+0000 7f049dd9b640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04880a2b80 0x7f04880a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0494005ab0 tx=0x7f0494000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:02:43.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.188+0000 7f048f7fe640 1 -- 172.21.15.155:0/773693423 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f049009c3f0 con 0x7f04a0100390 2024-09-10T08:02:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T08:02:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T08:02:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: from='osd.7 [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:02:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:02:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: Detected new or changed devices on smithi107 2024-09-10T08:02:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi107 to 5695M 2024-09-10T08:02:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:02:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:43.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:43.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T08:02:43.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T08:02:43.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:43.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: from='osd.7 [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:02:43.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:02:43.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: Detected new or changed devices on smithi107 2024-09-10T08:02:43.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:43.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:43.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:43.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:43.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:02:43.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi107 to 5695M 2024-09-10T08:02:43.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:43.302 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:43.302 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:02:43.302 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:02:43.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:02:43.364+0000 7f04a4c0a640 1 -- 172.21.15.155:0/773693423 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f04a0062630 con 0x7f04880a2b80 2024-09-10T08:02:43.666 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:02:43 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:02:43.300+0000 7f05b8491640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T08:02:43.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:02:43 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:02:43.310+0000 7f05b02b1640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:02:43.936 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:43 smithi155 ceph-mon[35405]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T08:02:43.936 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:43 smithi155 ceph-mon[35405]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T08:02:43.937 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:43 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:43.937 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:43 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:43.937 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:43 smithi155 ceph-mon[35405]: from='osd.7 ' entity='osd.7' 2024-09-10T08:02:43.937 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:43 smithi155 ceph-mon[35405]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:02:43.937 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:43 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:02:43.937 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:43 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:02:43.937 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:43 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:44.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:43 smithi002 ceph-mon[30563]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T08:02:44.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:43 smithi002 ceph-mon[30563]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T08:02:44.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:43 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:44.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:43 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:44.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:43 smithi002 ceph-mon[30563]: from='osd.7 ' entity='osd.7' 2024-09-10T08:02:44.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:43 smithi002 ceph-mon[30563]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:02:44.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:43 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:02:44.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:43 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:02:44.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:43 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:44.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:43 smithi107 ceph-mon[35048]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T08:02:44.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:43 smithi107 ceph-mon[35048]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T08:02:44.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:43 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:44.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:43 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:44.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:43 smithi107 ceph-mon[35048]: from='osd.7 ' entity='osd.7' 2024-09-10T08:02:44.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:43 smithi107 ceph-mon[35048]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:02:44.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:43 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:02:44.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:43 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:02:44.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:43 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:45.047 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:44 smithi155 ceph-mon[35405]: purged_snaps scrub starts 2024-09-10T08:02:45.047 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:44 smithi155 ceph-mon[35405]: purged_snaps scrub ok 2024-09-10T08:02:45.047 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:44 smithi155 ceph-mon[35405]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:45.048 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:44 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:45.048 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:44 smithi155 ceph-mon[35405]: osd.7 [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994] boot 2024-09-10T08:02:45.048 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:44 smithi155 ceph-mon[35405]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T08:02:45.048 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:44 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:45.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:44 smithi002 ceph-mon[30563]: purged_snaps scrub starts 2024-09-10T08:02:45.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:44 smithi002 ceph-mon[30563]: purged_snaps scrub ok 2024-09-10T08:02:45.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:44 smithi002 ceph-mon[30563]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:45.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:44 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:45.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:44 smithi002 ceph-mon[30563]: osd.7 [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994] boot 2024-09-10T08:02:45.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:44 smithi002 ceph-mon[30563]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T08:02:45.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:44 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:45.319 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:44 smithi107 ceph-mon[35048]: purged_snaps scrub starts 2024-09-10T08:02:45.319 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:44 smithi107 ceph-mon[35048]: purged_snaps scrub ok 2024-09-10T08:02:45.319 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:44 smithi107 ceph-mon[35048]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:02:45.319 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:44 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:45.319 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:44 smithi107 ceph-mon[35048]: osd.7 [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994] boot 2024-09-10T08:02:45.319 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:44 smithi107 ceph-mon[35048]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T08:02:45.319 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:44 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:02:46.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:45 smithi002 ceph-mon[30563]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T08:02:46.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:45 smithi002 ceph-mon[30563]: from='client.? 172.21.15.155:0/3143932736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f9de660-f542-428d-bbdb-a932abc8d4cb"}]: dispatch 2024-09-10T08:02:46.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:45 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f9de660-f542-428d-bbdb-a932abc8d4cb"}]: dispatch 2024-09-10T08:02:46.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:45 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f9de660-f542-428d-bbdb-a932abc8d4cb"}]': finished 2024-09-10T08:02:46.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:45 smithi002 ceph-mon[30563]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T08:02:46.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:45 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:02:46.276 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:45 smithi155 ceph-mon[35405]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T08:02:46.276 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:45 smithi155 ceph-mon[35405]: from='client.? 172.21.15.155:0/3143932736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f9de660-f542-428d-bbdb-a932abc8d4cb"}]: dispatch 2024-09-10T08:02:46.276 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:45 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f9de660-f542-428d-bbdb-a932abc8d4cb"}]: dispatch 2024-09-10T08:02:46.276 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:45 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f9de660-f542-428d-bbdb-a932abc8d4cb"}]': finished 2024-09-10T08:02:46.276 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:45 smithi155 ceph-mon[35405]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T08:02:46.276 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:45 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:02:46.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:45 smithi107 ceph-mon[35048]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T08:02:46.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:45 smithi107 ceph-mon[35048]: from='client.? 172.21.15.155:0/3143932736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f9de660-f542-428d-bbdb-a932abc8d4cb"}]: dispatch 2024-09-10T08:02:46.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:45 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f9de660-f542-428d-bbdb-a932abc8d4cb"}]: dispatch 2024-09-10T08:02:46.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:45 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f9de660-f542-428d-bbdb-a932abc8d4cb"}]': finished 2024-09-10T08:02:46.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:45 smithi107 ceph-mon[35048]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T08:02:46.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:45 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:02:47.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:46 smithi002 ceph-mon[30563]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:47.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:46 smithi002 ceph-mon[30563]: from='client.? 172.21.15.155:0/3303300457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:02:47.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:46 smithi155 ceph-mon[35405]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:47.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:46 smithi155 ceph-mon[35405]: from='client.? 172.21.15.155:0/3303300457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:02:47.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:46 smithi107 ceph-mon[35048]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:47.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:46 smithi107 ceph-mon[35048]: from='client.? 172.21.15.155:0/3303300457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:02:48.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:47 smithi002 ceph-mon[30563]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T08:02:48.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:47 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:02:48.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:47 smithi155 ceph-mon[35405]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T08:02:48.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:47 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:02:48.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:47 smithi107 ceph-mon[35048]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T08:02:48.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:47 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:02:49.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:48 smithi002 ceph-mon[30563]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:49.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:48 smithi002 ceph-mon[30563]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T08:02:49.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:48 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:02:49.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:48 smithi155 ceph-mon[35405]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:49.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:48 smithi155 ceph-mon[35405]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T08:02:49.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:48 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:02:49.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:48 smithi107 ceph-mon[35048]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:49.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:48 smithi107 ceph-mon[35048]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T08:02:49.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:48 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:02:51.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:50 smithi002 ceph-mon[30563]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:51.349 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:50 smithi155 ceph-mon[35405]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:51.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:50 smithi107 ceph-mon[35048]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:53.107 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:52 smithi155 ceph-mon[35405]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:52 smithi002 ceph-mon[30563]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:53.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:52 smithi107 ceph-mon[35048]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:55.152 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:54 smithi155 ceph-mon[35405]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:55.152 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T08:02:55.152 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:55.152 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:54 smithi155 ceph-mon[35405]: Deploying daemon osd.8 on smithi155 2024-09-10T08:02:55.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:54 smithi002 ceph-mon[30563]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:55.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T08:02:55.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:55.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:54 smithi002 ceph-mon[30563]: Deploying daemon osd.8 on smithi155 2024-09-10T08:02:55.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:54 smithi107 ceph-mon[35048]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:55.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T08:02:55.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:02:55.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:54 smithi107 ceph-mon[35048]: Deploying daemon osd.8 on smithi155 2024-09-10T08:02:57.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:56 smithi002 ceph-mon[30563]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:57.382 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:56 smithi155 ceph-mon[35405]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:57.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:56 smithi107 ceph-mon[35048]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:59.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:02:58 smithi155 ceph-mon[35405]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:59.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:02:58 smithi002 ceph-mon[30563]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:02:59.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:02:58 smithi107 ceph-mon[35048]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:01.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:00 smithi002 ceph-mon[30563]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:01.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:00 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:01.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:00 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:01.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:00 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:01.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:00 smithi155 ceph-mon[35405]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:01.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:00 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:01.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:00 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:01.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:00 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:01.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:00 smithi107 ceph-mon[35048]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:01.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:00 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:01.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:00 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:01.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:00 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:03.242 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:02 smithi155 ceph-mon[35405]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:03.242 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:02 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:03.242 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:02 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:03.243 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:02 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:03.243 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:02 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:03.243 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:02 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:03.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:02 smithi002 ceph-mon[30563]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:03.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:02 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:03.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:02 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:03.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:02 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:03.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:02 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:03.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:02 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:03.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:02 smithi107 ceph-mon[35048]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:03.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:02 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:03.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:02 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:03.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:02 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:03.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:02 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:03.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:02 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:03.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:03:03.916+0000 7f048f7fe640 1 -- 172.21.15.155:0/773693423 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f04a0062630 con 0x7f04880a2b80 2024-09-10T08:03:03.916 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 8 on host 'smithi155' 2024-09-10T08:03:03.918 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:03:03.918+0000 7f04a4c0a640 1 -- 172.21.15.155:0/773693423 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04880a2b80 msgr2=0x7f04880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:03:03.918 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:03:03.918+0000 7f04a4c0a640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04880a2b80 0x7f04880a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0494005ab0 tx=0x7f0494000bc0 comp rx=0 tx=0).stop 2024-09-10T08:03:03.918 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:03:03.918+0000 7f04a4c0a640 1 -- 172.21.15.155:0/773693423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a0100390 msgr2=0x7f04a0102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:03:03.918 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:03:03.918+0000 7f04a4c0a640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a0100390 0x7f04a0102950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f049000e9c0 tx=0x7f049000ee80 comp rx=0 tx=0).stop 2024-09-10T08:03:03.918 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:03:03.918+0000 7f04a4c0a640 1 -- 172.21.15.155:0/773693423 shutdown_connections 2024-09-10T08:03:03.918 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:03:03.918+0000 7f04a4c0a640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a0103c20 0x7f04a01014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:03.919 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:03:03.918+0000 7f04a4c0a640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04a0103250 0x7f04a0100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:03.919 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:03:03.918+0000 7f04a4c0a640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04880a2b80 0x7f04880a5040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:03.919 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:03:03.918+0000 7f04a4c0a640 1 --2- 172.21.15.155:0/773693423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04a0100390 0x7f04a0102950 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:03.919 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:03:03.918+0000 7f04a4c0a640 1 -- 172.21.15.155:0/773693423 >> 172.21.15.155:0/773693423 conn(0x7f04a00fba80 msgr2=0x7f04a0106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:03:03.919 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:03:03.918+0000 7f04a4c0a640 1 -- 172.21.15.155:0/773693423 shutdown_connections 2024-09-10T08:03:03.919 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:03:03.919+0000 7f04a4c0a640 1 -- 172.21.15.155:0/773693423 wait complete. 2024-09-10T08:03:04.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:04.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:04.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:04.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:04.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:04.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:04.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:04.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:04.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:04.582 DEBUG:teuthology.orchestra.run.smithi155:osd.8> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T08:03:04.585 INFO:tasks.cephadm:Deploying osd.9 on smithi002 with /dev/vg_nvme/lv_1... 2024-09-10T08:03:04.585 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T08:03:04.967 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:03:05.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:04 smithi002 ceph-mon[30563]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:05.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:04 smithi155 ceph-mon[35405]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:05.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:04 smithi107 ceph-mon[35048]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:06.656 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:03:06 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:03:06.475+0000 7f2ece712640 -1 osd.8 0 log_to_monitors true 2024-09-10T08:03:07.125 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:06 smithi155 ceph-mon[35405]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:07.125 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:06 smithi155 ceph-mon[35405]: from='osd.8 [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:03:07.125 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:06 smithi155 ceph-mon[35405]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:03:07.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:06 smithi002 ceph-mon[30563]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:07.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:06 smithi002 ceph-mon[30563]: from='osd.8 [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:03:07.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:06 smithi002 ceph-mon[30563]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:03:07.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:06 smithi107 ceph-mon[35048]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:07.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:06 smithi107 ceph-mon[35048]: from='osd.8 [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:03:07.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:06 smithi107 ceph-mon[35048]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:03:08.167 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T08:03:08.167 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T08:03:08.167 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:08.167 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: from='osd.8 [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:03:08.167 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:03:08.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: Detected new or changed devices on smithi155 2024-09-10T08:03:08.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:08.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:08.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:08.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:08.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:08.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi155 to 5696M 2024-09-10T08:03:08.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:08.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:08.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:08.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:08.385 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T08:03:08.385 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T08:03:08.385 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:08.385 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: from='osd.8 [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:03:08.385 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:03:08.386 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: Detected new or changed devices on smithi155 2024-09-10T08:03:08.386 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:08.386 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:08.386 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:08.386 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:08.386 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:08.386 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi155 to 5696M 2024-09-10T08:03:08.386 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:08.386 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:08.386 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:08.386 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:08.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T08:03:08.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T08:03:08.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:08.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: from='osd.8 [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:03:08.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:03:08.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: Detected new or changed devices on smithi155 2024-09-10T08:03:08.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:08.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:08.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:08.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:08.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:08.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi155 to 5696M 2024-09-10T08:03:08.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:08.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:08.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:08.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:03:08.464 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi002:vg_nvme/lv_1 2024-09-10T08:03:08.656 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:03:08.385+0000 7f2eca695640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T08:03:08.656 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:03:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:03:08.394+0000 7f2ec24b5640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:03:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:03:09.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:09 smithi002 ceph-mon[30563]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:09.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:09 smithi002 ceph-mon[30563]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-10T08:03:09.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:09 smithi002 ceph-mon[30563]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T08:03:09.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:09.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:09 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:09.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:09 smithi002 ceph-mon[30563]: from='osd.8 ' entity='osd.8' 2024-09-10T08:03:09.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:09 smithi155 ceph-mon[35405]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:09.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:09 smithi155 ceph-mon[35405]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-10T08:03:09.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:09 smithi155 ceph-mon[35405]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T08:03:09.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:09.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:09 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:09.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:09 smithi155 ceph-mon[35405]: from='osd.8 ' entity='osd.8' 2024-09-10T08:03:09.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:09 smithi107 ceph-mon[35048]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:09.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:09 smithi107 ceph-mon[35048]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-10T08:03:09.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:09 smithi107 ceph-mon[35048]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T08:03:09.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:09.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:09 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:09.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:09 smithi107 ceph-mon[35048]: from='osd.8 ' entity='osd.8' 2024-09-10T08:03:09.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.434+0000 7fe446a00640 1 -- 172.21.15.2:0/4264991907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe440078790 msgr2=0x7fe440079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:03:09.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.434+0000 7fe446a00640 1 --2- 172.21.15.2:0/4264991907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe440078790 0x7fe440079030 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe438009ca0 tx=0x7fe43802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:03:09.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.434+0000 7fe446a00640 1 -- 172.21.15.2:0/4264991907 shutdown_connections 2024-09-10T08:03:09.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.434+0000 7fe446a00640 1 --2- 172.21.15.2:0/4264991907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe440078790 0x7fe440079030 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.434+0000 7fe446a00640 1 --2- 172.21.15.2:0/4264991907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe440077df0 0x7fe440078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.434+0000 7fe446a00640 1 --2- 172.21.15.2:0/4264991907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe440076ba0 0x7fe440076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.434+0000 7fe446a00640 1 -- 172.21.15.2:0/4264991907 >> 172.21.15.2:0/4264991907 conn(0x7fe4400ff5b0 msgr2=0x7fe4401019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:03:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.434+0000 7fe446a00640 1 -- 172.21.15.2:0/4264991907 shutdown_connections 2024-09-10T08:03:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.434+0000 7fe446a00640 1 -- 172.21.15.2:0/4264991907 wait complete. 2024-09-10T08:03:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.435+0000 7fe446a00640 1 Processor -- start 2024-09-10T08:03:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.435+0000 7fe446a00640 1 -- start start 2024-09-10T08:03:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.436+0000 7fe446a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe440076ba0 0x7fe44019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:03:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.436+0000 7fe446a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe440077df0 0x7fe44019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:03:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.436+0000 7fe446a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe440078790 0x7fe4401a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:03:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.436+0000 7fe446a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe440113ec0 con 0x7fe440076ba0 2024-09-10T08:03:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.436+0000 7fe446a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe440113d40 con 0x7fe440077df0 2024-09-10T08:03:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.436+0000 7fe446a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe440114040 con 0x7fe440078790 2024-09-10T08:03:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.436+0000 7fe43fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe440076ba0 0x7fe44019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:03:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.436+0000 7fe43fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe440076ba0 0x7fe44019e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47580/0 (socket says 172.21.15.2:47580) 2024-09-10T08:03:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.436+0000 7fe43fe00640 1 -- 172.21.15.2:0/2747378132 learned_addr learned my addr 172.21.15.2:0/2747378132 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:03:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.436+0000 7fe43f400640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe440077df0 0x7fe44019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:03:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.436+0000 7fe444a00640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe440078790 0x7fe4401a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:03:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.437+0000 7fe43fe00640 1 -- 172.21.15.2:0/2747378132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe440078790 msgr2=0x7fe4401a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:03:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.437+0000 7fe43fe00640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe440078790 0x7fe4401a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.437+0000 7fe43fe00640 1 -- 172.21.15.2:0/2747378132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe440077df0 msgr2=0x7fe44019e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:03:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.437+0000 7fe43fe00640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe440077df0 0x7fe44019e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.437+0000 7fe43fe00640 1 -- 172.21.15.2:0/2747378132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4401a60e0 con 0x7fe440076ba0 2024-09-10T08:03:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.437+0000 7fe43fe00640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe440076ba0 0x7fe44019e4b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe43000e9f0 tx=0x7fe43000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:03:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.437+0000 7fe43cc00640 1 -- 172.21.15.2:0/2747378132 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43000cd80 con 0x7fe440076ba0 2024-09-10T08:03:09.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.437+0000 7fe43cc00640 1 -- 172.21.15.2:0/2747378132 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe430005500 con 0x7fe440076ba0 2024-09-10T08:03:09.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.437+0000 7fe446a00640 1 -- 172.21.15.2:0/2747378132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4401a63d0 con 0x7fe440076ba0 2024-09-10T08:03:09.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.437+0000 7fe43cc00640 1 -- 172.21.15.2:0/2747378132 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430010690 con 0x7fe440076ba0 2024-09-10T08:03:09.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.437+0000 7fe446a00640 1 -- 172.21.15.2:0/2747378132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4401a6910 con 0x7fe440076ba0 2024-09-10T08:03:09.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.438+0000 7fe446a00640 1 -- 172.21.15.2:0/2747378132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe440110780 con 0x7fe440076ba0 2024-09-10T08:03:09.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.440+0000 7fe43cc00640 1 -- 172.21.15.2:0/2747378132 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe4300046c0 con 0x7fe440076ba0 2024-09-10T08:03:09.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.442+0000 7fe43cc00640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4240a2c30 0x7fe4240a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:03:09.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.442+0000 7fe43cc00640 1 -- 172.21.15.2:0/2747378132 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5862+0+0 (secure 0 0 0) 0x7fe4300ceb60 con 0x7fe440076ba0 2024-09-10T08:03:09.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.442+0000 7fe43f400640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4240a2c30 0x7fe4240a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:03:09.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.442+0000 7fe43f400640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4240a2c30 0x7fe4240a50f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe44019f9d0 tx=0x7fe42c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:03:09.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.446+0000 7fe43cc00640 1 -- 172.21.15.2:0/2747378132 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe43009c3b0 con 0x7fe440076ba0 2024-09-10T08:03:09.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:09.640+0000 7fe446a00640 1 -- 172.21.15.2:0/2747378132 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe440062630 con 0x7fe4240a2c30 2024-09-10T08:03:10.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:10 smithi002 ceph-mon[30563]: purged_snaps scrub starts 2024-09-10T08:03:10.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:10 smithi002 ceph-mon[30563]: purged_snaps scrub ok 2024-09-10T08:03:10.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:10 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:10.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:10 smithi002 ceph-mon[30563]: osd.8 [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864] boot 2024-09-10T08:03:10.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:10 smithi002 ceph-mon[30563]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T08:03:10.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:10 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:10.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:10 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:03:10.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:10 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:03:10.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:10 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:10.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:10 smithi155 ceph-mon[35405]: purged_snaps scrub starts 2024-09-10T08:03:10.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:10 smithi155 ceph-mon[35405]: purged_snaps scrub ok 2024-09-10T08:03:10.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:10 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:10.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:10 smithi155 ceph-mon[35405]: osd.8 [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864] boot 2024-09-10T08:03:10.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:10 smithi155 ceph-mon[35405]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T08:03:10.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:10 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:10.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:10 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:03:10.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:10 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:03:10.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:10 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:10.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:10 smithi107 ceph-mon[35048]: purged_snaps scrub starts 2024-09-10T08:03:10.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:10 smithi107 ceph-mon[35048]: purged_snaps scrub ok 2024-09-10T08:03:10.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:10 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:10.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:10 smithi107 ceph-mon[35048]: osd.8 [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864] boot 2024-09-10T08:03:10.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:10 smithi107 ceph-mon[35048]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T08:03:10.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:10 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:03:10.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:10 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:03:10.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:10 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:03:10.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:10 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:11.291 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:11 smithi002 ceph-mon[30563]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:11.291 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:11 smithi002 ceph-mon[30563]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:03:11.291 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:11 smithi002 ceph-mon[30563]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T08:03:11.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:11 smithi155 ceph-mon[35405]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:11.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:11 smithi155 ceph-mon[35405]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:03:11.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:11 smithi155 ceph-mon[35405]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T08:03:11.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:11 smithi107 ceph-mon[35048]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:03:11.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:11 smithi107 ceph-mon[35048]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:03:11.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:11 smithi107 ceph-mon[35048]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T08:03:12.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:12 smithi002 ceph-mon[30563]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T08:03:12.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:12 smithi155 ceph-mon[35405]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T08:03:12.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:12 smithi107 ceph-mon[35048]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T08:03:13.328 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:13 smithi002 ceph-mon[30563]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:13.328 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:13 smithi002 ceph-mon[30563]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T08:03:13.328 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:13 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3540478391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07f3cee5-642d-4887-86ed-46293be9fe24"}]: dispatch 2024-09-10T08:03:13.329 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:13 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3540478391' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07f3cee5-642d-4887-86ed-46293be9fe24"}]': finished 2024-09-10T08:03:13.329 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:13 smithi002 ceph-mon[30563]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T08:03:13.329 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:13 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:13.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:13 smithi155 ceph-mon[35405]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:13.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:13 smithi155 ceph-mon[35405]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T08:03:13.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:13 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3540478391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07f3cee5-642d-4887-86ed-46293be9fe24"}]: dispatch 2024-09-10T08:03:13.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:13 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3540478391' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07f3cee5-642d-4887-86ed-46293be9fe24"}]': finished 2024-09-10T08:03:13.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:13 smithi155 ceph-mon[35405]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T08:03:13.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:13 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:13.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:13 smithi107 ceph-mon[35048]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:13.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:13 smithi107 ceph-mon[35048]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T08:03:13.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:13 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3540478391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07f3cee5-642d-4887-86ed-46293be9fe24"}]: dispatch 2024-09-10T08:03:13.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:13 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3540478391' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07f3cee5-642d-4887-86ed-46293be9fe24"}]': finished 2024-09-10T08:03:13.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:13 smithi107 ceph-mon[35048]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T08:03:13.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:13 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:14.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:14 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2354539077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:03:14.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:14 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2354539077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:03:14.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:14 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2354539077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:03:15.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:15 smithi107 ceph-mon[35048]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:15.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:15 smithi155 ceph-mon[35405]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:15.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:15 smithi002 ceph-mon[30563]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:17.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:17 smithi002 ceph-mon[30563]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-10T08:03:17.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:17 smithi155 ceph-mon[35405]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-10T08:03:17.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:17 smithi107 ceph-mon[35048]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-10T08:03:19.314 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:19 smithi002 ceph-mon[30563]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T08:03:19.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:19 smithi155 ceph-mon[35405]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T08:03:19.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:19 smithi107 ceph-mon[35048]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T08:03:21.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:21 smithi155 ceph-mon[35405]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:03:21.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:21 smithi107 ceph-mon[35048]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:03:21.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:21 smithi002 ceph-mon[30563]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:03:22.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:22 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T08:03:22.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:22 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:22.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:22 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T08:03:22.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:22 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:22.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:22 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T08:03:22.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:22 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:23.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:23 smithi002 ceph-mon[30563]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T08:03:23.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:23 smithi002 ceph-mon[30563]: Deploying daemon osd.9 on smithi002 2024-09-10T08:03:23.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:23 smithi155 ceph-mon[35405]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T08:03:23.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:23 smithi155 ceph-mon[35405]: Deploying daemon osd.9 on smithi002 2024-09-10T08:03:23.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:23 smithi107 ceph-mon[35048]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T08:03:23.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:23 smithi107 ceph-mon[35048]: Deploying daemon osd.9 on smithi002 2024-09-10T08:03:25.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:25 smithi002 ceph-mon[30563]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T08:03:25.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:25 smithi155 ceph-mon[35405]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T08:03:25.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:25 smithi107 ceph-mon[35048]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T08:03:27.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:27 smithi002 ceph-mon[30563]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:03:27.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:27 smithi155 ceph-mon[35405]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:03:27.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:27 smithi107 ceph-mon[35048]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:03:29.156 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:29 smithi002 ceph-mon[30563]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:29.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:29 smithi155 ceph-mon[35405]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:29.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:29 smithi107 ceph-mon[35048]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:30.372 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:30.373 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:30.373 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:30.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:30.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:30.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:30.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:30.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:30.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:31.353 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:31 smithi002 ceph-mon[30563]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:31.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:31 smithi155 ceph-mon[35405]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:31.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:31 smithi107 ceph-mon[35048]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:33.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:33 smithi155 ceph-mon[35405]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:33.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:33 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:33.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:33 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:33.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:33 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:33.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:33 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:33.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:33 smithi107 ceph-mon[35048]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:33.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:33 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:33.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:33 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:33.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:33 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:33.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:33 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:33.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:33 smithi002 ceph-mon[30563]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:33.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:33 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:33.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:33 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:33.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:33 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:33.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:33 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:34.243 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 9 on host 'smithi002' 2024-09-10T08:03:34.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:34.243+0000 7fe43cc00640 1 -- 172.21.15.2:0/2747378132 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe440062630 con 0x7fe4240a2c30 2024-09-10T08:03:34.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:34.246+0000 7fe446a00640 1 -- 172.21.15.2:0/2747378132 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4240a2c30 msgr2=0x7fe4240a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:03:34.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:34.246+0000 7fe446a00640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4240a2c30 0x7fe4240a50f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe44019f9d0 tx=0x7fe42c005f90 comp rx=0 tx=0).stop 2024-09-10T08:03:34.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:34.246+0000 7fe446a00640 1 -- 172.21.15.2:0/2747378132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe440076ba0 msgr2=0x7fe44019e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:03:34.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:34.246+0000 7fe446a00640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe440076ba0 0x7fe44019e4b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe43000e9f0 tx=0x7fe43000edf0 comp rx=0 tx=0).stop 2024-09-10T08:03:34.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:34.246+0000 7fe446a00640 1 -- 172.21.15.2:0/2747378132 shutdown_connections 2024-09-10T08:03:34.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:34.246+0000 7fe446a00640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4240a2c30 0x7fe4240a50f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:34.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:34.246+0000 7fe446a00640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe440078790 0x7fe4401a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:34.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:34.246+0000 7fe446a00640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe440077df0 0x7fe44019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:34.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:34.246+0000 7fe446a00640 1 --2- 172.21.15.2:0/2747378132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe440076ba0 0x7fe44019e4b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:34.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:34.246+0000 7fe446a00640 1 -- 172.21.15.2:0/2747378132 >> 172.21.15.2:0/2747378132 conn(0x7fe4400ff5b0 msgr2=0x7fe440101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:03:34.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:34.246+0000 7fe446a00640 1 -- 172.21.15.2:0/2747378132 shutdown_connections 2024-09-10T08:03:34.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:03:34.247+0000 7fe446a00640 1 -- 172.21.15.2:0/2747378132 wait complete. 2024-09-10T08:03:34.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:34.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:34.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:35.105 DEBUG:teuthology.orchestra.run.smithi002:osd.9> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T08:03:35.109 INFO:tasks.cephadm:Deploying osd.10 on smithi107 with /dev/vg_nvme/lv_1... 2024-09-10T08:03:35.110 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T08:03:35.313 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:35 smithi002 ceph-mon[30563]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:35.314 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:35 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:35.314 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:35 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:35.314 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:35 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:35.360 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:03:35.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:35 smithi155 ceph-mon[35405]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:35.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:35 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:35.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:35 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:35.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:35 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:35.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:35 smithi107 ceph-mon[35048]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:35.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:35 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:35.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:35 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:35.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:35 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:35.659 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:03:35 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:03:35.313+0000 7ff7bf22b640 -1 osd.9 0 log_to_monitors true 2024-09-10T08:03:36.370 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:36 smithi002 ceph-mon[30563]: from='osd.9 [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T08:03:36.370 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:36 smithi002 ceph-mon[30563]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T08:03:36.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:36 smithi155 ceph-mon[35405]: from='osd.9 [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T08:03:36.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:36 smithi155 ceph-mon[35405]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T08:03:36.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:36 smithi107 ceph-mon[35048]: from='osd.9 [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T08:03:36.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:36 smithi107 ceph-mon[35048]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T08:03:37.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:37 smithi107 ceph-mon[35048]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:37.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:37 smithi107 ceph-mon[35048]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T08:03:37.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:37 smithi107 ceph-mon[35048]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T08:03:37.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:37 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:37.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:37 smithi107 ceph-mon[35048]: from='osd.9 [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:03:37.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:37 smithi107 ceph-mon[35048]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:03:37.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:37 smithi002 ceph-mon[30563]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:37.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:37 smithi002 ceph-mon[30563]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T08:03:37.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:37 smithi002 ceph-mon[30563]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T08:03:37.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:37 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:37.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:37 smithi002 ceph-mon[30563]: from='osd.9 [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:03:37.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:37 smithi002 ceph-mon[30563]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:03:37.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:37 smithi155 ceph-mon[35405]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:37.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:37 smithi155 ceph-mon[35405]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T08:03:37.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:37 smithi155 ceph-mon[35405]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T08:03:37.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:37 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:37.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:37 smithi155 ceph-mon[35405]: from='osd.9 [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:03:37.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:37 smithi155 ceph-mon[35405]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-09-10T08:03:38.020 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:03:37 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:03:37.720+0000 7ff7b9e00640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T08:03:38.020 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:03:37 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:03:37.753+0000 7ff7afe00640 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:03:38.129 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T08:03:38.155 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2024-09-10T08:03:38.394 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:38 smithi002 ceph-mon[30563]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-09-10T08:03:38.394 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:38 smithi002 ceph-mon[30563]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T08:03:38.394 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:38.394 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:38.394 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:38 smithi002 ceph-mon[30563]: from='osd.9 ' entity='osd.9' 2024-09-10T08:03:38.400 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:03:38.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:38 smithi155 ceph-mon[35405]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-09-10T08:03:38.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:38 smithi155 ceph-mon[35405]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T08:03:38.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:38.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:38.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:38 smithi155 ceph-mon[35405]: from='osd.9 ' entity='osd.9' 2024-09-10T08:03:38.409 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:38 smithi107 ceph-mon[35048]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-09-10T08:03:38.409 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:38 smithi107 ceph-mon[35048]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T08:03:38.409 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:38.409 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:38.409 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:38 smithi107 ceph-mon[35048]: from='osd.9 ' entity='osd.9' 2024-09-10T08:03:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.875+0000 7f7053179640 1 -- 172.21.15.107:0/801184381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c102e00 msgr2=0x7f704c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:03:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.875+0000 7f7053179640 1 --2- 172.21.15.107:0/801184381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c102e00 0x7f704c103280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7038009900 tx=0x7f703802a410 comp rx=0 tx=0).stop 2024-09-10T08:03:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.875+0000 7f7053179640 1 -- 172.21.15.107:0/801184381 shutdown_connections 2024-09-10T08:03:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.875+0000 7f7053179640 1 --2- 172.21.15.107:0/801184381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c1037c0 0x7f704c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.875+0000 7f7053179640 1 --2- 172.21.15.107:0/801184381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c102e00 0x7f704c103280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.875+0000 7f7053179640 1 --2- 172.21.15.107:0/801184381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c101c00 0x7f704c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.875+0000 7f7053179640 1 -- 172.21.15.107:0/801184381 >> 172.21.15.107:0/801184381 conn(0x7f704c0fd390 msgr2=0x7f704c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:03:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.875+0000 7f7053179640 1 -- 172.21.15.107:0/801184381 shutdown_connections 2024-09-10T08:03:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.875+0000 7f7053179640 1 -- 172.21.15.107:0/801184381 wait complete. 2024-09-10T08:03:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f7053179640 1 Processor -- start 2024-09-10T08:03:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f7053179640 1 -- start start 2024-09-10T08:03:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f7053179640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c101c00 0x7f704c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:03:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f7053179640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c102e00 0x7f704c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:03:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f7053179640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c1037c0 0x7f704c19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:03:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f7053179640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10d860 con 0x7f704c101c00 2024-09-10T08:03:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f7053179640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10d6e0 con 0x7f704c102e00 2024-09-10T08:03:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f7053179640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10d9e0 con 0x7f704c1037c0 2024-09-10T08:03:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f704bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c102e00 0x7f704c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:03:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f704bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c102e00 0x7f704c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46310/0 (socket says 172.21.15.107:46310) 2024-09-10T08:03:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f704bfff640 1 -- 172.21.15.107:0/3885874509 learned_addr learned my addr 172.21.15.107:0/3885874509 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T08:03:38.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f7051716640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c1037c0 0x7f704c19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:03:38.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f7050f15640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c101c00 0x7f704c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:03:38.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f704bfff640 1 -- 172.21.15.107:0/3885874509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c1037c0 msgr2=0x7f704c19f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:03:38.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f704bfff640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c1037c0 0x7f704c19f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:38.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f704bfff640 1 -- 172.21.15.107:0/3885874509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c101c00 msgr2=0x7f704c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:03:38.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f704bfff640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c101c00 0x7f704c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:38.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f704bfff640 1 -- 172.21.15.107:0/3885874509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c19fad0 con 0x7f704c102e00 2024-09-10T08:03:38.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.876+0000 7f7050f15640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c101c00 0x7f704c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:03:38.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.877+0000 7f704bfff640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c102e00 0x7f704c1983e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7038009230 tx=0x7f7038004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:03:38.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.877+0000 7f7049ffb640 1 -- 172.21.15.107:0/3885874509 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703803f030 con 0x7f704c102e00 2024-09-10T08:03:38.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.877+0000 7f7053179640 1 -- 172.21.15.107:0/3885874509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c19fd60 con 0x7f704c102e00 2024-09-10T08:03:38.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.877+0000 7f7049ffb640 1 -- 172.21.15.107:0/3885874509 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7038003ce0 con 0x7f704c102e00 2024-09-10T08:03:38.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.877+0000 7f7049ffb640 1 -- 172.21.15.107:0/3885874509 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038035070 con 0x7f704c102e00 2024-09-10T08:03:38.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.877+0000 7f7053179640 1 -- 172.21.15.107:0/3885874509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c1a01e0 con 0x7f704c102e00 2024-09-10T08:03:38.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.878+0000 7f7053179640 1 -- 172.21.15.107:0/3885874509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704c108fa0 con 0x7f704c102e00 2024-09-10T08:03:38.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.879+0000 7f7049ffb640 1 -- 172.21.15.107:0/3885874509 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7038042030 con 0x7f704c102e00 2024-09-10T08:03:38.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.879+0000 7f7049ffb640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f703c0a2bb0 0x7f703c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:03:38.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.880+0000 7f7049ffb640 1 -- 172.21.15.107:0/3885874509 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6261+0+0 (secure 0 0 0) 0x7f70380eee50 con 0x7f704c102e00 2024-09-10T08:03:38.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.880+0000 7f7050f15640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f703c0a2bb0 0x7f703c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:03:38.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.880+0000 7f7050f15640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f703c0a2bb0 0x7f703c0a5070 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f70400058c0 tx=0x7f7040005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:03:38.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:38.884+0000 7f7049ffb640 1 -- 172.21.15.107:0/3885874509 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70380bc5b0 con 0x7f704c102e00 2024-09-10T08:03:39.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:39.068+0000 7f7053179640 1 -- 172.21.15.107:0/3885874509 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f704c062630 con 0x7f703c0a2bb0 2024-09-10T08:03:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: purged_snaps scrub starts 2024-09-10T08:03:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: purged_snaps scrub ok 2024-09-10T08:03:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: osd.9 [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099] boot 2024-09-10T08:03:39.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: osdmap e64: 10 total, 10 up, 10 in 2024-09-10T08:03:39.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:39.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:39.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:39.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:39.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:39.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:39.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:39.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:39.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:39.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:39.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:39.272 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:03:39.272 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:03:39.272 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:39.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: purged_snaps scrub starts 2024-09-10T08:03:39.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: purged_snaps scrub ok 2024-09-10T08:03:39.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:39.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:39.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: osd.9 [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099] boot 2024-09-10T08:03:39.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: osdmap e64: 10 total, 10 up, 10 in 2024-09-10T08:03:39.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:39.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:39.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:39.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:39.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:39.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:39.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:39.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:39.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:39.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:39.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:39.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:03:39.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:03:39.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:39.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: purged_snaps scrub starts 2024-09-10T08:03:39.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: purged_snaps scrub ok 2024-09-10T08:03:39.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:03:39.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:39.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: osd.9 [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099] boot 2024-09-10T08:03:39.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: osdmap e64: 10 total, 10 up, 10 in 2024-09-10T08:03:39.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:03:39.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:39.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:39.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:39.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:39.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:39.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:03:39.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:39.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:39.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:39.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:39.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:03:39.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:03:39.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:40.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:40 smithi155 ceph-mon[35405]: Detected new or changed devices on smithi002 2024-09-10T08:03:40.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:40 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi002 to 4273M 2024-09-10T08:03:40.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:40 smithi155 ceph-mon[35405]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:03:40.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:40 smithi107 ceph-mon[35048]: Detected new or changed devices on smithi002 2024-09-10T08:03:40.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:40 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi002 to 4273M 2024-09-10T08:03:40.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:40 smithi107 ceph-mon[35048]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:03:40.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:40 smithi002 ceph-mon[30563]: Detected new or changed devices on smithi002 2024-09-10T08:03:40.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:40 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi002 to 4273M 2024-09-10T08:03:40.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:40 smithi002 ceph-mon[30563]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:03:41.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:41 smithi155 ceph-mon[35405]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:41.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:41 smithi155 ceph-mon[35405]: osdmap e65: 10 total, 10 up, 10 in 2024-09-10T08:03:41.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:41 smithi107 ceph-mon[35048]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:41.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:41 smithi107 ceph-mon[35048]: osdmap e65: 10 total, 10 up, 10 in 2024-09-10T08:03:41.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:41 smithi002 ceph-mon[30563]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:41.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:41 smithi002 ceph-mon[30563]: osdmap e65: 10 total, 10 up, 10 in 2024-09-10T08:03:42.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:42 smithi155 ceph-mon[35405]: from='client.? 172.21.15.107:0/367522955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fbd17c0-c632-4ab1-aea8-b6db91498bf7"}]: dispatch 2024-09-10T08:03:42.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:42 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fbd17c0-c632-4ab1-aea8-b6db91498bf7"}]: dispatch 2024-09-10T08:03:42.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:42 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fbd17c0-c632-4ab1-aea8-b6db91498bf7"}]': finished 2024-09-10T08:03:42.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:42 smithi155 ceph-mon[35405]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T08:03:42.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:03:42.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:42 smithi155 ceph-mon[35405]: from='client.? 172.21.15.107:0/2863532785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:03:42.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:42 smithi107 ceph-mon[35048]: from='client.? 172.21.15.107:0/367522955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fbd17c0-c632-4ab1-aea8-b6db91498bf7"}]: dispatch 2024-09-10T08:03:42.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:42 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fbd17c0-c632-4ab1-aea8-b6db91498bf7"}]: dispatch 2024-09-10T08:03:42.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:42 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fbd17c0-c632-4ab1-aea8-b6db91498bf7"}]': finished 2024-09-10T08:03:42.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:42 smithi107 ceph-mon[35048]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T08:03:42.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:03:42.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:42 smithi107 ceph-mon[35048]: from='client.? 172.21.15.107:0/2863532785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:03:42.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:42 smithi002 ceph-mon[30563]: from='client.? 172.21.15.107:0/367522955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fbd17c0-c632-4ab1-aea8-b6db91498bf7"}]: dispatch 2024-09-10T08:03:42.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:42 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fbd17c0-c632-4ab1-aea8-b6db91498bf7"}]: dispatch 2024-09-10T08:03:42.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:42 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fbd17c0-c632-4ab1-aea8-b6db91498bf7"}]': finished 2024-09-10T08:03:42.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:42 smithi002 ceph-mon[30563]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T08:03:42.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:03:42.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:42 smithi002 ceph-mon[30563]: from='client.? 172.21.15.107:0/2863532785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:03:43.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:43 smithi002 ceph-mon[30563]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:43 smithi155 ceph-mon[35405]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:43.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:43 smithi107 ceph-mon[35048]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:45.773 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:45 smithi002 ceph-mon[30563]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:45.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:45 smithi155 ceph-mon[35405]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:45.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:45 smithi107 ceph-mon[35048]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:47.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:47 smithi002 ceph-mon[30563]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:47.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:47 smithi155 ceph-mon[35405]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:47.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:47 smithi107 ceph-mon[35048]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:49.590 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:49 smithi107 ceph-mon[35048]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:49.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:49 smithi002 ceph-mon[30563]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:49.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:49 smithi155 ceph-mon[35405]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:50.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T08:03:50.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:50.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T08:03:50.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T08:03:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:51.727 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:51 smithi107 ceph-mon[35048]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:51.727 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:51 smithi107 ceph-mon[35048]: Deploying daemon osd.10 on smithi107 2024-09-10T08:03:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:51 smithi155 ceph-mon[35405]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:51 smithi155 ceph-mon[35405]: Deploying daemon osd.10 on smithi107 2024-09-10T08:03:52.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:51 smithi002 ceph-mon[30563]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:52.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:51 smithi002 ceph-mon[30563]: Deploying daemon osd.10 on smithi107 2024-09-10T08:03:53.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:53 smithi107 ceph-mon[35048]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:53.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:53 smithi155 ceph-mon[35405]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:54.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:53 smithi002 ceph-mon[30563]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:55.794 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:55 smithi107 ceph-mon[35048]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:55.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:55 smithi155 ceph-mon[35405]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:56.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:55 smithi002 ceph-mon[30563]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:56.847 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:56.847 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:56.847 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:56.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:56.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:56.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:57.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:03:57.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:57.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:57.885 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:57 smithi107 ceph-mon[35048]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:57 smithi155 ceph-mon[35405]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:58.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:57 smithi002 ceph-mon[30563]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:59.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:59 smithi107 ceph-mon[35048]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:59.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:03:59 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:59.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:59.718+0000 7f7049ffb640 1 -- 172.21.15.107:0/3885874509 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f704c062630 con 0x7f703c0a2bb0 2024-09-10T08:03:59.718 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 10 on host 'smithi107' 2024-09-10T08:03:59.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:59.720+0000 7f7053179640 1 -- 172.21.15.107:0/3885874509 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f703c0a2bb0 msgr2=0x7f703c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:03:59.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:59.720+0000 7f7053179640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f703c0a2bb0 0x7f703c0a5070 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f70400058c0 tx=0x7f7040005850 comp rx=0 tx=0).stop 2024-09-10T08:03:59.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:59.720+0000 7f7053179640 1 -- 172.21.15.107:0/3885874509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c102e00 msgr2=0x7f704c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:03:59.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:59.720+0000 7f7053179640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c102e00 0x7f704c1983e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7038009230 tx=0x7f7038004360 comp rx=0 tx=0).stop 2024-09-10T08:03:59.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:59.720+0000 7f7053179640 1 -- 172.21.15.107:0/3885874509 shutdown_connections 2024-09-10T08:03:59.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:59.720+0000 7f7053179640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f703c0a2bb0 0x7f703c0a5070 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:59.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:59.720+0000 7f7053179640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c1037c0 0x7f704c19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:59.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:59.720+0000 7f7053179640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c102e00 0x7f704c1983e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:59.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:59.720+0000 7f7053179640 1 --2- 172.21.15.107:0/3885874509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c101c00 0x7f704c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:03:59.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:59.720+0000 7f7053179640 1 -- 172.21.15.107:0/3885874509 >> 172.21.15.107:0/3885874509 conn(0x7f704c0fd390 msgr2=0x7f704c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:03:59.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:59.720+0000 7f7053179640 1 -- 172.21.15.107:0/3885874509 shutdown_connections 2024-09-10T08:03:59.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T08:03:59.720+0000 7f7053179640 1 -- 172.21.15.107:0/3885874509 wait complete. 2024-09-10T08:03:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:59 smithi155 ceph-mon[35405]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:03:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:03:59.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:03:59.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:03:59.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:03:59 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:59 smithi002 ceph-mon[30563]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:04:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:04:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:03:59 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:00.547 DEBUG:teuthology.orchestra.run.smithi107:osd.10> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T08:04:00.549 INFO:tasks.cephadm:Deploying osd.11 on smithi155 with /dev/vg_nvme/lv_1... 2024-09-10T08:04:00.550 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T08:04:00.770 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:00 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:04:00.770 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:00 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:00.771 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:00 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:00.804 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:04:00.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:00 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:04:00.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:00 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:00.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:00 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:01.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:00 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:04:01.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:00 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:01.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:00 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:01.884 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:01 smithi107 ceph-mon[35048]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:04:01.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:01 smithi155 ceph-mon[35405]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:04:02.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:01 smithi002 ceph-mon[30563]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:04:02.416 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:04:02 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:04:02.290+0000 7fc2d5f00640 -1 osd.10 0 log_to_monitors true 2024-09-10T08:04:02.875 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:02 smithi155 ceph-mon[35405]: from='osd.10 [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:04:02.875 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:02 smithi155 ceph-mon[35405]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:04:02.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:02 smithi107 ceph-mon[35048]: from='osd.10 [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:04:02.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:02 smithi107 ceph-mon[35048]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:04:03.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:02 smithi002 ceph-mon[30563]: from='osd.10 [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:04:03.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:02 smithi002 ceph-mon[30563]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:04:03.651 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-10T08:04:03.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:04:03.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T08:04:03.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='osd.10 [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:04:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T08:04:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:04:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:04:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:03.677 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_1 2024-09-10T08:04:03.834 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:04:03.834 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T08:04:03.834 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='osd.10 [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:04:03.834 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T08:04:03.834 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:03.834 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:04:03.834 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:03.834 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:03.834 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:03.835 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:03.835 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:03.835 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:03.835 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:03.835 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:03.835 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:04:03.835 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:03.932 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:04:04.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:04:04.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T08:04:04.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='osd.10 [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:04:04.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T08:04:04.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:04.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T08:04:04.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:04:04.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:04.416 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:04:03.936+0000 7fc2d1e83640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T08:04:04.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:04:03 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:04:03.946+0000 7fc2c9ca3640 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:04:04.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.437+0000 7f7940701640 1 -- 172.21.15.155:0/2473954284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938101cd0 msgr2=0x7f7938102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:04.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.437+0000 7f7940701640 1 --2- 172.21.15.155:0/2473954284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938101cd0 0x7f7938102150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7928009900 tx=0x7f792802a410 comp rx=0 tx=0).stop 2024-09-10T08:04:04.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.437+0000 7f7940701640 1 -- 172.21.15.155:0/2473954284 shutdown_connections 2024-09-10T08:04:04.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.437+0000 7f7940701640 1 --2- 172.21.15.155:0/2473954284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7938102690 0x7f7938108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:04.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.437+0000 7f7940701640 1 --2- 172.21.15.155:0/2473954284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938101cd0 0x7f7938102150 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:04.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.437+0000 7f7940701640 1 --2- 172.21.15.155:0/2473954284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938100ad0 0x7f7938100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:04.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.437+0000 7f7940701640 1 -- 172.21.15.155:0/2473954284 >> 172.21.15.155:0/2473954284 conn(0x7f79380fc280 msgr2=0x7f79380fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:04.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.437+0000 7f7940701640 1 -- 172.21.15.155:0/2473954284 shutdown_connections 2024-09-10T08:04:04.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.437+0000 7f7940701640 1 -- 172.21.15.155:0/2473954284 wait complete. 2024-09-10T08:04:04.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.437+0000 7f7940701640 1 Processor -- start 2024-09-10T08:04:04.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f7940701640 1 -- start start 2024-09-10T08:04:04.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f7940701640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7938100ad0 0x7f7938104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:04.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f7940701640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938101cd0 0x7f79381030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:04.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f7940701640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938102690 0x7f79381035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:04.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f7940701640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f793810c800 con 0x7f7938101cd0 2024-09-10T08:04:04.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f7940701640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f793810c680 con 0x7f7938100ad0 2024-09-10T08:04:04.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f7940701640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f793810c980 con 0x7f7938102690 2024-09-10T08:04:04.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f793ec9e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938102690 0x7f79381035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:04.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f793ec9e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938102690 0x7f79381035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:37576/0 (socket says 172.21.15.155:37576) 2024-09-10T08:04:04.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f793ec9e640 1 -- 172.21.15.155:0/599423301 learned_addr learned my addr 172.21.15.155:0/599423301 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-10T08:04:04.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f793e49d640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7938100ad0 0x7f7938104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:04.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f793dc9c640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938101cd0 0x7f79381030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:04.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f793ec9e640 1 -- 172.21.15.155:0/599423301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7938100ad0 msgr2=0x7f7938104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:04.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f793ec9e640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7938100ad0 0x7f7938104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:04.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f793ec9e640 1 -- 172.21.15.155:0/599423301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938101cd0 msgr2=0x7f79381030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:04.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f793ec9e640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938101cd0 0x7f79381030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:04.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f793ec9e640 1 -- 172.21.15.155:0/599423301 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7938103eb0 con 0x7f7938102690 2024-09-10T08:04:04.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f793dc9c640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938101cd0 0x7f79381030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:04:04.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f793e49d640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7938100ad0 0x7f7938104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:04:04.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.438+0000 7f793ec9e640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938102690 0x7f79381035f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f793000eaa0 tx=0x7f793000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:04.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.439+0000 7f792f7fe640 1 -- 172.21.15.155:0/599423301 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793000cdb0 con 0x7f7938102690 2024-09-10T08:04:04.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.439+0000 7f792f7fe640 1 -- 172.21.15.155:0/599423301 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7930005020 con 0x7f7938102690 2024-09-10T08:04:04.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.439+0000 7f7940701640 1 -- 172.21.15.155:0/599423301 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79381a2110 con 0x7f7938102690 2024-09-10T08:04:04.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.439+0000 7f792f7fe640 1 -- 172.21.15.155:0/599423301 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79300116b0 con 0x7f7938102690 2024-09-10T08:04:04.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.439+0000 7f7940701640 1 -- 172.21.15.155:0/599423301 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79381a26f0 con 0x7f7938102690 2024-09-10T08:04:04.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.440+0000 7f7940701640 1 -- 172.21.15.155:0/599423301 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79381a2980 con 0x7f7938102690 2024-09-10T08:04:04.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.441+0000 7f792f7fe640 1 -- 172.21.15.155:0/599423301 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7930005560 con 0x7f7938102690 2024-09-10T08:04:04.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.441+0000 7f792f7fe640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79240a2c00 0x7f79240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:04.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.441+0000 7f792f7fe640 1 -- 172.21.15.155:0/599423301 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6414+0+0 (secure 0 0 0) 0x7f79300ceb30 con 0x7f7938102690 2024-09-10T08:04:04.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.442+0000 7f793e49d640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79240a2c00 0x7f79240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:04.443 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.442+0000 7f793e49d640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79240a2c00 0x7f79240a50c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7938101b30 tx=0x7f7934008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:04.445 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.445+0000 7f792f7fe640 1 -- 172.21.15.155:0/599423301 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7930016030 con 0x7f7938102690 2024-09-10T08:04:04.644 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:04.644+0000 7f7940701640 1 -- 172.21.15.155:0/599423301 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f793806a770 con 0x7f79240a2c00 2024-09-10T08:04:04.842 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:04 smithi155 ceph-mon[35405]: Detected new or changed devices on smithi107 2024-09-10T08:04:04.843 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:04 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi107 to 4271M 2024-09-10T08:04:04.843 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:04 smithi155 ceph-mon[35405]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T08:04:04.843 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:04 smithi155 ceph-mon[35405]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T08:04:04.843 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:04.843 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:04.843 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:04 smithi155 ceph-mon[35405]: from='osd.10 ' entity='osd.10' 2024-09-10T08:04:04.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:04 smithi107 ceph-mon[35048]: Detected new or changed devices on smithi107 2024-09-10T08:04:04.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:04 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi107 to 4271M 2024-09-10T08:04:04.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:04 smithi107 ceph-mon[35048]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T08:04:04.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:04 smithi107 ceph-mon[35048]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T08:04:04.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:04.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:04.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:04 smithi107 ceph-mon[35048]: from='osd.10 ' entity='osd.10' 2024-09-10T08:04:05.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:04 smithi002 ceph-mon[30563]: Detected new or changed devices on smithi107 2024-09-10T08:04:05.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:04 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi107 to 4271M 2024-09-10T08:04:05.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:04 smithi002 ceph-mon[30563]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T08:04:05.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:04 smithi002 ceph-mon[30563]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T08:04:05.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:05.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:05.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:04 smithi002 ceph-mon[30563]: from='osd.10 ' entity='osd.10' 2024-09-10T08:04:05.817 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:05 smithi155 ceph-mon[35405]: purged_snaps scrub starts 2024-09-10T08:04:05.817 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:05 smithi155 ceph-mon[35405]: purged_snaps scrub ok 2024-09-10T08:04:05.817 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:05 smithi155 ceph-mon[35405]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:04:05.817 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:05.817 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:05 smithi155 ceph-mon[35405]: osd.10 [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493] boot 2024-09-10T08:04:05.818 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:05 smithi155 ceph-mon[35405]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T08:04:05.818 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:05.818 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:05 smithi155 ceph-mon[35405]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:04:05.818 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:04:05.818 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:04:05.818 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:05.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:05 smithi107 ceph-mon[35048]: purged_snaps scrub starts 2024-09-10T08:04:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:05 smithi107 ceph-mon[35048]: purged_snaps scrub ok 2024-09-10T08:04:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:05 smithi107 ceph-mon[35048]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:04:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:05 smithi107 ceph-mon[35048]: osd.10 [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493] boot 2024-09-10T08:04:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:05 smithi107 ceph-mon[35048]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T08:04:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:05 smithi107 ceph-mon[35048]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:04:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:04:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:04:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:05 smithi002 ceph-mon[30563]: purged_snaps scrub starts 2024-09-10T08:04:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:05 smithi002 ceph-mon[30563]: purged_snaps scrub ok 2024-09-10T08:04:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:05 smithi002 ceph-mon[30563]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:04:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:05 smithi002 ceph-mon[30563]: osd.10 [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493] boot 2024-09-10T08:04:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:05 smithi002 ceph-mon[30563]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T08:04:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:04:06.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:05 smithi002 ceph-mon[30563]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:04:06.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:04:06.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:04:06.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:06.705 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:06 smithi155 ceph-mon[35405]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T08:04:06.705 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:06 smithi155 ceph-mon[35405]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:06.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:06 smithi107 ceph-mon[35048]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T08:04:06.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:06 smithi107 ceph-mon[35048]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:07.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:06 smithi002 ceph-mon[30563]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T08:04:07.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:06 smithi002 ceph-mon[30563]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:07.745 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:07 smithi155 ceph-mon[35405]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T08:04:07.745 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:07 smithi155 ceph-mon[35405]: from='client.? 172.21.15.155:0/2291975362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4caf962-e25f-4f3f-b3a7-a4495561ce6f"}]: dispatch 2024-09-10T08:04:07.745 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:07 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4caf962-e25f-4f3f-b3a7-a4495561ce6f"}]: dispatch 2024-09-10T08:04:07.746 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:07 smithi155 ceph-mon[35405]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4caf962-e25f-4f3f-b3a7-a4495561ce6f"}]': finished 2024-09-10T08:04:07.746 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:07 smithi155 ceph-mon[35405]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T08:04:07.746 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:07 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:07.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:07 smithi107 ceph-mon[35048]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T08:04:07.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:07 smithi107 ceph-mon[35048]: from='client.? 172.21.15.155:0/2291975362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4caf962-e25f-4f3f-b3a7-a4495561ce6f"}]: dispatch 2024-09-10T08:04:07.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:07 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4caf962-e25f-4f3f-b3a7-a4495561ce6f"}]: dispatch 2024-09-10T08:04:07.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:07 smithi107 ceph-mon[35048]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4caf962-e25f-4f3f-b3a7-a4495561ce6f"}]': finished 2024-09-10T08:04:07.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:07 smithi107 ceph-mon[35048]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T08:04:07.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:07 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:08.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:07 smithi002 ceph-mon[30563]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T08:04:08.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:07 smithi002 ceph-mon[30563]: from='client.? 172.21.15.155:0/2291975362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4caf962-e25f-4f3f-b3a7-a4495561ce6f"}]: dispatch 2024-09-10T08:04:08.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:07 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4caf962-e25f-4f3f-b3a7-a4495561ce6f"}]: dispatch 2024-09-10T08:04:08.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:07 smithi002 ceph-mon[30563]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4caf962-e25f-4f3f-b3a7-a4495561ce6f"}]': finished 2024-09-10T08:04:08.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:07 smithi002 ceph-mon[30563]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T08:04:08.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:07 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:08.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:08 smithi155 ceph-mon[35405]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:08.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:08 smithi155 ceph-mon[35405]: from='client.? 172.21.15.155:0/1623413994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:04:08.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:08 smithi155 ceph-mon[35405]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T08:04:08.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:08.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:08 smithi107 ceph-mon[35048]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:08.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:08 smithi107 ceph-mon[35048]: from='client.? 172.21.15.155:0/1623413994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:04:08.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:08 smithi107 ceph-mon[35048]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T08:04:08.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:09.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:08 smithi002 ceph-mon[30563]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:09.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:08 smithi002 ceph-mon[30563]: from='client.? 172.21.15.155:0/1623413994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:04:09.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:08 smithi002 ceph-mon[30563]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T08:04:09.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:11.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:10 smithi002 ceph-mon[30563]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:11.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:10 smithi155 ceph-mon[35405]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:11.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:10 smithi107 ceph-mon[35048]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:13.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:12 smithi002 ceph-mon[30563]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:04:13.069 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:12 smithi155 ceph-mon[35405]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:04:13.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:12 smithi107 ceph-mon[35048]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:04:14.836 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:14 smithi155 ceph-mon[35405]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T08:04:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:14 smithi002 ceph-mon[30563]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T08:04:15.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:14 smithi107 ceph-mon[35048]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T08:04:16.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:16 smithi155 ceph-mon[35405]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T08:04:16.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:16 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T08:04:16.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:16 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:16.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:16 smithi155 ceph-mon[35405]: Deploying daemon osd.11 on smithi155 2024-09-10T08:04:16.969 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:16 smithi107 ceph-mon[35048]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T08:04:16.969 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:16 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T08:04:16.969 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:16 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:16.969 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:16 smithi107 ceph-mon[35048]: Deploying daemon osd.11 on smithi155 2024-09-10T08:04:17.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:16 smithi002 ceph-mon[30563]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T08:04:17.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:16 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T08:04:17.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:16 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:17.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:16 smithi002 ceph-mon[30563]: Deploying daemon osd.11 on smithi155 2024-09-10T08:04:18.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:18 smithi155 ceph-mon[35405]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:04:19.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:18 smithi002 ceph-mon[30563]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:04:19.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:18 smithi107 ceph-mon[35048]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:04:20.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:20 smithi155 ceph-mon[35405]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T08:04:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:20 smithi002 ceph-mon[30563]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T08:04:21.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:20 smithi107 ceph-mon[35048]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T08:04:22.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:22 smithi155 ceph-mon[35405]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:04:22.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:22 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:04:22.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:22 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:22.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:22 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:22 smithi002 ceph-mon[30563]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:04:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:22 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:04:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:22 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:22 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:23.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:22 smithi107 ceph-mon[35048]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:04:23.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:22 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:04:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:22 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:22 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:24.895 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 11 on host 'smithi155' 2024-09-10T08:04:24.896 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:24.895+0000 7f792f7fe640 1 -- 172.21.15.155:0/599423301 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f793806a770 con 0x7f79240a2c00 2024-09-10T08:04:24.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:24.898+0000 7f7940701640 1 -- 172.21.15.155:0/599423301 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79240a2c00 msgr2=0x7f79240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:24.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:24.898+0000 7f7940701640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79240a2c00 0x7f79240a50c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7938101b30 tx=0x7f7934008040 comp rx=0 tx=0).stop 2024-09-10T08:04:24.897 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:24.898+0000 7f7940701640 1 -- 172.21.15.155:0/599423301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938102690 msgr2=0x7f79381035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:24.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:24.898+0000 7f7940701640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938102690 0x7f79381035f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f793000eaa0 tx=0x7f793000ef60 comp rx=0 tx=0).stop 2024-09-10T08:04:24.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:24.898+0000 7f7940701640 1 -- 172.21.15.155:0/599423301 shutdown_connections 2024-09-10T08:04:24.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:24.898+0000 7f7940701640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79240a2c00 0x7f79240a50c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:24.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:24.898+0000 7f7940701640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938102690 0x7f79381035f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:24.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:24.898+0000 7f7940701640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938101cd0 0x7f79381030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:24.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:24.898+0000 7f7940701640 1 --2- 172.21.15.155:0/599423301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7938100ad0 0x7f7938104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:24.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:24.898+0000 7f7940701640 1 -- 172.21.15.155:0/599423301 >> 172.21.15.155:0/599423301 conn(0x7f79380fc280 msgr2=0x7f793806a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:24.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:24.898+0000 7f7940701640 1 -- 172.21.15.155:0/599423301 shutdown_connections 2024-09-10T08:04:24.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:24.898+0000 7f7940701640 1 -- 172.21.15.155:0/599423301 wait complete. 2024-09-10T08:04:24.979 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:24 smithi155 ceph-mon[35405]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:25.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:24 smithi002 ceph-mon[30563]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:25.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:24 smithi107 ceph-mon[35048]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:25.804 DEBUG:teuthology.orchestra.run.smithi155:osd.11> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T08:04:25.806 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T08:04:25.807 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T08:04:26.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:26.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:26.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:26.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:26.194 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:04:26.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:26.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.763+0000 7fd636400640 1 -- 172.21.15.2:0/1573384167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6301037c0 msgr2=0x7fd63010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.763+0000 7fd636400640 1 --2- 172.21.15.2:0/1573384167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6301037c0 0x7fd63010a050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd628009ca0 tx=0x7fd62802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:04:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.763+0000 7fd636400640 1 -- 172.21.15.2:0/1573384167 shutdown_connections 2024-09-10T08:04:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.763+0000 7fd636400640 1 --2- 172.21.15.2:0/1573384167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6301037c0 0x7fd63010a050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.763+0000 7fd636400640 1 --2- 172.21.15.2:0/1573384167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd630102e00 0x7fd630103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.763+0000 7fd636400640 1 --2- 172.21.15.2:0/1573384167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd630101c00 0x7fd630102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.763+0000 7fd636400640 1 -- 172.21.15.2:0/1573384167 >> 172.21.15.2:0/1573384167 conn(0x7fd6300fd390 msgr2=0x7fd6300ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:26.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.763+0000 7fd636400640 1 -- 172.21.15.2:0/1573384167 shutdown_connections 2024-09-10T08:04:26.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.763+0000 7fd636400640 1 -- 172.21.15.2:0/1573384167 wait complete. 2024-09-10T08:04:26.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.764+0000 7fd636400640 1 Processor -- start 2024-09-10T08:04:26.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.764+0000 7fd636400640 1 -- start start 2024-09-10T08:04:26.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.764+0000 7fd636400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd630101c00 0x7fd630075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:26.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.764+0000 7fd636400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd630102e00 0x7fd6300745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:26.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.764+0000 7fd636400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6301037c0 0x7fd630074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:26.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.764+0000 7fd636400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd63010d8d0 con 0x7fd630101c00 2024-09-10T08:04:26.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.764+0000 7fd636400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd63010d750 con 0x7fd630102e00 2024-09-10T08:04:26.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.764+0000 7fd636400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd63010da50 con 0x7fd6301037c0 2024-09-10T08:04:26.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.765+0000 7fd62f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd630101c00 0x7fd630075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.765+0000 7fd62f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd630101c00 0x7fd630075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46484/0 (socket says 172.21.15.2:46484) 2024-09-10T08:04:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.765+0000 7fd62f400640 1 -- 172.21.15.2:0/3960032176 learned_addr learned my addr 172.21.15.2:0/3960032176 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:04:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.765+0000 7fd627e00640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd630102e00 0x7fd6300745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.765+0000 7fd62fe00640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6301037c0 0x7fd630074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.765+0000 7fd62f400640 1 -- 172.21.15.2:0/3960032176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6301037c0 msgr2=0x7fd630074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.765+0000 7fd62f400640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6301037c0 0x7fd630074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.765+0000 7fd62f400640 1 -- 172.21.15.2:0/3960032176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd630102e00 msgr2=0x7fd6300745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.765+0000 7fd62f400640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd630102e00 0x7fd6300745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.765+0000 7fd62f400640 1 -- 172.21.15.2:0/3960032176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6300753e0 con 0x7fd630101c00 2024-09-10T08:04:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.765+0000 7fd62f400640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd630101c00 0x7fd630075f90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd62000e9c0 tx=0x7fd62000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.766+0000 7fd62cc00640 1 -- 172.21.15.2:0/3960032176 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62000cd80 con 0x7fd630101c00 2024-09-10T08:04:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.766+0000 7fd62cc00640 1 -- 172.21.15.2:0/3960032176 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6200035f0 con 0x7fd630101c00 2024-09-10T08:04:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.766+0000 7fd636400640 1 -- 172.21.15.2:0/3960032176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6301a1fd0 con 0x7fd630101c00 2024-09-10T08:04:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.766+0000 7fd62cc00640 1 -- 172.21.15.2:0/3960032176 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620010690 con 0x7fd630101c00 2024-09-10T08:04:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.766+0000 7fd636400640 1 -- 172.21.15.2:0/3960032176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6301a2430 con 0x7fd630101c00 2024-09-10T08:04:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.767+0000 7fd636400640 1 -- 172.21.15.2:0/3960032176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd630108f80 con 0x7fd630101c00 2024-09-10T08:04:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.769+0000 7fd62cc00640 1 -- 172.21.15.2:0/3960032176 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd6200037a0 con 0x7fd630101c00 2024-09-10T08:04:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.770+0000 7fd62cc00640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd6140a2b80 0x7fd6140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.770+0000 7fd62cc00640 1 -- 172.21.15.2:0/3960032176 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6809+0+0 (secure 0 0 0) 0x7fd6200cef70 con 0x7fd630101c00 2024-09-10T08:04:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.771+0000 7fd627e00640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd6140a2b80 0x7fd6140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.771+0000 7fd627e00640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd6140a2b80 0x7fd6140a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd630075d10 tx=0x7fd618005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.772+0000 7fd62cc00640 1 -- 172.21.15.2:0/3960032176 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd62009c4a0 con 0x7fd630101c00 2024-09-10T08:04:26.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.957+0000 7fd636400640 1 -- 172.21.15.2:0/3960032176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fd630062e90 con 0x7fd630101c00 2024-09-10T08:04:26.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.958+0000 7fd62cc00640 1 -- 172.21.15.2:0/3960032176 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7fd62009bda0 con 0x7fd630101c00 2024-09-10T08:04:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:04:26.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.962+0000 7fd636400640 1 -- 172.21.15.2:0/3960032176 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd6140a2b80 msgr2=0x7fd6140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:26.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.962+0000 7fd636400640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd6140a2b80 0x7fd6140a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd630075d10 tx=0x7fd618005f90 comp rx=0 tx=0).stop 2024-09-10T08:04:26.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.962+0000 7fd636400640 1 -- 172.21.15.2:0/3960032176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd630101c00 msgr2=0x7fd630075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:26.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.962+0000 7fd636400640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd630101c00 0x7fd630075f90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd62000e9c0 tx=0x7fd62000ee80 comp rx=0 tx=0).stop 2024-09-10T08:04:26.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.962+0000 7fd636400640 1 -- 172.21.15.2:0/3960032176 shutdown_connections 2024-09-10T08:04:26.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.962+0000 7fd636400640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd6140a2b80 0x7fd6140a5040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:26.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.962+0000 7fd636400640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6301037c0 0x7fd630074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:26.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.962+0000 7fd636400640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd630102e00 0x7fd6300745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:26.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.962+0000 7fd636400640 1 --2- 172.21.15.2:0/3960032176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd630101c00 0x7fd630075f90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:26.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.962+0000 7fd636400640 1 -- 172.21.15.2:0/3960032176 >> 172.21.15.2:0/3960032176 conn(0x7fd6300fd390 msgr2=0x7fd630106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:26.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.962+0000 7fd636400640 1 -- 172.21.15.2:0/3960032176 shutdown_connections 2024-09-10T08:04:26.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:26.962+0000 7fd636400640 1 -- 172.21.15.2:0/3960032176 wait complete. 2024-09-10T08:04:27.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:26 smithi155 ceph-mon[35405]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:27.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:26 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:27.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:26 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:27.082 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:26 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3960032176' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:04:27.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:26 smithi002 ceph-mon[30563]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:27.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:26 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:27.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:26 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:27.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:26 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3960032176' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:04:27.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:26 smithi107 ceph-mon[35048]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:27.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:26 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:27.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:26 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:27.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:26 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3960032176' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:04:27.834 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1725955444,"num_in_osds":12,"osd_in_since":1725955447,"num_remapped_pgs":0} 2024-09-10T08:04:27.906 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:04:27 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:04:27.751+0000 7f9423943640 -1 osd.11 0 log_to_monitors true 2024-09-10T08:04:28.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:27 smithi002 ceph-mon[30563]: from='osd.11 [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:04:28.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:27 smithi002 ceph-mon[30563]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:04:28.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:27 smithi155 ceph-mon[35405]: from='osd.11 [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:04:28.321 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:27 smithi155 ceph-mon[35405]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:04:28.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:27 smithi107 ceph-mon[35048]: from='osd.11 [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:04:28.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:27 smithi107 ceph-mon[35048]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:04:28.835 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T08:04:29.209 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:28 smithi002 ceph-mon[30563]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:29.209 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:28 smithi002 ceph-mon[30563]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T08:04:29.209 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:28 smithi002 ceph-mon[30563]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T08:04:29.209 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:28 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:29.209 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:28 smithi002 ceph-mon[30563]: from='osd.11 [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:04:29.209 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:28 smithi002 ceph-mon[30563]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:04:29.242 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:04:29.258 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:28 smithi107 ceph-mon[35048]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:29.258 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:28 smithi107 ceph-mon[35048]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T08:04:29.259 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:28 smithi107 ceph-mon[35048]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T08:04:29.259 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:28 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:29.259 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:28 smithi107 ceph-mon[35048]: from='osd.11 [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:04:29.259 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:28 smithi107 ceph-mon[35048]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:04:29.406 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:04:28 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:04:28.991+0000 7f941f8c6640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T08:04:29.406 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:04:29.005+0000 7f941b6ee640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:04:29.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:28 smithi155 ceph-mon[35405]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:29.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:28 smithi155 ceph-mon[35405]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T08:04:29.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:28 smithi155 ceph-mon[35405]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T08:04:29.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:28 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:29.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:28 smithi155 ceph-mon[35405]: from='osd.11 [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:04:29.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:28 smithi155 ceph-mon[35405]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-09-10T08:04:29.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.822+0000 7f3871e00640 1 -- 172.21.15.2:0/463695080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f386c1037c0 msgr2=0x7f386c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:29.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.822+0000 7f3871e00640 1 --2- 172.21.15.2:0/463695080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f386c1037c0 0x7f386c10a050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3864009ca0 tx=0x7f386402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:04:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.822+0000 7f3871e00640 1 -- 172.21.15.2:0/463695080 shutdown_connections 2024-09-10T08:04:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.822+0000 7f3871e00640 1 --2- 172.21.15.2:0/463695080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f386c1037c0 0x7f386c10a050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.822+0000 7f3871e00640 1 --2- 172.21.15.2:0/463695080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f386c102e00 0x7f386c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.822+0000 7f3871e00640 1 --2- 172.21.15.2:0/463695080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f386c101c00 0x7f386c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.822+0000 7f3871e00640 1 -- 172.21.15.2:0/463695080 >> 172.21.15.2:0/463695080 conn(0x7f386c0fd390 msgr2=0x7f386c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.822+0000 7f3871e00640 1 -- 172.21.15.2:0/463695080 shutdown_connections 2024-09-10T08:04:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.823+0000 7f3871e00640 1 -- 172.21.15.2:0/463695080 wait complete. 2024-09-10T08:04:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.824+0000 7f3871e00640 1 Processor -- start 2024-09-10T08:04:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.824+0000 7f3871e00640 1 -- start start 2024-09-10T08:04:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.824+0000 7f3871e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f386c101c00 0x7f386c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.824+0000 7f3871e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f386c102e00 0x7f386c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.824+0000 7f3871e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f386c1037c0 0x7f386c1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:29.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.824+0000 7f3871e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f386c10d880 con 0x7f386c102e00 2024-09-10T08:04:29.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.824+0000 7f3871e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f386c10d700 con 0x7f386c1037c0 2024-09-10T08:04:29.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.824+0000 7f3871e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f386c10da00 con 0x7f386c101c00 2024-09-10T08:04:29.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.824+0000 7f386a000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f386c102e00 0x7f386c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:29.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.824+0000 7f386a000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f386c102e00 0x7f386c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48280/0 (socket says 172.21.15.2:48280) 2024-09-10T08:04:29.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.824+0000 7f386b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f386c1037c0 0x7f386c1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:29.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.825+0000 7f386a000640 1 -- 172.21.15.2:0/1600255852 learned_addr learned my addr 172.21.15.2:0/1600255852 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:04:29.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.825+0000 7f386aa00640 1 --2- 172.21.15.2:0/1600255852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f386c101c00 0x7f386c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:29.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.825+0000 7f386b400640 1 -- 172.21.15.2:0/1600255852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f386c101c00 msgr2=0x7f386c19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:29.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.825+0000 7f386b400640 1 --2- 172.21.15.2:0/1600255852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f386c101c00 0x7f386c19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:29.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.825+0000 7f386b400640 1 -- 172.21.15.2:0/1600255852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f386c102e00 msgr2=0x7f386c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:29.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.825+0000 7f386b400640 1 --2- 172.21.15.2:0/1600255852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f386c102e00 0x7f386c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:29.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.825+0000 7f386b400640 1 -- 172.21.15.2:0/1600255852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386c1a1cb0 con 0x7f386c1037c0 2024-09-10T08:04:29.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.825+0000 7f386b400640 1 --2- 172.21.15.2:0/1600255852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f386c1037c0 0x7f386c1a1640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3864031ae0 tx=0x7f3864031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:29.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.826+0000 7f3870a00640 1 -- 172.21.15.2:0/1600255852 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864003c30 con 0x7f386c1037c0 2024-09-10T08:04:29.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.826+0000 7f3870a00640 1 -- 172.21.15.2:0/1600255852 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3864003dd0 con 0x7f386c1037c0 2024-09-10T08:04:29.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.826+0000 7f3870a00640 1 -- 172.21.15.2:0/1600255852 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386402fcc0 con 0x7f386c1037c0 2024-09-10T08:04:29.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.826+0000 7f3871e00640 1 -- 172.21.15.2:0/1600255852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386c1a1f40 con 0x7f386c1037c0 2024-09-10T08:04:29.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.826+0000 7f3871e00640 1 -- 172.21.15.2:0/1600255852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f386c1a23f0 con 0x7f386c1037c0 2024-09-10T08:04:29.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.827+0000 7f3871e00640 1 -- 172.21.15.2:0/1600255852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f386c1a2620 con 0x7f386c1037c0 2024-09-10T08:04:29.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.829+0000 7f3870a00640 1 -- 172.21.15.2:0/1600255852 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3864004050 con 0x7f386c1037c0 2024-09-10T08:04:29.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.830+0000 7f3870a00640 1 --2- 172.21.15.2:0/1600255852 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38500a2bb0 0x7f38500a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:29.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.830+0000 7f3870a00640 1 -- 172.21.15.2:0/1600255852 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6847+0+0 (secure 0 0 0) 0x7f38640efbb0 con 0x7f386c1037c0 2024-09-10T08:04:29.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.830+0000 7f386aa00640 1 --2- 172.21.15.2:0/1600255852 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38500a2bb0 0x7f38500a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:29.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.831+0000 7f386aa00640 1 --2- 172.21.15.2:0/1600255852 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38500a2bb0 0x7f38500a5070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f386c102c60 tx=0x7f385c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:29.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:29.834+0000 7f3870a00640 1 -- 172.21.15.2:0/1600255852 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38640bd030 con 0x7f386c1037c0 2024-09-10T08:04:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.025+0000 7f3871e00640 1 -- 172.21.15.2:0/1600255852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f386c1a2d80 con 0x7f386c1037c0 2024-09-10T08:04:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.026+0000 7f3870a00640 1 -- 172.21.15.2:0/1600255852 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f38640bc930 con 0x7f386c1037c0 2024-09-10T08:04:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:04:30.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.028+0000 7f3871e00640 1 -- 172.21.15.2:0/1600255852 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38500a2bb0 msgr2=0x7f38500a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:30.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.028+0000 7f3871e00640 1 --2- 172.21.15.2:0/1600255852 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38500a2bb0 0x7f38500a5070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f386c102c60 tx=0x7f385c005f90 comp rx=0 tx=0).stop 2024-09-10T08:04:30.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.028+0000 7f3871e00640 1 -- 172.21.15.2:0/1600255852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f386c1037c0 msgr2=0x7f386c1a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:30.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.028+0000 7f3871e00640 1 --2- 172.21.15.2:0/1600255852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f386c1037c0 0x7f386c1a1640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3864031ae0 tx=0x7f3864031ee0 comp rx=0 tx=0).stop 2024-09-10T08:04:30.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.029+0000 7f3871e00640 1 -- 172.21.15.2:0/1600255852 shutdown_connections 2024-09-10T08:04:30.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.029+0000 7f3871e00640 1 --2- 172.21.15.2:0/1600255852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f386c1037c0 0x7f386c1a1640 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:30.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.029+0000 7f3871e00640 1 --2- 172.21.15.2:0/1600255852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f386c102e00 0x7f386c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:30.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.029+0000 7f3871e00640 1 --2- 172.21.15.2:0/1600255852 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38500a2bb0 0x7f38500a5070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:30.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.029+0000 7f3871e00640 1 --2- 172.21.15.2:0/1600255852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f386c101c00 0x7f386c19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:30.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.029+0000 7f3871e00640 1 -- 172.21.15.2:0/1600255852 >> 172.21.15.2:0/1600255852 conn(0x7f386c0fd390 msgr2=0x7f386c074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:30.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.029+0000 7f3871e00640 1 -- 172.21.15.2:0/1600255852 shutdown_connections 2024-09-10T08:04:30.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:30.029+0000 7f3871e00640 1 -- 172.21.15.2:0/1600255852 wait complete. 2024-09-10T08:04:30.263 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-10T08:04:30.263 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T08:04:30.263 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:30.263 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:30.263 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: Detected new or changed devices on smithi155 2024-09-10T08:04:30.263 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.264 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.264 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:30.264 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:30.264 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:30.264 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:30.264 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi155 to 4272M 2024-09-10T08:04:30.264 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.264 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:30.264 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:04:30.264 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.264 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:04:30.264 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:30.264 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:04:30.265 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:29 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-10T08:04:30.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T08:04:30.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:30.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:30.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: Detected new or changed devices on smithi155 2024-09-10T08:04:30.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:30.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:30.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:30.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:30.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi155 to 4272M 2024-09-10T08:04:30.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:30.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:04:30.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:04:30.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:30.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:04:30.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:29 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-09-10T08:04:30.444 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T08:04:30.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:30.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:30.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: Detected new or changed devices on smithi155 2024-09-10T08:04:30.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:30.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:30.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:30.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:04:30.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi155 to 4272M 2024-09-10T08:04:30.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.446 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:30.446 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:04:30.446 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.446 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:04:30.446 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:30.446 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:04:30.446 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:29 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:30.942 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1725955470,"num_in_osds":12,"osd_in_since":1725955447,"num_remapped_pgs":0} 2024-09-10T08:04:30.943 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:04:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:30 smithi002 ceph-mon[30563]: purged_snaps scrub starts 2024-09-10T08:04:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:30 smithi002 ceph-mon[30563]: purged_snaps scrub ok 2024-09-10T08:04:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:30 smithi002 ceph-mon[30563]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:30 smithi002 ceph-mon[30563]: osd.11 [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241] boot 2024-09-10T08:04:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:30 smithi002 ceph-mon[30563]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T08:04:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:30 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1600255852' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:04:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:04:31.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:30 smithi155 ceph-mon[35405]: purged_snaps scrub starts 2024-09-10T08:04:31.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:30 smithi155 ceph-mon[35405]: purged_snaps scrub ok 2024-09-10T08:04:31.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:30 smithi155 ceph-mon[35405]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:31.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:31.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:30 smithi155 ceph-mon[35405]: osd.11 [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241] boot 2024-09-10T08:04:31.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:30 smithi155 ceph-mon[35405]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T08:04:31.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:31.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:30 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1600255852' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:04:31.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:30 smithi107 ceph-mon[35048]: purged_snaps scrub starts 2024-09-10T08:04:31.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:30 smithi107 ceph-mon[35048]: purged_snaps scrub ok 2024-09-10T08:04:31.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:30 smithi107 ceph-mon[35048]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:04:31.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:31.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:30 smithi107 ceph-mon[35048]: osd.11 [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241] boot 2024-09-10T08:04:31.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:30 smithi107 ceph-mon[35048]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T08:04:31.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:04:31.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:30 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1600255852' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:04:31.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.871+0000 7fcaa1000640 1 -- 172.21.15.2:0/2143197881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca9c1019c0 msgr2=0x7fca9c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:31.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.871+0000 7fcaa1000640 1 --2- 172.21.15.2:0/2143197881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca9c1019c0 0x7fca9c101e20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fca84009930 tx=0x7fca8402a420 comp rx=0 tx=0).stop 2024-09-10T08:04:31.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.871+0000 7fcaa1000640 1 -- 172.21.15.2:0/2143197881 shutdown_connections 2024-09-10T08:04:31.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.871+0000 7fcaa1000640 1 --2- 172.21.15.2:0/2143197881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca9c102360 0x7fca9c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:31.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.871+0000 7fcaa1000640 1 --2- 172.21.15.2:0/2143197881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca9c1019c0 0x7fca9c101e20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:31.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.871+0000 7fcaa1000640 1 --2- 172.21.15.2:0/2143197881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c1079c0 0x7fca9c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:31.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.871+0000 7fcaa1000640 1 -- 172.21.15.2:0/2143197881 >> 172.21.15.2:0/2143197881 conn(0x7fca9c0fd740 msgr2=0x7fca9c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:31.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.871+0000 7fcaa1000640 1 -- 172.21.15.2:0/2143197881 shutdown_connections 2024-09-10T08:04:31.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.871+0000 7fcaa1000640 1 -- 172.21.15.2:0/2143197881 wait complete. 2024-09-10T08:04:31.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.872+0000 7fcaa1000640 1 Processor -- start 2024-09-10T08:04:31.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.872+0000 7fcaa1000640 1 -- start start 2024-09-10T08:04:31.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fcaa1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c1019c0 0x7fca9c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:31.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fcaa1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca9c102360 0x7fca9c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:31.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fcaa1000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca9c1079c0 0x7fca9c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:31.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fcaa1000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c111e10 con 0x7fca9c1079c0 2024-09-10T08:04:31.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fcaa1000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c111c90 con 0x7fca9c102360 2024-09-10T08:04:31.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fcaa1000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c111f90 con 0x7fca9c1019c0 2024-09-10T08:04:31.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fca9b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c1019c0 0x7fca9c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:31.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fca9b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c1019c0 0x7fca9c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35550/0 (socket says 172.21.15.2:35550) 2024-09-10T08:04:31.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fca9b400640 1 -- 172.21.15.2:0/4057685039 learned_addr learned my addr 172.21.15.2:0/4057685039 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:04:31.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fca9be00640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca9c1079c0 0x7fca9c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:31.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fca93e00640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca9c102360 0x7fca9c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:31.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fca9b400640 1 -- 172.21.15.2:0/4057685039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca9c102360 msgr2=0x7fca9c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:31.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fca9b400640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca9c102360 0x7fca9c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:31.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fca9b400640 1 -- 172.21.15.2:0/4057685039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca9c1079c0 msgr2=0x7fca9c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:31.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fca9b400640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca9c1079c0 0x7fca9c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:31.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fca9b400640 1 -- 172.21.15.2:0/4057685039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c19d320 con 0x7fca9c1019c0 2024-09-10T08:04:31.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.873+0000 7fca9be00640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca9c1079c0 0x7fca9c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:04:31.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.874+0000 7fca9b400640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c1019c0 0x7fca9c198370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d770 tx=0x7fca8c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:31.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.874+0000 7fca98c00640 1 -- 172.21.15.2:0/4057685039 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c017070 con 0x7fca9c1019c0 2024-09-10T08:04:31.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.874+0000 7fcaa1000640 1 -- 172.21.15.2:0/4057685039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9c19d610 con 0x7fca9c1019c0 2024-09-10T08:04:31.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.874+0000 7fca98c00640 1 -- 172.21.15.2:0/4057685039 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca8c005500 con 0x7fca9c1019c0 2024-09-10T08:04:31.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.874+0000 7fca98c00640 1 -- 172.21.15.2:0/4057685039 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c004e00 con 0x7fca9c1019c0 2024-09-10T08:04:31.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.874+0000 7fcaa1000640 1 -- 172.21.15.2:0/4057685039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9c074420 con 0x7fca9c1019c0 2024-09-10T08:04:31.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.875+0000 7fcaa1000640 1 -- 172.21.15.2:0/4057685039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9c103100 con 0x7fca9c1019c0 2024-09-10T08:04:31.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.877+0000 7fca98c00640 1 -- 172.21.15.2:0/4057685039 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fca8c004fa0 con 0x7fca9c1019c0 2024-09-10T08:04:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.878+0000 7fca98c00640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca800a2c00 0x7fca800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.879+0000 7fca98c00640 1 -- 172.21.15.2:0/4057685039 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fca8c0ce510 con 0x7fca9c1019c0 2024-09-10T08:04:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.879+0000 7fca93e00640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca800a2c00 0x7fca800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:31.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.880+0000 7fca93e00640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca800a2c00 0x7fca800a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fca84006010 tx=0x7fca84000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:31.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:31.881+0000 7fca98c00640 1 -- 172.21.15.2:0/4057685039 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca8c09b2a0 con 0x7fca9c1019c0 2024-09-10T08:04:32.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.065+0000 7fcaa1000640 1 -- 172.21.15.2:0/4057685039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca9c109f30 con 0x7fca9c1019c0 2024-09-10T08:04:32.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.066+0000 7fca98c00640 1 -- 172.21.15.2:0/4057685039 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18384 (secure 0 0 0) 0x7fca8c09f5a0 con 0x7fca9c1019c0 2024-09-10T08:04:32.066 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:04:32.067 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":77,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:04:31.190185+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:04:32.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.069+0000 7fcaa1000640 1 -- 172.21.15.2:0/4057685039 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca800a2c00 msgr2=0x7fca800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:32.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.069+0000 7fcaa1000640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca800a2c00 0x7fca800a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fca84006010 tx=0x7fca84000bc0 comp rx=0 tx=0).stop 2024-09-10T08:04:32.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.069+0000 7fcaa1000640 1 -- 172.21.15.2:0/4057685039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c1019c0 msgr2=0x7fca9c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:32.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.069+0000 7fcaa1000640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c1019c0 0x7fca9c198370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d770 tx=0x7fca8c00dc30 comp rx=0 tx=0).stop 2024-09-10T08:04:32.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.069+0000 7fcaa1000640 1 -- 172.21.15.2:0/4057685039 shutdown_connections 2024-09-10T08:04:32.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.069+0000 7fcaa1000640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca9c1079c0 0x7fca9c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:32.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.069+0000 7fcaa1000640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca9c102360 0x7fca9c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:32.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.069+0000 7fcaa1000640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca800a2c00 0x7fca800a50c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:32.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.069+0000 7fcaa1000640 1 --2- 172.21.15.2:0/4057685039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c1019c0 0x7fca9c198370 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:32.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.069+0000 7fcaa1000640 1 -- 172.21.15.2:0/4057685039 >> 172.21.15.2:0/4057685039 conn(0x7fca9c0fd740 msgr2=0x7fca9c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:32.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.069+0000 7fcaa1000640 1 -- 172.21.15.2:0/4057685039 shutdown_connections 2024-09-10T08:04:32.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:32.070+0000 7fcaa1000640 1 -- 172.21.15.2:0/4057685039 wait complete. 2024-09-10T08:04:32.456 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:32 smithi002 ceph-mon[30563]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T08:04:32.456 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:32 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/4057685039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:04:32.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:32 smithi155 ceph-mon[35405]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T08:04:32.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:32 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/4057685039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:04:32.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:32 smithi107 ceph-mon[35048]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T08:04:32.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:32 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/4057685039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:04:32.942 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T08:00:26.170464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T08:04:32.943 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T08:04:33.317 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:04:33.456 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:33 smithi002 ceph-mon[30563]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:33.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:33 smithi155 ceph-mon[35405]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:33.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:33 smithi107 ceph-mon[35048]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:33.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.890+0000 7fa1ce000640 1 -- 172.21.15.2:0/1882867157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1c8103ff0 msgr2=0x7fa1c81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:33.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.890+0000 7fa1ce000640 1 --2- 172.21.15.2:0/1882867157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1c8103ff0 0x7fa1c81043d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009900 tx=0x7fa1b802a410 comp rx=0 tx=0).stop 2024-09-10T08:04:33.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.890+0000 7fa1ce000640 1 -- 172.21.15.2:0/1882867157 shutdown_connections 2024-09-10T08:04:33.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.890+0000 7fa1ce000640 1 --2- 172.21.15.2:0/1882867157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c8109120 0x7fa1c8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:33.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.890+0000 7fa1ce000640 1 --2- 172.21.15.2:0/1882867157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1c81049a0 0x7fa1c81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:33.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.890+0000 7fa1ce000640 1 --2- 172.21.15.2:0/1882867157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1c8103ff0 0x7fa1c81043d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:33.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.890+0000 7fa1ce000640 1 -- 172.21.15.2:0/1882867157 >> 172.21.15.2:0/1882867157 conn(0x7fa1c80ffa90 msgr2=0x7fa1c8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:33.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.890+0000 7fa1ce000640 1 -- 172.21.15.2:0/1882867157 shutdown_connections 2024-09-10T08:04:33.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.890+0000 7fa1ce000640 1 -- 172.21.15.2:0/1882867157 wait complete. 2024-09-10T08:04:33.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.891+0000 7fa1ce000640 1 Processor -- start 2024-09-10T08:04:33.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.891+0000 7fa1ce000640 1 -- start start 2024-09-10T08:04:33.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.891+0000 7fa1ce000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c8103ff0 0x7fa1c819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:33.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.891+0000 7fa1ce000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1c81049a0 0x7fa1c819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:33.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.891+0000 7fa1ce000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1c8109120 0x7fa1c819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:33.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.891+0000 7fa1ce000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c8114150 con 0x7fa1c81049a0 2024-09-10T08:04:33.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.891+0000 7fa1ce000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c8113fd0 con 0x7fa1c8109120 2024-09-10T08:04:33.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.891+0000 7fa1ce000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c81142d0 con 0x7fa1c8103ff0 2024-09-10T08:04:33.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.892+0000 7fa1bea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1c81049a0 0x7fa1c819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:33.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.892+0000 7fa1c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1c8109120 0x7fa1c819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:33.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.892+0000 7fa1c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1c8109120 0x7fa1c819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38654/0 (socket says 172.21.15.2:38654) 2024-09-10T08:04:33.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.892+0000 7fa1c7e00640 1 -- 172.21.15.2:0/3277476575 learned_addr learned my addr 172.21.15.2:0/3277476575 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:04:33.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.892+0000 7fa1c7400640 1 --2- 172.21.15.2:0/3277476575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c8103ff0 0x7fa1c819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:33.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.892+0000 7fa1bea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1c81049a0 0x7fa1c819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48316/0 (socket says 172.21.15.2:48316) 2024-09-10T08:04:33.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.892+0000 7fa1c7e00640 1 -- 172.21.15.2:0/3277476575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c8103ff0 msgr2=0x7fa1c819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:33.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.892+0000 7fa1c7e00640 1 --2- 172.21.15.2:0/3277476575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c8103ff0 0x7fa1c819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:33.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.892+0000 7fa1c7e00640 1 -- 172.21.15.2:0/3277476575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1c81049a0 msgr2=0x7fa1c819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:33.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.892+0000 7fa1c7e00640 1 --2- 172.21.15.2:0/3277476575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1c81049a0 0x7fa1c819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:33.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.892+0000 7fa1c7e00640 1 -- 172.21.15.2:0/3277476575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c819f5a0 con 0x7fa1c8109120 2024-09-10T08:04:33.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.892+0000 7fa1c7e00640 1 --2- 172.21.15.2:0/3277476575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1c8109120 0x7fa1c819eec0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000eb10 tx=0x7fa1c000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:33.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.893+0000 7fa1c4c00640 1 -- 172.21.15.2:0/3277476575 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0004b20 con 0x7fa1c8109120 2024-09-10T08:04:33.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.893+0000 7fa1ce000640 1 -- 172.21.15.2:0/3277476575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c819f890 con 0x7fa1c8109120 2024-09-10T08:04:33.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.893+0000 7fa1c4c00640 1 -- 172.21.15.2:0/3277476575 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c0004cc0 con 0x7fa1c8109120 2024-09-10T08:04:33.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.893+0000 7fa1c4c00640 1 -- 172.21.15.2:0/3277476575 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0010760 con 0x7fa1c8109120 2024-09-10T08:04:33.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.893+0000 7fa1ce000640 1 -- 172.21.15.2:0/3277476575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c81a7170 con 0x7fa1c8109120 2024-09-10T08:04:33.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.894+0000 7fa1ce000640 1 -- 172.21.15.2:0/3277476575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c81043d0 con 0x7fa1c8109120 2024-09-10T08:04:33.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.896+0000 7fa1c4c00640 1 -- 172.21.15.2:0/3277476575 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa1c0045bd0 con 0x7fa1c8109120 2024-09-10T08:04:33.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.897+0000 7fa1c4c00640 1 --2- 172.21.15.2:0/3277476575 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1ac0a2b80 0x7fa1ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:33.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.897+0000 7fa1c4c00640 1 -- 172.21.15.2:0/3277476575 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fa1c00cf2a0 con 0x7fa1c8109120 2024-09-10T08:04:33.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.897+0000 7fa1c7400640 1 --2- 172.21.15.2:0/3277476575 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1ac0a2b80 0x7fa1ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:33.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.898+0000 7fa1c7400640 1 --2- 172.21.15.2:0/3277476575 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1ac0a2b80 0x7fa1ac0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009230 tx=0x7fa1b80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:33.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:33.899+0000 7fa1c4c00640 1 -- 172.21.15.2:0/3277476575 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1c009c600 con 0x7fa1c8109120 2024-09-10T08:04:34.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.082+0000 7fa1ce000640 1 -- 172.21.15.2:0/3277476575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1c819b950 con 0x7fa1c8109120 2024-09-10T08:04:34.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.083+0000 7fa1c4c00640 1 -- 172.21.15.2:0/3277476575 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fa1c009bf00 con 0x7fa1c8109120 2024-09-10T08:04:34.083 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-09-10T08:04:34.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.085+0000 7fa1ce000640 1 -- 172.21.15.2:0/3277476575 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1ac0a2b80 msgr2=0x7fa1ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:34.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.085+0000 7fa1ce000640 1 --2- 172.21.15.2:0/3277476575 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1ac0a2b80 0x7fa1ac0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009230 tx=0x7fa1b80058d0 comp rx=0 tx=0).stop 2024-09-10T08:04:34.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.085+0000 7fa1ce000640 1 -- 172.21.15.2:0/3277476575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1c8109120 msgr2=0x7fa1c819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:34.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.085+0000 7fa1ce000640 1 --2- 172.21.15.2:0/3277476575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1c8109120 0x7fa1c819eec0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000eb10 tx=0x7fa1c000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:04:34.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.085+0000 7fa1ce000640 1 -- 172.21.15.2:0/3277476575 shutdown_connections 2024-09-10T08:04:34.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.085+0000 7fa1ce000640 1 --2- 172.21.15.2:0/3277476575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1c8109120 0x7fa1c819eec0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:34.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.085+0000 7fa1ce000640 1 --2- 172.21.15.2:0/3277476575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1c81049a0 0x7fa1c819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:34.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.085+0000 7fa1ce000640 1 --2- 172.21.15.2:0/3277476575 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1ac0a2b80 0x7fa1ac0a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:34.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.085+0000 7fa1ce000640 1 --2- 172.21.15.2:0/3277476575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c8103ff0 0x7fa1c819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:34.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.085+0000 7fa1ce000640 1 -- 172.21.15.2:0/3277476575 >> 172.21.15.2:0/3277476575 conn(0x7fa1c80ffa90 msgr2=0x7fa1c8100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:34.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.086+0000 7fa1ce000640 1 -- 172.21.15.2:0/3277476575 shutdown_connections 2024-09-10T08:04:34.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:34.086+0000 7fa1ce000640 1 -- 172.21.15.2:0/3277476575 wait complete. 2024-09-10T08:04:34.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:34 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3277476575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:04:34.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:34 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3277476575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:04:34.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:34 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3277476575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:04:35.109 INFO:tasks.cephadm:Adding mds.a on smithi002 2024-09-10T08:04:35.109 INFO:tasks.cephadm:Adding mds.d on smithi002 2024-09-10T08:04:35.109 INFO:tasks.cephadm:Adding mds.g on smithi002 2024-09-10T08:04:35.109 INFO:tasks.cephadm:Adding mds.j on smithi002 2024-09-10T08:04:35.110 INFO:tasks.cephadm:Adding mds.b on smithi107 2024-09-10T08:04:35.110 INFO:tasks.cephadm:Adding mds.e on smithi107 2024-09-10T08:04:35.110 INFO:tasks.cephadm:Adding mds.h on smithi107 2024-09-10T08:04:35.110 INFO:tasks.cephadm:Adding mds.k on smithi107 2024-09-10T08:04:35.110 INFO:tasks.cephadm:Adding mds.c on smithi155 2024-09-10T08:04:35.110 INFO:tasks.cephadm:Adding mds.f on smithi155 2024-09-10T08:04:35.110 INFO:tasks.cephadm:Adding mds.i on smithi155 2024-09-10T08:04:35.110 INFO:tasks.cephadm:Adding mds.l on smithi155 2024-09-10T08:04:35.110 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l' 2024-09-10T08:04:35.371 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:04:35.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:35 smithi155 ceph-mon[35405]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:35.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:35 smithi002 ceph-mon[30563]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:35.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:35 smithi107 ceph-mon[35048]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:35.861 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.847+0000 7f86b6df2640 1 -- 172.21.15.155:0/359092870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b0100b10 msgr2=0x7f86b0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:35.861 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.847+0000 7f86b6df2640 1 --2- 172.21.15.155:0/359092870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b0100b10 0x7f86b0100f10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009900 tx=0x7f86a402a410 comp rx=0 tx=0).stop 2024-09-10T08:04:35.862 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.847+0000 7f86b6df2640 1 -- 172.21.15.155:0/359092870 shutdown_connections 2024-09-10T08:04:35.862 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.847+0000 7f86b6df2640 1 --2- 172.21.15.155:0/359092870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86b01026d0 0x7f86b0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:35.862 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.847+0000 7f86b6df2640 1 --2- 172.21.15.155:0/359092870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86b0101d10 0x7f86b0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:35.862 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.847+0000 7f86b6df2640 1 --2- 172.21.15.155:0/359092870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b0100b10 0x7f86b0100f10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:35.862 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.847+0000 7f86b6df2640 1 -- 172.21.15.155:0/359092870 >> 172.21.15.155:0/359092870 conn(0x7f86b00fc2a0 msgr2=0x7f86b00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:35.862 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.847+0000 7f86b6df2640 1 -- 172.21.15.155:0/359092870 shutdown_connections 2024-09-10T08:04:35.862 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.847+0000 7f86b6df2640 1 -- 172.21.15.155:0/359092870 wait complete. 2024-09-10T08:04:35.862 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86b6df2640 1 Processor -- start 2024-09-10T08:04:35.862 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86b6df2640 1 -- start start 2024-09-10T08:04:35.862 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86b6df2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86b0100b10 0x7f86b0198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:35.862 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86b6df2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b0101d10 0x7f86b0198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:35.862 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86b6df2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86b01026d0 0x7f86b019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86b6df2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f86b010c870 con 0x7f86b0100b10 2024-09-10T08:04:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86b6df2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f86b010c6f0 con 0x7f86b01026d0 2024-09-10T08:04:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86b6df2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f86b010c9f0 con 0x7f86b0101d10 2024-09-10T08:04:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86affff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b0101d10 0x7f86b0198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86affff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b0101d10 0x7f86b0198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:36214/0 (socket says 172.21.15.155:36214) 2024-09-10T08:04:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86affff640 1 -- 172.21.15.155:0/4104743193 learned_addr learned my addr 172.21.15.155:0/4104743193 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-09-10T08:04:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86b4b8e640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86b0100b10 0x7f86b0198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86affff640 1 -- 172.21.15.155:0/4104743193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86b01026d0 msgr2=0x7f86b019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86affff640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86b01026d0 0x7f86b019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86affff640 1 -- 172.21.15.155:0/4104743193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86b0100b10 msgr2=0x7f86b0198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86affff640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86b0100b10 0x7f86b0198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86affff640 1 -- 172.21.15.155:0/4104743193 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b019fd50 con 0x7f86b0101d10 2024-09-10T08:04:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86b4b8e640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86b0100b10 0x7f86b0198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:04:35.864 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86affff640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b0101d10 0x7f86b0198660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f869c00b4a0 tx=0x7f869c00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:35.864 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.848+0000 7f86adffb640 1 -- 172.21.15.155:0/4104743193 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c014070 con 0x7f86b0101d10 2024-09-10T08:04:35.864 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.849+0000 7f86adffb640 1 -- 172.21.15.155:0/4104743193 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869c010af0 con 0x7f86b0101d10 2024-09-10T08:04:35.864 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.849+0000 7f86b6df2640 1 -- 172.21.15.155:0/4104743193 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b019ffe0 con 0x7f86b0101d10 2024-09-10T08:04:35.864 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.849+0000 7f86adffb640 1 -- 172.21.15.155:0/4104743193 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c017d10 con 0x7f86b0101d10 2024-09-10T08:04:35.864 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.849+0000 7f86b6df2640 1 -- 172.21.15.155:0/4104743193 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b01a0520 con 0x7f86b0101d10 2024-09-10T08:04:35.864 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.850+0000 7f86b6df2640 1 -- 172.21.15.155:0/4104743193 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b01a07b0 con 0x7f86b0101d10 2024-09-10T08:04:35.864 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.851+0000 7f86adffb640 1 -- 172.21.15.155:0/4104743193 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f869c017070 con 0x7f86b0101d10 2024-09-10T08:04:35.864 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.851+0000 7f86adffb640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f86a00a2bb0 0x7f86a00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:35.864 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.851+0000 7f86adffb640 1 -- 172.21.15.155:0/4104743193 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f869c0cdfe0 con 0x7f86b0101d10 2024-09-10T08:04:35.864 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.852+0000 7f86b4b8e640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f86a00a2bb0 0x7f86a00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:35.864 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.852+0000 7f86b4b8e640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f86a00a2bb0 0x7f86a00a5070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f86a40096c0 tx=0x7f86a4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:35.865 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:35.856+0000 7f86adffb640 1 -- 172.21.15.155:0/4104743193 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f869c09b3e0 con 0x7f86b0101d10 2024-09-10T08:04:36.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.042+0000 7f86b6df2640 1 -- 172.21.15.155:0/4104743193 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}) -- 0x7f86b006af30 con 0x7f86a00a2bb0 2024-09-10T08:04:36.051 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.051+0000 7f86adffb640 1 -- 172.21.15.155:0/4104743193 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f86b006af30 con 0x7f86a00a2bb0 2024-09-10T08:04:36.052 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mds.all update... 2024-09-10T08:04:36.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.053+0000 7f86b6df2640 1 -- 172.21.15.155:0/4104743193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f86a00a2bb0 msgr2=0x7f86a00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:36.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.053+0000 7f86b6df2640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f86a00a2bb0 0x7f86a00a5070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f86a40096c0 tx=0x7f86a4031400 comp rx=0 tx=0).stop 2024-09-10T08:04:36.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.053+0000 7f86b6df2640 1 -- 172.21.15.155:0/4104743193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b0101d10 msgr2=0x7f86b0198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:36.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.053+0000 7f86b6df2640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b0101d10 0x7f86b0198660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f869c00b4a0 tx=0x7f869c00b960 comp rx=0 tx=0).stop 2024-09-10T08:04:36.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.053+0000 7f86b6df2640 1 -- 172.21.15.155:0/4104743193 shutdown_connections 2024-09-10T08:04:36.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.053+0000 7f86b6df2640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86b01026d0 0x7f86b019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:36.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.053+0000 7f86b6df2640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f86a00a2bb0 0x7f86a00a5070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:36.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.053+0000 7f86b6df2640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b0101d10 0x7f86b0198660 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:36.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.053+0000 7f86b6df2640 1 --2- 172.21.15.155:0/4104743193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86b0100b10 0x7f86b0198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:36.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.053+0000 7f86b6df2640 1 -- 172.21.15.155:0/4104743193 >> 172.21.15.155:0/4104743193 conn(0x7f86b00fc2a0 msgr2=0x7f86b006a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:36.055 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.053+0000 7f86b6df2640 1 -- 172.21.15.155:0/4104743193 shutdown_connections 2024-09-10T08:04:36.055 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-10T08:04:36.054+0000 7f86b6df2640 1 -- 172.21.15.155:0/4104743193 wait complete. 2024-09-10T08:04:36.830 DEBUG:teuthology.orchestra.run.smithi002:mds.a> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T08:04:36.835 DEBUG:teuthology.orchestra.run.smithi002:mds.d> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T08:04:36.881 DEBUG:teuthology.orchestra.run.smithi002:mds.g> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T08:04:36.926 DEBUG:teuthology.orchestra.run.smithi002:mds.j> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T08:04:36.973 DEBUG:teuthology.orchestra.run.smithi107:mds.b> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T08:04:36.976 DEBUG:teuthology.orchestra.run.smithi107:mds.e> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T08:04:37.019 DEBUG:teuthology.orchestra.run.smithi107:mds.h> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T08:04:37.062 DEBUG:teuthology.orchestra.run.smithi107:mds.k> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T08:04:37.105 DEBUG:teuthology.orchestra.run.smithi155:mds.c> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T08:04:37.108 DEBUG:teuthology.orchestra.run.smithi155:mds.f> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T08:04:37.150 DEBUG:teuthology.orchestra.run.smithi155:mds.i> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T08:04:37.193 DEBUG:teuthology.orchestra.run.smithi155:mds.l> sudo journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T08:04:37.236 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T08:04:37.236 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:04:37.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:37 smithi002 ceph-mon[30563]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:37.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:37 smithi002 ceph-mon[30563]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:04:37.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:37 smithi002 ceph-mon[30563]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-09-10T08:04:37.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:37 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:37.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:37 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:04:37.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:37 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:37.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:37 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:04:37.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:37 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:37.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:37 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:37.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:37 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:37.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:37 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:37.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:37 smithi002 ceph-mon[30563]: Deploying daemon mds.l on smithi155 2024-09-10T08:04:37.386 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:37 smithi155 ceph-mon[35405]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:37.387 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:37 smithi155 ceph-mon[35405]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:04:37.387 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:37 smithi155 ceph-mon[35405]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-09-10T08:04:37.387 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:37 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:37.387 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:37 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:04:37.387 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:37 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:37.387 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:37 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:04:37.387 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:37 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:37.387 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:37 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:37.387 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:37 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:37.387 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:37 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:37.387 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:37 smithi155 ceph-mon[35405]: Deploying daemon mds.l on smithi155 2024-09-10T08:04:37.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:37 smithi107 ceph-mon[35048]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:37.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:37 smithi107 ceph-mon[35048]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:04:37.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:37 smithi107 ceph-mon[35048]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-09-10T08:04:37.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:37 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:37.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:37 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:04:37.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:37 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:37.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:37 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:04:37.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:37 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:37.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:37 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:37.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:37 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:37.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:37 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:37.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:37 smithi107 ceph-mon[35048]: Deploying daemon mds.l on smithi155 2024-09-10T08:04:37.607 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:04:38.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.179+0000 7f417a200640 1 -- 172.21.15.2:0/1175432262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4174102790 msgr2=0x7f41741067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:38.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.179+0000 7f417a200640 1 --2- 172.21.15.2:0/1175432262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4174102790 0x7f41741067e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4160009900 tx=0x7f416002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:04:38.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.179+0000 7f417a200640 1 -- 172.21.15.2:0/1175432262 shutdown_connections 2024-09-10T08:04:38.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.179+0000 7f417a200640 1 --2- 172.21.15.2:0/1175432262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174106f10 0x7f417410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:38.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.179+0000 7f417a200640 1 --2- 172.21.15.2:0/1175432262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4174102790 0x7f41741067e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:38.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.179+0000 7f417a200640 1 --2- 172.21.15.2:0/1175432262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4174101de0 0x7f41741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:38.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.179+0000 7f417a200640 1 -- 172.21.15.2:0/1175432262 >> 172.21.15.2:0/1175432262 conn(0x7f41740fd8c0 msgr2=0x7f41740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:38.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.179+0000 7f417a200640 1 -- 172.21.15.2:0/1175432262 shutdown_connections 2024-09-10T08:04:38.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.179+0000 7f417a200640 1 -- 172.21.15.2:0/1175432262 wait complete. 2024-09-10T08:04:38.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.180+0000 7f417a200640 1 Processor -- start 2024-09-10T08:04:38.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.180+0000 7f417a200640 1 -- start start 2024-09-10T08:04:38.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.181+0000 7f417a200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174101de0 0x7f41740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:38.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.181+0000 7f417a200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4174102790 0x7f4174074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:38.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.181+0000 7f417a200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4174106f10 0x7f4174074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:38.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.181+0000 7f417a200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4174111f70 con 0x7f4174106f10 2024-09-10T08:04:38.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.181+0000 7f417a200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4174111df0 con 0x7f4174102790 2024-09-10T08:04:38.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.181+0000 7f417a200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41741120f0 con 0x7f4174101de0 2024-09-10T08:04:38.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.181+0000 7f4173400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174101de0 0x7f41740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:38.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.181+0000 7f4173e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4174106f10 0x7f4174074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:38.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.181+0000 7f4173400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174101de0 0x7f41740744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49866/0 (socket says 172.21.15.2:49866) 2024-09-10T08:04:38.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.181+0000 7f4173400640 1 -- 172.21.15.2:0/1102335093 learned_addr learned my addr 172.21.15.2:0/1102335093 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:04:38.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.181+0000 7f4172a00640 1 --2- 172.21.15.2:0/1102335093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4174102790 0x7f4174074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:38.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.181+0000 7f4173400640 1 -- 172.21.15.2:0/1102335093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4174102790 msgr2=0x7f4174074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:38.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.182+0000 7f4173400640 1 --2- 172.21.15.2:0/1102335093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4174102790 0x7f4174074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:38.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.182+0000 7f4173400640 1 -- 172.21.15.2:0/1102335093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4174106f10 msgr2=0x7f4174074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:38.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.182+0000 7f4173400640 1 --2- 172.21.15.2:0/1102335093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4174106f10 0x7f4174074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:38.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.182+0000 7f4173400640 1 -- 172.21.15.2:0/1102335093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41741aada0 con 0x7f4174101de0 2024-09-10T08:04:38.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.182+0000 7f4173400640 1 --2- 172.21.15.2:0/1102335093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174101de0 0x7f41740744f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f416400e9f0 tx=0x7f416400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:38.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.182+0000 7f416be00640 1 -- 172.21.15.2:0/1102335093 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416400cd80 con 0x7f4174101de0 2024-09-10T08:04:38.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.182+0000 7f416be00640 1 -- 172.21.15.2:0/1102335093 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4164010910 con 0x7f4174101de0 2024-09-10T08:04:38.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.182+0000 7f416be00640 1 -- 172.21.15.2:0/1102335093 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164018990 con 0x7f4174101de0 2024-09-10T08:04:38.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.182+0000 7f417a200640 1 -- 172.21.15.2:0/1102335093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41741aafd0 con 0x7f4174101de0 2024-09-10T08:04:38.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.182+0000 7f417a200640 1 -- 172.21.15.2:0/1102335093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41741ab430 con 0x7f4174101de0 2024-09-10T08:04:38.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.183+0000 7f417a200640 1 -- 172.21.15.2:0/1102335093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41741032c0 con 0x7f4174101de0 2024-09-10T08:04:38.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.185+0000 7f416be00640 1 -- 172.21.15.2:0/1102335093 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f4164010ac0 con 0x7f4174101de0 2024-09-10T08:04:38.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.186+0000 7f416be00640 1 --2- 172.21.15.2:0/1102335093 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41580a2b80 0x7f41580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:38.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.186+0000 7f416be00640 1 -- 172.21.15.2:0/1102335093 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f41640ceed0 con 0x7f4174101de0 2024-09-10T08:04:38.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.187+0000 7f4172a00640 1 --2- 172.21.15.2:0/1102335093 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41580a2b80 0x7f41580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:38.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.187+0000 7f4172a00640 1 --2- 172.21.15.2:0/1102335093 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41580a2b80 0x7f41580a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f417419c530 tx=0x7f41600058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:38.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.190+0000 7f416be00640 1 -- 172.21.15.2:0/1102335093 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f416409bbe0 con 0x7f4174101de0 2024-09-10T08:04:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.412+0000 7f417a200640 1 -- 172.21.15.2:0/1102335093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41741021c0 con 0x7f4174101de0 2024-09-10T08:04:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.413+0000 7f416be00640 1 -- 172.21.15.2:0/1102335093 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f41741021c0 con 0x7f4174101de0 2024-09-10T08:04:38.413 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:04:38.413 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":1,"btime":"2024-09-10T07:57:11:261417+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T08:04:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 1 2024-09-10T08:04:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.415+0000 7f417a200640 1 -- 172.21.15.2:0/1102335093 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41580a2b80 msgr2=0x7f41580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.415+0000 7f417a200640 1 --2- 172.21.15.2:0/1102335093 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41580a2b80 0x7f41580a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f417419c530 tx=0x7f41600058d0 comp rx=0 tx=0).stop 2024-09-10T08:04:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.415+0000 7f417a200640 1 -- 172.21.15.2:0/1102335093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174101de0 msgr2=0x7f41740744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.415+0000 7f417a200640 1 --2- 172.21.15.2:0/1102335093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174101de0 0x7f41740744f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f416400e9f0 tx=0x7f416400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:04:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.415+0000 7f417a200640 1 -- 172.21.15.2:0/1102335093 shutdown_connections 2024-09-10T08:04:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.415+0000 7f417a200640 1 --2- 172.21.15.2:0/1102335093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4174106f10 0x7f4174074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.415+0000 7f417a200640 1 --2- 172.21.15.2:0/1102335093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4174102790 0x7f4174074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.415+0000 7f417a200640 1 --2- 172.21.15.2:0/1102335093 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41580a2b80 0x7f41580a5040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.415+0000 7f417a200640 1 --2- 172.21.15.2:0/1102335093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174101de0 0x7f41740744f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.415+0000 7f417a200640 1 -- 172.21.15.2:0/1102335093 >> 172.21.15.2:0/1102335093 conn(0x7f41740fd8c0 msgr2=0x7f41740fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.415+0000 7f417a200640 1 -- 172.21.15.2:0/1102335093 shutdown_connections 2024-09-10T08:04:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:38.416+0000 7f417a200640 1 -- 172.21.15.2:0/1102335093 wait complete. 2024-09-10T08:04:38.656 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:04:38 smithi155 systemd[1]: Starting Ceph mds.l for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:04:38.970 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:04:38 smithi155 podman[64510]: 2024-09-10 08:04:38.592051476 +0000 UTC m=+0.015580063 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:04:38.970 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:04:38 smithi155 podman[64510]: 2024-09-10 08:04:38.702456027 +0000 UTC m=+0.125984653 container create 23f2a8c101e06a22a8ddfeb18c36275c3d849c119731fac590f68180e57a7963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:04:38.970 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:04:38 smithi155 podman[64510]: 2024-09-10 08:04:38.971539206 +0000 UTC m=+0.395067800 container init 23f2a8c101e06a22a8ddfeb18c36275c3d849c119731fac590f68180e57a7963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T08:04:39.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:39 smithi155 ceph-mon[35405]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:39.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:39 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1102335093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:39.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:39 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.155:6832/1276815741,v1:172.21.15.155:6833/1276815741] up:boot 2024-09-10T08:04:39.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:39 smithi155 ceph-mon[35405]: fsmap 1 up:standby 2024-09-10T08:04:39.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T08:04:39.407 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:04:38 smithi155 podman[64510]: 2024-09-10 08:04:38.974516927 +0000 UTC m=+0.398045518 container start 23f2a8c101e06a22a8ddfeb18c36275c3d849c119731fac590f68180e57a7963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T08:04:39.407 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:04:38 smithi155 ceph-mds[64529]: -- 172.21.15.155:0/3562099182 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe4d239860 con 0x55fe4d217000 2024-09-10T08:04:39.407 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:04:38 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-l[64525]: starting mds.l at 2024-09-10T08:04:39.407 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:04:39 smithi155 bash[64510]: 23f2a8c101e06a22a8ddfeb18c36275c3d849c119731fac590f68180e57a7963 2024-09-10T08:04:39.407 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:04:39 smithi155 systemd[1]: Started Ceph mds.l for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:04:39.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:39 smithi107 ceph-mon[35048]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:39.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:39 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1102335093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:39.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:39 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.155:6832/1276815741,v1:172.21.15.155:6833/1276815741] up:boot 2024-09-10T08:04:39.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:39 smithi107 ceph-mon[35048]: fsmap 1 up:standby 2024-09-10T08:04:39.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T08:04:39.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:39 smithi002 ceph-mon[30563]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:39.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:39 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1102335093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:39.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:39 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.155:6832/1276815741,v1:172.21.15.155:6833/1276815741] up:boot 2024-09-10T08:04:39.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:39 smithi002 ceph-mon[30563]: fsmap 1 up:standby 2024-09-10T08:04:39.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T08:04:40.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:40.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:40.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:40.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:40.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:40.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:40.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:40 smithi155 ceph-mon[35405]: Deploying daemon mds.k on smithi107 2024-09-10T08:04:40.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:40.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:40.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:40.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:40.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:40.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:40.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:40 smithi107 ceph-mon[35048]: Deploying daemon mds.k on smithi107 2024-09-10T08:04:40.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:40.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:40.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:40.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:40.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:40.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:40.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:40 smithi002 ceph-mon[30563]: Deploying daemon mds.k on smithi107 2024-09-10T08:04:41.165 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:04:41.379 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:41 smithi107 ceph-mon[35048]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:41.379 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:04:41 smithi107 systemd[1]: Starting Ceph mds.k for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:04:41.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:41 smithi155 ceph-mon[35405]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:41.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:41 smithi002 ceph-mon[30563]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:41.549 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:04:41.652 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:04:41 smithi107 podman[63563]: 2024-09-10 08:04:41.282218339 +0000 UTC m=+0.015639663 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:04:41.652 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:04:41 smithi107 podman[63563]: 2024-09-10 08:04:41.389813357 +0000 UTC m=+0.123234672 container create 0740ab8b90aeaa9ba04163831fbe04bbe0a4a45c547d2ff6dd19bf72f901c7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T08:04:41.652 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:04:41 smithi107 podman[63563]: 2024-09-10 08:04:41.6257665 +0000 UTC m=+0.359187814 container init 0740ab8b90aeaa9ba04163831fbe04bbe0a4a45c547d2ff6dd19bf72f901c7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T08:04:41.652 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:04:41 smithi107 podman[63563]: 2024-09-10 08:04:41.628600444 +0000 UTC m=+0.362021771 container start 0740ab8b90aeaa9ba04163831fbe04bbe0a4a45c547d2ff6dd19bf72f901c7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T08:04:41.917 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:04:41 smithi107 ceph-mds[63582]: -- 172.21.15.107:0/2132892700 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f45845860 con 0x560f45823000 2024-09-10T08:04:41.917 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:04:41 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-k[63578]: starting mds.k at 2024-09-10T08:04:41.917 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:04:41 smithi107 bash[63563]: 0740ab8b90aeaa9ba04163831fbe04bbe0a4a45c547d2ff6dd19bf72f901c7a4 2024-09-10T08:04:41.917 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:04:41 smithi107 systemd[1]: Started Ceph mds.k for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:04:42.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.098+0000 7f5cfd400640 1 -- 172.21.15.2:0/476100172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf81079c0 msgr2=0x7f5cf8107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:42.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.098+0000 7f5cfd400640 1 --2- 172.21.15.2:0/476100172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf81079c0 0x7f5cf8107da0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009860 tx=0x7f5ce802a340 comp rx=0 tx=0).stop 2024-09-10T08:04:42.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.098+0000 7f5cfd400640 1 -- 172.21.15.2:0/476100172 shutdown_connections 2024-09-10T08:04:42.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.098+0000 7f5cfd400640 1 --2- 172.21.15.2:0/476100172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf8102360 0x7f5cf810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:42.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.098+0000 7f5cfd400640 1 --2- 172.21.15.2:0/476100172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf81019c0 0x7f5cf8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:42.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.098+0000 7f5cfd400640 1 --2- 172.21.15.2:0/476100172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf81079c0 0x7f5cf8107da0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:42.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.098+0000 7f5cfd400640 1 -- 172.21.15.2:0/476100172 >> 172.21.15.2:0/476100172 conn(0x7f5cf80fd740 msgr2=0x7f5cf80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:42.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.098+0000 7f5cfd400640 1 -- 172.21.15.2:0/476100172 shutdown_connections 2024-09-10T08:04:42.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.098+0000 7f5cfd400640 1 -- 172.21.15.2:0/476100172 wait complete. 2024-09-10T08:04:42.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.100+0000 7f5cfd400640 1 Processor -- start 2024-09-10T08:04:42.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.100+0000 7f5cfd400640 1 -- start start 2024-09-10T08:04:42.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.100+0000 7f5cfd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf81019c0 0x7f5cf819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:42.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.100+0000 7f5cfd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf8102360 0x7f5cf819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:42.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cfd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf81079c0 0x7f5cf819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:42.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cfd400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8111dc0 con 0x7f5cf8102360 2024-09-10T08:04:42.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cfd400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8111c40 con 0x7f5cf81079c0 2024-09-10T08:04:42.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cfd400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8111f40 con 0x7f5cf81019c0 2024-09-10T08:04:42.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cf7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf81019c0 0x7f5cf819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:42.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cf7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf81019c0 0x7f5cf819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49898/0 (socket says 172.21.15.2:49898) 2024-09-10T08:04:42.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cf7e00640 1 -- 172.21.15.2:0/3279402748 learned_addr learned my addr 172.21.15.2:0/3279402748 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:04:42.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cf7400640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf8102360 0x7f5cf819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:42.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cfca00640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf81079c0 0x7f5cf819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:42.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cf7e00640 1 -- 172.21.15.2:0/3279402748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf81079c0 msgr2=0x7f5cf819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:42.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cf7e00640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf81079c0 0x7f5cf819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:42.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cf7e00640 1 -- 172.21.15.2:0/3279402748 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf8102360 msgr2=0x7f5cf819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:42.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cf7e00640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf8102360 0x7f5cf819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:42.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.101+0000 7f5cf7e00640 1 -- 172.21.15.2:0/3279402748 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf819f480 con 0x7f5cf81019c0 2024-09-10T08:04:42.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.102+0000 7f5cf7e00640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf81019c0 0x7f5cf819a4d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8035040 tx=0x7f5ce8033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:42.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.102+0000 7f5cf4c00640 1 -- 172.21.15.2:0/3279402748 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8037040 con 0x7f5cf81019c0 2024-09-10T08:04:42.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.102+0000 7f5cf4c00640 1 -- 172.21.15.2:0/3279402748 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ce8040070 con 0x7f5cf81019c0 2024-09-10T08:04:42.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.102+0000 7f5cf4c00640 1 -- 172.21.15.2:0/3279402748 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8003a10 con 0x7f5cf81019c0 2024-09-10T08:04:42.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.102+0000 7f5cfd400640 1 -- 172.21.15.2:0/3279402748 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf819f710 con 0x7f5cf81019c0 2024-09-10T08:04:42.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.102+0000 7f5cfd400640 1 -- 172.21.15.2:0/3279402748 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf81a7050 con 0x7f5cf81019c0 2024-09-10T08:04:42.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.103+0000 7f5cfd400640 1 -- 172.21.15.2:0/3279402748 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf8101e20 con 0x7f5cf81019c0 2024-09-10T08:04:42.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.107+0000 7f5cf4c00640 1 -- 172.21.15.2:0/3279402748 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5ce8004150 con 0x7f5cf81019c0 2024-09-10T08:04:42.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.108+0000 7f5cf4c00640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5cdc0a2c00 0x7f5cdc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:42.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.108+0000 7f5cf4c00640 1 -- 172.21.15.2:0/3279402748 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f5ce80ef200 con 0x7f5cf81019c0 2024-09-10T08:04:42.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.108+0000 7f5cf7400640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5cdc0a2c00 0x7f5cdc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:42.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.109+0000 7f5cf4c00640 1 -- 172.21.15.2:0/3279402748 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ce80bc560 con 0x7f5cf81019c0 2024-09-10T08:04:42.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.109+0000 7f5cf7400640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5cdc0a2c00 0x7f5cdc0a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5cf810e610 tx=0x7f5ce4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:42.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.354+0000 7f5cfd400640 1 -- 172.21.15.2:0/3279402748 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cf81a7220 con 0x7f5cf81019c0 2024-09-10T08:04:42.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.354+0000 7f5cf4c00640 1 -- 172.21.15.2:0/3279402748 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7f5ce80bbe60 con 0x7f5cf81019c0 2024-09-10T08:04:42.354 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:04:42.355 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":3,"btime":"2024-09-10T08:04:42:147162+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T08:04:42.355 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 3 2024-09-10T08:04:42.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.357+0000 7f5cfd400640 1 -- 172.21.15.2:0/3279402748 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5cdc0a2c00 msgr2=0x7f5cdc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.357+0000 7f5cfd400640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5cdc0a2c00 0x7f5cdc0a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5cf810e610 tx=0x7f5ce4005f90 comp rx=0 tx=0).stop 2024-09-10T08:04:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.357+0000 7f5cfd400640 1 -- 172.21.15.2:0/3279402748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf81019c0 msgr2=0x7f5cf819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.357+0000 7f5cfd400640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf81019c0 0x7f5cf819a4d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8035040 tx=0x7f5ce8033b60 comp rx=0 tx=0).stop 2024-09-10T08:04:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.357+0000 7f5cfd400640 1 -- 172.21.15.2:0/3279402748 shutdown_connections 2024-09-10T08:04:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.357+0000 7f5cfd400640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf81079c0 0x7f5cf819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.357+0000 7f5cfd400640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cf8102360 0x7f5cf819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.357+0000 7f5cfd400640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5cdc0a2c00 0x7f5cdc0a50c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.357+0000 7f5cfd400640 1 --2- 172.21.15.2:0/3279402748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf81019c0 0x7f5cf819a4d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.357+0000 7f5cfd400640 1 -- 172.21.15.2:0/3279402748 >> 172.21.15.2:0/3279402748 conn(0x7f5cf80fd740 msgr2=0x7f5cf80fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.357+0000 7f5cfd400640 1 -- 172.21.15.2:0/3279402748 shutdown_connections 2024-09-10T08:04:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:42.357+0000 7f5cfd400640 1 -- 172.21.15.2:0/3279402748 wait complete. 2024-09-10T08:04:43.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:42 smithi155 ceph-mon[35405]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:43.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:43.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:43.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:43.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:43.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:43.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:43.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:42 smithi155 ceph-mon[35405]: Deploying daemon mds.j on smithi002 2024-09-10T08:04:43.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:42 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.107:6832/3176720601,v1:172.21.15.107:6833/3176720601] up:boot 2024-09-10T08:04:43.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:42 smithi155 ceph-mon[35405]: fsmap 2 up:standby 2024-09-10T08:04:43.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T08:04:43.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:42 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3279402748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:43.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:42 smithi107 ceph-mon[35048]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:43.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:43.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:43.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:43.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:43.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:43.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:43.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:42 smithi107 ceph-mon[35048]: Deploying daemon mds.j on smithi002 2024-09-10T08:04:43.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:42 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.107:6832/3176720601,v1:172.21.15.107:6833/3176720601] up:boot 2024-09-10T08:04:43.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:42 smithi107 ceph-mon[35048]: fsmap 2 up:standby 2024-09-10T08:04:43.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T08:04:43.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:42 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3279402748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:42 smithi002 ceph-mon[30563]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:42 smithi002 ceph-mon[30563]: Deploying daemon mds.j on smithi002 2024-09-10T08:04:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:42 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.107:6832/3176720601,v1:172.21.15.107:6833/3176720601] up:boot 2024-09-10T08:04:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:42 smithi002 ceph-mon[30563]: fsmap 2 up:standby 2024-09-10T08:04:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T08:04:43.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:42 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3279402748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:44.770 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:04:44 smithi002 systemd[1]: Starting Ceph mds.j for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:04:45.108 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:44 smithi002 ceph-mon[30563]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:45.109 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:04:44 smithi002 podman[67331]: 2024-09-10 08:04:44.75381953 +0000 UTC m=+0.027248710 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:04:45.109 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:04:44 smithi002 podman[67331]: 2024-09-10 08:04:44.857239998 +0000 UTC m=+0.130669161 container create c5db02483c0ca20828e3bef2d03cebfc648df72cd97a2bdef9d32293b2d68444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-j, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:04:45.109 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:04:45 smithi002 podman[67331]: 2024-09-10 08:04:45.109807397 +0000 UTC m=+0.383236577 container init c5db02483c0ca20828e3bef2d03cebfc648df72cd97a2bdef9d32293b2d68444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-j, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:04:45.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:44 smithi155 ceph-mon[35405]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:45.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:44 smithi107 ceph-mon[35048]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:45.520 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:04:45 smithi002 podman[67331]: 2024-09-10 08:04:45.116397329 +0000 UTC m=+0.389826492 container start c5db02483c0ca20828e3bef2d03cebfc648df72cd97a2bdef9d32293b2d68444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T08:04:45.520 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:04:45 smithi002 ceph-mds[67350]: -- 172.21.15.2:0/4007939459 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562b55a03c20 con 0x562b559e0800 2024-09-10T08:04:45.520 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:04:45 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-j[67346]: starting mds.j at 2024-09-10T08:04:45.520 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:04:45 smithi002 bash[67331]: c5db02483c0ca20828e3bef2d03cebfc648df72cd97a2bdef9d32293b2d68444 2024-09-10T08:04:45.520 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:04:45 smithi002 systemd[1]: Started Ceph mds.j for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:04:46.413 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:04:46.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:46.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:46 smithi155 ceph-mon[35405]: Deploying daemon mds.d on smithi002 2024-09-10T08:04:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:46 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.2:6834/1979555440,v1:172.21.15.2:6835/1979555440] up:boot 2024-09-10T08:04:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:46 smithi155 ceph-mon[35405]: fsmap 3 up:standby 2024-09-10T08:04:46.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:46 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T08:04:46.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:46 smithi107 ceph-mon[35048]: Deploying daemon mds.d on smithi002 2024-09-10T08:04:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:46 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.2:6834/1979555440,v1:172.21.15.2:6835/1979555440] up:boot 2024-09-10T08:04:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:46 smithi107 ceph-mon[35048]: fsmap 3 up:standby 2024-09-10T08:04:46.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:46 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T08:04:46.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:46.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:46.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:46.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:46.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:46.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:46.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:46 smithi002 ceph-mon[30563]: Deploying daemon mds.d on smithi002 2024-09-10T08:04:46.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:46 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.2:6834/1979555440,v1:172.21.15.2:6835/1979555440] up:boot 2024-09-10T08:04:46.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:46 smithi002 ceph-mon[30563]: fsmap 3 up:standby 2024-09-10T08:04:46.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:46 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T08:04:47.205 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:04:47.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:47 smithi155 ceph-mon[35405]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:47.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:47 smithi002 ceph-mon[30563]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:47.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:47 smithi107 ceph-mon[35048]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:48.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.045+0000 7faad3200640 1 -- 172.21.15.2:0/1764041475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad4102360 msgr2=0x7faad410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:48.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.045+0000 7faad3200640 1 --2- 172.21.15.2:0/1764041475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad4102360 0x7faad410e890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faacc009cd0 tx=0x7faacc02a8e0 comp rx=0 tx=0).stop 2024-09-10T08:04:48.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.045+0000 7faad3200640 1 -- 172.21.15.2:0/1764041475 shutdown_connections 2024-09-10T08:04:48.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.045+0000 7faad3200640 1 --2- 172.21.15.2:0/1764041475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad4102360 0x7faad410e890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:48.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.045+0000 7faad3200640 1 --2- 172.21.15.2:0/1764041475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faad41019c0 0x7faad4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:48.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.045+0000 7faad3200640 1 --2- 172.21.15.2:0/1764041475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad41079c0 0x7faad4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:48.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.045+0000 7faad3200640 1 -- 172.21.15.2:0/1764041475 >> 172.21.15.2:0/1764041475 conn(0x7faad40fd740 msgr2=0x7faad40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:48.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.045+0000 7faad3200640 1 -- 172.21.15.2:0/1764041475 shutdown_connections 2024-09-10T08:04:48.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.045+0000 7faad3200640 1 -- 172.21.15.2:0/1764041475 wait complete. 2024-09-10T08:04:48.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.046+0000 7faad3200640 1 Processor -- start 2024-09-10T08:04:48.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.047+0000 7faad3200640 1 -- start start 2024-09-10T08:04:48.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.047+0000 7faad3200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad41019c0 0x7faad419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:48.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.047+0000 7faad3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad4102360 0x7faad419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:48.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.047+0000 7faad3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faad41079c0 0x7faad419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:48.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.047+0000 7faad3200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faad4111e10 con 0x7faad41019c0 2024-09-10T08:04:48.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.047+0000 7faad3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faad4111c90 con 0x7faad41079c0 2024-09-10T08:04:48.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.047+0000 7faad3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faad4111f90 con 0x7faad4102360 2024-09-10T08:04:48.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.047+0000 7faacbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad4102360 0x7faad419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:48.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.047+0000 7faad1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad41019c0 0x7faad419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:48.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.047+0000 7faacbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad4102360 0x7faad419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42654/0 (socket says 172.21.15.2:42654) 2024-09-10T08:04:48.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.047+0000 7faacbe00640 1 -- 172.21.15.2:0/1131129965 learned_addr learned my addr 172.21.15.2:0/1131129965 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:04:48.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.048+0000 7faad2800640 1 --2- 172.21.15.2:0/1131129965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faad41079c0 0x7faad419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:48.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.048+0000 7faacbe00640 1 -- 172.21.15.2:0/1131129965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faad41079c0 msgr2=0x7faad419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:48.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.048+0000 7faacbe00640 1 --2- 172.21.15.2:0/1131129965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faad41079c0 0x7faad419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:48.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.048+0000 7faacbe00640 1 -- 172.21.15.2:0/1131129965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad41019c0 msgr2=0x7faad419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:48.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.048+0000 7faacbe00640 1 --2- 172.21.15.2:0/1131129965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad41019c0 0x7faad419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:48.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.048+0000 7faacbe00640 1 -- 172.21.15.2:0/1131129965 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad419f4b0 con 0x7faad4102360 2024-09-10T08:04:48.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.048+0000 7faacbe00640 1 --2- 172.21.15.2:0/1131129965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad4102360 0x7faad419aa40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faabc00e9f0 tx=0x7faabc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:48.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.048+0000 7faacaa00640 1 -- 172.21.15.2:0/1131129965 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc00cd80 con 0x7faad4102360 2024-09-10T08:04:48.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.049+0000 7faad3200640 1 -- 172.21.15.2:0/1131129965 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad419f7a0 con 0x7faad4102360 2024-09-10T08:04:48.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.049+0000 7faad3200640 1 -- 172.21.15.2:0/1131129965 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad41a7080 con 0x7faad4102360 2024-09-10T08:04:48.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.049+0000 7faacaa00640 1 -- 172.21.15.2:0/1131129965 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faabc005500 con 0x7faad4102360 2024-09-10T08:04:48.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.049+0000 7faacaa00640 1 -- 172.21.15.2:0/1131129965 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc010690 con 0x7faad4102360 2024-09-10T08:04:48.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.050+0000 7faad3200640 1 -- 172.21.15.2:0/1131129965 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad4101e20 con 0x7faad4102360 2024-09-10T08:04:48.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.052+0000 7faacaa00640 1 -- 172.21.15.2:0/1131129965 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7faabc0046c0 con 0x7faad4102360 2024-09-10T08:04:48.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.053+0000 7faacaa00640 1 --2- 172.21.15.2:0/1131129965 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faab80a2c00 0x7faab80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:48.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.053+0000 7faacaa00640 1 -- 172.21.15.2:0/1131129965 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7faabc0cefc0 con 0x7faad4102360 2024-09-10T08:04:48.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.054+0000 7faad1e00640 1 --2- 172.21.15.2:0/1131129965 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faab80a2c00 0x7faab80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:48.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.054+0000 7faad1e00640 1 --2- 172.21.15.2:0/1131129965 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faab80a2c00 0x7faab80a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faac4009b50 tx=0x7faac4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:48.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.055+0000 7faacaa00640 1 -- 172.21.15.2:0/1131129965 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faabc09c320 con 0x7faad4102360 2024-09-10T08:04:48.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.318+0000 7faad3200640 1 -- 172.21.15.2:0/1131129965 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faad419b860 con 0x7faad4102360 2024-09-10T08:04:48.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.319+0000 7faacaa00640 1 -- 172.21.15.2:0/1131129965 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7faad419b860 con 0x7faad4102360 2024-09-10T08:04:48.319 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:04:48.319 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":4,"btime":"2024-09-10T08:04:45:778081+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T08:04:48.319 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 4 2024-09-10T08:04:48.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.321+0000 7faad3200640 1 -- 172.21.15.2:0/1131129965 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faab80a2c00 msgr2=0x7faab80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:48.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.321+0000 7faad3200640 1 --2- 172.21.15.2:0/1131129965 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faab80a2c00 0x7faab80a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faac4009b50 tx=0x7faac4005f90 comp rx=0 tx=0).stop 2024-09-10T08:04:48.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.322+0000 7faad3200640 1 -- 172.21.15.2:0/1131129965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad4102360 msgr2=0x7faad419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.322+0000 7faad3200640 1 --2- 172.21.15.2:0/1131129965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad4102360 0x7faad419aa40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faabc00e9f0 tx=0x7faabc00edf0 comp rx=0 tx=0).stop 2024-09-10T08:04:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.322+0000 7faad3200640 1 -- 172.21.15.2:0/1131129965 shutdown_connections 2024-09-10T08:04:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.322+0000 7faad3200640 1 --2- 172.21.15.2:0/1131129965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faad41079c0 0x7faad419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.322+0000 7faad3200640 1 --2- 172.21.15.2:0/1131129965 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faab80a2c00 0x7faab80a50c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.322+0000 7faad3200640 1 --2- 172.21.15.2:0/1131129965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad4102360 0x7faad419aa40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.322+0000 7faad3200640 1 --2- 172.21.15.2:0/1131129965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad41019c0 0x7faad419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.322+0000 7faad3200640 1 -- 172.21.15.2:0/1131129965 >> 172.21.15.2:0/1131129965 conn(0x7faad40fd740 msgr2=0x7faad410ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.322+0000 7faad3200640 1 -- 172.21.15.2:0/1131129965 shutdown_connections 2024-09-10T08:04:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:48.322+0000 7faad3200640 1 -- 172.21.15.2:0/1131129965 wait complete. 2024-09-10T08:04:49.480 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:49 smithi002 ceph-mon[30563]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:49.480 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:49 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1131129965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:49.481 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 10 08:04:49 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-d[67753]: starting mds.d at 2024-09-10T08:04:49.481 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 10 08:04:49 smithi002 bash[67724]: ab11db6cfd54f8e49d332eb25a2e4493ddf0b759ced1b4fa482b615dba4c0000 2024-09-10T08:04:49.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:49 smithi155 ceph-mon[35405]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:49.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:49 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1131129965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:49.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:49 smithi107 ceph-mon[35048]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:49.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:49 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1131129965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:49.770 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 10 08:04:49 smithi002 systemd[1]: Started Ceph mds.d for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:04:50.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:50.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:50.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:50.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:50.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:50.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:50.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:50 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.2:6836/2248723056,v1:172.21.15.2:6837/2248723056] up:boot 2024-09-10T08:04:50.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:50 smithi155 ceph-mon[35405]: fsmap 4 up:standby 2024-09-10T08:04:50.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T08:04:50.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:50.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:50.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:50.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:50.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:50.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:50.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:50 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.2:6836/2248723056,v1:172.21.15.2:6837/2248723056] up:boot 2024-09-10T08:04:50.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:50 smithi107 ceph-mon[35048]: fsmap 4 up:standby 2024-09-10T08:04:50.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T08:04:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:50 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.2:6836/2248723056,v1:172.21.15.2:6837/2248723056] up:boot 2024-09-10T08:04:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:50 smithi002 ceph-mon[30563]: fsmap 4 up:standby 2024-09-10T08:04:51.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T08:04:51.236 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:04:51.624 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:04:51.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:51 smithi107 ceph-mon[35048]: Deploying daemon mds.e on smithi107 2024-09-10T08:04:51.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:51 smithi107 ceph-mon[35048]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:51.667 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:04:51 smithi107 systemd[1]: Starting Ceph mds.e for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:04:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:51 smithi155 ceph-mon[35405]: Deploying daemon mds.e on smithi107 2024-09-10T08:04:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:51 smithi155 ceph-mon[35405]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:51.958 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:51 smithi002 ceph-mon[30563]: Deploying daemon mds.e on smithi107 2024-09-10T08:04:51.958 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:51 smithi002 ceph-mon[30563]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:52.087 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:04:51 smithi107 podman[63876]: 2024-09-10 08:04:51.712151034 +0000 UTC m=+0.015602746 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:04:52.088 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:04:51 smithi107 podman[63876]: 2024-09-10 08:04:51.82792557 +0000 UTC m=+0.131377327 container create f1a76df045aaee3aefda7aeb6325b004ff8ad57b08e7bfda19b2096fe81a35f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:04:52.088 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:04:52 smithi107 podman[63876]: 2024-09-10 08:04:52.089055343 +0000 UTC m=+0.392507055 container init f1a76df045aaee3aefda7aeb6325b004ff8ad57b08e7bfda19b2096fe81a35f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:04:52.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.345+0000 7fd165200640 1 -- 172.21.15.2:0/4234291716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd160109140 msgr2=0x7fd160110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:52.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.345+0000 7fd165200640 1 --2- 172.21.15.2:0/4234291716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd160109140 0x7fd160110cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd158009ca0 tx=0x7fd15802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:04:52.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.345+0000 7fd165200640 1 -- 172.21.15.2:0/4234291716 shutdown_connections 2024-09-10T08:04:52.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.345+0000 7fd165200640 1 --2- 172.21.15.2:0/4234291716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd160109140 0x7fd160110cc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:52.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.345+0000 7fd165200640 1 --2- 172.21.15.2:0/4234291716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1601049c0 0x7fd160108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.345+0000 7fd165200640 1 --2- 172.21.15.2:0/4234291716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd160104010 0x7fd1601043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.345+0000 7fd165200640 1 -- 172.21.15.2:0/4234291716 >> 172.21.15.2:0/4234291716 conn(0x7fd1600ffa50 msgr2=0x7fd160101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.345+0000 7fd165200640 1 -- 172.21.15.2:0/4234291716 shutdown_connections 2024-09-10T08:04:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.345+0000 7fd165200640 1 -- 172.21.15.2:0/4234291716 wait complete. 2024-09-10T08:04:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.346+0000 7fd165200640 1 Processor -- start 2024-09-10T08:04:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.346+0000 7fd165200640 1 -- start start 2024-09-10T08:04:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd165200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd160104010 0x7fd16019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd165200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1601049c0 0x7fd16019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd165200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd160109140 0x7fd16019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd165200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601141c0 con 0x7fd1601049c0 2024-09-10T08:04:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd165200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd160114040 con 0x7fd160109140 2024-09-10T08:04:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd165200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd160114340 con 0x7fd160104010 2024-09-10T08:04:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd15de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd160104010 0x7fd16019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd15de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd160104010 0x7fd16019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42696/0 (socket says 172.21.15.2:42696) 2024-09-10T08:04:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd157e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1601049c0 0x7fd16019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd15de00640 1 -- 172.21.15.2:0/3918576735 learned_addr learned my addr 172.21.15.2:0/3918576735 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:04:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd15e800640 1 --2- 172.21.15.2:0/3918576735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd160109140 0x7fd16019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd15de00640 1 -- 172.21.15.2:0/3918576735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd160109140 msgr2=0x7fd16019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd15de00640 1 --2- 172.21.15.2:0/3918576735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd160109140 0x7fd16019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd15de00640 1 -- 172.21.15.2:0/3918576735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1601049c0 msgr2=0x7fd16019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.347+0000 7fd15de00640 1 --2- 172.21.15.2:0/3918576735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1601049c0 0x7fd16019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.348+0000 7fd15de00640 1 -- 172.21.15.2:0/3918576735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16019f670 con 0x7fd160104010 2024-09-10T08:04:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.348+0000 7fd15de00640 1 --2- 172.21.15.2:0/3918576735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd160104010 0x7fd16019a6c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd15000e9f0 tx=0x7fd15000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.348+0000 7fd156a00640 1 -- 172.21.15.2:0/3918576735 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15000cd80 con 0x7fd160104010 2024-09-10T08:04:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.348+0000 7fd156a00640 1 -- 172.21.15.2:0/3918576735 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd150010910 con 0x7fd160104010 2024-09-10T08:04:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.348+0000 7fd156a00640 1 -- 172.21.15.2:0/3918576735 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150018990 con 0x7fd160104010 2024-09-10T08:04:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.348+0000 7fd165200640 1 -- 172.21.15.2:0/3918576735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16019f960 con 0x7fd160104010 2024-09-10T08:04:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.348+0000 7fd165200640 1 -- 172.21.15.2:0/3918576735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1601a7240 con 0x7fd160104010 2024-09-10T08:04:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.349+0000 7fd165200640 1 -- 172.21.15.2:0/3918576735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1601043f0 con 0x7fd160104010 2024-09-10T08:04:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.351+0000 7fd156a00640 1 -- 172.21.15.2:0/3918576735 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd150010ac0 con 0x7fd160104010 2024-09-10T08:04:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.352+0000 7fd156a00640 1 --2- 172.21.15.2:0/3918576735 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd1440a2b80 0x7fd1440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.353+0000 7fd156a00640 1 -- 172.21.15.2:0/3918576735 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fd1500cf4f0 con 0x7fd160104010 2024-09-10T08:04:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.353+0000 7fd157e00640 1 --2- 172.21.15.2:0/3918576735 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd1440a2b80 0x7fd1440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.353+0000 7fd157e00640 1 --2- 172.21.15.2:0/3918576735 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd1440a2b80 0x7fd1440a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd16010ab10 tx=0x7fd148005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.355+0000 7fd156a00640 1 -- 172.21.15.2:0/3918576735 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd15009c200 con 0x7fd160104010 2024-09-10T08:04:52.416 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:04:52 smithi107 podman[63876]: 2024-09-10 08:04:52.09200333 +0000 UTC m=+0.395455048 container start f1a76df045aaee3aefda7aeb6325b004ff8ad57b08e7bfda19b2096fe81a35f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD) 2024-09-10T08:04:52.416 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:04:52 smithi107 ceph-mds[63895]: -- 172.21.15.107:0/824614526 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5634472af860 con 0x56344728c800 2024-09-10T08:04:52.417 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:04:52 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-e[63891]: starting mds.e at 2024-09-10T08:04:52.417 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:04:52 smithi107 bash[63876]: f1a76df045aaee3aefda7aeb6325b004ff8ad57b08e7bfda19b2096fe81a35f9 2024-09-10T08:04:52.417 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:04:52 smithi107 systemd[1]: Started Ceph mds.e for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:04:52.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.577+0000 7fd165200640 1 -- 172.21.15.2:0/3918576735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd16010c380 con 0x7fd160104010 2024-09-10T08:04:52.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.581+0000 7fd156a00640 1 -- 172.21.15.2:0/3918576735 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7fd1500a0500 con 0x7fd160104010 2024-09-10T08:04:52.580 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:04:52.581 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":5,"btime":"2024-09-10T08:04:50:318688+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T08:04:52.581 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 5 2024-09-10T08:04:52.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.583+0000 7fd165200640 1 -- 172.21.15.2:0/3918576735 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd1440a2b80 msgr2=0x7fd1440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:52.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.583+0000 7fd165200640 1 --2- 172.21.15.2:0/3918576735 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd1440a2b80 0x7fd1440a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd16010ab10 tx=0x7fd148005f90 comp rx=0 tx=0).stop 2024-09-10T08:04:52.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.583+0000 7fd165200640 1 -- 172.21.15.2:0/3918576735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd160104010 msgr2=0x7fd16019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:52.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.583+0000 7fd165200640 1 --2- 172.21.15.2:0/3918576735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd160104010 0x7fd16019a6c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd15000e9f0 tx=0x7fd15000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:04:52.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.583+0000 7fd165200640 1 -- 172.21.15.2:0/3918576735 shutdown_connections 2024-09-10T08:04:52.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.583+0000 7fd165200640 1 --2- 172.21.15.2:0/3918576735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd160109140 0x7fd16019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:52.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.583+0000 7fd165200640 1 --2- 172.21.15.2:0/3918576735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1601049c0 0x7fd16019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:52.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.583+0000 7fd165200640 1 --2- 172.21.15.2:0/3918576735 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd1440a2b80 0x7fd1440a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:52.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.583+0000 7fd165200640 1 --2- 172.21.15.2:0/3918576735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd160104010 0x7fd16019a6c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:52.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.583+0000 7fd165200640 1 -- 172.21.15.2:0/3918576735 >> 172.21.15.2:0/3918576735 conn(0x7fd1600ffa50 msgr2=0x7fd160100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:52.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.584+0000 7fd165200640 1 -- 172.21.15.2:0/3918576735 shutdown_connections 2024-09-10T08:04:52.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:52.584+0000 7fd165200640 1 -- 172.21.15.2:0/3918576735 wait complete. 2024-09-10T08:04:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:53 smithi002 ceph-mon[30563]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:53 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:53 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:53 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:53 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:53 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:53 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:53 smithi002 ceph-mon[30563]: Deploying daemon mds.f on smithi155 2024-09-10T08:04:53.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:53 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3918576735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:53.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:53 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.107:6834/1775771400,v1:172.21.15.107:6835/1775771400] up:boot 2024-09-10T08:04:53.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:53 smithi002 ceph-mon[30563]: fsmap 5 up:standby 2024-09-10T08:04:53.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:53 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:04:53.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:53 smithi155 ceph-mon[35405]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:53.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:53 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:53.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:53 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:53.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:53 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:53.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:53 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:53.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:53 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:53.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:53 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:53.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:53 smithi155 ceph-mon[35405]: Deploying daemon mds.f on smithi155 2024-09-10T08:04:53.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:53 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3918576735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:53.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:53 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.107:6834/1775771400,v1:172.21.15.107:6835/1775771400] up:boot 2024-09-10T08:04:53.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:53 smithi155 ceph-mon[35405]: fsmap 5 up:standby 2024-09-10T08:04:53.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:53 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:04:53.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:53 smithi107 ceph-mon[35048]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:53.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:53 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:53.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:53 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:53.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:53 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:53.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:53 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:53.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:53 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:53.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:53 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:53.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:53 smithi107 ceph-mon[35048]: Deploying daemon mds.f on smithi155 2024-09-10T08:04:53.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:53 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3918576735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:53.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:53 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.107:6834/1775771400,v1:172.21.15.107:6835/1775771400] up:boot 2024-09-10T08:04:53.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:53 smithi107 ceph-mon[35048]: fsmap 5 up:standby 2024-09-10T08:04:53.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:53 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:04:54.406 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:04:54 smithi155 systemd[1]: Starting Ceph mds.f for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:04:54.906 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:04:54 smithi155 podman[64825]: 2024-09-10 08:04:54.479396252 +0000 UTC m=+0.110643135 container create 71aa353b688ce3828cf1a49f8998f51917f694419b866baeb4af6ddf6f13b5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:04:54.906 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:04:54 smithi155 podman[64825]: 2024-09-10 08:04:54.384680053 +0000 UTC m=+0.015926950 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:04:54.907 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:04:54 smithi155 podman[64825]: 2024-09-10 08:04:54.639917993 +0000 UTC m=+0.271164892 container init 71aa353b688ce3828cf1a49f8998f51917f694419b866baeb4af6ddf6f13b5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-f, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux ) 2024-09-10T08:04:54.907 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:04:54 smithi155 podman[64825]: 2024-09-10 08:04:54.64258626 +0000 UTC m=+0.273833157 container start 71aa353b688ce3828cf1a49f8998f51917f694419b866baeb4af6ddf6f13b5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:04:54.907 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:04:54 smithi155 ceph-mds[64844]: -- 172.21.15.155:0/2612560223 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f8455af860 con 0x55f84558d400 2024-09-10T08:04:54.907 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:04:54 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-f[64840]: starting mds.f at 2024-09-10T08:04:54.907 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:04:54 smithi155 bash[64825]: 71aa353b688ce3828cf1a49f8998f51917f694419b866baeb4af6ddf6f13b5e8 2024-09-10T08:04:54.907 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:04:54 smithi155 systemd[1]: Started Ceph mds.f for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:04:55.430 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:04:55.467 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:55 smithi155 ceph-mon[35405]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:55.468 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:55.468 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:55.468 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:55.468 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:55.468 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:55.468 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:55.487 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:55 smithi002 ceph-mon[30563]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:55.487 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:55.487 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:55.487 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:55.487 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:55.487 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:55.487 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:55.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:55 smithi107 ceph-mon[35048]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:55.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:55.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:55.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:55.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:55.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:55.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:55.821 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:04:56.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.416+0000 7fe2aaa00640 1 -- 172.21.15.2:0/1371102819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a4076720 msgr2=0x7fe2a4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:56.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.416+0000 7fe2aaa00640 1 --2- 172.21.15.2:0/1371102819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a4076720 0x7fe2a4076b00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe298009860 tx=0x7fe29802a340 comp rx=0 tx=0).stop 2024-09-10T08:04:56.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.416+0000 7fe2aaa00640 1 -- 172.21.15.2:0/1371102819 shutdown_connections 2024-09-10T08:04:56.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.416+0000 7fe2aaa00640 1 --2- 172.21.15.2:0/1371102819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2a4112870 0x7fe2a4114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:56.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.416+0000 7fe2aaa00640 1 --2- 172.21.15.2:0/1371102819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a4077040 0x7fe2a4112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:56.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.416+0000 7fe2aaa00640 1 --2- 172.21.15.2:0/1371102819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a4076720 0x7fe2a4076b00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:56.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.416+0000 7fe2aaa00640 1 -- 172.21.15.2:0/1371102819 >> 172.21.15.2:0/1371102819 conn(0x7fe2a40ff910 msgr2=0x7fe2a4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:56.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.416+0000 7fe2aaa00640 1 -- 172.21.15.2:0/1371102819 shutdown_connections 2024-09-10T08:04:56.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.416+0000 7fe2aaa00640 1 -- 172.21.15.2:0/1371102819 wait complete. 2024-09-10T08:04:56.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.417+0000 7fe2aaa00640 1 Processor -- start 2024-09-10T08:04:56.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.417+0000 7fe2aaa00640 1 -- start start 2024-09-10T08:04:56.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.418+0000 7fe2aaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a4076720 0x7fe2a41a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:56.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.418+0000 7fe2aaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2a4077040 0x7fe2a41a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:56.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.418+0000 7fe2aaa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a4112870 0x7fe2a419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:56.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.418+0000 7fe2aaa00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a41183e0 con 0x7fe2a4112870 2024-09-10T08:04:56.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.418+0000 7fe2aaa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a4118260 con 0x7fe2a4077040 2024-09-10T08:04:56.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.418+0000 7fe2aaa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a4118560 con 0x7fe2a4076720 2024-09-10T08:04:56.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.418+0000 7fe2aa000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a4112870 0x7fe2a419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:56.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.418+0000 7fe2aa000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a4112870 0x7fe2a419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45186/0 (socket says 172.21.15.2:45186) 2024-09-10T08:04:56.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.418+0000 7fe2aa000640 1 -- 172.21.15.2:0/3248209524 learned_addr learned my addr 172.21.15.2:0/3248209524 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:04:56.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.418+0000 7fe2a9600640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a4076720 0x7fe2a41a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:56.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.418+0000 7fe2a8c00640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2a4077040 0x7fe2a41a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:56.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.419+0000 7fe2aa000640 1 -- 172.21.15.2:0/3248209524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a4076720 msgr2=0x7fe2a41a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:56.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.419+0000 7fe2aa000640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a4076720 0x7fe2a41a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:56.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.419+0000 7fe2aa000640 1 -- 172.21.15.2:0/3248209524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2a4077040 msgr2=0x7fe2a41a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:56.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.419+0000 7fe2aa000640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2a4077040 0x7fe2a41a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:56.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.419+0000 7fe2aa000640 1 -- 172.21.15.2:0/3248209524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a419ef20 con 0x7fe2a4112870 2024-09-10T08:04:56.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.419+0000 7fe2aa000640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a4112870 0x7fe2a419e780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00eb10 tx=0x7fe29c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:56.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.419+0000 7fe2a2000640 1 -- 172.21.15.2:0/3248209524 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c004b20 con 0x7fe2a4112870 2024-09-10T08:04:56.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.419+0000 7fe2a2000640 1 -- 172.21.15.2:0/3248209524 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe29c004cc0 con 0x7fe2a4112870 2024-09-10T08:04:56.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.419+0000 7fe2a2000640 1 -- 172.21.15.2:0/3248209524 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c010790 con 0x7fe2a4112870 2024-09-10T08:04:56.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.419+0000 7fe2aaa00640 1 -- 172.21.15.2:0/3248209524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a419f210 con 0x7fe2a4112870 2024-09-10T08:04:56.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.419+0000 7fe2aaa00640 1 -- 172.21.15.2:0/3248209524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a41ab460 con 0x7fe2a4112870 2024-09-10T08:04:56.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.421+0000 7fe2aaa00640 1 -- 172.21.15.2:0/3248209524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a4077e60 con 0x7fe2a4112870 2024-09-10T08:04:56.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.423+0000 7fe2a2000640 1 -- 172.21.15.2:0/3248209524 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe29c005020 con 0x7fe2a4112870 2024-09-10T08:04:56.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.424+0000 7fe2a2000640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe28c0a2c00 0x7fe28c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:04:56.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.424+0000 7fe2a2000640 1 -- 172.21.15.2:0/3248209524 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fe29c0cf2c0 con 0x7fe2a4112870 2024-09-10T08:04:56.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.424+0000 7fe2a9600640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe28c0a2c00 0x7fe28c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:04:56.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.425+0000 7fe2a9600640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe28c0a2c00 0x7fe28c0a50c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe29802a930 tx=0x7fe2980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:04:56.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.429+0000 7fe2a2000640 1 -- 172.21.15.2:0/3248209524 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe29c09c620 con 0x7fe2a4112870 2024-09-10T08:04:56.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:56 smithi002 ceph-mon[30563]: Deploying daemon mds.c on smithi155 2024-09-10T08:04:56.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:56 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.155:6834/1068995103,v1:172.21.15.155:6835/1068995103] up:boot 2024-09-10T08:04:56.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:56 smithi002 ceph-mon[30563]: fsmap 6 up:standby 2024-09-10T08:04:56.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:04:56.576 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:56 smithi155 ceph-mon[35405]: Deploying daemon mds.c on smithi155 2024-09-10T08:04:56.576 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:56 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.155:6834/1068995103,v1:172.21.15.155:6835/1068995103] up:boot 2024-09-10T08:04:56.576 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:56 smithi155 ceph-mon[35405]: fsmap 6 up:standby 2024-09-10T08:04:56.576 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:04:56.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.650+0000 7fe2aaa00640 1 -- 172.21.15.2:0/3248209524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2a410da30 con 0x7fe2a4112870 2024-09-10T08:04:56.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.651+0000 7fe2a2000640 1 -- 172.21.15.2:0/3248209524 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5566 (secure 0 0 0) 0x7fe29c09bf20 con 0x7fe2a4112870 2024-09-10T08:04:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:04:56.651 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":7,"btime":"2024-09-10T08:04:55:230759+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-10T08:04:56.651 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 7 2024-09-10T08:04:56.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.653+0000 7fe2aaa00640 1 -- 172.21.15.2:0/3248209524 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe28c0a2c00 msgr2=0x7fe28c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:56.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.653+0000 7fe2aaa00640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe28c0a2c00 0x7fe28c0a50c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe29802a930 tx=0x7fe2980058d0 comp rx=0 tx=0).stop 2024-09-10T08:04:56.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.653+0000 7fe2aaa00640 1 -- 172.21.15.2:0/3248209524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a4112870 msgr2=0x7fe2a419e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:04:56.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.653+0000 7fe2aaa00640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a4112870 0x7fe2a419e780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00eb10 tx=0x7fe29c00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:04:56.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.653+0000 7fe2aaa00640 1 -- 172.21.15.2:0/3248209524 shutdown_connections 2024-09-10T08:04:56.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.653+0000 7fe2aaa00640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2a4112870 0x7fe2a419e780 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:56.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.653+0000 7fe2aaa00640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2a4077040 0x7fe2a41a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:56.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.653+0000 7fe2aaa00640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe28c0a2c00 0x7fe28c0a50c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:56.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.653+0000 7fe2aaa00640 1 --2- 172.21.15.2:0/3248209524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a4076720 0x7fe2a41a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:04:56.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.653+0000 7fe2aaa00640 1 -- 172.21.15.2:0/3248209524 >> 172.21.15.2:0/3248209524 conn(0x7fe2a40ff910 msgr2=0x7fe2a4101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:04:56.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.653+0000 7fe2aaa00640 1 -- 172.21.15.2:0/3248209524 shutdown_connections 2024-09-10T08:04:56.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:04:56.653+0000 7fe2aaa00640 1 -- 172.21.15.2:0/3248209524 wait complete. 2024-09-10T08:04:56.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:56 smithi107 ceph-mon[35048]: Deploying daemon mds.c on smithi155 2024-09-10T08:04:56.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:56 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.155:6834/1068995103,v1:172.21.15.155:6835/1068995103] up:boot 2024-09-10T08:04:56.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:56 smithi107 ceph-mon[35048]: fsmap 6 up:standby 2024-09-10T08:04:56.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:04:56.906 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:04:56 smithi155 systemd[1]: Starting Ceph mds.c for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:04:57.243 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:04:56 smithi155 podman[65124]: 2024-09-10 08:04:56.881352396 +0000 UTC m=+0.015647209 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:04:57.243 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:04:56 smithi155 podman[65124]: 2024-09-10 08:04:56.99283359 +0000 UTC m=+0.127128392 container create 4ad1423fd2e4a8578b0c083e331f057e6d08445d671976c966ae210f548d1522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-c, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T08:04:57.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:57 smithi002 ceph-mon[30563]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:57.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:57 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3248209524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:57.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:57 smithi155 ceph-mon[35405]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:57.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:57 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3248209524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:57.657 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:04:57 smithi155 podman[65124]: 2024-09-10 08:04:57.245340928 +0000 UTC m=+0.379635749 container init 4ad1423fd2e4a8578b0c083e331f057e6d08445d671976c966ae210f548d1522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-c, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:04:57.657 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:04:57 smithi155 podman[65124]: 2024-09-10 08:04:57.248444439 +0000 UTC m=+0.382739252 container start 4ad1423fd2e4a8578b0c083e331f057e6d08445d671976c966ae210f548d1522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T08:04:57.657 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:04:57 smithi155 ceph-mds[65143]: -- 172.21.15.155:0/3440091734 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55763a595860 con 0x55763a573000 2024-09-10T08:04:57.657 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:04:57 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-c[65139]: starting mds.c at 2024-09-10T08:04:57.657 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:04:57 smithi155 bash[65124]: 4ad1423fd2e4a8578b0c083e331f057e6d08445d671976c966ae210f548d1522 2024-09-10T08:04:57.657 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:04:57 smithi155 systemd[1]: Started Ceph mds.c for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:04:57.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:57 smithi107 ceph-mon[35048]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:57.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:57 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3248209524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:04:58.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:58 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:58.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:58 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:58.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:58 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:58.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:58 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:58.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:58 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:58.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:58 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:58.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:58 smithi155 ceph-mon[35405]: Deploying daemon mds.h on smithi107 2024-09-10T08:04:58.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:58 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.155:6836/3780222531,v1:172.21.15.155:6837/3780222531] up:boot 2024-09-10T08:04:58.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:58 smithi155 ceph-mon[35405]: fsmap 7 up:standby 2024-09-10T08:04:58.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:58 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:04:58.629 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:58 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:58.629 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:58 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:58.629 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:58 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:58.629 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:58 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:58.629 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:58 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:58.630 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:58 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:58.630 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:58 smithi002 ceph-mon[30563]: Deploying daemon mds.h on smithi107 2024-09-10T08:04:58.630 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:58 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.155:6836/3780222531,v1:172.21.15.155:6837/3780222531] up:boot 2024-09-10T08:04:58.630 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:58 smithi002 ceph-mon[30563]: fsmap 7 up:standby 2024-09-10T08:04:58.630 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:58 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:04:58.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:58 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:58.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:58 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:58.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:58 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:04:58.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:58 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:04:58.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:58 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:04:58.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:58 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:04:58.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:58 smithi107 ceph-mon[35048]: Deploying daemon mds.h on smithi107 2024-09-10T08:04:58.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:58 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.155:6836/3780222531,v1:172.21.15.155:6837/3780222531] up:boot 2024-09-10T08:04:58.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:58 smithi107 ceph-mon[35048]: fsmap 7 up:standby 2024-09-10T08:04:58.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:58 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:04:59.614 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:04:59.626 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:04:59 smithi107 ceph-mon[35048]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:59.627 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:04:59 smithi107 systemd[1]: Starting Ceph mds.h for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:04:59.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:04:59 smithi155 ceph-mon[35405]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:59.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:04:59 smithi002 ceph-mon[30563]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:04:59.893 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:04:59 smithi107 podman[64179]: 2024-09-10 08:04:59.530107574 +0000 UTC m=+0.015549795 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:04:59.893 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:04:59 smithi107 podman[64179]: 2024-09-10 08:04:59.657773247 +0000 UTC m=+0.143215455 container create ebd0d0571fa618e43165d7594aba409190be4d4c2234cc2dde79aca0b98c0574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-h, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:05:00.008 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:00.416 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:05:00 smithi107 podman[64179]: 2024-09-10 08:05:00.04388306 +0000 UTC m=+0.529325277 container init ebd0d0571fa618e43165d7594aba409190be4d4c2234cc2dde79aca0b98c0574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T08:05:00.417 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:05:00 smithi107 podman[64179]: 2024-09-10 08:05:00.0468172 +0000 UTC m=+0.532259414 container start ebd0d0571fa618e43165d7594aba409190be4d4c2234cc2dde79aca0b98c0574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T08:05:00.417 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:05:00 smithi107 ceph-mds[64197]: -- 172.21.15.107:0/848087975 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e03d2d860 con 0x555e03d0b400 2024-09-10T08:05:00.417 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:05:00 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-h[64193]: starting mds.h at 2024-09-10T08:05:00.417 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:05:00 smithi107 bash[64179]: ebd0d0571fa618e43165d7594aba409190be4d4c2234cc2dde79aca0b98c0574 2024-09-10T08:05:00.417 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:05:00 smithi107 systemd[1]: Started Ceph mds.h for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:05:00.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.614+0000 7f13b2400640 1 -- 172.21.15.2:0/2963865772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13ac1019c0 msgr2=0x7f13ac101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:00.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.614+0000 7f13b2400640 1 --2- 172.21.15.2:0/2963865772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13ac1019c0 0x7f13ac101e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1394009900 tx=0x7f139402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:05:00.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.614+0000 7f13b2400640 1 -- 172.21.15.2:0/2963865772 shutdown_connections 2024-09-10T08:05:00.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.614+0000 7f13b2400640 1 --2- 172.21.15.2:0/2963865772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13ac102360 0x7f13ac10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:00.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.614+0000 7f13b2400640 1 --2- 172.21.15.2:0/2963865772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13ac1019c0 0x7f13ac101e20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:00.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.614+0000 7f13b2400640 1 --2- 172.21.15.2:0/2963865772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13ac1079c0 0x7f13ac107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:00.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.614+0000 7f13b2400640 1 -- 172.21.15.2:0/2963865772 >> 172.21.15.2:0/2963865772 conn(0x7f13ac0fd740 msgr2=0x7f13ac0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:00.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.614+0000 7f13b2400640 1 -- 172.21.15.2:0/2963865772 shutdown_connections 2024-09-10T08:05:00.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.614+0000 7f13b2400640 1 -- 172.21.15.2:0/2963865772 wait complete. 2024-09-10T08:05:00.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.615+0000 7f13b2400640 1 Processor -- start 2024-09-10T08:05:00.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.616+0000 7f13b2400640 1 -- start start 2024-09-10T08:05:00.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.616+0000 7f13b2400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13ac1019c0 0x7f13ac198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:00.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.616+0000 7f13b2400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13ac102360 0x7f13ac198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:00.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.616+0000 7f13b2400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13ac1079c0 0x7f13ac19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:00.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.616+0000 7f13b2400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13ac111e10 con 0x7f13ac1019c0 2024-09-10T08:05:00.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.616+0000 7f13b2400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13ac111c90 con 0x7f13ac102360 2024-09-10T08:05:00.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.616+0000 7f13b2400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13ac111f90 con 0x7f13ac1079c0 2024-09-10T08:05:00.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.617+0000 7f13b1000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13ac1019c0 0x7f13ac198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:00.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.617+0000 7f13a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13ac102360 0x7f13ac198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:00.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.617+0000 7f13a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13ac102360 0x7f13ac198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58414/0 (socket says 172.21.15.2:58414) 2024-09-10T08:05:00.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.617+0000 7f13a3e00640 1 -- 172.21.15.2:0/1181219771 learned_addr learned my addr 172.21.15.2:0/1181219771 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:00.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.617+0000 7f13b1a00640 1 --2- 172.21.15.2:0/1181219771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13ac1079c0 0x7f13ac19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:00.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.617+0000 7f13a3e00640 1 -- 172.21.15.2:0/1181219771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13ac1079c0 msgr2=0x7f13ac19cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:00.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.617+0000 7f13a3e00640 1 --2- 172.21.15.2:0/1181219771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13ac1079c0 0x7f13ac19cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:00.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.617+0000 7f13a3e00640 1 -- 172.21.15.2:0/1181219771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13ac1019c0 msgr2=0x7f13ac198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:00.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.617+0000 7f13a3e00640 1 --2- 172.21.15.2:0/1181219771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13ac1019c0 0x7f13ac198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:00.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.617+0000 7f13a3e00640 1 -- 172.21.15.2:0/1181219771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ac19d200 con 0x7f13ac102360 2024-09-10T08:05:00.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.617+0000 7f13a3e00640 1 --2- 172.21.15.2:0/1181219771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13ac102360 0x7f13ac198790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f139402a9e0 tx=0x7f1394003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:00.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.618+0000 7f13aa000640 1 -- 172.21.15.2:0/1181219771 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139403f070 con 0x7f13ac102360 2024-09-10T08:05:00.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.618+0000 7f13aa000640 1 -- 172.21.15.2:0/1181219771 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1394031c30 con 0x7f13ac102360 2024-09-10T08:05:00.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.618+0000 7f13b2400640 1 -- 172.21.15.2:0/1181219771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13ac19d490 con 0x7f13ac102360 2024-09-10T08:05:00.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.618+0000 7f13aa000640 1 -- 172.21.15.2:0/1181219771 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394035030 con 0x7f13ac102360 2024-09-10T08:05:00.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.618+0000 7f13b2400640 1 -- 172.21.15.2:0/1181219771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13ac1a4dd0 con 0x7f13ac102360 2024-09-10T08:05:00.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.619+0000 7f13b2400640 1 -- 172.21.15.2:0/1181219771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13ac101e20 con 0x7f13ac102360 2024-09-10T08:05:00.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.621+0000 7f13aa000640 1 -- 172.21.15.2:0/1181219771 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1394042070 con 0x7f13ac102360 2024-09-10T08:05:00.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.622+0000 7f13aa000640 1 --2- 172.21.15.2:0/1181219771 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13900a2c00 0x7f13900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:00.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.622+0000 7f13aa000640 1 -- 172.21.15.2:0/1181219771 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f13940eee10 con 0x7f13ac102360 2024-09-10T08:05:00.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.622+0000 7f13b1000640 1 --2- 172.21.15.2:0/1181219771 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13900a2c00 0x7f13900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:00.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.623+0000 7f13b1000640 1 --2- 172.21.15.2:0/1181219771 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13900a2c00 0x7f13900a50c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f139c0095f0 tx=0x7f139c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:00.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.626+0000 7f13aa000640 1 -- 172.21.15.2:0/1181219771 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13940f2030 con 0x7f13ac102360 2024-09-10T08:05:00.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.853+0000 7f13b2400640 1 -- 172.21.15.2:0/1181219771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13ac1995b0 con 0x7f13ac102360 2024-09-10T08:05:00.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.854+0000 7f13aa000640 1 -- 172.21.15.2:0/1181219771 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7222 (secure 0 0 0) 0x7f13ac1995b0 con 0x7f13ac102360 2024-09-10T08:05:00.853 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:05:00.853 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":9,"btime":"2024-09-10T08:05:00:399898+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T08:05:00.854 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 9 2024-09-10T08:05:00.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.856+0000 7f13b2400640 1 -- 172.21.15.2:0/1181219771 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13900a2c00 msgr2=0x7f13900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:00.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.856+0000 7f13b2400640 1 --2- 172.21.15.2:0/1181219771 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13900a2c00 0x7f13900a50c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f139c0095f0 tx=0x7f139c005f90 comp rx=0 tx=0).stop 2024-09-10T08:05:00.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.856+0000 7f13b2400640 1 -- 172.21.15.2:0/1181219771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13ac102360 msgr2=0x7f13ac198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:00.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.856+0000 7f13b2400640 1 --2- 172.21.15.2:0/1181219771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13ac102360 0x7f13ac198790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f139402a9e0 tx=0x7f1394003c10 comp rx=0 tx=0).stop 2024-09-10T08:05:00.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.856+0000 7f13b2400640 1 -- 172.21.15.2:0/1181219771 shutdown_connections 2024-09-10T08:05:00.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.856+0000 7f13b2400640 1 --2- 172.21.15.2:0/1181219771 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13900a2c00 0x7f13900a50c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:00.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.856+0000 7f13b2400640 1 --2- 172.21.15.2:0/1181219771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13ac1079c0 0x7f13ac19cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:00.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.856+0000 7f13b2400640 1 --2- 172.21.15.2:0/1181219771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13ac102360 0x7f13ac198790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:00.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.856+0000 7f13b2400640 1 --2- 172.21.15.2:0/1181219771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13ac1019c0 0x7f13ac198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:00.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.856+0000 7f13b2400640 1 -- 172.21.15.2:0/1181219771 >> 172.21.15.2:0/1181219771 conn(0x7f13ac0fd740 msgr2=0x7f13ac0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:00.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.856+0000 7f13b2400640 1 -- 172.21.15.2:0/1181219771 shutdown_connections 2024-09-10T08:05:00.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:00.856+0000 7f13b2400640 1 -- 172.21.15.2:0/1181219771 wait complete. 2024-09-10T08:05:01.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:01 smithi155 ceph-mon[35405]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:01.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:01.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:01.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:01.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:05:01.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:05:01.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:01.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:01 smithi155 ceph-mon[35405]: Deploying daemon mds.i on smithi155 2024-09-10T08:05:01.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:01 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] up:boot 2024-09-10T08:05:01.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:01 smithi155 ceph-mon[35405]: fsmap 8 up:standby 2024-09-10T08:05:01.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:01 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:05:01.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:01 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1181219771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:01 smithi002 ceph-mon[30563]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:05:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:05:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:01 smithi002 ceph-mon[30563]: Deploying daemon mds.i on smithi155 2024-09-10T08:05:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:01 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] up:boot 2024-09-10T08:05:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:01 smithi002 ceph-mon[30563]: fsmap 8 up:standby 2024-09-10T08:05:01.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:01 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:05:01.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:01 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1181219771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:01.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:01 smithi107 ceph-mon[35048]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:01.677 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:01.677 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:01.677 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:01.677 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:05:01.677 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:05:01.677 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:01.678 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:01 smithi107 ceph-mon[35048]: Deploying daemon mds.i on smithi155 2024-09-10T08:05:01.678 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:01 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] up:boot 2024-09-10T08:05:01.678 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:01 smithi107 ceph-mon[35048]: fsmap 8 up:standby 2024-09-10T08:05:01.678 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:01 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:05:01.678 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:01 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1181219771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:02.406 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:05:02 smithi155 systemd[1]: Starting Ceph mds.i for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:05:02.701 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:05:02 smithi155 podman[65425]: 2024-09-10 08:05:02.332933555 +0000 UTC m=+0.015699004 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:05:02.701 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:05:02 smithi155 podman[65425]: 2024-09-10 08:05:02.472898291 +0000 UTC m=+0.155663725 container create af41c8137399c635edb08f7b28270e4bca75fe268f749364b712d821b35574d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-i, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T08:05:02.701 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:05:02 smithi155 podman[65425]: 2024-09-10 08:05:02.67539854 +0000 UTC m=+0.358163973 container init af41c8137399c635edb08f7b28270e4bca75fe268f749364b712d821b35574d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-i, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:05:02.701 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:05:02 smithi155 podman[65425]: 2024-09-10 08:05:02.678124431 +0000 UTC m=+0.360889866 container start af41c8137399c635edb08f7b28270e4bca75fe268f749364b712d821b35574d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T08:05:03.156 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:05:02 smithi155 ceph-mds[65444]: -- 172.21.15.155:0/3516256280 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d620a87860 con 0x55d620a65000 2024-09-10T08:05:03.156 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:05:02 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-i[65440]: starting mds.i at 2024-09-10T08:05:03.156 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:05:02 smithi155 bash[65425]: af41c8137399c635edb08f7b28270e4bca75fe268f749364b712d821b35574d3 2024-09-10T08:05:03.156 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:05:02 smithi155 systemd[1]: Started Ceph mds.i for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:05:03.466 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:03 smithi002 ceph-mon[30563]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:03.466 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:03.466 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:03.466 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:03.467 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:05:03.467 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:05:03.467 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:03.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:03 smithi155 ceph-mon[35405]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:03.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:03.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:03.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:03.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:05:03.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:05:03.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:03.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:03 smithi107 ceph-mon[35048]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:03.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:03.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:03.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:03.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:05:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:05:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:03.667 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:05:04.470 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:04 smithi002 ceph-mon[30563]: Deploying daemon mds.g on smithi002 2024-09-10T08:05:04.471 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:04 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] up:boot 2024-09-10T08:05:04.471 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:04 smithi002 ceph-mon[30563]: fsmap 9 up:standby 2024-09-10T08:05:04.471 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:05:04.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:04 smithi155 ceph-mon[35405]: Deploying daemon mds.g on smithi002 2024-09-10T08:05:04.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:04 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] up:boot 2024-09-10T08:05:04.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:04 smithi155 ceph-mon[35405]: fsmap 9 up:standby 2024-09-10T08:05:04.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:05:04.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:04 smithi107 ceph-mon[35048]: Deploying daemon mds.g on smithi002 2024-09-10T08:05:04.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:04 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] up:boot 2024-09-10T08:05:04.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:04 smithi107 ceph-mon[35048]: fsmap 9 up:standby 2024-09-10T08:05:04.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:05:04.708 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:05.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:05 smithi002 ceph-mon[30563]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.562+0000 7f25fe400640 1 -- 172.21.15.2:0/502749294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25f8103ff0 msgr2=0x7f25f81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.562+0000 7f25fe400640 1 --2- 172.21.15.2:0/502749294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25f8103ff0 0x7f25f81043d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009900 tx=0x7f25e802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:05:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.562+0000 7f25fe400640 1 -- 172.21.15.2:0/502749294 shutdown_connections 2024-09-10T08:05:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.562+0000 7f25fe400640 1 --2- 172.21.15.2:0/502749294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f8109120 0x7f25f8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.562+0000 7f25fe400640 1 --2- 172.21.15.2:0/502749294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25f81049a0 0x7f25f81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.562+0000 7f25fe400640 1 --2- 172.21.15.2:0/502749294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25f8103ff0 0x7f25f81043d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.562+0000 7f25fe400640 1 -- 172.21.15.2:0/502749294 >> 172.21.15.2:0/502749294 conn(0x7f25f80ffa90 msgr2=0x7f25f8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.562+0000 7f25fe400640 1 -- 172.21.15.2:0/502749294 shutdown_connections 2024-09-10T08:05:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.562+0000 7f25fe400640 1 -- 172.21.15.2:0/502749294 wait complete. 2024-09-10T08:05:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.563+0000 7f25fe400640 1 Processor -- start 2024-09-10T08:05:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.563+0000 7f25fe400640 1 -- start start 2024-09-10T08:05:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.563+0000 7f25fe400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25f8103ff0 0x7f25f819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.563+0000 7f25fe400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f81049a0 0x7f25f819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.563+0000 7f25fe400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25f8109120 0x7f25f819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.563+0000 7f25fe400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f25f8114170 con 0x7f25f8109120 2024-09-10T08:05:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.563+0000 7f25fe400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f25f8113ff0 con 0x7f25f81049a0 2024-09-10T08:05:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.563+0000 7f25fe400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f25f81142f0 con 0x7f25f8103ff0 2024-09-10T08:05:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.563+0000 7f25f7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25f8109120 0x7f25f819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.563+0000 7f25f7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25f8109120 0x7f25f819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42712/0 (socket says 172.21.15.2:42712) 2024-09-10T08:05:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.563+0000 7f25f7e00640 1 -- 172.21.15.2:0/3693977384 learned_addr learned my addr 172.21.15.2:0/3693977384 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.563+0000 7f25f7400640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25f8103ff0 0x7f25f819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.564+0000 7f25eea00640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f81049a0 0x7f25f819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.564+0000 7f25f7e00640 1 -- 172.21.15.2:0/3693977384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25f8103ff0 msgr2=0x7f25f819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.564+0000 7f25f7e00640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25f8103ff0 0x7f25f819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.564+0000 7f25f7e00640 1 -- 172.21.15.2:0/3693977384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f81049a0 msgr2=0x7f25f819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.564+0000 7f25f7e00640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f81049a0 0x7f25f819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.564+0000 7f25f7e00640 1 -- 172.21.15.2:0/3693977384 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f819f620 con 0x7f25f8109120 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.564+0000 7f25f7e00640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25f8109120 0x7f25f819ef40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f25f000eb10 tx=0x7f25f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.564+0000 7f25f4c00640 1 -- 172.21.15.2:0/3693977384 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0004b20 con 0x7f25f8109120 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.564+0000 7f25f4c00640 1 -- 172.21.15.2:0/3693977384 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f0004cc0 con 0x7f25f8109120 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.564+0000 7f25fe400640 1 -- 172.21.15.2:0/3693977384 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f819f910 con 0x7f25f8109120 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.565+0000 7f25f4c00640 1 -- 172.21.15.2:0/3693977384 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0010760 con 0x7f25f8109120 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.565+0000 7f25fe400640 1 -- 172.21.15.2:0/3693977384 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f81a71f0 con 0x7f25f8109120 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.566+0000 7f25fe400640 1 -- 172.21.15.2:0/3693977384 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f81043d0 con 0x7f25f8109120 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.568+0000 7f25f4c00640 1 -- 172.21.15.2:0/3693977384 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f25f0005000 con 0x7f25f8109120 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.569+0000 7f25f4c00640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f25dc0a2b80 0x7f25dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.569+0000 7f25f4c00640 1 -- 172.21.15.2:0/3693977384 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f25f00cf350 con 0x7f25f8109120 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.569+0000 7f25f7400640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f25dc0a2b80 0x7f25dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.570+0000 7f25f7400640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f25dc0a2b80 0x7f25dc0a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f25e8000c00 tx=0x7f25e8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:05.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.572+0000 7f25f4c00640 1 -- 172.21.15.2:0/3693977384 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f25f00541b0 con 0x7f25f8109120 2024-09-10T08:05:05.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:05 smithi155 ceph-mon[35405]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:05.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:05 smithi107 ceph-mon[35048]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:05.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.817+0000 7f25fe400640 1 -- 172.21.15.2:0/3693977384 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25f819b9d0 con 0x7f25f8109120 2024-09-10T08:05:05.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.817+0000 7f25f4c00640 1 -- 172.21.15.2:0/3693977384 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8052 (secure 0 0 0) 0x7f25f819b9d0 con 0x7f25f8109120 2024-09-10T08:05:05.818 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:05:05.818 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":10,"btime":"2024-09-10T08:05:03:196276+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T08:05:05.819 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 10 2024-09-10T08:05:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.820+0000 7f25fe400640 1 -- 172.21.15.2:0/3693977384 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f25dc0a2b80 msgr2=0x7f25dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.820+0000 7f25fe400640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f25dc0a2b80 0x7f25dc0a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f25e8000c00 tx=0x7f25e8005b50 comp rx=0 tx=0).stop 2024-09-10T08:05:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.820+0000 7f25fe400640 1 -- 172.21.15.2:0/3693977384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25f8109120 msgr2=0x7f25f819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:05.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.820+0000 7f25fe400640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25f8109120 0x7f25f819ef40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f25f000eb10 tx=0x7f25f000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:05:05.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.820+0000 7f25fe400640 1 -- 172.21.15.2:0/3693977384 shutdown_connections 2024-09-10T08:05:05.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.820+0000 7f25fe400640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25f8109120 0x7f25f819ef40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:05.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.820+0000 7f25fe400640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f81049a0 0x7f25f819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:05.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.820+0000 7f25fe400640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f25dc0a2b80 0x7f25dc0a5040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:05.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.820+0000 7f25fe400640 1 --2- 172.21.15.2:0/3693977384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25f8103ff0 0x7f25f819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:05.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.820+0000 7f25fe400640 1 -- 172.21.15.2:0/3693977384 >> 172.21.15.2:0/3693977384 conn(0x7f25f80ffa90 msgr2=0x7f25f8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:05.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.820+0000 7f25fe400640 1 -- 172.21.15.2:0/3693977384 shutdown_connections 2024-09-10T08:05:05.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:05.820+0000 7f25fe400640 1 -- 172.21.15.2:0/3693977384 wait complete. 2024-09-10T08:05:06.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:06 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3693977384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:06.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:06 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3693977384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:06.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:06 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3693977384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:07.216 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 10 08:05:07 smithi002 systemd[1]: Started Ceph mds.g for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:05:07.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:07 smithi002 ceph-mon[30563]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:07.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:07 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:07.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:07 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:07.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:07 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:07.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:07 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:05:07.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:07 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:05:07.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:07 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:07.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:07 smithi155 ceph-mon[35405]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:07.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:07 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:07.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:07 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:07.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:07 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:07.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:07 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:05:07.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:07 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:05:07.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:07 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:07.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:07 smithi107 ceph-mon[35048]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:07.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:07 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:07.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:07 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:07.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:07 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:07.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:07 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:05:07.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:07 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:05:07.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:07 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:08.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:08 smithi002 ceph-mon[30563]: Deploying daemon mds.a on smithi002 2024-09-10T08:05:08.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:08 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.2:6838/851586467,v1:172.21.15.2:6839/851586467] up:boot 2024-09-10T08:05:08.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:08 smithi002 ceph-mon[30563]: fsmap 10 up:standby 2024-09-10T08:05:08.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:08 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T08:05:08.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:08 smithi155 ceph-mon[35405]: Deploying daemon mds.a on smithi002 2024-09-10T08:05:08.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:08 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.2:6838/851586467,v1:172.21.15.2:6839/851586467] up:boot 2024-09-10T08:05:08.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:08 smithi155 ceph-mon[35405]: fsmap 10 up:standby 2024-09-10T08:05:08.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:08 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T08:05:08.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:08 smithi107 ceph-mon[35048]: Deploying daemon mds.a on smithi002 2024-09-10T08:05:08.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:08 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.2:6838/851586467,v1:172.21.15.2:6839/851586467] up:boot 2024-09-10T08:05:08.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:08 smithi107 ceph-mon[35048]: fsmap 10 up:standby 2024-09-10T08:05:08.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:08 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T08:05:08.834 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:05:09.235 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:09.449 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:09 smithi002 ceph-mon[30563]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:09.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:09 smithi155 ceph-mon[35405]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:09.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:09 smithi107 ceph-mon[35048]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:10.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.033+0000 7fe4f4200640 1 -- 172.21.15.2:0/3946856910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4ec102790 msgr2=0x7fe4ec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:10.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.033+0000 7fe4f4200640 1 --2- 172.21.15.2:0/3946856910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4ec102790 0x7fe4ec1067e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009900 tx=0x7fe4d802a410 comp rx=0 tx=0).stop 2024-09-10T08:05:10.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.034+0000 7fe4f4200640 1 -- 172.21.15.2:0/3946856910 shutdown_connections 2024-09-10T08:05:10.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.034+0000 7fe4f4200640 1 --2- 172.21.15.2:0/3946856910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4ec106f10 0x7fe4ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:10.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.034+0000 7fe4f4200640 1 --2- 172.21.15.2:0/3946856910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4ec102790 0x7fe4ec1067e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:10.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.034+0000 7fe4f4200640 1 --2- 172.21.15.2:0/3946856910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4ec101de0 0x7fe4ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:10.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.034+0000 7fe4f4200640 1 -- 172.21.15.2:0/3946856910 >> 172.21.15.2:0/3946856910 conn(0x7fe4ec0fd8c0 msgr2=0x7fe4ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:10.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.034+0000 7fe4f4200640 1 -- 172.21.15.2:0/3946856910 shutdown_connections 2024-09-10T08:05:10.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.034+0000 7fe4f4200640 1 -- 172.21.15.2:0/3946856910 wait complete. 2024-09-10T08:05:10.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.035+0000 7fe4f4200640 1 Processor -- start 2024-09-10T08:05:10.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.035+0000 7fe4f4200640 1 -- start start 2024-09-10T08:05:10.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.035+0000 7fe4f4200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4ec101de0 0x7fe4ec0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.035+0000 7fe4f4200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4ec102790 0x7fe4ec074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.035+0000 7fe4f4200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4ec106f10 0x7fe4ec074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.035+0000 7fe4f4200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec111fc0 con 0x7fe4ec101de0 2024-09-10T08:05:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.035+0000 7fe4f4200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec111e40 con 0x7fe4ec102790 2024-09-10T08:05:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.035+0000 7fe4f4200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ec112140 con 0x7fe4ec106f10 2024-09-10T08:05:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.035+0000 7fe4f1800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4ec101de0 0x7fe4ec0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.036+0000 7fe4f1800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4ec101de0 0x7fe4ec0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55928/0 (socket says 172.21.15.2:55928) 2024-09-10T08:05:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.036+0000 7fe4f1800640 1 -- 172.21.15.2:0/4018859517 learned_addr learned my addr 172.21.15.2:0/4018859517 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.036+0000 7fe4f0e00640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4ec102790 0x7fe4ec074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.036+0000 7fe4f2200640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4ec106f10 0x7fe4ec074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:10.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.036+0000 7fe4f1800640 1 -- 172.21.15.2:0/4018859517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4ec106f10 msgr2=0x7fe4ec074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:10.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.036+0000 7fe4f1800640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4ec106f10 0x7fe4ec074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:10.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.036+0000 7fe4f1800640 1 -- 172.21.15.2:0/4018859517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4ec102790 msgr2=0x7fe4ec074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:10.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.036+0000 7fe4f1800640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4ec102790 0x7fe4ec074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:10.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.036+0000 7fe4f1800640 1 -- 172.21.15.2:0/4018859517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ec1a68e0 con 0x7fe4ec101de0 2024-09-10T08:05:10.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.036+0000 7fe4f0e00640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4ec102790 0x7fe4ec074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:10.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.037+0000 7fe4f1800640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4ec101de0 0x7fe4ec0744f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000e9f0 tx=0x7fe4e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:10.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.037+0000 7fe4ea000640 1 -- 172.21.15.2:0/4018859517 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e000cd80 con 0x7fe4ec101de0 2024-09-10T08:05:10.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.037+0000 7fe4ea000640 1 -- 172.21.15.2:0/4018859517 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e0010910 con 0x7fe4ec101de0 2024-09-10T08:05:10.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.037+0000 7fe4ea000640 1 -- 172.21.15.2:0/4018859517 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0018990 con 0x7fe4ec101de0 2024-09-10T08:05:10.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.037+0000 7fe4f4200640 1 -- 172.21.15.2:0/4018859517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ec1a6bd0 con 0x7fe4ec101de0 2024-09-10T08:05:10.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.037+0000 7fe4f4200640 1 -- 172.21.15.2:0/4018859517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ec1a7090 con 0x7fe4ec101de0 2024-09-10T08:05:10.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.039+0000 7fe4f4200640 1 -- 172.21.15.2:0/4018859517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4ec1021c0 con 0x7fe4ec101de0 2024-09-10T08:05:10.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.040+0000 7fe4ea000640 1 -- 172.21.15.2:0/4018859517 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe4e0010ac0 con 0x7fe4ec101de0 2024-09-10T08:05:10.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.042+0000 7fe4ea000640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4d40a2c00 0x7fe4d40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:10.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.042+0000 7fe4ea000640 1 -- 172.21.15.2:0/4018859517 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fe4e00ceeb0 con 0x7fe4ec101de0 2024-09-10T08:05:10.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.042+0000 7fe4f0e00640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4d40a2c00 0x7fe4d40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:10.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.043+0000 7fe4f0e00640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4d40a2c00 0x7fe4d40a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8000c00 tx=0x7fe4d8031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:10.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.047+0000 7fe4ea000640 1 -- 172.21.15.2:0/4018859517 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4e009bbc0 con 0x7fe4ec101de0 2024-09-10T08:05:10.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.276+0000 7fe4f4200640 1 -- 172.21.15.2:0/4018859517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4ec02d810 con 0x7fe4ec101de0 2024-09-10T08:05:10.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.277+0000 7fe4ea000640 1 -- 172.21.15.2:0/4018859517 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8872 (secure 0 0 0) 0x7fe4e009fec0 con 0x7fe4ec101de0 2024-09-10T08:05:10.276 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:05:10.276 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":11,"btime":"2024-09-10T08:05:07:216344+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T08:05:10.276 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 11 2024-09-10T08:05:10.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.279+0000 7fe4f4200640 1 -- 172.21.15.2:0/4018859517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4d40a2c00 msgr2=0x7fe4d40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:10.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.279+0000 7fe4f4200640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4d40a2c00 0x7fe4d40a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8000c00 tx=0x7fe4d8031510 comp rx=0 tx=0).stop 2024-09-10T08:05:10.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.279+0000 7fe4f4200640 1 -- 172.21.15.2:0/4018859517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4ec101de0 msgr2=0x7fe4ec0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:10.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.279+0000 7fe4f4200640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4ec101de0 0x7fe4ec0744f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000e9f0 tx=0x7fe4e000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:05:10.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.279+0000 7fe4f4200640 1 -- 172.21.15.2:0/4018859517 shutdown_connections 2024-09-10T08:05:10.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.279+0000 7fe4f4200640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4d40a2c00 0x7fe4d40a50c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:10.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.279+0000 7fe4f4200640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4ec106f10 0x7fe4ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:10.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.279+0000 7fe4f4200640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4ec102790 0x7fe4ec074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:10.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.279+0000 7fe4f4200640 1 --2- 172.21.15.2:0/4018859517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4ec101de0 0x7fe4ec0744f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:10.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.279+0000 7fe4f4200640 1 -- 172.21.15.2:0/4018859517 >> 172.21.15.2:0/4018859517 conn(0x7fe4ec0fd8c0 msgr2=0x7fe4ec0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:10.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.279+0000 7fe4f4200640 1 -- 172.21.15.2:0/4018859517 shutdown_connections 2024-09-10T08:05:10.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:10.280+0000 7fe4f4200640 1 -- 172.21.15.2:0/4018859517 wait complete. 2024-09-10T08:05:11.239 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 10 08:05:11 smithi002 systemd[1]: Started Ceph mds.a for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:05:11.492 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:11 smithi002 ceph-mon[30563]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:11.492 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:11 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/4018859517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:11.492 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:11 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:11.492 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:11 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:11.492 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:11 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:11.492 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:11 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:05:11.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:11 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:05:11.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:11 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:11.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:11 smithi107 ceph-mon[35048]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:11.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:11 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/4018859517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:11.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:11 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:11.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:11 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:11.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:11 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:11.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:11 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:05:11.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:11 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:05:11.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:11 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:11.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:11 smithi155 ceph-mon[35405]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:11.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:11 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/4018859517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:11.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:11 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:11.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:11 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:11.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:11 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:11.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:11 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:05:11.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:11 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:05:11.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:11 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:12.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:12 smithi002 ceph-mon[30563]: Deploying daemon mds.b on smithi107 2024-09-10T08:05:12.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:12 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.2:6840/91146142,v1:172.21.15.2:6841/91146142] up:boot 2024-09-10T08:05:12.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:12 smithi002 ceph-mon[30563]: fsmap 11 up:standby 2024-09-10T08:05:12.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:12 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T08:05:12.622 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:12 smithi107 ceph-mon[35048]: Deploying daemon mds.b on smithi107 2024-09-10T08:05:12.622 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:12 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.2:6840/91146142,v1:172.21.15.2:6841/91146142] up:boot 2024-09-10T08:05:12.622 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:12 smithi107 ceph-mon[35048]: fsmap 11 up:standby 2024-09-10T08:05:12.622 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:12 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T08:05:12.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:12 smithi155 ceph-mon[35405]: Deploying daemon mds.b on smithi107 2024-09-10T08:05:12.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:12 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.2:6840/91146142,v1:172.21.15.2:6841/91146142] up:boot 2024-09-10T08:05:12.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:12 smithi155 ceph-mon[35405]: fsmap 11 up:standby 2024-09-10T08:05:12.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:12 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T08:05:13.253 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:05:13 smithi107 systemd[1]: Starting Ceph mds.b for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:05:13.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:13 smithi002 ceph-mon[30563]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:13.527 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:05:13.580 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:13 smithi107 ceph-mon[35048]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:13.580 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:05:13 smithi107 podman[64478]: 2024-09-10 08:05:13.220137921 +0000 UTC m=+0.016204597 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:05:13.580 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:05:13 smithi107 podman[64478]: 2024-09-10 08:05:13.328980397 +0000 UTC m=+0.125047104 container create 9f5ccb59747e4ad01c4b6d1bdaa205fa46bf08f2cd53e94cd4d41cd26f6467ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T08:05:13.580 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:05:13 smithi107 podman[64478]: 2024-09-10 08:05:13.581641995 +0000 UTC m=+0.377708660 container init 9f5ccb59747e4ad01c4b6d1bdaa205fa46bf08f2cd53e94cd4d41cd26f6467ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:05:13.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:13 smithi155 ceph-mon[35405]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:13.917 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:05:13 smithi107 podman[64478]: 2024-09-10 08:05:13.584523308 +0000 UTC m=+0.380589979 container start 9f5ccb59747e4ad01c4b6d1bdaa205fa46bf08f2cd53e94cd4d41cd26f6467ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T08:05:13.917 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:05:13 smithi107 ceph-mds[64497]: -- 172.21.15.107:0/73416430 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa6547b860 con 0x55aa65458800 2024-09-10T08:05:13.917 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:05:13 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-b[64493]: starting mds.b at 2024-09-10T08:05:13.917 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:05:13 smithi107 bash[64478]: 9f5ccb59747e4ad01c4b6d1bdaa205fa46bf08f2cd53e94cd4d41cd26f6467ad 2024-09-10T08:05:13.917 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:05:13 smithi107 systemd[1]: Started Ceph mds.b for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:05:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:14.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.512+0000 7f6264a00640 1 -- 172.21.15.2:0/3118423740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6260103e30 msgr2=0x7f6260104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:14.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.512+0000 7f6264a00640 1 --2- 172.21.15.2:0/3118423740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6260103e30 0x7f6260104210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6250009900 tx=0x7f625002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:05:14.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.512+0000 7f6264a00640 1 -- 172.21.15.2:0/3118423740 shutdown_connections 2024-09-10T08:05:14.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.512+0000 7f6264a00640 1 --2- 172.21.15.2:0/3118423740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6260108f60 0x7f6260110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:14.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.512+0000 7f6264a00640 1 --2- 172.21.15.2:0/3118423740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62601047e0 0x7f6260108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:14.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.512+0000 7f6264a00640 1 --2- 172.21.15.2:0/3118423740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6260103e30 0x7f6260104210 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:14.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.512+0000 7f6264a00640 1 -- 172.21.15.2:0/3118423740 >> 172.21.15.2:0/3118423740 conn(0x7f62600ff910 msgr2=0x7f6260101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:14.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.512+0000 7f6264a00640 1 -- 172.21.15.2:0/3118423740 shutdown_connections 2024-09-10T08:05:14.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.512+0000 7f6264a00640 1 -- 172.21.15.2:0/3118423740 wait complete. 2024-09-10T08:05:14.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.514+0000 7f6264a00640 1 Processor -- start 2024-09-10T08:05:14.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.514+0000 7f6264a00640 1 -- start start 2024-09-10T08:05:14.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.514+0000 7f6264a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6260103e30 0x7f626019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:14.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.514+0000 7f6264a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62601047e0 0x7f626019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:14.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.514+0000 7f6264a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6260108f60 0x7f626019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:14.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.514+0000 7f6264a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6260113fc0 con 0x7f6260108f60 2024-09-10T08:05:14.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.514+0000 7f6264a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6260113e40 con 0x7f6260103e30 2024-09-10T08:05:14.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.514+0000 7f6264a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6260114140 con 0x7f62601047e0 2024-09-10T08:05:14.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.514+0000 7f625f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6260103e30 0x7f626019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:14.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.515+0000 7f625f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6260103e30 0x7f626019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:47050/0 (socket says 172.21.15.2:47050) 2024-09-10T08:05:14.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.515+0000 7f625f400640 1 -- 172.21.15.2:0/1677011302 learned_addr learned my addr 172.21.15.2:0/1677011302 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:14.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.515+0000 7f625ea00640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62601047e0 0x7f626019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:14.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.515+0000 7f625fe00640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6260108f60 0x7f626019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:14.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.515+0000 7f625f400640 1 -- 172.21.15.2:0/1677011302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62601047e0 msgr2=0x7f626019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:14.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.515+0000 7f625f400640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62601047e0 0x7f626019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:14.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.515+0000 7f625f400640 1 -- 172.21.15.2:0/1677011302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6260108f60 msgr2=0x7f626019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:14.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.515+0000 7f625f400640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6260108f60 0x7f626019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:14.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.515+0000 7f625f400640 1 -- 172.21.15.2:0/1677011302 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626019f510 con 0x7f6260103e30 2024-09-10T08:05:14.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.515+0000 7f625ea00640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62601047e0 0x7f626019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:14.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.515+0000 7f625f400640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6260103e30 0x7f626019a560 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6250006010 tx=0x7f62500042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:14.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.516+0000 7f6257e00640 1 -- 172.21.15.2:0/1677011302 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625003f030 con 0x7f6260103e30 2024-09-10T08:05:14.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.516+0000 7f6257e00640 1 -- 172.21.15.2:0/1677011302 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6250003ce0 con 0x7f6260103e30 2024-09-10T08:05:14.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.516+0000 7f6264a00640 1 -- 172.21.15.2:0/1677011302 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626019f7a0 con 0x7f6260103e30 2024-09-10T08:05:14.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.516+0000 7f6257e00640 1 -- 172.21.15.2:0/1677011302 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250035070 con 0x7f6260103e30 2024-09-10T08:05:14.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.516+0000 7f6264a00640 1 -- 172.21.15.2:0/1677011302 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62601a70e0 con 0x7f6260103e30 2024-09-10T08:05:14.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.517+0000 7f6264a00640 1 -- 172.21.15.2:0/1677011302 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6260105310 con 0x7f6260103e30 2024-09-10T08:05:14.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.519+0000 7f6257e00640 1 -- 172.21.15.2:0/1677011302 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6250042030 con 0x7f6260103e30 2024-09-10T08:05:14.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.520+0000 7f6257e00640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62440a2c00 0x7f62440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:14.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.520+0000 7f6257e00640 1 -- 172.21.15.2:0/1677011302 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f62500ef300 con 0x7f6260103e30 2024-09-10T08:05:14.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.520+0000 7f625ea00640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62440a2c00 0x7f62440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:14.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.521+0000 7f625ea00640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62440a2c00 0x7f62440a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f626010a800 tx=0x7f624c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:14.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.523+0000 7f6257e00640 1 -- 172.21.15.2:0/1677011302 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62500bc0d0 con 0x7f6260103e30 2024-09-10T08:05:14.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.767+0000 7f6264a00640 1 -- 172.21.15.2:0/1677011302 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6260104210 con 0x7f6260103e30 2024-09-10T08:05:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.768+0000 7f6257e00640 1 -- 172.21.15.2:0/1677011302 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10518 (secure 0 0 0) 0x7f62500c03d0 con 0x7f6260103e30 2024-09-10T08:05:14.768 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:05:14.768 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":13,"btime":"2024-09-10T08:05:14:254339+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-10T08:05:14.769 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 13 2024-09-10T08:05:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.771+0000 7f6264a00640 1 -- 172.21.15.2:0/1677011302 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62440a2c00 msgr2=0x7f62440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.771+0000 7f6264a00640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62440a2c00 0x7f62440a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f626010a800 tx=0x7f624c00b390 comp rx=0 tx=0).stop 2024-09-10T08:05:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.771+0000 7f6264a00640 1 -- 172.21.15.2:0/1677011302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6260103e30 msgr2=0x7f626019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.771+0000 7f6264a00640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6260103e30 0x7f626019a560 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6250006010 tx=0x7f62500042f0 comp rx=0 tx=0).stop 2024-09-10T08:05:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.771+0000 7f6264a00640 1 -- 172.21.15.2:0/1677011302 shutdown_connections 2024-09-10T08:05:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.771+0000 7f6264a00640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6260108f60 0x7f626019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.771+0000 7f6264a00640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62440a2c00 0x7f62440a50c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.771+0000 7f6264a00640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62601047e0 0x7f626019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.771+0000 7f6264a00640 1 --2- 172.21.15.2:0/1677011302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6260103e30 0x7f626019a560 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.771+0000 7f6264a00640 1 -- 172.21.15.2:0/1677011302 >> 172.21.15.2:0/1677011302 conn(0x7f62600ff910 msgr2=0x7f6260100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.771+0000 7f6264a00640 1 -- 172.21.15.2:0/1677011302 shutdown_connections 2024-09-10T08:05:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:14.771+0000 7f6264a00640 1 -- 172.21.15.2:0/1677011302 wait complete. 2024-09-10T08:05:14.776 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:14 smithi107 ceph-mon[35048]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:14.776 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:14 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:14.776 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:14 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:14.776 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:14 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:14.776 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:14 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:14.776 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:14 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:14.776 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:14 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:05:14.776 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:14 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] up:boot 2024-09-10T08:05:14.776 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:14 smithi107 ceph-mon[35048]: fsmap 12 up:standby 2024-09-10T08:05:14.776 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:14 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:05:14.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:14 smithi155 ceph-mon[35405]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:14.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:14 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:14.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:14 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:14.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:14 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:14.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:14 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:14.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:14 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:14.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:14 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:05:14.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:14 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] up:boot 2024-09-10T08:05:14.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:14 smithi155 ceph-mon[35405]: fsmap 12 up:standby 2024-09-10T08:05:14.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:14 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:05:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:14 smithi002 ceph-mon[30563]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:14 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:14 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:14 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:14 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:14 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:14 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:05:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:14 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] up:boot 2024-09-10T08:05:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:14 smithi002 ceph-mon[30563]: fsmap 12 up:standby 2024-09-10T08:05:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:14 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:05:15.988 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:15 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1677011302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:15.988 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:15 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:15.988 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:15 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:15.988 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:15 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:15.989 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:15 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:16.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:15 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1677011302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:16.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:15 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:16.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:15 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:16.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:15 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:16.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:15 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:16.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:15 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1677011302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:16.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:15 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:16.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:15 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:16.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:15 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:16.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:15 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:16.342 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T08:05:16.760 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.375+0000 7f20ee600640 1 -- 172.21.15.2:0/1560635716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20e8104010 msgr2=0x7f20e81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.375+0000 7f20ee600640 1 --2- 172.21.15.2:0/1560635716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20e8104010 0x7f20e81043f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009900 tx=0x7f20d802a410 comp rx=0 tx=0).stop 2024-09-10T08:05:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.375+0000 7f20ee600640 1 -- 172.21.15.2:0/1560635716 shutdown_connections 2024-09-10T08:05:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.375+0000 7f20ee600640 1 --2- 172.21.15.2:0/1560635716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e8109140 0x7f20e8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.375+0000 7f20ee600640 1 --2- 172.21.15.2:0/1560635716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e81049c0 0x7f20e8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.375+0000 7f20ee600640 1 --2- 172.21.15.2:0/1560635716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20e8104010 0x7f20e81043f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.375+0000 7f20ee600640 1 -- 172.21.15.2:0/1560635716 >> 172.21.15.2:0/1560635716 conn(0x7f20e80ffa50 msgr2=0x7f20e8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.375+0000 7f20ee600640 1 -- 172.21.15.2:0/1560635716 shutdown_connections 2024-09-10T08:05:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.375+0000 7f20ee600640 1 -- 172.21.15.2:0/1560635716 wait complete. 2024-09-10T08:05:17.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.376+0000 7f20ee600640 1 Processor -- start 2024-09-10T08:05:17.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.377+0000 7f20ee600640 1 -- start start 2024-09-10T08:05:17.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.377+0000 7f20ee600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e8104010 0x7f20e819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:17.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.377+0000 7f20ee600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e81049c0 0x7f20e819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:17.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.377+0000 7f20ee600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20e8109140 0x7f20e819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:17.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.377+0000 7f20ee600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8114170 con 0x7f20e8109140 2024-09-10T08:05:17.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.377+0000 7f20ee600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8113ff0 con 0x7f20e8104010 2024-09-10T08:05:17.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.377+0000 7f20ee600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81142f0 con 0x7f20e81049c0 2024-09-10T08:05:17.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.377+0000 7f20e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e8104010 0x7f20e819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:17.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.377+0000 7f20e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e8104010 0x7f20e819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:56304/0 (socket says 172.21.15.2:56304) 2024-09-10T08:05:17.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.377+0000 7f20e7400640 1 -- 172.21.15.2:0/513742002 learned_addr learned my addr 172.21.15.2:0/513742002 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:17.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.377+0000 7f20e7e00640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20e8109140 0x7f20e819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:17.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.377+0000 7f20e6a00640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e81049c0 0x7f20e819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:17.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.378+0000 7f20e7400640 1 -- 172.21.15.2:0/513742002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e81049c0 msgr2=0x7f20e819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:17.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.378+0000 7f20e7400640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e81049c0 0x7f20e819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:17.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.378+0000 7f20e7400640 1 -- 172.21.15.2:0/513742002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20e8109140 msgr2=0x7f20e819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:17.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.378+0000 7f20e7400640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20e8109140 0x7f20e819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:17.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.378+0000 7f20e7400640 1 -- 172.21.15.2:0/513742002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e819f600 con 0x7f20e8104010 2024-09-10T08:05:17.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.378+0000 7f20e7400640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e8104010 0x7f20e819a650 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f20d8006010 tx=0x7f20d8004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:17.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.378+0000 7f20dfe00640 1 -- 172.21.15.2:0/513742002 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d803f030 con 0x7f20e8104010 2024-09-10T08:05:17.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.378+0000 7f20dfe00640 1 -- 172.21.15.2:0/513742002 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d8004440 con 0x7f20e8104010 2024-09-10T08:05:17.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.378+0000 7f20dfe00640 1 -- 172.21.15.2:0/513742002 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8035070 con 0x7f20e8104010 2024-09-10T08:05:17.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.378+0000 7f20ee600640 1 -- 172.21.15.2:0/513742002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e819f890 con 0x7f20e8104010 2024-09-10T08:05:17.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.379+0000 7f20ee600640 1 -- 172.21.15.2:0/513742002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e81a71d0 con 0x7f20e8104010 2024-09-10T08:05:17.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.380+0000 7f20ee600640 1 -- 172.21.15.2:0/513742002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e81043f0 con 0x7f20e8104010 2024-09-10T08:05:17.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.381+0000 7f20dfe00640 1 -- 172.21.15.2:0/513742002 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f20d8042030 con 0x7f20e8104010 2024-09-10T08:05:17.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.383+0000 7f20dfe00640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20cc0a2c00 0x7f20cc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:17.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.383+0000 7f20dfe00640 1 -- 172.21.15.2:0/513742002 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f20d80eef10 con 0x7f20e8104010 2024-09-10T08:05:17.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.383+0000 7f20e6a00640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20cc0a2c00 0x7f20cc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:17.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.384+0000 7f20e6a00640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20cc0a2c00 0x7f20cc0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20d40095f0 tx=0x7f20d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.387+0000 7f20dfe00640 1 -- 172.21.15.2:0/513742002 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20d80bbc20 con 0x7f20e8104010 2024-09-10T08:05:17.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:17 smithi002 ceph-mon[30563]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:17.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:17 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:17.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:17 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:17.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.575+0000 7f20ee600640 1 -- 172.21.15.2:0/513742002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f20e819b9b0 con 0x7f20e8104010 2024-09-10T08:05:17.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.576+0000 7f20dfe00640 1 -- 172.21.15.2:0/513742002 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7f20d80a6060 con 0x7f20e8104010 2024-09-10T08:05:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:05:17.576 INFO:teuthology.orchestra.run.smithi002.stdout:[ 2024-09-10T08:05:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:05:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "poolnum": 1, 2024-09-10T08:05:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "poolname": ".mgr" 2024-09-10T08:05:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:05:17.577 INFO:teuthology.orchestra.run.smithi002.stdout:] 2024-09-10T08:05:17.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.579+0000 7f20ee600640 1 -- 172.21.15.2:0/513742002 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20cc0a2c00 msgr2=0x7f20cc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:17.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.579+0000 7f20ee600640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20cc0a2c00 0x7f20cc0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20d40095f0 tx=0x7f20d4005f90 comp rx=0 tx=0).stop 2024-09-10T08:05:17.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.579+0000 7f20ee600640 1 -- 172.21.15.2:0/513742002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e8104010 msgr2=0x7f20e819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:17.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.579+0000 7f20ee600640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e8104010 0x7f20e819a650 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f20d8006010 tx=0x7f20d8004040 comp rx=0 tx=0).stop 2024-09-10T08:05:17.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.579+0000 7f20ee600640 1 -- 172.21.15.2:0/513742002 shutdown_connections 2024-09-10T08:05:17.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.579+0000 7f20ee600640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20e8109140 0x7f20e819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:17.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.579+0000 7f20ee600640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20cc0a2c00 0x7f20cc0a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:17.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.579+0000 7f20ee600640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e81049c0 0x7f20e819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:17.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.579+0000 7f20ee600640 1 --2- 172.21.15.2:0/513742002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e8104010 0x7f20e819a650 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:17.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.579+0000 7f20ee600640 1 -- 172.21.15.2:0/513742002 >> 172.21.15.2:0/513742002 conn(0x7f20e80ffa50 msgr2=0x7f20e8100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:17.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.579+0000 7f20ee600640 1 -- 172.21.15.2:0/513742002 shutdown_connections 2024-09-10T08:05:17.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:17.579+0000 7f20ee600640 1 -- 172.21.15.2:0/513742002 wait complete. 2024-09-10T08:05:17.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:17 smithi155 ceph-mon[35405]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:17.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:17 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:17.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:17 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:17.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:17 smithi107 ceph-mon[35048]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:17.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:17 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:17.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:17 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:18.430 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T08:05:18.430 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T08:05:18.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:18 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/513742002' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T08:05:18.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:18 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:18.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:18 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:05:18.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:18 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:18.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:18 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:05:18.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:18 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/513742002' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T08:05:18.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:18 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:18.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:18 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:05:18.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:18 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:18.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:18 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:05:18.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:18 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/513742002' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T08:05:18.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:18 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:18.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:18 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:05:18.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:18 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:18.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:18 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:05:19.622 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:19.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:19 smithi155 ceph-mon[35405]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:19.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:19.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:19 smithi107 ceph-mon[35048]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:19.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:19.767 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:19 smithi002 ceph-mon[30563]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:19.768 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:20.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.247+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1243847352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcc4103e30 msgr2=0x7fdcc4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:20.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.247+0000 7fdcc3e00640 1 --2- 172.21.15.2:0/1243847352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcc4103e30 0x7fdcc4104210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009900 tx=0x7fdcb402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:05:20.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.247+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1243847352 shutdown_connections 2024-09-10T08:05:20.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.247+0000 7fdcc3e00640 1 --2- 172.21.15.2:0/1243847352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc4108f60 0x7fdcc4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:20.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.247+0000 7fdcc3e00640 1 --2- 172.21.15.2:0/1243847352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc41047e0 0x7fdcc4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:20.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.248+0000 7fdcc3e00640 1 --2- 172.21.15.2:0/1243847352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcc4103e30 0x7fdcc4104210 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.248+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1243847352 >> 172.21.15.2:0/1243847352 conn(0x7fdcc40ff910 msgr2=0x7fdcc4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.248+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1243847352 shutdown_connections 2024-09-10T08:05:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.248+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1243847352 wait complete. 2024-09-10T08:05:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.249+0000 7fdcc3e00640 1 Processor -- start 2024-09-10T08:05:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.249+0000 7fdcc3e00640 1 -- start start 2024-09-10T08:05:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.250+0000 7fdcc3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcc4103e30 0x7fdcc4076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:20.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.250+0000 7fdcc3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc41047e0 0x7fdcc4076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:20.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.250+0000 7fdcc3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc4108f60 0x7fdcc4077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:20.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.250+0000 7fdcc3e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4114010 con 0x7fdcc4108f60 2024-09-10T08:05:20.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.250+0000 7fdcc3e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4113e90 con 0x7fdcc4103e30 2024-09-10T08:05:20.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.250+0000 7fdcc3e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4114190 con 0x7fdcc41047e0 2024-09-10T08:05:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.250+0000 7fdcc2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc41047e0 0x7fdcc4076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.250+0000 7fdcc2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcc4103e30 0x7fdcc4076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.250+0000 7fdcc2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc41047e0 0x7fdcc4076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42642/0 (socket says 172.21.15.2:42642) 2024-09-10T08:05:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.250+0000 7fdcc2000640 1 -- 172.21.15.2:0/1269000488 learned_addr learned my addr 172.21.15.2:0/1269000488 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.250+0000 7fdcc3400640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc4108f60 0x7fdcc4077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.251+0000 7fdcc2000640 1 -- 172.21.15.2:0/1269000488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcc4103e30 msgr2=0x7fdcc4076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.251+0000 7fdcc2000640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcc4103e30 0x7fdcc4076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.251+0000 7fdcc2000640 1 -- 172.21.15.2:0/1269000488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc4108f60 msgr2=0x7fdcc4077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.251+0000 7fdcc2000640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc4108f60 0x7fdcc4077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.251+0000 7fdcc2000640 1 -- 172.21.15.2:0/1269000488 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc41aaad0 con 0x7fdcc41047e0 2024-09-10T08:05:20.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.251+0000 7fdcc2a00640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcc4103e30 0x7fdcc4076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:20.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.251+0000 7fdcc3400640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc4108f60 0x7fdcc4077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:20.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.251+0000 7fdcc2000640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc41047e0 0x7fdcc4076c40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000ea20 tx=0x7fdcb000eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:20.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.252+0000 7fdcbb400640 1 -- 172.21.15.2:0/1269000488 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb000cd80 con 0x7fdcc41047e0 2024-09-10T08:05:20.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.252+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1269000488 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc41aadc0 con 0x7fdcc41047e0 2024-09-10T08:05:20.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.252+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1269000488 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc41ab280 con 0x7fdcc41047e0 2024-09-10T08:05:20.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.252+0000 7fdcbb400640 1 -- 172.21.15.2:0/1269000488 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb0010910 con 0x7fdcc41047e0 2024-09-10T08:05:20.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.252+0000 7fdcbb400640 1 -- 172.21.15.2:0/1269000488 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0018920 con 0x7fdcc41047e0 2024-09-10T08:05:20.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.253+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1269000488 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca4005290 con 0x7fdcc41047e0 2024-09-10T08:05:20.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.258+0000 7fdcbb400640 1 -- 172.21.15.2:0/1269000488 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fdcb0010430 con 0x7fdcc41047e0 2024-09-10T08:05:20.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.260+0000 7fdcbb400640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdca80a2c00 0x7fdca80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:20.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.260+0000 7fdcbb400640 1 -- 172.21.15.2:0/1269000488 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fdcb00cedf0 con 0x7fdcc41047e0 2024-09-10T08:05:20.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.261+0000 7fdcc2a00640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdca80a2c00 0x7fdca80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:20.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.262+0000 7fdcc2a00640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdca80a2c00 0x7fdca80a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4000c00 tx=0x7fdcb4031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:20.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.263+0000 7fdcbb400640 1 -- 172.21.15.2:0/1269000488 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdcb009bb00 con 0x7fdcc41047e0 2024-09-10T08:05:20.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:20.470+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1269000488 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fdca4005580 con 0x7fdcc41047e0 2024-09-10T08:05:20.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:20 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:20.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:20 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:20.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:20 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:20.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:20 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:20.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:20 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:20.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:20 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:20.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:20 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:20.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:20 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:20.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:20 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:20.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:20 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:20.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:20 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:20.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:20 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:21.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:21.445+0000 7fdcbb400640 1 -- 172.21.15.2:0/1269000488 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7fdcb009fe00 con 0x7fdcc41047e0 2024-09-10T08:05:21.446 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' created 2024-09-10T08:05:21.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:21.450+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1269000488 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdca80a2c00 msgr2=0x7fdca80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:21.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:21.450+0000 7fdcc3e00640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdca80a2c00 0x7fdca80a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4000c00 tx=0x7fdcb4031510 comp rx=0 tx=0).stop 2024-09-10T08:05:21.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:21.450+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1269000488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc41047e0 msgr2=0x7fdcc4076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:21.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:21.450+0000 7fdcc3e00640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc41047e0 0x7fdcc4076c40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000ea20 tx=0x7fdcb000eee0 comp rx=0 tx=0).stop 2024-09-10T08:05:21.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:21.451+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1269000488 shutdown_connections 2024-09-10T08:05:21.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:21.451+0000 7fdcc3e00640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc4108f60 0x7fdcc4077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:21.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:21.451+0000 7fdcc3e00640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdca80a2c00 0x7fdca80a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:21.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:21.451+0000 7fdcc3e00640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc41047e0 0x7fdcc4076c40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:21.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:21.451+0000 7fdcc3e00640 1 --2- 172.21.15.2:0/1269000488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcc4103e30 0x7fdcc4076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:21.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:21.451+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1269000488 >> 172.21.15.2:0/1269000488 conn(0x7fdcc40ff910 msgr2=0x7fdcc4100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:21.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:21.451+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1269000488 shutdown_connections 2024-09-10T08:05:21.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:21.451+0000 7fdcc3e00640 1 -- 172.21.15.2:0/1269000488 wait complete. 2024-09-10T08:05:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:21 smithi002 ceph-mon[30563]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:21 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1269000488' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:05:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:21 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:05:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:05:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:21 smithi155 ceph-mon[35405]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:21 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1269000488' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:05:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:21 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:05:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:21.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:05:21.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:21.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:21 smithi107 ceph-mon[35048]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:21.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:21 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1269000488' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:05:21.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:21 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:05:21.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:21.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:21.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:05:21.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:05:21.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:05:22.353 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T08:05:22.744 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:22.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:22 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T08:05:22.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:22 smithi002 ceph-mon[30563]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T08:05:22.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:22 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T08:05:22.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:22 smithi155 ceph-mon[35405]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T08:05:22.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:22 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T08:05:22.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:22 smithi107 ceph-mon[35048]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T08:05:23.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.388+0000 7f5e43200640 1 -- 172.21.15.2:0/1038248732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e44108f60 msgr2=0x7f5e44110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:23.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.388+0000 7f5e43200640 1 --2- 172.21.15.2:0/1038248732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e44108f60 0x7f5e44110ae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009ca0 tx=0x7f5e3c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:05:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.388+0000 7f5e43200640 1 -- 172.21.15.2:0/1038248732 shutdown_connections 2024-09-10T08:05:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.388+0000 7f5e43200640 1 --2- 172.21.15.2:0/1038248732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e44108f60 0x7f5e44110ae0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.388+0000 7f5e43200640 1 --2- 172.21.15.2:0/1038248732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e441047e0 0x7f5e44108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.388+0000 7f5e43200640 1 --2- 172.21.15.2:0/1038248732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e44103e30 0x7f5e44104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.388+0000 7f5e43200640 1 -- 172.21.15.2:0/1038248732 >> 172.21.15.2:0/1038248732 conn(0x7f5e440ff8d0 msgr2=0x7f5e44101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.388+0000 7f5e43200640 1 -- 172.21.15.2:0/1038248732 shutdown_connections 2024-09-10T08:05:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.388+0000 7f5e43200640 1 -- 172.21.15.2:0/1038248732 wait complete. 2024-09-10T08:05:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.389+0000 7f5e43200640 1 Processor -- start 2024-09-10T08:05:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.390+0000 7f5e43200640 1 -- start start 2024-09-10T08:05:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.390+0000 7f5e43200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e44103e30 0x7f5e4419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:23.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.390+0000 7f5e43200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e441047e0 0x7f5e4419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:23.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.390+0000 7f5e43200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e44108f60 0x7f5e4419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:23.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.390+0000 7f5e43200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5e44113fc0 con 0x7f5e44103e30 2024-09-10T08:05:23.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.390+0000 7f5e43200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e44113e40 con 0x7f5e44108f60 2024-09-10T08:05:23.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.390+0000 7f5e43200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e44114140 con 0x7f5e441047e0 2024-09-10T08:05:23.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.390+0000 7f5e41e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e44103e30 0x7f5e4419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:23.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.390+0000 7f5e41e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e44103e30 0x7f5e4419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34410/0 (socket says 172.21.15.2:34410) 2024-09-10T08:05:23.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.390+0000 7f5e41e00640 1 -- 172.21.15.2:0/2325660784 learned_addr learned my addr 172.21.15.2:0/2325660784 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:23.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.390+0000 7f5e39400640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e441047e0 0x7f5e4419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:23.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.390+0000 7f5e42800640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e44108f60 0x7f5e4419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:23.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.391+0000 7f5e41e00640 1 -- 172.21.15.2:0/2325660784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e441047e0 msgr2=0x7f5e4419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:23.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.391+0000 7f5e41e00640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e441047e0 0x7f5e4419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:23.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.391+0000 7f5e41e00640 1 -- 172.21.15.2:0/2325660784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e44108f60 msgr2=0x7f5e4419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:23.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.391+0000 7f5e41e00640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e44108f60 0x7f5e4419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:23.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.391+0000 7f5e41e00640 1 -- 172.21.15.2:0/2325660784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4419f390 con 0x7f5e44103e30 2024-09-10T08:05:23.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.391+0000 7f5e42800640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e44108f60 0x7f5e4419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:23.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.391+0000 7f5e39400640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e441047e0 0x7f5e4419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:23.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.391+0000 7f5e41e00640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e44103e30 0x7f5e4419a3e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400ea60 tx=0x7f5e3400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:23.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.391+0000 7f5e3b400640 1 -- 172.21.15.2:0/2325660784 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400cde0 con 0x7f5e44103e30 2024-09-10T08:05:23.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.391+0000 7f5e3b400640 1 -- 172.21.15.2:0/2325660784 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e340035f0 con 0x7f5e44103e30 2024-09-10T08:05:23.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.392+0000 7f5e43200640 1 -- 172.21.15.2:0/2325660784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4419f680 con 0x7f5e44103e30 2024-09-10T08:05:23.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.392+0000 7f5e3b400640 1 -- 172.21.15.2:0/2325660784 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34010670 con 0x7f5e44103e30 2024-09-10T08:05:23.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.392+0000 7f5e43200640 1 -- 172.21.15.2:0/2325660784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e441a6f60 con 0x7f5e44103e30 2024-09-10T08:05:23.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.393+0000 7f5e43200640 1 -- 172.21.15.2:0/2325660784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e44104210 con 0x7f5e44103e30 2024-09-10T08:05:23.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.395+0000 7f5e3b400640 1 -- 172.21.15.2:0/2325660784 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5e340037a0 con 0x7f5e44103e30 2024-09-10T08:05:23.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.396+0000 7f5e3b400640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5e280a2c00 0x7f5e280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:23.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.396+0000 7f5e3b400640 1 -- 172.21.15.2:0/2325660784 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7f5e340cf0d0 con 0x7f5e44103e30 2024-09-10T08:05:23.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.396+0000 7f5e39400640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5e280a2c00 0x7f5e280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:23.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.397+0000 7f5e39400640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5e280a2c00 0x7f5e280a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e440ffcf0 tx=0x7f5e2c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:23.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.399+0000 7f5e3b400640 1 -- 172.21.15.2:0/2325660784 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e3409c350 con 0x7f5e44103e30 2024-09-10T08:05:23.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:23.587+0000 7f5e43200640 1 -- 172.21.15.2:0/2325660784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f5e4410c1a0 con 0x7f5e44103e30 2024-09-10T08:05:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:23 smithi002 ceph-mon[30563]: pgmap v287: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:23 smithi002 ceph-mon[30563]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:05:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:23 smithi002 ceph-mon[30563]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T08:05:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:23 smithi155 ceph-mon[35405]: pgmap v287: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:23 smithi155 ceph-mon[35405]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:05:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:23 smithi155 ceph-mon[35405]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T08:05:23.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:23 smithi107 ceph-mon[35048]: pgmap v287: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:23.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:23 smithi107 ceph-mon[35048]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:05:23.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:23 smithi107 ceph-mon[35048]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T08:05:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:24.459+0000 7f5e3b400640 1 -- 172.21.15.2:0/2325660784 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v81) ==== 171+0+0 (secure 0 0 0) 0x7f5e3409bc50 con 0x7f5e44103e30 2024-09-10T08:05:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' created 2024-09-10T08:05:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:24.463+0000 7f5e43200640 1 -- 172.21.15.2:0/2325660784 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5e280a2c00 msgr2=0x7f5e280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:24.463+0000 7f5e43200640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5e280a2c00 0x7f5e280a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e440ffcf0 tx=0x7f5e2c005f90 comp rx=0 tx=0).stop 2024-09-10T08:05:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:24.463+0000 7f5e43200640 1 -- 172.21.15.2:0/2325660784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e44103e30 msgr2=0x7f5e4419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:24.463+0000 7f5e43200640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e44103e30 0x7f5e4419a3e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400ea60 tx=0x7f5e3400ef20 comp rx=0 tx=0).stop 2024-09-10T08:05:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:24.463+0000 7f5e43200640 1 -- 172.21.15.2:0/2325660784 shutdown_connections 2024-09-10T08:05:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:24.463+0000 7f5e43200640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e44108f60 0x7f5e4419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:24.463+0000 7f5e43200640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5e280a2c00 0x7f5e280a50c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:24.463+0000 7f5e43200640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e441047e0 0x7f5e4419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:24.463+0000 7f5e43200640 1 --2- 172.21.15.2:0/2325660784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e44103e30 0x7f5e4419a3e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:24.463+0000 7f5e43200640 1 -- 172.21.15.2:0/2325660784 >> 172.21.15.2:0/2325660784 conn(0x7f5e440ff8d0 msgr2=0x7f5e44100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:24.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:24.463+0000 7f5e43200640 1 -- 172.21.15.2:0/2325660784 shutdown_connections 2024-09-10T08:05:24.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:24.464+0000 7f5e43200640 1 -- 172.21.15.2:0/2325660784 wait complete. 2024-09-10T08:05:24.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:24 smithi002 ceph-mon[30563]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T08:05:24.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:24 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2325660784' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T08:05:24.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:24 smithi155 ceph-mon[35405]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T08:05:24.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:24 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2325660784' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T08:05:24.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:24 smithi107 ceph-mon[35048]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T08:05:24.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:24 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2325660784' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T08:05:25.397 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T08:05:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:25 smithi002 ceph-mon[30563]: pgmap v290: 33 pgs: 1 creating+activating, 8 creating+peering, 21 unknown, 3 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:25 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2325660784' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T08:05:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:25 smithi002 ceph-mon[30563]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T08:05:25.794 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:25 smithi155 ceph-mon[35405]: pgmap v290: 33 pgs: 1 creating+activating, 8 creating+peering, 21 unknown, 3 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:25 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2325660784' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T08:05:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:25 smithi155 ceph-mon[35405]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T08:05:25.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:25 smithi107 ceph-mon[35048]: pgmap v290: 33 pgs: 1 creating+activating, 8 creating+peering, 21 unknown, 3 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:25.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:25 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2325660784' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T08:05:25.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:25 smithi107 ceph-mon[35048]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T08:05:26.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.333+0000 7fa92a400640 1 -- 172.21.15.2:0/2301374128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa924076720 msgr2=0x7fa924076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:26.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.333+0000 7fa92a400640 1 --2- 172.21.15.2:0/2301374128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa924076720 0x7fa924076b00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa918009900 tx=0x7fa91802a400 comp rx=0 tx=0).stop 2024-09-10T08:05:26.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.333+0000 7fa92a400640 1 -- 172.21.15.2:0/2301374128 shutdown_connections 2024-09-10T08:05:26.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.333+0000 7fa92a400640 1 --2- 172.21.15.2:0/2301374128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa924112890 0x7fa924114c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:26.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.333+0000 7fa92a400640 1 --2- 172.21.15.2:0/2301374128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa924077040 0x7fa924112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:26.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.333+0000 7fa92a400640 1 --2- 172.21.15.2:0/2301374128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa924076720 0x7fa924076b00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:26.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.333+0000 7fa92a400640 1 -- 172.21.15.2:0/2301374128 >> 172.21.15.2:0/2301374128 conn(0x7fa9240ff8b0 msgr2=0x7fa924101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:26.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.333+0000 7fa92a400640 1 -- 172.21.15.2:0/2301374128 shutdown_connections 2024-09-10T08:05:26.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.334+0000 7fa92a400640 1 -- 172.21.15.2:0/2301374128 wait complete. 2024-09-10T08:05:26.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.334+0000 7fa92a400640 1 Processor -- start 2024-09-10T08:05:26.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.335+0000 7fa92a400640 1 -- start start 2024-09-10T08:05:26.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.335+0000 7fa92a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa924076720 0x7fa9241a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:26.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.335+0000 7fa92a400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa924077040 0x7fa9241a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:26.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.335+0000 7fa92a400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa924112890 0x7fa92419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:26.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.335+0000 7fa92a400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa924118570 con 0x7fa924112890 2024-09-10T08:05:26.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.335+0000 7fa92a400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa9241183f0 con 0x7fa924076720 2024-09-10T08:05:26.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.335+0000 7fa92a400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9241186f0 con 0x7fa924077040 2024-09-10T08:05:26.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.335+0000 7fa929000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa924076720 0x7fa9241a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.335+0000 7fa929a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa924112890 0x7fa92419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.335+0000 7fa929000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa924076720 0x7fa9241a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:56346/0 (socket says 172.21.15.2:56346) 2024-09-10T08:05:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.335+0000 7fa929000640 1 -- 172.21.15.2:0/406983340 learned_addr learned my addr 172.21.15.2:0/406983340 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.335+0000 7fa923e00640 1 --2- 172.21.15.2:0/406983340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa924077040 0x7fa9241a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.336+0000 7fa929a00640 1 -- 172.21.15.2:0/406983340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa924077040 msgr2=0x7fa9241a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.336+0000 7fa929a00640 1 --2- 172.21.15.2:0/406983340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa924077040 0x7fa9241a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.336+0000 7fa929a00640 1 -- 172.21.15.2:0/406983340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa924076720 msgr2=0x7fa9241a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.336+0000 7fa929a00640 1 --2- 172.21.15.2:0/406983340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa924076720 0x7fa9241a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.336+0000 7fa929a00640 1 -- 172.21.15.2:0/406983340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92419efd0 con 0x7fa924112890 2024-09-10T08:05:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.336+0000 7fa929a00640 1 --2- 172.21.15.2:0/406983340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa924112890 0x7fa92419e8a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa92407aa10 tx=0x7fa91c00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.336+0000 7fa921600640 1 -- 172.21.15.2:0/406983340 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c004d20 con 0x7fa924112890 2024-09-10T08:05:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.336+0000 7fa92a400640 1 -- 172.21.15.2:0/406983340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92419f2c0 con 0x7fa924112890 2024-09-10T08:05:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.336+0000 7fa921600640 1 -- 172.21.15.2:0/406983340 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa91c005020 con 0x7fa924112890 2024-09-10T08:05:26.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.336+0000 7fa921600640 1 -- 172.21.15.2:0/406983340 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0116d0 con 0x7fa924112890 2024-09-10T08:05:26.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.337+0000 7fa92a400640 1 -- 172.21.15.2:0/406983340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9241ab5b0 con 0x7fa924112890 2024-09-10T08:05:26.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.338+0000 7fa92a400640 1 -- 172.21.15.2:0/406983340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa924077e60 con 0x7fa924112890 2024-09-10T08:05:26.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.340+0000 7fa921600640 1 -- 172.21.15.2:0/406983340 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa91c005540 con 0x7fa924112890 2024-09-10T08:05:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.341+0000 7fa921600640 1 --2- 172.21.15.2:0/406983340 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa90c0a2b80 0x7fa90c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.341+0000 7fa921600640 1 -- 172.21.15.2:0/406983340 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7879+0+0 (secure 0 0 0) 0x7fa91c0cef70 con 0x7fa924112890 2024-09-10T08:05:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.341+0000 7fa929000640 1 --2- 172.21.15.2:0/406983340 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa90c0a2b80 0x7fa90c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:26.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.342+0000 7fa929000640 1 --2- 172.21.15.2:0/406983340 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa90c0a2b80 0x7fa90c0a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa9180096c0 tx=0x7fa9180046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:26.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.346+0000 7fa921600640 1 -- 172.21.15.2:0/406983340 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa91c016030 con 0x7fa924112890 2024-09-10T08:05:26.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:26.572+0000 7fa92a400640 1 -- 172.21.15.2:0/406983340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa92410da20 con 0x7fa924112890 2024-09-10T08:05:26.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:26 smithi002 ceph-mon[30563]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T08:05:26.909 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:26 smithi155 ceph-mon[35405]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T08:05:26.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:26 smithi107 ceph-mon[35048]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T08:05:27.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:27.487+0000 7fa921600640 1 -- 172.21.15.2:0/406983340 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa91c09b990 con 0x7fa924112890 2024-09-10T08:05:27.487 INFO:teuthology.orchestra.run.smithi002.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T08:05:27.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:27.490+0000 7fa92a400640 1 -- 172.21.15.2:0/406983340 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa90c0a2b80 msgr2=0x7fa90c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:27.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:27.490+0000 7fa92a400640 1 --2- 172.21.15.2:0/406983340 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa90c0a2b80 0x7fa90c0a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa9180096c0 tx=0x7fa9180046c0 comp rx=0 tx=0).stop 2024-09-10T08:05:27.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:27.490+0000 7fa92a400640 1 -- 172.21.15.2:0/406983340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa924112890 msgr2=0x7fa92419e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:27.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:27.490+0000 7fa92a400640 1 --2- 172.21.15.2:0/406983340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa924112890 0x7fa92419e8a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa92407aa10 tx=0x7fa91c00edd0 comp rx=0 tx=0).stop 2024-09-10T08:05:27.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:27.490+0000 7fa92a400640 1 -- 172.21.15.2:0/406983340 shutdown_connections 2024-09-10T08:05:27.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:27.490+0000 7fa92a400640 1 --2- 172.21.15.2:0/406983340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa924112890 0x7fa92419e8a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:27.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:27.490+0000 7fa92a400640 1 --2- 172.21.15.2:0/406983340 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa90c0a2b80 0x7fa90c0a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:27.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:27.490+0000 7fa92a400640 1 --2- 172.21.15.2:0/406983340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa924077040 0x7fa9241a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:27.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:27.490+0000 7fa92a400640 1 --2- 172.21.15.2:0/406983340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa924076720 0x7fa9241a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:27.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:27.490+0000 7fa92a400640 1 -- 172.21.15.2:0/406983340 >> 172.21.15.2:0/406983340 conn(0x7fa9240ff8b0 msgr2=0x7fa924101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:27.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:27.490+0000 7fa92a400640 1 -- 172.21.15.2:0/406983340 shutdown_connections 2024-09-10T08:05:27.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:27.490+0000 7fa92a400640 1 -- 172.21.15.2:0/406983340 wait complete. 2024-09-10T08:05:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:27 smithi002 ceph-mon[30563]: pgmap v293: 97 pgs: 1 creating+activating, 34 creating+peering, 39 unknown, 23 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:27 smithi002 ceph-mon[30563]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T08:05:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:27 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/406983340' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T08:05:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:27 smithi002 ceph-mon[30563]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T08:05:27.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:27 smithi155 ceph-mon[35405]: pgmap v293: 97 pgs: 1 creating+activating, 34 creating+peering, 39 unknown, 23 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:27.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:27 smithi155 ceph-mon[35405]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T08:05:27.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:27 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/406983340' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T08:05:27.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:27 smithi155 ceph-mon[35405]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T08:05:27.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:27 smithi107 ceph-mon[35048]: pgmap v293: 97 pgs: 1 creating+activating, 34 creating+peering, 39 unknown, 23 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:27.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:27 smithi107 ceph-mon[35048]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T08:05:27.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:27 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/406983340' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T08:05:27.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:27 smithi107 ceph-mon[35048]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T08:05:28.466 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T08:05:28.467 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T08:05:28.523 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:28 smithi002 ceph-mon[30563]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T08:05:28.523 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:28 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/406983340' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T08:05:28.523 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:28 smithi002 ceph-mon[30563]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T08:05:28.523 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:28 smithi002 ceph-mon[30563]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T08:05:28.853 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:28.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:28 smithi155 ceph-mon[35405]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T08:05:28.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:28 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/406983340' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T08:05:28.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:28 smithi155 ceph-mon[35405]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T08:05:28.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:28 smithi155 ceph-mon[35405]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T08:05:28.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:28 smithi107 ceph-mon[35048]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T08:05:28.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:28 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/406983340' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T08:05:28.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:28 smithi107 ceph-mon[35048]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T08:05:28.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:28 smithi107 ceph-mon[35048]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T08:05:29.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.411+0000 7fd4ede00640 1 -- 172.21.15.2:0/2320413489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4e8074ee0 msgr2=0x7fd4e810dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:29.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.411+0000 7fd4ede00640 1 --2- 172.21.15.2:0/2320413489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4e8074ee0 0x7fd4e810dee0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009900 tx=0x7fd4d002a480 comp rx=0 tx=0).stop 2024-09-10T08:05:29.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.411+0000 7fd4ede00640 1 -- 172.21.15.2:0/2320413489 shutdown_connections 2024-09-10T08:05:29.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.411+0000 7fd4ede00640 1 --2- 172.21.15.2:0/2320413489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4e810e420 0x7fd4e8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:29.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.411+0000 7fd4ede00640 1 --2- 172.21.15.2:0/2320413489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4e8074ee0 0x7fd4e810dee0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:29.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.411+0000 7fd4ede00640 1 --2- 172.21.15.2:0/2320413489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e80745c0 0x7fd4e80749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:29.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.411+0000 7fd4ede00640 1 -- 172.21.15.2:0/2320413489 >> 172.21.15.2:0/2320413489 conn(0x7fd4e80fd8c0 msgr2=0x7fd4e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:29.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.411+0000 7fd4ede00640 1 -- 172.21.15.2:0/2320413489 shutdown_connections 2024-09-10T08:05:29.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.411+0000 7fd4ede00640 1 -- 172.21.15.2:0/2320413489 wait complete. 2024-09-10T08:05:29.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.412+0000 7fd4ede00640 1 Processor -- start 2024-09-10T08:05:29.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.412+0000 7fd4ede00640 1 -- start start 2024-09-10T08:05:29.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.412+0000 7fd4ede00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e80745c0 0x7fd4e819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:29.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.412+0000 7fd4ede00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4e8074ee0 0x7fd4e819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:29.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.412+0000 7fd4ede00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4e810e420 0x7fd4e819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:29.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.412+0000 7fd4ede00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8114160 con 0x7fd4e8074ee0 2024-09-10T08:05:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.412+0000 7fd4ede00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8113fe0 con 0x7fd4e80745c0 2024-09-10T08:05:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.412+0000 7fd4ede00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81142e0 con 0x7fd4e810e420 2024-09-10T08:05:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.412+0000 7fd4dfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4e8074ee0 0x7fd4e819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.412+0000 7fd4e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e80745c0 0x7fd4e819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.412+0000 7fd4e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e80745c0 0x7fd4e819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:57556/0 (socket says 172.21.15.2:57556) 2024-09-10T08:05:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.412+0000 7fd4e6a00640 1 -- 172.21.15.2:0/1438572778 learned_addr learned my addr 172.21.15.2:0/1438572778 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.413+0000 7fd4e7400640 1 --2- 172.21.15.2:0/1438572778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4e810e420 0x7fd4e819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.413+0000 7fd4dfe00640 1 -- 172.21.15.2:0/1438572778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4e810e420 msgr2=0x7fd4e819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.413+0000 7fd4dfe00640 1 --2- 172.21.15.2:0/1438572778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4e810e420 0x7fd4e819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.413+0000 7fd4dfe00640 1 -- 172.21.15.2:0/1438572778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e80745c0 msgr2=0x7fd4e819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.413+0000 7fd4dfe00640 1 --2- 172.21.15.2:0/1438572778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e80745c0 0x7fd4e819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.413+0000 7fd4dfe00640 1 -- 172.21.15.2:0/1438572778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e819f630 con 0x7fd4e8074ee0 2024-09-10T08:05:29.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.413+0000 7fd4dfe00640 1 --2- 172.21.15.2:0/1438572778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4e8074ee0 0x7fd4e819abc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005940 tx=0x7fd4d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:29.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.414+0000 7fd4eca00640 1 -- 172.21.15.2:0/1438572778 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d003f030 con 0x7fd4e8074ee0 2024-09-10T08:05:29.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.414+0000 7fd4eca00640 1 -- 172.21.15.2:0/1438572778 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d0003ad0 con 0x7fd4e8074ee0 2024-09-10T08:05:29.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.414+0000 7fd4eca00640 1 -- 172.21.15.2:0/1438572778 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0035070 con 0x7fd4e8074ee0 2024-09-10T08:05:29.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.414+0000 7fd4ede00640 1 -- 172.21.15.2:0/1438572778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e819f8c0 con 0x7fd4e8074ee0 2024-09-10T08:05:29.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.414+0000 7fd4ede00640 1 -- 172.21.15.2:0/1438572778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e819fd50 con 0x7fd4e8074ee0 2024-09-10T08:05:29.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.415+0000 7fd4ede00640 1 -- 172.21.15.2:0/1438572778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e80749a0 con 0x7fd4e8074ee0 2024-09-10T08:05:29.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.417+0000 7fd4eca00640 1 -- 172.21.15.2:0/1438572778 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd4d0042030 con 0x7fd4e8074ee0 2024-09-10T08:05:29.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.418+0000 7fd4eca00640 1 --2- 172.21.15.2:0/1438572778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4cc0a2c00 0x7fd4cc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:29.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.418+0000 7fd4eca00640 1 -- 172.21.15.2:0/1438572778 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fd4d00ef4c0 con 0x7fd4e8074ee0 2024-09-10T08:05:29.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.418+0000 7fd4e6a00640 1 --2- 172.21.15.2:0/1438572778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4cc0a2c00 0x7fd4cc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:29.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.419+0000 7fd4e6a00640 1 --2- 172.21.15.2:0/1438572778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4cc0a2c00 0x7fd4cc0a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009c90 tx=0x7fd4d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:29.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.422+0000 7fd4eca00640 1 -- 172.21.15.2:0/1438572778 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4d00bbe80 con 0x7fd4e8074ee0 2024-09-10T08:05:29.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:29.615+0000 7fd4ede00640 1 -- 172.21.15.2:0/1438572778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fd4e802d7c0 con 0x7fd4e8074ee0 2024-09-10T08:05:29.615 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:29 smithi002 ceph-mon[30563]: pgmap v296: 97 pgs: 26 creating+peering, 2 unknown, 69 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T08:05:29.615 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:29 smithi002 ceph-mon[30563]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T08:05:29.615 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:29 smithi002 ceph-mon[30563]: Cluster is now healthy 2024-09-10T08:05:29.615 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:29 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] up:active 2024-09-10T08:05:29.615 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:29 smithi002 ceph-mon[30563]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:05:29.615 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:29 smithi002 ceph-mon[30563]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:05:29.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:29 smithi107 ceph-mon[35048]: pgmap v296: 97 pgs: 26 creating+peering, 2 unknown, 69 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T08:05:29.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:29 smithi107 ceph-mon[35048]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T08:05:29.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:29 smithi107 ceph-mon[35048]: Cluster is now healthy 2024-09-10T08:05:29.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:29 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] up:active 2024-09-10T08:05:29.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:29 smithi107 ceph-mon[35048]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:05:29.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:29 smithi107 ceph-mon[35048]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:05:29.667 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:05:29 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-b[64493]: 2024-09-10T08:05:29.266+0000 7fe29f1a5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T08:05:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:29 smithi155 ceph-mon[35405]: pgmap v296: 97 pgs: 26 creating+peering, 2 unknown, 69 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T08:05:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:29 smithi155 ceph-mon[35405]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T08:05:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:29 smithi155 ceph-mon[35405]: Cluster is now healthy 2024-09-10T08:05:29.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:29 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] up:active 2024-09-10T08:05:29.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:29 smithi155 ceph-mon[35405]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:05:29.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:29 smithi155 ceph-mon[35405]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:05:30.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:30.493+0000 7fd4eca00640 1 -- 172.21.15.2:0/1438572778 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7fd4d00c0180 con 0x7fd4e8074ee0 2024-09-10T08:05:30.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:30.496+0000 7fd4ede00640 1 -- 172.21.15.2:0/1438572778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4cc0a2c00 msgr2=0x7fd4cc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:30.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:30.496+0000 7fd4ede00640 1 --2- 172.21.15.2:0/1438572778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4cc0a2c00 0x7fd4cc0a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009c90 tx=0x7fd4d8005f90 comp rx=0 tx=0).stop 2024-09-10T08:05:30.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:30.496+0000 7fd4ede00640 1 -- 172.21.15.2:0/1438572778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4e8074ee0 msgr2=0x7fd4e819abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:30.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:30.496+0000 7fd4ede00640 1 --2- 172.21.15.2:0/1438572778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4e8074ee0 0x7fd4e819abc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005940 tx=0x7fd4d0004270 comp rx=0 tx=0).stop 2024-09-10T08:05:30.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:30.497+0000 7fd4ede00640 1 -- 172.21.15.2:0/1438572778 shutdown_connections 2024-09-10T08:05:30.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:30.497+0000 7fd4ede00640 1 --2- 172.21.15.2:0/1438572778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4cc0a2c00 0x7fd4cc0a50c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:30.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:30.497+0000 7fd4ede00640 1 --2- 172.21.15.2:0/1438572778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4e810e420 0x7fd4e819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:30.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:30.497+0000 7fd4ede00640 1 --2- 172.21.15.2:0/1438572778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4e8074ee0 0x7fd4e819abc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:30.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:30.497+0000 7fd4ede00640 1 --2- 172.21.15.2:0/1438572778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e80745c0 0x7fd4e819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:30.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:30.497+0000 7fd4ede00640 1 -- 172.21.15.2:0/1438572778 >> 172.21.15.2:0/1438572778 conn(0x7fd4e80fd8c0 msgr2=0x7fd4e80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:30.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:30.497+0000 7fd4ede00640 1 -- 172.21.15.2:0/1438572778 shutdown_connections 2024-09-10T08:05:30.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:30.497+0000 7fd4ede00640 1 -- 172.21.15.2:0/1438572778 wait complete. 2024-09-10T08:05:30.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:30 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1438572778' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T08:05:30.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:30 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1438572778' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T08:05:30.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:30 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1438572778' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T08:05:31.362 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T08:05:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:31.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:31 smithi002 ceph-mon[30563]: pgmap v297: 97 pgs: 26 creating+peering, 71 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 197 B/s wr, 1 op/s 2024-09-10T08:05:31.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:31 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1438572778' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T08:05:31.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:31 smithi002 ceph-mon[30563]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T08:05:31.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:31 smithi155 ceph-mon[35405]: pgmap v297: 97 pgs: 26 creating+peering, 71 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 197 B/s wr, 1 op/s 2024-09-10T08:05:31.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:31 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1438572778' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T08:05:31.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:31 smithi155 ceph-mon[35405]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T08:05:31.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:31 smithi107 ceph-mon[35048]: pgmap v297: 97 pgs: 26 creating+peering, 71 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 197 B/s wr, 1 op/s 2024-09-10T08:05:31.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:31 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1438572778' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T08:05:31.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:31 smithi107 ceph-mon[35048]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T08:05:32.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.350+0000 7f0dce600640 1 -- 172.21.15.2:0/3024849947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8103ff0 msgr2=0x7f0dc81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:32.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.350+0000 7f0dce600640 1 --2- 172.21.15.2:0/3024849947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8103ff0 0x7f0dc81043d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009860 tx=0x7f0db802a340 comp rx=0 tx=0).stop 2024-09-10T08:05:32.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.350+0000 7f0dce600640 1 -- 172.21.15.2:0/3024849947 shutdown_connections 2024-09-10T08:05:32.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.350+0000 7f0dce600640 1 --2- 172.21.15.2:0/3024849947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0dc8109120 0x7f0dc8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:32.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.350+0000 7f0dce600640 1 --2- 172.21.15.2:0/3024849947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dc81049a0 0x7f0dc81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:32.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.350+0000 7f0dce600640 1 --2- 172.21.15.2:0/3024849947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8103ff0 0x7f0dc81043d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.350+0000 7f0dce600640 1 -- 172.21.15.2:0/3024849947 >> 172.21.15.2:0/3024849947 conn(0x7f0dc80ffa90 msgr2=0x7f0dc8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.350+0000 7f0dce600640 1 -- 172.21.15.2:0/3024849947 shutdown_connections 2024-09-10T08:05:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.350+0000 7f0dce600640 1 -- 172.21.15.2:0/3024849947 wait complete. 2024-09-10T08:05:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.351+0000 7f0dce600640 1 Processor -- start 2024-09-10T08:05:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.351+0000 7f0dce600640 1 -- start start 2024-09-10T08:05:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.351+0000 7f0dce600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8103ff0 0x7f0dc819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.351+0000 7f0dce600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dc81049a0 0x7f0dc819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.351+0000 7f0dce600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0dc8109120 0x7f0dc819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dce600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8114120 con 0x7f0dc81049a0 2024-09-10T08:05:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dce600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8113fa0 con 0x7f0dc8103ff0 2024-09-10T08:05:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dce600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81142a0 con 0x7f0dc8109120 2024-09-10T08:05:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dc7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8103ff0 0x7f0dc819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dc7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8103ff0 0x7f0dc819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:57576/0 (socket says 172.21.15.2:57576) 2024-09-10T08:05:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dc7400640 1 -- 172.21.15.2:0/3136366257 learned_addr learned my addr 172.21.15.2:0/3136366257 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dbea00640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dc81049a0 0x7f0dc819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dc7e00640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0dc8109120 0x7f0dc819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dc7e00640 1 -- 172.21.15.2:0/3136366257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8103ff0 msgr2=0x7f0dc819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dc7e00640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8103ff0 0x7f0dc819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dc7e00640 1 -- 172.21.15.2:0/3136366257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dc81049a0 msgr2=0x7f0dc819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dc7e00640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dc81049a0 0x7f0dc819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dc7e00640 1 -- 172.21.15.2:0/3136366257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc819f590 con 0x7f0dc8109120 2024-09-10T08:05:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.352+0000 7f0dbea00640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dc81049a0 0x7f0dc819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.353+0000 7f0dc7e00640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0dc8109120 0x7f0dc819eeb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009fd0 tx=0x7f0dc000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.353+0000 7f0dc4c00640 1 -- 172.21.15.2:0/3136366257 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc000cd60 con 0x7f0dc8109120 2024-09-10T08:05:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.353+0000 7f0dc4c00640 1 -- 172.21.15.2:0/3136366257 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc0005020 con 0x7f0dc8109120 2024-09-10T08:05:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.353+0000 7f0dce600640 1 -- 172.21.15.2:0/3136366257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc819f880 con 0x7f0dc8109120 2024-09-10T08:05:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.353+0000 7f0dc4c00640 1 -- 172.21.15.2:0/3136366257 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0010690 con 0x7f0dc8109120 2024-09-10T08:05:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.353+0000 7f0dce600640 1 -- 172.21.15.2:0/3136366257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc81a7160 con 0x7f0dc8109120 2024-09-10T08:05:32.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.354+0000 7f0dce600640 1 -- 172.21.15.2:0/3136366257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc81043d0 con 0x7f0dc8109120 2024-09-10T08:05:32.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.356+0000 7f0dc4c00640 1 -- 172.21.15.2:0/3136366257 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0dc0005540 con 0x7f0dc8109120 2024-09-10T08:05:32.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.357+0000 7f0dc4c00640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0dac0a2b80 0x7f0dac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:32.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.358+0000 7f0dc4c00640 1 -- 172.21.15.2:0/3136366257 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f0dc00cf670 con 0x7f0dc8109120 2024-09-10T08:05:32.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.358+0000 7f0dc7400640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0dac0a2b80 0x7f0dac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:32.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.358+0000 7f0dc7400640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0dac0a2b80 0x7f0dac0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0db80095f0 tx=0x7f0db80056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:32.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.360+0000 7f0dc4c00640 1 -- 172.21.15.2:0/3136366257 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0dc009c540 con 0x7f0dc8109120 2024-09-10T08:05:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:32.548+0000 7f0dce600640 1 -- 172.21.15.2:0/3136366257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0dc819b940 con 0x7f0dc8109120 2024-09-10T08:05:34.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:34 smithi155 ceph-mon[35405]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-10T08:05:34.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:34 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3136366257' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T08:05:34.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:34 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T08:05:34.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:34 smithi107 ceph-mon[35048]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-10T08:05:34.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:34 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3136366257' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T08:05:34.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:34 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T08:05:34.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:34.514+0000 7f0dc4c00640 1 -- 172.21.15.2:0/3136366257 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f0dc819b940 con 0x7f0dc8109120 2024-09-10T08:05:34.515 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data_ec' created 2024-09-10T08:05:34.515 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:34 smithi002 ceph-mon[30563]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-10T08:05:34.516 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:34 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3136366257' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T08:05:34.516 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:34 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T08:05:34.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:34.519+0000 7f0dce600640 1 -- 172.21.15.2:0/3136366257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0dac0a2b80 msgr2=0x7f0dac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:34.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:34.519+0000 7f0dce600640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0dac0a2b80 0x7f0dac0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0db80095f0 tx=0x7f0db80056d0 comp rx=0 tx=0).stop 2024-09-10T08:05:34.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:34.519+0000 7f0dce600640 1 -- 172.21.15.2:0/3136366257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0dc8109120 msgr2=0x7f0dc819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:34.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:34.519+0000 7f0dce600640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0dc8109120 0x7f0dc819eeb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009fd0 tx=0x7f0dc000ee50 comp rx=0 tx=0).stop 2024-09-10T08:05:34.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:34.520+0000 7f0dce600640 1 -- 172.21.15.2:0/3136366257 shutdown_connections 2024-09-10T08:05:34.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:34.520+0000 7f0dce600640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0dac0a2b80 0x7f0dac0a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:34.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:34.520+0000 7f0dce600640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0dc8109120 0x7f0dc819eeb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:34.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:34.520+0000 7f0dce600640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0dc81049a0 0x7f0dc819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:34.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:34.520+0000 7f0dce600640 1 --2- 172.21.15.2:0/3136366257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dc8103ff0 0x7f0dc819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:34.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:34.520+0000 7f0dce600640 1 -- 172.21.15.2:0/3136366257 >> 172.21.15.2:0/3136366257 conn(0x7f0dc80ffa90 msgr2=0x7f0dc8100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:34.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:34.520+0000 7f0dce600640 1 -- 172.21.15.2:0/3136366257 shutdown_connections 2024-09-10T08:05:34.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:34.520+0000 7f0dce600640 1 -- 172.21.15.2:0/3136366257 wait complete. 2024-09-10T08:05:35.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:35 smithi002 ceph-mon[30563]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T08:05:35.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:35 smithi002 ceph-mon[30563]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T08:05:35.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:35 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T08:05:35.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:35 smithi002 ceph-mon[30563]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T08:05:35.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:35 smithi155 ceph-mon[35405]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T08:05:35.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:35 smithi155 ceph-mon[35405]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T08:05:35.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:35 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T08:05:35.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:35 smithi155 ceph-mon[35405]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T08:05:35.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:35 smithi107 ceph-mon[35048]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T08:05:35.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:35 smithi107 ceph-mon[35048]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T08:05:35.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:35 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T08:05:35.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:35 smithi107 ceph-mon[35048]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T08:05:35.480 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T08:05:35.873 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.459+0000 7f7744c00640 1 -- 172.21.15.2:0/2095167641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7740112870 msgr2=0x7f7740114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.459+0000 7f7744c00640 1 --2- 172.21.15.2:0/2095167641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7740112870 0x7f7740114c60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7738009ca0 tx=0x7f773802a8b0 comp rx=0 tx=0).stop 2024-09-10T08:05:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.459+0000 7f7744c00640 1 -- 172.21.15.2:0/2095167641 shutdown_connections 2024-09-10T08:05:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.459+0000 7f7744c00640 1 --2- 172.21.15.2:0/2095167641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7740112870 0x7f7740114c60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.459+0000 7f7744c00640 1 --2- 172.21.15.2:0/2095167641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7740077040 0x7f7740112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.459+0000 7f7744c00640 1 --2- 172.21.15.2:0/2095167641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7740076720 0x7f7740076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.459+0000 7f7744c00640 1 -- 172.21.15.2:0/2095167641 >> 172.21.15.2:0/2095167641 conn(0x7f77400ff910 msgr2=0x7f7740101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.459+0000 7f7744c00640 1 -- 172.21.15.2:0/2095167641 shutdown_connections 2024-09-10T08:05:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.459+0000 7f7744c00640 1 -- 172.21.15.2:0/2095167641 wait complete. 2024-09-10T08:05:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.460+0000 7f7744c00640 1 Processor -- start 2024-09-10T08:05:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.460+0000 7f7744c00640 1 -- start start 2024-09-10T08:05:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.461+0000 7f7744c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7740076720 0x7f77401a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.461+0000 7f7744c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7740077040 0x7f77401a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.461+0000 7f7744c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7740112870 0x7f774019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.461+0000 7f7744c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7740118400 con 0x7f7740077040 2024-09-10T08:05:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.461+0000 7f7744c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7740118280 con 0x7f7740076720 2024-09-10T08:05:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.461+0000 7f7744c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7740118580 con 0x7f7740112870 2024-09-10T08:05:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.461+0000 7f7736a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7740077040 0x7f77401a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.461+0000 7f7736a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7740077040 0x7f77401a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35468/0 (socket says 172.21.15.2:35468) 2024-09-10T08:05:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.461+0000 7f7736a00640 1 -- 172.21.15.2:0/2354356925 learned_addr learned my addr 172.21.15.2:0/2354356925 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.461+0000 7f773f400640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7740076720 0x7f77401a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.461+0000 7f773fe00640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7740112870 0x7f774019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.462+0000 7f7736a00640 1 -- 172.21.15.2:0/2354356925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7740112870 msgr2=0x7f774019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.462+0000 7f7736a00640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7740112870 0x7f774019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.462+0000 7f7736a00640 1 -- 172.21.15.2:0/2354356925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7740076720 msgr2=0x7f77401a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.462+0000 7f7736a00640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7740076720 0x7f77401a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.462+0000 7f7736a00640 1 -- 172.21.15.2:0/2354356925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774019ef80 con 0x7f7740077040 2024-09-10T08:05:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.462+0000 7f773f400640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7740076720 0x7f77401a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.462+0000 7f7736a00640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7740077040 0x7f77401a4b80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f772800d950 tx=0x7f772800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.462+0000 7f773cc00640 1 -- 172.21.15.2:0/2354356925 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728017030 con 0x7f7740077040 2024-09-10T08:05:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.462+0000 7f773cc00640 1 -- 172.21.15.2:0/2354356925 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7728004020 con 0x7f7740077040 2024-09-10T08:05:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.462+0000 7f773cc00640 1 -- 172.21.15.2:0/2354356925 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77280047a0 con 0x7f7740077040 2024-09-10T08:05:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.462+0000 7f7744c00640 1 -- 172.21.15.2:0/2354356925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774019f270 con 0x7f7740077040 2024-09-10T08:05:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.462+0000 7f7744c00640 1 -- 172.21.15.2:0/2354356925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77401ab5c0 con 0x7f7740077040 2024-09-10T08:05:36.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.463+0000 7f7744c00640 1 -- 172.21.15.2:0/2354356925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7740077e60 con 0x7f7740077040 2024-09-10T08:05:36.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.466+0000 7f773cc00640 1 -- 172.21.15.2:0/2354356925 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f77280041d0 con 0x7f7740077040 2024-09-10T08:05:36.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.467+0000 7f773cc00640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f77240a2b80 0x7f77240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:36.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.468+0000 7f773cc00640 1 -- 172.21.15.2:0/2354356925 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f77280cf200 con 0x7f7740077040 2024-09-10T08:05:36.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.468+0000 7f773f400640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f77240a2b80 0x7f77240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:36.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.468+0000 7f773f400640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f77240a2b80 0x7f77240a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f77300095f0 tx=0x7f7730005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:36.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.470+0000 7f773cc00640 1 -- 172.21.15.2:0/2354356925 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f772809bee0 con 0x7f7740077040 2024-09-10T08:05:36.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:36.654+0000 7f7744c00640 1 -- 172.21.15.2:0/2354356925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f774010da00 con 0x7f7740077040 2024-09-10T08:05:36.654 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:36 smithi002 ceph-mon[30563]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T08:05:36.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:36 smithi155 ceph-mon[35405]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T08:05:36.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:36 smithi107 ceph-mon[35048]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T08:05:37.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:37.519+0000 7f773cc00640 1 -- 172.21.15.2:0/2354356925 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v90) ==== 169+0+0 (secure 0 0 0) 0x7f772809b7e0 con 0x7f7740077040 2024-09-10T08:05:37.519 INFO:teuthology.orchestra.run.smithi002.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T08:05:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:37.522+0000 7f7744c00640 1 -- 172.21.15.2:0/2354356925 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f77240a2b80 msgr2=0x7f77240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:37.522+0000 7f7744c00640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f77240a2b80 0x7f77240a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f77300095f0 tx=0x7f7730005f90 comp rx=0 tx=0).stop 2024-09-10T08:05:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:37.522+0000 7f7744c00640 1 -- 172.21.15.2:0/2354356925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7740077040 msgr2=0x7f77401a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:37.522+0000 7f7744c00640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7740077040 0x7f77401a4b80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f772800d950 tx=0x7f772800de10 comp rx=0 tx=0).stop 2024-09-10T08:05:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:37.522+0000 7f7744c00640 1 -- 172.21.15.2:0/2354356925 shutdown_connections 2024-09-10T08:05:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:37.522+0000 7f7744c00640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f77240a2b80 0x7f77240a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:37.522+0000 7f7744c00640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7740112870 0x7f774019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:37.522+0000 7f7744c00640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7740077040 0x7f77401a4b80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:37.522+0000 7f7744c00640 1 --2- 172.21.15.2:0/2354356925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7740076720 0x7f77401a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:37.522+0000 7f7744c00640 1 -- 172.21.15.2:0/2354356925 >> 172.21.15.2:0/2354356925 conn(0x7f77400ff910 msgr2=0x7f7740101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:37.523+0000 7f7744c00640 1 -- 172.21.15.2:0/2354356925 shutdown_connections 2024-09-10T08:05:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:37.523+0000 7f7744c00640 1 -- 172.21.15.2:0/2354356925 wait complete. 2024-09-10T08:05:37.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:37 smithi002 ceph-mon[30563]: pgmap v304: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 3 op/s 2024-09-10T08:05:37.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:37 smithi002 ceph-mon[30563]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:05:37.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:37 smithi002 ceph-mon[30563]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T08:05:37.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:37 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2354356925' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T08:05:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:37 smithi155 ceph-mon[35405]: pgmap v304: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 3 op/s 2024-09-10T08:05:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:37 smithi155 ceph-mon[35405]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:05:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:37 smithi155 ceph-mon[35405]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T08:05:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:37 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2354356925' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T08:05:37.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:37 smithi107 ceph-mon[35048]: pgmap v304: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 3 op/s 2024-09-10T08:05:37.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:37 smithi107 ceph-mon[35048]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:05:37.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:37 smithi107 ceph-mon[35048]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T08:05:37.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:37 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2354356925' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T08:05:38.523 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T08:05:38.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:38 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2354356925' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T08:05:38.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:38 smithi002 ceph-mon[30563]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T08:05:38.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:38 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2354356925' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T08:05:38.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:38 smithi155 ceph-mon[35405]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T08:05:38.909 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:38.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:38 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2354356925' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T08:05:38.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:38 smithi107 ceph-mon[35048]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T08:05:39.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.475+0000 7f4da5400640 1 -- 172.21.15.2:0/1382063854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4da01019c0 msgr2=0x7f4da0101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:39.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.475+0000 7f4da5400640 1 --2- 172.21.15.2:0/1382063854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4da01019c0 0x7f4da0101e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009930 tx=0x7f4d8802a420 comp rx=0 tx=0).stop 2024-09-10T08:05:39.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.475+0000 7f4da5400640 1 -- 172.21.15.2:0/1382063854 shutdown_connections 2024-09-10T08:05:39.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.475+0000 7f4da5400640 1 --2- 172.21.15.2:0/1382063854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da0102360 0x7f4da010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:39.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.475+0000 7f4da5400640 1 --2- 172.21.15.2:0/1382063854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4da01019c0 0x7f4da0101e20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:39.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.475+0000 7f4da5400640 1 --2- 172.21.15.2:0/1382063854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4da01079c0 0x7f4da0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:39.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.475+0000 7f4da5400640 1 -- 172.21.15.2:0/1382063854 >> 172.21.15.2:0/1382063854 conn(0x7f4da00fd740 msgr2=0x7f4da00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:39.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.475+0000 7f4da5400640 1 -- 172.21.15.2:0/1382063854 shutdown_connections 2024-09-10T08:05:39.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.475+0000 7f4da5400640 1 -- 172.21.15.2:0/1382063854 wait complete. 2024-09-10T08:05:39.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.476+0000 7f4da5400640 1 Processor -- start 2024-09-10T08:05:39.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.476+0000 7f4da5400640 1 -- start start 2024-09-10T08:05:39.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4da5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da01019c0 0x7f4da019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:39.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4da5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4da0102360 0x7f4da019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:39.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4da5400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4da01079c0 0x7f4da019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4da5400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0111e10 con 0x7f4da01079c0 2024-09-10T08:05:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4da5400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0111c90 con 0x7f4da01019c0 2024-09-10T08:05:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4da5400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0111f90 con 0x7f4da0102360 2024-09-10T08:05:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4d9fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da01019c0 0x7f4da019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4d9fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da01019c0 0x7f4da019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41440/0 (socket says 172.21.15.2:41440) 2024-09-10T08:05:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4d9fe00640 1 -- 172.21.15.2:0/251910251 learned_addr learned my addr 172.21.15.2:0/251910251 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4da4a00640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4da01079c0 0x7f4da019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4d97e00640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4da0102360 0x7f4da019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4d9fe00640 1 -- 172.21.15.2:0/251910251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4da0102360 msgr2=0x7f4da019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4d9fe00640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4da0102360 0x7f4da019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4d9fe00640 1 -- 172.21.15.2:0/251910251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4da01079c0 msgr2=0x7f4da019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4d9fe00640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4da01079c0 0x7f4da019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.477+0000 7f4d9fe00640 1 -- 172.21.15.2:0/251910251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da019f4e0 con 0x7f4da01019c0 2024-09-10T08:05:39.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.478+0000 7f4d9fe00640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da01019c0 0x7f4da019a530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000e9f0 tx=0x7f4d9000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:39.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.478+0000 7f4d9d600640 1 -- 172.21.15.2:0/251910251 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9000ccc0 con 0x7f4da01019c0 2024-09-10T08:05:39.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.478+0000 7f4da5400640 1 -- 172.21.15.2:0/251910251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da019f7d0 con 0x7f4da01019c0 2024-09-10T08:05:39.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.478+0000 7f4d9d600640 1 -- 172.21.15.2:0/251910251 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d9000ce60 con 0x7f4da01019c0 2024-09-10T08:05:39.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.478+0000 7f4d9d600640 1 -- 172.21.15.2:0/251910251 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d90010690 con 0x7f4da01019c0 2024-09-10T08:05:39.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.478+0000 7f4da5400640 1 -- 172.21.15.2:0/251910251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da01a70b0 con 0x7f4da01019c0 2024-09-10T08:05:39.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.479+0000 7f4da5400640 1 -- 172.21.15.2:0/251910251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da0101e20 con 0x7f4da01019c0 2024-09-10T08:05:39.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.481+0000 7f4d9d600640 1 -- 172.21.15.2:0/251910251 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f4d900046c0 con 0x7f4da01019c0 2024-09-10T08:05:39.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.483+0000 7f4d9d600640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d840a2c00 0x7f4d840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:39.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.483+0000 7f4d9d600640 1 -- 172.21.15.2:0/251910251 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8770+0+0 (secure 0 0 0) 0x7f4d900cf7c0 con 0x7f4da01019c0 2024-09-10T08:05:39.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.483+0000 7f4d97e00640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d840a2c00 0x7f4d840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:39.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.483+0000 7f4d97e00640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d840a2c00 0x7f4d840a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d88006010 tx=0x7f4d88000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:39.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.485+0000 7f4d9d600640 1 -- 172.21.15.2:0/251910251 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d900541b0 con 0x7f4da01019c0 2024-09-10T08:05:39.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:39.708+0000 7f4da5400640 1 -- 172.21.15.2:0/251910251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f4da01a72e0 con 0x7f4da01019c0 2024-09-10T08:05:39.708 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:39 smithi002 ceph-mon[30563]: pgmap v307: 129 pgs: 5 creating+peering, 4 unknown, 120 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T08:05:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:39 smithi155 ceph-mon[35405]: pgmap v307: 129 pgs: 5 creating+peering, 4 unknown, 120 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T08:05:39.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:39 smithi107 ceph-mon[35048]: pgmap v307: 129 pgs: 5 creating+peering, 4 unknown, 120 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T08:05:40.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:40.541+0000 7f4d9d600640 1 -- 172.21.15.2:0/251910251 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f4d9009c4a0 con 0x7f4da01019c0 2024-09-10T08:05:40.541 INFO:teuthology.orchestra.run.smithi002.stderr:added data pool 4 to fsmap 2024-09-10T08:05:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:40.544+0000 7f4da5400640 1 -- 172.21.15.2:0/251910251 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d840a2c00 msgr2=0x7f4d840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:40.544+0000 7f4da5400640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d840a2c00 0x7f4d840a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d88006010 tx=0x7f4d88000bc0 comp rx=0 tx=0).stop 2024-09-10T08:05:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:40.544+0000 7f4da5400640 1 -- 172.21.15.2:0/251910251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da01019c0 msgr2=0x7f4da019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:40.544+0000 7f4da5400640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da01019c0 0x7f4da019a530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000e9f0 tx=0x7f4d9000edf0 comp rx=0 tx=0).stop 2024-09-10T08:05:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:40.544+0000 7f4da5400640 1 -- 172.21.15.2:0/251910251 shutdown_connections 2024-09-10T08:05:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:40.544+0000 7f4da5400640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4da01079c0 0x7f4da019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:40.544+0000 7f4da5400640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d840a2c00 0x7f4d840a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:40.544+0000 7f4da5400640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4da0102360 0x7f4da019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:40.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:40.544+0000 7f4da5400640 1 --2- 172.21.15.2:0/251910251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da01019c0 0x7f4da019a530 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:40.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:40.544+0000 7f4da5400640 1 -- 172.21.15.2:0/251910251 >> 172.21.15.2:0/251910251 conn(0x7f4da00fd740 msgr2=0x7f4da010cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:40.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:40.544+0000 7f4da5400640 1 -- 172.21.15.2:0/251910251 shutdown_connections 2024-09-10T08:05:40.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:40.545+0000 7f4da5400640 1 -- 172.21.15.2:0/251910251 wait complete. 2024-09-10T08:05:40.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:40 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/251910251' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T08:05:40.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:40 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T08:05:40.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:40 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/251910251' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T08:05:40.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:40 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T08:05:41.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:40 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/251910251' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T08:05:41.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:40 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T08:05:41.648 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:05:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:41 smithi155 ceph-mon[35405]: pgmap v308: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:41 smithi155 ceph-mon[35405]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T08:05:41.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:41 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T08:05:41.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:41 smithi155 ceph-mon[35405]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:05:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:41 smithi107 ceph-mon[35048]: pgmap v308: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:41 smithi107 ceph-mon[35048]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T08:05:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:41 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T08:05:41.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:41 smithi107 ceph-mon[35048]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:05:42.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:41 smithi002 ceph-mon[30563]: pgmap v308: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:42.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:41 smithi002 ceph-mon[30563]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T08:05:42.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:41 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T08:05:42.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:41 smithi002 ceph-mon[30563]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:05:42.031 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:42.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.592+0000 7f6b61c00640 1 -- 172.21.15.2:0/2904192534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b5c1079c0 msgr2=0x7f6b5c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:42.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.592+0000 7f6b61c00640 1 --2- 172.21.15.2:0/2904192534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b5c1079c0 0x7f6b5c107da0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c009930 tx=0x7f6b4c02a420 comp rx=0 tx=0).stop 2024-09-10T08:05:42.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.592+0000 7f6b61c00640 1 -- 172.21.15.2:0/2904192534 shutdown_connections 2024-09-10T08:05:42.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.592+0000 7f6b61c00640 1 --2- 172.21.15.2:0/2904192534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b5c102360 0x7f6b5c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:42.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.592+0000 7f6b61c00640 1 --2- 172.21.15.2:0/2904192534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b5c1019c0 0x7f6b5c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:42.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.592+0000 7f6b61c00640 1 --2- 172.21.15.2:0/2904192534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b5c1079c0 0x7f6b5c107da0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:42.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.592+0000 7f6b61c00640 1 -- 172.21.15.2:0/2904192534 >> 172.21.15.2:0/2904192534 conn(0x7f6b5c0fd740 msgr2=0x7f6b5c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:42.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.593+0000 7f6b61c00640 1 -- 172.21.15.2:0/2904192534 shutdown_connections 2024-09-10T08:05:42.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.593+0000 7f6b61c00640 1 -- 172.21.15.2:0/2904192534 wait complete. 2024-09-10T08:05:42.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.593+0000 7f6b61c00640 1 Processor -- start 2024-09-10T08:05:42.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.594+0000 7f6b61c00640 1 -- start start 2024-09-10T08:05:42.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.594+0000 7f6b61c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b5c1019c0 0x7f6b5c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:42.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.594+0000 7f6b61c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b5c102360 0x7f6b5c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:42.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.594+0000 7f6b61c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b5c1079c0 0x7f6b5c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:42.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.594+0000 7f6b61c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c111dd0 con 0x7f6b5c1079c0 2024-09-10T08:05:42.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.594+0000 7f6b61c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c111c50 con 0x7f6b5c1019c0 2024-09-10T08:05:42.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.594+0000 7f6b61c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c111f50 con 0x7f6b5c102360 2024-09-10T08:05:42.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.594+0000 7f6b61200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b5c1079c0 0x7f6b5c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:42.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.594+0000 7f6b61200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b5c1079c0 0x7f6b5c19ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41440/0 (socket says 172.21.15.2:41440) 2024-09-10T08:05:42.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.594+0000 7f6b61200640 1 -- 172.21.15.2:0/220469314 learned_addr learned my addr 172.21.15.2:0/220469314 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:42.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.594+0000 7f6b5be00640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b5c1019c0 0x7f6b5c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:42.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.594+0000 7f6b53400640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b5c102360 0x7f6b5c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:42.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.595+0000 7f6b5be00640 1 -- 172.21.15.2:0/220469314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b5c102360 msgr2=0x7f6b5c19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:42.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.595+0000 7f6b5be00640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b5c102360 0x7f6b5c19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:42.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.595+0000 7f6b5be00640 1 -- 172.21.15.2:0/220469314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b5c1079c0 msgr2=0x7f6b5c19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:42.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.595+0000 7f6b5be00640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b5c1079c0 0x7f6b5c19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:42.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.595+0000 7f6b5be00640 1 -- 172.21.15.2:0/220469314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c19f3f0 con 0x7f6b5c1019c0 2024-09-10T08:05:42.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.595+0000 7f6b61200640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b5c1079c0 0x7f6b5c19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:42.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.595+0000 7f6b5be00640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b5c1019c0 0x7f6b5c19a440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c02a940 tx=0x7f6b4c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:42.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.595+0000 7f6b59600640 1 -- 172.21.15.2:0/220469314 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c03f070 con 0x7f6b5c1019c0 2024-09-10T08:05:42.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.595+0000 7f6b59600640 1 -- 172.21.15.2:0/220469314 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b4c031c90 con 0x7f6b5c1019c0 2024-09-10T08:05:42.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.595+0000 7f6b59600640 1 -- 172.21.15.2:0/220469314 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c035030 con 0x7f6b5c1019c0 2024-09-10T08:05:42.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.595+0000 7f6b61c00640 1 -- 172.21.15.2:0/220469314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5c19f680 con 0x7f6b5c1019c0 2024-09-10T08:05:42.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.596+0000 7f6b61c00640 1 -- 172.21.15.2:0/220469314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b5c19fb10 con 0x7f6b5c1019c0 2024-09-10T08:05:42.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.597+0000 7f6b61c00640 1 -- 172.21.15.2:0/220469314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b5c101e20 con 0x7f6b5c1019c0 2024-09-10T08:05:42.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.599+0000 7f6b59600640 1 -- 172.21.15.2:0/220469314 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6b4c042070 con 0x7f6b5c1019c0 2024-09-10T08:05:42.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.600+0000 7f6b59600640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6b400a2c00 0x7f6b400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:42.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.600+0000 7f6b59600640 1 -- 172.21.15.2:0/220469314 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f6b4c0efd10 con 0x7f6b5c1019c0 2024-09-10T08:05:42.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.600+0000 7f6b53400640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6b400a2c00 0x7f6b400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:42.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.601+0000 7f6b53400640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6b400a2c00 0x7f6b400a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c10ec90 tx=0x7f6b4400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:42.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.602+0000 7f6b59600640 1 -- 172.21.15.2:0/220469314 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b4c0bc9d0 con 0x7f6b5c1019c0 2024-09-10T08:05:42.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:42 smithi002 ceph-mon[30563]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T08:05:42.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:42 smithi002 ceph-mon[30563]: Cluster is now healthy 2024-09-10T08:05:42.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.826+0000 7f6b61c00640 1 -- 172.21.15.2:0/220469314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b5c1a7180 con 0x7f6b5c1019c0 2024-09-10T08:05:42.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.827+0000 7f6b59600640 1 -- 172.21.15.2:0/220469314 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11836 (secure 0 0 0) 0x7f6b4c0bc2d0 con 0x7f6b5c1019c0 2024-09-10T08:05:42.828 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:05:42.828 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"btime":"2024-09-10T08:05:40:527941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:05:39.710415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24536,"qdb_cluster":[24536]},"id":1}]} 2024-09-10T08:05:42.829 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-09-10T08:05:42.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.830+0000 7f6b61c00640 1 -- 172.21.15.2:0/220469314 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6b400a2c00 msgr2=0x7f6b400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:42.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.830+0000 7f6b61c00640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6b400a2c00 0x7f6b400a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c10ec90 tx=0x7f6b4400a2d0 comp rx=0 tx=0).stop 2024-09-10T08:05:42.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.830+0000 7f6b61c00640 1 -- 172.21.15.2:0/220469314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b5c1019c0 msgr2=0x7f6b5c19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:42.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.830+0000 7f6b61c00640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b5c1019c0 0x7f6b5c19a440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c02a940 tx=0x7f6b4c003b90 comp rx=0 tx=0).stop 2024-09-10T08:05:42.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.830+0000 7f6b61c00640 1 -- 172.21.15.2:0/220469314 shutdown_connections 2024-09-10T08:05:42.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.830+0000 7f6b61c00640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b5c1079c0 0x7f6b5c19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:42.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.830+0000 7f6b61c00640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6b400a2c00 0x7f6b400a50c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.830+0000 7f6b61c00640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b5c102360 0x7f6b5c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.830+0000 7f6b61c00640 1 --2- 172.21.15.2:0/220469314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b5c1019c0 0x7f6b5c19a440 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.830+0000 7f6b61c00640 1 -- 172.21.15.2:0/220469314 >> 172.21.15.2:0/220469314 conn(0x7f6b5c0fd740 msgr2=0x7f6b5c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.830+0000 7f6b61c00640 1 -- 172.21.15.2:0/220469314 shutdown_connections 2024-09-10T08:05:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:42.830+0000 7f6b61c00640 1 -- 172.21.15.2:0/220469314 wait complete. 2024-09-10T08:05:42.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:42 smithi155 ceph-mon[35405]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T08:05:42.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:42 smithi155 ceph-mon[35405]: Cluster is now healthy 2024-09-10T08:05:42.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:42 smithi107 ceph-mon[35048]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T08:05:42.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:42 smithi107 ceph-mon[35048]: Cluster is now healthy 2024-09-10T08:05:43.657 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:05:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:43 smithi155 ceph-mon[35405]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:43 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/220469314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:43.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:43 smithi107 ceph-mon[35048]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:43.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:43 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/220469314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:44.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:43 smithi002 ceph-mon[30563]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:44.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:43 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/220469314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:44.034 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:44.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.650+0000 7f7f2b200640 1 -- 172.21.15.2:0/1983155398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f24106f10 msgr2=0x7f7f2410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:44.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.650+0000 7f7f2b200640 1 --2- 172.21.15.2:0/1983155398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f24106f10 0x7f7f2410ea90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009ca0 tx=0x7f7f1c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:05:44.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.651+0000 7f7f2b200640 1 -- 172.21.15.2:0/1983155398 shutdown_connections 2024-09-10T08:05:44.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.651+0000 7f7f2b200640 1 --2- 172.21.15.2:0/1983155398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f24106f10 0x7f7f2410ea90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:44.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.651+0000 7f7f2b200640 1 --2- 172.21.15.2:0/1983155398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24102790 0x7f7f241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:44.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.651+0000 7f7f2b200640 1 --2- 172.21.15.2:0/1983155398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f24101de0 0x7f7f241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:44.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.651+0000 7f7f2b200640 1 -- 172.21.15.2:0/1983155398 >> 172.21.15.2:0/1983155398 conn(0x7f7f240fd8c0 msgr2=0x7f7f240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:44.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.651+0000 7f7f2b200640 1 -- 172.21.15.2:0/1983155398 shutdown_connections 2024-09-10T08:05:44.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.651+0000 7f7f2b200640 1 -- 172.21.15.2:0/1983155398 wait complete. 2024-09-10T08:05:44.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.652+0000 7f7f2b200640 1 Processor -- start 2024-09-10T08:05:44.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.652+0000 7f7f2b200640 1 -- start start 2024-09-10T08:05:44.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.652+0000 7f7f2b200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f24101de0 0x7f7f2419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:44.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.652+0000 7f7f2b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f24102790 0x7f7f2419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:44.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.652+0000 7f7f2b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24106f10 0x7f7f2419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:44.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.652+0000 7f7f2b200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24111fc0 con 0x7f7f24101de0 2024-09-10T08:05:44.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.652+0000 7f7f2b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24111e40 con 0x7f7f24102790 2024-09-10T08:05:44.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.652+0000 7f7f2b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24112140 con 0x7f7f24106f10 2024-09-10T08:05:44.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.652+0000 7f7f23e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f24101de0 0x7f7f2419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:44.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.652+0000 7f7f23e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f24101de0 0x7f7f2419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41462/0 (socket says 172.21.15.2:41462) 2024-09-10T08:05:44.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.652+0000 7f7f23e00640 1 -- 172.21.15.2:0/2106716849 learned_addr learned my addr 172.21.15.2:0/2106716849 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:44.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.653+0000 7f7f1b400640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f24102790 0x7f7f2419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:44.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.653+0000 7f7f29200640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24106f10 0x7f7f2419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:44.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.653+0000 7f7f23e00640 1 -- 172.21.15.2:0/2106716849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24106f10 msgr2=0x7f7f2419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:44.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.653+0000 7f7f23e00640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24106f10 0x7f7f2419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:44.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.653+0000 7f7f23e00640 1 -- 172.21.15.2:0/2106716849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f24102790 msgr2=0x7f7f2419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:44.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.653+0000 7f7f23e00640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f24102790 0x7f7f2419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:44.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.653+0000 7f7f23e00640 1 -- 172.21.15.2:0/2106716849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2419f590 con 0x7f7f24101de0 2024-09-10T08:05:44.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.653+0000 7f7f29200640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24106f10 0x7f7f2419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:05:44.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.653+0000 7f7f23e00640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f24101de0 0x7f7f2419a5e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400e9f0 tx=0x7f7f1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:44.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.653+0000 7f7f21600640 1 -- 172.21.15.2:0/2106716849 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400cd80 con 0x7f7f24101de0 2024-09-10T08:05:44.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.653+0000 7f7f21600640 1 -- 172.21.15.2:0/2106716849 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f14010910 con 0x7f7f24101de0 2024-09-10T08:05:44.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.653+0000 7f7f2b200640 1 -- 172.21.15.2:0/2106716849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2419f880 con 0x7f7f24101de0 2024-09-10T08:05:44.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.654+0000 7f7f21600640 1 -- 172.21.15.2:0/2106716849 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14018990 con 0x7f7f24101de0 2024-09-10T08:05:44.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.654+0000 7f7f2b200640 1 -- 172.21.15.2:0/2106716849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241a7160 con 0x7f7f24101de0 2024-09-10T08:05:44.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.655+0000 7f7f2b200640 1 -- 172.21.15.2:0/2106716849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f241021c0 con 0x7f7f24101de0 2024-09-10T08:05:44.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.657+0000 7f7f21600640 1 -- 172.21.15.2:0/2106716849 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7f14010ac0 con 0x7f7f24101de0 2024-09-10T08:05:44.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.658+0000 7f7f21600640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f080a2b80 0x7f7f080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:44.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.658+0000 7f7f21600640 1 -- 172.21.15.2:0/2106716849 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f7f140cf780 con 0x7f7f24101de0 2024-09-10T08:05:44.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.658+0000 7f7f1b400640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f080a2b80 0x7f7f080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:44.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.659+0000 7f7f1b400640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f080a2b80 0x7f7f080a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f24076480 tx=0x7f7f0c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:44.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.662+0000 7f7f21600640 1 -- 172.21.15.2:0/2106716849 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f1409c440 con 0x7f7f24101de0 2024-09-10T08:05:44.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.849+0000 7f7f2b200640 1 -- 172.21.15.2:0/2106716849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f241a7400 con 0x7f7f24101de0 2024-09-10T08:05:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.851+0000 7f7f21600640 1 -- 172.21.15.2:0/2106716849 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24128 (secure 0 0 0) 0x7f7f1409bd40 con 0x7f7f24101de0 2024-09-10T08:05:44.851 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:05:44.851 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":91,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:05:39.710277+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:05:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.855+0000 7f7f2b200640 1 -- 172.21.15.2:0/2106716849 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f080a2b80 msgr2=0x7f7f080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.855+0000 7f7f2b200640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f080a2b80 0x7f7f080a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f24076480 tx=0x7f7f0c00b410 comp rx=0 tx=0).stop 2024-09-10T08:05:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.855+0000 7f7f2b200640 1 -- 172.21.15.2:0/2106716849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f24101de0 msgr2=0x7f7f2419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.855+0000 7f7f2b200640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f24101de0 0x7f7f2419a5e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400e9f0 tx=0x7f7f1400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:05:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.855+0000 7f7f2b200640 1 -- 172.21.15.2:0/2106716849 shutdown_connections 2024-09-10T08:05:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.855+0000 7f7f2b200640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f080a2b80 0x7f7f080a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.855+0000 7f7f2b200640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24106f10 0x7f7f2419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.855+0000 7f7f2b200640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f24102790 0x7f7f2419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:44.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.855+0000 7f7f2b200640 1 --2- 172.21.15.2:0/2106716849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f24101de0 0x7f7f2419a5e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:44.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.855+0000 7f7f2b200640 1 -- 172.21.15.2:0/2106716849 >> 172.21.15.2:0/2106716849 conn(0x7f7f240fd8c0 msgr2=0x7f7f240fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:44.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.855+0000 7f7f2b200640 1 -- 172.21.15.2:0/2106716849 shutdown_connections 2024-09-10T08:05:44.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:44.855+0000 7f7f2b200640 1 -- 172.21.15.2:0/2106716849 wait complete. 2024-09-10T08:05:45.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:45 smithi002 ceph-mon[30563]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:45.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:45 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2106716849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:05:45.859 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:05:45.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:45 smithi155 ceph-mon[35405]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:45.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:45 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2106716849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:05:45.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:45 smithi107 ceph-mon[35048]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:45.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:45 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2106716849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:05:46.240 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:46.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:46 smithi002 ceph-mon[30563]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:46.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.901+0000 7f415ac00640 1 -- 172.21.15.2:0/1561264637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41541019c0 msgr2=0x7f4154101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:46.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.901+0000 7f415ac00640 1 --2- 172.21.15.2:0/1561264637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41541019c0 0x7f4154101e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4140009930 tx=0x7f414002a420 comp rx=0 tx=0).stop 2024-09-10T08:05:46.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.901+0000 7f415ac00640 1 -- 172.21.15.2:0/1561264637 shutdown_connections 2024-09-10T08:05:46.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.901+0000 7f415ac00640 1 --2- 172.21.15.2:0/1561264637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4154102360 0x7f415410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:46.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.901+0000 7f415ac00640 1 --2- 172.21.15.2:0/1561264637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41541019c0 0x7f4154101e20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:46.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.901+0000 7f415ac00640 1 --2- 172.21.15.2:0/1561264637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41541079c0 0x7f4154107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:46.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.901+0000 7f415ac00640 1 -- 172.21.15.2:0/1561264637 >> 172.21.15.2:0/1561264637 conn(0x7f41540fd6e0 msgr2=0x7f41540ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:46.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.901+0000 7f415ac00640 1 -- 172.21.15.2:0/1561264637 shutdown_connections 2024-09-10T08:05:46.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.901+0000 7f415ac00640 1 -- 172.21.15.2:0/1561264637 wait complete. 2024-09-10T08:05:46.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.902+0000 7f415ac00640 1 Processor -- start 2024-09-10T08:05:46.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.902+0000 7f415ac00640 1 -- start start 2024-09-10T08:05:46.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.902+0000 7f415ac00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41541019c0 0x7f415419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:46.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.902+0000 7f415ac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4154102360 0x7f415419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:46.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.902+0000 7f415ac00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41541079c0 0x7f415419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:46.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.902+0000 7f415ac00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4154111dc0 con 0x7f41541019c0 2024-09-10T08:05:46.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.902+0000 7f415ac00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4154111c40 con 0x7f41541079c0 2024-09-10T08:05:46.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.902+0000 7f415ac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4154111f40 con 0x7f4154102360 2024-09-10T08:05:46.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.903+0000 7f4159800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41541019c0 0x7f415419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.903+0000 7f4159800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41541019c0 0x7f415419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45890/0 (socket says 172.21.15.2:45890) 2024-09-10T08:05:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.903+0000 7f4159800640 1 -- 172.21.15.2:0/702656176 learned_addr learned my addr 172.21.15.2:0/702656176 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.903+0000 7f4158e00640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4154102360 0x7f415419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.903+0000 7f415a200640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41541079c0 0x7f415419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.903+0000 7f4159800640 1 -- 172.21.15.2:0/702656176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4154102360 msgr2=0x7f415419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.903+0000 7f4159800640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4154102360 0x7f415419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.903+0000 7f4159800640 1 -- 172.21.15.2:0/702656176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41541079c0 msgr2=0x7f415419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.903+0000 7f4159800640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41541079c0 0x7f415419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.903+0000 7f4159800640 1 -- 172.21.15.2:0/702656176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415419f450 con 0x7f41541019c0 2024-09-10T08:05:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.903+0000 7f4158e00640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4154102360 0x7f415419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.904+0000 7f4159800640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41541019c0 0x7f415419a4a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f414800e9f0 tx=0x7f414800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:46.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.904+0000 7f4152000640 1 -- 172.21.15.2:0/702656176 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414800cd80 con 0x7f41541019c0 2024-09-10T08:05:46.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.904+0000 7f4152000640 1 -- 172.21.15.2:0/702656176 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4148010910 con 0x7f41541019c0 2024-09-10T08:05:46.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.904+0000 7f415ac00640 1 -- 172.21.15.2:0/702656176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415419f740 con 0x7f41541019c0 2024-09-10T08:05:46.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.904+0000 7f4152000640 1 -- 172.21.15.2:0/702656176 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148018990 con 0x7f41541019c0 2024-09-10T08:05:46.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.904+0000 7f415ac00640 1 -- 172.21.15.2:0/702656176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41541a7020 con 0x7f41541019c0 2024-09-10T08:05:46.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.905+0000 7f415ac00640 1 -- 172.21.15.2:0/702656176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4154101e20 con 0x7f41541019c0 2024-09-10T08:05:46.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:46 smithi155 ceph-mon[35405]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:46.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.907+0000 7f4152000640 1 -- 172.21.15.2:0/702656176 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f4148010ac0 con 0x7f41541019c0 2024-09-10T08:05:46.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.908+0000 7f4152000640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f413c0a2b80 0x7f413c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:46.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.908+0000 7f4152000640 1 -- 172.21.15.2:0/702656176 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f41480cf570 con 0x7f41541019c0 2024-09-10T08:05:46.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.908+0000 7f4158e00640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f413c0a2b80 0x7f413c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:46.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.909+0000 7f4158e00640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f413c0a2b80 0x7f413c0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4140009900 tx=0x7f41400058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:46.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:46.912+0000 7f4152000640 1 -- 172.21.15.2:0/702656176 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f414809bbe0 con 0x7f41541019c0 2024-09-10T08:05:46.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:46 smithi107 ceph-mon[35048]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:47.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.100+0000 7f415ac00640 1 -- 172.21.15.2:0/702656176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41541a7250 con 0x7f41541019c0 2024-09-10T08:05:47.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.102+0000 7f4152000640 1 -- 172.21.15.2:0/702656176 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24128 (secure 0 0 0) 0x7f414809fee0 con 0x7f41541019c0 2024-09-10T08:05:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:05:47.102 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":91,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:05:39.710277+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:05:47.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.105+0000 7f415ac00640 1 -- 172.21.15.2:0/702656176 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f413c0a2b80 msgr2=0x7f413c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:47.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.105+0000 7f415ac00640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f413c0a2b80 0x7f413c0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4140009900 tx=0x7f41400058d0 comp rx=0 tx=0).stop 2024-09-10T08:05:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.105+0000 7f415ac00640 1 -- 172.21.15.2:0/702656176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41541019c0 msgr2=0x7f415419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.105+0000 7f415ac00640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41541019c0 0x7f415419a4a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f414800e9f0 tx=0x7f414800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:05:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.105+0000 7f415ac00640 1 -- 172.21.15.2:0/702656176 shutdown_connections 2024-09-10T08:05:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.105+0000 7f415ac00640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41541079c0 0x7f415419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.105+0000 7f415ac00640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f413c0a2b80 0x7f413c0a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.105+0000 7f415ac00640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4154102360 0x7f415419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.105+0000 7f415ac00640 1 --2- 172.21.15.2:0/702656176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41541019c0 0x7f415419a4a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.105+0000 7f415ac00640 1 -- 172.21.15.2:0/702656176 >> 172.21.15.2:0/702656176 conn(0x7f41540fd6e0 msgr2=0x7f41540fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.105+0000 7f415ac00640 1 -- 172.21.15.2:0/702656176 shutdown_connections 2024-09-10T08:05:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:47.105+0000 7f415ac00640 1 -- 172.21.15.2:0/702656176 wait complete. 2024-09-10T08:05:47.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:47 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/702656176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:05:47.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:47 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/702656176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:05:47.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:47 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/702656176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:05:48.008 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:05:48.390 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:48.787 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:48 smithi002 ceph-mon[30563]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:48.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:48 smithi155 ceph-mon[35405]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:48.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:48 smithi107 ceph-mon[35048]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:49.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.009+0000 7f3adf000640 1 -- 172.21.15.2:0/2188744497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad8076720 msgr2=0x7f3ad8076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.009+0000 7f3adf000640 1 --2- 172.21.15.2:0/2188744497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad8076720 0x7f3ad8076b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009900 tx=0x7f3ac802a410 comp rx=0 tx=0).stop 2024-09-10T08:05:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.010+0000 7f3adf000640 1 -- 172.21.15.2:0/2188744497 shutdown_connections 2024-09-10T08:05:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.010+0000 7f3adf000640 1 --2- 172.21.15.2:0/2188744497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ad8112a50 0x7f3ad8114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.010+0000 7f3adf000640 1 --2- 172.21.15.2:0/2188744497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ad8077040 0x7f3ad8112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.010+0000 7f3adf000640 1 --2- 172.21.15.2:0/2188744497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad8076720 0x7f3ad8076b00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.010+0000 7f3adf000640 1 -- 172.21.15.2:0/2188744497 >> 172.21.15.2:0/2188744497 conn(0x7f3ad80ffa50 msgr2=0x7f3ad8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.010+0000 7f3adf000640 1 -- 172.21.15.2:0/2188744497 shutdown_connections 2024-09-10T08:05:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.010+0000 7f3adf000640 1 -- 172.21.15.2:0/2188744497 wait complete. 2024-09-10T08:05:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.011+0000 7f3adf000640 1 Processor -- start 2024-09-10T08:05:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.011+0000 7f3adf000640 1 -- start start 2024-09-10T08:05:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.011+0000 7f3adf000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ad8076720 0x7f3ad81a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.011+0000 7f3adf000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ad8077040 0x7f3ad81a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.011+0000 7f3adf000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad8112a50 0x7f3ad819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.011+0000 7f3adf000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad8118630 con 0x7f3ad8112a50 2024-09-10T08:05:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.011+0000 7f3adf000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81184b0 con 0x7f3ad8077040 2024-09-10T08:05:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.011+0000 7f3adf000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81187b0 con 0x7f3ad8076720 2024-09-10T08:05:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.011+0000 7f3add000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad8112a50 0x7f3ad819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.011+0000 7f3add000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad8112a50 0x7f3ad819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45902/0 (socket says 172.21.15.2:45902) 2024-09-10T08:05:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.011+0000 7f3add000640 1 -- 172.21.15.2:0/2892832903 learned_addr learned my addr 172.21.15.2:0/2892832903 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.012+0000 7f3ad7400640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ad8077040 0x7f3ad81a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.012+0000 7f3ad7e00640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ad8076720 0x7f3ad81a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.012+0000 7f3add000640 1 -- 172.21.15.2:0/2892832903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ad8076720 msgr2=0x7f3ad81a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.012+0000 7f3add000640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ad8076720 0x7f3ad81a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.012+0000 7f3add000640 1 -- 172.21.15.2:0/2892832903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ad8077040 msgr2=0x7f3ad81a4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.012+0000 7f3add000640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ad8077040 0x7f3ad81a4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.012+0000 7f3add000640 1 -- 172.21.15.2:0/2892832903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad819f0b0 con 0x7f3ad8112a50 2024-09-10T08:05:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.012+0000 7f3add000640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad8112a50 0x7f3ad819e980 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000dac0 tx=0x7f3ad000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:49.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.013+0000 7f3ad4c00640 1 -- 172.21.15.2:0/2892832903 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0017030 con 0x7f3ad8112a50 2024-09-10T08:05:49.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.013+0000 7f3ad4c00640 1 -- 172.21.15.2:0/2892832903 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ad0003e50 con 0x7f3ad8112a50 2024-09-10T08:05:49.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.013+0000 7f3adf000640 1 -- 172.21.15.2:0/2892832903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad819f3a0 con 0x7f3ad8112a50 2024-09-10T08:05:49.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.013+0000 7f3ad4c00640 1 -- 172.21.15.2:0/2892832903 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0010460 con 0x7f3ad8112a50 2024-09-10T08:05:49.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.013+0000 7f3adf000640 1 -- 172.21.15.2:0/2892832903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad81ab690 con 0x7f3ad8112a50 2024-09-10T08:05:49.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.014+0000 7f3adf000640 1 -- 172.21.15.2:0/2892832903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad8076b00 con 0x7f3ad8112a50 2024-09-10T08:05:49.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.016+0000 7f3ad4c00640 1 -- 172.21.15.2:0/2892832903 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3ad001a070 con 0x7f3ad8112a50 2024-09-10T08:05:49.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.017+0000 7f3ad4c00640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3abc0a2b80 0x7f3abc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:49.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.017+0000 7f3ad4c00640 1 -- 172.21.15.2:0/2892832903 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f3ad00cefc0 con 0x7f3ad8112a50 2024-09-10T08:05:49.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.017+0000 7f3ad7e00640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3abc0a2b80 0x7f3abc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:49.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.018+0000 7f3ad7e00640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3abc0a2b80 0x7f3abc0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8006010 tx=0x7f3ac8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:49.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.021+0000 7f3ad4c00640 1 -- 172.21.15.2:0/2892832903 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ad009bc80 con 0x7f3ad8112a50 2024-09-10T08:05:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.244+0000 7f3adf000640 1 -- 172.21.15.2:0/2892832903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ad802d7c0 con 0x7f3ad8112a50 2024-09-10T08:05:49.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.244+0000 7f3ad4c00640 1 -- 172.21.15.2:0/2892832903 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11836 (secure 0 0 0) 0x7f3ad009b580 con 0x7f3ad8112a50 2024-09-10T08:05:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:05:49.243 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"btime":"2024-09-10T08:05:40:527941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:05:39.710415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24536,"qdb_cluster":[24536]},"id":1}]} 2024-09-10T08:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-09-10T08:05:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.246+0000 7f3adf000640 1 -- 172.21.15.2:0/2892832903 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3abc0a2b80 msgr2=0x7f3abc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.246+0000 7f3adf000640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3abc0a2b80 0x7f3abc0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8006010 tx=0x7f3ac8000bc0 comp rx=0 tx=0).stop 2024-09-10T08:05:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.246+0000 7f3adf000640 1 -- 172.21.15.2:0/2892832903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad8112a50 msgr2=0x7f3ad819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.246+0000 7f3adf000640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad8112a50 0x7f3ad819e980 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000dac0 tx=0x7f3ad000df80 comp rx=0 tx=0).stop 2024-09-10T08:05:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.246+0000 7f3adf000640 1 -- 172.21.15.2:0/2892832903 shutdown_connections 2024-09-10T08:05:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.246+0000 7f3adf000640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad8112a50 0x7f3ad819e980 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.246+0000 7f3adf000640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ad8077040 0x7f3ad81a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:49.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.246+0000 7f3adf000640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3abc0a2b80 0x7f3abc0a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:49.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.246+0000 7f3adf000640 1 --2- 172.21.15.2:0/2892832903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ad8076720 0x7f3ad81a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:49.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.246+0000 7f3adf000640 1 -- 172.21.15.2:0/2892832903 >> 172.21.15.2:0/2892832903 conn(0x7f3ad80ffa50 msgr2=0x7f3ad8113650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:49.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.246+0000 7f3adf000640 1 -- 172.21.15.2:0/2892832903 shutdown_connections 2024-09-10T08:05:49.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:49.246+0000 7f3adf000640 1 -- 172.21.15.2:0/2892832903 wait complete. 2024-09-10T08:05:49.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:49 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2892832903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:49.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:49 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2892832903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:49.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:49 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2892832903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:05:50.034 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:05:50.418 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:50.812 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:50 smithi002 ceph-mon[30563]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:50.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:50 smithi155 ceph-mon[35405]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:50.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:50 smithi107 ceph-mon[35048]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:51.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.070+0000 7f0039400640 1 -- 172.21.15.2:0/293563306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0034108f60 msgr2=0x7f0034110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:51.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.070+0000 7f0039400640 1 --2- 172.21.15.2:0/293563306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0034108f60 0x7f0034110ae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f002c009be0 tx=0x7f002c02a6d0 comp rx=0 tx=0).stop 2024-09-10T08:05:51.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.070+0000 7f0039400640 1 -- 172.21.15.2:0/293563306 shutdown_connections 2024-09-10T08:05:51.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.070+0000 7f0039400640 1 --2- 172.21.15.2:0/293563306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0034108f60 0x7f0034110ae0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:51.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.070+0000 7f0039400640 1 --2- 172.21.15.2:0/293563306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00341047e0 0x7f0034108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:51.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.070+0000 7f0039400640 1 --2- 172.21.15.2:0/293563306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0034103e30 0x7f0034104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:51.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.070+0000 7f0039400640 1 -- 172.21.15.2:0/293563306 >> 172.21.15.2:0/293563306 conn(0x7f00340ff8d0 msgr2=0x7f0034101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:51.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.070+0000 7f0039400640 1 -- 172.21.15.2:0/293563306 shutdown_connections 2024-09-10T08:05:51.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.070+0000 7f0039400640 1 -- 172.21.15.2:0/293563306 wait complete. 2024-09-10T08:05:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.071+0000 7f0039400640 1 Processor -- start 2024-09-10T08:05:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.072+0000 7f0039400640 1 -- start start 2024-09-10T08:05:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.072+0000 7f0039400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0034103e30 0x7f0034076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.072+0000 7f0039400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00341047e0 0x7f0034076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.072+0000 7f0039400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0034108f60 0x7f0034077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.072+0000 7f0039400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0034113fc0 con 0x7f0034103e30 2024-09-10T08:05:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.072+0000 7f0039400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0034113e40 con 0x7f0034108f60 2024-09-10T08:05:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.072+0000 7f0039400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0034114140 con 0x7f00341047e0 2024-09-10T08:05:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.072+0000 7f0033e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0034103e30 0x7f0034076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.072+0000 7f0033e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0034103e30 0x7f0034076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45918/0 (socket says 172.21.15.2:45918) 2024-09-10T08:05:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.072+0000 7f0033e00640 1 -- 172.21.15.2:0/4087860733 learned_addr learned my addr 172.21.15.2:0/4087860733 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:51.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.072+0000 7f002b400640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00341047e0 0x7f0034076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.072+0000 7f0038a00640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0034108f60 0x7f0034077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.073+0000 7f0033e00640 1 -- 172.21.15.2:0/4087860733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00341047e0 msgr2=0x7f0034076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.073+0000 7f0033e00640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00341047e0 0x7f0034076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.073+0000 7f0033e00640 1 -- 172.21.15.2:0/4087860733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0034108f60 msgr2=0x7f0034077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.073+0000 7f0033e00640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0034108f60 0x7f0034077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.073+0000 7f0033e00640 1 -- 172.21.15.2:0/4087860733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00341aaaa0 con 0x7f0034103e30 2024-09-10T08:05:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.073+0000 7f002b400640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00341047e0 0x7f0034076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.073+0000 7f0038a00640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0034108f60 0x7f0034077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.073+0000 7f0033e00640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0034103e30 0x7f0034076700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f002400ea90 tx=0x7f002400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.073+0000 7f0031600640 1 -- 172.21.15.2:0/4087860733 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002400ce40 con 0x7f0034103e30 2024-09-10T08:05:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.073+0000 7f0031600640 1 -- 172.21.15.2:0/4087860733 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00240035f0 con 0x7f0034103e30 2024-09-10T08:05:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.073+0000 7f0031600640 1 -- 172.21.15.2:0/4087860733 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024010690 con 0x7f0034103e30 2024-09-10T08:05:51.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.073+0000 7f0039400640 1 -- 172.21.15.2:0/4087860733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00341aad90 con 0x7f0034103e30 2024-09-10T08:05:51.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.073+0000 7f0039400640 1 -- 172.21.15.2:0/4087860733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00341ab220 con 0x7f0034103e30 2024-09-10T08:05:51.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.075+0000 7f0039400640 1 -- 172.21.15.2:0/4087860733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0034105310 con 0x7f0034103e30 2024-09-10T08:05:51.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.076+0000 7f0031600640 1 -- 172.21.15.2:0/4087860733 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f00240037a0 con 0x7f0034103e30 2024-09-10T08:05:51.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.077+0000 7f0031600640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00180a2c00 0x7f00180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:51.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.078+0000 7f0031600640 1 -- 172.21.15.2:0/4087860733 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f00240cf5e0 con 0x7f0034103e30 2024-09-10T08:05:51.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.078+0000 7f002b400640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00180a2c00 0x7f00180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:51.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.078+0000 7f002b400640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00180a2c00 0x7f00180a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f001c0096c0 tx=0x7f001c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:51.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.081+0000 7f0031600640 1 -- 172.21.15.2:0/4087860733 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f002409c2a0 con 0x7f0034103e30 2024-09-10T08:05:51.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.268+0000 7f0039400640 1 -- 172.21.15.2:0/4087860733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f003402d7c0 con 0x7f0034103e30 2024-09-10T08:05:51.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.270+0000 7f0031600640 1 -- 172.21.15.2:0/4087860733 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24128 (secure 0 0 0) 0x7f002409bba0 con 0x7f0034103e30 2024-09-10T08:05:51.269 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:05:51.269 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":91,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:05:39.710277+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:05:51.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.272+0000 7f0039400640 1 -- 172.21.15.2:0/4087860733 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00180a2c00 msgr2=0x7f00180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:51.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.272+0000 7f0039400640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00180a2c00 0x7f00180a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f001c0096c0 tx=0x7f001c009260 comp rx=0 tx=0).stop 2024-09-10T08:05:51.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.272+0000 7f0039400640 1 -- 172.21.15.2:0/4087860733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0034103e30 msgr2=0x7f0034076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:51.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.272+0000 7f0039400640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0034103e30 0x7f0034076700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f002400ea90 tx=0x7f002400ef50 comp rx=0 tx=0).stop 2024-09-10T08:05:51.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.272+0000 7f0039400640 1 -- 172.21.15.2:0/4087860733 shutdown_connections 2024-09-10T08:05:51.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.272+0000 7f0039400640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0034108f60 0x7f0034077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:51.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.272+0000 7f0039400640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00180a2c00 0x7f00180a50c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:51.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.272+0000 7f0039400640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00341047e0 0x7f0034076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:51.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.272+0000 7f0039400640 1 --2- 172.21.15.2:0/4087860733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0034103e30 0x7f0034076700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:51.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.272+0000 7f0039400640 1 -- 172.21.15.2:0/4087860733 >> 172.21.15.2:0/4087860733 conn(0x7f00340ff8d0 msgr2=0x7f0034100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:51.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.272+0000 7f0039400640 1 -- 172.21.15.2:0/4087860733 shutdown_connections 2024-09-10T08:05:51.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:51.272+0000 7f0039400640 1 -- 172.21.15.2:0/4087860733 wait complete. 2024-09-10T08:05:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:51 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/4087860733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:05:51.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:51 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/4087860733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:05:52.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:51 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/4087860733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:05:52.150 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T08:05:26.574488+0000', 'modified': '2024-09-10T08:05:39.710415+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24536}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.107:6839/3674188134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 3674188134}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 3674188134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24536, 'qdb_cluster': [24536]} 2024-09-10T08:05:52.150 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T08:05:52.150 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T08:05:52.530 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:52.883 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:52 smithi002 ceph-mon[30563]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:52.939 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:52 smithi107 ceph-mon[35048]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:52.939 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:52 smithi155 ceph-mon[35405]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:53.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.115+0000 7fc012a00640 1 -- 172.21.15.2:0/245541016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc00c0745c0 msgr2=0x7fc00c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:53.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.115+0000 7fc012a00640 1 --2- 172.21.15.2:0/245541016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc00c0745c0 0x7fc00c0749a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc000009ca0 tx=0x7fc00002a950 comp rx=0 tx=0).stop 2024-09-10T08:05:53.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.115+0000 7fc012a00640 1 -- 172.21.15.2:0/245541016 shutdown_connections 2024-09-10T08:05:53.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.115+0000 7fc012a00640 1 --2- 172.21.15.2:0/245541016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00c10e260 0x7fc00c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.115+0000 7fc012a00640 1 --2- 172.21.15.2:0/245541016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc00c074ee0 0x7fc00c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.115+0000 7fc012a00640 1 --2- 172.21.15.2:0/245541016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc00c0745c0 0x7fc00c0749a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.115+0000 7fc012a00640 1 -- 172.21.15.2:0/245541016 >> 172.21.15.2:0/245541016 conn(0x7fc00c0fd740 msgr2=0x7fc00c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:53.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.115+0000 7fc012a00640 1 -- 172.21.15.2:0/245541016 shutdown_connections 2024-09-10T08:05:53.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.115+0000 7fc012a00640 1 -- 172.21.15.2:0/245541016 wait complete. 2024-09-10T08:05:53.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.116+0000 7fc012a00640 1 Processor -- start 2024-09-10T08:05:53.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.116+0000 7fc012a00640 1 -- start start 2024-09-10T08:05:53.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc012a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc00c0745c0 0x7fc00c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc012a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc00c074ee0 0x7fc00c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc012a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00c10e260 0x7fc00c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc012a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c114000 con 0x7fc00c0745c0 2024-09-10T08:05:53.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc012a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c113e80 con 0x7fc00c074ee0 2024-09-10T08:05:53.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc012a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c114180 con 0x7fc00c10e260 2024-09-10T08:05:53.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc011600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc00c0745c0 0x7fc00c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc011600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc00c0745c0 0x7fc00c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45934/0 (socket says 172.21.15.2:45934) 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc011600640 1 -- 172.21.15.2:0/2089810566 learned_addr learned my addr 172.21.15.2:0/2089810566 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc012000640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00c10e260 0x7fc00c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc010c00640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc00c074ee0 0x7fc00c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc011600640 1 -- 172.21.15.2:0/2089810566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00c10e260 msgr2=0x7fc00c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc011600640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00c10e260 0x7fc00c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc011600640 1 -- 172.21.15.2:0/2089810566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc00c074ee0 msgr2=0x7fc00c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc011600640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc00c074ee0 0x7fc00c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.117+0000 7fc011600640 1 -- 172.21.15.2:0/2089810566 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c19f450 con 0x7fc00c0745c0 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.118+0000 7fc011600640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc00c0745c0 0x7fc00c19a4a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc000005e30 tx=0x7fc000003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.118+0000 7fc00a000640 1 -- 172.21.15.2:0/2089810566 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000034040 con 0x7fc00c0745c0 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.118+0000 7fc00a000640 1 -- 172.21.15.2:0/2089810566 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00002fe20 con 0x7fc00c0745c0 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.118+0000 7fc00a000640 1 -- 172.21.15.2:0/2089810566 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0000047c0 con 0x7fc00c0745c0 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.118+0000 7fc012a00640 1 -- 172.21.15.2:0/2089810566 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c19f6e0 con 0x7fc00c0745c0 2024-09-10T08:05:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.118+0000 7fc012a00640 1 -- 172.21.15.2:0/2089810566 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c1a7020 con 0x7fc00c0745c0 2024-09-10T08:05:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.121+0000 7fc00a000640 1 -- 172.21.15.2:0/2089810566 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fc000037020 con 0x7fc00c0745c0 2024-09-10T08:05:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.122+0000 7fc016bde740 1 Processor -- start 2024-09-10T08:05:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.122+0000 7fc016bde740 1 -- start start 2024-09-10T08:05:53.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.122+0000 7fc016bde740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26a84e0 0x5626e26a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.122+0000 7fc016bde740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a8e00 0x5626e26b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.122+0000 7fc016bde740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26b36c0 0x5626e26b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.122+0000 7fc016bde740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5626e260e0b0 con 0x5626e26b36c0 2024-09-10T08:05:53.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.122+0000 7fc016bde740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5626e25b5ab0 con 0x5626e26a8e00 2024-09-10T08:05:53.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.122+0000 7fc016bde740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5626e260a070 con 0x5626e26a84e0 2024-09-10T08:05:53.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.122+0000 7fc012000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26a84e0 0x5626e26a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.122+0000 7fc012000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26a84e0 0x5626e26a88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:44332/0 (socket says 172.21.15.2:44332) 2024-09-10T08:05:53.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.122+0000 7fc012000640 1 -- 172.21.15.2:0/264402457 learned_addr learned my addr 172.21.15.2:0/264402457 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:53.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.122+0000 7fc010c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26b36c0 0x5626e26b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.122+0000 7fc011600640 1 --2- 172.21.15.2:0/264402457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a8e00 0x5626e26b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.123+0000 7fc012000640 1 -- 172.21.15.2:0/264402457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a8e00 msgr2=0x5626e26b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:05:53.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.123+0000 7fc012000640 1 --2- 172.21.15.2:0/264402457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a8e00 0x5626e26b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.123+0000 7fc012000640 1 -- 172.21.15.2:0/264402457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26b36c0 msgr2=0x5626e26b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:05:53.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.123+0000 7fc012000640 1 --2- 172.21.15.2:0/264402457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26b36c0 0x5626e26b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.123+0000 7fc012000640 1 -- 172.21.15.2:0/264402457 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e25c24a0 con 0x5626e26a84e0 2024-09-10T08:05:53.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.123+0000 7fc011600640 1 --2- 172.21.15.2:0/264402457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a8e00 0x5626e26b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:53.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.123+0000 7fc010c00640 1 --2- 172.21.15.2:0/264402457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26b36c0 0x5626e26b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.123+0000 7fc00a000640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbff40a2c00 0x7fbff40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.123+0000 7fc012000640 1 --2- 172.21.15.2:0/264402457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26a84e0 0x5626e26a88c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc004009ba0 tx=0x7fc004011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c1a0a0172a60769 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.123+0000 7fc00a000640 1 -- 172.21.15.2:0/2089810566 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fc0000f0090 con 0x7fc00c0745c0 2024-09-10T08:05:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.124+0000 7fc011600640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbff40a2c00 0x7fbff40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.124+0000 7fbffe600640 1 -- 172.21.15.2:0/264402457 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004004230 con 0x5626e26a84e0 2024-09-10T08:05:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.124+0000 7fbffe600640 1 -- 172.21.15.2:0/264402457 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc004009e10 con 0x5626e26a84e0 2024-09-10T08:05:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.124+0000 7fbffe600640 1 -- 172.21.15.2:0/264402457 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004004c90 con 0x5626e26a84e0 2024-09-10T08:05:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.125+0000 7fc016bde740 1 -- 172.21.15.2:0/264402457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26a84e0 msgr2=0x5626e26a88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.125+0000 7fc011600640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbff40a2c00 0x7fbff40a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc0000bc5e0 tx=0x7fc0000f3ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.125+0000 7fc016bde740 1 --2- 172.21.15.2:0/264402457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26a84e0 0x5626e26a88c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc004009ba0 tx=0x7fc004011810 comp rx=0 tx=0).stop 2024-09-10T08:05:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.125+0000 7fc016bde740 1 -- 172.21.15.2:0/264402457 shutdown_connections 2024-09-10T08:05:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.125+0000 7fc016bde740 1 --2- 172.21.15.2:0/264402457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26b36c0 0x5626e26b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.125+0000 7fc016bde740 1 --2- 172.21.15.2:0/264402457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a8e00 0x5626e26b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.125+0000 7fc016bde740 1 --2- 172.21.15.2:0/264402457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26a84e0 0x5626e26a88c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.125+0000 7fc016bde740 1 -- 172.21.15.2:0/264402457 >> 172.21.15.2:0/264402457 conn(0x5626e26a4a70 msgr2=0x5626e26a4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.125+0000 7fc016bde740 1 -- 172.21.15.2:0/264402457 shutdown_connections 2024-09-10T08:05:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.125+0000 7fc016bde740 1 -- 172.21.15.2:0/264402457 wait complete. 2024-09-10T08:05:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.125+0000 7fc016bde740 1 -- start start 2024-09-10T08:05:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.126+0000 7fc016bde740 1 Processor -- start 2024-09-10T08:05:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.126+0000 7fc016bde740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a84e0 0x5626e26b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.126+0000 7fc016bde740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.126+0000 7fc016bde740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26b36c0 0x5626e26b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.126+0000 7fc016bde740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5626e26b6330 con 0x5626e26a8e00 2024-09-10T08:05:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.126+0000 7fc016bde740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5626e2596920 con 0x5626e26a84e0 2024-09-10T08:05:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.126+0000 7fc016bde740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5626e26b64b0 con 0x5626e26b36c0 2024-09-10T08:05:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.126+0000 7fc016bde740 10 client.0 _ll_register_callbacks cb 0x5626e25ba8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:05:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.126+0000 7fc010c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.126+0000 7fc012000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a84e0 0x5626e26b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.126+0000 7fc010c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45964/0 (socket says 172.21.15.2:45964) 2024-09-10T08:05:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.126+0000 7fc010c00640 1 -- 172.21.15.2:0/4198412356 learned_addr learned my addr 172.21.15.2:0/4198412356 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.126+0000 7fc012000640 1 --2- 172.21.15.2:0/4198412356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26b36c0 0x5626e26b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.127+0000 7fc010c00640 1 -- 172.21.15.2:0/4198412356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26b36c0 msgr2=0x5626e26b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:05:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.127+0000 7fc010c00640 1 --2- 172.21.15.2:0/4198412356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26b36c0 0x5626e26b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.127+0000 7fc010c00640 1 -- 172.21.15.2:0/4198412356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a84e0 msgr2=0x5626e26b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:05:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.127+0000 7fc010c00640 1 --2- 172.21.15.2:0/4198412356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a84e0 0x5626e26b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.127+0000 7fc010c00640 1 -- 172.21.15.2:0/4198412356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e237f2b0 con 0x5626e26a8e00 2024-09-10T08:05:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.127+0000 7fc010c00640 1 --2- 172.21.15.2:0/4198412356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26b0ab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbff8012700 tx=0x7fbff8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a776f6ca4dcd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.127+0000 7fbff2a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:05:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.127+0000 7fbff2a00640 1 -- 172.21.15.2:0/4198412356 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff8004020 con 0x5626e26a8e00 2024-09-10T08:05:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.127+0000 7fbff2a00640 1 -- 172.21.15.2:0/4198412356 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fbff80041c0 con 0x5626e26a8e00 2024-09-10T08:05:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.127+0000 7fc016bde740 1 -- 172.21.15.2:0/4198412356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc004009e10 con 0x5626e26a8e00 2024-09-10T08:05:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.128+0000 7fbff2a00640 1 -- 172.21.15.2:0/4198412356 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fbff80043c0 con 0x5626e26a8e00 2024-09-10T08:05:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.128+0000 7fc016bde740 10 client.14613 fetch_fsmap learned FSMap version 17 2024-09-10T08:05:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.128+0000 7fc016bde740 1 -- 172.21.15.2:0/4198412356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5626e26c2010 con 0x5626e26a8e00 2024-09-10T08:05:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.128+0000 7fbff2a00640 1 -- 172.21.15.2:0/4198412356 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc004009e10 con 0x5626e26a8e00 2024-09-10T08:05:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.128+0000 7fbff2a00640 1 -- 172.21.15.2:0/4198412356 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fbff8003af0 con 0x5626e26a8e00 2024-09-10T08:05:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.128+0000 7fc016bde740 10 client.14613 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T08:05:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.128+0000 7fc016bde740 10 client.14613 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:05:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.128+0000 7fc016bde740 20 client.14613 populate_metadata read hostname 'smithi002' 2024-09-10T08:05:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.128+0000 7fc016bde740 1 --2- 172.21.15.2:0/4198412356 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626e26c4640 0x5626e26c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.128+0000 7fc016bde740 4 client.14613 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:05:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.128+0000 7fc016bde740 1 -- 172.21.15.2:0/4198412356 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626e261b620 con 0x5626e26c4640 2024-09-10T08:05:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.129+0000 7fc012000640 1 --2- 172.21.15.2:0/4198412356 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626e26c4640 0x5626e26c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.129+0000 7fc012000640 1 --2- 172.21.15.2:0/4198412356 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626e26c4640 0x5626e26c5ab0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e5a0efb7bb4c0a3 server_cookie=b42c001e18ca28ac in_seq=0 out_seq=0 2024-09-10T08:05:53.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.129+0000 7fbff2a00640 10 client.14613 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:05:53.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.130+0000 7fbff2a00640 1 -- 172.21.15.2:0/4198412356 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626e261b620 con 0x5626e26c4640 2024-09-10T08:05:53.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.130+0000 7fbff2a00640 10 client.14613 handle_command_reply: tid=0 2024-09-10T08:05:53.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.131+0000 7fc016bde740 1 client.14613 shutdown 2024-09-10T08:05:53.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.131+0000 7fc016bde740 1 -- 172.21.15.2:0/4198412356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 msgr2=0x5626e26b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:53.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.131+0000 7fc016bde740 1 --2- 172.21.15.2:0/4198412356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26b0ab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbff8012700 tx=0x7fbff8012bc0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.131+0000 7fc016bde740 1 -- 172.21.15.2:0/4198412356 shutdown_connections 2024-09-10T08:05:53.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.131+0000 7fc016bde740 1 --2- 172.21.15.2:0/4198412356 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626e26c4640 0x5626e26c5ab0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.131+0000 7fc016bde740 1 --2- 172.21.15.2:0/4198412356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26b36c0 0x5626e26b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.131+0000 7fc016bde740 1 --2- 172.21.15.2:0/4198412356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26b0ab0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.131+0000 7fbff2a00640 0 client.14613 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:05:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.131+0000 7fc016bde740 1 --2- 172.21.15.2:0/4198412356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a84e0 0x5626e26b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.131+0000 7fc016bde740 1 -- 172.21.15.2:0/4198412356 >> 172.21.15.2:0/4198412356 conn(0x5626e26a4a70 msgr2=0x5626e26aa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.131+0000 7fc016bde740 1 -- 172.21.15.2:0/4198412356 shutdown_connections 2024-09-10T08:05:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.131+0000 7fc016bde740 1 -- 172.21.15.2:0/4198412356 wait complete. 2024-09-10T08:05:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.131+0000 7fc016bde740 20 client.14613 trim_cache size 0 max 16384 2024-09-10T08:05:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.150+0000 7fc016bde740 1 Processor -- start 2024-09-10T08:05:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.150+0000 7fc016bde740 1 -- start start 2024-09-10T08:05:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.150+0000 7fc016bde740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a84e0 0x5626e26c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.150+0000 7fc016bde740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.150+0000 7fc016bde740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26b36c0 0x5626e26b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.150+0000 7fc016bde740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5626e26c3df0 con 0x5626e26a8e00 2024-09-10T08:05:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.150+0000 7fc016bde740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5626e26c44c0 con 0x5626e26a84e0 2024-09-10T08:05:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.150+0000 7fc016bde740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5626e26c4b90 con 0x5626e26b36c0 2024-09-10T08:05:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.151+0000 7fc010c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.151+0000 7fc012000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a84e0 0x5626e26c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.151+0000 7fc010c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45972/0 (socket says 172.21.15.2:45972) 2024-09-10T08:05:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.151+0000 7fc010c00640 1 -- 172.21.15.2:0/1350571172 learned_addr learned my addr 172.21.15.2:0/1350571172 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.151+0000 7fc012000640 1 --2- 172.21.15.2:0/1350571172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26b36c0 0x5626e26b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.151+0000 7fc010c00640 1 -- 172.21.15.2:0/1350571172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26b36c0 msgr2=0x5626e26b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:05:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.151+0000 7fc010c00640 1 --2- 172.21.15.2:0/1350571172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26b36c0 0x5626e26b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.151+0000 7fc010c00640 1 -- 172.21.15.2:0/1350571172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a84e0 msgr2=0x5626e26c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:05:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.151+0000 7fc010c00640 1 --2- 172.21.15.2:0/1350571172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a84e0 0x5626e26c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.151+0000 7fc010c00640 1 -- 172.21.15.2:0/1350571172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e26af520 con 0x5626e26a8e00 2024-09-10T08:05:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.151+0000 7fc012000640 1 --2- 172.21.15.2:0/1350571172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26b36c0 0x5626e26b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:05:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.151+0000 7fc010c00640 1 --2- 172.21.15.2:0/1350571172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26a4e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbff80058a0 tx=0x7fbff8009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e34fdb9d94149a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.151+0000 7fbffdc00640 1 -- 172.21.15.2:0/1350571172 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff801c4c0 con 0x5626e26a8e00 2024-09-10T08:05:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.152+0000 7fbffdc00640 1 -- 172.21.15.2:0/1350571172 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff801cdf0 con 0x5626e26a8e00 2024-09-10T08:05:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.152+0000 7fbffdc00640 1 -- 172.21.15.2:0/1350571172 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff800fe20 con 0x5626e26a8e00 2024-09-10T08:05:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.152+0000 7fc016bde740 1 -- 172.21.15.2:0/1350571172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 msgr2=0x5626e26a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.152+0000 7fc016bde740 1 --2- 172.21.15.2:0/1350571172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26a4e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbff80058a0 tx=0x7fbff8009d80 comp rx=0 tx=0).stop 2024-09-10T08:05:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.152+0000 7fc016bde740 1 -- 172.21.15.2:0/1350571172 shutdown_connections 2024-09-10T08:05:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.152+0000 7fc016bde740 1 --2- 172.21.15.2:0/1350571172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26b36c0 0x5626e26b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.152+0000 7fc016bde740 1 --2- 172.21.15.2:0/1350571172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26a4e50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.152+0000 7fc016bde740 1 --2- 172.21.15.2:0/1350571172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26a84e0 0x5626e26c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.152+0000 7fc016bde740 1 -- 172.21.15.2:0/1350571172 >> 172.21.15.2:0/1350571172 conn(0x5626e26a4a70 msgr2=0x5626e26a9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:53.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.152+0000 7fc016bde740 1 -- 172.21.15.2:0/1350571172 shutdown_connections 2024-09-10T08:05:53.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.152+0000 7fc016bde740 1 -- 172.21.15.2:0/1350571172 wait complete. 2024-09-10T08:05:53.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.153+0000 7fc016bde740 1 -- start start 2024-09-10T08:05:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.153+0000 7fc016bde740 1 Processor -- start 2024-09-10T08:05:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.153+0000 7fc016bde740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26a84e0 0x5626e26c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.153+0000 7fc016bde740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.153+0000 7fc016bde740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26b36c0 0x5626e26cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.153+0000 7fc016bde740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5626e26c5930 con 0x5626e26a8e00 2024-09-10T08:05:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.153+0000 7fc016bde740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5626e26c5260 con 0x5626e26b36c0 2024-09-10T08:05:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.153+0000 7fc016bde740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5626e26c6000 con 0x5626e26a84e0 2024-09-10T08:05:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.153+0000 7fc016bde740 10 client.0 _ll_register_callbacks cb 0x7fbff8009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:05:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.154+0000 7fc010c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.154+0000 7fc012000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26a84e0 0x5626e26c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.154+0000 7fc010c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45976/0 (socket says 172.21.15.2:45976) 2024-09-10T08:05:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.154+0000 7fc010c00640 1 -- 172.21.15.2:0/118483104 learned_addr learned my addr 172.21.15.2:0/118483104 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.154+0000 7fc012000640 1 --2- 172.21.15.2:0/118483104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26b36c0 0x5626e26cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.154+0000 7fc010c00640 1 -- 172.21.15.2:0/118483104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26a84e0 msgr2=0x5626e26c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:05:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.154+0000 7fc010c00640 1 --2- 172.21.15.2:0/118483104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26a84e0 0x5626e26c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.154+0000 7fc010c00640 1 -- 172.21.15.2:0/118483104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26b36c0 msgr2=0x5626e26cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:05:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.154+0000 7fc010c00640 1 --2- 172.21.15.2:0/118483104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26b36c0 0x5626e26cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.154+0000 7fc010c00640 1 -- 172.21.15.2:0/118483104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e26ccda0 con 0x5626e26a8e00 2024-09-10T08:05:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.154+0000 7fc012000640 1 --2- 172.21.15.2:0/118483104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26a84e0 0x5626e26c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.154+0000 7fc010c00640 1 --2- 172.21.15.2:0/118483104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26c87b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009940 tx=0x7fbff800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83c68da4b213ae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.154+0000 7fbff2a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:05:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.155+0000 7fbff2a00640 1 -- 172.21.15.2:0/118483104 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff8005040 con 0x5626e26a8e00 2024-09-10T08:05:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.155+0000 7fbff2a00640 1 -- 172.21.15.2:0/118483104 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fbff80051e0 con 0x5626e26a8e00 2024-09-10T08:05:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.155+0000 7fc016bde740 1 -- 172.21.15.2:0/118483104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbff801cdf0 con 0x5626e26a8e00 2024-09-10T08:05:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.155+0000 7fbff2a00640 1 -- 172.21.15.2:0/118483104 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fbff803d050 con 0x5626e26a8e00 2024-09-10T08:05:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.155+0000 7fc016bde740 10 client.14619 fetch_fsmap learned FSMap version 17 2024-09-10T08:05:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.155+0000 7fc016bde740 1 -- 172.21.15.2:0/118483104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5626e272f5b0 con 0x5626e26a8e00 2024-09-10T08:05:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.155+0000 7fbff2a00640 1 -- 172.21.15.2:0/118483104 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff801cdf0 con 0x5626e26a8e00 2024-09-10T08:05:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.155+0000 7fbff2a00640 1 -- 172.21.15.2:0/118483104 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fbff802b070 con 0x5626e26a8e00 2024-09-10T08:05:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.155+0000 7fc016bde740 10 client.14619 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T08:05:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.155+0000 7fc016bde740 10 client.14619 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:05:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.156+0000 7fc016bde740 20 client.14619 populate_metadata read hostname 'smithi002' 2024-09-10T08:05:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.156+0000 7fc016bde740 1 --2- 172.21.15.2:0/118483104 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626e2733f60 0x5626e2732c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.156+0000 7fc016bde740 4 client.14619 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:05:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.156+0000 7fc016bde740 1 -- 172.21.15.2:0/118483104 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "status"}) -- 0x5626e2615b30 con 0x5626e2733f60 2024-09-10T08:05:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.156+0000 7fc012000640 1 --2- 172.21.15.2:0/118483104 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626e2733f60 0x5626e2732c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.156+0000 7fc012000640 1 --2- 172.21.15.2:0/118483104 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626e2733f60 0x5626e2732c70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c54d303b08017ced server_cookie=e9d47f1a547acd80 in_seq=0 out_seq=0 2024-09-10T08:05:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.156+0000 7fbff2a00640 10 client.14619 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:05:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fbff2a00640 1 -- 172.21.15.2:0/118483104 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5626e2615b30 con 0x5626e2733f60 2024-09-10T08:05:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fbff2a00640 10 client.14619 handle_command_reply: tid=0 2024-09-10T08:05:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fc016bde740 1 client.14619 shutdown 2024-09-10T08:05:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fc016bde740 1 -- 172.21.15.2:0/118483104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 msgr2=0x5626e26c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fc016bde740 1 --2- 172.21.15.2:0/118483104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26c87b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009940 tx=0x7fbff800fc90 comp rx=0 tx=0).stop 2024-09-10T08:05:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fc016bde740 1 -- 172.21.15.2:0/118483104 shutdown_connections 2024-09-10T08:05:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fc016bde740 1 --2- 172.21.15.2:0/118483104 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626e2733f60 0x5626e2732c70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fc016bde740 1 --2- 172.21.15.2:0/118483104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626e26b36c0 0x5626e26cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fc016bde740 1 --2- 172.21.15.2:0/118483104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626e26a8e00 0x5626e26c87b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fc016bde740 1 --2- 172.21.15.2:0/118483104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e26a84e0 0x5626e26c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fc016bde740 1 -- 172.21.15.2:0/118483104 >> 172.21.15.2:0/118483104 conn(0x5626e26a4a70 msgr2=0x5626e26b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fbff2a00640 0 client.14619 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:05:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fc016bde740 1 -- 172.21.15.2:0/118483104 shutdown_connections 2024-09-10T08:05:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fc016bde740 1 -- 172.21.15.2:0/118483104 wait complete. 2024-09-10T08:05:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.157+0000 7fc016bde740 20 client.14619 trim_cache size 0 max 16384 2024-09-10T08:05:53.160 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:05:53.160 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "26ec64b0-6f4a-11ef-bcea-c7b262605968", 2024-09-10T08:05:53.160 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-09-10T08:05:53.160 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24536, 2024-09-10T08:05:53.161 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-09-10T08:05:53.161 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-09-10T08:05:53.161 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-09-10T08:05:53.161 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 25.667582887999998, 2024-09-10T08:05:53.161 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 17, 2024-09-10T08:05:53.161 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 91, 2024-09-10T08:05:53.161 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 91, 2024-09-10T08:05:53.161 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 39.546161544 2024-09-10T08:05:53.161 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:05:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.158+0000 7fc012a00640 1 -- 172.21.15.2:0/2089810566 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbff40a2c00 msgr2=0x7fbff40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.158+0000 7fc012a00640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbff40a2c00 0x7fbff40a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc0000bc5e0 tx=0x7fc0000f3ad0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.158+0000 7fc012a00640 1 -- 172.21.15.2:0/2089810566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc00c0745c0 msgr2=0x7fc00c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.158+0000 7fc012a00640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc00c0745c0 0x7fc00c19a4a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc000005e30 tx=0x7fc000003c90 comp rx=0 tx=0).stop 2024-09-10T08:05:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.158+0000 7fc012a00640 1 -- 172.21.15.2:0/2089810566 shutdown_connections 2024-09-10T08:05:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.158+0000 7fc012a00640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbff40a2c00 0x7fbff40a50c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.158+0000 7fc012a00640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00c10e260 0x7fc00c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.158+0000 7fc012a00640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc00c074ee0 0x7fc00c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.158+0000 7fc012a00640 1 --2- 172.21.15.2:0/2089810566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc00c0745c0 0x7fc00c19a4a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.158+0000 7fc012a00640 1 -- 172.21.15.2:0/2089810566 >> 172.21.15.2:0/2089810566 conn(0x7fc00c0fd740 msgr2=0x7fc00c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:53.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.158+0000 7fc012a00640 1 -- 172.21.15.2:0/2089810566 shutdown_connections 2024-09-10T08:05:53.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:53.158+0000 7fc012a00640 1 -- 172.21.15.2:0/2089810566 wait complete. 2024-09-10T08:05:54.211 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T08:05:54.212 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:05:54.212 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T08:05:54.602 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:05:55.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:54 smithi002 ceph-mon[30563]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:55.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:54 smithi155 ceph-mon[35405]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:55.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.165+0000 7fc39f200640 1 -- 172.21.15.2:0/3571672777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3981049a0 msgr2=0x7fc3981089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:55.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.165+0000 7fc39f200640 1 --2- 172.21.15.2:0/3571672777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3981049a0 0x7fc3981089f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc380009860 tx=0x7fc38002a340 comp rx=0 tx=0).stop 2024-09-10T08:05:55.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.165+0000 7fc39f200640 1 -- 172.21.15.2:0/3571672777 shutdown_connections 2024-09-10T08:05:55.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.165+0000 7fc39f200640 1 --2- 172.21.15.2:0/3571672777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc398109120 0x7fc398110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:55.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.165+0000 7fc39f200640 1 --2- 172.21.15.2:0/3571672777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3981049a0 0x7fc3981089f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:55.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.165+0000 7fc39f200640 1 --2- 172.21.15.2:0/3571672777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc398103ff0 0x7fc3981043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:55.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.165+0000 7fc39f200640 1 -- 172.21.15.2:0/3571672777 >> 172.21.15.2:0/3571672777 conn(0x7fc3980ffa90 msgr2=0x7fc398101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:55.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.165+0000 7fc39f200640 1 -- 172.21.15.2:0/3571672777 shutdown_connections 2024-09-10T08:05:55.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.165+0000 7fc39f200640 1 -- 172.21.15.2:0/3571672777 wait complete. 2024-09-10T08:05:55.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.166+0000 7fc39f200640 1 Processor -- start 2024-09-10T08:05:55.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.167+0000 7fc39f200640 1 -- start start 2024-09-10T08:05:55.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.167+0000 7fc39f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc398103ff0 0x7fc39819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:55.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.167+0000 7fc39f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3981049a0 0x7fc39819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:55.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.167+0000 7fc39f200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc398109120 0x7fc39819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:55.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.167+0000 7fc39f200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc398114120 con 0x7fc398109120 2024-09-10T08:05:55.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.167+0000 7fc39f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc398113fa0 con 0x7fc398103ff0 2024-09-10T08:05:55.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.167+0000 7fc39f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3981142a0 con 0x7fc3981049a0 2024-09-10T08:05:55.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:54 smithi107 ceph-mon[35048]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:55.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.167+0000 7fc397e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc398103ff0 0x7fc39819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:55.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.167+0000 7fc39d200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc398109120 0x7fc39819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:55.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.167+0000 7fc397e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc398103ff0 0x7fc39819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52718/0 (socket says 172.21.15.2:52718) 2024-09-10T08:05:55.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.167+0000 7fc397e00640 1 -- 172.21.15.2:0/4094568708 learned_addr learned my addr 172.21.15.2:0/4094568708 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:05:55.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.167+0000 7fc38f400640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3981049a0 0x7fc39819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:55.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.168+0000 7fc397e00640 1 -- 172.21.15.2:0/4094568708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3981049a0 msgr2=0x7fc39819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:55.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.168+0000 7fc397e00640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3981049a0 0x7fc39819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:55.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.168+0000 7fc397e00640 1 -- 172.21.15.2:0/4094568708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc398109120 msgr2=0x7fc39819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:55.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.168+0000 7fc397e00640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc398109120 0x7fc39819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:55.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.168+0000 7fc397e00640 1 -- 172.21.15.2:0/4094568708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39819e6b0 con 0x7fc398103ff0 2024-09-10T08:05:55.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.168+0000 7fc38f400640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3981049a0 0x7fc39819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:55.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.168+0000 7fc39d200640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc398109120 0x7fc39819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:05:55.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.168+0000 7fc397e00640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc398103ff0 0x7fc39819a760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc38800e9c0 tx=0x7fc38800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:55.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.169+0000 7fc395600640 1 -- 172.21.15.2:0/4094568708 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38800ccc0 con 0x7fc398103ff0 2024-09-10T08:05:55.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.169+0000 7fc39f200640 1 -- 172.21.15.2:0/4094568708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39819e9a0 con 0x7fc398103ff0 2024-09-10T08:05:55.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.169+0000 7fc395600640 1 -- 172.21.15.2:0/4094568708 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc388005500 con 0x7fc398103ff0 2024-09-10T08:05:55.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.169+0000 7fc395600640 1 -- 172.21.15.2:0/4094568708 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388010430 con 0x7fc398103ff0 2024-09-10T08:05:55.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.169+0000 7fc39f200640 1 -- 172.21.15.2:0/4094568708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3981a7310 con 0x7fc398103ff0 2024-09-10T08:05:55.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.170+0000 7fc39f200640 1 -- 172.21.15.2:0/4094568708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3981043d0 con 0x7fc398103ff0 2024-09-10T08:05:55.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.172+0000 7fc395600640 1 -- 172.21.15.2:0/4094568708 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fc388044b30 con 0x7fc398103ff0 2024-09-10T08:05:55.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.173+0000 7fc395600640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc37c0a2c00 0x7fc37c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:05:55.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.173+0000 7fc395600640 1 -- 172.21.15.2:0/4094568708 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fc3880cf560 con 0x7fc398103ff0 2024-09-10T08:05:55.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.173+0000 7fc38f400640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc37c0a2c00 0x7fc37c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:05:55.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.174+0000 7fc38f400640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc37c0a2c00 0x7fc37c0a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc38002a930 tx=0x7fc3800058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:05:55.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.175+0000 7fc395600640 1 -- 172.21.15.2:0/4094568708 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc38809c220 con 0x7fc398103ff0 2024-09-10T08:05:55.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.397+0000 7fc39f200640 1 -- 172.21.15.2:0/4094568708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc39819bac0 con 0x7fc398103ff0 2024-09-10T08:05:55.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.397+0000 7fc395600640 1 -- 172.21.15.2:0/4094568708 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fc388044e20 con 0x7fc398103ff0 2024-09-10T08:05:55.396 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T08:05:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.400+0000 7fc39f200640 1 -- 172.21.15.2:0/4094568708 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc37c0a2c00 msgr2=0x7fc37c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.400+0000 7fc39f200640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc37c0a2c00 0x7fc37c0a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc38002a930 tx=0x7fc3800058d0 comp rx=0 tx=0).stop 2024-09-10T08:05:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.400+0000 7fc39f200640 1 -- 172.21.15.2:0/4094568708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc398103ff0 msgr2=0x7fc39819a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:05:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.400+0000 7fc39f200640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc398103ff0 0x7fc39819a760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc38800e9c0 tx=0x7fc38800edc0 comp rx=0 tx=0).stop 2024-09-10T08:05:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.400+0000 7fc39f200640 1 -- 172.21.15.2:0/4094568708 shutdown_connections 2024-09-10T08:05:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.400+0000 7fc39f200640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc398109120 0x7fc39819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.400+0000 7fc39f200640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc37c0a2c00 0x7fc37c0a50c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.400+0000 7fc39f200640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3981049a0 0x7fc39819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.400+0000 7fc39f200640 1 --2- 172.21.15.2:0/4094568708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc398103ff0 0x7fc39819a760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:05:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.400+0000 7fc39f200640 1 -- 172.21.15.2:0/4094568708 >> 172.21.15.2:0/4094568708 conn(0x7fc3980ffa90 msgr2=0x7fc398100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:05:55.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.400+0000 7fc39f200640 1 -- 172.21.15.2:0/4094568708 shutdown_connections 2024-09-10T08:05:55.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:55.400+0000 7fc39f200640 1 -- 172.21.15.2:0/4094568708 wait complete. 2024-09-10T08:05:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:55 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/4094568708' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:05:56.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:55 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/4094568708' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:05:56.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:55 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/4094568708' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:05:56.378 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T08:05:56.378 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T08:05:56.378 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2024-09-10T08:05:56.378 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T08:05:56.378 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:05:56.378 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T08:05:56.378 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T08:05:56.378 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T08:05:56.378 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T08:05:56.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:05:56.379 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2024-09-10T08:05:56.416 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T08:05:56.416 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T08:05:56.416 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T08:05:56.416 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:05:56.416 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2024-09-10T08:05:56.416 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:05:56.416 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:05:56.416 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:05:56.416 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f0 2024-09-10T08:05:56.416 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:05:56.417 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:05:56.417 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f1 2024-09-10T08:05:56.417 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T08:05:56.417 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:05:56.417 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f0 2024-09-10T08:05:56.417 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T08:05:56.417 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 42169sec preferred_lft 42169sec 2024-09-10T08:05:56.417 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link 2024-09-10T08:05:56.417 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:05:56.417 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:05:56.417 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:05:56.417 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f1 2024-09-10T08:05:56.418 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T08:05:56.418 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:05:56.418 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-10T08:05:56.418 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add name ceph-brx type bridge 2024-09-10T08:05:56.419 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr flush dev ceph-brx 2024-09-10T08:05:56.419 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set ceph-brx up 2024-09-10T08:05:56.419 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T08:05:56.419 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-10T08:05:56.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:05:56.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:05:56.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:05:56.839 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T08:05:56.897 INFO:teuthology.orchestra.run.smithi002.stdout:1 2024-09-10T08:05:56.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:05:56.900 DEBUG:teuthology.orchestra.run.smithi002:> route 2024-09-10T08:05:56.952 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2024-09-10T08:05:56.952 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T08:05:56.953 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T08:05:56.953 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T08:05:56.953 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T08:05:56.953 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:05:56.953 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-10T08:05:56.953 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T08:05:56.953 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T08:05:56.954 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T08:05:56.954 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-10T08:05:57.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:56 smithi002 ceph-mon[30563]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:57.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:05:57.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:56 smithi155 ceph-mon[35405]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:57.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:56 smithi107 ceph-mon[35048]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:57.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:05:57.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:05:57.544 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-09-10T08:05:57.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:05:57.581 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2024-09-10T08:05:57.656 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:05:57.656 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-10T08:05:57.656 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:05:57.656 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T08:05:57.657 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-10T08:05:57.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:05:57.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:05:57.837 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T08:05:57.840 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:05:57.841 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-10T08:05:57.841 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T08:05:57.841 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T08:05:57.841 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T08:05:57.841 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T08:05:57.841 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T08:05:57.841 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-10T08:05:57.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:05:58.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:05:58.086 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:05:58.086 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-10T08:05:58.086 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2024-09-10T08:05:58.086 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T08:05:58.086 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-10T08:05:58.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:05:58.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:05:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:05:58.261 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T08:05:58.261 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T08:05:58.261 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:05:58.298 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T08:05:58.299 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T08:05:58.299 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:05:58.374 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2024-09-10T08:05:58.473 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T08:05:58.514 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2024-09-10T08:05:58.514 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2024-09-10T08:05:58.514 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2024-09-10T08:05:58.514 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/bpf 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2024-09-10T08:05:58.515 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2024-09-10T08:05:58.516 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2024-09-10T08:05:58.516 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T08:05:58.516 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T08:05:58.516 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T08:05:58.516 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T08:05:58.516 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2024-09-10T08:05:58.516 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2024-09-10T08:05:58.516 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2024-09-10T08:05:58.516 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/8b77180d02a50dc7d41c6fb350e9176823ad7fce8da98fab8a6f65171caeb7ed/merged 2024-09-10T08:05:58.516 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/2a5bf613df4ba93bca82549aa9fdfe1d50ce528d7a461571ea734b7c206f084f/merged 2024-09-10T08:05:58.516 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2024-09-10T08:05:58.516 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:05:58.516 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/5f04e6afa080343dc3615275d9e0bfb3b4fe8b782642c22863904f11f1aca3ae/merged 2024-09-10T08:05:58.517 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/df622566e4be8abf7f540fce7fdfd71b997e644268da88d9bb0d8ef06b0edf18/merged 2024-09-10T08:05:58.517 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/ce9685761d5ba088f8b347f9d075a8a7d5bc7d8a508909cf8ec37115efd5d195/merged 2024-09-10T08:05:58.517 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f81ada5a8f2bfa637762fe19587729579d5d285cd6ff693c5b4288b701c2e401/merged 2024-09-10T08:05:58.517 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/6d0cc48f14e10ae3106e59cad6426e62477e1fe4163188782d13437891207b8a/merged 2024-09-10T08:05:58.517 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/bc453b281eed1decaa02993d4bb617bc76ed6335a8568b1ebfefe5f03b2f1b3a/merged 2024-09-10T08:05:58.517 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/c4a0f7485db800a7aad3d1625ef1b6bedd06c518e9bf1dc07af4dda5fdab5976/merged 2024-09-10T08:05:58.517 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/4e4e159b20073d0bd769dc41715209d02bfb9262974e817f16a6eb94a6374a04/merged 2024-09-10T08:05:58.517 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2024-09-10T08:05:58.517 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:05:58.517 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:05:58.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:05:58.518 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2024-09-10T08:05:58.594 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T08:05:58.594 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T08:05:58.639 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2024-09-10T08:05:58.698 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T08:05:58.735 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2024-09-10T08:05:58.735 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2024-09-10T08:05:58.735 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2024-09-10T08:05:58.735 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2024-09-10T08:05:58.735 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2024-09-10T08:05:58.736 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2024-09-10T08:05:58.736 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2024-09-10T08:05:58.736 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2024-09-10T08:05:58.736 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2024-09-10T08:05:58.736 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/bpf 2024-09-10T08:05:58.736 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2024-09-10T08:05:58.736 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2024-09-10T08:05:58.736 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:05:58.736 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2024-09-10T08:05:58.736 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2024-09-10T08:05:58.736 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2024-09-10T08:05:58.736 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2024-09-10T08:05:58.736 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2024-09-10T08:05:58.737 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2024-09-10T08:05:58.737 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T08:05:58.737 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T08:05:58.737 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T08:05:58.737 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T08:05:58.737 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2024-09-10T08:05:58.737 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2024-09-10T08:05:58.737 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2024-09-10T08:05:58.737 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/8b77180d02a50dc7d41c6fb350e9176823ad7fce8da98fab8a6f65171caeb7ed/merged 2024-09-10T08:05:58.737 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/2a5bf613df4ba93bca82549aa9fdfe1d50ce528d7a461571ea734b7c206f084f/merged 2024-09-10T08:05:58.737 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2024-09-10T08:05:58.737 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:05:58.738 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/5f04e6afa080343dc3615275d9e0bfb3b4fe8b782642c22863904f11f1aca3ae/merged 2024-09-10T08:05:58.738 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/df622566e4be8abf7f540fce7fdfd71b997e644268da88d9bb0d8ef06b0edf18/merged 2024-09-10T08:05:58.738 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/ce9685761d5ba088f8b347f9d075a8a7d5bc7d8a508909cf8ec37115efd5d195/merged 2024-09-10T08:05:58.738 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f81ada5a8f2bfa637762fe19587729579d5d285cd6ff693c5b4288b701c2e401/merged 2024-09-10T08:05:58.738 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/6d0cc48f14e10ae3106e59cad6426e62477e1fe4163188782d13437891207b8a/merged 2024-09-10T08:05:58.738 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/bc453b281eed1decaa02993d4bb617bc76ed6335a8568b1ebfefe5f03b2f1b3a/merged 2024-09-10T08:05:58.738 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/c4a0f7485db800a7aad3d1625ef1b6bedd06c518e9bf1dc07af4dda5fdab5976/merged 2024-09-10T08:05:58.738 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/4e4e159b20073d0bd769dc41715209d02bfb9262974e817f16a6eb94a6374a04/merged 2024-09-10T08:05:58.738 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2024-09-10T08:05:58.738 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:05:58.738 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:05:58.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:05:58.739 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2024-09-10T08:05:58.740 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:05:58 smithi002 ceph-mon[30563]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:58.849 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:2024-09-10T08:05:58.849+0000 7f30e0508480 -1 init, newargv = 0x55c458600410 newargc=15 2024-09-10T08:05:58.849 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:ceph-fuse[72070]: starting ceph client 2024-09-10T08:05:58.860 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:ceph-fuse[72070]: starting fuse 2024-09-10T08:05:59.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:05:58 smithi155 ceph-mon[35405]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:59.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:05:58 smithi107 ceph-mon[35048]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:05:59.916 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2024-09-10T08:05:59.982 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T08:06:00.021 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2024-09-10T08:06:00.021 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2024-09-10T08:06:00.021 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2024-09-10T08:06:00.021 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2024-09-10T08:06:00.021 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2024-09-10T08:06:00.021 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2024-09-10T08:06:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2024-09-10T08:06:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2024-09-10T08:06:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2024-09-10T08:06:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/bpf 2024-09-10T08:06:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2024-09-10T08:06:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2024-09-10T08:06:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:06:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2024-09-10T08:06:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2024-09-10T08:06:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2024-09-10T08:06:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2024-09-10T08:06:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2024-09-10T08:06:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2024-09-10T08:06:00.023 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T08:06:00.023 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T08:06:00.023 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T08:06:00.023 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T08:06:00.023 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2024-09-10T08:06:00.023 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2024-09-10T08:06:00.023 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2024-09-10T08:06:00.023 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/8b77180d02a50dc7d41c6fb350e9176823ad7fce8da98fab8a6f65171caeb7ed/merged 2024-09-10T08:06:00.023 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/2a5bf613df4ba93bca82549aa9fdfe1d50ce528d7a461571ea734b7c206f084f/merged 2024-09-10T08:06:00.023 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2024-09-10T08:06:00.023 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:06:00.023 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/5f04e6afa080343dc3615275d9e0bfb3b4fe8b782642c22863904f11f1aca3ae/merged 2024-09-10T08:06:00.023 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/df622566e4be8abf7f540fce7fdfd71b997e644268da88d9bb0d8ef06b0edf18/merged 2024-09-10T08:06:00.024 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/ce9685761d5ba088f8b347f9d075a8a7d5bc7d8a508909cf8ec37115efd5d195/merged 2024-09-10T08:06:00.024 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f81ada5a8f2bfa637762fe19587729579d5d285cd6ff693c5b4288b701c2e401/merged 2024-09-10T08:06:00.024 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/6d0cc48f14e10ae3106e59cad6426e62477e1fe4163188782d13437891207b8a/merged 2024-09-10T08:06:00.024 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/bc453b281eed1decaa02993d4bb617bc76ed6335a8568b1ebfefe5f03b2f1b3a/merged 2024-09-10T08:06:00.024 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/c4a0f7485db800a7aad3d1625ef1b6bedd06c518e9bf1dc07af4dda5fdab5976/merged 2024-09-10T08:06:00.024 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/4e4e159b20073d0bd769dc41715209d02bfb9262974e817f16a6eb94a6374a04/merged 2024-09-10T08:06:00.024 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2024-09-10T08:06:00.024 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:06:00.024 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:06:00.024 INFO:teuthology.orchestra.run.smithi002.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-10T08:06:00.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:06:00.025 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2024-09-10T08:06:00.098 INFO:teuthology.orchestra.run.smithi002.stdout:84 2024-09-10T08:06:00.098 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [84] 2024-09-10T08:06:00.098 DEBUG:teuthology.orchestra.run.smithi002:> sudo stdin-killer -- python3 -c ' 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> import glob 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> import re 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> import os 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> import subprocess 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> def _find_admin_socket(client_name): 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2024-09-10T08:06:00.099 DEBUG:teuthology.orchestra.run.smithi002:> return files[0] 2024-09-10T08:06:00.100 DEBUG:teuthology.orchestra.run.smithi002:> 2024-09-10T08:06:00.100 DEBUG:teuthology.orchestra.run.smithi002:> for f in files: 2024-09-10T08:06:00.100 DEBUG:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T08:06:00.100 DEBUG:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T08:06:00.100 DEBUG:teuthology.orchestra.run.smithi002:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T08:06:00.100 DEBUG:teuthology.orchestra.run.smithi002:> contents = proc_f.read() 2024-09-10T08:06:00.100 DEBUG:teuthology.orchestra.run.smithi002:> if mountpoint in contents: 2024-09-10T08:06:00.100 DEBUG:teuthology.orchestra.run.smithi002:> return f 2024-09-10T08:06:00.100 DEBUG:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T08:06:00.100 DEBUG:teuthology.orchestra.run.smithi002:> 2024-09-10T08:06:00.100 DEBUG:teuthology.orchestra.run.smithi002:> print(_find_admin_socket("client.admin")) 2024-09-10T08:06:00.100 DEBUG:teuthology.orchestra.run.smithi002:> ' 2024-09-10T08:06:00.263 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.admin.72070.asok 2024-09-10T08:06:00.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:06:00.278 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72070.asok 2024-09-10T08:06:00.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:06:00.278 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72070.asok status 2024-09-10T08:06:00.463 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:06:00.463 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2024-09-10T08:06:00.463 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T08:06:00.463 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T08:06:00.463 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "admin", 2024-09-10T08:06:00.464 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2024-09-10T08:06:00.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T08:06:00.464 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "72070", 2024-09-10T08:06:00.464 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2024-09-10T08:06:00.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:06:00.464 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2024-09-10T08:06:00.464 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2024-09-10T08:06:00.464 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24650, 2024-09-10T08:06:00.464 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2024-09-10T08:06:00.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2024-09-10T08:06:00.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2024-09-10T08:06:00.464 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 24650 2024-09-10T08:06:00.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:06:00.465 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2024-09-10T08:06:00.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-10T08:06:00.465 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2024-09-10T08:06:00.465 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3471084590 2024-09-10T08:06:00.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:06:00.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:06:00.465 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2024-09-10T08:06:00.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-10T08:06:00.465 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2024-09-10T08:06:00.465 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3471084590 2024-09-10T08:06:00.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:06:00.465 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.24650 192.168.0.1:0/3471084590", 2024-09-10T08:06:00.465 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "192.168.0.1:0/3471084590", 2024-09-10T08:06:00.466 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2024-09-10T08:06:00.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 17, 2024-09-10T08:06:00.466 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 91, 2024-09-10T08:06:00.466 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2024-09-10T08:06:00.466 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklisted": false, 2024-09-10T08:06:00.466 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs" 2024-09-10T08:06:00.466 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:06:00.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:06:00.476 DEBUG:teuthology.orchestra.run.smithi002:> stat --file-system '--printf=%T 2024-09-10T08:06:00.476 DEBUG:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:06:00.515 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2024-09-10T08:06:00.516 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:06:00.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:06:00.516 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:06:00.614 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T08:06:00.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T08:06:00.719 INFO:teuthology.orchestra.run.smithi002.stdout:# file: . 2024-09-10T08:06:00.719 INFO:teuthology.orchestra.run.smithi002.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T08:06:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:06:00.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:06:00.727 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T08:06:00.727 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T08:06:00.765 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T08:06:00.855 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:00 smithi002 ceph-mon[30563]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:06:01.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:00 smithi155 ceph-mon[35405]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:06:01.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:00 smithi107 ceph-mon[35048]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:06:01.184 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:01.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.885+0000 7fbcaf400640 1 -- 172.21.15.2:0/2861116094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbca8103ff0 msgr2=0x7fbca81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:01.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.885+0000 7fbcaf400640 1 --2- 172.21.15.2:0/2861116094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbca8103ff0 0x7fbca81043d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009900 tx=0x7fbc9c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:06:01.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.885+0000 7fbcaf400640 1 -- 172.21.15.2:0/2861116094 shutdown_connections 2024-09-10T08:06:01.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.885+0000 7fbcaf400640 1 --2- 172.21.15.2:0/2861116094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca8109120 0x7fbca8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:01.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.885+0000 7fbcaf400640 1 --2- 172.21.15.2:0/2861116094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbca81049a0 0x7fbca81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:01.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.885+0000 7fbcaf400640 1 --2- 172.21.15.2:0/2861116094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbca8103ff0 0x7fbca81043d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:01.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.885+0000 7fbcaf400640 1 -- 172.21.15.2:0/2861116094 >> 172.21.15.2:0/2861116094 conn(0x7fbca80ffa90 msgr2=0x7fbca8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:01.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.885+0000 7fbcaf400640 1 -- 172.21.15.2:0/2861116094 shutdown_connections 2024-09-10T08:06:01.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.885+0000 7fbcaf400640 1 -- 172.21.15.2:0/2861116094 wait complete. 2024-09-10T08:06:01.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.886+0000 7fbcaf400640 1 Processor -- start 2024-09-10T08:06:01.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.886+0000 7fbcaf400640 1 -- start start 2024-09-10T08:06:01.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.887+0000 7fbcaf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbca8103ff0 0x7fbca819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:01.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.887+0000 7fbcaf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca81049a0 0x7fbca819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:01.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.887+0000 7fbcaf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbca8109120 0x7fbca819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:01.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.887+0000 7fbcaf400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbca8114170 con 0x7fbca8103ff0 2024-09-10T08:06:01.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.887+0000 7fbcaf400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbca8113ff0 con 0x7fbca81049a0 2024-09-10T08:06:01.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.887+0000 7fbcaf400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbca81142f0 con 0x7fbca8109120 2024-09-10T08:06:01.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.887+0000 7fbcaca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbca8103ff0 0x7fbca819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:01.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.887+0000 7fbcaca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbca8103ff0 0x7fbca819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57782/0 (socket says 172.21.15.2:57782) 2024-09-10T08:06:01.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.887+0000 7fbcaca00640 1 -- 172.21.15.2:0/1059204175 learned_addr learned my addr 172.21.15.2:0/1059204175 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:01.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.887+0000 7fbca7e00640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca81049a0 0x7fbca819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:01.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.887+0000 7fbcad400640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbca8109120 0x7fbca819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:01.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.888+0000 7fbca7e00640 1 -- 172.21.15.2:0/1059204175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbca8109120 msgr2=0x7fbca819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:01.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.888+0000 7fbca7e00640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbca8109120 0x7fbca819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:01.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.888+0000 7fbca7e00640 1 -- 172.21.15.2:0/1059204175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbca8103ff0 msgr2=0x7fbca819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:01.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.888+0000 7fbca7e00640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbca8103ff0 0x7fbca819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:01.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.888+0000 7fbca7e00640 1 -- 172.21.15.2:0/1059204175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca819f5b0 con 0x7fbca81049a0 2024-09-10T08:06:01.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.888+0000 7fbca7e00640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca81049a0 0x7fbca819ab40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400e9f0 tx=0x7fbc9400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:01.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.888+0000 7fbca5600640 1 -- 172.21.15.2:0/1059204175 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9400ccc0 con 0x7fbca81049a0 2024-09-10T08:06:01.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.888+0000 7fbca5600640 1 -- 172.21.15.2:0/1059204175 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc94010910 con 0x7fbca81049a0 2024-09-10T08:06:01.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.888+0000 7fbcaf400640 1 -- 172.21.15.2:0/1059204175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca819f8a0 con 0x7fbca81049a0 2024-09-10T08:06:01.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.888+0000 7fbca5600640 1 -- 172.21.15.2:0/1059204175 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94018960 con 0x7fbca81049a0 2024-09-10T08:06:01.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.888+0000 7fbcaf400640 1 -- 172.21.15.2:0/1059204175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca81a7180 con 0x7fbca81049a0 2024-09-10T08:06:01.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.889+0000 7fbcaf400640 1 -- 172.21.15.2:0/1059204175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca81043d0 con 0x7fbca81049a0 2024-09-10T08:06:01.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.892+0000 7fbca5600640 1 -- 172.21.15.2:0/1059204175 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fbc94010ac0 con 0x7fbca81049a0 2024-09-10T08:06:01.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.893+0000 7fbca5600640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbc900a2c00 0x7fbc900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:01.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.893+0000 7fbca5600640 1 -- 172.21.15.2:0/1059204175 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fbc940cf5d0 con 0x7fbca81049a0 2024-09-10T08:06:01.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.893+0000 7fbcaca00640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbc900a2c00 0x7fbc900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:01.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.894+0000 7fbcaca00640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbc900a2c00 0x7fbc900a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c0096c0 tx=0x7fbc9c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:01.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:01.897+0000 7fbca5600640 1 -- 172.21.15.2:0/1059204175 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc9409bc40 con 0x7fbca81049a0 2024-09-10T08:06:02.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.097+0000 7fbcaf400640 1 -- 172.21.15.2:0/1059204175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbca802d7c0 con 0x7fbca81049a0 2024-09-10T08:06:02.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.098+0000 7fbca5600640 1 -- 172.21.15.2:0/1059204175 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+578 (secure 0 0 0) 0x7fbc9409ff40 con 0x7fbca81049a0 2024-09-10T08:06:02.098 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3590511604 2024-09-11T07:58:03.548782+0000 2024-09-10T08:06:02.098 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/939674007 2024-09-11T07:58:03.548782+0000 2024-09-10T08:06:02.098 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2447931330 2024-09-11T07:58:03.548782+0000 2024-09-10T08:06:02.098 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/387296843 2024-09-11T07:58:03.548782+0000 2024-09-10T08:06:02.099 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1896108724 2024-09-11T07:57:36.359026+0000 2024-09-10T08:06:02.099 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2121891210 2024-09-11T07:57:36.359026+0000 2024-09-10T08:06:02.099 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/712393349 2024-09-11T07:57:36.359026+0000 2024-09-10T08:06:02.099 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/712393349 2024-09-11T07:57:36.359026+0000 2024-09-10T08:06:02.099 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3590511604 2024-09-11T07:58:03.548782+0000 2024-09-10T08:06:02.099 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1616837180 2024-09-11T07:57:36.359026+0000 2024-09-10T08:06:02.099 INFO:teuthology.orchestra.run.smithi002.stderr:listed 10 entries 2024-09-10T08:06:02.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.101+0000 7fbcaf400640 1 -- 172.21.15.2:0/1059204175 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbc900a2c00 msgr2=0x7fbc900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:02.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.101+0000 7fbcaf400640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbc900a2c00 0x7fbc900a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c0096c0 tx=0x7fbc9c0046c0 comp rx=0 tx=0).stop 2024-09-10T08:06:02.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.101+0000 7fbcaf400640 1 -- 172.21.15.2:0/1059204175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca81049a0 msgr2=0x7fbca819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:02.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.101+0000 7fbcaf400640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca81049a0 0x7fbca819ab40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400e9f0 tx=0x7fbc9400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:06:02.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.101+0000 7fbcaf400640 1 -- 172.21.15.2:0/1059204175 shutdown_connections 2024-09-10T08:06:02.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.101+0000 7fbcaf400640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbc900a2c00 0x7fbc900a50c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:02.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.101+0000 7fbcaf400640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbca8109120 0x7fbca819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:02.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.101+0000 7fbcaf400640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca81049a0 0x7fbca819ab40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:02.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.101+0000 7fbcaf400640 1 --2- 172.21.15.2:0/1059204175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbca8103ff0 0x7fbca819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:02.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.101+0000 7fbcaf400640 1 -- 172.21.15.2:0/1059204175 >> 172.21.15.2:0/1059204175 conn(0x7fbca80ffa90 msgr2=0x7fbca8100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:02.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.101+0000 7fbcaf400640 1 -- 172.21.15.2:0/1059204175 shutdown_connections 2024-09-10T08:06:02.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:02.101+0000 7fbcaf400640 1 -- 172.21.15.2:0/1059204175 wait complete. 2024-09-10T08:06:02.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:02 smithi002 ceph-mon[30563]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T08:06:02.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:02 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1059204175' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:06:02.972 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T08:06:02.973 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T08:06:03.009 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T08:06:03.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:02 smithi155 ceph-mon[35405]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T08:06:03.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:02 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1059204175' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:06:03.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:02 smithi107 ceph-mon[35048]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T08:06:03.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:02 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1059204175' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:06:03.428 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:03.832 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-10T08:06:03.832 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-09-10T08:06:03.832 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-10T08:06:03.832 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-10T08:06:03.832 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-09-10T08:06:03.832 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-10T08:06:03.832 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-09-10T08:06:03.832 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-10T08:06:03.832 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-09-10T08:06:03.833 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:03 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-09-10T08:06:04.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.094+0000 7f2359600640 1 -- 172.21.15.2:0/3548702672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2354104010 msgr2=0x7f23541043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:04.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.094+0000 7f2359600640 1 --2- 172.21.15.2:0/3548702672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2354104010 0x7f23541043f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2344009900 tx=0x7f234402a410 comp rx=0 tx=0).stop 2024-09-10T08:06:04.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.094+0000 7f2359600640 1 -- 172.21.15.2:0/3548702672 shutdown_connections 2024-09-10T08:06:04.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.094+0000 7f2359600640 1 --2- 172.21.15.2:0/3548702672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2354109140 0x7f2354110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:04.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.094+0000 7f2359600640 1 --2- 172.21.15.2:0/3548702672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23541049c0 0x7f2354108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:04.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.094+0000 7f2359600640 1 --2- 172.21.15.2:0/3548702672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2354104010 0x7f23541043f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:04.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.094+0000 7f2359600640 1 -- 172.21.15.2:0/3548702672 >> 172.21.15.2:0/3548702672 conn(0x7f23540ffa50 msgr2=0x7f2354101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:04.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.094+0000 7f2359600640 1 -- 172.21.15.2:0/3548702672 shutdown_connections 2024-09-10T08:06:04.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.094+0000 7f2359600640 1 -- 172.21.15.2:0/3548702672 wait complete. 2024-09-10T08:06:04.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.095+0000 7f2359600640 1 Processor -- start 2024-09-10T08:06:04.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.095+0000 7f2359600640 1 -- start start 2024-09-10T08:06:04.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.095+0000 7f2359600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2354104010 0x7f235419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:04.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.095+0000 7f2359600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23541049c0 0x7f235419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:04.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f2359600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2354109140 0x7f235419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:04.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f2359600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f23541141b0 con 0x7f2354109140 2024-09-10T08:06:04.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f2359600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2354114030 con 0x7f2354104010 2024-09-10T08:06:04.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f2359600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2354114330 con 0x7f23541049c0 2024-09-10T08:06:04.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f2353400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2354109140 0x7f235419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:04.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f234be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23541049c0 0x7f235419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:04.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f2353400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2354109140 0x7f235419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57796/0 (socket says 172.21.15.2:57796) 2024-09-10T08:06:04.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f2353400640 1 -- 172.21.15.2:0/3878554642 learned_addr learned my addr 172.21.15.2:0/3878554642 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:04.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f2352a00640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2354104010 0x7f235419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:04.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f234be00640 1 -- 172.21.15.2:0/3878554642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2354104010 msgr2=0x7f235419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:04.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f234be00640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2354104010 0x7f235419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:04.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f234be00640 1 -- 172.21.15.2:0/3878554642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2354109140 msgr2=0x7f235419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:04.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f234be00640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2354109140 0x7f235419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:04.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f234be00640 1 -- 172.21.15.2:0/3878554642 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235419f600 con 0x7f23541049c0 2024-09-10T08:06:04.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.096+0000 7f2353400640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2354109140 0x7f235419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:06:04.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.097+0000 7f234be00640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23541049c0 0x7f235419ab90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f233c00e9f0 tx=0x7f233c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:04.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.097+0000 7f234b400640 1 -- 172.21.15.2:0/3878554642 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c00ccc0 con 0x7f23541049c0 2024-09-10T08:06:04.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.097+0000 7f2359600640 1 -- 172.21.15.2:0/3878554642 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235419f8f0 con 0x7f23541049c0 2024-09-10T08:06:04.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.097+0000 7f234b400640 1 -- 172.21.15.2:0/3878554642 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f233c005020 con 0x7f23541049c0 2024-09-10T08:06:04.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.097+0000 7f234b400640 1 -- 172.21.15.2:0/3878554642 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c0049e0 con 0x7f23541049c0 2024-09-10T08:06:04.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.097+0000 7f2359600640 1 -- 172.21.15.2:0/3878554642 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23541a71d0 con 0x7f23541049c0 2024-09-10T08:06:04.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.098+0000 7f2359600640 1 -- 172.21.15.2:0/3878554642 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23541043f0 con 0x7f23541049c0 2024-09-10T08:06:04.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.100+0000 7f234b400640 1 -- 172.21.15.2:0/3878554642 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f233c005560 con 0x7f23541049c0 2024-09-10T08:06:04.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.101+0000 7f234b400640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23380a2b80 0x7f23380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:04.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.102+0000 7f234b400640 1 -- 172.21.15.2:0/3878554642 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f233c0cf3e0 con 0x7f23541049c0 2024-09-10T08:06:04.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.102+0000 7f2352a00640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23380a2b80 0x7f23380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:04.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.102+0000 7f2352a00640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23380a2b80 0x7f23380a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2344000c00 tx=0x7f2344005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:04.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.104+0000 7f234b400640 1 -- 172.21.15.2:0/3878554642 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f233c0051d0 con 0x7f23541049c0 2024-09-10T08:06:04.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-10T08:06:04.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-09-10T08:06:04.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-10T08:06:04.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-10T08:06:04.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-09-10T08:06:04.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-10T08:06:04.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-09-10T08:06:04.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-10T08:06:04.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-09-10T08:06:04.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:03 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-09-10T08:06:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-10T08:06:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-09-10T08:06:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-10T08:06:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-10T08:06:04.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-09-10T08:06:04.168 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-10T08:06:04.168 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-09-10T08:06:04.168 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-10T08:06:04.168 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-09-10T08:06:04.168 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:03 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-09-10T08:06:04.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.306+0000 7f2359600640 1 -- 172.21.15.2:0/3878554642 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f235419b9b0 con 0x7f23541049c0 2024-09-10T08:06:04.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.306+0000 7f234b400640 1 -- 172.21.15.2:0/3878554642 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+578 (secure 0 0 0) 0x7f235419b9b0 con 0x7f23541049c0 2024-09-10T08:06:04.306 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3590511604 2024-09-11T07:58:03.548782+0000 2024-09-10T08:06:04.306 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/939674007 2024-09-11T07:58:03.548782+0000 2024-09-10T08:06:04.306 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2447931330 2024-09-11T07:58:03.548782+0000 2024-09-10T08:06:04.306 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/387296843 2024-09-11T07:58:03.548782+0000 2024-09-10T08:06:04.307 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1896108724 2024-09-11T07:57:36.359026+0000 2024-09-10T08:06:04.307 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2121891210 2024-09-11T07:57:36.359026+0000 2024-09-10T08:06:04.307 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/712393349 2024-09-11T07:57:36.359026+0000 2024-09-10T08:06:04.307 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/712393349 2024-09-11T07:57:36.359026+0000 2024-09-10T08:06:04.307 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3590511604 2024-09-11T07:58:03.548782+0000 2024-09-10T08:06:04.307 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1616837180 2024-09-11T07:57:36.359026+0000 2024-09-10T08:06:04.307 INFO:teuthology.orchestra.run.smithi002.stderr:listed 10 entries 2024-09-10T08:06:04.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.309+0000 7f2359600640 1 -- 172.21.15.2:0/3878554642 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23380a2b80 msgr2=0x7f23380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:04.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.309+0000 7f2359600640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23380a2b80 0x7f23380a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2344000c00 tx=0x7f2344005b50 comp rx=0 tx=0).stop 2024-09-10T08:06:04.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.309+0000 7f2359600640 1 -- 172.21.15.2:0/3878554642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23541049c0 msgr2=0x7f235419ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:04.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.309+0000 7f2359600640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23541049c0 0x7f235419ab90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f233c00e9f0 tx=0x7f233c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:06:04.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.309+0000 7f2359600640 1 -- 172.21.15.2:0/3878554642 shutdown_connections 2024-09-10T08:06:04.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.309+0000 7f2359600640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2354109140 0x7f235419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:04.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.309+0000 7f2359600640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23380a2b80 0x7f23380a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:04.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.309+0000 7f2359600640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23541049c0 0x7f235419ab90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:04.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.309+0000 7f2359600640 1 --2- 172.21.15.2:0/3878554642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2354104010 0x7f235419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:04.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.309+0000 7f2359600640 1 -- 172.21.15.2:0/3878554642 >> 172.21.15.2:0/3878554642 conn(0x7f23540ffa50 msgr2=0x7f2354100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:04.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.309+0000 7f2359600640 1 -- 172.21.15.2:0/3878554642 shutdown_connections 2024-09-10T08:06:04.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:04.309+0000 7f2359600640 1 -- 172.21.15.2:0/3878554642 wait complete. 2024-09-10T08:06:04.975 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:04 smithi002 ceph-mon[30563]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T08:06:04.975 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-10T08:06:04.975 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-10T08:06:04.975 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-10T08:06:04.975 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-10T08:06:04.975 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-10T08:06:04.975 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-10T08:06:04.975 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-09-10T08:06:04.975 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-10T08:06:04.975 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-10T08:06:04.976 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-10T08:06:04.976 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:04 smithi002 ceph-mon[30563]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T08:06:04.976 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:04 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3878554642' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:06:05.013 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2024-09-10T08:06:05.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:06:05.014 DEBUG:teuthology.orchestra.run.smithi002:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:06:05.069 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:ceph-fuse[72070]: fuse finished with error 0 and tester_r 0 2024-09-10T08:06:05.073 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T08:06:05.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:04 smithi155 ceph-mon[35405]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T08:06:05.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-10T08:06:05.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-10T08:06:05.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-10T08:06:05.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-10T08:06:05.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-10T08:06:05.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-10T08:06:05.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-09-10T08:06:05.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-10T08:06:05.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-10T08:06:05.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-10T08:06:05.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:04 smithi155 ceph-mon[35405]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T08:06:05.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:04 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3878554642' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:06:05.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:04 smithi107 ceph-mon[35048]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T08:06:05.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-10T08:06:05.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-10T08:06:05.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-10T08:06:05.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-10T08:06:05.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-10T08:06:05.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-10T08:06:05.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-09-10T08:06:05.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-10T08:06:05.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-10T08:06:05.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-10T08:06:05.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:04 smithi107 ceph-mon[35048]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T08:06:05.168 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:04 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3878554642' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:06:06.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:05 smithi002 ceph-mon[30563]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T08:06:06.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:05 smithi155 ceph-mon[35405]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T08:06:06.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:05 smithi107 ceph-mon[35048]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T08:06:07.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:06 smithi002 ceph-mon[30563]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 462 B/s, 0 objects/s recovering 2024-09-10T08:06:07.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:06 smithi155 ceph-mon[35405]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 462 B/s, 0 objects/s recovering 2024-09-10T08:06:07.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:06 smithi107 ceph-mon[35048]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 462 B/s, 0 objects/s recovering 2024-09-10T08:06:09.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:08 smithi002 ceph-mon[30563]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 465 B/s, 0 objects/s recovering 2024-09-10T08:06:09.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:08 smithi155 ceph-mon[35405]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 465 B/s, 0 objects/s recovering 2024-09-10T08:06:09.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:08 smithi107 ceph-mon[35048]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 465 B/s, 0 objects/s recovering 2024-09-10T08:06:11.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:10 smithi002 ceph-mon[30563]: pgmap v326: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 465 B/s, 0 objects/s recovering 2024-09-10T08:06:11.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:10 smithi155 ceph-mon[35405]: pgmap v326: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 465 B/s, 0 objects/s recovering 2024-09-10T08:06:11.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:10 smithi107 ceph-mon[35048]: pgmap v326: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 465 B/s, 0 objects/s recovering 2024-09-10T08:06:11.174 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2024-09-10T08:06:11.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:06:11.174 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T08:06:11.236 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:06:11.616 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:12.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.218+0000 7f2eb5400640 1 -- 172.21.15.2:0/2468061250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb0102790 msgr2=0x7f2eb01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:12.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.218+0000 7f2eb5400640 1 --2- 172.21.15.2:0/2468061250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb0102790 0x7f2eb01067e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009900 tx=0x7f2e9802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:06:12.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.218+0000 7f2eb5400640 1 -- 172.21.15.2:0/2468061250 shutdown_connections 2024-09-10T08:06:12.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.218+0000 7f2eb5400640 1 --2- 172.21.15.2:0/2468061250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2eb0106f10 0x7f2eb010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:12.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.218+0000 7f2eb5400640 1 --2- 172.21.15.2:0/2468061250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb0102790 0x7f2eb01067e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:12.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.218+0000 7f2eb5400640 1 --2- 172.21.15.2:0/2468061250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb0101de0 0x7f2eb01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:12.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.218+0000 7f2eb5400640 1 -- 172.21.15.2:0/2468061250 >> 172.21.15.2:0/2468061250 conn(0x7f2eb00fd8c0 msgr2=0x7f2eb00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:12.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.218+0000 7f2eb5400640 1 -- 172.21.15.2:0/2468061250 shutdown_connections 2024-09-10T08:06:12.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.219+0000 7f2eb5400640 1 -- 172.21.15.2:0/2468061250 wait complete. 2024-09-10T08:06:12.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.220+0000 7f2eb5400640 1 Processor -- start 2024-09-10T08:06:12.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.220+0000 7f2eb5400640 1 -- start start 2024-09-10T08:06:12.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.220+0000 7f2eb5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2eb0101de0 0x7f2eb0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:12.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.220+0000 7f2eb5400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb0102790 0x7f2eb01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:12.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.220+0000 7f2eb5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb0106f10 0x7f2eb019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:12.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.221+0000 7f2eb5400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb0111fc0 con 0x7f2eb0102790 2024-09-10T08:06:12.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.221+0000 7f2eb5400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb0111e40 con 0x7f2eb0106f10 2024-09-10T08:06:12.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.221+0000 7f2eb5400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb0112140 con 0x7f2eb0101de0 2024-09-10T08:06:12.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.221+0000 7f2eae800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2eb0101de0 0x7f2eb0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:12.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.221+0000 7f2eae800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2eb0101de0 0x7f2eb0198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50172/0 (socket says 172.21.15.2:50172) 2024-09-10T08:06:12.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.221+0000 7f2eae800640 1 -- 172.21.15.2:0/912362599 learned_addr learned my addr 172.21.15.2:0/912362599 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:12.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.221+0000 7f2ea5e00640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb0102790 0x7f2eb01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:12.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.221+0000 7f2eaf200640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb0106f10 0x7f2eb019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:12.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.221+0000 7f2eae800640 1 -- 172.21.15.2:0/912362599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb0106f10 msgr2=0x7f2eb019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:12.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.221+0000 7f2eae800640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb0106f10 0x7f2eb019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:12.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.221+0000 7f2eae800640 1 -- 172.21.15.2:0/912362599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb0102790 msgr2=0x7f2eb01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:12.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.221+0000 7f2eae800640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb0102790 0x7f2eb01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:12.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.221+0000 7f2eae800640 1 -- 172.21.15.2:0/912362599 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb019d410 con 0x7f2eb0101de0 2024-09-10T08:06:12.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.222+0000 7f2eae800640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2eb0101de0 0x7f2eb0198460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000e9f0 tx=0x7f2ea000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:12.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.222+0000 7f2ea7e00640 1 -- 172.21.15.2:0/912362599 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea000cd80 con 0x7f2eb0101de0 2024-09-10T08:06:12.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.222+0000 7f2ea7e00640 1 -- 172.21.15.2:0/912362599 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ea0010910 con 0x7f2eb0101de0 2024-09-10T08:06:12.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.222+0000 7f2eb5400640 1 -- 172.21.15.2:0/912362599 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb019d700 con 0x7f2eb0101de0 2024-09-10T08:06:12.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.222+0000 7f2ea7e00640 1 -- 172.21.15.2:0/912362599 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0018990 con 0x7f2eb0101de0 2024-09-10T08:06:12.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.222+0000 7f2eb5400640 1 -- 172.21.15.2:0/912362599 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb01a4fe0 con 0x7f2eb0101de0 2024-09-10T08:06:12.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.223+0000 7f2eb5400640 1 -- 172.21.15.2:0/912362599 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb01021c0 con 0x7f2eb0101de0 2024-09-10T08:06:12.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.225+0000 7f2ea7e00640 1 -- 172.21.15.2:0/912362599 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2ea0010ac0 con 0x7f2eb0101de0 2024-09-10T08:06:12.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.226+0000 7f2ea7e00640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2e940a2c00 0x7f2e940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:12.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.227+0000 7f2ea7e00640 1 -- 172.21.15.2:0/912362599 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f2ea00cfc20 con 0x7f2eb0101de0 2024-09-10T08:06:12.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.227+0000 7f2ea5e00640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2e940a2c00 0x7f2e940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:12.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.227+0000 7f2ea5e00640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2e940a2c00 0x7f2e940a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000c00 tx=0x7f2e98005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:12.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.228+0000 7f2ea7e00640 1 -- 172.21.15.2:0/912362599 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ea009c170 con 0x7f2eb0101de0 2024-09-10T08:06:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.415+0000 7f2eb5400640 1 -- 172.21.15.2:0/912362599 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2eb010a0f0 con 0x7f2eb0101de0 2024-09-10T08:06:12.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.417+0000 7f2ea7e00640 1 -- 172.21.15.2:0/912362599 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24653 (secure 0 0 0) 0x7f2ea00a0470 con 0x7f2eb0101de0 2024-09-10T08:06:12.417 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:06:12.417 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":93,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:06:04.721464+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:06:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.419+0000 7f2eb5400640 1 -- 172.21.15.2:0/912362599 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2e940a2c00 msgr2=0x7f2e940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.419+0000 7f2eb5400640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2e940a2c00 0x7f2e940a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000c00 tx=0x7f2e98005b50 comp rx=0 tx=0).stop 2024-09-10T08:06:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.419+0000 7f2eb5400640 1 -- 172.21.15.2:0/912362599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2eb0101de0 msgr2=0x7f2eb0198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.419+0000 7f2eb5400640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2eb0101de0 0x7f2eb0198460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000e9f0 tx=0x7f2ea000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:06:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.419+0000 7f2eb5400640 1 -- 172.21.15.2:0/912362599 shutdown_connections 2024-09-10T08:06:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.419+0000 7f2eb5400640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb0106f10 0x7f2eb019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.419+0000 7f2eb5400640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb0102790 0x7f2eb01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.419+0000 7f2eb5400640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2e940a2c00 0x7f2e940a50c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.419+0000 7f2eb5400640 1 --2- 172.21.15.2:0/912362599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2eb0101de0 0x7f2eb0198460 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.419+0000 7f2eb5400640 1 -- 172.21.15.2:0/912362599 >> 172.21.15.2:0/912362599 conn(0x7f2eb00fd8c0 msgr2=0x7f2eb00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.419+0000 7f2eb5400640 1 -- 172.21.15.2:0/912362599 shutdown_connections 2024-09-10T08:06:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:12.420+0000 7f2eb5400640 1 -- 172.21.15.2:0/912362599 wait complete. 2024-09-10T08:06:13.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:12 smithi002 ceph-mon[30563]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 465 B/s, 0 objects/s recovering 2024-09-10T08:06:13.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:12 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/912362599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:13.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:12 smithi155 ceph-mon[35405]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 465 B/s, 0 objects/s recovering 2024-09-10T08:06:13.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:12 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/912362599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:13.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:12 smithi107 ceph-mon[35048]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 465 B/s, 0 objects/s recovering 2024-09-10T08:06:13.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:12 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/912362599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:13.272 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:06:13.654 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:14.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.246+0000 7ff425200640 1 -- 172.21.15.2:0/545405161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff420076720 msgr2=0x7ff420076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:14.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.246+0000 7ff425200640 1 --2- 172.21.15.2:0/545405161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff420076720 0x7ff420076b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff410009860 tx=0x7ff41002a340 comp rx=0 tx=0).stop 2024-09-10T08:06:14.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.246+0000 7ff425200640 1 -- 172.21.15.2:0/545405161 shutdown_connections 2024-09-10T08:06:14.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.246+0000 7ff425200640 1 --2- 172.21.15.2:0/545405161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420112a30 0x7ff420114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:14.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.246+0000 7ff425200640 1 --2- 172.21.15.2:0/545405161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff420077040 0x7ff4201124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.246+0000 7ff425200640 1 --2- 172.21.15.2:0/545405161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff420076720 0x7ff420076b00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.246+0000 7ff425200640 1 -- 172.21.15.2:0/545405161 >> 172.21.15.2:0/545405161 conn(0x7ff4200ffa90 msgr2=0x7ff420101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.247+0000 7ff425200640 1 -- 172.21.15.2:0/545405161 shutdown_connections 2024-09-10T08:06:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.247+0000 7ff425200640 1 -- 172.21.15.2:0/545405161 wait complete. 2024-09-10T08:06:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.247+0000 7ff425200640 1 Processor -- start 2024-09-10T08:06:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.248+0000 7ff425200640 1 -- start start 2024-09-10T08:06:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.248+0000 7ff425200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420076720 0x7ff4201a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.248+0000 7ff425200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff420077040 0x7ff4201a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.248+0000 7ff425200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff420112a30 0x7ff42019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.248+0000 7ff425200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff4201185c0 con 0x7ff420077040 2024-09-10T08:06:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.248+0000 7ff425200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff420118440 con 0x7ff420112a30 2024-09-10T08:06:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.248+0000 7ff425200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff420118740 con 0x7ff420076720 2024-09-10T08:06:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.248+0000 7ff415400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff420077040 0x7ff4201a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.248+0000 7ff415400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff420077040 0x7ff4201a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42786/0 (socket says 172.21.15.2:42786) 2024-09-10T08:06:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.248+0000 7ff415400640 1 -- 172.21.15.2:0/1502607093 learned_addr learned my addr 172.21.15.2:0/1502607093 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.248+0000 7ff41de00640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420076720 0x7ff4201a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.248+0000 7ff41e800640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff420112a30 0x7ff42019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.249+0000 7ff41de00640 1 -- 172.21.15.2:0/1502607093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff420112a30 msgr2=0x7ff42019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.249+0000 7ff41de00640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff420112a30 0x7ff42019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.249+0000 7ff41de00640 1 -- 172.21.15.2:0/1502607093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff420077040 msgr2=0x7ff4201a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.249+0000 7ff41de00640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff420077040 0x7ff4201a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.249+0000 7ff41de00640 1 -- 172.21.15.2:0/1502607093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42019f010 con 0x7ff420076720 2024-09-10T08:06:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.249+0000 7ff415400640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff420077040 0x7ff4201a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:06:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.249+0000 7ff41de00640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420076720 0x7ff4201a4720 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff41002a860 tx=0x7ff410033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.249+0000 7ff417400640 1 -- 172.21.15.2:0/1502607093 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410037040 con 0x7ff420076720 2024-09-10T08:06:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.249+0000 7ff425200640 1 -- 172.21.15.2:0/1502607093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42019f2a0 con 0x7ff420076720 2024-09-10T08:06:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.249+0000 7ff417400640 1 -- 172.21.15.2:0/1502607093 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff410033d50 con 0x7ff420076720 2024-09-10T08:06:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.249+0000 7ff417400640 1 -- 172.21.15.2:0/1502607093 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410003960 con 0x7ff420076720 2024-09-10T08:06:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.249+0000 7ff425200640 1 -- 172.21.15.2:0/1502607093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4201ab5f0 con 0x7ff420076720 2024-09-10T08:06:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.250+0000 7ff425200640 1 -- 172.21.15.2:0/1502607093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff420076b00 con 0x7ff420076720 2024-09-10T08:06:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.252+0000 7ff417400640 1 -- 172.21.15.2:0/1502607093 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff41006bd10 con 0x7ff420076720 2024-09-10T08:06:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.254+0000 7ff417400640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4040a2b80 0x7ff4040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.254+0000 7ff417400640 1 -- 172.21.15.2:0/1502607093 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7ff4100eec60 con 0x7ff420076720 2024-09-10T08:06:14.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.254+0000 7ff415400640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4040a2b80 0x7ff4040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:14.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.255+0000 7ff415400640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4040a2b80 0x7ff4040a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff4200793e0 tx=0x7ff408009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:14.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.256+0000 7ff417400640 1 -- 172.21.15.2:0/1502607093 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff410049020 con 0x7ff420076720 2024-09-10T08:06:14.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.442+0000 7ff425200640 1 -- 172.21.15.2:0/1502607093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff42010dbc0 con 0x7ff420076720 2024-09-10T08:06:14.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.444+0000 7ff417400640 1 -- 172.21.15.2:0/1502607093 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24653 (secure 0 0 0) 0x7ff4100bb800 con 0x7ff420076720 2024-09-10T08:06:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:06:14.444 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":93,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:06:04.721464+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:06:14.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.446+0000 7ff425200640 1 -- 172.21.15.2:0/1502607093 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4040a2b80 msgr2=0x7ff4040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:14.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.446+0000 7ff425200640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4040a2b80 0x7ff4040a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff4200793e0 tx=0x7ff408009260 comp rx=0 tx=0).stop 2024-09-10T08:06:14.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.446+0000 7ff425200640 1 -- 172.21.15.2:0/1502607093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420076720 msgr2=0x7ff4201a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:14.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.446+0000 7ff425200640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420076720 0x7ff4201a4720 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff41002a860 tx=0x7ff410033b60 comp rx=0 tx=0).stop 2024-09-10T08:06:14.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.447+0000 7ff425200640 1 -- 172.21.15.2:0/1502607093 shutdown_connections 2024-09-10T08:06:14.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.447+0000 7ff425200640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff420112a30 0x7ff42019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:14.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.447+0000 7ff425200640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff420077040 0x7ff4201a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:14.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.447+0000 7ff425200640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4040a2b80 0x7ff4040a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:14.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.447+0000 7ff425200640 1 --2- 172.21.15.2:0/1502607093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420076720 0x7ff4201a4720 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:14.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.447+0000 7ff425200640 1 -- 172.21.15.2:0/1502607093 >> 172.21.15.2:0/1502607093 conn(0x7ff4200ffa90 msgr2=0x7ff420101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:14.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.447+0000 7ff425200640 1 -- 172.21.15.2:0/1502607093 shutdown_connections 2024-09-10T08:06:14.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:14.447+0000 7ff425200640 1 -- 172.21.15.2:0/1502607093 wait complete. 2024-09-10T08:06:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:14 smithi002 ceph-mon[30563]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 309 B/s wr, 0 op/s; 375 B/s, 0 objects/s recovering 2024-09-10T08:06:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:14 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1502607093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:15.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:14 smithi155 ceph-mon[35405]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 309 B/s wr, 0 op/s; 375 B/s, 0 objects/s recovering 2024-09-10T08:06:15.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:14 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1502607093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:15.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:14 smithi107 ceph-mon[35048]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 309 B/s wr, 0 op/s; 375 B/s, 0 objects/s recovering 2024-09-10T08:06:15.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:14 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1502607093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:15.220 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T08:06:15.612 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:16.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.178+0000 7f6512e00640 1 -- 172.21.15.2:0/1021591919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f650c1049a0 msgr2=0x7f650c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:16.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.178+0000 7f6512e00640 1 --2- 172.21.15.2:0/1021591919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f650c1049a0 0x7f650c1089f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009900 tx=0x7f64f802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:06:16.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.178+0000 7f6512e00640 1 -- 172.21.15.2:0/1021591919 shutdown_connections 2024-09-10T08:06:16.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.178+0000 7f6512e00640 1 --2- 172.21.15.2:0/1021591919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f650c109120 0x7f650c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:16.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.178+0000 7f6512e00640 1 --2- 172.21.15.2:0/1021591919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f650c1049a0 0x7f650c1089f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:16.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.178+0000 7f6512e00640 1 --2- 172.21.15.2:0/1021591919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f650c103ff0 0x7f650c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:16.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.178+0000 7f6512e00640 1 -- 172.21.15.2:0/1021591919 >> 172.21.15.2:0/1021591919 conn(0x7f650c0ffa90 msgr2=0x7f650c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:16.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.178+0000 7f6512e00640 1 -- 172.21.15.2:0/1021591919 shutdown_connections 2024-09-10T08:06:16.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.178+0000 7f6512e00640 1 -- 172.21.15.2:0/1021591919 wait complete. 2024-09-10T08:06:16.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.179+0000 7f6512e00640 1 Processor -- start 2024-09-10T08:06:16.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.179+0000 7f6512e00640 1 -- start start 2024-09-10T08:06:16.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f6512e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f650c103ff0 0x7f650c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:16.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f6512e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f650c1049a0 0x7f650c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:16.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f6512e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f650c109120 0x7f650c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:16.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f6512e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f650c114120 con 0x7f650c109120 2024-09-10T08:06:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f6512e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f650c113fa0 con 0x7f650c1049a0 2024-09-10T08:06:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f6512e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f650c1142a0 con 0x7f650c103ff0 2024-09-10T08:06:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f6510e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f650c109120 0x7f650c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f6510e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f650c109120 0x7f650c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42804/0 (socket says 172.21.15.2:42804) 2024-09-10T08:06:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f6510e00640 1 -- 172.21.15.2:0/1184947257 learned_addr learned my addr 172.21.15.2:0/1184947257 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f650be00640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f650c103ff0 0x7f650c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f650b400640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f650c1049a0 0x7f650c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f650be00640 1 -- 172.21.15.2:0/1184947257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f650c1049a0 msgr2=0x7f650c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f650be00640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f650c1049a0 0x7f650c19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f650be00640 1 -- 172.21.15.2:0/1184947257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f650c109120 msgr2=0x7f650c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f650be00640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f650c109120 0x7f650c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f650be00640 1 -- 172.21.15.2:0/1184947257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c19f650 con 0x7f650c103ff0 2024-09-10T08:06:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.180+0000 7f6510e00640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f650c109120 0x7f650c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:06:16.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.181+0000 7f650be00640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f650c103ff0 0x7f650c19a6a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00e9f0 tx=0x7f64fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:16.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.181+0000 7f6508c00640 1 -- 172.21.15.2:0/1184947257 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00ccc0 con 0x7f650c103ff0 2024-09-10T08:06:16.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.181+0000 7f6508c00640 1 -- 172.21.15.2:0/1184947257 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64fc010910 con 0x7f650c103ff0 2024-09-10T08:06:16.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.181+0000 7f6508c00640 1 -- 172.21.15.2:0/1184947257 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc018960 con 0x7f650c103ff0 2024-09-10T08:06:16.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.181+0000 7f6512e00640 1 -- 172.21.15.2:0/1184947257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650c19f940 con 0x7f650c103ff0 2024-09-10T08:06:16.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.181+0000 7f6512e00640 1 -- 172.21.15.2:0/1184947257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650c1a7220 con 0x7f650c103ff0 2024-09-10T08:06:16.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.182+0000 7f6512e00640 1 -- 172.21.15.2:0/1184947257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f650c1043d0 con 0x7f650c103ff0 2024-09-10T08:06:16.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.184+0000 7f6508c00640 1 -- 172.21.15.2:0/1184947257 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f64fc010ac0 con 0x7f650c103ff0 2024-09-10T08:06:16.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.186+0000 7f6508c00640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f64f00a2b80 0x7f64f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:16.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.186+0000 7f6508c00640 1 -- 172.21.15.2:0/1184947257 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f64fc0cfb10 con 0x7f650c103ff0 2024-09-10T08:06:16.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.186+0000 7f650b400640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f64f00a2b80 0x7f64f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:16.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.187+0000 7f650b400640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f64f00a2b80 0x7f64f00a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009230 tx=0x7f64f80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:16.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.188+0000 7f6508c00640 1 -- 172.21.15.2:0/1184947257 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64fc09c060 con 0x7f650c103ff0 2024-09-10T08:06:16.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.412+0000 7f6512e00640 1 -- 172.21.15.2:0/1184947257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f650c10c360 con 0x7f650c103ff0 2024-09-10T08:06:16.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.759+0000 7f6508c00640 1 -- 172.21.15.2:0/1184947257 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f64fc0a0360 con 0x7f650c103ff0 2024-09-10T08:06:16.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.762+0000 7f6512e00640 1 -- 172.21.15.2:0/1184947257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f64f00a2b80 msgr2=0x7f64f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:16.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.762+0000 7f6512e00640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f64f00a2b80 0x7f64f00a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009230 tx=0x7f64f80058d0 comp rx=0 tx=0).stop 2024-09-10T08:06:16.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.762+0000 7f6512e00640 1 -- 172.21.15.2:0/1184947257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f650c103ff0 msgr2=0x7f650c19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:16.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.762+0000 7f6512e00640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f650c103ff0 0x7f650c19a6a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00e9f0 tx=0x7f64fc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:06:16.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.762+0000 7f6512e00640 1 -- 172.21.15.2:0/1184947257 shutdown_connections 2024-09-10T08:06:16.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.762+0000 7f6512e00640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f650c109120 0x7f650c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:16.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.762+0000 7f6512e00640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f650c1049a0 0x7f650c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:16.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.762+0000 7f6512e00640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f64f00a2b80 0x7f64f00a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:16.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.762+0000 7f6512e00640 1 --2- 172.21.15.2:0/1184947257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f650c103ff0 0x7f650c19a6a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:16.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.762+0000 7f6512e00640 1 -- 172.21.15.2:0/1184947257 >> 172.21.15.2:0/1184947257 conn(0x7f650c0ffa90 msgr2=0x7f650c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:16.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.762+0000 7f6512e00640 1 -- 172.21.15.2:0/1184947257 shutdown_connections 2024-09-10T08:06:16.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:16.762+0000 7f6512e00640 1 -- 172.21.15.2:0/1184947257 wait complete. 2024-09-10T08:06:17.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:16 smithi002 ceph-mon[30563]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 281 B/s wr, 0 op/s; 340 B/s, 0 objects/s recovering 2024-09-10T08:06:17.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:16 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1184947257' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:06:17.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:16 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:06:17.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:16 smithi155 ceph-mon[35405]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 281 B/s wr, 0 op/s; 340 B/s, 0 objects/s recovering 2024-09-10T08:06:17.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:16 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1184947257' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:06:17.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:16 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:06:17.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:16 smithi107 ceph-mon[35048]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 281 B/s wr, 0 op/s; 340 B/s, 0 objects/s recovering 2024-09-10T08:06:17.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:16 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1184947257' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:06:17.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:16 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:06:17.567 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-10T08:06:17.567 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-10T08:06:17.953 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:18.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:17 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T08:06:18.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:17 smithi002 ceph-mon[30563]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:06:18.057 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:17 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T08:06:18.057 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:17 smithi155 ceph-mon[35405]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:06:18.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:17 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T08:06:18.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:17 smithi107 ceph-mon[35048]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:06:18.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.572+0000 7f611fa00640 1 -- 172.21.15.2:0/288634912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f611810e440 msgr2=0x7f6118110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:18.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.572+0000 7f611fa00640 1 --2- 172.21.15.2:0/288634912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f611810e440 0x7f6118110830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6110009be0 tx=0x7f611002a750 comp rx=0 tx=0).stop 2024-09-10T08:06:18.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.572+0000 7f611fa00640 1 -- 172.21.15.2:0/288634912 shutdown_connections 2024-09-10T08:06:18.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.572+0000 7f611fa00640 1 --2- 172.21.15.2:0/288634912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f611810e440 0x7f6118110830 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:18.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.572+0000 7f611fa00640 1 --2- 172.21.15.2:0/288634912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6118074e30 0x7f611810df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:18.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.572+0000 7f611fa00640 1 --2- 172.21.15.2:0/288634912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6118074510 0x7f61180748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:18.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.572+0000 7f611fa00640 1 -- 172.21.15.2:0/288634912 >> 172.21.15.2:0/288634912 conn(0x7f61180fd8e0 msgr2=0x7f61180ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:18.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.573+0000 7f611fa00640 1 -- 172.21.15.2:0/288634912 shutdown_connections 2024-09-10T08:06:18.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.573+0000 7f611fa00640 1 -- 172.21.15.2:0/288634912 wait complete. 2024-09-10T08:06:18.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.574+0000 7f611fa00640 1 Processor -- start 2024-09-10T08:06:18.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.574+0000 7f611fa00640 1 -- start start 2024-09-10T08:06:18.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.574+0000 7f611fa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6118074510 0x7f61181966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:18.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.574+0000 7f611fa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6118074e30 0x7f6118196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:18.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.574+0000 7f611fa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f611810e440 0x7f611819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:18.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.575+0000 7f611fa00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6118104420 con 0x7f6118074e30 2024-09-10T08:06:18.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.575+0000 7f611fa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61181042a0 con 0x7f611810e440 2024-09-10T08:06:18.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.575+0000 7f611fa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f61181045a0 con 0x7f6118074510 2024-09-10T08:06:18.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.575+0000 7f611d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6118074510 0x7f61181966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:18.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.575+0000 7f6117e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6118074e30 0x7f6118196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:18.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.575+0000 7f611d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6118074510 0x7f61181966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36058/0 (socket says 172.21.15.2:36058) 2024-09-10T08:06:18.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.575+0000 7f611d000640 1 -- 172.21.15.2:0/2410425508 learned_addr learned my addr 172.21.15.2:0/2410425508 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:18.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.575+0000 7f611da00640 1 --2- 172.21.15.2:0/2410425508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f611810e440 0x7f611819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:18.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.575+0000 7f611d000640 1 -- 172.21.15.2:0/2410425508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f611810e440 msgr2=0x7f611819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:18.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.575+0000 7f611d000640 1 --2- 172.21.15.2:0/2410425508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f611810e440 0x7f611819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:18.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.575+0000 7f611d000640 1 -- 172.21.15.2:0/2410425508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6118074e30 msgr2=0x7f6118196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:18.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.575+0000 7f611d000640 1 --2- 172.21.15.2:0/2410425508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6118074e30 0x7f6118196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:18.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.575+0000 7f611d000640 1 -- 172.21.15.2:0/2410425508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611819b190 con 0x7f6118074510 2024-09-10T08:06:18.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.576+0000 7f611d000640 1 --2- 172.21.15.2:0/2410425508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6118074510 0x7f61181966c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f610c009fd0 tx=0x7f610c007590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:18.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.576+0000 7f6115600640 1 -- 172.21.15.2:0/2410425508 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c01d5f0 con 0x7f6118074510 2024-09-10T08:06:18.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.576+0000 7f611fa00640 1 -- 172.21.15.2:0/2410425508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611819b480 con 0x7f6118074510 2024-09-10T08:06:18.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.576+0000 7f6115600640 1 -- 172.21.15.2:0/2410425508 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f610c022070 con 0x7f6118074510 2024-09-10T08:06:18.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.576+0000 7f6115600640 1 -- 172.21.15.2:0/2410425508 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c01cb40 con 0x7f6118074510 2024-09-10T08:06:18.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.576+0000 7f611fa00640 1 -- 172.21.15.2:0/2410425508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61181a2d60 con 0x7f6118074510 2024-09-10T08:06:18.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.577+0000 7f611fa00640 1 -- 172.21.15.2:0/2410425508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61180748f0 con 0x7f6118074510 2024-09-10T08:06:18.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.579+0000 7f6115600640 1 -- 172.21.15.2:0/2410425508 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f610c01cce0 con 0x7f6118074510 2024-09-10T08:06:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.581+0000 7f6115600640 1 --2- 172.21.15.2:0/2410425508 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f61000a2b80 0x7f61000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.581+0000 7f6115600640 1 -- 172.21.15.2:0/2410425508 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f610c0d76c0 con 0x7f6118074510 2024-09-10T08:06:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.581+0000 7f6117e00640 1 --2- 172.21.15.2:0/2410425508 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f61000a2b80 0x7f61000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.582+0000 7f6117e00640 1 --2- 172.21.15.2:0/2410425508 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f61000a2b80 0x7f61000a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f611810f840 tx=0x7f6104009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:18.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.585+0000 7f6115600640 1 -- 172.21.15.2:0/2410425508 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f610c01a070 con 0x7f6118074510 2024-09-10T08:06:18.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:18.813+0000 7f611fa00640 1 -- 172.21.15.2:0/2410425508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f611802d7c0 con 0x7f6118074510 2024-09-10T08:06:19.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:18 smithi002 ceph-mon[30563]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1 B/s, 0 objects/s recovering 2024-09-10T08:06:19.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:18 smithi155 ceph-mon[35405]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1 B/s, 0 objects/s recovering 2024-09-10T08:06:19.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:18 smithi107 ceph-mon[35048]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1 B/s, 0 objects/s recovering 2024-09-10T08:06:19.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:19.774+0000 7f6115600640 1 -- 172.21.15.2:0/2410425508 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f610c0a4260 con 0x7f6118074510 2024-09-10T08:06:19.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:19.777+0000 7f611fa00640 1 -- 172.21.15.2:0/2410425508 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f61000a2b80 msgr2=0x7f61000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:19.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:19.777+0000 7f611fa00640 1 --2- 172.21.15.2:0/2410425508 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f61000a2b80 0x7f61000a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f611810f840 tx=0x7f6104009260 comp rx=0 tx=0).stop 2024-09-10T08:06:19.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:19.777+0000 7f611fa00640 1 -- 172.21.15.2:0/2410425508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6118074510 msgr2=0x7f61181966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:19.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:19.777+0000 7f611fa00640 1 --2- 172.21.15.2:0/2410425508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6118074510 0x7f61181966c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f610c009fd0 tx=0x7f610c007590 comp rx=0 tx=0).stop 2024-09-10T08:06:19.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:19.777+0000 7f611fa00640 1 -- 172.21.15.2:0/2410425508 shutdown_connections 2024-09-10T08:06:19.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:19.777+0000 7f611fa00640 1 --2- 172.21.15.2:0/2410425508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f611810e440 0x7f611819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:19.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:19.777+0000 7f611fa00640 1 --2- 172.21.15.2:0/2410425508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6118074e30 0x7f6118196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:19.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:19.777+0000 7f611fa00640 1 --2- 172.21.15.2:0/2410425508 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f61000a2b80 0x7f61000a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:19.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:19.777+0000 7f611fa00640 1 --2- 172.21.15.2:0/2410425508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6118074510 0x7f61181966c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:19.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:19.777+0000 7f611fa00640 1 -- 172.21.15.2:0/2410425508 >> 172.21.15.2:0/2410425508 conn(0x7f61180fd8e0 msgr2=0x7f61180fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:19.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:19.777+0000 7f611fa00640 1 -- 172.21.15.2:0/2410425508 shutdown_connections 2024-09-10T08:06:19.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:19.778+0000 7f611fa00640 1 -- 172.21.15.2:0/2410425508 wait complete. 2024-09-10T08:06:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:19 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2410425508' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-10T08:06:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:19 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-10T08:06:20.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:19 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2410425508' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-10T08:06:20.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:19 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-10T08:06:20.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:19 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2410425508' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-10T08:06:20.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:19 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-10T08:06:20.606 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T08:06:20.995 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:20 smithi002 ceph-mon[30563]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-10T08:06:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:20 smithi002 ceph-mon[30563]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T08:06:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:20 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-10T08:06:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:20 smithi002 ceph-mon[30563]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T08:06:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:20 smithi002 ceph-mon[30563]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:06:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:20 smithi002 ceph-mon[30563]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-10T08:06:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:20 smithi002 ceph-mon[30563]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-10T08:06:21.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:20 smithi155 ceph-mon[35405]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-10T08:06:21.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:20 smithi155 ceph-mon[35405]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T08:06:21.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:20 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-10T08:06:21.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:20 smithi155 ceph-mon[35405]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T08:06:21.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:20 smithi155 ceph-mon[35405]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:06:21.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:20 smithi155 ceph-mon[35405]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-10T08:06:21.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:20 smithi155 ceph-mon[35405]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-10T08:06:21.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:20 smithi107 ceph-mon[35048]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-10T08:06:21.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:20 smithi107 ceph-mon[35048]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T08:06:21.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:20 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-10T08:06:21.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:20 smithi107 ceph-mon[35048]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T08:06:21.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:20 smithi107 ceph-mon[35048]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:06:21.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:20 smithi107 ceph-mon[35048]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-10T08:06:21.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:20 smithi107 ceph-mon[35048]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-10T08:06:21.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.575+0000 7f0ceb200640 1 -- 172.21.15.2:0/1291076724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ce4104000 msgr2=0x7f0ce41043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:21.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.575+0000 7f0ceb200640 1 --2- 172.21.15.2:0/1291076724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ce4104000 0x7f0ce41043e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009900 tx=0x7f0cd402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:06:21.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.575+0000 7f0ceb200640 1 -- 172.21.15.2:0/1291076724 shutdown_connections 2024-09-10T08:06:21.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.575+0000 7f0ceb200640 1 --2- 172.21.15.2:0/1291076724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ce4109130 0x7f0ce4110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:21.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.575+0000 7f0ceb200640 1 --2- 172.21.15.2:0/1291076724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce41049b0 0x7f0ce4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:21.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.575+0000 7f0ceb200640 1 --2- 172.21.15.2:0/1291076724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ce4104000 0x7f0ce41043e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:21.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.575+0000 7f0ceb200640 1 -- 172.21.15.2:0/1291076724 >> 172.21.15.2:0/1291076724 conn(0x7f0ce40ffa20 msgr2=0x7f0ce4101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:21.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.575+0000 7f0ceb200640 1 -- 172.21.15.2:0/1291076724 shutdown_connections 2024-09-10T08:06:21.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.575+0000 7f0ceb200640 1 -- 172.21.15.2:0/1291076724 wait complete. 2024-09-10T08:06:21.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.576+0000 7f0ceb200640 1 Processor -- start 2024-09-10T08:06:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.576+0000 7f0ceb200640 1 -- start start 2024-09-10T08:06:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0ceb200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce4104000 0x7f0ce419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0ceb200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ce41049b0 0x7f0ce419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0ceb200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ce4109130 0x7f0ce419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0ceb200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4114180 con 0x7f0ce4109130 2024-09-10T08:06:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0ceb200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4114000 con 0x7f0ce4104000 2024-09-10T08:06:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0ceb200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4114300 con 0x7f0ce41049b0 2024-09-10T08:06:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0cdbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ce41049b0 0x7f0ce419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0cdbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ce41049b0 0x7f0ce419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36080/0 (socket says 172.21.15.2:36080) 2024-09-10T08:06:21.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0cdbe00640 1 -- 172.21.15.2:0/1037011394 learned_addr learned my addr 172.21.15.2:0/1037011394 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:21.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0ce3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce4104000 0x7f0ce419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:21.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0ce9200640 1 --2- 172.21.15.2:0/1037011394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ce4109130 0x7f0ce419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:21.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0cdbe00640 1 -- 172.21.15.2:0/1037011394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce4104000 msgr2=0x7f0ce419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:21.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0cdbe00640 1 --2- 172.21.15.2:0/1037011394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce4104000 0x7f0ce419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:21.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0cdbe00640 1 -- 172.21.15.2:0/1037011394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ce4109130 msgr2=0x7f0ce419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:21.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0cdbe00640 1 --2- 172.21.15.2:0/1037011394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ce4109130 0x7f0ce419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:21.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.577+0000 7f0cdbe00640 1 -- 172.21.15.2:0/1037011394 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce419f620 con 0x7f0ce41049b0 2024-09-10T08:06:21.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.578+0000 7f0cdbe00640 1 --2- 172.21.15.2:0/1037011394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ce41049b0 0x7f0ce419abb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00ea60 tx=0x7f0ccc00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:21.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.578+0000 7f0ce1600640 1 -- 172.21.15.2:0/1037011394 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00cde0 con 0x7f0ce41049b0 2024-09-10T08:06:21.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.578+0000 7f0ceb200640 1 -- 172.21.15.2:0/1037011394 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce419f910 con 0x7f0ce41049b0 2024-09-10T08:06:21.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.578+0000 7f0ce1600640 1 -- 172.21.15.2:0/1037011394 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ccc0035f0 con 0x7f0ce41049b0 2024-09-10T08:06:21.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.578+0000 7f0ce1600640 1 -- 172.21.15.2:0/1037011394 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc010670 con 0x7f0ce41049b0 2024-09-10T08:06:21.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.578+0000 7f0ceb200640 1 -- 172.21.15.2:0/1037011394 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce41a71f0 con 0x7f0ce41049b0 2024-09-10T08:06:21.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.579+0000 7f0ceb200640 1 -- 172.21.15.2:0/1037011394 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce41043e0 con 0x7f0ce41049b0 2024-09-10T08:06:21.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.581+0000 7f0ce1600640 1 -- 172.21.15.2:0/1037011394 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0ccc0037a0 con 0x7f0ce41049b0 2024-09-10T08:06:21.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.583+0000 7f0ce1600640 1 --2- 172.21.15.2:0/1037011394 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0cc80a2c00 0x7f0cc80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:21.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.583+0000 7f0ce1600640 1 -- 172.21.15.2:0/1037011394 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f0ccc0cf880 con 0x7f0ce41049b0 2024-09-10T08:06:21.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.583+0000 7f0ce3e00640 1 --2- 172.21.15.2:0/1037011394 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0cc80a2c00 0x7f0cc80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:21.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.584+0000 7f0ce3e00640 1 --2- 172.21.15.2:0/1037011394 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0cc80a2c00 0x7f0cc80a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4000c00 tx=0x7f0cd4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:21.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.585+0000 7f0ce1600640 1 -- 172.21.15.2:0/1037011394 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ccc09c420 con 0x7f0ce41049b0 2024-09-10T08:06:21.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:21.878+0000 7f0ceb200640 1 -- 172.21.15.2:0/1037011394 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0ce410bd80 con 0x7f0ce41049b0 2024-09-10T08:06:22.068 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:21 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] up:active 2024-09-10T08:06:22.068 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:21 smithi107 ceph-mon[35048]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T08:06:22.068 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:21 smithi107 ceph-mon[35048]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T08:06:22.068 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:21 smithi107 ceph-mon[35048]: Cluster is now healthy 2024-09-10T08:06:22.069 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:21 smithi107 ceph-mon[35048]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-10T08:06:22.069 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:21 smithi107 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-10T08:06:22.069 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:21 smithi107 ceph-mon[35048]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-10T08:06:22.069 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:21 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:06:22.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:21 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] up:active 2024-09-10T08:06:22.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:21 smithi155 ceph-mon[35405]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T08:06:22.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:21 smithi155 ceph-mon[35405]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T08:06:22.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:21 smithi155 ceph-mon[35405]: Cluster is now healthy 2024-09-10T08:06:22.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:21 smithi155 ceph-mon[35405]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-10T08:06:22.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:21 smithi155 ceph-mon[35405]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-10T08:06:22.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:21 smithi155 ceph-mon[35405]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-10T08:06:22.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:21 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:06:22.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:21 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] up:active 2024-09-10T08:06:22.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:21 smithi002 ceph-mon[30563]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T08:06:22.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:21 smithi002 ceph-mon[30563]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T08:06:22.169 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:21 smithi002 ceph-mon[30563]: Cluster is now healthy 2024-09-10T08:06:22.169 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:21 smithi002 ceph-mon[30563]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-10T08:06:22.169 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:21 smithi002 ceph-mon[30563]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-10T08:06:22.169 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:21 smithi002 ceph-mon[30563]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-10T08:06:22.169 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:21 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:06:22.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:22.259+0000 7f0ce1600640 1 -- 172.21.15.2:0/1037011394 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f0ccc09bd20 con 0x7f0ce41049b0 2024-09-10T08:06:22.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:22.263+0000 7f0ceb200640 1 -- 172.21.15.2:0/1037011394 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0cc80a2c00 msgr2=0x7f0cc80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:22.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:22.263+0000 7f0ceb200640 1 --2- 172.21.15.2:0/1037011394 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0cc80a2c00 0x7f0cc80a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4000c00 tx=0x7f0cd4005b50 comp rx=0 tx=0).stop 2024-09-10T08:06:22.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:22.263+0000 7f0ceb200640 1 -- 172.21.15.2:0/1037011394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ce41049b0 msgr2=0x7f0ce419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:22.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:22.263+0000 7f0ceb200640 1 --2- 172.21.15.2:0/1037011394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ce41049b0 0x7f0ce419abb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00ea60 tx=0x7f0ccc00ef20 comp rx=0 tx=0).stop 2024-09-10T08:06:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:22.263+0000 7f0ceb200640 1 -- 172.21.15.2:0/1037011394 shutdown_connections 2024-09-10T08:06:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:22.263+0000 7f0ceb200640 1 --2- 172.21.15.2:0/1037011394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ce4109130 0x7f0ce419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:22.263+0000 7f0ceb200640 1 --2- 172.21.15.2:0/1037011394 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0cc80a2c00 0x7f0cc80a50c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:22.263+0000 7f0ceb200640 1 --2- 172.21.15.2:0/1037011394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ce41049b0 0x7f0ce419abb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:22.263+0000 7f0ceb200640 1 --2- 172.21.15.2:0/1037011394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ce4104000 0x7f0ce419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:22.263+0000 7f0ceb200640 1 -- 172.21.15.2:0/1037011394 >> 172.21.15.2:0/1037011394 conn(0x7f0ce40ffa20 msgr2=0x7f0ce4100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:22.264+0000 7f0ceb200640 1 -- 172.21.15.2:0/1037011394 shutdown_connections 2024-09-10T08:06:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:22.264+0000 7f0ceb200640 1 -- 172.21.15.2:0/1037011394 wait complete. 2024-09-10T08:06:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:22 smithi002 ceph-mon[30563]: pgmap v332: 129 pgs: 129 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 2 op/s 2024-09-10T08:06:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:22 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] up:active 2024-09-10T08:06:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:22 smithi002 ceph-mon[30563]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-10T08:06:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:22 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1037011394' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T08:06:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:22 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T08:06:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:22 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T08:06:23.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:22 smithi002 ceph-mon[30563]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:06:23.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:22 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:23.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:22 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:23.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:22 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:23.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:22 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:23.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:22 smithi155 ceph-mon[35405]: pgmap v332: 129 pgs: 129 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 2 op/s 2024-09-10T08:06:23.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:22 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] up:active 2024-09-10T08:06:23.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:22 smithi155 ceph-mon[35405]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-10T08:06:23.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:22 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1037011394' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T08:06:23.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:22 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T08:06:23.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:22 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T08:06:23.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:22 smithi155 ceph-mon[35405]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:06:23.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:22 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:23.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:22 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:23.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:22 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:23.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:22 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:23.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:22 smithi107 ceph-mon[35048]: pgmap v332: 129 pgs: 129 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 2 op/s 2024-09-10T08:06:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:22 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] up:active 2024-09-10T08:06:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:22 smithi107 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-10T08:06:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:22 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1037011394' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T08:06:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:22 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T08:06:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:22 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T08:06:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:22 smithi107 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:06:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:22 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:22 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:22 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:22 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:23.625 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T08:06:24.053 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:24.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.641+0000 7fe9ea400640 1 -- 172.21.15.2:0/3236106773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e41049a0 msgr2=0x7fe9e41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:24.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.641+0000 7fe9ea400640 1 --2- 172.21.15.2:0/3236106773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e41049a0 0x7fe9e41089f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009900 tx=0x7fe9d002a410 comp rx=0 tx=0).stop 2024-09-10T08:06:24.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.641+0000 7fe9ea400640 1 -- 172.21.15.2:0/3236106773 shutdown_connections 2024-09-10T08:06:24.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.641+0000 7fe9ea400640 1 --2- 172.21.15.2:0/3236106773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e4109120 0x7fe9e4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:24.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.641+0000 7fe9ea400640 1 --2- 172.21.15.2:0/3236106773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e41049a0 0x7fe9e41089f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:24.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.641+0000 7fe9ea400640 1 --2- 172.21.15.2:0/3236106773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9e4103ff0 0x7fe9e41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:24.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.641+0000 7fe9ea400640 1 -- 172.21.15.2:0/3236106773 >> 172.21.15.2:0/3236106773 conn(0x7fe9e40ffa90 msgr2=0x7fe9e4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:24.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.641+0000 7fe9ea400640 1 -- 172.21.15.2:0/3236106773 shutdown_connections 2024-09-10T08:06:24.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.641+0000 7fe9ea400640 1 -- 172.21.15.2:0/3236106773 wait complete. 2024-09-10T08:06:24.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.642+0000 7fe9ea400640 1 Processor -- start 2024-09-10T08:06:24.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.642+0000 7fe9ea400640 1 -- start start 2024-09-10T08:06:24.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.643+0000 7fe9ea400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e4103ff0 0x7fe9e419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:24.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.643+0000 7fe9ea400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9e41049a0 0x7fe9e419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:24.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.643+0000 7fe9ea400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4109120 0x7fe9e419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:24.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.643+0000 7fe9ea400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e4114140 con 0x7fe9e4109120 2024-09-10T08:06:24.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.643+0000 7fe9ea400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e4113fc0 con 0x7fe9e4103ff0 2024-09-10T08:06:24.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.643+0000 7fe9ea400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41142c0 con 0x7fe9e41049a0 2024-09-10T08:06:24.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.643+0000 7fe9e3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e4103ff0 0x7fe9e419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:24.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.643+0000 7fe9e3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4109120 0x7fe9e419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:24.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.643+0000 7fe9e3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e4103ff0 0x7fe9e419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50950/0 (socket says 172.21.15.2:50950) 2024-09-10T08:06:24.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.643+0000 7fe9e3400640 1 -- 172.21.15.2:0/1927890589 learned_addr learned my addr 172.21.15.2:0/1927890589 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.643+0000 7fe9e3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4109120 0x7fe9e419eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38658/0 (socket says 172.21.15.2:38658) 2024-09-10T08:06:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.643+0000 7fe9e2a00640 1 --2- 172.21.15.2:0/1927890589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9e41049a0 0x7fe9e419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.644+0000 7fe9e3400640 1 -- 172.21.15.2:0/1927890589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9e41049a0 msgr2=0x7fe9e419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.644+0000 7fe9e3400640 1 --2- 172.21.15.2:0/1927890589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9e41049a0 0x7fe9e419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.644+0000 7fe9e3400640 1 -- 172.21.15.2:0/1927890589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4109120 msgr2=0x7fe9e419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.644+0000 7fe9e3400640 1 --2- 172.21.15.2:0/1927890589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4109120 0x7fe9e419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.644+0000 7fe9e3400640 1 -- 172.21.15.2:0/1927890589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e419f5d0 con 0x7fe9e4103ff0 2024-09-10T08:06:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.644+0000 7fe9e3400640 1 --2- 172.21.15.2:0/1927890589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e4103ff0 0x7fe9e419a620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400e9f0 tx=0x7fe9d400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.644+0000 7fe9dbe00640 1 -- 172.21.15.2:0/1927890589 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400cd80 con 0x7fe9e4103ff0 2024-09-10T08:06:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.644+0000 7fe9ea400640 1 -- 172.21.15.2:0/1927890589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e419f8c0 con 0x7fe9e4103ff0 2024-09-10T08:06:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.644+0000 7fe9dbe00640 1 -- 172.21.15.2:0/1927890589 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d4005500 con 0x7fe9e4103ff0 2024-09-10T08:06:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.644+0000 7fe9dbe00640 1 -- 172.21.15.2:0/1927890589 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4010690 con 0x7fe9e4103ff0 2024-09-10T08:06:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.644+0000 7fe9ea400640 1 -- 172.21.15.2:0/1927890589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e4076630 con 0x7fe9e4103ff0 2024-09-10T08:06:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.646+0000 7fe9ea400640 1 -- 172.21.15.2:0/1927890589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e41054d0 con 0x7fe9e4103ff0 2024-09-10T08:06:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.647+0000 7fe9dbe00640 1 -- 172.21.15.2:0/1927890589 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe9d40046c0 con 0x7fe9e4103ff0 2024-09-10T08:06:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.649+0000 7fe9dbe00640 1 --2- 172.21.15.2:0/1927890589 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe9c80a2b80 0x7fe9c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.649+0000 7fe9dbe00640 1 -- 172.21.15.2:0/1927890589 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fe9d40cf890 con 0x7fe9e4103ff0 2024-09-10T08:06:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.649+0000 7fe9e2a00640 1 --2- 172.21.15.2:0/1927890589 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe9c80a2b80 0x7fe9c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.650+0000 7fe9e2a00640 1 --2- 172.21.15.2:0/1927890589 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe9c80a2b80 0x7fe9c80a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40ffeb0 tx=0x7fe9d0040040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:24.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.653+0000 7fe9dbe00640 1 -- 172.21.15.2:0/1927890589 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9d409c430 con 0x7fe9e4103ff0 2024-09-10T08:06:24.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:24.881+0000 7fe9ea400640 1 -- 172.21.15.2:0/1927890589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fe9e410c360 con 0x7fe9e4103ff0 2024-09-10T08:06:25.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:25.376+0000 7fe9dbe00640 1 -- 172.21.15.2:0/1927890589 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fe9d409bd30 con 0x7fe9e4103ff0 2024-09-10T08:06:25.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:25.378+0000 7fe9ea400640 1 -- 172.21.15.2:0/1927890589 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe9c80a2b80 msgr2=0x7fe9c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:25.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:25.378+0000 7fe9ea400640 1 --2- 172.21.15.2:0/1927890589 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe9c80a2b80 0x7fe9c80a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40ffeb0 tx=0x7fe9d0040040 comp rx=0 tx=0).stop 2024-09-10T08:06:25.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:25.378+0000 7fe9ea400640 1 -- 172.21.15.2:0/1927890589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e4103ff0 msgr2=0x7fe9e419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:25.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:25.378+0000 7fe9ea400640 1 --2- 172.21.15.2:0/1927890589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e4103ff0 0x7fe9e419a620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400e9f0 tx=0x7fe9d400edf0 comp rx=0 tx=0).stop 2024-09-10T08:06:25.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:25.378+0000 7fe9ea400640 1 -- 172.21.15.2:0/1927890589 shutdown_connections 2024-09-10T08:06:25.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:25.378+0000 7fe9ea400640 1 --2- 172.21.15.2:0/1927890589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9e4109120 0x7fe9e419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:25.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:25.378+0000 7fe9ea400640 1 --2- 172.21.15.2:0/1927890589 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe9c80a2b80 0x7fe9c80a5040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:25.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:25.378+0000 7fe9ea400640 1 --2- 172.21.15.2:0/1927890589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9e41049a0 0x7fe9e419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:25.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:25.378+0000 7fe9ea400640 1 --2- 172.21.15.2:0/1927890589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e4103ff0 0x7fe9e419a620 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:25.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:25.378+0000 7fe9ea400640 1 -- 172.21.15.2:0/1927890589 >> 172.21.15.2:0/1927890589 conn(0x7fe9e40ffa90 msgr2=0x7fe9e4100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:25.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:25.378+0000 7fe9ea400640 1 -- 172.21.15.2:0/1927890589 shutdown_connections 2024-09-10T08:06:25.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:25.379+0000 7fe9ea400640 1 -- 172.21.15.2:0/1927890589 wait complete. 2024-09-10T08:06:25.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:25 smithi155 ceph-mon[35405]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T08:06:25.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:25.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:25.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:25 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1927890589' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T08:06:25.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:25 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T08:06:25.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:06:25.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:06:25.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:25.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:25 smithi107 ceph-mon[35048]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T08:06:25.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:25.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:25.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:25 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1927890589' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T08:06:25.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:25 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T08:06:25.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:06:25.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:06:25.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:25 smithi002 ceph-mon[30563]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T08:06:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:25 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1927890589' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T08:06:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:25 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T08:06:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:06:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:06:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:06:26.187 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-10T08:06:26.187 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T08:06:26.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:26 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T08:06:26.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:26 smithi002 ceph-mon[30563]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:06:26.567 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:26.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:26 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T08:06:26.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:26 smithi155 ceph-mon[35405]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:06:26.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:26 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T08:06:26.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:26 smithi107 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:06:27.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.193+0000 7febb8600640 1 -- 172.21.15.2:0/3470110427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febb0101df0 msgr2=0x7febb01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:27.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.193+0000 7febb8600640 1 --2- 172.21.15.2:0/3470110427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febb0101df0 0x7febb01021d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feba4009900 tx=0x7feba402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:06:27.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.194+0000 7febb8600640 1 -- 172.21.15.2:0/3470110427 shutdown_connections 2024-09-10T08:06:27.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.194+0000 7febb8600640 1 --2- 172.21.15.2:0/3470110427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febb0106f20 0x7febb010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:27.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.194+0000 7febb8600640 1 --2- 172.21.15.2:0/3470110427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febb01027a0 0x7febb01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:27.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.194+0000 7febb8600640 1 --2- 172.21.15.2:0/3470110427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febb0101df0 0x7febb01021d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:27.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.194+0000 7febb8600640 1 -- 172.21.15.2:0/3470110427 >> 172.21.15.2:0/3470110427 conn(0x7febb00fd8b0 msgr2=0x7febb00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:27.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.194+0000 7febb8600640 1 -- 172.21.15.2:0/3470110427 shutdown_connections 2024-09-10T08:06:27.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.194+0000 7febb8600640 1 -- 172.21.15.2:0/3470110427 wait complete. 2024-09-10T08:06:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.195+0000 7febb8600640 1 Processor -- start 2024-09-10T08:06:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.195+0000 7febb8600640 1 -- start start 2024-09-10T08:06:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb8600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febb0101df0 0x7febb01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb8600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febb01027a0 0x7febb0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb8600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febb0106f20 0x7febb019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb8600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7febb0111f80 con 0x7febb0106f20 2024-09-10T08:06:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb8600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7febb0111e00 con 0x7febb01027a0 2024-09-10T08:06:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb8600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7febb0112100 con 0x7febb0101df0 2024-09-10T08:06:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb5c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febb0101df0 0x7febb01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb6600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febb0106f20 0x7febb019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb5200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febb01027a0 0x7febb0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb5c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febb0101df0 0x7febb01984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:32910/0 (socket says 172.21.15.2:32910) 2024-09-10T08:06:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb5c00640 1 -- 172.21.15.2:0/2005794376 learned_addr learned my addr 172.21.15.2:0/2005794376 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb5c00640 1 -- 172.21.15.2:0/2005794376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febb01027a0 msgr2=0x7febb0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb5c00640 1 --2- 172.21.15.2:0/2005794376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febb01027a0 0x7febb0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb5c00640 1 -- 172.21.15.2:0/2005794376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febb0106f20 msgr2=0x7febb019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb5c00640 1 --2- 172.21.15.2:0/2005794376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febb0106f20 0x7febb019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb5c00640 1 -- 172.21.15.2:0/2005794376 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb019d490 con 0x7febb0101df0 2024-09-10T08:06:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.196+0000 7febb6600640 1 --2- 172.21.15.2:0/2005794376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febb0106f20 0x7febb019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:06:27.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.197+0000 7febb5c00640 1 --2- 172.21.15.2:0/2005794376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febb0101df0 0x7febb01984e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feba4006010 tx=0x7feba40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:27.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.197+0000 7febae000640 1 -- 172.21.15.2:0/2005794376 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba403f030 con 0x7febb0101df0 2024-09-10T08:06:27.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.197+0000 7febb8600640 1 -- 172.21.15.2:0/2005794376 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb019d720 con 0x7febb0101df0 2024-09-10T08:06:27.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.197+0000 7febae000640 1 -- 172.21.15.2:0/2005794376 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba4003ce0 con 0x7febb0101df0 2024-09-10T08:06:27.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.197+0000 7febae000640 1 -- 172.21.15.2:0/2005794376 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4035070 con 0x7febb0101df0 2024-09-10T08:06:27.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.197+0000 7febb8600640 1 -- 172.21.15.2:0/2005794376 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb01a5060 con 0x7febb0101df0 2024-09-10T08:06:27.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.198+0000 7febb8600640 1 -- 172.21.15.2:0/2005794376 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb01021d0 con 0x7febb0101df0 2024-09-10T08:06:27.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.200+0000 7febae000640 1 -- 172.21.15.2:0/2005794376 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7feba4042030 con 0x7febb0101df0 2024-09-10T08:06:27.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.202+0000 7febae000640 1 --2- 172.21.15.2:0/2005794376 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7feb980a2b80 0x7feb980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:27.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.202+0000 7febae000640 1 -- 172.21.15.2:0/2005794376 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7feba40efcb0 con 0x7febb0101df0 2024-09-10T08:06:27.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.202+0000 7febb5200640 1 --2- 172.21.15.2:0/2005794376 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7feb980a2b80 0x7feb980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:27.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.203+0000 7febb5200640 1 --2- 172.21.15.2:0/2005794376 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7feb980a2b80 0x7feb980a5040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7febb00fde10 tx=0x7feb9c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:27.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.204+0000 7febae000640 1 -- 172.21.15.2:0/2005794376 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feba40bc2c0 con 0x7febb0101df0 2024-09-10T08:06:27.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.415+0000 7febb8600640 1 -- 172.21.15.2:0/2005794376 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7febb0062630 con 0x7feb980a2b80 2024-09-10T08:06:27.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.443+0000 7febae000640 1 -- 172.21.15.2:0/2005794376 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7febb0062630 con 0x7feb980a2b80 2024-09-10T08:06:27.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.445+0000 7febb8600640 1 -- 172.21.15.2:0/2005794376 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7feb980a2b80 msgr2=0x7feb980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:27.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.445+0000 7febb8600640 1 --2- 172.21.15.2:0/2005794376 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7feb980a2b80 0x7feb980a5040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7febb00fde10 tx=0x7feb9c00b390 comp rx=0 tx=0).stop 2024-09-10T08:06:27.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.445+0000 7febb8600640 1 -- 172.21.15.2:0/2005794376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febb0101df0 msgr2=0x7febb01984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:27.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.445+0000 7febb8600640 1 --2- 172.21.15.2:0/2005794376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febb0101df0 0x7febb01984e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feba4006010 tx=0x7feba40042f0 comp rx=0 tx=0).stop 2024-09-10T08:06:27.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.445+0000 7febb8600640 1 -- 172.21.15.2:0/2005794376 shutdown_connections 2024-09-10T08:06:27.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.445+0000 7febb8600640 1 --2- 172.21.15.2:0/2005794376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febb0106f20 0x7febb019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:27.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.445+0000 7febb8600640 1 --2- 172.21.15.2:0/2005794376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febb01027a0 0x7febb0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:27.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.445+0000 7febb8600640 1 --2- 172.21.15.2:0/2005794376 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7feb980a2b80 0x7feb980a5040 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:27.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.445+0000 7febb8600640 1 --2- 172.21.15.2:0/2005794376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febb0101df0 0x7febb01984e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:27.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.446+0000 7febb8600640 1 -- 172.21.15.2:0/2005794376 >> 172.21.15.2:0/2005794376 conn(0x7febb00fd8b0 msgr2=0x7febb00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:27.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.446+0000 7febb8600640 1 -- 172.21.15.2:0/2005794376 shutdown_connections 2024-09-10T08:06:27.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:27.446+0000 7febb8600640 1 -- 172.21.15.2:0/2005794376 wait complete. 2024-09-10T08:06:27.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:27 smithi155 ceph-mon[35405]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T08:06:27.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:27 smithi107 ceph-mon[35048]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T08:06:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:27 smithi002 ceph-mon[30563]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T08:06:27.770 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:06:27 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T08:06:27.422+0000 7f6348400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:06:27.770 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:06:27 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T08:06:27.422+0000 7f6348400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:06:27.770 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:06:27 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T08:06:27.422+0000 7f6348400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:06:27.770 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:06:27 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T08:06:27.422+0000 7f6348400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:06:27.771 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:06:27 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T08:06:27.422+0000 7f6348400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:06:28.292 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T08:06:28.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:28 smithi002 ceph-mon[30563]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:28.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:28 smithi155 ceph-mon[35405]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:28.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:28 smithi107 ceph-mon[35048]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:28.677 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:29.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.208+0000 7f210d200640 1 -- 172.21.15.2:0/310730083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21081049a0 msgr2=0x7f21081089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:29.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.208+0000 7f210d200640 1 --2- 172.21.15.2:0/310730083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21081049a0 0x7f21081089f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009900 tx=0x7f20f402a410 comp rx=0 tx=0).stop 2024-09-10T08:06:29.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.208+0000 7f210d200640 1 -- 172.21.15.2:0/310730083 shutdown_connections 2024-09-10T08:06:29.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.208+0000 7f210d200640 1 --2- 172.21.15.2:0/310730083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2108109120 0x7f2108110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:29.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.208+0000 7f210d200640 1 --2- 172.21.15.2:0/310730083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21081049a0 0x7f21081089f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:29.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.208+0000 7f210d200640 1 --2- 172.21.15.2:0/310730083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2108103ff0 0x7f21081043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:29.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.208+0000 7f210d200640 1 -- 172.21.15.2:0/310730083 >> 172.21.15.2:0/310730083 conn(0x7f21080ffa90 msgr2=0x7f2108101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:29.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.209+0000 7f210d200640 1 -- 172.21.15.2:0/310730083 shutdown_connections 2024-09-10T08:06:29.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.209+0000 7f210d200640 1 -- 172.21.15.2:0/310730083 wait complete. 2024-09-10T08:06:29.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.210+0000 7f210d200640 1 Processor -- start 2024-09-10T08:06:29.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.210+0000 7f210d200640 1 -- start start 2024-09-10T08:06:29.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.211+0000 7f210d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2108103ff0 0x7f210819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:29.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.211+0000 7f210d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21081049a0 0x7f210819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:29.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.211+0000 7f210d200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2108109120 0x7f210819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:29.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.211+0000 7f210d200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2108114120 con 0x7f2108109120 2024-09-10T08:06:29.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.211+0000 7f210d200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2108113fa0 con 0x7f2108103ff0 2024-09-10T08:06:29.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.211+0000 7f210d200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f21081142a0 con 0x7f21081049a0 2024-09-10T08:06:29.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.211+0000 7f2105e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2108103ff0 0x7f210819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:29.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.211+0000 7f2105e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2108103ff0 0x7f210819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60436/0 (socket says 172.21.15.2:60436) 2024-09-10T08:06:29.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.211+0000 7f2105e00640 1 -- 172.21.15.2:0/4181735823 learned_addr learned my addr 172.21.15.2:0/4181735823 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:29.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.211+0000 7f2106800640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2108109120 0x7f210819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:29.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.211+0000 7f2105400640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21081049a0 0x7f210819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:29.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.211+0000 7f2105e00640 1 -- 172.21.15.2:0/4181735823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21081049a0 msgr2=0x7f210819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:29.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.211+0000 7f2105e00640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21081049a0 0x7f210819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:29.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.212+0000 7f2105e00640 1 -- 172.21.15.2:0/4181735823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2108109120 msgr2=0x7f210819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:29.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.212+0000 7f2105e00640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2108109120 0x7f210819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:29.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.212+0000 7f2105e00640 1 -- 172.21.15.2:0/4181735823 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210819f650 con 0x7f2108103ff0 2024-09-10T08:06:29.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.212+0000 7f2106800640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2108109120 0x7f210819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:06:29.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.212+0000 7f2105e00640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2108103ff0 0x7f210819a6a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f20f800e9f0 tx=0x7f20f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:29.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.212+0000 7f20fea00640 1 -- 172.21.15.2:0/4181735823 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f800cd80 con 0x7f2108103ff0 2024-09-10T08:06:29.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.212+0000 7f210d200640 1 -- 172.21.15.2:0/4181735823 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210819f940 con 0x7f2108103ff0 2024-09-10T08:06:29.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.212+0000 7f20fea00640 1 -- 172.21.15.2:0/4181735823 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20f8010910 con 0x7f2108103ff0 2024-09-10T08:06:29.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.212+0000 7f20fea00640 1 -- 172.21.15.2:0/4181735823 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f80188f0 con 0x7f2108103ff0 2024-09-10T08:06:29.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.212+0000 7f210d200640 1 -- 172.21.15.2:0/4181735823 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081a7220 con 0x7f2108103ff0 2024-09-10T08:06:29.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.214+0000 7f210d200640 1 -- 172.21.15.2:0/4181735823 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21081043d0 con 0x7f2108103ff0 2024-09-10T08:06:29.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.216+0000 7f20fea00640 1 -- 172.21.15.2:0/4181735823 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f20f8010ac0 con 0x7f2108103ff0 2024-09-10T08:06:29.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.217+0000 7f20fea00640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20ec0a2c10 0x7f20ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:29.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.217+0000 7f20fea00640 1 -- 172.21.15.2:0/4181735823 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f20f80cf920 con 0x7f2108103ff0 2024-09-10T08:06:29.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.217+0000 7f2105400640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20ec0a2c10 0x7f20ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:29.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.218+0000 7f2105400640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20ec0a2c10 0x7f20ec0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009230 tx=0x7f20f40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:29.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.221+0000 7f20fea00640 1 -- 172.21.15.2:0/4181735823 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20f809bd10 con 0x7f2108103ff0 2024-09-10T08:06:29.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.439+0000 7f210d200640 1 -- 172.21.15.2:0/4181735823 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f210810c390 con 0x7f20ec0a2c10 2024-09-10T08:06:29.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.464+0000 7f20fea00640 1 -- 172.21.15.2:0/4181735823 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f210810c390 con 0x7f20ec0a2c10 2024-09-10T08:06:29.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.467+0000 7f210d200640 1 -- 172.21.15.2:0/4181735823 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20ec0a2c10 msgr2=0x7f20ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:29.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.467+0000 7f210d200640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20ec0a2c10 0x7f20ec0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009230 tx=0x7f20f40058d0 comp rx=0 tx=0).stop 2024-09-10T08:06:29.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.467+0000 7f210d200640 1 -- 172.21.15.2:0/4181735823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2108103ff0 msgr2=0x7f210819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:29.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.467+0000 7f210d200640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2108103ff0 0x7f210819a6a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f20f800e9f0 tx=0x7f20f800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:06:29.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.467+0000 7f210d200640 1 -- 172.21.15.2:0/4181735823 shutdown_connections 2024-09-10T08:06:29.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.467+0000 7f210d200640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2108109120 0x7f210819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:29.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.467+0000 7f210d200640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20ec0a2c10 0x7f20ec0a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:29.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.467+0000 7f210d200640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21081049a0 0x7f210819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:29.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.467+0000 7f210d200640 1 --2- 172.21.15.2:0/4181735823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2108103ff0 0x7f210819a6a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:29.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.467+0000 7f210d200640 1 -- 172.21.15.2:0/4181735823 >> 172.21.15.2:0/4181735823 conn(0x7f21080ffa90 msgr2=0x7f2108100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:29.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.467+0000 7f210d200640 1 -- 172.21.15.2:0/4181735823 shutdown_connections 2024-09-10T08:06:29.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:29.467+0000 7f210d200640 1 -- 172.21.15.2:0/4181735823 wait complete. 2024-09-10T08:06:29.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:29 smithi155 ceph-mon[35405]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-10T08:06:29.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:29 smithi155 ceph-mon[35405]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T08:06:29.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:29 smithi107 ceph-mon[35048]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-10T08:06:29.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:29 smithi107 ceph-mon[35048]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T08:06:29.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:29 smithi002 ceph-mon[30563]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-10T08:06:29.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:29 smithi002 ceph-mon[30563]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T08:06:30.366 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T08:06:31.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:30 smithi107 ceph-mon[35048]: from='client.24716 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:31.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:31 smithi107 ceph-mon[35048]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-10T08:06:31.932 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:30 smithi155 ceph-mon[35405]: from='client.24716 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:31.933 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:31 smithi155 ceph-mon[35405]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-10T08:06:31.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:30 smithi002 ceph-mon[30563]: from='client.24716 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:31.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:31 smithi002 ceph-mon[30563]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-10T08:06:32.325 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:32.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.927+0000 7fc999200640 1 -- 172.21.15.2:0/2018081439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9941079c0 msgr2=0x7fc994107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:32.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.927+0000 7fc999200640 1 --2- 172.21.15.2:0/2018081439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9941079c0 0x7fc994107da0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc984009cd0 tx=0x7fc98402a8e0 comp rx=0 tx=0).stop 2024-09-10T08:06:32.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.927+0000 7fc999200640 1 -- 172.21.15.2:0/2018081439 shutdown_connections 2024-09-10T08:06:32.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.927+0000 7fc999200640 1 --2- 172.21.15.2:0/2018081439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc994102360 0x7fc99410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:32.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.927+0000 7fc999200640 1 --2- 172.21.15.2:0/2018081439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9941019c0 0x7fc994101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:32.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.927+0000 7fc999200640 1 --2- 172.21.15.2:0/2018081439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9941079c0 0x7fc994107da0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:32.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.927+0000 7fc999200640 1 -- 172.21.15.2:0/2018081439 >> 172.21.15.2:0/2018081439 conn(0x7fc9940fd740 msgr2=0x7fc9940ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:32.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.927+0000 7fc999200640 1 -- 172.21.15.2:0/2018081439 shutdown_connections 2024-09-10T08:06:32.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.927+0000 7fc999200640 1 -- 172.21.15.2:0/2018081439 wait complete. 2024-09-10T08:06:32.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.928+0000 7fc999200640 1 Processor -- start 2024-09-10T08:06:32.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.928+0000 7fc999200640 1 -- start start 2024-09-10T08:06:32.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.929+0000 7fc999200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9941019c0 0x7fc99419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:32.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.929+0000 7fc999200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc994102360 0x7fc99419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:32.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.929+0000 7fc999200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9941079c0 0x7fc99419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:32.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.929+0000 7fc999200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc994111e10 con 0x7fc9941079c0 2024-09-10T08:06:32.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.929+0000 7fc999200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc994111c90 con 0x7fc9941019c0 2024-09-10T08:06:32.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.929+0000 7fc999200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc994111f90 con 0x7fc994102360 2024-09-10T08:06:32.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.929+0000 7fc993400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9941019c0 0x7fc99419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:32.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.929+0000 7fc993400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9941019c0 0x7fc99419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60464/0 (socket says 172.21.15.2:60464) 2024-09-10T08:06:32.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.929+0000 7fc993400640 1 -- 172.21.15.2:0/1330908902 learned_addr learned my addr 172.21.15.2:0/1330908902 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:32.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.929+0000 7fc993e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9941079c0 0x7fc99419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:32.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.929+0000 7fc98aa00640 1 --2- 172.21.15.2:0/1330908902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc994102360 0x7fc99419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:32.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.930+0000 7fc993400640 1 -- 172.21.15.2:0/1330908902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc994102360 msgr2=0x7fc99419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:32.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.930+0000 7fc993400640 1 --2- 172.21.15.2:0/1330908902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc994102360 0x7fc99419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:32.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.930+0000 7fc993400640 1 -- 172.21.15.2:0/1330908902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9941079c0 msgr2=0x7fc99419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:32.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.930+0000 7fc993400640 1 --2- 172.21.15.2:0/1330908902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9941079c0 0x7fc99419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:32.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.930+0000 7fc993400640 1 -- 172.21.15.2:0/1330908902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99419f4a0 con 0x7fc9941019c0 2024-09-10T08:06:32.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.930+0000 7fc993400640 1 --2- 172.21.15.2:0/1330908902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9941019c0 0x7fc99419a4f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc984033040 tx=0x7fc9840093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:32.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.930+0000 7fc990c00640 1 -- 172.21.15.2:0/1330908902 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984004040 con 0x7fc9941019c0 2024-09-10T08:06:32.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.930+0000 7fc990c00640 1 -- 172.21.15.2:0/1330908902 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9840046c0 con 0x7fc9941019c0 2024-09-10T08:06:32.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.930+0000 7fc999200640 1 -- 172.21.15.2:0/1330908902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99419f730 con 0x7fc9941019c0 2024-09-10T08:06:32.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.930+0000 7fc990c00640 1 -- 172.21.15.2:0/1330908902 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984030ec0 con 0x7fc9941019c0 2024-09-10T08:06:32.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.930+0000 7fc999200640 1 -- 172.21.15.2:0/1330908902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9941a7070 con 0x7fc9941019c0 2024-09-10T08:06:32.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.931+0000 7fc999200640 1 -- 172.21.15.2:0/1330908902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc994101e20 con 0x7fc9941019c0 2024-09-10T08:06:32.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.933+0000 7fc990c00640 1 -- 172.21.15.2:0/1330908902 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc9840305a0 con 0x7fc9941019c0 2024-09-10T08:06:32.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.934+0000 7fc990c00640 1 --2- 172.21.15.2:0/1330908902 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc9780a2e30 0x7fc9780a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:32.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.935+0000 7fc990c00640 1 -- 172.21.15.2:0/1330908902 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fc98402e030 con 0x7fc9941019c0 2024-09-10T08:06:32.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.935+0000 7fc98aa00640 1 --2- 172.21.15.2:0/1330908902 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc9780a2e30 0x7fc9780a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:32.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.935+0000 7fc98aa00640 1 --2- 172.21.15.2:0/1330908902 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc9780a2e30 0x7fc9780a52f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc9940764a0 tx=0x7fc97c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:32.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:32.936+0000 7fc990c00640 1 -- 172.21.15.2:0/1330908902 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc98404b060 con 0x7fc9941019c0 2024-09-10T08:06:33.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.149+0000 7fc999200640 1 -- 172.21.15.2:0/1330908902 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fc9940fec80 con 0x7fc9780a2e30 2024-09-10T08:06:33.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.156+0000 7fc990c00640 1 -- 172.21.15.2:0/1330908902 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc9940fec80 con 0x7fc9780a2e30 2024-09-10T08:06:33.155 INFO:teuthology.orchestra.run.smithi002.stdout:/volumes/qa/sv_0/764ff92e-f4f6-4110-b026-52f01fbdd136 2024-09-10T08:06:33.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.158+0000 7fc999200640 1 -- 172.21.15.2:0/1330908902 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc9780a2e30 msgr2=0x7fc9780a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:33.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.158+0000 7fc999200640 1 --2- 172.21.15.2:0/1330908902 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc9780a2e30 0x7fc9780a52f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc9940764a0 tx=0x7fc97c005f90 comp rx=0 tx=0).stop 2024-09-10T08:06:33.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.158+0000 7fc999200640 1 -- 172.21.15.2:0/1330908902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9941019c0 msgr2=0x7fc99419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:33.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.158+0000 7fc999200640 1 --2- 172.21.15.2:0/1330908902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9941019c0 0x7fc99419a4f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc984033040 tx=0x7fc9840093d0 comp rx=0 tx=0).stop 2024-09-10T08:06:33.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.158+0000 7fc999200640 1 -- 172.21.15.2:0/1330908902 shutdown_connections 2024-09-10T08:06:33.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.158+0000 7fc999200640 1 --2- 172.21.15.2:0/1330908902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9941079c0 0x7fc99419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:33.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.158+0000 7fc999200640 1 --2- 172.21.15.2:0/1330908902 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc9780a2e30 0x7fc9780a52f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:33.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.158+0000 7fc999200640 1 --2- 172.21.15.2:0/1330908902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc994102360 0x7fc99419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:33.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.158+0000 7fc999200640 1 --2- 172.21.15.2:0/1330908902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9941019c0 0x7fc99419a4f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:33.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.158+0000 7fc999200640 1 -- 172.21.15.2:0/1330908902 >> 172.21.15.2:0/1330908902 conn(0x7fc9940fd740 msgr2=0x7fc99410d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:33.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.159+0000 7fc999200640 1 -- 172.21.15.2:0/1330908902 shutdown_connections 2024-09-10T08:06:33.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:33.159+0000 7fc999200640 1 -- 172.21.15.2:0/1330908902 wait complete. 2024-09-10T08:06:33.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:33 smithi155 ceph-mon[35405]: pgmap v337: 129 pgs: 129 active+clean; 604 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-10T08:06:33.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:33 smithi107 ceph-mon[35048]: pgmap v337: 129 pgs: 129 active+clean; 604 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-10T08:06:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:33 smithi002 ceph-mon[30563]: pgmap v337: 129 pgs: 129 active+clean; 604 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-10T08:06:33.963 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T08:06:34.343 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:34.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:34 smithi155 ceph-mon[35405]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:34.668 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:34 smithi107 ceph-mon[35048]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:34.680 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:34 smithi002 ceph-mon[30563]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:34.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.976+0000 7fc0c6400640 1 -- 172.21.15.2:0/1947647258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0c0104800 msgr2=0x7fc0c0108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:34.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.976+0000 7fc0c6400640 1 --2- 172.21.15.2:0/1947647258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0c0104800 0x7fc0c0108960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009900 tx=0x7fc0ac02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:06:34.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.976+0000 7fc0c6400640 1 -- 172.21.15.2:0/1947647258 shutdown_connections 2024-09-10T08:06:34.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.976+0000 7fc0c6400640 1 --2- 172.21.15.2:0/1947647258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c0109090 0x7fc0c0110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:34.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.976+0000 7fc0c6400640 1 --2- 172.21.15.2:0/1947647258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0c0104800 0x7fc0c0108960 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:34.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.976+0000 7fc0c6400640 1 --2- 172.21.15.2:0/1947647258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0c0103e50 0x7fc0c0104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:34.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.976+0000 7fc0c6400640 1 -- 172.21.15.2:0/1947647258 >> 172.21.15.2:0/1947647258 conn(0x7fc0c00ff8b0 msgr2=0x7fc0c0101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:34.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.976+0000 7fc0c6400640 1 -- 172.21.15.2:0/1947647258 shutdown_connections 2024-09-10T08:06:34.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.976+0000 7fc0c6400640 1 -- 172.21.15.2:0/1947647258 wait complete. 2024-09-10T08:06:34.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.977+0000 7fc0c6400640 1 Processor -- start 2024-09-10T08:06:34.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.977+0000 7fc0c6400640 1 -- start start 2024-09-10T08:06:34.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.977+0000 7fc0c6400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0c0103e50 0x7fc0c019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:34.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.977+0000 7fc0c6400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c0104800 0x7fc0c019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:34.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.977+0000 7fc0c6400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0c0109090 0x7fc0c019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:34.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.977+0000 7fc0c6400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0114100 con 0x7fc0c0103e50 2024-09-10T08:06:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.977+0000 7fc0c6400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0113f80 con 0x7fc0c0104800 2024-09-10T08:06:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.977+0000 7fc0c6400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0114280 con 0x7fc0c0109090 2024-09-10T08:06:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.978+0000 7fc0c5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0c0103e50 0x7fc0c019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.978+0000 7fc0bfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c0104800 0x7fc0c019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.978+0000 7fc0bfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c0104800 0x7fc0c019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60480/0 (socket says 172.21.15.2:60480) 2024-09-10T08:06:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.978+0000 7fc0c5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0c0103e50 0x7fc0c019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36236/0 (socket says 172.21.15.2:36236) 2024-09-10T08:06:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.978+0000 7fc0bfe00640 1 -- 172.21.15.2:0/1222526319 learned_addr learned my addr 172.21.15.2:0/1222526319 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.978+0000 7fc0c5a00640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0c0109090 0x7fc0c019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.978+0000 7fc0bfe00640 1 -- 172.21.15.2:0/1222526319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0c0109090 msgr2=0x7fc0c019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.978+0000 7fc0bfe00640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0c0109090 0x7fc0c019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.978+0000 7fc0bfe00640 1 -- 172.21.15.2:0/1222526319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0c0103e50 msgr2=0x7fc0c019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.978+0000 7fc0bfe00640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0c0103e50 0x7fc0c019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.978+0000 7fc0bfe00640 1 -- 172.21.15.2:0/1222526319 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c019f550 con 0x7fc0c0104800 2024-09-10T08:06:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.978+0000 7fc0c5000640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0c0103e50 0x7fc0c019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:06:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.979+0000 7fc0bfe00640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c0104800 0x7fc0c019aae0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac004770 tx=0x7fc0ac003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.979+0000 7fc0bd600640 1 -- 172.21.15.2:0/1222526319 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac03f070 con 0x7fc0c0104800 2024-09-10T08:06:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.979+0000 7fc0c6400640 1 -- 172.21.15.2:0/1222526319 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c019f7e0 con 0x7fc0c0104800 2024-09-10T08:06:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.979+0000 7fc0bd600640 1 -- 172.21.15.2:0/1222526319 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0ac031ca0 con 0x7fc0c0104800 2024-09-10T08:06:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.979+0000 7fc0bd600640 1 -- 172.21.15.2:0/1222526319 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac035030 con 0x7fc0c0104800 2024-09-10T08:06:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.979+0000 7fc0c6400640 1 -- 172.21.15.2:0/1222526319 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c01a7120 con 0x7fc0c0104800 2024-09-10T08:06:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.980+0000 7fc0c6400640 1 -- 172.21.15.2:0/1222526319 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c0105330 con 0x7fc0c0104800 2024-09-10T08:06:34.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.982+0000 7fc0bd600640 1 -- 172.21.15.2:0/1222526319 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc0ac042070 con 0x7fc0c0104800 2024-09-10T08:06:34.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.983+0000 7fc0bd600640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc0a80a2c10 0x7fc0a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:34.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.984+0000 7fc0bd600640 1 -- 172.21.15.2:0/1222526319 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fc0ac0efc30 con 0x7fc0c0104800 2024-09-10T08:06:34.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.984+0000 7fc0c5000640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc0a80a2c10 0x7fc0a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:34.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.984+0000 7fc0c5000640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc0a80a2c10 0x7fc0a80a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4005fd0 tx=0x7fc0b4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:34.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:34.988+0000 7fc0bd600640 1 -- 172.21.15.2:0/1222526319 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc0ac0bc020 con 0x7fc0c0104800 2024-09-10T08:06:35.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.201+0000 7fc0c6400640 1 -- 172.21.15.2:0/1222526319 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc0c0062630 con 0x7fc0a80a2c10 2024-09-10T08:06:35.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.228+0000 7fc0bd600640 1 -- 172.21.15.2:0/1222526319 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc0c0062630 con 0x7fc0a80a2c10 2024-09-10T08:06:35.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.231+0000 7fc0c6400640 1 -- 172.21.15.2:0/1222526319 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc0a80a2c10 msgr2=0x7fc0a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:35.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.231+0000 7fc0c6400640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc0a80a2c10 0x7fc0a80a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4005fd0 tx=0x7fc0b4009500 comp rx=0 tx=0).stop 2024-09-10T08:06:35.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.231+0000 7fc0c6400640 1 -- 172.21.15.2:0/1222526319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c0104800 msgr2=0x7fc0c019aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:35.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.231+0000 7fc0c6400640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c0104800 0x7fc0c019aae0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac004770 tx=0x7fc0ac003c10 comp rx=0 tx=0).stop 2024-09-10T08:06:35.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.231+0000 7fc0c6400640 1 -- 172.21.15.2:0/1222526319 shutdown_connections 2024-09-10T08:06:35.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.231+0000 7fc0c6400640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc0a80a2c10 0x7fc0a80a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:35.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.231+0000 7fc0c6400640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0c0109090 0x7fc0c019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:35.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.231+0000 7fc0c6400640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0c0104800 0x7fc0c019aae0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:35.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.231+0000 7fc0c6400640 1 --2- 172.21.15.2:0/1222526319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0c0103e50 0x7fc0c019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:35.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.231+0000 7fc0c6400640 1 -- 172.21.15.2:0/1222526319 >> 172.21.15.2:0/1222526319 conn(0x7fc0c00ff8b0 msgr2=0x7fc0c0100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:35.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.231+0000 7fc0c6400640 1 -- 172.21.15.2:0/1222526319 shutdown_connections 2024-09-10T08:06:35.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:35.231+0000 7fc0c6400640 1 -- 172.21.15.2:0/1222526319 wait complete. 2024-09-10T08:06:35.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:35 smithi002 ceph-mon[30563]: pgmap v338: 129 pgs: 129 active+clean; 605 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-10T08:06:35.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:35 smithi107 ceph-mon[35048]: pgmap v338: 129 pgs: 129 active+clean; 605 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-10T08:06:35.896 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T08:06:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:35 smithi155 ceph-mon[35405]: pgmap v338: 129 pgs: 129 active+clean; 605 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-10T08:06:36.273 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:36.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:36 smithi107 ceph-mon[35048]: from='client.24728 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:36.675 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:36 smithi002 ceph-mon[30563]: from='client.24728 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:36.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:36 smithi155 ceph-mon[35405]: from='client.24728 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:36.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.958+0000 7f76ba600640 1 -- 172.21.15.2:0/717559029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76b4104010 msgr2=0x7f76b41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:36.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.958+0000 7f76ba600640 1 --2- 172.21.15.2:0/717559029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76b4104010 0x7f76b41043f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009900 tx=0x7f76a402a410 comp rx=0 tx=0).stop 2024-09-10T08:06:36.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.958+0000 7f76ba600640 1 -- 172.21.15.2:0/717559029 shutdown_connections 2024-09-10T08:06:36.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.958+0000 7f76ba600640 1 --2- 172.21.15.2:0/717559029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b4109140 0x7f76b4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:36.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.958+0000 7f76ba600640 1 --2- 172.21.15.2:0/717559029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76b41049c0 0x7f76b4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:36.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.958+0000 7f76ba600640 1 --2- 172.21.15.2:0/717559029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76b4104010 0x7f76b41043f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:36.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.958+0000 7f76ba600640 1 -- 172.21.15.2:0/717559029 >> 172.21.15.2:0/717559029 conn(0x7f76b40ffa50 msgr2=0x7f76b4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:36.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.958+0000 7f76ba600640 1 -- 172.21.15.2:0/717559029 shutdown_connections 2024-09-10T08:06:36.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.958+0000 7f76ba600640 1 -- 172.21.15.2:0/717559029 wait complete. 2024-09-10T08:06:36.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.959+0000 7f76ba600640 1 Processor -- start 2024-09-10T08:06:36.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.959+0000 7f76ba600640 1 -- start start 2024-09-10T08:06:36.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.959+0000 7f76ba600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76b4104010 0x7f76b419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:36.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.959+0000 7f76ba600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76b41049c0 0x7f76b419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:36.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.959+0000 7f76ba600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b4109140 0x7f76b419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:36.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.959+0000 7f76ba600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f76b4114140 con 0x7f76b4104010 2024-09-10T08:06:36.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.959+0000 7f76ba600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f76b4113fc0 con 0x7f76b41049c0 2024-09-10T08:06:36.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.959+0000 7f76ba600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76b41142c0 con 0x7f76b4109140 2024-09-10T08:06:36.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.959+0000 7f76b3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76b4104010 0x7f76b419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:36.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.960+0000 7f76abe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76b41049c0 0x7f76b419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:36.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.960+0000 7f76b3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76b4104010 0x7f76b419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60536/0 (socket says 172.21.15.2:60536) 2024-09-10T08:06:36.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.960+0000 7f76b3400640 1 -- 172.21.15.2:0/2077519402 learned_addr learned my addr 172.21.15.2:0/2077519402 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:36.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.960+0000 7f76b3e00640 1 --2- 172.21.15.2:0/2077519402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b4109140 0x7f76b419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:36.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.960+0000 7f76abe00640 1 -- 172.21.15.2:0/2077519402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b4109140 msgr2=0x7f76b419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:36.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.960+0000 7f76abe00640 1 --2- 172.21.15.2:0/2077519402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b4109140 0x7f76b419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:36.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.960+0000 7f76abe00640 1 -- 172.21.15.2:0/2077519402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76b4104010 msgr2=0x7f76b419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:36.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.960+0000 7f76abe00640 1 --2- 172.21.15.2:0/2077519402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76b4104010 0x7f76b419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:36.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.960+0000 7f76abe00640 1 -- 172.21.15.2:0/2077519402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b419f5d0 con 0x7f76b41049c0 2024-09-10T08:06:36.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.960+0000 7f76abe00640 1 --2- 172.21.15.2:0/2077519402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76b41049c0 0x7f76b419ab60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f769c00e9f0 tx=0x7f769c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:36.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.961+0000 7f76b0c00640 1 -- 172.21.15.2:0/2077519402 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c00cd80 con 0x7f76b41049c0 2024-09-10T08:06:36.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.961+0000 7f76b0c00640 1 -- 172.21.15.2:0/2077519402 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f769c0035f0 con 0x7f76b41049c0 2024-09-10T08:06:36.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.961+0000 7f76ba600640 1 -- 172.21.15.2:0/2077519402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b419f8c0 con 0x7f76b41049c0 2024-09-10T08:06:36.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.961+0000 7f76b0c00640 1 -- 172.21.15.2:0/2077519402 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c010690 con 0x7f76b41049c0 2024-09-10T08:06:36.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.961+0000 7f76ba600640 1 -- 172.21.15.2:0/2077519402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b41a71a0 con 0x7f76b41049c0 2024-09-10T08:06:36.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.962+0000 7f76ba600640 1 -- 172.21.15.2:0/2077519402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b41043f0 con 0x7f76b41049c0 2024-09-10T08:06:36.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.964+0000 7f76b0c00640 1 -- 172.21.15.2:0/2077519402 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f769c003b30 con 0x7f76b41049c0 2024-09-10T08:06:36.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.965+0000 7f76b0c00640 1 --2- 172.21.15.2:0/2077519402 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76980a2c10 0x7f76980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:36.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.965+0000 7f76b0c00640 1 -- 172.21.15.2:0/2077519402 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f769c0cf800 con 0x7f76b41049c0 2024-09-10T08:06:36.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.965+0000 7f76b3400640 1 --2- 172.21.15.2:0/2077519402 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76980a2c10 0x7f76980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:36.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.966+0000 7f76b3400640 1 --2- 172.21.15.2:0/2077519402 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76980a2c10 0x7f76980a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f76a40096c0 tx=0x7f76a4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:36.967+0000 7f76b0c00640 1 -- 172.21.15.2:0/2077519402 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f769c09c240 con 0x7f76b41049c0 2024-09-10T08:06:37.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.179+0000 7f76ba600640 1 -- 172.21.15.2:0/2077519402 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f76b4062630 con 0x7f76980a2c10 2024-09-10T08:06:37.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.184+0000 7f76b0c00640 1 -- 172.21.15.2:0/2077519402 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f76b4062630 con 0x7f76980a2c10 2024-09-10T08:06:37.182 INFO:teuthology.orchestra.run.smithi002.stdout:/volumes/qa/sv_1/d78be939-d6c3-4b42-854d-10cdcae584fb 2024-09-10T08:06:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.186+0000 7f76ba600640 1 -- 172.21.15.2:0/2077519402 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76980a2c10 msgr2=0x7f76980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.186+0000 7f76ba600640 1 --2- 172.21.15.2:0/2077519402 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76980a2c10 0x7f76980a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f76a40096c0 tx=0x7f76a4005a20 comp rx=0 tx=0).stop 2024-09-10T08:06:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.186+0000 7f76ba600640 1 -- 172.21.15.2:0/2077519402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76b41049c0 msgr2=0x7f76b419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.186+0000 7f76ba600640 1 --2- 172.21.15.2:0/2077519402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76b41049c0 0x7f76b419ab60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f769c00e9f0 tx=0x7f769c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:06:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.186+0000 7f76ba600640 1 -- 172.21.15.2:0/2077519402 shutdown_connections 2024-09-10T08:06:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.186+0000 7f76ba600640 1 --2- 172.21.15.2:0/2077519402 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76980a2c10 0x7f76980a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.186+0000 7f76ba600640 1 --2- 172.21.15.2:0/2077519402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b4109140 0x7f76b419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.186+0000 7f76ba600640 1 --2- 172.21.15.2:0/2077519402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76b41049c0 0x7f76b419ab60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.186+0000 7f76ba600640 1 --2- 172.21.15.2:0/2077519402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76b4104010 0x7f76b419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.186+0000 7f76ba600640 1 -- 172.21.15.2:0/2077519402 >> 172.21.15.2:0/2077519402 conn(0x7f76b40ffa50 msgr2=0x7f76b4100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.186+0000 7f76ba600640 1 -- 172.21.15.2:0/2077519402 shutdown_connections 2024-09-10T08:06:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:37.186+0000 7f76ba600640 1 -- 172.21.15.2:0/2077519402 wait complete. 2024-09-10T08:06:37.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:37 smithi002 ceph-mon[30563]: pgmap v339: 129 pgs: 129 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-09-10T08:06:37.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:37 smithi107 ceph-mon[35048]: pgmap v339: 129 pgs: 129 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-09-10T08:06:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:37 smithi155 ceph-mon[35405]: pgmap v339: 129 pgs: 129 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-09-10T08:06:38.143 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:06:38.522 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:38.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:38 smithi002 ceph-mon[30563]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:38.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:38 smithi155 ceph-mon[35405]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:38.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:38 smithi107 ceph-mon[35048]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:06:39.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.106+0000 7fcd3e600640 1 -- 172.21.15.2:0/4078728893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd38074eb0 msgr2=0x7fcd38112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:39.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.106+0000 7fcd3e600640 1 --2- 172.21.15.2:0/4078728893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd38074eb0 0x7fcd38112650 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009900 tx=0x7fcd2002a480 comp rx=0 tx=0).stop 2024-09-10T08:06:39.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.107+0000 7fcd3e600640 1 -- 172.21.15.2:0/4078728893 shutdown_connections 2024-09-10T08:06:39.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.107+0000 7fcd3e600640 1 --2- 172.21.15.2:0/4078728893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd38112b90 0x7fcd38114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:39.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.107+0000 7fcd3e600640 1 --2- 172.21.15.2:0/4078728893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd38074eb0 0x7fcd38112650 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:39.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.107+0000 7fcd3e600640 1 --2- 172.21.15.2:0/4078728893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd38074590 0x7fcd38074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:39.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.107+0000 7fcd3e600640 1 -- 172.21.15.2:0/4078728893 >> 172.21.15.2:0/4078728893 conn(0x7fcd380fd8c0 msgr2=0x7fcd380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:39.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.107+0000 7fcd3e600640 1 -- 172.21.15.2:0/4078728893 shutdown_connections 2024-09-10T08:06:39.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.107+0000 7fcd3e600640 1 -- 172.21.15.2:0/4078728893 wait complete. 2024-09-10T08:06:39.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.108+0000 7fcd3e600640 1 Processor -- start 2024-09-10T08:06:39.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.108+0000 7fcd3e600640 1 -- start start 2024-09-10T08:06:39.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.108+0000 7fcd3e600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd38074590 0x7fcd3819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:39.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.108+0000 7fcd3e600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd38074eb0 0x7fcd3819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:39.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.108+0000 7fcd3e600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd38112b90 0x7fcd381a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:39.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.108+0000 7fcd3e600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381185b0 con 0x7fcd38074590 2024-09-10T08:06:39.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.108+0000 7fcd3e600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcd38118430 con 0x7fcd38112b90 2024-09-10T08:06:39.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.108+0000 7fcd3e600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcd38118730 con 0x7fcd38074eb0 2024-09-10T08:06:39.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.109+0000 7fcd37400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd38074590 0x7fcd3819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:39.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.109+0000 7fcd37400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd38074590 0x7fcd3819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60544/0 (socket says 172.21.15.2:60544) 2024-09-10T08:06:39.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.109+0000 7fcd37400640 1 -- 172.21.15.2:0/2688157861 learned_addr learned my addr 172.21.15.2:0/2688157861 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:39.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.109+0000 7fcd2ea00640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd38074eb0 0x7fcd3819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:39.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.109+0000 7fcd37e00640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd38112b90 0x7fcd381a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:39.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.109+0000 7fcd37400640 1 -- 172.21.15.2:0/2688157861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd38074eb0 msgr2=0x7fcd3819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:39.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.109+0000 7fcd37400640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd38074eb0 0x7fcd3819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:39.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.109+0000 7fcd37400640 1 -- 172.21.15.2:0/2688157861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd38112b90 msgr2=0x7fcd381a34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:39.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.109+0000 7fcd37400640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd38112b90 0x7fcd381a34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:39.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.109+0000 7fcd37400640 1 -- 172.21.15.2:0/2688157861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd381a3c00 con 0x7fcd38074590 2024-09-10T08:06:39.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.109+0000 7fcd37400640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd38074590 0x7fcd3819ebd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800e9f0 tx=0x7fcd2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:39.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.110+0000 7fcd34c00640 1 -- 172.21.15.2:0/2688157861 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2800cd80 con 0x7fcd38074590 2024-09-10T08:06:39.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.110+0000 7fcd34c00640 1 -- 172.21.15.2:0/2688157861 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd28010910 con 0x7fcd38074590 2024-09-10T08:06:39.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.110+0000 7fcd34c00640 1 -- 172.21.15.2:0/2688157861 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28018990 con 0x7fcd38074590 2024-09-10T08:06:39.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.110+0000 7fcd3e600640 1 -- 172.21.15.2:0/2688157861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd381a3ef0 con 0x7fcd38074590 2024-09-10T08:06:39.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.110+0000 7fcd3e600640 1 -- 172.21.15.2:0/2688157861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd381ab780 con 0x7fcd38074590 2024-09-10T08:06:39.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.111+0000 7fcd3e600640 1 -- 172.21.15.2:0/2688157861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd38074970 con 0x7fcd38074590 2024-09-10T08:06:39.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.113+0000 7fcd34c00640 1 -- 172.21.15.2:0/2688157861 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fcd28010ac0 con 0x7fcd38074590 2024-09-10T08:06:39.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.114+0000 7fcd34c00640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcd1c0a2c90 0x7fcd1c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:39.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.114+0000 7fcd34c00640 1 -- 172.21.15.2:0/2688157861 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fcd280cfc40 con 0x7fcd38074590 2024-09-10T08:06:39.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.115+0000 7fcd2ea00640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcd1c0a2c90 0x7fcd1c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:39.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.115+0000 7fcd2ea00640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcd1c0a2c90 0x7fcd1c0a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcd20000c00 tx=0x7fcd20005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:39.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.117+0000 7fcd34c00640 1 -- 172.21.15.2:0/2688157861 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd2809c030 con 0x7fcd38074590 2024-09-10T08:06:39.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.346+0000 7fcd3e600640 1 -- 172.21.15.2:0/2688157861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd3819ff30 con 0x7fcd38074590 2024-09-10T08:06:39.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.347+0000 7fcd34c00640 1 -- 172.21.15.2:0/2688157861 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7fcd280a0330 con 0x7fcd38074590 2024-09-10T08:06:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:06:39.347 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":25,"btime":"2024-09-10T08:06:25:369375+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:06:24.883542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24494,24499,24536]},"id":1}]} 2024-09-10T08:06:39.347 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 25 2024-09-10T08:06:39.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.350+0000 7fcd3e600640 1 -- 172.21.15.2:0/2688157861 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcd1c0a2c90 msgr2=0x7fcd1c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:39.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.350+0000 7fcd3e600640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcd1c0a2c90 0x7fcd1c0a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcd20000c00 tx=0x7fcd20005b50 comp rx=0 tx=0).stop 2024-09-10T08:06:39.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.350+0000 7fcd3e600640 1 -- 172.21.15.2:0/2688157861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd38074590 msgr2=0x7fcd3819ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:39.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.350+0000 7fcd3e600640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd38074590 0x7fcd3819ebd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800e9f0 tx=0x7fcd2800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:06:39.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.350+0000 7fcd3e600640 1 -- 172.21.15.2:0/2688157861 shutdown_connections 2024-09-10T08:06:39.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.350+0000 7fcd3e600640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd38112b90 0x7fcd381a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:39.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.350+0000 7fcd3e600640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcd1c0a2c90 0x7fcd1c0a5150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:39.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.350+0000 7fcd3e600640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd38074eb0 0x7fcd3819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:39.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.350+0000 7fcd3e600640 1 --2- 172.21.15.2:0/2688157861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd38074590 0x7fcd3819ebd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:39.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.350+0000 7fcd3e600640 1 -- 172.21.15.2:0/2688157861 >> 172.21.15.2:0/2688157861 conn(0x7fcd380fd8c0 msgr2=0x7fcd380ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:39.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.351+0000 7fcd3e600640 1 -- 172.21.15.2:0/2688157861 shutdown_connections 2024-09-10T08:06:39.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:39.351+0000 7fcd3e600640 1 -- 172.21.15.2:0/2688157861 wait complete. 2024-09-10T08:06:39.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:39 smithi002 ceph-mon[30563]: pgmap v340: 129 pgs: 129 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-10T08:06:39.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:39 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2688157861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:06:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:39 smithi155 ceph-mon[35405]: pgmap v340: 129 pgs: 129 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-10T08:06:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:39 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2688157861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:06:39.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:39 smithi107 ceph-mon[35048]: pgmap v340: 129 pgs: 129 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-10T08:06:39.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:39 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2688157861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:06:40.177 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:06:40.552 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:41.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.165+0000 7f7562a00640 1 -- 172.21.15.2:0/1357997899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c109120 msgr2=0x7f755c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:41.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.165+0000 7f7562a00640 1 --2- 172.21.15.2:0/1357997899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c109120 0x7f755c110ca0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7554009ca0 tx=0x7f755402a8e0 comp rx=0 tx=0).stop 2024-09-10T08:06:41.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.165+0000 7f7562a00640 1 -- 172.21.15.2:0/1357997899 shutdown_connections 2024-09-10T08:06:41.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.165+0000 7f7562a00640 1 --2- 172.21.15.2:0/1357997899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c109120 0x7f755c110ca0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:41.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.165+0000 7f7562a00640 1 --2- 172.21.15.2:0/1357997899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755c1049a0 0x7f755c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:41.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.165+0000 7f7562a00640 1 --2- 172.21.15.2:0/1357997899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f755c103ff0 0x7f755c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:41.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.165+0000 7f7562a00640 1 -- 172.21.15.2:0/1357997899 >> 172.21.15.2:0/1357997899 conn(0x7f755c0ffa90 msgr2=0x7f755c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:41.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.165+0000 7f7562a00640 1 -- 172.21.15.2:0/1357997899 shutdown_connections 2024-09-10T08:06:41.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.166+0000 7f7562a00640 1 -- 172.21.15.2:0/1357997899 wait complete. 2024-09-10T08:06:41.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.166+0000 7f7562a00640 1 Processor -- start 2024-09-10T08:06:41.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.167+0000 7f7562a00640 1 -- start start 2024-09-10T08:06:41.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.167+0000 7f7562a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c103ff0 0x7f755c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:41.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.167+0000 7f7562a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755c1049a0 0x7f755c19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:41.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.167+0000 7f7562a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f755c109120 0x7f755c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:41.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.167+0000 7f7562a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f755c114210 con 0x7f755c103ff0 2024-09-10T08:06:41.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.167+0000 7f7562a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f755c114090 con 0x7f755c1049a0 2024-09-10T08:06:41.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.167+0000 7f7562a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f755c114390 con 0x7f755c109120 2024-09-10T08:06:41.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.167+0000 7f755be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c103ff0 0x7f755c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:41.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.167+0000 7f755b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755c1049a0 0x7f755c19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:41.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.167+0000 7f755be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c103ff0 0x7f755c19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60558/0 (socket says 172.21.15.2:60558) 2024-09-10T08:06:41.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.168+0000 7f755be00640 1 -- 172.21.15.2:0/2861284323 learned_addr learned my addr 172.21.15.2:0/2861284323 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:41.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.168+0000 7f7560a00640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f755c109120 0x7f755c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:41.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.168+0000 7f755b400640 1 -- 172.21.15.2:0/2861284323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f755c109120 msgr2=0x7f755c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:41.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.168+0000 7f755b400640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f755c109120 0x7f755c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:41.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.168+0000 7f755b400640 1 -- 172.21.15.2:0/2861284323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c103ff0 msgr2=0x7f755c19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:41.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.168+0000 7f755b400640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c103ff0 0x7f755c19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:41.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.168+0000 7f755b400640 1 -- 172.21.15.2:0/2861284323 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755c19f680 con 0x7f755c1049a0 2024-09-10T08:06:41.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.168+0000 7f755be00640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c103ff0 0x7f755c19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:06:41.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.168+0000 7f755b400640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755c1049a0 0x7f755c19ac10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f754800e9f0 tx=0x7f754800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:41.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.169+0000 7f7558c00640 1 -- 172.21.15.2:0/2861284323 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754800ccc0 con 0x7f755c1049a0 2024-09-10T08:06:41.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.169+0000 7f7558c00640 1 -- 172.21.15.2:0/2861284323 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7548010910 con 0x7f755c1049a0 2024-09-10T08:06:41.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.169+0000 7f7558c00640 1 -- 172.21.15.2:0/2861284323 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548018960 con 0x7f755c1049a0 2024-09-10T08:06:41.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.169+0000 7f7562a00640 1 -- 172.21.15.2:0/2861284323 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755c19f970 con 0x7f755c1049a0 2024-09-10T08:06:41.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.169+0000 7f7562a00640 1 -- 172.21.15.2:0/2861284323 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f755c1a7250 con 0x7f755c1049a0 2024-09-10T08:06:41.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.170+0000 7f7562a00640 1 -- 172.21.15.2:0/2861284323 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f755c1043d0 con 0x7f755c1049a0 2024-09-10T08:06:41.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.172+0000 7f7558c00640 1 -- 172.21.15.2:0/2861284323 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7548010ac0 con 0x7f755c1049a0 2024-09-10T08:06:41.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.173+0000 7f7558c00640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75400a2c90 0x7f75400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:41.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.173+0000 7f7558c00640 1 -- 172.21.15.2:0/2861284323 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f75480cfc20 con 0x7f755c1049a0 2024-09-10T08:06:41.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.173+0000 7f755be00640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75400a2c90 0x7f75400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:41.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.174+0000 7f755be00640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75400a2c90 0x7f75400a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f754c009e00 tx=0x7f754c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:41.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.175+0000 7f7558c00640 1 -- 172.21.15.2:0/2861284323 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f754809c010 con 0x7f755c1049a0 2024-09-10T08:06:41.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.360+0000 7f7562a00640 1 -- 172.21.15.2:0/2861284323 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f755c1a7420 con 0x7f755c1049a0 2024-09-10T08:06:41.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.362+0000 7f7558c00640 1 -- 172.21.15.2:0/2861284323 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24653 (secure 0 0 0) 0x7f75480a4e20 con 0x7f755c1049a0 2024-09-10T08:06:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:06:41.363 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":93,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:06:04.721464+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:06:41.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.365+0000 7f7562a00640 1 -- 172.21.15.2:0/2861284323 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75400a2c90 msgr2=0x7f75400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:41.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.365+0000 7f7562a00640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75400a2c90 0x7f75400a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f754c009e00 tx=0x7f754c009190 comp rx=0 tx=0).stop 2024-09-10T08:06:41.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.365+0000 7f7562a00640 1 -- 172.21.15.2:0/2861284323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755c1049a0 msgr2=0x7f755c19ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:41.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.365+0000 7f7562a00640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755c1049a0 0x7f755c19ac10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f754800e9f0 tx=0x7f754800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:06:41.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.365+0000 7f7562a00640 1 -- 172.21.15.2:0/2861284323 shutdown_connections 2024-09-10T08:06:41.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.365+0000 7f7562a00640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75400a2c90 0x7f75400a5150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:41.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.365+0000 7f7562a00640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f755c109120 0x7f755c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:41.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.365+0000 7f7562a00640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755c1049a0 0x7f755c19ac10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:41.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.365+0000 7f7562a00640 1 --2- 172.21.15.2:0/2861284323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c103ff0 0x7f755c19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:41.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.365+0000 7f7562a00640 1 -- 172.21.15.2:0/2861284323 >> 172.21.15.2:0/2861284323 conn(0x7f755c0ffa90 msgr2=0x7f755c100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:41.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.365+0000 7f7562a00640 1 -- 172.21.15.2:0/2861284323 shutdown_connections 2024-09-10T08:06:41.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:41.365+0000 7f7562a00640 1 -- 172.21.15.2:0/2861284323 wait complete. 2024-09-10T08:06:41.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:41 smithi002 ceph-mon[30563]: pgmap v341: 129 pgs: 129 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-10T08:06:41.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:41 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2861284323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:41 smithi155 ceph-mon[35405]: pgmap v341: 129 pgs: 129 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-10T08:06:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:41 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2861284323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:41 smithi107 ceph-mon[35048]: pgmap v341: 129 pgs: 129 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-10T08:06:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:41 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2861284323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:42.136 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T08:06:42.136 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:06:42.514 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:43.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.224+0000 7f0040c00640 1 -- 172.21.15.2:0/342397342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f003c106f10 msgr2=0x7f003c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:43.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.224+0000 7f0040c00640 1 --2- 172.21.15.2:0/342397342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f003c106f10 0x7f003c10ea90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0034009ca0 tx=0x7f003402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:06:43.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.224+0000 7f0040c00640 1 -- 172.21.15.2:0/342397342 shutdown_connections 2024-09-10T08:06:43.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.224+0000 7f0040c00640 1 --2- 172.21.15.2:0/342397342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f003c106f10 0x7f003c10ea90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:43.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.224+0000 7f0040c00640 1 --2- 172.21.15.2:0/342397342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003c102790 0x7f003c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:43.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.224+0000 7f0040c00640 1 --2- 172.21.15.2:0/342397342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f003c101de0 0x7f003c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.224+0000 7f0040c00640 1 -- 172.21.15.2:0/342397342 >> 172.21.15.2:0/342397342 conn(0x7f003c0fd8c0 msgr2=0x7f003c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.224+0000 7f0040c00640 1 -- 172.21.15.2:0/342397342 shutdown_connections 2024-09-10T08:06:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.224+0000 7f0040c00640 1 -- 172.21.15.2:0/342397342 wait complete. 2024-09-10T08:06:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.225+0000 7f0040c00640 1 Processor -- start 2024-09-10T08:06:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.225+0000 7f0040c00640 1 -- start start 2024-09-10T08:06:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0040c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003c101de0 0x7f003c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0040c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f003c102790 0x7f003c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0040c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f003c106f10 0x7f003c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0040c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f003c111fc0 con 0x7f003c102790 2024-09-10T08:06:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0040c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f003c111e40 con 0x7f003c106f10 2024-09-10T08:06:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0040c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f003c112140 con 0x7f003c101de0 2024-09-10T08:06:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0039400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f003c102790 0x7f003c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f003a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f003c106f10 0x7f003c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0039400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f003c102790 0x7f003c19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60576/0 (socket says 172.21.15.2:60576) 2024-09-10T08:06:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0039400640 1 -- 172.21.15.2:0/2453728508 learned_addr learned my addr 172.21.15.2:0/2453728508 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0039e00640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003c101de0 0x7f003c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0039400640 1 -- 172.21.15.2:0/2453728508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003c101de0 msgr2=0x7f003c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0039400640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003c101de0 0x7f003c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0039400640 1 -- 172.21.15.2:0/2453728508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f003c106f10 msgr2=0x7f003c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.226+0000 7f0039400640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f003c106f10 0x7f003c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.227+0000 7f0039400640 1 -- 172.21.15.2:0/2453728508 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003c19f610 con 0x7f003c102790 2024-09-10T08:06:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.227+0000 7f0039e00640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003c101de0 0x7f003c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:06:43.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.227+0000 7f0039400640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f003c102790 0x7f003c19aba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f002800e9f0 tx=0x7f002800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.227+0000 7f0032a00640 1 -- 172.21.15.2:0/2453728508 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002800cd80 con 0x7f003c102790 2024-09-10T08:06:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.227+0000 7f0032a00640 1 -- 172.21.15.2:0/2453728508 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0028005500 con 0x7f003c102790 2024-09-10T08:06:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.227+0000 7f0032a00640 1 -- 172.21.15.2:0/2453728508 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028010690 con 0x7f003c102790 2024-09-10T08:06:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.227+0000 7f0040c00640 1 -- 172.21.15.2:0/2453728508 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003c19f900 con 0x7f003c102790 2024-09-10T08:06:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.227+0000 7f0040c00640 1 -- 172.21.15.2:0/2453728508 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003c1a71e0 con 0x7f003c102790 2024-09-10T08:06:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.229+0000 7f0040c00640 1 -- 172.21.15.2:0/2453728508 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f003c1021c0 con 0x7f003c102790 2024-09-10T08:06:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.231+0000 7f0032a00640 1 -- 172.21.15.2:0/2453728508 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f00280046c0 con 0x7f003c102790 2024-09-10T08:06:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.232+0000 7f0032a00640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00200a2c10 0x7f00200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.232+0000 7f0032a00640 1 -- 172.21.15.2:0/2453728508 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f00280cf9e0 con 0x7f003c102790 2024-09-10T08:06:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.233+0000 7f0039e00640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00200a2c10 0x7f00200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.233+0000 7f0039e00640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00200a2c10 0x7f00200a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f002c009b50 tx=0x7f002c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:43.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.237+0000 7f0032a00640 1 -- 172.21.15.2:0/2453728508 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f002809c420 con 0x7f003c102790 2024-09-10T08:06:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.418+0000 7f0040c00640 1 -- 172.21.15.2:0/2453728508 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f003c075430 con 0x7f00200a2c10 2024-09-10T08:06:43.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.430+0000 7f0032a00640 1 -- 172.21.15.2:0/2453728508 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340601 (secure 0 0 0) 0x7f003c075430 con 0x7f00200a2c10 2024-09-10T08:06:43.434 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:06:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-10T08:06:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.434+0000 7f0040c00640 1 -- 172.21.15.2:0/2453728508 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00200a2c10 msgr2=0x7f00200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.434+0000 7f0040c00640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00200a2c10 0x7f00200a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f002c009b50 tx=0x7f002c005f90 comp rx=0 tx=0).stop 2024-09-10T08:06:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.434+0000 7f0040c00640 1 -- 172.21.15.2:0/2453728508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f003c102790 msgr2=0x7f003c19aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:43.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.434+0000 7f0040c00640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f003c102790 0x7f003c19aba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f002800e9f0 tx=0x7f002800edf0 comp rx=0 tx=0).stop 2024-09-10T08:06:43.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.435+0000 7f0040c00640 1 -- 172.21.15.2:0/2453728508 shutdown_connections 2024-09-10T08:06:43.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.435+0000 7f0040c00640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f003c106f10 0x7f003c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:43.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.435+0000 7f0040c00640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f003c102790 0x7f003c19aba0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:43.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.435+0000 7f0040c00640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00200a2c10 0x7f00200a50d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:43.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.435+0000 7f0040c00640 1 --2- 172.21.15.2:0/2453728508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003c101de0 0x7f003c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:43.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.435+0000 7f0040c00640 1 -- 172.21.15.2:0/2453728508 >> 172.21.15.2:0/2453728508 conn(0x7f003c0fd8c0 msgr2=0x7f003c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:43.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.435+0000 7f0040c00640 1 -- 172.21.15.2:0/2453728508 shutdown_connections 2024-09-10T08:06:43.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:43.435+0000 7f0040c00640 1 -- 172.21.15.2:0/2453728508 wait complete. 2024-09-10T08:06:43.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:43 smithi002 ceph-mon[30563]: pgmap v342: 129 pgs: 129 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-09-10T08:06:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:43 smithi155 ceph-mon[35405]: pgmap v342: 129 pgs: 129 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-09-10T08:06:43.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:43 smithi107 ceph-mon[35048]: pgmap v342: 129 pgs: 129 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-09-10T08:06:44.306 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-10T08:06:41.658543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651863,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":257,"num_read_kb":241,"num_write":341,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1184461,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361712,"kb_used_data":12464,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580112,"statfs":{"total":1151940427776,"available":1151570034688,"internally_reserved":0,"allocated":12763136,"data_stored":8266300,"data_compressed":426582,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54041,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":95,"num_write":29,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003229"},"pg_stats":[{"pgid":"2.1f","version":"93'9","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.794582+0000","last_change":"2024-09-10T08:05:22.462062+0000","last_active":"2024-09-10T08:06:20.794582+0000","last_peered":"2024-09-10T08:06:20.794582+0000","last_clean":"2024-09-10T08:06:20.794582+0000","last_became_active":"2024-09-10T08:05:22.460973+0000","last_became_peered":"2024-09-10T08:05:22.460973+0000","last_unstale":"2024-09-10T08:06:20.794582+0000","last_undegraded":"2024-09-10T08:06:20.794582+0000","last_fullsized":"2024-09-10T08:06:20.794582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:01.357446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.735468+0000","last_change":"2024-09-10T08:05:25.482873+0000","last_active":"2024-09-10T08:06:04.735468+0000","last_peered":"2024-09-10T08:06:04.735468+0000","last_clean":"2024-09-10T08:06:04.735468+0000","last_became_active":"2024-09-10T08:05:25.482559+0000","last_became_peered":"2024-09-10T08:05:25.482559+0000","last_unstale":"2024-09-10T08:06:04.735468+0000","last_undegraded":"2024-09-10T08:06:04.735468+0000","last_fullsized":"2024-09-10T08:06:04.735468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:10:17.366160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.736356+0000","last_change":"2024-09-10T08:06:04.736356+0000","last_active":"2024-09-10T08:06:04.736356+0000","last_peered":"2024-09-10T08:06:04.736356+0000","last_clean":"2024-09-10T08:06:04.736356+0000","last_became_active":"2024-09-10T08:06:04.735859+0000","last_became_peered":"2024-09-10T08:06:04.735859+0000","last_unstale":"2024-09-10T08:06:04.736356+0000","last_undegraded":"2024-09-10T08:06:04.736356+0000","last_fullsized":"2024-09-10T08:06:04.736356+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:56.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"93'2","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.735769+0000","last_change":"2024-09-10T08:05:22.569504+0000","last_active":"2024-09-10T08:06:04.735769+0000","last_peered":"2024-09-10T08:06:04.735769+0000","last_clean":"2024-09-10T08:06:04.735769+0000","last_became_active":"2024-09-10T08:05:22.569164+0000","last_became_peered":"2024-09-10T08:05:22.569164+0000","last_unstale":"2024-09-10T08:06:04.735769+0000","last_undegraded":"2024-09-10T08:06:04.735769+0000","last_fullsized":"2024-09-10T08:06:04.735769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:41.628963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733116+0000","last_change":"2024-09-10T08:05:25.493017+0000","last_active":"2024-09-10T08:06:04.733116+0000","last_peered":"2024-09-10T08:06:04.733116+0000","last_clean":"2024-09-10T08:06:04.733116+0000","last_became_active":"2024-09-10T08:05:25.491370+0000","last_became_peered":"2024-09-10T08:05:25.491370+0000","last_unstale":"2024-09-10T08:06:04.733116+0000","last_undegraded":"2024-09-10T08:06:04.733116+0000","last_fullsized":"2024-09-10T08:06:04.733116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:41.616549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728519+0000","last_change":"2024-09-10T08:05:35.523751+0000","last_active":"2024-09-10T08:06:04.728519+0000","last_peered":"2024-09-10T08:06:04.728519+0000","last_clean":"2024-09-10T08:06:04.728519+0000","last_became_active":"2024-09-10T08:05:35.523372+0000","last_became_peered":"2024-09-10T08:05:35.523372+0000","last_unstale":"2024-09-10T08:06:04.728519+0000","last_undegraded":"2024-09-10T08:06:04.728519+0000","last_fullsized":"2024-09-10T08:06:04.728519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:20.971596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'4","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.793322+0000","last_change":"2024-09-10T08:05:22.460590+0000","last_active":"2024-09-10T08:06:20.793322+0000","last_peered":"2024-09-10T08:06:20.793322+0000","last_clean":"2024-09-10T08:06:20.793322+0000","last_became_active":"2024-09-10T08:05:22.460238+0000","last_became_peered":"2024-09-10T08:05:22.460238+0000","last_unstale":"2024-09-10T08:06:20.793322+0000","last_undegraded":"2024-09-10T08:06:20.793322+0000","last_fullsized":"2024-09-10T08:06:20.793322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:18.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728483+0000","last_change":"2024-09-10T08:05:25.487392+0000","last_active":"2024-09-10T08:06:04.728483+0000","last_peered":"2024-09-10T08:06:04.728483+0000","last_clean":"2024-09-10T08:06:04.728483+0000","last_became_active":"2024-09-10T08:05:25.487054+0000","last_became_peered":"2024-09-10T08:05:25.487054+0000","last_unstale":"2024-09-10T08:06:04.728483+0000","last_undegraded":"2024-09-10T08:06:04.728483+0000","last_fullsized":"2024-09-10T08:06:04.728483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:16.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.735778+0000","last_change":"2024-09-10T08:06:04.735778+0000","last_active":"2024-09-10T08:06:04.735778+0000","last_peered":"2024-09-10T08:06:04.735778+0000","last_clean":"2024-09-10T08:06:04.735778+0000","last_became_active":"2024-09-10T08:06:04.735208+0000","last_became_peered":"2024-09-10T08:06:04.735208+0000","last_unstale":"2024-09-10T08:06:04.735778+0000","last_undegraded":"2024-09-10T08:06:04.735778+0000","last_fullsized":"2024-09-10T08:06:04.735778+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:36.982246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729026+0000","last_change":"2024-09-10T08:05:22.458323+0000","last_active":"2024-09-10T08:06:04.729026+0000","last_peered":"2024-09-10T08:06:04.729026+0000","last_clean":"2024-09-10T08:06:04.729026+0000","last_became_active":"2024-09-10T08:05:22.457965+0000","last_became_peered":"2024-09-10T08:05:22.457965+0000","last_unstale":"2024-09-10T08:06:04.729026+0000","last_undegraded":"2024-09-10T08:06:04.729026+0000","last_fullsized":"2024-09-10T08:06:04.729026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:15.021305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.726424+0000","last_change":"2024-09-10T08:05:25.489883+0000","last_active":"2024-09-10T08:06:04.726424+0000","last_peered":"2024-09-10T08:06:04.726424+0000","last_clean":"2024-09-10T08:06:04.726424+0000","last_became_active":"2024-09-10T08:05:25.489482+0000","last_became_peered":"2024-09-10T08:05:25.489482+0000","last_unstale":"2024-09-10T08:06:04.726424+0000","last_undegraded":"2024-09-10T08:06:04.726424+0000","last_fullsized":"2024-09-10T08:06:04.726424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:52.080628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728497+0000","last_change":"2024-09-10T08:05:35.531458+0000","last_active":"2024-09-10T08:06:04.728497+0000","last_peered":"2024-09-10T08:06:04.728497+0000","last_clean":"2024-09-10T08:06:04.728497+0000","last_became_active":"2024-09-10T08:05:35.531072+0000","last_became_peered":"2024-09-10T08:05:35.531072+0000","last_unstale":"2024-09-10T08:06:04.728497+0000","last_undegraded":"2024-09-10T08:06:04.728497+0000","last_fullsized":"2024-09-10T08:06:04.728497+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:04.792664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731989+0000","last_change":"2024-09-10T08:05:25.486865+0000","last_active":"2024-09-10T08:06:04.731989+0000","last_peered":"2024-09-10T08:06:04.731989+0000","last_clean":"2024-09-10T08:06:04.731989+0000","last_became_active":"2024-09-10T08:05:25.486531+0000","last_became_peered":"2024-09-10T08:05:25.486531+0000","last_unstale":"2024-09-10T08:06:04.731989+0000","last_undegraded":"2024-09-10T08:06:04.731989+0000","last_fullsized":"2024-09-10T08:06:04.731989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:26.854682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"93'4","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.791172+0000","last_change":"2024-09-10T08:06:04.743303+0000","last_active":"2024-09-10T08:06:19.791172+0000","last_peered":"2024-09-10T08:06:19.791172+0000","last_clean":"2024-09-10T08:06:19.791172+0000","last_became_active":"2024-09-10T08:06:04.733294+0000","last_became_peered":"2024-09-10T08:06:04.733294+0000","last_unstale":"2024-09-10T08:06:19.791172+0000","last_undegraded":"2024-09-10T08:06:19.791172+0000","last_fullsized":"2024-09-10T08:06:19.791172+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:18.542658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728881+0000","last_change":"2024-09-10T08:05:35.642083+0000","last_active":"2024-09-10T08:06:04.728881+0000","last_peered":"2024-09-10T08:06:04.728881+0000","last_clean":"2024-09-10T08:06:04.728881+0000","last_became_active":"2024-09-10T08:05:35.641214+0000","last_became_peered":"2024-09-10T08:05:35.641214+0000","last_unstale":"2024-09-10T08:06:04.728881+0000","last_undegraded":"2024-09-10T08:06:04.728881+0000","last_fullsized":"2024-09-10T08:06:04.728881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:00:21.731769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733707+0000","last_change":"2024-09-10T08:05:25.501993+0000","last_active":"2024-09-10T08:06:04.733707+0000","last_peered":"2024-09-10T08:06:04.733707+0000","last_clean":"2024-09-10T08:06:04.733707+0000","last_became_active":"2024-09-10T08:05:25.499912+0000","last_became_peered":"2024-09-10T08:05:25.499912+0000","last_unstale":"2024-09-10T08:06:04.733707+0000","last_undegraded":"2024-09-10T08:06:04.733707+0000","last_fullsized":"2024-09-10T08:06:04.733707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:06.780067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"93'1","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732696+0000","last_change":"2024-09-10T08:05:22.461786+0000","last_active":"2024-09-10T08:06:04.732696+0000","last_peered":"2024-09-10T08:06:04.732696+0000","last_clean":"2024-09-10T08:06:04.732696+0000","last_became_active":"2024-09-10T08:05:22.460821+0000","last_became_peered":"2024-09-10T08:05:22.460821+0000","last_unstale":"2024-09-10T08:06:04.732696+0000","last_undegraded":"2024-09-10T08:06:04.732696+0000","last_fullsized":"2024-09-10T08:06:04.732696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:25.083538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.735635+0000","last_change":"2024-09-10T08:06:04.734470+0000","last_active":"2024-09-10T08:06:04.735635+0000","last_peered":"2024-09-10T08:06:04.735635+0000","last_clean":"2024-09-10T08:06:04.735635+0000","last_became_active":"2024-09-10T08:06:04.734040+0000","last_became_peered":"2024-09-10T08:06:04.734040+0000","last_unstale":"2024-09-10T08:06:04.735635+0000","last_undegraded":"2024-09-10T08:06:04.735635+0000","last_fullsized":"2024-09-10T08:06:04.735635+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:36.312144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728729+0000","last_change":"2024-09-10T08:05:25.492711+0000","last_active":"2024-09-10T08:06:04.728729+0000","last_peered":"2024-09-10T08:06:04.728729+0000","last_clean":"2024-09-10T08:06:04.728729+0000","last_became_active":"2024-09-10T08:05:25.492336+0000","last_became_peered":"2024-09-10T08:05:25.492336+0000","last_unstale":"2024-09-10T08:06:04.728729+0000","last_undegraded":"2024-09-10T08:06:04.728729+0000","last_fullsized":"2024-09-10T08:06:04.728729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:04.168884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"93'3","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.790001+0000","last_change":"2024-09-10T08:05:22.461535+0000","last_active":"2024-09-10T08:06:19.790001+0000","last_peered":"2024-09-10T08:06:19.790001+0000","last_clean":"2024-09-10T08:06:19.790001+0000","last_became_active":"2024-09-10T08:05:22.461208+0000","last_became_peered":"2024-09-10T08:05:22.461208+0000","last_unstale":"2024-09-10T08:06:19.790001+0000","last_undegraded":"2024-09-10T08:06:19.790001+0000","last_fullsized":"2024-09-10T08:06:19.790001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:26.009242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731554+0000","last_change":"2024-09-10T08:05:35.535442+0000","last_active":"2024-09-10T08:06:04.731554+0000","last_peered":"2024-09-10T08:06:04.731554+0000","last_clean":"2024-09-10T08:06:04.731554+0000","last_became_active":"2024-09-10T08:05:35.535058+0000","last_became_peered":"2024-09-10T08:05:35.535058+0000","last_unstale":"2024-09-10T08:06:04.731554+0000","last_undegraded":"2024-09-10T08:06:04.731554+0000","last_fullsized":"2024-09-10T08:06:04.731554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:39.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.727471+0000","last_change":"2024-09-10T08:05:25.483440+0000","last_active":"2024-09-10T08:06:04.727471+0000","last_peered":"2024-09-10T08:06:04.727471+0000","last_clean":"2024-09-10T08:06:04.727471+0000","last_became_active":"2024-09-10T08:05:25.483141+0000","last_became_peered":"2024-09-10T08:05:25.483141+0000","last_unstale":"2024-09-10T08:06:04.727471+0000","last_undegraded":"2024-09-10T08:06:04.727471+0000","last_fullsized":"2024-09-10T08:06:04.727471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:00.953188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"93'2","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.791516+0000","last_change":"2024-09-10T08:05:22.568141+0000","last_active":"2024-09-10T08:06:19.791516+0000","last_peered":"2024-09-10T08:06:19.791516+0000","last_clean":"2024-09-10T08:06:19.791516+0000","last_became_active":"2024-09-10T08:05:22.567857+0000","last_became_peered":"2024-09-10T08:05:22.567857+0000","last_unstale":"2024-09-10T08:06:19.791516+0000","last_undegraded":"2024-09-10T08:06:19.791516+0000","last_fullsized":"2024-09-10T08:06:19.791516+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:22.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729420+0000","last_change":"2024-09-10T08:05:35.641993+0000","last_active":"2024-09-10T08:06:04.729420+0000","last_peered":"2024-09-10T08:06:04.729420+0000","last_clean":"2024-09-10T08:06:04.729420+0000","last_became_active":"2024-09-10T08:05:35.641376+0000","last_became_peered":"2024-09-10T08:05:35.641376+0000","last_unstale":"2024-09-10T08:06:04.729420+0000","last_undegraded":"2024-09-10T08:06:04.729420+0000","last_fullsized":"2024-09-10T08:06:04.729420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:16:34.038924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.737560+0000","last_change":"2024-09-10T08:06:04.737560+0000","last_active":"2024-09-10T08:06:04.737560+0000","last_peered":"2024-09-10T08:06:04.737560+0000","last_clean":"2024-09-10T08:06:04.737560+0000","last_became_active":"2024-09-10T08:06:04.736656+0000","last_became_peered":"2024-09-10T08:06:04.736656+0000","last_unstale":"2024-09-10T08:06:04.737560+0000","last_undegraded":"2024-09-10T08:06:04.737560+0000","last_fullsized":"2024-09-10T08:06:04.737560+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:35.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"93'4","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.792092+0000","last_change":"2024-09-10T08:05:22.567077+0000","last_active":"2024-09-10T08:06:19.792092+0000","last_peered":"2024-09-10T08:06:19.792092+0000","last_clean":"2024-09-10T08:06:19.792092+0000","last_became_active":"2024-09-10T08:05:22.566757+0000","last_became_peered":"2024-09-10T08:05:22.566757+0000","last_unstale":"2024-09-10T08:06:19.792092+0000","last_undegraded":"2024-09-10T08:06:19.792092+0000","last_fullsized":"2024-09-10T08:06:19.792092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:06.881510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731567+0000","last_change":"2024-09-10T08:05:35.519539+0000","last_active":"2024-09-10T08:06:04.731567+0000","last_peered":"2024-09-10T08:06:04.731567+0000","last_clean":"2024-09-10T08:06:04.731567+0000","last_became_active":"2024-09-10T08:05:35.519044+0000","last_became_peered":"2024-09-10T08:05:35.519044+0000","last_unstale":"2024-09-10T08:06:04.731567+0000","last_undegraded":"2024-09-10T08:06:04.731567+0000","last_fullsized":"2024-09-10T08:06:04.731567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:10.546949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733989+0000","last_change":"2024-09-10T08:05:25.489753+0000","last_active":"2024-09-10T08:06:04.733989+0000","last_peered":"2024-09-10T08:06:04.733989+0000","last_clean":"2024-09-10T08:06:04.733989+0000","last_became_active":"2024-09-10T08:05:25.489455+0000","last_became_peered":"2024-09-10T08:05:25.489455+0000","last_unstale":"2024-09-10T08:06:04.733989+0000","last_undegraded":"2024-09-10T08:06:04.733989+0000","last_fullsized":"2024-09-10T08:06:04.733989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:23:53.667875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'8","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:35.504609+0000","last_change":"2024-09-10T08:06:04.741199+0000","last_active":"2024-09-10T08:06:35.504609+0000","last_peered":"2024-09-10T08:06:35.504609+0000","last_clean":"2024-09-10T08:06:35.504609+0000","last_became_active":"2024-09-10T08:06:04.738076+0000","last_became_peered":"2024-09-10T08:06:04.738076+0000","last_unstale":"2024-09-10T08:06:35.504609+0000","last_undegraded":"2024-09-10T08:06:35.504609+0000","last_fullsized":"2024-09-10T08:06:35.504609+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:45.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733304+0000","last_change":"2024-09-10T08:05:35.642358+0000","last_active":"2024-09-10T08:06:04.733304+0000","last_peered":"2024-09-10T08:06:04.733304+0000","last_clean":"2024-09-10T08:06:04.733304+0000","last_became_active":"2024-09-10T08:05:35.641408+0000","last_became_peered":"2024-09-10T08:05:35.641408+0000","last_unstale":"2024-09-10T08:06:04.733304+0000","last_undegraded":"2024-09-10T08:06:04.733304+0000","last_fullsized":"2024-09-10T08:06:04.733304+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:42.015139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732023+0000","last_change":"2024-09-10T08:05:25.486310+0000","last_active":"2024-09-10T08:06:04.732023+0000","last_peered":"2024-09-10T08:06:04.732023+0000","last_clean":"2024-09-10T08:06:04.732023+0000","last_became_active":"2024-09-10T08:05:25.485915+0000","last_became_peered":"2024-09-10T08:05:25.485915+0000","last_unstale":"2024-09-10T08:06:04.732023+0000","last_undegraded":"2024-09-10T08:06:04.732023+0000","last_fullsized":"2024-09-10T08:06:04.732023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:56.064361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"93'1","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.791751+0000","last_change":"2024-09-10T08:05:22.570595+0000","last_active":"2024-09-10T08:06:19.791751+0000","last_peered":"2024-09-10T08:06:19.791751+0000","last_clean":"2024-09-10T08:06:19.791751+0000","last_became_active":"2024-09-10T08:05:22.570219+0000","last_became_peered":"2024-09-10T08:05:22.570219+0000","last_unstale":"2024-09-10T08:06:19.791751+0000","last_undegraded":"2024-09-10T08:06:19.791751+0000","last_fullsized":"2024-09-10T08:06:19.791751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:03.230325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729161+0000","last_change":"2024-09-10T08:05:35.530472+0000","last_active":"2024-09-10T08:06:04.729161+0000","last_peered":"2024-09-10T08:06:04.729161+0000","last_clean":"2024-09-10T08:06:04.729161+0000","last_became_active":"2024-09-10T08:05:35.530066+0000","last_became_peered":"2024-09-10T08:05:35.530066+0000","last_unstale":"2024-09-10T08:06:04.729161+0000","last_undegraded":"2024-09-10T08:06:04.729161+0000","last_fullsized":"2024-09-10T08:06:04.729161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:58.915767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725911+0000","last_change":"2024-09-10T08:05:25.500524+0000","last_active":"2024-09-10T08:06:04.725911+0000","last_peered":"2024-09-10T08:06:04.725911+0000","last_clean":"2024-09-10T08:06:04.725911+0000","last_became_active":"2024-09-10T08:05:25.500186+0000","last_became_peered":"2024-09-10T08:05:25.500186+0000","last_unstale":"2024-09-10T08:06:04.725911+0000","last_undegraded":"2024-09-10T08:06:04.725911+0000","last_fullsized":"2024-09-10T08:06:04.725911+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:51.936222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731033+0000","last_change":"2024-09-10T08:05:22.569326+0000","last_active":"2024-09-10T08:06:04.731033+0000","last_peered":"2024-09-10T08:06:04.731033+0000","last_clean":"2024-09-10T08:06:04.731033+0000","last_became_active":"2024-09-10T08:05:22.568895+0000","last_became_peered":"2024-09-10T08:05:22.568895+0000","last_unstale":"2024-09-10T08:06:04.731033+0000","last_undegraded":"2024-09-10T08:06:04.731033+0000","last_fullsized":"2024-09-10T08:06:04.731033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:47.212073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728741+0000","last_change":"2024-09-10T08:05:35.641600+0000","last_active":"2024-09-10T08:06:04.728741+0000","last_peered":"2024-09-10T08:06:04.728741+0000","last_clean":"2024-09-10T08:06:04.728741+0000","last_became_active":"2024-09-10T08:05:35.641205+0000","last_became_peered":"2024-09-10T08:05:35.641205+0000","last_unstale":"2024-09-10T08:06:04.728741+0000","last_undegraded":"2024-09-10T08:06:04.728741+0000","last_fullsized":"2024-09-10T08:06:04.728741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:40.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725888+0000","last_change":"2024-09-10T08:05:25.495035+0000","last_active":"2024-09-10T08:06:04.725888+0000","last_peered":"2024-09-10T08:06:04.725888+0000","last_clean":"2024-09-10T08:06:04.725888+0000","last_became_active":"2024-09-10T08:05:25.494733+0000","last_became_peered":"2024-09-10T08:05:25.494733+0000","last_unstale":"2024-09-10T08:06:04.725888+0000","last_undegraded":"2024-09-10T08:06:04.725888+0000","last_fullsized":"2024-09-10T08:06:04.725888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.875867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"93'1","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732432+0000","last_change":"2024-09-10T08:05:22.458727+0000","last_active":"2024-09-10T08:06:04.732432+0000","last_peered":"2024-09-10T08:06:04.732432+0000","last_clean":"2024-09-10T08:06:04.732432+0000","last_became_active":"2024-09-10T08:05:22.458397+0000","last_became_peered":"2024-09-10T08:05:22.458397+0000","last_unstale":"2024-09-10T08:06:04.732432+0000","last_undegraded":"2024-09-10T08:06:04.732432+0000","last_fullsized":"2024-09-10T08:06:04.732432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:07.875559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731455+0000","last_change":"2024-09-10T08:05:35.534192+0000","last_active":"2024-09-10T08:06:04.731455+0000","last_peered":"2024-09-10T08:06:04.731455+0000","last_clean":"2024-09-10T08:06:04.731455+0000","last_became_active":"2024-09-10T08:05:35.530450+0000","last_became_peered":"2024-09-10T08:05:35.530450+0000","last_unstale":"2024-09-10T08:06:04.731455+0000","last_undegraded":"2024-09-10T08:06:04.731455+0000","last_fullsized":"2024-09-10T08:06:04.731455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:10.039144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725579+0000","last_change":"2024-09-10T08:05:25.482252+0000","last_active":"2024-09-10T08:06:04.725579+0000","last_peered":"2024-09-10T08:06:04.725579+0000","last_clean":"2024-09-10T08:06:04.725579+0000","last_became_active":"2024-09-10T08:05:25.481901+0000","last_became_peered":"2024-09-10T08:05:25.481901+0000","last_unstale":"2024-09-10T08:06:04.725579+0000","last_undegraded":"2024-09-10T08:06:04.725579+0000","last_fullsized":"2024-09-10T08:06:04.725579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:17.063340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'4","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:36.284676+0000","last_change":"2024-09-10T08:05:22.569199+0000","last_active":"2024-09-10T08:06:36.284676+0000","last_peered":"2024-09-10T08:06:36.284676+0000","last_clean":"2024-09-10T08:06:36.284676+0000","last_became_active":"2024-09-10T08:05:22.568787+0000","last_became_peered":"2024-09-10T08:05:22.568787+0000","last_unstale":"2024-09-10T08:06:36.284676+0000","last_undegraded":"2024-09-10T08:06:36.284676+0000","last_fullsized":"2024-09-10T08:06:36.284676+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:59.610130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.734072+0000","last_change":"2024-09-10T08:05:35.642446+0000","last_active":"2024-09-10T08:06:04.734072+0000","last_peered":"2024-09-10T08:06:04.734072+0000","last_clean":"2024-09-10T08:06:04.734072+0000","last_became_active":"2024-09-10T08:05:35.641543+0000","last_became_peered":"2024-09-10T08:05:35.641543+0000","last_unstale":"2024-09-10T08:06:04.734072+0000","last_undegraded":"2024-09-10T08:06:04.734072+0000","last_fullsized":"2024-09-10T08:06:04.734072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:33.658406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730139+0000","last_change":"2024-09-10T08:05:25.487598+0000","last_active":"2024-09-10T08:06:04.730139+0000","last_peered":"2024-09-10T08:06:04.730139+0000","last_clean":"2024-09-10T08:06:04.730139+0000","last_became_active":"2024-09-10T08:05:25.486879+0000","last_became_peered":"2024-09-10T08:05:25.486879+0000","last_unstale":"2024-09-10T08:06:04.730139+0000","last_undegraded":"2024-09-10T08:06:04.730139+0000","last_fullsized":"2024-09-10T08:06:04.730139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:20.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'3","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.792775+0000","last_change":"2024-09-10T08:05:22.463416+0000","last_active":"2024-09-10T08:06:20.792775+0000","last_peered":"2024-09-10T08:06:20.792775+0000","last_clean":"2024-09-10T08:06:20.792775+0000","last_became_active":"2024-09-10T08:05:22.463004+0000","last_became_peered":"2024-09-10T08:05:22.463004+0000","last_unstale":"2024-09-10T08:06:20.792775+0000","last_undegraded":"2024-09-10T08:06:20.792775+0000","last_fullsized":"2024-09-10T08:06:20.792775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:56.257812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.726319+0000","last_change":"2024-09-10T08:05:35.642059+0000","last_active":"2024-09-10T08:06:04.726319+0000","last_peered":"2024-09-10T08:06:04.726319+0000","last_clean":"2024-09-10T08:06:04.726319+0000","last_became_active":"2024-09-10T08:05:35.641540+0000","last_became_peered":"2024-09-10T08:05:35.641540+0000","last_unstale":"2024-09-10T08:06:04.726319+0000","last_undegraded":"2024-09-10T08:06:04.726319+0000","last_fullsized":"2024-09-10T08:06:04.726319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:28.557584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.734223+0000","last_change":"2024-09-10T08:05:25.496111+0000","last_active":"2024-09-10T08:06:04.734223+0000","last_peered":"2024-09-10T08:06:04.734223+0000","last_clean":"2024-09-10T08:06:04.734223+0000","last_became_active":"2024-09-10T08:05:25.495757+0000","last_became_peered":"2024-09-10T08:05:25.495757+0000","last_unstale":"2024-09-10T08:06:04.734223+0000","last_undegraded":"2024-09-10T08:06:04.734223+0000","last_fullsized":"2024-09-10T08:06:04.734223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:41.078205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'3","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:22.260153+0000","last_change":"2024-09-10T08:05:22.460069+0000","last_active":"2024-09-10T08:06:22.260153+0000","last_peered":"2024-09-10T08:06:22.260153+0000","last_clean":"2024-09-10T08:06:22.260153+0000","last_became_active":"2024-09-10T08:05:22.459557+0000","last_became_peered":"2024-09-10T08:05:22.459557+0000","last_unstale":"2024-09-10T08:06:22.260153+0000","last_undegraded":"2024-09-10T08:06:22.260153+0000","last_fullsized":"2024-09-10T08:06:22.260153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:49.058507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732140+0000","last_change":"2024-09-10T08:04:08.188262+0000","last_active":"2024-09-10T08:06:04.732140+0000","last_peered":"2024-09-10T08:06:04.732140+0000","last_clean":"2024-09-10T08:06:04.732140+0000","last_became_active":"2024-09-10T08:04:08.187691+0000","last_became_peered":"2024-09-10T08:04:08.187691+0000","last_unstale":"2024-09-10T08:06:04.732140+0000","last_undegraded":"2024-09-10T08:06:04.732140+0000","last_fullsized":"2024-09-10T08:06:04.732140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_clean_scrub_stamp":"2024-09-10T08:00:26.235127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:15.358818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732657+0000","last_change":"2024-09-10T08:05:35.528371+0000","last_active":"2024-09-10T08:06:04.732657+0000","last_peered":"2024-09-10T08:06:04.732657+0000","last_clean":"2024-09-10T08:06:04.732657+0000","last_became_active":"2024-09-10T08:05:35.527925+0000","last_became_peered":"2024-09-10T08:05:35.527925+0000","last_unstale":"2024-09-10T08:06:04.732657+0000","last_undegraded":"2024-09-10T08:06:04.732657+0000","last_fullsized":"2024-09-10T08:06:04.732657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:37:00.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732862+0000","last_change":"2024-09-10T08:05:25.490753+0000","last_active":"2024-09-10T08:06:04.732862+0000","last_peered":"2024-09-10T08:06:04.732862+0000","last_clean":"2024-09-10T08:06:04.732862+0000","last_became_active":"2024-09-10T08:05:25.489996+0000","last_became_peered":"2024-09-10T08:05:25.489996+0000","last_unstale":"2024-09-10T08:06:04.732862+0000","last_undegraded":"2024-09-10T08:06:04.732862+0000","last_fullsized":"2024-09-10T08:06:04.732862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:15.637199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'3","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.795368+0000","last_change":"2024-09-10T08:05:22.568696+0000","last_active":"2024-09-10T08:06:20.795368+0000","last_peered":"2024-09-10T08:06:20.795368+0000","last_clean":"2024-09-10T08:06:20.795368+0000","last_became_active":"2024-09-10T08:05:22.568331+0000","last_became_peered":"2024-09-10T08:05:22.568331+0000","last_unstale":"2024-09-10T08:06:20.795368+0000","last_undegraded":"2024-09-10T08:06:20.795368+0000","last_fullsized":"2024-09-10T08:06:20.795368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:21.699397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728181+0000","last_change":"2024-09-10T08:05:35.530804+0000","last_active":"2024-09-10T08:06:04.728181+0000","last_peered":"2024-09-10T08:06:04.728181+0000","last_clean":"2024-09-10T08:06:04.728181+0000","last_became_active":"2024-09-10T08:05:35.530356+0000","last_became_peered":"2024-09-10T08:05:35.530356+0000","last_unstale":"2024-09-10T08:06:04.728181+0000","last_undegraded":"2024-09-10T08:06:04.728181+0000","last_fullsized":"2024-09-10T08:06:04.728181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:59.318761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732403+0000","last_change":"2024-09-10T08:05:25.489546+0000","last_active":"2024-09-10T08:06:04.732403+0000","last_peered":"2024-09-10T08:06:04.732403+0000","last_clean":"2024-09-10T08:06:04.732403+0000","last_became_active":"2024-09-10T08:05:25.489188+0000","last_became_peered":"2024-09-10T08:05:25.489188+0000","last_unstale":"2024-09-10T08:06:04.732403+0000","last_undegraded":"2024-09-10T08:06:04.732403+0000","last_fullsized":"2024-09-10T08:06:04.732403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:26.471681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.726696+0000","last_change":"2024-09-10T08:05:22.463912+0000","last_active":"2024-09-10T08:06:04.726696+0000","last_peered":"2024-09-10T08:06:04.726696+0000","last_clean":"2024-09-10T08:06:04.726696+0000","last_became_active":"2024-09-10T08:05:22.463561+0000","last_became_peered":"2024-09-10T08:05:22.463561+0000","last_unstale":"2024-09-10T08:06:04.726696+0000","last_undegraded":"2024-09-10T08:06:04.726696+0000","last_fullsized":"2024-09-10T08:06:04.726696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:22.407604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729701+0000","last_change":"2024-09-10T08:05:35.528795+0000","last_active":"2024-09-10T08:06:04.729701+0000","last_peered":"2024-09-10T08:06:04.729701+0000","last_clean":"2024-09-10T08:06:04.729701+0000","last_became_active":"2024-09-10T08:05:35.528402+0000","last_became_peered":"2024-09-10T08:05:35.528402+0000","last_unstale":"2024-09-10T08:06:04.729701+0000","last_undegraded":"2024-09-10T08:06:04.729701+0000","last_fullsized":"2024-09-10T08:06:04.729701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:14.606371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732856+0000","last_change":"2024-09-10T08:05:25.484086+0000","last_active":"2024-09-10T08:06:04.732856+0000","last_peered":"2024-09-10T08:06:04.732856+0000","last_clean":"2024-09-10T08:06:04.732856+0000","last_became_active":"2024-09-10T08:05:25.483757+0000","last_became_peered":"2024-09-10T08:05:25.483757+0000","last_unstale":"2024-09-10T08:06:04.732856+0000","last_undegraded":"2024-09-10T08:06:04.732856+0000","last_fullsized":"2024-09-10T08:06:04.732856+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:16:16.362739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733080+0000","last_change":"2024-09-10T08:05:22.462220+0000","last_active":"2024-09-10T08:06:04.733080+0000","last_peered":"2024-09-10T08:06:04.733080+0000","last_clean":"2024-09-10T08:06:04.733080+0000","last_became_active":"2024-09-10T08:05:22.461726+0000","last_became_peered":"2024-09-10T08:05:22.461726+0000","last_unstale":"2024-09-10T08:06:04.733080+0000","last_undegraded":"2024-09-10T08:06:04.733080+0000","last_fullsized":"2024-09-10T08:06:04.733080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:56.381034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.735443+0000","last_change":"2024-09-10T08:06:04.734827+0000","last_active":"2024-09-10T08:06:04.735443+0000","last_peered":"2024-09-10T08:06:04.735443+0000","last_clean":"2024-09-10T08:06:04.735443+0000","last_became_active":"2024-09-10T08:06:04.734402+0000","last_became_peered":"2024-09-10T08:06:04.734402+0000","last_unstale":"2024-09-10T08:06:04.735443+0000","last_undegraded":"2024-09-10T08:06:04.735443+0000","last_fullsized":"2024-09-10T08:06:04.735443+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:24.806538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730162+0000","last_change":"2024-09-10T08:05:25.486100+0000","last_active":"2024-09-10T08:06:04.730162+0000","last_peered":"2024-09-10T08:06:04.730162+0000","last_clean":"2024-09-10T08:06:04.730162+0000","last_became_active":"2024-09-10T08:05:25.485586+0000","last_became_peered":"2024-09-10T08:05:25.485586+0000","last_unstale":"2024-09-10T08:06:04.730162+0000","last_undegraded":"2024-09-10T08:06:04.730162+0000","last_fullsized":"2024-09-10T08:06:04.730162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:33.565054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"93'2","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:22.263114+0000","last_change":"2024-09-10T08:06:04.743408+0000","last_active":"2024-09-10T08:06:22.263114+0000","last_peered":"2024-09-10T08:06:22.263114+0000","last_clean":"2024-09-10T08:06:22.263114+0000","last_became_active":"2024-09-10T08:06:04.732830+0000","last_became_peered":"2024-09-10T08:06:04.732830+0000","last_unstale":"2024-09-10T08:06:22.263114+0000","last_undegraded":"2024-09-10T08:06:22.263114+0000","last_fullsized":"2024-09-10T08:06:22.263114+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:29.381259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725544+0000","last_change":"2024-09-10T08:05:35.533015+0000","last_active":"2024-09-10T08:06:04.725544+0000","last_peered":"2024-09-10T08:06:04.725544+0000","last_clean":"2024-09-10T08:06:04.725544+0000","last_became_active":"2024-09-10T08:05:35.532590+0000","last_became_peered":"2024-09-10T08:05:35.532590+0000","last_unstale":"2024-09-10T08:06:04.725544+0000","last_undegraded":"2024-09-10T08:06:04.725544+0000","last_fullsized":"2024-09-10T08:06:04.725544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:23.984273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725636+0000","last_change":"2024-09-10T08:05:25.489994+0000","last_active":"2024-09-10T08:06:04.725636+0000","last_peered":"2024-09-10T08:06:04.725636+0000","last_clean":"2024-09-10T08:06:04.725636+0000","last_became_active":"2024-09-10T08:05:25.489471+0000","last_became_peered":"2024-09-10T08:05:25.489471+0000","last_unstale":"2024-09-10T08:06:04.725636+0000","last_undegraded":"2024-09-10T08:06:04.725636+0000","last_fullsized":"2024-09-10T08:06:04.725636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:53.866570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"93'15","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.795609+0000","last_change":"2024-09-10T08:05:22.568900+0000","last_active":"2024-09-10T08:06:20.795609+0000","last_peered":"2024-09-10T08:06:20.795609+0000","last_clean":"2024-09-10T08:06:20.795609+0000","last_became_active":"2024-09-10T08:05:22.568542+0000","last_became_peered":"2024-09-10T08:05:22.568542+0000","last_unstale":"2024-09-10T08:06:20.795609+0000","last_undegraded":"2024-09-10T08:06:20.795609+0000","last_fullsized":"2024-09-10T08:06:20.795609+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:58:45.669769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732976+0000","last_change":"2024-09-10T08:05:35.537364+0000","last_active":"2024-09-10T08:06:04.732976+0000","last_peered":"2024-09-10T08:06:04.732976+0000","last_clean":"2024-09-10T08:06:04.732976+0000","last_became_active":"2024-09-10T08:05:35.535958+0000","last_became_peered":"2024-09-10T08:05:35.535958+0000","last_unstale":"2024-09-10T08:06:04.732976+0000","last_undegraded":"2024-09-10T08:06:04.732976+0000","last_fullsized":"2024-09-10T08:06:04.732976+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:51.443798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731922+0000","last_change":"2024-09-10T08:05:25.485427+0000","last_active":"2024-09-10T08:06:04.731922+0000","last_peered":"2024-09-10T08:06:04.731922+0000","last_clean":"2024-09-10T08:06:04.731922+0000","last_became_active":"2024-09-10T08:05:25.485099+0000","last_became_peered":"2024-09-10T08:05:25.485099+0000","last_unstale":"2024-09-10T08:06:04.731922+0000","last_undegraded":"2024-09-10T08:06:04.731922+0000","last_fullsized":"2024-09-10T08:06:04.731922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:24:48.474323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'3","reported_seq":60,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:36.285093+0000","last_change":"2024-09-10T08:05:22.459963+0000","last_active":"2024-09-10T08:06:36.285093+0000","last_peered":"2024-09-10T08:06:36.285093+0000","last_clean":"2024-09-10T08:06:36.285093+0000","last_became_active":"2024-09-10T08:05:22.459497+0000","last_became_peered":"2024-09-10T08:05:22.459497+0000","last_unstale":"2024-09-10T08:06:36.285093+0000","last_undegraded":"2024-09-10T08:06:36.285093+0000","last_fullsized":"2024-09-10T08:06:36.285093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:29.887980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731560+0000","last_change":"2024-09-10T08:05:35.534570+0000","last_active":"2024-09-10T08:06:04.731560+0000","last_peered":"2024-09-10T08:06:04.731560+0000","last_clean":"2024-09-10T08:06:04.731560+0000","last_became_active":"2024-09-10T08:05:35.534195+0000","last_became_peered":"2024-09-10T08:05:35.534195+0000","last_unstale":"2024-09-10T08:06:04.731560+0000","last_undegraded":"2024-09-10T08:06:04.731560+0000","last_fullsized":"2024-09-10T08:06:04.731560+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:17.393491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731947+0000","last_change":"2024-09-10T08:05:25.488551+0000","last_active":"2024-09-10T08:06:04.731947+0000","last_peered":"2024-09-10T08:06:04.731947+0000","last_clean":"2024-09-10T08:06:04.731947+0000","last_became_active":"2024-09-10T08:05:25.488086+0000","last_became_peered":"2024-09-10T08:05:25.488086+0000","last_unstale":"2024-09-10T08:06:04.731947+0000","last_undegraded":"2024-09-10T08:06:04.731947+0000","last_fullsized":"2024-09-10T08:06:04.731947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:38.044721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"93'3","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.790500+0000","last_change":"2024-09-10T08:05:22.462269+0000","last_active":"2024-09-10T08:06:19.790500+0000","last_peered":"2024-09-10T08:06:19.790500+0000","last_clean":"2024-09-10T08:06:19.790500+0000","last_became_active":"2024-09-10T08:05:22.461924+0000","last_became_peered":"2024-09-10T08:05:22.461924+0000","last_unstale":"2024-09-10T08:06:19.790500+0000","last_undegraded":"2024-09-10T08:06:19.790500+0000","last_fullsized":"2024-09-10T08:06:19.790500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:54.996777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729289+0000","last_change":"2024-09-10T08:05:35.531596+0000","last_active":"2024-09-10T08:06:04.729289+0000","last_peered":"2024-09-10T08:06:04.729289+0000","last_clean":"2024-09-10T08:06:04.729289+0000","last_became_active":"2024-09-10T08:05:35.530179+0000","last_became_peered":"2024-09-10T08:05:35.530179+0000","last_unstale":"2024-09-10T08:06:04.729289+0000","last_undegraded":"2024-09-10T08:06:04.729289+0000","last_fullsized":"2024-09-10T08:06:04.729289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:32.056426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729989+0000","last_change":"2024-09-10T08:05:25.503504+0000","last_active":"2024-09-10T08:06:04.729989+0000","last_peered":"2024-09-10T08:06:04.729989+0000","last_clean":"2024-09-10T08:06:04.729989+0000","last_became_active":"2024-09-10T08:05:25.503105+0000","last_became_peered":"2024-09-10T08:05:25.503105+0000","last_unstale":"2024-09-10T08:06:04.729989+0000","last_undegraded":"2024-09-10T08:06:04.729989+0000","last_fullsized":"2024-09-10T08:06:04.729989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:40.885654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"93'6","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.794132+0000","last_change":"2024-09-10T08:05:22.462271+0000","last_active":"2024-09-10T08:06:20.794132+0000","last_peered":"2024-09-10T08:06:20.794132+0000","last_clean":"2024-09-10T08:06:20.794132+0000","last_became_active":"2024-09-10T08:05:22.461195+0000","last_became_peered":"2024-09-10T08:05:22.461195+0000","last_unstale":"2024-09-10T08:06:20.794132+0000","last_undegraded":"2024-09-10T08:06:20.794132+0000","last_fullsized":"2024-09-10T08:06:20.794132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:20.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.726535+0000","last_change":"2024-09-10T08:05:35.533218+0000","last_active":"2024-09-10T08:06:04.726535+0000","last_peered":"2024-09-10T08:06:04.726535+0000","last_clean":"2024-09-10T08:06:04.726535+0000","last_became_active":"2024-09-10T08:05:35.532706+0000","last_became_peered":"2024-09-10T08:05:35.532706+0000","last_unstale":"2024-09-10T08:06:04.726535+0000","last_undegraded":"2024-09-10T08:06:04.726535+0000","last_fullsized":"2024-09-10T08:06:04.726535+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:22.150563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.727795+0000","last_change":"2024-09-10T08:05:25.478978+0000","last_active":"2024-09-10T08:06:04.727795+0000","last_peered":"2024-09-10T08:06:04.727795+0000","last_clean":"2024-09-10T08:06:04.727795+0000","last_became_active":"2024-09-10T08:05:25.478649+0000","last_became_peered":"2024-09-10T08:05:25.478649+0000","last_unstale":"2024-09-10T08:06:04.727795+0000","last_undegraded":"2024-09-10T08:06:04.727795+0000","last_fullsized":"2024-09-10T08:06:04.727795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:17.493446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'29","reported_seq":129,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:40.292071+0000","last_change":"2024-09-10T08:06:04.740972+0000","last_active":"2024-09-10T08:06:40.292071+0000","last_peered":"2024-09-10T08:06:40.292071+0000","last_clean":"2024-09-10T08:06:40.292071+0000","last_became_active":"2024-09-10T08:06:04.731200+0000","last_became_peered":"2024-09-10T08:06:04.731200+0000","last_unstale":"2024-09-10T08:06:40.292071+0000","last_undegraded":"2024-09-10T08:06:40.292071+0000","last_fullsized":"2024-09-10T08:06:40.292071+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:17.169708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57258,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":78,"num_write":30,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3703,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730227+0000","last_change":"2024-09-10T08:05:35.532594+0000","last_active":"2024-09-10T08:06:04.730227+0000","last_peered":"2024-09-10T08:06:04.730227+0000","last_clean":"2024-09-10T08:06:04.730227+0000","last_became_active":"2024-09-10T08:05:35.532132+0000","last_became_peered":"2024-09-10T08:05:35.532132+0000","last_unstale":"2024-09-10T08:06:04.730227+0000","last_undegraded":"2024-09-10T08:06:04.730227+0000","last_fullsized":"2024-09-10T08:06:04.730227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:15.348648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728585+0000","last_change":"2024-09-10T08:05:25.485297+0000","last_active":"2024-09-10T08:06:04.728585+0000","last_peered":"2024-09-10T08:06:04.728585+0000","last_clean":"2024-09-10T08:06:04.728585+0000","last_became_active":"2024-09-10T08:05:25.484883+0000","last_became_peered":"2024-09-10T08:05:25.484883+0000","last_unstale":"2024-09-10T08:06:04.728585+0000","last_undegraded":"2024-09-10T08:06:04.728585+0000","last_fullsized":"2024-09-10T08:06:04.728585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:37.052435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"93'4","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.789580+0000","last_change":"2024-09-10T08:05:22.568993+0000","last_active":"2024-09-10T08:06:19.789580+0000","last_peered":"2024-09-10T08:06:19.789580+0000","last_clean":"2024-09-10T08:06:19.789580+0000","last_became_active":"2024-09-10T08:05:22.568564+0000","last_became_peered":"2024-09-10T08:05:22.568564+0000","last_unstale":"2024-09-10T08:06:19.789580+0000","last_undegraded":"2024-09-10T08:06:19.789580+0000","last_fullsized":"2024-09-10T08:06:19.789580+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:23.767174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729038+0000","last_change":"2024-09-10T08:05:35.524025+0000","last_active":"2024-09-10T08:06:04.729038+0000","last_peered":"2024-09-10T08:06:04.729038+0000","last_clean":"2024-09-10T08:06:04.729038+0000","last_became_active":"2024-09-10T08:05:35.523628+0000","last_became_peered":"2024-09-10T08:05:35.523628+0000","last_unstale":"2024-09-10T08:06:04.729038+0000","last_undegraded":"2024-09-10T08:06:04.729038+0000","last_fullsized":"2024-09-10T08:06:04.729038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:48.593779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733218+0000","last_change":"2024-09-10T08:05:25.496721+0000","last_active":"2024-09-10T08:06:04.733218+0000","last_peered":"2024-09-10T08:06:04.733218+0000","last_clean":"2024-09-10T08:06:04.733218+0000","last_became_active":"2024-09-10T08:05:25.495058+0000","last_became_peered":"2024-09-10T08:05:25.495058+0000","last_unstale":"2024-09-10T08:06:04.733218+0000","last_undegraded":"2024-09-10T08:06:04.733218+0000","last_fullsized":"2024-09-10T08:06:04.733218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:49.832511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"93'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.790645+0000","last_change":"2024-09-10T08:05:22.460487+0000","last_active":"2024-09-10T08:06:19.790645+0000","last_peered":"2024-09-10T08:06:19.790645+0000","last_clean":"2024-09-10T08:06:19.790645+0000","last_became_active":"2024-09-10T08:05:22.460158+0000","last_became_peered":"2024-09-10T08:05:22.460158+0000","last_unstale":"2024-09-10T08:06:19.790645+0000","last_undegraded":"2024-09-10T08:06:19.790645+0000","last_fullsized":"2024-09-10T08:06:19.790645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:53.285513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729478+0000","last_change":"2024-09-10T08:05:35.531399+0000","last_active":"2024-09-10T08:06:04.729478+0000","last_peered":"2024-09-10T08:06:04.729478+0000","last_clean":"2024-09-10T08:06:04.729478+0000","last_became_active":"2024-09-10T08:05:35.530320+0000","last_became_peered":"2024-09-10T08:05:35.530320+0000","last_unstale":"2024-09-10T08:06:04.729478+0000","last_undegraded":"2024-09-10T08:06:04.729478+0000","last_fullsized":"2024-09-10T08:06:04.729478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:19.814046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729028+0000","last_change":"2024-09-10T08:05:25.488208+0000","last_active":"2024-09-10T08:06:04.729028+0000","last_peered":"2024-09-10T08:06:04.729028+0000","last_clean":"2024-09-10T08:06:04.729028+0000","last_became_active":"2024-09-10T08:05:25.487893+0000","last_became_peered":"2024-09-10T08:05:25.487893+0000","last_unstale":"2024-09-10T08:06:04.729028+0000","last_undegraded":"2024-09-10T08:06:04.729028+0000","last_fullsized":"2024-09-10T08:06:04.729028+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:10.999416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"93'3","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:39.289458+0000","last_change":"2024-09-10T08:05:22.461321+0000","last_active":"2024-09-10T08:06:39.289458+0000","last_peered":"2024-09-10T08:06:39.289458+0000","last_clean":"2024-09-10T08:06:39.289458+0000","last_became_active":"2024-09-10T08:05:22.460950+0000","last_became_peered":"2024-09-10T08:05:22.460950+0000","last_unstale":"2024-09-10T08:06:39.289458+0000","last_undegraded":"2024-09-10T08:06:39.289458+0000","last_fullsized":"2024-09-10T08:06:39.289458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:55.647273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733452+0000","last_change":"2024-09-10T08:05:35.530578+0000","last_active":"2024-09-10T08:06:04.733452+0000","last_peered":"2024-09-10T08:06:04.733452+0000","last_clean":"2024-09-10T08:06:04.733452+0000","last_became_active":"2024-09-10T08:05:35.530162+0000","last_became_peered":"2024-09-10T08:05:35.530162+0000","last_unstale":"2024-09-10T08:06:04.733452+0000","last_undegraded":"2024-09-10T08:06:04.733452+0000","last_fullsized":"2024-09-10T08:06:04.733452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:19.290973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729879+0000","last_change":"2024-09-10T08:05:35.532685+0000","last_active":"2024-09-10T08:06:04.729879+0000","last_peered":"2024-09-10T08:06:04.729879+0000","last_clean":"2024-09-10T08:06:04.729879+0000","last_became_active":"2024-09-10T08:05:35.532257+0000","last_became_peered":"2024-09-10T08:05:35.532257+0000","last_unstale":"2024-09-10T08:06:04.729879+0000","last_undegraded":"2024-09-10T08:06:04.729879+0000","last_fullsized":"2024-09-10T08:06:04.729879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:07.412435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730714+0000","last_change":"2024-09-10T08:05:25.494472+0000","last_active":"2024-09-10T08:06:04.730714+0000","last_peered":"2024-09-10T08:06:04.730714+0000","last_clean":"2024-09-10T08:06:04.730714+0000","last_became_active":"2024-09-10T08:05:25.493928+0000","last_became_peered":"2024-09-10T08:05:25.493928+0000","last_unstale":"2024-09-10T08:06:04.730714+0000","last_undegraded":"2024-09-10T08:06:04.730714+0000","last_fullsized":"2024-09-10T08:06:04.730714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:00.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"93'1","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725770+0000","last_change":"2024-09-10T08:05:22.460585+0000","last_active":"2024-09-10T08:06:04.725770+0000","last_peered":"2024-09-10T08:06:04.725770+0000","last_clean":"2024-09-10T08:06:04.725770+0000","last_became_active":"2024-09-10T08:05:22.460259+0000","last_became_peered":"2024-09-10T08:05:22.460259+0000","last_unstale":"2024-09-10T08:06:04.725770+0000","last_undegraded":"2024-09-10T08:06:04.725770+0000","last_fullsized":"2024-09-10T08:06:04.725770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:45.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730310+0000","last_change":"2024-09-10T08:05:35.524000+0000","last_active":"2024-09-10T08:06:04.730310+0000","last_peered":"2024-09-10T08:06:04.730310+0000","last_clean":"2024-09-10T08:06:04.730310+0000","last_became_active":"2024-09-10T08:05:35.523647+0000","last_became_peered":"2024-09-10T08:05:35.523647+0000","last_unstale":"2024-09-10T08:06:04.730310+0000","last_undegraded":"2024-09-10T08:06:04.730310+0000","last_fullsized":"2024-09-10T08:06:04.730310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:10:54.664865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.734307+0000","last_change":"2024-09-10T08:05:25.499701+0000","last_active":"2024-09-10T08:06:04.734307+0000","last_peered":"2024-09-10T08:06:04.734307+0000","last_clean":"2024-09-10T08:06:04.734307+0000","last_became_active":"2024-09-10T08:05:25.496890+0000","last_became_peered":"2024-09-10T08:05:25.496890+0000","last_unstale":"2024-09-10T08:06:04.734307+0000","last_undegraded":"2024-09-10T08:06:04.734307+0000","last_fullsized":"2024-09-10T08:06:04.734307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:58.166365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"93'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.795335+0000","last_change":"2024-09-10T08:05:22.460880+0000","last_active":"2024-09-10T08:06:20.795335+0000","last_peered":"2024-09-10T08:06:20.795335+0000","last_clean":"2024-09-10T08:06:20.795335+0000","last_became_active":"2024-09-10T08:05:22.459932+0000","last_became_peered":"2024-09-10T08:05:22.459932+0000","last_unstale":"2024-09-10T08:06:20.795335+0000","last_undegraded":"2024-09-10T08:06:20.795335+0000","last_fullsized":"2024-09-10T08:06:20.795335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:55.826526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"93'3","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.794861+0000","last_change":"2024-09-10T08:05:22.460578+0000","last_active":"2024-09-10T08:06:20.794861+0000","last_peered":"2024-09-10T08:06:20.794861+0000","last_clean":"2024-09-10T08:06:20.794861+0000","last_became_active":"2024-09-10T08:05:22.460239+0000","last_became_peered":"2024-09-10T08:05:22.460239+0000","last_unstale":"2024-09-10T08:06:20.794861+0000","last_undegraded":"2024-09-10T08:06:20.794861+0000","last_fullsized":"2024-09-10T08:06:20.794861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:07.606243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.726058+0000","last_change":"2024-09-10T08:05:25.489846+0000","last_active":"2024-09-10T08:06:04.726058+0000","last_peered":"2024-09-10T08:06:04.726058+0000","last_clean":"2024-09-10T08:06:04.726058+0000","last_became_active":"2024-09-10T08:05:25.489326+0000","last_became_peered":"2024-09-10T08:05:25.489326+0000","last_unstale":"2024-09-10T08:06:04.726058+0000","last_undegraded":"2024-09-10T08:06:04.726058+0000","last_fullsized":"2024-09-10T08:06:04.726058+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:43.458468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731382+0000","last_change":"2024-09-10T08:05:35.530522+0000","last_active":"2024-09-10T08:06:04.731382+0000","last_peered":"2024-09-10T08:06:04.731382+0000","last_clean":"2024-09-10T08:06:04.731382+0000","last_became_active":"2024-09-10T08:05:35.530099+0000","last_became_peered":"2024-09-10T08:05:35.530099+0000","last_unstale":"2024-09-10T08:06:04.731382+0000","last_undegraded":"2024-09-10T08:06:04.731382+0000","last_fullsized":"2024-09-10T08:06:04.731382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:58.469119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'6","reported_seq":64,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:36.285877+0000","last_change":"2024-09-10T08:05:22.461391+0000","last_active":"2024-09-10T08:06:36.285877+0000","last_peered":"2024-09-10T08:06:36.285877+0000","last_clean":"2024-09-10T08:06:36.285877+0000","last_became_active":"2024-09-10T08:05:22.461042+0000","last_became_peered":"2024-09-10T08:05:22.461042+0000","last_unstale":"2024-09-10T08:06:36.285877+0000","last_undegraded":"2024-09-10T08:06:36.285877+0000","last_fullsized":"2024-09-10T08:06:36.285877+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:12.635710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728879+0000","last_change":"2024-09-10T08:05:25.487378+0000","last_active":"2024-09-10T08:06:04.728879+0000","last_peered":"2024-09-10T08:06:04.728879+0000","last_clean":"2024-09-10T08:06:04.728879+0000","last_became_active":"2024-09-10T08:05:25.487055+0000","last_became_peered":"2024-09-10T08:05:25.487055+0000","last_unstale":"2024-09-10T08:06:04.728879+0000","last_undegraded":"2024-09-10T08:06:04.728879+0000","last_fullsized":"2024-09-10T08:06:04.728879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:59.522014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733027+0000","last_change":"2024-09-10T08:05:35.522765+0000","last_active":"2024-09-10T08:06:04.733027+0000","last_peered":"2024-09-10T08:06:04.733027+0000","last_clean":"2024-09-10T08:06:04.733027+0000","last_became_active":"2024-09-10T08:05:35.522359+0000","last_became_peered":"2024-09-10T08:05:35.522359+0000","last_unstale":"2024-09-10T08:06:04.733027+0000","last_undegraded":"2024-09-10T08:06:04.733027+0000","last_fullsized":"2024-09-10T08:06:04.733027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:15.280867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728630+0000","last_change":"2024-09-10T08:05:25.475388+0000","last_active":"2024-09-10T08:06:04.728630+0000","last_peered":"2024-09-10T08:06:04.728630+0000","last_clean":"2024-09-10T08:06:04.728630+0000","last_became_active":"2024-09-10T08:05:25.474331+0000","last_became_peered":"2024-09-10T08:05:25.474331+0000","last_unstale":"2024-09-10T08:06:04.728630+0000","last_undegraded":"2024-09-10T08:06:04.728630+0000","last_fullsized":"2024-09-10T08:06:04.728630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:58.713148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731811+0000","last_change":"2024-09-10T08:05:25.481548+0000","last_active":"2024-09-10T08:06:04.731811+0000","last_peered":"2024-09-10T08:06:04.731811+0000","last_clean":"2024-09-10T08:06:04.731811+0000","last_became_active":"2024-09-10T08:05:25.481152+0000","last_became_peered":"2024-09-10T08:05:25.481152+0000","last_unstale":"2024-09-10T08:06:04.731811+0000","last_undegraded":"2024-09-10T08:06:04.731811+0000","last_fullsized":"2024-09-10T08:06:04.731811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:14:37.938977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731491+0000","last_change":"2024-09-10T08:05:25.491585+0000","last_active":"2024-09-10T08:06:04.731491+0000","last_peered":"2024-09-10T08:06:04.731491+0000","last_clean":"2024-09-10T08:06:04.731491+0000","last_became_active":"2024-09-10T08:05:25.491224+0000","last_became_peered":"2024-09-10T08:05:25.491224+0000","last_unstale":"2024-09-10T08:06:04.731491+0000","last_undegraded":"2024-09-10T08:06:04.731491+0000","last_fullsized":"2024-09-10T08:06:04.731491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:56.066808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729798+0000","last_change":"2024-09-10T08:05:25.499578+0000","last_active":"2024-09-10T08:06:04.729798+0000","last_peered":"2024-09-10T08:06:04.729798+0000","last_clean":"2024-09-10T08:06:04.729798+0000","last_became_active":"2024-09-10T08:05:25.498445+0000","last_became_peered":"2024-09-10T08:05:25.498445+0000","last_unstale":"2024-09-10T08:06:04.729798+0000","last_undegraded":"2024-09-10T08:06:04.729798+0000","last_fullsized":"2024-09-10T08:06:04.729798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:58.961699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.735764+0000","last_change":"2024-09-10T08:06:04.735764+0000","last_active":"2024-09-10T08:06:04.735764+0000","last_peered":"2024-09-10T08:06:04.735764+0000","last_clean":"2024-09-10T08:06:04.735764+0000","last_became_active":"2024-09-10T08:06:04.735366+0000","last_became_peered":"2024-09-10T08:06:04.735366+0000","last_unstale":"2024-09-10T08:06:04.735764+0000","last_undegraded":"2024-09-10T08:06:04.735764+0000","last_fullsized":"2024-09-10T08:06:04.735764+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:03.757890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729806+0000","last_change":"2024-09-10T08:05:25.475872+0000","last_active":"2024-09-10T08:06:04.729806+0000","last_peered":"2024-09-10T08:06:04.729806+0000","last_clean":"2024-09-10T08:06:04.729806+0000","last_became_active":"2024-09-10T08:05:25.475549+0000","last_became_peered":"2024-09-10T08:05:25.475549+0000","last_unstale":"2024-09-10T08:06:04.729806+0000","last_undegraded":"2024-09-10T08:06:04.729806+0000","last_fullsized":"2024-09-10T08:06:04.729806+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:37.575267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733499+0000","last_change":"2024-09-10T08:05:25.474210+0000","last_active":"2024-09-10T08:06:04.733499+0000","last_peered":"2024-09-10T08:06:04.733499+0000","last_clean":"2024-09-10T08:06:04.733499+0000","last_became_active":"2024-09-10T08:05:25.473882+0000","last_became_peered":"2024-09-10T08:05:25.473882+0000","last_unstale":"2024-09-10T08:06:04.733499+0000","last_undegraded":"2024-09-10T08:06:04.733499+0000","last_fullsized":"2024-09-10T08:06:04.733499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:05.164370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733896+0000","last_change":"2024-09-10T08:05:25.490406+0000","last_active":"2024-09-10T08:06:04.733896+0000","last_peered":"2024-09-10T08:06:04.733896+0000","last_clean":"2024-09-10T08:06:04.733896+0000","last_became_active":"2024-09-10T08:05:25.490061+0000","last_became_peered":"2024-09-10T08:05:25.490061+0000","last_unstale":"2024-09-10T08:06:04.733896+0000","last_undegraded":"2024-09-10T08:06:04.733896+0000","last_fullsized":"2024-09-10T08:06:04.733896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:12.988229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725565+0000","last_change":"2024-09-10T08:05:25.504049+0000","last_active":"2024-09-10T08:06:04.725565+0000","last_peered":"2024-09-10T08:06:04.725565+0000","last_clean":"2024-09-10T08:06:04.725565+0000","last_became_active":"2024-09-10T08:05:25.503770+0000","last_became_peered":"2024-09-10T08:05:25.503770+0000","last_unstale":"2024-09-10T08:06:04.725565+0000","last_undegraded":"2024-09-10T08:06:04.725565+0000","last_fullsized":"2024-09-10T08:06:04.725565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:52.850945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733622+0000","last_change":"2024-09-10T08:05:25.491682+0000","last_active":"2024-09-10T08:06:04.733622+0000","last_peered":"2024-09-10T08:06:04.733622+0000","last_clean":"2024-09-10T08:06:04.733622+0000","last_became_active":"2024-09-10T08:05:25.491296+0000","last_became_peered":"2024-09-10T08:05:25.491296+0000","last_unstale":"2024-09-10T08:06:04.733622+0000","last_undegraded":"2024-09-10T08:06:04.733622+0000","last_fullsized":"2024-09-10T08:06:04.733622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:30.075842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728374+0000","last_change":"2024-09-10T08:05:25.489648+0000","last_active":"2024-09-10T08:06:04.728374+0000","last_peered":"2024-09-10T08:06:04.728374+0000","last_clean":"2024-09-10T08:06:04.728374+0000","last_became_active":"2024-09-10T08:05:25.489267+0000","last_became_peered":"2024-09-10T08:05:25.489267+0000","last_unstale":"2024-09-10T08:06:04.728374+0000","last_undegraded":"2024-09-10T08:06:04.728374+0000","last_fullsized":"2024-09-10T08:06:04.728374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:21.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733968+0000","last_change":"2024-09-10T08:05:25.497862+0000","last_active":"2024-09-10T08:06:04.733968+0000","last_peered":"2024-09-10T08:06:04.733968+0000","last_clean":"2024-09-10T08:06:04.733968+0000","last_became_active":"2024-09-10T08:05:25.497504+0000","last_became_peered":"2024-09-10T08:05:25.497504+0000","last_unstale":"2024-09-10T08:06:04.733968+0000","last_undegraded":"2024-09-10T08:06:04.733968+0000","last_fullsized":"2024-09-10T08:06:04.733968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:40.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729400+0000","last_change":"2024-09-10T08:05:25.485924+0000","last_active":"2024-09-10T08:06:04.729400+0000","last_peered":"2024-09-10T08:06:04.729400+0000","last_clean":"2024-09-10T08:06:04.729400+0000","last_became_active":"2024-09-10T08:05:25.485545+0000","last_became_peered":"2024-09-10T08:05:25.485545+0000","last_unstale":"2024-09-10T08:06:04.729400+0000","last_undegraded":"2024-09-10T08:06:04.729400+0000","last_fullsized":"2024-09-10T08:06:04.729400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:33:20.233720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733183+0000","last_change":"2024-09-10T08:05:25.491911+0000","last_active":"2024-09-10T08:06:04.733183+0000","last_peered":"2024-09-10T08:06:04.733183+0000","last_clean":"2024-09-10T08:06:04.733183+0000","last_became_active":"2024-09-10T08:05:25.491330+0000","last_became_peered":"2024-09-10T08:05:25.491330+0000","last_unstale":"2024-09-10T08:06:04.733183+0000","last_undegraded":"2024-09-10T08:06:04.733183+0000","last_fullsized":"2024-09-10T08:06:04.733183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:06.043331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731733+0000","last_change":"2024-09-10T08:05:25.492220+0000","last_active":"2024-09-10T08:06:04.731733+0000","last_peered":"2024-09-10T08:06:04.731733+0000","last_clean":"2024-09-10T08:06:04.731733+0000","last_became_active":"2024-09-10T08:05:25.491473+0000","last_became_peered":"2024-09-10T08:05:25.491473+0000","last_unstale":"2024-09-10T08:06:04.731733+0000","last_undegraded":"2024-09-10T08:06:04.731733+0000","last_fullsized":"2024-09-10T08:06:04.731733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:13:30.278434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730671+0000","last_change":"2024-09-10T08:05:25.488927+0000","last_active":"2024-09-10T08:06:04.730671+0000","last_peered":"2024-09-10T08:06:04.730671+0000","last_clean":"2024-09-10T08:06:04.730671+0000","last_became_active":"2024-09-10T08:05:25.488550+0000","last_became_peered":"2024-09-10T08:05:25.488550+0000","last_unstale":"2024-09-10T08:06:04.730671+0000","last_undegraded":"2024-09-10T08:06:04.730671+0000","last_fullsized":"2024-09-10T08:06:04.730671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:44.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728218+0000","last_change":"2024-09-10T08:05:25.490125+0000","last_active":"2024-09-10T08:06:04.728218+0000","last_peered":"2024-09-10T08:06:04.728218+0000","last_clean":"2024-09-10T08:06:04.728218+0000","last_became_active":"2024-09-10T08:05:25.487805+0000","last_became_peered":"2024-09-10T08:05:25.487805+0000","last_unstale":"2024-09-10T08:06:04.728218+0000","last_undegraded":"2024-09-10T08:06:04.728218+0000","last_fullsized":"2024-09-10T08:06:04.728218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:33.479009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731155+0000","last_change":"2024-09-10T08:05:25.500346+0000","last_active":"2024-09-10T08:06:04.731155+0000","last_peered":"2024-09-10T08:06:04.731155+0000","last_clean":"2024-09-10T08:06:04.731155+0000","last_became_active":"2024-09-10T08:05:25.498289+0000","last_became_peered":"2024-09-10T08:05:25.498289+0000","last_unstale":"2024-09-10T08:06:04.731155+0000","last_undegraded":"2024-09-10T08:06:04.731155+0000","last_fullsized":"2024-09-10T08:06:04.731155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:12.054926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731420+0000","last_change":"2024-09-10T08:05:25.490001+0000","last_active":"2024-09-10T08:06:04.731420+0000","last_peered":"2024-09-10T08:06:04.731420+0000","last_clean":"2024-09-10T08:06:04.731420+0000","last_became_active":"2024-09-10T08:05:25.489706+0000","last_became_peered":"2024-09-10T08:05:25.489706+0000","last_unstale":"2024-09-10T08:06:04.731420+0000","last_undegraded":"2024-09-10T08:06:04.731420+0000","last_fullsized":"2024-09-10T08:06:04.731420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:38.822140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732652+0000","last_change":"2024-09-10T08:05:25.493874+0000","last_active":"2024-09-10T08:06:04.732652+0000","last_peered":"2024-09-10T08:06:04.732652+0000","last_clean":"2024-09-10T08:06:04.732652+0000","last_became_active":"2024-09-10T08:05:25.489911+0000","last_became_peered":"2024-09-10T08:05:25.489911+0000","last_unstale":"2024-09-10T08:06:04.732652+0000","last_undegraded":"2024-09-10T08:06:04.732652+0000","last_fullsized":"2024-09-10T08:06:04.732652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:46.509657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729372+0000","last_change":"2024-09-10T08:05:25.490277+0000","last_active":"2024-09-10T08:06:04.729372+0000","last_peered":"2024-09-10T08:06:04.729372+0000","last_clean":"2024-09-10T08:06:04.729372+0000","last_became_active":"2024-09-10T08:05:25.486863+0000","last_became_peered":"2024-09-10T08:05:25.486863+0000","last_unstale":"2024-09-10T08:06:04.729372+0000","last_undegraded":"2024-09-10T08:06:04.729372+0000","last_fullsized":"2024-09-10T08:06:04.729372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:21.353830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732812+0000","last_change":"2024-09-10T08:05:25.491028+0000","last_active":"2024-09-10T08:06:04.732812+0000","last_peered":"2024-09-10T08:06:04.732812+0000","last_clean":"2024-09-10T08:06:04.732812+0000","last_became_active":"2024-09-10T08:05:25.490452+0000","last_became_peered":"2024-09-10T08:05:25.490452+0000","last_unstale":"2024-09-10T08:06:04.732812+0000","last_undegraded":"2024-09-10T08:06:04.732812+0000","last_fullsized":"2024-09-10T08:06:04.732812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:53.059077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.726402+0000","last_change":"2024-09-10T08:05:25.489991+0000","last_active":"2024-09-10T08:06:04.726402+0000","last_peered":"2024-09-10T08:06:04.726402+0000","last_clean":"2024-09-10T08:06:04.726402+0000","last_became_active":"2024-09-10T08:05:25.489662+0000","last_became_peered":"2024-09-10T08:05:25.489662+0000","last_unstale":"2024-09-10T08:06:04.726402+0000","last_undegraded":"2024-09-10T08:06:04.726402+0000","last_fullsized":"2024-09-10T08:06:04.726402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:47.940712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728907+0000","last_change":"2024-09-10T08:05:25.475752+0000","last_active":"2024-09-10T08:06:04.728907+0000","last_peered":"2024-09-10T08:06:04.728907+0000","last_clean":"2024-09-10T08:06:04.728907+0000","last_became_active":"2024-09-10T08:05:25.475395+0000","last_became_peered":"2024-09-10T08:05:25.475395+0000","last_unstale":"2024-09-10T08:06:04.728907+0000","last_undegraded":"2024-09-10T08:06:04.728907+0000","last_fullsized":"2024-09-10T08:06:04.728907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.240154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.734209+0000","last_change":"2024-09-10T08:05:25.503061+0000","last_active":"2024-09-10T08:06:04.734209+0000","last_peered":"2024-09-10T08:06:04.734209+0000","last_clean":"2024-09-10T08:06:04.734209+0000","last_became_active":"2024-09-10T08:05:25.501333+0000","last_became_peered":"2024-09-10T08:05:25.501333+0000","last_unstale":"2024-09-10T08:06:04.734209+0000","last_undegraded":"2024-09-10T08:06:04.734209+0000","last_fullsized":"2024-09-10T08:06:04.734209+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:38.254777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732218+0000","last_change":"2024-09-10T08:05:25.482628+0000","last_active":"2024-09-10T08:06:04.732218+0000","last_peered":"2024-09-10T08:06:04.732218+0000","last_clean":"2024-09-10T08:06:04.732218+0000","last_became_active":"2024-09-10T08:05:25.482214+0000","last_became_peered":"2024-09-10T08:05:25.482214+0000","last_unstale":"2024-09-10T08:06:04.732218+0000","last_undegraded":"2024-09-10T08:06:04.732218+0000","last_fullsized":"2024-09-10T08:06:04.732218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:49.045496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725832+0000","last_change":"2024-09-10T08:05:25.491457+0000","last_active":"2024-09-10T08:06:04.725832+0000","last_peered":"2024-09-10T08:06:04.725832+0000","last_clean":"2024-09-10T08:06:04.725832+0000","last_became_active":"2024-09-10T08:05:25.491162+0000","last_became_peered":"2024-09-10T08:05:25.491162+0000","last_unstale":"2024-09-10T08:06:04.725832+0000","last_undegraded":"2024-09-10T08:06:04.725832+0000","last_fullsized":"2024-09-10T08:06:04.725832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:14.389752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.727533+0000","last_change":"2024-09-10T08:05:25.487874+0000","last_active":"2024-09-10T08:06:04.727533+0000","last_peered":"2024-09-10T08:06:04.727533+0000","last_clean":"2024-09-10T08:06:04.727533+0000","last_became_active":"2024-09-10T08:05:25.487497+0000","last_became_peered":"2024-09-10T08:05:25.487497+0000","last_unstale":"2024-09-10T08:06:04.727533+0000","last_undegraded":"2024-09-10T08:06:04.727533+0000","last_fullsized":"2024-09-10T08:06:04.727533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:26.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729279+0000","last_change":"2024-09-10T08:05:25.490018+0000","last_active":"2024-09-10T08:06:04.729279+0000","last_peered":"2024-09-10T08:06:04.729279+0000","last_clean":"2024-09-10T08:06:04.729279+0000","last_became_active":"2024-09-10T08:05:25.484951+0000","last_became_peered":"2024-09-10T08:05:25.484951+0000","last_unstale":"2024-09-10T08:06:04.729279+0000","last_undegraded":"2024-09-10T08:06:04.729279+0000","last_fullsized":"2024-09-10T08:06:04.729279+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:07.516154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731689+0000","last_change":"2024-09-10T08:05:25.501581+0000","last_active":"2024-09-10T08:06:04.731689+0000","last_peered":"2024-09-10T08:06:04.731689+0000","last_clean":"2024-09-10T08:06:04.731689+0000","last_became_active":"2024-09-10T08:05:25.499690+0000","last_became_peered":"2024-09-10T08:05:25.499690+0000","last_unstale":"2024-09-10T08:06:04.731689+0000","last_undegraded":"2024-09-10T08:06:04.731689+0000","last_fullsized":"2024-09-10T08:06:04.731689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:14.569791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729987+0000","last_change":"2024-09-10T08:05:25.487945+0000","last_active":"2024-09-10T08:06:04.729987+0000","last_peered":"2024-09-10T08:06:04.729987+0000","last_clean":"2024-09-10T08:06:04.729987+0000","last_became_active":"2024-09-10T08:05:25.487655+0000","last_became_peered":"2024-09-10T08:05:25.487655+0000","last_unstale":"2024-09-10T08:06:04.729987+0000","last_undegraded":"2024-09-10T08:06:04.729987+0000","last_fullsized":"2024-09-10T08:06:04.729987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:37.746037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730664+0000","last_change":"2024-09-10T08:05:25.489348+0000","last_active":"2024-09-10T08:06:04.730664+0000","last_peered":"2024-09-10T08:06:04.730664+0000","last_clean":"2024-09-10T08:06:04.730664+0000","last_became_active":"2024-09-10T08:05:25.489004+0000","last_became_peered":"2024-09-10T08:05:25.489004+0000","last_unstale":"2024-09-10T08:06:04.730664+0000","last_undegraded":"2024-09-10T08:06:04.730664+0000","last_fullsized":"2024-09-10T08:06:04.730664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:54.554719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61261,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":131,"num_read_kb":132,"num_write":158,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3725,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":171008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214976,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514525,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":528985,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743458,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1167360,"data_stored":1177585,"data_compressed":62832,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":529199,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135917,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":529097,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364848,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":1125030,"data_compressed":62832,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528374,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":524375,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":1120618,"data_compressed":62832,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953537,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":573508,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":524569,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":526028,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":581543,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":525763,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:06:44.314 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:06:44.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:44 smithi002 ceph-mon[30563]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:06:44.694 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:44.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:44 smithi155 ceph-mon[35405]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:06:44.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:44 smithi107 ceph-mon[35048]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:06:45.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.340+0000 7f0d4e000640 1 -- 172.21.15.2:0/357271359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d481079c0 msgr2=0x7f0d48107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:45.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.340+0000 7f0d4e000640 1 --2- 172.21.15.2:0/357271359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d481079c0 0x7f0d48107da0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009930 tx=0x7f0d3c02a420 comp rx=0 tx=0).stop 2024-09-10T08:06:45.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.340+0000 7f0d4e000640 1 -- 172.21.15.2:0/357271359 shutdown_connections 2024-09-10T08:06:45.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.340+0000 7f0d4e000640 1 --2- 172.21.15.2:0/357271359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d48102360 0x7f0d4810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:45.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.340+0000 7f0d4e000640 1 --2- 172.21.15.2:0/357271359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d481019c0 0x7f0d48101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:45.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.340+0000 7f0d4e000640 1 --2- 172.21.15.2:0/357271359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d481079c0 0x7f0d48107da0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:45.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.340+0000 7f0d4e000640 1 -- 172.21.15.2:0/357271359 >> 172.21.15.2:0/357271359 conn(0x7f0d480fd740 msgr2=0x7f0d480ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:45.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.340+0000 7f0d4e000640 1 -- 172.21.15.2:0/357271359 shutdown_connections 2024-09-10T08:06:45.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.340+0000 7f0d4e000640 1 -- 172.21.15.2:0/357271359 wait complete. 2024-09-10T08:06:45.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.341+0000 7f0d4e000640 1 Processor -- start 2024-09-10T08:06:45.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.341+0000 7f0d4e000640 1 -- start start 2024-09-10T08:06:45.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.341+0000 7f0d4e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d481019c0 0x7f0d481982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:45.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.341+0000 7f0d4e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d48102360 0x7f0d48198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:45.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d4e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d481079c0 0x7f0d4819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:45.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d4e000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48111dc0 con 0x7f0d481019c0 2024-09-10T08:06:45.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d4e000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48111c40 con 0x7f0d481079c0 2024-09-10T08:06:45.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d4e000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48111f40 con 0x7f0d48102360 2024-09-10T08:06:45.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d4cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d481019c0 0x7f0d481982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:45.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d4cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d481019c0 0x7f0d481982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60588/0 (socket says 172.21.15.2:60588) 2024-09-10T08:06:45.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d4cc00640 1 -- 172.21.15.2:0/1316942134 learned_addr learned my addr 172.21.15.2:0/1316942134 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:45.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d47e00640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d48102360 0x7f0d48198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:45.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d4d600640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d481079c0 0x7f0d4819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:45.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d4cc00640 1 -- 172.21.15.2:0/1316942134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d48102360 msgr2=0x7f0d48198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:45.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d4cc00640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d48102360 0x7f0d48198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:45.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d4cc00640 1 -- 172.21.15.2:0/1316942134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d481079c0 msgr2=0x7f0d4819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:45.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d4cc00640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d481079c0 0x7f0d4819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:45.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.342+0000 7f0d4cc00640 1 -- 172.21.15.2:0/1316942134 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4819d270 con 0x7f0d481019c0 2024-09-10T08:06:45.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.343+0000 7f0d4cc00640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d481019c0 0x7f0d481982c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009230 tx=0x7f0d3c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:45.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.343+0000 7f0d45600640 1 -- 172.21.15.2:0/1316942134 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c03f030 con 0x7f0d481019c0 2024-09-10T08:06:45.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.343+0000 7f0d45600640 1 -- 172.21.15.2:0/1316942134 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c004440 con 0x7f0d481019c0 2024-09-10T08:06:45.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.343+0000 7f0d45600640 1 -- 172.21.15.2:0/1316942134 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c035070 con 0x7f0d481019c0 2024-09-10T08:06:45.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.343+0000 7f0d4e000640 1 -- 172.21.15.2:0/1316942134 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4819d500 con 0x7f0d481019c0 2024-09-10T08:06:45.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.343+0000 7f0d4e000640 1 -- 172.21.15.2:0/1316942134 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4819d990 con 0x7f0d481019c0 2024-09-10T08:06:45.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.344+0000 7f0d4e000640 1 -- 172.21.15.2:0/1316942134 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d48101e20 con 0x7f0d481019c0 2024-09-10T08:06:45.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.346+0000 7f0d45600640 1 -- 172.21.15.2:0/1316942134 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0d3c042030 con 0x7f0d481019c0 2024-09-10T08:06:45.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.347+0000 7f0d45600640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d300a2c90 0x7f0d300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:45.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.347+0000 7f0d45600640 1 -- 172.21.15.2:0/1316942134 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f0d3c0ef7d0 con 0x7f0d481019c0 2024-09-10T08:06:45.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.348+0000 7f0d47e00640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d300a2c90 0x7f0d300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:45.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.348+0000 7f0d47e00640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d300a2c90 0x7f0d300a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0d480ffd10 tx=0x7f0d34005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:45.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.351+0000 7f0d45600640 1 -- 172.21.15.2:0/1316942134 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d3c0bbbc0 con 0x7f0d481019c0 2024-09-10T08:06:45.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.541+0000 7f0d4e000640 1 -- 172.21.15.2:0/1316942134 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0d48062630 con 0x7f0d300a2c90 2024-09-10T08:06:45.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.550+0000 7f0d45600640 1 -- 172.21.15.2:0/1316942134 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340601 (secure 0 0 0) 0x7f0d48062630 con 0x7f0d300a2c90 2024-09-10T08:06:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:06:45.555 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-10T08:06:45.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.555+0000 7f0d4e000640 1 -- 172.21.15.2:0/1316942134 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d300a2c90 msgr2=0x7f0d300a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:45.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.555+0000 7f0d4e000640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d300a2c90 0x7f0d300a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0d480ffd10 tx=0x7f0d34005f90 comp rx=0 tx=0).stop 2024-09-10T08:06:45.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.555+0000 7f0d4e000640 1 -- 172.21.15.2:0/1316942134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d481019c0 msgr2=0x7f0d481982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:45.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.555+0000 7f0d4e000640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d481019c0 0x7f0d481982c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009230 tx=0x7f0d3c004040 comp rx=0 tx=0).stop 2024-09-10T08:06:45.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.555+0000 7f0d4e000640 1 -- 172.21.15.2:0/1316942134 shutdown_connections 2024-09-10T08:06:45.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.555+0000 7f0d4e000640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d481079c0 0x7f0d4819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:45.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.555+0000 7f0d4e000640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d300a2c90 0x7f0d300a5150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:45.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.555+0000 7f0d4e000640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d48102360 0x7f0d48198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:45.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.555+0000 7f0d4e000640 1 --2- 172.21.15.2:0/1316942134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d481019c0 0x7f0d481982c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:45.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.555+0000 7f0d4e000640 1 -- 172.21.15.2:0/1316942134 >> 172.21.15.2:0/1316942134 conn(0x7f0d480fd740 msgr2=0x7f0d480fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:45.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.555+0000 7f0d4e000640 1 -- 172.21.15.2:0/1316942134 shutdown_connections 2024-09-10T08:06:45.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:45.555+0000 7f0d4e000640 1 -- 172.21.15.2:0/1316942134 wait complete. 2024-09-10T08:06:45.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:45 smithi002 ceph-mon[30563]: pgmap v343: 129 pgs: 129 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-10T08:06:45.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:45 smithi155 ceph-mon[35405]: pgmap v343: 129 pgs: 129 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-10T08:06:45.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:45 smithi107 ceph-mon[35048]: pgmap v343: 129 pgs: 129 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-10T08:06:46.457 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-10T08:06:43.659084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651863,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":272,"num_read_kb":246,"num_write":341,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1184461,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361724,"kb_used_data":12476,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580100,"statfs":{"total":1151940427776,"available":1151570022400,"internally_reserved":0,"allocated":12775424,"data_stored":8279075,"data_compressed":426582,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33188,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":69,"num_write":20,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003315"},"pg_stats":[{"pgid":"2.1f","version":"93'9","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.794582+0000","last_change":"2024-09-10T08:05:22.462062+0000","last_active":"2024-09-10T08:06:20.794582+0000","last_peered":"2024-09-10T08:06:20.794582+0000","last_clean":"2024-09-10T08:06:20.794582+0000","last_became_active":"2024-09-10T08:05:22.460973+0000","last_became_peered":"2024-09-10T08:05:22.460973+0000","last_unstale":"2024-09-10T08:06:20.794582+0000","last_undegraded":"2024-09-10T08:06:20.794582+0000","last_fullsized":"2024-09-10T08:06:20.794582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:01.357446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.735468+0000","last_change":"2024-09-10T08:05:25.482873+0000","last_active":"2024-09-10T08:06:04.735468+0000","last_peered":"2024-09-10T08:06:04.735468+0000","last_clean":"2024-09-10T08:06:04.735468+0000","last_became_active":"2024-09-10T08:05:25.482559+0000","last_became_peered":"2024-09-10T08:05:25.482559+0000","last_unstale":"2024-09-10T08:06:04.735468+0000","last_undegraded":"2024-09-10T08:06:04.735468+0000","last_fullsized":"2024-09-10T08:06:04.735468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:10:17.366160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.736356+0000","last_change":"2024-09-10T08:06:04.736356+0000","last_active":"2024-09-10T08:06:04.736356+0000","last_peered":"2024-09-10T08:06:04.736356+0000","last_clean":"2024-09-10T08:06:04.736356+0000","last_became_active":"2024-09-10T08:06:04.735859+0000","last_became_peered":"2024-09-10T08:06:04.735859+0000","last_unstale":"2024-09-10T08:06:04.736356+0000","last_undegraded":"2024-09-10T08:06:04.736356+0000","last_fullsized":"2024-09-10T08:06:04.736356+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:56.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"93'2","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.735769+0000","last_change":"2024-09-10T08:05:22.569504+0000","last_active":"2024-09-10T08:06:04.735769+0000","last_peered":"2024-09-10T08:06:04.735769+0000","last_clean":"2024-09-10T08:06:04.735769+0000","last_became_active":"2024-09-10T08:05:22.569164+0000","last_became_peered":"2024-09-10T08:05:22.569164+0000","last_unstale":"2024-09-10T08:06:04.735769+0000","last_undegraded":"2024-09-10T08:06:04.735769+0000","last_fullsized":"2024-09-10T08:06:04.735769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:41.628963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733116+0000","last_change":"2024-09-10T08:05:25.493017+0000","last_active":"2024-09-10T08:06:04.733116+0000","last_peered":"2024-09-10T08:06:04.733116+0000","last_clean":"2024-09-10T08:06:04.733116+0000","last_became_active":"2024-09-10T08:05:25.491370+0000","last_became_peered":"2024-09-10T08:05:25.491370+0000","last_unstale":"2024-09-10T08:06:04.733116+0000","last_undegraded":"2024-09-10T08:06:04.733116+0000","last_fullsized":"2024-09-10T08:06:04.733116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:41.616549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728519+0000","last_change":"2024-09-10T08:05:35.523751+0000","last_active":"2024-09-10T08:06:04.728519+0000","last_peered":"2024-09-10T08:06:04.728519+0000","last_clean":"2024-09-10T08:06:04.728519+0000","last_became_active":"2024-09-10T08:05:35.523372+0000","last_became_peered":"2024-09-10T08:05:35.523372+0000","last_unstale":"2024-09-10T08:06:04.728519+0000","last_undegraded":"2024-09-10T08:06:04.728519+0000","last_fullsized":"2024-09-10T08:06:04.728519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:20.971596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'4","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.793322+0000","last_change":"2024-09-10T08:05:22.460590+0000","last_active":"2024-09-10T08:06:20.793322+0000","last_peered":"2024-09-10T08:06:20.793322+0000","last_clean":"2024-09-10T08:06:20.793322+0000","last_became_active":"2024-09-10T08:05:22.460238+0000","last_became_peered":"2024-09-10T08:05:22.460238+0000","last_unstale":"2024-09-10T08:06:20.793322+0000","last_undegraded":"2024-09-10T08:06:20.793322+0000","last_fullsized":"2024-09-10T08:06:20.793322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:18.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728483+0000","last_change":"2024-09-10T08:05:25.487392+0000","last_active":"2024-09-10T08:06:04.728483+0000","last_peered":"2024-09-10T08:06:04.728483+0000","last_clean":"2024-09-10T08:06:04.728483+0000","last_became_active":"2024-09-10T08:05:25.487054+0000","last_became_peered":"2024-09-10T08:05:25.487054+0000","last_unstale":"2024-09-10T08:06:04.728483+0000","last_undegraded":"2024-09-10T08:06:04.728483+0000","last_fullsized":"2024-09-10T08:06:04.728483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:16.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.735778+0000","last_change":"2024-09-10T08:06:04.735778+0000","last_active":"2024-09-10T08:06:04.735778+0000","last_peered":"2024-09-10T08:06:04.735778+0000","last_clean":"2024-09-10T08:06:04.735778+0000","last_became_active":"2024-09-10T08:06:04.735208+0000","last_became_peered":"2024-09-10T08:06:04.735208+0000","last_unstale":"2024-09-10T08:06:04.735778+0000","last_undegraded":"2024-09-10T08:06:04.735778+0000","last_fullsized":"2024-09-10T08:06:04.735778+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:36.982246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729026+0000","last_change":"2024-09-10T08:05:22.458323+0000","last_active":"2024-09-10T08:06:04.729026+0000","last_peered":"2024-09-10T08:06:04.729026+0000","last_clean":"2024-09-10T08:06:04.729026+0000","last_became_active":"2024-09-10T08:05:22.457965+0000","last_became_peered":"2024-09-10T08:05:22.457965+0000","last_unstale":"2024-09-10T08:06:04.729026+0000","last_undegraded":"2024-09-10T08:06:04.729026+0000","last_fullsized":"2024-09-10T08:06:04.729026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:15.021305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.726424+0000","last_change":"2024-09-10T08:05:25.489883+0000","last_active":"2024-09-10T08:06:04.726424+0000","last_peered":"2024-09-10T08:06:04.726424+0000","last_clean":"2024-09-10T08:06:04.726424+0000","last_became_active":"2024-09-10T08:05:25.489482+0000","last_became_peered":"2024-09-10T08:05:25.489482+0000","last_unstale":"2024-09-10T08:06:04.726424+0000","last_undegraded":"2024-09-10T08:06:04.726424+0000","last_fullsized":"2024-09-10T08:06:04.726424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:52.080628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728497+0000","last_change":"2024-09-10T08:05:35.531458+0000","last_active":"2024-09-10T08:06:04.728497+0000","last_peered":"2024-09-10T08:06:04.728497+0000","last_clean":"2024-09-10T08:06:04.728497+0000","last_became_active":"2024-09-10T08:05:35.531072+0000","last_became_peered":"2024-09-10T08:05:35.531072+0000","last_unstale":"2024-09-10T08:06:04.728497+0000","last_undegraded":"2024-09-10T08:06:04.728497+0000","last_fullsized":"2024-09-10T08:06:04.728497+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:04.792664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731989+0000","last_change":"2024-09-10T08:05:25.486865+0000","last_active":"2024-09-10T08:06:04.731989+0000","last_peered":"2024-09-10T08:06:04.731989+0000","last_clean":"2024-09-10T08:06:04.731989+0000","last_became_active":"2024-09-10T08:05:25.486531+0000","last_became_peered":"2024-09-10T08:05:25.486531+0000","last_unstale":"2024-09-10T08:06:04.731989+0000","last_undegraded":"2024-09-10T08:06:04.731989+0000","last_fullsized":"2024-09-10T08:06:04.731989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:26.854682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"93'4","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.791172+0000","last_change":"2024-09-10T08:06:04.743303+0000","last_active":"2024-09-10T08:06:19.791172+0000","last_peered":"2024-09-10T08:06:19.791172+0000","last_clean":"2024-09-10T08:06:19.791172+0000","last_became_active":"2024-09-10T08:06:04.733294+0000","last_became_peered":"2024-09-10T08:06:04.733294+0000","last_unstale":"2024-09-10T08:06:19.791172+0000","last_undegraded":"2024-09-10T08:06:19.791172+0000","last_fullsized":"2024-09-10T08:06:19.791172+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:18.542658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728881+0000","last_change":"2024-09-10T08:05:35.642083+0000","last_active":"2024-09-10T08:06:04.728881+0000","last_peered":"2024-09-10T08:06:04.728881+0000","last_clean":"2024-09-10T08:06:04.728881+0000","last_became_active":"2024-09-10T08:05:35.641214+0000","last_became_peered":"2024-09-10T08:05:35.641214+0000","last_unstale":"2024-09-10T08:06:04.728881+0000","last_undegraded":"2024-09-10T08:06:04.728881+0000","last_fullsized":"2024-09-10T08:06:04.728881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:00:21.731769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733707+0000","last_change":"2024-09-10T08:05:25.501993+0000","last_active":"2024-09-10T08:06:04.733707+0000","last_peered":"2024-09-10T08:06:04.733707+0000","last_clean":"2024-09-10T08:06:04.733707+0000","last_became_active":"2024-09-10T08:05:25.499912+0000","last_became_peered":"2024-09-10T08:05:25.499912+0000","last_unstale":"2024-09-10T08:06:04.733707+0000","last_undegraded":"2024-09-10T08:06:04.733707+0000","last_fullsized":"2024-09-10T08:06:04.733707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:06.780067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"93'1","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732696+0000","last_change":"2024-09-10T08:05:22.461786+0000","last_active":"2024-09-10T08:06:04.732696+0000","last_peered":"2024-09-10T08:06:04.732696+0000","last_clean":"2024-09-10T08:06:04.732696+0000","last_became_active":"2024-09-10T08:05:22.460821+0000","last_became_peered":"2024-09-10T08:05:22.460821+0000","last_unstale":"2024-09-10T08:06:04.732696+0000","last_undegraded":"2024-09-10T08:06:04.732696+0000","last_fullsized":"2024-09-10T08:06:04.732696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:25.083538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.735635+0000","last_change":"2024-09-10T08:06:04.734470+0000","last_active":"2024-09-10T08:06:04.735635+0000","last_peered":"2024-09-10T08:06:04.735635+0000","last_clean":"2024-09-10T08:06:04.735635+0000","last_became_active":"2024-09-10T08:06:04.734040+0000","last_became_peered":"2024-09-10T08:06:04.734040+0000","last_unstale":"2024-09-10T08:06:04.735635+0000","last_undegraded":"2024-09-10T08:06:04.735635+0000","last_fullsized":"2024-09-10T08:06:04.735635+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:36.312144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728729+0000","last_change":"2024-09-10T08:05:25.492711+0000","last_active":"2024-09-10T08:06:04.728729+0000","last_peered":"2024-09-10T08:06:04.728729+0000","last_clean":"2024-09-10T08:06:04.728729+0000","last_became_active":"2024-09-10T08:05:25.492336+0000","last_became_peered":"2024-09-10T08:05:25.492336+0000","last_unstale":"2024-09-10T08:06:04.728729+0000","last_undegraded":"2024-09-10T08:06:04.728729+0000","last_fullsized":"2024-09-10T08:06:04.728729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:04.168884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"93'3","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.790001+0000","last_change":"2024-09-10T08:05:22.461535+0000","last_active":"2024-09-10T08:06:19.790001+0000","last_peered":"2024-09-10T08:06:19.790001+0000","last_clean":"2024-09-10T08:06:19.790001+0000","last_became_active":"2024-09-10T08:05:22.461208+0000","last_became_peered":"2024-09-10T08:05:22.461208+0000","last_unstale":"2024-09-10T08:06:19.790001+0000","last_undegraded":"2024-09-10T08:06:19.790001+0000","last_fullsized":"2024-09-10T08:06:19.790001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:26.009242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731554+0000","last_change":"2024-09-10T08:05:35.535442+0000","last_active":"2024-09-10T08:06:04.731554+0000","last_peered":"2024-09-10T08:06:04.731554+0000","last_clean":"2024-09-10T08:06:04.731554+0000","last_became_active":"2024-09-10T08:05:35.535058+0000","last_became_peered":"2024-09-10T08:05:35.535058+0000","last_unstale":"2024-09-10T08:06:04.731554+0000","last_undegraded":"2024-09-10T08:06:04.731554+0000","last_fullsized":"2024-09-10T08:06:04.731554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:39.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.727471+0000","last_change":"2024-09-10T08:05:25.483440+0000","last_active":"2024-09-10T08:06:04.727471+0000","last_peered":"2024-09-10T08:06:04.727471+0000","last_clean":"2024-09-10T08:06:04.727471+0000","last_became_active":"2024-09-10T08:05:25.483141+0000","last_became_peered":"2024-09-10T08:05:25.483141+0000","last_unstale":"2024-09-10T08:06:04.727471+0000","last_undegraded":"2024-09-10T08:06:04.727471+0000","last_fullsized":"2024-09-10T08:06:04.727471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:00.953188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"93'2","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.791516+0000","last_change":"2024-09-10T08:05:22.568141+0000","last_active":"2024-09-10T08:06:19.791516+0000","last_peered":"2024-09-10T08:06:19.791516+0000","last_clean":"2024-09-10T08:06:19.791516+0000","last_became_active":"2024-09-10T08:05:22.567857+0000","last_became_peered":"2024-09-10T08:05:22.567857+0000","last_unstale":"2024-09-10T08:06:19.791516+0000","last_undegraded":"2024-09-10T08:06:19.791516+0000","last_fullsized":"2024-09-10T08:06:19.791516+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:22.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729420+0000","last_change":"2024-09-10T08:05:35.641993+0000","last_active":"2024-09-10T08:06:04.729420+0000","last_peered":"2024-09-10T08:06:04.729420+0000","last_clean":"2024-09-10T08:06:04.729420+0000","last_became_active":"2024-09-10T08:05:35.641376+0000","last_became_peered":"2024-09-10T08:05:35.641376+0000","last_unstale":"2024-09-10T08:06:04.729420+0000","last_undegraded":"2024-09-10T08:06:04.729420+0000","last_fullsized":"2024-09-10T08:06:04.729420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:16:34.038924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.737560+0000","last_change":"2024-09-10T08:06:04.737560+0000","last_active":"2024-09-10T08:06:04.737560+0000","last_peered":"2024-09-10T08:06:04.737560+0000","last_clean":"2024-09-10T08:06:04.737560+0000","last_became_active":"2024-09-10T08:06:04.736656+0000","last_became_peered":"2024-09-10T08:06:04.736656+0000","last_unstale":"2024-09-10T08:06:04.737560+0000","last_undegraded":"2024-09-10T08:06:04.737560+0000","last_fullsized":"2024-09-10T08:06:04.737560+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:35.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"93'4","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.792092+0000","last_change":"2024-09-10T08:05:22.567077+0000","last_active":"2024-09-10T08:06:19.792092+0000","last_peered":"2024-09-10T08:06:19.792092+0000","last_clean":"2024-09-10T08:06:19.792092+0000","last_became_active":"2024-09-10T08:05:22.566757+0000","last_became_peered":"2024-09-10T08:05:22.566757+0000","last_unstale":"2024-09-10T08:06:19.792092+0000","last_undegraded":"2024-09-10T08:06:19.792092+0000","last_fullsized":"2024-09-10T08:06:19.792092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:06.881510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731567+0000","last_change":"2024-09-10T08:05:35.519539+0000","last_active":"2024-09-10T08:06:04.731567+0000","last_peered":"2024-09-10T08:06:04.731567+0000","last_clean":"2024-09-10T08:06:04.731567+0000","last_became_active":"2024-09-10T08:05:35.519044+0000","last_became_peered":"2024-09-10T08:05:35.519044+0000","last_unstale":"2024-09-10T08:06:04.731567+0000","last_undegraded":"2024-09-10T08:06:04.731567+0000","last_fullsized":"2024-09-10T08:06:04.731567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:10.546949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733989+0000","last_change":"2024-09-10T08:05:25.489753+0000","last_active":"2024-09-10T08:06:04.733989+0000","last_peered":"2024-09-10T08:06:04.733989+0000","last_clean":"2024-09-10T08:06:04.733989+0000","last_became_active":"2024-09-10T08:05:25.489455+0000","last_became_peered":"2024-09-10T08:05:25.489455+0000","last_unstale":"2024-09-10T08:06:04.733989+0000","last_undegraded":"2024-09-10T08:06:04.733989+0000","last_fullsized":"2024-09-10T08:06:04.733989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:23:53.667875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'8","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:35.504609+0000","last_change":"2024-09-10T08:06:04.741199+0000","last_active":"2024-09-10T08:06:35.504609+0000","last_peered":"2024-09-10T08:06:35.504609+0000","last_clean":"2024-09-10T08:06:35.504609+0000","last_became_active":"2024-09-10T08:06:04.738076+0000","last_became_peered":"2024-09-10T08:06:04.738076+0000","last_unstale":"2024-09-10T08:06:35.504609+0000","last_undegraded":"2024-09-10T08:06:35.504609+0000","last_fullsized":"2024-09-10T08:06:35.504609+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:45.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733304+0000","last_change":"2024-09-10T08:05:35.642358+0000","last_active":"2024-09-10T08:06:04.733304+0000","last_peered":"2024-09-10T08:06:04.733304+0000","last_clean":"2024-09-10T08:06:04.733304+0000","last_became_active":"2024-09-10T08:05:35.641408+0000","last_became_peered":"2024-09-10T08:05:35.641408+0000","last_unstale":"2024-09-10T08:06:04.733304+0000","last_undegraded":"2024-09-10T08:06:04.733304+0000","last_fullsized":"2024-09-10T08:06:04.733304+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:42.015139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732023+0000","last_change":"2024-09-10T08:05:25.486310+0000","last_active":"2024-09-10T08:06:04.732023+0000","last_peered":"2024-09-10T08:06:04.732023+0000","last_clean":"2024-09-10T08:06:04.732023+0000","last_became_active":"2024-09-10T08:05:25.485915+0000","last_became_peered":"2024-09-10T08:05:25.485915+0000","last_unstale":"2024-09-10T08:06:04.732023+0000","last_undegraded":"2024-09-10T08:06:04.732023+0000","last_fullsized":"2024-09-10T08:06:04.732023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:56.064361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"93'1","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.791751+0000","last_change":"2024-09-10T08:05:22.570595+0000","last_active":"2024-09-10T08:06:19.791751+0000","last_peered":"2024-09-10T08:06:19.791751+0000","last_clean":"2024-09-10T08:06:19.791751+0000","last_became_active":"2024-09-10T08:05:22.570219+0000","last_became_peered":"2024-09-10T08:05:22.570219+0000","last_unstale":"2024-09-10T08:06:19.791751+0000","last_undegraded":"2024-09-10T08:06:19.791751+0000","last_fullsized":"2024-09-10T08:06:19.791751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:03.230325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729161+0000","last_change":"2024-09-10T08:05:35.530472+0000","last_active":"2024-09-10T08:06:04.729161+0000","last_peered":"2024-09-10T08:06:04.729161+0000","last_clean":"2024-09-10T08:06:04.729161+0000","last_became_active":"2024-09-10T08:05:35.530066+0000","last_became_peered":"2024-09-10T08:05:35.530066+0000","last_unstale":"2024-09-10T08:06:04.729161+0000","last_undegraded":"2024-09-10T08:06:04.729161+0000","last_fullsized":"2024-09-10T08:06:04.729161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:58.915767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725911+0000","last_change":"2024-09-10T08:05:25.500524+0000","last_active":"2024-09-10T08:06:04.725911+0000","last_peered":"2024-09-10T08:06:04.725911+0000","last_clean":"2024-09-10T08:06:04.725911+0000","last_became_active":"2024-09-10T08:05:25.500186+0000","last_became_peered":"2024-09-10T08:05:25.500186+0000","last_unstale":"2024-09-10T08:06:04.725911+0000","last_undegraded":"2024-09-10T08:06:04.725911+0000","last_fullsized":"2024-09-10T08:06:04.725911+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:51.936222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731033+0000","last_change":"2024-09-10T08:05:22.569326+0000","last_active":"2024-09-10T08:06:04.731033+0000","last_peered":"2024-09-10T08:06:04.731033+0000","last_clean":"2024-09-10T08:06:04.731033+0000","last_became_active":"2024-09-10T08:05:22.568895+0000","last_became_peered":"2024-09-10T08:05:22.568895+0000","last_unstale":"2024-09-10T08:06:04.731033+0000","last_undegraded":"2024-09-10T08:06:04.731033+0000","last_fullsized":"2024-09-10T08:06:04.731033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:47.212073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728741+0000","last_change":"2024-09-10T08:05:35.641600+0000","last_active":"2024-09-10T08:06:04.728741+0000","last_peered":"2024-09-10T08:06:04.728741+0000","last_clean":"2024-09-10T08:06:04.728741+0000","last_became_active":"2024-09-10T08:05:35.641205+0000","last_became_peered":"2024-09-10T08:05:35.641205+0000","last_unstale":"2024-09-10T08:06:04.728741+0000","last_undegraded":"2024-09-10T08:06:04.728741+0000","last_fullsized":"2024-09-10T08:06:04.728741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:40.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725888+0000","last_change":"2024-09-10T08:05:25.495035+0000","last_active":"2024-09-10T08:06:04.725888+0000","last_peered":"2024-09-10T08:06:04.725888+0000","last_clean":"2024-09-10T08:06:04.725888+0000","last_became_active":"2024-09-10T08:05:25.494733+0000","last_became_peered":"2024-09-10T08:05:25.494733+0000","last_unstale":"2024-09-10T08:06:04.725888+0000","last_undegraded":"2024-09-10T08:06:04.725888+0000","last_fullsized":"2024-09-10T08:06:04.725888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.875867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"93'1","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732432+0000","last_change":"2024-09-10T08:05:22.458727+0000","last_active":"2024-09-10T08:06:04.732432+0000","last_peered":"2024-09-10T08:06:04.732432+0000","last_clean":"2024-09-10T08:06:04.732432+0000","last_became_active":"2024-09-10T08:05:22.458397+0000","last_became_peered":"2024-09-10T08:05:22.458397+0000","last_unstale":"2024-09-10T08:06:04.732432+0000","last_undegraded":"2024-09-10T08:06:04.732432+0000","last_fullsized":"2024-09-10T08:06:04.732432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:07.875559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731455+0000","last_change":"2024-09-10T08:05:35.534192+0000","last_active":"2024-09-10T08:06:04.731455+0000","last_peered":"2024-09-10T08:06:04.731455+0000","last_clean":"2024-09-10T08:06:04.731455+0000","last_became_active":"2024-09-10T08:05:35.530450+0000","last_became_peered":"2024-09-10T08:05:35.530450+0000","last_unstale":"2024-09-10T08:06:04.731455+0000","last_undegraded":"2024-09-10T08:06:04.731455+0000","last_fullsized":"2024-09-10T08:06:04.731455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:10.039144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725579+0000","last_change":"2024-09-10T08:05:25.482252+0000","last_active":"2024-09-10T08:06:04.725579+0000","last_peered":"2024-09-10T08:06:04.725579+0000","last_clean":"2024-09-10T08:06:04.725579+0000","last_became_active":"2024-09-10T08:05:25.481901+0000","last_became_peered":"2024-09-10T08:05:25.481901+0000","last_unstale":"2024-09-10T08:06:04.725579+0000","last_undegraded":"2024-09-10T08:06:04.725579+0000","last_fullsized":"2024-09-10T08:06:04.725579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:17.063340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'4","reported_seq":64,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:41.292000+0000","last_change":"2024-09-10T08:05:22.569199+0000","last_active":"2024-09-10T08:06:41.292000+0000","last_peered":"2024-09-10T08:06:41.292000+0000","last_clean":"2024-09-10T08:06:41.292000+0000","last_became_active":"2024-09-10T08:05:22.568787+0000","last_became_peered":"2024-09-10T08:05:22.568787+0000","last_unstale":"2024-09-10T08:06:41.292000+0000","last_undegraded":"2024-09-10T08:06:41.292000+0000","last_fullsized":"2024-09-10T08:06:41.292000+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:59.610130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.734072+0000","last_change":"2024-09-10T08:05:35.642446+0000","last_active":"2024-09-10T08:06:04.734072+0000","last_peered":"2024-09-10T08:06:04.734072+0000","last_clean":"2024-09-10T08:06:04.734072+0000","last_became_active":"2024-09-10T08:05:35.641543+0000","last_became_peered":"2024-09-10T08:05:35.641543+0000","last_unstale":"2024-09-10T08:06:04.734072+0000","last_undegraded":"2024-09-10T08:06:04.734072+0000","last_fullsized":"2024-09-10T08:06:04.734072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:33.658406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730139+0000","last_change":"2024-09-10T08:05:25.487598+0000","last_active":"2024-09-10T08:06:04.730139+0000","last_peered":"2024-09-10T08:06:04.730139+0000","last_clean":"2024-09-10T08:06:04.730139+0000","last_became_active":"2024-09-10T08:05:25.486879+0000","last_became_peered":"2024-09-10T08:05:25.486879+0000","last_unstale":"2024-09-10T08:06:04.730139+0000","last_undegraded":"2024-09-10T08:06:04.730139+0000","last_fullsized":"2024-09-10T08:06:04.730139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:20.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'3","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.792775+0000","last_change":"2024-09-10T08:05:22.463416+0000","last_active":"2024-09-10T08:06:20.792775+0000","last_peered":"2024-09-10T08:06:20.792775+0000","last_clean":"2024-09-10T08:06:20.792775+0000","last_became_active":"2024-09-10T08:05:22.463004+0000","last_became_peered":"2024-09-10T08:05:22.463004+0000","last_unstale":"2024-09-10T08:06:20.792775+0000","last_undegraded":"2024-09-10T08:06:20.792775+0000","last_fullsized":"2024-09-10T08:06:20.792775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:56.257812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.726319+0000","last_change":"2024-09-10T08:05:35.642059+0000","last_active":"2024-09-10T08:06:04.726319+0000","last_peered":"2024-09-10T08:06:04.726319+0000","last_clean":"2024-09-10T08:06:04.726319+0000","last_became_active":"2024-09-10T08:05:35.641540+0000","last_became_peered":"2024-09-10T08:05:35.641540+0000","last_unstale":"2024-09-10T08:06:04.726319+0000","last_undegraded":"2024-09-10T08:06:04.726319+0000","last_fullsized":"2024-09-10T08:06:04.726319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:28.557584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.734223+0000","last_change":"2024-09-10T08:05:25.496111+0000","last_active":"2024-09-10T08:06:04.734223+0000","last_peered":"2024-09-10T08:06:04.734223+0000","last_clean":"2024-09-10T08:06:04.734223+0000","last_became_active":"2024-09-10T08:05:25.495757+0000","last_became_peered":"2024-09-10T08:05:25.495757+0000","last_unstale":"2024-09-10T08:06:04.734223+0000","last_undegraded":"2024-09-10T08:06:04.734223+0000","last_fullsized":"2024-09-10T08:06:04.734223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:41.078205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'3","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:22.260153+0000","last_change":"2024-09-10T08:05:22.460069+0000","last_active":"2024-09-10T08:06:22.260153+0000","last_peered":"2024-09-10T08:06:22.260153+0000","last_clean":"2024-09-10T08:06:22.260153+0000","last_became_active":"2024-09-10T08:05:22.459557+0000","last_became_peered":"2024-09-10T08:05:22.459557+0000","last_unstale":"2024-09-10T08:06:22.260153+0000","last_undegraded":"2024-09-10T08:06:22.260153+0000","last_fullsized":"2024-09-10T08:06:22.260153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:49.058507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732140+0000","last_change":"2024-09-10T08:04:08.188262+0000","last_active":"2024-09-10T08:06:04.732140+0000","last_peered":"2024-09-10T08:06:04.732140+0000","last_clean":"2024-09-10T08:06:04.732140+0000","last_became_active":"2024-09-10T08:04:08.187691+0000","last_became_peered":"2024-09-10T08:04:08.187691+0000","last_unstale":"2024-09-10T08:06:04.732140+0000","last_undegraded":"2024-09-10T08:06:04.732140+0000","last_fullsized":"2024-09-10T08:06:04.732140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_clean_scrub_stamp":"2024-09-10T08:00:26.235127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:15.358818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732657+0000","last_change":"2024-09-10T08:05:35.528371+0000","last_active":"2024-09-10T08:06:04.732657+0000","last_peered":"2024-09-10T08:06:04.732657+0000","last_clean":"2024-09-10T08:06:04.732657+0000","last_became_active":"2024-09-10T08:05:35.527925+0000","last_became_peered":"2024-09-10T08:05:35.527925+0000","last_unstale":"2024-09-10T08:06:04.732657+0000","last_undegraded":"2024-09-10T08:06:04.732657+0000","last_fullsized":"2024-09-10T08:06:04.732657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:37:00.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732862+0000","last_change":"2024-09-10T08:05:25.490753+0000","last_active":"2024-09-10T08:06:04.732862+0000","last_peered":"2024-09-10T08:06:04.732862+0000","last_clean":"2024-09-10T08:06:04.732862+0000","last_became_active":"2024-09-10T08:05:25.489996+0000","last_became_peered":"2024-09-10T08:05:25.489996+0000","last_unstale":"2024-09-10T08:06:04.732862+0000","last_undegraded":"2024-09-10T08:06:04.732862+0000","last_fullsized":"2024-09-10T08:06:04.732862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:15.637199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'3","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.795368+0000","last_change":"2024-09-10T08:05:22.568696+0000","last_active":"2024-09-10T08:06:20.795368+0000","last_peered":"2024-09-10T08:06:20.795368+0000","last_clean":"2024-09-10T08:06:20.795368+0000","last_became_active":"2024-09-10T08:05:22.568331+0000","last_became_peered":"2024-09-10T08:05:22.568331+0000","last_unstale":"2024-09-10T08:06:20.795368+0000","last_undegraded":"2024-09-10T08:06:20.795368+0000","last_fullsized":"2024-09-10T08:06:20.795368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:21.699397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728181+0000","last_change":"2024-09-10T08:05:35.530804+0000","last_active":"2024-09-10T08:06:04.728181+0000","last_peered":"2024-09-10T08:06:04.728181+0000","last_clean":"2024-09-10T08:06:04.728181+0000","last_became_active":"2024-09-10T08:05:35.530356+0000","last_became_peered":"2024-09-10T08:05:35.530356+0000","last_unstale":"2024-09-10T08:06:04.728181+0000","last_undegraded":"2024-09-10T08:06:04.728181+0000","last_fullsized":"2024-09-10T08:06:04.728181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:59.318761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732403+0000","last_change":"2024-09-10T08:05:25.489546+0000","last_active":"2024-09-10T08:06:04.732403+0000","last_peered":"2024-09-10T08:06:04.732403+0000","last_clean":"2024-09-10T08:06:04.732403+0000","last_became_active":"2024-09-10T08:05:25.489188+0000","last_became_peered":"2024-09-10T08:05:25.489188+0000","last_unstale":"2024-09-10T08:06:04.732403+0000","last_undegraded":"2024-09-10T08:06:04.732403+0000","last_fullsized":"2024-09-10T08:06:04.732403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:26.471681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.726696+0000","last_change":"2024-09-10T08:05:22.463912+0000","last_active":"2024-09-10T08:06:04.726696+0000","last_peered":"2024-09-10T08:06:04.726696+0000","last_clean":"2024-09-10T08:06:04.726696+0000","last_became_active":"2024-09-10T08:05:22.463561+0000","last_became_peered":"2024-09-10T08:05:22.463561+0000","last_unstale":"2024-09-10T08:06:04.726696+0000","last_undegraded":"2024-09-10T08:06:04.726696+0000","last_fullsized":"2024-09-10T08:06:04.726696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:22.407604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729701+0000","last_change":"2024-09-10T08:05:35.528795+0000","last_active":"2024-09-10T08:06:04.729701+0000","last_peered":"2024-09-10T08:06:04.729701+0000","last_clean":"2024-09-10T08:06:04.729701+0000","last_became_active":"2024-09-10T08:05:35.528402+0000","last_became_peered":"2024-09-10T08:05:35.528402+0000","last_unstale":"2024-09-10T08:06:04.729701+0000","last_undegraded":"2024-09-10T08:06:04.729701+0000","last_fullsized":"2024-09-10T08:06:04.729701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:14.606371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732856+0000","last_change":"2024-09-10T08:05:25.484086+0000","last_active":"2024-09-10T08:06:04.732856+0000","last_peered":"2024-09-10T08:06:04.732856+0000","last_clean":"2024-09-10T08:06:04.732856+0000","last_became_active":"2024-09-10T08:05:25.483757+0000","last_became_peered":"2024-09-10T08:05:25.483757+0000","last_unstale":"2024-09-10T08:06:04.732856+0000","last_undegraded":"2024-09-10T08:06:04.732856+0000","last_fullsized":"2024-09-10T08:06:04.732856+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:16:16.362739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733080+0000","last_change":"2024-09-10T08:05:22.462220+0000","last_active":"2024-09-10T08:06:04.733080+0000","last_peered":"2024-09-10T08:06:04.733080+0000","last_clean":"2024-09-10T08:06:04.733080+0000","last_became_active":"2024-09-10T08:05:22.461726+0000","last_became_peered":"2024-09-10T08:05:22.461726+0000","last_unstale":"2024-09-10T08:06:04.733080+0000","last_undegraded":"2024-09-10T08:06:04.733080+0000","last_fullsized":"2024-09-10T08:06:04.733080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:56.381034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.735443+0000","last_change":"2024-09-10T08:06:04.734827+0000","last_active":"2024-09-10T08:06:04.735443+0000","last_peered":"2024-09-10T08:06:04.735443+0000","last_clean":"2024-09-10T08:06:04.735443+0000","last_became_active":"2024-09-10T08:06:04.734402+0000","last_became_peered":"2024-09-10T08:06:04.734402+0000","last_unstale":"2024-09-10T08:06:04.735443+0000","last_undegraded":"2024-09-10T08:06:04.735443+0000","last_fullsized":"2024-09-10T08:06:04.735443+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:24.806538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730162+0000","last_change":"2024-09-10T08:05:25.486100+0000","last_active":"2024-09-10T08:06:04.730162+0000","last_peered":"2024-09-10T08:06:04.730162+0000","last_clean":"2024-09-10T08:06:04.730162+0000","last_became_active":"2024-09-10T08:05:25.485586+0000","last_became_peered":"2024-09-10T08:05:25.485586+0000","last_unstale":"2024-09-10T08:06:04.730162+0000","last_undegraded":"2024-09-10T08:06:04.730162+0000","last_fullsized":"2024-09-10T08:06:04.730162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:33.565054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"93'2","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:22.263114+0000","last_change":"2024-09-10T08:06:04.743408+0000","last_active":"2024-09-10T08:06:22.263114+0000","last_peered":"2024-09-10T08:06:22.263114+0000","last_clean":"2024-09-10T08:06:22.263114+0000","last_became_active":"2024-09-10T08:06:04.732830+0000","last_became_peered":"2024-09-10T08:06:04.732830+0000","last_unstale":"2024-09-10T08:06:22.263114+0000","last_undegraded":"2024-09-10T08:06:22.263114+0000","last_fullsized":"2024-09-10T08:06:22.263114+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:29.381259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725544+0000","last_change":"2024-09-10T08:05:35.533015+0000","last_active":"2024-09-10T08:06:04.725544+0000","last_peered":"2024-09-10T08:06:04.725544+0000","last_clean":"2024-09-10T08:06:04.725544+0000","last_became_active":"2024-09-10T08:05:35.532590+0000","last_became_peered":"2024-09-10T08:05:35.532590+0000","last_unstale":"2024-09-10T08:06:04.725544+0000","last_undegraded":"2024-09-10T08:06:04.725544+0000","last_fullsized":"2024-09-10T08:06:04.725544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:23.984273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725636+0000","last_change":"2024-09-10T08:05:25.489994+0000","last_active":"2024-09-10T08:06:04.725636+0000","last_peered":"2024-09-10T08:06:04.725636+0000","last_clean":"2024-09-10T08:06:04.725636+0000","last_became_active":"2024-09-10T08:05:25.489471+0000","last_became_peered":"2024-09-10T08:05:25.489471+0000","last_unstale":"2024-09-10T08:06:04.725636+0000","last_undegraded":"2024-09-10T08:06:04.725636+0000","last_fullsized":"2024-09-10T08:06:04.725636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:53.866570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"93'15","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.795609+0000","last_change":"2024-09-10T08:05:22.568900+0000","last_active":"2024-09-10T08:06:20.795609+0000","last_peered":"2024-09-10T08:06:20.795609+0000","last_clean":"2024-09-10T08:06:20.795609+0000","last_became_active":"2024-09-10T08:05:22.568542+0000","last_became_peered":"2024-09-10T08:05:22.568542+0000","last_unstale":"2024-09-10T08:06:20.795609+0000","last_undegraded":"2024-09-10T08:06:20.795609+0000","last_fullsized":"2024-09-10T08:06:20.795609+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:58:45.669769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732976+0000","last_change":"2024-09-10T08:05:35.537364+0000","last_active":"2024-09-10T08:06:04.732976+0000","last_peered":"2024-09-10T08:06:04.732976+0000","last_clean":"2024-09-10T08:06:04.732976+0000","last_became_active":"2024-09-10T08:05:35.535958+0000","last_became_peered":"2024-09-10T08:05:35.535958+0000","last_unstale":"2024-09-10T08:06:04.732976+0000","last_undegraded":"2024-09-10T08:06:04.732976+0000","last_fullsized":"2024-09-10T08:06:04.732976+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:51.443798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731922+0000","last_change":"2024-09-10T08:05:25.485427+0000","last_active":"2024-09-10T08:06:04.731922+0000","last_peered":"2024-09-10T08:06:04.731922+0000","last_clean":"2024-09-10T08:06:04.731922+0000","last_became_active":"2024-09-10T08:05:25.485099+0000","last_became_peered":"2024-09-10T08:05:25.485099+0000","last_unstale":"2024-09-10T08:06:04.731922+0000","last_undegraded":"2024-09-10T08:06:04.731922+0000","last_fullsized":"2024-09-10T08:06:04.731922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:24:48.474323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'3","reported_seq":65,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:41.292360+0000","last_change":"2024-09-10T08:05:22.459963+0000","last_active":"2024-09-10T08:06:41.292360+0000","last_peered":"2024-09-10T08:06:41.292360+0000","last_clean":"2024-09-10T08:06:41.292360+0000","last_became_active":"2024-09-10T08:05:22.459497+0000","last_became_peered":"2024-09-10T08:05:22.459497+0000","last_unstale":"2024-09-10T08:06:41.292360+0000","last_undegraded":"2024-09-10T08:06:41.292360+0000","last_fullsized":"2024-09-10T08:06:41.292360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:29.887980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731560+0000","last_change":"2024-09-10T08:05:35.534570+0000","last_active":"2024-09-10T08:06:04.731560+0000","last_peered":"2024-09-10T08:06:04.731560+0000","last_clean":"2024-09-10T08:06:04.731560+0000","last_became_active":"2024-09-10T08:05:35.534195+0000","last_became_peered":"2024-09-10T08:05:35.534195+0000","last_unstale":"2024-09-10T08:06:04.731560+0000","last_undegraded":"2024-09-10T08:06:04.731560+0000","last_fullsized":"2024-09-10T08:06:04.731560+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:17.393491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731947+0000","last_change":"2024-09-10T08:05:25.488551+0000","last_active":"2024-09-10T08:06:04.731947+0000","last_peered":"2024-09-10T08:06:04.731947+0000","last_clean":"2024-09-10T08:06:04.731947+0000","last_became_active":"2024-09-10T08:05:25.488086+0000","last_became_peered":"2024-09-10T08:05:25.488086+0000","last_unstale":"2024-09-10T08:06:04.731947+0000","last_undegraded":"2024-09-10T08:06:04.731947+0000","last_fullsized":"2024-09-10T08:06:04.731947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:38.044721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"93'3","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.790500+0000","last_change":"2024-09-10T08:05:22.462269+0000","last_active":"2024-09-10T08:06:19.790500+0000","last_peered":"2024-09-10T08:06:19.790500+0000","last_clean":"2024-09-10T08:06:19.790500+0000","last_became_active":"2024-09-10T08:05:22.461924+0000","last_became_peered":"2024-09-10T08:05:22.461924+0000","last_unstale":"2024-09-10T08:06:19.790500+0000","last_undegraded":"2024-09-10T08:06:19.790500+0000","last_fullsized":"2024-09-10T08:06:19.790500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:54.996777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729289+0000","last_change":"2024-09-10T08:05:35.531596+0000","last_active":"2024-09-10T08:06:04.729289+0000","last_peered":"2024-09-10T08:06:04.729289+0000","last_clean":"2024-09-10T08:06:04.729289+0000","last_became_active":"2024-09-10T08:05:35.530179+0000","last_became_peered":"2024-09-10T08:05:35.530179+0000","last_unstale":"2024-09-10T08:06:04.729289+0000","last_undegraded":"2024-09-10T08:06:04.729289+0000","last_fullsized":"2024-09-10T08:06:04.729289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:32.056426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729989+0000","last_change":"2024-09-10T08:05:25.503504+0000","last_active":"2024-09-10T08:06:04.729989+0000","last_peered":"2024-09-10T08:06:04.729989+0000","last_clean":"2024-09-10T08:06:04.729989+0000","last_became_active":"2024-09-10T08:05:25.503105+0000","last_became_peered":"2024-09-10T08:05:25.503105+0000","last_unstale":"2024-09-10T08:06:04.729989+0000","last_undegraded":"2024-09-10T08:06:04.729989+0000","last_fullsized":"2024-09-10T08:06:04.729989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:40.885654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"93'6","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.794132+0000","last_change":"2024-09-10T08:05:22.462271+0000","last_active":"2024-09-10T08:06:20.794132+0000","last_peered":"2024-09-10T08:06:20.794132+0000","last_clean":"2024-09-10T08:06:20.794132+0000","last_became_active":"2024-09-10T08:05:22.461195+0000","last_became_peered":"2024-09-10T08:05:22.461195+0000","last_unstale":"2024-09-10T08:06:20.794132+0000","last_undegraded":"2024-09-10T08:06:20.794132+0000","last_fullsized":"2024-09-10T08:06:20.794132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:20.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.726535+0000","last_change":"2024-09-10T08:05:35.533218+0000","last_active":"2024-09-10T08:06:04.726535+0000","last_peered":"2024-09-10T08:06:04.726535+0000","last_clean":"2024-09-10T08:06:04.726535+0000","last_became_active":"2024-09-10T08:05:35.532706+0000","last_became_peered":"2024-09-10T08:05:35.532706+0000","last_unstale":"2024-09-10T08:06:04.726535+0000","last_undegraded":"2024-09-10T08:06:04.726535+0000","last_fullsized":"2024-09-10T08:06:04.726535+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:22.150563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.727795+0000","last_change":"2024-09-10T08:05:25.478978+0000","last_active":"2024-09-10T08:06:04.727795+0000","last_peered":"2024-09-10T08:06:04.727795+0000","last_clean":"2024-09-10T08:06:04.727795+0000","last_became_active":"2024-09-10T08:05:25.478649+0000","last_became_peered":"2024-09-10T08:05:25.478649+0000","last_unstale":"2024-09-10T08:06:04.727795+0000","last_undegraded":"2024-09-10T08:06:04.727795+0000","last_fullsized":"2024-09-10T08:06:04.727795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:17.493446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'29","reported_seq":129,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:40.292071+0000","last_change":"2024-09-10T08:06:04.740972+0000","last_active":"2024-09-10T08:06:40.292071+0000","last_peered":"2024-09-10T08:06:40.292071+0000","last_clean":"2024-09-10T08:06:40.292071+0000","last_became_active":"2024-09-10T08:06:04.731200+0000","last_became_peered":"2024-09-10T08:06:04.731200+0000","last_unstale":"2024-09-10T08:06:40.292071+0000","last_undegraded":"2024-09-10T08:06:40.292071+0000","last_fullsized":"2024-09-10T08:06:40.292071+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:17.169708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57258,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":78,"num_write":30,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3703,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730227+0000","last_change":"2024-09-10T08:05:35.532594+0000","last_active":"2024-09-10T08:06:04.730227+0000","last_peered":"2024-09-10T08:06:04.730227+0000","last_clean":"2024-09-10T08:06:04.730227+0000","last_became_active":"2024-09-10T08:05:35.532132+0000","last_became_peered":"2024-09-10T08:05:35.532132+0000","last_unstale":"2024-09-10T08:06:04.730227+0000","last_undegraded":"2024-09-10T08:06:04.730227+0000","last_fullsized":"2024-09-10T08:06:04.730227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:15.348648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728585+0000","last_change":"2024-09-10T08:05:25.485297+0000","last_active":"2024-09-10T08:06:04.728585+0000","last_peered":"2024-09-10T08:06:04.728585+0000","last_clean":"2024-09-10T08:06:04.728585+0000","last_became_active":"2024-09-10T08:05:25.484883+0000","last_became_peered":"2024-09-10T08:05:25.484883+0000","last_unstale":"2024-09-10T08:06:04.728585+0000","last_undegraded":"2024-09-10T08:06:04.728585+0000","last_fullsized":"2024-09-10T08:06:04.728585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:37.052435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"93'4","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.789580+0000","last_change":"2024-09-10T08:05:22.568993+0000","last_active":"2024-09-10T08:06:19.789580+0000","last_peered":"2024-09-10T08:06:19.789580+0000","last_clean":"2024-09-10T08:06:19.789580+0000","last_became_active":"2024-09-10T08:05:22.568564+0000","last_became_peered":"2024-09-10T08:05:22.568564+0000","last_unstale":"2024-09-10T08:06:19.789580+0000","last_undegraded":"2024-09-10T08:06:19.789580+0000","last_fullsized":"2024-09-10T08:06:19.789580+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:23.767174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729038+0000","last_change":"2024-09-10T08:05:35.524025+0000","last_active":"2024-09-10T08:06:04.729038+0000","last_peered":"2024-09-10T08:06:04.729038+0000","last_clean":"2024-09-10T08:06:04.729038+0000","last_became_active":"2024-09-10T08:05:35.523628+0000","last_became_peered":"2024-09-10T08:05:35.523628+0000","last_unstale":"2024-09-10T08:06:04.729038+0000","last_undegraded":"2024-09-10T08:06:04.729038+0000","last_fullsized":"2024-09-10T08:06:04.729038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:48.593779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733218+0000","last_change":"2024-09-10T08:05:25.496721+0000","last_active":"2024-09-10T08:06:04.733218+0000","last_peered":"2024-09-10T08:06:04.733218+0000","last_clean":"2024-09-10T08:06:04.733218+0000","last_became_active":"2024-09-10T08:05:25.495058+0000","last_became_peered":"2024-09-10T08:05:25.495058+0000","last_unstale":"2024-09-10T08:06:04.733218+0000","last_undegraded":"2024-09-10T08:06:04.733218+0000","last_fullsized":"2024-09-10T08:06:04.733218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:49.832511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"93'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:19.790645+0000","last_change":"2024-09-10T08:05:22.460487+0000","last_active":"2024-09-10T08:06:19.790645+0000","last_peered":"2024-09-10T08:06:19.790645+0000","last_clean":"2024-09-10T08:06:19.790645+0000","last_became_active":"2024-09-10T08:05:22.460158+0000","last_became_peered":"2024-09-10T08:05:22.460158+0000","last_unstale":"2024-09-10T08:06:19.790645+0000","last_undegraded":"2024-09-10T08:06:19.790645+0000","last_fullsized":"2024-09-10T08:06:19.790645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:53.285513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729478+0000","last_change":"2024-09-10T08:05:35.531399+0000","last_active":"2024-09-10T08:06:04.729478+0000","last_peered":"2024-09-10T08:06:04.729478+0000","last_clean":"2024-09-10T08:06:04.729478+0000","last_became_active":"2024-09-10T08:05:35.530320+0000","last_became_peered":"2024-09-10T08:05:35.530320+0000","last_unstale":"2024-09-10T08:06:04.729478+0000","last_undegraded":"2024-09-10T08:06:04.729478+0000","last_fullsized":"2024-09-10T08:06:04.729478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:19.814046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729028+0000","last_change":"2024-09-10T08:05:25.488208+0000","last_active":"2024-09-10T08:06:04.729028+0000","last_peered":"2024-09-10T08:06:04.729028+0000","last_clean":"2024-09-10T08:06:04.729028+0000","last_became_active":"2024-09-10T08:05:25.487893+0000","last_became_peered":"2024-09-10T08:05:25.487893+0000","last_unstale":"2024-09-10T08:06:04.729028+0000","last_undegraded":"2024-09-10T08:06:04.729028+0000","last_fullsized":"2024-09-10T08:06:04.729028+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:10.999416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"93'3","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:39.289458+0000","last_change":"2024-09-10T08:05:22.461321+0000","last_active":"2024-09-10T08:06:39.289458+0000","last_peered":"2024-09-10T08:06:39.289458+0000","last_clean":"2024-09-10T08:06:39.289458+0000","last_became_active":"2024-09-10T08:05:22.460950+0000","last_became_peered":"2024-09-10T08:05:22.460950+0000","last_unstale":"2024-09-10T08:06:39.289458+0000","last_undegraded":"2024-09-10T08:06:39.289458+0000","last_fullsized":"2024-09-10T08:06:39.289458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:55.647273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733452+0000","last_change":"2024-09-10T08:05:35.530578+0000","last_active":"2024-09-10T08:06:04.733452+0000","last_peered":"2024-09-10T08:06:04.733452+0000","last_clean":"2024-09-10T08:06:04.733452+0000","last_became_active":"2024-09-10T08:05:35.530162+0000","last_became_peered":"2024-09-10T08:05:35.530162+0000","last_unstale":"2024-09-10T08:06:04.733452+0000","last_undegraded":"2024-09-10T08:06:04.733452+0000","last_fullsized":"2024-09-10T08:06:04.733452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:19.290973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729879+0000","last_change":"2024-09-10T08:05:35.532685+0000","last_active":"2024-09-10T08:06:04.729879+0000","last_peered":"2024-09-10T08:06:04.729879+0000","last_clean":"2024-09-10T08:06:04.729879+0000","last_became_active":"2024-09-10T08:05:35.532257+0000","last_became_peered":"2024-09-10T08:05:35.532257+0000","last_unstale":"2024-09-10T08:06:04.729879+0000","last_undegraded":"2024-09-10T08:06:04.729879+0000","last_fullsized":"2024-09-10T08:06:04.729879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:07.412435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730714+0000","last_change":"2024-09-10T08:05:25.494472+0000","last_active":"2024-09-10T08:06:04.730714+0000","last_peered":"2024-09-10T08:06:04.730714+0000","last_clean":"2024-09-10T08:06:04.730714+0000","last_became_active":"2024-09-10T08:05:25.493928+0000","last_became_peered":"2024-09-10T08:05:25.493928+0000","last_unstale":"2024-09-10T08:06:04.730714+0000","last_undegraded":"2024-09-10T08:06:04.730714+0000","last_fullsized":"2024-09-10T08:06:04.730714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:00.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"93'1","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725770+0000","last_change":"2024-09-10T08:05:22.460585+0000","last_active":"2024-09-10T08:06:04.725770+0000","last_peered":"2024-09-10T08:06:04.725770+0000","last_clean":"2024-09-10T08:06:04.725770+0000","last_became_active":"2024-09-10T08:05:22.460259+0000","last_became_peered":"2024-09-10T08:05:22.460259+0000","last_unstale":"2024-09-10T08:06:04.725770+0000","last_undegraded":"2024-09-10T08:06:04.725770+0000","last_fullsized":"2024-09-10T08:06:04.725770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:45.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730310+0000","last_change":"2024-09-10T08:05:35.524000+0000","last_active":"2024-09-10T08:06:04.730310+0000","last_peered":"2024-09-10T08:06:04.730310+0000","last_clean":"2024-09-10T08:06:04.730310+0000","last_became_active":"2024-09-10T08:05:35.523647+0000","last_became_peered":"2024-09-10T08:05:35.523647+0000","last_unstale":"2024-09-10T08:06:04.730310+0000","last_undegraded":"2024-09-10T08:06:04.730310+0000","last_fullsized":"2024-09-10T08:06:04.730310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:10:54.664865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.734307+0000","last_change":"2024-09-10T08:05:25.499701+0000","last_active":"2024-09-10T08:06:04.734307+0000","last_peered":"2024-09-10T08:06:04.734307+0000","last_clean":"2024-09-10T08:06:04.734307+0000","last_became_active":"2024-09-10T08:05:25.496890+0000","last_became_peered":"2024-09-10T08:05:25.496890+0000","last_unstale":"2024-09-10T08:06:04.734307+0000","last_undegraded":"2024-09-10T08:06:04.734307+0000","last_fullsized":"2024-09-10T08:06:04.734307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:58.166365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"93'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.795335+0000","last_change":"2024-09-10T08:05:22.460880+0000","last_active":"2024-09-10T08:06:20.795335+0000","last_peered":"2024-09-10T08:06:20.795335+0000","last_clean":"2024-09-10T08:06:20.795335+0000","last_became_active":"2024-09-10T08:05:22.459932+0000","last_became_peered":"2024-09-10T08:05:22.459932+0000","last_unstale":"2024-09-10T08:06:20.795335+0000","last_undegraded":"2024-09-10T08:06:20.795335+0000","last_fullsized":"2024-09-10T08:06:20.795335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:55.826526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"93'3","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:20.794861+0000","last_change":"2024-09-10T08:05:22.460578+0000","last_active":"2024-09-10T08:06:20.794861+0000","last_peered":"2024-09-10T08:06:20.794861+0000","last_clean":"2024-09-10T08:06:20.794861+0000","last_became_active":"2024-09-10T08:05:22.460239+0000","last_became_peered":"2024-09-10T08:05:22.460239+0000","last_unstale":"2024-09-10T08:06:20.794861+0000","last_undegraded":"2024-09-10T08:06:20.794861+0000","last_fullsized":"2024-09-10T08:06:20.794861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:07.606243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.726058+0000","last_change":"2024-09-10T08:05:25.489846+0000","last_active":"2024-09-10T08:06:04.726058+0000","last_peered":"2024-09-10T08:06:04.726058+0000","last_clean":"2024-09-10T08:06:04.726058+0000","last_became_active":"2024-09-10T08:05:25.489326+0000","last_became_peered":"2024-09-10T08:05:25.489326+0000","last_unstale":"2024-09-10T08:06:04.726058+0000","last_undegraded":"2024-09-10T08:06:04.726058+0000","last_fullsized":"2024-09-10T08:06:04.726058+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:43.458468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731382+0000","last_change":"2024-09-10T08:05:35.530522+0000","last_active":"2024-09-10T08:06:04.731382+0000","last_peered":"2024-09-10T08:06:04.731382+0000","last_clean":"2024-09-10T08:06:04.731382+0000","last_became_active":"2024-09-10T08:05:35.530099+0000","last_became_peered":"2024-09-10T08:05:35.530099+0000","last_unstale":"2024-09-10T08:06:04.731382+0000","last_undegraded":"2024-09-10T08:06:04.731382+0000","last_fullsized":"2024-09-10T08:06:04.731382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:58.469119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'6","reported_seq":69,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:41.293207+0000","last_change":"2024-09-10T08:05:22.461391+0000","last_active":"2024-09-10T08:06:41.293207+0000","last_peered":"2024-09-10T08:06:41.293207+0000","last_clean":"2024-09-10T08:06:41.293207+0000","last_became_active":"2024-09-10T08:05:22.461042+0000","last_became_peered":"2024-09-10T08:05:22.461042+0000","last_unstale":"2024-09-10T08:06:41.293207+0000","last_undegraded":"2024-09-10T08:06:41.293207+0000","last_fullsized":"2024-09-10T08:06:41.293207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:12.635710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728879+0000","last_change":"2024-09-10T08:05:25.487378+0000","last_active":"2024-09-10T08:06:04.728879+0000","last_peered":"2024-09-10T08:06:04.728879+0000","last_clean":"2024-09-10T08:06:04.728879+0000","last_became_active":"2024-09-10T08:05:25.487055+0000","last_became_peered":"2024-09-10T08:05:25.487055+0000","last_unstale":"2024-09-10T08:06:04.728879+0000","last_undegraded":"2024-09-10T08:06:04.728879+0000","last_fullsized":"2024-09-10T08:06:04.728879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:59.522014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733027+0000","last_change":"2024-09-10T08:05:35.522765+0000","last_active":"2024-09-10T08:06:04.733027+0000","last_peered":"2024-09-10T08:06:04.733027+0000","last_clean":"2024-09-10T08:06:04.733027+0000","last_became_active":"2024-09-10T08:05:35.522359+0000","last_became_peered":"2024-09-10T08:05:35.522359+0000","last_unstale":"2024-09-10T08:06:04.733027+0000","last_undegraded":"2024-09-10T08:06:04.733027+0000","last_fullsized":"2024-09-10T08:06:04.733027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:15.280867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728630+0000","last_change":"2024-09-10T08:05:25.475388+0000","last_active":"2024-09-10T08:06:04.728630+0000","last_peered":"2024-09-10T08:06:04.728630+0000","last_clean":"2024-09-10T08:06:04.728630+0000","last_became_active":"2024-09-10T08:05:25.474331+0000","last_became_peered":"2024-09-10T08:05:25.474331+0000","last_unstale":"2024-09-10T08:06:04.728630+0000","last_undegraded":"2024-09-10T08:06:04.728630+0000","last_fullsized":"2024-09-10T08:06:04.728630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:58.713148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731811+0000","last_change":"2024-09-10T08:05:25.481548+0000","last_active":"2024-09-10T08:06:04.731811+0000","last_peered":"2024-09-10T08:06:04.731811+0000","last_clean":"2024-09-10T08:06:04.731811+0000","last_became_active":"2024-09-10T08:05:25.481152+0000","last_became_peered":"2024-09-10T08:05:25.481152+0000","last_unstale":"2024-09-10T08:06:04.731811+0000","last_undegraded":"2024-09-10T08:06:04.731811+0000","last_fullsized":"2024-09-10T08:06:04.731811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:14:37.938977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731491+0000","last_change":"2024-09-10T08:05:25.491585+0000","last_active":"2024-09-10T08:06:04.731491+0000","last_peered":"2024-09-10T08:06:04.731491+0000","last_clean":"2024-09-10T08:06:04.731491+0000","last_became_active":"2024-09-10T08:05:25.491224+0000","last_became_peered":"2024-09-10T08:05:25.491224+0000","last_unstale":"2024-09-10T08:06:04.731491+0000","last_undegraded":"2024-09-10T08:06:04.731491+0000","last_fullsized":"2024-09-10T08:06:04.731491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:56.066808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729798+0000","last_change":"2024-09-10T08:05:25.499578+0000","last_active":"2024-09-10T08:06:04.729798+0000","last_peered":"2024-09-10T08:06:04.729798+0000","last_clean":"2024-09-10T08:06:04.729798+0000","last_became_active":"2024-09-10T08:05:25.498445+0000","last_became_peered":"2024-09-10T08:05:25.498445+0000","last_unstale":"2024-09-10T08:06:04.729798+0000","last_undegraded":"2024-09-10T08:06:04.729798+0000","last_fullsized":"2024-09-10T08:06:04.729798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:58.961699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.735764+0000","last_change":"2024-09-10T08:06:04.735764+0000","last_active":"2024-09-10T08:06:04.735764+0000","last_peered":"2024-09-10T08:06:04.735764+0000","last_clean":"2024-09-10T08:06:04.735764+0000","last_became_active":"2024-09-10T08:06:04.735366+0000","last_became_peered":"2024-09-10T08:06:04.735366+0000","last_unstale":"2024-09-10T08:06:04.735764+0000","last_undegraded":"2024-09-10T08:06:04.735764+0000","last_fullsized":"2024-09-10T08:06:04.735764+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:03.757890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729806+0000","last_change":"2024-09-10T08:05:25.475872+0000","last_active":"2024-09-10T08:06:04.729806+0000","last_peered":"2024-09-10T08:06:04.729806+0000","last_clean":"2024-09-10T08:06:04.729806+0000","last_became_active":"2024-09-10T08:05:25.475549+0000","last_became_peered":"2024-09-10T08:05:25.475549+0000","last_unstale":"2024-09-10T08:06:04.729806+0000","last_undegraded":"2024-09-10T08:06:04.729806+0000","last_fullsized":"2024-09-10T08:06:04.729806+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:37.575267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733499+0000","last_change":"2024-09-10T08:05:25.474210+0000","last_active":"2024-09-10T08:06:04.733499+0000","last_peered":"2024-09-10T08:06:04.733499+0000","last_clean":"2024-09-10T08:06:04.733499+0000","last_became_active":"2024-09-10T08:05:25.473882+0000","last_became_peered":"2024-09-10T08:05:25.473882+0000","last_unstale":"2024-09-10T08:06:04.733499+0000","last_undegraded":"2024-09-10T08:06:04.733499+0000","last_fullsized":"2024-09-10T08:06:04.733499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:05.164370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733896+0000","last_change":"2024-09-10T08:05:25.490406+0000","last_active":"2024-09-10T08:06:04.733896+0000","last_peered":"2024-09-10T08:06:04.733896+0000","last_clean":"2024-09-10T08:06:04.733896+0000","last_became_active":"2024-09-10T08:05:25.490061+0000","last_became_peered":"2024-09-10T08:05:25.490061+0000","last_unstale":"2024-09-10T08:06:04.733896+0000","last_undegraded":"2024-09-10T08:06:04.733896+0000","last_fullsized":"2024-09-10T08:06:04.733896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:12.988229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725565+0000","last_change":"2024-09-10T08:05:25.504049+0000","last_active":"2024-09-10T08:06:04.725565+0000","last_peered":"2024-09-10T08:06:04.725565+0000","last_clean":"2024-09-10T08:06:04.725565+0000","last_became_active":"2024-09-10T08:05:25.503770+0000","last_became_peered":"2024-09-10T08:05:25.503770+0000","last_unstale":"2024-09-10T08:06:04.725565+0000","last_undegraded":"2024-09-10T08:06:04.725565+0000","last_fullsized":"2024-09-10T08:06:04.725565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:52.850945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733622+0000","last_change":"2024-09-10T08:05:25.491682+0000","last_active":"2024-09-10T08:06:04.733622+0000","last_peered":"2024-09-10T08:06:04.733622+0000","last_clean":"2024-09-10T08:06:04.733622+0000","last_became_active":"2024-09-10T08:05:25.491296+0000","last_became_peered":"2024-09-10T08:05:25.491296+0000","last_unstale":"2024-09-10T08:06:04.733622+0000","last_undegraded":"2024-09-10T08:06:04.733622+0000","last_fullsized":"2024-09-10T08:06:04.733622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:30.075842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728374+0000","last_change":"2024-09-10T08:05:25.489648+0000","last_active":"2024-09-10T08:06:04.728374+0000","last_peered":"2024-09-10T08:06:04.728374+0000","last_clean":"2024-09-10T08:06:04.728374+0000","last_became_active":"2024-09-10T08:05:25.489267+0000","last_became_peered":"2024-09-10T08:05:25.489267+0000","last_unstale":"2024-09-10T08:06:04.728374+0000","last_undegraded":"2024-09-10T08:06:04.728374+0000","last_fullsized":"2024-09-10T08:06:04.728374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:21.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733968+0000","last_change":"2024-09-10T08:05:25.497862+0000","last_active":"2024-09-10T08:06:04.733968+0000","last_peered":"2024-09-10T08:06:04.733968+0000","last_clean":"2024-09-10T08:06:04.733968+0000","last_became_active":"2024-09-10T08:05:25.497504+0000","last_became_peered":"2024-09-10T08:05:25.497504+0000","last_unstale":"2024-09-10T08:06:04.733968+0000","last_undegraded":"2024-09-10T08:06:04.733968+0000","last_fullsized":"2024-09-10T08:06:04.733968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:40.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729400+0000","last_change":"2024-09-10T08:05:25.485924+0000","last_active":"2024-09-10T08:06:04.729400+0000","last_peered":"2024-09-10T08:06:04.729400+0000","last_clean":"2024-09-10T08:06:04.729400+0000","last_became_active":"2024-09-10T08:05:25.485545+0000","last_became_peered":"2024-09-10T08:05:25.485545+0000","last_unstale":"2024-09-10T08:06:04.729400+0000","last_undegraded":"2024-09-10T08:06:04.729400+0000","last_fullsized":"2024-09-10T08:06:04.729400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:33:20.233720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.733183+0000","last_change":"2024-09-10T08:05:25.491911+0000","last_active":"2024-09-10T08:06:04.733183+0000","last_peered":"2024-09-10T08:06:04.733183+0000","last_clean":"2024-09-10T08:06:04.733183+0000","last_became_active":"2024-09-10T08:05:25.491330+0000","last_became_peered":"2024-09-10T08:05:25.491330+0000","last_unstale":"2024-09-10T08:06:04.733183+0000","last_undegraded":"2024-09-10T08:06:04.733183+0000","last_fullsized":"2024-09-10T08:06:04.733183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:06.043331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731733+0000","last_change":"2024-09-10T08:05:25.492220+0000","last_active":"2024-09-10T08:06:04.731733+0000","last_peered":"2024-09-10T08:06:04.731733+0000","last_clean":"2024-09-10T08:06:04.731733+0000","last_became_active":"2024-09-10T08:05:25.491473+0000","last_became_peered":"2024-09-10T08:05:25.491473+0000","last_unstale":"2024-09-10T08:06:04.731733+0000","last_undegraded":"2024-09-10T08:06:04.731733+0000","last_fullsized":"2024-09-10T08:06:04.731733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:13:30.278434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730671+0000","last_change":"2024-09-10T08:05:25.488927+0000","last_active":"2024-09-10T08:06:04.730671+0000","last_peered":"2024-09-10T08:06:04.730671+0000","last_clean":"2024-09-10T08:06:04.730671+0000","last_became_active":"2024-09-10T08:05:25.488550+0000","last_became_peered":"2024-09-10T08:05:25.488550+0000","last_unstale":"2024-09-10T08:06:04.730671+0000","last_undegraded":"2024-09-10T08:06:04.730671+0000","last_fullsized":"2024-09-10T08:06:04.730671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:44.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728218+0000","last_change":"2024-09-10T08:05:25.490125+0000","last_active":"2024-09-10T08:06:04.728218+0000","last_peered":"2024-09-10T08:06:04.728218+0000","last_clean":"2024-09-10T08:06:04.728218+0000","last_became_active":"2024-09-10T08:05:25.487805+0000","last_became_peered":"2024-09-10T08:05:25.487805+0000","last_unstale":"2024-09-10T08:06:04.728218+0000","last_undegraded":"2024-09-10T08:06:04.728218+0000","last_fullsized":"2024-09-10T08:06:04.728218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:33.479009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731155+0000","last_change":"2024-09-10T08:05:25.500346+0000","last_active":"2024-09-10T08:06:04.731155+0000","last_peered":"2024-09-10T08:06:04.731155+0000","last_clean":"2024-09-10T08:06:04.731155+0000","last_became_active":"2024-09-10T08:05:25.498289+0000","last_became_peered":"2024-09-10T08:05:25.498289+0000","last_unstale":"2024-09-10T08:06:04.731155+0000","last_undegraded":"2024-09-10T08:06:04.731155+0000","last_fullsized":"2024-09-10T08:06:04.731155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:12.054926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731420+0000","last_change":"2024-09-10T08:05:25.490001+0000","last_active":"2024-09-10T08:06:04.731420+0000","last_peered":"2024-09-10T08:06:04.731420+0000","last_clean":"2024-09-10T08:06:04.731420+0000","last_became_active":"2024-09-10T08:05:25.489706+0000","last_became_peered":"2024-09-10T08:05:25.489706+0000","last_unstale":"2024-09-10T08:06:04.731420+0000","last_undegraded":"2024-09-10T08:06:04.731420+0000","last_fullsized":"2024-09-10T08:06:04.731420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:38.822140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732652+0000","last_change":"2024-09-10T08:05:25.493874+0000","last_active":"2024-09-10T08:06:04.732652+0000","last_peered":"2024-09-10T08:06:04.732652+0000","last_clean":"2024-09-10T08:06:04.732652+0000","last_became_active":"2024-09-10T08:05:25.489911+0000","last_became_peered":"2024-09-10T08:05:25.489911+0000","last_unstale":"2024-09-10T08:06:04.732652+0000","last_undegraded":"2024-09-10T08:06:04.732652+0000","last_fullsized":"2024-09-10T08:06:04.732652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:46.509657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729372+0000","last_change":"2024-09-10T08:05:25.490277+0000","last_active":"2024-09-10T08:06:04.729372+0000","last_peered":"2024-09-10T08:06:04.729372+0000","last_clean":"2024-09-10T08:06:04.729372+0000","last_became_active":"2024-09-10T08:05:25.486863+0000","last_became_peered":"2024-09-10T08:05:25.486863+0000","last_unstale":"2024-09-10T08:06:04.729372+0000","last_undegraded":"2024-09-10T08:06:04.729372+0000","last_fullsized":"2024-09-10T08:06:04.729372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:21.353830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732812+0000","last_change":"2024-09-10T08:05:25.491028+0000","last_active":"2024-09-10T08:06:04.732812+0000","last_peered":"2024-09-10T08:06:04.732812+0000","last_clean":"2024-09-10T08:06:04.732812+0000","last_became_active":"2024-09-10T08:05:25.490452+0000","last_became_peered":"2024-09-10T08:05:25.490452+0000","last_unstale":"2024-09-10T08:06:04.732812+0000","last_undegraded":"2024-09-10T08:06:04.732812+0000","last_fullsized":"2024-09-10T08:06:04.732812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:53.059077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.726402+0000","last_change":"2024-09-10T08:05:25.489991+0000","last_active":"2024-09-10T08:06:04.726402+0000","last_peered":"2024-09-10T08:06:04.726402+0000","last_clean":"2024-09-10T08:06:04.726402+0000","last_became_active":"2024-09-10T08:05:25.489662+0000","last_became_peered":"2024-09-10T08:05:25.489662+0000","last_unstale":"2024-09-10T08:06:04.726402+0000","last_undegraded":"2024-09-10T08:06:04.726402+0000","last_fullsized":"2024-09-10T08:06:04.726402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:47.940712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.728907+0000","last_change":"2024-09-10T08:05:25.475752+0000","last_active":"2024-09-10T08:06:04.728907+0000","last_peered":"2024-09-10T08:06:04.728907+0000","last_clean":"2024-09-10T08:06:04.728907+0000","last_became_active":"2024-09-10T08:05:25.475395+0000","last_became_peered":"2024-09-10T08:05:25.475395+0000","last_unstale":"2024-09-10T08:06:04.728907+0000","last_undegraded":"2024-09-10T08:06:04.728907+0000","last_fullsized":"2024-09-10T08:06:04.728907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.240154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.734209+0000","last_change":"2024-09-10T08:05:25.503061+0000","last_active":"2024-09-10T08:06:04.734209+0000","last_peered":"2024-09-10T08:06:04.734209+0000","last_clean":"2024-09-10T08:06:04.734209+0000","last_became_active":"2024-09-10T08:05:25.501333+0000","last_became_peered":"2024-09-10T08:05:25.501333+0000","last_unstale":"2024-09-10T08:06:04.734209+0000","last_undegraded":"2024-09-10T08:06:04.734209+0000","last_fullsized":"2024-09-10T08:06:04.734209+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:38.254777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.732218+0000","last_change":"2024-09-10T08:05:25.482628+0000","last_active":"2024-09-10T08:06:04.732218+0000","last_peered":"2024-09-10T08:06:04.732218+0000","last_clean":"2024-09-10T08:06:04.732218+0000","last_became_active":"2024-09-10T08:05:25.482214+0000","last_became_peered":"2024-09-10T08:05:25.482214+0000","last_unstale":"2024-09-10T08:06:04.732218+0000","last_undegraded":"2024-09-10T08:06:04.732218+0000","last_fullsized":"2024-09-10T08:06:04.732218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:49.045496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.725832+0000","last_change":"2024-09-10T08:05:25.491457+0000","last_active":"2024-09-10T08:06:04.725832+0000","last_peered":"2024-09-10T08:06:04.725832+0000","last_clean":"2024-09-10T08:06:04.725832+0000","last_became_active":"2024-09-10T08:05:25.491162+0000","last_became_peered":"2024-09-10T08:05:25.491162+0000","last_unstale":"2024-09-10T08:06:04.725832+0000","last_undegraded":"2024-09-10T08:06:04.725832+0000","last_fullsized":"2024-09-10T08:06:04.725832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:14.389752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.727533+0000","last_change":"2024-09-10T08:05:25.487874+0000","last_active":"2024-09-10T08:06:04.727533+0000","last_peered":"2024-09-10T08:06:04.727533+0000","last_clean":"2024-09-10T08:06:04.727533+0000","last_became_active":"2024-09-10T08:05:25.487497+0000","last_became_peered":"2024-09-10T08:05:25.487497+0000","last_unstale":"2024-09-10T08:06:04.727533+0000","last_undegraded":"2024-09-10T08:06:04.727533+0000","last_fullsized":"2024-09-10T08:06:04.727533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:26.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729279+0000","last_change":"2024-09-10T08:05:25.490018+0000","last_active":"2024-09-10T08:06:04.729279+0000","last_peered":"2024-09-10T08:06:04.729279+0000","last_clean":"2024-09-10T08:06:04.729279+0000","last_became_active":"2024-09-10T08:05:25.484951+0000","last_became_peered":"2024-09-10T08:05:25.484951+0000","last_unstale":"2024-09-10T08:06:04.729279+0000","last_undegraded":"2024-09-10T08:06:04.729279+0000","last_fullsized":"2024-09-10T08:06:04.729279+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:07.516154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.731689+0000","last_change":"2024-09-10T08:05:25.501581+0000","last_active":"2024-09-10T08:06:04.731689+0000","last_peered":"2024-09-10T08:06:04.731689+0000","last_clean":"2024-09-10T08:06:04.731689+0000","last_became_active":"2024-09-10T08:05:25.499690+0000","last_became_peered":"2024-09-10T08:05:25.499690+0000","last_unstale":"2024-09-10T08:06:04.731689+0000","last_undegraded":"2024-09-10T08:06:04.731689+0000","last_fullsized":"2024-09-10T08:06:04.731689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:14.569791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.729987+0000","last_change":"2024-09-10T08:05:25.487945+0000","last_active":"2024-09-10T08:06:04.729987+0000","last_peered":"2024-09-10T08:06:04.729987+0000","last_clean":"2024-09-10T08:06:04.729987+0000","last_became_active":"2024-09-10T08:05:25.487655+0000","last_became_peered":"2024-09-10T08:05:25.487655+0000","last_unstale":"2024-09-10T08:06:04.729987+0000","last_undegraded":"2024-09-10T08:06:04.729987+0000","last_fullsized":"2024-09-10T08:06:04.729987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:37.746037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-10T08:06:04.730664+0000","last_change":"2024-09-10T08:05:25.489348+0000","last_active":"2024-09-10T08:06:04.730664+0000","last_peered":"2024-09-10T08:06:04.730664+0000","last_clean":"2024-09-10T08:06:04.730664+0000","last_became_active":"2024-09-10T08:05:25.489004+0000","last_became_peered":"2024-09-10T08:05:25.489004+0000","last_unstale":"2024-09-10T08:06:04.730664+0000","last_undegraded":"2024-09-10T08:06:04.730664+0000","last_fullsized":"2024-09-10T08:06:04.730664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:54.554719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61261,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":146,"num_read_kb":137,"num_write":158,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3725,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":183783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214976,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514525,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":528985,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743458,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1167360,"data_stored":1177585,"data_compressed":62832,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":529199,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135917,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":529097,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364848,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":1125030,"data_compressed":62832,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528375,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":524375,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790014,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":1120618,"data_compressed":62832,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953538,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":586283,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":524569,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":526028,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":581543,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":525763,"data_compressed":26454,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":57902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:06:46.463 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T08:06:46.463 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T08:06:46.464 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T08:06:46.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:46 smithi002 ceph-mon[30563]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:06:46.818 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:46 smithi107 ceph-mon[35048]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:06:46.840 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:46.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:46 smithi155 ceph-mon[35405]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:06:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.453+0000 7fb986800640 1 -- 172.21.15.2:0/1968110360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb980103e00 msgr2=0x7fb980104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.453+0000 7fb986800640 1 --2- 172.21.15.2:0/1968110360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb980103e00 0x7fb980104200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb970009900 tx=0x7fb97002a410 comp rx=0 tx=0).stop 2024-09-10T08:06:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.453+0000 7fb986800640 1 -- 172.21.15.2:0/1968110360 shutdown_connections 2024-09-10T08:06:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.453+0000 7fb986800640 1 --2- 172.21.15.2:0/1968110360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb980105920 0x7fb98010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.453+0000 7fb986800640 1 --2- 172.21.15.2:0/1968110360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb980104f60 0x7fb9801053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.453+0000 7fb986800640 1 --2- 172.21.15.2:0/1968110360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb980103e00 0x7fb980104200 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.453+0000 7fb986800640 1 -- 172.21.15.2:0/1968110360 >> 172.21.15.2:0/1968110360 conn(0x7fb9800ff5b0 msgr2=0x7fb9801019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.453+0000 7fb986800640 1 -- 172.21.15.2:0/1968110360 shutdown_connections 2024-09-10T08:06:47.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.453+0000 7fb986800640 1 -- 172.21.15.2:0/1968110360 wait complete. 2024-09-10T08:06:47.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.454+0000 7fb986800640 1 Processor -- start 2024-09-10T08:06:47.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.454+0000 7fb986800640 1 -- start start 2024-09-10T08:06:47.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.454+0000 7fb986800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb980103e00 0x7fb98019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:47.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.454+0000 7fb986800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb980104f60 0x7fb98019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:47.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.454+0000 7fb986800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb980105920 0x7fb9801a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:47.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.454+0000 7fb986800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb98010fb00 con 0x7fb980104f60 2024-09-10T08:06:47.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.454+0000 7fb986800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb98010f980 con 0x7fb980105920 2024-09-10T08:06:47.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.454+0000 7fb986800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb98010fc80 con 0x7fb980103e00 2024-09-10T08:06:47.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.455+0000 7fb977e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb980104f60 0x7fb98019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:47.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.455+0000 7fb97f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb980103e00 0x7fb98019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.455+0000 7fb97fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb980105920 0x7fb9801a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.455+0000 7fb977e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb980104f60 0x7fb98019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38952/0 (socket says 172.21.15.2:38952) 2024-09-10T08:06:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.455+0000 7fb977e00640 1 -- 172.21.15.2:0/3910928761 learned_addr learned my addr 172.21.15.2:0/3910928761 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.455+0000 7fb97f400640 1 -- 172.21.15.2:0/3910928761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb980105920 msgr2=0x7fb9801a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.455+0000 7fb97f400640 1 --2- 172.21.15.2:0/3910928761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb980105920 0x7fb9801a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.455+0000 7fb97f400640 1 -- 172.21.15.2:0/3910928761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb980104f60 msgr2=0x7fb98019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.455+0000 7fb97f400640 1 --2- 172.21.15.2:0/3910928761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb980104f60 0x7fb98019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.455+0000 7fb97f400640 1 -- 172.21.15.2:0/3910928761 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9801a1d00 con 0x7fb980103e00 2024-09-10T08:06:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.455+0000 7fb977e00640 1 --2- 172.21.15.2:0/3910928761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb980104f60 0x7fb98019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:06:47.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.456+0000 7fb97f400640 1 --2- 172.21.15.2:0/3910928761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb980103e00 0x7fb98019a0d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb97002a930 tx=0x7fb970032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:47.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.456+0000 7fb97cc00640 1 -- 172.21.15.2:0/3910928761 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97003f030 con 0x7fb980103e00 2024-09-10T08:06:47.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.456+0000 7fb986800640 1 -- 172.21.15.2:0/3910928761 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9801a1f90 con 0x7fb980103e00 2024-09-10T08:06:47.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.456+0000 7fb97cc00640 1 -- 172.21.15.2:0/3910928761 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb970003ce0 con 0x7fb980103e00 2024-09-10T08:06:47.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.456+0000 7fb97cc00640 1 -- 172.21.15.2:0/3910928761 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970035070 con 0x7fb980103e00 2024-09-10T08:06:47.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.456+0000 7fb986800640 1 -- 172.21.15.2:0/3910928761 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9801a23e0 con 0x7fb980103e00 2024-09-10T08:06:47.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.457+0000 7fb986800640 1 -- 172.21.15.2:0/3910928761 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98010b170 con 0x7fb980103e00 2024-09-10T08:06:47.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.459+0000 7fb97cc00640 1 -- 172.21.15.2:0/3910928761 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb970042030 con 0x7fb980103e00 2024-09-10T08:06:47.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.460+0000 7fb97cc00640 1 --2- 172.21.15.2:0/3910928761 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9640a2c40 0x7fb9640a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:47.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.460+0000 7fb97cc00640 1 -- 172.21.15.2:0/3910928761 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fb9700ef8d0 con 0x7fb980103e00 2024-09-10T08:06:47.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.460+0000 7fb977e00640 1 --2- 172.21.15.2:0/3910928761 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9640a2c40 0x7fb9640a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:47.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.461+0000 7fb977e00640 1 --2- 172.21.15.2:0/3910928761 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9640a2c40 0x7fb9640a5100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb98019b5f0 tx=0x7fb968005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:47.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.464+0000 7fb97cc00640 1 -- 172.21.15.2:0/3910928761 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9700bbcc0 con 0x7fb980103e00 2024-09-10T08:06:47.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.718+0000 7fb986800640 1 -- 172.21.15.2:0/3910928761 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb98002d9e0 con 0x7fb980103e00 2024-09-10T08:06:47.718 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:47 smithi002 ceph-mon[30563]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-10T08:06:47.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.725+0000 7fb97cc00640 1 -- 172.21.15.2:0/3910928761 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fb9700bffc0 con 0x7fb980103e00 2024-09-10T08:06:47.725 INFO:teuthology.orchestra.run.smithi002.stdout:[client.0] 2024-09-10T08:06:47.725 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQAX/t9mtVf7KhAA6syTS+TceVMO8N58wRlAcQ== 2024-09-10T08:06:47.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.727+0000 7fb986800640 1 -- 172.21.15.2:0/3910928761 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9640a2c40 msgr2=0x7fb9640a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:47.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.727+0000 7fb986800640 1 --2- 172.21.15.2:0/3910928761 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9640a2c40 0x7fb9640a5100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb98019b5f0 tx=0x7fb968005f90 comp rx=0 tx=0).stop 2024-09-10T08:06:47.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.727+0000 7fb986800640 1 -- 172.21.15.2:0/3910928761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb980103e00 msgr2=0x7fb98019a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:47.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.727+0000 7fb986800640 1 --2- 172.21.15.2:0/3910928761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb980103e00 0x7fb98019a0d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb97002a930 tx=0x7fb970032ce0 comp rx=0 tx=0).stop 2024-09-10T08:06:47.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.727+0000 7fb986800640 1 -- 172.21.15.2:0/3910928761 shutdown_connections 2024-09-10T08:06:47.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.727+0000 7fb986800640 1 --2- 172.21.15.2:0/3910928761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb980105920 0x7fb9801a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:47.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.727+0000 7fb986800640 1 --2- 172.21.15.2:0/3910928761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb980104f60 0x7fb98019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:47.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.727+0000 7fb986800640 1 --2- 172.21.15.2:0/3910928761 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9640a2c40 0x7fb9640a5100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:47.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.727+0000 7fb986800640 1 --2- 172.21.15.2:0/3910928761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb980103e00 0x7fb98019a0d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:47.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.727+0000 7fb986800640 1 -- 172.21.15.2:0/3910928761 >> 172.21.15.2:0/3910928761 conn(0x7fb9800ff5b0 msgr2=0x7fb980108340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:47.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.727+0000 7fb986800640 1 -- 172.21.15.2:0/3910928761 shutdown_connections 2024-09-10T08:06:47.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:47.727+0000 7fb986800640 1 -- 172.21.15.2:0/3910928761 wait complete. 2024-09-10T08:06:47.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:47 smithi155 ceph-mon[35405]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-10T08:06:47.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:47 smithi107 ceph-mon[35048]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-10T08:06:48.476 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T08:06:48.476 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T08:06:48.476 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T08:06:48.556 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T08:06:48.556 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T08:06:48.557 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T08:06:48.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:48 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3910928761' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T08:06:48.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:48 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T08:06:48.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:48 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T08:06:48.808 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:48 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3910928761' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T08:06:48.808 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:48 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T08:06:48.808 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:48 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T08:06:48.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:48 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3910928761' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T08:06:48.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:48 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T08:06:48.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:48 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T08:06:48.972 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:49.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.568+0000 7f2046e00640 1 -- 172.21.15.2:0/1432976219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2040112a30 msgr2=0x7f2040114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:49.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.568+0000 7f2046e00640 1 --2- 172.21.15.2:0/1432976219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2040112a30 0x7f2040114e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2038009ca0 tx=0x7f203802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:06:49.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.568+0000 7f2046e00640 1 -- 172.21.15.2:0/1432976219 shutdown_connections 2024-09-10T08:06:49.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.568+0000 7f2046e00640 1 --2- 172.21.15.2:0/1432976219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2040112a30 0x7f2040114e20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:49.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.568+0000 7f2046e00640 1 --2- 172.21.15.2:0/1432976219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2040077040 0x7f20401124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:49.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.568+0000 7f2046e00640 1 --2- 172.21.15.2:0/1432976219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2040076720 0x7f2040076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:49.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.568+0000 7f2046e00640 1 -- 172.21.15.2:0/1432976219 >> 172.21.15.2:0/1432976219 conn(0x7f20400ffa90 msgr2=0x7f2040101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:49.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.568+0000 7f2046e00640 1 -- 172.21.15.2:0/1432976219 shutdown_connections 2024-09-10T08:06:49.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.568+0000 7f2046e00640 1 -- 172.21.15.2:0/1432976219 wait complete. 2024-09-10T08:06:49.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.569+0000 7f2046e00640 1 Processor -- start 2024-09-10T08:06:49.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.569+0000 7f2046e00640 1 -- start start 2024-09-10T08:06:49.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.570+0000 7f2046e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2040076720 0x7f20401a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:49.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.570+0000 7f2046e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2040077040 0x7f20401a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:49.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.570+0000 7f2046e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2040112a30 0x7f204019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:49.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.570+0000 7f2046e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f20401185e0 con 0x7f2040112a30 2024-09-10T08:06:49.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.570+0000 7f2046e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2040118460 con 0x7f2040076720 2024-09-10T08:06:49.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.570+0000 7f2046e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2040118760 con 0x7f2040077040 2024-09-10T08:06:49.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.570+0000 7f2044e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2040112a30 0x7f204019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:49.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.570+0000 7f203fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2040076720 0x7f20401a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:49.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.570+0000 7f203f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2040077040 0x7f20401a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:49.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.570+0000 7f203fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2040076720 0x7f20401a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37658/0 (socket says 172.21.15.2:37658) 2024-09-10T08:06:49.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.570+0000 7f203fe00640 1 -- 172.21.15.2:0/3858801042 learned_addr learned my addr 172.21.15.2:0/3858801042 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:49.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.571+0000 7f203f400640 1 -- 172.21.15.2:0/3858801042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2040076720 msgr2=0x7f20401a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:49.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.571+0000 7f203f400640 1 --2- 172.21.15.2:0/3858801042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2040076720 0x7f20401a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:49.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.571+0000 7f203f400640 1 -- 172.21.15.2:0/3858801042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2040112a30 msgr2=0x7f204019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:49.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.571+0000 7f203f400640 1 --2- 172.21.15.2:0/3858801042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2040112a30 0x7f204019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:49.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.571+0000 7f203f400640 1 -- 172.21.15.2:0/3858801042 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204019f040 con 0x7f2040077040 2024-09-10T08:06:49.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.571+0000 7f203f400640 1 --2- 172.21.15.2:0/3858801042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2040077040 0x7f20401a4c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f202c00e9f0 tx=0x7f202c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:49.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.571+0000 7f203cc00640 1 -- 172.21.15.2:0/3858801042 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c00ccc0 con 0x7f2040077040 2024-09-10T08:06:49.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.571+0000 7f203cc00640 1 -- 172.21.15.2:0/3858801042 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f202c005020 con 0x7f2040077040 2024-09-10T08:06:49.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.571+0000 7f2046e00640 1 -- 172.21.15.2:0/3858801042 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204019f330 con 0x7f2040077040 2024-09-10T08:06:49.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.571+0000 7f203cc00640 1 -- 172.21.15.2:0/3858801042 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c0049e0 con 0x7f2040077040 2024-09-10T08:06:49.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.571+0000 7f2046e00640 1 -- 172.21.15.2:0/3858801042 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20401ab620 con 0x7f2040077040 2024-09-10T08:06:49.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.573+0000 7f2046e00640 1 -- 172.21.15.2:0/3858801042 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2040076b00 con 0x7f2040077040 2024-09-10T08:06:49.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.575+0000 7f203cc00640 1 -- 172.21.15.2:0/3858801042 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f202c004be0 con 0x7f2040077040 2024-09-10T08:06:49.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.576+0000 7f203cc00640 1 --2- 172.21.15.2:0/3858801042 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20240a2c90 0x7f20240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:49.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.576+0000 7f203cc00640 1 -- 172.21.15.2:0/3858801042 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f202c0cf3f0 con 0x7f2040077040 2024-09-10T08:06:49.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.576+0000 7f203fe00640 1 --2- 172.21.15.2:0/3858801042 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20240a2c90 0x7f20240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:49.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.577+0000 7f203fe00640 1 --2- 172.21.15.2:0/3858801042 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20240a2c90 0x7f20240a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f204019fa50 tx=0x7f2030005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:49.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.581+0000 7f203cc00640 1 -- 172.21.15.2:0/3858801042 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f202c09be30 con 0x7f2040077040 2024-09-10T08:06:49.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:49 smithi002 ceph-mon[30563]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-10T08:06:49.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.814+0000 7f2046e00640 1 -- 172.21.15.2:0/3858801042 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f204010dbc0 con 0x7f2040077040 2024-09-10T08:06:49.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.819+0000 7f203cc00640 1 -- 172.21.15.2:0/3858801042 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263191 (secure 0 0 0) 0x7f202c0051d0 con 0x7f2040077040 2024-09-10T08:06:49.823 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:06:49.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.824+0000 7f2046e00640 1 -- 172.21.15.2:0/3858801042 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20240a2c90 msgr2=0x7f20240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:49.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.824+0000 7f2046e00640 1 --2- 172.21.15.2:0/3858801042 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20240a2c90 0x7f20240a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f204019fa50 tx=0x7f2030005f90 comp rx=0 tx=0).stop 2024-09-10T08:06:49.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.824+0000 7f2046e00640 1 -- 172.21.15.2:0/3858801042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2040077040 msgr2=0x7f20401a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:49.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.824+0000 7f2046e00640 1 --2- 172.21.15.2:0/3858801042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2040077040 0x7f20401a4c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f202c00e9f0 tx=0x7f202c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:06:49.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.824+0000 7f2046e00640 1 -- 172.21.15.2:0/3858801042 shutdown_connections 2024-09-10T08:06:49.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.824+0000 7f2046e00640 1 --2- 172.21.15.2:0/3858801042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2040112a30 0x7f204019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:49.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.824+0000 7f2046e00640 1 --2- 172.21.15.2:0/3858801042 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f20240a2c90 0x7f20240a5150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:49.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.824+0000 7f2046e00640 1 --2- 172.21.15.2:0/3858801042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2040077040 0x7f20401a4c90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:49.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.824+0000 7f2046e00640 1 --2- 172.21.15.2:0/3858801042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2040076720 0x7f20401a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:49.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.824+0000 7f2046e00640 1 -- 172.21.15.2:0/3858801042 >> 172.21.15.2:0/3858801042 conn(0x7f20400ffa90 msgr2=0x7f2040113630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:49.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.824+0000 7f2046e00640 1 -- 172.21.15.2:0/3858801042 shutdown_connections 2024-09-10T08:06:49.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:49.824+0000 7f2046e00640 1 -- 172.21.15.2:0/3858801042 wait complete. 2024-09-10T08:06:49.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:49 smithi155 ceph-mon[35405]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-10T08:06:49.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:49 smithi107 ceph-mon[35048]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-10T08:06:50.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:50 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3858801042' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T08:06:50.791 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3751597725},{"type":"v1","addr":"172.21.15.2:6801","nonce":3751597725}]},"active_addr":"172.21.15.2:6801/3751597725","active_change":"2024-09-10T07:58:03.548889+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.2:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":145571003}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":4119824153}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":1700134961}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":1380984487}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":3234023605}]}]} 2024-09-10T08:06:50.797 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T08:06:50.797 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T08:06:50.797 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:06:50.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:50 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3858801042' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T08:06:50.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:50 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3858801042' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T08:06:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:51.665 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:51 smithi002 ceph-mon[30563]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-09-10T08:06:51.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.778+0000 7f36d1a00640 1 -- 172.21.15.2:0/1512685021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36cc1047e0 msgr2=0x7f36cc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:51.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.778+0000 7f36d1a00640 1 --2- 172.21.15.2:0/1512685021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36cc1047e0 0x7f36cc108830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009900 tx=0x7f36b802a410 comp rx=0 tx=0).stop 2024-09-10T08:06:51.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.778+0000 7f36d1a00640 1 -- 172.21.15.2:0/1512685021 shutdown_connections 2024-09-10T08:06:51.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.778+0000 7f36d1a00640 1 --2- 172.21.15.2:0/1512685021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36cc108f60 0x7f36cc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:51.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.778+0000 7f36d1a00640 1 --2- 172.21.15.2:0/1512685021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36cc1047e0 0x7f36cc108830 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:51.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.778+0000 7f36d1a00640 1 --2- 172.21.15.2:0/1512685021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc103e30 0x7f36cc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:51.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.778+0000 7f36d1a00640 1 -- 172.21.15.2:0/1512685021 >> 172.21.15.2:0/1512685021 conn(0x7f36cc0ff910 msgr2=0x7f36cc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:51.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.778+0000 7f36d1a00640 1 -- 172.21.15.2:0/1512685021 shutdown_connections 2024-09-10T08:06:51.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.779+0000 7f36d1a00640 1 -- 172.21.15.2:0/1512685021 wait complete. 2024-09-10T08:06:51.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.780+0000 7f36d1a00640 1 Processor -- start 2024-09-10T08:06:51.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.780+0000 7f36d1a00640 1 -- start start 2024-09-10T08:06:51.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.780+0000 7f36d1a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36cc103e30 0x7f36cc19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:51.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.780+0000 7f36d1a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36cc1047e0 0x7f36cc19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:51.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.780+0000 7f36d1a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc108f60 0x7f36cc19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:51.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.780+0000 7f36d1a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc114010 con 0x7f36cc103e30 2024-09-10T08:06:51.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.780+0000 7f36d1a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc113e90 con 0x7f36cc108f60 2024-09-10T08:06:51.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.780+0000 7f36d1a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc114190 con 0x7f36cc1047e0 2024-09-10T08:06:51.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.781+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36cc103e30 0x7f36cc19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:51.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.781+0000 7f36cb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36cc1047e0 0x7f36cc19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:51.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.781+0000 7f36d1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc108f60 0x7f36cc19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:51.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.781+0000 7f36d1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc108f60 0x7f36cc19ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37670/0 (socket says 172.21.15.2:37670) 2024-09-10T08:06:51.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.781+0000 7f36cb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36cc1047e0 0x7f36cc19a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42828/0 (socket says 172.21.15.2:42828) 2024-09-10T08:06:51.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.781+0000 7f36d1000640 1 -- 172.21.15.2:0/201500738 learned_addr learned my addr 172.21.15.2:0/201500738 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:51.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.781+0000 7f36d1000640 1 -- 172.21.15.2:0/201500738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36cc1047e0 msgr2=0x7f36cc19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:51.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.781+0000 7f36d1000640 1 --2- 172.21.15.2:0/201500738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36cc1047e0 0x7f36cc19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:51.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.781+0000 7f36d1000640 1 -- 172.21.15.2:0/201500738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36cc103e30 msgr2=0x7f36cc19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:51.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.781+0000 7f36d1000640 1 --2- 172.21.15.2:0/201500738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36cc103e30 0x7f36cc19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:51.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.781+0000 7f36d1000640 1 -- 172.21.15.2:0/201500738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36cc19f420 con 0x7f36cc108f60 2024-09-10T08:06:51.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.781+0000 7f36cb400640 1 --2- 172.21.15.2:0/201500738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36cc1047e0 0x7f36cc19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:06:51.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.782+0000 7f36d1000640 1 --2- 172.21.15.2:0/201500738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc108f60 0x7f36cc19ed40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f36c400eb10 tx=0x7f36c400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:51.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.782+0000 7f36c8c00640 1 -- 172.21.15.2:0/201500738 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4004b20 con 0x7f36cc108f60 2024-09-10T08:06:51.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.782+0000 7f36c8c00640 1 -- 172.21.15.2:0/201500738 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c4004cc0 con 0x7f36cc108f60 2024-09-10T08:06:51.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.782+0000 7f36d1a00640 1 -- 172.21.15.2:0/201500738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36cc19f710 con 0x7f36cc108f60 2024-09-10T08:06:51.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.782+0000 7f36c8c00640 1 -- 172.21.15.2:0/201500738 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4010760 con 0x7f36cc108f60 2024-09-10T08:06:51.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.782+0000 7f36d1a00640 1 -- 172.21.15.2:0/201500738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36cc1a6ff0 con 0x7f36cc108f60 2024-09-10T08:06:51.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.783+0000 7f36d1a00640 1 -- 172.21.15.2:0/201500738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36cc105310 con 0x7f36cc108f60 2024-09-10T08:06:51.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.785+0000 7f36c8c00640 1 -- 172.21.15.2:0/201500738 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f36c4005020 con 0x7f36cc108f60 2024-09-10T08:06:51.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.786+0000 7f36c8c00640 1 --2- 172.21.15.2:0/201500738 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36b00a2c90 0x7f36b00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:51.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.786+0000 7f36c8c00640 1 -- 172.21.15.2:0/201500738 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f36c40cfb60 con 0x7f36cc108f60 2024-09-10T08:06:51.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.786+0000 7f36cbe00640 1 --2- 172.21.15.2:0/201500738 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36b00a2c90 0x7f36b00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:51.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.787+0000 7f36cbe00640 1 --2- 172.21.15.2:0/201500738 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36b00a2c90 0x7f36b00a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009bd0 tx=0x7f36bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:51.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.790+0000 7f36c8c00640 1 -- 172.21.15.2:0/201500738 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f36c409c5a0 con 0x7f36cc108f60 2024-09-10T08:06:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:51 smithi155 ceph-mon[35405]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-09-10T08:06:51.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:51 smithi107 ceph-mon[35048]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-09-10T08:06:51.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.976+0000 7f36d1a00640 1 -- 172.21.15.2:0/201500738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36cc10bbb0 con 0x7f36cc108f60 2024-09-10T08:06:51.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.978+0000 7f36c8c00640 1 -- 172.21.15.2:0/201500738 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24653 (secure 0 0 0) 0x7f36c409bea0 con 0x7f36cc108f60 2024-09-10T08:06:51.978 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:06:51.978 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":93,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:06:04.721464+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:06:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.981+0000 7f36d1a00640 1 -- 172.21.15.2:0/201500738 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36b00a2c90 msgr2=0x7f36b00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.981+0000 7f36d1a00640 1 --2- 172.21.15.2:0/201500738 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36b00a2c90 0x7f36b00a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009bd0 tx=0x7f36bc005f90 comp rx=0 tx=0).stop 2024-09-10T08:06:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.981+0000 7f36d1a00640 1 -- 172.21.15.2:0/201500738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc108f60 msgr2=0x7f36cc19ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.981+0000 7f36d1a00640 1 --2- 172.21.15.2:0/201500738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc108f60 0x7f36cc19ed40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f36c400eb10 tx=0x7f36c400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:06:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.981+0000 7f36d1a00640 1 -- 172.21.15.2:0/201500738 shutdown_connections 2024-09-10T08:06:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.981+0000 7f36d1a00640 1 --2- 172.21.15.2:0/201500738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc108f60 0x7f36cc19ed40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.981+0000 7f36d1a00640 1 --2- 172.21.15.2:0/201500738 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36b00a2c90 0x7f36b00a5150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.981+0000 7f36d1a00640 1 --2- 172.21.15.2:0/201500738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36cc1047e0 0x7f36cc19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.981+0000 7f36d1a00640 1 --2- 172.21.15.2:0/201500738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36cc103e30 0x7f36cc19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.981+0000 7f36d1a00640 1 -- 172.21.15.2:0/201500738 >> 172.21.15.2:0/201500738 conn(0x7f36cc0ff910 msgr2=0x7f36cc100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.981+0000 7f36d1a00640 1 -- 172.21.15.2:0/201500738 shutdown_connections 2024-09-10T08:06:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:51.981+0000 7f36d1a00640 1 -- 172.21.15.2:0/201500738 wait complete. 2024-09-10T08:06:52.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:52 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/201500738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:52.818 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T08:06:52.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:06:52.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:52 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/201500738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:52.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:52 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/201500738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:53.196 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:53.656 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:53 smithi002 ceph-mon[30563]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-10T08:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.770+0000 7f7d3e600640 1 -- 172.21.15.2:0/1625674299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d38102360 msgr2=0x7f7d3810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.770+0000 7f7d3e600640 1 --2- 172.21.15.2:0/1625674299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d38102360 0x7f7d3810e890 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009cd0 tx=0x7f7d3002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.770+0000 7f7d3e600640 1 -- 172.21.15.2:0/1625674299 shutdown_connections 2024-09-10T08:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.770+0000 7f7d3e600640 1 --2- 172.21.15.2:0/1625674299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d38102360 0x7f7d3810e890 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.770+0000 7f7d3e600640 1 --2- 172.21.15.2:0/1625674299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d381019c0 0x7f7d38101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.770+0000 7f7d3e600640 1 --2- 172.21.15.2:0/1625674299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d381079c0 0x7f7d38107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.770+0000 7f7d3e600640 1 -- 172.21.15.2:0/1625674299 >> 172.21.15.2:0/1625674299 conn(0x7f7d380fd740 msgr2=0x7f7d380ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.770+0000 7f7d3e600640 1 -- 172.21.15.2:0/1625674299 shutdown_connections 2024-09-10T08:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.770+0000 7f7d3e600640 1 -- 172.21.15.2:0/1625674299 wait complete. 2024-09-10T08:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.771+0000 7f7d3e600640 1 Processor -- start 2024-09-10T08:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.771+0000 7f7d3e600640 1 -- start start 2024-09-10T08:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.771+0000 7f7d3e600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d381019c0 0x7f7d380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.771+0000 7f7d3e600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d38102360 0x7f7d38074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d3e600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d381079c0 0x7f7d38074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d3e600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7d38111e10 con 0x7f7d381019c0 2024-09-10T08:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d3e600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7d38111c90 con 0x7f7d38102360 2024-09-10T08:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d3e600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d38111f90 con 0x7f7d381079c0 2024-09-10T08:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d3d200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d381019c0 0x7f7d380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d3d200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d381019c0 0x7f7d380744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39004/0 (socket says 172.21.15.2:39004) 2024-09-10T08:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d3d200640 1 -- 172.21.15.2:0/4228114286 learned_addr learned my addr 172.21.15.2:0/4228114286 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d37e00640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d38102360 0x7f7d38074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d3dc00640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d381079c0 0x7f7d38074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d3d200640 1 -- 172.21.15.2:0/4228114286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d381079c0 msgr2=0x7f7d38074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d3d200640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d381079c0 0x7f7d38074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d3d200640 1 -- 172.21.15.2:0/4228114286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d38102360 msgr2=0x7f7d38074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d3d200640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d38102360 0x7f7d38074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d3d200640 1 -- 172.21.15.2:0/4228114286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d381a6750 con 0x7f7d381019c0 2024-09-10T08:06:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.772+0000 7f7d37e00640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d38102360 0x7f7d38074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:06:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.773+0000 7f7d3d200640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d381019c0 0x7f7d380744f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e9f0 tx=0x7f7d2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.773+0000 7f7d35600640 1 -- 172.21.15.2:0/4228114286 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00cd80 con 0x7f7d381019c0 2024-09-10T08:06:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.773+0000 7f7d3e600640 1 -- 172.21.15.2:0/4228114286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d381a6a40 con 0x7f7d381019c0 2024-09-10T08:06:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.773+0000 7f7d35600640 1 -- 172.21.15.2:0/4228114286 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c005500 con 0x7f7d381019c0 2024-09-10T08:06:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.773+0000 7f7d35600640 1 -- 172.21.15.2:0/4228114286 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c010690 con 0x7f7d381019c0 2024-09-10T08:06:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.773+0000 7f7d3e600640 1 -- 172.21.15.2:0/4228114286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d381a6f30 con 0x7f7d381019c0 2024-09-10T08:06:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.774+0000 7f7d3e600640 1 -- 172.21.15.2:0/4228114286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d38101e20 con 0x7f7d381019c0 2024-09-10T08:06:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.777+0000 7f7d35600640 1 -- 172.21.15.2:0/4228114286 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7d2c0046c0 con 0x7f7d381019c0 2024-09-10T08:06:53.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.778+0000 7f7d35600640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d200a2c10 0x7f7d200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:53.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.778+0000 7f7d35600640 1 -- 172.21.15.2:0/4228114286 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f7d2c0cf930 con 0x7f7d381019c0 2024-09-10T08:06:53.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.778+0000 7f7d37e00640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d200a2c10 0x7f7d200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:53.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.779+0000 7f7d37e00640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d200a2c10 0x7f7d200a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7d38075f80 tx=0x7f7d24005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:53.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.782+0000 7f7d35600640 1 -- 172.21.15.2:0/4228114286 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d2c09c370 con 0x7f7d381019c0 2024-09-10T08:06:53.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:53 smithi155 ceph-mon[35405]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-10T08:06:53.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:53 smithi107 ceph-mon[35048]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-10T08:06:53.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.970+0000 7f7d3e600640 1 -- 172.21.15.2:0/4228114286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d381099a0 con 0x7f7d381019c0 2024-09-10T08:06:53.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.972+0000 7f7d35600640 1 -- 172.21.15.2:0/4228114286 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24653 (secure 0 0 0) 0x7f7d2c09bc70 con 0x7f7d381019c0 2024-09-10T08:06:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:06:53.972 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":93,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:06:04.721464+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:06:53.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.974+0000 7f7d3e600640 1 -- 172.21.15.2:0/4228114286 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d200a2c10 msgr2=0x7f7d200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:53.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.974+0000 7f7d3e600640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d200a2c10 0x7f7d200a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7d38075f80 tx=0x7f7d24005f90 comp rx=0 tx=0).stop 2024-09-10T08:06:53.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.974+0000 7f7d3e600640 1 -- 172.21.15.2:0/4228114286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d381019c0 msgr2=0x7f7d380744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:53.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.974+0000 7f7d3e600640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d381019c0 0x7f7d380744f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e9f0 tx=0x7f7d2c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:06:53.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.975+0000 7f7d3e600640 1 -- 172.21.15.2:0/4228114286 shutdown_connections 2024-09-10T08:06:53.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.975+0000 7f7d3e600640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d200a2c10 0x7f7d200a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:53.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.975+0000 7f7d3e600640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d381079c0 0x7f7d38074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:53.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.975+0000 7f7d3e600640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d38102360 0x7f7d38074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:53.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.975+0000 7f7d3e600640 1 --2- 172.21.15.2:0/4228114286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d381019c0 0x7f7d380744f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:53.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.975+0000 7f7d3e600640 1 -- 172.21.15.2:0/4228114286 >> 172.21.15.2:0/4228114286 conn(0x7f7d380fd740 msgr2=0x7f7d380fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:53.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.975+0000 7f7d3e600640 1 -- 172.21.15.2:0/4228114286 shutdown_connections 2024-09-10T08:06:53.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:53.975+0000 7f7d3e600640 1 -- 172.21.15.2:0/4228114286 wait complete. 2024-09-10T08:06:54.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:54 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/4228114286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:54.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:54 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/4228114286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:54.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:54 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/4228114286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:06:54.982 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T08:06:54.982 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T08:06:54.983 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T08:06:54.983 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T08:06:54.984 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T08:06:54.984 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T08:06:54.984 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T08:06:54.985 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T08:06:54.985 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T08:06:54.986 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T08:06:54.986 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T08:06:54.986 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T08:06:55.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:55 smithi002 ceph-mon[30563]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:06:55.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:55 smithi155 ceph-mon[35405]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:06:55.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:55 smithi107 ceph-mon[35048]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:06:55.983 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:56.140 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:56.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.866+0000 7f2672800640 1 -- 172.21.15.2:0/3197638151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c103e30 msgr2=0x7f266c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:56.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.866+0000 7f2672800640 1 --2- 172.21.15.2:0/3197638151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c103e30 0x7f266c104210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2660009900 tx=0x7f266002a410 comp rx=0 tx=0).stop 2024-09-10T08:06:56.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.866+0000 7f2672800640 1 -- 172.21.15.2:0/3197638151 shutdown_connections 2024-09-10T08:06:56.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.866+0000 7f2672800640 1 --2- 172.21.15.2:0/3197638151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f266c108f60 0x7f266c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:56.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.866+0000 7f2672800640 1 --2- 172.21.15.2:0/3197638151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266c1047e0 0x7f266c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:56.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.866+0000 7f2672800640 1 --2- 172.21.15.2:0/3197638151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c103e30 0x7f266c104210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:56.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.866+0000 7f2672800640 1 -- 172.21.15.2:0/3197638151 >> 172.21.15.2:0/3197638151 conn(0x7f266c0ff8d0 msgr2=0x7f266c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:56.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.866+0000 7f2672800640 1 -- 172.21.15.2:0/3197638151 shutdown_connections 2024-09-10T08:06:56.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.866+0000 7f2672800640 1 -- 172.21.15.2:0/3197638151 wait complete. 2024-09-10T08:06:56.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.867+0000 7f2672800640 1 Processor -- start 2024-09-10T08:06:56.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.867+0000 7f2672800640 1 -- start start 2024-09-10T08:06:56.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.868+0000 7f2672800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266c103e30 0x7f266c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:56.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.868+0000 7f2672800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c1047e0 0x7f266c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:56.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.868+0000 7f2672800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f266c108f60 0x7f266c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:56.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.868+0000 7f2672800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f266c113fc0 con 0x7f266c1047e0 2024-09-10T08:06:56.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.868+0000 7f2672800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f266c113e40 con 0x7f266c108f60 2024-09-10T08:06:56.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.868+0000 7f2672800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f266c114140 con 0x7f266c103e30 2024-09-10T08:06:56.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.868+0000 7f2671400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266c103e30 0x7f266c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:56.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.868+0000 7f2670a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c1047e0 0x7f266c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:56.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.868+0000 7f2671400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266c103e30 0x7f266c19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36934/0 (socket says 172.21.15.2:36934) 2024-09-10T08:06:56.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.868+0000 7f2671400640 1 -- 172.21.15.2:0/557916599 learned_addr learned my addr 172.21.15.2:0/557916599 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:56.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.868+0000 7f2671e00640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f266c108f60 0x7f266c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:56.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.869+0000 7f2670a00640 1 -- 172.21.15.2:0/557916599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266c103e30 msgr2=0x7f266c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:56.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.869+0000 7f2670a00640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266c103e30 0x7f266c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:56.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.869+0000 7f2670a00640 1 -- 172.21.15.2:0/557916599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f266c108f60 msgr2=0x7f266c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:56.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.869+0000 7f2670a00640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f266c108f60 0x7f266c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:56.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.869+0000 7f2670a00640 1 -- 172.21.15.2:0/557916599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266c19f430 con 0x7f266c1047e0 2024-09-10T08:06:56.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.869+0000 7f2671e00640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f266c108f60 0x7f266c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:06:56.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.869+0000 7f2670a00640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c1047e0 0x7f266c19a9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f265800e9f0 tx=0x7f265800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:56.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.869+0000 7f266a000640 1 -- 172.21.15.2:0/557916599 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265800ccc0 con 0x7f266c1047e0 2024-09-10T08:06:56.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.869+0000 7f266a000640 1 -- 172.21.15.2:0/557916599 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2658005020 con 0x7f266c1047e0 2024-09-10T08:06:56.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.869+0000 7f266a000640 1 -- 172.21.15.2:0/557916599 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580049e0 con 0x7f266c1047e0 2024-09-10T08:06:56.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.869+0000 7f2672800640 1 -- 172.21.15.2:0/557916599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f266c19f720 con 0x7f266c1047e0 2024-09-10T08:06:56.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.869+0000 7f2672800640 1 -- 172.21.15.2:0/557916599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f266c1a7000 con 0x7f266c1047e0 2024-09-10T08:06:56.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.870+0000 7f2672800640 1 -- 172.21.15.2:0/557916599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f266c062980 con 0x7f266c1047e0 2024-09-10T08:06:56.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.872+0000 7f266a000640 1 -- 172.21.15.2:0/557916599 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2658005560 con 0x7f266c1047e0 2024-09-10T08:06:56.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.874+0000 7f266a000640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f26540a2c90 0x7f26540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.874+0000 7f266a000640 1 -- 172.21.15.2:0/557916599 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f26580cf750 con 0x7f266c1047e0 2024-09-10T08:06:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.874+0000 7f2671400640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f26540a2c90 0x7f26540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.874+0000 7f266a000640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994] conn(0x7f26540ade60 0x7f26540b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.874+0000 7f266a000640 1 -- 172.21.15.2:0/557916599 --> [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f26540b09d0 con 0x7f26540ade60 2024-09-10T08:06:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.874+0000 7f266a000640 1 -- 172.21.15.2:0/557916599 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f26580cfb00 con 0x7f266c1047e0 2024-09-10T08:06:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.875+0000 7f2671e00640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994] conn(0x7f26540ade60 0x7f26540b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:56.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.875+0000 7f2671400640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f26540a2c90 0x7f26540a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f26600096c0 tx=0x7f2660005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:56.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.875+0000 7f2671e00640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994] conn(0x7f26540ade60 0x7f26540b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:56.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.876+0000 7f266a000640 1 -- 172.21.15.2:0/557916599 <== osd.7 v2:172.21.15.107:6816/1479477994 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f26540b09d0 con 0x7f26540ade60 2024-09-10T08:06:56.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.890+0000 7f2672800640 1 -- 172.21.15.2:0/557916599 --> [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f266c062e00 con 0x7f26540ade60 2024-09-10T08:06:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.891+0000 7f266a000640 1 -- 172.21.15.2:0/557916599 <== osd.7 v2:172.21.15.107:6816/1479477994 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f266c062e00 con 0x7f26540ade60 2024-09-10T08:06:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.891+0000 7f2672800640 1 -- 172.21.15.2:0/557916599 >> [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994] conn(0x7f26540ade60 msgr2=0x7f26540b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.891+0000 7f2672800640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994] conn(0x7f26540ade60 0x7f26540b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.892+0000 7f2672800640 1 -- 172.21.15.2:0/557916599 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f26540a2c90 msgr2=0x7f26540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.892+0000 7f2672800640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f26540a2c90 0x7f26540a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f26600096c0 tx=0x7f2660005a20 comp rx=0 tx=0).stop 2024-09-10T08:06:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.892+0000 7f2672800640 1 -- 172.21.15.2:0/557916599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c1047e0 msgr2=0x7f266c19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.892+0000 7f2672800640 1 --2- 172.21.15.2:0/557916599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c1047e0 0x7f266c19a9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f265800e9f0 tx=0x7f265800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:06:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.892+0000 7f2671e00640 1 -- 172.21.15.2:0/557916599 reap_dead start 2024-09-10T08:06:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.892+0000 7f2672800640 1 -- 172.21.15.2:0/557916599 shutdown_connections 2024-09-10T08:06:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.892+0000 7f2672800640 1 -- 172.21.15.2:0/557916599 >> 172.21.15.2:0/557916599 conn(0x7f266c0ff8d0 msgr2=0x7f266c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.892+0000 7f2672800640 1 -- 172.21.15.2:0/557916599 shutdown_connections 2024-09-10T08:06:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:56.892+0000 7f2672800640 1 -- 172.21.15.2:0/557916599 wait complete. 2024-09-10T08:06:57.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.065+0000 7f2c28a00640 1 -- 172.21.15.2:0/3345450737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24102790 msgr2=0x7f2c241067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:57.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.065+0000 7f2c28a00640 1 --2- 172.21.15.2:0/3345450737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24102790 0x7f2c241067e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009900 tx=0x7f2c0c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:06:57.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.066+0000 7f2c28a00640 1 -- 172.21.15.2:0/3345450737 shutdown_connections 2024-09-10T08:06:57.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.066+0000 7f2c28a00640 1 --2- 172.21.15.2:0/3345450737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c24106f10 0x7f2c2410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:57.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.066+0000 7f2c28a00640 1 --2- 172.21.15.2:0/3345450737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24102790 0x7f2c241067e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:57.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.066+0000 7f2c28a00640 1 --2- 172.21.15.2:0/3345450737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24101de0 0x7f2c241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.066+0000 7f2c28a00640 1 -- 172.21.15.2:0/3345450737 >> 172.21.15.2:0/3345450737 conn(0x7f2c240fd8c0 msgr2=0x7f2c240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.066+0000 7f2c28a00640 1 -- 172.21.15.2:0/3345450737 shutdown_connections 2024-09-10T08:06:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.066+0000 7f2c28a00640 1 -- 172.21.15.2:0/3345450737 wait complete. 2024-09-10T08:06:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.067+0000 7f2c28a00640 1 Processor -- start 2024-09-10T08:06:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.067+0000 7f2c28a00640 1 -- start start 2024-09-10T08:06:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.067+0000 7f2c28a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c24101de0 0x7f2c2419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.067+0000 7f2c28a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24102790 0x7f2c2419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.067+0000 7f2c28a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24106f10 0x7f2c2419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.067+0000 7f2c28a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c24111f30 con 0x7f2c24106f10 2024-09-10T08:06:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.067+0000 7f2c28a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c24111db0 con 0x7f2c24102790 2024-09-10T08:06:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.067+0000 7f2c28a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2c241120b0 con 0x7f2c24101de0 2024-09-10T08:06:57.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.068+0000 7f2c22800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24106f10 0x7f2c2419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.068+0000 7f2c22800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24106f10 0x7f2c2419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45750/0 (socket says 172.21.15.2:45750) 2024-09-10T08:06:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.068+0000 7f2c22800640 1 -- 172.21.15.2:0/3761705018 learned_addr learned my addr 172.21.15.2:0/3761705018 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.068+0000 7f2c21e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c24101de0 0x7f2c2419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.068+0000 7f2c19400640 1 --2- 172.21.15.2:0/3761705018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24102790 0x7f2c2419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.068+0000 7f2c22800640 1 -- 172.21.15.2:0/3761705018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c24101de0 msgr2=0x7f2c2419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.068+0000 7f2c22800640 1 --2- 172.21.15.2:0/3761705018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c24101de0 0x7f2c2419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.068+0000 7f2c22800640 1 -- 172.21.15.2:0/3761705018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24102790 msgr2=0x7f2c2419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.068+0000 7f2c22800640 1 --2- 172.21.15.2:0/3761705018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24102790 0x7f2c2419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.068+0000 7f2c22800640 1 -- 172.21.15.2:0/3761705018 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2419f600 con 0x7f2c24106f10 2024-09-10T08:06:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.068+0000 7f2c22800640 1 --2- 172.21.15.2:0/3761705018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24106f10 0x7f2c2419ef20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00eb10 tx=0x7f2c1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:57.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.069+0000 7f2c1b400640 1 -- 172.21.15.2:0/3761705018 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c004b20 con 0x7f2c24106f10 2024-09-10T08:06:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.069+0000 7f2c1b400640 1 -- 172.21.15.2:0/3761705018 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c1c004cc0 con 0x7f2c24106f10 2024-09-10T08:06:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.069+0000 7f2c1b400640 1 -- 172.21.15.2:0/3761705018 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c010760 con 0x7f2c24106f10 2024-09-10T08:06:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.069+0000 7f2c28a00640 1 -- 172.21.15.2:0/3761705018 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2419f8f0 con 0x7f2c24106f10 2024-09-10T08:06:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.069+0000 7f2c28a00640 1 -- 172.21.15.2:0/3761705018 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c241a71d0 con 0x7f2c24106f10 2024-09-10T08:06:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.070+0000 7f2c28a00640 1 -- 172.21.15.2:0/3761705018 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2c24075780 con 0x7f2c24106f10 2024-09-10T08:06:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.072+0000 7f2c1b400640 1 -- 172.21.15.2:0/3761705018 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2c1c005020 con 0x7f2c24106f10 2024-09-10T08:06:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.073+0000 7f2c1b400640 1 --2- 172.21.15.2:0/3761705018 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c080a2c90 0x7f2c080a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.073+0000 7f2c1b400640 1 -- 172.21.15.2:0/3761705018 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f2c1c0cfde0 con 0x7f2c24106f10 2024-09-10T08:06:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.073+0000 7f2c21e00640 1 --2- 172.21.15.2:0/3761705018 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c080a2c90 0x7f2c080a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.073+0000 7f2c1b400640 1 --2- 172.21.15.2:0/3761705018 >> [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864] conn(0x7f2c080ade60 0x7f2c080b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.074+0000 7f2c1b400640 1 -- 172.21.15.2:0/3761705018 --> [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c080b09d0 con 0x7f2c080ade60 2024-09-10T08:06:57.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.074+0000 7f2c1b400640 1 -- 172.21.15.2:0/3761705018 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f2c1c0d0230 con 0x7f2c24106f10 2024-09-10T08:06:57.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.074+0000 7f2c19400640 1 --2- 172.21.15.2:0/3761705018 >> [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864] conn(0x7f2c080ade60 0x7f2c080b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:57.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.074+0000 7f2c21e00640 1 --2- 172.21.15.2:0/3761705018 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c080a2c90 0x7f2c080a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009bd0 tx=0x7f2c14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:57.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.075+0000 7f2c19400640 1 --2- 172.21.15.2:0/3761705018 >> [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864] conn(0x7f2c080ade60 0x7f2c080b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:57.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.076+0000 7f2c1b400640 1 -- 172.21.15.2:0/3761705018 <== osd.8 v2:172.21.15.155:6816/359284864 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c080b09d0 con 0x7f2c080ade60 2024-09-10T08:06:57.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.090+0000 7f2c28a00640 1 -- 172.21.15.2:0/3761705018 --> [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2c241032c0 con 0x7f2c080ade60 2024-09-10T08:06:57.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.091+0000 7f2c1b400640 1 -- 172.21.15.2:0/3761705018 <== osd.8 v2:172.21.15.155:6816/359284864 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2c241032c0 con 0x7f2c080ade60 2024-09-10T08:06:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.091+0000 7f2c28a00640 1 -- 172.21.15.2:0/3761705018 >> [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864] conn(0x7f2c080ade60 msgr2=0x7f2c080b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.091+0000 7f2c28a00640 1 --2- 172.21.15.2:0/3761705018 >> [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864] conn(0x7f2c080ade60 0x7f2c080b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.091+0000 7f2c28a00640 1 -- 172.21.15.2:0/3761705018 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c080a2c90 msgr2=0x7f2c080a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.091+0000 7f2c28a00640 1 --2- 172.21.15.2:0/3761705018 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c080a2c90 0x7f2c080a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009bd0 tx=0x7f2c14005f90 comp rx=0 tx=0).stop 2024-09-10T08:06:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.091+0000 7f2c28a00640 1 -- 172.21.15.2:0/3761705018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24106f10 msgr2=0x7f2c2419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.091+0000 7f2c28a00640 1 --2- 172.21.15.2:0/3761705018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c24106f10 0x7f2c2419ef20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00eb10 tx=0x7f2c1c00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:06:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.092+0000 7f2c22800640 1 -- 172.21.15.2:0/3761705018 reap_dead start 2024-09-10T08:06:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.092+0000 7f2c28a00640 1 -- 172.21.15.2:0/3761705018 shutdown_connections 2024-09-10T08:06:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.092+0000 7f2c28a00640 1 -- 172.21.15.2:0/3761705018 >> 172.21.15.2:0/3761705018 conn(0x7f2c240fd8c0 msgr2=0x7f2c240fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.092+0000 7f2c28a00640 1 -- 172.21.15.2:0/3761705018 shutdown_connections 2024-09-10T08:06:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:57.092+0000 7f2c28a00640 1 -- 172.21.15.2:0/3761705018 wait complete. 2024-09-10T08:06:57.488 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:57.489 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:57 smithi002 ceph-mon[30563]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:06:57.491 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:57.492 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:57.494 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:57.500 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:57.507 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:06:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:57 smithi155 ceph-mon[35405]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:06:57.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:57 smithi107 ceph-mon[35048]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:06:58.304 INFO:teuthology.orchestra.run.smithi002.stdout:244813135921 2024-09-10T08:06:58.304 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T08:06:59.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.856+0000 7f694aa00640 1 -- 172.21.15.2:0/3973439879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6944103e30 msgr2=0x7f6944104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:59.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.856+0000 7f694aa00640 1 --2- 172.21.15.2:0/3973439879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6944103e30 0x7f6944104210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6938009900 tx=0x7f693802a410 comp rx=0 tx=0).stop 2024-09-10T08:06:59.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.856+0000 7f694aa00640 1 -- 172.21.15.2:0/3973439879 shutdown_connections 2024-09-10T08:06:59.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.856+0000 7f694aa00640 1 --2- 172.21.15.2:0/3973439879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6944108f60 0x7f6944110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:59.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.856+0000 7f694aa00640 1 --2- 172.21.15.2:0/3973439879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69441047e0 0x7f6944108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.856+0000 7f694aa00640 1 --2- 172.21.15.2:0/3973439879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6944103e30 0x7f6944104210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.856+0000 7f694aa00640 1 -- 172.21.15.2:0/3973439879 >> 172.21.15.2:0/3973439879 conn(0x7f69440ff910 msgr2=0x7f6944101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.856+0000 7f694aa00640 1 -- 172.21.15.2:0/3973439879 shutdown_connections 2024-09-10T08:06:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.857+0000 7f694aa00640 1 -- 172.21.15.2:0/3973439879 wait complete. 2024-09-10T08:06:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.858+0000 7f694aa00640 1 Processor -- start 2024-09-10T08:06:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.858+0000 7f694aa00640 1 -- start start 2024-09-10T08:06:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.858+0000 7f694aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6944103e30 0x7f694419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.858+0000 7f694aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69441047e0 0x7f694419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.858+0000 7f694aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6944108f60 0x7f694419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.858+0000 7f694aa00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6944113fc0 con 0x7f6944108f60 2024-09-10T08:06:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.858+0000 7f694aa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6944113e40 con 0x7f6944103e30 2024-09-10T08:06:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.858+0000 7f694aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6944114140 con 0x7f69441047e0 2024-09-10T08:06:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.858+0000 7f694a000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6944108f60 0x7f694419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.858+0000 7f694a000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6944108f60 0x7f694419ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45768/0 (socket says 172.21.15.2:45768) 2024-09-10T08:06:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.858+0000 7f694a000640 1 -- 172.21.15.2:0/1068876336 learned_addr learned my addr 172.21.15.2:0/1068876336 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:06:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.859+0000 7f6949600640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6944103e30 0x7f694419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.859+0000 7f6948c00640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69441047e0 0x7f694419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.859+0000 7f694a000640 1 -- 172.21.15.2:0/1068876336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69441047e0 msgr2=0x7f694419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.859+0000 7f694a000640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69441047e0 0x7f694419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.859+0000 7f694a000640 1 -- 172.21.15.2:0/1068876336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6944103e30 msgr2=0x7f694419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.859+0000 7f694a000640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6944103e30 0x7f694419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.859+0000 7f694a000640 1 -- 172.21.15.2:0/1068876336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694419f3d0 con 0x7f6944108f60 2024-09-10T08:06:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.859+0000 7f694a000640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6944108f60 0x7f694419ecf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f693c00eb10 tx=0x7f693c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.859+0000 7f6942000640 1 -- 172.21.15.2:0/1068876336 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c004b20 con 0x7f6944108f60 2024-09-10T08:06:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.860+0000 7f6942000640 1 -- 172.21.15.2:0/1068876336 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f693c004cc0 con 0x7f6944108f60 2024-09-10T08:06:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.860+0000 7f6942000640 1 -- 172.21.15.2:0/1068876336 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c010760 con 0x7f6944108f60 2024-09-10T08:06:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.860+0000 7f694aa00640 1 -- 172.21.15.2:0/1068876336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694419f6c0 con 0x7f6944108f60 2024-09-10T08:06:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.860+0000 7f694aa00640 1 -- 172.21.15.2:0/1068876336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69441a6fa0 con 0x7f6944108f60 2024-09-10T08:06:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.861+0000 7f694aa00640 1 -- 172.21.15.2:0/1068876336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6944062940 con 0x7f6944108f60 2024-09-10T08:06:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.863+0000 7f6942000640 1 -- 172.21.15.2:0/1068876336 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f693c005000 con 0x7f6944108f60 2024-09-10T08:06:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.864+0000 7f6942000640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f692c0a2c90 0x7f692c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.864+0000 7f6942000640 1 -- 172.21.15.2:0/1068876336 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f693c0d03d0 con 0x7f6944108f60 2024-09-10T08:06:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.864+0000 7f6949600640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f692c0a2c90 0x7f692c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.864+0000 7f6942000640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241] conn(0x7f692c0ade60 0x7f692c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:06:59.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.864+0000 7f6942000640 1 -- 172.21.15.2:0/1068876336 --> [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f693c01aec0 con 0x7f692c0ade60 2024-09-10T08:06:59.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.864+0000 7f6942000640 1 -- 172.21.15.2:0/1068876336 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f693c0d0780 con 0x7f6944108f60 2024-09-10T08:06:59.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.865+0000 7f6948c00640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241] conn(0x7f692c0ade60 0x7f692c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:06:59.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.865+0000 7f6949600640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f692c0a2c90 0x7f692c0a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6938009230 tx=0x7f69380058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:59.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.865+0000 7f6948c00640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241] conn(0x7f692c0ade60 0x7f692c0b0300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:06:59.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.866+0000 7f6942000640 1 -- 172.21.15.2:0/1068876336 <== osd.11 v2:172.21.15.155:6824/4052924241 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f693c01aec0 con 0x7f692c0ade60 2024-09-10T08:06:59.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.881+0000 7f694aa00640 1 -- 172.21.15.2:0/1068876336 --> [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6944062e10 con 0x7f692c0ade60 2024-09-10T08:06:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.882+0000 7f6942000640 1 -- 172.21.15.2:0/1068876336 <== osd.11 v2:172.21.15.155:6824/4052924241 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6944062e10 con 0x7f692c0ade60 2024-09-10T08:06:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.883+0000 7f694aa00640 1 -- 172.21.15.2:0/1068876336 >> [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241] conn(0x7f692c0ade60 msgr2=0x7f692c0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.883+0000 7f694aa00640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241] conn(0x7f692c0ade60 0x7f692c0b0300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:06:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.883+0000 7f694aa00640 1 -- 172.21.15.2:0/1068876336 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f692c0a2c90 msgr2=0x7f692c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.883+0000 7f694aa00640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f692c0a2c90 0x7f692c0a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6938009230 tx=0x7f69380058d0 comp rx=0 tx=0).stop 2024-09-10T08:06:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.883+0000 7f694aa00640 1 -- 172.21.15.2:0/1068876336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6944108f60 msgr2=0x7f694419ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:06:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.883+0000 7f694aa00640 1 --2- 172.21.15.2:0/1068876336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6944108f60 0x7f694419ecf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f693c00eb10 tx=0x7f693c00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:06:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.883+0000 7f694a000640 1 -- 172.21.15.2:0/1068876336 reap_dead start 2024-09-10T08:06:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.883+0000 7f694aa00640 1 -- 172.21.15.2:0/1068876336 shutdown_connections 2024-09-10T08:06:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.883+0000 7f694aa00640 1 -- 172.21.15.2:0/1068876336 >> 172.21.15.2:0/1068876336 conn(0x7f69440ff910 msgr2=0x7f6944100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:06:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.883+0000 7f694aa00640 1 -- 172.21.15.2:0/1068876336 shutdown_connections 2024-09-10T08:06:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:06:58.883+0000 7f694aa00640 1 -- 172.21.15.2:0/1068876336 wait complete. 2024-09-10T08:06:59.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:06:59 smithi002 ceph-mon[30563]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:06:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:06:59 smithi155 ceph-mon[35405]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:06:59.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:06:59 smithi107 ceph-mon[35048]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:01.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.789+0000 7f59f2a00640 1 -- 172.21.15.2:0/2345788422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec1019c0 msgr2=0x7f59ec101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:01.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.789+0000 7f59f2a00640 1 --2- 172.21.15.2:0/2345788422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec1019c0 0x7f59ec101e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009930 tx=0x7f59d802a420 comp rx=0 tx=0).stop 2024-09-10T08:07:01.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.789+0000 7f59f2a00640 1 -- 172.21.15.2:0/2345788422 shutdown_connections 2024-09-10T08:07:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.789+0000 7f59f2a00640 1 --2- 172.21.15.2:0/2345788422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec102360 0x7f59ec10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.789+0000 7f59f2a00640 1 --2- 172.21.15.2:0/2345788422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec1019c0 0x7f59ec101e20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.789+0000 7f59f2a00640 1 --2- 172.21.15.2:0/2345788422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59ec1079c0 0x7f59ec107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.789+0000 7f59f2a00640 1 -- 172.21.15.2:0/2345788422 >> 172.21.15.2:0/2345788422 conn(0x7f59ec0fd740 msgr2=0x7f59ec0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.789+0000 7f59f2a00640 1 -- 172.21.15.2:0/2345788422 shutdown_connections 2024-09-10T08:07:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.789+0000 7f59f2a00640 1 -- 172.21.15.2:0/2345788422 wait complete. 2024-09-10T08:07:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.790+0000 7f59f2a00640 1 Processor -- start 2024-09-10T08:07:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.791+0000 7f59f2a00640 1 -- start start 2024-09-10T08:07:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.791+0000 7f59f2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec1019c0 0x7f59ec19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.791+0000 7f59f2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec102360 0x7f59ec19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.791+0000 7f59f2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59ec1079c0 0x7f59ec19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.791+0000 7f59f2a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec111e10 con 0x7f59ec102360 2024-09-10T08:07:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.791+0000 7f59f2a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec111c90 con 0x7f59ec1019c0 2024-09-10T08:07:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.791+0000 7f59f2a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec111f90 con 0x7f59ec1079c0 2024-09-10T08:07:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.791+0000 7f59f0c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec102360 0x7f59ec19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.791+0000 7f59f1600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec1019c0 0x7f59ec19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.792+0000 7f59f0c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec102360 0x7f59ec19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45794/0 (socket says 172.21.15.2:45794) 2024-09-10T08:07:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.792+0000 7f59f0c00640 1 -- 172.21.15.2:0/4178548724 learned_addr learned my addr 172.21.15.2:0/4178548724 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.792+0000 7f59f2000640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59ec1079c0 0x7f59ec19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.792+0000 7f59f0c00640 1 -- 172.21.15.2:0/4178548724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59ec1079c0 msgr2=0x7f59ec19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.792+0000 7f59f0c00640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59ec1079c0 0x7f59ec19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.792+0000 7f59f0c00640 1 -- 172.21.15.2:0/4178548724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec1019c0 msgr2=0x7f59ec19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.792+0000 7f59f0c00640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec1019c0 0x7f59ec19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.792+0000 7f59f0c00640 1 -- 172.21.15.2:0/4178548724 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ec19f4e0 con 0x7f59ec102360 2024-09-10T08:07:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.792+0000 7f59f1600640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59ec1019c0 0x7f59ec19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.792+0000 7f59f0c00640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec102360 0x7f59ec19aa70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f59d802a940 tx=0x7f59d8003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.793+0000 7f59ea000640 1 -- 172.21.15.2:0/4178548724 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d803f070 con 0x7f59ec102360 2024-09-10T08:07:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.793+0000 7f59ea000640 1 -- 172.21.15.2:0/4178548724 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59d8031c90 con 0x7f59ec102360 2024-09-10T08:07:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.793+0000 7f59ea000640 1 -- 172.21.15.2:0/4178548724 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8035030 con 0x7f59ec102360 2024-09-10T08:07:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.793+0000 7f59f2a00640 1 -- 172.21.15.2:0/4178548724 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ec19f770 con 0x7f59ec102360 2024-09-10T08:07:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.793+0000 7f59f2a00640 1 -- 172.21.15.2:0/4178548724 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ec19fc00 con 0x7f59ec102360 2024-09-10T08:07:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.794+0000 7f59f2a00640 1 -- 172.21.15.2:0/4178548724 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59ec0ff0d0 con 0x7f59ec102360 2024-09-10T08:07:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.796+0000 7f59ea000640 1 -- 172.21.15.2:0/4178548724 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f59d8042070 con 0x7f59ec102360 2024-09-10T08:07:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.798+0000 7f59ea000640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59d40a2c10 0x7f59d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.798+0000 7f59ea000640 1 -- 172.21.15.2:0/4178548724 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f59d80effb0 con 0x7f59ec102360 2024-09-10T08:07:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.798+0000 7f59f1600640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59d40a2c10 0x7f59d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.798+0000 7f59ea000640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707] conn(0x7f59d40adde0 0x7f59d40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.798+0000 7f59ea000640 1 -- 172.21.15.2:0/4178548724 --> [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59d40b0950 con 0x7f59d40adde0 2024-09-10T08:07:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.798+0000 7f59ea000640 1 -- 172.21.15.2:0/4178548724 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f59d80f0360 con 0x7f59ec102360 2024-09-10T08:07:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.799+0000 7f59f2000640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707] conn(0x7f59d40adde0 0x7f59d40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.799+0000 7f59f1600640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59d40a2c10 0x7f59d40a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009f40 tx=0x7f59e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.800+0000 7f59f2000640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707] conn(0x7f59d40adde0 0x7f59d40b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.800+0000 7f59ea000640 1 -- 172.21.15.2:0/4178548724 <== osd.2 v2:172.21.15.155:6800/158516707 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f59d40b0950 con 0x7f59d40adde0 2024-09-10T08:07:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.815+0000 7f59f2a00640 1 -- 172.21.15.2:0/4178548724 --> [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f59ec103100 con 0x7f59d40adde0 2024-09-10T08:07:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.816+0000 7f59ea000640 1 -- 172.21.15.2:0/4178548724 <== osd.2 v2:172.21.15.155:6800/158516707 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f59ec103100 con 0x7f59d40adde0 2024-09-10T08:07:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.816+0000 7f59f2a00640 1 -- 172.21.15.2:0/4178548724 >> [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707] conn(0x7f59d40adde0 msgr2=0x7f59d40b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.816+0000 7f59f2a00640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707] conn(0x7f59d40adde0 0x7f59d40b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.816+0000 7f59f2a00640 1 -- 172.21.15.2:0/4178548724 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59d40a2c10 msgr2=0x7f59d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.816+0000 7f59f2a00640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59d40a2c10 0x7f59d40a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009f40 tx=0x7f59e0005f90 comp rx=0 tx=0).stop 2024-09-10T08:07:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.816+0000 7f59f2a00640 1 -- 172.21.15.2:0/4178548724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec102360 msgr2=0x7f59ec19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.816+0000 7f59f2a00640 1 --2- 172.21.15.2:0/4178548724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59ec102360 0x7f59ec19aa70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f59d802a940 tx=0x7f59d8003b20 comp rx=0 tx=0).stop 2024-09-10T08:07:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.817+0000 7f59f2000640 1 -- 172.21.15.2:0/4178548724 reap_dead start 2024-09-10T08:07:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.817+0000 7f59f2a00640 1 -- 172.21.15.2:0/4178548724 shutdown_connections 2024-09-10T08:07:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.817+0000 7f59f2a00640 1 -- 172.21.15.2:0/4178548724 >> 172.21.15.2:0/4178548724 conn(0x7f59ec0fd740 msgr2=0x7f59ec10cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.817+0000 7f59f2a00640 1 -- 172.21.15.2:0/4178548724 shutdown_connections 2024-09-10T08:07:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:00.817+0000 7f59f2a00640 1 -- 172.21.15.2:0/4178548724 wait complete. 2024-09-10T08:07:01.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.684+0000 7f969d800640 1 -- 172.21.15.2:0/2517912630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96981079c0 msgr2=0x7f9698107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:01.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.684+0000 7f969d800640 1 --2- 172.21.15.2:0/2517912630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96981079c0 0x7f9698107da0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9688009900 tx=0x7f968802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.684+0000 7f969d800640 1 -- 172.21.15.2:0/2517912630 shutdown_connections 2024-09-10T08:07:01.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.684+0000 7f969d800640 1 --2- 172.21.15.2:0/2517912630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698102360 0x7f969810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.684+0000 7f969d800640 1 --2- 172.21.15.2:0/2517912630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96981019c0 0x7f9698101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.684+0000 7f969d800640 1 --2- 172.21.15.2:0/2517912630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96981079c0 0x7f9698107da0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.684+0000 7f969d800640 1 -- 172.21.15.2:0/2517912630 >> 172.21.15.2:0/2517912630 conn(0x7f96980fd740 msgr2=0x7f96980ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:01.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.684+0000 7f969d800640 1 -- 172.21.15.2:0/2517912630 shutdown_connections 2024-09-10T08:07:01.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.684+0000 7f969d800640 1 -- 172.21.15.2:0/2517912630 wait complete. 2024-09-10T08:07:01.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.685+0000 7f969d800640 1 Processor -- start 2024-09-10T08:07:01.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.685+0000 7f969d800640 1 -- start start 2024-09-10T08:07:01.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.685+0000 7f969d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96981019c0 0x7f9698198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:01.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.685+0000 7f969d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698102360 0x7f96981987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:01.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.685+0000 7f969d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96981079c0 0x7f969819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:01.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.685+0000 7f969d800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9698111e10 con 0x7f96981079c0 2024-09-10T08:07:01.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.685+0000 7f969d800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9698111c90 con 0x7f9698102360 2024-09-10T08:07:01.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.685+0000 7f969d800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9698111f90 con 0x7f96981019c0 2024-09-10T08:07:01.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.686+0000 7f969ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96981079c0 0x7f969819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:01.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.686+0000 7f969ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96981079c0 0x7f969819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45808/0 (socket says 172.21.15.2:45808) 2024-09-10T08:07:01.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.686+0000 7f969ce00640 1 -- 172.21.15.2:0/3829559315 learned_addr learned my addr 172.21.15.2:0/3829559315 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:01.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.686+0000 7f968f400640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698102360 0x7f96981987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:01.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.686+0000 7f9697e00640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96981019c0 0x7f9698198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:01.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.686+0000 7f969ce00640 1 -- 172.21.15.2:0/3829559315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96981019c0 msgr2=0x7f9698198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:01.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.686+0000 7f969ce00640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96981019c0 0x7f9698198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.686+0000 7f969ce00640 1 -- 172.21.15.2:0/3829559315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698102360 msgr2=0x7f96981987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:01.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.686+0000 7f969ce00640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698102360 0x7f96981987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.686+0000 7f969ce00640 1 -- 172.21.15.2:0/3829559315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969819d240 con 0x7f96981079c0 2024-09-10T08:07:01.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.686+0000 7f9697e00640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96981019c0 0x7f9698198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:07:01.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.687+0000 7f969ce00640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96981079c0 0x7f969819cb60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f969000eb10 tx=0x7f969000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:01.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.687+0000 7f9695600640 1 -- 172.21.15.2:0/3829559315 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690004b20 con 0x7f96981079c0 2024-09-10T08:07:01.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.687+0000 7f9695600640 1 -- 172.21.15.2:0/3829559315 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9690004cc0 con 0x7f96981079c0 2024-09-10T08:07:01.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.687+0000 7f9695600640 1 -- 172.21.15.2:0/3829559315 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690010760 con 0x7f96981079c0 2024-09-10T08:07:01.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.687+0000 7f969d800640 1 -- 172.21.15.2:0/3829559315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969819d530 con 0x7f96981079c0 2024-09-10T08:07:01.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.687+0000 7f969d800640 1 -- 172.21.15.2:0/3829559315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96981a4e10 con 0x7f96981079c0 2024-09-10T08:07:01.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.688+0000 7f969d800640 1 -- 172.21.15.2:0/3829559315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9698062980 con 0x7f96981079c0 2024-09-10T08:07:01.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.690+0000 7f9695600640 1 -- 172.21.15.2:0/3829559315 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9690005020 con 0x7f96981079c0 2024-09-10T08:07:01.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.691+0000 7f9695600640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f967c0a2c90 0x7f967c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:01.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.691+0000 7f9695600640 1 -- 172.21.15.2:0/3829559315 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f96900cfee0 con 0x7f96981079c0 2024-09-10T08:07:01.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.692+0000 7f9697e00640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f967c0a2c90 0x7f967c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:01.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.692+0000 7f9695600640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441] conn(0x7f967c0ade60 0x7f967c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:01.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.692+0000 7f9695600640 1 -- 172.21.15.2:0/3829559315 --> [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f967c0b09d0 con 0x7f967c0ade60 2024-09-10T08:07:01.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.692+0000 7f9695600640 1 -- 172.21.15.2:0/3829559315 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f96900d0290 con 0x7f96981079c0 2024-09-10T08:07:01.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.692+0000 7f968f400640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441] conn(0x7f967c0ade60 0x7f967c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:01.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.692+0000 7f9697e00640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f967c0a2c90 0x7f967c0a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9688000c00 tx=0x7f9688005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:01.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.693+0000 7f968f400640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441] conn(0x7f967c0ade60 0x7f967c0b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:01.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.694+0000 7f9695600640 1 -- 172.21.15.2:0/3829559315 <== osd.1 v2:172.21.15.107:6800/3710092441 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f967c0b09d0 con 0x7f967c0ade60 2024-09-10T08:07:01.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.708+0000 7f969d800640 1 -- 172.21.15.2:0/3829559315 --> [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9698062e00 con 0x7f967c0ade60 2024-09-10T08:07:01.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.709+0000 7f9695600640 1 -- 172.21.15.2:0/3829559315 <== osd.1 v2:172.21.15.107:6800/3710092441 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9698062e00 con 0x7f967c0ade60 2024-09-10T08:07:01.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.709+0000 7f969d800640 1 -- 172.21.15.2:0/3829559315 >> [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441] conn(0x7f967c0ade60 msgr2=0x7f967c0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:01.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.709+0000 7f969d800640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441] conn(0x7f967c0ade60 0x7f967c0b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.709+0000 7f969d800640 1 -- 172.21.15.2:0/3829559315 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f967c0a2c90 msgr2=0x7f967c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:01.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.709+0000 7f969d800640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f967c0a2c90 0x7f967c0a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9688000c00 tx=0x7f9688005a40 comp rx=0 tx=0).stop 2024-09-10T08:07:01.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.709+0000 7f969d800640 1 -- 172.21.15.2:0/3829559315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96981079c0 msgr2=0x7f969819cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:01.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.709+0000 7f969d800640 1 --2- 172.21.15.2:0/3829559315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96981079c0 0x7f969819cb60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f969000eb10 tx=0x7f969000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:07:01.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.709+0000 7f969ce00640 1 -- 172.21.15.2:0/3829559315 reap_dead start 2024-09-10T08:07:01.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.709+0000 7f969d800640 1 -- 172.21.15.2:0/3829559315 shutdown_connections 2024-09-10T08:07:01.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.709+0000 7f969d800640 1 -- 172.21.15.2:0/3829559315 >> 172.21.15.2:0/3829559315 conn(0x7f96980fd740 msgr2=0x7f96980fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:01.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.709+0000 7f969d800640 1 -- 172.21.15.2:0/3829559315 shutdown_connections 2024-09-10T08:07:01.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:01.709+0000 7f969d800640 1 -- 172.21.15.2:0/3829559315 wait complete. 2024-09-10T08:07:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:01 smithi002 ceph-mon[30563]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:01.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:01 smithi155 ceph-mon[35405]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:01.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:01 smithi107 ceph-mon[35048]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:02.660 INFO:teuthology.orchestra.run.smithi002.stdout:214748364852 2024-09-10T08:07:02.661 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T08:07:02.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.640+0000 7f1226400640 1 -- 172.21.15.2:0/3502028893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1220101de0 msgr2=0x7f12201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:02.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.640+0000 7f1226400640 1 --2- 172.21.15.2:0/3502028893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1220101de0 0x7f12201021c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1210009900 tx=0x7f121002a410 comp rx=0 tx=0).stop 2024-09-10T08:07:02.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.640+0000 7f1226400640 1 -- 172.21.15.2:0/3502028893 shutdown_connections 2024-09-10T08:07:02.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.640+0000 7f1226400640 1 --2- 172.21.15.2:0/3502028893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220106f10 0x7f122010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:02.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.640+0000 7f1226400640 1 --2- 172.21.15.2:0/3502028893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1220102790 0x7f12201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:02.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.640+0000 7f1226400640 1 --2- 172.21.15.2:0/3502028893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1220101de0 0x7f12201021c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.641+0000 7f1226400640 1 -- 172.21.15.2:0/3502028893 >> 172.21.15.2:0/3502028893 conn(0x7f12200fd8c0 msgr2=0x7f12200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.641+0000 7f1226400640 1 -- 172.21.15.2:0/3502028893 shutdown_connections 2024-09-10T08:07:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.641+0000 7f1226400640 1 -- 172.21.15.2:0/3502028893 wait complete. 2024-09-10T08:07:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.642+0000 7f1226400640 1 Processor -- start 2024-09-10T08:07:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.642+0000 7f1226400640 1 -- start start 2024-09-10T08:07:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.642+0000 7f1226400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1220101de0 0x7f1220108ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:02.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.642+0000 7f1226400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220102790 0x7f1220108ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:02.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.642+0000 7f1226400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1220106f10 0x7f1220109530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:02.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.642+0000 7f1226400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1220111fc0 con 0x7f1220106f10 2024-09-10T08:07:02.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.642+0000 7f1226400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1220111e40 con 0x7f1220101de0 2024-09-10T08:07:02.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.642+0000 7f1226400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1220112140 con 0x7f1220102790 2024-09-10T08:07:02.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.642+0000 7f121fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1220106f10 0x7f1220109530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:02.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.642+0000 7f121fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1220106f10 0x7f1220109530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45828/0 (socket says 172.21.15.2:45828) 2024-09-10T08:07:02.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.642+0000 7f121fe00640 1 -- 172.21.15.2:0/1021187147 learned_addr learned my addr 172.21.15.2:0/1021187147 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:02.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.643+0000 7f121f400640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1220101de0 0x7f1220108ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:02.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.643+0000 7f121ea00640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220102790 0x7f1220108ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:02.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.643+0000 7f121fe00640 1 -- 172.21.15.2:0/1021187147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220102790 msgr2=0x7f1220108ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.643+0000 7f121fe00640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220102790 0x7f1220108ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.643+0000 7f121fe00640 1 -- 172.21.15.2:0/1021187147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1220101de0 msgr2=0x7f1220108ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.643+0000 7f121fe00640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1220101de0 0x7f1220108ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.643+0000 7f121fe00640 1 -- 172.21.15.2:0/1021187147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12201aacf0 con 0x7f1220106f10 2024-09-10T08:07:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.643+0000 7f121fe00640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1220106f10 0x7f1220109530 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f121800eb10 tx=0x7f121800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.644+0000 7f1217e00640 1 -- 172.21.15.2:0/1021187147 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218004b20 con 0x7f1220106f10 2024-09-10T08:07:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.644+0000 7f1217e00640 1 -- 172.21.15.2:0/1021187147 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1218004cc0 con 0x7f1220106f10 2024-09-10T08:07:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.644+0000 7f1217e00640 1 -- 172.21.15.2:0/1021187147 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218010760 con 0x7f1220106f10 2024-09-10T08:07:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.644+0000 7f1226400640 1 -- 172.21.15.2:0/1021187147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12201aafe0 con 0x7f1220106f10 2024-09-10T08:07:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.644+0000 7f1226400640 1 -- 172.21.15.2:0/1021187147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12201ab440 con 0x7f1220106f10 2024-09-10T08:07:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.645+0000 7f1226400640 1 -- 172.21.15.2:0/1021187147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1220075700 con 0x7f1220106f10 2024-09-10T08:07:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.647+0000 7f1217e00640 1 -- 172.21.15.2:0/1021187147 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1218005020 con 0x7f1220106f10 2024-09-10T08:07:02.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.648+0000 7f1217e00640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f12040a2c10 0x7f12040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:02.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.649+0000 7f1217e00640 1 -- 172.21.15.2:0/1021187147 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f12180d0140 con 0x7f1220106f10 2024-09-10T08:07:02.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.649+0000 7f1217e00640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099] conn(0x7f12040add60 0x7f12040b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:02.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.649+0000 7f1217e00640 1 -- 172.21.15.2:0/1021187147 --> [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f121801ee50 con 0x7f12040add60 2024-09-10T08:07:02.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.649+0000 7f121f400640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f12040a2c10 0x7f12040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.649+0000 7f1217e00640 1 -- 172.21.15.2:0/1021187147 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f12180d04f0 con 0x7f1220106f10 2024-09-10T08:07:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.649+0000 7f121ea00640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099] conn(0x7f12040add60 0x7f12040b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.650+0000 7f121f400640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f12040a2c10 0x7f12040a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1210000c00 tx=0x7f1210005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.650+0000 7f121ea00640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099] conn(0x7f12040add60 0x7f12040b0200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.651+0000 7f1217e00640 1 -- 172.21.15.2:0/1021187147 <== osd.9 v2:172.21.15.2:6826/758080099 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f121801ee50 con 0x7f12040add60 2024-09-10T08:07:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.668+0000 7f1226400640 1 -- 172.21.15.2:0/1021187147 --> [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f12201032c0 con 0x7f12040add60 2024-09-10T08:07:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.669+0000 7f1217e00640 1 -- 172.21.15.2:0/1021187147 <== osd.9 v2:172.21.15.2:6826/758080099 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f12201032c0 con 0x7f12040add60 2024-09-10T08:07:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.669+0000 7f1226400640 1 -- 172.21.15.2:0/1021187147 >> [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099] conn(0x7f12040add60 msgr2=0x7f12040b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.669+0000 7f1226400640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099] conn(0x7f12040add60 0x7f12040b0200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.670+0000 7f1226400640 1 -- 172.21.15.2:0/1021187147 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f12040a2c10 msgr2=0x7f12040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.670+0000 7f1226400640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f12040a2c10 0x7f12040a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1210000c00 tx=0x7f1210005b50 comp rx=0 tx=0).stop 2024-09-10T08:07:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.670+0000 7f1226400640 1 -- 172.21.15.2:0/1021187147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1220106f10 msgr2=0x7f1220109530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.670+0000 7f1226400640 1 --2- 172.21.15.2:0/1021187147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1220106f10 0x7f1220109530 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f121800eb10 tx=0x7f121800c2d0 comp rx=0 tx=0).stop 2024-09-10T08:07:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.670+0000 7f121fe00640 1 -- 172.21.15.2:0/1021187147 reap_dead start 2024-09-10T08:07:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.670+0000 7f1226400640 1 -- 172.21.15.2:0/1021187147 shutdown_connections 2024-09-10T08:07:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.670+0000 7f1226400640 1 -- 172.21.15.2:0/1021187147 >> 172.21.15.2:0/1021187147 conn(0x7f12200fd8c0 msgr2=0x7f12200feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.670+0000 7f1226400640 1 -- 172.21.15.2:0/1021187147 shutdown_connections 2024-09-10T08:07:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:02.670+0000 7f1226400640 1 -- 172.21.15.2:0/1021187147 wait complete. 2024-09-10T08:07:03.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.244+0000 7f97d9200640 1 -- 172.21.15.2:0/2145204977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97d410d530 msgr2=0x7f97d41150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.244+0000 7f97d9200640 1 --2- 172.21.15.2:0/2145204977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97d410d530 0x7f97d41150b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009ca0 tx=0x7f97cc02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.245+0000 7f97d9200640 1 -- 172.21.15.2:0/2145204977 shutdown_connections 2024-09-10T08:07:03.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.245+0000 7f97d9200640 1 --2- 172.21.15.2:0/2145204977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97d410d530 0x7f97d41150b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.245+0000 7f97d9200640 1 --2- 172.21.15.2:0/2145204977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97d4108e00 0x7f97d410ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.245+0000 7f97d9200640 1 --2- 172.21.15.2:0/2145204977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97d4108450 0x7f97d4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.245+0000 7f97d9200640 1 -- 172.21.15.2:0/2145204977 >> 172.21.15.2:0/2145204977 conn(0x7f97d4077ba0 msgr2=0x7f97d4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.245+0000 7f97d9200640 1 -- 172.21.15.2:0/2145204977 shutdown_connections 2024-09-10T08:07:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.245+0000 7f97d9200640 1 -- 172.21.15.2:0/2145204977 wait complete. 2024-09-10T08:07:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.246+0000 7f97d9200640 1 Processor -- start 2024-09-10T08:07:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.246+0000 7f97d9200640 1 -- start start 2024-09-10T08:07:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97d4108450 0x7f97d41a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97d4108e00 0x7f97d41a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97d410d530 0x7f97d419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d9200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f97d4118590 con 0x7f97d410d530 2024-09-10T08:07:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d9200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f97d4118410 con 0x7f97d4108e00 2024-09-10T08:07:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d9200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f97d4118710 con 0x7f97d4108450 2024-09-10T08:07:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d2800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97d410d530 0x7f97d419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d2800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97d410d530 0x7f97d419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45836/0 (socket says 172.21.15.2:45836) 2024-09-10T08:07:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d2800640 1 -- 172.21.15.2:0/1092672537 learned_addr learned my addr 172.21.15.2:0/1092672537 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d1e00640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97d4108450 0x7f97d41a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97cbe00640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97d4108e00 0x7f97d41a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d2800640 1 -- 172.21.15.2:0/1092672537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97d4108450 msgr2=0x7f97d41a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d2800640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97d4108450 0x7f97d41a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d2800640 1 -- 172.21.15.2:0/1092672537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97d4108e00 msgr2=0x7f97d41a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d2800640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97d4108e00 0x7f97d41a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.247+0000 7f97d2800640 1 -- 172.21.15.2:0/1092672537 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d419efb0 con 0x7f97d410d530 2024-09-10T08:07:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.248+0000 7f97d2800640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97d410d530 0x7f97d419e880 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97cc005e30 tx=0x7f97cc003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.248+0000 7f97caa00640 1 -- 172.21.15.2:0/1092672537 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc031650 con 0x7f97d410d530 2024-09-10T08:07:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.248+0000 7f97caa00640 1 -- 172.21.15.2:0/1092672537 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97cc0317f0 con 0x7f97d410d530 2024-09-10T08:07:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.248+0000 7f97caa00640 1 -- 172.21.15.2:0/1092672537 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc02f9d0 con 0x7f97d410d530 2024-09-10T08:07:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.248+0000 7f97d9200640 1 -- 172.21.15.2:0/1092672537 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d419f240 con 0x7f97d410d530 2024-09-10T08:07:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.248+0000 7f97d9200640 1 -- 172.21.15.2:0/1092672537 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d41ab590 con 0x7f97d410d530 2024-09-10T08:07:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.249+0000 7f97d9200640 1 -- 172.21.15.2:0/1092672537 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f97d4062900 con 0x7f97d410d530 2024-09-10T08:07:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.251+0000 7f97caa00640 1 -- 172.21.15.2:0/1092672537 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f97cc02fb70 con 0x7f97d410d530 2024-09-10T08:07:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.252+0000 7f97caa00640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f97b80a2c90 0x7f97b80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.253+0000 7f97caa00640 1 -- 172.21.15.2:0/1092672537 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f97cc0f0ab0 con 0x7f97d410d530 2024-09-10T08:07:03.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.253+0000 7f97d1e00640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f97b80a2c90 0x7f97b80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.253+0000 7f97caa00640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493] conn(0x7f97b80ade60 0x7f97b80b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.253+0000 7f97caa00640 1 -- 172.21.15.2:0/1092672537 --> [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f97cc00b150 con 0x7f97b80ade60 2024-09-10T08:07:03.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.253+0000 7f97caa00640 1 -- 172.21.15.2:0/1092672537 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f97cc0f0e60 con 0x7f97d410d530 2024-09-10T08:07:03.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.253+0000 7f97cbe00640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493] conn(0x7f97b80ade60 0x7f97b80b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.253+0000 7f97d1e00640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f97b80a2c90 0x7f97b80a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f97d419f9c0 tx=0x7f97c400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:03.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.254+0000 7f97cbe00640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493] conn(0x7f97b80ade60 0x7f97b80b0300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:03.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.255+0000 7f97caa00640 1 -- 172.21.15.2:0/1092672537 <== osd.10 v2:172.21.15.107:6824/3646489493 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f97cc00b150 con 0x7f97b80ade60 2024-09-10T08:07:03.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.269+0000 7f97d9200640 1 -- 172.21.15.2:0/1092672537 --> [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f97d4062d00 con 0x7f97b80ade60 2024-09-10T08:07:03.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.270+0000 7f97caa00640 1 -- 172.21.15.2:0/1092672537 <== osd.10 v2:172.21.15.107:6824/3646489493 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f97d4062d00 con 0x7f97b80ade60 2024-09-10T08:07:03.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.270+0000 7f97d9200640 1 -- 172.21.15.2:0/1092672537 >> [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493] conn(0x7f97b80ade60 msgr2=0x7f97b80b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.270+0000 7f97d9200640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493] conn(0x7f97b80ade60 0x7f97b80b0300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.270+0000 7f97d9200640 1 -- 172.21.15.2:0/1092672537 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f97b80a2c90 msgr2=0x7f97b80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.270+0000 7f97d9200640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f97b80a2c90 0x7f97b80a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f97d419f9c0 tx=0x7f97c400b410 comp rx=0 tx=0).stop 2024-09-10T08:07:03.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.270+0000 7f97d9200640 1 -- 172.21.15.2:0/1092672537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97d410d530 msgr2=0x7f97d419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.270+0000 7f97d9200640 1 --2- 172.21.15.2:0/1092672537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97d410d530 0x7f97d419e880 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97cc005e30 tx=0x7f97cc003d00 comp rx=0 tx=0).stop 2024-09-10T08:07:03.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.270+0000 7f97d2800640 1 -- 172.21.15.2:0/1092672537 reap_dead start 2024-09-10T08:07:03.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.270+0000 7f97d9200640 1 -- 172.21.15.2:0/1092672537 shutdown_connections 2024-09-10T08:07:03.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.271+0000 7f97d9200640 1 -- 172.21.15.2:0/1092672537 >> 172.21.15.2:0/1092672537 conn(0x7f97d4077ba0 msgr2=0x7f97d4078e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:03.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.271+0000 7f97d9200640 1 -- 172.21.15.2:0/1092672537 shutdown_connections 2024-09-10T08:07:03.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.271+0000 7f97d9200640 1 -- 172.21.15.2:0/1092672537 wait complete. 2024-09-10T08:07:03.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.494+0000 7f0c7a600640 1 -- 172.21.15.2:0/2342863016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c7410e290 msgr2=0x7f0c74110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.494+0000 7f0c7a600640 1 --2- 172.21.15.2:0/2342863016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c7410e290 0x7f0c74110680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009ca0 tx=0x7f0c6c02a950 comp rx=0 tx=0).stop 2024-09-10T08:07:03.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.494+0000 7f0c7a600640 1 -- 172.21.15.2:0/2342863016 shutdown_connections 2024-09-10T08:07:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.494+0000 7f0c7a600640 1 --2- 172.21.15.2:0/2342863016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c7410e290 0x7f0c74110680 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.494+0000 7f0c7a600640 1 --2- 172.21.15.2:0/2342863016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c74074e30 0x7f0c7410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.494+0000 7f0c7a600640 1 --2- 172.21.15.2:0/2342863016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c74074510 0x7f0c740748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.494+0000 7f0c7a600640 1 -- 172.21.15.2:0/2342863016 >> 172.21.15.2:0/2342863016 conn(0x7f0c740fd710 msgr2=0x7f0c740ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.494+0000 7f0c7a600640 1 -- 172.21.15.2:0/2342863016 shutdown_connections 2024-09-10T08:07:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.494+0000 7f0c7a600640 1 -- 172.21.15.2:0/2342863016 wait complete. 2024-09-10T08:07:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.495+0000 7f0c7a600640 1 Processor -- start 2024-09-10T08:07:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.496+0000 7f0c7a600640 1 -- start start 2024-09-10T08:07:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.496+0000 7f0c7a600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c74074510 0x7f0c7419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.496+0000 7f0c7a600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c74074e30 0x7f0c7419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.496+0000 7f0c7a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c7410e290 0x7f0c7419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.496+0000 7f0c7a600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0c74113fa0 con 0x7f0c74074510 2024-09-10T08:07:03.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.496+0000 7f0c7a600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c74113e20 con 0x7f0c7410e290 2024-09-10T08:07:03.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.496+0000 7f0c7a600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0c74114120 con 0x7f0c74074e30 2024-09-10T08:07:03.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.496+0000 7f0c79200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c74074510 0x7f0c7419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.496+0000 7f0c79200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c74074510 0x7f0c7419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45840/0 (socket says 172.21.15.2:45840) 2024-09-10T08:07:03.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.496+0000 7f0c79200640 1 -- 172.21.15.2:0/1566433531 learned_addr learned my addr 172.21.15.2:0/1566433531 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:03.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.496+0000 7f0c73e00640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c74074e30 0x7f0c7419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.496+0000 7f0c79c00640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c7410e290 0x7f0c7419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.497+0000 7f0c73e00640 1 -- 172.21.15.2:0/1566433531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c7410e290 msgr2=0x7f0c7419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.497+0000 7f0c73e00640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c7410e290 0x7f0c7419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.497+0000 7f0c73e00640 1 -- 172.21.15.2:0/1566433531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c74074510 msgr2=0x7f0c7419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.497+0000 7f0c73e00640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c74074510 0x7f0c7419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.497+0000 7f0c73e00640 1 -- 172.21.15.2:0/1566433531 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7419f430 con 0x7f0c74074e30 2024-09-10T08:07:03.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.497+0000 7f0c79200640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c74074510 0x7f0c7419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:03.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.497+0000 7f0c73e00640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c74074e30 0x7f0c7419a9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000e9f0 tx=0x7f0c6000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:03.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.501+0000 7f0c71600640 1 -- 172.21.15.2:0/1566433531 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6000ccc0 con 0x7f0c74074e30 2024-09-10T08:07:03.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.501+0000 7f0c71600640 1 -- 172.21.15.2:0/1566433531 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c60005020 con 0x7f0c74074e30 2024-09-10T08:07:03.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.501+0000 7f0c7a600640 1 -- 172.21.15.2:0/1566433531 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c7419f720 con 0x7f0c74074e30 2024-09-10T08:07:03.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.501+0000 7f0c71600640 1 -- 172.21.15.2:0/1566433531 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c600049e0 con 0x7f0c74074e30 2024-09-10T08:07:03.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.501+0000 7f0c7a600640 1 -- 172.21.15.2:0/1566433531 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c741a7000 con 0x7f0c74074e30 2024-09-10T08:07:03.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.502+0000 7f0c7a600640 1 -- 172.21.15.2:0/1566433531 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c74062980 con 0x7f0c74074e30 2024-09-10T08:07:03.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.504+0000 7f0c71600640 1 -- 172.21.15.2:0/1566433531 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0c60005560 con 0x7f0c74074e30 2024-09-10T08:07:03.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.505+0000 7f0c71600640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0c5c0a2c90 0x7f0c5c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.506+0000 7f0c71600640 1 -- 172.21.15.2:0/1566433531 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f0c600cf3e0 con 0x7f0c74074e30 2024-09-10T08:07:03.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.506+0000 7f0c79200640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0c5c0a2c90 0x7f0c5c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.506+0000 7f0c71600640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508] conn(0x7f0c5c0ade60 0x7f0c5c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.506+0000 7f0c71600640 1 -- 172.21.15.2:0/1566433531 --> [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c5c0b09d0 con 0x7f0c5c0ade60 2024-09-10T08:07:03.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.506+0000 7f0c71600640 1 -- 172.21.15.2:0/1566433531 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f0c600d6050 con 0x7f0c74074e30 2024-09-10T08:07:03.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.506+0000 7f0c79c00640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508] conn(0x7f0c5c0ade60 0x7f0c5c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.507+0000 7f0c79200640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0c5c0a2c90 0x7f0c5c0a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c68011550 tx=0x7f0c68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:03.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.507+0000 7f0c79c00640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508] conn(0x7f0c5c0ade60 0x7f0c5c0b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:03.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.508+0000 7f0c71600640 1 -- 172.21.15.2:0/1566433531 <== osd.6 v2:172.21.15.2:6818/854683508 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c5c0b09d0 con 0x7f0c5c0ade60 2024-09-10T08:07:03.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.524+0000 7f0c7a600640 1 -- 172.21.15.2:0/1566433531 --> [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c74062e00 con 0x7f0c5c0ade60 2024-09-10T08:07:03.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.524+0000 7f0c71600640 1 -- 172.21.15.2:0/1566433531 <== osd.6 v2:172.21.15.2:6818/854683508 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0c74062e00 con 0x7f0c5c0ade60 2024-09-10T08:07:03.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.525+0000 7f0c7a600640 1 -- 172.21.15.2:0/1566433531 >> [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508] conn(0x7f0c5c0ade60 msgr2=0x7f0c5c0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.525+0000 7f0c7a600640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508] conn(0x7f0c5c0ade60 0x7f0c5c0b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.525+0000 7f0c7a600640 1 -- 172.21.15.2:0/1566433531 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0c5c0a2c90 msgr2=0x7f0c5c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.525+0000 7f0c7a600640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0c5c0a2c90 0x7f0c5c0a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c68011550 tx=0x7f0c68005f90 comp rx=0 tx=0).stop 2024-09-10T08:07:03.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.525+0000 7f0c7a600640 1 -- 172.21.15.2:0/1566433531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c74074e30 msgr2=0x7f0c7419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.525+0000 7f0c7a600640 1 --2- 172.21.15.2:0/1566433531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c74074e30 0x7f0c7419a9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000e9f0 tx=0x7f0c6000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.525+0000 7f0c79c00640 1 -- 172.21.15.2:0/1566433531 reap_dead start 2024-09-10T08:07:03.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.525+0000 7f0c7a600640 1 -- 172.21.15.2:0/1566433531 shutdown_connections 2024-09-10T08:07:03.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.525+0000 7f0c7a600640 1 -- 172.21.15.2:0/1566433531 >> 172.21.15.2:0/1566433531 conn(0x7f0c740fd710 msgr2=0x7f0c740fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:03.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.525+0000 7f0c7a600640 1 -- 172.21.15.2:0/1566433531 shutdown_connections 2024-09-10T08:07:03.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.525+0000 7f0c7a600640 1 -- 172.21.15.2:0/1566433531 wait complete. 2024-09-10T08:07:03.705 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:03 smithi002 ceph-mon[30563]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.705+0000 7f3aa8a00640 1 -- 172.21.15.2:0/787663556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa41079c0 msgr2=0x7f3aa4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.705+0000 7f3aa8a00640 1 --2- 172.21.15.2:0/787663556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa41079c0 0x7f3aa4107da0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009930 tx=0x7f3a9402a440 comp rx=0 tx=0).stop 2024-09-10T08:07:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.705+0000 7f3aa8a00640 1 -- 172.21.15.2:0/787663556 shutdown_connections 2024-09-10T08:07:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.705+0000 7f3aa8a00640 1 --2- 172.21.15.2:0/787663556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aa4102360 0x7f3aa410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.705+0000 7f3aa8a00640 1 --2- 172.21.15.2:0/787663556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3aa41019c0 0x7f3aa4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.705+0000 7f3aa8a00640 1 --2- 172.21.15.2:0/787663556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa41079c0 0x7f3aa4107da0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.705+0000 7f3aa8a00640 1 -- 172.21.15.2:0/787663556 >> 172.21.15.2:0/787663556 conn(0x7f3aa40fd740 msgr2=0x7f3aa40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.705+0000 7f3aa8a00640 1 -- 172.21.15.2:0/787663556 shutdown_connections 2024-09-10T08:07:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.705+0000 7f3aa8a00640 1 -- 172.21.15.2:0/787663556 wait complete. 2024-09-10T08:07:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.707+0000 7f3aa8a00640 1 Processor -- start 2024-09-10T08:07:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.707+0000 7f3aa8a00640 1 -- start start 2024-09-10T08:07:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa8a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3aa41019c0 0x7f3aa40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa8a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aa4102360 0x7f3aa4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa8a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa41079c0 0x7f3aa4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa8a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4111e10 con 0x7f3aa4102360 2024-09-10T08:07:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa8a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4111c90 con 0x7f3aa41019c0 2024-09-10T08:07:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa8a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa4111f90 con 0x7f3aa41079c0 2024-09-10T08:07:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3a9be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aa4102360 0x7f3aa4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3aa41019c0 0x7f3aa40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3aa41019c0 0x7f3aa40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52818/0 (socket says 172.21.15.2:52818) 2024-09-10T08:07:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa3400640 1 -- 172.21.15.2:0/980567614 learned_addr learned my addr 172.21.15.2:0/980567614 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3a9be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aa4102360 0x7f3aa4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45862/0 (socket says 172.21.15.2:45862) 2024-09-10T08:07:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa3e00640 1 --2- 172.21.15.2:0/980567614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa41079c0 0x7f3aa4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa3400640 1 -- 172.21.15.2:0/980567614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa41079c0 msgr2=0x7f3aa4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa3400640 1 --2- 172.21.15.2:0/980567614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa41079c0 0x7f3aa4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa3400640 1 -- 172.21.15.2:0/980567614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aa4102360 msgr2=0x7f3aa4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa3400640 1 --2- 172.21.15.2:0/980567614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aa4102360 0x7f3aa4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.708+0000 7f3aa3400640 1 -- 172.21.15.2:0/980567614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa41a67e0 con 0x7f3aa41019c0 2024-09-10T08:07:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.709+0000 7f3aa3400640 1 --2- 172.21.15.2:0/980567614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3aa41019c0 0x7f3aa40744f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a9402a960 tx=0x7f3a94003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.709+0000 7f3aa0c00640 1 -- 172.21.15.2:0/980567614 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9403f070 con 0x7f3aa41019c0 2024-09-10T08:07:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.709+0000 7f3aa0c00640 1 -- 172.21.15.2:0/980567614 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a94031c90 con 0x7f3aa41019c0 2024-09-10T08:07:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.709+0000 7f3aa8a00640 1 -- 172.21.15.2:0/980567614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa41a6a70 con 0x7f3aa41019c0 2024-09-10T08:07:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.709+0000 7f3aa0c00640 1 -- 172.21.15.2:0/980567614 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94035030 con 0x7f3aa41019c0 2024-09-10T08:07:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.709+0000 7f3aa8a00640 1 -- 172.21.15.2:0/980567614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa41a6fa0 con 0x7f3aa41019c0 2024-09-10T08:07:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.710+0000 7f3aa8a00640 1 -- 172.21.15.2:0/980567614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3aa4062900 con 0x7f3aa41019c0 2024-09-10T08:07:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.712+0000 7f3aa0c00640 1 -- 172.21.15.2:0/980567614 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3a94042070 con 0x7f3aa41019c0 2024-09-10T08:07:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.714+0000 7f3aa0c00640 1 --2- 172.21.15.2:0/980567614 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3a880a2c90 0x7f3a880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.714+0000 7f3aa0c00640 1 -- 172.21.15.2:0/980567614 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f3a940f02a0 con 0x7f3aa41019c0 2024-09-10T08:07:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.714+0000 7f3a9be00640 1 --2- 172.21.15.2:0/980567614 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3a880a2c90 0x7f3a880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.714+0000 7f3aa0c00640 1 --2- 172.21.15.2:0/980567614 >> [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144] conn(0x7f3a880ade60 0x7f3a880b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.714+0000 7f3aa0c00640 1 -- 172.21.15.2:0/980567614 --> [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a94038e90 con 0x7f3a880ade60 2024-09-10T08:07:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.714+0000 7f3aa0c00640 1 -- 172.21.15.2:0/980567614 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f3a940f0650 con 0x7f3aa41019c0 2024-09-10T08:07:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.715+0000 7f3aa3e00640 1 --2- 172.21.15.2:0/980567614 >> [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144] conn(0x7f3a880ade60 0x7f3a880b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.715+0000 7f3a9be00640 1 --2- 172.21.15.2:0/980567614 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3a880a2c90 0x7f3a880a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4106f40 tx=0x7f3a8c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.715+0000 7f3aa3e00640 1 --2- 172.21.15.2:0/980567614 >> [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144] conn(0x7f3a880ade60 0x7f3a880b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.716+0000 7f3aa0c00640 1 -- 172.21.15.2:0/980567614 <== osd.3 v2:172.21.15.2:6810/713998144 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a94038e90 con 0x7f3a880ade60 2024-09-10T08:07:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.736+0000 7f3aa8a00640 1 -- 172.21.15.2:0/980567614 --> [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3aa4062d00 con 0x7f3a880ade60 2024-09-10T08:07:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.737+0000 7f3aa0c00640 1 -- 172.21.15.2:0/980567614 <== osd.3 v2:172.21.15.2:6810/713998144 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3aa4062d00 con 0x7f3a880ade60 2024-09-10T08:07:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.737+0000 7f3aa8a00640 1 -- 172.21.15.2:0/980567614 >> [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144] conn(0x7f3a880ade60 msgr2=0x7f3a880b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.737+0000 7f3aa8a00640 1 --2- 172.21.15.2:0/980567614 >> [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144] conn(0x7f3a880ade60 0x7f3a880b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.738+0000 7f3aa8a00640 1 -- 172.21.15.2:0/980567614 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3a880a2c90 msgr2=0x7f3a880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.738+0000 7f3aa8a00640 1 --2- 172.21.15.2:0/980567614 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3a880a2c90 0x7f3a880a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4106f40 tx=0x7f3a8c005f90 comp rx=0 tx=0).stop 2024-09-10T08:07:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.738+0000 7f3aa8a00640 1 -- 172.21.15.2:0/980567614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3aa41019c0 msgr2=0x7f3aa40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.738+0000 7f3aa8a00640 1 --2- 172.21.15.2:0/980567614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3aa41019c0 0x7f3aa40744f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a9402a960 tx=0x7f3a94003b20 comp rx=0 tx=0).stop 2024-09-10T08:07:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.738+0000 7f3aa3e00640 1 -- 172.21.15.2:0/980567614 reap_dead start 2024-09-10T08:07:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.738+0000 7f3aa8a00640 1 -- 172.21.15.2:0/980567614 shutdown_connections 2024-09-10T08:07:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.738+0000 7f3aa8a00640 1 -- 172.21.15.2:0/980567614 >> 172.21.15.2:0/980567614 conn(0x7f3aa40fd740 msgr2=0x7f3aa40fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.738+0000 7f3aa8a00640 1 -- 172.21.15.2:0/980567614 shutdown_connections 2024-09-10T08:07:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:03.738+0000 7f3aa8a00640 1 -- 172.21.15.2:0/980567614 wait complete. 2024-09-10T08:07:03.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:03 smithi155 ceph-mon[35405]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:03.919 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:03 smithi107 ceph-mon[35048]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:04.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-10T08:07:04.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-10T08:07:04.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-10T08:07:04.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-09-10T08:07:04.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:04 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-09-10T08:07:04.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-10T08:07:04.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-10T08:07:04.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-10T08:07:04.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-09-10T08:07:04.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:04 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-09-10T08:07:04.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-10T08:07:04.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-10T08:07:04.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-10T08:07:04.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-09-10T08:07:04.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:04 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-09-10T08:07:05.512 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:05 smithi002 ceph-mon[30563]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:05.512 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-10T08:07:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-10T08:07:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-10T08:07:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-09-10T08:07:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:05 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-09-10T08:07:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:05 smithi002 ceph-mon[30563]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T08:07:05.771 INFO:teuthology.orchestra.run.smithi002.stdout:193273528380 2024-09-10T08:07:05.772 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T08:07:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.113+0000 7f36e6a00640 1 -- 172.21.15.2:0/3276002944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36e0103ff0 msgr2=0x7f36e01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:05.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.113+0000 7f36e6a00640 1 --2- 172.21.15.2:0/3276002944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36e0103ff0 0x7f36e01043d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009900 tx=0x7f36d002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:05.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.113+0000 7f36e6a00640 1 -- 172.21.15.2:0/3276002944 shutdown_connections 2024-09-10T08:07:05.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.113+0000 7f36e6a00640 1 --2- 172.21.15.2:0/3276002944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36e0109120 0x7f36e0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:05.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.113+0000 7f36e6a00640 1 --2- 172.21.15.2:0/3276002944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36e01049a0 0x7f36e01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:05.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.113+0000 7f36e6a00640 1 --2- 172.21.15.2:0/3276002944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36e0103ff0 0x7f36e01043d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:05.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.113+0000 7f36e6a00640 1 -- 172.21.15.2:0/3276002944 >> 172.21.15.2:0/3276002944 conn(0x7f36e00ffa90 msgr2=0x7f36e0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:05.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.113+0000 7f36e6a00640 1 -- 172.21.15.2:0/3276002944 shutdown_connections 2024-09-10T08:07:05.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.113+0000 7f36e6a00640 1 -- 172.21.15.2:0/3276002944 wait complete. 2024-09-10T08:07:05.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.114+0000 7f36e6a00640 1 Processor -- start 2024-09-10T08:07:05.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.114+0000 7f36e6a00640 1 -- start start 2024-09-10T08:07:05.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.114+0000 7f36e6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36e0103ff0 0x7f36e019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:05.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.114+0000 7f36e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36e01049a0 0x7f36e019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:05.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.114+0000 7f36e6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36e0109120 0x7f36e019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:05.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.114+0000 7f36e6a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f36e01141a0 con 0x7f36e0103ff0 2024-09-10T08:07:05.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.114+0000 7f36e6a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36e0114020 con 0x7f36e01049a0 2024-09-10T08:07:05.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.114+0000 7f36e6a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f36e0114320 con 0x7f36e0109120 2024-09-10T08:07:05.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.115+0000 7f36dfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36e0103ff0 0x7f36e019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:05.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.115+0000 7f36dfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36e0103ff0 0x7f36e019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45884/0 (socket says 172.21.15.2:45884) 2024-09-10T08:07:05.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.115+0000 7f36dfe00640 1 -- 172.21.15.2:0/1029429491 learned_addr learned my addr 172.21.15.2:0/1029429491 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:05.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.115+0000 7f36df400640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36e01049a0 0x7f36e019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:05.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.115+0000 7f36e4a00640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36e0109120 0x7f36e019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:05.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.115+0000 7f36dfe00640 1 -- 172.21.15.2:0/1029429491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36e0109120 msgr2=0x7f36e019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:05.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.115+0000 7f36dfe00640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36e0109120 0x7f36e019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:05.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.115+0000 7f36dfe00640 1 -- 172.21.15.2:0/1029429491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36e01049a0 msgr2=0x7f36e019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:05.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.115+0000 7f36dfe00640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36e01049a0 0x7f36e019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:05.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.115+0000 7f36dfe00640 1 -- 172.21.15.2:0/1029429491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e019f6f0 con 0x7f36e0103ff0 2024-09-10T08:07:05.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.115+0000 7f36dfe00640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36e0103ff0 0x7f36e019a740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f36d0006010 tx=0x7f36d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:05.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.116+0000 7f36dcc00640 1 -- 172.21.15.2:0/1029429491 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d003f030 con 0x7f36e0103ff0 2024-09-10T08:07:05.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.116+0000 7f36dcc00640 1 -- 172.21.15.2:0/1029429491 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d0003ce0 con 0x7f36e0103ff0 2024-09-10T08:07:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.116+0000 7f36e6a00640 1 -- 172.21.15.2:0/1029429491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e019f980 con 0x7f36e0103ff0 2024-09-10T08:07:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.116+0000 7f36dcc00640 1 -- 172.21.15.2:0/1029429491 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0035070 con 0x7f36e0103ff0 2024-09-10T08:07:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.116+0000 7f36e6a00640 1 -- 172.21.15.2:0/1029429491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e01a72c0 con 0x7f36e0103ff0 2024-09-10T08:07:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.117+0000 7f36e6a00640 1 -- 172.21.15.2:0/1029429491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36e0062900 con 0x7f36e0103ff0 2024-09-10T08:07:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.119+0000 7f36dcc00640 1 -- 172.21.15.2:0/1029429491 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f36d0042030 con 0x7f36e0103ff0 2024-09-10T08:07:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.120+0000 7f36dcc00640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36c40a2c10 0x7f36c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.120+0000 7f36dcc00640 1 -- 172.21.15.2:0/1029429491 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f36d00eff60 con 0x7f36e0103ff0 2024-09-10T08:07:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.121+0000 7f36df400640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36c40a2c10 0x7f36c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.121+0000 7f36dcc00640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971] conn(0x7f36c40ae060 0x7f36c40b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.121+0000 7f36dcc00640 1 -- 172.21.15.2:0/1029429491 --> [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36d0001d30 con 0x7f36c40ae060 2024-09-10T08:07:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.121+0000 7f36dcc00640 1 -- 172.21.15.2:0/1029429491 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f36d00f0310 con 0x7f36e0103ff0 2024-09-10T08:07:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.121+0000 7f36e4a00640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971] conn(0x7f36c40ae060 0x7f36c40b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.121+0000 7f36df400640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36c40a2c10 0x7f36c40a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f36e0100020 tx=0x7f36cc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:05.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.122+0000 7f36e4a00640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971] conn(0x7f36c40ae060 0x7f36c40b0500 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:05.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.122+0000 7f36dcc00640 1 -- 172.21.15.2:0/1029429491 <== osd.4 v2:172.21.15.107:6808/485793971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36d0001d30 con 0x7f36c40ae060 2024-09-10T08:07:05.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.137+0000 7f36e6a00640 1 -- 172.21.15.2:0/1029429491 --> [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f36e0062d80 con 0x7f36c40ae060 2024-09-10T08:07:05.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.138+0000 7f36dcc00640 1 -- 172.21.15.2:0/1029429491 <== osd.4 v2:172.21.15.107:6808/485793971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f36e0062d80 con 0x7f36c40ae060 2024-09-10T08:07:05.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.138+0000 7f36e6a00640 1 -- 172.21.15.2:0/1029429491 >> [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971] conn(0x7f36c40ae060 msgr2=0x7f36c40b0500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:05.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.138+0000 7f36e6a00640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971] conn(0x7f36c40ae060 0x7f36c40b0500 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:05.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.139+0000 7f36e6a00640 1 -- 172.21.15.2:0/1029429491 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36c40a2c10 msgr2=0x7f36c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:05.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.139+0000 7f36e6a00640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36c40a2c10 0x7f36c40a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f36e0100020 tx=0x7f36cc009110 comp rx=0 tx=0).stop 2024-09-10T08:07:05.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.139+0000 7f36e6a00640 1 -- 172.21.15.2:0/1029429491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36e0103ff0 msgr2=0x7f36e019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:05.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.139+0000 7f36e6a00640 1 --2- 172.21.15.2:0/1029429491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36e0103ff0 0x7f36e019a740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f36d0006010 tx=0x7f36d00042f0 comp rx=0 tx=0).stop 2024-09-10T08:07:05.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.139+0000 7f36e4a00640 1 -- 172.21.15.2:0/1029429491 reap_dead start 2024-09-10T08:07:05.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.139+0000 7f36e6a00640 1 -- 172.21.15.2:0/1029429491 shutdown_connections 2024-09-10T08:07:05.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.139+0000 7f36e6a00640 1 -- 172.21.15.2:0/1029429491 >> 172.21.15.2:0/1029429491 conn(0x7f36e00ffa90 msgr2=0x7f36e0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:05.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.139+0000 7f36e6a00640 1 -- 172.21.15.2:0/1029429491 shutdown_connections 2024-09-10T08:07:05.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:05.139+0000 7f36e6a00640 1 -- 172.21.15.2:0/1029429491 wait complete. 2024-09-10T08:07:05.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:05 smithi155 ceph-mon[35405]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:05.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-10T08:07:05.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-10T08:07:05.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-10T08:07:05.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-09-10T08:07:05.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:05 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-09-10T08:07:05.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:05 smithi155 ceph-mon[35405]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T08:07:05.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:05 smithi107 ceph-mon[35048]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:05.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-10T08:07:05.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-10T08:07:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-10T08:07:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-09-10T08:07:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:05 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-09-10T08:07:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:05 smithi107 ceph-mon[35048]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T08:07:06.626 INFO:teuthology.orchestra.run.smithi002.stdout:326417514529 2024-09-10T08:07:06.627 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T08:07:06.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:06 smithi002 ceph-mon[30563]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T08:07:06.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:06 smithi155 ceph-mon[35405]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T08:07:06.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:06 smithi107 ceph-mon[35048]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T08:07:06.988 INFO:teuthology.orchestra.run.smithi002.stdout:77309411410 2024-09-10T08:07:06.988 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T08:07:07.585 INFO:teuthology.orchestra.run.smithi002.stdout:274877906986 2024-09-10T08:07:07.585 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T08:07:07.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:07 smithi002 ceph-mon[30563]: pgmap v356: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-10T08:07:07.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:07 smithi002 ceph-mon[30563]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-10T08:07:07.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:07 smithi155 ceph-mon[35405]: pgmap v356: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-10T08:07:07.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:07 smithi155 ceph-mon[35405]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-10T08:07:07.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:07 smithi107 ceph-mon[35048]: pgmap v356: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-10T08:07:07.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:07 smithi107 ceph-mon[35048]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-10T08:07:07.930 INFO:teuthology.orchestra.run.smithi002.stdout:296352743463 2024-09-10T08:07:07.930 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T08:07:08.417 INFO:teuthology.orchestra.run.smithi002.stdout:107374182476 2024-09-10T08:07:08.417 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T08:07:08.752 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:08.753 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:08.788 INFO:teuthology.orchestra.run.smithi002.stdout:55834574936 2024-09-10T08:07:08.788 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T08:07:09.264 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:09.273 INFO:teuthology.orchestra.run.smithi002.stdout:137438953543 2024-09-10T08:07:09.273 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T08:07:09.799 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:09 smithi002 ceph-mon[30563]: pgmap v357: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:07:09.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:09 smithi155 ceph-mon[35405]: pgmap v357: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:07:09.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:09 smithi107 ceph-mon[35048]: pgmap v357: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:07:10.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.288+0000 7f89f2800640 1 -- 172.21.15.2:0/11556065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec106f10 msgr2=0x7f89ec10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:10.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.288+0000 7f89f2800640 1 --2- 172.21.15.2:0/11556065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec106f10 0x7f89ec10ea90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009ca0 tx=0x7f89e402a8b0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.288+0000 7f89f2800640 1 -- 172.21.15.2:0/11556065 shutdown_connections 2024-09-10T08:07:10.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.288+0000 7f89f2800640 1 --2- 172.21.15.2:0/11556065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec106f10 0x7f89ec10ea90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.288+0000 7f89f2800640 1 --2- 172.21.15.2:0/11556065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89ec102790 0x7f89ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.288+0000 7f89f2800640 1 --2- 172.21.15.2:0/11556065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89ec101de0 0x7f89ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.288+0000 7f89f2800640 1 -- 172.21.15.2:0/11556065 >> 172.21.15.2:0/11556065 conn(0x7f89ec0fd8c0 msgr2=0x7f89ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:10.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.288+0000 7f89f2800640 1 -- 172.21.15.2:0/11556065 shutdown_connections 2024-09-10T08:07:10.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.288+0000 7f89f2800640 1 -- 172.21.15.2:0/11556065 wait complete. 2024-09-10T08:07:10.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.289+0000 7f89f2800640 1 Processor -- start 2024-09-10T08:07:10.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.289+0000 7f89f2800640 1 -- start start 2024-09-10T08:07:10.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.289+0000 7f89f2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89ec101de0 0x7f89ec19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:10.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.289+0000 7f89f2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec102790 0x7f89ec19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:10.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.289+0000 7f89f2800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89ec106f10 0x7f89ec19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:10.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.289+0000 7f89f2800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec111f70 con 0x7f89ec106f10 2024-09-10T08:07:10.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.289+0000 7f89f2800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec111df0 con 0x7f89ec101de0 2024-09-10T08:07:10.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.289+0000 7f89f2800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec1120f0 con 0x7f89ec102790 2024-09-10T08:07:10.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.290+0000 7f89ebe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89ec106f10 0x7f89ec19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:10.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.290+0000 7f89eb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89ec101de0 0x7f89ec19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.290+0000 7f89ebe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89ec106f10 0x7f89ec19eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53264/0 (socket says 172.21.15.2:53264) 2024-09-10T08:07:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.290+0000 7f89eb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89ec101de0 0x7f89ec19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52680/0 (socket says 172.21.15.2:52680) 2024-09-10T08:07:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.290+0000 7f89ebe00640 1 -- 172.21.15.2:0/1372579727 learned_addr learned my addr 172.21.15.2:0/1372579727 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.290+0000 7f89eaa00640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec102790 0x7f89ec19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.290+0000 7f89ebe00640 1 -- 172.21.15.2:0/1372579727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec102790 msgr2=0x7f89ec19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.290+0000 7f89ebe00640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec102790 0x7f89ec19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.290+0000 7f89ebe00640 1 -- 172.21.15.2:0/1372579727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89ec101de0 msgr2=0x7f89ec19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.290+0000 7f89ebe00640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89ec101de0 0x7f89ec19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.290+0000 7f89ebe00640 1 -- 172.21.15.2:0/1372579727 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec19f5a0 con 0x7f89ec106f10 2024-09-10T08:07:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.290+0000 7f89eb400640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89ec101de0 0x7f89ec19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.290+0000 7f89ebe00640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89ec106f10 0x7f89ec19eec0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f89e4006010 tx=0x7f89e4004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.291+0000 7f89e3e00640 1 -- 172.21.15.2:0/1372579727 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4003d80 con 0x7f89ec106f10 2024-09-10T08:07:10.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.291+0000 7f89e3e00640 1 -- 172.21.15.2:0/1372579727 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89e4040070 con 0x7f89ec106f10 2024-09-10T08:07:10.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.291+0000 7f89e3e00640 1 -- 172.21.15.2:0/1372579727 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4045ad0 con 0x7f89ec106f10 2024-09-10T08:07:10.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.291+0000 7f89f2800640 1 -- 172.21.15.2:0/1372579727 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ec19f830 con 0x7f89ec106f10 2024-09-10T08:07:10.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.291+0000 7f89f2800640 1 -- 172.21.15.2:0/1372579727 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ec1a7170 con 0x7f89ec106f10 2024-09-10T08:07:10.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.292+0000 7f89f2800640 1 -- 172.21.15.2:0/1372579727 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ec1021c0 con 0x7f89ec106f10 2024-09-10T08:07:10.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.294+0000 7f89e3e00640 1 -- 172.21.15.2:0/1372579727 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f89e4045c70 con 0x7f89ec106f10 2024-09-10T08:07:10.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.295+0000 7f89e3e00640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f89d00a2c10 0x7f89d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:10.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.295+0000 7f89e3e00640 1 -- 172.21.15.2:0/1372579727 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f89e40f05c0 con 0x7f89ec106f10 2024-09-10T08:07:10.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.295+0000 7f89eb400640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f89d00a2c10 0x7f89d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:10.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.296+0000 7f89eb400640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f89d00a2c10 0x7f89d00a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0095f0 tx=0x7f89dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:10.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.299+0000 7f89e3e00640 1 -- 172.21.15.2:0/1372579727 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89e40bd080 con 0x7f89ec106f10 2024-09-10T08:07:10.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.495+0000 7f89f2800640 1 -- 172.21.15.2:0/1372579727 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f89ec1a73d0 con 0x7f89ec106f10 2024-09-10T08:07:10.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.496+0000 7f89e3e00640 1 -- 172.21.15.2:0/1372579727 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f89e40bc980 con 0x7f89ec106f10 2024-09-10T08:07:10.497 INFO:teuthology.orchestra.run.smithi002.stdout:214748364854 2024-09-10T08:07:10.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.498+0000 7f89f2800640 1 -- 172.21.15.2:0/1372579727 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f89d00a2c10 msgr2=0x7f89d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:10.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.498+0000 7f89f2800640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f89d00a2c10 0x7f89d00a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0095f0 tx=0x7f89dc005f90 comp rx=0 tx=0).stop 2024-09-10T08:07:10.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.498+0000 7f89f2800640 1 -- 172.21.15.2:0/1372579727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89ec106f10 msgr2=0x7f89ec19eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.498+0000 7f89f2800640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89ec106f10 0x7f89ec19eec0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f89e4006010 tx=0x7f89e4004a60 comp rx=0 tx=0).stop 2024-09-10T08:07:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.498+0000 7f89f2800640 1 -- 172.21.15.2:0/1372579727 shutdown_connections 2024-09-10T08:07:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.498+0000 7f89f2800640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89ec106f10 0x7f89ec19eec0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.498+0000 7f89f2800640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f89d00a2c10 0x7f89d00a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.498+0000 7f89f2800640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec102790 0x7f89ec19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.498+0000 7f89f2800640 1 --2- 172.21.15.2:0/1372579727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89ec101de0 0x7f89ec19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.498+0000 7f89f2800640 1 -- 172.21.15.2:0/1372579727 >> 172.21.15.2:0/1372579727 conn(0x7f89ec0fd8c0 msgr2=0x7f89ec0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.498+0000 7f89f2800640 1 -- 172.21.15.2:0/1372579727 shutdown_connections 2024-09-10T08:07:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.498+0000 7f89f2800640 1 -- 172.21.15.2:0/1372579727 wait complete. 2024-09-10T08:07:10.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.194+0000 7fd8f5200640 1 -- 172.21.15.2:0/852727357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8f0103ff0 msgr2=0x7fd8f01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:10.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.194+0000 7fd8f5200640 1 --2- 172.21.15.2:0/852727357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8f0103ff0 0x7fd8f01043d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009ca0 tx=0x7fd8e002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.194+0000 7fd8f5200640 1 -- 172.21.15.2:0/852727357 shutdown_connections 2024-09-10T08:07:10.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.194+0000 7fd8f5200640 1 --2- 172.21.15.2:0/852727357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f0109120 0x7fd8f0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.194+0000 7fd8f5200640 1 --2- 172.21.15.2:0/852727357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f01049a0 0x7fd8f01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.194+0000 7fd8f5200640 1 --2- 172.21.15.2:0/852727357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8f0103ff0 0x7fd8f01043d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.194+0000 7fd8f5200640 1 -- 172.21.15.2:0/852727357 >> 172.21.15.2:0/852727357 conn(0x7fd8f00ffa90 msgr2=0x7fd8f0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:10.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.194+0000 7fd8f5200640 1 -- 172.21.15.2:0/852727357 shutdown_connections 2024-09-10T08:07:10.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.194+0000 7fd8f5200640 1 -- 172.21.15.2:0/852727357 wait complete. 2024-09-10T08:07:10.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.195+0000 7fd8f5200640 1 Processor -- start 2024-09-10T08:07:10.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.195+0000 7fd8f5200640 1 -- start start 2024-09-10T08:07:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.195+0000 7fd8f5200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8f0103ff0 0x7fd8f019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.195+0000 7fd8f5200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f01049a0 0x7fd8f019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.195+0000 7fd8f5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f0109120 0x7fd8f019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.195+0000 7fd8f5200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f0114120 con 0x7fd8f0103ff0 2024-09-10T08:07:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.195+0000 7fd8f5200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f0113fa0 con 0x7fd8f01049a0 2024-09-10T08:07:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.195+0000 7fd8f5200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f01142a0 con 0x7fd8f0109120 2024-09-10T08:07:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.196+0000 7fd8ede00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8f0103ff0 0x7fd8f019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.196+0000 7fd8e5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f01049a0 0x7fd8f019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.196+0000 7fd8ede00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8f0103ff0 0x7fd8f019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53246/0 (socket says 172.21.15.2:53246) 2024-09-10T08:07:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.196+0000 7fd8ede00640 1 -- 172.21.15.2:0/1186436434 learned_addr learned my addr 172.21.15.2:0/1186436434 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.196+0000 7fd8ee800640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f0109120 0x7fd8f019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.196+0000 7fd8ede00640 1 -- 172.21.15.2:0/1186436434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f0109120 msgr2=0x7fd8f019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.196+0000 7fd8ede00640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f0109120 0x7fd8f019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.196+0000 7fd8ede00640 1 -- 172.21.15.2:0/1186436434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f01049a0 msgr2=0x7fd8f019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.196+0000 7fd8ede00640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f01049a0 0x7fd8f019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.196+0000 7fd8ede00640 1 -- 172.21.15.2:0/1186436434 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f019f5d0 con 0x7fd8f0103ff0 2024-09-10T08:07:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.196+0000 7fd8e5400640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f01049a0 0x7fd8f019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.197+0000 7fd8ede00640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8f0103ff0 0x7fd8f019a620 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd8e002ade0 tx=0x7fd8e00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.197+0000 7fd8e7400640 1 -- 172.21.15.2:0/1186436434 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e002fa20 con 0x7fd8f0103ff0 2024-09-10T08:07:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.197+0000 7fd8e7400640 1 -- 172.21.15.2:0/1186436434 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e002fbc0 con 0x7fd8f0103ff0 2024-09-10T08:07:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.197+0000 7fd8f5200640 1 -- 172.21.15.2:0/1186436434 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f019f860 con 0x7fd8f0103ff0 2024-09-10T08:07:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.197+0000 7fd8e7400640 1 -- 172.21.15.2:0/1186436434 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00467b0 con 0x7fd8f0103ff0 2024-09-10T08:07:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.197+0000 7fd8f5200640 1 -- 172.21.15.2:0/1186436434 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f01a71a0 con 0x7fd8f0103ff0 2024-09-10T08:07:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.198+0000 7fd8f5200640 1 -- 172.21.15.2:0/1186436434 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd8f0062980 con 0x7fd8f0103ff0 2024-09-10T08:07:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.200+0000 7fd8e7400640 1 -- 172.21.15.2:0/1186436434 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd8e0046950 con 0x7fd8f0103ff0 2024-09-10T08:07:10.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.201+0000 7fd8e7400640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8d40a2c10 0x7fd8d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:10.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.202+0000 7fd8e7400640 1 -- 172.21.15.2:0/1186436434 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd8e00f0d60 con 0x7fd8f0103ff0 2024-09-10T08:07:10.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.202+0000 7fd8e5400640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8d40a2c10 0x7fd8d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:10.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.202+0000 7fd8e7400640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139] conn(0x7fd8d40ae060 0x7fd8d40b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:10.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.202+0000 7fd8e7400640 1 -- 172.21.15.2:0/1186436434 --> [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8d40b0bd0 con 0x7fd8d40ae060 2024-09-10T08:07:10.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.202+0000 7fd8e7400640 1 -- 172.21.15.2:0/1186436434 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fd8e00f1110 con 0x7fd8f0103ff0 2024-09-10T08:07:10.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.202+0000 7fd8ee800640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139] conn(0x7fd8d40ae060 0x7fd8d40b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:10.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.203+0000 7fd8e5400640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8d40a2c10 0x7fd8d40a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00ffeb0 tx=0x7fd8d800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:10.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.203+0000 7fd8ee800640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139] conn(0x7fd8d40ae060 0x7fd8d40b0500 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:10.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.204+0000 7fd8e7400640 1 -- 172.21.15.2:0/1186436434 <== osd.0 v2:172.21.15.2:6802/3825226139 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd8d40b0bd0 con 0x7fd8d40ae060 2024-09-10T08:07:10.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.218+0000 7fd8f5200640 1 -- 172.21.15.2:0/1186436434 --> [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd8f0062e00 con 0x7fd8d40ae060 2024-09-10T08:07:10.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.219+0000 7fd8e7400640 1 -- 172.21.15.2:0/1186436434 <== osd.0 v2:172.21.15.2:6802/3825226139 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd8f0062e00 con 0x7fd8d40ae060 2024-09-10T08:07:10.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.219+0000 7fd8f5200640 1 -- 172.21.15.2:0/1186436434 >> [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139] conn(0x7fd8d40ae060 msgr2=0x7fd8d40b0500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:10.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.219+0000 7fd8f5200640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139] conn(0x7fd8d40ae060 0x7fd8d40b0500 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.219+0000 7fd8f5200640 1 -- 172.21.15.2:0/1186436434 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8d40a2c10 msgr2=0x7fd8d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:10.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.219+0000 7fd8f5200640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8d40a2c10 0x7fd8d40a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00ffeb0 tx=0x7fd8d800b390 comp rx=0 tx=0).stop 2024-09-10T08:07:10.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.219+0000 7fd8f5200640 1 -- 172.21.15.2:0/1186436434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8f0103ff0 msgr2=0x7fd8f019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:10.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.219+0000 7fd8f5200640 1 --2- 172.21.15.2:0/1186436434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8f0103ff0 0x7fd8f019a620 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd8e002ade0 tx=0x7fd8e00319f0 comp rx=0 tx=0).stop 2024-09-10T08:07:10.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.219+0000 7fd8ee800640 1 -- 172.21.15.2:0/1186436434 reap_dead start 2024-09-10T08:07:10.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.220+0000 7fd8f5200640 1 -- 172.21.15.2:0/1186436434 shutdown_connections 2024-09-10T08:07:10.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.220+0000 7fd8f5200640 1 -- 172.21.15.2:0/1186436434 >> 172.21.15.2:0/1186436434 conn(0x7fd8f00ffa90 msgr2=0x7fd8f0100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:10.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.220+0000 7fd8f5200640 1 -- 172.21.15.2:0/1186436434 shutdown_connections 2024-09-10T08:07:10.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:10.220+0000 7fd8f5200640 1 -- 172.21.15.2:0/1186436434 wait complete. 2024-09-10T08:07:10.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:10 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1372579727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T08:07:10.909 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:10 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1372579727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T08:07:10.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:10 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1372579727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T08:07:11.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.619+0000 7fc3abe00640 1 -- 172.21.15.2:0/282193542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ac10bfc0 msgr2=0x7fc3ac10e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:11.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.619+0000 7fc3abe00640 1 --2- 172.21.15.2:0/282193542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ac10bfc0 0x7fc3ac10e460 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009ca0 tx=0x7fc3a402a950 comp rx=0 tx=0).stop 2024-09-10T08:07:11.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.619+0000 7fc3abe00640 1 -- 172.21.15.2:0/282193542 shutdown_connections 2024-09-10T08:07:11.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.619+0000 7fc3abe00640 1 --2- 172.21.15.2:0/282193542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ac10bfc0 0x7fc3ac10e460 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:11.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.619+0000 7fc3abe00640 1 --2- 172.21.15.2:0/282193542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3ac1034b0 0x7fc3ac10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:11.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.619+0000 7fc3abe00640 1 --2- 172.21.15.2:0/282193542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ac102b90 0x7fc3ac102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:11.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.619+0000 7fc3abe00640 1 -- 172.21.15.2:0/282193542 >> 172.21.15.2:0/282193542 conn(0x7fc3ac0fc610 msgr2=0x7fc3ac0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:11.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.619+0000 7fc3abe00640 1 -- 172.21.15.2:0/282193542 shutdown_connections 2024-09-10T08:07:11.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.619+0000 7fc3abe00640 1 -- 172.21.15.2:0/282193542 wait complete. 2024-09-10T08:07:11.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.620+0000 7fc3abe00640 1 Processor -- start 2024-09-10T08:07:11.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.620+0000 7fc3abe00640 1 -- start start 2024-09-10T08:07:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.621+0000 7fc3abe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3ac102b90 0x7fc3ac19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.621+0000 7fc3abe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ac1034b0 0x7fc3ac19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.621+0000 7fc3abe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ac10bfc0 0x7fc3ac19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.621+0000 7fc3abe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac111e10 con 0x7fc3ac10bfc0 2024-09-10T08:07:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.621+0000 7fc3abe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac111c90 con 0x7fc3ac102b90 2024-09-10T08:07:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.621+0000 7fc3abe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac111f90 con 0x7fc3ac1034b0 2024-09-10T08:07:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.621+0000 7fc3ab400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ac10bfc0 0x7fc3ac19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.621+0000 7fc3a2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ac1034b0 0x7fc3ac19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.621+0000 7fc3aaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3ac102b90 0x7fc3ac19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.621+0000 7fc3ab400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ac10bfc0 0x7fc3ac19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53284/0 (socket says 172.21.15.2:53284) 2024-09-10T08:07:11.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.621+0000 7fc3aaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3ac102b90 0x7fc3ac19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52688/0 (socket says 172.21.15.2:52688) 2024-09-10T08:07:11.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.621+0000 7fc3ab400640 1 -- 172.21.15.2:0/3678761895 learned_addr learned my addr 172.21.15.2:0/3678761895 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:11.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.621+0000 7fc3a2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ac1034b0 0x7fc3ac19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51244/0 (socket says 172.21.15.2:51244) 2024-09-10T08:07:11.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.622+0000 7fc3a2000640 1 -- 172.21.15.2:0/3678761895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3ac102b90 msgr2=0x7fc3ac19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:11.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.622+0000 7fc3a2000640 1 --2- 172.21.15.2:0/3678761895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3ac102b90 0x7fc3ac19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:11.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.622+0000 7fc3a2000640 1 -- 172.21.15.2:0/3678761895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ac10bfc0 msgr2=0x7fc3ac19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:11.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.622+0000 7fc3a2000640 1 --2- 172.21.15.2:0/3678761895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ac10bfc0 0x7fc3ac19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:11.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.622+0000 7fc3a2000640 1 -- 172.21.15.2:0/3678761895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac19f410 con 0x7fc3ac1034b0 2024-09-10T08:07:11.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.622+0000 7fc3a2000640 1 --2- 172.21.15.2:0/3678761895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ac1034b0 0x7fc3ac19a9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc39400e7a0 tx=0x7fc39400ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:11.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.622+0000 7fc3b0a00640 1 -- 172.21.15.2:0/3678761895 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39400ccc0 con 0x7fc3ac1034b0 2024-09-10T08:07:11.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.622+0000 7fc3b0a00640 1 -- 172.21.15.2:0/3678761895 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc394009530 con 0x7fc3ac1034b0 2024-09-10T08:07:11.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.622+0000 7fc3abe00640 1 -- 172.21.15.2:0/3678761895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ac19f700 con 0x7fc3ac1034b0 2024-09-10T08:07:11.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.622+0000 7fc3b0a00640 1 -- 172.21.15.2:0/3678761895 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394018880 con 0x7fc3ac1034b0 2024-09-10T08:07:11.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.622+0000 7fc3abe00640 1 -- 172.21.15.2:0/3678761895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ac1a6fe0 con 0x7fc3ac1034b0 2024-09-10T08:07:11.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.623+0000 7fc3abe00640 1 -- 172.21.15.2:0/3678761895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc3ac102260 con 0x7fc3ac1034b0 2024-09-10T08:07:11.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.626+0000 7fc3b0a00640 1 -- 172.21.15.2:0/3678761895 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc394018a20 con 0x7fc3ac1034b0 2024-09-10T08:07:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.627+0000 7fc3b0a00640 1 --2- 172.21.15.2:0/3678761895 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3900a2c10 0x7fc3900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.628+0000 7fc3b0a00640 1 -- 172.21.15.2:0/3678761895 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc3940cff80 con 0x7fc3ac1034b0 2024-09-10T08:07:11.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.628+0000 7fc3aaa00640 1 --2- 172.21.15.2:0/3678761895 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3900a2c10 0x7fc3900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.628+0000 7fc3b0a00640 1 --2- 172.21.15.2:0/3678761895 >> [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362] conn(0x7fc3900ae060 0x7fc3900b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.628+0000 7fc3b0a00640 1 -- 172.21.15.2:0/3678761895 --> [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc394028ea0 con 0x7fc3900ae060 2024-09-10T08:07:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.628+0000 7fc3b0a00640 1 -- 172.21.15.2:0/3678761895 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fc3940d0330 con 0x7fc3ac1034b0 2024-09-10T08:07:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.628+0000 7fc3ab400640 1 --2- 172.21.15.2:0/3678761895 >> [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362] conn(0x7fc3900ae060 0x7fc3900b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.628+0000 7fc3aaa00640 1 --2- 172.21.15.2:0/3678761895 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3900a2c10 0x7fc3900a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009bd0 tx=0x7fc39c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:11.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.629+0000 7fc3ab400640 1 --2- 172.21.15.2:0/3678761895 >> [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362] conn(0x7fc3900ae060 0x7fc3900b0500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:11.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.630+0000 7fc3b0a00640 1 -- 172.21.15.2:0/3678761895 <== osd.5 v2:172.21.15.155:6808/2180279362 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc394028ea0 con 0x7fc3900ae060 2024-09-10T08:07:11.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.644+0000 7fc3abe00640 1 -- 172.21.15.2:0/3678761895 --> [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc3ac06a850 con 0x7fc3900ae060 2024-09-10T08:07:11.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.645+0000 7fc3b0a00640 1 -- 172.21.15.2:0/3678761895 <== osd.5 v2:172.21.15.155:6808/2180279362 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc3ac06a850 con 0x7fc3900ae060 2024-09-10T08:07:11.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.645+0000 7fc3abe00640 1 -- 172.21.15.2:0/3678761895 >> [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362] conn(0x7fc3900ae060 msgr2=0x7fc3900b0500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:11.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.645+0000 7fc3abe00640 1 --2- 172.21.15.2:0/3678761895 >> [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362] conn(0x7fc3900ae060 0x7fc3900b0500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:11.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.646+0000 7fc3abe00640 1 -- 172.21.15.2:0/3678761895 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3900a2c10 msgr2=0x7fc3900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:11.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.646+0000 7fc3abe00640 1 --2- 172.21.15.2:0/3678761895 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3900a2c10 0x7fc3900a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009bd0 tx=0x7fc39c005f90 comp rx=0 tx=0).stop 2024-09-10T08:07:11.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.646+0000 7fc3abe00640 1 -- 172.21.15.2:0/3678761895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ac1034b0 msgr2=0x7fc3ac19a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:11.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.646+0000 7fc3abe00640 1 --2- 172.21.15.2:0/3678761895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ac1034b0 0x7fc3ac19a9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc39400e7a0 tx=0x7fc39400ec60 comp rx=0 tx=0).stop 2024-09-10T08:07:11.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.646+0000 7fc3ab400640 1 -- 172.21.15.2:0/3678761895 reap_dead start 2024-09-10T08:07:11.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.646+0000 7fc3abe00640 1 -- 172.21.15.2:0/3678761895 shutdown_connections 2024-09-10T08:07:11.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.646+0000 7fc3abe00640 1 -- 172.21.15.2:0/3678761895 >> 172.21.15.2:0/3678761895 conn(0x7fc3ac0fc610 msgr2=0x7fc3ac10d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:11.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.646+0000 7fc3abe00640 1 -- 172.21.15.2:0/3678761895 shutdown_connections 2024-09-10T08:07:11.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:11.646+0000 7fc3abe00640 1 -- 172.21.15.2:0/3678761895 wait complete. 2024-09-10T08:07:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:11 smithi002 ceph-mon[30563]: pgmap v358: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T08:07:11.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:11 smithi155 ceph-mon[35405]: pgmap v358: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T08:07:11.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:11 smithi107 ceph-mon[35048]: pgmap v358: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T08:07:12.704 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364852 got 214748364854 for osd.7 2024-09-10T08:07:12.705 DEBUG:teuthology.parallel:result is None 2024-09-10T08:07:12.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:12 smithi155 ceph-mon[35405]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-10T08:07:12.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:12 smithi155 ceph-mon[35405]: Cluster is now healthy 2024-09-10T08:07:12.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:12 smithi107 ceph-mon[35048]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-10T08:07:12.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:12 smithi107 ceph-mon[35048]: Cluster is now healthy 2024-09-10T08:07:13.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:12 smithi002 ceph-mon[30563]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-10T08:07:13.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:12 smithi002 ceph-mon[30563]: Cluster is now healthy 2024-09-10T08:07:13.156 INFO:teuthology.orchestra.run.smithi002.stdout:34359738461 2024-09-10T08:07:13.156 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T08:07:13.643 INFO:teuthology.orchestra.run.smithi002.stdout:158913790020 2024-09-10T08:07:13.643 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T08:07:13.687 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:13.687 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:13.694 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:13.702 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:13.703 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:13.708 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:13.866 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:13 smithi002 ceph-mon[30563]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T08:07:13.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:13 smithi155 ceph-mon[35405]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T08:07:13.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:13 smithi107 ceph-mon[35048]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-10T08:07:14.790 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:14.794 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:14.794 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.604+0000 7fce4dc00640 1 -- 172.21.15.2:0/2352010790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce48103ff0 msgr2=0x7fce481043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.604+0000 7fce4dc00640 1 --2- 172.21.15.2:0/2352010790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce48103ff0 0x7fce481043d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fce38009ca0 tx=0x7fce3802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.604+0000 7fce4dc00640 1 -- 172.21.15.2:0/2352010790 shutdown_connections 2024-09-10T08:07:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.604+0000 7fce4dc00640 1 --2- 172.21.15.2:0/2352010790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce48109120 0x7fce48110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.604+0000 7fce4dc00640 1 --2- 172.21.15.2:0/2352010790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce481049a0 0x7fce481089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.604+0000 7fce4dc00640 1 --2- 172.21.15.2:0/2352010790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce48103ff0 0x7fce481043d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.604+0000 7fce4dc00640 1 -- 172.21.15.2:0/2352010790 >> 172.21.15.2:0/2352010790 conn(0x7fce480ffa90 msgr2=0x7fce48101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.604+0000 7fce4dc00640 1 -- 172.21.15.2:0/2352010790 shutdown_connections 2024-09-10T08:07:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.605+0000 7fce4dc00640 1 -- 172.21.15.2:0/2352010790 wait complete. 2024-09-10T08:07:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.605+0000 7fce4dc00640 1 Processor -- start 2024-09-10T08:07:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.606+0000 7fce4dc00640 1 -- start start 2024-09-10T08:07:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.606+0000 7fce4dc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce48103ff0 0x7fce4819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.606+0000 7fce4dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce481049a0 0x7fce4819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.606+0000 7fce4dc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce48109120 0x7fce4819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.606+0000 7fce4dc00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fce48114190 con 0x7fce48109120 2024-09-10T08:07:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.606+0000 7fce4dc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fce48114010 con 0x7fce481049a0 2024-09-10T08:07:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.606+0000 7fce4dc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce48114310 con 0x7fce48103ff0 2024-09-10T08:07:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.606+0000 7fce47400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce48109120 0x7fce4819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.606+0000 7fce47400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce48109120 0x7fce4819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53296/0 (socket says 172.21.15.2:53296) 2024-09-10T08:07:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.606+0000 7fce47400640 1 -- 172.21.15.2:0/966506808 learned_addr learned my addr 172.21.15.2:0/966506808 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.606+0000 7fce3e000640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce481049a0 0x7fce4819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.606+0000 7fce46a00640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce48103ff0 0x7fce4819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.607+0000 7fce47400640 1 -- 172.21.15.2:0/966506808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce48103ff0 msgr2=0x7fce4819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.607+0000 7fce47400640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce48103ff0 0x7fce4819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.607+0000 7fce47400640 1 -- 172.21.15.2:0/966506808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce481049a0 msgr2=0x7fce4819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.607+0000 7fce47400640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce481049a0 0x7fce4819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.607+0000 7fce47400640 1 -- 172.21.15.2:0/966506808 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4819f640 con 0x7fce48109120 2024-09-10T08:07:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.607+0000 7fce46a00640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce48103ff0 0x7fce4819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:07:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.607+0000 7fce47400640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce48109120 0x7fce4819ef60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fce4000d750 tx=0x7fce4000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.607+0000 7fce3fe00640 1 -- 172.21.15.2:0/966506808 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40017070 con 0x7fce48109120 2024-09-10T08:07:15.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.607+0000 7fce3fe00640 1 -- 172.21.15.2:0/966506808 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce40005500 con 0x7fce48109120 2024-09-10T08:07:15.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.607+0000 7fce3fe00640 1 -- 172.21.15.2:0/966506808 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40004d90 con 0x7fce48109120 2024-09-10T08:07:15.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.607+0000 7fce4dc00640 1 -- 172.21.15.2:0/966506808 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4819f930 con 0x7fce48109120 2024-09-10T08:07:15.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.607+0000 7fce4dc00640 1 -- 172.21.15.2:0/966506808 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce481a7210 con 0x7fce48109120 2024-09-10T08:07:15.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.609+0000 7fce4dc00640 1 -- 172.21.15.2:0/966506808 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce481043d0 con 0x7fce48109120 2024-09-10T08:07:15.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.610+0000 7fce3fe00640 1 -- 172.21.15.2:0/966506808 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fce40004f30 con 0x7fce48109120 2024-09-10T08:07:15.315 INFO:teuthology.orchestra.run.smithi002.stdout:326417514532 2024-09-10T08:07:15.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.611+0000 7fce3fe00640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce2c0a2c10 0x7fce2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:15.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.612+0000 7fce3fe00640 1 -- 172.21.15.2:0/966506808 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fce400cef90 con 0x7fce48109120 2024-09-10T08:07:15.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.612+0000 7fce46a00640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce2c0a2c10 0x7fce2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:15.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.612+0000 7fce46a00640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce2c0a2c10 0x7fce2c0a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce38000f80 tx=0x7fce38005fa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:15.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.616+0000 7fce3fe00640 1 -- 172.21.15.2:0/966506808 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce4009b940 con 0x7fce48109120 2024-09-10T08:07:15.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.828+0000 7fce4dc00640 1 -- 172.21.15.2:0/966506808 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fce4810c360 con 0x7fce48109120 2024-09-10T08:07:15.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.828+0000 7fce3fe00640 1 -- 172.21.15.2:0/966506808 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fce4009b240 con 0x7fce48109120 2024-09-10T08:07:15.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.830+0000 7fce4dc00640 1 -- 172.21.15.2:0/966506808 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce2c0a2c10 msgr2=0x7fce2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:15.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.830+0000 7fce4dc00640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce2c0a2c10 0x7fce2c0a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce38000f80 tx=0x7fce38005fa0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.830+0000 7fce4dc00640 1 -- 172.21.15.2:0/966506808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce48109120 msgr2=0x7fce4819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:15.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.830+0000 7fce4dc00640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce48109120 0x7fce4819ef60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fce4000d750 tx=0x7fce4000dc10 comp rx=0 tx=0).stop 2024-09-10T08:07:15.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.831+0000 7fce4dc00640 1 -- 172.21.15.2:0/966506808 shutdown_connections 2024-09-10T08:07:15.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.831+0000 7fce4dc00640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce48109120 0x7fce4819ef60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.831+0000 7fce4dc00640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce481049a0 0x7fce4819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.831+0000 7fce4dc00640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce2c0a2c10 0x7fce2c0a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.831+0000 7fce4dc00640 1 --2- 172.21.15.2:0/966506808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce48103ff0 0x7fce4819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.831+0000 7fce4dc00640 1 -- 172.21.15.2:0/966506808 >> 172.21.15.2:0/966506808 conn(0x7fce480ffa90 msgr2=0x7fce48100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:15.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.831+0000 7fce4dc00640 1 -- 172.21.15.2:0/966506808 shutdown_connections 2024-09-10T08:07:15.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:14.831+0000 7fce4dc00640 1 -- 172.21.15.2:0/966506808 wait complete. 2024-09-10T08:07:15.857 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:15 smithi002 ceph-mon[30563]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T08:07:15.857 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:15 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/966506808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T08:07:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:15 smithi155 ceph-mon[35405]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T08:07:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:15 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/966506808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T08:07:15.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:15 smithi107 ceph-mon[35048]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T08:07:15.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:15 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/966506808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T08:07:15.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.822+0000 7f834f800640 1 -- 172.21.15.2:0/2608519781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83481027a0 msgr2=0x7f83481067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:15.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.822+0000 7f834f800640 1 --2- 172.21.15.2:0/2608519781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83481027a0 0x7f83481067f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8334009860 tx=0x7f833402a340 comp rx=0 tx=0).stop 2024-09-10T08:07:15.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.823+0000 7f834f800640 1 -- 172.21.15.2:0/2608519781 shutdown_connections 2024-09-10T08:07:15.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.823+0000 7f834f800640 1 --2- 172.21.15.2:0/2608519781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8348106f20 0x7f834810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.823+0000 7f834f800640 1 --2- 172.21.15.2:0/2608519781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83481027a0 0x7f83481067f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.823+0000 7f834f800640 1 --2- 172.21.15.2:0/2608519781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8348101df0 0x7f83481021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.823+0000 7f834f800640 1 -- 172.21.15.2:0/2608519781 >> 172.21.15.2:0/2608519781 conn(0x7f83480fd8b0 msgr2=0x7f83480ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:15.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.823+0000 7f834f800640 1 -- 172.21.15.2:0/2608519781 shutdown_connections 2024-09-10T08:07:15.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.823+0000 7f834f800640 1 -- 172.21.15.2:0/2608519781 wait complete. 2024-09-10T08:07:15.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.823+0000 7f834f800640 1 Processor -- start 2024-09-10T08:07:15.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.824+0000 7f834f800640 1 -- start start 2024-09-10T08:07:15.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.824+0000 7f834f800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8348101df0 0x7f83480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:15.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.824+0000 7f834f800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83481027a0 0x7f8348074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:15.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.824+0000 7f834f800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8348106f20 0x7f8348074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:15.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.824+0000 7f834f800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8348111f40 con 0x7f8348106f20 2024-09-10T08:07:15.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.824+0000 7f834f800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8348111dc0 con 0x7f83481027a0 2024-09-10T08:07:15.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.824+0000 7f834f800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f83481120c0 con 0x7f8348101df0 2024-09-10T08:07:15.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.824+0000 7f834d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8348106f20 0x7f8348074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:15.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.824+0000 7f834ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8348101df0 0x7f83480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:15.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.824+0000 7f834d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8348106f20 0x7f8348074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53312/0 (socket says 172.21.15.2:53312) 2024-09-10T08:07:15.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.824+0000 7f834d800640 1 -- 172.21.15.2:0/2283659608 learned_addr learned my addr 172.21.15.2:0/2283659608 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:15.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.824+0000 7f834ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8348101df0 0x7f83480744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51268/0 (socket says 172.21.15.2:51268) 2024-09-10T08:07:15.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.824+0000 7f8347e00640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83481027a0 0x7f8348074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:15.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.825+0000 7f834d800640 1 -- 172.21.15.2:0/2283659608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8348101df0 msgr2=0x7f83480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:15.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.825+0000 7f834d800640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8348101df0 0x7f83480744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.825+0000 7f834d800640 1 -- 172.21.15.2:0/2283659608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83481027a0 msgr2=0x7f8348074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:15.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.825+0000 7f834d800640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83481027a0 0x7f8348074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.825+0000 7f834d800640 1 -- 172.21.15.2:0/2283659608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83481a6990 con 0x7f8348106f20 2024-09-10T08:07:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.825+0000 7f834ce00640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8348101df0 0x7f83480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.825+0000 7f834d800640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8348106f20 0x7f8348074f70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f834000eb10 tx=0x7f834000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.825+0000 7f8345600640 1 -- 172.21.15.2:0/2283659608 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834000ce20 con 0x7f8348106f20 2024-09-10T08:07:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.826+0000 7f8345600640 1 -- 172.21.15.2:0/2283659608 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8340005020 con 0x7f8348106f20 2024-09-10T08:07:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.826+0000 7f8345600640 1 -- 172.21.15.2:0/2283659608 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8340010690 con 0x7f8348106f20 2024-09-10T08:07:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.826+0000 7f834f800640 1 -- 172.21.15.2:0/2283659608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83481a6c80 con 0x7f8348106f20 2024-09-10T08:07:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.826+0000 7f834f800640 1 -- 172.21.15.2:0/2283659608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83481a7140 con 0x7f8348106f20 2024-09-10T08:07:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.827+0000 7f834f800640 1 -- 172.21.15.2:0/2283659608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83481021d0 con 0x7f8348106f20 2024-09-10T08:07:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.829+0000 7f8345600640 1 -- 172.21.15.2:0/2283659608 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8340005540 con 0x7f8348106f20 2024-09-10T08:07:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.830+0000 7f8345600640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83300a2c90 0x7f83300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.830+0000 7f8345600640 1 -- 172.21.15.2:0/2283659608 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f83400cfac0 con 0x7f8348106f20 2024-09-10T08:07:15.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.831+0000 7f834ce00640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83300a2c90 0x7f83300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:15.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.831+0000 7f834ce00640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83300a2c90 0x7f83300a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f833c005fd0 tx=0x7f833c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:15.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:15.834+0000 7f8345600640 1 -- 172.21.15.2:0/2283659608 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f834009c470 con 0x7f8348106f20 2024-09-10T08:07:16.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.027+0000 7f834f800640 1 -- 172.21.15.2:0/2283659608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f834810a160 con 0x7f8348106f20 2024-09-10T08:07:16.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.027+0000 7f8345600640 1 -- 172.21.15.2:0/2283659608 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f834009bd70 con 0x7f8348106f20 2024-09-10T08:07:16.031 INFO:teuthology.orchestra.run.smithi002.stdout:244813135924 2024-09-10T08:07:16.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.030+0000 7f834f800640 1 -- 172.21.15.2:0/2283659608 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83300a2c90 msgr2=0x7f83300a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:16.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.030+0000 7f834f800640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83300a2c90 0x7f83300a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f833c005fd0 tx=0x7f833c009260 comp rx=0 tx=0).stop 2024-09-10T08:07:16.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.030+0000 7f834f800640 1 -- 172.21.15.2:0/2283659608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8348106f20 msgr2=0x7f8348074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:16.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.030+0000 7f834f800640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8348106f20 0x7f8348074f70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f834000eb10 tx=0x7f834000ef10 comp rx=0 tx=0).stop 2024-09-10T08:07:16.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.030+0000 7f834f800640 1 -- 172.21.15.2:0/2283659608 shutdown_connections 2024-09-10T08:07:16.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.030+0000 7f834f800640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8348106f20 0x7f8348074f70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.030+0000 7f834f800640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83481027a0 0x7f8348074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.030+0000 7f834f800640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83300a2c90 0x7f83300a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.030+0000 7f834f800640 1 --2- 172.21.15.2:0/2283659608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8348101df0 0x7f83480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.030+0000 7f834f800640 1 -- 172.21.15.2:0/2283659608 >> 172.21.15.2:0/2283659608 conn(0x7f83480fd8b0 msgr2=0x7f83480febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:16.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.030+0000 7f834f800640 1 -- 172.21.15.2:0/2283659608 shutdown_connections 2024-09-10T08:07:16.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.030+0000 7f834f800640 1 -- 172.21.15.2:0/2283659608 wait complete. 2024-09-10T08:07:16.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.158+0000 7f9851a00640 1 -- 172.21.15.2:0/1243571669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f984c074ee0 msgr2=0x7f984c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:16.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.158+0000 7f9851a00640 1 --2- 172.21.15.2:0/1243571669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f984c074ee0 0x7f984c10dd20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9834009860 tx=0x7f983402a3d0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.158+0000 7f9851a00640 1 -- 172.21.15.2:0/1243571669 shutdown_connections 2024-09-10T08:07:16.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.158+0000 7f9851a00640 1 --2- 172.21.15.2:0/1243571669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f984c10e260 0x7f984c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.158+0000 7f9851a00640 1 --2- 172.21.15.2:0/1243571669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f984c074ee0 0x7f984c10dd20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.158+0000 7f9851a00640 1 --2- 172.21.15.2:0/1243571669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c0745c0 0x7f984c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.158+0000 7f9851a00640 1 -- 172.21.15.2:0/1243571669 >> 172.21.15.2:0/1243571669 conn(0x7f984c0fd740 msgr2=0x7f984c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:16.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.158+0000 7f9851a00640 1 -- 172.21.15.2:0/1243571669 shutdown_connections 2024-09-10T08:07:16.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.159+0000 7f9851a00640 1 -- 172.21.15.2:0/1243571669 wait complete. 2024-09-10T08:07:16.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.160+0000 7f9851a00640 1 Processor -- start 2024-09-10T08:07:16.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.160+0000 7f9851a00640 1 -- start start 2024-09-10T08:07:16.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9851a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f984c0745c0 0x7f984c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:16.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9851a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c074ee0 0x7f984c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:16.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9851a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f984c10e260 0x7f984c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:16.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9851a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f984c113fb0 con 0x7f984c10e260 2024-09-10T08:07:16.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9851a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f984c113e30 con 0x7f984c074ee0 2024-09-10T08:07:16.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9851a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f984c114130 con 0x7f984c0745c0 2024-09-10T08:07:16.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9843400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c074ee0 0x7f984c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:16.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f984be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f984c0745c0 0x7f984c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:16.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9843400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c074ee0 0x7f984c19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52746/0 (socket says 172.21.15.2:52746) 2024-09-10T08:07:16.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9843400640 1 -- 172.21.15.2:0/3923264335 learned_addr learned my addr 172.21.15.2:0/3923264335 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:16.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9851000640 1 --2- 172.21.15.2:0/3923264335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f984c10e260 0x7f984c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:16.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9843400640 1 -- 172.21.15.2:0/3923264335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f984c0745c0 msgr2=0x7f984c19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:16.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9843400640 1 --2- 172.21.15.2:0/3923264335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f984c0745c0 0x7f984c19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9843400640 1 -- 172.21.15.2:0/3923264335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f984c10e260 msgr2=0x7f984c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:16.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9843400640 1 --2- 172.21.15.2:0/3923264335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f984c10e260 0x7f984c19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.161+0000 7f9843400640 1 -- 172.21.15.2:0/3923264335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984c19f4e0 con 0x7f984c074ee0 2024-09-10T08:07:16.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.162+0000 7f9843400640 1 --2- 172.21.15.2:0/3923264335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c074ee0 0x7f984c19aa70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f983402a8f0 tx=0x7f9834030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:16.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.162+0000 7f9849600640 1 -- 172.21.15.2:0/3923264335 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834040020 con 0x7f984c074ee0 2024-09-10T08:07:16.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.162+0000 7f9849600640 1 -- 172.21.15.2:0/3923264335 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9834004460 con 0x7f984c074ee0 2024-09-10T08:07:16.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.162+0000 7f9851a00640 1 -- 172.21.15.2:0/3923264335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984c19f770 con 0x7f984c074ee0 2024-09-10T08:07:16.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.162+0000 7f9849600640 1 -- 172.21.15.2:0/3923264335 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98340046c0 con 0x7f984c074ee0 2024-09-10T08:07:16.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.162+0000 7f9851a00640 1 -- 172.21.15.2:0/3923264335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984c1a70b0 con 0x7f984c074ee0 2024-09-10T08:07:16.700 INFO:teuthology.orchestra.run.smithi002.stdout:296352743465 2024-09-10T08:07:16.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.163+0000 7f9851a00640 1 -- 172.21.15.2:0/3923264335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984c075c70 con 0x7f984c074ee0 2024-09-10T08:07:16.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.165+0000 7f9849600640 1 -- 172.21.15.2:0/3923264335 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9834043020 con 0x7f984c074ee0 2024-09-10T08:07:16.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.166+0000 7f9849600640 1 --2- 172.21.15.2:0/3923264335 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f98300a2c10 0x7f98300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:16.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.166+0000 7f9849600640 1 -- 172.21.15.2:0/3923264335 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f98340f0800 con 0x7f984c074ee0 2024-09-10T08:07:16.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.167+0000 7f984be00640 1 --2- 172.21.15.2:0/3923264335 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f98300a2c10 0x7f98300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:16.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.167+0000 7f984be00640 1 --2- 172.21.15.2:0/3923264335 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f98300a2c10 0x7f98300a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f983c005fd0 tx=0x7f983c010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:16.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.168+0000 7f9849600640 1 -- 172.21.15.2:0/3923264335 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f983404a020 con 0x7f984c074ee0 2024-09-10T08:07:16.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.358+0000 7f9851a00640 1 -- 172.21.15.2:0/3923264335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f984c1093e0 con 0x7f984c074ee0 2024-09-10T08:07:16.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.358+0000 7f9849600640 1 -- 172.21.15.2:0/3923264335 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f98340bd1b0 con 0x7f984c074ee0 2024-09-10T08:07:16.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.361+0000 7f9851a00640 1 -- 172.21.15.2:0/3923264335 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f98300a2c10 msgr2=0x7f98300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:16.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.361+0000 7f9851a00640 1 --2- 172.21.15.2:0/3923264335 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f98300a2c10 0x7f98300a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f983c005fd0 tx=0x7f983c010040 comp rx=0 tx=0).stop 2024-09-10T08:07:16.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.361+0000 7f9851a00640 1 -- 172.21.15.2:0/3923264335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c074ee0 msgr2=0x7f984c19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:16.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.361+0000 7f9851a00640 1 --2- 172.21.15.2:0/3923264335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c074ee0 0x7f984c19aa70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f983402a8f0 tx=0x7f9834030650 comp rx=0 tx=0).stop 2024-09-10T08:07:16.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.361+0000 7f9851a00640 1 -- 172.21.15.2:0/3923264335 shutdown_connections 2024-09-10T08:07:16.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.361+0000 7f9851a00640 1 --2- 172.21.15.2:0/3923264335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f984c10e260 0x7f984c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.361+0000 7f9851a00640 1 --2- 172.21.15.2:0/3923264335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c074ee0 0x7f984c19aa70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.361+0000 7f9851a00640 1 --2- 172.21.15.2:0/3923264335 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f98300a2c10 0x7f98300a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.361+0000 7f9851a00640 1 --2- 172.21.15.2:0/3923264335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f984c0745c0 0x7f984c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:16.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.361+0000 7f9851a00640 1 -- 172.21.15.2:0/3923264335 >> 172.21.15.2:0/3923264335 conn(0x7f984c0fd740 msgr2=0x7f984c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:16.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.361+0000 7f9851a00640 1 -- 172.21.15.2:0/3923264335 shutdown_connections 2024-09-10T08:07:16.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:16.361+0000 7f9851a00640 1 -- 172.21.15.2:0/3923264335 wait complete. 2024-09-10T08:07:16.703 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:16 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2283659608' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T08:07:16.703 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:16 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3923264335' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T08:07:16.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:16 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2283659608' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T08:07:16.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:16 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3923264335' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T08:07:16.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:16 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2283659608' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T08:07:16.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:16 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3923264335' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T08:07:17.394 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514529 got 326417514532 for osd.11 2024-09-10T08:07:17.394 DEBUG:teuthology.parallel:result is None 2024-09-10T08:07:17.638 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:17 smithi002 ceph-mon[30563]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-10T08:07:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:17 smithi155 ceph-mon[35405]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-10T08:07:17.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:17 smithi107 ceph-mon[35048]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-10T08:07:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.341+0000 7fe078c00640 1 -- 172.21.15.2:0/2463830746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe074101df0 msgr2=0x7fe0741021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.341+0000 7fe078c00640 1 --2- 172.21.15.2:0/2463830746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe074101df0 0x7fe0741021d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe064009900 tx=0x7fe06402a410 comp rx=0 tx=0).stop 2024-09-10T08:07:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.341+0000 7fe078c00640 1 -- 172.21.15.2:0/2463830746 shutdown_connections 2024-09-10T08:07:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.341+0000 7fe078c00640 1 --2- 172.21.15.2:0/2463830746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe074106f20 0x7fe07410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.341+0000 7fe078c00640 1 --2- 172.21.15.2:0/2463830746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0741027a0 0x7fe0741067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.341+0000 7fe078c00640 1 --2- 172.21.15.2:0/2463830746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe074101df0 0x7fe0741021d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.341+0000 7fe078c00640 1 -- 172.21.15.2:0/2463830746 >> 172.21.15.2:0/2463830746 conn(0x7fe0740fd8b0 msgr2=0x7fe0740ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.341+0000 7fe078c00640 1 -- 172.21.15.2:0/2463830746 shutdown_connections 2024-09-10T08:07:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.341+0000 7fe078c00640 1 -- 172.21.15.2:0/2463830746 wait complete. 2024-09-10T08:07:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.342+0000 7fe078c00640 1 Processor -- start 2024-09-10T08:07:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.342+0000 7fe078c00640 1 -- start start 2024-09-10T08:07:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.342+0000 7fe078c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe074101df0 0x7fe0740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.342+0000 7fe078c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0741027a0 0x7fe074074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe078c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe074106f20 0x7fe074074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe078c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe074111fd0 con 0x7fe0741027a0 2024-09-10T08:07:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe078c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe074111e50 con 0x7fe074106f20 2024-09-10T08:07:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe078c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe074112150 con 0x7fe074101df0 2024-09-10T08:07:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe071e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe074101df0 0x7fe0740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe071e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe074101df0 0x7fe0740744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:40466/0 (socket says 172.21.15.2:40466) 2024-09-10T08:07:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe071e00640 1 -- 172.21.15.2:0/759566105 learned_addr learned my addr 172.21.15.2:0/759566105 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe06be00640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0741027a0 0x7fe074074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe072800640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe074106f20 0x7fe074074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe071e00640 1 -- 172.21.15.2:0/759566105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe074106f20 msgr2=0x7fe074074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe071e00640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe074106f20 0x7fe074074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe071e00640 1 -- 172.21.15.2:0/759566105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0741027a0 msgr2=0x7fe074074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe071e00640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0741027a0 0x7fe074074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.343+0000 7fe071e00640 1 -- 172.21.15.2:0/759566105 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0741a68b0 con 0x7fe074101df0 2024-09-10T08:07:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.344+0000 7fe071e00640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe074101df0 0x7fe0740744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe064004770 tx=0x7fe064003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.344+0000 7fe06aa00640 1 -- 172.21.15.2:0/759566105 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06403f070 con 0x7fe074101df0 2024-09-10T08:07:18.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.344+0000 7fe06aa00640 1 -- 172.21.15.2:0/759566105 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe064031ca0 con 0x7fe074101df0 2024-09-10T08:07:18.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.344+0000 7fe06aa00640 1 -- 172.21.15.2:0/759566105 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064035030 con 0x7fe074101df0 2024-09-10T08:07:18.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.344+0000 7fe078c00640 1 -- 172.21.15.2:0/759566105 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0741a6b40 con 0x7fe074101df0 2024-09-10T08:07:18.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.344+0000 7fe078c00640 1 -- 172.21.15.2:0/759566105 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0741a7070 con 0x7fe074101df0 2024-09-10T08:07:18.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.345+0000 7fe078c00640 1 -- 172.21.15.2:0/759566105 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0741021d0 con 0x7fe074101df0 2024-09-10T08:07:18.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.348+0000 7fe06aa00640 1 -- 172.21.15.2:0/759566105 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe064042070 con 0x7fe074101df0 2024-09-10T08:07:18.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.350+0000 7fe06aa00640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe0580a2c90 0x7fe0580a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:18.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.350+0000 7fe06aa00640 1 -- 172.21.15.2:0/759566105 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe0640f0180 con 0x7fe074101df0 2024-09-10T08:07:18.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.350+0000 7fe06be00640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe0580a2c90 0x7fe0580a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:18.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.351+0000 7fe06be00640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe0580a2c90 0x7fe0580a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe074075f80 tx=0x7fe05c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:18.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.352+0000 7fe06aa00640 1 -- 172.21.15.2:0/759566105 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe064049030 con 0x7fe074101df0 2024-09-10T08:07:18.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.558+0000 7fe078c00640 1 -- 172.21.15.2:0/759566105 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe07410a100 con 0x7fe074101df0 2024-09-10T08:07:18.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.559+0000 7fe06aa00640 1 -- 172.21.15.2:0/759566105 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe0640bcb30 con 0x7fe074101df0 2024-09-10T08:07:18.557 INFO:teuthology.orchestra.run.smithi002.stdout:193273528382 2024-09-10T08:07:18.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.561+0000 7fe078c00640 1 -- 172.21.15.2:0/759566105 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe0580a2c90 msgr2=0x7fe0580a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:18.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.561+0000 7fe078c00640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe0580a2c90 0x7fe0580a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe074075f80 tx=0x7fe05c005f90 comp rx=0 tx=0).stop 2024-09-10T08:07:18.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.561+0000 7fe078c00640 1 -- 172.21.15.2:0/759566105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe074101df0 msgr2=0x7fe0740744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:18.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.561+0000 7fe078c00640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe074101df0 0x7fe0740744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe064004770 tx=0x7fe064003c10 comp rx=0 tx=0).stop 2024-09-10T08:07:18.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.561+0000 7fe078c00640 1 -- 172.21.15.2:0/759566105 shutdown_connections 2024-09-10T08:07:18.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.561+0000 7fe078c00640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe074106f20 0x7fe074074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.561+0000 7fe078c00640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0741027a0 0x7fe074074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.561+0000 7fe078c00640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe0580a2c90 0x7fe0580a5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.561+0000 7fe078c00640 1 --2- 172.21.15.2:0/759566105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe074101df0 0x7fe0740744f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.561+0000 7fe078c00640 1 -- 172.21.15.2:0/759566105 >> 172.21.15.2:0/759566105 conn(0x7fe0740fd8b0 msgr2=0x7fe0740febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:18.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.561+0000 7fe078c00640 1 -- 172.21.15.2:0/759566105 shutdown_connections 2024-09-10T08:07:18.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.561+0000 7fe078c00640 1 -- 172.21.15.2:0/759566105 wait complete. 2024-09-10T08:07:18.902 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:18 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3837532240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T08:07:18.902 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:18 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1786283484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T08:07:18.902 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:18 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/759566105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T08:07:18.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:18 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3837532240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T08:07:18.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:18 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1786283484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T08:07:18.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:18 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/759566105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T08:07:18.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:18 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3837532240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T08:07:18.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:18 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1786283484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T08:07:18.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:18 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/759566105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T08:07:18.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.703+0000 7fb7f8400640 1 -- 172.21.15.2:0/2980369248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7f0109140 msgr2=0x7fb7f0110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.703+0000 7fb7f8400640 1 --2- 172.21.15.2:0/2980369248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7f0109140 0x7fb7f0110cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009ca0 tx=0x7fb7e802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.703+0000 7fb7f8400640 1 -- 172.21.15.2:0/2980369248 shutdown_connections 2024-09-10T08:07:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.703+0000 7fb7f8400640 1 --2- 172.21.15.2:0/2980369248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7f0109140 0x7fb7f0110cc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.703+0000 7fb7f8400640 1 --2- 172.21.15.2:0/2980369248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f01049c0 0x7fb7f0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.703+0000 7fb7f8400640 1 --2- 172.21.15.2:0/2980369248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f0104010 0x7fb7f01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.703+0000 7fb7f8400640 1 -- 172.21.15.2:0/2980369248 >> 172.21.15.2:0/2980369248 conn(0x7fb7f00ffa50 msgr2=0x7fb7f0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.704+0000 7fb7f8400640 1 -- 172.21.15.2:0/2980369248 shutdown_connections 2024-09-10T08:07:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.704+0000 7fb7f8400640 1 -- 172.21.15.2:0/2980369248 wait complete. 2024-09-10T08:07:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.705+0000 7fb7f8400640 1 Processor -- start 2024-09-10T08:07:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.705+0000 7fb7f8400640 1 -- start start 2024-09-10T08:07:18.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.705+0000 7fb7f8400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7f0104010 0x7fb7f019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:18.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.705+0000 7fb7f8400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f01049c0 0x7fb7f019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:18.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.705+0000 7fb7f8400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f0109140 0x7fb7f019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:18.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.705+0000 7fb7f8400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f0114170 con 0x7fb7f0104010 2024-09-10T08:07:18.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.705+0000 7fb7f8400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f0113ff0 con 0x7fb7f0109140 2024-09-10T08:07:18.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.705+0000 7fb7f8400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f01142f0 con 0x7fb7f01049c0 2024-09-10T08:07:18.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.705+0000 7fb7f5a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7f0104010 0x7fb7f019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:18.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.705+0000 7fb7f6400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f0109140 0x7fb7f019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:18.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.705+0000 7fb7f6400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f0109140 0x7fb7f019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58362/0 (socket says 172.21.15.2:58362) 2024-09-10T08:07:18.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.705+0000 7fb7f6400640 1 -- 172.21.15.2:0/3448797834 learned_addr learned my addr 172.21.15.2:0/3448797834 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:18.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.705+0000 7fb7f5a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7f0104010 0x7fb7f019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44742/0 (socket says 172.21.15.2:44742) 2024-09-10T08:07:18.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.706+0000 7fb7f5000640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f01049c0 0x7fb7f019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:18.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.706+0000 7fb7f5a00640 1 -- 172.21.15.2:0/3448797834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f01049c0 msgr2=0x7fb7f019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:18.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.706+0000 7fb7f5a00640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f01049c0 0x7fb7f019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.706+0000 7fb7f5a00640 1 -- 172.21.15.2:0/3448797834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f0109140 msgr2=0x7fb7f019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:18.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.706+0000 7fb7f5a00640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f0109140 0x7fb7f019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.706+0000 7fb7f5a00640 1 -- 172.21.15.2:0/3448797834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f019f600 con 0x7fb7f0104010 2024-09-10T08:07:18.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.706+0000 7fb7f5000640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f01049c0 0x7fb7f019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:18.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.706+0000 7fb7f6400640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f0109140 0x7fb7f019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:18.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.706+0000 7fb7f5a00640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7f0104010 0x7fb7f019a650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400e9f0 tx=0x7fb7e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:18.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.707+0000 7fb7ee000640 1 -- 172.21.15.2:0/3448797834 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e400ccc0 con 0x7fb7f0104010 2024-09-10T08:07:18.975 INFO:teuthology.orchestra.run.smithi002.stdout:55834574939 2024-09-10T08:07:18.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.707+0000 7fb7f8400640 1 -- 172.21.15.2:0/3448797834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f019f8f0 con 0x7fb7f0104010 2024-09-10T08:07:18.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.707+0000 7fb7ee000640 1 -- 172.21.15.2:0/3448797834 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7e4005020 con 0x7fb7f0104010 2024-09-10T08:07:18.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.707+0000 7fb7ee000640 1 -- 172.21.15.2:0/3448797834 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40049e0 con 0x7fb7f0104010 2024-09-10T08:07:18.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.707+0000 7fb7f8400640 1 -- 172.21.15.2:0/3448797834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f01a71d0 con 0x7fb7f0104010 2024-09-10T08:07:18.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.708+0000 7fb7f8400640 1 -- 172.21.15.2:0/3448797834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f01043f0 con 0x7fb7f0104010 2024-09-10T08:07:18.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.710+0000 7fb7ee000640 1 -- 172.21.15.2:0/3448797834 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb7e4005560 con 0x7fb7f0104010 2024-09-10T08:07:18.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.712+0000 7fb7ee000640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7d80a2c90 0x7fb7d80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:18.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.712+0000 7fb7ee000640 1 -- 172.21.15.2:0/3448797834 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb7e40cf490 con 0x7fb7f0104010 2024-09-10T08:07:18.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.712+0000 7fb7f5000640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7d80a2c90 0x7fb7d80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:18.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.713+0000 7fb7f5000640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7d80a2c90 0x7fb7d80a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb7f010ab10 tx=0x7fb7dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:18.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.716+0000 7fb7ee000640 1 -- 172.21.15.2:0/3448797834 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7e40051d0 con 0x7fb7f0104010 2024-09-10T08:07:18.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.903+0000 7fb7f8400640 1 -- 172.21.15.2:0/3448797834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb7f019b9b0 con 0x7fb7f0104010 2024-09-10T08:07:18.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.903+0000 7fb7ee000640 1 -- 172.21.15.2:0/3448797834 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb7e409bea0 con 0x7fb7f0104010 2024-09-10T08:07:18.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.905+0000 7fb7f8400640 1 -- 172.21.15.2:0/3448797834 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7d80a2c90 msgr2=0x7fb7d80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:18.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.905+0000 7fb7f8400640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7d80a2c90 0x7fb7d80a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb7f010ab10 tx=0x7fb7dc005f90 comp rx=0 tx=0).stop 2024-09-10T08:07:18.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.905+0000 7fb7f8400640 1 -- 172.21.15.2:0/3448797834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7f0104010 msgr2=0x7fb7f019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:18.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.905+0000 7fb7f8400640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7f0104010 0x7fb7f019a650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400e9f0 tx=0x7fb7e400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.905+0000 7fb7f8400640 1 -- 172.21.15.2:0/3448797834 shutdown_connections 2024-09-10T08:07:18.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.905+0000 7fb7f8400640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f0109140 0x7fb7f019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.905+0000 7fb7f8400640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7d80a2c90 0x7fb7d80a5150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.905+0000 7fb7f8400640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7f01049c0 0x7fb7f019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.906+0000 7fb7f8400640 1 --2- 172.21.15.2:0/3448797834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7f0104010 0x7fb7f019a650 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:18.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.906+0000 7fb7f8400640 1 -- 172.21.15.2:0/3448797834 >> 172.21.15.2:0/3448797834 conn(0x7fb7f00ffa50 msgr2=0x7fb7f0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:18.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.906+0000 7fb7f8400640 1 -- 172.21.15.2:0/3448797834 shutdown_connections 2024-09-10T08:07:18.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.906+0000 7fb7f8400640 1 -- 172.21.15.2:0/3448797834 wait complete. 2024-09-10T08:07:19.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.757+0000 7f6cdf200640 1 -- 172.21.15.2:0/1107790035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce00745c0 msgr2=0x7f6ce00749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.757+0000 7f6cdf200640 1 --2- 172.21.15.2:0/1107790035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce00745c0 0x7f6ce00749a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009900 tx=0x7f6cd002a480 comp rx=0 tx=0).stop 2024-09-10T08:07:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.757+0000 7f6cdf200640 1 -- 172.21.15.2:0/1107790035 shutdown_connections 2024-09-10T08:07:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.757+0000 7f6cdf200640 1 --2- 172.21.15.2:0/1107790035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ce010e260 0x7f6ce0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.757+0000 7f6cdf200640 1 --2- 172.21.15.2:0/1107790035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ce0074ee0 0x7f6ce010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.757+0000 7f6cdf200640 1 --2- 172.21.15.2:0/1107790035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce00745c0 0x7f6ce00749a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.757+0000 7f6cdf200640 1 -- 172.21.15.2:0/1107790035 >> 172.21.15.2:0/1107790035 conn(0x7f6ce00fd740 msgr2=0x7f6ce00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.757+0000 7f6cdf200640 1 -- 172.21.15.2:0/1107790035 shutdown_connections 2024-09-10T08:07:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.757+0000 7f6cdf200640 1 -- 172.21.15.2:0/1107790035 wait complete. 2024-09-10T08:07:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.758+0000 7f6cdf200640 1 Processor -- start 2024-09-10T08:07:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.758+0000 7f6cdf200640 1 -- start start 2024-09-10T08:07:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.759+0000 7f6cdf200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ce00745c0 0x7f6ce019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.759+0000 7f6cdf200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ce0074ee0 0x7f6ce019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.759+0000 7f6cdf200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce010e260 0x7f6ce019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.759+0000 7f6cdf200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce0113fb0 con 0x7f6ce010e260 2024-09-10T08:07:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.759+0000 7f6cdf200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce0113e30 con 0x7f6ce0074ee0 2024-09-10T08:07:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.759+0000 7f6cdf200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce0114130 con 0x7f6ce00745c0 2024-09-10T08:07:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.759+0000 7f6cde800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce010e260 0x7f6ce019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.759+0000 7f6cdde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ce00745c0 0x7f6ce019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.759+0000 7f6cde800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce010e260 0x7f6ce019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44670/0 (socket says 172.21.15.2:44670) 2024-09-10T08:07:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.759+0000 7f6cde800640 1 -- 172.21.15.2:0/3837532240 learned_addr learned my addr 172.21.15.2:0/3837532240 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.759+0000 7f6cd7e00640 1 --2- 172.21.15.2:0/3837532240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ce0074ee0 0x7f6ce019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.760+0000 7f6cde800640 1 -- 172.21.15.2:0/3837532240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ce00745c0 msgr2=0x7f6ce019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.760+0000 7f6cde800640 1 --2- 172.21.15.2:0/3837532240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ce00745c0 0x7f6ce019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.760+0000 7f6cde800640 1 -- 172.21.15.2:0/3837532240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ce0074ee0 msgr2=0x7f6ce019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.760+0000 7f6cde800640 1 --2- 172.21.15.2:0/3837532240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ce0074ee0 0x7f6ce019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.760+0000 7f6cde800640 1 -- 172.21.15.2:0/3837532240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce019f440 con 0x7f6ce010e260 2024-09-10T08:07:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.760+0000 7f6cde800640 1 --2- 172.21.15.2:0/3837532240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce010e260 0x7f6ce019ed60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800eb40 tx=0x7f6cd800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.760+0000 7f6cd6a00640 1 -- 172.21.15.2:0/3837532240 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8004b20 con 0x7f6ce010e260 2024-09-10T08:07:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.760+0000 7f6cd6a00640 1 -- 172.21.15.2:0/3837532240 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd8004cc0 con 0x7f6ce010e260 2024-09-10T08:07:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.760+0000 7f6cd6a00640 1 -- 172.21.15.2:0/3837532240 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8010850 con 0x7f6ce010e260 2024-09-10T08:07:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.760+0000 7f6cdf200640 1 -- 172.21.15.2:0/3837532240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce019f730 con 0x7f6ce010e260 2024-09-10T08:07:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.760+0000 7f6cdf200640 1 -- 172.21.15.2:0/3837532240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce01a7010 con 0x7f6ce010e260 2024-09-10T08:07:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.761+0000 7f6cdf200640 1 -- 172.21.15.2:0/3837532240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce00749a0 con 0x7f6ce010e260 2024-09-10T08:07:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.763+0000 7f6cd6a00640 1 -- 172.21.15.2:0/3837532240 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6cd8005020 con 0x7f6ce010e260 2024-09-10T08:07:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.764+0000 7f6cd6a00640 1 --2- 172.21.15.2:0/3837532240 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6cc40a2c90 0x7f6cc40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.765+0000 7f6cd6a00640 1 -- 172.21.15.2:0/3837532240 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6cd80cfcb0 con 0x7f6ce010e260 2024-09-10T08:07:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.765+0000 7f6cdde00640 1 --2- 172.21.15.2:0/3837532240 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6cc40a2c90 0x7f6cc40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.765+0000 7f6cdde00640 1 --2- 172.21.15.2:0/3837532240 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6cc40a2c90 0x7f6cc40a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009230 tx=0x7f6cd00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.769+0000 7f6cd6a00640 1 -- 172.21.15.2:0/3837532240 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6cd809c660 con 0x7f6ce010e260 2024-09-10T08:07:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.963+0000 7f6cdf200640 1 -- 172.21.15.2:0/3837532240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6ce002d7c0 con 0x7f6ce010e260 2024-09-10T08:07:19.312 INFO:teuthology.orchestra.run.smithi002.stdout:107374182478 2024-09-10T08:07:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.963+0000 7f6cd6a00640 1 -- 172.21.15.2:0/3837532240 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6cd809bf60 con 0x7f6ce010e260 2024-09-10T08:07:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.966+0000 7f6cdf200640 1 -- 172.21.15.2:0/3837532240 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6cc40a2c90 msgr2=0x7f6cc40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.966+0000 7f6cdf200640 1 --2- 172.21.15.2:0/3837532240 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6cc40a2c90 0x7f6cc40a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009230 tx=0x7f6cd00058d0 comp rx=0 tx=0).stop 2024-09-10T08:07:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.966+0000 7f6cdf200640 1 -- 172.21.15.2:0/3837532240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce010e260 msgr2=0x7f6ce019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:19.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.966+0000 7f6cdf200640 1 --2- 172.21.15.2:0/3837532240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce010e260 0x7f6ce019ed60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800eb40 tx=0x7f6cd800c2d0 comp rx=0 tx=0).stop 2024-09-10T08:07:19.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.966+0000 7f6cdf200640 1 -- 172.21.15.2:0/3837532240 shutdown_connections 2024-09-10T08:07:19.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.966+0000 7f6cdf200640 1 --2- 172.21.15.2:0/3837532240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce010e260 0x7f6ce019ed60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:19.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.966+0000 7f6cdf200640 1 --2- 172.21.15.2:0/3837532240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ce0074ee0 0x7f6ce019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:19.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.966+0000 7f6cdf200640 1 --2- 172.21.15.2:0/3837532240 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6cc40a2c90 0x7f6cc40a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:19.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.966+0000 7f6cdf200640 1 --2- 172.21.15.2:0/3837532240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ce00745c0 0x7f6ce019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:19.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.966+0000 7f6cdf200640 1 -- 172.21.15.2:0/3837532240 >> 172.21.15.2:0/3837532240 conn(0x7f6ce00fd740 msgr2=0x7f6ce00fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:19.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.966+0000 7f6cdf200640 1 -- 172.21.15.2:0/3837532240 shutdown_connections 2024-09-10T08:07:19.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:17.967+0000 7f6cdf200640 1 -- 172.21.15.2:0/3837532240 wait complete. 2024-09-10T08:07:19.816 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743463 got 296352743465 for osd.10 2024-09-10T08:07:19.816 DEBUG:teuthology.parallel:result is None 2024-09-10T08:07:19.899 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:19 smithi002 ceph-mon[30563]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-10T08:07:19.899 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:19 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3448797834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T08:07:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:19 smithi155 ceph-mon[35405]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-10T08:07:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:19 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3448797834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T08:07:19.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:19 smithi107 ceph-mon[35048]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-10T08:07:19.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:19 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3448797834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T08:07:20.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.181+0000 7fb1c7c00640 1 -- 172.21.15.2:0/3083227603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1c0103ff0 msgr2=0x7fb1c01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:20.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.181+0000 7fb1c7c00640 1 --2- 172.21.15.2:0/3083227603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1c0103ff0 0x7fb1c01043d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009900 tx=0x7fb1b402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.181+0000 7fb1c7c00640 1 -- 172.21.15.2:0/3083227603 shutdown_connections 2024-09-10T08:07:20.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.181+0000 7fb1c7c00640 1 --2- 172.21.15.2:0/3083227603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1c0109120 0x7fb1c0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.181+0000 7fb1c7c00640 1 --2- 172.21.15.2:0/3083227603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c01049a0 0x7fb1c01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.181+0000 7fb1c7c00640 1 --2- 172.21.15.2:0/3083227603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1c0103ff0 0x7fb1c01043d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.181+0000 7fb1c7c00640 1 -- 172.21.15.2:0/3083227603 >> 172.21.15.2:0/3083227603 conn(0x7fb1c00ffa90 msgr2=0x7fb1c0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:20.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.181+0000 7fb1c7c00640 1 -- 172.21.15.2:0/3083227603 shutdown_connections 2024-09-10T08:07:20.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.181+0000 7fb1c7c00640 1 -- 172.21.15.2:0/3083227603 wait complete. 2024-09-10T08:07:20.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.183+0000 7fb1c7c00640 1 Processor -- start 2024-09-10T08:07:20.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.183+0000 7fb1c7c00640 1 -- start start 2024-09-10T08:07:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.183+0000 7fb1c7c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c0103ff0 0x7fb1c019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1c7c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1c01049a0 0x7fb1c019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1c7c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1c0109120 0x7fb1c019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1c7c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0114190 con 0x7fb1c01049a0 2024-09-10T08:07:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1c7c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0114010 con 0x7fb1c0109120 2024-09-10T08:07:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1c7c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0114310 con 0x7fb1c0103ff0 2024-09-10T08:07:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1bfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1c01049a0 0x7fb1c019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1c5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c0103ff0 0x7fb1c019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1c5c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1c0109120 0x7fb1c019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1bfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1c01049a0 0x7fb1c019abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44688/0 (socket says 172.21.15.2:44688) 2024-09-10T08:07:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1c5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c0103ff0 0x7fb1c019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:40446/0 (socket says 172.21.15.2:40446) 2024-09-10T08:07:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1bfe00640 1 -- 172.21.15.2:0/1786283484 learned_addr learned my addr 172.21.15.2:0/1786283484 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1bfe00640 1 -- 172.21.15.2:0/1786283484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c0103ff0 msgr2=0x7fb1c019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1bfe00640 1 --2- 172.21.15.2:0/1786283484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c0103ff0 0x7fb1c019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1bfe00640 1 -- 172.21.15.2:0/1786283484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1c0109120 msgr2=0x7fb1c019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1bfe00640 1 --2- 172.21.15.2:0/1786283484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1c0109120 0x7fb1c019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.184+0000 7fb1bfe00640 1 -- 172.21.15.2:0/1786283484 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c019f640 con 0x7fb1c01049a0 2024-09-10T08:07:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.185+0000 7fb1bfe00640 1 --2- 172.21.15.2:0/1786283484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1c01049a0 0x7fb1c019abd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00e9f0 tx=0x7fb1ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.185+0000 7fb1bd600640 1 -- 172.21.15.2:0/1786283484 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00cd80 con 0x7fb1c01049a0 2024-09-10T08:07:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.185+0000 7fb1bd600640 1 -- 172.21.15.2:0/1786283484 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1ac005500 con 0x7fb1c01049a0 2024-09-10T08:07:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.185+0000 7fb1c7c00640 1 -- 172.21.15.2:0/1786283484 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c019f930 con 0x7fb1c01049a0 2024-09-10T08:07:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.185+0000 7fb1bd600640 1 -- 172.21.15.2:0/1786283484 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac010690 con 0x7fb1c01049a0 2024-09-10T08:07:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.185+0000 7fb1c7c00640 1 -- 172.21.15.2:0/1786283484 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c01a7210 con 0x7fb1c01049a0 2024-09-10T08:07:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.186+0000 7fb1c7c00640 1 -- 172.21.15.2:0/1786283484 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c01043d0 con 0x7fb1c01049a0 2024-09-10T08:07:20.016 INFO:teuthology.orchestra.run.smithi002.stdout:274877906989 2024-09-10T08:07:20.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.189+0000 7fb1bd600640 1 -- 172.21.15.2:0/1786283484 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb1ac0046c0 con 0x7fb1c01049a0 2024-09-10T08:07:20.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.190+0000 7fb1bd600640 1 --2- 172.21.15.2:0/1786283484 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb1a80a2c10 0x7fb1a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:20.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.190+0000 7fb1bd600640 1 -- 172.21.15.2:0/1786283484 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb1ac0cf9a0 con 0x7fb1c01049a0 2024-09-10T08:07:20.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.190+0000 7fb1c5200640 1 --2- 172.21.15.2:0/1786283484 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb1a80a2c10 0x7fb1a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.191+0000 7fb1c5200640 1 --2- 172.21.15.2:0/1786283484 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb1a80a2c10 0x7fb1a80a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4000c00 tx=0x7fb1b4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.194+0000 7fb1bd600640 1 -- 172.21.15.2:0/1786283484 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1ac09c3b0 con 0x7fb1c01049a0 2024-09-10T08:07:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.387+0000 7fb1c7c00640 1 -- 172.21.15.2:0/1786283484 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fb1c010bd70 con 0x7fb1c01049a0 2024-09-10T08:07:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.387+0000 7fb1bd600640 1 -- 172.21.15.2:0/1786283484 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb1ac09bcb0 con 0x7fb1c01049a0 2024-09-10T08:07:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.390+0000 7fb1c7c00640 1 -- 172.21.15.2:0/1786283484 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb1a80a2c10 msgr2=0x7fb1a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.390+0000 7fb1c7c00640 1 --2- 172.21.15.2:0/1786283484 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb1a80a2c10 0x7fb1a80a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4000c00 tx=0x7fb1b4005b50 comp rx=0 tx=0).stop 2024-09-10T08:07:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.390+0000 7fb1c7c00640 1 -- 172.21.15.2:0/1786283484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1c01049a0 msgr2=0x7fb1c019abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.390+0000 7fb1c7c00640 1 --2- 172.21.15.2:0/1786283484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1c01049a0 0x7fb1c019abd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00e9f0 tx=0x7fb1ac00edf0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.390+0000 7fb1c7c00640 1 -- 172.21.15.2:0/1786283484 shutdown_connections 2024-09-10T08:07:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.390+0000 7fb1c7c00640 1 --2- 172.21.15.2:0/1786283484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1c0109120 0x7fb1c019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.390+0000 7fb1c7c00640 1 --2- 172.21.15.2:0/1786283484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1c01049a0 0x7fb1c019abd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.390+0000 7fb1c7c00640 1 --2- 172.21.15.2:0/1786283484 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb1a80a2c10 0x7fb1a80a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.390+0000 7fb1c7c00640 1 --2- 172.21.15.2:0/1786283484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c0103ff0 0x7fb1c019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.390+0000 7fb1c7c00640 1 -- 172.21.15.2:0/1786283484 >> 172.21.15.2:0/1786283484 conn(0x7fb1c00ffa90 msgr2=0x7fb1c0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:20.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.390+0000 7fb1c7c00640 1 -- 172.21.15.2:0/1786283484 shutdown_connections 2024-09-10T08:07:20.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:18.390+0000 7fb1c7c00640 1 -- 172.21.15.2:0/1786283484 wait complete. 2024-09-10T08:07:20.741 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:20 smithi002 ceph-mon[30563]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:20.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:20 smithi155 ceph-mon[35405]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:20.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:20 smithi107 ceph-mon[35048]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:20.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.986+0000 7ff7c4a00640 1 -- 172.21.15.2:0/585004326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7c0102360 msgr2=0x7ff7c010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:20.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.986+0000 7ff7c4a00640 1 --2- 172.21.15.2:0/585004326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7c0102360 0x7ff7c010e890 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009cd0 tx=0x7ff7b802a8f0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.986+0000 7ff7c4a00640 1 -- 172.21.15.2:0/585004326 shutdown_connections 2024-09-10T08:07:20.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.986+0000 7ff7c4a00640 1 --2- 172.21.15.2:0/585004326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7c0102360 0x7ff7c010e890 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.986+0000 7ff7c4a00640 1 --2- 172.21.15.2:0/585004326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c01019c0 0x7ff7c0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.986+0000 7ff7c4a00640 1 --2- 172.21.15.2:0/585004326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c01079c0 0x7ff7c0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.986+0000 7ff7c4a00640 1 -- 172.21.15.2:0/585004326 >> 172.21.15.2:0/585004326 conn(0x7ff7c00fd740 msgr2=0x7ff7c00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:20.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.987+0000 7ff7c4a00640 1 -- 172.21.15.2:0/585004326 shutdown_connections 2024-09-10T08:07:20.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.987+0000 7ff7c4a00640 1 -- 172.21.15.2:0/585004326 wait complete. 2024-09-10T08:07:20.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.988+0000 7ff7c4a00640 1 Processor -- start 2024-09-10T08:07:20.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.988+0000 7ff7c4a00640 1 -- start start 2024-09-10T08:07:20.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.988+0000 7ff7c4a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c01019c0 0x7ff7c0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:20.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.988+0000 7ff7c4a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c0102360 0x7ff7c01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:20.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.988+0000 7ff7c4a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7c01079c0 0x7ff7c019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:20.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.988+0000 7ff7c4a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0111e10 con 0x7ff7c01079c0 2024-09-10T08:07:20.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.988+0000 7ff7c4a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0111c90 con 0x7ff7c01019c0 2024-09-10T08:07:20.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.988+0000 7ff7c4a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0111f90 con 0x7ff7c0102360 2024-09-10T08:07:20.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.989+0000 7ff7bf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c01019c0 0x7ff7c0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:20.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.989+0000 7ff7bf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c01019c0 0x7ff7c0198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58382/0 (socket says 172.21.15.2:58382) 2024-09-10T08:07:20.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.989+0000 7ff7bf400640 1 -- 172.21.15.2:0/3600610752 learned_addr learned my addr 172.21.15.2:0/3600610752 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:20.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.989+0000 7ff7bea00640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c0102360 0x7ff7c01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:20.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.989+0000 7ff7bfe00640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7c01079c0 0x7ff7c019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:20.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.989+0000 7ff7bf400640 1 -- 172.21.15.2:0/3600610752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c0102360 msgr2=0x7ff7c01987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:20.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.989+0000 7ff7bf400640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c0102360 0x7ff7c01987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.989+0000 7ff7bf400640 1 -- 172.21.15.2:0/3600610752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7c01079c0 msgr2=0x7ff7c019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:20.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.989+0000 7ff7bf400640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7c01079c0 0x7ff7c019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:20.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.989+0000 7ff7bf400640 1 -- 172.21.15.2:0/3600610752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c019d240 con 0x7ff7c01019c0 2024-09-10T08:07:20.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.989+0000 7ff7bea00640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c0102360 0x7ff7c01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:07:20.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.990+0000 7ff7bf400640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c01019c0 0x7ff7c0198290 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000e9f0 tx=0x7ff7b000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:20.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.990+0000 7ff7b7e00640 1 -- 172.21.15.2:0/3600610752 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000cd80 con 0x7ff7c01019c0 2024-09-10T08:07:20.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.990+0000 7ff7b7e00640 1 -- 172.21.15.2:0/3600610752 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b0005500 con 0x7ff7c01019c0 2024-09-10T08:07:20.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.990+0000 7ff7c4a00640 1 -- 172.21.15.2:0/3600610752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c019d530 con 0x7ff7c01019c0 2024-09-10T08:07:20.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.990+0000 7ff7b7e00640 1 -- 172.21.15.2:0/3600610752 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0010690 con 0x7ff7c01019c0 2024-09-10T08:07:20.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.990+0000 7ff7c4a00640 1 -- 172.21.15.2:0/3600610752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c01a4e10 con 0x7ff7c01019c0 2024-09-10T08:07:20.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.991+0000 7ff7c4a00640 1 -- 172.21.15.2:0/3600610752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c0101e20 con 0x7ff7c01019c0 2024-09-10T08:07:20.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.993+0000 7ff7b7e00640 1 -- 172.21.15.2:0/3600610752 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff7b00046c0 con 0x7ff7c01019c0 2024-09-10T08:07:20.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.994+0000 7ff7b7e00640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff7a40a2c10 0x7ff7a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:20.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.995+0000 7ff7b7e00640 1 -- 172.21.15.2:0/3600610752 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff7b00cfa70 con 0x7ff7c01019c0 2024-09-10T08:07:20.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.995+0000 7ff7bea00640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff7a40a2c10 0x7ff7a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:20.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.996+0000 7ff7bea00640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff7a40a2c10 0x7ff7a40a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00ffcd0 tx=0x7ff7ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:20.999+0000 7ff7b7e00640 1 -- 172.21.15.2:0/3600610752 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7b009c420 con 0x7ff7c01019c0 2024-09-10T08:07:21.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.188+0000 7ff7c4a00640 1 -- 172.21.15.2:0/3600610752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff7c01995f0 con 0x7ff7c01019c0 2024-09-10T08:07:21.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.189+0000 7ff7b7e00640 1 -- 172.21.15.2:0/3600610752 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff7b009bd20 con 0x7ff7c01019c0 2024-09-10T08:07:21.189 INFO:teuthology.orchestra.run.smithi002.stdout:77309411414 2024-09-10T08:07:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.191+0000 7ff7c4a00640 1 -- 172.21.15.2:0/3600610752 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff7a40a2c10 msgr2=0x7ff7a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.191+0000 7ff7c4a00640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff7a40a2c10 0x7ff7a40a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00ffcd0 tx=0x7ff7ac005f90 comp rx=0 tx=0).stop 2024-09-10T08:07:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.191+0000 7ff7c4a00640 1 -- 172.21.15.2:0/3600610752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c01019c0 msgr2=0x7ff7c0198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.191+0000 7ff7c4a00640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c01019c0 0x7ff7c0198290 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000e9f0 tx=0x7ff7b000edf0 comp rx=0 tx=0).stop 2024-09-10T08:07:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.191+0000 7ff7c4a00640 1 -- 172.21.15.2:0/3600610752 shutdown_connections 2024-09-10T08:07:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.191+0000 7ff7c4a00640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7c01079c0 0x7ff7c019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.191+0000 7ff7c4a00640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff7a40a2c10 0x7ff7a40a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.191+0000 7ff7c4a00640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c0102360 0x7ff7c01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.191+0000 7ff7c4a00640 1 --2- 172.21.15.2:0/3600610752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c01019c0 0x7ff7c0198290 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.191+0000 7ff7c4a00640 1 -- 172.21.15.2:0/3600610752 >> 172.21.15.2:0/3600610752 conn(0x7ff7c00fd740 msgr2=0x7ff7c00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.191+0000 7ff7c4a00640 1 -- 172.21.15.2:0/3600610752 shutdown_connections 2024-09-10T08:07:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.191+0000 7ff7c4a00640 1 -- 172.21.15.2:0/3600610752 wait complete. 2024-09-10T08:07:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:21 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3600610752' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T08:07:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:21 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3233708911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T08:07:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:21 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3600610752' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T08:07:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:21 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3233708911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T08:07:21.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:21 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3600610752' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T08:07:21.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:21 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3233708911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T08:07:21.998 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906986 got 274877906989 for osd.9 2024-09-10T08:07:21.998 DEBUG:teuthology.parallel:result is None 2024-09-10T08:07:22.327 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135921 got 244813135924 for osd.8 2024-09-10T08:07:22.327 DEBUG:teuthology.parallel:result is None 2024-09-10T08:07:22.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:22 smithi107 ceph-mon[35048]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:22.968 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:22 smithi002 ceph-mon[30563]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.250+0000 7fe5fb000640 1 -- 172.21.15.2:0/2426879989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f4101dd0 msgr2=0x7fe5f41021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.250+0000 7fe5fb000640 1 --2- 172.21.15.2:0/2426879989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f4101dd0 0x7fe5f41021b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4009860 tx=0x7fe5e402a340 comp rx=0 tx=0).stop 2024-09-10T08:07:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.250+0000 7fe5fb000640 1 -- 172.21.15.2:0/2426879989 shutdown_connections 2024-09-10T08:07:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.250+0000 7fe5fb000640 1 --2- 172.21.15.2:0/2426879989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5f4106f00 0x7fe5f410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.250+0000 7fe5fb000640 1 --2- 172.21.15.2:0/2426879989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f4102780 0x7fe5f41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.250+0000 7fe5fb000640 1 --2- 172.21.15.2:0/2426879989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f4101dd0 0x7fe5f41021b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.250+0000 7fe5fb000640 1 -- 172.21.15.2:0/2426879989 >> 172.21.15.2:0/2426879989 conn(0x7fe5f40fd890 msgr2=0x7fe5f40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.250+0000 7fe5fb000640 1 -- 172.21.15.2:0/2426879989 shutdown_connections 2024-09-10T08:07:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.250+0000 7fe5fb000640 1 -- 172.21.15.2:0/2426879989 wait complete. 2024-09-10T08:07:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.251+0000 7fe5fb000640 1 Processor -- start 2024-09-10T08:07:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.251+0000 7fe5fb000640 1 -- start start 2024-09-10T08:07:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.251+0000 7fe5fb000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5f4101dd0 0x7fe5f40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.251+0000 7fe5fb000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f4102780 0x7fe5f4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.251+0000 7fe5fb000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f4106f00 0x7fe5f4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.252+0000 7fe5fb000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f4111f60 con 0x7fe5f4102780 2024-09-10T08:07:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.252+0000 7fe5fb000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f4111de0 con 0x7fe5f4101dd0 2024-09-10T08:07:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.252+0000 7fe5fb000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f41120e0 con 0x7fe5f4106f00 2024-09-10T08:07:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.252+0000 7fe5f3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5f4101dd0 0x7fe5f40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.252+0000 7fe5f3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f4102780 0x7fe5f4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.252+0000 7fe5f3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5f4101dd0 0x7fe5f40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58390/0 (socket says 172.21.15.2:58390) 2024-09-10T08:07:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.252+0000 7fe5f3e00640 1 -- 172.21.15.2:0/3233708911 learned_addr learned my addr 172.21.15.2:0/3233708911 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:23.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.252+0000 7fe5f9000640 1 --2- 172.21.15.2:0/3233708911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f4106f00 0x7fe5f4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:23.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.252+0000 7fe5f3e00640 1 -- 172.21.15.2:0/3233708911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f4106f00 msgr2=0x7fe5f4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:23.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.252+0000 7fe5f3e00640 1 --2- 172.21.15.2:0/3233708911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f4106f00 0x7fe5f4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:23.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.252+0000 7fe5f3e00640 1 -- 172.21.15.2:0/3233708911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f4102780 msgr2=0x7fe5f4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:23.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.252+0000 7fe5f3e00640 1 --2- 172.21.15.2:0/3233708911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f4102780 0x7fe5f4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:23.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.252+0000 7fe5f3e00640 1 -- 172.21.15.2:0/3233708911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f41a6920 con 0x7fe5f4101dd0 2024-09-10T08:07:23.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.253+0000 7fe5f3e00640 1 --2- 172.21.15.2:0/3233708911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5f4101dd0 0x7fe5f40744f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4035040 tx=0x7fe5e4033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:23.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.253+0000 7fe5f0c00640 1 -- 172.21.15.2:0/3233708911 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4037040 con 0x7fe5f4101dd0 2024-09-10T08:07:23.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.253+0000 7fe5f0c00640 1 -- 172.21.15.2:0/3233708911 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5e4040070 con 0x7fe5f4101dd0 2024-09-10T08:07:23.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.253+0000 7fe5f0c00640 1 -- 172.21.15.2:0/3233708911 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4003a50 con 0x7fe5f4101dd0 2024-09-10T08:07:23.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.253+0000 7fe5fb000640 1 -- 172.21.15.2:0/3233708911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f41a6bb0 con 0x7fe5f4101dd0 2024-09-10T08:07:23.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.253+0000 7fe5fb000640 1 -- 172.21.15.2:0/3233708911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f41a7040 con 0x7fe5f4101dd0 2024-09-10T08:07:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.254+0000 7fe5fb000640 1 -- 172.21.15.2:0/3233708911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f41021b0 con 0x7fe5f4101dd0 2024-09-10T08:07:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.258+0000 7fe5f0c00640 1 -- 172.21.15.2:0/3233708911 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe5e4030670 con 0x7fe5f4101dd0 2024-09-10T08:07:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.259+0000 7fe5f0c00640 1 --2- 172.21.15.2:0/3233708911 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5d80a2c10 0x7fe5d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.259+0000 7fe5f0c00640 1 -- 172.21.15.2:0/3233708911 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe5e40ef720 con 0x7fe5f4101dd0 2024-09-10T08:07:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.260+0000 7fe5f3400640 1 --2- 172.21.15.2:0/3233708911 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5d80a2c10 0x7fe5d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.260+0000 7fe5f0c00640 1 -- 172.21.15.2:0/3233708911 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5e40bc1e0 con 0x7fe5f4101dd0 2024-09-10T08:07:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.260+0000 7fe5f3400640 1 --2- 172.21.15.2:0/3233708911 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5d80a2c10 0x7fe5d80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4075f80 tx=0x7fe5e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.451+0000 7fe5fb000640 1 -- 172.21.15.2:0/3233708911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe5f4075a20 con 0x7fe5f4101dd0 2024-09-10T08:07:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.452+0000 7fe5f0c00640 1 -- 172.21.15.2:0/3233708911 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe5f4075a20 con 0x7fe5f4101dd0 2024-09-10T08:07:23.065 INFO:teuthology.orchestra.run.smithi002.stdout:137438953546 2024-09-10T08:07:23.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.454+0000 7fe5fb000640 1 -- 172.21.15.2:0/3233708911 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5d80a2c10 msgr2=0x7fe5d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:23.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.454+0000 7fe5fb000640 1 --2- 172.21.15.2:0/3233708911 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5d80a2c10 0x7fe5d80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4075f80 tx=0x7fe5e0009260 comp rx=0 tx=0).stop 2024-09-10T08:07:23.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.454+0000 7fe5fb000640 1 -- 172.21.15.2:0/3233708911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5f4101dd0 msgr2=0x7fe5f40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:23.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.454+0000 7fe5fb000640 1 --2- 172.21.15.2:0/3233708911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5f4101dd0 0x7fe5f40744f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4035040 tx=0x7fe5e4033b60 comp rx=0 tx=0).stop 2024-09-10T08:07:23.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.454+0000 7fe5fb000640 1 -- 172.21.15.2:0/3233708911 shutdown_connections 2024-09-10T08:07:23.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.454+0000 7fe5fb000640 1 --2- 172.21.15.2:0/3233708911 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5d80a2c10 0x7fe5d80a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:23.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.454+0000 7fe5fb000640 1 --2- 172.21.15.2:0/3233708911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f4106f00 0x7fe5f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:23.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.454+0000 7fe5fb000640 1 --2- 172.21.15.2:0/3233708911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f4102780 0x7fe5f4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:23.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.454+0000 7fe5fb000640 1 --2- 172.21.15.2:0/3233708911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5f4101dd0 0x7fe5f40744f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:23.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.454+0000 7fe5fb000640 1 -- 172.21.15.2:0/3233708911 >> 172.21.15.2:0/3233708911 conn(0x7fe5f40fd890 msgr2=0x7fe5f40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:23.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.454+0000 7fe5fb000640 1 -- 172.21.15.2:0/3233708911 shutdown_connections 2024-09-10T08:07:23.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:21.454+0000 7fe5fb000640 1 -- 172.21.15.2:0/3233708911 wait complete. 2024-09-10T08:07:23.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:22 smithi155 ceph-mon[35405]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:23.450 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:23.841 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953543 got 137438953546 for osd.4 2024-09-10T08:07:23.841 DEBUG:teuthology.parallel:result is None 2024-09-10T08:07:23.845 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182476 got 107374182478 for osd.3 2024-09-10T08:07:23.845 DEBUG:teuthology.parallel:result is None 2024-09-10T08:07:24.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574936 got 55834574939 for osd.1 2024-09-10T08:07:24.439 DEBUG:teuthology.parallel:result is None 2024-09-10T08:07:24.982 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:24 smithi002 ceph-mon[30563]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T08:07:25.020 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411410 got 77309411414 for osd.2 2024-09-10T08:07:25.020 DEBUG:teuthology.parallel:result is None 2024-09-10T08:07:25.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:24 smithi155 ceph-mon[35405]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T08:07:25.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:24 smithi107 ceph-mon[35048]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T08:07:25.375 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528380 got 193273528382 for osd.6 2024-09-10T08:07:25.375 DEBUG:teuthology.parallel:result is None 2024-09-10T08:07:25.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.588+0000 7f06f4600640 1 -- 172.21.15.2:0/2843679286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec1049a0 msgr2=0x7f06ec1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:25.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.588+0000 7f06f4600640 1 --2- 172.21.15.2:0/2843679286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec1049a0 0x7f06ec1089f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009900 tx=0x7f06d802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:25.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.588+0000 7f06f4600640 1 -- 172.21.15.2:0/2843679286 shutdown_connections 2024-09-10T08:07:25.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.588+0000 7f06f4600640 1 --2- 172.21.15.2:0/2843679286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06ec109120 0x7f06ec110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:25.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.588+0000 7f06f4600640 1 --2- 172.21.15.2:0/2843679286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec1049a0 0x7f06ec1089f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:25.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.588+0000 7f06f4600640 1 --2- 172.21.15.2:0/2843679286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ec103ff0 0x7f06ec1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:25.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.588+0000 7f06f4600640 1 -- 172.21.15.2:0/2843679286 >> 172.21.15.2:0/2843679286 conn(0x7f06ec0ffa90 msgr2=0x7f06ec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:25.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.588+0000 7f06f4600640 1 -- 172.21.15.2:0/2843679286 shutdown_connections 2024-09-10T08:07:25.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.588+0000 7f06f4600640 1 -- 172.21.15.2:0/2843679286 wait complete. 2024-09-10T08:07:25.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.589+0000 7f06f4600640 1 Processor -- start 2024-09-10T08:07:25.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.590+0000 7f06f4600640 1 -- start start 2024-09-10T08:07:25.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.590+0000 7f06f4600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06ec103ff0 0x7f06ec19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:25.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.590+0000 7f06f4600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ec1049a0 0x7f06ec19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:25.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.590+0000 7f06f4600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec109120 0x7f06ec19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:25.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.590+0000 7f06f4600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec114140 con 0x7f06ec109120 2024-09-10T08:07:25.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.590+0000 7f06f4600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec113fc0 con 0x7f06ec1049a0 2024-09-10T08:07:25.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.590+0000 7f06f4600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1142c0 con 0x7f06ec103ff0 2024-09-10T08:07:25.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.590+0000 7f06f1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ec1049a0 0x7f06ec19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:25.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.590+0000 7f06f1c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06ec103ff0 0x7f06ec19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.590+0000 7f06f1c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06ec103ff0 0x7f06ec19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:40580/0 (socket says 172.21.15.2:40580) 2024-09-10T08:07:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.590+0000 7f06f1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ec1049a0 0x7f06ec19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58414/0 (socket says 172.21.15.2:58414) 2024-09-10T08:07:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.590+0000 7f06f1c00640 1 -- 172.21.15.2:0/1922476772 learned_addr learned my addr 172.21.15.2:0/1922476772 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.590+0000 7f06f2600640 1 --2- 172.21.15.2:0/1922476772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec109120 0x7f06ec19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.591+0000 7f06f1c00640 1 -- 172.21.15.2:0/1922476772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ec1049a0 msgr2=0x7f06ec19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.591+0000 7f06f1c00640 1 --2- 172.21.15.2:0/1922476772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ec1049a0 0x7f06ec19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.591+0000 7f06f1c00640 1 -- 172.21.15.2:0/1922476772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec109120 msgr2=0x7f06ec19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.591+0000 7f06f1c00640 1 --2- 172.21.15.2:0/1922476772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec109120 0x7f06ec19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.591+0000 7f06f1c00640 1 -- 172.21.15.2:0/1922476772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ec19f5d0 con 0x7f06ec103ff0 2024-09-10T08:07:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.591+0000 7f06f1c00640 1 --2- 172.21.15.2:0/1922476772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06ec103ff0 0x7f06ec19a620 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f06e000ea60 tx=0x7f06e000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.591+0000 7f06ea000640 1 -- 172.21.15.2:0/1922476772 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e000cde0 con 0x7f06ec103ff0 2024-09-10T08:07:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.591+0000 7f06ea000640 1 -- 172.21.15.2:0/1922476772 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e00035f0 con 0x7f06ec103ff0 2024-09-10T08:07:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.591+0000 7f06ea000640 1 -- 172.21.15.2:0/1922476772 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0010670 con 0x7f06ec103ff0 2024-09-10T08:07:25.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.591+0000 7f06f4600640 1 -- 172.21.15.2:0/1922476772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ec19f8c0 con 0x7f06ec103ff0 2024-09-10T08:07:25.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.592+0000 7f06f4600640 1 -- 172.21.15.2:0/1922476772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ec1a71a0 con 0x7f06ec103ff0 2024-09-10T08:07:25.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.593+0000 7f06f4600640 1 -- 172.21.15.2:0/1922476772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06ec1043d0 con 0x7f06ec103ff0 2024-09-10T08:07:25.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.594+0000 7f06ea000640 1 -- 172.21.15.2:0/1922476772 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f06e00037a0 con 0x7f06ec103ff0 2024-09-10T08:07:25.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.596+0000 7f06ea000640 1 --2- 172.21.15.2:0/1922476772 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06d40a2c90 0x7f06d40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:25.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.596+0000 7f06ea000640 1 -- 172.21.15.2:0/1922476772 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f06e00cf960 con 0x7f06ec103ff0 2024-09-10T08:07:25.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.596+0000 7f06f1200640 1 --2- 172.21.15.2:0/1922476772 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06d40a2c90 0x7f06d40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:25.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.597+0000 7f06f1200640 1 --2- 172.21.15.2:0/1922476772 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06d40a2c90 0x7f06d40a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009230 tx=0x7f06d80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:25.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.600+0000 7f06ea000640 1 -- 172.21.15.2:0/1922476772 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f06e009c310 con 0x7f06ec103ff0 2024-09-10T08:07:25.630 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:25.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.808+0000 7f06f4600640 1 -- 172.21.15.2:0/1922476772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f06ec19b980 con 0x7f06ec103ff0 2024-09-10T08:07:25.809 INFO:teuthology.orchestra.run.smithi002.stdout:34359738464 2024-09-10T08:07:25.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.809+0000 7f06ea000640 1 -- 172.21.15.2:0/1922476772 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f06e009bc10 con 0x7f06ec103ff0 2024-09-10T08:07:25.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.811+0000 7f06f4600640 1 -- 172.21.15.2:0/1922476772 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06d40a2c90 msgr2=0x7f06d40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:25.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.811+0000 7f06f4600640 1 --2- 172.21.15.2:0/1922476772 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06d40a2c90 0x7f06d40a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009230 tx=0x7f06d80058d0 comp rx=0 tx=0).stop 2024-09-10T08:07:25.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.811+0000 7f06f4600640 1 -- 172.21.15.2:0/1922476772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06ec103ff0 msgr2=0x7f06ec19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:25.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.811+0000 7f06f4600640 1 --2- 172.21.15.2:0/1922476772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06ec103ff0 0x7f06ec19a620 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f06e000ea60 tx=0x7f06e000ef20 comp rx=0 tx=0).stop 2024-09-10T08:07:25.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.812+0000 7f06f4600640 1 -- 172.21.15.2:0/1922476772 shutdown_connections 2024-09-10T08:07:25.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.812+0000 7f06f4600640 1 --2- 172.21.15.2:0/1922476772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06ec109120 0x7f06ec19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:25.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.812+0000 7f06f4600640 1 --2- 172.21.15.2:0/1922476772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ec1049a0 0x7f06ec19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:25.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.812+0000 7f06f4600640 1 --2- 172.21.15.2:0/1922476772 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06d40a2c90 0x7f06d40a5150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:25.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.812+0000 7f06f4600640 1 --2- 172.21.15.2:0/1922476772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06ec103ff0 0x7f06ec19a620 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:25.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.812+0000 7f06f4600640 1 -- 172.21.15.2:0/1922476772 >> 172.21.15.2:0/1922476772 conn(0x7f06ec0ffa90 msgr2=0x7f06ec100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:25.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.812+0000 7f06f4600640 1 -- 172.21.15.2:0/1922476772 shutdown_connections 2024-09-10T08:07:25.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:25.812+0000 7f06f4600640 1 -- 172.21.15.2:0/1922476772 wait complete. 2024-09-10T08:07:25.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:25 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:07:26.012 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:25 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:07:26.019 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:25 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.363+0000 7f69a9600640 1 -- 172.21.15.2:0/1318180485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69a41027a0 msgr2=0x7f69a41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.363+0000 7f69a9600640 1 --2- 172.21.15.2:0/1318180485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69a41027a0 0x7f69a41067f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6990009900 tx=0x7f699002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.363+0000 7f69a9600640 1 -- 172.21.15.2:0/1318180485 shutdown_connections 2024-09-10T08:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.363+0000 7f69a9600640 1 --2- 172.21.15.2:0/1318180485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69a4106f20 0x7f69a410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.363+0000 7f69a9600640 1 --2- 172.21.15.2:0/1318180485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69a41027a0 0x7f69a41067f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.363+0000 7f69a9600640 1 --2- 172.21.15.2:0/1318180485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69a4101df0 0x7f69a41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.363+0000 7f69a9600640 1 -- 172.21.15.2:0/1318180485 >> 172.21.15.2:0/1318180485 conn(0x7f69a40fd8b0 msgr2=0x7f69a40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.363+0000 7f69a9600640 1 -- 172.21.15.2:0/1318180485 shutdown_connections 2024-09-10T08:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.363+0000 7f69a9600640 1 -- 172.21.15.2:0/1318180485 wait complete. 2024-09-10T08:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.364+0000 7f69a9600640 1 Processor -- start 2024-09-10T08:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a9600640 1 -- start start 2024-09-10T08:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a9600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69a4101df0 0x7f69a419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a9600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69a41027a0 0x7f69a419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a9600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69a4106f20 0x7f69a419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a9600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4111fd0 con 0x7f69a41027a0 2024-09-10T08:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a9600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4111e50 con 0x7f69a4101df0 2024-09-10T08:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a9600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4112150 con 0x7f69a4106f20 2024-09-10T08:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69a41027a0 0x7f69a419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69a4101df0 0x7f69a419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69a4106f20 0x7f69a419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69a4106f20 0x7f69a419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:40598/0 (socket says 172.21.15.2:40598) 2024-09-10T08:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a3400640 1 -- 172.21.15.2:0/1124372932 learned_addr learned my addr 172.21.15.2:0/1124372932 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69a4101df0 0x7f69a419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58428/0 (socket says 172.21.15.2:58428) 2024-09-10T08:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.365+0000 7f69a2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69a41027a0 0x7f69a419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44820/0 (socket says 172.21.15.2:44820) 2024-09-10T08:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.366+0000 7f69a3400640 1 -- 172.21.15.2:0/1124372932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69a4101df0 msgr2=0x7f69a419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.366+0000 7f69a3400640 1 --2- 172.21.15.2:0/1124372932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69a4101df0 0x7f69a419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.366+0000 7f69a3400640 1 -- 172.21.15.2:0/1124372932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69a41027a0 msgr2=0x7f69a419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.366+0000 7f69a3400640 1 --2- 172.21.15.2:0/1124372932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69a41027a0 0x7f69a419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.366+0000 7f69a3400640 1 -- 172.21.15.2:0/1124372932 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a419f5f0 con 0x7f69a4106f20 2024-09-10T08:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.366+0000 7f69a2000640 1 --2- 172.21.15.2:0/1124372932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69a41027a0 0x7f69a419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.366+0000 7f69a3400640 1 --2- 172.21.15.2:0/1124372932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69a4106f20 0x7f69a419ef10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f699c00eb10 tx=0x7f699c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.366+0000 7f699b400640 1 -- 172.21.15.2:0/1124372932 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c004b20 con 0x7f69a4106f20 2024-09-10T08:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.366+0000 7f699b400640 1 -- 172.21.15.2:0/1124372932 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f699c004cc0 con 0x7f69a4106f20 2024-09-10T08:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.366+0000 7f699b400640 1 -- 172.21.15.2:0/1124372932 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c010760 con 0x7f69a4106f20 2024-09-10T08:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.366+0000 7f69a9600640 1 -- 172.21.15.2:0/1124372932 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a419f8e0 con 0x7f69a4106f20 2024-09-10T08:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.366+0000 7f69a9600640 1 -- 172.21.15.2:0/1124372932 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a41a71c0 con 0x7f69a4106f20 2024-09-10T08:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.368+0000 7f69a9600640 1 -- 172.21.15.2:0/1124372932 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a41021d0 con 0x7f69a4106f20 2024-09-10T08:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.370+0000 7f699b400640 1 -- 172.21.15.2:0/1124372932 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f699c005020 con 0x7f69a4106f20 2024-09-10T08:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.371+0000 7f699b400640 1 --2- 172.21.15.2:0/1124372932 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f69880a2c90 0x7f69880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.371+0000 7f699b400640 1 -- 172.21.15.2:0/1124372932 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f699c0cfbd0 con 0x7f69a4106f20 2024-09-10T08:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.371+0000 7f69a2a00640 1 --2- 172.21.15.2:0/1124372932 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f69880a2c90 0x7f69880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.372+0000 7f69a2a00640 1 --2- 172.21.15.2:0/1124372932 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f69880a2c90 0x7f69880a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6994009bd0 tx=0x7f6994005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.375+0000 7f699b400640 1 -- 172.21.15.2:0/1124372932 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f699c09c580 con 0x7f69a4106f20 2024-09-10T08:07:26.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.576+0000 7f69a9600640 1 -- 172.21.15.2:0/1124372932 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f69a41a74a0 con 0x7f69a4106f20 2024-09-10T08:07:26.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.576+0000 7f699b400640 1 -- 172.21.15.2:0/1124372932 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f699c09be80 con 0x7f69a4106f20 2024-09-10T08:07:26.576 INFO:teuthology.orchestra.run.smithi002.stdout:158913790023 2024-09-10T08:07:26.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.578+0000 7f69a9600640 1 -- 172.21.15.2:0/1124372932 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f69880a2c90 msgr2=0x7f69880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:26.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.578+0000 7f69a9600640 1 --2- 172.21.15.2:0/1124372932 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f69880a2c90 0x7f69880a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6994009bd0 tx=0x7f6994005f90 comp rx=0 tx=0).stop 2024-09-10T08:07:26.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.578+0000 7f69a9600640 1 -- 172.21.15.2:0/1124372932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69a4106f20 msgr2=0x7f69a419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:26.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.578+0000 7f69a9600640 1 --2- 172.21.15.2:0/1124372932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69a4106f20 0x7f69a419ef10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f699c00eb10 tx=0x7f699c00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:07:26.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.578+0000 7f69a9600640 1 -- 172.21.15.2:0/1124372932 shutdown_connections 2024-09-10T08:07:26.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.578+0000 7f69a9600640 1 --2- 172.21.15.2:0/1124372932 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f69880a2c90 0x7f69880a5150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:26.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.578+0000 7f69a9600640 1 --2- 172.21.15.2:0/1124372932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69a4106f20 0x7f69a419ef10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:26.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.578+0000 7f69a9600640 1 --2- 172.21.15.2:0/1124372932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69a41027a0 0x7f69a419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:26.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.578+0000 7f69a9600640 1 --2- 172.21.15.2:0/1124372932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69a4101df0 0x7f69a419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:26.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.578+0000 7f69a9600640 1 -- 172.21.15.2:0/1124372932 >> 172.21.15.2:0/1124372932 conn(0x7f69a40fd8b0 msgr2=0x7f69a40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:26.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.578+0000 7f69a9600640 1 -- 172.21.15.2:0/1124372932 shutdown_connections 2024-09-10T08:07:26.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:26.579+0000 7f69a9600640 1 -- 172.21.15.2:0/1124372932 wait complete. 2024-09-10T08:07:27.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:26 smithi002 ceph-mon[30563]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:27.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:26 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1922476772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T08:07:27.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:26 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1124372932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T08:07:27.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:26 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:27.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:26 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:27.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:26 smithi155 ceph-mon[35405]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:27.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:26 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1922476772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T08:07:27.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:26 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1124372932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T08:07:27.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:26 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:27.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:26 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:27.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:26 smithi107 ceph-mon[35048]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:27.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:26 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1922476772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T08:07:27.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:26 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1124372932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T08:07:27.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:26 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:27.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:26 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:27.358 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738461 got 34359738464 for osd.0 2024-09-10T08:07:27.358 DEBUG:teuthology.parallel:result is None 2024-09-10T08:07:28.002 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:28.002 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:27 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:28.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:27 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:28.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:27 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:28.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:27 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:28.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:27 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:28.570 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790020 got 158913790023 for osd.5 2024-09-10T08:07:28.570 DEBUG:teuthology.parallel:result is None 2024-09-10T08:07:28.570 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T08:07:28.571 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:07:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:29.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:28 smithi002 ceph-mon[30563]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:29.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:28 smithi155 ceph-mon[35405]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:29.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:28 smithi107 ceph-mon[35048]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:29.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.582+0000 7f68ea400640 1 -- 172.21.15.2:0/1128240488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e4076720 msgr2=0x7f68e4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:29.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.582+0000 7f68ea400640 1 --2- 172.21.15.2:0/1128240488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e4076720 0x7f68e4076b00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009900 tx=0x7f68d402a410 comp rx=0 tx=0).stop 2024-09-10T08:07:29.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.582+0000 7f68ea400640 1 -- 172.21.15.2:0/1128240488 shutdown_connections 2024-09-10T08:07:29.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.582+0000 7f68ea400640 1 --2- 172.21.15.2:0/1128240488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e4112a30 0x7f68e4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:29.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.582+0000 7f68ea400640 1 --2- 172.21.15.2:0/1128240488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e4077040 0x7f68e41124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:29.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.582+0000 7f68ea400640 1 --2- 172.21.15.2:0/1128240488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e4076720 0x7f68e4076b00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:29.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.582+0000 7f68ea400640 1 -- 172.21.15.2:0/1128240488 >> 172.21.15.2:0/1128240488 conn(0x7f68e40ffa90 msgr2=0x7f68e4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:29.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.582+0000 7f68ea400640 1 -- 172.21.15.2:0/1128240488 shutdown_connections 2024-09-10T08:07:29.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.582+0000 7f68ea400640 1 -- 172.21.15.2:0/1128240488 wait complete. 2024-09-10T08:07:29.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.584+0000 7f68ea400640 1 Processor -- start 2024-09-10T08:07:29.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.584+0000 7f68ea400640 1 -- start start 2024-09-10T08:07:29.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.584+0000 7f68ea400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e4076720 0x7f68e41a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:29.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.584+0000 7f68ea400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e4077040 0x7f68e41a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:29.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.584+0000 7f68ea400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e4112a30 0x7f68e419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:29.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.584+0000 7f68ea400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f68e41185a0 con 0x7f68e4112a30 2024-09-10T08:07:29.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.584+0000 7f68ea400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4118420 con 0x7f68e4076720 2024-09-10T08:07:29.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.584+0000 7f68ea400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4118720 con 0x7f68e4077040 2024-09-10T08:07:29.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.585+0000 7f68e3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e4112a30 0x7f68e419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:29.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.585+0000 7f68e3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e4076720 0x7f68e41a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:29.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.585+0000 7f68e3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e4076720 0x7f68e41a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:56872/0 (socket says 172.21.15.2:56872) 2024-09-10T08:07:29.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.585+0000 7f68e3400640 1 -- 172.21.15.2:0/4112623585 learned_addr learned my addr 172.21.15.2:0/4112623585 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:29.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.585+0000 7f68e2a00640 1 --2- 172.21.15.2:0/4112623585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e4077040 0x7f68e41a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:29.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.585+0000 7f68e3e00640 1 -- 172.21.15.2:0/4112623585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e4077040 msgr2=0x7f68e41a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:29.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.585+0000 7f68e3e00640 1 --2- 172.21.15.2:0/4112623585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e4077040 0x7f68e41a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:29.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.585+0000 7f68e3e00640 1 -- 172.21.15.2:0/4112623585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e4076720 msgr2=0x7f68e41a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:29.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.585+0000 7f68e3e00640 1 --2- 172.21.15.2:0/4112623585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e4076720 0x7f68e41a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:29.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.585+0000 7f68e3e00640 1 -- 172.21.15.2:0/4112623585 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e419efe0 con 0x7f68e4112a30 2024-09-10T08:07:29.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.586+0000 7f68e3e00640 1 --2- 172.21.15.2:0/4112623585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e4112a30 0x7f68e419e8b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f68e407aa10 tx=0x7f68dc00dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:29.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.586+0000 7f68dbe00640 1 -- 172.21.15.2:0/4112623585 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc00bd40 con 0x7f68e4112a30 2024-09-10T08:07:29.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.586+0000 7f68dbe00640 1 -- 172.21.15.2:0/4112623585 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68dc003e50 con 0x7f68e4112a30 2024-09-10T08:07:29.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.586+0000 7f68dbe00640 1 -- 172.21.15.2:0/4112623585 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc0106c0 con 0x7f68e4112a30 2024-09-10T08:07:29.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.586+0000 7f68ea400640 1 -- 172.21.15.2:0/4112623585 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e419f2d0 con 0x7f68e4112a30 2024-09-10T08:07:29.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.586+0000 7f68ea400640 1 -- 172.21.15.2:0/4112623585 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e41ab5c0 con 0x7f68e4112a30 2024-09-10T08:07:29.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.587+0000 7f68ea400640 1 -- 172.21.15.2:0/4112623585 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e4076b00 con 0x7f68e4112a30 2024-09-10T08:07:29.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.589+0000 7f68dbe00640 1 -- 172.21.15.2:0/4112623585 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f68dc003970 con 0x7f68e4112a30 2024-09-10T08:07:29.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.591+0000 7f68dbe00640 1 --2- 172.21.15.2:0/4112623585 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f68c80a2c10 0x7f68c80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:29.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.591+0000 7f68dbe00640 1 -- 172.21.15.2:0/4112623585 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f68dc0cf890 con 0x7f68e4112a30 2024-09-10T08:07:29.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.591+0000 7f68e3400640 1 --2- 172.21.15.2:0/4112623585 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f68c80a2c10 0x7f68c80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:29.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.592+0000 7f68e3400640 1 --2- 172.21.15.2:0/4112623585 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f68c80a2c10 0x7f68c80a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f68d40096c0 tx=0x7f68d4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:29.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.595+0000 7f68dbe00640 1 -- 172.21.15.2:0/4112623585 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68dc09c240 con 0x7f68e4112a30 2024-09-10T08:07:29.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.794+0000 7f68ea400640 1 -- 172.21.15.2:0/4112623585 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f68e4062630 con 0x7f68c80a2c10 2024-09-10T08:07:29.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.805+0000 7f68dbe00640 1 -- 172.21.15.2:0/4112623585 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340606 (secure 0 0 0) 0x7f68e4062630 con 0x7f68c80a2c10 2024-09-10T08:07:29.808 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:07:29.810 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-10T08:07:29.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.810+0000 7f68ea400640 1 -- 172.21.15.2:0/4112623585 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f68c80a2c10 msgr2=0x7f68c80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:29.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.810+0000 7f68ea400640 1 --2- 172.21.15.2:0/4112623585 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f68c80a2c10 0x7f68c80a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f68d40096c0 tx=0x7f68d4005a20 comp rx=0 tx=0).stop 2024-09-10T08:07:29.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.810+0000 7f68ea400640 1 -- 172.21.15.2:0/4112623585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e4112a30 msgr2=0x7f68e419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:29.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.810+0000 7f68ea400640 1 --2- 172.21.15.2:0/4112623585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e4112a30 0x7f68e419e8b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f68e407aa10 tx=0x7f68dc00dea0 comp rx=0 tx=0).stop 2024-09-10T08:07:29.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.810+0000 7f68ea400640 1 -- 172.21.15.2:0/4112623585 shutdown_connections 2024-09-10T08:07:29.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.810+0000 7f68ea400640 1 --2- 172.21.15.2:0/4112623585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e4112a30 0x7f68e419e8b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:29.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.810+0000 7f68ea400640 1 --2- 172.21.15.2:0/4112623585 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f68c80a2c10 0x7f68c80a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:29.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.810+0000 7f68ea400640 1 --2- 172.21.15.2:0/4112623585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e4077040 0x7f68e41a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:29.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.810+0000 7f68ea400640 1 --2- 172.21.15.2:0/4112623585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e4076720 0x7f68e41a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:29.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.810+0000 7f68ea400640 1 -- 172.21.15.2:0/4112623585 >> 172.21.15.2:0/4112623585 conn(0x7f68e40ffa90 msgr2=0x7f68e4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:29.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.810+0000 7f68ea400640 1 -- 172.21.15.2:0/4112623585 shutdown_connections 2024-09-10T08:07:29.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:29.810+0000 7f68ea400640 1 -- 172.21.15.2:0/4112623585 wait complete. 2024-09-10T08:07:30.615 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-10T08:07:29.672377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657266,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":542,"num_read_kb":387,"num_write":350,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1184461,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362112,"kb_used_data":12788,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579712,"statfs":{"total":1151940427776,"available":1151569625088,"internally_reserved":0,"allocated":13094912,"data_stored":8523896,"data_compressed":497802,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003361"},"pg_stats":[{"pgid":"2.1f","version":"93'9","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514795+0000","last_change":"2024-09-10T08:05:22.462062+0000","last_active":"2024-09-10T08:07:05.514795+0000","last_peered":"2024-09-10T08:07:05.514795+0000","last_clean":"2024-09-10T08:07:05.514795+0000","last_became_active":"2024-09-10T08:05:22.460973+0000","last_became_peered":"2024-09-10T08:05:22.460973+0000","last_unstale":"2024-09-10T08:07:05.514795+0000","last_undegraded":"2024-09-10T08:07:05.514795+0000","last_fullsized":"2024-09-10T08:07:05.514795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:01.357446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.877341+0000","last_change":"2024-09-10T08:05:25.482873+0000","last_active":"2024-09-10T08:07:05.877341+0000","last_peered":"2024-09-10T08:07:05.877341+0000","last_clean":"2024-09-10T08:07:05.877341+0000","last_became_active":"2024-09-10T08:05:25.482559+0000","last_became_peered":"2024-09-10T08:05:25.482559+0000","last_unstale":"2024-09-10T08:07:05.877341+0000","last_undegraded":"2024-09-10T08:07:05.877341+0000","last_fullsized":"2024-09-10T08:07:05.877341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:10:17.366160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511328+0000","last_change":"2024-09-10T08:06:04.736356+0000","last_active":"2024-09-10T08:07:05.511328+0000","last_peered":"2024-09-10T08:07:05.511328+0000","last_clean":"2024-09-10T08:07:05.511328+0000","last_became_active":"2024-09-10T08:06:04.735859+0000","last_became_peered":"2024-09-10T08:06:04.735859+0000","last_unstale":"2024-09-10T08:07:05.511328+0000","last_undegraded":"2024-09-10T08:07:05.511328+0000","last_fullsized":"2024-09-10T08:07:05.511328+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:56.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"93'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.877433+0000","last_change":"2024-09-10T08:05:22.569504+0000","last_active":"2024-09-10T08:07:05.877433+0000","last_peered":"2024-09-10T08:07:05.877433+0000","last_clean":"2024-09-10T08:07:05.877433+0000","last_became_active":"2024-09-10T08:05:22.569164+0000","last_became_peered":"2024-09-10T08:05:22.569164+0000","last_unstale":"2024-09-10T08:07:05.877433+0000","last_undegraded":"2024-09-10T08:07:05.877433+0000","last_fullsized":"2024-09-10T08:07:05.877433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:41.628963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513937+0000","last_change":"2024-09-10T08:05:25.493017+0000","last_active":"2024-09-10T08:07:05.513937+0000","last_peered":"2024-09-10T08:07:05.513937+0000","last_clean":"2024-09-10T08:07:05.513937+0000","last_became_active":"2024-09-10T08:05:25.491370+0000","last_became_peered":"2024-09-10T08:05:25.491370+0000","last_unstale":"2024-09-10T08:07:05.513937+0000","last_undegraded":"2024-09-10T08:07:05.513937+0000","last_fullsized":"2024-09-10T08:07:05.513937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:41.616549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914030+0000","last_change":"2024-09-10T08:05:35.523751+0000","last_active":"2024-09-10T08:07:05.914030+0000","last_peered":"2024-09-10T08:07:05.914030+0000","last_clean":"2024-09-10T08:07:05.914030+0000","last_became_active":"2024-09-10T08:05:35.523372+0000","last_became_peered":"2024-09-10T08:05:35.523372+0000","last_unstale":"2024-09-10T08:07:05.914030+0000","last_undegraded":"2024-09-10T08:07:05.914030+0000","last_fullsized":"2024-09-10T08:07:05.914030+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:20.971596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'4","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508861+0000","last_change":"2024-09-10T08:05:22.460590+0000","last_active":"2024-09-10T08:07:05.508861+0000","last_peered":"2024-09-10T08:07:05.508861+0000","last_clean":"2024-09-10T08:07:05.508861+0000","last_became_active":"2024-09-10T08:05:22.460238+0000","last_became_peered":"2024-09-10T08:05:22.460238+0000","last_unstale":"2024-09-10T08:07:05.508861+0000","last_undegraded":"2024-09-10T08:07:05.508861+0000","last_fullsized":"2024-09-10T08:07:05.508861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:18.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914856+0000","last_change":"2024-09-10T08:05:25.487392+0000","last_active":"2024-09-10T08:07:05.914856+0000","last_peered":"2024-09-10T08:07:05.914856+0000","last_clean":"2024-09-10T08:07:05.914856+0000","last_became_active":"2024-09-10T08:05:25.487054+0000","last_became_peered":"2024-09-10T08:05:25.487054+0000","last_unstale":"2024-09-10T08:07:05.914856+0000","last_undegraded":"2024-09-10T08:07:05.914856+0000","last_fullsized":"2024-09-10T08:07:05.914856+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:16.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914757+0000","last_change":"2024-09-10T08:06:04.735778+0000","last_active":"2024-09-10T08:07:05.914757+0000","last_peered":"2024-09-10T08:07:05.914757+0000","last_clean":"2024-09-10T08:07:05.914757+0000","last_became_active":"2024-09-10T08:06:04.735208+0000","last_became_peered":"2024-09-10T08:06:04.735208+0000","last_unstale":"2024-09-10T08:07:05.914757+0000","last_undegraded":"2024-09-10T08:07:05.914757+0000","last_fullsized":"2024-09-10T08:07:05.914757+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:36.982246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508892+0000","last_change":"2024-09-10T08:05:22.458323+0000","last_active":"2024-09-10T08:07:05.508892+0000","last_peered":"2024-09-10T08:07:05.508892+0000","last_clean":"2024-09-10T08:07:05.508892+0000","last_became_active":"2024-09-10T08:05:22.457965+0000","last_became_peered":"2024-09-10T08:05:22.457965+0000","last_unstale":"2024-09-10T08:07:05.508892+0000","last_undegraded":"2024-09-10T08:07:05.508892+0000","last_fullsized":"2024-09-10T08:07:05.508892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:15.021305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569413+0000","last_change":"2024-09-10T08:05:25.489883+0000","last_active":"2024-09-10T08:07:05.569413+0000","last_peered":"2024-09-10T08:07:05.569413+0000","last_clean":"2024-09-10T08:07:05.569413+0000","last_became_active":"2024-09-10T08:05:25.489482+0000","last_became_peered":"2024-09-10T08:05:25.489482+0000","last_unstale":"2024-09-10T08:07:05.569413+0000","last_undegraded":"2024-09-10T08:07:05.569413+0000","last_fullsized":"2024-09-10T08:07:05.569413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:52.080628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914006+0000","last_change":"2024-09-10T08:05:35.531458+0000","last_active":"2024-09-10T08:07:05.914006+0000","last_peered":"2024-09-10T08:07:05.914006+0000","last_clean":"2024-09-10T08:07:05.914006+0000","last_became_active":"2024-09-10T08:05:35.531072+0000","last_became_peered":"2024-09-10T08:05:35.531072+0000","last_unstale":"2024-09-10T08:07:05.914006+0000","last_undegraded":"2024-09-10T08:07:05.914006+0000","last_fullsized":"2024-09-10T08:07:05.914006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:04.792664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509274+0000","last_change":"2024-09-10T08:05:25.486865+0000","last_active":"2024-09-10T08:07:05.509274+0000","last_peered":"2024-09-10T08:07:05.509274+0000","last_clean":"2024-09-10T08:07:05.509274+0000","last_became_active":"2024-09-10T08:05:25.486531+0000","last_became_peered":"2024-09-10T08:05:25.486531+0000","last_unstale":"2024-09-10T08:07:05.509274+0000","last_undegraded":"2024-09-10T08:07:05.509274+0000","last_fullsized":"2024-09-10T08:07:05.509274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:26.854682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"93'4","reported_seq":59,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508843+0000","last_change":"2024-09-10T08:06:04.743303+0000","last_active":"2024-09-10T08:07:05.508843+0000","last_peered":"2024-09-10T08:07:05.508843+0000","last_clean":"2024-09-10T08:07:05.508843+0000","last_became_active":"2024-09-10T08:06:04.733294+0000","last_became_peered":"2024-09-10T08:06:04.733294+0000","last_unstale":"2024-09-10T08:07:05.508843+0000","last_undegraded":"2024-09-10T08:07:05.508843+0000","last_fullsized":"2024-09-10T08:07:05.508843+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:18.542658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914638+0000","last_change":"2024-09-10T08:05:35.642083+0000","last_active":"2024-09-10T08:07:05.914638+0000","last_peered":"2024-09-10T08:07:05.914638+0000","last_clean":"2024-09-10T08:07:05.914638+0000","last_became_active":"2024-09-10T08:05:35.641214+0000","last_became_peered":"2024-09-10T08:05:35.641214+0000","last_unstale":"2024-09-10T08:07:05.914638+0000","last_undegraded":"2024-09-10T08:07:05.914638+0000","last_fullsized":"2024-09-10T08:07:05.914638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:00:21.731769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876974+0000","last_change":"2024-09-10T08:05:25.501993+0000","last_active":"2024-09-10T08:07:05.876974+0000","last_peered":"2024-09-10T08:07:05.876974+0000","last_clean":"2024-09-10T08:07:05.876974+0000","last_became_active":"2024-09-10T08:05:25.499912+0000","last_became_peered":"2024-09-10T08:05:25.499912+0000","last_unstale":"2024-09-10T08:07:05.876974+0000","last_undegraded":"2024-09-10T08:07:05.876974+0000","last_fullsized":"2024-09-10T08:07:05.876974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:06.780067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"93'1","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511452+0000","last_change":"2024-09-10T08:05:22.461786+0000","last_active":"2024-09-10T08:07:05.511452+0000","last_peered":"2024-09-10T08:07:05.511452+0000","last_clean":"2024-09-10T08:07:05.511452+0000","last_became_active":"2024-09-10T08:05:22.460821+0000","last_became_peered":"2024-09-10T08:05:22.460821+0000","last_unstale":"2024-09-10T08:07:05.511452+0000","last_undegraded":"2024-09-10T08:07:05.511452+0000","last_fullsized":"2024-09-10T08:07:05.511452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:25.083538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914505+0000","last_change":"2024-09-10T08:06:04.734470+0000","last_active":"2024-09-10T08:07:05.914505+0000","last_peered":"2024-09-10T08:07:05.914505+0000","last_clean":"2024-09-10T08:07:05.914505+0000","last_became_active":"2024-09-10T08:06:04.734040+0000","last_became_peered":"2024-09-10T08:06:04.734040+0000","last_unstale":"2024-09-10T08:07:05.914505+0000","last_undegraded":"2024-09-10T08:07:05.914505+0000","last_fullsized":"2024-09-10T08:07:05.914505+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:36.312144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769794+0000","last_change":"2024-09-10T08:05:25.492711+0000","last_active":"2024-09-10T08:07:05.769794+0000","last_peered":"2024-09-10T08:07:05.769794+0000","last_clean":"2024-09-10T08:07:05.769794+0000","last_became_active":"2024-09-10T08:05:25.492336+0000","last_became_peered":"2024-09-10T08:05:25.492336+0000","last_unstale":"2024-09-10T08:07:05.769794+0000","last_undegraded":"2024-09-10T08:07:05.769794+0000","last_fullsized":"2024-09-10T08:07:05.769794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:04.168884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"93'3","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876730+0000","last_change":"2024-09-10T08:05:22.461535+0000","last_active":"2024-09-10T08:07:05.876730+0000","last_peered":"2024-09-10T08:07:05.876730+0000","last_clean":"2024-09-10T08:07:05.876730+0000","last_became_active":"2024-09-10T08:05:22.461208+0000","last_became_peered":"2024-09-10T08:05:22.461208+0000","last_unstale":"2024-09-10T08:07:05.876730+0000","last_undegraded":"2024-09-10T08:07:05.876730+0000","last_fullsized":"2024-09-10T08:07:05.876730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:26.009242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513843+0000","last_change":"2024-09-10T08:05:35.535442+0000","last_active":"2024-09-10T08:07:05.513843+0000","last_peered":"2024-09-10T08:07:05.513843+0000","last_clean":"2024-09-10T08:07:05.513843+0000","last_became_active":"2024-09-10T08:05:35.535058+0000","last_became_peered":"2024-09-10T08:05:35.535058+0000","last_unstale":"2024-09-10T08:07:05.513843+0000","last_undegraded":"2024-09-10T08:07:05.513843+0000","last_fullsized":"2024-09-10T08:07:05.513843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:39.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508321+0000","last_change":"2024-09-10T08:05:25.483440+0000","last_active":"2024-09-10T08:07:05.508321+0000","last_peered":"2024-09-10T08:07:05.508321+0000","last_clean":"2024-09-10T08:07:05.508321+0000","last_became_active":"2024-09-10T08:05:25.483141+0000","last_became_peered":"2024-09-10T08:05:25.483141+0000","last_unstale":"2024-09-10T08:07:05.508321+0000","last_undegraded":"2024-09-10T08:07:05.508321+0000","last_fullsized":"2024-09-10T08:07:05.508321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:00.953188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914286+0000","last_change":"2024-09-10T08:05:22.568141+0000","last_active":"2024-09-10T08:07:05.914286+0000","last_peered":"2024-09-10T08:07:05.914286+0000","last_clean":"2024-09-10T08:07:05.914286+0000","last_became_active":"2024-09-10T08:05:22.567857+0000","last_became_peered":"2024-09-10T08:05:22.567857+0000","last_unstale":"2024-09-10T08:07:05.914286+0000","last_undegraded":"2024-09-10T08:07:05.914286+0000","last_fullsized":"2024-09-10T08:07:05.914286+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:22.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914747+0000","last_change":"2024-09-10T08:05:35.641993+0000","last_active":"2024-09-10T08:07:05.914747+0000","last_peered":"2024-09-10T08:07:05.914747+0000","last_clean":"2024-09-10T08:07:05.914747+0000","last_became_active":"2024-09-10T08:05:35.641376+0000","last_became_peered":"2024-09-10T08:05:35.641376+0000","last_unstale":"2024-09-10T08:07:05.914747+0000","last_undegraded":"2024-09-10T08:07:05.914747+0000","last_fullsized":"2024-09-10T08:07:05.914747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:16:34.038924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513778+0000","last_change":"2024-09-10T08:06:04.737560+0000","last_active":"2024-09-10T08:07:05.513778+0000","last_peered":"2024-09-10T08:07:05.513778+0000","last_clean":"2024-09-10T08:07:05.513778+0000","last_became_active":"2024-09-10T08:06:04.736656+0000","last_became_peered":"2024-09-10T08:06:04.736656+0000","last_unstale":"2024-09-10T08:07:05.513778+0000","last_undegraded":"2024-09-10T08:07:05.513778+0000","last_fullsized":"2024-09-10T08:07:05.513778+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:35.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"93'4","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915938+0000","last_change":"2024-09-10T08:05:22.567077+0000","last_active":"2024-09-10T08:07:05.915938+0000","last_peered":"2024-09-10T08:07:05.915938+0000","last_clean":"2024-09-10T08:07:05.915938+0000","last_became_active":"2024-09-10T08:05:22.566757+0000","last_became_peered":"2024-09-10T08:05:22.566757+0000","last_unstale":"2024-09-10T08:07:05.915938+0000","last_undegraded":"2024-09-10T08:07:05.915938+0000","last_fullsized":"2024-09-10T08:07:05.915938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:06.881510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511464+0000","last_change":"2024-09-10T08:05:35.519539+0000","last_active":"2024-09-10T08:07:05.511464+0000","last_peered":"2024-09-10T08:07:05.511464+0000","last_clean":"2024-09-10T08:07:05.511464+0000","last_became_active":"2024-09-10T08:05:35.519044+0000","last_became_peered":"2024-09-10T08:05:35.519044+0000","last_unstale":"2024-09-10T08:07:05.511464+0000","last_undegraded":"2024-09-10T08:07:05.511464+0000","last_fullsized":"2024-09-10T08:07:05.511464+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:10.546949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.772349+0000","last_change":"2024-09-10T08:07:05.771774+0000","last_active":"2024-09-10T08:07:05.772349+0000","last_peered":"2024-09-10T08:07:05.772349+0000","last_clean":"2024-09-10T08:07:05.772349+0000","last_became_active":"2024-09-10T08:07:05.771336+0000","last_became_peered":"2024-09-10T08:07:05.771336+0000","last_unstale":"2024-09-10T08:07:05.772349+0000","last_undegraded":"2024-09-10T08:07:05.772349+0000","last_fullsized":"2024-09-10T08:07:05.772349+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:11.482222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"93'11","reported_seq":64,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513571+0000","last_change":"2024-09-10T08:06:04.741199+0000","last_active":"2024-09-10T08:07:05.513571+0000","last_peered":"2024-09-10T08:07:05.513571+0000","last_clean":"2024-09-10T08:07:05.513571+0000","last_became_active":"2024-09-10T08:06:04.738076+0000","last_became_peered":"2024-09-10T08:06:04.738076+0000","last_unstale":"2024-09-10T08:07:05.513571+0000","last_undegraded":"2024-09-10T08:07:05.513571+0000","last_fullsized":"2024-09-10T08:07:05.513571+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:45.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876403+0000","last_change":"2024-09-10T08:05:35.642358+0000","last_active":"2024-09-10T08:07:05.876403+0000","last_peered":"2024-09-10T08:07:05.876403+0000","last_clean":"2024-09-10T08:07:05.876403+0000","last_became_active":"2024-09-10T08:05:35.641408+0000","last_became_peered":"2024-09-10T08:05:35.641408+0000","last_unstale":"2024-09-10T08:07:05.876403+0000","last_undegraded":"2024-09-10T08:07:05.876403+0000","last_fullsized":"2024-09-10T08:07:05.876403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:42.015139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914499+0000","last_change":"2024-09-10T08:05:25.486310+0000","last_active":"2024-09-10T08:07:05.914499+0000","last_peered":"2024-09-10T08:07:05.914499+0000","last_clean":"2024-09-10T08:07:05.914499+0000","last_became_active":"2024-09-10T08:05:25.485915+0000","last_became_peered":"2024-09-10T08:05:25.485915+0000","last_unstale":"2024-09-10T08:07:05.914499+0000","last_undegraded":"2024-09-10T08:07:05.914499+0000","last_fullsized":"2024-09-10T08:07:05.914499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:56.064361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"93'1","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508937+0000","last_change":"2024-09-10T08:05:22.570595+0000","last_active":"2024-09-10T08:07:05.508937+0000","last_peered":"2024-09-10T08:07:05.508937+0000","last_clean":"2024-09-10T08:07:05.508937+0000","last_became_active":"2024-09-10T08:05:22.570219+0000","last_became_peered":"2024-09-10T08:05:22.570219+0000","last_unstale":"2024-09-10T08:07:05.508937+0000","last_undegraded":"2024-09-10T08:07:05.508937+0000","last_fullsized":"2024-09-10T08:07:05.508937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:03.230325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769829+0000","last_change":"2024-09-10T08:05:35.530472+0000","last_active":"2024-09-10T08:07:05.769829+0000","last_peered":"2024-09-10T08:07:05.769829+0000","last_clean":"2024-09-10T08:07:05.769829+0000","last_became_active":"2024-09-10T08:05:35.530066+0000","last_became_peered":"2024-09-10T08:05:35.530066+0000","last_unstale":"2024-09-10T08:07:05.769829+0000","last_undegraded":"2024-09-10T08:07:05.769829+0000","last_fullsized":"2024-09-10T08:07:05.769829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:58.915767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914775+0000","last_change":"2024-09-10T08:05:25.500524+0000","last_active":"2024-09-10T08:07:05.914775+0000","last_peered":"2024-09-10T08:07:05.914775+0000","last_clean":"2024-09-10T08:07:05.914775+0000","last_became_active":"2024-09-10T08:05:25.500186+0000","last_became_peered":"2024-09-10T08:05:25.500186+0000","last_unstale":"2024-09-10T08:07:05.914775+0000","last_undegraded":"2024-09-10T08:07:05.914775+0000","last_fullsized":"2024-09-10T08:07:05.914775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:51.936222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513172+0000","last_change":"2024-09-10T08:05:22.569326+0000","last_active":"2024-09-10T08:07:05.513172+0000","last_peered":"2024-09-10T08:07:05.513172+0000","last_clean":"2024-09-10T08:07:05.513172+0000","last_became_active":"2024-09-10T08:05:22.568895+0000","last_became_peered":"2024-09-10T08:05:22.568895+0000","last_unstale":"2024-09-10T08:07:05.513172+0000","last_undegraded":"2024-09-10T08:07:05.513172+0000","last_fullsized":"2024-09-10T08:07:05.513172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:47.212073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508317+0000","last_change":"2024-09-10T08:05:35.641600+0000","last_active":"2024-09-10T08:07:05.508317+0000","last_peered":"2024-09-10T08:07:05.508317+0000","last_clean":"2024-09-10T08:07:05.508317+0000","last_became_active":"2024-09-10T08:05:35.641205+0000","last_became_peered":"2024-09-10T08:05:35.641205+0000","last_unstale":"2024-09-10T08:07:05.508317+0000","last_undegraded":"2024-09-10T08:07:05.508317+0000","last_fullsized":"2024-09-10T08:07:05.508317+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:40.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915501+0000","last_change":"2024-09-10T08:05:25.495035+0000","last_active":"2024-09-10T08:07:05.915501+0000","last_peered":"2024-09-10T08:07:05.915501+0000","last_clean":"2024-09-10T08:07:05.915501+0000","last_became_active":"2024-09-10T08:05:25.494733+0000","last_became_peered":"2024-09-10T08:05:25.494733+0000","last_unstale":"2024-09-10T08:07:05.915501+0000","last_undegraded":"2024-09-10T08:07:05.915501+0000","last_fullsized":"2024-09-10T08:07:05.915501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.875867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"93'1","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509414+0000","last_change":"2024-09-10T08:05:22.458727+0000","last_active":"2024-09-10T08:07:05.509414+0000","last_peered":"2024-09-10T08:07:05.509414+0000","last_clean":"2024-09-10T08:07:05.509414+0000","last_became_active":"2024-09-10T08:05:22.458397+0000","last_became_peered":"2024-09-10T08:05:22.458397+0000","last_unstale":"2024-09-10T08:07:05.509414+0000","last_undegraded":"2024-09-10T08:07:05.509414+0000","last_fullsized":"2024-09-10T08:07:05.509414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:07.875559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914333+0000","last_change":"2024-09-10T08:05:35.534192+0000","last_active":"2024-09-10T08:07:05.914333+0000","last_peered":"2024-09-10T08:07:05.914333+0000","last_clean":"2024-09-10T08:07:05.914333+0000","last_became_active":"2024-09-10T08:05:35.530450+0000","last_became_peered":"2024-09-10T08:05:35.530450+0000","last_unstale":"2024-09-10T08:07:05.914333+0000","last_undegraded":"2024-09-10T08:07:05.914333+0000","last_fullsized":"2024-09-10T08:07:05.914333+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:10.039144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568765+0000","last_change":"2024-09-10T08:05:25.482252+0000","last_active":"2024-09-10T08:07:05.568765+0000","last_peered":"2024-09-10T08:07:05.568765+0000","last_clean":"2024-09-10T08:07:05.568765+0000","last_became_active":"2024-09-10T08:05:25.481901+0000","last_became_peered":"2024-09-10T08:05:25.481901+0000","last_unstale":"2024-09-10T08:07:05.568765+0000","last_undegraded":"2024-09-10T08:07:05.568765+0000","last_fullsized":"2024-09-10T08:07:05.568765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:17.063340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'4","reported_seq":114,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:27.362804+0000","last_change":"2024-09-10T08:05:22.569199+0000","last_active":"2024-09-10T08:07:27.362804+0000","last_peered":"2024-09-10T08:07:27.362804+0000","last_clean":"2024-09-10T08:07:27.362804+0000","last_became_active":"2024-09-10T08:05:22.568787+0000","last_became_peered":"2024-09-10T08:05:22.568787+0000","last_unstale":"2024-09-10T08:07:27.362804+0000","last_undegraded":"2024-09-10T08:07:27.362804+0000","last_fullsized":"2024-09-10T08:07:27.362804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:59.610130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914604+0000","last_change":"2024-09-10T08:05:35.642446+0000","last_active":"2024-09-10T08:07:05.914604+0000","last_peered":"2024-09-10T08:07:05.914604+0000","last_clean":"2024-09-10T08:07:05.914604+0000","last_became_active":"2024-09-10T08:05:35.641543+0000","last_became_peered":"2024-09-10T08:05:35.641543+0000","last_unstale":"2024-09-10T08:07:05.914604+0000","last_undegraded":"2024-09-10T08:07:05.914604+0000","last_fullsized":"2024-09-10T08:07:05.914604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:33.658406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.913993+0000","last_change":"2024-09-10T08:05:25.487598+0000","last_active":"2024-09-10T08:07:05.913993+0000","last_peered":"2024-09-10T08:07:05.913993+0000","last_clean":"2024-09-10T08:07:05.913993+0000","last_became_active":"2024-09-10T08:05:25.486879+0000","last_became_peered":"2024-09-10T08:05:25.486879+0000","last_unstale":"2024-09-10T08:07:05.913993+0000","last_undegraded":"2024-09-10T08:07:05.913993+0000","last_fullsized":"2024-09-10T08:07:05.913993+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:20.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509934+0000","last_change":"2024-09-10T08:05:22.463416+0000","last_active":"2024-09-10T08:07:05.509934+0000","last_peered":"2024-09-10T08:07:05.509934+0000","last_clean":"2024-09-10T08:07:05.509934+0000","last_became_active":"2024-09-10T08:05:22.463004+0000","last_became_peered":"2024-09-10T08:05:22.463004+0000","last_unstale":"2024-09-10T08:07:05.509934+0000","last_undegraded":"2024-09-10T08:07:05.509934+0000","last_fullsized":"2024-09-10T08:07:05.509934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:56.257812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569233+0000","last_change":"2024-09-10T08:05:35.642059+0000","last_active":"2024-09-10T08:07:05.569233+0000","last_peered":"2024-09-10T08:07:05.569233+0000","last_clean":"2024-09-10T08:07:05.569233+0000","last_became_active":"2024-09-10T08:05:35.641540+0000","last_became_peered":"2024-09-10T08:05:35.641540+0000","last_unstale":"2024-09-10T08:07:05.569233+0000","last_undegraded":"2024-09-10T08:07:05.569233+0000","last_fullsized":"2024-09-10T08:07:05.569233+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:28.557584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876775+0000","last_change":"2024-09-10T08:05:25.496111+0000","last_active":"2024-09-10T08:07:05.876775+0000","last_peered":"2024-09-10T08:07:05.876775+0000","last_clean":"2024-09-10T08:07:05.876775+0000","last_became_active":"2024-09-10T08:05:25.495757+0000","last_became_peered":"2024-09-10T08:05:25.495757+0000","last_unstale":"2024-09-10T08:07:05.876775+0000","last_undegraded":"2024-09-10T08:07:05.876775+0000","last_fullsized":"2024-09-10T08:07:05.876775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:41.078205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'3","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915005+0000","last_change":"2024-09-10T08:05:22.460069+0000","last_active":"2024-09-10T08:07:05.915005+0000","last_peered":"2024-09-10T08:07:05.915005+0000","last_clean":"2024-09-10T08:07:05.915005+0000","last_became_active":"2024-09-10T08:05:22.459557+0000","last_became_peered":"2024-09-10T08:05:22.459557+0000","last_unstale":"2024-09-10T08:07:05.915005+0000","last_undegraded":"2024-09-10T08:07:05.915005+0000","last_fullsized":"2024-09-10T08:07:05.915005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:49.058507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509959+0000","last_change":"2024-09-10T08:04:08.188262+0000","last_active":"2024-09-10T08:07:05.509959+0000","last_peered":"2024-09-10T08:07:05.509959+0000","last_clean":"2024-09-10T08:07:05.509959+0000","last_became_active":"2024-09-10T08:04:08.187691+0000","last_became_peered":"2024-09-10T08:04:08.187691+0000","last_unstale":"2024-09-10T08:07:05.509959+0000","last_undegraded":"2024-09-10T08:07:05.509959+0000","last_fullsized":"2024-09-10T08:07:05.509959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_clean_scrub_stamp":"2024-09-10T08:00:26.235127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:15.358818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509459+0000","last_change":"2024-09-10T08:05:35.528371+0000","last_active":"2024-09-10T08:07:05.509459+0000","last_peered":"2024-09-10T08:07:05.509459+0000","last_clean":"2024-09-10T08:07:05.509459+0000","last_became_active":"2024-09-10T08:05:35.527925+0000","last_became_peered":"2024-09-10T08:05:35.527925+0000","last_unstale":"2024-09-10T08:07:05.509459+0000","last_undegraded":"2024-09-10T08:07:05.509459+0000","last_fullsized":"2024-09-10T08:07:05.509459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:37:00.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509689+0000","last_change":"2024-09-10T08:05:25.490753+0000","last_active":"2024-09-10T08:07:05.509689+0000","last_peered":"2024-09-10T08:07:05.509689+0000","last_clean":"2024-09-10T08:07:05.509689+0000","last_became_active":"2024-09-10T08:05:25.489996+0000","last_became_peered":"2024-09-10T08:05:25.489996+0000","last_unstale":"2024-09-10T08:07:05.509689+0000","last_undegraded":"2024-09-10T08:07:05.509689+0000","last_fullsized":"2024-09-10T08:07:05.509689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:15.637199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511724+0000","last_change":"2024-09-10T08:05:22.568696+0000","last_active":"2024-09-10T08:07:05.511724+0000","last_peered":"2024-09-10T08:07:05.511724+0000","last_clean":"2024-09-10T08:07:05.511724+0000","last_became_active":"2024-09-10T08:05:22.568331+0000","last_became_peered":"2024-09-10T08:05:22.568331+0000","last_unstale":"2024-09-10T08:07:05.511724+0000","last_undegraded":"2024-09-10T08:07:05.511724+0000","last_fullsized":"2024-09-10T08:07:05.511724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:21.699397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914379+0000","last_change":"2024-09-10T08:05:35.530804+0000","last_active":"2024-09-10T08:07:05.914379+0000","last_peered":"2024-09-10T08:07:05.914379+0000","last_clean":"2024-09-10T08:07:05.914379+0000","last_became_active":"2024-09-10T08:05:35.530356+0000","last_became_peered":"2024-09-10T08:05:35.530356+0000","last_unstale":"2024-09-10T08:07:05.914379+0000","last_undegraded":"2024-09-10T08:07:05.914379+0000","last_fullsized":"2024-09-10T08:07:05.914379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:59.318761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509566+0000","last_change":"2024-09-10T08:05:25.489546+0000","last_active":"2024-09-10T08:07:05.509566+0000","last_peered":"2024-09-10T08:07:05.509566+0000","last_clean":"2024-09-10T08:07:05.509566+0000","last_became_active":"2024-09-10T08:05:25.489188+0000","last_became_peered":"2024-09-10T08:05:25.489188+0000","last_unstale":"2024-09-10T08:07:05.509566+0000","last_undegraded":"2024-09-10T08:07:05.509566+0000","last_fullsized":"2024-09-10T08:07:05.509566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:26.471681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569669+0000","last_change":"2024-09-10T08:05:22.463912+0000","last_active":"2024-09-10T08:07:05.569669+0000","last_peered":"2024-09-10T08:07:05.569669+0000","last_clean":"2024-09-10T08:07:05.569669+0000","last_became_active":"2024-09-10T08:05:22.463561+0000","last_became_peered":"2024-09-10T08:05:22.463561+0000","last_unstale":"2024-09-10T08:07:05.569669+0000","last_undegraded":"2024-09-10T08:07:05.569669+0000","last_fullsized":"2024-09-10T08:07:05.569669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:22.407604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514223+0000","last_change":"2024-09-10T08:05:35.528795+0000","last_active":"2024-09-10T08:07:05.514223+0000","last_peered":"2024-09-10T08:07:05.514223+0000","last_clean":"2024-09-10T08:07:05.514223+0000","last_became_active":"2024-09-10T08:05:35.528402+0000","last_became_peered":"2024-09-10T08:05:35.528402+0000","last_unstale":"2024-09-10T08:07:05.514223+0000","last_undegraded":"2024-09-10T08:07:05.514223+0000","last_fullsized":"2024-09-10T08:07:05.514223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:14.606371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509938+0000","last_change":"2024-09-10T08:05:25.484086+0000","last_active":"2024-09-10T08:07:05.509938+0000","last_peered":"2024-09-10T08:07:05.509938+0000","last_clean":"2024-09-10T08:07:05.509938+0000","last_became_active":"2024-09-10T08:05:25.483757+0000","last_became_peered":"2024-09-10T08:05:25.483757+0000","last_unstale":"2024-09-10T08:07:05.509938+0000","last_undegraded":"2024-09-10T08:07:05.509938+0000","last_fullsized":"2024-09-10T08:07:05.509938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:16:16.362739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914631+0000","last_change":"2024-09-10T08:05:22.462220+0000","last_active":"2024-09-10T08:07:05.914631+0000","last_peered":"2024-09-10T08:07:05.914631+0000","last_clean":"2024-09-10T08:07:05.914631+0000","last_became_active":"2024-09-10T08:05:22.461726+0000","last_became_peered":"2024-09-10T08:05:22.461726+0000","last_unstale":"2024-09-10T08:07:05.914631+0000","last_undegraded":"2024-09-10T08:07:05.914631+0000","last_fullsized":"2024-09-10T08:07:05.914631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:56.381034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508201+0000","last_change":"2024-09-10T08:06:04.734827+0000","last_active":"2024-09-10T08:07:05.508201+0000","last_peered":"2024-09-10T08:07:05.508201+0000","last_clean":"2024-09-10T08:07:05.508201+0000","last_became_active":"2024-09-10T08:06:04.734402+0000","last_became_peered":"2024-09-10T08:06:04.734402+0000","last_unstale":"2024-09-10T08:07:05.508201+0000","last_undegraded":"2024-09-10T08:07:05.508201+0000","last_fullsized":"2024-09-10T08:07:05.508201+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:24.806538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770235+0000","last_change":"2024-09-10T08:05:25.486100+0000","last_active":"2024-09-10T08:07:05.770235+0000","last_peered":"2024-09-10T08:07:05.770235+0000","last_clean":"2024-09-10T08:07:05.770235+0000","last_became_active":"2024-09-10T08:05:25.485586+0000","last_became_peered":"2024-09-10T08:05:25.485586+0000","last_unstale":"2024-09-10T08:07:05.770235+0000","last_undegraded":"2024-09-10T08:07:05.770235+0000","last_fullsized":"2024-09-10T08:07:05.770235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:33.565054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"93'2","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770365+0000","last_change":"2024-09-10T08:06:04.743408+0000","last_active":"2024-09-10T08:07:05.770365+0000","last_peered":"2024-09-10T08:07:05.770365+0000","last_clean":"2024-09-10T08:07:05.770365+0000","last_became_active":"2024-09-10T08:06:04.732830+0000","last_became_peered":"2024-09-10T08:06:04.732830+0000","last_unstale":"2024-09-10T08:07:05.770365+0000","last_undegraded":"2024-09-10T08:07:05.770365+0000","last_fullsized":"2024-09-10T08:07:05.770365+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:29.381259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568797+0000","last_change":"2024-09-10T08:05:35.533015+0000","last_active":"2024-09-10T08:07:05.568797+0000","last_peered":"2024-09-10T08:07:05.568797+0000","last_clean":"2024-09-10T08:07:05.568797+0000","last_became_active":"2024-09-10T08:05:35.532590+0000","last_became_peered":"2024-09-10T08:05:35.532590+0000","last_unstale":"2024-09-10T08:07:05.568797+0000","last_undegraded":"2024-09-10T08:07:05.568797+0000","last_fullsized":"2024-09-10T08:07:05.568797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:23.984273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568914+0000","last_change":"2024-09-10T08:05:25.489994+0000","last_active":"2024-09-10T08:07:05.568914+0000","last_peered":"2024-09-10T08:07:05.568914+0000","last_clean":"2024-09-10T08:07:05.568914+0000","last_became_active":"2024-09-10T08:05:25.489471+0000","last_became_peered":"2024-09-10T08:05:25.489471+0000","last_unstale":"2024-09-10T08:07:05.568914+0000","last_undegraded":"2024-09-10T08:07:05.568914+0000","last_fullsized":"2024-09-10T08:07:05.568914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:53.866570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"93'15","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508330+0000","last_change":"2024-09-10T08:05:22.568900+0000","last_active":"2024-09-10T08:07:05.508330+0000","last_peered":"2024-09-10T08:07:05.508330+0000","last_clean":"2024-09-10T08:07:05.508330+0000","last_became_active":"2024-09-10T08:05:22.568542+0000","last_became_peered":"2024-09-10T08:05:22.568542+0000","last_unstale":"2024-09-10T08:07:05.508330+0000","last_undegraded":"2024-09-10T08:07:05.508330+0000","last_fullsized":"2024-09-10T08:07:05.508330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:58:45.669769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914605+0000","last_change":"2024-09-10T08:05:35.537364+0000","last_active":"2024-09-10T08:07:05.914605+0000","last_peered":"2024-09-10T08:07:05.914605+0000","last_clean":"2024-09-10T08:07:05.914605+0000","last_became_active":"2024-09-10T08:05:35.535958+0000","last_became_peered":"2024-09-10T08:05:35.535958+0000","last_unstale":"2024-09-10T08:07:05.914605+0000","last_undegraded":"2024-09-10T08:07:05.914605+0000","last_fullsized":"2024-09-10T08:07:05.914605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:51.443798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.917030+0000","last_change":"2024-09-10T08:07:05.917030+0000","last_active":"2024-09-10T08:07:05.917030+0000","last_peered":"2024-09-10T08:07:05.917030+0000","last_clean":"2024-09-10T08:07:05.917030+0000","last_became_active":"2024-09-10T08:07:05.916281+0000","last_became_peered":"2024-09-10T08:07:05.916281+0000","last_unstale":"2024-09-10T08:07:05.917030+0000","last_undegraded":"2024-09-10T08:07:05.917030+0000","last_fullsized":"2024-09-10T08:07:05.917030+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:15.060009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'3","reported_seq":113,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:25.360466+0000","last_change":"2024-09-10T08:05:22.459963+0000","last_active":"2024-09-10T08:07:25.360466+0000","last_peered":"2024-09-10T08:07:25.360466+0000","last_clean":"2024-09-10T08:07:25.360466+0000","last_became_active":"2024-09-10T08:05:22.459497+0000","last_became_peered":"2024-09-10T08:05:22.459497+0000","last_unstale":"2024-09-10T08:07:25.360466+0000","last_undegraded":"2024-09-10T08:07:25.360466+0000","last_fullsized":"2024-09-10T08:07:25.360466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:29.887980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513941+0000","last_change":"2024-09-10T08:05:35.534570+0000","last_active":"2024-09-10T08:07:05.513941+0000","last_peered":"2024-09-10T08:07:05.513941+0000","last_clean":"2024-09-10T08:07:05.513941+0000","last_became_active":"2024-09-10T08:05:35.534195+0000","last_became_peered":"2024-09-10T08:05:35.534195+0000","last_unstale":"2024-09-10T08:07:05.513941+0000","last_undegraded":"2024-09-10T08:07:05.513941+0000","last_fullsized":"2024-09-10T08:07:05.513941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:17.393491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509832+0000","last_change":"2024-09-10T08:05:25.488551+0000","last_active":"2024-09-10T08:07:05.509832+0000","last_peered":"2024-09-10T08:07:05.509832+0000","last_clean":"2024-09-10T08:07:05.509832+0000","last_became_active":"2024-09-10T08:05:25.488086+0000","last_became_peered":"2024-09-10T08:05:25.488086+0000","last_unstale":"2024-09-10T08:07:05.509832+0000","last_undegraded":"2024-09-10T08:07:05.509832+0000","last_fullsized":"2024-09-10T08:07:05.509832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:38.044721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"93'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568982+0000","last_change":"2024-09-10T08:05:22.462269+0000","last_active":"2024-09-10T08:07:05.568982+0000","last_peered":"2024-09-10T08:07:05.568982+0000","last_clean":"2024-09-10T08:07:05.568982+0000","last_became_active":"2024-09-10T08:05:22.461924+0000","last_became_peered":"2024-09-10T08:05:22.461924+0000","last_unstale":"2024-09-10T08:07:05.568982+0000","last_undegraded":"2024-09-10T08:07:05.568982+0000","last_fullsized":"2024-09-10T08:07:05.568982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:54.996777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769980+0000","last_change":"2024-09-10T08:05:35.531596+0000","last_active":"2024-09-10T08:07:05.769980+0000","last_peered":"2024-09-10T08:07:05.769980+0000","last_clean":"2024-09-10T08:07:05.769980+0000","last_became_active":"2024-09-10T08:05:35.530179+0000","last_became_peered":"2024-09-10T08:05:35.530179+0000","last_unstale":"2024-09-10T08:07:05.769980+0000","last_undegraded":"2024-09-10T08:07:05.769980+0000","last_fullsized":"2024-09-10T08:07:05.769980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:32.056426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914190+0000","last_change":"2024-09-10T08:05:25.503504+0000","last_active":"2024-09-10T08:07:05.914190+0000","last_peered":"2024-09-10T08:07:05.914190+0000","last_clean":"2024-09-10T08:07:05.914190+0000","last_became_active":"2024-09-10T08:05:25.503105+0000","last_became_peered":"2024-09-10T08:05:25.503105+0000","last_unstale":"2024-09-10T08:07:05.914190+0000","last_undegraded":"2024-09-10T08:07:05.914190+0000","last_fullsized":"2024-09-10T08:07:05.914190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:40.885654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"93'6","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511569+0000","last_change":"2024-09-10T08:05:22.462271+0000","last_active":"2024-09-10T08:07:05.511569+0000","last_peered":"2024-09-10T08:07:05.511569+0000","last_clean":"2024-09-10T08:07:05.511569+0000","last_became_active":"2024-09-10T08:05:22.461195+0000","last_became_peered":"2024-09-10T08:05:22.461195+0000","last_unstale":"2024-09-10T08:07:05.511569+0000","last_undegraded":"2024-09-10T08:07:05.511569+0000","last_fullsized":"2024-09-10T08:07:05.511569+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:20.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569404+0000","last_change":"2024-09-10T08:05:35.533218+0000","last_active":"2024-09-10T08:07:05.569404+0000","last_peered":"2024-09-10T08:07:05.569404+0000","last_clean":"2024-09-10T08:07:05.569404+0000","last_became_active":"2024-09-10T08:05:35.532706+0000","last_became_peered":"2024-09-10T08:05:35.532706+0000","last_unstale":"2024-09-10T08:07:05.569404+0000","last_undegraded":"2024-09-10T08:07:05.569404+0000","last_fullsized":"2024-09-10T08:07:05.569404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:22.150563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508629+0000","last_change":"2024-09-10T08:05:25.478978+0000","last_active":"2024-09-10T08:07:05.508629+0000","last_peered":"2024-09-10T08:07:05.508629+0000","last_clean":"2024-09-10T08:07:05.508629+0000","last_became_active":"2024-09-10T08:05:25.478649+0000","last_became_peered":"2024-09-10T08:05:25.478649+0000","last_unstale":"2024-09-10T08:07:05.508629+0000","last_undegraded":"2024-09-10T08:07:05.508629+0000","last_fullsized":"2024-09-10T08:07:05.508629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:17.493446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'32","reported_seq":227,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:25.360377+0000","last_change":"2024-09-10T08:06:04.740972+0000","last_active":"2024-09-10T08:07:25.360377+0000","last_peered":"2024-09-10T08:07:25.360377+0000","last_clean":"2024-09-10T08:07:25.360377+0000","last_became_active":"2024-09-10T08:06:04.731200+0000","last_became_peered":"2024-09-10T08:06:04.731200+0000","last_unstale":"2024-09-10T08:07:25.360377+0000","last_undegraded":"2024-09-10T08:07:25.360377+0000","last_fullsized":"2024-09-10T08:07:25.360377+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:17.169708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62661,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":136,"num_read_kb":129,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3703,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512731+0000","last_change":"2024-09-10T08:05:35.532594+0000","last_active":"2024-09-10T08:07:05.512731+0000","last_peered":"2024-09-10T08:07:05.512731+0000","last_clean":"2024-09-10T08:07:05.512731+0000","last_became_active":"2024-09-10T08:05:35.532132+0000","last_became_peered":"2024-09-10T08:05:35.532132+0000","last_unstale":"2024-09-10T08:07:05.512731+0000","last_undegraded":"2024-09-10T08:07:05.512731+0000","last_fullsized":"2024-09-10T08:07:05.512731+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:15.348648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914930+0000","last_change":"2024-09-10T08:05:25.485297+0000","last_active":"2024-09-10T08:07:05.914930+0000","last_peered":"2024-09-10T08:07:05.914930+0000","last_clean":"2024-09-10T08:07:05.914930+0000","last_became_active":"2024-09-10T08:05:25.484883+0000","last_became_peered":"2024-09-10T08:05:25.484883+0000","last_unstale":"2024-09-10T08:07:05.914930+0000","last_undegraded":"2024-09-10T08:07:05.914930+0000","last_fullsized":"2024-09-10T08:07:05.914930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:37.052435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"93'4","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569659+0000","last_change":"2024-09-10T08:05:22.568993+0000","last_active":"2024-09-10T08:07:05.569659+0000","last_peered":"2024-09-10T08:07:05.569659+0000","last_clean":"2024-09-10T08:07:05.569659+0000","last_became_active":"2024-09-10T08:05:22.568564+0000","last_became_peered":"2024-09-10T08:05:22.568564+0000","last_unstale":"2024-09-10T08:07:05.569659+0000","last_undegraded":"2024-09-10T08:07:05.569659+0000","last_fullsized":"2024-09-10T08:07:05.569659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:23.767174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914503+0000","last_change":"2024-09-10T08:05:35.524025+0000","last_active":"2024-09-10T08:07:05.914503+0000","last_peered":"2024-09-10T08:07:05.914503+0000","last_clean":"2024-09-10T08:07:05.914503+0000","last_became_active":"2024-09-10T08:05:35.523628+0000","last_became_peered":"2024-09-10T08:05:35.523628+0000","last_unstale":"2024-09-10T08:07:05.914503+0000","last_undegraded":"2024-09-10T08:07:05.914503+0000","last_fullsized":"2024-09-10T08:07:05.914503+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:48.593779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.570537+0000","last_change":"2024-09-10T08:07:05.570537+0000","last_active":"2024-09-10T08:07:05.570537+0000","last_peered":"2024-09-10T08:07:05.570537+0000","last_clean":"2024-09-10T08:07:05.570537+0000","last_became_active":"2024-09-10T08:07:05.570278+0000","last_became_peered":"2024-09-10T08:07:05.570278+0000","last_unstale":"2024-09-10T08:07:05.570537+0000","last_undegraded":"2024-09-10T08:07:05.570537+0000","last_fullsized":"2024-09-10T08:07:05.570537+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:56.490462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"93'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509561+0000","last_change":"2024-09-10T08:05:22.460487+0000","last_active":"2024-09-10T08:07:05.509561+0000","last_peered":"2024-09-10T08:07:05.509561+0000","last_clean":"2024-09-10T08:07:05.509561+0000","last_became_active":"2024-09-10T08:05:22.460158+0000","last_became_peered":"2024-09-10T08:05:22.460158+0000","last_unstale":"2024-09-10T08:07:05.509561+0000","last_undegraded":"2024-09-10T08:07:05.509561+0000","last_fullsized":"2024-09-10T08:07:05.509561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:53.285513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512437+0000","last_change":"2024-09-10T08:05:35.531399+0000","last_active":"2024-09-10T08:07:05.512437+0000","last_peered":"2024-09-10T08:07:05.512437+0000","last_clean":"2024-09-10T08:07:05.512437+0000","last_became_active":"2024-09-10T08:05:35.530320+0000","last_became_peered":"2024-09-10T08:05:35.530320+0000","last_unstale":"2024-09-10T08:07:05.512437+0000","last_undegraded":"2024-09-10T08:07:05.512437+0000","last_fullsized":"2024-09-10T08:07:05.512437+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:19.814046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915161+0000","last_change":"2024-09-10T08:05:25.488208+0000","last_active":"2024-09-10T08:07:05.915161+0000","last_peered":"2024-09-10T08:07:05.915161+0000","last_clean":"2024-09-10T08:07:05.915161+0000","last_became_active":"2024-09-10T08:05:25.487893+0000","last_became_peered":"2024-09-10T08:05:25.487893+0000","last_unstale":"2024-09-10T08:07:05.915161+0000","last_undegraded":"2024-09-10T08:07:05.915161+0000","last_fullsized":"2024-09-10T08:07:05.915161+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:10.999416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"93'3","reported_seq":110,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:24.359029+0000","last_change":"2024-09-10T08:05:22.461321+0000","last_active":"2024-09-10T08:07:24.359029+0000","last_peered":"2024-09-10T08:07:24.359029+0000","last_clean":"2024-09-10T08:07:24.359029+0000","last_became_active":"2024-09-10T08:05:22.460950+0000","last_became_peered":"2024-09-10T08:05:22.460950+0000","last_unstale":"2024-09-10T08:07:24.359029+0000","last_undegraded":"2024-09-10T08:07:24.359029+0000","last_fullsized":"2024-09-10T08:07:24.359029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:55.647273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876486+0000","last_change":"2024-09-10T08:05:35.530578+0000","last_active":"2024-09-10T08:07:05.876486+0000","last_peered":"2024-09-10T08:07:05.876486+0000","last_clean":"2024-09-10T08:07:05.876486+0000","last_became_active":"2024-09-10T08:05:35.530162+0000","last_became_peered":"2024-09-10T08:05:35.530162+0000","last_unstale":"2024-09-10T08:07:05.876486+0000","last_undegraded":"2024-09-10T08:07:05.876486+0000","last_fullsized":"2024-09-10T08:07:05.876486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:19.290973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513556+0000","last_change":"2024-09-10T08:05:35.532685+0000","last_active":"2024-09-10T08:07:05.513556+0000","last_peered":"2024-09-10T08:07:05.513556+0000","last_clean":"2024-09-10T08:07:05.513556+0000","last_became_active":"2024-09-10T08:05:35.532257+0000","last_became_peered":"2024-09-10T08:05:35.532257+0000","last_unstale":"2024-09-10T08:07:05.513556+0000","last_undegraded":"2024-09-10T08:07:05.513556+0000","last_fullsized":"2024-09-10T08:07:05.513556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:07.412435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512520+0000","last_change":"2024-09-10T08:05:25.494472+0000","last_active":"2024-09-10T08:07:05.512520+0000","last_peered":"2024-09-10T08:07:05.512520+0000","last_clean":"2024-09-10T08:07:05.512520+0000","last_became_active":"2024-09-10T08:05:25.493928+0000","last_became_peered":"2024-09-10T08:05:25.493928+0000","last_unstale":"2024-09-10T08:07:05.512520+0000","last_undegraded":"2024-09-10T08:07:05.512520+0000","last_fullsized":"2024-09-10T08:07:05.512520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:00.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"93'1","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568866+0000","last_change":"2024-09-10T08:05:22.460585+0000","last_active":"2024-09-10T08:07:05.568866+0000","last_peered":"2024-09-10T08:07:05.568866+0000","last_clean":"2024-09-10T08:07:05.568866+0000","last_became_active":"2024-09-10T08:05:22.460259+0000","last_became_peered":"2024-09-10T08:05:22.460259+0000","last_unstale":"2024-09-10T08:07:05.568866+0000","last_undegraded":"2024-09-10T08:07:05.568866+0000","last_fullsized":"2024-09-10T08:07:05.568866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:45.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770172+0000","last_change":"2024-09-10T08:05:35.524000+0000","last_active":"2024-09-10T08:07:05.770172+0000","last_peered":"2024-09-10T08:07:05.770172+0000","last_clean":"2024-09-10T08:07:05.770172+0000","last_became_active":"2024-09-10T08:05:35.523647+0000","last_became_peered":"2024-09-10T08:05:35.523647+0000","last_unstale":"2024-09-10T08:07:05.770172+0000","last_undegraded":"2024-09-10T08:07:05.770172+0000","last_fullsized":"2024-09-10T08:07:05.770172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:10:54.664865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876547+0000","last_change":"2024-09-10T08:05:25.499701+0000","last_active":"2024-09-10T08:07:05.876547+0000","last_peered":"2024-09-10T08:07:05.876547+0000","last_clean":"2024-09-10T08:07:05.876547+0000","last_became_active":"2024-09-10T08:05:25.496890+0000","last_became_peered":"2024-09-10T08:05:25.496890+0000","last_unstale":"2024-09-10T08:07:05.876547+0000","last_undegraded":"2024-09-10T08:07:05.876547+0000","last_fullsized":"2024-09-10T08:07:05.876547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:58.166365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"93'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914017+0000","last_change":"2024-09-10T08:05:22.460880+0000","last_active":"2024-09-10T08:07:05.914017+0000","last_peered":"2024-09-10T08:07:05.914017+0000","last_clean":"2024-09-10T08:07:05.914017+0000","last_became_active":"2024-09-10T08:05:22.459932+0000","last_became_peered":"2024-09-10T08:05:22.459932+0000","last_unstale":"2024-09-10T08:07:05.914017+0000","last_undegraded":"2024-09-10T08:07:05.914017+0000","last_fullsized":"2024-09-10T08:07:05.914017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:55.826526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"93'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915361+0000","last_change":"2024-09-10T08:05:22.460578+0000","last_active":"2024-09-10T08:07:05.915361+0000","last_peered":"2024-09-10T08:07:05.915361+0000","last_clean":"2024-09-10T08:07:05.915361+0000","last_became_active":"2024-09-10T08:05:22.460239+0000","last_became_peered":"2024-09-10T08:05:22.460239+0000","last_unstale":"2024-09-10T08:07:05.915361+0000","last_undegraded":"2024-09-10T08:07:05.915361+0000","last_fullsized":"2024-09-10T08:07:05.915361+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:07.606243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569247+0000","last_change":"2024-09-10T08:05:25.489846+0000","last_active":"2024-09-10T08:07:05.569247+0000","last_peered":"2024-09-10T08:07:05.569247+0000","last_clean":"2024-09-10T08:07:05.569247+0000","last_became_active":"2024-09-10T08:05:25.489326+0000","last_became_peered":"2024-09-10T08:05:25.489326+0000","last_unstale":"2024-09-10T08:07:05.569247+0000","last_undegraded":"2024-09-10T08:07:05.569247+0000","last_fullsized":"2024-09-10T08:07:05.569247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:43.458468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511441+0000","last_change":"2024-09-10T08:05:35.530522+0000","last_active":"2024-09-10T08:07:05.511441+0000","last_peered":"2024-09-10T08:07:05.511441+0000","last_clean":"2024-09-10T08:07:05.511441+0000","last_became_active":"2024-09-10T08:05:35.530099+0000","last_became_peered":"2024-09-10T08:05:35.530099+0000","last_unstale":"2024-09-10T08:07:05.511441+0000","last_undegraded":"2024-09-10T08:07:05.511441+0000","last_fullsized":"2024-09-10T08:07:05.511441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:58.469119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'6","reported_seq":118,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:26.362173+0000","last_change":"2024-09-10T08:05:22.461391+0000","last_active":"2024-09-10T08:07:26.362173+0000","last_peered":"2024-09-10T08:07:26.362173+0000","last_clean":"2024-09-10T08:07:26.362173+0000","last_became_active":"2024-09-10T08:05:22.461042+0000","last_became_peered":"2024-09-10T08:05:22.461042+0000","last_unstale":"2024-09-10T08:07:26.362173+0000","last_undegraded":"2024-09-10T08:07:26.362173+0000","last_fullsized":"2024-09-10T08:07:26.362173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:12.635710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508510+0000","last_change":"2024-09-10T08:05:25.487378+0000","last_active":"2024-09-10T08:07:05.508510+0000","last_peered":"2024-09-10T08:07:05.508510+0000","last_clean":"2024-09-10T08:07:05.508510+0000","last_became_active":"2024-09-10T08:05:25.487055+0000","last_became_peered":"2024-09-10T08:05:25.487055+0000","last_unstale":"2024-09-10T08:07:05.508510+0000","last_undegraded":"2024-09-10T08:07:05.508510+0000","last_fullsized":"2024-09-10T08:07:05.508510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:59.522014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511645+0000","last_change":"2024-09-10T08:05:35.522765+0000","last_active":"2024-09-10T08:07:05.511645+0000","last_peered":"2024-09-10T08:07:05.511645+0000","last_clean":"2024-09-10T08:07:05.511645+0000","last_became_active":"2024-09-10T08:05:35.522359+0000","last_became_peered":"2024-09-10T08:05:35.522359+0000","last_unstale":"2024-09-10T08:07:05.511645+0000","last_undegraded":"2024-09-10T08:07:05.511645+0000","last_fullsized":"2024-09-10T08:07:05.511645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:15.280867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769725+0000","last_change":"2024-09-10T08:05:25.475388+0000","last_active":"2024-09-10T08:07:05.769725+0000","last_peered":"2024-09-10T08:07:05.769725+0000","last_clean":"2024-09-10T08:07:05.769725+0000","last_became_active":"2024-09-10T08:05:25.474331+0000","last_became_peered":"2024-09-10T08:05:25.474331+0000","last_unstale":"2024-09-10T08:07:05.769725+0000","last_undegraded":"2024-09-10T08:07:05.769725+0000","last_fullsized":"2024-09-10T08:07:05.769725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:58.713148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.570505+0000","last_change":"2024-09-10T08:07:05.570505+0000","last_active":"2024-09-10T08:07:05.570505+0000","last_peered":"2024-09-10T08:07:05.570505+0000","last_clean":"2024-09-10T08:07:05.570505+0000","last_became_active":"2024-09-10T08:07:05.569970+0000","last_became_peered":"2024-09-10T08:07:05.569970+0000","last_unstale":"2024-09-10T08:07:05.570505+0000","last_undegraded":"2024-09-10T08:07:05.570505+0000","last_fullsized":"2024-09-10T08:07:05.570505+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:13.282503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511389+0000","last_change":"2024-09-10T08:05:25.491585+0000","last_active":"2024-09-10T08:07:05.511389+0000","last_peered":"2024-09-10T08:07:05.511389+0000","last_clean":"2024-09-10T08:07:05.511389+0000","last_became_active":"2024-09-10T08:05:25.491224+0000","last_became_peered":"2024-09-10T08:05:25.491224+0000","last_unstale":"2024-09-10T08:07:05.511389+0000","last_undegraded":"2024-09-10T08:07:05.511389+0000","last_fullsized":"2024-09-10T08:07:05.511389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:56.066808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514849+0000","last_change":"2024-09-10T08:05:25.499578+0000","last_active":"2024-09-10T08:07:05.514849+0000","last_peered":"2024-09-10T08:07:05.514849+0000","last_clean":"2024-09-10T08:07:05.514849+0000","last_became_active":"2024-09-10T08:05:25.498445+0000","last_became_peered":"2024-09-10T08:05:25.498445+0000","last_unstale":"2024-09-10T08:07:05.514849+0000","last_undegraded":"2024-09-10T08:07:05.514849+0000","last_fullsized":"2024-09-10T08:07:05.514849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:58.961699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914503+0000","last_change":"2024-09-10T08:06:04.735764+0000","last_active":"2024-09-10T08:07:05.914503+0000","last_peered":"2024-09-10T08:07:05.914503+0000","last_clean":"2024-09-10T08:07:05.914503+0000","last_became_active":"2024-09-10T08:06:04.735366+0000","last_became_peered":"2024-09-10T08:06:04.735366+0000","last_unstale":"2024-09-10T08:07:05.914503+0000","last_undegraded":"2024-09-10T08:07:05.914503+0000","last_fullsized":"2024-09-10T08:07:05.914503+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:03.757890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770279+0000","last_change":"2024-09-10T08:05:25.475872+0000","last_active":"2024-09-10T08:07:05.770279+0000","last_peered":"2024-09-10T08:07:05.770279+0000","last_clean":"2024-09-10T08:07:05.770279+0000","last_became_active":"2024-09-10T08:05:25.475549+0000","last_became_peered":"2024-09-10T08:05:25.475549+0000","last_unstale":"2024-09-10T08:07:05.770279+0000","last_undegraded":"2024-09-10T08:07:05.770279+0000","last_fullsized":"2024-09-10T08:07:05.770279+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:37.575267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511808+0000","last_change":"2024-09-10T08:05:25.474210+0000","last_active":"2024-09-10T08:07:05.511808+0000","last_peered":"2024-09-10T08:07:05.511808+0000","last_clean":"2024-09-10T08:07:05.511808+0000","last_became_active":"2024-09-10T08:05:25.473882+0000","last_became_peered":"2024-09-10T08:05:25.473882+0000","last_unstale":"2024-09-10T08:07:05.511808+0000","last_undegraded":"2024-09-10T08:07:05.511808+0000","last_fullsized":"2024-09-10T08:07:05.511808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:05.164370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.877060+0000","last_change":"2024-09-10T08:05:25.490406+0000","last_active":"2024-09-10T08:07:05.877060+0000","last_peered":"2024-09-10T08:07:05.877060+0000","last_clean":"2024-09-10T08:07:05.877060+0000","last_became_active":"2024-09-10T08:05:25.490061+0000","last_became_peered":"2024-09-10T08:05:25.490061+0000","last_unstale":"2024-09-10T08:07:05.877060+0000","last_undegraded":"2024-09-10T08:07:05.877060+0000","last_fullsized":"2024-09-10T08:07:05.877060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:12.988229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568674+0000","last_change":"2024-09-10T08:05:25.504049+0000","last_active":"2024-09-10T08:07:05.568674+0000","last_peered":"2024-09-10T08:07:05.568674+0000","last_clean":"2024-09-10T08:07:05.568674+0000","last_became_active":"2024-09-10T08:05:25.503770+0000","last_became_peered":"2024-09-10T08:05:25.503770+0000","last_unstale":"2024-09-10T08:07:05.568674+0000","last_undegraded":"2024-09-10T08:07:05.568674+0000","last_fullsized":"2024-09-10T08:07:05.568674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:52.850945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876620+0000","last_change":"2024-09-10T08:05:25.491682+0000","last_active":"2024-09-10T08:07:05.876620+0000","last_peered":"2024-09-10T08:07:05.876620+0000","last_clean":"2024-09-10T08:07:05.876620+0000","last_became_active":"2024-09-10T08:05:25.491296+0000","last_became_peered":"2024-09-10T08:05:25.491296+0000","last_unstale":"2024-09-10T08:07:05.876620+0000","last_undegraded":"2024-09-10T08:07:05.876620+0000","last_fullsized":"2024-09-10T08:07:05.876620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:30.075842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914579+0000","last_change":"2024-09-10T08:05:25.489648+0000","last_active":"2024-09-10T08:07:05.914579+0000","last_peered":"2024-09-10T08:07:05.914579+0000","last_clean":"2024-09-10T08:07:05.914579+0000","last_became_active":"2024-09-10T08:05:25.489267+0000","last_became_peered":"2024-09-10T08:05:25.489267+0000","last_unstale":"2024-09-10T08:07:05.914579+0000","last_undegraded":"2024-09-10T08:07:05.914579+0000","last_fullsized":"2024-09-10T08:07:05.914579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:21.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914393+0000","last_change":"2024-09-10T08:05:25.497862+0000","last_active":"2024-09-10T08:07:05.914393+0000","last_peered":"2024-09-10T08:07:05.914393+0000","last_clean":"2024-09-10T08:07:05.914393+0000","last_became_active":"2024-09-10T08:05:25.497504+0000","last_became_peered":"2024-09-10T08:05:25.497504+0000","last_unstale":"2024-09-10T08:07:05.914393+0000","last_undegraded":"2024-09-10T08:07:05.914393+0000","last_fullsized":"2024-09-10T08:07:05.914393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:40.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770039+0000","last_change":"2024-09-10T08:05:25.485924+0000","last_active":"2024-09-10T08:07:05.770039+0000","last_peered":"2024-09-10T08:07:05.770039+0000","last_clean":"2024-09-10T08:07:05.770039+0000","last_became_active":"2024-09-10T08:05:25.485545+0000","last_became_peered":"2024-09-10T08:05:25.485545+0000","last_unstale":"2024-09-10T08:07:05.770039+0000","last_undegraded":"2024-09-10T08:07:05.770039+0000","last_fullsized":"2024-09-10T08:07:05.770039+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:33:20.233720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511912+0000","last_change":"2024-09-10T08:05:25.491911+0000","last_active":"2024-09-10T08:07:05.511912+0000","last_peered":"2024-09-10T08:07:05.511912+0000","last_clean":"2024-09-10T08:07:05.511912+0000","last_became_active":"2024-09-10T08:05:25.491330+0000","last_became_peered":"2024-09-10T08:05:25.491330+0000","last_unstale":"2024-09-10T08:07:05.511912+0000","last_undegraded":"2024-09-10T08:07:05.511912+0000","last_fullsized":"2024-09-10T08:07:05.511912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:06.043331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513499+0000","last_change":"2024-09-10T08:05:25.492220+0000","last_active":"2024-09-10T08:07:05.513499+0000","last_peered":"2024-09-10T08:07:05.513499+0000","last_clean":"2024-09-10T08:07:05.513499+0000","last_became_active":"2024-09-10T08:05:25.491473+0000","last_became_peered":"2024-09-10T08:05:25.491473+0000","last_unstale":"2024-09-10T08:07:05.513499+0000","last_undegraded":"2024-09-10T08:07:05.513499+0000","last_fullsized":"2024-09-10T08:07:05.513499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:13:30.278434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770467+0000","last_change":"2024-09-10T08:05:25.488927+0000","last_active":"2024-09-10T08:07:05.770467+0000","last_peered":"2024-09-10T08:07:05.770467+0000","last_clean":"2024-09-10T08:07:05.770467+0000","last_became_active":"2024-09-10T08:05:25.488550+0000","last_became_peered":"2024-09-10T08:05:25.488550+0000","last_unstale":"2024-09-10T08:07:05.770467+0000","last_undegraded":"2024-09-10T08:07:05.770467+0000","last_fullsized":"2024-09-10T08:07:05.770467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:44.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914091+0000","last_change":"2024-09-10T08:05:25.490125+0000","last_active":"2024-09-10T08:07:05.914091+0000","last_peered":"2024-09-10T08:07:05.914091+0000","last_clean":"2024-09-10T08:07:05.914091+0000","last_became_active":"2024-09-10T08:05:25.487805+0000","last_became_peered":"2024-09-10T08:05:25.487805+0000","last_unstale":"2024-09-10T08:07:05.914091+0000","last_undegraded":"2024-09-10T08:07:05.914091+0000","last_fullsized":"2024-09-10T08:07:05.914091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:33.479009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513789+0000","last_change":"2024-09-10T08:05:25.500346+0000","last_active":"2024-09-10T08:07:05.513789+0000","last_peered":"2024-09-10T08:07:05.513789+0000","last_clean":"2024-09-10T08:07:05.513789+0000","last_became_active":"2024-09-10T08:05:25.498289+0000","last_became_peered":"2024-09-10T08:05:25.498289+0000","last_unstale":"2024-09-10T08:07:05.513789+0000","last_undegraded":"2024-09-10T08:07:05.513789+0000","last_fullsized":"2024-09-10T08:07:05.513789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:12.054926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514016+0000","last_change":"2024-09-10T08:05:25.490001+0000","last_active":"2024-09-10T08:07:05.514016+0000","last_peered":"2024-09-10T08:07:05.514016+0000","last_clean":"2024-09-10T08:07:05.514016+0000","last_became_active":"2024-09-10T08:05:25.489706+0000","last_became_peered":"2024-09-10T08:05:25.489706+0000","last_unstale":"2024-09-10T08:07:05.514016+0000","last_undegraded":"2024-09-10T08:07:05.514016+0000","last_fullsized":"2024-09-10T08:07:05.514016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:38.822140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914315+0000","last_change":"2024-09-10T08:05:25.493874+0000","last_active":"2024-09-10T08:07:05.914315+0000","last_peered":"2024-09-10T08:07:05.914315+0000","last_clean":"2024-09-10T08:07:05.914315+0000","last_became_active":"2024-09-10T08:05:25.489911+0000","last_became_peered":"2024-09-10T08:05:25.489911+0000","last_unstale":"2024-09-10T08:07:05.914315+0000","last_undegraded":"2024-09-10T08:07:05.914315+0000","last_fullsized":"2024-09-10T08:07:05.914315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:46.509657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914747+0000","last_change":"2024-09-10T08:05:25.490277+0000","last_active":"2024-09-10T08:07:05.914747+0000","last_peered":"2024-09-10T08:07:05.914747+0000","last_clean":"2024-09-10T08:07:05.914747+0000","last_became_active":"2024-09-10T08:05:25.486863+0000","last_became_peered":"2024-09-10T08:05:25.486863+0000","last_unstale":"2024-09-10T08:07:05.914747+0000","last_undegraded":"2024-09-10T08:07:05.914747+0000","last_fullsized":"2024-09-10T08:07:05.914747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:21.353830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509623+0000","last_change":"2024-09-10T08:05:25.491028+0000","last_active":"2024-09-10T08:07:05.509623+0000","last_peered":"2024-09-10T08:07:05.509623+0000","last_clean":"2024-09-10T08:07:05.509623+0000","last_became_active":"2024-09-10T08:05:25.490452+0000","last_became_peered":"2024-09-10T08:05:25.490452+0000","last_unstale":"2024-09-10T08:07:05.509623+0000","last_undegraded":"2024-09-10T08:07:05.509623+0000","last_fullsized":"2024-09-10T08:07:05.509623+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:53.059077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915382+0000","last_change":"2024-09-10T08:05:25.489991+0000","last_active":"2024-09-10T08:07:05.915382+0000","last_peered":"2024-09-10T08:07:05.915382+0000","last_clean":"2024-09-10T08:07:05.915382+0000","last_became_active":"2024-09-10T08:05:25.489662+0000","last_became_peered":"2024-09-10T08:05:25.489662+0000","last_unstale":"2024-09-10T08:07:05.915382+0000","last_undegraded":"2024-09-10T08:07:05.915382+0000","last_fullsized":"2024-09-10T08:07:05.915382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:47.940712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915269+0000","last_change":"2024-09-10T08:05:25.475752+0000","last_active":"2024-09-10T08:07:05.915269+0000","last_peered":"2024-09-10T08:07:05.915269+0000","last_clean":"2024-09-10T08:07:05.915269+0000","last_became_active":"2024-09-10T08:05:25.475395+0000","last_became_peered":"2024-09-10T08:05:25.475395+0000","last_unstale":"2024-09-10T08:07:05.915269+0000","last_undegraded":"2024-09-10T08:07:05.915269+0000","last_fullsized":"2024-09-10T08:07:05.915269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.240154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876525+0000","last_change":"2024-09-10T08:05:25.503061+0000","last_active":"2024-09-10T08:07:05.876525+0000","last_peered":"2024-09-10T08:07:05.876525+0000","last_clean":"2024-09-10T08:07:05.876525+0000","last_became_active":"2024-09-10T08:05:25.501333+0000","last_became_peered":"2024-09-10T08:05:25.501333+0000","last_unstale":"2024-09-10T08:07:05.876525+0000","last_undegraded":"2024-09-10T08:07:05.876525+0000","last_fullsized":"2024-09-10T08:07:05.876525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:38.254777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514631+0000","last_change":"2024-09-10T08:07:05.514631+0000","last_active":"2024-09-10T08:07:05.514631+0000","last_peered":"2024-09-10T08:07:05.514631+0000","last_clean":"2024-09-10T08:07:05.514631+0000","last_became_active":"2024-09-10T08:07:05.514285+0000","last_became_peered":"2024-09-10T08:07:05.514285+0000","last_unstale":"2024-09-10T08:07:05.514631+0000","last_undegraded":"2024-09-10T08:07:05.514631+0000","last_fullsized":"2024-09-10T08:07:05.514631+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:40.293121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915209+0000","last_change":"2024-09-10T08:05:25.491457+0000","last_active":"2024-09-10T08:07:05.915209+0000","last_peered":"2024-09-10T08:07:05.915209+0000","last_clean":"2024-09-10T08:07:05.915209+0000","last_became_active":"2024-09-10T08:05:25.491162+0000","last_became_peered":"2024-09-10T08:05:25.491162+0000","last_unstale":"2024-09-10T08:07:05.915209+0000","last_undegraded":"2024-09-10T08:07:05.915209+0000","last_fullsized":"2024-09-10T08:07:05.915209+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:14.389752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508872+0000","last_change":"2024-09-10T08:05:25.487874+0000","last_active":"2024-09-10T08:07:05.508872+0000","last_peered":"2024-09-10T08:07:05.508872+0000","last_clean":"2024-09-10T08:07:05.508872+0000","last_became_active":"2024-09-10T08:05:25.487497+0000","last_became_peered":"2024-09-10T08:05:25.487497+0000","last_unstale":"2024-09-10T08:07:05.508872+0000","last_undegraded":"2024-09-10T08:07:05.508872+0000","last_fullsized":"2024-09-10T08:07:05.508872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:26.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914692+0000","last_change":"2024-09-10T08:05:25.490018+0000","last_active":"2024-09-10T08:07:05.914692+0000","last_peered":"2024-09-10T08:07:05.914692+0000","last_clean":"2024-09-10T08:07:05.914692+0000","last_became_active":"2024-09-10T08:05:25.484951+0000","last_became_peered":"2024-09-10T08:05:25.484951+0000","last_unstale":"2024-09-10T08:07:05.914692+0000","last_undegraded":"2024-09-10T08:07:05.914692+0000","last_fullsized":"2024-09-10T08:07:05.914692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:07.516154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514980+0000","last_change":"2024-09-10T08:05:25.501581+0000","last_active":"2024-09-10T08:07:05.514980+0000","last_peered":"2024-09-10T08:07:05.514980+0000","last_clean":"2024-09-10T08:07:05.514980+0000","last_became_active":"2024-09-10T08:05:25.499690+0000","last_became_peered":"2024-09-10T08:05:25.499690+0000","last_unstale":"2024-09-10T08:07:05.514980+0000","last_undegraded":"2024-09-10T08:07:05.514980+0000","last_fullsized":"2024-09-10T08:07:05.514980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:14.569791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513674+0000","last_change":"2024-09-10T08:05:25.487945+0000","last_active":"2024-09-10T08:07:05.513674+0000","last_peered":"2024-09-10T08:07:05.513674+0000","last_clean":"2024-09-10T08:07:05.513674+0000","last_became_active":"2024-09-10T08:05:25.487655+0000","last_became_peered":"2024-09-10T08:05:25.487655+0000","last_unstale":"2024-09-10T08:07:05.513674+0000","last_undegraded":"2024-09-10T08:07:05.513674+0000","last_fullsized":"2024-09-10T08:07:05.513674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:37.746037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770451+0000","last_change":"2024-09-10T08:05:25.489348+0000","last_active":"2024-09-10T08:07:05.770451+0000","last_peered":"2024-09-10T08:07:05.770451+0000","last_clean":"2024-09-10T08:07:05.770451+0000","last_became_active":"2024-09-10T08:05:25.489004+0000","last_became_peered":"2024-09-10T08:05:25.489004+0000","last_unstale":"2024-09-10T08:07:05.770451+0000","last_undegraded":"2024-09-10T08:07:05.770451+0000","last_fullsized":"2024-09-10T08:07:05.770451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:54.554719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66664,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":416,"num_read_kb":278,"num_write":167,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3725,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214976,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514535,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":548036,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743468,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32360,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712792,"statfs":{"total":95995035648,"available":95961899008,"internally_reserved":0,"allocated":1200128,"data_stored":1202039,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906992,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1064960,"data_stored":548250,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135927,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1060864,"data_stored":548148,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1114112,"data_stored":1144081,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528385,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":543426,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1122304,"data_stored":1139669,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953548,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1130496,"data_stored":610737,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1056768,"data_stored":543620,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1064960,"data_stored":545079,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1114112,"data_stored":605997,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1069056,"data_stored":544814,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:07:30.622 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:07:30.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:30 smithi002 ceph-mon[30563]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:30.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:30.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:30.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:30 smithi002 ceph-mon[30563]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:07:30.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:07:30.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:07:30.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:31.008 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:31.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:30 smithi155 ceph-mon[35405]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:31.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:31.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:31.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:30 smithi155 ceph-mon[35405]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:07:31.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:07:31.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:07:31.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:31.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:30 smithi107 ceph-mon[35048]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:31.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:31.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:31.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:30 smithi107 ceph-mon[35048]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:07:31.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:07:31.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:07:31.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:07:31.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.638+0000 7f9a0cc00640 1 -- 172.21.15.2:0/1419310052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081049a0 msgr2=0x7f9a081089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:31.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.638+0000 7f9a0cc00640 1 --2- 172.21.15.2:0/1419310052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081049a0 0x7f9a081089f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009900 tx=0x7f99f402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:31.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.638+0000 7f9a0cc00640 1 -- 172.21.15.2:0/1419310052 shutdown_connections 2024-09-10T08:07:31.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.638+0000 7f9a0cc00640 1 --2- 172.21.15.2:0/1419310052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a08109120 0x7f9a08110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:31.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.638+0000 7f9a0cc00640 1 --2- 172.21.15.2:0/1419310052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081049a0 0x7f9a081089f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:31.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.638+0000 7f9a0cc00640 1 --2- 172.21.15.2:0/1419310052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a08103ff0 0x7f9a081043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:31.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.638+0000 7f9a0cc00640 1 -- 172.21.15.2:0/1419310052 >> 172.21.15.2:0/1419310052 conn(0x7f9a080ffa90 msgr2=0x7f9a08101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:31.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.638+0000 7f9a0cc00640 1 -- 172.21.15.2:0/1419310052 shutdown_connections 2024-09-10T08:07:31.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.638+0000 7f9a0cc00640 1 -- 172.21.15.2:0/1419310052 wait complete. 2024-09-10T08:07:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.639+0000 7f9a0cc00640 1 Processor -- start 2024-09-10T08:07:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.639+0000 7f9a0cc00640 1 -- start start 2024-09-10T08:07:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.639+0000 7f9a0cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a08103ff0 0x7f9a0819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.639+0000 7f9a0cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a081049a0 0x7f9a0819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.639+0000 7f9a0cc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a08109120 0x7f9a0819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.639+0000 7f9a0cc00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9a08114120 con 0x7f9a08103ff0 2024-09-10T08:07:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.639+0000 7f9a0cc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9a08113fa0 con 0x7f9a081049a0 2024-09-10T08:07:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.639+0000 7f9a0cc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9a081142a0 con 0x7f9a08109120 2024-09-10T08:07:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.640+0000 7f9a05e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a08103ff0 0x7f9a0819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.640+0000 7f9a05400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a081049a0 0x7f9a0819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.640+0000 7f9a05e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a08103ff0 0x7f9a0819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40058/0 (socket says 172.21.15.2:40058) 2024-09-10T08:07:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.640+0000 7f9a05400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a081049a0 0x7f9a0819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:56884/0 (socket says 172.21.15.2:56884) 2024-09-10T08:07:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.640+0000 7f9a05e00640 1 -- 172.21.15.2:0/672678163 learned_addr learned my addr 172.21.15.2:0/672678163 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.640+0000 7f9a06800640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a08109120 0x7f9a0819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.640+0000 7f9a05e00640 1 -- 172.21.15.2:0/672678163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a08109120 msgr2=0x7f9a0819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.640+0000 7f9a05e00640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a08109120 0x7f9a0819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.640+0000 7f9a05e00640 1 -- 172.21.15.2:0/672678163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a081049a0 msgr2=0x7f9a0819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.640+0000 7f9a05e00640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a081049a0 0x7f9a0819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.640+0000 7f9a05e00640 1 -- 172.21.15.2:0/672678163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0819f650 con 0x7f9a08103ff0 2024-09-10T08:07:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.640+0000 7f9a05400640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a081049a0 0x7f9a0819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.641+0000 7f9a05e00640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a08103ff0 0x7f9a0819a6a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f99f800e9f0 tx=0x7f99f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.641+0000 7f99fea00640 1 -- 172.21.15.2:0/672678163 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f800cd80 con 0x7f9a08103ff0 2024-09-10T08:07:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.641+0000 7f99fea00640 1 -- 172.21.15.2:0/672678163 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99f8010910 con 0x7f9a08103ff0 2024-09-10T08:07:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.641+0000 7f9a0cc00640 1 -- 172.21.15.2:0/672678163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0819f940 con 0x7f9a08103ff0 2024-09-10T08:07:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.641+0000 7f99fea00640 1 -- 172.21.15.2:0/672678163 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8018990 con 0x7f9a08103ff0 2024-09-10T08:07:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.641+0000 7f9a0cc00640 1 -- 172.21.15.2:0/672678163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a081a7220 con 0x7f9a08103ff0 2024-09-10T08:07:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.642+0000 7f9a0cc00640 1 -- 172.21.15.2:0/672678163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a081043d0 con 0x7f9a08103ff0 2024-09-10T08:07:31.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.646+0000 7f99fea00640 1 -- 172.21.15.2:0/672678163 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f99f8010ac0 con 0x7f9a08103ff0 2024-09-10T08:07:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.647+0000 7f99fea00640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f99ec0a2c10 0x7f99ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.647+0000 7f99fea00640 1 -- 172.21.15.2:0/672678163 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f99f80d0130 con 0x7f9a08103ff0 2024-09-10T08:07:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.648+0000 7f9a05400640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f99ec0a2c10 0x7f99ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.648+0000 7f9a05400640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f99ec0a2c10 0x7f99ec0a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009230 tx=0x7f99f40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:31.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.650+0000 7f99fea00640 1 -- 172.21.15.2:0/672678163 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99f809cae0 con 0x7f9a08103ff0 2024-09-10T08:07:31.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.832+0000 7f9a0cc00640 1 -- 172.21.15.2:0/672678163 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a08062630 con 0x7f99ec0a2c10 2024-09-10T08:07:31.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.849+0000 7f99fea00640 1 -- 172.21.15.2:0/672678163 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340606 (secure 0 0 0) 0x7f9a08062630 con 0x7f99ec0a2c10 2024-09-10T08:07:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:07:31.856 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-10T08:07:31.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.855+0000 7f9a0cc00640 1 -- 172.21.15.2:0/672678163 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f99ec0a2c10 msgr2=0x7f99ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:31.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.855+0000 7f9a0cc00640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f99ec0a2c10 0x7f99ec0a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009230 tx=0x7f99f40058d0 comp rx=0 tx=0).stop 2024-09-10T08:07:31.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.855+0000 7f9a0cc00640 1 -- 172.21.15.2:0/672678163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a08103ff0 msgr2=0x7f9a0819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:31.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.855+0000 7f9a0cc00640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a08103ff0 0x7f9a0819a6a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f99f800e9f0 tx=0x7f99f800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:07:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.855+0000 7f9a0cc00640 1 -- 172.21.15.2:0/672678163 shutdown_connections 2024-09-10T08:07:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.855+0000 7f9a0cc00640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f99ec0a2c10 0x7f99ec0a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.855+0000 7f9a0cc00640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a08109120 0x7f9a0819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.855+0000 7f9a0cc00640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a081049a0 0x7f9a0819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.855+0000 7f9a0cc00640 1 --2- 172.21.15.2:0/672678163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a08103ff0 0x7f9a0819a6a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.855+0000 7f9a0cc00640 1 -- 172.21.15.2:0/672678163 >> 172.21.15.2:0/672678163 conn(0x7f9a080ffa90 msgr2=0x7f9a08100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.855+0000 7f9a0cc00640 1 -- 172.21.15.2:0/672678163 shutdown_connections 2024-09-10T08:07:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:31.856+0000 7f9a0cc00640 1 -- 172.21.15.2:0/672678163 wait complete. 2024-09-10T08:07:32.720 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-10T08:07:31.672843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657266,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":557,"num_read_kb":397,"num_write":350,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1184461,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362112,"kb_used_data":12788,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579712,"statfs":{"total":1151940427776,"available":1151569625088,"internally_reserved":0,"allocated":13094912,"data_stored":8523896,"data_compressed":497802,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003259"},"pg_stats":[{"pgid":"2.1f","version":"93'9","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514795+0000","last_change":"2024-09-10T08:05:22.462062+0000","last_active":"2024-09-10T08:07:05.514795+0000","last_peered":"2024-09-10T08:07:05.514795+0000","last_clean":"2024-09-10T08:07:05.514795+0000","last_became_active":"2024-09-10T08:05:22.460973+0000","last_became_peered":"2024-09-10T08:05:22.460973+0000","last_unstale":"2024-09-10T08:07:05.514795+0000","last_undegraded":"2024-09-10T08:07:05.514795+0000","last_fullsized":"2024-09-10T08:07:05.514795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:01.357446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.877341+0000","last_change":"2024-09-10T08:05:25.482873+0000","last_active":"2024-09-10T08:07:05.877341+0000","last_peered":"2024-09-10T08:07:05.877341+0000","last_clean":"2024-09-10T08:07:05.877341+0000","last_became_active":"2024-09-10T08:05:25.482559+0000","last_became_peered":"2024-09-10T08:05:25.482559+0000","last_unstale":"2024-09-10T08:07:05.877341+0000","last_undegraded":"2024-09-10T08:07:05.877341+0000","last_fullsized":"2024-09-10T08:07:05.877341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:10:17.366160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511328+0000","last_change":"2024-09-10T08:06:04.736356+0000","last_active":"2024-09-10T08:07:05.511328+0000","last_peered":"2024-09-10T08:07:05.511328+0000","last_clean":"2024-09-10T08:07:05.511328+0000","last_became_active":"2024-09-10T08:06:04.735859+0000","last_became_peered":"2024-09-10T08:06:04.735859+0000","last_unstale":"2024-09-10T08:07:05.511328+0000","last_undegraded":"2024-09-10T08:07:05.511328+0000","last_fullsized":"2024-09-10T08:07:05.511328+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:56.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"93'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.877433+0000","last_change":"2024-09-10T08:05:22.569504+0000","last_active":"2024-09-10T08:07:05.877433+0000","last_peered":"2024-09-10T08:07:05.877433+0000","last_clean":"2024-09-10T08:07:05.877433+0000","last_became_active":"2024-09-10T08:05:22.569164+0000","last_became_peered":"2024-09-10T08:05:22.569164+0000","last_unstale":"2024-09-10T08:07:05.877433+0000","last_undegraded":"2024-09-10T08:07:05.877433+0000","last_fullsized":"2024-09-10T08:07:05.877433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:41.628963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513985+0000","last_change":"2024-09-10T08:05:25.493017+0000","last_active":"2024-09-10T08:07:05.513985+0000","last_peered":"2024-09-10T08:07:05.513985+0000","last_clean":"2024-09-10T08:07:05.513985+0000","last_became_active":"2024-09-10T08:05:25.491370+0000","last_became_peered":"2024-09-10T08:05:25.491370+0000","last_unstale":"2024-09-10T08:07:05.513985+0000","last_undegraded":"2024-09-10T08:07:05.513985+0000","last_fullsized":"2024-09-10T08:07:05.513985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:41.616549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914030+0000","last_change":"2024-09-10T08:05:35.523751+0000","last_active":"2024-09-10T08:07:05.914030+0000","last_peered":"2024-09-10T08:07:05.914030+0000","last_clean":"2024-09-10T08:07:05.914030+0000","last_became_active":"2024-09-10T08:05:35.523372+0000","last_became_peered":"2024-09-10T08:05:35.523372+0000","last_unstale":"2024-09-10T08:07:05.914030+0000","last_undegraded":"2024-09-10T08:07:05.914030+0000","last_fullsized":"2024-09-10T08:07:05.914030+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:20.971596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'4","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508861+0000","last_change":"2024-09-10T08:05:22.460590+0000","last_active":"2024-09-10T08:07:05.508861+0000","last_peered":"2024-09-10T08:07:05.508861+0000","last_clean":"2024-09-10T08:07:05.508861+0000","last_became_active":"2024-09-10T08:05:22.460238+0000","last_became_peered":"2024-09-10T08:05:22.460238+0000","last_unstale":"2024-09-10T08:07:05.508861+0000","last_undegraded":"2024-09-10T08:07:05.508861+0000","last_fullsized":"2024-09-10T08:07:05.508861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:18.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914856+0000","last_change":"2024-09-10T08:05:25.487392+0000","last_active":"2024-09-10T08:07:05.914856+0000","last_peered":"2024-09-10T08:07:05.914856+0000","last_clean":"2024-09-10T08:07:05.914856+0000","last_became_active":"2024-09-10T08:05:25.487054+0000","last_became_peered":"2024-09-10T08:05:25.487054+0000","last_unstale":"2024-09-10T08:07:05.914856+0000","last_undegraded":"2024-09-10T08:07:05.914856+0000","last_fullsized":"2024-09-10T08:07:05.914856+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:16.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914757+0000","last_change":"2024-09-10T08:06:04.735778+0000","last_active":"2024-09-10T08:07:05.914757+0000","last_peered":"2024-09-10T08:07:05.914757+0000","last_clean":"2024-09-10T08:07:05.914757+0000","last_became_active":"2024-09-10T08:06:04.735208+0000","last_became_peered":"2024-09-10T08:06:04.735208+0000","last_unstale":"2024-09-10T08:07:05.914757+0000","last_undegraded":"2024-09-10T08:07:05.914757+0000","last_fullsized":"2024-09-10T08:07:05.914757+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:36.982246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508892+0000","last_change":"2024-09-10T08:05:22.458323+0000","last_active":"2024-09-10T08:07:05.508892+0000","last_peered":"2024-09-10T08:07:05.508892+0000","last_clean":"2024-09-10T08:07:05.508892+0000","last_became_active":"2024-09-10T08:05:22.457965+0000","last_became_peered":"2024-09-10T08:05:22.457965+0000","last_unstale":"2024-09-10T08:07:05.508892+0000","last_undegraded":"2024-09-10T08:07:05.508892+0000","last_fullsized":"2024-09-10T08:07:05.508892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:15.021305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569413+0000","last_change":"2024-09-10T08:05:25.489883+0000","last_active":"2024-09-10T08:07:05.569413+0000","last_peered":"2024-09-10T08:07:05.569413+0000","last_clean":"2024-09-10T08:07:05.569413+0000","last_became_active":"2024-09-10T08:05:25.489482+0000","last_became_peered":"2024-09-10T08:05:25.489482+0000","last_unstale":"2024-09-10T08:07:05.569413+0000","last_undegraded":"2024-09-10T08:07:05.569413+0000","last_fullsized":"2024-09-10T08:07:05.569413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:52.080628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914006+0000","last_change":"2024-09-10T08:05:35.531458+0000","last_active":"2024-09-10T08:07:05.914006+0000","last_peered":"2024-09-10T08:07:05.914006+0000","last_clean":"2024-09-10T08:07:05.914006+0000","last_became_active":"2024-09-10T08:05:35.531072+0000","last_became_peered":"2024-09-10T08:05:35.531072+0000","last_unstale":"2024-09-10T08:07:05.914006+0000","last_undegraded":"2024-09-10T08:07:05.914006+0000","last_fullsized":"2024-09-10T08:07:05.914006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:04.792664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509274+0000","last_change":"2024-09-10T08:05:25.486865+0000","last_active":"2024-09-10T08:07:05.509274+0000","last_peered":"2024-09-10T08:07:05.509274+0000","last_clean":"2024-09-10T08:07:05.509274+0000","last_became_active":"2024-09-10T08:05:25.486531+0000","last_became_peered":"2024-09-10T08:05:25.486531+0000","last_unstale":"2024-09-10T08:07:05.509274+0000","last_undegraded":"2024-09-10T08:07:05.509274+0000","last_fullsized":"2024-09-10T08:07:05.509274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:26.854682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"93'4","reported_seq":59,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508843+0000","last_change":"2024-09-10T08:06:04.743303+0000","last_active":"2024-09-10T08:07:05.508843+0000","last_peered":"2024-09-10T08:07:05.508843+0000","last_clean":"2024-09-10T08:07:05.508843+0000","last_became_active":"2024-09-10T08:06:04.733294+0000","last_became_peered":"2024-09-10T08:06:04.733294+0000","last_unstale":"2024-09-10T08:07:05.508843+0000","last_undegraded":"2024-09-10T08:07:05.508843+0000","last_fullsized":"2024-09-10T08:07:05.508843+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:18.542658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914638+0000","last_change":"2024-09-10T08:05:35.642083+0000","last_active":"2024-09-10T08:07:05.914638+0000","last_peered":"2024-09-10T08:07:05.914638+0000","last_clean":"2024-09-10T08:07:05.914638+0000","last_became_active":"2024-09-10T08:05:35.641214+0000","last_became_peered":"2024-09-10T08:05:35.641214+0000","last_unstale":"2024-09-10T08:07:05.914638+0000","last_undegraded":"2024-09-10T08:07:05.914638+0000","last_fullsized":"2024-09-10T08:07:05.914638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:00:21.731769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876974+0000","last_change":"2024-09-10T08:05:25.501993+0000","last_active":"2024-09-10T08:07:05.876974+0000","last_peered":"2024-09-10T08:07:05.876974+0000","last_clean":"2024-09-10T08:07:05.876974+0000","last_became_active":"2024-09-10T08:05:25.499912+0000","last_became_peered":"2024-09-10T08:05:25.499912+0000","last_unstale":"2024-09-10T08:07:05.876974+0000","last_undegraded":"2024-09-10T08:07:05.876974+0000","last_fullsized":"2024-09-10T08:07:05.876974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:06.780067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"93'1","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511452+0000","last_change":"2024-09-10T08:05:22.461786+0000","last_active":"2024-09-10T08:07:05.511452+0000","last_peered":"2024-09-10T08:07:05.511452+0000","last_clean":"2024-09-10T08:07:05.511452+0000","last_became_active":"2024-09-10T08:05:22.460821+0000","last_became_peered":"2024-09-10T08:05:22.460821+0000","last_unstale":"2024-09-10T08:07:05.511452+0000","last_undegraded":"2024-09-10T08:07:05.511452+0000","last_fullsized":"2024-09-10T08:07:05.511452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:25.083538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914505+0000","last_change":"2024-09-10T08:06:04.734470+0000","last_active":"2024-09-10T08:07:05.914505+0000","last_peered":"2024-09-10T08:07:05.914505+0000","last_clean":"2024-09-10T08:07:05.914505+0000","last_became_active":"2024-09-10T08:06:04.734040+0000","last_became_peered":"2024-09-10T08:06:04.734040+0000","last_unstale":"2024-09-10T08:07:05.914505+0000","last_undegraded":"2024-09-10T08:07:05.914505+0000","last_fullsized":"2024-09-10T08:07:05.914505+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:36.312144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769794+0000","last_change":"2024-09-10T08:05:25.492711+0000","last_active":"2024-09-10T08:07:05.769794+0000","last_peered":"2024-09-10T08:07:05.769794+0000","last_clean":"2024-09-10T08:07:05.769794+0000","last_became_active":"2024-09-10T08:05:25.492336+0000","last_became_peered":"2024-09-10T08:05:25.492336+0000","last_unstale":"2024-09-10T08:07:05.769794+0000","last_undegraded":"2024-09-10T08:07:05.769794+0000","last_fullsized":"2024-09-10T08:07:05.769794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:04.168884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"93'3","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876730+0000","last_change":"2024-09-10T08:05:22.461535+0000","last_active":"2024-09-10T08:07:05.876730+0000","last_peered":"2024-09-10T08:07:05.876730+0000","last_clean":"2024-09-10T08:07:05.876730+0000","last_became_active":"2024-09-10T08:05:22.461208+0000","last_became_peered":"2024-09-10T08:05:22.461208+0000","last_unstale":"2024-09-10T08:07:05.876730+0000","last_undegraded":"2024-09-10T08:07:05.876730+0000","last_fullsized":"2024-09-10T08:07:05.876730+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:26.009242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513843+0000","last_change":"2024-09-10T08:05:35.535442+0000","last_active":"2024-09-10T08:07:05.513843+0000","last_peered":"2024-09-10T08:07:05.513843+0000","last_clean":"2024-09-10T08:07:05.513843+0000","last_became_active":"2024-09-10T08:05:35.535058+0000","last_became_peered":"2024-09-10T08:05:35.535058+0000","last_unstale":"2024-09-10T08:07:05.513843+0000","last_undegraded":"2024-09-10T08:07:05.513843+0000","last_fullsized":"2024-09-10T08:07:05.513843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:39.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508321+0000","last_change":"2024-09-10T08:05:25.483440+0000","last_active":"2024-09-10T08:07:05.508321+0000","last_peered":"2024-09-10T08:07:05.508321+0000","last_clean":"2024-09-10T08:07:05.508321+0000","last_became_active":"2024-09-10T08:05:25.483141+0000","last_became_peered":"2024-09-10T08:05:25.483141+0000","last_unstale":"2024-09-10T08:07:05.508321+0000","last_undegraded":"2024-09-10T08:07:05.508321+0000","last_fullsized":"2024-09-10T08:07:05.508321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:00.953188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914286+0000","last_change":"2024-09-10T08:05:22.568141+0000","last_active":"2024-09-10T08:07:05.914286+0000","last_peered":"2024-09-10T08:07:05.914286+0000","last_clean":"2024-09-10T08:07:05.914286+0000","last_became_active":"2024-09-10T08:05:22.567857+0000","last_became_peered":"2024-09-10T08:05:22.567857+0000","last_unstale":"2024-09-10T08:07:05.914286+0000","last_undegraded":"2024-09-10T08:07:05.914286+0000","last_fullsized":"2024-09-10T08:07:05.914286+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:22.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914747+0000","last_change":"2024-09-10T08:05:35.641993+0000","last_active":"2024-09-10T08:07:05.914747+0000","last_peered":"2024-09-10T08:07:05.914747+0000","last_clean":"2024-09-10T08:07:05.914747+0000","last_became_active":"2024-09-10T08:05:35.641376+0000","last_became_peered":"2024-09-10T08:05:35.641376+0000","last_unstale":"2024-09-10T08:07:05.914747+0000","last_undegraded":"2024-09-10T08:07:05.914747+0000","last_fullsized":"2024-09-10T08:07:05.914747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:16:34.038924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513778+0000","last_change":"2024-09-10T08:06:04.737560+0000","last_active":"2024-09-10T08:07:05.513778+0000","last_peered":"2024-09-10T08:07:05.513778+0000","last_clean":"2024-09-10T08:07:05.513778+0000","last_became_active":"2024-09-10T08:06:04.736656+0000","last_became_peered":"2024-09-10T08:06:04.736656+0000","last_unstale":"2024-09-10T08:07:05.513778+0000","last_undegraded":"2024-09-10T08:07:05.513778+0000","last_fullsized":"2024-09-10T08:07:05.513778+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:35.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"93'4","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915938+0000","last_change":"2024-09-10T08:05:22.567077+0000","last_active":"2024-09-10T08:07:05.915938+0000","last_peered":"2024-09-10T08:07:05.915938+0000","last_clean":"2024-09-10T08:07:05.915938+0000","last_became_active":"2024-09-10T08:05:22.566757+0000","last_became_peered":"2024-09-10T08:05:22.566757+0000","last_unstale":"2024-09-10T08:07:05.915938+0000","last_undegraded":"2024-09-10T08:07:05.915938+0000","last_fullsized":"2024-09-10T08:07:05.915938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:06.881510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511464+0000","last_change":"2024-09-10T08:05:35.519539+0000","last_active":"2024-09-10T08:07:05.511464+0000","last_peered":"2024-09-10T08:07:05.511464+0000","last_clean":"2024-09-10T08:07:05.511464+0000","last_became_active":"2024-09-10T08:05:35.519044+0000","last_became_peered":"2024-09-10T08:05:35.519044+0000","last_unstale":"2024-09-10T08:07:05.511464+0000","last_undegraded":"2024-09-10T08:07:05.511464+0000","last_fullsized":"2024-09-10T08:07:05.511464+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:10.546949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.772349+0000","last_change":"2024-09-10T08:07:05.771774+0000","last_active":"2024-09-10T08:07:05.772349+0000","last_peered":"2024-09-10T08:07:05.772349+0000","last_clean":"2024-09-10T08:07:05.772349+0000","last_became_active":"2024-09-10T08:07:05.771336+0000","last_became_peered":"2024-09-10T08:07:05.771336+0000","last_unstale":"2024-09-10T08:07:05.772349+0000","last_undegraded":"2024-09-10T08:07:05.772349+0000","last_fullsized":"2024-09-10T08:07:05.772349+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:11.482222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"93'11","reported_seq":64,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513571+0000","last_change":"2024-09-10T08:06:04.741199+0000","last_active":"2024-09-10T08:07:05.513571+0000","last_peered":"2024-09-10T08:07:05.513571+0000","last_clean":"2024-09-10T08:07:05.513571+0000","last_became_active":"2024-09-10T08:06:04.738076+0000","last_became_peered":"2024-09-10T08:06:04.738076+0000","last_unstale":"2024-09-10T08:07:05.513571+0000","last_undegraded":"2024-09-10T08:07:05.513571+0000","last_fullsized":"2024-09-10T08:07:05.513571+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:45.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876403+0000","last_change":"2024-09-10T08:05:35.642358+0000","last_active":"2024-09-10T08:07:05.876403+0000","last_peered":"2024-09-10T08:07:05.876403+0000","last_clean":"2024-09-10T08:07:05.876403+0000","last_became_active":"2024-09-10T08:05:35.641408+0000","last_became_peered":"2024-09-10T08:05:35.641408+0000","last_unstale":"2024-09-10T08:07:05.876403+0000","last_undegraded":"2024-09-10T08:07:05.876403+0000","last_fullsized":"2024-09-10T08:07:05.876403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:42.015139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914499+0000","last_change":"2024-09-10T08:05:25.486310+0000","last_active":"2024-09-10T08:07:05.914499+0000","last_peered":"2024-09-10T08:07:05.914499+0000","last_clean":"2024-09-10T08:07:05.914499+0000","last_became_active":"2024-09-10T08:05:25.485915+0000","last_became_peered":"2024-09-10T08:05:25.485915+0000","last_unstale":"2024-09-10T08:07:05.914499+0000","last_undegraded":"2024-09-10T08:07:05.914499+0000","last_fullsized":"2024-09-10T08:07:05.914499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:56.064361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"93'1","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508987+0000","last_change":"2024-09-10T08:05:22.570595+0000","last_active":"2024-09-10T08:07:05.508987+0000","last_peered":"2024-09-10T08:07:05.508987+0000","last_clean":"2024-09-10T08:07:05.508987+0000","last_became_active":"2024-09-10T08:05:22.570219+0000","last_became_peered":"2024-09-10T08:05:22.570219+0000","last_unstale":"2024-09-10T08:07:05.508987+0000","last_undegraded":"2024-09-10T08:07:05.508987+0000","last_fullsized":"2024-09-10T08:07:05.508987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:03.230325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769829+0000","last_change":"2024-09-10T08:05:35.530472+0000","last_active":"2024-09-10T08:07:05.769829+0000","last_peered":"2024-09-10T08:07:05.769829+0000","last_clean":"2024-09-10T08:07:05.769829+0000","last_became_active":"2024-09-10T08:05:35.530066+0000","last_became_peered":"2024-09-10T08:05:35.530066+0000","last_unstale":"2024-09-10T08:07:05.769829+0000","last_undegraded":"2024-09-10T08:07:05.769829+0000","last_fullsized":"2024-09-10T08:07:05.769829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:58.915767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914831+0000","last_change":"2024-09-10T08:05:25.500524+0000","last_active":"2024-09-10T08:07:05.914831+0000","last_peered":"2024-09-10T08:07:05.914831+0000","last_clean":"2024-09-10T08:07:05.914831+0000","last_became_active":"2024-09-10T08:05:25.500186+0000","last_became_peered":"2024-09-10T08:05:25.500186+0000","last_unstale":"2024-09-10T08:07:05.914831+0000","last_undegraded":"2024-09-10T08:07:05.914831+0000","last_fullsized":"2024-09-10T08:07:05.914831+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:51.936222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513172+0000","last_change":"2024-09-10T08:05:22.569326+0000","last_active":"2024-09-10T08:07:05.513172+0000","last_peered":"2024-09-10T08:07:05.513172+0000","last_clean":"2024-09-10T08:07:05.513172+0000","last_became_active":"2024-09-10T08:05:22.568895+0000","last_became_peered":"2024-09-10T08:05:22.568895+0000","last_unstale":"2024-09-10T08:07:05.513172+0000","last_undegraded":"2024-09-10T08:07:05.513172+0000","last_fullsized":"2024-09-10T08:07:05.513172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:47.212073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508317+0000","last_change":"2024-09-10T08:05:35.641600+0000","last_active":"2024-09-10T08:07:05.508317+0000","last_peered":"2024-09-10T08:07:05.508317+0000","last_clean":"2024-09-10T08:07:05.508317+0000","last_became_active":"2024-09-10T08:05:35.641205+0000","last_became_peered":"2024-09-10T08:05:35.641205+0000","last_unstale":"2024-09-10T08:07:05.508317+0000","last_undegraded":"2024-09-10T08:07:05.508317+0000","last_fullsized":"2024-09-10T08:07:05.508317+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:40.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915549+0000","last_change":"2024-09-10T08:05:25.495035+0000","last_active":"2024-09-10T08:07:05.915549+0000","last_peered":"2024-09-10T08:07:05.915549+0000","last_clean":"2024-09-10T08:07:05.915549+0000","last_became_active":"2024-09-10T08:05:25.494733+0000","last_became_peered":"2024-09-10T08:05:25.494733+0000","last_unstale":"2024-09-10T08:07:05.915549+0000","last_undegraded":"2024-09-10T08:07:05.915549+0000","last_fullsized":"2024-09-10T08:07:05.915549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.875867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"93'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509485+0000","last_change":"2024-09-10T08:05:22.458727+0000","last_active":"2024-09-10T08:07:05.509485+0000","last_peered":"2024-09-10T08:07:05.509485+0000","last_clean":"2024-09-10T08:07:05.509485+0000","last_became_active":"2024-09-10T08:05:22.458397+0000","last_became_peered":"2024-09-10T08:05:22.458397+0000","last_unstale":"2024-09-10T08:07:05.509485+0000","last_undegraded":"2024-09-10T08:07:05.509485+0000","last_fullsized":"2024-09-10T08:07:05.509485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:07.875559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914333+0000","last_change":"2024-09-10T08:05:35.534192+0000","last_active":"2024-09-10T08:07:05.914333+0000","last_peered":"2024-09-10T08:07:05.914333+0000","last_clean":"2024-09-10T08:07:05.914333+0000","last_became_active":"2024-09-10T08:05:35.530450+0000","last_became_peered":"2024-09-10T08:05:35.530450+0000","last_unstale":"2024-09-10T08:07:05.914333+0000","last_undegraded":"2024-09-10T08:07:05.914333+0000","last_fullsized":"2024-09-10T08:07:05.914333+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:10.039144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568765+0000","last_change":"2024-09-10T08:05:25.482252+0000","last_active":"2024-09-10T08:07:05.568765+0000","last_peered":"2024-09-10T08:07:05.568765+0000","last_clean":"2024-09-10T08:07:05.568765+0000","last_became_active":"2024-09-10T08:05:25.481901+0000","last_became_peered":"2024-09-10T08:05:25.481901+0000","last_unstale":"2024-09-10T08:07:05.568765+0000","last_undegraded":"2024-09-10T08:07:05.568765+0000","last_fullsized":"2024-09-10T08:07:05.568765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:17.063340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'4","reported_seq":114,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:27.362804+0000","last_change":"2024-09-10T08:05:22.569199+0000","last_active":"2024-09-10T08:07:27.362804+0000","last_peered":"2024-09-10T08:07:27.362804+0000","last_clean":"2024-09-10T08:07:27.362804+0000","last_became_active":"2024-09-10T08:05:22.568787+0000","last_became_peered":"2024-09-10T08:05:22.568787+0000","last_unstale":"2024-09-10T08:07:27.362804+0000","last_undegraded":"2024-09-10T08:07:27.362804+0000","last_fullsized":"2024-09-10T08:07:27.362804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:59.610130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914604+0000","last_change":"2024-09-10T08:05:35.642446+0000","last_active":"2024-09-10T08:07:05.914604+0000","last_peered":"2024-09-10T08:07:05.914604+0000","last_clean":"2024-09-10T08:07:05.914604+0000","last_became_active":"2024-09-10T08:05:35.641543+0000","last_became_peered":"2024-09-10T08:05:35.641543+0000","last_unstale":"2024-09-10T08:07:05.914604+0000","last_undegraded":"2024-09-10T08:07:05.914604+0000","last_fullsized":"2024-09-10T08:07:05.914604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:33.658406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.913993+0000","last_change":"2024-09-10T08:05:25.487598+0000","last_active":"2024-09-10T08:07:05.913993+0000","last_peered":"2024-09-10T08:07:05.913993+0000","last_clean":"2024-09-10T08:07:05.913993+0000","last_became_active":"2024-09-10T08:05:25.486879+0000","last_became_peered":"2024-09-10T08:05:25.486879+0000","last_unstale":"2024-09-10T08:07:05.913993+0000","last_undegraded":"2024-09-10T08:07:05.913993+0000","last_fullsized":"2024-09-10T08:07:05.913993+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:20.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509934+0000","last_change":"2024-09-10T08:05:22.463416+0000","last_active":"2024-09-10T08:07:05.509934+0000","last_peered":"2024-09-10T08:07:05.509934+0000","last_clean":"2024-09-10T08:07:05.509934+0000","last_became_active":"2024-09-10T08:05:22.463004+0000","last_became_peered":"2024-09-10T08:05:22.463004+0000","last_unstale":"2024-09-10T08:07:05.509934+0000","last_undegraded":"2024-09-10T08:07:05.509934+0000","last_fullsized":"2024-09-10T08:07:05.509934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:56.257812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569233+0000","last_change":"2024-09-10T08:05:35.642059+0000","last_active":"2024-09-10T08:07:05.569233+0000","last_peered":"2024-09-10T08:07:05.569233+0000","last_clean":"2024-09-10T08:07:05.569233+0000","last_became_active":"2024-09-10T08:05:35.641540+0000","last_became_peered":"2024-09-10T08:05:35.641540+0000","last_unstale":"2024-09-10T08:07:05.569233+0000","last_undegraded":"2024-09-10T08:07:05.569233+0000","last_fullsized":"2024-09-10T08:07:05.569233+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:28.557584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876830+0000","last_change":"2024-09-10T08:05:25.496111+0000","last_active":"2024-09-10T08:07:05.876830+0000","last_peered":"2024-09-10T08:07:05.876830+0000","last_clean":"2024-09-10T08:07:05.876830+0000","last_became_active":"2024-09-10T08:05:25.495757+0000","last_became_peered":"2024-09-10T08:05:25.495757+0000","last_unstale":"2024-09-10T08:07:05.876830+0000","last_undegraded":"2024-09-10T08:07:05.876830+0000","last_fullsized":"2024-09-10T08:07:05.876830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:41.078205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'3","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915005+0000","last_change":"2024-09-10T08:05:22.460069+0000","last_active":"2024-09-10T08:07:05.915005+0000","last_peered":"2024-09-10T08:07:05.915005+0000","last_clean":"2024-09-10T08:07:05.915005+0000","last_became_active":"2024-09-10T08:05:22.459557+0000","last_became_peered":"2024-09-10T08:05:22.459557+0000","last_unstale":"2024-09-10T08:07:05.915005+0000","last_undegraded":"2024-09-10T08:07:05.915005+0000","last_fullsized":"2024-09-10T08:07:05.915005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:49.058507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509959+0000","last_change":"2024-09-10T08:04:08.188262+0000","last_active":"2024-09-10T08:07:05.509959+0000","last_peered":"2024-09-10T08:07:05.509959+0000","last_clean":"2024-09-10T08:07:05.509959+0000","last_became_active":"2024-09-10T08:04:08.187691+0000","last_became_peered":"2024-09-10T08:04:08.187691+0000","last_unstale":"2024-09-10T08:07:05.509959+0000","last_undegraded":"2024-09-10T08:07:05.509959+0000","last_fullsized":"2024-09-10T08:07:05.509959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_clean_scrub_stamp":"2024-09-10T08:00:26.235127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:15.358818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509459+0000","last_change":"2024-09-10T08:05:35.528371+0000","last_active":"2024-09-10T08:07:05.509459+0000","last_peered":"2024-09-10T08:07:05.509459+0000","last_clean":"2024-09-10T08:07:05.509459+0000","last_became_active":"2024-09-10T08:05:35.527925+0000","last_became_peered":"2024-09-10T08:05:35.527925+0000","last_unstale":"2024-09-10T08:07:05.509459+0000","last_undegraded":"2024-09-10T08:07:05.509459+0000","last_fullsized":"2024-09-10T08:07:05.509459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:37:00.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509689+0000","last_change":"2024-09-10T08:05:25.490753+0000","last_active":"2024-09-10T08:07:05.509689+0000","last_peered":"2024-09-10T08:07:05.509689+0000","last_clean":"2024-09-10T08:07:05.509689+0000","last_became_active":"2024-09-10T08:05:25.489996+0000","last_became_peered":"2024-09-10T08:05:25.489996+0000","last_unstale":"2024-09-10T08:07:05.509689+0000","last_undegraded":"2024-09-10T08:07:05.509689+0000","last_fullsized":"2024-09-10T08:07:05.509689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:15.637199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511724+0000","last_change":"2024-09-10T08:05:22.568696+0000","last_active":"2024-09-10T08:07:05.511724+0000","last_peered":"2024-09-10T08:07:05.511724+0000","last_clean":"2024-09-10T08:07:05.511724+0000","last_became_active":"2024-09-10T08:05:22.568331+0000","last_became_peered":"2024-09-10T08:05:22.568331+0000","last_unstale":"2024-09-10T08:07:05.511724+0000","last_undegraded":"2024-09-10T08:07:05.511724+0000","last_fullsized":"2024-09-10T08:07:05.511724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:21.699397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914379+0000","last_change":"2024-09-10T08:05:35.530804+0000","last_active":"2024-09-10T08:07:05.914379+0000","last_peered":"2024-09-10T08:07:05.914379+0000","last_clean":"2024-09-10T08:07:05.914379+0000","last_became_active":"2024-09-10T08:05:35.530356+0000","last_became_peered":"2024-09-10T08:05:35.530356+0000","last_unstale":"2024-09-10T08:07:05.914379+0000","last_undegraded":"2024-09-10T08:07:05.914379+0000","last_fullsized":"2024-09-10T08:07:05.914379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:59.318761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509566+0000","last_change":"2024-09-10T08:05:25.489546+0000","last_active":"2024-09-10T08:07:05.509566+0000","last_peered":"2024-09-10T08:07:05.509566+0000","last_clean":"2024-09-10T08:07:05.509566+0000","last_became_active":"2024-09-10T08:05:25.489188+0000","last_became_peered":"2024-09-10T08:05:25.489188+0000","last_unstale":"2024-09-10T08:07:05.509566+0000","last_undegraded":"2024-09-10T08:07:05.509566+0000","last_fullsized":"2024-09-10T08:07:05.509566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:26.471681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569669+0000","last_change":"2024-09-10T08:05:22.463912+0000","last_active":"2024-09-10T08:07:05.569669+0000","last_peered":"2024-09-10T08:07:05.569669+0000","last_clean":"2024-09-10T08:07:05.569669+0000","last_became_active":"2024-09-10T08:05:22.463561+0000","last_became_peered":"2024-09-10T08:05:22.463561+0000","last_unstale":"2024-09-10T08:07:05.569669+0000","last_undegraded":"2024-09-10T08:07:05.569669+0000","last_fullsized":"2024-09-10T08:07:05.569669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:22.407604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514223+0000","last_change":"2024-09-10T08:05:35.528795+0000","last_active":"2024-09-10T08:07:05.514223+0000","last_peered":"2024-09-10T08:07:05.514223+0000","last_clean":"2024-09-10T08:07:05.514223+0000","last_became_active":"2024-09-10T08:05:35.528402+0000","last_became_peered":"2024-09-10T08:05:35.528402+0000","last_unstale":"2024-09-10T08:07:05.514223+0000","last_undegraded":"2024-09-10T08:07:05.514223+0000","last_fullsized":"2024-09-10T08:07:05.514223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:14.606371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509938+0000","last_change":"2024-09-10T08:05:25.484086+0000","last_active":"2024-09-10T08:07:05.509938+0000","last_peered":"2024-09-10T08:07:05.509938+0000","last_clean":"2024-09-10T08:07:05.509938+0000","last_became_active":"2024-09-10T08:05:25.483757+0000","last_became_peered":"2024-09-10T08:05:25.483757+0000","last_unstale":"2024-09-10T08:07:05.509938+0000","last_undegraded":"2024-09-10T08:07:05.509938+0000","last_fullsized":"2024-09-10T08:07:05.509938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:16:16.362739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914631+0000","last_change":"2024-09-10T08:05:22.462220+0000","last_active":"2024-09-10T08:07:05.914631+0000","last_peered":"2024-09-10T08:07:05.914631+0000","last_clean":"2024-09-10T08:07:05.914631+0000","last_became_active":"2024-09-10T08:05:22.461726+0000","last_became_peered":"2024-09-10T08:05:22.461726+0000","last_unstale":"2024-09-10T08:07:05.914631+0000","last_undegraded":"2024-09-10T08:07:05.914631+0000","last_fullsized":"2024-09-10T08:07:05.914631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:56.381034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508201+0000","last_change":"2024-09-10T08:06:04.734827+0000","last_active":"2024-09-10T08:07:05.508201+0000","last_peered":"2024-09-10T08:07:05.508201+0000","last_clean":"2024-09-10T08:07:05.508201+0000","last_became_active":"2024-09-10T08:06:04.734402+0000","last_became_peered":"2024-09-10T08:06:04.734402+0000","last_unstale":"2024-09-10T08:07:05.508201+0000","last_undegraded":"2024-09-10T08:07:05.508201+0000","last_fullsized":"2024-09-10T08:07:05.508201+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:24.806538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770235+0000","last_change":"2024-09-10T08:05:25.486100+0000","last_active":"2024-09-10T08:07:05.770235+0000","last_peered":"2024-09-10T08:07:05.770235+0000","last_clean":"2024-09-10T08:07:05.770235+0000","last_became_active":"2024-09-10T08:05:25.485586+0000","last_became_peered":"2024-09-10T08:05:25.485586+0000","last_unstale":"2024-09-10T08:07:05.770235+0000","last_undegraded":"2024-09-10T08:07:05.770235+0000","last_fullsized":"2024-09-10T08:07:05.770235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:33.565054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"93'2","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770365+0000","last_change":"2024-09-10T08:06:04.743408+0000","last_active":"2024-09-10T08:07:05.770365+0000","last_peered":"2024-09-10T08:07:05.770365+0000","last_clean":"2024-09-10T08:07:05.770365+0000","last_became_active":"2024-09-10T08:06:04.732830+0000","last_became_peered":"2024-09-10T08:06:04.732830+0000","last_unstale":"2024-09-10T08:07:05.770365+0000","last_undegraded":"2024-09-10T08:07:05.770365+0000","last_fullsized":"2024-09-10T08:07:05.770365+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:29.381259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568797+0000","last_change":"2024-09-10T08:05:35.533015+0000","last_active":"2024-09-10T08:07:05.568797+0000","last_peered":"2024-09-10T08:07:05.568797+0000","last_clean":"2024-09-10T08:07:05.568797+0000","last_became_active":"2024-09-10T08:05:35.532590+0000","last_became_peered":"2024-09-10T08:05:35.532590+0000","last_unstale":"2024-09-10T08:07:05.568797+0000","last_undegraded":"2024-09-10T08:07:05.568797+0000","last_fullsized":"2024-09-10T08:07:05.568797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:23.984273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568914+0000","last_change":"2024-09-10T08:05:25.489994+0000","last_active":"2024-09-10T08:07:05.568914+0000","last_peered":"2024-09-10T08:07:05.568914+0000","last_clean":"2024-09-10T08:07:05.568914+0000","last_became_active":"2024-09-10T08:05:25.489471+0000","last_became_peered":"2024-09-10T08:05:25.489471+0000","last_unstale":"2024-09-10T08:07:05.568914+0000","last_undegraded":"2024-09-10T08:07:05.568914+0000","last_fullsized":"2024-09-10T08:07:05.568914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:53.866570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"93'15","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508330+0000","last_change":"2024-09-10T08:05:22.568900+0000","last_active":"2024-09-10T08:07:05.508330+0000","last_peered":"2024-09-10T08:07:05.508330+0000","last_clean":"2024-09-10T08:07:05.508330+0000","last_became_active":"2024-09-10T08:05:22.568542+0000","last_became_peered":"2024-09-10T08:05:22.568542+0000","last_unstale":"2024-09-10T08:07:05.508330+0000","last_undegraded":"2024-09-10T08:07:05.508330+0000","last_fullsized":"2024-09-10T08:07:05.508330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:58:45.669769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914605+0000","last_change":"2024-09-10T08:05:35.537364+0000","last_active":"2024-09-10T08:07:05.914605+0000","last_peered":"2024-09-10T08:07:05.914605+0000","last_clean":"2024-09-10T08:07:05.914605+0000","last_became_active":"2024-09-10T08:05:35.535958+0000","last_became_peered":"2024-09-10T08:05:35.535958+0000","last_unstale":"2024-09-10T08:07:05.914605+0000","last_undegraded":"2024-09-10T08:07:05.914605+0000","last_fullsized":"2024-09-10T08:07:05.914605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:51.443798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.917030+0000","last_change":"2024-09-10T08:07:05.917030+0000","last_active":"2024-09-10T08:07:05.917030+0000","last_peered":"2024-09-10T08:07:05.917030+0000","last_clean":"2024-09-10T08:07:05.917030+0000","last_became_active":"2024-09-10T08:07:05.916281+0000","last_became_peered":"2024-09-10T08:07:05.916281+0000","last_unstale":"2024-09-10T08:07:05.917030+0000","last_undegraded":"2024-09-10T08:07:05.917030+0000","last_fullsized":"2024-09-10T08:07:05.917030+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:15.060009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'3","reported_seq":113,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:25.360466+0000","last_change":"2024-09-10T08:05:22.459963+0000","last_active":"2024-09-10T08:07:25.360466+0000","last_peered":"2024-09-10T08:07:25.360466+0000","last_clean":"2024-09-10T08:07:25.360466+0000","last_became_active":"2024-09-10T08:05:22.459497+0000","last_became_peered":"2024-09-10T08:05:22.459497+0000","last_unstale":"2024-09-10T08:07:25.360466+0000","last_undegraded":"2024-09-10T08:07:25.360466+0000","last_fullsized":"2024-09-10T08:07:25.360466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:29.887980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513941+0000","last_change":"2024-09-10T08:05:35.534570+0000","last_active":"2024-09-10T08:07:05.513941+0000","last_peered":"2024-09-10T08:07:05.513941+0000","last_clean":"2024-09-10T08:07:05.513941+0000","last_became_active":"2024-09-10T08:05:35.534195+0000","last_became_peered":"2024-09-10T08:05:35.534195+0000","last_unstale":"2024-09-10T08:07:05.513941+0000","last_undegraded":"2024-09-10T08:07:05.513941+0000","last_fullsized":"2024-09-10T08:07:05.513941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:17.393491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509832+0000","last_change":"2024-09-10T08:05:25.488551+0000","last_active":"2024-09-10T08:07:05.509832+0000","last_peered":"2024-09-10T08:07:05.509832+0000","last_clean":"2024-09-10T08:07:05.509832+0000","last_became_active":"2024-09-10T08:05:25.488086+0000","last_became_peered":"2024-09-10T08:05:25.488086+0000","last_unstale":"2024-09-10T08:07:05.509832+0000","last_undegraded":"2024-09-10T08:07:05.509832+0000","last_fullsized":"2024-09-10T08:07:05.509832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:38.044721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"93'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568982+0000","last_change":"2024-09-10T08:05:22.462269+0000","last_active":"2024-09-10T08:07:05.568982+0000","last_peered":"2024-09-10T08:07:05.568982+0000","last_clean":"2024-09-10T08:07:05.568982+0000","last_became_active":"2024-09-10T08:05:22.461924+0000","last_became_peered":"2024-09-10T08:05:22.461924+0000","last_unstale":"2024-09-10T08:07:05.568982+0000","last_undegraded":"2024-09-10T08:07:05.568982+0000","last_fullsized":"2024-09-10T08:07:05.568982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:54.996777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769980+0000","last_change":"2024-09-10T08:05:35.531596+0000","last_active":"2024-09-10T08:07:05.769980+0000","last_peered":"2024-09-10T08:07:05.769980+0000","last_clean":"2024-09-10T08:07:05.769980+0000","last_became_active":"2024-09-10T08:05:35.530179+0000","last_became_peered":"2024-09-10T08:05:35.530179+0000","last_unstale":"2024-09-10T08:07:05.769980+0000","last_undegraded":"2024-09-10T08:07:05.769980+0000","last_fullsized":"2024-09-10T08:07:05.769980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:32.056426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914190+0000","last_change":"2024-09-10T08:05:25.503504+0000","last_active":"2024-09-10T08:07:05.914190+0000","last_peered":"2024-09-10T08:07:05.914190+0000","last_clean":"2024-09-10T08:07:05.914190+0000","last_became_active":"2024-09-10T08:05:25.503105+0000","last_became_peered":"2024-09-10T08:05:25.503105+0000","last_unstale":"2024-09-10T08:07:05.914190+0000","last_undegraded":"2024-09-10T08:07:05.914190+0000","last_fullsized":"2024-09-10T08:07:05.914190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:40.885654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"93'6","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511569+0000","last_change":"2024-09-10T08:05:22.462271+0000","last_active":"2024-09-10T08:07:05.511569+0000","last_peered":"2024-09-10T08:07:05.511569+0000","last_clean":"2024-09-10T08:07:05.511569+0000","last_became_active":"2024-09-10T08:05:22.461195+0000","last_became_peered":"2024-09-10T08:05:22.461195+0000","last_unstale":"2024-09-10T08:07:05.511569+0000","last_undegraded":"2024-09-10T08:07:05.511569+0000","last_fullsized":"2024-09-10T08:07:05.511569+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:20.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569404+0000","last_change":"2024-09-10T08:05:35.533218+0000","last_active":"2024-09-10T08:07:05.569404+0000","last_peered":"2024-09-10T08:07:05.569404+0000","last_clean":"2024-09-10T08:07:05.569404+0000","last_became_active":"2024-09-10T08:05:35.532706+0000","last_became_peered":"2024-09-10T08:05:35.532706+0000","last_unstale":"2024-09-10T08:07:05.569404+0000","last_undegraded":"2024-09-10T08:07:05.569404+0000","last_fullsized":"2024-09-10T08:07:05.569404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:22.150563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508629+0000","last_change":"2024-09-10T08:05:25.478978+0000","last_active":"2024-09-10T08:07:05.508629+0000","last_peered":"2024-09-10T08:07:05.508629+0000","last_clean":"2024-09-10T08:07:05.508629+0000","last_became_active":"2024-09-10T08:05:25.478649+0000","last_became_peered":"2024-09-10T08:05:25.478649+0000","last_unstale":"2024-09-10T08:07:05.508629+0000","last_undegraded":"2024-09-10T08:07:05.508629+0000","last_fullsized":"2024-09-10T08:07:05.508629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:17.493446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'32","reported_seq":237,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:30.367674+0000","last_change":"2024-09-10T08:06:04.740972+0000","last_active":"2024-09-10T08:07:30.367674+0000","last_peered":"2024-09-10T08:07:30.367674+0000","last_clean":"2024-09-10T08:07:30.367674+0000","last_became_active":"2024-09-10T08:06:04.731200+0000","last_became_peered":"2024-09-10T08:06:04.731200+0000","last_unstale":"2024-09-10T08:07:30.367674+0000","last_undegraded":"2024-09-10T08:07:30.367674+0000","last_fullsized":"2024-09-10T08:07:30.367674+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:17.169708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62661,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":146,"num_read_kb":134,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3703,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512731+0000","last_change":"2024-09-10T08:05:35.532594+0000","last_active":"2024-09-10T08:07:05.512731+0000","last_peered":"2024-09-10T08:07:05.512731+0000","last_clean":"2024-09-10T08:07:05.512731+0000","last_became_active":"2024-09-10T08:05:35.532132+0000","last_became_peered":"2024-09-10T08:05:35.532132+0000","last_unstale":"2024-09-10T08:07:05.512731+0000","last_undegraded":"2024-09-10T08:07:05.512731+0000","last_fullsized":"2024-09-10T08:07:05.512731+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:15.348648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914930+0000","last_change":"2024-09-10T08:05:25.485297+0000","last_active":"2024-09-10T08:07:05.914930+0000","last_peered":"2024-09-10T08:07:05.914930+0000","last_clean":"2024-09-10T08:07:05.914930+0000","last_became_active":"2024-09-10T08:05:25.484883+0000","last_became_peered":"2024-09-10T08:05:25.484883+0000","last_unstale":"2024-09-10T08:07:05.914930+0000","last_undegraded":"2024-09-10T08:07:05.914930+0000","last_fullsized":"2024-09-10T08:07:05.914930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:37.052435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"93'4","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569659+0000","last_change":"2024-09-10T08:05:22.568993+0000","last_active":"2024-09-10T08:07:05.569659+0000","last_peered":"2024-09-10T08:07:05.569659+0000","last_clean":"2024-09-10T08:07:05.569659+0000","last_became_active":"2024-09-10T08:05:22.568564+0000","last_became_peered":"2024-09-10T08:05:22.568564+0000","last_unstale":"2024-09-10T08:07:05.569659+0000","last_undegraded":"2024-09-10T08:07:05.569659+0000","last_fullsized":"2024-09-10T08:07:05.569659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:23.767174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914503+0000","last_change":"2024-09-10T08:05:35.524025+0000","last_active":"2024-09-10T08:07:05.914503+0000","last_peered":"2024-09-10T08:07:05.914503+0000","last_clean":"2024-09-10T08:07:05.914503+0000","last_became_active":"2024-09-10T08:05:35.523628+0000","last_became_peered":"2024-09-10T08:05:35.523628+0000","last_unstale":"2024-09-10T08:07:05.914503+0000","last_undegraded":"2024-09-10T08:07:05.914503+0000","last_fullsized":"2024-09-10T08:07:05.914503+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:48.593779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.570537+0000","last_change":"2024-09-10T08:07:05.570537+0000","last_active":"2024-09-10T08:07:05.570537+0000","last_peered":"2024-09-10T08:07:05.570537+0000","last_clean":"2024-09-10T08:07:05.570537+0000","last_became_active":"2024-09-10T08:07:05.570278+0000","last_became_peered":"2024-09-10T08:07:05.570278+0000","last_unstale":"2024-09-10T08:07:05.570537+0000","last_undegraded":"2024-09-10T08:07:05.570537+0000","last_fullsized":"2024-09-10T08:07:05.570537+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:56.490462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"93'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509561+0000","last_change":"2024-09-10T08:05:22.460487+0000","last_active":"2024-09-10T08:07:05.509561+0000","last_peered":"2024-09-10T08:07:05.509561+0000","last_clean":"2024-09-10T08:07:05.509561+0000","last_became_active":"2024-09-10T08:05:22.460158+0000","last_became_peered":"2024-09-10T08:05:22.460158+0000","last_unstale":"2024-09-10T08:07:05.509561+0000","last_undegraded":"2024-09-10T08:07:05.509561+0000","last_fullsized":"2024-09-10T08:07:05.509561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:53.285513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512437+0000","last_change":"2024-09-10T08:05:35.531399+0000","last_active":"2024-09-10T08:07:05.512437+0000","last_peered":"2024-09-10T08:07:05.512437+0000","last_clean":"2024-09-10T08:07:05.512437+0000","last_became_active":"2024-09-10T08:05:35.530320+0000","last_became_peered":"2024-09-10T08:05:35.530320+0000","last_unstale":"2024-09-10T08:07:05.512437+0000","last_undegraded":"2024-09-10T08:07:05.512437+0000","last_fullsized":"2024-09-10T08:07:05.512437+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:19.814046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915161+0000","last_change":"2024-09-10T08:05:25.488208+0000","last_active":"2024-09-10T08:07:05.915161+0000","last_peered":"2024-09-10T08:07:05.915161+0000","last_clean":"2024-09-10T08:07:05.915161+0000","last_became_active":"2024-09-10T08:05:25.487893+0000","last_became_peered":"2024-09-10T08:05:25.487893+0000","last_unstale":"2024-09-10T08:07:05.915161+0000","last_undegraded":"2024-09-10T08:07:05.915161+0000","last_fullsized":"2024-09-10T08:07:05.915161+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:10.999416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"93'3","reported_seq":115,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:29.366161+0000","last_change":"2024-09-10T08:05:22.461321+0000","last_active":"2024-09-10T08:07:29.366161+0000","last_peered":"2024-09-10T08:07:29.366161+0000","last_clean":"2024-09-10T08:07:29.366161+0000","last_became_active":"2024-09-10T08:05:22.460950+0000","last_became_peered":"2024-09-10T08:05:22.460950+0000","last_unstale":"2024-09-10T08:07:29.366161+0000","last_undegraded":"2024-09-10T08:07:29.366161+0000","last_fullsized":"2024-09-10T08:07:29.366161+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:55.647273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876486+0000","last_change":"2024-09-10T08:05:35.530578+0000","last_active":"2024-09-10T08:07:05.876486+0000","last_peered":"2024-09-10T08:07:05.876486+0000","last_clean":"2024-09-10T08:07:05.876486+0000","last_became_active":"2024-09-10T08:05:35.530162+0000","last_became_peered":"2024-09-10T08:05:35.530162+0000","last_unstale":"2024-09-10T08:07:05.876486+0000","last_undegraded":"2024-09-10T08:07:05.876486+0000","last_fullsized":"2024-09-10T08:07:05.876486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:19.290973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513556+0000","last_change":"2024-09-10T08:05:35.532685+0000","last_active":"2024-09-10T08:07:05.513556+0000","last_peered":"2024-09-10T08:07:05.513556+0000","last_clean":"2024-09-10T08:07:05.513556+0000","last_became_active":"2024-09-10T08:05:35.532257+0000","last_became_peered":"2024-09-10T08:05:35.532257+0000","last_unstale":"2024-09-10T08:07:05.513556+0000","last_undegraded":"2024-09-10T08:07:05.513556+0000","last_fullsized":"2024-09-10T08:07:05.513556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:07.412435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512520+0000","last_change":"2024-09-10T08:05:25.494472+0000","last_active":"2024-09-10T08:07:05.512520+0000","last_peered":"2024-09-10T08:07:05.512520+0000","last_clean":"2024-09-10T08:07:05.512520+0000","last_became_active":"2024-09-10T08:05:25.493928+0000","last_became_peered":"2024-09-10T08:05:25.493928+0000","last_unstale":"2024-09-10T08:07:05.512520+0000","last_undegraded":"2024-09-10T08:07:05.512520+0000","last_fullsized":"2024-09-10T08:07:05.512520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:00.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"93'1","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568866+0000","last_change":"2024-09-10T08:05:22.460585+0000","last_active":"2024-09-10T08:07:05.568866+0000","last_peered":"2024-09-10T08:07:05.568866+0000","last_clean":"2024-09-10T08:07:05.568866+0000","last_became_active":"2024-09-10T08:05:22.460259+0000","last_became_peered":"2024-09-10T08:05:22.460259+0000","last_unstale":"2024-09-10T08:07:05.568866+0000","last_undegraded":"2024-09-10T08:07:05.568866+0000","last_fullsized":"2024-09-10T08:07:05.568866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:45.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770172+0000","last_change":"2024-09-10T08:05:35.524000+0000","last_active":"2024-09-10T08:07:05.770172+0000","last_peered":"2024-09-10T08:07:05.770172+0000","last_clean":"2024-09-10T08:07:05.770172+0000","last_became_active":"2024-09-10T08:05:35.523647+0000","last_became_peered":"2024-09-10T08:05:35.523647+0000","last_unstale":"2024-09-10T08:07:05.770172+0000","last_undegraded":"2024-09-10T08:07:05.770172+0000","last_fullsized":"2024-09-10T08:07:05.770172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:10:54.664865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876547+0000","last_change":"2024-09-10T08:05:25.499701+0000","last_active":"2024-09-10T08:07:05.876547+0000","last_peered":"2024-09-10T08:07:05.876547+0000","last_clean":"2024-09-10T08:07:05.876547+0000","last_became_active":"2024-09-10T08:05:25.496890+0000","last_became_peered":"2024-09-10T08:05:25.496890+0000","last_unstale":"2024-09-10T08:07:05.876547+0000","last_undegraded":"2024-09-10T08:07:05.876547+0000","last_fullsized":"2024-09-10T08:07:05.876547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:58.166365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"93'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914017+0000","last_change":"2024-09-10T08:05:22.460880+0000","last_active":"2024-09-10T08:07:05.914017+0000","last_peered":"2024-09-10T08:07:05.914017+0000","last_clean":"2024-09-10T08:07:05.914017+0000","last_became_active":"2024-09-10T08:05:22.459932+0000","last_became_peered":"2024-09-10T08:05:22.459932+0000","last_unstale":"2024-09-10T08:07:05.914017+0000","last_undegraded":"2024-09-10T08:07:05.914017+0000","last_fullsized":"2024-09-10T08:07:05.914017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:55.826526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"93'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915361+0000","last_change":"2024-09-10T08:05:22.460578+0000","last_active":"2024-09-10T08:07:05.915361+0000","last_peered":"2024-09-10T08:07:05.915361+0000","last_clean":"2024-09-10T08:07:05.915361+0000","last_became_active":"2024-09-10T08:05:22.460239+0000","last_became_peered":"2024-09-10T08:05:22.460239+0000","last_unstale":"2024-09-10T08:07:05.915361+0000","last_undegraded":"2024-09-10T08:07:05.915361+0000","last_fullsized":"2024-09-10T08:07:05.915361+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:07.606243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569247+0000","last_change":"2024-09-10T08:05:25.489846+0000","last_active":"2024-09-10T08:07:05.569247+0000","last_peered":"2024-09-10T08:07:05.569247+0000","last_clean":"2024-09-10T08:07:05.569247+0000","last_became_active":"2024-09-10T08:05:25.489326+0000","last_became_peered":"2024-09-10T08:05:25.489326+0000","last_unstale":"2024-09-10T08:07:05.569247+0000","last_undegraded":"2024-09-10T08:07:05.569247+0000","last_fullsized":"2024-09-10T08:07:05.569247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:43.458468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511441+0000","last_change":"2024-09-10T08:05:35.530522+0000","last_active":"2024-09-10T08:07:05.511441+0000","last_peered":"2024-09-10T08:07:05.511441+0000","last_clean":"2024-09-10T08:07:05.511441+0000","last_became_active":"2024-09-10T08:05:35.530099+0000","last_became_peered":"2024-09-10T08:05:35.530099+0000","last_unstale":"2024-09-10T08:07:05.511441+0000","last_undegraded":"2024-09-10T08:07:05.511441+0000","last_fullsized":"2024-09-10T08:07:05.511441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:58.469119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'6","reported_seq":118,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:26.362173+0000","last_change":"2024-09-10T08:05:22.461391+0000","last_active":"2024-09-10T08:07:26.362173+0000","last_peered":"2024-09-10T08:07:26.362173+0000","last_clean":"2024-09-10T08:07:26.362173+0000","last_became_active":"2024-09-10T08:05:22.461042+0000","last_became_peered":"2024-09-10T08:05:22.461042+0000","last_unstale":"2024-09-10T08:07:26.362173+0000","last_undegraded":"2024-09-10T08:07:26.362173+0000","last_fullsized":"2024-09-10T08:07:26.362173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:12.635710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508510+0000","last_change":"2024-09-10T08:05:25.487378+0000","last_active":"2024-09-10T08:07:05.508510+0000","last_peered":"2024-09-10T08:07:05.508510+0000","last_clean":"2024-09-10T08:07:05.508510+0000","last_became_active":"2024-09-10T08:05:25.487055+0000","last_became_peered":"2024-09-10T08:05:25.487055+0000","last_unstale":"2024-09-10T08:07:05.508510+0000","last_undegraded":"2024-09-10T08:07:05.508510+0000","last_fullsized":"2024-09-10T08:07:05.508510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:59.522014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511645+0000","last_change":"2024-09-10T08:05:35.522765+0000","last_active":"2024-09-10T08:07:05.511645+0000","last_peered":"2024-09-10T08:07:05.511645+0000","last_clean":"2024-09-10T08:07:05.511645+0000","last_became_active":"2024-09-10T08:05:35.522359+0000","last_became_peered":"2024-09-10T08:05:35.522359+0000","last_unstale":"2024-09-10T08:07:05.511645+0000","last_undegraded":"2024-09-10T08:07:05.511645+0000","last_fullsized":"2024-09-10T08:07:05.511645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:15.280867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769794+0000","last_change":"2024-09-10T08:05:25.475388+0000","last_active":"2024-09-10T08:07:05.769794+0000","last_peered":"2024-09-10T08:07:05.769794+0000","last_clean":"2024-09-10T08:07:05.769794+0000","last_became_active":"2024-09-10T08:05:25.474331+0000","last_became_peered":"2024-09-10T08:05:25.474331+0000","last_unstale":"2024-09-10T08:07:05.769794+0000","last_undegraded":"2024-09-10T08:07:05.769794+0000","last_fullsized":"2024-09-10T08:07:05.769794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:58.713148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.571225+0000","last_change":"2024-09-10T08:07:05.570505+0000","last_active":"2024-09-10T08:07:05.571225+0000","last_peered":"2024-09-10T08:07:05.571225+0000","last_clean":"2024-09-10T08:07:05.571225+0000","last_became_active":"2024-09-10T08:07:05.569970+0000","last_became_peered":"2024-09-10T08:07:05.569970+0000","last_unstale":"2024-09-10T08:07:05.571225+0000","last_undegraded":"2024-09-10T08:07:05.571225+0000","last_fullsized":"2024-09-10T08:07:05.571225+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:13.282503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511470+0000","last_change":"2024-09-10T08:05:25.491585+0000","last_active":"2024-09-10T08:07:05.511470+0000","last_peered":"2024-09-10T08:07:05.511470+0000","last_clean":"2024-09-10T08:07:05.511470+0000","last_became_active":"2024-09-10T08:05:25.491224+0000","last_became_peered":"2024-09-10T08:05:25.491224+0000","last_unstale":"2024-09-10T08:07:05.511470+0000","last_undegraded":"2024-09-10T08:07:05.511470+0000","last_fullsized":"2024-09-10T08:07:05.511470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:56.066808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514897+0000","last_change":"2024-09-10T08:05:25.499578+0000","last_active":"2024-09-10T08:07:05.514897+0000","last_peered":"2024-09-10T08:07:05.514897+0000","last_clean":"2024-09-10T08:07:05.514897+0000","last_became_active":"2024-09-10T08:05:25.498445+0000","last_became_peered":"2024-09-10T08:05:25.498445+0000","last_unstale":"2024-09-10T08:07:05.514897+0000","last_undegraded":"2024-09-10T08:07:05.514897+0000","last_fullsized":"2024-09-10T08:07:05.514897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:58.961699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914503+0000","last_change":"2024-09-10T08:06:04.735764+0000","last_active":"2024-09-10T08:07:05.914503+0000","last_peered":"2024-09-10T08:07:05.914503+0000","last_clean":"2024-09-10T08:07:05.914503+0000","last_became_active":"2024-09-10T08:06:04.735366+0000","last_became_peered":"2024-09-10T08:06:04.735366+0000","last_unstale":"2024-09-10T08:07:05.914503+0000","last_undegraded":"2024-09-10T08:07:05.914503+0000","last_fullsized":"2024-09-10T08:07:05.914503+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:03.757890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770279+0000","last_change":"2024-09-10T08:05:25.475872+0000","last_active":"2024-09-10T08:07:05.770279+0000","last_peered":"2024-09-10T08:07:05.770279+0000","last_clean":"2024-09-10T08:07:05.770279+0000","last_became_active":"2024-09-10T08:05:25.475549+0000","last_became_peered":"2024-09-10T08:05:25.475549+0000","last_unstale":"2024-09-10T08:07:05.770279+0000","last_undegraded":"2024-09-10T08:07:05.770279+0000","last_fullsized":"2024-09-10T08:07:05.770279+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:37.575267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511808+0000","last_change":"2024-09-10T08:05:25.474210+0000","last_active":"2024-09-10T08:07:05.511808+0000","last_peered":"2024-09-10T08:07:05.511808+0000","last_clean":"2024-09-10T08:07:05.511808+0000","last_became_active":"2024-09-10T08:05:25.473882+0000","last_became_peered":"2024-09-10T08:05:25.473882+0000","last_unstale":"2024-09-10T08:07:05.511808+0000","last_undegraded":"2024-09-10T08:07:05.511808+0000","last_fullsized":"2024-09-10T08:07:05.511808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:05.164370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.877060+0000","last_change":"2024-09-10T08:05:25.490406+0000","last_active":"2024-09-10T08:07:05.877060+0000","last_peered":"2024-09-10T08:07:05.877060+0000","last_clean":"2024-09-10T08:07:05.877060+0000","last_became_active":"2024-09-10T08:05:25.490061+0000","last_became_peered":"2024-09-10T08:05:25.490061+0000","last_unstale":"2024-09-10T08:07:05.877060+0000","last_undegraded":"2024-09-10T08:07:05.877060+0000","last_fullsized":"2024-09-10T08:07:05.877060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:12.988229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568674+0000","last_change":"2024-09-10T08:05:25.504049+0000","last_active":"2024-09-10T08:07:05.568674+0000","last_peered":"2024-09-10T08:07:05.568674+0000","last_clean":"2024-09-10T08:07:05.568674+0000","last_became_active":"2024-09-10T08:05:25.503770+0000","last_became_peered":"2024-09-10T08:05:25.503770+0000","last_unstale":"2024-09-10T08:07:05.568674+0000","last_undegraded":"2024-09-10T08:07:05.568674+0000","last_fullsized":"2024-09-10T08:07:05.568674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:52.850945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876620+0000","last_change":"2024-09-10T08:05:25.491682+0000","last_active":"2024-09-10T08:07:05.876620+0000","last_peered":"2024-09-10T08:07:05.876620+0000","last_clean":"2024-09-10T08:07:05.876620+0000","last_became_active":"2024-09-10T08:05:25.491296+0000","last_became_peered":"2024-09-10T08:05:25.491296+0000","last_unstale":"2024-09-10T08:07:05.876620+0000","last_undegraded":"2024-09-10T08:07:05.876620+0000","last_fullsized":"2024-09-10T08:07:05.876620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:30.075842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914579+0000","last_change":"2024-09-10T08:05:25.489648+0000","last_active":"2024-09-10T08:07:05.914579+0000","last_peered":"2024-09-10T08:07:05.914579+0000","last_clean":"2024-09-10T08:07:05.914579+0000","last_became_active":"2024-09-10T08:05:25.489267+0000","last_became_peered":"2024-09-10T08:05:25.489267+0000","last_unstale":"2024-09-10T08:07:05.914579+0000","last_undegraded":"2024-09-10T08:07:05.914579+0000","last_fullsized":"2024-09-10T08:07:05.914579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:21.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914393+0000","last_change":"2024-09-10T08:05:25.497862+0000","last_active":"2024-09-10T08:07:05.914393+0000","last_peered":"2024-09-10T08:07:05.914393+0000","last_clean":"2024-09-10T08:07:05.914393+0000","last_became_active":"2024-09-10T08:05:25.497504+0000","last_became_peered":"2024-09-10T08:05:25.497504+0000","last_unstale":"2024-09-10T08:07:05.914393+0000","last_undegraded":"2024-09-10T08:07:05.914393+0000","last_fullsized":"2024-09-10T08:07:05.914393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:40.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770039+0000","last_change":"2024-09-10T08:05:25.485924+0000","last_active":"2024-09-10T08:07:05.770039+0000","last_peered":"2024-09-10T08:07:05.770039+0000","last_clean":"2024-09-10T08:07:05.770039+0000","last_became_active":"2024-09-10T08:05:25.485545+0000","last_became_peered":"2024-09-10T08:05:25.485545+0000","last_unstale":"2024-09-10T08:07:05.770039+0000","last_undegraded":"2024-09-10T08:07:05.770039+0000","last_fullsized":"2024-09-10T08:07:05.770039+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:33:20.233720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511912+0000","last_change":"2024-09-10T08:05:25.491911+0000","last_active":"2024-09-10T08:07:05.511912+0000","last_peered":"2024-09-10T08:07:05.511912+0000","last_clean":"2024-09-10T08:07:05.511912+0000","last_became_active":"2024-09-10T08:05:25.491330+0000","last_became_peered":"2024-09-10T08:05:25.491330+0000","last_unstale":"2024-09-10T08:07:05.511912+0000","last_undegraded":"2024-09-10T08:07:05.511912+0000","last_fullsized":"2024-09-10T08:07:05.511912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:06.043331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513499+0000","last_change":"2024-09-10T08:05:25.492220+0000","last_active":"2024-09-10T08:07:05.513499+0000","last_peered":"2024-09-10T08:07:05.513499+0000","last_clean":"2024-09-10T08:07:05.513499+0000","last_became_active":"2024-09-10T08:05:25.491473+0000","last_became_peered":"2024-09-10T08:05:25.491473+0000","last_unstale":"2024-09-10T08:07:05.513499+0000","last_undegraded":"2024-09-10T08:07:05.513499+0000","last_fullsized":"2024-09-10T08:07:05.513499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:13:30.278434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770467+0000","last_change":"2024-09-10T08:05:25.488927+0000","last_active":"2024-09-10T08:07:05.770467+0000","last_peered":"2024-09-10T08:07:05.770467+0000","last_clean":"2024-09-10T08:07:05.770467+0000","last_became_active":"2024-09-10T08:05:25.488550+0000","last_became_peered":"2024-09-10T08:05:25.488550+0000","last_unstale":"2024-09-10T08:07:05.770467+0000","last_undegraded":"2024-09-10T08:07:05.770467+0000","last_fullsized":"2024-09-10T08:07:05.770467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:44.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914091+0000","last_change":"2024-09-10T08:05:25.490125+0000","last_active":"2024-09-10T08:07:05.914091+0000","last_peered":"2024-09-10T08:07:05.914091+0000","last_clean":"2024-09-10T08:07:05.914091+0000","last_became_active":"2024-09-10T08:05:25.487805+0000","last_became_peered":"2024-09-10T08:05:25.487805+0000","last_unstale":"2024-09-10T08:07:05.914091+0000","last_undegraded":"2024-09-10T08:07:05.914091+0000","last_fullsized":"2024-09-10T08:07:05.914091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:33.479009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513789+0000","last_change":"2024-09-10T08:05:25.500346+0000","last_active":"2024-09-10T08:07:05.513789+0000","last_peered":"2024-09-10T08:07:05.513789+0000","last_clean":"2024-09-10T08:07:05.513789+0000","last_became_active":"2024-09-10T08:05:25.498289+0000","last_became_peered":"2024-09-10T08:05:25.498289+0000","last_unstale":"2024-09-10T08:07:05.513789+0000","last_undegraded":"2024-09-10T08:07:05.513789+0000","last_fullsized":"2024-09-10T08:07:05.513789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:12.054926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514016+0000","last_change":"2024-09-10T08:05:25.490001+0000","last_active":"2024-09-10T08:07:05.514016+0000","last_peered":"2024-09-10T08:07:05.514016+0000","last_clean":"2024-09-10T08:07:05.514016+0000","last_became_active":"2024-09-10T08:05:25.489706+0000","last_became_peered":"2024-09-10T08:05:25.489706+0000","last_unstale":"2024-09-10T08:07:05.514016+0000","last_undegraded":"2024-09-10T08:07:05.514016+0000","last_fullsized":"2024-09-10T08:07:05.514016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:38.822140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914315+0000","last_change":"2024-09-10T08:05:25.493874+0000","last_active":"2024-09-10T08:07:05.914315+0000","last_peered":"2024-09-10T08:07:05.914315+0000","last_clean":"2024-09-10T08:07:05.914315+0000","last_became_active":"2024-09-10T08:05:25.489911+0000","last_became_peered":"2024-09-10T08:05:25.489911+0000","last_unstale":"2024-09-10T08:07:05.914315+0000","last_undegraded":"2024-09-10T08:07:05.914315+0000","last_fullsized":"2024-09-10T08:07:05.914315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:46.509657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914747+0000","last_change":"2024-09-10T08:05:25.490277+0000","last_active":"2024-09-10T08:07:05.914747+0000","last_peered":"2024-09-10T08:07:05.914747+0000","last_clean":"2024-09-10T08:07:05.914747+0000","last_became_active":"2024-09-10T08:05:25.486863+0000","last_became_peered":"2024-09-10T08:05:25.486863+0000","last_unstale":"2024-09-10T08:07:05.914747+0000","last_undegraded":"2024-09-10T08:07:05.914747+0000","last_fullsized":"2024-09-10T08:07:05.914747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:21.353830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509623+0000","last_change":"2024-09-10T08:05:25.491028+0000","last_active":"2024-09-10T08:07:05.509623+0000","last_peered":"2024-09-10T08:07:05.509623+0000","last_clean":"2024-09-10T08:07:05.509623+0000","last_became_active":"2024-09-10T08:05:25.490452+0000","last_became_peered":"2024-09-10T08:05:25.490452+0000","last_unstale":"2024-09-10T08:07:05.509623+0000","last_undegraded":"2024-09-10T08:07:05.509623+0000","last_fullsized":"2024-09-10T08:07:05.509623+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:53.059077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915382+0000","last_change":"2024-09-10T08:05:25.489991+0000","last_active":"2024-09-10T08:07:05.915382+0000","last_peered":"2024-09-10T08:07:05.915382+0000","last_clean":"2024-09-10T08:07:05.915382+0000","last_became_active":"2024-09-10T08:05:25.489662+0000","last_became_peered":"2024-09-10T08:05:25.489662+0000","last_unstale":"2024-09-10T08:07:05.915382+0000","last_undegraded":"2024-09-10T08:07:05.915382+0000","last_fullsized":"2024-09-10T08:07:05.915382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:47.940712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915269+0000","last_change":"2024-09-10T08:05:25.475752+0000","last_active":"2024-09-10T08:07:05.915269+0000","last_peered":"2024-09-10T08:07:05.915269+0000","last_clean":"2024-09-10T08:07:05.915269+0000","last_became_active":"2024-09-10T08:05:25.475395+0000","last_became_peered":"2024-09-10T08:05:25.475395+0000","last_unstale":"2024-09-10T08:07:05.915269+0000","last_undegraded":"2024-09-10T08:07:05.915269+0000","last_fullsized":"2024-09-10T08:07:05.915269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.240154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876525+0000","last_change":"2024-09-10T08:05:25.503061+0000","last_active":"2024-09-10T08:07:05.876525+0000","last_peered":"2024-09-10T08:07:05.876525+0000","last_clean":"2024-09-10T08:07:05.876525+0000","last_became_active":"2024-09-10T08:05:25.501333+0000","last_became_peered":"2024-09-10T08:05:25.501333+0000","last_unstale":"2024-09-10T08:07:05.876525+0000","last_undegraded":"2024-09-10T08:07:05.876525+0000","last_fullsized":"2024-09-10T08:07:05.876525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:38.254777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514631+0000","last_change":"2024-09-10T08:07:05.514631+0000","last_active":"2024-09-10T08:07:05.514631+0000","last_peered":"2024-09-10T08:07:05.514631+0000","last_clean":"2024-09-10T08:07:05.514631+0000","last_became_active":"2024-09-10T08:07:05.514285+0000","last_became_peered":"2024-09-10T08:07:05.514285+0000","last_unstale":"2024-09-10T08:07:05.514631+0000","last_undegraded":"2024-09-10T08:07:05.514631+0000","last_fullsized":"2024-09-10T08:07:05.514631+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:40.293121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915209+0000","last_change":"2024-09-10T08:05:25.491457+0000","last_active":"2024-09-10T08:07:05.915209+0000","last_peered":"2024-09-10T08:07:05.915209+0000","last_clean":"2024-09-10T08:07:05.915209+0000","last_became_active":"2024-09-10T08:05:25.491162+0000","last_became_peered":"2024-09-10T08:05:25.491162+0000","last_unstale":"2024-09-10T08:07:05.915209+0000","last_undegraded":"2024-09-10T08:07:05.915209+0000","last_fullsized":"2024-09-10T08:07:05.915209+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:14.389752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508872+0000","last_change":"2024-09-10T08:05:25.487874+0000","last_active":"2024-09-10T08:07:05.508872+0000","last_peered":"2024-09-10T08:07:05.508872+0000","last_clean":"2024-09-10T08:07:05.508872+0000","last_became_active":"2024-09-10T08:05:25.487497+0000","last_became_peered":"2024-09-10T08:05:25.487497+0000","last_unstale":"2024-09-10T08:07:05.508872+0000","last_undegraded":"2024-09-10T08:07:05.508872+0000","last_fullsized":"2024-09-10T08:07:05.508872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:26.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914692+0000","last_change":"2024-09-10T08:05:25.490018+0000","last_active":"2024-09-10T08:07:05.914692+0000","last_peered":"2024-09-10T08:07:05.914692+0000","last_clean":"2024-09-10T08:07:05.914692+0000","last_became_active":"2024-09-10T08:05:25.484951+0000","last_became_peered":"2024-09-10T08:05:25.484951+0000","last_unstale":"2024-09-10T08:07:05.914692+0000","last_undegraded":"2024-09-10T08:07:05.914692+0000","last_fullsized":"2024-09-10T08:07:05.914692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:07.516154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.515027+0000","last_change":"2024-09-10T08:05:25.501581+0000","last_active":"2024-09-10T08:07:05.515027+0000","last_peered":"2024-09-10T08:07:05.515027+0000","last_clean":"2024-09-10T08:07:05.515027+0000","last_became_active":"2024-09-10T08:05:25.499690+0000","last_became_peered":"2024-09-10T08:05:25.499690+0000","last_unstale":"2024-09-10T08:07:05.515027+0000","last_undegraded":"2024-09-10T08:07:05.515027+0000","last_fullsized":"2024-09-10T08:07:05.515027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:14.569791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513726+0000","last_change":"2024-09-10T08:05:25.487945+0000","last_active":"2024-09-10T08:07:05.513726+0000","last_peered":"2024-09-10T08:07:05.513726+0000","last_clean":"2024-09-10T08:07:05.513726+0000","last_became_active":"2024-09-10T08:05:25.487655+0000","last_became_peered":"2024-09-10T08:05:25.487655+0000","last_unstale":"2024-09-10T08:07:05.513726+0000","last_undegraded":"2024-09-10T08:07:05.513726+0000","last_fullsized":"2024-09-10T08:07:05.513726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:37.746037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770478+0000","last_change":"2024-09-10T08:05:25.489348+0000","last_active":"2024-09-10T08:07:05.770478+0000","last_peered":"2024-09-10T08:07:05.770478+0000","last_clean":"2024-09-10T08:07:05.770478+0000","last_became_active":"2024-09-10T08:05:25.489004+0000","last_became_peered":"2024-09-10T08:05:25.489004+0000","last_unstale":"2024-09-10T08:07:05.770478+0000","last_undegraded":"2024-09-10T08:07:05.770478+0000","last_fullsized":"2024-09-10T08:07:05.770478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:54.554719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66664,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":431,"num_read_kb":288,"num_write":167,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3725,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214976,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514536,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":548036,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743469,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32360,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712792,"statfs":{"total":95995035648,"available":95961899008,"internally_reserved":0,"allocated":1200128,"data_stored":1202039,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906992,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1064960,"data_stored":548250,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135928,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1060864,"data_stored":548148,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1114112,"data_stored":1144081,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528385,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":543426,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1122304,"data_stored":1139669,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953548,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1130496,"data_stored":610737,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1056768,"data_stored":543620,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1064960,"data_stored":545079,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1114112,"data_stored":605997,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1069056,"data_stored":544814,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:07:32.726 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T08:07:32.726 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T08:07:32.726 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T08:07:32.726 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T08:07:33.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:32 smithi002 ceph-mon[30563]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:33.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:32 smithi002 ceph-mon[30563]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:07:33.105 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:33.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:32 smithi155 ceph-mon[35405]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:33.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:32 smithi155 ceph-mon[35405]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:07:33.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:32 smithi107 ceph-mon[35048]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:33.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:32 smithi107 ceph-mon[35048]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:07:33.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.707+0000 7f1c53400640 1 -- 172.21.15.2:0/510940632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c4c101df0 msgr2=0x7f1c4c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:33.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.707+0000 7f1c53400640 1 --2- 172.21.15.2:0/510940632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c4c101df0 0x7f1c4c1021d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009900 tx=0x7f1c4002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.707+0000 7f1c53400640 1 -- 172.21.15.2:0/510940632 shutdown_connections 2024-09-10T08:07:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.707+0000 7f1c53400640 1 --2- 172.21.15.2:0/510940632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c4c106f20 0x7f1c4c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.707+0000 7f1c53400640 1 --2- 172.21.15.2:0/510940632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c1027a0 0x7f1c4c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.707+0000 7f1c53400640 1 --2- 172.21.15.2:0/510940632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c4c101df0 0x7f1c4c1021d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.707+0000 7f1c53400640 1 -- 172.21.15.2:0/510940632 >> 172.21.15.2:0/510940632 conn(0x7f1c4c0fd8b0 msgr2=0x7f1c4c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.707+0000 7f1c53400640 1 -- 172.21.15.2:0/510940632 shutdown_connections 2024-09-10T08:07:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.707+0000 7f1c53400640 1 -- 172.21.15.2:0/510940632 wait complete. 2024-09-10T08:07:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.708+0000 7f1c53400640 1 Processor -- start 2024-09-10T08:07:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.708+0000 7f1c53400640 1 -- start start 2024-09-10T08:07:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c53400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c101df0 0x7f1c4c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c53400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c4c1027a0 0x7f1c4c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c53400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c4c106f20 0x7f1c4c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c53400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c111f40 con 0x7f1c4c1027a0 2024-09-10T08:07:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c53400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c111dc0 con 0x7f1c4c101df0 2024-09-10T08:07:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c53400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1120c0 con 0x7f1c4c106f20 2024-09-10T08:07:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c4be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c4c1027a0 0x7f1c4c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c4be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c4c1027a0 0x7f1c4c19ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40064/0 (socket says 172.21.15.2:40064) 2024-09-10T08:07:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c4be00640 1 -- 172.21.15.2:0/2334029746 learned_addr learned my addr 172.21.15.2:0/2334029746 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c50a00640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c101df0 0x7f1c4c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c51400640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c4c106f20 0x7f1c4c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c4be00640 1 -- 172.21.15.2:0/2334029746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c4c106f20 msgr2=0x7f1c4c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c4be00640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c4c106f20 0x7f1c4c19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c4be00640 1 -- 172.21.15.2:0/2334029746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c101df0 msgr2=0x7f1c4c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c4be00640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c101df0 0x7f1c4c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.709+0000 7f1c4be00640 1 -- 172.21.15.2:0/2334029746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4c19f570 con 0x7f1c4c1027a0 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.710+0000 7f1c4be00640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c4c1027a0 0x7f1c4c19ab00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800e9f0 tx=0x7f1c3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.710+0000 7f1c49600640 1 -- 172.21.15.2:0/2334029746 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800cd80 con 0x7f1c4c1027a0 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.710+0000 7f1c49600640 1 -- 172.21.15.2:0/2334029746 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c38010910 con 0x7f1c4c1027a0 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.710+0000 7f1c49600640 1 -- 172.21.15.2:0/2334029746 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38018990 con 0x7f1c4c1027a0 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.710+0000 7f1c53400640 1 -- 172.21.15.2:0/2334029746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4c19f860 con 0x7f1c4c1027a0 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.710+0000 7f1c53400640 1 -- 172.21.15.2:0/2334029746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c4c1a7140 con 0x7f1c4c1027a0 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.711+0000 7f1c53400640 1 -- 172.21.15.2:0/2334029746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c4c1021d0 con 0x7f1c4c1027a0 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.714+0000 7f1c49600640 1 -- 172.21.15.2:0/2334029746 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1c38010ac0 con 0x7f1c4c1027a0 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.716+0000 7f1c49600640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1c340a2c90 0x7f1c340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.716+0000 7f1c49600640 1 -- 172.21.15.2:0/2334029746 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1c380cf890 con 0x7f1c4c1027a0 2024-09-10T08:07:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.716+0000 7f1c50a00640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1c340a2c90 0x7f1c340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:33.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.717+0000 7f1c50a00640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1c340a2c90 0x7f1c340a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009230 tx=0x7f1c400058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:33.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.720+0000 7f1c49600640 1 -- 172.21.15.2:0/2334029746 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c3809bbf0 con 0x7f1c4c1027a0 2024-09-10T08:07:33.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.944+0000 7f1c53400640 1 -- 172.21.15.2:0/2334029746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1c4c1a7470 con 0x7f1c4c1027a0 2024-09-10T08:07:33.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.944+0000 7f1c49600640 1 -- 172.21.15.2:0/2334029746 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f1c3809fef0 con 0x7f1c4c1027a0 2024-09-10T08:07:33.944 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:07:33.944 INFO:teuthology.orchestra.run.smithi002.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T08:07:33.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.946+0000 7f1c53400640 1 -- 172.21.15.2:0/2334029746 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1c340a2c90 msgr2=0x7f1c340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:33.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.946+0000 7f1c53400640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1c340a2c90 0x7f1c340a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009230 tx=0x7f1c400058d0 comp rx=0 tx=0).stop 2024-09-10T08:07:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.946+0000 7f1c53400640 1 -- 172.21.15.2:0/2334029746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c4c1027a0 msgr2=0x7f1c4c19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.946+0000 7f1c53400640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c4c1027a0 0x7f1c4c19ab00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800e9f0 tx=0x7f1c3800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:07:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.947+0000 7f1c53400640 1 -- 172.21.15.2:0/2334029746 shutdown_connections 2024-09-10T08:07:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.947+0000 7f1c53400640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1c340a2c90 0x7f1c340a5150 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.947+0000 7f1c53400640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c4c106f20 0x7f1c4c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.947+0000 7f1c53400640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c4c1027a0 0x7f1c4c19ab00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.947+0000 7f1c53400640 1 --2- 172.21.15.2:0/2334029746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c101df0 0x7f1c4c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.947+0000 7f1c53400640 1 -- 172.21.15.2:0/2334029746 >> 172.21.15.2:0/2334029746 conn(0x7f1c4c0fd8b0 msgr2=0x7f1c4c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.947+0000 7f1c53400640 1 -- 172.21.15.2:0/2334029746 shutdown_connections 2024-09-10T08:07:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:33.947+0000 7f1c53400640 1 -- 172.21.15.2:0/2334029746 wait complete. 2024-09-10T08:07:34.712 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T08:07:34.712 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:07:35.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:34 smithi002 ceph-mon[30563]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:35.602 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:34 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2334029746' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T08:07:35.603 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:34 smithi107 ceph-mon[35048]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:35.603 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:34 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2334029746' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T08:07:35.603 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:34 smithi155 ceph-mon[35405]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:35.603 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:34 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2334029746' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T08:07:35.603 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:35.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.682+0000 7f24cea00640 1 -- 172.21.15.2:0/3526900763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24c8077040 msgr2=0x7f24c81124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:35.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.682+0000 7f24cea00640 1 --2- 172.21.15.2:0/3526900763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24c8077040 0x7f24c81124f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009900 tx=0x7f24b402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:35.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.682+0000 7f24cea00640 1 -- 172.21.15.2:0/3526900763 shutdown_connections 2024-09-10T08:07:35.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.682+0000 7f24cea00640 1 --2- 172.21.15.2:0/3526900763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24c8112a30 0x7f24c8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:35.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.682+0000 7f24cea00640 1 --2- 172.21.15.2:0/3526900763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24c8077040 0x7f24c81124f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.682+0000 7f24cea00640 1 --2- 172.21.15.2:0/3526900763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c8076720 0x7f24c8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.682+0000 7f24cea00640 1 -- 172.21.15.2:0/3526900763 >> 172.21.15.2:0/3526900763 conn(0x7f24c80ffa90 msgr2=0x7f24c8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.682+0000 7f24cea00640 1 -- 172.21.15.2:0/3526900763 shutdown_connections 2024-09-10T08:07:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.682+0000 7f24cea00640 1 -- 172.21.15.2:0/3526900763 wait complete. 2024-09-10T08:07:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.683+0000 7f24cea00640 1 Processor -- start 2024-09-10T08:07:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.683+0000 7f24cea00640 1 -- start start 2024-09-10T08:07:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24cea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24c8076720 0x7f24c81a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24cea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24c8077040 0x7f24c81a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24cea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c8112a30 0x7f24c819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24cea00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8118610 con 0x7f24c8076720 2024-09-10T08:07:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24cea00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8118490 con 0x7f24c8077040 2024-09-10T08:07:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24cea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8118790 con 0x7f24c8112a30 2024-09-10T08:07:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24c7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24c8076720 0x7f24c81a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24c7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24c8076720 0x7f24c81a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40068/0 (socket says 172.21.15.2:40068) 2024-09-10T08:07:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24c7e00640 1 -- 172.21.15.2:0/4012023167 learned_addr learned my addr 172.21.15.2:0/4012023167 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24c7400640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24c8077040 0x7f24c81a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24cca00640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c8112a30 0x7f24c819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24c7e00640 1 -- 172.21.15.2:0/4012023167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c8112a30 msgr2=0x7f24c819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24c7e00640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c8112a30 0x7f24c819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24c7e00640 1 -- 172.21.15.2:0/4012023167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24c8077040 msgr2=0x7f24c81a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24c7e00640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24c8077040 0x7f24c81a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.684+0000 7f24c7e00640 1 -- 172.21.15.2:0/4012023167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c819f050 con 0x7f24c8076720 2024-09-10T08:07:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.685+0000 7f24c7e00640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24c8076720 0x7f24c81a4760 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f24b800e9f0 tx=0x7f24b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.685+0000 7f24c4c00640 1 -- 172.21.15.2:0/4012023167 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b800cd80 con 0x7f24c8076720 2024-09-10T08:07:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.685+0000 7f24cea00640 1 -- 172.21.15.2:0/4012023167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c819f340 con 0x7f24c8076720 2024-09-10T08:07:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.685+0000 7f24c4c00640 1 -- 172.21.15.2:0/4012023167 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b8005500 con 0x7f24c8076720 2024-09-10T08:07:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.685+0000 7f24c4c00640 1 -- 172.21.15.2:0/4012023167 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8018630 con 0x7f24c8076720 2024-09-10T08:07:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.685+0000 7f24cea00640 1 -- 172.21.15.2:0/4012023167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c81ab630 con 0x7f24c8076720 2024-09-10T08:07:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.686+0000 7f24cea00640 1 -- 172.21.15.2:0/4012023167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c8076b00 con 0x7f24c8076720 2024-09-10T08:07:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.688+0000 7f24c4c00640 1 -- 172.21.15.2:0/4012023167 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f24b80046c0 con 0x7f24c8076720 2024-09-10T08:07:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.689+0000 7f24c4c00640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f24ac0a2c10 0x7f24ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.690+0000 7f24c4c00640 1 -- 172.21.15.2:0/4012023167 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f24b80cfaa0 con 0x7f24c8076720 2024-09-10T08:07:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.690+0000 7f24c7400640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f24ac0a2c10 0x7f24ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.690+0000 7f24c7400640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f24ac0a2c10 0x7f24ac0a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f24b4000c00 tx=0x7f24b4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.693+0000 7f24c4c00640 1 -- 172.21.15.2:0/4012023167 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24b809c4b0 con 0x7f24c8076720 2024-09-10T08:07:35.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.917+0000 7f24cea00640 1 -- 172.21.15.2:0/4012023167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24c810dc20 con 0x7f24c8076720 2024-09-10T08:07:35.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.918+0000 7f24c4c00640 1 -- 172.21.15.2:0/4012023167 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7f24b809bdb0 con 0x7f24c8076720 2024-09-10T08:07:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:07:35.919 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":25,"btime":"2024-09-10T08:06:25:369375+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:06:24.883542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24494,24499,24536]},"id":1}]} 2024-09-10T08:07:35.919 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 25 2024-09-10T08:07:35.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.921+0000 7f24cea00640 1 -- 172.21.15.2:0/4012023167 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f24ac0a2c10 msgr2=0x7f24ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:35.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.921+0000 7f24cea00640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f24ac0a2c10 0x7f24ac0a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f24b4000c00 tx=0x7f24b4005b50 comp rx=0 tx=0).stop 2024-09-10T08:07:35.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.921+0000 7f24cea00640 1 -- 172.21.15.2:0/4012023167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24c8076720 msgr2=0x7f24c81a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:35.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.921+0000 7f24cea00640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24c8076720 0x7f24c81a4760 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f24b800e9f0 tx=0x7f24b800edf0 comp rx=0 tx=0).stop 2024-09-10T08:07:35.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.921+0000 7f24cea00640 1 -- 172.21.15.2:0/4012023167 shutdown_connections 2024-09-10T08:07:35.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.921+0000 7f24cea00640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f24ac0a2c10 0x7f24ac0a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:35.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.921+0000 7f24cea00640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c8112a30 0x7f24c819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:35.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.921+0000 7f24cea00640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24c8077040 0x7f24c81a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:35.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.921+0000 7f24cea00640 1 --2- 172.21.15.2:0/4012023167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24c8076720 0x7f24c81a4760 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:35.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.921+0000 7f24cea00640 1 -- 172.21.15.2:0/4012023167 >> 172.21.15.2:0/4012023167 conn(0x7f24c80ffa90 msgr2=0x7f24c8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:35.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.921+0000 7f24cea00640 1 -- 172.21.15.2:0/4012023167 shutdown_connections 2024-09-10T08:07:35.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:35.921+0000 7f24cea00640 1 -- 172.21.15.2:0/4012023167 wait complete. 2024-09-10T08:07:36.894 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:07:37.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:36 smithi002 ceph-mon[30563]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:37.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:36 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/4012023167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:07:37.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:36 smithi155 ceph-mon[35405]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:37.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:36 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/4012023167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:07:37.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:36 smithi107 ceph-mon[35048]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:37.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:36 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/4012023167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:07:37.273 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:37.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.849+0000 7f7105200640 1 -- 172.21.15.2:0/696008979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100106f10 msgr2=0x7f710010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:37.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.849+0000 7f7105200640 1 --2- 172.21.15.2:0/696008979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100106f10 0x7f710010ea90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009ca0 tx=0x7f70f802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:07:37.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.849+0000 7f7105200640 1 -- 172.21.15.2:0/696008979 shutdown_connections 2024-09-10T08:07:37.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.849+0000 7f7105200640 1 --2- 172.21.15.2:0/696008979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100106f10 0x7f710010ea90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:37.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.849+0000 7f7105200640 1 --2- 172.21.15.2:0/696008979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100102790 0x7f71001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:37.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.849+0000 7f7105200640 1 --2- 172.21.15.2:0/696008979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100101de0 0x7f71001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:37.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.849+0000 7f7105200640 1 -- 172.21.15.2:0/696008979 >> 172.21.15.2:0/696008979 conn(0x7f71000fd8c0 msgr2=0x7f71000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:37.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.849+0000 7f7105200640 1 -- 172.21.15.2:0/696008979 shutdown_connections 2024-09-10T08:07:37.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.849+0000 7f7105200640 1 -- 172.21.15.2:0/696008979 wait complete. 2024-09-10T08:07:37.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.850+0000 7f7105200640 1 Processor -- start 2024-09-10T08:07:37.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.850+0000 7f7105200640 1 -- start start 2024-09-10T08:07:37.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.851+0000 7f7105200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100101de0 0x7f7100198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:37.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.851+0000 7f7105200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100102790 0x7f7100198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:37.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.851+0000 7f7105200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100106f10 0x7f710019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:37.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.851+0000 7f7105200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7100111fc0 con 0x7f7100106f10 2024-09-10T08:07:37.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.851+0000 7f7105200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7100111e40 con 0x7f7100102790 2024-09-10T08:07:37.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.851+0000 7f7105200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7100112140 con 0x7f7100101de0 2024-09-10T08:07:37.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.851+0000 7f70fde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100101de0 0x7f7100198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:37.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.851+0000 7f70fe800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100106f10 0x7f710019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:37.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.851+0000 7f70fde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100101de0 0x7f7100198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:32930/0 (socket says 172.21.15.2:32930) 2024-09-10T08:07:37.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.851+0000 7f70fde00640 1 -- 172.21.15.2:0/632002694 learned_addr learned my addr 172.21.15.2:0/632002694 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:37.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.851+0000 7f70f7e00640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100102790 0x7f7100198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:37.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.852+0000 7f70fde00640 1 -- 172.21.15.2:0/632002694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100102790 msgr2=0x7f7100198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:37.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.852+0000 7f70fde00640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100102790 0x7f7100198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:37.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.852+0000 7f70fde00640 1 -- 172.21.15.2:0/632002694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100106f10 msgr2=0x7f710019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:37.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.852+0000 7f70fde00640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100106f10 0x7f710019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:37.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.852+0000 7f70fde00640 1 -- 172.21.15.2:0/632002694 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710019d400 con 0x7f7100101de0 2024-09-10T08:07:37.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.852+0000 7f70fe800640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100106f10 0x7f710019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:37.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.852+0000 7f70fde00640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100101de0 0x7f7100198450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f70f000d950 tx=0x7f70f000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:37.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.852+0000 7f70f6a00640 1 -- 172.21.15.2:0/632002694 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0017030 con 0x7f7100101de0 2024-09-10T08:07:37.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.852+0000 7f7105200640 1 -- 172.21.15.2:0/632002694 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710019d6f0 con 0x7f7100101de0 2024-09-10T08:07:37.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.852+0000 7f70f6a00640 1 -- 172.21.15.2:0/632002694 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f0004020 con 0x7f7100101de0 2024-09-10T08:07:37.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.852+0000 7f70f6a00640 1 -- 172.21.15.2:0/632002694 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f00047a0 con 0x7f7100101de0 2024-09-10T08:07:37.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.853+0000 7f7105200640 1 -- 172.21.15.2:0/632002694 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71001a4fd0 con 0x7f7100101de0 2024-09-10T08:07:37.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.854+0000 7f7105200640 1 -- 172.21.15.2:0/632002694 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71001021c0 con 0x7f7100101de0 2024-09-10T08:07:37.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.855+0000 7f70f6a00640 1 -- 172.21.15.2:0/632002694 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f70f00041d0 con 0x7f7100101de0 2024-09-10T08:07:37.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.857+0000 7f70f6a00640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70e40a2c10 0x7f70e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:37.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.857+0000 7f70f6a00640 1 -- 172.21.15.2:0/632002694 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f70f00cef50 con 0x7f7100101de0 2024-09-10T08:07:37.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.857+0000 7f70f7e00640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70e40a2c10 0x7f70e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:37.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.858+0000 7f70f7e00640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70e40a2c10 0x7f70e40a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f71001087d0 tx=0x7f70e800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:37.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:37.861+0000 7f70f6a00640 1 -- 172.21.15.2:0/632002694 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70f009b900 con 0x7f7100101de0 2024-09-10T08:07:38.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.086+0000 7f7105200640 1 -- 172.21.15.2:0/632002694 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71001997b0 con 0x7f7100101de0 2024-09-10T08:07:38.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.087+0000 7f70f6a00640 1 -- 172.21.15.2:0/632002694 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7f70f009b200 con 0x7f7100101de0 2024-09-10T08:07:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:07:38.087 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":25,"btime":"2024-09-10T08:06:25:369375+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:06:24.883542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24494,24499,24536]},"id":1}]} 2024-09-10T08:07:38.088 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 25 2024-09-10T08:07:38.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.090+0000 7f7105200640 1 -- 172.21.15.2:0/632002694 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70e40a2c10 msgr2=0x7f70e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.090+0000 7f7105200640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70e40a2c10 0x7f70e40a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f71001087d0 tx=0x7f70e800b390 comp rx=0 tx=0).stop 2024-09-10T08:07:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.090+0000 7f7105200640 1 -- 172.21.15.2:0/632002694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100101de0 msgr2=0x7f7100198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.090+0000 7f7105200640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100101de0 0x7f7100198450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f70f000d950 tx=0x7f70f000de10 comp rx=0 tx=0).stop 2024-09-10T08:07:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.090+0000 7f7105200640 1 -- 172.21.15.2:0/632002694 shutdown_connections 2024-09-10T08:07:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.090+0000 7f7105200640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100106f10 0x7f710019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.090+0000 7f7105200640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100102790 0x7f7100198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.090+0000 7f7105200640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70e40a2c10 0x7f70e40a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.090+0000 7f7105200640 1 --2- 172.21.15.2:0/632002694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100101de0 0x7f7100198450 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.090+0000 7f7105200640 1 -- 172.21.15.2:0/632002694 >> 172.21.15.2:0/632002694 conn(0x7f71000fd8c0 msgr2=0x7f71000fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.090+0000 7f7105200640 1 -- 172.21.15.2:0/632002694 shutdown_connections 2024-09-10T08:07:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:38.090+0000 7f7105200640 1 -- 172.21.15.2:0/632002694 wait complete. 2024-09-10T08:07:38.890 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:07:39.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:38 smithi002 ceph-mon[30563]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:39.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:38 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/632002694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:07:39.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:38 smithi155 ceph-mon[35405]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:39.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:38 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/632002694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:07:39.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:38 smithi107 ceph-mon[35048]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:39.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:38 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/632002694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:07:39.266 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:39.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.825+0000 7ff5e6000640 1 -- 172.21.15.2:0/561694560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5e0103ff0 msgr2=0x7ff5e01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:39.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.825+0000 7ff5e6000640 1 --2- 172.21.15.2:0/561694560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5e0103ff0 0x7ff5e01043d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0009900 tx=0x7ff5d002a410 comp rx=0 tx=0).stop 2024-09-10T08:07:39.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.825+0000 7ff5e6000640 1 -- 172.21.15.2:0/561694560 shutdown_connections 2024-09-10T08:07:39.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.825+0000 7ff5e6000640 1 --2- 172.21.15.2:0/561694560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e0109120 0x7ff5e0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:39.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.825+0000 7ff5e6000640 1 --2- 172.21.15.2:0/561694560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e01049a0 0x7ff5e01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:39.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.825+0000 7ff5e6000640 1 --2- 172.21.15.2:0/561694560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5e0103ff0 0x7ff5e01043d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:39.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.825+0000 7ff5e6000640 1 -- 172.21.15.2:0/561694560 >> 172.21.15.2:0/561694560 conn(0x7ff5e00ffa90 msgr2=0x7ff5e0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:39.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.825+0000 7ff5e6000640 1 -- 172.21.15.2:0/561694560 shutdown_connections 2024-09-10T08:07:39.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.825+0000 7ff5e6000640 1 -- 172.21.15.2:0/561694560 wait complete. 2024-09-10T08:07:39.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.826+0000 7ff5e6000640 1 Processor -- start 2024-09-10T08:07:39.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.826+0000 7ff5e6000640 1 -- start start 2024-09-10T08:07:39.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5e6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5e0103ff0 0x7ff5e019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:39.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5e6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e01049a0 0x7ff5e019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:39.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5e6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e0109120 0x7ff5e019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:39.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5e6000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0114190 con 0x7ff5e0103ff0 2024-09-10T08:07:39.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5e6000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0114010 con 0x7ff5e01049a0 2024-09-10T08:07:39.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5e6000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0114310 con 0x7ff5e0109120 2024-09-10T08:07:39.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5df400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5e0103ff0 0x7ff5e019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:39.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5df400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5e0103ff0 0x7ff5e019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51724/0 (socket says 172.21.15.2:51724) 2024-09-10T08:07:39.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5df400640 1 -- 172.21.15.2:0/870853824 learned_addr learned my addr 172.21.15.2:0/870853824 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:39.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5d7e00640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e01049a0 0x7ff5e019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:39.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5dfe00640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e0109120 0x7ff5e019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:39.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5df400640 1 -- 172.21.15.2:0/870853824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e0109120 msgr2=0x7ff5e019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:39.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5df400640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e0109120 0x7ff5e019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:39.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5df400640 1 -- 172.21.15.2:0/870853824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e01049a0 msgr2=0x7ff5e019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:39.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5df400640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e01049a0 0x7ff5e019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:39.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.827+0000 7ff5df400640 1 -- 172.21.15.2:0/870853824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e019f640 con 0x7ff5e0103ff0 2024-09-10T08:07:39.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.828+0000 7ff5df400640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5e0103ff0 0x7ff5e019a690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0004770 tx=0x7ff5d0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:39.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.828+0000 7ff5dcc00640 1 -- 172.21.15.2:0/870853824 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d003f070 con 0x7ff5e0103ff0 2024-09-10T08:07:39.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.828+0000 7ff5dcc00640 1 -- 172.21.15.2:0/870853824 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5d0031ca0 con 0x7ff5e0103ff0 2024-09-10T08:07:39.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.828+0000 7ff5e6000640 1 -- 172.21.15.2:0/870853824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e019f8d0 con 0x7ff5e0103ff0 2024-09-10T08:07:39.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.828+0000 7ff5dcc00640 1 -- 172.21.15.2:0/870853824 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0035030 con 0x7ff5e0103ff0 2024-09-10T08:07:39.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.828+0000 7ff5e6000640 1 -- 172.21.15.2:0/870853824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e01a7210 con 0x7ff5e0103ff0 2024-09-10T08:07:39.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.829+0000 7ff5e6000640 1 -- 172.21.15.2:0/870853824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e01043d0 con 0x7ff5e0103ff0 2024-09-10T08:07:39.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.831+0000 7ff5dcc00640 1 -- 172.21.15.2:0/870853824 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff5d0042070 con 0x7ff5e0103ff0 2024-09-10T08:07:39.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.833+0000 7ff5dcc00640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5c40a2c90 0x7ff5c40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.833+0000 7ff5dcc00640 1 -- 172.21.15.2:0/870853824 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff5d00f02c0 con 0x7ff5e0103ff0 2024-09-10T08:07:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.833+0000 7ff5d7e00640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5c40a2c90 0x7ff5c40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.834+0000 7ff5d7e00640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5c40a2c90 0x7ff5c40a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00fffb0 tx=0x7ff5c800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:39.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:39.835+0000 7ff5dcc00640 1 -- 172.21.15.2:0/870853824 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5d00bcc70 con 0x7ff5e0103ff0 2024-09-10T08:07:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.022+0000 7ff5e6000640 1 -- 172.21.15.2:0/870853824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5e010c300 con 0x7ff5e0103ff0 2024-09-10T08:07:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.024+0000 7ff5dcc00640 1 -- 172.21.15.2:0/870853824 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24887 (secure 0 0 0) 0x7ff5d00dc070 con 0x7ff5e0103ff0 2024-09-10T08:07:40.024 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:07:40.024 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":95,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:07:05.504273+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:07:40.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.026+0000 7ff5e6000640 1 -- 172.21.15.2:0/870853824 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5c40a2c90 msgr2=0x7ff5c40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:40.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.026+0000 7ff5e6000640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5c40a2c90 0x7ff5c40a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00fffb0 tx=0x7ff5c800b390 comp rx=0 tx=0).stop 2024-09-10T08:07:40.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.026+0000 7ff5e6000640 1 -- 172.21.15.2:0/870853824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5e0103ff0 msgr2=0x7ff5e019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:40.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.026+0000 7ff5e6000640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5e0103ff0 0x7ff5e019a690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0004770 tx=0x7ff5d0003c10 comp rx=0 tx=0).stop 2024-09-10T08:07:40.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.026+0000 7ff5e6000640 1 -- 172.21.15.2:0/870853824 shutdown_connections 2024-09-10T08:07:40.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.026+0000 7ff5e6000640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5c40a2c90 0x7ff5c40a5150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:40.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.026+0000 7ff5e6000640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e0109120 0x7ff5e019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:40.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.026+0000 7ff5e6000640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e01049a0 0x7ff5e019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:40.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.026+0000 7ff5e6000640 1 --2- 172.21.15.2:0/870853824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5e0103ff0 0x7ff5e019a690 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:40.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.026+0000 7ff5e6000640 1 -- 172.21.15.2:0/870853824 >> 172.21.15.2:0/870853824 conn(0x7ff5e00ffa90 msgr2=0x7ff5e0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:40.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.026+0000 7ff5e6000640 1 -- 172.21.15.2:0/870853824 shutdown_connections 2024-09-10T08:07:40.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:40.026+0000 7ff5e6000640 1 -- 172.21.15.2:0/870853824 wait complete. 2024-09-10T08:07:40.838 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:07:41.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:40 smithi002 ceph-mon[30563]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:41.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:40 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/870853824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:07:41.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:40 smithi155 ceph-mon[35405]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:41.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:40 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/870853824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:07:41.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:40 smithi107 ceph-mon[35048]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:41.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:40 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/870853824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:07:41.216 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:41.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.774+0000 7f5a1f200640 1 -- 172.21.15.2:0/391573934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a18104010 msgr2=0x7f5a181043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:41.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.774+0000 7f5a1f200640 1 --2- 172.21.15.2:0/391573934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a18104010 0x7f5a181043f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009900 tx=0x7f5a0802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:41.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.774+0000 7f5a1f200640 1 -- 172.21.15.2:0/391573934 shutdown_connections 2024-09-10T08:07:41.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.774+0000 7f5a1f200640 1 --2- 172.21.15.2:0/391573934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a18109140 0x7f5a18110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:41.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.774+0000 7f5a1f200640 1 --2- 172.21.15.2:0/391573934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a181049c0 0x7f5a18108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:41.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.774+0000 7f5a1f200640 1 --2- 172.21.15.2:0/391573934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a18104010 0x7f5a181043f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:41.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.774+0000 7f5a1f200640 1 -- 172.21.15.2:0/391573934 >> 172.21.15.2:0/391573934 conn(0x7f5a180ffa50 msgr2=0x7f5a18101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:41.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.774+0000 7f5a1f200640 1 -- 172.21.15.2:0/391573934 shutdown_connections 2024-09-10T08:07:41.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.774+0000 7f5a1f200640 1 -- 172.21.15.2:0/391573934 wait complete. 2024-09-10T08:07:41.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.775+0000 7f5a1f200640 1 Processor -- start 2024-09-10T08:07:41.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.775+0000 7f5a1f200640 1 -- start start 2024-09-10T08:07:41.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.776+0000 7f5a1f200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a18104010 0x7f5a1819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:41.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.776+0000 7f5a1f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a181049c0 0x7f5a1819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:41.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.776+0000 7f5a1f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a18109140 0x7f5a1819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:41.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.776+0000 7f5a1f200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5a18114160 con 0x7f5a18104010 2024-09-10T08:07:41.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.776+0000 7f5a1f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5a18113fe0 con 0x7f5a181049c0 2024-09-10T08:07:41.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.776+0000 7f5a1f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181142e0 con 0x7f5a18109140 2024-09-10T08:07:41.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.776+0000 7f5a17e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a18104010 0x7f5a1819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:41.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.776+0000 7f5a17e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a18104010 0x7f5a1819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51736/0 (socket says 172.21.15.2:51736) 2024-09-10T08:07:41.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.776+0000 7f5a1d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a18109140 0x7f5a1819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.776+0000 7f5a17e00640 1 -- 172.21.15.2:0/1853689973 learned_addr learned my addr 172.21.15.2:0/1853689973 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.776+0000 7f5a17400640 1 --2- 172.21.15.2:0/1853689973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a181049c0 0x7f5a1819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.777+0000 7f5a17e00640 1 -- 172.21.15.2:0/1853689973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a18109140 msgr2=0x7f5a1819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.777+0000 7f5a17e00640 1 --2- 172.21.15.2:0/1853689973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a18109140 0x7f5a1819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.777+0000 7f5a17e00640 1 -- 172.21.15.2:0/1853689973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a181049c0 msgr2=0x7f5a1819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.777+0000 7f5a17e00640 1 --2- 172.21.15.2:0/1853689973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a181049c0 0x7f5a1819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.777+0000 7f5a17e00640 1 -- 172.21.15.2:0/1853689973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1819f5d0 con 0x7f5a18104010 2024-09-10T08:07:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.777+0000 7f5a17e00640 1 --2- 172.21.15.2:0/1853689973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a18104010 0x7f5a1819a620 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5a08006010 tx=0x7f5a080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.777+0000 7f5a14c00640 1 -- 172.21.15.2:0/1853689973 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0803f030 con 0x7f5a18104010 2024-09-10T08:07:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.777+0000 7f5a14c00640 1 -- 172.21.15.2:0/1853689973 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a08003ce0 con 0x7f5a18104010 2024-09-10T08:07:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.777+0000 7f5a14c00640 1 -- 172.21.15.2:0/1853689973 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08035070 con 0x7f5a18104010 2024-09-10T08:07:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.777+0000 7f5a1f200640 1 -- 172.21.15.2:0/1853689973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1819f860 con 0x7f5a18104010 2024-09-10T08:07:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.778+0000 7f5a1f200640 1 -- 172.21.15.2:0/1853689973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a181a71a0 con 0x7f5a18104010 2024-09-10T08:07:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.779+0000 7f5a1f200640 1 -- 172.21.15.2:0/1853689973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a181043f0 con 0x7f5a18104010 2024-09-10T08:07:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.780+0000 7f5a14c00640 1 -- 172.21.15.2:0/1853689973 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5a08042030 con 0x7f5a18104010 2024-09-10T08:07:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.782+0000 7f5a14c00640 1 --2- 172.21.15.2:0/1853689973 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59fc0a2c90 0x7f59fc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.782+0000 7f5a14c00640 1 -- 172.21.15.2:0/1853689973 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5a080efad0 con 0x7f5a18104010 2024-09-10T08:07:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.782+0000 7f5a17400640 1 --2- 172.21.15.2:0/1853689973 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59fc0a2c90 0x7f59fc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.783+0000 7f5a17400640 1 --2- 172.21.15.2:0/1853689973 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59fc0a2c90 0x7f59fc0a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5a180ffe70 tx=0x7f5a0400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:41.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:41.786+0000 7f5a14c00640 1 -- 172.21.15.2:0/1853689973 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a080bbef0 con 0x7f5a18104010 2024-09-10T08:07:42.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.007+0000 7f5a1f200640 1 -- 172.21.15.2:0/1853689973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a1810c320 con 0x7f5a18104010 2024-09-10T08:07:42.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.008+0000 7f5a14c00640 1 -- 172.21.15.2:0/1853689973 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7f5a080c01f0 con 0x7f5a18104010 2024-09-10T08:07:42.008 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:07:42.008 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":25,"btime":"2024-09-10T08:06:25:369375+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:06:24.883542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24494,24499,24536]},"id":1}]} 2024-09-10T08:07:42.009 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 25 2024-09-10T08:07:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.011+0000 7f5a1f200640 1 -- 172.21.15.2:0/1853689973 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59fc0a2c90 msgr2=0x7f59fc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.011+0000 7f5a1f200640 1 --2- 172.21.15.2:0/1853689973 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59fc0a2c90 0x7f59fc0a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5a180ffe70 tx=0x7f5a0400b390 comp rx=0 tx=0).stop 2024-09-10T08:07:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.011+0000 7f5a1f200640 1 -- 172.21.15.2:0/1853689973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a18104010 msgr2=0x7f5a1819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.011+0000 7f5a1f200640 1 --2- 172.21.15.2:0/1853689973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a18104010 0x7f5a1819a620 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5a08006010 tx=0x7f5a080042f0 comp rx=0 tx=0).stop 2024-09-10T08:07:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.011+0000 7f5a1f200640 1 -- 172.21.15.2:0/1853689973 shutdown_connections 2024-09-10T08:07:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.011+0000 7f5a1f200640 1 --2- 172.21.15.2:0/1853689973 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59fc0a2c90 0x7f59fc0a5150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.011+0000 7f5a1f200640 1 --2- 172.21.15.2:0/1853689973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a18109140 0x7f5a1819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.011+0000 7f5a1f200640 1 --2- 172.21.15.2:0/1853689973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a181049c0 0x7f5a1819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.011+0000 7f5a1f200640 1 --2- 172.21.15.2:0/1853689973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a18104010 0x7f5a1819a620 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.011+0000 7f5a1f200640 1 -- 172.21.15.2:0/1853689973 >> 172.21.15.2:0/1853689973 conn(0x7f5a180ffa50 msgr2=0x7f5a18100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.011+0000 7f5a1f200640 1 -- 172.21.15.2:0/1853689973 shutdown_connections 2024-09-10T08:07:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:42.011+0000 7f5a1f200640 1 -- 172.21.15.2:0/1853689973 wait complete. 2024-09-10T08:07:42.739 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T08:05:26.574488+0000', 'modified': '2024-09-10T08:06:24.883542+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24536, 'mds_1': 24499, 'mds_2': 24494}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6835/1775771400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 1775771400}, {'type': 'v1', 'addr': '172.21.15.107:6835', 'nonce': 1775771400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6835/1068995103', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 1068995103}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 1068995103}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/3780222531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 3780222531}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 3780222531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.107:6837/3130379099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6836', 'nonce': 3130379099}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 3130379099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.155:6839/2871792117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 2871792117}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 2871792117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.107:6839/3674188134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 3674188134}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 3674188134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24536, 'qdb_cluster': [24494, 24499, 24536]} 2024-09-10T08:07:42.739 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T08:07:42.739 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T08:07:43.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:42 smithi002 ceph-mon[30563]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:43.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:42 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1853689973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:07:43.122 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:43.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:42 smithi155 ceph-mon[35405]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:43.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:42 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1853689973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:07:43.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:42 smithi107 ceph-mon[35048]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:43.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:42 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1853689973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:07:43.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.644+0000 7fb8b7a00640 1 -- 172.21.15.2:0/3496425274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8b0101de0 msgr2=0x7fb8b01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:43.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.644+0000 7fb8b7a00640 1 --2- 172.21.15.2:0/3496425274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8b0101de0 0x7fb8b01021c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009900 tx=0x7fb8a402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.644+0000 7fb8b7a00640 1 -- 172.21.15.2:0/3496425274 shutdown_connections 2024-09-10T08:07:43.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.644+0000 7fb8b7a00640 1 --2- 172.21.15.2:0/3496425274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b0106f10 0x7fb8b010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.644+0000 7fb8b7a00640 1 --2- 172.21.15.2:0/3496425274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8b0102790 0x7fb8b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.644+0000 7fb8b7a00640 1 --2- 172.21.15.2:0/3496425274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8b0101de0 0x7fb8b01021c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.644+0000 7fb8b7a00640 1 -- 172.21.15.2:0/3496425274 >> 172.21.15.2:0/3496425274 conn(0x7fb8b00fd8c0 msgr2=0x7fb8b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:43.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.644+0000 7fb8b7a00640 1 -- 172.21.15.2:0/3496425274 shutdown_connections 2024-09-10T08:07:43.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.644+0000 7fb8b7a00640 1 -- 172.21.15.2:0/3496425274 wait complete. 2024-09-10T08:07:43.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.645+0000 7fb8b7a00640 1 Processor -- start 2024-09-10T08:07:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.645+0000 7fb8b7a00640 1 -- start start 2024-09-10T08:07:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.646+0000 7fb8b7a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b0101de0 0x7fb8b019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.646+0000 7fb8b7a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8b0102790 0x7fb8b019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.646+0000 7fb8b5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b0101de0 0x7fb8b019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.646+0000 7fb8b5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b0101de0 0x7fb8b019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46596/0 (socket says 172.21.15.2:46596) 2024-09-10T08:07:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.646+0000 7fb8b7a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8b0106f10 0x7fb8b019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.646+0000 7fb8b7a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b0111f70 con 0x7fb8b0102790 2024-09-10T08:07:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.646+0000 7fb8b7a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b0111df0 con 0x7fb8b0101de0 2024-09-10T08:07:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.646+0000 7fb8b7a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01120f0 con 0x7fb8b0106f10 2024-09-10T08:07:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.647+0000 7fb8afe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8b0102790 0x7fb8b019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.647+0000 7fb8afe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8b0102790 0x7fb8b019ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51746/0 (socket says 172.21.15.2:51746) 2024-09-10T08:07:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.647+0000 7fb8afe00640 1 -- 172.21.15.2:0/1388959694 learned_addr learned my addr 172.21.15.2:0/1388959694 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.647+0000 7fb8b5a00640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8b0106f10 0x7fb8b019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.647+0000 7fb8afe00640 1 -- 172.21.15.2:0/1388959694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8b0106f10 msgr2=0x7fb8b019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.647+0000 7fb8afe00640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8b0106f10 0x7fb8b019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.647+0000 7fb8afe00640 1 -- 172.21.15.2:0/1388959694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b0101de0 msgr2=0x7fb8b019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.647+0000 7fb8afe00640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b0101de0 0x7fb8b019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.647+0000 7fb8afe00640 1 -- 172.21.15.2:0/1388959694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b019f670 con 0x7fb8b0102790 2024-09-10T08:07:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.647+0000 7fb8b5000640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b0101de0 0x7fb8b019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:07:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.648+0000 7fb8afe00640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8b0102790 0x7fb8b019ac00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00d770 tx=0x7fb89c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.648+0000 7fb8ad600640 1 -- 172.21.15.2:0/1388959694 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c017070 con 0x7fb8b0102790 2024-09-10T08:07:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.648+0000 7fb8ad600640 1 -- 172.21.15.2:0/1388959694 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb89c005500 con 0x7fb8b0102790 2024-09-10T08:07:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.648+0000 7fb8ad600640 1 -- 172.21.15.2:0/1388959694 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c004e00 con 0x7fb8b0102790 2024-09-10T08:07:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.648+0000 7fb8b7a00640 1 -- 172.21.15.2:0/1388959694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b019f960 con 0x7fb8b0102790 2024-09-10T08:07:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.648+0000 7fb8b7a00640 1 -- 172.21.15.2:0/1388959694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b01a7240 con 0x7fb8b0102790 2024-09-10T08:07:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.651+0000 7fb8ba631740 1 Processor -- start 2024-09-10T08:07:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8ba631740 1 -- start start 2024-09-10T08:07:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8ba631740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20a330 0x56099b20a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8ba631740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20ac50 0x56099b214fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8ba631740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b215510 0x56099b217900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8ba631740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56099b1700b0 con 0x56099b215510 2024-09-10T08:07:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8ba631740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56099b117ab0 con 0x56099b20ac50 2024-09-10T08:07:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8ba631740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56099b16c070 con 0x56099b20a330 2024-09-10T08:07:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8ad600640 1 -- 172.21.15.2:0/1388959694 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb89c004fa0 con 0x7fb8b0102790 2024-09-10T08:07:43.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8afe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b215510 0x56099b217900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8b5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20ac50 0x56099b214fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8b5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20a330 0x56099b20a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8afe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b215510 0x56099b217900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51754/0 (socket says 172.21.15.2:51754) 2024-09-10T08:07:43.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8afe00640 1 -- 172.21.15.2:0/1576759532 learned_addr learned my addr 172.21.15.2:0/1576759532 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:43.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.652+0000 7fb8b5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20a330 0x56099b20a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:33004/0 (socket says 172.21.15.2:33004) 2024-09-10T08:07:43.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.653+0000 7fb8b5000640 1 -- 172.21.15.2:0/1576759532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20a330 msgr2=0x56099b20a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:43.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.653+0000 7fb8b5000640 1 --2- 172.21.15.2:0/1576759532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20a330 0x56099b20a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.653+0000 7fb8b5000640 1 -- 172.21.15.2:0/1576759532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b215510 msgr2=0x56099b217900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:43.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.653+0000 7fb8b5000640 1 --2- 172.21.15.2:0/1576759532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b215510 0x56099b217900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.653+0000 7fb8b5000640 1 -- 172.21.15.2:0/1576759532 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56099b1244a0 con 0x56099b20ac50 2024-09-10T08:07:43.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.653+0000 7fb8afe00640 1 --2- 172.21.15.2:0/1576759532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b215510 0x56099b217900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:43.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.653+0000 7fb8b5000640 1 --2- 172.21.15.2:0/1576759532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20ac50 0x56099b214fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009230 tx=0x7fb8a4005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e84a2a5a19200c6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:43.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.654+0000 7fb8a2000640 1 -- 172.21.15.2:0/1576759532 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a403f070 con 0x56099b20ac50 2024-09-10T08:07:43.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.654+0000 7fb8a2000640 1 -- 172.21.15.2:0/1576759532 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a4004030 con 0x56099b20ac50 2024-09-10T08:07:43.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.654+0000 7fb8a2000640 1 -- 172.21.15.2:0/1576759532 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4035070 con 0x56099b20ac50 2024-09-10T08:07:43.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.654+0000 7fb8ad600640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb8980a2c10 0x7fb8980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.654+0000 7fb8ad600640 1 -- 172.21.15.2:0/1388959694 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb89c0cf0a0 con 0x7fb8b0102790 2024-09-10T08:07:43.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.654+0000 7fb8b5a00640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb8980a2c10 0x7fb8980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.655+0000 7fb8ba631740 1 -- 172.21.15.2:0/1576759532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20ac50 msgr2=0x56099b214fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:43.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.655+0000 7fb8ba631740 1 --2- 172.21.15.2:0/1576759532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20ac50 0x56099b214fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009230 tx=0x7fb8a4005e60 comp rx=0 tx=0).stop 2024-09-10T08:07:43.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.655+0000 7fb8ba631740 1 -- 172.21.15.2:0/1576759532 shutdown_connections 2024-09-10T08:07:43.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.655+0000 7fb8ba631740 1 --2- 172.21.15.2:0/1576759532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b215510 0x56099b217900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.655+0000 7fb8ba631740 1 --2- 172.21.15.2:0/1576759532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20ac50 0x56099b214fd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.655+0000 7fb8ba631740 1 --2- 172.21.15.2:0/1576759532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20a330 0x56099b20a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.655+0000 7fb8ba631740 1 -- 172.21.15.2:0/1576759532 >> 172.21.15.2:0/1576759532 conn(0x56099b2068c0 msgr2=0x56099b206ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:43.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.655+0000 7fb8b5a00640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb8980a2c10 0x7fb8980a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8005fd0 tx=0x7fb8a8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:43.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.655+0000 7fb8ba631740 1 -- 172.21.15.2:0/1576759532 shutdown_connections 2024-09-10T08:07:43.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.655+0000 7fb8ba631740 1 -- 172.21.15.2:0/1576759532 wait complete. 2024-09-10T08:07:43.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.655+0000 7fb8ba631740 1 -- start start 2024-09-10T08:07:43.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.656+0000 7fb8ba631740 1 Processor -- start 2024-09-10T08:07:43.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.656+0000 7fb8ba631740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b20a330 0x56099b2125c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.656+0000 7fb8ba631740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20ac50 0x56099b212b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.656+0000 7fb8ba631740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b215510 0x56099b213040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.656+0000 7fb8ba631740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56099b218060 con 0x56099b20a330 2024-09-10T08:07:43.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.656+0000 7fb8ba631740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56099b0f8920 con 0x56099b215510 2024-09-10T08:07:43.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.656+0000 7fb8ba631740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56099b2181e0 con 0x56099b20ac50 2024-09-10T08:07:43.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.656+0000 7fb8ba631740 10 client.0 _ll_register_callbacks cb 0x56099b11c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:07:43.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.656+0000 7fb8b5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b20a330 0x56099b2125c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.656+0000 7fb8b5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b20a330 0x56099b2125c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51756/0 (socket says 172.21.15.2:51756) 2024-09-10T08:07:43.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.657+0000 7fb8b5000640 1 -- 172.21.15.2:0/1384904870 learned_addr learned my addr 172.21.15.2:0/1384904870 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:43.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.657+0000 7fb8afe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20ac50 0x56099b212b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.657+0000 7fb8b5a00640 1 --2- 172.21.15.2:0/1384904870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b215510 0x56099b213040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.657+0000 7fb8afe00640 1 -- 172.21.15.2:0/1384904870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b215510 msgr2=0x56099b213040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:43.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.657+0000 7fb8afe00640 1 --2- 172.21.15.2:0/1384904870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b215510 0x56099b213040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.657+0000 7fb8afe00640 1 -- 172.21.15.2:0/1384904870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b20a330 msgr2=0x56099b2125c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:43.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.657+0000 7fb8afe00640 1 --2- 172.21.15.2:0/1384904870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b20a330 0x56099b2125c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.657+0000 7fb8afe00640 1 -- 172.21.15.2:0/1384904870 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56099aee12b0 con 0x56099b20ac50 2024-09-10T08:07:43.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.657+0000 7fb8afe00640 1 --2- 172.21.15.2:0/1384904870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20ac50 0x56099b212b00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0d28f0 tx=0x7fb89c0048a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc8acd61a73ba4fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:43.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.657+0000 7fb896a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:07:43.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.657+0000 7fb896a00640 1 -- 172.21.15.2:0/1384904870 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c020070 con 0x56099b20ac50 2024-09-10T08:07:43.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.657+0000 7fb896a00640 1 -- 172.21.15.2:0/1384904870 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb89c0979c0 con 0x56099b20ac50 2024-09-10T08:07:43.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.657+0000 7fb8ba631740 1 -- 172.21.15.2:0/1384904870 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a4004030 con 0x56099b20ac50 2024-09-10T08:07:43.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.658+0000 7fb896a00640 1 -- 172.21.15.2:0/1384904870 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb89c09acb0 con 0x56099b20ac50 2024-09-10T08:07:43.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.658+0000 7fb8ba631740 10 client.24955 fetch_fsmap learned FSMap version 25 2024-09-10T08:07:43.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.658+0000 7fb8ba631740 1 -- 172.21.15.2:0/1384904870 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56099b223eb0 con 0x56099b20ac50 2024-09-10T08:07:43.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.658+0000 7fb896a00640 1 -- 172.21.15.2:0/1384904870 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a4004030 con 0x56099b20ac50 2024-09-10T08:07:43.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.658+0000 7fb896a00640 1 -- 172.21.15.2:0/1384904870 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb89c0cc850 con 0x56099b20ac50 2024-09-10T08:07:43.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.658+0000 7fb8ba631740 10 client.24955 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:07:43.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.658+0000 7fb8ba631740 10 client.24955 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:07:43.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.658+0000 7fb8ba631740 20 client.24955 populate_metadata read hostname 'smithi002' 2024-09-10T08:07:43.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.658+0000 7fb8ba631740 1 --2- 172.21.15.2:0/1384904870 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56099b2264d0 0x56099b227940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.658+0000 7fb8ba631740 4 client.24955 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:07:43.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.658+0000 7fb8ba631740 1 -- 172.21.15.2:0/1384904870 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56099b17d620 con 0x56099b2264d0 2024-09-10T08:07:43.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.659+0000 7fb8b5000640 1 --2- 172.21.15.2:0/1384904870 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56099b2264d0 0x56099b227940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.659+0000 7fb8b5000640 1 --2- 172.21.15.2:0/1384904870 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56099b2264d0 0x56099b227940 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69c4d6f14ce52966 server_cookie=3e272171fd6dc401 in_seq=0 out_seq=0 2024-09-10T08:07:43.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.659+0000 7fb896a00640 10 client.24955 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:07:43.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.660+0000 7fb896a00640 1 -- 172.21.15.2:0/1384904870 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56099b17d620 con 0x56099b2264d0 2024-09-10T08:07:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.660+0000 7fb896a00640 10 client.24955 handle_command_reply: tid=0 2024-09-10T08:07:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.660+0000 7fb8ba631740 1 client.24955 shutdown 2024-09-10T08:07:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.661+0000 7fb8ba631740 1 -- 172.21.15.2:0/1384904870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20ac50 msgr2=0x56099b212b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.661+0000 7fb8ba631740 1 --2- 172.21.15.2:0/1384904870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20ac50 0x56099b212b00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0d28f0 tx=0x7fb89c0048a0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.661+0000 7fb8ba631740 1 -- 172.21.15.2:0/1384904870 shutdown_connections 2024-09-10T08:07:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.661+0000 7fb8ba631740 1 --2- 172.21.15.2:0/1384904870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b215510 0x56099b213040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.661+0000 7fb8ba631740 1 --2- 172.21.15.2:0/1384904870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20ac50 0x56099b212b00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.661+0000 7fb8ba631740 1 --2- 172.21.15.2:0/1384904870 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56099b2264d0 0x56099b227940 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.661+0000 7fb8ba631740 1 --2- 172.21.15.2:0/1384904870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b20a330 0x56099b2125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.661+0000 7fb8ba631740 1 -- 172.21.15.2:0/1384904870 >> 172.21.15.2:0/1384904870 conn(0x56099b2068c0 msgr2=0x56099b20c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.661+0000 7fb896a00640 0 client.24955 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:07:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.661+0000 7fb8ba631740 1 -- 172.21.15.2:0/1384904870 shutdown_connections 2024-09-10T08:07:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.661+0000 7fb8ba631740 1 -- 172.21.15.2:0/1384904870 wait complete. 2024-09-10T08:07:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.661+0000 7fb8ba631740 20 client.24955 trim_cache size 0 max 16384 2024-09-10T08:07:43.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.680+0000 7fb8ba631740 1 Processor -- start 2024-09-10T08:07:43.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.681+0000 7fb8ba631740 1 -- start start 2024-09-10T08:07:43.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.681+0000 7fb8ba631740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20a330 0x56099b20cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.681+0000 7fb8ba631740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20ac50 0x56099b22fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.681+0000 7fb8ba631740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b215510 0x56099b230160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.681+0000 7fb8ba631740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56099b225c80 con 0x56099b215510 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.681+0000 7fb8ba631740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56099b226350 con 0x56099b20ac50 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.681+0000 7fb8ba631740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56099b226a20 con 0x56099b20a330 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.681+0000 7fb8b5000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20a330 0x56099b20cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.681+0000 7fb8b5a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b215510 0x56099b230160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.681+0000 7fb8b5000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20a330 0x56099b20cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:33018/0 (socket says 172.21.15.2:33018) 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.681+0000 7fb8b5000640 1 -- 172.21.15.2:0/3161001285 learned_addr learned my addr 172.21.15.2:0/3161001285 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.681+0000 7fb8afe00640 1 --2- 172.21.15.2:0/3161001285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20ac50 0x56099b22fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.682+0000 7fb8b5000640 1 -- 172.21.15.2:0/3161001285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20ac50 msgr2=0x56099b22fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.682+0000 7fb8b5000640 1 --2- 172.21.15.2:0/3161001285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20ac50 0x56099b22fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.682+0000 7fb8b5000640 1 -- 172.21.15.2:0/3161001285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b215510 msgr2=0x56099b230160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.682+0000 7fb8b5000640 1 --2- 172.21.15.2:0/3161001285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b215510 0x56099b230160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.682+0000 7fb8b5000640 1 -- 172.21.15.2:0/3161001285 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56099b226ed0 con 0x56099b20a330 2024-09-10T08:07:43.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.682+0000 7fb8b5000640 1 --2- 172.21.15.2:0/3161001285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20a330 0x56099b20cd90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40051c0 tx=0x7fb8a4005150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b97ad7e9fa01f316 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:43.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.682+0000 7fb8a1600640 1 -- 172.21.15.2:0/3161001285 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4035040 con 0x56099b20a330 2024-09-10T08:07:43.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.682+0000 7fb8a1600640 1 -- 172.21.15.2:0/3161001285 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a4004460 con 0x56099b20a330 2024-09-10T08:07:43.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.682+0000 7fb8a1600640 1 -- 172.21.15.2:0/3161001285 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a403f040 con 0x56099b20a330 2024-09-10T08:07:43.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.683+0000 7fb8ba631740 1 -- 172.21.15.2:0/3161001285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20a330 msgr2=0x56099b20cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:43.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.683+0000 7fb8ba631740 1 --2- 172.21.15.2:0/3161001285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20a330 0x56099b20cd90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40051c0 tx=0x7fb8a4005150 comp rx=0 tx=0).stop 2024-09-10T08:07:43.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.683+0000 7fb8ba631740 1 -- 172.21.15.2:0/3161001285 shutdown_connections 2024-09-10T08:07:43.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.683+0000 7fb8ba631740 1 --2- 172.21.15.2:0/3161001285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b215510 0x56099b230160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.683+0000 7fb8ba631740 1 --2- 172.21.15.2:0/3161001285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20ac50 0x56099b22fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.683+0000 7fb8ba631740 1 --2- 172.21.15.2:0/3161001285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b20a330 0x56099b20cd90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.683+0000 7fb8ba631740 1 -- 172.21.15.2:0/3161001285 >> 172.21.15.2:0/3161001285 conn(0x56099b2068c0 msgr2=0x56099b210eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:43.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.683+0000 7fb8ba631740 1 -- 172.21.15.2:0/3161001285 shutdown_connections 2024-09-10T08:07:43.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.683+0000 7fb8ba631740 1 -- 172.21.15.2:0/3161001285 wait complete. 2024-09-10T08:07:43.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.683+0000 7fb8ba631740 1 -- start start 2024-09-10T08:07:43.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.684+0000 7fb8ba631740 1 Processor -- start 2024-09-10T08:07:43.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.684+0000 7fb8ba631740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20a330 0x56099b230440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.684+0000 7fb8ba631740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b20ac50 0x56099b20f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.684+0000 7fb8ba631740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b215510 0x56099b20fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.684+0000 7fb8ba631740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56099b2277c0 con 0x56099b20ac50 2024-09-10T08:07:43.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.684+0000 7fb8ba631740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56099b2270f0 con 0x56099b20a330 2024-09-10T08:07:43.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.684+0000 7fb8ba631740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56099b227e90 con 0x56099b215510 2024-09-10T08:07:43.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.684+0000 7fb8ba631740 10 client.0 _ll_register_callbacks cb 0x7fb89c0cc570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:07:43.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.684+0000 7fb8b5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20a330 0x56099b230440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.684+0000 7fb8afe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b20ac50 0x56099b20f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.684+0000 7fb8b5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20a330 0x56099b230440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46622/0 (socket says 172.21.15.2:46622) 2024-09-10T08:07:43.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.684+0000 7fb8b5000640 1 -- 172.21.15.2:0/835194546 learned_addr learned my addr 172.21.15.2:0/835194546 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:43.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.685+0000 7fb8b5a00640 1 --2- 172.21.15.2:0/835194546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b215510 0x56099b20fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.685+0000 7fb8b5000640 1 -- 172.21.15.2:0/835194546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b215510 msgr2=0x56099b20fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:43.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.685+0000 7fb8b5000640 1 --2- 172.21.15.2:0/835194546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b215510 0x56099b20fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.685+0000 7fb8b5000640 1 -- 172.21.15.2:0/835194546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b20ac50 msgr2=0x56099b20f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:43.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.685+0000 7fb8b5000640 1 --2- 172.21.15.2:0/835194546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b20ac50 0x56099b20f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.685+0000 7fb8b5000640 1 -- 172.21.15.2:0/835194546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56099b210430 con 0x56099b20a330 2024-09-10T08:07:43.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.685+0000 7fb8b5000640 1 --2- 172.21.15.2:0/835194546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20a330 0x56099b230440 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb8a402a910 tx=0x7fb8a4031a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c40a2bef5f6f59a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:43.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.685+0000 7fb896a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:07:43.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.685+0000 7fb896a00640 1 -- 172.21.15.2:0/835194546 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a405b070 con 0x56099b20a330 2024-09-10T08:07:43.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.685+0000 7fb8ba631740 1 -- 172.21.15.2:0/835194546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a4004460 con 0x56099b20a330 2024-09-10T08:07:43.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.685+0000 7fb896a00640 1 -- 172.21.15.2:0/835194546 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb8a40051f0 con 0x56099b20a330 2024-09-10T08:07:43.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.686+0000 7fb896a00640 1 -- 172.21.15.2:0/835194546 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8a403f050 con 0x56099b20a330 2024-09-10T08:07:43.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.686+0000 7fb8ba631740 10 client.24986 fetch_fsmap learned FSMap version 25 2024-09-10T08:07:43.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.686+0000 7fb8ba631740 1 -- 172.21.15.2:0/835194546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56099b290d10 con 0x56099b20a330 2024-09-10T08:07:43.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.686+0000 7fb896a00640 1 -- 172.21.15.2:0/835194546 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a4004460 con 0x56099b20a330 2024-09-10T08:07:43.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.686+0000 7fb896a00640 1 -- 172.21.15.2:0/835194546 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb8a405e030 con 0x56099b20a330 2024-09-10T08:07:43.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.686+0000 7fb8ba631740 10 client.24986 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:07:43.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.686+0000 7fb8ba631740 10 client.24986 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:07:43.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.686+0000 7fb8ba631740 20 client.24986 populate_metadata read hostname 'smithi002' 2024-09-10T08:07:43.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.686+0000 7fb8ba631740 1 --2- 172.21.15.2:0/835194546 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56099b295230 0x56099b293f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:43.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.686+0000 7fb8ba631740 4 client.24986 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:07:43.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.686+0000 7fb8ba631740 1 -- 172.21.15.2:0/835194546 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "status"}) -- 0x56099b177b30 con 0x56099b295230 2024-09-10T08:07:43.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.687+0000 7fb8afe00640 1 --2- 172.21.15.2:0/835194546 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56099b295230 0x56099b293f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:43.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.687+0000 7fb8afe00640 1 --2- 172.21.15.2:0/835194546 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56099b295230 0x56099b293f40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f01f0f294dc066f0 server_cookie=71d0276dea5a6cc9 in_seq=0 out_seq=0 2024-09-10T08:07:43.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.687+0000 7fb896a00640 10 client.24986 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:07:43.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.687+0000 7fb896a00640 1 -- 172.21.15.2:0/835194546 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56099b177b30 con 0x56099b295230 2024-09-10T08:07:43.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.687+0000 7fb896a00640 10 client.24986 handle_command_reply: tid=0 2024-09-10T08:07:43.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.687+0000 7fb8ba631740 1 client.24986 shutdown 2024-09-10T08:07:43.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.688+0000 7fb8ba631740 1 -- 172.21.15.2:0/835194546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20a330 msgr2=0x56099b230440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:43.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.688+0000 7fb8ba631740 1 --2- 172.21.15.2:0/835194546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20a330 0x56099b230440 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb8a402a910 tx=0x7fb8a4031a90 comp rx=0 tx=0).stop 2024-09-10T08:07:43.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.688+0000 7fb8ba631740 1 -- 172.21.15.2:0/835194546 shutdown_connections 2024-09-10T08:07:43.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.688+0000 7fb8ba631740 1 --2- 172.21.15.2:0/835194546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56099b215510 0x56099b20fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.688+0000 7fb8ba631740 1 --2- 172.21.15.2:0/835194546 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56099b295230 0x56099b293f40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.688+0000 7fb8ba631740 1 --2- 172.21.15.2:0/835194546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56099b20ac50 0x56099b20f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.688+0000 7fb8ba631740 1 --2- 172.21.15.2:0/835194546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56099b20a330 0x56099b230440 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.688+0000 7fb896a00640 0 client.24986 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:07:43.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.688+0000 7fb8ba631740 1 -- 172.21.15.2:0/835194546 >> 172.21.15.2:0/835194546 conn(0x56099b2068c0 msgr2=0x56099b220430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:43.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.688+0000 7fb8ba631740 1 -- 172.21.15.2:0/835194546 shutdown_connections 2024-09-10T08:07:43.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.688+0000 7fb8ba631740 1 -- 172.21.15.2:0/835194546 wait complete. 2024-09-10T08:07:43.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.688+0000 7fb8ba631740 20 client.24986 trim_cache size 0 max 16384 2024-09-10T08:07:43.698 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:07:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "26ec64b0-6f4a-11ef-bcea-c7b262605968", 2024-09-10T08:07:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2024-09-10T08:07:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24494, 2024-09-10T08:07:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-09-10T08:07:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-09-10T08:07:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-09-10T08:07:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 82.904195630999993, 2024-09-10T08:07:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 25, 2024-09-10T08:07:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 95, 2024-09-10T08:07:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 93, 2024-09-10T08:07:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 163.61631850099999 2024-09-10T08:07:43.699 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:07:43.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.689+0000 7fb8b7a00640 1 -- 172.21.15.2:0/1388959694 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb8980a2c10 msgr2=0x7fb8980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:43.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.689+0000 7fb8b7a00640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb8980a2c10 0x7fb8980a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8005fd0 tx=0x7fb8a8009500 comp rx=0 tx=0).stop 2024-09-10T08:07:43.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.689+0000 7fb8b7a00640 1 -- 172.21.15.2:0/1388959694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8b0102790 msgr2=0x7fb8b019ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:43.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.689+0000 7fb8b7a00640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8b0102790 0x7fb8b019ac00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00d770 tx=0x7fb89c00dc30 comp rx=0 tx=0).stop 2024-09-10T08:07:43.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.689+0000 7fb8b7a00640 1 -- 172.21.15.2:0/1388959694 shutdown_connections 2024-09-10T08:07:43.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.689+0000 7fb8b7a00640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb8980a2c10 0x7fb8980a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.689+0000 7fb8b7a00640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8b0106f10 0x7fb8b019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.689+0000 7fb8b7a00640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8b0102790 0x7fb8b019ac00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.689+0000 7fb8b7a00640 1 --2- 172.21.15.2:0/1388959694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b0101de0 0x7fb8b019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:43.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.689+0000 7fb8b7a00640 1 -- 172.21.15.2:0/1388959694 >> 172.21.15.2:0/1388959694 conn(0x7fb8b00fd8c0 msgr2=0x7fb8b00febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:43.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.689+0000 7fb8b7a00640 1 -- 172.21.15.2:0/1388959694 shutdown_connections 2024-09-10T08:07:43.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:43.689+0000 7fb8b7a00640 1 -- 172.21.15.2:0/1388959694 wait complete. 2024-09-10T08:07:44.453 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T08:07:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:45.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:44 smithi002 ceph-mon[30563]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:45.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:44 smithi155 ceph-mon[35405]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:45.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:44 smithi107 ceph-mon[35048]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:45.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.391+0000 7f0199400640 1 -- 172.21.15.2:0/1278242614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0194109120 msgr2=0x7f0194110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:45.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.391+0000 7f0199400640 1 --2- 172.21.15.2:0/1278242614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0194109120 0x7f0194110ca0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f018c009900 tx=0x7f018c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.391+0000 7f0199400640 1 -- 172.21.15.2:0/1278242614 shutdown_connections 2024-09-10T08:07:45.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.391+0000 7f0199400640 1 --2- 172.21.15.2:0/1278242614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0194109120 0x7f0194110ca0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.391+0000 7f0199400640 1 --2- 172.21.15.2:0/1278242614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01941049a0 0x7f01941089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.391+0000 7f0199400640 1 --2- 172.21.15.2:0/1278242614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0194103ff0 0x7f01941043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.391+0000 7f0199400640 1 -- 172.21.15.2:0/1278242614 >> 172.21.15.2:0/1278242614 conn(0x7f01940ffa90 msgr2=0x7f0194101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.391+0000 7f0199400640 1 -- 172.21.15.2:0/1278242614 shutdown_connections 2024-09-10T08:07:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.391+0000 7f0199400640 1 -- 172.21.15.2:0/1278242614 wait complete. 2024-09-10T08:07:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.392+0000 7f0199400640 1 Processor -- start 2024-09-10T08:07:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.392+0000 7f0199400640 1 -- start start 2024-09-10T08:07:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.392+0000 7f0199400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0194103ff0 0x7f019419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.392+0000 7f0199400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01941049a0 0x7f019419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.392+0000 7f0199400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0194109120 0x7f019419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.392+0000 7f0199400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0194114180 con 0x7f01941049a0 2024-09-10T08:07:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.392+0000 7f0199400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0194114000 con 0x7f0194109120 2024-09-10T08:07:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.392+0000 7f0199400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0194114300 con 0x7f0194103ff0 2024-09-10T08:07:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.392+0000 7f0193200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0194109120 0x7f019419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.393+0000 7f0192800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0194103ff0 0x7f019419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.393+0000 7f0189e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01941049a0 0x7f019419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.393+0000 7f0192800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0194103ff0 0x7f019419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:33046/0 (socket says 172.21.15.2:33046) 2024-09-10T08:07:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.393+0000 7f0193200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0194109120 0x7f019419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46636/0 (socket says 172.21.15.2:46636) 2024-09-10T08:07:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.393+0000 7f0192800640 1 -- 172.21.15.2:0/1371894745 learned_addr learned my addr 172.21.15.2:0/1371894745 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.393+0000 7f0192800640 1 -- 172.21.15.2:0/1371894745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0194109120 msgr2=0x7f019419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.393+0000 7f0192800640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0194109120 0x7f019419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.393+0000 7f0192800640 1 -- 172.21.15.2:0/1371894745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01941049a0 msgr2=0x7f019419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.393+0000 7f0192800640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01941049a0 0x7f019419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.393+0000 7f0192800640 1 -- 172.21.15.2:0/1371894745 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019419f590 con 0x7f0194103ff0 2024-09-10T08:07:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.393+0000 7f0189e00640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01941049a0 0x7f019419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.393+0000 7f0193200640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0194109120 0x7f019419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.393+0000 7f0192800640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0194103ff0 0x7f019419a5e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f018400eab0 tx=0x7f018400ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.394+0000 7f018be00640 1 -- 172.21.15.2:0/1371894745 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018400ce40 con 0x7f0194103ff0 2024-09-10T08:07:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.394+0000 7f0199400640 1 -- 172.21.15.2:0/1371894745 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019419f880 con 0x7f0194103ff0 2024-09-10T08:07:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.394+0000 7f018be00640 1 -- 172.21.15.2:0/1371894745 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0184011950 con 0x7f0194103ff0 2024-09-10T08:07:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.394+0000 7f018be00640 1 -- 172.21.15.2:0/1371894745 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0184010ae0 con 0x7f0194103ff0 2024-09-10T08:07:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.394+0000 7f0199400640 1 -- 172.21.15.2:0/1371894745 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01941a7160 con 0x7f0194103ff0 2024-09-10T08:07:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.397+0000 7f019be8d740 1 Processor -- start 2024-09-10T08:07:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.397+0000 7f019be8d740 1 -- start start 2024-09-10T08:07:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.397+0000 7f018be00640 1 -- 172.21.15.2:0/1371894745 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0184011b00 con 0x7f0194103ff0 2024-09-10T08:07:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.397+0000 7f019be8d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 0x55e8fcb5f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.397+0000 7f019be8d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb69fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.397+0000 7f019be8d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb6c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.397+0000 7f019be8d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e8fcac50b0 con 0x55e8fcb6a510 2024-09-10T08:07:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.397+0000 7f019be8d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e8fca6cab0 con 0x55e8fcb5fc50 2024-09-10T08:07:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.397+0000 7f019be8d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e8fcac1070 con 0x55e8fcb5f330 2024-09-10T08:07:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.398+0000 7f0192800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb69fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.398+0000 7f0189e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb6c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.398+0000 7f0193200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 0x55e8fcb5f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.398+0000 7f0192800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb69fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46646/0 (socket says 172.21.15.2:46646) 2024-09-10T08:07:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.398+0000 7f0192800640 1 -- 172.21.15.2:0/3552886994 learned_addr learned my addr 172.21.15.2:0/3552886994 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.398+0000 7f0193200640 1 -- 172.21.15.2:0/3552886994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 msgr2=0x55e8fcb69fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.398+0000 7f0193200640 1 --2- 172.21.15.2:0/3552886994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb69fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.398+0000 7f0193200640 1 -- 172.21.15.2:0/3552886994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 msgr2=0x55e8fcb6c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.398+0000 7f0193200640 1 --2- 172.21.15.2:0/3552886994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb6c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.398+0000 7f0193200640 1 -- 172.21.15.2:0/3552886994 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8fca794a0 con 0x55e8fcb5f330 2024-09-10T08:07:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.398+0000 7f0192800640 1 --2- 172.21.15.2:0/3552886994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb69fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.398+0000 7f0189e00640 1 --2- 172.21.15.2:0/3552886994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb6c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.399+0000 7f0193200640 1 --2- 172.21.15.2:0/3552886994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 0x55e8fcb5f710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f018c02a910 tx=0x7f018c005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d78235026cc707c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.399+0000 7f0183400640 1 -- 172.21.15.2:0/3552886994 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c040070 con 0x55e8fcb5f330 2024-09-10T08:07:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.399+0000 7f0183400640 1 -- 172.21.15.2:0/3552886994 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f018c003cd0 con 0x55e8fcb5f330 2024-09-10T08:07:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.399+0000 7f0183400640 1 -- 172.21.15.2:0/3552886994 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c035040 con 0x55e8fcb5f330 2024-09-10T08:07:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.399+0000 7f018be00640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f01780a2c90 0x7f01780a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.399+0000 7f018be00640 1 -- 172.21.15.2:0/1371894745 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f01840cfbe0 con 0x7f0194103ff0 2024-09-10T08:07:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.399+0000 7f0192800640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f01780a2c90 0x7f01780a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.400+0000 7f019be8d740 1 -- 172.21.15.2:0/3552886994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 msgr2=0x55e8fcb5f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.400+0000 7f019be8d740 1 --2- 172.21.15.2:0/3552886994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 0x55e8fcb5f710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f018c02a910 tx=0x7f018c005e00 comp rx=0 tx=0).stop 2024-09-10T08:07:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.400+0000 7f019be8d740 1 -- 172.21.15.2:0/3552886994 shutdown_connections 2024-09-10T08:07:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.400+0000 7f019be8d740 1 --2- 172.21.15.2:0/3552886994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb6c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.400+0000 7f019be8d740 1 --2- 172.21.15.2:0/3552886994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb69fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.400+0000 7f019be8d740 1 --2- 172.21.15.2:0/3552886994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 0x55e8fcb5f710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.400+0000 7f019be8d740 1 -- 172.21.15.2:0/3552886994 >> 172.21.15.2:0/3552886994 conn(0x55e8fcb5b8c0 msgr2=0x55e8fcb5bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.400+0000 7f019be8d740 1 -- 172.21.15.2:0/3552886994 shutdown_connections 2024-09-10T08:07:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.400+0000 7f019be8d740 1 -- 172.21.15.2:0/3552886994 wait complete. 2024-09-10T08:07:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.400+0000 7f0192800640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f01780a2c90 0x7f01780a5150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f01840982f0 tx=0x7f01840d3690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.400+0000 7f019be8d740 1 -- start start 2024-09-10T08:07:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.401+0000 7f019be8d740 1 Processor -- start 2024-09-10T08:07:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.401+0000 7f019be8d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 0x55e8fcb67560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.401+0000 7f019be8d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb67aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.401+0000 7f019be8d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb67fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.401+0000 7f019be8d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e8fcb6d060 con 0x55e8fcb6a510 2024-09-10T08:07:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.401+0000 7f019be8d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e8fca4d920 con 0x55e8fcb5fc50 2024-09-10T08:07:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.401+0000 7f019be8d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e8fcb6d1e0 con 0x55e8fcb5f330 2024-09-10T08:07:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.401+0000 7f019be8d740 10 client.0 _ll_register_callbacks cb 0x55e8fca718a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:07:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.401+0000 7f0193200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb67fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.401+0000 7f0193200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb67fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51810/0 (socket says 172.21.15.2:51810) 2024-09-10T08:07:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.401+0000 7f0193200640 1 -- 172.21.15.2:0/3772375479 learned_addr learned my addr 172.21.15.2:0/3772375479 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.401+0000 7f0193200640 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 0x55e8fcb67560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.402+0000 7f0189e00640 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb67aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.402+0000 7f0193200640 1 -- 172.21.15.2:0/3772375479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 msgr2=0x55e8fcb67560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.402+0000 7f0193200640 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 0x55e8fcb67560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.402+0000 7f0193200640 1 -- 172.21.15.2:0/3772375479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 msgr2=0x55e8fcb67aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.402+0000 7f0193200640 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb67aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.402+0000 7f0193200640 1 -- 172.21.15.2:0/3772375479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8fc8362b0 con 0x55e8fcb6a510 2024-09-10T08:07:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.402+0000 7f0189e00640 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb67aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:07:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.402+0000 7f0193200640 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb67fe0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f018c031fd0 tx=0x7f018c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc53add44227d1e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.402+0000 7f0177e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:07:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.402+0000 7f0177e00640 1 -- 172.21.15.2:0/3772375479 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c035030 con 0x55e8fcb6a510 2024-09-10T08:07:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.402+0000 7f019be8d740 1 -- 172.21.15.2:0/3772375479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f018c003cd0 con 0x55e8fcb6a510 2024-09-10T08:07:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.402+0000 7f0177e00640 1 -- 172.21.15.2:0/3772375479 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f018c0317c0 con 0x55e8fcb6a510 2024-09-10T08:07:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.403+0000 7f0177e00640 1 -- 172.21.15.2:0/3772375479 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f018c05b050 con 0x55e8fcb6a510 2024-09-10T08:07:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.403+0000 7f019be8d740 10 client.14979 fetch_fsmap learned FSMap version 25 2024-09-10T08:07:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.403+0000 7f019be8d740 1 -- 172.21.15.2:0/3772375479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8fcb78df0 con 0x55e8fcb6a510 2024-09-10T08:07:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.403+0000 7f0177e00640 1 -- 172.21.15.2:0/3772375479 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f018c003cd0 con 0x55e8fcb6a510 2024-09-10T08:07:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.403+0000 7f0177e00640 1 -- 172.21.15.2:0/3772375479 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f018c0458a0 con 0x55e8fcb6a510 2024-09-10T08:07:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.403+0000 7f019be8d740 10 client.14979 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:07:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.403+0000 7f019be8d740 10 client.14979 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:07:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.403+0000 7f019be8d740 20 client.14979 populate_metadata read hostname 'smithi002' 2024-09-10T08:07:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.403+0000 7f019be8d740 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55e8fcb7af00 0x55e8fcb7c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.403+0000 7f019be8d740 4 client.14979 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:07:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.403+0000 7f019be8d740 1 -- 172.21.15.2:0/3772375479 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8fcad2620 con 0x55e8fcb7af00 2024-09-10T08:07:45.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.404+0000 7f0189e00640 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55e8fcb7af00 0x55e8fcb7c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.404+0000 7f0189e00640 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55e8fcb7af00 0x55e8fcb7c370 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=780d62d80d74ddd3 server_cookie=a58ef0ba8083bc6c in_seq=0 out_seq=0 2024-09-10T08:07:45.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.404+0000 7f0177e00640 10 client.14979 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:07:45.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.405+0000 7f0177e00640 1 -- 172.21.15.2:0/3772375479 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8fcad2620 con 0x55e8fcb7af00 2024-09-10T08:07:45.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.405+0000 7f0177e00640 10 client.14979 handle_command_reply: tid=0 2024-09-10T08:07:45.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.405+0000 7f019be8d740 1 client.14979 shutdown 2024-09-10T08:07:45.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.406+0000 7f019be8d740 1 -- 172.21.15.2:0/3772375479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 msgr2=0x55e8fcb67fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:45.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.406+0000 7f019be8d740 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb67fe0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f018c031fd0 tx=0x7f018c005a20 comp rx=0 tx=0).stop 2024-09-10T08:07:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.406+0000 7f019be8d740 1 -- 172.21.15.2:0/3772375479 shutdown_connections 2024-09-10T08:07:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.406+0000 7f019be8d740 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55e8fcb7af00 0x55e8fcb7c370 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.406+0000 7f019be8d740 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb67fe0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.406+0000 7f019be8d740 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb67aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.406+0000 7f019be8d740 1 --2- 172.21.15.2:0/3772375479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 0x55e8fcb67560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.406+0000 7f019be8d740 1 -- 172.21.15.2:0/3772375479 >> 172.21.15.2:0/3772375479 conn(0x55e8fcb5b8c0 msgr2=0x55e8fcb615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.406+0000 7f0177e00640 0 client.14979 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:07:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.406+0000 7f019be8d740 1 -- 172.21.15.2:0/3772375479 shutdown_connections 2024-09-10T08:07:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.406+0000 7f019be8d740 1 -- 172.21.15.2:0/3772375479 wait complete. 2024-09-10T08:07:45.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.406+0000 7f019be8d740 20 client.14979 trim_cache size 0 max 16384 2024-09-10T08:07:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.424+0000 7f019be8d740 1 Processor -- start 2024-09-10T08:07:45.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.425+0000 7f019be8d740 1 -- start start 2024-09-10T08:07:45.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.425+0000 7f019be8d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb5f330 0x55e8fcb7bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.425+0000 7f019be8d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb66e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.425+0000 7f019be8d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb6a510 0x55e8fcb61d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.425+0000 7f019be8d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e8fcb7a6b0 con 0x55e8fcb5f330 2024-09-10T08:07:45.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.425+0000 7f019be8d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e8fcb7ad80 con 0x55e8fcb5fc50 2024-09-10T08:07:45.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.425+0000 7f019be8d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e8fcb7b450 con 0x55e8fcb6a510 2024-09-10T08:07:45.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.425+0000 7f0193200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb5f330 0x55e8fcb7bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.425+0000 7f0193200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb5f330 0x55e8fcb7bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51812/0 (socket says 172.21.15.2:51812) 2024-09-10T08:07:45.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.425+0000 7f0193200640 1 -- 172.21.15.2:0/335108618 learned_addr learned my addr 172.21.15.2:0/335108618 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:45.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.425+0000 7f0189e00640 1 --2- 172.21.15.2:0/335108618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb66e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.425+0000 7f0193200640 1 --2- 172.21.15.2:0/335108618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb6a510 0x55e8fcb61d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.426+0000 7f0189e00640 1 -- 172.21.15.2:0/335108618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb6a510 msgr2=0x55e8fcb61d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:45.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.426+0000 7f0189e00640 1 --2- 172.21.15.2:0/335108618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb6a510 0x55e8fcb61d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.426+0000 7f0189e00640 1 -- 172.21.15.2:0/335108618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb5f330 msgr2=0x55e8fcb7bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:45.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.426+0000 7f0189e00640 1 --2- 172.21.15.2:0/335108618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb5f330 0x55e8fcb7bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.426+0000 7f0189e00640 1 -- 172.21.15.2:0/335108618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8fcb5c550 con 0x55e8fcb5fc50 2024-09-10T08:07:45.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.426+0000 7f0193200640 1 --2- 172.21.15.2:0/335108618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb5f330 0x55e8fcb7bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:45.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.426+0000 7f0189e00640 1 --2- 172.21.15.2:0/335108618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb66e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55e8fcb7cb40 tx=0x7f017c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bb61306ee5ed592 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:45.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.426+0000 7f0182a00640 1 -- 172.21.15.2:0/335108618 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c012310 con 0x55e8fcb5fc50 2024-09-10T08:07:45.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.426+0000 7f0182a00640 1 -- 172.21.15.2:0/335108618 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f017c0124b0 con 0x55e8fcb5fc50 2024-09-10T08:07:45.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.426+0000 7f0182a00640 1 -- 172.21.15.2:0/335108618 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c0106b0 con 0x55e8fcb5fc50 2024-09-10T08:07:45.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.427+0000 7f019be8d740 1 -- 172.21.15.2:0/335108618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 msgr2=0x55e8fcb66e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:45.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.427+0000 7f019be8d740 1 --2- 172.21.15.2:0/335108618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb66e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55e8fcb7cb40 tx=0x7f017c0046c0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.427+0000 7f019be8d740 1 -- 172.21.15.2:0/335108618 shutdown_connections 2024-09-10T08:07:45.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.427+0000 7f019be8d740 1 --2- 172.21.15.2:0/335108618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb6a510 0x55e8fcb61d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.427+0000 7f019be8d740 1 --2- 172.21.15.2:0/335108618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb66e20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.427+0000 7f019be8d740 1 --2- 172.21.15.2:0/335108618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb5f330 0x55e8fcb7bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.427+0000 7f019be8d740 1 -- 172.21.15.2:0/335108618 >> 172.21.15.2:0/335108618 conn(0x55e8fcb5b8c0 msgr2=0x55e8fcb65e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:45.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.427+0000 7f019be8d740 1 -- 172.21.15.2:0/335108618 shutdown_connections 2024-09-10T08:07:45.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.427+0000 7f019be8d740 1 -- 172.21.15.2:0/335108618 wait complete. 2024-09-10T08:07:45.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.427+0000 7f019be8d740 1 -- start start 2024-09-10T08:07:45.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.428+0000 7f019be8d740 1 Processor -- start 2024-09-10T08:07:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.428+0000 7f019be8d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 0x55e8fcb64760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.428+0000 7f019be8d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb64ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.428+0000 7f019be8d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb651e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.428+0000 7f019be8d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e8fcb7c1f0 con 0x55e8fcb6a510 2024-09-10T08:07:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.428+0000 7f019be8d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e8fcb7bb20 con 0x55e8fcb5fc50 2024-09-10T08:07:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.428+0000 7f019be8d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e8fcb7c8c0 con 0x55e8fcb5f330 2024-09-10T08:07:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.428+0000 7f019be8d740 10 client.0 _ll_register_callbacks cb 0x7f018c031b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:07:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.428+0000 7f0193200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.429+0000 7f0193200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb651e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51818/0 (socket says 172.21.15.2:51818) 2024-09-10T08:07:45.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.429+0000 7f0189e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb64ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.429+0000 7f0193200640 1 -- 172.21.15.2:0/3641590675 learned_addr learned my addr 172.21.15.2:0/3641590675 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:45.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.429+0000 7f0193200640 1 --2- 172.21.15.2:0/3641590675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 0x55e8fcb64760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.429+0000 7f0193200640 1 -- 172.21.15.2:0/3641590675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 msgr2=0x55e8fcb64760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:45.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.429+0000 7f0193200640 1 --2- 172.21.15.2:0/3641590675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 0x55e8fcb64760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.429+0000 7f0193200640 1 -- 172.21.15.2:0/3641590675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 msgr2=0x55e8fcb64ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:45.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.429+0000 7f0193200640 1 --2- 172.21.15.2:0/3641590675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb64ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.429+0000 7f0193200640 1 -- 172.21.15.2:0/3641590675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8fcb82be0 con 0x55e8fcb6a510 2024-09-10T08:07:45.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.429+0000 7f0189e00640 1 --2- 172.21.15.2:0/3641590675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb64ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:45.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.429+0000 7f0193200640 1 --2- 172.21.15.2:0/3641590675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb651e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f018c031520 tx=0x7f018c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04a2ded4af43f1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:45.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.429+0000 7f0177e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:07:45.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.429+0000 7f0177e00640 1 -- 172.21.15.2:0/3641590675 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c035040 con 0x55e8fcb6a510 2024-09-10T08:07:45.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.430+0000 7f019be8d740 1 -- 172.21.15.2:0/3641590675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f017c0124b0 con 0x55e8fcb6a510 2024-09-10T08:07:45.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.430+0000 7f0177e00640 1 -- 172.21.15.2:0/3641590675 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f018c045670 con 0x55e8fcb6a510 2024-09-10T08:07:45.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.430+0000 7f0177e00640 1 -- 172.21.15.2:0/3641590675 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f018c06b050 con 0x55e8fcb6a510 2024-09-10T08:07:45.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.430+0000 7f019be8d740 10 client.14985 fetch_fsmap learned FSMap version 25 2024-09-10T08:07:45.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.430+0000 7f019be8d740 1 -- 172.21.15.2:0/3641590675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8fcbe5940 con 0x55e8fcb6a510 2024-09-10T08:07:45.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.430+0000 7f0177e00640 1 -- 172.21.15.2:0/3641590675 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f017c0124b0 con 0x55e8fcb6a510 2024-09-10T08:07:45.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.430+0000 7f0177e00640 1 -- 172.21.15.2:0/3641590675 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f018c005030 con 0x55e8fcb6a510 2024-09-10T08:07:45.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.430+0000 7f019be8d740 10 client.14985 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:07:45.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.430+0000 7f019be8d740 10 client.14985 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:07:45.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.430+0000 7f019be8d740 20 client.14985 populate_metadata read hostname 'smithi002' 2024-09-10T08:07:45.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.430+0000 7f019be8d740 1 --2- 172.21.15.2:0/3641590675 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55e8fcbea400 0x55e8fcbe9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:45.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.430+0000 7f019be8d740 4 client.14985 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:07:45.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.430+0000 7f019be8d740 1 -- 172.21.15.2:0/3641590675 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "status"}) -- 0x55e8fcaccb30 con 0x55e8fcbea400 2024-09-10T08:07:45.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.431+0000 7f0189e00640 1 --2- 172.21.15.2:0/3641590675 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55e8fcbea400 0x55e8fcbe9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:45.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.431+0000 7f0189e00640 1 --2- 172.21.15.2:0/3641590675 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55e8fcbea400 0x55e8fcbe9110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91b6b4fd9d706ac9 server_cookie=7592c3541662f82 in_seq=0 out_seq=0 2024-09-10T08:07:45.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.431+0000 7f0177e00640 10 client.14985 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:07:45.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f0177e00640 1 -- 172.21.15.2:0/3641590675 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e8fcaccb30 con 0x55e8fcbea400 2024-09-10T08:07:45.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f0177e00640 10 client.14985 handle_command_reply: tid=0 2024-09-10T08:07:45.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f019be8d740 1 client.14985 shutdown 2024-09-10T08:07:45.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f019be8d740 1 -- 172.21.15.2:0/3641590675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 msgr2=0x55e8fcb651e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:45.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f019be8d740 1 --2- 172.21.15.2:0/3641590675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb651e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f018c031520 tx=0x7f018c005fd0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f019be8d740 1 -- 172.21.15.2:0/3641590675 shutdown_connections 2024-09-10T08:07:45.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f019be8d740 1 --2- 172.21.15.2:0/3641590675 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55e8fcbea400 0x55e8fcbe9110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f019be8d740 1 --2- 172.21.15.2:0/3641590675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e8fcb6a510 0x55e8fcb651e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f019be8d740 1 --2- 172.21.15.2:0/3641590675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e8fcb5fc50 0x55e8fcb64ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f019be8d740 1 --2- 172.21.15.2:0/3641590675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8fcb5f330 0x55e8fcb64760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f019be8d740 1 -- 172.21.15.2:0/3641590675 >> 172.21.15.2:0/3641590675 conn(0x55e8fcb5b8c0 msgr2=0x55e8fcb65d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:45.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f0177e00640 0 client.14985 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:07:45.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f019be8d740 1 -- 172.21.15.2:0/3641590675 shutdown_connections 2024-09-10T08:07:45.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f019be8d740 1 -- 172.21.15.2:0/3641590675 wait complete. 2024-09-10T08:07:45.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.432+0000 7f019be8d740 20 client.14985 trim_cache size 0 max 16384 2024-09-10T08:07:45.438 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:07:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "26ec64b0-6f4a-11ef-bcea-c7b262605968", 2024-09-10T08:07:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2024-09-10T08:07:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24499, 2024-09-10T08:07:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-09-10T08:07:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-09-10T08:07:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-09-10T08:07:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 85.651677363000005, 2024-09-10T08:07:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 25, 2024-09-10T08:07:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 95, 2024-09-10T08:07:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 93, 2024-09-10T08:07:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 162.72823980800001 2024-09-10T08:07:45.439 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:07:45.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.433+0000 7f0199400640 1 -- 172.21.15.2:0/1371894745 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f01780a2c90 msgr2=0x7f01780a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:45.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.433+0000 7f0199400640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f01780a2c90 0x7f01780a5150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f01840982f0 tx=0x7f01840d3690 comp rx=0 tx=0).stop 2024-09-10T08:07:45.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.433+0000 7f0199400640 1 -- 172.21.15.2:0/1371894745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0194103ff0 msgr2=0x7f019419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:45.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.433+0000 7f0199400640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0194103ff0 0x7f019419a5e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f018400eab0 tx=0x7f018400ef70 comp rx=0 tx=0).stop 2024-09-10T08:07:45.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.433+0000 7f0199400640 1 -- 172.21.15.2:0/1371894745 shutdown_connections 2024-09-10T08:07:45.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.433+0000 7f0199400640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0194109120 0x7f019419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.433+0000 7f0199400640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01941049a0 0x7f019419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.433+0000 7f0199400640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f01780a2c90 0x7f01780a5150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.433+0000 7f0199400640 1 --2- 172.21.15.2:0/1371894745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0194103ff0 0x7f019419a5e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:45.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.433+0000 7f0199400640 1 -- 172.21.15.2:0/1371894745 >> 172.21.15.2:0/1371894745 conn(0x7f01940ffa90 msgr2=0x7f0194101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:45.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.433+0000 7f0199400640 1 -- 172.21.15.2:0/1371894745 shutdown_connections 2024-09-10T08:07:45.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:45.433+0000 7f0199400640 1 -- 172.21.15.2:0/1371894745 wait complete. 2024-09-10T08:07:46.188 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T08:07:46.567 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:47.002 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:46 smithi002 ceph-mon[30563]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:47.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.114+0000 7f62f7800640 1 -- 172.21.15.2:0/417378457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62f0102790 msgr2=0x7f62f01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:47.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.114+0000 7f62f7800640 1 --2- 172.21.15.2:0/417378457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62f0102790 0x7f62f01067e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009900 tx=0x7f62dc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.114+0000 7f62f7800640 1 -- 172.21.15.2:0/417378457 shutdown_connections 2024-09-10T08:07:47.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.114+0000 7f62f7800640 1 --2- 172.21.15.2:0/417378457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62f0106f10 0x7f62f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.114+0000 7f62f7800640 1 --2- 172.21.15.2:0/417378457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62f0102790 0x7f62f01067e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.114+0000 7f62f7800640 1 --2- 172.21.15.2:0/417378457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0101de0 0x7f62f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.114+0000 7f62f7800640 1 -- 172.21.15.2:0/417378457 >> 172.21.15.2:0/417378457 conn(0x7f62f00fd8c0 msgr2=0x7f62f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:47.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.114+0000 7f62f7800640 1 -- 172.21.15.2:0/417378457 shutdown_connections 2024-09-10T08:07:47.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.114+0000 7f62f7800640 1 -- 172.21.15.2:0/417378457 wait complete. 2024-09-10T08:07:47.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.115+0000 7f62f7800640 1 Processor -- start 2024-09-10T08:07:47.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.115+0000 7f62f7800640 1 -- start start 2024-09-10T08:07:47.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.116+0000 7f62f7800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0101de0 0x7f62f019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.116+0000 7f62f7800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62f0102790 0x7f62f019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.116+0000 7f62f7800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62f0106f10 0x7f62f019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.116+0000 7f62f7800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0111fc0 con 0x7f62f0102790 2024-09-10T08:07:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.116+0000 7f62f7800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0111e40 con 0x7f62f0101de0 2024-09-10T08:07:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.116+0000 7f62f7800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0112140 con 0x7f62f0106f10 2024-09-10T08:07:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.116+0000 7f62efe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62f0102790 0x7f62f019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.116+0000 7f62efe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62f0102790 0x7f62f019abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34506/0 (socket says 172.21.15.2:34506) 2024-09-10T08:07:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.116+0000 7f62efe00640 1 -- 172.21.15.2:0/1981478017 learned_addr learned my addr 172.21.15.2:0/1981478017 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.116+0000 7f62f4e00640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0101de0 0x7f62f019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.116+0000 7f62f5800640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62f0106f10 0x7f62f019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.117+0000 7f62efe00640 1 -- 172.21.15.2:0/1981478017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62f0106f10 msgr2=0x7f62f019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.117+0000 7f62efe00640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62f0106f10 0x7f62f019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.117+0000 7f62efe00640 1 -- 172.21.15.2:0/1981478017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0101de0 msgr2=0x7f62f019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:47.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.117+0000 7f62efe00640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0101de0 0x7f62f019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.117+0000 7f62efe00640 1 -- 172.21.15.2:0/1981478017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f019f640 con 0x7f62f0102790 2024-09-10T08:07:47.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.117+0000 7f62f4e00640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0101de0 0x7f62f019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:47.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.117+0000 7f62efe00640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62f0102790 0x7f62f019abd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f62dc004770 tx=0x7f62dc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:47.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.117+0000 7f62ed600640 1 -- 172.21.15.2:0/1981478017 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc03f070 con 0x7f62f0102790 2024-09-10T08:07:47.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.117+0000 7f62ed600640 1 -- 172.21.15.2:0/1981478017 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62dc031ca0 con 0x7f62f0102790 2024-09-10T08:07:47.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.117+0000 7f62ed600640 1 -- 172.21.15.2:0/1981478017 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc035030 con 0x7f62f0102790 2024-09-10T08:07:47.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.117+0000 7f62f7800640 1 -- 172.21.15.2:0/1981478017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f019f8d0 con 0x7f62f0102790 2024-09-10T08:07:47.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.117+0000 7f62f7800640 1 -- 172.21.15.2:0/1981478017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f01a7210 con 0x7f62f0102790 2024-09-10T08:07:47.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.120+0000 7f62fa45e740 1 Processor -- start 2024-09-10T08:07:47.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.121+0000 7f62ed600640 1 -- 172.21.15.2:0/1981478017 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f62dc042070 con 0x7f62f0102790 2024-09-10T08:07:47.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.121+0000 7f62fa45e740 1 -- start start 2024-09-10T08:07:47.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.121+0000 7f62fa45e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33250 0x55eef3c33630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.121+0000 7f62fa45e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.121+0000 7f62fa45e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 0x55eef3c49110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.121+0000 7f62fa45e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55eef3b990b0 con 0x55eef3c33250 2024-09-10T08:07:47.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.121+0000 7f62fa45e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eef3b40ab0 con 0x55eef3c46d20 2024-09-10T08:07:47.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.121+0000 7f62fa45e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eef3b95070 con 0x55eef3c33b70 2024-09-10T08:07:47.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.121+0000 7f62f5800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33250 0x55eef3c33630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.121+0000 7f62f4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.121+0000 7f62f4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54786/0 (socket says 172.21.15.2:54786) 2024-09-10T08:07:47.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.121+0000 7f62f4e00640 1 -- 172.21.15.2:0/605212906 learned_addr learned my addr 172.21.15.2:0/605212906 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:47.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.121+0000 7f62efe00640 1 --2- 172.21.15.2:0/605212906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 0x55eef3c49110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.122+0000 7f62f4e00640 1 -- 172.21.15.2:0/605212906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 msgr2=0x55eef3c49110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.122+0000 7f62f4e00640 1 --2- 172.21.15.2:0/605212906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 0x55eef3c49110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.122+0000 7f62f4e00640 1 -- 172.21.15.2:0/605212906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33250 msgr2=0x55eef3c33630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.122+0000 7f62f4e00640 1 --2- 172.21.15.2:0/605212906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33250 0x55eef3c33630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.122+0000 7f62f4e00640 1 -- 172.21.15.2:0/605212906 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef3b4d4a0 con 0x55eef3c33b70 2024-09-10T08:07:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.122+0000 7f62f4e00640 1 --2- 172.21.15.2:0/605212906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c467e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f62e40115f0 tx=0x7f62e4011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98710f77eb9b4899 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.122+0000 7f62ed600640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62d80a2c90 0x7f62d80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.122+0000 7f62ed600640 1 -- 172.21.15.2:0/1981478017 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f62dc0ef7f0 con 0x7f62f0102790 2024-09-10T08:07:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.122+0000 7f62e2000640 1 -- 172.21.15.2:0/605212906 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e401e070 con 0x55eef3c33b70 2024-09-10T08:07:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.123+0000 7f62e2000640 1 -- 172.21.15.2:0/605212906 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e400fb20 con 0x55eef3c33b70 2024-09-10T08:07:47.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.123+0000 7f62e2000640 1 -- 172.21.15.2:0/605212906 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4019c40 con 0x55eef3c33b70 2024-09-10T08:07:47.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.123+0000 7f62f5800640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62d80a2c90 0x7f62d80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.123+0000 7f62fa45e740 1 -- 172.21.15.2:0/605212906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 msgr2=0x55eef3c467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:47.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.123+0000 7f62fa45e740 1 --2- 172.21.15.2:0/605212906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c467e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f62e40115f0 tx=0x7f62e4011ab0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.123+0000 7f62f5800640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62d80a2c90 0x7f62d80a5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005fd0 tx=0x7f62e8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:47.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.123+0000 7f62fa45e740 1 -- 172.21.15.2:0/605212906 shutdown_connections 2024-09-10T08:07:47.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.123+0000 7f62fa45e740 1 --2- 172.21.15.2:0/605212906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 0x55eef3c49110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.123+0000 7f62fa45e740 1 --2- 172.21.15.2:0/605212906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c467e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.123+0000 7f62fa45e740 1 --2- 172.21.15.2:0/605212906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33250 0x55eef3c33630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.123+0000 7f62fa45e740 1 -- 172.21.15.2:0/605212906 >> 172.21.15.2:0/605212906 conn(0x55eef3c2f8c0 msgr2=0x55eef3c2fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:47.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.124+0000 7f62fa45e740 1 -- 172.21.15.2:0/605212906 shutdown_connections 2024-09-10T08:07:47.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.124+0000 7f62fa45e740 1 -- 172.21.15.2:0/605212906 wait complete. 2024-09-10T08:07:47.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.124+0000 7f62fa45e740 1 -- start start 2024-09-10T08:07:47.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.124+0000 7f62fa45e740 1 Processor -- start 2024-09-10T08:07:47.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62fa45e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c33250 0x55eef3c43cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62fa45e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c44210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62fa45e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c46d20 0x55eef3c44750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62fa45e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55eef3c49930 con 0x55eef3c46d20 2024-09-10T08:07:47.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62fa45e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eef3b21920 con 0x55eef3c33250 2024-09-10T08:07:47.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62fa45e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eef3c49ab0 con 0x55eef3c33b70 2024-09-10T08:07:47.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62fa45e740 10 client.0 _ll_register_callbacks cb 0x55eef3b458a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:07:47.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62f5800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c46d20 0x55eef3c44750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62efe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c44210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62efe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c44210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54794/0 (socket says 172.21.15.2:54794) 2024-09-10T08:07:47.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62efe00640 1 -- 172.21.15.2:0/646586770 learned_addr learned my addr 172.21.15.2:0/646586770 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:47.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62f5800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c46d20 0x55eef3c44750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34518/0 (socket says 172.21.15.2:34518) 2024-09-10T08:07:47.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62f4e00640 1 --2- 172.21.15.2:0/646586770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c33250 0x55eef3c43cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62efe00640 1 -- 172.21.15.2:0/646586770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c33250 msgr2=0x55eef3c43cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:47.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62efe00640 1 --2- 172.21.15.2:0/646586770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c33250 0x55eef3c43cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62efe00640 1 -- 172.21.15.2:0/646586770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c46d20 msgr2=0x55eef3c44750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:47.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62efe00640 1 --2- 172.21.15.2:0/646586770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c46d20 0x55eef3c44750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.125+0000 7f62efe00640 1 -- 172.21.15.2:0/646586770 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef390a2b0 con 0x55eef3c33b70 2024-09-10T08:07:47.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.126+0000 7f62efe00640 1 --2- 172.21.15.2:0/646586770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c44210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0b85e0 tx=0x7f62dc0b7f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab2518c1748f2c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:47.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.126+0000 7f62d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:07:47.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.126+0000 7f62d6a00640 1 -- 172.21.15.2:0/646586770 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc049070 con 0x55eef3c33b70 2024-09-10T08:07:47.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.126+0000 7f62fa45e740 1 -- 172.21.15.2:0/646586770 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e400fb20 con 0x55eef3c33b70 2024-09-10T08:07:47.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.126+0000 7f62d6a00640 1 -- 172.21.15.2:0/646586770 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f62dc0ecfa0 con 0x55eef3c33b70 2024-09-10T08:07:47.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.126+0000 7f62d6a00640 1 -- 172.21.15.2:0/646586770 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62dc0bb570 con 0x55eef3c33b70 2024-09-10T08:07:47.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.126+0000 7f62fa45e740 10 client.24991 fetch_fsmap learned FSMap version 25 2024-09-10T08:07:47.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.126+0000 7f62fa45e740 1 -- 172.21.15.2:0/646586770 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eef3c49990 con 0x55eef3c33b70 2024-09-10T08:07:47.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.127+0000 7f62d6a00640 1 -- 172.21.15.2:0/646586770 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e400fb20 con 0x55eef3c33b70 2024-09-10T08:07:47.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.127+0000 7f62d6a00640 1 -- 172.21.15.2:0/646586770 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f62dc04c060 con 0x55eef3c33b70 2024-09-10T08:07:47.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.127+0000 7f62fa45e740 10 client.24991 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:07:47.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.127+0000 7f62fa45e740 10 client.24991 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:07:47.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.127+0000 7f62fa45e740 20 client.24991 populate_metadata read hostname 'smithi002' 2024-09-10T08:07:47.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.127+0000 7f62fa45e740 1 --2- 172.21.15.2:0/646586770 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55eef3c57b70 0x55eef3c58fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.127+0000 7f62fa45e740 4 client.24991 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:07:47.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.127+0000 7f62fa45e740 1 -- 172.21.15.2:0/646586770 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eef3ba6620 con 0x55eef3c57b70 2024-09-10T08:07:47.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.127+0000 7f62f4e00640 1 --2- 172.21.15.2:0/646586770 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55eef3c57b70 0x55eef3c58fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.128+0000 7f62f4e00640 1 --2- 172.21.15.2:0/646586770 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55eef3c57b70 0x55eef3c58fe0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fc928b79ffd560a server_cookie=94b192e3d98185f9 in_seq=0 out_seq=0 2024-09-10T08:07:47.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.128+0000 7f62d6a00640 10 client.24991 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:07:47.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62d6a00640 1 -- 172.21.15.2:0/646586770 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eef3ba6620 con 0x55eef3c57b70 2024-09-10T08:07:47.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62d6a00640 10 client.24991 handle_command_reply: tid=0 2024-09-10T08:07:47.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62fa45e740 1 client.24991 shutdown 2024-09-10T08:07:47.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62fa45e740 1 -- 172.21.15.2:0/646586770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 msgr2=0x55eef3c44210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:47.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62fa45e740 1 --2- 172.21.15.2:0/646586770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c44210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0b85e0 tx=0x7f62dc0b7f70 comp rx=0 tx=0).stop 2024-09-10T08:07:47.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62fa45e740 1 -- 172.21.15.2:0/646586770 shutdown_connections 2024-09-10T08:07:47.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62fa45e740 1 --2- 172.21.15.2:0/646586770 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55eef3c57b70 0x55eef3c58fe0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62fa45e740 1 --2- 172.21.15.2:0/646586770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c46d20 0x55eef3c44750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62fa45e740 1 --2- 172.21.15.2:0/646586770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c44210 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62fa45e740 1 --2- 172.21.15.2:0/646586770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c33250 0x55eef3c43cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62d6a00640 0 client.24991 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:07:47.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62fa45e740 1 -- 172.21.15.2:0/646586770 >> 172.21.15.2:0/646586770 conn(0x55eef3c2f8c0 msgr2=0x55eef3c2ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:47.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62fa45e740 1 -- 172.21.15.2:0/646586770 shutdown_connections 2024-09-10T08:07:47.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62fa45e740 1 -- 172.21.15.2:0/646586770 wait complete. 2024-09-10T08:07:47.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.129+0000 7f62fa45e740 20 client.24991 trim_cache size 0 max 16384 2024-09-10T08:07:47.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.148+0000 7f62fa45e740 1 Processor -- start 2024-09-10T08:07:47.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.148+0000 7f62fa45e740 1 -- start start 2024-09-10T08:07:47.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62fa45e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33250 0x55eef3c58fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62fa45e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62fa45e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 0x55eef3c61800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62fa45e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55eef3c57320 con 0x55eef3c33250 2024-09-10T08:07:47.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62fa45e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eef3c579f0 con 0x55eef3c46d20 2024-09-10T08:07:47.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62fa45e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eef3c580c0 con 0x55eef3c33b70 2024-09-10T08:07:47.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62f4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33250 0x55eef3c58fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62f4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33250 0x55eef3c58fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34534/0 (socket says 172.21.15.2:34534) 2024-09-10T08:07:47.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62f4e00640 1 -- 172.21.15.2:0/7632654 learned_addr learned my addr 172.21.15.2:0/7632654 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:47.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62f5800640 1 --2- 172.21.15.2:0/7632654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 0x55eef3c61800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62efe00640 1 --2- 172.21.15.2:0/7632654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62f4e00640 1 -- 172.21.15.2:0/7632654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 msgr2=0x55eef3c612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:47.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62f4e00640 1 --2- 172.21.15.2:0/7632654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62f4e00640 1 -- 172.21.15.2:0/7632654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 msgr2=0x55eef3c61800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:47.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62f4e00640 1 --2- 172.21.15.2:0/7632654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 0x55eef3c61800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.149+0000 7f62f4e00640 1 -- 172.21.15.2:0/7632654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef3c34580 con 0x55eef3c33250 2024-09-10T08:07:47.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.150+0000 7f62f4e00640 1 --2- 172.21.15.2:0/7632654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33250 0x55eef3c58fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f62e40115f0 tx=0x7f62e4019820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476d7f442dc7e585 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:47.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.150+0000 7f62e1600640 1 -- 172.21.15.2:0/7632654 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4028030 con 0x55eef3c33250 2024-09-10T08:07:47.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.150+0000 7f62e1600640 1 -- 172.21.15.2:0/7632654 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e400fb20 con 0x55eef3c33250 2024-09-10T08:07:47.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.150+0000 7f62e1600640 1 -- 172.21.15.2:0/7632654 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e401e040 con 0x55eef3c33250 2024-09-10T08:07:47.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.151+0000 7f62fa45e740 1 -- 172.21.15.2:0/7632654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33250 msgr2=0x55eef3c58fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:47.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.151+0000 7f62fa45e740 1 --2- 172.21.15.2:0/7632654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33250 0x55eef3c58fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f62e40115f0 tx=0x7f62e4019820 comp rx=0 tx=0).stop 2024-09-10T08:07:47.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.151+0000 7f62fa45e740 1 -- 172.21.15.2:0/7632654 shutdown_connections 2024-09-10T08:07:47.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.151+0000 7f62fa45e740 1 --2- 172.21.15.2:0/7632654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 0x55eef3c61800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.151+0000 7f62fa45e740 1 --2- 172.21.15.2:0/7632654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33b70 0x55eef3c612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.151+0000 7f62fa45e740 1 --2- 172.21.15.2:0/7632654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33250 0x55eef3c58fe0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.151+0000 7f62fa45e740 1 -- 172.21.15.2:0/7632654 >> 172.21.15.2:0/7632654 conn(0x55eef3c2f8c0 msgr2=0x55eef3c425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:47.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.151+0000 7f62fa45e740 1 -- 172.21.15.2:0/7632654 shutdown_connections 2024-09-10T08:07:47.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.151+0000 7f62fa45e740 1 -- 172.21.15.2:0/7632654 wait complete. 2024-09-10T08:07:47.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.151+0000 7f62fa45e740 1 -- start start 2024-09-10T08:07:47.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.152+0000 7f62fa45e740 1 Processor -- start 2024-09-10T08:07:47.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.152+0000 7f62fa45e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33250 0x55eef3c5b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.152+0000 7f62fa45e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33b70 0x55eef3c5b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.152+0000 7f62fa45e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 0x55eef3c5bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.152+0000 7f62fa45e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55eef3c58e60 con 0x55eef3c33b70 2024-09-10T08:07:47.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.152+0000 7f62fa45e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eef3c58790 con 0x55eef3c46d20 2024-09-10T08:07:47.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.152+0000 7f62fa45e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eef3c59530 con 0x55eef3c33250 2024-09-10T08:07:47.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.152+0000 7f62fa45e740 10 client.0 _ll_register_callbacks cb 0x7f62dc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:07:47.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.152+0000 7f62f4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33250 0x55eef3c5b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.152+0000 7f62efe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33b70 0x55eef3c5b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.152+0000 7f62f4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33250 0x55eef3c5b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54812/0 (socket says 172.21.15.2:54812) 2024-09-10T08:07:47.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.152+0000 7f62f4e00640 1 -- 172.21.15.2:0/3164666274 learned_addr learned my addr 172.21.15.2:0/3164666274 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:47.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.152+0000 7f62f5800640 1 --2- 172.21.15.2:0/3164666274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 0x55eef3c5bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.153+0000 7f62f4e00640 1 -- 172.21.15.2:0/3164666274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 msgr2=0x55eef3c5bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:47.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.153+0000 7f62f4e00640 1 --2- 172.21.15.2:0/3164666274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 0x55eef3c5bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.153+0000 7f62f4e00640 1 -- 172.21.15.2:0/3164666274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33b70 msgr2=0x55eef3c5b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:07:47.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.153+0000 7f62f4e00640 1 --2- 172.21.15.2:0/3164666274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33b70 0x55eef3c5b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.153+0000 7f62f4e00640 1 -- 172.21.15.2:0/3164666274 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef3c5fd10 con 0x55eef3c33250 2024-09-10T08:07:47.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.153+0000 7f62f4e00640 1 --2- 172.21.15.2:0/3164666274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33250 0x55eef3c5b2d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f62e401c040 tx=0x7f62e4005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf34d1b9cc999f16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:47.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.153+0000 7f62d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:07:47.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.153+0000 7f62d6a00640 1 -- 172.21.15.2:0/3164666274 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4028070 con 0x55eef3c33250 2024-09-10T08:07:47.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.153+0000 7f62fa45e740 1 -- 172.21.15.2:0/3164666274 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e400fb20 con 0x55eef3c33250 2024-09-10T08:07:47.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.153+0000 7f62d6a00640 1 -- 172.21.15.2:0/3164666274 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f62e400fe10 con 0x55eef3c33250 2024-09-10T08:07:47.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:46 smithi155 ceph-mon[35405]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:47.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.154+0000 7f62d6a00640 1 -- 172.21.15.2:0/3164666274 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62e4037050 con 0x55eef3c33250 2024-09-10T08:07:47.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.154+0000 7f62fa45e740 10 client.24997 fetch_fsmap learned FSMap version 25 2024-09-10T08:07:47.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.154+0000 7f62fa45e740 1 -- 172.21.15.2:0/3164666274 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eef3cc27d0 con 0x55eef3c33250 2024-09-10T08:07:47.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.154+0000 7f62d6a00640 1 -- 172.21.15.2:0/3164666274 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e400fb20 con 0x55eef3c33250 2024-09-10T08:07:47.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.154+0000 7f62d6a00640 1 -- 172.21.15.2:0/3164666274 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f62e40035f0 con 0x55eef3c33250 2024-09-10T08:07:47.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.154+0000 7f62fa45e740 10 client.24997 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:07:47.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.154+0000 7f62fa45e740 10 client.24997 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:07:47.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.154+0000 7f62fa45e740 20 client.24997 populate_metadata read hostname 'smithi002' 2024-09-10T08:07:47.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.154+0000 7f62fa45e740 1 --2- 172.21.15.2:0/3164666274 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55eef3cc6c10 0x55eef3cc5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:47.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.154+0000 7f62fa45e740 4 client.24997 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:07:47.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.154+0000 7f62fa45e740 1 -- 172.21.15.2:0/3164666274 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "status"}) -- 0x55eef3ba0b30 con 0x55eef3cc6c10 2024-09-10T08:07:47.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.155+0000 7f62efe00640 1 --2- 172.21.15.2:0/3164666274 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55eef3cc6c10 0x55eef3cc5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:47.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.155+0000 7f62efe00640 1 --2- 172.21.15.2:0/3164666274 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55eef3cc6c10 0x55eef3cc5920 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e948e5043c470095 server_cookie=5d101d11a58a0343 in_seq=0 out_seq=0 2024-09-10T08:07:47.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.155+0000 7f62d6a00640 10 client.24997 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:07:47.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.155+0000 7f62d6a00640 1 -- 172.21.15.2:0/3164666274 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55eef3ba0b30 con 0x55eef3cc6c10 2024-09-10T08:07:47.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.155+0000 7f62d6a00640 10 client.24997 handle_command_reply: tid=0 2024-09-10T08:07:47.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.155+0000 7f62fa45e740 1 client.24997 shutdown 2024-09-10T08:07:47.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.156+0000 7f62fa45e740 1 -- 172.21.15.2:0/3164666274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33250 msgr2=0x55eef3c5b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:47.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.156+0000 7f62fa45e740 1 --2- 172.21.15.2:0/3164666274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33250 0x55eef3c5b2d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f62e401c040 tx=0x7f62e4005d90 comp rx=0 tx=0).stop 2024-09-10T08:07:47.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.156+0000 7f62fa45e740 1 -- 172.21.15.2:0/3164666274 shutdown_connections 2024-09-10T08:07:47.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.156+0000 7f62fa45e740 1 --2- 172.21.15.2:0/3164666274 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55eef3cc6c10 0x55eef3cc5920 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.156+0000 7f62fa45e740 1 --2- 172.21.15.2:0/3164666274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef3c46d20 0x55eef3c5bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.156+0000 7f62fa45e740 1 --2- 172.21.15.2:0/3164666274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eef3c33b70 0x55eef3c5b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.156+0000 7f62fa45e740 1 --2- 172.21.15.2:0/3164666274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef3c33250 0x55eef3c5b2d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.156+0000 7f62d6a00640 0 client.24997 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:07:47.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.156+0000 7f62fa45e740 1 -- 172.21.15.2:0/3164666274 >> 172.21.15.2:0/3164666274 conn(0x55eef3c2f8c0 msgr2=0x55eef3c51d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:47.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.156+0000 7f62fa45e740 1 -- 172.21.15.2:0/3164666274 shutdown_connections 2024-09-10T08:07:47.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.156+0000 7f62fa45e740 1 -- 172.21.15.2:0/3164666274 wait complete. 2024-09-10T08:07:47.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.156+0000 7f62fa45e740 20 client.24997 trim_cache size 0 max 16384 2024-09-10T08:07:47.161 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:07:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "26ec64b0-6f4a-11ef-bcea-c7b262605968", 2024-09-10T08:07:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-09-10T08:07:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24536, 2024-09-10T08:07:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-09-10T08:07:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-09-10T08:07:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-09-10T08:07:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 139.667400034, 2024-09-10T08:07:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 25, 2024-09-10T08:07:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 95, 2024-09-10T08:07:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 93, 2024-09-10T08:07:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 153.545968985 2024-09-10T08:07:47.162 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:07:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.157+0000 7f62f7800640 1 -- 172.21.15.2:0/1981478017 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62d80a2c90 msgr2=0x7f62d80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.157+0000 7f62f7800640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62d80a2c90 0x7f62d80a5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005fd0 tx=0x7f62e8009500 comp rx=0 tx=0).stop 2024-09-10T08:07:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.157+0000 7f62f7800640 1 -- 172.21.15.2:0/1981478017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62f0102790 msgr2=0x7f62f019abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.157+0000 7f62f7800640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62f0102790 0x7f62f019abd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f62dc004770 tx=0x7f62dc003c10 comp rx=0 tx=0).stop 2024-09-10T08:07:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.157+0000 7f62f7800640 1 -- 172.21.15.2:0/1981478017 shutdown_connections 2024-09-10T08:07:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.157+0000 7f62f7800640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62d80a2c90 0x7f62d80a5150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.157+0000 7f62f7800640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62f0106f10 0x7f62f019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.157+0000 7f62f7800640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62f0102790 0x7f62f019abd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.157+0000 7f62f7800640 1 --2- 172.21.15.2:0/1981478017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0101de0 0x7f62f019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.157+0000 7f62f7800640 1 -- 172.21.15.2:0/1981478017 >> 172.21.15.2:0/1981478017 conn(0x7f62f00fd8c0 msgr2=0x7f62f00ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.157+0000 7f62f7800640 1 -- 172.21.15.2:0/1981478017 shutdown_connections 2024-09-10T08:07:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:47.157+0000 7f62f7800640 1 -- 172.21.15.2:0/1981478017 wait complete. 2024-09-10T08:07:47.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:46 smithi107 ceph-mon[35048]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:48.004 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T08:07:48.004 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T08:07:48.019 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-09-10T08:07:48.020 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T08:07:48.400 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:48.853 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:48 smithi002 ceph-mon[30563]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:48.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.967+0000 7f79cf200640 1 -- 172.21.15.2:0/3657363327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79d0103600 msgr2=0x7f79d0109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.967+0000 7f79cf200640 1 --2- 172.21.15.2:0/3657363327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79d0103600 0x7f79d0109e90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009ca0 tx=0x7f79c802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:07:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.967+0000 7f79cf200640 1 -- 172.21.15.2:0/3657363327 shutdown_connections 2024-09-10T08:07:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.967+0000 7f79cf200640 1 --2- 172.21.15.2:0/3657363327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79d0103600 0x7f79d0109e90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.967+0000 7f79cf200640 1 --2- 172.21.15.2:0/3657363327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79d0102c40 0x7f79d01030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.967+0000 7f79cf200640 1 --2- 172.21.15.2:0/3657363327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0101a40 0x7f79d0101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.967+0000 7f79cf200640 1 -- 172.21.15.2:0/3657363327 >> 172.21.15.2:0/3657363327 conn(0x7f79d00fd1f0 msgr2=0x7f79d00ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.967+0000 7f79cf200640 1 -- 172.21.15.2:0/3657363327 shutdown_connections 2024-09-10T08:07:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.967+0000 7f79cf200640 1 -- 172.21.15.2:0/3657363327 wait complete. 2024-09-10T08:07:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.968+0000 7f79cf200640 1 Processor -- start 2024-09-10T08:07:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.968+0000 7f79cf200640 1 -- start start 2024-09-10T08:07:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cf200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0101a40 0x7f79d0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:48.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cf200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79d0102c40 0x7f79d00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cf200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79d0103600 0x7f79d0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cf200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f79d010d6a0 con 0x7f79d0103600 2024-09-10T08:07:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cf200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f79d010d520 con 0x7f79d0102c40 2024-09-10T08:07:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cf200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79d010d820 con 0x7f79d0101a40 2024-09-10T08:07:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0101a40 0x7f79d0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0101a40 0x7f79d0075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54838/0 (socket says 172.21.15.2:54838) 2024-09-10T08:07:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cde00640 1 -- 172.21.15.2:0/2275715009 learned_addr learned my addr 172.21.15.2:0/2275715009 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79ce800640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79d0103600 0x7f79d0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cd400640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79d0102c40 0x7f79d00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cde00640 1 -- 172.21.15.2:0/2275715009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79d0102c40 msgr2=0x7f79d00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cde00640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79d0102c40 0x7f79d00745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cde00640 1 -- 172.21.15.2:0/2275715009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79d0103600 msgr2=0x7f79d0074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:48.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cde00640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79d0103600 0x7f79d0074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:48.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.969+0000 7f79cde00640 1 -- 172.21.15.2:0/2275715009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d00753e0 con 0x7f79d0101a40 2024-09-10T08:07:48.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.970+0000 7f79cde00640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0101a40 0x7f79d0075f90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f79c000d770 tx=0x7f79c000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:48.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.970+0000 7f79c6a00640 1 -- 172.21.15.2:0/2275715009 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0017070 con 0x7f79d0101a40 2024-09-10T08:07:48.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.970+0000 7f79c6a00640 1 -- 172.21.15.2:0/2275715009 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c0009c30 con 0x7f79d0101a40 2024-09-10T08:07:48.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.970+0000 7f79cf200640 1 -- 172.21.15.2:0/2275715009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d01a1e20 con 0x7f79d0101a40 2024-09-10T08:07:48.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.970+0000 7f79c6a00640 1 -- 172.21.15.2:0/2275715009 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0003dd0 con 0x7f79d0101a40 2024-09-10T08:07:48.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.970+0000 7f79cf200640 1 -- 172.21.15.2:0/2275715009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d01a2210 con 0x7f79d0101a40 2024-09-10T08:07:48.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.971+0000 7f79cf200640 1 -- 172.21.15.2:0/2275715009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d0108df0 con 0x7f79d0101a40 2024-09-10T08:07:48.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.973+0000 7f79c6a00640 1 -- 172.21.15.2:0/2275715009 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f79c0003fb0 con 0x7f79d0101a40 2024-09-10T08:07:48.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.974+0000 7f79c6a00640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79b40a2c10 0x7f79b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:48.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.974+0000 7f79c6a00640 1 -- 172.21.15.2:0/2275715009 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f79c00cf060 con 0x7f79d0101a40 2024-09-10T08:07:48.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.975+0000 7f79cd400640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79b40a2c10 0x7f79b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:48.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.975+0000 7f79cd400640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79b40a2c10 0x7f79b40a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f79d0075d10 tx=0x7f79bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:48.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:48.977+0000 7f79c6a00640 1 -- 172.21.15.2:0/2275715009 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79c0009de0 con 0x7f79d0101a40 2024-09-10T08:07:49.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:48 smithi155 ceph-mon[35405]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:49.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:48 smithi107 ceph-mon[35048]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:49.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.180+0000 7f79cf200640 1 -- 172.21.15.2:0/2275715009 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f79d0062630 con 0x7f79b40a2c10 2024-09-10T08:07:49.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.181+0000 7f79c6a00640 1 -- 172.21.15.2:0/2275715009 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f79d0062630 con 0x7f79b40a2c10 2024-09-10T08:07:49.181 INFO:teuthology.orchestra.run.smithi002.stdout:Backend: cephadm 2024-09-10T08:07:49.181 INFO:teuthology.orchestra.run.smithi002.stdout:Available: Yes 2024-09-10T08:07:49.181 INFO:teuthology.orchestra.run.smithi002.stdout:Paused: No 2024-09-10T08:07:49.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.183+0000 7f79cf200640 1 -- 172.21.15.2:0/2275715009 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79b40a2c10 msgr2=0x7f79b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:49.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.183+0000 7f79cf200640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79b40a2c10 0x7f79b40a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f79d0075d10 tx=0x7f79bc005f90 comp rx=0 tx=0).stop 2024-09-10T08:07:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.183+0000 7f79cf200640 1 -- 172.21.15.2:0/2275715009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0101a40 msgr2=0x7f79d0075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.183+0000 7f79cf200640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0101a40 0x7f79d0075f90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f79c000d770 tx=0x7f79c000dc30 comp rx=0 tx=0).stop 2024-09-10T08:07:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.184+0000 7f79cf200640 1 -- 172.21.15.2:0/2275715009 shutdown_connections 2024-09-10T08:07:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.184+0000 7f79cf200640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79d0103600 0x7f79d0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.184+0000 7f79cf200640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79d0102c40 0x7f79d00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.184+0000 7f79cf200640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79b40a2c10 0x7f79b40a50d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.184+0000 7f79cf200640 1 --2- 172.21.15.2:0/2275715009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0101a40 0x7f79d0075f90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.184+0000 7f79cf200640 1 -- 172.21.15.2:0/2275715009 >> 172.21.15.2:0/2275715009 conn(0x7f79d00fd1f0 msgr2=0x7f79d0105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:49.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.184+0000 7f79cf200640 1 -- 172.21.15.2:0/2275715009 shutdown_connections 2024-09-10T08:07:49.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:49.184+0000 7f79cf200640 1 -- 172.21.15.2:0/2275715009 wait complete. 2024-09-10T08:07:49.882 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:49 smithi002 ceph-mon[30563]: from='client.25003 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:49.924 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T08:07:50.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:49 smithi155 ceph-mon[35405]: from='client.25003 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:50.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:49 smithi107 ceph-mon[35048]: from='client.25003 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:50.310 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:50.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.915+0000 7f2fd0a00640 1 -- 172.21.15.2:0/1606652690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fcc077df0 msgr2=0x7f2fcc078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:50.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.915+0000 7f2fd0a00640 1 --2- 172.21.15.2:0/1606652690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fcc077df0 0x7f2fcc078250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009900 tx=0x7f2fb802a410 comp rx=0 tx=0).stop 2024-09-10T08:07:50.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.915+0000 7f2fd0a00640 1 -- 172.21.15.2:0/1606652690 shutdown_connections 2024-09-10T08:07:50.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.915+0000 7f2fd0a00640 1 --2- 172.21.15.2:0/1606652690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fcc078790 0x7f2fcc079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:50.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.915+0000 7f2fd0a00640 1 --2- 172.21.15.2:0/1606652690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fcc077df0 0x7f2fcc078250 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:50.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.915+0000 7f2fd0a00640 1 --2- 172.21.15.2:0/1606652690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fcc076ba0 0x7f2fcc076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:50.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.915+0000 7f2fd0a00640 1 -- 172.21.15.2:0/1606652690 >> 172.21.15.2:0/1606652690 conn(0x7f2fcc0ff3e0 msgr2=0x7f2fcc101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:50.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.915+0000 7f2fd0a00640 1 -- 172.21.15.2:0/1606652690 shutdown_connections 2024-09-10T08:07:50.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.915+0000 7f2fd0a00640 1 -- 172.21.15.2:0/1606652690 wait complete. 2024-09-10T08:07:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.916+0000 7f2fd0a00640 1 Processor -- start 2024-09-10T08:07:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.917+0000 7f2fd0a00640 1 -- start start 2024-09-10T08:07:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.917+0000 7f2fd0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fcc076ba0 0x7f2fcc19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.917+0000 7f2fd0a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fcc077df0 0x7f2fcc19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.917+0000 7f2fd0a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fcc078790 0x7f2fcc1a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.917+0000 7f2fd0a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc113d60 con 0x7f2fcc078790 2024-09-10T08:07:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.917+0000 7f2fd0a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc113be0 con 0x7f2fcc076ba0 2024-09-10T08:07:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.917+0000 7f2fd0a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc113ee0 con 0x7f2fcc077df0 2024-09-10T08:07:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.917+0000 7f2fcbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fcc078790 0x7f2fcc1a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.917+0000 7f2fcaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fcc077df0 0x7f2fcc19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.917+0000 7f2fcb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fcc076ba0 0x7f2fcc19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.917+0000 7f2fcbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fcc078790 0x7f2fcc1a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34580/0 (socket says 172.21.15.2:34580) 2024-09-10T08:07:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.917+0000 7f2fcaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fcc077df0 0x7f2fcc19e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54852/0 (socket says 172.21.15.2:54852) 2024-09-10T08:07:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.917+0000 7f2fcbe00640 1 -- 172.21.15.2:0/464180477 learned_addr learned my addr 172.21.15.2:0/464180477 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.918+0000 7f2fcaa00640 1 -- 172.21.15.2:0/464180477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fcc076ba0 msgr2=0x7f2fcc19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.918+0000 7f2fcaa00640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fcc076ba0 0x7f2fcc19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.918+0000 7f2fcaa00640 1 -- 172.21.15.2:0/464180477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fcc078790 msgr2=0x7f2fcc1a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:50.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.918+0000 7f2fcaa00640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fcc078790 0x7f2fcc1a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:50.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.918+0000 7f2fcaa00640 1 -- 172.21.15.2:0/464180477 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc1a5f90 con 0x7f2fcc077df0 2024-09-10T08:07:50.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.918+0000 7f2fcbe00640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fcc078790 0x7f2fcc1a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:50.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.918+0000 7f2fcb400640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fcc076ba0 0x7f2fcc19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:50.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.918+0000 7f2fcaa00640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fcc077df0 0x7f2fcc19e8a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2fb802a930 tx=0x7f2fb8032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.919+0000 7f2fc3e00640 1 -- 172.21.15.2:0/464180477 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb803f030 con 0x7f2fcc077df0 2024-09-10T08:07:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.919+0000 7f2fc3e00640 1 -- 172.21.15.2:0/464180477 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb8003ce0 con 0x7f2fcc077df0 2024-09-10T08:07:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.919+0000 7f2fc3e00640 1 -- 172.21.15.2:0/464180477 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8035070 con 0x7f2fcc077df0 2024-09-10T08:07:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.919+0000 7f2fd0a00640 1 -- 172.21.15.2:0/464180477 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fcc1a6220 con 0x7f2fcc077df0 2024-09-10T08:07:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.919+0000 7f2fd0a00640 1 -- 172.21.15.2:0/464180477 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fcc1a66d0 con 0x7f2fcc077df0 2024-09-10T08:07:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.920+0000 7f2fd0a00640 1 -- 172.21.15.2:0/464180477 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fcc1105d0 con 0x7f2fcc077df0 2024-09-10T08:07:50.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.922+0000 7f2fc3e00640 1 -- 172.21.15.2:0/464180477 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2fb8042030 con 0x7f2fcc077df0 2024-09-10T08:07:50.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.924+0000 7f2fc3e00640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2fb00a2c40 0x7f2fb00a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:50.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.924+0000 7f2fc3e00640 1 -- 172.21.15.2:0/464180477 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2fb80ef980 con 0x7f2fcc077df0 2024-09-10T08:07:50.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.924+0000 7f2fcb400640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2fb00a2c40 0x7f2fb00a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:50.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.925+0000 7f2fcb400640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2fb00a2c40 0x7f2fb00a5100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc009fc0 tx=0x7f2fbc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:50.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:50.927+0000 7f2fc3e00640 1 -- 172.21.15.2:0/464180477 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2fb80bbce0 con 0x7f2fcc077df0 2024-09-10T08:07:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:50 smithi002 ceph-mon[30563]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:51.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.122+0000 7f2fd0a00640 1 -- 172.21.15.2:0/464180477 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2fcc062630 con 0x7f2fb00a2c40 2024-09-10T08:07:51.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.134+0000 7f2fc3e00640 1 -- 172.21.15.2:0/464180477 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f2fcc062630 con 0x7f2fb00a2c40 2024-09-10T08:07:51.134 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T08:07:51.134 INFO:teuthology.orchestra.run.smithi002.stdout:mds.a smithi002 running (2m) 21s ago 2m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 aee775cee0bf 2024-09-10T08:07:51.135 INFO:teuthology.orchestra.run.smithi002.stdout:mds.b smithi107 running (2m) 24s ago 2m 26.2M - 19.1.1-305-g1cf819b7 41d671128357 9f5ccb59747e 2024-09-10T08:07:51.135 INFO:teuthology.orchestra.run.smithi002.stdout:mds.c smithi155 running (2m) 24s ago 2m 19.2M - 19.1.1-305-g1cf819b7 41d671128357 4ad1423fd2e4 2024-09-10T08:07:51.135 INFO:teuthology.orchestra.run.smithi002.stdout:mds.d smithi002 running (3m) 21s ago 3m 13.2M - 19.1.1-305-g1cf819b7 41d671128357 ab11db6cfd54 2024-09-10T08:07:51.135 INFO:teuthology.orchestra.run.smithi002.stdout:mds.e smithi107 running (2m) 24s ago 2m 19.5M - 19.1.1-305-g1cf819b7 41d671128357 f1a76df045aa 2024-09-10T08:07:51.135 INFO:teuthology.orchestra.run.smithi002.stdout:mds.f smithi155 running (2m) 24s ago 2m 18.0M - 19.1.1-305-g1cf819b7 41d671128357 71aa353b688c 2024-09-10T08:07:51.135 INFO:teuthology.orchestra.run.smithi002.stdout:mds.g smithi002 running (2m) 21s ago 2m 12.9M - 19.1.1-305-g1cf819b7 41d671128357 0cf6862e27b0 2024-09-10T08:07:51.135 INFO:teuthology.orchestra.run.smithi002.stdout:mds.h smithi107 running (2m) 24s ago 2m 18.9M - 19.1.1-305-g1cf819b7 41d671128357 ebd0d0571fa6 2024-09-10T08:07:51.135 INFO:teuthology.orchestra.run.smithi002.stdout:mds.i smithi155 running (2m) 24s ago 2m 20.6M - 19.1.1-305-g1cf819b7 41d671128357 af41c8137399 2024-09-10T08:07:51.135 INFO:teuthology.orchestra.run.smithi002.stdout:mds.j smithi002 running (3m) 21s ago 3m 13.1M - 19.1.1-305-g1cf819b7 41d671128357 c5db02483c0c 2024-09-10T08:07:51.135 INFO:teuthology.orchestra.run.smithi002.stdout:mds.k smithi107 running (3m) 24s ago 3m 14.7M - 19.1.1-305-g1cf819b7 41d671128357 0740ab8b90ae 2024-09-10T08:07:51.135 INFO:teuthology.orchestra.run.smithi002.stdout:mds.l smithi155 running (3m) 24s ago 3m 14.7M - 19.1.1-305-g1cf819b7 41d671128357 23f2a8c101e0 2024-09-10T08:07:51.135 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.x smithi002 *:9283,8765 running (10m) 21s ago 10m 509M - 19.1.1-305-g1cf819b7 41d671128357 3688b0cf814b 2024-09-10T08:07:51.136 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.y smithi107 *:8443,8765 running (8m) 24s ago 8m 442M - 19.1.1-305-g1cf819b7 41d671128357 eeb53500d432 2024-09-10T08:07:51.136 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.z smithi155 *:8443,8765 running (8m) 24s ago 8m 443M - 19.1.1-305-g1cf819b7 41d671128357 30f2ea8a4b4d 2024-09-10T08:07:51.136 INFO:teuthology.orchestra.run.smithi002.stdout:mon.a smithi002 running (10m) 21s ago 10m 49.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 f3a068f1b964 2024-09-10T08:07:51.136 INFO:teuthology.orchestra.run.smithi002.stdout:mon.b smithi107 running (9m) 24s ago 9m 42.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 e8e08e82260c 2024-09-10T08:07:51.136 INFO:teuthology.orchestra.run.smithi002.stdout:mon.c smithi155 running (9m) 24s ago 9m 41.8M 2048M 19.1.1-305-g1cf819b7 41d671128357 39a0f8f0def5 2024-09-10T08:07:51.136 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (8m) 21s ago 8m 44.9M 4273M 19.1.1-305-g1cf819b7 41d671128357 f7b6a8b63940 2024-09-10T08:07:51.136 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi107 running (8m) 24s ago 7m 46.6M 4271M 19.1.1-305-g1cf819b7 41d671128357 b807476a103b 2024-09-10T08:07:51.136 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi155 running (7m) 24s ago 7m 49.7M 4272M 19.1.1-305-g1cf819b7 41d671128357 b497a2e23209 2024-09-10T08:07:51.136 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi002 running (7m) 21s ago 7m 41.9M 4273M 19.1.1-305-g1cf819b7 41d671128357 697e3a316d81 2024-09-10T08:07:51.136 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi107 running (6m) 24s ago 6m 43.3M 4271M 19.1.1-305-g1cf819b7 41d671128357 1a80374265dc 2024-09-10T08:07:51.136 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi155 running (6m) 24s ago 6m 55.4M 4272M 19.1.1-305-g1cf819b7 41d671128357 e4200126d24a 2024-09-10T08:07:51.136 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 smithi002 running (5m) 21s ago 5m 42.3M 4273M 19.1.1-305-g1cf819b7 41d671128357 370556ae7ce8 2024-09-10T08:07:51.137 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 smithi107 running (5m) 24s ago 5m 58.0M 4271M 19.1.1-305-g1cf819b7 41d671128357 c59e4d9ba134 2024-09-10T08:07:51.137 INFO:teuthology.orchestra.run.smithi002.stdout:osd.8 smithi155 running (4m) 24s ago 4m 56.6M 4272M 19.1.1-305-g1cf819b7 41d671128357 83bf4c2aa07e 2024-09-10T08:07:51.137 INFO:teuthology.orchestra.run.smithi002.stdout:osd.9 smithi002 running (4m) 21s ago 4m 52.3M 4273M 19.1.1-305-g1cf819b7 41d671128357 f6dbc8f39405 2024-09-10T08:07:51.137 INFO:teuthology.orchestra.run.smithi002.stdout:osd.10 smithi107 running (3m) 24s ago 3m 57.7M 4271M 19.1.1-305-g1cf819b7 41d671128357 f0ec50d5c682 2024-09-10T08:07:51.137 INFO:teuthology.orchestra.run.smithi002.stdout:osd.11 smithi155 running (3m) 24s ago 3m 44.1M 4272M 19.1.1-305-g1cf819b7 41d671128357 c9dcc9c9105e 2024-09-10T08:07:51.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.136+0000 7f2fd0a00640 1 -- 172.21.15.2:0/464180477 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2fb00a2c40 msgr2=0x7f2fb00a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:51.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.136+0000 7f2fd0a00640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2fb00a2c40 0x7f2fb00a5100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc009fc0 tx=0x7f2fbc005f90 comp rx=0 tx=0).stop 2024-09-10T08:07:51.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.136+0000 7f2fd0a00640 1 -- 172.21.15.2:0/464180477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fcc077df0 msgr2=0x7f2fcc19e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:51.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.136+0000 7f2fd0a00640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fcc077df0 0x7f2fcc19e8a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2fb802a930 tx=0x7f2fb8032ce0 comp rx=0 tx=0).stop 2024-09-10T08:07:51.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.137+0000 7f2fd0a00640 1 -- 172.21.15.2:0/464180477 shutdown_connections 2024-09-10T08:07:51.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.137+0000 7f2fd0a00640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fcc078790 0x7f2fcc1a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:51.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.137+0000 7f2fd0a00640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2fb00a2c40 0x7f2fb00a5100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:51.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.137+0000 7f2fd0a00640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fcc077df0 0x7f2fcc19e8a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:51.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.137+0000 7f2fd0a00640 1 --2- 172.21.15.2:0/464180477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fcc076ba0 0x7f2fcc19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:51.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.137+0000 7f2fd0a00640 1 -- 172.21.15.2:0/464180477 >> 172.21.15.2:0/464180477 conn(0x7f2fcc0ff3e0 msgr2=0x7f2fcc100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:51.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.137+0000 7f2fd0a00640 1 -- 172.21.15.2:0/464180477 shutdown_connections 2024-09-10T08:07:51.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:51.137+0000 7f2fd0a00640 1 -- 172.21.15.2:0/464180477 wait complete. 2024-09-10T08:07:51.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:50 smithi155 ceph-mon[35405]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:51.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:50 smithi107 ceph-mon[35048]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:07:51.891 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:51 smithi002 ceph-mon[30563]: from='client.25009 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:51.929 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T08:07:52.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:51 smithi155 ceph-mon[35405]: from='client.25009 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:52.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:51 smithi107 ceph-mon[35048]: from='client.25009 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:52.309 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.951+0000 7f3ba0800640 1 -- 172.21.15.2:0/3931464557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b98102e00 msgr2=0x7f3b98103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.951+0000 7f3ba0800640 1 --2- 172.21.15.2:0/3931464557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b98102e00 0x7f3b98103280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009900 tx=0x7f3b8802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.952+0000 7f3ba0800640 1 -- 172.21.15.2:0/3931464557 shutdown_connections 2024-09-10T08:07:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.952+0000 7f3ba0800640 1 --2- 172.21.15.2:0/3931464557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b981037c0 0x7f3b9810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.952+0000 7f3ba0800640 1 --2- 172.21.15.2:0/3931464557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b98102e00 0x7f3b98103280 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.952+0000 7f3ba0800640 1 --2- 172.21.15.2:0/3931464557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b98101c00 0x7f3b98102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.952+0000 7f3ba0800640 1 -- 172.21.15.2:0/3931464557 >> 172.21.15.2:0/3931464557 conn(0x7f3b980fd390 msgr2=0x7f3b980ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:52.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.952+0000 7f3ba0800640 1 -- 172.21.15.2:0/3931464557 shutdown_connections 2024-09-10T08:07:52.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.952+0000 7f3ba0800640 1 -- 172.21.15.2:0/3931464557 wait complete. 2024-09-10T08:07:52.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.953+0000 7f3ba0800640 1 Processor -- start 2024-09-10T08:07:52.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.953+0000 7f3ba0800640 1 -- start start 2024-09-10T08:07:52.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.953+0000 7f3ba0800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b98101c00 0x7f3b9819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:52.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.953+0000 7f3ba0800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b98102e00 0x7f3b9819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:52.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.953+0000 7f3ba0800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b981037c0 0x7f3b981a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:52.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.953+0000 7f3ba0800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9810d860 con 0x7f3b981037c0 2024-09-10T08:07:52.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.953+0000 7f3ba0800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9810d6e0 con 0x7f3b98101c00 2024-09-10T08:07:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.953+0000 7f3ba0800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9810d9e0 con 0x7f3b98102e00 2024-09-10T08:07:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.954+0000 7f3b9e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b981037c0 0x7f3b981a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.954+0000 7f3b9d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b98102e00 0x7f3b9819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.954+0000 7f3b9d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b98102e00 0x7f3b9819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54866/0 (socket says 172.21.15.2:54866) 2024-09-10T08:07:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.954+0000 7f3b9de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b98101c00 0x7f3b9819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.954+0000 7f3b9d400640 1 -- 172.21.15.2:0/2674682287 learned_addr learned my addr 172.21.15.2:0/2674682287 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.954+0000 7f3b9e800640 1 -- 172.21.15.2:0/2674682287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b98102e00 msgr2=0x7f3b9819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.954+0000 7f3b9e800640 1 --2- 172.21.15.2:0/2674682287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b98102e00 0x7f3b9819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.954+0000 7f3b9e800640 1 -- 172.21.15.2:0/2674682287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b98101c00 msgr2=0x7f3b9819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.954+0000 7f3b9e800640 1 --2- 172.21.15.2:0/2674682287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b98101c00 0x7f3b9819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.954+0000 7f3b9e800640 1 -- 172.21.15.2:0/2674682287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b981a1d30 con 0x7f3b981037c0 2024-09-10T08:07:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.954+0000 7f3b9e800640 1 --2- 172.21.15.2:0/2674682287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b981037c0 0x7f3b981a16c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400da10 tx=0x7f3b9400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.955+0000 7f3b92a00640 1 -- 172.21.15.2:0/2674682287 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9400bd70 con 0x7f3b981037c0 2024-09-10T08:07:52.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.955+0000 7f3b92a00640 1 -- 172.21.15.2:0/2674682287 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b94003e50 con 0x7f3b981037c0 2024-09-10T08:07:52.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.955+0000 7f3b92a00640 1 -- 172.21.15.2:0/2674682287 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b940106a0 con 0x7f3b981037c0 2024-09-10T08:07:52.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.955+0000 7f3ba0800640 1 -- 172.21.15.2:0/2674682287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b981a1fc0 con 0x7f3b981037c0 2024-09-10T08:07:52.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.955+0000 7f3ba0800640 1 -- 172.21.15.2:0/2674682287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b981a2480 con 0x7f3b981037c0 2024-09-10T08:07:52.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.956+0000 7f3ba0800640 1 -- 172.21.15.2:0/2674682287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b981a2710 con 0x7f3b981037c0 2024-09-10T08:07:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.958+0000 7f3b92a00640 1 -- 172.21.15.2:0/2674682287 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3b94004b40 con 0x7f3b981037c0 2024-09-10T08:07:52.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.959+0000 7f3b92a00640 1 --2- 172.21.15.2:0/2674682287 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3b800a2c40 0x7f3b800a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:52.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.959+0000 7f3b92a00640 1 -- 172.21.15.2:0/2674682287 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3b940cfc90 con 0x7f3b981037c0 2024-09-10T08:07:52.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.960+0000 7f3b9de00640 1 --2- 172.21.15.2:0/2674682287 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3b800a2c40 0x7f3b800a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:52.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.960+0000 7f3b9de00640 1 --2- 172.21.15.2:0/2674682287 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3b800a2c40 0x7f3b800a5100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3b98102c60 tx=0x7f3b8c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:52.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:52.962+0000 7f3b92a00640 1 -- 172.21.15.2:0/2674682287 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b9409c640 con 0x7f3b981037c0 2024-09-10T08:07:53.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:52 smithi002 ceph-mon[30563]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:53.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:52 smithi155 ceph-mon[35405]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.160+0000 7f3ba0800640 1 -- 172.21.15.2:0/2674682287 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3b981a28e0 con 0x7f3b800a2c40 2024-09-10T08:07:53.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.166+0000 7f3b92a00640 1 -- 172.21.15.2:0/2674682287 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3b981a28e0 con 0x7f3b800a2c40 2024-09-10T08:07:53.166 INFO:teuthology.orchestra.run.smithi002.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T08:07:53.166 INFO:teuthology.orchestra.run.smithi002.stdout:mds.all 12/12 26s ago 3m smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-09-10T08:07:53.166 INFO:teuthology.orchestra.run.smithi002.stdout:mgr 3/3 26s ago 8m smithi002=x;smithi107=y;smithi155=z;count:3 2024-09-10T08:07:53.166 INFO:teuthology.orchestra.run.smithi002.stdout:mon 3/3 26s ago 9m smithi002:172.21.15.2=a;smithi107:172.21.15.107=b;smithi155:172.21.15.155=c;count:3 2024-09-10T08:07:53.166 INFO:teuthology.orchestra.run.smithi002.stdout:osd 12 26s ago - 2024-09-10T08:07:53.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:52 smithi107 ceph-mon[35048]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:53.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.168+0000 7f3ba0800640 1 -- 172.21.15.2:0/2674682287 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3b800a2c40 msgr2=0x7f3b800a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.168+0000 7f3ba0800640 1 --2- 172.21.15.2:0/2674682287 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3b800a2c40 0x7f3b800a5100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3b98102c60 tx=0x7f3b8c00b390 comp rx=0 tx=0).stop 2024-09-10T08:07:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.168+0000 7f3ba0800640 1 -- 172.21.15.2:0/2674682287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b981037c0 msgr2=0x7f3b981a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.168+0000 7f3ba0800640 1 --2- 172.21.15.2:0/2674682287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b981037c0 0x7f3b981a16c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400da10 tx=0x7f3b9400ded0 comp rx=0 tx=0).stop 2024-09-10T08:07:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.168+0000 7f3ba0800640 1 -- 172.21.15.2:0/2674682287 shutdown_connections 2024-09-10T08:07:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.168+0000 7f3ba0800640 1 --2- 172.21.15.2:0/2674682287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b981037c0 0x7f3b981a16c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.168+0000 7f3ba0800640 1 --2- 172.21.15.2:0/2674682287 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3b800a2c40 0x7f3b800a5100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.168+0000 7f3ba0800640 1 --2- 172.21.15.2:0/2674682287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b98102e00 0x7f3b9819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.168+0000 7f3ba0800640 1 --2- 172.21.15.2:0/2674682287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b98101c00 0x7f3b9819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.168+0000 7f3ba0800640 1 -- 172.21.15.2:0/2674682287 >> 172.21.15.2:0/2674682287 conn(0x7f3b980fd390 msgr2=0x7f3b98074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:53.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.169+0000 7f3ba0800640 1 -- 172.21.15.2:0/2674682287 shutdown_connections 2024-09-10T08:07:53.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:53.169+0000 7f3ba0800640 1 -- 172.21.15.2:0/2674682287 wait complete. 2024-09-10T08:07:53.974 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:53 smithi002 ceph-mon[30563]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:54.012 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T08:07:54.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:53 smithi155 ceph-mon[35405]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:54.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:53 smithi107 ceph-mon[35048]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:54.391 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:54.836 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:54 smithi002 ceph-mon[30563]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:54.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.952+0000 7f83a1400640 1 -- 172.21.15.2:0/3056534157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f839c105920 msgr2=0x7f839c10c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:54.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.952+0000 7f83a1400640 1 --2- 172.21.15.2:0/3056534157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f839c105920 0x7f839c10c240 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8394009900 tx=0x7f839402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:54.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.953+0000 7f83a1400640 1 -- 172.21.15.2:0/3056534157 shutdown_connections 2024-09-10T08:07:54.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.953+0000 7f83a1400640 1 --2- 172.21.15.2:0/3056534157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f839c105920 0x7f839c10c240 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:54.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.953+0000 7f83a1400640 1 --2- 172.21.15.2:0/3056534157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f839c104f60 0x7f839c1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:54.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.953+0000 7f83a1400640 1 --2- 172.21.15.2:0/3056534157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f839c103e00 0x7f839c104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:54.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.953+0000 7f83a1400640 1 -- 172.21.15.2:0/3056534157 >> 172.21.15.2:0/3056534157 conn(0x7f839c0ff5b0 msgr2=0x7f839c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:54.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.953+0000 7f83a1400640 1 -- 172.21.15.2:0/3056534157 shutdown_connections 2024-09-10T08:07:54.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.953+0000 7f83a1400640 1 -- 172.21.15.2:0/3056534157 wait complete. 2024-09-10T08:07:54.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.954+0000 7f83a1400640 1 Processor -- start 2024-09-10T08:07:54.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.954+0000 7f83a1400640 1 -- start start 2024-09-10T08:07:54.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.954+0000 7f83a1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f839c103e00 0x7f839c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:54.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.954+0000 7f83a1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f839c104f60 0x7f839c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:54.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.954+0000 7f83a1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f839c105920 0x7f839c1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:54.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.954+0000 7f83a1400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f839c10fae0 con 0x7f839c103e00 2024-09-10T08:07:54.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.954+0000 7f83a1400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f839c10f960 con 0x7f839c105920 2024-09-10T08:07:54.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.954+0000 7f83a1400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f839c10fc60 con 0x7f839c104f60 2024-09-10T08:07:54.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.954+0000 7f839a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f839c103e00 0x7f839c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:54.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.954+0000 7f839a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f839c103e00 0x7f839c19a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34626/0 (socket says 172.21.15.2:34626) 2024-09-10T08:07:54.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.954+0000 7f839a800640 1 -- 172.21.15.2:0/500993573 learned_addr learned my addr 172.21.15.2:0/500993573 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:54.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.955+0000 7f839b200640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f839c105920 0x7f839c1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:54.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.955+0000 7f8391e00640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f839c104f60 0x7f839c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:54.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.955+0000 7f839a800640 1 -- 172.21.15.2:0/500993573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f839c104f60 msgr2=0x7f839c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:54.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.955+0000 7f839a800640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f839c104f60 0x7f839c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:54.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.955+0000 7f839a800640 1 -- 172.21.15.2:0/500993573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f839c105920 msgr2=0x7f839c1a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:54.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.955+0000 7f839a800640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f839c105920 0x7f839c1a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:54.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.955+0000 7f839a800640 1 -- 172.21.15.2:0/500993573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c1a1c80 con 0x7f839c103e00 2024-09-10T08:07:54.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.955+0000 7f839a800640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f839c103e00 0x7f839c19a050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f838c00ebc0 tx=0x7f838c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:54.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.955+0000 7f8393e00640 1 -- 172.21.15.2:0/500993573 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c010630 con 0x7f839c103e00 2024-09-10T08:07:54.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.956+0000 7f83a1400640 1 -- 172.21.15.2:0/500993573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c1a1f70 con 0x7f839c103e00 2024-09-10T08:07:54.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.956+0000 7f8393e00640 1 -- 172.21.15.2:0/500993573 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f838c010cb0 con 0x7f839c103e00 2024-09-10T08:07:54.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.956+0000 7f83a1400640 1 -- 172.21.15.2:0/500993573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c1a23d0 con 0x7f839c103e00 2024-09-10T08:07:54.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.956+0000 7f8393e00640 1 -- 172.21.15.2:0/500993573 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c018b70 con 0x7f839c103e00 2024-09-10T08:07:54.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.957+0000 7f83a1400640 1 -- 172.21.15.2:0/500993573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c10b120 con 0x7f839c103e00 2024-09-10T08:07:54.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.959+0000 7f8393e00640 1 -- 172.21.15.2:0/500993573 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f838c018d10 con 0x7f839c103e00 2024-09-10T08:07:54.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.960+0000 7f8393e00640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83800a2c40 0x7f83800a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:54.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.960+0000 7f8393e00640 1 -- 172.21.15.2:0/500993573 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f838c0cfba0 con 0x7f839c103e00 2024-09-10T08:07:54.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.960+0000 7f8391e00640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83800a2c40 0x7f83800a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:54.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.961+0000 7f8391e00640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83800a2c40 0x7f83800a5100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f839c19b570 tx=0x7f8384005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:54.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:54.964+0000 7f8393e00640 1 -- 172.21.15.2:0/500993573 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f838c09bf00 con 0x7f839c103e00 2024-09-10T08:07:55.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.155+0000 7f83a1400640 1 -- 172.21.15.2:0/500993573 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f839c062630 con 0x7f83800a2c40 2024-09-10T08:07:55.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:54 smithi155 ceph-mon[35405]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:55.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.156+0000 7f8393e00640 1 -- 172.21.15.2:0/500993573 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f839c062630 con 0x7f83800a2c40 2024-09-10T08:07:55.157 INFO:teuthology.orchestra.run.smithi002.stdout:HOST ADDR LABELS STATUS 2024-09-10T08:07:55.157 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 172.21.15.2 2024-09-10T08:07:55.157 INFO:teuthology.orchestra.run.smithi002.stdout:smithi107 172.21.15.107 2024-09-10T08:07:55.157 INFO:teuthology.orchestra.run.smithi002.stdout:smithi155 172.21.15.155 2024-09-10T08:07:55.158 INFO:teuthology.orchestra.run.smithi002.stdout:3 hosts in cluster 2024-09-10T08:07:55.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.159+0000 7f83a1400640 1 -- 172.21.15.2:0/500993573 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83800a2c40 msgr2=0x7f83800a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:55.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.159+0000 7f83a1400640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83800a2c40 0x7f83800a5100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f839c19b570 tx=0x7f8384005f90 comp rx=0 tx=0).stop 2024-09-10T08:07:55.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.159+0000 7f83a1400640 1 -- 172.21.15.2:0/500993573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f839c103e00 msgr2=0x7f839c19a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:55.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.159+0000 7f83a1400640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f839c103e00 0x7f839c19a050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f838c00ebc0 tx=0x7f838c00c6a0 comp rx=0 tx=0).stop 2024-09-10T08:07:55.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.159+0000 7f83a1400640 1 -- 172.21.15.2:0/500993573 shutdown_connections 2024-09-10T08:07:55.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.159+0000 7f83a1400640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f839c105920 0x7f839c1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:55.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.159+0000 7f83a1400640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83800a2c40 0x7f83800a5100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:55.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.159+0000 7f83a1400640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f839c104f60 0x7f839c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:55.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.159+0000 7f83a1400640 1 --2- 172.21.15.2:0/500993573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f839c103e00 0x7f839c19a050 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:55.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.159+0000 7f83a1400640 1 -- 172.21.15.2:0/500993573 >> 172.21.15.2:0/500993573 conn(0x7f839c0ff5b0 msgr2=0x7f839c1082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:55.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.159+0000 7f83a1400640 1 -- 172.21.15.2:0/500993573 shutdown_connections 2024-09-10T08:07:55.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:55.159+0000 7f83a1400640 1 -- 172.21.15.2:0/500993573 wait complete. 2024-09-10T08:07:55.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:54 smithi107 ceph-mon[35048]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:55.900 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:55 smithi002 ceph-mon[30563]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:55.939 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T08:07:56.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:55 smithi155 ceph-mon[35405]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:56.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:55 smithi107 ceph-mon[35048]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:56.317 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:56.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.909+0000 7fcda3e00640 1 -- 172.21.15.2:0/1657008613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda4104e50 msgr2=0x7fcda41052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:56.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.909+0000 7fcda3e00640 1 --2- 172.21.15.2:0/1657008613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda4104e50 0x7fcda41052d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009900 tx=0x7fcd8c02a410 comp rx=0 tx=0).stop 2024-09-10T08:07:56.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.909+0000 7fcda3e00640 1 -- 172.21.15.2:0/1657008613 shutdown_connections 2024-09-10T08:07:56.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.909+0000 7fcda3e00640 1 --2- 172.21.15.2:0/1657008613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda4105810 0x7fcda410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:56.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.909+0000 7fcda3e00640 1 --2- 172.21.15.2:0/1657008613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda4104e50 0x7fcda41052d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:56.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.909+0000 7fcda3e00640 1 --2- 172.21.15.2:0/1657008613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcda4103c50 0x7fcda4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:56.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.909+0000 7fcda3e00640 1 -- 172.21.15.2:0/1657008613 >> 172.21.15.2:0/1657008613 conn(0x7fcda40ff3e0 msgr2=0x7fcda4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:56.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.909+0000 7fcda3e00640 1 -- 172.21.15.2:0/1657008613 shutdown_connections 2024-09-10T08:07:56.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.909+0000 7fcda3e00640 1 -- 172.21.15.2:0/1657008613 wait complete. 2024-09-10T08:07:56.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.910+0000 7fcda3e00640 1 Processor -- start 2024-09-10T08:07:56.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.910+0000 7fcda3e00640 1 -- start start 2024-09-10T08:07:56.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.911+0000 7fcda3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcda4103c50 0x7fcda4199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:56.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.911+0000 7fcda3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda4104e50 0x7fcda419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:56.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.911+0000 7fcda3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda4105810 0x7fcda41a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:56.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.911+0000 7fcda3e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcda410f8b0 con 0x7fcda4104e50 2024-09-10T08:07:56.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.911+0000 7fcda3e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcda410f730 con 0x7fcda4105810 2024-09-10T08:07:56.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.911+0000 7fcda3e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcda410fa30 con 0x7fcda4103c50 2024-09-10T08:07:56.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.911+0000 7fcd9be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda4104e50 0x7fcda419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:56.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.911+0000 7fcd9be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda4104e50 0x7fcda419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54132/0 (socket says 172.21.15.2:54132) 2024-09-10T08:07:56.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.911+0000 7fcd9be00640 1 -- 172.21.15.2:0/1324080165 learned_addr learned my addr 172.21.15.2:0/1324080165 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:56.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.911+0000 7fcda2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcda4103c50 0x7fcda4199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:56.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.911+0000 7fcda3400640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda4105810 0x7fcda41a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:56.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.912+0000 7fcda2a00640 1 -- 172.21.15.2:0/1324080165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda4105810 msgr2=0x7fcda41a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:56.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.912+0000 7fcda2a00640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda4105810 0x7fcda41a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:56.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.912+0000 7fcda2a00640 1 -- 172.21.15.2:0/1324080165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda4104e50 msgr2=0x7fcda419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:56.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.912+0000 7fcda2a00640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda4104e50 0x7fcda419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:56.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.912+0000 7fcda2a00640 1 -- 172.21.15.2:0/1324080165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda41a1b30 con 0x7fcda4103c50 2024-09-10T08:07:56.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.912+0000 7fcd9be00640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda4104e50 0x7fcda419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:56.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.912+0000 7fcda2a00640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcda4103c50 0x7fcda4199f00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcda4104cb0 tx=0x7fcd9400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:56.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.912+0000 7fcda8a00640 1 -- 172.21.15.2:0/1324080165 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94017070 con 0x7fcda4103c50 2024-09-10T08:07:56.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.912+0000 7fcda8a00640 1 -- 172.21.15.2:0/1324080165 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd94009c30 con 0x7fcda4103c50 2024-09-10T08:07:56.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.912+0000 7fcda3e00640 1 -- 172.21.15.2:0/1324080165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda41a1e20 con 0x7fcda4103c50 2024-09-10T08:07:56.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.912+0000 7fcda8a00640 1 -- 172.21.15.2:0/1324080165 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9400f620 con 0x7fcda4103c50 2024-09-10T08:07:56.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.912+0000 7fcda3e00640 1 -- 172.21.15.2:0/1324080165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda41a2280 con 0x7fcda4103c50 2024-09-10T08:07:56.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.913+0000 7fcda3e00640 1 -- 172.21.15.2:0/1324080165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda410b050 con 0x7fcda4103c50 2024-09-10T08:07:56.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.915+0000 7fcda8a00640 1 -- 172.21.15.2:0/1324080165 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fcd9401a070 con 0x7fcda4103c50 2024-09-10T08:07:56.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.917+0000 7fcda8a00640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcd880a2cc0 0x7fcd880a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:56.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.917+0000 7fcda8a00640 1 -- 172.21.15.2:0/1324080165 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcd940cef40 con 0x7fcda4103c50 2024-09-10T08:07:56.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.917+0000 7fcd9be00640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcd880a2cc0 0x7fcd880a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:56.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.917+0000 7fcd9be00640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcd880a2cc0 0x7fcd880a5180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c031440 tx=0x7fcd8c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:56.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:56.921+0000 7fcda8a00640 1 -- 172.21.15.2:0/1324080165 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd9409b2a0 con 0x7fcda4103c50 2024-09-10T08:07:57.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.112+0000 7fcda3e00640 1 -- 172.21.15.2:0/1324080165 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fcda4062630 con 0x7fcd880a2cc0 2024-09-10T08:07:57.111 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:56 smithi002 ceph-mon[30563]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:57.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.114+0000 7fcda8a00640 1 -- 172.21.15.2:0/1324080165 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fcda4062630 con 0x7fcd880a2cc0 2024-09-10T08:07:57.113 INFO:teuthology.orchestra.run.smithi002.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T08:07:57.113 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EV480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-09-10T08:07:57.113 INFO:teuthology.orchestra.run.smithi002.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T08:07:57.113 INFO:teuthology.orchestra.run.smithi002.stdout:smithi155 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GW400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T08:07:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.116+0000 7fcda3e00640 1 -- 172.21.15.2:0/1324080165 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcd880a2cc0 msgr2=0x7fcd880a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.116+0000 7fcda3e00640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcd880a2cc0 0x7fcd880a5180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c031440 tx=0x7fcd8c0315b0 comp rx=0 tx=0).stop 2024-09-10T08:07:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.116+0000 7fcda3e00640 1 -- 172.21.15.2:0/1324080165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcda4103c50 msgr2=0x7fcda4199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.116+0000 7fcda3e00640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcda4103c50 0x7fcda4199f00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcda4104cb0 tx=0x7fcd9400dde0 comp rx=0 tx=0).stop 2024-09-10T08:07:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.116+0000 7fcda3e00640 1 -- 172.21.15.2:0/1324080165 shutdown_connections 2024-09-10T08:07:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.116+0000 7fcda3e00640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda4105810 0x7fcda41a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:57.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.116+0000 7fcda3e00640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda4104e50 0x7fcda419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:57.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.116+0000 7fcda3e00640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcd880a2cc0 0x7fcd880a5180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:57.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.116+0000 7fcda3e00640 1 --2- 172.21.15.2:0/1324080165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcda4103c50 0x7fcda4199f00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:57.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.116+0000 7fcda3e00640 1 -- 172.21.15.2:0/1324080165 >> 172.21.15.2:0/1324080165 conn(0x7fcda40ff3e0 msgr2=0x7fcda4108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:57.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.116+0000 7fcda3e00640 1 -- 172.21.15.2:0/1324080165 shutdown_connections 2024-09-10T08:07:57.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:57.117+0000 7fcda3e00640 1 -- 172.21.15.2:0/1324080165 wait complete. 2024-09-10T08:07:57.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:56 smithi155 ceph-mon[35405]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:57.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:56 smithi107 ceph-mon[35048]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:07:57.945 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T08:07:57.955 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-09-10T08:07:57.956 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T08:07:58.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:57 smithi155 ceph-mon[35405]: from='client.25027 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:58.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:57 smithi107 ceph-mon[35048]: from='client.25027 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:58.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:57 smithi002 ceph-mon[30563]: from='client.25027 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:07:58.334 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:07:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.922+0000 7f2bbd800640 1 -- 172.21.15.2:0/3360609806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bb8101c00 msgr2=0x7f2bb8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.922+0000 7f2bbd800640 1 --2- 172.21.15.2:0/3360609806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bb8101c00 0x7f2bb8102000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009900 tx=0x7f2ba802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:07:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.923+0000 7f2bbd800640 1 -- 172.21.15.2:0/3360609806 shutdown_connections 2024-09-10T08:07:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.923+0000 7f2bbd800640 1 --2- 172.21.15.2:0/3360609806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bb81037c0 0x7f2bb810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.923+0000 7f2bbd800640 1 --2- 172.21.15.2:0/3360609806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb8102e00 0x7f2bb8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.923+0000 7f2bbd800640 1 --2- 172.21.15.2:0/3360609806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bb8101c00 0x7f2bb8102000 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.923+0000 7f2bbd800640 1 -- 172.21.15.2:0/3360609806 >> 172.21.15.2:0/3360609806 conn(0x7f2bb80fd390 msgr2=0x7f2bb80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.923+0000 7f2bbd800640 1 -- 172.21.15.2:0/3360609806 shutdown_connections 2024-09-10T08:07:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.923+0000 7f2bbd800640 1 -- 172.21.15.2:0/3360609806 wait complete. 2024-09-10T08:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.924+0000 7f2bbd800640 1 Processor -- start 2024-09-10T08:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.924+0000 7f2bbd800640 1 -- start start 2024-09-10T08:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.924+0000 7f2bbd800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb8101c00 0x7f2bb819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.924+0000 7f2bbd800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bb8102e00 0x7f2bb819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.924+0000 7f2bbd800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bb81037c0 0x7f2bb81a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.924+0000 7f2bbd800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb810d860 con 0x7f2bb81037c0 2024-09-10T08:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.924+0000 7f2bbd800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb810d6e0 con 0x7f2bb8101c00 2024-09-10T08:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.924+0000 7f2bbd800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb810d9e0 con 0x7f2bb8102e00 2024-09-10T08:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bb7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bb81037c0 0x7f2bb81a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bb6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb8101c00 0x7f2bb819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bb6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb8101c00 0x7f2bb819a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:47086/0 (socket says 172.21.15.2:47086) 2024-09-10T08:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bb7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bb81037c0 0x7f2bb81a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54142/0 (socket says 172.21.15.2:54142) 2024-09-10T08:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bb6a00640 1 -- 172.21.15.2:0/2029760438 learned_addr learned my addr 172.21.15.2:0/2029760438 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bafe00640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bb8102e00 0x7f2bb819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bb7400640 1 -- 172.21.15.2:0/2029760438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bb8102e00 msgr2=0x7f2bb819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bb7400640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bb8102e00 0x7f2bb819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bb7400640 1 -- 172.21.15.2:0/2029760438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb8101c00 msgr2=0x7f2bb819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bb7400640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb8101c00 0x7f2bb819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bb7400640 1 -- 172.21.15.2:0/2029760438 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb81a1d30 con 0x7f2bb81037c0 2024-09-10T08:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bafe00640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bb8102e00 0x7f2bb819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:07:58.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bb6a00640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb8101c00 0x7f2bb819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:07:58.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.925+0000 7f2bb7400640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bb81037c0 0x7f2bb81a16c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009fd0 tx=0x7f2bb000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:58.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.926+0000 7f2baf400640 1 -- 172.21.15.2:0/2029760438 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000ce20 con 0x7f2bb81037c0 2024-09-10T08:07:58.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.926+0000 7f2baf400640 1 -- 172.21.15.2:0/2029760438 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb0005020 con 0x7f2bb81037c0 2024-09-10T08:07:58.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.926+0000 7f2baf400640 1 -- 172.21.15.2:0/2029760438 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0010720 con 0x7f2bb81037c0 2024-09-10T08:07:58.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.926+0000 7f2bbd800640 1 -- 172.21.15.2:0/2029760438 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb81a1fc0 con 0x7f2bb81037c0 2024-09-10T08:07:58.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.926+0000 7f2bbd800640 1 -- 172.21.15.2:0/2029760438 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb81a24d0 con 0x7f2bb81037c0 2024-09-10T08:07:58.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.927+0000 7f2bbd800640 1 -- 172.21.15.2:0/2029760438 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb81a2790 con 0x7f2bb81037c0 2024-09-10T08:07:58.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.929+0000 7f2baf400640 1 -- 172.21.15.2:0/2029760438 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2bb0005540 con 0x7f2bb81037c0 2024-09-10T08:07:58.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.930+0000 7f2baf400640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2b9c0a2cc0 0x7f2b9c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:07:58.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.930+0000 7f2baf400640 1 -- 172.21.15.2:0/2029760438 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2bb00cfcf0 con 0x7f2bb81037c0 2024-09-10T08:07:58.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.930+0000 7f2bb6a00640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2b9c0a2cc0 0x7f2b9c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:07:58.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.931+0000 7f2bb6a00640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2b9c0a2cc0 0x7f2b9c0a5180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8000c00 tx=0x7f2ba80046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:58.934+0000 7f2baf400640 1 -- 172.21.15.2:0/2029760438 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2bb009c6a0 con 0x7f2bb81037c0 2024-09-10T08:07:59.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:58 smithi155 ceph-mon[35405]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:59.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.159+0000 7f2bbd800640 1 -- 172.21.15.2:0/2029760438 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2bb81a2a80 con 0x7f2bb81037c0 2024-09-10T08:07:59.158 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:58 smithi002 ceph-mon[30563]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:59.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.159+0000 7f2baf400640 1 -- 172.21.15.2:0/2029760438 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3086 (secure 0 0 0) 0x7f2bb009bfa0 con 0x7f2bb81037c0 2024-09-10T08:07:59.158 INFO:teuthology.orchestra.run.smithi002.stdout:e25 2024-09-10T08:07:59.159 INFO:teuthology.orchestra.run.smithi002.stdout:btime 2024-09-10T08:06:25:369375+0000 2024-09-10T08:07:59.159 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T08:07:59.159 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T08:07:59.159 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-09-10T08:07:59.159 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:07:59.159 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-09-10T08:07:59.159 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-09-10T08:07:59.159 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 25 2024-09-10T08:07:59.159 INFO:teuthology.orchestra.run.smithi002.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T08:07:59.159 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-09-10T08:05:26.574488+0000 2024-09-10T08:07:59.159 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-09-10T08:06:24.883542+0000 2024-09-10T08:07:59.159 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-09-10T08:07:59.160 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-09-10T08:07:59.160 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 300 2024-09-10T08:07:59.160 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-09-10T08:07:59.160 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-09-10T08:07:59.160 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-09-10T08:07:59.160 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-09-10T08:07:59.160 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-09-10T08:07:59.160 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 0 2024-09-10T08:07:59.160 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T08:07:59.160 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 3 2024-09-10T08:07:59.160 INFO:teuthology.orchestra.run.smithi002.stdout:in 0,1,2 2024-09-10T08:07:59.160 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24536,1=24499,2=24494} 2024-09-10T08:07:59.161 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-09-10T08:07:59.161 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-09-10T08:07:59.161 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-09-10T08:07:59.161 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3,4] 2024-09-10T08:07:59.161 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-09-10T08:07:59.161 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data disabled 2024-09-10T08:07:59.161 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-09-10T08:07:59.161 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-09-10T08:07:59.161 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 0 2024-09-10T08:07:59.161 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 24536 members: 24494,24499,24536 2024-09-10T08:07:59.161 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b{0:24536} state up:active seq 5 addr [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:07:59.161 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.c{0:24487} state up:standby-replay seq 1 addr [v2:172.21.15.155:6836/3780222531,v1:172.21.15.155:6837/3780222531] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:07:59.161 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.i{1:24499} state up:active seq 21 addr [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:07:59.162 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.f{1:24475} state up:standby-replay seq 1 addr [v2:172.21.15.155:6834/1068995103,v1:172.21.15.155:6835/1068995103] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:07:59.162 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.h{2:24494} state up:active seq 22 addr [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:07:59.162 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.e{2:24473} state up:standby-replay seq 1 addr [v2:172.21.15.107:6834/1775771400,v1:172.21.15.107:6835/1775771400] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:07:59.162 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:07:59.162 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:07:59.162 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-09-10T08:07:59.162 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:07:59.162 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.j{-1:14454} state up:standby seq 1 addr [v2:172.21.15.2:6834/1979555440,v1:172.21.15.2:6835/1979555440] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:07:59.162 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.d{-1:14466} state up:standby seq 1 addr [v2:172.21.15.2:6836/2248723056,v1:172.21.15.2:6837/2248723056] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:07:59.162 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.g{-1:14502} state up:standby seq 1 addr [v2:172.21.15.2:6838/851586467,v1:172.21.15.2:6839/851586467] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:07:59.162 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a{-1:14514} state up:standby seq 1 addr [v2:172.21.15.2:6840/91146142,v1:172.21.15.2:6841/91146142] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:07:59.162 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.l{-1:24436} state up:standby seq 1 addr [v2:172.21.15.155:6832/1276815741,v1:172.21.15.155:6833/1276815741] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:07:59.163 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.k{-1:24443} state up:standby seq 1 addr [v2:172.21.15.107:6832/3176720601,v1:172.21.15.107:6833/3176720601] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:07:59.163 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 25 2024-09-10T08:07:59.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.162+0000 7f2bbd800640 1 -- 172.21.15.2:0/2029760438 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2b9c0a2cc0 msgr2=0x7f2b9c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:59.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.162+0000 7f2bbd800640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2b9c0a2cc0 0x7f2b9c0a5180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8000c00 tx=0x7f2ba80046c0 comp rx=0 tx=0).stop 2024-09-10T08:07:59.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.162+0000 7f2bbd800640 1 -- 172.21.15.2:0/2029760438 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bb81037c0 msgr2=0x7f2bb81a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:07:59.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.162+0000 7f2bbd800640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bb81037c0 0x7f2bb81a16c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009fd0 tx=0x7f2bb000ef10 comp rx=0 tx=0).stop 2024-09-10T08:07:59.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.163+0000 7f2bbd800640 1 -- 172.21.15.2:0/2029760438 shutdown_connections 2024-09-10T08:07:59.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.163+0000 7f2bbd800640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bb81037c0 0x7f2bb81a16c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:59.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.163+0000 7f2bbd800640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2b9c0a2cc0 0x7f2b9c0a5180 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:59.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.163+0000 7f2bbd800640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bb8102e00 0x7f2bb819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:59.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.163+0000 7f2bbd800640 1 --2- 172.21.15.2:0/2029760438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb8101c00 0x7f2bb819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:07:59.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.163+0000 7f2bbd800640 1 -- 172.21.15.2:0/2029760438 >> 172.21.15.2:0/2029760438 conn(0x7f2bb80fd390 msgr2=0x7f2bb8074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:07:59.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.163+0000 7f2bbd800640 1 -- 172.21.15.2:0/2029760438 shutdown_connections 2024-09-10T08:07:59.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:07:59.163+0000 7f2bbd800640 1 -- 172.21.15.2:0/2029760438 wait complete. 2024-09-10T08:07:59.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:58 smithi107 ceph-mon[35048]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:07:59.883 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:07:59 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2029760438' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T08:07:59.919 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T08:08:00.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:07:59 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2029760438' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T08:08:00.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:07:59 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2029760438' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T08:08:00.302 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:00.885 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:00 smithi002 ceph-mon[30563]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:08:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.914+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3604694248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bf4103c50 msgr2=0x7f6bf4104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.914+0000 7f6bf8a00640 1 --2- 172.21.15.2:0/3604694248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bf4103c50 0x7f6bf4104050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009900 tx=0x7f6be402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:08:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.914+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3604694248 shutdown_connections 2024-09-10T08:08:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.914+0000 7f6bf8a00640 1 --2- 172.21.15.2:0/3604694248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bf4105810 0x7f6bf410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.914+0000 7f6bf8a00640 1 --2- 172.21.15.2:0/3604694248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bf4104e50 0x7f6bf41052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.914+0000 7f6bf8a00640 1 --2- 172.21.15.2:0/3604694248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bf4103c50 0x7f6bf4104050 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.914+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3604694248 >> 172.21.15.2:0/3604694248 conn(0x7f6bf40ff3e0 msgr2=0x7f6bf4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.914+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3604694248 shutdown_connections 2024-09-10T08:08:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.914+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3604694248 wait complete. 2024-09-10T08:08:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.915+0000 7f6bf8a00640 1 Processor -- start 2024-09-10T08:08:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.915+0000 7f6bf8a00640 1 -- start start 2024-09-10T08:08:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.915+0000 7f6bf8a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bf4103c50 0x7f6bf4199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.915+0000 7f6bf8a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bf4104e50 0x7f6bf419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.915+0000 7f6bf8a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bf4105810 0x7f6bf41a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.915+0000 7f6bf8a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf410f930 con 0x7f6bf4104e50 2024-09-10T08:08:00.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.915+0000 7f6bf8a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf410f7b0 con 0x7f6bf4105810 2024-09-10T08:08:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.915+0000 7f6bf8a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf410fab0 con 0x7f6bf4103c50 2024-09-10T08:08:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.916+0000 7f6bf2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bf4104e50 0x7f6bf419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.916+0000 7f6bf3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bf4103c50 0x7f6bf4199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.916+0000 7f6bf3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bf4103c50 0x7f6bf4199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56844/0 (socket says 172.21.15.2:56844) 2024-09-10T08:08:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.916+0000 7f6bf3400640 1 -- 172.21.15.2:0/3403628132 learned_addr learned my addr 172.21.15.2:0/3403628132 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.916+0000 7f6bf3e00640 1 --2- 172.21.15.2:0/3403628132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bf4105810 0x7f6bf41a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.916+0000 7f6bf2a00640 1 -- 172.21.15.2:0/3403628132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bf4103c50 msgr2=0x7f6bf4199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.916+0000 7f6bf2a00640 1 --2- 172.21.15.2:0/3403628132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bf4103c50 0x7f6bf4199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.916+0000 7f6bf2a00640 1 -- 172.21.15.2:0/3403628132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bf4105810 msgr2=0x7f6bf41a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.916+0000 7f6bf2a00640 1 --2- 172.21.15.2:0/3403628132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bf4105810 0x7f6bf41a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.916+0000 7f6bf2a00640 1 -- 172.21.15.2:0/3403628132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf41a1b50 con 0x7f6bf4104e50 2024-09-10T08:08:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.916+0000 7f6bf2a00640 1 --2- 172.21.15.2:0/3403628132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bf4104e50 0x7f6bf419a460 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6be000e9f0 tx=0x7f6be000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:00.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.917+0000 7f6bebe00640 1 -- 172.21.15.2:0/3403628132 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be000ccc0 con 0x7f6bf4104e50 2024-09-10T08:08:00.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.917+0000 7f6bebe00640 1 -- 172.21.15.2:0/3403628132 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be0010910 con 0x7f6bf4104e50 2024-09-10T08:08:00.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.917+0000 7f6bebe00640 1 -- 172.21.15.2:0/3403628132 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0018960 con 0x7f6bf4104e50 2024-09-10T08:08:00.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.917+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3403628132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf41a1e40 con 0x7f6bf4104e50 2024-09-10T08:08:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.917+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3403628132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf41a2340 con 0x7f6bf4104e50 2024-09-10T08:08:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.918+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3403628132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf410af80 con 0x7f6bf4104e50 2024-09-10T08:08:00.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.921+0000 7f6bebe00640 1 -- 172.21.15.2:0/3403628132 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6be0010ac0 con 0x7f6bf4104e50 2024-09-10T08:08:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.922+0000 7f6bebe00640 1 --2- 172.21.15.2:0/3403628132 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6bd80a2cc0 0x7f6bd80a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.922+0000 7f6bebe00640 1 -- 172.21.15.2:0/3403628132 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6be00cf8b0 con 0x7f6bf4104e50 2024-09-10T08:08:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.922+0000 7f6bf3400640 1 --2- 172.21.15.2:0/3403628132 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6bd80a2cc0 0x7f6bd80a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.923+0000 7f6bf3400640 1 --2- 172.21.15.2:0/3403628132 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6bd80a2cc0 0x7f6bd80a5180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6be4000c00 tx=0x7f6be40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:00.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:00.926+0000 7f6bebe00640 1 -- 172.21.15.2:0/3403628132 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6be009bc10 con 0x7f6bf4104e50 2024-09-10T08:08:01.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.114+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3403628132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f6bf402da10 con 0x7f6bf4104e50 2024-09-10T08:08:01.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.116+0000 7f6bebe00640 1 -- 172.21.15.2:0/3403628132 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v95) ==== 56+0+5689 (secure 0 0 0) 0x7f6be009ff10 con 0x7f6bf4104e50 2024-09-10T08:08:01.115 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 95 2024-09-10T08:08:01.115 INFO:teuthology.orchestra.run.smithi002.stdout:fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T08:08:01.115 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-09-10T07:57:11.261625+0000 2024-09-10T08:08:01.115 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-09-10T08:07:05.504273+0000 2024-09-10T08:08:01.115 INFO:teuthology.orchestra.run.smithi002.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T08:08:01.115 INFO:teuthology.orchestra.run.smithi002.stdout:crush_version 27 2024-09-10T08:08:01.115 INFO:teuthology.orchestra.run.smithi002.stdout:full_ratio 0.95 2024-09-10T08:08:01.115 INFO:teuthology.orchestra.run.smithi002.stdout:backfillfull_ratio 0.9 2024-09-10T08:08:01.115 INFO:teuthology.orchestra.run.smithi002.stdout:nearfull_ratio 0.85 2024-09-10T08:08:01.115 INFO:teuthology.orchestra.run.smithi002.stdout:require_min_compat_client luminous 2024-09-10T08:08:01.116 INFO:teuthology.orchestra.run.smithi002.stdout:min_compat_client luminous 2024-09-10T08:08:01.116 INFO:teuthology.orchestra.run.smithi002.stdout:require_osd_release squid 2024-09-10T08:08:01.116 INFO:teuthology.orchestra.run.smithi002.stdout:stretch_mode_enabled false 2024-09-10T08:08:01.116 INFO:teuthology.orchestra.run.smithi002.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T08:08:01.116 INFO:teuthology.orchestra.run.smithi002.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-10T08:08:01.116 INFO:teuthology.orchestra.run.smithi002.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-10T08:08:01.116 INFO:teuthology.orchestra.run.smithi002.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 91 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-10T08:08:01.116 INFO:teuthology.orchestra.run.smithi002.stdout:max_osd 12 2024-09-10T08:08:01.116 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 up in weight 1 up_from 8 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6802/3825226139,v1:172.21.15.2:6803/3825226139] [v2:172.21.15.2:6804/3825226139,v1:172.21.15.2:6805/3825226139] exists,up 41eb8073-0ba0-4e03-a60a-e7b27476ba0d 2024-09-10T08:08:01.116 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6800/3710092441,v1:172.21.15.107:6801/3710092441] [v2:172.21.15.107:6802/3710092441,v1:172.21.15.107:6803/3710092441] exists,up c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6 2024-09-10T08:08:01.116 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 up in weight 1 up_from 18 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6800/158516707,v1:172.21.15.155:6801/158516707] [v2:172.21.15.155:6802/158516707,v1:172.21.15.155:6803/158516707] exists,up f1c5e964-c621-4441-a87f-17c96c05de10 2024-09-10T08:08:01.116 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6810/713998144,v1:172.21.15.2:6811/713998144] [v2:172.21.15.2:6812/713998144,v1:172.21.15.2:6813/713998144] exists,up 400fc872-8c48-40e3-8ecb-b34778d142a9 2024-09-10T08:08:01.116 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 up in weight 1 up_from 32 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6808/485793971,v1:172.21.15.107:6809/485793971] [v2:172.21.15.107:6810/485793971,v1:172.21.15.107:6811/485793971] exists,up 269f71aa-0f17-4260-ad01-452fdd1b997f 2024-09-10T08:08:01.117 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 up in weight 1 up_from 37 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6808/2180279362,v1:172.21.15.155:6809/2180279362] [v2:172.21.15.155:6810/2180279362,v1:172.21.15.155:6811/2180279362] exists,up c7f0af26-b1a3-428b-a75f-f54956e40bf0 2024-09-10T08:08:01.117 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 up in weight 1 up_from 45 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6818/854683508,v1:172.21.15.2:6819/854683508] [v2:172.21.15.2:6820/854683508,v1:172.21.15.2:6821/854683508] exists,up b0dc96e4-e62c-44ce-9902-7c168af07b9e 2024-09-10T08:08:01.117 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 up in weight 1 up_from 50 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6816/1479477994,v1:172.21.15.107:6817/1479477994] [v2:172.21.15.107:6818/1479477994,v1:172.21.15.107:6819/1479477994] exists,up 8feffc44-c4f3-4e5d-a59f-8ab00b949179 2024-09-10T08:08:01.117 INFO:teuthology.orchestra.run.smithi002.stdout:osd.8 up in weight 1 up_from 57 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6816/359284864,v1:172.21.15.155:6817/359284864] [v2:172.21.15.155:6818/359284864,v1:172.21.15.155:6819/359284864] exists,up 4f9de660-f542-428d-bbdb-a932abc8d4cb 2024-09-10T08:08:01.117 INFO:teuthology.orchestra.run.smithi002.stdout:osd.9 up in weight 1 up_from 64 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6826/758080099,v1:172.21.15.2:6827/758080099] [v2:172.21.15.2:6828/758080099,v1:172.21.15.2:6829/758080099] exists,up 07f3cee5-642d-4887-86ed-46293be9fe24 2024-09-10T08:08:01.117 INFO:teuthology.orchestra.run.smithi002.stdout:osd.10 up in weight 1 up_from 69 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6824/3646489493,v1:172.21.15.107:6825/3646489493] [v2:172.21.15.107:6826/3646489493,v1:172.21.15.107:6827/3646489493] exists,up 2fbd17c0-c632-4ab1-aea8-b6db91498bf7 2024-09-10T08:08:01.117 INFO:teuthology.orchestra.run.smithi002.stdout:osd.11 up in weight 1 up_from 76 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6824/4052924241,v1:172.21.15.155:6825/4052924241] [v2:172.21.15.155:6826/4052924241,v1:172.21.15.155:6827/4052924241] exists,up e4caf962-e25f-4f3f-b3a7-a4495561ce6f 2024-09-10T08:08:01.117 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.6 [8,11] 2024-09-10T08:08:01.117 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.b [2,1] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.f [8,11] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.14 [2,4] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.6 [4,6] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.7 [3,9] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.10 [5,9] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.17 [4,8] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.21 [4,7] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.24 [4,9] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.39 [5,8] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.8 [5,8] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.c [9,10] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.19 [9,8] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.1b [9,8] 2024-09-10T08:08:01.118 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6801/3590511604 expires 2024-09-11T07:58:03.548782+0000 2024-09-10T08:08:01.119 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/939674007 expires 2024-09-11T07:58:03.548782+0000 2024-09-10T08:08:01.119 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/2447931330 expires 2024-09-11T07:58:03.548782+0000 2024-09-10T08:08:01.119 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/387296843 expires 2024-09-11T07:58:03.548782+0000 2024-09-10T08:08:01.119 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/1896108724 expires 2024-09-11T07:57:36.359026+0000 2024-09-10T08:08:01.119 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/2121891210 expires 2024-09-11T07:57:36.359026+0000 2024-09-10T08:08:01.119 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6801/712393349 expires 2024-09-11T07:57:36.359026+0000 2024-09-10T08:08:01.119 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6800/712393349 expires 2024-09-11T07:57:36.359026+0000 2024-09-10T08:08:01.119 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6800/3590511604 expires 2024-09-11T07:58:03.548782+0000 2024-09-10T08:08:01.119 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/1616837180 expires 2024-09-11T07:57:36.359026+0000 2024-09-10T08:08:01.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.118+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3403628132 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6bd80a2cc0 msgr2=0x7f6bd80a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:01.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.118+0000 7f6bf8a00640 1 --2- 172.21.15.2:0/3403628132 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6bd80a2cc0 0x7f6bd80a5180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6be4000c00 tx=0x7f6be40315b0 comp rx=0 tx=0).stop 2024-09-10T08:08:01.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.118+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3403628132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bf4104e50 msgr2=0x7f6bf419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:01.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.118+0000 7f6bf8a00640 1 --2- 172.21.15.2:0/3403628132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bf4104e50 0x7f6bf419a460 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6be000e9f0 tx=0x7f6be000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:08:01.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.118+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3403628132 shutdown_connections 2024-09-10T08:08:01.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.118+0000 7f6bf8a00640 1 --2- 172.21.15.2:0/3403628132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bf4105810 0x7f6bf41a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:01.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.118+0000 7f6bf8a00640 1 --2- 172.21.15.2:0/3403628132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bf4104e50 0x7f6bf419a460 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:01.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.118+0000 7f6bf8a00640 1 --2- 172.21.15.2:0/3403628132 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6bd80a2cc0 0x7f6bd80a5180 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:01.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.118+0000 7f6bf8a00640 1 --2- 172.21.15.2:0/3403628132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bf4103c50 0x7f6bf4199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:01.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.118+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3403628132 >> 172.21.15.2:0/3403628132 conn(0x7f6bf40ff3e0 msgr2=0x7f6bf4108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:01.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.118+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3403628132 shutdown_connections 2024-09-10T08:08:01.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:01.118+0000 7f6bf8a00640 1 -- 172.21.15.2:0/3403628132 wait complete. 2024-09-10T08:08:01.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:00 smithi155 ceph-mon[35405]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:08:01.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:00 smithi107 ceph-mon[35048]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:08:02.046 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T08:08:02.061 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:08:02.111 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:01 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3403628132' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T08:08:02.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:01 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3403628132' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T08:08:02.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:01 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3403628132' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T08:08:02.438 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:02.878 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:02 smithi002 ceph-mon[30563]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:08:02.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.991+0000 7fc3f9400640 1 -- 172.21.15.2:0/1618415406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f4103ff0 msgr2=0x7fc3f41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:02.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.991+0000 7fc3f9400640 1 --2- 172.21.15.2:0/1618415406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f4103ff0 0x7fc3f41043d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009900 tx=0x7fc3e402a410 comp rx=0 tx=0).stop 2024-09-10T08:08:02.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.991+0000 7fc3f9400640 1 -- 172.21.15.2:0/1618415406 shutdown_connections 2024-09-10T08:08:02.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.991+0000 7fc3f9400640 1 --2- 172.21.15.2:0/1618415406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f4109120 0x7fc3f4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:02.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.991+0000 7fc3f9400640 1 --2- 172.21.15.2:0/1618415406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f41049a0 0x7fc3f41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:02.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.991+0000 7fc3f9400640 1 --2- 172.21.15.2:0/1618415406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f4103ff0 0x7fc3f41043d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:02.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.991+0000 7fc3f9400640 1 -- 172.21.15.2:0/1618415406 >> 172.21.15.2:0/1618415406 conn(0x7fc3f40ffa90 msgr2=0x7fc3f4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:02.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.991+0000 7fc3f9400640 1 -- 172.21.15.2:0/1618415406 shutdown_connections 2024-09-10T08:08:02.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.991+0000 7fc3f9400640 1 -- 172.21.15.2:0/1618415406 wait complete. 2024-09-10T08:08:02.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.992+0000 7fc3f9400640 1 Processor -- start 2024-09-10T08:08:02.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.992+0000 7fc3f9400640 1 -- start start 2024-09-10T08:08:02.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.992+0000 7fc3f9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f4103ff0 0x7fc3f419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:02.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.992+0000 7fc3f9400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f41049a0 0x7fc3f419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:02.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.992+0000 7fc3f9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f4109120 0x7fc3f419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:02.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.992+0000 7fc3f9400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f4114190 con 0x7fc3f41049a0 2024-09-10T08:08:02.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.992+0000 7fc3f9400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f4114010 con 0x7fc3f4109120 2024-09-10T08:08:02.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.992+0000 7fc3f9400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f4114310 con 0x7fc3f4103ff0 2024-09-10T08:08:02.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.993+0000 7fc3f2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f4103ff0 0x7fc3f419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:02.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.993+0000 7fc3f3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f4109120 0x7fc3f419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.993+0000 7fc3f2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f4103ff0 0x7fc3f419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56852/0 (socket says 172.21.15.2:56852) 2024-09-10T08:08:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.993+0000 7fc3f2800640 1 -- 172.21.15.2:0/2162082805 learned_addr learned my addr 172.21.15.2:0/2162082805 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.993+0000 7fc3e9e00640 1 --2- 172.21.15.2:0/2162082805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f41049a0 0x7fc3f419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.993+0000 7fc3f2800640 1 -- 172.21.15.2:0/2162082805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f4109120 msgr2=0x7fc3f419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.993+0000 7fc3f2800640 1 --2- 172.21.15.2:0/2162082805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f4109120 0x7fc3f419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.993+0000 7fc3f2800640 1 -- 172.21.15.2:0/2162082805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f41049a0 msgr2=0x7fc3f419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.993+0000 7fc3f2800640 1 --2- 172.21.15.2:0/2162082805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f41049a0 0x7fc3f419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.993+0000 7fc3f2800640 1 -- 172.21.15.2:0/2162082805 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f419f640 con 0x7fc3f4103ff0 2024-09-10T08:08:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.994+0000 7fc3f2800640 1 --2- 172.21.15.2:0/2162082805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f4103ff0 0x7fc3f419a690 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4004770 tx=0x7fc3e4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.994+0000 7fc3ebe00640 1 -- 172.21.15.2:0/2162082805 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e403f070 con 0x7fc3f4103ff0 2024-09-10T08:08:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.994+0000 7fc3ebe00640 1 -- 172.21.15.2:0/2162082805 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e4031ca0 con 0x7fc3f4103ff0 2024-09-10T08:08:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.994+0000 7fc3f9400640 1 -- 172.21.15.2:0/2162082805 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f419f8d0 con 0x7fc3f4103ff0 2024-09-10T08:08:02.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.994+0000 7fc3ebe00640 1 -- 172.21.15.2:0/2162082805 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4035030 con 0x7fc3f4103ff0 2024-09-10T08:08:02.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.994+0000 7fc3f9400640 1 -- 172.21.15.2:0/2162082805 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f41a7210 con 0x7fc3f4103ff0 2024-09-10T08:08:02.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.995+0000 7fc3f9400640 1 -- 172.21.15.2:0/2162082805 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f41054d0 con 0x7fc3f4103ff0 2024-09-10T08:08:02.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.997+0000 7fc3ebe00640 1 -- 172.21.15.2:0/2162082805 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc3e4042070 con 0x7fc3f4103ff0 2024-09-10T08:08:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.998+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/2162082805 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3d80a2c10 0x7fc3d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.998+0000 7fc3ebe00640 1 -- 172.21.15.2:0/2162082805 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc3e40ef950 con 0x7fc3f4103ff0 2024-09-10T08:08:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.999+0000 7fc3e9e00640 1 --2- 172.21.15.2:0/2162082805 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3d80a2c10 0x7fc3d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:02.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:02.999+0000 7fc3e9e00640 1 --2- 172.21.15.2:0/2162082805 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3d80a2c10 0x7fc3d80a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40fffb0 tx=0x7fc3dc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:03.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.003+0000 7fc3ebe00640 1 -- 172.21.15.2:0/2162082805 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3e40bbcb0 con 0x7fc3f4103ff0 2024-09-10T08:08:03.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:02 smithi155 ceph-mon[35405]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:08:03.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:02 smithi107 ceph-mon[35048]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:08:03.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.225+0000 7fc3f9400640 1 -- 172.21.15.2:0/2162082805 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3f410c300 con 0x7fc3f4103ff0 2024-09-10T08:08:03.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.225+0000 7fc3ebe00640 1 -- 172.21.15.2:0/2162082805 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7fc3e40bffb0 con 0x7fc3f4103ff0 2024-09-10T08:08:03.226 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:03.226 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":25,"btime":"2024-09-10T08:06:25:369375+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:06:24.883542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24494,24499,24536]},"id":1}]} 2024-09-10T08:08:03.226 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 25 2024-09-10T08:08:03.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.228+0000 7fc3f9400640 1 -- 172.21.15.2:0/2162082805 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3d80a2c10 msgr2=0x7fc3d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:03.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.228+0000 7fc3f9400640 1 --2- 172.21.15.2:0/2162082805 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3d80a2c10 0x7fc3d80a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40fffb0 tx=0x7fc3dc00b390 comp rx=0 tx=0).stop 2024-09-10T08:08:03.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.228+0000 7fc3f9400640 1 -- 172.21.15.2:0/2162082805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f4103ff0 msgr2=0x7fc3f419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:03.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.228+0000 7fc3f9400640 1 --2- 172.21.15.2:0/2162082805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f4103ff0 0x7fc3f419a690 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4004770 tx=0x7fc3e4003c10 comp rx=0 tx=0).stop 2024-09-10T08:08:03.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.228+0000 7fc3f9400640 1 -- 172.21.15.2:0/2162082805 shutdown_connections 2024-09-10T08:08:03.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.228+0000 7fc3f9400640 1 --2- 172.21.15.2:0/2162082805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f4109120 0x7fc3f419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:03.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.228+0000 7fc3f9400640 1 --2- 172.21.15.2:0/2162082805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3f41049a0 0x7fc3f419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:03.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.228+0000 7fc3f9400640 1 --2- 172.21.15.2:0/2162082805 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3d80a2c10 0x7fc3d80a50d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:03.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.228+0000 7fc3f9400640 1 --2- 172.21.15.2:0/2162082805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f4103ff0 0x7fc3f419a690 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:03.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.228+0000 7fc3f9400640 1 -- 172.21.15.2:0/2162082805 >> 172.21.15.2:0/2162082805 conn(0x7fc3f40ffa90 msgr2=0x7fc3f4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:03.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.229+0000 7fc3f9400640 1 -- 172.21.15.2:0/2162082805 shutdown_connections 2024-09-10T08:08:03.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:03.229+0000 7fc3f9400640 1 -- 172.21.15.2:0/2162082805 wait complete. 2024-09-10T08:08:03.952 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:03 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2162082805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:04.022 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:08:04.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:03 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2162082805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:04.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:03 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2162082805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:04.417 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:05.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.041+0000 7f5979800640 1 -- 172.21.15.2:0/854810424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5974105f50 msgr2=0x7f59741063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:05.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.041+0000 7f5979800640 1 --2- 172.21.15.2:0/854810424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5974105f50 0x7f59741063e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f596c009ca0 tx=0x7f596c02a940 comp rx=0 tx=0).stop 2024-09-10T08:08:05.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.042+0000 7f5979800640 1 -- 172.21.15.2:0/854810424 shutdown_connections 2024-09-10T08:08:05.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.042+0000 7f5979800640 1 --2- 172.21.15.2:0/854810424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5974105f50 0x7f59741063e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:05.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.042+0000 7f5979800640 1 --2- 172.21.15.2:0/854810424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5974101460 0x7f5974105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:05.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.042+0000 7f5979800640 1 --2- 172.21.15.2:0/854810424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5974100b40 0x7f5974100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.042+0000 7f5979800640 1 -- 172.21.15.2:0/854810424 >> 172.21.15.2:0/854810424 conn(0x7f59740fc7d0 msgr2=0x7f59740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.042+0000 7f5979800640 1 -- 172.21.15.2:0/854810424 shutdown_connections 2024-09-10T08:08:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.042+0000 7f5979800640 1 -- 172.21.15.2:0/854810424 wait complete. 2024-09-10T08:08:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.043+0000 7f5979800640 1 Processor -- start 2024-09-10T08:08:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.043+0000 7f5979800640 1 -- start start 2024-09-10T08:08:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5979800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5974100b40 0x7f59741a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5979800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5974101460 0x7f59741a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5979800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5974105f50 0x7f597419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5979800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f59741185e0 con 0x7f5974105f50 2024-09-10T08:08:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5979800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5974118460 con 0x7f5974100b40 2024-09-10T08:08:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5979800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5974118760 con 0x7f5974101460 2024-09-10T08:08:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5972a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5974100b40 0x7f59741a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5972a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5974100b40 0x7f59741a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:47142/0 (socket says 172.21.15.2:47142) 2024-09-10T08:08:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5972a00640 1 -- 172.21.15.2:0/2810113417 learned_addr learned my addr 172.21.15.2:0/2810113417 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5972000640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5974101460 0x7f59741a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5973400640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5974105f50 0x7f597419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5972a00640 1 -- 172.21.15.2:0/2810113417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5974101460 msgr2=0x7f59741a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5972a00640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5974101460 0x7f59741a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5972a00640 1 -- 172.21.15.2:0/2810113417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5974105f50 msgr2=0x7f597419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5972a00640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5974105f50 0x7f597419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:05.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5972a00640 1 -- 172.21.15.2:0/2810113417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597419eff0 con 0x7f5974100b40 2024-09-10T08:08:05.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.044+0000 7f5972000640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5974101460 0x7f59741a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:05.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.045+0000 7f5972a00640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5974100b40 0x7f59741a4700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f596400ea60 tx=0x7f596400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:05.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.045+0000 7f596b400640 1 -- 172.21.15.2:0/2810113417 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596400cde0 con 0x7f5974100b40 2024-09-10T08:08:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.045+0000 7f596b400640 1 -- 172.21.15.2:0/2810113417 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59640035f0 con 0x7f5974100b40 2024-09-10T08:08:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.045+0000 7f596b400640 1 -- 172.21.15.2:0/2810113417 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964010670 con 0x7f5974100b40 2024-09-10T08:08:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.045+0000 7f5979800640 1 -- 172.21.15.2:0/2810113417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597419f2e0 con 0x7f5974100b40 2024-09-10T08:08:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.045+0000 7f5979800640 1 -- 172.21.15.2:0/2810113417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59741ab5d0 con 0x7f5974100b40 2024-09-10T08:08:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.046+0000 7f5979800640 1 -- 172.21.15.2:0/2810113417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5974105930 con 0x7f5974100b40 2024-09-10T08:08:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.048+0000 7f596b400640 1 -- 172.21.15.2:0/2810113417 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f59640037a0 con 0x7f5974100b40 2024-09-10T08:08:05.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.049+0000 7f596b400640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59580a2c00 0x7f59580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:05.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.049+0000 7f596b400640 1 -- 172.21.15.2:0/2810113417 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f59640cf700 con 0x7f5974100b40 2024-09-10T08:08:05.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.050+0000 7f5972000640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59580a2c00 0x7f59580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:05.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.050+0000 7f5972000640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59580a2c00 0x7f59580a50c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f59741065c0 tx=0x7f5960005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:05.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.053+0000 7f596b400640 1 -- 172.21.15.2:0/2810113417 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f596409c210 con 0x7f5974100b40 2024-09-10T08:08:05.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:04 smithi155 ceph-mon[35405]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:08:05.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:04 smithi155 ceph-mon[35405]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-10T08:08:05.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:04 smithi107 ceph-mon[35048]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:08:05.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:04 smithi107 ceph-mon[35048]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-10T08:08:05.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:04 smithi002 ceph-mon[30563]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:08:05.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:04 smithi002 ceph-mon[30563]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-10T08:08:05.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.274+0000 7f5979800640 1 -- 172.21.15.2:0/2810113417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f597419fdf0 con 0x7f5974100b40 2024-09-10T08:08:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.275+0000 7f596b400640 1 -- 172.21.15.2:0/2810113417 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7f596409bb10 con 0x7f5974100b40 2024-09-10T08:08:05.277 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:05.277 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":25,"btime":"2024-09-10T08:06:25:369375+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:06:24.883542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24494,24499,24536]},"id":1}]} 2024-09-10T08:08:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 25 2024-09-10T08:08:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.278+0000 7f5979800640 1 -- 172.21.15.2:0/2810113417 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59580a2c00 msgr2=0x7f59580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.278+0000 7f5979800640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59580a2c00 0x7f59580a50c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f59741065c0 tx=0x7f5960005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.278+0000 7f5979800640 1 -- 172.21.15.2:0/2810113417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5974100b40 msgr2=0x7f59741a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.278+0000 7f5979800640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5974100b40 0x7f59741a4700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f596400ea60 tx=0x7f596400ef20 comp rx=0 tx=0).stop 2024-09-10T08:08:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.278+0000 7f5979800640 1 -- 172.21.15.2:0/2810113417 shutdown_connections 2024-09-10T08:08:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.278+0000 7f5979800640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5974105f50 0x7f597419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.278+0000 7f5979800640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59580a2c00 0x7f59580a50c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.278+0000 7f5979800640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5974101460 0x7f59741a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.278+0000 7f5979800640 1 --2- 172.21.15.2:0/2810113417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5974100b40 0x7f59741a4700 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.278+0000 7f5979800640 1 -- 172.21.15.2:0/2810113417 >> 172.21.15.2:0/2810113417 conn(0x7f59740fc7d0 msgr2=0x7f59741038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.278+0000 7f5979800640 1 -- 172.21.15.2:0/2810113417 shutdown_connections 2024-09-10T08:08:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:05.278+0000 7f5979800640 1 -- 172.21.15.2:0/2810113417 wait complete. 2024-09-10T08:08:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:05 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2810113417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:06.154 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:08:06.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:05 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2810113417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:06.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:05 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2810113417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:06.536 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:06.970 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:06 smithi002 ceph-mon[30563]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:08:07.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.083+0000 7f3bb5000640 1 -- 172.21.15.2:0/2585945053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb0074590 msgr2=0x7f3bb0074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:07.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.083+0000 7f3bb5000640 1 --2- 172.21.15.2:0/2585945053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb0074590 0x7f3bb0074970 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009ca0 tx=0x7f3ba002a950 comp rx=0 tx=0).stop 2024-09-10T08:08:07.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.083+0000 7f3bb5000640 1 -- 172.21.15.2:0/2585945053 shutdown_connections 2024-09-10T08:08:07.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.083+0000 7f3bb5000640 1 --2- 172.21.15.2:0/2585945053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bb01129e0 0x7f3bb0114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:07.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.083+0000 7f3bb5000640 1 --2- 172.21.15.2:0/2585945053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bb0074eb0 0x7f3bb01124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:07.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.083+0000 7f3bb5000640 1 --2- 172.21.15.2:0/2585945053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb0074590 0x7f3bb0074970 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:07.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.083+0000 7f3bb5000640 1 -- 172.21.15.2:0/2585945053 >> 172.21.15.2:0/2585945053 conn(0x7f3bb00fd710 msgr2=0x7f3bb00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:07.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.083+0000 7f3bb5000640 1 -- 172.21.15.2:0/2585945053 shutdown_connections 2024-09-10T08:08:07.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.083+0000 7f3bb5000640 1 -- 172.21.15.2:0/2585945053 wait complete. 2024-09-10T08:08:07.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.084+0000 7f3bb5000640 1 Processor -- start 2024-09-10T08:08:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.084+0000 7f3bb5000640 1 -- start start 2024-09-10T08:08:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.085+0000 7f3bb5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb0074590 0x7f3bb01a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.085+0000 7f3bb5000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bb0074eb0 0x7f3bb01a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.085+0000 7f3bb5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bb01129e0 0x7f3bb019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.085+0000 7f3bb5000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01183e0 con 0x7f3bb0074590 2024-09-10T08:08:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.085+0000 7f3bb5000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0118260 con 0x7f3bb01129e0 2024-09-10T08:08:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.085+0000 7f3bb5000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0118560 con 0x7f3bb0074eb0 2024-09-10T08:08:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.085+0000 7f3baf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb0074590 0x7f3bb01a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.085+0000 7f3baf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb0074590 0x7f3bb01a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40936/0 (socket says 172.21.15.2:40936) 2024-09-10T08:08:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.085+0000 7f3baf400640 1 -- 172.21.15.2:0/847534602 learned_addr learned my addr 172.21.15.2:0/847534602 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.085+0000 7f3ba6a00640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bb0074eb0 0x7f3bb01a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.085+0000 7f3bafe00640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bb01129e0 0x7f3bb019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.085+0000 7f3baf400640 1 -- 172.21.15.2:0/847534602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bb0074eb0 msgr2=0x7f3bb01a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.085+0000 7f3baf400640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bb0074eb0 0x7f3bb01a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.086+0000 7f3baf400640 1 -- 172.21.15.2:0/847534602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bb01129e0 msgr2=0x7f3bb019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.086+0000 7f3baf400640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bb01129e0 0x7f3bb019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.086+0000 7f3baf400640 1 -- 172.21.15.2:0/847534602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb019efc0 con 0x7f3bb0074590 2024-09-10T08:08:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.086+0000 7f3baf400640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb0074590 0x7f3bb01a4680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0005cc0 tx=0x7f3ba002f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.086+0000 7f3bacc00640 1 -- 172.21.15.2:0/847534602 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0034040 con 0x7f3bb0074590 2024-09-10T08:08:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.086+0000 7f3bacc00640 1 -- 172.21.15.2:0/847534602 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba0030920 con 0x7f3bb0074590 2024-09-10T08:08:07.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.086+0000 7f3bb5000640 1 -- 172.21.15.2:0/847534602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb019f250 con 0x7f3bb0074590 2024-09-10T08:08:07.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.086+0000 7f3bacc00640 1 -- 172.21.15.2:0/847534602 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00046e0 con 0x7f3bb0074590 2024-09-10T08:08:07.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.086+0000 7f3bb5000640 1 -- 172.21.15.2:0/847534602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb01ab430 con 0x7f3bb0074590 2024-09-10T08:08:07.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.088+0000 7f3bb5000640 1 -- 172.21.15.2:0/847534602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb0075cd0 con 0x7f3bb0074590 2024-09-10T08:08:07.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.090+0000 7f3bacc00640 1 -- 172.21.15.2:0/847534602 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3ba0037020 con 0x7f3bb0074590 2024-09-10T08:08:07.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.091+0000 7f3bacc00640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3b940a2b80 0x7f3b940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:07.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.091+0000 7f3bacc00640 1 -- 172.21.15.2:0/847534602 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3ba00f02b0 con 0x7f3bb0074590 2024-09-10T08:08:07.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.091+0000 7f3ba6a00640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3b940a2b80 0x7f3b940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:07.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.092+0000 7f3ba6a00640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3b940a2b80 0x7f3b940a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3bb019ffa0 tx=0x7f3b98005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:07.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.095+0000 7f3bacc00640 1 -- 172.21.15.2:0/847534602 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ba00bcdc0 con 0x7f3bb0074590 2024-09-10T08:08:07.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:06 smithi155 ceph-mon[35405]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:08:07.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:06 smithi107 ceph-mon[35048]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:08:07.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.285+0000 7f3bb5000640 1 -- 172.21.15.2:0/847534602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bb0104e20 con 0x7f3bb0074590 2024-09-10T08:08:07.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.286+0000 7f3bacc00640 1 -- 172.21.15.2:0/847534602 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24887 (secure 0 0 0) 0x7f3ba00bc6c0 con 0x7f3bb0074590 2024-09-10T08:08:07.287 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:07.287 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":95,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:07:05.504273+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:08:07.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.289+0000 7f3bb5000640 1 -- 172.21.15.2:0/847534602 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3b940a2b80 msgr2=0x7f3b940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:07.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.289+0000 7f3bb5000640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3b940a2b80 0x7f3b940a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3bb019ffa0 tx=0x7f3b98005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:07.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.289+0000 7f3bb5000640 1 -- 172.21.15.2:0/847534602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb0074590 msgr2=0x7f3bb01a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:07.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.289+0000 7f3bb5000640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb0074590 0x7f3bb01a4680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0005cc0 tx=0x7f3ba002f420 comp rx=0 tx=0).stop 2024-09-10T08:08:07.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.289+0000 7f3bb5000640 1 -- 172.21.15.2:0/847534602 shutdown_connections 2024-09-10T08:08:07.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.289+0000 7f3bb5000640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bb01129e0 0x7f3bb019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:07.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.289+0000 7f3bb5000640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3b940a2b80 0x7f3b940a5040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:07.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.289+0000 7f3bb5000640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bb0074eb0 0x7f3bb01a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:07.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.289+0000 7f3bb5000640 1 --2- 172.21.15.2:0/847534602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb0074590 0x7f3bb01a4680 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:07.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.289+0000 7f3bb5000640 1 -- 172.21.15.2:0/847534602 >> 172.21.15.2:0/847534602 conn(0x7f3bb00fd710 msgr2=0x7f3bb00ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:07.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.289+0000 7f3bb5000640 1 -- 172.21.15.2:0/847534602 shutdown_connections 2024-09-10T08:08:07.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:07.289+0000 7f3bb5000640 1 -- 172.21.15.2:0/847534602 wait complete. 2024-09-10T08:08:08.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:07 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/847534602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:08:08.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:07 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/847534602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:08:08.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:07 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/847534602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:08:08.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T08:05:26.574488+0000', 'modified': '2024-09-10T08:06:24.883542+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24536, 'mds_1': 24499, 'mds_2': 24494}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6835/1775771400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 1775771400}, {'type': 'v1', 'addr': '172.21.15.107:6835', 'nonce': 1775771400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6835/1068995103', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 1068995103}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 1068995103}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/3780222531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 3780222531}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 3780222531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.107:6837/3130379099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6836', 'nonce': 3130379099}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 3130379099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.155:6839/2871792117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 2871792117}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 2871792117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.107:6839/3674188134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 3674188134}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 3674188134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24536, 'qdb_cluster': [24494, 24499, 24536]} 2024-09-10T08:08:08.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T08:08:08.198 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T08:08:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:09.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:08 smithi002 ceph-mon[30563]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:08:09.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.151+0000 7fa4a8400640 1 -- 172.21.15.2:0/9107850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4a0106f10 msgr2=0x7fa4a010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:09.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.151+0000 7fa4a8400640 1 --2- 172.21.15.2:0/9107850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4a0106f10 0x7fa4a010ea90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa498009ca0 tx=0x7fa49802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.151+0000 7fa4a8400640 1 -- 172.21.15.2:0/9107850 shutdown_connections 2024-09-10T08:08:09.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.151+0000 7fa4a8400640 1 --2- 172.21.15.2:0/9107850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4a0106f10 0x7fa4a010ea90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.151+0000 7fa4a8400640 1 --2- 172.21.15.2:0/9107850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a0102790 0x7fa4a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.151+0000 7fa4a8400640 1 --2- 172.21.15.2:0/9107850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a0101de0 0x7fa4a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.151+0000 7fa4a8400640 1 -- 172.21.15.2:0/9107850 >> 172.21.15.2:0/9107850 conn(0x7fa4a00fd8e0 msgr2=0x7fa4a00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:09.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.151+0000 7fa4a8400640 1 -- 172.21.15.2:0/9107850 shutdown_connections 2024-09-10T08:08:09.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.151+0000 7fa4a8400640 1 -- 172.21.15.2:0/9107850 wait complete. 2024-09-10T08:08:09.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.152+0000 7fa4a8400640 1 Processor -- start 2024-09-10T08:08:09.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.152+0000 7fa4a8400640 1 -- start start 2024-09-10T08:08:09.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.152+0000 7fa4a8400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a0101de0 0x7fa4a019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.152+0000 7fa4a8400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4a0102790 0x7fa4a019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a8400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a0106f10 0x7fa4a019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a8400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a0111f70 con 0x7fa4a0102790 2024-09-10T08:08:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a8400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a0111df0 con 0x7fa4a0101de0 2024-09-10T08:08:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a8400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01120f0 con 0x7fa4a0106f10 2024-09-10T08:08:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4a0102790 0x7fa4a019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4a0102790 0x7fa4a019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40942/0 (socket says 172.21.15.2:40942) 2024-09-10T08:08:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a5000640 1 -- 172.21.15.2:0/3006739330 learned_addr learned my addr 172.21.15.2:0/3006739330 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a5a00640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a0101de0 0x7fa4a019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a6400640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a0106f10 0x7fa4a019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a5000640 1 -- 172.21.15.2:0/3006739330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a0106f10 msgr2=0x7fa4a019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a5000640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a0106f10 0x7fa4a019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a5000640 1 -- 172.21.15.2:0/3006739330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a0101de0 msgr2=0x7fa4a019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a5000640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a0101de0 0x7fa4a019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a5000640 1 -- 172.21.15.2:0/3006739330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a019f580 con 0x7fa4a0102790 2024-09-10T08:08:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.153+0000 7fa4a5a00640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a0101de0 0x7fa4a019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.154+0000 7fa4a5000640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4a0102790 0x7fa4a019ab10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00ea60 tx=0x7fa48c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.154+0000 7fa49e000640 1 -- 172.21.15.2:0/3006739330 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c00cde0 con 0x7fa4a0102790 2024-09-10T08:08:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.154+0000 7fa49e000640 1 -- 172.21.15.2:0/3006739330 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48c0035f0 con 0x7fa4a0102790 2024-09-10T08:08:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.154+0000 7fa49e000640 1 -- 172.21.15.2:0/3006739330 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c010670 con 0x7fa4a0102790 2024-09-10T08:08:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.154+0000 7fa4a8400640 1 -- 172.21.15.2:0/3006739330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a019f870 con 0x7fa4a0102790 2024-09-10T08:08:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.154+0000 7fa4a8400640 1 -- 172.21.15.2:0/3006739330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a01a7150 con 0x7fa4a0102790 2024-09-10T08:08:09.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:08 smithi155 ceph-mon[35405]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:08:09.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.157+0000 7fa4aadfd740 1 Processor -- start 2024-09-10T08:08:09.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.157+0000 7fa4aadfd740 1 -- start start 2024-09-10T08:08:09.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.158+0000 7fa49e000640 1 -- 172.21.15.2:0/3006739330 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa48c0037a0 con 0x7fa4a0102790 2024-09-10T08:08:09.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.158+0000 7fa4aadfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859330 0x55c967859710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.158+0000 7fa4aadfd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859c50 0x55c967863fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.158+0000 7fa4aadfd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967864510 0x55c967866900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.158+0000 7fa4aadfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c9677bf0b0 con 0x55c967859330 2024-09-10T08:08:09.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.158+0000 7fa4aadfd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c967766ab0 con 0x55c967864510 2024-09-10T08:08:09.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.158+0000 7fa4aadfd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c9677bb070 con 0x55c967859c50 2024-09-10T08:08:09.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.158+0000 7fa4a6400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859330 0x55c967859710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.158+0000 7fa4a6400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859330 0x55c967859710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40950/0 (socket says 172.21.15.2:40950) 2024-09-10T08:08:09.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.158+0000 7fa4a6400640 1 -- 172.21.15.2:0/1962477414 learned_addr learned my addr 172.21.15.2:0/1962477414 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.158+0000 7fa4a5000640 1 --2- 172.21.15.2:0/1962477414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967864510 0x55c967866900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.158+0000 7fa4a5a00640 1 --2- 172.21.15.2:0/1962477414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859c50 0x55c967863fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.159+0000 7fa4a6400640 1 -- 172.21.15.2:0/1962477414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859c50 msgr2=0x55c967863fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.159+0000 7fa4a6400640 1 --2- 172.21.15.2:0/1962477414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859c50 0x55c967863fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.159+0000 7fa4a6400640 1 -- 172.21.15.2:0/1962477414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967864510 msgr2=0x55c967866900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.159+0000 7fa4a6400640 1 --2- 172.21.15.2:0/1962477414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967864510 0x55c967866900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.159+0000 7fa4a6400640 1 -- 172.21.15.2:0/1962477414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9677734a0 con 0x55c967859330 2024-09-10T08:08:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.159+0000 7fa4a6400640 1 --2- 172.21.15.2:0/1962477414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859330 0x55c967859710 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa498009ca0 tx=0x7fa498005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c558c1b1f6c96e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.159+0000 7fa492a00640 1 -- 172.21.15.2:0/1962477414 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498031db0 con 0x55c967859330 2024-09-10T08:08:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.159+0000 7fa492a00640 1 -- 172.21.15.2:0/1962477414 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa498036040 con 0x55c967859330 2024-09-10T08:08:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.159+0000 7fa492a00640 1 -- 172.21.15.2:0/1962477414 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498004330 con 0x55c967859330 2024-09-10T08:08:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.160+0000 7fa49e000640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa4880a2c00 0x7fa4880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.160+0000 7fa49e000640 1 -- 172.21.15.2:0/3006739330 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa48c0cf660 con 0x7fa4a0102790 2024-09-10T08:08:09.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:08 smithi107 ceph-mon[35048]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:08:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.160+0000 7fa4a5a00640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa4880a2c00 0x7fa4880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.160+0000 7fa4aadfd740 1 -- 172.21.15.2:0/1962477414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859330 msgr2=0x55c967859710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.160+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/1962477414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859330 0x55c967859710 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa498009ca0 tx=0x7fa498005f10 comp rx=0 tx=0).stop 2024-09-10T08:08:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.160+0000 7fa4aadfd740 1 -- 172.21.15.2:0/1962477414 shutdown_connections 2024-09-10T08:08:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.160+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/1962477414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967864510 0x55c967866900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.160+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/1962477414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859c50 0x55c967863fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.160+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/1962477414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859330 0x55c967859710 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.160+0000 7fa4aadfd740 1 -- 172.21.15.2:0/1962477414 >> 172.21.15.2:0/1962477414 conn(0x55c9678558c0 msgr2=0x55c967855ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.161+0000 7fa4aadfd740 1 -- 172.21.15.2:0/1962477414 shutdown_connections 2024-09-10T08:08:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.161+0000 7fa4aadfd740 1 -- 172.21.15.2:0/1962477414 wait complete. 2024-09-10T08:08:09.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.161+0000 7fa4a5a00640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa4880a2c00 0x7fa4880a50c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55c967856720 tx=0x7fa494005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.161+0000 7fa4aadfd740 1 -- start start 2024-09-10T08:08:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.161+0000 7fa4aadfd740 1 Processor -- start 2024-09-10T08:08:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.162+0000 7fa4aadfd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859330 0x55c967861450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.162+0000 7fa4aadfd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967859c50 0x55c967861990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.162+0000 7fa4aadfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967864510 0x55c967861ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.162+0000 7fa4aadfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c967867000 con 0x55c967864510 2024-09-10T08:08:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.162+0000 7fa4aadfd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c967747920 con 0x55c967859c50 2024-09-10T08:08:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.162+0000 7fa4aadfd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c967867180 con 0x55c967859330 2024-09-10T08:08:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.162+0000 7fa4aadfd740 10 client.0 _ll_register_callbacks cb 0x55c96776b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.162+0000 7fa4a6400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967864510 0x55c967861ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.162+0000 7fa4a5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859330 0x55c967861450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.162+0000 7fa4a5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967859c50 0x55c967861990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.162+0000 7fa4a5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859330 0x55c967861450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56772/0 (socket says 172.21.15.2:56772) 2024-09-10T08:08:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.162+0000 7fa4a6400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967864510 0x55c967861ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40956/0 (socket says 172.21.15.2:40956) 2024-09-10T08:08:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.162+0000 7fa4a5a00640 1 -- 172.21.15.2:0/2517524459 learned_addr learned my addr 172.21.15.2:0/2517524459 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.163+0000 7fa4a5000640 1 -- 172.21.15.2:0/2517524459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859330 msgr2=0x55c967861450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.163+0000 7fa4a5000640 1 --2- 172.21.15.2:0/2517524459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859330 0x55c967861450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.163+0000 7fa4a5000640 1 -- 172.21.15.2:0/2517524459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967864510 msgr2=0x55c967861ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.163+0000 7fa4a5000640 1 --2- 172.21.15.2:0/2517524459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967864510 0x55c967861ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.163+0000 7fa4a5000640 1 -- 172.21.15.2:0/2517524459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9675302b0 con 0x55c967859c50 2024-09-10T08:08:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.163+0000 7fa4a5000640 1 --2- 172.21.15.2:0/2517524459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967859c50 0x55c967861990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55c967856c40 tx=0x7fa48c0976b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=704f3b62024c0bc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.163+0000 7fa487400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:08:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.163+0000 7fa487400640 1 -- 172.21.15.2:0/2517524459 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c09b5b0 con 0x55c967859c50 2024-09-10T08:08:09.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.163+0000 7fa487400640 1 -- 172.21.15.2:0/2517524459 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa48c003b30 con 0x55c967859c50 2024-09-10T08:08:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.163+0000 7fa4aadfd740 1 -- 172.21.15.2:0/2517524459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa498036040 con 0x55c967859c50 2024-09-10T08:08:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.163+0000 7fa487400640 1 -- 172.21.15.2:0/2517524459 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa48c09b3d0 con 0x55c967859c50 2024-09-10T08:08:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.163+0000 7fa4aadfd740 10 client.25094 fetch_fsmap learned FSMap version 25 2024-09-10T08:08:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.163+0000 7fa4aadfd740 1 -- 172.21.15.2:0/2517524459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c967872bf0 con 0x55c967859c50 2024-09-10T08:08:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.164+0000 7fa487400640 1 -- 172.21.15.2:0/2517524459 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa498036040 con 0x55c967859c50 2024-09-10T08:08:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.164+0000 7fa487400640 1 -- 172.21.15.2:0/2517524459 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa48c0cf280 con 0x55c967859c50 2024-09-10T08:08:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.164+0000 7fa4aadfd740 10 client.25094 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:08:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.164+0000 7fa4aadfd740 10 client.25094 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:08:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.164+0000 7fa4aadfd740 20 client.25094 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.164+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/2517524459 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c967875250 0x55c9678766c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.164+0000 7fa4aadfd740 4 client.25094 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:08:09.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.164+0000 7fa4aadfd740 1 -- 172.21.15.2:0/2517524459 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9677cc620 con 0x55c967875250 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.164+0000 7fa4a5a00640 1 --2- 172.21.15.2:0/2517524459 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c967875250 0x55c9678766c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.165+0000 7fa4a5a00640 1 --2- 172.21.15.2:0/2517524459 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c967875250 0x55c9678766c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=abff8cbd7ce664e5 server_cookie=8b5bf335d60b4a1a in_seq=0 out_seq=0 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.165+0000 7fa487400640 10 client.25094 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.166+0000 7fa487400640 1 -- 172.21.15.2:0/2517524459 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9677cc620 con 0x55c967875250 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.166+0000 7fa487400640 10 client.25094 handle_command_reply: tid=0 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.166+0000 7fa4aadfd740 1 client.25094 shutdown 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.166+0000 7fa4aadfd740 1 -- 172.21.15.2:0/2517524459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967859c50 msgr2=0x55c967861990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.166+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/2517524459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967859c50 0x55c967861990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55c967856c40 tx=0x7fa48c0976b0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.166+0000 7fa4aadfd740 1 -- 172.21.15.2:0/2517524459 shutdown_connections 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.166+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/2517524459 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c967875250 0x55c9678766c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.166+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/2517524459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967864510 0x55c967861ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.166+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/2517524459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967859c50 0x55c967861990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.166+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/2517524459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859330 0x55c967861450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.166+0000 7fa4aadfd740 1 -- 172.21.15.2:0/2517524459 >> 172.21.15.2:0/2517524459 conn(0x55c9678558c0 msgr2=0x55c96785b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:09.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.166+0000 7fa487400640 0 client.25094 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:08:09.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.166+0000 7fa4aadfd740 1 -- 172.21.15.2:0/2517524459 shutdown_connections 2024-09-10T08:08:09.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.167+0000 7fa4aadfd740 1 -- 172.21.15.2:0/2517524459 wait complete. 2024-09-10T08:08:09.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.167+0000 7fa4aadfd740 20 client.25094 trim_cache size 0 max 16384 2024-09-10T08:08:09.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.185+0000 7fa4aadfd740 1 Processor -- start 2024-09-10T08:08:09.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.185+0000 7fa4aadfd740 1 -- start start 2024-09-10T08:08:09.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.186+0000 7fa4aadfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859330 0x55c96785fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.186+0000 7fa4aadfd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967859c50 0x55c967860260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.186+0000 7fa4aadfd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967864510 0x55c96785e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.186+0000 7fa4aadfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c967874a00 con 0x55c967859330 2024-09-10T08:08:09.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.186+0000 7fa4aadfd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c9678750d0 con 0x55c967859c50 2024-09-10T08:08:09.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.186+0000 7fa4aadfd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c9678757a0 con 0x55c967864510 2024-09-10T08:08:09.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.186+0000 7fa4a5a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859330 0x55c96785fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.186+0000 7fa4a6400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967864510 0x55c96785e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.186+0000 7fa4a5a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859330 0x55c96785fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40960/0 (socket says 172.21.15.2:40960) 2024-09-10T08:08:09.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.186+0000 7fa4a5a00640 1 -- 172.21.15.2:0/3098689601 learned_addr learned my addr 172.21.15.2:0/3098689601 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:09.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.186+0000 7fa4a5000640 1 --2- 172.21.15.2:0/3098689601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967859c50 0x55c967860260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.187+0000 7fa4a6400640 1 -- 172.21.15.2:0/3098689601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967859c50 msgr2=0x55c967860260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:09.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.187+0000 7fa4a6400640 1 --2- 172.21.15.2:0/3098689601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967859c50 0x55c967860260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.187+0000 7fa4a6400640 1 -- 172.21.15.2:0/3098689601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859330 msgr2=0x55c96785fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:09.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.187+0000 7fa4a6400640 1 --2- 172.21.15.2:0/3098689601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859330 0x55c96785fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.187+0000 7fa4a6400640 1 -- 172.21.15.2:0/3098689601 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9678627d0 con 0x55c967864510 2024-09-10T08:08:09.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.187+0000 7fa4a6400640 1 --2- 172.21.15.2:0/3098689601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967864510 0x55c96785e6d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa498005e30 tx=0x7fa498031980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98207d5119871400 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:09.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.187+0000 7fa492000640 1 -- 172.21.15.2:0/3098689601 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498031650 con 0x55c967864510 2024-09-10T08:08:09.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.187+0000 7fa492000640 1 -- 172.21.15.2:0/3098689601 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa498036050 con 0x55c967864510 2024-09-10T08:08:09.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.187+0000 7fa492000640 1 -- 172.21.15.2:0/3098689601 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498004830 con 0x55c967864510 2024-09-10T08:08:09.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.188+0000 7fa4aadfd740 1 -- 172.21.15.2:0/3098689601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967864510 msgr2=0x55c96785e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:09.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.188+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/3098689601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967864510 0x55c96785e6d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa498005e30 tx=0x7fa498031980 comp rx=0 tx=0).stop 2024-09-10T08:08:09.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.188+0000 7fa4aadfd740 1 -- 172.21.15.2:0/3098689601 shutdown_connections 2024-09-10T08:08:09.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.188+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/3098689601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967864510 0x55c96785e6d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.188+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/3098689601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967859c50 0x55c967860260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.188+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/3098689601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859330 0x55c96785fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.188+0000 7fa4aadfd740 1 -- 172.21.15.2:0/3098689601 >> 172.21.15.2:0/3098689601 conn(0x55c9678558c0 msgr2=0x55c96785c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:09.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.188+0000 7fa4aadfd740 1 -- 172.21.15.2:0/3098689601 shutdown_connections 2024-09-10T08:08:09.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.188+0000 7fa4aadfd740 1 -- 172.21.15.2:0/3098689601 wait complete. 2024-09-10T08:08:09.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.188+0000 7fa4aadfd740 1 -- start start 2024-09-10T08:08:09.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.189+0000 7fa4aadfd740 1 Processor -- start 2024-09-10T08:08:09.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.189+0000 7fa4aadfd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859330 0x55c96787ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.189+0000 7fa4aadfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859c50 0x55c96787f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.189+0000 7fa4aadfd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967864510 0x55c96785de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.189+0000 7fa4aadfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c967876540 con 0x55c967859c50 2024-09-10T08:08:09.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.189+0000 7fa4aadfd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c967875e70 con 0x55c967864510 2024-09-10T08:08:09.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.189+0000 7fa4aadfd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c967876c10 con 0x55c967859330 2024-09-10T08:08:09.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.189+0000 7fa4aadfd740 10 client.0 _ll_register_callbacks cb 0x7fa48c009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:09.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.189+0000 7fa4a5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859c50 0x55c96787f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.189+0000 7fa4a5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859c50 0x55c96787f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40962/0 (socket says 172.21.15.2:40962) 2024-09-10T08:08:09.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.189+0000 7fa4a5000640 1 -- 172.21.15.2:0/3843889940 learned_addr learned my addr 172.21.15.2:0/3843889940 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:09.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.189+0000 7fa4a6400640 1 --2- 172.21.15.2:0/3843889940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967864510 0x55c96785de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.190+0000 7fa4a5a00640 1 --2- 172.21.15.2:0/3843889940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859330 0x55c96787ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.190+0000 7fa4a5000640 1 -- 172.21.15.2:0/3843889940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859330 msgr2=0x55c96787ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:09.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.190+0000 7fa4a5000640 1 --2- 172.21.15.2:0/3843889940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859330 0x55c96787ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.190+0000 7fa4a5000640 1 -- 172.21.15.2:0/3843889940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967864510 msgr2=0x55c96785de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:09.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.190+0000 7fa4a5000640 1 --2- 172.21.15.2:0/3843889940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967864510 0x55c96785de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.190+0000 7fa4a5000640 1 -- 172.21.15.2:0/3843889940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96785e610 con 0x55c967859c50 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.190+0000 7fa4a5000640 1 --2- 172.21.15.2:0/3843889940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859c50 0x55c96787f1d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0983f0 tx=0x7fa48c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5752fc76b57fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.190+0000 7fa487400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.190+0000 7fa487400640 1 -- 172.21.15.2:0/3843889940 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c09b5b0 con 0x55c967859c50 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.190+0000 7fa4aadfd740 1 -- 172.21.15.2:0/3843889940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa498036050 con 0x55c967859c50 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.190+0000 7fa487400640 1 -- 172.21.15.2:0/3843889940 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa48c09b0d0 con 0x55c967859c50 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.191+0000 7fa487400640 1 -- 172.21.15.2:0/3843889940 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa48c09fe20 con 0x55c967859c50 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.191+0000 7fa4aadfd740 10 client.15081 fetch_fsmap learned FSMap version 25 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.191+0000 7fa4aadfd740 1 -- 172.21.15.2:0/3843889940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c96787e5b0 con 0x55c967859c50 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.191+0000 7fa487400640 1 -- 172.21.15.2:0/3843889940 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa498036050 con 0x55c967859c50 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.191+0000 7fa487400640 1 -- 172.21.15.2:0/3843889940 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa48c0977c0 con 0x55c967859c50 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.191+0000 7fa4aadfd740 10 client.15081 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.191+0000 7fa4aadfd740 10 client.15081 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.191+0000 7fa4aadfd740 20 client.15081 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:09.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.191+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/3843889940 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c9678e3fd0 0x55c9678e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:09.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.191+0000 7fa4aadfd740 4 client.15081 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:08:09.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.191+0000 7fa4aadfd740 1 -- 172.21.15.2:0/3843889940 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "status"}) -- 0x55c9677c6b30 con 0x55c9678e3fd0 2024-09-10T08:08:09.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.192+0000 7fa4a5a00640 1 --2- 172.21.15.2:0/3843889940 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c9678e3fd0 0x55c9678e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:09.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.192+0000 7fa4a5a00640 1 --2- 172.21.15.2:0/3843889940 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c9678e3fd0 0x55c9678e2ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7dec5f4e5b505bd server_cookie=3fc33dcd5710ca9b in_seq=0 out_seq=0 2024-09-10T08:08:09.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.192+0000 7fa487400640 10 client.15081 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:08:09.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.192+0000 7fa487400640 1 -- 172.21.15.2:0/3843889940 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c9677c6b30 con 0x55c9678e3fd0 2024-09-10T08:08:09.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.192+0000 7fa487400640 10 client.15081 handle_command_reply: tid=0 2024-09-10T08:08:09.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.192+0000 7fa4aadfd740 1 client.15081 shutdown 2024-09-10T08:08:09.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.193+0000 7fa4aadfd740 1 -- 172.21.15.2:0/3843889940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859c50 msgr2=0x55c96787f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:09.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.193+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/3843889940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859c50 0x55c96787f1d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0983f0 tx=0x7fa48c003d40 comp rx=0 tx=0).stop 2024-09-10T08:08:09.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.193+0000 7fa4aadfd740 1 -- 172.21.15.2:0/3843889940 shutdown_connections 2024-09-10T08:08:09.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.193+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/3843889940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c967864510 0x55c96785de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.193+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/3843889940 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c9678e3fd0 0x55c9678e2ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.193+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/3843889940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c967859c50 0x55c96787f1d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.193+0000 7fa4aadfd740 1 --2- 172.21.15.2:0/3843889940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c967859330 0x55c96787ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.193+0000 7fa4aadfd740 1 -- 172.21.15.2:0/3843889940 >> 172.21.15.2:0/3843889940 conn(0x55c9678558c0 msgr2=0x55c96785f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:09.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.193+0000 7fa487400640 0 client.15081 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:08:09.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.193+0000 7fa4aadfd740 1 -- 172.21.15.2:0/3843889940 shutdown_connections 2024-09-10T08:08:09.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.193+0000 7fa4aadfd740 1 -- 172.21.15.2:0/3843889940 wait complete. 2024-09-10T08:08:09.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.193+0000 7fa4aadfd740 20 client.15081 trim_cache size 0 max 16384 2024-09-10T08:08:09.197 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:08:09.198 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "26ec64b0-6f4a-11ef-bcea-c7b262605968", 2024-09-10T08:08:09.198 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2024-09-10T08:08:09.198 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24494, 2024-09-10T08:08:09.198 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-09-10T08:08:09.198 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-09-10T08:08:09.198 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-09-10T08:08:09.198 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 108.40930079899999, 2024-09-10T08:08:09.198 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 25, 2024-09-10T08:08:09.198 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 95, 2024-09-10T08:08:09.198 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 93, 2024-09-10T08:08:09.198 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 189.12142395000001 2024-09-10T08:08:09.198 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:08:09.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.194+0000 7fa4a8400640 1 -- 172.21.15.2:0/3006739330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa4880a2c00 msgr2=0x7fa4880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:09.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.194+0000 7fa4a8400640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa4880a2c00 0x7fa4880a50c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55c967856720 tx=0x7fa494005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:09.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.194+0000 7fa4a8400640 1 -- 172.21.15.2:0/3006739330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4a0102790 msgr2=0x7fa4a019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:09.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.194+0000 7fa4a8400640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4a0102790 0x7fa4a019ab10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00ea60 tx=0x7fa48c00ef20 comp rx=0 tx=0).stop 2024-09-10T08:08:09.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.194+0000 7fa4a8400640 1 -- 172.21.15.2:0/3006739330 shutdown_connections 2024-09-10T08:08:09.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.194+0000 7fa4a8400640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa4880a2c00 0x7fa4880a50c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.194+0000 7fa4a8400640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a0106f10 0x7fa4a019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.194+0000 7fa4a8400640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4a0102790 0x7fa4a019ab10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.194+0000 7fa4a8400640 1 --2- 172.21.15.2:0/3006739330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a0101de0 0x7fa4a019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:09.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.194+0000 7fa4a8400640 1 -- 172.21.15.2:0/3006739330 >> 172.21.15.2:0/3006739330 conn(0x7fa4a00fd8e0 msgr2=0x7fa4a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:09.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.194+0000 7fa4a8400640 1 -- 172.21.15.2:0/3006739330 shutdown_connections 2024-09-10T08:08:09.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:09.194+0000 7fa4a8400640 1 -- 172.21.15.2:0/3006739330 wait complete. 2024-09-10T08:08:10.078 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T08:08:10.457 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:10.905 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:10 smithi002 ceph-mon[30563]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:08:11.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.018+0000 7fb978200640 1 -- 172.21.15.2:0/485793909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9701049a0 msgr2=0x7fb9701089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:11.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.018+0000 7fb978200640 1 --2- 172.21.15.2:0/485793909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9701049a0 0x7fb9701089f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009900 tx=0x7fb95c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.018+0000 7fb978200640 1 -- 172.21.15.2:0/485793909 shutdown_connections 2024-09-10T08:08:11.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.018+0000 7fb978200640 1 --2- 172.21.15.2:0/485793909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb970109120 0x7fb970110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.018+0000 7fb978200640 1 --2- 172.21.15.2:0/485793909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9701049a0 0x7fb9701089f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.018+0000 7fb978200640 1 --2- 172.21.15.2:0/485793909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb970103ff0 0x7fb9701043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.018+0000 7fb978200640 1 -- 172.21.15.2:0/485793909 >> 172.21.15.2:0/485793909 conn(0x7fb9700ffa90 msgr2=0x7fb970101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:11.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.018+0000 7fb978200640 1 -- 172.21.15.2:0/485793909 shutdown_connections 2024-09-10T08:08:11.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.018+0000 7fb978200640 1 -- 172.21.15.2:0/485793909 wait complete. 2024-09-10T08:08:11.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.019+0000 7fb978200640 1 Processor -- start 2024-09-10T08:08:11.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.019+0000 7fb978200640 1 -- start start 2024-09-10T08:08:11.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.020+0000 7fb978200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb970103ff0 0x7fb97019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.020+0000 7fb978200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9701049a0 0x7fb97019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.020+0000 7fb978200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb970109120 0x7fb97019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.020+0000 7fb978200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb970114140 con 0x7fb9701049a0 2024-09-10T08:08:11.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.020+0000 7fb978200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb970113fc0 con 0x7fb970103ff0 2024-09-10T08:08:11.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.020+0000 7fb978200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb9701142c0 con 0x7fb970109120 2024-09-10T08:08:11.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.020+0000 7fb974e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9701049a0 0x7fb97019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.020+0000 7fb974e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9701049a0 0x7fb97019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40990/0 (socket says 172.21.15.2:40990) 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.020+0000 7fb974e00640 1 -- 172.21.15.2:0/2952198025 learned_addr learned my addr 172.21.15.2:0/2952198025 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.020+0000 7fb975800640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb970103ff0 0x7fb97019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.020+0000 7fb976200640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb970109120 0x7fb97019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.021+0000 7fb974e00640 1 -- 172.21.15.2:0/2952198025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb970109120 msgr2=0x7fb97019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.021+0000 7fb974e00640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb970109120 0x7fb97019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.021+0000 7fb974e00640 1 -- 172.21.15.2:0/2952198025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb970103ff0 msgr2=0x7fb97019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.021+0000 7fb974e00640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb970103ff0 0x7fb97019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.021+0000 7fb974e00640 1 -- 172.21.15.2:0/2952198025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97019f5d0 con 0x7fb9701049a0 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.021+0000 7fb976200640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb970109120 0x7fb97019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.021+0000 7fb975800640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb970103ff0 0x7fb97019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.021+0000 7fb974e00640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9701049a0 0x7fb97019ab60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb95c006010 tx=0x7fb95c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.021+0000 7fb96e000640 1 -- 172.21.15.2:0/2952198025 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c03f030 con 0x7fb9701049a0 2024-09-10T08:08:11.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.021+0000 7fb96e000640 1 -- 172.21.15.2:0/2952198025 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb95c004440 con 0x7fb9701049a0 2024-09-10T08:08:11.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.021+0000 7fb96e000640 1 -- 172.21.15.2:0/2952198025 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c035070 con 0x7fb9701049a0 2024-09-10T08:08:11.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.022+0000 7fb978200640 1 -- 172.21.15.2:0/2952198025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97019f860 con 0x7fb9701049a0 2024-09-10T08:08:11.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.022+0000 7fb978200640 1 -- 172.21.15.2:0/2952198025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97019fcc0 con 0x7fb9701049a0 2024-09-10T08:08:11.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.025+0000 7fb97ac06740 1 Processor -- start 2024-09-10T08:08:11.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.025+0000 7fb96e000640 1 -- 172.21.15.2:0/2952198025 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb95c042030 con 0x7fb9701049a0 2024-09-10T08:08:11.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.026+0000 7fb97ac06740 1 -- start start 2024-09-10T08:08:11.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.026+0000 7fb97ac06740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad84a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.026+0000 7fb97ac06740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad854fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.026+0000 7fb97ac06740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad855510 0x5634ad857900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.026+0000 7fb97ac06740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5634ad7b00b0 con 0x5634ad84a330 2024-09-10T08:08:11.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.026+0000 7fb97ac06740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5634ad757ab0 con 0x5634ad84ac50 2024-09-10T08:08:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.026+0000 7fb97ac06740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5634ad7ac070 con 0x5634ad855510 2024-09-10T08:08:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.026+0000 7fb976200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad84a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.026+0000 7fb975800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad854fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.026+0000 7fb976200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad84a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41004/0 (socket says 172.21.15.2:41004) 2024-09-10T08:08:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.026+0000 7fb975800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad854fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52254/0 (socket says 172.21.15.2:52254) 2024-09-10T08:08:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.026+0000 7fb976200640 1 -- 172.21.15.2:0/3690255021 learned_addr learned my addr 172.21.15.2:0/3690255021 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.026+0000 7fb974e00640 1 --2- 172.21.15.2:0/3690255021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad855510 0x5634ad857900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.027+0000 7fb975800640 1 -- 172.21.15.2:0/3690255021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad855510 msgr2=0x5634ad857900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.027+0000 7fb975800640 1 --2- 172.21.15.2:0/3690255021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad855510 0x5634ad857900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.027+0000 7fb975800640 1 -- 172.21.15.2:0/3690255021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 msgr2=0x5634ad84a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.027+0000 7fb975800640 1 --2- 172.21.15.2:0/3690255021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad84a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.027+0000 7fb975800640 1 -- 172.21.15.2:0/3690255021 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634ad7644a0 con 0x5634ad84ac50 2024-09-10T08:08:11.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.027+0000 7fb975800640 1 --2- 172.21.15.2:0/3690255021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad854fd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb964009fd0 tx=0x7fb9640118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e0467892e4313c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:11.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.027+0000 7fb96e000640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9580a2b80 0x7fb9580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.027+0000 7fb962a00640 1 -- 172.21.15.2:0/3690255021 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964004310 con 0x5634ad84ac50 2024-09-10T08:08:11.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.027+0000 7fb962a00640 1 -- 172.21.15.2:0/3690255021 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9640044b0 con 0x5634ad84ac50 2024-09-10T08:08:11.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.027+0000 7fb962a00640 1 -- 172.21.15.2:0/3690255021 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964003650 con 0x5634ad84ac50 2024-09-10T08:08:11.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.027+0000 7fb96e000640 1 -- 172.21.15.2:0/2952198025 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb95c0efb80 con 0x7fb9701049a0 2024-09-10T08:08:11.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.028+0000 7fb976200640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9580a2b80 0x7fb9580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.028+0000 7fb97ac06740 1 -- 172.21.15.2:0/3690255021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 msgr2=0x5634ad854fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.028+0000 7fb97ac06740 1 --2- 172.21.15.2:0/3690255021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad854fd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb964009fd0 tx=0x7fb9640118d0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.028+0000 7fb976200640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9580a2b80 0x7fb9580a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb968009fd0 tx=0x7fb968009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.028+0000 7fb97ac06740 1 -- 172.21.15.2:0/3690255021 shutdown_connections 2024-09-10T08:08:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.028+0000 7fb97ac06740 1 --2- 172.21.15.2:0/3690255021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad855510 0x5634ad857900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.028+0000 7fb97ac06740 1 --2- 172.21.15.2:0/3690255021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad854fd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.028+0000 7fb97ac06740 1 --2- 172.21.15.2:0/3690255021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad84a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.028+0000 7fb97ac06740 1 -- 172.21.15.2:0/3690255021 >> 172.21.15.2:0/3690255021 conn(0x5634ad8468c0 msgr2=0x5634ad846ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.028+0000 7fb97ac06740 1 -- 172.21.15.2:0/3690255021 shutdown_connections 2024-09-10T08:08:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.029+0000 7fb97ac06740 1 -- 172.21.15.2:0/3690255021 wait complete. 2024-09-10T08:08:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.029+0000 7fb97ac06740 1 -- start start 2024-09-10T08:08:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.029+0000 7fb97ac06740 1 Processor -- start 2024-09-10T08:08:11.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.030+0000 7fb97ac06740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad8525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.030+0000 7fb97ac06740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad852b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.030+0000 7fb97ac06740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad855510 0x5634ad853070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.030+0000 7fb97ac06740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5634ad858060 con 0x5634ad84a330 2024-09-10T08:08:11.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.030+0000 7fb97ac06740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5634ad738920 con 0x5634ad84ac50 2024-09-10T08:08:11.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.030+0000 7fb97ac06740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5634ad8581e0 con 0x5634ad855510 2024-09-10T08:08:11.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.030+0000 7fb97ac06740 10 client.0 _ll_register_callbacks cb 0x5634ad75c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:11.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.030+0000 7fb975800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad8525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.030+0000 7fb975800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad8525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41016/0 (socket says 172.21.15.2:41016) 2024-09-10T08:08:11.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.030+0000 7fb975800640 1 -- 172.21.15.2:0/3464607232 learned_addr learned my addr 172.21.15.2:0/3464607232 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:11.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.030+0000 7fb974e00640 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad852b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.030+0000 7fb976200640 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad855510 0x5634ad853070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb974e00640 1 -- 172.21.15.2:0/3464607232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad855510 msgr2=0x5634ad853070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:11.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb974e00640 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad855510 0x5634ad853070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb974e00640 1 -- 172.21.15.2:0/3464607232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 msgr2=0x5634ad8525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:11.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb974e00640 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad8525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb974e00640 1 -- 172.21.15.2:0/3464607232 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634ad5212b0 con 0x5634ad84ac50 2024-09-10T08:08:11.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb975800640 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad8525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:11.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb974e00640 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad852b30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x5634ad847c40 tx=0x7fb95c0b82f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25201574a6417a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:11.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb957400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:08:11.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb957400640 1 -- 172.21.15.2:0/3464607232 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0bb8f0 con 0x5634ad84ac50 2024-09-10T08:08:11.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb957400640 1 -- 172.21.15.2:0/3464607232 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb95c0b8600 con 0x5634ad84ac50 2024-09-10T08:08:11.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb97ac06740 1 -- 172.21.15.2:0/3464607232 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9640044b0 con 0x5634ad84ac50 2024-09-10T08:08:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb957400640 1 -- 172.21.15.2:0/3464607232 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb95c0bc690 con 0x5634ad84ac50 2024-09-10T08:08:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb97ac06740 10 client.25112 fetch_fsmap learned FSMap version 25 2024-09-10T08:08:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.031+0000 7fb97ac06740 1 -- 172.21.15.2:0/3464607232 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634ad858060 con 0x5634ad84ac50 2024-09-10T08:08:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.032+0000 7fb957400640 1 -- 172.21.15.2:0/3464607232 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9640044b0 con 0x5634ad84ac50 2024-09-10T08:08:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.032+0000 7fb957400640 1 -- 172.21.15.2:0/3464607232 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb95c04c060 con 0x5634ad84ac50 2024-09-10T08:08:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.032+0000 7fb97ac06740 10 client.25112 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:08:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.032+0000 7fb97ac06740 10 client.25112 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:08:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.032+0000 7fb97ac06740 20 client.25112 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.032+0000 7fb97ac06740 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5634ad866480 0x5634ad8678f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.032+0000 7fb97ac06740 4 client.25112 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:08:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.032+0000 7fb97ac06740 1 -- 172.21.15.2:0/3464607232 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634ad7bd620 con 0x5634ad866480 2024-09-10T08:08:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.032+0000 7fb975800640 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5634ad866480 0x5634ad8678f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.033+0000 7fb975800640 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5634ad866480 0x5634ad8678f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc7dd3997faa191e server_cookie=e2ceada1529c135 in_seq=0 out_seq=0 2024-09-10T08:08:11.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.033+0000 7fb957400640 10 client.25112 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:08:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb957400640 1 -- 172.21.15.2:0/3464607232 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634ad7bd620 con 0x5634ad866480 2024-09-10T08:08:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb957400640 10 client.25112 handle_command_reply: tid=0 2024-09-10T08:08:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb97ac06740 1 client.25112 shutdown 2024-09-10T08:08:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb97ac06740 1 -- 172.21.15.2:0/3464607232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 msgr2=0x5634ad852b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb97ac06740 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad852b30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x5634ad847c40 tx=0x7fb95c0b82f0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb97ac06740 1 -- 172.21.15.2:0/3464607232 shutdown_connections 2024-09-10T08:08:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb97ac06740 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5634ad866480 0x5634ad8678f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb97ac06740 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad855510 0x5634ad853070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb97ac06740 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad852b30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb97ac06740 1 --2- 172.21.15.2:0/3464607232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad8525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb97ac06740 1 -- 172.21.15.2:0/3464607232 >> 172.21.15.2:0/3464607232 conn(0x5634ad8468c0 msgr2=0x5634ad84c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:11.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb957400640 0 client.25112 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:08:11.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb97ac06740 1 -- 172.21.15.2:0/3464607232 shutdown_connections 2024-09-10T08:08:11.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.034+0000 7fb97ac06740 1 -- 172.21.15.2:0/3464607232 wait complete. 2024-09-10T08:08:11.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.035+0000 7fb97ac06740 20 client.25112 trim_cache size 0 max 16384 2024-09-10T08:08:11.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.053+0000 7fb97ac06740 1 Processor -- start 2024-09-10T08:08:11.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.053+0000 7fb97ac06740 1 -- start start 2024-09-10T08:08:11.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.054+0000 7fb97ac06740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad852ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.054+0000 7fb97ac06740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad84ac50 0x5634ad864f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.054+0000 7fb97ac06740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad855510 0x5634ad8654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.054+0000 7fb97ac06740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5634ad865c30 con 0x5634ad84a330 2024-09-10T08:08:11.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.054+0000 7fb97ac06740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5634ad866300 con 0x5634ad855510 2024-09-10T08:08:11.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.054+0000 7fb97ac06740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5634ad8669d0 con 0x5634ad84ac50 2024-09-10T08:08:11.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.054+0000 7fb975800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad852ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.054+0000 7fb975800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad852ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41032/0 (socket says 172.21.15.2:41032) 2024-09-10T08:08:11.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.054+0000 7fb975800640 1 -- 172.21.15.2:0/651528489 learned_addr learned my addr 172.21.15.2:0/651528489 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:11.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.054+0000 7fb974e00640 1 --2- 172.21.15.2:0/651528489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad84ac50 0x5634ad864f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.054+0000 7fb976200640 1 --2- 172.21.15.2:0/651528489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad855510 0x5634ad8654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.054+0000 7fb975800640 1 -- 172.21.15.2:0/651528489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad84ac50 msgr2=0x5634ad864f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:11.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.054+0000 7fb975800640 1 --2- 172.21.15.2:0/651528489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad84ac50 0x5634ad864f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.055+0000 7fb975800640 1 -- 172.21.15.2:0/651528489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad855510 msgr2=0x5634ad8654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:11.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.055+0000 7fb975800640 1 --2- 172.21.15.2:0/651528489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad855510 0x5634ad8654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.055+0000 7fb975800640 1 -- 172.21.15.2:0/651528489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634ad84aa80 con 0x5634ad84a330 2024-09-10T08:08:11.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.055+0000 7fb975800640 1 --2- 172.21.15.2:0/651528489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad852ee0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb964009710 tx=0x7fb964004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ffc687ec886568 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:11.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.055+0000 7fb962000640 1 -- 172.21.15.2:0/651528489 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964004230 con 0x5634ad84a330 2024-09-10T08:08:11.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.055+0000 7fb962000640 1 -- 172.21.15.2:0/651528489 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9640043d0 con 0x5634ad84a330 2024-09-10T08:08:11.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.055+0000 7fb962000640 1 -- 172.21.15.2:0/651528489 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96402e6a0 con 0x5634ad84a330 2024-09-10T08:08:11.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.056+0000 7fb97ac06740 1 -- 172.21.15.2:0/651528489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 msgr2=0x5634ad852ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:11.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.056+0000 7fb97ac06740 1 --2- 172.21.15.2:0/651528489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad852ee0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb964009710 tx=0x7fb964004c10 comp rx=0 tx=0).stop 2024-09-10T08:08:11.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.056+0000 7fb97ac06740 1 -- 172.21.15.2:0/651528489 shutdown_connections 2024-09-10T08:08:11.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.056+0000 7fb97ac06740 1 --2- 172.21.15.2:0/651528489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad855510 0x5634ad8654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.056+0000 7fb97ac06740 1 --2- 172.21.15.2:0/651528489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad84ac50 0x5634ad864f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.056+0000 7fb97ac06740 1 --2- 172.21.15.2:0/651528489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad84a330 0x5634ad852ee0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.056+0000 7fb97ac06740 1 -- 172.21.15.2:0/651528489 >> 172.21.15.2:0/651528489 conn(0x5634ad8468c0 msgr2=0x5634ad850f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:11.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.056+0000 7fb97ac06740 1 -- 172.21.15.2:0/651528489 shutdown_connections 2024-09-10T08:08:11.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.056+0000 7fb97ac06740 1 -- 172.21.15.2:0/651528489 wait complete. 2024-09-10T08:08:11.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.056+0000 7fb97ac06740 1 -- start start 2024-09-10T08:08:11.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.057+0000 7fb97ac06740 1 Processor -- start 2024-09-10T08:08:11.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.057+0000 7fb97ac06740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad84a330 0x5634ad86ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.057+0000 7fb97ac06740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad870460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.057+0000 7fb97ac06740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad855510 0x5634ad84f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.057+0000 7fb97ac06740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5634ad867770 con 0x5634ad855510 2024-09-10T08:08:11.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.057+0000 7fb97ac06740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5634ad8670a0 con 0x5634ad84ac50 2024-09-10T08:08:11.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.057+0000 7fb97ac06740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5634ad867e40 con 0x5634ad84a330 2024-09-10T08:08:11.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.057+0000 7fb97ac06740 10 client.0 _ll_register_callbacks cb 0x7fb95c0095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:11.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.057+0000 7fb976200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad855510 0x5634ad84f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.057+0000 7fb976200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad855510 0x5634ad84f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41036/0 (socket says 172.21.15.2:41036) 2024-09-10T08:08:11.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.057+0000 7fb976200640 1 -- 172.21.15.2:0/2205935962 learned_addr learned my addr 172.21.15.2:0/2205935962 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:11.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb975800640 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad84a330 0x5634ad86ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb974e00640 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad870460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb976200640 1 -- 172.21.15.2:0/2205935962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad84a330 msgr2=0x5634ad86ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:11.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb976200640 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad84a330 0x5634ad86ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb976200640 1 -- 172.21.15.2:0/2205935962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 msgr2=0x5634ad870460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:11.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb976200640 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad870460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb976200640 1 -- 172.21.15.2:0/2205935962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634ad84fb30 con 0x5634ad855510 2024-09-10T08:08:11.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb975800640 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad84a330 0x5634ad86ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:08:11.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb974e00640 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad870460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:08:11.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb976200640 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad855510 0x5634ad84f300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb968005fd0 tx=0x7fb968004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3ea7010f9278f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:11.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb957400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:08:11.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb957400640 1 -- 172.21.15.2:0/2205935962 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96801eb80 con 0x5634ad855510 2024-09-10T08:08:11.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb97ac06740 1 -- 172.21.15.2:0/2205935962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9640043d0 con 0x5634ad855510 2024-09-10T08:08:11.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.058+0000 7fb957400640 1 -- 172.21.15.2:0/2205935962 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb96801ed20 con 0x5634ad855510 2024-09-10T08:08:11.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.059+0000 7fb957400640 1 -- 172.21.15.2:0/2205935962 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb968034050 con 0x5634ad855510 2024-09-10T08:08:11.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.059+0000 7fb97ac06740 10 client.15099 fetch_fsmap learned FSMap version 25 2024-09-10T08:08:11.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.059+0000 7fb97ac06740 1 -- 172.21.15.2:0/2205935962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634ad86e750 con 0x5634ad855510 2024-09-10T08:08:11.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.059+0000 7fb957400640 1 -- 172.21.15.2:0/2205935962 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9640043d0 con 0x5634ad855510 2024-09-10T08:08:11.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.059+0000 7fb957400640 1 -- 172.21.15.2:0/2205935962 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb96801d410 con 0x5634ad855510 2024-09-10T08:08:11.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.059+0000 7fb97ac06740 10 client.15099 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:08:11.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.059+0000 7fb97ac06740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:08:11.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.059+0000 7fb97ac06740 20 client.15099 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:11.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.059+0000 7fb97ac06740 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5634ad8d5590 0x5634ad8d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:11.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.059+0000 7fb97ac06740 4 client.15099 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:08:11.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.059+0000 7fb97ac06740 1 -- 172.21.15.2:0/2205935962 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "status"}) -- 0x5634ad7b7b30 con 0x5634ad8d5590 2024-09-10T08:08:11.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.060+0000 7fb975800640 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5634ad8d5590 0x5634ad8d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:11.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.060+0000 7fb975800640 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5634ad8d5590 0x5634ad8d42a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8250885ce1424e0 server_cookie=ec568b5c9a5b76a in_seq=0 out_seq=0 2024-09-10T08:08:11.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.060+0000 7fb957400640 10 client.15099 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:08:11.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb957400640 1 -- 172.21.15.2:0/2205935962 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5634ad7b7b30 con 0x5634ad8d5590 2024-09-10T08:08:11.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb957400640 10 client.15099 handle_command_reply: tid=0 2024-09-10T08:08:11.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb97ac06740 1 client.15099 shutdown 2024-09-10T08:08:11.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb97ac06740 1 -- 172.21.15.2:0/2205935962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad855510 msgr2=0x5634ad84f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:11.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb97ac06740 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad855510 0x5634ad84f300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb968005fd0 tx=0x7fb968004a40 comp rx=0 tx=0).stop 2024-09-10T08:08:11.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb97ac06740 1 -- 172.21.15.2:0/2205935962 shutdown_connections 2024-09-10T08:08:11.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb97ac06740 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5634ad8d5590 0x5634ad8d42a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb97ac06740 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634ad855510 0x5634ad84f300 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb97ac06740 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634ad84ac50 0x5634ad870460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb97ac06740 1 --2- 172.21.15.2:0/2205935962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634ad84a330 0x5634ad86ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb97ac06740 1 -- 172.21.15.2:0/2205935962 >> 172.21.15.2:0/2205935962 conn(0x5634ad8468c0 msgr2=0x5634ad850ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:11.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb957400640 0 client.15099 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:08:11.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb97ac06740 1 -- 172.21.15.2:0/2205935962 shutdown_connections 2024-09-10T08:08:11.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb97ac06740 1 -- 172.21.15.2:0/2205935962 wait complete. 2024-09-10T08:08:11.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.061+0000 7fb97ac06740 20 client.15099 trim_cache size 0 max 16384 2024-09-10T08:08:11.066 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:08:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "26ec64b0-6f4a-11ef-bcea-c7b262605968", 2024-09-10T08:08:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2024-09-10T08:08:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24499, 2024-09-10T08:08:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-09-10T08:08:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-09-10T08:08:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-09-10T08:08:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 111.28058224, 2024-09-10T08:08:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 25, 2024-09-10T08:08:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 95, 2024-09-10T08:08:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 93, 2024-09-10T08:08:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 188.357145269 2024-09-10T08:08:11.067 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:08:11.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.062+0000 7fb978200640 1 -- 172.21.15.2:0/2952198025 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9580a2b80 msgr2=0x7fb9580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:11.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.062+0000 7fb978200640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9580a2b80 0x7fb9580a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb968009fd0 tx=0x7fb968009500 comp rx=0 tx=0).stop 2024-09-10T08:08:11.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.062+0000 7fb978200640 1 -- 172.21.15.2:0/2952198025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9701049a0 msgr2=0x7fb97019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:11.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.062+0000 7fb978200640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9701049a0 0x7fb97019ab60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb95c006010 tx=0x7fb95c004040 comp rx=0 tx=0).stop 2024-09-10T08:08:11.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.062+0000 7fb978200640 1 -- 172.21.15.2:0/2952198025 shutdown_connections 2024-09-10T08:08:11.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.062+0000 7fb978200640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9580a2b80 0x7fb9580a5040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.062+0000 7fb978200640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb970109120 0x7fb97019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.062+0000 7fb978200640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9701049a0 0x7fb97019ab60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.062+0000 7fb978200640 1 --2- 172.21.15.2:0/2952198025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb970103ff0 0x7fb97019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:11.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.062+0000 7fb978200640 1 -- 172.21.15.2:0/2952198025 >> 172.21.15.2:0/2952198025 conn(0x7fb9700ffa90 msgr2=0x7fb970100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:11.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.062+0000 7fb978200640 1 -- 172.21.15.2:0/2952198025 shutdown_connections 2024-09-10T08:08:11.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:11.062+0000 7fb978200640 1 -- 172.21.15.2:0/2952198025 wait complete. 2024-09-10T08:08:11.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:10 smithi155 ceph-mon[35405]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:08:11.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:10 smithi107 ceph-mon[35048]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:08:11.953 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T08:08:12.331 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:12.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.885+0000 7f1939600640 1 -- 172.21.15.2:0/2762386739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19340745c0 msgr2=0x7f19340749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:12.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.885+0000 7f1939600640 1 --2- 172.21.15.2:0/2762386739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19340745c0 0x7f19340749a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1924009900 tx=0x7f192402a480 comp rx=0 tx=0).stop 2024-09-10T08:08:12.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.885+0000 7f1939600640 1 -- 172.21.15.2:0/2762386739 shutdown_connections 2024-09-10T08:08:12.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.885+0000 7f1939600640 1 --2- 172.21.15.2:0/2762386739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f193410e430 0x7f1934110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.885+0000 7f1939600640 1 --2- 172.21.15.2:0/2762386739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1934074ee0 0x7f193410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.885+0000 7f1939600640 1 --2- 172.21.15.2:0/2762386739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19340745c0 0x7f19340749a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.885+0000 7f1939600640 1 -- 172.21.15.2:0/2762386739 >> 172.21.15.2:0/2762386739 conn(0x7f19340fd8b0 msgr2=0x7f19340ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:12.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.885+0000 7f1939600640 1 -- 172.21.15.2:0/2762386739 shutdown_connections 2024-09-10T08:08:12.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.885+0000 7f1939600640 1 -- 172.21.15.2:0/2762386739 wait complete. 2024-09-10T08:08:12.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.886+0000 7f1939600640 1 Processor -- start 2024-09-10T08:08:12.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.886+0000 7f1939600640 1 -- start start 2024-09-10T08:08:12.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.887+0000 7f1939600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19340745c0 0x7f193419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.887+0000 7f1939600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1934074ee0 0x7f193419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.887+0000 7f1939600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f193410e430 0x7f193419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.887+0000 7f1939600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f19341141d0 con 0x7f193410e430 2024-09-10T08:08:12.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.887+0000 7f1939600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1934114050 con 0x7f19340745c0 2024-09-10T08:08:12.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.887+0000 7f1939600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1934114350 con 0x7f1934074ee0 2024-09-10T08:08:12.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.887+0000 7f1933400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f193410e430 0x7f193419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.887+0000 7f192a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1934074ee0 0x7f193419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.887+0000 7f192a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1934074ee0 0x7f193419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56860/0 (socket says 172.21.15.2:56860) 2024-09-10T08:08:12.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.887+0000 7f192a000640 1 -- 172.21.15.2:0/4089967617 learned_addr learned my addr 172.21.15.2:0/4089967617 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:12.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.887+0000 7f1932a00640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19340745c0 0x7f193419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.888+0000 7f1932a00640 1 -- 172.21.15.2:0/4089967617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1934074ee0 msgr2=0x7f193419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:12.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.888+0000 7f1932a00640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1934074ee0 0x7f193419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.888+0000 7f1932a00640 1 -- 172.21.15.2:0/4089967617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f193410e430 msgr2=0x7f193419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:12.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.888+0000 7f1932a00640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f193410e430 0x7f193419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.888+0000 7f1932a00640 1 -- 172.21.15.2:0/4089967617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193419f5e0 con 0x7f19340745c0 2024-09-10T08:08:12.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.888+0000 7f192a000640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1934074ee0 0x7f193419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:12.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.888+0000 7f1933400640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f193410e430 0x7f193419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:12.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.888+0000 7f1932a00640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19340745c0 0x7f193419a630 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f192402a9a0 tx=0x7f1924003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:12.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.888+0000 7f192be00640 1 -- 172.21.15.2:0/4089967617 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192403f070 con 0x7f19340745c0 2024-09-10T08:08:12.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.888+0000 7f192be00640 1 -- 172.21.15.2:0/4089967617 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1924031c90 con 0x7f19340745c0 2024-09-10T08:08:12.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.888+0000 7f1939600640 1 -- 172.21.15.2:0/4089967617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193419f870 con 0x7f19340745c0 2024-09-10T08:08:12.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.888+0000 7f192be00640 1 -- 172.21.15.2:0/4089967617 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924035030 con 0x7f19340745c0 2024-09-10T08:08:12.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.889+0000 7f1939600640 1 -- 172.21.15.2:0/4089967617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19341a71b0 con 0x7f19340745c0 2024-09-10T08:08:12.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.891+0000 7f192be00640 1 -- 172.21.15.2:0/4089967617 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1924042070 con 0x7f19340745c0 2024-09-10T08:08:12.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.892+0000 7f193c035740 1 Processor -- start 2024-09-10T08:08:12.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.892+0000 7f193c035740 1 -- start start 2024-09-10T08:08:12.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.892+0000 7f193c035740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6b330 0x55943eb6b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.892+0000 7f193c035740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6bc50 0x55943eb75fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.892+0000 7f193c035740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb76510 0x55943eb78900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.892+0000 7f193c035740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55943ead10b0 con 0x55943eb6b330 2024-09-10T08:08:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.892+0000 7f193c035740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55943ea78ab0 con 0x55943eb6bc50 2024-09-10T08:08:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.892+0000 7f193c035740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55943eacd070 con 0x55943eb76510 2024-09-10T08:08:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.893+0000 7f1933400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6b330 0x55943eb6b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.893+0000 7f1933400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6b330 0x55943eb6b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41074/0 (socket says 172.21.15.2:41074) 2024-09-10T08:08:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.893+0000 7f1932a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6bc50 0x55943eb75fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.893+0000 7f1933400640 1 -- 172.21.15.2:0/2481136640 learned_addr learned my addr 172.21.15.2:0/2481136640 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:12.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.893+0000 7f192a000640 1 --2- 172.21.15.2:0/2481136640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb76510 0x55943eb78900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.893+0000 7f1933400640 1 -- 172.21.15.2:0/2481136640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb76510 msgr2=0x55943eb78900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:12.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.893+0000 7f1933400640 1 --2- 172.21.15.2:0/2481136640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb76510 0x55943eb78900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.893+0000 7f1933400640 1 -- 172.21.15.2:0/2481136640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6bc50 msgr2=0x55943eb75fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:12.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.893+0000 7f1933400640 1 --2- 172.21.15.2:0/2481136640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6bc50 0x55943eb75fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.893+0000 7f1933400640 1 -- 172.21.15.2:0/2481136640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55943ea854a0 con 0x55943eb6b330 2024-09-10T08:08:12.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.893+0000 7f1932a00640 1 --2- 172.21.15.2:0/2481136640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6bc50 0x55943eb75fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:12.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.894+0000 7f1933400640 1 --2- 172.21.15.2:0/2481136640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6b330 0x55943eb6b710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f192c0106c0 tx=0x7f192c010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d72c9efa0aa313 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:12.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.894+0000 7f192be00640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19180a2c00 0x7f19180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.894+0000 7f1923400640 1 -- 172.21.15.2:0/2481136640 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c00e8d0 con 0x55943eb6b330 2024-09-10T08:08:12.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.894+0000 7f1923400640 1 -- 172.21.15.2:0/2481136640 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f192c012450 con 0x55943eb6b330 2024-09-10T08:08:12.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.894+0000 7f192be00640 1 -- 172.21.15.2:0/4089967617 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f19240efa70 con 0x7f19340745c0 2024-09-10T08:08:12.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.894+0000 7f1923400640 1 -- 172.21.15.2:0/2481136640 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c01c630 con 0x55943eb6b330 2024-09-10T08:08:12.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.894+0000 7f1932a00640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19180a2c00 0x7f19180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.895+0000 7f193c035740 1 -- 172.21.15.2:0/2481136640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6b330 msgr2=0x55943eb6b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.895+0000 7f1932a00640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19180a2c00 0x7f19180a50c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f19240bbc30 tx=0x7f19240f5510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.895+0000 7f193c035740 1 --2- 172.21.15.2:0/2481136640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6b330 0x55943eb6b710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f192c0106c0 tx=0x7f192c010b80 comp rx=0 tx=0).stop 2024-09-10T08:08:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.895+0000 7f193c035740 1 -- 172.21.15.2:0/2481136640 shutdown_connections 2024-09-10T08:08:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.895+0000 7f193c035740 1 --2- 172.21.15.2:0/2481136640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb76510 0x55943eb78900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.895+0000 7f193c035740 1 --2- 172.21.15.2:0/2481136640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6bc50 0x55943eb75fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.895+0000 7f193c035740 1 --2- 172.21.15.2:0/2481136640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6b330 0x55943eb6b710 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.895+0000 7f193c035740 1 -- 172.21.15.2:0/2481136640 >> 172.21.15.2:0/2481136640 conn(0x55943eb678c0 msgr2=0x55943eb67ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:12.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.895+0000 7f193c035740 1 -- 172.21.15.2:0/2481136640 shutdown_connections 2024-09-10T08:08:12.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.895+0000 7f193c035740 1 -- 172.21.15.2:0/2481136640 wait complete. 2024-09-10T08:08:12.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.896+0000 7f193c035740 1 -- start start 2024-09-10T08:08:12.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.896+0000 7f193c035740 1 Processor -- start 2024-09-10T08:08:12.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.896+0000 7f193c035740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6b330 0x55943eb73590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.896+0000 7f193c035740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb6bc50 0x55943eb73ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.896+0000 7f193c035740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb76510 0x55943eb74010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.896+0000 7f193c035740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55943eb79060 con 0x55943eb76510 2024-09-10T08:08:12.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.896+0000 7f193c035740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55943ea59920 con 0x55943eb6b330 2024-09-10T08:08:12.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.896+0000 7f193c035740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55943eb791e0 con 0x55943eb6bc50 2024-09-10T08:08:12.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.896+0000 7f193c035740 10 client.0 _ll_register_callbacks cb 0x55943ea7d8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:12.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.897+0000 7f192a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb6bc50 0x55943eb73ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.897+0000 7f192a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb6bc50 0x55943eb73ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56878/0 (socket says 172.21.15.2:56878) 2024-09-10T08:08:12.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.897+0000 7f1933400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb76510 0x55943eb74010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.897+0000 7f192a000640 1 -- 172.21.15.2:0/2694406673 learned_addr learned my addr 172.21.15.2:0/2694406673 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:12.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.897+0000 7f1933400640 1 --2- 172.21.15.2:0/2694406673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6b330 0x55943eb73590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.897+0000 7f192a000640 1 -- 172.21.15.2:0/2694406673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6b330 msgr2=0x55943eb73590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:12.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.897+0000 7f192a000640 1 --2- 172.21.15.2:0/2694406673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6b330 0x55943eb73590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.897+0000 7f192a000640 1 -- 172.21.15.2:0/2694406673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb76510 msgr2=0x55943eb74010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:12.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.897+0000 7f192a000640 1 --2- 172.21.15.2:0/2694406673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb76510 0x55943eb74010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.897+0000 7f192a000640 1 -- 172.21.15.2:0/2694406673 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55943e8422b0 con 0x55943eb6bc50 2024-09-10T08:08:12.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.897+0000 7f1933400640 1 --2- 172.21.15.2:0/2694406673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb76510 0x55943eb74010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:08:12.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.898+0000 7f192a000640 1 --2- 172.21.15.2:0/2694406673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb6bc50 0x55943eb73ad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f191c012d20 tx=0x7f191c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b427549d33add403 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:12.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.898+0000 7f1917e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:08:12.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.898+0000 7f1917e00640 1 -- 172.21.15.2:0/2694406673 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c012110 con 0x55943eb6bc50 2024-09-10T08:08:12.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.898+0000 7f193c035740 1 -- 172.21.15.2:0/2694406673 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f192c012450 con 0x55943eb6bc50 2024-09-10T08:08:12.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.898+0000 7f1917e00640 1 -- 172.21.15.2:0/2694406673 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f191c004050 con 0x55943eb6bc50 2024-09-10T08:08:12.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.898+0000 7f1917e00640 1 -- 172.21.15.2:0/2694406673 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f191c025050 con 0x55943eb6bc50 2024-09-10T08:08:12.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.898+0000 7f193c035740 10 client.25105 fetch_fsmap learned FSMap version 25 2024-09-10T08:08:12.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.898+0000 7f193c035740 1 -- 172.21.15.2:0/2694406673 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55943eb84f00 con 0x55943eb6bc50 2024-09-10T08:08:12.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.899+0000 7f1917e00640 1 -- 172.21.15.2:0/2694406673 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f192c012450 con 0x55943eb6bc50 2024-09-10T08:08:12.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.899+0000 7f1917e00640 1 -- 172.21.15.2:0/2694406673 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f191c010430 con 0x55943eb6bc50 2024-09-10T08:08:12.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.899+0000 7f193c035740 10 client.25105 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:08:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.899+0000 7f193c035740 10 client.25105 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:08:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.899+0000 7f193c035740 20 client.25105 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.899+0000 7f193c035740 1 --2- 172.21.15.2:0/2694406673 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55943eb874e0 0x55943eb88950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.899+0000 7f193c035740 4 client.25105 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:08:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.899+0000 7f193c035740 1 -- 172.21.15.2:0/2694406673 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55943eade620 con 0x55943eb874e0 2024-09-10T08:08:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.899+0000 7f1933400640 1 --2- 172.21.15.2:0/2694406673 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55943eb874e0 0x55943eb88950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.900+0000 7f1933400640 1 --2- 172.21.15.2:0/2694406673 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55943eb874e0 0x55943eb88950 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95a31359b40d0b0d server_cookie=e656940262c78cfb in_seq=0 out_seq=0 2024-09-10T08:08:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.900+0000 7f1917e00640 10 client.25105 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f1917e00640 1 -- 172.21.15.2:0/2694406673 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55943eade620 con 0x55943eb874e0 2024-09-10T08:08:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f1917e00640 10 client.25105 handle_command_reply: tid=0 2024-09-10T08:08:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f193c035740 1 client.25105 shutdown 2024-09-10T08:08:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f193c035740 1 -- 172.21.15.2:0/2694406673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb6bc50 msgr2=0x55943eb73ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:12.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f193c035740 1 --2- 172.21.15.2:0/2694406673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb6bc50 0x55943eb73ad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f191c012d20 tx=0x7f191c00f790 comp rx=0 tx=0).stop 2024-09-10T08:08:12.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f193c035740 1 -- 172.21.15.2:0/2694406673 shutdown_connections 2024-09-10T08:08:12.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f193c035740 1 --2- 172.21.15.2:0/2694406673 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55943eb874e0 0x55943eb88950 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f193c035740 1 --2- 172.21.15.2:0/2694406673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb76510 0x55943eb74010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f193c035740 1 --2- 172.21.15.2:0/2694406673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb6bc50 0x55943eb73ad0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f193c035740 1 --2- 172.21.15.2:0/2694406673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6b330 0x55943eb73590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f1917e00640 0 client.25105 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:12.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f193c035740 1 -- 172.21.15.2:0/2694406673 >> 172.21.15.2:0/2694406673 conn(0x55943eb678c0 msgr2=0x55943eb6d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:12.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f193c035740 1 -- 172.21.15.2:0/2694406673 shutdown_connections 2024-09-10T08:08:12.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.901+0000 7f193c035740 1 -- 172.21.15.2:0/2694406673 wait complete. 2024-09-10T08:08:12.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.902+0000 7f193c035740 20 client.25105 trim_cache size 0 max 16384 2024-09-10T08:08:12.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.920+0000 7f193c035740 1 Processor -- start 2024-09-10T08:08:12.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.920+0000 7f193c035740 1 -- start start 2024-09-10T08:08:12.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.920+0000 7f193c035740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6b330 0x55943eb73e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.920+0000 7f193c035740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6bc50 0x55943eb70810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.920+0000 7f193c035740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb76510 0x55943eb70d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.920+0000 7f193c035740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55943eb86c90 con 0x55943eb6bc50 2024-09-10T08:08:12.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.920+0000 7f193c035740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55943eb87360 con 0x55943eb6b330 2024-09-10T08:08:12.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.920+0000 7f193c035740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55943eb87a30 con 0x55943eb76510 2024-09-10T08:08:12.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.921+0000 7f192a000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6bc50 0x55943eb70810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.921+0000 7f192a000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6bc50 0x55943eb70810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41090/0 (socket says 172.21.15.2:41090) 2024-09-10T08:08:12.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.921+0000 7f192a000640 1 -- 172.21.15.2:0/4267855916 learned_addr learned my addr 172.21.15.2:0/4267855916 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:12.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.921+0000 7f1933400640 1 --2- 172.21.15.2:0/4267855916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6b330 0x55943eb73e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.921+0000 7f1933400640 1 --2- 172.21.15.2:0/4267855916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb76510 0x55943eb70d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.921+0000 7f192a000640 1 -- 172.21.15.2:0/4267855916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb76510 msgr2=0x55943eb70d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:12.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.921+0000 7f192a000640 1 --2- 172.21.15.2:0/4267855916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb76510 0x55943eb70d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.921+0000 7f192a000640 1 -- 172.21.15.2:0/4267855916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6b330 msgr2=0x55943eb73e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:12.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.921+0000 7f192a000640 1 --2- 172.21.15.2:0/4267855916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6b330 0x55943eb73e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.921+0000 7f192a000640 1 -- 172.21.15.2:0/4267855916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55943eb71ae0 con 0x55943eb6bc50 2024-09-10T08:08:12.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.921+0000 7f192a000640 1 --2- 172.21.15.2:0/4267855916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6bc50 0x55943eb70810 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f191c0097f0 tx=0x7f191c0123a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a06e5716d68b9d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:12.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.922+0000 7f1922a00640 1 -- 172.21.15.2:0/4267855916 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c010a20 con 0x55943eb6bc50 2024-09-10T08:08:12.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.922+0000 7f1922a00640 1 -- 172.21.15.2:0/4267855916 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f191c010bc0 con 0x55943eb6bc50 2024-09-10T08:08:12.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.922+0000 7f1922a00640 1 -- 172.21.15.2:0/4267855916 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c003ab0 con 0x55943eb6bc50 2024-09-10T08:08:12.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.922+0000 7f193c035740 1 -- 172.21.15.2:0/4267855916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6bc50 msgr2=0x55943eb70810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:12.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.922+0000 7f193c035740 1 --2- 172.21.15.2:0/4267855916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6bc50 0x55943eb70810 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f191c0097f0 tx=0x7f191c0123a0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.922+0000 7f193c035740 1 -- 172.21.15.2:0/4267855916 shutdown_connections 2024-09-10T08:08:12.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.922+0000 7f193c035740 1 --2- 172.21.15.2:0/4267855916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb76510 0x55943eb70d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.922+0000 7f193c035740 1 --2- 172.21.15.2:0/4267855916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb6bc50 0x55943eb70810 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.922+0000 7f193c035740 1 --2- 172.21.15.2:0/4267855916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6b330 0x55943eb73e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.922+0000 7f193c035740 1 -- 172.21.15.2:0/4267855916 >> 172.21.15.2:0/4267855916 conn(0x55943eb678c0 msgr2=0x55943eb6e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:12.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.923+0000 7f193c035740 1 -- 172.21.15.2:0/4267855916 shutdown_connections 2024-09-10T08:08:12.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.923+0000 7f193c035740 1 -- 172.21.15.2:0/4267855916 wait complete. 2024-09-10T08:08:12.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.923+0000 7f193c035740 1 -- start start 2024-09-10T08:08:12.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.923+0000 7f193c035740 1 Processor -- start 2024-09-10T08:08:12.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f193c035740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb6b330 0x55943eb91450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f193c035740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6bc50 0x55943eb8baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f193c035740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb76510 0x55943eb8c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f193c035740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55943eb887d0 con 0x55943eb76510 2024-09-10T08:08:12.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f193c035740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55943eb88100 con 0x55943eb6bc50 2024-09-10T08:08:12.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f193c035740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55943eb88ea0 con 0x55943eb6b330 2024-09-10T08:08:12.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f193c035740 10 client.0 _ll_register_callbacks cb 0x7f191c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:12.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f1933400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb76510 0x55943eb8c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f192a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6bc50 0x55943eb8baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f1933400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb6b330 0x55943eb91450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f192a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6bc50 0x55943eb8baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52330/0 (socket says 172.21.15.2:52330) 2024-09-10T08:08:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f192a000640 1 -- 172.21.15.2:0/3097103552 learned_addr learned my addr 172.21.15.2:0/3097103552 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f192a000640 1 -- 172.21.15.2:0/3097103552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb6b330 msgr2=0x55943eb91450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f192a000640 1 --2- 172.21.15.2:0/3097103552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb6b330 0x55943eb91450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f192a000640 1 -- 172.21.15.2:0/3097103552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb76510 msgr2=0x55943eb8c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f192a000640 1 --2- 172.21.15.2:0/3097103552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb76510 0x55943eb8c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.924+0000 7f192a000640 1 -- 172.21.15.2:0/3097103552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55943eb8c8a0 con 0x55943eb6bc50 2024-09-10T08:08:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.925+0000 7f192a000640 1 --2- 172.21.15.2:0/3097103552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6bc50 0x55943eb8baf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f191c000c00 tx=0x7f191c009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b216475ae10f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.925+0000 7f1917e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:08:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.925+0000 7f1917e00640 1 -- 172.21.15.2:0/3097103552 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c010a20 con 0x55943eb6bc50 2024-09-10T08:08:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.925+0000 7f193c035740 1 -- 172.21.15.2:0/3097103552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f191c010bc0 con 0x55943eb6bc50 2024-09-10T08:08:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.925+0000 7f1917e00640 1 -- 172.21.15.2:0/3097103552 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f191c0052e0 con 0x55943eb6bc50 2024-09-10T08:08:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.925+0000 7f1917e00640 1 -- 172.21.15.2:0/3097103552 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f191c0054e0 con 0x55943eb6bc50 2024-09-10T08:08:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.925+0000 7f193c035740 10 client.25136 fetch_fsmap learned FSMap version 25 2024-09-10T08:08:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.925+0000 7f193c035740 1 -- 172.21.15.2:0/3097103552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55943ebf2c90 con 0x55943eb6bc50 2024-09-10T08:08:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.925+0000 7f1917e00640 1 -- 172.21.15.2:0/3097103552 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f191c010bc0 con 0x55943eb6bc50 2024-09-10T08:08:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.926+0000 7f1917e00640 1 -- 172.21.15.2:0/3097103552 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f191c01c5e0 con 0x55943eb6bc50 2024-09-10T08:08:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.926+0000 7f193c035740 10 client.25136 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:08:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.926+0000 7f193c035740 10 client.25136 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:08:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.926+0000 7f193c035740 20 client.25136 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.926+0000 7f193c035740 1 --2- 172.21.15.2:0/3097103552 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55943ebf7500 0x55943ebf6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.926+0000 7f193c035740 4 client.25136 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:08:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.926+0000 7f193c035740 1 -- 172.21.15.2:0/3097103552 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "status"}) -- 0x55943eb68470 con 0x55943ebf7500 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.926+0000 7f1933400640 1 --2- 172.21.15.2:0/3097103552 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55943ebf7500 0x55943ebf6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f1933400640 1 --2- 172.21.15.2:0/3097103552 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55943ebf7500 0x55943ebf6210 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=226704b4a931a89 server_cookie=3a407a33552c6c70 in_seq=0 out_seq=0 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f1917e00640 10 client.25136 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f1917e00640 1 -- 172.21.15.2:0/3097103552 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55943eb68470 con 0x55943ebf7500 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f1917e00640 10 client.25136 handle_command_reply: tid=0 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f193c035740 1 client.25136 shutdown 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f193c035740 1 -- 172.21.15.2:0/3097103552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6bc50 msgr2=0x55943eb8baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f193c035740 1 --2- 172.21.15.2:0/3097103552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6bc50 0x55943eb8baf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f191c000c00 tx=0x7f191c009340 comp rx=0 tx=0).stop 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f193c035740 1 -- 172.21.15.2:0/3097103552 shutdown_connections 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f193c035740 1 --2- 172.21.15.2:0/3097103552 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55943ebf7500 0x55943ebf6210 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f193c035740 1 --2- 172.21.15.2:0/3097103552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55943eb76510 0x55943eb8c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f193c035740 1 --2- 172.21.15.2:0/3097103552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55943eb6bc50 0x55943eb8baf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f193c035740 1 --2- 172.21.15.2:0/3097103552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943eb6b330 0x55943eb91450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f1917e00640 0 client.25136 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f193c035740 1 -- 172.21.15.2:0/3097103552 >> 172.21.15.2:0/3097103552 conn(0x55943eb678c0 msgr2=0x55943eb6cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f193c035740 1 -- 172.21.15.2:0/3097103552 shutdown_connections 2024-09-10T08:08:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f193c035740 1 -- 172.21.15.2:0/3097103552 wait complete. 2024-09-10T08:08:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.927+0000 7f193c035740 20 client.25136 trim_cache size 0 max 16384 2024-09-10T08:08:12.931 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:08:12.931 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "26ec64b0-6f4a-11ef-bcea-c7b262605968", 2024-09-10T08:08:12.931 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-09-10T08:08:12.931 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24536, 2024-09-10T08:08:12.931 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-09-10T08:08:12.931 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-09-10T08:08:12.931 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-09-10T08:08:12.932 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 165.43881789700001, 2024-09-10T08:08:12.932 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 25, 2024-09-10T08:08:12.932 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 95, 2024-09-10T08:08:12.932 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 93, 2024-09-10T08:08:12.932 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 179.31738743700001 2024-09-10T08:08:12.932 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:08:12.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.928+0000 7f1939600640 1 -- 172.21.15.2:0/4089967617 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19180a2c00 msgr2=0x7f19180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.928+0000 7f1939600640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19180a2c00 0x7f19180a50c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f19240bbc30 tx=0x7f19240f5510 comp rx=0 tx=0).stop 2024-09-10T08:08:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.928+0000 7f1939600640 1 -- 172.21.15.2:0/4089967617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19340745c0 msgr2=0x7f193419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.928+0000 7f1939600640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19340745c0 0x7f193419a630 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f192402a9a0 tx=0x7f1924003b90 comp rx=0 tx=0).stop 2024-09-10T08:08:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.928+0000 7f1939600640 1 -- 172.21.15.2:0/4089967617 shutdown_connections 2024-09-10T08:08:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.928+0000 7f1939600640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f193410e430 0x7f193419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.928+0000 7f1939600640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19180a2c00 0x7f19180a50c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.928+0000 7f1939600640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1934074ee0 0x7f193419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.928+0000 7f1939600640 1 --2- 172.21.15.2:0/4089967617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19340745c0 0x7f193419a630 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.928+0000 7f1939600640 1 -- 172.21.15.2:0/4089967617 >> 172.21.15.2:0/4089967617 conn(0x7f19340fd8b0 msgr2=0x7f19340fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.928+0000 7f1939600640 1 -- 172.21.15.2:0/4089967617 shutdown_connections 2024-09-10T08:08:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:12.928+0000 7f1939600640 1 -- 172.21.15.2:0/4089967617 wait complete. 2024-09-10T08:08:13.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:12 smithi155 ceph-mon[35405]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:08:13.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:12 smithi107 ceph-mon[35048]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:08:13.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:12 smithi002 ceph-mon[30563]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:08:13.741 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T08:08:13.751 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T08:08:13.751 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T08:08:13.752 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T08:08:13.752 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T08:08:13.766 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T08:08:13.766 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-10T08:08:13.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:08:13.766 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-09-10T08:08:13.802 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T08:08:13.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:08:13.803 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:08:13.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:08:13.902 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2024-09-10T08:08:14.243 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-10T08:08:14.243 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T08:08:14.243 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T08:08:14.244 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T08:08:14.626 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:14 smithi002 ceph-mon[30563]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:08:15.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:14 smithi155 ceph-mon[35405]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:08:15.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:14 smithi107 ceph-mon[35048]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:08:15.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.220+0000 7f4b52e00640 1 -- 172.21.15.2:0/3791378134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b4c10e260 msgr2=0x7f4b4c110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:15.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.220+0000 7f4b52e00640 1 --2- 172.21.15.2:0/3791378134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b4c10e260 0x7f4b4c110650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009900 tx=0x7f4b4402a480 comp rx=0 tx=0).stop 2024-09-10T08:08:15.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.220+0000 7f4b52e00640 1 -- 172.21.15.2:0/3791378134 shutdown_connections 2024-09-10T08:08:15.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.220+0000 7f4b52e00640 1 --2- 172.21.15.2:0/3791378134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b4c10e260 0x7f4b4c110650 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:15.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.220+0000 7f4b52e00640 1 --2- 172.21.15.2:0/3791378134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b4c074ee0 0x7f4b4c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:15.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.220+0000 7f4b52e00640 1 --2- 172.21.15.2:0/3791378134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b4c0745c0 0x7f4b4c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:15.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.220+0000 7f4b52e00640 1 -- 172.21.15.2:0/3791378134 >> 172.21.15.2:0/3791378134 conn(0x7f4b4c0fd740 msgr2=0x7f4b4c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:15.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.220+0000 7f4b52e00640 1 -- 172.21.15.2:0/3791378134 shutdown_connections 2024-09-10T08:08:15.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.220+0000 7f4b52e00640 1 -- 172.21.15.2:0/3791378134 wait complete. 2024-09-10T08:08:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.221+0000 7f4b52e00640 1 Processor -- start 2024-09-10T08:08:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.222+0000 7f4b52e00640 1 -- start start 2024-09-10T08:08:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.222+0000 7f4b52e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b4c0745c0 0x7f4b4c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.222+0000 7f4b52e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b4c074ee0 0x7f4b4c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.222+0000 7f4b52e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b4c10e260 0x7f4b4c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.222+0000 7f4b52e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c113f70 con 0x7f4b4c10e260 2024-09-10T08:08:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.222+0000 7f4b52e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c113df0 con 0x7f4b4c0745c0 2024-09-10T08:08:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.222+0000 7f4b52e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1140f0 con 0x7f4b4c074ee0 2024-09-10T08:08:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.222+0000 7f4b52400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b4c10e260 0x7f4b4c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.222+0000 7f4b51a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b4c0745c0 0x7f4b4c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.222+0000 7f4b51a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b4c0745c0 0x7f4b4c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52354/0 (socket says 172.21.15.2:52354) 2024-09-10T08:08:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.222+0000 7f4b51a00640 1 -- 172.21.15.2:0/461787607 learned_addr learned my addr 172.21.15.2:0/461787607 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.222+0000 7f4b52400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b4c10e260 0x7f4b4c19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41110/0 (socket says 172.21.15.2:41110) 2024-09-10T08:08:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.222+0000 7f4b51000640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b4c074ee0 0x7f4b4c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.223+0000 7f4b51a00640 1 -- 172.21.15.2:0/461787607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b4c074ee0 msgr2=0x7f4b4c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.223+0000 7f4b51a00640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b4c074ee0 0x7f4b4c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.223+0000 7f4b51a00640 1 -- 172.21.15.2:0/461787607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b4c10e260 msgr2=0x7f4b4c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.223+0000 7f4b51a00640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b4c10e260 0x7f4b4c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.223+0000 7f4b51a00640 1 -- 172.21.15.2:0/461787607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c19f400 con 0x7f4b4c0745c0 2024-09-10T08:08:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.223+0000 7f4b52400640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b4c10e260 0x7f4b4c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.223+0000 7f4b51a00640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b4c0745c0 0x7f4b4c19a450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000ebc0 tx=0x7f4b4000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:15.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.223+0000 7f4b4a000640 1 -- 172.21.15.2:0/461787607 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40010630 con 0x7f4b4c0745c0 2024-09-10T08:08:15.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.223+0000 7f4b4a000640 1 -- 172.21.15.2:0/461787607 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b40010cb0 con 0x7f4b4c0745c0 2024-09-10T08:08:15.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.223+0000 7f4b52e00640 1 -- 172.21.15.2:0/461787607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c19f6f0 con 0x7f4b4c0745c0 2024-09-10T08:08:15.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.223+0000 7f4b4a000640 1 -- 172.21.15.2:0/461787607 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40018bd0 con 0x7f4b4c0745c0 2024-09-10T08:08:15.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.224+0000 7f4b52e00640 1 -- 172.21.15.2:0/461787607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c1a6fd0 con 0x7f4b4c0745c0 2024-09-10T08:08:15.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.225+0000 7f4b52e00640 1 -- 172.21.15.2:0/461787607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b4c0749a0 con 0x7f4b4c0745c0 2024-09-10T08:08:15.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.227+0000 7f4b4a000640 1 -- 172.21.15.2:0/461787607 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4b400107d0 con 0x7f4b4c0745c0 2024-09-10T08:08:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.228+0000 7f4b4a000640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b340a2c00 0x7f4b340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.228+0000 7f4b4a000640 1 -- 172.21.15.2:0/461787607 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4b400cfa30 con 0x7f4b4c0745c0 2024-09-10T08:08:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.228+0000 7f4b51000640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b340a2c00 0x7f4b340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.229+0000 7f4b51000640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b340a2c00 0x7f4b340a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c10f580 tx=0x7f4b38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:15.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.232+0000 7f4b4a000640 1 -- 172.21.15.2:0/461787607 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b4009bef0 con 0x7f4b4c0745c0 2024-09-10T08:08:15.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.452+0000 7f4b52e00640 1 -- 172.21.15.2:0/461787607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4b4c02d7c0 con 0x7f4b4c0745c0 2024-09-10T08:08:15.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.453+0000 7f4b4a000640 1 -- 172.21.15.2:0/461787607 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7f4b400a01f0 con 0x7f4b4c0745c0 2024-09-10T08:08:15.453 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T08:08:15.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.455+0000 7f4b52e00640 1 -- 172.21.15.2:0/461787607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b340a2c00 msgr2=0x7f4b340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:15.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.455+0000 7f4b52e00640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b340a2c00 0x7f4b340a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c10f580 tx=0x7f4b38005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:15.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.455+0000 7f4b52e00640 1 -- 172.21.15.2:0/461787607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b4c0745c0 msgr2=0x7f4b4c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:15.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.455+0000 7f4b52e00640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b4c0745c0 0x7f4b4c19a450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000ebc0 tx=0x7f4b4000c6a0 comp rx=0 tx=0).stop 2024-09-10T08:08:15.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.455+0000 7f4b52e00640 1 -- 172.21.15.2:0/461787607 shutdown_connections 2024-09-10T08:08:15.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.455+0000 7f4b52e00640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b4c10e260 0x7f4b4c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:15.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.455+0000 7f4b52e00640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b340a2c00 0x7f4b340a50c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:15.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.455+0000 7f4b52e00640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b4c074ee0 0x7f4b4c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:15.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.455+0000 7f4b52e00640 1 --2- 172.21.15.2:0/461787607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b4c0745c0 0x7f4b4c19a450 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:15.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.455+0000 7f4b52e00640 1 -- 172.21.15.2:0/461787607 >> 172.21.15.2:0/461787607 conn(0x7f4b4c0fd740 msgr2=0x7f4b4c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:15.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.456+0000 7f4b52e00640 1 -- 172.21.15.2:0/461787607 shutdown_connections 2024-09-10T08:08:15.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:15.456+0000 7f4b52e00640 1 -- 172.21.15.2:0/461787607 wait complete. 2024-09-10T08:08:16.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:15 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/461787607' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:08:16.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:15 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/461787607' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:08:16.187 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:15 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/461787607' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:08:16.224 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T08:08:16.224 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T08:08:16.225 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2024-09-10T08:08:16.225 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T08:08:16.225 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T08:08:16.225 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T08:08:16.225 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T08:08:16.225 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T08:08:16.225 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T08:08:16.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:08:16.226 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2024-09-10T08:08:16.262 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T08:08:16.263 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T08:08:16.263 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T08:08:16.263 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:08:16.263 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2024-09-10T08:08:16.263 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:08:16.263 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:08:16.263 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:08:16.263 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f0 2024-09-10T08:08:16.263 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:08:16.263 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:08:16.263 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f1 2024-09-10T08:08:16.264 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T08:08:16.264 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:08:16.264 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f0 2024-09-10T08:08:16.264 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T08:08:16.264 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 42029sec preferred_lft 42029sec 2024-09-10T08:08:16.264 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link 2024-09-10T08:08:16.264 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:08:16.264 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:08:16.264 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:08:16.264 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f1 2024-09-10T08:08:16.264 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T08:08:16.265 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:08:16.265 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-10T08:08:16.265 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add name ceph-brx type bridge 2024-09-10T08:08:16.265 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr flush dev ceph-brx 2024-09-10T08:08:16.265 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set ceph-brx up 2024-09-10T08:08:16.265 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T08:08:16.265 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-10T08:08:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:08:16.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:08:16.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:08:16.491 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T08:08:16.546 INFO:teuthology.orchestra.run.smithi002.stdout:1 2024-09-10T08:08:16.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:08:16.550 DEBUG:teuthology.orchestra.run.smithi002:> route 2024-09-10T08:08:16.587 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2024-09-10T08:08:16.587 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T08:08:16.587 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T08:08:16.588 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T08:08:16.588 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T08:08:16.588 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:08:16.588 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-10T08:08:16.588 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T08:08:16.589 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T08:08:16.589 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T08:08:16.589 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-10T08:08:16.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:08:16.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:08:16.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:08:16.788 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-09-10T08:08:16.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:08:16.824 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2024-09-10T08:08:16.900 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:08:16.900 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-10T08:08:16.900 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T08:08:16.900 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T08:08:16.900 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-10T08:08:17.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:16 smithi002 ceph-mon[30563]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:08:17.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:08:17.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:08:17.081 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T08:08:17.081 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:08:17.082 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-10T08:08:17.082 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T08:08:17.082 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T08:08:17.082 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T08:08:17.082 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T08:08:17.082 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T08:08:17.082 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-10T08:08:17.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:16 smithi155 ceph-mon[35405]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:08:17.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:08:17.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:16 smithi107 ceph-mon[35048]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:08:17.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:08:17.302 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:08:17.302 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-10T08:08:17.302 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2024-09-10T08:08:17.303 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T08:08:17.303 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-10T08:08:17.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:08:17.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:08:17.470 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T08:08:17.470 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T08:08:17.505 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T08:08:17.505 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T08:08:17.506 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T08:08:17.581 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-10T08:08:17.581 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-10T08:08:17.581 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/d78be939-d6c3-4b42-854d-10cdcae584fb 2024-09-10T08:08:17.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:08:17.582 DEBUG:teuthology.orchestra.run.smithi002:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/d78be939-d6c3-4b42-854d-10cdcae584fb /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-09-10T08:08:17.711 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-09-10T08:08:17.711 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-09-10T08:08:18.353 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-09-10T08:08:18.353 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-09-10T08:08:18.353 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:6789,172.21.15.107:6789,172.21.15.155:6789" 2024-09-10T08:08:18.353 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: trying mount with new device syntax: 0@26ec64b0-6f4a-11ef-bcea-c7b262605968.cephfs=/volumes/qa/sv_1/d78be939-d6c3-4b42-854d-10cdcae584fb 2024-09-10T08:08:18.353 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.2:6789/172.21.15.107:6789/172.21.15.155:6789" will pass to kernel 2024-09-10T08:08:18.356 INFO:teuthology.orchestra.run.smithi002.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T08:08:18.356 INFO:teuthology.orchestra.run.smithi002.stdout: You just mounted a file system that supports labels which does not 2024-09-10T08:08:18.356 INFO:teuthology.orchestra.run.smithi002.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T08:08:18.356 INFO:teuthology.orchestra.run.smithi002.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T08:08:18.356 INFO:teuthology.orchestra.run.smithi002.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T08:08:18.359 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T08:08:18.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:08:18.360 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T08:08:18.419 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T08:08:18.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T08:08:18.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:08:18.512 INFO:teuthology.orchestra.run.smithi002.stdout:client25120 2024-09-10T08:08:18.512 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T08:08:18.512 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T08:08:18.549 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T08:08:18.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T08:08:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:08:18.679 INFO:teuthology.orchestra.run.smithi002.stdout:client25120 2024-09-10T08:08:18.680 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T08:08:18.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T08:08:18.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:08:18.773 INFO:teuthology.orchestra.run.smithi002.stdout:client25120 2024-09-10T08:08:18.774 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968.client25120/status') 2024-09-10T08:08:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:08:18.877 INFO:teuthology.orchestra.run.smithi002.stdout:instance: client.25120 (3)172.21.15.2:0/2395163174 2024-09-10T08:08:18.877 INFO:teuthology.orchestra.run.smithi002.stdout:blocklisted: false 2024-09-10T08:08:18.878 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2024-09-10T08:08:18.878 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2024-09-10T08:08:18.878 INFO:teuthology.orchestra.run.smithi002.stderr:70 bytes copied, 5.7805e-05 s, 1.2 MB/s 2024-09-10T08:08:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:08:18.888 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T08:08:18.900 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T08:08:18.900 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-09-10T08:08:18.901 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-10T08:08:19.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.059+0000 7fa176200640 1 -- 172.21.15.2:0/3522626336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa170074bd0 msgr2=0x7fa170075030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:19.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.059+0000 7fa176200640 1 --2- 172.21.15.2:0/3522626336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa170074bd0 0x7fa170075030 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009900 tx=0x7fa15c02a480 comp rx=0 tx=0).stop 2024-09-10T08:08:19.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.059+0000 7fa176200640 1 -- 172.21.15.2:0/3522626336 shutdown_connections 2024-09-10T08:08:19.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.059+0000 7fa176200640 1 --2- 172.21.15.2:0/3522626336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa170109a10 0x7fa17010bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:19.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.059+0000 7fa176200640 1 --2- 172.21.15.2:0/3522626336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa170074bd0 0x7fa170075030 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:19.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.059+0000 7fa176200640 1 --2- 172.21.15.2:0/3522626336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1700761b0 0x7fa170074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:19.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.059+0000 7fa176200640 1 -- 172.21.15.2:0/3522626336 >> 172.21.15.2:0/3522626336 conn(0x7fa1700fd2e0 msgr2=0x7fa1700ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:19.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.059+0000 7fa176200640 1 -- 172.21.15.2:0/3522626336 shutdown_connections 2024-09-10T08:08:19.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.059+0000 7fa176200640 1 -- 172.21.15.2:0/3522626336 wait complete. 2024-09-10T08:08:19.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.060+0000 7fa176200640 1 Processor -- start 2024-09-10T08:08:19.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.060+0000 7fa176200640 1 -- start start 2024-09-10T08:08:19.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.060+0000 7fa176200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa170074bd0 0x7fa17019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:19.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.060+0000 7fa176200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1700761b0 0x7fa17019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:19.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa176200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa170109a10 0x7fa1701a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:19.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa176200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa17010fa30 con 0x7fa1700761b0 2024-09-10T08:08:19.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa176200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa17010f8b0 con 0x7fa170109a10 2024-09-10T08:08:19.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa176200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa17010fbb0 con 0x7fa170074bd0 2024-09-10T08:08:19.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa16ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1700761b0 0x7fa17019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:19.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa16f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa170074bd0 0x7fa17019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:19.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa16ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1700761b0 0x7fa17019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39816/0 (socket says 172.21.15.2:39816) 2024-09-10T08:08:19.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa16ea00640 1 -- 172.21.15.2:0/2211771052 learned_addr learned my addr 172.21.15.2:0/2211771052 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:19.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa16fe00640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa170109a10 0x7fa1701a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:19.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa16f400640 1 -- 172.21.15.2:0/2211771052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa170109a10 msgr2=0x7fa1701a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:19.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa16f400640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa170109a10 0x7fa1701a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:19.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa16f400640 1 -- 172.21.15.2:0/2211771052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1700761b0 msgr2=0x7fa17019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:19.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa16f400640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1700761b0 0x7fa17019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:19.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa16f400640 1 -- 172.21.15.2:0/2211771052 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1701a1c20 con 0x7fa170074bd0 2024-09-10T08:08:19.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.061+0000 7fa16ea00640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1700761b0 0x7fa17019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:19.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.062+0000 7fa16f400640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa170074bd0 0x7fa17019a000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa16000e9f0 tx=0x7fa16000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:19.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.062+0000 7fa167e00640 1 -- 172.21.15.2:0/2211771052 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16000ccc0 con 0x7fa170074bd0 2024-09-10T08:08:19.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.062+0000 7fa167e00640 1 -- 172.21.15.2:0/2211771052 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa160010910 con 0x7fa170074bd0 2024-09-10T08:08:19.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.062+0000 7fa176200640 1 -- 172.21.15.2:0/2211771052 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1701a1f10 con 0x7fa170074bd0 2024-09-10T08:08:19.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.062+0000 7fa167e00640 1 -- 172.21.15.2:0/2211771052 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa160018960 con 0x7fa170074bd0 2024-09-10T08:08:19.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.062+0000 7fa176200640 1 -- 172.21.15.2:0/2211771052 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1701a2370 con 0x7fa170074bd0 2024-09-10T08:08:19.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.063+0000 7fa176200640 1 -- 172.21.15.2:0/2211771052 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1701087d0 con 0x7fa170074bd0 2024-09-10T08:08:19.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.065+0000 7fa167e00640 1 -- 172.21.15.2:0/2211771052 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa160010ac0 con 0x7fa170074bd0 2024-09-10T08:08:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.066+0000 7fa167e00640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1540a2c30 0x7fa1540a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.067+0000 7fa167e00640 1 -- 172.21.15.2:0/2211771052 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa1600cfb80 con 0x7fa170074bd0 2024-09-10T08:08:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.067+0000 7fa16ea00640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1540a2c30 0x7fa1540a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.067+0000 7fa16ea00640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1540a2c30 0x7fa1540a50f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa17019b510 tx=0x7fa15c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:19.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.069+0000 7fa167e00640 1 -- 172.21.15.2:0/2211771052 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa16009c040 con 0x7fa170074bd0 2024-09-10T08:08:19.114 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:18 smithi002 ceph-mon[30563]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T08:08:19.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:18 smithi155 ceph-mon[35405]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T08:08:19.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:18 smithi107 ceph-mon[35048]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-10T08:08:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.292+0000 7fa176200640 1 -- 172.21.15.2:0/2211771052 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fa17002d7c0 con 0x7fa170074bd0 2024-09-10T08:08:19.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.867+0000 7fa167e00640 1 -- 172.21.15.2:0/2211771052 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7fa1600a0340 con 0x7fa170074bd0 2024-09-10T08:08:19.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.869+0000 7fa176200640 1 -- 172.21.15.2:0/2211771052 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1540a2c30 msgr2=0x7fa1540a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:19.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.869+0000 7fa176200640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1540a2c30 0x7fa1540a50f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa17019b510 tx=0x7fa15c0315b0 comp rx=0 tx=0).stop 2024-09-10T08:08:19.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.869+0000 7fa176200640 1 -- 172.21.15.2:0/2211771052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa170074bd0 msgr2=0x7fa17019a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:19.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.869+0000 7fa176200640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa170074bd0 0x7fa17019a000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa16000e9f0 tx=0x7fa16000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:08:19.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.869+0000 7fa176200640 1 -- 172.21.15.2:0/2211771052 shutdown_connections 2024-09-10T08:08:19.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.869+0000 7fa176200640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa170109a10 0x7fa1701a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:19.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.869+0000 7fa176200640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1700761b0 0x7fa17019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:19.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.869+0000 7fa176200640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1540a2c30 0x7fa1540a50f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:19.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.869+0000 7fa176200640 1 --2- 172.21.15.2:0/2211771052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa170074bd0 0x7fa17019a000 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:19.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.869+0000 7fa176200640 1 -- 172.21.15.2:0/2211771052 >> 172.21.15.2:0/2211771052 conn(0x7fa1700fd2e0 msgr2=0x7fa1701059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:19.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.869+0000 7fa176200640 1 -- 172.21.15.2:0/2211771052 shutdown_connections 2024-09-10T08:08:19.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:19.870+0000 7fa176200640 1 -- 172.21.15.2:0/2211771052 wait complete. 2024-09-10T08:08:19.886 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-10T08:08:20.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.047+0000 7fa5cec00640 1 -- 172.21.15.2:0/3741093632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8105910 msgr2=0x7fa5c810c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:20.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.047+0000 7fa5cec00640 1 --2- 172.21.15.2:0/3741093632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8105910 0x7fa5c810c1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009ca0 tx=0x7fa5c002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.047+0000 7fa5cec00640 1 -- 172.21.15.2:0/3741093632 shutdown_connections 2024-09-10T08:08:20.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.047+0000 7fa5cec00640 1 --2- 172.21.15.2:0/3741093632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8105910 0x7fa5c810c1a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.047+0000 7fa5cec00640 1 --2- 172.21.15.2:0/3741093632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8104f50 0x7fa5c81053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.047+0000 7fa5cec00640 1 --2- 172.21.15.2:0/3741093632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8103d60 0x7fa5c8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.047+0000 7fa5cec00640 1 -- 172.21.15.2:0/3741093632 >> 172.21.15.2:0/3741093632 conn(0x7fa5c80ff4e0 msgr2=0x7fa5c8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:20.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.047+0000 7fa5cec00640 1 -- 172.21.15.2:0/3741093632 shutdown_connections 2024-09-10T08:08:20.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.047+0000 7fa5cec00640 1 -- 172.21.15.2:0/3741093632 wait complete. 2024-09-10T08:08:20.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.048+0000 7fa5cec00640 1 Processor -- start 2024-09-10T08:08:20.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.048+0000 7fa5cec00640 1 -- start start 2024-09-10T08:08:20.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.048+0000 7fa5cec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8103d60 0x7fa5c8199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:20.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.048+0000 7fa5cec00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8104f50 0x7fa5c819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:20.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.048+0000 7fa5cec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8105910 0x7fa5c81a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:20.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.048+0000 7fa5cec00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c810f9c0 con 0x7fa5c8104f50 2024-09-10T08:08:20.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.048+0000 7fa5cec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c810f840 con 0x7fa5c8105910 2024-09-10T08:08:20.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.048+0000 7fa5cec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c810fb40 con 0x7fa5c8103d60 2024-09-10T08:08:20.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.048+0000 7fa5c7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8104f50 0x7fa5c819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:20.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.048+0000 7fa5c7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8104f50 0x7fa5c819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39840/0 (socket says 172.21.15.2:39840) 2024-09-10T08:08:20.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.049+0000 7fa5c7400640 1 -- 172.21.15.2:0/2921563080 learned_addr learned my addr 172.21.15.2:0/2921563080 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:20.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.049+0000 7fa5ccc00640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8105910 0x7fa5c81a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:20.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.049+0000 7fa5c7e00640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8103d60 0x7fa5c8199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:20.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.049+0000 7fa5c7400640 1 -- 172.21.15.2:0/2921563080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8103d60 msgr2=0x7fa5c8199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:20.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.049+0000 7fa5c7400640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8103d60 0x7fa5c8199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.049+0000 7fa5c7400640 1 -- 172.21.15.2:0/2921563080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8105910 msgr2=0x7fa5c81a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:20.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.049+0000 7fa5c7400640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8105910 0x7fa5c81a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.049+0000 7fa5c7400640 1 -- 172.21.15.2:0/2921563080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c81a1b40 con 0x7fa5c8104f50 2024-09-10T08:08:20.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.049+0000 7fa5ccc00640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8105910 0x7fa5c81a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:20.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.049+0000 7fa5c7400640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8104f50 0x7fa5c819a460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400d950 tx=0x7fa5b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:20.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.050+0000 7fa5c4c00640 1 -- 172.21.15.2:0/2921563080 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4017030 con 0x7fa5c8104f50 2024-09-10T08:08:20.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.050+0000 7fa5c4c00640 1 -- 172.21.15.2:0/2921563080 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4004020 con 0x7fa5c8104f50 2024-09-10T08:08:20.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.050+0000 7fa5cec00640 1 -- 172.21.15.2:0/2921563080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c81a1e30 con 0x7fa5c8104f50 2024-09-10T08:08:20.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.050+0000 7fa5c4c00640 1 -- 172.21.15.2:0/2921563080 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40047a0 con 0x7fa5c8104f50 2024-09-10T08:08:20.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.050+0000 7fa5cec00640 1 -- 172.21.15.2:0/2921563080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c81a22f0 con 0x7fa5c8104f50 2024-09-10T08:08:20.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.051+0000 7fa5cec00640 1 -- 172.21.15.2:0/2921563080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c810b020 con 0x7fa5c8104f50 2024-09-10T08:08:20.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.054+0000 7fa5c4c00640 1 -- 172.21.15.2:0/2921563080 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa5b40041d0 con 0x7fa5c8104f50 2024-09-10T08:08:20.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.055+0000 7fa5c4c00640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa5ac0a2c30 0x7fa5ac0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:20.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.055+0000 7fa5c4c00640 1 -- 172.21.15.2:0/2921563080 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa5b40ced90 con 0x7fa5c8104f50 2024-09-10T08:08:20.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.055+0000 7fa5c7e00640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa5ac0a2c30 0x7fa5ac0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:20.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.056+0000 7fa5c7e00640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa5ac0a2c30 0x7fa5ac0a50f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80095f0 tx=0x7fa5b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:20.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.063+0000 7fa5c4c00640 1 -- 172.21.15.2:0/2921563080 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5b409b8a0 con 0x7fa5c8104f50 2024-09-10T08:08:20.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:19 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2211771052' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T08:08:20.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:19 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T08:08:20.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:19 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2211771052' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T08:08:20.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:19 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T08:08:20.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:19 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2211771052' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T08:08:20.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:19 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T08:08:20.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.281+0000 7fa5cec00640 1 -- 172.21.15.2:0/2921563080 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fa5c80626d0 con 0x7fa5ac0a2c30 2024-09-10T08:08:20.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.303+0000 7fa5c4c00640 1 -- 172.21.15.2:0/2921563080 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fa5c80626d0 con 0x7fa5ac0a2c30 2024-09-10T08:08:20.303 INFO:teuthology.orchestra.run.smithi002.stdout:{} 2024-09-10T08:08:20.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.306+0000 7fa5cec00640 1 -- 172.21.15.2:0/2921563080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa5ac0a2c30 msgr2=0x7fa5ac0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:20.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.306+0000 7fa5cec00640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa5ac0a2c30 0x7fa5ac0a50f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80095f0 tx=0x7fa5b8005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:20.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.306+0000 7fa5cec00640 1 -- 172.21.15.2:0/2921563080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8104f50 msgr2=0x7fa5c819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:20.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.306+0000 7fa5cec00640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8104f50 0x7fa5c819a460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400d950 tx=0x7fa5b400de10 comp rx=0 tx=0).stop 2024-09-10T08:08:20.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.306+0000 7fa5cec00640 1 -- 172.21.15.2:0/2921563080 shutdown_connections 2024-09-10T08:08:20.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.306+0000 7fa5cec00640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8105910 0x7fa5c81a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.306+0000 7fa5cec00640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8104f50 0x7fa5c819a460 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.306+0000 7fa5cec00640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa5ac0a2c30 0x7fa5ac0a50f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.306+0000 7fa5cec00640 1 --2- 172.21.15.2:0/2921563080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8103d60 0x7fa5c8199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.306+0000 7fa5cec00640 1 -- 172.21.15.2:0/2921563080 >> 172.21.15.2:0/2921563080 conn(0x7fa5c80ff4e0 msgr2=0x7fa5c81081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:20.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.306+0000 7fa5cec00640 1 -- 172.21.15.2:0/2921563080 shutdown_connections 2024-09-10T08:08:20.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.306+0000 7fa5cec00640 1 -- 172.21.15.2:0/2921563080 wait complete. 2024-09-10T08:08:20.322 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T08:08:20.336 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-10T08:08:20.348 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T08:08:20.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.507+0000 7f297f200640 1 -- 172.21.15.2:0/1247697797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29801023e0 msgr2=0x7f29801047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:20.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.507+0000 7f297f200640 1 --2- 172.21.15.2:0/1247697797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29801023e0 0x7f29801047d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2970009900 tx=0x7f297002a410 comp rx=0 tx=0).stop 2024-09-10T08:08:20.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.507+0000 7f297f200640 1 -- 172.21.15.2:0/1247697797 shutdown_connections 2024-09-10T08:08:20.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.507+0000 7f297f200640 1 --2- 172.21.15.2:0/1247697797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2980107640 0x7f2980109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.507+0000 7f297f200640 1 --2- 172.21.15.2:0/1247697797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2980104d10 0x7f2980107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.507+0000 7f297f200640 1 --2- 172.21.15.2:0/1247697797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29801023e0 0x7f29801047d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.507+0000 7f297f200640 1 -- 172.21.15.2:0/1247697797 >> 172.21.15.2:0/1247697797 conn(0x7f29800fc000 msgr2=0x7f29800fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:20.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.507+0000 7f297f200640 1 -- 172.21.15.2:0/1247697797 shutdown_connections 2024-09-10T08:08:20.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.507+0000 7f297f200640 1 -- 172.21.15.2:0/1247697797 wait complete. 2024-09-10T08:08:20.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.508+0000 7f297f200640 1 Processor -- start 2024-09-10T08:08:20.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.508+0000 7f297f200640 1 -- start start 2024-09-10T08:08:20.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.508+0000 7f297f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29801023e0 0x7f2980197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:20.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2980104d10 0x7f2980198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:20.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297f200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2980107640 0x7f298019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:20.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297f200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f298010d650 con 0x7f2980107640 2024-09-10T08:08:20.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f298010d4d0 con 0x7f29801023e0 2024-09-10T08:08:20.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f298010d7d0 con 0x7f2980104d10 2024-09-10T08:08:20.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2980107640 0x7f298019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:20.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29801023e0 0x7f2980197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:20.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2980107640 0x7f298019f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39886/0 (socket says 172.21.15.2:39886) 2024-09-10T08:08:20.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297e800640 1 -- 172.21.15.2:0/3516294450 learned_addr learned my addr 172.21.15.2:0/3516294450 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:20.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f2975400640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2980104d10 0x7f2980198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:20.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297de00640 1 -- 172.21.15.2:0/3516294450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2980104d10 msgr2=0x7f2980198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:20.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297de00640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2980104d10 0x7f2980198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297de00640 1 -- 172.21.15.2:0/3516294450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2980107640 msgr2=0x7f298019f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:20.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297de00640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2980107640 0x7f298019f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297de00640 1 -- 172.21.15.2:0/3516294450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298019f910 con 0x7f29801023e0 2024-09-10T08:08:20.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.509+0000 7f297e800640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2980107640 0x7f298019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:20.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.510+0000 7f297de00640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29801023e0 0x7f2980197d00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2970031440 tx=0x7f2970004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:20.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.510+0000 7f2977400640 1 -- 172.21.15.2:0/3516294450 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297003f070 con 0x7f29801023e0 2024-09-10T08:08:20.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.510+0000 7f297f200640 1 -- 172.21.15.2:0/3516294450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298019fba0 con 0x7f29801023e0 2024-09-10T08:08:20.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.510+0000 7f2977400640 1 -- 172.21.15.2:0/3516294450 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2970003cd0 con 0x7f29801023e0 2024-09-10T08:08:20.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.510+0000 7f2977400640 1 -- 172.21.15.2:0/3516294450 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970035070 con 0x7f29801023e0 2024-09-10T08:08:20.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.510+0000 7f297f200640 1 -- 172.21.15.2:0/3516294450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29801a0060 con 0x7f29801023e0 2024-09-10T08:08:20.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.511+0000 7f297f200640 1 -- 172.21.15.2:0/3516294450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29800fdaf0 con 0x7f29801023e0 2024-09-10T08:08:20.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.513+0000 7f2977400640 1 -- 172.21.15.2:0/3516294450 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f297003b4f0 con 0x7f29801023e0 2024-09-10T08:08:20.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.514+0000 7f2977400640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f29640a2b90 0x7f29640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:20.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.515+0000 7f2977400640 1 -- 172.21.15.2:0/3516294450 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f29700efe30 con 0x7f29801023e0 2024-09-10T08:08:20.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.515+0000 7f2975400640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f29640a2b90 0x7f29640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:20.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.515+0000 7f2975400640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f29640a2b90 0x7f29640a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2980069ee0 tx=0x7f2968005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:20.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.517+0000 7f2977400640 1 -- 172.21.15.2:0/3516294450 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29700bc2f0 con 0x7f29801023e0 2024-09-10T08:08:20.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.707+0000 7f297f200640 1 -- 172.21.15.2:0/3516294450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f298002d990 con 0x7f29801023e0 2024-09-10T08:08:20.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.708+0000 7f2977400640 1 -- 172.21.15.2:0/3516294450 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24887 (secure 0 0 0) 0x7f29700c05f0 con 0x7f29801023e0 2024-09-10T08:08:20.708 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:20.709 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":95,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:07:05.504273+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:08:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.711+0000 7f297f200640 1 -- 172.21.15.2:0/3516294450 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f29640a2b90 msgr2=0x7f29640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.711+0000 7f297f200640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f29640a2b90 0x7f29640a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2980069ee0 tx=0x7f2968005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.711+0000 7f297f200640 1 -- 172.21.15.2:0/3516294450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29801023e0 msgr2=0x7f2980197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.711+0000 7f297f200640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29801023e0 0x7f2980197d00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2970031440 tx=0x7f2970004270 comp rx=0 tx=0).stop 2024-09-10T08:08:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.711+0000 7f297f200640 1 -- 172.21.15.2:0/3516294450 shutdown_connections 2024-09-10T08:08:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.711+0000 7f297f200640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2980107640 0x7f298019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.711+0000 7f297f200640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f29640a2b90 0x7f29640a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.711+0000 7f297f200640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2980104d10 0x7f2980198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.711+0000 7f297f200640 1 --2- 172.21.15.2:0/3516294450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29801023e0 0x7f2980197d00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.711+0000 7f297f200640 1 -- 172.21.15.2:0/3516294450 >> 172.21.15.2:0/3516294450 conn(0x7f29800fc000 msgr2=0x7f2980102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.711+0000 7f297f200640 1 -- 172.21.15.2:0/3516294450 shutdown_connections 2024-09-10T08:08:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.711+0000 7f297f200640 1 -- 172.21.15.2:0/3516294450 wait complete. 2024-09-10T08:08:20.724 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:08:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T08:08:20.287+0000 7f6348400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:08:20.724 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:08:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T08:08:20.287+0000 7f6348400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:08:20.725 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:08:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T08:08:20.287+0000 7f6348400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:08:20.725 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:08:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T08:08:20.287+0000 7f6348400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:08:20.725 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:08:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T08:08:20.287+0000 7f6348400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:08:20.727 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T08:00:26.170464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T08:05:21.042136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T08:05:24.120081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T08:05:34.042103+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T08:08:20.727 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T08:08:20.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.890+0000 7fd551a00640 1 -- 172.21.15.2:0/2059885230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd54c102cf0 msgr2=0x7fd54c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.890+0000 7fd551a00640 1 --2- 172.21.15.2:0/2059885230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd54c102cf0 0x7fd54c103170 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd538009900 tx=0x7fd53802a410 comp rx=0 tx=0).stop 2024-09-10T08:08:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.890+0000 7fd551a00640 1 -- 172.21.15.2:0/2059885230 shutdown_connections 2024-09-10T08:08:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.890+0000 7fd551a00640 1 --2- 172.21.15.2:0/2059885230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd54c1036b0 0x7fd54c109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.890+0000 7fd551a00640 1 --2- 172.21.15.2:0/2059885230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd54c102cf0 0x7fd54c103170 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.890+0000 7fd551a00640 1 --2- 172.21.15.2:0/2059885230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd54c101b00 0x7fd54c101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.890+0000 7fd551a00640 1 -- 172.21.15.2:0/2059885230 >> 172.21.15.2:0/2059885230 conn(0x7fd54c0fd2a0 msgr2=0x7fd54c0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.890+0000 7fd551a00640 1 -- 172.21.15.2:0/2059885230 shutdown_connections 2024-09-10T08:08:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.890+0000 7fd551a00640 1 -- 172.21.15.2:0/2059885230 wait complete. 2024-09-10T08:08:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.891+0000 7fd551a00640 1 Processor -- start 2024-09-10T08:08:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.892+0000 7fd551a00640 1 -- start start 2024-09-10T08:08:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.892+0000 7fd551a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd54c101b00 0x7fd54c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.892+0000 7fd551a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd54c102cf0 0x7fd54c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.892+0000 7fd551a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd54c1036b0 0x7fd54c1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.892+0000 7fd551a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c10d780 con 0x7fd54c102cf0 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.892+0000 7fd551a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c10d600 con 0x7fd54c101b00 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.892+0000 7fd551a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c10d900 con 0x7fd54c1036b0 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.892+0000 7fd54a000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd54c102cf0 0x7fd54c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.892+0000 7fd54a000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd54c102cf0 0x7fd54c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39896/0 (socket says 172.21.15.2:39896) 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.892+0000 7fd54a000640 1 -- 172.21.15.2:0/2931977028 learned_addr learned my addr 172.21.15.2:0/2931977028 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.892+0000 7fd54aa00640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd54c101b00 0x7fd54c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.892+0000 7fd54b400640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd54c1036b0 0x7fd54c1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.893+0000 7fd54a000640 1 -- 172.21.15.2:0/2931977028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd54c1036b0 msgr2=0x7fd54c1a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.893+0000 7fd54a000640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd54c1036b0 0x7fd54c1a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.893+0000 7fd54a000640 1 -- 172.21.15.2:0/2931977028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd54c101b00 msgr2=0x7fd54c19a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.893+0000 7fd54a000640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd54c101b00 0x7fd54c19a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.893+0000 7fd54a000640 1 -- 172.21.15.2:0/2931977028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c1a1cf0 con 0x7fd54c102cf0 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.893+0000 7fd54a000640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd54c102cf0 0x7fd54c19a610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd538009230 tx=0x7fd5380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.893+0000 7fd543400640 1 -- 172.21.15.2:0/2931977028 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53803f030 con 0x7fd54c102cf0 2024-09-10T08:08:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.893+0000 7fd543400640 1 -- 172.21.15.2:0/2931977028 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd538003ce0 con 0x7fd54c102cf0 2024-09-10T08:08:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.893+0000 7fd543400640 1 -- 172.21.15.2:0/2931977028 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538035070 con 0x7fd54c102cf0 2024-09-10T08:08:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.893+0000 7fd551a00640 1 -- 172.21.15.2:0/2931977028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54c1a1f80 con 0x7fd54c102cf0 2024-09-10T08:08:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.893+0000 7fd551a00640 1 -- 172.21.15.2:0/2931977028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54c1a2390 con 0x7fd54c102cf0 2024-09-10T08:08:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.895+0000 7fd551a00640 1 -- 172.21.15.2:0/2931977028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd54c1a2680 con 0x7fd54c102cf0 2024-09-10T08:08:20.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.896+0000 7fd543400640 1 -- 172.21.15.2:0/2931977028 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd538042030 con 0x7fd54c102cf0 2024-09-10T08:08:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.898+0000 7fd543400640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5300a2c10 0x7fd5300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.898+0000 7fd543400640 1 -- 172.21.15.2:0/2931977028 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd5380ef540 con 0x7fd54c102cf0 2024-09-10T08:08:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.898+0000 7fd54aa00640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5300a2c10 0x7fd5300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.899+0000 7fd54aa00640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5300a2c10 0x7fd5300a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd54c102b50 tx=0x7fd53c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:20.902+0000 7fd543400640 1 -- 172.21.15.2:0/2931977028 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5380bba00 con 0x7fd54c102cf0 2024-09-10T08:08:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:20 smithi002 ceph-mon[30563]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T08:08:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:20 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T08:08:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:20 smithi002 ceph-mon[30563]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:08:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:20 smithi002 ceph-mon[30563]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:08:21.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:20 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3516294450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:08:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.085+0000 7fd551a00640 1 -- 172.21.15.2:0/2931977028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd54c1a2a10 con 0x7fd54c102cf0 2024-09-10T08:08:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.085+0000 7fd543400640 1 -- 172.21.15.2:0/2931977028 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fd5380bfd00 con 0x7fd54c102cf0 2024-09-10T08:08:21.085 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-09-10T08:08:21.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.087+0000 7fd551a00640 1 -- 172.21.15.2:0/2931977028 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5300a2c10 msgr2=0x7fd5300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:21.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.087+0000 7fd551a00640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5300a2c10 0x7fd5300a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd54c102b50 tx=0x7fd53c005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.087+0000 7fd551a00640 1 -- 172.21.15.2:0/2931977028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd54c102cf0 msgr2=0x7fd54c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.087+0000 7fd551a00640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd54c102cf0 0x7fd54c19a610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd538009230 tx=0x7fd5380042f0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.088+0000 7fd551a00640 1 -- 172.21.15.2:0/2931977028 shutdown_connections 2024-09-10T08:08:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.088+0000 7fd551a00640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5300a2c10 0x7fd5300a50d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.088+0000 7fd551a00640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd54c1036b0 0x7fd54c1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.088+0000 7fd551a00640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd54c102cf0 0x7fd54c19a610 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.088+0000 7fd551a00640 1 --2- 172.21.15.2:0/2931977028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd54c101b00 0x7fd54c19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.088+0000 7fd551a00640 1 -- 172.21.15.2:0/2931977028 >> 172.21.15.2:0/2931977028 conn(0x7fd54c0fd2a0 msgr2=0x7fd54c074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.088+0000 7fd551a00640 1 -- 172.21.15.2:0/2931977028 shutdown_connections 2024-09-10T08:08:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.088+0000 7fd551a00640 1 -- 172.21.15.2:0/2931977028 wait complete. 2024-09-10T08:08:21.103 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T08:08:21.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:20 smithi155 ceph-mon[35405]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T08:08:21.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:20 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T08:08:21.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:20 smithi155 ceph-mon[35405]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:08:21.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:20 smithi155 ceph-mon[35405]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:08:21.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:20 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3516294450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:08:21.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:20 smithi107 ceph-mon[35048]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T08:08:21.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:20 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T08:08:21.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:20 smithi107 ceph-mon[35048]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:08:21.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:20 smithi107 ceph-mon[35048]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:08:21.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:20 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3516294450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:08:21.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.265+0000 7fe147200640 1 -- 172.21.15.2:0/4176627689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1481057c0 msgr2=0x7fe14810c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:21.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.265+0000 7fe147200640 1 --2- 172.21.15.2:0/4176627689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1481057c0 0x7fe14810c050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe140009ca0 tx=0x7fe14002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.265+0000 7fe147200640 1 -- 172.21.15.2:0/4176627689 shutdown_connections 2024-09-10T08:08:21.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.265+0000 7fe147200640 1 --2- 172.21.15.2:0/4176627689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1481057c0 0x7fe14810c050 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.265+0000 7fe147200640 1 --2- 172.21.15.2:0/4176627689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe148104e00 0x7fe148105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.265+0000 7fe147200640 1 --2- 172.21.15.2:0/4176627689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe148103c10 0x7fe148104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.265+0000 7fe147200640 1 -- 172.21.15.2:0/4176627689 >> 172.21.15.2:0/4176627689 conn(0x7fe1480ff390 msgr2=0x7fe1481017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:21.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.265+0000 7fe147200640 1 -- 172.21.15.2:0/4176627689 shutdown_connections 2024-09-10T08:08:21.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.265+0000 7fe147200640 1 -- 172.21.15.2:0/4176627689 wait complete. 2024-09-10T08:08:21.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.266+0000 7fe147200640 1 Processor -- start 2024-09-10T08:08:21.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.266+0000 7fe147200640 1 -- start start 2024-09-10T08:08:21.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.267+0000 7fe147200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe148103c10 0x7fe148199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:21.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.267+0000 7fe147200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe148104e00 0x7fe14819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:21.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.267+0000 7fe147200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1481057c0 0x7fe1481a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:21.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.267+0000 7fe147200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe14810f890 con 0x7fe1481057c0 2024-09-10T08:08:21.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.267+0000 7fe147200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe14810f710 con 0x7fe148103c10 2024-09-10T08:08:21.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.267+0000 7fe147200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe14810fa10 con 0x7fe148104e00 2024-09-10T08:08:21.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.267+0000 7fe146800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1481057c0 0x7fe1481a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:21.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.267+0000 7fe145e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe148103c10 0x7fe148199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:21.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.267+0000 7fe145e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe148103c10 0x7fe148199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:49166/0 (socket says 172.21.15.2:49166) 2024-09-10T08:08:21.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.267+0000 7fe145e00640 1 -- 172.21.15.2:0/1172218737 learned_addr learned my addr 172.21.15.2:0/1172218737 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:21.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.267+0000 7fe145400640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe148104e00 0x7fe14819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:21.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.267+0000 7fe145e00640 1 -- 172.21.15.2:0/1172218737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe148104e00 msgr2=0x7fe14819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:21.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.268+0000 7fe145e00640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe148104e00 0x7fe14819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.268+0000 7fe145e00640 1 -- 172.21.15.2:0/1172218737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1481057c0 msgr2=0x7fe1481a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:21.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.268+0000 7fe145e00640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1481057c0 0x7fe1481a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.268+0000 7fe145e00640 1 -- 172.21.15.2:0/1172218737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1481a1ab0 con 0x7fe148103c10 2024-09-10T08:08:21.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.268+0000 7fe146800640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1481057c0 0x7fe1481a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:21.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.268+0000 7fe145e00640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe148103c10 0x7fe148199e90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe13800e9c0 tx=0x7fe13800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:21.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.268+0000 7fe13ea00640 1 -- 172.21.15.2:0/1172218737 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13800cd80 con 0x7fe148103c10 2024-09-10T08:08:21.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.268+0000 7fe147200640 1 -- 172.21.15.2:0/1172218737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1481a1da0 con 0x7fe148103c10 2024-09-10T08:08:21.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.268+0000 7fe13ea00640 1 -- 172.21.15.2:0/1172218737 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1380035f0 con 0x7fe148103c10 2024-09-10T08:08:21.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.268+0000 7fe13ea00640 1 -- 172.21.15.2:0/1172218737 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138010690 con 0x7fe148103c10 2024-09-10T08:08:21.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.268+0000 7fe147200640 1 -- 172.21.15.2:0/1172218737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1481a2210 con 0x7fe148103c10 2024-09-10T08:08:21.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.269+0000 7fe147200640 1 -- 172.21.15.2:0/1172218737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe14810af80 con 0x7fe148103c10 2024-09-10T08:08:21.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.271+0000 7fe13ea00640 1 -- 172.21.15.2:0/1172218737 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe1380037a0 con 0x7fe148103c10 2024-09-10T08:08:21.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.273+0000 7fe13ea00640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe12c0a2b90 0x7fe12c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:21.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.273+0000 7fe13ea00640 1 -- 172.21.15.2:0/1172218737 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe1380cf880 con 0x7fe148103c10 2024-09-10T08:08:21.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.273+0000 7fe145400640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe12c0a2b90 0x7fe12c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:21.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.273+0000 7fe145400640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe12c0a2b90 0x7fe12c0a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe14819b3a0 tx=0x7fe134005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:21.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.275+0000 7fe13ea00640 1 -- 172.21.15.2:0/1172218737 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe13809c390 con 0x7fe148103c10 2024-09-10T08:08:21.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.462+0000 7fe147200640 1 -- 172.21.15.2:0/1172218737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe14802da00 con 0x7fe148103c10 2024-09-10T08:08:21.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.462+0000 7fe13ea00640 1 -- 172.21.15.2:0/1172218737 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7fe13809bc90 con 0x7fe148103c10 2024-09-10T08:08:21.462 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-09-10T08:08:21.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.465+0000 7fe147200640 1 -- 172.21.15.2:0/1172218737 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe12c0a2b90 msgr2=0x7fe12c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.465+0000 7fe147200640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe12c0a2b90 0x7fe12c0a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe14819b3a0 tx=0x7fe134005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.465+0000 7fe147200640 1 -- 172.21.15.2:0/1172218737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe148103c10 msgr2=0x7fe148199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.465+0000 7fe147200640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe148103c10 0x7fe148199e90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe13800e9c0 tx=0x7fe13800ee80 comp rx=0 tx=0).stop 2024-09-10T08:08:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.465+0000 7fe147200640 1 -- 172.21.15.2:0/1172218737 shutdown_connections 2024-09-10T08:08:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.465+0000 7fe147200640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1481057c0 0x7fe1481a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.465+0000 7fe147200640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe12c0a2b90 0x7fe12c0a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.465+0000 7fe147200640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe148104e00 0x7fe14819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.465+0000 7fe147200640 1 --2- 172.21.15.2:0/1172218737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe148103c10 0x7fe148199e90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.465+0000 7fe147200640 1 -- 172.21.15.2:0/1172218737 >> 172.21.15.2:0/1172218737 conn(0x7fe1480ff390 msgr2=0x7fe148108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.465+0000 7fe147200640 1 -- 172.21.15.2:0/1172218737 shutdown_connections 2024-09-10T08:08:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.465+0000 7fe147200640 1 -- 172.21.15.2:0/1172218737 wait complete. 2024-09-10T08:08:21.480 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T08:08:21.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.644+0000 7fb69a200640 1 -- 172.21.15.2:0/2543446226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6941058d0 msgr2=0x7fb69410c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:21.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.644+0000 7fb69a200640 1 --2- 172.21.15.2:0/2543446226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6941058d0 0x7fb69410c160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009ca0 tx=0x7fb68c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.645+0000 7fb69a200640 1 -- 172.21.15.2:0/2543446226 shutdown_connections 2024-09-10T08:08:21.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.645+0000 7fb69a200640 1 --2- 172.21.15.2:0/2543446226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6941058d0 0x7fb69410c160 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.645+0000 7fb69a200640 1 --2- 172.21.15.2:0/2543446226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb694104f10 0x7fb694105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.645+0000 7fb69a200640 1 --2- 172.21.15.2:0/2543446226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb694103d20 0x7fb694104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.645+0000 7fb69a200640 1 -- 172.21.15.2:0/2543446226 >> 172.21.15.2:0/2543446226 conn(0x7fb6940ff4a0 msgr2=0x7fb6941018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:21.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.645+0000 7fb69a200640 1 -- 172.21.15.2:0/2543446226 shutdown_connections 2024-09-10T08:08:21.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.645+0000 7fb69a200640 1 -- 172.21.15.2:0/2543446226 wait complete. 2024-09-10T08:08:21.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.646+0000 7fb69a200640 1 Processor -- start 2024-09-10T08:08:21.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.646+0000 7fb69a200640 1 -- start start 2024-09-10T08:08:21.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.646+0000 7fb69a200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb694103d20 0x7fb694199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:21.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.646+0000 7fb69a200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb694104f10 0x7fb69419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:21.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.646+0000 7fb69a200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6941058d0 0x7fb6941a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:21.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.646+0000 7fb69a200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb69410f9d0 con 0x7fb694104f10 2024-09-10T08:08:21.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.646+0000 7fb69a200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb69410f850 con 0x7fb694103d20 2024-09-10T08:08:21.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.646+0000 7fb69a200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb69410fb50 con 0x7fb6941058d0 2024-09-10T08:08:21.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.646+0000 7fb68be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb694104f10 0x7fb69419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:21.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.646+0000 7fb68be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb694104f10 0x7fb69419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39936/0 (socket says 172.21.15.2:39936) 2024-09-10T08:08:21.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.646+0000 7fb68be00640 1 -- 172.21.15.2:0/3277957872 learned_addr learned my addr 172.21.15.2:0/3277957872 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:21.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.646+0000 7fb693400640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb694103d20 0x7fb694199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:21.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.646+0000 7fb693e00640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6941058d0 0x7fb6941a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:21.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.647+0000 7fb68be00640 1 -- 172.21.15.2:0/3277957872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6941058d0 msgr2=0x7fb6941a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:21.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.647+0000 7fb68be00640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6941058d0 0x7fb6941a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.647+0000 7fb68be00640 1 -- 172.21.15.2:0/3277957872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb694103d20 msgr2=0x7fb694199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:21.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.647+0000 7fb68be00640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb694103d20 0x7fb694199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.647+0000 7fb68be00640 1 -- 172.21.15.2:0/3277957872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6941a1c10 con 0x7fb694104f10 2024-09-10T08:08:21.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.647+0000 7fb68be00640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb694104f10 0x7fb69419a530 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00d950 tx=0x7fb67c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:21.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.647+0000 7fb690c00640 1 -- 172.21.15.2:0/3277957872 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c017070 con 0x7fb694104f10 2024-09-10T08:08:21.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.647+0000 7fb690c00640 1 -- 172.21.15.2:0/3277957872 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb67c009c30 con 0x7fb694104f10 2024-09-10T08:08:21.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.647+0000 7fb69a200640 1 -- 172.21.15.2:0/3277957872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6941a1f00 con 0x7fb694104f10 2024-09-10T08:08:21.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.647+0000 7fb690c00640 1 -- 172.21.15.2:0/3277957872 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c00f6b0 con 0x7fb694104f10 2024-09-10T08:08:21.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.647+0000 7fb69a200640 1 -- 172.21.15.2:0/3277957872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6941a2410 con 0x7fb694104f10 2024-09-10T08:08:21.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.648+0000 7fb69a200640 1 -- 172.21.15.2:0/3277957872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb69410b0c0 con 0x7fb694104f10 2024-09-10T08:08:21.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.650+0000 7fb690c00640 1 -- 172.21.15.2:0/3277957872 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb67c0109d0 con 0x7fb694104f10 2024-09-10T08:08:21.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.652+0000 7fb690c00640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb6780a2c10 0x7fb6780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:21.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.652+0000 7fb690c00640 1 -- 172.21.15.2:0/3277957872 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb67c0cef80 con 0x7fb694104f10 2024-09-10T08:08:21.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.652+0000 7fb693400640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb6780a2c10 0x7fb6780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:21.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.652+0000 7fb693400640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb6780a2c10 0x7fb6780a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb684005fd0 tx=0x7fb68400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:21.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.655+0000 7fb690c00640 1 -- 172.21.15.2:0/3277957872 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb67c009de0 con 0x7fb694104f10 2024-09-10T08:08:21.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.841+0000 7fb69a200640 1 -- 172.21.15.2:0/3277957872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb694062e90 con 0x7fb694104f10 2024-09-10T08:08:21.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.841+0000 7fb690c00640 1 -- 172.21.15.2:0/3277957872 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fb67c09ba90 con 0x7fb694104f10 2024-09-10T08:08:21.841 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-09-10T08:08:21.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.843+0000 7fb69a200640 1 -- 172.21.15.2:0/3277957872 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb6780a2c10 msgr2=0x7fb6780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:21.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.843+0000 7fb69a200640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb6780a2c10 0x7fb6780a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb684005fd0 tx=0x7fb68400b410 comp rx=0 tx=0).stop 2024-09-10T08:08:21.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.843+0000 7fb69a200640 1 -- 172.21.15.2:0/3277957872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb694104f10 msgr2=0x7fb69419a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:21.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.843+0000 7fb69a200640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb694104f10 0x7fb69419a530 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00d950 tx=0x7fb67c00de10 comp rx=0 tx=0).stop 2024-09-10T08:08:21.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.843+0000 7fb69a200640 1 -- 172.21.15.2:0/3277957872 shutdown_connections 2024-09-10T08:08:21.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.844+0000 7fb69a200640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb6780a2c10 0x7fb6780a50d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.844+0000 7fb69a200640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6941058d0 0x7fb6941a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.844+0000 7fb69a200640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb694104f10 0x7fb69419a530 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.844+0000 7fb69a200640 1 --2- 172.21.15.2:0/3277957872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb694103d20 0x7fb694199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:21.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.844+0000 7fb69a200640 1 -- 172.21.15.2:0/3277957872 >> 172.21.15.2:0/3277957872 conn(0x7fb6940ff4a0 msgr2=0x7fb694108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:21.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.844+0000 7fb69a200640 1 -- 172.21.15.2:0/3277957872 shutdown_connections 2024-09-10T08:08:21.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:21.844+0000 7fb69a200640 1 -- 172.21.15.2:0/3277957872 wait complete. 2024-09-10T08:08:21.859 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T08:08:22.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.029+0000 7fce42600640 1 -- 172.21.15.2:0/1725132789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c1036c0 msgr2=0x7fce3c109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:22.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.029+0000 7fce42600640 1 --2- 172.21.15.2:0/1725132789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c1036c0 0x7fce3c109f50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce34009ca0 tx=0x7fce3402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:08:22.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.029+0000 7fce42600640 1 -- 172.21.15.2:0/1725132789 shutdown_connections 2024-09-10T08:08:22.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.029+0000 7fce42600640 1 --2- 172.21.15.2:0/1725132789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c1036c0 0x7fce3c109f50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:22.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.029+0000 7fce42600640 1 --2- 172.21.15.2:0/1725132789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce3c102d00 0x7fce3c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:22.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.029+0000 7fce42600640 1 --2- 172.21.15.2:0/1725132789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce3c101b10 0x7fce3c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:22.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.029+0000 7fce42600640 1 -- 172.21.15.2:0/1725132789 >> 172.21.15.2:0/1725132789 conn(0x7fce3c0fd2b0 msgr2=0x7fce3c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:22.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.029+0000 7fce42600640 1 -- 172.21.15.2:0/1725132789 shutdown_connections 2024-09-10T08:08:22.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.029+0000 7fce42600640 1 -- 172.21.15.2:0/1725132789 wait complete. 2024-09-10T08:08:22.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.030+0000 7fce42600640 1 Processor -- start 2024-09-10T08:08:22.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.030+0000 7fce42600640 1 -- start start 2024-09-10T08:08:22.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce42600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce3c101b10 0x7fce3c075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:22.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce42600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c102d00 0x7fce3c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce42600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce3c1036c0 0x7fce3c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce42600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c10d790 con 0x7fce3c102d00 2024-09-10T08:08:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce42600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c10d610 con 0x7fce3c101b10 2024-09-10T08:08:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce42600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c10d910 con 0x7fce3c1036c0 2024-09-10T08:08:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce3aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c102d00 0x7fce3c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce3b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce3c101b10 0x7fce3c075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce3b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce3c101b10 0x7fce3c075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:49192/0 (socket says 172.21.15.2:49192) 2024-09-10T08:08:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce3b400640 1 -- 172.21.15.2:0/1088956563 learned_addr learned my addr 172.21.15.2:0/1088956563 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce3be00640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce3c1036c0 0x7fce3c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce3b400640 1 -- 172.21.15.2:0/1088956563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce3c1036c0 msgr2=0x7fce3c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce3b400640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce3c1036c0 0x7fce3c074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce3b400640 1 -- 172.21.15.2:0/1088956563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c102d00 msgr2=0x7fce3c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce3b400640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c102d00 0x7fce3c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce3b400640 1 -- 172.21.15.2:0/1088956563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c0753b0 con 0x7fce3c101b10 2024-09-10T08:08:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.031+0000 7fce3aa00640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c102d00 0x7fce3c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.032+0000 7fce3b400640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce3c101b10 0x7fce3c075fa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00e9c0 tx=0x7fce2c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.032+0000 7fce33e00640 1 -- 172.21.15.2:0/1088956563 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c00cd80 con 0x7fce3c101b10 2024-09-10T08:08:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.032+0000 7fce33e00640 1 -- 172.21.15.2:0/1088956563 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce2c0035f0 con 0x7fce3c101b10 2024-09-10T08:08:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.032+0000 7fce42600640 1 -- 172.21.15.2:0/1088956563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3c1a1f60 con 0x7fce3c101b10 2024-09-10T08:08:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.032+0000 7fce33e00640 1 -- 172.21.15.2:0/1088956563 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c010690 con 0x7fce3c101b10 2024-09-10T08:08:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.032+0000 7fce42600640 1 -- 172.21.15.2:0/1088956563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3c1a2360 con 0x7fce3c101b10 2024-09-10T08:08:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.033+0000 7fce42600640 1 -- 172.21.15.2:0/1088956563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce3c108f00 con 0x7fce3c101b10 2024-09-10T08:08:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.035+0000 7fce33e00640 1 -- 172.21.15.2:0/1088956563 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fce2c0037a0 con 0x7fce3c101b10 2024-09-10T08:08:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.036+0000 7fce33e00640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce200a2c20 0x7fce200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.036+0000 7fce33e00640 1 -- 172.21.15.2:0/1088956563 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fce2c0cfa40 con 0x7fce3c101b10 2024-09-10T08:08:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.037+0000 7fce3aa00640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce200a2c20 0x7fce200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.037+0000 7fce3aa00640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce200a2c20 0x7fce200a50e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fce3c075d10 tx=0x7fce28005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:22.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.040+0000 7fce33e00640 1 -- 172.21.15.2:0/1088956563 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce2c09c3f0 con 0x7fce3c101b10 2024-09-10T08:08:22.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:21 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2931977028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:08:22.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:21 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1172218737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:08:22.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:21 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3277957872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:08:22.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:21 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2931977028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:08:22.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:21 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1172218737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:08:22.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:21 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3277957872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:08:22.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.227+0000 7fce42600640 1 -- 172.21.15.2:0/1088956563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fce3c062e90 con 0x7fce3c101b10 2024-09-10T08:08:22.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.228+0000 7fce33e00640 1 -- 172.21.15.2:0/1088956563 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7fce2c09bcf0 con 0x7fce3c101b10 2024-09-10T08:08:22.228 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-09-10T08:08:22.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.230+0000 7fce42600640 1 -- 172.21.15.2:0/1088956563 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce200a2c20 msgr2=0x7fce200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:22.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.230+0000 7fce42600640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce200a2c20 0x7fce200a50e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fce3c075d10 tx=0x7fce28005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:22.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.230+0000 7fce42600640 1 -- 172.21.15.2:0/1088956563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce3c101b10 msgr2=0x7fce3c075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:22.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.230+0000 7fce42600640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce3c101b10 0x7fce3c075fa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00e9c0 tx=0x7fce2c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:08:22.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.231+0000 7fce42600640 1 -- 172.21.15.2:0/1088956563 shutdown_connections 2024-09-10T08:08:22.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.231+0000 7fce42600640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fce200a2c20 0x7fce200a50e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:22.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.231+0000 7fce42600640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce3c1036c0 0x7fce3c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:22.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.231+0000 7fce42600640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c102d00 0x7fce3c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:22.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.231+0000 7fce42600640 1 --2- 172.21.15.2:0/1088956563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce3c101b10 0x7fce3c075fa0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:22.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.231+0000 7fce42600640 1 -- 172.21.15.2:0/1088956563 >> 172.21.15.2:0/1088956563 conn(0x7fce3c0fd2b0 msgr2=0x7fce3c1060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:22.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.231+0000 7fce42600640 1 -- 172.21.15.2:0/1088956563 shutdown_connections 2024-09-10T08:08:22.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:22.231+0000 7fce42600640 1 -- 172.21.15.2:0/1088956563 wait complete. 2024-09-10T08:08:22.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:21 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2931977028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:08:22.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:21 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1172218737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:08:22.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:21 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3277957872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:08:22.247 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-10T08:08:22.247 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:08:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:22 smithi002 ceph-mon[30563]: pgmap v394: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-10T08:08:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:22 smithi002 ceph-mon[30563]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-10T08:08:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:22 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1088956563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:08:23.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:22 smithi155 ceph-mon[35405]: pgmap v394: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-10T08:08:23.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:22 smithi155 ceph-mon[35405]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-10T08:08:23.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:22 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1088956563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:08:23.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:22 smithi107 ceph-mon[35048]: pgmap v394: 129 pgs: 129 active+clean; 648 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-10T08:08:23.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:22 smithi107 ceph-mon[35048]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-10T08:08:23.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:22 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1088956563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:08:23.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.308+0000 7f43ebe00640 1 -- 172.21.15.2:0/263460537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e4103ff0 msgr2=0x7f43e41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:23.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.308+0000 7f43ebe00640 1 --2- 172.21.15.2:0/263460537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e4103ff0 0x7f43e41043d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009900 tx=0x7f43d802a410 comp rx=0 tx=0).stop 2024-09-10T08:08:23.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.308+0000 7f43ebe00640 1 -- 172.21.15.2:0/263460537 shutdown_connections 2024-09-10T08:08:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.308+0000 7f43ebe00640 1 --2- 172.21.15.2:0/263460537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43e4109120 0x7f43e4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.308+0000 7f43ebe00640 1 --2- 172.21.15.2:0/263460537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43e41049a0 0x7f43e41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.308+0000 7f43ebe00640 1 --2- 172.21.15.2:0/263460537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e4103ff0 0x7f43e41043d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.308+0000 7f43ebe00640 1 -- 172.21.15.2:0/263460537 >> 172.21.15.2:0/263460537 conn(0x7f43e40ffa90 msgr2=0x7f43e4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.308+0000 7f43ebe00640 1 -- 172.21.15.2:0/263460537 shutdown_connections 2024-09-10T08:08:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.308+0000 7f43ebe00640 1 -- 172.21.15.2:0/263460537 wait complete. 2024-09-10T08:08:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.309+0000 7f43ebe00640 1 Processor -- start 2024-09-10T08:08:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.309+0000 7f43ebe00640 1 -- start start 2024-09-10T08:08:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.310+0000 7f43ebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43e4103ff0 0x7f43e419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.310+0000 7f43ebe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43e41049a0 0x7f43e419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.310+0000 7f43ebe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e4109120 0x7f43e419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.310+0000 7f43ebe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4114120 con 0x7f43e4109120 2024-09-10T08:08:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.310+0000 7f43ebe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4113fa0 con 0x7f43e41049a0 2024-09-10T08:08:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.310+0000 7f43ebe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41142a0 con 0x7f43e4103ff0 2024-09-10T08:08:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.310+0000 7f43e9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43e4103ff0 0x7f43e419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.310+0000 7f43e9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43e4103ff0 0x7f43e419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51208/0 (socket says 172.21.15.2:51208) 2024-09-10T08:08:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.310+0000 7f43e9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e4109120 0x7f43e419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.310+0000 7f43e9400640 1 -- 172.21.15.2:0/836462605 learned_addr learned my addr 172.21.15.2:0/836462605 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:23.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.310+0000 7f43e8a00640 1 --2- 172.21.15.2:0/836462605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43e41049a0 0x7f43e419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.311+0000 7f43e9e00640 1 -- 172.21.15.2:0/836462605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43e4103ff0 msgr2=0x7f43e419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.311+0000 7f43e9e00640 1 --2- 172.21.15.2:0/836462605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43e4103ff0 0x7f43e419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.311+0000 7f43e9e00640 1 -- 172.21.15.2:0/836462605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43e41049a0 msgr2=0x7f43e419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.311+0000 7f43e9e00640 1 --2- 172.21.15.2:0/836462605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43e41049a0 0x7f43e419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.311+0000 7f43e9e00640 1 -- 172.21.15.2:0/836462605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e419f590 con 0x7f43e4109120 2024-09-10T08:08:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.311+0000 7f43e9e00640 1 --2- 172.21.15.2:0/836462605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e4109120 0x7f43e419eeb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009fd0 tx=0x7f43dc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.311+0000 7f43e2000640 1 -- 172.21.15.2:0/836462605 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00ce20 con 0x7f43e4109120 2024-09-10T08:08:23.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.311+0000 7f43e2000640 1 -- 172.21.15.2:0/836462605 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43dc005020 con 0x7f43e4109120 2024-09-10T08:08:23.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.311+0000 7f43ebe00640 1 -- 172.21.15.2:0/836462605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e419f880 con 0x7f43e4109120 2024-09-10T08:08:23.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.311+0000 7f43e2000640 1 -- 172.21.15.2:0/836462605 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc010720 con 0x7f43e4109120 2024-09-10T08:08:23.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.311+0000 7f43ebe00640 1 -- 172.21.15.2:0/836462605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e41a7160 con 0x7f43e4109120 2024-09-10T08:08:23.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.312+0000 7f43ebe00640 1 -- 172.21.15.2:0/836462605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e41043d0 con 0x7f43e4109120 2024-09-10T08:08:23.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.314+0000 7f43e2000640 1 -- 172.21.15.2:0/836462605 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f43dc005540 con 0x7f43e4109120 2024-09-10T08:08:23.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.316+0000 7f43e2000640 1 --2- 172.21.15.2:0/836462605 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f43cc0a2c10 0x7f43cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:23.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.316+0000 7f43e2000640 1 -- 172.21.15.2:0/836462605 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f43dc0cfca0 con 0x7f43e4109120 2024-09-10T08:08:23.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.316+0000 7f43e9400640 1 --2- 172.21.15.2:0/836462605 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f43cc0a2c10 0x7f43cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:23.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.317+0000 7f43e9400640 1 --2- 172.21.15.2:0/836462605 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f43cc0a2c10 0x7f43cc0a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f43d80096c0 tx=0x7f43d8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:23.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.320+0000 7f43e2000640 1 -- 172.21.15.2:0/836462605 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43dc09c650 con 0x7f43e4109120 2024-09-10T08:08:23.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.547+0000 7f43ebe00640 1 -- 172.21.15.2:0/836462605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43e402d7c0 con 0x7f43e4109120 2024-09-10T08:08:23.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.547+0000 7f43e2000640 1 -- 172.21.15.2:0/836462605 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f43dc09bf50 con 0x7f43e4109120 2024-09-10T08:08:23.547 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:23.548 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:08:23.548 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:08:23.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.550+0000 7f43ebe00640 1 -- 172.21.15.2:0/836462605 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f43cc0a2c10 msgr2=0x7f43cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:23.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.550+0000 7f43ebe00640 1 --2- 172.21.15.2:0/836462605 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f43cc0a2c10 0x7f43cc0a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f43d80096c0 tx=0x7f43d8005a20 comp rx=0 tx=0).stop 2024-09-10T08:08:23.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.550+0000 7f43ebe00640 1 -- 172.21.15.2:0/836462605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e4109120 msgr2=0x7f43e419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:23.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.550+0000 7f43ebe00640 1 --2- 172.21.15.2:0/836462605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e4109120 0x7f43e419eeb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009fd0 tx=0x7f43dc00ef10 comp rx=0 tx=0).stop 2024-09-10T08:08:23.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.550+0000 7f43ebe00640 1 -- 172.21.15.2:0/836462605 shutdown_connections 2024-09-10T08:08:23.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.550+0000 7f43ebe00640 1 --2- 172.21.15.2:0/836462605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43e4109120 0x7f43e419eeb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:23.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.550+0000 7f43ebe00640 1 --2- 172.21.15.2:0/836462605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43e41049a0 0x7f43e419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:23.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.550+0000 7f43ebe00640 1 --2- 172.21.15.2:0/836462605 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f43cc0a2c10 0x7f43cc0a50d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:23.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.550+0000 7f43ebe00640 1 --2- 172.21.15.2:0/836462605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43e4103ff0 0x7f43e419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:23.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.550+0000 7f43ebe00640 1 -- 172.21.15.2:0/836462605 >> 172.21.15.2:0/836462605 conn(0x7f43e40ffa90 msgr2=0x7f43e4100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:23.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.550+0000 7f43ebe00640 1 -- 172.21.15.2:0/836462605 shutdown_connections 2024-09-10T08:08:23.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:23.550+0000 7f43ebe00640 1 -- 172.21.15.2:0/836462605 wait complete. 2024-09-10T08:08:24.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:23 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/836462605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:24.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:23 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/836462605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:24.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:23 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/836462605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:24.701 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-10T08:08:24.701 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-10T08:08:24.701 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T08:08:24.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.867+0000 7f9108800640 1 -- 172.21.15.2:0/2099345192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9100074bd0 msgr2=0x7f9100075030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:24.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.867+0000 7f9108800640 1 --2- 172.21.15.2:0/2099345192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9100074bd0 0x7f9100075030 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009900 tx=0x7f90f002a480 comp rx=0 tx=0).stop 2024-09-10T08:08:24.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.867+0000 7f9108800640 1 -- 172.21.15.2:0/2099345192 shutdown_connections 2024-09-10T08:08:24.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.867+0000 7f9108800640 1 --2- 172.21.15.2:0/2099345192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91001099c0 0x7f910010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:24.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.867+0000 7f9108800640 1 --2- 172.21.15.2:0/2099345192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9100074bd0 0x7f9100075030 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:24.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.867+0000 7f9108800640 1 --2- 172.21.15.2:0/2099345192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91000761b0 0x7f9100074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:24.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.867+0000 7f9108800640 1 -- 172.21.15.2:0/2099345192 >> 172.21.15.2:0/2099345192 conn(0x7f91000fd2b0 msgr2=0x7f91000ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:24.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.867+0000 7f9108800640 1 -- 172.21.15.2:0/2099345192 shutdown_connections 2024-09-10T08:08:24.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.867+0000 7f9108800640 1 -- 172.21.15.2:0/2099345192 wait complete. 2024-09-10T08:08:24.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.868+0000 7f9108800640 1 Processor -- start 2024-09-10T08:08:24.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.868+0000 7f9108800640 1 -- start start 2024-09-10T08:08:24.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9108800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9100074bd0 0x7f910019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:24.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9108800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91000761b0 0x7f910019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:24.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9108800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91001099c0 0x7f91001a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:24.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9108800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f910010f9e0 con 0x7f9100074bd0 2024-09-10T08:08:24.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9108800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f910010f860 con 0x7f91001099c0 2024-09-10T08:08:24.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9108800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f910010fb60 con 0x7f91000761b0 2024-09-10T08:08:24.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9105e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9100074bd0 0x7f910019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9105e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9100074bd0 0x7f910019a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39986/0 (socket says 172.21.15.2:39986) 2024-09-10T08:08:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9105e00640 1 -- 172.21.15.2:0/3386794307 learned_addr learned my addr 172.21.15.2:0/3386794307 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9105400640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91000761b0 0x7f910019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9106800640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91001099c0 0x7f91001a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9105e00640 1 -- 172.21.15.2:0/3386794307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91000761b0 msgr2=0x7f910019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9105e00640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91000761b0 0x7f910019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9105e00640 1 -- 172.21.15.2:0/3386794307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91001099c0 msgr2=0x7f91001a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9105e00640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91001099c0 0x7f91001a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9105e00640 1 -- 172.21.15.2:0/3386794307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91001a1c50 con 0x7f9100074bd0 2024-09-10T08:08:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.869+0000 7f9106800640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91001099c0 0x7f91001a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:08:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.870+0000 7f9105e00640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9100074bd0 0x7f910019a030 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f90f400e9f0 tx=0x7f90f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.870+0000 7f90faa00640 1 -- 172.21.15.2:0/3386794307 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f400cd80 con 0x7f9100074bd0 2024-09-10T08:08:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.870+0000 7f90faa00640 1 -- 172.21.15.2:0/3386794307 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f4010910 con 0x7f9100074bd0 2024-09-10T08:08:24.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.870+0000 7f9108800640 1 -- 172.21.15.2:0/3386794307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91001a1f40 con 0x7f9100074bd0 2024-09-10T08:08:24.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.870+0000 7f90faa00640 1 -- 172.21.15.2:0/3386794307 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4018990 con 0x7f9100074bd0 2024-09-10T08:08:24.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.870+0000 7f9108800640 1 -- 172.21.15.2:0/3386794307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91001a23b0 con 0x7f9100074bd0 2024-09-10T08:08:24.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.871+0000 7f9108800640 1 -- 172.21.15.2:0/3386794307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9100108780 con 0x7f9100074bd0 2024-09-10T08:08:24.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.873+0000 7f90faa00640 1 -- 172.21.15.2:0/3386794307 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f90f4010ac0 con 0x7f9100074bd0 2024-09-10T08:08:24.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.875+0000 7f90faa00640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f90e80a2ca0 0x7f90e80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:24.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.875+0000 7f90faa00640 1 -- 172.21.15.2:0/3386794307 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f90f40cf940 con 0x7f9100074bd0 2024-09-10T08:08:24.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.875+0000 7f9105400640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f90e80a2ca0 0x7f90e80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:24.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.876+0000 7f9105400640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f90e80a2ca0 0x7f90e80a5160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90f0031440 tx=0x7f90f00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:24.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:24.879+0000 7f90faa00640 1 -- 172.21.15.2:0/3386794307 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90f409c2f0 con 0x7f9100074bd0 2024-09-10T08:08:25.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:24 smithi002 ceph-mon[30563]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-10T08:08:25.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.065+0000 7f9108800640 1 -- 172.21.15.2:0/3386794307 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9100062630 con 0x7f90e80a2ca0 2024-09-10T08:08:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.074+0000 7f90faa00640 1 -- 172.21.15.2:0/3386794307 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340724 (secure 0 0 0) 0x7f9100062630 con 0x7f90e80a2ca0 2024-09-10T08:08:25.079 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:25.081 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-10T08:08:25.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.078+0000 7f9108800640 1 -- 172.21.15.2:0/3386794307 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f90e80a2ca0 msgr2=0x7f90e80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:25.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.078+0000 7f9108800640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f90e80a2ca0 0x7f90e80a5160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90f0031440 tx=0x7f90f00315b0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.079+0000 7f9108800640 1 -- 172.21.15.2:0/3386794307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9100074bd0 msgr2=0x7f910019a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:25.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.079+0000 7f9108800640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9100074bd0 0x7f910019a030 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f90f400e9f0 tx=0x7f90f400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.079+0000 7f9108800640 1 -- 172.21.15.2:0/3386794307 shutdown_connections 2024-09-10T08:08:25.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.079+0000 7f9108800640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91001099c0 0x7f91001a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.079+0000 7f9108800640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f90e80a2ca0 0x7f90e80a5160 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.079+0000 7f9108800640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91000761b0 0x7f910019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.079+0000 7f9108800640 1 --2- 172.21.15.2:0/3386794307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9100074bd0 0x7f910019a030 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.079+0000 7f9108800640 1 -- 172.21.15.2:0/3386794307 >> 172.21.15.2:0/3386794307 conn(0x7f91000fd2b0 msgr2=0x7f9100105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:25.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.079+0000 7f9108800640 1 -- 172.21.15.2:0/3386794307 shutdown_connections 2024-09-10T08:08:25.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.079+0000 7f9108800640 1 -- 172.21.15.2:0/3386794307 wait complete. 2024-09-10T08:08:25.094 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-10T08:08:23.693024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664830,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":879,"num_read_kb":565,"num_write":370,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1184461,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":798,"ondisk_log_size":798,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362048,"kb_used_data":12800,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579776,"statfs":{"total":1151940427776,"available":1151569690624,"internally_reserved":0,"allocated":13107200,"data_stored":8545409,"data_compressed":497802,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7174,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":46,"num_write":13,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003532"},"pg_stats":[{"pgid":"2.1f","version":"93'9","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514795+0000","last_change":"2024-09-10T08:05:22.462062+0000","last_active":"2024-09-10T08:07:05.514795+0000","last_peered":"2024-09-10T08:07:05.514795+0000","last_clean":"2024-09-10T08:07:05.514795+0000","last_became_active":"2024-09-10T08:05:22.460973+0000","last_became_peered":"2024-09-10T08:05:22.460973+0000","last_unstale":"2024-09-10T08:07:05.514795+0000","last_undegraded":"2024-09-10T08:07:05.514795+0000","last_fullsized":"2024-09-10T08:07:05.514795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:01.357446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656803+0000","last_change":"2024-09-10T08:05:25.482873+0000","last_active":"2024-09-10T08:08:03.656803+0000","last_peered":"2024-09-10T08:08:03.656803+0000","last_clean":"2024-09-10T08:08:03.656803+0000","last_became_active":"2024-09-10T08:05:25.482559+0000","last_became_peered":"2024-09-10T08:05:25.482559+0000","last_unstale":"2024-09-10T08:08:03.656803+0000","last_undegraded":"2024-09-10T08:08:03.656803+0000","last_fullsized":"2024-09-10T08:08:03.656803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:10:17.366160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511384+0000","last_change":"2024-09-10T08:06:04.736356+0000","last_active":"2024-09-10T08:07:05.511384+0000","last_peered":"2024-09-10T08:07:05.511384+0000","last_clean":"2024-09-10T08:07:05.511384+0000","last_became_active":"2024-09-10T08:06:04.735859+0000","last_became_peered":"2024-09-10T08:06:04.735859+0000","last_unstale":"2024-09-10T08:07:05.511384+0000","last_undegraded":"2024-09-10T08:07:05.511384+0000","last_fullsized":"2024-09-10T08:07:05.511384+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:56.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"93'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.877433+0000","last_change":"2024-09-10T08:05:22.569504+0000","last_active":"2024-09-10T08:07:05.877433+0000","last_peered":"2024-09-10T08:07:05.877433+0000","last_clean":"2024-09-10T08:07:05.877433+0000","last_became_active":"2024-09-10T08:05:22.569164+0000","last_became_peered":"2024-09-10T08:05:22.569164+0000","last_unstale":"2024-09-10T08:07:05.877433+0000","last_undegraded":"2024-09-10T08:07:05.877433+0000","last_fullsized":"2024-09-10T08:07:05.877433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:41.628963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.680208+0000","last_change":"2024-09-10T08:05:25.493017+0000","last_active":"2024-09-10T08:08:03.680208+0000","last_peered":"2024-09-10T08:08:03.680208+0000","last_clean":"2024-09-10T08:08:03.680208+0000","last_became_active":"2024-09-10T08:05:25.491370+0000","last_became_peered":"2024-09-10T08:05:25.491370+0000","last_unstale":"2024-09-10T08:08:03.680208+0000","last_undegraded":"2024-09-10T08:08:03.680208+0000","last_fullsized":"2024-09-10T08:08:03.680208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:41.616549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914087+0000","last_change":"2024-09-10T08:05:35.523751+0000","last_active":"2024-09-10T08:07:05.914087+0000","last_peered":"2024-09-10T08:07:05.914087+0000","last_clean":"2024-09-10T08:07:05.914087+0000","last_became_active":"2024-09-10T08:05:35.523372+0000","last_became_peered":"2024-09-10T08:05:35.523372+0000","last_unstale":"2024-09-10T08:07:05.914087+0000","last_undegraded":"2024-09-10T08:07:05.914087+0000","last_fullsized":"2024-09-10T08:07:05.914087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:20.971596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508934+0000","last_change":"2024-09-10T08:05:22.460590+0000","last_active":"2024-09-10T08:07:05.508934+0000","last_peered":"2024-09-10T08:07:05.508934+0000","last_clean":"2024-09-10T08:07:05.508934+0000","last_became_active":"2024-09-10T08:05:22.460238+0000","last_became_peered":"2024-09-10T08:05:22.460238+0000","last_unstale":"2024-09-10T08:07:05.508934+0000","last_undegraded":"2024-09-10T08:07:05.508934+0000","last_fullsized":"2024-09-10T08:07:05.508934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:18.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669035+0000","last_change":"2024-09-10T08:05:25.487392+0000","last_active":"2024-09-10T08:08:03.669035+0000","last_peered":"2024-09-10T08:08:03.669035+0000","last_clean":"2024-09-10T08:08:03.669035+0000","last_became_active":"2024-09-10T08:05:25.487054+0000","last_became_peered":"2024-09-10T08:05:25.487054+0000","last_unstale":"2024-09-10T08:08:03.669035+0000","last_undegraded":"2024-09-10T08:08:03.669035+0000","last_fullsized":"2024-09-10T08:08:03.669035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:16.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914818+0000","last_change":"2024-09-10T08:06:04.735778+0000","last_active":"2024-09-10T08:07:05.914818+0000","last_peered":"2024-09-10T08:07:05.914818+0000","last_clean":"2024-09-10T08:07:05.914818+0000","last_became_active":"2024-09-10T08:06:04.735208+0000","last_became_peered":"2024-09-10T08:06:04.735208+0000","last_unstale":"2024-09-10T08:07:05.914818+0000","last_undegraded":"2024-09-10T08:07:05.914818+0000","last_fullsized":"2024-09-10T08:07:05.914818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:36.982246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508944+0000","last_change":"2024-09-10T08:05:22.458323+0000","last_active":"2024-09-10T08:07:05.508944+0000","last_peered":"2024-09-10T08:07:05.508944+0000","last_clean":"2024-09-10T08:07:05.508944+0000","last_became_active":"2024-09-10T08:05:22.457965+0000","last_became_peered":"2024-09-10T08:05:22.457965+0000","last_unstale":"2024-09-10T08:07:05.508944+0000","last_undegraded":"2024-09-10T08:07:05.508944+0000","last_fullsized":"2024-09-10T08:07:05.508944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:15.021305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.663997+0000","last_change":"2024-09-10T08:05:25.489883+0000","last_active":"2024-09-10T08:08:03.663997+0000","last_peered":"2024-09-10T08:08:03.663997+0000","last_clean":"2024-09-10T08:08:03.663997+0000","last_became_active":"2024-09-10T08:05:25.489482+0000","last_became_peered":"2024-09-10T08:05:25.489482+0000","last_unstale":"2024-09-10T08:08:03.663997+0000","last_undegraded":"2024-09-10T08:08:03.663997+0000","last_fullsized":"2024-09-10T08:08:03.663997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:52.080628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914064+0000","last_change":"2024-09-10T08:05:35.531458+0000","last_active":"2024-09-10T08:07:05.914064+0000","last_peered":"2024-09-10T08:07:05.914064+0000","last_clean":"2024-09-10T08:07:05.914064+0000","last_became_active":"2024-09-10T08:05:35.531072+0000","last_became_peered":"2024-09-10T08:05:35.531072+0000","last_unstale":"2024-09-10T08:07:05.914064+0000","last_undegraded":"2024-09-10T08:07:05.914064+0000","last_fullsized":"2024-09-10T08:07:05.914064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:04.792664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667650+0000","last_change":"2024-09-10T08:05:25.486865+0000","last_active":"2024-09-10T08:08:03.667650+0000","last_peered":"2024-09-10T08:08:03.667650+0000","last_clean":"2024-09-10T08:08:03.667650+0000","last_became_active":"2024-09-10T08:05:25.486531+0000","last_became_peered":"2024-09-10T08:05:25.486531+0000","last_unstale":"2024-09-10T08:08:03.667650+0000","last_undegraded":"2024-09-10T08:08:03.667650+0000","last_fullsized":"2024-09-10T08:08:03.667650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:26.854682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"93'4","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508917+0000","last_change":"2024-09-10T08:06:04.743303+0000","last_active":"2024-09-10T08:07:05.508917+0000","last_peered":"2024-09-10T08:07:05.508917+0000","last_clean":"2024-09-10T08:07:05.508917+0000","last_became_active":"2024-09-10T08:06:04.733294+0000","last_became_peered":"2024-09-10T08:06:04.733294+0000","last_unstale":"2024-09-10T08:07:05.508917+0000","last_undegraded":"2024-09-10T08:07:05.508917+0000","last_fullsized":"2024-09-10T08:07:05.508917+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:18.542658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914822+0000","last_change":"2024-09-10T08:05:35.642083+0000","last_active":"2024-09-10T08:07:05.914822+0000","last_peered":"2024-09-10T08:07:05.914822+0000","last_clean":"2024-09-10T08:07:05.914822+0000","last_became_active":"2024-09-10T08:05:35.641214+0000","last_became_peered":"2024-09-10T08:05:35.641214+0000","last_unstale":"2024-09-10T08:07:05.914822+0000","last_undegraded":"2024-09-10T08:07:05.914822+0000","last_fullsized":"2024-09-10T08:07:05.914822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:00:21.731769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681852+0000","last_change":"2024-09-10T08:05:25.501993+0000","last_active":"2024-09-10T08:08:03.681852+0000","last_peered":"2024-09-10T08:08:03.681852+0000","last_clean":"2024-09-10T08:08:03.681852+0000","last_became_active":"2024-09-10T08:05:25.499912+0000","last_became_peered":"2024-09-10T08:05:25.499912+0000","last_unstale":"2024-09-10T08:08:03.681852+0000","last_undegraded":"2024-09-10T08:08:03.681852+0000","last_fullsized":"2024-09-10T08:08:03.681852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:06.780067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:36.268711+0000","last_change":"2024-09-10T08:05:22.461786+0000","last_active":"2024-09-10T08:07:36.268711+0000","last_peered":"2024-09-10T08:07:36.268711+0000","last_clean":"2024-09-10T08:07:36.268711+0000","last_became_active":"2024-09-10T08:05:22.460821+0000","last_became_peered":"2024-09-10T08:05:22.460821+0000","last_unstale":"2024-09-10T08:07:36.268711+0000","last_undegraded":"2024-09-10T08:07:36.268711+0000","last_fullsized":"2024-09-10T08:07:36.268711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:25.083538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914568+0000","last_change":"2024-09-10T08:06:04.734470+0000","last_active":"2024-09-10T08:07:05.914568+0000","last_peered":"2024-09-10T08:07:05.914568+0000","last_clean":"2024-09-10T08:07:05.914568+0000","last_became_active":"2024-09-10T08:06:04.734040+0000","last_became_peered":"2024-09-10T08:06:04.734040+0000","last_unstale":"2024-09-10T08:07:05.914568+0000","last_undegraded":"2024-09-10T08:07:05.914568+0000","last_fullsized":"2024-09-10T08:07:05.914568+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:36.312144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675820+0000","last_change":"2024-09-10T08:05:25.492711+0000","last_active":"2024-09-10T08:08:03.675820+0000","last_peered":"2024-09-10T08:08:03.675820+0000","last_clean":"2024-09-10T08:08:03.675820+0000","last_became_active":"2024-09-10T08:05:25.492336+0000","last_became_peered":"2024-09-10T08:05:25.492336+0000","last_unstale":"2024-09-10T08:08:03.675820+0000","last_undegraded":"2024-09-10T08:08:03.675820+0000","last_fullsized":"2024-09-10T08:08:03.675820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:04.168884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"93'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876783+0000","last_change":"2024-09-10T08:05:22.461535+0000","last_active":"2024-09-10T08:07:05.876783+0000","last_peered":"2024-09-10T08:07:05.876783+0000","last_clean":"2024-09-10T08:07:05.876783+0000","last_became_active":"2024-09-10T08:05:22.461208+0000","last_became_peered":"2024-09-10T08:05:22.461208+0000","last_unstale":"2024-09-10T08:07:05.876783+0000","last_undegraded":"2024-09-10T08:07:05.876783+0000","last_fullsized":"2024-09-10T08:07:05.876783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:26.009242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513889+0000","last_change":"2024-09-10T08:05:35.535442+0000","last_active":"2024-09-10T08:07:05.513889+0000","last_peered":"2024-09-10T08:07:05.513889+0000","last_clean":"2024-09-10T08:07:05.513889+0000","last_became_active":"2024-09-10T08:05:35.535058+0000","last_became_peered":"2024-09-10T08:05:35.535058+0000","last_unstale":"2024-09-10T08:07:05.513889+0000","last_undegraded":"2024-09-10T08:07:05.513889+0000","last_fullsized":"2024-09-10T08:07:05.513889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:39.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.627539+0000","last_change":"2024-09-10T08:05:25.483440+0000","last_active":"2024-09-10T08:08:03.627539+0000","last_peered":"2024-09-10T08:08:03.627539+0000","last_clean":"2024-09-10T08:08:03.627539+0000","last_became_active":"2024-09-10T08:05:25.483141+0000","last_became_peered":"2024-09-10T08:05:25.483141+0000","last_unstale":"2024-09-10T08:08:03.627539+0000","last_undegraded":"2024-09-10T08:08:03.627539+0000","last_fullsized":"2024-09-10T08:08:03.627539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:00.953188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"93'2","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914337+0000","last_change":"2024-09-10T08:05:22.568141+0000","last_active":"2024-09-10T08:07:05.914337+0000","last_peered":"2024-09-10T08:07:05.914337+0000","last_clean":"2024-09-10T08:07:05.914337+0000","last_became_active":"2024-09-10T08:05:22.567857+0000","last_became_peered":"2024-09-10T08:05:22.567857+0000","last_unstale":"2024-09-10T08:07:05.914337+0000","last_undegraded":"2024-09-10T08:07:05.914337+0000","last_fullsized":"2024-09-10T08:07:05.914337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:22.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914846+0000","last_change":"2024-09-10T08:05:35.641993+0000","last_active":"2024-09-10T08:07:05.914846+0000","last_peered":"2024-09-10T08:07:05.914846+0000","last_clean":"2024-09-10T08:07:05.914846+0000","last_became_active":"2024-09-10T08:05:35.641376+0000","last_became_peered":"2024-09-10T08:05:35.641376+0000","last_unstale":"2024-09-10T08:07:05.914846+0000","last_undegraded":"2024-09-10T08:07:05.914846+0000","last_fullsized":"2024-09-10T08:07:05.914846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:16:34.038924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.678764+0000","last_change":"2024-09-10T08:06:04.737560+0000","last_active":"2024-09-10T08:08:03.678764+0000","last_peered":"2024-09-10T08:08:03.678764+0000","last_clean":"2024-09-10T08:08:03.678764+0000","last_became_active":"2024-09-10T08:06:04.736656+0000","last_became_peered":"2024-09-10T08:06:04.736656+0000","last_unstale":"2024-09-10T08:08:03.678764+0000","last_undegraded":"2024-09-10T08:08:03.678764+0000","last_fullsized":"2024-09-10T08:08:03.678764+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:35.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"93'4","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915999+0000","last_change":"2024-09-10T08:05:22.567077+0000","last_active":"2024-09-10T08:07:05.915999+0000","last_peered":"2024-09-10T08:07:05.915999+0000","last_clean":"2024-09-10T08:07:05.915999+0000","last_became_active":"2024-09-10T08:05:22.566757+0000","last_became_peered":"2024-09-10T08:05:22.566757+0000","last_unstale":"2024-09-10T08:07:05.915999+0000","last_undegraded":"2024-09-10T08:07:05.915999+0000","last_fullsized":"2024-09-10T08:07:05.915999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:06.881510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511524+0000","last_change":"2024-09-10T08:05:35.519539+0000","last_active":"2024-09-10T08:07:05.511524+0000","last_peered":"2024-09-10T08:07:05.511524+0000","last_clean":"2024-09-10T08:07:05.511524+0000","last_became_active":"2024-09-10T08:05:35.519044+0000","last_became_peered":"2024-09-10T08:05:35.519044+0000","last_unstale":"2024-09-10T08:07:05.511524+0000","last_undegraded":"2024-09-10T08:07:05.511524+0000","last_fullsized":"2024-09-10T08:07:05.511524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:10.546949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658164+0000","last_change":"2024-09-10T08:07:05.771774+0000","last_active":"2024-09-10T08:08:03.658164+0000","last_peered":"2024-09-10T08:08:03.658164+0000","last_clean":"2024-09-10T08:08:03.658164+0000","last_became_active":"2024-09-10T08:07:05.771336+0000","last_became_peered":"2024-09-10T08:07:05.771336+0000","last_unstale":"2024-09-10T08:08:03.658164+0000","last_undegraded":"2024-09-10T08:08:03.658164+0000","last_fullsized":"2024-09-10T08:08:03.658164+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:11.482222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"95'13","reported_seq":67,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:04.516173+0000","last_change":"2024-09-10T08:06:04.741199+0000","last_active":"2024-09-10T08:08:04.516173+0000","last_peered":"2024-09-10T08:08:04.516173+0000","last_clean":"2024-09-10T08:08:04.516173+0000","last_became_active":"2024-09-10T08:06:04.738076+0000","last_became_peered":"2024-09-10T08:06:04.738076+0000","last_unstale":"2024-09-10T08:08:04.516173+0000","last_undegraded":"2024-09-10T08:08:04.516173+0000","last_fullsized":"2024-09-10T08:08:04.516173+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:45.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876462+0000","last_change":"2024-09-10T08:05:35.642358+0000","last_active":"2024-09-10T08:07:05.876462+0000","last_peered":"2024-09-10T08:07:05.876462+0000","last_clean":"2024-09-10T08:07:05.876462+0000","last_became_active":"2024-09-10T08:05:35.641408+0000","last_became_peered":"2024-09-10T08:05:35.641408+0000","last_unstale":"2024-09-10T08:07:05.876462+0000","last_undegraded":"2024-09-10T08:07:05.876462+0000","last_fullsized":"2024-09-10T08:07:05.876462+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:42.015139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658430+0000","last_change":"2024-09-10T08:05:25.486310+0000","last_active":"2024-09-10T08:08:03.658430+0000","last_peered":"2024-09-10T08:08:03.658430+0000","last_clean":"2024-09-10T08:08:03.658430+0000","last_became_active":"2024-09-10T08:05:25.485915+0000","last_became_peered":"2024-09-10T08:05:25.485915+0000","last_unstale":"2024-09-10T08:08:03.658430+0000","last_undegraded":"2024-09-10T08:08:03.658430+0000","last_fullsized":"2024-09-10T08:08:03.658430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:56.064361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"93'1","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508987+0000","last_change":"2024-09-10T08:05:22.570595+0000","last_active":"2024-09-10T08:07:05.508987+0000","last_peered":"2024-09-10T08:07:05.508987+0000","last_clean":"2024-09-10T08:07:05.508987+0000","last_became_active":"2024-09-10T08:05:22.570219+0000","last_became_peered":"2024-09-10T08:05:22.570219+0000","last_unstale":"2024-09-10T08:07:05.508987+0000","last_undegraded":"2024-09-10T08:07:05.508987+0000","last_fullsized":"2024-09-10T08:07:05.508987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:03.230325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769927+0000","last_change":"2024-09-10T08:05:35.530472+0000","last_active":"2024-09-10T08:07:05.769927+0000","last_peered":"2024-09-10T08:07:05.769927+0000","last_clean":"2024-09-10T08:07:05.769927+0000","last_became_active":"2024-09-10T08:05:35.530066+0000","last_became_peered":"2024-09-10T08:05:35.530066+0000","last_unstale":"2024-09-10T08:07:05.769927+0000","last_undegraded":"2024-09-10T08:07:05.769927+0000","last_fullsized":"2024-09-10T08:07:05.769927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:58.915767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671834+0000","last_change":"2024-09-10T08:05:25.500524+0000","last_active":"2024-09-10T08:08:03.671834+0000","last_peered":"2024-09-10T08:08:03.671834+0000","last_clean":"2024-09-10T08:08:03.671834+0000","last_became_active":"2024-09-10T08:05:25.500186+0000","last_became_peered":"2024-09-10T08:05:25.500186+0000","last_unstale":"2024-09-10T08:08:03.671834+0000","last_undegraded":"2024-09-10T08:08:03.671834+0000","last_fullsized":"2024-09-10T08:08:03.671834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:51.936222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513172+0000","last_change":"2024-09-10T08:05:22.569326+0000","last_active":"2024-09-10T08:07:05.513172+0000","last_peered":"2024-09-10T08:07:05.513172+0000","last_clean":"2024-09-10T08:07:05.513172+0000","last_became_active":"2024-09-10T08:05:22.568895+0000","last_became_peered":"2024-09-10T08:05:22.568895+0000","last_unstale":"2024-09-10T08:07:05.513172+0000","last_undegraded":"2024-09-10T08:07:05.513172+0000","last_fullsized":"2024-09-10T08:07:05.513172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:47.212073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508368+0000","last_change":"2024-09-10T08:05:35.641600+0000","last_active":"2024-09-10T08:07:05.508368+0000","last_peered":"2024-09-10T08:07:05.508368+0000","last_clean":"2024-09-10T08:07:05.508368+0000","last_became_active":"2024-09-10T08:05:35.641205+0000","last_became_peered":"2024-09-10T08:05:35.641205+0000","last_unstale":"2024-09-10T08:07:05.508368+0000","last_undegraded":"2024-09-10T08:07:05.508368+0000","last_fullsized":"2024-09-10T08:07:05.508368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:40.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.659935+0000","last_change":"2024-09-10T08:05:25.495035+0000","last_active":"2024-09-10T08:08:03.659935+0000","last_peered":"2024-09-10T08:08:03.659935+0000","last_clean":"2024-09-10T08:08:03.659935+0000","last_became_active":"2024-09-10T08:05:25.494733+0000","last_became_peered":"2024-09-10T08:05:25.494733+0000","last_unstale":"2024-09-10T08:08:03.659935+0000","last_undegraded":"2024-09-10T08:08:03.659935+0000","last_fullsized":"2024-09-10T08:08:03.659935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.875867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"93'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509485+0000","last_change":"2024-09-10T08:05:22.458727+0000","last_active":"2024-09-10T08:07:05.509485+0000","last_peered":"2024-09-10T08:07:05.509485+0000","last_clean":"2024-09-10T08:07:05.509485+0000","last_became_active":"2024-09-10T08:05:22.458397+0000","last_became_peered":"2024-09-10T08:05:22.458397+0000","last_unstale":"2024-09-10T08:07:05.509485+0000","last_undegraded":"2024-09-10T08:07:05.509485+0000","last_fullsized":"2024-09-10T08:07:05.509485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:07.875559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914403+0000","last_change":"2024-09-10T08:05:35.534192+0000","last_active":"2024-09-10T08:07:05.914403+0000","last_peered":"2024-09-10T08:07:05.914403+0000","last_clean":"2024-09-10T08:07:05.914403+0000","last_became_active":"2024-09-10T08:05:35.530450+0000","last_became_peered":"2024-09-10T08:05:35.530450+0000","last_unstale":"2024-09-10T08:07:05.914403+0000","last_undegraded":"2024-09-10T08:07:05.914403+0000","last_fullsized":"2024-09-10T08:07:05.914403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:10.039144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651611+0000","last_change":"2024-09-10T08:05:25.482252+0000","last_active":"2024-09-10T08:08:03.651611+0000","last_peered":"2024-09-10T08:08:03.651611+0000","last_clean":"2024-09-10T08:08:03.651611+0000","last_became_active":"2024-09-10T08:05:25.481901+0000","last_became_peered":"2024-09-10T08:05:25.481901+0000","last_unstale":"2024-09-10T08:08:03.651611+0000","last_undegraded":"2024-09-10T08:08:03.651611+0000","last_fullsized":"2024-09-10T08:08:03.651611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:17.063340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":170,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:21.446702+0000","last_change":"2024-09-10T08:05:22.569199+0000","last_active":"2024-09-10T08:08:21.446702+0000","last_peered":"2024-09-10T08:08:21.446702+0000","last_clean":"2024-09-10T08:08:21.446702+0000","last_became_active":"2024-09-10T08:05:22.568787+0000","last_became_peered":"2024-09-10T08:05:22.568787+0000","last_unstale":"2024-09-10T08:08:21.446702+0000","last_undegraded":"2024-09-10T08:08:21.446702+0000","last_fullsized":"2024-09-10T08:08:21.446702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:59.610130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914677+0000","last_change":"2024-09-10T08:05:35.642446+0000","last_active":"2024-09-10T08:07:05.914677+0000","last_peered":"2024-09-10T08:07:05.914677+0000","last_clean":"2024-09-10T08:07:05.914677+0000","last_became_active":"2024-09-10T08:05:35.641543+0000","last_became_peered":"2024-09-10T08:05:35.641543+0000","last_unstale":"2024-09-10T08:07:05.914677+0000","last_undegraded":"2024-09-10T08:07:05.914677+0000","last_fullsized":"2024-09-10T08:07:05.914677+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:33.658406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.652192+0000","last_change":"2024-09-10T08:05:25.487598+0000","last_active":"2024-09-10T08:08:03.652192+0000","last_peered":"2024-09-10T08:08:03.652192+0000","last_clean":"2024-09-10T08:08:03.652192+0000","last_became_active":"2024-09-10T08:05:25.486879+0000","last_became_peered":"2024-09-10T08:05:25.486879+0000","last_unstale":"2024-09-10T08:08:03.652192+0000","last_undegraded":"2024-09-10T08:08:03.652192+0000","last_fullsized":"2024-09-10T08:08:03.652192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:20.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509934+0000","last_change":"2024-09-10T08:05:22.463416+0000","last_active":"2024-09-10T08:07:05.509934+0000","last_peered":"2024-09-10T08:07:05.509934+0000","last_clean":"2024-09-10T08:07:05.509934+0000","last_became_active":"2024-09-10T08:05:22.463004+0000","last_became_peered":"2024-09-10T08:05:22.463004+0000","last_unstale":"2024-09-10T08:07:05.509934+0000","last_undegraded":"2024-09-10T08:07:05.509934+0000","last_fullsized":"2024-09-10T08:07:05.509934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:56.257812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569364+0000","last_change":"2024-09-10T08:05:35.642059+0000","last_active":"2024-09-10T08:07:05.569364+0000","last_peered":"2024-09-10T08:07:05.569364+0000","last_clean":"2024-09-10T08:07:05.569364+0000","last_became_active":"2024-09-10T08:05:35.641540+0000","last_became_peered":"2024-09-10T08:05:35.641540+0000","last_unstale":"2024-09-10T08:07:05.569364+0000","last_undegraded":"2024-09-10T08:07:05.569364+0000","last_fullsized":"2024-09-10T08:07:05.569364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:28.557584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.670571+0000","last_change":"2024-09-10T08:05:25.496111+0000","last_active":"2024-09-10T08:08:03.670571+0000","last_peered":"2024-09-10T08:08:03.670571+0000","last_clean":"2024-09-10T08:08:03.670571+0000","last_became_active":"2024-09-10T08:05:25.495757+0000","last_became_peered":"2024-09-10T08:05:25.495757+0000","last_unstale":"2024-09-10T08:08:03.670571+0000","last_undegraded":"2024-09-10T08:08:03.670571+0000","last_fullsized":"2024-09-10T08:08:03.670571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:41.078205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'3","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915005+0000","last_change":"2024-09-10T08:05:22.460069+0000","last_active":"2024-09-10T08:07:05.915005+0000","last_peered":"2024-09-10T08:07:05.915005+0000","last_clean":"2024-09-10T08:07:05.915005+0000","last_became_active":"2024-09-10T08:05:22.459557+0000","last_became_peered":"2024-09-10T08:05:22.459557+0000","last_unstale":"2024-09-10T08:07:05.915005+0000","last_undegraded":"2024-09-10T08:07:05.915005+0000","last_fullsized":"2024-09-10T08:07:05.915005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:49.058507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509959+0000","last_change":"2024-09-10T08:04:08.188262+0000","last_active":"2024-09-10T08:07:05.509959+0000","last_peered":"2024-09-10T08:07:05.509959+0000","last_clean":"2024-09-10T08:07:05.509959+0000","last_became_active":"2024-09-10T08:04:08.187691+0000","last_became_peered":"2024-09-10T08:04:08.187691+0000","last_unstale":"2024-09-10T08:07:05.509959+0000","last_undegraded":"2024-09-10T08:07:05.509959+0000","last_fullsized":"2024-09-10T08:07:05.509959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_clean_scrub_stamp":"2024-09-10T08:00:26.235127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:15.358818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509550+0000","last_change":"2024-09-10T08:05:35.528371+0000","last_active":"2024-09-10T08:07:05.509550+0000","last_peered":"2024-09-10T08:07:05.509550+0000","last_clean":"2024-09-10T08:07:05.509550+0000","last_became_active":"2024-09-10T08:05:35.527925+0000","last_became_peered":"2024-09-10T08:05:35.527925+0000","last_unstale":"2024-09-10T08:07:05.509550+0000","last_undegraded":"2024-09-10T08:07:05.509550+0000","last_fullsized":"2024-09-10T08:07:05.509550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:37:00.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667384+0000","last_change":"2024-09-10T08:05:25.490753+0000","last_active":"2024-09-10T08:08:03.667384+0000","last_peered":"2024-09-10T08:08:03.667384+0000","last_clean":"2024-09-10T08:08:03.667384+0000","last_became_active":"2024-09-10T08:05:25.489996+0000","last_became_peered":"2024-09-10T08:05:25.489996+0000","last_unstale":"2024-09-10T08:08:03.667384+0000","last_undegraded":"2024-09-10T08:08:03.667384+0000","last_fullsized":"2024-09-10T08:08:03.667384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:15.637199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'3","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511918+0000","last_change":"2024-09-10T08:05:22.568696+0000","last_active":"2024-09-10T08:07:05.511918+0000","last_peered":"2024-09-10T08:07:05.511918+0000","last_clean":"2024-09-10T08:07:05.511918+0000","last_became_active":"2024-09-10T08:05:22.568331+0000","last_became_peered":"2024-09-10T08:05:22.568331+0000","last_unstale":"2024-09-10T08:07:05.511918+0000","last_undegraded":"2024-09-10T08:07:05.511918+0000","last_fullsized":"2024-09-10T08:07:05.511918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:21.699397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914428+0000","last_change":"2024-09-10T08:05:35.530804+0000","last_active":"2024-09-10T08:07:05.914428+0000","last_peered":"2024-09-10T08:07:05.914428+0000","last_clean":"2024-09-10T08:07:05.914428+0000","last_became_active":"2024-09-10T08:05:35.530356+0000","last_became_peered":"2024-09-10T08:05:35.530356+0000","last_unstale":"2024-09-10T08:07:05.914428+0000","last_undegraded":"2024-09-10T08:07:05.914428+0000","last_fullsized":"2024-09-10T08:07:05.914428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:59.318761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.662813+0000","last_change":"2024-09-10T08:05:25.489546+0000","last_active":"2024-09-10T08:08:03.662813+0000","last_peered":"2024-09-10T08:08:03.662813+0000","last_clean":"2024-09-10T08:08:03.662813+0000","last_became_active":"2024-09-10T08:05:25.489188+0000","last_became_peered":"2024-09-10T08:05:25.489188+0000","last_unstale":"2024-09-10T08:08:03.662813+0000","last_undegraded":"2024-09-10T08:08:03.662813+0000","last_fullsized":"2024-09-10T08:08:03.662813+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:26.471681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569736+0000","last_change":"2024-09-10T08:05:22.463912+0000","last_active":"2024-09-10T08:07:05.569736+0000","last_peered":"2024-09-10T08:07:05.569736+0000","last_clean":"2024-09-10T08:07:05.569736+0000","last_became_active":"2024-09-10T08:05:22.463561+0000","last_became_peered":"2024-09-10T08:05:22.463561+0000","last_unstale":"2024-09-10T08:07:05.569736+0000","last_undegraded":"2024-09-10T08:07:05.569736+0000","last_fullsized":"2024-09-10T08:07:05.569736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:22.407604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514959+0000","last_change":"2024-09-10T08:05:35.528795+0000","last_active":"2024-09-10T08:07:05.514959+0000","last_peered":"2024-09-10T08:07:05.514959+0000","last_clean":"2024-09-10T08:07:05.514959+0000","last_became_active":"2024-09-10T08:05:35.528402+0000","last_became_peered":"2024-09-10T08:05:35.528402+0000","last_unstale":"2024-09-10T08:07:05.514959+0000","last_undegraded":"2024-09-10T08:07:05.514959+0000","last_fullsized":"2024-09-10T08:07:05.514959+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:14.606371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.657641+0000","last_change":"2024-09-10T08:05:25.484086+0000","last_active":"2024-09-10T08:08:03.657641+0000","last_peered":"2024-09-10T08:08:03.657641+0000","last_clean":"2024-09-10T08:08:03.657641+0000","last_became_active":"2024-09-10T08:05:25.483757+0000","last_became_peered":"2024-09-10T08:05:25.483757+0000","last_unstale":"2024-09-10T08:08:03.657641+0000","last_undegraded":"2024-09-10T08:08:03.657641+0000","last_fullsized":"2024-09-10T08:08:03.657641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:16:16.362739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914678+0000","last_change":"2024-09-10T08:05:22.462220+0000","last_active":"2024-09-10T08:07:05.914678+0000","last_peered":"2024-09-10T08:07:05.914678+0000","last_clean":"2024-09-10T08:07:05.914678+0000","last_became_active":"2024-09-10T08:05:22.461726+0000","last_became_peered":"2024-09-10T08:05:22.461726+0000","last_unstale":"2024-09-10T08:07:05.914678+0000","last_undegraded":"2024-09-10T08:07:05.914678+0000","last_fullsized":"2024-09-10T08:07:05.914678+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:56.381034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508271+0000","last_change":"2024-09-10T08:06:04.734827+0000","last_active":"2024-09-10T08:07:05.508271+0000","last_peered":"2024-09-10T08:07:05.508271+0000","last_clean":"2024-09-10T08:07:05.508271+0000","last_became_active":"2024-09-10T08:06:04.734402+0000","last_became_peered":"2024-09-10T08:06:04.734402+0000","last_unstale":"2024-09-10T08:07:05.508271+0000","last_undegraded":"2024-09-10T08:07:05.508271+0000","last_fullsized":"2024-09-10T08:07:05.508271+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:24.806538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654332+0000","last_change":"2024-09-10T08:05:25.486100+0000","last_active":"2024-09-10T08:08:03.654332+0000","last_peered":"2024-09-10T08:08:03.654332+0000","last_clean":"2024-09-10T08:08:03.654332+0000","last_became_active":"2024-09-10T08:05:25.485586+0000","last_became_peered":"2024-09-10T08:05:25.485586+0000","last_unstale":"2024-09-10T08:08:03.654332+0000","last_undegraded":"2024-09-10T08:08:03.654332+0000","last_fullsized":"2024-09-10T08:08:03.654332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:33.565054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"93'2","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770403+0000","last_change":"2024-09-10T08:06:04.743408+0000","last_active":"2024-09-10T08:07:05.770403+0000","last_peered":"2024-09-10T08:07:05.770403+0000","last_clean":"2024-09-10T08:07:05.770403+0000","last_became_active":"2024-09-10T08:06:04.732830+0000","last_became_peered":"2024-09-10T08:06:04.732830+0000","last_unstale":"2024-09-10T08:07:05.770403+0000","last_undegraded":"2024-09-10T08:07:05.770403+0000","last_fullsized":"2024-09-10T08:07:05.770403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:29.381259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568977+0000","last_change":"2024-09-10T08:05:35.533015+0000","last_active":"2024-09-10T08:07:05.568977+0000","last_peered":"2024-09-10T08:07:05.568977+0000","last_clean":"2024-09-10T08:07:05.568977+0000","last_became_active":"2024-09-10T08:05:35.532590+0000","last_became_peered":"2024-09-10T08:05:35.532590+0000","last_unstale":"2024-09-10T08:07:05.568977+0000","last_undegraded":"2024-09-10T08:07:05.568977+0000","last_fullsized":"2024-09-10T08:07:05.568977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:23.984273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.650197+0000","last_change":"2024-09-10T08:05:25.489994+0000","last_active":"2024-09-10T08:08:03.650197+0000","last_peered":"2024-09-10T08:08:03.650197+0000","last_clean":"2024-09-10T08:08:03.650197+0000","last_became_active":"2024-09-10T08:05:25.489471+0000","last_became_peered":"2024-09-10T08:05:25.489471+0000","last_unstale":"2024-09-10T08:08:03.650197+0000","last_undegraded":"2024-09-10T08:08:03.650197+0000","last_fullsized":"2024-09-10T08:08:03.650197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:53.866570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"93'15","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508405+0000","last_change":"2024-09-10T08:05:22.568900+0000","last_active":"2024-09-10T08:07:05.508405+0000","last_peered":"2024-09-10T08:07:05.508405+0000","last_clean":"2024-09-10T08:07:05.508405+0000","last_became_active":"2024-09-10T08:05:22.568542+0000","last_became_peered":"2024-09-10T08:05:22.568542+0000","last_unstale":"2024-09-10T08:07:05.508405+0000","last_undegraded":"2024-09-10T08:07:05.508405+0000","last_fullsized":"2024-09-10T08:07:05.508405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:58:45.669769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914683+0000","last_change":"2024-09-10T08:05:35.537364+0000","last_active":"2024-09-10T08:07:05.914683+0000","last_peered":"2024-09-10T08:07:05.914683+0000","last_clean":"2024-09-10T08:07:05.914683+0000","last_became_active":"2024-09-10T08:05:35.535958+0000","last_became_peered":"2024-09-10T08:05:35.535958+0000","last_unstale":"2024-09-10T08:07:05.914683+0000","last_undegraded":"2024-09-10T08:07:05.914683+0000","last_fullsized":"2024-09-10T08:07:05.914683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:51.443798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660149+0000","last_change":"2024-09-10T08:07:05.917030+0000","last_active":"2024-09-10T08:08:03.660149+0000","last_peered":"2024-09-10T08:08:03.660149+0000","last_clean":"2024-09-10T08:08:03.660149+0000","last_became_active":"2024-09-10T08:07:05.916281+0000","last_became_peered":"2024-09-10T08:07:05.916281+0000","last_unstale":"2024-09-10T08:08:03.660149+0000","last_undegraded":"2024-09-10T08:08:03.660149+0000","last_fullsized":"2024-09-10T08:08:03.660149+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:15.060009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'4","reported_seq":171,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:20.445646+0000","last_change":"2024-09-10T08:05:22.459963+0000","last_active":"2024-09-10T08:08:20.445646+0000","last_peered":"2024-09-10T08:08:20.445646+0000","last_clean":"2024-09-10T08:08:20.445646+0000","last_became_active":"2024-09-10T08:05:22.459497+0000","last_became_peered":"2024-09-10T08:05:22.459497+0000","last_unstale":"2024-09-10T08:08:20.445646+0000","last_undegraded":"2024-09-10T08:08:20.445646+0000","last_fullsized":"2024-09-10T08:08:20.445646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:29.887980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513986+0000","last_change":"2024-09-10T08:05:35.534570+0000","last_active":"2024-09-10T08:07:05.513986+0000","last_peered":"2024-09-10T08:07:05.513986+0000","last_clean":"2024-09-10T08:07:05.513986+0000","last_became_active":"2024-09-10T08:05:35.534195+0000","last_became_peered":"2024-09-10T08:05:35.534195+0000","last_unstale":"2024-09-10T08:07:05.513986+0000","last_undegraded":"2024-09-10T08:07:05.513986+0000","last_fullsized":"2024-09-10T08:07:05.513986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:17.393491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.668227+0000","last_change":"2024-09-10T08:05:25.488551+0000","last_active":"2024-09-10T08:08:03.668227+0000","last_peered":"2024-09-10T08:08:03.668227+0000","last_clean":"2024-09-10T08:08:03.668227+0000","last_became_active":"2024-09-10T08:05:25.488086+0000","last_became_peered":"2024-09-10T08:05:25.488086+0000","last_unstale":"2024-09-10T08:08:03.668227+0000","last_undegraded":"2024-09-10T08:08:03.668227+0000","last_fullsized":"2024-09-10T08:08:03.668227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:38.044721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569070+0000","last_change":"2024-09-10T08:05:22.462269+0000","last_active":"2024-09-10T08:07:05.569070+0000","last_peered":"2024-09-10T08:07:05.569070+0000","last_clean":"2024-09-10T08:07:05.569070+0000","last_became_active":"2024-09-10T08:05:22.461924+0000","last_became_peered":"2024-09-10T08:05:22.461924+0000","last_unstale":"2024-09-10T08:07:05.569070+0000","last_undegraded":"2024-09-10T08:07:05.569070+0000","last_fullsized":"2024-09-10T08:07:05.569070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:54.996777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770191+0000","last_change":"2024-09-10T08:05:35.531596+0000","last_active":"2024-09-10T08:07:05.770191+0000","last_peered":"2024-09-10T08:07:05.770191+0000","last_clean":"2024-09-10T08:07:05.770191+0000","last_became_active":"2024-09-10T08:05:35.530179+0000","last_became_peered":"2024-09-10T08:05:35.530179+0000","last_unstale":"2024-09-10T08:07:05.770191+0000","last_undegraded":"2024-09-10T08:07:05.770191+0000","last_fullsized":"2024-09-10T08:07:05.770191+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:32.056426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.663218+0000","last_change":"2024-09-10T08:05:25.503504+0000","last_active":"2024-09-10T08:08:03.663218+0000","last_peered":"2024-09-10T08:08:03.663218+0000","last_clean":"2024-09-10T08:08:03.663218+0000","last_became_active":"2024-09-10T08:05:25.503105+0000","last_became_peered":"2024-09-10T08:05:25.503105+0000","last_unstale":"2024-09-10T08:08:03.663218+0000","last_undegraded":"2024-09-10T08:08:03.663218+0000","last_fullsized":"2024-09-10T08:08:03.663218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:40.885654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"93'6","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511623+0000","last_change":"2024-09-10T08:05:22.462271+0000","last_active":"2024-09-10T08:07:05.511623+0000","last_peered":"2024-09-10T08:07:05.511623+0000","last_clean":"2024-09-10T08:07:05.511623+0000","last_became_active":"2024-09-10T08:05:22.461195+0000","last_became_peered":"2024-09-10T08:05:22.461195+0000","last_unstale":"2024-09-10T08:07:05.511623+0000","last_undegraded":"2024-09-10T08:07:05.511623+0000","last_fullsized":"2024-09-10T08:07:05.511623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:20.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569486+0000","last_change":"2024-09-10T08:05:35.533218+0000","last_active":"2024-09-10T08:07:05.569486+0000","last_peered":"2024-09-10T08:07:05.569486+0000","last_clean":"2024-09-10T08:07:05.569486+0000","last_became_active":"2024-09-10T08:05:35.532706+0000","last_became_peered":"2024-09-10T08:05:35.532706+0000","last_unstale":"2024-09-10T08:07:05.569486+0000","last_undegraded":"2024-09-10T08:07:05.569486+0000","last_fullsized":"2024-09-10T08:07:05.569486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:22.150563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.655456+0000","last_change":"2024-09-10T08:05:25.478978+0000","last_active":"2024-09-10T08:08:03.655456+0000","last_peered":"2024-09-10T08:08:03.655456+0000","last_clean":"2024-09-10T08:08:03.655456+0000","last_became_active":"2024-09-10T08:05:25.478649+0000","last_became_peered":"2024-09-10T08:05:25.478649+0000","last_unstale":"2024-09-10T08:08:03.655456+0000","last_undegraded":"2024-09-10T08:08:03.655456+0000","last_fullsized":"2024-09-10T08:08:03.655456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:17.493446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'39","reported_seq":349,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:20.446317+0000","last_change":"2024-09-10T08:06:04.740972+0000","last_active":"2024-09-10T08:08:20.446317+0000","last_peered":"2024-09-10T08:08:20.446317+0000","last_clean":"2024-09-10T08:08:20.446317+0000","last_became_active":"2024-09-10T08:06:04.731200+0000","last_became_peered":"2024-09-10T08:06:04.731200+0000","last_unstale":"2024-09-10T08:08:20.446317+0000","last_undegraded":"2024-09-10T08:08:20.446317+0000","last_fullsized":"2024-09-10T08:08:20.446317+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:17.169708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68987,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":193,"num_write":40,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3703,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512781+0000","last_change":"2024-09-10T08:05:35.532594+0000","last_active":"2024-09-10T08:07:05.512781+0000","last_peered":"2024-09-10T08:07:05.512781+0000","last_clean":"2024-09-10T08:07:05.512781+0000","last_became_active":"2024-09-10T08:05:35.532132+0000","last_became_peered":"2024-09-10T08:05:35.532132+0000","last_unstale":"2024-09-10T08:07:05.512781+0000","last_undegraded":"2024-09-10T08:07:05.512781+0000","last_fullsized":"2024-09-10T08:07:05.512781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:15.348648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.666577+0000","last_change":"2024-09-10T08:05:25.485297+0000","last_active":"2024-09-10T08:08:03.666577+0000","last_peered":"2024-09-10T08:08:03.666577+0000","last_clean":"2024-09-10T08:08:03.666577+0000","last_became_active":"2024-09-10T08:05:25.484883+0000","last_became_peered":"2024-09-10T08:05:25.484883+0000","last_unstale":"2024-09-10T08:08:03.666577+0000","last_undegraded":"2024-09-10T08:08:03.666577+0000","last_fullsized":"2024-09-10T08:08:03.666577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:37.052435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"93'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569713+0000","last_change":"2024-09-10T08:05:22.568993+0000","last_active":"2024-09-10T08:07:05.569713+0000","last_peered":"2024-09-10T08:07:05.569713+0000","last_clean":"2024-09-10T08:07:05.569713+0000","last_became_active":"2024-09-10T08:05:22.568564+0000","last_became_peered":"2024-09-10T08:05:22.568564+0000","last_unstale":"2024-09-10T08:07:05.569713+0000","last_undegraded":"2024-09-10T08:07:05.569713+0000","last_fullsized":"2024-09-10T08:07:05.569713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:23.767174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914556+0000","last_change":"2024-09-10T08:05:35.524025+0000","last_active":"2024-09-10T08:07:05.914556+0000","last_peered":"2024-09-10T08:07:05.914556+0000","last_clean":"2024-09-10T08:07:05.914556+0000","last_became_active":"2024-09-10T08:05:35.523628+0000","last_became_peered":"2024-09-10T08:05:35.523628+0000","last_unstale":"2024-09-10T08:07:05.914556+0000","last_undegraded":"2024-09-10T08:07:05.914556+0000","last_fullsized":"2024-09-10T08:07:05.914556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:48.593779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.680405+0000","last_change":"2024-09-10T08:07:05.570537+0000","last_active":"2024-09-10T08:08:03.680405+0000","last_peered":"2024-09-10T08:08:03.680405+0000","last_clean":"2024-09-10T08:08:03.680405+0000","last_became_active":"2024-09-10T08:07:05.570278+0000","last_became_peered":"2024-09-10T08:07:05.570278+0000","last_unstale":"2024-09-10T08:08:03.680405+0000","last_undegraded":"2024-09-10T08:08:03.680405+0000","last_fullsized":"2024-09-10T08:08:03.680405+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:56.490462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509635+0000","last_change":"2024-09-10T08:05:22.460487+0000","last_active":"2024-09-10T08:07:05.509635+0000","last_peered":"2024-09-10T08:07:05.509635+0000","last_clean":"2024-09-10T08:07:05.509635+0000","last_became_active":"2024-09-10T08:05:22.460158+0000","last_became_peered":"2024-09-10T08:05:22.460158+0000","last_unstale":"2024-09-10T08:07:05.509635+0000","last_undegraded":"2024-09-10T08:07:05.509635+0000","last_fullsized":"2024-09-10T08:07:05.509635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:53.285513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512495+0000","last_change":"2024-09-10T08:05:35.531399+0000","last_active":"2024-09-10T08:07:05.512495+0000","last_peered":"2024-09-10T08:07:05.512495+0000","last_clean":"2024-09-10T08:07:05.512495+0000","last_became_active":"2024-09-10T08:05:35.530320+0000","last_became_peered":"2024-09-10T08:05:35.530320+0000","last_unstale":"2024-09-10T08:07:05.512495+0000","last_undegraded":"2024-09-10T08:07:05.512495+0000","last_fullsized":"2024-09-10T08:07:05.512495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:19.814046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669147+0000","last_change":"2024-09-10T08:05:25.488208+0000","last_active":"2024-09-10T08:08:03.669147+0000","last_peered":"2024-09-10T08:08:03.669147+0000","last_clean":"2024-09-10T08:08:03.669147+0000","last_became_active":"2024-09-10T08:05:25.487893+0000","last_became_peered":"2024-09-10T08:05:25.487893+0000","last_unstale":"2024-09-10T08:08:03.669147+0000","last_undegraded":"2024-09-10T08:08:03.669147+0000","last_fullsized":"2024-09-10T08:08:03.669147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:10.999416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:20.296166+0000","last_change":"2024-09-10T08:05:22.461321+0000","last_active":"2024-09-10T08:08:20.296166+0000","last_peered":"2024-09-10T08:08:20.296166+0000","last_clean":"2024-09-10T08:08:20.296166+0000","last_became_active":"2024-09-10T08:05:22.460950+0000","last_became_peered":"2024-09-10T08:05:22.460950+0000","last_unstale":"2024-09-10T08:08:20.296166+0000","last_undegraded":"2024-09-10T08:08:20.296166+0000","last_fullsized":"2024-09-10T08:08:20.296166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:55.647273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876534+0000","last_change":"2024-09-10T08:05:35.530578+0000","last_active":"2024-09-10T08:07:05.876534+0000","last_peered":"2024-09-10T08:07:05.876534+0000","last_clean":"2024-09-10T08:07:05.876534+0000","last_became_active":"2024-09-10T08:05:35.530162+0000","last_became_peered":"2024-09-10T08:05:35.530162+0000","last_unstale":"2024-09-10T08:07:05.876534+0000","last_undegraded":"2024-09-10T08:07:05.876534+0000","last_fullsized":"2024-09-10T08:07:05.876534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:19.290973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513617+0000","last_change":"2024-09-10T08:05:35.532685+0000","last_active":"2024-09-10T08:07:05.513617+0000","last_peered":"2024-09-10T08:07:05.513617+0000","last_clean":"2024-09-10T08:07:05.513617+0000","last_became_active":"2024-09-10T08:05:35.532257+0000","last_became_peered":"2024-09-10T08:05:35.532257+0000","last_unstale":"2024-09-10T08:07:05.513617+0000","last_undegraded":"2024-09-10T08:07:05.513617+0000","last_fullsized":"2024-09-10T08:07:05.513617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:07.412435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.662135+0000","last_change":"2024-09-10T08:05:25.494472+0000","last_active":"2024-09-10T08:08:03.662135+0000","last_peered":"2024-09-10T08:08:03.662135+0000","last_clean":"2024-09-10T08:08:03.662135+0000","last_became_active":"2024-09-10T08:05:25.493928+0000","last_became_peered":"2024-09-10T08:05:25.493928+0000","last_unstale":"2024-09-10T08:08:03.662135+0000","last_undegraded":"2024-09-10T08:08:03.662135+0000","last_fullsized":"2024-09-10T08:08:03.662135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:00.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"93'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568928+0000","last_change":"2024-09-10T08:05:22.460585+0000","last_active":"2024-09-10T08:07:05.568928+0000","last_peered":"2024-09-10T08:07:05.568928+0000","last_clean":"2024-09-10T08:07:05.568928+0000","last_became_active":"2024-09-10T08:05:22.460259+0000","last_became_peered":"2024-09-10T08:05:22.460259+0000","last_unstale":"2024-09-10T08:07:05.568928+0000","last_undegraded":"2024-09-10T08:07:05.568928+0000","last_fullsized":"2024-09-10T08:07:05.568928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:45.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770275+0000","last_change":"2024-09-10T08:05:35.524000+0000","last_active":"2024-09-10T08:07:05.770275+0000","last_peered":"2024-09-10T08:07:05.770275+0000","last_clean":"2024-09-10T08:07:05.770275+0000","last_became_active":"2024-09-10T08:05:35.523647+0000","last_became_peered":"2024-09-10T08:05:35.523647+0000","last_unstale":"2024-09-10T08:07:05.770275+0000","last_undegraded":"2024-09-10T08:07:05.770275+0000","last_fullsized":"2024-09-10T08:07:05.770275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:10:54.664865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660563+0000","last_change":"2024-09-10T08:05:25.499701+0000","last_active":"2024-09-10T08:08:03.660563+0000","last_peered":"2024-09-10T08:08:03.660563+0000","last_clean":"2024-09-10T08:08:03.660563+0000","last_became_active":"2024-09-10T08:05:25.496890+0000","last_became_peered":"2024-09-10T08:05:25.496890+0000","last_unstale":"2024-09-10T08:08:03.660563+0000","last_undegraded":"2024-09-10T08:08:03.660563+0000","last_fullsized":"2024-09-10T08:08:03.660563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:58.166365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914070+0000","last_change":"2024-09-10T08:05:22.460880+0000","last_active":"2024-09-10T08:07:05.914070+0000","last_peered":"2024-09-10T08:07:05.914070+0000","last_clean":"2024-09-10T08:07:05.914070+0000","last_became_active":"2024-09-10T08:05:22.459932+0000","last_became_peered":"2024-09-10T08:05:22.459932+0000","last_unstale":"2024-09-10T08:07:05.914070+0000","last_undegraded":"2024-09-10T08:07:05.914070+0000","last_fullsized":"2024-09-10T08:07:05.914070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:55.826526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915405+0000","last_change":"2024-09-10T08:05:22.460578+0000","last_active":"2024-09-10T08:07:05.915405+0000","last_peered":"2024-09-10T08:07:05.915405+0000","last_clean":"2024-09-10T08:07:05.915405+0000","last_became_active":"2024-09-10T08:05:22.460239+0000","last_became_peered":"2024-09-10T08:05:22.460239+0000","last_unstale":"2024-09-10T08:07:05.915405+0000","last_undegraded":"2024-09-10T08:07:05.915405+0000","last_fullsized":"2024-09-10T08:07:05.915405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:07.606243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.661142+0000","last_change":"2024-09-10T08:05:25.489846+0000","last_active":"2024-09-10T08:08:03.661142+0000","last_peered":"2024-09-10T08:08:03.661142+0000","last_clean":"2024-09-10T08:08:03.661142+0000","last_became_active":"2024-09-10T08:05:25.489326+0000","last_became_peered":"2024-09-10T08:05:25.489326+0000","last_unstale":"2024-09-10T08:08:03.661142+0000","last_undegraded":"2024-09-10T08:08:03.661142+0000","last_fullsized":"2024-09-10T08:08:03.661142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:43.458468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511635+0000","last_change":"2024-09-10T08:05:35.530522+0000","last_active":"2024-09-10T08:07:05.511635+0000","last_peered":"2024-09-10T08:07:05.511635+0000","last_clean":"2024-09-10T08:07:05.511635+0000","last_became_active":"2024-09-10T08:05:35.530099+0000","last_became_peered":"2024-09-10T08:05:35.530099+0000","last_unstale":"2024-09-10T08:07:05.511635+0000","last_undegraded":"2024-09-10T08:07:05.511635+0000","last_fullsized":"2024-09-10T08:07:05.511635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:58.469119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":179,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:20.446528+0000","last_change":"2024-09-10T08:05:22.461391+0000","last_active":"2024-09-10T08:08:20.446528+0000","last_peered":"2024-09-10T08:08:20.446528+0000","last_clean":"2024-09-10T08:08:20.446528+0000","last_became_active":"2024-09-10T08:05:22.461042+0000","last_became_peered":"2024-09-10T08:05:22.461042+0000","last_unstale":"2024-09-10T08:08:20.446528+0000","last_undegraded":"2024-09-10T08:08:20.446528+0000","last_fullsized":"2024-09-10T08:08:20.446528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:12.635710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1860,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.649481+0000","last_change":"2024-09-10T08:05:25.487378+0000","last_active":"2024-09-10T08:08:03.649481+0000","last_peered":"2024-09-10T08:08:03.649481+0000","last_clean":"2024-09-10T08:08:03.649481+0000","last_became_active":"2024-09-10T08:05:25.487055+0000","last_became_peered":"2024-09-10T08:05:25.487055+0000","last_unstale":"2024-09-10T08:08:03.649481+0000","last_undegraded":"2024-09-10T08:08:03.649481+0000","last_fullsized":"2024-09-10T08:08:03.649481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:59.522014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511748+0000","last_change":"2024-09-10T08:05:35.522765+0000","last_active":"2024-09-10T08:07:05.511748+0000","last_peered":"2024-09-10T08:07:05.511748+0000","last_clean":"2024-09-10T08:07:05.511748+0000","last_became_active":"2024-09-10T08:05:35.522359+0000","last_became_peered":"2024-09-10T08:05:35.522359+0000","last_unstale":"2024-09-10T08:07:05.511748+0000","last_undegraded":"2024-09-10T08:07:05.511748+0000","last_fullsized":"2024-09-10T08:07:05.511748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:15.280867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.652869+0000","last_change":"2024-09-10T08:05:25.475388+0000","last_active":"2024-09-10T08:08:03.652869+0000","last_peered":"2024-09-10T08:08:03.652869+0000","last_clean":"2024-09-10T08:08:03.652869+0000","last_became_active":"2024-09-10T08:05:25.474331+0000","last_became_peered":"2024-09-10T08:05:25.474331+0000","last_unstale":"2024-09-10T08:08:03.652869+0000","last_undegraded":"2024-09-10T08:08:03.652869+0000","last_fullsized":"2024-09-10T08:08:03.652869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:58.713148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656383+0000","last_change":"2024-09-10T08:07:05.570505+0000","last_active":"2024-09-10T08:08:03.656383+0000","last_peered":"2024-09-10T08:08:03.656383+0000","last_clean":"2024-09-10T08:08:03.656383+0000","last_became_active":"2024-09-10T08:07:05.569970+0000","last_became_peered":"2024-09-10T08:07:05.569970+0000","last_unstale":"2024-09-10T08:08:03.656383+0000","last_undegraded":"2024-09-10T08:08:03.656383+0000","last_fullsized":"2024-09-10T08:08:03.656383+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:13.282503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671951+0000","last_change":"2024-09-10T08:05:25.491585+0000","last_active":"2024-09-10T08:08:03.671951+0000","last_peered":"2024-09-10T08:08:03.671951+0000","last_clean":"2024-09-10T08:08:03.671951+0000","last_became_active":"2024-09-10T08:05:25.491224+0000","last_became_peered":"2024-09-10T08:05:25.491224+0000","last_unstale":"2024-09-10T08:08:03.671951+0000","last_undegraded":"2024-09-10T08:08:03.671951+0000","last_fullsized":"2024-09-10T08:08:03.671951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:56.066808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681502+0000","last_change":"2024-09-10T08:05:25.499578+0000","last_active":"2024-09-10T08:08:03.681502+0000","last_peered":"2024-09-10T08:08:03.681502+0000","last_clean":"2024-09-10T08:08:03.681502+0000","last_became_active":"2024-09-10T08:05:25.498445+0000","last_became_peered":"2024-09-10T08:05:25.498445+0000","last_unstale":"2024-09-10T08:08:03.681502+0000","last_undegraded":"2024-09-10T08:08:03.681502+0000","last_fullsized":"2024-09-10T08:08:03.681502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:58.961699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675717+0000","last_change":"2024-09-10T08:06:04.735764+0000","last_active":"2024-09-10T08:08:03.675717+0000","last_peered":"2024-09-10T08:08:03.675717+0000","last_clean":"2024-09-10T08:08:03.675717+0000","last_became_active":"2024-09-10T08:06:04.735366+0000","last_became_peered":"2024-09-10T08:06:04.735366+0000","last_unstale":"2024-09-10T08:08:03.675717+0000","last_undegraded":"2024-09-10T08:08:03.675717+0000","last_fullsized":"2024-09-10T08:08:03.675717+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:03.757890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651587+0000","last_change":"2024-09-10T08:05:25.475872+0000","last_active":"2024-09-10T08:08:03.651587+0000","last_peered":"2024-09-10T08:08:03.651587+0000","last_clean":"2024-09-10T08:08:03.651587+0000","last_became_active":"2024-09-10T08:05:25.475549+0000","last_became_peered":"2024-09-10T08:05:25.475549+0000","last_unstale":"2024-09-10T08:08:03.651587+0000","last_undegraded":"2024-09-10T08:08:03.651587+0000","last_fullsized":"2024-09-10T08:08:03.651587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:37.575267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.664667+0000","last_change":"2024-09-10T08:05:25.474210+0000","last_active":"2024-09-10T08:08:03.664667+0000","last_peered":"2024-09-10T08:08:03.664667+0000","last_clean":"2024-09-10T08:08:03.664667+0000","last_became_active":"2024-09-10T08:05:25.473882+0000","last_became_peered":"2024-09-10T08:05:25.473882+0000","last_unstale":"2024-09-10T08:08:03.664667+0000","last_undegraded":"2024-09-10T08:08:03.664667+0000","last_fullsized":"2024-09-10T08:08:03.664667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:05.164370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651519+0000","last_change":"2024-09-10T08:05:25.490406+0000","last_active":"2024-09-10T08:08:03.651519+0000","last_peered":"2024-09-10T08:08:03.651519+0000","last_clean":"2024-09-10T08:08:03.651519+0000","last_became_active":"2024-09-10T08:05:25.490061+0000","last_became_peered":"2024-09-10T08:05:25.490061+0000","last_unstale":"2024-09-10T08:08:03.651519+0000","last_undegraded":"2024-09-10T08:08:03.651519+0000","last_fullsized":"2024-09-10T08:08:03.651519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:12.988229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.664045+0000","last_change":"2024-09-10T08:05:25.504049+0000","last_active":"2024-09-10T08:08:03.664045+0000","last_peered":"2024-09-10T08:08:03.664045+0000","last_clean":"2024-09-10T08:08:03.664045+0000","last_became_active":"2024-09-10T08:05:25.503770+0000","last_became_peered":"2024-09-10T08:05:25.503770+0000","last_unstale":"2024-09-10T08:08:03.664045+0000","last_undegraded":"2024-09-10T08:08:03.664045+0000","last_fullsized":"2024-09-10T08:08:03.664045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:52.850945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.672614+0000","last_change":"2024-09-10T08:05:25.491682+0000","last_active":"2024-09-10T08:08:03.672614+0000","last_peered":"2024-09-10T08:08:03.672614+0000","last_clean":"2024-09-10T08:08:03.672614+0000","last_became_active":"2024-09-10T08:05:25.491296+0000","last_became_peered":"2024-09-10T08:05:25.491296+0000","last_unstale":"2024-09-10T08:08:03.672614+0000","last_undegraded":"2024-09-10T08:08:03.672614+0000","last_fullsized":"2024-09-10T08:08:03.672614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:30.075842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.673509+0000","last_change":"2024-09-10T08:05:25.489648+0000","last_active":"2024-09-10T08:08:03.673509+0000","last_peered":"2024-09-10T08:08:03.673509+0000","last_clean":"2024-09-10T08:08:03.673509+0000","last_became_active":"2024-09-10T08:05:25.489267+0000","last_became_peered":"2024-09-10T08:05:25.489267+0000","last_unstale":"2024-09-10T08:08:03.673509+0000","last_undegraded":"2024-09-10T08:08:03.673509+0000","last_fullsized":"2024-09-10T08:08:03.673509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:21.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681690+0000","last_change":"2024-09-10T08:05:25.497862+0000","last_active":"2024-09-10T08:08:03.681690+0000","last_peered":"2024-09-10T08:08:03.681690+0000","last_clean":"2024-09-10T08:08:03.681690+0000","last_became_active":"2024-09-10T08:05:25.497504+0000","last_became_peered":"2024-09-10T08:05:25.497504+0000","last_unstale":"2024-09-10T08:08:03.681690+0000","last_undegraded":"2024-09-10T08:08:03.681690+0000","last_fullsized":"2024-09-10T08:08:03.681690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:40.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654484+0000","last_change":"2024-09-10T08:05:25.485924+0000","last_active":"2024-09-10T08:08:03.654484+0000","last_peered":"2024-09-10T08:08:03.654484+0000","last_clean":"2024-09-10T08:08:03.654484+0000","last_became_active":"2024-09-10T08:05:25.485545+0000","last_became_peered":"2024-09-10T08:05:25.485545+0000","last_unstale":"2024-09-10T08:08:03.654484+0000","last_undegraded":"2024-09-10T08:08:03.654484+0000","last_fullsized":"2024-09-10T08:08:03.654484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:33:20.233720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671966+0000","last_change":"2024-09-10T08:05:25.491911+0000","last_active":"2024-09-10T08:08:03.671966+0000","last_peered":"2024-09-10T08:08:03.671966+0000","last_clean":"2024-09-10T08:08:03.671966+0000","last_became_active":"2024-09-10T08:05:25.491330+0000","last_became_peered":"2024-09-10T08:05:25.491330+0000","last_unstale":"2024-09-10T08:08:03.671966+0000","last_undegraded":"2024-09-10T08:08:03.671966+0000","last_fullsized":"2024-09-10T08:08:03.671966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:06.043331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.655352+0000","last_change":"2024-09-10T08:05:25.492220+0000","last_active":"2024-09-10T08:08:03.655352+0000","last_peered":"2024-09-10T08:08:03.655352+0000","last_clean":"2024-09-10T08:08:03.655352+0000","last_became_active":"2024-09-10T08:05:25.491473+0000","last_became_peered":"2024-09-10T08:05:25.491473+0000","last_unstale":"2024-09-10T08:08:03.655352+0000","last_undegraded":"2024-09-10T08:08:03.655352+0000","last_fullsized":"2024-09-10T08:08:03.655352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:13:30.278434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676184+0000","last_change":"2024-09-10T08:05:25.488927+0000","last_active":"2024-09-10T08:08:03.676184+0000","last_peered":"2024-09-10T08:08:03.676184+0000","last_clean":"2024-09-10T08:08:03.676184+0000","last_became_active":"2024-09-10T08:05:25.488550+0000","last_became_peered":"2024-09-10T08:05:25.488550+0000","last_unstale":"2024-09-10T08:08:03.676184+0000","last_undegraded":"2024-09-10T08:08:03.676184+0000","last_fullsized":"2024-09-10T08:08:03.676184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:44.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.677654+0000","last_change":"2024-09-10T08:05:25.490125+0000","last_active":"2024-09-10T08:08:03.677654+0000","last_peered":"2024-09-10T08:08:03.677654+0000","last_clean":"2024-09-10T08:08:03.677654+0000","last_became_active":"2024-09-10T08:05:25.487805+0000","last_became_peered":"2024-09-10T08:05:25.487805+0000","last_unstale":"2024-09-10T08:08:03.677654+0000","last_undegraded":"2024-09-10T08:08:03.677654+0000","last_fullsized":"2024-09-10T08:08:03.677654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:33.479009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676794+0000","last_change":"2024-09-10T08:05:25.500346+0000","last_active":"2024-09-10T08:08:03.676794+0000","last_peered":"2024-09-10T08:08:03.676794+0000","last_clean":"2024-09-10T08:08:03.676794+0000","last_became_active":"2024-09-10T08:05:25.498289+0000","last_became_peered":"2024-09-10T08:05:25.498289+0000","last_unstale":"2024-09-10T08:08:03.676794+0000","last_undegraded":"2024-09-10T08:08:03.676794+0000","last_fullsized":"2024-09-10T08:08:03.676794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:12.054926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676196+0000","last_change":"2024-09-10T08:05:25.490001+0000","last_active":"2024-09-10T08:08:03.676196+0000","last_peered":"2024-09-10T08:08:03.676196+0000","last_clean":"2024-09-10T08:08:03.676196+0000","last_became_active":"2024-09-10T08:05:25.489706+0000","last_became_peered":"2024-09-10T08:05:25.489706+0000","last_unstale":"2024-09-10T08:08:03.676196+0000","last_undegraded":"2024-09-10T08:08:03.676196+0000","last_fullsized":"2024-09-10T08:08:03.676196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:38.822140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669334+0000","last_change":"2024-09-10T08:05:25.493874+0000","last_active":"2024-09-10T08:08:03.669334+0000","last_peered":"2024-09-10T08:08:03.669334+0000","last_clean":"2024-09-10T08:08:03.669334+0000","last_became_active":"2024-09-10T08:05:25.489911+0000","last_became_peered":"2024-09-10T08:05:25.489911+0000","last_unstale":"2024-09-10T08:08:03.669334+0000","last_undegraded":"2024-09-10T08:08:03.669334+0000","last_fullsized":"2024-09-10T08:08:03.669334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:46.509657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654960+0000","last_change":"2024-09-10T08:05:25.490277+0000","last_active":"2024-09-10T08:08:03.654960+0000","last_peered":"2024-09-10T08:08:03.654960+0000","last_clean":"2024-09-10T08:08:03.654960+0000","last_became_active":"2024-09-10T08:05:25.486863+0000","last_became_peered":"2024-09-10T08:05:25.486863+0000","last_unstale":"2024-09-10T08:08:03.654960+0000","last_undegraded":"2024-09-10T08:08:03.654960+0000","last_fullsized":"2024-09-10T08:08:03.654960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:21.353830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656794+0000","last_change":"2024-09-10T08:05:25.491028+0000","last_active":"2024-09-10T08:08:03.656794+0000","last_peered":"2024-09-10T08:08:03.656794+0000","last_clean":"2024-09-10T08:08:03.656794+0000","last_became_active":"2024-09-10T08:05:25.490452+0000","last_became_peered":"2024-09-10T08:05:25.490452+0000","last_unstale":"2024-09-10T08:08:03.656794+0000","last_undegraded":"2024-09-10T08:08:03.656794+0000","last_fullsized":"2024-09-10T08:08:03.656794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:53.059077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660628+0000","last_change":"2024-09-10T08:05:25.489991+0000","last_active":"2024-09-10T08:08:03.660628+0000","last_peered":"2024-09-10T08:08:03.660628+0000","last_clean":"2024-09-10T08:08:03.660628+0000","last_became_active":"2024-09-10T08:05:25.489662+0000","last_became_peered":"2024-09-10T08:05:25.489662+0000","last_unstale":"2024-09-10T08:08:03.660628+0000","last_undegraded":"2024-09-10T08:08:03.660628+0000","last_fullsized":"2024-09-10T08:08:03.660628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:47.940712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667243+0000","last_change":"2024-09-10T08:05:25.475752+0000","last_active":"2024-09-10T08:08:03.667243+0000","last_peered":"2024-09-10T08:08:03.667243+0000","last_clean":"2024-09-10T08:08:03.667243+0000","last_became_active":"2024-09-10T08:05:25.475395+0000","last_became_peered":"2024-09-10T08:05:25.475395+0000","last_unstale":"2024-09-10T08:08:03.667243+0000","last_undegraded":"2024-09-10T08:08:03.667243+0000","last_fullsized":"2024-09-10T08:08:03.667243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.240154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.672245+0000","last_change":"2024-09-10T08:05:25.503061+0000","last_active":"2024-09-10T08:08:03.672245+0000","last_peered":"2024-09-10T08:08:03.672245+0000","last_clean":"2024-09-10T08:08:03.672245+0000","last_became_active":"2024-09-10T08:05:25.501333+0000","last_became_peered":"2024-09-10T08:05:25.501333+0000","last_unstale":"2024-09-10T08:08:03.672245+0000","last_undegraded":"2024-09-10T08:08:03.672245+0000","last_fullsized":"2024-09-10T08:08:03.672245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:38.254777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.661256+0000","last_change":"2024-09-10T08:07:05.514631+0000","last_active":"2024-09-10T08:08:03.661256+0000","last_peered":"2024-09-10T08:08:03.661256+0000","last_clean":"2024-09-10T08:08:03.661256+0000","last_became_active":"2024-09-10T08:07:05.514285+0000","last_became_peered":"2024-09-10T08:07:05.514285+0000","last_unstale":"2024-09-10T08:08:03.661256+0000","last_undegraded":"2024-09-10T08:08:03.661256+0000","last_fullsized":"2024-09-10T08:08:03.661256+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:40.293121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660092+0000","last_change":"2024-09-10T08:05:25.491457+0000","last_active":"2024-09-10T08:08:03.660092+0000","last_peered":"2024-09-10T08:08:03.660092+0000","last_clean":"2024-09-10T08:08:03.660092+0000","last_became_active":"2024-09-10T08:05:25.491162+0000","last_became_peered":"2024-09-10T08:05:25.491162+0000","last_unstale":"2024-09-10T08:08:03.660092+0000","last_undegraded":"2024-09-10T08:08:03.660092+0000","last_fullsized":"2024-09-10T08:08:03.660092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:14.389752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660737+0000","last_change":"2024-09-10T08:05:25.487874+0000","last_active":"2024-09-10T08:08:03.660737+0000","last_peered":"2024-09-10T08:08:03.660737+0000","last_clean":"2024-09-10T08:08:03.660737+0000","last_became_active":"2024-09-10T08:05:25.487497+0000","last_became_peered":"2024-09-10T08:05:25.487497+0000","last_unstale":"2024-09-10T08:08:03.660737+0000","last_undegraded":"2024-09-10T08:08:03.660737+0000","last_fullsized":"2024-09-10T08:08:03.660737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:26.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658875+0000","last_change":"2024-09-10T08:05:25.490018+0000","last_active":"2024-09-10T08:08:03.658875+0000","last_peered":"2024-09-10T08:08:03.658875+0000","last_clean":"2024-09-10T08:08:03.658875+0000","last_became_active":"2024-09-10T08:05:25.484951+0000","last_became_peered":"2024-09-10T08:05:25.484951+0000","last_unstale":"2024-09-10T08:08:03.658875+0000","last_undegraded":"2024-09-10T08:08:03.658875+0000","last_fullsized":"2024-09-10T08:08:03.658875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:07.516154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.677343+0000","last_change":"2024-09-10T08:05:25.501581+0000","last_active":"2024-09-10T08:08:03.677343+0000","last_peered":"2024-09-10T08:08:03.677343+0000","last_clean":"2024-09-10T08:08:03.677343+0000","last_became_active":"2024-09-10T08:05:25.499690+0000","last_became_peered":"2024-09-10T08:05:25.499690+0000","last_unstale":"2024-09-10T08:08:03.677343+0000","last_undegraded":"2024-09-10T08:08:03.677343+0000","last_fullsized":"2024-09-10T08:08:03.677343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:14.569791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.678243+0000","last_change":"2024-09-10T08:05:25.487945+0000","last_active":"2024-09-10T08:08:03.678243+0000","last_peered":"2024-09-10T08:08:03.678243+0000","last_clean":"2024-09-10T08:08:03.678243+0000","last_became_active":"2024-09-10T08:05:25.487655+0000","last_became_peered":"2024-09-10T08:05:25.487655+0000","last_unstale":"2024-09-10T08:08:03.678243+0000","last_undegraded":"2024-09-10T08:08:03.678243+0000","last_fullsized":"2024-09-10T08:08:03.678243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:37.746037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675926+0000","last_change":"2024-09-10T08:05:25.489348+0000","last_active":"2024-09-10T08:08:03.675926+0000","last_peered":"2024-09-10T08:08:03.675926+0000","last_clean":"2024-09-10T08:08:03.675926+0000","last_became_active":"2024-09-10T08:05:25.489004+0000","last_became_peered":"2024-09-10T08:05:25.489004+0000","last_unstale":"2024-09-10T08:08:03.675926+0000","last_undegraded":"2024-09-10T08:08:03.675926+0000","last_fullsized":"2024-09-10T08:08:03.675926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:54.554719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74228,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":753,"num_read_kb":456,"num_write":187,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3725,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":221898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214976,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514546,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":548036,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743479,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712792,"statfs":{"total":95995035648,"available":95961899008,"internally_reserved":0,"allocated":1204224,"data_stored":1208758,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907002,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":548250,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135938,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":548148,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364869,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1114112,"data_stored":1144533,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528396,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":543426,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1122304,"data_stored":1140514,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":617456,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":543620,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":545924,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":611930,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":544814,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:08:25.158 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T08:08:25.158 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:24 smithi155 ceph-mon[35405]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-10T08:08:25.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:24 smithi107 ceph-mon[35048]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-10T08:08:25.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.317+0000 7fb8e5800640 1 -- 172.21.15.2:0/2636612369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e0104f20 msgr2=0x7fb8e01053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:25.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.317+0000 7fb8e5800640 1 --2- 172.21.15.2:0/2636612369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e0104f20 0x7fb8e01053a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009900 tx=0x7fb8c802a410 comp rx=0 tx=0).stop 2024-09-10T08:08:25.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.317+0000 7fb8e5800640 1 -- 172.21.15.2:0/2636612369 shutdown_connections 2024-09-10T08:08:25.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.317+0000 7fb8e5800640 1 --2- 172.21.15.2:0/2636612369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8e01058e0 0x7fb8e010c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.317+0000 7fb8e5800640 1 --2- 172.21.15.2:0/2636612369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e0104f20 0x7fb8e01053a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.317+0000 7fb8e5800640 1 --2- 172.21.15.2:0/2636612369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8e0103d30 0x7fb8e0104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.317+0000 7fb8e5800640 1 -- 172.21.15.2:0/2636612369 >> 172.21.15.2:0/2636612369 conn(0x7fb8e00ff4b0 msgr2=0x7fb8e01018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:25.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.317+0000 7fb8e5800640 1 -- 172.21.15.2:0/2636612369 shutdown_connections 2024-09-10T08:08:25.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.317+0000 7fb8e5800640 1 -- 172.21.15.2:0/2636612369 wait complete. 2024-09-10T08:08:25.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.318+0000 7fb8e5800640 1 Processor -- start 2024-09-10T08:08:25.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.318+0000 7fb8e5800640 1 -- start start 2024-09-10T08:08:25.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8e5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8e0103d30 0x7fb8e019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:25.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8e5800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e0104f20 0x7fb8e019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:25.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8e5800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8e01058e0 0x7fb8e01a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:25.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8e5800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e010f9b0 con 0x7fb8e0104f20 2024-09-10T08:08:25.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8e5800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e010f830 con 0x7fb8e01058e0 2024-09-10T08:08:25.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8e5800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e010fb30 con 0x7fb8e0103d30 2024-09-10T08:08:25.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8d7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e0104f20 0x7fb8e019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:25.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8dea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8e0103d30 0x7fb8e019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:25.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8df400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8e01058e0 0x7fb8e01a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:25.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8d7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e0104f20 0x7fb8e019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40006/0 (socket says 172.21.15.2:40006) 2024-09-10T08:08:25.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8d7e00640 1 -- 172.21.15.2:0/2727296273 learned_addr learned my addr 172.21.15.2:0/2727296273 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:25.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8df400640 1 -- 172.21.15.2:0/2727296273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8e0103d30 msgr2=0x7fb8e019a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:25.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8df400640 1 --2- 172.21.15.2:0/2727296273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8e0103d30 0x7fb8e019a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8df400640 1 -- 172.21.15.2:0/2727296273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e0104f20 msgr2=0x7fb8e019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:25.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8df400640 1 --2- 172.21.15.2:0/2727296273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e0104f20 0x7fb8e019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8df400640 1 -- 172.21.15.2:0/2727296273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e01a1cb0 con 0x7fb8e01058e0 2024-09-10T08:08:25.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.319+0000 7fb8dea00640 1 --2- 172.21.15.2:0/2727296273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8e0103d30 0x7fb8e019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:25.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.320+0000 7fb8df400640 1 --2- 172.21.15.2:0/2727296273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8e01058e0 0x7fb8e01a1640 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800da10 tx=0x7fb8d800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:25.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.320+0000 7fb8d7400640 1 -- 172.21.15.2:0/2727296273 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d800bd70 con 0x7fb8e01058e0 2024-09-10T08:08:25.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.320+0000 7fb8d7400640 1 -- 172.21.15.2:0/2727296273 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8d8003e50 con 0x7fb8e01058e0 2024-09-10T08:08:25.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.320+0000 7fb8e5800640 1 -- 172.21.15.2:0/2727296273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e01a1fa0 con 0x7fb8e01058e0 2024-09-10T08:08:25.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.320+0000 7fb8d7400640 1 -- 172.21.15.2:0/2727296273 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d80106a0 con 0x7fb8e01058e0 2024-09-10T08:08:25.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.320+0000 7fb8e5800640 1 -- 172.21.15.2:0/2727296273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e01a2410 con 0x7fb8e01058e0 2024-09-10T08:08:25.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.321+0000 7fb8e5800640 1 -- 172.21.15.2:0/2727296273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e010b1c0 con 0x7fb8e01058e0 2024-09-10T08:08:25.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.323+0000 7fb8d7400640 1 -- 172.21.15.2:0/2727296273 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fb8d8004b40 con 0x7fb8e01058e0 2024-09-10T08:08:25.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.324+0000 7fb8d7400640 1 --2- 172.21.15.2:0/2727296273 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb8c40a2c20 0x7fb8c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:25.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.324+0000 7fb8d7400640 1 -- 172.21.15.2:0/2727296273 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb8d80cfb20 con 0x7fb8e01058e0 2024-09-10T08:08:25.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.325+0000 7fb8dea00640 1 --2- 172.21.15.2:0/2727296273 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb8c40a2c20 0x7fb8c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:25.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.325+0000 7fb8dea00640 1 --2- 172.21.15.2:0/2727296273 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb8c40a2c20 0x7fb8c40a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0104d80 tx=0x7fb8d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:25.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.327+0000 7fb8d7400640 1 -- 172.21.15.2:0/2727296273 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8d809c4d0 con 0x7fb8e01058e0 2024-09-10T08:08:25.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.508+0000 7fb8e5800640 1 -- 172.21.15.2:0/2727296273 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8e0062630 con 0x7fb8c40a2c20 2024-09-10T08:08:25.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.516+0000 7fb8d7400640 1 -- 172.21.15.2:0/2727296273 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340724 (secure 0 0 0) 0x7fb8e0062630 con 0x7fb8c40a2c20 2024-09-10T08:08:25.520 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:25.522 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-10T08:08:25.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.521+0000 7fb8e5800640 1 -- 172.21.15.2:0/2727296273 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb8c40a2c20 msgr2=0x7fb8c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.521+0000 7fb8e5800640 1 --2- 172.21.15.2:0/2727296273 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb8c40a2c20 0x7fb8c40a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0104d80 tx=0x7fb8d0005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.521+0000 7fb8e5800640 1 -- 172.21.15.2:0/2727296273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8e01058e0 msgr2=0x7fb8e01a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.521+0000 7fb8e5800640 1 --2- 172.21.15.2:0/2727296273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8e01058e0 0x7fb8e01a1640 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800da10 tx=0x7fb8d800ded0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.521+0000 7fb8e5800640 1 -- 172.21.15.2:0/2727296273 shutdown_connections 2024-09-10T08:08:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.521+0000 7fb8e5800640 1 --2- 172.21.15.2:0/2727296273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8e01058e0 0x7fb8e01a1640 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.521+0000 7fb8e5800640 1 --2- 172.21.15.2:0/2727296273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e0104f20 0x7fb8e019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.521+0000 7fb8e5800640 1 --2- 172.21.15.2:0/2727296273 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb8c40a2c20 0x7fb8c40a50e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.521+0000 7fb8e5800640 1 --2- 172.21.15.2:0/2727296273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8e0103d30 0x7fb8e019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.521+0000 7fb8e5800640 1 -- 172.21.15.2:0/2727296273 >> 172.21.15.2:0/2727296273 conn(0x7fb8e00ff4b0 msgr2=0x7fb8e0108390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.521+0000 7fb8e5800640 1 -- 172.21.15.2:0/2727296273 shutdown_connections 2024-09-10T08:08:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.521+0000 7fb8e5800640 1 -- 172.21.15.2:0/2727296273 wait complete. 2024-09-10T08:08:25.537 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-10T08:08:23.693024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664830,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":879,"num_read_kb":565,"num_write":370,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1184461,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":798,"ondisk_log_size":798,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362048,"kb_used_data":12800,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579776,"statfs":{"total":1151940427776,"available":1151569690624,"internally_reserved":0,"allocated":13107200,"data_stored":8545409,"data_compressed":497802,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7174,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":46,"num_write":13,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003532"},"pg_stats":[{"pgid":"2.1f","version":"93'9","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514795+0000","last_change":"2024-09-10T08:05:22.462062+0000","last_active":"2024-09-10T08:07:05.514795+0000","last_peered":"2024-09-10T08:07:05.514795+0000","last_clean":"2024-09-10T08:07:05.514795+0000","last_became_active":"2024-09-10T08:05:22.460973+0000","last_became_peered":"2024-09-10T08:05:22.460973+0000","last_unstale":"2024-09-10T08:07:05.514795+0000","last_undegraded":"2024-09-10T08:07:05.514795+0000","last_fullsized":"2024-09-10T08:07:05.514795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:01.357446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656803+0000","last_change":"2024-09-10T08:05:25.482873+0000","last_active":"2024-09-10T08:08:03.656803+0000","last_peered":"2024-09-10T08:08:03.656803+0000","last_clean":"2024-09-10T08:08:03.656803+0000","last_became_active":"2024-09-10T08:05:25.482559+0000","last_became_peered":"2024-09-10T08:05:25.482559+0000","last_unstale":"2024-09-10T08:08:03.656803+0000","last_undegraded":"2024-09-10T08:08:03.656803+0000","last_fullsized":"2024-09-10T08:08:03.656803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:10:17.366160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511384+0000","last_change":"2024-09-10T08:06:04.736356+0000","last_active":"2024-09-10T08:07:05.511384+0000","last_peered":"2024-09-10T08:07:05.511384+0000","last_clean":"2024-09-10T08:07:05.511384+0000","last_became_active":"2024-09-10T08:06:04.735859+0000","last_became_peered":"2024-09-10T08:06:04.735859+0000","last_unstale":"2024-09-10T08:07:05.511384+0000","last_undegraded":"2024-09-10T08:07:05.511384+0000","last_fullsized":"2024-09-10T08:07:05.511384+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:56.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"93'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.877433+0000","last_change":"2024-09-10T08:05:22.569504+0000","last_active":"2024-09-10T08:07:05.877433+0000","last_peered":"2024-09-10T08:07:05.877433+0000","last_clean":"2024-09-10T08:07:05.877433+0000","last_became_active":"2024-09-10T08:05:22.569164+0000","last_became_peered":"2024-09-10T08:05:22.569164+0000","last_unstale":"2024-09-10T08:07:05.877433+0000","last_undegraded":"2024-09-10T08:07:05.877433+0000","last_fullsized":"2024-09-10T08:07:05.877433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:41.628963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.680208+0000","last_change":"2024-09-10T08:05:25.493017+0000","last_active":"2024-09-10T08:08:03.680208+0000","last_peered":"2024-09-10T08:08:03.680208+0000","last_clean":"2024-09-10T08:08:03.680208+0000","last_became_active":"2024-09-10T08:05:25.491370+0000","last_became_peered":"2024-09-10T08:05:25.491370+0000","last_unstale":"2024-09-10T08:08:03.680208+0000","last_undegraded":"2024-09-10T08:08:03.680208+0000","last_fullsized":"2024-09-10T08:08:03.680208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:41.616549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914087+0000","last_change":"2024-09-10T08:05:35.523751+0000","last_active":"2024-09-10T08:07:05.914087+0000","last_peered":"2024-09-10T08:07:05.914087+0000","last_clean":"2024-09-10T08:07:05.914087+0000","last_became_active":"2024-09-10T08:05:35.523372+0000","last_became_peered":"2024-09-10T08:05:35.523372+0000","last_unstale":"2024-09-10T08:07:05.914087+0000","last_undegraded":"2024-09-10T08:07:05.914087+0000","last_fullsized":"2024-09-10T08:07:05.914087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:20.971596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508934+0000","last_change":"2024-09-10T08:05:22.460590+0000","last_active":"2024-09-10T08:07:05.508934+0000","last_peered":"2024-09-10T08:07:05.508934+0000","last_clean":"2024-09-10T08:07:05.508934+0000","last_became_active":"2024-09-10T08:05:22.460238+0000","last_became_peered":"2024-09-10T08:05:22.460238+0000","last_unstale":"2024-09-10T08:07:05.508934+0000","last_undegraded":"2024-09-10T08:07:05.508934+0000","last_fullsized":"2024-09-10T08:07:05.508934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:18.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669035+0000","last_change":"2024-09-10T08:05:25.487392+0000","last_active":"2024-09-10T08:08:03.669035+0000","last_peered":"2024-09-10T08:08:03.669035+0000","last_clean":"2024-09-10T08:08:03.669035+0000","last_became_active":"2024-09-10T08:05:25.487054+0000","last_became_peered":"2024-09-10T08:05:25.487054+0000","last_unstale":"2024-09-10T08:08:03.669035+0000","last_undegraded":"2024-09-10T08:08:03.669035+0000","last_fullsized":"2024-09-10T08:08:03.669035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:16.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914818+0000","last_change":"2024-09-10T08:06:04.735778+0000","last_active":"2024-09-10T08:07:05.914818+0000","last_peered":"2024-09-10T08:07:05.914818+0000","last_clean":"2024-09-10T08:07:05.914818+0000","last_became_active":"2024-09-10T08:06:04.735208+0000","last_became_peered":"2024-09-10T08:06:04.735208+0000","last_unstale":"2024-09-10T08:07:05.914818+0000","last_undegraded":"2024-09-10T08:07:05.914818+0000","last_fullsized":"2024-09-10T08:07:05.914818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:36.982246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508944+0000","last_change":"2024-09-10T08:05:22.458323+0000","last_active":"2024-09-10T08:07:05.508944+0000","last_peered":"2024-09-10T08:07:05.508944+0000","last_clean":"2024-09-10T08:07:05.508944+0000","last_became_active":"2024-09-10T08:05:22.457965+0000","last_became_peered":"2024-09-10T08:05:22.457965+0000","last_unstale":"2024-09-10T08:07:05.508944+0000","last_undegraded":"2024-09-10T08:07:05.508944+0000","last_fullsized":"2024-09-10T08:07:05.508944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:15.021305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.663997+0000","last_change":"2024-09-10T08:05:25.489883+0000","last_active":"2024-09-10T08:08:03.663997+0000","last_peered":"2024-09-10T08:08:03.663997+0000","last_clean":"2024-09-10T08:08:03.663997+0000","last_became_active":"2024-09-10T08:05:25.489482+0000","last_became_peered":"2024-09-10T08:05:25.489482+0000","last_unstale":"2024-09-10T08:08:03.663997+0000","last_undegraded":"2024-09-10T08:08:03.663997+0000","last_fullsized":"2024-09-10T08:08:03.663997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:52.080628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914064+0000","last_change":"2024-09-10T08:05:35.531458+0000","last_active":"2024-09-10T08:07:05.914064+0000","last_peered":"2024-09-10T08:07:05.914064+0000","last_clean":"2024-09-10T08:07:05.914064+0000","last_became_active":"2024-09-10T08:05:35.531072+0000","last_became_peered":"2024-09-10T08:05:35.531072+0000","last_unstale":"2024-09-10T08:07:05.914064+0000","last_undegraded":"2024-09-10T08:07:05.914064+0000","last_fullsized":"2024-09-10T08:07:05.914064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:04.792664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667650+0000","last_change":"2024-09-10T08:05:25.486865+0000","last_active":"2024-09-10T08:08:03.667650+0000","last_peered":"2024-09-10T08:08:03.667650+0000","last_clean":"2024-09-10T08:08:03.667650+0000","last_became_active":"2024-09-10T08:05:25.486531+0000","last_became_peered":"2024-09-10T08:05:25.486531+0000","last_unstale":"2024-09-10T08:08:03.667650+0000","last_undegraded":"2024-09-10T08:08:03.667650+0000","last_fullsized":"2024-09-10T08:08:03.667650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:26.854682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"93'4","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508917+0000","last_change":"2024-09-10T08:06:04.743303+0000","last_active":"2024-09-10T08:07:05.508917+0000","last_peered":"2024-09-10T08:07:05.508917+0000","last_clean":"2024-09-10T08:07:05.508917+0000","last_became_active":"2024-09-10T08:06:04.733294+0000","last_became_peered":"2024-09-10T08:06:04.733294+0000","last_unstale":"2024-09-10T08:07:05.508917+0000","last_undegraded":"2024-09-10T08:07:05.508917+0000","last_fullsized":"2024-09-10T08:07:05.508917+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:18.542658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914822+0000","last_change":"2024-09-10T08:05:35.642083+0000","last_active":"2024-09-10T08:07:05.914822+0000","last_peered":"2024-09-10T08:07:05.914822+0000","last_clean":"2024-09-10T08:07:05.914822+0000","last_became_active":"2024-09-10T08:05:35.641214+0000","last_became_peered":"2024-09-10T08:05:35.641214+0000","last_unstale":"2024-09-10T08:07:05.914822+0000","last_undegraded":"2024-09-10T08:07:05.914822+0000","last_fullsized":"2024-09-10T08:07:05.914822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:00:21.731769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681852+0000","last_change":"2024-09-10T08:05:25.501993+0000","last_active":"2024-09-10T08:08:03.681852+0000","last_peered":"2024-09-10T08:08:03.681852+0000","last_clean":"2024-09-10T08:08:03.681852+0000","last_became_active":"2024-09-10T08:05:25.499912+0000","last_became_peered":"2024-09-10T08:05:25.499912+0000","last_unstale":"2024-09-10T08:08:03.681852+0000","last_undegraded":"2024-09-10T08:08:03.681852+0000","last_fullsized":"2024-09-10T08:08:03.681852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:06.780067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:36.268711+0000","last_change":"2024-09-10T08:05:22.461786+0000","last_active":"2024-09-10T08:07:36.268711+0000","last_peered":"2024-09-10T08:07:36.268711+0000","last_clean":"2024-09-10T08:07:36.268711+0000","last_became_active":"2024-09-10T08:05:22.460821+0000","last_became_peered":"2024-09-10T08:05:22.460821+0000","last_unstale":"2024-09-10T08:07:36.268711+0000","last_undegraded":"2024-09-10T08:07:36.268711+0000","last_fullsized":"2024-09-10T08:07:36.268711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:25.083538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914568+0000","last_change":"2024-09-10T08:06:04.734470+0000","last_active":"2024-09-10T08:07:05.914568+0000","last_peered":"2024-09-10T08:07:05.914568+0000","last_clean":"2024-09-10T08:07:05.914568+0000","last_became_active":"2024-09-10T08:06:04.734040+0000","last_became_peered":"2024-09-10T08:06:04.734040+0000","last_unstale":"2024-09-10T08:07:05.914568+0000","last_undegraded":"2024-09-10T08:07:05.914568+0000","last_fullsized":"2024-09-10T08:07:05.914568+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:36.312144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675820+0000","last_change":"2024-09-10T08:05:25.492711+0000","last_active":"2024-09-10T08:08:03.675820+0000","last_peered":"2024-09-10T08:08:03.675820+0000","last_clean":"2024-09-10T08:08:03.675820+0000","last_became_active":"2024-09-10T08:05:25.492336+0000","last_became_peered":"2024-09-10T08:05:25.492336+0000","last_unstale":"2024-09-10T08:08:03.675820+0000","last_undegraded":"2024-09-10T08:08:03.675820+0000","last_fullsized":"2024-09-10T08:08:03.675820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:04.168884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"93'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876783+0000","last_change":"2024-09-10T08:05:22.461535+0000","last_active":"2024-09-10T08:07:05.876783+0000","last_peered":"2024-09-10T08:07:05.876783+0000","last_clean":"2024-09-10T08:07:05.876783+0000","last_became_active":"2024-09-10T08:05:22.461208+0000","last_became_peered":"2024-09-10T08:05:22.461208+0000","last_unstale":"2024-09-10T08:07:05.876783+0000","last_undegraded":"2024-09-10T08:07:05.876783+0000","last_fullsized":"2024-09-10T08:07:05.876783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:26.009242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513889+0000","last_change":"2024-09-10T08:05:35.535442+0000","last_active":"2024-09-10T08:07:05.513889+0000","last_peered":"2024-09-10T08:07:05.513889+0000","last_clean":"2024-09-10T08:07:05.513889+0000","last_became_active":"2024-09-10T08:05:35.535058+0000","last_became_peered":"2024-09-10T08:05:35.535058+0000","last_unstale":"2024-09-10T08:07:05.513889+0000","last_undegraded":"2024-09-10T08:07:05.513889+0000","last_fullsized":"2024-09-10T08:07:05.513889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:39.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.627539+0000","last_change":"2024-09-10T08:05:25.483440+0000","last_active":"2024-09-10T08:08:03.627539+0000","last_peered":"2024-09-10T08:08:03.627539+0000","last_clean":"2024-09-10T08:08:03.627539+0000","last_became_active":"2024-09-10T08:05:25.483141+0000","last_became_peered":"2024-09-10T08:05:25.483141+0000","last_unstale":"2024-09-10T08:08:03.627539+0000","last_undegraded":"2024-09-10T08:08:03.627539+0000","last_fullsized":"2024-09-10T08:08:03.627539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:00.953188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"93'2","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914337+0000","last_change":"2024-09-10T08:05:22.568141+0000","last_active":"2024-09-10T08:07:05.914337+0000","last_peered":"2024-09-10T08:07:05.914337+0000","last_clean":"2024-09-10T08:07:05.914337+0000","last_became_active":"2024-09-10T08:05:22.567857+0000","last_became_peered":"2024-09-10T08:05:22.567857+0000","last_unstale":"2024-09-10T08:07:05.914337+0000","last_undegraded":"2024-09-10T08:07:05.914337+0000","last_fullsized":"2024-09-10T08:07:05.914337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:22.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914846+0000","last_change":"2024-09-10T08:05:35.641993+0000","last_active":"2024-09-10T08:07:05.914846+0000","last_peered":"2024-09-10T08:07:05.914846+0000","last_clean":"2024-09-10T08:07:05.914846+0000","last_became_active":"2024-09-10T08:05:35.641376+0000","last_became_peered":"2024-09-10T08:05:35.641376+0000","last_unstale":"2024-09-10T08:07:05.914846+0000","last_undegraded":"2024-09-10T08:07:05.914846+0000","last_fullsized":"2024-09-10T08:07:05.914846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:16:34.038924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.678764+0000","last_change":"2024-09-10T08:06:04.737560+0000","last_active":"2024-09-10T08:08:03.678764+0000","last_peered":"2024-09-10T08:08:03.678764+0000","last_clean":"2024-09-10T08:08:03.678764+0000","last_became_active":"2024-09-10T08:06:04.736656+0000","last_became_peered":"2024-09-10T08:06:04.736656+0000","last_unstale":"2024-09-10T08:08:03.678764+0000","last_undegraded":"2024-09-10T08:08:03.678764+0000","last_fullsized":"2024-09-10T08:08:03.678764+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:35.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"93'4","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915999+0000","last_change":"2024-09-10T08:05:22.567077+0000","last_active":"2024-09-10T08:07:05.915999+0000","last_peered":"2024-09-10T08:07:05.915999+0000","last_clean":"2024-09-10T08:07:05.915999+0000","last_became_active":"2024-09-10T08:05:22.566757+0000","last_became_peered":"2024-09-10T08:05:22.566757+0000","last_unstale":"2024-09-10T08:07:05.915999+0000","last_undegraded":"2024-09-10T08:07:05.915999+0000","last_fullsized":"2024-09-10T08:07:05.915999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:06.881510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511524+0000","last_change":"2024-09-10T08:05:35.519539+0000","last_active":"2024-09-10T08:07:05.511524+0000","last_peered":"2024-09-10T08:07:05.511524+0000","last_clean":"2024-09-10T08:07:05.511524+0000","last_became_active":"2024-09-10T08:05:35.519044+0000","last_became_peered":"2024-09-10T08:05:35.519044+0000","last_unstale":"2024-09-10T08:07:05.511524+0000","last_undegraded":"2024-09-10T08:07:05.511524+0000","last_fullsized":"2024-09-10T08:07:05.511524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:10.546949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658164+0000","last_change":"2024-09-10T08:07:05.771774+0000","last_active":"2024-09-10T08:08:03.658164+0000","last_peered":"2024-09-10T08:08:03.658164+0000","last_clean":"2024-09-10T08:08:03.658164+0000","last_became_active":"2024-09-10T08:07:05.771336+0000","last_became_peered":"2024-09-10T08:07:05.771336+0000","last_unstale":"2024-09-10T08:08:03.658164+0000","last_undegraded":"2024-09-10T08:08:03.658164+0000","last_fullsized":"2024-09-10T08:08:03.658164+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:11.482222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"95'13","reported_seq":67,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:04.516173+0000","last_change":"2024-09-10T08:06:04.741199+0000","last_active":"2024-09-10T08:08:04.516173+0000","last_peered":"2024-09-10T08:08:04.516173+0000","last_clean":"2024-09-10T08:08:04.516173+0000","last_became_active":"2024-09-10T08:06:04.738076+0000","last_became_peered":"2024-09-10T08:06:04.738076+0000","last_unstale":"2024-09-10T08:08:04.516173+0000","last_undegraded":"2024-09-10T08:08:04.516173+0000","last_fullsized":"2024-09-10T08:08:04.516173+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:45.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876462+0000","last_change":"2024-09-10T08:05:35.642358+0000","last_active":"2024-09-10T08:07:05.876462+0000","last_peered":"2024-09-10T08:07:05.876462+0000","last_clean":"2024-09-10T08:07:05.876462+0000","last_became_active":"2024-09-10T08:05:35.641408+0000","last_became_peered":"2024-09-10T08:05:35.641408+0000","last_unstale":"2024-09-10T08:07:05.876462+0000","last_undegraded":"2024-09-10T08:07:05.876462+0000","last_fullsized":"2024-09-10T08:07:05.876462+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:42.015139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658430+0000","last_change":"2024-09-10T08:05:25.486310+0000","last_active":"2024-09-10T08:08:03.658430+0000","last_peered":"2024-09-10T08:08:03.658430+0000","last_clean":"2024-09-10T08:08:03.658430+0000","last_became_active":"2024-09-10T08:05:25.485915+0000","last_became_peered":"2024-09-10T08:05:25.485915+0000","last_unstale":"2024-09-10T08:08:03.658430+0000","last_undegraded":"2024-09-10T08:08:03.658430+0000","last_fullsized":"2024-09-10T08:08:03.658430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:56.064361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"93'1","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508987+0000","last_change":"2024-09-10T08:05:22.570595+0000","last_active":"2024-09-10T08:07:05.508987+0000","last_peered":"2024-09-10T08:07:05.508987+0000","last_clean":"2024-09-10T08:07:05.508987+0000","last_became_active":"2024-09-10T08:05:22.570219+0000","last_became_peered":"2024-09-10T08:05:22.570219+0000","last_unstale":"2024-09-10T08:07:05.508987+0000","last_undegraded":"2024-09-10T08:07:05.508987+0000","last_fullsized":"2024-09-10T08:07:05.508987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:03.230325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769927+0000","last_change":"2024-09-10T08:05:35.530472+0000","last_active":"2024-09-10T08:07:05.769927+0000","last_peered":"2024-09-10T08:07:05.769927+0000","last_clean":"2024-09-10T08:07:05.769927+0000","last_became_active":"2024-09-10T08:05:35.530066+0000","last_became_peered":"2024-09-10T08:05:35.530066+0000","last_unstale":"2024-09-10T08:07:05.769927+0000","last_undegraded":"2024-09-10T08:07:05.769927+0000","last_fullsized":"2024-09-10T08:07:05.769927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:58.915767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671834+0000","last_change":"2024-09-10T08:05:25.500524+0000","last_active":"2024-09-10T08:08:03.671834+0000","last_peered":"2024-09-10T08:08:03.671834+0000","last_clean":"2024-09-10T08:08:03.671834+0000","last_became_active":"2024-09-10T08:05:25.500186+0000","last_became_peered":"2024-09-10T08:05:25.500186+0000","last_unstale":"2024-09-10T08:08:03.671834+0000","last_undegraded":"2024-09-10T08:08:03.671834+0000","last_fullsized":"2024-09-10T08:08:03.671834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:51.936222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513172+0000","last_change":"2024-09-10T08:05:22.569326+0000","last_active":"2024-09-10T08:07:05.513172+0000","last_peered":"2024-09-10T08:07:05.513172+0000","last_clean":"2024-09-10T08:07:05.513172+0000","last_became_active":"2024-09-10T08:05:22.568895+0000","last_became_peered":"2024-09-10T08:05:22.568895+0000","last_unstale":"2024-09-10T08:07:05.513172+0000","last_undegraded":"2024-09-10T08:07:05.513172+0000","last_fullsized":"2024-09-10T08:07:05.513172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:47.212073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508368+0000","last_change":"2024-09-10T08:05:35.641600+0000","last_active":"2024-09-10T08:07:05.508368+0000","last_peered":"2024-09-10T08:07:05.508368+0000","last_clean":"2024-09-10T08:07:05.508368+0000","last_became_active":"2024-09-10T08:05:35.641205+0000","last_became_peered":"2024-09-10T08:05:35.641205+0000","last_unstale":"2024-09-10T08:07:05.508368+0000","last_undegraded":"2024-09-10T08:07:05.508368+0000","last_fullsized":"2024-09-10T08:07:05.508368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:40.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.659935+0000","last_change":"2024-09-10T08:05:25.495035+0000","last_active":"2024-09-10T08:08:03.659935+0000","last_peered":"2024-09-10T08:08:03.659935+0000","last_clean":"2024-09-10T08:08:03.659935+0000","last_became_active":"2024-09-10T08:05:25.494733+0000","last_became_peered":"2024-09-10T08:05:25.494733+0000","last_unstale":"2024-09-10T08:08:03.659935+0000","last_undegraded":"2024-09-10T08:08:03.659935+0000","last_fullsized":"2024-09-10T08:08:03.659935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.875867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"93'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509485+0000","last_change":"2024-09-10T08:05:22.458727+0000","last_active":"2024-09-10T08:07:05.509485+0000","last_peered":"2024-09-10T08:07:05.509485+0000","last_clean":"2024-09-10T08:07:05.509485+0000","last_became_active":"2024-09-10T08:05:22.458397+0000","last_became_peered":"2024-09-10T08:05:22.458397+0000","last_unstale":"2024-09-10T08:07:05.509485+0000","last_undegraded":"2024-09-10T08:07:05.509485+0000","last_fullsized":"2024-09-10T08:07:05.509485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:07.875559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914403+0000","last_change":"2024-09-10T08:05:35.534192+0000","last_active":"2024-09-10T08:07:05.914403+0000","last_peered":"2024-09-10T08:07:05.914403+0000","last_clean":"2024-09-10T08:07:05.914403+0000","last_became_active":"2024-09-10T08:05:35.530450+0000","last_became_peered":"2024-09-10T08:05:35.530450+0000","last_unstale":"2024-09-10T08:07:05.914403+0000","last_undegraded":"2024-09-10T08:07:05.914403+0000","last_fullsized":"2024-09-10T08:07:05.914403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:10.039144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651611+0000","last_change":"2024-09-10T08:05:25.482252+0000","last_active":"2024-09-10T08:08:03.651611+0000","last_peered":"2024-09-10T08:08:03.651611+0000","last_clean":"2024-09-10T08:08:03.651611+0000","last_became_active":"2024-09-10T08:05:25.481901+0000","last_became_peered":"2024-09-10T08:05:25.481901+0000","last_unstale":"2024-09-10T08:08:03.651611+0000","last_undegraded":"2024-09-10T08:08:03.651611+0000","last_fullsized":"2024-09-10T08:08:03.651611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:17.063340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":170,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:21.446702+0000","last_change":"2024-09-10T08:05:22.569199+0000","last_active":"2024-09-10T08:08:21.446702+0000","last_peered":"2024-09-10T08:08:21.446702+0000","last_clean":"2024-09-10T08:08:21.446702+0000","last_became_active":"2024-09-10T08:05:22.568787+0000","last_became_peered":"2024-09-10T08:05:22.568787+0000","last_unstale":"2024-09-10T08:08:21.446702+0000","last_undegraded":"2024-09-10T08:08:21.446702+0000","last_fullsized":"2024-09-10T08:08:21.446702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:59.610130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914677+0000","last_change":"2024-09-10T08:05:35.642446+0000","last_active":"2024-09-10T08:07:05.914677+0000","last_peered":"2024-09-10T08:07:05.914677+0000","last_clean":"2024-09-10T08:07:05.914677+0000","last_became_active":"2024-09-10T08:05:35.641543+0000","last_became_peered":"2024-09-10T08:05:35.641543+0000","last_unstale":"2024-09-10T08:07:05.914677+0000","last_undegraded":"2024-09-10T08:07:05.914677+0000","last_fullsized":"2024-09-10T08:07:05.914677+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:33.658406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.652192+0000","last_change":"2024-09-10T08:05:25.487598+0000","last_active":"2024-09-10T08:08:03.652192+0000","last_peered":"2024-09-10T08:08:03.652192+0000","last_clean":"2024-09-10T08:08:03.652192+0000","last_became_active":"2024-09-10T08:05:25.486879+0000","last_became_peered":"2024-09-10T08:05:25.486879+0000","last_unstale":"2024-09-10T08:08:03.652192+0000","last_undegraded":"2024-09-10T08:08:03.652192+0000","last_fullsized":"2024-09-10T08:08:03.652192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:20.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509934+0000","last_change":"2024-09-10T08:05:22.463416+0000","last_active":"2024-09-10T08:07:05.509934+0000","last_peered":"2024-09-10T08:07:05.509934+0000","last_clean":"2024-09-10T08:07:05.509934+0000","last_became_active":"2024-09-10T08:05:22.463004+0000","last_became_peered":"2024-09-10T08:05:22.463004+0000","last_unstale":"2024-09-10T08:07:05.509934+0000","last_undegraded":"2024-09-10T08:07:05.509934+0000","last_fullsized":"2024-09-10T08:07:05.509934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:56.257812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569364+0000","last_change":"2024-09-10T08:05:35.642059+0000","last_active":"2024-09-10T08:07:05.569364+0000","last_peered":"2024-09-10T08:07:05.569364+0000","last_clean":"2024-09-10T08:07:05.569364+0000","last_became_active":"2024-09-10T08:05:35.641540+0000","last_became_peered":"2024-09-10T08:05:35.641540+0000","last_unstale":"2024-09-10T08:07:05.569364+0000","last_undegraded":"2024-09-10T08:07:05.569364+0000","last_fullsized":"2024-09-10T08:07:05.569364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:28.557584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.670571+0000","last_change":"2024-09-10T08:05:25.496111+0000","last_active":"2024-09-10T08:08:03.670571+0000","last_peered":"2024-09-10T08:08:03.670571+0000","last_clean":"2024-09-10T08:08:03.670571+0000","last_became_active":"2024-09-10T08:05:25.495757+0000","last_became_peered":"2024-09-10T08:05:25.495757+0000","last_unstale":"2024-09-10T08:08:03.670571+0000","last_undegraded":"2024-09-10T08:08:03.670571+0000","last_fullsized":"2024-09-10T08:08:03.670571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:41.078205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'3","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915005+0000","last_change":"2024-09-10T08:05:22.460069+0000","last_active":"2024-09-10T08:07:05.915005+0000","last_peered":"2024-09-10T08:07:05.915005+0000","last_clean":"2024-09-10T08:07:05.915005+0000","last_became_active":"2024-09-10T08:05:22.459557+0000","last_became_peered":"2024-09-10T08:05:22.459557+0000","last_unstale":"2024-09-10T08:07:05.915005+0000","last_undegraded":"2024-09-10T08:07:05.915005+0000","last_fullsized":"2024-09-10T08:07:05.915005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:49.058507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509959+0000","last_change":"2024-09-10T08:04:08.188262+0000","last_active":"2024-09-10T08:07:05.509959+0000","last_peered":"2024-09-10T08:07:05.509959+0000","last_clean":"2024-09-10T08:07:05.509959+0000","last_became_active":"2024-09-10T08:04:08.187691+0000","last_became_peered":"2024-09-10T08:04:08.187691+0000","last_unstale":"2024-09-10T08:07:05.509959+0000","last_undegraded":"2024-09-10T08:07:05.509959+0000","last_fullsized":"2024-09-10T08:07:05.509959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_clean_scrub_stamp":"2024-09-10T08:00:26.235127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:15.358818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509550+0000","last_change":"2024-09-10T08:05:35.528371+0000","last_active":"2024-09-10T08:07:05.509550+0000","last_peered":"2024-09-10T08:07:05.509550+0000","last_clean":"2024-09-10T08:07:05.509550+0000","last_became_active":"2024-09-10T08:05:35.527925+0000","last_became_peered":"2024-09-10T08:05:35.527925+0000","last_unstale":"2024-09-10T08:07:05.509550+0000","last_undegraded":"2024-09-10T08:07:05.509550+0000","last_fullsized":"2024-09-10T08:07:05.509550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:37:00.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667384+0000","last_change":"2024-09-10T08:05:25.490753+0000","last_active":"2024-09-10T08:08:03.667384+0000","last_peered":"2024-09-10T08:08:03.667384+0000","last_clean":"2024-09-10T08:08:03.667384+0000","last_became_active":"2024-09-10T08:05:25.489996+0000","last_became_peered":"2024-09-10T08:05:25.489996+0000","last_unstale":"2024-09-10T08:08:03.667384+0000","last_undegraded":"2024-09-10T08:08:03.667384+0000","last_fullsized":"2024-09-10T08:08:03.667384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:15.637199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'3","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511918+0000","last_change":"2024-09-10T08:05:22.568696+0000","last_active":"2024-09-10T08:07:05.511918+0000","last_peered":"2024-09-10T08:07:05.511918+0000","last_clean":"2024-09-10T08:07:05.511918+0000","last_became_active":"2024-09-10T08:05:22.568331+0000","last_became_peered":"2024-09-10T08:05:22.568331+0000","last_unstale":"2024-09-10T08:07:05.511918+0000","last_undegraded":"2024-09-10T08:07:05.511918+0000","last_fullsized":"2024-09-10T08:07:05.511918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:21.699397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914428+0000","last_change":"2024-09-10T08:05:35.530804+0000","last_active":"2024-09-10T08:07:05.914428+0000","last_peered":"2024-09-10T08:07:05.914428+0000","last_clean":"2024-09-10T08:07:05.914428+0000","last_became_active":"2024-09-10T08:05:35.530356+0000","last_became_peered":"2024-09-10T08:05:35.530356+0000","last_unstale":"2024-09-10T08:07:05.914428+0000","last_undegraded":"2024-09-10T08:07:05.914428+0000","last_fullsized":"2024-09-10T08:07:05.914428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:59.318761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.662813+0000","last_change":"2024-09-10T08:05:25.489546+0000","last_active":"2024-09-10T08:08:03.662813+0000","last_peered":"2024-09-10T08:08:03.662813+0000","last_clean":"2024-09-10T08:08:03.662813+0000","last_became_active":"2024-09-10T08:05:25.489188+0000","last_became_peered":"2024-09-10T08:05:25.489188+0000","last_unstale":"2024-09-10T08:08:03.662813+0000","last_undegraded":"2024-09-10T08:08:03.662813+0000","last_fullsized":"2024-09-10T08:08:03.662813+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:26.471681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569736+0000","last_change":"2024-09-10T08:05:22.463912+0000","last_active":"2024-09-10T08:07:05.569736+0000","last_peered":"2024-09-10T08:07:05.569736+0000","last_clean":"2024-09-10T08:07:05.569736+0000","last_became_active":"2024-09-10T08:05:22.463561+0000","last_became_peered":"2024-09-10T08:05:22.463561+0000","last_unstale":"2024-09-10T08:07:05.569736+0000","last_undegraded":"2024-09-10T08:07:05.569736+0000","last_fullsized":"2024-09-10T08:07:05.569736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:22.407604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514959+0000","last_change":"2024-09-10T08:05:35.528795+0000","last_active":"2024-09-10T08:07:05.514959+0000","last_peered":"2024-09-10T08:07:05.514959+0000","last_clean":"2024-09-10T08:07:05.514959+0000","last_became_active":"2024-09-10T08:05:35.528402+0000","last_became_peered":"2024-09-10T08:05:35.528402+0000","last_unstale":"2024-09-10T08:07:05.514959+0000","last_undegraded":"2024-09-10T08:07:05.514959+0000","last_fullsized":"2024-09-10T08:07:05.514959+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:14.606371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.657641+0000","last_change":"2024-09-10T08:05:25.484086+0000","last_active":"2024-09-10T08:08:03.657641+0000","last_peered":"2024-09-10T08:08:03.657641+0000","last_clean":"2024-09-10T08:08:03.657641+0000","last_became_active":"2024-09-10T08:05:25.483757+0000","last_became_peered":"2024-09-10T08:05:25.483757+0000","last_unstale":"2024-09-10T08:08:03.657641+0000","last_undegraded":"2024-09-10T08:08:03.657641+0000","last_fullsized":"2024-09-10T08:08:03.657641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:16:16.362739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914678+0000","last_change":"2024-09-10T08:05:22.462220+0000","last_active":"2024-09-10T08:07:05.914678+0000","last_peered":"2024-09-10T08:07:05.914678+0000","last_clean":"2024-09-10T08:07:05.914678+0000","last_became_active":"2024-09-10T08:05:22.461726+0000","last_became_peered":"2024-09-10T08:05:22.461726+0000","last_unstale":"2024-09-10T08:07:05.914678+0000","last_undegraded":"2024-09-10T08:07:05.914678+0000","last_fullsized":"2024-09-10T08:07:05.914678+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:56.381034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508271+0000","last_change":"2024-09-10T08:06:04.734827+0000","last_active":"2024-09-10T08:07:05.508271+0000","last_peered":"2024-09-10T08:07:05.508271+0000","last_clean":"2024-09-10T08:07:05.508271+0000","last_became_active":"2024-09-10T08:06:04.734402+0000","last_became_peered":"2024-09-10T08:06:04.734402+0000","last_unstale":"2024-09-10T08:07:05.508271+0000","last_undegraded":"2024-09-10T08:07:05.508271+0000","last_fullsized":"2024-09-10T08:07:05.508271+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:24.806538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654332+0000","last_change":"2024-09-10T08:05:25.486100+0000","last_active":"2024-09-10T08:08:03.654332+0000","last_peered":"2024-09-10T08:08:03.654332+0000","last_clean":"2024-09-10T08:08:03.654332+0000","last_became_active":"2024-09-10T08:05:25.485586+0000","last_became_peered":"2024-09-10T08:05:25.485586+0000","last_unstale":"2024-09-10T08:08:03.654332+0000","last_undegraded":"2024-09-10T08:08:03.654332+0000","last_fullsized":"2024-09-10T08:08:03.654332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:33.565054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"93'2","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770403+0000","last_change":"2024-09-10T08:06:04.743408+0000","last_active":"2024-09-10T08:07:05.770403+0000","last_peered":"2024-09-10T08:07:05.770403+0000","last_clean":"2024-09-10T08:07:05.770403+0000","last_became_active":"2024-09-10T08:06:04.732830+0000","last_became_peered":"2024-09-10T08:06:04.732830+0000","last_unstale":"2024-09-10T08:07:05.770403+0000","last_undegraded":"2024-09-10T08:07:05.770403+0000","last_fullsized":"2024-09-10T08:07:05.770403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:29.381259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568977+0000","last_change":"2024-09-10T08:05:35.533015+0000","last_active":"2024-09-10T08:07:05.568977+0000","last_peered":"2024-09-10T08:07:05.568977+0000","last_clean":"2024-09-10T08:07:05.568977+0000","last_became_active":"2024-09-10T08:05:35.532590+0000","last_became_peered":"2024-09-10T08:05:35.532590+0000","last_unstale":"2024-09-10T08:07:05.568977+0000","last_undegraded":"2024-09-10T08:07:05.568977+0000","last_fullsized":"2024-09-10T08:07:05.568977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:23.984273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.650197+0000","last_change":"2024-09-10T08:05:25.489994+0000","last_active":"2024-09-10T08:08:03.650197+0000","last_peered":"2024-09-10T08:08:03.650197+0000","last_clean":"2024-09-10T08:08:03.650197+0000","last_became_active":"2024-09-10T08:05:25.489471+0000","last_became_peered":"2024-09-10T08:05:25.489471+0000","last_unstale":"2024-09-10T08:08:03.650197+0000","last_undegraded":"2024-09-10T08:08:03.650197+0000","last_fullsized":"2024-09-10T08:08:03.650197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:53.866570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"93'15","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508405+0000","last_change":"2024-09-10T08:05:22.568900+0000","last_active":"2024-09-10T08:07:05.508405+0000","last_peered":"2024-09-10T08:07:05.508405+0000","last_clean":"2024-09-10T08:07:05.508405+0000","last_became_active":"2024-09-10T08:05:22.568542+0000","last_became_peered":"2024-09-10T08:05:22.568542+0000","last_unstale":"2024-09-10T08:07:05.508405+0000","last_undegraded":"2024-09-10T08:07:05.508405+0000","last_fullsized":"2024-09-10T08:07:05.508405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:58:45.669769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914683+0000","last_change":"2024-09-10T08:05:35.537364+0000","last_active":"2024-09-10T08:07:05.914683+0000","last_peered":"2024-09-10T08:07:05.914683+0000","last_clean":"2024-09-10T08:07:05.914683+0000","last_became_active":"2024-09-10T08:05:35.535958+0000","last_became_peered":"2024-09-10T08:05:35.535958+0000","last_unstale":"2024-09-10T08:07:05.914683+0000","last_undegraded":"2024-09-10T08:07:05.914683+0000","last_fullsized":"2024-09-10T08:07:05.914683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:51.443798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660149+0000","last_change":"2024-09-10T08:07:05.917030+0000","last_active":"2024-09-10T08:08:03.660149+0000","last_peered":"2024-09-10T08:08:03.660149+0000","last_clean":"2024-09-10T08:08:03.660149+0000","last_became_active":"2024-09-10T08:07:05.916281+0000","last_became_peered":"2024-09-10T08:07:05.916281+0000","last_unstale":"2024-09-10T08:08:03.660149+0000","last_undegraded":"2024-09-10T08:08:03.660149+0000","last_fullsized":"2024-09-10T08:08:03.660149+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:15.060009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'4","reported_seq":171,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:20.445646+0000","last_change":"2024-09-10T08:05:22.459963+0000","last_active":"2024-09-10T08:08:20.445646+0000","last_peered":"2024-09-10T08:08:20.445646+0000","last_clean":"2024-09-10T08:08:20.445646+0000","last_became_active":"2024-09-10T08:05:22.459497+0000","last_became_peered":"2024-09-10T08:05:22.459497+0000","last_unstale":"2024-09-10T08:08:20.445646+0000","last_undegraded":"2024-09-10T08:08:20.445646+0000","last_fullsized":"2024-09-10T08:08:20.445646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:29.887980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513986+0000","last_change":"2024-09-10T08:05:35.534570+0000","last_active":"2024-09-10T08:07:05.513986+0000","last_peered":"2024-09-10T08:07:05.513986+0000","last_clean":"2024-09-10T08:07:05.513986+0000","last_became_active":"2024-09-10T08:05:35.534195+0000","last_became_peered":"2024-09-10T08:05:35.534195+0000","last_unstale":"2024-09-10T08:07:05.513986+0000","last_undegraded":"2024-09-10T08:07:05.513986+0000","last_fullsized":"2024-09-10T08:07:05.513986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:17.393491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.668227+0000","last_change":"2024-09-10T08:05:25.488551+0000","last_active":"2024-09-10T08:08:03.668227+0000","last_peered":"2024-09-10T08:08:03.668227+0000","last_clean":"2024-09-10T08:08:03.668227+0000","last_became_active":"2024-09-10T08:05:25.488086+0000","last_became_peered":"2024-09-10T08:05:25.488086+0000","last_unstale":"2024-09-10T08:08:03.668227+0000","last_undegraded":"2024-09-10T08:08:03.668227+0000","last_fullsized":"2024-09-10T08:08:03.668227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:38.044721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569070+0000","last_change":"2024-09-10T08:05:22.462269+0000","last_active":"2024-09-10T08:07:05.569070+0000","last_peered":"2024-09-10T08:07:05.569070+0000","last_clean":"2024-09-10T08:07:05.569070+0000","last_became_active":"2024-09-10T08:05:22.461924+0000","last_became_peered":"2024-09-10T08:05:22.461924+0000","last_unstale":"2024-09-10T08:07:05.569070+0000","last_undegraded":"2024-09-10T08:07:05.569070+0000","last_fullsized":"2024-09-10T08:07:05.569070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:54.996777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770191+0000","last_change":"2024-09-10T08:05:35.531596+0000","last_active":"2024-09-10T08:07:05.770191+0000","last_peered":"2024-09-10T08:07:05.770191+0000","last_clean":"2024-09-10T08:07:05.770191+0000","last_became_active":"2024-09-10T08:05:35.530179+0000","last_became_peered":"2024-09-10T08:05:35.530179+0000","last_unstale":"2024-09-10T08:07:05.770191+0000","last_undegraded":"2024-09-10T08:07:05.770191+0000","last_fullsized":"2024-09-10T08:07:05.770191+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:32.056426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.663218+0000","last_change":"2024-09-10T08:05:25.503504+0000","last_active":"2024-09-10T08:08:03.663218+0000","last_peered":"2024-09-10T08:08:03.663218+0000","last_clean":"2024-09-10T08:08:03.663218+0000","last_became_active":"2024-09-10T08:05:25.503105+0000","last_became_peered":"2024-09-10T08:05:25.503105+0000","last_unstale":"2024-09-10T08:08:03.663218+0000","last_undegraded":"2024-09-10T08:08:03.663218+0000","last_fullsized":"2024-09-10T08:08:03.663218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:40.885654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"93'6","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511623+0000","last_change":"2024-09-10T08:05:22.462271+0000","last_active":"2024-09-10T08:07:05.511623+0000","last_peered":"2024-09-10T08:07:05.511623+0000","last_clean":"2024-09-10T08:07:05.511623+0000","last_became_active":"2024-09-10T08:05:22.461195+0000","last_became_peered":"2024-09-10T08:05:22.461195+0000","last_unstale":"2024-09-10T08:07:05.511623+0000","last_undegraded":"2024-09-10T08:07:05.511623+0000","last_fullsized":"2024-09-10T08:07:05.511623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:20.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569486+0000","last_change":"2024-09-10T08:05:35.533218+0000","last_active":"2024-09-10T08:07:05.569486+0000","last_peered":"2024-09-10T08:07:05.569486+0000","last_clean":"2024-09-10T08:07:05.569486+0000","last_became_active":"2024-09-10T08:05:35.532706+0000","last_became_peered":"2024-09-10T08:05:35.532706+0000","last_unstale":"2024-09-10T08:07:05.569486+0000","last_undegraded":"2024-09-10T08:07:05.569486+0000","last_fullsized":"2024-09-10T08:07:05.569486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:22.150563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.655456+0000","last_change":"2024-09-10T08:05:25.478978+0000","last_active":"2024-09-10T08:08:03.655456+0000","last_peered":"2024-09-10T08:08:03.655456+0000","last_clean":"2024-09-10T08:08:03.655456+0000","last_became_active":"2024-09-10T08:05:25.478649+0000","last_became_peered":"2024-09-10T08:05:25.478649+0000","last_unstale":"2024-09-10T08:08:03.655456+0000","last_undegraded":"2024-09-10T08:08:03.655456+0000","last_fullsized":"2024-09-10T08:08:03.655456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:17.493446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'39","reported_seq":349,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:20.446317+0000","last_change":"2024-09-10T08:06:04.740972+0000","last_active":"2024-09-10T08:08:20.446317+0000","last_peered":"2024-09-10T08:08:20.446317+0000","last_clean":"2024-09-10T08:08:20.446317+0000","last_became_active":"2024-09-10T08:06:04.731200+0000","last_became_peered":"2024-09-10T08:06:04.731200+0000","last_unstale":"2024-09-10T08:08:20.446317+0000","last_undegraded":"2024-09-10T08:08:20.446317+0000","last_fullsized":"2024-09-10T08:08:20.446317+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:17.169708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68987,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":193,"num_write":40,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3703,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512781+0000","last_change":"2024-09-10T08:05:35.532594+0000","last_active":"2024-09-10T08:07:05.512781+0000","last_peered":"2024-09-10T08:07:05.512781+0000","last_clean":"2024-09-10T08:07:05.512781+0000","last_became_active":"2024-09-10T08:05:35.532132+0000","last_became_peered":"2024-09-10T08:05:35.532132+0000","last_unstale":"2024-09-10T08:07:05.512781+0000","last_undegraded":"2024-09-10T08:07:05.512781+0000","last_fullsized":"2024-09-10T08:07:05.512781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:15.348648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.666577+0000","last_change":"2024-09-10T08:05:25.485297+0000","last_active":"2024-09-10T08:08:03.666577+0000","last_peered":"2024-09-10T08:08:03.666577+0000","last_clean":"2024-09-10T08:08:03.666577+0000","last_became_active":"2024-09-10T08:05:25.484883+0000","last_became_peered":"2024-09-10T08:05:25.484883+0000","last_unstale":"2024-09-10T08:08:03.666577+0000","last_undegraded":"2024-09-10T08:08:03.666577+0000","last_fullsized":"2024-09-10T08:08:03.666577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:37.052435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"93'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569713+0000","last_change":"2024-09-10T08:05:22.568993+0000","last_active":"2024-09-10T08:07:05.569713+0000","last_peered":"2024-09-10T08:07:05.569713+0000","last_clean":"2024-09-10T08:07:05.569713+0000","last_became_active":"2024-09-10T08:05:22.568564+0000","last_became_peered":"2024-09-10T08:05:22.568564+0000","last_unstale":"2024-09-10T08:07:05.569713+0000","last_undegraded":"2024-09-10T08:07:05.569713+0000","last_fullsized":"2024-09-10T08:07:05.569713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:23.767174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914556+0000","last_change":"2024-09-10T08:05:35.524025+0000","last_active":"2024-09-10T08:07:05.914556+0000","last_peered":"2024-09-10T08:07:05.914556+0000","last_clean":"2024-09-10T08:07:05.914556+0000","last_became_active":"2024-09-10T08:05:35.523628+0000","last_became_peered":"2024-09-10T08:05:35.523628+0000","last_unstale":"2024-09-10T08:07:05.914556+0000","last_undegraded":"2024-09-10T08:07:05.914556+0000","last_fullsized":"2024-09-10T08:07:05.914556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:48.593779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.680405+0000","last_change":"2024-09-10T08:07:05.570537+0000","last_active":"2024-09-10T08:08:03.680405+0000","last_peered":"2024-09-10T08:08:03.680405+0000","last_clean":"2024-09-10T08:08:03.680405+0000","last_became_active":"2024-09-10T08:07:05.570278+0000","last_became_peered":"2024-09-10T08:07:05.570278+0000","last_unstale":"2024-09-10T08:08:03.680405+0000","last_undegraded":"2024-09-10T08:08:03.680405+0000","last_fullsized":"2024-09-10T08:08:03.680405+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:56.490462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509635+0000","last_change":"2024-09-10T08:05:22.460487+0000","last_active":"2024-09-10T08:07:05.509635+0000","last_peered":"2024-09-10T08:07:05.509635+0000","last_clean":"2024-09-10T08:07:05.509635+0000","last_became_active":"2024-09-10T08:05:22.460158+0000","last_became_peered":"2024-09-10T08:05:22.460158+0000","last_unstale":"2024-09-10T08:07:05.509635+0000","last_undegraded":"2024-09-10T08:07:05.509635+0000","last_fullsized":"2024-09-10T08:07:05.509635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:53.285513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512495+0000","last_change":"2024-09-10T08:05:35.531399+0000","last_active":"2024-09-10T08:07:05.512495+0000","last_peered":"2024-09-10T08:07:05.512495+0000","last_clean":"2024-09-10T08:07:05.512495+0000","last_became_active":"2024-09-10T08:05:35.530320+0000","last_became_peered":"2024-09-10T08:05:35.530320+0000","last_unstale":"2024-09-10T08:07:05.512495+0000","last_undegraded":"2024-09-10T08:07:05.512495+0000","last_fullsized":"2024-09-10T08:07:05.512495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:19.814046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669147+0000","last_change":"2024-09-10T08:05:25.488208+0000","last_active":"2024-09-10T08:08:03.669147+0000","last_peered":"2024-09-10T08:08:03.669147+0000","last_clean":"2024-09-10T08:08:03.669147+0000","last_became_active":"2024-09-10T08:05:25.487893+0000","last_became_peered":"2024-09-10T08:05:25.487893+0000","last_unstale":"2024-09-10T08:08:03.669147+0000","last_undegraded":"2024-09-10T08:08:03.669147+0000","last_fullsized":"2024-09-10T08:08:03.669147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:10.999416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:20.296166+0000","last_change":"2024-09-10T08:05:22.461321+0000","last_active":"2024-09-10T08:08:20.296166+0000","last_peered":"2024-09-10T08:08:20.296166+0000","last_clean":"2024-09-10T08:08:20.296166+0000","last_became_active":"2024-09-10T08:05:22.460950+0000","last_became_peered":"2024-09-10T08:05:22.460950+0000","last_unstale":"2024-09-10T08:08:20.296166+0000","last_undegraded":"2024-09-10T08:08:20.296166+0000","last_fullsized":"2024-09-10T08:08:20.296166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:55.647273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876534+0000","last_change":"2024-09-10T08:05:35.530578+0000","last_active":"2024-09-10T08:07:05.876534+0000","last_peered":"2024-09-10T08:07:05.876534+0000","last_clean":"2024-09-10T08:07:05.876534+0000","last_became_active":"2024-09-10T08:05:35.530162+0000","last_became_peered":"2024-09-10T08:05:35.530162+0000","last_unstale":"2024-09-10T08:07:05.876534+0000","last_undegraded":"2024-09-10T08:07:05.876534+0000","last_fullsized":"2024-09-10T08:07:05.876534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:19.290973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513617+0000","last_change":"2024-09-10T08:05:35.532685+0000","last_active":"2024-09-10T08:07:05.513617+0000","last_peered":"2024-09-10T08:07:05.513617+0000","last_clean":"2024-09-10T08:07:05.513617+0000","last_became_active":"2024-09-10T08:05:35.532257+0000","last_became_peered":"2024-09-10T08:05:35.532257+0000","last_unstale":"2024-09-10T08:07:05.513617+0000","last_undegraded":"2024-09-10T08:07:05.513617+0000","last_fullsized":"2024-09-10T08:07:05.513617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:07.412435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.662135+0000","last_change":"2024-09-10T08:05:25.494472+0000","last_active":"2024-09-10T08:08:03.662135+0000","last_peered":"2024-09-10T08:08:03.662135+0000","last_clean":"2024-09-10T08:08:03.662135+0000","last_became_active":"2024-09-10T08:05:25.493928+0000","last_became_peered":"2024-09-10T08:05:25.493928+0000","last_unstale":"2024-09-10T08:08:03.662135+0000","last_undegraded":"2024-09-10T08:08:03.662135+0000","last_fullsized":"2024-09-10T08:08:03.662135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:00.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"93'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568928+0000","last_change":"2024-09-10T08:05:22.460585+0000","last_active":"2024-09-10T08:07:05.568928+0000","last_peered":"2024-09-10T08:07:05.568928+0000","last_clean":"2024-09-10T08:07:05.568928+0000","last_became_active":"2024-09-10T08:05:22.460259+0000","last_became_peered":"2024-09-10T08:05:22.460259+0000","last_unstale":"2024-09-10T08:07:05.568928+0000","last_undegraded":"2024-09-10T08:07:05.568928+0000","last_fullsized":"2024-09-10T08:07:05.568928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:45.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770275+0000","last_change":"2024-09-10T08:05:35.524000+0000","last_active":"2024-09-10T08:07:05.770275+0000","last_peered":"2024-09-10T08:07:05.770275+0000","last_clean":"2024-09-10T08:07:05.770275+0000","last_became_active":"2024-09-10T08:05:35.523647+0000","last_became_peered":"2024-09-10T08:05:35.523647+0000","last_unstale":"2024-09-10T08:07:05.770275+0000","last_undegraded":"2024-09-10T08:07:05.770275+0000","last_fullsized":"2024-09-10T08:07:05.770275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:10:54.664865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660563+0000","last_change":"2024-09-10T08:05:25.499701+0000","last_active":"2024-09-10T08:08:03.660563+0000","last_peered":"2024-09-10T08:08:03.660563+0000","last_clean":"2024-09-10T08:08:03.660563+0000","last_became_active":"2024-09-10T08:05:25.496890+0000","last_became_peered":"2024-09-10T08:05:25.496890+0000","last_unstale":"2024-09-10T08:08:03.660563+0000","last_undegraded":"2024-09-10T08:08:03.660563+0000","last_fullsized":"2024-09-10T08:08:03.660563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:58.166365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914070+0000","last_change":"2024-09-10T08:05:22.460880+0000","last_active":"2024-09-10T08:07:05.914070+0000","last_peered":"2024-09-10T08:07:05.914070+0000","last_clean":"2024-09-10T08:07:05.914070+0000","last_became_active":"2024-09-10T08:05:22.459932+0000","last_became_peered":"2024-09-10T08:05:22.459932+0000","last_unstale":"2024-09-10T08:07:05.914070+0000","last_undegraded":"2024-09-10T08:07:05.914070+0000","last_fullsized":"2024-09-10T08:07:05.914070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:55.826526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915405+0000","last_change":"2024-09-10T08:05:22.460578+0000","last_active":"2024-09-10T08:07:05.915405+0000","last_peered":"2024-09-10T08:07:05.915405+0000","last_clean":"2024-09-10T08:07:05.915405+0000","last_became_active":"2024-09-10T08:05:22.460239+0000","last_became_peered":"2024-09-10T08:05:22.460239+0000","last_unstale":"2024-09-10T08:07:05.915405+0000","last_undegraded":"2024-09-10T08:07:05.915405+0000","last_fullsized":"2024-09-10T08:07:05.915405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:07.606243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.661142+0000","last_change":"2024-09-10T08:05:25.489846+0000","last_active":"2024-09-10T08:08:03.661142+0000","last_peered":"2024-09-10T08:08:03.661142+0000","last_clean":"2024-09-10T08:08:03.661142+0000","last_became_active":"2024-09-10T08:05:25.489326+0000","last_became_peered":"2024-09-10T08:05:25.489326+0000","last_unstale":"2024-09-10T08:08:03.661142+0000","last_undegraded":"2024-09-10T08:08:03.661142+0000","last_fullsized":"2024-09-10T08:08:03.661142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:43.458468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511635+0000","last_change":"2024-09-10T08:05:35.530522+0000","last_active":"2024-09-10T08:07:05.511635+0000","last_peered":"2024-09-10T08:07:05.511635+0000","last_clean":"2024-09-10T08:07:05.511635+0000","last_became_active":"2024-09-10T08:05:35.530099+0000","last_became_peered":"2024-09-10T08:05:35.530099+0000","last_unstale":"2024-09-10T08:07:05.511635+0000","last_undegraded":"2024-09-10T08:07:05.511635+0000","last_fullsized":"2024-09-10T08:07:05.511635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:58.469119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":179,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:20.446528+0000","last_change":"2024-09-10T08:05:22.461391+0000","last_active":"2024-09-10T08:08:20.446528+0000","last_peered":"2024-09-10T08:08:20.446528+0000","last_clean":"2024-09-10T08:08:20.446528+0000","last_became_active":"2024-09-10T08:05:22.461042+0000","last_became_peered":"2024-09-10T08:05:22.461042+0000","last_unstale":"2024-09-10T08:08:20.446528+0000","last_undegraded":"2024-09-10T08:08:20.446528+0000","last_fullsized":"2024-09-10T08:08:20.446528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:12.635710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1860,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.649481+0000","last_change":"2024-09-10T08:05:25.487378+0000","last_active":"2024-09-10T08:08:03.649481+0000","last_peered":"2024-09-10T08:08:03.649481+0000","last_clean":"2024-09-10T08:08:03.649481+0000","last_became_active":"2024-09-10T08:05:25.487055+0000","last_became_peered":"2024-09-10T08:05:25.487055+0000","last_unstale":"2024-09-10T08:08:03.649481+0000","last_undegraded":"2024-09-10T08:08:03.649481+0000","last_fullsized":"2024-09-10T08:08:03.649481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:59.522014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511748+0000","last_change":"2024-09-10T08:05:35.522765+0000","last_active":"2024-09-10T08:07:05.511748+0000","last_peered":"2024-09-10T08:07:05.511748+0000","last_clean":"2024-09-10T08:07:05.511748+0000","last_became_active":"2024-09-10T08:05:35.522359+0000","last_became_peered":"2024-09-10T08:05:35.522359+0000","last_unstale":"2024-09-10T08:07:05.511748+0000","last_undegraded":"2024-09-10T08:07:05.511748+0000","last_fullsized":"2024-09-10T08:07:05.511748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:15.280867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.652869+0000","last_change":"2024-09-10T08:05:25.475388+0000","last_active":"2024-09-10T08:08:03.652869+0000","last_peered":"2024-09-10T08:08:03.652869+0000","last_clean":"2024-09-10T08:08:03.652869+0000","last_became_active":"2024-09-10T08:05:25.474331+0000","last_became_peered":"2024-09-10T08:05:25.474331+0000","last_unstale":"2024-09-10T08:08:03.652869+0000","last_undegraded":"2024-09-10T08:08:03.652869+0000","last_fullsized":"2024-09-10T08:08:03.652869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:58.713148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656383+0000","last_change":"2024-09-10T08:07:05.570505+0000","last_active":"2024-09-10T08:08:03.656383+0000","last_peered":"2024-09-10T08:08:03.656383+0000","last_clean":"2024-09-10T08:08:03.656383+0000","last_became_active":"2024-09-10T08:07:05.569970+0000","last_became_peered":"2024-09-10T08:07:05.569970+0000","last_unstale":"2024-09-10T08:08:03.656383+0000","last_undegraded":"2024-09-10T08:08:03.656383+0000","last_fullsized":"2024-09-10T08:08:03.656383+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:13.282503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671951+0000","last_change":"2024-09-10T08:05:25.491585+0000","last_active":"2024-09-10T08:08:03.671951+0000","last_peered":"2024-09-10T08:08:03.671951+0000","last_clean":"2024-09-10T08:08:03.671951+0000","last_became_active":"2024-09-10T08:05:25.491224+0000","last_became_peered":"2024-09-10T08:05:25.491224+0000","last_unstale":"2024-09-10T08:08:03.671951+0000","last_undegraded":"2024-09-10T08:08:03.671951+0000","last_fullsized":"2024-09-10T08:08:03.671951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:56.066808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681502+0000","last_change":"2024-09-10T08:05:25.499578+0000","last_active":"2024-09-10T08:08:03.681502+0000","last_peered":"2024-09-10T08:08:03.681502+0000","last_clean":"2024-09-10T08:08:03.681502+0000","last_became_active":"2024-09-10T08:05:25.498445+0000","last_became_peered":"2024-09-10T08:05:25.498445+0000","last_unstale":"2024-09-10T08:08:03.681502+0000","last_undegraded":"2024-09-10T08:08:03.681502+0000","last_fullsized":"2024-09-10T08:08:03.681502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:58.961699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675717+0000","last_change":"2024-09-10T08:06:04.735764+0000","last_active":"2024-09-10T08:08:03.675717+0000","last_peered":"2024-09-10T08:08:03.675717+0000","last_clean":"2024-09-10T08:08:03.675717+0000","last_became_active":"2024-09-10T08:06:04.735366+0000","last_became_peered":"2024-09-10T08:06:04.735366+0000","last_unstale":"2024-09-10T08:08:03.675717+0000","last_undegraded":"2024-09-10T08:08:03.675717+0000","last_fullsized":"2024-09-10T08:08:03.675717+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:03.757890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651587+0000","last_change":"2024-09-10T08:05:25.475872+0000","last_active":"2024-09-10T08:08:03.651587+0000","last_peered":"2024-09-10T08:08:03.651587+0000","last_clean":"2024-09-10T08:08:03.651587+0000","last_became_active":"2024-09-10T08:05:25.475549+0000","last_became_peered":"2024-09-10T08:05:25.475549+0000","last_unstale":"2024-09-10T08:08:03.651587+0000","last_undegraded":"2024-09-10T08:08:03.651587+0000","last_fullsized":"2024-09-10T08:08:03.651587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:37.575267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.664667+0000","last_change":"2024-09-10T08:05:25.474210+0000","last_active":"2024-09-10T08:08:03.664667+0000","last_peered":"2024-09-10T08:08:03.664667+0000","last_clean":"2024-09-10T08:08:03.664667+0000","last_became_active":"2024-09-10T08:05:25.473882+0000","last_became_peered":"2024-09-10T08:05:25.473882+0000","last_unstale":"2024-09-10T08:08:03.664667+0000","last_undegraded":"2024-09-10T08:08:03.664667+0000","last_fullsized":"2024-09-10T08:08:03.664667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:05.164370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651519+0000","last_change":"2024-09-10T08:05:25.490406+0000","last_active":"2024-09-10T08:08:03.651519+0000","last_peered":"2024-09-10T08:08:03.651519+0000","last_clean":"2024-09-10T08:08:03.651519+0000","last_became_active":"2024-09-10T08:05:25.490061+0000","last_became_peered":"2024-09-10T08:05:25.490061+0000","last_unstale":"2024-09-10T08:08:03.651519+0000","last_undegraded":"2024-09-10T08:08:03.651519+0000","last_fullsized":"2024-09-10T08:08:03.651519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:12.988229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.664045+0000","last_change":"2024-09-10T08:05:25.504049+0000","last_active":"2024-09-10T08:08:03.664045+0000","last_peered":"2024-09-10T08:08:03.664045+0000","last_clean":"2024-09-10T08:08:03.664045+0000","last_became_active":"2024-09-10T08:05:25.503770+0000","last_became_peered":"2024-09-10T08:05:25.503770+0000","last_unstale":"2024-09-10T08:08:03.664045+0000","last_undegraded":"2024-09-10T08:08:03.664045+0000","last_fullsized":"2024-09-10T08:08:03.664045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:52.850945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.672614+0000","last_change":"2024-09-10T08:05:25.491682+0000","last_active":"2024-09-10T08:08:03.672614+0000","last_peered":"2024-09-10T08:08:03.672614+0000","last_clean":"2024-09-10T08:08:03.672614+0000","last_became_active":"2024-09-10T08:05:25.491296+0000","last_became_peered":"2024-09-10T08:05:25.491296+0000","last_unstale":"2024-09-10T08:08:03.672614+0000","last_undegraded":"2024-09-10T08:08:03.672614+0000","last_fullsized":"2024-09-10T08:08:03.672614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:30.075842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.673509+0000","last_change":"2024-09-10T08:05:25.489648+0000","last_active":"2024-09-10T08:08:03.673509+0000","last_peered":"2024-09-10T08:08:03.673509+0000","last_clean":"2024-09-10T08:08:03.673509+0000","last_became_active":"2024-09-10T08:05:25.489267+0000","last_became_peered":"2024-09-10T08:05:25.489267+0000","last_unstale":"2024-09-10T08:08:03.673509+0000","last_undegraded":"2024-09-10T08:08:03.673509+0000","last_fullsized":"2024-09-10T08:08:03.673509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:21.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681690+0000","last_change":"2024-09-10T08:05:25.497862+0000","last_active":"2024-09-10T08:08:03.681690+0000","last_peered":"2024-09-10T08:08:03.681690+0000","last_clean":"2024-09-10T08:08:03.681690+0000","last_became_active":"2024-09-10T08:05:25.497504+0000","last_became_peered":"2024-09-10T08:05:25.497504+0000","last_unstale":"2024-09-10T08:08:03.681690+0000","last_undegraded":"2024-09-10T08:08:03.681690+0000","last_fullsized":"2024-09-10T08:08:03.681690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:40.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654484+0000","last_change":"2024-09-10T08:05:25.485924+0000","last_active":"2024-09-10T08:08:03.654484+0000","last_peered":"2024-09-10T08:08:03.654484+0000","last_clean":"2024-09-10T08:08:03.654484+0000","last_became_active":"2024-09-10T08:05:25.485545+0000","last_became_peered":"2024-09-10T08:05:25.485545+0000","last_unstale":"2024-09-10T08:08:03.654484+0000","last_undegraded":"2024-09-10T08:08:03.654484+0000","last_fullsized":"2024-09-10T08:08:03.654484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:33:20.233720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671966+0000","last_change":"2024-09-10T08:05:25.491911+0000","last_active":"2024-09-10T08:08:03.671966+0000","last_peered":"2024-09-10T08:08:03.671966+0000","last_clean":"2024-09-10T08:08:03.671966+0000","last_became_active":"2024-09-10T08:05:25.491330+0000","last_became_peered":"2024-09-10T08:05:25.491330+0000","last_unstale":"2024-09-10T08:08:03.671966+0000","last_undegraded":"2024-09-10T08:08:03.671966+0000","last_fullsized":"2024-09-10T08:08:03.671966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:06.043331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.655352+0000","last_change":"2024-09-10T08:05:25.492220+0000","last_active":"2024-09-10T08:08:03.655352+0000","last_peered":"2024-09-10T08:08:03.655352+0000","last_clean":"2024-09-10T08:08:03.655352+0000","last_became_active":"2024-09-10T08:05:25.491473+0000","last_became_peered":"2024-09-10T08:05:25.491473+0000","last_unstale":"2024-09-10T08:08:03.655352+0000","last_undegraded":"2024-09-10T08:08:03.655352+0000","last_fullsized":"2024-09-10T08:08:03.655352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:13:30.278434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676184+0000","last_change":"2024-09-10T08:05:25.488927+0000","last_active":"2024-09-10T08:08:03.676184+0000","last_peered":"2024-09-10T08:08:03.676184+0000","last_clean":"2024-09-10T08:08:03.676184+0000","last_became_active":"2024-09-10T08:05:25.488550+0000","last_became_peered":"2024-09-10T08:05:25.488550+0000","last_unstale":"2024-09-10T08:08:03.676184+0000","last_undegraded":"2024-09-10T08:08:03.676184+0000","last_fullsized":"2024-09-10T08:08:03.676184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:44.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.677654+0000","last_change":"2024-09-10T08:05:25.490125+0000","last_active":"2024-09-10T08:08:03.677654+0000","last_peered":"2024-09-10T08:08:03.677654+0000","last_clean":"2024-09-10T08:08:03.677654+0000","last_became_active":"2024-09-10T08:05:25.487805+0000","last_became_peered":"2024-09-10T08:05:25.487805+0000","last_unstale":"2024-09-10T08:08:03.677654+0000","last_undegraded":"2024-09-10T08:08:03.677654+0000","last_fullsized":"2024-09-10T08:08:03.677654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:33.479009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676794+0000","last_change":"2024-09-10T08:05:25.500346+0000","last_active":"2024-09-10T08:08:03.676794+0000","last_peered":"2024-09-10T08:08:03.676794+0000","last_clean":"2024-09-10T08:08:03.676794+0000","last_became_active":"2024-09-10T08:05:25.498289+0000","last_became_peered":"2024-09-10T08:05:25.498289+0000","last_unstale":"2024-09-10T08:08:03.676794+0000","last_undegraded":"2024-09-10T08:08:03.676794+0000","last_fullsized":"2024-09-10T08:08:03.676794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:12.054926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676196+0000","last_change":"2024-09-10T08:05:25.490001+0000","last_active":"2024-09-10T08:08:03.676196+0000","last_peered":"2024-09-10T08:08:03.676196+0000","last_clean":"2024-09-10T08:08:03.676196+0000","last_became_active":"2024-09-10T08:05:25.489706+0000","last_became_peered":"2024-09-10T08:05:25.489706+0000","last_unstale":"2024-09-10T08:08:03.676196+0000","last_undegraded":"2024-09-10T08:08:03.676196+0000","last_fullsized":"2024-09-10T08:08:03.676196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:38.822140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669334+0000","last_change":"2024-09-10T08:05:25.493874+0000","last_active":"2024-09-10T08:08:03.669334+0000","last_peered":"2024-09-10T08:08:03.669334+0000","last_clean":"2024-09-10T08:08:03.669334+0000","last_became_active":"2024-09-10T08:05:25.489911+0000","last_became_peered":"2024-09-10T08:05:25.489911+0000","last_unstale":"2024-09-10T08:08:03.669334+0000","last_undegraded":"2024-09-10T08:08:03.669334+0000","last_fullsized":"2024-09-10T08:08:03.669334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:46.509657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654960+0000","last_change":"2024-09-10T08:05:25.490277+0000","last_active":"2024-09-10T08:08:03.654960+0000","last_peered":"2024-09-10T08:08:03.654960+0000","last_clean":"2024-09-10T08:08:03.654960+0000","last_became_active":"2024-09-10T08:05:25.486863+0000","last_became_peered":"2024-09-10T08:05:25.486863+0000","last_unstale":"2024-09-10T08:08:03.654960+0000","last_undegraded":"2024-09-10T08:08:03.654960+0000","last_fullsized":"2024-09-10T08:08:03.654960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:21.353830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656794+0000","last_change":"2024-09-10T08:05:25.491028+0000","last_active":"2024-09-10T08:08:03.656794+0000","last_peered":"2024-09-10T08:08:03.656794+0000","last_clean":"2024-09-10T08:08:03.656794+0000","last_became_active":"2024-09-10T08:05:25.490452+0000","last_became_peered":"2024-09-10T08:05:25.490452+0000","last_unstale":"2024-09-10T08:08:03.656794+0000","last_undegraded":"2024-09-10T08:08:03.656794+0000","last_fullsized":"2024-09-10T08:08:03.656794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:53.059077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660628+0000","last_change":"2024-09-10T08:05:25.489991+0000","last_active":"2024-09-10T08:08:03.660628+0000","last_peered":"2024-09-10T08:08:03.660628+0000","last_clean":"2024-09-10T08:08:03.660628+0000","last_became_active":"2024-09-10T08:05:25.489662+0000","last_became_peered":"2024-09-10T08:05:25.489662+0000","last_unstale":"2024-09-10T08:08:03.660628+0000","last_undegraded":"2024-09-10T08:08:03.660628+0000","last_fullsized":"2024-09-10T08:08:03.660628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:47.940712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667243+0000","last_change":"2024-09-10T08:05:25.475752+0000","last_active":"2024-09-10T08:08:03.667243+0000","last_peered":"2024-09-10T08:08:03.667243+0000","last_clean":"2024-09-10T08:08:03.667243+0000","last_became_active":"2024-09-10T08:05:25.475395+0000","last_became_peered":"2024-09-10T08:05:25.475395+0000","last_unstale":"2024-09-10T08:08:03.667243+0000","last_undegraded":"2024-09-10T08:08:03.667243+0000","last_fullsized":"2024-09-10T08:08:03.667243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.240154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.672245+0000","last_change":"2024-09-10T08:05:25.503061+0000","last_active":"2024-09-10T08:08:03.672245+0000","last_peered":"2024-09-10T08:08:03.672245+0000","last_clean":"2024-09-10T08:08:03.672245+0000","last_became_active":"2024-09-10T08:05:25.501333+0000","last_became_peered":"2024-09-10T08:05:25.501333+0000","last_unstale":"2024-09-10T08:08:03.672245+0000","last_undegraded":"2024-09-10T08:08:03.672245+0000","last_fullsized":"2024-09-10T08:08:03.672245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:38.254777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.661256+0000","last_change":"2024-09-10T08:07:05.514631+0000","last_active":"2024-09-10T08:08:03.661256+0000","last_peered":"2024-09-10T08:08:03.661256+0000","last_clean":"2024-09-10T08:08:03.661256+0000","last_became_active":"2024-09-10T08:07:05.514285+0000","last_became_peered":"2024-09-10T08:07:05.514285+0000","last_unstale":"2024-09-10T08:08:03.661256+0000","last_undegraded":"2024-09-10T08:08:03.661256+0000","last_fullsized":"2024-09-10T08:08:03.661256+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:40.293121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660092+0000","last_change":"2024-09-10T08:05:25.491457+0000","last_active":"2024-09-10T08:08:03.660092+0000","last_peered":"2024-09-10T08:08:03.660092+0000","last_clean":"2024-09-10T08:08:03.660092+0000","last_became_active":"2024-09-10T08:05:25.491162+0000","last_became_peered":"2024-09-10T08:05:25.491162+0000","last_unstale":"2024-09-10T08:08:03.660092+0000","last_undegraded":"2024-09-10T08:08:03.660092+0000","last_fullsized":"2024-09-10T08:08:03.660092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:14.389752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660737+0000","last_change":"2024-09-10T08:05:25.487874+0000","last_active":"2024-09-10T08:08:03.660737+0000","last_peered":"2024-09-10T08:08:03.660737+0000","last_clean":"2024-09-10T08:08:03.660737+0000","last_became_active":"2024-09-10T08:05:25.487497+0000","last_became_peered":"2024-09-10T08:05:25.487497+0000","last_unstale":"2024-09-10T08:08:03.660737+0000","last_undegraded":"2024-09-10T08:08:03.660737+0000","last_fullsized":"2024-09-10T08:08:03.660737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:26.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658875+0000","last_change":"2024-09-10T08:05:25.490018+0000","last_active":"2024-09-10T08:08:03.658875+0000","last_peered":"2024-09-10T08:08:03.658875+0000","last_clean":"2024-09-10T08:08:03.658875+0000","last_became_active":"2024-09-10T08:05:25.484951+0000","last_became_peered":"2024-09-10T08:05:25.484951+0000","last_unstale":"2024-09-10T08:08:03.658875+0000","last_undegraded":"2024-09-10T08:08:03.658875+0000","last_fullsized":"2024-09-10T08:08:03.658875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:07.516154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.677343+0000","last_change":"2024-09-10T08:05:25.501581+0000","last_active":"2024-09-10T08:08:03.677343+0000","last_peered":"2024-09-10T08:08:03.677343+0000","last_clean":"2024-09-10T08:08:03.677343+0000","last_became_active":"2024-09-10T08:05:25.499690+0000","last_became_peered":"2024-09-10T08:05:25.499690+0000","last_unstale":"2024-09-10T08:08:03.677343+0000","last_undegraded":"2024-09-10T08:08:03.677343+0000","last_fullsized":"2024-09-10T08:08:03.677343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:14.569791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.678243+0000","last_change":"2024-09-10T08:05:25.487945+0000","last_active":"2024-09-10T08:08:03.678243+0000","last_peered":"2024-09-10T08:08:03.678243+0000","last_clean":"2024-09-10T08:08:03.678243+0000","last_became_active":"2024-09-10T08:05:25.487655+0000","last_became_peered":"2024-09-10T08:05:25.487655+0000","last_unstale":"2024-09-10T08:08:03.678243+0000","last_undegraded":"2024-09-10T08:08:03.678243+0000","last_fullsized":"2024-09-10T08:08:03.678243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:37.746037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675926+0000","last_change":"2024-09-10T08:05:25.489348+0000","last_active":"2024-09-10T08:08:03.675926+0000","last_peered":"2024-09-10T08:08:03.675926+0000","last_clean":"2024-09-10T08:08:03.675926+0000","last_became_active":"2024-09-10T08:05:25.489004+0000","last_became_peered":"2024-09-10T08:05:25.489004+0000","last_unstale":"2024-09-10T08:08:03.675926+0000","last_undegraded":"2024-09-10T08:08:03.675926+0000","last_fullsized":"2024-09-10T08:08:03.675926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:54.554719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74228,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":753,"num_read_kb":456,"num_write":187,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3725,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":221898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214976,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514546,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":548036,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743479,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712792,"statfs":{"total":95995035648,"available":95961899008,"internally_reserved":0,"allocated":1204224,"data_stored":1208758,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907002,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":548250,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135938,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":548148,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364869,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1114112,"data_stored":1144533,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528396,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":543426,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1122304,"data_stored":1140514,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":617456,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":543620,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":545924,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":611930,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":544814,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:08:25.542 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-10T08:08:25.542 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T08:08:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.736+0000 7f059be00640 1 -- 172.21.15.2:0/359540129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f059c104d60 msgr2=0x7f059c1051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.736+0000 7f059be00640 1 --2- 172.21.15.2:0/359540129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f059c104d60 0x7f059c1051e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0588009900 tx=0x7f058802a410 comp rx=0 tx=0).stop 2024-09-10T08:08:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.736+0000 7f059be00640 1 -- 172.21.15.2:0/359540129 shutdown_connections 2024-09-10T08:08:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.736+0000 7f059be00640 1 --2- 172.21.15.2:0/359540129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f059c105720 0x7f059c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.736+0000 7f059be00640 1 --2- 172.21.15.2:0/359540129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f059c104d60 0x7f059c1051e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.736+0000 7f059be00640 1 --2- 172.21.15.2:0/359540129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c103b70 0x7f059c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.736+0000 7f059be00640 1 -- 172.21.15.2:0/359540129 >> 172.21.15.2:0/359540129 conn(0x7f059c0ff2f0 msgr2=0x7f059c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.736+0000 7f059be00640 1 -- 172.21.15.2:0/359540129 shutdown_connections 2024-09-10T08:08:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.736+0000 7f059be00640 1 -- 172.21.15.2:0/359540129 wait complete. 2024-09-10T08:08:25.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.737+0000 7f059be00640 1 Processor -- start 2024-09-10T08:08:25.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.737+0000 7f059be00640 1 -- start start 2024-09-10T08:08:25.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.737+0000 7f059be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f059c103b70 0x7f059c199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:25.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.737+0000 7f059be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c104d60 0x7f059c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:25.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.737+0000 7f059be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f059c105720 0x7f059c1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:25.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.737+0000 7f059be00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f059c10f7d0 con 0x7f059c103b70 2024-09-10T08:08:25.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.737+0000 7f059be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f059c10f650 con 0x7f059c104d60 2024-09-10T08:08:25.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.737+0000 7f059be00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f059c10f950 con 0x7f059c105720 2024-09-10T08:08:25.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.738+0000 7f059aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f059c103b70 0x7f059c199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:25.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.738+0000 7f059aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f059c103b70 0x7f059c199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40032/0 (socket says 172.21.15.2:40032) 2024-09-10T08:08:25.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.738+0000 7f059aa00640 1 -- 172.21.15.2:0/985771432 learned_addr learned my addr 172.21.15.2:0/985771432 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.738+0000 7f059a000640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c104d60 0x7f059c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.738+0000 7f059b400640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f059c105720 0x7f059c1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.738+0000 7f059aa00640 1 -- 172.21.15.2:0/985771432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f059c105720 msgr2=0x7f059c1a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.738+0000 7f059aa00640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f059c105720 0x7f059c1a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.738+0000 7f059aa00640 1 -- 172.21.15.2:0/985771432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c104d60 msgr2=0x7f059c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.738+0000 7f059aa00640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c104d60 0x7f059c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.738+0000 7f059aa00640 1 -- 172.21.15.2:0/985771432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059c1a1b00 con 0x7f059c103b70 2024-09-10T08:08:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.738+0000 7f059b400640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f059c105720 0x7f059c1a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:08:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.738+0000 7f059aa00640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f059c103b70 0x7f059c199ee0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f058c00e9c0 tx=0x7f058c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.739+0000 7f0593400640 1 -- 172.21.15.2:0/985771432 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c00cde0 con 0x7f059c103b70 2024-09-10T08:08:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.739+0000 7f0593400640 1 -- 172.21.15.2:0/985771432 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f058c010910 con 0x7f059c103b70 2024-09-10T08:08:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.739+0000 7f0593400640 1 -- 172.21.15.2:0/985771432 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c018a00 con 0x7f059c103b70 2024-09-10T08:08:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.739+0000 7f059be00640 1 -- 172.21.15.2:0/985771432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059c1a1df0 con 0x7f059c103b70 2024-09-10T08:08:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.739+0000 7f059be00640 1 -- 172.21.15.2:0/985771432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059c1a2200 con 0x7f059c103b70 2024-09-10T08:08:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.740+0000 7f059be00640 1 -- 172.21.15.2:0/985771432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f059c10afb0 con 0x7f059c103b70 2024-09-10T08:08:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.742+0000 7f0593400640 1 -- 172.21.15.2:0/985771432 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f058c01f020 con 0x7f059c103b70 2024-09-10T08:08:25.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.743+0000 7f0593400640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f05800a2c20 0x7f05800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:25.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.743+0000 7f0593400640 1 -- 172.21.15.2:0/985771432 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f058c0cf830 con 0x7f059c103b70 2024-09-10T08:08:25.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.743+0000 7f059a000640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f05800a2c20 0x7f05800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:25.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.744+0000 7f059a000640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f05800a2c20 0x7f05800a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f05880314e0 tx=0x7f0588000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:25.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.747+0000 7f0593400640 1 -- 172.21.15.2:0/985771432 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f058c09c240 con 0x7f059c103b70 2024-09-10T08:08:25.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.933+0000 7f059be00640 1 -- 172.21.15.2:0/985771432 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f059c062630 con 0x7f05800a2c20 2024-09-10T08:08:25.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.942+0000 7f0593400640 1 -- 172.21.15.2:0/985771432 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340724 (secure 0 0 0) 0x7f059c062630 con 0x7f05800a2c20 2024-09-10T08:08:25.945 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:25.947 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-10T08:08:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.946+0000 7f059be00640 1 -- 172.21.15.2:0/985771432 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f05800a2c20 msgr2=0x7f05800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.946+0000 7f059be00640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f05800a2c20 0x7f05800a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f05880314e0 tx=0x7f0588000bc0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.946+0000 7f059be00640 1 -- 172.21.15.2:0/985771432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f059c103b70 msgr2=0x7f059c199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.946+0000 7f059be00640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f059c103b70 0x7f059c199ee0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f058c00e9c0 tx=0x7f058c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:08:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.947+0000 7f059be00640 1 -- 172.21.15.2:0/985771432 shutdown_connections 2024-09-10T08:08:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.947+0000 7f059be00640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f05800a2c20 0x7f05800a50e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.947+0000 7f059be00640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f059c105720 0x7f059c1a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.947+0000 7f059be00640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c104d60 0x7f059c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.947+0000 7f059be00640 1 --2- 172.21.15.2:0/985771432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f059c103b70 0x7f059c199ee0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.947+0000 7f059be00640 1 -- 172.21.15.2:0/985771432 >> 172.21.15.2:0/985771432 conn(0x7f059c0ff2f0 msgr2=0x7f059c108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.947+0000 7f059be00640 1 -- 172.21.15.2:0/985771432 shutdown_connections 2024-09-10T08:08:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:25.947+0000 7f059be00640 1 -- 172.21.15.2:0/985771432 wait complete. 2024-09-10T08:08:25.962 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-10T08:08:25.693511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667350,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":895,"num_read_kb":578,"num_write":373,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1184461,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":800,"ondisk_log_size":800,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362056,"kb_used_data":12808,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579768,"statfs":{"total":1151940427776,"available":1151569682432,"internally_reserved":0,"allocated":13115392,"data_stored":8551628,"data_compressed":497802,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9694,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":54,"num_write":16,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003269"},"pg_stats":[{"pgid":"2.1f","version":"93'9","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514795+0000","last_change":"2024-09-10T08:05:22.462062+0000","last_active":"2024-09-10T08:07:05.514795+0000","last_peered":"2024-09-10T08:07:05.514795+0000","last_clean":"2024-09-10T08:07:05.514795+0000","last_became_active":"2024-09-10T08:05:22.460973+0000","last_became_peered":"2024-09-10T08:05:22.460973+0000","last_unstale":"2024-09-10T08:07:05.514795+0000","last_undegraded":"2024-09-10T08:07:05.514795+0000","last_fullsized":"2024-09-10T08:07:05.514795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:01.357446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656803+0000","last_change":"2024-09-10T08:05:25.482873+0000","last_active":"2024-09-10T08:08:03.656803+0000","last_peered":"2024-09-10T08:08:03.656803+0000","last_clean":"2024-09-10T08:08:03.656803+0000","last_became_active":"2024-09-10T08:05:25.482559+0000","last_became_peered":"2024-09-10T08:05:25.482559+0000","last_unstale":"2024-09-10T08:08:03.656803+0000","last_undegraded":"2024-09-10T08:08:03.656803+0000","last_fullsized":"2024-09-10T08:08:03.656803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:10:17.366160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511384+0000","last_change":"2024-09-10T08:06:04.736356+0000","last_active":"2024-09-10T08:07:05.511384+0000","last_peered":"2024-09-10T08:07:05.511384+0000","last_clean":"2024-09-10T08:07:05.511384+0000","last_became_active":"2024-09-10T08:06:04.735859+0000","last_became_peered":"2024-09-10T08:06:04.735859+0000","last_unstale":"2024-09-10T08:07:05.511384+0000","last_undegraded":"2024-09-10T08:07:05.511384+0000","last_fullsized":"2024-09-10T08:07:05.511384+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:56.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"93'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.877433+0000","last_change":"2024-09-10T08:05:22.569504+0000","last_active":"2024-09-10T08:07:05.877433+0000","last_peered":"2024-09-10T08:07:05.877433+0000","last_clean":"2024-09-10T08:07:05.877433+0000","last_became_active":"2024-09-10T08:05:22.569164+0000","last_became_peered":"2024-09-10T08:05:22.569164+0000","last_unstale":"2024-09-10T08:07:05.877433+0000","last_undegraded":"2024-09-10T08:07:05.877433+0000","last_fullsized":"2024-09-10T08:07:05.877433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:41.628963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.680208+0000","last_change":"2024-09-10T08:05:25.493017+0000","last_active":"2024-09-10T08:08:03.680208+0000","last_peered":"2024-09-10T08:08:03.680208+0000","last_clean":"2024-09-10T08:08:03.680208+0000","last_became_active":"2024-09-10T08:05:25.491370+0000","last_became_peered":"2024-09-10T08:05:25.491370+0000","last_unstale":"2024-09-10T08:08:03.680208+0000","last_undegraded":"2024-09-10T08:08:03.680208+0000","last_fullsized":"2024-09-10T08:08:03.680208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:41.616549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914087+0000","last_change":"2024-09-10T08:05:35.523751+0000","last_active":"2024-09-10T08:07:05.914087+0000","last_peered":"2024-09-10T08:07:05.914087+0000","last_clean":"2024-09-10T08:07:05.914087+0000","last_became_active":"2024-09-10T08:05:35.523372+0000","last_became_peered":"2024-09-10T08:05:35.523372+0000","last_unstale":"2024-09-10T08:07:05.914087+0000","last_undegraded":"2024-09-10T08:07:05.914087+0000","last_fullsized":"2024-09-10T08:07:05.914087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:20.971596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508934+0000","last_change":"2024-09-10T08:05:22.460590+0000","last_active":"2024-09-10T08:07:05.508934+0000","last_peered":"2024-09-10T08:07:05.508934+0000","last_clean":"2024-09-10T08:07:05.508934+0000","last_became_active":"2024-09-10T08:05:22.460238+0000","last_became_peered":"2024-09-10T08:05:22.460238+0000","last_unstale":"2024-09-10T08:07:05.508934+0000","last_undegraded":"2024-09-10T08:07:05.508934+0000","last_fullsized":"2024-09-10T08:07:05.508934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:18.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669035+0000","last_change":"2024-09-10T08:05:25.487392+0000","last_active":"2024-09-10T08:08:03.669035+0000","last_peered":"2024-09-10T08:08:03.669035+0000","last_clean":"2024-09-10T08:08:03.669035+0000","last_became_active":"2024-09-10T08:05:25.487054+0000","last_became_peered":"2024-09-10T08:05:25.487054+0000","last_unstale":"2024-09-10T08:08:03.669035+0000","last_undegraded":"2024-09-10T08:08:03.669035+0000","last_fullsized":"2024-09-10T08:08:03.669035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:16.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914818+0000","last_change":"2024-09-10T08:06:04.735778+0000","last_active":"2024-09-10T08:07:05.914818+0000","last_peered":"2024-09-10T08:07:05.914818+0000","last_clean":"2024-09-10T08:07:05.914818+0000","last_became_active":"2024-09-10T08:06:04.735208+0000","last_became_peered":"2024-09-10T08:06:04.735208+0000","last_unstale":"2024-09-10T08:07:05.914818+0000","last_undegraded":"2024-09-10T08:07:05.914818+0000","last_fullsized":"2024-09-10T08:07:05.914818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:36.982246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508944+0000","last_change":"2024-09-10T08:05:22.458323+0000","last_active":"2024-09-10T08:07:05.508944+0000","last_peered":"2024-09-10T08:07:05.508944+0000","last_clean":"2024-09-10T08:07:05.508944+0000","last_became_active":"2024-09-10T08:05:22.457965+0000","last_became_peered":"2024-09-10T08:05:22.457965+0000","last_unstale":"2024-09-10T08:07:05.508944+0000","last_undegraded":"2024-09-10T08:07:05.508944+0000","last_fullsized":"2024-09-10T08:07:05.508944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:15.021305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.663997+0000","last_change":"2024-09-10T08:05:25.489883+0000","last_active":"2024-09-10T08:08:03.663997+0000","last_peered":"2024-09-10T08:08:03.663997+0000","last_clean":"2024-09-10T08:08:03.663997+0000","last_became_active":"2024-09-10T08:05:25.489482+0000","last_became_peered":"2024-09-10T08:05:25.489482+0000","last_unstale":"2024-09-10T08:08:03.663997+0000","last_undegraded":"2024-09-10T08:08:03.663997+0000","last_fullsized":"2024-09-10T08:08:03.663997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:52.080628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914064+0000","last_change":"2024-09-10T08:05:35.531458+0000","last_active":"2024-09-10T08:07:05.914064+0000","last_peered":"2024-09-10T08:07:05.914064+0000","last_clean":"2024-09-10T08:07:05.914064+0000","last_became_active":"2024-09-10T08:05:35.531072+0000","last_became_peered":"2024-09-10T08:05:35.531072+0000","last_unstale":"2024-09-10T08:07:05.914064+0000","last_undegraded":"2024-09-10T08:07:05.914064+0000","last_fullsized":"2024-09-10T08:07:05.914064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:04.792664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667650+0000","last_change":"2024-09-10T08:05:25.486865+0000","last_active":"2024-09-10T08:08:03.667650+0000","last_peered":"2024-09-10T08:08:03.667650+0000","last_clean":"2024-09-10T08:08:03.667650+0000","last_became_active":"2024-09-10T08:05:25.486531+0000","last_became_peered":"2024-09-10T08:05:25.486531+0000","last_unstale":"2024-09-10T08:08:03.667650+0000","last_undegraded":"2024-09-10T08:08:03.667650+0000","last_fullsized":"2024-09-10T08:08:03.667650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:26.854682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"93'4","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508917+0000","last_change":"2024-09-10T08:06:04.743303+0000","last_active":"2024-09-10T08:07:05.508917+0000","last_peered":"2024-09-10T08:07:05.508917+0000","last_clean":"2024-09-10T08:07:05.508917+0000","last_became_active":"2024-09-10T08:06:04.733294+0000","last_became_peered":"2024-09-10T08:06:04.733294+0000","last_unstale":"2024-09-10T08:07:05.508917+0000","last_undegraded":"2024-09-10T08:07:05.508917+0000","last_fullsized":"2024-09-10T08:07:05.508917+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:18.542658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914822+0000","last_change":"2024-09-10T08:05:35.642083+0000","last_active":"2024-09-10T08:07:05.914822+0000","last_peered":"2024-09-10T08:07:05.914822+0000","last_clean":"2024-09-10T08:07:05.914822+0000","last_became_active":"2024-09-10T08:05:35.641214+0000","last_became_peered":"2024-09-10T08:05:35.641214+0000","last_unstale":"2024-09-10T08:07:05.914822+0000","last_undegraded":"2024-09-10T08:07:05.914822+0000","last_fullsized":"2024-09-10T08:07:05.914822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:00:21.731769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681852+0000","last_change":"2024-09-10T08:05:25.501993+0000","last_active":"2024-09-10T08:08:03.681852+0000","last_peered":"2024-09-10T08:08:03.681852+0000","last_clean":"2024-09-10T08:08:03.681852+0000","last_became_active":"2024-09-10T08:05:25.499912+0000","last_became_peered":"2024-09-10T08:05:25.499912+0000","last_unstale":"2024-09-10T08:08:03.681852+0000","last_undegraded":"2024-09-10T08:08:03.681852+0000","last_fullsized":"2024-09-10T08:08:03.681852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:06.780067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:36.268711+0000","last_change":"2024-09-10T08:05:22.461786+0000","last_active":"2024-09-10T08:07:36.268711+0000","last_peered":"2024-09-10T08:07:36.268711+0000","last_clean":"2024-09-10T08:07:36.268711+0000","last_became_active":"2024-09-10T08:05:22.460821+0000","last_became_peered":"2024-09-10T08:05:22.460821+0000","last_unstale":"2024-09-10T08:07:36.268711+0000","last_undegraded":"2024-09-10T08:07:36.268711+0000","last_fullsized":"2024-09-10T08:07:36.268711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:25.083538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914568+0000","last_change":"2024-09-10T08:06:04.734470+0000","last_active":"2024-09-10T08:07:05.914568+0000","last_peered":"2024-09-10T08:07:05.914568+0000","last_clean":"2024-09-10T08:07:05.914568+0000","last_became_active":"2024-09-10T08:06:04.734040+0000","last_became_peered":"2024-09-10T08:06:04.734040+0000","last_unstale":"2024-09-10T08:07:05.914568+0000","last_undegraded":"2024-09-10T08:07:05.914568+0000","last_fullsized":"2024-09-10T08:07:05.914568+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:36.312144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675820+0000","last_change":"2024-09-10T08:05:25.492711+0000","last_active":"2024-09-10T08:08:03.675820+0000","last_peered":"2024-09-10T08:08:03.675820+0000","last_clean":"2024-09-10T08:08:03.675820+0000","last_became_active":"2024-09-10T08:05:25.492336+0000","last_became_peered":"2024-09-10T08:05:25.492336+0000","last_unstale":"2024-09-10T08:08:03.675820+0000","last_undegraded":"2024-09-10T08:08:03.675820+0000","last_fullsized":"2024-09-10T08:08:03.675820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:04.168884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"93'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876783+0000","last_change":"2024-09-10T08:05:22.461535+0000","last_active":"2024-09-10T08:07:05.876783+0000","last_peered":"2024-09-10T08:07:05.876783+0000","last_clean":"2024-09-10T08:07:05.876783+0000","last_became_active":"2024-09-10T08:05:22.461208+0000","last_became_peered":"2024-09-10T08:05:22.461208+0000","last_unstale":"2024-09-10T08:07:05.876783+0000","last_undegraded":"2024-09-10T08:07:05.876783+0000","last_fullsized":"2024-09-10T08:07:05.876783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:26.009242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513889+0000","last_change":"2024-09-10T08:05:35.535442+0000","last_active":"2024-09-10T08:07:05.513889+0000","last_peered":"2024-09-10T08:07:05.513889+0000","last_clean":"2024-09-10T08:07:05.513889+0000","last_became_active":"2024-09-10T08:05:35.535058+0000","last_became_peered":"2024-09-10T08:05:35.535058+0000","last_unstale":"2024-09-10T08:07:05.513889+0000","last_undegraded":"2024-09-10T08:07:05.513889+0000","last_fullsized":"2024-09-10T08:07:05.513889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:39.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.627539+0000","last_change":"2024-09-10T08:05:25.483440+0000","last_active":"2024-09-10T08:08:03.627539+0000","last_peered":"2024-09-10T08:08:03.627539+0000","last_clean":"2024-09-10T08:08:03.627539+0000","last_became_active":"2024-09-10T08:05:25.483141+0000","last_became_peered":"2024-09-10T08:05:25.483141+0000","last_unstale":"2024-09-10T08:08:03.627539+0000","last_undegraded":"2024-09-10T08:08:03.627539+0000","last_fullsized":"2024-09-10T08:08:03.627539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:00.953188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"93'2","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914337+0000","last_change":"2024-09-10T08:05:22.568141+0000","last_active":"2024-09-10T08:07:05.914337+0000","last_peered":"2024-09-10T08:07:05.914337+0000","last_clean":"2024-09-10T08:07:05.914337+0000","last_became_active":"2024-09-10T08:05:22.567857+0000","last_became_peered":"2024-09-10T08:05:22.567857+0000","last_unstale":"2024-09-10T08:07:05.914337+0000","last_undegraded":"2024-09-10T08:07:05.914337+0000","last_fullsized":"2024-09-10T08:07:05.914337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:22.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914846+0000","last_change":"2024-09-10T08:05:35.641993+0000","last_active":"2024-09-10T08:07:05.914846+0000","last_peered":"2024-09-10T08:07:05.914846+0000","last_clean":"2024-09-10T08:07:05.914846+0000","last_became_active":"2024-09-10T08:05:35.641376+0000","last_became_peered":"2024-09-10T08:05:35.641376+0000","last_unstale":"2024-09-10T08:07:05.914846+0000","last_undegraded":"2024-09-10T08:07:05.914846+0000","last_fullsized":"2024-09-10T08:07:05.914846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:16:34.038924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.678764+0000","last_change":"2024-09-10T08:06:04.737560+0000","last_active":"2024-09-10T08:08:03.678764+0000","last_peered":"2024-09-10T08:08:03.678764+0000","last_clean":"2024-09-10T08:08:03.678764+0000","last_became_active":"2024-09-10T08:06:04.736656+0000","last_became_peered":"2024-09-10T08:06:04.736656+0000","last_unstale":"2024-09-10T08:08:03.678764+0000","last_undegraded":"2024-09-10T08:08:03.678764+0000","last_fullsized":"2024-09-10T08:08:03.678764+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:35.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"93'4","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915999+0000","last_change":"2024-09-10T08:05:22.567077+0000","last_active":"2024-09-10T08:07:05.915999+0000","last_peered":"2024-09-10T08:07:05.915999+0000","last_clean":"2024-09-10T08:07:05.915999+0000","last_became_active":"2024-09-10T08:05:22.566757+0000","last_became_peered":"2024-09-10T08:05:22.566757+0000","last_unstale":"2024-09-10T08:07:05.915999+0000","last_undegraded":"2024-09-10T08:07:05.915999+0000","last_fullsized":"2024-09-10T08:07:05.915999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:06.881510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511524+0000","last_change":"2024-09-10T08:05:35.519539+0000","last_active":"2024-09-10T08:07:05.511524+0000","last_peered":"2024-09-10T08:07:05.511524+0000","last_clean":"2024-09-10T08:07:05.511524+0000","last_became_active":"2024-09-10T08:05:35.519044+0000","last_became_peered":"2024-09-10T08:05:35.519044+0000","last_unstale":"2024-09-10T08:07:05.511524+0000","last_undegraded":"2024-09-10T08:07:05.511524+0000","last_fullsized":"2024-09-10T08:07:05.511524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:10.546949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658164+0000","last_change":"2024-09-10T08:07:05.771774+0000","last_active":"2024-09-10T08:08:03.658164+0000","last_peered":"2024-09-10T08:08:03.658164+0000","last_clean":"2024-09-10T08:08:03.658164+0000","last_became_active":"2024-09-10T08:07:05.771336+0000","last_became_peered":"2024-09-10T08:07:05.771336+0000","last_unstale":"2024-09-10T08:08:03.658164+0000","last_undegraded":"2024-09-10T08:08:03.658164+0000","last_fullsized":"2024-09-10T08:08:03.658164+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:11.482222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"95'14","reported_seq":68,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:18.518255+0000","last_change":"2024-09-10T08:06:04.741199+0000","last_active":"2024-09-10T08:08:18.518255+0000","last_peered":"2024-09-10T08:08:18.518255+0000","last_clean":"2024-09-10T08:08:18.518255+0000","last_became_active":"2024-09-10T08:06:04.738076+0000","last_became_peered":"2024-09-10T08:06:04.738076+0000","last_unstale":"2024-09-10T08:08:18.518255+0000","last_undegraded":"2024-09-10T08:08:18.518255+0000","last_fullsized":"2024-09-10T08:08:18.518255+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:45.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876462+0000","last_change":"2024-09-10T08:05:35.642358+0000","last_active":"2024-09-10T08:07:05.876462+0000","last_peered":"2024-09-10T08:07:05.876462+0000","last_clean":"2024-09-10T08:07:05.876462+0000","last_became_active":"2024-09-10T08:05:35.641408+0000","last_became_peered":"2024-09-10T08:05:35.641408+0000","last_unstale":"2024-09-10T08:07:05.876462+0000","last_undegraded":"2024-09-10T08:07:05.876462+0000","last_fullsized":"2024-09-10T08:07:05.876462+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:42.015139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658430+0000","last_change":"2024-09-10T08:05:25.486310+0000","last_active":"2024-09-10T08:08:03.658430+0000","last_peered":"2024-09-10T08:08:03.658430+0000","last_clean":"2024-09-10T08:08:03.658430+0000","last_became_active":"2024-09-10T08:05:25.485915+0000","last_became_peered":"2024-09-10T08:05:25.485915+0000","last_unstale":"2024-09-10T08:08:03.658430+0000","last_undegraded":"2024-09-10T08:08:03.658430+0000","last_fullsized":"2024-09-10T08:08:03.658430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:56.064361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"93'1","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508987+0000","last_change":"2024-09-10T08:05:22.570595+0000","last_active":"2024-09-10T08:07:05.508987+0000","last_peered":"2024-09-10T08:07:05.508987+0000","last_clean":"2024-09-10T08:07:05.508987+0000","last_became_active":"2024-09-10T08:05:22.570219+0000","last_became_peered":"2024-09-10T08:05:22.570219+0000","last_unstale":"2024-09-10T08:07:05.508987+0000","last_undegraded":"2024-09-10T08:07:05.508987+0000","last_fullsized":"2024-09-10T08:07:05.508987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:03.230325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769927+0000","last_change":"2024-09-10T08:05:35.530472+0000","last_active":"2024-09-10T08:07:05.769927+0000","last_peered":"2024-09-10T08:07:05.769927+0000","last_clean":"2024-09-10T08:07:05.769927+0000","last_became_active":"2024-09-10T08:05:35.530066+0000","last_became_peered":"2024-09-10T08:05:35.530066+0000","last_unstale":"2024-09-10T08:07:05.769927+0000","last_undegraded":"2024-09-10T08:07:05.769927+0000","last_fullsized":"2024-09-10T08:07:05.769927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:58.915767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671834+0000","last_change":"2024-09-10T08:05:25.500524+0000","last_active":"2024-09-10T08:08:03.671834+0000","last_peered":"2024-09-10T08:08:03.671834+0000","last_clean":"2024-09-10T08:08:03.671834+0000","last_became_active":"2024-09-10T08:05:25.500186+0000","last_became_peered":"2024-09-10T08:05:25.500186+0000","last_unstale":"2024-09-10T08:08:03.671834+0000","last_undegraded":"2024-09-10T08:08:03.671834+0000","last_fullsized":"2024-09-10T08:08:03.671834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:51.936222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513172+0000","last_change":"2024-09-10T08:05:22.569326+0000","last_active":"2024-09-10T08:07:05.513172+0000","last_peered":"2024-09-10T08:07:05.513172+0000","last_clean":"2024-09-10T08:07:05.513172+0000","last_became_active":"2024-09-10T08:05:22.568895+0000","last_became_peered":"2024-09-10T08:05:22.568895+0000","last_unstale":"2024-09-10T08:07:05.513172+0000","last_undegraded":"2024-09-10T08:07:05.513172+0000","last_fullsized":"2024-09-10T08:07:05.513172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:47.212073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508368+0000","last_change":"2024-09-10T08:05:35.641600+0000","last_active":"2024-09-10T08:07:05.508368+0000","last_peered":"2024-09-10T08:07:05.508368+0000","last_clean":"2024-09-10T08:07:05.508368+0000","last_became_active":"2024-09-10T08:05:35.641205+0000","last_became_peered":"2024-09-10T08:05:35.641205+0000","last_unstale":"2024-09-10T08:07:05.508368+0000","last_undegraded":"2024-09-10T08:07:05.508368+0000","last_fullsized":"2024-09-10T08:07:05.508368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:40.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.659935+0000","last_change":"2024-09-10T08:05:25.495035+0000","last_active":"2024-09-10T08:08:03.659935+0000","last_peered":"2024-09-10T08:08:03.659935+0000","last_clean":"2024-09-10T08:08:03.659935+0000","last_became_active":"2024-09-10T08:05:25.494733+0000","last_became_peered":"2024-09-10T08:05:25.494733+0000","last_unstale":"2024-09-10T08:08:03.659935+0000","last_undegraded":"2024-09-10T08:08:03.659935+0000","last_fullsized":"2024-09-10T08:08:03.659935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.875867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"93'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509485+0000","last_change":"2024-09-10T08:05:22.458727+0000","last_active":"2024-09-10T08:07:05.509485+0000","last_peered":"2024-09-10T08:07:05.509485+0000","last_clean":"2024-09-10T08:07:05.509485+0000","last_became_active":"2024-09-10T08:05:22.458397+0000","last_became_peered":"2024-09-10T08:05:22.458397+0000","last_unstale":"2024-09-10T08:07:05.509485+0000","last_undegraded":"2024-09-10T08:07:05.509485+0000","last_fullsized":"2024-09-10T08:07:05.509485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:07.875559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914403+0000","last_change":"2024-09-10T08:05:35.534192+0000","last_active":"2024-09-10T08:07:05.914403+0000","last_peered":"2024-09-10T08:07:05.914403+0000","last_clean":"2024-09-10T08:07:05.914403+0000","last_became_active":"2024-09-10T08:05:35.530450+0000","last_became_peered":"2024-09-10T08:05:35.530450+0000","last_unstale":"2024-09-10T08:07:05.914403+0000","last_undegraded":"2024-09-10T08:07:05.914403+0000","last_fullsized":"2024-09-10T08:07:05.914403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:10.039144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651611+0000","last_change":"2024-09-10T08:05:25.482252+0000","last_active":"2024-09-10T08:08:03.651611+0000","last_peered":"2024-09-10T08:08:03.651611+0000","last_clean":"2024-09-10T08:08:03.651611+0000","last_became_active":"2024-09-10T08:05:25.481901+0000","last_became_peered":"2024-09-10T08:05:25.481901+0000","last_unstale":"2024-09-10T08:08:03.651611+0000","last_undegraded":"2024-09-10T08:08:03.651611+0000","last_fullsized":"2024-09-10T08:08:03.651611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:17.063340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":170,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:21.446702+0000","last_change":"2024-09-10T08:05:22.569199+0000","last_active":"2024-09-10T08:08:21.446702+0000","last_peered":"2024-09-10T08:08:21.446702+0000","last_clean":"2024-09-10T08:08:21.446702+0000","last_became_active":"2024-09-10T08:05:22.568787+0000","last_became_peered":"2024-09-10T08:05:22.568787+0000","last_unstale":"2024-09-10T08:08:21.446702+0000","last_undegraded":"2024-09-10T08:08:21.446702+0000","last_fullsized":"2024-09-10T08:08:21.446702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:59.610130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914677+0000","last_change":"2024-09-10T08:05:35.642446+0000","last_active":"2024-09-10T08:07:05.914677+0000","last_peered":"2024-09-10T08:07:05.914677+0000","last_clean":"2024-09-10T08:07:05.914677+0000","last_became_active":"2024-09-10T08:05:35.641543+0000","last_became_peered":"2024-09-10T08:05:35.641543+0000","last_unstale":"2024-09-10T08:07:05.914677+0000","last_undegraded":"2024-09-10T08:07:05.914677+0000","last_fullsized":"2024-09-10T08:07:05.914677+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:33.658406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.652192+0000","last_change":"2024-09-10T08:05:25.487598+0000","last_active":"2024-09-10T08:08:03.652192+0000","last_peered":"2024-09-10T08:08:03.652192+0000","last_clean":"2024-09-10T08:08:03.652192+0000","last_became_active":"2024-09-10T08:05:25.486879+0000","last_became_peered":"2024-09-10T08:05:25.486879+0000","last_unstale":"2024-09-10T08:08:03.652192+0000","last_undegraded":"2024-09-10T08:08:03.652192+0000","last_fullsized":"2024-09-10T08:08:03.652192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:20.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509934+0000","last_change":"2024-09-10T08:05:22.463416+0000","last_active":"2024-09-10T08:07:05.509934+0000","last_peered":"2024-09-10T08:07:05.509934+0000","last_clean":"2024-09-10T08:07:05.509934+0000","last_became_active":"2024-09-10T08:05:22.463004+0000","last_became_peered":"2024-09-10T08:05:22.463004+0000","last_unstale":"2024-09-10T08:07:05.509934+0000","last_undegraded":"2024-09-10T08:07:05.509934+0000","last_fullsized":"2024-09-10T08:07:05.509934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:56.257812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569364+0000","last_change":"2024-09-10T08:05:35.642059+0000","last_active":"2024-09-10T08:07:05.569364+0000","last_peered":"2024-09-10T08:07:05.569364+0000","last_clean":"2024-09-10T08:07:05.569364+0000","last_became_active":"2024-09-10T08:05:35.641540+0000","last_became_peered":"2024-09-10T08:05:35.641540+0000","last_unstale":"2024-09-10T08:07:05.569364+0000","last_undegraded":"2024-09-10T08:07:05.569364+0000","last_fullsized":"2024-09-10T08:07:05.569364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:28.557584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.670571+0000","last_change":"2024-09-10T08:05:25.496111+0000","last_active":"2024-09-10T08:08:03.670571+0000","last_peered":"2024-09-10T08:08:03.670571+0000","last_clean":"2024-09-10T08:08:03.670571+0000","last_became_active":"2024-09-10T08:05:25.495757+0000","last_became_peered":"2024-09-10T08:05:25.495757+0000","last_unstale":"2024-09-10T08:08:03.670571+0000","last_undegraded":"2024-09-10T08:08:03.670571+0000","last_fullsized":"2024-09-10T08:08:03.670571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:41.078205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'3","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915005+0000","last_change":"2024-09-10T08:05:22.460069+0000","last_active":"2024-09-10T08:07:05.915005+0000","last_peered":"2024-09-10T08:07:05.915005+0000","last_clean":"2024-09-10T08:07:05.915005+0000","last_became_active":"2024-09-10T08:05:22.459557+0000","last_became_peered":"2024-09-10T08:05:22.459557+0000","last_unstale":"2024-09-10T08:07:05.915005+0000","last_undegraded":"2024-09-10T08:07:05.915005+0000","last_fullsized":"2024-09-10T08:07:05.915005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:49.058507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509959+0000","last_change":"2024-09-10T08:04:08.188262+0000","last_active":"2024-09-10T08:07:05.509959+0000","last_peered":"2024-09-10T08:07:05.509959+0000","last_clean":"2024-09-10T08:07:05.509959+0000","last_became_active":"2024-09-10T08:04:08.187691+0000","last_became_peered":"2024-09-10T08:04:08.187691+0000","last_unstale":"2024-09-10T08:07:05.509959+0000","last_undegraded":"2024-09-10T08:07:05.509959+0000","last_fullsized":"2024-09-10T08:07:05.509959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_clean_scrub_stamp":"2024-09-10T08:00:26.235127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:15.358818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509550+0000","last_change":"2024-09-10T08:05:35.528371+0000","last_active":"2024-09-10T08:07:05.509550+0000","last_peered":"2024-09-10T08:07:05.509550+0000","last_clean":"2024-09-10T08:07:05.509550+0000","last_became_active":"2024-09-10T08:05:35.527925+0000","last_became_peered":"2024-09-10T08:05:35.527925+0000","last_unstale":"2024-09-10T08:07:05.509550+0000","last_undegraded":"2024-09-10T08:07:05.509550+0000","last_fullsized":"2024-09-10T08:07:05.509550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:37:00.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667384+0000","last_change":"2024-09-10T08:05:25.490753+0000","last_active":"2024-09-10T08:08:03.667384+0000","last_peered":"2024-09-10T08:08:03.667384+0000","last_clean":"2024-09-10T08:08:03.667384+0000","last_became_active":"2024-09-10T08:05:25.489996+0000","last_became_peered":"2024-09-10T08:05:25.489996+0000","last_unstale":"2024-09-10T08:08:03.667384+0000","last_undegraded":"2024-09-10T08:08:03.667384+0000","last_fullsized":"2024-09-10T08:08:03.667384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:15.637199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'3","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511918+0000","last_change":"2024-09-10T08:05:22.568696+0000","last_active":"2024-09-10T08:07:05.511918+0000","last_peered":"2024-09-10T08:07:05.511918+0000","last_clean":"2024-09-10T08:07:05.511918+0000","last_became_active":"2024-09-10T08:05:22.568331+0000","last_became_peered":"2024-09-10T08:05:22.568331+0000","last_unstale":"2024-09-10T08:07:05.511918+0000","last_undegraded":"2024-09-10T08:07:05.511918+0000","last_fullsized":"2024-09-10T08:07:05.511918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:21.699397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914428+0000","last_change":"2024-09-10T08:05:35.530804+0000","last_active":"2024-09-10T08:07:05.914428+0000","last_peered":"2024-09-10T08:07:05.914428+0000","last_clean":"2024-09-10T08:07:05.914428+0000","last_became_active":"2024-09-10T08:05:35.530356+0000","last_became_peered":"2024-09-10T08:05:35.530356+0000","last_unstale":"2024-09-10T08:07:05.914428+0000","last_undegraded":"2024-09-10T08:07:05.914428+0000","last_fullsized":"2024-09-10T08:07:05.914428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:59.318761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.662813+0000","last_change":"2024-09-10T08:05:25.489546+0000","last_active":"2024-09-10T08:08:03.662813+0000","last_peered":"2024-09-10T08:08:03.662813+0000","last_clean":"2024-09-10T08:08:03.662813+0000","last_became_active":"2024-09-10T08:05:25.489188+0000","last_became_peered":"2024-09-10T08:05:25.489188+0000","last_unstale":"2024-09-10T08:08:03.662813+0000","last_undegraded":"2024-09-10T08:08:03.662813+0000","last_fullsized":"2024-09-10T08:08:03.662813+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:26.471681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569736+0000","last_change":"2024-09-10T08:05:22.463912+0000","last_active":"2024-09-10T08:07:05.569736+0000","last_peered":"2024-09-10T08:07:05.569736+0000","last_clean":"2024-09-10T08:07:05.569736+0000","last_became_active":"2024-09-10T08:05:22.463561+0000","last_became_peered":"2024-09-10T08:05:22.463561+0000","last_unstale":"2024-09-10T08:07:05.569736+0000","last_undegraded":"2024-09-10T08:07:05.569736+0000","last_fullsized":"2024-09-10T08:07:05.569736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:22.407604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514959+0000","last_change":"2024-09-10T08:05:35.528795+0000","last_active":"2024-09-10T08:07:05.514959+0000","last_peered":"2024-09-10T08:07:05.514959+0000","last_clean":"2024-09-10T08:07:05.514959+0000","last_became_active":"2024-09-10T08:05:35.528402+0000","last_became_peered":"2024-09-10T08:05:35.528402+0000","last_unstale":"2024-09-10T08:07:05.514959+0000","last_undegraded":"2024-09-10T08:07:05.514959+0000","last_fullsized":"2024-09-10T08:07:05.514959+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:14.606371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.657641+0000","last_change":"2024-09-10T08:05:25.484086+0000","last_active":"2024-09-10T08:08:03.657641+0000","last_peered":"2024-09-10T08:08:03.657641+0000","last_clean":"2024-09-10T08:08:03.657641+0000","last_became_active":"2024-09-10T08:05:25.483757+0000","last_became_peered":"2024-09-10T08:05:25.483757+0000","last_unstale":"2024-09-10T08:08:03.657641+0000","last_undegraded":"2024-09-10T08:08:03.657641+0000","last_fullsized":"2024-09-10T08:08:03.657641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:16:16.362739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914678+0000","last_change":"2024-09-10T08:05:22.462220+0000","last_active":"2024-09-10T08:07:05.914678+0000","last_peered":"2024-09-10T08:07:05.914678+0000","last_clean":"2024-09-10T08:07:05.914678+0000","last_became_active":"2024-09-10T08:05:22.461726+0000","last_became_peered":"2024-09-10T08:05:22.461726+0000","last_unstale":"2024-09-10T08:07:05.914678+0000","last_undegraded":"2024-09-10T08:07:05.914678+0000","last_fullsized":"2024-09-10T08:07:05.914678+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:56.381034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508271+0000","last_change":"2024-09-10T08:06:04.734827+0000","last_active":"2024-09-10T08:07:05.508271+0000","last_peered":"2024-09-10T08:07:05.508271+0000","last_clean":"2024-09-10T08:07:05.508271+0000","last_became_active":"2024-09-10T08:06:04.734402+0000","last_became_peered":"2024-09-10T08:06:04.734402+0000","last_unstale":"2024-09-10T08:07:05.508271+0000","last_undegraded":"2024-09-10T08:07:05.508271+0000","last_fullsized":"2024-09-10T08:07:05.508271+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:24.806538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654332+0000","last_change":"2024-09-10T08:05:25.486100+0000","last_active":"2024-09-10T08:08:03.654332+0000","last_peered":"2024-09-10T08:08:03.654332+0000","last_clean":"2024-09-10T08:08:03.654332+0000","last_became_active":"2024-09-10T08:05:25.485586+0000","last_became_peered":"2024-09-10T08:05:25.485586+0000","last_unstale":"2024-09-10T08:08:03.654332+0000","last_undegraded":"2024-09-10T08:08:03.654332+0000","last_fullsized":"2024-09-10T08:08:03.654332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:33.565054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"93'2","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770403+0000","last_change":"2024-09-10T08:06:04.743408+0000","last_active":"2024-09-10T08:07:05.770403+0000","last_peered":"2024-09-10T08:07:05.770403+0000","last_clean":"2024-09-10T08:07:05.770403+0000","last_became_active":"2024-09-10T08:06:04.732830+0000","last_became_peered":"2024-09-10T08:06:04.732830+0000","last_unstale":"2024-09-10T08:07:05.770403+0000","last_undegraded":"2024-09-10T08:07:05.770403+0000","last_fullsized":"2024-09-10T08:07:05.770403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:29.381259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568977+0000","last_change":"2024-09-10T08:05:35.533015+0000","last_active":"2024-09-10T08:07:05.568977+0000","last_peered":"2024-09-10T08:07:05.568977+0000","last_clean":"2024-09-10T08:07:05.568977+0000","last_became_active":"2024-09-10T08:05:35.532590+0000","last_became_peered":"2024-09-10T08:05:35.532590+0000","last_unstale":"2024-09-10T08:07:05.568977+0000","last_undegraded":"2024-09-10T08:07:05.568977+0000","last_fullsized":"2024-09-10T08:07:05.568977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:23.984273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.650197+0000","last_change":"2024-09-10T08:05:25.489994+0000","last_active":"2024-09-10T08:08:03.650197+0000","last_peered":"2024-09-10T08:08:03.650197+0000","last_clean":"2024-09-10T08:08:03.650197+0000","last_became_active":"2024-09-10T08:05:25.489471+0000","last_became_peered":"2024-09-10T08:05:25.489471+0000","last_unstale":"2024-09-10T08:08:03.650197+0000","last_undegraded":"2024-09-10T08:08:03.650197+0000","last_fullsized":"2024-09-10T08:08:03.650197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:53.866570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"93'15","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508405+0000","last_change":"2024-09-10T08:05:22.568900+0000","last_active":"2024-09-10T08:07:05.508405+0000","last_peered":"2024-09-10T08:07:05.508405+0000","last_clean":"2024-09-10T08:07:05.508405+0000","last_became_active":"2024-09-10T08:05:22.568542+0000","last_became_peered":"2024-09-10T08:05:22.568542+0000","last_unstale":"2024-09-10T08:07:05.508405+0000","last_undegraded":"2024-09-10T08:07:05.508405+0000","last_fullsized":"2024-09-10T08:07:05.508405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:58:45.669769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914683+0000","last_change":"2024-09-10T08:05:35.537364+0000","last_active":"2024-09-10T08:07:05.914683+0000","last_peered":"2024-09-10T08:07:05.914683+0000","last_clean":"2024-09-10T08:07:05.914683+0000","last_became_active":"2024-09-10T08:05:35.535958+0000","last_became_peered":"2024-09-10T08:05:35.535958+0000","last_unstale":"2024-09-10T08:07:05.914683+0000","last_undegraded":"2024-09-10T08:07:05.914683+0000","last_fullsized":"2024-09-10T08:07:05.914683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:51.443798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660149+0000","last_change":"2024-09-10T08:07:05.917030+0000","last_active":"2024-09-10T08:08:03.660149+0000","last_peered":"2024-09-10T08:08:03.660149+0000","last_clean":"2024-09-10T08:08:03.660149+0000","last_became_active":"2024-09-10T08:07:05.916281+0000","last_became_peered":"2024-09-10T08:07:05.916281+0000","last_unstale":"2024-09-10T08:08:03.660149+0000","last_undegraded":"2024-09-10T08:08:03.660149+0000","last_fullsized":"2024-09-10T08:08:03.660149+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:15.060009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'4","reported_seq":171,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:20.445646+0000","last_change":"2024-09-10T08:05:22.459963+0000","last_active":"2024-09-10T08:08:20.445646+0000","last_peered":"2024-09-10T08:08:20.445646+0000","last_clean":"2024-09-10T08:08:20.445646+0000","last_became_active":"2024-09-10T08:05:22.459497+0000","last_became_peered":"2024-09-10T08:05:22.459497+0000","last_unstale":"2024-09-10T08:08:20.445646+0000","last_undegraded":"2024-09-10T08:08:20.445646+0000","last_fullsized":"2024-09-10T08:08:20.445646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:29.887980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513986+0000","last_change":"2024-09-10T08:05:35.534570+0000","last_active":"2024-09-10T08:07:05.513986+0000","last_peered":"2024-09-10T08:07:05.513986+0000","last_clean":"2024-09-10T08:07:05.513986+0000","last_became_active":"2024-09-10T08:05:35.534195+0000","last_became_peered":"2024-09-10T08:05:35.534195+0000","last_unstale":"2024-09-10T08:07:05.513986+0000","last_undegraded":"2024-09-10T08:07:05.513986+0000","last_fullsized":"2024-09-10T08:07:05.513986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:17.393491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.668227+0000","last_change":"2024-09-10T08:05:25.488551+0000","last_active":"2024-09-10T08:08:03.668227+0000","last_peered":"2024-09-10T08:08:03.668227+0000","last_clean":"2024-09-10T08:08:03.668227+0000","last_became_active":"2024-09-10T08:05:25.488086+0000","last_became_peered":"2024-09-10T08:05:25.488086+0000","last_unstale":"2024-09-10T08:08:03.668227+0000","last_undegraded":"2024-09-10T08:08:03.668227+0000","last_fullsized":"2024-09-10T08:08:03.668227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:38.044721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569070+0000","last_change":"2024-09-10T08:05:22.462269+0000","last_active":"2024-09-10T08:07:05.569070+0000","last_peered":"2024-09-10T08:07:05.569070+0000","last_clean":"2024-09-10T08:07:05.569070+0000","last_became_active":"2024-09-10T08:05:22.461924+0000","last_became_peered":"2024-09-10T08:05:22.461924+0000","last_unstale":"2024-09-10T08:07:05.569070+0000","last_undegraded":"2024-09-10T08:07:05.569070+0000","last_fullsized":"2024-09-10T08:07:05.569070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:54.996777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770191+0000","last_change":"2024-09-10T08:05:35.531596+0000","last_active":"2024-09-10T08:07:05.770191+0000","last_peered":"2024-09-10T08:07:05.770191+0000","last_clean":"2024-09-10T08:07:05.770191+0000","last_became_active":"2024-09-10T08:05:35.530179+0000","last_became_peered":"2024-09-10T08:05:35.530179+0000","last_unstale":"2024-09-10T08:07:05.770191+0000","last_undegraded":"2024-09-10T08:07:05.770191+0000","last_fullsized":"2024-09-10T08:07:05.770191+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:32.056426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.663218+0000","last_change":"2024-09-10T08:05:25.503504+0000","last_active":"2024-09-10T08:08:03.663218+0000","last_peered":"2024-09-10T08:08:03.663218+0000","last_clean":"2024-09-10T08:08:03.663218+0000","last_became_active":"2024-09-10T08:05:25.503105+0000","last_became_peered":"2024-09-10T08:05:25.503105+0000","last_unstale":"2024-09-10T08:08:03.663218+0000","last_undegraded":"2024-09-10T08:08:03.663218+0000","last_fullsized":"2024-09-10T08:08:03.663218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:40.885654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"93'6","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511623+0000","last_change":"2024-09-10T08:05:22.462271+0000","last_active":"2024-09-10T08:07:05.511623+0000","last_peered":"2024-09-10T08:07:05.511623+0000","last_clean":"2024-09-10T08:07:05.511623+0000","last_became_active":"2024-09-10T08:05:22.461195+0000","last_became_peered":"2024-09-10T08:05:22.461195+0000","last_unstale":"2024-09-10T08:07:05.511623+0000","last_undegraded":"2024-09-10T08:07:05.511623+0000","last_fullsized":"2024-09-10T08:07:05.511623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:20.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569486+0000","last_change":"2024-09-10T08:05:35.533218+0000","last_active":"2024-09-10T08:07:05.569486+0000","last_peered":"2024-09-10T08:07:05.569486+0000","last_clean":"2024-09-10T08:07:05.569486+0000","last_became_active":"2024-09-10T08:05:35.532706+0000","last_became_peered":"2024-09-10T08:05:35.532706+0000","last_unstale":"2024-09-10T08:07:05.569486+0000","last_undegraded":"2024-09-10T08:07:05.569486+0000","last_fullsized":"2024-09-10T08:07:05.569486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:22.150563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.655456+0000","last_change":"2024-09-10T08:05:25.478978+0000","last_active":"2024-09-10T08:08:03.655456+0000","last_peered":"2024-09-10T08:08:03.655456+0000","last_clean":"2024-09-10T08:08:03.655456+0000","last_became_active":"2024-09-10T08:05:25.478649+0000","last_became_peered":"2024-09-10T08:05:25.478649+0000","last_unstale":"2024-09-10T08:08:03.655456+0000","last_undegraded":"2024-09-10T08:08:03.655456+0000","last_fullsized":"2024-09-10T08:08:03.655456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:17.493446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'40","reported_seq":361,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:25.453422+0000","last_change":"2024-09-10T08:06:04.740972+0000","last_active":"2024-09-10T08:08:25.453422+0000","last_peered":"2024-09-10T08:08:25.453422+0000","last_clean":"2024-09-10T08:08:25.453422+0000","last_became_active":"2024-09-10T08:06:04.731200+0000","last_became_peered":"2024-09-10T08:06:04.731200+0000","last_unstale":"2024-09-10T08:08:25.453422+0000","last_undegraded":"2024-09-10T08:08:25.453422+0000","last_fullsized":"2024-09-10T08:08:25.453422+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:17.169708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71507,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":261,"num_read_kb":201,"num_write":41,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3703,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512781+0000","last_change":"2024-09-10T08:05:35.532594+0000","last_active":"2024-09-10T08:07:05.512781+0000","last_peered":"2024-09-10T08:07:05.512781+0000","last_clean":"2024-09-10T08:07:05.512781+0000","last_became_active":"2024-09-10T08:05:35.532132+0000","last_became_peered":"2024-09-10T08:05:35.532132+0000","last_unstale":"2024-09-10T08:07:05.512781+0000","last_undegraded":"2024-09-10T08:07:05.512781+0000","last_fullsized":"2024-09-10T08:07:05.512781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:15.348648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.666577+0000","last_change":"2024-09-10T08:05:25.485297+0000","last_active":"2024-09-10T08:08:03.666577+0000","last_peered":"2024-09-10T08:08:03.666577+0000","last_clean":"2024-09-10T08:08:03.666577+0000","last_became_active":"2024-09-10T08:05:25.484883+0000","last_became_peered":"2024-09-10T08:05:25.484883+0000","last_unstale":"2024-09-10T08:08:03.666577+0000","last_undegraded":"2024-09-10T08:08:03.666577+0000","last_fullsized":"2024-09-10T08:08:03.666577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:37.052435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"93'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569713+0000","last_change":"2024-09-10T08:05:22.568993+0000","last_active":"2024-09-10T08:07:05.569713+0000","last_peered":"2024-09-10T08:07:05.569713+0000","last_clean":"2024-09-10T08:07:05.569713+0000","last_became_active":"2024-09-10T08:05:22.568564+0000","last_became_peered":"2024-09-10T08:05:22.568564+0000","last_unstale":"2024-09-10T08:07:05.569713+0000","last_undegraded":"2024-09-10T08:07:05.569713+0000","last_fullsized":"2024-09-10T08:07:05.569713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:23.767174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914556+0000","last_change":"2024-09-10T08:05:35.524025+0000","last_active":"2024-09-10T08:07:05.914556+0000","last_peered":"2024-09-10T08:07:05.914556+0000","last_clean":"2024-09-10T08:07:05.914556+0000","last_became_active":"2024-09-10T08:05:35.523628+0000","last_became_peered":"2024-09-10T08:05:35.523628+0000","last_unstale":"2024-09-10T08:07:05.914556+0000","last_undegraded":"2024-09-10T08:07:05.914556+0000","last_fullsized":"2024-09-10T08:07:05.914556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:48.593779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.680405+0000","last_change":"2024-09-10T08:07:05.570537+0000","last_active":"2024-09-10T08:08:03.680405+0000","last_peered":"2024-09-10T08:08:03.680405+0000","last_clean":"2024-09-10T08:08:03.680405+0000","last_became_active":"2024-09-10T08:07:05.570278+0000","last_became_peered":"2024-09-10T08:07:05.570278+0000","last_unstale":"2024-09-10T08:08:03.680405+0000","last_undegraded":"2024-09-10T08:08:03.680405+0000","last_fullsized":"2024-09-10T08:08:03.680405+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:56.490462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509635+0000","last_change":"2024-09-10T08:05:22.460487+0000","last_active":"2024-09-10T08:07:05.509635+0000","last_peered":"2024-09-10T08:07:05.509635+0000","last_clean":"2024-09-10T08:07:05.509635+0000","last_became_active":"2024-09-10T08:05:22.460158+0000","last_became_peered":"2024-09-10T08:05:22.460158+0000","last_unstale":"2024-09-10T08:07:05.509635+0000","last_undegraded":"2024-09-10T08:07:05.509635+0000","last_fullsized":"2024-09-10T08:07:05.509635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:53.285513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512495+0000","last_change":"2024-09-10T08:05:35.531399+0000","last_active":"2024-09-10T08:07:05.512495+0000","last_peered":"2024-09-10T08:07:05.512495+0000","last_clean":"2024-09-10T08:07:05.512495+0000","last_became_active":"2024-09-10T08:05:35.530320+0000","last_became_peered":"2024-09-10T08:05:35.530320+0000","last_unstale":"2024-09-10T08:07:05.512495+0000","last_undegraded":"2024-09-10T08:07:05.512495+0000","last_fullsized":"2024-09-10T08:07:05.512495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:19.814046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669147+0000","last_change":"2024-09-10T08:05:25.488208+0000","last_active":"2024-09-10T08:08:03.669147+0000","last_peered":"2024-09-10T08:08:03.669147+0000","last_clean":"2024-09-10T08:08:03.669147+0000","last_became_active":"2024-09-10T08:05:25.487893+0000","last_became_peered":"2024-09-10T08:05:25.487893+0000","last_unstale":"2024-09-10T08:08:03.669147+0000","last_undegraded":"2024-09-10T08:08:03.669147+0000","last_fullsized":"2024-09-10T08:08:03.669147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:10.999416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":173,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:24.452115+0000","last_change":"2024-09-10T08:05:22.461321+0000","last_active":"2024-09-10T08:08:24.452115+0000","last_peered":"2024-09-10T08:08:24.452115+0000","last_clean":"2024-09-10T08:08:24.452115+0000","last_became_active":"2024-09-10T08:05:22.460950+0000","last_became_peered":"2024-09-10T08:05:22.460950+0000","last_unstale":"2024-09-10T08:08:24.452115+0000","last_undegraded":"2024-09-10T08:08:24.452115+0000","last_fullsized":"2024-09-10T08:08:24.452115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:55.647273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876534+0000","last_change":"2024-09-10T08:05:35.530578+0000","last_active":"2024-09-10T08:07:05.876534+0000","last_peered":"2024-09-10T08:07:05.876534+0000","last_clean":"2024-09-10T08:07:05.876534+0000","last_became_active":"2024-09-10T08:05:35.530162+0000","last_became_peered":"2024-09-10T08:05:35.530162+0000","last_unstale":"2024-09-10T08:07:05.876534+0000","last_undegraded":"2024-09-10T08:07:05.876534+0000","last_fullsized":"2024-09-10T08:07:05.876534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:19.290973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513617+0000","last_change":"2024-09-10T08:05:35.532685+0000","last_active":"2024-09-10T08:07:05.513617+0000","last_peered":"2024-09-10T08:07:05.513617+0000","last_clean":"2024-09-10T08:07:05.513617+0000","last_became_active":"2024-09-10T08:05:35.532257+0000","last_became_peered":"2024-09-10T08:05:35.532257+0000","last_unstale":"2024-09-10T08:07:05.513617+0000","last_undegraded":"2024-09-10T08:07:05.513617+0000","last_fullsized":"2024-09-10T08:07:05.513617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:07.412435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.662135+0000","last_change":"2024-09-10T08:05:25.494472+0000","last_active":"2024-09-10T08:08:03.662135+0000","last_peered":"2024-09-10T08:08:03.662135+0000","last_clean":"2024-09-10T08:08:03.662135+0000","last_became_active":"2024-09-10T08:05:25.493928+0000","last_became_peered":"2024-09-10T08:05:25.493928+0000","last_unstale":"2024-09-10T08:08:03.662135+0000","last_undegraded":"2024-09-10T08:08:03.662135+0000","last_fullsized":"2024-09-10T08:08:03.662135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:00.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"93'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568928+0000","last_change":"2024-09-10T08:05:22.460585+0000","last_active":"2024-09-10T08:07:05.568928+0000","last_peered":"2024-09-10T08:07:05.568928+0000","last_clean":"2024-09-10T08:07:05.568928+0000","last_became_active":"2024-09-10T08:05:22.460259+0000","last_became_peered":"2024-09-10T08:05:22.460259+0000","last_unstale":"2024-09-10T08:07:05.568928+0000","last_undegraded":"2024-09-10T08:07:05.568928+0000","last_fullsized":"2024-09-10T08:07:05.568928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:45.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770275+0000","last_change":"2024-09-10T08:05:35.524000+0000","last_active":"2024-09-10T08:07:05.770275+0000","last_peered":"2024-09-10T08:07:05.770275+0000","last_clean":"2024-09-10T08:07:05.770275+0000","last_became_active":"2024-09-10T08:05:35.523647+0000","last_became_peered":"2024-09-10T08:05:35.523647+0000","last_unstale":"2024-09-10T08:07:05.770275+0000","last_undegraded":"2024-09-10T08:07:05.770275+0000","last_fullsized":"2024-09-10T08:07:05.770275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:10:54.664865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660563+0000","last_change":"2024-09-10T08:05:25.499701+0000","last_active":"2024-09-10T08:08:03.660563+0000","last_peered":"2024-09-10T08:08:03.660563+0000","last_clean":"2024-09-10T08:08:03.660563+0000","last_became_active":"2024-09-10T08:05:25.496890+0000","last_became_peered":"2024-09-10T08:05:25.496890+0000","last_unstale":"2024-09-10T08:08:03.660563+0000","last_undegraded":"2024-09-10T08:08:03.660563+0000","last_fullsized":"2024-09-10T08:08:03.660563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:58.166365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914070+0000","last_change":"2024-09-10T08:05:22.460880+0000","last_active":"2024-09-10T08:07:05.914070+0000","last_peered":"2024-09-10T08:07:05.914070+0000","last_clean":"2024-09-10T08:07:05.914070+0000","last_became_active":"2024-09-10T08:05:22.459932+0000","last_became_peered":"2024-09-10T08:05:22.459932+0000","last_unstale":"2024-09-10T08:07:05.914070+0000","last_undegraded":"2024-09-10T08:07:05.914070+0000","last_fullsized":"2024-09-10T08:07:05.914070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:55.826526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915405+0000","last_change":"2024-09-10T08:05:22.460578+0000","last_active":"2024-09-10T08:07:05.915405+0000","last_peered":"2024-09-10T08:07:05.915405+0000","last_clean":"2024-09-10T08:07:05.915405+0000","last_became_active":"2024-09-10T08:05:22.460239+0000","last_became_peered":"2024-09-10T08:05:22.460239+0000","last_unstale":"2024-09-10T08:07:05.915405+0000","last_undegraded":"2024-09-10T08:07:05.915405+0000","last_fullsized":"2024-09-10T08:07:05.915405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:07.606243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.661142+0000","last_change":"2024-09-10T08:05:25.489846+0000","last_active":"2024-09-10T08:08:03.661142+0000","last_peered":"2024-09-10T08:08:03.661142+0000","last_clean":"2024-09-10T08:08:03.661142+0000","last_became_active":"2024-09-10T08:05:25.489326+0000","last_became_peered":"2024-09-10T08:05:25.489326+0000","last_unstale":"2024-09-10T08:08:03.661142+0000","last_undegraded":"2024-09-10T08:08:03.661142+0000","last_fullsized":"2024-09-10T08:08:03.661142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:43.458468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511635+0000","last_change":"2024-09-10T08:05:35.530522+0000","last_active":"2024-09-10T08:07:05.511635+0000","last_peered":"2024-09-10T08:07:05.511635+0000","last_clean":"2024-09-10T08:07:05.511635+0000","last_became_active":"2024-09-10T08:05:35.530099+0000","last_became_peered":"2024-09-10T08:05:35.530099+0000","last_unstale":"2024-09-10T08:07:05.511635+0000","last_undegraded":"2024-09-10T08:07:05.511635+0000","last_fullsized":"2024-09-10T08:07:05.511635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:58.469119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":179,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:20.446528+0000","last_change":"2024-09-10T08:05:22.461391+0000","last_active":"2024-09-10T08:08:20.446528+0000","last_peered":"2024-09-10T08:08:20.446528+0000","last_clean":"2024-09-10T08:08:20.446528+0000","last_became_active":"2024-09-10T08:05:22.461042+0000","last_became_peered":"2024-09-10T08:05:22.461042+0000","last_unstale":"2024-09-10T08:08:20.446528+0000","last_undegraded":"2024-09-10T08:08:20.446528+0000","last_fullsized":"2024-09-10T08:08:20.446528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:12.635710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1860,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.649481+0000","last_change":"2024-09-10T08:05:25.487378+0000","last_active":"2024-09-10T08:08:03.649481+0000","last_peered":"2024-09-10T08:08:03.649481+0000","last_clean":"2024-09-10T08:08:03.649481+0000","last_became_active":"2024-09-10T08:05:25.487055+0000","last_became_peered":"2024-09-10T08:05:25.487055+0000","last_unstale":"2024-09-10T08:08:03.649481+0000","last_undegraded":"2024-09-10T08:08:03.649481+0000","last_fullsized":"2024-09-10T08:08:03.649481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:59.522014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511748+0000","last_change":"2024-09-10T08:05:35.522765+0000","last_active":"2024-09-10T08:07:05.511748+0000","last_peered":"2024-09-10T08:07:05.511748+0000","last_clean":"2024-09-10T08:07:05.511748+0000","last_became_active":"2024-09-10T08:05:35.522359+0000","last_became_peered":"2024-09-10T08:05:35.522359+0000","last_unstale":"2024-09-10T08:07:05.511748+0000","last_undegraded":"2024-09-10T08:07:05.511748+0000","last_fullsized":"2024-09-10T08:07:05.511748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:15.280867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.652869+0000","last_change":"2024-09-10T08:05:25.475388+0000","last_active":"2024-09-10T08:08:03.652869+0000","last_peered":"2024-09-10T08:08:03.652869+0000","last_clean":"2024-09-10T08:08:03.652869+0000","last_became_active":"2024-09-10T08:05:25.474331+0000","last_became_peered":"2024-09-10T08:05:25.474331+0000","last_unstale":"2024-09-10T08:08:03.652869+0000","last_undegraded":"2024-09-10T08:08:03.652869+0000","last_fullsized":"2024-09-10T08:08:03.652869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:58.713148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656383+0000","last_change":"2024-09-10T08:07:05.570505+0000","last_active":"2024-09-10T08:08:03.656383+0000","last_peered":"2024-09-10T08:08:03.656383+0000","last_clean":"2024-09-10T08:08:03.656383+0000","last_became_active":"2024-09-10T08:07:05.569970+0000","last_became_peered":"2024-09-10T08:07:05.569970+0000","last_unstale":"2024-09-10T08:08:03.656383+0000","last_undegraded":"2024-09-10T08:08:03.656383+0000","last_fullsized":"2024-09-10T08:08:03.656383+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:13.282503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671951+0000","last_change":"2024-09-10T08:05:25.491585+0000","last_active":"2024-09-10T08:08:03.671951+0000","last_peered":"2024-09-10T08:08:03.671951+0000","last_clean":"2024-09-10T08:08:03.671951+0000","last_became_active":"2024-09-10T08:05:25.491224+0000","last_became_peered":"2024-09-10T08:05:25.491224+0000","last_unstale":"2024-09-10T08:08:03.671951+0000","last_undegraded":"2024-09-10T08:08:03.671951+0000","last_fullsized":"2024-09-10T08:08:03.671951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:56.066808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681502+0000","last_change":"2024-09-10T08:05:25.499578+0000","last_active":"2024-09-10T08:08:03.681502+0000","last_peered":"2024-09-10T08:08:03.681502+0000","last_clean":"2024-09-10T08:08:03.681502+0000","last_became_active":"2024-09-10T08:05:25.498445+0000","last_became_peered":"2024-09-10T08:05:25.498445+0000","last_unstale":"2024-09-10T08:08:03.681502+0000","last_undegraded":"2024-09-10T08:08:03.681502+0000","last_fullsized":"2024-09-10T08:08:03.681502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:58.961699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675717+0000","last_change":"2024-09-10T08:06:04.735764+0000","last_active":"2024-09-10T08:08:03.675717+0000","last_peered":"2024-09-10T08:08:03.675717+0000","last_clean":"2024-09-10T08:08:03.675717+0000","last_became_active":"2024-09-10T08:06:04.735366+0000","last_became_peered":"2024-09-10T08:06:04.735366+0000","last_unstale":"2024-09-10T08:08:03.675717+0000","last_undegraded":"2024-09-10T08:08:03.675717+0000","last_fullsized":"2024-09-10T08:08:03.675717+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:03.757890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651587+0000","last_change":"2024-09-10T08:05:25.475872+0000","last_active":"2024-09-10T08:08:03.651587+0000","last_peered":"2024-09-10T08:08:03.651587+0000","last_clean":"2024-09-10T08:08:03.651587+0000","last_became_active":"2024-09-10T08:05:25.475549+0000","last_became_peered":"2024-09-10T08:05:25.475549+0000","last_unstale":"2024-09-10T08:08:03.651587+0000","last_undegraded":"2024-09-10T08:08:03.651587+0000","last_fullsized":"2024-09-10T08:08:03.651587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:37.575267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.664667+0000","last_change":"2024-09-10T08:05:25.474210+0000","last_active":"2024-09-10T08:08:03.664667+0000","last_peered":"2024-09-10T08:08:03.664667+0000","last_clean":"2024-09-10T08:08:03.664667+0000","last_became_active":"2024-09-10T08:05:25.473882+0000","last_became_peered":"2024-09-10T08:05:25.473882+0000","last_unstale":"2024-09-10T08:08:03.664667+0000","last_undegraded":"2024-09-10T08:08:03.664667+0000","last_fullsized":"2024-09-10T08:08:03.664667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:05.164370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651519+0000","last_change":"2024-09-10T08:05:25.490406+0000","last_active":"2024-09-10T08:08:03.651519+0000","last_peered":"2024-09-10T08:08:03.651519+0000","last_clean":"2024-09-10T08:08:03.651519+0000","last_became_active":"2024-09-10T08:05:25.490061+0000","last_became_peered":"2024-09-10T08:05:25.490061+0000","last_unstale":"2024-09-10T08:08:03.651519+0000","last_undegraded":"2024-09-10T08:08:03.651519+0000","last_fullsized":"2024-09-10T08:08:03.651519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:12.988229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.664045+0000","last_change":"2024-09-10T08:05:25.504049+0000","last_active":"2024-09-10T08:08:03.664045+0000","last_peered":"2024-09-10T08:08:03.664045+0000","last_clean":"2024-09-10T08:08:03.664045+0000","last_became_active":"2024-09-10T08:05:25.503770+0000","last_became_peered":"2024-09-10T08:05:25.503770+0000","last_unstale":"2024-09-10T08:08:03.664045+0000","last_undegraded":"2024-09-10T08:08:03.664045+0000","last_fullsized":"2024-09-10T08:08:03.664045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:52.850945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.672614+0000","last_change":"2024-09-10T08:05:25.491682+0000","last_active":"2024-09-10T08:08:03.672614+0000","last_peered":"2024-09-10T08:08:03.672614+0000","last_clean":"2024-09-10T08:08:03.672614+0000","last_became_active":"2024-09-10T08:05:25.491296+0000","last_became_peered":"2024-09-10T08:05:25.491296+0000","last_unstale":"2024-09-10T08:08:03.672614+0000","last_undegraded":"2024-09-10T08:08:03.672614+0000","last_fullsized":"2024-09-10T08:08:03.672614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:30.075842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.673509+0000","last_change":"2024-09-10T08:05:25.489648+0000","last_active":"2024-09-10T08:08:03.673509+0000","last_peered":"2024-09-10T08:08:03.673509+0000","last_clean":"2024-09-10T08:08:03.673509+0000","last_became_active":"2024-09-10T08:05:25.489267+0000","last_became_peered":"2024-09-10T08:05:25.489267+0000","last_unstale":"2024-09-10T08:08:03.673509+0000","last_undegraded":"2024-09-10T08:08:03.673509+0000","last_fullsized":"2024-09-10T08:08:03.673509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:21.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681690+0000","last_change":"2024-09-10T08:05:25.497862+0000","last_active":"2024-09-10T08:08:03.681690+0000","last_peered":"2024-09-10T08:08:03.681690+0000","last_clean":"2024-09-10T08:08:03.681690+0000","last_became_active":"2024-09-10T08:05:25.497504+0000","last_became_peered":"2024-09-10T08:05:25.497504+0000","last_unstale":"2024-09-10T08:08:03.681690+0000","last_undegraded":"2024-09-10T08:08:03.681690+0000","last_fullsized":"2024-09-10T08:08:03.681690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:40.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654484+0000","last_change":"2024-09-10T08:05:25.485924+0000","last_active":"2024-09-10T08:08:03.654484+0000","last_peered":"2024-09-10T08:08:03.654484+0000","last_clean":"2024-09-10T08:08:03.654484+0000","last_became_active":"2024-09-10T08:05:25.485545+0000","last_became_peered":"2024-09-10T08:05:25.485545+0000","last_unstale":"2024-09-10T08:08:03.654484+0000","last_undegraded":"2024-09-10T08:08:03.654484+0000","last_fullsized":"2024-09-10T08:08:03.654484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:33:20.233720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671966+0000","last_change":"2024-09-10T08:05:25.491911+0000","last_active":"2024-09-10T08:08:03.671966+0000","last_peered":"2024-09-10T08:08:03.671966+0000","last_clean":"2024-09-10T08:08:03.671966+0000","last_became_active":"2024-09-10T08:05:25.491330+0000","last_became_peered":"2024-09-10T08:05:25.491330+0000","last_unstale":"2024-09-10T08:08:03.671966+0000","last_undegraded":"2024-09-10T08:08:03.671966+0000","last_fullsized":"2024-09-10T08:08:03.671966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:06.043331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.655352+0000","last_change":"2024-09-10T08:05:25.492220+0000","last_active":"2024-09-10T08:08:03.655352+0000","last_peered":"2024-09-10T08:08:03.655352+0000","last_clean":"2024-09-10T08:08:03.655352+0000","last_became_active":"2024-09-10T08:05:25.491473+0000","last_became_peered":"2024-09-10T08:05:25.491473+0000","last_unstale":"2024-09-10T08:08:03.655352+0000","last_undegraded":"2024-09-10T08:08:03.655352+0000","last_fullsized":"2024-09-10T08:08:03.655352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:13:30.278434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676184+0000","last_change":"2024-09-10T08:05:25.488927+0000","last_active":"2024-09-10T08:08:03.676184+0000","last_peered":"2024-09-10T08:08:03.676184+0000","last_clean":"2024-09-10T08:08:03.676184+0000","last_became_active":"2024-09-10T08:05:25.488550+0000","last_became_peered":"2024-09-10T08:05:25.488550+0000","last_unstale":"2024-09-10T08:08:03.676184+0000","last_undegraded":"2024-09-10T08:08:03.676184+0000","last_fullsized":"2024-09-10T08:08:03.676184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:44.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.677654+0000","last_change":"2024-09-10T08:05:25.490125+0000","last_active":"2024-09-10T08:08:03.677654+0000","last_peered":"2024-09-10T08:08:03.677654+0000","last_clean":"2024-09-10T08:08:03.677654+0000","last_became_active":"2024-09-10T08:05:25.487805+0000","last_became_peered":"2024-09-10T08:05:25.487805+0000","last_unstale":"2024-09-10T08:08:03.677654+0000","last_undegraded":"2024-09-10T08:08:03.677654+0000","last_fullsized":"2024-09-10T08:08:03.677654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:33.479009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676794+0000","last_change":"2024-09-10T08:05:25.500346+0000","last_active":"2024-09-10T08:08:03.676794+0000","last_peered":"2024-09-10T08:08:03.676794+0000","last_clean":"2024-09-10T08:08:03.676794+0000","last_became_active":"2024-09-10T08:05:25.498289+0000","last_became_peered":"2024-09-10T08:05:25.498289+0000","last_unstale":"2024-09-10T08:08:03.676794+0000","last_undegraded":"2024-09-10T08:08:03.676794+0000","last_fullsized":"2024-09-10T08:08:03.676794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:12.054926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676196+0000","last_change":"2024-09-10T08:05:25.490001+0000","last_active":"2024-09-10T08:08:03.676196+0000","last_peered":"2024-09-10T08:08:03.676196+0000","last_clean":"2024-09-10T08:08:03.676196+0000","last_became_active":"2024-09-10T08:05:25.489706+0000","last_became_peered":"2024-09-10T08:05:25.489706+0000","last_unstale":"2024-09-10T08:08:03.676196+0000","last_undegraded":"2024-09-10T08:08:03.676196+0000","last_fullsized":"2024-09-10T08:08:03.676196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:38.822140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669334+0000","last_change":"2024-09-10T08:05:25.493874+0000","last_active":"2024-09-10T08:08:03.669334+0000","last_peered":"2024-09-10T08:08:03.669334+0000","last_clean":"2024-09-10T08:08:03.669334+0000","last_became_active":"2024-09-10T08:05:25.489911+0000","last_became_peered":"2024-09-10T08:05:25.489911+0000","last_unstale":"2024-09-10T08:08:03.669334+0000","last_undegraded":"2024-09-10T08:08:03.669334+0000","last_fullsized":"2024-09-10T08:08:03.669334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:46.509657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654960+0000","last_change":"2024-09-10T08:05:25.490277+0000","last_active":"2024-09-10T08:08:03.654960+0000","last_peered":"2024-09-10T08:08:03.654960+0000","last_clean":"2024-09-10T08:08:03.654960+0000","last_became_active":"2024-09-10T08:05:25.486863+0000","last_became_peered":"2024-09-10T08:05:25.486863+0000","last_unstale":"2024-09-10T08:08:03.654960+0000","last_undegraded":"2024-09-10T08:08:03.654960+0000","last_fullsized":"2024-09-10T08:08:03.654960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:21.353830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656794+0000","last_change":"2024-09-10T08:05:25.491028+0000","last_active":"2024-09-10T08:08:03.656794+0000","last_peered":"2024-09-10T08:08:03.656794+0000","last_clean":"2024-09-10T08:08:03.656794+0000","last_became_active":"2024-09-10T08:05:25.490452+0000","last_became_peered":"2024-09-10T08:05:25.490452+0000","last_unstale":"2024-09-10T08:08:03.656794+0000","last_undegraded":"2024-09-10T08:08:03.656794+0000","last_fullsized":"2024-09-10T08:08:03.656794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:53.059077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660628+0000","last_change":"2024-09-10T08:05:25.489991+0000","last_active":"2024-09-10T08:08:03.660628+0000","last_peered":"2024-09-10T08:08:03.660628+0000","last_clean":"2024-09-10T08:08:03.660628+0000","last_became_active":"2024-09-10T08:05:25.489662+0000","last_became_peered":"2024-09-10T08:05:25.489662+0000","last_unstale":"2024-09-10T08:08:03.660628+0000","last_undegraded":"2024-09-10T08:08:03.660628+0000","last_fullsized":"2024-09-10T08:08:03.660628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:47.940712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667243+0000","last_change":"2024-09-10T08:05:25.475752+0000","last_active":"2024-09-10T08:08:03.667243+0000","last_peered":"2024-09-10T08:08:03.667243+0000","last_clean":"2024-09-10T08:08:03.667243+0000","last_became_active":"2024-09-10T08:05:25.475395+0000","last_became_peered":"2024-09-10T08:05:25.475395+0000","last_unstale":"2024-09-10T08:08:03.667243+0000","last_undegraded":"2024-09-10T08:08:03.667243+0000","last_fullsized":"2024-09-10T08:08:03.667243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.240154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.672245+0000","last_change":"2024-09-10T08:05:25.503061+0000","last_active":"2024-09-10T08:08:03.672245+0000","last_peered":"2024-09-10T08:08:03.672245+0000","last_clean":"2024-09-10T08:08:03.672245+0000","last_became_active":"2024-09-10T08:05:25.501333+0000","last_became_peered":"2024-09-10T08:05:25.501333+0000","last_unstale":"2024-09-10T08:08:03.672245+0000","last_undegraded":"2024-09-10T08:08:03.672245+0000","last_fullsized":"2024-09-10T08:08:03.672245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:38.254777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.661256+0000","last_change":"2024-09-10T08:07:05.514631+0000","last_active":"2024-09-10T08:08:03.661256+0000","last_peered":"2024-09-10T08:08:03.661256+0000","last_clean":"2024-09-10T08:08:03.661256+0000","last_became_active":"2024-09-10T08:07:05.514285+0000","last_became_peered":"2024-09-10T08:07:05.514285+0000","last_unstale":"2024-09-10T08:08:03.661256+0000","last_undegraded":"2024-09-10T08:08:03.661256+0000","last_fullsized":"2024-09-10T08:08:03.661256+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:40.293121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660092+0000","last_change":"2024-09-10T08:05:25.491457+0000","last_active":"2024-09-10T08:08:03.660092+0000","last_peered":"2024-09-10T08:08:03.660092+0000","last_clean":"2024-09-10T08:08:03.660092+0000","last_became_active":"2024-09-10T08:05:25.491162+0000","last_became_peered":"2024-09-10T08:05:25.491162+0000","last_unstale":"2024-09-10T08:08:03.660092+0000","last_undegraded":"2024-09-10T08:08:03.660092+0000","last_fullsized":"2024-09-10T08:08:03.660092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:14.389752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660737+0000","last_change":"2024-09-10T08:05:25.487874+0000","last_active":"2024-09-10T08:08:03.660737+0000","last_peered":"2024-09-10T08:08:03.660737+0000","last_clean":"2024-09-10T08:08:03.660737+0000","last_became_active":"2024-09-10T08:05:25.487497+0000","last_became_peered":"2024-09-10T08:05:25.487497+0000","last_unstale":"2024-09-10T08:08:03.660737+0000","last_undegraded":"2024-09-10T08:08:03.660737+0000","last_fullsized":"2024-09-10T08:08:03.660737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:26.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658875+0000","last_change":"2024-09-10T08:05:25.490018+0000","last_active":"2024-09-10T08:08:03.658875+0000","last_peered":"2024-09-10T08:08:03.658875+0000","last_clean":"2024-09-10T08:08:03.658875+0000","last_became_active":"2024-09-10T08:05:25.484951+0000","last_became_peered":"2024-09-10T08:05:25.484951+0000","last_unstale":"2024-09-10T08:08:03.658875+0000","last_undegraded":"2024-09-10T08:08:03.658875+0000","last_fullsized":"2024-09-10T08:08:03.658875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:07.516154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.677343+0000","last_change":"2024-09-10T08:05:25.501581+0000","last_active":"2024-09-10T08:08:03.677343+0000","last_peered":"2024-09-10T08:08:03.677343+0000","last_clean":"2024-09-10T08:08:03.677343+0000","last_became_active":"2024-09-10T08:05:25.499690+0000","last_became_peered":"2024-09-10T08:05:25.499690+0000","last_unstale":"2024-09-10T08:08:03.677343+0000","last_undegraded":"2024-09-10T08:08:03.677343+0000","last_fullsized":"2024-09-10T08:08:03.677343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:14.569791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.678243+0000","last_change":"2024-09-10T08:05:25.487945+0000","last_active":"2024-09-10T08:08:03.678243+0000","last_peered":"2024-09-10T08:08:03.678243+0000","last_clean":"2024-09-10T08:08:03.678243+0000","last_became_active":"2024-09-10T08:05:25.487655+0000","last_became_peered":"2024-09-10T08:05:25.487655+0000","last_unstale":"2024-09-10T08:08:03.678243+0000","last_undegraded":"2024-09-10T08:08:03.678243+0000","last_fullsized":"2024-09-10T08:08:03.678243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:37.746037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675926+0000","last_change":"2024-09-10T08:05:25.489348+0000","last_active":"2024-09-10T08:08:03.675926+0000","last_peered":"2024-09-10T08:08:03.675926+0000","last_clean":"2024-09-10T08:08:03.675926+0000","last_became_active":"2024-09-10T08:05:25.489004+0000","last_became_peered":"2024-09-10T08:05:25.489004+0000","last_unstale":"2024-09-10T08:08:03.675926+0000","last_undegraded":"2024-09-10T08:08:03.675926+0000","last_fullsized":"2024-09-10T08:08:03.675926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:54.554719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76748,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":769,"num_read_kb":469,"num_write":190,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3725,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":450560,"data_stored":227724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214976,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514546,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":548036,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743480,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32364,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712788,"statfs":{"total":95995035648,"available":95961894912,"internally_reserved":0,"allocated":1208320,"data_stored":1211278,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907003,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":548250,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135939,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":548148,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364870,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1114112,"data_stored":1145319,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528396,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":543426,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1122304,"data_stored":1140514,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":617456,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":543620,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":545924,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":614843,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":544814,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":73478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:08:25.968 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:08:26.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:25 smithi155 ceph-mon[35405]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:26.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:25 smithi155 ceph-mon[35405]: from='client.25208 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:26.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:25 smithi107 ceph-mon[35048]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:26.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:25 smithi107 ceph-mon[35048]: from='client.25208 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:26.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:25 smithi002 ceph-mon[30563]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:26.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:25 smithi002 ceph-mon[30563]: from='client.25208 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:26.378 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.901+0000 7f38f2000640 1 -- 172.21.15.2:0/3946731074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38ec074620 msgr2=0x7f38ec074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.901+0000 7f38f2000640 1 --2- 172.21.15.2:0/3946731074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38ec074620 0x7f38ec074a00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009900 tx=0x7f38dc02a480 comp rx=0 tx=0).stop 2024-09-10T08:08:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.902+0000 7f38f2000640 1 -- 172.21.15.2:0/3946731074 shutdown_connections 2024-09-10T08:08:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.902+0000 7f38f2000640 1 --2- 172.21.15.2:0/3946731074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec10e3c0 0x7f38ec110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.902+0000 7f38f2000640 1 --2- 172.21.15.2:0/3946731074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38ec074f40 0x7f38ec10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.902+0000 7f38f2000640 1 --2- 172.21.15.2:0/3946731074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38ec074620 0x7f38ec074a00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.902+0000 7f38f2000640 1 -- 172.21.15.2:0/3946731074 >> 172.21.15.2:0/3946731074 conn(0x7f38ec0fd8c0 msgr2=0x7f38ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.902+0000 7f38f2000640 1 -- 172.21.15.2:0/3946731074 shutdown_connections 2024-09-10T08:08:26.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.902+0000 7f38f2000640 1 -- 172.21.15.2:0/3946731074 wait complete. 2024-09-10T08:08:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.903+0000 7f38f2000640 1 Processor -- start 2024-09-10T08:08:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.903+0000 7f38f2000640 1 -- start start 2024-09-10T08:08:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.903+0000 7f38f2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38ec074620 0x7f38ec19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.903+0000 7f38f2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec074f40 0x7f38ec19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.903+0000 7f38f2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38ec10e3c0 0x7f38ec19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.903+0000 7f38f2000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1141a0 con 0x7f38ec10e3c0 2024-09-10T08:08:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.903+0000 7f38f2000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec114020 con 0x7f38ec074f40 2024-09-10T08:08:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.903+0000 7f38f2000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec114320 con 0x7f38ec074620 2024-09-10T08:08:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.904+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38ec10e3c0 0x7f38ec19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.904+0000 7f38e3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec074f40 0x7f38ec19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.904+0000 7f38eb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38ec074620 0x7f38ec19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.904+0000 7f38ebe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38ec10e3c0 0x7f38ec19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39754/0 (socket says 172.21.15.2:39754) 2024-09-10T08:08:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.904+0000 7f38ebe00640 1 -- 172.21.15.2:0/1020615197 learned_addr learned my addr 172.21.15.2:0/1020615197 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.904+0000 7f38e3e00640 1 -- 172.21.15.2:0/1020615197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38ec074620 msgr2=0x7f38ec19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.904+0000 7f38e3e00640 1 --2- 172.21.15.2:0/1020615197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38ec074620 0x7f38ec19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.904+0000 7f38e3e00640 1 -- 172.21.15.2:0/1020615197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38ec10e3c0 msgr2=0x7f38ec19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.904+0000 7f38e3e00640 1 --2- 172.21.15.2:0/1020615197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38ec10e3c0 0x7f38ec19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.904+0000 7f38e3e00640 1 -- 172.21.15.2:0/1020615197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec19f6d0 con 0x7f38ec074f40 2024-09-10T08:08:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.904+0000 7f38e3e00640 1 --2- 172.21.15.2:0/1020615197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec074f40 0x7f38ec19ac60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005fd0 tx=0x7f38d400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.905+0000 7f38e8c00640 1 -- 172.21.15.2:0/1020615197 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4009c30 con 0x7f38ec074f40 2024-09-10T08:08:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.905+0000 7f38e8c00640 1 -- 172.21.15.2:0/1020615197 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d4009dd0 con 0x7f38ec074f40 2024-09-10T08:08:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.905+0000 7f38e8c00640 1 -- 172.21.15.2:0/1020615197 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4010460 con 0x7f38ec074f40 2024-09-10T08:08:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.905+0000 7f38f2000640 1 -- 172.21.15.2:0/1020615197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec19f9c0 con 0x7f38ec074f40 2024-09-10T08:08:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.905+0000 7f38f2000640 1 -- 172.21.15.2:0/1020615197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec1a72a0 con 0x7f38ec074f40 2024-09-10T08:08:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.906+0000 7f38f2000640 1 -- 172.21.15.2:0/1020615197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ec074a00 con 0x7f38ec074f40 2024-09-10T08:08:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.908+0000 7f38e8c00640 1 -- 172.21.15.2:0/1020615197 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f38d4003ef0 con 0x7f38ec074f40 2024-09-10T08:08:26.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.909+0000 7f38e8c00640 1 --2- 172.21.15.2:0/1020615197 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38d00a2c90 0x7f38d00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:26.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.909+0000 7f38e8c00640 1 -- 172.21.15.2:0/1020615197 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f38d40cf0b0 con 0x7f38ec074f40 2024-09-10T08:08:26.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.909+0000 7f38eb400640 1 --2- 172.21.15.2:0/1020615197 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38d00a2c90 0x7f38d00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:26.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.910+0000 7f38eb400640 1 --2- 172.21.15.2:0/1020615197 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38d00a2c90 0x7f38d00a5150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f38dc000c00 tx=0x7f38dc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:26.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:26.913+0000 7f38e8c00640 1 -- 172.21.15.2:0/1020615197 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38d4017020 con 0x7f38ec074f40 2024-09-10T08:08:27.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.134+0000 7f38f2000640 1 -- 172.21.15.2:0/1020615197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38ec109540 con 0x7f38ec074f40 2024-09-10T08:08:27.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:26 smithi002 ceph-mon[30563]: pgmap v396: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:08:27.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:26 smithi002 ceph-mon[30563]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:27.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.135+0000 7f38e8c00640 1 -- 172.21.15.2:0/1020615197 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f38d409b410 con 0x7f38ec074f40 2024-09-10T08:08:27.135 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:27.136 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:08:27.136 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:08:27.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.137+0000 7f38f2000640 1 -- 172.21.15.2:0/1020615197 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38d00a2c90 msgr2=0x7f38d00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:27.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.137+0000 7f38f2000640 1 --2- 172.21.15.2:0/1020615197 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38d00a2c90 0x7f38d00a5150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f38dc000c00 tx=0x7f38dc005b50 comp rx=0 tx=0).stop 2024-09-10T08:08:27.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.137+0000 7f38f2000640 1 -- 172.21.15.2:0/1020615197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec074f40 msgr2=0x7f38ec19ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:27.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.137+0000 7f38f2000640 1 --2- 172.21.15.2:0/1020615197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec074f40 0x7f38ec19ac60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005fd0 tx=0x7f38d400dde0 comp rx=0 tx=0).stop 2024-09-10T08:08:27.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.137+0000 7f38f2000640 1 -- 172.21.15.2:0/1020615197 shutdown_connections 2024-09-10T08:08:27.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.137+0000 7f38f2000640 1 --2- 172.21.15.2:0/1020615197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38ec10e3c0 0x7f38ec19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:27.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.137+0000 7f38f2000640 1 --2- 172.21.15.2:0/1020615197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38ec074f40 0x7f38ec19ac60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:27.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.137+0000 7f38f2000640 1 --2- 172.21.15.2:0/1020615197 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38d00a2c90 0x7f38d00a5150 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:27.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.137+0000 7f38f2000640 1 --2- 172.21.15.2:0/1020615197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38ec074620 0x7f38ec19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:27.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.137+0000 7f38f2000640 1 -- 172.21.15.2:0/1020615197 >> 172.21.15.2:0/1020615197 conn(0x7f38ec0fd8c0 msgr2=0x7f38ec0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:27.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.138+0000 7f38f2000640 1 -- 172.21.15.2:0/1020615197 shutdown_connections 2024-09-10T08:08:27.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:27.138+0000 7f38f2000640 1 -- 172.21.15.2:0/1020615197 wait complete. 2024-09-10T08:08:27.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:26 smithi155 ceph-mon[35405]: pgmap v396: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:08:27.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:26 smithi155 ceph-mon[35405]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:27.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:26 smithi107 ceph-mon[35048]: pgmap v396: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:08:27.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:26 smithi107 ceph-mon[35048]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:27.837 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:08:28.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:27 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1020615197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:28.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:27 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1020615197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:28.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:27 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1020615197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:28.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.820+0000 7f38b0000640 1 -- 172.21.15.2:0/1027448673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a81049a0 msgr2=0x7f38a81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:28.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.820+0000 7f38b0000640 1 --2- 172.21.15.2:0/1027448673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a81049a0 0x7f38a81089f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3894009900 tx=0x7f389402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:08:28.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.820+0000 7f38b0000640 1 -- 172.21.15.2:0/1027448673 shutdown_connections 2024-09-10T08:08:28.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.820+0000 7f38b0000640 1 --2- 172.21.15.2:0/1027448673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a8109120 0x7f38a8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:28.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.820+0000 7f38b0000640 1 --2- 172.21.15.2:0/1027448673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a81049a0 0x7f38a81089f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:28.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.820+0000 7f38b0000640 1 --2- 172.21.15.2:0/1027448673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a8103ff0 0x7f38a81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:28.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.820+0000 7f38b0000640 1 -- 172.21.15.2:0/1027448673 >> 172.21.15.2:0/1027448673 conn(0x7f38a80ffa90 msgr2=0x7f38a8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:28.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.820+0000 7f38b0000640 1 -- 172.21.15.2:0/1027448673 shutdown_connections 2024-09-10T08:08:28.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.820+0000 7f38b0000640 1 -- 172.21.15.2:0/1027448673 wait complete. 2024-09-10T08:08:28.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.821+0000 7f38b0000640 1 Processor -- start 2024-09-10T08:08:28.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.821+0000 7f38b0000640 1 -- start start 2024-09-10T08:08:28.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.821+0000 7f38b0000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a8103ff0 0x7f38a819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:28.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.821+0000 7f38b0000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a81049a0 0x7f38a819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:28.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.821+0000 7f38b0000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a8109120 0x7f38a819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:28.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.821+0000 7f38b0000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8114190 con 0x7f38a8109120 2024-09-10T08:08:28.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.821+0000 7f38b0000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8114010 con 0x7f38a81049a0 2024-09-10T08:08:28.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.821+0000 7f38b0000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8114310 con 0x7f38a8103ff0 2024-09-10T08:08:28.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.822+0000 7f38ae000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a8109120 0x7f38a819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:28.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.822+0000 7f38ad600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a8103ff0 0x7f38a819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:28.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.822+0000 7f38ad600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a8103ff0 0x7f38a819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60124/0 (socket says 172.21.15.2:60124) 2024-09-10T08:08:28.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.822+0000 7f38ad600640 1 -- 172.21.15.2:0/3866708467 learned_addr learned my addr 172.21.15.2:0/3866708467 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:28.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.822+0000 7f38acc00640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a81049a0 0x7f38a819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:28.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.822+0000 7f38ad600640 1 -- 172.21.15.2:0/3866708467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a81049a0 msgr2=0x7f38a819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:28.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.822+0000 7f38ad600640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a81049a0 0x7f38a819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:28.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.822+0000 7f38ad600640 1 -- 172.21.15.2:0/3866708467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a8109120 msgr2=0x7f38a819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:28.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.822+0000 7f38ad600640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a8109120 0x7f38a819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:28.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.822+0000 7f38ad600640 1 -- 172.21.15.2:0/3866708467 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a819f5c0 con 0x7f38a8103ff0 2024-09-10T08:08:28.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.822+0000 7f38ae000640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a8109120 0x7f38a819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:28.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.822+0000 7f38ad600640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a8103ff0 0x7f38a819a610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f389c00eb90 tx=0x7f389c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:28.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.823+0000 7f38a6000640 1 -- 172.21.15.2:0/3866708467 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c004d20 con 0x7f38a8103ff0 2024-09-10T08:08:28.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.823+0000 7f38b0000640 1 -- 172.21.15.2:0/3866708467 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a819f8b0 con 0x7f38a8103ff0 2024-09-10T08:08:28.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.823+0000 7f38a6000640 1 -- 172.21.15.2:0/3866708467 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c00ee90 con 0x7f38a8103ff0 2024-09-10T08:08:28.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.823+0000 7f38a6000640 1 -- 172.21.15.2:0/3866708467 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c010830 con 0x7f38a8103ff0 2024-09-10T08:08:28.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.823+0000 7f38b0000640 1 -- 172.21.15.2:0/3866708467 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a81a7190 con 0x7f38a8103ff0 2024-09-10T08:08:28.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.824+0000 7f38b0000640 1 -- 172.21.15.2:0/3866708467 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a81043d0 con 0x7f38a8103ff0 2024-09-10T08:08:28.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.826+0000 7f38a6000640 1 -- 172.21.15.2:0/3866708467 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f389c044b60 con 0x7f38a8103ff0 2024-09-10T08:08:28.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.827+0000 7f38a6000640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38900a2c10 0x7f38900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:28.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.827+0000 7f38a6000640 1 -- 172.21.15.2:0/3866708467 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f389c0cfc90 con 0x7f38a8103ff0 2024-09-10T08:08:28.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.828+0000 7f38acc00640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38900a2c10 0x7f38900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:28.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.828+0000 7f38acc00640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38900a2c10 0x7f38900a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3894000c00 tx=0x7f3894005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:28.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:28.830+0000 7f38a6000640 1 -- 172.21.15.2:0/3866708467 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f389c09c640 con 0x7f38a8103ff0 2024-09-10T08:08:29.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.015+0000 7f38b0000640 1 -- 172.21.15.2:0/3866708467 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38a819b970 con 0x7f38a8103ff0 2024-09-10T08:08:29.015 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:28 smithi002 ceph-mon[30563]: pgmap v397: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:08:29.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.017+0000 7f38a6000640 1 -- 172.21.15.2:0/3866708467 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24887 (secure 0 0 0) 0x7f389c09bf40 con 0x7f38a8103ff0 2024-09-10T08:08:29.017 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:29.017 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":95,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:07:05.504273+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:08:29.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.020+0000 7f38b0000640 1 -- 172.21.15.2:0/3866708467 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38900a2c10 msgr2=0x7f38900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:29.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.020+0000 7f38b0000640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38900a2c10 0x7f38900a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3894000c00 tx=0x7f3894005b50 comp rx=0 tx=0).stop 2024-09-10T08:08:29.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.020+0000 7f38b0000640 1 -- 172.21.15.2:0/3866708467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a8103ff0 msgr2=0x7f38a819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:29.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.020+0000 7f38b0000640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a8103ff0 0x7f38a819a610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f389c00eb90 tx=0x7f389c00c6a0 comp rx=0 tx=0).stop 2024-09-10T08:08:29.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.020+0000 7f38b0000640 1 -- 172.21.15.2:0/3866708467 shutdown_connections 2024-09-10T08:08:29.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.020+0000 7f38b0000640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38a8109120 0x7f38a819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:29.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.020+0000 7f38b0000640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38a81049a0 0x7f38a819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:29.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.020+0000 7f38b0000640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38900a2c10 0x7f38900a50d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:29.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.020+0000 7f38b0000640 1 --2- 172.21.15.2:0/3866708467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a8103ff0 0x7f38a819a610 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:29.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.020+0000 7f38b0000640 1 -- 172.21.15.2:0/3866708467 >> 172.21.15.2:0/3866708467 conn(0x7f38a80ffa90 msgr2=0x7f38a8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:29.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.021+0000 7f38b0000640 1 -- 172.21.15.2:0/3866708467 shutdown_connections 2024-09-10T08:08:29.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:29.021+0000 7f38b0000640 1 -- 172.21.15.2:0/3866708467 wait complete. 2024-09-10T08:08:29.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:28 smithi155 ceph-mon[35405]: pgmap v397: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:08:29.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:28 smithi107 ceph-mon[35048]: pgmap v397: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:08:29.821 DEBUG:tasks.fwd_scrub:Yielding 2024-09-10T08:08:29.821 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-10T08:08:29.831 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-09-10T08:08:29.832 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-10T08:08:29.832 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-10T08:08:29.832 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:08:29.832 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:08:29.837 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-10T08:08:29.869 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T08:08:29.869 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:08:30.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:29 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3866708467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:08:30.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:29 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3866708467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:08:30.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:29 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3866708467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:08:30.230 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:30.264 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.998+0000 7f4ad7e00640 1 -- 172.21.15.2:0/1823337714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad80745c0 msgr2=0x7f4ad80749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.998+0000 7f4ad7e00640 1 --2- 172.21.15.2:0/1823337714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad80745c0 0x7f4ad80749a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009860 tx=0x7f4ac802a3d0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.998+0000 7f4ad7e00640 1 -- 172.21.15.2:0/1823337714 shutdown_connections 2024-09-10T08:08:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.998+0000 7f4ad7e00640 1 --2- 172.21.15.2:0/1823337714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad810e260 0x7f4ad8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.998+0000 7f4ad7e00640 1 --2- 172.21.15.2:0/1823337714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ad8074ee0 0x7f4ad810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.998+0000 7f4ad7e00640 1 --2- 172.21.15.2:0/1823337714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad80745c0 0x7f4ad80749a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.998+0000 7f4ad7e00640 1 -- 172.21.15.2:0/1823337714 >> 172.21.15.2:0/1823337714 conn(0x7f4ad80fd740 msgr2=0x7f4ad80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.998+0000 7f4ad7e00640 1 -- 172.21.15.2:0/1823337714 shutdown_connections 2024-09-10T08:08:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.998+0000 7f4ad7e00640 1 -- 172.21.15.2:0/1823337714 wait complete. 2024-09-10T08:08:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.999+0000 7f4ad7e00640 1 Processor -- start 2024-09-10T08:08:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.999+0000 7f4ad7e00640 1 -- start start 2024-09-10T08:08:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.999+0000 7f4ad7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad80745c0 0x7f4ad819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.999+0000 7f4ad7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ad8074ee0 0x7f4ad819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.999+0000 7f4ad7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad810e260 0x7f4ad819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.999+0000 7f4ad7e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8113fb0 con 0x7f4ad80745c0 2024-09-10T08:08:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.999+0000 7f4ad7e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8113e30 con 0x7f4ad8074ee0 2024-09-10T08:08:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:30.999+0000 7f4ad7e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8114130 con 0x7f4ad810e260 2024-09-10T08:08:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.000+0000 7f4ad6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad80745c0 0x7f4ad819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.000+0000 7f4ace000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ad8074ee0 0x7f4ad819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.000+0000 7f4ad7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad810e260 0x7f4ad819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.000+0000 7f4ace000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ad8074ee0 0x7f4ad819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:59386/0 (socket says 172.21.15.2:59386) 2024-09-10T08:08:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.000+0000 7f4ad6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad80745c0 0x7f4ad819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39780/0 (socket says 172.21.15.2:39780) 2024-09-10T08:08:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.000+0000 7f4ace000640 1 -- 172.21.15.2:0/2890759316 learned_addr learned my addr 172.21.15.2:0/2890759316 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.000+0000 7f4ace000640 1 -- 172.21.15.2:0/2890759316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad810e260 msgr2=0x7f4ad819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.000+0000 7f4ace000640 1 --2- 172.21.15.2:0/2890759316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad810e260 0x7f4ad819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.000+0000 7f4ace000640 1 -- 172.21.15.2:0/2890759316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad80745c0 msgr2=0x7f4ad819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.000+0000 7f4ace000640 1 --2- 172.21.15.2:0/2890759316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad80745c0 0x7f4ad819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.000+0000 7f4ace000640 1 -- 172.21.15.2:0/2890759316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad819f4a0 con 0x7f4ad8074ee0 2024-09-10T08:08:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.000+0000 7f4ace000640 1 --2- 172.21.15.2:0/2890759316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ad8074ee0 0x7f4ad819aa30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000e9c0 tx=0x7f4ac000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.001+0000 7f4adca00640 1 -- 172.21.15.2:0/2890759316 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac000cd80 con 0x7f4ad8074ee0 2024-09-10T08:08:31.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.001+0000 7f4ad7e00640 1 -- 172.21.15.2:0/2890759316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad819f790 con 0x7f4ad8074ee0 2024-09-10T08:08:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.001+0000 7f4adca00640 1 -- 172.21.15.2:0/2890759316 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac00035f0 con 0x7f4ad8074ee0 2024-09-10T08:08:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.001+0000 7f4adca00640 1 -- 172.21.15.2:0/2890759316 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0010670 con 0x7f4ad8074ee0 2024-09-10T08:08:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.001+0000 7f4ad7e00640 1 -- 172.21.15.2:0/2890759316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad81a7070 con 0x7f4ad8074ee0 2024-09-10T08:08:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.004+0000 7f4adca00640 1 -- 172.21.15.2:0/2890759316 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f4ac00037a0 con 0x7f4ad8074ee0 2024-09-10T08:08:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.004+0000 7f4ae08d5740 1 Processor -- start 2024-09-10T08:08:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.004+0000 7f4ae08d5740 1 -- start start 2024-09-10T08:08:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.004+0000 7f4ae08d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd39cb0 0x55f22bd3a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.004+0000 7f4ae08d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd3a5d0 0x55f22bd44950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.004+0000 7f4ae08d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd44e90 0x55f22bd47280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.004+0000 7f4ae08d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f22bca0290 con 0x55f22bd39cb0 2024-09-10T08:08:31.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.004+0000 7f4ae08d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f22bc47b40 con 0x55f22bd44e90 2024-09-10T08:08:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.004+0000 7f4ae08d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f22bc9c250 con 0x55f22bd3a5d0 2024-09-10T08:08:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.005+0000 7f4ad6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd3a5d0 0x55f22bd44950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.005+0000 7f4ace000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd44e90 0x55f22bd47280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.005+0000 7f4ad6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd3a5d0 0x55f22bd44950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60162/0 (socket says 172.21.15.2:60162) 2024-09-10T08:08:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.005+0000 7f4ad6a00640 1 -- 172.21.15.2:0/3144095403 learned_addr learned my addr 172.21.15.2:0/3144095403 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.005+0000 7f4ad7400640 1 --2- 172.21.15.2:0/3144095403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd39cb0 0x55f22bd3a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.005+0000 7f4ad6a00640 1 -- 172.21.15.2:0/3144095403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd44e90 msgr2=0x55f22bd47280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.005+0000 7f4ad6a00640 1 --2- 172.21.15.2:0/3144095403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd44e90 0x55f22bd47280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.005+0000 7f4ad6a00640 1 -- 172.21.15.2:0/3144095403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd39cb0 msgr2=0x55f22bd3a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.005+0000 7f4ad6a00640 1 --2- 172.21.15.2:0/3144095403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd39cb0 0x55f22bd3a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.005+0000 7f4ad6a00640 1 -- 172.21.15.2:0/3144095403 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f22ba116a0 con 0x55f22bd3a5d0 2024-09-10T08:08:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.005+0000 7f4adca00640 1 --2- 172.21.15.2:0/2890759316 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4abc0a2c10 0x7f4abc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.006+0000 7f4ad6a00640 1 --2- 172.21.15.2:0/3144095403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd3a5d0 0x55f22bd44950 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8034000 tx=0x7f4ac8032f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=910603cb7fb09ba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.006+0000 7f4adca00640 1 -- 172.21.15.2:0/2890759316 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4ac00cf840 con 0x7f4ad8074ee0 2024-09-10T08:08:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.006+0000 7f4ad7400640 1 --2- 172.21.15.2:0/2890759316 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4abc0a2c10 0x7f4abc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.006+0000 7f4ac7e00640 1 -- 172.21.15.2:0/3144095403 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8030cb0 con 0x55f22bd3a5d0 2024-09-10T08:08:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.006+0000 7f4ac7e00640 1 -- 172.21.15.2:0/3144095403 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac8030e50 con 0x55f22bd3a5d0 2024-09-10T08:08:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.006+0000 7f4ac7e00640 1 -- 172.21.15.2:0/3144095403 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8004920 con 0x55f22bd3a5d0 2024-09-10T08:08:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.007+0000 7f4ae08d5740 1 -- 172.21.15.2:0/3144095403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd3a5d0 msgr2=0x55f22bd44950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.007+0000 7f4ad7400640 1 --2- 172.21.15.2:0/2890759316 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4abc0a2c10 0x7f4abc0a50d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4ad806a6f0 tx=0x7f4ad0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.007+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/3144095403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd3a5d0 0x55f22bd44950 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8034000 tx=0x7f4ac8032f60 comp rx=0 tx=0).stop 2024-09-10T08:08:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.007+0000 7f4ae08d5740 1 -- 172.21.15.2:0/3144095403 shutdown_connections 2024-09-10T08:08:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.007+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/3144095403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd44e90 0x55f22bd47280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.007+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/3144095403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd3a5d0 0x55f22bd44950 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.007+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/3144095403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd39cb0 0x55f22bd3a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.007+0000 7f4ae08d5740 1 -- 172.21.15.2:0/3144095403 >> 172.21.15.2:0/3144095403 conn(0x55f22bd36240 msgr2=0x55f22bd36620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.007+0000 7f4ae08d5740 1 -- 172.21.15.2:0/3144095403 shutdown_connections 2024-09-10T08:08:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.007+0000 7f4ae08d5740 1 -- 172.21.15.2:0/3144095403 wait complete. 2024-09-10T08:08:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.008+0000 7f4ae08d5740 1 -- start start 2024-09-10T08:08:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.008+0000 7f4ae08d5740 1 Processor -- start 2024-09-10T08:08:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.008+0000 7f4ae08d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd39cb0 0x55f22bd44730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.008+0000 7f4ae08d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd3a5d0 0x55f22bd419e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.008+0000 7f4ae08d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd44e90 0x55f22bd41f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.008+0000 7f4ae08d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f22bd47b00 con 0x55f22bd3a5d0 2024-09-10T08:08:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.008+0000 7f4ae08d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f22bc5afa0 con 0x55f22bd39cb0 2024-09-10T08:08:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.008+0000 7f4ae08d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f22bd47c80 con 0x55f22bd44e90 2024-09-10T08:08:31.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.009+0000 7f4ae08d5740 10 client.0 _ll_register_callbacks cb 0x55f22b9aeba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.009+0000 7f4ad7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd44e90 0x55f22bd41f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.009+0000 7f4ad6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd39cb0 0x55f22bd44730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.009+0000 7f4ad6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd39cb0 0x55f22bd44730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:59418/0 (socket says 172.21.15.2:59418) 2024-09-10T08:08:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.009+0000 7f4ad6a00640 1 -- 172.21.15.2:0/1469049112 learned_addr learned my addr 172.21.15.2:0/1469049112 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.009+0000 7f4ad7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd44e90 0x55f22bd41f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60164/0 (socket says 172.21.15.2:60164) 2024-09-10T08:08:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.009+0000 7f4ad6a00640 1 -- 172.21.15.2:0/1469049112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd44e90 msgr2=0x55f22bd41f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.009+0000 7f4ad6a00640 1 --2- 172.21.15.2:0/1469049112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd44e90 0x55f22bd41f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.009+0000 7f4ad6a00640 1 -- 172.21.15.2:0/1469049112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd3a5d0 msgr2=0x55f22bd419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.009+0000 7f4ace000640 1 --2- 172.21.15.2:0/1469049112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd3a5d0 0x55f22bd419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.009+0000 7f4ad6a00640 1 --2- 172.21.15.2:0/1469049112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd3a5d0 0x55f22bd419e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.009+0000 7f4ad6a00640 1 -- 172.21.15.2:0/1469049112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f22bd42720 con 0x55f22bd39cb0 2024-09-10T08:08:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.009+0000 7f4ace000640 1 --2- 172.21.15.2:0/1469049112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd3a5d0 0x55f22bd419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:08:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.010+0000 7f4ad6a00640 1 --2- 172.21.15.2:0/1469049112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd39cb0 0x55f22bd44730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8032ff0 tx=0x7f4ac802a3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeb898ff77aabf5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.010+0000 7f4ac4c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:08:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.010+0000 7f4ac4c00640 1 -- 172.21.15.2:0/1469049112 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8055060 con 0x55f22bd39cb0 2024-09-10T08:08:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.010+0000 7f4ac4c00640 1 -- 172.21.15.2:0/1469049112 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4ac80305b0 con 0x55f22bd39cb0 2024-09-10T08:08:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.010+0000 7f4ae08d5740 1 -- 172.21.15.2:0/1469049112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac8030e50 con 0x55f22bd39cb0 2024-09-10T08:08:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.010+0000 7f4ac4c00640 1 -- 172.21.15.2:0/1469049112 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ac8009450 con 0x55f22bd39cb0 2024-09-10T08:08:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.010+0000 7f4ae08d5740 10 client.25238 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.010+0000 7f4ae08d5740 1 -- 172.21.15.2:0/1469049112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f22bd47b00 con 0x55f22bd39cb0 2024-09-10T08:08:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.010+0000 7f4ac4c00640 1 -- 172.21.15.2:0/1469049112 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac8030e50 con 0x55f22bd39cb0 2024-09-10T08:08:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.011+0000 7f4ac4c00640 1 -- 172.21.15.2:0/1469049112 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4ac802f570 con 0x55f22bd39cb0 2024-09-10T08:08:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.011+0000 7f4ae08d5740 10 client.25238 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.011+0000 7f4ae08d5740 10 client.25238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:08:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.011+0000 7f4ae08d5740 20 client.25238 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.011+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/1469049112 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f22bd54b00 0x55f22bd56ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.011+0000 7f4ae08d5740 4 client.25238 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:08:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.011+0000 7f4ae08d5740 1 -- 172.21.15.2:0/1469049112 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f22bcad800 con 0x55f22bd54b00 2024-09-10T08:08:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.011+0000 7f4ace000640 1 --2- 172.21.15.2:0/1469049112 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f22bd54b00 0x55f22bd56ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.011+0000 7f4ace000640 1 --2- 172.21.15.2:0/1469049112 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f22bd54b00 0x55f22bd56ef0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1088628692240d server_cookie=ff8aa2814f3f1631 in_seq=0 out_seq=0 2024-09-10T08:08:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.011+0000 7f4ac4c00640 10 client.25238 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.012+0000 7f4ac4c00640 1 -- 172.21.15.2:0/1469049112 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f22bcad800 con 0x55f22bd54b00 2024-09-10T08:08:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.012+0000 7f4ac4c00640 10 client.25238 handle_command_reply: tid=0 2024-09-10T08:08:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.012+0000 7f4ae08d5740 1 client.25238 shutdown 2024-09-10T08:08:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.012+0000 7f4ae08d5740 1 -- 172.21.15.2:0/1469049112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd39cb0 msgr2=0x55f22bd44730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.012+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/1469049112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd39cb0 0x55f22bd44730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8032ff0 tx=0x7f4ac802a3d0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.012+0000 7f4ae08d5740 1 -- 172.21.15.2:0/1469049112 shutdown_connections 2024-09-10T08:08:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.012+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/1469049112 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f22bd54b00 0x55f22bd56ef0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.012+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/1469049112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd44e90 0x55f22bd41f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.013+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/1469049112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd3a5d0 0x55f22bd419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.013+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/1469049112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd39cb0 0x55f22bd44730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.013+0000 7f4ae08d5740 1 -- 172.21.15.2:0/1469049112 >> 172.21.15.2:0/1469049112 conn(0x55f22bd36240 msgr2=0x55f22bd3bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.013+0000 7f4ac4c00640 0 client.25238 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.013+0000 7f4ae08d5740 1 -- 172.21.15.2:0/1469049112 shutdown_connections 2024-09-10T08:08:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.013+0000 7f4ae08d5740 1 -- 172.21.15.2:0/1469049112 wait complete. 2024-09-10T08:08:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.013+0000 7f4ae08d5740 20 client.25238 trim_cache size 0 max 16384 2024-09-10T08:08:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.034+0000 7f4ae08d5740 1 Processor -- start 2024-09-10T08:08:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.034+0000 7f4ae08d5740 1 -- start start 2024-09-10T08:08:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.034+0000 7f4ae08d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd39cb0 0x55f22bd41e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.034+0000 7f4ae08d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd3a5d0 0x55f22bd34f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.034+0000 7f4ae08d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd44e90 0x55f22bd354d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.034+0000 7f4ae08d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad81b3810 con 0x55f22bd44e90 2024-09-10T08:08:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.034+0000 7f4ae08d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad81b10e0 con 0x55f22bd3a5d0 2024-09-10T08:08:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.034+0000 7f4ae08d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad81aef30 con 0x55f22bd39cb0 2024-09-10T08:08:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.034+0000 7f4ace000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd3a5d0 0x55f22bd34f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.035+0000 7f4ad6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd39cb0 0x55f22bd41e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.035+0000 7f4ace000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd3a5d0 0x55f22bd34f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:59428/0 (socket says 172.21.15.2:59428) 2024-09-10T08:08:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.035+0000 7f4ace000640 1 -- 172.21.15.2:0/1654031143 learned_addr learned my addr 172.21.15.2:0/1654031143 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.035+0000 7f4ad6a00640 1 -- 172.21.15.2:0/1654031143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd3a5d0 msgr2=0x55f22bd34f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.035+0000 7f4ad6a00640 1 --2- 172.21.15.2:0/1654031143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd3a5d0 0x55f22bd34f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.035+0000 7f4ad6a00640 1 -- 172.21.15.2:0/1654031143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd44e90 msgr2=0x55f22bd354d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.035+0000 7f4ad7400640 1 --2- 172.21.15.2:0/1654031143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd44e90 0x55f22bd354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.035+0000 7f4ad6a00640 1 --2- 172.21.15.2:0/1654031143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd44e90 0x55f22bd354d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.035+0000 7f4ad6a00640 1 -- 172.21.15.2:0/1654031143 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f22bd3ff50 con 0x55f22bd39cb0 2024-09-10T08:08:31.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.035+0000 7f4ad7400640 1 --2- 172.21.15.2:0/1654031143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd44e90 0x55f22bd354d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:08:31.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.035+0000 7f4ad6a00640 1 --2- 172.21.15.2:0/1654031143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd39cb0 0x55f22bd41e00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ac802fb00 tx=0x7f4ac802ff50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e8f80fca47edc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:31.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.037+0000 7f4ac7400640 1 -- 172.21.15.2:0/1654031143 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8055040 con 0x55f22bd39cb0 2024-09-10T08:08:31.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.037+0000 7f4ac7400640 1 -- 172.21.15.2:0/1654031143 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac8005250 con 0x55f22bd39cb0 2024-09-10T08:08:31.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.037+0000 7f4ac7400640 1 -- 172.21.15.2:0/1654031143 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80307b0 con 0x55f22bd39cb0 2024-09-10T08:08:31.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.038+0000 7f4ae08d5740 1 -- 172.21.15.2:0/1654031143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd39cb0 msgr2=0x55f22bd41e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.038+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/1654031143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd39cb0 0x55f22bd41e00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ac802fb00 tx=0x7f4ac802ff50 comp rx=0 tx=0).stop 2024-09-10T08:08:31.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.038+0000 7f4ae08d5740 1 -- 172.21.15.2:0/1654031143 shutdown_connections 2024-09-10T08:08:31.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.038+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/1654031143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd44e90 0x55f22bd354d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.038+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/1654031143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd3a5d0 0x55f22bd34f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.038+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/1654031143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd39cb0 0x55f22bd41e00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.038+0000 7f4ae08d5740 1 -- 172.21.15.2:0/1654031143 >> 172.21.15.2:0/1654031143 conn(0x55f22bd36240 msgr2=0x55f22bd34880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.038+0000 7f4ae08d5740 1 -- 172.21.15.2:0/1654031143 shutdown_connections 2024-09-10T08:08:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.038+0000 7f4ae08d5740 1 -- 172.21.15.2:0/1654031143 wait complete. 2024-09-10T08:08:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.039+0000 7f4ae08d5740 1 -- start start 2024-09-10T08:08:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.039+0000 7f4ae08d5740 1 Processor -- start 2024-09-10T08:08:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.040+0000 7f4ae08d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd39cb0 0x55f22bd4d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.040+0000 7f4ae08d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd3a5d0 0x55f22bd4ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.040+0000 7f4ae08d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd44e90 0x55f22bd4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.040+0000 7f4ae08d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad81af5a0 con 0x55f22bd39cb0 2024-09-10T08:08:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.040+0000 7f4ae08d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad81b0a10 con 0x55f22bd3a5d0 2024-09-10T08:08:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.040+0000 7f4ae08d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad81afc70 con 0x55f22bd44e90 2024-09-10T08:08:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.040+0000 7f4ae08d5740 10 client.0 _ll_register_callbacks cb 0x7f4ad0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:31.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.040+0000 7f4ace000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd3a5d0 0x55f22bd4ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.040+0000 7f4ace000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd3a5d0 0x55f22bd4ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:59438/0 (socket says 172.21.15.2:59438) 2024-09-10T08:08:31.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.040+0000 7f4ad6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd39cb0 0x55f22bd4d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.041+0000 7f4ace000640 1 -- 172.21.15.2:0/844077572 learned_addr learned my addr 172.21.15.2:0/844077572 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:31.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.041+0000 7f4ad7400640 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd44e90 0x55f22bd4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.041+0000 7f4ace000640 1 -- 172.21.15.2:0/844077572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd44e90 msgr2=0x55f22bd4b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:31.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.041+0000 7f4ace000640 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd44e90 0x55f22bd4b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.041+0000 7f4ace000640 1 -- 172.21.15.2:0/844077572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd39cb0 msgr2=0x55f22bd4d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:31.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.041+0000 7f4ace000640 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd39cb0 0x55f22bd4d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.041+0000 7f4ace000640 1 -- 172.21.15.2:0/844077572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f22bdbe5f0 con 0x55f22bd3a5d0 2024-09-10T08:08:31.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.041+0000 7f4ad6a00640 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd39cb0 0x55f22bd4d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:08:31.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.041+0000 7f4ad7400640 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd44e90 0x55f22bd4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:08:31.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.042+0000 7f4ace000640 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd3a5d0 0x55f22bd4ab30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4ac009b7f0 tx=0x7f4ac00d1c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78e4b31c79c51ffd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:31.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.042+0000 7f4ac4c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:08:31.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.042+0000 7f4ac4c00640 1 -- 172.21.15.2:0/844077572 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac009c1f0 con 0x55f22bd3a5d0 2024-09-10T08:08:31.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.042+0000 7f4ac4c00640 1 -- 172.21.15.2:0/844077572 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4ac0003b10 con 0x55f22bd3a5d0 2024-09-10T08:08:31.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.042+0000 7f4ae08d5740 1 -- 172.21.15.2:0/844077572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac8005250 con 0x55f22bd3a5d0 2024-09-10T08:08:31.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.042+0000 7f4ac4c00640 1 -- 172.21.15.2:0/844077572 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ac00b9030 con 0x55f22bd3a5d0 2024-09-10T08:08:31.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.043+0000 7f4ae08d5740 10 client.25244 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:31.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.043+0000 7f4ae08d5740 1 -- 172.21.15.2:0/844077572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f22bdbf780 con 0x55f22bd3a5d0 2024-09-10T08:08:31.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.043+0000 7f4ac4c00640 1 -- 172.21.15.2:0/844077572 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac8005250 con 0x55f22bd3a5d0 2024-09-10T08:08:31.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.043+0000 7f4ac4c00640 1 -- 172.21.15.2:0/844077572 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4ac00ccff0 con 0x55f22bd3a5d0 2024-09-10T08:08:31.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.043+0000 7f4ae08d5740 10 client.25244 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.043+0000 7f4ae08d5740 10 client.25244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:08:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.043+0000 7f4ae08d5740 20 client.25244 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.043+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f22bdbfe00 0x55f22bdc2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.043+0000 7f4ae08d5740 4 client.25244 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:08:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.043+0000 7f4ae08d5740 1 -- 172.21.15.2:0/844077572 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f22bca7d10 con 0x55f22bdbfe00 2024-09-10T08:08:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.044+0000 7f4ad6a00640 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f22bdbfe00 0x55f22bdc2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.044+0000 7f4ac4c00640 10 client.25244 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.044+0000 7f4ad6a00640 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f22bdbfe00 0x55f22bdc2220 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5088b5993d17c5 server_cookie=d14e2396fccff5d5 in_seq=0 out_seq=0 2024-09-10T08:08:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.045+0000 7f4ac4c00640 1 -- 172.21.15.2:0/844077572 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f22bca7d10 con 0x55f22bdbfe00 2024-09-10T08:08:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.045+0000 7f4ac4c00640 10 client.25244 handle_command_reply: tid=0 2024-09-10T08:08:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.045+0000 7f4ae08d5740 1 client.25244 shutdown 2024-09-10T08:08:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.046+0000 7f4ae08d5740 1 -- 172.21.15.2:0/844077572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd3a5d0 msgr2=0x55f22bd4ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.046+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd3a5d0 0x55f22bd4ab30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4ac009b7f0 tx=0x7f4ac00d1c90 comp rx=0 tx=0).stop 2024-09-10T08:08:31.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.046+0000 7f4ae08d5740 1 -- 172.21.15.2:0/844077572 shutdown_connections 2024-09-10T08:08:31.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.046+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f22bdbfe00 0x55f22bdc2220 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.046+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f22bd44e90 0x55f22bd4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.046+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f22bd3a5d0 0x55f22bd4ab30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.046+0000 7f4ae08d5740 1 --2- 172.21.15.2:0/844077572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f22bd39cb0 0x55f22bd4d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.046+0000 7f4ae08d5740 1 -- 172.21.15.2:0/844077572 >> 172.21.15.2:0/844077572 conn(0x55f22bd36240 msgr2=0x55f22bd350a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:31.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.046+0000 7f4ac4c00640 0 client.25244 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:31.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.046+0000 7f4ae08d5740 1 -- 172.21.15.2:0/844077572 shutdown_connections 2024-09-10T08:08:31.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.046+0000 7f4ae08d5740 1 -- 172.21.15.2:0/844077572 wait complete. 2024-09-10T08:08:31.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.046+0000 7f4ae08d5740 20 client.25244 trim_cache size 0 max 16384 2024-09-10T08:08:31.051 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:08:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:08:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "401195cc-8ebd-46b5-8355-5a423a6ee37e", 2024-09-10T08:08:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:08:31.051 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:08:31.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.047+0000 7f4ad7e00640 1 -- 172.21.15.2:0/2890759316 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4abc0a2c10 msgr2=0x7f4abc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.047+0000 7f4ad7e00640 1 --2- 172.21.15.2:0/2890759316 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4abc0a2c10 0x7f4abc0a50d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4ad806a6f0 tx=0x7f4ad0009500 comp rx=0 tx=0).stop 2024-09-10T08:08:31.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.047+0000 7f4ad7e00640 1 -- 172.21.15.2:0/2890759316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ad8074ee0 msgr2=0x7f4ad819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.047+0000 7f4ad7e00640 1 --2- 172.21.15.2:0/2890759316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ad8074ee0 0x7f4ad819aa30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000e9c0 tx=0x7f4ac000edc0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.048+0000 7f4ad7e00640 1 -- 172.21.15.2:0/2890759316 shutdown_connections 2024-09-10T08:08:31.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.048+0000 7f4ad7e00640 1 --2- 172.21.15.2:0/2890759316 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4abc0a2c10 0x7f4abc0a50d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.048+0000 7f4ad7e00640 1 --2- 172.21.15.2:0/2890759316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad810e260 0x7f4ad819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.048+0000 7f4ad7e00640 1 --2- 172.21.15.2:0/2890759316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ad8074ee0 0x7f4ad819aa30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.048+0000 7f4ad7e00640 1 --2- 172.21.15.2:0/2890759316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad80745c0 0x7f4ad819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.048+0000 7f4ad7e00640 1 -- 172.21.15.2:0/2890759316 >> 172.21.15.2:0/2890759316 conn(0x7f4ad80fd740 msgr2=0x7f4ad80fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:31.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.048+0000 7f4ad7e00640 1 -- 172.21.15.2:0/2890759316 shutdown_connections 2024-09-10T08:08:31.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.048+0000 7f4ad7e00640 1 -- 172.21.15.2:0/2890759316 wait complete. 2024-09-10T08:08:31.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:30 smithi002 ceph-mon[30563]: pgmap v398: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T08:08:31.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:30 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:08:31.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:30 smithi155 ceph-mon[35405]: pgmap v398: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T08:08:31.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:30 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:08:31.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:30 smithi107 ceph-mon[35048]: pgmap v398: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T08:08:31.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:30 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:08:31.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.259+0000 7fd4e2200640 1 -- 172.21.15.2:0/2439669328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc104010 msgr2=0x7fd4dc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.259+0000 7fd4e2200640 1 --2- 172.21.15.2:0/2439669328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc104010 0x7fd4dc1043f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009900 tx=0x7fd4cc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.259+0000 7fd4e2200640 1 -- 172.21.15.2:0/2439669328 shutdown_connections 2024-09-10T08:08:31.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.259+0000 7fd4e2200640 1 --2- 172.21.15.2:0/2439669328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4dc109140 0x7fd4dc110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.259+0000 7fd4e2200640 1 --2- 172.21.15.2:0/2439669328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4dc1049c0 0x7fd4dc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.259+0000 7fd4e2200640 1 --2- 172.21.15.2:0/2439669328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc104010 0x7fd4dc1043f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.259+0000 7fd4e2200640 1 -- 172.21.15.2:0/2439669328 >> 172.21.15.2:0/2439669328 conn(0x7fd4dc0ffa50 msgr2=0x7fd4dc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:31.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.260+0000 7fd4e2200640 1 -- 172.21.15.2:0/2439669328 shutdown_connections 2024-09-10T08:08:31.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.260+0000 7fd4e2200640 1 -- 172.21.15.2:0/2439669328 wait complete. 2024-09-10T08:08:31.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.260+0000 7fd4e2200640 1 Processor -- start 2024-09-10T08:08:31.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.261+0000 7fd4e2200640 1 -- start start 2024-09-10T08:08:31.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.261+0000 7fd4e2200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4dc104010 0x7fd4dc19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.261+0000 7fd4e2200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4dc1049c0 0x7fd4dc19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.261+0000 7fd4e2200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc109140 0x7fd4dc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.261+0000 7fd4e2200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc114190 con 0x7fd4dc1049c0 2024-09-10T08:08:31.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.261+0000 7fd4e2200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc114010 con 0x7fd4dc104010 2024-09-10T08:08:31.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.261+0000 7fd4e2200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc114310 con 0x7fd4dc109140 2024-09-10T08:08:31.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.261+0000 7fd4d3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4dc1049c0 0x7fd4dc19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.261+0000 7fd4db400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4dc104010 0x7fd4dc19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.261+0000 7fd4db400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4dc104010 0x7fd4dc19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:59456/0 (socket says 172.21.15.2:59456) 2024-09-10T08:08:31.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.261+0000 7fd4dbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc109140 0x7fd4dc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.261+0000 7fd4db400640 1 -- 172.21.15.2:0/116620590 learned_addr learned my addr 172.21.15.2:0/116620590 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.262+0000 7fd4dbe00640 1 -- 172.21.15.2:0/116620590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4dc104010 msgr2=0x7fd4dc19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.262+0000 7fd4dbe00640 1 --2- 172.21.15.2:0/116620590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4dc104010 0x7fd4dc19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.262+0000 7fd4dbe00640 1 -- 172.21.15.2:0/116620590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4dc1049c0 msgr2=0x7fd4dc19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.262+0000 7fd4dbe00640 1 --2- 172.21.15.2:0/116620590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4dc1049c0 0x7fd4dc19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.262+0000 7fd4dbe00640 1 -- 172.21.15.2:0/116620590 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4dc19f660 con 0x7fd4dc109140 2024-09-10T08:08:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.262+0000 7fd4dbe00640 1 --2- 172.21.15.2:0/116620590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc109140 0x7fd4dc19ef80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009fd0 tx=0x7fd4d400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.262+0000 7fd4d8c00640 1 -- 172.21.15.2:0/116620590 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400ce20 con 0x7fd4dc109140 2024-09-10T08:08:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.262+0000 7fd4e2200640 1 -- 172.21.15.2:0/116620590 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4dc19f950 con 0x7fd4dc109140 2024-09-10T08:08:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.262+0000 7fd4d8c00640 1 -- 172.21.15.2:0/116620590 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d4005020 con 0x7fd4dc109140 2024-09-10T08:08:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.262+0000 7fd4d8c00640 1 -- 172.21.15.2:0/116620590 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4010720 con 0x7fd4dc109140 2024-09-10T08:08:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.263+0000 7fd4e2200640 1 -- 172.21.15.2:0/116620590 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4dc1a7230 con 0x7fd4dc109140 2024-09-10T08:08:31.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.263+0000 7fd4e2200640 1 -- 172.21.15.2:0/116620590 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4dc1043f0 con 0x7fd4dc109140 2024-09-10T08:08:31.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.265+0000 7fd4d8c00640 1 -- 172.21.15.2:0/116620590 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fd4d4005540 con 0x7fd4dc109140 2024-09-10T08:08:31.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.267+0000 7fd4d8c00640 1 --2- 172.21.15.2:0/116620590 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4c00a2c10 0x7fd4c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:31.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.267+0000 7fd4d8c00640 1 -- 172.21.15.2:0/116620590 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd4d40cfc60 con 0x7fd4dc109140 2024-09-10T08:08:31.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.267+0000 7fd4db400640 1 --2- 172.21.15.2:0/116620590 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4c00a2c10 0x7fd4c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:31.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.268+0000 7fd4db400640 1 --2- 172.21.15.2:0/116620590 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4c00a2c10 0x7fd4c00a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0096c0 tx=0x7fd4cc0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:31.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.271+0000 7fd4d8c00640 1 -- 172.21.15.2:0/116620590 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4d409c610 con 0x7fd4dc109140 2024-09-10T08:08:31.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.483+0000 7fd4e2200640 1 -- 172.21.15.2:0/116620590 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4dc062630 con 0x7fd4c00a2c10 2024-09-10T08:08:31.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.492+0000 7fd4d8c00640 1 -- 172.21.15.2:0/116620590 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340724 (secure 0 0 0) 0x7fd4dc062630 con 0x7fd4c00a2c10 2024-09-10T08:08:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:31.689 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-10T08:08:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.684+0000 7fd4e2200640 1 -- 172.21.15.2:0/116620590 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4c00a2c10 msgr2=0x7fd4c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.684+0000 7fd4e2200640 1 --2- 172.21.15.2:0/116620590 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4c00a2c10 0x7fd4c00a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0096c0 tx=0x7fd4cc0046c0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.684+0000 7fd4e2200640 1 -- 172.21.15.2:0/116620590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc109140 msgr2=0x7fd4dc19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.684+0000 7fd4e2200640 1 --2- 172.21.15.2:0/116620590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc109140 0x7fd4dc19ef80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009fd0 tx=0x7fd4d400ef10 comp rx=0 tx=0).stop 2024-09-10T08:08:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.684+0000 7fd4e2200640 1 -- 172.21.15.2:0/116620590 shutdown_connections 2024-09-10T08:08:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.684+0000 7fd4e2200640 1 --2- 172.21.15.2:0/116620590 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4c00a2c10 0x7fd4c00a50d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.684+0000 7fd4e2200640 1 --2- 172.21.15.2:0/116620590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4dc109140 0x7fd4dc19ef80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.684+0000 7fd4e2200640 1 --2- 172.21.15.2:0/116620590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4dc1049c0 0x7fd4dc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.684+0000 7fd4e2200640 1 --2- 172.21.15.2:0/116620590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4dc104010 0x7fd4dc19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.684+0000 7fd4e2200640 1 -- 172.21.15.2:0/116620590 >> 172.21.15.2:0/116620590 conn(0x7fd4dc0ffa50 msgr2=0x7fd4dc100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.684+0000 7fd4e2200640 1 -- 172.21.15.2:0/116620590 shutdown_connections 2024-09-10T08:08:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:31.684+0000 7fd4e2200640 1 -- 172.21.15.2:0/116620590 wait complete. 2024-09-10T08:08:32.162 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:08:32.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:32 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:32.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:32 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:32.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:32 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:32.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:32 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:32.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:32 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:08:32.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:32 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:08:32.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:32 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:08:32.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:32 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:08:32.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:32 smithi155 ceph-mon[35405]: from='client.25228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:32.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:32 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:32 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:32 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:32 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:32 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:08:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:32 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:08:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:32 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:08:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:32 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:08:32.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:32 smithi107 ceph-mon[35048]: from='client.25228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:32.490 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:32 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:32.490 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:32 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:32.490 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:32 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:32.490 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:32 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:32.490 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:32 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:08:32.491 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:32 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:08:32.491 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:32 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:08:32.491 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:32 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:08:32.491 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:32 smithi002 ceph-mon[30563]: from='client.25228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:32.870 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-10T08:08:29.694595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667350,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":910,"num_read_kb":583,"num_write":373,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1184461,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":800,"ondisk_log_size":800,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362060,"kb_used_data":12812,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579764,"statfs":{"total":1151940427776,"available":1151569678336,"internally_reserved":0,"allocated":13119488,"data_stored":8554148,"data_compressed":497802,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9694,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":44,"num_write":16,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003378"},"pg_stats":[{"pgid":"2.1f","version":"93'9","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514795+0000","last_change":"2024-09-10T08:05:22.462062+0000","last_active":"2024-09-10T08:07:05.514795+0000","last_peered":"2024-09-10T08:07:05.514795+0000","last_clean":"2024-09-10T08:07:05.514795+0000","last_became_active":"2024-09-10T08:05:22.460973+0000","last_became_peered":"2024-09-10T08:05:22.460973+0000","last_unstale":"2024-09-10T08:07:05.514795+0000","last_undegraded":"2024-09-10T08:07:05.514795+0000","last_fullsized":"2024-09-10T08:07:05.514795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:01.357446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656803+0000","last_change":"2024-09-10T08:05:25.482873+0000","last_active":"2024-09-10T08:08:03.656803+0000","last_peered":"2024-09-10T08:08:03.656803+0000","last_clean":"2024-09-10T08:08:03.656803+0000","last_became_active":"2024-09-10T08:05:25.482559+0000","last_became_peered":"2024-09-10T08:05:25.482559+0000","last_unstale":"2024-09-10T08:08:03.656803+0000","last_undegraded":"2024-09-10T08:08:03.656803+0000","last_fullsized":"2024-09-10T08:08:03.656803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:10:17.366160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511384+0000","last_change":"2024-09-10T08:06:04.736356+0000","last_active":"2024-09-10T08:07:05.511384+0000","last_peered":"2024-09-10T08:07:05.511384+0000","last_clean":"2024-09-10T08:07:05.511384+0000","last_became_active":"2024-09-10T08:06:04.735859+0000","last_became_peered":"2024-09-10T08:06:04.735859+0000","last_unstale":"2024-09-10T08:07:05.511384+0000","last_undegraded":"2024-09-10T08:07:05.511384+0000","last_fullsized":"2024-09-10T08:07:05.511384+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:56.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"93'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.877433+0000","last_change":"2024-09-10T08:05:22.569504+0000","last_active":"2024-09-10T08:07:05.877433+0000","last_peered":"2024-09-10T08:07:05.877433+0000","last_clean":"2024-09-10T08:07:05.877433+0000","last_became_active":"2024-09-10T08:05:22.569164+0000","last_became_peered":"2024-09-10T08:05:22.569164+0000","last_unstale":"2024-09-10T08:07:05.877433+0000","last_undegraded":"2024-09-10T08:07:05.877433+0000","last_fullsized":"2024-09-10T08:07:05.877433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:41.628963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.680208+0000","last_change":"2024-09-10T08:05:25.493017+0000","last_active":"2024-09-10T08:08:03.680208+0000","last_peered":"2024-09-10T08:08:03.680208+0000","last_clean":"2024-09-10T08:08:03.680208+0000","last_became_active":"2024-09-10T08:05:25.491370+0000","last_became_peered":"2024-09-10T08:05:25.491370+0000","last_unstale":"2024-09-10T08:08:03.680208+0000","last_undegraded":"2024-09-10T08:08:03.680208+0000","last_fullsized":"2024-09-10T08:08:03.680208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:41.616549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914087+0000","last_change":"2024-09-10T08:05:35.523751+0000","last_active":"2024-09-10T08:07:05.914087+0000","last_peered":"2024-09-10T08:07:05.914087+0000","last_clean":"2024-09-10T08:07:05.914087+0000","last_became_active":"2024-09-10T08:05:35.523372+0000","last_became_peered":"2024-09-10T08:05:35.523372+0000","last_unstale":"2024-09-10T08:07:05.914087+0000","last_undegraded":"2024-09-10T08:07:05.914087+0000","last_fullsized":"2024-09-10T08:07:05.914087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:20.971596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508934+0000","last_change":"2024-09-10T08:05:22.460590+0000","last_active":"2024-09-10T08:07:05.508934+0000","last_peered":"2024-09-10T08:07:05.508934+0000","last_clean":"2024-09-10T08:07:05.508934+0000","last_became_active":"2024-09-10T08:05:22.460238+0000","last_became_peered":"2024-09-10T08:05:22.460238+0000","last_unstale":"2024-09-10T08:07:05.508934+0000","last_undegraded":"2024-09-10T08:07:05.508934+0000","last_fullsized":"2024-09-10T08:07:05.508934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:18.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669035+0000","last_change":"2024-09-10T08:05:25.487392+0000","last_active":"2024-09-10T08:08:03.669035+0000","last_peered":"2024-09-10T08:08:03.669035+0000","last_clean":"2024-09-10T08:08:03.669035+0000","last_became_active":"2024-09-10T08:05:25.487054+0000","last_became_peered":"2024-09-10T08:05:25.487054+0000","last_unstale":"2024-09-10T08:08:03.669035+0000","last_undegraded":"2024-09-10T08:08:03.669035+0000","last_fullsized":"2024-09-10T08:08:03.669035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:16.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914818+0000","last_change":"2024-09-10T08:06:04.735778+0000","last_active":"2024-09-10T08:07:05.914818+0000","last_peered":"2024-09-10T08:07:05.914818+0000","last_clean":"2024-09-10T08:07:05.914818+0000","last_became_active":"2024-09-10T08:06:04.735208+0000","last_became_peered":"2024-09-10T08:06:04.735208+0000","last_unstale":"2024-09-10T08:07:05.914818+0000","last_undegraded":"2024-09-10T08:07:05.914818+0000","last_fullsized":"2024-09-10T08:07:05.914818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:36.982246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508944+0000","last_change":"2024-09-10T08:05:22.458323+0000","last_active":"2024-09-10T08:07:05.508944+0000","last_peered":"2024-09-10T08:07:05.508944+0000","last_clean":"2024-09-10T08:07:05.508944+0000","last_became_active":"2024-09-10T08:05:22.457965+0000","last_became_peered":"2024-09-10T08:05:22.457965+0000","last_unstale":"2024-09-10T08:07:05.508944+0000","last_undegraded":"2024-09-10T08:07:05.508944+0000","last_fullsized":"2024-09-10T08:07:05.508944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:15.021305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.663997+0000","last_change":"2024-09-10T08:05:25.489883+0000","last_active":"2024-09-10T08:08:03.663997+0000","last_peered":"2024-09-10T08:08:03.663997+0000","last_clean":"2024-09-10T08:08:03.663997+0000","last_became_active":"2024-09-10T08:05:25.489482+0000","last_became_peered":"2024-09-10T08:05:25.489482+0000","last_unstale":"2024-09-10T08:08:03.663997+0000","last_undegraded":"2024-09-10T08:08:03.663997+0000","last_fullsized":"2024-09-10T08:08:03.663997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:52.080628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914064+0000","last_change":"2024-09-10T08:05:35.531458+0000","last_active":"2024-09-10T08:07:05.914064+0000","last_peered":"2024-09-10T08:07:05.914064+0000","last_clean":"2024-09-10T08:07:05.914064+0000","last_became_active":"2024-09-10T08:05:35.531072+0000","last_became_peered":"2024-09-10T08:05:35.531072+0000","last_unstale":"2024-09-10T08:07:05.914064+0000","last_undegraded":"2024-09-10T08:07:05.914064+0000","last_fullsized":"2024-09-10T08:07:05.914064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:04.792664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667650+0000","last_change":"2024-09-10T08:05:25.486865+0000","last_active":"2024-09-10T08:08:03.667650+0000","last_peered":"2024-09-10T08:08:03.667650+0000","last_clean":"2024-09-10T08:08:03.667650+0000","last_became_active":"2024-09-10T08:05:25.486531+0000","last_became_peered":"2024-09-10T08:05:25.486531+0000","last_unstale":"2024-09-10T08:08:03.667650+0000","last_undegraded":"2024-09-10T08:08:03.667650+0000","last_fullsized":"2024-09-10T08:08:03.667650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:26.854682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"93'4","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508917+0000","last_change":"2024-09-10T08:06:04.743303+0000","last_active":"2024-09-10T08:07:05.508917+0000","last_peered":"2024-09-10T08:07:05.508917+0000","last_clean":"2024-09-10T08:07:05.508917+0000","last_became_active":"2024-09-10T08:06:04.733294+0000","last_became_peered":"2024-09-10T08:06:04.733294+0000","last_unstale":"2024-09-10T08:07:05.508917+0000","last_undegraded":"2024-09-10T08:07:05.508917+0000","last_fullsized":"2024-09-10T08:07:05.508917+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:18.542658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914822+0000","last_change":"2024-09-10T08:05:35.642083+0000","last_active":"2024-09-10T08:07:05.914822+0000","last_peered":"2024-09-10T08:07:05.914822+0000","last_clean":"2024-09-10T08:07:05.914822+0000","last_became_active":"2024-09-10T08:05:35.641214+0000","last_became_peered":"2024-09-10T08:05:35.641214+0000","last_unstale":"2024-09-10T08:07:05.914822+0000","last_undegraded":"2024-09-10T08:07:05.914822+0000","last_fullsized":"2024-09-10T08:07:05.914822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:00:21.731769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681852+0000","last_change":"2024-09-10T08:05:25.501993+0000","last_active":"2024-09-10T08:08:03.681852+0000","last_peered":"2024-09-10T08:08:03.681852+0000","last_clean":"2024-09-10T08:08:03.681852+0000","last_became_active":"2024-09-10T08:05:25.499912+0000","last_became_peered":"2024-09-10T08:05:25.499912+0000","last_unstale":"2024-09-10T08:08:03.681852+0000","last_undegraded":"2024-09-10T08:08:03.681852+0000","last_fullsized":"2024-09-10T08:08:03.681852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:06.780067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:36.268711+0000","last_change":"2024-09-10T08:05:22.461786+0000","last_active":"2024-09-10T08:07:36.268711+0000","last_peered":"2024-09-10T08:07:36.268711+0000","last_clean":"2024-09-10T08:07:36.268711+0000","last_became_active":"2024-09-10T08:05:22.460821+0000","last_became_peered":"2024-09-10T08:05:22.460821+0000","last_unstale":"2024-09-10T08:07:36.268711+0000","last_undegraded":"2024-09-10T08:07:36.268711+0000","last_fullsized":"2024-09-10T08:07:36.268711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:25.083538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914568+0000","last_change":"2024-09-10T08:06:04.734470+0000","last_active":"2024-09-10T08:07:05.914568+0000","last_peered":"2024-09-10T08:07:05.914568+0000","last_clean":"2024-09-10T08:07:05.914568+0000","last_became_active":"2024-09-10T08:06:04.734040+0000","last_became_peered":"2024-09-10T08:06:04.734040+0000","last_unstale":"2024-09-10T08:07:05.914568+0000","last_undegraded":"2024-09-10T08:07:05.914568+0000","last_fullsized":"2024-09-10T08:07:05.914568+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:36.312144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675820+0000","last_change":"2024-09-10T08:05:25.492711+0000","last_active":"2024-09-10T08:08:03.675820+0000","last_peered":"2024-09-10T08:08:03.675820+0000","last_clean":"2024-09-10T08:08:03.675820+0000","last_became_active":"2024-09-10T08:05:25.492336+0000","last_became_peered":"2024-09-10T08:05:25.492336+0000","last_unstale":"2024-09-10T08:08:03.675820+0000","last_undegraded":"2024-09-10T08:08:03.675820+0000","last_fullsized":"2024-09-10T08:08:03.675820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:04.168884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"93'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876783+0000","last_change":"2024-09-10T08:05:22.461535+0000","last_active":"2024-09-10T08:07:05.876783+0000","last_peered":"2024-09-10T08:07:05.876783+0000","last_clean":"2024-09-10T08:07:05.876783+0000","last_became_active":"2024-09-10T08:05:22.461208+0000","last_became_peered":"2024-09-10T08:05:22.461208+0000","last_unstale":"2024-09-10T08:07:05.876783+0000","last_undegraded":"2024-09-10T08:07:05.876783+0000","last_fullsized":"2024-09-10T08:07:05.876783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:26.009242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513889+0000","last_change":"2024-09-10T08:05:35.535442+0000","last_active":"2024-09-10T08:07:05.513889+0000","last_peered":"2024-09-10T08:07:05.513889+0000","last_clean":"2024-09-10T08:07:05.513889+0000","last_became_active":"2024-09-10T08:05:35.535058+0000","last_became_peered":"2024-09-10T08:05:35.535058+0000","last_unstale":"2024-09-10T08:07:05.513889+0000","last_undegraded":"2024-09-10T08:07:05.513889+0000","last_fullsized":"2024-09-10T08:07:05.513889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:39.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.627539+0000","last_change":"2024-09-10T08:05:25.483440+0000","last_active":"2024-09-10T08:08:03.627539+0000","last_peered":"2024-09-10T08:08:03.627539+0000","last_clean":"2024-09-10T08:08:03.627539+0000","last_became_active":"2024-09-10T08:05:25.483141+0000","last_became_peered":"2024-09-10T08:05:25.483141+0000","last_unstale":"2024-09-10T08:08:03.627539+0000","last_undegraded":"2024-09-10T08:08:03.627539+0000","last_fullsized":"2024-09-10T08:08:03.627539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:00.953188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"93'2","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914337+0000","last_change":"2024-09-10T08:05:22.568141+0000","last_active":"2024-09-10T08:07:05.914337+0000","last_peered":"2024-09-10T08:07:05.914337+0000","last_clean":"2024-09-10T08:07:05.914337+0000","last_became_active":"2024-09-10T08:05:22.567857+0000","last_became_peered":"2024-09-10T08:05:22.567857+0000","last_unstale":"2024-09-10T08:07:05.914337+0000","last_undegraded":"2024-09-10T08:07:05.914337+0000","last_fullsized":"2024-09-10T08:07:05.914337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:22.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914846+0000","last_change":"2024-09-10T08:05:35.641993+0000","last_active":"2024-09-10T08:07:05.914846+0000","last_peered":"2024-09-10T08:07:05.914846+0000","last_clean":"2024-09-10T08:07:05.914846+0000","last_became_active":"2024-09-10T08:05:35.641376+0000","last_became_peered":"2024-09-10T08:05:35.641376+0000","last_unstale":"2024-09-10T08:07:05.914846+0000","last_undegraded":"2024-09-10T08:07:05.914846+0000","last_fullsized":"2024-09-10T08:07:05.914846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:16:34.038924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.678764+0000","last_change":"2024-09-10T08:06:04.737560+0000","last_active":"2024-09-10T08:08:03.678764+0000","last_peered":"2024-09-10T08:08:03.678764+0000","last_clean":"2024-09-10T08:08:03.678764+0000","last_became_active":"2024-09-10T08:06:04.736656+0000","last_became_peered":"2024-09-10T08:06:04.736656+0000","last_unstale":"2024-09-10T08:08:03.678764+0000","last_undegraded":"2024-09-10T08:08:03.678764+0000","last_fullsized":"2024-09-10T08:08:03.678764+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:35.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"93'4","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915999+0000","last_change":"2024-09-10T08:05:22.567077+0000","last_active":"2024-09-10T08:07:05.915999+0000","last_peered":"2024-09-10T08:07:05.915999+0000","last_clean":"2024-09-10T08:07:05.915999+0000","last_became_active":"2024-09-10T08:05:22.566757+0000","last_became_peered":"2024-09-10T08:05:22.566757+0000","last_unstale":"2024-09-10T08:07:05.915999+0000","last_undegraded":"2024-09-10T08:07:05.915999+0000","last_fullsized":"2024-09-10T08:07:05.915999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:06.881510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511524+0000","last_change":"2024-09-10T08:05:35.519539+0000","last_active":"2024-09-10T08:07:05.511524+0000","last_peered":"2024-09-10T08:07:05.511524+0000","last_clean":"2024-09-10T08:07:05.511524+0000","last_became_active":"2024-09-10T08:05:35.519044+0000","last_became_peered":"2024-09-10T08:05:35.519044+0000","last_unstale":"2024-09-10T08:07:05.511524+0000","last_undegraded":"2024-09-10T08:07:05.511524+0000","last_fullsized":"2024-09-10T08:07:05.511524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:10.546949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658164+0000","last_change":"2024-09-10T08:07:05.771774+0000","last_active":"2024-09-10T08:08:03.658164+0000","last_peered":"2024-09-10T08:08:03.658164+0000","last_clean":"2024-09-10T08:08:03.658164+0000","last_became_active":"2024-09-10T08:07:05.771336+0000","last_became_peered":"2024-09-10T08:07:05.771336+0000","last_unstale":"2024-09-10T08:08:03.658164+0000","last_undegraded":"2024-09-10T08:08:03.658164+0000","last_fullsized":"2024-09-10T08:08:03.658164+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:11.482222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"95'14","reported_seq":68,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:18.518255+0000","last_change":"2024-09-10T08:06:04.741199+0000","last_active":"2024-09-10T08:08:18.518255+0000","last_peered":"2024-09-10T08:08:18.518255+0000","last_clean":"2024-09-10T08:08:18.518255+0000","last_became_active":"2024-09-10T08:06:04.738076+0000","last_became_peered":"2024-09-10T08:06:04.738076+0000","last_unstale":"2024-09-10T08:08:18.518255+0000","last_undegraded":"2024-09-10T08:08:18.518255+0000","last_fullsized":"2024-09-10T08:08:18.518255+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:45.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876462+0000","last_change":"2024-09-10T08:05:35.642358+0000","last_active":"2024-09-10T08:07:05.876462+0000","last_peered":"2024-09-10T08:07:05.876462+0000","last_clean":"2024-09-10T08:07:05.876462+0000","last_became_active":"2024-09-10T08:05:35.641408+0000","last_became_peered":"2024-09-10T08:05:35.641408+0000","last_unstale":"2024-09-10T08:07:05.876462+0000","last_undegraded":"2024-09-10T08:07:05.876462+0000","last_fullsized":"2024-09-10T08:07:05.876462+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:42.015139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658430+0000","last_change":"2024-09-10T08:05:25.486310+0000","last_active":"2024-09-10T08:08:03.658430+0000","last_peered":"2024-09-10T08:08:03.658430+0000","last_clean":"2024-09-10T08:08:03.658430+0000","last_became_active":"2024-09-10T08:05:25.485915+0000","last_became_peered":"2024-09-10T08:05:25.485915+0000","last_unstale":"2024-09-10T08:08:03.658430+0000","last_undegraded":"2024-09-10T08:08:03.658430+0000","last_fullsized":"2024-09-10T08:08:03.658430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:56.064361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"93'1","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508987+0000","last_change":"2024-09-10T08:05:22.570595+0000","last_active":"2024-09-10T08:07:05.508987+0000","last_peered":"2024-09-10T08:07:05.508987+0000","last_clean":"2024-09-10T08:07:05.508987+0000","last_became_active":"2024-09-10T08:05:22.570219+0000","last_became_peered":"2024-09-10T08:05:22.570219+0000","last_unstale":"2024-09-10T08:07:05.508987+0000","last_undegraded":"2024-09-10T08:07:05.508987+0000","last_fullsized":"2024-09-10T08:07:05.508987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:03.230325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769927+0000","last_change":"2024-09-10T08:05:35.530472+0000","last_active":"2024-09-10T08:07:05.769927+0000","last_peered":"2024-09-10T08:07:05.769927+0000","last_clean":"2024-09-10T08:07:05.769927+0000","last_became_active":"2024-09-10T08:05:35.530066+0000","last_became_peered":"2024-09-10T08:05:35.530066+0000","last_unstale":"2024-09-10T08:07:05.769927+0000","last_undegraded":"2024-09-10T08:07:05.769927+0000","last_fullsized":"2024-09-10T08:07:05.769927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:58.915767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671834+0000","last_change":"2024-09-10T08:05:25.500524+0000","last_active":"2024-09-10T08:08:03.671834+0000","last_peered":"2024-09-10T08:08:03.671834+0000","last_clean":"2024-09-10T08:08:03.671834+0000","last_became_active":"2024-09-10T08:05:25.500186+0000","last_became_peered":"2024-09-10T08:05:25.500186+0000","last_unstale":"2024-09-10T08:08:03.671834+0000","last_undegraded":"2024-09-10T08:08:03.671834+0000","last_fullsized":"2024-09-10T08:08:03.671834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:51.936222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513172+0000","last_change":"2024-09-10T08:05:22.569326+0000","last_active":"2024-09-10T08:07:05.513172+0000","last_peered":"2024-09-10T08:07:05.513172+0000","last_clean":"2024-09-10T08:07:05.513172+0000","last_became_active":"2024-09-10T08:05:22.568895+0000","last_became_peered":"2024-09-10T08:05:22.568895+0000","last_unstale":"2024-09-10T08:07:05.513172+0000","last_undegraded":"2024-09-10T08:07:05.513172+0000","last_fullsized":"2024-09-10T08:07:05.513172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:47.212073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508368+0000","last_change":"2024-09-10T08:05:35.641600+0000","last_active":"2024-09-10T08:07:05.508368+0000","last_peered":"2024-09-10T08:07:05.508368+0000","last_clean":"2024-09-10T08:07:05.508368+0000","last_became_active":"2024-09-10T08:05:35.641205+0000","last_became_peered":"2024-09-10T08:05:35.641205+0000","last_unstale":"2024-09-10T08:07:05.508368+0000","last_undegraded":"2024-09-10T08:07:05.508368+0000","last_fullsized":"2024-09-10T08:07:05.508368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:40.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.659935+0000","last_change":"2024-09-10T08:05:25.495035+0000","last_active":"2024-09-10T08:08:03.659935+0000","last_peered":"2024-09-10T08:08:03.659935+0000","last_clean":"2024-09-10T08:08:03.659935+0000","last_became_active":"2024-09-10T08:05:25.494733+0000","last_became_peered":"2024-09-10T08:05:25.494733+0000","last_unstale":"2024-09-10T08:08:03.659935+0000","last_undegraded":"2024-09-10T08:08:03.659935+0000","last_fullsized":"2024-09-10T08:08:03.659935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.875867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"93'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509485+0000","last_change":"2024-09-10T08:05:22.458727+0000","last_active":"2024-09-10T08:07:05.509485+0000","last_peered":"2024-09-10T08:07:05.509485+0000","last_clean":"2024-09-10T08:07:05.509485+0000","last_became_active":"2024-09-10T08:05:22.458397+0000","last_became_peered":"2024-09-10T08:05:22.458397+0000","last_unstale":"2024-09-10T08:07:05.509485+0000","last_undegraded":"2024-09-10T08:07:05.509485+0000","last_fullsized":"2024-09-10T08:07:05.509485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:07.875559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914403+0000","last_change":"2024-09-10T08:05:35.534192+0000","last_active":"2024-09-10T08:07:05.914403+0000","last_peered":"2024-09-10T08:07:05.914403+0000","last_clean":"2024-09-10T08:07:05.914403+0000","last_became_active":"2024-09-10T08:05:35.530450+0000","last_became_peered":"2024-09-10T08:05:35.530450+0000","last_unstale":"2024-09-10T08:07:05.914403+0000","last_undegraded":"2024-09-10T08:07:05.914403+0000","last_fullsized":"2024-09-10T08:07:05.914403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:10.039144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651611+0000","last_change":"2024-09-10T08:05:25.482252+0000","last_active":"2024-09-10T08:08:03.651611+0000","last_peered":"2024-09-10T08:08:03.651611+0000","last_clean":"2024-09-10T08:08:03.651611+0000","last_became_active":"2024-09-10T08:05:25.481901+0000","last_became_peered":"2024-09-10T08:05:25.481901+0000","last_unstale":"2024-09-10T08:08:03.651611+0000","last_undegraded":"2024-09-10T08:08:03.651611+0000","last_fullsized":"2024-09-10T08:08:03.651611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:17.063340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":175,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:26.454478+0000","last_change":"2024-09-10T08:05:22.569199+0000","last_active":"2024-09-10T08:08:26.454478+0000","last_peered":"2024-09-10T08:08:26.454478+0000","last_clean":"2024-09-10T08:08:26.454478+0000","last_became_active":"2024-09-10T08:05:22.568787+0000","last_became_peered":"2024-09-10T08:05:22.568787+0000","last_unstale":"2024-09-10T08:08:26.454478+0000","last_undegraded":"2024-09-10T08:08:26.454478+0000","last_fullsized":"2024-09-10T08:08:26.454478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:59.610130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914677+0000","last_change":"2024-09-10T08:05:35.642446+0000","last_active":"2024-09-10T08:07:05.914677+0000","last_peered":"2024-09-10T08:07:05.914677+0000","last_clean":"2024-09-10T08:07:05.914677+0000","last_became_active":"2024-09-10T08:05:35.641543+0000","last_became_peered":"2024-09-10T08:05:35.641543+0000","last_unstale":"2024-09-10T08:07:05.914677+0000","last_undegraded":"2024-09-10T08:07:05.914677+0000","last_fullsized":"2024-09-10T08:07:05.914677+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:33.658406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.652192+0000","last_change":"2024-09-10T08:05:25.487598+0000","last_active":"2024-09-10T08:08:03.652192+0000","last_peered":"2024-09-10T08:08:03.652192+0000","last_clean":"2024-09-10T08:08:03.652192+0000","last_became_active":"2024-09-10T08:05:25.486879+0000","last_became_peered":"2024-09-10T08:05:25.486879+0000","last_unstale":"2024-09-10T08:08:03.652192+0000","last_undegraded":"2024-09-10T08:08:03.652192+0000","last_fullsized":"2024-09-10T08:08:03.652192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:20.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509934+0000","last_change":"2024-09-10T08:05:22.463416+0000","last_active":"2024-09-10T08:07:05.509934+0000","last_peered":"2024-09-10T08:07:05.509934+0000","last_clean":"2024-09-10T08:07:05.509934+0000","last_became_active":"2024-09-10T08:05:22.463004+0000","last_became_peered":"2024-09-10T08:05:22.463004+0000","last_unstale":"2024-09-10T08:07:05.509934+0000","last_undegraded":"2024-09-10T08:07:05.509934+0000","last_fullsized":"2024-09-10T08:07:05.509934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:56.257812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569364+0000","last_change":"2024-09-10T08:05:35.642059+0000","last_active":"2024-09-10T08:07:05.569364+0000","last_peered":"2024-09-10T08:07:05.569364+0000","last_clean":"2024-09-10T08:07:05.569364+0000","last_became_active":"2024-09-10T08:05:35.641540+0000","last_became_peered":"2024-09-10T08:05:35.641540+0000","last_unstale":"2024-09-10T08:07:05.569364+0000","last_undegraded":"2024-09-10T08:07:05.569364+0000","last_fullsized":"2024-09-10T08:07:05.569364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:28.557584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.670571+0000","last_change":"2024-09-10T08:05:25.496111+0000","last_active":"2024-09-10T08:08:03.670571+0000","last_peered":"2024-09-10T08:08:03.670571+0000","last_clean":"2024-09-10T08:08:03.670571+0000","last_became_active":"2024-09-10T08:05:25.495757+0000","last_became_peered":"2024-09-10T08:05:25.495757+0000","last_unstale":"2024-09-10T08:08:03.670571+0000","last_undegraded":"2024-09-10T08:08:03.670571+0000","last_fullsized":"2024-09-10T08:08:03.670571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:41.078205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'3","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915005+0000","last_change":"2024-09-10T08:05:22.460069+0000","last_active":"2024-09-10T08:07:05.915005+0000","last_peered":"2024-09-10T08:07:05.915005+0000","last_clean":"2024-09-10T08:07:05.915005+0000","last_became_active":"2024-09-10T08:05:22.459557+0000","last_became_peered":"2024-09-10T08:05:22.459557+0000","last_unstale":"2024-09-10T08:07:05.915005+0000","last_undegraded":"2024-09-10T08:07:05.915005+0000","last_fullsized":"2024-09-10T08:07:05.915005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:49.058507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509959+0000","last_change":"2024-09-10T08:04:08.188262+0000","last_active":"2024-09-10T08:07:05.509959+0000","last_peered":"2024-09-10T08:07:05.509959+0000","last_clean":"2024-09-10T08:07:05.509959+0000","last_became_active":"2024-09-10T08:04:08.187691+0000","last_became_peered":"2024-09-10T08:04:08.187691+0000","last_unstale":"2024-09-10T08:07:05.509959+0000","last_undegraded":"2024-09-10T08:07:05.509959+0000","last_fullsized":"2024-09-10T08:07:05.509959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_clean_scrub_stamp":"2024-09-10T08:00:26.235127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:15.358818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509550+0000","last_change":"2024-09-10T08:05:35.528371+0000","last_active":"2024-09-10T08:07:05.509550+0000","last_peered":"2024-09-10T08:07:05.509550+0000","last_clean":"2024-09-10T08:07:05.509550+0000","last_became_active":"2024-09-10T08:05:35.527925+0000","last_became_peered":"2024-09-10T08:05:35.527925+0000","last_unstale":"2024-09-10T08:07:05.509550+0000","last_undegraded":"2024-09-10T08:07:05.509550+0000","last_fullsized":"2024-09-10T08:07:05.509550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:37:00.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667384+0000","last_change":"2024-09-10T08:05:25.490753+0000","last_active":"2024-09-10T08:08:03.667384+0000","last_peered":"2024-09-10T08:08:03.667384+0000","last_clean":"2024-09-10T08:08:03.667384+0000","last_became_active":"2024-09-10T08:05:25.489996+0000","last_became_peered":"2024-09-10T08:05:25.489996+0000","last_unstale":"2024-09-10T08:08:03.667384+0000","last_undegraded":"2024-09-10T08:08:03.667384+0000","last_fullsized":"2024-09-10T08:08:03.667384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:15.637199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'3","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511918+0000","last_change":"2024-09-10T08:05:22.568696+0000","last_active":"2024-09-10T08:07:05.511918+0000","last_peered":"2024-09-10T08:07:05.511918+0000","last_clean":"2024-09-10T08:07:05.511918+0000","last_became_active":"2024-09-10T08:05:22.568331+0000","last_became_peered":"2024-09-10T08:05:22.568331+0000","last_unstale":"2024-09-10T08:07:05.511918+0000","last_undegraded":"2024-09-10T08:07:05.511918+0000","last_fullsized":"2024-09-10T08:07:05.511918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:21.699397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914428+0000","last_change":"2024-09-10T08:05:35.530804+0000","last_active":"2024-09-10T08:07:05.914428+0000","last_peered":"2024-09-10T08:07:05.914428+0000","last_clean":"2024-09-10T08:07:05.914428+0000","last_became_active":"2024-09-10T08:05:35.530356+0000","last_became_peered":"2024-09-10T08:05:35.530356+0000","last_unstale":"2024-09-10T08:07:05.914428+0000","last_undegraded":"2024-09-10T08:07:05.914428+0000","last_fullsized":"2024-09-10T08:07:05.914428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:59.318761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.662813+0000","last_change":"2024-09-10T08:05:25.489546+0000","last_active":"2024-09-10T08:08:03.662813+0000","last_peered":"2024-09-10T08:08:03.662813+0000","last_clean":"2024-09-10T08:08:03.662813+0000","last_became_active":"2024-09-10T08:05:25.489188+0000","last_became_peered":"2024-09-10T08:05:25.489188+0000","last_unstale":"2024-09-10T08:08:03.662813+0000","last_undegraded":"2024-09-10T08:08:03.662813+0000","last_fullsized":"2024-09-10T08:08:03.662813+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:26.471681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569736+0000","last_change":"2024-09-10T08:05:22.463912+0000","last_active":"2024-09-10T08:07:05.569736+0000","last_peered":"2024-09-10T08:07:05.569736+0000","last_clean":"2024-09-10T08:07:05.569736+0000","last_became_active":"2024-09-10T08:05:22.463561+0000","last_became_peered":"2024-09-10T08:05:22.463561+0000","last_unstale":"2024-09-10T08:07:05.569736+0000","last_undegraded":"2024-09-10T08:07:05.569736+0000","last_fullsized":"2024-09-10T08:07:05.569736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:22.407604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514959+0000","last_change":"2024-09-10T08:05:35.528795+0000","last_active":"2024-09-10T08:07:05.514959+0000","last_peered":"2024-09-10T08:07:05.514959+0000","last_clean":"2024-09-10T08:07:05.514959+0000","last_became_active":"2024-09-10T08:05:35.528402+0000","last_became_peered":"2024-09-10T08:05:35.528402+0000","last_unstale":"2024-09-10T08:07:05.514959+0000","last_undegraded":"2024-09-10T08:07:05.514959+0000","last_fullsized":"2024-09-10T08:07:05.514959+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:14.606371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.657641+0000","last_change":"2024-09-10T08:05:25.484086+0000","last_active":"2024-09-10T08:08:03.657641+0000","last_peered":"2024-09-10T08:08:03.657641+0000","last_clean":"2024-09-10T08:08:03.657641+0000","last_became_active":"2024-09-10T08:05:25.483757+0000","last_became_peered":"2024-09-10T08:05:25.483757+0000","last_unstale":"2024-09-10T08:08:03.657641+0000","last_undegraded":"2024-09-10T08:08:03.657641+0000","last_fullsized":"2024-09-10T08:08:03.657641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:16:16.362739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914678+0000","last_change":"2024-09-10T08:05:22.462220+0000","last_active":"2024-09-10T08:07:05.914678+0000","last_peered":"2024-09-10T08:07:05.914678+0000","last_clean":"2024-09-10T08:07:05.914678+0000","last_became_active":"2024-09-10T08:05:22.461726+0000","last_became_peered":"2024-09-10T08:05:22.461726+0000","last_unstale":"2024-09-10T08:07:05.914678+0000","last_undegraded":"2024-09-10T08:07:05.914678+0000","last_fullsized":"2024-09-10T08:07:05.914678+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:56.381034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508271+0000","last_change":"2024-09-10T08:06:04.734827+0000","last_active":"2024-09-10T08:07:05.508271+0000","last_peered":"2024-09-10T08:07:05.508271+0000","last_clean":"2024-09-10T08:07:05.508271+0000","last_became_active":"2024-09-10T08:06:04.734402+0000","last_became_peered":"2024-09-10T08:06:04.734402+0000","last_unstale":"2024-09-10T08:07:05.508271+0000","last_undegraded":"2024-09-10T08:07:05.508271+0000","last_fullsized":"2024-09-10T08:07:05.508271+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:24.806538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654332+0000","last_change":"2024-09-10T08:05:25.486100+0000","last_active":"2024-09-10T08:08:03.654332+0000","last_peered":"2024-09-10T08:08:03.654332+0000","last_clean":"2024-09-10T08:08:03.654332+0000","last_became_active":"2024-09-10T08:05:25.485586+0000","last_became_peered":"2024-09-10T08:05:25.485586+0000","last_unstale":"2024-09-10T08:08:03.654332+0000","last_undegraded":"2024-09-10T08:08:03.654332+0000","last_fullsized":"2024-09-10T08:08:03.654332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:33.565054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"93'2","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770403+0000","last_change":"2024-09-10T08:06:04.743408+0000","last_active":"2024-09-10T08:07:05.770403+0000","last_peered":"2024-09-10T08:07:05.770403+0000","last_clean":"2024-09-10T08:07:05.770403+0000","last_became_active":"2024-09-10T08:06:04.732830+0000","last_became_peered":"2024-09-10T08:06:04.732830+0000","last_unstale":"2024-09-10T08:07:05.770403+0000","last_undegraded":"2024-09-10T08:07:05.770403+0000","last_fullsized":"2024-09-10T08:07:05.770403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:29.381259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568977+0000","last_change":"2024-09-10T08:05:35.533015+0000","last_active":"2024-09-10T08:07:05.568977+0000","last_peered":"2024-09-10T08:07:05.568977+0000","last_clean":"2024-09-10T08:07:05.568977+0000","last_became_active":"2024-09-10T08:05:35.532590+0000","last_became_peered":"2024-09-10T08:05:35.532590+0000","last_unstale":"2024-09-10T08:07:05.568977+0000","last_undegraded":"2024-09-10T08:07:05.568977+0000","last_fullsized":"2024-09-10T08:07:05.568977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:23.984273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.650197+0000","last_change":"2024-09-10T08:05:25.489994+0000","last_active":"2024-09-10T08:08:03.650197+0000","last_peered":"2024-09-10T08:08:03.650197+0000","last_clean":"2024-09-10T08:08:03.650197+0000","last_became_active":"2024-09-10T08:05:25.489471+0000","last_became_peered":"2024-09-10T08:05:25.489471+0000","last_unstale":"2024-09-10T08:08:03.650197+0000","last_undegraded":"2024-09-10T08:08:03.650197+0000","last_fullsized":"2024-09-10T08:08:03.650197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:53.866570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"93'15","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508405+0000","last_change":"2024-09-10T08:05:22.568900+0000","last_active":"2024-09-10T08:07:05.508405+0000","last_peered":"2024-09-10T08:07:05.508405+0000","last_clean":"2024-09-10T08:07:05.508405+0000","last_became_active":"2024-09-10T08:05:22.568542+0000","last_became_peered":"2024-09-10T08:05:22.568542+0000","last_unstale":"2024-09-10T08:07:05.508405+0000","last_undegraded":"2024-09-10T08:07:05.508405+0000","last_fullsized":"2024-09-10T08:07:05.508405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:58:45.669769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914683+0000","last_change":"2024-09-10T08:05:35.537364+0000","last_active":"2024-09-10T08:07:05.914683+0000","last_peered":"2024-09-10T08:07:05.914683+0000","last_clean":"2024-09-10T08:07:05.914683+0000","last_became_active":"2024-09-10T08:05:35.535958+0000","last_became_peered":"2024-09-10T08:05:35.535958+0000","last_unstale":"2024-09-10T08:07:05.914683+0000","last_undegraded":"2024-09-10T08:07:05.914683+0000","last_fullsized":"2024-09-10T08:07:05.914683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:51.443798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660149+0000","last_change":"2024-09-10T08:07:05.917030+0000","last_active":"2024-09-10T08:08:03.660149+0000","last_peered":"2024-09-10T08:08:03.660149+0000","last_clean":"2024-09-10T08:08:03.660149+0000","last_became_active":"2024-09-10T08:07:05.916281+0000","last_became_peered":"2024-09-10T08:07:05.916281+0000","last_unstale":"2024-09-10T08:08:03.660149+0000","last_undegraded":"2024-09-10T08:08:03.660149+0000","last_fullsized":"2024-09-10T08:08:03.660149+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:15.060009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'4","reported_seq":176,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:25.453378+0000","last_change":"2024-09-10T08:05:22.459963+0000","last_active":"2024-09-10T08:08:25.453378+0000","last_peered":"2024-09-10T08:08:25.453378+0000","last_clean":"2024-09-10T08:08:25.453378+0000","last_became_active":"2024-09-10T08:05:22.459497+0000","last_became_peered":"2024-09-10T08:05:22.459497+0000","last_unstale":"2024-09-10T08:08:25.453378+0000","last_undegraded":"2024-09-10T08:08:25.453378+0000","last_fullsized":"2024-09-10T08:08:25.453378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:29.887980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513986+0000","last_change":"2024-09-10T08:05:35.534570+0000","last_active":"2024-09-10T08:07:05.513986+0000","last_peered":"2024-09-10T08:07:05.513986+0000","last_clean":"2024-09-10T08:07:05.513986+0000","last_became_active":"2024-09-10T08:05:35.534195+0000","last_became_peered":"2024-09-10T08:05:35.534195+0000","last_unstale":"2024-09-10T08:07:05.513986+0000","last_undegraded":"2024-09-10T08:07:05.513986+0000","last_fullsized":"2024-09-10T08:07:05.513986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:17.393491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.668227+0000","last_change":"2024-09-10T08:05:25.488551+0000","last_active":"2024-09-10T08:08:03.668227+0000","last_peered":"2024-09-10T08:08:03.668227+0000","last_clean":"2024-09-10T08:08:03.668227+0000","last_became_active":"2024-09-10T08:05:25.488086+0000","last_became_peered":"2024-09-10T08:05:25.488086+0000","last_unstale":"2024-09-10T08:08:03.668227+0000","last_undegraded":"2024-09-10T08:08:03.668227+0000","last_fullsized":"2024-09-10T08:08:03.668227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:38.044721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569070+0000","last_change":"2024-09-10T08:05:22.462269+0000","last_active":"2024-09-10T08:07:05.569070+0000","last_peered":"2024-09-10T08:07:05.569070+0000","last_clean":"2024-09-10T08:07:05.569070+0000","last_became_active":"2024-09-10T08:05:22.461924+0000","last_became_peered":"2024-09-10T08:05:22.461924+0000","last_unstale":"2024-09-10T08:07:05.569070+0000","last_undegraded":"2024-09-10T08:07:05.569070+0000","last_fullsized":"2024-09-10T08:07:05.569070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:54.996777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770191+0000","last_change":"2024-09-10T08:05:35.531596+0000","last_active":"2024-09-10T08:07:05.770191+0000","last_peered":"2024-09-10T08:07:05.770191+0000","last_clean":"2024-09-10T08:07:05.770191+0000","last_became_active":"2024-09-10T08:05:35.530179+0000","last_became_peered":"2024-09-10T08:05:35.530179+0000","last_unstale":"2024-09-10T08:07:05.770191+0000","last_undegraded":"2024-09-10T08:07:05.770191+0000","last_fullsized":"2024-09-10T08:07:05.770191+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:32.056426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.663218+0000","last_change":"2024-09-10T08:05:25.503504+0000","last_active":"2024-09-10T08:08:03.663218+0000","last_peered":"2024-09-10T08:08:03.663218+0000","last_clean":"2024-09-10T08:08:03.663218+0000","last_became_active":"2024-09-10T08:05:25.503105+0000","last_became_peered":"2024-09-10T08:05:25.503105+0000","last_unstale":"2024-09-10T08:08:03.663218+0000","last_undegraded":"2024-09-10T08:08:03.663218+0000","last_fullsized":"2024-09-10T08:08:03.663218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:40.885654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"93'6","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511623+0000","last_change":"2024-09-10T08:05:22.462271+0000","last_active":"2024-09-10T08:07:05.511623+0000","last_peered":"2024-09-10T08:07:05.511623+0000","last_clean":"2024-09-10T08:07:05.511623+0000","last_became_active":"2024-09-10T08:05:22.461195+0000","last_became_peered":"2024-09-10T08:05:22.461195+0000","last_unstale":"2024-09-10T08:07:05.511623+0000","last_undegraded":"2024-09-10T08:07:05.511623+0000","last_fullsized":"2024-09-10T08:07:05.511623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:20.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569486+0000","last_change":"2024-09-10T08:05:35.533218+0000","last_active":"2024-09-10T08:07:05.569486+0000","last_peered":"2024-09-10T08:07:05.569486+0000","last_clean":"2024-09-10T08:07:05.569486+0000","last_became_active":"2024-09-10T08:05:35.532706+0000","last_became_peered":"2024-09-10T08:05:35.532706+0000","last_unstale":"2024-09-10T08:07:05.569486+0000","last_undegraded":"2024-09-10T08:07:05.569486+0000","last_fullsized":"2024-09-10T08:07:05.569486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:22.150563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.655456+0000","last_change":"2024-09-10T08:05:25.478978+0000","last_active":"2024-09-10T08:08:03.655456+0000","last_peered":"2024-09-10T08:08:03.655456+0000","last_clean":"2024-09-10T08:08:03.655456+0000","last_became_active":"2024-09-10T08:05:25.478649+0000","last_became_peered":"2024-09-10T08:05:25.478649+0000","last_unstale":"2024-09-10T08:08:03.655456+0000","last_undegraded":"2024-09-10T08:08:03.655456+0000","last_fullsized":"2024-09-10T08:08:03.655456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:17.493446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'40","reported_seq":361,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:25.453422+0000","last_change":"2024-09-10T08:06:04.740972+0000","last_active":"2024-09-10T08:08:25.453422+0000","last_peered":"2024-09-10T08:08:25.453422+0000","last_clean":"2024-09-10T08:08:25.453422+0000","last_became_active":"2024-09-10T08:06:04.731200+0000","last_became_peered":"2024-09-10T08:06:04.731200+0000","last_unstale":"2024-09-10T08:08:25.453422+0000","last_undegraded":"2024-09-10T08:08:25.453422+0000","last_fullsized":"2024-09-10T08:08:25.453422+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:17.169708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71507,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":261,"num_read_kb":201,"num_write":41,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3703,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512781+0000","last_change":"2024-09-10T08:05:35.532594+0000","last_active":"2024-09-10T08:07:05.512781+0000","last_peered":"2024-09-10T08:07:05.512781+0000","last_clean":"2024-09-10T08:07:05.512781+0000","last_became_active":"2024-09-10T08:05:35.532132+0000","last_became_peered":"2024-09-10T08:05:35.532132+0000","last_unstale":"2024-09-10T08:07:05.512781+0000","last_undegraded":"2024-09-10T08:07:05.512781+0000","last_fullsized":"2024-09-10T08:07:05.512781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:15.348648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.666577+0000","last_change":"2024-09-10T08:05:25.485297+0000","last_active":"2024-09-10T08:08:03.666577+0000","last_peered":"2024-09-10T08:08:03.666577+0000","last_clean":"2024-09-10T08:08:03.666577+0000","last_became_active":"2024-09-10T08:05:25.484883+0000","last_became_peered":"2024-09-10T08:05:25.484883+0000","last_unstale":"2024-09-10T08:08:03.666577+0000","last_undegraded":"2024-09-10T08:08:03.666577+0000","last_fullsized":"2024-09-10T08:08:03.666577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:37.052435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"93'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569713+0000","last_change":"2024-09-10T08:05:22.568993+0000","last_active":"2024-09-10T08:07:05.569713+0000","last_peered":"2024-09-10T08:07:05.569713+0000","last_clean":"2024-09-10T08:07:05.569713+0000","last_became_active":"2024-09-10T08:05:22.568564+0000","last_became_peered":"2024-09-10T08:05:22.568564+0000","last_unstale":"2024-09-10T08:07:05.569713+0000","last_undegraded":"2024-09-10T08:07:05.569713+0000","last_fullsized":"2024-09-10T08:07:05.569713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:23.767174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914556+0000","last_change":"2024-09-10T08:05:35.524025+0000","last_active":"2024-09-10T08:07:05.914556+0000","last_peered":"2024-09-10T08:07:05.914556+0000","last_clean":"2024-09-10T08:07:05.914556+0000","last_became_active":"2024-09-10T08:05:35.523628+0000","last_became_peered":"2024-09-10T08:05:35.523628+0000","last_unstale":"2024-09-10T08:07:05.914556+0000","last_undegraded":"2024-09-10T08:07:05.914556+0000","last_fullsized":"2024-09-10T08:07:05.914556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:48.593779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.680405+0000","last_change":"2024-09-10T08:07:05.570537+0000","last_active":"2024-09-10T08:08:03.680405+0000","last_peered":"2024-09-10T08:08:03.680405+0000","last_clean":"2024-09-10T08:08:03.680405+0000","last_became_active":"2024-09-10T08:07:05.570278+0000","last_became_peered":"2024-09-10T08:07:05.570278+0000","last_unstale":"2024-09-10T08:08:03.680405+0000","last_undegraded":"2024-09-10T08:08:03.680405+0000","last_fullsized":"2024-09-10T08:08:03.680405+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:56.490462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509635+0000","last_change":"2024-09-10T08:05:22.460487+0000","last_active":"2024-09-10T08:07:05.509635+0000","last_peered":"2024-09-10T08:07:05.509635+0000","last_clean":"2024-09-10T08:07:05.509635+0000","last_became_active":"2024-09-10T08:05:22.460158+0000","last_became_peered":"2024-09-10T08:05:22.460158+0000","last_unstale":"2024-09-10T08:07:05.509635+0000","last_undegraded":"2024-09-10T08:07:05.509635+0000","last_fullsized":"2024-09-10T08:07:05.509635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:53.285513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512495+0000","last_change":"2024-09-10T08:05:35.531399+0000","last_active":"2024-09-10T08:07:05.512495+0000","last_peered":"2024-09-10T08:07:05.512495+0000","last_clean":"2024-09-10T08:07:05.512495+0000","last_became_active":"2024-09-10T08:05:35.530320+0000","last_became_peered":"2024-09-10T08:05:35.530320+0000","last_unstale":"2024-09-10T08:07:05.512495+0000","last_undegraded":"2024-09-10T08:07:05.512495+0000","last_fullsized":"2024-09-10T08:07:05.512495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:19.814046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669147+0000","last_change":"2024-09-10T08:05:25.488208+0000","last_active":"2024-09-10T08:08:03.669147+0000","last_peered":"2024-09-10T08:08:03.669147+0000","last_clean":"2024-09-10T08:08:03.669147+0000","last_became_active":"2024-09-10T08:05:25.487893+0000","last_became_peered":"2024-09-10T08:05:25.487893+0000","last_unstale":"2024-09-10T08:08:03.669147+0000","last_undegraded":"2024-09-10T08:08:03.669147+0000","last_fullsized":"2024-09-10T08:08:03.669147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:10.999416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":173,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:24.452115+0000","last_change":"2024-09-10T08:05:22.461321+0000","last_active":"2024-09-10T08:08:24.452115+0000","last_peered":"2024-09-10T08:08:24.452115+0000","last_clean":"2024-09-10T08:08:24.452115+0000","last_became_active":"2024-09-10T08:05:22.460950+0000","last_became_peered":"2024-09-10T08:05:22.460950+0000","last_unstale":"2024-09-10T08:08:24.452115+0000","last_undegraded":"2024-09-10T08:08:24.452115+0000","last_fullsized":"2024-09-10T08:08:24.452115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:55.647273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876534+0000","last_change":"2024-09-10T08:05:35.530578+0000","last_active":"2024-09-10T08:07:05.876534+0000","last_peered":"2024-09-10T08:07:05.876534+0000","last_clean":"2024-09-10T08:07:05.876534+0000","last_became_active":"2024-09-10T08:05:35.530162+0000","last_became_peered":"2024-09-10T08:05:35.530162+0000","last_unstale":"2024-09-10T08:07:05.876534+0000","last_undegraded":"2024-09-10T08:07:05.876534+0000","last_fullsized":"2024-09-10T08:07:05.876534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:19.290973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513617+0000","last_change":"2024-09-10T08:05:35.532685+0000","last_active":"2024-09-10T08:07:05.513617+0000","last_peered":"2024-09-10T08:07:05.513617+0000","last_clean":"2024-09-10T08:07:05.513617+0000","last_became_active":"2024-09-10T08:05:35.532257+0000","last_became_peered":"2024-09-10T08:05:35.532257+0000","last_unstale":"2024-09-10T08:07:05.513617+0000","last_undegraded":"2024-09-10T08:07:05.513617+0000","last_fullsized":"2024-09-10T08:07:05.513617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:07.412435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.662135+0000","last_change":"2024-09-10T08:05:25.494472+0000","last_active":"2024-09-10T08:08:03.662135+0000","last_peered":"2024-09-10T08:08:03.662135+0000","last_clean":"2024-09-10T08:08:03.662135+0000","last_became_active":"2024-09-10T08:05:25.493928+0000","last_became_peered":"2024-09-10T08:05:25.493928+0000","last_unstale":"2024-09-10T08:08:03.662135+0000","last_undegraded":"2024-09-10T08:08:03.662135+0000","last_fullsized":"2024-09-10T08:08:03.662135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:00.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"93'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568928+0000","last_change":"2024-09-10T08:05:22.460585+0000","last_active":"2024-09-10T08:07:05.568928+0000","last_peered":"2024-09-10T08:07:05.568928+0000","last_clean":"2024-09-10T08:07:05.568928+0000","last_became_active":"2024-09-10T08:05:22.460259+0000","last_became_peered":"2024-09-10T08:05:22.460259+0000","last_unstale":"2024-09-10T08:07:05.568928+0000","last_undegraded":"2024-09-10T08:07:05.568928+0000","last_fullsized":"2024-09-10T08:07:05.568928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:45.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770275+0000","last_change":"2024-09-10T08:05:35.524000+0000","last_active":"2024-09-10T08:07:05.770275+0000","last_peered":"2024-09-10T08:07:05.770275+0000","last_clean":"2024-09-10T08:07:05.770275+0000","last_became_active":"2024-09-10T08:05:35.523647+0000","last_became_peered":"2024-09-10T08:05:35.523647+0000","last_unstale":"2024-09-10T08:07:05.770275+0000","last_undegraded":"2024-09-10T08:07:05.770275+0000","last_fullsized":"2024-09-10T08:07:05.770275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:10:54.664865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660563+0000","last_change":"2024-09-10T08:05:25.499701+0000","last_active":"2024-09-10T08:08:03.660563+0000","last_peered":"2024-09-10T08:08:03.660563+0000","last_clean":"2024-09-10T08:08:03.660563+0000","last_became_active":"2024-09-10T08:05:25.496890+0000","last_became_peered":"2024-09-10T08:05:25.496890+0000","last_unstale":"2024-09-10T08:08:03.660563+0000","last_undegraded":"2024-09-10T08:08:03.660563+0000","last_fullsized":"2024-09-10T08:08:03.660563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:58.166365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914070+0000","last_change":"2024-09-10T08:05:22.460880+0000","last_active":"2024-09-10T08:07:05.914070+0000","last_peered":"2024-09-10T08:07:05.914070+0000","last_clean":"2024-09-10T08:07:05.914070+0000","last_became_active":"2024-09-10T08:05:22.459932+0000","last_became_peered":"2024-09-10T08:05:22.459932+0000","last_unstale":"2024-09-10T08:07:05.914070+0000","last_undegraded":"2024-09-10T08:07:05.914070+0000","last_fullsized":"2024-09-10T08:07:05.914070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:55.826526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915405+0000","last_change":"2024-09-10T08:05:22.460578+0000","last_active":"2024-09-10T08:07:05.915405+0000","last_peered":"2024-09-10T08:07:05.915405+0000","last_clean":"2024-09-10T08:07:05.915405+0000","last_became_active":"2024-09-10T08:05:22.460239+0000","last_became_peered":"2024-09-10T08:05:22.460239+0000","last_unstale":"2024-09-10T08:07:05.915405+0000","last_undegraded":"2024-09-10T08:07:05.915405+0000","last_fullsized":"2024-09-10T08:07:05.915405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:07.606243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.661142+0000","last_change":"2024-09-10T08:05:25.489846+0000","last_active":"2024-09-10T08:08:03.661142+0000","last_peered":"2024-09-10T08:08:03.661142+0000","last_clean":"2024-09-10T08:08:03.661142+0000","last_became_active":"2024-09-10T08:05:25.489326+0000","last_became_peered":"2024-09-10T08:05:25.489326+0000","last_unstale":"2024-09-10T08:08:03.661142+0000","last_undegraded":"2024-09-10T08:08:03.661142+0000","last_fullsized":"2024-09-10T08:08:03.661142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:43.458468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511635+0000","last_change":"2024-09-10T08:05:35.530522+0000","last_active":"2024-09-10T08:07:05.511635+0000","last_peered":"2024-09-10T08:07:05.511635+0000","last_clean":"2024-09-10T08:07:05.511635+0000","last_became_active":"2024-09-10T08:05:35.530099+0000","last_became_peered":"2024-09-10T08:05:35.530099+0000","last_unstale":"2024-09-10T08:07:05.511635+0000","last_undegraded":"2024-09-10T08:07:05.511635+0000","last_fullsized":"2024-09-10T08:07:05.511635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:58.469119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":184,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:25.454135+0000","last_change":"2024-09-10T08:05:22.461391+0000","last_active":"2024-09-10T08:08:25.454135+0000","last_peered":"2024-09-10T08:08:25.454135+0000","last_clean":"2024-09-10T08:08:25.454135+0000","last_became_active":"2024-09-10T08:05:22.461042+0000","last_became_peered":"2024-09-10T08:05:22.461042+0000","last_unstale":"2024-09-10T08:08:25.454135+0000","last_undegraded":"2024-09-10T08:08:25.454135+0000","last_fullsized":"2024-09-10T08:08:25.454135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:12.635710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1860,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.649481+0000","last_change":"2024-09-10T08:05:25.487378+0000","last_active":"2024-09-10T08:08:03.649481+0000","last_peered":"2024-09-10T08:08:03.649481+0000","last_clean":"2024-09-10T08:08:03.649481+0000","last_became_active":"2024-09-10T08:05:25.487055+0000","last_became_peered":"2024-09-10T08:05:25.487055+0000","last_unstale":"2024-09-10T08:08:03.649481+0000","last_undegraded":"2024-09-10T08:08:03.649481+0000","last_fullsized":"2024-09-10T08:08:03.649481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:59.522014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511748+0000","last_change":"2024-09-10T08:05:35.522765+0000","last_active":"2024-09-10T08:07:05.511748+0000","last_peered":"2024-09-10T08:07:05.511748+0000","last_clean":"2024-09-10T08:07:05.511748+0000","last_became_active":"2024-09-10T08:05:35.522359+0000","last_became_peered":"2024-09-10T08:05:35.522359+0000","last_unstale":"2024-09-10T08:07:05.511748+0000","last_undegraded":"2024-09-10T08:07:05.511748+0000","last_fullsized":"2024-09-10T08:07:05.511748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:15.280867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.652869+0000","last_change":"2024-09-10T08:05:25.475388+0000","last_active":"2024-09-10T08:08:03.652869+0000","last_peered":"2024-09-10T08:08:03.652869+0000","last_clean":"2024-09-10T08:08:03.652869+0000","last_became_active":"2024-09-10T08:05:25.474331+0000","last_became_peered":"2024-09-10T08:05:25.474331+0000","last_unstale":"2024-09-10T08:08:03.652869+0000","last_undegraded":"2024-09-10T08:08:03.652869+0000","last_fullsized":"2024-09-10T08:08:03.652869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:58.713148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656383+0000","last_change":"2024-09-10T08:07:05.570505+0000","last_active":"2024-09-10T08:08:03.656383+0000","last_peered":"2024-09-10T08:08:03.656383+0000","last_clean":"2024-09-10T08:08:03.656383+0000","last_became_active":"2024-09-10T08:07:05.569970+0000","last_became_peered":"2024-09-10T08:07:05.569970+0000","last_unstale":"2024-09-10T08:08:03.656383+0000","last_undegraded":"2024-09-10T08:08:03.656383+0000","last_fullsized":"2024-09-10T08:08:03.656383+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:13.282503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671951+0000","last_change":"2024-09-10T08:05:25.491585+0000","last_active":"2024-09-10T08:08:03.671951+0000","last_peered":"2024-09-10T08:08:03.671951+0000","last_clean":"2024-09-10T08:08:03.671951+0000","last_became_active":"2024-09-10T08:05:25.491224+0000","last_became_peered":"2024-09-10T08:05:25.491224+0000","last_unstale":"2024-09-10T08:08:03.671951+0000","last_undegraded":"2024-09-10T08:08:03.671951+0000","last_fullsized":"2024-09-10T08:08:03.671951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:56.066808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681502+0000","last_change":"2024-09-10T08:05:25.499578+0000","last_active":"2024-09-10T08:08:03.681502+0000","last_peered":"2024-09-10T08:08:03.681502+0000","last_clean":"2024-09-10T08:08:03.681502+0000","last_became_active":"2024-09-10T08:05:25.498445+0000","last_became_peered":"2024-09-10T08:05:25.498445+0000","last_unstale":"2024-09-10T08:08:03.681502+0000","last_undegraded":"2024-09-10T08:08:03.681502+0000","last_fullsized":"2024-09-10T08:08:03.681502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:58.961699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675717+0000","last_change":"2024-09-10T08:06:04.735764+0000","last_active":"2024-09-10T08:08:03.675717+0000","last_peered":"2024-09-10T08:08:03.675717+0000","last_clean":"2024-09-10T08:08:03.675717+0000","last_became_active":"2024-09-10T08:06:04.735366+0000","last_became_peered":"2024-09-10T08:06:04.735366+0000","last_unstale":"2024-09-10T08:08:03.675717+0000","last_undegraded":"2024-09-10T08:08:03.675717+0000","last_fullsized":"2024-09-10T08:08:03.675717+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:03.757890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651587+0000","last_change":"2024-09-10T08:05:25.475872+0000","last_active":"2024-09-10T08:08:03.651587+0000","last_peered":"2024-09-10T08:08:03.651587+0000","last_clean":"2024-09-10T08:08:03.651587+0000","last_became_active":"2024-09-10T08:05:25.475549+0000","last_became_peered":"2024-09-10T08:05:25.475549+0000","last_unstale":"2024-09-10T08:08:03.651587+0000","last_undegraded":"2024-09-10T08:08:03.651587+0000","last_fullsized":"2024-09-10T08:08:03.651587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:37.575267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.664667+0000","last_change":"2024-09-10T08:05:25.474210+0000","last_active":"2024-09-10T08:08:03.664667+0000","last_peered":"2024-09-10T08:08:03.664667+0000","last_clean":"2024-09-10T08:08:03.664667+0000","last_became_active":"2024-09-10T08:05:25.473882+0000","last_became_peered":"2024-09-10T08:05:25.473882+0000","last_unstale":"2024-09-10T08:08:03.664667+0000","last_undegraded":"2024-09-10T08:08:03.664667+0000","last_fullsized":"2024-09-10T08:08:03.664667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:05.164370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651519+0000","last_change":"2024-09-10T08:05:25.490406+0000","last_active":"2024-09-10T08:08:03.651519+0000","last_peered":"2024-09-10T08:08:03.651519+0000","last_clean":"2024-09-10T08:08:03.651519+0000","last_became_active":"2024-09-10T08:05:25.490061+0000","last_became_peered":"2024-09-10T08:05:25.490061+0000","last_unstale":"2024-09-10T08:08:03.651519+0000","last_undegraded":"2024-09-10T08:08:03.651519+0000","last_fullsized":"2024-09-10T08:08:03.651519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:12.988229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.664045+0000","last_change":"2024-09-10T08:05:25.504049+0000","last_active":"2024-09-10T08:08:03.664045+0000","last_peered":"2024-09-10T08:08:03.664045+0000","last_clean":"2024-09-10T08:08:03.664045+0000","last_became_active":"2024-09-10T08:05:25.503770+0000","last_became_peered":"2024-09-10T08:05:25.503770+0000","last_unstale":"2024-09-10T08:08:03.664045+0000","last_undegraded":"2024-09-10T08:08:03.664045+0000","last_fullsized":"2024-09-10T08:08:03.664045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:52.850945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.672614+0000","last_change":"2024-09-10T08:05:25.491682+0000","last_active":"2024-09-10T08:08:03.672614+0000","last_peered":"2024-09-10T08:08:03.672614+0000","last_clean":"2024-09-10T08:08:03.672614+0000","last_became_active":"2024-09-10T08:05:25.491296+0000","last_became_peered":"2024-09-10T08:05:25.491296+0000","last_unstale":"2024-09-10T08:08:03.672614+0000","last_undegraded":"2024-09-10T08:08:03.672614+0000","last_fullsized":"2024-09-10T08:08:03.672614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:30.075842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.673509+0000","last_change":"2024-09-10T08:05:25.489648+0000","last_active":"2024-09-10T08:08:03.673509+0000","last_peered":"2024-09-10T08:08:03.673509+0000","last_clean":"2024-09-10T08:08:03.673509+0000","last_became_active":"2024-09-10T08:05:25.489267+0000","last_became_peered":"2024-09-10T08:05:25.489267+0000","last_unstale":"2024-09-10T08:08:03.673509+0000","last_undegraded":"2024-09-10T08:08:03.673509+0000","last_fullsized":"2024-09-10T08:08:03.673509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:21.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681690+0000","last_change":"2024-09-10T08:05:25.497862+0000","last_active":"2024-09-10T08:08:03.681690+0000","last_peered":"2024-09-10T08:08:03.681690+0000","last_clean":"2024-09-10T08:08:03.681690+0000","last_became_active":"2024-09-10T08:05:25.497504+0000","last_became_peered":"2024-09-10T08:05:25.497504+0000","last_unstale":"2024-09-10T08:08:03.681690+0000","last_undegraded":"2024-09-10T08:08:03.681690+0000","last_fullsized":"2024-09-10T08:08:03.681690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:40.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654484+0000","last_change":"2024-09-10T08:05:25.485924+0000","last_active":"2024-09-10T08:08:03.654484+0000","last_peered":"2024-09-10T08:08:03.654484+0000","last_clean":"2024-09-10T08:08:03.654484+0000","last_became_active":"2024-09-10T08:05:25.485545+0000","last_became_peered":"2024-09-10T08:05:25.485545+0000","last_unstale":"2024-09-10T08:08:03.654484+0000","last_undegraded":"2024-09-10T08:08:03.654484+0000","last_fullsized":"2024-09-10T08:08:03.654484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:33:20.233720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671966+0000","last_change":"2024-09-10T08:05:25.491911+0000","last_active":"2024-09-10T08:08:03.671966+0000","last_peered":"2024-09-10T08:08:03.671966+0000","last_clean":"2024-09-10T08:08:03.671966+0000","last_became_active":"2024-09-10T08:05:25.491330+0000","last_became_peered":"2024-09-10T08:05:25.491330+0000","last_unstale":"2024-09-10T08:08:03.671966+0000","last_undegraded":"2024-09-10T08:08:03.671966+0000","last_fullsized":"2024-09-10T08:08:03.671966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:06.043331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.655352+0000","last_change":"2024-09-10T08:05:25.492220+0000","last_active":"2024-09-10T08:08:03.655352+0000","last_peered":"2024-09-10T08:08:03.655352+0000","last_clean":"2024-09-10T08:08:03.655352+0000","last_became_active":"2024-09-10T08:05:25.491473+0000","last_became_peered":"2024-09-10T08:05:25.491473+0000","last_unstale":"2024-09-10T08:08:03.655352+0000","last_undegraded":"2024-09-10T08:08:03.655352+0000","last_fullsized":"2024-09-10T08:08:03.655352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:13:30.278434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676184+0000","last_change":"2024-09-10T08:05:25.488927+0000","last_active":"2024-09-10T08:08:03.676184+0000","last_peered":"2024-09-10T08:08:03.676184+0000","last_clean":"2024-09-10T08:08:03.676184+0000","last_became_active":"2024-09-10T08:05:25.488550+0000","last_became_peered":"2024-09-10T08:05:25.488550+0000","last_unstale":"2024-09-10T08:08:03.676184+0000","last_undegraded":"2024-09-10T08:08:03.676184+0000","last_fullsized":"2024-09-10T08:08:03.676184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:44.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.677654+0000","last_change":"2024-09-10T08:05:25.490125+0000","last_active":"2024-09-10T08:08:03.677654+0000","last_peered":"2024-09-10T08:08:03.677654+0000","last_clean":"2024-09-10T08:08:03.677654+0000","last_became_active":"2024-09-10T08:05:25.487805+0000","last_became_peered":"2024-09-10T08:05:25.487805+0000","last_unstale":"2024-09-10T08:08:03.677654+0000","last_undegraded":"2024-09-10T08:08:03.677654+0000","last_fullsized":"2024-09-10T08:08:03.677654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:33.479009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676794+0000","last_change":"2024-09-10T08:05:25.500346+0000","last_active":"2024-09-10T08:08:03.676794+0000","last_peered":"2024-09-10T08:08:03.676794+0000","last_clean":"2024-09-10T08:08:03.676794+0000","last_became_active":"2024-09-10T08:05:25.498289+0000","last_became_peered":"2024-09-10T08:05:25.498289+0000","last_unstale":"2024-09-10T08:08:03.676794+0000","last_undegraded":"2024-09-10T08:08:03.676794+0000","last_fullsized":"2024-09-10T08:08:03.676794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:12.054926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676196+0000","last_change":"2024-09-10T08:05:25.490001+0000","last_active":"2024-09-10T08:08:03.676196+0000","last_peered":"2024-09-10T08:08:03.676196+0000","last_clean":"2024-09-10T08:08:03.676196+0000","last_became_active":"2024-09-10T08:05:25.489706+0000","last_became_peered":"2024-09-10T08:05:25.489706+0000","last_unstale":"2024-09-10T08:08:03.676196+0000","last_undegraded":"2024-09-10T08:08:03.676196+0000","last_fullsized":"2024-09-10T08:08:03.676196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:38.822140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669334+0000","last_change":"2024-09-10T08:05:25.493874+0000","last_active":"2024-09-10T08:08:03.669334+0000","last_peered":"2024-09-10T08:08:03.669334+0000","last_clean":"2024-09-10T08:08:03.669334+0000","last_became_active":"2024-09-10T08:05:25.489911+0000","last_became_peered":"2024-09-10T08:05:25.489911+0000","last_unstale":"2024-09-10T08:08:03.669334+0000","last_undegraded":"2024-09-10T08:08:03.669334+0000","last_fullsized":"2024-09-10T08:08:03.669334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:46.509657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654960+0000","last_change":"2024-09-10T08:05:25.490277+0000","last_active":"2024-09-10T08:08:03.654960+0000","last_peered":"2024-09-10T08:08:03.654960+0000","last_clean":"2024-09-10T08:08:03.654960+0000","last_became_active":"2024-09-10T08:05:25.486863+0000","last_became_peered":"2024-09-10T08:05:25.486863+0000","last_unstale":"2024-09-10T08:08:03.654960+0000","last_undegraded":"2024-09-10T08:08:03.654960+0000","last_fullsized":"2024-09-10T08:08:03.654960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:21.353830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656794+0000","last_change":"2024-09-10T08:05:25.491028+0000","last_active":"2024-09-10T08:08:03.656794+0000","last_peered":"2024-09-10T08:08:03.656794+0000","last_clean":"2024-09-10T08:08:03.656794+0000","last_became_active":"2024-09-10T08:05:25.490452+0000","last_became_peered":"2024-09-10T08:05:25.490452+0000","last_unstale":"2024-09-10T08:08:03.656794+0000","last_undegraded":"2024-09-10T08:08:03.656794+0000","last_fullsized":"2024-09-10T08:08:03.656794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:53.059077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660628+0000","last_change":"2024-09-10T08:05:25.489991+0000","last_active":"2024-09-10T08:08:03.660628+0000","last_peered":"2024-09-10T08:08:03.660628+0000","last_clean":"2024-09-10T08:08:03.660628+0000","last_became_active":"2024-09-10T08:05:25.489662+0000","last_became_peered":"2024-09-10T08:05:25.489662+0000","last_unstale":"2024-09-10T08:08:03.660628+0000","last_undegraded":"2024-09-10T08:08:03.660628+0000","last_fullsized":"2024-09-10T08:08:03.660628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:47.940712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667243+0000","last_change":"2024-09-10T08:05:25.475752+0000","last_active":"2024-09-10T08:08:03.667243+0000","last_peered":"2024-09-10T08:08:03.667243+0000","last_clean":"2024-09-10T08:08:03.667243+0000","last_became_active":"2024-09-10T08:05:25.475395+0000","last_became_peered":"2024-09-10T08:05:25.475395+0000","last_unstale":"2024-09-10T08:08:03.667243+0000","last_undegraded":"2024-09-10T08:08:03.667243+0000","last_fullsized":"2024-09-10T08:08:03.667243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.240154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.672245+0000","last_change":"2024-09-10T08:05:25.503061+0000","last_active":"2024-09-10T08:08:03.672245+0000","last_peered":"2024-09-10T08:08:03.672245+0000","last_clean":"2024-09-10T08:08:03.672245+0000","last_became_active":"2024-09-10T08:05:25.501333+0000","last_became_peered":"2024-09-10T08:05:25.501333+0000","last_unstale":"2024-09-10T08:08:03.672245+0000","last_undegraded":"2024-09-10T08:08:03.672245+0000","last_fullsized":"2024-09-10T08:08:03.672245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:38.254777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.661256+0000","last_change":"2024-09-10T08:07:05.514631+0000","last_active":"2024-09-10T08:08:03.661256+0000","last_peered":"2024-09-10T08:08:03.661256+0000","last_clean":"2024-09-10T08:08:03.661256+0000","last_became_active":"2024-09-10T08:07:05.514285+0000","last_became_peered":"2024-09-10T08:07:05.514285+0000","last_unstale":"2024-09-10T08:08:03.661256+0000","last_undegraded":"2024-09-10T08:08:03.661256+0000","last_fullsized":"2024-09-10T08:08:03.661256+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:40.293121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660092+0000","last_change":"2024-09-10T08:05:25.491457+0000","last_active":"2024-09-10T08:08:03.660092+0000","last_peered":"2024-09-10T08:08:03.660092+0000","last_clean":"2024-09-10T08:08:03.660092+0000","last_became_active":"2024-09-10T08:05:25.491162+0000","last_became_peered":"2024-09-10T08:05:25.491162+0000","last_unstale":"2024-09-10T08:08:03.660092+0000","last_undegraded":"2024-09-10T08:08:03.660092+0000","last_fullsized":"2024-09-10T08:08:03.660092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:14.389752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660737+0000","last_change":"2024-09-10T08:05:25.487874+0000","last_active":"2024-09-10T08:08:03.660737+0000","last_peered":"2024-09-10T08:08:03.660737+0000","last_clean":"2024-09-10T08:08:03.660737+0000","last_became_active":"2024-09-10T08:05:25.487497+0000","last_became_peered":"2024-09-10T08:05:25.487497+0000","last_unstale":"2024-09-10T08:08:03.660737+0000","last_undegraded":"2024-09-10T08:08:03.660737+0000","last_fullsized":"2024-09-10T08:08:03.660737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:26.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658875+0000","last_change":"2024-09-10T08:05:25.490018+0000","last_active":"2024-09-10T08:08:03.658875+0000","last_peered":"2024-09-10T08:08:03.658875+0000","last_clean":"2024-09-10T08:08:03.658875+0000","last_became_active":"2024-09-10T08:05:25.484951+0000","last_became_peered":"2024-09-10T08:05:25.484951+0000","last_unstale":"2024-09-10T08:08:03.658875+0000","last_undegraded":"2024-09-10T08:08:03.658875+0000","last_fullsized":"2024-09-10T08:08:03.658875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:07.516154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.677343+0000","last_change":"2024-09-10T08:05:25.501581+0000","last_active":"2024-09-10T08:08:03.677343+0000","last_peered":"2024-09-10T08:08:03.677343+0000","last_clean":"2024-09-10T08:08:03.677343+0000","last_became_active":"2024-09-10T08:05:25.499690+0000","last_became_peered":"2024-09-10T08:05:25.499690+0000","last_unstale":"2024-09-10T08:08:03.677343+0000","last_undegraded":"2024-09-10T08:08:03.677343+0000","last_fullsized":"2024-09-10T08:08:03.677343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:14.569791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.678243+0000","last_change":"2024-09-10T08:05:25.487945+0000","last_active":"2024-09-10T08:08:03.678243+0000","last_peered":"2024-09-10T08:08:03.678243+0000","last_clean":"2024-09-10T08:08:03.678243+0000","last_became_active":"2024-09-10T08:05:25.487655+0000","last_became_peered":"2024-09-10T08:05:25.487655+0000","last_unstale":"2024-09-10T08:08:03.678243+0000","last_undegraded":"2024-09-10T08:08:03.678243+0000","last_fullsized":"2024-09-10T08:08:03.678243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:37.746037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675926+0000","last_change":"2024-09-10T08:05:25.489348+0000","last_active":"2024-09-10T08:08:03.675926+0000","last_peered":"2024-09-10T08:08:03.675926+0000","last_clean":"2024-09-10T08:08:03.675926+0000","last_became_active":"2024-09-10T08:05:25.489004+0000","last_became_peered":"2024-09-10T08:05:25.489004+0000","last_unstale":"2024-09-10T08:08:03.675926+0000","last_undegraded":"2024-09-10T08:08:03.675926+0000","last_fullsized":"2024-09-10T08:08:03.675926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:54.554719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76748,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":784,"num_read_kb":474,"num_write":190,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3725,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":230244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214976,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514547,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":548036,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743480,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32364,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712788,"statfs":{"total":95995035648,"available":95961894912,"internally_reserved":0,"allocated":1208320,"data_stored":1211278,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907004,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":548250,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135939,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":548148,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364870,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1114112,"data_stored":1145319,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528397,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":543426,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790036,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1122304,"data_stored":1140514,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953560,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":619976,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182493,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":543620,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":545924,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":614843,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":544814,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":73478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:08:32.876 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:08:32.972 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:33.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.562+0000 7fcabfe00640 1 -- 172.21.15.2:0/3013725101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcac01068d0 msgr2=0x7fcac0106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:33.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.562+0000 7fcabfe00640 1 --2- 172.21.15.2:0/3013725101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcac01068d0 0x7fcac0106cb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009930 tx=0x7fcab002a420 comp rx=0 tx=0).stop 2024-09-10T08:08:33.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.562+0000 7fcabfe00640 1 -- 172.21.15.2:0/3013725101 shutdown_connections 2024-09-10T08:08:33.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.562+0000 7fcabfe00640 1 --2- 172.21.15.2:0/3013725101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0101270 0x7fcac010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.562+0000 7fcabfe00640 1 --2- 172.21.15.2:0/3013725101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcac01008d0 0x7fcac0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.562+0000 7fcabfe00640 1 --2- 172.21.15.2:0/3013725101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcac01068d0 0x7fcac0106cb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.562+0000 7fcabfe00640 1 -- 172.21.15.2:0/3013725101 >> 172.21.15.2:0/3013725101 conn(0x7fcac00fc610 msgr2=0x7fcac00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:33.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.562+0000 7fcabfe00640 1 -- 172.21.15.2:0/3013725101 shutdown_connections 2024-09-10T08:08:33.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.562+0000 7fcabfe00640 1 -- 172.21.15.2:0/3013725101 wait complete. 2024-09-10T08:08:33.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.563+0000 7fcabfe00640 1 Processor -- start 2024-09-10T08:08:33.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.564+0000 7fcabfe00640 1 -- start start 2024-09-10T08:08:33.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.564+0000 7fcabfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcac01008d0 0x7fcac019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.564+0000 7fcabfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcac0101270 0x7fcac019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.564+0000 7fcabfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac01068d0 0x7fcac019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.564+0000 7fcabfe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0110cf0 con 0x7fcac01008d0 2024-09-10T08:08:33.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.564+0000 7fcabfe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0110b70 con 0x7fcac01068d0 2024-09-10T08:08:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.564+0000 7fcabfe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0110e70 con 0x7fcac0101270 2024-09-10T08:08:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.564+0000 7fcabea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcac01008d0 0x7fcac019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.564+0000 7fcabea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcac01008d0 0x7fcac019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39850/0 (socket says 172.21.15.2:39850) 2024-09-10T08:08:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.564+0000 7fcabea00640 1 -- 172.21.15.2:0/1977272689 learned_addr learned my addr 172.21.15.2:0/1977272689 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.564+0000 7fcabf400640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac01068d0 0x7fcac019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.565+0000 7fcabe000640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcac0101270 0x7fcac019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.565+0000 7fcabf400640 1 -- 172.21.15.2:0/1977272689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcac0101270 msgr2=0x7fcac019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.565+0000 7fcabf400640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcac0101270 0x7fcac019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.565+0000 7fcabf400640 1 -- 172.21.15.2:0/1977272689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcac01008d0 msgr2=0x7fcac019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.565+0000 7fcabf400640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcac01008d0 0x7fcac019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.565+0000 7fcabf400640 1 -- 172.21.15.2:0/1977272689 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac019f460 con 0x7fcac01068d0 2024-09-10T08:08:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.565+0000 7fcabea00640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcac01008d0 0x7fcac019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:33.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.565+0000 7fcabf400640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac01068d0 0x7fcac019ed80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009fd0 tx=0x7fcab800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:33.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.566+0000 7fcab7400640 1 -- 172.21.15.2:0/1977272689 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab800ce20 con 0x7fcac01068d0 2024-09-10T08:08:33.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.566+0000 7fcab7400640 1 -- 172.21.15.2:0/1977272689 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab8005020 con 0x7fcac01068d0 2024-09-10T08:08:33.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.566+0000 7fcabfe00640 1 -- 172.21.15.2:0/1977272689 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac019f750 con 0x7fcac01068d0 2024-09-10T08:08:33.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.566+0000 7fcab7400640 1 -- 172.21.15.2:0/1977272689 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8010720 con 0x7fcac01068d0 2024-09-10T08:08:33.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.566+0000 7fcabfe00640 1 -- 172.21.15.2:0/1977272689 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac01a7030 con 0x7fcac01068d0 2024-09-10T08:08:33.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.569+0000 7fcab7400640 1 -- 172.21.15.2:0/1977272689 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fcab8005540 con 0x7fcac01068d0 2024-09-10T08:08:33.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.570+0000 7fcac880f740 1 Processor -- start 2024-09-10T08:08:33.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.570+0000 7fcab7400640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcaa40a2c10 0x7fcaa40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.570+0000 7fcac880f740 1 -- start start 2024-09-10T08:08:33.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.571+0000 7fcab7400640 1 -- 172.21.15.2:0/1977272689 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcab80cfc60 con 0x7fcac01068d0 2024-09-10T08:08:33.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.571+0000 7fcabe000640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcaa40a2c10 0x7fcaa40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.571+0000 7fcac880f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2e9c70 0x558fba2ea050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.571+0000 7fcac880f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2ea590 0x558fba2fd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.571+0000 7fcac880f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2fd740 0x558fba2ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.571+0000 7fcabea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2e9c70 0x558fba2ea050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.571+0000 7fcac880f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558fba2503e0 con 0x558fba2e9c70 2024-09-10T08:08:33.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.571+0000 7fcac880f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558fba1f7d80 con 0x558fba2ea590 2024-09-10T08:08:33.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.571+0000 7fcac880f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558fba24c3a0 con 0x558fba2fd740 2024-09-10T08:08:33.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.571+0000 7fcabea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2e9c70 0x558fba2ea050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39862/0 (socket says 172.21.15.2:39862) 2024-09-10T08:08:33.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.571+0000 7fcabea00640 1 -- 172.21.15.2:0/6092320 learned_addr learned my addr 172.21.15.2:0/6092320 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:33.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.572+0000 7fcabf400640 1 --2- 172.21.15.2:0/6092320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2fd740 0x558fba2ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.572+0000 7fcabf400640 1 -- 172.21.15.2:0/6092320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2ea590 msgr2=0x558fba2fd200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:08:33.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.572+0000 7fcabe000640 1 --2- 172.21.15.2:0/6092320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2ea590 0x558fba2fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.573+0000 7fcabf400640 1 --2- 172.21.15.2:0/6092320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2ea590 0x558fba2fd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.573+0000 7fcabf400640 1 -- 172.21.15.2:0/6092320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2e9c70 msgr2=0x558fba2ea050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:33.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.573+0000 7fcabf400640 1 --2- 172.21.15.2:0/6092320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2e9c70 0x558fba2ea050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.573+0000 7fcabf400640 1 -- 172.21.15.2:0/6092320 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fba216ae0 con 0x558fba2fd740 2024-09-10T08:08:33.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.573+0000 7fcabea00640 1 --2- 172.21.15.2:0/6092320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2e9c70 0x558fba2ea050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:33.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.573+0000 7fcabe000640 1 --2- 172.21.15.2:0/6092320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2ea590 0x558fba2fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:08:33.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.573+0000 7fcabe000640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcaa40a2c10 0x7fcaa40a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcac010dba0 tx=0x7fcaac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:33.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.573+0000 7fcabf400640 1 --2- 172.21.15.2:0/6092320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2fd740 0x558fba2ffb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcab80cd1c0 tx=0x7fcab809c610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b2e7dfe65ea6712 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:33.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.573+0000 7fcaabe00640 1 -- 172.21.15.2:0/6092320 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80b7070 con 0x558fba2fd740 2024-09-10T08:08:33.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.573+0000 7fcaabe00640 1 -- 172.21.15.2:0/6092320 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab80a4b80 con 0x558fba2fd740 2024-09-10T08:08:33.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.573+0000 7fcaabe00640 1 -- 172.21.15.2:0/6092320 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80b4020 con 0x558fba2fd740 2024-09-10T08:08:33.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.574+0000 7fcac880f740 1 -- 172.21.15.2:0/6092320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2fd740 msgr2=0x558fba2ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:33.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.574+0000 7fcac880f740 1 --2- 172.21.15.2:0/6092320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2fd740 0x558fba2ffb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcab80cd1c0 tx=0x7fcab809c610 comp rx=0 tx=0).stop 2024-09-10T08:08:33.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.575+0000 7fcac880f740 1 -- 172.21.15.2:0/6092320 shutdown_connections 2024-09-10T08:08:33.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.575+0000 7fcac880f740 1 --2- 172.21.15.2:0/6092320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2fd740 0x558fba2ffb30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.575+0000 7fcac880f740 1 --2- 172.21.15.2:0/6092320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2ea590 0x558fba2fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.575+0000 7fcac880f740 1 --2- 172.21.15.2:0/6092320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2e9c70 0x558fba2ea050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.575+0000 7fcac880f740 1 -- 172.21.15.2:0/6092320 >> 172.21.15.2:0/6092320 conn(0x558fba2e62e0 msgr2=0x558fba2e66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:33.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.575+0000 7fcac880f740 1 -- 172.21.15.2:0/6092320 shutdown_connections 2024-09-10T08:08:33.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.575+0000 7fcac880f740 1 -- 172.21.15.2:0/6092320 wait complete. 2024-09-10T08:08:33.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.576+0000 7fcac880f740 1 -- start start 2024-09-10T08:08:33.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.576+0000 7fcac880f740 1 Processor -- start 2024-09-10T08:08:33.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.577+0000 7fcac880f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2e9c70 0x558fba2fa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.577+0000 7fcac880f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2ea590 0x558fba2faae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.577+0000 7fcac880f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2fd740 0x558fba2fb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.577+0000 7fcac880f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558fba300350 con 0x558fba2fd740 2024-09-10T08:08:33.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.577+0000 7fcac880f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558fba20b0f0 con 0x558fba2ea590 2024-09-10T08:08:33.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.577+0000 7fcac880f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558fba3004d0 con 0x558fba2e9c70 2024-09-10T08:08:33.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.577+0000 7fcac880f740 10 client.0 _ll_register_callbacks cb 0x558fba1fc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:33.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.577+0000 7fcabea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2e9c70 0x558fba2fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.577+0000 7fcabe000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2ea590 0x558fba2faae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.577+0000 7fcabe000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2ea590 0x558fba2faae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:59482/0 (socket says 172.21.15.2:59482) 2024-09-10T08:08:33.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.577+0000 7fcabf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2fd740 0x558fba2fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.577+0000 7fcabe000640 1 -- 172.21.15.2:0/1371143905 learned_addr learned my addr 172.21.15.2:0/1371143905 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:33.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.578+0000 7fcabf400640 1 -- 172.21.15.2:0/1371143905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2e9c70 msgr2=0x558fba2fa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:33.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.578+0000 7fcabf400640 1 --2- 172.21.15.2:0/1371143905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2e9c70 0x558fba2fa5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.578+0000 7fcabf400640 1 -- 172.21.15.2:0/1371143905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2ea590 msgr2=0x558fba2faae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:33.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.578+0000 7fcabf400640 1 --2- 172.21.15.2:0/1371143905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2ea590 0x558fba2faae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.578+0000 7fcabf400640 1 -- 172.21.15.2:0/1371143905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fb9fc13c0 con 0x558fba2fd740 2024-09-10T08:08:33.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.578+0000 7fcabf400640 1 --2- 172.21.15.2:0/1371143905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2fd740 0x558fba2fb020 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcab800c760 tx=0x7fcab80cd640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8690597d43872ab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:33.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.578+0000 7fcaa8c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:08:33.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.579+0000 7fcaa8c00640 1 -- 172.21.15.2:0/1371143905 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab807e020 con 0x558fba2fd740 2024-09-10T08:08:33.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.579+0000 7fcaa8c00640 1 -- 172.21.15.2:0/1371143905 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcab80a48f0 con 0x558fba2fd740 2024-09-10T08:08:33.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.579+0000 7fcac880f740 1 -- 172.21.15.2:0/1371143905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab80a4b80 con 0x558fba2fd740 2024-09-10T08:08:33.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.579+0000 7fcaa8c00640 1 -- 172.21.15.2:0/1371143905 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcab80b4070 con 0x558fba2fd740 2024-09-10T08:08:33.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.579+0000 7fcac880f740 10 client.15237 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:33.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.579+0000 7fcac880f740 1 -- 172.21.15.2:0/1371143905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558fba30bf30 con 0x558fba2fd740 2024-09-10T08:08:33.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.579+0000 7fcaa8c00640 1 -- 172.21.15.2:0/1371143905 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab80a4b80 con 0x558fba2fd740 2024-09-10T08:08:33.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.580+0000 7fcaa8c00640 1 -- 172.21.15.2:0/1371143905 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcab809b360 con 0x558fba2fd740 2024-09-10T08:08:33.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.580+0000 7fcac880f740 10 client.15237 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:33.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.580+0000 7fcac880f740 10 client.15237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:08:33.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.580+0000 7fcac880f740 20 client.15237 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:33.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.580+0000 7fcac880f740 1 --2- 172.21.15.2:0/1371143905 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558fba30d280 0x558fba30f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.580+0000 7fcac880f740 4 client.15237 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:08:33.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.580+0000 7fcac880f740 1 -- 172.21.15.2:0/1371143905 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fba25d950 con 0x558fba30d280 2024-09-10T08:08:33.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.580+0000 7fcabea00640 1 --2- 172.21.15.2:0/1371143905 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558fba30d280 0x558fba30f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.581+0000 7fcabea00640 1 --2- 172.21.15.2:0/1371143905 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558fba30d280 0x558fba30f670 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1491d55c9257e2f server_cookie=bdd3f24754416f13 in_seq=0 out_seq=0 2024-09-10T08:08:33.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.581+0000 7fcaa8c00640 10 client.15237 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:33.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcaa8c00640 1 -- 172.21.15.2:0/1371143905 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fba25d950 con 0x558fba30d280 2024-09-10T08:08:33.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcaa8c00640 10 client.15237 handle_command_reply: tid=0 2024-09-10T08:08:33.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcac880f740 1 client.15237 shutdown 2024-09-10T08:08:33.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcac880f740 1 -- 172.21.15.2:0/1371143905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2fd740 msgr2=0x558fba2fb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:33.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcac880f740 1 --2- 172.21.15.2:0/1371143905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2fd740 0x558fba2fb020 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcab800c760 tx=0x7fcab80cd640 comp rx=0 tx=0).stop 2024-09-10T08:08:33.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcac880f740 1 -- 172.21.15.2:0/1371143905 shutdown_connections 2024-09-10T08:08:33.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcac880f740 1 --2- 172.21.15.2:0/1371143905 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558fba30d280 0x558fba30f670 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcac880f740 1 --2- 172.21.15.2:0/1371143905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2fd740 0x558fba2fb020 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcac880f740 1 --2- 172.21.15.2:0/1371143905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2ea590 0x558fba2faae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcac880f740 1 --2- 172.21.15.2:0/1371143905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2e9c70 0x558fba2fa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcac880f740 1 -- 172.21.15.2:0/1371143905 >> 172.21.15.2:0/1371143905 conn(0x558fba2e62e0 msgr2=0x558fba2ebf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:33.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcaa8c00640 0 client.15237 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:33.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcac880f740 1 -- 172.21.15.2:0/1371143905 shutdown_connections 2024-09-10T08:08:33.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcac880f740 1 -- 172.21.15.2:0/1371143905 wait complete. 2024-09-10T08:08:33.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.582+0000 7fcac880f740 20 client.15237 trim_cache size 0 max 16384 2024-09-10T08:08:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.602+0000 7fcac880f740 1 Processor -- start 2024-09-10T08:08:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.602+0000 7fcac880f740 1 -- start start 2024-09-10T08:08:33.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.602+0000 7fcac880f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2e9c70 0x558fba2e1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.602+0000 7fcac880f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2ea590 0x558fba308e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.602+0000 7fcac880f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2fd740 0x558fba312690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.602+0000 7fcac880f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01af350 con 0x558fba2ea590 2024-09-10T08:08:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.602+0000 7fcac880f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01acc20 con 0x558fba2e9c70 2024-09-10T08:08:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.602+0000 7fcac880f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01aaab0 con 0x558fba2fd740 2024-09-10T08:08:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.602+0000 7fcabe000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2ea590 0x558fba308e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.602+0000 7fcabe000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2ea590 0x558fba308e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39880/0 (socket says 172.21.15.2:39880) 2024-09-10T08:08:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.602+0000 7fcabe000640 1 -- 172.21.15.2:0/719953397 learned_addr learned my addr 172.21.15.2:0/719953397 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.602+0000 7fcabea00640 1 --2- 172.21.15.2:0/719953397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2e9c70 0x558fba2e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.603+0000 7fcabf400640 1 --2- 172.21.15.2:0/719953397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2fd740 0x558fba312690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.603+0000 7fcabea00640 1 -- 172.21.15.2:0/719953397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2fd740 msgr2=0x558fba312690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.603+0000 7fcabea00640 1 --2- 172.21.15.2:0/719953397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2fd740 0x558fba312690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.603+0000 7fcabea00640 1 -- 172.21.15.2:0/719953397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2ea590 msgr2=0x558fba308e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.603+0000 7fcabea00640 1 --2- 172.21.15.2:0/719953397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2ea590 0x558fba308e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.603+0000 7fcabea00640 1 -- 172.21.15.2:0/719953397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fba2f9550 con 0x558fba2e9c70 2024-09-10T08:08:33.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.603+0000 7fcabe000640 1 --2- 172.21.15.2:0/719953397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2ea590 0x558fba308e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:33.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.603+0000 7fcabf400640 1 --2- 172.21.15.2:0/719953397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2fd740 0x558fba312690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:08:33.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.603+0000 7fcabea00640 1 --2- 172.21.15.2:0/719953397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2e9c70 0x558fba2e1060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x558fba20b240 tx=0x7fcab0031510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c66373603cf0394 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:33.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.604+0000 7fcaab400640 1 -- 172.21.15.2:0/719953397 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab004a070 con 0x558fba2e9c70 2024-09-10T08:08:33.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.604+0000 7fcaab400640 1 -- 172.21.15.2:0/719953397 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab0005430 con 0x558fba2e9c70 2024-09-10T08:08:33.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.604+0000 7fcaab400640 1 -- 172.21.15.2:0/719953397 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0035040 con 0x558fba2e9c70 2024-09-10T08:08:33.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.604+0000 7fcac880f740 1 -- 172.21.15.2:0/719953397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2e9c70 msgr2=0x558fba2e1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:33.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.604+0000 7fcac880f740 1 --2- 172.21.15.2:0/719953397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2e9c70 0x558fba2e1060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x558fba20b240 tx=0x7fcab0031510 comp rx=0 tx=0).stop 2024-09-10T08:08:33.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.605+0000 7fcac880f740 1 -- 172.21.15.2:0/719953397 shutdown_connections 2024-09-10T08:08:33.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.605+0000 7fcac880f740 1 --2- 172.21.15.2:0/719953397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2fd740 0x558fba312690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.605+0000 7fcac880f740 1 --2- 172.21.15.2:0/719953397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2ea590 0x558fba308e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.605+0000 7fcac880f740 1 --2- 172.21.15.2:0/719953397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2e9c70 0x558fba2e1060 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.605+0000 7fcac880f740 1 -- 172.21.15.2:0/719953397 >> 172.21.15.2:0/719953397 conn(0x558fba2e62e0 msgr2=0x558fba308760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:33.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.605+0000 7fcac880f740 1 -- 172.21.15.2:0/719953397 shutdown_connections 2024-09-10T08:08:33.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.605+0000 7fcac880f740 1 -- 172.21.15.2:0/719953397 wait complete. 2024-09-10T08:08:33.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.605+0000 7fcac880f740 1 -- start start 2024-09-10T08:08:33.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.605+0000 7fcac880f740 1 Processor -- start 2024-09-10T08:08:33.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.606+0000 7fcac880f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2e9c70 0x558fba30eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.606+0000 7fcac880f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2ea590 0x558fba30f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.606+0000 7fcac880f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2fd740 0x558fba376bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.606+0000 7fcac880f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ab100 con 0x558fba2fd740 2024-09-10T08:08:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.606+0000 7fcac880f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ac550 con 0x558fba2e9c70 2024-09-10T08:08:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.606+0000 7fcac880f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ab7b0 con 0x558fba2ea590 2024-09-10T08:08:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.606+0000 7fcac880f740 10 client.0 _ll_register_callbacks cb 0x7fcab809bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.606+0000 7fcabea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2e9c70 0x558fba30eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.606+0000 7fcabf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2fd740 0x558fba376bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.606+0000 7fcabea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2e9c70 0x558fba30eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:59506/0 (socket says 172.21.15.2:59506) 2024-09-10T08:08:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.606+0000 7fcabe000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2ea590 0x558fba30f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.606+0000 7fcabea00640 1 -- 172.21.15.2:0/1466139662 learned_addr learned my addr 172.21.15.2:0/1466139662 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.607+0000 7fcabea00640 1 -- 172.21.15.2:0/1466139662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2ea590 msgr2=0x558fba30f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.607+0000 7fcabea00640 1 --2- 172.21.15.2:0/1466139662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2ea590 0x558fba30f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.607+0000 7fcabea00640 1 -- 172.21.15.2:0/1466139662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2fd740 msgr2=0x558fba376bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.607+0000 7fcabea00640 1 --2- 172.21.15.2:0/1466139662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2fd740 0x558fba376bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.607+0000 7fcabea00640 1 -- 172.21.15.2:0/1466139662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fba377170 con 0x558fba2e9c70 2024-09-10T08:08:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.607+0000 7fcabe000640 1 --2- 172.21.15.2:0/1466139662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2ea590 0x558fba30f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.607+0000 7fcabea00640 1 --2- 172.21.15.2:0/1466139662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2e9c70 0x558fba30eb80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcab002a940 tx=0x7fcab00315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6f7d8b7978d4400 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.607+0000 7fcaa8c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:08:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.607+0000 7fcaa8c00640 1 -- 172.21.15.2:0/1466139662 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0053060 con 0x558fba2e9c70 2024-09-10T08:08:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.608+0000 7fcaa8c00640 1 -- 172.21.15.2:0/1466139662 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcab0031890 con 0x558fba2e9c70 2024-09-10T08:08:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.608+0000 7fcac880f740 1 -- 172.21.15.2:0/1466139662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab0005430 con 0x558fba2e9c70 2024-09-10T08:08:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.608+0000 7fcaa8c00640 1 -- 172.21.15.2:0/1466139662 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcab005a050 con 0x558fba2e9c70 2024-09-10T08:08:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.608+0000 7fcac880f740 10 client.25265 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.608+0000 7fcac880f740 1 -- 172.21.15.2:0/1466139662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558fba3783b0 con 0x558fba2e9c70 2024-09-10T08:08:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.608+0000 7fcaa8c00640 1 -- 172.21.15.2:0/1466139662 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab0005430 con 0x558fba2e9c70 2024-09-10T08:08:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.609+0000 7fcaa8c00640 1 -- 172.21.15.2:0/1466139662 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcab004d030 con 0x558fba2e9c70 2024-09-10T08:08:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.609+0000 7fcac880f740 10 client.25265 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.609+0000 7fcac880f740 10 client.25265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:08:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.609+0000 7fcac880f740 20 client.25265 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.609+0000 7fcac880f740 1 --2- 172.21.15.2:0/1466139662 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558fba378b00 0x558fba37af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.609+0000 7fcac880f740 4 client.25265 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:08:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.609+0000 7fcac880f740 1 -- 172.21.15.2:0/1466139662 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558fba257e60 con 0x558fba378b00 2024-09-10T08:08:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.609+0000 7fcabe000640 1 --2- 172.21.15.2:0/1466139662 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558fba378b00 0x558fba37af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcabe000640 1 --2- 172.21.15.2:0/1466139662 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558fba378b00 0x558fba37af20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd7aa3a407340f8c server_cookie=62daf7111fbacb8f in_seq=0 out_seq=0 2024-09-10T08:08:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcaa8c00640 10 client.25265 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcaa8c00640 1 -- 172.21.15.2:0/1466139662 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558fba257e60 con 0x558fba378b00 2024-09-10T08:08:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcaa8c00640 10 client.25265 handle_command_reply: tid=0 2024-09-10T08:08:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcac880f740 1 client.25265 shutdown 2024-09-10T08:08:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcac880f740 1 -- 172.21.15.2:0/1466139662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2e9c70 msgr2=0x558fba30eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcac880f740 1 --2- 172.21.15.2:0/1466139662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2e9c70 0x558fba30eb80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcab002a940 tx=0x7fcab00315b0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcac880f740 1 -- 172.21.15.2:0/1466139662 shutdown_connections 2024-09-10T08:08:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcac880f740 1 --2- 172.21.15.2:0/1466139662 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558fba378b00 0x558fba37af20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcac880f740 1 --2- 172.21.15.2:0/1466139662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fba2fd740 0x558fba376bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcac880f740 1 --2- 172.21.15.2:0/1466139662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fba2ea590 0x558fba30f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcaa8c00640 0 client.25265 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcac880f740 1 --2- 172.21.15.2:0/1466139662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fba2e9c70 0x558fba30eb80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.610+0000 7fcac880f740 1 -- 172.21.15.2:0/1466139662 >> 172.21.15.2:0/1466139662 conn(0x558fba2e62e0 msgr2=0x558fba2e5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.611+0000 7fcac880f740 1 -- 172.21.15.2:0/1466139662 shutdown_connections 2024-09-10T08:08:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.611+0000 7fcac880f740 1 -- 172.21.15.2:0/1466139662 wait complete. 2024-09-10T08:08:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.611+0000 7fcac880f740 20 client.25265 trim_cache size 0 max 16384 2024-09-10T08:08:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:08:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:08:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:08:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "401195cc-8ebd-46b5-8355-5a423a6ee37e": { 2024-09-10T08:08:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:08:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "401195cc-8ebd-46b5-8355-5a423a6ee37e", 2024-09-10T08:08:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:08:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:08:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:08:33.614 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:08:33.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.612+0000 7fcabfe00640 1 -- 172.21.15.2:0/1977272689 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcaa40a2c10 msgr2=0x7fcaa40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:33.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.612+0000 7fcabfe00640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcaa40a2c10 0x7fcaa40a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcac010dba0 tx=0x7fcaac005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:33.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.612+0000 7fcabfe00640 1 -- 172.21.15.2:0/1977272689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac01068d0 msgr2=0x7fcac019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:33.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.612+0000 7fcabfe00640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac01068d0 0x7fcac019ed80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009fd0 tx=0x7fcab800ef10 comp rx=0 tx=0).stop 2024-09-10T08:08:33.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.612+0000 7fcabfe00640 1 -- 172.21.15.2:0/1977272689 shutdown_connections 2024-09-10T08:08:33.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.612+0000 7fcabfe00640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac01068d0 0x7fcac019ed80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.612+0000 7fcabfe00640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcaa40a2c10 0x7fcaa40a50d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.612+0000 7fcabfe00640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcac0101270 0x7fcac019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.612+0000 7fcabfe00640 1 --2- 172.21.15.2:0/1977272689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcac01008d0 0x7fcac019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:33.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.612+0000 7fcabfe00640 1 -- 172.21.15.2:0/1977272689 >> 172.21.15.2:0/1977272689 conn(0x7fcac00fc610 msgr2=0x7fcac006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:33.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.612+0000 7fcabfe00640 1 -- 172.21.15.2:0/1977272689 shutdown_connections 2024-09-10T08:08:33.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:33.612+0000 7fcabfe00640 1 -- 172.21.15.2:0/1977272689 wait complete. 2024-09-10T08:08:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:33 smithi002 ceph-mon[30563]: pgmap v399: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-10T08:08:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:33 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:33 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:33 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:08:33.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:33 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:08:33.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:33 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:33.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:33 smithi155 ceph-mon[35405]: pgmap v399: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-10T08:08:33.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:33 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:33.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:33 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:33.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:33 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:08:33.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:33 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:08:33.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:33 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:33.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:33 smithi107 ceph-mon[35048]: pgmap v399: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-10T08:08:33.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:33 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:33.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:33 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:33.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:33 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:08:33.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:33 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:08:33.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:33 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:08:33.936 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:34.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.514+0000 7f852b000640 1 -- 172.21.15.2:0/3591603749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85241047e0 msgr2=0x7f8524108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:34.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.514+0000 7f852b000640 1 --2- 172.21.15.2:0/3591603749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85241047e0 0x7f8524108830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8510009900 tx=0x7f851002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:08:34.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.514+0000 7f852b000640 1 -- 172.21.15.2:0/3591603749 shutdown_connections 2024-09-10T08:08:34.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.514+0000 7f852b000640 1 --2- 172.21.15.2:0/3591603749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8524108f60 0x7f8524110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:34.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.514+0000 7f852b000640 1 --2- 172.21.15.2:0/3591603749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85241047e0 0x7f8524108830 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.514+0000 7f852b000640 1 --2- 172.21.15.2:0/3591603749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8524103e30 0x7f8524104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.514+0000 7f852b000640 1 -- 172.21.15.2:0/3591603749 >> 172.21.15.2:0/3591603749 conn(0x7f85240ff910 msgr2=0x7f8524101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.514+0000 7f852b000640 1 -- 172.21.15.2:0/3591603749 shutdown_connections 2024-09-10T08:08:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.514+0000 7f852b000640 1 -- 172.21.15.2:0/3591603749 wait complete. 2024-09-10T08:08:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.515+0000 7f852b000640 1 Processor -- start 2024-09-10T08:08:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.515+0000 7f852b000640 1 -- start start 2024-09-10T08:08:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.516+0000 7f852b000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8524103e30 0x7f8524076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.516+0000 7f852b000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85241047e0 0x7f8524076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.516+0000 7f852b000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8524108f60 0x7f8524077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.516+0000 7f852b000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8524113fc0 con 0x7f85241047e0 2024-09-10T08:08:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.516+0000 7f852b000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8524113e40 con 0x7f8524103e30 2024-09-10T08:08:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.516+0000 7f852b000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8524114140 con 0x7f8524108f60 2024-09-10T08:08:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.516+0000 7f8529200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85241047e0 0x7f8524076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.516+0000 7f8529200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85241047e0 0x7f8524076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39904/0 (socket says 172.21.15.2:39904) 2024-09-10T08:08:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.516+0000 7f8529200640 1 -- 172.21.15.2:0/3111325011 learned_addr learned my addr 172.21.15.2:0/3111325011 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.516+0000 7f8529c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8524103e30 0x7f8524076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.516+0000 7f852a600640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8524108f60 0x7f8524077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.517+0000 7f8529c00640 1 -- 172.21.15.2:0/3111325011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8524108f60 msgr2=0x7f8524077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.517+0000 7f8529c00640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8524108f60 0x7f8524077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.517+0000 7f8529c00640 1 -- 172.21.15.2:0/3111325011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85241047e0 msgr2=0x7f8524076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.517+0000 7f8529c00640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85241047e0 0x7f8524076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.517+0000 7f8529c00640 1 -- 172.21.15.2:0/3111325011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85241aab50 con 0x7f8524103e30 2024-09-10T08:08:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.517+0000 7f852a600640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8524108f60 0x7f8524077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:08:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.517+0000 7f8529200640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85241047e0 0x7f8524076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.517+0000 7f8529c00640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8524103e30 0x7f8524076700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f851800ea50 tx=0x7f851800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.517+0000 7f8522000640 1 -- 172.21.15.2:0/3111325011 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851800cd00 con 0x7f8524103e30 2024-09-10T08:08:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.517+0000 7f852b000640 1 -- 172.21.15.2:0/3111325011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85241aae40 con 0x7f8524103e30 2024-09-10T08:08:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.517+0000 7f8522000640 1 -- 172.21.15.2:0/3111325011 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8518010910 con 0x7f8524103e30 2024-09-10T08:08:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.517+0000 7f852b000640 1 -- 172.21.15.2:0/3111325011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85241ab340 con 0x7f8524103e30 2024-09-10T08:08:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.517+0000 7f8522000640 1 -- 172.21.15.2:0/3111325011 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518018920 con 0x7f8524103e30 2024-09-10T08:08:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.519+0000 7f852b000640 1 -- 172.21.15.2:0/3111325011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8504005290 con 0x7f8524103e30 2024-09-10T08:08:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.520+0000 7f8522000640 1 -- 172.21.15.2:0/3111325011 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f8518010ac0 con 0x7f8524103e30 2024-09-10T08:08:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.522+0000 7f8522000640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f850c0a2c90 0x7f850c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.522+0000 7f8522000640 1 -- 172.21.15.2:0/3111325011 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f85180cf880 con 0x7f8524103e30 2024-09-10T08:08:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.522+0000 7f8529200640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f850c0a2c90 0x7f850c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.523+0000 7f8529200640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f850c0a2c90 0x7f850c0a5150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8524077ec0 tx=0x7f8510005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:34.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.526+0000 7f8522000640 1 -- 172.21.15.2:0/3111325011 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f851809c230 con 0x7f8524103e30 2024-09-10T08:08:34.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.716+0000 7f852b000640 1 -- 172.21.15.2:0/3111325011 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8504000bc0 con 0x7f850c0a2c90 2024-09-10T08:08:34.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.733+0000 7f8522000640 1 -- 172.21.15.2:0/3111325011 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340722 (secure 0 0 0) 0x7f8504000bc0 con 0x7f850c0a2c90 2024-09-10T08:08:34.737 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:34.738 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-10T08:08:34.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.738+0000 7f852b000640 1 -- 172.21.15.2:0/3111325011 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f850c0a2c90 msgr2=0x7f850c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:34.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.738+0000 7f852b000640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f850c0a2c90 0x7f850c0a5150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8524077ec0 tx=0x7f8510005a20 comp rx=0 tx=0).stop 2024-09-10T08:08:34.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.738+0000 7f852b000640 1 -- 172.21.15.2:0/3111325011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8524103e30 msgr2=0x7f8524076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:34.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.738+0000 7f852b000640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8524103e30 0x7f8524076700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f851800ea50 tx=0x7f851800ef10 comp rx=0 tx=0).stop 2024-09-10T08:08:34.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.739+0000 7f852b000640 1 -- 172.21.15.2:0/3111325011 shutdown_connections 2024-09-10T08:08:34.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.739+0000 7f852b000640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f850c0a2c90 0x7f850c0a5150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:34.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.739+0000 7f852b000640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8524108f60 0x7f8524077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:34.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.739+0000 7f852b000640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85241047e0 0x7f8524076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:34.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.739+0000 7f852b000640 1 --2- 172.21.15.2:0/3111325011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8524103e30 0x7f8524076700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:34.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.739+0000 7f852b000640 1 -- 172.21.15.2:0/3111325011 >> 172.21.15.2:0/3111325011 conn(0x7f85240ff910 msgr2=0x7f8524100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:34.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.739+0000 7f852b000640 1 -- 172.21.15.2:0/3111325011 shutdown_connections 2024-09-10T08:08:34.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:34.739+0000 7f852b000640 1 -- 172.21.15.2:0/3111325011 wait complete. 2024-09-10T08:08:35.382 INFO:tasks.cephfs.filesystem:scrub status for tag:401195cc-8ebd-46b5-8355-5a423a6ee37e - {'path': '/', 'tag': '401195cc-8ebd-46b5-8355-5a423a6ee37e', 'options': 'recursive,force'} 2024-09-10T08:08:35.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:35 smithi002 ceph-mon[30563]: pgmap v400: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-10T08:08:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:35 smithi155 ceph-mon[35405]: pgmap v400: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-10T08:08:35.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:35 smithi107 ceph-mon[35048]: pgmap v400: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-10T08:08:36.264 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-09-10T08:08:33.695536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667350,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":940,"num_read_kb":598,"num_write":373,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1184461,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":800,"ondisk_log_size":800,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362060,"kb_used_data":12812,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579764,"statfs":{"total":1151940427776,"available":1151569678336,"internally_reserved":0,"allocated":13119488,"data_stored":8554148,"data_compressed":497802,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3617,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":41,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003318"},"pg_stats":[{"pgid":"2.1f","version":"93'9","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514795+0000","last_change":"2024-09-10T08:05:22.462062+0000","last_active":"2024-09-10T08:07:05.514795+0000","last_peered":"2024-09-10T08:07:05.514795+0000","last_clean":"2024-09-10T08:07:05.514795+0000","last_became_active":"2024-09-10T08:05:22.460973+0000","last_became_peered":"2024-09-10T08:05:22.460973+0000","last_unstale":"2024-09-10T08:07:05.514795+0000","last_undegraded":"2024-09-10T08:07:05.514795+0000","last_fullsized":"2024-09-10T08:07:05.514795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:01.357446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656803+0000","last_change":"2024-09-10T08:05:25.482873+0000","last_active":"2024-09-10T08:08:03.656803+0000","last_peered":"2024-09-10T08:08:03.656803+0000","last_clean":"2024-09-10T08:08:03.656803+0000","last_became_active":"2024-09-10T08:05:25.482559+0000","last_became_peered":"2024-09-10T08:05:25.482559+0000","last_unstale":"2024-09-10T08:08:03.656803+0000","last_undegraded":"2024-09-10T08:08:03.656803+0000","last_fullsized":"2024-09-10T08:08:03.656803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:10:17.366160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511384+0000","last_change":"2024-09-10T08:06:04.736356+0000","last_active":"2024-09-10T08:07:05.511384+0000","last_peered":"2024-09-10T08:07:05.511384+0000","last_clean":"2024-09-10T08:07:05.511384+0000","last_became_active":"2024-09-10T08:06:04.735859+0000","last_became_peered":"2024-09-10T08:06:04.735859+0000","last_unstale":"2024-09-10T08:07:05.511384+0000","last_undegraded":"2024-09-10T08:07:05.511384+0000","last_fullsized":"2024-09-10T08:07:05.511384+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:56.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"93'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.877433+0000","last_change":"2024-09-10T08:05:22.569504+0000","last_active":"2024-09-10T08:07:05.877433+0000","last_peered":"2024-09-10T08:07:05.877433+0000","last_clean":"2024-09-10T08:07:05.877433+0000","last_became_active":"2024-09-10T08:05:22.569164+0000","last_became_peered":"2024-09-10T08:05:22.569164+0000","last_unstale":"2024-09-10T08:07:05.877433+0000","last_undegraded":"2024-09-10T08:07:05.877433+0000","last_fullsized":"2024-09-10T08:07:05.877433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:41.628963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.680208+0000","last_change":"2024-09-10T08:05:25.493017+0000","last_active":"2024-09-10T08:08:03.680208+0000","last_peered":"2024-09-10T08:08:03.680208+0000","last_clean":"2024-09-10T08:08:03.680208+0000","last_became_active":"2024-09-10T08:05:25.491370+0000","last_became_peered":"2024-09-10T08:05:25.491370+0000","last_unstale":"2024-09-10T08:08:03.680208+0000","last_undegraded":"2024-09-10T08:08:03.680208+0000","last_fullsized":"2024-09-10T08:08:03.680208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:41.616549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914087+0000","last_change":"2024-09-10T08:05:35.523751+0000","last_active":"2024-09-10T08:07:05.914087+0000","last_peered":"2024-09-10T08:07:05.914087+0000","last_clean":"2024-09-10T08:07:05.914087+0000","last_became_active":"2024-09-10T08:05:35.523372+0000","last_became_peered":"2024-09-10T08:05:35.523372+0000","last_unstale":"2024-09-10T08:07:05.914087+0000","last_undegraded":"2024-09-10T08:07:05.914087+0000","last_fullsized":"2024-09-10T08:07:05.914087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:20.971596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508934+0000","last_change":"2024-09-10T08:05:22.460590+0000","last_active":"2024-09-10T08:07:05.508934+0000","last_peered":"2024-09-10T08:07:05.508934+0000","last_clean":"2024-09-10T08:07:05.508934+0000","last_became_active":"2024-09-10T08:05:22.460238+0000","last_became_peered":"2024-09-10T08:05:22.460238+0000","last_unstale":"2024-09-10T08:07:05.508934+0000","last_undegraded":"2024-09-10T08:07:05.508934+0000","last_fullsized":"2024-09-10T08:07:05.508934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:18.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669035+0000","last_change":"2024-09-10T08:05:25.487392+0000","last_active":"2024-09-10T08:08:03.669035+0000","last_peered":"2024-09-10T08:08:03.669035+0000","last_clean":"2024-09-10T08:08:03.669035+0000","last_became_active":"2024-09-10T08:05:25.487054+0000","last_became_peered":"2024-09-10T08:05:25.487054+0000","last_unstale":"2024-09-10T08:08:03.669035+0000","last_undegraded":"2024-09-10T08:08:03.669035+0000","last_fullsized":"2024-09-10T08:08:03.669035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:16.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914818+0000","last_change":"2024-09-10T08:06:04.735778+0000","last_active":"2024-09-10T08:07:05.914818+0000","last_peered":"2024-09-10T08:07:05.914818+0000","last_clean":"2024-09-10T08:07:05.914818+0000","last_became_active":"2024-09-10T08:06:04.735208+0000","last_became_peered":"2024-09-10T08:06:04.735208+0000","last_unstale":"2024-09-10T08:07:05.914818+0000","last_undegraded":"2024-09-10T08:07:05.914818+0000","last_fullsized":"2024-09-10T08:07:05.914818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:36.982246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508944+0000","last_change":"2024-09-10T08:05:22.458323+0000","last_active":"2024-09-10T08:07:05.508944+0000","last_peered":"2024-09-10T08:07:05.508944+0000","last_clean":"2024-09-10T08:07:05.508944+0000","last_became_active":"2024-09-10T08:05:22.457965+0000","last_became_peered":"2024-09-10T08:05:22.457965+0000","last_unstale":"2024-09-10T08:07:05.508944+0000","last_undegraded":"2024-09-10T08:07:05.508944+0000","last_fullsized":"2024-09-10T08:07:05.508944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:15.021305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.663997+0000","last_change":"2024-09-10T08:05:25.489883+0000","last_active":"2024-09-10T08:08:03.663997+0000","last_peered":"2024-09-10T08:08:03.663997+0000","last_clean":"2024-09-10T08:08:03.663997+0000","last_became_active":"2024-09-10T08:05:25.489482+0000","last_became_peered":"2024-09-10T08:05:25.489482+0000","last_unstale":"2024-09-10T08:08:03.663997+0000","last_undegraded":"2024-09-10T08:08:03.663997+0000","last_fullsized":"2024-09-10T08:08:03.663997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:52.080628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914064+0000","last_change":"2024-09-10T08:05:35.531458+0000","last_active":"2024-09-10T08:07:05.914064+0000","last_peered":"2024-09-10T08:07:05.914064+0000","last_clean":"2024-09-10T08:07:05.914064+0000","last_became_active":"2024-09-10T08:05:35.531072+0000","last_became_peered":"2024-09-10T08:05:35.531072+0000","last_unstale":"2024-09-10T08:07:05.914064+0000","last_undegraded":"2024-09-10T08:07:05.914064+0000","last_fullsized":"2024-09-10T08:07:05.914064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:04.792664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667650+0000","last_change":"2024-09-10T08:05:25.486865+0000","last_active":"2024-09-10T08:08:03.667650+0000","last_peered":"2024-09-10T08:08:03.667650+0000","last_clean":"2024-09-10T08:08:03.667650+0000","last_became_active":"2024-09-10T08:05:25.486531+0000","last_became_peered":"2024-09-10T08:05:25.486531+0000","last_unstale":"2024-09-10T08:08:03.667650+0000","last_undegraded":"2024-09-10T08:08:03.667650+0000","last_fullsized":"2024-09-10T08:08:03.667650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:26.854682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"93'4","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508917+0000","last_change":"2024-09-10T08:06:04.743303+0000","last_active":"2024-09-10T08:07:05.508917+0000","last_peered":"2024-09-10T08:07:05.508917+0000","last_clean":"2024-09-10T08:07:05.508917+0000","last_became_active":"2024-09-10T08:06:04.733294+0000","last_became_peered":"2024-09-10T08:06:04.733294+0000","last_unstale":"2024-09-10T08:07:05.508917+0000","last_undegraded":"2024-09-10T08:07:05.508917+0000","last_fullsized":"2024-09-10T08:07:05.508917+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:18.542658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914822+0000","last_change":"2024-09-10T08:05:35.642083+0000","last_active":"2024-09-10T08:07:05.914822+0000","last_peered":"2024-09-10T08:07:05.914822+0000","last_clean":"2024-09-10T08:07:05.914822+0000","last_became_active":"2024-09-10T08:05:35.641214+0000","last_became_peered":"2024-09-10T08:05:35.641214+0000","last_unstale":"2024-09-10T08:07:05.914822+0000","last_undegraded":"2024-09-10T08:07:05.914822+0000","last_fullsized":"2024-09-10T08:07:05.914822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:00:21.731769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681852+0000","last_change":"2024-09-10T08:05:25.501993+0000","last_active":"2024-09-10T08:08:03.681852+0000","last_peered":"2024-09-10T08:08:03.681852+0000","last_clean":"2024-09-10T08:08:03.681852+0000","last_became_active":"2024-09-10T08:05:25.499912+0000","last_became_peered":"2024-09-10T08:05:25.499912+0000","last_unstale":"2024-09-10T08:08:03.681852+0000","last_undegraded":"2024-09-10T08:08:03.681852+0000","last_fullsized":"2024-09-10T08:08:03.681852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:06.780067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:36.268711+0000","last_change":"2024-09-10T08:05:22.461786+0000","last_active":"2024-09-10T08:07:36.268711+0000","last_peered":"2024-09-10T08:07:36.268711+0000","last_clean":"2024-09-10T08:07:36.268711+0000","last_became_active":"2024-09-10T08:05:22.460821+0000","last_became_peered":"2024-09-10T08:05:22.460821+0000","last_unstale":"2024-09-10T08:07:36.268711+0000","last_undegraded":"2024-09-10T08:07:36.268711+0000","last_fullsized":"2024-09-10T08:07:36.268711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:25.083538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914568+0000","last_change":"2024-09-10T08:06:04.734470+0000","last_active":"2024-09-10T08:07:05.914568+0000","last_peered":"2024-09-10T08:07:05.914568+0000","last_clean":"2024-09-10T08:07:05.914568+0000","last_became_active":"2024-09-10T08:06:04.734040+0000","last_became_peered":"2024-09-10T08:06:04.734040+0000","last_unstale":"2024-09-10T08:07:05.914568+0000","last_undegraded":"2024-09-10T08:07:05.914568+0000","last_fullsized":"2024-09-10T08:07:05.914568+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:36.312144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675820+0000","last_change":"2024-09-10T08:05:25.492711+0000","last_active":"2024-09-10T08:08:03.675820+0000","last_peered":"2024-09-10T08:08:03.675820+0000","last_clean":"2024-09-10T08:08:03.675820+0000","last_became_active":"2024-09-10T08:05:25.492336+0000","last_became_peered":"2024-09-10T08:05:25.492336+0000","last_unstale":"2024-09-10T08:08:03.675820+0000","last_undegraded":"2024-09-10T08:08:03.675820+0000","last_fullsized":"2024-09-10T08:08:03.675820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:04.168884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"93'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876783+0000","last_change":"2024-09-10T08:05:22.461535+0000","last_active":"2024-09-10T08:07:05.876783+0000","last_peered":"2024-09-10T08:07:05.876783+0000","last_clean":"2024-09-10T08:07:05.876783+0000","last_became_active":"2024-09-10T08:05:22.461208+0000","last_became_peered":"2024-09-10T08:05:22.461208+0000","last_unstale":"2024-09-10T08:07:05.876783+0000","last_undegraded":"2024-09-10T08:07:05.876783+0000","last_fullsized":"2024-09-10T08:07:05.876783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:26.009242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513889+0000","last_change":"2024-09-10T08:05:35.535442+0000","last_active":"2024-09-10T08:07:05.513889+0000","last_peered":"2024-09-10T08:07:05.513889+0000","last_clean":"2024-09-10T08:07:05.513889+0000","last_became_active":"2024-09-10T08:05:35.535058+0000","last_became_peered":"2024-09-10T08:05:35.535058+0000","last_unstale":"2024-09-10T08:07:05.513889+0000","last_undegraded":"2024-09-10T08:07:05.513889+0000","last_fullsized":"2024-09-10T08:07:05.513889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:39.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.627539+0000","last_change":"2024-09-10T08:05:25.483440+0000","last_active":"2024-09-10T08:08:03.627539+0000","last_peered":"2024-09-10T08:08:03.627539+0000","last_clean":"2024-09-10T08:08:03.627539+0000","last_became_active":"2024-09-10T08:05:25.483141+0000","last_became_peered":"2024-09-10T08:05:25.483141+0000","last_unstale":"2024-09-10T08:08:03.627539+0000","last_undegraded":"2024-09-10T08:08:03.627539+0000","last_fullsized":"2024-09-10T08:08:03.627539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:00.953188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"93'2","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914337+0000","last_change":"2024-09-10T08:05:22.568141+0000","last_active":"2024-09-10T08:07:05.914337+0000","last_peered":"2024-09-10T08:07:05.914337+0000","last_clean":"2024-09-10T08:07:05.914337+0000","last_became_active":"2024-09-10T08:05:22.567857+0000","last_became_peered":"2024-09-10T08:05:22.567857+0000","last_unstale":"2024-09-10T08:07:05.914337+0000","last_undegraded":"2024-09-10T08:07:05.914337+0000","last_fullsized":"2024-09-10T08:07:05.914337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:22.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914846+0000","last_change":"2024-09-10T08:05:35.641993+0000","last_active":"2024-09-10T08:07:05.914846+0000","last_peered":"2024-09-10T08:07:05.914846+0000","last_clean":"2024-09-10T08:07:05.914846+0000","last_became_active":"2024-09-10T08:05:35.641376+0000","last_became_peered":"2024-09-10T08:05:35.641376+0000","last_unstale":"2024-09-10T08:07:05.914846+0000","last_undegraded":"2024-09-10T08:07:05.914846+0000","last_fullsized":"2024-09-10T08:07:05.914846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:16:34.038924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.678764+0000","last_change":"2024-09-10T08:06:04.737560+0000","last_active":"2024-09-10T08:08:03.678764+0000","last_peered":"2024-09-10T08:08:03.678764+0000","last_clean":"2024-09-10T08:08:03.678764+0000","last_became_active":"2024-09-10T08:06:04.736656+0000","last_became_peered":"2024-09-10T08:06:04.736656+0000","last_unstale":"2024-09-10T08:08:03.678764+0000","last_undegraded":"2024-09-10T08:08:03.678764+0000","last_fullsized":"2024-09-10T08:08:03.678764+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:35.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"93'4","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915999+0000","last_change":"2024-09-10T08:05:22.567077+0000","last_active":"2024-09-10T08:07:05.915999+0000","last_peered":"2024-09-10T08:07:05.915999+0000","last_clean":"2024-09-10T08:07:05.915999+0000","last_became_active":"2024-09-10T08:05:22.566757+0000","last_became_peered":"2024-09-10T08:05:22.566757+0000","last_unstale":"2024-09-10T08:07:05.915999+0000","last_undegraded":"2024-09-10T08:07:05.915999+0000","last_fullsized":"2024-09-10T08:07:05.915999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:06.881510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511524+0000","last_change":"2024-09-10T08:05:35.519539+0000","last_active":"2024-09-10T08:07:05.511524+0000","last_peered":"2024-09-10T08:07:05.511524+0000","last_clean":"2024-09-10T08:07:05.511524+0000","last_became_active":"2024-09-10T08:05:35.519044+0000","last_became_peered":"2024-09-10T08:05:35.519044+0000","last_unstale":"2024-09-10T08:07:05.511524+0000","last_undegraded":"2024-09-10T08:07:05.511524+0000","last_fullsized":"2024-09-10T08:07:05.511524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:10.546949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658164+0000","last_change":"2024-09-10T08:07:05.771774+0000","last_active":"2024-09-10T08:08:03.658164+0000","last_peered":"2024-09-10T08:08:03.658164+0000","last_clean":"2024-09-10T08:08:03.658164+0000","last_became_active":"2024-09-10T08:07:05.771336+0000","last_became_peered":"2024-09-10T08:07:05.771336+0000","last_unstale":"2024-09-10T08:08:03.658164+0000","last_undegraded":"2024-09-10T08:08:03.658164+0000","last_fullsized":"2024-09-10T08:08:03.658164+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:11.482222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"95'14","reported_seq":68,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:18.518255+0000","last_change":"2024-09-10T08:06:04.741199+0000","last_active":"2024-09-10T08:08:18.518255+0000","last_peered":"2024-09-10T08:08:18.518255+0000","last_clean":"2024-09-10T08:08:18.518255+0000","last_became_active":"2024-09-10T08:06:04.738076+0000","last_became_peered":"2024-09-10T08:06:04.738076+0000","last_unstale":"2024-09-10T08:08:18.518255+0000","last_undegraded":"2024-09-10T08:08:18.518255+0000","last_fullsized":"2024-09-10T08:08:18.518255+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:45.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876462+0000","last_change":"2024-09-10T08:05:35.642358+0000","last_active":"2024-09-10T08:07:05.876462+0000","last_peered":"2024-09-10T08:07:05.876462+0000","last_clean":"2024-09-10T08:07:05.876462+0000","last_became_active":"2024-09-10T08:05:35.641408+0000","last_became_peered":"2024-09-10T08:05:35.641408+0000","last_unstale":"2024-09-10T08:07:05.876462+0000","last_undegraded":"2024-09-10T08:07:05.876462+0000","last_fullsized":"2024-09-10T08:07:05.876462+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:42.015139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658430+0000","last_change":"2024-09-10T08:05:25.486310+0000","last_active":"2024-09-10T08:08:03.658430+0000","last_peered":"2024-09-10T08:08:03.658430+0000","last_clean":"2024-09-10T08:08:03.658430+0000","last_became_active":"2024-09-10T08:05:25.485915+0000","last_became_peered":"2024-09-10T08:05:25.485915+0000","last_unstale":"2024-09-10T08:08:03.658430+0000","last_undegraded":"2024-09-10T08:08:03.658430+0000","last_fullsized":"2024-09-10T08:08:03.658430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:56.064361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"93'1","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508987+0000","last_change":"2024-09-10T08:05:22.570595+0000","last_active":"2024-09-10T08:07:05.508987+0000","last_peered":"2024-09-10T08:07:05.508987+0000","last_clean":"2024-09-10T08:07:05.508987+0000","last_became_active":"2024-09-10T08:05:22.570219+0000","last_became_peered":"2024-09-10T08:05:22.570219+0000","last_unstale":"2024-09-10T08:07:05.508987+0000","last_undegraded":"2024-09-10T08:07:05.508987+0000","last_fullsized":"2024-09-10T08:07:05.508987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:03.230325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769927+0000","last_change":"2024-09-10T08:05:35.530472+0000","last_active":"2024-09-10T08:07:05.769927+0000","last_peered":"2024-09-10T08:07:05.769927+0000","last_clean":"2024-09-10T08:07:05.769927+0000","last_became_active":"2024-09-10T08:05:35.530066+0000","last_became_peered":"2024-09-10T08:05:35.530066+0000","last_unstale":"2024-09-10T08:07:05.769927+0000","last_undegraded":"2024-09-10T08:07:05.769927+0000","last_fullsized":"2024-09-10T08:07:05.769927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:58.915767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671834+0000","last_change":"2024-09-10T08:05:25.500524+0000","last_active":"2024-09-10T08:08:03.671834+0000","last_peered":"2024-09-10T08:08:03.671834+0000","last_clean":"2024-09-10T08:08:03.671834+0000","last_became_active":"2024-09-10T08:05:25.500186+0000","last_became_peered":"2024-09-10T08:05:25.500186+0000","last_unstale":"2024-09-10T08:08:03.671834+0000","last_undegraded":"2024-09-10T08:08:03.671834+0000","last_fullsized":"2024-09-10T08:08:03.671834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:51.936222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513172+0000","last_change":"2024-09-10T08:05:22.569326+0000","last_active":"2024-09-10T08:07:05.513172+0000","last_peered":"2024-09-10T08:07:05.513172+0000","last_clean":"2024-09-10T08:07:05.513172+0000","last_became_active":"2024-09-10T08:05:22.568895+0000","last_became_peered":"2024-09-10T08:05:22.568895+0000","last_unstale":"2024-09-10T08:07:05.513172+0000","last_undegraded":"2024-09-10T08:07:05.513172+0000","last_fullsized":"2024-09-10T08:07:05.513172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:47.212073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508368+0000","last_change":"2024-09-10T08:05:35.641600+0000","last_active":"2024-09-10T08:07:05.508368+0000","last_peered":"2024-09-10T08:07:05.508368+0000","last_clean":"2024-09-10T08:07:05.508368+0000","last_became_active":"2024-09-10T08:05:35.641205+0000","last_became_peered":"2024-09-10T08:05:35.641205+0000","last_unstale":"2024-09-10T08:07:05.508368+0000","last_undegraded":"2024-09-10T08:07:05.508368+0000","last_fullsized":"2024-09-10T08:07:05.508368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:40.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.659935+0000","last_change":"2024-09-10T08:05:25.495035+0000","last_active":"2024-09-10T08:08:03.659935+0000","last_peered":"2024-09-10T08:08:03.659935+0000","last_clean":"2024-09-10T08:08:03.659935+0000","last_became_active":"2024-09-10T08:05:25.494733+0000","last_became_peered":"2024-09-10T08:05:25.494733+0000","last_unstale":"2024-09-10T08:08:03.659935+0000","last_undegraded":"2024-09-10T08:08:03.659935+0000","last_fullsized":"2024-09-10T08:08:03.659935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.875867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"93'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509485+0000","last_change":"2024-09-10T08:05:22.458727+0000","last_active":"2024-09-10T08:07:05.509485+0000","last_peered":"2024-09-10T08:07:05.509485+0000","last_clean":"2024-09-10T08:07:05.509485+0000","last_became_active":"2024-09-10T08:05:22.458397+0000","last_became_peered":"2024-09-10T08:05:22.458397+0000","last_unstale":"2024-09-10T08:07:05.509485+0000","last_undegraded":"2024-09-10T08:07:05.509485+0000","last_fullsized":"2024-09-10T08:07:05.509485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:07.875559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914403+0000","last_change":"2024-09-10T08:05:35.534192+0000","last_active":"2024-09-10T08:07:05.914403+0000","last_peered":"2024-09-10T08:07:05.914403+0000","last_clean":"2024-09-10T08:07:05.914403+0000","last_became_active":"2024-09-10T08:05:35.530450+0000","last_became_peered":"2024-09-10T08:05:35.530450+0000","last_unstale":"2024-09-10T08:07:05.914403+0000","last_undegraded":"2024-09-10T08:07:05.914403+0000","last_fullsized":"2024-09-10T08:07:05.914403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:10.039144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651611+0000","last_change":"2024-09-10T08:05:25.482252+0000","last_active":"2024-09-10T08:08:03.651611+0000","last_peered":"2024-09-10T08:08:03.651611+0000","last_clean":"2024-09-10T08:08:03.651611+0000","last_became_active":"2024-09-10T08:05:25.481901+0000","last_became_peered":"2024-09-10T08:05:25.481901+0000","last_unstale":"2024-09-10T08:08:03.651611+0000","last_undegraded":"2024-09-10T08:08:03.651611+0000","last_fullsized":"2024-09-10T08:08:03.651611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:17.063340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":180,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:31.462022+0000","last_change":"2024-09-10T08:05:22.569199+0000","last_active":"2024-09-10T08:08:31.462022+0000","last_peered":"2024-09-10T08:08:31.462022+0000","last_clean":"2024-09-10T08:08:31.462022+0000","last_became_active":"2024-09-10T08:05:22.568787+0000","last_became_peered":"2024-09-10T08:05:22.568787+0000","last_unstale":"2024-09-10T08:08:31.462022+0000","last_undegraded":"2024-09-10T08:08:31.462022+0000","last_fullsized":"2024-09-10T08:08:31.462022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:59.610130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914677+0000","last_change":"2024-09-10T08:05:35.642446+0000","last_active":"2024-09-10T08:07:05.914677+0000","last_peered":"2024-09-10T08:07:05.914677+0000","last_clean":"2024-09-10T08:07:05.914677+0000","last_became_active":"2024-09-10T08:05:35.641543+0000","last_became_peered":"2024-09-10T08:05:35.641543+0000","last_unstale":"2024-09-10T08:07:05.914677+0000","last_undegraded":"2024-09-10T08:07:05.914677+0000","last_fullsized":"2024-09-10T08:07:05.914677+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:33.658406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.652192+0000","last_change":"2024-09-10T08:05:25.487598+0000","last_active":"2024-09-10T08:08:03.652192+0000","last_peered":"2024-09-10T08:08:03.652192+0000","last_clean":"2024-09-10T08:08:03.652192+0000","last_became_active":"2024-09-10T08:05:25.486879+0000","last_became_peered":"2024-09-10T08:05:25.486879+0000","last_unstale":"2024-09-10T08:08:03.652192+0000","last_undegraded":"2024-09-10T08:08:03.652192+0000","last_fullsized":"2024-09-10T08:08:03.652192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:20.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509934+0000","last_change":"2024-09-10T08:05:22.463416+0000","last_active":"2024-09-10T08:07:05.509934+0000","last_peered":"2024-09-10T08:07:05.509934+0000","last_clean":"2024-09-10T08:07:05.509934+0000","last_became_active":"2024-09-10T08:05:22.463004+0000","last_became_peered":"2024-09-10T08:05:22.463004+0000","last_unstale":"2024-09-10T08:07:05.509934+0000","last_undegraded":"2024-09-10T08:07:05.509934+0000","last_fullsized":"2024-09-10T08:07:05.509934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:56.257812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569364+0000","last_change":"2024-09-10T08:05:35.642059+0000","last_active":"2024-09-10T08:07:05.569364+0000","last_peered":"2024-09-10T08:07:05.569364+0000","last_clean":"2024-09-10T08:07:05.569364+0000","last_became_active":"2024-09-10T08:05:35.641540+0000","last_became_peered":"2024-09-10T08:05:35.641540+0000","last_unstale":"2024-09-10T08:07:05.569364+0000","last_undegraded":"2024-09-10T08:07:05.569364+0000","last_fullsized":"2024-09-10T08:07:05.569364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:28.557584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.670571+0000","last_change":"2024-09-10T08:05:25.496111+0000","last_active":"2024-09-10T08:08:03.670571+0000","last_peered":"2024-09-10T08:08:03.670571+0000","last_clean":"2024-09-10T08:08:03.670571+0000","last_became_active":"2024-09-10T08:05:25.495757+0000","last_became_peered":"2024-09-10T08:05:25.495757+0000","last_unstale":"2024-09-10T08:08:03.670571+0000","last_undegraded":"2024-09-10T08:08:03.670571+0000","last_fullsized":"2024-09-10T08:08:03.670571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:41.078205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'3","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915005+0000","last_change":"2024-09-10T08:05:22.460069+0000","last_active":"2024-09-10T08:07:05.915005+0000","last_peered":"2024-09-10T08:07:05.915005+0000","last_clean":"2024-09-10T08:07:05.915005+0000","last_became_active":"2024-09-10T08:05:22.459557+0000","last_became_peered":"2024-09-10T08:05:22.459557+0000","last_unstale":"2024-09-10T08:07:05.915005+0000","last_undegraded":"2024-09-10T08:07:05.915005+0000","last_fullsized":"2024-09-10T08:07:05.915005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:49.058507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509959+0000","last_change":"2024-09-10T08:04:08.188262+0000","last_active":"2024-09-10T08:07:05.509959+0000","last_peered":"2024-09-10T08:07:05.509959+0000","last_clean":"2024-09-10T08:07:05.509959+0000","last_became_active":"2024-09-10T08:04:08.187691+0000","last_became_peered":"2024-09-10T08:04:08.187691+0000","last_unstale":"2024-09-10T08:07:05.509959+0000","last_undegraded":"2024-09-10T08:07:05.509959+0000","last_fullsized":"2024-09-10T08:07:05.509959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_clean_scrub_stamp":"2024-09-10T08:00:26.235127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:15.358818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509550+0000","last_change":"2024-09-10T08:05:35.528371+0000","last_active":"2024-09-10T08:07:05.509550+0000","last_peered":"2024-09-10T08:07:05.509550+0000","last_clean":"2024-09-10T08:07:05.509550+0000","last_became_active":"2024-09-10T08:05:35.527925+0000","last_became_peered":"2024-09-10T08:05:35.527925+0000","last_unstale":"2024-09-10T08:07:05.509550+0000","last_undegraded":"2024-09-10T08:07:05.509550+0000","last_fullsized":"2024-09-10T08:07:05.509550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:37:00.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667384+0000","last_change":"2024-09-10T08:05:25.490753+0000","last_active":"2024-09-10T08:08:03.667384+0000","last_peered":"2024-09-10T08:08:03.667384+0000","last_clean":"2024-09-10T08:08:03.667384+0000","last_became_active":"2024-09-10T08:05:25.489996+0000","last_became_peered":"2024-09-10T08:05:25.489996+0000","last_unstale":"2024-09-10T08:08:03.667384+0000","last_undegraded":"2024-09-10T08:08:03.667384+0000","last_fullsized":"2024-09-10T08:08:03.667384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:15.637199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'3","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511918+0000","last_change":"2024-09-10T08:05:22.568696+0000","last_active":"2024-09-10T08:07:05.511918+0000","last_peered":"2024-09-10T08:07:05.511918+0000","last_clean":"2024-09-10T08:07:05.511918+0000","last_became_active":"2024-09-10T08:05:22.568331+0000","last_became_peered":"2024-09-10T08:05:22.568331+0000","last_unstale":"2024-09-10T08:07:05.511918+0000","last_undegraded":"2024-09-10T08:07:05.511918+0000","last_fullsized":"2024-09-10T08:07:05.511918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:21.699397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914428+0000","last_change":"2024-09-10T08:05:35.530804+0000","last_active":"2024-09-10T08:07:05.914428+0000","last_peered":"2024-09-10T08:07:05.914428+0000","last_clean":"2024-09-10T08:07:05.914428+0000","last_became_active":"2024-09-10T08:05:35.530356+0000","last_became_peered":"2024-09-10T08:05:35.530356+0000","last_unstale":"2024-09-10T08:07:05.914428+0000","last_undegraded":"2024-09-10T08:07:05.914428+0000","last_fullsized":"2024-09-10T08:07:05.914428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:59.318761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.662813+0000","last_change":"2024-09-10T08:05:25.489546+0000","last_active":"2024-09-10T08:08:03.662813+0000","last_peered":"2024-09-10T08:08:03.662813+0000","last_clean":"2024-09-10T08:08:03.662813+0000","last_became_active":"2024-09-10T08:05:25.489188+0000","last_became_peered":"2024-09-10T08:05:25.489188+0000","last_unstale":"2024-09-10T08:08:03.662813+0000","last_undegraded":"2024-09-10T08:08:03.662813+0000","last_fullsized":"2024-09-10T08:08:03.662813+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:26.471681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569736+0000","last_change":"2024-09-10T08:05:22.463912+0000","last_active":"2024-09-10T08:07:05.569736+0000","last_peered":"2024-09-10T08:07:05.569736+0000","last_clean":"2024-09-10T08:07:05.569736+0000","last_became_active":"2024-09-10T08:05:22.463561+0000","last_became_peered":"2024-09-10T08:05:22.463561+0000","last_unstale":"2024-09-10T08:07:05.569736+0000","last_undegraded":"2024-09-10T08:07:05.569736+0000","last_fullsized":"2024-09-10T08:07:05.569736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:22.407604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514959+0000","last_change":"2024-09-10T08:05:35.528795+0000","last_active":"2024-09-10T08:07:05.514959+0000","last_peered":"2024-09-10T08:07:05.514959+0000","last_clean":"2024-09-10T08:07:05.514959+0000","last_became_active":"2024-09-10T08:05:35.528402+0000","last_became_peered":"2024-09-10T08:05:35.528402+0000","last_unstale":"2024-09-10T08:07:05.514959+0000","last_undegraded":"2024-09-10T08:07:05.514959+0000","last_fullsized":"2024-09-10T08:07:05.514959+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:14.606371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.657641+0000","last_change":"2024-09-10T08:05:25.484086+0000","last_active":"2024-09-10T08:08:03.657641+0000","last_peered":"2024-09-10T08:08:03.657641+0000","last_clean":"2024-09-10T08:08:03.657641+0000","last_became_active":"2024-09-10T08:05:25.483757+0000","last_became_peered":"2024-09-10T08:05:25.483757+0000","last_unstale":"2024-09-10T08:08:03.657641+0000","last_undegraded":"2024-09-10T08:08:03.657641+0000","last_fullsized":"2024-09-10T08:08:03.657641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:16:16.362739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914678+0000","last_change":"2024-09-10T08:05:22.462220+0000","last_active":"2024-09-10T08:07:05.914678+0000","last_peered":"2024-09-10T08:07:05.914678+0000","last_clean":"2024-09-10T08:07:05.914678+0000","last_became_active":"2024-09-10T08:05:22.461726+0000","last_became_peered":"2024-09-10T08:05:22.461726+0000","last_unstale":"2024-09-10T08:07:05.914678+0000","last_undegraded":"2024-09-10T08:07:05.914678+0000","last_fullsized":"2024-09-10T08:07:05.914678+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:56.381034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508271+0000","last_change":"2024-09-10T08:06:04.734827+0000","last_active":"2024-09-10T08:07:05.508271+0000","last_peered":"2024-09-10T08:07:05.508271+0000","last_clean":"2024-09-10T08:07:05.508271+0000","last_became_active":"2024-09-10T08:06:04.734402+0000","last_became_peered":"2024-09-10T08:06:04.734402+0000","last_unstale":"2024-09-10T08:07:05.508271+0000","last_undegraded":"2024-09-10T08:07:05.508271+0000","last_fullsized":"2024-09-10T08:07:05.508271+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:24.806538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654332+0000","last_change":"2024-09-10T08:05:25.486100+0000","last_active":"2024-09-10T08:08:03.654332+0000","last_peered":"2024-09-10T08:08:03.654332+0000","last_clean":"2024-09-10T08:08:03.654332+0000","last_became_active":"2024-09-10T08:05:25.485586+0000","last_became_peered":"2024-09-10T08:05:25.485586+0000","last_unstale":"2024-09-10T08:08:03.654332+0000","last_undegraded":"2024-09-10T08:08:03.654332+0000","last_fullsized":"2024-09-10T08:08:03.654332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:33.565054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"93'2","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770403+0000","last_change":"2024-09-10T08:06:04.743408+0000","last_active":"2024-09-10T08:07:05.770403+0000","last_peered":"2024-09-10T08:07:05.770403+0000","last_clean":"2024-09-10T08:07:05.770403+0000","last_became_active":"2024-09-10T08:06:04.732830+0000","last_became_peered":"2024-09-10T08:06:04.732830+0000","last_unstale":"2024-09-10T08:07:05.770403+0000","last_undegraded":"2024-09-10T08:07:05.770403+0000","last_fullsized":"2024-09-10T08:07:05.770403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:29.381259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568977+0000","last_change":"2024-09-10T08:05:35.533015+0000","last_active":"2024-09-10T08:07:05.568977+0000","last_peered":"2024-09-10T08:07:05.568977+0000","last_clean":"2024-09-10T08:07:05.568977+0000","last_became_active":"2024-09-10T08:05:35.532590+0000","last_became_peered":"2024-09-10T08:05:35.532590+0000","last_unstale":"2024-09-10T08:07:05.568977+0000","last_undegraded":"2024-09-10T08:07:05.568977+0000","last_fullsized":"2024-09-10T08:07:05.568977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:23.984273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.650197+0000","last_change":"2024-09-10T08:05:25.489994+0000","last_active":"2024-09-10T08:08:03.650197+0000","last_peered":"2024-09-10T08:08:03.650197+0000","last_clean":"2024-09-10T08:08:03.650197+0000","last_became_active":"2024-09-10T08:05:25.489471+0000","last_became_peered":"2024-09-10T08:05:25.489471+0000","last_unstale":"2024-09-10T08:08:03.650197+0000","last_undegraded":"2024-09-10T08:08:03.650197+0000","last_fullsized":"2024-09-10T08:08:03.650197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:53.866570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"93'15","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508405+0000","last_change":"2024-09-10T08:05:22.568900+0000","last_active":"2024-09-10T08:07:05.508405+0000","last_peered":"2024-09-10T08:07:05.508405+0000","last_clean":"2024-09-10T08:07:05.508405+0000","last_became_active":"2024-09-10T08:05:22.568542+0000","last_became_peered":"2024-09-10T08:05:22.568542+0000","last_unstale":"2024-09-10T08:07:05.508405+0000","last_undegraded":"2024-09-10T08:07:05.508405+0000","last_fullsized":"2024-09-10T08:07:05.508405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:58:45.669769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914683+0000","last_change":"2024-09-10T08:05:35.537364+0000","last_active":"2024-09-10T08:07:05.914683+0000","last_peered":"2024-09-10T08:07:05.914683+0000","last_clean":"2024-09-10T08:07:05.914683+0000","last_became_active":"2024-09-10T08:05:35.535958+0000","last_became_peered":"2024-09-10T08:05:35.535958+0000","last_unstale":"2024-09-10T08:07:05.914683+0000","last_undegraded":"2024-09-10T08:07:05.914683+0000","last_fullsized":"2024-09-10T08:07:05.914683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:51.443798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660149+0000","last_change":"2024-09-10T08:07:05.917030+0000","last_active":"2024-09-10T08:08:03.660149+0000","last_peered":"2024-09-10T08:08:03.660149+0000","last_clean":"2024-09-10T08:08:03.660149+0000","last_became_active":"2024-09-10T08:07:05.916281+0000","last_became_peered":"2024-09-10T08:07:05.916281+0000","last_unstale":"2024-09-10T08:08:03.660149+0000","last_undegraded":"2024-09-10T08:08:03.660149+0000","last_fullsized":"2024-09-10T08:08:03.660149+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:15.060009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'4","reported_seq":181,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:30.460882+0000","last_change":"2024-09-10T08:05:22.459963+0000","last_active":"2024-09-10T08:08:30.460882+0000","last_peered":"2024-09-10T08:08:30.460882+0000","last_clean":"2024-09-10T08:08:30.460882+0000","last_became_active":"2024-09-10T08:05:22.459497+0000","last_became_peered":"2024-09-10T08:05:22.459497+0000","last_unstale":"2024-09-10T08:08:30.460882+0000","last_undegraded":"2024-09-10T08:08:30.460882+0000","last_fullsized":"2024-09-10T08:08:30.460882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:29.887980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513986+0000","last_change":"2024-09-10T08:05:35.534570+0000","last_active":"2024-09-10T08:07:05.513986+0000","last_peered":"2024-09-10T08:07:05.513986+0000","last_clean":"2024-09-10T08:07:05.513986+0000","last_became_active":"2024-09-10T08:05:35.534195+0000","last_became_peered":"2024-09-10T08:05:35.534195+0000","last_unstale":"2024-09-10T08:07:05.513986+0000","last_undegraded":"2024-09-10T08:07:05.513986+0000","last_fullsized":"2024-09-10T08:07:05.513986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:17.393491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.668227+0000","last_change":"2024-09-10T08:05:25.488551+0000","last_active":"2024-09-10T08:08:03.668227+0000","last_peered":"2024-09-10T08:08:03.668227+0000","last_clean":"2024-09-10T08:08:03.668227+0000","last_became_active":"2024-09-10T08:05:25.488086+0000","last_became_peered":"2024-09-10T08:05:25.488086+0000","last_unstale":"2024-09-10T08:08:03.668227+0000","last_undegraded":"2024-09-10T08:08:03.668227+0000","last_fullsized":"2024-09-10T08:08:03.668227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:38.044721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569070+0000","last_change":"2024-09-10T08:05:22.462269+0000","last_active":"2024-09-10T08:07:05.569070+0000","last_peered":"2024-09-10T08:07:05.569070+0000","last_clean":"2024-09-10T08:07:05.569070+0000","last_became_active":"2024-09-10T08:05:22.461924+0000","last_became_peered":"2024-09-10T08:05:22.461924+0000","last_unstale":"2024-09-10T08:07:05.569070+0000","last_undegraded":"2024-09-10T08:07:05.569070+0000","last_fullsized":"2024-09-10T08:07:05.569070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:54.996777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770191+0000","last_change":"2024-09-10T08:05:35.531596+0000","last_active":"2024-09-10T08:07:05.770191+0000","last_peered":"2024-09-10T08:07:05.770191+0000","last_clean":"2024-09-10T08:07:05.770191+0000","last_became_active":"2024-09-10T08:05:35.530179+0000","last_became_peered":"2024-09-10T08:05:35.530179+0000","last_unstale":"2024-09-10T08:07:05.770191+0000","last_undegraded":"2024-09-10T08:07:05.770191+0000","last_fullsized":"2024-09-10T08:07:05.770191+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:32.056426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.663218+0000","last_change":"2024-09-10T08:05:25.503504+0000","last_active":"2024-09-10T08:08:03.663218+0000","last_peered":"2024-09-10T08:08:03.663218+0000","last_clean":"2024-09-10T08:08:03.663218+0000","last_became_active":"2024-09-10T08:05:25.503105+0000","last_became_peered":"2024-09-10T08:05:25.503105+0000","last_unstale":"2024-09-10T08:08:03.663218+0000","last_undegraded":"2024-09-10T08:08:03.663218+0000","last_fullsized":"2024-09-10T08:08:03.663218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:40.885654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"93'6","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511623+0000","last_change":"2024-09-10T08:05:22.462271+0000","last_active":"2024-09-10T08:07:05.511623+0000","last_peered":"2024-09-10T08:07:05.511623+0000","last_clean":"2024-09-10T08:07:05.511623+0000","last_became_active":"2024-09-10T08:05:22.461195+0000","last_became_peered":"2024-09-10T08:05:22.461195+0000","last_unstale":"2024-09-10T08:07:05.511623+0000","last_undegraded":"2024-09-10T08:07:05.511623+0000","last_fullsized":"2024-09-10T08:07:05.511623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:20.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569486+0000","last_change":"2024-09-10T08:05:35.533218+0000","last_active":"2024-09-10T08:07:05.569486+0000","last_peered":"2024-09-10T08:07:05.569486+0000","last_clean":"2024-09-10T08:07:05.569486+0000","last_became_active":"2024-09-10T08:05:35.532706+0000","last_became_peered":"2024-09-10T08:05:35.532706+0000","last_unstale":"2024-09-10T08:07:05.569486+0000","last_undegraded":"2024-09-10T08:07:05.569486+0000","last_fullsized":"2024-09-10T08:07:05.569486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:22.150563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.655456+0000","last_change":"2024-09-10T08:05:25.478978+0000","last_active":"2024-09-10T08:08:03.655456+0000","last_peered":"2024-09-10T08:08:03.655456+0000","last_clean":"2024-09-10T08:08:03.655456+0000","last_became_active":"2024-09-10T08:05:25.478649+0000","last_became_peered":"2024-09-10T08:05:25.478649+0000","last_unstale":"2024-09-10T08:08:03.655456+0000","last_undegraded":"2024-09-10T08:08:03.655456+0000","last_fullsized":"2024-09-10T08:08:03.655456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:17.493446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'40","reported_seq":371,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:30.460828+0000","last_change":"2024-09-10T08:06:04.740972+0000","last_active":"2024-09-10T08:08:30.460828+0000","last_peered":"2024-09-10T08:08:30.460828+0000","last_clean":"2024-09-10T08:08:30.460828+0000","last_became_active":"2024-09-10T08:06:04.731200+0000","last_became_peered":"2024-09-10T08:06:04.731200+0000","last_unstale":"2024-09-10T08:08:30.460828+0000","last_undegraded":"2024-09-10T08:08:30.460828+0000","last_fullsized":"2024-09-10T08:08:30.460828+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:17.169708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71507,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":271,"num_read_kb":206,"num_write":41,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3703,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512781+0000","last_change":"2024-09-10T08:05:35.532594+0000","last_active":"2024-09-10T08:07:05.512781+0000","last_peered":"2024-09-10T08:07:05.512781+0000","last_clean":"2024-09-10T08:07:05.512781+0000","last_became_active":"2024-09-10T08:05:35.532132+0000","last_became_peered":"2024-09-10T08:05:35.532132+0000","last_unstale":"2024-09-10T08:07:05.512781+0000","last_undegraded":"2024-09-10T08:07:05.512781+0000","last_fullsized":"2024-09-10T08:07:05.512781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:15.348648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.666577+0000","last_change":"2024-09-10T08:05:25.485297+0000","last_active":"2024-09-10T08:08:03.666577+0000","last_peered":"2024-09-10T08:08:03.666577+0000","last_clean":"2024-09-10T08:08:03.666577+0000","last_became_active":"2024-09-10T08:05:25.484883+0000","last_became_peered":"2024-09-10T08:05:25.484883+0000","last_unstale":"2024-09-10T08:08:03.666577+0000","last_undegraded":"2024-09-10T08:08:03.666577+0000","last_fullsized":"2024-09-10T08:08:03.666577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:37.052435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"93'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569713+0000","last_change":"2024-09-10T08:05:22.568993+0000","last_active":"2024-09-10T08:07:05.569713+0000","last_peered":"2024-09-10T08:07:05.569713+0000","last_clean":"2024-09-10T08:07:05.569713+0000","last_became_active":"2024-09-10T08:05:22.568564+0000","last_became_peered":"2024-09-10T08:05:22.568564+0000","last_unstale":"2024-09-10T08:07:05.569713+0000","last_undegraded":"2024-09-10T08:07:05.569713+0000","last_fullsized":"2024-09-10T08:07:05.569713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:23.767174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914556+0000","last_change":"2024-09-10T08:05:35.524025+0000","last_active":"2024-09-10T08:07:05.914556+0000","last_peered":"2024-09-10T08:07:05.914556+0000","last_clean":"2024-09-10T08:07:05.914556+0000","last_became_active":"2024-09-10T08:05:35.523628+0000","last_became_peered":"2024-09-10T08:05:35.523628+0000","last_unstale":"2024-09-10T08:07:05.914556+0000","last_undegraded":"2024-09-10T08:07:05.914556+0000","last_fullsized":"2024-09-10T08:07:05.914556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:48.593779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.680405+0000","last_change":"2024-09-10T08:07:05.570537+0000","last_active":"2024-09-10T08:08:03.680405+0000","last_peered":"2024-09-10T08:08:03.680405+0000","last_clean":"2024-09-10T08:08:03.680405+0000","last_became_active":"2024-09-10T08:07:05.570278+0000","last_became_peered":"2024-09-10T08:07:05.570278+0000","last_unstale":"2024-09-10T08:08:03.680405+0000","last_undegraded":"2024-09-10T08:08:03.680405+0000","last_fullsized":"2024-09-10T08:08:03.680405+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:56.490462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509635+0000","last_change":"2024-09-10T08:05:22.460487+0000","last_active":"2024-09-10T08:07:05.509635+0000","last_peered":"2024-09-10T08:07:05.509635+0000","last_clean":"2024-09-10T08:07:05.509635+0000","last_became_active":"2024-09-10T08:05:22.460158+0000","last_became_peered":"2024-09-10T08:05:22.460158+0000","last_unstale":"2024-09-10T08:07:05.509635+0000","last_undegraded":"2024-09-10T08:07:05.509635+0000","last_fullsized":"2024-09-10T08:07:05.509635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:53.285513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512495+0000","last_change":"2024-09-10T08:05:35.531399+0000","last_active":"2024-09-10T08:07:05.512495+0000","last_peered":"2024-09-10T08:07:05.512495+0000","last_clean":"2024-09-10T08:07:05.512495+0000","last_became_active":"2024-09-10T08:05:35.530320+0000","last_became_peered":"2024-09-10T08:05:35.530320+0000","last_unstale":"2024-09-10T08:07:05.512495+0000","last_undegraded":"2024-09-10T08:07:05.512495+0000","last_fullsized":"2024-09-10T08:07:05.512495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:19.814046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669147+0000","last_change":"2024-09-10T08:05:25.488208+0000","last_active":"2024-09-10T08:08:03.669147+0000","last_peered":"2024-09-10T08:08:03.669147+0000","last_clean":"2024-09-10T08:08:03.669147+0000","last_became_active":"2024-09-10T08:05:25.487893+0000","last_became_peered":"2024-09-10T08:05:25.487893+0000","last_unstale":"2024-09-10T08:08:03.669147+0000","last_undegraded":"2024-09-10T08:08:03.669147+0000","last_fullsized":"2024-09-10T08:08:03.669147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:10.999416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":178,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:29.459251+0000","last_change":"2024-09-10T08:05:22.461321+0000","last_active":"2024-09-10T08:08:29.459251+0000","last_peered":"2024-09-10T08:08:29.459251+0000","last_clean":"2024-09-10T08:08:29.459251+0000","last_became_active":"2024-09-10T08:05:22.460950+0000","last_became_peered":"2024-09-10T08:05:22.460950+0000","last_unstale":"2024-09-10T08:08:29.459251+0000","last_undegraded":"2024-09-10T08:08:29.459251+0000","last_fullsized":"2024-09-10T08:08:29.459251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:55.647273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876534+0000","last_change":"2024-09-10T08:05:35.530578+0000","last_active":"2024-09-10T08:07:05.876534+0000","last_peered":"2024-09-10T08:07:05.876534+0000","last_clean":"2024-09-10T08:07:05.876534+0000","last_became_active":"2024-09-10T08:05:35.530162+0000","last_became_peered":"2024-09-10T08:05:35.530162+0000","last_unstale":"2024-09-10T08:07:05.876534+0000","last_undegraded":"2024-09-10T08:07:05.876534+0000","last_fullsized":"2024-09-10T08:07:05.876534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:19.290973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513617+0000","last_change":"2024-09-10T08:05:35.532685+0000","last_active":"2024-09-10T08:07:05.513617+0000","last_peered":"2024-09-10T08:07:05.513617+0000","last_clean":"2024-09-10T08:07:05.513617+0000","last_became_active":"2024-09-10T08:05:35.532257+0000","last_became_peered":"2024-09-10T08:05:35.532257+0000","last_unstale":"2024-09-10T08:07:05.513617+0000","last_undegraded":"2024-09-10T08:07:05.513617+0000","last_fullsized":"2024-09-10T08:07:05.513617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:07.412435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.662135+0000","last_change":"2024-09-10T08:05:25.494472+0000","last_active":"2024-09-10T08:08:03.662135+0000","last_peered":"2024-09-10T08:08:03.662135+0000","last_clean":"2024-09-10T08:08:03.662135+0000","last_became_active":"2024-09-10T08:05:25.493928+0000","last_became_peered":"2024-09-10T08:05:25.493928+0000","last_unstale":"2024-09-10T08:08:03.662135+0000","last_undegraded":"2024-09-10T08:08:03.662135+0000","last_fullsized":"2024-09-10T08:08:03.662135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:00.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"93'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568928+0000","last_change":"2024-09-10T08:05:22.460585+0000","last_active":"2024-09-10T08:07:05.568928+0000","last_peered":"2024-09-10T08:07:05.568928+0000","last_clean":"2024-09-10T08:07:05.568928+0000","last_became_active":"2024-09-10T08:05:22.460259+0000","last_became_peered":"2024-09-10T08:05:22.460259+0000","last_unstale":"2024-09-10T08:07:05.568928+0000","last_undegraded":"2024-09-10T08:07:05.568928+0000","last_fullsized":"2024-09-10T08:07:05.568928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:45.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770275+0000","last_change":"2024-09-10T08:05:35.524000+0000","last_active":"2024-09-10T08:07:05.770275+0000","last_peered":"2024-09-10T08:07:05.770275+0000","last_clean":"2024-09-10T08:07:05.770275+0000","last_became_active":"2024-09-10T08:05:35.523647+0000","last_became_peered":"2024-09-10T08:05:35.523647+0000","last_unstale":"2024-09-10T08:07:05.770275+0000","last_undegraded":"2024-09-10T08:07:05.770275+0000","last_fullsized":"2024-09-10T08:07:05.770275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:10:54.664865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660563+0000","last_change":"2024-09-10T08:05:25.499701+0000","last_active":"2024-09-10T08:08:03.660563+0000","last_peered":"2024-09-10T08:08:03.660563+0000","last_clean":"2024-09-10T08:08:03.660563+0000","last_became_active":"2024-09-10T08:05:25.496890+0000","last_became_peered":"2024-09-10T08:05:25.496890+0000","last_unstale":"2024-09-10T08:08:03.660563+0000","last_undegraded":"2024-09-10T08:08:03.660563+0000","last_fullsized":"2024-09-10T08:08:03.660563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:58.166365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914070+0000","last_change":"2024-09-10T08:05:22.460880+0000","last_active":"2024-09-10T08:07:05.914070+0000","last_peered":"2024-09-10T08:07:05.914070+0000","last_clean":"2024-09-10T08:07:05.914070+0000","last_became_active":"2024-09-10T08:05:22.459932+0000","last_became_peered":"2024-09-10T08:05:22.459932+0000","last_unstale":"2024-09-10T08:07:05.914070+0000","last_undegraded":"2024-09-10T08:07:05.914070+0000","last_fullsized":"2024-09-10T08:07:05.914070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:55.826526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915405+0000","last_change":"2024-09-10T08:05:22.460578+0000","last_active":"2024-09-10T08:07:05.915405+0000","last_peered":"2024-09-10T08:07:05.915405+0000","last_clean":"2024-09-10T08:07:05.915405+0000","last_became_active":"2024-09-10T08:05:22.460239+0000","last_became_peered":"2024-09-10T08:05:22.460239+0000","last_unstale":"2024-09-10T08:07:05.915405+0000","last_undegraded":"2024-09-10T08:07:05.915405+0000","last_fullsized":"2024-09-10T08:07:05.915405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:07.606243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.661142+0000","last_change":"2024-09-10T08:05:25.489846+0000","last_active":"2024-09-10T08:08:03.661142+0000","last_peered":"2024-09-10T08:08:03.661142+0000","last_clean":"2024-09-10T08:08:03.661142+0000","last_became_active":"2024-09-10T08:05:25.489326+0000","last_became_peered":"2024-09-10T08:05:25.489326+0000","last_unstale":"2024-09-10T08:08:03.661142+0000","last_undegraded":"2024-09-10T08:08:03.661142+0000","last_fullsized":"2024-09-10T08:08:03.661142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:43.458468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511635+0000","last_change":"2024-09-10T08:05:35.530522+0000","last_active":"2024-09-10T08:07:05.511635+0000","last_peered":"2024-09-10T08:07:05.511635+0000","last_clean":"2024-09-10T08:07:05.511635+0000","last_became_active":"2024-09-10T08:05:35.530099+0000","last_became_peered":"2024-09-10T08:05:35.530099+0000","last_unstale":"2024-09-10T08:07:05.511635+0000","last_undegraded":"2024-09-10T08:07:05.511635+0000","last_fullsized":"2024-09-10T08:07:05.511635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:58.469119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":189,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:30.461416+0000","last_change":"2024-09-10T08:05:22.461391+0000","last_active":"2024-09-10T08:08:30.461416+0000","last_peered":"2024-09-10T08:08:30.461416+0000","last_clean":"2024-09-10T08:08:30.461416+0000","last_became_active":"2024-09-10T08:05:22.461042+0000","last_became_peered":"2024-09-10T08:05:22.461042+0000","last_unstale":"2024-09-10T08:08:30.461416+0000","last_undegraded":"2024-09-10T08:08:30.461416+0000","last_fullsized":"2024-09-10T08:08:30.461416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:12.635710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1860,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.649481+0000","last_change":"2024-09-10T08:05:25.487378+0000","last_active":"2024-09-10T08:08:03.649481+0000","last_peered":"2024-09-10T08:08:03.649481+0000","last_clean":"2024-09-10T08:08:03.649481+0000","last_became_active":"2024-09-10T08:05:25.487055+0000","last_became_peered":"2024-09-10T08:05:25.487055+0000","last_unstale":"2024-09-10T08:08:03.649481+0000","last_undegraded":"2024-09-10T08:08:03.649481+0000","last_fullsized":"2024-09-10T08:08:03.649481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:59.522014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511748+0000","last_change":"2024-09-10T08:05:35.522765+0000","last_active":"2024-09-10T08:07:05.511748+0000","last_peered":"2024-09-10T08:07:05.511748+0000","last_clean":"2024-09-10T08:07:05.511748+0000","last_became_active":"2024-09-10T08:05:35.522359+0000","last_became_peered":"2024-09-10T08:05:35.522359+0000","last_unstale":"2024-09-10T08:07:05.511748+0000","last_undegraded":"2024-09-10T08:07:05.511748+0000","last_fullsized":"2024-09-10T08:07:05.511748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:15.280867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.652869+0000","last_change":"2024-09-10T08:05:25.475388+0000","last_active":"2024-09-10T08:08:03.652869+0000","last_peered":"2024-09-10T08:08:03.652869+0000","last_clean":"2024-09-10T08:08:03.652869+0000","last_became_active":"2024-09-10T08:05:25.474331+0000","last_became_peered":"2024-09-10T08:05:25.474331+0000","last_unstale":"2024-09-10T08:08:03.652869+0000","last_undegraded":"2024-09-10T08:08:03.652869+0000","last_fullsized":"2024-09-10T08:08:03.652869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:58.713148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656383+0000","last_change":"2024-09-10T08:07:05.570505+0000","last_active":"2024-09-10T08:08:03.656383+0000","last_peered":"2024-09-10T08:08:03.656383+0000","last_clean":"2024-09-10T08:08:03.656383+0000","last_became_active":"2024-09-10T08:07:05.569970+0000","last_became_peered":"2024-09-10T08:07:05.569970+0000","last_unstale":"2024-09-10T08:08:03.656383+0000","last_undegraded":"2024-09-10T08:08:03.656383+0000","last_fullsized":"2024-09-10T08:08:03.656383+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:13.282503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671951+0000","last_change":"2024-09-10T08:05:25.491585+0000","last_active":"2024-09-10T08:08:03.671951+0000","last_peered":"2024-09-10T08:08:03.671951+0000","last_clean":"2024-09-10T08:08:03.671951+0000","last_became_active":"2024-09-10T08:05:25.491224+0000","last_became_peered":"2024-09-10T08:05:25.491224+0000","last_unstale":"2024-09-10T08:08:03.671951+0000","last_undegraded":"2024-09-10T08:08:03.671951+0000","last_fullsized":"2024-09-10T08:08:03.671951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:56.066808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681502+0000","last_change":"2024-09-10T08:05:25.499578+0000","last_active":"2024-09-10T08:08:03.681502+0000","last_peered":"2024-09-10T08:08:03.681502+0000","last_clean":"2024-09-10T08:08:03.681502+0000","last_became_active":"2024-09-10T08:05:25.498445+0000","last_became_peered":"2024-09-10T08:05:25.498445+0000","last_unstale":"2024-09-10T08:08:03.681502+0000","last_undegraded":"2024-09-10T08:08:03.681502+0000","last_fullsized":"2024-09-10T08:08:03.681502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:58.961699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675717+0000","last_change":"2024-09-10T08:06:04.735764+0000","last_active":"2024-09-10T08:08:03.675717+0000","last_peered":"2024-09-10T08:08:03.675717+0000","last_clean":"2024-09-10T08:08:03.675717+0000","last_became_active":"2024-09-10T08:06:04.735366+0000","last_became_peered":"2024-09-10T08:06:04.735366+0000","last_unstale":"2024-09-10T08:08:03.675717+0000","last_undegraded":"2024-09-10T08:08:03.675717+0000","last_fullsized":"2024-09-10T08:08:03.675717+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:03.757890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651587+0000","last_change":"2024-09-10T08:05:25.475872+0000","last_active":"2024-09-10T08:08:03.651587+0000","last_peered":"2024-09-10T08:08:03.651587+0000","last_clean":"2024-09-10T08:08:03.651587+0000","last_became_active":"2024-09-10T08:05:25.475549+0000","last_became_peered":"2024-09-10T08:05:25.475549+0000","last_unstale":"2024-09-10T08:08:03.651587+0000","last_undegraded":"2024-09-10T08:08:03.651587+0000","last_fullsized":"2024-09-10T08:08:03.651587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:37.575267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.664667+0000","last_change":"2024-09-10T08:05:25.474210+0000","last_active":"2024-09-10T08:08:03.664667+0000","last_peered":"2024-09-10T08:08:03.664667+0000","last_clean":"2024-09-10T08:08:03.664667+0000","last_became_active":"2024-09-10T08:05:25.473882+0000","last_became_peered":"2024-09-10T08:05:25.473882+0000","last_unstale":"2024-09-10T08:08:03.664667+0000","last_undegraded":"2024-09-10T08:08:03.664667+0000","last_fullsized":"2024-09-10T08:08:03.664667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:05.164370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651519+0000","last_change":"2024-09-10T08:05:25.490406+0000","last_active":"2024-09-10T08:08:03.651519+0000","last_peered":"2024-09-10T08:08:03.651519+0000","last_clean":"2024-09-10T08:08:03.651519+0000","last_became_active":"2024-09-10T08:05:25.490061+0000","last_became_peered":"2024-09-10T08:05:25.490061+0000","last_unstale":"2024-09-10T08:08:03.651519+0000","last_undegraded":"2024-09-10T08:08:03.651519+0000","last_fullsized":"2024-09-10T08:08:03.651519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:12.988229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.664045+0000","last_change":"2024-09-10T08:05:25.504049+0000","last_active":"2024-09-10T08:08:03.664045+0000","last_peered":"2024-09-10T08:08:03.664045+0000","last_clean":"2024-09-10T08:08:03.664045+0000","last_became_active":"2024-09-10T08:05:25.503770+0000","last_became_peered":"2024-09-10T08:05:25.503770+0000","last_unstale":"2024-09-10T08:08:03.664045+0000","last_undegraded":"2024-09-10T08:08:03.664045+0000","last_fullsized":"2024-09-10T08:08:03.664045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:52.850945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.672614+0000","last_change":"2024-09-10T08:05:25.491682+0000","last_active":"2024-09-10T08:08:03.672614+0000","last_peered":"2024-09-10T08:08:03.672614+0000","last_clean":"2024-09-10T08:08:03.672614+0000","last_became_active":"2024-09-10T08:05:25.491296+0000","last_became_peered":"2024-09-10T08:05:25.491296+0000","last_unstale":"2024-09-10T08:08:03.672614+0000","last_undegraded":"2024-09-10T08:08:03.672614+0000","last_fullsized":"2024-09-10T08:08:03.672614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:30.075842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.673509+0000","last_change":"2024-09-10T08:05:25.489648+0000","last_active":"2024-09-10T08:08:03.673509+0000","last_peered":"2024-09-10T08:08:03.673509+0000","last_clean":"2024-09-10T08:08:03.673509+0000","last_became_active":"2024-09-10T08:05:25.489267+0000","last_became_peered":"2024-09-10T08:05:25.489267+0000","last_unstale":"2024-09-10T08:08:03.673509+0000","last_undegraded":"2024-09-10T08:08:03.673509+0000","last_fullsized":"2024-09-10T08:08:03.673509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:21.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681690+0000","last_change":"2024-09-10T08:05:25.497862+0000","last_active":"2024-09-10T08:08:03.681690+0000","last_peered":"2024-09-10T08:08:03.681690+0000","last_clean":"2024-09-10T08:08:03.681690+0000","last_became_active":"2024-09-10T08:05:25.497504+0000","last_became_peered":"2024-09-10T08:05:25.497504+0000","last_unstale":"2024-09-10T08:08:03.681690+0000","last_undegraded":"2024-09-10T08:08:03.681690+0000","last_fullsized":"2024-09-10T08:08:03.681690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:40.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654484+0000","last_change":"2024-09-10T08:05:25.485924+0000","last_active":"2024-09-10T08:08:03.654484+0000","last_peered":"2024-09-10T08:08:03.654484+0000","last_clean":"2024-09-10T08:08:03.654484+0000","last_became_active":"2024-09-10T08:05:25.485545+0000","last_became_peered":"2024-09-10T08:05:25.485545+0000","last_unstale":"2024-09-10T08:08:03.654484+0000","last_undegraded":"2024-09-10T08:08:03.654484+0000","last_fullsized":"2024-09-10T08:08:03.654484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:33:20.233720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671966+0000","last_change":"2024-09-10T08:05:25.491911+0000","last_active":"2024-09-10T08:08:03.671966+0000","last_peered":"2024-09-10T08:08:03.671966+0000","last_clean":"2024-09-10T08:08:03.671966+0000","last_became_active":"2024-09-10T08:05:25.491330+0000","last_became_peered":"2024-09-10T08:05:25.491330+0000","last_unstale":"2024-09-10T08:08:03.671966+0000","last_undegraded":"2024-09-10T08:08:03.671966+0000","last_fullsized":"2024-09-10T08:08:03.671966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:06.043331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.655352+0000","last_change":"2024-09-10T08:05:25.492220+0000","last_active":"2024-09-10T08:08:03.655352+0000","last_peered":"2024-09-10T08:08:03.655352+0000","last_clean":"2024-09-10T08:08:03.655352+0000","last_became_active":"2024-09-10T08:05:25.491473+0000","last_became_peered":"2024-09-10T08:05:25.491473+0000","last_unstale":"2024-09-10T08:08:03.655352+0000","last_undegraded":"2024-09-10T08:08:03.655352+0000","last_fullsized":"2024-09-10T08:08:03.655352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:13:30.278434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676184+0000","last_change":"2024-09-10T08:05:25.488927+0000","last_active":"2024-09-10T08:08:03.676184+0000","last_peered":"2024-09-10T08:08:03.676184+0000","last_clean":"2024-09-10T08:08:03.676184+0000","last_became_active":"2024-09-10T08:05:25.488550+0000","last_became_peered":"2024-09-10T08:05:25.488550+0000","last_unstale":"2024-09-10T08:08:03.676184+0000","last_undegraded":"2024-09-10T08:08:03.676184+0000","last_fullsized":"2024-09-10T08:08:03.676184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:44.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.677654+0000","last_change":"2024-09-10T08:05:25.490125+0000","last_active":"2024-09-10T08:08:03.677654+0000","last_peered":"2024-09-10T08:08:03.677654+0000","last_clean":"2024-09-10T08:08:03.677654+0000","last_became_active":"2024-09-10T08:05:25.487805+0000","last_became_peered":"2024-09-10T08:05:25.487805+0000","last_unstale":"2024-09-10T08:08:03.677654+0000","last_undegraded":"2024-09-10T08:08:03.677654+0000","last_fullsized":"2024-09-10T08:08:03.677654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:33.479009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676794+0000","last_change":"2024-09-10T08:05:25.500346+0000","last_active":"2024-09-10T08:08:03.676794+0000","last_peered":"2024-09-10T08:08:03.676794+0000","last_clean":"2024-09-10T08:08:03.676794+0000","last_became_active":"2024-09-10T08:05:25.498289+0000","last_became_peered":"2024-09-10T08:05:25.498289+0000","last_unstale":"2024-09-10T08:08:03.676794+0000","last_undegraded":"2024-09-10T08:08:03.676794+0000","last_fullsized":"2024-09-10T08:08:03.676794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:12.054926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676196+0000","last_change":"2024-09-10T08:05:25.490001+0000","last_active":"2024-09-10T08:08:03.676196+0000","last_peered":"2024-09-10T08:08:03.676196+0000","last_clean":"2024-09-10T08:08:03.676196+0000","last_became_active":"2024-09-10T08:05:25.489706+0000","last_became_peered":"2024-09-10T08:05:25.489706+0000","last_unstale":"2024-09-10T08:08:03.676196+0000","last_undegraded":"2024-09-10T08:08:03.676196+0000","last_fullsized":"2024-09-10T08:08:03.676196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:38.822140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669334+0000","last_change":"2024-09-10T08:05:25.493874+0000","last_active":"2024-09-10T08:08:03.669334+0000","last_peered":"2024-09-10T08:08:03.669334+0000","last_clean":"2024-09-10T08:08:03.669334+0000","last_became_active":"2024-09-10T08:05:25.489911+0000","last_became_peered":"2024-09-10T08:05:25.489911+0000","last_unstale":"2024-09-10T08:08:03.669334+0000","last_undegraded":"2024-09-10T08:08:03.669334+0000","last_fullsized":"2024-09-10T08:08:03.669334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:46.509657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654960+0000","last_change":"2024-09-10T08:05:25.490277+0000","last_active":"2024-09-10T08:08:03.654960+0000","last_peered":"2024-09-10T08:08:03.654960+0000","last_clean":"2024-09-10T08:08:03.654960+0000","last_became_active":"2024-09-10T08:05:25.486863+0000","last_became_peered":"2024-09-10T08:05:25.486863+0000","last_unstale":"2024-09-10T08:08:03.654960+0000","last_undegraded":"2024-09-10T08:08:03.654960+0000","last_fullsized":"2024-09-10T08:08:03.654960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:21.353830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656794+0000","last_change":"2024-09-10T08:05:25.491028+0000","last_active":"2024-09-10T08:08:03.656794+0000","last_peered":"2024-09-10T08:08:03.656794+0000","last_clean":"2024-09-10T08:08:03.656794+0000","last_became_active":"2024-09-10T08:05:25.490452+0000","last_became_peered":"2024-09-10T08:05:25.490452+0000","last_unstale":"2024-09-10T08:08:03.656794+0000","last_undegraded":"2024-09-10T08:08:03.656794+0000","last_fullsized":"2024-09-10T08:08:03.656794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:53.059077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660628+0000","last_change":"2024-09-10T08:05:25.489991+0000","last_active":"2024-09-10T08:08:03.660628+0000","last_peered":"2024-09-10T08:08:03.660628+0000","last_clean":"2024-09-10T08:08:03.660628+0000","last_became_active":"2024-09-10T08:05:25.489662+0000","last_became_peered":"2024-09-10T08:05:25.489662+0000","last_unstale":"2024-09-10T08:08:03.660628+0000","last_undegraded":"2024-09-10T08:08:03.660628+0000","last_fullsized":"2024-09-10T08:08:03.660628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:47.940712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667243+0000","last_change":"2024-09-10T08:05:25.475752+0000","last_active":"2024-09-10T08:08:03.667243+0000","last_peered":"2024-09-10T08:08:03.667243+0000","last_clean":"2024-09-10T08:08:03.667243+0000","last_became_active":"2024-09-10T08:05:25.475395+0000","last_became_peered":"2024-09-10T08:05:25.475395+0000","last_unstale":"2024-09-10T08:08:03.667243+0000","last_undegraded":"2024-09-10T08:08:03.667243+0000","last_fullsized":"2024-09-10T08:08:03.667243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.240154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.672245+0000","last_change":"2024-09-10T08:05:25.503061+0000","last_active":"2024-09-10T08:08:03.672245+0000","last_peered":"2024-09-10T08:08:03.672245+0000","last_clean":"2024-09-10T08:08:03.672245+0000","last_became_active":"2024-09-10T08:05:25.501333+0000","last_became_peered":"2024-09-10T08:05:25.501333+0000","last_unstale":"2024-09-10T08:08:03.672245+0000","last_undegraded":"2024-09-10T08:08:03.672245+0000","last_fullsized":"2024-09-10T08:08:03.672245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:38.254777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.661256+0000","last_change":"2024-09-10T08:07:05.514631+0000","last_active":"2024-09-10T08:08:03.661256+0000","last_peered":"2024-09-10T08:08:03.661256+0000","last_clean":"2024-09-10T08:08:03.661256+0000","last_became_active":"2024-09-10T08:07:05.514285+0000","last_became_peered":"2024-09-10T08:07:05.514285+0000","last_unstale":"2024-09-10T08:08:03.661256+0000","last_undegraded":"2024-09-10T08:08:03.661256+0000","last_fullsized":"2024-09-10T08:08:03.661256+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:40.293121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660092+0000","last_change":"2024-09-10T08:05:25.491457+0000","last_active":"2024-09-10T08:08:03.660092+0000","last_peered":"2024-09-10T08:08:03.660092+0000","last_clean":"2024-09-10T08:08:03.660092+0000","last_became_active":"2024-09-10T08:05:25.491162+0000","last_became_peered":"2024-09-10T08:05:25.491162+0000","last_unstale":"2024-09-10T08:08:03.660092+0000","last_undegraded":"2024-09-10T08:08:03.660092+0000","last_fullsized":"2024-09-10T08:08:03.660092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:14.389752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660737+0000","last_change":"2024-09-10T08:05:25.487874+0000","last_active":"2024-09-10T08:08:03.660737+0000","last_peered":"2024-09-10T08:08:03.660737+0000","last_clean":"2024-09-10T08:08:03.660737+0000","last_became_active":"2024-09-10T08:05:25.487497+0000","last_became_peered":"2024-09-10T08:05:25.487497+0000","last_unstale":"2024-09-10T08:08:03.660737+0000","last_undegraded":"2024-09-10T08:08:03.660737+0000","last_fullsized":"2024-09-10T08:08:03.660737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:26.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658875+0000","last_change":"2024-09-10T08:05:25.490018+0000","last_active":"2024-09-10T08:08:03.658875+0000","last_peered":"2024-09-10T08:08:03.658875+0000","last_clean":"2024-09-10T08:08:03.658875+0000","last_became_active":"2024-09-10T08:05:25.484951+0000","last_became_peered":"2024-09-10T08:05:25.484951+0000","last_unstale":"2024-09-10T08:08:03.658875+0000","last_undegraded":"2024-09-10T08:08:03.658875+0000","last_fullsized":"2024-09-10T08:08:03.658875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:07.516154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.677343+0000","last_change":"2024-09-10T08:05:25.501581+0000","last_active":"2024-09-10T08:08:03.677343+0000","last_peered":"2024-09-10T08:08:03.677343+0000","last_clean":"2024-09-10T08:08:03.677343+0000","last_became_active":"2024-09-10T08:05:25.499690+0000","last_became_peered":"2024-09-10T08:05:25.499690+0000","last_unstale":"2024-09-10T08:08:03.677343+0000","last_undegraded":"2024-09-10T08:08:03.677343+0000","last_fullsized":"2024-09-10T08:08:03.677343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:14.569791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.678243+0000","last_change":"2024-09-10T08:05:25.487945+0000","last_active":"2024-09-10T08:08:03.678243+0000","last_peered":"2024-09-10T08:08:03.678243+0000","last_clean":"2024-09-10T08:08:03.678243+0000","last_became_active":"2024-09-10T08:05:25.487655+0000","last_became_peered":"2024-09-10T08:05:25.487655+0000","last_unstale":"2024-09-10T08:08:03.678243+0000","last_undegraded":"2024-09-10T08:08:03.678243+0000","last_fullsized":"2024-09-10T08:08:03.678243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:37.746037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675926+0000","last_change":"2024-09-10T08:05:25.489348+0000","last_active":"2024-09-10T08:08:03.675926+0000","last_peered":"2024-09-10T08:08:03.675926+0000","last_clean":"2024-09-10T08:08:03.675926+0000","last_became_active":"2024-09-10T08:05:25.489004+0000","last_became_peered":"2024-09-10T08:05:25.489004+0000","last_unstale":"2024-09-10T08:08:03.675926+0000","last_undegraded":"2024-09-10T08:08:03.675926+0000","last_fullsized":"2024-09-10T08:08:03.675926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:54.554719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76748,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":814,"num_read_kb":489,"num_write":190,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3725,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":230244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214976,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514548,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":548036,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743481,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32364,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712788,"statfs":{"total":95995035648,"available":95961894912,"internally_reserved":0,"allocated":1208320,"data_stored":1211278,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907004,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":548250,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135940,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":548148,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364871,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1114112,"data_stored":1145319,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528398,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":543426,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790037,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1122304,"data_stored":1140514,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953561,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":619976,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":543620,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":545924,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":614843,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":544814,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":73478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:08:36.270 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T08:08:36.270 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:08:36.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:36 smithi002 ceph-mon[30563]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:36.651 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:36.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:36 smithi155 ceph-mon[35405]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:36.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:36 smithi107 ceph-mon[35048]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:37.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.272+0000 7fa227000640 1 -- 172.21.15.2:0/2679890468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2201049a0 msgr2=0x7fa2201089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:37.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.272+0000 7fa227000640 1 --2- 172.21.15.2:0/2679890468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2201049a0 0x7fa2201089f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009900 tx=0x7fa20c02a410 comp rx=0 tx=0).stop 2024-09-10T08:08:37.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.272+0000 7fa227000640 1 -- 172.21.15.2:0/2679890468 shutdown_connections 2024-09-10T08:08:37.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.272+0000 7fa227000640 1 --2- 172.21.15.2:0/2679890468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa220109120 0x7fa220110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:37.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.272+0000 7fa227000640 1 --2- 172.21.15.2:0/2679890468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2201049a0 0x7fa2201089f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:37.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.272+0000 7fa227000640 1 --2- 172.21.15.2:0/2679890468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa220103ff0 0x7fa2201043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:37.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.272+0000 7fa227000640 1 -- 172.21.15.2:0/2679890468 >> 172.21.15.2:0/2679890468 conn(0x7fa2200ffa90 msgr2=0x7fa220101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:37.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.272+0000 7fa227000640 1 -- 172.21.15.2:0/2679890468 shutdown_connections 2024-09-10T08:08:37.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.272+0000 7fa227000640 1 -- 172.21.15.2:0/2679890468 wait complete. 2024-09-10T08:08:37.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.273+0000 7fa227000640 1 Processor -- start 2024-09-10T08:08:37.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.273+0000 7fa227000640 1 -- start start 2024-09-10T08:08:37.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.273+0000 7fa227000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa220103ff0 0x7fa22019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:37.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.273+0000 7fa227000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2201049a0 0x7fa22019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:37.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.273+0000 7fa227000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa220109120 0x7fa22019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:37.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.273+0000 7fa227000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa220114140 con 0x7fa220109120 2024-09-10T08:08:37.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.273+0000 7fa227000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113fc0 con 0x7fa220103ff0 2024-09-10T08:08:37.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.273+0000 7fa227000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201142c0 con 0x7fa2201049a0 2024-09-10T08:08:37.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.274+0000 7fa225000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa220109120 0x7fa22019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:37.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.274+0000 7fa21f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2201049a0 0x7fa22019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:37.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.274+0000 7fa21fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa220103ff0 0x7fa22019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:37.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.274+0000 7fa225000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa220109120 0x7fa22019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40340/0 (socket says 172.21.15.2:40340) 2024-09-10T08:08:37.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.274+0000 7fa225000640 1 -- 172.21.15.2:0/544864753 learned_addr learned my addr 172.21.15.2:0/544864753 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:37.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.274+0000 7fa225000640 1 -- 172.21.15.2:0/544864753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2201049a0 msgr2=0x7fa22019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:37.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.274+0000 7fa225000640 1 --2- 172.21.15.2:0/544864753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2201049a0 0x7fa22019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:37.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.274+0000 7fa225000640 1 -- 172.21.15.2:0/544864753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa220103ff0 msgr2=0x7fa22019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:37.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.274+0000 7fa225000640 1 --2- 172.21.15.2:0/544864753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa220103ff0 0x7fa22019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:37.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.274+0000 7fa225000640 1 -- 172.21.15.2:0/544864753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22019f5d0 con 0x7fa220109120 2024-09-10T08:08:37.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.275+0000 7fa225000640 1 --2- 172.21.15.2:0/544864753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa220109120 0x7fa22019eef0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa218009fd0 tx=0x7fa21800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:37.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.275+0000 7fa21cc00640 1 -- 172.21.15.2:0/544864753 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218017070 con 0x7fa220109120 2024-09-10T08:08:37.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.275+0000 7fa21cc00640 1 -- 172.21.15.2:0/544864753 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa218003e50 con 0x7fa220109120 2024-09-10T08:08:37.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.275+0000 7fa21cc00640 1 -- 172.21.15.2:0/544864753 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2180106c0 con 0x7fa220109120 2024-09-10T08:08:37.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.275+0000 7fa227000640 1 -- 172.21.15.2:0/544864753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22019f8c0 con 0x7fa220109120 2024-09-10T08:08:37.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.275+0000 7fa227000640 1 -- 172.21.15.2:0/544864753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2201a71a0 con 0x7fa220109120 2024-09-10T08:08:37.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.276+0000 7fa227000640 1 -- 172.21.15.2:0/544864753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2201043d0 con 0x7fa220109120 2024-09-10T08:08:37.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.278+0000 7fa21cc00640 1 -- 172.21.15.2:0/544864753 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fa218003970 con 0x7fa220109120 2024-09-10T08:08:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.280+0000 7fa21cc00640 1 --2- 172.21.15.2:0/544864753 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa2040a2c90 0x7fa2040a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.280+0000 7fa21cc00640 1 -- 172.21.15.2:0/544864753 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa2180cf1d0 con 0x7fa220109120 2024-09-10T08:08:37.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.280+0000 7fa21fe00640 1 --2- 172.21.15.2:0/544864753 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa2040a2c90 0x7fa2040a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:37.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.281+0000 7fa21fe00640 1 --2- 172.21.15.2:0/544864753 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa2040a2c90 0x7fa2040a5150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa210009bd0 tx=0x7fa210005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:37.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.282+0000 7fa21cc00640 1 -- 172.21.15.2:0/544864753 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa21809b5b0 con 0x7fa220109120 2024-09-10T08:08:37.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.468+0000 7fa227000640 1 -- 172.21.15.2:0/544864753 --> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa22010c3d0 con 0x7fa2040a2c90 2024-09-10T08:08:37.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.477+0000 7fa21cc00640 1 -- 172.21.15.2:0/544864753 <== mgr.14150 v2:172.21.15.2:6800/3751597725 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340722 (secure 0 0 0) 0x7fa210004030 con 0x7fa2040a2c90 2024-09-10T08:08:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:37.482 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-10T08:08:37.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.481+0000 7fa227000640 1 -- 172.21.15.2:0/544864753 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa2040a2c90 msgr2=0x7fa2040a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:37.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.481+0000 7fa227000640 1 --2- 172.21.15.2:0/544864753 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa2040a2c90 0x7fa2040a5150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa210009bd0 tx=0x7fa210005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:37.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.481+0000 7fa227000640 1 -- 172.21.15.2:0/544864753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa220109120 msgr2=0x7fa22019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:37.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.481+0000 7fa227000640 1 --2- 172.21.15.2:0/544864753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa220109120 0x7fa22019eef0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa218009fd0 tx=0x7fa21800b260 comp rx=0 tx=0).stop 2024-09-10T08:08:37.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.482+0000 7fa227000640 1 -- 172.21.15.2:0/544864753 shutdown_connections 2024-09-10T08:08:37.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.482+0000 7fa227000640 1 --2- 172.21.15.2:0/544864753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa220109120 0x7fa22019eef0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:37.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.482+0000 7fa227000640 1 --2- 172.21.15.2:0/544864753 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa2040a2c90 0x7fa2040a5150 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:37.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.482+0000 7fa227000640 1 --2- 172.21.15.2:0/544864753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2201049a0 0x7fa22019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:37.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.482+0000 7fa227000640 1 --2- 172.21.15.2:0/544864753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa220103ff0 0x7fa22019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:37.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.482+0000 7fa227000640 1 -- 172.21.15.2:0/544864753 >> 172.21.15.2:0/544864753 conn(0x7fa2200ffa90 msgr2=0x7fa220100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:37.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.482+0000 7fa227000640 1 -- 172.21.15.2:0/544864753 shutdown_connections 2024-09-10T08:08:37.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:37.482+0000 7fa227000640 1 -- 172.21.15.2:0/544864753 wait complete. 2024-09-10T08:08:37.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:37 smithi002 ceph-mon[30563]: pgmap v401: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:08:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:37 smithi155 ceph-mon[35405]: pgmap v401: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:08:37.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:37 smithi107 ceph-mon[35048]: pgmap v401: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:08:38.383 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-10T08:08:35.696105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667350,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":957,"num_read_kb":610,"num_write":373,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1184461,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":800,"ondisk_log_size":800,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362060,"kb_used_data":12812,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579764,"statfs":{"total":1151940427776,"available":1151569678336,"internally_reserved":0,"allocated":13119488,"data_stored":8554148,"data_compressed":497802,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2520,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":45,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003080"},"pg_stats":[{"pgid":"2.1f","version":"93'9","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:29.118945+0000","last_change":"2024-09-10T08:05:22.462062+0000","last_active":"2024-09-10T08:07:29.118945+0000","last_peered":"2024-09-10T08:07:29.118945+0000","last_clean":"2024-09-10T08:07:29.118945+0000","last_became_active":"2024-09-10T08:05:22.460973+0000","last_became_peered":"2024-09-10T08:05:22.460973+0000","last_unstale":"2024-09-10T08:07:29.118945+0000","last_undegraded":"2024-09-10T08:07:29.118945+0000","last_fullsized":"2024-09-10T08:07:29.118945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:01.357446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656803+0000","last_change":"2024-09-10T08:05:25.482873+0000","last_active":"2024-09-10T08:08:03.656803+0000","last_peered":"2024-09-10T08:08:03.656803+0000","last_clean":"2024-09-10T08:08:03.656803+0000","last_became_active":"2024-09-10T08:05:25.482559+0000","last_became_peered":"2024-09-10T08:05:25.482559+0000","last_unstale":"2024-09-10T08:08:03.656803+0000","last_undegraded":"2024-09-10T08:08:03.656803+0000","last_fullsized":"2024-09-10T08:08:03.656803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:10:17.366160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511384+0000","last_change":"2024-09-10T08:06:04.736356+0000","last_active":"2024-09-10T08:07:05.511384+0000","last_peered":"2024-09-10T08:07:05.511384+0000","last_clean":"2024-09-10T08:07:05.511384+0000","last_became_active":"2024-09-10T08:06:04.735859+0000","last_became_peered":"2024-09-10T08:06:04.735859+0000","last_unstale":"2024-09-10T08:07:05.511384+0000","last_undegraded":"2024-09-10T08:07:05.511384+0000","last_fullsized":"2024-09-10T08:07:05.511384+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:56.411678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"93'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.877433+0000","last_change":"2024-09-10T08:05:22.569504+0000","last_active":"2024-09-10T08:07:05.877433+0000","last_peered":"2024-09-10T08:07:05.877433+0000","last_clean":"2024-09-10T08:07:05.877433+0000","last_became_active":"2024-09-10T08:05:22.569164+0000","last_became_peered":"2024-09-10T08:05:22.569164+0000","last_unstale":"2024-09-10T08:07:05.877433+0000","last_undegraded":"2024-09-10T08:07:05.877433+0000","last_fullsized":"2024-09-10T08:07:05.877433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:41.628963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.680208+0000","last_change":"2024-09-10T08:05:25.493017+0000","last_active":"2024-09-10T08:08:03.680208+0000","last_peered":"2024-09-10T08:08:03.680208+0000","last_clean":"2024-09-10T08:08:03.680208+0000","last_became_active":"2024-09-10T08:05:25.491370+0000","last_became_peered":"2024-09-10T08:05:25.491370+0000","last_unstale":"2024-09-10T08:08:03.680208+0000","last_undegraded":"2024-09-10T08:08:03.680208+0000","last_fullsized":"2024-09-10T08:08:03.680208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:54:41.616549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914087+0000","last_change":"2024-09-10T08:05:35.523751+0000","last_active":"2024-09-10T08:07:05.914087+0000","last_peered":"2024-09-10T08:07:05.914087+0000","last_clean":"2024-09-10T08:07:05.914087+0000","last_became_active":"2024-09-10T08:05:35.523372+0000","last_became_peered":"2024-09-10T08:05:35.523372+0000","last_unstale":"2024-09-10T08:07:05.914087+0000","last_undegraded":"2024-09-10T08:07:05.914087+0000","last_fullsized":"2024-09-10T08:07:05.914087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:15:20.971596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508934+0000","last_change":"2024-09-10T08:05:22.460590+0000","last_active":"2024-09-10T08:07:05.508934+0000","last_peered":"2024-09-10T08:07:05.508934+0000","last_clean":"2024-09-10T08:07:05.508934+0000","last_became_active":"2024-09-10T08:05:22.460238+0000","last_became_peered":"2024-09-10T08:05:22.460238+0000","last_unstale":"2024-09-10T08:07:05.508934+0000","last_undegraded":"2024-09-10T08:07:05.508934+0000","last_fullsized":"2024-09-10T08:07:05.508934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:18.459509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669035+0000","last_change":"2024-09-10T08:05:25.487392+0000","last_active":"2024-09-10T08:08:03.669035+0000","last_peered":"2024-09-10T08:08:03.669035+0000","last_clean":"2024-09-10T08:08:03.669035+0000","last_became_active":"2024-09-10T08:05:25.487054+0000","last_became_peered":"2024-09-10T08:05:25.487054+0000","last_unstale":"2024-09-10T08:08:03.669035+0000","last_undegraded":"2024-09-10T08:08:03.669035+0000","last_fullsized":"2024-09-10T08:08:03.669035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:16.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914818+0000","last_change":"2024-09-10T08:06:04.735778+0000","last_active":"2024-09-10T08:07:05.914818+0000","last_peered":"2024-09-10T08:07:05.914818+0000","last_clean":"2024-09-10T08:07:05.914818+0000","last_became_active":"2024-09-10T08:06:04.735208+0000","last_became_peered":"2024-09-10T08:06:04.735208+0000","last_unstale":"2024-09-10T08:07:05.914818+0000","last_undegraded":"2024-09-10T08:07:05.914818+0000","last_fullsized":"2024-09-10T08:07:05.914818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:36.982246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508944+0000","last_change":"2024-09-10T08:05:22.458323+0000","last_active":"2024-09-10T08:07:05.508944+0000","last_peered":"2024-09-10T08:07:05.508944+0000","last_clean":"2024-09-10T08:07:05.508944+0000","last_became_active":"2024-09-10T08:05:22.457965+0000","last_became_peered":"2024-09-10T08:05:22.457965+0000","last_unstale":"2024-09-10T08:07:05.508944+0000","last_undegraded":"2024-09-10T08:07:05.508944+0000","last_fullsized":"2024-09-10T08:07:05.508944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:15.021305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.663997+0000","last_change":"2024-09-10T08:05:25.489883+0000","last_active":"2024-09-10T08:08:03.663997+0000","last_peered":"2024-09-10T08:08:03.663997+0000","last_clean":"2024-09-10T08:08:03.663997+0000","last_became_active":"2024-09-10T08:05:25.489482+0000","last_became_peered":"2024-09-10T08:05:25.489482+0000","last_unstale":"2024-09-10T08:08:03.663997+0000","last_undegraded":"2024-09-10T08:08:03.663997+0000","last_fullsized":"2024-09-10T08:08:03.663997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:52.080628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914064+0000","last_change":"2024-09-10T08:05:35.531458+0000","last_active":"2024-09-10T08:07:05.914064+0000","last_peered":"2024-09-10T08:07:05.914064+0000","last_clean":"2024-09-10T08:07:05.914064+0000","last_became_active":"2024-09-10T08:05:35.531072+0000","last_became_peered":"2024-09-10T08:05:35.531072+0000","last_unstale":"2024-09-10T08:07:05.914064+0000","last_undegraded":"2024-09-10T08:07:05.914064+0000","last_fullsized":"2024-09-10T08:07:05.914064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:04.792664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667650+0000","last_change":"2024-09-10T08:05:25.486865+0000","last_active":"2024-09-10T08:08:03.667650+0000","last_peered":"2024-09-10T08:08:03.667650+0000","last_clean":"2024-09-10T08:08:03.667650+0000","last_became_active":"2024-09-10T08:05:25.486531+0000","last_became_peered":"2024-09-10T08:05:25.486531+0000","last_unstale":"2024-09-10T08:08:03.667650+0000","last_undegraded":"2024-09-10T08:08:03.667650+0000","last_fullsized":"2024-09-10T08:08:03.667650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:26.854682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"93'4","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508917+0000","last_change":"2024-09-10T08:06:04.743303+0000","last_active":"2024-09-10T08:07:05.508917+0000","last_peered":"2024-09-10T08:07:05.508917+0000","last_clean":"2024-09-10T08:07:05.508917+0000","last_became_active":"2024-09-10T08:06:04.733294+0000","last_became_peered":"2024-09-10T08:06:04.733294+0000","last_unstale":"2024-09-10T08:07:05.508917+0000","last_undegraded":"2024-09-10T08:07:05.508917+0000","last_fullsized":"2024-09-10T08:07:05.508917+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:18.542658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914822+0000","last_change":"2024-09-10T08:05:35.642083+0000","last_active":"2024-09-10T08:07:05.914822+0000","last_peered":"2024-09-10T08:07:05.914822+0000","last_clean":"2024-09-10T08:07:05.914822+0000","last_became_active":"2024-09-10T08:05:35.641214+0000","last_became_peered":"2024-09-10T08:05:35.641214+0000","last_unstale":"2024-09-10T08:07:05.914822+0000","last_undegraded":"2024-09-10T08:07:05.914822+0000","last_fullsized":"2024-09-10T08:07:05.914822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:00:21.731769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681852+0000","last_change":"2024-09-10T08:05:25.501993+0000","last_active":"2024-09-10T08:08:03.681852+0000","last_peered":"2024-09-10T08:08:03.681852+0000","last_clean":"2024-09-10T08:08:03.681852+0000","last_became_active":"2024-09-10T08:05:25.499912+0000","last_became_peered":"2024-09-10T08:05:25.499912+0000","last_unstale":"2024-09-10T08:08:03.681852+0000","last_undegraded":"2024-09-10T08:08:03.681852+0000","last_fullsized":"2024-09-10T08:08:03.681852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:06.780067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:36.268711+0000","last_change":"2024-09-10T08:05:22.461786+0000","last_active":"2024-09-10T08:07:36.268711+0000","last_peered":"2024-09-10T08:07:36.268711+0000","last_clean":"2024-09-10T08:07:36.268711+0000","last_became_active":"2024-09-10T08:05:22.460821+0000","last_became_peered":"2024-09-10T08:05:22.460821+0000","last_unstale":"2024-09-10T08:07:36.268711+0000","last_undegraded":"2024-09-10T08:07:36.268711+0000","last_fullsized":"2024-09-10T08:07:36.268711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:25.083538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914568+0000","last_change":"2024-09-10T08:06:04.734470+0000","last_active":"2024-09-10T08:07:05.914568+0000","last_peered":"2024-09-10T08:07:05.914568+0000","last_clean":"2024-09-10T08:07:05.914568+0000","last_became_active":"2024-09-10T08:06:04.734040+0000","last_became_peered":"2024-09-10T08:06:04.734040+0000","last_unstale":"2024-09-10T08:07:05.914568+0000","last_undegraded":"2024-09-10T08:07:05.914568+0000","last_fullsized":"2024-09-10T08:07:05.914568+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:36.312144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675820+0000","last_change":"2024-09-10T08:05:25.492711+0000","last_active":"2024-09-10T08:08:03.675820+0000","last_peered":"2024-09-10T08:08:03.675820+0000","last_clean":"2024-09-10T08:08:03.675820+0000","last_became_active":"2024-09-10T08:05:25.492336+0000","last_became_peered":"2024-09-10T08:05:25.492336+0000","last_unstale":"2024-09-10T08:08:03.675820+0000","last_undegraded":"2024-09-10T08:08:03.675820+0000","last_fullsized":"2024-09-10T08:08:03.675820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:04.168884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"93'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876783+0000","last_change":"2024-09-10T08:05:22.461535+0000","last_active":"2024-09-10T08:07:05.876783+0000","last_peered":"2024-09-10T08:07:05.876783+0000","last_clean":"2024-09-10T08:07:05.876783+0000","last_became_active":"2024-09-10T08:05:22.461208+0000","last_became_peered":"2024-09-10T08:05:22.461208+0000","last_unstale":"2024-09-10T08:07:05.876783+0000","last_undegraded":"2024-09-10T08:07:05.876783+0000","last_fullsized":"2024-09-10T08:07:05.876783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:26.009242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513889+0000","last_change":"2024-09-10T08:05:35.535442+0000","last_active":"2024-09-10T08:07:05.513889+0000","last_peered":"2024-09-10T08:07:05.513889+0000","last_clean":"2024-09-10T08:07:05.513889+0000","last_became_active":"2024-09-10T08:05:35.535058+0000","last_became_peered":"2024-09-10T08:05:35.535058+0000","last_unstale":"2024-09-10T08:07:05.513889+0000","last_undegraded":"2024-09-10T08:07:05.513889+0000","last_fullsized":"2024-09-10T08:07:05.513889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:39.020463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.627539+0000","last_change":"2024-09-10T08:05:25.483440+0000","last_active":"2024-09-10T08:08:03.627539+0000","last_peered":"2024-09-10T08:08:03.627539+0000","last_clean":"2024-09-10T08:08:03.627539+0000","last_became_active":"2024-09-10T08:05:25.483141+0000","last_became_peered":"2024-09-10T08:05:25.483141+0000","last_unstale":"2024-09-10T08:08:03.627539+0000","last_undegraded":"2024-09-10T08:08:03.627539+0000","last_fullsized":"2024-09-10T08:08:03.627539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:00.953188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"93'2","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914337+0000","last_change":"2024-09-10T08:05:22.568141+0000","last_active":"2024-09-10T08:07:05.914337+0000","last_peered":"2024-09-10T08:07:05.914337+0000","last_clean":"2024-09-10T08:07:05.914337+0000","last_became_active":"2024-09-10T08:05:22.567857+0000","last_became_peered":"2024-09-10T08:05:22.567857+0000","last_unstale":"2024-09-10T08:07:05.914337+0000","last_undegraded":"2024-09-10T08:07:05.914337+0000","last_fullsized":"2024-09-10T08:07:05.914337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:22.958747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914846+0000","last_change":"2024-09-10T08:05:35.641993+0000","last_active":"2024-09-10T08:07:05.914846+0000","last_peered":"2024-09-10T08:07:05.914846+0000","last_clean":"2024-09-10T08:07:05.914846+0000","last_became_active":"2024-09-10T08:05:35.641376+0000","last_became_peered":"2024-09-10T08:05:35.641376+0000","last_unstale":"2024-09-10T08:07:05.914846+0000","last_undegraded":"2024-09-10T08:07:05.914846+0000","last_fullsized":"2024-09-10T08:07:05.914846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:16:34.038924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.678764+0000","last_change":"2024-09-10T08:06:04.737560+0000","last_active":"2024-09-10T08:08:03.678764+0000","last_peered":"2024-09-10T08:08:03.678764+0000","last_clean":"2024-09-10T08:08:03.678764+0000","last_became_active":"2024-09-10T08:06:04.736656+0000","last_became_peered":"2024-09-10T08:06:04.736656+0000","last_unstale":"2024-09-10T08:08:03.678764+0000","last_undegraded":"2024-09-10T08:08:03.678764+0000","last_fullsized":"2024-09-10T08:08:03.678764+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:35.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"93'4","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915999+0000","last_change":"2024-09-10T08:05:22.567077+0000","last_active":"2024-09-10T08:07:05.915999+0000","last_peered":"2024-09-10T08:07:05.915999+0000","last_clean":"2024-09-10T08:07:05.915999+0000","last_became_active":"2024-09-10T08:05:22.566757+0000","last_became_peered":"2024-09-10T08:05:22.566757+0000","last_unstale":"2024-09-10T08:07:05.915999+0000","last_undegraded":"2024-09-10T08:07:05.915999+0000","last_fullsized":"2024-09-10T08:07:05.915999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:06.881510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511524+0000","last_change":"2024-09-10T08:05:35.519539+0000","last_active":"2024-09-10T08:07:05.511524+0000","last_peered":"2024-09-10T08:07:05.511524+0000","last_clean":"2024-09-10T08:07:05.511524+0000","last_became_active":"2024-09-10T08:05:35.519044+0000","last_became_peered":"2024-09-10T08:05:35.519044+0000","last_unstale":"2024-09-10T08:07:05.511524+0000","last_undegraded":"2024-09-10T08:07:05.511524+0000","last_fullsized":"2024-09-10T08:07:05.511524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:10.546949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658164+0000","last_change":"2024-09-10T08:07:05.771774+0000","last_active":"2024-09-10T08:08:03.658164+0000","last_peered":"2024-09-10T08:08:03.658164+0000","last_clean":"2024-09-10T08:08:03.658164+0000","last_became_active":"2024-09-10T08:07:05.771336+0000","last_became_peered":"2024-09-10T08:07:05.771336+0000","last_unstale":"2024-09-10T08:08:03.658164+0000","last_undegraded":"2024-09-10T08:08:03.658164+0000","last_fullsized":"2024-09-10T08:08:03.658164+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:11.482222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"95'14","reported_seq":68,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:18.518255+0000","last_change":"2024-09-10T08:06:04.741199+0000","last_active":"2024-09-10T08:08:18.518255+0000","last_peered":"2024-09-10T08:08:18.518255+0000","last_clean":"2024-09-10T08:08:18.518255+0000","last_became_active":"2024-09-10T08:06:04.738076+0000","last_became_peered":"2024-09-10T08:06:04.738076+0000","last_unstale":"2024-09-10T08:08:18.518255+0000","last_undegraded":"2024-09-10T08:08:18.518255+0000","last_fullsized":"2024-09-10T08:08:18.518255+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:45.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876462+0000","last_change":"2024-09-10T08:05:35.642358+0000","last_active":"2024-09-10T08:07:05.876462+0000","last_peered":"2024-09-10T08:07:05.876462+0000","last_clean":"2024-09-10T08:07:05.876462+0000","last_became_active":"2024-09-10T08:05:35.641408+0000","last_became_peered":"2024-09-10T08:05:35.641408+0000","last_unstale":"2024-09-10T08:07:05.876462+0000","last_undegraded":"2024-09-10T08:07:05.876462+0000","last_fullsized":"2024-09-10T08:07:05.876462+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:42.015139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658430+0000","last_change":"2024-09-10T08:05:25.486310+0000","last_active":"2024-09-10T08:08:03.658430+0000","last_peered":"2024-09-10T08:08:03.658430+0000","last_clean":"2024-09-10T08:08:03.658430+0000","last_became_active":"2024-09-10T08:05:25.485915+0000","last_became_peered":"2024-09-10T08:05:25.485915+0000","last_unstale":"2024-09-10T08:08:03.658430+0000","last_undegraded":"2024-09-10T08:08:03.658430+0000","last_fullsized":"2024-09-10T08:08:03.658430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:56.064361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"93'1","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508987+0000","last_change":"2024-09-10T08:05:22.570595+0000","last_active":"2024-09-10T08:07:05.508987+0000","last_peered":"2024-09-10T08:07:05.508987+0000","last_clean":"2024-09-10T08:07:05.508987+0000","last_became_active":"2024-09-10T08:05:22.570219+0000","last_became_peered":"2024-09-10T08:05:22.570219+0000","last_unstale":"2024-09-10T08:07:05.508987+0000","last_undegraded":"2024-09-10T08:07:05.508987+0000","last_fullsized":"2024-09-10T08:07:05.508987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:03.230325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.769927+0000","last_change":"2024-09-10T08:05:35.530472+0000","last_active":"2024-09-10T08:07:05.769927+0000","last_peered":"2024-09-10T08:07:05.769927+0000","last_clean":"2024-09-10T08:07:05.769927+0000","last_became_active":"2024-09-10T08:05:35.530066+0000","last_became_peered":"2024-09-10T08:05:35.530066+0000","last_unstale":"2024-09-10T08:07:05.769927+0000","last_undegraded":"2024-09-10T08:07:05.769927+0000","last_fullsized":"2024-09-10T08:07:05.769927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:58.915767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671834+0000","last_change":"2024-09-10T08:05:25.500524+0000","last_active":"2024-09-10T08:08:03.671834+0000","last_peered":"2024-09-10T08:08:03.671834+0000","last_clean":"2024-09-10T08:08:03.671834+0000","last_became_active":"2024-09-10T08:05:25.500186+0000","last_became_peered":"2024-09-10T08:05:25.500186+0000","last_unstale":"2024-09-10T08:08:03.671834+0000","last_undegraded":"2024-09-10T08:08:03.671834+0000","last_fullsized":"2024-09-10T08:08:03.671834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:51.936222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513172+0000","last_change":"2024-09-10T08:05:22.569326+0000","last_active":"2024-09-10T08:07:05.513172+0000","last_peered":"2024-09-10T08:07:05.513172+0000","last_clean":"2024-09-10T08:07:05.513172+0000","last_became_active":"2024-09-10T08:05:22.568895+0000","last_became_peered":"2024-09-10T08:05:22.568895+0000","last_unstale":"2024-09-10T08:07:05.513172+0000","last_undegraded":"2024-09-10T08:07:05.513172+0000","last_fullsized":"2024-09-10T08:07:05.513172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:47.212073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508368+0000","last_change":"2024-09-10T08:05:35.641600+0000","last_active":"2024-09-10T08:07:05.508368+0000","last_peered":"2024-09-10T08:07:05.508368+0000","last_clean":"2024-09-10T08:07:05.508368+0000","last_became_active":"2024-09-10T08:05:35.641205+0000","last_became_peered":"2024-09-10T08:05:35.641205+0000","last_unstale":"2024-09-10T08:07:05.508368+0000","last_undegraded":"2024-09-10T08:07:05.508368+0000","last_fullsized":"2024-09-10T08:07:05.508368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:40.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.659935+0000","last_change":"2024-09-10T08:05:25.495035+0000","last_active":"2024-09-10T08:08:03.659935+0000","last_peered":"2024-09-10T08:08:03.659935+0000","last_clean":"2024-09-10T08:08:03.659935+0000","last_became_active":"2024-09-10T08:05:25.494733+0000","last_became_peered":"2024-09-10T08:05:25.494733+0000","last_unstale":"2024-09-10T08:08:03.659935+0000","last_undegraded":"2024-09-10T08:08:03.659935+0000","last_fullsized":"2024-09-10T08:08:03.659935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.875867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"93'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509485+0000","last_change":"2024-09-10T08:05:22.458727+0000","last_active":"2024-09-10T08:07:05.509485+0000","last_peered":"2024-09-10T08:07:05.509485+0000","last_clean":"2024-09-10T08:07:05.509485+0000","last_became_active":"2024-09-10T08:05:22.458397+0000","last_became_peered":"2024-09-10T08:05:22.458397+0000","last_unstale":"2024-09-10T08:07:05.509485+0000","last_undegraded":"2024-09-10T08:07:05.509485+0000","last_fullsized":"2024-09-10T08:07:05.509485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:07.875559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914403+0000","last_change":"2024-09-10T08:05:35.534192+0000","last_active":"2024-09-10T08:07:05.914403+0000","last_peered":"2024-09-10T08:07:05.914403+0000","last_clean":"2024-09-10T08:07:05.914403+0000","last_became_active":"2024-09-10T08:05:35.530450+0000","last_became_peered":"2024-09-10T08:05:35.530450+0000","last_unstale":"2024-09-10T08:07:05.914403+0000","last_undegraded":"2024-09-10T08:07:05.914403+0000","last_fullsized":"2024-09-10T08:07:05.914403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:10.039144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651611+0000","last_change":"2024-09-10T08:05:25.482252+0000","last_active":"2024-09-10T08:08:03.651611+0000","last_peered":"2024-09-10T08:08:03.651611+0000","last_clean":"2024-09-10T08:08:03.651611+0000","last_became_active":"2024-09-10T08:05:25.481901+0000","last_became_peered":"2024-09-10T08:05:25.481901+0000","last_unstale":"2024-09-10T08:08:03.651611+0000","last_undegraded":"2024-09-10T08:08:03.651611+0000","last_fullsized":"2024-09-10T08:08:03.651611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:17.063340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":180,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:31.462022+0000","last_change":"2024-09-10T08:05:22.569199+0000","last_active":"2024-09-10T08:08:31.462022+0000","last_peered":"2024-09-10T08:08:31.462022+0000","last_clean":"2024-09-10T08:08:31.462022+0000","last_became_active":"2024-09-10T08:05:22.568787+0000","last_became_peered":"2024-09-10T08:05:22.568787+0000","last_unstale":"2024-09-10T08:08:31.462022+0000","last_undegraded":"2024-09-10T08:08:31.462022+0000","last_fullsized":"2024-09-10T08:08:31.462022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:59.610130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914677+0000","last_change":"2024-09-10T08:05:35.642446+0000","last_active":"2024-09-10T08:07:05.914677+0000","last_peered":"2024-09-10T08:07:05.914677+0000","last_clean":"2024-09-10T08:07:05.914677+0000","last_became_active":"2024-09-10T08:05:35.641543+0000","last_became_peered":"2024-09-10T08:05:35.641543+0000","last_unstale":"2024-09-10T08:07:05.914677+0000","last_undegraded":"2024-09-10T08:07:05.914677+0000","last_fullsized":"2024-09-10T08:07:05.914677+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:33.658406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.652192+0000","last_change":"2024-09-10T08:05:25.487598+0000","last_active":"2024-09-10T08:08:03.652192+0000","last_peered":"2024-09-10T08:08:03.652192+0000","last_clean":"2024-09-10T08:08:03.652192+0000","last_became_active":"2024-09-10T08:05:25.486879+0000","last_became_peered":"2024-09-10T08:05:25.486879+0000","last_unstale":"2024-09-10T08:08:03.652192+0000","last_undegraded":"2024-09-10T08:08:03.652192+0000","last_fullsized":"2024-09-10T08:08:03.652192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:20.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509934+0000","last_change":"2024-09-10T08:05:22.463416+0000","last_active":"2024-09-10T08:07:05.509934+0000","last_peered":"2024-09-10T08:07:05.509934+0000","last_clean":"2024-09-10T08:07:05.509934+0000","last_became_active":"2024-09-10T08:05:22.463004+0000","last_became_peered":"2024-09-10T08:05:22.463004+0000","last_unstale":"2024-09-10T08:07:05.509934+0000","last_undegraded":"2024-09-10T08:07:05.509934+0000","last_fullsized":"2024-09-10T08:07:05.509934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:56.257812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569364+0000","last_change":"2024-09-10T08:05:35.642059+0000","last_active":"2024-09-10T08:07:05.569364+0000","last_peered":"2024-09-10T08:07:05.569364+0000","last_clean":"2024-09-10T08:07:05.569364+0000","last_became_active":"2024-09-10T08:05:35.641540+0000","last_became_peered":"2024-09-10T08:05:35.641540+0000","last_unstale":"2024-09-10T08:07:05.569364+0000","last_undegraded":"2024-09-10T08:07:05.569364+0000","last_fullsized":"2024-09-10T08:07:05.569364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:28.557584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.670571+0000","last_change":"2024-09-10T08:05:25.496111+0000","last_active":"2024-09-10T08:08:03.670571+0000","last_peered":"2024-09-10T08:08:03.670571+0000","last_clean":"2024-09-10T08:08:03.670571+0000","last_became_active":"2024-09-10T08:05:25.495757+0000","last_became_peered":"2024-09-10T08:05:25.495757+0000","last_unstale":"2024-09-10T08:08:03.670571+0000","last_undegraded":"2024-09-10T08:08:03.670571+0000","last_fullsized":"2024-09-10T08:08:03.670571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:41.078205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'3","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915005+0000","last_change":"2024-09-10T08:05:22.460069+0000","last_active":"2024-09-10T08:07:05.915005+0000","last_peered":"2024-09-10T08:07:05.915005+0000","last_clean":"2024-09-10T08:07:05.915005+0000","last_became_active":"2024-09-10T08:05:22.459557+0000","last_became_peered":"2024-09-10T08:05:22.459557+0000","last_unstale":"2024-09-10T08:07:05.915005+0000","last_undegraded":"2024-09-10T08:07:05.915005+0000","last_fullsized":"2024-09-10T08:07:05.915005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:49.058507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509959+0000","last_change":"2024-09-10T08:04:08.188262+0000","last_active":"2024-09-10T08:07:05.509959+0000","last_peered":"2024-09-10T08:07:05.509959+0000","last_clean":"2024-09-10T08:07:05.509959+0000","last_became_active":"2024-09-10T08:04:08.187691+0000","last_became_peered":"2024-09-10T08:04:08.187691+0000","last_unstale":"2024-09-10T08:07:05.509959+0000","last_undegraded":"2024-09-10T08:07:05.509959+0000","last_fullsized":"2024-09-10T08:07:05.509959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:00:26.235127+0000","last_clean_scrub_stamp":"2024-09-10T08:00:26.235127+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:15.358818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509550+0000","last_change":"2024-09-10T08:05:35.528371+0000","last_active":"2024-09-10T08:07:05.509550+0000","last_peered":"2024-09-10T08:07:05.509550+0000","last_clean":"2024-09-10T08:07:05.509550+0000","last_became_active":"2024-09-10T08:05:35.527925+0000","last_became_peered":"2024-09-10T08:05:35.527925+0000","last_unstale":"2024-09-10T08:07:05.509550+0000","last_undegraded":"2024-09-10T08:07:05.509550+0000","last_fullsized":"2024-09-10T08:07:05.509550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:37:00.431911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667384+0000","last_change":"2024-09-10T08:05:25.490753+0000","last_active":"2024-09-10T08:08:03.667384+0000","last_peered":"2024-09-10T08:08:03.667384+0000","last_clean":"2024-09-10T08:08:03.667384+0000","last_became_active":"2024-09-10T08:05:25.489996+0000","last_became_peered":"2024-09-10T08:05:25.489996+0000","last_unstale":"2024-09-10T08:08:03.667384+0000","last_undegraded":"2024-09-10T08:08:03.667384+0000","last_fullsized":"2024-09-10T08:08:03.667384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:15.637199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'3","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511918+0000","last_change":"2024-09-10T08:05:22.568696+0000","last_active":"2024-09-10T08:07:05.511918+0000","last_peered":"2024-09-10T08:07:05.511918+0000","last_clean":"2024-09-10T08:07:05.511918+0000","last_became_active":"2024-09-10T08:05:22.568331+0000","last_became_peered":"2024-09-10T08:05:22.568331+0000","last_unstale":"2024-09-10T08:07:05.511918+0000","last_undegraded":"2024-09-10T08:07:05.511918+0000","last_fullsized":"2024-09-10T08:07:05.511918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:21.699397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914428+0000","last_change":"2024-09-10T08:05:35.530804+0000","last_active":"2024-09-10T08:07:05.914428+0000","last_peered":"2024-09-10T08:07:05.914428+0000","last_clean":"2024-09-10T08:07:05.914428+0000","last_became_active":"2024-09-10T08:05:35.530356+0000","last_became_peered":"2024-09-10T08:05:35.530356+0000","last_unstale":"2024-09-10T08:07:05.914428+0000","last_undegraded":"2024-09-10T08:07:05.914428+0000","last_fullsized":"2024-09-10T08:07:05.914428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:59.318761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.662813+0000","last_change":"2024-09-10T08:05:25.489546+0000","last_active":"2024-09-10T08:08:03.662813+0000","last_peered":"2024-09-10T08:08:03.662813+0000","last_clean":"2024-09-10T08:08:03.662813+0000","last_became_active":"2024-09-10T08:05:25.489188+0000","last_became_peered":"2024-09-10T08:05:25.489188+0000","last_unstale":"2024-09-10T08:08:03.662813+0000","last_undegraded":"2024-09-10T08:08:03.662813+0000","last_fullsized":"2024-09-10T08:08:03.662813+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:51:26.471681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569736+0000","last_change":"2024-09-10T08:05:22.463912+0000","last_active":"2024-09-10T08:07:05.569736+0000","last_peered":"2024-09-10T08:07:05.569736+0000","last_clean":"2024-09-10T08:07:05.569736+0000","last_became_active":"2024-09-10T08:05:22.463561+0000","last_became_peered":"2024-09-10T08:05:22.463561+0000","last_unstale":"2024-09-10T08:07:05.569736+0000","last_undegraded":"2024-09-10T08:07:05.569736+0000","last_fullsized":"2024-09-10T08:07:05.569736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:22.407604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.514959+0000","last_change":"2024-09-10T08:05:35.528795+0000","last_active":"2024-09-10T08:07:05.514959+0000","last_peered":"2024-09-10T08:07:05.514959+0000","last_clean":"2024-09-10T08:07:05.514959+0000","last_became_active":"2024-09-10T08:05:35.528402+0000","last_became_peered":"2024-09-10T08:05:35.528402+0000","last_unstale":"2024-09-10T08:07:05.514959+0000","last_undegraded":"2024-09-10T08:07:05.514959+0000","last_fullsized":"2024-09-10T08:07:05.514959+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:14.606371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.657641+0000","last_change":"2024-09-10T08:05:25.484086+0000","last_active":"2024-09-10T08:08:03.657641+0000","last_peered":"2024-09-10T08:08:03.657641+0000","last_clean":"2024-09-10T08:08:03.657641+0000","last_became_active":"2024-09-10T08:05:25.483757+0000","last_became_peered":"2024-09-10T08:05:25.483757+0000","last_unstale":"2024-09-10T08:08:03.657641+0000","last_undegraded":"2024-09-10T08:08:03.657641+0000","last_fullsized":"2024-09-10T08:08:03.657641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:16:16.362739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914678+0000","last_change":"2024-09-10T08:05:22.462220+0000","last_active":"2024-09-10T08:07:05.914678+0000","last_peered":"2024-09-10T08:07:05.914678+0000","last_clean":"2024-09-10T08:07:05.914678+0000","last_became_active":"2024-09-10T08:05:22.461726+0000","last_became_peered":"2024-09-10T08:05:22.461726+0000","last_unstale":"2024-09-10T08:07:05.914678+0000","last_undegraded":"2024-09-10T08:07:05.914678+0000","last_fullsized":"2024-09-10T08:07:05.914678+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:56.381034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508271+0000","last_change":"2024-09-10T08:06:04.734827+0000","last_active":"2024-09-10T08:07:05.508271+0000","last_peered":"2024-09-10T08:07:05.508271+0000","last_clean":"2024-09-10T08:07:05.508271+0000","last_became_active":"2024-09-10T08:06:04.734402+0000","last_became_peered":"2024-09-10T08:06:04.734402+0000","last_unstale":"2024-09-10T08:07:05.508271+0000","last_undegraded":"2024-09-10T08:07:05.508271+0000","last_fullsized":"2024-09-10T08:07:05.508271+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:24.806538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654332+0000","last_change":"2024-09-10T08:05:25.486100+0000","last_active":"2024-09-10T08:08:03.654332+0000","last_peered":"2024-09-10T08:08:03.654332+0000","last_clean":"2024-09-10T08:08:03.654332+0000","last_became_active":"2024-09-10T08:05:25.485586+0000","last_became_peered":"2024-09-10T08:05:25.485586+0000","last_unstale":"2024-09-10T08:08:03.654332+0000","last_undegraded":"2024-09-10T08:08:03.654332+0000","last_fullsized":"2024-09-10T08:08:03.654332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:33.565054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"93'2","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770403+0000","last_change":"2024-09-10T08:06:04.743408+0000","last_active":"2024-09-10T08:07:05.770403+0000","last_peered":"2024-09-10T08:07:05.770403+0000","last_clean":"2024-09-10T08:07:05.770403+0000","last_became_active":"2024-09-10T08:06:04.732830+0000","last_became_peered":"2024-09-10T08:06:04.732830+0000","last_unstale":"2024-09-10T08:07:05.770403+0000","last_undegraded":"2024-09-10T08:07:05.770403+0000","last_fullsized":"2024-09-10T08:07:05.770403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:29.381259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568977+0000","last_change":"2024-09-10T08:05:35.533015+0000","last_active":"2024-09-10T08:07:05.568977+0000","last_peered":"2024-09-10T08:07:05.568977+0000","last_clean":"2024-09-10T08:07:05.568977+0000","last_became_active":"2024-09-10T08:05:35.532590+0000","last_became_peered":"2024-09-10T08:05:35.532590+0000","last_unstale":"2024-09-10T08:07:05.568977+0000","last_undegraded":"2024-09-10T08:07:05.568977+0000","last_fullsized":"2024-09-10T08:07:05.568977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:23.984273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.650197+0000","last_change":"2024-09-10T08:05:25.489994+0000","last_active":"2024-09-10T08:08:03.650197+0000","last_peered":"2024-09-10T08:08:03.650197+0000","last_clean":"2024-09-10T08:08:03.650197+0000","last_became_active":"2024-09-10T08:05:25.489471+0000","last_became_peered":"2024-09-10T08:05:25.489471+0000","last_unstale":"2024-09-10T08:08:03.650197+0000","last_undegraded":"2024-09-10T08:08:03.650197+0000","last_fullsized":"2024-09-10T08:08:03.650197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:53.866570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"93'15","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.508405+0000","last_change":"2024-09-10T08:05:22.568900+0000","last_active":"2024-09-10T08:07:05.508405+0000","last_peered":"2024-09-10T08:07:05.508405+0000","last_clean":"2024-09-10T08:07:05.508405+0000","last_became_active":"2024-09-10T08:05:22.568542+0000","last_became_peered":"2024-09-10T08:05:22.568542+0000","last_unstale":"2024-09-10T08:07:05.508405+0000","last_undegraded":"2024-09-10T08:07:05.508405+0000","last_fullsized":"2024-09-10T08:07:05.508405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:58:45.669769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914683+0000","last_change":"2024-09-10T08:05:35.537364+0000","last_active":"2024-09-10T08:07:05.914683+0000","last_peered":"2024-09-10T08:07:05.914683+0000","last_clean":"2024-09-10T08:07:05.914683+0000","last_became_active":"2024-09-10T08:05:35.535958+0000","last_became_peered":"2024-09-10T08:05:35.535958+0000","last_unstale":"2024-09-10T08:07:05.914683+0000","last_undegraded":"2024-09-10T08:07:05.914683+0000","last_fullsized":"2024-09-10T08:07:05.914683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:51.443798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660149+0000","last_change":"2024-09-10T08:07:05.917030+0000","last_active":"2024-09-10T08:08:03.660149+0000","last_peered":"2024-09-10T08:08:03.660149+0000","last_clean":"2024-09-10T08:08:03.660149+0000","last_became_active":"2024-09-10T08:07:05.916281+0000","last_became_peered":"2024-09-10T08:07:05.916281+0000","last_unstale":"2024-09-10T08:08:03.660149+0000","last_undegraded":"2024-09-10T08:08:03.660149+0000","last_fullsized":"2024-09-10T08:08:03.660149+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:15.060009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'4","reported_seq":181,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:30.460882+0000","last_change":"2024-09-10T08:05:22.459963+0000","last_active":"2024-09-10T08:08:30.460882+0000","last_peered":"2024-09-10T08:08:30.460882+0000","last_clean":"2024-09-10T08:08:30.460882+0000","last_became_active":"2024-09-10T08:05:22.459497+0000","last_became_peered":"2024-09-10T08:05:22.459497+0000","last_unstale":"2024-09-10T08:08:30.460882+0000","last_undegraded":"2024-09-10T08:08:30.460882+0000","last_fullsized":"2024-09-10T08:08:30.460882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:29.887980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513986+0000","last_change":"2024-09-10T08:05:35.534570+0000","last_active":"2024-09-10T08:07:05.513986+0000","last_peered":"2024-09-10T08:07:05.513986+0000","last_clean":"2024-09-10T08:07:05.513986+0000","last_became_active":"2024-09-10T08:05:35.534195+0000","last_became_peered":"2024-09-10T08:05:35.534195+0000","last_unstale":"2024-09-10T08:07:05.513986+0000","last_undegraded":"2024-09-10T08:07:05.513986+0000","last_fullsized":"2024-09-10T08:07:05.513986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:17.393491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.668227+0000","last_change":"2024-09-10T08:05:25.488551+0000","last_active":"2024-09-10T08:08:03.668227+0000","last_peered":"2024-09-10T08:08:03.668227+0000","last_clean":"2024-09-10T08:08:03.668227+0000","last_became_active":"2024-09-10T08:05:25.488086+0000","last_became_peered":"2024-09-10T08:05:25.488086+0000","last_unstale":"2024-09-10T08:08:03.668227+0000","last_undegraded":"2024-09-10T08:08:03.668227+0000","last_fullsized":"2024-09-10T08:08:03.668227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:38.044721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569070+0000","last_change":"2024-09-10T08:05:22.462269+0000","last_active":"2024-09-10T08:07:05.569070+0000","last_peered":"2024-09-10T08:07:05.569070+0000","last_clean":"2024-09-10T08:07:05.569070+0000","last_became_active":"2024-09-10T08:05:22.461924+0000","last_became_peered":"2024-09-10T08:05:22.461924+0000","last_unstale":"2024-09-10T08:07:05.569070+0000","last_undegraded":"2024-09-10T08:07:05.569070+0000","last_fullsized":"2024-09-10T08:07:05.569070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:54.996777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770191+0000","last_change":"2024-09-10T08:05:35.531596+0000","last_active":"2024-09-10T08:07:05.770191+0000","last_peered":"2024-09-10T08:07:05.770191+0000","last_clean":"2024-09-10T08:07:05.770191+0000","last_became_active":"2024-09-10T08:05:35.530179+0000","last_became_peered":"2024-09-10T08:05:35.530179+0000","last_unstale":"2024-09-10T08:07:05.770191+0000","last_undegraded":"2024-09-10T08:07:05.770191+0000","last_fullsized":"2024-09-10T08:07:05.770191+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:32.056426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.663218+0000","last_change":"2024-09-10T08:05:25.503504+0000","last_active":"2024-09-10T08:08:03.663218+0000","last_peered":"2024-09-10T08:08:03.663218+0000","last_clean":"2024-09-10T08:08:03.663218+0000","last_became_active":"2024-09-10T08:05:25.503105+0000","last_became_peered":"2024-09-10T08:05:25.503105+0000","last_unstale":"2024-09-10T08:08:03.663218+0000","last_undegraded":"2024-09-10T08:08:03.663218+0000","last_fullsized":"2024-09-10T08:08:03.663218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:40.885654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"93'6","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511623+0000","last_change":"2024-09-10T08:05:22.462271+0000","last_active":"2024-09-10T08:07:05.511623+0000","last_peered":"2024-09-10T08:07:05.511623+0000","last_clean":"2024-09-10T08:07:05.511623+0000","last_became_active":"2024-09-10T08:05:22.461195+0000","last_became_peered":"2024-09-10T08:05:22.461195+0000","last_unstale":"2024-09-10T08:07:05.511623+0000","last_undegraded":"2024-09-10T08:07:05.511623+0000","last_fullsized":"2024-09-10T08:07:05.511623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:20.269483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569486+0000","last_change":"2024-09-10T08:05:35.533218+0000","last_active":"2024-09-10T08:07:05.569486+0000","last_peered":"2024-09-10T08:07:05.569486+0000","last_clean":"2024-09-10T08:07:05.569486+0000","last_became_active":"2024-09-10T08:05:35.532706+0000","last_became_peered":"2024-09-10T08:05:35.532706+0000","last_unstale":"2024-09-10T08:07:05.569486+0000","last_undegraded":"2024-09-10T08:07:05.569486+0000","last_fullsized":"2024-09-10T08:07:05.569486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:22.150563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.655456+0000","last_change":"2024-09-10T08:05:25.478978+0000","last_active":"2024-09-10T08:08:03.655456+0000","last_peered":"2024-09-10T08:08:03.655456+0000","last_clean":"2024-09-10T08:08:03.655456+0000","last_became_active":"2024-09-10T08:05:25.478649+0000","last_became_peered":"2024-09-10T08:05:25.478649+0000","last_unstale":"2024-09-10T08:08:03.655456+0000","last_undegraded":"2024-09-10T08:08:03.655456+0000","last_fullsized":"2024-09-10T08:08:03.655456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:17.493446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'40","reported_seq":382,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:35.468522+0000","last_change":"2024-09-10T08:06:04.740972+0000","last_active":"2024-09-10T08:08:35.468522+0000","last_peered":"2024-09-10T08:08:35.468522+0000","last_clean":"2024-09-10T08:08:35.468522+0000","last_became_active":"2024-09-10T08:06:04.731200+0000","last_became_peered":"2024-09-10T08:06:04.731200+0000","last_unstale":"2024-09-10T08:08:35.468522+0000","last_undegraded":"2024-09-10T08:08:35.468522+0000","last_fullsized":"2024-09-10T08:08:35.468522+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:17.169708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71507,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":282,"num_read_kb":212,"num_write":41,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3703,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512781+0000","last_change":"2024-09-10T08:05:35.532594+0000","last_active":"2024-09-10T08:07:05.512781+0000","last_peered":"2024-09-10T08:07:05.512781+0000","last_clean":"2024-09-10T08:07:05.512781+0000","last_became_active":"2024-09-10T08:05:35.532132+0000","last_became_peered":"2024-09-10T08:05:35.532132+0000","last_unstale":"2024-09-10T08:07:05.512781+0000","last_undegraded":"2024-09-10T08:07:05.512781+0000","last_fullsized":"2024-09-10T08:07:05.512781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:11:15.348648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.666577+0000","last_change":"2024-09-10T08:05:25.485297+0000","last_active":"2024-09-10T08:08:03.666577+0000","last_peered":"2024-09-10T08:08:03.666577+0000","last_clean":"2024-09-10T08:08:03.666577+0000","last_became_active":"2024-09-10T08:05:25.484883+0000","last_became_peered":"2024-09-10T08:05:25.484883+0000","last_unstale":"2024-09-10T08:08:03.666577+0000","last_undegraded":"2024-09-10T08:08:03.666577+0000","last_fullsized":"2024-09-10T08:08:03.666577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:37.052435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"93'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.569713+0000","last_change":"2024-09-10T08:05:22.568993+0000","last_active":"2024-09-10T08:07:05.569713+0000","last_peered":"2024-09-10T08:07:05.569713+0000","last_clean":"2024-09-10T08:07:05.569713+0000","last_became_active":"2024-09-10T08:05:22.568564+0000","last_became_peered":"2024-09-10T08:05:22.568564+0000","last_unstale":"2024-09-10T08:07:05.569713+0000","last_undegraded":"2024-09-10T08:07:05.569713+0000","last_fullsized":"2024-09-10T08:07:05.569713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:23.767174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914556+0000","last_change":"2024-09-10T08:05:35.524025+0000","last_active":"2024-09-10T08:07:05.914556+0000","last_peered":"2024-09-10T08:07:05.914556+0000","last_clean":"2024-09-10T08:07:05.914556+0000","last_became_active":"2024-09-10T08:05:35.523628+0000","last_became_peered":"2024-09-10T08:05:35.523628+0000","last_unstale":"2024-09-10T08:07:05.914556+0000","last_undegraded":"2024-09-10T08:07:05.914556+0000","last_fullsized":"2024-09-10T08:07:05.914556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:48.593779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.680405+0000","last_change":"2024-09-10T08:07:05.570537+0000","last_active":"2024-09-10T08:08:03.680405+0000","last_peered":"2024-09-10T08:08:03.680405+0000","last_clean":"2024-09-10T08:08:03.680405+0000","last_became_active":"2024-09-10T08:07:05.570278+0000","last_became_peered":"2024-09-10T08:07:05.570278+0000","last_unstale":"2024-09-10T08:08:03.680405+0000","last_undegraded":"2024-09-10T08:08:03.680405+0000","last_fullsized":"2024-09-10T08:08:03.680405+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:56.490462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.509635+0000","last_change":"2024-09-10T08:05:22.460487+0000","last_active":"2024-09-10T08:07:05.509635+0000","last_peered":"2024-09-10T08:07:05.509635+0000","last_clean":"2024-09-10T08:07:05.509635+0000","last_became_active":"2024-09-10T08:05:22.460158+0000","last_became_peered":"2024-09-10T08:05:22.460158+0000","last_unstale":"2024-09-10T08:07:05.509635+0000","last_undegraded":"2024-09-10T08:07:05.509635+0000","last_fullsized":"2024-09-10T08:07:05.509635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:53.285513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.512495+0000","last_change":"2024-09-10T08:05:35.531399+0000","last_active":"2024-09-10T08:07:05.512495+0000","last_peered":"2024-09-10T08:07:05.512495+0000","last_clean":"2024-09-10T08:07:05.512495+0000","last_became_active":"2024-09-10T08:05:35.530320+0000","last_became_peered":"2024-09-10T08:05:35.530320+0000","last_unstale":"2024-09-10T08:07:05.512495+0000","last_undegraded":"2024-09-10T08:07:05.512495+0000","last_fullsized":"2024-09-10T08:07:05.512495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:19.814046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669147+0000","last_change":"2024-09-10T08:05:25.488208+0000","last_active":"2024-09-10T08:08:03.669147+0000","last_peered":"2024-09-10T08:08:03.669147+0000","last_clean":"2024-09-10T08:08:03.669147+0000","last_became_active":"2024-09-10T08:05:25.487893+0000","last_became_peered":"2024-09-10T08:05:25.487893+0000","last_unstale":"2024-09-10T08:08:03.669147+0000","last_undegraded":"2024-09-10T08:08:03.669147+0000","last_fullsized":"2024-09-10T08:08:03.669147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:10.999416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":183,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:34.467437+0000","last_change":"2024-09-10T08:05:22.461321+0000","last_active":"2024-09-10T08:08:34.467437+0000","last_peered":"2024-09-10T08:08:34.467437+0000","last_clean":"2024-09-10T08:08:34.467437+0000","last_became_active":"2024-09-10T08:05:22.460950+0000","last_became_peered":"2024-09-10T08:05:22.460950+0000","last_unstale":"2024-09-10T08:08:34.467437+0000","last_undegraded":"2024-09-10T08:08:34.467437+0000","last_fullsized":"2024-09-10T08:08:34.467437+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:55.647273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.876534+0000","last_change":"2024-09-10T08:05:35.530578+0000","last_active":"2024-09-10T08:07:05.876534+0000","last_peered":"2024-09-10T08:07:05.876534+0000","last_clean":"2024-09-10T08:07:05.876534+0000","last_became_active":"2024-09-10T08:05:35.530162+0000","last_became_peered":"2024-09-10T08:05:35.530162+0000","last_unstale":"2024-09-10T08:07:05.876534+0000","last_undegraded":"2024-09-10T08:07:05.876534+0000","last_fullsized":"2024-09-10T08:07:05.876534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:19.290973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.513617+0000","last_change":"2024-09-10T08:05:35.532685+0000","last_active":"2024-09-10T08:07:05.513617+0000","last_peered":"2024-09-10T08:07:05.513617+0000","last_clean":"2024-09-10T08:07:05.513617+0000","last_became_active":"2024-09-10T08:05:35.532257+0000","last_became_peered":"2024-09-10T08:05:35.532257+0000","last_unstale":"2024-09-10T08:07:05.513617+0000","last_undegraded":"2024-09-10T08:07:05.513617+0000","last_fullsized":"2024-09-10T08:07:05.513617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:07.412435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.662135+0000","last_change":"2024-09-10T08:05:25.494472+0000","last_active":"2024-09-10T08:08:03.662135+0000","last_peered":"2024-09-10T08:08:03.662135+0000","last_clean":"2024-09-10T08:08:03.662135+0000","last_became_active":"2024-09-10T08:05:25.493928+0000","last_became_peered":"2024-09-10T08:05:25.493928+0000","last_unstale":"2024-09-10T08:08:03.662135+0000","last_undegraded":"2024-09-10T08:08:03.662135+0000","last_fullsized":"2024-09-10T08:08:03.662135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:00.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"93'1","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.568928+0000","last_change":"2024-09-10T08:05:22.460585+0000","last_active":"2024-09-10T08:07:05.568928+0000","last_peered":"2024-09-10T08:07:05.568928+0000","last_clean":"2024-09-10T08:07:05.568928+0000","last_became_active":"2024-09-10T08:05:22.460259+0000","last_became_peered":"2024-09-10T08:05:22.460259+0000","last_unstale":"2024-09-10T08:07:05.568928+0000","last_undegraded":"2024-09-10T08:07:05.568928+0000","last_fullsized":"2024-09-10T08:07:05.568928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:45.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.770275+0000","last_change":"2024-09-10T08:05:35.524000+0000","last_active":"2024-09-10T08:07:05.770275+0000","last_peered":"2024-09-10T08:07:05.770275+0000","last_clean":"2024-09-10T08:07:05.770275+0000","last_became_active":"2024-09-10T08:05:35.523647+0000","last_became_peered":"2024-09-10T08:05:35.523647+0000","last_unstale":"2024-09-10T08:07:05.770275+0000","last_undegraded":"2024-09-10T08:07:05.770275+0000","last_fullsized":"2024-09-10T08:07:05.770275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:10:54.664865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660563+0000","last_change":"2024-09-10T08:05:25.499701+0000","last_active":"2024-09-10T08:08:03.660563+0000","last_peered":"2024-09-10T08:08:03.660563+0000","last_clean":"2024-09-10T08:08:03.660563+0000","last_became_active":"2024-09-10T08:05:25.496890+0000","last_became_peered":"2024-09-10T08:05:25.496890+0000","last_unstale":"2024-09-10T08:08:03.660563+0000","last_undegraded":"2024-09-10T08:08:03.660563+0000","last_fullsized":"2024-09-10T08:08:03.660563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:58.166365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"93'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.914070+0000","last_change":"2024-09-10T08:05:22.460880+0000","last_active":"2024-09-10T08:07:05.914070+0000","last_peered":"2024-09-10T08:07:05.914070+0000","last_clean":"2024-09-10T08:07:05.914070+0000","last_became_active":"2024-09-10T08:05:22.459932+0000","last_became_peered":"2024-09-10T08:05:22.459932+0000","last_unstale":"2024-09-10T08:07:05.914070+0000","last_undegraded":"2024-09-10T08:07:05.914070+0000","last_fullsized":"2024-09-10T08:07:05.914070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:55.826526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"93'3","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.915405+0000","last_change":"2024-09-10T08:05:22.460578+0000","last_active":"2024-09-10T08:07:05.915405+0000","last_peered":"2024-09-10T08:07:05.915405+0000","last_clean":"2024-09-10T08:07:05.915405+0000","last_became_active":"2024-09-10T08:05:22.460239+0000","last_became_peered":"2024-09-10T08:05:22.460239+0000","last_unstale":"2024-09-10T08:07:05.915405+0000","last_undegraded":"2024-09-10T08:07:05.915405+0000","last_fullsized":"2024-09-10T08:07:05.915405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:07.606243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.661142+0000","last_change":"2024-09-10T08:05:25.489846+0000","last_active":"2024-09-10T08:08:03.661142+0000","last_peered":"2024-09-10T08:08:03.661142+0000","last_clean":"2024-09-10T08:08:03.661142+0000","last_became_active":"2024-09-10T08:05:25.489326+0000","last_became_peered":"2024-09-10T08:05:25.489326+0000","last_unstale":"2024-09-10T08:08:03.661142+0000","last_undegraded":"2024-09-10T08:08:03.661142+0000","last_fullsized":"2024-09-10T08:08:03.661142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:43.458468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511635+0000","last_change":"2024-09-10T08:05:35.530522+0000","last_active":"2024-09-10T08:07:05.511635+0000","last_peered":"2024-09-10T08:07:05.511635+0000","last_clean":"2024-09-10T08:07:05.511635+0000","last_became_active":"2024-09-10T08:05:35.530099+0000","last_became_peered":"2024-09-10T08:05:35.530099+0000","last_unstale":"2024-09-10T08:07:05.511635+0000","last_undegraded":"2024-09-10T08:07:05.511635+0000","last_fullsized":"2024-09-10T08:07:05.511635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:58.469119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":189,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:30.461416+0000","last_change":"2024-09-10T08:05:22.461391+0000","last_active":"2024-09-10T08:08:30.461416+0000","last_peered":"2024-09-10T08:08:30.461416+0000","last_clean":"2024-09-10T08:08:30.461416+0000","last_became_active":"2024-09-10T08:05:22.461042+0000","last_became_peered":"2024-09-10T08:05:22.461042+0000","last_unstale":"2024-09-10T08:08:30.461416+0000","last_undegraded":"2024-09-10T08:08:30.461416+0000","last_fullsized":"2024-09-10T08:08:30.461416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:21.436345+0000","last_clean_scrub_stamp":"2024-09-10T08:05:21.436345+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:12.635710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1860,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.649481+0000","last_change":"2024-09-10T08:05:25.487378+0000","last_active":"2024-09-10T08:08:03.649481+0000","last_peered":"2024-09-10T08:08:03.649481+0000","last_clean":"2024-09-10T08:08:03.649481+0000","last_became_active":"2024-09-10T08:05:25.487055+0000","last_became_peered":"2024-09-10T08:05:25.487055+0000","last_unstale":"2024-09-10T08:08:03.649481+0000","last_undegraded":"2024-09-10T08:08:03.649481+0000","last_fullsized":"2024-09-10T08:08:03.649481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:59.522014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:07:05.511748+0000","last_change":"2024-09-10T08:05:35.522765+0000","last_active":"2024-09-10T08:07:05.511748+0000","last_peered":"2024-09-10T08:07:05.511748+0000","last_clean":"2024-09-10T08:07:05.511748+0000","last_became_active":"2024-09-10T08:05:35.522359+0000","last_became_peered":"2024-09-10T08:05:35.522359+0000","last_unstale":"2024-09-10T08:07:05.511748+0000","last_undegraded":"2024-09-10T08:07:05.511748+0000","last_fullsized":"2024-09-10T08:07:05.511748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:34.500035+0000","last_clean_scrub_stamp":"2024-09-10T08:05:34.500035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:15.280867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.652869+0000","last_change":"2024-09-10T08:05:25.475388+0000","last_active":"2024-09-10T08:08:03.652869+0000","last_peered":"2024-09-10T08:08:03.652869+0000","last_clean":"2024-09-10T08:08:03.652869+0000","last_became_active":"2024-09-10T08:05:25.474331+0000","last_became_peered":"2024-09-10T08:05:25.474331+0000","last_unstale":"2024-09-10T08:08:03.652869+0000","last_undegraded":"2024-09-10T08:08:03.652869+0000","last_fullsized":"2024-09-10T08:08:03.652869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:58.713148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656383+0000","last_change":"2024-09-10T08:07:05.570505+0000","last_active":"2024-09-10T08:08:03.656383+0000","last_peered":"2024-09-10T08:08:03.656383+0000","last_clean":"2024-09-10T08:08:03.656383+0000","last_became_active":"2024-09-10T08:07:05.569970+0000","last_became_peered":"2024-09-10T08:07:05.569970+0000","last_unstale":"2024-09-10T08:08:03.656383+0000","last_undegraded":"2024-09-10T08:08:03.656383+0000","last_fullsized":"2024-09-10T08:08:03.656383+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:13.282503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671951+0000","last_change":"2024-09-10T08:05:25.491585+0000","last_active":"2024-09-10T08:08:03.671951+0000","last_peered":"2024-09-10T08:08:03.671951+0000","last_clean":"2024-09-10T08:08:03.671951+0000","last_became_active":"2024-09-10T08:05:25.491224+0000","last_became_peered":"2024-09-10T08:05:25.491224+0000","last_unstale":"2024-09-10T08:08:03.671951+0000","last_undegraded":"2024-09-10T08:08:03.671951+0000","last_fullsized":"2024-09-10T08:08:03.671951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:56.066808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681502+0000","last_change":"2024-09-10T08:05:25.499578+0000","last_active":"2024-09-10T08:08:03.681502+0000","last_peered":"2024-09-10T08:08:03.681502+0000","last_clean":"2024-09-10T08:08:03.681502+0000","last_became_active":"2024-09-10T08:05:25.498445+0000","last_became_peered":"2024-09-10T08:05:25.498445+0000","last_unstale":"2024-09-10T08:08:03.681502+0000","last_undegraded":"2024-09-10T08:08:03.681502+0000","last_fullsized":"2024-09-10T08:08:03.681502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:58.961699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675717+0000","last_change":"2024-09-10T08:06:04.735764+0000","last_active":"2024-09-10T08:08:03.675717+0000","last_peered":"2024-09-10T08:08:03.675717+0000","last_clean":"2024-09-10T08:08:03.675717+0000","last_became_active":"2024-09-10T08:06:04.735366+0000","last_became_peered":"2024-09-10T08:06:04.735366+0000","last_unstale":"2024-09-10T08:08:03.675717+0000","last_undegraded":"2024-09-10T08:08:03.675717+0000","last_fullsized":"2024-09-10T08:08:03.675717+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:03.757890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651587+0000","last_change":"2024-09-10T08:05:25.475872+0000","last_active":"2024-09-10T08:08:03.651587+0000","last_peered":"2024-09-10T08:08:03.651587+0000","last_clean":"2024-09-10T08:08:03.651587+0000","last_became_active":"2024-09-10T08:05:25.475549+0000","last_became_peered":"2024-09-10T08:05:25.475549+0000","last_unstale":"2024-09-10T08:08:03.651587+0000","last_undegraded":"2024-09-10T08:08:03.651587+0000","last_fullsized":"2024-09-10T08:08:03.651587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:37.575267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":54,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.664667+0000","last_change":"2024-09-10T08:05:25.474210+0000","last_active":"2024-09-10T08:08:03.664667+0000","last_peered":"2024-09-10T08:08:03.664667+0000","last_clean":"2024-09-10T08:08:03.664667+0000","last_became_active":"2024-09-10T08:05:25.473882+0000","last_became_peered":"2024-09-10T08:05:25.473882+0000","last_unstale":"2024-09-10T08:08:03.664667+0000","last_undegraded":"2024-09-10T08:08:03.664667+0000","last_fullsized":"2024-09-10T08:08:03.664667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:05.164370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.651519+0000","last_change":"2024-09-10T08:05:25.490406+0000","last_active":"2024-09-10T08:08:03.651519+0000","last_peered":"2024-09-10T08:08:03.651519+0000","last_clean":"2024-09-10T08:08:03.651519+0000","last_became_active":"2024-09-10T08:05:25.490061+0000","last_became_peered":"2024-09-10T08:05:25.490061+0000","last_unstale":"2024-09-10T08:08:03.651519+0000","last_undegraded":"2024-09-10T08:08:03.651519+0000","last_fullsized":"2024-09-10T08:08:03.651519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:12.988229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.664045+0000","last_change":"2024-09-10T08:05:25.504049+0000","last_active":"2024-09-10T08:08:03.664045+0000","last_peered":"2024-09-10T08:08:03.664045+0000","last_clean":"2024-09-10T08:08:03.664045+0000","last_became_active":"2024-09-10T08:05:25.503770+0000","last_became_peered":"2024-09-10T08:05:25.503770+0000","last_unstale":"2024-09-10T08:08:03.664045+0000","last_undegraded":"2024-09-10T08:08:03.664045+0000","last_fullsized":"2024-09-10T08:08:03.664045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:52.850945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.672614+0000","last_change":"2024-09-10T08:05:25.491682+0000","last_active":"2024-09-10T08:08:03.672614+0000","last_peered":"2024-09-10T08:08:03.672614+0000","last_clean":"2024-09-10T08:08:03.672614+0000","last_became_active":"2024-09-10T08:05:25.491296+0000","last_became_peered":"2024-09-10T08:05:25.491296+0000","last_unstale":"2024-09-10T08:08:03.672614+0000","last_undegraded":"2024-09-10T08:08:03.672614+0000","last_fullsized":"2024-09-10T08:08:03.672614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:30.075842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.673509+0000","last_change":"2024-09-10T08:05:25.489648+0000","last_active":"2024-09-10T08:08:03.673509+0000","last_peered":"2024-09-10T08:08:03.673509+0000","last_clean":"2024-09-10T08:08:03.673509+0000","last_became_active":"2024-09-10T08:05:25.489267+0000","last_became_peered":"2024-09-10T08:05:25.489267+0000","last_unstale":"2024-09-10T08:08:03.673509+0000","last_undegraded":"2024-09-10T08:08:03.673509+0000","last_fullsized":"2024-09-10T08:08:03.673509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:21.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.681690+0000","last_change":"2024-09-10T08:05:25.497862+0000","last_active":"2024-09-10T08:08:03.681690+0000","last_peered":"2024-09-10T08:08:03.681690+0000","last_clean":"2024-09-10T08:08:03.681690+0000","last_became_active":"2024-09-10T08:05:25.497504+0000","last_became_peered":"2024-09-10T08:05:25.497504+0000","last_unstale":"2024-09-10T08:08:03.681690+0000","last_undegraded":"2024-09-10T08:08:03.681690+0000","last_fullsized":"2024-09-10T08:08:03.681690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:40.291598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654484+0000","last_change":"2024-09-10T08:05:25.485924+0000","last_active":"2024-09-10T08:08:03.654484+0000","last_peered":"2024-09-10T08:08:03.654484+0000","last_clean":"2024-09-10T08:08:03.654484+0000","last_became_active":"2024-09-10T08:05:25.485545+0000","last_became_peered":"2024-09-10T08:05:25.485545+0000","last_unstale":"2024-09-10T08:08:03.654484+0000","last_undegraded":"2024-09-10T08:08:03.654484+0000","last_fullsized":"2024-09-10T08:08:03.654484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:33:20.233720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.671966+0000","last_change":"2024-09-10T08:05:25.491911+0000","last_active":"2024-09-10T08:08:03.671966+0000","last_peered":"2024-09-10T08:08:03.671966+0000","last_clean":"2024-09-10T08:08:03.671966+0000","last_became_active":"2024-09-10T08:05:25.491330+0000","last_became_peered":"2024-09-10T08:05:25.491330+0000","last_unstale":"2024-09-10T08:08:03.671966+0000","last_undegraded":"2024-09-10T08:08:03.671966+0000","last_fullsized":"2024-09-10T08:08:03.671966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:24:06.043331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.655352+0000","last_change":"2024-09-10T08:05:25.492220+0000","last_active":"2024-09-10T08:08:03.655352+0000","last_peered":"2024-09-10T08:08:03.655352+0000","last_clean":"2024-09-10T08:08:03.655352+0000","last_became_active":"2024-09-10T08:05:25.491473+0000","last_became_peered":"2024-09-10T08:05:25.491473+0000","last_unstale":"2024-09-10T08:08:03.655352+0000","last_undegraded":"2024-09-10T08:08:03.655352+0000","last_fullsized":"2024-09-10T08:08:03.655352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:13:30.278434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676184+0000","last_change":"2024-09-10T08:05:25.488927+0000","last_active":"2024-09-10T08:08:03.676184+0000","last_peered":"2024-09-10T08:08:03.676184+0000","last_clean":"2024-09-10T08:08:03.676184+0000","last_became_active":"2024-09-10T08:05:25.488550+0000","last_became_peered":"2024-09-10T08:05:25.488550+0000","last_unstale":"2024-09-10T08:08:03.676184+0000","last_undegraded":"2024-09-10T08:08:03.676184+0000","last_fullsized":"2024-09-10T08:08:03.676184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:44.682999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.677654+0000","last_change":"2024-09-10T08:05:25.490125+0000","last_active":"2024-09-10T08:08:03.677654+0000","last_peered":"2024-09-10T08:08:03.677654+0000","last_clean":"2024-09-10T08:08:03.677654+0000","last_became_active":"2024-09-10T08:05:25.487805+0000","last_became_peered":"2024-09-10T08:05:25.487805+0000","last_unstale":"2024-09-10T08:08:03.677654+0000","last_undegraded":"2024-09-10T08:08:03.677654+0000","last_fullsized":"2024-09-10T08:08:03.677654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:33.479009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676794+0000","last_change":"2024-09-10T08:05:25.500346+0000","last_active":"2024-09-10T08:08:03.676794+0000","last_peered":"2024-09-10T08:08:03.676794+0000","last_clean":"2024-09-10T08:08:03.676794+0000","last_became_active":"2024-09-10T08:05:25.498289+0000","last_became_peered":"2024-09-10T08:05:25.498289+0000","last_unstale":"2024-09-10T08:08:03.676794+0000","last_undegraded":"2024-09-10T08:08:03.676794+0000","last_fullsized":"2024-09-10T08:08:03.676794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:12.054926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.676196+0000","last_change":"2024-09-10T08:05:25.490001+0000","last_active":"2024-09-10T08:08:03.676196+0000","last_peered":"2024-09-10T08:08:03.676196+0000","last_clean":"2024-09-10T08:08:03.676196+0000","last_became_active":"2024-09-10T08:05:25.489706+0000","last_became_peered":"2024-09-10T08:05:25.489706+0000","last_unstale":"2024-09-10T08:08:03.676196+0000","last_undegraded":"2024-09-10T08:08:03.676196+0000","last_fullsized":"2024-09-10T08:08:03.676196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:38.822140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.669334+0000","last_change":"2024-09-10T08:05:25.493874+0000","last_active":"2024-09-10T08:08:03.669334+0000","last_peered":"2024-09-10T08:08:03.669334+0000","last_clean":"2024-09-10T08:08:03.669334+0000","last_became_active":"2024-09-10T08:05:25.489911+0000","last_became_peered":"2024-09-10T08:05:25.489911+0000","last_unstale":"2024-09-10T08:08:03.669334+0000","last_undegraded":"2024-09-10T08:08:03.669334+0000","last_fullsized":"2024-09-10T08:08:03.669334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:46.509657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.654960+0000","last_change":"2024-09-10T08:05:25.490277+0000","last_active":"2024-09-10T08:08:03.654960+0000","last_peered":"2024-09-10T08:08:03.654960+0000","last_clean":"2024-09-10T08:08:03.654960+0000","last_became_active":"2024-09-10T08:05:25.486863+0000","last_became_peered":"2024-09-10T08:05:25.486863+0000","last_unstale":"2024-09-10T08:08:03.654960+0000","last_undegraded":"2024-09-10T08:08:03.654960+0000","last_fullsized":"2024-09-10T08:08:03.654960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:44:21.353830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.656794+0000","last_change":"2024-09-10T08:05:25.491028+0000","last_active":"2024-09-10T08:08:03.656794+0000","last_peered":"2024-09-10T08:08:03.656794+0000","last_clean":"2024-09-10T08:08:03.656794+0000","last_became_active":"2024-09-10T08:05:25.490452+0000","last_became_peered":"2024-09-10T08:05:25.490452+0000","last_unstale":"2024-09-10T08:08:03.656794+0000","last_undegraded":"2024-09-10T08:08:03.656794+0000","last_fullsized":"2024-09-10T08:08:03.656794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:53.059077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660628+0000","last_change":"2024-09-10T08:05:25.489991+0000","last_active":"2024-09-10T08:08:03.660628+0000","last_peered":"2024-09-10T08:08:03.660628+0000","last_clean":"2024-09-10T08:08:03.660628+0000","last_became_active":"2024-09-10T08:05:25.489662+0000","last_became_peered":"2024-09-10T08:05:25.489662+0000","last_unstale":"2024-09-10T08:08:03.660628+0000","last_undegraded":"2024-09-10T08:08:03.660628+0000","last_fullsized":"2024-09-10T08:08:03.660628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:55:47.940712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.667243+0000","last_change":"2024-09-10T08:05:25.475752+0000","last_active":"2024-09-10T08:08:03.667243+0000","last_peered":"2024-09-10T08:08:03.667243+0000","last_clean":"2024-09-10T08:08:03.667243+0000","last_became_active":"2024-09-10T08:05:25.475395+0000","last_became_peered":"2024-09-10T08:05:25.475395+0000","last_unstale":"2024-09-10T08:08:03.667243+0000","last_undegraded":"2024-09-10T08:08:03.667243+0000","last_fullsized":"2024-09-10T08:08:03.667243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.240154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.672245+0000","last_change":"2024-09-10T08:05:25.503061+0000","last_active":"2024-09-10T08:08:03.672245+0000","last_peered":"2024-09-10T08:08:03.672245+0000","last_clean":"2024-09-10T08:08:03.672245+0000","last_became_active":"2024-09-10T08:05:25.501333+0000","last_became_peered":"2024-09-10T08:05:25.501333+0000","last_unstale":"2024-09-10T08:08:03.672245+0000","last_undegraded":"2024-09-10T08:08:03.672245+0000","last_fullsized":"2024-09-10T08:08:03.672245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:38.254777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.661256+0000","last_change":"2024-09-10T08:07:05.514631+0000","last_active":"2024-09-10T08:08:03.661256+0000","last_peered":"2024-09-10T08:08:03.661256+0000","last_clean":"2024-09-10T08:08:03.661256+0000","last_became_active":"2024-09-10T08:07:05.514285+0000","last_became_peered":"2024-09-10T08:07:05.514285+0000","last_unstale":"2024-09-10T08:08:03.661256+0000","last_undegraded":"2024-09-10T08:08:03.661256+0000","last_fullsized":"2024-09-10T08:08:03.661256+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:40.293121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660092+0000","last_change":"2024-09-10T08:05:25.491457+0000","last_active":"2024-09-10T08:08:03.660092+0000","last_peered":"2024-09-10T08:08:03.660092+0000","last_clean":"2024-09-10T08:08:03.660092+0000","last_became_active":"2024-09-10T08:05:25.491162+0000","last_became_peered":"2024-09-10T08:05:25.491162+0000","last_unstale":"2024-09-10T08:08:03.660092+0000","last_undegraded":"2024-09-10T08:08:03.660092+0000","last_fullsized":"2024-09-10T08:08:03.660092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:14.389752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.660737+0000","last_change":"2024-09-10T08:05:25.487874+0000","last_active":"2024-09-10T08:08:03.660737+0000","last_peered":"2024-09-10T08:08:03.660737+0000","last_clean":"2024-09-10T08:08:03.660737+0000","last_became_active":"2024-09-10T08:05:25.487497+0000","last_became_peered":"2024-09-10T08:05:25.487497+0000","last_unstale":"2024-09-10T08:08:03.660737+0000","last_undegraded":"2024-09-10T08:08:03.660737+0000","last_fullsized":"2024-09-10T08:08:03.660737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:26.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.658875+0000","last_change":"2024-09-10T08:05:25.490018+0000","last_active":"2024-09-10T08:08:03.658875+0000","last_peered":"2024-09-10T08:08:03.658875+0000","last_clean":"2024-09-10T08:08:03.658875+0000","last_became_active":"2024-09-10T08:05:25.484951+0000","last_became_peered":"2024-09-10T08:05:25.484951+0000","last_unstale":"2024-09-10T08:08:03.658875+0000","last_undegraded":"2024-09-10T08:08:03.658875+0000","last_fullsized":"2024-09-10T08:08:03.658875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:07.516154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.677343+0000","last_change":"2024-09-10T08:05:25.501581+0000","last_active":"2024-09-10T08:08:03.677343+0000","last_peered":"2024-09-10T08:08:03.677343+0000","last_clean":"2024-09-10T08:08:03.677343+0000","last_became_active":"2024-09-10T08:05:25.499690+0000","last_became_peered":"2024-09-10T08:05:25.499690+0000","last_unstale":"2024-09-10T08:08:03.677343+0000","last_undegraded":"2024-09-10T08:08:03.677343+0000","last_fullsized":"2024-09-10T08:08:03.677343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:14.569791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.678243+0000","last_change":"2024-09-10T08:05:25.487945+0000","last_active":"2024-09-10T08:08:03.678243+0000","last_peered":"2024-09-10T08:08:03.678243+0000","last_clean":"2024-09-10T08:08:03.678243+0000","last_became_active":"2024-09-10T08:05:25.487655+0000","last_became_peered":"2024-09-10T08:05:25.487655+0000","last_unstale":"2024-09-10T08:08:03.678243+0000","last_undegraded":"2024-09-10T08:08:03.678243+0000","last_fullsized":"2024-09-10T08:08:03.678243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:37.746037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"95'8","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T08:08:03.675926+0000","last_change":"2024-09-10T08:05:25.489348+0000","last_active":"2024-09-10T08:08:03.675926+0000","last_peered":"2024-09-10T08:08:03.675926+0000","last_clean":"2024-09-10T08:08:03.675926+0000","last_became_active":"2024-09-10T08:05:25.489004+0000","last_became_peered":"2024-09-10T08:05:25.489004+0000","last_unstale":"2024-09-10T08:08:03.675926+0000","last_undegraded":"2024-09-10T08:08:03.675926+0000","last_fullsized":"2024-09-10T08:08:03.675926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:05:24.455239+0000","last_clean_scrub_stamp":"2024-09-10T08:05:24.455239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:54.554719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76748,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":831,"num_read_kb":501,"num_write":190,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3725,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":230244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214976,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514548,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":548036,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743482,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32364,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712788,"statfs":{"total":95995035648,"available":95961894912,"internally_reserved":0,"allocated":1208320,"data_stored":1211278,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907005,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":548250,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135941,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":548148,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364872,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1114112,"data_stored":1145319,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528398,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":543426,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790037,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1122304,"data_stored":1140514,"data_compressed":68767,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953561,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":619976,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":543620,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":545924,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":614843,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":544814,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34732,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36378,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":73478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:08:38.390 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:08:38.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:38 smithi002 ceph-mon[30563]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:38.770 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:38.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:38 smithi155 ceph-mon[35405]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:38.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:38 smithi107 ceph-mon[35048]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:08:39.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.343+0000 7fd557200640 1 -- 172.21.15.2:0/546970941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd550103ff0 msgr2=0x7fd5501043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:39.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.343+0000 7fd557200640 1 --2- 172.21.15.2:0/546970941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd550103ff0 0x7fd5501043d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd540009900 tx=0x7fd54002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:08:39.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.343+0000 7fd557200640 1 -- 172.21.15.2:0/546970941 shutdown_connections 2024-09-10T08:08:39.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.343+0000 7fd557200640 1 --2- 172.21.15.2:0/546970941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd550109120 0x7fd550110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:39.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.343+0000 7fd557200640 1 --2- 172.21.15.2:0/546970941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5501049a0 0x7fd5501089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:39.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.343+0000 7fd557200640 1 --2- 172.21.15.2:0/546970941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd550103ff0 0x7fd5501043d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:39.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.343+0000 7fd557200640 1 -- 172.21.15.2:0/546970941 >> 172.21.15.2:0/546970941 conn(0x7fd5500ffa90 msgr2=0x7fd550101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:39.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.343+0000 7fd557200640 1 -- 172.21.15.2:0/546970941 shutdown_connections 2024-09-10T08:08:39.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.343+0000 7fd557200640 1 -- 172.21.15.2:0/546970941 wait complete. 2024-09-10T08:08:39.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.344+0000 7fd557200640 1 Processor -- start 2024-09-10T08:08:39.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.344+0000 7fd557200640 1 -- start start 2024-09-10T08:08:39.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.344+0000 7fd557200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd550103ff0 0x7fd55019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:39.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.344+0000 7fd557200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5501049a0 0x7fd55019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:39.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.344+0000 7fd557200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd550109120 0x7fd55019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:39.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.344+0000 7fd557200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd550114120 con 0x7fd550103ff0 2024-09-10T08:08:39.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.344+0000 7fd557200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd550113fa0 con 0x7fd550109120 2024-09-10T08:08:39.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.344+0000 7fd557200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5501142a0 con 0x7fd5501049a0 2024-09-10T08:08:39.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.345+0000 7fd54fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd550103ff0 0x7fd55019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:39.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.345+0000 7fd54fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd550103ff0 0x7fd55019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40352/0 (socket says 172.21.15.2:40352) 2024-09-10T08:08:39.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.345+0000 7fd54fe00640 1 -- 172.21.15.2:0/1132048151 learned_addr learned my addr 172.21.15.2:0/1132048151 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:39.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.345+0000 7fd555200640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd550109120 0x7fd55019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:39.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.345+0000 7fd547e00640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5501049a0 0x7fd55019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:39.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.345+0000 7fd54fe00640 1 -- 172.21.15.2:0/1132048151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5501049a0 msgr2=0x7fd55019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:39.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.345+0000 7fd54fe00640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5501049a0 0x7fd55019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:39.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.345+0000 7fd54fe00640 1 -- 172.21.15.2:0/1132048151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd550109120 msgr2=0x7fd55019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:39.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.345+0000 7fd54fe00640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd550109120 0x7fd55019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:39.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.345+0000 7fd54fe00640 1 -- 172.21.15.2:0/1132048151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55019f590 con 0x7fd550103ff0 2024-09-10T08:08:39.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.345+0000 7fd54fe00640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd550103ff0 0x7fd55019a5e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd540006010 tx=0x7fd5400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:39.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.346+0000 7fd54d600640 1 -- 172.21.15.2:0/1132048151 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54003f030 con 0x7fd550103ff0 2024-09-10T08:08:39.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.346+0000 7fd54d600640 1 -- 172.21.15.2:0/1132048151 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd540003ce0 con 0x7fd550103ff0 2024-09-10T08:08:39.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.346+0000 7fd54d600640 1 -- 172.21.15.2:0/1132048151 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540035070 con 0x7fd550103ff0 2024-09-10T08:08:39.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.346+0000 7fd557200640 1 -- 172.21.15.2:0/1132048151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55019f820 con 0x7fd550103ff0 2024-09-10T08:08:39.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.346+0000 7fd557200640 1 -- 172.21.15.2:0/1132048151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd55019fcb0 con 0x7fd550103ff0 2024-09-10T08:08:39.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.347+0000 7fd557200640 1 -- 172.21.15.2:0/1132048151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5501043d0 con 0x7fd550103ff0 2024-09-10T08:08:39.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.350+0000 7fd54d600640 1 -- 172.21.15.2:0/1132048151 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fd540042030 con 0x7fd550103ff0 2024-09-10T08:08:39.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.351+0000 7fd54d600640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5340a2c10 0x7fd5340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:39.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.351+0000 7fd54d600640 1 -- 172.21.15.2:0/1132048151 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd5400efa80 con 0x7fd550103ff0 2024-09-10T08:08:39.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.351+0000 7fd547e00640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5340a2c10 0x7fd5340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:39.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.351+0000 7fd547e00640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5340a2c10 0x7fd5340a50d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd5500ffeb0 tx=0x7fd53800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:39.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.355+0000 7fd54d600640 1 -- 172.21.15.2:0/1132048151 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5400f1e20 con 0x7fd550103ff0 2024-09-10T08:08:39.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:39 smithi002 ceph-mon[30563]: pgmap v402: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-10T08:08:39.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.580+0000 7fd557200640 1 -- 172.21.15.2:0/1132048151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd55002d7c0 con 0x7fd550103ff0 2024-09-10T08:08:39.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.581+0000 7fd54d600640 1 -- 172.21.15.2:0/1132048151 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fd5400bbea0 con 0x7fd550103ff0 2024-09-10T08:08:39.580 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:39.580 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:08:39.580 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:08:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.583+0000 7fd557200640 1 -- 172.21.15.2:0/1132048151 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5340a2c10 msgr2=0x7fd5340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.583+0000 7fd557200640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5340a2c10 0x7fd5340a50d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd5500ffeb0 tx=0x7fd53800b410 comp rx=0 tx=0).stop 2024-09-10T08:08:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.583+0000 7fd557200640 1 -- 172.21.15.2:0/1132048151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd550103ff0 msgr2=0x7fd55019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.583+0000 7fd557200640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd550103ff0 0x7fd55019a5e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd540006010 tx=0x7fd5400042f0 comp rx=0 tx=0).stop 2024-09-10T08:08:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.583+0000 7fd557200640 1 -- 172.21.15.2:0/1132048151 shutdown_connections 2024-09-10T08:08:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.583+0000 7fd557200640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd550109120 0x7fd55019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.583+0000 7fd557200640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5340a2c10 0x7fd5340a50d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.583+0000 7fd557200640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5501049a0 0x7fd55019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.583+0000 7fd557200640 1 --2- 172.21.15.2:0/1132048151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd550103ff0 0x7fd55019a5e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.583+0000 7fd557200640 1 -- 172.21.15.2:0/1132048151 >> 172.21.15.2:0/1132048151 conn(0x7fd5500ffa90 msgr2=0x7fd550100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.583+0000 7fd557200640 1 -- 172.21.15.2:0/1132048151 shutdown_connections 2024-09-10T08:08:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:39.583+0000 7fd557200640 1 -- 172.21.15.2:0/1132048151 wait complete. 2024-09-10T08:08:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:39 smithi155 ceph-mon[35405]: pgmap v402: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-10T08:08:39.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:39 smithi107 ceph-mon[35048]: pgmap v402: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-10T08:08:40.378 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:08:40.757 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:40.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:40 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1132048151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:40.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:40 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1132048151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:40.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:40 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1132048151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:41.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.335+0000 7f28f0c00640 1 -- 172.21.15.2:0/1346213012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec109140 msgr2=0x7f28ec110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:41.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.335+0000 7f28f0c00640 1 --2- 172.21.15.2:0/1346213012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec109140 0x7f28ec110cc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009ca0 tx=0x7f28e402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:08:41.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.335+0000 7f28f0c00640 1 -- 172.21.15.2:0/1346213012 shutdown_connections 2024-09-10T08:08:41.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.335+0000 7f28f0c00640 1 --2- 172.21.15.2:0/1346213012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec109140 0x7f28ec110cc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:41.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.335+0000 7f28f0c00640 1 --2- 172.21.15.2:0/1346213012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ec1049c0 0x7f28ec108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:41.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.335+0000 7f28f0c00640 1 --2- 172.21.15.2:0/1346213012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28ec104010 0x7f28ec1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:41.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.335+0000 7f28f0c00640 1 -- 172.21.15.2:0/1346213012 >> 172.21.15.2:0/1346213012 conn(0x7f28ec0ffa50 msgr2=0x7f28ec101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:41.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.335+0000 7f28f0c00640 1 -- 172.21.15.2:0/1346213012 shutdown_connections 2024-09-10T08:08:41.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.335+0000 7f28f0c00640 1 -- 172.21.15.2:0/1346213012 wait complete. 2024-09-10T08:08:41.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.336+0000 7f28f0c00640 1 Processor -- start 2024-09-10T08:08:41.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.336+0000 7f28f0c00640 1 -- start start 2024-09-10T08:08:41.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.337+0000 7f28f0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ec104010 0x7f28ec19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:41.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.337+0000 7f28f0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28ec1049c0 0x7f28ec19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:41.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.337+0000 7f28f0c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec109140 0x7f28ec19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:41.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.337+0000 7f28f0c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec114170 con 0x7f28ec109140 2024-09-10T08:08:41.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.337+0000 7f28f0c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec113ff0 con 0x7f28ec104010 2024-09-10T08:08:41.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.337+0000 7f28f0c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1142f0 con 0x7f28ec1049c0 2024-09-10T08:08:41.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.337+0000 7f28e9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ec104010 0x7f28ec19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:41.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.337+0000 7f28ea800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec109140 0x7f28ec19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:41.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.337+0000 7f28e9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ec104010 0x7f28ec19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36046/0 (socket says 172.21.15.2:36046) 2024-09-10T08:08:41.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.337+0000 7f28e9e00640 1 -- 172.21.15.2:0/208159085 learned_addr learned my addr 172.21.15.2:0/208159085 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:41.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.337+0000 7f28e9400640 1 --2- 172.21.15.2:0/208159085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28ec1049c0 0x7f28ec19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:41.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.338+0000 7f28e9e00640 1 -- 172.21.15.2:0/208159085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28ec1049c0 msgr2=0x7f28ec19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:41.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.338+0000 7f28e9e00640 1 --2- 172.21.15.2:0/208159085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28ec1049c0 0x7f28ec19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:41.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.338+0000 7f28e9e00640 1 -- 172.21.15.2:0/208159085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec109140 msgr2=0x7f28ec19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:41.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.338+0000 7f28e9e00640 1 --2- 172.21.15.2:0/208159085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec109140 0x7f28ec19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:41.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.338+0000 7f28e9e00640 1 -- 172.21.15.2:0/208159085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec19f5c0 con 0x7f28ec104010 2024-09-10T08:08:41.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.338+0000 7f28e9e00640 1 --2- 172.21.15.2:0/208159085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ec104010 0x7f28ec19a610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00e9f0 tx=0x7f28dc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:41.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.338+0000 7f28e2a00640 1 -- 172.21.15.2:0/208159085 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc00cd80 con 0x7f28ec104010 2024-09-10T08:08:41.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.338+0000 7f28f0c00640 1 -- 172.21.15.2:0/208159085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec19f8b0 con 0x7f28ec104010 2024-09-10T08:08:41.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.338+0000 7f28e2a00640 1 -- 172.21.15.2:0/208159085 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28dc005500 con 0x7f28ec104010 2024-09-10T08:08:41.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.338+0000 7f28e2a00640 1 -- 172.21.15.2:0/208159085 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc010690 con 0x7f28ec104010 2024-09-10T08:08:41.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.338+0000 7f28f0c00640 1 -- 172.21.15.2:0/208159085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec1a7190 con 0x7f28ec104010 2024-09-10T08:08:41.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.339+0000 7f28f0c00640 1 -- 172.21.15.2:0/208159085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ec1043f0 con 0x7f28ec104010 2024-09-10T08:08:41.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.341+0000 7f28e2a00640 1 -- 172.21.15.2:0/208159085 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f28dc0046c0 con 0x7f28ec104010 2024-09-10T08:08:41.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.343+0000 7f28e2a00640 1 --2- 172.21.15.2:0/208159085 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f28d00a2c10 0x7f28d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:41.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.343+0000 7f28e2a00640 1 -- 172.21.15.2:0/208159085 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f28dc0cf9e0 con 0x7f28ec104010 2024-09-10T08:08:41.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.343+0000 7f28e9400640 1 --2- 172.21.15.2:0/208159085 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f28d00a2c10 0x7f28d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:41.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.343+0000 7f28e9400640 1 --2- 172.21.15.2:0/208159085 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f28d00a2c10 0x7f28d00a50d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f28ec10ab10 tx=0x7f28d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:41.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.345+0000 7f28e2a00640 1 -- 172.21.15.2:0/208159085 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28dc09c390 con 0x7f28ec104010 2024-09-10T08:08:41.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:41 smithi002 ceph-mon[30563]: pgmap v403: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T08:08:41.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.574+0000 7f28f0c00640 1 -- 172.21.15.2:0/208159085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28ec19b970 con 0x7f28ec104010 2024-09-10T08:08:41.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.575+0000 7f28e2a00640 1 -- 172.21.15.2:0/208159085 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f28dc0b3ea0 con 0x7f28ec104010 2024-09-10T08:08:41.574 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:41.575 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:08:41.575 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:08:41.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.578+0000 7f28f0c00640 1 -- 172.21.15.2:0/208159085 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f28d00a2c10 msgr2=0x7f28d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:41.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.578+0000 7f28f0c00640 1 --2- 172.21.15.2:0/208159085 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f28d00a2c10 0x7f28d00a50d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f28ec10ab10 tx=0x7f28d8005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:41.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.578+0000 7f28f0c00640 1 -- 172.21.15.2:0/208159085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ec104010 msgr2=0x7f28ec19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:41.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.578+0000 7f28f0c00640 1 --2- 172.21.15.2:0/208159085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ec104010 0x7f28ec19a610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00e9f0 tx=0x7f28dc00edf0 comp rx=0 tx=0).stop 2024-09-10T08:08:41.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.578+0000 7f28f0c00640 1 -- 172.21.15.2:0/208159085 shutdown_connections 2024-09-10T08:08:41.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.578+0000 7f28f0c00640 1 --2- 172.21.15.2:0/208159085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec109140 0x7f28ec19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:41.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.578+0000 7f28f0c00640 1 --2- 172.21.15.2:0/208159085 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f28d00a2c10 0x7f28d00a50d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:41.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.578+0000 7f28f0c00640 1 --2- 172.21.15.2:0/208159085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28ec1049c0 0x7f28ec19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:41.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.578+0000 7f28f0c00640 1 --2- 172.21.15.2:0/208159085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ec104010 0x7f28ec19a610 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:41.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.578+0000 7f28f0c00640 1 -- 172.21.15.2:0/208159085 >> 172.21.15.2:0/208159085 conn(0x7f28ec0ffa50 msgr2=0x7f28ec100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:41.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.578+0000 7f28f0c00640 1 -- 172.21.15.2:0/208159085 shutdown_connections 2024-09-10T08:08:41.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:41.578+0000 7f28f0c00640 1 -- 172.21.15.2:0/208159085 wait complete. 2024-09-10T08:08:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:41 smithi155 ceph-mon[35405]: pgmap v403: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T08:08:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:41 smithi107 ceph-mon[35048]: pgmap v403: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-10T08:08:42.400 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:08:42.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:42 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/208159085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:42.781 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:42.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:42 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/208159085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:42.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:42 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/208159085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.039+0000 7f0f16e00640 1 -- 172.21.15.2:0/360334674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f10104f50 msgr2=0x7f0f101053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.039+0000 7f0f16e00640 1 --2- 172.21.15.2:0/360334674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f10104f50 0x7f0f101053d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009900 tx=0x7f0ef802a410 comp rx=0 tx=0).stop 2024-09-10T08:08:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.039+0000 7f0f16e00640 1 -- 172.21.15.2:0/360334674 shutdown_connections 2024-09-10T08:08:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.039+0000 7f0f16e00640 1 --2- 172.21.15.2:0/360334674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f10105910 0x7f0f1010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.039+0000 7f0f16e00640 1 --2- 172.21.15.2:0/360334674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f10104f50 0x7f0f101053d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.039+0000 7f0f16e00640 1 --2- 172.21.15.2:0/360334674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f10103d60 0x7f0f10104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.039+0000 7f0f16e00640 1 -- 172.21.15.2:0/360334674 >> 172.21.15.2:0/360334674 conn(0x7f0f100ff4e0 msgr2=0x7f0f10101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.039+0000 7f0f16e00640 1 -- 172.21.15.2:0/360334674 shutdown_connections 2024-09-10T08:08:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.039+0000 7f0f16e00640 1 -- 172.21.15.2:0/360334674 wait complete. 2024-09-10T08:08:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.040+0000 7f0f16e00640 1 Processor -- start 2024-09-10T08:08:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.040+0000 7f0f16e00640 1 -- start start 2024-09-10T08:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.040+0000 7f0f16e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f10103d60 0x7f0f10199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.040+0000 7f0f16e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f10104f50 0x7f0f1019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.040+0000 7f0f16e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f10105910 0x7f0f101a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.040+0000 7f0f16e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1010fa30 con 0x7f0f10104f50 2024-09-10T08:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.040+0000 7f0f16e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1010f8b0 con 0x7f0f10103d60 2024-09-10T08:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.040+0000 7f0f16e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1010fbb0 con 0x7f0f10105910 2024-09-10T08:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.040+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f10104f50 0x7f0f1019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.041+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f10103d60 0x7f0f10199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.041+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f10104f50 0x7f0f1019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40380/0 (socket says 172.21.15.2:40380) 2024-09-10T08:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.041+0000 7f0f07e00640 1 -- 172.21.15.2:0/1482208131 learned_addr learned my addr 172.21.15.2:0/1482208131 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.041+0000 7f0f14e00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f10105910 0x7f0f101a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.041+0000 7f0f0fe00640 1 -- 172.21.15.2:0/1482208131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f10105910 msgr2=0x7f0f101a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.041+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f10105910 0x7f0f101a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.041+0000 7f0f0fe00640 1 -- 172.21.15.2:0/1482208131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f10104f50 msgr2=0x7f0f1019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.041+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f10104f50 0x7f0f1019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.041+0000 7f0f0fe00640 1 -- 172.21.15.2:0/1482208131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f101a1bc0 con 0x7f0f10103d60 2024-09-10T08:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.041+0000 7f0f14e00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f10105910 0x7f0f101a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.041+0000 7f0f07e00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f10104f50 0x7f0f1019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.041+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f10103d60 0x7f0f10199fa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000e9c0 tx=0x7f0f0000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.041+0000 7f0f0d600640 1 -- 172.21.15.2:0/1482208131 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000ccc0 con 0x7f0f10103d60 2024-09-10T08:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.042+0000 7f0f0d600640 1 -- 172.21.15.2:0/1482208131 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f000035f0 con 0x7f0f10103d60 2024-09-10T08:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.042+0000 7f0f16e00640 1 -- 172.21.15.2:0/1482208131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f101a1eb0 con 0x7f0f10103d60 2024-09-10T08:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.042+0000 7f0f0d600640 1 -- 172.21.15.2:0/1482208131 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000ce60 con 0x7f0f10103d60 2024-09-10T08:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.042+0000 7f0f16e00640 1 -- 172.21.15.2:0/1482208131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f101a2310 con 0x7f0f10103d60 2024-09-10T08:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f19c71740 1 Processor -- start 2024-09-10T08:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f19c71740 1 -- start start 2024-09-10T08:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f0d600640 1 -- 172.21.15.2:0/1482208131 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f0f000037a0 con 0x7f0f10103d60 2024-09-10T08:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f19c71740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb107090 0x55f9bb107490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f19c71740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb1079d0 0x55f9bb10c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f19c71740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb10ca60 0x55f9bb10ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f19c71740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb06c100 con 0x55f9bb1079d0 2024-09-10T08:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f19c71740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb0141b0 con 0x55f9bb10ca60 2024-09-10T08:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f19c71740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb0680c0 con 0x55f9bb107090 2024-09-10T08:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb1079d0 0x55f9bb10c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f14e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb107090 0x55f9bb107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb1079d0 0x55f9bb10c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40396/0 (socket says 172.21.15.2:40396) 2024-09-10T08:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f0fe00640 1 -- 172.21.15.2:0/4292099561 learned_addr learned my addr 172.21.15.2:0/4292099561 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.045+0000 7f0f07e00640 1 --2- 172.21.15.2:0/4292099561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb10ca60 0x55f9bb10ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.046+0000 7f0f0fe00640 1 -- 172.21.15.2:0/4292099561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb107090 msgr2=0x55f9bb107490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.046+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/4292099561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb107090 0x55f9bb107490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.046+0000 7f0f0fe00640 1 -- 172.21.15.2:0/4292099561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb10ca60 msgr2=0x55f9bb10ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.046+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/4292099561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb10ca60 0x55f9bb10ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.046+0000 7f0f0fe00640 1 -- 172.21.15.2:0/4292099561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bad7f210 con 0x55f9bb1079d0 2024-09-10T08:08:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.046+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/4292099561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb1079d0 0x55f9bb10c490 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0f000d3ab0 tx=0x7f0f000d3f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fe10e48a18122 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.046+0000 7f0f05600640 1 -- 172.21.15.2:0/4292099561 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f000d8d90 con 0x55f9bb1079d0 2024-09-10T08:08:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.047+0000 7f0f05600640 1 -- 172.21.15.2:0/4292099561 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f000dd920 con 0x55f9bb1079d0 2024-09-10T08:08:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.047+0000 7f0f05600640 1 -- 172.21.15.2:0/4292099561 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f000dc9c0 con 0x55f9bb1079d0 2024-09-10T08:08:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.047+0000 7f0f0d600640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0ef40a2cd0 0x7f0ef40a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.047+0000 7f0f0d600640 1 -- 172.21.15.2:0/1482208131 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0f000cfad0 con 0x7f0f10103d60 2024-09-10T08:08:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.047+0000 7f0f14e00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0ef40a2cd0 0x7f0ef40a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.047+0000 7f0f19c71740 1 -- 172.21.15.2:0/4292099561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb1079d0 msgr2=0x55f9bb10c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.047+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4292099561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb1079d0 0x55f9bb10c490 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0f000d3ab0 tx=0x7f0f000d3f70 comp rx=0 tx=0).stop 2024-09-10T08:08:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.047+0000 7f0f19c71740 1 -- 172.21.15.2:0/4292099561 shutdown_connections 2024-09-10T08:08:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.047+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4292099561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb10ca60 0x55f9bb10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.047+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4292099561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb1079d0 0x55f9bb10c490 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.047+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4292099561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb107090 0x55f9bb107490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.047+0000 7f0f19c71740 1 -- 172.21.15.2:0/4292099561 >> 172.21.15.2:0/4292099561 conn(0x55f9bb102700 msgr2=0x55f9bb102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.048+0000 7f0f19c71740 1 -- 172.21.15.2:0/4292099561 shutdown_connections 2024-09-10T08:08:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.048+0000 7f0f14e00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0ef40a2cd0 0x7f0ef40a5190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0f08005fd0 tx=0x7f0f08009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.048+0000 7f0f19c71740 1 -- 172.21.15.2:0/4292099561 wait complete. 2024-09-10T08:08:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.048+0000 7f0f19c71740 1 -- start start 2024-09-10T08:08:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.048+0000 7f0f19c71740 1 Processor -- start 2024-09-10T08:08:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f19c71740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb107090 0x55f9bb111190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f19c71740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 0x55f9bb1116d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f19c71740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb10ca60 0x55f9bb118710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f19c71740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb10fa30 con 0x55f9bb10ca60 2024-09-10T08:08:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f19c71740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f9baff5090 con 0x55f9bb107090 2024-09-10T08:08:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f19c71740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb10fbb0 con 0x55f9bb1079d0 2024-09-10T08:08:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f19c71740 10 client.0 _ll_register_callbacks cb 0x55f9bb049f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f14e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb10ca60 0x55f9bb118710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f14e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb10ca60 0x55f9bb118710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40404/0 (socket says 172.21.15.2:40404) 2024-09-10T08:08:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f14e00640 1 -- 172.21.15.2:0/1127699744 learned_addr learned my addr 172.21.15.2:0/1127699744 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/1127699744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb107090 0x55f9bb111190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f07e00640 1 --2- 172.21.15.2:0/1127699744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 0x55f9bb1116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f14e00640 1 -- 172.21.15.2:0/1127699744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 msgr2=0x55f9bb1116d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f14e00640 1 --2- 172.21.15.2:0/1127699744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 0x55f9bb1116d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f14e00640 1 -- 172.21.15.2:0/1127699744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb107090 msgr2=0x55f9bb111190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f14e00640 1 --2- 172.21.15.2:0/1127699744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb107090 0x55f9bb111190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.049+0000 7f0f14e00640 1 -- 172.21.15.2:0/1127699744 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bae4b620 con 0x55f9bb10ca60 2024-09-10T08:08:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.050+0000 7f0f14e00640 1 --2- 172.21.15.2:0/1127699744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb10ca60 0x55f9bb118710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0f08010bb0 tx=0x7f0f08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a22ce6a5dbee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.050+0000 7f0efe000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:08:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.050+0000 7f0efe000640 1 -- 172.21.15.2:0/1127699744 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0802c060 con 0x55f9bb10ca60 2024-09-10T08:08:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.050+0000 7f0efe000640 1 -- 172.21.15.2:0/1127699744 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0f08017030 con 0x55f9bb10ca60 2024-09-10T08:08:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.050+0000 7f0f19c71740 1 -- 172.21.15.2:0/1127699744 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f000dd920 con 0x55f9bb10ca60 2024-09-10T08:08:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.050+0000 7f0efe000640 1 -- 172.21.15.2:0/1127699744 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f08033050 con 0x55f9bb10ca60 2024-09-10T08:08:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.050+0000 7f0f19c71740 10 client.15279 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.050+0000 7f0f19c71740 1 -- 172.21.15.2:0/1127699744 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9bb11b5d0 con 0x55f9bb10ca60 2024-09-10T08:08:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.050+0000 7f0efe000640 1 -- 172.21.15.2:0/1127699744 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f000dd920 con 0x55f9bb10ca60 2024-09-10T08:08:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.051+0000 7f0efe000640 1 -- 172.21.15.2:0/1127699744 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0f0801c410 con 0x55f9bb10ca60 2024-09-10T08:08:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.051+0000 7f0f19c71740 10 client.15279 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.051+0000 7f0f19c71740 10 client.15279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:08:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.051+0000 7f0f19c71740 20 client.15279 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.051+0000 7f0f19c71740 1 --2- 172.21.15.2:0/1127699744 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f9bb11cca0 0x55f9bb11f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.051+0000 7f0f19c71740 4 client.15279 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:08:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.051+0000 7f0f19c71740 1 -- 172.21.15.2:0/1127699744 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9bb079830 con 0x55f9bb11cca0 2024-09-10T08:08:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.051+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/1127699744 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f9bb11cca0 0x55f9bb11f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.052+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/1127699744 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f9bb11cca0 0x55f9bb11f0b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d7b88f05cc958f9 server_cookie=2736a134ca492232 in_seq=0 out_seq=0 2024-09-10T08:08:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.052+0000 7f0efe000640 10 client.15279 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.052+0000 7f0efe000640 1 -- 172.21.15.2:0/1127699744 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9bb079830 con 0x55f9bb11cca0 2024-09-10T08:08:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.052+0000 7f0efe000640 10 client.15279 handle_command_reply: tid=0 2024-09-10T08:08:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.052+0000 7f0f19c71740 1 client.15279 shutdown 2024-09-10T08:08:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.052+0000 7f0f19c71740 1 -- 172.21.15.2:0/1127699744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb10ca60 msgr2=0x55f9bb118710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.052+0000 7f0f19c71740 1 --2- 172.21.15.2:0/1127699744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb10ca60 0x55f9bb118710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0f08010bb0 tx=0x7f0f08004a40 comp rx=0 tx=0).stop 2024-09-10T08:08:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.053+0000 7f0f19c71740 1 -- 172.21.15.2:0/1127699744 shutdown_connections 2024-09-10T08:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.053+0000 7f0f19c71740 1 --2- 172.21.15.2:0/1127699744 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f9bb11cca0 0x55f9bb11f0b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.053+0000 7f0f19c71740 1 --2- 172.21.15.2:0/1127699744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb10ca60 0x55f9bb118710 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.053+0000 7f0f19c71740 1 --2- 172.21.15.2:0/1127699744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 0x55f9bb1116d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.053+0000 7f0f19c71740 1 --2- 172.21.15.2:0/1127699744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb107090 0x55f9bb111190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.053+0000 7f0efe000640 0 client.15279 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.053+0000 7f0f19c71740 1 -- 172.21.15.2:0/1127699744 >> 172.21.15.2:0/1127699744 conn(0x55f9bb102700 msgr2=0x55f9bb103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.053+0000 7f0f19c71740 1 -- 172.21.15.2:0/1127699744 shutdown_connections 2024-09-10T08:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.053+0000 7f0f19c71740 1 -- 172.21.15.2:0/1127699744 wait complete. 2024-09-10T08:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.053+0000 7f0f19c71740 20 client.15279 trim_cache size 0 max 16384 2024-09-10T08:08:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.074+0000 7f0f19c71740 1 Processor -- start 2024-09-10T08:08:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.074+0000 7f0f19c71740 1 -- start start 2024-09-10T08:08:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.074+0000 7f0f19c71740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb107090 0x55f9bb101770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.074+0000 7f0f19c71740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 0x55f9bb1186d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.074+0000 7f0f19c71740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb10ca60 0x55f9bb103af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.074+0000 7f0f19c71740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101af3e0 con 0x55f9bb107090 2024-09-10T08:08:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.074+0000 7f0f19c71740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101accb0 con 0x55f9bb10ca60 2024-09-10T08:08:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.074+0000 7f0f19c71740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101aab00 con 0x55f9bb1079d0 2024-09-10T08:08:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.074+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb107090 0x55f9bb101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.074+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb107090 0x55f9bb101770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40418/0 (socket says 172.21.15.2:40418) 2024-09-10T08:08:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.074+0000 7f0f0fe00640 1 -- 172.21.15.2:0/4065542706 learned_addr learned my addr 172.21.15.2:0/4065542706 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.075+0000 7f0f07e00640 1 --2- 172.21.15.2:0/4065542706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 0x55f9bb1186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.075+0000 7f0f14e00640 1 --2- 172.21.15.2:0/4065542706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb10ca60 0x55f9bb103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.075+0000 7f0f0fe00640 1 -- 172.21.15.2:0/4065542706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 msgr2=0x55f9bb1186d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.075+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/4065542706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 0x55f9bb1186d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.075+0000 7f0f0fe00640 1 -- 172.21.15.2:0/4065542706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb10ca60 msgr2=0x55f9bb103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.075+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/4065542706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb10ca60 0x55f9bb103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.075+0000 7f0f0fe00640 1 -- 172.21.15.2:0/4065542706 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bb0fdd60 con 0x55f9bb107090 2024-09-10T08:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.075+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/4065542706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb107090 0x55f9bb101770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0f000d3870 tx=0x7f0f000dc3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bf1923e91d24f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.075+0000 7f0f04c00640 1 -- 172.21.15.2:0/4065542706 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0009bb50 con 0x55f9bb107090 2024-09-10T08:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.075+0000 7f0f04c00640 1 -- 172.21.15.2:0/4065542706 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f0009c480 con 0x55f9bb107090 2024-09-10T08:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.075+0000 7f0f04c00640 1 -- 172.21.15.2:0/4065542706 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0009b6e0 con 0x55f9bb107090 2024-09-10T08:08:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.076+0000 7f0f19c71740 1 -- 172.21.15.2:0/4065542706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb107090 msgr2=0x55f9bb101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.076+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4065542706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb107090 0x55f9bb101770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0f000d3870 tx=0x7f0f000dc3e0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.076+0000 7f0f19c71740 1 -- 172.21.15.2:0/4065542706 shutdown_connections 2024-09-10T08:08:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.076+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4065542706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb10ca60 0x55f9bb103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.076+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4065542706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 0x55f9bb1186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.076+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4065542706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb107090 0x55f9bb101770 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.076+0000 7f0f19c71740 1 -- 172.21.15.2:0/4065542706 >> 172.21.15.2:0/4065542706 conn(0x55f9bb102700 msgr2=0x55f9bb101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.076+0000 7f0f19c71740 1 -- 172.21.15.2:0/4065542706 shutdown_connections 2024-09-10T08:08:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.076+0000 7f0f19c71740 1 -- 172.21.15.2:0/4065542706 wait complete. 2024-09-10T08:08:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.077+0000 7f0f19c71740 1 -- start start 2024-09-10T08:08:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.077+0000 7f0f19c71740 1 Processor -- start 2024-09-10T08:08:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.077+0000 7f0f19c71740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb107090 0x55f9bb115b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.077+0000 7f0f19c71740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 0x55f9bb116040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.077+0000 7f0f19c71740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb10ca60 0x55f9bb117050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.077+0000 7f0f19c71740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101ab170 con 0x55f9bb10ca60 2024-09-10T08:08:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.077+0000 7f0f19c71740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101ac5e0 con 0x55f9bb107090 2024-09-10T08:08:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.077+0000 7f0f19c71740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101ab840 con 0x55f9bb1079d0 2024-09-10T08:08:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.077+0000 7f0f19c71740 10 client.0 _ll_register_callbacks cb 0x7f0f08009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb107090 0x55f9bb115b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0f14e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb10ca60 0x55f9bb117050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0f0fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb107090 0x55f9bb115b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36092/0 (socket says 172.21.15.2:36092) 2024-09-10T08:08:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0f0fe00640 1 -- 172.21.15.2:0/4146184080 learned_addr learned my addr 172.21.15.2:0/4146184080 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0f07e00640 1 --2- 172.21.15.2:0/4146184080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 0x55f9bb116040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0f0fe00640 1 -- 172.21.15.2:0/4146184080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 msgr2=0x55f9bb116040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/4146184080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 0x55f9bb116040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0f0fe00640 1 -- 172.21.15.2:0/4146184080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb10ca60 msgr2=0x55f9bb117050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/4146184080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb10ca60 0x55f9bb117050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0f0fe00640 1 -- 172.21.15.2:0/4146184080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bb117660 con 0x55f9bb107090 2024-09-10T08:08:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0f14e00640 1 --2- 172.21.15.2:0/4146184080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb10ca60 0x55f9bb117050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0f0fe00640 1 --2- 172.21.15.2:0/4146184080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb107090 0x55f9bb115b00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0f000d3ab0 tx=0x7f0f000d8790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1da9f2a58a4b7611 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0efe000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:08:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0efe000640 1 -- 172.21.15.2:0/4146184080 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0009b8c0 con 0x55f9bb107090 2024-09-10T08:08:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0f19c71740 1 -- 172.21.15.2:0/4146184080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f0009c480 con 0x55f9bb107090 2024-09-10T08:08:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.078+0000 7f0efe000640 1 -- 172.21.15.2:0/4146184080 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0f00003b10 con 0x55f9bb107090 2024-09-10T08:08:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.079+0000 7f0efe000640 1 -- 172.21.15.2:0/4146184080 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f0009b6e0 con 0x55f9bb107090 2024-09-10T08:08:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.079+0000 7f0f19c71740 10 client.25307 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.079+0000 7f0f19c71740 1 -- 172.21.15.2:0/4146184080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9bb118970 con 0x55f9bb107090 2024-09-10T08:08:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.079+0000 7f0efe000640 1 -- 172.21.15.2:0/4146184080 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f0009c480 con 0x55f9bb107090 2024-09-10T08:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.079+0000 7f0efe000640 1 -- 172.21.15.2:0/4146184080 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0f000b2030 con 0x55f9bb107090 2024-09-10T08:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.079+0000 7f0f19c71740 10 client.25307 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.079+0000 7f0f19c71740 10 client.25307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.079+0000 7f0f19c71740 20 client.25307 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.079+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4146184080 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f9bb187aa0 0x55f9bb189e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.079+0000 7f0f19c71740 4 client.25307 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.079+0000 7f0f19c71740 1 -- 172.21.15.2:0/4146184080 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9bb073b80 con 0x55f9bb187aa0 2024-09-10T08:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.080+0000 7f0f07e00640 1 --2- 172.21.15.2:0/4146184080 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f9bb187aa0 0x55f9bb189e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.080+0000 7f0f07e00640 1 --2- 172.21.15.2:0/4146184080 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f9bb187aa0 0x55f9bb189e90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f7e740130100f3 server_cookie=dc7e564962b00068 in_seq=0 out_seq=0 2024-09-10T08:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.080+0000 7f0efe000640 10 client.25307 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.095+0000 7f0efe000640 1 -- 172.21.15.2:0/4146184080 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9bb073b80 con 0x55f9bb187aa0 2024-09-10T08:08:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.095+0000 7f0efe000640 10 client.25307 handle_command_reply: tid=0 2024-09-10T08:08:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.095+0000 7f0f19c71740 1 client.25307 shutdown 2024-09-10T08:08:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.095+0000 7f0f19c71740 1 -- 172.21.15.2:0/4146184080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb107090 msgr2=0x55f9bb115b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.095+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4146184080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb107090 0x55f9bb115b00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0f000d3ab0 tx=0x7f0f000d8790 comp rx=0 tx=0).stop 2024-09-10T08:08:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.095+0000 7f0f19c71740 1 -- 172.21.15.2:0/4146184080 shutdown_connections 2024-09-10T08:08:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.095+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4146184080 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f9bb187aa0 0x55f9bb189e90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.095+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4146184080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f9bb10ca60 0x55f9bb117050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.095+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4146184080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bb1079d0 0x55f9bb116040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.095+0000 7f0f19c71740 1 --2- 172.21.15.2:0/4146184080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9bb107090 0x55f9bb115b00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.095+0000 7f0efe000640 0 client.25307 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.095+0000 7f0f19c71740 1 -- 172.21.15.2:0/4146184080 >> 172.21.15.2:0/4146184080 conn(0x55f9bb102700 msgr2=0x55f9bb101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.095+0000 7f0f19c71740 1 -- 172.21.15.2:0/4146184080 shutdown_connections 2024-09-10T08:08:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.096+0000 7f0f19c71740 1 -- 172.21.15.2:0/4146184080 wait complete. 2024-09-10T08:08:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.096+0000 7f0f19c71740 20 client.25307 trim_cache size 0 max 16384 2024-09-10T08:08:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:08:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:08:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:08:43.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:08:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.096+0000 7f0f16e00640 1 -- 172.21.15.2:0/1482208131 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0ef40a2cd0 msgr2=0x7f0ef40a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.096+0000 7f0f16e00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0ef40a2cd0 0x7f0ef40a5190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0f08005fd0 tx=0x7f0f08009500 comp rx=0 tx=0).stop 2024-09-10T08:08:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.096+0000 7f0f16e00640 1 -- 172.21.15.2:0/1482208131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f10103d60 msgr2=0x7f0f10199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.096+0000 7f0f16e00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f10103d60 0x7f0f10199fa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000e9c0 tx=0x7f0f0000ee80 comp rx=0 tx=0).stop 2024-09-10T08:08:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.097+0000 7f0f16e00640 1 -- 172.21.15.2:0/1482208131 shutdown_connections 2024-09-10T08:08:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.097+0000 7f0f16e00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0ef40a2cd0 0x7f0ef40a5190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.097+0000 7f0f16e00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f10105910 0x7f0f101a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.097+0000 7f0f16e00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f10104f50 0x7f0f1019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.097+0000 7f0f16e00640 1 --2- 172.21.15.2:0/1482208131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f10103d60 0x7f0f10199fa0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.097+0000 7f0f16e00640 1 -- 172.21.15.2:0/1482208131 >> 172.21.15.2:0/1482208131 conn(0x7f0f100ff4e0 msgr2=0x7f0f10108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.097+0000 7f0f16e00640 1 -- 172.21.15.2:0/1482208131 shutdown_connections 2024-09-10T08:08:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:43.097+0000 7f0f16e00640 1 -- 172.21.15.2:0/1482208131 wait complete. 2024-09-10T08:08:43.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.360+0000 7f8bb6a00640 1 -- 172.21.15.2:0/4127455370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb01049a0 msgr2=0x7f8bb01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.360+0000 7f8bb6a00640 1 --2- 172.21.15.2:0/4127455370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb01049a0 0x7f8bb01089f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009900 tx=0x7f8b9802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.361+0000 7f8bb6a00640 1 -- 172.21.15.2:0/4127455370 shutdown_connections 2024-09-10T08:08:43.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.361+0000 7f8bb6a00640 1 --2- 172.21.15.2:0/4127455370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb0109120 0x7f8bb0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.361+0000 7f8bb6a00640 1 --2- 172.21.15.2:0/4127455370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb01049a0 0x7f8bb01089f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.361+0000 7f8bb6a00640 1 --2- 172.21.15.2:0/4127455370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bb0103ff0 0x7f8bb01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.361+0000 7f8bb6a00640 1 -- 172.21.15.2:0/4127455370 >> 172.21.15.2:0/4127455370 conn(0x7f8bb00ffa90 msgr2=0x7f8bb0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:43.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.361+0000 7f8bb6a00640 1 -- 172.21.15.2:0/4127455370 shutdown_connections 2024-09-10T08:08:43.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.361+0000 7f8bb6a00640 1 -- 172.21.15.2:0/4127455370 wait complete. 2024-09-10T08:08:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.362+0000 7f8bb6a00640 1 Processor -- start 2024-09-10T08:08:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.362+0000 7f8bb6a00640 1 -- start start 2024-09-10T08:08:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.362+0000 7f8bb6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb0103ff0 0x7f8bb019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.362+0000 7f8bb6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bb01049a0 0x7f8bb019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.362+0000 7f8bb6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb0109120 0x7f8bb019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.362+0000 7f8bb6a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0114140 con 0x7f8bb0109120 2024-09-10T08:08:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.362+0000 7f8bb6a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0113fc0 con 0x7f8bb0103ff0 2024-09-10T08:08:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.362+0000 7f8bb6a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01142c0 con 0x7f8bb01049a0 2024-09-10T08:08:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.362+0000 7f8bb4a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb0109120 0x7f8bb019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.362+0000 7f8bb4a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb0109120 0x7f8bb019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40442/0 (socket says 172.21.15.2:40442) 2024-09-10T08:08:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.362+0000 7f8bb4a00640 1 -- 172.21.15.2:0/2969565168 learned_addr learned my addr 172.21.15.2:0/2969565168 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.362+0000 7f8bafe00640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb0103ff0 0x7f8bb019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.363+0000 7f8ba7e00640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bb01049a0 0x7f8bb019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.363+0000 7f8bb4a00640 1 -- 172.21.15.2:0/2969565168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bb01049a0 msgr2=0x7f8bb019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.363+0000 7f8bb4a00640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bb01049a0 0x7f8bb019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.363+0000 7f8bb4a00640 1 -- 172.21.15.2:0/2969565168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb0103ff0 msgr2=0x7f8bb019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.363+0000 7f8bb4a00640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb0103ff0 0x7f8bb019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.363+0000 7f8bb4a00640 1 -- 172.21.15.2:0/2969565168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb019f5d0 con 0x7f8bb0109120 2024-09-10T08:08:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.363+0000 7f8bb4a00640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb0109120 0x7f8bb019eef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800eb10 tx=0x7f8ba800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.363+0000 7f8bad600640 1 -- 172.21.15.2:0/2969565168 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8004b20 con 0x7f8bb0109120 2024-09-10T08:08:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.363+0000 7f8bad600640 1 -- 172.21.15.2:0/2969565168 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ba8004cc0 con 0x7f8bb0109120 2024-09-10T08:08:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.363+0000 7f8bb6a00640 1 -- 172.21.15.2:0/2969565168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb019f8c0 con 0x7f8bb0109120 2024-09-10T08:08:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.363+0000 7f8bad600640 1 -- 172.21.15.2:0/2969565168 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8010760 con 0x7f8bb0109120 2024-09-10T08:08:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.364+0000 7f8bb6a00640 1 -- 172.21.15.2:0/2969565168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb01a71a0 con 0x7f8bb0109120 2024-09-10T08:08:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.365+0000 7f8bb6a00640 1 -- 172.21.15.2:0/2969565168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb01043d0 con 0x7f8bb0109120 2024-09-10T08:08:43.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.366+0000 7f8bad600640 1 -- 172.21.15.2:0/2969565168 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f8ba8005020 con 0x7f8bb0109120 2024-09-10T08:08:43.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.368+0000 7f8bad600640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8b940a2c10 0x7f8b940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:43.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.368+0000 7f8bad600640 1 -- 172.21.15.2:0/2969565168 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8ba80cfb10 con 0x7f8bb0109120 2024-09-10T08:08:43.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.368+0000 7f8bafe00640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8b940a2c10 0x7f8b940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:43.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.369+0000 7f8bafe00640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8b940a2c10 0x7f8b940a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009bd0 tx=0x7f8ba0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:43.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.372+0000 7f8bad600640 1 -- 172.21.15.2:0/2969565168 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ba809c4c0 con 0x7f8bb0109120 2024-09-10T08:08:43.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:43 smithi002 ceph-mon[30563]: pgmap v404: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-10T08:08:43.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.560+0000 7f8bb6a00640 1 -- 172.21.15.2:0/2969565168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8bb010c3c0 con 0x7f8bb0109120 2024-09-10T08:08:43.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.561+0000 7f8bad600640 1 -- 172.21.15.2:0/2969565168 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24887 (secure 0 0 0) 0x7f8ba809bdc0 con 0x7f8bb0109120 2024-09-10T08:08:43.560 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:43.561 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":95,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:07:05.504273+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:08:43.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.564+0000 7f8bb6a00640 1 -- 172.21.15.2:0/2969565168 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8b940a2c10 msgr2=0x7f8b940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.564+0000 7f8bb6a00640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8b940a2c10 0x7f8b940a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009bd0 tx=0x7f8ba0005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:43.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.564+0000 7f8bb6a00640 1 -- 172.21.15.2:0/2969565168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb0109120 msgr2=0x7f8bb019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:43.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.564+0000 7f8bb6a00640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb0109120 0x7f8bb019eef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800eb10 tx=0x7f8ba800c2d0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.564+0000 7f8bb6a00640 1 -- 172.21.15.2:0/2969565168 shutdown_connections 2024-09-10T08:08:43.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.564+0000 7f8bb6a00640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bb0109120 0x7f8bb019eef0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.564+0000 7f8bb6a00640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8b940a2c10 0x7f8b940a50d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.564+0000 7f8bb6a00640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bb01049a0 0x7f8bb019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.564+0000 7f8bb6a00640 1 --2- 172.21.15.2:0/2969565168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb0103ff0 0x7f8bb019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:43.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.564+0000 7f8bb6a00640 1 -- 172.21.15.2:0/2969565168 >> 172.21.15.2:0/2969565168 conn(0x7f8bb00ffa90 msgr2=0x7f8bb0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:43.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.564+0000 7f8bb6a00640 1 -- 172.21.15.2:0/2969565168 shutdown_connections 2024-09-10T08:08:43.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:43.564+0000 7f8bb6a00640 1 -- 172.21.15.2:0/2969565168 wait complete. 2024-09-10T08:08:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:43 smithi155 ceph-mon[35405]: pgmap v404: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-10T08:08:43.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:43 smithi107 ceph-mon[35048]: pgmap v404: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-10T08:08:44.348 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-10T08:08:44.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:44 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2969565168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:08:44.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:44 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:08:44.729 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:44.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:44 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2969565168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:08:44.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:44 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:08:44.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:44 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2969565168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:08:44.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:44 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:08:45.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:45 smithi002 ceph-mon[30563]: pgmap v405: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-10T08:08:45.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:45 smithi155 ceph-mon[35405]: pgmap v405: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-10T08:08:45.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:45 smithi107 ceph-mon[35048]: pgmap v405: 129 pgs: 129 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-10T08:08:46.066 INFO:teuthology.orchestra.run.smithi002.stdout:{"run_dir":"/var/run/ceph"} 2024-09-10T08:08:46.067 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-10T08:08:46.067 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 9893 2024-09-10T08:08:46.067 DEBUG:tasks.quiescer:Yielding 2024-09-10T08:08:46.067 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T08:08:46.082 INFO:tasks.workunit:Pulling workunits from ref 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:08:46.083 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T08:08:46.083 INFO:tasks.workunit:timeout=3h 2024-09-10T08:08:46.083 INFO:tasks.workunit:cleanup=True 2024-09-10T08:08:46.083 DEBUG:teuthology.orchestra.run.smithi002:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T08:08:46.084 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:08:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T08:08:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-10T08:08:46.122 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 54h/84d Inode: 1099511628283 Links: 2 2024-09-10T08:08:46.123 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:08:46.123 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-10T08:08:46.123 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-10 08:06:35.206471793 +0000 2024-09-10T08:08:46.123 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-10 08:06:35.206471793 +0000 2024-09-10T08:08:46.123 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-10 08:08:18.414630554 +0000 2024-09-10T08:08:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-10 08:06:35.206471793 +0000 2024-09-10T08:08:46.123 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T08:08:46.123 DEBUG:teuthology.orchestra.run.smithi002:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T08:08:46.229 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:08:46.268 INFO:tasks.workunit.client.0.smithi002.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T08:08:46.492 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:47.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.078+0000 7fa518200640 1 -- 172.21.15.2:0/3841842732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa51010e430 msgr2=0x7fa510110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:47.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.078+0000 7fa518200640 1 --2- 172.21.15.2:0/3841842732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa51010e430 0x7fa510110820 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa508009ca0 tx=0x7fa50802a950 comp rx=0 tx=0).stop 2024-09-10T08:08:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.078+0000 7fa518200640 1 -- 172.21.15.2:0/3841842732 shutdown_connections 2024-09-10T08:08:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.078+0000 7fa518200640 1 --2- 172.21.15.2:0/3841842732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa51010e430 0x7fa510110820 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.078+0000 7fa518200640 1 --2- 172.21.15.2:0/3841842732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa510074ee0 0x7fa51010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.078+0000 7fa518200640 1 --2- 172.21.15.2:0/3841842732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5100745c0 0x7fa5100749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.078+0000 7fa518200640 1 -- 172.21.15.2:0/3841842732 >> 172.21.15.2:0/3841842732 conn(0x7fa5100fd8b0 msgr2=0x7fa5100ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.078+0000 7fa518200640 1 -- 172.21.15.2:0/3841842732 shutdown_connections 2024-09-10T08:08:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.078+0000 7fa518200640 1 -- 172.21.15.2:0/3841842732 wait complete. 2024-09-10T08:08:47.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.079+0000 7fa518200640 1 Processor -- start 2024-09-10T08:08:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.079+0000 7fa518200640 1 -- start start 2024-09-10T08:08:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.079+0000 7fa518200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5100745c0 0x7fa51019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.079+0000 7fa518200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa510074ee0 0x7fa51019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.079+0000 7fa518200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa51010e430 0x7fa51019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:47.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.079+0000 7fa518200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa510114160 con 0x7fa5100745c0 2024-09-10T08:08:47.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.079+0000 7fa518200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa510113fe0 con 0x7fa51010e430 2024-09-10T08:08:47.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.079+0000 7fa518200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5101142e0 con 0x7fa510074ee0 2024-09-10T08:08:47.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.080+0000 7fa515800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5100745c0 0x7fa51019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:47.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.080+0000 7fa515800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5100745c0 0x7fa51019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55532/0 (socket says 172.21.15.2:55532) 2024-09-10T08:08:47.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.080+0000 7fa515800640 1 -- 172.21.15.2:0/2150547811 learned_addr learned my addr 172.21.15.2:0/2150547811 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:47.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.080+0000 7fa516200640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa51010e430 0x7fa51019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:47.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.080+0000 7fa514e00640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa510074ee0 0x7fa51019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:47.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.080+0000 7fa516200640 1 -- 172.21.15.2:0/2150547811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa510074ee0 msgr2=0x7fa51019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:47.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.080+0000 7fa516200640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa510074ee0 0x7fa51019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:47.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.080+0000 7fa516200640 1 -- 172.21.15.2:0/2150547811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5100745c0 msgr2=0x7fa51019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:47.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.080+0000 7fa516200640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5100745c0 0x7fa51019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:47.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.080+0000 7fa516200640 1 -- 172.21.15.2:0/2150547811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51019f670 con 0x7fa51010e430 2024-09-10T08:08:47.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.081+0000 7fa516200640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa51010e430 0x7fa51019ef90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa50802ae70 tx=0x7fa50802f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:47.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.081+0000 7fa50e000640 1 -- 172.21.15.2:0/2150547811 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508034040 con 0x7fa51010e430 2024-09-10T08:08:47.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.081+0000 7fa50e000640 1 -- 172.21.15.2:0/2150547811 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa508030920 con 0x7fa51010e430 2024-09-10T08:08:47.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.081+0000 7fa518200640 1 -- 172.21.15.2:0/2150547811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51019f900 con 0x7fa51010e430 2024-09-10T08:08:47.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.081+0000 7fa50e000640 1 -- 172.21.15.2:0/2150547811 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508004610 con 0x7fa51010e430 2024-09-10T08:08:47.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.081+0000 7fa518200640 1 -- 172.21.15.2:0/2150547811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5101a7240 con 0x7fa51010e430 2024-09-10T08:08:47.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.082+0000 7fa518200640 1 -- 172.21.15.2:0/2150547811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5100749a0 con 0x7fa51010e430 2024-09-10T08:08:47.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.084+0000 7fa50e000640 1 -- 172.21.15.2:0/2150547811 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fa508037020 con 0x7fa51010e430 2024-09-10T08:08:47.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.085+0000 7fa50e000640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa4f80a2c90 0x7fa4f80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:47.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.086+0000 7fa50e000640 1 -- 172.21.15.2:0/2150547811 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa5080f0450 con 0x7fa51010e430 2024-09-10T08:08:47.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.086+0000 7fa515800640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa4f80a2c90 0x7fa4f80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:47.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.086+0000 7fa515800640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa4f80a2c90 0x7fa4f80a5150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa504009c90 tx=0x7fa504005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:47.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.088+0000 7fa50e000640 1 -- 172.21.15.2:0/2150547811 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5080811b0 con 0x7fa51010e430 2024-09-10T08:08:47.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.312+0000 7fa518200640 1 -- 172.21.15.2:0/2150547811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5101095b0 con 0x7fa51010e430 2024-09-10T08:08:47.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.313+0000 7fa50e000640 1 -- 172.21.15.2:0/2150547811 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fa5080bcf10 con 0x7fa51010e430 2024-09-10T08:08:47.313 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:08:47.313 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:08:47.314 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:08:47.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.315+0000 7fa518200640 1 -- 172.21.15.2:0/2150547811 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa4f80a2c90 msgr2=0x7fa4f80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:47.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.315+0000 7fa518200640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa4f80a2c90 0x7fa4f80a5150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa504009c90 tx=0x7fa504005f90 comp rx=0 tx=0).stop 2024-09-10T08:08:47.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.315+0000 7fa518200640 1 -- 172.21.15.2:0/2150547811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa51010e430 msgr2=0x7fa51019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.315+0000 7fa518200640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa51010e430 0x7fa51019ef90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa50802ae70 tx=0x7fa50802f420 comp rx=0 tx=0).stop 2024-09-10T08:08:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.315+0000 7fa518200640 1 -- 172.21.15.2:0/2150547811 shutdown_connections 2024-09-10T08:08:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.315+0000 7fa518200640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa51010e430 0x7fa51019ef90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.315+0000 7fa518200640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa4f80a2c90 0x7fa4f80a5150 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.315+0000 7fa518200640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa510074ee0 0x7fa51019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.315+0000 7fa518200640 1 --2- 172.21.15.2:0/2150547811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5100745c0 0x7fa51019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.315+0000 7fa518200640 1 -- 172.21.15.2:0/2150547811 >> 172.21.15.2:0/2150547811 conn(0x7fa5100fd8b0 msgr2=0x7fa5100fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.316+0000 7fa518200640 1 -- 172.21.15.2:0/2150547811 shutdown_connections 2024-09-10T08:08:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:47.316+0000 7fa518200640 1 -- 172.21.15.2:0/2150547811 wait complete. 2024-09-10T08:08:47.695 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:47 smithi002 ceph-mon[30563]: pgmap v406: 129 pgs: 129 active+clean; 657 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-10T08:08:47.695 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:47 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2150547811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:47.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:47 smithi155 ceph-mon[35405]: pgmap v406: 129 pgs: 129 active+clean; 657 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-10T08:08:47.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:47 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2150547811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:47.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:47 smithi107 ceph-mon[35048]: pgmap v406: 129 pgs: 129 active+clean; 657 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-10T08:08:47.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:47 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2150547811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:08:48.014 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T08:05:26.574488+0000', 'modified': '2024-09-10T08:08:19.294639+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24536, 'mds_1': 24499, 'mds_2': 24494}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6835/1775771400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 1775771400}, {'type': 'v1', 'addr': '172.21.15.107:6835', 'nonce': 1775771400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6835/1068995103', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 1068995103}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 1068995103}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/3780222531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 3780222531}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 3780222531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.107:6837/3130379099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6836', 'nonce': 3130379099}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 3130379099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.155:6839/2871792117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 2871792117}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 2871792117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.107:6839/3674188134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 3674188134}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 3674188134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24536, 'qdb_cluster': [24536, 24499, 24494]} 2024-09-10T08:08:48.015 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T08:08:48.015 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T08:08:48.437 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:49.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.003+0000 7f949bc00640 1 -- 172.21.15.2:0/1410968315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9494076720 msgr2=0x7f9494076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:49.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.003+0000 7f949bc00640 1 --2- 172.21.15.2:0/1410968315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9494076720 0x7f9494076b00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9484009900 tx=0x7f948402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.003+0000 7f949bc00640 1 -- 172.21.15.2:0/1410968315 shutdown_connections 2024-09-10T08:08:49.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.003+0000 7f949bc00640 1 --2- 172.21.15.2:0/1410968315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9494112a30 0x7f9494114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.003+0000 7f949bc00640 1 --2- 172.21.15.2:0/1410968315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9494077040 0x7f94941124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.003+0000 7f949bc00640 1 --2- 172.21.15.2:0/1410968315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9494076720 0x7f9494076b00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.003+0000 7f949bc00640 1 -- 172.21.15.2:0/1410968315 >> 172.21.15.2:0/1410968315 conn(0x7f94940ffa90 msgr2=0x7f9494101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.003+0000 7f949bc00640 1 -- 172.21.15.2:0/1410968315 shutdown_connections 2024-09-10T08:08:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.004+0000 7f949bc00640 1 -- 172.21.15.2:0/1410968315 wait complete. 2024-09-10T08:08:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.005+0000 7f949bc00640 1 Processor -- start 2024-09-10T08:08:49.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.005+0000 7f949bc00640 1 -- start start 2024-09-10T08:08:49.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.005+0000 7f949bc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9494076720 0x7f94941a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.005+0000 7f949bc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9494077040 0x7f94941a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.005+0000 7f949bc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9494112a30 0x7f949419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.005+0000 7f949bc00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9494118610 con 0x7f9494112a30 2024-09-10T08:08:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.005+0000 7f949bc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9494118490 con 0x7f9494077040 2024-09-10T08:08:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.005+0000 7f949bc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9494118790 con 0x7f9494076720 2024-09-10T08:08:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.005+0000 7f9499c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9494112a30 0x7f949419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.005+0000 7f9499c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9494112a30 0x7f949419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55548/0 (socket says 172.21.15.2:55548) 2024-09-10T08:08:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.005+0000 7f9499c00640 1 -- 172.21.15.2:0/3139857829 learned_addr learned my addr 172.21.15.2:0/3139857829 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.006+0000 7f9499200640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9494076720 0x7f94941a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.006+0000 7f948be00640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9494077040 0x7f94941a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.006+0000 7f9499c00640 1 -- 172.21.15.2:0/3139857829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9494076720 msgr2=0x7f94941a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.006+0000 7f9499c00640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9494076720 0x7f94941a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.006+0000 7f9499c00640 1 -- 172.21.15.2:0/3139857829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9494077040 msgr2=0x7f94941a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.006+0000 7f9499c00640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9494077040 0x7f94941a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.006+0000 7f9499c00640 1 -- 172.21.15.2:0/3139857829 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949419efe0 con 0x7f9494112a30 2024-09-10T08:08:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.006+0000 7f9499c00640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9494112a30 0x7f949419e8b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f948c009fd0 tx=0x7f948c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.007+0000 7f9492000640 1 -- 172.21.15.2:0/3139857829 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c004d20 con 0x7f9494112a30 2024-09-10T08:08:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.007+0000 7f949bc00640 1 -- 172.21.15.2:0/3139857829 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949419f2d0 con 0x7f9494112a30 2024-09-10T08:08:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.007+0000 7f9492000640 1 -- 172.21.15.2:0/3139857829 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f948c005020 con 0x7f9494112a30 2024-09-10T08:08:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.007+0000 7f9492000640 1 -- 172.21.15.2:0/3139857829 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c0116d0 con 0x7f9494112a30 2024-09-10T08:08:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.007+0000 7f949bc00640 1 -- 172.21.15.2:0/3139857829 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94941ab5c0 con 0x7f9494112a30 2024-09-10T08:08:49.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.010+0000 7f949e665740 1 Processor -- start 2024-09-10T08:08:49.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.010+0000 7f9492000640 1 -- 172.21.15.2:0/3139857829 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f948c0051d0 con 0x7f9494112a30 2024-09-10T08:08:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.010+0000 7f949e665740 1 -- start start 2024-09-10T08:08:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.010+0000 7f949e665740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45eb8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.010+0000 7f949e665740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8c50 0x55ba45ec2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.010+0000 7f949e665740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 0x55ba45ec5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.010+0000 7f949e665740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ba45e1e0b0 con 0x55ba45eb8c50 2024-09-10T08:08:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.010+0000 7f949e665740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ba45dc5ab0 con 0x55ba45eb8330 2024-09-10T08:08:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.010+0000 7f949e665740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ba45e1a070 con 0x55ba45ec3510 2024-09-10T08:08:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.011+0000 7f9499200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8c50 0x55ba45ec2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.011+0000 7f948be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45eb8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.011+0000 7f948be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45eb8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:34904/0 (socket says 172.21.15.2:34904) 2024-09-10T08:08:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.011+0000 7f948be00640 1 -- 172.21.15.2:0/3765969841 learned_addr learned my addr 172.21.15.2:0/3765969841 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.011+0000 7f948be00640 1 --2- 172.21.15.2:0/3765969841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 0x55ba45ec5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.011+0000 7f9499200640 1 -- 172.21.15.2:0/3765969841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 msgr2=0x55ba45ec5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.011+0000 7f9499200640 1 --2- 172.21.15.2:0/3765969841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 0x55ba45ec5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.011+0000 7f9499200640 1 -- 172.21.15.2:0/3765969841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 msgr2=0x55ba45eb8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.011+0000 7f9499200640 1 --2- 172.21.15.2:0/3765969841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45eb8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.011+0000 7f9499200640 1 -- 172.21.15.2:0/3765969841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba45dd24a0 con 0x55ba45eb8c50 2024-09-10T08:08:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.011+0000 7f9492000640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94780a2c10 0x7f94780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.012+0000 7f9499200640 1 --2- 172.21.15.2:0/3765969841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8c50 0x55ba45ec2fd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f948402a9e0 tx=0x7f9484005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61bf99bb4d46735 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.012+0000 7f9492000640 1 -- 172.21.15.2:0/3139857829 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f948c0cfa50 con 0x7f9494112a30 2024-09-10T08:08:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.012+0000 7f948be00640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94780a2c10 0x7f94780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.012+0000 7f948a000640 1 -- 172.21.15.2:0/3765969841 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484035040 con 0x55ba45eb8c50 2024-09-10T08:08:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.012+0000 7f948a000640 1 -- 172.21.15.2:0/3765969841 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9484004470 con 0x55ba45eb8c50 2024-09-10T08:08:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.012+0000 7f948a000640 1 -- 172.21.15.2:0/3765969841 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94840037f0 con 0x55ba45eb8c50 2024-09-10T08:08:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.013+0000 7f948be00640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94780a2c10 0x7f94780a50d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f947c00bfd0 tx=0x7f947c004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.013+0000 7f949e665740 1 -- 172.21.15.2:0/3765969841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8c50 msgr2=0x55ba45ec2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.013+0000 7f949e665740 1 --2- 172.21.15.2:0/3765969841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8c50 0x55ba45ec2fd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f948402a9e0 tx=0x7f9484005c00 comp rx=0 tx=0).stop 2024-09-10T08:08:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.013+0000 7f949e665740 1 -- 172.21.15.2:0/3765969841 shutdown_connections 2024-09-10T08:08:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.013+0000 7f949e665740 1 --2- 172.21.15.2:0/3765969841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 0x55ba45ec5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.013+0000 7f949e665740 1 --2- 172.21.15.2:0/3765969841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8c50 0x55ba45ec2fd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.013+0000 7f949e665740 1 --2- 172.21.15.2:0/3765969841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45eb8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.013+0000 7f949e665740 1 -- 172.21.15.2:0/3765969841 >> 172.21.15.2:0/3765969841 conn(0x55ba45eb48c0 msgr2=0x55ba45eb4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.013+0000 7f949e665740 1 -- 172.21.15.2:0/3765969841 shutdown_connections 2024-09-10T08:08:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.013+0000 7f949e665740 1 -- 172.21.15.2:0/3765969841 wait complete. 2024-09-10T08:08:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.014+0000 7f949e665740 1 -- start start 2024-09-10T08:08:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.014+0000 7f949e665740 1 Processor -- start 2024-09-10T08:08:49.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.014+0000 7f949e665740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45ec05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.014+0000 7f949e665740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45eb8c50 0x55ba45ec0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.014+0000 7f949e665740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45ec3510 0x55ba45ec1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.014+0000 7f949e665740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ba45ec6060 con 0x55ba45ec3510 2024-09-10T08:08:49.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.014+0000 7f949e665740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ba45da6920 con 0x55ba45eb8330 2024-09-10T08:08:49.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.014+0000 7f949e665740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ba45ec61e0 con 0x55ba45eb8c50 2024-09-10T08:08:49.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.014+0000 7f949e665740 10 client.0 _ll_register_callbacks cb 0x55ba45dca8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:49.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.015+0000 7f9499c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45ec3510 0x55ba45ec1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.015+0000 7f948be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45eb8c50 0x55ba45ec0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.015+0000 7f9499200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45ec05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.015+0000 7f9499200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45ec05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:34916/0 (socket says 172.21.15.2:34916) 2024-09-10T08:08:49.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.015+0000 7f9499200640 1 -- 172.21.15.2:0/39340157 learned_addr learned my addr 172.21.15.2:0/39340157 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:49.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.015+0000 7f9499c00640 1 -- 172.21.15.2:0/39340157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45eb8c50 msgr2=0x55ba45ec0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:49.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.015+0000 7f9499c00640 1 --2- 172.21.15.2:0/39340157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45eb8c50 0x55ba45ec0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.015+0000 7f9499c00640 1 -- 172.21.15.2:0/39340157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 msgr2=0x55ba45ec05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:49.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.015+0000 7f9499c00640 1 --2- 172.21.15.2:0/39340157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45ec05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.015+0000 7f9499c00640 1 -- 172.21.15.2:0/39340157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba45b8f2b0 con 0x55ba45ec3510 2024-09-10T08:08:49.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.015+0000 7f9499200640 1 --2- 172.21.15.2:0/39340157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45ec05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:49.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.015+0000 7f9499c00640 1 --2- 172.21.15.2:0/39340157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45ec3510 0x55ba45ec1050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f948c0ccfb0 tx=0x7f948c09b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35492ed3da1ebf3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:49.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.015+0000 7f9482a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:08:49.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.016+0000 7f9482a00640 1 -- 172.21.15.2:0/39340157 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c01f030 con 0x55ba45ec3510 2024-09-10T08:08:49.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.016+0000 7f9482a00640 1 -- 172.21.15.2:0/39340157 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f948c0a44e0 con 0x55ba45ec3510 2024-09-10T08:08:49.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.016+0000 7f949e665740 1 -- 172.21.15.2:0/39340157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9484004470 con 0x55ba45ec3510 2024-09-10T08:08:49.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.016+0000 7f9482a00640 1 -- 172.21.15.2:0/39340157 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f948c0a4720 con 0x55ba45ec3510 2024-09-10T08:08:49.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.016+0000 7f949e665740 10 client.15309 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:49.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.016+0000 7f949e665740 1 -- 172.21.15.2:0/39340157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba45ed1eb0 con 0x55ba45ec3510 2024-09-10T08:08:49.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.016+0000 7f9482a00640 1 -- 172.21.15.2:0/39340157 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9484004470 con 0x55ba45ec3510 2024-09-10T08:08:49.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.016+0000 7f9482a00640 1 -- 172.21.15.2:0/39340157 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f948c00f040 con 0x55ba45ec3510 2024-09-10T08:08:49.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.016+0000 7f949e665740 10 client.15309 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:49.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.017+0000 7f949e665740 10 client.15309 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:08:49.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.017+0000 7f949e665740 20 client.15309 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:49.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.017+0000 7f949e665740 1 --2- 172.21.15.2:0/39340157 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55ba45ed44a0 0x55ba45ed5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.017+0000 7f949e665740 4 client.15309 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:08:49.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.017+0000 7f949e665740 1 -- 172.21.15.2:0/39340157 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba45e2b620 con 0x55ba45ed44a0 2024-09-10T08:08:49.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.017+0000 7f9499200640 1 --2- 172.21.15.2:0/39340157 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55ba45ed44a0 0x55ba45ed5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.018+0000 7f9499200640 1 --2- 172.21.15.2:0/39340157 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55ba45ed44a0 0x55ba45ed5910 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9fe741358f39eb2d server_cookie=fb4ab626ded2702c in_seq=0 out_seq=0 2024-09-10T08:08:49.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.018+0000 7f9482a00640 10 client.15309 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:08:49.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.018+0000 7f9482a00640 1 -- 172.21.15.2:0/39340157 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba45e2b620 con 0x55ba45ed44a0 2024-09-10T08:08:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.018+0000 7f9482a00640 10 client.15309 handle_command_reply: tid=0 2024-09-10T08:08:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.018+0000 7f949e665740 1 client.15309 shutdown 2024-09-10T08:08:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.019+0000 7f949e665740 1 -- 172.21.15.2:0/39340157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45ec3510 msgr2=0x55ba45ec1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.019+0000 7f949e665740 1 --2- 172.21.15.2:0/39340157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45ec3510 0x55ba45ec1050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f948c0ccfb0 tx=0x7f948c09b7e0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.019+0000 7f949e665740 1 -- 172.21.15.2:0/39340157 shutdown_connections 2024-09-10T08:08:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.019+0000 7f949e665740 1 --2- 172.21.15.2:0/39340157 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55ba45ed44a0 0x55ba45ed5910 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.019+0000 7f949e665740 1 --2- 172.21.15.2:0/39340157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45ec3510 0x55ba45ec1050 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.019+0000 7f949e665740 1 --2- 172.21.15.2:0/39340157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45eb8c50 0x55ba45ec0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.019+0000 7f949e665740 1 --2- 172.21.15.2:0/39340157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45ec05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.019+0000 7f9482a00640 0 client.15309 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:08:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.019+0000 7f949e665740 1 -- 172.21.15.2:0/39340157 >> 172.21.15.2:0/39340157 conn(0x55ba45eb48c0 msgr2=0x55ba45eba5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.019+0000 7f949e665740 1 -- 172.21.15.2:0/39340157 shutdown_connections 2024-09-10T08:08:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.019+0000 7f949e665740 1 -- 172.21.15.2:0/39340157 wait complete. 2024-09-10T08:08:49.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.019+0000 7f949e665740 20 client.15309 trim_cache size 0 max 16384 2024-09-10T08:08:49.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.038+0000 7f949e665740 1 Processor -- start 2024-09-10T08:08:49.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.038+0000 7f949e665740 1 -- start start 2024-09-10T08:08:49.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.038+0000 7f949e665740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45eb4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.038+0000 7f949e665740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8c50 0x55ba45ebfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.038+0000 7f949e665740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 0x55ba45ebeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.038+0000 7f949e665740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ba45ed3c50 con 0x55ba45eb8c50 2024-09-10T08:08:49.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.038+0000 7f949e665740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ba45ed4320 con 0x55ba45eb8330 2024-09-10T08:08:49.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.038+0000 7f949e665740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ba45ed49f0 con 0x55ba45ec3510 2024-09-10T08:08:49.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.039+0000 7f948be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8c50 0x55ba45ebfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.039+0000 7f948be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8c50 0x55ba45ebfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55560/0 (socket says 172.21.15.2:55560) 2024-09-10T08:08:49.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.039+0000 7f948be00640 1 -- 172.21.15.2:0/1060808714 learned_addr learned my addr 172.21.15.2:0/1060808714 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:49.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.039+0000 7f9499200640 1 --2- 172.21.15.2:0/1060808714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45eb4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.039+0000 7f9499c00640 1 --2- 172.21.15.2:0/1060808714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 0x55ba45ebeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.039+0000 7f948be00640 1 -- 172.21.15.2:0/1060808714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 msgr2=0x55ba45ebeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:49.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.039+0000 7f948be00640 1 --2- 172.21.15.2:0/1060808714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 0x55ba45ebeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.039+0000 7f948be00640 1 -- 172.21.15.2:0/1060808714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 msgr2=0x55ba45eb4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:49.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.039+0000 7f948be00640 1 --2- 172.21.15.2:0/1060808714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45eb4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.039+0000 7f948be00640 1 -- 172.21.15.2:0/1060808714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba45ec2f40 con 0x55ba45eb8c50 2024-09-10T08:08:49.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.039+0000 7f9499200640 1 --2- 172.21.15.2:0/1060808714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45eb4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:49.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.040+0000 7f948be00640 1 --2- 172.21.15.2:0/1060808714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8c50 0x55ba45ebfe90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f947c00e840 tx=0x7f947c014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f57e99e2c2e3e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:49.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.040+0000 7f9489600640 1 -- 172.21.15.2:0/1060808714 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c005020 con 0x55ba45eb8c50 2024-09-10T08:08:49.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.040+0000 7f9489600640 1 -- 172.21.15.2:0/1060808714 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f947c0047a0 con 0x55ba45eb8c50 2024-09-10T08:08:49.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.040+0000 7f9489600640 1 -- 172.21.15.2:0/1060808714 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c02b6d0 con 0x55ba45eb8c50 2024-09-10T08:08:49.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.040+0000 7f949e665740 1 -- 172.21.15.2:0/1060808714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8c50 msgr2=0x55ba45ebfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:49.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.040+0000 7f949e665740 1 --2- 172.21.15.2:0/1060808714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8c50 0x55ba45ebfe90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f947c00e840 tx=0x7f947c014070 comp rx=0 tx=0).stop 2024-09-10T08:08:49.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.041+0000 7f949e665740 1 -- 172.21.15.2:0/1060808714 shutdown_connections 2024-09-10T08:08:49.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.041+0000 7f949e665740 1 --2- 172.21.15.2:0/1060808714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 0x55ba45ebeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.041+0000 7f949e665740 1 --2- 172.21.15.2:0/1060808714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8c50 0x55ba45ebfe90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.041+0000 7f949e665740 1 --2- 172.21.15.2:0/1060808714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8330 0x55ba45eb4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.041+0000 7f949e665740 1 -- 172.21.15.2:0/1060808714 >> 172.21.15.2:0/1060808714 conn(0x55ba45eb48c0 msgr2=0x55ba45ebb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:49.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.041+0000 7f949e665740 1 -- 172.21.15.2:0/1060808714 shutdown_connections 2024-09-10T08:08:49.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.041+0000 7f949e665740 1 -- 172.21.15.2:0/1060808714 wait complete. 2024-09-10T08:08:49.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.041+0000 7f949e665740 1 -- start start 2024-09-10T08:08:49.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.041+0000 7f949e665740 1 Processor -- start 2024-09-10T08:08:49.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f949e665740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8330 0x55ba45ede110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f949e665740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8c50 0x55ba45ed7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f949e665740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 0x55ba45ed7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f949e665740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ba45ed5790 con 0x55ba45eb8330 2024-09-10T08:08:49.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f949e665740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ba45ed50c0 con 0x55ba45eb8c50 2024-09-10T08:08:49.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f949e665740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ba45ed5e60 con 0x55ba45ec3510 2024-09-10T08:08:49.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f949e665740 10 client.0 _ll_register_callbacks cb 0x7f948c09b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:49.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f9499200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8330 0x55ba45ede110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f9499200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8330 0x55ba45ede110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55562/0 (socket says 172.21.15.2:55562) 2024-09-10T08:08:49.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f9499200640 1 -- 172.21.15.2:0/706137637 learned_addr learned my addr 172.21.15.2:0/706137637 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:49.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f948be00640 1 --2- 172.21.15.2:0/706137637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8c50 0x55ba45ed7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f9499c00640 1 --2- 172.21.15.2:0/706137637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 0x55ba45ed7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f9499200640 1 -- 172.21.15.2:0/706137637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 msgr2=0x55ba45ed7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:49.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f9499200640 1 --2- 172.21.15.2:0/706137637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 0x55ba45ed7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f9499200640 1 -- 172.21.15.2:0/706137637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8c50 msgr2=0x55ba45ed7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:49.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f9499200640 1 --2- 172.21.15.2:0/706137637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8c50 0x55ba45ed7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.042+0000 7f9499200640 1 -- 172.21.15.2:0/706137637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba45ed7f40 con 0x55ba45eb8330 2024-09-10T08:08:49.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.043+0000 7f9499200640 1 --2- 172.21.15.2:0/706137637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8330 0x55ba45ede110 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9484009900 tx=0x7f9484004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b34c6ac5f19f80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:49.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.043+0000 7f9482a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:08:49.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.043+0000 7f9482a00640 1 -- 172.21.15.2:0/706137637 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484035040 con 0x55ba45eb8330 2024-09-10T08:08:49.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.043+0000 7f9482a00640 1 -- 172.21.15.2:0/706137637 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f94840317b0 con 0x55ba45eb8330 2024-09-10T08:08:49.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.043+0000 7f949e665740 1 -- 172.21.15.2:0/706137637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f947c0047a0 con 0x55ba45eb8330 2024-09-10T08:08:49.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.043+0000 7f9482a00640 1 -- 172.21.15.2:0/706137637 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f948405e050 con 0x55ba45eb8330 2024-09-10T08:08:49.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.043+0000 7f949e665740 10 client.15315 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:49.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.043+0000 7f949e665740 1 -- 172.21.15.2:0/706137637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba45edca20 con 0x55ba45eb8330 2024-09-10T08:08:49.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.044+0000 7f9482a00640 1 -- 172.21.15.2:0/706137637 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f947c0047a0 con 0x55ba45eb8330 2024-09-10T08:08:49.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.044+0000 7f9482a00640 1 -- 172.21.15.2:0/706137637 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9484003a00 con 0x55ba45eb8330 2024-09-10T08:08:49.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.044+0000 7f949e665740 10 client.15315 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:49.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.044+0000 7f949e665740 10 client.15315 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:08:49.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.044+0000 7f949e665740 20 client.15315 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:49.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.044+0000 7f949e665740 1 --2- 172.21.15.2:0/706137637 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55ba45f431a0 0x55ba45f41eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:49.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.044+0000 7f949e665740 4 client.15315 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:08:49.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.044+0000 7f949e665740 1 -- 172.21.15.2:0/706137637 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "status"}) -- 0x55ba45e25b30 con 0x55ba45f431a0 2024-09-10T08:08:49.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.044+0000 7f948be00640 1 --2- 172.21.15.2:0/706137637 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55ba45f431a0 0x55ba45f41eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:49.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f948be00640 1 --2- 172.21.15.2:0/706137637 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55ba45f431a0 0x55ba45f41eb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=140c6f11a86e65c6 server_cookie=dc6a727b2f5dbcc8 in_seq=0 out_seq=0 2024-09-10T08:08:49.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f9482a00640 10 client.15315 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:08:49.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f9482a00640 1 -- 172.21.15.2:0/706137637 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ba45e25b30 con 0x55ba45f431a0 2024-09-10T08:08:49.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f9482a00640 10 client.15315 handle_command_reply: tid=0 2024-09-10T08:08:49.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f949e665740 1 client.15315 shutdown 2024-09-10T08:08:49.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f949e665740 1 -- 172.21.15.2:0/706137637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8330 msgr2=0x55ba45ede110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:49.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f949e665740 1 --2- 172.21.15.2:0/706137637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8330 0x55ba45ede110 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9484009900 tx=0x7f9484004bd0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f949e665740 1 -- 172.21.15.2:0/706137637 shutdown_connections 2024-09-10T08:08:49.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f949e665740 1 --2- 172.21.15.2:0/706137637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba45ec3510 0x55ba45ed7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f949e665740 1 --2- 172.21.15.2:0/706137637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba45eb8c50 0x55ba45ed7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f949e665740 1 --2- 172.21.15.2:0/706137637 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55ba45f431a0 0x55ba45f41eb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f949e665740 1 --2- 172.21.15.2:0/706137637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba45eb8330 0x55ba45ede110 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f949e665740 1 -- 172.21.15.2:0/706137637 >> 172.21.15.2:0/706137637 conn(0x55ba45eb48c0 msgr2=0x55ba45ebb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:49.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.045+0000 7f9482a00640 0 client.15315 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:08:49.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.046+0000 7f949e665740 1 -- 172.21.15.2:0/706137637 shutdown_connections 2024-09-10T08:08:49.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.046+0000 7f949e665740 1 -- 172.21.15.2:0/706137637 wait complete. 2024-09-10T08:08:49.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.046+0000 7f949e665740 20 client.15315 trim_cache size 0 max 16384 2024-09-10T08:08:49.049 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:08:49.049 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "26ec64b0-6f4a-11ef-bcea-c7b262605968", 2024-09-10T08:08:49.049 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2024-09-10T08:08:49.049 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24494, 2024-09-10T08:08:49.049 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-09-10T08:08:49.050 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-09-10T08:08:49.050 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-09-10T08:08:49.050 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 148.26193595500001, 2024-09-10T08:08:49.050 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 26, 2024-09-10T08:08:49.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 95, 2024-09-10T08:08:49.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 95, 2024-09-10T08:08:49.050 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 228.97405863200001 2024-09-10T08:08:49.050 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:08:49.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.046+0000 7f949bc00640 1 -- 172.21.15.2:0/3139857829 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94780a2c10 msgr2=0x7f94780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:49.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.046+0000 7f949bc00640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94780a2c10 0x7f94780a50d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f947c00bfd0 tx=0x7f947c004730 comp rx=0 tx=0).stop 2024-09-10T08:08:49.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.046+0000 7f949bc00640 1 -- 172.21.15.2:0/3139857829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9494112a30 msgr2=0x7f949419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:49.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.046+0000 7f949bc00640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9494112a30 0x7f949419e8b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f948c009fd0 tx=0x7f948c00ee60 comp rx=0 tx=0).stop 2024-09-10T08:08:49.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.046+0000 7f949bc00640 1 -- 172.21.15.2:0/3139857829 shutdown_connections 2024-09-10T08:08:49.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.046+0000 7f949bc00640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9494112a30 0x7f949419e8b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.046+0000 7f949bc00640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9494077040 0x7f94941a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.046+0000 7f949bc00640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94780a2c10 0x7f94780a50d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.046+0000 7f949bc00640 1 --2- 172.21.15.2:0/3139857829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9494076720 0x7f94941a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:49.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.046+0000 7f949bc00640 1 -- 172.21.15.2:0/3139857829 >> 172.21.15.2:0/3139857829 conn(0x7f94940ffa90 msgr2=0x7f9494101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:49.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.047+0000 7f949bc00640 1 -- 172.21.15.2:0/3139857829 shutdown_connections 2024-09-10T08:08:49.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:49.047+0000 7f949bc00640 1 -- 172.21.15.2:0/3139857829 wait complete. 2024-09-10T08:08:49.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:49 smithi002 ceph-mon[30563]: pgmap v407: 129 pgs: 129 active+clean; 658 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-10T08:08:49.859 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T08:08:49.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:49 smithi155 ceph-mon[35405]: pgmap v407: 129 pgs: 129 active+clean; 658 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-10T08:08:49.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:49 smithi107 ceph-mon[35048]: pgmap v407: 129 pgs: 129 active+clean; 658 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-10T08:08:50.244 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.873+0000 7f5aeee00640 1 -- 172.21.15.2:0/1204734535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ae8074620 msgr2=0x7f5ae8074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.873+0000 7f5aeee00640 1 --2- 172.21.15.2:0/1204734535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ae8074620 0x7f5ae8074a00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009900 tx=0x7f5ad802a480 comp rx=0 tx=0).stop 2024-09-10T08:08:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.873+0000 7f5aeee00640 1 -- 172.21.15.2:0/1204734535 shutdown_connections 2024-09-10T08:08:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.873+0000 7f5aeee00640 1 --2- 172.21.15.2:0/1204734535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae810e3c0 0x7f5ae8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.873+0000 7f5aeee00640 1 --2- 172.21.15.2:0/1204734535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ae8074f40 0x7f5ae810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.873+0000 7f5aeee00640 1 --2- 172.21.15.2:0/1204734535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ae8074620 0x7f5ae8074a00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.873+0000 7f5aeee00640 1 -- 172.21.15.2:0/1204734535 >> 172.21.15.2:0/1204734535 conn(0x7f5ae80fd8c0 msgr2=0x7f5ae80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.873+0000 7f5aeee00640 1 -- 172.21.15.2:0/1204734535 shutdown_connections 2024-09-10T08:08:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.873+0000 7f5aeee00640 1 -- 172.21.15.2:0/1204734535 wait complete. 2024-09-10T08:08:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.874+0000 7f5aeee00640 1 Processor -- start 2024-09-10T08:08:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.874+0000 7f5aeee00640 1 -- start start 2024-09-10T08:08:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5aeee00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ae8074620 0x7f5ae819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5aeee00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ae8074f40 0x7f5ae819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5aeee00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae810e3c0 0x7f5ae819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5aeee00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae81141c0 con 0x7f5ae8074f40 2024-09-10T08:08:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5aeee00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae8114040 con 0x7f5ae8074620 2024-09-10T08:08:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5aeee00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae8114340 con 0x7f5ae810e3c0 2024-09-10T08:08:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5ae7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ae8074620 0x7f5ae819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5ae7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ae8074620 0x7f5ae819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:34964/0 (socket says 172.21.15.2:34964) 2024-09-10T08:08:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5ae7e00640 1 -- 172.21.15.2:0/1455372519 learned_addr learned my addr 172.21.15.2:0/1455372519 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5adfe00640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ae8074f40 0x7f5ae819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5aece00640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae810e3c0 0x7f5ae819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5ae7e00640 1 -- 172.21.15.2:0/1455372519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae810e3c0 msgr2=0x7f5ae819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5ae7e00640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae810e3c0 0x7f5ae819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5ae7e00640 1 -- 172.21.15.2:0/1455372519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ae8074f40 msgr2=0x7f5ae819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5ae7e00640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ae8074f40 0x7f5ae819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.875+0000 7f5ae7e00640 1 -- 172.21.15.2:0/1455372519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae819f630 con 0x7f5ae8074620 2024-09-10T08:08:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.876+0000 7f5ae7e00640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ae8074620 0x7f5ae819a680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8004770 tx=0x7f5ad8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.876+0000 7f5ae5600640 1 -- 172.21.15.2:0/1455372519 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad803f070 con 0x7f5ae8074620 2024-09-10T08:08:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.876+0000 7f5ae5600640 1 -- 172.21.15.2:0/1455372519 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ad8031ca0 con 0x7f5ae8074620 2024-09-10T08:08:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.876+0000 7f5aeee00640 1 -- 172.21.15.2:0/1455372519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae819f8c0 con 0x7f5ae8074620 2024-09-10T08:08:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.876+0000 7f5ae5600640 1 -- 172.21.15.2:0/1455372519 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8035030 con 0x7f5ae8074620 2024-09-10T08:08:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.876+0000 7f5aeee00640 1 -- 172.21.15.2:0/1455372519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae81a7200 con 0x7f5ae8074620 2024-09-10T08:08:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.879+0000 7f5af19f7740 1 Processor -- start 2024-09-10T08:08:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.879+0000 7f5af19f7740 1 -- start start 2024-09-10T08:08:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.879+0000 7f5ae5600640 1 -- 172.21.15.2:0/1455372519 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f5ad8042070 con 0x7f5ae8074620 2024-09-10T08:08:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.879+0000 7f5af19f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4a330 0x55f62ee4a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.879+0000 7f5af19f7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee4ac50 0x55f62ee54fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.879+0000 7f5af19f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee55510 0x55f62ee57900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.879+0000 7f5af19f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f62edb00b0 con 0x55f62ee55510 2024-09-10T08:08:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.879+0000 7f5af19f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f62ed57ab0 con 0x55f62ee4a330 2024-09-10T08:08:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.879+0000 7f5af19f7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f62edac070 con 0x55f62ee4ac50 2024-09-10T08:08:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.880+0000 7f5aece00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4a330 0x55f62ee4a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.880+0000 7f5ae7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee4ac50 0x55f62ee54fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.880+0000 7f5aece00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4a330 0x55f62ee4a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:34972/0 (socket says 172.21.15.2:34972) 2024-09-10T08:08:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.880+0000 7f5aece00640 1 -- 172.21.15.2:0/881314357 learned_addr learned my addr 172.21.15.2:0/881314357 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.880+0000 7f5adfe00640 1 --2- 172.21.15.2:0/881314357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee55510 0x55f62ee57900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.880+0000 7f5aece00640 1 -- 172.21.15.2:0/881314357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee4ac50 msgr2=0x55f62ee54fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.880+0000 7f5aece00640 1 --2- 172.21.15.2:0/881314357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee4ac50 0x55f62ee54fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.880+0000 7f5aece00640 1 -- 172.21.15.2:0/881314357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee55510 msgr2=0x55f62ee57900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.880+0000 7f5aece00640 1 --2- 172.21.15.2:0/881314357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee55510 0x55f62ee57900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.880+0000 7f5aece00640 1 -- 172.21.15.2:0/881314357 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f62ed644a0 con 0x55f62ee4a330 2024-09-10T08:08:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.880+0000 7f5aece00640 1 --2- 172.21.15.2:0/881314357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4a330 0x55f62ee4a710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009f50 tx=0x7f5ae00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1df3de3c736a59d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.881+0000 7f5add600640 1 -- 172.21.15.2:0/881314357 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0004d20 con 0x55f62ee4a330 2024-09-10T08:08:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.881+0000 7f5add600640 1 -- 172.21.15.2:0/881314357 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae0004050 con 0x55f62ee4a330 2024-09-10T08:08:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.881+0000 7f5add600640 1 -- 172.21.15.2:0/881314357 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0005280 con 0x55f62ee4a330 2024-09-10T08:08:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.881+0000 7f5ae5600640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5acc0a2c10 0x7f5acc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.881+0000 7f5ae5600640 1 -- 172.21.15.2:0/1455372519 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5ad80efc00 con 0x7f5ae8074620 2024-09-10T08:08:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.881+0000 7f5ae7e00640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5acc0a2c10 0x7f5acc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.881+0000 7f5af19f7740 1 -- 172.21.15.2:0/881314357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4a330 msgr2=0x55f62ee4a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.881+0000 7f5af19f7740 1 --2- 172.21.15.2:0/881314357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4a330 0x55f62ee4a710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009f50 tx=0x7f5ae00118d0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.882+0000 7f5af19f7740 1 -- 172.21.15.2:0/881314357 shutdown_connections 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.882+0000 7f5af19f7740 1 --2- 172.21.15.2:0/881314357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee55510 0x55f62ee57900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.882+0000 7f5af19f7740 1 --2- 172.21.15.2:0/881314357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee4ac50 0x55f62ee54fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.882+0000 7f5af19f7740 1 --2- 172.21.15.2:0/881314357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4a330 0x55f62ee4a710 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.882+0000 7f5af19f7740 1 -- 172.21.15.2:0/881314357 >> 172.21.15.2:0/881314357 conn(0x55f62ee468c0 msgr2=0x55f62ee46ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.882+0000 7f5af19f7740 1 -- 172.21.15.2:0/881314357 shutdown_connections 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.882+0000 7f5af19f7740 1 -- 172.21.15.2:0/881314357 wait complete. 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.882+0000 7f5ae7e00640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5acc0a2c10 0x7f5acc0a50d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80bbfb0 tx=0x7f5ad80f36b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.882+0000 7f5af19f7740 1 -- start start 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.883+0000 7f5af19f7740 1 Processor -- start 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.883+0000 7f5af19f7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee4a330 0x55f62ee52580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.883+0000 7f5af19f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4ac50 0x55f62ee52ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.883+0000 7f5af19f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee55510 0x55f62ee53000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.883+0000 7f5af19f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f62ee58060 con 0x55f62ee55510 2024-09-10T08:08:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.883+0000 7f5af19f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f62ed38920 con 0x55f62ee4ac50 2024-09-10T08:08:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.883+0000 7f5af19f7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f62ee581e0 con 0x55f62ee4a330 2024-09-10T08:08:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.883+0000 7f5af19f7740 10 client.0 _ll_register_callbacks cb 0x55f62ed5c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.883+0000 7f5aece00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee55510 0x55f62ee53000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.883+0000 7f5adfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4ac50 0x55f62ee52ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.883+0000 7f5aece00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee55510 0x55f62ee53000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55594/0 (socket says 172.21.15.2:55594) 2024-09-10T08:08:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.883+0000 7f5aece00640 1 -- 172.21.15.2:0/830767545 learned_addr learned my addr 172.21.15.2:0/830767545 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.884+0000 7f5aece00640 1 --2- 172.21.15.2:0/830767545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee4a330 0x55f62ee52580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.884+0000 7f5adfe00640 1 -- 172.21.15.2:0/830767545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee4a330 msgr2=0x55f62ee52580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.884+0000 7f5adfe00640 1 --2- 172.21.15.2:0/830767545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee4a330 0x55f62ee52580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.884+0000 7f5adfe00640 1 -- 172.21.15.2:0/830767545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee55510 msgr2=0x55f62ee53000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.884+0000 7f5adfe00640 1 --2- 172.21.15.2:0/830767545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee55510 0x55f62ee53000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.884+0000 7f5adfe00640 1 -- 172.21.15.2:0/830767545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f62eb212b0 con 0x55f62ee4ac50 2024-09-10T08:08:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.884+0000 7f5aece00640 1 --2- 172.21.15.2:0/830767545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee55510 0x55f62ee53000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.884+0000 7f5adfe00640 1 --2- 172.21.15.2:0/830767545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4ac50 0x55f62ee52ac0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5ae810f820 tx=0x7f5ad0012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f381354a81a252f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.884+0000 7f5ad6000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:08:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.884+0000 7f5ad6000640 1 -- 172.21.15.2:0/830767545 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad000f920 con 0x55f62ee4ac50 2024-09-10T08:08:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.884+0000 7f5ad6000640 1 -- 172.21.15.2:0/830767545 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5ad000fac0 con 0x55f62ee4ac50 2024-09-10T08:08:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.884+0000 7f5af19f7740 1 -- 172.21.15.2:0/830767545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ae0004050 con 0x55f62ee4ac50 2024-09-10T08:08:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.885+0000 7f5ad6000640 1 -- 172.21.15.2:0/830767545 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ad000fcc0 con 0x55f62ee4ac50 2024-09-10T08:08:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.885+0000 7f5af19f7740 10 client.25349 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.885+0000 7f5af19f7740 1 -- 172.21.15.2:0/830767545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f62ee63e70 con 0x55f62ee4ac50 2024-09-10T08:08:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.885+0000 7f5ad6000640 1 -- 172.21.15.2:0/830767545 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae0004050 con 0x55f62ee4ac50 2024-09-10T08:08:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.885+0000 7f5ad6000640 1 -- 172.21.15.2:0/830767545 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5ad001c420 con 0x55f62ee4ac50 2024-09-10T08:08:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.885+0000 7f5af19f7740 10 client.25349 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:50.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.885+0000 7f5af19f7740 10 client.25349 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:08:50.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.885+0000 7f5af19f7740 20 client.25349 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:50.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.885+0000 7f5af19f7740 1 --2- 172.21.15.2:0/830767545 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f62ee66510 0x55f62ee67980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.885+0000 7f5af19f7740 4 client.25349 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:08:50.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.885+0000 7f5af19f7740 1 -- 172.21.15.2:0/830767545 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f62edbd620 con 0x55f62ee66510 2024-09-10T08:08:50.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.886+0000 7f5aece00640 1 --2- 172.21.15.2:0/830767545 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f62ee66510 0x55f62ee67980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.886+0000 7f5aece00640 1 --2- 172.21.15.2:0/830767545 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f62ee66510 0x55f62ee67980 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c008bb55fa6a157 server_cookie=7d4f2165b6f8a244 in_seq=0 out_seq=0 2024-09-10T08:08:50.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.886+0000 7f5ad6000640 10 client.25349 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:08:50.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.887+0000 7f5ad6000640 1 -- 172.21.15.2:0/830767545 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f62edbd620 con 0x55f62ee66510 2024-09-10T08:08:50.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.887+0000 7f5ad6000640 10 client.25349 handle_command_reply: tid=0 2024-09-10T08:08:50.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.887+0000 7f5af19f7740 1 client.25349 shutdown 2024-09-10T08:08:50.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.887+0000 7f5af19f7740 1 -- 172.21.15.2:0/830767545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4ac50 msgr2=0x55f62ee52ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:50.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.887+0000 7f5af19f7740 1 --2- 172.21.15.2:0/830767545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4ac50 0x55f62ee52ac0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5ae810f820 tx=0x7f5ad0012b60 comp rx=0 tx=0).stop 2024-09-10T08:08:50.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.888+0000 7f5af19f7740 1 -- 172.21.15.2:0/830767545 shutdown_connections 2024-09-10T08:08:50.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.888+0000 7f5af19f7740 1 --2- 172.21.15.2:0/830767545 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f62ee66510 0x55f62ee67980 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.888+0000 7f5af19f7740 1 --2- 172.21.15.2:0/830767545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee55510 0x55f62ee53000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.888+0000 7f5af19f7740 1 --2- 172.21.15.2:0/830767545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4ac50 0x55f62ee52ac0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.888+0000 7f5ad6000640 0 client.25349 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:08:50.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.888+0000 7f5af19f7740 1 --2- 172.21.15.2:0/830767545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee4a330 0x55f62ee52580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.888+0000 7f5af19f7740 1 -- 172.21.15.2:0/830767545 >> 172.21.15.2:0/830767545 conn(0x55f62ee468c0 msgr2=0x55f62ee4c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:50.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.888+0000 7f5af19f7740 1 -- 172.21.15.2:0/830767545 shutdown_connections 2024-09-10T08:08:50.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.888+0000 7f5af19f7740 1 -- 172.21.15.2:0/830767545 wait complete. 2024-09-10T08:08:50.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.888+0000 7f5af19f7740 20 client.25349 trim_cache size 0 max 16384 2024-09-10T08:08:50.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.907+0000 7f5af19f7740 1 Processor -- start 2024-09-10T08:08:50.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.907+0000 7f5af19f7740 1 -- start start 2024-09-10T08:08:50.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.908+0000 7f5af19f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4a330 0x55f62ee50eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.908+0000 7f5af19f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4ac50 0x55f62ee513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.908+0000 7f5af19f7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee55510 0x55f62ee4f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.908+0000 7f5af19f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f62ee65cc0 con 0x55f62ee4a330 2024-09-10T08:08:50.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.908+0000 7f5af19f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f62ee66390 con 0x55f62ee4ac50 2024-09-10T08:08:50.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.908+0000 7f5af19f7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f62ee66a60 con 0x55f62ee55510 2024-09-10T08:08:50.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.908+0000 7f5aece00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4a330 0x55f62ee50eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.908+0000 7f5aece00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4a330 0x55f62ee50eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55598/0 (socket says 172.21.15.2:55598) 2024-09-10T08:08:50.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.908+0000 7f5aece00640 1 -- 172.21.15.2:0/4241859679 learned_addr learned my addr 172.21.15.2:0/4241859679 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:50.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.908+0000 7f5adfe00640 1 --2- 172.21.15.2:0/4241859679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4ac50 0x55f62ee513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.908+0000 7f5aece00640 1 --2- 172.21.15.2:0/4241859679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee55510 0x55f62ee4f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.908+0000 7f5aece00640 1 -- 172.21.15.2:0/4241859679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee55510 msgr2=0x55f62ee4f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:50.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.909+0000 7f5aece00640 1 --2- 172.21.15.2:0/4241859679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee55510 0x55f62ee4f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.909+0000 7f5aece00640 1 -- 172.21.15.2:0/4241859679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4ac50 msgr2=0x55f62ee513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:50.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.909+0000 7f5aece00640 1 --2- 172.21.15.2:0/4241859679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4ac50 0x55f62ee513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.909+0000 7f5aece00640 1 -- 172.21.15.2:0/4241859679 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f62ee4aa40 con 0x55f62ee4a330 2024-09-10T08:08:50.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.909+0000 7f5adfe00640 1 --2- 172.21.15.2:0/4241859679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4ac50 0x55f62ee513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:50.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.909+0000 7f5aece00640 1 --2- 172.21.15.2:0/4241859679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4a330 0x55f62ee50eb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0011dd0 tx=0x7f5ae0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbc0a2ac792439 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:50.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.909+0000 7f5adcc00640 1 -- 172.21.15.2:0/4241859679 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0003a30 con 0x55f62ee4a330 2024-09-10T08:08:50.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.909+0000 7f5adcc00640 1 -- 172.21.15.2:0/4241859679 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae0003bd0 con 0x55f62ee4a330 2024-09-10T08:08:50.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.909+0000 7f5adcc00640 1 -- 172.21.15.2:0/4241859679 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000f6e0 con 0x55f62ee4a330 2024-09-10T08:08:50.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.910+0000 7f5af19f7740 1 -- 172.21.15.2:0/4241859679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4a330 msgr2=0x55f62ee50eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:50.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.910+0000 7f5af19f7740 1 --2- 172.21.15.2:0/4241859679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4a330 0x55f62ee50eb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0011dd0 tx=0x7f5ae0009500 comp rx=0 tx=0).stop 2024-09-10T08:08:50.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.910+0000 7f5af19f7740 1 -- 172.21.15.2:0/4241859679 shutdown_connections 2024-09-10T08:08:50.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.910+0000 7f5af19f7740 1 --2- 172.21.15.2:0/4241859679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee55510 0x55f62ee4f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.910+0000 7f5af19f7740 1 --2- 172.21.15.2:0/4241859679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4ac50 0x55f62ee513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.910+0000 7f5af19f7740 1 --2- 172.21.15.2:0/4241859679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4a330 0x55f62ee50eb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.910+0000 7f5af19f7740 1 -- 172.21.15.2:0/4241859679 >> 172.21.15.2:0/4241859679 conn(0x55f62ee468c0 msgr2=0x55f62ee4d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:50.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.910+0000 7f5af19f7740 1 -- 172.21.15.2:0/4241859679 shutdown_connections 2024-09-10T08:08:50.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.910+0000 7f5af19f7740 1 -- 172.21.15.2:0/4241859679 wait complete. 2024-09-10T08:08:50.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.911+0000 7f5af19f7740 1 -- start start 2024-09-10T08:08:50.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.911+0000 7f5af19f7740 1 Processor -- start 2024-09-10T08:08:50.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.911+0000 7f5af19f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4a330 0x55f62ee704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.911+0000 7f5af19f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4ac50 0x55f62ee52540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.911+0000 7f5af19f7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee55510 0x55f62ee52b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.911+0000 7f5af19f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f62ee67800 con 0x55f62ee4ac50 2024-09-10T08:08:50.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.911+0000 7f5af19f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f62ee67130 con 0x55f62ee4a330 2024-09-10T08:08:50.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.911+0000 7f5af19f7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f62ee67ed0 con 0x55f62ee55510 2024-09-10T08:08:50.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.911+0000 7f5af19f7740 10 client.0 _ll_register_callbacks cb 0x7f5ad0009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:50.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.911+0000 7f5adfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4ac50 0x55f62ee52540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.911+0000 7f5adfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4ac50 0x55f62ee52540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55608/0 (socket says 172.21.15.2:55608) 2024-09-10T08:08:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.912+0000 7f5adfe00640 1 -- 172.21.15.2:0/2936104211 learned_addr learned my addr 172.21.15.2:0/2936104211 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.912+0000 7f5aece00640 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4a330 0x55f62ee704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.912+0000 7f5aece00640 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee55510 0x55f62ee52b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.912+0000 7f5adfe00640 1 -- 172.21.15.2:0/2936104211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee55510 msgr2=0x55f62ee52b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.912+0000 7f5adfe00640 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee55510 0x55f62ee52b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.912+0000 7f5adfe00640 1 -- 172.21.15.2:0/2936104211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4a330 msgr2=0x55f62ee704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.912+0000 7f5adfe00640 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4a330 0x55f62ee704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.912+0000 7f5adfe00640 1 -- 172.21.15.2:0/2936104211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f62ee6df40 con 0x55f62ee4ac50 2024-09-10T08:08:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.912+0000 7f5aece00640 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee55510 0x55f62ee52b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:08:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.912+0000 7f5adfe00640 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4ac50 0x55f62ee52540 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0000c00 tx=0x7f5ad0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d0d70e6a6be6fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:50.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.912+0000 7f5ad6000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:08:50.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.913+0000 7f5ad6000640 1 -- 172.21.15.2:0/2936104211 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad002f060 con 0x55f62ee4ac50 2024-09-10T08:08:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.913+0000 7f5ad6000640 1 -- 172.21.15.2:0/2936104211 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5ad001c630 con 0x55f62ee4ac50 2024-09-10T08:08:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.913+0000 7f5af19f7740 1 -- 172.21.15.2:0/2936104211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ae0003bd0 con 0x55f62ee4ac50 2024-09-10T08:08:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.913+0000 7f5ad6000640 1 -- 172.21.15.2:0/2936104211 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ad00050d0 con 0x55f62ee4ac50 2024-09-10T08:08:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.913+0000 7f5af19f7740 10 client.15333 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.913+0000 7f5af19f7740 1 -- 172.21.15.2:0/2936104211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f62eed0d30 con 0x55f62ee4ac50 2024-09-10T08:08:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.913+0000 7f5ad6000640 1 -- 172.21.15.2:0/2936104211 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae0003bd0 con 0x55f62ee4ac50 2024-09-10T08:08:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.914+0000 7f5ad6000640 1 -- 172.21.15.2:0/2936104211 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5ad0003ae0 con 0x55f62ee4ac50 2024-09-10T08:08:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.914+0000 7f5af19f7740 10 client.15333 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.914+0000 7f5af19f7740 10 client.15333 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:08:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.914+0000 7f5af19f7740 20 client.15333 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:50.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.914+0000 7f5af19f7740 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f62eed5940 0x55f62eed4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.914+0000 7f5af19f7740 4 client.15333 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:08:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.914+0000 7f5af19f7740 1 -- 172.21.15.2:0/2936104211 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "status"}) -- 0x55f62edb7b30 con 0x55f62eed5940 2024-09-10T08:08:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.914+0000 7f5aece00640 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f62eed5940 0x55f62eed4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5aece00640 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f62eed5940 0x55f62eed4650 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79bfdd965fdbd622 server_cookie=d2e830f5482f304b in_seq=0 out_seq=0 2024-09-10T08:08:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5ad6000640 10 client.15333 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:08:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5ad6000640 1 -- 172.21.15.2:0/2936104211 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55f62edb7b30 con 0x55f62eed5940 2024-09-10T08:08:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5ad6000640 10 client.15333 handle_command_reply: tid=0 2024-09-10T08:08:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5af19f7740 1 client.15333 shutdown 2024-09-10T08:08:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5af19f7740 1 -- 172.21.15.2:0/2936104211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4ac50 msgr2=0x55f62ee52540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5af19f7740 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4ac50 0x55f62ee52540 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0000c00 tx=0x7f5ad0005dc0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5af19f7740 1 -- 172.21.15.2:0/2936104211 shutdown_connections 2024-09-10T08:08:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5af19f7740 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f62eed5940 0x55f62eed4650 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5af19f7740 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f62ee55510 0x55f62ee52b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5af19f7740 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f62ee4ac50 0x55f62ee52540 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5af19f7740 1 --2- 172.21.15.2:0/2936104211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f62ee4a330 0x55f62ee704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5af19f7740 1 -- 172.21.15.2:0/2936104211 >> 172.21.15.2:0/2936104211 conn(0x55f62ee468c0 msgr2=0x55f62ee50ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.915+0000 7f5ad6000640 0 client.15333 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:08:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.916+0000 7f5af19f7740 1 -- 172.21.15.2:0/2936104211 shutdown_connections 2024-09-10T08:08:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.916+0000 7f5af19f7740 1 -- 172.21.15.2:0/2936104211 wait complete. 2024-09-10T08:08:50.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.916+0000 7f5af19f7740 20 client.15333 trim_cache size 0 max 16384 2024-09-10T08:08:50.919 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:08:50.919 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "26ec64b0-6f4a-11ef-bcea-c7b262605968", 2024-09-10T08:08:50.919 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2024-09-10T08:08:50.920 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24499, 2024-09-10T08:08:50.920 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-09-10T08:08:50.920 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-09-10T08:08:50.920 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-09-10T08:08:50.920 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 151.13502208, 2024-09-10T08:08:50.920 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 26, 2024-09-10T08:08:50.920 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 95, 2024-09-10T08:08:50.920 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 95, 2024-09-10T08:08:50.920 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 228.21158447799999 2024-09-10T08:08:50.920 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:08:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.916+0000 7f5aeee00640 1 -- 172.21.15.2:0/1455372519 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5acc0a2c10 msgr2=0x7f5acc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.916+0000 7f5aeee00640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5acc0a2c10 0x7f5acc0a50d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80bbfb0 tx=0x7f5ad80f36b0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.916+0000 7f5aeee00640 1 -- 172.21.15.2:0/1455372519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ae8074620 msgr2=0x7f5ae819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.916+0000 7f5aeee00640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ae8074620 0x7f5ae819a680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8004770 tx=0x7f5ad8003c10 comp rx=0 tx=0).stop 2024-09-10T08:08:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.916+0000 7f5aeee00640 1 -- 172.21.15.2:0/1455372519 shutdown_connections 2024-09-10T08:08:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.916+0000 7f5aeee00640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5acc0a2c10 0x7f5acc0a50d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.916+0000 7f5aeee00640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae810e3c0 0x7f5ae819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.916+0000 7f5aeee00640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ae8074f40 0x7f5ae819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.916+0000 7f5aeee00640 1 --2- 172.21.15.2:0/1455372519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ae8074620 0x7f5ae819a680 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.916+0000 7f5aeee00640 1 -- 172.21.15.2:0/1455372519 >> 172.21.15.2:0/1455372519 conn(0x7f5ae80fd8c0 msgr2=0x7f5ae80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.917+0000 7f5aeee00640 1 -- 172.21.15.2:0/1455372519 shutdown_connections 2024-09-10T08:08:50.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:50.917+0000 7f5aeee00640 1 -- 172.21.15.2:0/1455372519 wait complete. 2024-09-10T08:08:51.636 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:51 smithi002 ceph-mon[30563]: pgmap v408: 129 pgs: 129 active+clean; 658 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T08:08:51.672 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T08:08:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:51 smithi155 ceph-mon[35405]: pgmap v408: 129 pgs: 129 active+clean; 658 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T08:08:51.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:51 smithi107 ceph-mon[35048]: pgmap v408: 129 pgs: 129 active+clean; 658 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T08:08:52.056 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:08:52.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.656+0000 7ff5d8200640 1 -- 172.21.15.2:0/3758247168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0103ff0 msgr2=0x7ff5d01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:52.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.656+0000 7ff5d8200640 1 --2- 172.21.15.2:0/3758247168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0103ff0 0x7ff5d01043d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009900 tx=0x7ff5c402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.656+0000 7ff5d8200640 1 -- 172.21.15.2:0/3758247168 shutdown_connections 2024-09-10T08:08:52.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.656+0000 7ff5d8200640 1 --2- 172.21.15.2:0/3758247168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5d0109120 0x7ff5d0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.656+0000 7ff5d8200640 1 --2- 172.21.15.2:0/3758247168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5d01049a0 0x7ff5d01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.656+0000 7ff5d8200640 1 --2- 172.21.15.2:0/3758247168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0103ff0 0x7ff5d01043d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.656+0000 7ff5d8200640 1 -- 172.21.15.2:0/3758247168 >> 172.21.15.2:0/3758247168 conn(0x7ff5d00ffa90 msgr2=0x7ff5d0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:52.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.656+0000 7ff5d8200640 1 -- 172.21.15.2:0/3758247168 shutdown_connections 2024-09-10T08:08:52.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.656+0000 7ff5d8200640 1 -- 172.21.15.2:0/3758247168 wait complete. 2024-09-10T08:08:52.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.657+0000 7ff5d8200640 1 Processor -- start 2024-09-10T08:08:52.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.657+0000 7ff5d8200640 1 -- start start 2024-09-10T08:08:52.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d8200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5d0103ff0 0x7ff5d019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d8200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5d01049a0 0x7ff5d019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d8200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0109120 0x7ff5d019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d8200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0114170 con 0x7ff5d0103ff0 2024-09-10T08:08:52.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d8200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0113ff0 con 0x7ff5d0109120 2024-09-10T08:08:52.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d8200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01142f0 con 0x7ff5d01049a0 2024-09-10T08:08:52.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d5800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5d0103ff0 0x7ff5d019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d5800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5d0103ff0 0x7ff5d019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55626/0 (socket says 172.21.15.2:55626) 2024-09-10T08:08:52.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d5800640 1 -- 172.21.15.2:0/376939256 learned_addr learned my addr 172.21.15.2:0/376939256 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:52.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d4e00640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5d01049a0 0x7ff5d019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d6200640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0109120 0x7ff5d019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d5800640 1 -- 172.21.15.2:0/376939256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5d01049a0 msgr2=0x7ff5d019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:52.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d5800640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5d01049a0 0x7ff5d019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d5800640 1 -- 172.21.15.2:0/376939256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0109120 msgr2=0x7ff5d019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:52.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d5800640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0109120 0x7ff5d019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d5800640 1 -- 172.21.15.2:0/376939256 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d019f5c0 con 0x7ff5d0103ff0 2024-09-10T08:08:52.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.658+0000 7ff5d6200640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0109120 0x7ff5d019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:08:52.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.659+0000 7ff5d5800640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5d0103ff0 0x7ff5d019a610 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4004770 tx=0x7ff5c4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:52.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.659+0000 7ff5ce000640 1 -- 172.21.15.2:0/376939256 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c403f070 con 0x7ff5d0103ff0 2024-09-10T08:08:52.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.659+0000 7ff5ce000640 1 -- 172.21.15.2:0/376939256 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c4031ca0 con 0x7ff5d0103ff0 2024-09-10T08:08:52.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.659+0000 7ff5d8200640 1 -- 172.21.15.2:0/376939256 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d019f850 con 0x7ff5d0103ff0 2024-09-10T08:08:52.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.659+0000 7ff5ce000640 1 -- 172.21.15.2:0/376939256 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4035030 con 0x7ff5d0103ff0 2024-09-10T08:08:52.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.659+0000 7ff5d8200640 1 -- 172.21.15.2:0/376939256 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d01a7190 con 0x7ff5d0103ff0 2024-09-10T08:08:52.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.662+0000 7ff5dabbc740 1 Processor -- start 2024-09-10T08:08:52.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.662+0000 7ff5ce000640 1 -- 172.21.15.2:0/376939256 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7ff5c4042070 con 0x7ff5d0103ff0 2024-09-10T08:08:52.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.662+0000 7ff5dabbc740 1 -- start start 2024-09-10T08:08:52.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.663+0000 7ff5dabbc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e46710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.663+0000 7ff5dabbc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e50fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.663+0000 7ff5dabbc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e53900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.663+0000 7ff5dabbc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d9a2dac0b0 con 0x55d9a2e51510 2024-09-10T08:08:52.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.663+0000 7ff5dabbc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d9a2d53ab0 con 0x55d9a2e46330 2024-09-10T08:08:52.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.663+0000 7ff5dabbc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d9a2da8070 con 0x55d9a2e46c50 2024-09-10T08:08:52.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.663+0000 7ff5d4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e53900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.663+0000 7ff5d5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e50fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.663+0000 7ff5d4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e53900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55642/0 (socket says 172.21.15.2:55642) 2024-09-10T08:08:52.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.663+0000 7ff5d4e00640 1 -- 172.21.15.2:0/1529783782 learned_addr learned my addr 172.21.15.2:0/1529783782 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:52.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.663+0000 7ff5d6200640 1 --2- 172.21.15.2:0/1529783782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e46710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.664+0000 7ff5d6200640 1 -- 172.21.15.2:0/1529783782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 msgr2=0x55d9a2e50fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:52.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.664+0000 7ff5d6200640 1 --2- 172.21.15.2:0/1529783782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e50fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.664+0000 7ff5d6200640 1 -- 172.21.15.2:0/1529783782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 msgr2=0x55d9a2e53900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:52.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.664+0000 7ff5d6200640 1 --2- 172.21.15.2:0/1529783782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e53900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.664+0000 7ff5d6200640 1 -- 172.21.15.2:0/1529783782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a2d604a0 con 0x55d9a2e46330 2024-09-10T08:08:52.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.664+0000 7ff5d4e00640 1 --2- 172.21.15.2:0/1529783782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e53900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:52.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.664+0000 7ff5ce000640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5b80a2c10 0x7ff5b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.664+0000 7ff5d6200640 1 --2- 172.21.15.2:0/1529783782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e46710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009f10 tx=0x7ff5c80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e368c9db5b9925b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:52.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.664+0000 7ff5ce000640 1 -- 172.21.15.2:0/376939256 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff5c40efd70 con 0x7ff5d0103ff0 2024-09-10T08:08:52.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.664+0000 7ff5d5800640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5b80a2c10 0x7ff5b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.664+0000 7ff5c2a00640 1 -- 172.21.15.2:0/1529783782 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c801c680 con 0x55d9a2e46330 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.664+0000 7ff5c2a00640 1 -- 172.21.15.2:0/1529783782 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c801cd00 con 0x55d9a2e46330 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.664+0000 7ff5c2a00640 1 -- 172.21.15.2:0/1529783782 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c801a380 con 0x55d9a2e46330 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.665+0000 7ff5dabbc740 1 -- 172.21.15.2:0/1529783782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 msgr2=0x55d9a2e46710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.665+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/1529783782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e46710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009f10 tx=0x7ff5c80107a0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.665+0000 7ff5d5800640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5b80a2c10 0x7ff5b80a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40b8480 tx=0x7ff5c40f3820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.665+0000 7ff5dabbc740 1 -- 172.21.15.2:0/1529783782 shutdown_connections 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.665+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/1529783782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e53900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.665+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/1529783782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e50fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.665+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/1529783782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e46710 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.665+0000 7ff5dabbc740 1 -- 172.21.15.2:0/1529783782 >> 172.21.15.2:0/1529783782 conn(0x55d9a2e428c0 msgr2=0x55d9a2e42ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.665+0000 7ff5dabbc740 1 -- 172.21.15.2:0/1529783782 shutdown_connections 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.665+0000 7ff5dabbc740 1 -- 172.21.15.2:0/1529783782 wait complete. 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.666+0000 7ff5dabbc740 1 -- start start 2024-09-10T08:08:52.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.666+0000 7ff5dabbc740 1 Processor -- start 2024-09-10T08:08:52.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5dabbc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e4e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5dabbc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e46c50 0x55d9a2e4ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5dabbc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e51510 0x55d9a2e4f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5dabbc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d9a2e54060 con 0x55d9a2e46c50 2024-09-10T08:08:52.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5dabbc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d9a2d34920 con 0x55d9a2e46330 2024-09-10T08:08:52.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5dabbc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d9a2e541e0 con 0x55d9a2e51510 2024-09-10T08:08:52.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5dabbc740 10 client.0 _ll_register_callbacks cb 0x55d9a2d588a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:52.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5d4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e46c50 0x55d9a2e4ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5d4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e46c50 0x55d9a2e4ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55650/0 (socket says 172.21.15.2:55650) 2024-09-10T08:08:52.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5d4e00640 1 -- 172.21.15.2:0/4242645639 learned_addr learned my addr 172.21.15.2:0/4242645639 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:52.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5d6200640 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e4e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5d6200640 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e51510 0x55d9a2e4f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5d4e00640 1 -- 172.21.15.2:0/4242645639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e51510 msgr2=0x55d9a2e4f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:52.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5d4e00640 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e51510 0x55d9a2e4f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5d4e00640 1 -- 172.21.15.2:0/4242645639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 msgr2=0x55d9a2e4e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:52.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5d4e00640 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e4e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5d4e00640 1 -- 172.21.15.2:0/4242645639 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a2b1d2b0 con 0x55d9a2e46c50 2024-09-10T08:08:52.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.667+0000 7ff5d6200640 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e51510 0x55d9a2e4f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:08:52.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.668+0000 7ff5d4e00640 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e46c50 0x55d9a2e4ebb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00bf60 tx=0x7ff5bc00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3b9c47aa36709d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:52.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.668+0000 7ff5b7400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:08:52.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.668+0000 7ff5b7400640 1 -- 172.21.15.2:0/4242645639 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc01f060 con 0x55d9a2e46c50 2024-09-10T08:08:52.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.668+0000 7ff5b7400640 1 -- 172.21.15.2:0/4242645639 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff5bc010700 con 0x55d9a2e46c50 2024-09-10T08:08:52.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.668+0000 7ff5dabbc740 1 -- 172.21.15.2:0/4242645639 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c801cd00 con 0x55d9a2e46c50 2024-09-10T08:08:52.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.668+0000 7ff5b7400640 1 -- 172.21.15.2:0/4242645639 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff5bc026050 con 0x55d9a2e46c50 2024-09-10T08:08:52.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.668+0000 7ff5dabbc740 10 client.15345 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:52.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.668+0000 7ff5dabbc740 1 -- 172.21.15.2:0/4242645639 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9a2e5fed0 con 0x55d9a2e46c50 2024-09-10T08:08:52.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.669+0000 7ff5b7400640 1 -- 172.21.15.2:0/4242645639 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c801cd00 con 0x55d9a2e46c50 2024-09-10T08:08:52.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.669+0000 7ff5b7400640 1 -- 172.21.15.2:0/4242645639 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff5bc00fb60 con 0x55d9a2e46c50 2024-09-10T08:08:52.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.669+0000 7ff5dabbc740 10 client.15345 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:52.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.669+0000 7ff5dabbc740 10 client.15345 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:08:52.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.669+0000 7ff5dabbc740 20 client.15345 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:52.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.669+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d9a2e624d0 0x55d9a2e63940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.669+0000 7ff5dabbc740 4 client.15345 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:08:52.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.669+0000 7ff5dabbc740 1 -- 172.21.15.2:0/4242645639 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9a2db9620 con 0x55d9a2e624d0 2024-09-10T08:08:52.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.670+0000 7ff5d6200640 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d9a2e624d0 0x55d9a2e63940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.670+0000 7ff5d6200640 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d9a2e624d0 0x55d9a2e63940 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd8d0c1717f8a6fb server_cookie=91cc8da0d19cb4a3 in_seq=0 out_seq=0 2024-09-10T08:08:52.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.670+0000 7ff5b7400640 10 client.15345 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:52.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5b7400640 1 -- 172.21.15.2:0/4242645639 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9a2db9620 con 0x55d9a2e624d0 2024-09-10T08:08:52.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5b7400640 10 client.15345 handle_command_reply: tid=0 2024-09-10T08:08:52.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5dabbc740 1 client.15345 shutdown 2024-09-10T08:08:52.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5dabbc740 1 -- 172.21.15.2:0/4242645639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e46c50 msgr2=0x55d9a2e4ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:52.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e46c50 0x55d9a2e4ebb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00bf60 tx=0x7ff5bc00bf90 comp rx=0 tx=0).stop 2024-09-10T08:08:52.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5dabbc740 1 -- 172.21.15.2:0/4242645639 shutdown_connections 2024-09-10T08:08:52.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d9a2e624d0 0x55d9a2e63940 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e51510 0x55d9a2e4f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e46c50 0x55d9a2e4ebb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/4242645639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e4e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5b7400640 0 client.15345 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:52.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5dabbc740 1 -- 172.21.15.2:0/4242645639 >> 172.21.15.2:0/4242645639 conn(0x55d9a2e428c0 msgr2=0x55d9a2e485a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:52.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5dabbc740 1 -- 172.21.15.2:0/4242645639 shutdown_connections 2024-09-10T08:08:52.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.671+0000 7ff5dabbc740 1 -- 172.21.15.2:0/4242645639 wait complete. 2024-09-10T08:08:52.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.672+0000 7ff5dabbc740 20 client.15345 trim_cache size 0 max 16384 2024-09-10T08:08:52.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.690+0000 7ff5dabbc740 1 Processor -- start 2024-09-10T08:08:52.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.691+0000 7ff5dabbc740 1 -- start start 2024-09-10T08:08:52.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.691+0000 7ff5dabbc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e4ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.691+0000 7ff5dabbc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e4b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.691+0000 7ff5dabbc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e4be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.691+0000 7ff5dabbc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d9a2e61c80 con 0x55d9a2e51510 2024-09-10T08:08:52.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.691+0000 7ff5dabbc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d9a2e62350 con 0x55d9a2e46330 2024-09-10T08:08:52.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.691+0000 7ff5dabbc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d9a2e62a20 con 0x55d9a2e46c50 2024-09-10T08:08:52.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.691+0000 7ff5d6200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e4be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.691+0000 7ff5d6200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e4be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55656/0 (socket says 172.21.15.2:55656) 2024-09-10T08:08:52.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.691+0000 7ff5d6200640 1 -- 172.21.15.2:0/3708082301 learned_addr learned my addr 172.21.15.2:0/3708082301 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:52.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.691+0000 7ff5d6200640 1 --2- 172.21.15.2:0/3708082301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e4ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.691+0000 7ff5d4e00640 1 --2- 172.21.15.2:0/3708082301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e4b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.692+0000 7ff5d6200640 1 -- 172.21.15.2:0/3708082301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 msgr2=0x55d9a2e4b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:52.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.692+0000 7ff5d6200640 1 --2- 172.21.15.2:0/3708082301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e4b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.692+0000 7ff5d6200640 1 -- 172.21.15.2:0/3708082301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 msgr2=0x55d9a2e4ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:52.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.692+0000 7ff5d6200640 1 --2- 172.21.15.2:0/3708082301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e4ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.692+0000 7ff5d6200640 1 -- 172.21.15.2:0/3708082301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a2e4f480 con 0x55d9a2e51510 2024-09-10T08:08:52.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.692+0000 7ff5d6200640 1 --2- 172.21.15.2:0/3708082301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e4be70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8010ca0 tx=0x7ff5c8011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e358991b0df8ce24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:52.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.692+0000 7ff5c2000640 1 -- 172.21.15.2:0/3708082301 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80049c0 con 0x55d9a2e51510 2024-09-10T08:08:52.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.692+0000 7ff5c2000640 1 -- 172.21.15.2:0/3708082301 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c8004b60 con 0x55d9a2e51510 2024-09-10T08:08:52.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.692+0000 7ff5c2000640 1 -- 172.21.15.2:0/3708082301 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c802c350 con 0x55d9a2e51510 2024-09-10T08:08:52.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.693+0000 7ff5dabbc740 1 -- 172.21.15.2:0/3708082301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 msgr2=0x55d9a2e4be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:52.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.693+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/3708082301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e4be70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8010ca0 tx=0x7ff5c8011bb0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.693+0000 7ff5dabbc740 1 -- 172.21.15.2:0/3708082301 shutdown_connections 2024-09-10T08:08:52.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.693+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/3708082301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e4be70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.693+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/3708082301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e4b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.693+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/3708082301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e4ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.693+0000 7ff5dabbc740 1 -- 172.21.15.2:0/3708082301 >> 172.21.15.2:0/3708082301 conn(0x55d9a2e428c0 msgr2=0x55d9a2e499d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:52.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.693+0000 7ff5dabbc740 1 -- 172.21.15.2:0/3708082301 shutdown_connections 2024-09-10T08:08:52.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.693+0000 7ff5dabbc740 1 -- 172.21.15.2:0/3708082301 wait complete. 2024-09-10T08:08:52.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.693+0000 7ff5dabbc740 1 -- start start 2024-09-10T08:08:52.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.694+0000 7ff5dabbc740 1 Processor -- start 2024-09-10T08:08:52.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.694+0000 7ff5dabbc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e6bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.694+0000 7ff5dabbc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e6c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.694+0000 7ff5dabbc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e66c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.694+0000 7ff5dabbc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d9a2e637c0 con 0x55d9a2e51510 2024-09-10T08:08:52.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.694+0000 7ff5dabbc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d9a2e630f0 con 0x55d9a2e46330 2024-09-10T08:08:52.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.694+0000 7ff5dabbc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d9a2e63e90 con 0x55d9a2e46c50 2024-09-10T08:08:52.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.694+0000 7ff5dabbc740 10 client.0 _ll_register_callbacks cb 0x7ff5bc00bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:52.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.694+0000 7ff5d6200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e66c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.695+0000 7ff5d4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e6c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.695+0000 7ff5d6200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e66c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55672/0 (socket says 172.21.15.2:55672) 2024-09-10T08:08:52.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.695+0000 7ff5d4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e6c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:41336/0 (socket says 172.21.15.2:41336) 2024-09-10T08:08:52.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.695+0000 7ff5d6200640 1 -- 172.21.15.2:0/3137788517 learned_addr learned my addr 172.21.15.2:0/3137788517 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:52.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.695+0000 7ff5d6200640 1 --2- 172.21.15.2:0/3137788517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e6bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.695+0000 7ff5d4e00640 1 -- 172.21.15.2:0/3137788517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 msgr2=0x55d9a2e6bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.695+0000 7ff5d4e00640 1 --2- 172.21.15.2:0/3137788517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e6bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.695+0000 7ff5d4e00640 1 -- 172.21.15.2:0/3137788517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 msgr2=0x55d9a2e66c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.695+0000 7ff5d4e00640 1 --2- 172.21.15.2:0/3137788517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e66c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.695+0000 7ff5d4e00640 1 -- 172.21.15.2:0/3137788517 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a2e67490 con 0x55d9a2e46c50 2024-09-10T08:08:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.695+0000 7ff5d4e00640 1 --2- 172.21.15.2:0/3137788517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e6c450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00ff30 tx=0x7ff5bc005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bdf89d8922958fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.695+0000 7ff5b7400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:08:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.696+0000 7ff5b7400640 1 -- 172.21.15.2:0/3137788517 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc00b480 con 0x55d9a2e46c50 2024-09-10T08:08:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.696+0000 7ff5dabbc740 1 -- 172.21.15.2:0/3137788517 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c8004b60 con 0x55d9a2e46c50 2024-09-10T08:08:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.696+0000 7ff5b7400640 1 -- 172.21.15.2:0/3137788517 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff5bc010700 con 0x55d9a2e46c50 2024-09-10T08:08:52.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.696+0000 7ff5b7400640 1 -- 172.21.15.2:0/3137788517 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff5bc038050 con 0x55d9a2e46c50 2024-09-10T08:08:52.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.696+0000 7ff5dabbc740 10 client.25348 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:52.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.696+0000 7ff5dabbc740 1 -- 172.21.15.2:0/3137788517 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9a2ecd7e0 con 0x55d9a2e46c50 2024-09-10T08:08:52.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.696+0000 7ff5b7400640 1 -- 172.21.15.2:0/3137788517 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c8004b60 con 0x55d9a2e46c50 2024-09-10T08:08:52.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.697+0000 7ff5b7400640 1 -- 172.21.15.2:0/3137788517 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff5bc0049f0 con 0x55d9a2e46c50 2024-09-10T08:08:52.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.697+0000 7ff5dabbc740 10 client.25348 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:52.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.697+0000 7ff5dabbc740 10 client.25348 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:08:52.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.697+0000 7ff5dabbc740 20 client.25348 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:52.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.697+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/3137788517 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d9a2ed2430 0x55d9a2ed1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:52.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.697+0000 7ff5dabbc740 4 client.25348 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:08:52.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.697+0000 7ff5dabbc740 1 -- 172.21.15.2:0/3137788517 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "status"}) -- 0x55d9a2db3b30 con 0x55d9a2ed2430 2024-09-10T08:08:52.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.697+0000 7ff5d6200640 1 --2- 172.21.15.2:0/3137788517 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d9a2ed2430 0x55d9a2ed1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:52.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5d6200640 1 --2- 172.21.15.2:0/3137788517 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d9a2ed2430 0x55d9a2ed1140 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e39600018b608084 server_cookie=a450b7fc4d7dd894 in_seq=0 out_seq=0 2024-09-10T08:08:52.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5b7400640 10 client.25348 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:52.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5b7400640 1 -- 172.21.15.2:0/3137788517 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d9a2db3b30 con 0x55d9a2ed2430 2024-09-10T08:08:52.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5b7400640 10 client.25348 handle_command_reply: tid=0 2024-09-10T08:08:52.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5dabbc740 1 client.25348 shutdown 2024-09-10T08:08:52.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5dabbc740 1 -- 172.21.15.2:0/3137788517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 msgr2=0x55d9a2e6c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:52.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/3137788517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e6c450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00ff30 tx=0x7ff5bc005270 comp rx=0 tx=0).stop 2024-09-10T08:08:52.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5dabbc740 1 -- 172.21.15.2:0/3137788517 shutdown_connections 2024-09-10T08:08:52.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/3137788517 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d9a2ed2430 0x55d9a2ed1140 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/3137788517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9a2e51510 0x55d9a2e66c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/3137788517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9a2e46c50 0x55d9a2e6c450 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5b7400640 0 client.25348 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:52.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5dabbc740 1 --2- 172.21.15.2:0/3137788517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9a2e46330 0x55d9a2e6bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.698+0000 7ff5dabbc740 1 -- 172.21.15.2:0/3137788517 >> 172.21.15.2:0/3137788517 conn(0x55d9a2e428c0 msgr2=0x55d9a2e47d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:52.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.699+0000 7ff5dabbc740 1 -- 172.21.15.2:0/3137788517 shutdown_connections 2024-09-10T08:08:52.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.699+0000 7ff5dabbc740 1 -- 172.21.15.2:0/3137788517 wait complete. 2024-09-10T08:08:52.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.699+0000 7ff5dabbc740 20 client.25348 trim_cache size 0 max 16384 2024-09-10T08:08:52.703 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:08:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "26ec64b0-6f4a-11ef-bcea-c7b262605968", 2024-09-10T08:08:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-09-10T08:08:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24536, 2024-09-10T08:08:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-09-10T08:08:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-09-10T08:08:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-09-10T08:08:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 205.210085975, 2024-09-10T08:08:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 26, 2024-09-10T08:08:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 95, 2024-09-10T08:08:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 95, 2024-09-10T08:08:52.704 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 219.08865496799999 2024-09-10T08:08:52.704 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:08:52.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.699+0000 7ff5d8200640 1 -- 172.21.15.2:0/376939256 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5b80a2c10 msgr2=0x7ff5b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:52.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.699+0000 7ff5d8200640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5b80a2c10 0x7ff5b80a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40b8480 tx=0x7ff5c40f3820 comp rx=0 tx=0).stop 2024-09-10T08:08:52.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.699+0000 7ff5d8200640 1 -- 172.21.15.2:0/376939256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5d0103ff0 msgr2=0x7ff5d019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:52.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.700+0000 7ff5d8200640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5d0103ff0 0x7ff5d019a610 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4004770 tx=0x7ff5c4003c10 comp rx=0 tx=0).stop 2024-09-10T08:08:52.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.700+0000 7ff5d8200640 1 -- 172.21.15.2:0/376939256 shutdown_connections 2024-09-10T08:08:52.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.700+0000 7ff5d8200640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0109120 0x7ff5d019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.700+0000 7ff5d8200640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5b80a2c10 0x7ff5b80a50d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.700+0000 7ff5d8200640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5d01049a0 0x7ff5d019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.700+0000 7ff5d8200640 1 --2- 172.21.15.2:0/376939256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5d0103ff0 0x7ff5d019a610 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:52.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.700+0000 7ff5d8200640 1 -- 172.21.15.2:0/376939256 >> 172.21.15.2:0/376939256 conn(0x7ff5d00ffa90 msgr2=0x7ff5d0100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:52.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.700+0000 7ff5d8200640 1 -- 172.21.15.2:0/376939256 shutdown_connections 2024-09-10T08:08:52.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:08:52.700+0000 7ff5d8200640 1 -- 172.21.15.2:0/376939256 wait complete. 2024-09-10T08:08:53.493 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-10T08:08:53.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:53 smithi002 ceph-mon[30563]: pgmap v409: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-10T08:08:53.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:53 smithi155 ceph-mon[35405]: pgmap v409: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-10T08:08:53.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:53 smithi107 ceph-mon[35048]: pgmap v409: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-10T08:08:55.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:55 smithi002 ceph-mon[30563]: pgmap v410: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-10T08:08:55.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:55 smithi155 ceph-mon[35405]: pgmap v410: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-10T08:08:55.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:55 smithi107 ceph-mon[35048]: pgmap v410: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-10T08:08:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.224+0000 7f1ffa200640 1 -- 172.21.15.2:0/61425147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff40761d0 msgr2=0x7f1ff4074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.224+0000 7f1ffa200640 1 --2- 172.21.15.2:0/61425147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff40761d0 0x7f1ff4074620 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009900 tx=0x7f1fe402a480 comp rx=0 tx=0).stop 2024-09-10T08:08:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.224+0000 7f1ffa200640 1 -- 172.21.15.2:0/61425147 shutdown_connections 2024-09-10T08:08:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.224+0000 7f1ffa200640 1 --2- 172.21.15.2:0/61425147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ff41099f0 0x7f1ff410beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.224+0000 7f1ffa200640 1 --2- 172.21.15.2:0/61425147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ff4074bf0 0x7f1ff4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.224+0000 7f1ffa200640 1 --2- 172.21.15.2:0/61425147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff40761d0 0x7f1ff4074620 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.224+0000 7f1ffa200640 1 -- 172.21.15.2:0/61425147 >> 172.21.15.2:0/61425147 conn(0x7f1ff40fd2e0 msgr2=0x7f1ff40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.224+0000 7f1ffa200640 1 -- 172.21.15.2:0/61425147 shutdown_connections 2024-09-10T08:08:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.224+0000 7f1ffa200640 1 -- 172.21.15.2:0/61425147 wait complete. 2024-09-10T08:08:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.225+0000 7f1ffa200640 1 Processor -- start 2024-09-10T08:08:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.225+0000 7f1ffa200640 1 -- start start 2024-09-10T08:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.225+0000 7f1ffa200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ff4074bf0 0x7f1ff4104ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.225+0000 7f1ffa200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ff40761d0 0x7f1ff4105420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.225+0000 7f1ffa200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff41099f0 0x7f1ff4101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.225+0000 7f1ffa200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410f9f0 con 0x7f1ff41099f0 2024-09-10T08:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.225+0000 7f1ffa200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410f870 con 0x7f1ff4074bf0 2024-09-10T08:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.225+0000 7f1ffa200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410fb70 con 0x7f1ff40761d0 2024-09-10T08:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.225+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff41099f0 0x7f1ff4101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.226+0000 7f1ff3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ff4074bf0 0x7f1ff4104ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.226+0000 7f1ff3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ff4074bf0 0x7f1ff4104ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35086/0 (socket says 172.21.15.2:35086) 2024-09-10T08:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.226+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff41099f0 0x7f1ff4101a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55688/0 (socket says 172.21.15.2:55688) 2024-09-10T08:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.226+0000 7f1ff3400640 1 -- 172.21.15.2:0/3127123494 learned_addr learned my addr 172.21.15.2:0/3127123494 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.226+0000 7f1ff2a00640 1 --2- 172.21.15.2:0/3127123494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ff40761d0 0x7f1ff4105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.226+0000 7f1ff3400640 1 -- 172.21.15.2:0/3127123494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ff40761d0 msgr2=0x7f1ff4105420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.226+0000 7f1ff3400640 1 --2- 172.21.15.2:0/3127123494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ff40761d0 0x7f1ff4105420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.226+0000 7f1ff3400640 1 -- 172.21.15.2:0/3127123494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff41099f0 msgr2=0x7f1ff4101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.226+0000 7f1ff3400640 1 --2- 172.21.15.2:0/3127123494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff41099f0 0x7f1ff4101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.226+0000 7f1ff3400640 1 -- 172.21.15.2:0/3127123494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff4102310 con 0x7f1ff4074bf0 2024-09-10T08:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.226+0000 7f1ff3400640 1 --2- 172.21.15.2:0/3127123494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ff4074bf0 0x7f1ff4104ee0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009230 tx=0x7f1fe40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.227+0000 7f1febe00640 1 -- 172.21.15.2:0/3127123494 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe403f030 con 0x7f1ff4074bf0 2024-09-10T08:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.227+0000 7f1ffa200640 1 -- 172.21.15.2:0/3127123494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff41a6430 con 0x7f1ff4074bf0 2024-09-10T08:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.227+0000 7f1febe00640 1 -- 172.21.15.2:0/3127123494 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe4003ce0 con 0x7f1ff4074bf0 2024-09-10T08:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.227+0000 7f1febe00640 1 -- 172.21.15.2:0/3127123494 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4035070 con 0x7f1ff4074bf0 2024-09-10T08:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.227+0000 7f1ffa200640 1 -- 172.21.15.2:0/3127123494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff41a67e0 con 0x7f1ff4074bf0 2024-09-10T08:08:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.230+0000 7f1ffcfbf740 1 Processor -- start 2024-09-10T08:08:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.230+0000 7f1ffcfbf740 1 -- start start 2024-09-10T08:08:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.230+0000 7f1febe00640 1 -- 172.21.15.2:0/3127123494 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f1fe4042030 con 0x7f1ff4074bf0 2024-09-10T08:08:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.230+0000 7f1ffcfbf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0a090 0x55e57fe0a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.230+0000 7f1ffcfbf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a9d0 0x55e57fe0f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.230+0000 7f1ffcfbf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0fa60 0x55e57fe11f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.230+0000 7f1ffcfbf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e57fd6f100 con 0x55e57fe0a9d0 2024-09-10T08:08:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.230+0000 7f1ffcfbf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e57fd171b0 con 0x55e57fe0fa60 2024-09-10T08:08:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.230+0000 7f1ffcfbf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e57fd6b0c0 con 0x55e57fe0a090 2024-09-10T08:08:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.230+0000 7f1ff3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a9d0 0x55e57fe0f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.231+0000 7f1ff2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0fa60 0x55e57fe11f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.231+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0a090 0x55e57fe0a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.231+0000 7f1ff2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0fa60 0x55e57fe11f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35096/0 (socket says 172.21.15.2:35096) 2024-09-10T08:08:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.231+0000 7f1ff2a00640 1 -- 172.21.15.2:0/2902545535 learned_addr learned my addr 172.21.15.2:0/2902545535 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.231+0000 7f1ff3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a9d0 0x55e57fe0f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55700/0 (socket says 172.21.15.2:55700) 2024-09-10T08:08:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.231+0000 7f1ff3e00640 1 -- 172.21.15.2:0/2902545535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0fa60 msgr2=0x55e57fe11f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.231+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/2902545535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0fa60 0x55e57fe11f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.231+0000 7f1ff3e00640 1 -- 172.21.15.2:0/2902545535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a9d0 msgr2=0x55e57fe0f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.231+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/2902545535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a9d0 0x55e57fe0f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.231+0000 7f1ff3e00640 1 -- 172.21.15.2:0/2902545535 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e57fa82210 con 0x55e57fe0a090 2024-09-10T08:08:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.231+0000 7f1ff2a00640 1 --2- 172.21.15.2:0/2902545535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0fa60 0x55e57fe11f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.231+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/2902545535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0a090 0x55e57fe0a490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009f00 tx=0x7f1fec011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6854825fe7268303 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1febe00640 1 --2- 172.21.15.2:0/3127123494 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1fd80a2cd0 0x7f1fd80a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1fdfe00640 1 -- 172.21.15.2:0/2902545535 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec004d20 con 0x55e57fe0a090 2024-09-10T08:08:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1fdfe00640 1 -- 172.21.15.2:0/2902545535 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fec004030 con 0x55e57fe0a090 2024-09-10T08:08:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1fdfe00640 1 -- 172.21.15.2:0/2902545535 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec005220 con 0x55e57fe0a090 2024-09-10T08:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1febe00640 1 -- 172.21.15.2:0/3127123494 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1fe40efab0 con 0x7f1ff4074bf0 2024-09-10T08:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1ff3400640 1 --2- 172.21.15.2:0/3127123494 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1fd80a2cd0 0x7f1fd80a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/2902545535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0a090 msgr2=0x55e57fe0a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/2902545535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0a090 0x55e57fe0a490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009f00 tx=0x7f1fec011810 comp rx=0 tx=0).stop 2024-09-10T08:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/2902545535 shutdown_connections 2024-09-10T08:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/2902545535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0fa60 0x55e57fe11f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/2902545535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a9d0 0x55e57fe0f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/2902545535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0a090 0x55e57fe0a490 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/2902545535 >> 172.21.15.2:0/2902545535 conn(0x55e57fe05700 msgr2=0x55e57fe05b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.232+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/2902545535 shutdown_connections 2024-09-10T08:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.233+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/2902545535 wait complete. 2024-09-10T08:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.233+0000 7f1ff3400640 1 --2- 172.21.15.2:0/3127123494 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1fd80a2cd0 0x7f1fd80a5190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40bbde0 tx=0x7f1fe40f35a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.233+0000 7f1ffcfbf740 1 -- start start 2024-09-10T08:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.233+0000 7f1ffcfbf740 1 Processor -- start 2024-09-10T08:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.234+0000 7f1ffcfbf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a090 0x55e57fe141f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.234+0000 7f1ffcfbf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0a9d0 0x55e57fe14730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.234+0000 7f1ffcfbf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0fa60 0x55e57fe1b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.234+0000 7f1ffcfbf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e57fe12a30 con 0x55e57fe0fa60 2024-09-10T08:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.234+0000 7f1ffcfbf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e57fcf8090 con 0x55e57fe0a090 2024-09-10T08:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.234+0000 7f1ffcfbf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e57fe12bb0 con 0x55e57fe0a9d0 2024-09-10T08:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.234+0000 7f1ffcfbf740 10 client.0 _ll_register_callbacks cb 0x55e57fd4cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.234+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0fa60 0x55e57fe1b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.234+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0fa60 0x55e57fe1b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55704/0 (socket says 172.21.15.2:55704) 2024-09-10T08:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.234+0000 7f1ff3e00640 1 -- 172.21.15.2:0/3919909849 learned_addr learned my addr 172.21.15.2:0/3919909849 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.234+0000 7f1ff2a00640 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0a9d0 0x55e57fe14730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.234+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a090 0x55e57fe141f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1ff2a00640 1 -- 172.21.15.2:0/3919909849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a090 msgr2=0x55e57fe141f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1ff2a00640 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a090 0x55e57fe141f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1ff2a00640 1 -- 172.21.15.2:0/3919909849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0fa60 msgr2=0x55e57fe1b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1ff2a00640 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0fa60 0x55e57fe1b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1ff2a00640 1 -- 172.21.15.2:0/3919909849 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e57fb4e620 con 0x55e57fe0a9d0 2024-09-10T08:08:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0fa60 0x55e57fe1b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:08:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1ff2a00640 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0a9d0 0x55e57fe14730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00058c0 tx=0x7f1fe0012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43a6820042a8dffd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1fdcc00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:08:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1fdcc00640 1 -- 172.21.15.2:0/3919909849 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0004030 con 0x55e57fe0a9d0 2024-09-10T08:08:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/3919909849 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fec004030 con 0x55e57fe0a9d0 2024-09-10T08:08:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1fdcc00640 1 -- 172.21.15.2:0/3919909849 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1fe00041d0 con 0x55e57fe0a9d0 2024-09-10T08:08:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1fdcc00640 1 -- 172.21.15.2:0/3919909849 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1fe0025050 con 0x55e57fe0a9d0 2024-09-10T08:08:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1ffcfbf740 10 client.25360 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.235+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/3919909849 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e57fe1e630 con 0x55e57fe0a9d0 2024-09-10T08:08:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.236+0000 7f1fdcc00640 1 -- 172.21.15.2:0/3919909849 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fec004030 con 0x55e57fe0a9d0 2024-09-10T08:08:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.236+0000 7f1fdcc00640 1 -- 172.21.15.2:0/3919909849 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1fe00039e0 con 0x55e57fe0a9d0 2024-09-10T08:08:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.236+0000 7f1ffcfbf740 10 client.25360 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.236+0000 7f1ffcfbf740 10 client.25360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:08:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.236+0000 7f1ffcfbf740 20 client.25360 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:56.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.236+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e57fe1fd50 0x55e57fe22140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.236+0000 7f1ffcfbf740 4 client.25360 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:08:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.236+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/3919909849 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e57fd7c830 con 0x55e57fe1fd50 2024-09-10T08:08:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.237+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e57fe1fd50 0x55e57fe22140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.237+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e57fe1fd50 0x55e57fe22140 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9691ab4810c3c0cd server_cookie=88344b46a1a52d22 in_seq=0 out_seq=0 2024-09-10T08:08:56.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.237+0000 7f1fdcc00640 10 client.25360 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1fdcc00640 1 -- 172.21.15.2:0/3919909849 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e57fd7c830 con 0x55e57fe1fd50 2024-09-10T08:08:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1fdcc00640 10 client.25360 handle_command_reply: tid=0 2024-09-10T08:08:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1ffcfbf740 1 client.25360 shutdown 2024-09-10T08:08:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/3919909849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0a9d0 msgr2=0x55e57fe14730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0a9d0 0x55e57fe14730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00058c0 tx=0x7f1fe0012b60 comp rx=0 tx=0).stop 2024-09-10T08:08:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/3919909849 shutdown_connections 2024-09-10T08:08:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e57fe1fd50 0x55e57fe22140 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0fa60 0x55e57fe1b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0a9d0 0x55e57fe14730 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/3919909849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a090 0x55e57fe141f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1fdcc00640 0 client.25360 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/3919909849 >> 172.21.15.2:0/3919909849 conn(0x55e57fe05700 msgr2=0x55e57fe06f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/3919909849 shutdown_connections 2024-09-10T08:08:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/3919909849 wait complete. 2024-09-10T08:08:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.238+0000 7f1ffcfbf740 20 client.25360 trim_cache size 0 max 16384 2024-09-10T08:08:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.254+0000 7f1ffcfbf740 1 Processor -- start 2024-09-10T08:08:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.254+0000 7f1ffcfbf740 1 -- start start 2024-09-10T08:08:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.254+0000 7f1ffcfbf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a090 0x55e57fe0f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.254+0000 7f1ffcfbf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a9d0 0x55e57fe1b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.254+0000 7f1ffcfbf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0fa60 0x55e57fe04770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.255+0000 7f1ffcfbf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41b3710 con 0x55e57fe0a090 2024-09-10T08:08:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.255+0000 7f1ffcfbf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41b0fe0 con 0x55e57fe0a9d0 2024-09-10T08:08:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.255+0000 7f1ffcfbf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41aee30 con 0x55e57fe0fa60 2024-09-10T08:08:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.255+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a090 0x55e57fe0f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.255+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a090 0x55e57fe0f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55708/0 (socket says 172.21.15.2:55708) 2024-09-10T08:08:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.255+0000 7f1ff3e00640 1 -- 172.21.15.2:0/1357655488 learned_addr learned my addr 172.21.15.2:0/1357655488 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.255+0000 7f1ff2a00640 1 --2- 172.21.15.2:0/1357655488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a9d0 0x55e57fe1b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.255+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/1357655488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0fa60 0x55e57fe04770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.255+0000 7f1ff3e00640 1 -- 172.21.15.2:0/1357655488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0fa60 msgr2=0x55e57fe04770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.255+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/1357655488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0fa60 0x55e57fe04770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.255+0000 7f1ff3e00640 1 -- 172.21.15.2:0/1357655488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a9d0 msgr2=0x55e57fe1b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.255+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/1357655488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a9d0 0x55e57fe1b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.255+0000 7f1ff3e00640 1 -- 172.21.15.2:0/1357655488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e57fe00d60 con 0x55e57fe0a090 2024-09-10T08:08:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.256+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/1357655488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a090 0x55e57fe0f1d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009270 tx=0x7f1fec009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321810a7cb7b7f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.256+0000 7f1fdf400640 1 -- 172.21.15.2:0/1357655488 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec01e040 con 0x55e57fe0a090 2024-09-10T08:08:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.256+0000 7f1fdf400640 1 -- 172.21.15.2:0/1357655488 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fec004c10 con 0x55e57fe0a090 2024-09-10T08:08:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.256+0000 7f1fdf400640 1 -- 172.21.15.2:0/1357655488 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec005760 con 0x55e57fe0a090 2024-09-10T08:08:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.256+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/1357655488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a090 msgr2=0x55e57fe0f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.256+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/1357655488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a090 0x55e57fe0f1d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009270 tx=0x7f1fec009500 comp rx=0 tx=0).stop 2024-09-10T08:08:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.256+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/1357655488 shutdown_connections 2024-09-10T08:08:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.256+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/1357655488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0fa60 0x55e57fe04770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.256+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/1357655488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a9d0 0x55e57fe1b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.256+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/1357655488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a090 0x55e57fe0f1d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.256+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/1357655488 >> 172.21.15.2:0/1357655488 conn(0x55e57fe05700 msgr2=0x55e57fe04060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.257+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/1357655488 shutdown_connections 2024-09-10T08:08:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.257+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/1357655488 wait complete. 2024-09-10T08:08:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.257+0000 7f1ffcfbf740 1 -- start start 2024-09-10T08:08:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.257+0000 7f1ffcfbf740 1 Processor -- start 2024-09-10T08:08:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.257+0000 7f1ffcfbf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a090 0x55e57fe187e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ffcfbf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a9d0 0x55e57fe18d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ffcfbf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0fa60 0x55e57fe19d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ffcfbf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41af4a0 con 0x55e57fe0a090 2024-09-10T08:08:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ffcfbf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41b0910 con 0x55e57fe0a9d0 2024-09-10T08:08:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ffcfbf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41afb70 con 0x55e57fe0fa60 2024-09-10T08:08:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ffcfbf740 10 client.0 _ll_register_callbacks cb 0x7f1fe0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:08:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a090 0x55e57fe187e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ff2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a9d0 0x55e57fe18d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ff2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a9d0 0x55e57fe18d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35116/0 (socket says 172.21.15.2:35116) 2024-09-10T08:08:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ff2a00640 1 -- 172.21.15.2:0/4142247767 learned_addr learned my addr 172.21.15.2:0/4142247767 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:08:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/4142247767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0fa60 0x55e57fe19d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ff2a00640 1 -- 172.21.15.2:0/4142247767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0fa60 msgr2=0x55e57fe19d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ff2a00640 1 --2- 172.21.15.2:0/4142247767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0fa60 0x55e57fe19d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ff2a00640 1 -- 172.21.15.2:0/4142247767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a090 msgr2=0x55e57fe187e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:08:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ff2a00640 1 --2- 172.21.15.2:0/4142247767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a090 0x55e57fe187e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ff2a00640 1 -- 172.21.15.2:0/4142247767 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e57fe0b660 con 0x55e57fe0a9d0 2024-09-10T08:08:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/4142247767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0fa60 0x55e57fe19d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:08:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1ff2a00640 1 --2- 172.21.15.2:0/4142247767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a9d0 0x55e57fe18d20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009e70 tx=0x7f1fe000fb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f02c1031f90a0cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:08:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.258+0000 7f1fdcc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:08:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.259+0000 7f1fdcc00640 1 -- 172.21.15.2:0/4142247767 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0022040 con 0x55e57fe0a9d0 2024-09-10T08:08:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.259+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/4142247767 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fec004c10 con 0x55e57fe0a9d0 2024-09-10T08:08:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.259+0000 7f1fdcc00640 1 -- 172.21.15.2:0/4142247767 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1fe00035f0 con 0x55e57fe0a9d0 2024-09-10T08:08:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.259+0000 7f1fdcc00640 1 -- 172.21.15.2:0/4142247767 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1fe0037050 con 0x55e57fe0a9d0 2024-09-10T08:08:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.259+0000 7f1ffcfbf740 10 client.25391 fetch_fsmap learned FSMap version 26 2024-09-10T08:08:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.259+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/4142247767 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e57fe1b520 con 0x55e57fe0a9d0 2024-09-10T08:08:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.259+0000 7f1fdcc00640 1 -- 172.21.15.2:0/4142247767 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fec004c10 con 0x55e57fe0a9d0 2024-09-10T08:08:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.259+0000 7f1fdcc00640 1 -- 172.21.15.2:0/4142247767 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1fe0010480 con 0x55e57fe0a9d0 2024-09-10T08:08:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.259+0000 7f1ffcfbf740 10 client.25391 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:08:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.259+0000 7f1ffcfbf740 10 client.25391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:08:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.260+0000 7f1ffcfbf740 20 client.25391 populate_metadata read hostname 'smithi002' 2024-09-10T08:08:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.260+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/4142247767 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e57fe8ac60 0x55e57fe8d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:08:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.260+0000 7f1ffcfbf740 4 client.25391 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:08:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.260+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/4142247767 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e57fd76b80 con 0x55e57fe8ac60 2024-09-10T08:08:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.260+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/4142247767 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e57fe8ac60 0x55e57fe8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:08:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.260+0000 7f1ff3e00640 1 --2- 172.21.15.2:0/4142247767 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e57fe8ac60 0x55e57fe8d080 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4beae06aa2bcb2c1 server_cookie=261ad0dcbca1df9f in_seq=0 out_seq=0 2024-09-10T08:08:56.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.260+0000 7f1fdcc00640 10 client.25391 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1fdcc00640 1 -- 172.21.15.2:0/4142247767 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e57fd76b80 con 0x55e57fe8ac60 2024-09-10T08:08:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1fdcc00640 10 client.25391 handle_command_reply: tid=0 2024-09-10T08:08:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1ffcfbf740 1 client.25391 shutdown 2024-09-10T08:08:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/4142247767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a9d0 msgr2=0x55e57fe18d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/4142247767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a9d0 0x55e57fe18d20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009e70 tx=0x7f1fe000fb50 comp rx=0 tx=0).stop 2024-09-10T08:08:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/4142247767 shutdown_connections 2024-09-10T08:08:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/4142247767 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e57fe8ac60 0x55e57fe8d080 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/4142247767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e57fe0fa60 0x55e57fe19d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/4142247767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e57fe0a9d0 0x55e57fe18d20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1ffcfbf740 1 --2- 172.21.15.2:0/4142247767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e57fe0a090 0x55e57fe187e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1fdcc00640 0 client.25391 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:08:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/4142247767 >> 172.21.15.2:0/4142247767 conn(0x55e57fe05700 msgr2=0x55e57fe04060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/4142247767 shutdown_connections 2024-09-10T08:08:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.269+0000 7f1ffcfbf740 1 -- 172.21.15.2:0/4142247767 wait complete. 2024-09-10T08:08:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.270+0000 7f1ffcfbf740 20 client.25391 trim_cache size 0 max 16384 2024-09-10T08:08:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:08:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:08:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:08:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:08:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.270+0000 7f1ffa200640 1 -- 172.21.15.2:0/3127123494 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1fd80a2cd0 msgr2=0x7f1fd80a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.270+0000 7f1ffa200640 1 --2- 172.21.15.2:0/3127123494 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1fd80a2cd0 0x7f1fd80a5190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40bbde0 tx=0x7f1fe40f35a0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.270+0000 7f1ffa200640 1 -- 172.21.15.2:0/3127123494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ff4074bf0 msgr2=0x7f1ff4104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:08:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.270+0000 7f1ffa200640 1 --2- 172.21.15.2:0/3127123494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ff4074bf0 0x7f1ff4104ee0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009230 tx=0x7f1fe40042f0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.270+0000 7f1ffa200640 1 -- 172.21.15.2:0/3127123494 shutdown_connections 2024-09-10T08:08:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.270+0000 7f1ffa200640 1 --2- 172.21.15.2:0/3127123494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff41099f0 0x7f1ff4101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.270+0000 7f1ffa200640 1 --2- 172.21.15.2:0/3127123494 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1fd80a2cd0 0x7f1fd80a5190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.270+0000 7f1ffa200640 1 --2- 172.21.15.2:0/3127123494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ff40761d0 0x7f1ff4105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.270+0000 7f1ffa200640 1 --2- 172.21.15.2:0/3127123494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ff4074bf0 0x7f1ff4104ee0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:08:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.270+0000 7f1ffa200640 1 -- 172.21.15.2:0/3127123494 >> 172.21.15.2:0/3127123494 conn(0x7f1ff40fd2e0 msgr2=0x7f1ff4105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:08:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.270+0000 7f1ffa200640 1 -- 172.21.15.2:0/3127123494 shutdown_connections 2024-09-10T08:08:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:08:56.270+0000 7f1ffa200640 1 -- 172.21.15.2:0/3127123494 wait complete. 2024-09-10T08:08:57.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:57 smithi002 ceph-mon[30563]: pgmap v411: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-10T08:08:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:57 smithi155 ceph-mon[35405]: pgmap v411: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-10T08:08:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:57 smithi107 ceph-mon[35048]: pgmap v411: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-10T08:08:59.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:08:59 smithi002 ceph-mon[30563]: pgmap v412: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-10T08:08:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:08:59 smithi155 ceph-mon[35405]: pgmap v412: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-10T08:08:59.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:08:59 smithi107 ceph-mon[35048]: pgmap v412: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-10T08:09:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:01 smithi002 ceph-mon[30563]: pgmap v413: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-10T08:09:01.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:01 smithi155 ceph-mon[35405]: pgmap v413: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-10T08:09:01.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:01 smithi107 ceph-mon[35048]: pgmap v413: 129 pgs: 129 active+clean; 663 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-10T08:09:03.774 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:03 smithi002 ceph-mon[30563]: pgmap v414: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:09:03.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:03 smithi155 ceph-mon[35405]: pgmap v414: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:09:03.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:03 smithi107 ceph-mon[35048]: pgmap v414: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:09:05.383 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:09:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:05 smithi002 ceph-mon[30563]: pgmap v415: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-10T08:09:05.807 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:05.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:05 smithi155 ceph-mon[35405]: pgmap v415: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-10T08:09:05.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:05 smithi107 ceph-mon[35048]: pgmap v415: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-10T08:09:06.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.415+0000 7fc5c3200640 1 -- 172.21.15.2:0/3110945161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5bc10c590 msgr2=0x7fc5bc10c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:06.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.415+0000 7fc5c3200640 1 --2- 172.21.15.2:0/3110945161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5bc10c590 0x7fc5bc10c9f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009900 tx=0x7fc5a802a410 comp rx=0 tx=0).stop 2024-09-10T08:09:06.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.415+0000 7fc5c3200640 1 -- 172.21.15.2:0/3110945161 shutdown_connections 2024-09-10T08:09:06.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.415+0000 7fc5c3200640 1 --2- 172.21.15.2:0/3110945161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5bc10cf30 0x7fc5bc119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.415+0000 7fc5c3200640 1 --2- 172.21.15.2:0/3110945161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5bc10c590 0x7fc5bc10c9f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.415+0000 7fc5c3200640 1 --2- 172.21.15.2:0/3110945161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5bc112640 0x7fc5bc112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.415+0000 7fc5c3200640 1 -- 172.21.15.2:0/3110945161 >> 172.21.15.2:0/3110945161 conn(0x7fc5bc06cdc0 msgr2=0x7fc5bc06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:06.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.415+0000 7fc5c3200640 1 -- 172.21.15.2:0/3110945161 shutdown_connections 2024-09-10T08:09:06.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.415+0000 7fc5c3200640 1 -- 172.21.15.2:0/3110945161 wait complete. 2024-09-10T08:09:06.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.416+0000 7fc5c3200640 1 Processor -- start 2024-09-10T08:09:06.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.416+0000 7fc5c3200640 1 -- start start 2024-09-10T08:09:06.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.417+0000 7fc5c3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5bc10c590 0x7fc5bc1a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.417+0000 7fc5c3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5bc10cf30 0x7fc5bc1a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.417+0000 7fc5c3200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5bc112640 0x7fc5bc1a7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.417+0000 7fc5c3200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc11c9a0 con 0x7fc5bc112640 2024-09-10T08:09:06.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.417+0000 7fc5c3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc11c820 con 0x7fc5bc10cf30 2024-09-10T08:09:06.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.417+0000 7fc5c3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc11cb20 con 0x7fc5bc10c590 2024-09-10T08:09:06.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.417+0000 7fc5c1200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5bc112640 0x7fc5bc1a7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.417+0000 7fc5bbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5bc10c590 0x7fc5bc1a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.417+0000 7fc5c1200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5bc112640 0x7fc5bc1a7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56504/0 (socket says 172.21.15.2:56504) 2024-09-10T08:09:06.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.417+0000 7fc5c1200640 1 -- 172.21.15.2:0/657745019 learned_addr learned my addr 172.21.15.2:0/657745019 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:06.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.417+0000 7fc5bb400640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5bc10cf30 0x7fc5bc1a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.418+0000 7fc5c1200640 1 -- 172.21.15.2:0/657745019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5bc10c590 msgr2=0x7fc5bc1a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:06.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.418+0000 7fc5c1200640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5bc10c590 0x7fc5bc1a2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.418+0000 7fc5c1200640 1 -- 172.21.15.2:0/657745019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5bc10cf30 msgr2=0x7fc5bc1a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:06.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.418+0000 7fc5c1200640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5bc10cf30 0x7fc5bc1a32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.418+0000 7fc5c1200640 1 -- 172.21.15.2:0/657745019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc1a7d60 con 0x7fc5bc112640 2024-09-10T08:09:06.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.418+0000 7fc5bb400640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5bc10cf30 0x7fc5bc1a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:06.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.418+0000 7fc5c1200640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5bc112640 0x7fc5bc1a7680 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400eb10 tx=0x7fc5b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:06.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.418+0000 7fc5b8c00640 1 -- 172.21.15.2:0/657745019 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4004b20 con 0x7fc5bc112640 2024-09-10T08:09:06.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.418+0000 7fc5b8c00640 1 -- 172.21.15.2:0/657745019 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b4004cc0 con 0x7fc5bc112640 2024-09-10T08:09:06.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.418+0000 7fc5c3200640 1 -- 172.21.15.2:0/657745019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5bc1a8050 con 0x7fc5bc112640 2024-09-10T08:09:06.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.418+0000 7fc5b8c00640 1 -- 172.21.15.2:0/657745019 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4010760 con 0x7fc5bc112640 2024-09-10T08:09:06.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.418+0000 7fc5c3200640 1 -- 172.21.15.2:0/657745019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5bc1af930 con 0x7fc5bc112640 2024-09-10T08:09:06.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.422+0000 7fc5c5e22740 1 Processor -- start 2024-09-10T08:09:06.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.422+0000 7fc5b8c00640 1 -- 172.21.15.2:0/657745019 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fc5b4005020 con 0x7fc5bc112640 2024-09-10T08:09:06.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.422+0000 7fc5c5e22740 1 -- start start 2024-09-10T08:09:06.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.422+0000 7fc5c5e22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 0x555f6878ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.423+0000 7fc5c5e22740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878f4b0 0x555f68799830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.423+0000 7fc5c5e22740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 0x555f6879c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.423+0000 7fc5c5e22740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555f686f53e0 con 0x555f6878f4b0 2024-09-10T08:09:06.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.423+0000 7fc5c5e22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555f6869cd80 con 0x555f6878eb90 2024-09-10T08:09:06.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.423+0000 7fc5c5e22740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555f686f13a0 con 0x555f68799d70 2024-09-10T08:09:06.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.423+0000 7fc5bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 0x555f6878ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.423+0000 7fc5bbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878f4b0 0x555f68799830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.423+0000 7fc5bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 0x555f6878ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36320/0 (socket says 172.21.15.2:36320) 2024-09-10T08:09:06.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.423+0000 7fc5bb400640 1 -- 172.21.15.2:0/1603816879 learned_addr learned my addr 172.21.15.2:0/1603816879 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:06.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.423+0000 7fc5bb400640 1 --2- 172.21.15.2:0/1603816879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 0x555f6879c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.424+0000 7fc5bbe00640 1 -- 172.21.15.2:0/1603816879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 msgr2=0x555f6879c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:06.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.424+0000 7fc5bbe00640 1 --2- 172.21.15.2:0/1603816879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 0x555f6879c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.424+0000 7fc5bbe00640 1 -- 172.21.15.2:0/1603816879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 msgr2=0x555f6878ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:06.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.424+0000 7fc5bbe00640 1 --2- 172.21.15.2:0/1603816879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 0x555f6878ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.424+0000 7fc5bbe00640 1 -- 172.21.15.2:0/1603816879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f686bbae0 con 0x555f6878f4b0 2024-09-10T08:09:06.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.424+0000 7fc5bb400640 1 --2- 172.21.15.2:0/1603816879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 0x555f6878ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:06.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.424+0000 7fc5b8c00640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc5a00a2c10 0x7fc5a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.424+0000 7fc5b8c00640 1 -- 172.21.15.2:0/657745019 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc5b40cfc60 con 0x7fc5bc112640 2024-09-10T08:09:06.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.424+0000 7fc5bb400640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc5a00a2c10 0x7fc5a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.424+0000 7fc5bbe00640 1 --2- 172.21.15.2:0/1603816879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878f4b0 0x555f68799830 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009c00 tx=0x7fc5ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6702f6fa4b793a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:06.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.425+0000 7fc5b1600640 1 -- 172.21.15.2:0/1603816879 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac004230 con 0x555f6878f4b0 2024-09-10T08:09:06.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.425+0000 7fc5b1600640 1 -- 172.21.15.2:0/1603816879 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ac0043d0 con 0x555f6878f4b0 2024-09-10T08:09:06.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.425+0000 7fc5b1600640 1 -- 172.21.15.2:0/1603816879 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac004ce0 con 0x555f6878f4b0 2024-09-10T08:09:06.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.426+0000 7fc5c5e22740 1 -- 172.21.15.2:0/1603816879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878f4b0 msgr2=0x555f68799830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:06.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.426+0000 7fc5bb400640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc5a00a2c10 0x7fc5a00a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x555f6878c4a0 tx=0x7fc5a802a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:06.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.426+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/1603816879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878f4b0 0x555f68799830 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009c00 tx=0x7fc5ac011810 comp rx=0 tx=0).stop 2024-09-10T08:09:06.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.426+0000 7fc5c5e22740 1 -- 172.21.15.2:0/1603816879 shutdown_connections 2024-09-10T08:09:06.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.426+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/1603816879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 0x555f6879c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.426+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/1603816879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878f4b0 0x555f68799830 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.426+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/1603816879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 0x555f6878ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.426+0000 7fc5c5e22740 1 -- 172.21.15.2:0/1603816879 >> 172.21.15.2:0/1603816879 conn(0x555f6878b120 msgr2=0x555f6878b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:06.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.426+0000 7fc5c5e22740 1 -- 172.21.15.2:0/1603816879 shutdown_connections 2024-09-10T08:09:06.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.426+0000 7fc5c5e22740 1 -- 172.21.15.2:0/1603816879 wait complete. 2024-09-10T08:09:06.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.427+0000 7fc5c5e22740 1 -- start start 2024-09-10T08:09:06.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.427+0000 7fc5c5e22740 1 Processor -- start 2024-09-10T08:09:06.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.427+0000 7fc5c5e22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 0x555f68796bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.427+0000 7fc5c5e22740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878f4b0 0x555f68797110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.427+0000 7fc5c5e22740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 0x555f68797650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.427+0000 7fc5c5e22740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555f6879ca30 con 0x555f6878f4b0 2024-09-10T08:09:06.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.427+0000 7fc5c5e22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555f686b00f0 con 0x555f6878eb90 2024-09-10T08:09:06.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.427+0000 7fc5c5e22740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555f6879cbb0 con 0x555f68799d70 2024-09-10T08:09:06.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.427+0000 7fc5c5e22740 10 client.0 _ll_register_callbacks cb 0x555f686a19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:06.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.428+0000 7fc5bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 0x555f68796bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.428+0000 7fc5bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 0x555f68796bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36332/0 (socket says 172.21.15.2:36332) 2024-09-10T08:09:06.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.428+0000 7fc5bbe00640 1 -- 172.21.15.2:0/91427828 learned_addr learned my addr 172.21.15.2:0/91427828 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:06.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.428+0000 7fc5bb400640 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878f4b0 0x555f68797110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.428+0000 7fc5c1200640 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 0x555f68797650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.428+0000 7fc5bb400640 1 -- 172.21.15.2:0/91427828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 msgr2=0x555f68797650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:06.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.428+0000 7fc5bb400640 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 0x555f68797650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.428+0000 7fc5bb400640 1 -- 172.21.15.2:0/91427828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 msgr2=0x555f68796bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:06.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.428+0000 7fc5bb400640 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 0x555f68796bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.428+0000 7fc5bb400640 1 -- 172.21.15.2:0/91427828 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f684663c0 con 0x555f6878f4b0 2024-09-10T08:09:06.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.428+0000 7fc5c1200640 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 0x555f68797650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:06.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.428+0000 7fc5bbe00640 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 0x555f68796bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:06.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.429+0000 7fc5bb400640 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878f4b0 0x555f68797110 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8005ec0 tx=0x7fc5a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478060fddcbf11b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:06.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.429+0000 7fc5a6000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:09:06.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.429+0000 7fc5a6000640 1 -- 172.21.15.2:0/91427828 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8035040 con 0x555f6878f4b0 2024-09-10T08:09:06.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.429+0000 7fc5a6000640 1 -- 172.21.15.2:0/91427828 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc5a8031750 con 0x555f6878f4b0 2024-09-10T08:09:06.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.429+0000 7fc5c5e22740 1 -- 172.21.15.2:0/91427828 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5ac0043d0 con 0x555f6878f4b0 2024-09-10T08:09:06.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.429+0000 7fc5a6000640 1 -- 172.21.15.2:0/91427828 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc5a805c050 con 0x555f6878f4b0 2024-09-10T08:09:06.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.430+0000 7fc5c5e22740 10 client.15381 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:06.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.430+0000 7fc5c5e22740 1 -- 172.21.15.2:0/91427828 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555f687a8660 con 0x555f6878f4b0 2024-09-10T08:09:06.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.430+0000 7fc5a6000640 1 -- 172.21.15.2:0/91427828 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ac0043d0 con 0x555f6878f4b0 2024-09-10T08:09:06.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.430+0000 7fc5a6000640 1 -- 172.21.15.2:0/91427828 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc5a8048040 con 0x555f6878f4b0 2024-09-10T08:09:06.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.430+0000 7fc5c5e22740 10 client.15381 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:06.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.430+0000 7fc5c5e22740 10 client.15381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:06.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.430+0000 7fc5c5e22740 20 client.15381 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:06.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.430+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555f687a9b20 0x555f687abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.430+0000 7fc5c5e22740 4 client.15381 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:06.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.430+0000 7fc5c5e22740 1 -- 172.21.15.2:0/91427828 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f68702950 con 0x555f687a9b20 2024-09-10T08:09:06.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.431+0000 7fc5bbe00640 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555f687a9b20 0x555f687abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.431+0000 7fc5bbe00640 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555f687a9b20 0x555f687abf90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec2724304ae03a1 server_cookie=cc31282cadc7c44d in_seq=0 out_seq=0 2024-09-10T08:09:06.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.431+0000 7fc5a6000640 10 client.15381 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:06.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.432+0000 7fc5a6000640 1 -- 172.21.15.2:0/91427828 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f68702950 con 0x555f687a9b20 2024-09-10T08:09:06.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.432+0000 7fc5a6000640 10 client.15381 handle_command_reply: tid=0 2024-09-10T08:09:06.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.432+0000 7fc5c5e22740 1 client.15381 shutdown 2024-09-10T08:09:06.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.432+0000 7fc5c5e22740 1 -- 172.21.15.2:0/91427828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878f4b0 msgr2=0x555f68797110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:06.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.432+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878f4b0 0x555f68797110 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8005ec0 tx=0x7fc5a8009260 comp rx=0 tx=0).stop 2024-09-10T08:09:06.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.432+0000 7fc5c5e22740 1 -- 172.21.15.2:0/91427828 shutdown_connections 2024-09-10T08:09:06.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.432+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555f687a9b20 0x555f687abf90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.432+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 0x555f68797650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.432+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878f4b0 0x555f68797110 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.432+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/91427828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878eb90 0x555f68796bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.432+0000 7fc5c5e22740 1 -- 172.21.15.2:0/91427828 >> 172.21.15.2:0/91427828 conn(0x555f6878b120 msgr2=0x555f68790e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:06.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.433+0000 7fc5a6000640 0 client.15381 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:06.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.433+0000 7fc5c5e22740 1 -- 172.21.15.2:0/91427828 shutdown_connections 2024-09-10T08:09:06.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.433+0000 7fc5c5e22740 1 -- 172.21.15.2:0/91427828 wait complete. 2024-09-10T08:09:06.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.433+0000 7fc5c5e22740 20 client.15381 trim_cache size 0 max 16384 2024-09-10T08:09:06.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.451+0000 7fc5c5e22740 1 Processor -- start 2024-09-10T08:09:06.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.451+0000 7fc5c5e22740 1 -- start start 2024-09-10T08:09:06.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5c5e22740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878eb90 0x555f687abb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5c5e22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878f4b0 0x555f68790cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5c5e22740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 0x555f687b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5c5e22740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1bc040 con 0x555f6878eb90 2024-09-10T08:09:06.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5c5e22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1b9910 con 0x555f6878f4b0 2024-09-10T08:09:06.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5c5e22740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1b7760 con 0x555f68799d70 2024-09-10T08:09:06.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5bbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878eb90 0x555f687abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5bbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878eb90 0x555f687abb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56536/0 (socket says 172.21.15.2:56536) 2024-09-10T08:09:06.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5bbe00640 1 -- 172.21.15.2:0/3555170934 learned_addr learned my addr 172.21.15.2:0/3555170934 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:06.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5bb400640 1 --2- 172.21.15.2:0/3555170934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878f4b0 0x555f68790cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5c1200640 1 --2- 172.21.15.2:0/3555170934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 0x555f687b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5bbe00640 1 -- 172.21.15.2:0/3555170934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 msgr2=0x555f687b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:06.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5bbe00640 1 --2- 172.21.15.2:0/3555170934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 0x555f687b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5bbe00640 1 -- 172.21.15.2:0/3555170934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878f4b0 msgr2=0x555f68790cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:06.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5bbe00640 1 --2- 172.21.15.2:0/3555170934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878f4b0 0x555f68790cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5bbe00640 1 -- 172.21.15.2:0/3555170934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f68795b80 con 0x555f6878eb90 2024-09-10T08:09:06.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.452+0000 7fc5bb400640 1 --2- 172.21.15.2:0/3555170934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878f4b0 0x555f68790cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:06.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.453+0000 7fc5bbe00640 1 --2- 172.21.15.2:0/3555170934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878eb90 0x555f687abb30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac004950 tx=0x7fc5ac004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c372835c1b064 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:06.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.453+0000 7fc5b0c00640 1 -- 172.21.15.2:0/3555170934 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac032070 con 0x555f6878eb90 2024-09-10T08:09:06.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.453+0000 7fc5b0c00640 1 -- 172.21.15.2:0/3555170934 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ac02a070 con 0x555f6878eb90 2024-09-10T08:09:06.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.453+0000 7fc5b0c00640 1 -- 172.21.15.2:0/3555170934 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac01e040 con 0x555f6878eb90 2024-09-10T08:09:06.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.454+0000 7fc5c5e22740 1 -- 172.21.15.2:0/3555170934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878eb90 msgr2=0x555f687abb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:06.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.454+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/3555170934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878eb90 0x555f687abb30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac004950 tx=0x7fc5ac004980 comp rx=0 tx=0).stop 2024-09-10T08:09:06.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.454+0000 7fc5c5e22740 1 -- 172.21.15.2:0/3555170934 shutdown_connections 2024-09-10T08:09:06.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.454+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/3555170934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f68799d70 0x555f687b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.454+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/3555170934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878f4b0 0x555f68790cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.454+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/3555170934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f6878eb90 0x555f687abb30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.454+0000 7fc5c5e22740 1 -- 172.21.15.2:0/3555170934 >> 172.21.15.2:0/3555170934 conn(0x555f6878b120 msgr2=0x555f687905e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:06.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.454+0000 7fc5c5e22740 1 -- 172.21.15.2:0/3555170934 shutdown_connections 2024-09-10T08:09:06.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.454+0000 7fc5c5e22740 1 -- 172.21.15.2:0/3555170934 wait complete. 2024-09-10T08:09:06.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.455+0000 7fc5c5e22740 1 -- start start 2024-09-10T08:09:06.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.455+0000 7fc5c5e22740 1 Processor -- start 2024-09-10T08:09:06.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.455+0000 7fc5c5e22740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f6878eb90 0x555f687a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.455+0000 7fc5c5e22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878f4b0 0x555f687a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.455+0000 7fc5c5e22740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f68799d70 0x555f6879d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.455+0000 7fc5c5e22740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1b7dd0 con 0x555f68799d70 2024-09-10T08:09:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.455+0000 7fc5c5e22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1b9240 con 0x555f6878f4b0 2024-09-10T08:09:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.455+0000 7fc5c5e22740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1b84a0 con 0x555f6878eb90 2024-09-10T08:09:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5c5e22740 10 client.0 _ll_register_callbacks cb 0x7fc5a8003c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5bbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f6878eb90 0x555f687a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878f4b0 0x555f687a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5bbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f6878eb90 0x555f687a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:48662/0 (socket says 172.21.15.2:48662) 2024-09-10T08:09:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5bbe00640 1 -- 172.21.15.2:0/3719073740 learned_addr learned my addr 172.21.15.2:0/3719073740 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878f4b0 0x555f687a24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36346/0 (socket says 172.21.15.2:36346) 2024-09-10T08:09:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5c1200640 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f68799d70 0x555f6879d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5bbe00640 1 -- 172.21.15.2:0/3719073740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878f4b0 msgr2=0x555f687a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:06.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5bbe00640 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878f4b0 0x555f687a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5bbe00640 1 -- 172.21.15.2:0/3719073740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f68799d70 msgr2=0x555f6879d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:06.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5bbe00640 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f68799d70 0x555f6879d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5bbe00640 1 -- 172.21.15.2:0/3719073740 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f687ac300 con 0x555f6878eb90 2024-09-10T08:09:06.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5c1200640 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f68799d70 0x555f6879d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:06.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.456+0000 7fc5bb400640 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878f4b0 0x555f687a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.457+0000 7fc5bbe00640 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f6878eb90 0x555f687a1fa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac011850 tx=0x7fc5ac005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=929185e355968026 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.457+0000 7fc5a6000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:09:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.457+0000 7fc5a6000640 1 -- 172.21.15.2:0/3719073740 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac032070 con 0x555f6878eb90 2024-09-10T08:09:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.457+0000 7fc5c5e22740 1 -- 172.21.15.2:0/3719073740 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5ac02a070 con 0x555f6878eb90 2024-09-10T08:09:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.457+0000 7fc5a6000640 1 -- 172.21.15.2:0/3719073740 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc5ac02ca10 con 0x555f6878eb90 2024-09-10T08:09:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.457+0000 7fc5a6000640 1 -- 172.21.15.2:0/3719073740 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc5ac009780 con 0x555f6878eb90 2024-09-10T08:09:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.457+0000 7fc5c5e22740 10 client.25381 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.457+0000 7fc5c5e22740 1 -- 172.21.15.2:0/3719073740 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555f68814170 con 0x555f6878eb90 2024-09-10T08:09:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.458+0000 7fc5a6000640 1 -- 172.21.15.2:0/3719073740 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ac02a070 con 0x555f6878eb90 2024-09-10T08:09:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.458+0000 7fc5a6000640 1 -- 172.21.15.2:0/3719073740 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc5ac024040 con 0x555f6878eb90 2024-09-10T08:09:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.458+0000 7fc5c5e22740 10 client.25381 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.458+0000 7fc5c5e22740 10 client.25381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.458+0000 7fc5c5e22740 20 client.25381 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.458+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555f688145e0 0x555f688169d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.458+0000 7fc5c5e22740 4 client.25381 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:09:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.458+0000 7fc5c5e22740 1 -- 172.21.15.2:0/3719073740 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555f686fce60 con 0x555f688145e0 2024-09-10T08:09:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.459+0000 7fc5bb400640 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555f688145e0 0x555f688169d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.459+0000 7fc5bb400640 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555f688145e0 0x555f688169d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b2101d8dbb1d54 server_cookie=c258b41911d03211 in_seq=0 out_seq=0 2024-09-10T08:09:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.459+0000 7fc5a6000640 10 client.25381 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.459+0000 7fc5a6000640 1 -- 172.21.15.2:0/3719073740 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555f686fce60 con 0x555f688145e0 2024-09-10T08:09:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.459+0000 7fc5a6000640 10 client.25381 handle_command_reply: tid=0 2024-09-10T08:09:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.459+0000 7fc5c5e22740 1 client.25381 shutdown 2024-09-10T08:09:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.459+0000 7fc5c5e22740 1 -- 172.21.15.2:0/3719073740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f6878eb90 msgr2=0x555f687a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.459+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f6878eb90 0x555f687a1fa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac011850 tx=0x7fc5ac005f90 comp rx=0 tx=0).stop 2024-09-10T08:09:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.460+0000 7fc5c5e22740 1 -- 172.21.15.2:0/3719073740 shutdown_connections 2024-09-10T08:09:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.460+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555f688145e0 0x555f688169d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.460+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f68799d70 0x555f6879d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.460+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f6878f4b0 0x555f687a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.460+0000 7fc5c5e22740 1 --2- 172.21.15.2:0/3719073740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f6878eb90 0x555f687a1fa0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.460+0000 7fc5a6000640 0 client.25381 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:06.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.460+0000 7fc5c5e22740 1 -- 172.21.15.2:0/3719073740 >> 172.21.15.2:0/3719073740 conn(0x555f6878b120 msgr2=0x555f68790dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:06.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.460+0000 7fc5c5e22740 1 -- 172.21.15.2:0/3719073740 shutdown_connections 2024-09-10T08:09:06.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.460+0000 7fc5c5e22740 1 -- 172.21.15.2:0/3719073740 wait complete. 2024-09-10T08:09:06.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.460+0000 7fc5c5e22740 20 client.25381 trim_cache size 0 max 16384 2024-09-10T08:09:06.467 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:09:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:09:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:09:06.468 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:09:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.461+0000 7fc5c3200640 1 -- 172.21.15.2:0/657745019 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc5a00a2c10 msgr2=0x7fc5a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.461+0000 7fc5c3200640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc5a00a2c10 0x7fc5a00a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x555f6878c4a0 tx=0x7fc5a802a410 comp rx=0 tx=0).stop 2024-09-10T08:09:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.461+0000 7fc5c3200640 1 -- 172.21.15.2:0/657745019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5bc112640 msgr2=0x7fc5bc1a7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.461+0000 7fc5c3200640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5bc112640 0x7fc5bc1a7680 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400eb10 tx=0x7fc5b400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.461+0000 7fc5c3200640 1 -- 172.21.15.2:0/657745019 shutdown_connections 2024-09-10T08:09:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.461+0000 7fc5c3200640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5bc112640 0x7fc5bc1a7680 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.461+0000 7fc5c3200640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5bc10cf30 0x7fc5bc1a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.461+0000 7fc5c3200640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc5a00a2c10 0x7fc5a00a50d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.461+0000 7fc5c3200640 1 --2- 172.21.15.2:0/657745019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5bc10c590 0x7fc5bc1a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.461+0000 7fc5c3200640 1 -- 172.21.15.2:0/657745019 >> 172.21.15.2:0/657745019 conn(0x7fc5bc06cdc0 msgr2=0x7fc5bc10af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.461+0000 7fc5c3200640 1 -- 172.21.15.2:0/657745019 shutdown_connections 2024-09-10T08:09:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:06.461+0000 7fc5c3200640 1 -- 172.21.15.2:0/657745019 wait complete. 2024-09-10T08:09:07.227 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:09:07.227 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:09:07.685 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:07.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:07 smithi002 ceph-mon[30563]: pgmap v416: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-10T08:09:07.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:07 smithi155 ceph-mon[35405]: pgmap v416: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-10T08:09:07.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:07 smithi107 ceph-mon[35048]: pgmap v416: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-10T08:09:08.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.311+0000 7fc1bca00640 1 -- 172.21.15.2:0/1136668367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b8108f60 msgr2=0x7fc1b8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:08.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.311+0000 7fc1bca00640 1 --2- 172.21.15.2:0/1136668367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b8108f60 0x7fc1b8110ae0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009ca0 tx=0x7fc1b002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:09:08.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.311+0000 7fc1bca00640 1 -- 172.21.15.2:0/1136668367 shutdown_connections 2024-09-10T08:09:08.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.311+0000 7fc1bca00640 1 --2- 172.21.15.2:0/1136668367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b8108f60 0x7fc1b8110ae0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:08.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.311+0000 7fc1bca00640 1 --2- 172.21.15.2:0/1136668367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1b81047e0 0x7fc1b8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:08.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.311+0000 7fc1bca00640 1 --2- 172.21.15.2:0/1136668367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1b8103e30 0x7fc1b8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:08.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.311+0000 7fc1bca00640 1 -- 172.21.15.2:0/1136668367 >> 172.21.15.2:0/1136668367 conn(0x7fc1b80ff910 msgr2=0x7fc1b8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:08.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.312+0000 7fc1bca00640 1 -- 172.21.15.2:0/1136668367 shutdown_connections 2024-09-10T08:09:08.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.312+0000 7fc1bca00640 1 -- 172.21.15.2:0/1136668367 wait complete. 2024-09-10T08:09:08.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.313+0000 7fc1bca00640 1 Processor -- start 2024-09-10T08:09:08.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.313+0000 7fc1bca00640 1 -- start start 2024-09-10T08:09:08.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.313+0000 7fc1bca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1b8103e30 0x7fc1b819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:08.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.313+0000 7fc1bca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b81047e0 0x7fc1b819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:08.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.313+0000 7fc1bca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1b8108f60 0x7fc1b819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:08.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.313+0000 7fc1bca00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b8114010 con 0x7fc1b81047e0 2024-09-10T08:09:08.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.313+0000 7fc1bca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b8113e90 con 0x7fc1b8108f60 2024-09-10T08:09:08.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.313+0000 7fc1bca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b8114190 con 0x7fc1b8103e30 2024-09-10T08:09:08.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.314+0000 7fc1b7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1b8103e30 0x7fc1b819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:08.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.314+0000 7fc1b7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1b8103e30 0x7fc1b819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49684/0 (socket says 172.21.15.2:49684) 2024-09-10T08:09:08.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.314+0000 7fc1b7400640 1 -- 172.21.15.2:0/209299699 learned_addr learned my addr 172.21.15.2:0/209299699 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:08.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.314+0000 7fc1b6a00640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b81047e0 0x7fc1b819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:08.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.314+0000 7fc1b7e00640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1b8108f60 0x7fc1b819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:08.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.314+0000 7fc1b7400640 1 -- 172.21.15.2:0/209299699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1b8108f60 msgr2=0x7fc1b819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:08.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.314+0000 7fc1b7400640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1b8108f60 0x7fc1b819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:08.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.314+0000 7fc1b7400640 1 -- 172.21.15.2:0/209299699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b81047e0 msgr2=0x7fc1b819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:08.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.314+0000 7fc1b7400640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b81047e0 0x7fc1b819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:08.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.314+0000 7fc1b7400640 1 -- 172.21.15.2:0/209299699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b819f3b0 con 0x7fc1b8103e30 2024-09-10T08:09:08.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.315+0000 7fc1b7400640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1b8103e30 0x7fc1b819a400 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800e9f0 tx=0x7fc1a800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:08.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.315+0000 7fc1afe00640 1 -- 172.21.15.2:0/209299699 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a800cd80 con 0x7fc1b8103e30 2024-09-10T08:09:08.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.315+0000 7fc1afe00640 1 -- 172.21.15.2:0/209299699 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a8005500 con 0x7fc1b8103e30 2024-09-10T08:09:08.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.315+0000 7fc1afe00640 1 -- 172.21.15.2:0/209299699 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8010690 con 0x7fc1b8103e30 2024-09-10T08:09:08.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.316+0000 7fc1bca00640 1 -- 172.21.15.2:0/209299699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b819f6a0 con 0x7fc1b8103e30 2024-09-10T08:09:08.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.316+0000 7fc1bca00640 1 -- 172.21.15.2:0/209299699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b81a6f80 con 0x7fc1b8103e30 2024-09-10T08:09:08.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.317+0000 7fc1bca00640 1 -- 172.21.15.2:0/209299699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b8105310 con 0x7fc1b8103e30 2024-09-10T08:09:08.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.321+0000 7fc1afe00640 1 -- 172.21.15.2:0/209299699 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fc1a80046c0 con 0x7fc1b8103e30 2024-09-10T08:09:08.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.322+0000 7fc1afe00640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc19c0a2c90 0x7fc19c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:08.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.323+0000 7fc1afe00640 1 -- 172.21.15.2:0/209299699 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc1a80cfba0 con 0x7fc1b8103e30 2024-09-10T08:09:08.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.323+0000 7fc1b6a00640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc19c0a2c90 0x7fc19c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:08.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.323+0000 7fc1afe00640 1 -- 172.21.15.2:0/209299699 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1a809c550 con 0x7fc1b8103e30 2024-09-10T08:09:08.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.324+0000 7fc1b6a00640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc19c0a2c90 0x7fc19c0a5150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40095f0 tx=0x7fc1a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:08.495 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T08:09:08.495 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T08:09:08.495 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:09:08.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.598+0000 7fc1bca00640 1 -- 172.21.15.2:0/209299699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1b8104210 con 0x7fc1b8103e30 2024-09-10T08:09:08.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.599+0000 7fc1afe00640 1 -- 172.21.15.2:0/209299699 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fc1a809be50 con 0x7fc1b8103e30 2024-09-10T08:09:08.599 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:09:08.599 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:09:08.600 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:09:08.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.602+0000 7fc1bca00640 1 -- 172.21.15.2:0/209299699 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc19c0a2c90 msgr2=0x7fc19c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:08.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.602+0000 7fc1bca00640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc19c0a2c90 0x7fc19c0a5150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40095f0 tx=0x7fc1a4005f90 comp rx=0 tx=0).stop 2024-09-10T08:09:08.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.602+0000 7fc1bca00640 1 -- 172.21.15.2:0/209299699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1b8103e30 msgr2=0x7fc1b819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:08.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.602+0000 7fc1bca00640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1b8103e30 0x7fc1b819a400 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800e9f0 tx=0x7fc1a800edf0 comp rx=0 tx=0).stop 2024-09-10T08:09:08.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.602+0000 7fc1bca00640 1 -- 172.21.15.2:0/209299699 shutdown_connections 2024-09-10T08:09:08.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.602+0000 7fc1bca00640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1b8108f60 0x7fc1b819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:08.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.602+0000 7fc1bca00640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc1b81047e0 0x7fc1b819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:08.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.602+0000 7fc1bca00640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc19c0a2c90 0x7fc19c0a5150 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:08.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.602+0000 7fc1bca00640 1 --2- 172.21.15.2:0/209299699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1b8103e30 0x7fc1b819a400 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:08.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.602+0000 7fc1bca00640 1 -- 172.21.15.2:0/209299699 >> 172.21.15.2:0/209299699 conn(0x7fc1b80ff910 msgr2=0x7fc1b8100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:08.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.602+0000 7fc1bca00640 1 -- 172.21.15.2:0/209299699 shutdown_connections 2024-09-10T08:09:08.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:08.602+0000 7fc1bca00640 1 -- 172.21.15.2:0/209299699 wait complete. 2024-09-10T08:09:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.410+0000 7f1941000640 1 -- 172.21.15.2:0/2898079119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f193c101580 msgr2=0x7f193c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.410+0000 7f1941000640 1 --2- 172.21.15.2:0/2898079119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f193c101580 0x7f193c101a00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1928009900 tx=0x7f192802a400 comp rx=0 tx=0).stop 2024-09-10T08:09:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.413+0000 7f1941000640 1 -- 172.21.15.2:0/2898079119 shutdown_connections 2024-09-10T08:09:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.413+0000 7f1941000640 1 --2- 172.21.15.2:0/2898079119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f193c101f40 0x7f193c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.413+0000 7f1941000640 1 --2- 172.21.15.2:0/2898079119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f193c101580 0x7f193c101a00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.413+0000 7f1941000640 1 --2- 172.21.15.2:0/2898079119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f193c103050 0x7f193c103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.413+0000 7f1941000640 1 -- 172.21.15.2:0/2898079119 >> 172.21.15.2:0/2898079119 conn(0x7f193c0fd0d0 msgr2=0x7f193c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.413+0000 7f1941000640 1 -- 172.21.15.2:0/2898079119 shutdown_connections 2024-09-10T08:09:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.413+0000 7f1941000640 1 -- 172.21.15.2:0/2898079119 wait complete. 2024-09-10T08:09:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.414+0000 7f1941000640 1 Processor -- start 2024-09-10T08:09:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.415+0000 7f1941000640 1 -- start start 2024-09-10T08:09:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.415+0000 7f1941000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f193c101580 0x7f193c070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.415+0000 7f1941000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f193c101f40 0x7f193c071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.415+0000 7f1941000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f193c103050 0x7f193c1a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.415+0000 7f1941000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f193c10b570 con 0x7f193c101f40 2024-09-10T08:09:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.415+0000 7f1941000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f193c10b3f0 con 0x7f193c101580 2024-09-10T08:09:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.415+0000 7f1941000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f193c10b6f0 con 0x7f193c103050 2024-09-10T08:09:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.416+0000 7f193b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f193c101580 0x7f193c070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.416+0000 7f193b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f193c101580 0x7f193c070bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37980/0 (socket says 172.21.15.2:37980) 2024-09-10T08:09:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.416+0000 7f193b400640 1 -- 172.21.15.2:0/412052885 learned_addr learned my addr 172.21.15.2:0/412052885 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.416+0000 7f193aa00640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f193c101f40 0x7f193c071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.416+0000 7f193be00640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f193c103050 0x7f193c1a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.416+0000 7f193b400640 1 -- 172.21.15.2:0/412052885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f193c103050 msgr2=0x7f193c1a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.416+0000 7f193b400640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f193c103050 0x7f193c1a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.416+0000 7f193b400640 1 -- 172.21.15.2:0/412052885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f193c101f40 msgr2=0x7f193c071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.417+0000 7f193b400640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f193c101f40 0x7f193c071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.417+0000 7f193b400640 1 -- 172.21.15.2:0/412052885 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c1a6160 con 0x7f193c101580 2024-09-10T08:09:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.417+0000 7f193be00640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f193c103050 0x7f193c1a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:09:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.417+0000 7f193aa00640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f193c101f40 0x7f193c071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.417+0000 7f193b400640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f193c101580 0x7f193c070bc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f192c00e9c0 tx=0x7f192c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.417+0000 7f1933e00640 1 -- 172.21.15.2:0/412052885 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c00cde0 con 0x7f193c101580 2024-09-10T08:09:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.417+0000 7f1933e00640 1 -- 172.21.15.2:0/412052885 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f192c010910 con 0x7f193c101580 2024-09-10T08:09:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.417+0000 7f1933e00640 1 -- 172.21.15.2:0/412052885 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c018a00 con 0x7f193c101580 2024-09-10T08:09:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.418+0000 7f1941000640 1 -- 172.21.15.2:0/412052885 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193c1a6330 con 0x7f193c101580 2024-09-10T08:09:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.418+0000 7f1941000640 1 -- 172.21.15.2:0/412052885 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193c1a67a0 con 0x7f193c101580 2024-09-10T08:09:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.423+0000 7f1945489740 1 Processor -- start 2024-09-10T08:09:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.424+0000 7f1945489740 1 -- start start 2024-09-10T08:09:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.424+0000 7f1933e00640 1 -- 172.21.15.2:0/412052885 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f192c01f020 con 0x7f193c101580 2024-09-10T08:09:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.424+0000 7f1945489740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3b650 0x55936bb3ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.424+0000 7f1945489740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb3bf90 0x55936bb40a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.424+0000 7f1945489740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb41070 0x55936bb43530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.424+0000 7f1945489740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55936baa5100 con 0x55936bb3bf90 2024-09-10T08:09:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.424+0000 7f1945489740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55936ba4d1b0 con 0x55936bb41070 2024-09-10T08:09:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.424+0000 7f1945489740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55936baa10c0 con 0x55936bb3b650 2024-09-10T08:09:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.424+0000 7f193b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb3bf90 0x55936bb40a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.425+0000 7f193b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb3bf90 0x55936bb40a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45120/0 (socket says 172.21.15.2:45120) 2024-09-10T08:09:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.425+0000 7f193b400640 1 -- 172.21.15.2:0/3037081134 learned_addr learned my addr 172.21.15.2:0/3037081134 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.425+0000 7f193be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3b650 0x55936bb3ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.425+0000 7f193be00640 1 -- 172.21.15.2:0/3037081134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb41070 msgr2=0x55936bb43530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:09:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.425+0000 7f193be00640 1 --2- 172.21.15.2:0/3037081134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb41070 0x55936bb43530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.425+0000 7f193be00640 1 -- 172.21.15.2:0/3037081134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb3bf90 msgr2=0x55936bb40a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.425+0000 7f193be00640 1 --2- 172.21.15.2:0/3037081134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb3bf90 0x55936bb40a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.425+0000 7f193be00640 1 -- 172.21.15.2:0/3037081134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55936b7b8210 con 0x55936bb3b650 2024-09-10T08:09:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.425+0000 7f193b400640 1 --2- 172.21.15.2:0/3037081134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb3bf90 0x55936bb40a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.426+0000 7f193be00640 1 --2- 172.21.15.2:0/3037081134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3b650 0x55936bb3ba50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1934009ec0 tx=0x7f193400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30f2f255783c82b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.426+0000 7f1930c00640 1 -- 172.21.15.2:0/3037081134 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934003b70 con 0x55936bb3b650 2024-09-10T08:09:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.426+0000 7f1933e00640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19200a2c80 0x7f19200a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.426+0000 7f1930c00640 1 -- 172.21.15.2:0/3037081134 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1934003d10 con 0x55936bb3b650 2024-09-10T08:09:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.426+0000 7f1930c00640 1 -- 172.21.15.2:0/3037081134 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934011a30 con 0x55936bb3b650 2024-09-10T08:09:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.426+0000 7f1933e00640 1 -- 172.21.15.2:0/412052885 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f192c0cf7e0 con 0x7f193c101580 2024-09-10T08:09:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.427+0000 7f193b400640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19200a2c80 0x7f19200a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.427+0000 7f1945489740 1 -- 172.21.15.2:0/3037081134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3b650 msgr2=0x55936bb3ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.427+0000 7f1945489740 1 --2- 172.21.15.2:0/3037081134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3b650 0x55936bb3ba50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1934009ec0 tx=0x7f193400f730 comp rx=0 tx=0).stop 2024-09-10T08:09:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.427+0000 7f1945489740 1 -- 172.21.15.2:0/3037081134 shutdown_connections 2024-09-10T08:09:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.427+0000 7f1945489740 1 --2- 172.21.15.2:0/3037081134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb41070 0x55936bb43530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.427+0000 7f1945489740 1 --2- 172.21.15.2:0/3037081134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb3bf90 0x55936bb40a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.427+0000 7f1945489740 1 --2- 172.21.15.2:0/3037081134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3b650 0x55936bb3ba50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.427+0000 7f1945489740 1 -- 172.21.15.2:0/3037081134 >> 172.21.15.2:0/3037081134 conn(0x55936bb440e0 msgr2=0x55936bb444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.428+0000 7f1945489740 1 -- 172.21.15.2:0/3037081134 shutdown_connections 2024-09-10T08:09:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.428+0000 7f193b400640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19200a2c80 0x7f19200a5140 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f192c000c00 tx=0x7f192c0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.428+0000 7f1945489740 1 -- 172.21.15.2:0/3037081134 wait complete. 2024-09-10T08:09:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.429+0000 7f1945489740 1 -- start start 2024-09-10T08:09:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.429+0000 7f1945489740 1 Processor -- start 2024-09-10T08:09:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.429+0000 7f1945489740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3b650 0x55936bb5b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.430+0000 7f1945489740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3bf90 0x55936bb5b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.430+0000 7f1945489740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb41070 0x55936bb629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.430+0000 7f1945489740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55936bb59c30 con 0x55936bb41070 2024-09-10T08:09:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.430+0000 7f1945489740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55936ba2e090 con 0x55936bb3b650 2024-09-10T08:09:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.430+0000 7f1945489740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55936bb59db0 con 0x55936bb3bf90 2024-09-10T08:09:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.430+0000 7f1945489740 10 client.0 _ll_register_callbacks cb 0x55936ba82f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.430+0000 7f193be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3b650 0x55936bb5b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.431+0000 7f193be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3b650 0x55936bb5b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37982/0 (socket says 172.21.15.2:37982) 2024-09-10T08:09:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.431+0000 7f193be00640 1 -- 172.21.15.2:0/547382742 learned_addr learned my addr 172.21.15.2:0/547382742 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.431+0000 7f193be00640 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb41070 0x55936bb629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.431+0000 7f193aa00640 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3bf90 0x55936bb5b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.431+0000 7f193aa00640 1 -- 172.21.15.2:0/547382742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3b650 msgr2=0x55936bb5b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.431+0000 7f193aa00640 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3b650 0x55936bb5b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.431+0000 7f193aa00640 1 -- 172.21.15.2:0/547382742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb41070 msgr2=0x55936bb629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.432+0000 7f193aa00640 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb41070 0x55936bb629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.432+0000 7f193aa00640 1 -- 172.21.15.2:0/547382742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55936b884620 con 0x55936bb3bf90 2024-09-10T08:09:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.432+0000 7f193be00640 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb41070 0x55936bb629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.432+0000 7f193aa00640 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3bf90 0x55936bb5b960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f192802a920 tx=0x7f1928009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=762d711b6a0585ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.432+0000 7f1925600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:09:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.433+0000 7f1925600640 1 -- 172.21.15.2:0/547382742 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928035040 con 0x55936bb3bf90 2024-09-10T08:09:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.433+0000 7f1925600640 1 -- 172.21.15.2:0/547382742 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1928004770 con 0x55936bb3bf90 2024-09-10T08:09:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.433+0000 7f1945489740 1 -- 172.21.15.2:0/547382742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1934003d10 con 0x55936bb3bf90 2024-09-10T08:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.433+0000 7f1925600640 1 -- 172.21.15.2:0/547382742 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1928005610 con 0x55936bb3bf90 2024-09-10T08:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.434+0000 7f1945489740 10 client.25396 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.434+0000 7f1945489740 1 -- 172.21.15.2:0/547382742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55936bb65800 con 0x55936bb3bf90 2024-09-10T08:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.434+0000 7f1925600640 1 -- 172.21.15.2:0/547382742 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1934003d10 con 0x55936bb3bf90 2024-09-10T08:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.434+0000 7f1925600640 1 -- 172.21.15.2:0/547382742 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f19280035f0 con 0x55936bb3bf90 2024-09-10T08:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.435+0000 7f1945489740 10 client.25396 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.435+0000 7f1945489740 10 client.25396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.435+0000 7f1945489740 20 client.25396 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.435+0000 7f1945489740 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55936bb66ed0 0x55936bb692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.435+0000 7f1945489740 4 client.25396 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.435+0000 7f1945489740 1 -- 172.21.15.2:0/547382742 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55936bab2830 con 0x55936bb66ed0 2024-09-10T08:09:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.436+0000 7f193be00640 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55936bb66ed0 0x55936bb692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.436+0000 7f193be00640 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55936bb66ed0 0x55936bb692c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=532ccd9a782ada13 server_cookie=6a803dc468af1dca in_seq=0 out_seq=0 2024-09-10T08:09:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.436+0000 7f1925600640 10 client.25396 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:09.438 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:09:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.437+0000 7f1925600640 1 -- 172.21.15.2:0/547382742 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55936bab2830 con 0x55936bb66ed0 2024-09-10T08:09:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.437+0000 7f1925600640 10 client.25396 handle_command_reply: tid=0 2024-09-10T08:09:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.437+0000 7f1945489740 1 client.25396 shutdown 2024-09-10T08:09:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.437+0000 7f1945489740 1 -- 172.21.15.2:0/547382742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3bf90 msgr2=0x55936bb5b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.437+0000 7f1945489740 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3bf90 0x55936bb5b960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f192802a920 tx=0x7f1928009630 comp rx=0 tx=0).stop 2024-09-10T08:09:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.438+0000 7f1945489740 1 -- 172.21.15.2:0/547382742 shutdown_connections 2024-09-10T08:09:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.438+0000 7f1945489740 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55936bb66ed0 0x55936bb692c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.438+0000 7f1945489740 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb41070 0x55936bb629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.438+0000 7f1945489740 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3bf90 0x55936bb5b960 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.438+0000 7f1925600640 0 client.25396 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.438+0000 7f1945489740 1 --2- 172.21.15.2:0/547382742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3b650 0x55936bb5b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.438+0000 7f1945489740 1 -- 172.21.15.2:0/547382742 >> 172.21.15.2:0/547382742 conn(0x55936bb440e0 msgr2=0x55936bb458f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.438+0000 7f1945489740 1 -- 172.21.15.2:0/547382742 shutdown_connections 2024-09-10T08:09:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.438+0000 7f1945489740 1 -- 172.21.15.2:0/547382742 wait complete. 2024-09-10T08:09:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.438+0000 7f1945489740 20 client.25396 trim_cache size 0 max 16384 2024-09-10T08:09:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.469+0000 7f1945489740 1 Processor -- start 2024-09-10T08:09:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.469+0000 7f1945489740 1 -- start start 2024-09-10T08:09:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.469+0000 7f1945489740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb3b650 0x55936bb444c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.469+0000 7f1945489740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3bf90 0x55936bb62960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.469+0000 7f1945489740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb41070 0x55936bb44a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.469+0000 7f1945489740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1b3700 con 0x55936bb3b650 2024-09-10T08:09:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.469+0000 7f1945489740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1b0fd0 con 0x55936bb3bf90 2024-09-10T08:09:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.469+0000 7f1945489740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1aee20 con 0x55936bb41070 2024-09-10T08:09:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.470+0000 7f193be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb41070 0x55936bb44a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.470+0000 7f193aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3bf90 0x55936bb62960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.470+0000 7f193be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb41070 0x55936bb44a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49746/0 (socket says 172.21.15.2:49746) 2024-09-10T08:09:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.470+0000 7f193be00640 1 -- 172.21.15.2:0/1286941457 learned_addr learned my addr 172.21.15.2:0/1286941457 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.471+0000 7f193be00640 1 --2- 172.21.15.2:0/1286941457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb3b650 0x55936bb444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.471+0000 7f193aa00640 1 -- 172.21.15.2:0/1286941457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb41070 msgr2=0x55936bb44a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.471+0000 7f193aa00640 1 --2- 172.21.15.2:0/1286941457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb41070 0x55936bb44a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.471+0000 7f193aa00640 1 -- 172.21.15.2:0/1286941457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb3b650 msgr2=0x55936bb444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.471+0000 7f193aa00640 1 --2- 172.21.15.2:0/1286941457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb3b650 0x55936bb444c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.471+0000 7f193aa00640 1 -- 172.21.15.2:0/1286941457 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55936bb59f50 con 0x55936bb3bf90 2024-09-10T08:09:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.471+0000 7f193be00640 1 --2- 172.21.15.2:0/1286941457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb3b650 0x55936bb444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:09:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.471+0000 7f193aa00640 1 --2- 172.21.15.2:0/1286941457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3bf90 0x55936bb62960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19280314e0 tx=0x7f1928005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e6830497ad5ef5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.471+0000 7f1927e00640 1 -- 172.21.15.2:0/1286941457 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928035070 con 0x55936bb3bf90 2024-09-10T08:09:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.472+0000 7f1927e00640 1 -- 172.21.15.2:0/1286941457 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19280035f0 con 0x55936bb3bf90 2024-09-10T08:09:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.472+0000 7f1927e00640 1 -- 172.21.15.2:0/1286941457 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19280049d0 con 0x55936bb3bf90 2024-09-10T08:09:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.473+0000 7f1945489740 1 -- 172.21.15.2:0/1286941457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3bf90 msgr2=0x55936bb62960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.473+0000 7f1945489740 1 --2- 172.21.15.2:0/1286941457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3bf90 0x55936bb62960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19280314e0 tx=0x7f1928005fd0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.473+0000 7f1945489740 1 -- 172.21.15.2:0/1286941457 shutdown_connections 2024-09-10T08:09:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.473+0000 7f1945489740 1 --2- 172.21.15.2:0/1286941457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb41070 0x55936bb44a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.473+0000 7f1945489740 1 --2- 172.21.15.2:0/1286941457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3bf90 0x55936bb62960 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.473+0000 7f1945489740 1 --2- 172.21.15.2:0/1286941457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb3b650 0x55936bb444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.473+0000 7f1945489740 1 -- 172.21.15.2:0/1286941457 >> 172.21.15.2:0/1286941457 conn(0x55936bb440e0 msgr2=0x55936bb40790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.473+0000 7f1945489740 1 -- 172.21.15.2:0/1286941457 shutdown_connections 2024-09-10T08:09:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.473+0000 7f1945489740 1 -- 172.21.15.2:0/1286941457 wait complete. 2024-09-10T08:09:09.474 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.474+0000 7f1945489740 1 -- start start 2024-09-10T08:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.474+0000 7f1945489740 1 Processor -- start 2024-09-10T08:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.475+0000 7f1945489740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3b650 0x55936bb5fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.475+0000 7f1945489740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3bf90 0x55936bb60040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.475+0000 7f1945489740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb41070 0x55936bb61000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.475+0000 7f1945489740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1af490 con 0x55936bb41070 2024-09-10T08:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.475+0000 7f1945489740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1b0900 con 0x55936bb3b650 2024-09-10T08:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.475+0000 7f1945489740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1afb60 con 0x55936bb3bf90 2024-09-10T08:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.475+0000 7f1945489740 10 client.0 _ll_register_callbacks cb 0x7f19340093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.475+0000 7f193be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb41070 0x55936bb61000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.475+0000 7f193be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3b650 0x55936bb5fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.476+0000 7f193be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3b650 0x55936bb5fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37996/0 (socket says 172.21.15.2:37996) 2024-09-10T08:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.476+0000 7f193be00640 1 -- 172.21.15.2:0/200243788 learned_addr learned my addr 172.21.15.2:0/200243788 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.476+0000 7f193aa00640 1 --2- 172.21.15.2:0/200243788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3bf90 0x55936bb60040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.476+0000 7f193be00640 1 -- 172.21.15.2:0/200243788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3bf90 msgr2=0x55936bb60040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.476+0000 7f193be00640 1 --2- 172.21.15.2:0/200243788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3bf90 0x55936bb60040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.476+0000 7f193be00640 1 -- 172.21.15.2:0/200243788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb41070 msgr2=0x55936bb61000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.476+0000 7f193be00640 1 --2- 172.21.15.2:0/200243788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb41070 0x55936bb61000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.476+0000 7f193be00640 1 -- 172.21.15.2:0/200243788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55936bb3cc20 con 0x55936bb3b650 2024-09-10T08:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.476+0000 7f193be00640 1 --2- 172.21.15.2:0/200243788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3b650 0x55936bb5fb00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f19340099c0 tx=0x7f1934004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f45888bd6f944c0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.476+0000 7f1925600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.477+0000 7f1925600640 1 -- 172.21.15.2:0/200243788 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934024040 con 0x55936bb3b650 2024-09-10T08:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.477+0000 7f1925600640 1 -- 172.21.15.2:0/200243788 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1934004c00 con 0x55936bb3b650 2024-09-10T08:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.477+0000 7f1945489740 1 -- 172.21.15.2:0/200243788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19280035f0 con 0x55936bb3b650 2024-09-10T08:09:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.477+0000 7f1925600640 1 -- 172.21.15.2:0/200243788 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f193402c440 con 0x55936bb3b650 2024-09-10T08:09:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.477+0000 7f1945489740 10 client.25430 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.477+0000 7f1945489740 1 -- 172.21.15.2:0/200243788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55936bb62720 con 0x55936bb3b650 2024-09-10T08:09:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.478+0000 7f1925600640 1 -- 172.21.15.2:0/200243788 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19280035f0 con 0x55936bb3b650 2024-09-10T08:09:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.478+0000 7f1925600640 1 -- 172.21.15.2:0/200243788 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f19340119e0 con 0x55936bb3b650 2024-09-10T08:09:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.478+0000 7f1945489740 10 client.25430 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.478+0000 7f1945489740 10 client.25430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.478+0000 7f1945489740 20 client.25430 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.478+0000 7f1945489740 1 --2- 172.21.15.2:0/200243788 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55936bbd1b80 0x55936bbd3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.478+0000 7f1945489740 4 client.25430 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:09:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.478+0000 7f1945489740 1 -- 172.21.15.2:0/200243788 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55936baacb80 con 0x55936bbd1b80 2024-09-10T08:09:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.478+0000 7f193aa00640 1 --2- 172.21.15.2:0/200243788 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55936bbd1b80 0x55936bbd3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.479+0000 7f193aa00640 1 --2- 172.21.15.2:0/200243788 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55936bbd1b80 0x55936bbd3f70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2783b90253c02c68 server_cookie=c9f01327b9361f8b in_seq=0 out_seq=0 2024-09-10T08:09:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.479+0000 7f1925600640 10 client.25430 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1925600640 1 -- 172.21.15.2:0/200243788 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55936baacb80 con 0x55936bbd1b80 2024-09-10T08:09:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1925600640 10 client.25430 handle_command_reply: tid=0 2024-09-10T08:09:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1945489740 1 client.25430 shutdown 2024-09-10T08:09:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1945489740 1 -- 172.21.15.2:0/200243788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3b650 msgr2=0x55936bb5fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1945489740 1 --2- 172.21.15.2:0/200243788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3b650 0x55936bb5fb00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f19340099c0 tx=0x7f1934004990 comp rx=0 tx=0).stop 2024-09-10T08:09:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1945489740 1 -- 172.21.15.2:0/200243788 shutdown_connections 2024-09-10T08:09:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1945489740 1 --2- 172.21.15.2:0/200243788 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55936bbd1b80 0x55936bbd3f70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1945489740 1 --2- 172.21.15.2:0/200243788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55936bb41070 0x55936bb61000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1945489740 1 --2- 172.21.15.2:0/200243788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55936bb3bf90 0x55936bb60040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1945489740 1 --2- 172.21.15.2:0/200243788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55936bb3b650 0x55936bb5fb00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1945489740 1 -- 172.21.15.2:0/200243788 >> 172.21.15.2:0/200243788 conn(0x55936bb440e0 msgr2=0x55936bb3a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1925600640 0 client.25430 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1945489740 1 -- 172.21.15.2:0/200243788 shutdown_connections 2024-09-10T08:09:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.486+0000 7f1945489740 1 -- 172.21.15.2:0/200243788 wait complete. 2024-09-10T08:09:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.487+0000 7f1945489740 20 client.25430 trim_cache size 0 max 16384 2024-09-10T08:09:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:09:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:09:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:09:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.487+0000 7f1941000640 1 -- 172.21.15.2:0/412052885 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19200a2c80 msgr2=0x7f19200a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.487+0000 7f1941000640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19200a2c80 0x7f19200a5140 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f192c000c00 tx=0x7f192c0d5530 comp rx=0 tx=0).stop 2024-09-10T08:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.487+0000 7f1941000640 1 -- 172.21.15.2:0/412052885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f193c101580 msgr2=0x7f193c070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.487+0000 7f1941000640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f193c101580 0x7f193c070bc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f192c00e9c0 tx=0x7f192c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.488+0000 7f1941000640 1 -- 172.21.15.2:0/412052885 shutdown_connections 2024-09-10T08:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.488+0000 7f1941000640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19200a2c80 0x7f19200a5140 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.488+0000 7f1941000640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f193c103050 0x7f193c1a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.488+0000 7f1941000640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f193c101f40 0x7f193c071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.488+0000 7f1941000640 1 --2- 172.21.15.2:0/412052885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f193c101580 0x7f193c070bc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.488+0000 7f1941000640 1 -- 172.21.15.2:0/412052885 >> 172.21.15.2:0/412052885 conn(0x7f193c0fd0d0 msgr2=0x7f193c105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.488+0000 7f1941000640 1 -- 172.21.15.2:0/412052885 shutdown_connections 2024-09-10T08:09:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:09.488+0000 7f1941000640 1 -- 172.21.15.2:0/412052885 wait complete. 2024-09-10T08:09:09.635 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:09 smithi002 ceph-mon[30563]: pgmap v417: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-10T08:09:09.635 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:09 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/209299699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:09.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:09 smithi155 ceph-mon[35405]: pgmap v417: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-10T08:09:09.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:09 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/209299699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:09.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:09 smithi107 ceph-mon[35048]: pgmap v417: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-10T08:09:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:09 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/209299699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:10.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.180+0000 7f72c9200640 1 -- 172.21.15.2:0/603259526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c4104010 msgr2=0x7f72c41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:10.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.180+0000 7f72c9200640 1 --2- 172.21.15.2:0/603259526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c4104010 0x7f72c41043f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009900 tx=0x7f72b402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:09:10.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.181+0000 7f72c9200640 1 -- 172.21.15.2:0/603259526 shutdown_connections 2024-09-10T08:09:10.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.181+0000 7f72c9200640 1 --2- 172.21.15.2:0/603259526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c4109140 0x7f72c4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:10.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.181+0000 7f72c9200640 1 --2- 172.21.15.2:0/603259526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72c41049c0 0x7f72c4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:10.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.181+0000 7f72c9200640 1 --2- 172.21.15.2:0/603259526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c4104010 0x7f72c41043f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:10.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.181+0000 7f72c9200640 1 -- 172.21.15.2:0/603259526 >> 172.21.15.2:0/603259526 conn(0x7f72c40ffa50 msgr2=0x7f72c4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:10.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.181+0000 7f72c9200640 1 -- 172.21.15.2:0/603259526 shutdown_connections 2024-09-10T08:09:10.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.181+0000 7f72c9200640 1 -- 172.21.15.2:0/603259526 wait complete. 2024-09-10T08:09:10.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.182+0000 7f72c9200640 1 Processor -- start 2024-09-10T08:09:10.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.182+0000 7f72c9200640 1 -- start start 2024-09-10T08:09:10.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.182+0000 7f72c9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72c4104010 0x7f72c419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:10.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.182+0000 7f72c9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c41049c0 0x7f72c419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:10.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.182+0000 7f72c9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c4109140 0x7f72c419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:10.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.182+0000 7f72c9200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4114170 con 0x7f72c4104010 2024-09-10T08:09:10.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.182+0000 7f72c9200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4113ff0 con 0x7f72c41049c0 2024-09-10T08:09:10.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.182+0000 7f72c9200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72c41142f0 con 0x7f72c4109140 2024-09-10T08:09:10.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.183+0000 7f72c1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72c4104010 0x7f72c419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:10.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.183+0000 7f72c1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72c4104010 0x7f72c419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45144/0 (socket says 172.21.15.2:45144) 2024-09-10T08:09:10.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.183+0000 7f72c1e00640 1 -- 172.21.15.2:0/2097707330 learned_addr learned my addr 172.21.15.2:0/2097707330 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:10.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.183+0000 7f72c2800640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c4109140 0x7f72c419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:10.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.184+0000 7f72c1400640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c41049c0 0x7f72c419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:10.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.184+0000 7f72c1e00640 1 -- 172.21.15.2:0/2097707330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c4109140 msgr2=0x7f72c419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:10.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.184+0000 7f72c1e00640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c4109140 0x7f72c419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:10.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.184+0000 7f72c1e00640 1 -- 172.21.15.2:0/2097707330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c41049c0 msgr2=0x7f72c419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:10.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.184+0000 7f72c1e00640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c41049c0 0x7f72c419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:10.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.184+0000 7f72c1e00640 1 -- 172.21.15.2:0/2097707330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c419f5e0 con 0x7f72c4104010 2024-09-10T08:09:10.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.184+0000 7f72c1e00640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72c4104010 0x7f72c419a630 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f72b4006010 tx=0x7f72b4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:10.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.184+0000 7f72baa00640 1 -- 172.21.15.2:0/2097707330 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b403f030 con 0x7f72c4104010 2024-09-10T08:09:10.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.185+0000 7f72baa00640 1 -- 172.21.15.2:0/2097707330 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b4004440 con 0x7f72c4104010 2024-09-10T08:09:10.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.185+0000 7f72c9200640 1 -- 172.21.15.2:0/2097707330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c419f870 con 0x7f72c4104010 2024-09-10T08:09:10.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.185+0000 7f72c9200640 1 -- 172.21.15.2:0/2097707330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c41a71b0 con 0x7f72c4104010 2024-09-10T08:09:10.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.185+0000 7f72baa00640 1 -- 172.21.15.2:0/2097707330 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4035070 con 0x7f72c4104010 2024-09-10T08:09:10.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.186+0000 7f72c9200640 1 -- 172.21.15.2:0/2097707330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c41043f0 con 0x7f72c4104010 2024-09-10T08:09:10.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.190+0000 7f72baa00640 1 -- 172.21.15.2:0/2097707330 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f72b4042030 con 0x7f72c4104010 2024-09-10T08:09:10.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.192+0000 7f72baa00640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f72a80a2c10 0x7f72a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:10.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.192+0000 7f72baa00640 1 -- 172.21.15.2:0/2097707330 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f72b40efa50 con 0x7f72c4104010 2024-09-10T08:09:10.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.193+0000 7f72c1400640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f72a80a2c10 0x7f72a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:10.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.194+0000 7f72c1400640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f72a80a2c10 0x7f72a80a50d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f72c40ffe70 tx=0x7f72b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:10.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.197+0000 7f72baa00640 1 -- 172.21.15.2:0/2097707330 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f72b4049030 con 0x7f72c4104010 2024-09-10T08:09:10.218 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:10.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.433+0000 7f72c9200640 1 -- 172.21.15.2:0/2097707330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72c410c320 con 0x7f72c4104010 2024-09-10T08:09:10.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.434+0000 7f72baa00640 1 -- 172.21.15.2:0/2097707330 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f72b40bc400 con 0x7f72c4104010 2024-09-10T08:09:10.434 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:09:10.434 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:09:10.434 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:09:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.436+0000 7f72c9200640 1 -- 172.21.15.2:0/2097707330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f72a80a2c10 msgr2=0x7f72a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.436+0000 7f72c9200640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f72a80a2c10 0x7f72a80a50d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f72c40ffe70 tx=0x7f72b0005f90 comp rx=0 tx=0).stop 2024-09-10T08:09:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.436+0000 7f72c9200640 1 -- 172.21.15.2:0/2097707330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72c4104010 msgr2=0x7f72c419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.436+0000 7f72c9200640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72c4104010 0x7f72c419a630 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f72b4006010 tx=0x7f72b4004040 comp rx=0 tx=0).stop 2024-09-10T08:09:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.436+0000 7f72c9200640 1 -- 172.21.15.2:0/2097707330 shutdown_connections 2024-09-10T08:09:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.436+0000 7f72c9200640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f72a80a2c10 0x7f72a80a50d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.436+0000 7f72c9200640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c4109140 0x7f72c419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.436+0000 7f72c9200640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c41049c0 0x7f72c419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.436+0000 7f72c9200640 1 --2- 172.21.15.2:0/2097707330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72c4104010 0x7f72c419a630 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.437+0000 7f72c9200640 1 -- 172.21.15.2:0/2097707330 >> 172.21.15.2:0/2097707330 conn(0x7f72c40ffa50 msgr2=0x7f72c4100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.437+0000 7f72c9200640 1 -- 172.21.15.2:0/2097707330 shutdown_connections 2024-09-10T08:09:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:10.437+0000 7f72c9200640 1 -- 172.21.15.2:0/2097707330 wait complete. 2024-09-10T08:09:10.624 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:10 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2097707330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:10.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:10 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2097707330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:10.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:10 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2097707330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:11.263 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T08:09:11.264 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T08:09:11.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.536+0000 7f2dc1000640 1 -- 172.21.15.2:0/417911479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dbc103e20 msgr2=0x7f2dbc104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:11.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.536+0000 7f2dc1000640 1 --2- 172.21.15.2:0/417911479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dbc103e20 0x7f2dbc104200 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009900 tx=0x7f2dac02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.536+0000 7f2dc1000640 1 -- 172.21.15.2:0/417911479 shutdown_connections 2024-09-10T08:09:11.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.536+0000 7f2dc1000640 1 --2- 172.21.15.2:0/417911479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dbc108f50 0x7f2dbc110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.536+0000 7f2dc1000640 1 --2- 172.21.15.2:0/417911479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dbc1047d0 0x7f2dbc108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.536+0000 7f2dc1000640 1 --2- 172.21.15.2:0/417911479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dbc103e20 0x7f2dbc104200 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.536+0000 7f2dc1000640 1 -- 172.21.15.2:0/417911479 >> 172.21.15.2:0/417911479 conn(0x7f2dbc0ff860 msgr2=0x7f2dbc101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:11.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.537+0000 7f2dc1000640 1 -- 172.21.15.2:0/417911479 shutdown_connections 2024-09-10T08:09:11.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.537+0000 7f2dc1000640 1 -- 172.21.15.2:0/417911479 wait complete. 2024-09-10T08:09:11.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.538+0000 7f2dc1000640 1 Processor -- start 2024-09-10T08:09:11.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.538+0000 7f2dc1000640 1 -- start start 2024-09-10T08:09:11.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.539+0000 7f2dc1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dbc103e20 0x7f2dbc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.539+0000 7f2dc1000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dbc1047d0 0x7f2dbc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.539+0000 7f2dc1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dbc108f50 0x7f2dbc19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.539+0000 7f2dc1000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc114000 con 0x7f2dbc1047d0 2024-09-10T08:09:11.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.539+0000 7f2dc1000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc113e80 con 0x7f2dbc103e20 2024-09-10T08:09:11.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.539+0000 7f2dc1000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc114180 con 0x7f2dbc108f50 2024-09-10T08:09:11.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.540+0000 7f2dbaa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dbc1047d0 0x7f2dbc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.540+0000 7f2dbaa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dbc1047d0 0x7f2dbc19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45152/0 (socket says 172.21.15.2:45152) 2024-09-10T08:09:11.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.540+0000 7f2dbaa00640 1 -- 172.21.15.2:0/1966941858 learned_addr learned my addr 172.21.15.2:0/1966941858 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:11.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.540+0000 7f2dbb400640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dbc103e20 0x7f2dbc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.540+0000 7f2dbbe00640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dbc108f50 0x7f2dbc19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.540+0000 7f2dbaa00640 1 -- 172.21.15.2:0/1966941858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dbc108f50 msgr2=0x7f2dbc19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:11.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.540+0000 7f2dbaa00640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dbc108f50 0x7f2dbc19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.540+0000 7f2dbaa00640 1 -- 172.21.15.2:0/1966941858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dbc103e20 msgr2=0x7f2dbc19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:11.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.540+0000 7f2dbaa00640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dbc103e20 0x7f2dbc19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.540+0000 7f2dbaa00640 1 -- 172.21.15.2:0/1966941858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc19f450 con 0x7f2dbc1047d0 2024-09-10T08:09:11.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.541+0000 7f2dbaa00640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dbc1047d0 0x7f2dbc19a9e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2da800e9f0 tx=0x7f2da800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:11.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.541+0000 7f2db3e00640 1 -- 172.21.15.2:0/1966941858 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da800cd80 con 0x7f2dbc1047d0 2024-09-10T08:09:11.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.541+0000 7f2db3e00640 1 -- 172.21.15.2:0/1966941858 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da8005500 con 0x7f2dbc1047d0 2024-09-10T08:09:11.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.541+0000 7f2dc1000640 1 -- 172.21.15.2:0/1966941858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dbc19f740 con 0x7f2dbc1047d0 2024-09-10T08:09:11.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.542+0000 7f2dc1000640 1 -- 172.21.15.2:0/1966941858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dbc076630 con 0x7f2dbc1047d0 2024-09-10T08:09:11.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.541+0000 7f2db3e00640 1 -- 172.21.15.2:0/1966941858 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8010690 con 0x7f2dbc1047d0 2024-09-10T08:09:11.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.545+0000 7f2dc50b7740 1 Processor -- start 2024-09-10T08:09:11.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.545+0000 7f2dc50b7740 1 -- start start 2024-09-10T08:09:11.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.546+0000 7f2dc50b7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb44f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.546+0000 7f2dc50b7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb459ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.546+0000 7f2dc50b7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb45c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.546+0000 7f2dc50b7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b2bb3b53e0 con 0x55b2bb44f760 2024-09-10T08:09:11.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.546+0000 7f2dc50b7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b2bb35cd80 con 0x55b2bb44ee40 2024-09-10T08:09:11.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.546+0000 7f2dc50b7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b2bb3b13a0 con 0x55b2bb45a020 2024-09-10T08:09:11.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.546+0000 7f2dbbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb44f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.546+0000 7f2dbbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb44f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38034/0 (socket says 172.21.15.2:38034) 2024-09-10T08:09:11.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.546+0000 7f2dbbe00640 1 -- 172.21.15.2:0/452672715 learned_addr learned my addr 172.21.15.2:0/452672715 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:11.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.547+0000 7f2dbb400640 1 --2- 172.21.15.2:0/452672715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb459ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.547+0000 7f2dbbe00640 1 -- 172.21.15.2:0/452672715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 msgr2=0x55b2bb45c410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:09:11.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.547+0000 7f2dbbe00640 1 --2- 172.21.15.2:0/452672715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb45c410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.547+0000 7f2dbbe00640 1 -- 172.21.15.2:0/452672715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 msgr2=0x55b2bb459ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:11.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.547+0000 7f2dbbe00640 1 --2- 172.21.15.2:0/452672715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb459ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.547+0000 7f2dbbe00640 1 -- 172.21.15.2:0/452672715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2bb37bae0 con 0x55b2bb44ee40 2024-09-10T08:09:11.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.547+0000 7f2dbb400640 1 --2- 172.21.15.2:0/452672715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb459ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:09:11.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.547+0000 7f2dbbe00640 1 --2- 172.21.15.2:0/452672715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb44f220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009f50 tx=0x7f2db40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9985427391893c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:11.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.547+0000 7f2db3e00640 1 -- 172.21.15.2:0/1966941858 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f2da80046c0 con 0x7f2dbc1047d0 2024-09-10T08:09:11.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.547+0000 7f2db0c00640 1 -- 172.21.15.2:0/452672715 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4004b20 con 0x55b2bb44ee40 2024-09-10T08:09:11.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.547+0000 7f2db0c00640 1 -- 172.21.15.2:0/452672715 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db4004cc0 con 0x55b2bb44ee40 2024-09-10T08:09:11.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.547+0000 7f2db0c00640 1 -- 172.21.15.2:0/452672715 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4005490 con 0x55b2bb44ee40 2024-09-10T08:09:11.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.548+0000 7f2dc50b7740 1 -- 172.21.15.2:0/452672715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 msgr2=0x55b2bb44f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:11.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.548+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/452672715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb44f220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009f50 tx=0x7f2db40118d0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.549+0000 7f2dc50b7740 1 -- 172.21.15.2:0/452672715 shutdown_connections 2024-09-10T08:09:11.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.549+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/452672715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb45c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.549+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/452672715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb459ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.549+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/452672715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb44f220 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.549+0000 7f2dc50b7740 1 -- 172.21.15.2:0/452672715 >> 172.21.15.2:0/452672715 conn(0x55b2bb44b3d0 msgr2=0x55b2bb44b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:11.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.549+0000 7f2dc50b7740 1 -- 172.21.15.2:0/452672715 shutdown_connections 2024-09-10T08:09:11.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.549+0000 7f2dc50b7740 1 -- 172.21.15.2:0/452672715 wait complete. 2024-09-10T08:09:11.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.549+0000 7f2db3e00640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2da00a2c30 0x7f2da00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.550+0000 7f2dc50b7740 1 -- start start 2024-09-10T08:09:11.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.550+0000 7f2db3e00640 1 -- 172.21.15.2:0/1966941858 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2da80cf920 con 0x7f2dbc1047d0 2024-09-10T08:09:11.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.550+0000 7f2dc50b7740 1 Processor -- start 2024-09-10T08:09:11.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.550+0000 7f2dbbe00640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2da00a2c30 0x7f2da00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.551+0000 7f2dc50b7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb457030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.551+0000 7f2dc50b7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb457570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.551+0000 7f2dc50b7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb457ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.551+0000 7f2dc50b7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b2bb45cbf0 con 0x55b2bb44f760 2024-09-10T08:09:11.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.551+0000 7f2dc50b7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b2bb3700f0 con 0x55b2bb44ee40 2024-09-10T08:09:11.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.551+0000 7f2dc50b7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b2bb45cd70 con 0x55b2bb45a020 2024-09-10T08:09:11.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.551+0000 7f2dc50b7740 10 client.0 _ll_register_callbacks cb 0x55b2bb357bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:11.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.551+0000 7f2dbb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb457030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.551+0000 7f2dbb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb457030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38038/0 (socket says 172.21.15.2:38038) 2024-09-10T08:09:11.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.551+0000 7f2dbb400640 1 -- 172.21.15.2:0/3931844262 learned_addr learned my addr 172.21.15.2:0/3931844262 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:11.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.551+0000 7f2dbbe00640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2da00a2c30 0x7f2da00a50f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2db401b040 tx=0x7f2db4000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:11.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.551+0000 7f2dbbe00640 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb457ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.551+0000 7f2dbaa00640 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb457570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.552+0000 7f2dbb400640 1 -- 172.21.15.2:0/3931844262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 msgr2=0x55b2bb457ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:11.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.552+0000 7f2dbb400640 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb457ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.552+0000 7f2dbb400640 1 -- 172.21.15.2:0/3931844262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 msgr2=0x55b2bb457570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:11.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.552+0000 7f2dbb400640 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb457570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.552+0000 7f2dbb400640 1 -- 172.21.15.2:0/3931844262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2bb1263c0 con 0x55b2bb44ee40 2024-09-10T08:09:11.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.552+0000 7f2dbaa00640 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb457570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:11.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.552+0000 7f2dbb400640 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb457030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2dac033000 tx=0x7f2dac004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0384432db2ce3c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:11.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.552+0000 7f2da5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:09:11.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.552+0000 7f2da5600640 1 -- 172.21.15.2:0/3931844262 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac035040 con 0x55b2bb44ee40 2024-09-10T08:09:11.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.552+0000 7f2da5600640 1 -- 172.21.15.2:0/3931844262 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2dac005230 con 0x55b2bb44ee40 2024-09-10T08:09:11.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.552+0000 7f2dc50b7740 1 -- 172.21.15.2:0/3931844262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db4004cc0 con 0x55b2bb44ee40 2024-09-10T08:09:11.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.553+0000 7f2da5600640 1 -- 172.21.15.2:0/3931844262 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2dac04c050 con 0x55b2bb44ee40 2024-09-10T08:09:11.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.553+0000 7f2dc50b7740 10 client.25445 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:11.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.553+0000 7f2dc50b7740 1 -- 172.21.15.2:0/3931844262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b2bb468ca0 con 0x55b2bb44ee40 2024-09-10T08:09:11.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.553+0000 7f2da5600640 1 -- 172.21.15.2:0/3931844262 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db4004cc0 con 0x55b2bb44ee40 2024-09-10T08:09:11.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.554+0000 7f2da5600640 1 -- 172.21.15.2:0/3931844262 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2dac0318e0 con 0x55b2bb44ee40 2024-09-10T08:09:11.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.554+0000 7f2dc50b7740 10 client.25445 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:11.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.554+0000 7f2dc50b7740 10 client.25445 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:09:11.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.554+0000 7f2dc50b7740 20 client.25445 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:11.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.554+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b2bb46a160 0x55b2bb46c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.554+0000 7f2dc50b7740 4 client.25445 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:11.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.554+0000 7f2dc50b7740 1 -- 172.21.15.2:0/3931844262 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2bb3c2950 con 0x55b2bb46a160 2024-09-10T08:09:11.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.554+0000 7f2dbaa00640 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b2bb46a160 0x55b2bb46c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.555+0000 7f2dbaa00640 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b2bb46a160 0x55b2bb46c550 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a76ef1d1abd2d3fe server_cookie=87eeaa516c274709 in_seq=0 out_seq=0 2024-09-10T08:09:11.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.555+0000 7f2da5600640 10 client.25445 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:09:11.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.556+0000 7f2da5600640 1 -- 172.21.15.2:0/3931844262 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2bb3c2950 con 0x55b2bb46a160 2024-09-10T08:09:11.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.556+0000 7f2da5600640 10 client.25445 handle_command_reply: tid=0 2024-09-10T08:09:11.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.556+0000 7f2dc50b7740 1 client.25445 shutdown 2024-09-10T08:09:11.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.556+0000 7f2dc50b7740 1 -- 172.21.15.2:0/3931844262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 msgr2=0x55b2bb457030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:11.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.556+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb457030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2dac033000 tx=0x7f2dac004270 comp rx=0 tx=0).stop 2024-09-10T08:09:11.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.556+0000 7f2dc50b7740 1 -- 172.21.15.2:0/3931844262 shutdown_connections 2024-09-10T08:09:11.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.556+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb457ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.556+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b2bb46a160 0x55b2bb46c550 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.556+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb457570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.556+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/3931844262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb457030 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.556+0000 7f2dc50b7740 1 -- 172.21.15.2:0/3931844262 >> 172.21.15.2:0/3931844262 conn(0x55b2bb44b3d0 msgr2=0x55b2bb451100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:11.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.556+0000 7f2da5600640 0 client.25445 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:09:11.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.556+0000 7f2dc50b7740 1 -- 172.21.15.2:0/3931844262 shutdown_connections 2024-09-10T08:09:11.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.557+0000 7f2dc50b7740 1 -- 172.21.15.2:0/3931844262 wait complete. 2024-09-10T08:09:11.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.557+0000 7f2dc50b7740 20 client.25445 trim_cache size 0 max 16384 2024-09-10T08:09:11.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.585+0000 7f2dc50b7740 1 Processor -- start 2024-09-10T08:09:11.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.585+0000 7f2dc50b7740 1 -- start start 2024-09-10T08:09:11.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.585+0000 7f2dc50b7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44ee40 0x55b2bb458fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.585+0000 7f2dc50b7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44f760 0x55b2bb471e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.585+0000 7f2dc50b7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb472360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.585+0000 7f2dc50b7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1b7bb0 con 0x55b2bb44ee40 2024-09-10T08:09:11.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.585+0000 7f2dc50b7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1b5480 con 0x55b2bb44f760 2024-09-10T08:09:11.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.585+0000 7f2dc50b7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1b32f0 con 0x55b2bb45a020 2024-09-10T08:09:11.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.586+0000 7f2dbaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44f760 0x55b2bb471e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.586+0000 7f2dbb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44ee40 0x55b2bb458fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.586+0000 7f2dbaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44f760 0x55b2bb471e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38040/0 (socket says 172.21.15.2:38040) 2024-09-10T08:09:11.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.586+0000 7f2dbaa00640 1 -- 172.21.15.2:0/686158582 learned_addr learned my addr 172.21.15.2:0/686158582 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:11.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.586+0000 7f2dbb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44ee40 0x55b2bb458fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45188/0 (socket says 172.21.15.2:45188) 2024-09-10T08:09:11.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.586+0000 7f2dbbe00640 1 --2- 172.21.15.2:0/686158582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb472360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.586+0000 7f2dbb400640 1 -- 172.21.15.2:0/686158582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 msgr2=0x55b2bb472360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:11.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.586+0000 7f2dbb400640 1 --2- 172.21.15.2:0/686158582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb472360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.587+0000 7f2dbb400640 1 -- 172.21.15.2:0/686158582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44f760 msgr2=0x55b2bb471e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:11.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.587+0000 7f2dbb400640 1 --2- 172.21.15.2:0/686158582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44f760 0x55b2bb471e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.587+0000 7f2dbb400640 1 -- 172.21.15.2:0/686158582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2bb45c640 con 0x55b2bb44ee40 2024-09-10T08:09:11.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.587+0000 7f2dbaa00640 1 --2- 172.21.15.2:0/686158582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44f760 0x55b2bb471e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:11.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.587+0000 7f2dbb400640 1 --2- 172.21.15.2:0/686158582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44ee40 0x55b2bb458fa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2dac02a910 tx=0x7f2dac02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7028767321059c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:11.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.587+0000 7f2da7e00640 1 -- 172.21.15.2:0/686158582 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac055070 con 0x55b2bb44ee40 2024-09-10T08:09:11.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.588+0000 7f2da7e00640 1 -- 172.21.15.2:0/686158582 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dac049070 con 0x55b2bb44ee40 2024-09-10T08:09:11.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.588+0000 7f2da7e00640 1 -- 172.21.15.2:0/686158582 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac035040 con 0x55b2bb44ee40 2024-09-10T08:09:11.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.589+0000 7f2dc50b7740 1 -- 172.21.15.2:0/686158582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44ee40 msgr2=0x55b2bb458fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:11.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.589+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/686158582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44ee40 0x55b2bb458fa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2dac02a910 tx=0x7f2dac02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.589+0000 7f2dc50b7740 1 -- 172.21.15.2:0/686158582 shutdown_connections 2024-09-10T08:09:11.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.589+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/686158582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb472360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.589+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/686158582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44f760 0x55b2bb471e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.589+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/686158582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44ee40 0x55b2bb458fa0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.589+0000 7f2dc50b7740 1 -- 172.21.15.2:0/686158582 >> 172.21.15.2:0/686158582 conn(0x55b2bb44b3d0 msgr2=0x55b2bb458890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:11.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.589+0000 7f2dc50b7740 1 -- 172.21.15.2:0/686158582 shutdown_connections 2024-09-10T08:09:11.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.589+0000 7f2dc50b7740 1 -- 172.21.15.2:0/686158582 wait complete. 2024-09-10T08:09:11.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.590+0000 7f2dc50b7740 1 -- start start 2024-09-10T08:09:11.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.590+0000 7f2dc50b7740 1 Processor -- start 2024-09-10T08:09:11.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.590+0000 7f2dc50b7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb461fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.590+0000 7f2dc50b7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb462500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.590+0000 7f2dc50b7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb45ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.590+0000 7f2dc50b7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1b3940 con 0x55b2bb44f760 2024-09-10T08:09:11.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.590+0000 7f2dc50b7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1b4db0 con 0x55b2bb44ee40 2024-09-10T08:09:11.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.590+0000 7f2dc50b7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1b4010 con 0x55b2bb45a020 2024-09-10T08:09:11.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.590+0000 7f2dc50b7740 10 client.0 _ll_register_callbacks cb 0x7f2da8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:11.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.591+0000 7f2dbbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb45ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.591+0000 7f2dbbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb45ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49818/0 (socket says 172.21.15.2:49818) 2024-09-10T08:09:11.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.591+0000 7f2dbb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb461fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.591+0000 7f2dbbe00640 1 -- 172.21.15.2:0/4022116414 learned_addr learned my addr 172.21.15.2:0/4022116414 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:11.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.591+0000 7f2dbbe00640 1 -- 172.21.15.2:0/4022116414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 msgr2=0x55b2bb461fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:11.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.591+0000 7f2dbbe00640 1 --2- 172.21.15.2:0/4022116414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb461fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.591+0000 7f2dbbe00640 1 -- 172.21.15.2:0/4022116414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 msgr2=0x55b2bb462500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:11.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.591+0000 7f2dbaa00640 1 --2- 172.21.15.2:0/4022116414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb462500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.592+0000 7f2dbbe00640 1 --2- 172.21.15.2:0/4022116414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb462500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.592+0000 7f2dbbe00640 1 -- 172.21.15.2:0/4022116414 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2bb45f580 con 0x55b2bb45a020 2024-09-10T08:09:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.592+0000 7f2dbaa00640 1 --2- 172.21.15.2:0/4022116414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb462500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:09:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.592+0000 7f2dbbe00640 1 --2- 172.21.15.2:0/4022116414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb45ef90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2db400fe20 tx=0x7f2db400ffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da6b0e926b6a9e61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.592+0000 7f2da5600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:09:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.592+0000 7f2da5600640 1 -- 172.21.15.2:0/4022116414 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4034030 con 0x55b2bb45a020 2024-09-10T08:09:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.593+0000 7f2dc50b7740 1 -- 172.21.15.2:0/4022116414 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dac049070 con 0x55b2bb45a020 2024-09-10T08:09:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.593+0000 7f2da5600640 1 -- 172.21.15.2:0/4022116414 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2db4004260 con 0x55b2bb45a020 2024-09-10T08:09:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.593+0000 7f2da5600640 1 -- 172.21.15.2:0/4022116414 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2db4003e20 con 0x55b2bb45a020 2024-09-10T08:09:11.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.593+0000 7f2dc50b7740 10 client.25423 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:11.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.593+0000 7f2dc50b7740 1 -- 172.21.15.2:0/4022116414 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b2bb4d5730 con 0x55b2bb45a020 2024-09-10T08:09:11.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.593+0000 7f2da5600640 1 -- 172.21.15.2:0/4022116414 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dac049070 con 0x55b2bb45a020 2024-09-10T08:09:11.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.593+0000 7f2da5600640 1 -- 172.21.15.2:0/4022116414 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2db4022b50 con 0x55b2bb45a020 2024-09-10T08:09:11.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.594+0000 7f2dc50b7740 10 client.25423 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:11.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.594+0000 7f2dc50b7740 10 client.25423 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:09:11.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.594+0000 7f2dc50b7740 20 client.25423 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:11.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.594+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/4022116414 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b2bb4d5d20 0x55b2bb4d8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:11.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.594+0000 7f2dc50b7740 4 client.25423 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:09:11.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.594+0000 7f2dc50b7740 1 -- 172.21.15.2:0/4022116414 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2bb4469a0 con 0x55b2bb4d5d20 2024-09-10T08:09:11.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.594+0000 7f2dbb400640 1 --2- 172.21.15.2:0/4022116414 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b2bb4d5d20 0x55b2bb4d8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:11.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2da5600640 10 client.25423 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:09:11.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2dbb400640 1 --2- 172.21.15.2:0/4022116414 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b2bb4d5d20 0x55b2bb4d8140 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b818aa31d65d6c2 server_cookie=5d388c9162487cc7 in_seq=0 out_seq=0 2024-09-10T08:09:11.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2da5600640 1 -- 172.21.15.2:0/4022116414 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2bb4469a0 con 0x55b2bb4d5d20 2024-09-10T08:09:11.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2da5600640 10 client.25423 handle_command_reply: tid=0 2024-09-10T08:09:11.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2dc50b7740 1 client.25423 shutdown 2024-09-10T08:09:11.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2dc50b7740 1 -- 172.21.15.2:0/4022116414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 msgr2=0x55b2bb45ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:11.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/4022116414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb45ef90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2db400fe20 tx=0x7f2db400ffd0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2dc50b7740 1 -- 172.21.15.2:0/4022116414 shutdown_connections 2024-09-10T08:09:11.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/4022116414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2bb45a020 0x55b2bb45ef90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/4022116414 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b2bb4d5d20 0x55b2bb4d8140 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/4022116414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2bb44f760 0x55b2bb462500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2dc50b7740 1 --2- 172.21.15.2:0/4022116414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2bb44ee40 0x55b2bb461fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2dc50b7740 1 -- 172.21.15.2:0/4022116414 >> 172.21.15.2:0/4022116414 conn(0x55b2bb44b3d0 msgr2=0x55b2bb4590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:11.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.595+0000 7f2da5600640 0 client.25423 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:09:11.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.596+0000 7f2dc50b7740 1 -- 172.21.15.2:0/4022116414 shutdown_connections 2024-09-10T08:09:11.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.596+0000 7f2dc50b7740 1 -- 172.21.15.2:0/4022116414 wait complete. 2024-09-10T08:09:11.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.596+0000 7f2dc50b7740 20 client.25423 trim_cache size 0 max 16384 2024-09-10T08:09:11.598 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:09:11.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.597+0000 7f2dc1000640 1 -- 172.21.15.2:0/1966941858 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2da00a2c30 msgr2=0x7f2da00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:11.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.597+0000 7f2dc1000640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2da00a2c30 0x7f2da00a50f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2db401b040 tx=0x7f2db4000f40 comp rx=0 tx=0).stop 2024-09-10T08:09:11.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.597+0000 7f2dc1000640 1 -- 172.21.15.2:0/1966941858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dbc1047d0 msgr2=0x7f2dbc19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:11.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.597+0000 7f2dc1000640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dbc1047d0 0x7f2dbc19a9e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2da800e9f0 tx=0x7f2da800edf0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.597+0000 7f2dc1000640 1 -- 172.21.15.2:0/1966941858 shutdown_connections 2024-09-10T08:09:11.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.597+0000 7f2dc1000640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2da00a2c30 0x7f2da00a50f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.597+0000 7f2dc1000640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dbc108f50 0x7f2dbc19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.597+0000 7f2dc1000640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dbc1047d0 0x7f2dbc19a9e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.597+0000 7f2dc1000640 1 --2- 172.21.15.2:0/1966941858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dbc103e20 0x7f2dbc19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:11.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.597+0000 7f2dc1000640 1 -- 172.21.15.2:0/1966941858 >> 172.21.15.2:0/1966941858 conn(0x7f2dbc0ff860 msgr2=0x7f2dbc100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:11.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.598+0000 7f2dc1000640 1 -- 172.21.15.2:0/1966941858 shutdown_connections 2024-09-10T08:09:11.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:11.598+0000 7f2dc1000640 1 -- 172.21.15.2:0/1966941858 wait complete. 2024-09-10T08:09:11.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:11 smithi155 ceph-mon[35405]: pgmap v418: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-10T08:09:11.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:11 smithi107 ceph-mon[35048]: pgmap v418: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-10T08:09:11.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:11 smithi002 ceph-mon[30563]: pgmap v418: 129 pgs: 129 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-10T08:09:12.361 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:12.384 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:09:12.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.958+0000 7fe946200640 1 -- 172.21.15.2:0/1406548149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe940074ee0 msgr2=0x7fe94010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.958+0000 7fe946200640 1 --2- 172.21.15.2:0/1406548149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe940074ee0 0x7fe94010dd20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009900 tx=0x7fe92c02a480 comp rx=0 tx=0).stop 2024-09-10T08:09:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.958+0000 7fe946200640 1 -- 172.21.15.2:0/1406548149 shutdown_connections 2024-09-10T08:09:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.958+0000 7fe946200640 1 --2- 172.21.15.2:0/1406548149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe94010e260 0x7fe940110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.958+0000 7fe946200640 1 --2- 172.21.15.2:0/1406548149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe940074ee0 0x7fe94010dd20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.958+0000 7fe946200640 1 --2- 172.21.15.2:0/1406548149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9400745c0 0x7fe9400749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.958+0000 7fe946200640 1 -- 172.21.15.2:0/1406548149 >> 172.21.15.2:0/1406548149 conn(0x7fe9400fd740 msgr2=0x7fe9400ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.958+0000 7fe946200640 1 -- 172.21.15.2:0/1406548149 shutdown_connections 2024-09-10T08:09:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.958+0000 7fe946200640 1 -- 172.21.15.2:0/1406548149 wait complete. 2024-09-10T08:09:12.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.959+0000 7fe946200640 1 Processor -- start 2024-09-10T08:09:12.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.960+0000 7fe946200640 1 -- start start 2024-09-10T08:09:12.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.960+0000 7fe946200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9400745c0 0x7fe94019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:12.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.960+0000 7fe946200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe940074ee0 0x7fe94019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:12.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.960+0000 7fe946200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe94010e260 0x7fe94019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:12.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.960+0000 7fe946200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe940113fb0 con 0x7fe9400745c0 2024-09-10T08:09:12.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.960+0000 7fe946200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe940113e30 con 0x7fe940074ee0 2024-09-10T08:09:12.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.960+0000 7fe946200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe940114130 con 0x7fe94010e260 2024-09-10T08:09:12.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.961+0000 7fe93fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe940074ee0 0x7fe94019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:12.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.961+0000 7fe93fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe940074ee0 0x7fe94019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38074/0 (socket says 172.21.15.2:38074) 2024-09-10T08:09:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.961+0000 7fe93fe00640 1 -- 172.21.15.2:0/3237384650 learned_addr learned my addr 172.21.15.2:0/3237384650 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.961+0000 7fe945800640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe94010e260 0x7fe94019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.961+0000 7fe944e00640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9400745c0 0x7fe94019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.961+0000 7fe93fe00640 1 -- 172.21.15.2:0/3237384650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe94010e260 msgr2=0x7fe94019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.961+0000 7fe93fe00640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe94010e260 0x7fe94019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.961+0000 7fe93fe00640 1 -- 172.21.15.2:0/3237384650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9400745c0 msgr2=0x7fe94019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.961+0000 7fe93fe00640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9400745c0 0x7fe94019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.961+0000 7fe93fe00640 1 -- 172.21.15.2:0/3237384650 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94019f460 con 0x7fe940074ee0 2024-09-10T08:09:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.962+0000 7fe93fe00640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe940074ee0 0x7fe94019a9f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe92c006010 tx=0x7fe92c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.962+0000 7fe93d600640 1 -- 172.21.15.2:0/3237384650 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c03f030 con 0x7fe940074ee0 2024-09-10T08:09:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.962+0000 7fe93d600640 1 -- 172.21.15.2:0/3237384650 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe92c004440 con 0x7fe940074ee0 2024-09-10T08:09:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.962+0000 7fe93d600640 1 -- 172.21.15.2:0/3237384650 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c035070 con 0x7fe940074ee0 2024-09-10T08:09:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.962+0000 7fe946200640 1 -- 172.21.15.2:0/3237384650 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94019f6f0 con 0x7fe940074ee0 2024-09-10T08:09:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.962+0000 7fe946200640 1 -- 172.21.15.2:0/3237384650 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe94019fb80 con 0x7fe940074ee0 2024-09-10T08:09:12.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.966+0000 7fe94a1df740 1 Processor -- start 2024-09-10T08:09:12.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.967+0000 7fe94a1df740 1 -- start start 2024-09-10T08:09:12.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.967+0000 7fe94a1df740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216ce560 0x5557216ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:12.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.967+0000 7fe94a1df740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216cee80 0x5557216d9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:12.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.967+0000 7fe94a1df740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216d9740 0x5557216dbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:12.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.967+0000 7fe94a1df740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555721634530 con 0x5557216d9740 2024-09-10T08:09:12.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.967+0000 7fe94a1df740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5557215db860 con 0x5557216cee80 2024-09-10T08:09:12.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.967+0000 7fe94a1df740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5557216304f0 con 0x5557216ce560 2024-09-10T08:09:12.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.967+0000 7fe945800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216ce560 0x5557216ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:12.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.967+0000 7fe944e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216cee80 0x5557216d9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:12.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.967+0000 7fe945800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216ce560 0x5557216ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49856/0 (socket says 172.21.15.2:49856) 2024-09-10T08:09:12.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.967+0000 7fe945800640 1 -- 172.21.15.2:0/1879580747 learned_addr learned my addr 172.21.15.2:0/1879580747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:12.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.968+0000 7fe93d600640 1 -- 172.21.15.2:0/3237384650 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fe92c042030 con 0x7fe940074ee0 2024-09-10T08:09:12.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.968+0000 7fe945800640 1 -- 172.21.15.2:0/1879580747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216cee80 msgr2=0x5557216d9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:12.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.968+0000 7fe945800640 1 --2- 172.21.15.2:0/1879580747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216cee80 0x5557216d9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.968+0000 7fe945800640 1 -- 172.21.15.2:0/1879580747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216d9740 msgr2=0x5557216dbb30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:09:12.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.968+0000 7fe945800640 1 --2- 172.21.15.2:0/1879580747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216d9740 0x5557216dbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.968+0000 7fe945800640 1 -- 172.21.15.2:0/1879580747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557213a5b90 con 0x5557216ce560 2024-09-10T08:09:12.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.968+0000 7fe944e00640 1 --2- 172.21.15.2:0/1879580747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216cee80 0x5557216d9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:12.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.968+0000 7fe945800640 1 --2- 172.21.15.2:0/1879580747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216ce560 0x5557216ce940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe938009f30 tx=0x7fe9380118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41da06789a56d27a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:12.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.969+0000 7fe932000640 1 -- 172.21.15.2:0/1879580747 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938004d20 con 0x5557216ce560 2024-09-10T08:09:12.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.969+0000 7fe932000640 1 -- 172.21.15.2:0/1879580747 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe938004050 con 0x5557216ce560 2024-09-10T08:09:12.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.969+0000 7fe932000640 1 -- 172.21.15.2:0/1879580747 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938005260 con 0x5557216ce560 2024-09-10T08:09:12.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.969+0000 7fe93d600640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe9280a2c90 0x7fe9280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:12.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.970+0000 7fe93d600640 1 -- 172.21.15.2:0/3237384650 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe92c0ef920 con 0x7fe940074ee0 2024-09-10T08:09:12.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.970+0000 7fe94a1df740 1 -- 172.21.15.2:0/1879580747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216ce560 msgr2=0x5557216ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:12.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.970+0000 7fe94a1df740 1 --2- 172.21.15.2:0/1879580747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216ce560 0x5557216ce940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe938009f30 tx=0x7fe9380118d0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.970+0000 7fe94a1df740 1 -- 172.21.15.2:0/1879580747 shutdown_connections 2024-09-10T08:09:12.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.970+0000 7fe94a1df740 1 --2- 172.21.15.2:0/1879580747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216d9740 0x5557216dbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.970+0000 7fe94a1df740 1 --2- 172.21.15.2:0/1879580747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216cee80 0x5557216d9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.970+0000 7fe94a1df740 1 --2- 172.21.15.2:0/1879580747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216ce560 0x5557216ce940 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.970+0000 7fe94a1df740 1 -- 172.21.15.2:0/1879580747 >> 172.21.15.2:0/1879580747 conn(0x5557216caaf0 msgr2=0x5557216caed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:12.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.970+0000 7fe944e00640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe9280a2c90 0x7fe9280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:12.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.971+0000 7fe94a1df740 1 -- 172.21.15.2:0/1879580747 shutdown_connections 2024-09-10T08:09:12.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.971+0000 7fe94a1df740 1 -- 172.21.15.2:0/1879580747 wait complete. 2024-09-10T08:09:12.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.971+0000 7fe944e00640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe9280a2c90 0x7fe9280a5150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5557216cb950 tx=0x7fe934005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:12.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.971+0000 7fe94a1df740 1 -- start start 2024-09-10T08:09:12.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.972+0000 7fe94a1df740 1 Processor -- start 2024-09-10T08:09:12.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.972+0000 7fe94a1df740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216ce560 0x5557216d66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:12.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.972+0000 7fe94a1df740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216cee80 0x5557216d6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:12.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.972+0000 7fe94a1df740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216d7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:12.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.972+0000 7fe94a1df740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5557216dc310 con 0x5557216cee80 2024-09-10T08:09:12.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.972+0000 7fe94a1df740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5557215eee60 con 0x5557216ce560 2024-09-10T08:09:12.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.972+0000 7fe94a1df740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5557216dc490 con 0x5557216d9740 2024-09-10T08:09:12.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.972+0000 7fe94a1df740 10 client.0 _ll_register_callbacks cb 0x5557215f3cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:12.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.973+0000 7fe944e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216ce560 0x5557216d66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:12.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.973+0000 7fe93fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216cee80 0x5557216d6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:12.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.973+0000 7fe944e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216ce560 0x5557216d66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38094/0 (socket says 172.21.15.2:38094) 2024-09-10T08:09:12.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.973+0000 7fe944e00640 1 -- 172.21.15.2:0/751708881 learned_addr learned my addr 172.21.15.2:0/751708881 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:12.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.973+0000 7fe945800640 1 --2- 172.21.15.2:0/751708881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216d7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:12.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.973+0000 7fe944e00640 1 -- 172.21.15.2:0/751708881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 msgr2=0x5557216d7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:12.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.973+0000 7fe944e00640 1 --2- 172.21.15.2:0/751708881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216d7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.973+0000 7fe944e00640 1 -- 172.21.15.2:0/751708881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216cee80 msgr2=0x5557216d6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:12.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.973+0000 7fe944e00640 1 --2- 172.21.15.2:0/751708881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216cee80 0x5557216d6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.973+0000 7fe944e00640 1 -- 172.21.15.2:0/751708881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557215fa990 con 0x5557216ce560 2024-09-10T08:09:12.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.973+0000 7fe945800640 1 --2- 172.21.15.2:0/751708881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216d7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:12.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.974+0000 7fe944e00640 1 --2- 172.21.15.2:0/751708881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216ce560 0x5557216d66b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe934004610 tx=0x7fe934004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc46a1cfd16e6f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:12.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.974+0000 7fe926a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:09:12.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.974+0000 7fe926a00640 1 -- 172.21.15.2:0/751708881 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93401e470 con 0x5557216ce560 2024-09-10T08:09:12.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.974+0000 7fe94a1df740 1 -- 172.21.15.2:0/751708881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe938004050 con 0x5557216ce560 2024-09-10T08:09:12.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.974+0000 7fe926a00640 1 -- 172.21.15.2:0/751708881 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe93401e610 con 0x5557216ce560 2024-09-10T08:09:12.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.974+0000 7fe926a00640 1 -- 172.21.15.2:0/751708881 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe93401e810 con 0x5557216ce560 2024-09-10T08:09:12.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.974+0000 7fe94a1df740 10 client.25460 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:12.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.974+0000 7fe94a1df740 1 -- 172.21.15.2:0/751708881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5557216f0890 con 0x5557216ce560 2024-09-10T08:09:12.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.975+0000 7fe926a00640 1 -- 172.21.15.2:0/751708881 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe938004050 con 0x5557216ce560 2024-09-10T08:09:12.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.975+0000 7fe926a00640 1 -- 172.21.15.2:0/751708881 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe934027070 con 0x5557216ce560 2024-09-10T08:09:12.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.975+0000 7fe94a1df740 10 client.25460 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:12.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.975+0000 7fe94a1df740 10 client.25460 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:09:12.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.975+0000 7fe94a1df740 20 client.25460 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:12.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.975+0000 7fe94a1df740 1 --2- 172.21.15.2:0/751708881 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5557216f1ca0 0x5557216f4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:12.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.975+0000 7fe94a1df740 4 client.25460 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:12.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.975+0000 7fe94a1df740 1 -- 172.21.15.2:0/751708881 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557215d7310 con 0x5557216f1ca0 2024-09-10T08:09:12.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.976+0000 7fe93fe00640 1 --2- 172.21.15.2:0/751708881 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5557216f1ca0 0x5557216f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:12.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.976+0000 7fe93fe00640 1 --2- 172.21.15.2:0/751708881 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5557216f1ca0 0x5557216f4160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564387d37e07dcb8 server_cookie=f6c77affe99e5d45 in_seq=0 out_seq=0 2024-09-10T08:09:12.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.976+0000 7fe926a00640 10 client.25460 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:12.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.977+0000 7fe926a00640 1 -- 172.21.15.2:0/751708881 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557215d7310 con 0x5557216f1ca0 2024-09-10T08:09:12.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.977+0000 7fe926a00640 10 client.25460 handle_command_reply: tid=0 2024-09-10T08:09:12.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.977+0000 7fe94a1df740 1 client.25460 shutdown 2024-09-10T08:09:12.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.977+0000 7fe94a1df740 1 -- 172.21.15.2:0/751708881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216ce560 msgr2=0x5557216d66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:12.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.977+0000 7fe94a1df740 1 --2- 172.21.15.2:0/751708881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216ce560 0x5557216d66b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe934004610 tx=0x7fe934004730 comp rx=0 tx=0).stop 2024-09-10T08:09:12.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.977+0000 7fe94a1df740 1 -- 172.21.15.2:0/751708881 shutdown_connections 2024-09-10T08:09:12.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.977+0000 7fe94a1df740 1 --2- 172.21.15.2:0/751708881 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5557216f1ca0 0x5557216f4160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.978+0000 7fe94a1df740 1 --2- 172.21.15.2:0/751708881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216d7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.978+0000 7fe94a1df740 1 --2- 172.21.15.2:0/751708881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216cee80 0x5557216d6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.978+0000 7fe94a1df740 1 --2- 172.21.15.2:0/751708881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216ce560 0x5557216d66b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:12.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.978+0000 7fe94a1df740 1 -- 172.21.15.2:0/751708881 >> 172.21.15.2:0/751708881 conn(0x5557216caaf0 msgr2=0x5557216d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:12.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.978+0000 7fe926a00640 0 client.25460 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:12.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.978+0000 7fe94a1df740 1 -- 172.21.15.2:0/751708881 shutdown_connections 2024-09-10T08:09:12.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.978+0000 7fe94a1df740 1 -- 172.21.15.2:0/751708881 wait complete. 2024-09-10T08:09:12.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:12.978+0000 7fe94a1df740 20 client.25460 trim_cache size 0 max 16384 2024-09-10T08:09:13.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.015+0000 7fe94a1df740 1 Processor -- start 2024-09-10T08:09:13.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.015+0000 7fe94a1df740 1 -- start start 2024-09-10T08:09:13.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.015+0000 7fe94a1df740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216ce560 0x5557216d55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.015+0000 7fe94a1df740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216cee80 0x5557216d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.015+0000 7fe94a1df740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.015+0000 7fe94a1df740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401af380 con 0x5557216ce560 2024-09-10T08:09:13.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.015+0000 7fe94a1df740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401acc50 con 0x5557216cee80 2024-09-10T08:09:13.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.015+0000 7fe94a1df740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401aaaa0 con 0x5557216d9740 2024-09-10T08:09:13.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.016+0000 7fe945800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.016+0000 7fe93fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216cee80 0x5557216d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.016+0000 7fe945800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216f9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49866/0 (socket says 172.21.15.2:49866) 2024-09-10T08:09:13.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.016+0000 7fe93fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216cee80 0x5557216d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38106/0 (socket says 172.21.15.2:38106) 2024-09-10T08:09:13.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.016+0000 7fe945800640 1 -- 172.21.15.2:0/4156745885 learned_addr learned my addr 172.21.15.2:0/4156745885 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:13.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.016+0000 7fe944e00640 1 --2- 172.21.15.2:0/4156745885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216ce560 0x5557216d55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.016+0000 7fe945800640 1 -- 172.21.15.2:0/4156745885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216cee80 msgr2=0x5557216d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:13.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.016+0000 7fe945800640 1 --2- 172.21.15.2:0/4156745885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216cee80 0x5557216d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.016+0000 7fe945800640 1 -- 172.21.15.2:0/4156745885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216ce560 msgr2=0x5557216d55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:13.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.016+0000 7fe945800640 1 --2- 172.21.15.2:0/4156745885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216ce560 0x5557216d55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.016+0000 7fe945800640 1 -- 172.21.15.2:0/4156745885 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557216cbba0 con 0x5557216d9740 2024-09-10T08:09:13.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.016+0000 7fe93fe00640 1 --2- 172.21.15.2:0/4156745885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216cee80 0x5557216d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:13.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.017+0000 7fe945800640 1 --2- 172.21.15.2:0/4156745885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216f9a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe938014040 tx=0x7fe938003f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c36bb9649d1b29a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:13.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.017+0000 7fe931600640 1 -- 172.21.15.2:0/4156745885 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9380042d0 con 0x5557216d9740 2024-09-10T08:09:13.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.017+0000 7fe931600640 1 -- 172.21.15.2:0/4156745885 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe938004d20 con 0x5557216d9740 2024-09-10T08:09:13.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.017+0000 7fe931600640 1 -- 172.21.15.2:0/4156745885 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938005680 con 0x5557216d9740 2024-09-10T08:09:13.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.018+0000 7fe94a1df740 1 -- 172.21.15.2:0/4156745885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 msgr2=0x5557216f9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:13.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.018+0000 7fe94a1df740 1 --2- 172.21.15.2:0/4156745885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216f9a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe938014040 tx=0x7fe938003f40 comp rx=0 tx=0).stop 2024-09-10T08:09:13.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.018+0000 7fe94a1df740 1 -- 172.21.15.2:0/4156745885 shutdown_connections 2024-09-10T08:09:13.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.018+0000 7fe94a1df740 1 --2- 172.21.15.2:0/4156745885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216f9a80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.018+0000 7fe94a1df740 1 --2- 172.21.15.2:0/4156745885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216cee80 0x5557216d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.018+0000 7fe94a1df740 1 --2- 172.21.15.2:0/4156745885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216ce560 0x5557216d55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.018+0000 7fe94a1df740 1 -- 172.21.15.2:0/4156745885 >> 172.21.15.2:0/4156745885 conn(0x5557216caaf0 msgr2=0x5557216c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:13.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.018+0000 7fe94a1df740 1 -- 172.21.15.2:0/4156745885 shutdown_connections 2024-09-10T08:09:13.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.019+0000 7fe94a1df740 1 -- 172.21.15.2:0/4156745885 wait complete. 2024-09-10T08:09:13.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.019+0000 7fe94a1df740 1 -- start start 2024-09-10T08:09:13.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.019+0000 7fe94a1df740 1 Processor -- start 2024-09-10T08:09:13.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.020+0000 7fe94a1df740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216ce560 0x5557216ea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.020+0000 7fe94a1df740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216cee80 0x5557216d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.020+0000 7fe94a1df740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216d3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.020+0000 7fe94a1df740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401ab110 con 0x5557216cee80 2024-09-10T08:09:13.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.020+0000 7fe94a1df740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401ac580 con 0x5557216ce560 2024-09-10T08:09:13.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.020+0000 7fe94a1df740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401ab7e0 con 0x5557216d9740 2024-09-10T08:09:13.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.020+0000 7fe94a1df740 10 client.0 _ll_register_callbacks cb 0x7fe934009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:13.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.020+0000 7fe93fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216cee80 0x5557216d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.020+0000 7fe944e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216ce560 0x5557216ea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.020+0000 7fe93fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216cee80 0x5557216d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45256/0 (socket says 172.21.15.2:45256) 2024-09-10T08:09:13.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.020+0000 7fe93fe00640 1 -- 172.21.15.2:0/873263469 learned_addr learned my addr 172.21.15.2:0/873263469 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:13.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.020+0000 7fe945800640 1 --2- 172.21.15.2:0/873263469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216d3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.021+0000 7fe93fe00640 1 -- 172.21.15.2:0/873263469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 msgr2=0x5557216d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:13.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.021+0000 7fe93fe00640 1 --2- 172.21.15.2:0/873263469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216d3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.021+0000 7fe93fe00640 1 -- 172.21.15.2:0/873263469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216ce560 msgr2=0x5557216ea7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:13.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.021+0000 7fe93fe00640 1 --2- 172.21.15.2:0/873263469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216ce560 0x5557216ea7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.021+0000 7fe93fe00640 1 -- 172.21.15.2:0/873263469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557216d42d0 con 0x5557216cee80 2024-09-10T08:09:13.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.021+0000 7fe93fe00640 1 --2- 172.21.15.2:0/873263469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216cee80 0x5557216d34d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0bba40 tx=0x7fe92c0bbd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bad7f32883f6334 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:13.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.021+0000 7fe926a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:09:13.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.022+0000 7fe926a00640 1 -- 172.21.15.2:0/873263469 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c049040 con 0x5557216cee80 2024-09-10T08:09:13.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.022+0000 7fe926a00640 1 -- 172.21.15.2:0/873263469 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe92c0b83b0 con 0x5557216cee80 2024-09-10T08:09:13.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.022+0000 7fe94a1df740 1 -- 172.21.15.2:0/873263469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe938004d20 con 0x5557216cee80 2024-09-10T08:09:13.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.022+0000 7fe926a00640 1 -- 172.21.15.2:0/873263469 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe92c0ed0d0 con 0x5557216cee80 2024-09-10T08:09:13.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.022+0000 7fe94a1df740 10 client.15441 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:13.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.022+0000 7fe94a1df740 1 -- 172.21.15.2:0/873263469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55572175cb90 con 0x5557216cee80 2024-09-10T08:09:13.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.023+0000 7fe926a00640 1 -- 172.21.15.2:0/873263469 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe938004d20 con 0x5557216cee80 2024-09-10T08:09:13.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.023+0000 7fe926a00640 1 -- 172.21.15.2:0/873263469 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe92c031b50 con 0x5557216cee80 2024-09-10T08:09:13.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.023+0000 7fe94a1df740 10 client.15441 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:13.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.023+0000 7fe94a1df740 10 client.15441 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:09:13.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.023+0000 7fe94a1df740 20 client.15441 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:13.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.023+0000 7fe94a1df740 1 --2- 172.21.15.2:0/873263469 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55572175d2e0 0x55572175f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.023+0000 7fe94a1df740 4 client.15441 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T08:09:13.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.023+0000 7fe94a1df740 1 -- 172.21.15.2:0/873263469 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x555721641c60 con 0x55572175d2e0 2024-09-10T08:09:13.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.023+0000 7fe944e00640 1 --2- 172.21.15.2:0/873263469 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55572175d2e0 0x55572175f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.024+0000 7fe944e00640 1 --2- 172.21.15.2:0/873263469 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55572175d2e0 0x55572175f700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b78b7dff6fd870d server_cookie=7125345bbe8df2e6 in_seq=0 out_seq=0 2024-09-10T08:09:13.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.024+0000 7fe926a00640 10 client.15441 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:13.028 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:13.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.036+0000 7fe926a00640 1 -- 172.21.15.2:0/873263469 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x555721641c60 con 0x55572175d2e0 2024-09-10T08:09:13.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.036+0000 7fe926a00640 10 client.15441 handle_command_reply: tid=0 2024-09-10T08:09:13.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.036+0000 7fe94a1df740 1 client.15441 shutdown 2024-09-10T08:09:13.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.036+0000 7fe94a1df740 1 -- 172.21.15.2:0/873263469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216cee80 msgr2=0x5557216d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:13.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.036+0000 7fe94a1df740 1 --2- 172.21.15.2:0/873263469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216cee80 0x5557216d34d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0bba40 tx=0x7fe92c0bbd40 comp rx=0 tx=0).stop 2024-09-10T08:09:13.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.037+0000 7fe94a1df740 1 -- 172.21.15.2:0/873263469 shutdown_connections 2024-09-10T08:09:13.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.037+0000 7fe94a1df740 1 --2- 172.21.15.2:0/873263469 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55572175d2e0 0x55572175f700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.037+0000 7fe94a1df740 1 --2- 172.21.15.2:0/873263469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557216d9740 0x5557216d3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.037+0000 7fe94a1df740 1 --2- 172.21.15.2:0/873263469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557216cee80 0x5557216d34d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.037+0000 7fe94a1df740 1 --2- 172.21.15.2:0/873263469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557216ce560 0x5557216ea7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.037+0000 7fe926a00640 0 client.15441 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:13.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.037+0000 7fe94a1df740 1 -- 172.21.15.2:0/873263469 >> 172.21.15.2:0/873263469 conn(0x5557216caaf0 msgr2=0x5557216ece20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:13.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.037+0000 7fe94a1df740 1 -- 172.21.15.2:0/873263469 shutdown_connections 2024-09-10T08:09:13.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.037+0000 7fe94a1df740 1 -- 172.21.15.2:0/873263469 wait complete. 2024-09-10T08:09:13.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.038+0000 7fe94a1df740 20 client.15441 trim_cache size 0 max 16384 2024-09-10T08:09:13.040 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:09:13.040 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:09:13.040 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:09:13.041 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 2, 2024-09-10T08:09:13.041 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:09:13.041 INFO:teuthology.orchestra.run.smithi002.stdout: "db781f90": { 2024-09-10T08:09:13.041 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 2, 2024-09-10T08:09:13.041 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:09:13.041 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:09:13.041 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:09:13.041 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:09:13.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:09:13.041 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:09:13.042 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 150.0, 2024-09-10T08:09:13.042 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:09:13.042 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:09:13.042 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:09:13.042 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:09:13.042 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:09:13.042 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:09:13.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:13.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:13.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:13.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:13.043 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:13.043 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:09:13.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.039+0000 7fe946200640 1 -- 172.21.15.2:0/3237384650 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe9280a2c90 msgr2=0x7fe9280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:13.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.039+0000 7fe946200640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe9280a2c90 0x7fe9280a5150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5557216cb950 tx=0x7fe934005f90 comp rx=0 tx=0).stop 2024-09-10T08:09:13.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.039+0000 7fe946200640 1 -- 172.21.15.2:0/3237384650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe940074ee0 msgr2=0x7fe94019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:13.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.039+0000 7fe946200640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe940074ee0 0x7fe94019a9f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe92c006010 tx=0x7fe92c004040 comp rx=0 tx=0).stop 2024-09-10T08:09:13.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.039+0000 7fe946200640 1 -- 172.21.15.2:0/3237384650 shutdown_connections 2024-09-10T08:09:13.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.039+0000 7fe946200640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe9280a2c90 0x7fe9280a5150 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.039+0000 7fe946200640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe94010e260 0x7fe94019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.039+0000 7fe946200640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe940074ee0 0x7fe94019a9f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.039+0000 7fe946200640 1 --2- 172.21.15.2:0/3237384650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9400745c0 0x7fe94019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.039+0000 7fe946200640 1 -- 172.21.15.2:0/3237384650 >> 172.21.15.2:0/3237384650 conn(0x7fe9400fd740 msgr2=0x7fe9400fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:13.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.040+0000 7fe946200640 1 -- 172.21.15.2:0/3237384650 shutdown_connections 2024-09-10T08:09:13.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.040+0000 7fe946200640 1 -- 172.21.15.2:0/3237384650 wait complete. 2024-09-10T08:09:13.700 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:13 smithi002 ceph-mon[30563]: pgmap v419: 129 pgs: 129 active+clean; 667 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-10T08:09:13.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.888+0000 7f0ca3200640 1 -- 172.21.15.2:0/2373633821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca4102b90 msgr2=0x7f0ca4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:13.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.888+0000 7f0ca3200640 1 --2- 172.21.15.2:0/2373633821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca4102b90 0x7f0ca4102f70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009900 tx=0x7f0c9402a480 comp rx=0 tx=0).stop 2024-09-10T08:09:13.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.888+0000 7f0ca3200640 1 -- 172.21.15.2:0/2373633821 shutdown_connections 2024-09-10T08:09:13.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.888+0000 7f0ca3200640 1 --2- 172.21.15.2:0/2373633821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ca410bfc0 0x7f0ca410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.888+0000 7f0ca3200640 1 --2- 172.21.15.2:0/2373633821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca41034b0 0x7f0ca410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.888+0000 7f0ca3200640 1 --2- 172.21.15.2:0/2373633821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca4102b90 0x7f0ca4102f70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.888+0000 7f0ca3200640 1 -- 172.21.15.2:0/2373633821 >> 172.21.15.2:0/2373633821 conn(0x7f0ca40fc610 msgr2=0x7f0ca40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:13.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.888+0000 7f0ca3200640 1 -- 172.21.15.2:0/2373633821 shutdown_connections 2024-09-10T08:09:13.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.888+0000 7f0ca3200640 1 -- 172.21.15.2:0/2373633821 wait complete. 2024-09-10T08:09:13.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.889+0000 7f0ca3200640 1 Processor -- start 2024-09-10T08:09:13.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.889+0000 7f0ca3200640 1 -- start start 2024-09-10T08:09:13.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.889+0000 7f0ca3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ca4102b90 0x7f0ca419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca3200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca41034b0 0x7f0ca419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca410bfc0 0x7f0ca419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca3200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca4111dc0 con 0x7f0ca41034b0 2024-09-10T08:09:13.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca4111c40 con 0x7f0ca4102b90 2024-09-10T08:09:13.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca4111f40 con 0x7f0ca410bfc0 2024-09-10T08:09:13.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ca4102b90 0x7f0ca419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca41034b0 0x7f0ca419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ca4102b90 0x7f0ca419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38136/0 (socket says 172.21.15.2:38136) 2024-09-10T08:09:13.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca1e00640 1 -- 172.21.15.2:0/3293551881 learned_addr learned my addr 172.21.15.2:0/3293551881 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:13.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca2800640 1 --2- 172.21.15.2:0/3293551881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca410bfc0 0x7f0ca419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca1e00640 1 -- 172.21.15.2:0/3293551881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca410bfc0 msgr2=0x7f0ca419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:13.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca1e00640 1 --2- 172.21.15.2:0/3293551881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca410bfc0 0x7f0ca419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca1e00640 1 -- 172.21.15.2:0/3293551881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca41034b0 msgr2=0x7f0ca419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:13.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca1e00640 1 --2- 172.21.15.2:0/3293551881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca41034b0 0x7f0ca419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.890+0000 7f0ca1e00640 1 -- 172.21.15.2:0/3293551881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca419f460 con 0x7f0ca4102b90 2024-09-10T08:09:13.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.891+0000 7f0ca1e00640 1 --2- 172.21.15.2:0/3293551881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ca4102b90 0x7f0ca419a4b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0c94006010 tx=0x7f0c94004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:13.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.891+0000 7f0c9aa00640 1 -- 172.21.15.2:0/3293551881 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9403f030 con 0x7f0ca4102b90 2024-09-10T08:09:13.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.891+0000 7f0ca3200640 1 -- 172.21.15.2:0/3293551881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca419f6f0 con 0x7f0ca4102b90 2024-09-10T08:09:13.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.891+0000 7f0c9aa00640 1 -- 172.21.15.2:0/3293551881 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c94004440 con 0x7f0ca4102b90 2024-09-10T08:09:13.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.891+0000 7f0c9aa00640 1 -- 172.21.15.2:0/3293551881 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c94035070 con 0x7f0ca4102b90 2024-09-10T08:09:13.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.892+0000 7f0ca3200640 1 -- 172.21.15.2:0/3293551881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca41a7030 con 0x7f0ca4102b90 2024-09-10T08:09:13.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.895+0000 7f0cab7fb740 1 Processor -- start 2024-09-10T08:09:13.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.895+0000 7f0cab7fb740 1 -- start start 2024-09-10T08:09:13.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.895+0000 7f0cab7fb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a3e40 0x55b5b31a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.895+0000 7f0cab7fb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a4760 0x55b5b31aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.895+0000 7f0c9aa00640 1 -- 172.21.15.2:0/3293551881 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f0c94042030 con 0x7f0ca4102b90 2024-09-10T08:09:13.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.895+0000 7f0cab7fb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.895+0000 7f0cab7fb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b5b310a3e0 con 0x55b5b31a4760 2024-09-10T08:09:13.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.895+0000 7f0cab7fb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b5b30b1d80 con 0x55b5b31af020 2024-09-10T08:09:13.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.895+0000 7f0cab7fb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b5b31063a0 con 0x55b5b31a3e40 2024-09-10T08:09:13.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.896+0000 7f0ca1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.896+0000 7f0ca2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a3e40 0x55b5b31a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.896+0000 7f0ca1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38146/0 (socket says 172.21.15.2:38146) 2024-09-10T08:09:13.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.896+0000 7f0ca1400640 1 -- 172.21.15.2:0/3422008912 learned_addr learned my addr 172.21.15.2:0/3422008912 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:13.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.896+0000 7f0ca1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a4760 0x55b5b31aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.896+0000 7f0ca1400640 1 -- 172.21.15.2:0/3422008912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a3e40 msgr2=0x55b5b31a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:13.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.896+0000 7f0ca1400640 1 --2- 172.21.15.2:0/3422008912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a3e40 0x55b5b31a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.896+0000 7f0ca1400640 1 -- 172.21.15.2:0/3422008912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a4760 msgr2=0x55b5b31aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:13.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.896+0000 7f0ca1400640 1 --2- 172.21.15.2:0/3422008912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a4760 0x55b5b31aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.896+0000 7f0ca1400640 1 -- 172.21.15.2:0/3422008912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5b30d0ae0 con 0x55b5b31af020 2024-09-10T08:09:13.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.896+0000 7f0ca1400640 1 --2- 172.21.15.2:0/3422008912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31b1410 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009c90 tx=0x7f0c90011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ead807c26e84134 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:13.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.897+0000 7f0c8f400640 1 -- 172.21.15.2:0/3422008912 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90005220 con 0x55b5b31af020 2024-09-10T08:09:13.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.897+0000 7f0c8f400640 1 -- 172.21.15.2:0/3422008912 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c90004c80 con 0x55b5b31af020 2024-09-10T08:09:13.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.897+0000 7f0c8f400640 1 -- 172.21.15.2:0/3422008912 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90019830 con 0x55b5b31af020 2024-09-10T08:09:13.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.897+0000 7f0c9aa00640 1 --2- 172.21.15.2:0/3293551881 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0c880a2c10 0x7f0c880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.897+0000 7f0c9aa00640 1 -- 172.21.15.2:0/3293551881 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0c940ef970 con 0x7f0ca4102b90 2024-09-10T08:09:13.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.897+0000 7f0ca2800640 1 --2- 172.21.15.2:0/3293551881 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0c880a2c10 0x7f0c880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.897+0000 7f0cab7fb740 1 -- 172.21.15.2:0/3422008912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 msgr2=0x55b5b31b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:13.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.897+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/3422008912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31b1410 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009c90 tx=0x7f0c90011810 comp rx=0 tx=0).stop 2024-09-10T08:09:13.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.898+0000 7f0cab7fb740 1 -- 172.21.15.2:0/3422008912 shutdown_connections 2024-09-10T08:09:13.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.898+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/3422008912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31b1410 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.898+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/3422008912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a4760 0x55b5b31aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.898+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/3422008912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a3e40 0x55b5b31a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.898+0000 7f0cab7fb740 1 -- 172.21.15.2:0/3422008912 >> 172.21.15.2:0/3422008912 conn(0x55b5b31a03d0 msgr2=0x55b5b31a07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:13.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.898+0000 7f0cab7fb740 1 -- 172.21.15.2:0/3422008912 shutdown_connections 2024-09-10T08:09:13.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.898+0000 7f0cab7fb740 1 -- 172.21.15.2:0/3422008912 wait complete. 2024-09-10T08:09:13.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.898+0000 7f0ca2800640 1 --2- 172.21.15.2:0/3293551881 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0c880a2c10 0x7f0c880a50d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c005fd0 tx=0x7f0c9c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:13.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.898+0000 7f0cab7fb740 1 -- start start 2024-09-10T08:09:13.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.899+0000 7f0cab7fb740 1 Processor -- start 2024-09-10T08:09:13.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.899+0000 7f0cab7fb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a3e40 0x55b5b31abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.899+0000 7f0cab7fb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 0x55b5b31ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.899+0000 7f0cab7fb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31ac930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.899+0000 7f0cab7fb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b5b31b1bf0 con 0x55b5b31a3e40 2024-09-10T08:09:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.899+0000 7f0cab7fb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b5b30c50f0 con 0x55b5b31af020 2024-09-10T08:09:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.899+0000 7f0cab7fb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b5b31b1d70 con 0x55b5b31a4760 2024-09-10T08:09:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.899+0000 7f0cab7fb740 10 client.0 _ll_register_callbacks cb 0x55b5b30acbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.900+0000 7f0ca1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a3e40 0x55b5b31abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.900+0000 7f0ca1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 0x55b5b31ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.900+0000 7f0ca1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 0x55b5b31ac3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49908/0 (socket says 172.21.15.2:49908) 2024-09-10T08:09:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.900+0000 7f0ca1400640 1 -- 172.21.15.2:0/3186036414 learned_addr learned my addr 172.21.15.2:0/3186036414 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.900+0000 7f0ca1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a3e40 0x55b5b31abeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45280/0 (socket says 172.21.15.2:45280) 2024-09-10T08:09:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.900+0000 7f0ca2800640 1 --2- 172.21.15.2:0/3186036414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.900+0000 7f0ca1400640 1 -- 172.21.15.2:0/3186036414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 msgr2=0x55b5b31ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.900+0000 7f0ca1400640 1 --2- 172.21.15.2:0/3186036414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.900+0000 7f0ca1400640 1 -- 172.21.15.2:0/3186036414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a3e40 msgr2=0x55b5b31abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.900+0000 7f0ca1400640 1 --2- 172.21.15.2:0/3186036414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a3e40 0x55b5b31abeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.900+0000 7f0ca1400640 1 -- 172.21.15.2:0/3186036414 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5b2e7b3c0 con 0x55b5b31a4760 2024-09-10T08:09:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.901+0000 7f0ca1400640 1 --2- 172.21.15.2:0/3186036414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 0x55b5b31ac3f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0c9001b040 tx=0x7f0c90005200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba186c1eca65ea0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.901+0000 7f0c83e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:09:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.901+0000 7f0c83e00640 1 -- 172.21.15.2:0/3186036414 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90029070 con 0x55b5b31a4760 2024-09-10T08:09:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.901+0000 7f0cab7fb740 1 -- 172.21.15.2:0/3186036414 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c90004c80 con 0x55b5b31a4760 2024-09-10T08:09:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.901+0000 7f0c83e00640 1 -- 172.21.15.2:0/3186036414 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0c90019730 con 0x55b5b31a4760 2024-09-10T08:09:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.901+0000 7f0c83e00640 1 -- 172.21.15.2:0/3186036414 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0c90035050 con 0x55b5b31a4760 2024-09-10T08:09:13.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.901+0000 7f0cab7fb740 10 client.25453 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:13.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.901+0000 7f0cab7fb740 1 -- 172.21.15.2:0/3186036414 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b5b31bdbc0 con 0x55b5b31a4760 2024-09-10T08:09:13.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.902+0000 7f0c83e00640 1 -- 172.21.15.2:0/3186036414 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c90004c80 con 0x55b5b31a4760 2024-09-10T08:09:13.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.902+0000 7f0c83e00640 1 -- 172.21.15.2:0/3186036414 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0c90023730 con 0x55b5b31a4760 2024-09-10T08:09:13.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.902+0000 7f0cab7fb740 10 client.25453 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:13.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.902+0000 7f0cab7fb740 10 client.25453 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:09:13.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.902+0000 7f0cab7fb740 20 client.25453 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:13.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.902+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/3186036414 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b5b31bef10 0x55b5b31c1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.902+0000 7f0cab7fb740 4 client.25453 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:13.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.902+0000 7f0cab7fb740 1 -- 172.21.15.2:0/3186036414 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5b3117950 con 0x55b5b31bef10 2024-09-10T08:09:13.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.902+0000 7f0ca1e00640 1 --2- 172.21.15.2:0/3186036414 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b5b31bef10 0x55b5b31c1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.903+0000 7f0c83e00640 10 client.25453 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:09:13.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.903+0000 7f0ca1e00640 1 --2- 172.21.15.2:0/3186036414 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b5b31bef10 0x55b5b31c1300 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f503c6d3c1d9c2be server_cookie=9a281bd295d72c04 in_seq=0 out_seq=0 2024-09-10T08:09:13.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0c83e00640 1 -- 172.21.15.2:0/3186036414 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5b3117950 con 0x55b5b31bef10 2024-09-10T08:09:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0c83e00640 10 client.25453 handle_command_reply: tid=0 2024-09-10T08:09:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0cab7fb740 1 client.25453 shutdown 2024-09-10T08:09:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0cab7fb740 1 -- 172.21.15.2:0/3186036414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 msgr2=0x55b5b31ac3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/3186036414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 0x55b5b31ac3f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0c9001b040 tx=0x7f0c90005200 comp rx=0 tx=0).stop 2024-09-10T08:09:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0cab7fb740 1 -- 172.21.15.2:0/3186036414 shutdown_connections 2024-09-10T08:09:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/3186036414 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b5b31bef10 0x55b5b31c1300 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/3186036414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/3186036414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 0x55b5b31ac3f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0c83e00640 0 client.25453 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:09:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/3186036414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a3e40 0x55b5b31abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0cab7fb740 1 -- 172.21.15.2:0/3186036414 >> 172.21.15.2:0/3186036414 conn(0x55b5b31a03d0 msgr2=0x55b5b31a6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0cab7fb740 1 -- 172.21.15.2:0/3186036414 shutdown_connections 2024-09-10T08:09:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.904+0000 7f0cab7fb740 1 -- 172.21.15.2:0/3186036414 wait complete. 2024-09-10T08:09:13.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.905+0000 7f0cab7fb740 20 client.25453 trim_cache size 0 max 16384 2024-09-10T08:09:13.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:13 smithi155 ceph-mon[35405]: pgmap v419: 129 pgs: 129 active+clean; 667 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-10T08:09:13.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:13 smithi107 ceph-mon[35048]: pgmap v419: 129 pgs: 129 active+clean; 667 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-10T08:09:13.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.927+0000 7f0cab7fb740 1 Processor -- start 2024-09-10T08:09:13.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.927+0000 7f0cab7fb740 1 -- start start 2024-09-10T08:09:13.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.928+0000 7f0cab7fb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a3e40 0x55b5b31a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.928+0000 7f0cab7fb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a4760 0x55b5b31c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.928+0000 7f0cab7fb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.928+0000 7f0cab7fb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca41b36f0 con 0x55b5b31a4760 2024-09-10T08:09:13.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.928+0000 7f0cab7fb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca41b0fc0 con 0x55b5b31af020 2024-09-10T08:09:13.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.928+0000 7f0cab7fb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca41aee10 con 0x55b5b31a3e40 2024-09-10T08:09:13.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.928+0000 7f0ca1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a4760 0x55b5b31c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.928+0000 7f0ca1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a4760 0x55b5b31c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45284/0 (socket says 172.21.15.2:45284) 2024-09-10T08:09:13.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.928+0000 7f0ca1400640 1 -- 172.21.15.2:0/2369011596 learned_addr learned my addr 172.21.15.2:0/2369011596 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:13.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.928+0000 7f0ca1e00640 1 --2- 172.21.15.2:0/2369011596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a3e40 0x55b5b31a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.928+0000 7f0ca2800640 1 --2- 172.21.15.2:0/2369011596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.929+0000 7f0ca1400640 1 -- 172.21.15.2:0/2369011596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a3e40 msgr2=0x55b5b31a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:13.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.929+0000 7f0ca1400640 1 --2- 172.21.15.2:0/2369011596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a3e40 0x55b5b31a5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.929+0000 7f0ca1400640 1 -- 172.21.15.2:0/2369011596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 msgr2=0x55b5b31c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:13.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.929+0000 7f0ca1400640 1 --2- 172.21.15.2:0/2369011596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.929+0000 7f0ca1400640 1 -- 172.21.15.2:0/2369011596 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5b31a0200 con 0x55b5b31a4760 2024-09-10T08:09:13.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.929+0000 7f0ca1e00640 1 --2- 172.21.15.2:0/2369011596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a3e40 0x55b5b31a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:13.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.929+0000 7f0ca1400640 1 --2- 172.21.15.2:0/2369011596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a4760 0x55b5b31c6a20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0c90023c70 tx=0x7f0c90009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bb9835bd1f6198 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:13.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.929+0000 7f0c8ea00640 1 -- 172.21.15.2:0/2369011596 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90032030 con 0x55b5b31a4760 2024-09-10T08:09:13.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.929+0000 7f0c8ea00640 1 -- 172.21.15.2:0/2369011596 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c900194c0 con 0x55b5b31a4760 2024-09-10T08:09:13.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.929+0000 7f0c8ea00640 1 -- 172.21.15.2:0/2369011596 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90029070 con 0x55b5b31a4760 2024-09-10T08:09:13.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.930+0000 7f0cab7fb740 1 -- 172.21.15.2:0/2369011596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a4760 msgr2=0x55b5b31c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:13.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.930+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/2369011596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a4760 0x55b5b31c6a20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0c90023c70 tx=0x7f0c90009980 comp rx=0 tx=0).stop 2024-09-10T08:09:13.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.930+0000 7f0cab7fb740 1 -- 172.21.15.2:0/2369011596 shutdown_connections 2024-09-10T08:09:13.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.930+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/2369011596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.930+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/2369011596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a4760 0x55b5b31c6a20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.930+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/2369011596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a3e40 0x55b5b31a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.930+0000 7f0cab7fb740 1 -- 172.21.15.2:0/2369011596 >> 172.21.15.2:0/2369011596 conn(0x55b5b31a03d0 msgr2=0x55b5b31a5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:13.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.930+0000 7f0cab7fb740 1 -- 172.21.15.2:0/2369011596 shutdown_connections 2024-09-10T08:09:13.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.931+0000 7f0cab7fb740 1 -- 172.21.15.2:0/2369011596 wait complete. 2024-09-10T08:09:13.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.931+0000 7f0cab7fb740 1 -- start start 2024-09-10T08:09:13.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.931+0000 7f0cab7fb740 1 Processor -- start 2024-09-10T08:09:13.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0cab7fb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a3e40 0x55b5b31b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0cab7fb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 0x55b5b31b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0cab7fb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0cab7fb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca41af480 con 0x55b5b31a3e40 2024-09-10T08:09:13.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0cab7fb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca41b08f0 con 0x55b5b31af020 2024-09-10T08:09:13.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0cab7fb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca41afb50 con 0x55b5b31a4760 2024-09-10T08:09:13.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0cab7fb740 10 client.0 _ll_register_callbacks cb 0x7f0c9c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0ca1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 0x55b5b31b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0ca1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 0x55b5b31b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49922/0 (socket says 172.21.15.2:49922) 2024-09-10T08:09:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0ca1400640 1 -- 172.21.15.2:0/4177435648 learned_addr learned my addr 172.21.15.2:0/4177435648 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0ca2800640 1 --2- 172.21.15.2:0/4177435648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0ca1400640 1 -- 172.21.15.2:0/4177435648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 msgr2=0x55b5b31b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0ca1400640 1 --2- 172.21.15.2:0/4177435648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.932+0000 7f0ca1400640 1 -- 172.21.15.2:0/4177435648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a3e40 msgr2=0x55b5b31b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.933+0000 7f0ca1400640 1 --2- 172.21.15.2:0/4177435648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a3e40 0x55b5b31b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.933+0000 7f0ca1400640 1 -- 172.21.15.2:0/4177435648 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5b319fc80 con 0x55b5b31a4760 2024-09-10T08:09:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.933+0000 7f0ca1400640 1 --2- 172.21.15.2:0/4177435648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 0x55b5b31b3350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0c90011850 tx=0x7f0c90009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f446f8b8edf5162 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.933+0000 7f0c83e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:09:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.933+0000 7f0c83e00640 1 -- 172.21.15.2:0/4177435648 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90029050 con 0x55b5b31a4760 2024-09-10T08:09:13.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.933+0000 7f0c83e00640 1 -- 172.21.15.2:0/4177435648 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0c90023430 con 0x55b5b31a4760 2024-09-10T08:09:13.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.933+0000 7f0cab7fb740 1 -- 172.21.15.2:0/4177435648 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c900194c0 con 0x55b5b31a4760 2024-09-10T08:09:13.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.934+0000 7f0c83e00640 1 -- 172.21.15.2:0/4177435648 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0c9004e050 con 0x55b5b31a4760 2024-09-10T08:09:13.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.934+0000 7f0cab7fb740 10 client.25459 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:13.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.934+0000 7f0cab7fb740 1 -- 172.21.15.2:0/4177435648 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b5b3229c00 con 0x55b5b31a4760 2024-09-10T08:09:13.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.934+0000 7f0c83e00640 1 -- 172.21.15.2:0/4177435648 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c900194c0 con 0x55b5b31a4760 2024-09-10T08:09:13.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.934+0000 7f0c83e00640 1 -- 172.21.15.2:0/4177435648 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0c900236a0 con 0x55b5b31a4760 2024-09-10T08:09:13.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.934+0000 7f0cab7fb740 10 client.25459 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:13.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.934+0000 7f0cab7fb740 10 client.25459 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:09:13.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.934+0000 7f0cab7fb740 20 client.25459 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.934+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/4177435648 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b5b322a210 0x55b5b322c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.935+0000 7f0cab7fb740 4 client.25459 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:09:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.935+0000 7f0cab7fb740 1 -- 172.21.15.2:0/4177435648 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5b3111e60 con 0x55b5b322a210 2024-09-10T08:09:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.935+0000 7f0ca1e00640 1 --2- 172.21.15.2:0/4177435648 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b5b322a210 0x55b5b322c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.935+0000 7f0ca1e00640 1 --2- 172.21.15.2:0/4177435648 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b5b322a210 0x55b5b322c630 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93a166c5c4416a36 server_cookie=f928aac2d526f714 in_seq=0 out_seq=0 2024-09-10T08:09:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.935+0000 7f0c83e00640 10 client.25459 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:09:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0c83e00640 1 -- 172.21.15.2:0/4177435648 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5b3111e60 con 0x55b5b322a210 2024-09-10T08:09:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0c83e00640 10 client.25459 handle_command_reply: tid=0 2024-09-10T08:09:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0cab7fb740 1 client.25459 shutdown 2024-09-10T08:09:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0cab7fb740 1 -- 172.21.15.2:0/4177435648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 msgr2=0x55b5b31b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/4177435648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 0x55b5b31b3350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0c90011850 tx=0x7f0c90009f70 comp rx=0 tx=0).stop 2024-09-10T08:09:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0cab7fb740 1 -- 172.21.15.2:0/4177435648 shutdown_connections 2024-09-10T08:09:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/4177435648 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b5b322a210 0x55b5b322c630 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/4177435648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5b31af020 0x55b5b31b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/4177435648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5b31a4760 0x55b5b31b3350 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0cab7fb740 1 --2- 172.21.15.2:0/4177435648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b5b31a3e40 0x55b5b31b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0cab7fb740 1 -- 172.21.15.2:0/4177435648 >> 172.21.15.2:0/4177435648 conn(0x55b5b31a03d0 msgr2=0x55b5b31adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0c83e00640 0 client.25459 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:09:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0cab7fb740 1 -- 172.21.15.2:0/4177435648 shutdown_connections 2024-09-10T08:09:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0cab7fb740 1 -- 172.21.15.2:0/4177435648 wait complete. 2024-09-10T08:09:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.936+0000 7f0cab7fb740 20 client.25459 trim_cache size 0 max 16384 2024-09-10T08:09:13.939 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:09:13.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.937+0000 7f0ca3200640 1 -- 172.21.15.2:0/3293551881 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0c880a2c10 msgr2=0x7f0c880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:13.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.937+0000 7f0ca3200640 1 --2- 172.21.15.2:0/3293551881 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0c880a2c10 0x7f0c880a50d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c005fd0 tx=0x7f0c9c009500 comp rx=0 tx=0).stop 2024-09-10T08:09:13.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.937+0000 7f0ca3200640 1 -- 172.21.15.2:0/3293551881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ca4102b90 msgr2=0x7f0ca419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:13.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.937+0000 7f0ca3200640 1 --2- 172.21.15.2:0/3293551881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ca4102b90 0x7f0ca419a4b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0c94006010 tx=0x7f0c94004040 comp rx=0 tx=0).stop 2024-09-10T08:09:13.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.937+0000 7f0ca3200640 1 -- 172.21.15.2:0/3293551881 shutdown_connections 2024-09-10T08:09:13.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.937+0000 7f0ca3200640 1 --2- 172.21.15.2:0/3293551881 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0c880a2c10 0x7f0c880a50d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.937+0000 7f0ca3200640 1 --2- 172.21.15.2:0/3293551881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca410bfc0 0x7f0ca419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.937+0000 7f0ca3200640 1 --2- 172.21.15.2:0/3293551881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ca41034b0 0x7f0ca419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.937+0000 7f0ca3200640 1 --2- 172.21.15.2:0/3293551881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ca4102b90 0x7f0ca419a4b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:13.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.937+0000 7f0ca3200640 1 -- 172.21.15.2:0/3293551881 >> 172.21.15.2:0/3293551881 conn(0x7f0ca40fc610 msgr2=0x7f0ca410d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:13.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.938+0000 7f0ca3200640 1 -- 172.21.15.2:0/3293551881 shutdown_connections 2024-09-10T08:09:13.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:13.938+0000 7f0ca3200640 1 -- 172.21.15.2:0/3293551881 wait complete. 2024-09-10T08:09:14.320 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'db781f90', quiesce took 5.8 seconds. Will release after: 24.2 2024-09-10T08:09:15.106 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:09:15.498 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:15.792 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:15 smithi002 ceph-mon[30563]: pgmap v420: 129 pgs: 129 active+clean; 667 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:09:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:15 smithi155 ceph-mon[35405]: pgmap v420: 129 pgs: 129 active+clean; 667 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:09:15.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:15 smithi107 ceph-mon[35048]: pgmap v420: 129 pgs: 129 active+clean; 667 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:09:16.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.098+0000 7f4807e00640 1 -- 172.21.15.2:0/1572440195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48081034b0 msgr2=0x7f480810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:16.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.098+0000 7f4807e00640 1 --2- 172.21.15.2:0/1572440195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48081034b0 0x7f480810b9a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009900 tx=0x7f47f402a480 comp rx=0 tx=0).stop 2024-09-10T08:09:16.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.098+0000 7f4807e00640 1 -- 172.21.15.2:0/1572440195 shutdown_connections 2024-09-10T08:09:16.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.098+0000 7f4807e00640 1 --2- 172.21.15.2:0/1572440195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f480810bfc0 0x7f480810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.098+0000 7f4807e00640 1 --2- 172.21.15.2:0/1572440195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48081034b0 0x7f480810b9a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.098+0000 7f4807e00640 1 --2- 172.21.15.2:0/1572440195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4808102b90 0x7f4808102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.098+0000 7f4807e00640 1 -- 172.21.15.2:0/1572440195 >> 172.21.15.2:0/1572440195 conn(0x7f48080fc610 msgr2=0x7f48080fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:16.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.098+0000 7f4807e00640 1 -- 172.21.15.2:0/1572440195 shutdown_connections 2024-09-10T08:09:16.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.098+0000 7f4807e00640 1 -- 172.21.15.2:0/1572440195 wait complete. 2024-09-10T08:09:16.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.100+0000 7f4807e00640 1 Processor -- start 2024-09-10T08:09:16.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.100+0000 7f4807e00640 1 -- start start 2024-09-10T08:09:16.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.100+0000 7f4807e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4808102b90 0x7f480819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.100+0000 7f4807e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48081034b0 0x7f480819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.100+0000 7f4807e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f480810bfc0 0x7f480819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.100+0000 7f4807e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4808111e10 con 0x7f480810bfc0 2024-09-10T08:09:16.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.100+0000 7f4807e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4808111c90 con 0x7f48081034b0 2024-09-10T08:09:16.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.100+0000 7f4807e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4808111f90 con 0x7f4808102b90 2024-09-10T08:09:16.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.100+0000 7f4806000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48081034b0 0x7f480819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.101+0000 7f4806a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4808102b90 0x7f480819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.101+0000 7f4806000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48081034b0 0x7f480819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38192/0 (socket says 172.21.15.2:38192) 2024-09-10T08:09:16.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.101+0000 7f4806000640 1 -- 172.21.15.2:0/1378177579 learned_addr learned my addr 172.21.15.2:0/1378177579 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:16.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.101+0000 7f4807400640 1 --2- 172.21.15.2:0/1378177579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f480810bfc0 0x7f480819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.101+0000 7f4806000640 1 -- 172.21.15.2:0/1378177579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4808102b90 msgr2=0x7f480819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:16.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.101+0000 7f4806000640 1 --2- 172.21.15.2:0/1378177579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4808102b90 0x7f480819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.101+0000 7f4806000640 1 -- 172.21.15.2:0/1378177579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f480810bfc0 msgr2=0x7f480819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:16.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.101+0000 7f4806000640 1 --2- 172.21.15.2:0/1378177579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f480810bfc0 0x7f480819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.101+0000 7f4806000640 1 -- 172.21.15.2:0/1378177579 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480819f480 con 0x7f48081034b0 2024-09-10T08:09:16.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.101+0000 7f4806000640 1 --2- 172.21.15.2:0/1378177579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48081034b0 0x7f480819aa10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f47f4004770 tx=0x7f47f4003b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:16.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.102+0000 7f480ca00640 1 -- 172.21.15.2:0/1378177579 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f403f070 con 0x7f48081034b0 2024-09-10T08:09:16.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.102+0000 7f480ca00640 1 -- 172.21.15.2:0/1378177579 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f4031c90 con 0x7f48081034b0 2024-09-10T08:09:16.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.102+0000 7f4807e00640 1 -- 172.21.15.2:0/1378177579 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480819f710 con 0x7f48081034b0 2024-09-10T08:09:16.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.102+0000 7f480ca00640 1 -- 172.21.15.2:0/1378177579 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4035030 con 0x7f48081034b0 2024-09-10T08:09:16.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.102+0000 7f4807e00640 1 -- 172.21.15.2:0/1378177579 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48081a7050 con 0x7f48081034b0 2024-09-10T08:09:16.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.105+0000 7f4810862740 1 Processor -- start 2024-09-10T08:09:16.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.105+0000 7f4810862740 1 -- start start 2024-09-10T08:09:16.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.105+0000 7f480ca00640 1 -- 172.21.15.2:0/1378177579 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f47f4042070 con 0x7f48081034b0 2024-09-10T08:09:16.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.106+0000 7f4810862740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4cff220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.106+0000 7f4810862740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d09ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.106+0000 7f4810862740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4d0a020 0x5569a4d0c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.106+0000 7f4810862740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5569a4c653e0 con 0x5569a4d0a020 2024-09-10T08:09:16.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.106+0000 7f4810862740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5569a4c0cd80 con 0x5569a4cfee40 2024-09-10T08:09:16.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.106+0000 7f4810862740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5569a4c613a0 con 0x5569a4cff760 2024-09-10T08:09:16.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.106+0000 7f4806a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d09ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.106+0000 7f4806000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4d0a020 0x5569a4d0c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.106+0000 7f4806a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d09ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49950/0 (socket says 172.21.15.2:49950) 2024-09-10T08:09:16.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.106+0000 7f4806a00640 1 -- 172.21.15.2:0/420799457 learned_addr learned my addr 172.21.15.2:0/420799457 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:16.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.106+0000 7f4807400640 1 --2- 172.21.15.2:0/420799457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4cff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.107+0000 7f4806a00640 1 -- 172.21.15.2:0/420799457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 msgr2=0x5569a4cff220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:16.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.107+0000 7f4806a00640 1 --2- 172.21.15.2:0/420799457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4cff220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.107+0000 7f4806a00640 1 -- 172.21.15.2:0/420799457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4d0a020 msgr2=0x5569a4d0c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:16.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.107+0000 7f4806a00640 1 --2- 172.21.15.2:0/420799457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4d0a020 0x5569a4d0c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.107+0000 7f4806a00640 1 -- 172.21.15.2:0/420799457 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a4c2bae0 con 0x5569a4cff760 2024-09-10T08:09:16.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.107+0000 7f4806a00640 1 --2- 172.21.15.2:0/420799457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d09ae0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009cf0 tx=0x7f47f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb77ccba5e686114 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:16.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.107+0000 7f47f3e00640 1 -- 172.21.15.2:0/420799457 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8004230 con 0x5569a4cff760 2024-09-10T08:09:16.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.107+0000 7f47f3e00640 1 -- 172.21.15.2:0/420799457 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f80043d0 con 0x5569a4cff760 2024-09-10T08:09:16.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.107+0000 7f47f3e00640 1 -- 172.21.15.2:0/420799457 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8019960 con 0x5569a4cff760 2024-09-10T08:09:16.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.107+0000 7f480ca00640 1 --2- 172.21.15.2:0/1378177579 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47ec0a2c10 0x7f47ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.108+0000 7f480ca00640 1 -- 172.21.15.2:0/1378177579 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f47f40efe10 con 0x7f48081034b0 2024-09-10T08:09:16.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.108+0000 7f4807400640 1 --2- 172.21.15.2:0/1378177579 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47ec0a2c10 0x7f47ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.108+0000 7f4810862740 1 -- 172.21.15.2:0/420799457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 msgr2=0x5569a4d09ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:16.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.108+0000 7f4810862740 1 --2- 172.21.15.2:0/420799457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d09ae0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009cf0 tx=0x7f47f8011810 comp rx=0 tx=0).stop 2024-09-10T08:09:16.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.108+0000 7f4810862740 1 -- 172.21.15.2:0/420799457 shutdown_connections 2024-09-10T08:09:16.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.108+0000 7f4810862740 1 --2- 172.21.15.2:0/420799457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4d0a020 0x5569a4d0c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.108+0000 7f4810862740 1 --2- 172.21.15.2:0/420799457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d09ae0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.108+0000 7f4810862740 1 --2- 172.21.15.2:0/420799457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4cff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.108+0000 7f4810862740 1 -- 172.21.15.2:0/420799457 >> 172.21.15.2:0/420799457 conn(0x5569a4cfb3d0 msgr2=0x5569a4cfb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:16.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.109+0000 7f4810862740 1 -- 172.21.15.2:0/420799457 shutdown_connections 2024-09-10T08:09:16.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.109+0000 7f4810862740 1 -- 172.21.15.2:0/420799457 wait complete. 2024-09-10T08:09:16.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.109+0000 7f4807400640 1 --2- 172.21.15.2:0/1378177579 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47ec0a2c10 0x7f47ec0a50d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4800005fd0 tx=0x7f4800009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:16.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.109+0000 7f4810862740 1 -- start start 2024-09-10T08:09:16.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.110+0000 7f4810862740 1 Processor -- start 2024-09-10T08:09:16.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.110+0000 7f4810862740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4d06e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.110+0000 7f4810862740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cff760 0x5569a4d073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.110+0000 7f4810862740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4d0a020 0x5569a4d07900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.110+0000 7f4810862740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5569a4d0cbf0 con 0x5569a4cff760 2024-09-10T08:09:16.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.110+0000 7f4810862740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5569a4c200f0 con 0x5569a4cfee40 2024-09-10T08:09:16.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.110+0000 7f4810862740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5569a4d0cd70 con 0x5569a4d0a020 2024-09-10T08:09:16.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.110+0000 7f4810862740 10 client.0 _ll_register_callbacks cb 0x5569a4c07bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:16.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.110+0000 7f4806000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cff760 0x5569a4d073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.110+0000 7f4806a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4d06e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.111+0000 7f4806000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cff760 0x5569a4d073c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45310/0 (socket says 172.21.15.2:45310) 2024-09-10T08:09:16.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.111+0000 7f4806000640 1 -- 172.21.15.2:0/892334776 learned_addr learned my addr 172.21.15.2:0/892334776 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:16.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.111+0000 7f4807400640 1 --2- 172.21.15.2:0/892334776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4d0a020 0x5569a4d07900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.111+0000 7f4806a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4d06e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38206/0 (socket says 172.21.15.2:38206) 2024-09-10T08:09:16.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.111+0000 7f4806000640 1 -- 172.21.15.2:0/892334776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4d0a020 msgr2=0x5569a4d07900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:16.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.111+0000 7f4806000640 1 --2- 172.21.15.2:0/892334776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4d0a020 0x5569a4d07900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.111+0000 7f4806000640 1 -- 172.21.15.2:0/892334776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 msgr2=0x5569a4d06e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:16.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.111+0000 7f4806000640 1 --2- 172.21.15.2:0/892334776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4d06e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.111+0000 7f4806000640 1 -- 172.21.15.2:0/892334776 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a49d63c0 con 0x5569a4cff760 2024-09-10T08:09:16.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.111+0000 7f4807400640 1 --2- 172.21.15.2:0/892334776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4d0a020 0x5569a4d07900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:16.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.111+0000 7f4806000640 1 --2- 172.21.15.2:0/892334776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cff760 0x5569a4d073c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f47f40b8580 tx=0x7f47f40f3550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63cd22ccd249021 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:16.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.111+0000 7f47f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:09:16.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.112+0000 7f47f0c00640 1 -- 172.21.15.2:0/892334776 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4049030 con 0x5569a4cff760 2024-09-10T08:09:16.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.112+0000 7f47f0c00640 1 -- 172.21.15.2:0/892334776 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f47f40b8790 con 0x5569a4cff760 2024-09-10T08:09:16.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.112+0000 7f4810862740 1 -- 172.21.15.2:0/892334776 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47f80043d0 con 0x5569a4cff760 2024-09-10T08:09:16.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.112+0000 7f47f0c00640 1 -- 172.21.15.2:0/892334776 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47f40bbef0 con 0x5569a4cff760 2024-09-10T08:09:16.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.112+0000 7f4810862740 10 client.15468 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:16.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.112+0000 7f4810862740 1 -- 172.21.15.2:0/892334776 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5569a4d18be0 con 0x5569a4cff760 2024-09-10T08:09:16.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.113+0000 7f47f0c00640 1 -- 172.21.15.2:0/892334776 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f80043d0 con 0x5569a4cff760 2024-09-10T08:09:16.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.113+0000 7f47f0c00640 1 -- 172.21.15.2:0/892334776 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f47f40bc380 con 0x5569a4cff760 2024-09-10T08:09:16.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.113+0000 7f4810862740 10 client.15468 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:16.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.113+0000 7f4810862740 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:16.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.113+0000 7f4810862740 20 client.15468 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:16.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.113+0000 7f4810862740 1 --2- 172.21.15.2:0/892334776 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5569a4d1a090 0x5569a4d1c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.113+0000 7f4810862740 4 client.15468 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:16.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.113+0000 7f4810862740 1 -- 172.21.15.2:0/892334776 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569a4c72950 con 0x5569a4d1a090 2024-09-10T08:09:16.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.114+0000 7f4806a00640 1 --2- 172.21.15.2:0/892334776 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5569a4d1a090 0x5569a4d1c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.114+0000 7f4806a00640 1 --2- 172.21.15.2:0/892334776 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5569a4d1a090 0x5569a4d1c4b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba0246da6eed16a6 server_cookie=dcc4a555f1763fcd in_seq=0 out_seq=0 2024-09-10T08:09:16.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.114+0000 7f47f0c00640 10 client.15468 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:16.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.115+0000 7f47f0c00640 1 -- 172.21.15.2:0/892334776 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569a4c72950 con 0x5569a4d1a090 2024-09-10T08:09:16.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.115+0000 7f47f0c00640 10 client.15468 handle_command_reply: tid=0 2024-09-10T08:09:16.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.115+0000 7f4810862740 1 client.15468 shutdown 2024-09-10T08:09:16.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.115+0000 7f4810862740 1 -- 172.21.15.2:0/892334776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cff760 msgr2=0x5569a4d073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:16.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.115+0000 7f4810862740 1 --2- 172.21.15.2:0/892334776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cff760 0x5569a4d073c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f47f40b8580 tx=0x7f47f40f3550 comp rx=0 tx=0).stop 2024-09-10T08:09:16.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.115+0000 7f4810862740 1 -- 172.21.15.2:0/892334776 shutdown_connections 2024-09-10T08:09:16.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.115+0000 7f4810862740 1 --2- 172.21.15.2:0/892334776 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5569a4d1a090 0x5569a4d1c4b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.115+0000 7f4810862740 1 --2- 172.21.15.2:0/892334776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4d0a020 0x5569a4d07900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.115+0000 7f4810862740 1 --2- 172.21.15.2:0/892334776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cff760 0x5569a4d073c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.115+0000 7f4810862740 1 --2- 172.21.15.2:0/892334776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4d06e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.115+0000 7f4810862740 1 -- 172.21.15.2:0/892334776 >> 172.21.15.2:0/892334776 conn(0x5569a4cfb3d0 msgr2=0x5569a4d010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:16.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.116+0000 7f47f0c00640 0 client.15468 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:16.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.116+0000 7f4810862740 1 -- 172.21.15.2:0/892334776 shutdown_connections 2024-09-10T08:09:16.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.116+0000 7f4810862740 1 -- 172.21.15.2:0/892334776 wait complete. 2024-09-10T08:09:16.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.116+0000 7f4810862740 20 client.15468 trim_cache size 0 max 16384 2024-09-10T08:09:16.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.141+0000 7f4810862740 1 Processor -- start 2024-09-10T08:09:16.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.141+0000 7f4810862740 1 -- start start 2024-09-10T08:09:16.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.142+0000 7f4810862740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cfee40 0x5569a4cf6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.142+0000 7f4810862740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d08df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.142+0000 7f4810862740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4d0a020 0x5569a4d21d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.142+0000 7f4810862740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b37d0 con 0x5569a4cfee40 2024-09-10T08:09:16.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.142+0000 7f4810862740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b10a0 con 0x5569a4d0a020 2024-09-10T08:09:16.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.142+0000 7f4810862740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f48081aeef0 con 0x5569a4cff760 2024-09-10T08:09:16.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.142+0000 7f4806a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cfee40 0x5569a4cf6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.142+0000 7f4806a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cfee40 0x5569a4cf6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45316/0 (socket says 172.21.15.2:45316) 2024-09-10T08:09:16.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.142+0000 7f4806a00640 1 -- 172.21.15.2:0/268145343 learned_addr learned my addr 172.21.15.2:0/268145343 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:16.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.142+0000 7f4806000640 1 --2- 172.21.15.2:0/268145343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d08df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.142+0000 7f4807400640 1 --2- 172.21.15.2:0/268145343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4d0a020 0x5569a4d21d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.143+0000 7f4806a00640 1 -- 172.21.15.2:0/268145343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 msgr2=0x5569a4d08df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:16.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.143+0000 7f4806a00640 1 --2- 172.21.15.2:0/268145343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d08df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.143+0000 7f4806a00640 1 -- 172.21.15.2:0/268145343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4d0a020 msgr2=0x5569a4d21d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:16.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.143+0000 7f4806a00640 1 --2- 172.21.15.2:0/268145343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4d0a020 0x5569a4d21d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.143+0000 7f4806a00640 1 -- 172.21.15.2:0/268145343 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a4d05e30 con 0x5569a4cfee40 2024-09-10T08:09:16.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.143+0000 7f4807400640 1 --2- 172.21.15.2:0/268145343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4d0a020 0x5569a4d21d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:16.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.143+0000 7f4806a00640 1 --2- 172.21.15.2:0/268145343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cfee40 0x5569a4cf6300 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f47f8005fd0 tx=0x7f47f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa124c59865e9a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:16.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.143+0000 7f47f3400640 1 -- 172.21.15.2:0/268145343 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8031070 con 0x5569a4cfee40 2024-09-10T08:09:16.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.143+0000 7f47f3400640 1 -- 172.21.15.2:0/268145343 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f80035f0 con 0x5569a4cfee40 2024-09-10T08:09:16.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.143+0000 7f47f3400640 1 -- 172.21.15.2:0/268145343 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8019790 con 0x5569a4cfee40 2024-09-10T08:09:16.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.144+0000 7f4810862740 1 -- 172.21.15.2:0/268145343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cfee40 msgr2=0x5569a4cf6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:16.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.144+0000 7f4810862740 1 --2- 172.21.15.2:0/268145343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cfee40 0x5569a4cf6300 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f47f8005fd0 tx=0x7f47f8011810 comp rx=0 tx=0).stop 2024-09-10T08:09:16.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.144+0000 7f4810862740 1 -- 172.21.15.2:0/268145343 shutdown_connections 2024-09-10T08:09:16.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.144+0000 7f4810862740 1 --2- 172.21.15.2:0/268145343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4d0a020 0x5569a4d21d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.144+0000 7f4810862740 1 --2- 172.21.15.2:0/268145343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d08df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.144+0000 7f4810862740 1 --2- 172.21.15.2:0/268145343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4cfee40 0x5569a4cf6300 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.144+0000 7f4810862740 1 -- 172.21.15.2:0/268145343 >> 172.21.15.2:0/268145343 conn(0x5569a4cfb3d0 msgr2=0x5569a4d086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:16.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.144+0000 7f4810862740 1 -- 172.21.15.2:0/268145343 shutdown_connections 2024-09-10T08:09:16.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.144+0000 7f4810862740 1 -- 172.21.15.2:0/268145343 wait complete. 2024-09-10T08:09:16.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.145+0000 7f4810862740 1 -- start start 2024-09-10T08:09:16.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.145+0000 7f4810862740 1 Processor -- start 2024-09-10T08:09:16.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.145+0000 7f4810862740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4d0dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4810862740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d0e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4810862740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4d0a020 0x5569a4d0e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4810862740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f48081af560 con 0x5569a4d0a020 2024-09-10T08:09:16.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4810862740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b09d0 con 0x5569a4cfee40 2024-09-10T08:09:16.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4810862740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f48081afc30 con 0x5569a4cff760 2024-09-10T08:09:16.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4810862740 10 client.0 _ll_register_callbacks cb 0x7f47f4009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:16.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4806a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4d0dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4806a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4d0dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38224/0 (socket says 172.21.15.2:38224) 2024-09-10T08:09:16.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4806a00640 1 -- 172.21.15.2:0/3252548857 learned_addr learned my addr 172.21.15.2:0/3252548857 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:16.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4806000640 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d0e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4807400640 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4d0a020 0x5569a4d0e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4806a00640 1 -- 172.21.15.2:0/3252548857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 msgr2=0x5569a4d0e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:16.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4806a00640 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d0e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4806a00640 1 -- 172.21.15.2:0/3252548857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4d0a020 msgr2=0x5569a4d0e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:16.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4806a00640 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4d0a020 0x5569a4d0e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4806a00640 1 -- 172.21.15.2:0/3252548857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a4d83d50 con 0x5569a4cfee40 2024-09-10T08:09:16.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4806000640 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d0e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:16.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.146+0000 7f4807400640 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4d0a020 0x5569a4d0e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:16.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.147+0000 7f4806a00640 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4d0dce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009ee0 tx=0x7f47f8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cc1644c463a82c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:16.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.147+0000 7f47f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:09:16.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.147+0000 7f47f0c00640 1 -- 172.21.15.2:0/3252548857 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8039020 con 0x5569a4cfee40 2024-09-10T08:09:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.147+0000 7f47f0c00640 1 -- 172.21.15.2:0/3252548857 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f47f80039a0 con 0x5569a4cfee40 2024-09-10T08:09:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.147+0000 7f4810862740 1 -- 172.21.15.2:0/3252548857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47f80035f0 con 0x5569a4cfee40 2024-09-10T08:09:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.147+0000 7f47f0c00640 1 -- 172.21.15.2:0/3252548857 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47f8044050 con 0x5569a4cfee40 2024-09-10T08:09:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.148+0000 7f4810862740 10 client.25502 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.148+0000 7f4810862740 1 -- 172.21.15.2:0/3252548857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5569a4d84fd0 con 0x5569a4cfee40 2024-09-10T08:09:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.148+0000 7f47f0c00640 1 -- 172.21.15.2:0/3252548857 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f80035f0 con 0x5569a4cfee40 2024-09-10T08:09:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.148+0000 7f47f0c00640 1 -- 172.21.15.2:0/3252548857 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f47f8034070 con 0x5569a4cfee40 2024-09-10T08:09:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.148+0000 7f4810862740 10 client.25502 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.148+0000 7f4810862740 10 client.25502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.148+0000 7f4810862740 20 client.25502 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.148+0000 7f4810862740 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5569a4d854f0 0x5569a4d878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.148+0000 7f4810862740 4 client.25502 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:09:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.148+0000 7f4810862740 1 -- 172.21.15.2:0/3252548857 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569a4c6ce60 con 0x5569a4d854f0 2024-09-10T08:09:16.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.149+0000 7f4806000640 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5569a4d854f0 0x5569a4d878e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:16.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.149+0000 7f4806000640 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5569a4d854f0 0x5569a4d878e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8803593f87a5a1ea server_cookie=fca69563c69d3033 in_seq=0 out_seq=0 2024-09-10T08:09:16.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.149+0000 7f47f0c00640 10 client.25502 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:16.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.149+0000 7f47f0c00640 1 -- 172.21.15.2:0/3252548857 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569a4c6ce60 con 0x5569a4d854f0 2024-09-10T08:09:16.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f47f0c00640 10 client.25502 handle_command_reply: tid=0 2024-09-10T08:09:16.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f4810862740 1 client.25502 shutdown 2024-09-10T08:09:16.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f4810862740 1 -- 172.21.15.2:0/3252548857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 msgr2=0x5569a4d0dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:16.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f4810862740 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4d0dce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009ee0 tx=0x7f47f8009310 comp rx=0 tx=0).stop 2024-09-10T08:09:16.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f4810862740 1 -- 172.21.15.2:0/3252548857 shutdown_connections 2024-09-10T08:09:16.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f4810862740 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5569a4d854f0 0x5569a4d878e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f4810862740 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569a4d0a020 0x5569a4d0e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f4810862740 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a4cff760 0x5569a4d0e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f4810862740 1 --2- 172.21.15.2:0/3252548857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569a4cfee40 0x5569a4d0dce0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f4810862740 1 -- 172.21.15.2:0/3252548857 >> 172.21.15.2:0/3252548857 conn(0x5569a4cfb3d0 msgr2=0x5569a4d07660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:16.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f47f0c00640 0 client.25502 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:16.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f4810862740 1 -- 172.21.15.2:0/3252548857 shutdown_connections 2024-09-10T08:09:16.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f4810862740 1 -- 172.21.15.2:0/3252548857 wait complete. 2024-09-10T08:09:16.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.150+0000 7f4810862740 20 client.25502 trim_cache size 0 max 16384 2024-09-10T08:09:16.158 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:09:16.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.151+0000 7f4807e00640 1 -- 172.21.15.2:0/1378177579 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47ec0a2c10 msgr2=0x7f47ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:16.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.151+0000 7f4807e00640 1 --2- 172.21.15.2:0/1378177579 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47ec0a2c10 0x7f47ec0a50d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4800005fd0 tx=0x7f4800009500 comp rx=0 tx=0).stop 2024-09-10T08:09:16.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.151+0000 7f4807e00640 1 -- 172.21.15.2:0/1378177579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48081034b0 msgr2=0x7f480819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:16.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.151+0000 7f4807e00640 1 --2- 172.21.15.2:0/1378177579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48081034b0 0x7f480819aa10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f47f4004770 tx=0x7f47f4003b60 comp rx=0 tx=0).stop 2024-09-10T08:09:16.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.151+0000 7f4807e00640 1 -- 172.21.15.2:0/1378177579 shutdown_connections 2024-09-10T08:09:16.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.151+0000 7f4807e00640 1 --2- 172.21.15.2:0/1378177579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f480810bfc0 0x7f480819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.151+0000 7f4807e00640 1 --2- 172.21.15.2:0/1378177579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48081034b0 0x7f480819aa10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.151+0000 7f4807e00640 1 --2- 172.21.15.2:0/1378177579 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47ec0a2c10 0x7f47ec0a50d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.151+0000 7f4807e00640 1 --2- 172.21.15.2:0/1378177579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4808102b90 0x7f480819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:16.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.151+0000 7f4807e00640 1 -- 172.21.15.2:0/1378177579 >> 172.21.15.2:0/1378177579 conn(0x7f48080fc610 msgr2=0x7f4808101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:16.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.151+0000 7f4807e00640 1 -- 172.21.15.2:0/1378177579 shutdown_connections 2024-09-10T08:09:16.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:16.151+0000 7f4807e00640 1 -- 172.21.15.2:0/1378177579 wait complete. 2024-09-10T08:09:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:17 smithi155 ceph-mon[35405]: pgmap v421: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:09:17.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:17 smithi107 ceph-mon[35048]: pgmap v421: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:09:17.927 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:09:17.927 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:09:17.977 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:17 smithi002 ceph-mon[30563]: pgmap v421: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:09:18.359 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:19.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.011+0000 7f9e2f200640 1 -- 172.21.15.2:0/2451333587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e30105d90 msgr2=0x7f9e30106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:19.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.011+0000 7f9e2f200640 1 --2- 172.21.15.2:0/2451333587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e30105d90 0x7f9e30106220 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009be0 tx=0x7f9e2802a750 comp rx=0 tx=0).stop 2024-09-10T08:09:19.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.011+0000 7f9e2f200640 1 -- 172.21.15.2:0/2451333587 shutdown_connections 2024-09-10T08:09:19.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.011+0000 7f9e2f200640 1 --2- 172.21.15.2:0/2451333587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e30105d90 0x7f9e30106220 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.011+0000 7f9e2f200640 1 --2- 172.21.15.2:0/2451333587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e301012a0 0x7f9e30105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.011+0000 7f9e2f200640 1 --2- 172.21.15.2:0/2451333587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e30100980 0x7f9e30100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.011+0000 7f9e2f200640 1 -- 172.21.15.2:0/2451333587 >> 172.21.15.2:0/2451333587 conn(0x7f9e300fc610 msgr2=0x7f9e300fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:19.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.011+0000 7f9e2f200640 1 -- 172.21.15.2:0/2451333587 shutdown_connections 2024-09-10T08:09:19.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.011+0000 7f9e2f200640 1 -- 172.21.15.2:0/2451333587 wait complete. 2024-09-10T08:09:19.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.012+0000 7f9e2f200640 1 Processor -- start 2024-09-10T08:09:19.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.013+0000 7f9e2f200640 1 -- start start 2024-09-10T08:09:19.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.013+0000 7f9e2f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e30100980 0x7f9e301a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.013+0000 7f9e2f200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e301012a0 0x7f9e301a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.013+0000 7f9e2f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e30105d90 0x7f9e3019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.013+0000 7f9e2f200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30118310 con 0x7f9e301012a0 2024-09-10T08:09:19.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.013+0000 7f9e2f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30118190 con 0x7f9e30100980 2024-09-10T08:09:19.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.013+0000 7f9e2f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30118490 con 0x7f9e30105d90 2024-09-10T08:09:19.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.013+0000 7f9e2de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e30100980 0x7f9e301a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.013+0000 7f9e2de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e30100980 0x7f9e301a4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37566/0 (socket says 172.21.15.2:37566) 2024-09-10T08:09:19.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.013+0000 7f9e2de00640 1 -- 172.21.15.2:0/1172151441 learned_addr learned my addr 172.21.15.2:0/1172151441 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:19.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.014+0000 7f9e2e800640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e30105d90 0x7f9e3019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.014+0000 7f9e2d400640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e301012a0 0x7f9e301a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.014+0000 7f9e2de00640 1 -- 172.21.15.2:0/1172151441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e30105d90 msgr2=0x7f9e3019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:19.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.014+0000 7f9e2de00640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e30105d90 0x7f9e3019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.014+0000 7f9e2de00640 1 -- 172.21.15.2:0/1172151441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e301012a0 msgr2=0x7f9e301a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:19.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.014+0000 7f9e2de00640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e301012a0 0x7f9e301a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.014+0000 7f9e2de00640 1 -- 172.21.15.2:0/1172151441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3019eed0 con 0x7f9e30100980 2024-09-10T08:09:19.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.014+0000 7f9e2d400640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e301012a0 0x7f9e301a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:09:19.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.015+0000 7f9e2de00640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e30100980 0x7f9e301a4590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000d770 tx=0x7f9e2000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:19.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.015+0000 7f9e26a00640 1 -- 172.21.15.2:0/1172151441 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e20017070 con 0x7f9e30100980 2024-09-10T08:09:19.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.015+0000 7f9e2f200640 1 -- 172.21.15.2:0/1172151441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3019f1c0 con 0x7f9e30100980 2024-09-10T08:09:19.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.015+0000 7f9e26a00640 1 -- 172.21.15.2:0/1172151441 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e20005500 con 0x7f9e30100980 2024-09-10T08:09:19.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.015+0000 7f9e26a00640 1 -- 172.21.15.2:0/1172151441 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e20004e00 con 0x7f9e30100980 2024-09-10T08:09:19.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.015+0000 7f9e2f200640 1 -- 172.21.15.2:0/1172151441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e301ab510 con 0x7f9e30100980 2024-09-10T08:09:19.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.020+0000 7f9e3785c740 1 Processor -- start 2024-09-10T08:09:19.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.020+0000 7f9e3785c740 1 -- start start 2024-09-10T08:09:19.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.020+0000 7f9e3785c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cc9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.020+0000 7f9e3785c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc95d0 0x556357cd3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.020+0000 7f9e3785c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cd3e90 0x556357cd6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.020+0000 7f9e3785c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556357c2f290 con 0x556357cc8cb0 2024-09-10T08:09:19.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.020+0000 7f9e3785c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556357bd6b40 con 0x556357cc95d0 2024-09-10T08:09:19.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.020+0000 7f9e3785c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556357c2b250 con 0x556357cd3e90 2024-09-10T08:09:19.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.020+0000 7f9e26a00640 1 -- 172.21.15.2:0/1172151441 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f9e20004fa0 con 0x7f9e30100980 2024-09-10T08:09:19.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.021+0000 7f9e2de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc95d0 0x556357cd3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.021+0000 7f9e2de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc95d0 0x556357cd3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37582/0 (socket says 172.21.15.2:37582) 2024-09-10T08:09:19.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.021+0000 7f9e2de00640 1 -- 172.21.15.2:0/1606484501 learned_addr learned my addr 172.21.15.2:0/1606484501 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:19.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.021+0000 7f9e2e800640 1 --2- 172.21.15.2:0/1606484501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cc9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.021+0000 7f9e2d400640 1 --2- 172.21.15.2:0/1606484501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cd3e90 0x556357cd6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.022+0000 7f9e2de00640 1 -- 172.21.15.2:0/1606484501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cd3e90 msgr2=0x556357cd6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:19.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.022+0000 7f9e2de00640 1 --2- 172.21.15.2:0/1606484501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cd3e90 0x556357cd6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.022+0000 7f9e2de00640 1 -- 172.21.15.2:0/1606484501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 msgr2=0x556357cc9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:19.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.022+0000 7f9e2de00640 1 --2- 172.21.15.2:0/1606484501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cc9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.022+0000 7f9e2de00640 1 -- 172.21.15.2:0/1606484501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563579a06a0 con 0x556357cc95d0 2024-09-10T08:09:19.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.022+0000 7f9e2de00640 1 --2- 172.21.15.2:0/1606484501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc95d0 0x556357cd3950 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9e200d71b0 tx=0x7f9e200d75b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4cf882296aa5bd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:19.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.022+0000 7f9e26a00640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9e140a2c90 0x7f9e140a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.022+0000 7f9e1b400640 1 -- 172.21.15.2:0/1606484501 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e200e2070 con 0x556357cc95d0 2024-09-10T08:09:19.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.023+0000 7f9e1b400640 1 -- 172.21.15.2:0/1606484501 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e2009b990 con 0x556357cc95d0 2024-09-10T08:09:19.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.023+0000 7f9e1b400640 1 -- 172.21.15.2:0/1606484501 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e200b1680 con 0x556357cc95d0 2024-09-10T08:09:19.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.023+0000 7f9e26a00640 1 -- 172.21.15.2:0/1172151441 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9e200cefe0 con 0x7f9e30100980 2024-09-10T08:09:19.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.023+0000 7f9e2e800640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9e140a2c90 0x7f9e140a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.024+0000 7f9e3785c740 1 -- 172.21.15.2:0/1606484501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc95d0 msgr2=0x556357cd3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:19.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.024+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1606484501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc95d0 0x556357cd3950 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9e200d71b0 tx=0x7f9e200d75b0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.024+0000 7f9e3785c740 1 -- 172.21.15.2:0/1606484501 shutdown_connections 2024-09-10T08:09:19.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.024+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1606484501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cd3e90 0x556357cd6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.024+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1606484501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc95d0 0x556357cd3950 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.024+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1606484501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cc9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.024+0000 7f9e3785c740 1 -- 172.21.15.2:0/1606484501 >> 172.21.15.2:0/1606484501 conn(0x556357cc5240 msgr2=0x556357cc5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:19.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.024+0000 7f9e3785c740 1 -- 172.21.15.2:0/1606484501 shutdown_connections 2024-09-10T08:09:19.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.024+0000 7f9e3785c740 1 -- 172.21.15.2:0/1606484501 wait complete. 2024-09-10T08:09:19.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.024+0000 7f9e2e800640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9e140a2c90 0x7f9e140a5150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e28006010 tx=0x7f9e28009840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:19.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.025+0000 7f9e3785c740 1 -- start start 2024-09-10T08:09:19.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.025+0000 7f9e3785c740 1 Processor -- start 2024-09-10T08:09:19.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.026+0000 7f9e3785c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cd3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.026+0000 7f9e3785c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc95d0 0x556357cd0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.026+0000 7f9e3785c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cd3e90 0x556357cd0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.026+0000 7f9e3785c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556357cd6b00 con 0x556357cc8cb0 2024-09-10T08:09:19.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.026+0000 7f9e3785c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556357be9fa0 con 0x556357cc95d0 2024-09-10T08:09:19.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.026+0000 7f9e3785c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556357cd6c80 con 0x556357cd3e90 2024-09-10T08:09:19.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.026+0000 7f9e3785c740 10 client.0 _ll_register_callbacks cb 0x55635793dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:19.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.026+0000 7f9e2de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cd3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.027+0000 7f9e2de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cd3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35210/0 (socket says 172.21.15.2:35210) 2024-09-10T08:09:19.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.027+0000 7f9e2de00640 1 -- 172.21.15.2:0/3629665968 learned_addr learned my addr 172.21.15.2:0/3629665968 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:19.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.027+0000 7f9e2e800640 1 --2- 172.21.15.2:0/3629665968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cd3e90 0x556357cd0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.027+0000 7f9e2d400640 1 --2- 172.21.15.2:0/3629665968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc95d0 0x556357cd0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.027+0000 7f9e2de00640 1 -- 172.21.15.2:0/3629665968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cd3e90 msgr2=0x556357cd0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:19.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.027+0000 7f9e2de00640 1 --2- 172.21.15.2:0/3629665968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cd3e90 0x556357cd0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.027+0000 7f9e2de00640 1 -- 172.21.15.2:0/3629665968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc95d0 msgr2=0x556357cd0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:19.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.027+0000 7f9e2de00640 1 --2- 172.21.15.2:0/3629665968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc95d0 0x556357cd0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.027+0000 7f9e2de00640 1 -- 172.21.15.2:0/3629665968 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556357cd1740 con 0x556357cc8cb0 2024-09-10T08:09:19.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.027+0000 7f9e2de00640 1 --2- 172.21.15.2:0/3629665968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cd3770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000d740 tx=0x7f9e200cf260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a18f1c2eeed59d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:19.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.027+0000 7f9e0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:09:19.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.028+0000 7f9e0fe00640 1 -- 172.21.15.2:0/3629665968 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e20020070 con 0x556357cc8cb0 2024-09-10T08:09:19.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.028+0000 7f9e0fe00640 1 -- 172.21.15.2:0/3629665968 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9e20099590 con 0x556357cc8cb0 2024-09-10T08:09:19.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.028+0000 7f9e3785c740 1 -- 172.21.15.2:0/3629665968 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e2009b990 con 0x556357cc8cb0 2024-09-10T08:09:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.028+0000 7f9e0fe00640 1 -- 172.21.15.2:0/3629665968 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9e20097530 con 0x556357cc8cb0 2024-09-10T08:09:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.028+0000 7f9e3785c740 10 client.15483 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.028+0000 7f9e3785c740 1 -- 172.21.15.2:0/3629665968 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556357cd6b00 con 0x556357cc8cb0 2024-09-10T08:09:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.029+0000 7f9e0fe00640 1 -- 172.21.15.2:0/3629665968 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e2009b990 con 0x556357cc8cb0 2024-09-10T08:09:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.029+0000 7f9e0fe00640 1 -- 172.21.15.2:0/3629665968 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9e20097900 con 0x556357cc8cb0 2024-09-10T08:09:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.029+0000 7f9e3785c740 10 client.15483 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.029+0000 7f9e3785c740 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.029+0000 7f9e3785c740 20 client.15483 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.029+0000 7f9e3785c740 1 --2- 172.21.15.2:0/3629665968 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556357ce3b70 0x556357ce5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.029+0000 7f9e3785c740 4 client.15483 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.029+0000 7f9e3785c740 1 -- 172.21.15.2:0/3629665968 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556357c3c800 con 0x556357ce3b70 2024-09-10T08:09:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.030+0000 7f9e2d400640 1 --2- 172.21.15.2:0/3629665968 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556357ce3b70 0x556357ce5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.030+0000 7f9e2d400640 1 --2- 172.21.15.2:0/3629665968 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556357ce3b70 0x556357ce5f60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdc58cd7f23114e1 server_cookie=7326b6fe18d65c3 in_seq=0 out_seq=0 2024-09-10T08:09:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.030+0000 7f9e0fe00640 10 client.15483 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:19.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.031+0000 7f9e0fe00640 1 -- 172.21.15.2:0/3629665968 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556357c3c800 con 0x556357ce3b70 2024-09-10T08:09:19.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.031+0000 7f9e0fe00640 10 client.15483 handle_command_reply: tid=0 2024-09-10T08:09:19.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.032+0000 7f9e3785c740 1 client.15483 shutdown 2024-09-10T08:09:19.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.032+0000 7f9e3785c740 1 -- 172.21.15.2:0/3629665968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 msgr2=0x556357cd3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:19.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.032+0000 7f9e3785c740 1 --2- 172.21.15.2:0/3629665968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cd3770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000d740 tx=0x7f9e200cf260 comp rx=0 tx=0).stop 2024-09-10T08:09:19.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.032+0000 7f9e3785c740 1 -- 172.21.15.2:0/3629665968 shutdown_connections 2024-09-10T08:09:19.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.032+0000 7f9e3785c740 1 --2- 172.21.15.2:0/3629665968 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556357ce3b70 0x556357ce5f60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.032+0000 7f9e3785c740 1 --2- 172.21.15.2:0/3629665968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cd3e90 0x556357cd0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.032+0000 7f9e3785c740 1 --2- 172.21.15.2:0/3629665968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc95d0 0x556357cd0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.032+0000 7f9e3785c740 1 --2- 172.21.15.2:0/3629665968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cd3770 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.032+0000 7f9e3785c740 1 -- 172.21.15.2:0/3629665968 >> 172.21.15.2:0/3629665968 conn(0x556357cc5240 msgr2=0x556357ccaf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:19.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.032+0000 7f9e0fe00640 0 client.15483 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:19.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.032+0000 7f9e3785c740 1 -- 172.21.15.2:0/3629665968 shutdown_connections 2024-09-10T08:09:19.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.032+0000 7f9e3785c740 1 -- 172.21.15.2:0/3629665968 wait complete. 2024-09-10T08:09:19.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.033+0000 7f9e3785c740 20 client.15483 trim_cache size 0 max 16384 2024-09-10T08:09:19.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.067+0000 7f9e3785c740 1 Processor -- start 2024-09-10T08:09:19.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.067+0000 7f9e3785c740 1 -- start start 2024-09-10T08:09:19.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.068+0000 7f9e3785c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc8cb0 0x556357ceb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.068+0000 7f9e3785c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cc95d0 0x556357cebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.068+0000 7f9e3785c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cd3e90 0x556357ccdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.068+0000 7f9e3785c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301b7bf0 con 0x556357cd3e90 2024-09-10T08:09:19.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.068+0000 7f9e3785c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301b54c0 con 0x556357cc8cb0 2024-09-10T08:09:19.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.068+0000 7f9e3785c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301b3310 con 0x556357cc95d0 2024-09-10T08:09:19.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.068+0000 7f9e2de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc8cb0 0x556357ceb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.068+0000 7f9e2d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cc95d0 0x556357cebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.068+0000 7f9e2de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc8cb0 0x556357ceb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37602/0 (socket says 172.21.15.2:37602) 2024-09-10T08:09:19.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.068+0000 7f9e2de00640 1 -- 172.21.15.2:0/1406459932 learned_addr learned my addr 172.21.15.2:0/1406459932 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:19.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.068+0000 7f9e2e800640 1 --2- 172.21.15.2:0/1406459932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cd3e90 0x556357ccdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.069+0000 7f9e2d400640 1 -- 172.21.15.2:0/1406459932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc8cb0 msgr2=0x556357ceb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:19.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.069+0000 7f9e2d400640 1 --2- 172.21.15.2:0/1406459932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc8cb0 0x556357ceb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.069+0000 7f9e2d400640 1 -- 172.21.15.2:0/1406459932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cd3e90 msgr2=0x556357ccdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:19.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.069+0000 7f9e2d400640 1 --2- 172.21.15.2:0/1406459932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cd3e90 0x556357ccdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.069+0000 7f9e2d400640 1 -- 172.21.15.2:0/1406459932 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556357cca190 con 0x556357cc95d0 2024-09-10T08:09:19.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.069+0000 7f9e2e800640 1 --2- 172.21.15.2:0/1406459932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cd3e90 0x556357ccdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:19.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.069+0000 7f9e2d400640 1 --2- 172.21.15.2:0/1406459932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cc95d0 0x556357cebbc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x556357cd2060 tx=0x7f9e1c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb0c79d8f3f91760 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:19.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.070+0000 7f9e1aa00640 1 -- 172.21.15.2:0/1406459932 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c003ca0 con 0x556357cc95d0 2024-09-10T08:09:19.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.070+0000 7f9e1aa00640 1 -- 172.21.15.2:0/1406459932 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e1c003e40 con 0x556357cc95d0 2024-09-10T08:09:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.070+0000 7f9e1aa00640 1 -- 172.21.15.2:0/1406459932 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c01b350 con 0x556357cc95d0 2024-09-10T08:09:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.071+0000 7f9e3785c740 1 -- 172.21.15.2:0/1406459932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cc95d0 msgr2=0x556357cebbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.071+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1406459932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cc95d0 0x556357cebbc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x556357cd2060 tx=0x7f9e1c009260 comp rx=0 tx=0).stop 2024-09-10T08:09:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.071+0000 7f9e3785c740 1 -- 172.21.15.2:0/1406459932 shutdown_connections 2024-09-10T08:09:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.071+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1406459932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cd3e90 0x556357ccdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.071+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1406459932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cc95d0 0x556357cebbc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.071+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1406459932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cc8cb0 0x556357ceb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.071+0000 7f9e3785c740 1 -- 172.21.15.2:0/1406459932 >> 172.21.15.2:0/1406459932 conn(0x556357cc5240 msgr2=0x556357ccc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.071+0000 7f9e3785c740 1 -- 172.21.15.2:0/1406459932 shutdown_connections 2024-09-10T08:09:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.071+0000 7f9e3785c740 1 -- 172.21.15.2:0/1406459932 wait complete. 2024-09-10T08:09:19.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.071+0000 7f9e3785c740 1 -- start start 2024-09-10T08:09:19.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.072+0000 7f9e3785c740 1 Processor -- start 2024-09-10T08:09:19.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.072+0000 7f9e3785c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cdc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.072+0000 7f9e3785c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cc95d0 0x556357cd7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.072+0000 7f9e3785c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cd3e90 0x556357cd7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.072+0000 7f9e3785c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301b3980 con 0x556357cc8cb0 2024-09-10T08:09:19.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.072+0000 7f9e3785c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301b4df0 con 0x556357cd3e90 2024-09-10T08:09:19.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.072+0000 7f9e3785c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301b4050 con 0x556357cc95d0 2024-09-10T08:09:19.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.072+0000 7f9e3785c740 10 client.0 _ll_register_callbacks cb 0x7f9e200e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:19.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.073+0000 7f9e2de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cdc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.073+0000 7f9e2d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cc95d0 0x556357cd7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.073+0000 7f9e2de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cdc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35230/0 (socket says 172.21.15.2:35230) 2024-09-10T08:09:19.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.073+0000 7f9e2de00640 1 -- 172.21.15.2:0/1731835098 learned_addr learned my addr 172.21.15.2:0/1731835098 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:19.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.073+0000 7f9e2e800640 1 --2- 172.21.15.2:0/1731835098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cd3e90 0x556357cd7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.073+0000 7f9e2d400640 1 -- 172.21.15.2:0/1731835098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cd3e90 msgr2=0x556357cd7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:19.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.073+0000 7f9e2d400640 1 --2- 172.21.15.2:0/1731835098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cd3e90 0x556357cd7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.073+0000 7f9e2d400640 1 -- 172.21.15.2:0/1731835098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 msgr2=0x556357cdc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:19.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.073+0000 7f9e2d400640 1 --2- 172.21.15.2:0/1731835098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cdc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.073+0000 7f9e2d400640 1 -- 172.21.15.2:0/1731835098 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556357cd8190 con 0x556357cc95d0 2024-09-10T08:09:19.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.073+0000 7f9e2de00640 1 --2- 172.21.15.2:0/1731835098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cdc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:19.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.074+0000 7f9e2d400640 1 --2- 172.21.15.2:0/1731835098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cc95d0 0x556357cd7710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x556357bd6c20 tx=0x7f9e1c0043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9b5db271db9098b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:19.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.074+0000 7f9e0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:09:19.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.074+0000 7f9e0fe00640 1 -- 172.21.15.2:0/1731835098 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c004930 con 0x556357cc95d0 2024-09-10T08:09:19.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.074+0000 7f9e0fe00640 1 -- 172.21.15.2:0/1731835098 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9e1c004c00 con 0x556357cc95d0 2024-09-10T08:09:19.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.074+0000 7f9e3785c740 1 -- 172.21.15.2:0/1731835098 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e1c003e40 con 0x556357cc95d0 2024-09-10T08:09:19.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.074+0000 7f9e0fe00640 1 -- 172.21.15.2:0/1731835098 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9e1c037050 con 0x556357cc95d0 2024-09-10T08:09:19.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.074+0000 7f9e3785c740 10 client.25495 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:19.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.074+0000 7f9e3785c740 1 -- 172.21.15.2:0/1731835098 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556357d4e3e0 con 0x556357cc95d0 2024-09-10T08:09:19.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.075+0000 7f9e0fe00640 1 -- 172.21.15.2:0/1731835098 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e1c003e40 con 0x556357cc95d0 2024-09-10T08:09:19.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.076+0000 7f9e0fe00640 1 -- 172.21.15.2:0/1731835098 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9e1c02c020 con 0x556357cc95d0 2024-09-10T08:09:19.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.076+0000 7f9e3785c740 10 client.25495 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:19.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.076+0000 7f9e3785c740 10 client.25495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:19.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.076+0000 7f9e3785c740 20 client.25495 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:19.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.076+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1731835098 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556357d4ea40 0x556357d50e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:19.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.076+0000 7f9e3785c740 4 client.25495 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:09:19.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.076+0000 7f9e3785c740 1 -- 172.21.15.2:0/1731835098 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556357c36d10 con 0x556357d4ea40 2024-09-10T08:09:19.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.076+0000 7f9e2de00640 1 --2- 172.21.15.2:0/1731835098 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556357d4ea40 0x556357d50e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:19.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.077+0000 7f9e2de00640 1 --2- 172.21.15.2:0/1731835098 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556357d4ea40 0x556357d50e30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a8eff9dba25d5b2 server_cookie=c6c765117e5f4cf in_seq=0 out_seq=0 2024-09-10T08:09:19.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.077+0000 7f9e0fe00640 10 client.25495 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:19.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.078+0000 7f9e0fe00640 1 -- 172.21.15.2:0/1731835098 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556357c36d10 con 0x556357d4ea40 2024-09-10T08:09:19.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.078+0000 7f9e0fe00640 10 client.25495 handle_command_reply: tid=0 2024-09-10T08:09:19.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.078+0000 7f9e3785c740 1 client.25495 shutdown 2024-09-10T08:09:19.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.078+0000 7f9e3785c740 1 -- 172.21.15.2:0/1731835098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cc95d0 msgr2=0x556357cd7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:19.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.078+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1731835098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cc95d0 0x556357cd7710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x556357bd6c20 tx=0x7f9e1c0043f0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.079+0000 7f9e3785c740 1 -- 172.21.15.2:0/1731835098 shutdown_connections 2024-09-10T08:09:19.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.079+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1731835098 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556357d4ea40 0x556357d50e30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.079+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1731835098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556357cd3e90 0x556357cd7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.079+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1731835098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556357cc95d0 0x556357cd7710 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x556357bd6c20 tx=0x7f9e1c0043f0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.079+0000 7f9e3785c740 1 --2- 172.21.15.2:0/1731835098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556357cc8cb0 0x556357cdc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.079+0000 7f9e3785c740 1 -- 172.21.15.2:0/1731835098 >> 172.21.15.2:0/1731835098 conn(0x556357cc5240 msgr2=0x556357cc3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:19.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.079+0000 7f9e0fe00640 0 client.25495 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:19.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.079+0000 7f9e3785c740 1 -- 172.21.15.2:0/1731835098 shutdown_connections 2024-09-10T08:09:19.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.079+0000 7f9e3785c740 1 -- 172.21.15.2:0/1731835098 wait complete. 2024-09-10T08:09:19.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.079+0000 7f9e3785c740 20 client.25495 trim_cache size 0 max 16384 2024-09-10T08:09:19.081 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:09:19.081 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:09:19.081 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "3122b6d1-7437-41e4-acbc-5fd3ff849e18", 2024-09-10T08:09:19.081 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:09:19.081 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:09:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.080+0000 7f9e2f200640 1 -- 172.21.15.2:0/1172151441 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9e140a2c90 msgr2=0x7f9e140a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.080+0000 7f9e2f200640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9e140a2c90 0x7f9e140a5150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e28006010 tx=0x7f9e28009840 comp rx=0 tx=0).stop 2024-09-10T08:09:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.080+0000 7f9e2f200640 1 -- 172.21.15.2:0/1172151441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e30100980 msgr2=0x7f9e301a4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.080+0000 7f9e2f200640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e30100980 0x7f9e301a4590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000d770 tx=0x7f9e2000dc30 comp rx=0 tx=0).stop 2024-09-10T08:09:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.081+0000 7f9e2f200640 1 -- 172.21.15.2:0/1172151441 shutdown_connections 2024-09-10T08:09:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.081+0000 7f9e2f200640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9e140a2c90 0x7f9e140a5150 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.081+0000 7f9e2f200640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e30105d90 0x7f9e3019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.081+0000 7f9e2f200640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e301012a0 0x7f9e301a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.081+0000 7f9e2f200640 1 --2- 172.21.15.2:0/1172151441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e30100980 0x7f9e301a4590 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:19.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.081+0000 7f9e2f200640 1 -- 172.21.15.2:0/1172151441 >> 172.21.15.2:0/1172151441 conn(0x7f9e300fc610 msgr2=0x7f9e300fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:19.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.081+0000 7f9e2f200640 1 -- 172.21.15.2:0/1172151441 shutdown_connections 2024-09-10T08:09:19.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:19.081+0000 7f9e2f200640 1 -- 172.21.15.2:0/1172151441 wait complete. 2024-09-10T08:09:19.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:19 smithi002 ceph-mon[30563]: pgmap v422: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:09:19.889 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:09:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:19 smithi155 ceph-mon[35405]: pgmap v422: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:09:19.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:19 smithi107 ceph-mon[35048]: pgmap v422: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:09:20.302 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:20.741 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:20 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:09:20.741 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:20 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:09:20.741 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:20 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:09:20.741 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:20 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:09:20.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.867+0000 7fac25400640 1 -- 172.21.15.2:0/289527136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac20074510 msgr2=0x7fac200748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:20.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.867+0000 7fac25400640 1 --2- 172.21.15.2:0/289527136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac20074510 0x7fac200748f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fac10009900 tx=0x7fac1002a480 comp rx=0 tx=0).stop 2024-09-10T08:09:20.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.867+0000 7fac25400640 1 -- 172.21.15.2:0/289527136 shutdown_connections 2024-09-10T08:09:20.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.867+0000 7fac25400640 1 --2- 172.21.15.2:0/289527136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac2010e280 0x7fac20110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.867+0000 7fac25400640 1 --2- 172.21.15.2:0/289527136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac20074e30 0x7fac2010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.867+0000 7fac25400640 1 --2- 172.21.15.2:0/289527136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac20074510 0x7fac200748f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.867+0000 7fac25400640 1 -- 172.21.15.2:0/289527136 >> 172.21.15.2:0/289527136 conn(0x7fac200fd740 msgr2=0x7fac200ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:20.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.867+0000 7fac25400640 1 -- 172.21.15.2:0/289527136 shutdown_connections 2024-09-10T08:09:20.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.867+0000 7fac25400640 1 -- 172.21.15.2:0/289527136 wait complete. 2024-09-10T08:09:20.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.868+0000 7fac25400640 1 Processor -- start 2024-09-10T08:09:20.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.869+0000 7fac25400640 1 -- start start 2024-09-10T08:09:20.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.869+0000 7fac25400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac20074510 0x7fac2019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.869+0000 7fac25400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac20074e30 0x7fac2019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.869+0000 7fac25400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac2010e280 0x7fac2019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.869+0000 7fac25400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac20114000 con 0x7fac20074510 2024-09-10T08:09:20.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.869+0000 7fac25400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac20113e80 con 0x7fac20074e30 2024-09-10T08:09:20.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.869+0000 7fac25400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac20114180 con 0x7fac2010e280 2024-09-10T08:09:20.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.870+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac20074510 0x7fac2019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.870+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac20074510 0x7fac2019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35246/0 (socket says 172.21.15.2:35246) 2024-09-10T08:09:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.870+0000 7fac1fe00640 1 -- 172.21.15.2:0/1867903124 learned_addr learned my addr 172.21.15.2:0/1867903124 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.870+0000 7fac1f400640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac20074e30 0x7fac2019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.870+0000 7fac24a00640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac2010e280 0x7fac2019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.870+0000 7fac1f400640 1 -- 172.21.15.2:0/1867903124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac2010e280 msgr2=0x7fac2019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.870+0000 7fac1f400640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac2010e280 0x7fac2019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.870+0000 7fac1f400640 1 -- 172.21.15.2:0/1867903124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac20074510 msgr2=0x7fac2019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.870+0000 7fac1f400640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac20074510 0x7fac2019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.870+0000 7fac1f400640 1 -- 172.21.15.2:0/1867903124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2019f530 con 0x7fac20074e30 2024-09-10T08:09:20.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.871+0000 7fac1f400640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac20074e30 0x7fac2019aac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00e9f0 tx=0x7fac0c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:20.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.871+0000 7fac1cc00640 1 -- 172.21.15.2:0/1867903124 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c00ccc0 con 0x7fac20074e30 2024-09-10T08:09:20.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.871+0000 7fac1cc00640 1 -- 172.21.15.2:0/1867903124 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac0c005020 con 0x7fac20074e30 2024-09-10T08:09:20.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.871+0000 7fac1cc00640 1 -- 172.21.15.2:0/1867903124 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0049e0 con 0x7fac20074e30 2024-09-10T08:09:20.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.871+0000 7fac25400640 1 -- 172.21.15.2:0/1867903124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2019f820 con 0x7fac20074e30 2024-09-10T08:09:20.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.871+0000 7fac25400640 1 -- 172.21.15.2:0/1867903124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac201a7100 con 0x7fac20074e30 2024-09-10T08:09:20.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.875+0000 7fac293ef740 1 Processor -- start 2024-09-10T08:09:20.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.875+0000 7fac293ef740 1 -- start start 2024-09-10T08:09:20.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.875+0000 7fac293ef740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 0x55c571ce9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.875+0000 7fac293ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571ce9670 0x55c571cf39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.875+0000 7fac293ef740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571cf3f30 0x55c571cf6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.875+0000 7fac293ef740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c571c4f3e0 con 0x55c571cf3f30 2024-09-10T08:09:20.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.875+0000 7fac293ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c571bf6d80 con 0x55c571ce9670 2024-09-10T08:09:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.875+0000 7fac293ef740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c571c4b3a0 con 0x55c571ce8d50 2024-09-10T08:09:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.876+0000 7fac24a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 0x55c571ce9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.876+0000 7fac24a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 0x55c571ce9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:48704/0 (socket says 172.21.15.2:48704) 2024-09-10T08:09:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.876+0000 7fac24a00640 1 -- 172.21.15.2:0/168452699 learned_addr learned my addr 172.21.15.2:0/168452699 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.876+0000 7fac1fe00640 1 --2- 172.21.15.2:0/168452699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571ce9670 0x55c571cf39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.876+0000 7fac24a00640 1 -- 172.21.15.2:0/168452699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571ce9670 msgr2=0x55c571cf39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.876+0000 7fac24a00640 1 --2- 172.21.15.2:0/168452699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571ce9670 0x55c571cf39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.876+0000 7fac24a00640 1 -- 172.21.15.2:0/168452699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571cf3f30 msgr2=0x55c571cf6320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:09:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.876+0000 7fac1cc00640 1 -- 172.21.15.2:0/1867903124 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fac0c005560 con 0x7fac20074e30 2024-09-10T08:09:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.876+0000 7fac24a00640 1 --2- 172.21.15.2:0/168452699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571cf3f30 0x55c571cf6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.877+0000 7fac24a00640 1 -- 172.21.15.2:0/168452699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c571c15ae0 con 0x55c571ce8d50 2024-09-10T08:09:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.877+0000 7fac24a00640 1 --2- 172.21.15.2:0/168452699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 0x55c571ce9130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fac18009f30 tx=0x7fac180118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c3cc466821300cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.877+0000 7fac15200640 1 -- 172.21.15.2:0/168452699 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18004d20 con 0x55c571ce8d50 2024-09-10T08:09:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.877+0000 7fac15200640 1 -- 172.21.15.2:0/168452699 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac18004050 con 0x55c571ce8d50 2024-09-10T08:09:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.877+0000 7fac15200640 1 -- 172.21.15.2:0/168452699 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18005280 con 0x55c571ce8d50 2024-09-10T08:09:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.878+0000 7fac293ef740 1 -- 172.21.15.2:0/168452699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 msgr2=0x55c571ce9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.878+0000 7fac293ef740 1 --2- 172.21.15.2:0/168452699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 0x55c571ce9130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fac18009f30 tx=0x7fac180118d0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.879+0000 7fac293ef740 1 -- 172.21.15.2:0/168452699 shutdown_connections 2024-09-10T08:09:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.879+0000 7fac293ef740 1 --2- 172.21.15.2:0/168452699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571cf3f30 0x55c571cf6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.879+0000 7fac293ef740 1 --2- 172.21.15.2:0/168452699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571ce9670 0x55c571cf39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.879+0000 7fac293ef740 1 --2- 172.21.15.2:0/168452699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 0x55c571ce9130 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.879+0000 7fac293ef740 1 -- 172.21.15.2:0/168452699 >> 172.21.15.2:0/168452699 conn(0x55c571ce52e0 msgr2=0x55c571ce56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.879+0000 7fac293ef740 1 -- 172.21.15.2:0/168452699 shutdown_connections 2024-09-10T08:09:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.879+0000 7fac293ef740 1 -- 172.21.15.2:0/168452699 wait complete. 2024-09-10T08:09:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.879+0000 7fac1cc00640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fac040a2ba0 0x7fac040a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.879+0000 7fac1cc00640 1 -- 172.21.15.2:0/1867903124 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fac0c0cf5a0 con 0x7fac20074e30 2024-09-10T08:09:20.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.879+0000 7fac293ef740 1 -- start start 2024-09-10T08:09:20.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.880+0000 7fac293ef740 1 Processor -- start 2024-09-10T08:09:20.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.880+0000 7fac1fe00640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fac040a2ba0 0x7fac040a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.880+0000 7fac293ef740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 0x55c571cf0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.880+0000 7fac293ef740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571ce9670 0x55c571cf1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.880+0000 7fac293ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571cf3f30 0x55c571cf1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.880+0000 7fac293ef740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c571cf6ba0 con 0x55c571ce9670 2024-09-10T08:09:20.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.880+0000 7fac293ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c571c0a0f0 con 0x55c571cf3f30 2024-09-10T08:09:20.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.880+0000 7fac293ef740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c571cf6d20 con 0x55c571ce8d50 2024-09-10T08:09:20.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.881+0000 7fac293ef740 10 client.0 _ll_register_callbacks cb 0x55c571bfb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:20.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.881+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 0x55c571cf0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.881+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 0x55c571cf0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:48708/0 (socket says 172.21.15.2:48708) 2024-09-10T08:09:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.881+0000 7fac1fe00640 1 -- 172.21.15.2:0/1177006676 learned_addr learned my addr 172.21.15.2:0/1177006676 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.881+0000 7fac1fe00640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fac040a2ba0 0x7fac040a5060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fac100096c0 tx=0x7fac100046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.881+0000 7fac1f400640 1 --2- 172.21.15.2:0/1177006676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571ce9670 0x55c571cf1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.881+0000 7fac24a00640 1 --2- 172.21.15.2:0/1177006676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571cf3f30 0x55c571cf1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.882+0000 7fac1fe00640 1 -- 172.21.15.2:0/1177006676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571cf3f30 msgr2=0x55c571cf1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:20.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.882+0000 7fac1fe00640 1 --2- 172.21.15.2:0/1177006676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571cf3f30 0x55c571cf1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.882+0000 7fac1fe00640 1 -- 172.21.15.2:0/1177006676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571ce9670 msgr2=0x55c571cf1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:20.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.882+0000 7fac1fe00640 1 --2- 172.21.15.2:0/1177006676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571ce9670 0x55c571cf1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.882+0000 7fac1fe00640 1 -- 172.21.15.2:0/1177006676 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5719c03c0 con 0x55c571ce8d50 2024-09-10T08:09:20.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.882+0000 7fac1fe00640 1 --2- 172.21.15.2:0/1177006676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 0x55c571cf0ef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fac10006010 tx=0x7fac10009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2f67217f7a29b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.882+0000 7fac09600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:09:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.882+0000 7fac09600640 1 -- 172.21.15.2:0/1177006676 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10035040 con 0x55c571ce8d50 2024-09-10T08:09:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.882+0000 7fac09600640 1 -- 172.21.15.2:0/1177006676 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fac100317b0 con 0x55c571ce8d50 2024-09-10T08:09:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.882+0000 7fac293ef740 1 -- 172.21.15.2:0/1177006676 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac18004050 con 0x55c571ce8d50 2024-09-10T08:09:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.883+0000 7fac09600640 1 -- 172.21.15.2:0/1177006676 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fac1005a050 con 0x55c571ce8d50 2024-09-10T08:09:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.883+0000 7fac293ef740 10 client.25507 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.883+0000 7fac293ef740 1 -- 172.21.15.2:0/1177006676 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c571d0b130 con 0x55c571ce8d50 2024-09-10T08:09:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.883+0000 7fac09600640 1 -- 172.21.15.2:0/1177006676 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac18004050 con 0x55c571ce8d50 2024-09-10T08:09:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.884+0000 7fac09600640 1 -- 172.21.15.2:0/1177006676 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fac1003daf0 con 0x55c571ce8d50 2024-09-10T08:09:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.884+0000 7fac293ef740 10 client.25507 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.884+0000 7fac293ef740 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.884+0000 7fac293ef740 20 client.25507 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:20.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.884+0000 7fac293ef740 1 --2- 172.21.15.2:0/1177006676 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c571d0c5f0 0x55c571d0e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.884+0000 7fac293ef740 4 client.25507 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:20.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.884+0000 7fac293ef740 1 -- 172.21.15.2:0/1177006676 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c571c5c950 con 0x55c571d0c5f0 2024-09-10T08:09:20.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.884+0000 7fac1f400640 1 --2- 172.21.15.2:0/1177006676 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c571d0c5f0 0x55c571d0e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.885+0000 7fac1f400640 1 --2- 172.21.15.2:0/1177006676 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c571d0c5f0 0x55c571d0e9e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e660a5310c7b919c server_cookie=1514b9a9b300006b in_seq=0 out_seq=0 2024-09-10T08:09:20.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.885+0000 7fac09600640 10 client.25507 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.886+0000 7fac09600640 1 -- 172.21.15.2:0/1177006676 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c571c5c950 con 0x55c571d0c5f0 2024-09-10T08:09:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.886+0000 7fac09600640 10 client.25507 handle_command_reply: tid=0 2024-09-10T08:09:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.886+0000 7fac293ef740 1 client.25507 shutdown 2024-09-10T08:09:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.886+0000 7fac293ef740 1 -- 172.21.15.2:0/1177006676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 msgr2=0x55c571cf0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.886+0000 7fac293ef740 1 --2- 172.21.15.2:0/1177006676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 0x55c571cf0ef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fac10006010 tx=0x7fac10009260 comp rx=0 tx=0).stop 2024-09-10T08:09:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.886+0000 7fac293ef740 1 -- 172.21.15.2:0/1177006676 shutdown_connections 2024-09-10T08:09:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.886+0000 7fac293ef740 1 --2- 172.21.15.2:0/1177006676 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c571d0c5f0 0x55c571d0e9e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.886+0000 7fac09600640 0 client.25507 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.886+0000 7fac293ef740 1 --2- 172.21.15.2:0/1177006676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571cf3f30 0x55c571cf1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.886+0000 7fac293ef740 1 --2- 172.21.15.2:0/1177006676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571ce9670 0x55c571cf1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.886+0000 7fac293ef740 1 --2- 172.21.15.2:0/1177006676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce8d50 0x55c571cf0ef0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.886+0000 7fac293ef740 1 -- 172.21.15.2:0/1177006676 >> 172.21.15.2:0/1177006676 conn(0x55c571ce52e0 msgr2=0x55c571ceafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.887+0000 7fac293ef740 1 -- 172.21.15.2:0/1177006676 shutdown_connections 2024-09-10T08:09:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.887+0000 7fac293ef740 1 -- 172.21.15.2:0/1177006676 wait complete. 2024-09-10T08:09:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.887+0000 7fac293ef740 20 client.25507 trim_cache size 0 max 16384 2024-09-10T08:09:20.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:20 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:09:20.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:20 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:09:20.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:20 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:09:20.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:20 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:09:20.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.915+0000 7fac293ef740 1 Processor -- start 2024-09-10T08:09:20.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.915+0000 7fac293ef740 1 -- start start 2024-09-10T08:09:20.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.916+0000 7fac293ef740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571ce8d50 0x55c571cf2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.916+0000 7fac293ef740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce9670 0x55c571d14290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.916+0000 7fac293ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571cf3f30 0x55c571d147d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.916+0000 7fac293ef740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac201af370 con 0x55c571ce8d50 2024-09-10T08:09:20.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.916+0000 7fac293ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac201acc40 con 0x55c571cf3f30 2024-09-10T08:09:20.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.916+0000 7fac293ef740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac201aaa90 con 0x55c571ce9670 2024-09-10T08:09:20.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:20 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:09:20.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:20 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:09:20.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:20 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:09:20.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:20 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:09:20.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.916+0000 7fac1f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce9670 0x55c571d14290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.916+0000 7fac1f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce9670 0x55c571d14290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:48722/0 (socket says 172.21.15.2:48722) 2024-09-10T08:09:20.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.916+0000 7fac1f400640 1 -- 172.21.15.2:0/2434117835 learned_addr learned my addr 172.21.15.2:0/2434117835 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:20.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.916+0000 7fac24a00640 1 --2- 172.21.15.2:0/2434117835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571cf3f30 0x55c571d147d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.917+0000 7fac1fe00640 1 --2- 172.21.15.2:0/2434117835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571ce8d50 0x55c571cf2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.917+0000 7fac1f400640 1 -- 172.21.15.2:0/2434117835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571cf3f30 msgr2=0x55c571d147d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:20.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.917+0000 7fac1f400640 1 --2- 172.21.15.2:0/2434117835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571cf3f30 0x55c571d147d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.917+0000 7fac1f400640 1 -- 172.21.15.2:0/2434117835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571ce8d50 msgr2=0x55c571cf2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.917+0000 7fac1f400640 1 --2- 172.21.15.2:0/2434117835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571ce8d50 0x55c571cf2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.917+0000 7fac1f400640 1 -- 172.21.15.2:0/2434117835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c571ce6390 con 0x55c571ce9670 2024-09-10T08:09:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.917+0000 7fac24a00640 1 --2- 172.21.15.2:0/2434117835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571cf3f30 0x55c571d147d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.917+0000 7fac1f400640 1 --2- 172.21.15.2:0/2434117835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce9670 0x55c571d14290 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fac0c098220 tx=0x7fac0c0d1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfe9f5fdd2857b2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.918+0000 7fac0be00640 1 -- 172.21.15.2:0/2434117835 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c01c040 con 0x55c571ce9670 2024-09-10T08:09:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.918+0000 7fac0be00640 1 -- 172.21.15.2:0/2434117835 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac0c0cf040 con 0x55c571ce9670 2024-09-10T08:09:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.918+0000 7fac0be00640 1 -- 172.21.15.2:0/2434117835 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c098050 con 0x55c571ce9670 2024-09-10T08:09:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.919+0000 7fac293ef740 1 -- 172.21.15.2:0/2434117835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce9670 msgr2=0x55c571d14290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.919+0000 7fac293ef740 1 --2- 172.21.15.2:0/2434117835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce9670 0x55c571d14290 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fac0c098220 tx=0x7fac0c0d1040 comp rx=0 tx=0).stop 2024-09-10T08:09:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.919+0000 7fac293ef740 1 -- 172.21.15.2:0/2434117835 shutdown_connections 2024-09-10T08:09:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.919+0000 7fac293ef740 1 --2- 172.21.15.2:0/2434117835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571cf3f30 0x55c571d147d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.919+0000 7fac293ef740 1 --2- 172.21.15.2:0/2434117835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce9670 0x55c571d14290 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.919+0000 7fac293ef740 1 --2- 172.21.15.2:0/2434117835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571ce8d50 0x55c571cf2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.919+0000 7fac293ef740 1 -- 172.21.15.2:0/2434117835 >> 172.21.15.2:0/2434117835 conn(0x55c571ce52e0 msgr2=0x55c571cf2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.919+0000 7fac293ef740 1 -- 172.21.15.2:0/2434117835 shutdown_connections 2024-09-10T08:09:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.920+0000 7fac293ef740 1 -- 172.21.15.2:0/2434117835 wait complete. 2024-09-10T08:09:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.920+0000 7fac293ef740 1 -- start start 2024-09-10T08:09:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.920+0000 7fac293ef740 1 Processor -- start 2024-09-10T08:09:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.921+0000 7fac293ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571ce8d50 0x55c571cedec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.921+0000 7fac293ef740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce9670 0x55c571cee400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.921+0000 7fac293ef740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571cf3f30 0x55c571cee940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.921+0000 7fac293ef740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac201ab100 con 0x55c571cf3f30 2024-09-10T08:09:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.921+0000 7fac293ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac201ac570 con 0x55c571ce8d50 2024-09-10T08:09:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.921+0000 7fac293ef740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac201ab7d0 con 0x55c571ce9670 2024-09-10T08:09:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.921+0000 7fac293ef740 10 client.0 _ll_register_callbacks cb 0x7fac10004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.921+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571ce8d50 0x55c571cedec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.921+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571ce8d50 0x55c571cedec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37662/0 (socket says 172.21.15.2:37662) 2024-09-10T08:09:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.922+0000 7fac1fe00640 1 -- 172.21.15.2:0/1378194649 learned_addr learned my addr 172.21.15.2:0/1378194649 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.922+0000 7fac1f400640 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce9670 0x55c571cee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.922+0000 7fac24a00640 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571cf3f30 0x55c571cee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.922+0000 7fac1fe00640 1 -- 172.21.15.2:0/1378194649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce9670 msgr2=0x55c571cee400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.922+0000 7fac1fe00640 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce9670 0x55c571cee400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.922+0000 7fac1fe00640 1 -- 172.21.15.2:0/1378194649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571cf3f30 msgr2=0x55c571cee940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.922+0000 7fac1fe00640 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571cf3f30 0x55c571cee940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.922+0000 7fac1fe00640 1 -- 172.21.15.2:0/1378194649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c571d76260 con 0x55c571ce8d50 2024-09-10T08:09:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.922+0000 7fac24a00640 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571cf3f30 0x55c571cee940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:09:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.922+0000 7fac1fe00640 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571ce8d50 0x55c571cedec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fac1002aa70 tx=0x7fac100498b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b114326305a1fc72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.922+0000 7fac09600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:09:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.923+0000 7fac09600640 1 -- 172.21.15.2:0/1378194649 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10049e60 con 0x55c571ce8d50 2024-09-10T08:09:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.923+0000 7fac09600640 1 -- 172.21.15.2:0/1378194649 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fac1003d860 con 0x55c571ce8d50 2024-09-10T08:09:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.923+0000 7fac293ef740 1 -- 172.21.15.2:0/1378194649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac0c0cf040 con 0x55c571ce8d50 2024-09-10T08:09:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.923+0000 7fac09600640 1 -- 172.21.15.2:0/1378194649 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fac10003a80 con 0x55c571ce8d50 2024-09-10T08:09:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.923+0000 7fac293ef740 10 client.25535 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.923+0000 7fac293ef740 1 -- 172.21.15.2:0/1378194649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c571d774d0 con 0x55c571ce8d50 2024-09-10T08:09:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.923+0000 7fac09600640 1 -- 172.21.15.2:0/1378194649 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac0c0cf040 con 0x55c571ce8d50 2024-09-10T08:09:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.924+0000 7fac09600640 1 -- 172.21.15.2:0/1378194649 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fac1005d070 con 0x55c571ce8d50 2024-09-10T08:09:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.924+0000 7fac293ef740 10 client.25535 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.924+0000 7fac293ef740 10 client.25535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.924+0000 7fac293ef740 20 client.25535 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.924+0000 7fac293ef740 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c571d77a80 0x55c571d79ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.924+0000 7fac293ef740 4 client.25535 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:09:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.924+0000 7fac293ef740 1 -- 172.21.15.2:0/1378194649 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c571c56e60 con 0x55c571d77a80 2024-09-10T08:09:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.924+0000 7fac1f400640 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c571d77a80 0x55c571d79ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.925+0000 7fac1f400640 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c571d77a80 0x55c571d79ea0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7da5e488361b7f2 server_cookie=c41f3ad92c276763 in_seq=0 out_seq=0 2024-09-10T08:09:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.925+0000 7fac09600640 10 client.25535 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.925+0000 7fac09600640 1 -- 172.21.15.2:0/1378194649 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c571c56e60 con 0x55c571d77a80 2024-09-10T08:09:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.925+0000 7fac09600640 10 client.25535 handle_command_reply: tid=0 2024-09-10T08:09:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.925+0000 7fac293ef740 1 client.25535 shutdown 2024-09-10T08:09:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.925+0000 7fac293ef740 1 -- 172.21.15.2:0/1378194649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571ce8d50 msgr2=0x55c571cedec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.925+0000 7fac293ef740 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571ce8d50 0x55c571cedec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fac1002aa70 tx=0x7fac100498b0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.925+0000 7fac293ef740 1 -- 172.21.15.2:0/1378194649 shutdown_connections 2024-09-10T08:09:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.925+0000 7fac293ef740 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c571d77a80 0x55c571d79ea0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.925+0000 7fac293ef740 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c571cf3f30 0x55c571cee940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.925+0000 7fac293ef740 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c571ce9670 0x55c571cee400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.925+0000 7fac293ef740 1 --2- 172.21.15.2:0/1378194649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c571ce8d50 0x55c571cedec0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.925+0000 7fac293ef740 1 -- 172.21.15.2:0/1378194649 >> 172.21.15.2:0/1378194649 conn(0x55c571ce52e0 msgr2=0x55c571cef3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.926+0000 7fac09600640 0 client.25535 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.926+0000 7fac293ef740 1 -- 172.21.15.2:0/1378194649 shutdown_connections 2024-09-10T08:09:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.926+0000 7fac293ef740 1 -- 172.21.15.2:0/1378194649 wait complete. 2024-09-10T08:09:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.926+0000 7fac293ef740 20 client.25535 trim_cache size 0 max 16384 2024-09-10T08:09:20.931 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:09:20.931 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:09:20.931 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:09:20.931 INFO:teuthology.orchestra.run.smithi002.stdout: "3122b6d1-7437-41e4-acbc-5fd3ff849e18": { 2024-09-10T08:09:20.931 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:09:20.931 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "3122b6d1-7437-41e4-acbc-5fd3ff849e18", 2024-09-10T08:09:20.932 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:09:20.932 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:20.932 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:20.932 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:09:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.927+0000 7fac25400640 1 -- 172.21.15.2:0/1867903124 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fac040a2ba0 msgr2=0x7fac040a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.927+0000 7fac25400640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fac040a2ba0 0x7fac040a5060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fac100096c0 tx=0x7fac100046c0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.927+0000 7fac25400640 1 -- 172.21.15.2:0/1867903124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac20074e30 msgr2=0x7fac2019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.927+0000 7fac25400640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac20074e30 0x7fac2019aac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00e9f0 tx=0x7fac0c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.927+0000 7fac25400640 1 -- 172.21.15.2:0/1867903124 shutdown_connections 2024-09-10T08:09:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.927+0000 7fac25400640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fac040a2ba0 0x7fac040a5060 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.927+0000 7fac25400640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac2010e280 0x7fac2019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.927+0000 7fac25400640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac20074e30 0x7fac2019aac0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.927+0000 7fac25400640 1 --2- 172.21.15.2:0/1867903124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac20074510 0x7fac2019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.927+0000 7fac25400640 1 -- 172.21.15.2:0/1867903124 >> 172.21.15.2:0/1867903124 conn(0x7fac200fd740 msgr2=0x7fac200fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.928+0000 7fac25400640 1 -- 172.21.15.2:0/1867903124 shutdown_connections 2024-09-10T08:09:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:20.928+0000 7fac25400640 1 -- 172.21.15.2:0/1867903124 wait complete. 2024-09-10T08:09:21.597 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:21 smithi002 ceph-mon[30563]: pgmap v423: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-10T08:09:21.726 INFO:tasks.cephfs.filesystem:scrub status for tag:3122b6d1-7437-41e4-acbc-5fd3ff849e18 - {'path': '/', 'tag': '3122b6d1-7437-41e4-acbc-5fd3ff849e18', 'options': 'recursive,force'} 2024-09-10T08:09:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:21 smithi155 ceph-mon[35405]: pgmap v423: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-10T08:09:21.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:21 smithi107 ceph-mon[35048]: pgmap v423: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-10T08:09:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.620+0000 7f1a03200640 1 -- 172.21.15.2:0/3624582653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a04102440 msgr2=0x7f1a04104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.620+0000 7f1a03200640 1 --2- 172.21.15.2:0/3624582653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a04102440 0x7f1a04104830 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009900 tx=0x7f19f402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.620+0000 7f1a03200640 1 -- 172.21.15.2:0/3624582653 shutdown_connections 2024-09-10T08:09:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.620+0000 7f1a03200640 1 --2- 172.21.15.2:0/3624582653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a041076a0 0x7f1a04109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.620+0000 7f1a03200640 1 --2- 172.21.15.2:0/3624582653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a04104d70 0x7f1a04107160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.620+0000 7f1a03200640 1 --2- 172.21.15.2:0/3624582653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a04102440 0x7f1a04104830 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.620+0000 7f1a03200640 1 -- 172.21.15.2:0/3624582653 >> 172.21.15.2:0/3624582653 conn(0x7f1a040fbfa0 msgr2=0x7f1a040fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.621+0000 7f1a03200640 1 -- 172.21.15.2:0/3624582653 shutdown_connections 2024-09-10T08:09:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.621+0000 7f1a03200640 1 -- 172.21.15.2:0/3624582653 wait complete. 2024-09-10T08:09:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.622+0000 7f1a03200640 1 Processor -- start 2024-09-10T08:09:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.622+0000 7f1a03200640 1 -- start start 2024-09-10T08:09:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.622+0000 7f1a03200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a04102440 0x7f1a04199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.622+0000 7f1a03200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a04104d70 0x7f1a0419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.622+0000 7f1a03200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a041076a0 0x7f1a041a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.622+0000 7f1a03200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0410d680 con 0x7f1a04104d70 2024-09-10T08:09:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.622+0000 7f1a03200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0410d500 con 0x7f1a04102440 2024-09-10T08:09:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.622+0000 7f1a03200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0410d800 con 0x7f1a041076a0 2024-09-10T08:09:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.623+0000 7f1a01e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a04102440 0x7f1a04199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.623+0000 7f1a01e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a04102440 0x7f1a04199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37674/0 (socket says 172.21.15.2:37674) 2024-09-10T08:09:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.623+0000 7f1a01e00640 1 -- 172.21.15.2:0/1459885624 learned_addr learned my addr 172.21.15.2:0/1459885624 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.623+0000 7f1a01400640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a04104d70 0x7f1a0419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.623+0000 7f1a02800640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a041076a0 0x7f1a041a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.623+0000 7f1a01e00640 1 -- 172.21.15.2:0/1459885624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a041076a0 msgr2=0x7f1a041a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.623+0000 7f1a01e00640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a041076a0 0x7f1a041a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.623+0000 7f1a01e00640 1 -- 172.21.15.2:0/1459885624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a04104d70 msgr2=0x7f1a0419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.623+0000 7f1a01e00640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a04104d70 0x7f1a0419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.623+0000 7f1a01e00640 1 -- 172.21.15.2:0/1459885624 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a041a1b60 con 0x7f1a04102440 2024-09-10T08:09:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.623+0000 7f1a01e00640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a04102440 0x7f1a04199f40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f19f4031440 tx=0x7f19f4032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.624+0000 7f19faa00640 1 -- 172.21.15.2:0/1459885624 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f403f030 con 0x7f1a04102440 2024-09-10T08:09:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.624+0000 7f1a03200640 1 -- 172.21.15.2:0/1459885624 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a041a1df0 con 0x7f1a04102440 2024-09-10T08:09:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.624+0000 7f19faa00640 1 -- 172.21.15.2:0/1459885624 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f4003ce0 con 0x7f1a04102440 2024-09-10T08:09:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.624+0000 7f19faa00640 1 -- 172.21.15.2:0/1459885624 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4035070 con 0x7f1a04102440 2024-09-10T08:09:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.624+0000 7f1a03200640 1 -- 172.21.15.2:0/1459885624 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a041a2250 con 0x7f1a04102440 2024-09-10T08:09:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.627+0000 7f19faa00640 1 -- 172.21.15.2:0/1459885624 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f19f4042030 con 0x7f1a04102440 2024-09-10T08:09:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.628+0000 7f1a0be4a740 1 Processor -- start 2024-09-10T08:09:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.628+0000 7f1a0be4a740 1 -- start start 2024-09-10T08:09:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.629+0000 7f1a0be4a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 0x556570f14660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.629+0000 7f1a0be4a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f14ba0 0x556570f19660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.629+0000 7f1a0be4a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f19c30 0x556570f1c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.629+0000 7f1a0be4a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556570e79100 con 0x556570f14ba0 2024-09-10T08:09:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.629+0000 7f1a0be4a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556570e211b0 con 0x556570f14260 2024-09-10T08:09:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.629+0000 7f1a0be4a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556570e750c0 con 0x556570f19c30 2024-09-10T08:09:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.629+0000 7f1a02800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 0x556570f14660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.629+0000 7f1a02800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 0x556570f14660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37688/0 (socket says 172.21.15.2:37688) 2024-09-10T08:09:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.629+0000 7f1a02800640 1 -- 172.21.15.2:0/3316110021 learned_addr learned my addr 172.21.15.2:0/3316110021 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.629+0000 7f19faa00640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19e80a2c90 0x7f19e80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.629+0000 7f1a01e00640 1 --2- 172.21.15.2:0/3316110021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f14ba0 0x556570f19660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.629+0000 7f1a01400640 1 --2- 172.21.15.2:0/3316110021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f19c30 0x556570f1c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.630+0000 7f19faa00640 1 -- 172.21.15.2:0/1459885624 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f19f40efa10 con 0x7f1a04102440 2024-09-10T08:09:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.630+0000 7f1a02800640 1 -- 172.21.15.2:0/3316110021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f19c30 msgr2=0x556570f1c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.630+0000 7f1a02800640 1 --2- 172.21.15.2:0/3316110021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f19c30 0x556570f1c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.630+0000 7f1a02800640 1 -- 172.21.15.2:0/3316110021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f14ba0 msgr2=0x556570f19660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.630+0000 7f1a02800640 1 --2- 172.21.15.2:0/3316110021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f14ba0 0x556570f19660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.630+0000 7f1a02800640 1 -- 172.21.15.2:0/3316110021 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556570b8c210 con 0x556570f14260 2024-09-10T08:09:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.630+0000 7f1a02800640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19e80a2c90 0x7f19e80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.630+0000 7f1a02800640 1 --2- 172.21.15.2:0/3316110021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 0x556570f14660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f19fc013590 tx=0x7f19fc009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=843d9255bbb06067 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.631+0000 7f19ef400640 1 -- 172.21.15.2:0/3316110021 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc012110 con 0x556570f14260 2024-09-10T08:09:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.631+0000 7f1a02800640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19e80a2c90 0x7f19e80a5150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f19fc023000 tx=0x7f19fc01c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.631+0000 7f19ef400640 1 -- 172.21.15.2:0/3316110021 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19fc01b8c0 con 0x556570f14260 2024-09-10T08:09:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.631+0000 7f19ef400640 1 -- 172.21.15.2:0/3316110021 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc00e610 con 0x556570f14260 2024-09-10T08:09:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.632+0000 7f1a0be4a740 1 -- 172.21.15.2:0/3316110021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 msgr2=0x556570f14660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.632+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/3316110021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 0x556570f14660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f19fc013590 tx=0x7f19fc009f90 comp rx=0 tx=0).stop 2024-09-10T08:09:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.632+0000 7f1a0be4a740 1 -- 172.21.15.2:0/3316110021 shutdown_connections 2024-09-10T08:09:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.632+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/3316110021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f19c30 0x556570f1c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.632+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/3316110021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f14ba0 0x556570f19660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.632+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/3316110021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 0x556570f14660 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.632+0000 7f1a0be4a740 1 -- 172.21.15.2:0/3316110021 >> 172.21.15.2:0/3316110021 conn(0x556570f0f8d0 msgr2=0x556570f0fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.632+0000 7f1a0be4a740 1 -- 172.21.15.2:0/3316110021 shutdown_connections 2024-09-10T08:09:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.632+0000 7f1a0be4a740 1 -- 172.21.15.2:0/3316110021 wait complete. 2024-09-10T08:09:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.633+0000 7f1a0be4a740 1 -- start start 2024-09-10T08:09:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.633+0000 7f1a0be4a740 1 Processor -- start 2024-09-10T08:09:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.633+0000 7f1a0be4a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f14260 0x556570f1e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.633+0000 7f1a0be4a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14ba0 0x556570f1e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.633+0000 7f1a0be4a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f19c30 0x556570f25970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.633+0000 7f1a0be4a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556570f1cc00 con 0x556570f19c30 2024-09-10T08:09:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.633+0000 7f1a0be4a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556570e02090 con 0x556570f14ba0 2024-09-10T08:09:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.633+0000 7f1a0be4a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556570f1cd80 con 0x556570f14260 2024-09-10T08:09:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.633+0000 7f1a0be4a740 10 client.0 _ll_register_callbacks cb 0x556570e56f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.634+0000 7f1a01e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f14260 0x556570f1e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.634+0000 7f1a01e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f14260 0x556570f1e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:48752/0 (socket says 172.21.15.2:48752) 2024-09-10T08:09:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.634+0000 7f1a01e00640 1 -- 172.21.15.2:0/3259458345 learned_addr learned my addr 172.21.15.2:0/3259458345 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.634+0000 7f1a01400640 1 --2- 172.21.15.2:0/3259458345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14ba0 0x556570f1e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.634+0000 7f1a02800640 1 --2- 172.21.15.2:0/3259458345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f19c30 0x556570f25970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.634+0000 7f1a01e00640 1 -- 172.21.15.2:0/3259458345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14ba0 msgr2=0x556570f1e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.634+0000 7f1a01e00640 1 --2- 172.21.15.2:0/3259458345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14ba0 0x556570f1e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.634+0000 7f1a01e00640 1 -- 172.21.15.2:0/3259458345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f19c30 msgr2=0x556570f25970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.634+0000 7f1a01e00640 1 --2- 172.21.15.2:0/3259458345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f19c30 0x556570f25970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.634+0000 7f1a01e00640 1 -- 172.21.15.2:0/3259458345 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556570c58620 con 0x556570f14260 2024-09-10T08:09:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.634+0000 7f1a01e00640 1 --2- 172.21.15.2:0/3259458345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f14260 0x556570f1e3f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006010 tx=0x7f19f40b80d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=408e3e4e7301355c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.634+0000 7f19e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:09:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.635+0000 7f19e3e00640 1 -- 172.21.15.2:0/3259458345 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f40eb070 con 0x556570f14260 2024-09-10T08:09:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.635+0000 7f19e3e00640 1 -- 172.21.15.2:0/3259458345 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f19f4003800 con 0x556570f14260 2024-09-10T08:09:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.635+0000 7f1a0be4a740 1 -- 172.21.15.2:0/3259458345 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19fc01b8c0 con 0x556570f14260 2024-09-10T08:09:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.635+0000 7f19e3e00640 1 -- 172.21.15.2:0/3259458345 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19f4049040 con 0x556570f14260 2024-09-10T08:09:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.635+0000 7f1a0be4a740 10 client.25525 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.635+0000 7f1a0be4a740 1 -- 172.21.15.2:0/3259458345 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556570f1ccd0 con 0x556570f14260 2024-09-10T08:09:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.635+0000 7f19e3e00640 1 -- 172.21.15.2:0/3259458345 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19fc01b8c0 con 0x556570f14260 2024-09-10T08:09:22.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.635+0000 7f19e3e00640 1 -- 172.21.15.2:0/3259458345 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f19f40b83e0 con 0x556570f14260 2024-09-10T08:09:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.636+0000 7f1a0be4a740 10 client.25525 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.636+0000 7f1a0be4a740 10 client.25525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.636+0000 7f1a0be4a740 20 client.25525 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.636+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/3259458345 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556570f29ec0 0x556570f2c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.636+0000 7f1a0be4a740 4 client.25525 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.636+0000 7f1a0be4a740 1 -- 172.21.15.2:0/3259458345 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556570e86830 con 0x556570f29ec0 2024-09-10T08:09:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.636+0000 7f1a01400640 1 --2- 172.21.15.2:0/3259458345 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556570f29ec0 0x556570f2c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.637+0000 7f1a01400640 1 --2- 172.21.15.2:0/3259458345 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556570f29ec0 0x556570f2c310 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32719a16b6b106c server_cookie=a2e8e7cd45f39c8d in_seq=0 out_seq=0 2024-09-10T08:09:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.637+0000 7f19e3e00640 10 client.25525 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.637+0000 7f19e3e00640 1 -- 172.21.15.2:0/3259458345 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556570e86830 con 0x556570f29ec0 2024-09-10T08:09:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.637+0000 7f19e3e00640 10 client.25525 handle_command_reply: tid=0 2024-09-10T08:09:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.638+0000 7f1a0be4a740 1 client.25525 shutdown 2024-09-10T08:09:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.638+0000 7f1a0be4a740 1 -- 172.21.15.2:0/3259458345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f14260 msgr2=0x556570f1e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.638+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/3259458345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f14260 0x556570f1e3f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006010 tx=0x7f19f40b80d0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.638+0000 7f1a0be4a740 1 -- 172.21.15.2:0/3259458345 shutdown_connections 2024-09-10T08:09:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.638+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/3259458345 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556570f29ec0 0x556570f2c310 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.638+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/3259458345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f19c30 0x556570f25970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.638+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/3259458345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14ba0 0x556570f1e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.638+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/3259458345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f14260 0x556570f1e3f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.638+0000 7f19e3e00640 0 client.25525 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.638+0000 7f1a0be4a740 1 -- 172.21.15.2:0/3259458345 >> 172.21.15.2:0/3259458345 conn(0x556570f0f8d0 msgr2=0x556570f110e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.638+0000 7f1a0be4a740 1 -- 172.21.15.2:0/3259458345 shutdown_connections 2024-09-10T08:09:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.638+0000 7f1a0be4a740 1 -- 172.21.15.2:0/3259458345 wait complete. 2024-09-10T08:09:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.638+0000 7f1a0be4a740 20 client.25525 trim_cache size 0 max 16384 2024-09-10T08:09:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.667+0000 7f1a0be4a740 1 Processor -- start 2024-09-10T08:09:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.668+0000 7f1a0be4a740 1 -- start start 2024-09-10T08:09:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.668+0000 7f1a0be4a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f14260 0x556570f1e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.668+0000 7f1a0be4a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14ba0 0x556570f25900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.668+0000 7f1a0be4a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f19c30 0x556570f10780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.668+0000 7f1a0be4a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1a041af1c0 con 0x556570f19c30 2024-09-10T08:09:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.668+0000 7f1a0be4a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a041aca90 con 0x556570f14ba0 2024-09-10T08:09:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.668+0000 7f1a0be4a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a041aa8e0 con 0x556570f14260 2024-09-10T08:09:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.668+0000 7f1a01400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14ba0 0x556570f25900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.668+0000 7f1a01e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f14260 0x556570f1e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.668+0000 7f1a01400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14ba0 0x556570f25900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37704/0 (socket says 172.21.15.2:37704) 2024-09-10T08:09:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.668+0000 7f1a01400640 1 -- 172.21.15.2:0/1477895690 learned_addr learned my addr 172.21.15.2:0/1477895690 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.668+0000 7f1a02800640 1 --2- 172.21.15.2:0/1477895690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f19c30 0x556570f10780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.669+0000 7f1a01e00640 1 -- 172.21.15.2:0/1477895690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14ba0 msgr2=0x556570f25900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.669+0000 7f1a01e00640 1 --2- 172.21.15.2:0/1477895690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14ba0 0x556570f25900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.669+0000 7f1a01e00640 1 -- 172.21.15.2:0/1477895690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f19c30 msgr2=0x556570f10780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.669+0000 7f1a01e00640 1 --2- 172.21.15.2:0/1477895690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f19c30 0x556570f10780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.669+0000 7f1a01e00640 1 -- 172.21.15.2:0/1477895690 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556570f2c990 con 0x556570f14260 2024-09-10T08:09:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.669+0000 7f1a01e00640 1 --2- 172.21.15.2:0/1477895690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f14260 0x556570f1e8b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f19f40b81d0 tx=0x7f19f4005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14336123218f537d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.669+0000 7f19eea00640 1 -- 172.21.15.2:0/1477895690 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f40eb040 con 0x556570f14260 2024-09-10T08:09:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.669+0000 7f19eea00640 1 -- 172.21.15.2:0/1477895690 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f40c0070 con 0x556570f14260 2024-09-10T08:09:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.669+0000 7f19eea00640 1 -- 172.21.15.2:0/1477895690 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f40b83e0 con 0x556570f14260 2024-09-10T08:09:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.670+0000 7f1a0be4a740 1 -- 172.21.15.2:0/1477895690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f14260 msgr2=0x556570f1e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.670+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/1477895690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f14260 0x556570f1e8b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f19f40b81d0 tx=0x7f19f4005c90 comp rx=0 tx=0).stop 2024-09-10T08:09:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.670+0000 7f1a0be4a740 1 -- 172.21.15.2:0/1477895690 shutdown_connections 2024-09-10T08:09:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.670+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/1477895690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f19c30 0x556570f10780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.670+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/1477895690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14ba0 0x556570f25900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.670+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/1477895690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f14260 0x556570f1e8b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.670+0000 7f1a0be4a740 1 -- 172.21.15.2:0/1477895690 >> 172.21.15.2:0/1477895690 conn(0x556570f0f8d0 msgr2=0x556570f1e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.670+0000 7f1a0be4a740 1 -- 172.21.15.2:0/1477895690 shutdown_connections 2024-09-10T08:09:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.671+0000 7f1a0be4a740 1 -- 172.21.15.2:0/1477895690 wait complete. 2024-09-10T08:09:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.671+0000 7f1a0be4a740 1 -- start start 2024-09-10T08:09:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.671+0000 7f1a0be4a740 1 Processor -- start 2024-09-10T08:09:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.672+0000 7f1a0be4a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 0x556570f227b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.672+0000 7f1a0be4a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f14ba0 0x556570f22cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.672+0000 7f1a0be4a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f19c30 0x556570f92be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.672+0000 7f1a0be4a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1a041aaf50 con 0x556570f14ba0 2024-09-10T08:09:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.672+0000 7f1a0be4a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a041ac3c0 con 0x556570f14260 2024-09-10T08:09:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.672+0000 7f1a0be4a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a041ab620 con 0x556570f19c30 2024-09-10T08:09:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.672+0000 7f1a0be4a740 10 client.0 _ll_register_callbacks cb 0x7f19f4032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.672+0000 7f1a01e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 0x556570f227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.672+0000 7f1a01400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f14ba0 0x556570f22cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.672+0000 7f1a01e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 0x556570f227b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37716/0 (socket says 172.21.15.2:37716) 2024-09-10T08:09:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.672+0000 7f1a01e00640 1 -- 172.21.15.2:0/1043484479 learned_addr learned my addr 172.21.15.2:0/1043484479 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:22.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.672+0000 7f1a02800640 1 --2- 172.21.15.2:0/1043484479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f19c30 0x556570f92be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.673+0000 7f1a01e00640 1 -- 172.21.15.2:0/1043484479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f19c30 msgr2=0x556570f92be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:22.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.673+0000 7f1a01e00640 1 --2- 172.21.15.2:0/1043484479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f19c30 0x556570f92be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.673+0000 7f1a01e00640 1 -- 172.21.15.2:0/1043484479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f14ba0 msgr2=0x556570f22cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.673+0000 7f1a01e00640 1 --2- 172.21.15.2:0/1043484479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f14ba0 0x556570f22cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.673+0000 7f1a01e00640 1 -- 172.21.15.2:0/1043484479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556570f93120 con 0x556570f14260 2024-09-10T08:09:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.673+0000 7f1a01e00640 1 --2- 172.21.15.2:0/1043484479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 0x556570f227b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f19f40f1040 tx=0x7f19f40bbc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=761d70e7b54b1c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.673+0000 7f19e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:09:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.673+0000 7f19e3e00640 1 -- 172.21.15.2:0/1043484479 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f409e060 con 0x556570f14260 2024-09-10T08:09:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.673+0000 7f19e3e00640 1 -- 172.21.15.2:0/1043484479 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f19f409abe0 con 0x556570f14260 2024-09-10T08:09:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.673+0000 7f1a0be4a740 1 -- 172.21.15.2:0/1043484479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f40c0070 con 0x556570f14260 2024-09-10T08:09:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.674+0000 7f19e3e00640 1 -- 172.21.15.2:0/1043484479 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19f40eb070 con 0x556570f14260 2024-09-10T08:09:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.674+0000 7f1a0be4a740 10 client.25553 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.674+0000 7f1a0be4a740 1 -- 172.21.15.2:0/1043484479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556570f93120 con 0x556570f14260 2024-09-10T08:09:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.674+0000 7f19e3e00640 1 -- 172.21.15.2:0/1043484479 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f40c0070 con 0x556570f14260 2024-09-10T08:09:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.674+0000 7f19e3e00640 1 -- 172.21.15.2:0/1043484479 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f19f40c44a0 con 0x556570f14260 2024-09-10T08:09:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.674+0000 7f1a0be4a740 10 client.25553 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.674+0000 7f1a0be4a740 10 client.25553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.674+0000 7f1a0be4a740 20 client.25553 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.674+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/1043484479 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556570f94820 0x556570f96c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.674+0000 7f1a0be4a740 4 client.25553 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:09:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.674+0000 7f1a0be4a740 1 -- 172.21.15.2:0/1043484479 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556570e80b80 con 0x556570f94820 2024-09-10T08:09:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.675+0000 7f1a01400640 1 --2- 172.21.15.2:0/1043484479 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556570f94820 0x556570f96c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.675+0000 7f1a01400640 1 --2- 172.21.15.2:0/1043484479 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556570f94820 0x556570f96c40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8611e5abe9ec5bf server_cookie=64ac5efc4e70185a in_seq=0 out_seq=0 2024-09-10T08:09:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.675+0000 7f19e3e00640 10 client.25553 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.683+0000 7f19e3e00640 1 -- 172.21.15.2:0/1043484479 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556570e80b80 con 0x556570f94820 2024-09-10T08:09:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.683+0000 7f19e3e00640 10 client.25553 handle_command_reply: tid=0 2024-09-10T08:09:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.683+0000 7f1a0be4a740 1 client.25553 shutdown 2024-09-10T08:09:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.683+0000 7f1a0be4a740 1 -- 172.21.15.2:0/1043484479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 msgr2=0x556570f227b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.683+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/1043484479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 0x556570f227b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f19f40f1040 tx=0x7f19f40bbc00 comp rx=0 tx=0).stop 2024-09-10T08:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.683+0000 7f1a0be4a740 1 -- 172.21.15.2:0/1043484479 shutdown_connections 2024-09-10T08:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.683+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/1043484479 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556570f94820 0x556570f96c40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.683+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/1043484479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556570f19c30 0x556570f92be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.683+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/1043484479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556570f14ba0 0x556570f22cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.683+0000 7f1a0be4a740 1 --2- 172.21.15.2:0/1043484479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556570f14260 0x556570f227b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.683+0000 7f1a0be4a740 1 -- 172.21.15.2:0/1043484479 >> 172.21.15.2:0/1043484479 conn(0x556570f0f8d0 msgr2=0x556570f0e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.684+0000 7f19e3e00640 0 client.25553 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.684+0000 7f1a0be4a740 1 -- 172.21.15.2:0/1043484479 shutdown_connections 2024-09-10T08:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.684+0000 7f1a0be4a740 1 -- 172.21.15.2:0/1043484479 wait complete. 2024-09-10T08:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.684+0000 7f1a0be4a740 20 client.25553 trim_cache size 0 max 16384 2024-09-10T08:09:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:09:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:09:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:09:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:09:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.684+0000 7f1a03200640 1 -- 172.21.15.2:0/1459885624 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19e80a2c90 msgr2=0x7f19e80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.685+0000 7f1a03200640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19e80a2c90 0x7f19e80a5150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f19fc023000 tx=0x7f19fc01c000 comp rx=0 tx=0).stop 2024-09-10T08:09:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.685+0000 7f1a03200640 1 -- 172.21.15.2:0/1459885624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a04102440 msgr2=0x7f1a04199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.685+0000 7f1a03200640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a04102440 0x7f1a04199f40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f19f4031440 tx=0x7f19f4032ce0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.685+0000 7f1a03200640 1 -- 172.21.15.2:0/1459885624 shutdown_connections 2024-09-10T08:09:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.685+0000 7f1a03200640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f19e80a2c90 0x7f19e80a5150 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.685+0000 7f1a03200640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a041076a0 0x7f1a041a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.685+0000 7f1a03200640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a04104d70 0x7f1a0419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.685+0000 7f1a03200640 1 --2- 172.21.15.2:0/1459885624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a04102440 0x7f1a04199f40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.685+0000 7f1a03200640 1 -- 172.21.15.2:0/1459885624 >> 172.21.15.2:0/1459885624 conn(0x7f1a040fbfa0 msgr2=0x7f1a04101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.685+0000 7f1a03200640 1 -- 172.21.15.2:0/1459885624 shutdown_connections 2024-09-10T08:09:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:22.685+0000 7f1a03200640 1 -- 172.21.15.2:0/1459885624 wait complete. 2024-09-10T08:09:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:23 smithi155 ceph-mon[35405]: pgmap v424: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:09:23.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:23 smithi107 ceph-mon[35048]: pgmap v424: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:09:24.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:23 smithi002 ceph-mon[30563]: pgmap v424: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:09:24.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:24 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:09:24.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:24 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:09:25.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:24 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:09:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:25 smithi155 ceph-mon[35405]: pgmap v425: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T08:09:25.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:25 smithi107 ceph-mon[35048]: pgmap v425: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T08:09:26.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:25 smithi002 ceph-mon[30563]: pgmap v425: 129 pgs: 129 active+clean; 672 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T08:09:27.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:27 smithi155 ceph-mon[35405]: pgmap v426: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-10T08:09:28.185 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:27 smithi002 ceph-mon[30563]: pgmap v426: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-10T08:09:28.186 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:27 smithi107 ceph-mon[35048]: pgmap v426: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-10T08:09:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:29 smithi155 ceph-mon[35405]: pgmap v427: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-10T08:09:29.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:29 smithi107 ceph-mon[35048]: pgmap v427: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-10T08:09:30.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:29 smithi002 ceph-mon[30563]: pgmap v427: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-10T08:09:31.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:31 smithi155 ceph-mon[35405]: pgmap v428: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:09:31.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:31 smithi107 ceph-mon[35048]: pgmap v428: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:09:32.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:31 smithi002 ceph-mon[30563]: pgmap v428: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:09:33.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:33 smithi155 ceph-mon[35405]: pgmap v429: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:09:33.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:33 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:09:33.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:33 smithi107 ceph-mon[35048]: pgmap v429: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:09:33.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:33 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:09:34.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:33 smithi002 ceph-mon[30563]: pgmap v429: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:09:34.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:33 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:09:34.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:09:34.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:09:34.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:09:34.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:09:34.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:09:34.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:09:35.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:09:35.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:09:35.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:09:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.837+0000 7fb837e00640 1 -- 172.21.15.2:0/2684201659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb838101580 msgr2=0x7fb838101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.837+0000 7fb837e00640 1 --2- 172.21.15.2:0/2684201659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb838101580 0x7fb838101a00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb824009900 tx=0x7fb82402a410 comp rx=0 tx=0).stop 2024-09-10T08:09:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.838+0000 7fb837e00640 1 -- 172.21.15.2:0/2684201659 shutdown_connections 2024-09-10T08:09:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.838+0000 7fb837e00640 1 --2- 172.21.15.2:0/2684201659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb838101f40 0x7fb838109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.838+0000 7fb837e00640 1 --2- 172.21.15.2:0/2684201659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb838101580 0x7fb838101a00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.838+0000 7fb837e00640 1 --2- 172.21.15.2:0/2684201659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb838103050 0x7fb838103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.838+0000 7fb837e00640 1 -- 172.21.15.2:0/2684201659 >> 172.21.15.2:0/2684201659 conn(0x7fb8380fd0d0 msgr2=0x7fb8380ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.838+0000 7fb837e00640 1 -- 172.21.15.2:0/2684201659 shutdown_connections 2024-09-10T08:09:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.838+0000 7fb837e00640 1 -- 172.21.15.2:0/2684201659 wait complete. 2024-09-10T08:09:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.839+0000 7fb837e00640 1 Processor -- start 2024-09-10T08:09:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.839+0000 7fb837e00640 1 -- start start 2024-09-10T08:09:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.839+0000 7fb837e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb838101f40 0x7fb838195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.839+0000 7fb837e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb838103050 0x7fb838196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.839+0000 7fb837e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb83819b3a0 0x7fb83819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.839+0000 7fb837e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb83810b570 con 0x7fb838101f40 2024-09-10T08:09:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.839+0000 7fb837e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb83810b3f0 con 0x7fb838103050 2024-09-10T08:09:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.839+0000 7fb837e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb83810b6f0 con 0x7fb83819b3a0 2024-09-10T08:09:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.840+0000 7fb836000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb838103050 0x7fb838196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.840+0000 7fb836000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb838103050 0x7fb838196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39104/0 (socket says 172.21.15.2:39104) 2024-09-10T08:09:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.840+0000 7fb836000640 1 -- 172.21.15.2:0/2257129610 learned_addr learned my addr 172.21.15.2:0/2257129610 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.840+0000 7fb837400640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb83819b3a0 0x7fb83819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.840+0000 7fb836a00640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb838101f40 0x7fb838195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.840+0000 7fb836000640 1 -- 172.21.15.2:0/2257129610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb83819b3a0 msgr2=0x7fb83819d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.840+0000 7fb836000640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb83819b3a0 0x7fb83819d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.840+0000 7fb836000640 1 -- 172.21.15.2:0/2257129610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb838101f40 msgr2=0x7fb838195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.840+0000 7fb836000640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb838101f40 0x7fb838195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.840+0000 7fb836000640 1 -- 172.21.15.2:0/2257129610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83819de00 con 0x7fb838103050 2024-09-10T08:09:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.840+0000 7fb836a00640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb838101f40 0x7fb838195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.840+0000 7fb837400640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb83819b3a0 0x7fb83819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.841+0000 7fb836000640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb838103050 0x7fb838196340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb82402aa00 tx=0x7fb824032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.841+0000 7fb82f400640 1 -- 172.21.15.2:0/2257129610 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82403f030 con 0x7fb838103050 2024-09-10T08:09:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.841+0000 7fb82f400640 1 -- 172.21.15.2:0/2257129610 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb824003ce0 con 0x7fb838103050 2024-09-10T08:09:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.841+0000 7fb82f400640 1 -- 172.21.15.2:0/2257129610 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824035070 con 0x7fb838103050 2024-09-10T08:09:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.842+0000 7fb837e00640 1 -- 172.21.15.2:0/2257129610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83819e090 con 0x7fb838103050 2024-09-10T08:09:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.842+0000 7fb837e00640 1 -- 172.21.15.2:0/2257129610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83819e530 con 0x7fb838103050 2024-09-10T08:09:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.845+0000 7fb84089f740 1 Processor -- start 2024-09-10T08:09:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.845+0000 7fb84089f740 1 -- start start 2024-09-10T08:09:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb84089f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 0x55ede2995660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb84089f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995ba0 0x55ede299a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb84089f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 0x55ede299d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb84089f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ede28fa100 con 0x55ede299ac30 2024-09-10T08:09:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb84089f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ede28a21b0 con 0x55ede2995ba0 2024-09-10T08:09:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb84089f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ede28f60c0 con 0x55ede2995260 2024-09-10T08:09:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb837400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 0x55ede2995660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb837400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 0x55ede2995660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:58162/0 (socket says 172.21.15.2:58162) 2024-09-10T08:09:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb837400640 1 -- 172.21.15.2:0/403801039 learned_addr learned my addr 172.21.15.2:0/403801039 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb836a00640 1 --2- 172.21.15.2:0/403801039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995ba0 0x55ede299a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb837400640 1 -- 172.21.15.2:0/403801039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995ba0 msgr2=0x55ede299a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb837400640 1 --2- 172.21.15.2:0/403801039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995ba0 0x55ede299a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb837400640 1 -- 172.21.15.2:0/403801039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 msgr2=0x55ede299d0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:09:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb837400640 1 --2- 172.21.15.2:0/403801039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 0x55ede299d0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.846+0000 7fb837400640 1 -- 172.21.15.2:0/403801039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ede260d210 con 0x55ede2995260 2024-09-10T08:09:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.847+0000 7fb82f400640 1 -- 172.21.15.2:0/2257129610 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb824042030 con 0x7fb838103050 2024-09-10T08:09:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.847+0000 7fb837400640 1 --2- 172.21.15.2:0/403801039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 0x55ede2995660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb830011810 tx=0x7fb830011cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36aff573d9780aed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.847+0000 7fb823e00640 1 -- 172.21.15.2:0/403801039 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830011310 con 0x55ede2995260 2024-09-10T08:09:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.847+0000 7fb823e00640 1 -- 172.21.15.2:0/403801039 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83000f9a0 con 0x55ede2995260 2024-09-10T08:09:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.847+0000 7fb823e00640 1 -- 172.21.15.2:0/403801039 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8300056c0 con 0x55ede2995260 2024-09-10T08:09:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.848+0000 7fb84089f740 1 -- 172.21.15.2:0/403801039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 msgr2=0x55ede2995660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.848+0000 7fb84089f740 1 --2- 172.21.15.2:0/403801039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 0x55ede2995660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb830011810 tx=0x7fb830011cd0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.848+0000 7fb82f400640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb81c0a2b80 0x7fb81c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.849+0000 7fb82f400640 1 -- 172.21.15.2:0/2257129610 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb8240efc30 con 0x7fb838103050 2024-09-10T08:09:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.849+0000 7fb84089f740 1 -- 172.21.15.2:0/403801039 shutdown_connections 2024-09-10T08:09:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.849+0000 7fb84089f740 1 --2- 172.21.15.2:0/403801039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 0x55ede299d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.849+0000 7fb84089f740 1 --2- 172.21.15.2:0/403801039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995ba0 0x55ede299a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.849+0000 7fb84089f740 1 --2- 172.21.15.2:0/403801039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 0x55ede2995660 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.849+0000 7fb84089f740 1 -- 172.21.15.2:0/403801039 >> 172.21.15.2:0/403801039 conn(0x55ede29908d0 msgr2=0x55ede2990cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.849+0000 7fb84089f740 1 -- 172.21.15.2:0/403801039 shutdown_connections 2024-09-10T08:09:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.849+0000 7fb837400640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb81c0a2b80 0x7fb81c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.849+0000 7fb84089f740 1 -- 172.21.15.2:0/403801039 wait complete. 2024-09-10T08:09:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.850+0000 7fb84089f740 1 -- start start 2024-09-10T08:09:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.850+0000 7fb837400640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb81c0a2b80 0x7fb81c0a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb830005fd0 tx=0x7fb830009b10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.850+0000 7fb84089f740 1 Processor -- start 2024-09-10T08:09:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.851+0000 7fb84089f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995260 0x55ede299f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.851+0000 7fb84089f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede2995ba0 0x55ede299f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.851+0000 7fb84089f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede299ac30 0x55ede29a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.851+0000 7fb84089f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ede299dc00 con 0x55ede2995ba0 2024-09-10T08:09:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.851+0000 7fb84089f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ede2883090 con 0x55ede2995260 2024-09-10T08:09:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.851+0000 7fb84089f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ede299dd80 con 0x55ede299ac30 2024-09-10T08:09:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.851+0000 7fb84089f740 10 client.0 _ll_register_callbacks cb 0x55ede28d7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.851+0000 7fb837400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede299ac30 0x55ede29a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.851+0000 7fb837400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede299ac30 0x55ede29a69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:58168/0 (socket says 172.21.15.2:58168) 2024-09-10T08:09:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.851+0000 7fb837400640 1 -- 172.21.15.2:0/903400977 learned_addr learned my addr 172.21.15.2:0/903400977 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.851+0000 7fb836000640 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede2995ba0 0x55ede299f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.851+0000 7fb836a00640 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995260 0x55ede299f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.852+0000 7fb836a00640 1 -- 172.21.15.2:0/903400977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede299ac30 msgr2=0x55ede29a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.852+0000 7fb836a00640 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede299ac30 0x55ede29a69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.852+0000 7fb836a00640 1 -- 172.21.15.2:0/903400977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede2995ba0 msgr2=0x55ede299f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.852+0000 7fb836a00640 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede2995ba0 0x55ede299f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.852+0000 7fb836a00640 1 -- 172.21.15.2:0/903400977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ede26d9620 con 0x55ede2995260 2024-09-10T08:09:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.852+0000 7fb836000640 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede2995ba0 0x55ede299f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.852+0000 7fb836a00640 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995260 0x55ede299f470 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55ede29965d0 tx=0x7fb82800bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcd976d91eb11a66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.852+0000 7fb820c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:09:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.853+0000 7fb820c00640 1 -- 172.21.15.2:0/903400977 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828004460 con 0x55ede2995260 2024-09-10T08:09:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.853+0000 7fb820c00640 1 -- 172.21.15.2:0/903400977 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb828017860 con 0x55ede2995260 2024-09-10T08:09:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.853+0000 7fb84089f740 1 -- 172.21.15.2:0/903400977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb83000f9a0 con 0x55ede2995260 2024-09-10T08:09:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.853+0000 7fb820c00640 1 -- 172.21.15.2:0/903400977 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb828029050 con 0x55ede2995260 2024-09-10T08:09:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.853+0000 7fb84089f740 10 client.25565 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.853+0000 7fb84089f740 1 -- 172.21.15.2:0/903400977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ede29a98b0 con 0x55ede2995260 2024-09-10T08:09:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.853+0000 7fb820c00640 1 -- 172.21.15.2:0/903400977 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83000f9a0 con 0x55ede2995260 2024-09-10T08:09:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.854+0000 7fb820c00640 1 -- 172.21.15.2:0/903400977 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb82801a430 con 0x55ede2995260 2024-09-10T08:09:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.854+0000 7fb84089f740 10 client.25565 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.854+0000 7fb84089f740 10 client.25565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.854+0000 7fb84089f740 20 client.25565 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.854+0000 7fb84089f740 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ede29aaf80 0x55ede29ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.854+0000 7fb84089f740 4 client.25565 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.854+0000 7fb84089f740 1 -- 172.21.15.2:0/903400977 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ede2907830 con 0x55ede29aaf80 2024-09-10T08:09:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.854+0000 7fb836000640 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ede29aaf80 0x55ede29ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.855+0000 7fb836000640 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ede29aaf80 0x55ede29ad370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fed08acfdf5467a3 server_cookie=926c67514c36f3cc in_seq=0 out_seq=0 2024-09-10T08:09:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.855+0000 7fb820c00640 10 client.25565 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.855+0000 7fb820c00640 1 -- 172.21.15.2:0/903400977 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ede2907830 con 0x55ede29aaf80 2024-09-10T08:09:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.855+0000 7fb820c00640 10 client.25565 handle_command_reply: tid=0 2024-09-10T08:09:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.856+0000 7fb84089f740 1 client.25565 shutdown 2024-09-10T08:09:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.856+0000 7fb84089f740 1 -- 172.21.15.2:0/903400977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995260 msgr2=0x55ede299f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.856+0000 7fb84089f740 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995260 0x55ede299f470 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55ede29965d0 tx=0x7fb82800bc20 comp rx=0 tx=0).stop 2024-09-10T08:09:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.856+0000 7fb84089f740 1 -- 172.21.15.2:0/903400977 shutdown_connections 2024-09-10T08:09:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.856+0000 7fb84089f740 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ede29aaf80 0x55ede29ad370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.856+0000 7fb84089f740 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede299ac30 0x55ede29a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.856+0000 7fb84089f740 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede2995ba0 0x55ede299f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.856+0000 7fb84089f740 1 --2- 172.21.15.2:0/903400977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995260 0x55ede299f470 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.856+0000 7fb84089f740 1 -- 172.21.15.2:0/903400977 >> 172.21.15.2:0/903400977 conn(0x55ede29908d0 msgr2=0x55ede29920e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.856+0000 7fb820c00640 0 client.25565 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.856+0000 7fb84089f740 1 -- 172.21.15.2:0/903400977 shutdown_connections 2024-09-10T08:09:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.856+0000 7fb84089f740 1 -- 172.21.15.2:0/903400977 wait complete. 2024-09-10T08:09:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.856+0000 7fb84089f740 20 client.25565 trim_cache size 0 max 16384 2024-09-10T08:09:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.886+0000 7fb84089f740 1 Processor -- start 2024-09-10T08:09:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.886+0000 7fb84089f740 1 -- start start 2024-09-10T08:09:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.886+0000 7fb84089f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 0x55ede299f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.886+0000 7fb84089f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995ba0 0x55ede29a6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.886+0000 7fb84089f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 0x55ede2990cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.887+0000 7fb84089f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381ab410 con 0x55ede299ac30 2024-09-10T08:09:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.887+0000 7fb84089f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381a8ce0 con 0x55ede2995ba0 2024-09-10T08:09:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.887+0000 7fb84089f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381a6b30 con 0x55ede2995260 2024-09-10T08:09:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.887+0000 7fb836a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 0x55ede299f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.887+0000 7fb836a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 0x55ede299f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:58178/0 (socket says 172.21.15.2:58178) 2024-09-10T08:09:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.887+0000 7fb836a00640 1 -- 172.21.15.2:0/1543865277 learned_addr learned my addr 172.21.15.2:0/1543865277 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.887+0000 7fb836000640 1 --2- 172.21.15.2:0/1543865277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995ba0 0x55ede29a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.887+0000 7fb837400640 1 --2- 172.21.15.2:0/1543865277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 0x55ede2990cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.887+0000 7fb836a00640 1 -- 172.21.15.2:0/1543865277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995ba0 msgr2=0x55ede29a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.887+0000 7fb836a00640 1 --2- 172.21.15.2:0/1543865277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995ba0 0x55ede29a6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.887+0000 7fb836a00640 1 -- 172.21.15.2:0/1543865277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 msgr2=0x55ede2990cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.887+0000 7fb836a00640 1 --2- 172.21.15.2:0/1543865277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 0x55ede2990cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.887+0000 7fb836a00640 1 -- 172.21.15.2:0/1543865277 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ede2996080 con 0x55ede2995260 2024-09-10T08:09:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.888+0000 7fb836a00640 1 --2- 172.21.15.2:0/1543865277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 0x55ede299f930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb828001eb0 tx=0x7fb82800d640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8b0bc7b738d0c8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.888+0000 7fb823400640 1 -- 172.21.15.2:0/1543865277 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82801a430 con 0x55ede2995260 2024-09-10T08:09:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.888+0000 7fb823400640 1 -- 172.21.15.2:0/1543865277 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb82801a5d0 con 0x55ede2995260 2024-09-10T08:09:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.888+0000 7fb823400640 1 -- 172.21.15.2:0/1543865277 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828017bd0 con 0x55ede2995260 2024-09-10T08:09:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.889+0000 7fb84089f740 1 -- 172.21.15.2:0/1543865277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 msgr2=0x55ede299f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.889+0000 7fb84089f740 1 --2- 172.21.15.2:0/1543865277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 0x55ede299f930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb828001eb0 tx=0x7fb82800d640 comp rx=0 tx=0).stop 2024-09-10T08:09:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.889+0000 7fb84089f740 1 -- 172.21.15.2:0/1543865277 shutdown_connections 2024-09-10T08:09:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.889+0000 7fb84089f740 1 --2- 172.21.15.2:0/1543865277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 0x55ede2990cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.889+0000 7fb84089f740 1 --2- 172.21.15.2:0/1543865277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995ba0 0x55ede29a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.889+0000 7fb84089f740 1 --2- 172.21.15.2:0/1543865277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995260 0x55ede299f930 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.889+0000 7fb84089f740 1 -- 172.21.15.2:0/1543865277 >> 172.21.15.2:0/1543865277 conn(0x55ede29908d0 msgr2=0x55ede299f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.890+0000 7fb84089f740 1 -- 172.21.15.2:0/1543865277 shutdown_connections 2024-09-10T08:09:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.890+0000 7fb84089f740 1 -- 172.21.15.2:0/1543865277 wait complete. 2024-09-10T08:09:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.890+0000 7fb84089f740 1 -- start start 2024-09-10T08:09:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.890+0000 7fb84089f740 1 Processor -- start 2024-09-10T08:09:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.891+0000 7fb84089f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995260 0x55ede29a6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.891+0000 7fb84089f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995ba0 0x55ede29a3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.891+0000 7fb84089f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 0x55ede29a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.891+0000 7fb84089f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381a71a0 con 0x55ede299ac30 2024-09-10T08:09:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.891+0000 7fb84089f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381a8610 con 0x55ede2995260 2024-09-10T08:09:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.891+0000 7fb84089f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381a7870 con 0x55ede2995ba0 2024-09-10T08:09:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.891+0000 7fb84089f740 10 client.0 _ll_register_callbacks cb 0x7fb82800d300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.891+0000 7fb836000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995ba0 0x55ede29a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.891+0000 7fb836a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995260 0x55ede29a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.891+0000 7fb836000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995ba0 0x55ede29a3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:58184/0 (socket says 172.21.15.2:58184) 2024-09-10T08:09:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.891+0000 7fb836a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995260 0x55ede29a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39140/0 (socket says 172.21.15.2:39140) 2024-09-10T08:09:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.891+0000 7fb836000640 1 -- 172.21.15.2:0/1688981410 learned_addr learned my addr 172.21.15.2:0/1688981410 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.891+0000 7fb837400640 1 --2- 172.21.15.2:0/1688981410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 0x55ede29a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.892+0000 7fb836000640 1 -- 172.21.15.2:0/1688981410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995260 msgr2=0x55ede29a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.892+0000 7fb836000640 1 --2- 172.21.15.2:0/1688981410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995260 0x55ede29a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.892+0000 7fb836000640 1 -- 172.21.15.2:0/1688981410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 msgr2=0x55ede29a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.892+0000 7fb836000640 1 --2- 172.21.15.2:0/1688981410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 0x55ede29a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.892+0000 7fb836000640 1 -- 172.21.15.2:0/1688981410 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ede2a13cc0 con 0x55ede2995ba0 2024-09-10T08:09:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.892+0000 7fb836a00640 1 --2- 172.21.15.2:0/1688981410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995260 0x55ede29a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.892+0000 7fb836000640 1 --2- 172.21.15.2:0/1688981410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995ba0 0x55ede29a3450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb82402a9d0 tx=0x7fb824031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bf289797075aa69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.892+0000 7fb820c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:09:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.892+0000 7fb820c00640 1 -- 172.21.15.2:0/1688981410 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8240c03f0 con 0x55ede2995ba0 2024-09-10T08:09:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.892+0000 7fb820c00640 1 -- 172.21.15.2:0/1688981410 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb8240ed3f0 con 0x55ede2995ba0 2024-09-10T08:09:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.893+0000 7fb84089f740 1 -- 172.21.15.2:0/1688981410 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb82801a5d0 con 0x55ede2995ba0 2024-09-10T08:09:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.893+0000 7fb820c00640 1 -- 172.21.15.2:0/1688981410 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb8240c4cb0 con 0x55ede2995ba0 2024-09-10T08:09:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.893+0000 7fb84089f740 10 client.25549 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.893+0000 7fb84089f740 1 -- 172.21.15.2:0/1688981410 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ede2a15070 con 0x55ede2995ba0 2024-09-10T08:09:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.893+0000 7fb820c00640 1 -- 172.21.15.2:0/1688981410 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb82801a5d0 con 0x55ede2995ba0 2024-09-10T08:09:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.893+0000 7fb820c00640 1 -- 172.21.15.2:0/1688981410 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb824003800 con 0x55ede2995ba0 2024-09-10T08:09:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.893+0000 7fb84089f740 10 client.25549 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.893+0000 7fb84089f740 10 client.25549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.893+0000 7fb84089f740 20 client.25549 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.893+0000 7fb84089f740 1 --2- 172.21.15.2:0/1688981410 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ede2a15a20 0x55ede2a17e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.893+0000 7fb84089f740 4 client.25549 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:09:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.893+0000 7fb84089f740 1 -- 172.21.15.2:0/1688981410 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ede2901b80 con 0x55ede2a15a20 2024-09-10T08:09:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.894+0000 7fb836a00640 1 --2- 172.21.15.2:0/1688981410 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ede2a15a20 0x55ede2a17e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.894+0000 7fb836a00640 1 --2- 172.21.15.2:0/1688981410 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ede2a15a20 0x55ede2a17e40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af8d19ea730f3e8c server_cookie=dfb71d3c5ee7def4 in_seq=0 out_seq=0 2024-09-10T08:09:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.894+0000 7fb820c00640 10 client.25549 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb820c00640 1 -- 172.21.15.2:0/1688981410 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ede2901b80 con 0x55ede2a15a20 2024-09-10T08:09:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb820c00640 10 client.25549 handle_command_reply: tid=0 2024-09-10T08:09:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb84089f740 1 client.25549 shutdown 2024-09-10T08:09:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb84089f740 1 -- 172.21.15.2:0/1688981410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995ba0 msgr2=0x55ede29a3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb84089f740 1 --2- 172.21.15.2:0/1688981410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995ba0 0x55ede29a3450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb82402a9d0 tx=0x7fb824031400 comp rx=0 tx=0).stop 2024-09-10T08:09:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb84089f740 1 -- 172.21.15.2:0/1688981410 shutdown_connections 2024-09-10T08:09:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb84089f740 1 --2- 172.21.15.2:0/1688981410 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ede2a15a20 0x55ede2a17e40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb84089f740 1 --2- 172.21.15.2:0/1688981410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ede299ac30 0x55ede29a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb84089f740 1 --2- 172.21.15.2:0/1688981410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ede2995ba0 0x55ede29a3450 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb84089f740 1 --2- 172.21.15.2:0/1688981410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ede2995260 0x55ede29a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb820c00640 0 client.25549 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb84089f740 1 -- 172.21.15.2:0/1688981410 >> 172.21.15.2:0/1688981410 conn(0x55ede29908d0 msgr2=0x55ede298f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb84089f740 1 -- 172.21.15.2:0/1688981410 shutdown_connections 2024-09-10T08:09:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb84089f740 1 -- 172.21.15.2:0/1688981410 wait complete. 2024-09-10T08:09:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.901+0000 7fb84089f740 20 client.25549 trim_cache size 0 max 16384 2024-09-10T08:09:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:09:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:09:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:09:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:09:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.902+0000 7fb837e00640 1 -- 172.21.15.2:0/2257129610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb81c0a2b80 msgr2=0x7fb81c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.902+0000 7fb837e00640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb81c0a2b80 0x7fb81c0a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb830005fd0 tx=0x7fb830009b10 comp rx=0 tx=0).stop 2024-09-10T08:09:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.902+0000 7fb837e00640 1 -- 172.21.15.2:0/2257129610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb838103050 msgr2=0x7fb838196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.902+0000 7fb837e00640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb838103050 0x7fb838196340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb82402aa00 tx=0x7fb824032ce0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.902+0000 7fb837e00640 1 -- 172.21.15.2:0/2257129610 shutdown_connections 2024-09-10T08:09:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.902+0000 7fb837e00640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb81c0a2b80 0x7fb81c0a5040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.902+0000 7fb837e00640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb83819b3a0 0x7fb83819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.902+0000 7fb837e00640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb838103050 0x7fb838196340 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.902+0000 7fb837e00640 1 --2- 172.21.15.2:0/2257129610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb838101f40 0x7fb838195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.902+0000 7fb837e00640 1 -- 172.21.15.2:0/2257129610 >> 172.21.15.2:0/2257129610 conn(0x7fb8380fd0d0 msgr2=0x7fb838106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.903+0000 7fb837e00640 1 -- 172.21.15.2:0/2257129610 shutdown_connections 2024-09-10T08:09:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:35.903+0000 7fb837e00640 1 -- 172.21.15.2:0/2257129610 wait complete. 2024-09-10T08:09:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:35 smithi155 ceph-mon[35405]: pgmap v430: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:09:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:35 smithi155 ceph-mon[35405]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-09-10T08:09:35.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:35 smithi107 ceph-mon[35048]: pgmap v430: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:09:35.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:35 smithi107 ceph-mon[35048]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-09-10T08:09:36.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:35 smithi002 ceph-mon[30563]: pgmap v430: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:09:36.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:35 smithi002 ceph-mon[30563]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-09-10T08:09:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:37 smithi155 ceph-mon[35405]: pgmap v431: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 8 op/s 2024-09-10T08:09:37.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:37 smithi107 ceph-mon[35048]: pgmap v431: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 8 op/s 2024-09-10T08:09:38.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:37 smithi002 ceph-mon[30563]: pgmap v431: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 8 op/s 2024-09-10T08:09:38.522 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:09:38.972 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:39.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.652+0000 7f004a600640 1 -- 172.21.15.2:0/2622983584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00441027a0 msgr2=0x7f00441067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:39.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.652+0000 7f004a600640 1 --2- 172.21.15.2:0/2622983584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00441027a0 0x7f00441067f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0030009900 tx=0x7f003002a410 comp rx=0 tx=0).stop 2024-09-10T08:09:39.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.652+0000 7f004a600640 1 -- 172.21.15.2:0/2622983584 shutdown_connections 2024-09-10T08:09:39.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.652+0000 7f004a600640 1 --2- 172.21.15.2:0/2622983584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0044106f20 0x7f004410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:39.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.652+0000 7f004a600640 1 --2- 172.21.15.2:0/2622983584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00441027a0 0x7f00441067f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:39.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.652+0000 7f004a600640 1 --2- 172.21.15.2:0/2622983584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0044101df0 0x7f00441021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:39.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.652+0000 7f004a600640 1 -- 172.21.15.2:0/2622983584 >> 172.21.15.2:0/2622983584 conn(0x7f00440fd8b0 msgr2=0x7f00440ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:39.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.652+0000 7f004a600640 1 -- 172.21.15.2:0/2622983584 shutdown_connections 2024-09-10T08:09:39.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.652+0000 7f004a600640 1 -- 172.21.15.2:0/2622983584 wait complete. 2024-09-10T08:09:39.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.653+0000 7f004a600640 1 Processor -- start 2024-09-10T08:09:39.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.654+0000 7f004a600640 1 -- start start 2024-09-10T08:09:39.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.654+0000 7f004a600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0044101df0 0x7f0044196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:39.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.654+0000 7f004a600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00441027a0 0x7f0044196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:39.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.654+0000 7f004a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0044106f20 0x7f004419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:39.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.654+0000 7f004a600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f004410fde0 con 0x7f0044101df0 2024-09-10T08:09:39.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.654+0000 7f004a600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f004410fc60 con 0x7f0044106f20 2024-09-10T08:09:39.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.654+0000 7f004a600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f004410ff60 con 0x7f00441027a0 2024-09-10T08:09:39.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.655+0000 7f0043400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0044101df0 0x7f0044196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:39.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.655+0000 7f0043400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0044101df0 0x7f0044196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56686/0 (socket says 172.21.15.2:56686) 2024-09-10T08:09:39.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.655+0000 7f0043400640 1 -- 172.21.15.2:0/3793579627 learned_addr learned my addr 172.21.15.2:0/3793579627 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:39.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.655+0000 7f0042a00640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00441027a0 0x7f0044196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:39.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.655+0000 7f0043e00640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0044106f20 0x7f004419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:39.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.655+0000 7f0043400640 1 -- 172.21.15.2:0/3793579627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00441027a0 msgr2=0x7f0044196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:39.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.655+0000 7f0043400640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00441027a0 0x7f0044196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:39.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.655+0000 7f0043400640 1 -- 172.21.15.2:0/3793579627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0044106f20 msgr2=0x7f004419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:39.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.656+0000 7f0043400640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0044106f20 0x7f004419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:39.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.656+0000 7f0043400640 1 -- 172.21.15.2:0/3793579627 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004419b1d0 con 0x7f0044101df0 2024-09-10T08:09:39.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.656+0000 7f0042a00640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00441027a0 0x7f0044196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:39.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.656+0000 7f0043e00640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0044106f20 0x7f004419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:09:39.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.656+0000 7f0043400640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0044101df0 0x7f0044196220 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0034009ca0 tx=0x7f0034007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:39.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.657+0000 7f003be00640 1 -- 172.21.15.2:0/3793579627 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00340195f0 con 0x7f0044101df0 2024-09-10T08:09:39.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.657+0000 7f003be00640 1 -- 172.21.15.2:0/3793579627 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f003401e070 con 0x7f0044101df0 2024-09-10T08:09:39.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.657+0000 7f003be00640 1 -- 172.21.15.2:0/3793579627 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034018920 con 0x7f0044101df0 2024-09-10T08:09:39.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.657+0000 7f004a600640 1 -- 172.21.15.2:0/3793579627 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004419b4c0 con 0x7f0044101df0 2024-09-10T08:09:39.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.657+0000 7f004a600640 1 -- 172.21.15.2:0/3793579627 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00441a2da0 con 0x7f0044101df0 2024-09-10T08:09:39.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.659+0000 7f004a600640 1 -- 172.21.15.2:0/3793579627 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00441032d0 con 0x7f0044101df0 2024-09-10T08:09:39.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.662+0000 7f003be00640 1 -- 172.21.15.2:0/3793579627 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0034018ac0 con 0x7f0044101df0 2024-09-10T08:09:39.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.663+0000 7f003be00640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00280a2c00 0x7f00280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:39.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.664+0000 7f003be00640 1 -- 172.21.15.2:0/3793579627 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f00340d2eb0 con 0x7f0044101df0 2024-09-10T08:09:39.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.664+0000 7f0042a00640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00280a2c00 0x7f00280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:39.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.665+0000 7f0042a00640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00280a2c00 0x7f00280a50c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0030000c00 tx=0x7f0030005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:39.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.672+0000 7f003be00640 1 -- 172.21.15.2:0/3793579627 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0034027020 con 0x7f0044101df0 2024-09-10T08:09:39.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:39 smithi002 ceph-mon[30563]: pgmap v432: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T08:09:39.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.905+0000 7f004a600640 1 -- 172.21.15.2:0/3793579627 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f004410a160 con 0x7f0044101df0 2024-09-10T08:09:39.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.906+0000 7f003be00640 1 -- 172.21.15.2:0/3793579627 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f0034016070 con 0x7f0044101df0 2024-09-10T08:09:39.906 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:09:39.906 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:09:39.906 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:09:39.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:39 smithi155 ceph-mon[35405]: pgmap v432: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T08:09:39.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.908+0000 7f004a600640 1 -- 172.21.15.2:0/3793579627 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00280a2c00 msgr2=0x7f00280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:39.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.908+0000 7f004a600640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00280a2c00 0x7f00280a50c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0030000c00 tx=0x7f0030005a40 comp rx=0 tx=0).stop 2024-09-10T08:09:39.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.908+0000 7f004a600640 1 -- 172.21.15.2:0/3793579627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0044101df0 msgr2=0x7f0044196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:39.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.908+0000 7f004a600640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0044101df0 0x7f0044196220 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0034009ca0 tx=0x7f0034007590 comp rx=0 tx=0).stop 2024-09-10T08:09:39.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.909+0000 7f004a600640 1 -- 172.21.15.2:0/3793579627 shutdown_connections 2024-09-10T08:09:39.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.909+0000 7f004a600640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0044106f20 0x7f004419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:39.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.909+0000 7f004a600640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00280a2c00 0x7f00280a50c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:39.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.909+0000 7f004a600640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00441027a0 0x7f0044196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:39.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.909+0000 7f004a600640 1 --2- 172.21.15.2:0/3793579627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0044101df0 0x7f0044196220 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:39.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.909+0000 7f004a600640 1 -- 172.21.15.2:0/3793579627 >> 172.21.15.2:0/3793579627 conn(0x7f00440fd8b0 msgr2=0x7f00440feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:39.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.909+0000 7f004a600640 1 -- 172.21.15.2:0/3793579627 shutdown_connections 2024-09-10T08:09:39.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:39.910+0000 7f004a600640 1 -- 172.21.15.2:0/3793579627 wait complete. 2024-09-10T08:09:39.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:39 smithi107 ceph-mon[35048]: pgmap v432: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T08:09:40.841 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id db781f90 --release --await' 2024-09-10T08:09:40.842 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id db781f90 --release --await 2024-09-10T08:09:40.892 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:40 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3793579627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:40.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:40 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3793579627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:40.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:40 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3793579627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:41.245 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:41.718 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:41 smithi002 ceph-mon[30563]: pgmap v433: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T08:09:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:41 smithi155 ceph-mon[35405]: pgmap v433: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T08:09:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:41 smithi107 ceph-mon[35048]: pgmap v433: 129 pgs: 129 active+clean; 673 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T08:09:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.927+0000 7fdee9e00640 1 -- 172.21.15.2:0/2408918519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee4102b90 msgr2=0x7fdee4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.927+0000 7fdee9e00640 1 --2- 172.21.15.2:0/2408918519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee4102b90 0x7fdee4102f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fded8009900 tx=0x7fded802a480 comp rx=0 tx=0).stop 2024-09-10T08:09:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.927+0000 7fdee9e00640 1 -- 172.21.15.2:0/2408918519 shutdown_connections 2024-09-10T08:09:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.927+0000 7fdee9e00640 1 --2- 172.21.15.2:0/2408918519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdee410bfc0 0x7fdee410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.927+0000 7fdee9e00640 1 --2- 172.21.15.2:0/2408918519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdee41034b0 0x7fdee410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.927+0000 7fdee9e00640 1 --2- 172.21.15.2:0/2408918519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee4102b90 0x7fdee4102f70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.927+0000 7fdee9e00640 1 -- 172.21.15.2:0/2408918519 >> 172.21.15.2:0/2408918519 conn(0x7fdee40fc610 msgr2=0x7fdee40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:41.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.927+0000 7fdee9e00640 1 -- 172.21.15.2:0/2408918519 shutdown_connections 2024-09-10T08:09:41.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.927+0000 7fdee9e00640 1 -- 172.21.15.2:0/2408918519 wait complete. 2024-09-10T08:09:41.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.929+0000 7fdee9e00640 1 Processor -- start 2024-09-10T08:09:41.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.929+0000 7fdee9e00640 1 -- start start 2024-09-10T08:09:41.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.929+0000 7fdee9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee4102b90 0x7fdee419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.929+0000 7fdee9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdee41034b0 0x7fdee419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.929+0000 7fdee9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdee410bfc0 0x7fdee410ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.929+0000 7fdee9e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdee4111e10 con 0x7fdee410bfc0 2024-09-10T08:09:41.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.929+0000 7fdee9e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdee4111c90 con 0x7fdee4102b90 2024-09-10T08:09:41.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.929+0000 7fdee9e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdee4111f90 con 0x7fdee41034b0 2024-09-10T08:09:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.930+0000 7fdee8a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee4102b90 0x7fdee419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.930+0000 7fdee8a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee4102b90 0x7fdee419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:55592/0 (socket says 172.21.15.2:55592) 2024-09-10T08:09:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.930+0000 7fdee8a00640 1 -- 172.21.15.2:0/3582337700 learned_addr learned my addr 172.21.15.2:0/3582337700 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.930+0000 7fdee9400640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdee410bfc0 0x7fdee410ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.931+0000 7fdee3e00640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdee41034b0 0x7fdee419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.931+0000 7fdee9400640 1 -- 172.21.15.2:0/3582337700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdee41034b0 msgr2=0x7fdee419f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.931+0000 7fdee9400640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdee41034b0 0x7fdee419f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.931+0000 7fdee9400640 1 -- 172.21.15.2:0/3582337700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee4102b90 msgr2=0x7fdee419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.931+0000 7fdee9400640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee4102b90 0x7fdee419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.931+0000 7fdee9400640 1 -- 172.21.15.2:0/3582337700 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee410b520 con 0x7fdee410bfc0 2024-09-10T08:09:41.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.931+0000 7fdee8a00640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee4102b90 0x7fdee419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:41.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.932+0000 7fdee9400640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdee410bfc0 0x7fdee410ac60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00da10 tx=0x7fdedc00ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:41.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.932+0000 7fdee1600640 1 -- 172.21.15.2:0/3582337700 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc00bd70 con 0x7fdee410bfc0 2024-09-10T08:09:41.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.933+0000 7fdee1600640 1 -- 172.21.15.2:0/3582337700 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdedc003e50 con 0x7fdee410bfc0 2024-09-10T08:09:41.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.933+0000 7fdee1600640 1 -- 172.21.15.2:0/3582337700 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc010660 con 0x7fdee410bfc0 2024-09-10T08:09:41.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.933+0000 7fdee9e00640 1 -- 172.21.15.2:0/3582337700 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee410b810 con 0x7fdee410bfc0 2024-09-10T08:09:41.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.933+0000 7fdee9e00640 1 -- 172.21.15.2:0/3582337700 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee419ab80 con 0x7fdee410bfc0 2024-09-10T08:09:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.938+0000 7fdee1600640 1 -- 172.21.15.2:0/3582337700 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fdedc003970 con 0x7fdee410bfc0 2024-09-10T08:09:41.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.938+0000 7fdeeddd3740 1 Processor -- start 2024-09-10T08:09:41.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.939+0000 7fdeeddd3740 1 -- start start 2024-09-10T08:09:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.940+0000 7fdeeddd3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299520 0x55b9f7299900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.940+0000 7fdeeddd3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f7299e40 0x55b9f72a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.940+0000 7fdeeddd3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f72a4700 0x55b9f72a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.940+0000 7fdeeddd3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b9f71ff650 con 0x55b9f7299520 2024-09-10T08:09:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.940+0000 7fdeeddd3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b9f71a6f10 con 0x55b9f72a4700 2024-09-10T08:09:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.940+0000 7fdeeddd3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b9f71fb610 con 0x55b9f7299e40 2024-09-10T08:09:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.941+0000 7fdee1600640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdecc0a2c00 0x7fdecc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.941+0000 7fdee1600640 1 -- 172.21.15.2:0/3582337700 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fdedc0cf1e0 con 0x7fdee410bfc0 2024-09-10T08:09:41.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.942+0000 7fdee9400640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdecc0a2c00 0x7fdecc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:41.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.942+0000 7fdee8a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f7299e40 0x55b9f72a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:41.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.942+0000 7fdee8a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f7299e40 0x55b9f72a41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59310/0 (socket says 172.21.15.2:59310) 2024-09-10T08:09:41.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.942+0000 7fdee8a00640 1 -- 172.21.15.2:0/2021203019 learned_addr learned my addr 172.21.15.2:0/2021203019 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.942+0000 7fdee3e00640 1 --2- 172.21.15.2:0/2021203019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299520 0x55b9f7299900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.943+0000 7fdee9400640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdecc0a2c00 0x7fdecc0a50c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00b630 tx=0x7fdedc018f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.943+0000 7fdee3e00640 1 -- 172.21.15.2:0/2021203019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f7299e40 msgr2=0x55b9f72a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.943+0000 7fdee3e00640 1 --2- 172.21.15.2:0/2021203019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f7299e40 0x55b9f72a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.943+0000 7fdee3e00640 1 -- 172.21.15.2:0/2021203019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f72a4700 msgr2=0x55b9f72a6af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:09:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.943+0000 7fdee3e00640 1 --2- 172.21.15.2:0/2021203019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f72a4700 0x55b9f72a6af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.943+0000 7fdee3e00640 1 -- 172.21.15.2:0/2021203019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9f6f0e580 con 0x55b9f7299520 2024-09-10T08:09:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.944+0000 7fdee3e00640 1 --2- 172.21.15.2:0/2021203019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299520 0x55b9f7299900 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdee41976e0 tx=0x7fded00113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f382ba8b49431abe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:41.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.945+0000 7fded5c00640 1 -- 172.21.15.2:0/2021203019 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0011d80 con 0x55b9f7299520 2024-09-10T08:09:41.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.945+0000 7fded5c00640 1 -- 172.21.15.2:0/2021203019 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded0012930 con 0x55b9f7299520 2024-09-10T08:09:41.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.945+0000 7fded5c00640 1 -- 172.21.15.2:0/2021203019 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded000faa0 con 0x55b9f7299520 2024-09-10T08:09:41.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.946+0000 7fdeeddd3740 1 -- 172.21.15.2:0/2021203019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299520 msgr2=0x55b9f7299900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:41.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.946+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/2021203019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299520 0x55b9f7299900 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdee41976e0 tx=0x7fded00113e0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.946+0000 7fdeeddd3740 1 -- 172.21.15.2:0/2021203019 shutdown_connections 2024-09-10T08:09:41.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.946+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/2021203019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f72a4700 0x55b9f72a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.946+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/2021203019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f7299e40 0x55b9f72a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.946+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/2021203019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299520 0x55b9f7299900 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.946+0000 7fdeeddd3740 1 -- 172.21.15.2:0/2021203019 >> 172.21.15.2:0/2021203019 conn(0x55b9f7295ab0 msgr2=0x55b9f7295e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.946+0000 7fdeeddd3740 1 -- 172.21.15.2:0/2021203019 shutdown_connections 2024-09-10T08:09:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.947+0000 7fdeeddd3740 1 -- 172.21.15.2:0/2021203019 wait complete. 2024-09-10T08:09:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.947+0000 7fdeeddd3740 1 -- start start 2024-09-10T08:09:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.948+0000 7fdeeddd3740 1 Processor -- start 2024-09-10T08:09:41.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.948+0000 7fdeeddd3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72a1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.948+0000 7fdeeddd3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299e40 0x55b9f72a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.948+0000 7fdeeddd3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f72a4700 0x55b9f72a2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.948+0000 7fdeeddd3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b9f72a72d0 con 0x55b9f7299e40 2024-09-10T08:09:41.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.948+0000 7fdeeddd3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b9f71ba350 con 0x55b9f7299520 2024-09-10T08:09:41.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.948+0000 7fdeeddd3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b9f72a7450 con 0x55b9f72a4700 2024-09-10T08:09:41.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.949+0000 7fdeeddd3740 10 client.0 _ll_register_callbacks cb 0x55b9f71dd290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:41.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.949+0000 7fdee3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:41.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.949+0000 7fdee3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:55602/0 (socket says 172.21.15.2:55602) 2024-09-10T08:09:41.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.949+0000 7fdee3e00640 1 -- 172.21.15.2:0/1825295705 learned_addr learned my addr 172.21.15.2:0/1825295705 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:41.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.950+0000 7fdee3e00640 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f72a4700 0x55b9f72a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:41.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.950+0000 7fdee8a00640 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299e40 0x55b9f72a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:41.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.951+0000 7fdee3e00640 1 -- 172.21.15.2:0/1825295705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 msgr2=0x55b9f72a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:41.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.951+0000 7fdee3e00640 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.951+0000 7fdee3e00640 1 -- 172.21.15.2:0/1825295705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299e40 msgr2=0x55b9f72a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:41.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.951+0000 7fdee3e00640 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299e40 0x55b9f72a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.951+0000 7fdee3e00640 1 -- 172.21.15.2:0/1825295705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9f6fd8960 con 0x55b9f72a4700 2024-09-10T08:09:41.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.951+0000 7fdee8a00640 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299e40 0x55b9f72a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:41.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.952+0000 7fdee3e00640 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f72a4700 0x55b9f72a2100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fded0012ca0 tx=0x7fded0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b3de6460cbbc66c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:41.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.952+0000 7fdeca000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:09:41.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.953+0000 7fdeca000640 1 -- 172.21.15.2:0/1825295705 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0012650 con 0x55b9f72a4700 2024-09-10T08:09:41.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.953+0000 7fdeca000640 1 -- 172.21.15.2:0/1825295705 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fded000f820 con 0x55b9f72a4700 2024-09-10T08:09:41.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.953+0000 7fdeeddd3740 1 -- 172.21.15.2:0/1825295705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded0012930 con 0x55b9f72a4700 2024-09-10T08:09:41.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.953+0000 7fdeca000640 1 -- 172.21.15.2:0/1825295705 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fded0037050 con 0x55b9f72a4700 2024-09-10T08:09:41.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.954+0000 7fdeeddd3740 10 client.25567 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:41.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.954+0000 7fdeeddd3740 1 -- 172.21.15.2:0/1825295705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b9f72b3000 con 0x55b9f72a4700 2024-09-10T08:09:41.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.954+0000 7fdeca000640 1 -- 172.21.15.2:0/1825295705 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded0012930 con 0x55b9f72a4700 2024-09-10T08:09:41.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.954+0000 7fdeca000640 1 -- 172.21.15.2:0/1825295705 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fded0026040 con 0x55b9f72a4700 2024-09-10T08:09:41.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.955+0000 7fdeeddd3740 10 client.25567 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:41.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.955+0000 7fdeeddd3740 10 client.25567 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:09:41.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.955+0000 7fdeeddd3740 20 client.25567 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:41.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.955+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9f72b4460 0x55b9f72b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.955+0000 7fdeeddd3740 4 client.25567 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:41.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.955+0000 7fdeeddd3740 1 -- 172.21.15.2:0/1825295705 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9f720cbc0 con 0x55b9f72b4460 2024-09-10T08:09:41.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.956+0000 7fdee8a00640 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9f72b4460 0x55b9f72b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:41.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.956+0000 7fdee8a00640 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9f72b4460 0x55b9f72b6850 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9468e48313437aa4 server_cookie=cbcaa5cb9b6e2f4 in_seq=0 out_seq=0 2024-09-10T08:09:41.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.956+0000 7fdeca000640 10 client.25567 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:41.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.957+0000 7fdeca000640 1 -- 172.21.15.2:0/1825295705 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9f720cbc0 con 0x55b9f72b4460 2024-09-10T08:09:41.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.957+0000 7fdeca000640 10 client.25567 handle_command_reply: tid=0 2024-09-10T08:09:41.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.957+0000 7fdeeddd3740 1 client.25567 shutdown 2024-09-10T08:09:41.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.958+0000 7fdeeddd3740 1 -- 172.21.15.2:0/1825295705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f72a4700 msgr2=0x55b9f72a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:41.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.958+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f72a4700 0x55b9f72a2100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fded0012ca0 tx=0x7fded0000bc0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.958+0000 7fdeeddd3740 1 -- 172.21.15.2:0/1825295705 shutdown_connections 2024-09-10T08:09:41.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.958+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9f72b4460 0x55b9f72b6850 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.958+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f72a4700 0x55b9f72a2100 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.958+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299e40 0x55b9f72a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.958+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/1825295705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.958+0000 7fdeeddd3740 1 -- 172.21.15.2:0/1825295705 >> 172.21.15.2:0/1825295705 conn(0x55b9f7295ab0 msgr2=0x55b9f7294cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:41.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.958+0000 7fdeca000640 0 client.25567 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:41.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.958+0000 7fdeeddd3740 1 -- 172.21.15.2:0/1825295705 shutdown_connections 2024-09-10T08:09:41.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.959+0000 7fdeeddd3740 1 -- 172.21.15.2:0/1825295705 wait complete. 2024-09-10T08:09:41.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.959+0000 7fdeeddd3740 20 client.25567 trim_cache size 0 max 16384 2024-09-10T08:09:41.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.995+0000 7fdeeddd3740 1 Processor -- start 2024-09-10T08:09:41.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.996+0000 7fdeeddd3740 1 -- start start 2024-09-10T08:09:41.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.996+0000 7fdeeddd3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72bbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.996+0000 7fdeeddd3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f7299e40 0x55b9f72bc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.996+0000 7fdeeddd3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f72a4700 0x55b9f729e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:41.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.996+0000 7fdeeddd3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41c04e0 con 0x55b9f72a4700 2024-09-10T08:09:41.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.996+0000 7fdeeddd3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41bddb0 con 0x55b9f7299520 2024-09-10T08:09:41.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.996+0000 7fdeeddd3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41bbc00 con 0x55b9f7299e40 2024-09-10T08:09:41.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.997+0000 7fdee8a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f7299e40 0x55b9f72bc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:41.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.997+0000 7fdee8a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f7299e40 0x55b9f72bc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59330/0 (socket says 172.21.15.2:59330) 2024-09-10T08:09:41.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.997+0000 7fdee8a00640 1 -- 172.21.15.2:0/3793582864 learned_addr learned my addr 172.21.15.2:0/3793582864 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:41.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.997+0000 7fdee3e00640 1 --2- 172.21.15.2:0/3793582864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72bbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:41.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.997+0000 7fdee3e00640 1 --2- 172.21.15.2:0/3793582864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f72a4700 0x55b9f729e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:41.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.999+0000 7fdee3e00640 1 -- 172.21.15.2:0/3793582864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f7299e40 msgr2=0x55b9f72bc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:41.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.999+0000 7fdee3e00640 1 --2- 172.21.15.2:0/3793582864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f7299e40 0x55b9f72bc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.999+0000 7fdee3e00640 1 -- 172.21.15.2:0/3793582864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 msgr2=0x55b9f72bbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:41.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.999+0000 7fdee3e00640 1 --2- 172.21.15.2:0/3793582864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72bbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:41.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:41.999+0000 7fdee3e00640 1 -- 172.21.15.2:0/3793582864 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9f7296b60 con 0x55b9f72a4700 2024-09-10T08:09:41.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.000+0000 7fdee3e00640 1 --2- 172.21.15.2:0/3793582864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f72a4700 0x55b9f729e8e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fded0029b60 tx=0x7fded0011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44049a4384466e61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:42.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.001+0000 7fded5200640 1 -- 172.21.15.2:0/3793582864 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0029680 con 0x55b9f72a4700 2024-09-10T08:09:42.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.001+0000 7fded5200640 1 -- 172.21.15.2:0/3793582864 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded002e070 con 0x55b9f72a4700 2024-09-10T08:09:42.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.001+0000 7fded5200640 1 -- 172.21.15.2:0/3793582864 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0029680 con 0x55b9f72a4700 2024-09-10T08:09:42.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.002+0000 7fdeeddd3740 1 -- 172.21.15.2:0/3793582864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f72a4700 msgr2=0x55b9f729e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:42.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.002+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/3793582864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f72a4700 0x55b9f729e8e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fded0029b60 tx=0x7fded0011fd0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.003+0000 7fdeeddd3740 1 -- 172.21.15.2:0/3793582864 shutdown_connections 2024-09-10T08:09:42.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.003+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/3793582864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f72a4700 0x55b9f729e8e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.003+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/3793582864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f7299e40 0x55b9f72bc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.003+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/3793582864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72bbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.003+0000 7fdeeddd3740 1 -- 172.21.15.2:0/3793582864 >> 172.21.15.2:0/3793582864 conn(0x55b9f7295ab0 msgr2=0x55b9f72a35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:42.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.003+0000 7fdeeddd3740 1 -- 172.21.15.2:0/3793582864 shutdown_connections 2024-09-10T08:09:42.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.004+0000 7fdeeddd3740 1 -- 172.21.15.2:0/3793582864 wait complete. 2024-09-10T08:09:42.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.005+0000 7fdeeddd3740 1 -- start start 2024-09-10T08:09:42.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.005+0000 7fdeeddd3740 1 Processor -- start 2024-09-10T08:09:42.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.006+0000 7fdeeddd3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72acef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:42.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.007+0000 7fdeeddd3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299e40 0x55b9f72aa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:42.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.007+0000 7fdeeddd3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f72a4700 0x55b9f72aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:42.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.007+0000 7fdeeddd3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41bc270 con 0x55b9f7299e40 2024-09-10T08:09:42.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.007+0000 7fdeeddd3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41bd6e0 con 0x55b9f7299520 2024-09-10T08:09:42.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.007+0000 7fdeeddd3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41bc940 con 0x55b9f72a4700 2024-09-10T08:09:42.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.007+0000 7fdeeddd3740 10 client.0 _ll_register_callbacks cb 0x7fded00099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:42.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.007+0000 7fdee3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f72a4700 0x55b9f72aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:42.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.007+0000 7fdee3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:42.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.007+0000 7fdee3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72acef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:55616/0 (socket says 172.21.15.2:55616) 2024-09-10T08:09:42.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.007+0000 7fdee3e00640 1 -- 172.21.15.2:0/2012277363 learned_addr learned my addr 172.21.15.2:0/2012277363 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.009+0000 7fdee8a00640 1 --2- 172.21.15.2:0/2012277363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299e40 0x55b9f72aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.009+0000 7fdee3e00640 1 -- 172.21.15.2:0/2012277363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 msgr2=0x55b9f72acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.009+0000 7fdee3e00640 1 --2- 172.21.15.2:0/2012277363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72acef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.009+0000 7fdee3e00640 1 -- 172.21.15.2:0/2012277363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299e40 msgr2=0x55b9f72aa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.009+0000 7fdee3e00640 1 --2- 172.21.15.2:0/2012277363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299e40 0x55b9f72aa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.009+0000 7fdee3e00640 1 -- 172.21.15.2:0/2012277363 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9f72aafd0 con 0x55b9f72a4700 2024-09-10T08:09:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.009+0000 7fdee8a00640 1 --2- 172.21.15.2:0/2012277363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299e40 0x55b9f72aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:09:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.010+0000 7fdee3e00640 1 --2- 172.21.15.2:0/2012277363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f72a4700 0x55b9f72aa7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fded00093c0 tx=0x7fded00113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7e3a22d43cd4588 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.010+0000 7fdeca000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:09:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.011+0000 7fdeca000640 1 -- 172.21.15.2:0/2012277363 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded004d060 con 0x55b9f72a4700 2024-09-10T08:09:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.011+0000 7fdeca000640 1 -- 172.21.15.2:0/2012277363 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fded0029480 con 0x55b9f72a4700 2024-09-10T08:09:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.011+0000 7fdeeddd3740 1 -- 172.21.15.2:0/2012277363 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded002e070 con 0x55b9f72a4700 2024-09-10T08:09:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.012+0000 7fdeca000640 1 -- 172.21.15.2:0/2012277363 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fded0029680 con 0x55b9f72a4700 2024-09-10T08:09:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.012+0000 7fdeeddd3740 10 client.25573 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.012+0000 7fdeeddd3740 1 -- 172.21.15.2:0/2012277363 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b9f731eea0 con 0x55b9f72a4700 2024-09-10T08:09:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.013+0000 7fdeca000640 1 -- 172.21.15.2:0/2012277363 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded002e070 con 0x55b9f72a4700 2024-09-10T08:09:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.013+0000 7fdeca000640 1 -- 172.21.15.2:0/2012277363 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fded0047040 con 0x55b9f72a4700 2024-09-10T08:09:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.013+0000 7fdeeddd3740 10 client.25573 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.013+0000 7fdeeddd3740 10 client.25573 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:09:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.013+0000 7fdeeddd3740 20 client.25573 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.013+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/2012277363 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9f731f4b0 0x55b9f73218a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.013+0000 7fdeeddd3740 4 client.25573 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "db781f90", "release": true, "await": true}] 2024-09-10T08:09:42.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.013+0000 7fdeeddd3740 1 -- 172.21.15.2:0/2012277363 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "db781f90", "release": true, "await": true}) -- 0x55b9f72070d0 con 0x55b9f731f4b0 2024-09-10T08:09:42.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.014+0000 7fdee8a00640 1 --2- 172.21.15.2:0/2012277363 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9f731f4b0 0x55b9f73218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:42.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.015+0000 7fdee8a00640 1 --2- 172.21.15.2:0/2012277363 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9f731f4b0 0x55b9f73218a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302c00bb7118dd69 server_cookie=2819ba08c1a2cc63 in_seq=0 out_seq=0 2024-09-10T08:09:42.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.015+0000 7fdeca000640 10 client.25573 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:42.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.017+0000 7fdeca000640 1 -- 172.21.15.2:0/2012277363 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b9f72070d0 con 0x55b9f731f4b0 2024-09-10T08:09:42.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.017+0000 7fdeca000640 10 client.25573 handle_command_reply: tid=0 2024-09-10T08:09:42.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.017+0000 7fdeeddd3740 1 client.25573 shutdown 2024-09-10T08:09:42.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.018+0000 7fdeeddd3740 1 -- 172.21.15.2:0/2012277363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f72a4700 msgr2=0x55b9f72aa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:42.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.018+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/2012277363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f72a4700 0x55b9f72aa7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fded00093c0 tx=0x7fded00113e0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.018+0000 7fdeeddd3740 1 -- 172.21.15.2:0/2012277363 shutdown_connections 2024-09-10T08:09:42.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.018+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/2012277363 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9f731f4b0 0x55b9f73218a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.018+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/2012277363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9f72a4700 0x55b9f72aa7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.018+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/2012277363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9f7299e40 0x55b9f72aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.018+0000 7fdeeddd3740 1 --2- 172.21.15.2:0/2012277363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9f7299520 0x55b9f72acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.018+0000 7fdeeddd3740 1 -- 172.21.15.2:0/2012277363 >> 172.21.15.2:0/2012277363 conn(0x55b9f7295ab0 msgr2=0x55b9f72af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:42.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.018+0000 7fdeca000640 0 client.25573 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:42.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.020+0000 7fdeeddd3740 1 -- 172.21.15.2:0/2012277363 shutdown_connections 2024-09-10T08:09:42.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.020+0000 7fdeeddd3740 1 -- 172.21.15.2:0/2012277363 wait complete. 2024-09-10T08:09:42.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.020+0000 7fdeeddd3740 20 client.25573 trim_cache size 0 max 16384 2024-09-10T08:09:42.023 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:09:42.023 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:09:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:09:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 4, 2024-09-10T08:09:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:09:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "db781f90": { 2024-09-10T08:09:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 4, 2024-09-10T08:09:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:09:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:09:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:09:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:09:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:09:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:09:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 150.0, 2024-09-10T08:09:42.024 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:09:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:09:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:09:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:09:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:09:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:09:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:42.025 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:42.025 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:09:42.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.023+0000 7fdee9e00640 1 -- 172.21.15.2:0/3582337700 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdecc0a2c00 msgr2=0x7fdecc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:42.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.023+0000 7fdee9e00640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdecc0a2c00 0x7fdecc0a50c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00b630 tx=0x7fdedc018f70 comp rx=0 tx=0).stop 2024-09-10T08:09:42.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.023+0000 7fdee9e00640 1 -- 172.21.15.2:0/3582337700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdee410bfc0 msgr2=0x7fdee410ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:42.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.023+0000 7fdee9e00640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdee410bfc0 0x7fdee410ac60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00da10 tx=0x7fdedc00ded0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.024+0000 7fdee9e00640 1 -- 172.21.15.2:0/3582337700 shutdown_connections 2024-09-10T08:09:42.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.024+0000 7fdee9e00640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdee410bfc0 0x7fdee410ac60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.024+0000 7fdee9e00640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdecc0a2c00 0x7fdecc0a50c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.024+0000 7fdee9e00640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdee41034b0 0x7fdee419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.024+0000 7fdee9e00640 1 --2- 172.21.15.2:0/3582337700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee4102b90 0x7fdee419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:42.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.024+0000 7fdee9e00640 1 -- 172.21.15.2:0/3582337700 >> 172.21.15.2:0/3582337700 conn(0x7fdee40fc610 msgr2=0x7fdee410d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:42.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.025+0000 7fdee9e00640 1 -- 172.21.15.2:0/3582337700 shutdown_connections 2024-09-10T08:09:42.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:42.027+0000 7fdee9e00640 1 -- 172.21.15.2:0/3582337700 wait complete. 2024-09-10T08:09:42.981 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'db781f90', total seconds elapsed: 34.5 2024-09-10T08:09:42.982 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T08:09:42.982 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:09:43.544 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:43.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:43 smithi002 ceph-mon[30563]: pgmap v434: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:09:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:43 smithi155 ceph-mon[35405]: pgmap v434: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:09:43.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:43 smithi107 ceph-mon[35048]: pgmap v434: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:09:44.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.254+0000 7f29c0a00640 1 -- 172.21.15.2:0/400717348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29bc103e10 msgr2=0x7f29bc1041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:44.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.254+0000 7f29c0a00640 1 --2- 172.21.15.2:0/400717348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29bc103e10 0x7f29bc1041f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009900 tx=0x7f29ac02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:09:44.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.254+0000 7f29c0a00640 1 -- 172.21.15.2:0/400717348 shutdown_connections 2024-09-10T08:09:44.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.254+0000 7f29c0a00640 1 --2- 172.21.15.2:0/400717348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29bc108f40 0x7f29bc110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:44.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.254+0000 7f29c0a00640 1 --2- 172.21.15.2:0/400717348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29bc1047c0 0x7f29bc108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:44.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.254+0000 7f29c0a00640 1 --2- 172.21.15.2:0/400717348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29bc103e10 0x7f29bc1041f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:44.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.254+0000 7f29c0a00640 1 -- 172.21.15.2:0/400717348 >> 172.21.15.2:0/400717348 conn(0x7f29bc0ff870 msgr2=0x7f29bc101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:44.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.255+0000 7f29c0a00640 1 -- 172.21.15.2:0/400717348 shutdown_connections 2024-09-10T08:09:44.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.255+0000 7f29c0a00640 1 -- 172.21.15.2:0/400717348 wait complete. 2024-09-10T08:09:44.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.256+0000 7f29c0a00640 1 Processor -- start 2024-09-10T08:09:44.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.257+0000 7f29c0a00640 1 -- start start 2024-09-10T08:09:44.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.257+0000 7f29c0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29bc103e10 0x7f29bc19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.257+0000 7f29c0a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29bc1047c0 0x7f29bc19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.257+0000 7f29c0a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29bc108f40 0x7f29bc19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.257+0000 7f29c0a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc113ff0 con 0x7f29bc108f40 2024-09-10T08:09:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.257+0000 7f29c0a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc113e70 con 0x7f29bc103e10 2024-09-10T08:09:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.257+0000 7f29c0a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc114170 con 0x7f29bc1047c0 2024-09-10T08:09:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.258+0000 7f29baa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29bc1047c0 0x7f29bc19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.258+0000 7f29baa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29bc1047c0 0x7f29bc19a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59354/0 (socket says 172.21.15.2:59354) 2024-09-10T08:09:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.258+0000 7f29baa00640 1 -- 172.21.15.2:0/2290619109 learned_addr learned my addr 172.21.15.2:0/2290619109 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.258+0000 7f29bb400640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29bc103e10 0x7f29bc19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:44.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.258+0000 7f29bb400640 1 -- 172.21.15.2:0/2290619109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29bc1047c0 msgr2=0x7f29bc19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.258+0000 7f29bb400640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29bc1047c0 0x7f29bc19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.258+0000 7f29bb400640 1 -- 172.21.15.2:0/2290619109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29bc108f40 msgr2=0x7f29bc19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.258+0000 7f29bb400640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29bc108f40 0x7f29bc19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.258+0000 7f29bb400640 1 -- 172.21.15.2:0/2290619109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc19f400 con 0x7f29bc103e10 2024-09-10T08:09:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.258+0000 7f29baa00640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29bc1047c0 0x7f29bc19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.259+0000 7f29bb400640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29bc103e10 0x7f29bc19a450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f29ac02a910 tx=0x7f29ac003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.259+0000 7f29b3e00640 1 -- 172.21.15.2:0/2290619109 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac03f070 con 0x7f29bc103e10 2024-09-10T08:09:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.259+0000 7f29b3e00640 1 -- 172.21.15.2:0/2290619109 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29ac031c90 con 0x7f29bc103e10 2024-09-10T08:09:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.260+0000 7f29b3e00640 1 -- 172.21.15.2:0/2290619109 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac035030 con 0x7f29bc103e10 2024-09-10T08:09:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.260+0000 7f29c0a00640 1 -- 172.21.15.2:0/2290619109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29bc19f690 con 0x7f29bc103e10 2024-09-10T08:09:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.261+0000 7f29c0a00640 1 -- 172.21.15.2:0/2290619109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29bc1a6fd0 con 0x7f29bc103e10 2024-09-10T08:09:44.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.262+0000 7f29c0a00640 1 -- 172.21.15.2:0/2290619109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29bc1052f0 con 0x7f29bc103e10 2024-09-10T08:09:44.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.266+0000 7f29b3e00640 1 -- 172.21.15.2:0/2290619109 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f29ac042070 con 0x7f29bc103e10 2024-09-10T08:09:44.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.268+0000 7f29b3e00640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f29a00a2c00 0x7f29a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:44.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.268+0000 7f29b3e00640 1 -- 172.21.15.2:0/2290619109 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f29ac0eff80 con 0x7f29bc103e10 2024-09-10T08:09:44.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.268+0000 7f29baa00640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f29a00a2c00 0x7f29a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:44.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.269+0000 7f29baa00640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f29a00a2c00 0x7f29a00a50c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f29bc101dc0 tx=0x7f29a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.271+0000 7f29b3e00640 1 -- 172.21.15.2:0/2290619109 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29ac0bca90 con 0x7f29bc103e10 2024-09-10T08:09:44.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.723+0000 7f29c0a00640 1 -- 172.21.15.2:0/2290619109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29bc1041f0 con 0x7f29bc103e10 2024-09-10T08:09:44.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.724+0000 7f29b3e00640 1 -- 172.21.15.2:0/2290619109 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f29ac0bc390 con 0x7f29bc103e10 2024-09-10T08:09:44.725 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:09:44.725 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:09:44.725 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:09:44.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.727+0000 7f29c0a00640 1 -- 172.21.15.2:0/2290619109 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f29a00a2c00 msgr2=0x7f29a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:44.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.727+0000 7f29c0a00640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f29a00a2c00 0x7f29a00a50c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f29bc101dc0 tx=0x7f29a8005f90 comp rx=0 tx=0).stop 2024-09-10T08:09:44.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.727+0000 7f29c0a00640 1 -- 172.21.15.2:0/2290619109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29bc103e10 msgr2=0x7f29bc19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:44.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.727+0000 7f29c0a00640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29bc103e10 0x7f29bc19a450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f29ac02a910 tx=0x7f29ac003b20 comp rx=0 tx=0).stop 2024-09-10T08:09:44.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.727+0000 7f29c0a00640 1 -- 172.21.15.2:0/2290619109 shutdown_connections 2024-09-10T08:09:44.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.727+0000 7f29c0a00640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29bc108f40 0x7f29bc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:44.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.727+0000 7f29c0a00640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f29a00a2c00 0x7f29a00a50c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:44.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.727+0000 7f29c0a00640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29bc1047c0 0x7f29bc19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:44.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.727+0000 7f29c0a00640 1 --2- 172.21.15.2:0/2290619109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29bc103e10 0x7f29bc19a450 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:44.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.727+0000 7f29c0a00640 1 -- 172.21.15.2:0/2290619109 >> 172.21.15.2:0/2290619109 conn(0x7f29bc0ff870 msgr2=0x7f29bc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:44.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.727+0000 7f29c0a00640 1 -- 172.21.15.2:0/2290619109 shutdown_connections 2024-09-10T08:09:44.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:44.728+0000 7f29c0a00640 1 -- 172.21.15.2:0/2290619109 wait complete. 2024-09-10T08:09:45.635 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T08:09:45.635 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T08:09:45.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:45 smithi155 ceph-mon[35405]: pgmap v435: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:09:45.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:45 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2290619109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:45.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:45 smithi107 ceph-mon[35048]: pgmap v435: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:09:45.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:45 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2290619109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:46.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:45 smithi002 ceph-mon[30563]: pgmap v435: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:09:46.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:45 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2290619109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:46.062 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:46.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.719+0000 7f1c3a800640 1 -- 172.21.15.2:0/673598400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34074510 msgr2=0x7f1c340748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.719+0000 7f1c3a800640 1 --2- 172.21.15.2:0/673598400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34074510 0x7f1c340748f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009860 tx=0x7f1c2802a3d0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.719+0000 7f1c3a800640 1 -- 172.21.15.2:0/673598400 shutdown_connections 2024-09-10T08:09:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.719+0000 7f1c3a800640 1 --2- 172.21.15.2:0/673598400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c3410e280 0x7f1c34110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.719+0000 7f1c3a800640 1 --2- 172.21.15.2:0/673598400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c34074e30 0x7f1c3410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.719+0000 7f1c3a800640 1 --2- 172.21.15.2:0/673598400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34074510 0x7f1c340748f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.719+0000 7f1c3a800640 1 -- 172.21.15.2:0/673598400 >> 172.21.15.2:0/673598400 conn(0x7f1c340fd740 msgr2=0x7f1c340ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.719+0000 7f1c3a800640 1 -- 172.21.15.2:0/673598400 shutdown_connections 2024-09-10T08:09:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.719+0000 7f1c3a800640 1 -- 172.21.15.2:0/673598400 wait complete. 2024-09-10T08:09:46.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.721+0000 7f1c3a800640 1 Processor -- start 2024-09-10T08:09:46.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.721+0000 7f1c3a800640 1 -- start start 2024-09-10T08:09:46.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.721+0000 7f1c3a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c34074510 0x7f1c3419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.721+0000 7f1c3a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34074e30 0x7f1c3419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.721+0000 7f1c3a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3410e280 0x7f1c3419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.721+0000 7f1c3a800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34113fb0 con 0x7f1c34074e30 2024-09-10T08:09:46.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.722+0000 7f1c3a800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34113e30 con 0x7f1c3410e280 2024-09-10T08:09:46.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.722+0000 7f1c3a800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34114130 con 0x7f1c34074510 2024-09-10T08:09:46.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.722+0000 7f1c39400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c34074510 0x7f1c3419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.722+0000 7f1c39400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c34074510 0x7f1c3419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59368/0 (socket says 172.21.15.2:59368) 2024-09-10T08:09:46.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.722+0000 7f1c39400640 1 -- 172.21.15.2:0/3282271859 learned_addr learned my addr 172.21.15.2:0/3282271859 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:46.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.722+0000 7f1c38a00640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34074e30 0x7f1c3419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.722+0000 7f1c39e00640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3410e280 0x7f1c3419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.723+0000 7f1c39400640 1 -- 172.21.15.2:0/3282271859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3410e280 msgr2=0x7f1c3419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:46.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.723+0000 7f1c39400640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3410e280 0x7f1c3419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.723+0000 7f1c39400640 1 -- 172.21.15.2:0/3282271859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34074e30 msgr2=0x7f1c3419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:46.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.723+0000 7f1c39400640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34074e30 0x7f1c3419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.723+0000 7f1c39400640 1 -- 172.21.15.2:0/3282271859 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3419f450 con 0x7f1c34074510 2024-09-10T08:09:46.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.723+0000 7f1c38a00640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34074e30 0x7f1c3419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:46.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.723+0000 7f1c39e00640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3410e280 0x7f1c3419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:46.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.724+0000 7f1c39400640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c34074510 0x7f1c3419a4a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c2802a8f0 tx=0x7f1c28030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:46.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.724+0000 7f1c32000640 1 -- 172.21.15.2:0/3282271859 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28040020 con 0x7f1c34074510 2024-09-10T08:09:46.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.724+0000 7f1c3a800640 1 -- 172.21.15.2:0/3282271859 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3419f6e0 con 0x7f1c34074510 2024-09-10T08:09:46.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.725+0000 7f1c32000640 1 -- 172.21.15.2:0/3282271859 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c28004440 con 0x7f1c34074510 2024-09-10T08:09:46.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.725+0000 7f1c32000640 1 -- 172.21.15.2:0/3282271859 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c280046c0 con 0x7f1c34074510 2024-09-10T08:09:46.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.725+0000 7f1c3a800640 1 -- 172.21.15.2:0/3282271859 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c341a7020 con 0x7f1c34074510 2024-09-10T08:09:46.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.728+0000 7f1c32000640 1 -- 172.21.15.2:0/3282271859 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1c28043020 con 0x7f1c34074510 2024-09-10T08:09:46.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.729+0000 7f1c3ea15740 1 Processor -- start 2024-09-10T08:09:46.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.730+0000 7f1c3ea15740 1 -- start start 2024-09-10T08:09:46.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.730+0000 7f1c32000640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1c1c0a2c00 0x7f1c1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.730+0000 7f1c3ea15740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0e480 0x56328af0e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.730+0000 7f1c3ea15740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0eda0 0x56328af21a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.730+0000 7f1c3ea15740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af21f50 0x56328af24340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.730+0000 7f1c3ea15740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56328ae74530 con 0x56328af0e480 2024-09-10T08:09:46.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.730+0000 7f1c3ea15740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56328ae1b860 con 0x56328af21f50 2024-09-10T08:09:46.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.730+0000 7f1c3ea15740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56328ae704f0 con 0x56328af0eda0 2024-09-10T08:09:46.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.730+0000 7f1c32000640 1 -- 172.21.15.2:0/3282271859 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1c280eff70 con 0x7f1c34074510 2024-09-10T08:09:46.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.731+0000 7f1c38a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0eda0 0x56328af21a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.731+0000 7f1c38a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0eda0 0x56328af21a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59384/0 (socket says 172.21.15.2:59384) 2024-09-10T08:09:46.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.731+0000 7f1c38a00640 1 -- 172.21.15.2:0/40713123 learned_addr learned my addr 172.21.15.2:0/40713123 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:46.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.731+0000 7f1c39e00640 1 --2- 172.21.15.2:0/40713123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af21f50 0x56328af24340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.731+0000 7f1c39e00640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1c1c0a2c00 0x7f1c1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.731+0000 7f1c39400640 1 --2- 172.21.15.2:0/40713123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0e480 0x56328af0e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.732+0000 7f1c39e00640 1 -- 172.21.15.2:0/40713123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0eda0 msgr2=0x56328af21a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:46.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.732+0000 7f1c39e00640 1 --2- 172.21.15.2:0/40713123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0eda0 0x56328af21a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.732+0000 7f1c39e00640 1 -- 172.21.15.2:0/40713123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0e480 msgr2=0x56328af0e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:46.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.732+0000 7f1c39e00640 1 --2- 172.21.15.2:0/40713123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0e480 0x56328af0e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.732+0000 7f1c39e00640 1 -- 172.21.15.2:0/40713123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328abe5b90 con 0x56328af21f50 2024-09-10T08:09:46.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.732+0000 7f1c39e00640 1 --2- 172.21.15.2:0/40713123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af21f50 0x56328af24340 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009fd0 tx=0x7f1c2c00f950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bec61b5c05470f1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:46.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.733+0000 7f1c39e00640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1c1c0a2c00 0x7f1c1c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0099e0 tx=0x7f1c2c01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.733+0000 7f1c26600640 1 -- 172.21.15.2:0/40713123 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c02b070 con 0x56328af21f50 2024-09-10T08:09:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.733+0000 7f1c26600640 1 -- 172.21.15.2:0/40713123 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c2c011900 con 0x56328af21f50 2024-09-10T08:09:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.733+0000 7f1c26600640 1 -- 172.21.15.2:0/40713123 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c022070 con 0x56328af21f50 2024-09-10T08:09:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.734+0000 7f1c3ea15740 1 -- 172.21.15.2:0/40713123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af21f50 msgr2=0x56328af24340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.734+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/40713123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af21f50 0x56328af24340 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009fd0 tx=0x7f1c2c00f950 comp rx=0 tx=0).stop 2024-09-10T08:09:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.734+0000 7f1c3ea15740 1 -- 172.21.15.2:0/40713123 shutdown_connections 2024-09-10T08:09:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.734+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/40713123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af21f50 0x56328af24340 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.734+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/40713123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0eda0 0x56328af21a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.734+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/40713123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0e480 0x56328af0e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.734+0000 7f1c3ea15740 1 -- 172.21.15.2:0/40713123 >> 172.21.15.2:0/40713123 conn(0x56328af0aaf0 msgr2=0x56328af0aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.734+0000 7f1c3ea15740 1 -- 172.21.15.2:0/40713123 shutdown_connections 2024-09-10T08:09:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.734+0000 7f1c3ea15740 1 -- 172.21.15.2:0/40713123 wait complete. 2024-09-10T08:09:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.735+0000 7f1c3ea15740 1 -- start start 2024-09-10T08:09:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.735+0000 7f1c3ea15740 1 Processor -- start 2024-09-10T08:09:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.736+0000 7f1c3ea15740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0e480 0x56328af1ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.736+0000 7f1c3ea15740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0eda0 0x56328af1f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.736+0000 7f1c3ea15740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af21f50 0x56328af1f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.736+0000 7f1c3ea15740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56328af24a60 con 0x56328af0eda0 2024-09-10T08:09:46.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.736+0000 7f1c3ea15740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56328ae2ee60 con 0x56328af21f50 2024-09-10T08:09:46.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.736+0000 7f1c3ea15740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56328af24be0 con 0x56328af0e480 2024-09-10T08:09:46.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.736+0000 7f1c3ea15740 10 client.0 _ll_register_callbacks cb 0x56328ae33cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:46.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.736+0000 7f1c39400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0e480 0x56328af1ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.736+0000 7f1c39400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0e480 0x56328af1ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59390/0 (socket says 172.21.15.2:59390) 2024-09-10T08:09:46.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.736+0000 7f1c39400640 1 -- 172.21.15.2:0/1382881568 learned_addr learned my addr 172.21.15.2:0/1382881568 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:46.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.736+0000 7f1c38a00640 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0eda0 0x56328af1f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.736+0000 7f1c39e00640 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af21f50 0x56328af1f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.737+0000 7f1c39e00640 1 -- 172.21.15.2:0/1382881568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0e480 msgr2=0x56328af1ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:46.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.737+0000 7f1c39e00640 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0e480 0x56328af1ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.737+0000 7f1c39e00640 1 -- 172.21.15.2:0/1382881568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0eda0 msgr2=0x56328af1f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:46.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.737+0000 7f1c39e00640 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0eda0 0x56328af1f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.737+0000 7f1c39e00640 1 -- 172.21.15.2:0/1382881568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328ae3a990 con 0x56328af21f50 2024-09-10T08:09:46.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.737+0000 7f1c38a00640 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0eda0 0x56328af1f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:46.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.737+0000 7f1c39e00640 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af21f50 0x56328af1f9e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c005c10 tx=0x7f1c2c0268e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e95a85ef46b7c595 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:46.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.738+0000 7f1c1aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:09:46.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.738+0000 7f1c1aa00640 1 -- 172.21.15.2:0/1382881568 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c022070 con 0x56328af21f50 2024-09-10T08:09:46.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.738+0000 7f1c1aa00640 1 -- 172.21.15.2:0/1382881568 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1c2c011460 con 0x56328af21f50 2024-09-10T08:09:46.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.738+0000 7f1c3ea15740 1 -- 172.21.15.2:0/1382881568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c2c011900 con 0x56328af21f50 2024-09-10T08:09:46.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.739+0000 7f1c1aa00640 1 -- 172.21.15.2:0/1382881568 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c2c02b040 con 0x56328af21f50 2024-09-10T08:09:46.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.739+0000 7f1c3ea15740 10 client.25607 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:46.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.739+0000 7f1c3ea15740 1 -- 172.21.15.2:0/1382881568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56328af307d0 con 0x56328af21f50 2024-09-10T08:09:46.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.739+0000 7f1c1aa00640 1 -- 172.21.15.2:0/1382881568 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c2c011900 con 0x56328af21f50 2024-09-10T08:09:46.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.739+0000 7f1c1aa00640 1 -- 172.21.15.2:0/1382881568 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1c2c005630 con 0x56328af21f50 2024-09-10T08:09:46.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.740+0000 7f1c3ea15740 10 client.25607 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:46.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.740+0000 7f1c3ea15740 10 client.25607 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:09:46.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.740+0000 7f1c3ea15740 20 client.25607 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:46.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.740+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56328af31b80 0x56328af33f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.740+0000 7f1c3ea15740 4 client.25607 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:46.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.740+0000 7f1c3ea15740 1 -- 172.21.15.2:0/1382881568 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56328ae17310 con 0x56328af31b80 2024-09-10T08:09:46.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.740+0000 7f1c39400640 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56328af31b80 0x56328af33f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.741+0000 7f1c39400640 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56328af31b80 0x56328af33f70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71681fd89066baa7 server_cookie=383c017d761c63c2 in_seq=0 out_seq=0 2024-09-10T08:09:46.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.741+0000 7f1c1aa00640 10 client.25607 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:46.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.742+0000 7f1c1aa00640 1 -- 172.21.15.2:0/1382881568 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56328ae17310 con 0x56328af31b80 2024-09-10T08:09:46.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.742+0000 7f1c1aa00640 10 client.25607 handle_command_reply: tid=0 2024-09-10T08:09:46.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.742+0000 7f1c3ea15740 1 client.25607 shutdown 2024-09-10T08:09:46.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.742+0000 7f1c3ea15740 1 -- 172.21.15.2:0/1382881568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af21f50 msgr2=0x56328af1f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:46.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.742+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af21f50 0x56328af1f9e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c005c10 tx=0x7f1c2c0268e0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.742+0000 7f1c3ea15740 1 -- 172.21.15.2:0/1382881568 shutdown_connections 2024-09-10T08:09:46.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.742+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56328af31b80 0x56328af33f70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.742+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af21f50 0x56328af1f9e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.742+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0eda0 0x56328af1f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.742+0000 7f1c1aa00640 0 client.25607 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:46.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.742+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/1382881568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0e480 0x56328af1ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.742+0000 7f1c3ea15740 1 -- 172.21.15.2:0/1382881568 >> 172.21.15.2:0/1382881568 conn(0x56328af0aaf0 msgr2=0x56328af09c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:46.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.743+0000 7f1c3ea15740 1 -- 172.21.15.2:0/1382881568 shutdown_connections 2024-09-10T08:09:46.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.743+0000 7f1c3ea15740 1 -- 172.21.15.2:0/1382881568 wait complete. 2024-09-10T08:09:46.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.743+0000 7f1c3ea15740 20 client.25607 trim_cache size 0 max 16384 2024-09-10T08:09:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.761+0000 7f1c3ea15740 1 Processor -- start 2024-09-10T08:09:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.762+0000 7f1c3ea15740 1 -- start start 2024-09-10T08:09:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.762+0000 7f1c3ea15740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af0e480 0x56328af0a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.762+0000 7f1c3ea15740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0eda0 0x56328af1f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.762+0000 7f1c3ea15740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af21f50 0x56328af1fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.762+0000 7f1c3ea15740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341af320 con 0x56328af21f50 2024-09-10T08:09:46.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.762+0000 7f1c3ea15740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341acbf0 con 0x56328af0e480 2024-09-10T08:09:46.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.762+0000 7f1c3ea15740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341aaa60 con 0x56328af0eda0 2024-09-10T08:09:46.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.762+0000 7f1c39e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af21f50 0x56328af1fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.763+0000 7f1c38a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0eda0 0x56328af1f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.763+0000 7f1c38a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0eda0 0x56328af1f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59400/0 (socket says 172.21.15.2:59400) 2024-09-10T08:09:46.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.763+0000 7f1c38a00640 1 -- 172.21.15.2:0/695562362 learned_addr learned my addr 172.21.15.2:0/695562362 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:46.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.763+0000 7f1c39400640 1 --2- 172.21.15.2:0/695562362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af0e480 0x56328af0a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.763+0000 7f1c38a00640 1 -- 172.21.15.2:0/695562362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af0e480 msgr2=0x56328af0a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:46.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.763+0000 7f1c38a00640 1 --2- 172.21.15.2:0/695562362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af0e480 0x56328af0a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.763+0000 7f1c38a00640 1 -- 172.21.15.2:0/695562362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af21f50 msgr2=0x56328af1fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:46.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.763+0000 7f1c38a00640 1 --2- 172.21.15.2:0/695562362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af21f50 0x56328af1fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.763+0000 7f1c38a00640 1 -- 172.21.15.2:0/695562362 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328af342e0 con 0x56328af0eda0 2024-09-10T08:09:46.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.764+0000 7f1c38a00640 1 --2- 172.21.15.2:0/695562362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0eda0 0x56328af1f630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009530 tx=0x7f1c20009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddb09a16f3e9c389 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:46.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.764+0000 7f1c25c00640 1 -- 172.21.15.2:0/695562362 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20004020 con 0x56328af0eda0 2024-09-10T08:09:46.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.764+0000 7f1c25c00640 1 -- 172.21.15.2:0/695562362 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c2000fe00 con 0x56328af0eda0 2024-09-10T08:09:46.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.764+0000 7f1c25c00640 1 -- 172.21.15.2:0/695562362 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c200047a0 con 0x56328af0eda0 2024-09-10T08:09:46.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.765+0000 7f1c3ea15740 1 -- 172.21.15.2:0/695562362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0eda0 msgr2=0x56328af1f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:46.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.765+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/695562362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0eda0 0x56328af1f630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009530 tx=0x7f1c20009260 comp rx=0 tx=0).stop 2024-09-10T08:09:46.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.765+0000 7f1c3ea15740 1 -- 172.21.15.2:0/695562362 shutdown_connections 2024-09-10T08:09:46.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.765+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/695562362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af21f50 0x56328af1fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.765+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/695562362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af0eda0 0x56328af1f630 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.765+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/695562362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af0e480 0x56328af0a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.765+0000 7f1c3ea15740 1 -- 172.21.15.2:0/695562362 >> 172.21.15.2:0/695562362 conn(0x56328af0aaf0 msgr2=0x56328af1ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:46.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.765+0000 7f1c3ea15740 1 -- 172.21.15.2:0/695562362 shutdown_connections 2024-09-10T08:09:46.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.765+0000 7f1c3ea15740 1 -- 172.21.15.2:0/695562362 wait complete. 2024-09-10T08:09:46.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.766+0000 7f1c3ea15740 1 -- start start 2024-09-10T08:09:46.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.766+0000 7f1c3ea15740 1 Processor -- start 2024-09-10T08:09:46.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.766+0000 7f1c3ea15740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0e480 0x56328af1bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.766+0000 7f1c3ea15740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af0eda0 0x56328af1c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.767+0000 7f1c3ea15740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af21f50 0x56328af1ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.767+0000 7f1c3ea15740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341ab0b0 con 0x56328af0e480 2024-09-10T08:09:46.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.767+0000 7f1c3ea15740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341ac520 con 0x56328af0eda0 2024-09-10T08:09:46.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.767+0000 7f1c3ea15740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341ab780 con 0x56328af21f50 2024-09-10T08:09:46.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.767+0000 7f1c3ea15740 10 client.0 _ll_register_callbacks cb 0x7f1c2c029070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:46.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.767+0000 7f1c39400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0e480 0x56328af1bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.767+0000 7f1c39400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0e480 0x56328af1bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56816/0 (socket says 172.21.15.2:56816) 2024-09-10T08:09:46.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.767+0000 7f1c39400640 1 -- 172.21.15.2:0/2864936027 learned_addr learned my addr 172.21.15.2:0/2864936027 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:46.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.767+0000 7f1c39e00640 1 --2- 172.21.15.2:0/2864936027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af21f50 0x56328af1ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.767+0000 7f1c38a00640 1 --2- 172.21.15.2:0/2864936027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af0eda0 0x56328af1c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.768+0000 7f1c39400640 1 -- 172.21.15.2:0/2864936027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af21f50 msgr2=0x56328af1ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:46.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.768+0000 7f1c39400640 1 --2- 172.21.15.2:0/2864936027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af21f50 0x56328af1ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.768+0000 7f1c39400640 1 -- 172.21.15.2:0/2864936027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af0eda0 msgr2=0x56328af1c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:46.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.768+0000 7f1c39400640 1 --2- 172.21.15.2:0/2864936027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af0eda0 0x56328af1c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.768+0000 7f1c39400640 1 -- 172.21.15.2:0/2864936027 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328af9b750 con 0x56328af0e480 2024-09-10T08:09:46.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.768+0000 7f1c39400640 1 --2- 172.21.15.2:0/2864936027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0e480 0x56328af1bfc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1c280ed5c0 tx=0x7f1c280dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1d1faf07a527db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:46.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.768+0000 7f1c1aa00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:09:46.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.768+0000 7f1c1aa00640 1 -- 172.21.15.2:0/2864936027 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c280bc380 con 0x56328af0e480 2024-09-10T08:09:46.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.768+0000 7f1c1aa00640 1 -- 172.21.15.2:0/2864936027 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1c280ed730 con 0x56328af0e480 2024-09-10T08:09:46.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.768+0000 7f1c3ea15740 1 -- 172.21.15.2:0/2864936027 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c2000fe00 con 0x56328af0e480 2024-09-10T08:09:46.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.769+0000 7f1c1aa00640 1 -- 172.21.15.2:0/2864936027 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c280c0730 con 0x56328af0e480 2024-09-10T08:09:46.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.769+0000 7f1c3ea15740 10 client.15573 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:46.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.769+0000 7f1c3ea15740 1 -- 172.21.15.2:0/2864936027 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56328af9ca10 con 0x56328af0e480 2024-09-10T08:09:46.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.769+0000 7f1c1aa00640 1 -- 172.21.15.2:0/2864936027 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c2000fe00 con 0x56328af0e480 2024-09-10T08:09:46.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.769+0000 7f1c1aa00640 1 -- 172.21.15.2:0/2864936027 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1c2809e020 con 0x56328af0e480 2024-09-10T08:09:46.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.770+0000 7f1c3ea15740 10 client.15573 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:46.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.770+0000 7f1c3ea15740 10 client.15573 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:09:46.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.770+0000 7f1c3ea15740 20 client.15573 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:46.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.770+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/2864936027 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56328af9d130 0x56328af9f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:46.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.770+0000 7f1c3ea15740 4 client.15573 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T08:09:46.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.770+0000 7f1c3ea15740 1 -- 172.21.15.2:0/2864936027 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x56328ae81c60 con 0x56328af9d130 2024-09-10T08:09:46.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.770+0000 7f1c38a00640 1 --2- 172.21.15.2:0/2864936027 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56328af9d130 0x56328af9f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:46.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.771+0000 7f1c38a00640 1 --2- 172.21.15.2:0/2864936027 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56328af9d130 0x56328af9f550 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c40324a44851d1 server_cookie=b4b6681a94026db7 in_seq=0 out_seq=0 2024-09-10T08:09:46.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.771+0000 7f1c1aa00640 10 client.15573 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:46.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.780+0000 7f1c1aa00640 1 -- 172.21.15.2:0/2864936027 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56328ae81c60 con 0x56328af9d130 2024-09-10T08:09:46.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.780+0000 7f1c1aa00640 10 client.15573 handle_command_reply: tid=0 2024-09-10T08:09:46.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.780+0000 7f1c3ea15740 1 client.15573 shutdown 2024-09-10T08:09:46.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.780+0000 7f1c3ea15740 1 -- 172.21.15.2:0/2864936027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0e480 msgr2=0x56328af1bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:46.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.780+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/2864936027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0e480 0x56328af1bfc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1c280ed5c0 tx=0x7f1c280dc040 comp rx=0 tx=0).stop 2024-09-10T08:09:46.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.780+0000 7f1c3ea15740 1 -- 172.21.15.2:0/2864936027 shutdown_connections 2024-09-10T08:09:46.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.780+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/2864936027 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56328af9d130 0x56328af9f550 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.780+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/2864936027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56328af21f50 0x56328af1ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.780+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/2864936027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56328af0eda0 0x56328af1c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.780+0000 7f1c3ea15740 1 --2- 172.21.15.2:0/2864936027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56328af0e480 0x56328af1bfc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.780+0000 7f1c1aa00640 0 client.15573 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:46.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.780+0000 7f1c3ea15740 1 -- 172.21.15.2:0/2864936027 >> 172.21.15.2:0/2864936027 conn(0x56328af0aaf0 msgr2=0x56328af1f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:46.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.781+0000 7f1c3ea15740 1 -- 172.21.15.2:0/2864936027 shutdown_connections 2024-09-10T08:09:46.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.781+0000 7f1c3ea15740 1 -- 172.21.15.2:0/2864936027 wait complete. 2024-09-10T08:09:46.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.781+0000 7f1c3ea15740 20 client.15573 trim_cache size 0 max 16384 2024-09-10T08:09:46.782 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:09:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:09:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:09:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 6, 2024-09-10T08:09:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:09:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "391eb929": { 2024-09-10T08:09:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 6, 2024-09-10T08:09:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:09:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:09:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:09:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:09:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:09:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:09:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 150.0, 2024-09-10T08:09:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:09:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:09:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:09:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:09:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:09:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:09:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:09:46.784 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:09:46.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.782+0000 7f1c3a800640 1 -- 172.21.15.2:0/3282271859 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1c1c0a2c00 msgr2=0x7f1c1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:46.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.782+0000 7f1c3a800640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1c1c0a2c00 0x7f1c1c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0099e0 tx=0x7f1c2c01e000 comp rx=0 tx=0).stop 2024-09-10T08:09:46.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.782+0000 7f1c3a800640 1 -- 172.21.15.2:0/3282271859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c34074510 msgr2=0x7f1c3419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:46.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.782+0000 7f1c3a800640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c34074510 0x7f1c3419a4a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c2802a8f0 tx=0x7f1c28030650 comp rx=0 tx=0).stop 2024-09-10T08:09:46.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.782+0000 7f1c3a800640 1 -- 172.21.15.2:0/3282271859 shutdown_connections 2024-09-10T08:09:46.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.782+0000 7f1c3a800640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3410e280 0x7f1c3419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.782+0000 7f1c3a800640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c34074e30 0x7f1c3419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.782+0000 7f1c3a800640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1c1c0a2c00 0x7f1c1c0a50c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.782+0000 7f1c3a800640 1 --2- 172.21.15.2:0/3282271859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c34074510 0x7f1c3419a4a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:46.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.782+0000 7f1c3a800640 1 -- 172.21.15.2:0/3282271859 >> 172.21.15.2:0/3282271859 conn(0x7f1c340fd740 msgr2=0x7f1c340feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:46.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.782+0000 7f1c3a800640 1 -- 172.21.15.2:0/3282271859 shutdown_connections 2024-09-10T08:09:46.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:46.783+0000 7f1c3a800640 1 -- 172.21.15.2:0/3282271859 wait complete. 2024-09-10T08:09:47.679 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '391eb929', quiesce took 4.7 seconds. Will release after: 25.3 2024-09-10T08:09:47.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:47 smithi155 ceph-mon[35405]: pgmap v436: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:09:47.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:47 smithi107 ceph-mon[35048]: pgmap v436: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:09:48.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:47 smithi002 ceph-mon[30563]: pgmap v436: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:09:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.090+0000 7ff82a000640 1 -- 172.21.15.2:0/1129699284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff824074620 msgr2=0x7ff824074a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.090+0000 7ff82a000640 1 --2- 172.21.15.2:0/1129699284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff824074620 0x7ff824074a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff818009900 tx=0x7ff81802a400 comp rx=0 tx=0).stop 2024-09-10T08:09:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.090+0000 7ff82a000640 1 -- 172.21.15.2:0/1129699284 shutdown_connections 2024-09-10T08:09:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.090+0000 7ff82a000640 1 --2- 172.21.15.2:0/1129699284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff824109cf0 0x7ff82410c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.090+0000 7ff82a000640 1 --2- 172.21.15.2:0/1129699284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff824074f60 0x7ff8241097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.090+0000 7ff82a000640 1 --2- 172.21.15.2:0/1129699284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff824074620 0x7ff824074a20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.090+0000 7ff82a000640 1 -- 172.21.15.2:0/1129699284 >> 172.21.15.2:0/1129699284 conn(0x7ff8240fd0d0 msgr2=0x7ff8240ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.090+0000 7ff82a000640 1 -- 172.21.15.2:0/1129699284 shutdown_connections 2024-09-10T08:09:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.090+0000 7ff82a000640 1 -- 172.21.15.2:0/1129699284 wait complete. 2024-09-10T08:09:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.091+0000 7ff82a000640 1 Processor -- start 2024-09-10T08:09:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.091+0000 7ff82a000640 1 -- start start 2024-09-10T08:09:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.092+0000 7ff82a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff824074620 0x7ff824199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.092+0000 7ff82a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff824074f60 0x7ff82419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.092+0000 7ff82a000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff824109cf0 0x7ff8241a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.092+0000 7ff82a000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff82410f920 con 0x7ff824109cf0 2024-09-10T08:09:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.092+0000 7ff82a000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff82410f7a0 con 0x7ff824074f60 2024-09-10T08:09:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.092+0000 7ff82a000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff82410faa0 con 0x7ff824074620 2024-09-10T08:09:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.092+0000 7ff828c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff824074620 0x7ff824199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.092+0000 7ff829600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff824109cf0 0x7ff8241a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.092+0000 7ff828c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff824074620 0x7ff824199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:44872/0 (socket says 172.21.15.2:44872) 2024-09-10T08:09:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.092+0000 7ff828c00640 1 -- 172.21.15.2:0/1486685188 learned_addr learned my addr 172.21.15.2:0/1486685188 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.092+0000 7ff823e00640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff824074f60 0x7ff82419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.093+0000 7ff828c00640 1 -- 172.21.15.2:0/1486685188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff824074f60 msgr2=0x7ff82419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.093+0000 7ff828c00640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff824074f60 0x7ff82419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.093+0000 7ff828c00640 1 -- 172.21.15.2:0/1486685188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff824109cf0 msgr2=0x7ff8241a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.093+0000 7ff828c00640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff824109cf0 0x7ff8241a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.093+0000 7ff828c00640 1 -- 172.21.15.2:0/1486685188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8241a1bc0 con 0x7ff824074620 2024-09-10T08:09:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.093+0000 7ff829600640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff824109cf0 0x7ff8241a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.093+0000 7ff828c00640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff824074620 0x7ff824199fa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff818031440 tx=0x7ff818032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.094+0000 7ff821600640 1 -- 172.21.15.2:0/1486685188 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81803f030 con 0x7ff824074620 2024-09-10T08:09:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.094+0000 7ff821600640 1 -- 172.21.15.2:0/1486685188 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff818003ce0 con 0x7ff824074620 2024-09-10T08:09:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.094+0000 7ff82a000640 1 -- 172.21.15.2:0/1486685188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8241a1e50 con 0x7ff824074620 2024-09-10T08:09:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.094+0000 7ff821600640 1 -- 172.21.15.2:0/1486685188 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818035070 con 0x7ff824074620 2024-09-10T08:09:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.094+0000 7ff82a000640 1 -- 172.21.15.2:0/1486685188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8241a22b0 con 0x7ff824074620 2024-09-10T08:09:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.098+0000 7ff82e408740 1 Processor -- start 2024-09-10T08:09:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.098+0000 7ff82e408740 1 -- start start 2024-09-10T08:09:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.099+0000 7ff821600640 1 -- 172.21.15.2:0/1486685188 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff818042030 con 0x7ff824074620 2024-09-10T08:09:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.099+0000 7ff82e408740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089260 0x55ea2c089660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.099+0000 7ff82e408740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089ba0 0x55ea2c08e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.099+0000 7ff82e408740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c08ec30 0x55ea2c0910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.099+0000 7ff82e408740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ea2bfee100 con 0x55ea2c089ba0 2024-09-10T08:09:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.099+0000 7ff82e408740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ea2bf961b0 con 0x55ea2c089260 2024-09-10T08:09:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.099+0000 7ff82e408740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ea2bfea0c0 con 0x55ea2c08ec30 2024-09-10T08:09:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.099+0000 7ff823e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c08ec30 0x55ea2c0910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.099+0000 7ff829600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089260 0x55ea2c089660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.099+0000 7ff823e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c08ec30 0x55ea2c0910f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:44878/0 (socket says 172.21.15.2:44878) 2024-09-10T08:09:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.099+0000 7ff823e00640 1 -- 172.21.15.2:0/868817916 learned_addr learned my addr 172.21.15.2:0/868817916 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.099+0000 7ff828c00640 1 --2- 172.21.15.2:0/868817916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089ba0 0x55ea2c08e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.100+0000 7ff823e00640 1 -- 172.21.15.2:0/868817916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089260 msgr2=0x55ea2c089660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.100+0000 7ff823e00640 1 --2- 172.21.15.2:0/868817916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089260 0x55ea2c089660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.100+0000 7ff823e00640 1 -- 172.21.15.2:0/868817916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089ba0 msgr2=0x55ea2c08e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.100+0000 7ff823e00640 1 --2- 172.21.15.2:0/868817916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089ba0 0x55ea2c08e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.100+0000 7ff823e00640 1 -- 172.21.15.2:0/868817916 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea2bd01210 con 0x55ea2c08ec30 2024-09-10T08:09:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.100+0000 7ff828c00640 1 --2- 172.21.15.2:0/868817916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089ba0 0x55ea2c08e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.100+0000 7ff823e00640 1 --2- 172.21.15.2:0/868817916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c08ec30 0x55ea2c0910f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff810009c90 tx=0x7ff810011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98604c60d2ee1ee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.101+0000 7ff815c00640 1 -- 172.21.15.2:0/868817916 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81001e070 con 0x55ea2c08ec30 2024-09-10T08:09:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.101+0000 7ff815c00640 1 -- 172.21.15.2:0/868817916 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff810019b10 con 0x55ea2c08ec30 2024-09-10T08:09:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.101+0000 7ff815c00640 1 -- 172.21.15.2:0/868817916 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81000eac0 con 0x55ea2c08ec30 2024-09-10T08:09:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.102+0000 7ff82e408740 1 -- 172.21.15.2:0/868817916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c08ec30 msgr2=0x55ea2c0910f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.102+0000 7ff82e408740 1 --2- 172.21.15.2:0/868817916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c08ec30 0x55ea2c0910f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff810009c90 tx=0x7ff810011810 comp rx=0 tx=0).stop 2024-09-10T08:09:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.102+0000 7ff82e408740 1 -- 172.21.15.2:0/868817916 shutdown_connections 2024-09-10T08:09:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.102+0000 7ff82e408740 1 --2- 172.21.15.2:0/868817916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c08ec30 0x55ea2c0910f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.102+0000 7ff82e408740 1 --2- 172.21.15.2:0/868817916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089ba0 0x55ea2c08e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.102+0000 7ff82e408740 1 --2- 172.21.15.2:0/868817916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089260 0x55ea2c089660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.102+0000 7ff82e408740 1 -- 172.21.15.2:0/868817916 >> 172.21.15.2:0/868817916 conn(0x55ea2c0848d0 msgr2=0x55ea2c084cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.102+0000 7ff82e408740 1 -- 172.21.15.2:0/868817916 shutdown_connections 2024-09-10T08:09:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.102+0000 7ff82e408740 1 -- 172.21.15.2:0/868817916 wait complete. 2024-09-10T08:09:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.102+0000 7ff821600640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff80c0a2b10 0x7ff80c0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.102+0000 7ff821600640 1 -- 172.21.15.2:0/1486685188 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff8180ef820 con 0x7ff824074620 2024-09-10T08:09:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.102+0000 7ff823e00640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff80c0a2b10 0x7ff80c0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.102+0000 7ff82e408740 1 -- start start 2024-09-10T08:09:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.103+0000 7ff82e408740 1 Processor -- start 2024-09-10T08:09:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.103+0000 7ff823e00640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff80c0a2b10 0x7ff80c0a4fd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff810009a50 tx=0x7ff810000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.104+0000 7ff82e408740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089260 0x55ea2c0933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.104+0000 7ff82e408740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c089ba0 0x55ea2c0938e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.104+0000 7ff82e408740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c08ec30 0x55ea2c098910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.104+0000 7ff82e408740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ea2c091ba0 con 0x55ea2c089260 2024-09-10T08:09:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.104+0000 7ff82e408740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ea2bf77090 con 0x55ea2c08ec30 2024-09-10T08:09:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.104+0000 7ff82e408740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ea2c091d20 con 0x55ea2c089ba0 2024-09-10T08:09:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.104+0000 7ff82e408740 10 client.0 _ll_register_callbacks cb 0x55ea2bfcbf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.104+0000 7ff828c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089260 0x55ea2c0933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.104+0000 7ff828c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089260 0x55ea2c0933a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44638/0 (socket says 172.21.15.2:44638) 2024-09-10T08:09:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.104+0000 7ff828c00640 1 -- 172.21.15.2:0/29887573 learned_addr learned my addr 172.21.15.2:0/29887573 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.104+0000 7ff829600640 1 --2- 172.21.15.2:0/29887573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c08ec30 0x55ea2c098910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.105+0000 7ff828c00640 1 -- 172.21.15.2:0/29887573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c089ba0 msgr2=0x55ea2c0938e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.105+0000 7ff828c00640 1 --2- 172.21.15.2:0/29887573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c089ba0 0x55ea2c0938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.105+0000 7ff828c00640 1 -- 172.21.15.2:0/29887573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c08ec30 msgr2=0x55ea2c098910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.105+0000 7ff828c00640 1 --2- 172.21.15.2:0/29887573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c08ec30 0x55ea2c098910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.105+0000 7ff828c00640 1 -- 172.21.15.2:0/29887573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea2bdcd620 con 0x55ea2c089260 2024-09-10T08:09:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.105+0000 7ff829600640 1 --2- 172.21.15.2:0/29887573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c08ec30 0x55ea2c098910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.105+0000 7ff828c00640 1 --2- 172.21.15.2:0/29887573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089260 0x55ea2c0933a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff8180b8710 tx=0x7ff818009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47a9baee91e53e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.105+0000 7ff80a000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:09:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.105+0000 7ff80a000640 1 -- 172.21.15.2:0/29887573 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818049040 con 0x55ea2c089260 2024-09-10T08:09:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.105+0000 7ff80a000640 1 -- 172.21.15.2:0/29887573 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff81803bcd0 con 0x55ea2c089260 2024-09-10T08:09:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.105+0000 7ff82e408740 1 -- 172.21.15.2:0/29887573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff810019b10 con 0x55ea2c089260 2024-09-10T08:09:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.106+0000 7ff80a000640 1 -- 172.21.15.2:0/29887573 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff8180d9050 con 0x55ea2c089260 2024-09-10T08:09:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.106+0000 7ff82e408740 10 client.15585 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.106+0000 7ff82e408740 1 -- 172.21.15.2:0/29887573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ea2c09d830 con 0x55ea2c089260 2024-09-10T08:09:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.106+0000 7ff80a000640 1 -- 172.21.15.2:0/29887573 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff810019b10 con 0x55ea2c089260 2024-09-10T08:09:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.106+0000 7ff80a000640 1 -- 172.21.15.2:0/29887573 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff8180d4070 con 0x55ea2c089260 2024-09-10T08:09:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.106+0000 7ff82e408740 10 client.15585 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.106+0000 7ff82e408740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.106+0000 7ff82e408740 20 client.15585 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.106+0000 7ff82e408740 1 --2- 172.21.15.2:0/29887573 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea2c09eec0 0x55ea2c0a1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.106+0000 7ff82e408740 4 client.15585 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.106+0000 7ff82e408740 1 -- 172.21.15.2:0/29887573 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea2bffb830 con 0x55ea2c09eec0 2024-09-10T08:09:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.107+0000 7ff823e00640 1 --2- 172.21.15.2:0/29887573 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea2c09eec0 0x55ea2c0a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.107+0000 7ff80a000640 10 client.15585 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.107+0000 7ff823e00640 1 --2- 172.21.15.2:0/29887573 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea2c09eec0 0x55ea2c0a1360 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ed7a6e044d10a3 server_cookie=112c45359e36b53b in_seq=0 out_seq=0 2024-09-10T08:09:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.108+0000 7ff80a000640 1 -- 172.21.15.2:0/29887573 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea2bffb830 con 0x55ea2c09eec0 2024-09-10T08:09:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.108+0000 7ff80a000640 10 client.15585 handle_command_reply: tid=0 2024-09-10T08:09:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.108+0000 7ff82e408740 1 client.15585 shutdown 2024-09-10T08:09:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.109+0000 7ff82e408740 1 -- 172.21.15.2:0/29887573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089260 msgr2=0x55ea2c0933a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.109+0000 7ff82e408740 1 --2- 172.21.15.2:0/29887573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089260 0x55ea2c0933a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff8180b8710 tx=0x7ff818009620 comp rx=0 tx=0).stop 2024-09-10T08:09:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.109+0000 7ff82e408740 1 -- 172.21.15.2:0/29887573 shutdown_connections 2024-09-10T08:09:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.109+0000 7ff82e408740 1 --2- 172.21.15.2:0/29887573 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea2c09eec0 0x55ea2c0a1360 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.109+0000 7ff82e408740 1 --2- 172.21.15.2:0/29887573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c08ec30 0x55ea2c098910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.109+0000 7ff82e408740 1 --2- 172.21.15.2:0/29887573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c089ba0 0x55ea2c0938e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.109+0000 7ff82e408740 1 --2- 172.21.15.2:0/29887573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089260 0x55ea2c0933a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.109+0000 7ff82e408740 1 -- 172.21.15.2:0/29887573 >> 172.21.15.2:0/29887573 conn(0x55ea2c0848d0 msgr2=0x55ea2c0860e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.109+0000 7ff80a000640 0 client.15585 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.109+0000 7ff82e408740 1 -- 172.21.15.2:0/29887573 shutdown_connections 2024-09-10T08:09:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.109+0000 7ff82e408740 1 -- 172.21.15.2:0/29887573 wait complete. 2024-09-10T08:09:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.109+0000 7ff82e408740 20 client.15585 trim_cache size 0 max 16384 2024-09-10T08:09:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.139+0000 7ff82e408740 1 Processor -- start 2024-09-10T08:09:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.139+0000 7ff82e408740 1 -- start start 2024-09-10T08:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.140+0000 7ff82e408740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c0848d0 0x55ea2c085780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.140+0000 7ff82e408740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089260 0x55ea2c085cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.140+0000 7ff82e408740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089ba0 0x55ea2c086200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.140+0000 7ff82e408740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241aad50 con 0x55ea2c089ba0 2024-09-10T08:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.140+0000 7ff82e408740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241a8620 con 0x55ea2c089260 2024-09-10T08:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.140+0000 7ff82e408740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241a6470 con 0x55ea2c0848d0 2024-09-10T08:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.140+0000 7ff828c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c0848d0 0x55ea2c085780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.140+0000 7ff829600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089ba0 0x55ea2c086200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.140+0000 7ff828c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c0848d0 0x55ea2c085780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:44888/0 (socket says 172.21.15.2:44888) 2024-09-10T08:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.140+0000 7ff828c00640 1 -- 172.21.15.2:0/3586811706 learned_addr learned my addr 172.21.15.2:0/3586811706 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.140+0000 7ff823e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089260 0x55ea2c085cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.141+0000 7ff828c00640 1 -- 172.21.15.2:0/3586811706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089260 msgr2=0x55ea2c085cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.141+0000 7ff828c00640 1 --2- 172.21.15.2:0/3586811706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089260 0x55ea2c085cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.141+0000 7ff828c00640 1 -- 172.21.15.2:0/3586811706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089ba0 msgr2=0x55ea2c086200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.141+0000 7ff828c00640 1 --2- 172.21.15.2:0/3586811706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089ba0 0x55ea2c086200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.141+0000 7ff828c00640 1 -- 172.21.15.2:0/3586811706 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea2c09c000 con 0x55ea2c0848d0 2024-09-10T08:09:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.141+0000 7ff828c00640 1 --2- 172.21.15.2:0/3586811706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c0848d0 0x55ea2c085780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff8180bba40 tx=0x7ff818003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a168e20f540448a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.142+0000 7ff815200640 1 -- 172.21.15.2:0/3586811706 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81809f070 con 0x55ea2c0848d0 2024-09-10T08:09:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.142+0000 7ff815200640 1 -- 172.21.15.2:0/3586811706 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff818032e50 con 0x55ea2c0848d0 2024-09-10T08:09:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.142+0000 7ff815200640 1 -- 172.21.15.2:0/3586811706 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818049070 con 0x55ea2c0848d0 2024-09-10T08:09:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.143+0000 7ff82e408740 1 -- 172.21.15.2:0/3586811706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c0848d0 msgr2=0x55ea2c085780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.143+0000 7ff82e408740 1 --2- 172.21.15.2:0/3586811706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c0848d0 0x55ea2c085780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff8180bba40 tx=0x7ff818003eb0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.143+0000 7ff82e408740 1 -- 172.21.15.2:0/3586811706 shutdown_connections 2024-09-10T08:09:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.143+0000 7ff82e408740 1 --2- 172.21.15.2:0/3586811706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c089ba0 0x55ea2c086200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.143+0000 7ff82e408740 1 --2- 172.21.15.2:0/3586811706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089260 0x55ea2c085cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.143+0000 7ff82e408740 1 --2- 172.21.15.2:0/3586811706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c0848d0 0x55ea2c085780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.143+0000 7ff82e408740 1 -- 172.21.15.2:0/3586811706 >> 172.21.15.2:0/3586811706 conn(0x55ea2c07fa50 msgr2=0x55ea2c093360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.143+0000 7ff82e408740 1 -- 172.21.15.2:0/3586811706 shutdown_connections 2024-09-10T08:09:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.143+0000 7ff82e408740 1 -- 172.21.15.2:0/3586811706 wait complete. 2024-09-10T08:09:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.144+0000 7ff82e408740 1 -- start start 2024-09-10T08:09:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.144+0000 7ff82e408740 1 Processor -- start 2024-09-10T08:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.144+0000 7ff82e408740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c0848d0 0x55ea2c0b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.144+0000 7ff82e408740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c089260 0x55ea2c0b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.144+0000 7ff82e408740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089ba0 0x55ea2c0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.144+0000 7ff82e408740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241a6ae0 con 0x55ea2c0848d0 2024-09-10T08:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.145+0000 7ff82e408740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241a7f50 con 0x55ea2c089ba0 2024-09-10T08:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.145+0000 7ff82e408740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241a71b0 con 0x55ea2c089260 2024-09-10T08:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.145+0000 7ff82e408740 10 client.0 _ll_register_callbacks cb 0x7ff8180057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.145+0000 7ff828c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c0848d0 0x55ea2c0b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.145+0000 7ff828c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c0848d0 0x55ea2c0b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44654/0 (socket says 172.21.15.2:44654) 2024-09-10T08:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.145+0000 7ff828c00640 1 -- 172.21.15.2:0/3169276866 learned_addr learned my addr 172.21.15.2:0/3169276866 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.145+0000 7ff829600640 1 --2- 172.21.15.2:0/3169276866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089ba0 0x55ea2c0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.145+0000 7ff828c00640 1 -- 172.21.15.2:0/3169276866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c089260 msgr2=0x55ea2c0b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.145+0000 7ff828c00640 1 --2- 172.21.15.2:0/3169276866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c089260 0x55ea2c0b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.145+0000 7ff828c00640 1 -- 172.21.15.2:0/3169276866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089ba0 msgr2=0x55ea2c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.145+0000 7ff828c00640 1 --2- 172.21.15.2:0/3169276866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089ba0 0x55ea2c0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.145+0000 7ff828c00640 1 -- 172.21.15.2:0/3169276866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea2c0b29f0 con 0x55ea2c0848d0 2024-09-10T08:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.145+0000 7ff829600640 1 --2- 172.21.15.2:0/3169276866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089ba0 0x55ea2c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.146+0000 7ff828c00640 1 --2- 172.21.15.2:0/3169276866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c0848d0 0x55ea2c0b0f30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff818005ec0 tx=0x7ff818009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a60a8b4d66d5ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.146+0000 7ff80a000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.146+0000 7ff80a000640 1 -- 172.21.15.2:0/3169276866 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81809f070 con 0x55ea2c0848d0 2024-09-10T08:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.146+0000 7ff80a000640 1 -- 172.21.15.2:0/3169276866 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff8180c4670 con 0x55ea2c0848d0 2024-09-10T08:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.146+0000 7ff82e408740 1 -- 172.21.15.2:0/3169276866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff818032e50 con 0x55ea2c0848d0 2024-09-10T08:09:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.146+0000 7ff80a000640 1 -- 172.21.15.2:0/3169276866 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff8180cb050 con 0x55ea2c0848d0 2024-09-10T08:09:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.146+0000 7ff82e408740 10 client.15591 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.146+0000 7ff82e408740 1 -- 172.21.15.2:0/3169276866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ea2c109fb0 con 0x55ea2c0848d0 2024-09-10T08:09:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.147+0000 7ff80a000640 1 -- 172.21.15.2:0/3169276866 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8180bbd40 con 0x55ea2c0848d0 2024-09-10T08:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.147+0000 7ff80a000640 1 -- 172.21.15.2:0/3169276866 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff8180d4040 con 0x55ea2c0848d0 2024-09-10T08:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.147+0000 7ff82e408740 10 client.15591 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.147+0000 7ff82e408740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.147+0000 7ff82e408740 20 client.15591 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.147+0000 7ff82e408740 1 --2- 172.21.15.2:0/3169276866 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea2c08ec30 0x55ea2c10c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.147+0000 7ff82e408740 4 client.15591 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.147+0000 7ff82e408740 1 -- 172.21.15.2:0/3169276866 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff818032e50 con 0x55ea2c08ec30 2024-09-10T08:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.148+0000 7ff823e00640 1 --2- 172.21.15.2:0/3169276866 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea2c08ec30 0x55ea2c10c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.148+0000 7ff823e00640 1 --2- 172.21.15.2:0/3169276866 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea2c08ec30 0x55ea2c10c730 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1bcb1e0cf9c344 server_cookie=d39acb2360b54c4e in_seq=0 out_seq=0 2024-09-10T08:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.148+0000 7ff80a000640 10 client.15591 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.152+0000 7ff80a000640 1 -- 172.21.15.2:0/3169276866 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff818032e50 con 0x55ea2c08ec30 2024-09-10T08:09:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.152+0000 7ff80a000640 10 client.15591 handle_command_reply: tid=0 2024-09-10T08:09:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.152+0000 7ff82e408740 1 client.15591 shutdown 2024-09-10T08:09:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.152+0000 7ff82e408740 1 -- 172.21.15.2:0/3169276866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c0848d0 msgr2=0x55ea2c0b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.152+0000 7ff82e408740 1 --2- 172.21.15.2:0/3169276866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c0848d0 0x55ea2c0b0f30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff818005ec0 tx=0x7ff818009260 comp rx=0 tx=0).stop 2024-09-10T08:09:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.152+0000 7ff82e408740 1 -- 172.21.15.2:0/3169276866 shutdown_connections 2024-09-10T08:09:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.152+0000 7ff82e408740 1 --2- 172.21.15.2:0/3169276866 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea2c08ec30 0x55ea2c10c730 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.152+0000 7ff82e408740 1 --2- 172.21.15.2:0/3169276866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea2c089ba0 0x55ea2c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.152+0000 7ff82e408740 1 --2- 172.21.15.2:0/3169276866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea2c089260 0x55ea2c0b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.152+0000 7ff80a000640 0 client.15591 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.152+0000 7ff82e408740 1 --2- 172.21.15.2:0/3169276866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea2c0848d0 0x55ea2c0b0f30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.152+0000 7ff82e408740 1 -- 172.21.15.2:0/3169276866 >> 172.21.15.2:0/3169276866 conn(0x55ea2c07fa50 msgr2=0x55ea2c0831f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.153+0000 7ff82e408740 1 -- 172.21.15.2:0/3169276866 shutdown_connections 2024-09-10T08:09:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.153+0000 7ff82e408740 1 -- 172.21.15.2:0/3169276866 wait complete. 2024-09-10T08:09:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.153+0000 7ff82e408740 20 client.15591 trim_cache size 0 max 16384 2024-09-10T08:09:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:09:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:09:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:09:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:09:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.154+0000 7ff82a000640 1 -- 172.21.15.2:0/1486685188 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff80c0a2b10 msgr2=0x7ff80c0a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.154+0000 7ff82a000640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff80c0a2b10 0x7ff80c0a4fd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff810009a50 tx=0x7ff810000bc0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.154+0000 7ff82a000640 1 -- 172.21.15.2:0/1486685188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff824074620 msgr2=0x7ff824199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.154+0000 7ff82a000640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff824074620 0x7ff824199fa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff818031440 tx=0x7ff818032ce0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.154+0000 7ff82a000640 1 -- 172.21.15.2:0/1486685188 shutdown_connections 2024-09-10T08:09:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.154+0000 7ff82a000640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff824109cf0 0x7ff8241a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.154+0000 7ff82a000640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff824074f60 0x7ff82419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.154+0000 7ff82a000640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff80c0a2b10 0x7ff80c0a4fd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.154+0000 7ff82a000640 1 --2- 172.21.15.2:0/1486685188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff824074620 0x7ff824199fa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.154+0000 7ff82a000640 1 -- 172.21.15.2:0/1486685188 >> 172.21.15.2:0/1486685188 conn(0x7ff8240fd0d0 msgr2=0x7ff824105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.154+0000 7ff82a000640 1 -- 172.21.15.2:0/1486685188 shutdown_connections 2024-09-10T08:09:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:09:49.155+0000 7ff82a000640 1 -- 172.21.15.2:0/1486685188 wait complete. 2024-09-10T08:09:49.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:49 smithi155 ceph-mon[35405]: pgmap v437: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-10T08:09:49.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:49 smithi107 ceph-mon[35048]: pgmap v437: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-10T08:09:50.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:49 smithi002 ceph-mon[30563]: pgmap v437: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-10T08:09:51.727 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:09:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:51 smithi155 ceph-mon[35405]: pgmap v438: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:09:51.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:51 smithi107 ceph-mon[35048]: pgmap v438: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:09:52.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:51 smithi002 ceph-mon[30563]: pgmap v438: 129 pgs: 129 active+clean; 674 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:09:52.124 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:53.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.047+0000 7f1800e00640 1 -- 172.21.15.2:0/1135460370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17fc1047e0 msgr2=0x7f17fc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:53.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.047+0000 7f1800e00640 1 --2- 172.21.15.2:0/1135460370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17fc1047e0 0x7f17fc108830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009860 tx=0x7f17e802a340 comp rx=0 tx=0).stop 2024-09-10T08:09:53.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.047+0000 7f1800e00640 1 -- 172.21.15.2:0/1135460370 shutdown_connections 2024-09-10T08:09:53.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.047+0000 7f1800e00640 1 --2- 172.21.15.2:0/1135460370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17fc108f60 0x7f17fc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.047+0000 7f1800e00640 1 --2- 172.21.15.2:0/1135460370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17fc1047e0 0x7f17fc108830 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.047+0000 7f1800e00640 1 --2- 172.21.15.2:0/1135460370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17fc103e30 0x7f17fc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.047+0000 7f1800e00640 1 -- 172.21.15.2:0/1135460370 >> 172.21.15.2:0/1135460370 conn(0x7f17fc0ff910 msgr2=0x7f17fc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:53.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.047+0000 7f1800e00640 1 -- 172.21.15.2:0/1135460370 shutdown_connections 2024-09-10T08:09:53.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.047+0000 7f1800e00640 1 -- 172.21.15.2:0/1135460370 wait complete. 2024-09-10T08:09:53.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.048+0000 7f1800e00640 1 Processor -- start 2024-09-10T08:09:53.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.048+0000 7f1800e00640 1 -- start start 2024-09-10T08:09:53.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.048+0000 7f1800e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17fc103e30 0x7f17fc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.048+0000 7f1800e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17fc1047e0 0x7f17fc19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.048+0000 7f1800e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17fc108f60 0x7f17fc19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.048+0000 7f1800e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc113fc0 con 0x7f17fc1047e0 2024-09-10T08:09:53.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.048+0000 7f1800e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc113e40 con 0x7f17fc103e30 2024-09-10T08:09:53.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.048+0000 7f1800e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc114140 con 0x7f17fc108f60 2024-09-10T08:09:53.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.049+0000 7f17faa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17fc1047e0 0x7f17fc19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.049+0000 7f17faa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17fc1047e0 0x7f17fc19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44670/0 (socket says 172.21.15.2:44670) 2024-09-10T08:09:53.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.049+0000 7f17fbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17fc108f60 0x7f17fc19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.049+0000 7f17faa00640 1 -- 172.21.15.2:0/2533836815 learned_addr learned my addr 172.21.15.2:0/2533836815 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:53.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.049+0000 7f17fb400640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17fc103e30 0x7f17fc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.049+0000 7f17fbe00640 1 -- 172.21.15.2:0/2533836815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17fc103e30 msgr2=0x7f17fc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:53.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.049+0000 7f17fbe00640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17fc103e30 0x7f17fc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.049+0000 7f17fbe00640 1 -- 172.21.15.2:0/2533836815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17fc1047e0 msgr2=0x7f17fc19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:53.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.049+0000 7f17fbe00640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17fc1047e0 0x7f17fc19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.049+0000 7f17fbe00640 1 -- 172.21.15.2:0/2533836815 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc19f4a0 con 0x7f17fc108f60 2024-09-10T08:09:53.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.049+0000 7f17faa00640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17fc1047e0 0x7f17fc19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:53.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.049+0000 7f17fbe00640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17fc108f60 0x7f17fc19edc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f17f400efc0 tx=0x7f17f400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:53.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.050+0000 7f17f3e00640 1 -- 172.21.15.2:0/2533836815 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4003970 con 0x7f17fc108f60 2024-09-10T08:09:53.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.050+0000 7f17f3e00640 1 -- 172.21.15.2:0/2533836815 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f4004020 con 0x7f17fc108f60 2024-09-10T08:09:53.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.050+0000 7f1800e00640 1 -- 172.21.15.2:0/2533836815 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc19f790 con 0x7f17fc108f60 2024-09-10T08:09:53.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.050+0000 7f17f3e00640 1 -- 172.21.15.2:0/2533836815 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f400b9f0 con 0x7f17fc108f60 2024-09-10T08:09:53.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.050+0000 7f1800e00640 1 -- 172.21.15.2:0/2533836815 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc1a7070 con 0x7f17fc108f60 2024-09-10T08:09:53.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.053+0000 7f17f3e00640 1 -- 172.21.15.2:0/2533836815 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f17f401d020 con 0x7f17fc108f60 2024-09-10T08:09:53.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.054+0000 7f1805028740 1 Processor -- start 2024-09-10T08:09:53.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.054+0000 7f1805028740 1 -- start start 2024-09-10T08:09:53.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.054+0000 7f1805028740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 0x564539931130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.054+0000 7f1805028740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x56453993b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.054+0000 7f1805028740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 0x56453993e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.054+0000 7f1805028740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5645398973e0 con 0x564539931670 2024-09-10T08:09:53.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.054+0000 7f1805028740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56453983ed80 con 0x564539930d50 2024-09-10T08:09:53.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.054+0000 7f1805028740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5645398933a0 con 0x56453993bf30 2024-09-10T08:09:53.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.054+0000 7f17fb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x56453993b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.055+0000 7f17fb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x56453993b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44686/0 (socket says 172.21.15.2:44686) 2024-09-10T08:09:53.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.055+0000 7f17fb400640 1 -- 172.21.15.2:0/1386460350 learned_addr learned my addr 172.21.15.2:0/1386460350 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:53.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.055+0000 7f17faa00640 1 --2- 172.21.15.2:0/1386460350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 0x564539931130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.055+0000 7f17faa00640 1 --2- 172.21.15.2:0/1386460350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 0x56453993e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.055+0000 7f17f3e00640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f17e00a2c00 0x7f17e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.055+0000 7f17f3e00640 1 -- 172.21.15.2:0/2533836815 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f17f40cf2c0 con 0x7f17fc108f60 2024-09-10T08:09:53.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.055+0000 7f17fb400640 1 -- 172.21.15.2:0/1386460350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 msgr2=0x56453993e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:53.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.055+0000 7f17fb400640 1 --2- 172.21.15.2:0/1386460350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 0x56453993e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.055+0000 7f17fb400640 1 -- 172.21.15.2:0/1386460350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 msgr2=0x564539931130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:53.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.055+0000 7f17fb400640 1 --2- 172.21.15.2:0/1386460350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 0x564539931130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.055+0000 7f17fb400640 1 -- 172.21.15.2:0/1386460350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56453985dae0 con 0x564539931670 2024-09-10T08:09:53.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.055+0000 7f17fbe00640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f17e00a2c00 0x7f17e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.056+0000 7f17fb400640 1 --2- 172.21.15.2:0/1386460350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x56453993b9f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009c90 tx=0x7f17ec0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd58ccffac9866e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:53.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.056+0000 7f17f0c00640 1 -- 172.21.15.2:0/1386460350 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec012450 con 0x564539931670 2024-09-10T08:09:53.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.056+0000 7f17f0c00640 1 -- 172.21.15.2:0/1386460350 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ec012ad0 con 0x564539931670 2024-09-10T08:09:53.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.056+0000 7f17f0c00640 1 -- 172.21.15.2:0/1386460350 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00fbf0 con 0x564539931670 2024-09-10T08:09:53.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.056+0000 7f17fbe00640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f17e00a2c00 0x7f17e00a50c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f17f409b3d0 tx=0x7f17f400bb90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:53.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.057+0000 7f1805028740 1 -- 172.21.15.2:0/1386460350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 msgr2=0x56453993b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:53.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.057+0000 7f1805028740 1 --2- 172.21.15.2:0/1386460350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x56453993b9f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009c90 tx=0x7f17ec0117b0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.057+0000 7f1805028740 1 -- 172.21.15.2:0/1386460350 shutdown_connections 2024-09-10T08:09:53.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.057+0000 7f1805028740 1 --2- 172.21.15.2:0/1386460350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 0x56453993e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.057+0000 7f1805028740 1 --2- 172.21.15.2:0/1386460350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x56453993b9f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.057+0000 7f1805028740 1 --2- 172.21.15.2:0/1386460350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 0x564539931130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.057+0000 7f1805028740 1 -- 172.21.15.2:0/1386460350 >> 172.21.15.2:0/1386460350 conn(0x56453992d2e0 msgr2=0x56453992d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:53.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.057+0000 7f1805028740 1 -- 172.21.15.2:0/1386460350 shutdown_connections 2024-09-10T08:09:53.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.057+0000 7f1805028740 1 -- 172.21.15.2:0/1386460350 wait complete. 2024-09-10T08:09:53.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.058+0000 7f1805028740 1 -- start start 2024-09-10T08:09:53.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.058+0000 7f1805028740 1 Processor -- start 2024-09-10T08:09:53.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.058+0000 7f1805028740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539930d50 0x564539938df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.058+0000 7f1805028740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564539931670 0x564539939330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.058+0000 7f1805028740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56453993bf30 0x564539939870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f1805028740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56453993eba0 con 0x564539930d50 2024-09-10T08:09:53.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f1805028740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5645398520f0 con 0x56453993bf30 2024-09-10T08:09:53.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f1805028740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56453993ed20 con 0x564539931670 2024-09-10T08:09:53.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f1805028740 10 client.0 _ll_register_callbacks cb 0x5645398439e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:53.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f17faa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539930d50 0x564539938df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f17fb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564539931670 0x564539939330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f17faa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539930d50 0x564539938df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44698/0 (socket says 172.21.15.2:44698) 2024-09-10T08:09:53.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f17faa00640 1 -- 172.21.15.2:0/3590465188 learned_addr learned my addr 172.21.15.2:0/3590465188 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:53.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f17fb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564539931670 0x564539939330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:44932/0 (socket says 172.21.15.2:44932) 2024-09-10T08:09:53.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f17faa00640 1 --2- 172.21.15.2:0/3590465188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56453993bf30 0x564539939870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f17faa00640 1 -- 172.21.15.2:0/3590465188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564539931670 msgr2=0x564539939330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:53.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f17faa00640 1 --2- 172.21.15.2:0/3590465188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564539931670 0x564539939330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f17faa00640 1 -- 172.21.15.2:0/3590465188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56453993bf30 msgr2=0x564539939870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:53.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f17faa00640 1 --2- 172.21.15.2:0/3590465188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56453993bf30 0x564539939870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f17faa00640 1 -- 172.21.15.2:0/3590465188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645396083c0 con 0x564539930d50 2024-09-10T08:09:53.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.059+0000 7f17fb400640 1 --2- 172.21.15.2:0/3590465188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564539931670 0x564539939330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:09:53.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.060+0000 7f17faa00640 1 --2- 172.21.15.2:0/3590465188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539930d50 0x564539938df0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17e802c010 tx=0x7f17e802a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc8f3e306e70b2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:53.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.060+0000 7f17e5600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:09:53.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.060+0000 7f17e5600640 1 -- 172.21.15.2:0/3590465188 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8037040 con 0x564539930d50 2024-09-10T08:09:53.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.060+0000 7f17e5600640 1 -- 172.21.15.2:0/3590465188 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f17e8003a00 con 0x564539930d50 2024-09-10T08:09:53.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.060+0000 7f1805028740 1 -- 172.21.15.2:0/3590465188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17ec012ad0 con 0x564539930d50 2024-09-10T08:09:53.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.060+0000 7f17e5600640 1 -- 172.21.15.2:0/3590465188 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17e8051050 con 0x564539930d50 2024-09-10T08:09:53.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.060+0000 7f1805028740 10 client.15603 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:53.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.060+0000 7f1805028740 1 -- 172.21.15.2:0/3590465188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56453994a850 con 0x564539930d50 2024-09-10T08:09:53.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.061+0000 7f17e5600640 1 -- 172.21.15.2:0/3590465188 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ec012ad0 con 0x564539930d50 2024-09-10T08:09:53.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.061+0000 7f17e5600640 1 -- 172.21.15.2:0/3590465188 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f17e803d040 con 0x564539930d50 2024-09-10T08:09:53.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.061+0000 7f1805028740 10 client.15603 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:53.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.061+0000 7f1805028740 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:53.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.061+0000 7f1805028740 20 client.15603 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:53.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.061+0000 7f1805028740 1 --2- 172.21.15.2:0/3590465188 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56453994bd10 0x56453994e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.061+0000 7f1805028740 4 client.15603 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:09:53.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.061+0000 7f1805028740 1 -- 172.21.15.2:0/3590465188 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645398a4950 con 0x56453994bd10 2024-09-10T08:09:53.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.062+0000 7f17fb400640 1 --2- 172.21.15.2:0/3590465188 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56453994bd10 0x56453994e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.062+0000 7f17fb400640 1 --2- 172.21.15.2:0/3590465188 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56453994bd10 0x56453994e100 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6841f04b6107cb83 server_cookie=f1520a97bd26a19c in_seq=0 out_seq=0 2024-09-10T08:09:53.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.062+0000 7f17e5600640 10 client.15603 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:53.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.063+0000 7f17e5600640 1 -- 172.21.15.2:0/3590465188 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645398a4950 con 0x56453994bd10 2024-09-10T08:09:53.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.063+0000 7f17e5600640 10 client.15603 handle_command_reply: tid=0 2024-09-10T08:09:53.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.063+0000 7f1805028740 1 client.15603 shutdown 2024-09-10T08:09:53.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.063+0000 7f1805028740 1 -- 172.21.15.2:0/3590465188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539930d50 msgr2=0x564539938df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:53.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.063+0000 7f1805028740 1 --2- 172.21.15.2:0/3590465188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539930d50 0x564539938df0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17e802c010 tx=0x7f17e802a340 comp rx=0 tx=0).stop 2024-09-10T08:09:53.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.063+0000 7f1805028740 1 -- 172.21.15.2:0/3590465188 shutdown_connections 2024-09-10T08:09:53.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.063+0000 7f1805028740 1 --2- 172.21.15.2:0/3590465188 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56453994bd10 0x56453994e100 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.064+0000 7f1805028740 1 --2- 172.21.15.2:0/3590465188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56453993bf30 0x564539939870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.064+0000 7f1805028740 1 --2- 172.21.15.2:0/3590465188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564539931670 0x564539939330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.064+0000 7f1805028740 1 --2- 172.21.15.2:0/3590465188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539930d50 0x564539938df0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.064+0000 7f1805028740 1 -- 172.21.15.2:0/3590465188 >> 172.21.15.2:0/3590465188 conn(0x56453992d2e0 msgr2=0x564539933010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:53.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.064+0000 7f17e5600640 0 client.15603 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:53.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.064+0000 7f1805028740 1 -- 172.21.15.2:0/3590465188 shutdown_connections 2024-09-10T08:09:53.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.064+0000 7f1805028740 1 -- 172.21.15.2:0/3590465188 wait complete. 2024-09-10T08:09:53.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.064+0000 7f1805028740 20 client.15603 trim_cache size 0 max 16384 2024-09-10T08:09:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.082+0000 7f1805028740 1 Processor -- start 2024-09-10T08:09:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.083+0000 7f1805028740 1 -- start start 2024-09-10T08:09:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.083+0000 7f1805028740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 0x5645399337b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.083+0000 7f1805028740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x56453993ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.083+0000 7f1805028740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 0x5645399539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.083+0000 7f1805028740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1b36d0 con 0x564539931670 2024-09-10T08:09:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.083+0000 7f1805028740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1b0fa0 con 0x564539930d50 2024-09-10T08:09:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.083+0000 7f1805028740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1aedf0 con 0x56453993bf30 2024-09-10T08:09:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.083+0000 7f17fb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x56453993ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.083+0000 7f17fb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x56453993ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44704/0 (socket says 172.21.15.2:44704) 2024-09-10T08:09:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.083+0000 7f17fb400640 1 -- 172.21.15.2:0/303926246 learned_addr learned my addr 172.21.15.2:0/303926246 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.083+0000 7f17faa00640 1 --2- 172.21.15.2:0/303926246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 0x5645399337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.083+0000 7f17faa00640 1 --2- 172.21.15.2:0/303926246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 0x5645399539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.084+0000 7f17faa00640 1 -- 172.21.15.2:0/303926246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 msgr2=0x5645399539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.084+0000 7f17faa00640 1 --2- 172.21.15.2:0/303926246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 0x5645399539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.084+0000 7f17faa00640 1 -- 172.21.15.2:0/303926246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 msgr2=0x56453993ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.084+0000 7f17faa00640 1 --2- 172.21.15.2:0/303926246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x56453993ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.084+0000 7f17faa00640 1 -- 172.21.15.2:0/303926246 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564539937360 con 0x564539930d50 2024-09-10T08:09:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.084+0000 7f17faa00640 1 --2- 172.21.15.2:0/303926246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 0x5645399337b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f17e8035040 tx=0x7f17e8009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdb382ad31c66506 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.084+0000 7f17e7e00640 1 -- 172.21.15.2:0/303926246 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8037040 con 0x564539930d50 2024-09-10T08:09:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.084+0000 7f17e7e00640 1 -- 172.21.15.2:0/303926246 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e80044c0 con 0x564539930d50 2024-09-10T08:09:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.084+0000 7f17e7e00640 1 -- 172.21.15.2:0/303926246 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80046c0 con 0x564539930d50 2024-09-10T08:09:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.085+0000 7f1805028740 1 -- 172.21.15.2:0/303926246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 msgr2=0x5645399337b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.085+0000 7f1805028740 1 --2- 172.21.15.2:0/303926246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 0x5645399337b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f17e8035040 tx=0x7f17e8009570 comp rx=0 tx=0).stop 2024-09-10T08:09:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.085+0000 7f1805028740 1 -- 172.21.15.2:0/303926246 shutdown_connections 2024-09-10T08:09:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.085+0000 7f1805028740 1 --2- 172.21.15.2:0/303926246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 0x5645399539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.085+0000 7f1805028740 1 --2- 172.21.15.2:0/303926246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x56453993ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.085+0000 7f1805028740 1 --2- 172.21.15.2:0/303926246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 0x5645399337b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.085+0000 7f1805028740 1 -- 172.21.15.2:0/303926246 >> 172.21.15.2:0/303926246 conn(0x56453992d2e0 msgr2=0x56453993a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:53.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.085+0000 7f1805028740 1 -- 172.21.15.2:0/303926246 shutdown_connections 2024-09-10T08:09:53.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.085+0000 7f1805028740 1 -- 172.21.15.2:0/303926246 wait complete. 2024-09-10T08:09:53.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.086+0000 7f1805028740 1 -- start start 2024-09-10T08:09:53.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.086+0000 7f1805028740 1 Processor -- start 2024-09-10T08:09:53.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.086+0000 7f1805028740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 0x564539944520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.086+0000 7f1805028740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x564539935a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f1805028740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 0x564539935fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f1805028740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1af460 con 0x564539931670 2024-09-10T08:09:53.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f1805028740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1b08d0 con 0x564539930d50 2024-09-10T08:09:53.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f1805028740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1afb30 con 0x56453993bf30 2024-09-10T08:09:53.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f1805028740 10 client.0 _ll_register_callbacks cb 0x7f17e8030a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:09:53.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f17fb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x564539935a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f17fb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x564539935a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44720/0 (socket says 172.21.15.2:44720) 2024-09-10T08:09:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f17fb400640 1 -- 172.21.15.2:0/3170338082 learned_addr learned my addr 172.21.15.2:0/3170338082 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f17faa00640 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 0x564539944520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f17faa00640 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 0x564539935fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f17fb400640 1 -- 172.21.15.2:0/3170338082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 msgr2=0x564539935fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f17fb400640 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 0x564539935fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f17fb400640 1 -- 172.21.15.2:0/3170338082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 msgr2=0x564539944520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:09:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f17fb400640 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 0x564539944520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f17fb400640 1 -- 172.21.15.2:0/3170338082 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645399366b0 con 0x564539931670 2024-09-10T08:09:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.087+0000 7f17faa00640 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 0x564539935fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:09:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.088+0000 7f17fb400640 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x564539935a80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f17ec01c040 tx=0x7f17ec009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ed50ac46e1ef82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.088+0000 7f17e5600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:09:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.088+0000 7f17e5600640 1 -- 172.21.15.2:0/3170338082 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00fa10 con 0x564539931670 2024-09-10T08:09:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.088+0000 7f17e5600640 1 -- 172.21.15.2:0/3170338082 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f17ec00fbb0 con 0x564539931670 2024-09-10T08:09:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.088+0000 7f1805028740 1 -- 172.21.15.2:0/3170338082 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56453989ee60 con 0x564539931670 2024-09-10T08:09:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.088+0000 7f17e5600640 1 -- 172.21.15.2:0/3170338082 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17ec00fdb0 con 0x564539931670 2024-09-10T08:09:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.088+0000 7f1805028740 10 client.15609 fetch_fsmap learned FSMap version 26 2024-09-10T08:09:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.088+0000 7f1805028740 1 -- 172.21.15.2:0/3170338082 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5645399b6890 con 0x564539931670 2024-09-10T08:09:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.089+0000 7f17e5600640 1 -- 172.21.15.2:0/3170338082 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56453989ee60 con 0x564539931670 2024-09-10T08:09:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.089+0000 7f17e5600640 1 -- 172.21.15.2:0/3170338082 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f17ec0054f0 con 0x564539931670 2024-09-10T08:09:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.089+0000 7f1805028740 10 client.15609 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:09:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.089+0000 7f1805028740 10 client.15609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:09:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.089+0000 7f1805028740 20 client.15609 populate_metadata read hostname 'smithi002' 2024-09-10T08:09:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.089+0000 7f1805028740 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5645399b6e50 0x5645399b9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.089+0000 7f1805028740 4 client.15609 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:09:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.089+0000 7f1805028740 1 -- 172.21.15.2:0/3170338082 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56453981b0c0 con 0x5645399b6e50 2024-09-10T08:09:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.090+0000 7f17faa00640 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5645399b6e50 0x5645399b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.090+0000 7f17faa00640 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5645399b6e50 0x5645399b9270 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c00d8b5b723d2eca server_cookie=8a9b229ef391b8ae in_seq=0 out_seq=0 2024-09-10T08:09:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.090+0000 7f17e5600640 10 client.15609 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.090+0000 7f17e5600640 1 -- 172.21.15.2:0/3170338082 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56453981b0c0 con 0x5645399b6e50 2024-09-10T08:09:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.090+0000 7f17e5600640 10 client.15609 handle_command_reply: tid=0 2024-09-10T08:09:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.090+0000 7f1805028740 1 client.15609 shutdown 2024-09-10T08:09:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.090+0000 7f1805028740 1 -- 172.21.15.2:0/3170338082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 msgr2=0x564539935a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.090+0000 7f1805028740 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x564539935a80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f17ec01c040 tx=0x7f17ec009790 comp rx=0 tx=0).stop 2024-09-10T08:09:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.091+0000 7f1805028740 1 -- 172.21.15.2:0/3170338082 shutdown_connections 2024-09-10T08:09:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.091+0000 7f1805028740 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5645399b6e50 0x5645399b9270 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.091+0000 7f1805028740 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56453993bf30 0x564539935fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.091+0000 7f1805028740 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564539931670 0x564539935a80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.091+0000 7f1805028740 1 --2- 172.21.15.2:0/3170338082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564539930d50 0x564539944520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.091+0000 7f1805028740 1 -- 172.21.15.2:0/3170338082 >> 172.21.15.2:0/3170338082 conn(0x56453992d2e0 msgr2=0x5645399395d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.091+0000 7f17e5600640 0 client.15609 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:09:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.091+0000 7f1805028740 1 -- 172.21.15.2:0/3170338082 shutdown_connections 2024-09-10T08:09:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.091+0000 7f1805028740 1 -- 172.21.15.2:0/3170338082 wait complete. 2024-09-10T08:09:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.091+0000 7f1805028740 20 client.15609 trim_cache size 0 max 16384 2024-09-10T08:09:53.094 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:09:53.094 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:09:53.094 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:09:53.094 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:09:53.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.092+0000 7f1800e00640 1 -- 172.21.15.2:0/2533836815 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f17e00a2c00 msgr2=0x7f17e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:53.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.092+0000 7f1800e00640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f17e00a2c00 0x7f17e00a50c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f17f409b3d0 tx=0x7f17f400bb90 comp rx=0 tx=0).stop 2024-09-10T08:09:53.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.092+0000 7f1800e00640 1 -- 172.21.15.2:0/2533836815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17fc108f60 msgr2=0x7f17fc19edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:53.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.092+0000 7f1800e00640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17fc108f60 0x7f17fc19edc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f17f400efc0 tx=0x7f17f400c800 comp rx=0 tx=0).stop 2024-09-10T08:09:53.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.092+0000 7f1800e00640 1 -- 172.21.15.2:0/2533836815 shutdown_connections 2024-09-10T08:09:53.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.092+0000 7f1800e00640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f17e00a2c00 0x7f17e00a50c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.092+0000 7f1800e00640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17fc108f60 0x7f17fc19edc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.092+0000 7f1800e00640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17fc1047e0 0x7f17fc19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.092+0000 7f1800e00640 1 --2- 172.21.15.2:0/2533836815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17fc103e30 0x7f17fc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:53.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.092+0000 7f1800e00640 1 -- 172.21.15.2:0/2533836815 >> 172.21.15.2:0/2533836815 conn(0x7f17fc0ff910 msgr2=0x7f17fc100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:53.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.092+0000 7f1800e00640 1 -- 172.21.15.2:0/2533836815 shutdown_connections 2024-09-10T08:09:53.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:53.092+0000 7f1800e00640 1 -- 172.21.15.2:0/2533836815 wait complete. 2024-09-10T08:09:53.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:53 smithi155 ceph-mon[35405]: pgmap v439: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T08:09:53.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:53 smithi107 ceph-mon[35048]: pgmap v439: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T08:09:54.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:53 smithi002 ceph-mon[30563]: pgmap v439: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T08:09:55.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:55 smithi155 ceph-mon[35405]: pgmap v440: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:09:55.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:55 smithi107 ceph-mon[35048]: pgmap v440: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:09:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:55 smithi002 ceph-mon[30563]: pgmap v440: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:09:57.342 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:09:57.342 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:09:57.737 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:57 smithi155 ceph-mon[35405]: pgmap v441: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:09:57.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:57 smithi107 ceph-mon[35048]: pgmap v441: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:09:57.988 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:57 smithi002 ceph-mon[30563]: pgmap v441: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:09:58.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.427+0000 7fabe6000640 1 -- 172.21.15.2:0/1993268256 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd00319b0 con 0x7fabe8074510 2024-09-10T08:09:58.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.428+0000 7fabed200640 1 -- 172.21.15.2:0/1993268256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabe8074510 msgr2=0x7fabe80748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:58.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.428+0000 7fabed200640 1 --2- 172.21.15.2:0/1993268256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabe8074510 0x7fabe80748f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009900 tx=0x7fabd002a480 comp rx=0 tx=0).stop 2024-09-10T08:09:58.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.428+0000 7fabed200640 1 -- 172.21.15.2:0/1993268256 shutdown_connections 2024-09-10T08:09:58.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.428+0000 7fabed200640 1 --2- 172.21.15.2:0/1993268256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe810e280 0x7fabe8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:58.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.428+0000 7fabed200640 1 --2- 172.21.15.2:0/1993268256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabe8074e30 0x7fabe810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:58.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.428+0000 7fabed200640 1 --2- 172.21.15.2:0/1993268256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabe8074510 0x7fabe80748f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:58.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.428+0000 7fabed200640 1 -- 172.21.15.2:0/1993268256 >> 172.21.15.2:0/1993268256 conn(0x7fabe80fd740 msgr2=0x7fabe80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:58.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.428+0000 7fabed200640 1 -- 172.21.15.2:0/1993268256 shutdown_connections 2024-09-10T08:09:58.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.428+0000 7fabed200640 1 -- 172.21.15.2:0/1993268256 wait complete. 2024-09-10T08:09:58.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.430+0000 7fabed200640 1 Processor -- start 2024-09-10T08:09:58.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.430+0000 7fabed200640 1 -- start start 2024-09-10T08:09:58.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.430+0000 7fabed200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabe8074510 0x7fabe819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:58.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.430+0000 7fabed200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabe8074e30 0x7fabe819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:58.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.430+0000 7fabed200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe810e280 0x7fabe819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:58.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.430+0000 7fabed200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8113fb0 con 0x7fabe8074e30 2024-09-10T08:09:58.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.430+0000 7fabed200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8113e30 con 0x7fabe810e280 2024-09-10T08:09:58.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.430+0000 7fabed200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8114130 con 0x7fabe8074510 2024-09-10T08:09:58.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.430+0000 7fabe6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabe8074e30 0x7fabe819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:58.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.431+0000 7fabe6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabe8074e30 0x7fabe819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58410/0 (socket says 172.21.15.2:58410) 2024-09-10T08:09:58.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.431+0000 7fabe6a00640 1 -- 172.21.15.2:0/2234256270 learned_addr learned my addr 172.21.15.2:0/2234256270 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:09:58.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.431+0000 7fabe7400640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabe8074510 0x7fabe819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:58.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.431+0000 7fabe7e00640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe810e280 0x7fabe819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:58.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.431+0000 7fabe6a00640 1 -- 172.21.15.2:0/2234256270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabe8074510 msgr2=0x7fabe819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:58.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.431+0000 7fabe6a00640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabe8074510 0x7fabe819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:58.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.431+0000 7fabe6a00640 1 -- 172.21.15.2:0/2234256270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe810e280 msgr2=0x7fabe819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:58.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.431+0000 7fabe6a00640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe810e280 0x7fabe819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:58.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.431+0000 7fabe6a00640 1 -- 172.21.15.2:0/2234256270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe819f4f0 con 0x7fabe8074e30 2024-09-10T08:09:58.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.431+0000 7fabe6a00640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabe8074e30 0x7fabe819aa80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fabd800e9f0 tx=0x7fabd800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:58.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.432+0000 7fabdfe00640 1 -- 172.21.15.2:0/2234256270 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd800cd80 con 0x7fabe8074e30 2024-09-10T08:09:58.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.432+0000 7fabdfe00640 1 -- 172.21.15.2:0/2234256270 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd8005500 con 0x7fabe8074e30 2024-09-10T08:09:58.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.432+0000 7fabdfe00640 1 -- 172.21.15.2:0/2234256270 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd8010690 con 0x7fabe8074e30 2024-09-10T08:09:58.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.432+0000 7fabed200640 1 -- 172.21.15.2:0/2234256270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe819f7e0 con 0x7fabe8074e30 2024-09-10T08:09:58.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.432+0000 7fabed200640 1 -- 172.21.15.2:0/2234256270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe81a70c0 con 0x7fabe8074e30 2024-09-10T08:09:58.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.433+0000 7fabed200640 1 -- 172.21.15.2:0/2234256270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe80748f0 con 0x7fabe8074e30 2024-09-10T08:09:58.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.435+0000 7fabdfe00640 1 -- 172.21.15.2:0/2234256270 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fabd80046c0 con 0x7fabe8074e30 2024-09-10T08:09:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.436+0000 7fabdfe00640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fabcc0a2b80 0x7fabcc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:09:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.437+0000 7fabdfe00640 1 -- 172.21.15.2:0/2234256270 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fabd80cf870 con 0x7fabe8074e30 2024-09-10T08:09:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.437+0000 7fabe7400640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fabcc0a2b80 0x7fabcc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:09:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.438+0000 7fabe7400640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fabcc0a2b80 0x7fabcc0a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009230 tx=0x7fabd00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:09:58.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.441+0000 7fabdfe00640 1 -- 172.21.15.2:0/2234256270 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fabd809c380 con 0x7fabe8074e30 2024-09-10T08:09:58.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.666+0000 7fabed200640 1 -- 172.21.15.2:0/2234256270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabe8108e70 con 0x7fabe8074e30 2024-09-10T08:09:58.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.667+0000 7fabdfe00640 1 -- 172.21.15.2:0/2234256270 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fabd809bc80 con 0x7fabe8074e30 2024-09-10T08:09:58.667 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:09:58.667 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:09:58.668 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:09:58.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.670+0000 7fabed200640 1 -- 172.21.15.2:0/2234256270 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fabcc0a2b80 msgr2=0x7fabcc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:58.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.670+0000 7fabed200640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fabcc0a2b80 0x7fabcc0a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009230 tx=0x7fabd00058d0 comp rx=0 tx=0).stop 2024-09-10T08:09:58.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.670+0000 7fabed200640 1 -- 172.21.15.2:0/2234256270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabe8074e30 msgr2=0x7fabe819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:09:58.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.670+0000 7fabed200640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabe8074e30 0x7fabe819aa80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fabd800e9f0 tx=0x7fabd800edf0 comp rx=0 tx=0).stop 2024-09-10T08:09:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.671+0000 7fabed200640 1 -- 172.21.15.2:0/2234256270 shutdown_connections 2024-09-10T08:09:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.671+0000 7fabed200640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe810e280 0x7fabe819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.671+0000 7fabed200640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabe8074e30 0x7fabe819aa80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.671+0000 7fabed200640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fabcc0a2b80 0x7fabcc0a5040 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.671+0000 7fabed200640 1 --2- 172.21.15.2:0/2234256270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabe8074510 0x7fabe819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:09:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.671+0000 7fabed200640 1 -- 172.21.15.2:0/2234256270 >> 172.21.15.2:0/2234256270 conn(0x7fabe80fd740 msgr2=0x7fabe80fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:09:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.671+0000 7fabed200640 1 -- 172.21.15.2:0/2234256270 shutdown_connections 2024-09-10T08:09:58.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:09:58.671+0000 7fabed200640 1 -- 172.21.15.2:0/2234256270 wait complete. 2024-09-10T08:09:59.001 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:58 smithi002 ceph-mon[30563]: pgmap v442: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:09:59.001 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:09:58 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2234256270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:59.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:58 smithi155 ceph-mon[35405]: pgmap v442: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:09:59.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:09:58 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2234256270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:59.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:58 smithi107 ceph-mon[35048]: pgmap v442: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:09:59.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:09:58 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2234256270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:09:59.227 INFO:tasks.workunit.client.0.smithi002.stderr:Updating files: 57% (7242/12697) Updating files: 58% (7365/12697) Updating files: 59% (7492/12697) Updating files: 60% (7619/12697) Updating files: 61% (7746/12697) Updating files: 62% (7873/12697) Updating files: 63% (8000/12697) Updating files: 64% (8127/12697) Updating files: 65% (8254/12697) Updating files: 66% (8381/12697) Updating files: 67% (8507/12697) Updating files: 68% (8634/12697) Updating files: 69% (8761/12697) Updating files: 70% (8888/12697) Updating files: 71% (9015/12697) Updating files: 72% (9142/12697) Updating files: 73% (9269/12697) Updating files: 74% (9396/12697) Updating files: 75% (9523/12697) Updating files: 76% (9650/12697) Updating files: 77% (9777/12697) Updating files: 78% (9904/12697) Updating files: 79% (10031/12697) Updating files: 80% (10158/12697) Updating files: 81% (10285/12697) Updating files: 82% (10412/12697) Updating files: 83% (10539/12697) Updating files: 84% (10666/12697) Updating files: 85% (10793/12697) Updating files: 86% (10920/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 92% (11737/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T08:09:59.391 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:09:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:09:59.893 INFO:tasks.workunit.client.0.smithi002.stderr:Note: switching to '1cf819b7212a6048e92fc78c4eff963344f6d56b'. 2024-09-10T08:09:59.893 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-10T08:09:59.893 INFO:tasks.workunit.client.0.smithi002.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T08:09:59.893 INFO:tasks.workunit.client.0.smithi002.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T08:09:59.893 INFO:tasks.workunit.client.0.smithi002.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T08:09:59.894 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-10T08:09:59.894 INFO:tasks.workunit.client.0.smithi002.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T08:09:59.894 INFO:tasks.workunit.client.0.smithi002.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T08:09:59.894 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-10T08:09:59.894 INFO:tasks.workunit.client.0.smithi002.stderr: git switch -c 2024-09-10T08:09:59.894 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-10T08:09:59.894 INFO:tasks.workunit.client.0.smithi002.stderr:Or undo this operation with: 2024-09-10T08:09:59.894 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-10T08:09:59.894 INFO:tasks.workunit.client.0.smithi002.stderr: git switch - 2024-09-10T08:09:59.894 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-10T08:09:59.894 INFO:tasks.workunit.client.0.smithi002.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T08:09:59.894 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-10T08:09:59.894 INFO:tasks.workunit.client.0.smithi002.stderr:HEAD is now at 1cf819b7212 Merge PR #59672 into wip-rishabh-testing-20240909.165402-squid 2024-09-10T08:09:59.910 DEBUG:teuthology.orchestra.run.smithi002:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T08:10:00.017 INFO:tasks.workunit.client.0.smithi002.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T08:10:00.023 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T08:10:00.024 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T08:10:00.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.451+0000 7fbeb7200640 1 -- 172.21.15.2:0/4293142861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbeb81019c0 msgr2=0x7fbeb8101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:00.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.451+0000 7fbeb7200640 1 --2- 172.21.15.2:0/4293142861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbeb81019c0 0x7fbeb8101e20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009930 tx=0x7fbea002a420 comp rx=0 tx=0).stop 2024-09-10T08:10:00.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.451+0000 7fbeb7200640 1 -- 172.21.15.2:0/4293142861 shutdown_connections 2024-09-10T08:10:00.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.451+0000 7fbeb7200640 1 --2- 172.21.15.2:0/4293142861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb8102360 0x7fbeb810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.451+0000 7fbeb7200640 1 --2- 172.21.15.2:0/4293142861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbeb81019c0 0x7fbeb8101e20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.451+0000 7fbeb7200640 1 --2- 172.21.15.2:0/4293142861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb81079c0 0x7fbeb8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.451+0000 7fbeb7200640 1 -- 172.21.15.2:0/4293142861 >> 172.21.15.2:0/4293142861 conn(0x7fbeb80fd740 msgr2=0x7fbeb80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:00.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.451+0000 7fbeb7200640 1 -- 172.21.15.2:0/4293142861 shutdown_connections 2024-09-10T08:10:00.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.451+0000 7fbeb7200640 1 -- 172.21.15.2:0/4293142861 wait complete. 2024-09-10T08:10:00.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.452+0000 7fbeb7200640 1 Processor -- start 2024-09-10T08:10:00.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.452+0000 7fbeb7200640 1 -- start start 2024-09-10T08:10:00.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.453+0000 7fbeb7200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbeb81019c0 0x7fbeb819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.453+0000 7fbeb7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb8102360 0x7fbeb819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.453+0000 7fbeb7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb81079c0 0x7fbeb819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.453+0000 7fbeb7200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8111e10 con 0x7fbeb81019c0 2024-09-10T08:10:00.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.453+0000 7fbeb7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8111c90 con 0x7fbeb81079c0 2024-09-10T08:10:00.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.453+0000 7fbeb7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8111f90 con 0x7fbeb8102360 2024-09-10T08:10:00.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.453+0000 7fbeb5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbeb81019c0 0x7fbeb819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.453+0000 7fbeb5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbeb81019c0 0x7fbeb819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58438/0 (socket says 172.21.15.2:58438) 2024-09-10T08:10:00.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.453+0000 7fbeb5e00640 1 -- 172.21.15.2:0/962922613 learned_addr learned my addr 172.21.15.2:0/962922613 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:00.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.453+0000 7fbead400640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb8102360 0x7fbeb819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.453+0000 7fbeb6800640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb81079c0 0x7fbeb819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.454+0000 7fbeb5e00640 1 -- 172.21.15.2:0/962922613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb8102360 msgr2=0x7fbeb819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.454+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb8102360 0x7fbeb819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.454+0000 7fbeb5e00640 1 -- 172.21.15.2:0/962922613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb81079c0 msgr2=0x7fbeb819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.454+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb81079c0 0x7fbeb819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.454+0000 7fbeb5e00640 1 -- 172.21.15.2:0/962922613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb819f480 con 0x7fbeb81019c0 2024-09-10T08:10:00.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.454+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbeb81019c0 0x7fbeb819a4d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbea800ebc0 tx=0x7fbea800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:00.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.454+0000 7fbeaf400640 1 -- 172.21.15.2:0/962922613 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8010630 con 0x7fbeb81019c0 2024-09-10T08:10:00.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.454+0000 7fbeaf400640 1 -- 172.21.15.2:0/962922613 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea8010cb0 con 0x7fbeb81019c0 2024-09-10T08:10:00.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.454+0000 7fbeb7200640 1 -- 172.21.15.2:0/962922613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb819f770 con 0x7fbeb81019c0 2024-09-10T08:10:00.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.454+0000 7fbeaf400640 1 -- 172.21.15.2:0/962922613 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8018bd0 con 0x7fbeb81019c0 2024-09-10T08:10:00.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.454+0000 7fbeb7200640 1 -- 172.21.15.2:0/962922613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb81a7050 con 0x7fbeb81019c0 2024-09-10T08:10:00.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.457+0000 7fbeaf400640 1 -- 172.21.15.2:0/962922613 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbea80107d0 con 0x7fbeb81019c0 2024-09-10T08:10:00.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.458+0000 7fbebf5db740 1 Processor -- start 2024-09-10T08:10:00.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.458+0000 7fbebf5db740 1 -- start start 2024-09-10T08:10:00.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.458+0000 7fbebf5db740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0592a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.458+0000 7fbebf5db740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c05934ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.458+0000 7fbebf5db740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05937410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.458+0000 7fbebf5db740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557c058903e0 con 0x557c05935020 2024-09-10T08:10:00.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.458+0000 7fbebf5db740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557c05837d80 con 0x557c05929e40 2024-09-10T08:10:00.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.458+0000 7fbebf5db740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557c0588c3a0 con 0x557c0592a760 2024-09-10T08:10:00.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.459+0000 7fbead400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05937410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.459+0000 7fbeb5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c05934ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.459+0000 7fbead400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05937410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58446/0 (socket says 172.21.15.2:58446) 2024-09-10T08:10:00.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.459+0000 7fbead400640 1 -- 172.21.15.2:0/3313421709 learned_addr learned my addr 172.21.15.2:0/3313421709 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:00.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.459+0000 7fbeb6800640 1 --2- 172.21.15.2:0/3313421709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0592a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.459+0000 7fbeb5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c05934ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:41464/0 (socket says 172.21.15.2:41464) 2024-09-10T08:10:00.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.459+0000 7fbead400640 1 -- 172.21.15.2:0/3313421709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 msgr2=0x557c05934ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:00.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.459+0000 7fbead400640 1 --2- 172.21.15.2:0/3313421709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c05934ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.459+0000 7fbead400640 1 -- 172.21.15.2:0/3313421709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 msgr2=0x557c0592a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:00.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.459+0000 7fbead400640 1 --2- 172.21.15.2:0/3313421709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0592a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.459+0000 7fbead400640 1 -- 172.21.15.2:0/3313421709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c05856ae0 con 0x557c05935020 2024-09-10T08:10:00.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.459+0000 7fbead400640 1 --2- 172.21.15.2:0/3313421709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05937410 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009930 tx=0x7fbea0005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24971461e355ae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:00.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.459+0000 7fbeaf400640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbe9c0a2c00 0x7fbe9c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.460+0000 7fbeaf400640 1 -- 172.21.15.2:0/962922613 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbea80cf9a0 con 0x7fbeb81019c0 2024-09-10T08:10:00.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.460+0000 7fbea7000640 1 -- 172.21.15.2:0/3313421709 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0035040 con 0x557c05935020 2024-09-10T08:10:00.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.460+0000 7fbeb6800640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbe9c0a2c00 0x7fbe9c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.460+0000 7fbea7000640 1 -- 172.21.15.2:0/3313421709 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea0005520 con 0x557c05935020 2024-09-10T08:10:00.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.460+0000 7fbea7000640 1 -- 172.21.15.2:0/3313421709 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea00035f0 con 0x557c05935020 2024-09-10T08:10:00.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.461+0000 7fbebf5db740 1 -- 172.21.15.2:0/3313421709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 msgr2=0x557c05937410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:00.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.461+0000 7fbebf5db740 1 --2- 172.21.15.2:0/3313421709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05937410 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009930 tx=0x7fbea0005de0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.461+0000 7fbeb6800640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbe9c0a2c00 0x7fbe9c0a50c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009da0 tx=0x7fbeb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:00.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.461+0000 7fbebf5db740 1 -- 172.21.15.2:0/3313421709 shutdown_connections 2024-09-10T08:10:00.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.461+0000 7fbebf5db740 1 --2- 172.21.15.2:0/3313421709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05937410 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.461+0000 7fbebf5db740 1 --2- 172.21.15.2:0/3313421709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c05934ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.461+0000 7fbebf5db740 1 --2- 172.21.15.2:0/3313421709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0592a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.461+0000 7fbebf5db740 1 -- 172.21.15.2:0/3313421709 >> 172.21.15.2:0/3313421709 conn(0x557c059263d0 msgr2=0x557c059267b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:00.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.461+0000 7fbebf5db740 1 -- 172.21.15.2:0/3313421709 shutdown_connections 2024-09-10T08:10:00.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.461+0000 7fbebf5db740 1 -- 172.21.15.2:0/3313421709 wait complete. 2024-09-10T08:10:00.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.461+0000 7fbebf5db740 1 -- start start 2024-09-10T08:10:00.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbebf5db740 1 Processor -- start 2024-09-10T08:10:00.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbebf5db740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c05931f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbebf5db740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c059324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbebf5db740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05932a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbebf5db740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557c05937bf0 con 0x557c05935020 2024-09-10T08:10:00.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbebf5db740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557c0584b0f0 con 0x557c05929e40 2024-09-10T08:10:00.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbebf5db740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557c05937d70 con 0x557c0592a760 2024-09-10T08:10:00.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbebf5db740 10 client.0 _ll_register_callbacks cb 0x557c05832bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:00.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbeb5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c05931f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbead400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c059324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbeb6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05932a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbead400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c059324d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:41470/0 (socket says 172.21.15.2:41470) 2024-09-10T08:10:00.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbeb5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c05931f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36960/0 (socket says 172.21.15.2:36960) 2024-09-10T08:10:00.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.462+0000 7fbead400640 1 -- 172.21.15.2:0/3094329392 learned_addr learned my addr 172.21.15.2:0/3094329392 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:00.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.463+0000 7fbead400640 1 -- 172.21.15.2:0/3094329392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 msgr2=0x557c05931f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:00.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.463+0000 7fbead400640 1 --2- 172.21.15.2:0/3094329392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c05931f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.463+0000 7fbead400640 1 -- 172.21.15.2:0/3094329392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 msgr2=0x557c05932a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:00.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.463+0000 7fbead400640 1 --2- 172.21.15.2:0/3094329392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05932a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.463+0000 7fbead400640 1 -- 172.21.15.2:0/3094329392 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c056013c0 con 0x557c0592a760 2024-09-10T08:10:00.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.463+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/3094329392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c05931f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:00.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.463+0000 7fbead400640 1 --2- 172.21.15.2:0/3094329392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c059324d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbea00048f0 tx=0x7fbea0004a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32acd85498aa7d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:00.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.463+0000 7fbe9b400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:10:00.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.463+0000 7fbe9b400640 1 -- 172.21.15.2:0/3094329392 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0052060 con 0x557c0592a760 2024-09-10T08:10:00.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.463+0000 7fbebf5db740 1 -- 172.21.15.2:0/3094329392 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea0005520 con 0x557c0592a760 2024-09-10T08:10:00.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.463+0000 7fbe9b400640 1 -- 172.21.15.2:0/3094329392 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbea0004390 con 0x557c0592a760 2024-09-10T08:10:00.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.464+0000 7fbe9b400640 1 -- 172.21.15.2:0/3094329392 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbea0059050 con 0x557c0592a760 2024-09-10T08:10:00.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.464+0000 7fbebf5db740 10 client.25645 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:00.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.464+0000 7fbebf5db740 1 -- 172.21.15.2:0/3094329392 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c05937bf0 con 0x557c0592a760 2024-09-10T08:10:00.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.464+0000 7fbe9b400640 1 -- 172.21.15.2:0/3094329392 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea0005520 con 0x557c0592a760 2024-09-10T08:10:00.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.464+0000 7fbe9b400640 1 -- 172.21.15.2:0/3094329392 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbea00038c0 con 0x557c0592a760 2024-09-10T08:10:00.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.465+0000 7fbebf5db740 10 client.25645 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:00.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.465+0000 7fbebf5db740 10 client.25645 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:10:00.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.465+0000 7fbebf5db740 20 client.25645 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:00.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.465+0000 7fbebf5db740 1 --2- 172.21.15.2:0/3094329392 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557c05944e30 0x557c05947220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.465+0000 7fbebf5db740 4 client.25645 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:00.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.465+0000 7fbebf5db740 1 -- 172.21.15.2:0/3094329392 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c0589d950 con 0x557c05944e30 2024-09-10T08:10:00.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.465+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/3094329392 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557c05944e30 0x557c05947220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.465+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/3094329392 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557c05944e30 0x557c05947220 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93edb709e7636a2c server_cookie=29f741fb283d31e6 in_seq=0 out_seq=0 2024-09-10T08:10:00.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.465+0000 7fbe9b400640 10 client.25645 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:00.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.466+0000 7fbe9b400640 1 -- 172.21.15.2:0/3094329392 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c0589d950 con 0x557c05944e30 2024-09-10T08:10:00.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.466+0000 7fbe9b400640 10 client.25645 handle_command_reply: tid=0 2024-09-10T08:10:00.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.466+0000 7fbebf5db740 1 client.25645 shutdown 2024-09-10T08:10:00.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.466+0000 7fbebf5db740 1 -- 172.21.15.2:0/3094329392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 msgr2=0x557c059324d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:00.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.466+0000 7fbebf5db740 1 --2- 172.21.15.2:0/3094329392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c059324d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbea00048f0 tx=0x7fbea0004a80 comp rx=0 tx=0).stop 2024-09-10T08:10:00.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.467+0000 7fbebf5db740 1 -- 172.21.15.2:0/3094329392 shutdown_connections 2024-09-10T08:10:00.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.467+0000 7fbebf5db740 1 --2- 172.21.15.2:0/3094329392 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557c05944e30 0x557c05947220 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.467+0000 7fbebf5db740 1 --2- 172.21.15.2:0/3094329392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05932a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.467+0000 7fbe9b400640 0 client.25645 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:00.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.467+0000 7fbebf5db740 1 --2- 172.21.15.2:0/3094329392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c059324d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.467+0000 7fbebf5db740 1 --2- 172.21.15.2:0/3094329392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c05931f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.467+0000 7fbebf5db740 1 -- 172.21.15.2:0/3094329392 >> 172.21.15.2:0/3094329392 conn(0x557c059263d0 msgr2=0x557c0592c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:00.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.467+0000 7fbebf5db740 1 -- 172.21.15.2:0/3094329392 shutdown_connections 2024-09-10T08:10:00.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.467+0000 7fbebf5db740 1 -- 172.21.15.2:0/3094329392 wait complete. 2024-09-10T08:10:00.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.467+0000 7fbebf5db740 20 client.25645 trim_cache size 0 max 16384 2024-09-10T08:10:00.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.481+0000 7fbebf5db740 1 Processor -- start 2024-09-10T08:10:00.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.481+0000 7fbebf5db740 1 -- start start 2024-09-10T08:10:00.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.481+0000 7fbebf5db740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0594c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.481+0000 7fbebf5db740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c0594cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbebf5db740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05924c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbebf5db740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81b3760 con 0x557c05935020 2024-09-10T08:10:00.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbebf5db740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81b1030 con 0x557c05929e40 2024-09-10T08:10:00.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbebf5db740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81aee80 con 0x557c0592a760 2024-09-10T08:10:00.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbead400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c0594cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbeb6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05924c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbead400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c0594cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:41474/0 (socket says 172.21.15.2:41474) 2024-09-10T08:10:00.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbead400640 1 -- 172.21.15.2:0/375549149 learned_addr learned my addr 172.21.15.2:0/375549149 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:00.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/375549149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0594c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbeb5e00640 1 -- 172.21.15.2:0/375549149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 msgr2=0x557c0594cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:00.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/375549149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c0594cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbeb5e00640 1 -- 172.21.15.2:0/375549149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 msgr2=0x557c05924c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:00.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/375549149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05924c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.482+0000 7fbeb5e00640 1 -- 172.21.15.2:0/375549149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c05929c70 con 0x557c05929e40 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.483+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/375549149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0594c960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x557c05927230 tx=0x7fbea80d5530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dcb02b5df134101 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.483+0000 7fbea6600640 1 -- 172.21.15.2:0/375549149 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea80d1e60 con 0x557c05929e40 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.483+0000 7fbea6600640 1 -- 172.21.15.2:0/375549149 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea809be60 con 0x557c05929e40 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.483+0000 7fbea6600640 1 -- 172.21.15.2:0/375549149 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8010e60 con 0x557c05929e40 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.484+0000 7fbebf5db740 1 -- 172.21.15.2:0/375549149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 msgr2=0x557c0594c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.484+0000 7fbebf5db740 1 --2- 172.21.15.2:0/375549149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0594c960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x557c05927230 tx=0x7fbea80d5530 comp rx=0 tx=0).stop 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.484+0000 7fbebf5db740 1 -- 172.21.15.2:0/375549149 shutdown_connections 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.484+0000 7fbebf5db740 1 --2- 172.21.15.2:0/375549149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05924c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.484+0000 7fbebf5db740 1 --2- 172.21.15.2:0/375549149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c0594cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.484+0000 7fbebf5db740 1 --2- 172.21.15.2:0/375549149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0594c960 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.484+0000 7fbebf5db740 1 -- 172.21.15.2:0/375549149 >> 172.21.15.2:0/375549149 conn(0x557c059263d0 msgr2=0x557c05931080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.484+0000 7fbebf5db740 1 -- 172.21.15.2:0/375549149 shutdown_connections 2024-09-10T08:10:00.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.484+0000 7fbebf5db740 1 -- 172.21.15.2:0/375549149 wait complete. 2024-09-10T08:10:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.484+0000 7fbebf5db740 1 -- start start 2024-09-10T08:10:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.485+0000 7fbebf5db740 1 Processor -- start 2024-09-10T08:10:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.485+0000 7fbebf5db740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0593d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.485+0000 7fbebf5db740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c05938840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.485+0000 7fbebf5db740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05938d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.485+0000 7fbebf5db740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81af4f0 con 0x557c05935020 2024-09-10T08:10:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.485+0000 7fbebf5db740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81b0960 con 0x557c05929e40 2024-09-10T08:10:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.485+0000 7fbebf5db740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81afbc0 con 0x557c0592a760 2024-09-10T08:10:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.485+0000 7fbebf5db740 10 client.0 _ll_register_callbacks cb 0x7fbeb0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.485+0000 7fbeb6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05938d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.485+0000 7fbeb6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05938d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58466/0 (socket says 172.21.15.2:58466) 2024-09-10T08:10:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.485+0000 7fbeb6800640 1 -- 172.21.15.2:0/325861394 learned_addr learned my addr 172.21.15.2:0/325861394 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.485+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0593d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.485+0000 7fbead400640 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c05938840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.486+0000 7fbeb6800640 1 -- 172.21.15.2:0/325861394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 msgr2=0x557c05938840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.486+0000 7fbeb6800640 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c05938840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.486+0000 7fbeb6800640 1 -- 172.21.15.2:0/325861394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 msgr2=0x557c0593d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.486+0000 7fbeb6800640 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0593d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.486+0000 7fbeb6800640 1 -- 172.21.15.2:0/325861394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c05927060 con 0x557c05935020 2024-09-10T08:10:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.486+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0593d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.486+0000 7fbeb6800640 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05938d80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0004620 tx=0x7fbeb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49298577654e0cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.486+0000 7fbe9b400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.486+0000 7fbe9b400640 1 -- 172.21.15.2:0/325861394 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb002e070 con 0x557c05935020 2024-09-10T08:10:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.487+0000 7fbebf5db740 1 -- 172.21.15.2:0/325861394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea809be60 con 0x557c05935020 2024-09-10T08:10:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.487+0000 7fbe9b400640 1 -- 172.21.15.2:0/325861394 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbeb0005020 con 0x557c05935020 2024-09-10T08:10:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.487+0000 7fbe9b400640 1 -- 172.21.15.2:0/325861394 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbeb0037050 con 0x557c05935020 2024-09-10T08:10:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.487+0000 7fbebf5db740 10 client.15633 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.487+0000 7fbebf5db740 1 -- 172.21.15.2:0/325861394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c059af5d0 con 0x557c05935020 2024-09-10T08:10:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.487+0000 7fbe9b400640 1 -- 172.21.15.2:0/325861394 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea809be60 con 0x557c05935020 2024-09-10T08:10:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.487+0000 7fbe9b400640 1 -- 172.21.15.2:0/325861394 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbeb0031020 con 0x557c05935020 2024-09-10T08:10:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.487+0000 7fbebf5db740 10 client.15633 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.487+0000 7fbebf5db740 10 client.15633 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:10:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.487+0000 7fbebf5db740 20 client.15633 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.487+0000 7fbebf5db740 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557c059afda0 0x557c059b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.487+0000 7fbebf5db740 4 client.15633 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:10:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.487+0000 7fbebf5db740 1 -- 172.21.15.2:0/325861394 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c05897e60 con 0x557c059afda0 2024-09-10T08:10:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.488+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557c059afda0 0x557c059b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.488+0000 7fbeb5e00640 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557c059afda0 0x557c059b21c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e84362cc9f0cf4fe server_cookie=37524dd61b9fbde7 in_seq=0 out_seq=0 2024-09-10T08:10:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.488+0000 7fbe9b400640 10 client.15633 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.488+0000 7fbe9b400640 1 -- 172.21.15.2:0/325861394 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c05897e60 con 0x557c059afda0 2024-09-10T08:10:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.488+0000 7fbe9b400640 10 client.15633 handle_command_reply: tid=0 2024-09-10T08:10:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.488+0000 7fbebf5db740 1 client.15633 shutdown 2024-09-10T08:10:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.489+0000 7fbebf5db740 1 -- 172.21.15.2:0/325861394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 msgr2=0x557c05938d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.489+0000 7fbebf5db740 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05938d80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0004620 tx=0x7fbeb0004730 comp rx=0 tx=0).stop 2024-09-10T08:10:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.489+0000 7fbebf5db740 1 -- 172.21.15.2:0/325861394 shutdown_connections 2024-09-10T08:10:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.489+0000 7fbebf5db740 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557c059afda0 0x557c059b21c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.489+0000 7fbebf5db740 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c05935020 0x557c05938d80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.489+0000 7fbebf5db740 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c0592a760 0x557c05938840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.489+0000 7fbebf5db740 1 --2- 172.21.15.2:0/325861394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c05929e40 0x557c0593d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.489+0000 7fbebf5db740 1 -- 172.21.15.2:0/325861394 >> 172.21.15.2:0/325861394 conn(0x557c059263d0 msgr2=0x557c0592b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.489+0000 7fbe9b400640 0 client.15633 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.489+0000 7fbebf5db740 1 -- 172.21.15.2:0/325861394 shutdown_connections 2024-09-10T08:10:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.489+0000 7fbebf5db740 1 -- 172.21.15.2:0/325861394 wait complete. 2024-09-10T08:10:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.489+0000 7fbebf5db740 20 client.15633 trim_cache size 0 max 16384 2024-09-10T08:10:00.518 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:10:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.490+0000 7fbeb7200640 1 -- 172.21.15.2:0/962922613 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbe9c0a2c00 msgr2=0x7fbe9c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.490+0000 7fbeb7200640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbe9c0a2c00 0x7fbe9c0a50c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009da0 tx=0x7fbeb0005f90 comp rx=0 tx=0).stop 2024-09-10T08:10:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.490+0000 7fbeb7200640 1 -- 172.21.15.2:0/962922613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbeb81019c0 msgr2=0x7fbeb819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.490+0000 7fbeb7200640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbeb81019c0 0x7fbeb819a4d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbea800ebc0 tx=0x7fbea800c6a0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.490+0000 7fbeb7200640 1 -- 172.21.15.2:0/962922613 shutdown_connections 2024-09-10T08:10:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.490+0000 7fbeb7200640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb81079c0 0x7fbeb819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.490+0000 7fbeb7200640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbe9c0a2c00 0x7fbe9c0a50c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.490+0000 7fbeb7200640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb8102360 0x7fbeb819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.490+0000 7fbeb7200640 1 --2- 172.21.15.2:0/962922613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbeb81019c0 0x7fbeb819a4d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.490+0000 7fbeb7200640 1 -- 172.21.15.2:0/962922613 >> 172.21.15.2:0/962922613 conn(0x7fbeb80fd740 msgr2=0x7fbeb810d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.490+0000 7fbeb7200640 1 -- 172.21.15.2:0/962922613 shutdown_connections 2024-09-10T08:10:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:00.490+0000 7fbeb7200640 1 -- 172.21.15.2:0/962922613 wait complete. 2024-09-10T08:10:01.042 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:00 smithi002 ceph-mon[30563]: pgmap v443: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:10:01.042 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:00 smithi002 ceph-mon[30563]: overall HEALTH_OK 2024-09-10T08:10:01.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:00 smithi155 ceph-mon[35405]: pgmap v443: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:10:01.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:00 smithi155 ceph-mon[35405]: overall HEALTH_OK 2024-09-10T08:10:01.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:00 smithi107 ceph-mon[35048]: pgmap v443: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:10:01.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:00 smithi107 ceph-mon[35048]: overall HEALTH_OK 2024-09-10T08:10:01.636 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:10:01.775 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T08:10:01.874 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T08:10:01.950 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T08:10:01.953 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T08:10:01.953 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T08:10:02.026 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T08:10:02.033 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T08:10:02.033 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T08:10:02.040 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:02.070 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-09-10T08:10:02.071 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-09-10T08:10:02.071 DEBUG:teuthology.orchestra.run.smithi002:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-10T08:10:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.278+0000 7f7798c00640 1 -- 172.21.15.2:0/1844276571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7794074f60 msgr2=0x7f77941097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.278+0000 7f7798c00640 1 --2- 172.21.15.2:0/1844276571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7794074f60 0x7f77941097b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f777c009900 tx=0x7f777c02a410 comp rx=0 tx=0).stop 2024-09-10T08:10:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.278+0000 7f7798c00640 1 -- 172.21.15.2:0/1844276571 shutdown_connections 2024-09-10T08:10:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.278+0000 7f7798c00640 1 --2- 172.21.15.2:0/1844276571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7794109cf0 0x7f779410c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.278+0000 7f7798c00640 1 --2- 172.21.15.2:0/1844276571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7794074f60 0x7f77941097b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.278+0000 7f7798c00640 1 --2- 172.21.15.2:0/1844276571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7794074620 0x7f7794074a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.278+0000 7f7798c00640 1 -- 172.21.15.2:0/1844276571 >> 172.21.15.2:0/1844276571 conn(0x7f77940fd0d0 msgr2=0x7f77940ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.278+0000 7f7798c00640 1 -- 172.21.15.2:0/1844276571 shutdown_connections 2024-09-10T08:10:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.278+0000 7f7798c00640 1 -- 172.21.15.2:0/1844276571 wait complete. 2024-09-10T08:10:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.279+0000 7f7798c00640 1 Processor -- start 2024-09-10T08:10:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.279+0000 7f7798c00640 1 -- start start 2024-09-10T08:10:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.279+0000 7f7798c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7794074620 0x7f7794199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.279+0000 7f7798c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7794074f60 0x7f779419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.279+0000 7f7798c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7794109cf0 0x7f77941a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.279+0000 7f7798c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f779410f8c0 con 0x7f7794074f60 2024-09-10T08:10:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.279+0000 7f7798c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f779410f740 con 0x7f7794109cf0 2024-09-10T08:10:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.280+0000 7f7798c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f779410fa40 con 0x7f7794074620 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.280+0000 7f778be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7794074f60 0x7f779419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.280+0000 7f7793400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7794074620 0x7f7794199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.280+0000 7f778be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7794074f60 0x7f779419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58478/0 (socket says 172.21.15.2:58478) 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.280+0000 7f7793e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7794109cf0 0x7f77941a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.280+0000 7f7793400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7794074620 0x7f7794199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:41500/0 (socket says 172.21.15.2:41500) 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.280+0000 7f778be00640 1 -- 172.21.15.2:0/919092916 learned_addr learned my addr 172.21.15.2:0/919092916 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.280+0000 7f7793e00640 1 -- 172.21.15.2:0/919092916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7794074620 msgr2=0x7f7794199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.280+0000 7f7793e00640 1 --2- 172.21.15.2:0/919092916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7794074620 0x7f7794199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.280+0000 7f7793e00640 1 -- 172.21.15.2:0/919092916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7794074f60 msgr2=0x7f779419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.280+0000 7f7793e00640 1 --2- 172.21.15.2:0/919092916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7794074f60 0x7f779419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.280+0000 7f7793e00640 1 -- 172.21.15.2:0/919092916 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77941a1bc0 con 0x7f7794109cf0 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.280+0000 7f778be00640 1 --2- 172.21.15.2:0/919092916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7794074f60 0x7f779419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.281+0000 7f7793e00640 1 --2- 172.21.15.2:0/919092916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7794109cf0 0x7f77941a1550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f778c00da10 tx=0x7f778c00ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.281+0000 7f7790c00640 1 -- 172.21.15.2:0/919092916 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c00bd70 con 0x7f7794109cf0 2024-09-10T08:10:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.281+0000 7f7790c00640 1 -- 172.21.15.2:0/919092916 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f778c003e50 con 0x7f7794109cf0 2024-09-10T08:10:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.281+0000 7f7798c00640 1 -- 172.21.15.2:0/919092916 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77941a1eb0 con 0x7f7794109cf0 2024-09-10T08:10:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.281+0000 7f7790c00640 1 -- 172.21.15.2:0/919092916 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c0106a0 con 0x7f7794109cf0 2024-09-10T08:10:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.281+0000 7f7798c00640 1 -- 172.21.15.2:0/919092916 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77941a2320 con 0x7f7794109cf0 2024-09-10T08:10:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.284+0000 7f779d1dc740 1 Processor -- start 2024-09-10T08:10:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.284+0000 7f7790c00640 1 -- 172.21.15.2:0/919092916 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f778c003970 con 0x7f7794109cf0 2024-09-10T08:10:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.285+0000 7f779d1dc740 1 -- start start 2024-09-10T08:10:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.285+0000 7f779d1dc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9ab260 0x560d3b9ab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.285+0000 7f779d1dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9abba0 0x560d3b9b0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.285+0000 7f779d1dc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9b0c30 0x560d3b9b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.285+0000 7f779d1dc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560d3b910100 con 0x560d3b9b0c30 2024-09-10T08:10:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.285+0000 7f779d1dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560d3b8b81b0 con 0x560d3b9abba0 2024-09-10T08:10:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.285+0000 7f779d1dc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d3b90c0c0 con 0x560d3b9ab260 2024-09-10T08:10:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.285+0000 7f778be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9ab260 0x560d3b9ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.285+0000 7f7793400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9abba0 0x560d3b9b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.285+0000 7f778be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9ab260 0x560d3b9ab660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:41516/0 (socket says 172.21.15.2:41516) 2024-09-10T08:10:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.285+0000 7f778be00640 1 -- 172.21.15.2:0/301235249 learned_addr learned my addr 172.21.15.2:0/301235249 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.285+0000 7f7793400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9abba0 0x560d3b9b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37008/0 (socket says 172.21.15.2:37008) 2024-09-10T08:10:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.285+0000 7f778be00640 1 --2- 172.21.15.2:0/301235249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9b0c30 0x560d3b9b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.286+0000 7f7793400640 1 -- 172.21.15.2:0/301235249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9ab260 msgr2=0x560d3b9ab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.286+0000 7f7793400640 1 --2- 172.21.15.2:0/301235249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9ab260 0x560d3b9ab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.286+0000 7f7793400640 1 -- 172.21.15.2:0/301235249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9b0c30 msgr2=0x560d3b9b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.286+0000 7f7793400640 1 --2- 172.21.15.2:0/301235249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9b0c30 0x560d3b9b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.286+0000 7f7793400640 1 -- 172.21.15.2:0/301235249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d3b623210 con 0x560d3b9abba0 2024-09-10T08:10:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.286+0000 7f7793400640 1 --2- 172.21.15.2:0/301235249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9abba0 0x560d3b9b0660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7784009c20 tx=0x7f7784011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef4c527b3a6e428 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.286+0000 7f7790c00640 1 --2- 172.21.15.2:0/919092916 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f77780a2b80 0x7f77780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.286+0000 7f7783e00640 1 -- 172.21.15.2:0/301235249 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784004050 con 0x560d3b9abba0 2024-09-10T08:10:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.286+0000 7f7783e00640 1 -- 172.21.15.2:0/301235249 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7784005500 con 0x560d3b9abba0 2024-09-10T08:10:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.286+0000 7f7783e00640 1 -- 172.21.15.2:0/301235249 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784004d50 con 0x560d3b9abba0 2024-09-10T08:10:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.286+0000 7f7790c00640 1 -- 172.21.15.2:0/919092916 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f778c0cf840 con 0x7f7794109cf0 2024-09-10T08:10:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.287+0000 7f778be00640 1 --2- 172.21.15.2:0/919092916 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f77780a2b80 0x7f77780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.287+0000 7f779d1dc740 1 -- 172.21.15.2:0/301235249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9abba0 msgr2=0x560d3b9b0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.287+0000 7f779d1dc740 1 --2- 172.21.15.2:0/301235249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9abba0 0x560d3b9b0660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7784009c20 tx=0x7f7784011810 comp rx=0 tx=0).stop 2024-09-10T08:10:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.287+0000 7f779d1dc740 1 -- 172.21.15.2:0/301235249 shutdown_connections 2024-09-10T08:10:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.287+0000 7f779d1dc740 1 --2- 172.21.15.2:0/301235249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9b0c30 0x560d3b9b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.287+0000 7f779d1dc740 1 --2- 172.21.15.2:0/301235249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9abba0 0x560d3b9b0660 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.287+0000 7f779d1dc740 1 --2- 172.21.15.2:0/301235249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9ab260 0x560d3b9ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.287+0000 7f779d1dc740 1 -- 172.21.15.2:0/301235249 >> 172.21.15.2:0/301235249 conn(0x560d3b9a68d0 msgr2=0x560d3b9a6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.287+0000 7f778be00640 1 --2- 172.21.15.2:0/919092916 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f77780a2b80 0x7f77780a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f777c005c40 tx=0x7f777c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.287+0000 7f779d1dc740 1 -- 172.21.15.2:0/301235249 shutdown_connections 2024-09-10T08:10:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.287+0000 7f779d1dc740 1 -- 172.21.15.2:0/301235249 wait complete. 2024-09-10T08:10:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.288+0000 7f779d1dc740 1 -- start start 2024-09-10T08:10:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.288+0000 7f779d1dc740 1 Processor -- start 2024-09-10T08:10:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.288+0000 7f779d1dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.288+0000 7f779d1dc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9abba0 0x560d3b9b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.288+0000 7f779d1dc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9b0c30 0x560d3b9bc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.288+0000 7f779d1dc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560d3b9b3c00 con 0x560d3b9b0c30 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.288+0000 7f779d1dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560d3b899090 con 0x560d3b9ab260 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.288+0000 7f779d1dc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d3b9b3d80 con 0x560d3b9abba0 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.289+0000 7f779d1dc740 10 client.0 _ll_register_callbacks cb 0x560d3b8edf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.289+0000 7f7793400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.289+0000 7f7793e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9b0c30 0x560d3b9bc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.289+0000 7f7793400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37018/0 (socket says 172.21.15.2:37018) 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.289+0000 7f7793400640 1 -- 172.21.15.2:0/763135026 learned_addr learned my addr 172.21.15.2:0/763135026 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.289+0000 7f778be00640 1 --2- 172.21.15.2:0/763135026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9abba0 0x560d3b9b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.289+0000 7f7793400640 1 -- 172.21.15.2:0/763135026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9abba0 msgr2=0x560d3b9b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.289+0000 7f7793400640 1 --2- 172.21.15.2:0/763135026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9abba0 0x560d3b9b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.289+0000 7f7793400640 1 -- 172.21.15.2:0/763135026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9b0c30 msgr2=0x560d3b9bc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.289+0000 7f7793400640 1 --2- 172.21.15.2:0/763135026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9b0c30 0x560d3b9bc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.289+0000 7f7793400640 1 -- 172.21.15.2:0/763135026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d3b6ef620 con 0x560d3b9ab260 2024-09-10T08:10:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.289+0000 7f7793400640 1 --2- 172.21.15.2:0/763135026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9b53e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f778401b040 tx=0x7f7784005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a9c8b730c9ce7a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.289+0000 7f7780c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.290+0000 7f7780c00640 1 -- 172.21.15.2:0/763135026 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784029070 con 0x560d3b9ab260 2024-09-10T08:10:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.290+0000 7f779d1dc740 1 -- 172.21.15.2:0/763135026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7784005500 con 0x560d3b9ab260 2024-09-10T08:10:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.290+0000 7f7780c00640 1 -- 172.21.15.2:0/763135026 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7784003a00 con 0x560d3b9ab260 2024-09-10T08:10:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.290+0000 7f7780c00640 1 -- 172.21.15.2:0/763135026 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7784035050 con 0x560d3b9ab260 2024-09-10T08:10:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.290+0000 7f779d1dc740 10 client.25682 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.290+0000 7f779d1dc740 1 -- 172.21.15.2:0/763135026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d3b9bf7a0 con 0x560d3b9ab260 2024-09-10T08:10:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.290+0000 7f7780c00640 1 -- 172.21.15.2:0/763135026 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7784005500 con 0x560d3b9ab260 2024-09-10T08:10:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.290+0000 7f7780c00640 1 -- 172.21.15.2:0/763135026 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7784023430 con 0x560d3b9ab260 2024-09-10T08:10:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.291+0000 7f779d1dc740 10 client.25682 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.291+0000 7f779d1dc740 10 client.25682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.291+0000 7f779d1dc740 20 client.25682 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.291+0000 7f779d1dc740 1 --2- 172.21.15.2:0/763135026 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d3b9c0cf0 0x560d3b9c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.291+0000 7f779d1dc740 4 client.25682 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.291+0000 7f779d1dc740 1 -- 172.21.15.2:0/763135026 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d3b91d830 con 0x560d3b9c0cf0 2024-09-10T08:10:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.291+0000 7f778be00640 1 --2- 172.21.15.2:0/763135026 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d3b9c0cf0 0x560d3b9c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.291+0000 7f778be00640 1 --2- 172.21.15.2:0/763135026 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d3b9c0cf0 0x560d3b9c3170 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d525c3cdf0206eac server_cookie=53aa48ad526b2712 in_seq=0 out_seq=0 2024-09-10T08:10:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.291+0000 7f7780c00640 10 client.25682 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.292+0000 7f7780c00640 1 -- 172.21.15.2:0/763135026 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d3b91d830 con 0x560d3b9c0cf0 2024-09-10T08:10:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.292+0000 7f7780c00640 10 client.25682 handle_command_reply: tid=0 2024-09-10T08:10:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.292+0000 7f779d1dc740 1 client.25682 shutdown 2024-09-10T08:10:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.292+0000 7f779d1dc740 1 -- 172.21.15.2:0/763135026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 msgr2=0x560d3b9b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.292+0000 7f779d1dc740 1 --2- 172.21.15.2:0/763135026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9b53e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f778401b040 tx=0x7f7784005730 comp rx=0 tx=0).stop 2024-09-10T08:10:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.292+0000 7f779d1dc740 1 -- 172.21.15.2:0/763135026 shutdown_connections 2024-09-10T08:10:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.292+0000 7f779d1dc740 1 --2- 172.21.15.2:0/763135026 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d3b9c0cf0 0x560d3b9c3170 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.292+0000 7f779d1dc740 1 --2- 172.21.15.2:0/763135026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9b0c30 0x560d3b9bc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.292+0000 7f7780c00640 0 client.25682 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.292+0000 7f779d1dc740 1 --2- 172.21.15.2:0/763135026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9abba0 0x560d3b9b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.292+0000 7f779d1dc740 1 --2- 172.21.15.2:0/763135026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9b53e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.292+0000 7f779d1dc740 1 -- 172.21.15.2:0/763135026 >> 172.21.15.2:0/763135026 conn(0x560d3b9a68d0 msgr2=0x560d3b9a80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.293+0000 7f779d1dc740 1 -- 172.21.15.2:0/763135026 shutdown_connections 2024-09-10T08:10:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.293+0000 7f779d1dc740 1 -- 172.21.15.2:0/763135026 wait complete. 2024-09-10T08:10:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.293+0000 7f779d1dc740 20 client.25682 trim_cache size 0 max 16384 2024-09-10T08:10:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.308+0000 7f779d1dc740 1 Processor -- start 2024-09-10T08:10:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.308+0000 7f779d1dc740 1 -- start start 2024-09-10T08:10:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.308+0000 7f779d1dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.308+0000 7f779d1dc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 0x560d3b9bc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.308+0000 7f779d1dc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9b0c30 0x560d3b9a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.308+0000 7f779d1dc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f77941af1c0 con 0x560d3b9abba0 2024-09-10T08:10:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.308+0000 7f779d1dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77941aca90 con 0x560d3b9ab260 2024-09-10T08:10:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.308+0000 7f779d1dc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f77941aa8e0 con 0x560d3b9b0c30 2024-09-10T08:10:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.309+0000 7f778be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 0x560d3b9bc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.309+0000 7f778be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 0x560d3b9bc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58510/0 (socket says 172.21.15.2:58510) 2024-09-10T08:10:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.309+0000 7f778be00640 1 -- 172.21.15.2:0/3576447457 learned_addr learned my addr 172.21.15.2:0/3576447457 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.309+0000 7f7793400640 1 --2- 172.21.15.2:0/3576447457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.309+0000 7f7793e00640 1 --2- 172.21.15.2:0/3576447457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9b0c30 0x560d3b9a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.309+0000 7f778be00640 1 -- 172.21.15.2:0/3576447457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9b0c30 msgr2=0x560d3b9a7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.309+0000 7f778be00640 1 --2- 172.21.15.2:0/3576447457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9b0c30 0x560d3b9a7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.309+0000 7f778be00640 1 -- 172.21.15.2:0/3576447457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 msgr2=0x560d3b9b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.309+0000 7f778be00640 1 --2- 172.21.15.2:0/3576447457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.309+0000 7f778be00640 1 -- 172.21.15.2:0/3576447457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d3b9b3ae0 con 0x560d3b9abba0 2024-09-10T08:10:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.309+0000 7f7793400640 1 --2- 172.21.15.2:0/3576447457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.309+0000 7f778be00640 1 --2- 172.21.15.2:0/3576447457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 0x560d3b9bc880 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f777c009230 tx=0x7f777c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcc7c1b60a0b1ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.310+0000 7f7783400640 1 -- 172.21.15.2:0/3576447457 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c035040 con 0x560d3b9abba0 2024-09-10T08:10:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.310+0000 7f7783400640 1 -- 172.21.15.2:0/3576447457 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f777c003dc0 con 0x560d3b9abba0 2024-09-10T08:10:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.310+0000 7f7783400640 1 -- 172.21.15.2:0/3576447457 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c004a00 con 0x560d3b9abba0 2024-09-10T08:10:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.311+0000 7f779d1dc740 1 -- 172.21.15.2:0/3576447457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 msgr2=0x560d3b9bc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.311+0000 7f779d1dc740 1 --2- 172.21.15.2:0/3576447457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 0x560d3b9bc880 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f777c009230 tx=0x7f777c009260 comp rx=0 tx=0).stop 2024-09-10T08:10:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.311+0000 7f779d1dc740 1 -- 172.21.15.2:0/3576447457 shutdown_connections 2024-09-10T08:10:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.311+0000 7f779d1dc740 1 --2- 172.21.15.2:0/3576447457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9b0c30 0x560d3b9a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.311+0000 7f779d1dc740 1 --2- 172.21.15.2:0/3576447457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 0x560d3b9bc880 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.311+0000 7f779d1dc740 1 --2- 172.21.15.2:0/3576447457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.311+0000 7f779d1dc740 1 -- 172.21.15.2:0/3576447457 >> 172.21.15.2:0/3576447457 conn(0x560d3b9a68d0 msgr2=0x560d3b9a5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.311+0000 7f779d1dc740 1 -- 172.21.15.2:0/3576447457 shutdown_connections 2024-09-10T08:10:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.311+0000 7f779d1dc740 1 -- 172.21.15.2:0/3576447457 wait complete. 2024-09-10T08:10:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.311+0000 7f779d1dc740 1 -- start start 2024-09-10T08:10:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.311+0000 7f779d1dc740 1 Processor -- start 2024-09-10T08:10:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f779d1dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9ba310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f779d1dc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 0x560d3b9ba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f779d1dc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9b0c30 0x560d3b9bb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f779d1dc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f77941aaf50 con 0x560d3b9abba0 2024-09-10T08:10:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f779d1dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77941ac3c0 con 0x560d3b9ab260 2024-09-10T08:10:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f779d1dc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f77941ab620 con 0x560d3b9b0c30 2024-09-10T08:10:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f779d1dc740 10 client.0 _ll_register_callbacks cb 0x7f777c031860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f778be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 0x560d3b9ba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f778be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 0x560d3b9ba850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58520/0 (socket says 172.21.15.2:58520) 2024-09-10T08:10:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f778be00640 1 -- 172.21.15.2:0/2228381408 learned_addr learned my addr 172.21.15.2:0/2228381408 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f7793400640 1 --2- 172.21.15.2:0/2228381408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f778be00640 1 -- 172.21.15.2:0/2228381408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9b0c30 msgr2=0x560d3b9bb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f778be00640 1 --2- 172.21.15.2:0/2228381408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9b0c30 0x560d3b9bb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f778be00640 1 -- 172.21.15.2:0/2228381408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 msgr2=0x560d3b9ba310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f778be00640 1 --2- 172.21.15.2:0/2228381408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9ba310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f778be00640 1 -- 172.21.15.2:0/2228381408 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d3b9bbdb0 con 0x560d3b9abba0 2024-09-10T08:10:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.312+0000 7f7793400640 1 --2- 172.21.15.2:0/2228381408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.313+0000 7f778be00640 1 --2- 172.21.15.2:0/2228381408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 0x560d3b9ba850 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f777c031440 tx=0x7f777c031ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3ed5615d5e5519 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.313+0000 7f7780c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.313+0000 7f7780c00640 1 -- 172.21.15.2:0/2228381408 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c062020 con 0x560d3b9abba0 2024-09-10T08:10:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.313+0000 7f7780c00640 1 -- 172.21.15.2:0/2228381408 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f777c004460 con 0x560d3b9abba0 2024-09-10T08:10:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.313+0000 7f779d1dc740 1 -- 172.21.15.2:0/2228381408 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f777c003dc0 con 0x560d3b9abba0 2024-09-10T08:10:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.313+0000 7f7780c00640 1 -- 172.21.15.2:0/2228381408 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f777c035050 con 0x560d3b9abba0 2024-09-10T08:10:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.313+0000 7f779d1dc740 10 client.15651 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.313+0000 7f779d1dc740 1 -- 172.21.15.2:0/2228381408 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d3ba2bd10 con 0x560d3b9abba0 2024-09-10T08:10:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.313+0000 7f7780c00640 1 -- 172.21.15.2:0/2228381408 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f777c003dc0 con 0x560d3b9abba0 2024-09-10T08:10:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.314+0000 7f7780c00640 1 -- 172.21.15.2:0/2228381408 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f777c054350 con 0x560d3b9abba0 2024-09-10T08:10:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.314+0000 7f779d1dc740 10 client.15651 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.314+0000 7f779d1dc740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.314+0000 7f779d1dc740 20 client.15651 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.314+0000 7f779d1dc740 1 --2- 172.21.15.2:0/2228381408 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d3ba2c740 0x560d3ba2eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.314+0000 7f779d1dc740 4 client.15651 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:10:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.314+0000 7f779d1dc740 1 -- 172.21.15.2:0/2228381408 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d3b917b80 con 0x560d3ba2c740 2024-09-10T08:10:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.314+0000 7f7793400640 1 --2- 172.21.15.2:0/2228381408 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d3ba2c740 0x560d3ba2eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.314+0000 7f7793400640 1 --2- 172.21.15.2:0/2228381408 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d3ba2c740 0x560d3ba2eb60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787fbcf932a21078 server_cookie=98d9cbb122331e24 in_seq=0 out_seq=0 2024-09-10T08:10:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.314+0000 7f7780c00640 10 client.15651 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.318+0000 7f7780c00640 1 -- 172.21.15.2:0/2228381408 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d3b917b80 con 0x560d3ba2c740 2024-09-10T08:10:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.318+0000 7f7780c00640 10 client.15651 handle_command_reply: tid=0 2024-09-10T08:10:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.318+0000 7f779d1dc740 1 client.15651 shutdown 2024-09-10T08:10:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.318+0000 7f779d1dc740 1 -- 172.21.15.2:0/2228381408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 msgr2=0x560d3b9ba850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.318+0000 7f779d1dc740 1 --2- 172.21.15.2:0/2228381408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 0x560d3b9ba850 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f777c031440 tx=0x7f777c031ab0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.318+0000 7f779d1dc740 1 -- 172.21.15.2:0/2228381408 shutdown_connections 2024-09-10T08:10:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.318+0000 7f779d1dc740 1 --2- 172.21.15.2:0/2228381408 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d3ba2c740 0x560d3ba2eb60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.318+0000 7f779d1dc740 1 --2- 172.21.15.2:0/2228381408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d3b9b0c30 0x560d3b9bb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.318+0000 7f779d1dc740 1 --2- 172.21.15.2:0/2228381408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d3b9abba0 0x560d3b9ba850 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.318+0000 7f779d1dc740 1 --2- 172.21.15.2:0/2228381408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d3b9ab260 0x560d3b9ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.318+0000 7f7780c00640 0 client.15651 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.318+0000 7f779d1dc740 1 -- 172.21.15.2:0/2228381408 >> 172.21.15.2:0/2228381408 conn(0x560d3b9a68d0 msgr2=0x560d3b9a6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.319+0000 7f779d1dc740 1 -- 172.21.15.2:0/2228381408 shutdown_connections 2024-09-10T08:10:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.319+0000 7f779d1dc740 1 -- 172.21.15.2:0/2228381408 wait complete. 2024-09-10T08:10:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.319+0000 7f779d1dc740 20 client.15651 trim_cache size 0 max 16384 2024-09-10T08:10:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:10:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:10:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:10:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:10:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.319+0000 7f7798c00640 1 -- 172.21.15.2:0/919092916 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f77780a2b80 msgr2=0x7f77780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.319+0000 7f7798c00640 1 --2- 172.21.15.2:0/919092916 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f77780a2b80 0x7f77780a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f777c005c40 tx=0x7f777c0315b0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.319+0000 7f7798c00640 1 -- 172.21.15.2:0/919092916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7794109cf0 msgr2=0x7f77941a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.319+0000 7f7798c00640 1 --2- 172.21.15.2:0/919092916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7794109cf0 0x7f77941a1550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f778c00da10 tx=0x7f778c00ded0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.319+0000 7f7798c00640 1 -- 172.21.15.2:0/919092916 shutdown_connections 2024-09-10T08:10:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.319+0000 7f7798c00640 1 --2- 172.21.15.2:0/919092916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7794109cf0 0x7f77941a1550 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.319+0000 7f7798c00640 1 --2- 172.21.15.2:0/919092916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7794074f60 0x7f779419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.319+0000 7f7798c00640 1 --2- 172.21.15.2:0/919092916 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f77780a2b80 0x7f77780a5040 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.319+0000 7f7798c00640 1 --2- 172.21.15.2:0/919092916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7794074620 0x7f7794199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.319+0000 7f7798c00640 1 -- 172.21.15.2:0/919092916 >> 172.21.15.2:0/919092916 conn(0x7f77940fd0d0 msgr2=0x7f7794105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.320+0000 7f7798c00640 1 -- 172.21.15.2:0/919092916 shutdown_connections 2024-09-10T08:10:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:02.320+0000 7f7798c00640 1 -- 172.21.15.2:0/919092916 wait complete. 2024-09-10T08:10:02.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.708+0000 7f808aa00640 1 -- 172.21.15.2:0/3461558594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8084109120 msgr2=0x7f8084110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.708+0000 7f808aa00640 1 --2- 172.21.15.2:0/3461558594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8084109120 0x7f8084110ca0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f807c009ca0 tx=0x7f807c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.708+0000 7f808aa00640 1 -- 172.21.15.2:0/3461558594 shutdown_connections 2024-09-10T08:10:02.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.709+0000 7f808aa00640 1 --2- 172.21.15.2:0/3461558594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8084109120 0x7f8084110ca0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.709+0000 7f808aa00640 1 --2- 172.21.15.2:0/3461558594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80841049a0 0x7f80841089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.709+0000 7f808aa00640 1 --2- 172.21.15.2:0/3461558594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8084103ff0 0x7f80841043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.709+0000 7f808aa00640 1 -- 172.21.15.2:0/3461558594 >> 172.21.15.2:0/3461558594 conn(0x7f80840ffa90 msgr2=0x7f8084101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:02.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.709+0000 7f808aa00640 1 -- 172.21.15.2:0/3461558594 shutdown_connections 2024-09-10T08:10:02.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.709+0000 7f808aa00640 1 -- 172.21.15.2:0/3461558594 wait complete. 2024-09-10T08:10:02.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.710+0000 7f808aa00640 1 Processor -- start 2024-09-10T08:10:02.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.710+0000 7f808aa00640 1 -- start start 2024-09-10T08:10:02.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.710+0000 7f808aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8084103ff0 0x7f808419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.710+0000 7f808aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80841049a0 0x7f808419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.710+0000 7f808aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8084109120 0x7f808419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.710+0000 7f808aa00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8084114180 con 0x7f80841049a0 2024-09-10T08:10:02.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.710+0000 7f808aa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8084114000 con 0x7f8084109120 2024-09-10T08:10:02.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.710+0000 7f808aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8084114300 con 0x7f8084103ff0 2024-09-10T08:10:02.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.710+0000 7f807be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80841049a0 0x7f808419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.711+0000 7f807be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80841049a0 0x7f808419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58530/0 (socket says 172.21.15.2:58530) 2024-09-10T08:10:02.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.711+0000 7f807be00640 1 -- 172.21.15.2:0/3580780759 learned_addr learned my addr 172.21.15.2:0/3580780759 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:02.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.711+0000 7f8088a00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8084109120 0x7f808419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.711+0000 7f8083e00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8084103ff0 0x7f808419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.711+0000 7f807be00640 1 -- 172.21.15.2:0/3580780759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8084103ff0 msgr2=0x7f808419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.711+0000 7f807be00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8084103ff0 0x7f808419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.711+0000 7f807be00640 1 -- 172.21.15.2:0/3580780759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8084109120 msgr2=0x7f808419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.711+0000 7f807be00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8084109120 0x7f808419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.711+0000 7f807be00640 1 -- 172.21.15.2:0/3580780759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808419f5d0 con 0x7f80841049a0 2024-09-10T08:10:02.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.711+0000 7f8088a00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8084109120 0x7f808419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:02.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.711+0000 7f807be00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80841049a0 0x7f808419ab60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f806c00d950 tx=0x7f806c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:02.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.712+0000 7f8081600640 1 -- 172.21.15.2:0/3580780759 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c017030 con 0x7f80841049a0 2024-09-10T08:10:02.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.712+0000 7f8081600640 1 -- 172.21.15.2:0/3580780759 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f806c004020 con 0x7f80841049a0 2024-09-10T08:10:02.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.712+0000 7f808aa00640 1 -- 172.21.15.2:0/3580780759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808419f8c0 con 0x7f80841049a0 2024-09-10T08:10:02.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.712+0000 7f8081600640 1 -- 172.21.15.2:0/3580780759 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c0047a0 con 0x7f80841049a0 2024-09-10T08:10:02.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.712+0000 7f808aa00640 1 -- 172.21.15.2:0/3580780759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80841a71a0 con 0x7f80841049a0 2024-09-10T08:10:02.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.715+0000 7f8081600640 1 -- 172.21.15.2:0/3580780759 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f806c0041d0 con 0x7f80841049a0 2024-09-10T08:10:02.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.715+0000 7f808d48f740 1 Processor -- start 2024-09-10T08:10:02.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.715+0000 7f808d48f740 1 -- start start 2024-09-10T08:10:02.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.716+0000 7f808d48f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ad6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.716+0000 7f808d48f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7ae0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.716+0000 7f808d48f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7ae3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.716+0000 7f808d48f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b0f7a3c3e0 con 0x55b0f7ae0e50 2024-09-10T08:10:02.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.716+0000 7f808d48f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b0f79e3d80 con 0x55b0f7ad6590 2024-09-10T08:10:02.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.716+0000 7f808d48f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b0f7a383a0 con 0x55b0f7ad5c70 2024-09-10T08:10:02.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.716+0000 7f8088a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ad6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.716+0000 7f8083e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7ae0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.716+0000 7f8088a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ad6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:41580/0 (socket says 172.21.15.2:41580) 2024-09-10T08:10:02.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.716+0000 7f8088a00640 1 -- 172.21.15.2:0/1628478928 learned_addr learned my addr 172.21.15.2:0/1628478928 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:02.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.716+0000 7f807be00640 1 --2- 172.21.15.2:0/1628478928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7ae3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.716+0000 7f8083e00640 1 -- 172.21.15.2:0/1628478928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad5c70 msgr2=0x55b0f7ad6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.717+0000 7f8083e00640 1 --2- 172.21.15.2:0/1628478928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ad6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.717+0000 7f8083e00640 1 -- 172.21.15.2:0/1628478928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 msgr2=0x55b0f7ae3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.717+0000 7f8083e00640 1 --2- 172.21.15.2:0/1628478928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7ae3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.717+0000 7f8083e00640 1 -- 172.21.15.2:0/1628478928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f7a02ae0 con 0x55b0f7ad6590 2024-09-10T08:10:02.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.717+0000 7f8081600640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f80680a2c00 0x7f80680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.717+0000 7f8083e00640 1 --2- 172.21.15.2:0/1628478928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7ae0910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8074009c20 tx=0x7f8074011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=183def22b34663a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:02.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.717+0000 7f8081600640 1 -- 172.21.15.2:0/3580780759 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f806c0cecf0 con 0x7f80841049a0 2024-09-10T08:10:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.717+0000 7f8079200640 1 -- 172.21.15.2:0/1628478928 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074004050 con 0x55b0f7ad6590 2024-09-10T08:10:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.717+0000 7f8079200640 1 -- 172.21.15.2:0/1628478928 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8074005500 con 0x55b0f7ad6590 2024-09-10T08:10:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.717+0000 7f8088a00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f80680a2c00 0x7f80680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.717+0000 7f8079200640 1 -- 172.21.15.2:0/1628478928 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80740035f0 con 0x55b0f7ad6590 2024-09-10T08:10:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.718+0000 7f808d48f740 1 -- 172.21.15.2:0/1628478928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 msgr2=0x55b0f7ae0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.718+0000 7f808d48f740 1 --2- 172.21.15.2:0/1628478928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7ae0910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8074009c20 tx=0x7f8074011810 comp rx=0 tx=0).stop 2024-09-10T08:10:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.718+0000 7f8088a00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f80680a2c00 0x7f80680a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f807c02aeb0 tx=0x7f807c005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.718+0000 7f808d48f740 1 -- 172.21.15.2:0/1628478928 shutdown_connections 2024-09-10T08:10:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.718+0000 7f808d48f740 1 --2- 172.21.15.2:0/1628478928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7ae3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.718+0000 7f808d48f740 1 --2- 172.21.15.2:0/1628478928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7ae0910 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.718+0000 7f808d48f740 1 --2- 172.21.15.2:0/1628478928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ad6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.718+0000 7f808d48f740 1 -- 172.21.15.2:0/1628478928 >> 172.21.15.2:0/1628478928 conn(0x55b0f7ad2200 msgr2=0x55b0f7ad25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:02.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.718+0000 7f808d48f740 1 -- 172.21.15.2:0/1628478928 shutdown_connections 2024-09-10T08:10:02.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.718+0000 7f808d48f740 1 -- 172.21.15.2:0/1628478928 wait complete. 2024-09-10T08:10:02.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.719+0000 7f808d48f740 1 -- start start 2024-09-10T08:10:02.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.719+0000 7f808d48f740 1 Processor -- start 2024-09-10T08:10:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.719+0000 7f808d48f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad5c70 0x55b0f7addd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.719+0000 7f808d48f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7ade2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f808d48f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7ade7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f808d48f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b0f7ae3a20 con 0x55b0f7ae0e50 2024-09-10T08:10:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f808d48f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b0f79f70f0 con 0x55b0f7ad6590 2024-09-10T08:10:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f808d48f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b0f7ae3ba0 con 0x55b0f7ad5c70 2024-09-10T08:10:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f808d48f740 10 client.0 _ll_register_callbacks cb 0x55b0f79debd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f8083e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad5c70 0x55b0f7addd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f8088a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7ade7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f8083e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad5c70 0x55b0f7addd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:41594/0 (socket says 172.21.15.2:41594) 2024-09-10T08:10:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f8083e00640 1 -- 172.21.15.2:0/1264774149 learned_addr learned my addr 172.21.15.2:0/1264774149 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f807be00640 1 --2- 172.21.15.2:0/1264774149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7ade2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f8088a00640 1 -- 172.21.15.2:0/1264774149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad5c70 msgr2=0x55b0f7addd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f8088a00640 1 --2- 172.21.15.2:0/1264774149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad5c70 0x55b0f7addd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f8088a00640 1 -- 172.21.15.2:0/1264774149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 msgr2=0x55b0f7ade2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f8088a00640 1 --2- 172.21.15.2:0/1264774149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7ade2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f8088a00640 1 -- 172.21.15.2:0/1264774149 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f77ad3c0 con 0x55b0f7ae0e50 2024-09-10T08:10:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.720+0000 7f807be00640 1 --2- 172.21.15.2:0/1264774149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7ade2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.721+0000 7f8088a00640 1 --2- 172.21.15.2:0/1264774149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7ade7e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f807c02ade0 tx=0x7f807c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c969204103f4440 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.721+0000 7f8071600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.721+0000 7f8071600640 1 -- 172.21.15.2:0/1264774149 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c0319a0 con 0x55b0f7ae0e50 2024-09-10T08:10:02.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.721+0000 7f808d48f740 1 -- 172.21.15.2:0/1264774149 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8074005500 con 0x55b0f7ae0e50 2024-09-10T08:10:02.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.721+0000 7f8071600640 1 -- 172.21.15.2:0/1264774149 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f807c004250 con 0x55b0f7ae0e50 2024-09-10T08:10:02.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.721+0000 7f8071600640 1 -- 172.21.15.2:0/1264774149 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f807c0098a0 con 0x55b0f7ae0e50 2024-09-10T08:10:02.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.721+0000 7f808d48f740 10 client.15663 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:02.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.721+0000 7f808d48f740 1 -- 172.21.15.2:0/1264774149 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0f7aefa90 con 0x55b0f7ae0e50 2024-09-10T08:10:02.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.722+0000 7f8071600640 1 -- 172.21.15.2:0/1264774149 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8074005500 con 0x55b0f7ae0e50 2024-09-10T08:10:02.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.722+0000 7f8071600640 1 -- 172.21.15.2:0/1264774149 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f807c052070 con 0x55b0f7ae0e50 2024-09-10T08:10:02.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.722+0000 7f808d48f740 10 client.15663 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.722+0000 7f808d48f740 10 client.15663 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:10:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.722+0000 7f808d48f740 20 client.15663 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.722+0000 7f808d48f740 1 --2- 172.21.15.2:0/1264774149 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b0f7af0f40 0x55b0f7af33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.722+0000 7f808d48f740 4 client.15663 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.722+0000 7f808d48f740 1 -- 172.21.15.2:0/1264774149 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0f7a49950 con 0x55b0f7af0f40 2024-09-10T08:10:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.723+0000 7f8083e00640 1 --2- 172.21.15.2:0/1264774149 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b0f7af0f40 0x55b0f7af33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.723+0000 7f8083e00640 1 --2- 172.21.15.2:0/1264774149 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b0f7af0f40 0x55b0f7af33b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=631202145a4683a5 server_cookie=d22cff2e4799f705 in_seq=0 out_seq=0 2024-09-10T08:10:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.723+0000 7f8071600640 10 client.15663 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.724+0000 7f8071600640 1 -- 172.21.15.2:0/1264774149 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0f7a49950 con 0x55b0f7af0f40 2024-09-10T08:10:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.724+0000 7f8071600640 10 client.15663 handle_command_reply: tid=0 2024-09-10T08:10:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.724+0000 7f808d48f740 1 client.15663 shutdown 2024-09-10T08:10:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.724+0000 7f808d48f740 1 -- 172.21.15.2:0/1264774149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 msgr2=0x55b0f7ade7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.724+0000 7f808d48f740 1 --2- 172.21.15.2:0/1264774149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7ade7e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f807c02ade0 tx=0x7f807c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.725+0000 7f808d48f740 1 -- 172.21.15.2:0/1264774149 shutdown_connections 2024-09-10T08:10:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.725+0000 7f808d48f740 1 --2- 172.21.15.2:0/1264774149 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b0f7af0f40 0x55b0f7af33b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.725+0000 7f808d48f740 1 --2- 172.21.15.2:0/1264774149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7ade7e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.725+0000 7f808d48f740 1 --2- 172.21.15.2:0/1264774149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7ade2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.725+0000 7f808d48f740 1 --2- 172.21.15.2:0/1264774149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad5c70 0x55b0f7addd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.725+0000 7f808d48f740 1 -- 172.21.15.2:0/1264774149 >> 172.21.15.2:0/1264774149 conn(0x55b0f7ad2200 msgr2=0x55b0f7ad7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.725+0000 7f8071600640 0 client.15663 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.725+0000 7f808d48f740 1 -- 172.21.15.2:0/1264774149 shutdown_connections 2024-09-10T08:10:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.725+0000 7f808d48f740 1 -- 172.21.15.2:0/1264774149 wait complete. 2024-09-10T08:10:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.725+0000 7f808d48f740 20 client.15663 trim_cache size 0 max 16384 2024-09-10T08:10:02.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.739+0000 7f808d48f740 1 Processor -- start 2024-09-10T08:10:02.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.739+0000 7f808d48f740 1 -- start start 2024-09-10T08:10:02.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.739+0000 7f808d48f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ad7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.739+0000 7f808d48f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7af8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.739+0000 7f808d48f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ae0e50 0x55b0f7af91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.739+0000 7f808d48f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f80841b3940 con 0x55b0f7ad5c70 2024-09-10T08:10:02.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.739+0000 7f808d48f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f80841b1210 con 0x55b0f7ad6590 2024-09-10T08:10:02.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.739+0000 7f808d48f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80841af060 con 0x55b0f7ae0e50 2024-09-10T08:10:02.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.739+0000 7f8083e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ad7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.739+0000 7f8083e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ad7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58560/0 (socket says 172.21.15.2:58560) 2024-09-10T08:10:02.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.739+0000 7f8083e00640 1 -- 172.21.15.2:0/2838004283 learned_addr learned my addr 172.21.15.2:0/2838004283 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:02.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.739+0000 7f8088a00640 1 --2- 172.21.15.2:0/2838004283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ae0e50 0x55b0f7af91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.739+0000 7f807be00640 1 --2- 172.21.15.2:0/2838004283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7af8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.740+0000 7f8083e00640 1 -- 172.21.15.2:0/2838004283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ae0e50 msgr2=0x55b0f7af91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.740+0000 7f8083e00640 1 --2- 172.21.15.2:0/2838004283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ae0e50 0x55b0f7af91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.740+0000 7f8083e00640 1 -- 172.21.15.2:0/2838004283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 msgr2=0x55b0f7af8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.740+0000 7f8083e00640 1 --2- 172.21.15.2:0/2838004283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7af8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.740+0000 7f8083e00640 1 -- 172.21.15.2:0/2838004283 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f7acd770 con 0x55b0f7ad5c70 2024-09-10T08:10:02.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.740+0000 7f8083e00640 1 --2- 172.21.15.2:0/2838004283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ad7dd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8074014040 tx=0x7f80740096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1411093604a3a1d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:02.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.740+0000 7f8073e00640 1 -- 172.21.15.2:0/2838004283 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074004240 con 0x55b0f7ad5c70 2024-09-10T08:10:02.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.740+0000 7f8073e00640 1 -- 172.21.15.2:0/2838004283 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80740043e0 con 0x55b0f7ad5c70 2024-09-10T08:10:02.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.740+0000 7f8073e00640 1 -- 172.21.15.2:0/2838004283 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074019720 con 0x55b0f7ad5c70 2024-09-10T08:10:02.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.741+0000 7f808d48f740 1 -- 172.21.15.2:0/2838004283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ad5c70 msgr2=0x55b0f7ad7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.741+0000 7f808d48f740 1 --2- 172.21.15.2:0/2838004283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ad7dd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8074014040 tx=0x7f80740096a0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.741+0000 7f808d48f740 1 -- 172.21.15.2:0/2838004283 shutdown_connections 2024-09-10T08:10:02.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.741+0000 7f808d48f740 1 --2- 172.21.15.2:0/2838004283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ae0e50 0x55b0f7af91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.741+0000 7f808d48f740 1 --2- 172.21.15.2:0/2838004283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad6590 0x55b0f7af8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.741+0000 7f808d48f740 1 --2- 172.21.15.2:0/2838004283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ad7dd0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.741+0000 7f808d48f740 1 -- 172.21.15.2:0/2838004283 >> 172.21.15.2:0/2838004283 conn(0x55b0f7ad2200 msgr2=0x55b0f7ad76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:02.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.741+0000 7f808d48f740 1 -- 172.21.15.2:0/2838004283 shutdown_connections 2024-09-10T08:10:02.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.741+0000 7f808d48f740 1 -- 172.21.15.2:0/2838004283 wait complete. 2024-09-10T08:10:02.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.741+0000 7f808d48f740 1 -- start start 2024-09-10T08:10:02.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.742+0000 7f808d48f740 1 Processor -- start 2024-09-10T08:10:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.742+0000 7f808d48f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ae5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.742+0000 7f808d48f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad6590 0x55b0f7ae5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.742+0000 7f808d48f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7b5ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.742+0000 7f808d48f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f80841af6d0 con 0x55b0f7ae0e50 2024-09-10T08:10:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.742+0000 7f808d48f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f80841b0b40 con 0x55b0f7ad5c70 2024-09-10T08:10:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.742+0000 7f808d48f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80841afda0 con 0x55b0f7ad6590 2024-09-10T08:10:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.742+0000 7f808d48f740 10 client.0 _ll_register_callbacks cb 0x7f807c02f4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.743+0000 7f8083e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ae5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.743+0000 7f8088a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7b5ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.743+0000 7f8083e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ae5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37064/0 (socket says 172.21.15.2:37064) 2024-09-10T08:10:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.743+0000 7f807be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad6590 0x55b0f7ae5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.743+0000 7f8083e00640 1 -- 172.21.15.2:0/816652157 learned_addr learned my addr 172.21.15.2:0/816652157 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.743+0000 7f8088a00640 1 -- 172.21.15.2:0/816652157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad6590 msgr2=0x55b0f7ae5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.743+0000 7f8088a00640 1 --2- 172.21.15.2:0/816652157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad6590 0x55b0f7ae5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.743+0000 7f8088a00640 1 -- 172.21.15.2:0/816652157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad5c70 msgr2=0x55b0f7ae5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.743+0000 7f8088a00640 1 --2- 172.21.15.2:0/816652157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ae5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.743+0000 7f8088a00640 1 -- 172.21.15.2:0/816652157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f7b5b190 con 0x55b0f7ae0e50 2024-09-10T08:10:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.743+0000 7f8088a00640 1 --2- 172.21.15.2:0/816652157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7b5ac50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f807c02ffa0 tx=0x7f807c030910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d980043d6133faa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.743+0000 7f8071600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.744+0000 7f8071600640 1 -- 172.21.15.2:0/816652157 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c030b80 con 0x55b0f7ae0e50 2024-09-10T08:10:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.744+0000 7f8071600640 1 -- 172.21.15.2:0/816652157 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f807c030d20 con 0x55b0f7ae0e50 2024-09-10T08:10:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.744+0000 7f808d48f740 1 -- 172.21.15.2:0/816652157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80740043e0 con 0x55b0f7ae0e50 2024-09-10T08:10:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.744+0000 7f8071600640 1 -- 172.21.15.2:0/816652157 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f807c06b050 con 0x55b0f7ae0e50 2024-09-10T08:10:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.744+0000 7f808d48f740 10 client.15669 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:02.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.744+0000 7f808d48f740 1 -- 172.21.15.2:0/816652157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0f7b5c430 con 0x55b0f7ae0e50 2024-09-10T08:10:02.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.744+0000 7f8071600640 1 -- 172.21.15.2:0/816652157 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80740043e0 con 0x55b0f7ae0e50 2024-09-10T08:10:02.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.744+0000 7f8071600640 1 -- 172.21.15.2:0/816652157 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f807c060020 con 0x55b0f7ae0e50 2024-09-10T08:10:02.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.744+0000 7f808d48f740 10 client.15669 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:02.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.744+0000 7f808d48f740 10 client.15669 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:10:02.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.745+0000 7f808d48f740 20 client.15669 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:02.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.745+0000 7f808d48f740 1 --2- 172.21.15.2:0/816652157 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b0f7b5cb50 0x55b0f7b5ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:02.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.745+0000 7f808d48f740 4 client.15669 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:10:02.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.745+0000 7f808d48f740 1 -- 172.21.15.2:0/816652157 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0f7a43e60 con 0x55b0f7b5cb50 2024-09-10T08:10:02.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.745+0000 7f8083e00640 1 --2- 172.21.15.2:0/816652157 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b0f7b5cb50 0x55b0f7b5ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:02.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f8083e00640 1 --2- 172.21.15.2:0/816652157 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b0f7b5cb50 0x55b0f7b5ef70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0d368ababaeb63b server_cookie=5c1c72c3aef4ddbd in_seq=0 out_seq=0 2024-09-10T08:10:02.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f8071600640 10 client.15669 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:02.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f8071600640 1 -- 172.21.15.2:0/816652157 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0f7a43e60 con 0x55b0f7b5cb50 2024-09-10T08:10:02.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f8071600640 10 client.15669 handle_command_reply: tid=0 2024-09-10T08:10:02.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f808d48f740 1 client.15669 shutdown 2024-09-10T08:10:02.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f808d48f740 1 -- 172.21.15.2:0/816652157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 msgr2=0x55b0f7b5ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f808d48f740 1 --2- 172.21.15.2:0/816652157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7b5ac50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f807c02ffa0 tx=0x7f807c030910 comp rx=0 tx=0).stop 2024-09-10T08:10:02.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f808d48f740 1 -- 172.21.15.2:0/816652157 shutdown_connections 2024-09-10T08:10:02.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f808d48f740 1 --2- 172.21.15.2:0/816652157 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b0f7b5cb50 0x55b0f7b5ef70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f808d48f740 1 --2- 172.21.15.2:0/816652157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0f7ae0e50 0x55b0f7b5ac50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f808d48f740 1 --2- 172.21.15.2:0/816652157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0f7ad6590 0x55b0f7ae5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f808d48f740 1 --2- 172.21.15.2:0/816652157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0f7ad5c70 0x55b0f7ae5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f808d48f740 1 -- 172.21.15.2:0/816652157 >> 172.21.15.2:0/816652157 conn(0x55b0f7ad2200 msgr2=0x55b0f7ade540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:02.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.746+0000 7f8071600640 0 client.15669 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:02.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.747+0000 7f808d48f740 1 -- 172.21.15.2:0/816652157 shutdown_connections 2024-09-10T08:10:02.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.747+0000 7f808d48f740 1 -- 172.21.15.2:0/816652157 wait complete. 2024-09-10T08:10:02.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.747+0000 7f808d48f740 20 client.15669 trim_cache size 0 max 16384 2024-09-10T08:10:02.750 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:10:02.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.747+0000 7f808aa00640 1 -- 172.21.15.2:0/3580780759 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f80680a2c00 msgr2=0x7f80680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.747+0000 7f808aa00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f80680a2c00 0x7f80680a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f807c02aeb0 tx=0x7f807c005d80 comp rx=0 tx=0).stop 2024-09-10T08:10:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.747+0000 7f808aa00640 1 -- 172.21.15.2:0/3580780759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80841049a0 msgr2=0x7f808419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.747+0000 7f808aa00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80841049a0 0x7f808419ab60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f806c00d950 tx=0x7f806c00de10 comp rx=0 tx=0).stop 2024-09-10T08:10:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.747+0000 7f808aa00640 1 -- 172.21.15.2:0/3580780759 shutdown_connections 2024-09-10T08:10:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.747+0000 7f808aa00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8084109120 0x7f808419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.747+0000 7f808aa00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80841049a0 0x7f808419ab60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.747+0000 7f808aa00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f80680a2c00 0x7f80680a50c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.748+0000 7f808aa00640 1 --2- 172.21.15.2:0/3580780759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8084103ff0 0x7f808419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.748+0000 7f808aa00640 1 -- 172.21.15.2:0/3580780759 >> 172.21.15.2:0/3580780759 conn(0x7f80840ffa90 msgr2=0x7f8084100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.748+0000 7f808aa00640 1 -- 172.21.15.2:0/3580780759 shutdown_connections 2024-09-10T08:10:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:02.748+0000 7f808aa00640 1 -- 172.21.15.2:0/3580780759 wait complete. 2024-09-10T08:10:03.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:02 smithi002 ceph-mon[30563]: pgmap v444: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:10:03.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:02 smithi155 ceph-mon[35405]: pgmap v444: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:10:03.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:02 smithi107 ceph-mon[35048]: pgmap v444: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:10:03.634 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:10:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:04.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.589+0000 7f1ccf800640 1 -- 172.21.15.2:0/812643237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc8103ff0 msgr2=0x7f1cc81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:04.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.589+0000 7f1ccf800640 1 --2- 172.21.15.2:0/812643237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc8103ff0 0x7f1cc81043d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009900 tx=0x7f1cbc02a410 comp rx=0 tx=0).stop 2024-09-10T08:10:04.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.589+0000 7f1ccf800640 1 -- 172.21.15.2:0/812643237 shutdown_connections 2024-09-10T08:10:04.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.589+0000 7f1ccf800640 1 --2- 172.21.15.2:0/812643237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cc8109120 0x7f1cc8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.589+0000 7f1ccf800640 1 --2- 172.21.15.2:0/812643237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cc81049a0 0x7f1cc81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.589+0000 7f1ccf800640 1 --2- 172.21.15.2:0/812643237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc8103ff0 0x7f1cc81043d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.589+0000 7f1ccf800640 1 -- 172.21.15.2:0/812643237 >> 172.21.15.2:0/812643237 conn(0x7f1cc80ffa90 msgr2=0x7f1cc8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:04.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.589+0000 7f1ccf800640 1 -- 172.21.15.2:0/812643237 shutdown_connections 2024-09-10T08:10:04.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.589+0000 7f1ccf800640 1 -- 172.21.15.2:0/812643237 wait complete. 2024-09-10T08:10:04.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.590+0000 7f1ccf800640 1 Processor -- start 2024-09-10T08:10:04.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.591+0000 7f1ccf800640 1 -- start start 2024-09-10T08:10:04.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.591+0000 7f1ccf800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc8103ff0 0x7f1cc819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.591+0000 7f1ccf800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cc81049a0 0x7f1cc819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.591+0000 7f1ccf800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cc8109120 0x7f1cc819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.591+0000 7f1ccf800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc8114170 con 0x7f1cc8103ff0 2024-09-10T08:10:04.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.591+0000 7f1ccf800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc8113ff0 con 0x7f1cc8109120 2024-09-10T08:10:04.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.591+0000 7f1ccf800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81142f0 con 0x7f1cc81049a0 2024-09-10T08:10:04.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.591+0000 7f1ccce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc8103ff0 0x7f1cc819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.591+0000 7f1ccce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc8103ff0 0x7f1cc819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58584/0 (socket says 172.21.15.2:58584) 2024-09-10T08:10:04.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.591+0000 7f1ccce00640 1 -- 172.21.15.2:0/884310379 learned_addr learned my addr 172.21.15.2:0/884310379 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:04.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.591+0000 7f1cc7e00640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cc81049a0 0x7f1cc819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.592+0000 7f1ccd800640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cc8109120 0x7f1cc819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.592+0000 7f1ccce00640 1 -- 172.21.15.2:0/884310379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cc81049a0 msgr2=0x7f1cc819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:04.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.592+0000 7f1ccce00640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cc81049a0 0x7f1cc819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.592+0000 7f1ccce00640 1 -- 172.21.15.2:0/884310379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cc8109120 msgr2=0x7f1cc819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:04.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.592+0000 7f1ccce00640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cc8109120 0x7f1cc819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.592+0000 7f1ccce00640 1 -- 172.21.15.2:0/884310379 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc819f620 con 0x7f1cc8103ff0 2024-09-10T08:10:04.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.592+0000 7f1ccd800640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cc8109120 0x7f1cc819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:04.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.592+0000 7f1ccce00640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc8103ff0 0x7f1cc819a670 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc004770 tx=0x7f1cbc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:04.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.592+0000 7f1cc5600640 1 -- 172.21.15.2:0/884310379 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc03f070 con 0x7f1cc8103ff0 2024-09-10T08:10:04.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.592+0000 7f1cc5600640 1 -- 172.21.15.2:0/884310379 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc031ca0 con 0x7f1cc8103ff0 2024-09-10T08:10:04.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.592+0000 7f1cc5600640 1 -- 172.21.15.2:0/884310379 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc035030 con 0x7f1cc8103ff0 2024-09-10T08:10:04.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.592+0000 7f1ccf800640 1 -- 172.21.15.2:0/884310379 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc819f8b0 con 0x7f1cc8103ff0 2024-09-10T08:10:04.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.593+0000 7f1ccf800640 1 -- 172.21.15.2:0/884310379 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc819fd40 con 0x7f1cc8103ff0 2024-09-10T08:10:04.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.595+0000 7f1cd2212740 1 Processor -- start 2024-09-10T08:10:04.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.596+0000 7f1cc5600640 1 -- 172.21.15.2:0/884310379 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1cbc042070 con 0x7f1cc8103ff0 2024-09-10T08:10:04.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.596+0000 7f1cd2212740 1 -- start start 2024-09-10T08:10:04.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.596+0000 7f1cd2212740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee5c70 0x55a2a6ee6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.596+0000 7f1cd2212740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee6590 0x55a2a6ef0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.596+0000 7f1cd2212740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ef0e50 0x55a2a6ef3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.596+0000 7f1cd2212740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a2a6e4c3e0 con 0x55a2a6ef0e50 2024-09-10T08:10:04.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.596+0000 7f1cd2212740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a2a6df3d80 con 0x55a2a6ee6590 2024-09-10T08:10:04.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.596+0000 7f1cd2212740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a2a6e483a0 con 0x55a2a6ee5c70 2024-09-10T08:10:04.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.596+0000 7f1ccce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee6590 0x55a2a6ef0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.597+0000 7f1cc7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ef0e50 0x55a2a6ef3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.597+0000 7f1ccd800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee5c70 0x55a2a6ee6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.597+0000 7f1ccce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee6590 0x55a2a6ef0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37090/0 (socket says 172.21.15.2:37090) 2024-09-10T08:10:04.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.597+0000 7f1ccce00640 1 -- 172.21.15.2:0/3865266064 learned_addr learned my addr 172.21.15.2:0/3865266064 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:04.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.597+0000 7f1ccd800640 1 -- 172.21.15.2:0/3865266064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee6590 msgr2=0x55a2a6ef0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:04.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.597+0000 7f1ccd800640 1 --2- 172.21.15.2:0/3865266064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee6590 0x55a2a6ef0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.597+0000 7f1ccd800640 1 -- 172.21.15.2:0/3865266064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ef0e50 msgr2=0x55a2a6ef3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:04.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.597+0000 7f1ccd800640 1 --2- 172.21.15.2:0/3865266064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ef0e50 0x55a2a6ef3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.597+0000 7f1ccd800640 1 -- 172.21.15.2:0/3865266064 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2a6e12ae0 con 0x55a2a6ee5c70 2024-09-10T08:10:04.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.597+0000 7f1ccd800640 1 --2- 172.21.15.2:0/3865266064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee5c70 0x55a2a6ee6050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009f10 tx=0x7f1cc00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad94ed3ee2769074 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:04.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.598+0000 7f1cc5600640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1cb00a2c00 0x7f1cb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.598+0000 7f1cba000640 1 -- 172.21.15.2:0/3865266064 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc001c680 con 0x55a2a6ee5c70 2024-09-10T08:10:04.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.598+0000 7f1cba000640 1 -- 172.21.15.2:0/3865266064 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc001cd00 con 0x55a2a6ee5c70 2024-09-10T08:10:04.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.598+0000 7f1cba000640 1 -- 172.21.15.2:0/3865266064 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc001a380 con 0x55a2a6ee5c70 2024-09-10T08:10:04.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.598+0000 7f1cc5600640 1 -- 172.21.15.2:0/884310379 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1cbc0efa50 con 0x7f1cc8103ff0 2024-09-10T08:10:04.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.598+0000 7f1ccce00640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1cb00a2c00 0x7f1cb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.598+0000 7f1cd2212740 1 -- 172.21.15.2:0/3865266064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee5c70 msgr2=0x55a2a6ee6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:04.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.598+0000 7f1cd2212740 1 --2- 172.21.15.2:0/3865266064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee5c70 0x55a2a6ee6050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009f10 tx=0x7f1cc00107a0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.598+0000 7f1ccce00640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1cb00a2c00 0x7f1cb00a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc031550 tx=0x7f1cbc0f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:04.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.599+0000 7f1cd2212740 1 -- 172.21.15.2:0/3865266064 shutdown_connections 2024-09-10T08:10:04.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.599+0000 7f1cd2212740 1 --2- 172.21.15.2:0/3865266064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ef0e50 0x55a2a6ef3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.599+0000 7f1cd2212740 1 --2- 172.21.15.2:0/3865266064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee6590 0x55a2a6ef0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.599+0000 7f1cd2212740 1 --2- 172.21.15.2:0/3865266064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee5c70 0x55a2a6ee6050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.599+0000 7f1cd2212740 1 -- 172.21.15.2:0/3865266064 >> 172.21.15.2:0/3865266064 conn(0x55a2a6ee2200 msgr2=0x55a2a6ee25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:04.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.599+0000 7f1cd2212740 1 -- 172.21.15.2:0/3865266064 shutdown_connections 2024-09-10T08:10:04.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.599+0000 7f1cd2212740 1 -- 172.21.15.2:0/3865266064 wait complete. 2024-09-10T08:10:04.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.599+0000 7f1cd2212740 1 -- start start 2024-09-10T08:10:04.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.599+0000 7f1cd2212740 1 Processor -- start 2024-09-10T08:10:04.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.600+0000 7f1cd2212740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee5c70 0x55a2a6eedd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.600+0000 7f1cd2212740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 0x55a2a6eee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.600+0000 7f1cd2212740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ef0e50 0x55a2a6eee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.600+0000 7f1cd2212740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a2a6ef3a20 con 0x55a2a6ee6590 2024-09-10T08:10:04.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.600+0000 7f1cd2212740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a2a6e070f0 con 0x55a2a6ee5c70 2024-09-10T08:10:04.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.600+0000 7f1cd2212740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a2a6ef3ba0 con 0x55a2a6ef0e50 2024-09-10T08:10:04.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.600+0000 7f1cd2212740 10 client.0 _ll_register_callbacks cb 0x55a2a6deebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:04.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.600+0000 7f1cc7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 0x55a2a6eee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.600+0000 7f1cc7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 0x55a2a6eee2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58592/0 (socket says 172.21.15.2:58592) 2024-09-10T08:10:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.600+0000 7f1cc7e00640 1 -- 172.21.15.2:0/1220979260 learned_addr learned my addr 172.21.15.2:0/1220979260 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.600+0000 7f1ccd800640 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee5c70 0x55a2a6eedd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.600+0000 7f1ccd800640 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ef0e50 0x55a2a6eee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.601+0000 7f1cc7e00640 1 -- 172.21.15.2:0/1220979260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ef0e50 msgr2=0x55a2a6eee7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.601+0000 7f1cc7e00640 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ef0e50 0x55a2a6eee7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.601+0000 7f1cc7e00640 1 -- 172.21.15.2:0/1220979260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee5c70 msgr2=0x55a2a6eedd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.601+0000 7f1cc7e00640 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee5c70 0x55a2a6eedd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.601+0000 7f1cc7e00640 1 -- 172.21.15.2:0/1220979260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2a6bbd3c0 con 0x55a2a6ee6590 2024-09-10T08:10:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.601+0000 7f1ccd800640 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee5c70 0x55a2a6eedd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.601+0000 7f1cc7e00640 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 0x55a2a6eee2a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4012700 tx=0x7f1cb4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d0cf7c86890a98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.601+0000 7f1caea00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.601+0000 7f1caea00640 1 -- 172.21.15.2:0/1220979260 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400f8c0 con 0x55a2a6ee6590 2024-09-10T08:10:04.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.601+0000 7f1caea00640 1 -- 172.21.15.2:0/1220979260 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1cb400fa60 con 0x55a2a6ee6590 2024-09-10T08:10:04.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.601+0000 7f1cd2212740 1 -- 172.21.15.2:0/1220979260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc001cd00 con 0x55a2a6ee6590 2024-09-10T08:10:04.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.602+0000 7f1caea00640 1 -- 172.21.15.2:0/1220979260 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1cb4025050 con 0x55a2a6ee6590 2024-09-10T08:10:04.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.602+0000 7f1cd2212740 10 client.15681 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:04.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.602+0000 7f1cd2212740 1 -- 172.21.15.2:0/1220979260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2a6effaf0 con 0x55a2a6ee6590 2024-09-10T08:10:04.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.602+0000 7f1caea00640 1 -- 172.21.15.2:0/1220979260 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc001cd00 con 0x55a2a6ee6590 2024-09-10T08:10:04.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.602+0000 7f1caea00640 1 -- 172.21.15.2:0/1220979260 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1cb4010430 con 0x55a2a6ee6590 2024-09-10T08:10:04.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.602+0000 7f1cd2212740 10 client.15681 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:04.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.602+0000 7f1cd2212740 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:04.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.602+0000 7f1cd2212740 20 client.15681 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:04.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.602+0000 7f1cd2212740 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a2a6f00fb0 0x55a2a6f033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.602+0000 7f1cd2212740 4 client.15681 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:04.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.602+0000 7f1cd2212740 1 -- 172.21.15.2:0/1220979260 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2a6e59950 con 0x55a2a6f00fb0 2024-09-10T08:10:04.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.603+0000 7f1ccd800640 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a2a6f00fb0 0x55a2a6f033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.603+0000 7f1ccd800640 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a2a6f00fb0 0x55a2a6f033a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f40c3dbd4f259dbf server_cookie=d75cf6f7d343b916 in_seq=0 out_seq=0 2024-09-10T08:10:04.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.603+0000 7f1caea00640 10 client.15681 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:04.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1caea00640 1 -- 172.21.15.2:0/1220979260 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2a6e59950 con 0x55a2a6f00fb0 2024-09-10T08:10:04.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1caea00640 10 client.15681 handle_command_reply: tid=0 2024-09-10T08:10:04.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1cd2212740 1 client.15681 shutdown 2024-09-10T08:10:04.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1cd2212740 1 -- 172.21.15.2:0/1220979260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 msgr2=0x55a2a6eee2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:04.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1cd2212740 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 0x55a2a6eee2a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4012700 tx=0x7f1cb4012bc0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1cd2212740 1 -- 172.21.15.2:0/1220979260 shutdown_connections 2024-09-10T08:10:04.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1cd2212740 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a2a6f00fb0 0x55a2a6f033a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1cd2212740 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ef0e50 0x55a2a6eee7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1cd2212740 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 0x55a2a6eee2a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1cd2212740 1 --2- 172.21.15.2:0/1220979260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee5c70 0x55a2a6eedd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1cd2212740 1 -- 172.21.15.2:0/1220979260 >> 172.21.15.2:0/1220979260 conn(0x55a2a6ee2200 msgr2=0x55a2a6ee1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:04.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1caea00640 0 client.15681 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:04.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1cd2212740 1 -- 172.21.15.2:0/1220979260 shutdown_connections 2024-09-10T08:10:04.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1cd2212740 1 -- 172.21.15.2:0/1220979260 wait complete. 2024-09-10T08:10:04.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.604+0000 7f1cd2212740 20 client.15681 trim_cache size 0 max 16384 2024-09-10T08:10:04.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.618+0000 7f1cd2212740 1 Processor -- start 2024-09-10T08:10:04.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.618+0000 7f1cd2212740 1 -- start start 2024-09-10T08:10:04.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.618+0000 7f1cd2212740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee5c70 0x55a2a6ee86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.618+0000 7f1cd2212740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 0x55a2a6eefcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.618+0000 7f1cd2212740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ef0e50 0x55a2a6f08cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.618+0000 7f1cd2212740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81b38e0 con 0x55a2a6ee6590 2024-09-10T08:10:04.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.618+0000 7f1cd2212740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81b11b0 con 0x55a2a6ef0e50 2024-09-10T08:10:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.618+0000 7f1cd2212740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81af000 con 0x55a2a6ee5c70 2024-09-10T08:10:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.619+0000 7f1cc7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 0x55a2a6eefcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.619+0000 7f1cc7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 0x55a2a6eefcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58598/0 (socket says 172.21.15.2:58598) 2024-09-10T08:10:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.619+0000 7f1cc7e00640 1 -- 172.21.15.2:0/636183720 learned_addr learned my addr 172.21.15.2:0/636183720 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.619+0000 7f1ccd800640 1 --2- 172.21.15.2:0/636183720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee5c70 0x55a2a6ee86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.619+0000 7f1ccd800640 1 --2- 172.21.15.2:0/636183720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ef0e50 0x55a2a6f08cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.619+0000 7f1cc7e00640 1 -- 172.21.15.2:0/636183720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee5c70 msgr2=0x55a2a6ee86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.619+0000 7f1cc7e00640 1 --2- 172.21.15.2:0/636183720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee5c70 0x55a2a6ee86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.619+0000 7f1cc7e00640 1 -- 172.21.15.2:0/636183720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ef0e50 msgr2=0x55a2a6f08cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.619+0000 7f1cc7e00640 1 --2- 172.21.15.2:0/636183720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ef0e50 0x55a2a6f08cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.619+0000 7f1cc7e00640 1 -- 172.21.15.2:0/636183720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2a6edd770 con 0x55a2a6ee6590 2024-09-10T08:10:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.619+0000 7f1ccd800640 1 --2- 172.21.15.2:0/636183720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ef0e50 0x55a2a6f08cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:10:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.619+0000 7f1cc7e00640 1 --2- 172.21.15.2:0/636183720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 0x55a2a6eefcd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40109c0 tx=0x7f1cb40096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422f8d3d7706551e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:04.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.620+0000 7f1cb9600640 1 -- 172.21.15.2:0/636183720 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40104f0 con 0x55a2a6ee6590 2024-09-10T08:10:04.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.620+0000 7f1cb9600640 1 -- 172.21.15.2:0/636183720 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb40055b0 con 0x55a2a6ee6590 2024-09-10T08:10:04.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.620+0000 7f1cb9600640 1 -- 172.21.15.2:0/636183720 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400fa70 con 0x55a2a6ee6590 2024-09-10T08:10:04.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.620+0000 7f1cd2212740 1 -- 172.21.15.2:0/636183720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 msgr2=0x55a2a6eefcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:04.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.620+0000 7f1cd2212740 1 --2- 172.21.15.2:0/636183720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 0x55a2a6eefcd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40109c0 tx=0x7f1cb40096c0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.620+0000 7f1cd2212740 1 -- 172.21.15.2:0/636183720 shutdown_connections 2024-09-10T08:10:04.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.620+0000 7f1cd2212740 1 --2- 172.21.15.2:0/636183720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ef0e50 0x55a2a6f08cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.620+0000 7f1cd2212740 1 --2- 172.21.15.2:0/636183720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ee6590 0x55a2a6eefcd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.620+0000 7f1cd2212740 1 --2- 172.21.15.2:0/636183720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee5c70 0x55a2a6ee86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.620+0000 7f1cd2212740 1 -- 172.21.15.2:0/636183720 >> 172.21.15.2:0/636183720 conn(0x55a2a6ee2200 msgr2=0x55a2a6eef5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:04.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.620+0000 7f1cd2212740 1 -- 172.21.15.2:0/636183720 shutdown_connections 2024-09-10T08:10:04.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.620+0000 7f1cd2212740 1 -- 172.21.15.2:0/636183720 wait complete. 2024-09-10T08:10:04.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.621+0000 7f1cd2212740 1 -- start start 2024-09-10T08:10:04.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.621+0000 7f1cd2212740 1 Processor -- start 2024-09-10T08:10:04.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.621+0000 7f1cd2212740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee5c70 0x55a2a6ef94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.621+0000 7f1cd2212740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee6590 0x55a2a6ef4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.621+0000 7f1cd2212740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ef0e50 0x55a2a6ef4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.621+0000 7f1cd2212740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81af670 con 0x55a2a6ef0e50 2024-09-10T08:10:04.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.621+0000 7f1cd2212740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81b0ae0 con 0x55a2a6ee5c70 2024-09-10T08:10:04.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.621+0000 7f1cd2212740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81afd40 con 0x55a2a6ee6590 2024-09-10T08:10:04.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.621+0000 7f1cd2212740 10 client.0 _ll_register_callbacks cb 0x7f1cb4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:04.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.622+0000 7f1cc7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee6590 0x55a2a6ef4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.622+0000 7f1ccd800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee5c70 0x55a2a6ef94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.622+0000 7f1cc7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee6590 0x55a2a6ef4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:41696/0 (socket says 172.21.15.2:41696) 2024-09-10T08:10:04.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.622+0000 7f1cc7e00640 1 -- 172.21.15.2:0/3133219096 learned_addr learned my addr 172.21.15.2:0/3133219096 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:04.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.622+0000 7f1ccd800640 1 --2- 172.21.15.2:0/3133219096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ef0e50 0x55a2a6ef4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.622+0000 7f1cc7e00640 1 -- 172.21.15.2:0/3133219096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee5c70 msgr2=0x55a2a6ef94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:04.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.622+0000 7f1cc7e00640 1 --2- 172.21.15.2:0/3133219096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee5c70 0x55a2a6ef94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.622+0000 7f1cc7e00640 1 -- 172.21.15.2:0/3133219096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ef0e50 msgr2=0x55a2a6ef4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:04.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.622+0000 7f1cc7e00640 1 --2- 172.21.15.2:0/3133219096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ef0e50 0x55a2a6ef4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.622+0000 7f1cc7e00640 1 -- 172.21.15.2:0/3133219096 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2a6ef5570 con 0x55a2a6ee6590 2024-09-10T08:10:04.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.622+0000 7f1cc7e00640 1 --2- 172.21.15.2:0/3133219096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee6590 0x55a2a6ef4800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4012700 tx=0x7f1cb4005fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3565f1779a08ad1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:04.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.622+0000 7f1caea00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:10:04.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.623+0000 7f1caea00640 1 -- 172.21.15.2:0/3133219096 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb402f070 con 0x55a2a6ee6590 2024-09-10T08:10:04.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.623+0000 7f1caea00640 1 -- 172.21.15.2:0/3133219096 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1cb4022070 con 0x55a2a6ee6590 2024-09-10T08:10:04.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.623+0000 7f1cd2212740 1 -- 172.21.15.2:0/3133219096 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb40055b0 con 0x55a2a6ee6590 2024-09-10T08:10:04.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.623+0000 7f1caea00640 1 -- 172.21.15.2:0/3133219096 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1cb403b050 con 0x55a2a6ee6590 2024-09-10T08:10:04.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.623+0000 7f1cd2212740 10 client.25702 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:04.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.623+0000 7f1cd2212740 1 -- 172.21.15.2:0/3133219096 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2a6f6bb80 con 0x55a2a6ee6590 2024-09-10T08:10:04.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.624+0000 7f1caea00640 1 -- 172.21.15.2:0/3133219096 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb40055b0 con 0x55a2a6ee6590 2024-09-10T08:10:04.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.624+0000 7f1caea00640 1 -- 172.21.15.2:0/3133219096 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1cb4003710 con 0x55a2a6ee6590 2024-09-10T08:10:04.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.624+0000 7f1cd2212740 10 client.25702 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:04.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.624+0000 7f1cd2212740 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:04.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.624+0000 7f1cd2212740 20 client.25702 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:04.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.624+0000 7f1cd2212740 1 --2- 172.21.15.2:0/3133219096 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a2a6f6c130 0x55a2a6f6e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:04.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.624+0000 7f1cd2212740 4 client.25702 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:10:04.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.624+0000 7f1cd2212740 1 -- 172.21.15.2:0/3133219096 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2a6e53e60 con 0x55a2a6f6c130 2024-09-10T08:10:04.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.624+0000 7f1ccd800640 1 --2- 172.21.15.2:0/3133219096 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a2a6f6c130 0x55a2a6f6e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:04.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1ccd800640 1 --2- 172.21.15.2:0/3133219096 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a2a6f6c130 0x55a2a6f6e520 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb4a3870fe59e726 server_cookie=d5551b6837a3d106 in_seq=0 out_seq=0 2024-09-10T08:10:04.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1caea00640 10 client.25702 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1caea00640 1 -- 172.21.15.2:0/3133219096 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2a6e53e60 con 0x55a2a6f6c130 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1caea00640 10 client.25702 handle_command_reply: tid=0 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1cd2212740 1 client.25702 shutdown 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1cd2212740 1 -- 172.21.15.2:0/3133219096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee6590 msgr2=0x55a2a6ef4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1cd2212740 1 --2- 172.21.15.2:0/3133219096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee6590 0x55a2a6ef4800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4012700 tx=0x7f1cb4005fc0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1cd2212740 1 -- 172.21.15.2:0/3133219096 shutdown_connections 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1cd2212740 1 --2- 172.21.15.2:0/3133219096 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a2a6f6c130 0x55a2a6f6e520 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1cd2212740 1 --2- 172.21.15.2:0/3133219096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2a6ef0e50 0x55a2a6ef4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1cd2212740 1 --2- 172.21.15.2:0/3133219096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2a6ee6590 0x55a2a6ef4800 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1caea00640 0 client.25702 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1cd2212740 1 --2- 172.21.15.2:0/3133219096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2a6ee5c70 0x55a2a6ef94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.625+0000 7f1cd2212740 1 -- 172.21.15.2:0/3133219096 >> 172.21.15.2:0/3133219096 conn(0x55a2a6ee2200 msgr2=0x55a2a6ee1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.626+0000 7f1cd2212740 1 -- 172.21.15.2:0/3133219096 shutdown_connections 2024-09-10T08:10:04.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.626+0000 7f1cd2212740 1 -- 172.21.15.2:0/3133219096 wait complete. 2024-09-10T08:10:04.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.626+0000 7f1cd2212740 20 client.25702 trim_cache size 0 max 16384 2024-09-10T08:10:04.648 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:10:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.626+0000 7f1ccf800640 1 -- 172.21.15.2:0/884310379 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1cb00a2c00 msgr2=0x7f1cb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.626+0000 7f1ccf800640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1cb00a2c00 0x7f1cb00a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc031550 tx=0x7f1cbc0f5530 comp rx=0 tx=0).stop 2024-09-10T08:10:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.626+0000 7f1ccf800640 1 -- 172.21.15.2:0/884310379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc8103ff0 msgr2=0x7f1cc819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.626+0000 7f1ccf800640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc8103ff0 0x7f1cc819a670 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc004770 tx=0x7f1cbc003c10 comp rx=0 tx=0).stop 2024-09-10T08:10:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.626+0000 7f1ccf800640 1 -- 172.21.15.2:0/884310379 shutdown_connections 2024-09-10T08:10:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.626+0000 7f1ccf800640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cc8109120 0x7f1cc819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.626+0000 7f1ccf800640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1cb00a2c00 0x7f1cb00a50c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.626+0000 7f1ccf800640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cc81049a0 0x7f1cc819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.626+0000 7f1ccf800640 1 --2- 172.21.15.2:0/884310379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc8103ff0 0x7f1cc819a670 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.626+0000 7f1ccf800640 1 -- 172.21.15.2:0/884310379 >> 172.21.15.2:0/884310379 conn(0x7f1cc80ffa90 msgr2=0x7f1cc8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.627+0000 7f1ccf800640 1 -- 172.21.15.2:0/884310379 shutdown_connections 2024-09-10T08:10:04.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:04.627+0000 7f1ccf800640 1 -- 172.21.15.2:0/884310379 wait complete. 2024-09-10T08:10:05.032 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:04 smithi002 ceph-mon[30563]: pgmap v445: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T08:10:05.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:04 smithi155 ceph-mon[35405]: pgmap v445: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T08:10:05.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:04 smithi107 ceph-mon[35048]: pgmap v445: 129 pgs: 129 active+clean; 675 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T08:10:06.461 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:10:06.462 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:10:06.855 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:07.132 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:06 smithi002 ceph-mon[30563]: pgmap v446: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:10:07.159 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:06 smithi155 ceph-mon[35405]: pgmap v446: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:10:07.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:06 smithi107 ceph-mon[35048]: pgmap v446: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:10:07.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.460+0000 7f530d000640 1 -- 172.21.15.2:0/2698035106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53081049a0 msgr2=0x7f53081089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:07.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.460+0000 7f530d000640 1 --2- 172.21.15.2:0/2698035106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53081049a0 0x7f53081089f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009900 tx=0x7f52f002a410 comp rx=0 tx=0).stop 2024-09-10T08:10:07.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.460+0000 7f530d000640 1 -- 172.21.15.2:0/2698035106 shutdown_connections 2024-09-10T08:10:07.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.460+0000 7f530d000640 1 --2- 172.21.15.2:0/2698035106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5308109120 0x7f5308110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.460+0000 7f530d000640 1 --2- 172.21.15.2:0/2698035106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53081049a0 0x7f53081089f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.460+0000 7f530d000640 1 --2- 172.21.15.2:0/2698035106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5308103ff0 0x7f53081043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.460+0000 7f530d000640 1 -- 172.21.15.2:0/2698035106 >> 172.21.15.2:0/2698035106 conn(0x7f53080ffa90 msgr2=0x7f5308101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:07.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.460+0000 7f530d000640 1 -- 172.21.15.2:0/2698035106 shutdown_connections 2024-09-10T08:10:07.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.460+0000 7f530d000640 1 -- 172.21.15.2:0/2698035106 wait complete. 2024-09-10T08:10:07.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.461+0000 7f530d000640 1 Processor -- start 2024-09-10T08:10:07.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.462+0000 7f530d000640 1 -- start start 2024-09-10T08:10:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.462+0000 7f530d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5308103ff0 0x7f530819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.462+0000 7f530d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53081049a0 0x7f530819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.462+0000 7f530d000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5308109120 0x7f530819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.462+0000 7f530d000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5308114170 con 0x7f5308109120 2024-09-10T08:10:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.462+0000 7f530d000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5308113ff0 con 0x7f53081049a0 2024-09-10T08:10:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.462+0000 7f530d000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53081142f0 con 0x7f5308103ff0 2024-09-10T08:10:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.462+0000 7f5306800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5308109120 0x7f530819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.462+0000 7f5306800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5308109120 0x7f530819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35732/0 (socket says 172.21.15.2:35732) 2024-09-10T08:10:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.462+0000 7f5306800640 1 -- 172.21.15.2:0/490830579 learned_addr learned my addr 172.21.15.2:0/490830579 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.462+0000 7f52fd400640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53081049a0 0x7f530819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.462+0000 7f5305e00640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5308103ff0 0x7f530819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.463+0000 7f5306800640 1 -- 172.21.15.2:0/490830579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5308103ff0 msgr2=0x7f530819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.463+0000 7f5306800640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5308103ff0 0x7f530819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.463+0000 7f5306800640 1 -- 172.21.15.2:0/490830579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53081049a0 msgr2=0x7f530819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.463+0000 7f5306800640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53081049a0 0x7f530819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.463+0000 7f5306800640 1 -- 172.21.15.2:0/490830579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530819f600 con 0x7f5308109120 2024-09-10T08:10:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.463+0000 7f5305e00640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5308103ff0 0x7f530819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.463+0000 7f5306800640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5308109120 0x7f530819ef20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f530000eaa0 tx=0x7f530000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.463+0000 7f52ff400640 1 -- 172.21.15.2:0/490830579 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530000cdb0 con 0x7f5308109120 2024-09-10T08:10:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.464+0000 7f52ff400640 1 -- 172.21.15.2:0/490830579 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5300005020 con 0x7f5308109120 2024-09-10T08:10:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.464+0000 7f530d000640 1 -- 172.21.15.2:0/490830579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530819f8f0 con 0x7f5308109120 2024-09-10T08:10:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.464+0000 7f52ff400640 1 -- 172.21.15.2:0/490830579 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53000116b0 con 0x7f5308109120 2024-09-10T08:10:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.464+0000 7f530d000640 1 -- 172.21.15.2:0/490830579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53081a71d0 con 0x7f5308109120 2024-09-10T08:10:07.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.467+0000 7f52ff400640 1 -- 172.21.15.2:0/490830579 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5300005560 con 0x7f5308109120 2024-09-10T08:10:07.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.467+0000 7f530fa50740 1 Processor -- start 2024-09-10T08:10:07.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.467+0000 7f530fa50740 1 -- start start 2024-09-10T08:10:07.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.467+0000 7f530fa50740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d5ac0 0x560ba52d5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f530fa50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d63e0 0x560ba52e0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f530fa50740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52e0ca0 0x560ba52e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f530fa50740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560ba523c290 con 0x560ba52d5ac0 2024-09-10T08:10:07.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f530fa50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560ba51e3b40 con 0x560ba52d63e0 2024-09-10T08:10:07.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f530fa50740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560ba5238250 con 0x560ba52e0ca0 2024-09-10T08:10:07.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f5305e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d63e0 0x560ba52e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f5305e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d63e0 0x560ba52e0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58820/0 (socket says 172.21.15.2:58820) 2024-09-10T08:10:07.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f52fd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d5ac0 0x560ba52d5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f5305e00640 1 -- 172.21.15.2:0/3004474114 learned_addr learned my addr 172.21.15.2:0/3004474114 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:07.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f52fd400640 1 --2- 172.21.15.2:0/3004474114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52e0ca0 0x560ba52e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f5305e00640 1 -- 172.21.15.2:0/3004474114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52e0ca0 msgr2=0x560ba52e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:07.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f5305e00640 1 --2- 172.21.15.2:0/3004474114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52e0ca0 0x560ba52e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f5305e00640 1 -- 172.21.15.2:0/3004474114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d5ac0 msgr2=0x560ba52d5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:07.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f5305e00640 1 --2- 172.21.15.2:0/3004474114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d5ac0 0x560ba52d5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f5305e00640 1 -- 172.21.15.2:0/3004474114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ba4fad6a0 con 0x560ba52d63e0 2024-09-10T08:10:07.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.468+0000 7f52fd400640 1 --2- 172.21.15.2:0/3004474114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52e0ca0 0x560ba52e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:10:07.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.469+0000 7f52ff400640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52ec0a2c00 0x7f52ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.469+0000 7f5305e00640 1 --2- 172.21.15.2:0/3004474114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d63e0 0x560ba52e0760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f52f80115f0 tx=0x7f52f8011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7df759d7fc7b49e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:07.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.469+0000 7f52ff400640 1 -- 172.21.15.2:0/490830579 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f53000cf560 con 0x7f5308109120 2024-09-10T08:10:07.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.469+0000 7f52f7400640 1 -- 172.21.15.2:0/3004474114 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f801e070 con 0x560ba52d63e0 2024-09-10T08:10:07.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.469+0000 7f52fd400640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52ec0a2c00 0x7f52ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.469+0000 7f52f7400640 1 -- 172.21.15.2:0/3004474114 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f800fb20 con 0x560ba52d63e0 2024-09-10T08:10:07.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.469+0000 7f52f7400640 1 -- 172.21.15.2:0/3004474114 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8019c40 con 0x560ba52d63e0 2024-09-10T08:10:07.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.470+0000 7f530fa50740 1 -- 172.21.15.2:0/3004474114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d63e0 msgr2=0x560ba52e0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:07.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.470+0000 7f530fa50740 1 --2- 172.21.15.2:0/3004474114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d63e0 0x560ba52e0760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f52f80115f0 tx=0x7f52f8011ab0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.470+0000 7f52fd400640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52ec0a2c00 0x7f52ec0a50c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f52f0000c00 tx=0x7f52f0005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:07.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.470+0000 7f530fa50740 1 -- 172.21.15.2:0/3004474114 shutdown_connections 2024-09-10T08:10:07.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.470+0000 7f530fa50740 1 --2- 172.21.15.2:0/3004474114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52e0ca0 0x560ba52e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.470+0000 7f530fa50740 1 --2- 172.21.15.2:0/3004474114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d63e0 0x560ba52e0760 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.470+0000 7f530fa50740 1 --2- 172.21.15.2:0/3004474114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d5ac0 0x560ba52d5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.470+0000 7f530fa50740 1 -- 172.21.15.2:0/3004474114 >> 172.21.15.2:0/3004474114 conn(0x560ba52d2050 msgr2=0x560ba52d2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:07.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.470+0000 7f530fa50740 1 -- 172.21.15.2:0/3004474114 shutdown_connections 2024-09-10T08:10:07.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.470+0000 7f530fa50740 1 -- 172.21.15.2:0/3004474114 wait complete. 2024-09-10T08:10:07.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.471+0000 7f530fa50740 1 -- start start 2024-09-10T08:10:07.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.471+0000 7f530fa50740 1 Processor -- start 2024-09-10T08:10:07.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.471+0000 7f530fa50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 0x560ba52e0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.471+0000 7f530fa50740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d63e0 0x560ba52dd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.471+0000 7f530fa50740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52e0ca0 0x560ba52ddf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.471+0000 7f530fa50740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560ba52e3910 con 0x560ba52e0ca0 2024-09-10T08:10:07.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.471+0000 7f530fa50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560ba51f6fa0 con 0x560ba52d5ac0 2024-09-10T08:10:07.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.471+0000 7f530fa50740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560ba52e3a90 con 0x560ba52d63e0 2024-09-10T08:10:07.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.471+0000 7f530fa50740 10 client.0 _ll_register_callbacks cb 0x560ba4f4aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:07.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.472+0000 7f5305e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 0x560ba52e0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.472+0000 7f5305e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 0x560ba52e0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58826/0 (socket says 172.21.15.2:58826) 2024-09-10T08:10:07.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.472+0000 7f52fd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d63e0 0x560ba52dd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.472+0000 7f5305e00640 1 -- 172.21.15.2:0/3583701446 learned_addr learned my addr 172.21.15.2:0/3583701446 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:07.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.472+0000 7f5306800640 1 --2- 172.21.15.2:0/3583701446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52e0ca0 0x560ba52ddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.472+0000 7f5305e00640 1 -- 172.21.15.2:0/3583701446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d63e0 msgr2=0x560ba52dd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:07.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.472+0000 7f5305e00640 1 --2- 172.21.15.2:0/3583701446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d63e0 0x560ba52dd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.472+0000 7f5305e00640 1 -- 172.21.15.2:0/3583701446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52e0ca0 msgr2=0x560ba52ddf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:07.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.472+0000 7f5305e00640 1 --2- 172.21.15.2:0/3583701446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52e0ca0 0x560ba52ddf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.472+0000 7f5305e00640 1 -- 172.21.15.2:0/3583701446 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ba52de760 con 0x560ba52d5ac0 2024-09-10T08:10:07.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.472+0000 7f5306800640 1 --2- 172.21.15.2:0/3583701446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52e0ca0 0x560ba52ddf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:07.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.473+0000 7f5305e00640 1 --2- 172.21.15.2:0/3583701446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 0x560ba52e0690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f52f801c040 tx=0x7f52f800f870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a8bec022639983c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:07.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.473+0000 7f52ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:07.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.473+0000 7f52ebe00640 1 -- 172.21.15.2:0/3583701446 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f801e040 con 0x560ba52d5ac0 2024-09-10T08:10:07.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.473+0000 7f530fa50740 1 -- 172.21.15.2:0/3583701446 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f800fb20 con 0x560ba52d5ac0 2024-09-10T08:10:07.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.473+0000 7f52ebe00640 1 -- 172.21.15.2:0/3583701446 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f52f80199c0 con 0x560ba52d5ac0 2024-09-10T08:10:07.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.473+0000 7f52ebe00640 1 -- 172.21.15.2:0/3583701446 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f52f802f050 con 0x560ba52d5ac0 2024-09-10T08:10:07.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.473+0000 7f530fa50740 10 client.25733 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:07.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.473+0000 7f530fa50740 1 -- 172.21.15.2:0/3583701446 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ba52ef840 con 0x560ba52d5ac0 2024-09-10T08:10:07.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.474+0000 7f52ebe00640 1 -- 172.21.15.2:0/3583701446 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f800fb20 con 0x560ba52d5ac0 2024-09-10T08:10:07.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.474+0000 7f52ebe00640 1 -- 172.21.15.2:0/3583701446 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f52f80035f0 con 0x560ba52d5ac0 2024-09-10T08:10:07.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.474+0000 7f530fa50740 10 client.25733 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:07.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.474+0000 7f530fa50740 10 client.25733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:07.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.474+0000 7f530fa50740 20 client.25733 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:07.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.474+0000 7f530fa50740 1 --2- 172.21.15.2:0/3583701446 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560ba52f0c50 0x560ba52f3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.474+0000 7f530fa50740 4 client.25733 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:07.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.474+0000 7f530fa50740 1 -- 172.21.15.2:0/3583701446 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ba5249800 con 0x560ba52f0c50 2024-09-10T08:10:07.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.474+0000 7f52fd400640 1 --2- 172.21.15.2:0/3583701446 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560ba52f0c50 0x560ba52f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.475+0000 7f52fd400640 1 --2- 172.21.15.2:0/3583701446 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560ba52f0c50 0x560ba52f3040 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ad6a777b9c044e0 server_cookie=7370c717947c51f3 in_seq=0 out_seq=0 2024-09-10T08:10:07.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.475+0000 7f52ebe00640 10 client.25733 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:07.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f52ebe00640 1 -- 172.21.15.2:0/3583701446 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ba5249800 con 0x560ba52f0c50 2024-09-10T08:10:07.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f52ebe00640 10 client.25733 handle_command_reply: tid=0 2024-09-10T08:10:07.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f530fa50740 1 client.25733 shutdown 2024-09-10T08:10:07.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f530fa50740 1 -- 172.21.15.2:0/3583701446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 msgr2=0x560ba52e0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:07.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f530fa50740 1 --2- 172.21.15.2:0/3583701446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 0x560ba52e0690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f52f801c040 tx=0x7f52f800f870 comp rx=0 tx=0).stop 2024-09-10T08:10:07.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f530fa50740 1 -- 172.21.15.2:0/3583701446 shutdown_connections 2024-09-10T08:10:07.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f530fa50740 1 --2- 172.21.15.2:0/3583701446 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560ba52f0c50 0x560ba52f3040 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f530fa50740 1 --2- 172.21.15.2:0/3583701446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52e0ca0 0x560ba52ddf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f530fa50740 1 --2- 172.21.15.2:0/3583701446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d63e0 0x560ba52dd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f530fa50740 1 --2- 172.21.15.2:0/3583701446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 0x560ba52e0690 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f530fa50740 1 -- 172.21.15.2:0/3583701446 >> 172.21.15.2:0/3583701446 conn(0x560ba52d2050 msgr2=0x560ba52d7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:07.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f52ebe00640 0 client.25733 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:07.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f530fa50740 1 -- 172.21.15.2:0/3583701446 shutdown_connections 2024-09-10T08:10:07.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f530fa50740 1 -- 172.21.15.2:0/3583701446 wait complete. 2024-09-10T08:10:07.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.476+0000 7f530fa50740 20 client.25733 trim_cache size 0 max 16384 2024-09-10T08:10:07.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.495+0000 7f530fa50740 1 Processor -- start 2024-09-10T08:10:07.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.495+0000 7f530fa50740 1 -- start start 2024-09-10T08:10:07.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.495+0000 7f530fa50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d2050 0x560ba52f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.495+0000 7f530fa50740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d5ac0 0x560ba52f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.495+0000 7f530fa50740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d63e0 0x560ba52daba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.495+0000 7f530fa50740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f53081b3890 con 0x560ba52d63e0 2024-09-10T08:10:07.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.495+0000 7f530fa50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53081b1160 con 0x560ba52d2050 2024-09-10T08:10:07.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.495+0000 7f530fa50740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53081aefb0 con 0x560ba52d5ac0 2024-09-10T08:10:07.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.495+0000 7f5306800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d63e0 0x560ba52daba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.495+0000 7f5305e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d2050 0x560ba52f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.495+0000 7f52fd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d5ac0 0x560ba52f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.495+0000 7f5306800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d63e0 0x560ba52daba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35754/0 (socket says 172.21.15.2:35754) 2024-09-10T08:10:07.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.495+0000 7f5306800640 1 -- 172.21.15.2:0/1478268919 learned_addr learned my addr 172.21.15.2:0/1478268919 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:07.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.496+0000 7f5305e00640 1 -- 172.21.15.2:0/1478268919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d5ac0 msgr2=0x560ba52f8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:07.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.496+0000 7f5305e00640 1 --2- 172.21.15.2:0/1478268919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d5ac0 0x560ba52f8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.496+0000 7f5305e00640 1 -- 172.21.15.2:0/1478268919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d63e0 msgr2=0x560ba52daba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:07.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.496+0000 7f5305e00640 1 --2- 172.21.15.2:0/1478268919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d63e0 0x560ba52daba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.496+0000 7f5305e00640 1 -- 172.21.15.2:0/1478268919 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ba52de2a0 con 0x560ba52d2050 2024-09-10T08:10:07.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.496+0000 7f52fd400640 1 --2- 172.21.15.2:0/1478268919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d5ac0 0x560ba52f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:07.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.496+0000 7f5305e00640 1 --2- 172.21.15.2:0/1478268919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d2050 0x560ba52f87b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52f80039c0 tx=0x7f52f8011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64c41a47fa24206b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:07.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.496+0000 7f52f6a00640 1 -- 172.21.15.2:0/1478268919 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f800fb20 con 0x560ba52d2050 2024-09-10T08:10:07.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.496+0000 7f52f6a00640 1 -- 172.21.15.2:0/1478268919 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f8015030 con 0x560ba52d2050 2024-09-10T08:10:07.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.496+0000 7f52f6a00640 1 -- 172.21.15.2:0/1478268919 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8019890 con 0x560ba52d2050 2024-09-10T08:10:07.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.497+0000 7f530fa50740 1 -- 172.21.15.2:0/1478268919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d2050 msgr2=0x560ba52f87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:07.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.497+0000 7f530fa50740 1 --2- 172.21.15.2:0/1478268919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d2050 0x560ba52f87b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52f80039c0 tx=0x7f52f8011ab0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.497+0000 7f530fa50740 1 -- 172.21.15.2:0/1478268919 shutdown_connections 2024-09-10T08:10:07.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.497+0000 7f530fa50740 1 --2- 172.21.15.2:0/1478268919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d63e0 0x560ba52daba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.497+0000 7f530fa50740 1 --2- 172.21.15.2:0/1478268919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d5ac0 0x560ba52f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.497+0000 7f530fa50740 1 --2- 172.21.15.2:0/1478268919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d2050 0x560ba52f87b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.497+0000 7f530fa50740 1 -- 172.21.15.2:0/1478268919 >> 172.21.15.2:0/1478268919 conn(0x560ba52dfdd0 msgr2=0x560ba52d8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:07.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.497+0000 7f530fa50740 1 -- 172.21.15.2:0/1478268919 shutdown_connections 2024-09-10T08:10:07.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.497+0000 7f530fa50740 1 -- 172.21.15.2:0/1478268919 wait complete. 2024-09-10T08:10:07.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.497+0000 7f530fa50740 1 -- start start 2024-09-10T08:10:07.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.498+0000 7f530fa50740 1 Processor -- start 2024-09-10T08:10:07.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.498+0000 7f530fa50740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d2050 0x560ba52e5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.498+0000 7f530fa50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 0x560ba52e5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.498+0000 7f530fa50740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d63e0 0x560ba53547c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.498+0000 7f530fa50740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f53081af620 con 0x560ba52d63e0 2024-09-10T08:10:07.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.498+0000 7f530fa50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53081b0a90 con 0x560ba52d5ac0 2024-09-10T08:10:07.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.498+0000 7f530fa50740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53081afcf0 con 0x560ba52d2050 2024-09-10T08:10:07.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.498+0000 7f530fa50740 10 client.0 _ll_register_callbacks cb 0x7f52f801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:07.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.498+0000 7f5306800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d63e0 0x560ba53547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.498+0000 7f52fd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 0x560ba52e5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.499+0000 7f5305e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d2050 0x560ba52e5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.499+0000 7f5306800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d63e0 0x560ba53547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35764/0 (socket says 172.21.15.2:35764) 2024-09-10T08:10:07.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.499+0000 7f5306800640 1 -- 172.21.15.2:0/67539600 learned_addr learned my addr 172.21.15.2:0/67539600 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:07.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.499+0000 7f52fd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 0x560ba52e5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58856/0 (socket says 172.21.15.2:58856) 2024-09-10T08:10:07.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.499+0000 7f5305e00640 1 -- 172.21.15.2:0/67539600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 msgr2=0x560ba52e5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:07.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.499+0000 7f5305e00640 1 --2- 172.21.15.2:0/67539600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 0x560ba52e5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.499+0000 7f5305e00640 1 -- 172.21.15.2:0/67539600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d63e0 msgr2=0x560ba53547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:07.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.499+0000 7f5305e00640 1 --2- 172.21.15.2:0/67539600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d63e0 0x560ba53547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.499+0000 7f5305e00640 1 -- 172.21.15.2:0/67539600 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ba5354d80 con 0x560ba52d2050 2024-09-10T08:10:07.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.499+0000 7f52fd400640 1 --2- 172.21.15.2:0/67539600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 0x560ba52e5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:07.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.499+0000 7f5305e00640 1 --2- 172.21.15.2:0/67539600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d2050 0x560ba52e5300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x560ba52ed700 tx=0x7f52f8005a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f8b44c70275712f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:07.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.499+0000 7f52ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:10:07.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.500+0000 7f52ebe00640 1 -- 172.21.15.2:0/67539600 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f801e030 con 0x560ba52d2050 2024-09-10T08:10:07.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.500+0000 7f530fa50740 1 -- 172.21.15.2:0/67539600 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f8015030 con 0x560ba52d2050 2024-09-10T08:10:07.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.500+0000 7f52ebe00640 1 -- 172.21.15.2:0/67539600 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f52f80194b0 con 0x560ba52d2050 2024-09-10T08:10:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.500+0000 7f52ebe00640 1 -- 172.21.15.2:0/67539600 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f52f800fb20 con 0x560ba52d2050 2024-09-10T08:10:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.500+0000 7f530fa50740 10 client.25717 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.500+0000 7f530fa50740 1 -- 172.21.15.2:0/67539600 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ba535c170 con 0x560ba52d2050 2024-09-10T08:10:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.500+0000 7f52ebe00640 1 -- 172.21.15.2:0/67539600 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f8015030 con 0x560ba52d2050 2024-09-10T08:10:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.500+0000 7f52ebe00640 1 -- 172.21.15.2:0/67539600 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f52f80054d0 con 0x560ba52d2050 2024-09-10T08:10:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.501+0000 7f530fa50740 10 client.25717 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.501+0000 7f530fa50740 10 client.25717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.501+0000 7f530fa50740 20 client.25717 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.501+0000 7f530fa50740 1 --2- 172.21.15.2:0/67539600 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560ba52e0ca0 0x560ba535e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.501+0000 7f530fa50740 4 client.25717 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:10:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.501+0000 7f530fa50740 1 -- 172.21.15.2:0/67539600 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560ba5243d10 con 0x560ba52e0ca0 2024-09-10T08:10:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.501+0000 7f52fd400640 1 --2- 172.21.15.2:0/67539600 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560ba52e0ca0 0x560ba535e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:07.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.501+0000 7f52fd400640 1 --2- 172.21.15.2:0/67539600 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560ba52e0ca0 0x560ba535e7e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c4e6e64512bc26d server_cookie=4b2e797b839aa3ea in_seq=0 out_seq=0 2024-09-10T08:10:07.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.501+0000 7f52ebe00640 10 client.25717 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:07.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f52ebe00640 1 -- 172.21.15.2:0/67539600 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560ba5243d10 con 0x560ba52e0ca0 2024-09-10T08:10:07.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f52ebe00640 10 client.25717 handle_command_reply: tid=0 2024-09-10T08:10:07.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f530fa50740 1 client.25717 shutdown 2024-09-10T08:10:07.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f530fa50740 1 -- 172.21.15.2:0/67539600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d2050 msgr2=0x560ba52e5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:07.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f530fa50740 1 --2- 172.21.15.2:0/67539600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d2050 0x560ba52e5300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x560ba52ed700 tx=0x7f52f8005a70 comp rx=0 tx=0).stop 2024-09-10T08:10:07.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f530fa50740 1 -- 172.21.15.2:0/67539600 shutdown_connections 2024-09-10T08:10:07.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f530fa50740 1 --2- 172.21.15.2:0/67539600 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560ba52e0ca0 0x560ba535e7e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f530fa50740 1 --2- 172.21.15.2:0/67539600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ba52d63e0 0x560ba53547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f530fa50740 1 --2- 172.21.15.2:0/67539600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ba52d5ac0 0x560ba52e5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f530fa50740 1 --2- 172.21.15.2:0/67539600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ba52d2050 0x560ba52e5300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f530fa50740 1 -- 172.21.15.2:0/67539600 >> 172.21.15.2:0/67539600 conn(0x560ba52dfdd0 msgr2=0x560ba52d7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:07.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f52ebe00640 0 client.25717 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:07.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f530fa50740 1 -- 172.21.15.2:0/67539600 shutdown_connections 2024-09-10T08:10:07.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.503+0000 7f530fa50740 1 -- 172.21.15.2:0/67539600 wait complete. 2024-09-10T08:10:07.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.504+0000 7f530fa50740 20 client.25717 trim_cache size 0 max 16384 2024-09-10T08:10:07.505 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:10:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:10:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "acbe5b4b-424d-446d-8f37-2e855a640a3b", 2024-09-10T08:10:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:10:07.505 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:10:07.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.504+0000 7f530d000640 1 -- 172.21.15.2:0/490830579 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52ec0a2c00 msgr2=0x7f52ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:07.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.504+0000 7f530d000640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52ec0a2c00 0x7f52ec0a50c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f52f0000c00 tx=0x7f52f0005bc0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.504+0000 7f530d000640 1 -- 172.21.15.2:0/490830579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5308109120 msgr2=0x7f530819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:07.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.504+0000 7f530d000640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5308109120 0x7f530819ef20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f530000eaa0 tx=0x7f530000ef60 comp rx=0 tx=0).stop 2024-09-10T08:10:07.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.504+0000 7f530d000640 1 -- 172.21.15.2:0/490830579 shutdown_connections 2024-09-10T08:10:07.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.504+0000 7f530d000640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5308109120 0x7f530819ef20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.504+0000 7f530d000640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53081049a0 0x7f530819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.504+0000 7f530d000640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52ec0a2c00 0x7f52ec0a50c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.504+0000 7f530d000640 1 --2- 172.21.15.2:0/490830579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5308103ff0 0x7f530819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:07.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.504+0000 7f530d000640 1 -- 172.21.15.2:0/490830579 >> 172.21.15.2:0/490830579 conn(0x7f53080ffa90 msgr2=0x7f5308100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:07.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.504+0000 7f530d000640 1 -- 172.21.15.2:0/490830579 shutdown_connections 2024-09-10T08:10:07.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:07.505+0000 7f530d000640 1 -- 172.21.15.2:0/490830579 wait complete. 2024-09-10T08:10:08.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:07 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:10:08.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:07 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:08.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:07 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:10:08.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:07 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:08.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:07 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:10:08.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:07 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:08.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:07 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:10:08.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:07 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:08.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:07 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:10:08.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:07 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:08.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:07 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:10:08.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:07 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:08.461 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:10:08.840 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:09.113 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:08 smithi002 ceph-mon[30563]: pgmap v447: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:10:09.113 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:08 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:10:09.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:08 smithi155 ceph-mon[35405]: pgmap v447: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:10:09.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:08 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:10:09.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:08 smithi107 ceph-mon[35048]: pgmap v447: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:10:09.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:08 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:10:09.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.384+0000 7f4eb6200640 1 -- 172.21.15.2:0/2194378143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb01049a0 msgr2=0x7f4eb01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:09.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.384+0000 7f4eb6200640 1 --2- 172.21.15.2:0/2194378143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb01049a0 0x7f4eb01089f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009900 tx=0x7f4e9c02a410 comp rx=0 tx=0).stop 2024-09-10T08:10:09.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.384+0000 7f4eb6200640 1 -- 172.21.15.2:0/2194378143 shutdown_connections 2024-09-10T08:10:09.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.384+0000 7f4eb6200640 1 --2- 172.21.15.2:0/2194378143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb0109120 0x7f4eb0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.384+0000 7f4eb6200640 1 --2- 172.21.15.2:0/2194378143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb01049a0 0x7f4eb01089f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.384+0000 7f4eb6200640 1 --2- 172.21.15.2:0/2194378143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb0103ff0 0x7f4eb01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.384+0000 7f4eb6200640 1 -- 172.21.15.2:0/2194378143 >> 172.21.15.2:0/2194378143 conn(0x7f4eb00ffa90 msgr2=0x7f4eb0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:09.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.384+0000 7f4eb6200640 1 -- 172.21.15.2:0/2194378143 shutdown_connections 2024-09-10T08:10:09.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.384+0000 7f4eb6200640 1 -- 172.21.15.2:0/2194378143 wait complete. 2024-09-10T08:10:09.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.385+0000 7f4eb6200640 1 Processor -- start 2024-09-10T08:10:09.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.386+0000 7f4eb6200640 1 -- start start 2024-09-10T08:10:09.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.386+0000 7f4eb6200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb0103ff0 0x7f4eb019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.386+0000 7f4eb6200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb01049a0 0x7f4eb019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.386+0000 7f4eb6200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb0109120 0x7f4eb019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.386+0000 7f4eb6200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb0114190 con 0x7f4eb0109120 2024-09-10T08:10:09.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.386+0000 7f4eb6200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb0114010 con 0x7f4eb01049a0 2024-09-10T08:10:09.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.386+0000 7f4eb6200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb0114310 con 0x7f4eb0103ff0 2024-09-10T08:10:09.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.386+0000 7f4eafe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb0109120 0x7f4eb019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.386+0000 7f4eaf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb0103ff0 0x7f4eb019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.386+0000 7f4eafe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb0109120 0x7f4eb019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35790/0 (socket says 172.21.15.2:35790) 2024-09-10T08:10:09.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.386+0000 7f4eafe00640 1 -- 172.21.15.2:0/961203277 learned_addr learned my addr 172.21.15.2:0/961203277 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:09.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.386+0000 7f4eaea00640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb01049a0 0x7f4eb019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.387+0000 7f4eaf400640 1 -- 172.21.15.2:0/961203277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb01049a0 msgr2=0x7f4eb019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:09.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.387+0000 7f4eaf400640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb01049a0 0x7f4eb019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.387+0000 7f4eaf400640 1 -- 172.21.15.2:0/961203277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb0109120 msgr2=0x7f4eb019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:09.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.387+0000 7f4eaf400640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb0109120 0x7f4eb019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.387+0000 7f4eaf400640 1 -- 172.21.15.2:0/961203277 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb019f5e0 con 0x7f4eb0103ff0 2024-09-10T08:10:09.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.387+0000 7f4eafe00640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb0109120 0x7f4eb019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:09.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.387+0000 7f4eaf400640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb0103ff0 0x7f4eb019a630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000e9f0 tx=0x7f4ea000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:09.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.387+0000 7f4ea7e00640 1 -- 172.21.15.2:0/961203277 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea000ccc0 con 0x7f4eb0103ff0 2024-09-10T08:10:09.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.388+0000 7f4eb6200640 1 -- 172.21.15.2:0/961203277 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb019f8d0 con 0x7f4eb0103ff0 2024-09-10T08:10:09.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.388+0000 7f4ea7e00640 1 -- 172.21.15.2:0/961203277 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea0005020 con 0x7f4eb0103ff0 2024-09-10T08:10:09.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.388+0000 7f4ea7e00640 1 -- 172.21.15.2:0/961203277 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00049e0 con 0x7f4eb0103ff0 2024-09-10T08:10:09.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.388+0000 7f4eb6200640 1 -- 172.21.15.2:0/961203277 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb01a71b0 con 0x7f4eb0103ff0 2024-09-10T08:10:09.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.391+0000 7f4eb8e3c740 1 Processor -- start 2024-09-10T08:10:09.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.391+0000 7f4ea7e00640 1 -- 172.21.15.2:0/961203277 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4ea0005560 con 0x7f4eb0103ff0 2024-09-10T08:10:09.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.391+0000 7f4eb8e3c740 1 -- start start 2024-09-10T08:10:09.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.391+0000 7f4eb8e3c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b87b90 0x563c18b87f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.391+0000 7f4eb8e3c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b884b0 0x563c18b92830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eb8e3c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b92d70 0x563c18b95160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eb8e3c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563c18aee3e0 con 0x563c18b92d70 2024-09-10T08:10:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eb8e3c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563c18a95d80 con 0x563c18b87b90 2024-09-10T08:10:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eb8e3c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563c18aea3a0 con 0x563c18b884b0 2024-09-10T08:10:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eafe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b87b90 0x563c18b87f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eafe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b87b90 0x563c18b87f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58872/0 (socket says 172.21.15.2:58872) 2024-09-10T08:10:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eafe00640 1 -- 172.21.15.2:0/3186382797 learned_addr learned my addr 172.21.15.2:0/3186382797 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eaea00640 1 --2- 172.21.15.2:0/3186382797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b92d70 0x563c18b95160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eaf400640 1 --2- 172.21.15.2:0/3186382797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b884b0 0x563c18b92830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eafe00640 1 -- 172.21.15.2:0/3186382797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b884b0 msgr2=0x563c18b92830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:09.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eafe00640 1 --2- 172.21.15.2:0/3186382797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b884b0 0x563c18b92830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eafe00640 1 -- 172.21.15.2:0/3186382797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b92d70 msgr2=0x563c18b95160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:09.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eafe00640 1 --2- 172.21.15.2:0/3186382797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b92d70 0x563c18b95160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eafe00640 1 -- 172.21.15.2:0/3186382797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c18ab4ae0 con 0x563c18b87b90 2024-09-10T08:10:09.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.392+0000 7f4eaea00640 1 --2- 172.21.15.2:0/3186382797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b92d70 0x563c18b95160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:09.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.393+0000 7f4eafe00640 1 --2- 172.21.15.2:0/3186382797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b87b90 0x563c18b87f70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800e830 tx=0x7f4ea800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe6948596a788d78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:09.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.393+0000 7f4ea7e00640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4e940a2b80 0x7f4e940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.393+0000 7f4e9be00640 1 -- 172.21.15.2:0/3186382797 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8004d20 con 0x563c18b87b90 2024-09-10T08:10:09.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.393+0000 7f4ea7e00640 1 -- 172.21.15.2:0/961203277 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4ea00cf400 con 0x7f4eb0103ff0 2024-09-10T08:10:09.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.393+0000 7f4e9be00640 1 -- 172.21.15.2:0/3186382797 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea8005020 con 0x563c18b87b90 2024-09-10T08:10:09.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.393+0000 7f4e9be00640 1 -- 172.21.15.2:0/3186382797 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea800f620 con 0x563c18b87b90 2024-09-10T08:10:09.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.393+0000 7f4eaf400640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4e940a2b80 0x7f4e940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.394+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/3186382797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b87b90 msgr2=0x563c18b87f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:09.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.394+0000 7f4eaf400640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4e940a2b80 0x7f4e940a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00981e0 tx=0x7f4ea00054d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:09.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.394+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/3186382797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b87b90 0x563c18b87f70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800e830 tx=0x7f4ea800ecf0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.394+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/3186382797 shutdown_connections 2024-09-10T08:10:09.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.394+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/3186382797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b92d70 0x563c18b95160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.394+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/3186382797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b884b0 0x563c18b92830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.394+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/3186382797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b87b90 0x563c18b87f70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.394+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/3186382797 >> 172.21.15.2:0/3186382797 conn(0x563c18b84120 msgr2=0x563c18b84500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:09.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.394+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/3186382797 shutdown_connections 2024-09-10T08:10:09.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.394+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/3186382797 wait complete. 2024-09-10T08:10:09.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.395+0000 7f4eb8e3c740 1 -- start start 2024-09-10T08:10:09.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.395+0000 7f4eb8e3c740 1 Processor -- start 2024-09-10T08:10:09.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.395+0000 7f4eb8e3c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18b8fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.395+0000 7f4eb8e3c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 0x563c18b90230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.395+0000 7f4eb8e3c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b92d70 0x563c18b90770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.395+0000 7f4eb8e3c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563c18b959e0 con 0x563c18b884b0 2024-09-10T08:10:09.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.395+0000 7f4eb8e3c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563c18aa90f0 con 0x563c18b92d70 2024-09-10T08:10:09.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.395+0000 7f4eb8e3c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563c18b95b60 con 0x563c18b87b90 2024-09-10T08:10:09.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.395+0000 7f4eb8e3c740 10 client.0 _ll_register_callbacks cb 0x563c18a9a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:09.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.396+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 0x563c18b90230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.396+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 0x563c18b90230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35814/0 (socket says 172.21.15.2:35814) 2024-09-10T08:10:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.396+0000 7f4eafe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18b8fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.396+0000 7f4eaea00640 1 -- 172.21.15.2:0/923985388 learned_addr learned my addr 172.21.15.2:0/923985388 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.396+0000 7f4eafe00640 1 --2- 172.21.15.2:0/923985388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b92d70 0x563c18b90770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.396+0000 7f4eafe00640 1 -- 172.21.15.2:0/923985388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b92d70 msgr2=0x563c18b90770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.396+0000 7f4eafe00640 1 --2- 172.21.15.2:0/923985388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b92d70 0x563c18b90770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.396+0000 7f4eafe00640 1 -- 172.21.15.2:0/923985388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 msgr2=0x563c18b90230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.396+0000 7f4eafe00640 1 --2- 172.21.15.2:0/923985388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 0x563c18b90230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.396+0000 7f4eafe00640 1 -- 172.21.15.2:0/923985388 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c1885f3c0 con 0x563c18b87b90 2024-09-10T08:10:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.396+0000 7f4eaea00640 1 --2- 172.21.15.2:0/923985388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 0x563c18b90230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.396+0000 7f4eafe00640 1 --2- 172.21.15.2:0/923985388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18b8fcf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8000f80 tx=0x7f4ea8005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a36977c034ea792 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.396+0000 7f4e98c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:10:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.397+0000 7f4e98c00640 1 -- 172.21.15.2:0/923985388 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8024040 con 0x563c18b87b90 2024-09-10T08:10:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.397+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/923985388 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea8005020 con 0x563c18b87b90 2024-09-10T08:10:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.397+0000 7f4e98c00640 1 -- 172.21.15.2:0/923985388 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4ea8005600 con 0x563c18b87b90 2024-09-10T08:10:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.397+0000 7f4e98c00640 1 -- 172.21.15.2:0/923985388 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ea8034050 con 0x563c18b87b90 2024-09-10T08:10:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.397+0000 7f4eb8e3c740 10 client.25729 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.397+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/923985388 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c18ba1710 con 0x563c18b87b90 2024-09-10T08:10:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.397+0000 7f4e98c00640 1 -- 172.21.15.2:0/923985388 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea8005020 con 0x563c18b87b90 2024-09-10T08:10:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.398+0000 7f4e98c00640 1 -- 172.21.15.2:0/923985388 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4ea801bcc0 con 0x563c18b87b90 2024-09-10T08:10:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.398+0000 7f4eb8e3c740 10 client.25729 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.398+0000 7f4eb8e3c740 10 client.25729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.398+0000 7f4eb8e3c740 20 client.25729 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.398+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/923985388 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563c18ba2a60 0x563c18ba4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.398+0000 7f4eb8e3c740 4 client.25729 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:09.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.398+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/923985388 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c18afb950 con 0x563c18ba2a60 2024-09-10T08:10:09.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.398+0000 7f4eaea00640 1 --2- 172.21.15.2:0/923985388 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563c18ba2a60 0x563c18ba4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.399+0000 7f4eaea00640 1 --2- 172.21.15.2:0/923985388 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563c18ba2a60 0x563c18ba4e50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da33661f6043b28 server_cookie=ca9710465d02282d in_seq=0 out_seq=0 2024-09-10T08:10:09.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.399+0000 7f4e98c00640 10 client.25729 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:09.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.399+0000 7f4e98c00640 1 -- 172.21.15.2:0/923985388 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c18afb950 con 0x563c18ba2a60 2024-09-10T08:10:09.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.399+0000 7f4e98c00640 10 client.25729 handle_command_reply: tid=0 2024-09-10T08:10:09.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.400+0000 7f4eb8e3c740 1 client.25729 shutdown 2024-09-10T08:10:09.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.400+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/923985388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 msgr2=0x563c18b8fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:09.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.400+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/923985388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18b8fcf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8000f80 tx=0x7f4ea8005a00 comp rx=0 tx=0).stop 2024-09-10T08:10:09.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.400+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/923985388 shutdown_connections 2024-09-10T08:10:09.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.400+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/923985388 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563c18ba2a60 0x563c18ba4e50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.400+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/923985388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b92d70 0x563c18b90770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.400+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/923985388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 0x563c18b90230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.400+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/923985388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18b8fcf0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.400+0000 7f4e98c00640 0 client.25729 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:09.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.400+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/923985388 >> 172.21.15.2:0/923985388 conn(0x563c18b84120 msgr2=0x563c18b89e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:09.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.400+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/923985388 shutdown_connections 2024-09-10T08:10:09.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.400+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/923985388 wait complete. 2024-09-10T08:10:09.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.400+0000 7f4eb8e3c740 20 client.25729 trim_cache size 0 max 16384 2024-09-10T08:10:09.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.418+0000 7f4eb8e3c740 1 Processor -- start 2024-09-10T08:10:09.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.418+0000 7f4eb8e3c740 1 -- start start 2024-09-10T08:10:09.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.418+0000 7f4eb8e3c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18baa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.418+0000 7f4eb8e3c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 0x563c18baab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.418+0000 7f4eb8e3c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b92d70 0x563c18b8cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.418+0000 7f4eb8e3c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb01b3950 con 0x563c18b884b0 2024-09-10T08:10:09.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.418+0000 7f4eb8e3c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb01b1220 con 0x563c18b92d70 2024-09-10T08:10:09.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.418+0000 7f4eb8e3c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb01af070 con 0x563c18b87b90 2024-09-10T08:10:09.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.419+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 0x563c18baab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.419+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 0x563c18baab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35830/0 (socket says 172.21.15.2:35830) 2024-09-10T08:10:09.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.419+0000 7f4eaea00640 1 -- 172.21.15.2:0/2678319205 learned_addr learned my addr 172.21.15.2:0/2678319205 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:09.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.419+0000 7f4eafe00640 1 --2- 172.21.15.2:0/2678319205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18baa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.419+0000 7f4eafe00640 1 --2- 172.21.15.2:0/2678319205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b92d70 0x563c18b8cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.419+0000 7f4eaea00640 1 -- 172.21.15.2:0/2678319205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 msgr2=0x563c18baa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:09.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.419+0000 7f4eaea00640 1 --2- 172.21.15.2:0/2678319205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18baa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.419+0000 7f4eaea00640 1 -- 172.21.15.2:0/2678319205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b92d70 msgr2=0x563c18b8cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:09.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.419+0000 7f4eaea00640 1 --2- 172.21.15.2:0/2678319205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b92d70 0x563c18b8cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.419+0000 7f4eaea00640 1 -- 172.21.15.2:0/2678319205 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c18b84db0 con 0x563c18b884b0 2024-09-10T08:10:09.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.419+0000 7f4eaea00640 1 --2- 172.21.15.2:0/2678319205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 0x563c18baab70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c02a930 tx=0x7f4e9c0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cc23cead55b049 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:09.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.420+0000 7f4e9b400640 1 -- 172.21.15.2:0/2678319205 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c035040 con 0x563c18b884b0 2024-09-10T08:10:09.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.420+0000 7f4e9b400640 1 -- 172.21.15.2:0/2678319205 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c04a070 con 0x563c18b884b0 2024-09-10T08:10:09.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.420+0000 7f4e9b400640 1 -- 172.21.15.2:0/2678319205 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0315b0 con 0x563c18b884b0 2024-09-10T08:10:09.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.420+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/2678319205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 msgr2=0x563c18baab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:09.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.420+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/2678319205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 0x563c18baab70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c02a930 tx=0x7f4e9c0095a0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.420+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/2678319205 shutdown_connections 2024-09-10T08:10:09.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.420+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/2678319205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b92d70 0x563c18b8cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.420+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/2678319205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b884b0 0x563c18baab70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.420+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/2678319205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18baa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.420+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/2678319205 >> 172.21.15.2:0/2678319205 conn(0x563c18b84120 msgr2=0x563c18b8ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:09.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.420+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/2678319205 shutdown_connections 2024-09-10T08:10:09.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.421+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/2678319205 wait complete. 2024-09-10T08:10:09.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.421+0000 7f4eb8e3c740 1 -- start start 2024-09-10T08:10:09.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.421+0000 7f4eb8e3c740 1 Processor -- start 2024-09-10T08:10:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eb8e3c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18b9b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eb8e3c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b884b0 0x563c18b8c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eb8e3c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b92d70 0x563c18b8ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eb8e3c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb01af6e0 con 0x563c18b92d70 2024-09-10T08:10:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eb8e3c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb01b0b50 con 0x563c18b884b0 2024-09-10T08:10:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eb8e3c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb01afdb0 con 0x563c18b87b90 2024-09-10T08:10:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eb8e3c740 10 client.0 _ll_register_callbacks cb 0x7f4ea8011df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eafe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b92d70 0x563c18b8ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b884b0 0x563c18b8c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eafe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18b9b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b884b0 0x563c18b8c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58912/0 (socket says 172.21.15.2:58912) 2024-09-10T08:10:09.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eafe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18b9b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42678/0 (socket says 172.21.15.2:42678) 2024-09-10T08:10:09.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eaea00640 1 -- 172.21.15.2:0/1172382188 learned_addr learned my addr 172.21.15.2:0/1172382188 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:09.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eaea00640 1 -- 172.21.15.2:0/1172382188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 msgr2=0x563c18b9b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:09.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eaea00640 1 --2- 172.21.15.2:0/1172382188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18b9b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eaea00640 1 -- 172.21.15.2:0/1172382188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b92d70 msgr2=0x563c18b8ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:09.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eaea00640 1 --2- 172.21.15.2:0/1172382188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b92d70 0x563c18b8ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.422+0000 7f4eaea00640 1 -- 172.21.15.2:0/1172382188 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c18b8d470 con 0x563c18b884b0 2024-09-10T08:10:09.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.423+0000 7f4eaea00640 1 --2- 172.21.15.2:0/1172382188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b884b0 0x563c18b8c8c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009230 tx=0x7f4e9c005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f75668cbb69e7ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:09.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.423+0000 7f4e98c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:09.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.423+0000 7f4e98c00640 1 -- 172.21.15.2:0/1172382188 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c052070 con 0x563c18b884b0 2024-09-10T08:10:09.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.423+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/1172382188 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9c04a070 con 0x563c18b884b0 2024-09-10T08:10:09.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.423+0000 7f4e98c00640 1 -- 172.21.15.2:0/1172382188 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4e9c005520 con 0x563c18b884b0 2024-09-10T08:10:09.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.423+0000 7f4e98c00640 1 -- 172.21.15.2:0/1172382188 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4e9c035040 con 0x563c18b884b0 2024-09-10T08:10:09.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.423+0000 7f4eb8e3c740 10 client.25757 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:09.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.423+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/1172382188 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c18c0d480 con 0x563c18b884b0 2024-09-10T08:10:09.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.424+0000 7f4e98c00640 1 -- 172.21.15.2:0/1172382188 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c04a070 con 0x563c18b884b0 2024-09-10T08:10:09.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.424+0000 7f4e98c00640 1 -- 172.21.15.2:0/1172382188 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4e9c031b90 con 0x563c18b884b0 2024-09-10T08:10:09.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.424+0000 7f4eb8e3c740 10 client.25757 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:09.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.424+0000 7f4eb8e3c740 10 client.25757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:09.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.424+0000 7f4eb8e3c740 20 client.25757 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:09.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.424+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/1172382188 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563c18c0daf0 0x563c18c0ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:09.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.424+0000 7f4eb8e3c740 4 client.25757 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:10:09.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.424+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/1172382188 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c18af5e60 con 0x563c18c0daf0 2024-09-10T08:10:09.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.424+0000 7f4eafe00640 1 --2- 172.21.15.2:0/1172382188 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563c18c0daf0 0x563c18c0ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:09.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4eafe00640 1 --2- 172.21.15.2:0/1172382188 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563c18c0daf0 0x563c18c0ff10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f084a4cd69083 server_cookie=880c0bc07d813e3d in_seq=0 out_seq=0 2024-09-10T08:10:09.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4e98c00640 10 client.25757 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:09.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4e98c00640 1 -- 172.21.15.2:0/1172382188 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563c18af5e60 con 0x563c18c0daf0 2024-09-10T08:10:09.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4e98c00640 10 client.25757 handle_command_reply: tid=0 2024-09-10T08:10:09.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4eb8e3c740 1 client.25757 shutdown 2024-09-10T08:10:09.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/1172382188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b884b0 msgr2=0x563c18b8c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:09.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/1172382188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b884b0 0x563c18b8c8c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009230 tx=0x7f4e9c005750 comp rx=0 tx=0).stop 2024-09-10T08:10:09.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/1172382188 shutdown_connections 2024-09-10T08:10:09.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/1172382188 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563c18c0daf0 0x563c18c0ff10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/1172382188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c18b92d70 0x563c18b8ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/1172382188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c18b884b0 0x563c18b8c8c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4eb8e3c740 1 --2- 172.21.15.2:0/1172382188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c18b87b90 0x563c18b9b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4e98c00640 0 client.25757 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:09.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/1172382188 >> 172.21.15.2:0/1172382188 conn(0x563c18b84120 msgr2=0x563c18b829a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:09.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/1172382188 shutdown_connections 2024-09-10T08:10:09.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4eb8e3c740 1 -- 172.21.15.2:0/1172382188 wait complete. 2024-09-10T08:10:09.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.425+0000 7f4eb8e3c740 20 client.25757 trim_cache size 0 max 16384 2024-09-10T08:10:09.434 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:10:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:10:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:10:09.434 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:10:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.426+0000 7f4eb6200640 1 -- 172.21.15.2:0/961203277 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4e940a2b80 msgr2=0x7f4e940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.426+0000 7f4eb6200640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4e940a2b80 0x7f4e940a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00981e0 tx=0x7f4ea00054d0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.426+0000 7f4eb6200640 1 -- 172.21.15.2:0/961203277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb0103ff0 msgr2=0x7f4eb019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.426+0000 7f4eb6200640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb0103ff0 0x7f4eb019a630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000e9f0 tx=0x7f4ea000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.426+0000 7f4eb6200640 1 -- 172.21.15.2:0/961203277 shutdown_connections 2024-09-10T08:10:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.426+0000 7f4eb6200640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb0109120 0x7f4eb019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.426+0000 7f4eb6200640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb01049a0 0x7f4eb019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.426+0000 7f4eb6200640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4e940a2b80 0x7f4e940a5040 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.426+0000 7f4eb6200640 1 --2- 172.21.15.2:0/961203277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb0103ff0 0x7f4eb019a630 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.426+0000 7f4eb6200640 1 -- 172.21.15.2:0/961203277 >> 172.21.15.2:0/961203277 conn(0x7f4eb00ffa90 msgr2=0x7f4eb0100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.426+0000 7f4eb6200640 1 -- 172.21.15.2:0/961203277 shutdown_connections 2024-09-10T08:10:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:09.426+0000 7f4eb6200640 1 -- 172.21.15.2:0/961203277 wait complete. 2024-09-10T08:10:10.203 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:10:10.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:10:10.584 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:11.002 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:10 smithi002 ceph-mon[30563]: pgmap v448: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-10T08:10:11.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.115+0000 7f0d41a00640 1 -- 172.21.15.2:0/2706108842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d3c10e420 msgr2=0x7f0d3c110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.115+0000 7f0d41a00640 1 --2- 172.21.15.2:0/2706108842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d3c10e420 0x7f0d3c110810 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009ca0 tx=0x7f0d3402a940 comp rx=0 tx=0).stop 2024-09-10T08:10:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.116+0000 7f0d41a00640 1 -- 172.21.15.2:0/2706108842 shutdown_connections 2024-09-10T08:10:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.116+0000 7f0d41a00640 1 --2- 172.21.15.2:0/2706108842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d3c10e420 0x7f0d3c110810 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.116+0000 7f0d41a00640 1 --2- 172.21.15.2:0/2706108842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c074ee0 0x7f0d3c10dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.116+0000 7f0d41a00640 1 --2- 172.21.15.2:0/2706108842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c0745c0 0x7f0d3c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.116+0000 7f0d41a00640 1 -- 172.21.15.2:0/2706108842 >> 172.21.15.2:0/2706108842 conn(0x7f0d3c0fd8c0 msgr2=0x7f0d3c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.116+0000 7f0d41a00640 1 -- 172.21.15.2:0/2706108842 shutdown_connections 2024-09-10T08:10:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.116+0000 7f0d41a00640 1 -- 172.21.15.2:0/2706108842 wait complete. 2024-09-10T08:10:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.117+0000 7f0d41a00640 1 Processor -- start 2024-09-10T08:10:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.117+0000 7f0d41a00640 1 -- start start 2024-09-10T08:10:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.117+0000 7f0d41a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c0745c0 0x7f0d3c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:11.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.117+0000 7f0d41a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d3c074ee0 0x7f0d3c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.117+0000 7f0d41a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c10e420 0x7f0d3c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.117+0000 7f0d41a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1141a0 con 0x7f0d3c074ee0 2024-09-10T08:10:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.117+0000 7f0d41a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c114020 con 0x7f0d3c0745c0 2024-09-10T08:10:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.117+0000 7f0d41a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c114320 con 0x7f0d3c10e420 2024-09-10T08:10:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.118+0000 7f0d32000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d3c074ee0 0x7f0d3c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.118+0000 7f0d3aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c0745c0 0x7f0d3c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.118+0000 7f0d3aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c0745c0 0x7f0d3c19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58942/0 (socket says 172.21.15.2:58942) 2024-09-10T08:10:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.118+0000 7f0d32000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d3c074ee0 0x7f0d3c19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35864/0 (socket says 172.21.15.2:35864) 2024-09-10T08:10:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.118+0000 7f0d3aa00640 1 -- 172.21.15.2:0/927319905 learned_addr learned my addr 172.21.15.2:0/927319905 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.118+0000 7f0d3b400640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c10e420 0x7f0d3c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.118+0000 7f0d3aa00640 1 -- 172.21.15.2:0/927319905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c10e420 msgr2=0x7f0d3c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.118+0000 7f0d3aa00640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c10e420 0x7f0d3c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.118+0000 7f0d3aa00640 1 -- 172.21.15.2:0/927319905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d3c074ee0 msgr2=0x7f0d3c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.118+0000 7f0d3aa00640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d3c074ee0 0x7f0d3c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.118+0000 7f0d3aa00640 1 -- 172.21.15.2:0/927319905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c19f600 con 0x7f0d3c0745c0 2024-09-10T08:10:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.118+0000 7f0d32000640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d3c074ee0 0x7f0d3c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.118+0000 7f0d3aa00640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c0745c0 0x7f0d3c19a650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00e9f0 tx=0x7f0d2c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.119+0000 7f0d33e00640 1 -- 172.21.15.2:0/927319905 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c00cd80 con 0x7f0d3c0745c0 2024-09-10T08:10:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.119+0000 7f0d33e00640 1 -- 172.21.15.2:0/927319905 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d2c010910 con 0x7f0d3c0745c0 2024-09-10T08:10:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.119+0000 7f0d33e00640 1 -- 172.21.15.2:0/927319905 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c018990 con 0x7f0d3c0745c0 2024-09-10T08:10:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.119+0000 7f0d41a00640 1 -- 172.21.15.2:0/927319905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c19f8f0 con 0x7f0d3c0745c0 2024-09-10T08:10:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.119+0000 7f0d41a00640 1 -- 172.21.15.2:0/927319905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c1a71d0 con 0x7f0d3c0745c0 2024-09-10T08:10:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.120+0000 7f0d41a00640 1 -- 172.21.15.2:0/927319905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3c0749a0 con 0x7f0d3c0745c0 2024-09-10T08:10:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.123+0000 7f0d33e00640 1 -- 172.21.15.2:0/927319905 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0d2c010ac0 con 0x7f0d3c0745c0 2024-09-10T08:10:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.124+0000 7f0d33e00640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d200a2c00 0x7f0d200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.124+0000 7f0d33e00640 1 -- 172.21.15.2:0/927319905 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0d2c0cf700 con 0x7f0d3c0745c0 2024-09-10T08:10:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.124+0000 7f0d32000640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d200a2c00 0x7f0d200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.125+0000 7f0d32000640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d200a2c00 0x7f0d200a50c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c10f7b0 tx=0x7f0d24005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.128+0000 7f0d33e00640 1 -- 172.21.15.2:0/927319905 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d2c09bbc0 con 0x7f0d3c0745c0 2024-09-10T08:10:11.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:10 smithi155 ceph-mon[35405]: pgmap v448: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-10T08:10:11.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:10 smithi107 ceph-mon[35048]: pgmap v448: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-10T08:10:11.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.352+0000 7f0d41a00640 1 -- 172.21.15.2:0/927319905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d3c02d7c0 con 0x7f0d3c0745c0 2024-09-10T08:10:11.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.353+0000 7f0d33e00640 1 -- 172.21.15.2:0/927319905 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f0d2c09fec0 con 0x7f0d3c0745c0 2024-09-10T08:10:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:10:11.352 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:10:11.352 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:10:11.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.355+0000 7f0d41a00640 1 -- 172.21.15.2:0/927319905 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d200a2c00 msgr2=0x7f0d200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:11.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.355+0000 7f0d41a00640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d200a2c00 0x7f0d200a50c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c10f7b0 tx=0x7f0d24005f90 comp rx=0 tx=0).stop 2024-09-10T08:10:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.355+0000 7f0d41a00640 1 -- 172.21.15.2:0/927319905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c0745c0 msgr2=0x7f0d3c19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.355+0000 7f0d41a00640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c0745c0 0x7f0d3c19a650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00e9f0 tx=0x7f0d2c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:10:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.355+0000 7f0d41a00640 1 -- 172.21.15.2:0/927319905 shutdown_connections 2024-09-10T08:10:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.355+0000 7f0d41a00640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d200a2c00 0x7f0d200a50c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.355+0000 7f0d41a00640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c10e420 0x7f0d3c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.355+0000 7f0d41a00640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d3c074ee0 0x7f0d3c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.355+0000 7f0d41a00640 1 --2- 172.21.15.2:0/927319905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c0745c0 0x7f0d3c19a650 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.355+0000 7f0d41a00640 1 -- 172.21.15.2:0/927319905 >> 172.21.15.2:0/927319905 conn(0x7f0d3c0fd8c0 msgr2=0x7f0d3c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.356+0000 7f0d41a00640 1 -- 172.21.15.2:0/927319905 shutdown_connections 2024-09-10T08:10:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:11.356+0000 7f0d41a00640 1 -- 172.21.15.2:0/927319905 wait complete. 2024-09-10T08:10:12.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:11 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/927319905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:10:12.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:11 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/927319905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:10:12.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:11 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/927319905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:10:12.342 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:10:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:12.980 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:10:13.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:12 smithi155 ceph-mon[35405]: pgmap v449: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:10:13.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:12 smithi107 ceph-mon[35048]: pgmap v449: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:10:13.204 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:12 smithi002 ceph-mon[30563]: pgmap v449: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:10:13.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.322+0000 7f8437e00640 1 -- 172.21.15.2:0/3327542156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84301049a0 msgr2=0x7f84301089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:13.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.322+0000 7f8437e00640 1 --2- 172.21.15.2:0/3327542156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84301049a0 0x7f84301089f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f841c009900 tx=0x7f841c02a410 comp rx=0 tx=0).stop 2024-09-10T08:10:13.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.322+0000 7f8437e00640 1 -- 172.21.15.2:0/3327542156 shutdown_connections 2024-09-10T08:10:13.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.322+0000 7f8437e00640 1 --2- 172.21.15.2:0/3327542156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430109120 0x7f8430110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.322+0000 7f8437e00640 1 --2- 172.21.15.2:0/3327542156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84301049a0 0x7f84301089f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.322+0000 7f8437e00640 1 --2- 172.21.15.2:0/3327542156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8430103ff0 0x7f84301043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.322+0000 7f8437e00640 1 -- 172.21.15.2:0/3327542156 >> 172.21.15.2:0/3327542156 conn(0x7f84300ffa90 msgr2=0x7f8430101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:13.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.322+0000 7f8437e00640 1 -- 172.21.15.2:0/3327542156 shutdown_connections 2024-09-10T08:10:13.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.322+0000 7f8437e00640 1 -- 172.21.15.2:0/3327542156 wait complete. 2024-09-10T08:10:13.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.324+0000 7f8437e00640 1 Processor -- start 2024-09-10T08:10:13.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.324+0000 7f8437e00640 1 -- start start 2024-09-10T08:10:13.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.324+0000 7f8437e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8430103ff0 0x7f843019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.324+0000 7f8437e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84301049a0 0x7f843019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.324+0000 7f8437e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430109120 0x7f843019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.324+0000 7f8437e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8430114170 con 0x7f8430103ff0 2024-09-10T08:10:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.324+0000 7f8437e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8430113ff0 con 0x7f8430109120 2024-09-10T08:10:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.324+0000 7f8437e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84301142f0 con 0x7f84301049a0 2024-09-10T08:10:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.325+0000 7f8435400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8430103ff0 0x7f843019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.325+0000 7f8435400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8430103ff0 0x7f843019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35886/0 (socket says 172.21.15.2:35886) 2024-09-10T08:10:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.325+0000 7f8435400640 1 -- 172.21.15.2:0/1219460798 learned_addr learned my addr 172.21.15.2:0/1219460798 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.325+0000 7f8434a00640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84301049a0 0x7f843019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.325+0000 7f8435e00640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430109120 0x7f843019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.325+0000 7f8435400640 1 -- 172.21.15.2:0/1219460798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84301049a0 msgr2=0x7f843019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:13.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.325+0000 7f8435400640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84301049a0 0x7f843019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.325+0000 7f8435400640 1 -- 172.21.15.2:0/1219460798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430109120 msgr2=0x7f843019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:13.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.325+0000 7f8435400640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430109120 0x7f843019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.325+0000 7f8435400640 1 -- 172.21.15.2:0/1219460798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843019f600 con 0x7f8430103ff0 2024-09-10T08:10:13.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.326+0000 7f8435400640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8430103ff0 0x7f843019a650 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f842400d950 tx=0x7f842400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:13.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.326+0000 7f842e000640 1 -- 172.21.15.2:0/1219460798 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424017070 con 0x7f8430103ff0 2024-09-10T08:10:13.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.326+0000 7f842e000640 1 -- 172.21.15.2:0/1219460798 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8424009c30 con 0x7f8430103ff0 2024-09-10T08:10:13.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.326+0000 7f842e000640 1 -- 172.21.15.2:0/1219460798 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842400f6b0 con 0x7f8430103ff0 2024-09-10T08:10:13.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.326+0000 7f8437e00640 1 -- 172.21.15.2:0/1219460798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843019f8f0 con 0x7f8430103ff0 2024-09-10T08:10:13.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.326+0000 7f8437e00640 1 -- 172.21.15.2:0/1219460798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84301a71d0 con 0x7f8430103ff0 2024-09-10T08:10:13.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.329+0000 7f843aa12740 1 Processor -- start 2024-09-10T08:10:13.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.329+0000 7f842e000640 1 -- 172.21.15.2:0/1219460798 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f84240109d0 con 0x7f8430103ff0 2024-09-10T08:10:13.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.329+0000 7f843aa12740 1 -- start start 2024-09-10T08:10:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.330+0000 7f843aa12740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f7c70 0x5595016f8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.330+0000 7f843aa12740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595016f8590 0x559501702910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.330+0000 7f843aa12740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559501702e50 0x559501705240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.330+0000 7f843aa12740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55950165e3e0 con 0x5595016f7c70 2024-09-10T08:10:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.330+0000 7f843aa12740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559501605d80 con 0x5595016f8590 2024-09-10T08:10:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.330+0000 7f843aa12740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55950165a3a0 con 0x559501702e50 2024-09-10T08:10:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.330+0000 7f8435e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f7c70 0x5595016f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.330+0000 7f8435e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f7c70 0x5595016f8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35900/0 (socket says 172.21.15.2:35900) 2024-09-10T08:10:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.330+0000 7f8435e00640 1 -- 172.21.15.2:0/843924044 learned_addr learned my addr 172.21.15.2:0/843924044 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.330+0000 7f8435400640 1 --2- 172.21.15.2:0/843924044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595016f8590 0x559501702910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.331+0000 7f8434a00640 1 --2- 172.21.15.2:0/843924044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559501702e50 0x559501705240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.331+0000 7f8435e00640 1 -- 172.21.15.2:0/843924044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559501702e50 msgr2=0x559501705240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.331+0000 7f8435e00640 1 --2- 172.21.15.2:0/843924044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559501702e50 0x559501705240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.331+0000 7f8435e00640 1 -- 172.21.15.2:0/843924044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595016f8590 msgr2=0x559501702910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.331+0000 7f8435e00640 1 --2- 172.21.15.2:0/843924044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595016f8590 0x559501702910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.331+0000 7f8435e00640 1 -- 172.21.15.2:0/843924044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559501624ae0 con 0x5595016f7c70 2024-09-10T08:10:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.331+0000 7f8435e00640 1 --2- 172.21.15.2:0/843924044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f7c70 0x5595016f8050 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8428009f50 tx=0x7f84280118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc13ccd27fb89af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.331+0000 7f8422600640 1 -- 172.21.15.2:0/843924044 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428004b20 con 0x5595016f7c70 2024-09-10T08:10:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.332+0000 7f8422600640 1 -- 172.21.15.2:0/843924044 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8428004cc0 con 0x5595016f7c70 2024-09-10T08:10:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.332+0000 7f8422600640 1 -- 172.21.15.2:0/843924044 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428005020 con 0x5595016f7c70 2024-09-10T08:10:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.332+0000 7f842e000640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f84180a2c00 0x7f84180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.332+0000 7f842e000640 1 -- 172.21.15.2:0/1219460798 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f84240cefd0 con 0x7f8430103ff0 2024-09-10T08:10:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.332+0000 7f8435400640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f84180a2c00 0x7f84180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.332+0000 7f843aa12740 1 -- 172.21.15.2:0/843924044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f7c70 msgr2=0x5595016f8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.332+0000 7f843aa12740 1 --2- 172.21.15.2:0/843924044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f7c70 0x5595016f8050 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8428009f50 tx=0x7f84280118d0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.333+0000 7f843aa12740 1 -- 172.21.15.2:0/843924044 shutdown_connections 2024-09-10T08:10:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.333+0000 7f843aa12740 1 --2- 172.21.15.2:0/843924044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559501702e50 0x559501705240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.333+0000 7f843aa12740 1 --2- 172.21.15.2:0/843924044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595016f8590 0x559501702910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.333+0000 7f843aa12740 1 --2- 172.21.15.2:0/843924044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f7c70 0x5595016f8050 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.333+0000 7f843aa12740 1 -- 172.21.15.2:0/843924044 >> 172.21.15.2:0/843924044 conn(0x5595016f4200 msgr2=0x5595016f45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:13.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.333+0000 7f843aa12740 1 -- 172.21.15.2:0/843924044 shutdown_connections 2024-09-10T08:10:13.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.333+0000 7f843aa12740 1 -- 172.21.15.2:0/843924044 wait complete. 2024-09-10T08:10:13.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.333+0000 7f8435400640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f84180a2c00 0x7f84180a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8424097e60 tx=0x7f84240d2a10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:13.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.334+0000 7f843aa12740 1 -- start start 2024-09-10T08:10:13.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.334+0000 7f843aa12740 1 Processor -- start 2024-09-10T08:10:13.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.335+0000 7f843aa12740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f7c70 0x5595016ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.335+0000 7f843aa12740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f8590 0x5595017002a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.335+0000 7f843aa12740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559501702e50 0x5595017007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.335+0000 7f843aa12740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559501705a20 con 0x5595016f8590 2024-09-10T08:10:13.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.335+0000 7f843aa12740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5595016190f0 con 0x559501702e50 2024-09-10T08:10:13.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.335+0000 7f843aa12740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559501705ba0 con 0x5595016f7c70 2024-09-10T08:10:13.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.335+0000 7f843aa12740 10 client.0 _ll_register_callbacks cb 0x559501600bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:13.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.335+0000 7f8435e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f7c70 0x5595016ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.335+0000 7f8435e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f7c70 0x5595016ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42728/0 (socket says 172.21.15.2:42728) 2024-09-10T08:10:13.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.335+0000 7f8435e00640 1 -- 172.21.15.2:0/2903153530 learned_addr learned my addr 172.21.15.2:0/2903153530 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:13.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.335+0000 7f8434a00640 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f8590 0x5595017002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.335+0000 7f8435e00640 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559501702e50 0x5595017007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.336+0000 7f8435e00640 1 -- 172.21.15.2:0/2903153530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559501702e50 msgr2=0x5595017007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:13.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.336+0000 7f8435e00640 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559501702e50 0x5595017007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.336+0000 7f8435e00640 1 -- 172.21.15.2:0/2903153530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f8590 msgr2=0x5595017002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:13.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.336+0000 7f8435e00640 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f8590 0x5595017002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.336+0000 7f8435e00640 1 -- 172.21.15.2:0/2903153530 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595013cf3c0 con 0x5595016f7c70 2024-09-10T08:10:13.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.336+0000 7f8434a00640 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f8590 0x5595017002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:13.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.336+0000 7f8435e00640 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f7c70 0x5595016ffd60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f84280099e0 tx=0x7f84280045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9246c5927fbe979 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:13.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.336+0000 7f8416a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:10:13.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.336+0000 7f8416a00640 1 -- 172.21.15.2:0/2903153530 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842802c020 con 0x5595016f7c70 2024-09-10T08:10:13.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.336+0000 7f843aa12740 1 -- 172.21.15.2:0/2903153530 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8428004cc0 con 0x5595016f7c70 2024-09-10T08:10:13.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.336+0000 7f8416a00640 1 -- 172.21.15.2:0/2903153530 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8428016040 con 0x5595016f7c70 2024-09-10T08:10:13.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.337+0000 7f8416a00640 1 -- 172.21.15.2:0/2903153530 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8428037050 con 0x5595016f7c70 2024-09-10T08:10:13.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.337+0000 7f843aa12740 10 client.25753 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:13.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.337+0000 7f843aa12740 1 -- 172.21.15.2:0/2903153530 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559501711a90 con 0x5595016f7c70 2024-09-10T08:10:13.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.337+0000 7f8416a00640 1 -- 172.21.15.2:0/2903153530 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8428004cc0 con 0x5595016f7c70 2024-09-10T08:10:13.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.337+0000 7f8416a00640 1 -- 172.21.15.2:0/2903153530 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f842800f480 con 0x5595016f7c70 2024-09-10T08:10:13.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.338+0000 7f843aa12740 10 client.25753 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:13.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.338+0000 7f843aa12740 10 client.25753 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:10:13.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.338+0000 7f843aa12740 20 client.25753 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:13.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.338+0000 7f843aa12740 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x559501712ea0 0x559501715290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.338+0000 7f843aa12740 4 client.25753 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:13.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.338+0000 7f843aa12740 1 -- 172.21.15.2:0/2903153530 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55950166b950 con 0x559501712ea0 2024-09-10T08:10:13.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.338+0000 7f8434a00640 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x559501712ea0 0x559501715290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.339+0000 7f8434a00640 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x559501712ea0 0x559501715290 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe4fa6ce4565ba30 server_cookie=1f8e8bff10d89861 in_seq=0 out_seq=0 2024-09-10T08:10:13.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.339+0000 7f8416a00640 10 client.25753 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:13.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.339+0000 7f8416a00640 1 -- 172.21.15.2:0/2903153530 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55950166b950 con 0x559501712ea0 2024-09-10T08:10:13.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.339+0000 7f8416a00640 10 client.25753 handle_command_reply: tid=0 2024-09-10T08:10:13.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.339+0000 7f843aa12740 1 client.25753 shutdown 2024-09-10T08:10:13.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.340+0000 7f843aa12740 1 -- 172.21.15.2:0/2903153530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f7c70 msgr2=0x5595016ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:13.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.340+0000 7f843aa12740 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f7c70 0x5595016ffd60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f84280099e0 tx=0x7f84280045b0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.340+0000 7f843aa12740 1 -- 172.21.15.2:0/2903153530 shutdown_connections 2024-09-10T08:10:13.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.340+0000 7f843aa12740 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559501702e50 0x5595017007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.340+0000 7f843aa12740 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x559501712ea0 0x559501715290 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.340+0000 7f843aa12740 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f8590 0x5595017002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.340+0000 7f843aa12740 1 --2- 172.21.15.2:0/2903153530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f7c70 0x5595016ffd60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.340+0000 7f8416a00640 0 client.25753 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:13.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.340+0000 7f843aa12740 1 -- 172.21.15.2:0/2903153530 >> 172.21.15.2:0/2903153530 conn(0x5595016f4200 msgr2=0x5595016f3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:13.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.340+0000 7f843aa12740 1 -- 172.21.15.2:0/2903153530 shutdown_connections 2024-09-10T08:10:13.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.340+0000 7f843aa12740 1 -- 172.21.15.2:0/2903153530 wait complete. 2024-09-10T08:10:13.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.340+0000 7f843aa12740 20 client.25753 trim_cache size 0 max 16384 2024-09-10T08:10:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.354+0000 7f843aa12740 1 Processor -- start 2024-09-10T08:10:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.354+0000 7f843aa12740 1 -- start start 2024-09-10T08:10:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.355+0000 7f843aa12740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f7c70 0x559501702090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.355+0000 7f843aa12740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f8590 0x5595017002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.355+0000 7f843aa12740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559501702e50 0x559501700810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.355+0000 7f843aa12740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f84301b3990 con 0x5595016f8590 2024-09-10T08:10:13.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.355+0000 7f843aa12740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84301b1260 con 0x559501702e50 2024-09-10T08:10:13.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.355+0000 7f843aa12740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84301af0b0 con 0x5595016f7c70 2024-09-10T08:10:13.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.355+0000 7f8434a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f8590 0x5595017002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.355+0000 7f8434a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f8590 0x5595017002d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35914/0 (socket says 172.21.15.2:35914) 2024-09-10T08:10:13.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.355+0000 7f8435e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f7c70 0x559501702090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.355+0000 7f8434a00640 1 -- 172.21.15.2:0/2470641497 learned_addr learned my addr 172.21.15.2:0/2470641497 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:13.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.355+0000 7f8435e00640 1 --2- 172.21.15.2:0/2470641497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559501702e50 0x559501700810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.356+0000 7f8435e00640 1 -- 172.21.15.2:0/2470641497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559501702e50 msgr2=0x559501700810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:13.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.356+0000 7f8435e00640 1 --2- 172.21.15.2:0/2470641497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559501702e50 0x559501700810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.356+0000 7f8435e00640 1 -- 172.21.15.2:0/2470641497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f8590 msgr2=0x5595017002d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.356+0000 7f8435e00640 1 --2- 172.21.15.2:0/2470641497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f8590 0x5595017002d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.356+0000 7f8435e00640 1 -- 172.21.15.2:0/2470641497 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595016ef770 con 0x5595016f7c70 2024-09-10T08:10:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.356+0000 7f8434a00640 1 --2- 172.21.15.2:0/2470641497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f8590 0x5595017002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.356+0000 7f8435e00640 1 --2- 172.21.15.2:0/2470641497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f7c70 0x559501702090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8428003f10 tx=0x7f84280118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67f23f4702695264 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.356+0000 7f8421c00640 1 -- 172.21.15.2:0/2470641497 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842802c040 con 0x5595016f7c70 2024-09-10T08:10:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.356+0000 7f8421c00640 1 -- 172.21.15.2:0/2470641497 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f842800fac0 con 0x5595016f7c70 2024-09-10T08:10:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.356+0000 7f8421c00640 1 -- 172.21.15.2:0/2470641497 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84280286d0 con 0x5595016f7c70 2024-09-10T08:10:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.357+0000 7f843aa12740 1 -- 172.21.15.2:0/2470641497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f7c70 msgr2=0x559501702090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.357+0000 7f843aa12740 1 --2- 172.21.15.2:0/2470641497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f7c70 0x559501702090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8428003f10 tx=0x7f84280118d0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.357+0000 7f843aa12740 1 -- 172.21.15.2:0/2470641497 shutdown_connections 2024-09-10T08:10:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.357+0000 7f843aa12740 1 --2- 172.21.15.2:0/2470641497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559501702e50 0x559501700810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.357+0000 7f843aa12740 1 --2- 172.21.15.2:0/2470641497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595016f8590 0x5595017002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.357+0000 7f843aa12740 1 --2- 172.21.15.2:0/2470641497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f7c70 0x559501702090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.357+0000 7f843aa12740 1 -- 172.21.15.2:0/2470641497 >> 172.21.15.2:0/2470641497 conn(0x5595016f4200 msgr2=0x5595016f3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.357+0000 7f843aa12740 1 -- 172.21.15.2:0/2470641497 shutdown_connections 2024-09-10T08:10:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.357+0000 7f843aa12740 1 -- 172.21.15.2:0/2470641497 wait complete. 2024-09-10T08:10:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.358+0000 7f843aa12740 1 -- start start 2024-09-10T08:10:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.358+0000 7f843aa12740 1 Processor -- start 2024-09-10T08:10:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.358+0000 7f843aa12740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595016f7c70 0x55950170b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f843aa12740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f8590 0x559501706720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f843aa12740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559501702e50 0x559501706c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f843aa12740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f84301af720 con 0x559501702e50 2024-09-10T08:10:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f843aa12740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84301b0b90 con 0x5595016f7c70 2024-09-10T08:10:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f843aa12740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84301afdf0 con 0x5595016f8590 2024-09-10T08:10:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f843aa12740 10 client.0 _ll_register_callbacks cb 0x7f84240d28e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f8435e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595016f7c70 0x55950170b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f8435e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595016f7c70 0x55950170b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:59000/0 (socket says 172.21.15.2:59000) 2024-09-10T08:10:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f8435e00640 1 -- 172.21.15.2:0/2431201800 learned_addr learned my addr 172.21.15.2:0/2431201800 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f8435e00640 1 --2- 172.21.15.2:0/2431201800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559501702e50 0x559501706c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f8434a00640 1 --2- 172.21.15.2:0/2431201800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f8590 0x559501706720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f8435e00640 1 -- 172.21.15.2:0/2431201800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f8590 msgr2=0x559501706720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f8435e00640 1 --2- 172.21.15.2:0/2431201800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f8590 0x559501706720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f8435e00640 1 -- 172.21.15.2:0/2431201800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559501702e50 msgr2=0x559501706c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f8435e00640 1 --2- 172.21.15.2:0/2431201800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559501702e50 0x559501706c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.359+0000 7f8435e00640 1 -- 172.21.15.2:0/2431201800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595017072d0 con 0x5595016f7c70 2024-09-10T08:10:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.360+0000 7f8435e00640 1 --2- 172.21.15.2:0/2431201800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595016f7c70 0x55950170b3a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8428003d80 tx=0x7f8428003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7174d66d95addc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.360+0000 7f8416a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:13.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.360+0000 7f8416a00640 1 -- 172.21.15.2:0/2431201800 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842802c030 con 0x5595016f7c70 2024-09-10T08:10:13.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.360+0000 7f843aa12740 1 -- 172.21.15.2:0/2431201800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f842800fac0 con 0x5595016f7c70 2024-09-10T08:10:13.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.360+0000 7f8416a00640 1 -- 172.21.15.2:0/2431201800 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8428004340 con 0x5595016f7c70 2024-09-10T08:10:13.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.360+0000 7f8416a00640 1 -- 172.21.15.2:0/2431201800 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8428009b30 con 0x5595016f7c70 2024-09-10T08:10:13.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.360+0000 7f843aa12740 10 client.25781 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:13.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.360+0000 7f843aa12740 1 -- 172.21.15.2:0/2431201800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55950177d7f0 con 0x5595016f7c70 2024-09-10T08:10:13.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.361+0000 7f8416a00640 1 -- 172.21.15.2:0/2431201800 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f842800fac0 con 0x5595016f7c70 2024-09-10T08:10:13.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.361+0000 7f8416a00640 1 -- 172.21.15.2:0/2431201800 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8428028c90 con 0x5595016f7c70 2024-09-10T08:10:13.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.361+0000 7f843aa12740 10 client.25781 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.361+0000 7f843aa12740 10 client.25781 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:10:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.361+0000 7f843aa12740 20 client.25781 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.361+0000 7f843aa12740 1 --2- 172.21.15.2:0/2431201800 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55950177de00 0x5595017801f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.361+0000 7f843aa12740 4 client.25781 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:10:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.361+0000 7f843aa12740 1 -- 172.21.15.2:0/2431201800 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559501665e60 con 0x55950177de00 2024-09-10T08:10:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.361+0000 7f8434a00640 1 --2- 172.21.15.2:0/2431201800 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55950177de00 0x5595017801f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.362+0000 7f8434a00640 1 --2- 172.21.15.2:0/2431201800 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55950177de00 0x5595017801f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e7642af724fc830 server_cookie=71bcdd6605e4ee65 in_seq=0 out_seq=0 2024-09-10T08:10:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.362+0000 7f8416a00640 10 client.25781 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.362+0000 7f8416a00640 1 -- 172.21.15.2:0/2431201800 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559501665e60 con 0x55950177de00 2024-09-10T08:10:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.362+0000 7f8416a00640 10 client.25781 handle_command_reply: tid=0 2024-09-10T08:10:13.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.362+0000 7f843aa12740 1 client.25781 shutdown 2024-09-10T08:10:13.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.362+0000 7f843aa12740 1 -- 172.21.15.2:0/2431201800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595016f7c70 msgr2=0x55950170b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:13.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.362+0000 7f843aa12740 1 --2- 172.21.15.2:0/2431201800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595016f7c70 0x55950170b3a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8428003d80 tx=0x7f8428003e70 comp rx=0 tx=0).stop 2024-09-10T08:10:13.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.362+0000 7f843aa12740 1 -- 172.21.15.2:0/2431201800 shutdown_connections 2024-09-10T08:10:13.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.362+0000 7f843aa12740 1 --2- 172.21.15.2:0/2431201800 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55950177de00 0x5595017801f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.363+0000 7f843aa12740 1 --2- 172.21.15.2:0/2431201800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559501702e50 0x559501706c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.363+0000 7f843aa12740 1 --2- 172.21.15.2:0/2431201800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595016f8590 0x559501706720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.363+0000 7f843aa12740 1 --2- 172.21.15.2:0/2431201800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595016f7c70 0x55950170b3a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.363+0000 7f843aa12740 1 -- 172.21.15.2:0/2431201800 >> 172.21.15.2:0/2431201800 conn(0x5595016f4200 msgr2=0x5595016f3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:13.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.363+0000 7f8416a00640 0 client.25781 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:13.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.363+0000 7f843aa12740 1 -- 172.21.15.2:0/2431201800 shutdown_connections 2024-09-10T08:10:13.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.363+0000 7f843aa12740 1 -- 172.21.15.2:0/2431201800 wait complete. 2024-09-10T08:10:13.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.363+0000 7f843aa12740 20 client.25781 trim_cache size 0 max 16384 2024-09-10T08:10:13.366 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:10:13.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.364+0000 7f8437e00640 1 -- 172.21.15.2:0/1219460798 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f84180a2c00 msgr2=0x7f84180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:13.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.364+0000 7f8437e00640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f84180a2c00 0x7f84180a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8424097e60 tx=0x7f84240d2a10 comp rx=0 tx=0).stop 2024-09-10T08:10:13.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.364+0000 7f8437e00640 1 -- 172.21.15.2:0/1219460798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8430103ff0 msgr2=0x7f843019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:13.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.364+0000 7f8437e00640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8430103ff0 0x7f843019a650 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f842400d950 tx=0x7f842400de10 comp rx=0 tx=0).stop 2024-09-10T08:10:13.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.364+0000 7f8437e00640 1 -- 172.21.15.2:0/1219460798 shutdown_connections 2024-09-10T08:10:13.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.364+0000 7f8437e00640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430109120 0x7f843019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.364+0000 7f8437e00640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f84180a2c00 0x7f84180a50c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.364+0000 7f8437e00640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84301049a0 0x7f843019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.364+0000 7f8437e00640 1 --2- 172.21.15.2:0/1219460798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8430103ff0 0x7f843019a650 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:13.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.364+0000 7f8437e00640 1 -- 172.21.15.2:0/1219460798 >> 172.21.15.2:0/1219460798 conn(0x7f84300ffa90 msgr2=0x7f8430100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:13.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.364+0000 7f8437e00640 1 -- 172.21.15.2:0/1219460798 shutdown_connections 2024-09-10T08:10:13.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:13.364+0000 7f8437e00640 1 -- 172.21.15.2:0/1219460798 wait complete. 2024-09-10T08:10:14.123 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:10:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:14.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.803+0000 7fd054a00640 1 -- 172.21.15.2:0/1237728139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd050102360 msgr2=0x7fd05010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:14.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.803+0000 7fd054a00640 1 --2- 172.21.15.2:0/1237728139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd050102360 0x7fd05010e890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd048009c10 tx=0x7fd04802a6e0 comp rx=0 tx=0).stop 2024-09-10T08:10:14.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.803+0000 7fd054a00640 1 -- 172.21.15.2:0/1237728139 shutdown_connections 2024-09-10T08:10:14.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.803+0000 7fd054a00640 1 --2- 172.21.15.2:0/1237728139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd050102360 0x7fd05010e890 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:14.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.803+0000 7fd054a00640 1 --2- 172.21.15.2:0/1237728139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0501019c0 0x7fd050101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:14.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.803+0000 7fd054a00640 1 --2- 172.21.15.2:0/1237728139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0501079c0 0x7fd050107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:14.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.803+0000 7fd054a00640 1 -- 172.21.15.2:0/1237728139 >> 172.21.15.2:0/1237728139 conn(0x7fd0500fd740 msgr2=0x7fd0500ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:14.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.803+0000 7fd054a00640 1 -- 172.21.15.2:0/1237728139 shutdown_connections 2024-09-10T08:10:14.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.803+0000 7fd054a00640 1 -- 172.21.15.2:0/1237728139 wait complete. 2024-09-10T08:10:14.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.804+0000 7fd054a00640 1 Processor -- start 2024-09-10T08:10:14.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.805+0000 7fd054a00640 1 -- start start 2024-09-10T08:10:14.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.805+0000 7fd054a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0501019c0 0x7fd050075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:14.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.805+0000 7fd054a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd050102360 0x7fd050072040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:14.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.805+0000 7fd054a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0501079c0 0x7fd0500725a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:14.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.805+0000 7fd054a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd05006c620 con 0x7fd050102360 2024-09-10T08:10:14.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.805+0000 7fd054a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd05006c4a0 con 0x7fd0501019c0 2024-09-10T08:10:14.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.805+0000 7fd054a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd05006c7a0 con 0x7fd0501079c0 2024-09-10T08:10:14.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.806+0000 7fd04f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0501019c0 0x7fd050075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:14.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.806+0000 7fd04ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd050102360 0x7fd050072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:14.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.806+0000 7fd04f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0501019c0 0x7fd050075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:59026/0 (socket says 172.21.15.2:59026) 2024-09-10T08:10:14.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.806+0000 7fd04f400640 1 -- 172.21.15.2:0/3083968374 learned_addr learned my addr 172.21.15.2:0/3083968374 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:14.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.806+0000 7fd04fe00640 1 --2- 172.21.15.2:0/3083968374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0501079c0 0x7fd0500725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:14.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.806+0000 7fd04f400640 1 -- 172.21.15.2:0/3083968374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0501079c0 msgr2=0x7fd0500725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:14.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.806+0000 7fd04f400640 1 --2- 172.21.15.2:0/3083968374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0501079c0 0x7fd0500725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:14.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.806+0000 7fd04f400640 1 -- 172.21.15.2:0/3083968374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd050102360 msgr2=0x7fd050072040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:14.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.806+0000 7fd04f400640 1 --2- 172.21.15.2:0/3083968374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd050102360 0x7fd050072040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:14.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.806+0000 7fd04f400640 1 -- 172.21.15.2:0/3083968374 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd050072e60 con 0x7fd0501019c0 2024-09-10T08:10:14.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.806+0000 7fd04f400640 1 --2- 172.21.15.2:0/3083968374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0501019c0 0x7fd050075fa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd04000ea60 tx=0x7fd04000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:14.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.807+0000 7fd047e00640 1 -- 172.21.15.2:0/3083968374 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04000cd20 con 0x7fd0501019c0 2024-09-10T08:10:14.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.807+0000 7fd054a00640 1 -- 172.21.15.2:0/3083968374 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0501af1f0 con 0x7fd0501019c0 2024-09-10T08:10:14.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.807+0000 7fd047e00640 1 -- 172.21.15.2:0/3083968374 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0400035f0 con 0x7fd0501019c0 2024-09-10T08:10:14.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.807+0000 7fd047e00640 1 -- 172.21.15.2:0/3083968374 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040010690 con 0x7fd0501019c0 2024-09-10T08:10:14.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.807+0000 7fd054a00640 1 -- 172.21.15.2:0/3083968374 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0501af750 con 0x7fd0501019c0 2024-09-10T08:10:14.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.808+0000 7fd054a00640 1 -- 172.21.15.2:0/3083968374 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd050103100 con 0x7fd0501019c0 2024-09-10T08:10:14.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.810+0000 7fd047e00640 1 -- 172.21.15.2:0/3083968374 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd0400037a0 con 0x7fd0501019c0 2024-09-10T08:10:14.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.811+0000 7fd047e00640 1 --2- 172.21.15.2:0/3083968374 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0340a2c00 0x7fd0340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:14.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.811+0000 7fd047e00640 1 -- 172.21.15.2:0/3083968374 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd0400cf660 con 0x7fd0501019c0 2024-09-10T08:10:14.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.812+0000 7fd04ea00640 1 --2- 172.21.15.2:0/3083968374 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0340a2c00 0x7fd0340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:14.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.812+0000 7fd04ea00640 1 --2- 172.21.15.2:0/3083968374 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0340a2c00 0x7fd0340a50c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd050073800 tx=0x7fd03c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:14.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:14.813+0000 7fd047e00640 1 -- 172.21.15.2:0/3083968374 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd04009c170 con 0x7fd0501019c0 2024-09-10T08:10:14.855 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:14.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:14 smithi002 ceph-mon[30563]: pgmap v450: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:10:15.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.048+0000 7fd054a00640 1 -- 172.21.15.2:0/3083968374 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd050101e20 con 0x7fd0501019c0 2024-09-10T08:10:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.049+0000 7fd047e00640 1 -- 172.21.15.2:0/3083968374 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fd050101e20 con 0x7fd0501019c0 2024-09-10T08:10:15.049 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:10:15.049 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:10:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:10:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.052+0000 7fd054a00640 1 -- 172.21.15.2:0/3083968374 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0340a2c00 msgr2=0x7fd0340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.052+0000 7fd054a00640 1 --2- 172.21.15.2:0/3083968374 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0340a2c00 0x7fd0340a50c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd050073800 tx=0x7fd03c009260 comp rx=0 tx=0).stop 2024-09-10T08:10:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.052+0000 7fd054a00640 1 -- 172.21.15.2:0/3083968374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0501019c0 msgr2=0x7fd050075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.052+0000 7fd054a00640 1 --2- 172.21.15.2:0/3083968374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0501019c0 0x7fd050075fa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd04000ea60 tx=0x7fd04000ee60 comp rx=0 tx=0).stop 2024-09-10T08:10:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.052+0000 7fd054a00640 1 -- 172.21.15.2:0/3083968374 shutdown_connections 2024-09-10T08:10:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.052+0000 7fd054a00640 1 --2- 172.21.15.2:0/3083968374 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0340a2c00 0x7fd0340a50c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.052+0000 7fd054a00640 1 --2- 172.21.15.2:0/3083968374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0501079c0 0x7fd0500725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.052+0000 7fd054a00640 1 --2- 172.21.15.2:0/3083968374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd050102360 0x7fd050072040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.052+0000 7fd054a00640 1 --2- 172.21.15.2:0/3083968374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0501019c0 0x7fd050075fa0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.052+0000 7fd054a00640 1 -- 172.21.15.2:0/3083968374 >> 172.21.15.2:0/3083968374 conn(0x7fd0500fd740 msgr2=0x7fd0500fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.053+0000 7fd054a00640 1 -- 172.21.15.2:0/3083968374 shutdown_connections 2024-09-10T08:10:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.053+0000 7fd054a00640 1 -- 172.21.15.2:0/3083968374 wait complete. 2024-09-10T08:10:15.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:14 smithi155 ceph-mon[35405]: pgmap v450: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:10:15.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:14 smithi107 ceph-mon[35048]: pgmap v450: 129 pgs: 129 active+clean; 680 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:10:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.436+0000 7f4143200640 1 -- 172.21.15.2:0/5279156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4144105720 msgr2=0x7f414410bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.436+0000 7f4143200640 1 --2- 172.21.15.2:0/5279156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4144105720 0x7f414410bfb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f413c009ca0 tx=0x7f413c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.436+0000 7f4143200640 1 -- 172.21.15.2:0/5279156 shutdown_connections 2024-09-10T08:10:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.436+0000 7f4143200640 1 --2- 172.21.15.2:0/5279156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4144105720 0x7f414410bfb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.436+0000 7f4143200640 1 --2- 172.21.15.2:0/5279156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4144104d60 0x7f41441051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.436+0000 7f4143200640 1 --2- 172.21.15.2:0/5279156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4144103b70 0x7f4144103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.436+0000 7f4143200640 1 -- 172.21.15.2:0/5279156 >> 172.21.15.2:0/5279156 conn(0x7f41440ff2f0 msgr2=0x7f4144101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.436+0000 7f4143200640 1 -- 172.21.15.2:0/5279156 shutdown_connections 2024-09-10T08:10:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.437+0000 7f4143200640 1 -- 172.21.15.2:0/5279156 wait complete. 2024-09-10T08:10:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.437+0000 7f4143200640 1 Processor -- start 2024-09-10T08:10:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.437+0000 7f4143200640 1 -- start start 2024-09-10T08:10:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.438+0000 7f4143200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4144103b70 0x7f4144199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.438+0000 7f4143200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4144104d60 0x7f414419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.438+0000 7f4143200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4144105720 0x7f41441a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.438+0000 7f4143200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f414410f7f0 con 0x7f4144103b70 2024-09-10T08:10:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.438+0000 7f4143200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f414410f670 con 0x7f4144104d60 2024-09-10T08:10:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.438+0000 7f4143200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f414410f970 con 0x7f4144105720 2024-09-10T08:10:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.438+0000 7f4141e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4144103b70 0x7f4144199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.438+0000 7f4142800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4144105720 0x7f41441a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.438+0000 7f4141e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4144103b70 0x7f4144199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35958/0 (socket says 172.21.15.2:35958) 2024-09-10T08:10:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.438+0000 7f4141e00640 1 -- 172.21.15.2:0/1700427671 learned_addr learned my addr 172.21.15.2:0/1700427671 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.438+0000 7f4141400640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4144104d60 0x7f414419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.439+0000 7f4141e00640 1 -- 172.21.15.2:0/1700427671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4144105720 msgr2=0x7f41441a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.439+0000 7f4141e00640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4144105720 0x7f41441a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.439+0000 7f4141e00640 1 -- 172.21.15.2:0/1700427671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4144104d60 msgr2=0x7f414419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.439+0000 7f4141e00640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4144104d60 0x7f414419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.439+0000 7f4141e00640 1 -- 172.21.15.2:0/1700427671 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41441a1a60 con 0x7f4144103b70 2024-09-10T08:10:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.439+0000 7f4141400640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4144104d60 0x7f414419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.439+0000 7f4141e00640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4144103b70 0x7f4144199e40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f413400e9c0 tx=0x7f413400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.439+0000 7f413aa00640 1 -- 172.21.15.2:0/1700427671 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413400cd80 con 0x7f4144103b70 2024-09-10T08:10:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.439+0000 7f413aa00640 1 -- 172.21.15.2:0/1700427671 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41340035f0 con 0x7f4144103b70 2024-09-10T08:10:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.439+0000 7f413aa00640 1 -- 172.21.15.2:0/1700427671 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134010690 con 0x7f4144103b70 2024-09-10T08:10:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.439+0000 7f4143200640 1 -- 172.21.15.2:0/1700427671 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41441a1d50 con 0x7f4144103b70 2024-09-10T08:10:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.439+0000 7f4143200640 1 -- 172.21.15.2:0/1700427671 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41441a2190 con 0x7f4144103b70 2024-09-10T08:10:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.442+0000 7f414b9f3740 1 Processor -- start 2024-09-10T08:10:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.442+0000 7f414b9f3740 1 -- start start 2024-09-10T08:10:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.442+0000 7f413aa00640 1 -- 172.21.15.2:0/1700427671 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f41340037a0 con 0x7f4144103b70 2024-09-10T08:10:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f414b9f3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585260 0x559a22585660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f414b9f3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585ba0 0x559a2258a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f414b9f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a2258ac30 0x559a2258d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f414b9f3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559a224ea100 con 0x559a22585260 2024-09-10T08:10:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f414b9f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559a224921b0 con 0x559a2258ac30 2024-09-10T08:10:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f414b9f3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559a224e60c0 con 0x559a22585ba0 2024-09-10T08:10:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f4142800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585260 0x559a22585660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f4142800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585260 0x559a22585660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35974/0 (socket says 172.21.15.2:35974) 2024-09-10T08:10:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f4141400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a2258ac30 0x559a2258d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f4142800640 1 -- 172.21.15.2:0/3839771155 learned_addr learned my addr 172.21.15.2:0/3839771155 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f4141e00640 1 --2- 172.21.15.2:0/3839771155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585ba0 0x559a2258a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f4141400640 1 -- 172.21.15.2:0/3839771155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585ba0 msgr2=0x559a2258a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f4141400640 1 --2- 172.21.15.2:0/3839771155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585ba0 0x559a2258a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f4141400640 1 -- 172.21.15.2:0/3839771155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585260 msgr2=0x559a22585660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f4141400640 1 --2- 172.21.15.2:0/3839771155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585260 0x559a22585660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.443+0000 7f4141400640 1 -- 172.21.15.2:0/3839771155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a221fd210 con 0x559a2258ac30 2024-09-10T08:10:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.444+0000 7f4141400640 1 --2- 172.21.15.2:0/3839771155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a2258ac30 0x559a2258d0f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4130009fa0 tx=0x7f41300118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=519d360b8a1adc23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.444+0000 7f412f000640 1 -- 172.21.15.2:0/3839771155 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130004020 con 0x559a2258ac30 2024-09-10T08:10:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.444+0000 7f413aa00640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41280a2c00 0x7f41280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.444+0000 7f412f000640 1 -- 172.21.15.2:0/3839771155 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41300041c0 con 0x559a2258ac30 2024-09-10T08:10:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.444+0000 7f412f000640 1 -- 172.21.15.2:0/3839771155 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413000f8f0 con 0x559a2258ac30 2024-09-10T08:10:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.444+0000 7f413aa00640 1 -- 172.21.15.2:0/1700427671 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f41340cf770 con 0x7f4144103b70 2024-09-10T08:10:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.444+0000 7f4142800640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41280a2c00 0x7f41280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.445+0000 7f414b9f3740 1 -- 172.21.15.2:0/3839771155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a2258ac30 msgr2=0x559a2258d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.445+0000 7f414b9f3740 1 --2- 172.21.15.2:0/3839771155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a2258ac30 0x559a2258d0f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4130009fa0 tx=0x7f41300118d0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.445+0000 7f414b9f3740 1 -- 172.21.15.2:0/3839771155 shutdown_connections 2024-09-10T08:10:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.445+0000 7f414b9f3740 1 --2- 172.21.15.2:0/3839771155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a2258ac30 0x559a2258d0f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.445+0000 7f414b9f3740 1 --2- 172.21.15.2:0/3839771155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585ba0 0x559a2258a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.445+0000 7f414b9f3740 1 --2- 172.21.15.2:0/3839771155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585260 0x559a22585660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.445+0000 7f414b9f3740 1 -- 172.21.15.2:0/3839771155 >> 172.21.15.2:0/3839771155 conn(0x559a225808d0 msgr2=0x559a22580cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.445+0000 7f414b9f3740 1 -- 172.21.15.2:0/3839771155 shutdown_connections 2024-09-10T08:10:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.445+0000 7f4142800640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41280a2c00 0x7f41280a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f413c005e30 tx=0x7f413c03d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.445+0000 7f414b9f3740 1 -- 172.21.15.2:0/3839771155 wait complete. 2024-09-10T08:10:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.445+0000 7f414b9f3740 1 -- start start 2024-09-10T08:10:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.446+0000 7f414b9f3740 1 Processor -- start 2024-09-10T08:10:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.446+0000 7f414b9f3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585260 0x559a2258f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.446+0000 7f414b9f3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585ba0 0x559a2258f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.446+0000 7f414b9f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a2258ac30 0x559a22596960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.446+0000 7f414b9f3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559a2258dc00 con 0x559a22585ba0 2024-09-10T08:10:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.446+0000 7f414b9f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559a22473090 con 0x559a2258ac30 2024-09-10T08:10:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.446+0000 7f414b9f3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559a2258dd80 con 0x559a22585260 2024-09-10T08:10:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.446+0000 7f414b9f3740 10 client.0 _ll_register_callbacks cb 0x559a224c7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.446+0000 7f4141400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585ba0 0x559a2258f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.446+0000 7f4141400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585ba0 0x559a2258f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35976/0 (socket says 172.21.15.2:35976) 2024-09-10T08:10:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.446+0000 7f4141400640 1 -- 172.21.15.2:0/371335676 learned_addr learned my addr 172.21.15.2:0/371335676 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.446+0000 7f4142800640 1 --2- 172.21.15.2:0/371335676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a2258ac30 0x559a22596960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.446+0000 7f4141e00640 1 --2- 172.21.15.2:0/371335676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585260 0x559a2258f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.447+0000 7f4141400640 1 -- 172.21.15.2:0/371335676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585260 msgr2=0x559a2258f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.447+0000 7f4141400640 1 --2- 172.21.15.2:0/371335676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585260 0x559a2258f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.447+0000 7f4141400640 1 -- 172.21.15.2:0/371335676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a2258ac30 msgr2=0x559a22596960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.447+0000 7f4141400640 1 --2- 172.21.15.2:0/371335676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a2258ac30 0x559a22596960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.447+0000 7f4141400640 1 -- 172.21.15.2:0/371335676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a222c9620 con 0x559a22585ba0 2024-09-10T08:10:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.447+0000 7f4141400640 1 --2- 172.21.15.2:0/371335676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585ba0 0x559a2258f920 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f413001b040 tx=0x7f4130003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3097c359b511365 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.447+0000 7f4123400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.447+0000 7f4123400640 1 -- 172.21.15.2:0/371335676 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413001d050 con 0x559a22585ba0 2024-09-10T08:10:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.447+0000 7f4123400640 1 -- 172.21.15.2:0/371335676 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4130003fc0 con 0x559a22585ba0 2024-09-10T08:10:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.447+0000 7f414b9f3740 1 -- 172.21.15.2:0/371335676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41300041c0 con 0x559a22585ba0 2024-09-10T08:10:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.448+0000 7f4123400640 1 -- 172.21.15.2:0/371335676 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4130032050 con 0x559a22585ba0 2024-09-10T08:10:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.448+0000 7f414b9f3740 10 client.15765 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.448+0000 7f414b9f3740 1 -- 172.21.15.2:0/371335676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559a22599750 con 0x559a22585ba0 2024-09-10T08:10:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.448+0000 7f4123400640 1 -- 172.21.15.2:0/371335676 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41300041c0 con 0x559a22585ba0 2024-09-10T08:10:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.448+0000 7f4123400640 1 -- 172.21.15.2:0/371335676 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f413000f610 con 0x559a22585ba0 2024-09-10T08:10:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.448+0000 7f414b9f3740 10 client.15765 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.448+0000 7f414b9f3740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.448+0000 7f414b9f3740 20 client.15765 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.448+0000 7f414b9f3740 1 --2- 172.21.15.2:0/371335676 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559a2259ac20 0x559a2259d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.448+0000 7f414b9f3740 4 client.15765 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.448+0000 7f414b9f3740 1 -- 172.21.15.2:0/371335676 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a224f7830 con 0x559a2259ac20 2024-09-10T08:10:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.449+0000 7f4141e00640 1 --2- 172.21.15.2:0/371335676 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559a2259ac20 0x559a2259d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.449+0000 7f4141e00640 1 --2- 172.21.15.2:0/371335676 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559a2259ac20 0x559a2259d010 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=614ef00085a34db4 server_cookie=8b86536c1186bbe1 in_seq=0 out_seq=0 2024-09-10T08:10:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.449+0000 7f4123400640 10 client.15765 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.450+0000 7f4123400640 1 -- 172.21.15.2:0/371335676 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a224f7830 con 0x559a2259ac20 2024-09-10T08:10:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.450+0000 7f4123400640 10 client.15765 handle_command_reply: tid=0 2024-09-10T08:10:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.451+0000 7f414b9f3740 1 client.15765 shutdown 2024-09-10T08:10:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.451+0000 7f414b9f3740 1 -- 172.21.15.2:0/371335676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585ba0 msgr2=0x559a2258f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.451+0000 7f414b9f3740 1 --2- 172.21.15.2:0/371335676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585ba0 0x559a2258f920 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f413001b040 tx=0x7f4130003c90 comp rx=0 tx=0).stop 2024-09-10T08:10:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.451+0000 7f414b9f3740 1 -- 172.21.15.2:0/371335676 shutdown_connections 2024-09-10T08:10:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.451+0000 7f414b9f3740 1 --2- 172.21.15.2:0/371335676 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559a2259ac20 0x559a2259d010 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.451+0000 7f414b9f3740 1 --2- 172.21.15.2:0/371335676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a2258ac30 0x559a22596960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.451+0000 7f414b9f3740 1 --2- 172.21.15.2:0/371335676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585ba0 0x559a2258f920 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.451+0000 7f4123400640 0 client.15765 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.451+0000 7f414b9f3740 1 --2- 172.21.15.2:0/371335676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585260 0x559a2258f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.451+0000 7f414b9f3740 1 -- 172.21.15.2:0/371335676 >> 172.21.15.2:0/371335676 conn(0x559a225808d0 msgr2=0x559a225820e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.451+0000 7f414b9f3740 1 -- 172.21.15.2:0/371335676 shutdown_connections 2024-09-10T08:10:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.451+0000 7f414b9f3740 1 -- 172.21.15.2:0/371335676 wait complete. 2024-09-10T08:10:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.451+0000 7f414b9f3740 20 client.15765 trim_cache size 0 max 16384 2024-09-10T08:10:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.466+0000 7f414b9f3740 1 Processor -- start 2024-09-10T08:10:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.466+0000 7f414b9f3740 1 -- start start 2024-09-10T08:10:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f414b9f3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585260 0x559a2257f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f414b9f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a22585ba0 0x559a2257fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f414b9f3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a2258ac30 0x559a22596920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f414b9f3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f41441af1c0 con 0x559a22585260 2024-09-10T08:10:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f414b9f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f41441aca90 con 0x559a22585ba0 2024-09-10T08:10:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f414b9f3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41441aa8e0 con 0x559a2258ac30 2024-09-10T08:10:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f4141e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585260 0x559a2257f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f4141e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585260 0x559a2257f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35990/0 (socket says 172.21.15.2:35990) 2024-09-10T08:10:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f4141e00640 1 -- 172.21.15.2:0/407089626 learned_addr learned my addr 172.21.15.2:0/407089626 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f4141400640 1 --2- 172.21.15.2:0/407089626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a22585ba0 0x559a2257fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f4142800640 1 --2- 172.21.15.2:0/407089626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a2258ac30 0x559a22596920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f4141e00640 1 -- 172.21.15.2:0/407089626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a2258ac30 msgr2=0x559a22596920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f4141e00640 1 --2- 172.21.15.2:0/407089626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a2258ac30 0x559a22596920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f4141e00640 1 -- 172.21.15.2:0/407089626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a22585ba0 msgr2=0x559a2257fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f4141e00640 1 --2- 172.21.15.2:0/407089626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a22585ba0 0x559a2257fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f4141e00640 1 -- 172.21.15.2:0/407089626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a2259d380 con 0x559a22585260 2024-09-10T08:10:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f4141400640 1 --2- 172.21.15.2:0/407089626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a22585ba0 0x559a2257fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.467+0000 7f4141e00640 1 --2- 172.21.15.2:0/407089626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585260 0x559a2257f900 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f413409bb10 tx=0x7f41340045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e8ade37040772 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.468+0000 7f412e600640 1 -- 172.21.15.2:0/407089626 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413409bb80 con 0x559a22585260 2024-09-10T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.468+0000 7f412e600640 1 -- 172.21.15.2:0/407089626 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f413409ff30 con 0x559a22585260 2024-09-10T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.468+0000 7f412e600640 1 -- 172.21.15.2:0/407089626 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134003c30 con 0x559a22585260 2024-09-10T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.468+0000 7f414b9f3740 1 -- 172.21.15.2:0/407089626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585260 msgr2=0x559a2257f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.468+0000 7f414b9f3740 1 --2- 172.21.15.2:0/407089626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585260 0x559a2257f900 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f413409bb10 tx=0x7f41340045f0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.468+0000 7f414b9f3740 1 -- 172.21.15.2:0/407089626 shutdown_connections 2024-09-10T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.468+0000 7f414b9f3740 1 --2- 172.21.15.2:0/407089626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a2258ac30 0x559a22596920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.468+0000 7f414b9f3740 1 --2- 172.21.15.2:0/407089626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a22585ba0 0x559a2257fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.468+0000 7f414b9f3740 1 --2- 172.21.15.2:0/407089626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a22585260 0x559a2257f900 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.468+0000 7f414b9f3740 1 -- 172.21.15.2:0/407089626 >> 172.21.15.2:0/407089626 conn(0x559a225808d0 msgr2=0x559a2257f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.468+0000 7f414b9f3740 1 -- 172.21.15.2:0/407089626 shutdown_connections 2024-09-10T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.468+0000 7f414b9f3740 1 -- 172.21.15.2:0/407089626 wait complete. 2024-09-10T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.469+0000 7f414b9f3740 1 -- start start 2024-09-10T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.469+0000 7f414b9f3740 1 Processor -- start 2024-09-10T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.469+0000 7f414b9f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a22585260 0x559a22593830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.469+0000 7f414b9f3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585ba0 0x559a22593d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.469+0000 7f414b9f3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a2258ac30 0x559a226038c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.469+0000 7f414b9f3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f41441aaf50 con 0x559a2258ac30 2024-09-10T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.469+0000 7f414b9f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f41441ac3c0 con 0x559a22585260 2024-09-10T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.469+0000 7f414b9f3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41441ab620 con 0x559a22585ba0 2024-09-10T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.469+0000 7f414b9f3740 10 client.0 _ll_register_callbacks cb 0x7f4134005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4141e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a22585260 0x559a22593830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4142800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a2258ac30 0x559a226038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4141e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a22585260 0x559a22593830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:59092/0 (socket says 172.21.15.2:59092) 2024-09-10T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4141e00640 1 -- 172.21.15.2:0/414714285 learned_addr learned my addr 172.21.15.2:0/414714285 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4142800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a2258ac30 0x559a226038c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35994/0 (socket says 172.21.15.2:35994) 2024-09-10T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4141400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585ba0 0x559a22593d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4141e00640 1 -- 172.21.15.2:0/414714285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585ba0 msgr2=0x559a22593d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4141e00640 1 --2- 172.21.15.2:0/414714285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585ba0 0x559a22593d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4141e00640 1 -- 172.21.15.2:0/414714285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a2258ac30 msgr2=0x559a226038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4141e00640 1 --2- 172.21.15.2:0/414714285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a2258ac30 0x559a226038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4141e00640 1 -- 172.21.15.2:0/414714285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a22603e30 con 0x559a22585260 2024-09-10T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4141400640 1 --2- 172.21.15.2:0/414714285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585ba0 0x559a22593d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4142800640 1 --2- 172.21.15.2:0/414714285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a2258ac30 0x559a226038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4141e00640 1 --2- 172.21.15.2:0/414714285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a22585260 0x559a22593830 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f413400c6a0 tx=0x7f413409c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fa3ace7a6cb77a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4123400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.470+0000 7f4123400640 1 -- 172.21.15.2:0/414714285 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41340ba050 con 0x559a22585260 2024-09-10T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.471+0000 7f4123400640 1 -- 172.21.15.2:0/414714285 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f41340cf3c0 con 0x559a22585260 2024-09-10T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.471+0000 7f414b9f3740 1 -- 172.21.15.2:0/414714285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f413409ff30 con 0x559a22585260 2024-09-10T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.471+0000 7f4123400640 1 -- 172.21.15.2:0/414714285 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f413409bb80 con 0x559a22585260 2024-09-10T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.471+0000 7f414b9f3740 10 client.25805 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.471+0000 7f414b9f3740 1 -- 172.21.15.2:0/414714285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559a22605190 con 0x559a22585260 2024-09-10T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.471+0000 7f4123400640 1 -- 172.21.15.2:0/414714285 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f413409ff30 con 0x559a22585260 2024-09-10T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.471+0000 7f4123400640 1 -- 172.21.15.2:0/414714285 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f41340d2ce0 con 0x559a22585260 2024-09-10T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.471+0000 7f414b9f3740 10 client.25805 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.471+0000 7f414b9f3740 10 client.25805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.471+0000 7f414b9f3740 20 client.25805 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.471+0000 7f414b9f3740 1 --2- 172.21.15.2:0/414714285 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559a22605b10 0x559a22607f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.471+0000 7f414b9f3740 4 client.25805 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:10:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.471+0000 7f414b9f3740 1 -- 172.21.15.2:0/414714285 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a224f1b80 con 0x559a22605b10 2024-09-10T08:10:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.472+0000 7f4141400640 1 --2- 172.21.15.2:0/414714285 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559a22605b10 0x559a22607f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.472+0000 7f4141400640 1 --2- 172.21.15.2:0/414714285 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559a22605b10 0x559a22607f30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c6375a4d9f53888 server_cookie=630d18895558e072 in_seq=0 out_seq=0 2024-09-10T08:10:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.472+0000 7f4123400640 10 client.25805 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.475+0000 7f4123400640 1 -- 172.21.15.2:0/414714285 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a224f1b80 con 0x559a22605b10 2024-09-10T08:10:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f4123400640 10 client.25805 handle_command_reply: tid=0 2024-09-10T08:10:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f414b9f3740 1 client.25805 shutdown 2024-09-10T08:10:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f414b9f3740 1 -- 172.21.15.2:0/414714285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a22585260 msgr2=0x559a22593830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f414b9f3740 1 --2- 172.21.15.2:0/414714285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a22585260 0x559a22593830 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f413400c6a0 tx=0x7f413409c340 comp rx=0 tx=0).stop 2024-09-10T08:10:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f414b9f3740 1 -- 172.21.15.2:0/414714285 shutdown_connections 2024-09-10T08:10:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f414b9f3740 1 --2- 172.21.15.2:0/414714285 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559a22605b10 0x559a22607f30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f414b9f3740 1 --2- 172.21.15.2:0/414714285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a2258ac30 0x559a226038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f414b9f3740 1 --2- 172.21.15.2:0/414714285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a22585ba0 0x559a22593d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f414b9f3740 1 --2- 172.21.15.2:0/414714285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a22585260 0x559a22593830 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f414b9f3740 1 -- 172.21.15.2:0/414714285 >> 172.21.15.2:0/414714285 conn(0x559a225808d0 msgr2=0x559a2257fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f4123400640 0 client.25805 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f414b9f3740 1 -- 172.21.15.2:0/414714285 shutdown_connections 2024-09-10T08:10:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f414b9f3740 1 -- 172.21.15.2:0/414714285 wait complete. 2024-09-10T08:10:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.476+0000 7f414b9f3740 20 client.25805 trim_cache size 0 max 16384 2024-09-10T08:10:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:10:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:10:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:10:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:10:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.477+0000 7f4143200640 1 -- 172.21.15.2:0/1700427671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41280a2c00 msgr2=0x7f41280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.477+0000 7f4143200640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41280a2c00 0x7f41280a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f413c005e30 tx=0x7f413c03d040 comp rx=0 tx=0).stop 2024-09-10T08:10:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.477+0000 7f4143200640 1 -- 172.21.15.2:0/1700427671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4144103b70 msgr2=0x7f4144199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.477+0000 7f4143200640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4144103b70 0x7f4144199e40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f413400e9c0 tx=0x7f413400ee80 comp rx=0 tx=0).stop 2024-09-10T08:10:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.477+0000 7f4143200640 1 -- 172.21.15.2:0/1700427671 shutdown_connections 2024-09-10T08:10:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.477+0000 7f4143200640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41280a2c00 0x7f41280a50c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.477+0000 7f4143200640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4144105720 0x7f41441a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.477+0000 7f4143200640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4144104d60 0x7f414419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.477+0000 7f4143200640 1 --2- 172.21.15.2:0/1700427671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4144103b70 0x7f4144199e40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.477+0000 7f4143200640 1 -- 172.21.15.2:0/1700427671 >> 172.21.15.2:0/1700427671 conn(0x7f41440ff2f0 msgr2=0x7f4144108080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.477+0000 7f4143200640 1 -- 172.21.15.2:0/1700427671 shutdown_connections 2024-09-10T08:10:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:15.477+0000 7f4143200640 1 -- 172.21.15.2:0/1700427671 wait complete. 2024-09-10T08:10:15.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.983+0000 7f2d99600640 1 -- 172.21.15.2:0/2720670173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d94076720 msgr2=0x7f2d94076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.983+0000 7f2d99600640 1 --2- 172.21.15.2:0/2720670173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d94076720 0x7f2d94076b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009900 tx=0x7f2d8402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.984+0000 7f2d99600640 1 -- 172.21.15.2:0/2720670173 shutdown_connections 2024-09-10T08:10:15.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.984+0000 7f2d99600640 1 --2- 172.21.15.2:0/2720670173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d94112a30 0x7f2d94114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.984+0000 7f2d99600640 1 --2- 172.21.15.2:0/2720670173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d94077040 0x7f2d941124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.984+0000 7f2d99600640 1 --2- 172.21.15.2:0/2720670173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d94076720 0x7f2d94076b00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.984+0000 7f2d99600640 1 -- 172.21.15.2:0/2720670173 >> 172.21.15.2:0/2720670173 conn(0x7f2d940ffa90 msgr2=0x7f2d94101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:15.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.984+0000 7f2d99600640 1 -- 172.21.15.2:0/2720670173 shutdown_connections 2024-09-10T08:10:15.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.984+0000 7f2d99600640 1 -- 172.21.15.2:0/2720670173 wait complete. 2024-09-10T08:10:15.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.985+0000 7f2d99600640 1 Processor -- start 2024-09-10T08:10:15.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.985+0000 7f2d99600640 1 -- start start 2024-09-10T08:10:15.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.985+0000 7f2d99600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d94076720 0x7f2d941a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.985+0000 7f2d99600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d94077040 0x7f2d941a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.985+0000 7f2d99600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d94112a30 0x7f2d9419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.985+0000 7f2d99600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94118610 con 0x7f2d94112a30 2024-09-10T08:10:15.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.985+0000 7f2d99600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94118490 con 0x7f2d94076720 2024-09-10T08:10:15.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.985+0000 7f2d99600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94118790 con 0x7f2d94077040 2024-09-10T08:10:15.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.986+0000 7f2d92a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d94076720 0x7f2d941a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.986+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d94077040 0x7f2d941a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.986+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d94077040 0x7f2d941a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42836/0 (socket says 172.21.15.2:42836) 2024-09-10T08:10:15.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.986+0000 7f2d8be00640 1 -- 172.21.15.2:0/3411960517 learned_addr learned my addr 172.21.15.2:0/3411960517 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:15.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.986+0000 7f2d93400640 1 --2- 172.21.15.2:0/3411960517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d94112a30 0x7f2d9419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.986+0000 7f2d8be00640 1 -- 172.21.15.2:0/3411960517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d94076720 msgr2=0x7f2d941a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.986+0000 7f2d8be00640 1 --2- 172.21.15.2:0/3411960517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d94076720 0x7f2d941a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.986+0000 7f2d8be00640 1 -- 172.21.15.2:0/3411960517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d94112a30 msgr2=0x7f2d9419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.986+0000 7f2d8be00640 1 --2- 172.21.15.2:0/3411960517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d94112a30 0x7f2d9419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.986+0000 7f2d8be00640 1 -- 172.21.15.2:0/3411960517 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9419f010 con 0x7f2d94077040 2024-09-10T08:10:15.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.987+0000 7f2d8be00640 1 --2- 172.21.15.2:0/3411960517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d94077040 0x7f2d941a4c60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00e9f0 tx=0x7f2d7c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:15.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.987+0000 7f2d8b400640 1 -- 172.21.15.2:0/3411960517 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c00ccc0 con 0x7f2d94077040 2024-09-10T08:10:15.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.987+0000 7f2d8b400640 1 -- 172.21.15.2:0/3411960517 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d7c00ce60 con 0x7f2d94077040 2024-09-10T08:10:15.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.987+0000 7f2d99600640 1 -- 172.21.15.2:0/3411960517 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9419f300 con 0x7f2d94077040 2024-09-10T08:10:15.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.987+0000 7f2d8b400640 1 -- 172.21.15.2:0/3411960517 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c010690 con 0x7f2d94077040 2024-09-10T08:10:15.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.987+0000 7f2d99600640 1 -- 172.21.15.2:0/3411960517 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d941ab5f0 con 0x7f2d94077040 2024-09-10T08:10:15.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.991+0000 7f2d9c229740 1 Processor -- start 2024-09-10T08:10:15.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.991+0000 7f2d9c229740 1 -- start start 2024-09-10T08:10:15.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.992+0000 7f2d8b400640 1 -- 172.21.15.2:0/3411960517 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2d7c0046c0 con 0x7f2d94077040 2024-09-10T08:10:15.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.992+0000 7f2d9c229740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f6fc70 0x562aa9f70050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.992+0000 7f2d9c229740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f70590 0x562aa9f7a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.992+0000 7f2d9c229740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 0x562aa9f7d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.992+0000 7f2d9c229740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562aa9ed63e0 con 0x562aa9f70590 2024-09-10T08:10:15.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.992+0000 7f2d9c229740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562aa9e7dd80 con 0x562aa9f7ae50 2024-09-10T08:10:15.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.992+0000 7f2d9c229740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562aa9ed23a0 con 0x562aa9f6fc70 2024-09-10T08:10:15.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.992+0000 7f2d92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f70590 0x562aa9f7a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.992+0000 7f2d92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f70590 0x562aa9f7a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36012/0 (socket says 172.21.15.2:36012) 2024-09-10T08:10:15.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.992+0000 7f2d93400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f6fc70 0x562aa9f70050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.992+0000 7f2d92a00640 1 -- 172.21.15.2:0/2478033251 learned_addr learned my addr 172.21.15.2:0/2478033251 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:15.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.992+0000 7f2d8be00640 1 --2- 172.21.15.2:0/2478033251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 0x562aa9f7d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.993+0000 7f2d92a00640 1 -- 172.21.15.2:0/2478033251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f6fc70 msgr2=0x562aa9f70050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:15.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.993+0000 7f2d92a00640 1 --2- 172.21.15.2:0/2478033251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f6fc70 0x562aa9f70050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.993+0000 7f2d92a00640 1 -- 172.21.15.2:0/2478033251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 msgr2=0x562aa9f7d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:15.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.993+0000 7f2d92a00640 1 --2- 172.21.15.2:0/2478033251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 0x562aa9f7d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.993+0000 7f2d92a00640 1 -- 172.21.15.2:0/2478033251 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aa9e9cae0 con 0x562aa9f70590 2024-09-10T08:10:15.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.993+0000 7f2d8be00640 1 --2- 172.21.15.2:0/2478033251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 0x562aa9f7d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:15.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.993+0000 7f2d92a00640 1 --2- 172.21.15.2:0/2478033251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f70590 0x562aa9f7a910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009900 tx=0x7f2d84005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7558221c08a37c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:15.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.993+0000 7f2d83400640 1 -- 172.21.15.2:0/2478033251 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84035040 con 0x562aa9f70590 2024-09-10T08:10:15.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.993+0000 7f2d83400640 1 -- 172.21.15.2:0/2478033251 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d84005520 con 0x562aa9f70590 2024-09-10T08:10:15.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.993+0000 7f2d83400640 1 -- 172.21.15.2:0/2478033251 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d840035f0 con 0x562aa9f70590 2024-09-10T08:10:15.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.994+0000 7f2d8b400640 1 --2- 172.21.15.2:0/3411960517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d780a2c00 0x7f2d780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.994+0000 7f2d8b400640 1 -- 172.21.15.2:0/3411960517 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2d7c0cf7f0 con 0x7f2d94077040 2024-09-10T08:10:15.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.994+0000 7f2d93400640 1 --2- 172.21.15.2:0/3411960517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d780a2c00 0x7f2d780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:15.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.994+0000 7f2d9c229740 1 -- 172.21.15.2:0/2478033251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f70590 msgr2=0x562aa9f7a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:15.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.994+0000 7f2d9c229740 1 --2- 172.21.15.2:0/2478033251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f70590 0x562aa9f7a910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009900 tx=0x7f2d84005de0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.994+0000 7f2d9c229740 1 -- 172.21.15.2:0/2478033251 shutdown_connections 2024-09-10T08:10:15.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.994+0000 7f2d9c229740 1 --2- 172.21.15.2:0/2478033251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 0x562aa9f7d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.994+0000 7f2d9c229740 1 --2- 172.21.15.2:0/2478033251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f70590 0x562aa9f7a910 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.994+0000 7f2d9c229740 1 --2- 172.21.15.2:0/2478033251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f6fc70 0x562aa9f70050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:15.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.994+0000 7f2d9c229740 1 -- 172.21.15.2:0/2478033251 >> 172.21.15.2:0/2478033251 conn(0x562aa9f6c200 msgr2=0x562aa9f6c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:15.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.994+0000 7f2d9c229740 1 -- 172.21.15.2:0/2478033251 shutdown_connections 2024-09-10T08:10:15.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.995+0000 7f2d9c229740 1 -- 172.21.15.2:0/2478033251 wait complete. 2024-09-10T08:10:15.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.995+0000 7f2d93400640 1 --2- 172.21.15.2:0/3411960517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d780a2c00 0x7f2d780a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2d9406b460 tx=0x7f2d8c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:15.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.995+0000 7f2d9c229740 1 -- start start 2024-09-10T08:10:15.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.995+0000 7f2d9c229740 1 Processor -- start 2024-09-10T08:10:15.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.996+0000 7f2d9c229740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f77ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.996+0000 7f2d9c229740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f70590 0x562aa9f78220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.996+0000 7f2d9c229740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 0x562aa9f78760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:15.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.996+0000 7f2d9c229740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562aa9f7da20 con 0x562aa9f6fc70 2024-09-10T08:10:16.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.996+0000 7f2d9c229740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562aa9e910f0 con 0x562aa9f7ae50 2024-09-10T08:10:16.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.996+0000 7f2d9c229740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562aa9f7dba0 con 0x562aa9f70590 2024-09-10T08:10:16.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.996+0000 7f2d9c229740 10 client.0 _ll_register_callbacks cb 0x562aa9e78bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:16.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.996+0000 7f2d92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f77ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:16.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.996+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f70590 0x562aa9f78220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:16.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.996+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f70590 0x562aa9f78220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42846/0 (socket says 172.21.15.2:42846) 2024-09-10T08:10:16.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.996+0000 7f2d92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f77ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36020/0 (socket says 172.21.15.2:36020) 2024-09-10T08:10:16.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.996+0000 7f2d8be00640 1 -- 172.21.15.2:0/3217421666 learned_addr learned my addr 172.21.15.2:0/3217421666 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:16.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.996+0000 7f2d93400640 1 --2- 172.21.15.2:0/3217421666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 0x562aa9f78760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:16.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.997+0000 7f2d8be00640 1 -- 172.21.15.2:0/3217421666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 msgr2=0x562aa9f78760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:16.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.997+0000 7f2d8be00640 1 --2- 172.21.15.2:0/3217421666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 0x562aa9f78760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.997+0000 7f2d8be00640 1 -- 172.21.15.2:0/3217421666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 msgr2=0x562aa9f77ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:16.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.997+0000 7f2d8be00640 1 --2- 172.21.15.2:0/3217421666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f77ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.997+0000 7f2d8be00640 1 -- 172.21.15.2:0/3217421666 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aa9c473c0 con 0x562aa9f70590 2024-09-10T08:10:16.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.997+0000 7f2d8be00640 1 --2- 172.21.15.2:0/3217421666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f70590 0x562aa9f78220 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c09b900 tx=0x7f2d7c09b620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ba6dce521460b54 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:16.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.997+0000 7f2d77e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:10:16.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.997+0000 7f2d77e00640 1 -- 172.21.15.2:0/3217421666 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0b2060 con 0x562aa9f70590 2024-09-10T08:10:16.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.997+0000 7f2d77e00640 1 -- 172.21.15.2:0/3217421666 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2d7c0cfde0 con 0x562aa9f70590 2024-09-10T08:10:16.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.997+0000 7f2d9c229740 1 -- 172.21.15.2:0/3217421666 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d84005520 con 0x562aa9f70590 2024-09-10T08:10:16.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.997+0000 7f2d77e00640 1 -- 172.21.15.2:0/3217421666 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d7c09ffb0 con 0x562aa9f70590 2024-09-10T08:10:16.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.998+0000 7f2d9c229740 10 client.25795 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:16.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.998+0000 7f2d9c229740 1 -- 172.21.15.2:0/3217421666 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562aa9f89a50 con 0x562aa9f70590 2024-09-10T08:10:16.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.998+0000 7f2d77e00640 1 -- 172.21.15.2:0/3217421666 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d84005520 con 0x562aa9f70590 2024-09-10T08:10:16.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.998+0000 7f2d77e00640 1 -- 172.21.15.2:0/3217421666 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2d7c09b1e0 con 0x562aa9f70590 2024-09-10T08:10:16.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.998+0000 7f2d9c229740 10 client.25795 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:16.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.998+0000 7f2d9c229740 10 client.25795 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:10:16.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.998+0000 7f2d9c229740 20 client.25795 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:16.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.998+0000 7f2d9c229740 1 --2- 172.21.15.2:0/3217421666 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562aa9f8ada0 0x562aa9f8d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:16.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.998+0000 7f2d9c229740 4 client.25795 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:16.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.998+0000 7f2d9c229740 1 -- 172.21.15.2:0/3217421666 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aa9ee3950 con 0x562aa9f8ada0 2024-09-10T08:10:16.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.999+0000 7f2d92a00640 1 --2- 172.21.15.2:0/3217421666 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562aa9f8ada0 0x562aa9f8d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:16.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.999+0000 7f2d92a00640 1 --2- 172.21.15.2:0/3217421666 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562aa9f8ada0 0x562aa9f8d210 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7739c915889be341 server_cookie=25dd4f9331b3f0e in_seq=0 out_seq=0 2024-09-10T08:10:16.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:15.999+0000 7f2d77e00640 10 client.25795 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:16.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.000+0000 7f2d77e00640 1 -- 172.21.15.2:0/3217421666 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562aa9ee3950 con 0x562aa9f8ada0 2024-09-10T08:10:16.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.000+0000 7f2d77e00640 10 client.25795 handle_command_reply: tid=0 2024-09-10T08:10:16.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.000+0000 7f2d9c229740 1 client.25795 shutdown 2024-09-10T08:10:16.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.000+0000 7f2d9c229740 1 -- 172.21.15.2:0/3217421666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f70590 msgr2=0x562aa9f78220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:16.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.000+0000 7f2d9c229740 1 --2- 172.21.15.2:0/3217421666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f70590 0x562aa9f78220 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c09b900 tx=0x7f2d7c09b620 comp rx=0 tx=0).stop 2024-09-10T08:10:16.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.000+0000 7f2d9c229740 1 -- 172.21.15.2:0/3217421666 shutdown_connections 2024-09-10T08:10:16.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.000+0000 7f2d9c229740 1 --2- 172.21.15.2:0/3217421666 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562aa9f8ada0 0x562aa9f8d210 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.000+0000 7f2d9c229740 1 --2- 172.21.15.2:0/3217421666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 0x562aa9f78760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.000+0000 7f2d9c229740 1 --2- 172.21.15.2:0/3217421666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f70590 0x562aa9f78220 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.000+0000 7f2d9c229740 1 --2- 172.21.15.2:0/3217421666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f77ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.000+0000 7f2d77e00640 0 client.25795 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:16.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.000+0000 7f2d9c229740 1 -- 172.21.15.2:0/3217421666 >> 172.21.15.2:0/3217421666 conn(0x562aa9f6c200 msgr2=0x562aa9f6b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:16.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.001+0000 7f2d9c229740 1 -- 172.21.15.2:0/3217421666 shutdown_connections 2024-09-10T08:10:16.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.001+0000 7f2d9c229740 1 -- 172.21.15.2:0/3217421666 wait complete. 2024-09-10T08:10:16.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.001+0000 7f2d9c229740 20 client.25795 trim_cache size 0 max 16384 2024-09-10T08:10:16.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.014+0000 7f2d9c229740 1 Processor -- start 2024-09-10T08:10:16.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.014+0000 7f2d9c229740 1 -- start start 2024-09-10T08:10:16.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.015+0000 7f2d9c229740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f92ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:16.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.015+0000 7f2d9c229740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f70590 0x562aa9f92ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:16.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.015+0000 7f2d9c229740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f7ae50 0x562aa9f74f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:16.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.015+0000 7f2d9c229740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941b7cb0 con 0x562aa9f6fc70 2024-09-10T08:10:16.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.015+0000 7f2d9c229740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941b5580 con 0x562aa9f70590 2024-09-10T08:10:16.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.015+0000 7f2d9c229740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941b33d0 con 0x562aa9f7ae50 2024-09-10T08:10:16.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.015+0000 7f2d92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f92ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:16.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.015+0000 7f2d92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f92ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36032/0 (socket says 172.21.15.2:36032) 2024-09-10T08:10:16.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.015+0000 7f2d92a00640 1 -- 172.21.15.2:0/2188623781 learned_addr learned my addr 172.21.15.2:0/2188623781 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:16.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.015+0000 7f2d8be00640 1 --2- 172.21.15.2:0/2188623781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f70590 0x562aa9f92ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:16.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.015+0000 7f2d93400640 1 --2- 172.21.15.2:0/2188623781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f7ae50 0x562aa9f74f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:16.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.016+0000 7f2d92a00640 1 -- 172.21.15.2:0/2188623781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f7ae50 msgr2=0x562aa9f74f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:16.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.016+0000 7f2d92a00640 1 --2- 172.21.15.2:0/2188623781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f7ae50 0x562aa9f74f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.016+0000 7f2d92a00640 1 -- 172.21.15.2:0/2188623781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f70590 msgr2=0x562aa9f92ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:16.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.016+0000 7f2d92a00640 1 --2- 172.21.15.2:0/2188623781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f70590 0x562aa9f92ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.016+0000 7f2d92a00640 1 -- 172.21.15.2:0/2188623781 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aa9f70b00 con 0x562aa9f6fc70 2024-09-10T08:10:16.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.016+0000 7f2d8be00640 1 --2- 172.21.15.2:0/2188623781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f70590 0x562aa9f92ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:16.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.016+0000 7f2d92a00640 1 --2- 172.21.15.2:0/2188623781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f92ab0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009900 tx=0x7f2d84000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cf87964a62a1d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:16.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.016+0000 7f2d82a00640 1 -- 172.21.15.2:0/2188623781 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84035040 con 0x562aa9f6fc70 2024-09-10T08:10:16.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.016+0000 7f2d82a00640 1 -- 172.21.15.2:0/2188623781 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d840042c0 con 0x562aa9f6fc70 2024-09-10T08:10:16.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.016+0000 7f2d82a00640 1 -- 172.21.15.2:0/2188623781 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d840038e0 con 0x562aa9f6fc70 2024-09-10T08:10:16.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.017+0000 7f2d9c229740 1 -- 172.21.15.2:0/2188623781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 msgr2=0x562aa9f92ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:16.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.017+0000 7f2d9c229740 1 --2- 172.21.15.2:0/2188623781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f92ab0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009900 tx=0x7f2d84000bc0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.017+0000 7f2d9c229740 1 -- 172.21.15.2:0/2188623781 shutdown_connections 2024-09-10T08:10:16.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.017+0000 7f2d9c229740 1 --2- 172.21.15.2:0/2188623781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f7ae50 0x562aa9f74f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.017+0000 7f2d9c229740 1 --2- 172.21.15.2:0/2188623781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f70590 0x562aa9f92ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.017+0000 7f2d9c229740 1 --2- 172.21.15.2:0/2188623781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f92ab0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.017+0000 7f2d9c229740 1 -- 172.21.15.2:0/2188623781 >> 172.21.15.2:0/2188623781 conn(0x562aa9f6c200 msgr2=0x562aa9f79d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:16.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.017+0000 7f2d9c229740 1 -- 172.21.15.2:0/2188623781 shutdown_connections 2024-09-10T08:10:16.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.017+0000 7f2d9c229740 1 -- 172.21.15.2:0/2188623781 wait complete. 2024-09-10T08:10:16.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.017+0000 7f2d9c229740 1 -- start start 2024-09-10T08:10:16.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.018+0000 7f2d9c229740 1 Processor -- start 2024-09-10T08:10:16.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.018+0000 7f2d9c229740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f7eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:16.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.018+0000 7f2d9c229740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f70590 0x562aa9f7f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:16.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.018+0000 7f2d9c229740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 0x562aa9ff4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:16.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.018+0000 7f2d9c229740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941b3a40 con 0x562aa9f6fc70 2024-09-10T08:10:16.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.018+0000 7f2d9c229740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941b4eb0 con 0x562aa9f7ae50 2024-09-10T08:10:16.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.018+0000 7f2d9c229740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941b4110 con 0x562aa9f70590 2024-09-10T08:10:16.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.018+0000 7f2d9c229740 10 client.0 _ll_register_callbacks cb 0x7f2d7c09b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:16.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.018+0000 7f2d92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f7eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:16.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.018+0000 7f2d92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f7eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36040/0 (socket says 172.21.15.2:36040) 2024-09-10T08:10:16.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.018+0000 7f2d92a00640 1 -- 172.21.15.2:0/4020383626 learned_addr learned my addr 172.21.15.2:0/4020383626 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:16.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.019+0000 7f2d93400640 1 --2- 172.21.15.2:0/4020383626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 0x562aa9ff4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:16.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.019+0000 7f2d8be00640 1 --2- 172.21.15.2:0/4020383626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f70590 0x562aa9f7f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:16.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.019+0000 7f2d92a00640 1 -- 172.21.15.2:0/4020383626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f70590 msgr2=0x562aa9f7f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:16.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.019+0000 7f2d92a00640 1 --2- 172.21.15.2:0/4020383626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f70590 0x562aa9f7f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.019+0000 7f2d92a00640 1 -- 172.21.15.2:0/4020383626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 msgr2=0x562aa9ff4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:16.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.019+0000 7f2d92a00640 1 --2- 172.21.15.2:0/4020383626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 0x562aa9ff4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.019+0000 7f2d92a00640 1 -- 172.21.15.2:0/4020383626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aa9f67770 con 0x562aa9f6fc70 2024-09-10T08:10:16.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.019+0000 7f2d92a00640 1 --2- 172.21.15.2:0/4020383626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f7eec0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2d84006010 tx=0x7f2d84031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4a4304b9c568aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:16.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.019+0000 7f2d77e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:16.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d77e00640 1 -- 172.21.15.2:0/4020383626 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84035070 con 0x562aa9f6fc70 2024-09-10T08:10:16.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d77e00640 1 -- 172.21.15.2:0/4020383626 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2d840317c0 con 0x562aa9f6fc70 2024-09-10T08:10:16.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d9c229740 1 -- 172.21.15.2:0/4020383626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d840042c0 con 0x562aa9f6fc70 2024-09-10T08:10:16.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d77e00640 1 -- 172.21.15.2:0/4020383626 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d84068050 con 0x562aa9f6fc70 2024-09-10T08:10:16.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d9c229740 10 client.15789 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:16.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d9c229740 1 -- 172.21.15.2:0/4020383626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562aa9ff5f00 con 0x562aa9f6fc70 2024-09-10T08:10:16.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d77e00640 1 -- 172.21.15.2:0/4020383626 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d840042c0 con 0x562aa9f6fc70 2024-09-10T08:10:16.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d77e00640 1 -- 172.21.15.2:0/4020383626 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2d84050480 con 0x562aa9f6fc70 2024-09-10T08:10:16.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d9c229740 10 client.15789 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:16.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d9c229740 10 client.15789 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:10:16.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d9c229740 20 client.15789 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:16.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d9c229740 1 --2- 172.21.15.2:0/4020383626 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562aa9ff61e0 0x562aa9ff85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:16.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d9c229740 4 client.15789 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:10:16.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.020+0000 7f2d9c229740 1 -- 172.21.15.2:0/4020383626 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562aa9edde60 con 0x562aa9ff61e0 2024-09-10T08:10:16.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.021+0000 7f2d8be00640 1 --2- 172.21.15.2:0/4020383626 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562aa9ff61e0 0x562aa9ff85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:16.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.021+0000 7f2d8be00640 1 --2- 172.21.15.2:0/4020383626 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562aa9ff61e0 0x562aa9ff85d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef6b6d302df0e798 server_cookie=9cc56e8c58a64677 in_seq=0 out_seq=0 2024-09-10T08:10:16.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.021+0000 7f2d77e00640 10 client.15789 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:16.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d77e00640 1 -- 172.21.15.2:0/4020383626 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562aa9edde60 con 0x562aa9ff61e0 2024-09-10T08:10:16.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d77e00640 10 client.15789 handle_command_reply: tid=0 2024-09-10T08:10:16.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d9c229740 1 client.15789 shutdown 2024-09-10T08:10:16.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d9c229740 1 -- 172.21.15.2:0/4020383626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 msgr2=0x562aa9f7eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:16.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d9c229740 1 --2- 172.21.15.2:0/4020383626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f7eec0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2d84006010 tx=0x7f2d84031400 comp rx=0 tx=0).stop 2024-09-10T08:10:16.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d9c229740 1 -- 172.21.15.2:0/4020383626 shutdown_connections 2024-09-10T08:10:16.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d9c229740 1 --2- 172.21.15.2:0/4020383626 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562aa9ff61e0 0x562aa9ff85d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d9c229740 1 --2- 172.21.15.2:0/4020383626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562aa9f7ae50 0x562aa9ff4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d9c229740 1 --2- 172.21.15.2:0/4020383626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562aa9f70590 0x562aa9f7f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d9c229740 1 --2- 172.21.15.2:0/4020383626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562aa9f6fc70 0x562aa9f7eec0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d77e00640 0 client.15789 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:16.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d9c229740 1 -- 172.21.15.2:0/4020383626 >> 172.21.15.2:0/4020383626 conn(0x562aa9f6c200 msgr2=0x562aa9f6ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:16.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d9c229740 1 -- 172.21.15.2:0/4020383626 shutdown_connections 2024-09-10T08:10:16.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d9c229740 1 -- 172.21.15.2:0/4020383626 wait complete. 2024-09-10T08:10:16.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.022+0000 7f2d9c229740 20 client.15789 trim_cache size 0 max 16384 2024-09-10T08:10:16.027 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:10:16.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.023+0000 7f2d99600640 1 -- 172.21.15.2:0/3411960517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d780a2c00 msgr2=0x7f2d780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:16.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.023+0000 7f2d99600640 1 --2- 172.21.15.2:0/3411960517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d780a2c00 0x7f2d780a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2d9406b460 tx=0x7f2d8c009500 comp rx=0 tx=0).stop 2024-09-10T08:10:16.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.023+0000 7f2d99600640 1 -- 172.21.15.2:0/3411960517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d94077040 msgr2=0x7f2d941a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:16.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.023+0000 7f2d99600640 1 --2- 172.21.15.2:0/3411960517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d94077040 0x7f2d941a4c60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00e9f0 tx=0x7f2d7c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.023+0000 7f2d99600640 1 -- 172.21.15.2:0/3411960517 shutdown_connections 2024-09-10T08:10:16.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.023+0000 7f2d99600640 1 --2- 172.21.15.2:0/3411960517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d94112a30 0x7f2d9419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.023+0000 7f2d99600640 1 --2- 172.21.15.2:0/3411960517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d780a2c00 0x7f2d780a50c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.023+0000 7f2d99600640 1 --2- 172.21.15.2:0/3411960517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d94077040 0x7f2d941a4c60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.023+0000 7f2d99600640 1 --2- 172.21.15.2:0/3411960517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d94076720 0x7f2d941a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:16.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.023+0000 7f2d99600640 1 -- 172.21.15.2:0/3411960517 >> 172.21.15.2:0/3411960517 conn(0x7f2d940ffa90 msgr2=0x7f2d94101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:16.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.023+0000 7f2d99600640 1 -- 172.21.15.2:0/3411960517 shutdown_connections 2024-09-10T08:10:16.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:16.023+0000 7f2d99600640 1 -- 172.21.15.2:0/3411960517 wait complete. 2024-09-10T08:10:16.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:15 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3083968374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:10:16.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:15 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3083968374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:10:16.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:15 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3083968374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:10:16.887 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id 391eb929 --release --await' 2024-09-10T08:10:16.887 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id 391eb929 --release --await 2024-09-10T08:10:17.083 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:16 smithi002 ceph-mon[30563]: pgmap v451: 129 pgs: 129 active+clean; 681 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-10T08:10:17.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:16 smithi155 ceph-mon[35405]: pgmap v451: 129 pgs: 129 active+clean; 681 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-10T08:10:17.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:16 smithi107 ceph-mon[35048]: pgmap v451: 129 pgs: 129 active+clean; 681 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-10T08:10:17.558 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:10:17.705 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:18.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.320+0000 7f284d800640 1 -- 172.21.15.2:0/1260256643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2848074510 msgr2=0x7f28480748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:18.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.320+0000 7f284d800640 1 --2- 172.21.15.2:0/1260256643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2848074510 0x7f28480748f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2838009900 tx=0x7f283802a480 comp rx=0 tx=0).stop 2024-09-10T08:10:18.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.320+0000 7f284d800640 1 -- 172.21.15.2:0/1260256643 shutdown_connections 2024-09-10T08:10:18.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.320+0000 7f284d800640 1 --2- 172.21.15.2:0/1260256643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f284810e280 0x7f2848110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.320+0000 7f284d800640 1 --2- 172.21.15.2:0/1260256643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2848074e30 0x7f284810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.320+0000 7f284d800640 1 --2- 172.21.15.2:0/1260256643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2848074510 0x7f28480748f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.320+0000 7f284d800640 1 -- 172.21.15.2:0/1260256643 >> 172.21.15.2:0/1260256643 conn(0x7f28480fd740 msgr2=0x7f28480ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:18.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.320+0000 7f284d800640 1 -- 172.21.15.2:0/1260256643 shutdown_connections 2024-09-10T08:10:18.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.320+0000 7f284d800640 1 -- 172.21.15.2:0/1260256643 wait complete. 2024-09-10T08:10:18.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.321+0000 7f284d800640 1 Processor -- start 2024-09-10T08:10:18.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.322+0000 7f284d800640 1 -- start start 2024-09-10T08:10:18.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.322+0000 7f284d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2848074510 0x7f284819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.322+0000 7f284d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2848074e30 0x7f284819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.322+0000 7f284d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f284810e280 0x7f284819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.322+0000 7f284d800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2848114000 con 0x7f2848074510 2024-09-10T08:10:18.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.322+0000 7f284d800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2848113e80 con 0x7f2848074e30 2024-09-10T08:10:18.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.322+0000 7f284d800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2848114180 con 0x7f284810e280 2024-09-10T08:10:18.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.323+0000 7f2847e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2848074510 0x7f284819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.323+0000 7f2847e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2848074510 0x7f284819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37116/0 (socket says 172.21.15.2:37116) 2024-09-10T08:10:18.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.323+0000 7f2847e00640 1 -- 172.21.15.2:0/4178651073 learned_addr learned my addr 172.21.15.2:0/4178651073 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:18.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.323+0000 7f2847400640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2848074e30 0x7f284819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.323+0000 7f284ce00640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f284810e280 0x7f284819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.323+0000 7f2847400640 1 -- 172.21.15.2:0/4178651073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f284810e280 msgr2=0x7f284819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:18.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.323+0000 7f2847400640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f284810e280 0x7f284819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.323+0000 7f2847400640 1 -- 172.21.15.2:0/4178651073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2848074510 msgr2=0x7f284819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:18.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.323+0000 7f2847400640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2848074510 0x7f284819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.323+0000 7f2847400640 1 -- 172.21.15.2:0/4178651073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284819f4d0 con 0x7f2848074e30 2024-09-10T08:10:18.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.323+0000 7f2847e00640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2848074510 0x7f284819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:18.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.324+0000 7f2847400640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2848074e30 0x7f284819aa60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f283400d720 tx=0x7f283400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:18.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.324+0000 7f2844c00640 1 -- 172.21.15.2:0/4178651073 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834017070 con 0x7f2848074e30 2024-09-10T08:10:18.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.324+0000 7f2844c00640 1 -- 172.21.15.2:0/4178651073 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283400de30 con 0x7f2848074e30 2024-09-10T08:10:18.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.324+0000 7f284d800640 1 -- 172.21.15.2:0/4178651073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284819f7c0 con 0x7f2848074e30 2024-09-10T08:10:18.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.324+0000 7f2844c00640 1 -- 172.21.15.2:0/4178651073 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28340047a0 con 0x7f2848074e30 2024-09-10T08:10:18.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.324+0000 7f284d800640 1 -- 172.21.15.2:0/4178651073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28481a70a0 con 0x7f2848074e30 2024-09-10T08:10:18.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.328+0000 7f2844c00640 1 -- 172.21.15.2:0/4178651073 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f28340099d0 con 0x7f2848074e30 2024-09-10T08:10:18.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.328+0000 7f28517d5740 1 Processor -- start 2024-09-10T08:10:18.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.328+0000 7f28517d5740 1 -- start start 2024-09-10T08:10:18.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.328+0000 7f28517d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485e520 0x5620c485e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.329+0000 7f28517d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c485ee40 0x5620c48691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.329+0000 7f28517d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c4869700 0x5620c486baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.329+0000 7f28517d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5620c47c4650 con 0x5620c4869700 2024-09-10T08:10:18.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.329+0000 7f28517d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5620c476bf10 con 0x5620c485e520 2024-09-10T08:10:18.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.329+0000 7f28517d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5620c47c0610 con 0x5620c485ee40 2024-09-10T08:10:18.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.329+0000 7f284ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485e520 0x5620c485e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.329+0000 7f2847e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c485ee40 0x5620c48691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.329+0000 7f284ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485e520 0x5620c485e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:55914/0 (socket says 172.21.15.2:55914) 2024-09-10T08:10:18.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.329+0000 7f284ce00640 1 -- 172.21.15.2:0/3335224530 learned_addr learned my addr 172.21.15.2:0/3335224530 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:18.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.329+0000 7f2847400640 1 --2- 172.21.15.2:0/3335224530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c4869700 0x5620c486baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.329+0000 7f2847e00640 1 -- 172.21.15.2:0/3335224530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485e520 msgr2=0x5620c485e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:18.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.329+0000 7f2847e00640 1 --2- 172.21.15.2:0/3335224530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485e520 0x5620c485e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.330+0000 7f2847e00640 1 -- 172.21.15.2:0/3335224530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c4869700 msgr2=0x5620c486baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.330+0000 7f2847e00640 1 --2- 172.21.15.2:0/3335224530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c4869700 0x5620c486baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.330+0000 7f2847e00640 1 -- 172.21.15.2:0/3335224530 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620c44d3580 con 0x5620c485ee40 2024-09-10T08:10:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.330+0000 7f284ce00640 1 --2- 172.21.15.2:0/3335224530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485e520 0x5620c485e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.330+0000 7f2847400640 1 --2- 172.21.15.2:0/3335224530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c4869700 0x5620c486baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.330+0000 7f2844c00640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f282c0a2c00 0x7f282c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.330+0000 7f2844c00640 1 -- 172.21.15.2:0/4178651073 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f28340cefe0 con 0x7f2848074e30 2024-09-10T08:10:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.330+0000 7f2847e00640 1 --2- 172.21.15.2:0/3335224530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c485ee40 0x5620c48691c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2838031550 tx=0x7f2838005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=752ed3cf55ee28ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.330+0000 7f284ce00640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f282c0a2c00 0x7f282c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.331+0000 7f283d600640 1 -- 172.21.15.2:0/3335224530 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838040070 con 0x5620c485ee40 2024-09-10T08:10:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.331+0000 7f283d600640 1 -- 172.21.15.2:0/3335224530 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28380047a0 con 0x5620c485ee40 2024-09-10T08:10:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.331+0000 7f283d600640 1 -- 172.21.15.2:0/3335224530 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838035040 con 0x5620c485ee40 2024-09-10T08:10:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.332+0000 7f28517d5740 1 -- 172.21.15.2:0/3335224530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c485ee40 msgr2=0x5620c48691c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.332+0000 7f284ce00640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f282c0a2c00 0x7f282c0a50c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2840009fd0 tx=0x7f2840009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.332+0000 7f28517d5740 1 --2- 172.21.15.2:0/3335224530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c485ee40 0x5620c48691c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2838031550 tx=0x7f2838005e80 comp rx=0 tx=0).stop 2024-09-10T08:10:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.332+0000 7f28517d5740 1 -- 172.21.15.2:0/3335224530 shutdown_connections 2024-09-10T08:10:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.332+0000 7f28517d5740 1 --2- 172.21.15.2:0/3335224530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c4869700 0x5620c486baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.332+0000 7f28517d5740 1 --2- 172.21.15.2:0/3335224530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c485ee40 0x5620c48691c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.332+0000 7f28517d5740 1 --2- 172.21.15.2:0/3335224530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485e520 0x5620c485e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.332+0000 7f28517d5740 1 -- 172.21.15.2:0/3335224530 >> 172.21.15.2:0/3335224530 conn(0x5620c485aab0 msgr2=0x5620c485ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:18.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.332+0000 7f28517d5740 1 -- 172.21.15.2:0/3335224530 shutdown_connections 2024-09-10T08:10:18.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.332+0000 7f28517d5740 1 -- 172.21.15.2:0/3335224530 wait complete. 2024-09-10T08:10:18.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.333+0000 7f28517d5740 1 -- start start 2024-09-10T08:10:18.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.333+0000 7f28517d5740 1 Processor -- start 2024-09-10T08:10:18.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.334+0000 7f28517d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c485e520 0x5620c48665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.334+0000 7f28517d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485ee40 0x5620c4866af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.334+0000 7f28517d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c4869700 0x5620c4867030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.334+0000 7f28517d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5620c486c2d0 con 0x5620c485ee40 2024-09-10T08:10:18.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.334+0000 7f28517d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5620c477f350 con 0x5620c4869700 2024-09-10T08:10:18.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.334+0000 7f28517d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5620c486c450 con 0x5620c485e520 2024-09-10T08:10:18.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.334+0000 7f28517d5740 10 client.0 _ll_register_callbacks cb 0x5620c47a2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:18.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.334+0000 7f2847400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485ee40 0x5620c4866af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.334+0000 7f2847400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485ee40 0x5620c4866af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37126/0 (socket says 172.21.15.2:37126) 2024-09-10T08:10:18.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.334+0000 7f2847400640 1 -- 172.21.15.2:0/679919732 learned_addr learned my addr 172.21.15.2:0/679919732 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:18.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.334+0000 7f284ce00640 1 --2- 172.21.15.2:0/679919732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c4869700 0x5620c4867030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.335+0000 7f2847400640 1 -- 172.21.15.2:0/679919732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c485e520 msgr2=0x5620c48665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:18.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.335+0000 7f2847400640 1 --2- 172.21.15.2:0/679919732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c485e520 0x5620c48665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.335+0000 7f2847400640 1 -- 172.21.15.2:0/679919732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c4869700 msgr2=0x5620c4867030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:18.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.335+0000 7f2847400640 1 --2- 172.21.15.2:0/679919732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c4869700 0x5620c4867030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.335+0000 7f2847400640 1 -- 172.21.15.2:0/679919732 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620c459d960 con 0x5620c485ee40 2024-09-10T08:10:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.335+0000 7f284ce00640 1 --2- 172.21.15.2:0/679919732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c4869700 0x5620c4867030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.335+0000 7f2847400640 1 --2- 172.21.15.2:0/679919732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485ee40 0x5620c4866af0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2834005fd0 tx=0x7f2834005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9764e9b666c65917 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.335+0000 7f2832000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.335+0000 7f2832000640 1 -- 172.21.15.2:0/679919732 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283409af90 con 0x5620c485ee40 2024-09-10T08:10:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.335+0000 7f2832000640 1 -- 172.21.15.2:0/679919732 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2834097ac0 con 0x5620c485ee40 2024-09-10T08:10:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.336+0000 7f28517d5740 1 -- 172.21.15.2:0/679919732 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28380047a0 con 0x5620c485ee40 2024-09-10T08:10:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.336+0000 7f2832000640 1 -- 172.21.15.2:0/679919732 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f283409adb0 con 0x5620c485ee40 2024-09-10T08:10:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.336+0000 7f28517d5740 10 client.15801 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.336+0000 7f28517d5740 1 -- 172.21.15.2:0/679919732 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5620c4877ed0 con 0x5620c485ee40 2024-09-10T08:10:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.336+0000 7f2832000640 1 -- 172.21.15.2:0/679919732 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28380047a0 con 0x5620c485ee40 2024-09-10T08:10:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.336+0000 7f2832000640 1 -- 172.21.15.2:0/679919732 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2834009cc0 con 0x5620c485ee40 2024-09-10T08:10:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.337+0000 7f28517d5740 10 client.15801 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.337+0000 7f28517d5740 10 client.15801 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:10:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.337+0000 7f28517d5740 20 client.15801 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.337+0000 7f28517d5740 1 --2- 172.21.15.2:0/679919732 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5620c4879220 0x5620c487b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.337+0000 7f28517d5740 4 client.15801 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.337+0000 7f28517d5740 1 -- 172.21.15.2:0/679919732 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620c47d1bc0 con 0x5620c4879220 2024-09-10T08:10:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.337+0000 7f2847e00640 1 --2- 172.21.15.2:0/679919732 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5620c4879220 0x5620c487b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.338+0000 7f2847e00640 1 --2- 172.21.15.2:0/679919732 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5620c4879220 0x5620c487b690 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceabeb0a36fb735f server_cookie=2927fd21dc183f58 in_seq=0 out_seq=0 2024-09-10T08:10:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.338+0000 7f2832000640 10 client.15801 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.338+0000 7f2832000640 1 -- 172.21.15.2:0/679919732 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620c47d1bc0 con 0x5620c4879220 2024-09-10T08:10:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.338+0000 7f2832000640 10 client.15801 handle_command_reply: tid=0 2024-09-10T08:10:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.339+0000 7f28517d5740 1 client.15801 shutdown 2024-09-10T08:10:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.339+0000 7f28517d5740 1 -- 172.21.15.2:0/679919732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485ee40 msgr2=0x5620c4866af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.339+0000 7f28517d5740 1 --2- 172.21.15.2:0/679919732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485ee40 0x5620c4866af0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2834005fd0 tx=0x7f2834005d40 comp rx=0 tx=0).stop 2024-09-10T08:10:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.339+0000 7f28517d5740 1 -- 172.21.15.2:0/679919732 shutdown_connections 2024-09-10T08:10:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.339+0000 7f28517d5740 1 --2- 172.21.15.2:0/679919732 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5620c4879220 0x5620c487b690 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.339+0000 7f28517d5740 1 --2- 172.21.15.2:0/679919732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c4869700 0x5620c4867030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.339+0000 7f28517d5740 1 --2- 172.21.15.2:0/679919732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485ee40 0x5620c4866af0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.339+0000 7f28517d5740 1 --2- 172.21.15.2:0/679919732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c485e520 0x5620c48665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.339+0000 7f2832000640 0 client.15801 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.339+0000 7f28517d5740 1 -- 172.21.15.2:0/679919732 >> 172.21.15.2:0/679919732 conn(0x5620c485aab0 msgr2=0x5620c4859cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.339+0000 7f28517d5740 1 -- 172.21.15.2:0/679919732 shutdown_connections 2024-09-10T08:10:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.339+0000 7f28517d5740 1 -- 172.21.15.2:0/679919732 wait complete. 2024-09-10T08:10:18.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.340+0000 7f28517d5740 20 client.15801 trim_cache size 0 max 16384 2024-09-10T08:10:18.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.367+0000 7f28517d5740 1 Processor -- start 2024-09-10T08:10:18.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.368+0000 7f28517d5740 1 -- start start 2024-09-10T08:10:18.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.368+0000 7f28517d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485e520 0x5620c4860f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.368+0000 7f28517d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485ee40 0x5620c4880f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.368+0000 7f28517d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c4869700 0x5620c48814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.368+0000 7f28517d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f28481b3750 con 0x5620c485e520 2024-09-10T08:10:18.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.368+0000 7f28517d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28481b1020 con 0x5620c485ee40 2024-09-10T08:10:18.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.368+0000 7f28517d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f28481aee70 con 0x5620c4869700 2024-09-10T08:10:18.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.368+0000 7f2847e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485e520 0x5620c4860f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.368+0000 7f284ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c4869700 0x5620c48814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.368+0000 7f284ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c4869700 0x5620c48814a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:41394/0 (socket says 172.21.15.2:41394) 2024-09-10T08:10:18.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.368+0000 7f284ce00640 1 -- 172.21.15.2:0/3550670941 learned_addr learned my addr 172.21.15.2:0/3550670941 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:18.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.368+0000 7f2847400640 1 --2- 172.21.15.2:0/3550670941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485ee40 0x5620c4880f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.369+0000 7f284ce00640 1 -- 172.21.15.2:0/3550670941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485ee40 msgr2=0x5620c4880f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:18.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.369+0000 7f284ce00640 1 --2- 172.21.15.2:0/3550670941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485ee40 0x5620c4880f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.369+0000 7f284ce00640 1 -- 172.21.15.2:0/3550670941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485e520 msgr2=0x5620c4860f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:18.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.369+0000 7f284ce00640 1 --2- 172.21.15.2:0/3550670941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485e520 0x5620c4860f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.369+0000 7f284ce00640 1 -- 172.21.15.2:0/3550670941 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620c485a8e0 con 0x5620c4869700 2024-09-10T08:10:18.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.369+0000 7f284ce00640 1 --2- 172.21.15.2:0/3550670941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c4869700 0x5620c48814a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f284000f940 tx=0x7f2840019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d08a73c1a144e0d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:18.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.370+0000 7f283cc00640 1 -- 172.21.15.2:0/3550670941 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840004b20 con 0x5620c4869700 2024-09-10T08:10:18.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.370+0000 7f283cc00640 1 -- 172.21.15.2:0/3550670941 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2840004cc0 con 0x5620c4869700 2024-09-10T08:10:18.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.370+0000 7f283cc00640 1 -- 172.21.15.2:0/3550670941 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28400057f0 con 0x5620c4869700 2024-09-10T08:10:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.370+0000 7f28517d5740 1 -- 172.21.15.2:0/3550670941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c4869700 msgr2=0x5620c48814a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.370+0000 7f28517d5740 1 --2- 172.21.15.2:0/3550670941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c4869700 0x5620c48814a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f284000f940 tx=0x7f2840019000 comp rx=0 tx=0).stop 2024-09-10T08:10:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.370+0000 7f28517d5740 1 -- 172.21.15.2:0/3550670941 shutdown_connections 2024-09-10T08:10:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.370+0000 7f28517d5740 1 --2- 172.21.15.2:0/3550670941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c4869700 0x5620c48814a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.370+0000 7f28517d5740 1 --2- 172.21.15.2:0/3550670941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485ee40 0x5620c4880f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.370+0000 7f28517d5740 1 --2- 172.21.15.2:0/3550670941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485e520 0x5620c4860f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.371+0000 7f28517d5740 1 -- 172.21.15.2:0/3550670941 >> 172.21.15.2:0/3550670941 conn(0x5620c485aab0 msgr2=0x5620c4864e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.371+0000 7f28517d5740 1 -- 172.21.15.2:0/3550670941 shutdown_connections 2024-09-10T08:10:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.371+0000 7f28517d5740 1 -- 172.21.15.2:0/3550670941 wait complete. 2024-09-10T08:10:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.371+0000 7f28517d5740 1 -- start start 2024-09-10T08:10:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.371+0000 7f28517d5740 1 Processor -- start 2024-09-10T08:10:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.372+0000 7f28517d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485e520 0x5620c4871fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.372+0000 7f28517d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485ee40 0x5620c486d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.372+0000 7f28517d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c4869700 0x5620c486da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.372+0000 7f28517d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f28481af4e0 con 0x5620c485e520 2024-09-10T08:10:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.372+0000 7f28517d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28481b0950 con 0x5620c485ee40 2024-09-10T08:10:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.372+0000 7f28517d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f28481afbb0 con 0x5620c4869700 2024-09-10T08:10:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.372+0000 7f28517d5740 10 client.0 _ll_register_callbacks cb 0x7f28340970b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.372+0000 7f2847e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485e520 0x5620c4871fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.372+0000 7f284ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c4869700 0x5620c486da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.372+0000 7f2847e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485e520 0x5620c4871fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37152/0 (socket says 172.21.15.2:37152) 2024-09-10T08:10:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.372+0000 7f2847e00640 1 -- 172.21.15.2:0/1823675085 learned_addr learned my addr 172.21.15.2:0/1823675085 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.372+0000 7f2847400640 1 --2- 172.21.15.2:0/1823675085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485ee40 0x5620c486d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.373+0000 7f2847e00640 1 -- 172.21.15.2:0/1823675085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c4869700 msgr2=0x5620c486da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.373+0000 7f2847e00640 1 --2- 172.21.15.2:0/1823675085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c4869700 0x5620c486da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.373+0000 7f2847e00640 1 -- 172.21.15.2:0/1823675085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485ee40 msgr2=0x5620c486d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.373+0000 7f2847e00640 1 --2- 172.21.15.2:0/1823675085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485ee40 0x5620c486d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.373+0000 7f2847e00640 1 -- 172.21.15.2:0/1823675085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620c48eb740 con 0x5620c485e520 2024-09-10T08:10:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.373+0000 7f2847400640 1 --2- 172.21.15.2:0/1823675085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485ee40 0x5620c486d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.373+0000 7f2847e00640 1 --2- 172.21.15.2:0/1823675085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485e520 0x5620c4871fb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2838031520 tx=0x7f2838009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb30c4caf0d08710 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.373+0000 7f2832000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.373+0000 7f2832000640 1 -- 172.21.15.2:0/1823675085 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838040040 con 0x5620c485e520 2024-09-10T08:10:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.373+0000 7f2832000640 1 -- 172.21.15.2:0/1823675085 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2838003d00 con 0x5620c485e520 2024-09-10T08:10:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.373+0000 7f28517d5740 1 -- 172.21.15.2:0/1823675085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2840004cc0 con 0x5620c485e520 2024-09-10T08:10:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.374+0000 7f2832000640 1 -- 172.21.15.2:0/1823675085 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f283805b050 con 0x5620c485e520 2024-09-10T08:10:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.374+0000 7f28517d5740 10 client.15807 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.374+0000 7f28517d5740 1 -- 172.21.15.2:0/1823675085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5620c48ec920 con 0x5620c485e520 2024-09-10T08:10:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.374+0000 7f2832000640 1 -- 172.21.15.2:0/1823675085 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2840004cc0 con 0x5620c485e520 2024-09-10T08:10:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.374+0000 7f2832000640 1 -- 172.21.15.2:0/1823675085 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f283804d540 con 0x5620c485e520 2024-09-10T08:10:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.374+0000 7f28517d5740 10 client.15807 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.374+0000 7f28517d5740 10 client.15807 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:10:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.374+0000 7f28517d5740 20 client.15807 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.374+0000 7f28517d5740 1 --2- 172.21.15.2:0/1823675085 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5620c48ecbb0 0x5620c48eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.375+0000 7f28517d5740 4 client.15807 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "391eb929", "release": true, "await": true}] 2024-09-10T08:10:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.375+0000 7f28517d5740 1 -- 172.21.15.2:0/1823675085 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "391eb929", "release": true, "await": true}) -- 0x5620c47cc0d0 con 0x5620c48ecbb0 2024-09-10T08:10:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.375+0000 7f2847400640 1 --2- 172.21.15.2:0/1823675085 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5620c48ecbb0 0x5620c48eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.375+0000 7f2847400640 1 --2- 172.21.15.2:0/1823675085 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5620c48ecbb0 0x5620c48eefa0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dfe8c83cd7e875a server_cookie=b417233f65bc74c6 in_seq=0 out_seq=0 2024-09-10T08:10:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.375+0000 7f2832000640 10 client.15807 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.378+0000 7f2832000640 1 -- 172.21.15.2:0/1823675085 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5620c47cc0d0 con 0x5620c48ecbb0 2024-09-10T08:10:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.378+0000 7f2832000640 10 client.15807 handle_command_reply: tid=0 2024-09-10T08:10:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.378+0000 7f28517d5740 1 client.15807 shutdown 2024-09-10T08:10:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.378+0000 7f28517d5740 1 -- 172.21.15.2:0/1823675085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485e520 msgr2=0x5620c4871fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.378+0000 7f28517d5740 1 --2- 172.21.15.2:0/1823675085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485e520 0x5620c4871fb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2838031520 tx=0x7f2838009260 comp rx=0 tx=0).stop 2024-09-10T08:10:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.378+0000 7f28517d5740 1 -- 172.21.15.2:0/1823675085 shutdown_connections 2024-09-10T08:10:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.378+0000 7f28517d5740 1 --2- 172.21.15.2:0/1823675085 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5620c48ecbb0 0x5620c48eefa0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.378+0000 7f28517d5740 1 --2- 172.21.15.2:0/1823675085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620c4869700 0x5620c486da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.379+0000 7f28517d5740 1 --2- 172.21.15.2:0/1823675085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5620c485ee40 0x5620c486d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.379+0000 7f28517d5740 1 --2- 172.21.15.2:0/1823675085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5620c485e520 0x5620c4871fb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.379+0000 7f28517d5740 1 -- 172.21.15.2:0/1823675085 >> 172.21.15.2:0/1823675085 conn(0x5620c485aab0 msgr2=0x5620c4867c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:18.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.379+0000 7f2832000640 0 client.15807 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:18.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.379+0000 7f28517d5740 1 -- 172.21.15.2:0/1823675085 shutdown_connections 2024-09-10T08:10:18.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.379+0000 7f28517d5740 1 -- 172.21.15.2:0/1823675085 wait complete. 2024-09-10T08:10:18.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.379+0000 7f28517d5740 20 client.15807 trim_cache size 0 max 16384 2024-09-10T08:10:18.381 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:10:18.381 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:10:18.382 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:10:18.382 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 8, 2024-09-10T08:10:18.382 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:10:18.382 INFO:teuthology.orchestra.run.smithi002.stdout: "391eb929": { 2024-09-10T08:10:18.382 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 8, 2024-09-10T08:10:18.382 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:10:18.382 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:10:18.382 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:10:18.382 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:10:18.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:10:18.382 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:10:18.382 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 150.0, 2024-09-10T08:10:18.382 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:10:18.383 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:10:18.383 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:10:18.383 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:10:18.383 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:10:18.383 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:10:18.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:10:18.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:10:18.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:10:18.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:10:18.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:10:18.383 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:10:18.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.380+0000 7f284d800640 1 -- 172.21.15.2:0/4178651073 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f282c0a2c00 msgr2=0x7f282c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:18.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.380+0000 7f284d800640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f282c0a2c00 0x7f282c0a50c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2840009fd0 tx=0x7f2840009500 comp rx=0 tx=0).stop 2024-09-10T08:10:18.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.380+0000 7f284d800640 1 -- 172.21.15.2:0/4178651073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2848074e30 msgr2=0x7f284819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:18.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.380+0000 7f284d800640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2848074e30 0x7f284819aa60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f283400d720 tx=0x7f283400dbe0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.384 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.380+0000 7f284d800640 1 -- 172.21.15.2:0/4178651073 shutdown_connections 2024-09-10T08:10:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.380+0000 7f284d800640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f282c0a2c00 0x7f282c0a50c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.380+0000 7f284d800640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f284810e280 0x7f284819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.380+0000 7f284d800640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2848074e30 0x7f284819aa60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.380+0000 7f284d800640 1 --2- 172.21.15.2:0/4178651073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2848074510 0x7f284819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.380+0000 7f284d800640 1 -- 172.21.15.2:0/4178651073 >> 172.21.15.2:0/4178651073 conn(0x7f28480fd740 msgr2=0x7f28480fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.380+0000 7f284d800640 1 -- 172.21.15.2:0/4178651073 shutdown_connections 2024-09-10T08:10:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:18.380+0000 7f284d800640 1 -- 172.21.15.2:0/4178651073 wait complete. 2024-09-10T08:10:18.392 INFO:tasks.workunit.client.0.smithi002.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-10T08:10:18.394 INFO:tasks.workunit.client.0.smithi002.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-09-10T08:10:18.394 INFO:tasks.workunit.client.0.smithi002.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-09-10T08:10:19.109 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:18 smithi002 ceph-mon[30563]: pgmap v452: 129 pgs: 129 active+clean; 681 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T08:10:19.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:18 smithi155 ceph-mon[35405]: pgmap v452: 129 pgs: 129 active+clean; 681 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T08:10:19.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:18 smithi107 ceph-mon[35048]: pgmap v452: 129 pgs: 129 active+clean; 681 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T08:10:19.656 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '391eb929', total seconds elapsed: 36.7 2024-09-10T08:10:19.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.332+0000 7f7675200640 1 -- 172.21.15.2:0/163692979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7670112a50 msgr2=0x7f7670114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:19.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.332+0000 7f7675200640 1 --2- 172.21.15.2:0/163692979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7670112a50 0x7f7670114e40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7668009ca0 tx=0x7f766802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.332+0000 7f7675200640 1 -- 172.21.15.2:0/163692979 shutdown_connections 2024-09-10T08:10:19.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.332+0000 7f7675200640 1 --2- 172.21.15.2:0/163692979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7670112a50 0x7f7670114e40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.332+0000 7f7675200640 1 --2- 172.21.15.2:0/163692979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7670077040 0x7f7670112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.332+0000 7f7675200640 1 --2- 172.21.15.2:0/163692979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7670076720 0x7f7670076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.332+0000 7f7675200640 1 -- 172.21.15.2:0/163692979 >> 172.21.15.2:0/163692979 conn(0x7f76700ffa50 msgr2=0x7f7670101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:19.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.333+0000 7f7675200640 1 -- 172.21.15.2:0/163692979 shutdown_connections 2024-09-10T08:10:19.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.333+0000 7f7675200640 1 -- 172.21.15.2:0/163692979 wait complete. 2024-09-10T08:10:19.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.334+0000 7f7675200640 1 Processor -- start 2024-09-10T08:10:19.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.334+0000 7f7675200640 1 -- start start 2024-09-10T08:10:19.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.334+0000 7f7675200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7670076720 0x7f76701a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.334+0000 7f7675200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7670077040 0x7f76701a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.334+0000 7f7675200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7670112a50 0x7f767019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.334+0000 7f7675200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f76701185a0 con 0x7f7670076720 2024-09-10T08:10:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.334+0000 7f7675200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7670118420 con 0x7f7670112a50 2024-09-10T08:10:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.334+0000 7f7675200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7670118720 con 0x7f7670077040 2024-09-10T08:10:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.334+0000 7f766de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7670076720 0x7f76701a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.334+0000 7f766de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7670076720 0x7f76701a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37172/0 (socket says 172.21.15.2:37172) 2024-09-10T08:10:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.334+0000 7f766de00640 1 -- 172.21.15.2:0/725660304 learned_addr learned my addr 172.21.15.2:0/725660304 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.335+0000 7f766e800640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7670112a50 0x7f767019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.335+0000 7f7667e00640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7670077040 0x7f76701a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.335+0000 7f766de00640 1 -- 172.21.15.2:0/725660304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7670077040 msgr2=0x7f76701a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:19.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.335+0000 7f766de00640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7670077040 0x7f76701a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.335+0000 7f766de00640 1 -- 172.21.15.2:0/725660304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7670112a50 msgr2=0x7f767019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:19.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.335+0000 7f766de00640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7670112a50 0x7f767019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.335+0000 7f766de00640 1 -- 172.21.15.2:0/725660304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767019efb0 con 0x7f7670076720 2024-09-10T08:10:19.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.335+0000 7f7667e00640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7670077040 0x7f76701a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:19.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.335+0000 7f766de00640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7670076720 0x7f76701a46c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f766000d770 tx=0x7f766000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:19.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.335+0000 7f7666a00640 1 -- 172.21.15.2:0/725660304 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660017070 con 0x7f7670076720 2024-09-10T08:10:19.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.335+0000 7f7666a00640 1 -- 172.21.15.2:0/725660304 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7660005500 con 0x7f7670076720 2024-09-10T08:10:19.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.335+0000 7f7666a00640 1 -- 172.21.15.2:0/725660304 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660004e00 con 0x7f7670076720 2024-09-10T08:10:19.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.335+0000 7f7675200640 1 -- 172.21.15.2:0/725660304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767019f2a0 con 0x7f7670076720 2024-09-10T08:10:19.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.336+0000 7f7675200640 1 -- 172.21.15.2:0/725660304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76701ab590 con 0x7f7670076720 2024-09-10T08:10:19.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.339+0000 7f7677e10740 1 Processor -- start 2024-09-10T08:10:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.339+0000 7f7666a00640 1 -- 172.21.15.2:0/725660304 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7660004fa0 con 0x7f7670076720 2024-09-10T08:10:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.339+0000 7f7677e10740 1 -- start start 2024-09-10T08:10:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.339+0000 7f7677e10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2863c70 0x55c0b2864050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.339+0000 7f7677e10740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2864590 0x55c0b286e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.339+0000 7f7677e10740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b2871240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.339+0000 7f7677e10740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c0b27ca3e0 con 0x55c0b286ee50 2024-09-10T08:10:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.339+0000 7f7677e10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c0b2771d80 con 0x55c0b2863c70 2024-09-10T08:10:19.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.339+0000 7f7677e10740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c0b27c63a0 con 0x55c0b2864590 2024-09-10T08:10:19.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.339+0000 7f7667e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b2871240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.340+0000 7f766e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2863c70 0x55c0b2864050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.340+0000 7f766de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2864590 0x55c0b286e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.340+0000 7f7667e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b2871240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37180/0 (socket says 172.21.15.2:37180) 2024-09-10T08:10:19.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.340+0000 7f7667e00640 1 -- 172.21.15.2:0/467283781 learned_addr learned my addr 172.21.15.2:0/467283781 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:19.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.340+0000 7f7667e00640 1 -- 172.21.15.2:0/467283781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2864590 msgr2=0x55c0b286e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:19.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.340+0000 7f7667e00640 1 --2- 172.21.15.2:0/467283781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2864590 0x55c0b286e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.340+0000 7f7667e00640 1 -- 172.21.15.2:0/467283781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2863c70 msgr2=0x55c0b2864050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:19.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.340+0000 7f7667e00640 1 --2- 172.21.15.2:0/467283781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2863c70 0x55c0b2864050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.340+0000 7f7667e00640 1 -- 172.21.15.2:0/467283781 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b2790ae0 con 0x55c0b286ee50 2024-09-10T08:10:19.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.340+0000 7f766e800640 1 --2- 172.21.15.2:0/467283781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2863c70 0x55c0b2864050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:19.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.340+0000 7f7667e00640 1 --2- 172.21.15.2:0/467283781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b2871240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7658009c30 tx=0x7f76580117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fbee807be0106a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:19.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.341+0000 7f7666a00640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76540a2b80 0x7f76540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.341+0000 7f765f400640 1 -- 172.21.15.2:0/467283781 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658004020 con 0x55c0b286ee50 2024-09-10T08:10:19.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.341+0000 7f7666a00640 1 -- 172.21.15.2:0/725660304 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f76600cef10 con 0x7f7670076720 2024-09-10T08:10:19.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.341+0000 7f765f400640 1 -- 172.21.15.2:0/467283781 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76580041c0 con 0x55c0b286ee50 2024-09-10T08:10:19.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.341+0000 7f765f400640 1 -- 172.21.15.2:0/467283781 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658012780 con 0x55c0b286ee50 2024-09-10T08:10:19.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.341+0000 7f766e800640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76540a2b80 0x7f76540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.342+0000 7f7677e10740 1 -- 172.21.15.2:0/467283781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 msgr2=0x55c0b2871240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:19.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.342+0000 7f766e800640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76540a2b80 0x7f76540a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7668006010 tx=0x7f7668004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:19.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.342+0000 7f7677e10740 1 --2- 172.21.15.2:0/467283781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b2871240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7658009c30 tx=0x7f76580117b0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.342+0000 7f7677e10740 1 -- 172.21.15.2:0/467283781 shutdown_connections 2024-09-10T08:10:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.342+0000 7f7677e10740 1 --2- 172.21.15.2:0/467283781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b2871240 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.342+0000 7f7677e10740 1 --2- 172.21.15.2:0/467283781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2864590 0x55c0b286e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.342+0000 7f7677e10740 1 --2- 172.21.15.2:0/467283781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2863c70 0x55c0b2864050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.342+0000 7f7677e10740 1 -- 172.21.15.2:0/467283781 >> 172.21.15.2:0/467283781 conn(0x55c0b2860200 msgr2=0x55c0b28605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.342+0000 7f7677e10740 1 -- 172.21.15.2:0/467283781 shutdown_connections 2024-09-10T08:10:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.342+0000 7f7677e10740 1 -- 172.21.15.2:0/467283781 wait complete. 2024-09-10T08:10:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.343+0000 7f7677e10740 1 -- start start 2024-09-10T08:10:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.343+0000 7f7677e10740 1 Processor -- start 2024-09-10T08:10:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.343+0000 7f7677e10740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 0x55c0b286bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.343+0000 7f7677e10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2864590 0x55c0b286c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.343+0000 7f7677e10740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b286c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.343+0000 7f7677e10740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c0b2871a20 con 0x55c0b286ee50 2024-09-10T08:10:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.343+0000 7f7677e10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c0b27850f0 con 0x55c0b2864590 2024-09-10T08:10:19.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.343+0000 7f7677e10740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c0b2871ba0 con 0x55c0b2863c70 2024-09-10T08:10:19.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.343+0000 7f7677e10740 10 client.0 _ll_register_callbacks cb 0x55c0b276cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:19.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.344+0000 7f766e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b286c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.344+0000 7f7667e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2864590 0x55c0b286c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.344+0000 7f7667e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2864590 0x55c0b286c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:55978/0 (socket says 172.21.15.2:55978) 2024-09-10T08:10:19.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.344+0000 7f7667e00640 1 -- 172.21.15.2:0/635607815 learned_addr learned my addr 172.21.15.2:0/635607815 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:19.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.344+0000 7f766de00640 1 --2- 172.21.15.2:0/635607815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 0x55c0b286bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.344+0000 7f7667e00640 1 -- 172.21.15.2:0/635607815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 msgr2=0x55c0b286bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:19.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.344+0000 7f7667e00640 1 --2- 172.21.15.2:0/635607815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 0x55c0b286bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.344+0000 7f7667e00640 1 -- 172.21.15.2:0/635607815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 msgr2=0x55c0b286c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:19.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.344+0000 7f7667e00640 1 --2- 172.21.15.2:0/635607815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b286c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.344+0000 7f7667e00640 1 -- 172.21.15.2:0/635607815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b253b3c0 con 0x55c0b2864590 2024-09-10T08:10:19.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.344+0000 7f766e800640 1 --2- 172.21.15.2:0/635607815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b286c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:19.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.344+0000 7f7667e00640 1 --2- 172.21.15.2:0/635607815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2864590 0x55c0b286c1b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7658000c00 tx=0x7f7658004370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8c7118f878af0d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:19.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.344+0000 7f7653e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:19.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.345+0000 7f7653e00640 1 -- 172.21.15.2:0/635607815 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658027030 con 0x55c0b2864590 2024-09-10T08:10:19.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.345+0000 7f7677e10740 1 -- 172.21.15.2:0/635607815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76580041c0 con 0x55c0b2864590 2024-09-10T08:10:19.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.345+0000 7f7653e00640 1 -- 172.21.15.2:0/635607815 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f765801f070 con 0x55c0b2864590 2024-09-10T08:10:19.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.345+0000 7f7653e00640 1 -- 172.21.15.2:0/635607815 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7658031050 con 0x55c0b2864590 2024-09-10T08:10:19.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.345+0000 7f7677e10740 10 client.25853 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:19.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.345+0000 7f7677e10740 1 -- 172.21.15.2:0/635607815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c0b287d9f0 con 0x55c0b2864590 2024-09-10T08:10:19.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.345+0000 7f7653e00640 1 -- 172.21.15.2:0/635607815 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76580041c0 con 0x55c0b2864590 2024-09-10T08:10:19.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.346+0000 7f7653e00640 1 -- 172.21.15.2:0/635607815 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f76580046c0 con 0x55c0b2864590 2024-09-10T08:10:19.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.346+0000 7f7677e10740 10 client.25853 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:19.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.346+0000 7f7677e10740 10 client.25853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:19.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.346+0000 7f7677e10740 20 client.25853 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:19.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.346+0000 7f7677e10740 1 --2- 172.21.15.2:0/635607815 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c0b287ee00 0x55c0b28811f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.346+0000 7f7677e10740 4 client.25853 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:19.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.346+0000 7f7677e10740 1 -- 172.21.15.2:0/635607815 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0b27d7950 con 0x55c0b287ee00 2024-09-10T08:10:19.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.346+0000 7f766de00640 1 --2- 172.21.15.2:0/635607815 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c0b287ee00 0x55c0b28811f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.347+0000 7f766de00640 1 --2- 172.21.15.2:0/635607815 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c0b287ee00 0x55c0b28811f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed7f1f60b3e09d server_cookie=40a284edd9013e0c in_seq=0 out_seq=0 2024-09-10T08:10:19.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.347+0000 7f7653e00640 10 client.25853 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:19.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.347+0000 7f7653e00640 1 -- 172.21.15.2:0/635607815 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0b27d7950 con 0x55c0b287ee00 2024-09-10T08:10:19.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.347+0000 7f7653e00640 10 client.25853 handle_command_reply: tid=0 2024-09-10T08:10:19.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.347+0000 7f7677e10740 1 client.25853 shutdown 2024-09-10T08:10:19.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.348+0000 7f7677e10740 1 -- 172.21.15.2:0/635607815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2864590 msgr2=0x55c0b286c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:19.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.348+0000 7f7677e10740 1 --2- 172.21.15.2:0/635607815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2864590 0x55c0b286c1b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7658000c00 tx=0x7f7658004370 comp rx=0 tx=0).stop 2024-09-10T08:10:19.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.348+0000 7f7677e10740 1 -- 172.21.15.2:0/635607815 shutdown_connections 2024-09-10T08:10:19.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.348+0000 7f7677e10740 1 --2- 172.21.15.2:0/635607815 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c0b287ee00 0x55c0b28811f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.348+0000 7f7677e10740 1 --2- 172.21.15.2:0/635607815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b286c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.348+0000 7f7677e10740 1 --2- 172.21.15.2:0/635607815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2864590 0x55c0b286c1b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.348+0000 7f7653e00640 0 client.25853 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:19.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.348+0000 7f7677e10740 1 --2- 172.21.15.2:0/635607815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 0x55c0b286bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.348+0000 7f7677e10740 1 -- 172.21.15.2:0/635607815 >> 172.21.15.2:0/635607815 conn(0x55c0b2860200 msgr2=0x55c0b2865ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:19.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.348+0000 7f7677e10740 1 -- 172.21.15.2:0/635607815 shutdown_connections 2024-09-10T08:10:19.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.348+0000 7f7677e10740 1 -- 172.21.15.2:0/635607815 wait complete. 2024-09-10T08:10:19.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.348+0000 7f7677e10740 20 client.25853 trim_cache size 0 max 16384 2024-09-10T08:10:19.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.362+0000 7f7677e10740 1 Processor -- start 2024-09-10T08:10:19.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.362+0000 7f7677e10740 1 -- start start 2024-09-10T08:10:19.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.362+0000 7f7677e10740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 0x55c0b286aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.362+0000 7f7677e10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2864590 0x55c0b28868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.362+0000 7f7677e10740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b2886e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.362+0000 7f7677e10740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f76701b7d80 con 0x55c0b286ee50 2024-09-10T08:10:19.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.362+0000 7f7677e10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f76701b5650 con 0x55c0b2864590 2024-09-10T08:10:19.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.362+0000 7f7677e10740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76701b34a0 con 0x55c0b2863c70 2024-09-10T08:10:19.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.362+0000 7f766e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b2886e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.362+0000 7f766e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b2886e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37210/0 (socket says 172.21.15.2:37210) 2024-09-10T08:10:19.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.362+0000 7f766e800640 1 -- 172.21.15.2:0/1236821974 learned_addr learned my addr 172.21.15.2:0/1236821974 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:19.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.363+0000 7f766de00640 1 --2- 172.21.15.2:0/1236821974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 0x55c0b286aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.363+0000 7f7667e00640 1 --2- 172.21.15.2:0/1236821974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2864590 0x55c0b28868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.363+0000 7f766e800640 1 -- 172.21.15.2:0/1236821974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 msgr2=0x55c0b286aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:19.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.363+0000 7f766e800640 1 --2- 172.21.15.2:0/1236821974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 0x55c0b286aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.363+0000 7f766e800640 1 -- 172.21.15.2:0/1236821974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2864590 msgr2=0x55c0b28868e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:19.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.363+0000 7f766e800640 1 --2- 172.21.15.2:0/1236821974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2864590 0x55c0b28868e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.363+0000 7f766e800640 1 -- 172.21.15.2:0/1236821974 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b2860030 con 0x55c0b286ee50 2024-09-10T08:10:19.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.363+0000 7f766e800640 1 --2- 172.21.15.2:0/1236821974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b2886e20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7668009ca0 tx=0x7f7668041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19a68973a2b3a32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:19.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.364+0000 7f765ea00640 1 -- 172.21.15.2:0/1236821974 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668003c80 con 0x55c0b286ee50 2024-09-10T08:10:19.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.364+0000 7f765ea00640 1 -- 172.21.15.2:0/1236821974 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7668036040 con 0x55c0b286ee50 2024-09-10T08:10:19.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.364+0000 7f765ea00640 1 -- 172.21.15.2:0/1236821974 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766802fcd0 con 0x55c0b286ee50 2024-09-10T08:10:19.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.364+0000 7f7677e10740 1 -- 172.21.15.2:0/1236821974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 msgr2=0x55c0b2886e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:19.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.364+0000 7f7677e10740 1 --2- 172.21.15.2:0/1236821974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b2886e20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7668009ca0 tx=0x7f7668041000 comp rx=0 tx=0).stop 2024-09-10T08:10:19.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.365+0000 7f7677e10740 1 -- 172.21.15.2:0/1236821974 shutdown_connections 2024-09-10T08:10:19.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.365+0000 7f7677e10740 1 --2- 172.21.15.2:0/1236821974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b286ee50 0x55c0b2886e20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.365+0000 7f7677e10740 1 --2- 172.21.15.2:0/1236821974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b2864590 0x55c0b28868e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.365+0000 7f7677e10740 1 --2- 172.21.15.2:0/1236821974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 0x55c0b286aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.365+0000 7f7677e10740 1 -- 172.21.15.2:0/1236821974 >> 172.21.15.2:0/1236821974 conn(0x55c0b2860200 msgr2=0x55c0b286dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:19.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.365+0000 7f7677e10740 1 -- 172.21.15.2:0/1236821974 shutdown_connections 2024-09-10T08:10:19.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.365+0000 7f7677e10740 1 -- 172.21.15.2:0/1236821974 wait complete. 2024-09-10T08:10:19.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.365+0000 7f7677e10740 1 -- start start 2024-09-10T08:10:19.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.366+0000 7f7677e10740 1 Processor -- start 2024-09-10T08:10:19.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.366+0000 7f7677e10740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 0x55c0b2868690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.366+0000 7f7677e10740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b2864590 0x55c0b2868bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.366+0000 7f7677e10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b286ee50 0x55c0b2875df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.366+0000 7f7677e10740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f76701b3b10 con 0x55c0b2864590 2024-09-10T08:10:19.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.366+0000 7f7677e10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f76701b4f80 con 0x55c0b286ee50 2024-09-10T08:10:19.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.366+0000 7f7677e10740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76701b41e0 con 0x55c0b2863c70 2024-09-10T08:10:19.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.366+0000 7f7677e10740 10 client.0 _ll_register_callbacks cb 0x7f7668009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:19.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.367+0000 7f7667e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b2864590 0x55c0b2868bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.367+0000 7f766de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 0x55c0b2868690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.367+0000 7f7667e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b2864590 0x55c0b2868bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37216/0 (socket says 172.21.15.2:37216) 2024-09-10T08:10:19.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.367+0000 7f7667e00640 1 -- 172.21.15.2:0/728636527 learned_addr learned my addr 172.21.15.2:0/728636527 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:19.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.367+0000 7f766e800640 1 --2- 172.21.15.2:0/728636527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b286ee50 0x55c0b2875df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.367+0000 7f7667e00640 1 -- 172.21.15.2:0/728636527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 msgr2=0x55c0b2868690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:19.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.367+0000 7f7667e00640 1 --2- 172.21.15.2:0/728636527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 0x55c0b2868690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.367+0000 7f7667e00640 1 -- 172.21.15.2:0/728636527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b286ee50 msgr2=0x55c0b2875df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:19.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.367+0000 7f7667e00640 1 --2- 172.21.15.2:0/728636527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b286ee50 0x55c0b2875df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.367+0000 7f7667e00640 1 -- 172.21.15.2:0/728636527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b2860e90 con 0x55c0b2864590 2024-09-10T08:10:19.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.367+0000 7f7667e00640 1 --2- 172.21.15.2:0/728636527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b2864590 0x55c0b2868bd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7658003e60 tx=0x7f7658009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8af72310a30c28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:19.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.367+0000 7f7653e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:19.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.367+0000 7f7653e00640 1 -- 172.21.15.2:0/728636527 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765803a060 con 0x55c0b2864590 2024-09-10T08:10:19.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.368+0000 7f7677e10740 1 -- 172.21.15.2:0/728636527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7668036040 con 0x55c0b2864590 2024-09-10T08:10:19.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.368+0000 7f7653e00640 1 -- 172.21.15.2:0/728636527 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7658004140 con 0x55c0b2864590 2024-09-10T08:10:19.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.368+0000 7f7653e00640 1 -- 172.21.15.2:0/728636527 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7658027040 con 0x55c0b2864590 2024-09-10T08:10:19.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.368+0000 7f7677e10740 10 client.15825 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:19.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.368+0000 7f7677e10740 1 -- 172.21.15.2:0/728636527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c0b2877420 con 0x55c0b2864590 2024-09-10T08:10:19.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.368+0000 7f7653e00640 1 -- 172.21.15.2:0/728636527 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7668036040 con 0x55c0b2864590 2024-09-10T08:10:19.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.368+0000 7f7653e00640 1 -- 172.21.15.2:0/728636527 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7658004de0 con 0x55c0b2864590 2024-09-10T08:10:19.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.368+0000 7f7677e10740 10 client.15825 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:19.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.368+0000 7f7677e10740 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:19.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.368+0000 7f7677e10740 20 client.15825 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:19.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.369+0000 7f7677e10740 1 --2- 172.21.15.2:0/728636527 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c0b28e98b0 0x55c0b28ebca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:19.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.369+0000 7f7677e10740 4 client.15825 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:10:19.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.369+0000 7f7677e10740 1 -- 172.21.15.2:0/728636527 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0b27d1e60 con 0x55c0b28e98b0 2024-09-10T08:10:19.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.369+0000 7f766de00640 1 --2- 172.21.15.2:0/728636527 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c0b28e98b0 0x55c0b28ebca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:19.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.369+0000 7f766de00640 1 --2- 172.21.15.2:0/728636527 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c0b28e98b0 0x55c0b28ebca0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5d29da5d5915246 server_cookie=31ebc5921444194 in_seq=0 out_seq=0 2024-09-10T08:10:19.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.369+0000 7f7653e00640 10 client.15825 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:19.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7653e00640 1 -- 172.21.15.2:0/728636527 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0b27d1e60 con 0x55c0b28e98b0 2024-09-10T08:10:19.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7653e00640 10 client.15825 handle_command_reply: tid=0 2024-09-10T08:10:19.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7677e10740 1 client.15825 shutdown 2024-09-10T08:10:19.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7677e10740 1 -- 172.21.15.2:0/728636527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b2864590 msgr2=0x55c0b2868bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:19.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7677e10740 1 --2- 172.21.15.2:0/728636527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b2864590 0x55c0b2868bd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7658003e60 tx=0x7f7658009550 comp rx=0 tx=0).stop 2024-09-10T08:10:19.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7677e10740 1 -- 172.21.15.2:0/728636527 shutdown_connections 2024-09-10T08:10:19.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7677e10740 1 --2- 172.21.15.2:0/728636527 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c0b28e98b0 0x55c0b28ebca0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7677e10740 1 --2- 172.21.15.2:0/728636527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b286ee50 0x55c0b2875df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7677e10740 1 --2- 172.21.15.2:0/728636527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b2864590 0x55c0b2868bd0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7677e10740 1 --2- 172.21.15.2:0/728636527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0b2863c70 0x55c0b2868690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7653e00640 0 client.15825 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:19.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7677e10740 1 -- 172.21.15.2:0/728636527 >> 172.21.15.2:0/728636527 conn(0x55c0b2860200 msgr2=0x55c0b285eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:19.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7677e10740 1 -- 172.21.15.2:0/728636527 shutdown_connections 2024-09-10T08:10:19.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7677e10740 1 -- 172.21.15.2:0/728636527 wait complete. 2024-09-10T08:10:19.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.370+0000 7f7677e10740 20 client.15825 trim_cache size 0 max 16384 2024-09-10T08:10:19.827 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:10:19.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.371+0000 7f7675200640 1 -- 172.21.15.2:0/725660304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76540a2b80 msgr2=0x7f76540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:19.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.371+0000 7f7675200640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76540a2b80 0x7f76540a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7668006010 tx=0x7f7668004a80 comp rx=0 tx=0).stop 2024-09-10T08:10:19.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.371+0000 7f7675200640 1 -- 172.21.15.2:0/725660304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7670076720 msgr2=0x7f76701a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:19.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.371+0000 7f7675200640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7670076720 0x7f76701a46c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f766000d770 tx=0x7f766000dc30 comp rx=0 tx=0).stop 2024-09-10T08:10:19.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.371+0000 7f7675200640 1 -- 172.21.15.2:0/725660304 shutdown_connections 2024-09-10T08:10:19.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.371+0000 7f7675200640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7670112a50 0x7f767019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.371+0000 7f7675200640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f76540a2b80 0x7f76540a5040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.371+0000 7f7675200640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7670077040 0x7f76701a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.371+0000 7f7675200640 1 --2- 172.21.15.2:0/725660304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7670076720 0x7f76701a46c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:19.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.371+0000 7f7675200640 1 -- 172.21.15.2:0/725660304 >> 172.21.15.2:0/725660304 conn(0x7f76700ffa50 msgr2=0x7f7670101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:19.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.371+0000 7f7675200640 1 -- 172.21.15.2:0/725660304 shutdown_connections 2024-09-10T08:10:19.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:19.371+0000 7f7675200640 1 -- 172.21.15.2:0/725660304 wait complete. 2024-09-10T08:10:21.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:20 smithi155 ceph-mon[35405]: pgmap v453: 129 pgs: 129 active+clean; 681 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:10:21.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:20 smithi107 ceph-mon[35048]: pgmap v453: 129 pgs: 129 active+clean; 681 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:10:21.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:20 smithi002 ceph-mon[30563]: pgmap v453: 129 pgs: 129 active+clean; 681 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:10:21.518 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:10:21.518 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:10:21.906 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:22.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.522+0000 7fbdcd800640 1 -- 172.21.15.2:0/1238854920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc8106f10 msgr2=0x7fbdc810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:22.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.522+0000 7fbdcd800640 1 --2- 172.21.15.2:0/1238854920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc8106f10 0x7fbdc810ea90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009ca0 tx=0x7fbdc002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.522+0000 7fbdcd800640 1 -- 172.21.15.2:0/1238854920 shutdown_connections 2024-09-10T08:10:22.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.522+0000 7fbdcd800640 1 --2- 172.21.15.2:0/1238854920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc8106f10 0x7fbdc810ea90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.522+0000 7fbdcd800640 1 --2- 172.21.15.2:0/1238854920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdc8102790 0x7fbdc81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.522+0000 7fbdcd800640 1 --2- 172.21.15.2:0/1238854920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdc8101de0 0x7fbdc81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.522+0000 7fbdcd800640 1 -- 172.21.15.2:0/1238854920 >> 172.21.15.2:0/1238854920 conn(0x7fbdc80fd8c0 msgr2=0x7fbdc80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:22.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.523+0000 7fbdcd800640 1 -- 172.21.15.2:0/1238854920 shutdown_connections 2024-09-10T08:10:22.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.523+0000 7fbdcd800640 1 -- 172.21.15.2:0/1238854920 wait complete. 2024-09-10T08:10:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.523+0000 7fbdcd800640 1 Processor -- start 2024-09-10T08:10:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.524+0000 7fbdcd800640 1 -- start start 2024-09-10T08:10:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.524+0000 7fbdcd800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdc8101de0 0x7fbdc8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.524+0000 7fbdcd800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc8102790 0x7fbdc8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.524+0000 7fbdcd800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdc8106f10 0x7fbdc819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.524+0000 7fbdcd800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc8111fc0 con 0x7fbdc8102790 2024-09-10T08:10:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.524+0000 7fbdcd800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc8111e40 con 0x7fbdc8106f10 2024-09-10T08:10:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.524+0000 7fbdcd800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc8112140 con 0x7fbdc8101de0 2024-09-10T08:10:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.524+0000 7fbdbe000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc8102790 0x7fbdc8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.524+0000 7fbdbe000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc8102790 0x7fbdc8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37232/0 (socket says 172.21.15.2:37232) 2024-09-10T08:10:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.524+0000 7fbdbe000640 1 -- 172.21.15.2:0/432665348 learned_addr learned my addr 172.21.15.2:0/432665348 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.524+0000 7fbdc6a00640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdc8101de0 0x7fbdc8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.525+0000 7fbdc7400640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdc8106f10 0x7fbdc819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.525+0000 7fbdbe000640 1 -- 172.21.15.2:0/432665348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdc8101de0 msgr2=0x7fbdc8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.525+0000 7fbdbe000640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdc8101de0 0x7fbdc8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.525+0000 7fbdbe000640 1 -- 172.21.15.2:0/432665348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdc8106f10 msgr2=0x7fbdc819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.525+0000 7fbdbe000640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdc8106f10 0x7fbdc819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.525+0000 7fbdbe000640 1 -- 172.21.15.2:0/432665348 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc819d3f0 con 0x7fbdc8102790 2024-09-10T08:10:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.525+0000 7fbdbe000640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc8102790 0x7fbdc8198980 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000d950 tx=0x7fbdb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.525+0000 7fbdbfe00640 1 -- 172.21.15.2:0/432665348 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0017070 con 0x7fbdc8102790 2024-09-10T08:10:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.525+0000 7fbdbfe00640 1 -- 172.21.15.2:0/432665348 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb0009c30 con 0x7fbdc8102790 2024-09-10T08:10:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.525+0000 7fbdcd800640 1 -- 172.21.15.2:0/432665348 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc819d6e0 con 0x7fbdc8102790 2024-09-10T08:10:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.525+0000 7fbdbfe00640 1 -- 172.21.15.2:0/432665348 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb000f6b0 con 0x7fbdc8102790 2024-09-10T08:10:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.526+0000 7fbdcd800640 1 -- 172.21.15.2:0/432665348 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc81a4fc0 con 0x7fbdc8102790 2024-09-10T08:10:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.528+0000 7fbdd0210740 1 Processor -- start 2024-09-10T08:10:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.529+0000 7fbdd0210740 1 -- start start 2024-09-10T08:10:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.529+0000 7fbdbfe00640 1 -- 172.21.15.2:0/432665348 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbdb00109d0 con 0x7fbdc8102790 2024-09-10T08:10:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.529+0000 7fbdd0210740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 0x5574197c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.529+0000 7fbdd0210740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.529+0000 7fbdd0210740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197ceca0 0x5574197d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.529+0000 7fbdd0210740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55741972a290 con 0x5574197c3ac0 2024-09-10T08:10:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.529+0000 7fbdd0210740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5574196d1b40 con 0x5574197c43e0 2024-09-10T08:10:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.529+0000 7fbdd0210740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557419726250 con 0x5574197ceca0 2024-09-10T08:10:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.529+0000 7fbdc7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 0x5574197c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.529+0000 7fbdc7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 0x5574197c3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37242/0 (socket says 172.21.15.2:37242) 2024-09-10T08:10:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.529+0000 7fbdc7400640 1 -- 172.21.15.2:0/1975223894 learned_addr learned my addr 172.21.15.2:0/1975223894 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.530+0000 7fbdc6a00640 1 --2- 172.21.15.2:0/1975223894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.530+0000 7fbdbe000640 1 --2- 172.21.15.2:0/1975223894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197ceca0 0x5574197d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.530+0000 7fbdc7400640 1 -- 172.21.15.2:0/1975223894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197ceca0 msgr2=0x5574197d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.530+0000 7fbdc7400640 1 --2- 172.21.15.2:0/1975223894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197ceca0 0x5574197d1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.530+0000 7fbdc7400640 1 -- 172.21.15.2:0/1975223894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 msgr2=0x5574197ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.530+0000 7fbdc7400640 1 --2- 172.21.15.2:0/1975223894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.530+0000 7fbdc7400640 1 -- 172.21.15.2:0/1975223894 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55741949b6a0 con 0x5574197c3ac0 2024-09-10T08:10:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.530+0000 7fbdc7400640 1 --2- 172.21.15.2:0/1975223894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 0x5574197c3ea0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009920 tx=0x7fbdc0009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01957fdbf5cd930 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.531+0000 7fbdb7e00640 1 -- 172.21.15.2:0/1975223894 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0031e30 con 0x5574197c3ac0 2024-09-10T08:10:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.531+0000 7fbdb7e00640 1 -- 172.21.15.2:0/1975223894 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc0036040 con 0x5574197c3ac0 2024-09-10T08:10:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.531+0000 7fbdb7e00640 1 -- 172.21.15.2:0/1975223894 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00043a0 con 0x5574197c3ac0 2024-09-10T08:10:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.531+0000 7fbdbfe00640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbdac0a2b80 0x7fbdac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.531+0000 7fbdbfe00640 1 -- 172.21.15.2:0/432665348 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbdb00cefe0 con 0x7fbdc8102790 2024-09-10T08:10:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.531+0000 7fbdd0210740 1 -- 172.21.15.2:0/1975223894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 msgr2=0x5574197c3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:22.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.531+0000 7fbdc6a00640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbdac0a2b80 0x7fbdac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.531+0000 7fbdd0210740 1 --2- 172.21.15.2:0/1975223894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 0x5574197c3ea0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009920 tx=0x7fbdc0009950 comp rx=0 tx=0).stop 2024-09-10T08:10:22.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.532+0000 7fbdd0210740 1 -- 172.21.15.2:0/1975223894 shutdown_connections 2024-09-10T08:10:22.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.532+0000 7fbdd0210740 1 --2- 172.21.15.2:0/1975223894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197ceca0 0x5574197d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.532+0000 7fbdd0210740 1 --2- 172.21.15.2:0/1975223894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.532+0000 7fbdd0210740 1 --2- 172.21.15.2:0/1975223894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 0x5574197c3ea0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.532+0000 7fbdd0210740 1 -- 172.21.15.2:0/1975223894 >> 172.21.15.2:0/1975223894 conn(0x5574197c0050 msgr2=0x5574197c0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:22.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.532+0000 7fbdd0210740 1 -- 172.21.15.2:0/1975223894 shutdown_connections 2024-09-10T08:10:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.532+0000 7fbdd0210740 1 -- 172.21.15.2:0/1975223894 wait complete. 2024-09-10T08:10:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.532+0000 7fbdc6a00640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbdac0a2b80 0x7fbdac0a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x5574197c0eb0 tx=0x7fbdb800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.532+0000 7fbdd0210740 1 -- start start 2024-09-10T08:10:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.532+0000 7fbdd0210740 1 Processor -- start 2024-09-10T08:10:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.533+0000 7fbdd0210740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 0x5574197ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.533+0000 7fbdd0210740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.533+0000 7fbdd0210740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197ceca0 0x5574197cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.533+0000 7fbdd0210740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5574197d1910 con 0x5574197c3ac0 2024-09-10T08:10:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.533+0000 7fbdd0210740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5574196e4fa0 con 0x5574197c43e0 2024-09-10T08:10:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.533+0000 7fbdd0210740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5574197d1a90 con 0x5574197ceca0 2024-09-10T08:10:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.533+0000 7fbdd0210740 10 client.0 _ll_register_callbacks cb 0x557419438ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.533+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 0x5574197ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.533+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 0x5574197ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37250/0 (socket says 172.21.15.2:37250) 2024-09-10T08:10:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.533+0000 7fbdc6a00640 1 -- 172.21.15.2:0/1119997803 learned_addr learned my addr 172.21.15.2:0/1119997803 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.533+0000 7fbdbe000640 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.533+0000 7fbdc7400640 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197ceca0 0x5574197cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.534+0000 7fbdc6a00640 1 -- 172.21.15.2:0/1119997803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197ceca0 msgr2=0x5574197cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.534+0000 7fbdc6a00640 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197ceca0 0x5574197cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.534+0000 7fbdc6a00640 1 -- 172.21.15.2:0/1119997803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 msgr2=0x5574197cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.534+0000 7fbdc6a00640 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.534+0000 7fbdc6a00640 1 -- 172.21.15.2:0/1119997803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574197cc680 con 0x5574197c3ac0 2024-09-10T08:10:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.534+0000 7fbdc7400640 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197ceca0 0x5574197cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.534+0000 7fbdbe000640 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.534+0000 7fbdc6a00640 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 0x5574197ce620 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8003d30 tx=0x7fbdb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67967097ed1c0670 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.534+0000 7fbdb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.534+0000 7fbdb4c00640 1 -- 172.21.15.2:0/1119997803 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb802c060 con 0x5574197c3ac0 2024-09-10T08:10:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.534+0000 7fbdb4c00640 1 -- 172.21.15.2:0/1119997803 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbdb8017030 con 0x5574197c3ac0 2024-09-10T08:10:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.534+0000 7fbdd0210740 1 -- 172.21.15.2:0/1119997803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc0036040 con 0x5574197c3ac0 2024-09-10T08:10:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.535+0000 7fbdb4c00640 1 -- 172.21.15.2:0/1119997803 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbdb8033050 con 0x5574197c3ac0 2024-09-10T08:10:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.535+0000 7fbdd0210740 10 client.15837 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.535+0000 7fbdd0210740 1 -- 172.21.15.2:0/1119997803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5574197dd8c0 con 0x5574197c3ac0 2024-09-10T08:10:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.535+0000 7fbdb4c00640 1 -- 172.21.15.2:0/1119997803 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc0036040 con 0x5574197c3ac0 2024-09-10T08:10:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.535+0000 7fbdb4c00640 1 -- 172.21.15.2:0/1119997803 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbdb801c3f0 con 0x5574197c3ac0 2024-09-10T08:10:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.535+0000 7fbdd0210740 10 client.15837 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.535+0000 7fbdd0210740 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.535+0000 7fbdd0210740 20 client.15837 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.535+0000 7fbdd0210740 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5574197ded80 0x5574197e11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.535+0000 7fbdd0210740 4 client.15837 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.535+0000 7fbdd0210740 1 -- 172.21.15.2:0/1119997803 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557419737800 con 0x5574197ded80 2024-09-10T08:10:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.535+0000 7fbdbe000640 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5574197ded80 0x5574197e11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.536+0000 7fbdbe000640 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5574197ded80 0x5574197e11d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95729230d3efba01 server_cookie=c1efd142476a9a7f in_seq=0 out_seq=0 2024-09-10T08:10:22.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.536+0000 7fbdb4c00640 10 client.15837 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:22.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdb4c00640 1 -- 172.21.15.2:0/1119997803 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557419737800 con 0x5574197ded80 2024-09-10T08:10:22.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdb4c00640 10 client.15837 handle_command_reply: tid=0 2024-09-10T08:10:22.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdd0210740 1 client.15837 shutdown 2024-09-10T08:10:22.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdd0210740 1 -- 172.21.15.2:0/1119997803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 msgr2=0x5574197ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:22.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdd0210740 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 0x5574197ce620 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8003d30 tx=0x7fbdb8004730 comp rx=0 tx=0).stop 2024-09-10T08:10:22.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdd0210740 1 -- 172.21.15.2:0/1119997803 shutdown_connections 2024-09-10T08:10:22.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdd0210740 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5574197ded80 0x5574197e11d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdd0210740 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197ceca0 0x5574197cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdd0210740 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdd0210740 1 --2- 172.21.15.2:0/1119997803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c3ac0 0x5574197ce620 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdb4c00640 0 client.15837 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:22.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdd0210740 1 -- 172.21.15.2:0/1119997803 >> 172.21.15.2:0/1119997803 conn(0x5574197c0050 msgr2=0x5574197bf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:22.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdd0210740 1 -- 172.21.15.2:0/1119997803 shutdown_connections 2024-09-10T08:10:22.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdd0210740 1 -- 172.21.15.2:0/1119997803 wait complete. 2024-09-10T08:10:22.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.537+0000 7fbdd0210740 20 client.15837 trim_cache size 0 max 16384 2024-09-10T08:10:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.556+0000 7fbdd0210740 1 Processor -- start 2024-09-10T08:10:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.556+0000 7fbdd0210740 1 -- start start 2024-09-10T08:10:22.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.556+0000 7fbdd0210740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197c3ac0 0x5574197c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.556+0000 7fbdd0210740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c43e0 0x5574197e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.556+0000 7fbdd0210740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197ceca0 0x5574197e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.556+0000 7fbdd0210740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81b17b0 con 0x5574197c43e0 2024-09-10T08:10:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.556+0000 7fbdd0210740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81af080 con 0x5574197ceca0 2024-09-10T08:10:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.556+0000 7fbdd0210740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81aced0 con 0x5574197c3ac0 2024-09-10T08:10:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.556+0000 7fbdbe000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c43e0 0x5574197e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.556+0000 7fbdbe000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c43e0 0x5574197e6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37254/0 (socket says 172.21.15.2:37254) 2024-09-10T08:10:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.556+0000 7fbdbe000640 1 -- 172.21.15.2:0/2657491968 learned_addr learned my addr 172.21.15.2:0/2657491968 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.557+0000 7fbdc6a00640 1 --2- 172.21.15.2:0/2657491968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197c3ac0 0x5574197c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.557+0000 7fbdc7400640 1 --2- 172.21.15.2:0/2657491968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197ceca0 0x5574197e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.557+0000 7fbdbe000640 1 -- 172.21.15.2:0/2657491968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197c3ac0 msgr2=0x5574197c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:22.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.557+0000 7fbdbe000640 1 --2- 172.21.15.2:0/2657491968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197c3ac0 0x5574197c6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.557+0000 7fbdbe000640 1 -- 172.21.15.2:0/2657491968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197ceca0 msgr2=0x5574197e6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:22.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.557+0000 7fbdbe000640 1 --2- 172.21.15.2:0/2657491968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197ceca0 0x5574197e6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.557+0000 7fbdbe000640 1 -- 172.21.15.2:0/2657491968 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574197e1850 con 0x5574197c43e0 2024-09-10T08:10:22.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.557+0000 7fbdc7400640 1 --2- 172.21.15.2:0/2657491968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197ceca0 0x5574197e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:22.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.557+0000 7fbdc6a00640 1 --2- 172.21.15.2:0/2657491968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197c3ac0 0x5574197c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:22.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.557+0000 7fbdbe000640 1 --2- 172.21.15.2:0/2657491968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c43e0 0x5574197e6aa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0097e00 tx=0x7fbdb00d2a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203bcd766b6d6a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.557+0000 7fbdb7400640 1 -- 172.21.15.2:0/2657491968 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0020040 con 0x5574197c43e0 2024-09-10T08:10:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.558+0000 7fbdb7400640 1 -- 172.21.15.2:0/2657491968 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb0009de0 con 0x5574197c43e0 2024-09-10T08:10:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.558+0000 7fbdb7400640 1 -- 172.21.15.2:0/2657491968 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb00cc7e0 con 0x5574197c43e0 2024-09-10T08:10:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.558+0000 7fbdd0210740 1 -- 172.21.15.2:0/2657491968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c43e0 msgr2=0x5574197e6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.558+0000 7fbdd0210740 1 --2- 172.21.15.2:0/2657491968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c43e0 0x5574197e6aa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0097e00 tx=0x7fbdb00d2a60 comp rx=0 tx=0).stop 2024-09-10T08:10:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.558+0000 7fbdd0210740 1 -- 172.21.15.2:0/2657491968 shutdown_connections 2024-09-10T08:10:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.558+0000 7fbdd0210740 1 --2- 172.21.15.2:0/2657491968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197ceca0 0x5574197e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.558+0000 7fbdd0210740 1 --2- 172.21.15.2:0/2657491968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197c43e0 0x5574197e6aa0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.558+0000 7fbdd0210740 1 --2- 172.21.15.2:0/2657491968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197c3ac0 0x5574197c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.558+0000 7fbdd0210740 1 -- 172.21.15.2:0/2657491968 >> 172.21.15.2:0/2657491968 conn(0x5574197c0050 msgr2=0x5574197c65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.558+0000 7fbdd0210740 1 -- 172.21.15.2:0/2657491968 shutdown_connections 2024-09-10T08:10:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.559+0000 7fbdd0210740 1 -- 172.21.15.2:0/2657491968 wait complete. 2024-09-10T08:10:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.559+0000 7fbdd0210740 1 -- start start 2024-09-10T08:10:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.559+0000 7fbdd0210740 1 Processor -- start 2024-09-10T08:10:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdd0210740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197c3ac0 0x5574197d6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdd0210740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197d7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdd0210740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197ceca0 0x5574197d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdd0210740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81ad540 con 0x5574197ceca0 2024-09-10T08:10:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdd0210740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81ae9b0 con 0x5574197c43e0 2024-09-10T08:10:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdd0210740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81adc10 con 0x5574197c3ac0 2024-09-10T08:10:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdd0210740 10 client.0 _ll_register_callbacks cb 0x7fbdb800bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdc7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197ceca0 0x5574197d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdbe000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197c3ac0 0x5574197d6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdc7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197ceca0 0x5574197d3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37258/0 (socket says 172.21.15.2:37258) 2024-09-10T08:10:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdc7400640 1 -- 172.21.15.2:0/3517765264 learned_addr learned my addr 172.21.15.2:0/3517765264 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdbe000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197d7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:56080/0 (socket says 172.21.15.2:56080) 2024-09-10T08:10:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdc7400640 1 -- 172.21.15.2:0/3517765264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197c3ac0 msgr2=0x5574197d6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdc7400640 1 --2- 172.21.15.2:0/3517765264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197c3ac0 0x5574197d6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdc7400640 1 -- 172.21.15.2:0/3517765264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 msgr2=0x5574197d7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:22.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdc7400640 1 --2- 172.21.15.2:0/3517765264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197d7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdc7400640 1 -- 172.21.15.2:0/3517765264 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574197d41f0 con 0x5574197ceca0 2024-09-10T08:10:22.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdbe000640 1 --2- 172.21.15.2:0/3517765264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:22.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.560+0000 7fbdc6a00640 1 --2- 172.21.15.2:0/3517765264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197c3ac0 0x5574197d6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:22.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.561+0000 7fbdc7400640 1 --2- 172.21.15.2:0/3517765264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197ceca0 0x5574197d3cb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbdc002a900 tx=0x7fbdc00095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a12249972350c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:22.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.561+0000 7fbdb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:22.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.561+0000 7fbdb4c00640 1 -- 172.21.15.2:0/3517765264 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0003970 con 0x5574197ceca0 2024-09-10T08:10:22.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.561+0000 7fbdb4c00640 1 -- 172.21.15.2:0/3517765264 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbdc0003b10 con 0x5574197ceca0 2024-09-10T08:10:22.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.561+0000 7fbdd0210740 1 -- 172.21.15.2:0/3517765264 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdb0009de0 con 0x5574197ceca0 2024-09-10T08:10:22.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.561+0000 7fbdb4c00640 1 -- 172.21.15.2:0/3517765264 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbdc0003d10 con 0x5574197ceca0 2024-09-10T08:10:22.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.561+0000 7fbdd0210740 10 client.15843 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:22.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.561+0000 7fbdd0210740 1 -- 172.21.15.2:0/3517765264 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55741984e580 con 0x5574197ceca0 2024-09-10T08:10:22.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.562+0000 7fbdb4c00640 1 -- 172.21.15.2:0/3517765264 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb0009de0 con 0x5574197ceca0 2024-09-10T08:10:22.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.562+0000 7fbdb4c00640 1 -- 172.21.15.2:0/3517765264 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbdc002f870 con 0x5574197ceca0 2024-09-10T08:10:22.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.562+0000 7fbdd0210740 10 client.15843 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:22.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.562+0000 7fbdd0210740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:22.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.562+0000 7fbdd0210740 20 client.15843 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:22.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.562+0000 7fbdd0210740 1 --2- 172.21.15.2:0/3517765264 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55741984e860 0x557419850c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:22.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.562+0000 7fbdd0210740 4 client.15843 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:10:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.562+0000 7fbdd0210740 1 -- 172.21.15.2:0/3517765264 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557419731d10 con 0x55741984e860 2024-09-10T08:10:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.562+0000 7fbdc6a00640 1 --2- 172.21.15.2:0/3517765264 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55741984e860 0x557419850c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.563+0000 7fbdc6a00640 1 --2- 172.21.15.2:0/3517765264 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55741984e860 0x557419850c50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7baf782d0b398b1f server_cookie=912f40e125de4481 in_seq=0 out_seq=0 2024-09-10T08:10:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.563+0000 7fbdb4c00640 10 client.15843 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.564+0000 7fbdb4c00640 1 -- 172.21.15.2:0/3517765264 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557419731d10 con 0x55741984e860 2024-09-10T08:10:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.564+0000 7fbdb4c00640 10 client.15843 handle_command_reply: tid=0 2024-09-10T08:10:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.564+0000 7fbdd0210740 1 client.15843 shutdown 2024-09-10T08:10:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.564+0000 7fbdd0210740 1 -- 172.21.15.2:0/3517765264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197ceca0 msgr2=0x5574197d3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.564+0000 7fbdd0210740 1 --2- 172.21.15.2:0/3517765264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197ceca0 0x5574197d3cb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbdc002a900 tx=0x7fbdc00095d0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.564+0000 7fbdd0210740 1 -- 172.21.15.2:0/3517765264 shutdown_connections 2024-09-10T08:10:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.564+0000 7fbdd0210740 1 --2- 172.21.15.2:0/3517765264 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55741984e860 0x557419850c50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.564+0000 7fbdd0210740 1 --2- 172.21.15.2:0/3517765264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574197ceca0 0x5574197d3cb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.565+0000 7fbdd0210740 1 --2- 172.21.15.2:0/3517765264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574197c43e0 0x5574197d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.565+0000 7fbdd0210740 1 --2- 172.21.15.2:0/3517765264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574197c3ac0 0x5574197d6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.565+0000 7fbdd0210740 1 -- 172.21.15.2:0/3517765264 >> 172.21.15.2:0/3517765264 conn(0x5574197c0050 msgr2=0x5574197bf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.565+0000 7fbdb4c00640 0 client.15843 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:22.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.565+0000 7fbdd0210740 1 -- 172.21.15.2:0/3517765264 shutdown_connections 2024-09-10T08:10:22.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.565+0000 7fbdd0210740 1 -- 172.21.15.2:0/3517765264 wait complete. 2024-09-10T08:10:22.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.565+0000 7fbdd0210740 20 client.15843 trim_cache size 0 max 16384 2024-09-10T08:10:22.570 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:10:22.570 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:10:22.570 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "566f2d9b-c743-4e9e-898d-23cbb67109ca", 2024-09-10T08:10:22.570 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:10:22.570 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:10:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.566+0000 7fbdcd800640 1 -- 172.21.15.2:0/432665348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbdac0a2b80 msgr2=0x7fbdac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.566+0000 7fbdcd800640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbdac0a2b80 0x7fbdac0a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x5574197c0eb0 tx=0x7fbdb800b390 comp rx=0 tx=0).stop 2024-09-10T08:10:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.566+0000 7fbdcd800640 1 -- 172.21.15.2:0/432665348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc8102790 msgr2=0x7fbdc8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.566+0000 7fbdcd800640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc8102790 0x7fbdc8198980 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000d950 tx=0x7fbdb000de10 comp rx=0 tx=0).stop 2024-09-10T08:10:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.566+0000 7fbdcd800640 1 -- 172.21.15.2:0/432665348 shutdown_connections 2024-09-10T08:10:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.566+0000 7fbdcd800640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdc8106f10 0x7fbdc819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.566+0000 7fbdcd800640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdc8102790 0x7fbdc8198980 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.566+0000 7fbdcd800640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbdac0a2b80 0x7fbdac0a5040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.566+0000 7fbdcd800640 1 --2- 172.21.15.2:0/432665348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdc8101de0 0x7fbdc8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:22.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.566+0000 7fbdcd800640 1 -- 172.21.15.2:0/432665348 >> 172.21.15.2:0/432665348 conn(0x7fbdc80fd8c0 msgr2=0x7fbdc80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:22.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.566+0000 7fbdcd800640 1 -- 172.21.15.2:0/432665348 shutdown_connections 2024-09-10T08:10:22.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:22.566+0000 7fbdcd800640 1 -- 172.21.15.2:0/432665348 wait complete. 2024-09-10T08:10:23.122 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:22 smithi002 ceph-mon[30563]: pgmap v454: 129 pgs: 129 active+clean; 711 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T08:10:23.123 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:22 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:10:23.123 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:22 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:23.123 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:22 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:10:23.123 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:22 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:23.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:22 smithi155 ceph-mon[35405]: pgmap v454: 129 pgs: 129 active+clean; 711 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T08:10:23.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:22 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:10:23.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:22 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:23.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:22 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:10:23.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:22 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:23.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:22 smithi107 ceph-mon[35048]: pgmap v454: 129 pgs: 129 active+clean; 711 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T08:10:23.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:22 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:10:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:22 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:22 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:10:23.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:22 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:23.440 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:10:23.823 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:24.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:23 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:10:24.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:23 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:10:24.174 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:23 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:10:24.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.447+0000 7f45bea00640 1 -- 172.21.15.2:0/405143907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45b81049a0 msgr2=0x7f45b81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:24.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.447+0000 7f45bea00640 1 --2- 172.21.15.2:0/405143907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45b81049a0 0x7f45b81089f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009900 tx=0x7f45a002a410 comp rx=0 tx=0).stop 2024-09-10T08:10:24.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.447+0000 7f45bea00640 1 -- 172.21.15.2:0/405143907 shutdown_connections 2024-09-10T08:10:24.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.447+0000 7f45bea00640 1 --2- 172.21.15.2:0/405143907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45b8109120 0x7f45b8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.447+0000 7f45bea00640 1 --2- 172.21.15.2:0/405143907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45b81049a0 0x7f45b81089f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.447+0000 7f45bea00640 1 --2- 172.21.15.2:0/405143907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b8103ff0 0x7f45b81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.447+0000 7f45bea00640 1 -- 172.21.15.2:0/405143907 >> 172.21.15.2:0/405143907 conn(0x7f45b80ffa90 msgr2=0x7f45b8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:24.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.447+0000 7f45bea00640 1 -- 172.21.15.2:0/405143907 shutdown_connections 2024-09-10T08:10:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.447+0000 7f45bea00640 1 -- 172.21.15.2:0/405143907 wait complete. 2024-09-10T08:10:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.448+0000 7f45bea00640 1 Processor -- start 2024-09-10T08:10:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.448+0000 7f45bea00640 1 -- start start 2024-09-10T08:10:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.448+0000 7f45bea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45b8103ff0 0x7f45b819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.448+0000 7f45bea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b81049a0 0x7f45b819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.448+0000 7f45bea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45b8109120 0x7f45b819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.448+0000 7f45bea00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f45b8114140 con 0x7f45b8109120 2024-09-10T08:10:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.448+0000 7f45bea00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45b8113fc0 con 0x7f45b8103ff0 2024-09-10T08:10:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.448+0000 7f45bea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81142c0 con 0x7f45b81049a0 2024-09-10T08:10:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.449+0000 7f45b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45b8103ff0 0x7f45b819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.449+0000 7f45af400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b81049a0 0x7f45b819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.449+0000 7f45b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45b8103ff0 0x7f45b819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:56098/0 (socket says 172.21.15.2:56098) 2024-09-10T08:10:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.449+0000 7f45b7e00640 1 -- 172.21.15.2:0/2303028109 learned_addr learned my addr 172.21.15.2:0/2303028109 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.449+0000 7f45bca00640 1 --2- 172.21.15.2:0/2303028109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45b8109120 0x7f45b819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.449+0000 7f45b7e00640 1 -- 172.21.15.2:0/2303028109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b81049a0 msgr2=0x7f45b819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.449+0000 7f45b7e00640 1 --2- 172.21.15.2:0/2303028109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b81049a0 0x7f45b819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.449+0000 7f45b7e00640 1 -- 172.21.15.2:0/2303028109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45b8109120 msgr2=0x7f45b819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.449+0000 7f45b7e00640 1 --2- 172.21.15.2:0/2303028109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45b8109120 0x7f45b819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.449+0000 7f45b7e00640 1 -- 172.21.15.2:0/2303028109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b819f5d0 con 0x7f45b8103ff0 2024-09-10T08:10:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.449+0000 7f45b7e00640 1 --2- 172.21.15.2:0/2303028109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45b8103ff0 0x7f45b819a620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f45a800e9f0 tx=0x7f45a800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.450+0000 7f45b5600640 1 -- 172.21.15.2:0/2303028109 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a800cd80 con 0x7f45b8103ff0 2024-09-10T08:10:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.450+0000 7f45b5600640 1 -- 172.21.15.2:0/2303028109 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a8005500 con 0x7f45b8103ff0 2024-09-10T08:10:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.450+0000 7f45bea00640 1 -- 172.21.15.2:0/2303028109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b819f8c0 con 0x7f45b8103ff0 2024-09-10T08:10:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.450+0000 7f45b5600640 1 -- 172.21.15.2:0/2303028109 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8010690 con 0x7f45b8103ff0 2024-09-10T08:10:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.450+0000 7f45bea00640 1 -- 172.21.15.2:0/2303028109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b81a71a0 con 0x7f45b8103ff0 2024-09-10T08:10:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45c13cf740 1 Processor -- start 2024-09-10T08:10:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45c13cf740 1 -- start start 2024-09-10T08:10:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45c13cf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c1b90 0x5626948c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45c13cf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948c24b0 0x5626948cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45c13cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948ccd70 0x5626948cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45c13cf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5626948283e0 con 0x5626948c24b0 2024-09-10T08:10:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45c13cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5626947cfd80 con 0x5626948ccd70 2024-09-10T08:10:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45c13cf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5626948243a0 con 0x5626948c1b90 2024-09-10T08:10:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45b5600640 1 -- 172.21.15.2:0/2303028109 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f45a80046c0 con 0x7f45b8103ff0 2024-09-10T08:10:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45bca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c1b90 0x5626948c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45af400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948ccd70 0x5626948cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45af400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948ccd70 0x5626948cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:56106/0 (socket says 172.21.15.2:56106) 2024-09-10T08:10:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45af400640 1 -- 172.21.15.2:0/1551204291 learned_addr learned my addr 172.21.15.2:0/1551204291 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.453+0000 7f45bca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c1b90 0x5626948c1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:41550/0 (socket says 172.21.15.2:41550) 2024-09-10T08:10:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.454+0000 7f45b7e00640 1 --2- 172.21.15.2:0/1551204291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948c24b0 0x5626948cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.454+0000 7f45af400640 1 -- 172.21.15.2:0/1551204291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c1b90 msgr2=0x5626948c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.454+0000 7f45af400640 1 --2- 172.21.15.2:0/1551204291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c1b90 0x5626948c1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.454+0000 7f45af400640 1 -- 172.21.15.2:0/1551204291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948c24b0 msgr2=0x5626948cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.454+0000 7f45af400640 1 --2- 172.21.15.2:0/1551204291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948c24b0 0x5626948cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.454+0000 7f45af400640 1 -- 172.21.15.2:0/1551204291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626947eeae0 con 0x5626948ccd70 2024-09-10T08:10:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.454+0000 7f45af400640 1 --2- 172.21.15.2:0/1551204291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948ccd70 0x5626948cf160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009900 tx=0x7f45a0005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbfc49a2709e3e7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.454+0000 7f45ad200640 1 -- 172.21.15.2:0/1551204291 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0040070 con 0x5626948ccd70 2024-09-10T08:10:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.455+0000 7f45ad200640 1 -- 172.21.15.2:0/1551204291 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a0031870 con 0x5626948ccd70 2024-09-10T08:10:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.455+0000 7f45ad200640 1 -- 172.21.15.2:0/1551204291 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0035040 con 0x5626948ccd70 2024-09-10T08:10:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.455+0000 7f45b5600640 1 --2- 172.21.15.2:0/2303028109 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f459c0a2b80 0x7f459c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.455+0000 7f45b5600640 1 -- 172.21.15.2:0/2303028109 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f45a80cf810 con 0x7f45b8103ff0 2024-09-10T08:10:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.455+0000 7f45bca00640 1 --2- 172.21.15.2:0/2303028109 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f459c0a2b80 0x7f459c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.455+0000 7f45c13cf740 1 -- 172.21.15.2:0/1551204291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948ccd70 msgr2=0x5626948cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.455+0000 7f45c13cf740 1 --2- 172.21.15.2:0/1551204291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948ccd70 0x5626948cf160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009900 tx=0x7f45a0005be0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.455+0000 7f45c13cf740 1 -- 172.21.15.2:0/1551204291 shutdown_connections 2024-09-10T08:10:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.455+0000 7f45c13cf740 1 --2- 172.21.15.2:0/1551204291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948ccd70 0x5626948cf160 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.455+0000 7f45c13cf740 1 --2- 172.21.15.2:0/1551204291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948c24b0 0x5626948cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.455+0000 7f45c13cf740 1 --2- 172.21.15.2:0/1551204291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c1b90 0x5626948c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.455+0000 7f45c13cf740 1 -- 172.21.15.2:0/1551204291 >> 172.21.15.2:0/1551204291 conn(0x5626948be120 msgr2=0x5626948be500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.456+0000 7f45c13cf740 1 -- 172.21.15.2:0/1551204291 shutdown_connections 2024-09-10T08:10:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.456+0000 7f45c13cf740 1 -- 172.21.15.2:0/1551204291 wait complete. 2024-09-10T08:10:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.456+0000 7f45bca00640 1 --2- 172.21.15.2:0/2303028109 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f459c0a2b80 0x7f459c0a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f45b0005fd0 tx=0x7f45b0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.456+0000 7f45c13cf740 1 -- start start 2024-09-10T08:10:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.457+0000 7f45c13cf740 1 Processor -- start 2024-09-10T08:10:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.457+0000 7f45c13cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 0x5626948c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.457+0000 7f45c13cf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c24b0 0x5626948ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.457+0000 7f45c13cf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948ccd70 0x5626948ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.457+0000 7f45c13cf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5626948cf9e0 con 0x5626948ccd70 2024-09-10T08:10:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.457+0000 7f45c13cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5626947e30f0 con 0x5626948c1b90 2024-09-10T08:10:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.457+0000 7f45c13cf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5626948cfb60 con 0x5626948c24b0 2024-09-10T08:10:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.457+0000 7f45c13cf740 10 client.0 _ll_register_callbacks cb 0x5626947d49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:24.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.457+0000 7f45bca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948ccd70 0x5626948ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 0x5626948c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45af400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c24b0 0x5626948ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 0x5626948c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:56116/0 (socket says 172.21.15.2:56116) 2024-09-10T08:10:24.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.457+0000 7f45bca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948ccd70 0x5626948ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37292/0 (socket says 172.21.15.2:37292) 2024-09-10T08:10:24.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45b7e00640 1 -- 172.21.15.2:0/3248924950 learned_addr learned my addr 172.21.15.2:0/3248924950 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:24.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45b7e00640 1 -- 172.21.15.2:0/3248924950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c24b0 msgr2=0x5626948ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:24.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45b7e00640 1 --2- 172.21.15.2:0/3248924950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c24b0 0x5626948ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45b7e00640 1 -- 172.21.15.2:0/3248924950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948ccd70 msgr2=0x5626948ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:24.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45b7e00640 1 --2- 172.21.15.2:0/3248924950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948ccd70 0x5626948ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45b7e00640 1 -- 172.21.15.2:0/3248924950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626945993c0 con 0x5626948c1b90 2024-09-10T08:10:24.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45b7e00640 1 --2- 172.21.15.2:0/3248924950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 0x5626948c9c30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f45a800c6a0 tx=0x7f45a809b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ec9d8bd704262fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:24.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45a5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:24.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45a5600640 1 -- 172.21.15.2:0/3248924950 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a809bc80 con 0x5626948c1b90 2024-09-10T08:10:24.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45a5600640 1 -- 172.21.15.2:0/3248924950 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f45a80a4460 con 0x5626948c1b90 2024-09-10T08:10:24.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.458+0000 7f45c13cf740 1 -- 172.21.15.2:0/3248924950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a0031870 con 0x5626948c1b90 2024-09-10T08:10:24.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.459+0000 7f45a5600640 1 -- 172.21.15.2:0/3248924950 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45a80a0030 con 0x5626948c1b90 2024-09-10T08:10:24.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.459+0000 7f45c13cf740 10 client.25889 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:24.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.459+0000 7f45c13cf740 1 -- 172.21.15.2:0/3248924950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626948db660 con 0x5626948c1b90 2024-09-10T08:10:24.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.459+0000 7f45a5600640 1 -- 172.21.15.2:0/3248924950 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a0031870 con 0x5626948c1b90 2024-09-10T08:10:24.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.459+0000 7f45a5600640 1 -- 172.21.15.2:0/3248924950 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f45a809ab40 con 0x5626948c1b90 2024-09-10T08:10:24.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.459+0000 7f45c13cf740 10 client.25889 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:24.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.459+0000 7f45c13cf740 10 client.25889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:24.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.459+0000 7f45c13cf740 20 client.25889 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:24.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.459+0000 7f45c13cf740 1 --2- 172.21.15.2:0/3248924950 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626948dcb20 0x5626948def30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.459+0000 7f45c13cf740 4 client.25889 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:24.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.459+0000 7f45c13cf740 1 -- 172.21.15.2:0/3248924950 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562694835950 con 0x5626948dcb20 2024-09-10T08:10:24.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.460+0000 7f45af400640 1 --2- 172.21.15.2:0/3248924950 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626948dcb20 0x5626948def30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.460+0000 7f45af400640 1 --2- 172.21.15.2:0/3248924950 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626948dcb20 0x5626948def30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce19453281ba7db server_cookie=ec784a3768c39f3d in_seq=0 out_seq=0 2024-09-10T08:10:24.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.460+0000 7f45a5600640 10 client.25889 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:24.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.461+0000 7f45a5600640 1 -- 172.21.15.2:0/3248924950 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562694835950 con 0x5626948dcb20 2024-09-10T08:10:24.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.461+0000 7f45a5600640 10 client.25889 handle_command_reply: tid=0 2024-09-10T08:10:24.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.461+0000 7f45c13cf740 1 client.25889 shutdown 2024-09-10T08:10:24.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.461+0000 7f45c13cf740 1 -- 172.21.15.2:0/3248924950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 msgr2=0x5626948c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:24.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.461+0000 7f45c13cf740 1 --2- 172.21.15.2:0/3248924950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 0x5626948c9c30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f45a800c6a0 tx=0x7f45a809b7c0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.461+0000 7f45c13cf740 1 -- 172.21.15.2:0/3248924950 shutdown_connections 2024-09-10T08:10:24.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.461+0000 7f45c13cf740 1 --2- 172.21.15.2:0/3248924950 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626948dcb20 0x5626948def30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.461+0000 7f45c13cf740 1 --2- 172.21.15.2:0/3248924950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948ccd70 0x5626948ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.461+0000 7f45c13cf740 1 --2- 172.21.15.2:0/3248924950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c24b0 0x5626948ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.461+0000 7f45a5600640 0 client.25889 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:24.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.461+0000 7f45c13cf740 1 --2- 172.21.15.2:0/3248924950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 0x5626948c9c30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.461+0000 7f45c13cf740 1 -- 172.21.15.2:0/3248924950 >> 172.21.15.2:0/3248924950 conn(0x5626948be120 msgr2=0x5626948c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:24.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.462+0000 7f45c13cf740 1 -- 172.21.15.2:0/3248924950 shutdown_connections 2024-09-10T08:10:24.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.462+0000 7f45c13cf740 1 -- 172.21.15.2:0/3248924950 wait complete. 2024-09-10T08:10:24.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.462+0000 7f45c13cf740 20 client.25889 trim_cache size 0 max 16384 2024-09-10T08:10:24.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.480+0000 7f45c13cf740 1 Processor -- start 2024-09-10T08:10:24.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.480+0000 7f45c13cf740 1 -- start start 2024-09-10T08:10:24.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.480+0000 7f45c13cf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c1b90 0x5626948c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.480+0000 7f45c13cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c24b0 0x5626948e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.480+0000 7f45c13cf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948ccd70 0x5626948e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.480+0000 7f45c13cf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81b38a0 con 0x5626948ccd70 2024-09-10T08:10:24.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.480+0000 7f45c13cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81b1170 con 0x5626948c24b0 2024-09-10T08:10:24.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.480+0000 7f45c13cf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81aefc0 con 0x5626948c1b90 2024-09-10T08:10:24.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.480+0000 7f45bca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948ccd70 0x5626948e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.480+0000 7f45bca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948ccd70 0x5626948e4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37308/0 (socket says 172.21.15.2:37308) 2024-09-10T08:10:24.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.481+0000 7f45bca00640 1 -- 172.21.15.2:0/2835910385 learned_addr learned my addr 172.21.15.2:0/2835910385 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:24.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.481+0000 7f45af400640 1 --2- 172.21.15.2:0/2835910385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c24b0 0x5626948e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.481+0000 7f45b7e00640 1 --2- 172.21.15.2:0/2835910385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c1b90 0x5626948c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.481+0000 7f45bca00640 1 -- 172.21.15.2:0/2835910385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c1b90 msgr2=0x5626948c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:24.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.481+0000 7f45bca00640 1 --2- 172.21.15.2:0/2835910385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c1b90 0x5626948c45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.481+0000 7f45bca00640 1 -- 172.21.15.2:0/2835910385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c24b0 msgr2=0x5626948e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.481+0000 7f45bca00640 1 --2- 172.21.15.2:0/2835910385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c24b0 0x5626948e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.481+0000 7f45bca00640 1 -- 172.21.15.2:0/2835910385 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626948df5b0 con 0x5626948ccd70 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.481+0000 7f45af400640 1 --2- 172.21.15.2:0/2835910385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c24b0 0x5626948e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.481+0000 7f45bca00640 1 --2- 172.21.15.2:0/2835910385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948ccd70 0x5626948e4d40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f45b0003eb0 tx=0x7f45b001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34341efa42ebdcc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.482+0000 7f45a7e00640 1 -- 172.21.15.2:0/2835910385 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0004b20 con 0x5626948ccd70 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.482+0000 7f45a7e00640 1 -- 172.21.15.2:0/2835910385 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b0004cc0 con 0x5626948ccd70 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.482+0000 7f45a7e00640 1 -- 172.21.15.2:0/2835910385 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0005410 con 0x5626948ccd70 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.482+0000 7f45c13cf740 1 -- 172.21.15.2:0/2835910385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948ccd70 msgr2=0x5626948e4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.482+0000 7f45c13cf740 1 --2- 172.21.15.2:0/2835910385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948ccd70 0x5626948e4d40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f45b0003eb0 tx=0x7f45b001a000 comp rx=0 tx=0).stop 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.482+0000 7f45c13cf740 1 -- 172.21.15.2:0/2835910385 shutdown_connections 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.482+0000 7f45c13cf740 1 --2- 172.21.15.2:0/2835910385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948ccd70 0x5626948e4d40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.482+0000 7f45c13cf740 1 --2- 172.21.15.2:0/2835910385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c24b0 0x5626948e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.482+0000 7f45c13cf740 1 --2- 172.21.15.2:0/2835910385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948c1b90 0x5626948c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.482+0000 7f45c13cf740 1 -- 172.21.15.2:0/2835910385 >> 172.21.15.2:0/2835910385 conn(0x5626948be120 msgr2=0x5626948c8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:24.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.482+0000 7f45c13cf740 1 -- 172.21.15.2:0/2835910385 shutdown_connections 2024-09-10T08:10:24.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.483+0000 7f45c13cf740 1 -- 172.21.15.2:0/2835910385 wait complete. 2024-09-10T08:10:24.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.483+0000 7f45c13cf740 1 -- start start 2024-09-10T08:10:24.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.483+0000 7f45c13cf740 1 Processor -- start 2024-09-10T08:10:24.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45c13cf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 0x5626948c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45c13cf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948c24b0 0x5626948c7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45c13cf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948ccd70 0x5626948c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45c13cf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81af630 con 0x5626948c24b0 2024-09-10T08:10:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45c13cf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81b0aa0 con 0x5626948c1b90 2024-09-10T08:10:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45c13cf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81afd00 con 0x5626948ccd70 2024-09-10T08:10:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45c13cf740 10 client.0 _ll_register_callbacks cb 0x7f45a80056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45af400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948c24b0 0x5626948c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 0x5626948c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 0x5626948c6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:56128/0 (socket says 172.21.15.2:56128) 2024-09-10T08:10:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45b7e00640 1 -- 172.21.15.2:0/3952777423 learned_addr learned my addr 172.21.15.2:0/3952777423 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45bca00640 1 --2- 172.21.15.2:0/3952777423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948ccd70 0x5626948c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45b7e00640 1 -- 172.21.15.2:0/3952777423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948ccd70 msgr2=0x5626948c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45b7e00640 1 --2- 172.21.15.2:0/3952777423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948ccd70 0x5626948c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45b7e00640 1 -- 172.21.15.2:0/3952777423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948c24b0 msgr2=0x5626948c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45b7e00640 1 --2- 172.21.15.2:0/3952777423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948c24b0 0x5626948c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.484+0000 7f45b7e00640 1 -- 172.21.15.2:0/3952777423 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562694946930 con 0x5626948c1b90 2024-09-10T08:10:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.485+0000 7f45b7e00640 1 --2- 172.21.15.2:0/3952777423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 0x5626948c6d50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f45a80097d0 tx=0x7f45a809b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf7d279a0574a5d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.485+0000 7f45a5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.485+0000 7f45a5600640 1 -- 172.21.15.2:0/3952777423 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a80a8020 con 0x5626948c1b90 2024-09-10T08:10:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.485+0000 7f45a5600640 1 -- 172.21.15.2:0/3952777423 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f45a80a12b0 con 0x5626948c1b90 2024-09-10T08:10:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.485+0000 7f45c13cf740 1 -- 172.21.15.2:0/3952777423 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b0004cc0 con 0x5626948c1b90 2024-09-10T08:10:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.485+0000 7f45a5600640 1 -- 172.21.15.2:0/3952777423 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45a80d3070 con 0x5626948c1b90 2024-09-10T08:10:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.485+0000 7f45c13cf740 10 client.25895 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.485+0000 7f45c13cf740 1 -- 172.21.15.2:0/3952777423 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562694947be0 con 0x5626948c1b90 2024-09-10T08:10:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.485+0000 7f45a5600640 1 -- 172.21.15.2:0/3952777423 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b0004cc0 con 0x5626948c1b90 2024-09-10T08:10:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.486+0000 7f45a5600640 1 -- 172.21.15.2:0/3952777423 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f45a80a4c60 con 0x5626948c1b90 2024-09-10T08:10:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.486+0000 7f45c13cf740 10 client.25895 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.486+0000 7f45c13cf740 10 client.25895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.486+0000 7f45c13cf740 20 client.25895 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.486+0000 7f45c13cf740 1 --2- 172.21.15.2:0/3952777423 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626949481c0 0x56269494a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.486+0000 7f45c13cf740 4 client.25895 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:10:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.486+0000 7f45c13cf740 1 -- 172.21.15.2:0/3952777423 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56269482fe60 con 0x5626949481c0 2024-09-10T08:10:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.486+0000 7f45af400640 1 --2- 172.21.15.2:0/3952777423 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626949481c0 0x56269494a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45af400640 1 --2- 172.21.15.2:0/3952777423 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626949481c0 0x56269494a5b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27d4a941d6161d96 server_cookie=a82074abb51835b8 in_seq=0 out_seq=0 2024-09-10T08:10:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45a5600640 10 client.25895 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45a5600640 1 -- 172.21.15.2:0/3952777423 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56269482fe60 con 0x5626949481c0 2024-09-10T08:10:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45a5600640 10 client.25895 handle_command_reply: tid=0 2024-09-10T08:10:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45c13cf740 1 client.25895 shutdown 2024-09-10T08:10:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45c13cf740 1 -- 172.21.15.2:0/3952777423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 msgr2=0x5626948c6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45c13cf740 1 --2- 172.21.15.2:0/3952777423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 0x5626948c6d50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f45a80097d0 tx=0x7f45a809b8f0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45c13cf740 1 -- 172.21.15.2:0/3952777423 shutdown_connections 2024-09-10T08:10:24.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45c13cf740 1 --2- 172.21.15.2:0/3952777423 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5626949481c0 0x56269494a5b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45c13cf740 1 --2- 172.21.15.2:0/3952777423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626948ccd70 0x5626948c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45c13cf740 1 --2- 172.21.15.2:0/3952777423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626948c24b0 0x5626948c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45c13cf740 1 --2- 172.21.15.2:0/3952777423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626948c1b90 0x5626948c6d50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45c13cf740 1 -- 172.21.15.2:0/3952777423 >> 172.21.15.2:0/3952777423 conn(0x5626948be120 msgr2=0x5626948bd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:24.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45a5600640 0 client.25895 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:24.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45c13cf740 1 -- 172.21.15.2:0/3952777423 shutdown_connections 2024-09-10T08:10:24.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.487+0000 7f45c13cf740 1 -- 172.21.15.2:0/3952777423 wait complete. 2024-09-10T08:10:24.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.488+0000 7f45c13cf740 20 client.25895 trim_cache size 0 max 16384 2024-09-10T08:10:24.491 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:10:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:10:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:10:24.492 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:10:24.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.488+0000 7f45bea00640 1 -- 172.21.15.2:0/2303028109 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f459c0a2b80 msgr2=0x7f459c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:24.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.488+0000 7f45bea00640 1 --2- 172.21.15.2:0/2303028109 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f459c0a2b80 0x7f459c0a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f45b0005fd0 tx=0x7f45b0009500 comp rx=0 tx=0).stop 2024-09-10T08:10:24.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.488+0000 7f45bea00640 1 -- 172.21.15.2:0/2303028109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45b8103ff0 msgr2=0x7f45b819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:24.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.488+0000 7f45bea00640 1 --2- 172.21.15.2:0/2303028109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45b8103ff0 0x7f45b819a620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f45a800e9f0 tx=0x7f45a800edf0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.488+0000 7f45bea00640 1 -- 172.21.15.2:0/2303028109 shutdown_connections 2024-09-10T08:10:24.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.488+0000 7f45bea00640 1 --2- 172.21.15.2:0/2303028109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45b8109120 0x7f45b819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.488+0000 7f45bea00640 1 --2- 172.21.15.2:0/2303028109 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f459c0a2b80 0x7f459c0a5040 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.488+0000 7f45bea00640 1 --2- 172.21.15.2:0/2303028109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b81049a0 0x7f45b819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.488+0000 7f45bea00640 1 --2- 172.21.15.2:0/2303028109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45b8103ff0 0x7f45b819a620 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:24.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.488+0000 7f45bea00640 1 -- 172.21.15.2:0/2303028109 >> 172.21.15.2:0/2303028109 conn(0x7f45b80ffa90 msgr2=0x7f45b8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:24.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.488+0000 7f45bea00640 1 -- 172.21.15.2:0/2303028109 shutdown_connections 2024-09-10T08:10:24.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:24.489+0000 7f45bea00640 1 -- 172.21.15.2:0/2303028109 wait complete. 2024-09-10T08:10:25.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:24 smithi155 ceph-mon[35405]: pgmap v455: 129 pgs: 129 active+clean; 711 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T08:10:25.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:24 smithi107 ceph-mon[35048]: pgmap v455: 129 pgs: 129 active+clean; 711 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T08:10:25.181 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:24 smithi002 ceph-mon[30563]: pgmap v455: 129 pgs: 129 active+clean; 711 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T08:10:25.417 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:10:25.417 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:10:25.796 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.379+0000 7fb417600640 1 -- 172.21.15.2:0/4070252183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb410112a30 msgr2=0x7fb410114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.379+0000 7fb417600640 1 --2- 172.21.15.2:0/4070252183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb410112a30 0x7fb410114e20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb408009ca0 tx=0x7fb40802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:10:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.379+0000 7fb417600640 1 -- 172.21.15.2:0/4070252183 shutdown_connections 2024-09-10T08:10:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.379+0000 7fb417600640 1 --2- 172.21.15.2:0/4070252183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb410112a30 0x7fb410114e20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.379+0000 7fb417600640 1 --2- 172.21.15.2:0/4070252183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb410077040 0x7fb4101124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.379+0000 7fb417600640 1 --2- 172.21.15.2:0/4070252183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410076720 0x7fb410076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.379+0000 7fb417600640 1 -- 172.21.15.2:0/4070252183 >> 172.21.15.2:0/4070252183 conn(0x7fb4100ffa90 msgr2=0x7fb410101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.379+0000 7fb417600640 1 -- 172.21.15.2:0/4070252183 shutdown_connections 2024-09-10T08:10:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.379+0000 7fb417600640 1 -- 172.21.15.2:0/4070252183 wait complete. 2024-09-10T08:10:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.380+0000 7fb417600640 1 Processor -- start 2024-09-10T08:10:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.380+0000 7fb417600640 1 -- start start 2024-09-10T08:10:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.381+0000 7fb417600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb410076720 0x7fb4101a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.381+0000 7fb417600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb410077040 0x7fb4101a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.381+0000 7fb417600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410112a30 0x7fb41019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.381+0000 7fb417600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101185e0 con 0x7fb410076720 2024-09-10T08:10:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.381+0000 7fb417600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb410118460 con 0x7fb410112a30 2024-09-10T08:10:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.381+0000 7fb417600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb410118760 con 0x7fb410077040 2024-09-10T08:10:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.381+0000 7fb414c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb410076720 0x7fb4101a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.381+0000 7fb414c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb410076720 0x7fb4101a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37344/0 (socket says 172.21.15.2:37344) 2024-09-10T08:10:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.381+0000 7fb414c00640 1 -- 172.21.15.2:0/1898238924 learned_addr learned my addr 172.21.15.2:0/1898238924 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.381+0000 7fb407e00640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb410077040 0x7fb4101a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.381+0000 7fb415600640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410112a30 0x7fb41019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.382+0000 7fb414c00640 1 -- 172.21.15.2:0/1898238924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb410077040 msgr2=0x7fb4101a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.382+0000 7fb414c00640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb410077040 0x7fb4101a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.382+0000 7fb414c00640 1 -- 172.21.15.2:0/1898238924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410112a30 msgr2=0x7fb41019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.382+0000 7fb414c00640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410112a30 0x7fb41019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.382+0000 7fb414c00640 1 -- 172.21.15.2:0/1898238924 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41019f040 con 0x7fb410076720 2024-09-10T08:10:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.382+0000 7fb415600640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410112a30 0x7fb41019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:26.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.382+0000 7fb414c00640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb410076720 0x7fb4101a4750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb40000e9f0 tx=0x7fb40000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.382+0000 7fb40e000640 1 -- 172.21.15.2:0/1898238924 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40000cd80 con 0x7fb410076720 2024-09-10T08:10:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.382+0000 7fb40e000640 1 -- 172.21.15.2:0/1898238924 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb400010910 con 0x7fb410076720 2024-09-10T08:10:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.382+0000 7fb417600640 1 -- 172.21.15.2:0/1898238924 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41019f330 con 0x7fb410076720 2024-09-10T08:10:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.382+0000 7fb40e000640 1 -- 172.21.15.2:0/1898238924 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400018990 con 0x7fb410076720 2024-09-10T08:10:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.382+0000 7fb417600640 1 -- 172.21.15.2:0/1898238924 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4101ab620 con 0x7fb410076720 2024-09-10T08:10:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.384+0000 7fb417600640 1 -- 172.21.15.2:0/1898238924 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb410076b00 con 0x7fb410076720 2024-09-10T08:10:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.385+0000 7fb40e000640 1 -- 172.21.15.2:0/1898238924 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb400010ac0 con 0x7fb410076720 2024-09-10T08:10:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.388+0000 7fb40e000640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb3f40a2c00 0x7fb3f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.388+0000 7fb40e000640 1 -- 172.21.15.2:0/1898238924 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb4000cf7b0 con 0x7fb410076720 2024-09-10T08:10:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.388+0000 7fb407e00640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb3f40a2c00 0x7fb3f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.388+0000 7fb407e00640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb3f40a2c00 0x7fb3f40a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb4100796d0 tx=0x7fb3f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:26.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.392+0000 7fb40e000640 1 -- 172.21.15.2:0/1898238924 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb40009c2c0 con 0x7fb410076720 2024-09-10T08:10:26.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.621+0000 7fb417600640 1 -- 172.21.15.2:0/1898238924 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb41010dbf0 con 0x7fb410076720 2024-09-10T08:10:26.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.621+0000 7fb40e000640 1 -- 172.21.15.2:0/1898238924 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fb40009bbc0 con 0x7fb410076720 2024-09-10T08:10:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:10:26.622 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:10:26.622 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:10:26.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.624+0000 7fb417600640 1 -- 172.21.15.2:0/1898238924 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb3f40a2c00 msgr2=0x7fb3f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:26.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.624+0000 7fb417600640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb3f40a2c00 0x7fb3f40a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb4100796d0 tx=0x7fb3f8005f90 comp rx=0 tx=0).stop 2024-09-10T08:10:26.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.624+0000 7fb417600640 1 -- 172.21.15.2:0/1898238924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb410076720 msgr2=0x7fb4101a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:26.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.624+0000 7fb417600640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb410076720 0x7fb4101a4750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb40000e9f0 tx=0x7fb40000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:10:26.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.624+0000 7fb417600640 1 -- 172.21.15.2:0/1898238924 shutdown_connections 2024-09-10T08:10:26.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.624+0000 7fb417600640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410112a30 0x7fb41019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:26.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.624+0000 7fb417600640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb3f40a2c00 0x7fb3f40a50c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:26.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.624+0000 7fb417600640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb410077040 0x7fb4101a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:26.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.624+0000 7fb417600640 1 --2- 172.21.15.2:0/1898238924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb410076720 0x7fb4101a4750 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:26.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.624+0000 7fb417600640 1 -- 172.21.15.2:0/1898238924 >> 172.21.15.2:0/1898238924 conn(0x7fb4100ffa90 msgr2=0x7fb410113630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:26.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.624+0000 7fb417600640 1 -- 172.21.15.2:0/1898238924 shutdown_connections 2024-09-10T08:10:26.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:26.624+0000 7fb417600640 1 -- 172.21.15.2:0/1898238924 wait complete. 2024-09-10T08:10:26.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:26 smithi002 ceph-mon[30563]: pgmap v456: 129 pgs: 129 active+clean; 767 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 10 op/s 2024-09-10T08:10:26.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:26 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1898238924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:10:27.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:26 smithi155 ceph-mon[35405]: pgmap v456: 129 pgs: 129 active+clean; 767 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 10 op/s 2024-09-10T08:10:27.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:26 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1898238924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:10:27.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:26 smithi107 ceph-mon[35048]: pgmap v456: 129 pgs: 129 active+clean; 767 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 10 op/s 2024-09-10T08:10:27.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:26 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1898238924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:10:27.540 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:10:27.923 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:28.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.500+0000 7fee60c00640 1 -- 172.21.15.2:0/4038894556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee5c10e260 msgr2=0x7fee5c110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.500+0000 7fee60c00640 1 --2- 172.21.15.2:0/4038894556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee5c10e260 0x7fee5c110650 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fee54009ca0 tx=0x7fee5402a950 comp rx=0 tx=0).stop 2024-09-10T08:10:28.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.500+0000 7fee60c00640 1 -- 172.21.15.2:0/4038894556 shutdown_connections 2024-09-10T08:10:28.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.500+0000 7fee60c00640 1 --2- 172.21.15.2:0/4038894556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee5c10e260 0x7fee5c110650 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.500+0000 7fee60c00640 1 --2- 172.21.15.2:0/4038894556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee5c074ee0 0x7fee5c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.500+0000 7fee60c00640 1 --2- 172.21.15.2:0/4038894556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c0745c0 0x7fee5c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.500+0000 7fee60c00640 1 -- 172.21.15.2:0/4038894556 >> 172.21.15.2:0/4038894556 conn(0x7fee5c0fd740 msgr2=0x7fee5c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:28.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.500+0000 7fee60c00640 1 -- 172.21.15.2:0/4038894556 shutdown_connections 2024-09-10T08:10:28.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.500+0000 7fee60c00640 1 -- 172.21.15.2:0/4038894556 wait complete. 2024-09-10T08:10:28.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.502+0000 7fee60c00640 1 Processor -- start 2024-09-10T08:10:28.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.502+0000 7fee60c00640 1 -- start start 2024-09-10T08:10:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.503+0000 7fee60c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c0745c0 0x7fee5c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.503+0000 7fee60c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee5c074ee0 0x7fee5c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.503+0000 7fee60c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee5c10e260 0x7fee5c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.503+0000 7fee60c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c113f70 con 0x7fee5c074ee0 2024-09-10T08:10:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.503+0000 7fee60c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c113df0 con 0x7fee5c0745c0 2024-09-10T08:10:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.503+0000 7fee60c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c1140f0 con 0x7fee5c10e260 2024-09-10T08:10:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.503+0000 7fee5b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c0745c0 0x7fee5c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.503+0000 7fee5aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee5c074ee0 0x7fee5c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.503+0000 7fee5b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c0745c0 0x7fee5c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60022/0 (socket says 172.21.15.2:60022) 2024-09-10T08:10:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.503+0000 7fee5b400640 1 -- 172.21.15.2:0/592113609 learned_addr learned my addr 172.21.15.2:0/592113609 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.503+0000 7fee5be00640 1 --2- 172.21.15.2:0/592113609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee5c10e260 0x7fee5c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.503+0000 7fee5b400640 1 -- 172.21.15.2:0/592113609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee5c10e260 msgr2=0x7fee5c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.503+0000 7fee5b400640 1 --2- 172.21.15.2:0/592113609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee5c10e260 0x7fee5c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.504+0000 7fee5b400640 1 -- 172.21.15.2:0/592113609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee5c074ee0 msgr2=0x7fee5c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.504+0000 7fee5b400640 1 --2- 172.21.15.2:0/592113609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee5c074ee0 0x7fee5c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.504+0000 7fee5b400640 1 -- 172.21.15.2:0/592113609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c19f420 con 0x7fee5c0745c0 2024-09-10T08:10:28.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.504+0000 7fee5b400640 1 --2- 172.21.15.2:0/592113609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c0745c0 0x7fee5c19a470 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00e9f0 tx=0x7fee4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:28.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.504+0000 7fee53e00640 1 -- 172.21.15.2:0/592113609 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c00cd80 con 0x7fee5c0745c0 2024-09-10T08:10:28.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.504+0000 7fee60c00640 1 -- 172.21.15.2:0/592113609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5c19f710 con 0x7fee5c0745c0 2024-09-10T08:10:28.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.504+0000 7fee53e00640 1 -- 172.21.15.2:0/592113609 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee4c010910 con 0x7fee5c0745c0 2024-09-10T08:10:28.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.504+0000 7fee53e00640 1 -- 172.21.15.2:0/592113609 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0188f0 con 0x7fee5c0745c0 2024-09-10T08:10:28.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.504+0000 7fee60c00640 1 -- 172.21.15.2:0/592113609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5c1a6ff0 con 0x7fee5c0745c0 2024-09-10T08:10:28.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.507+0000 7fee64cb8740 1 Processor -- start 2024-09-10T08:10:28.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.507+0000 7fee53e00640 1 -- 172.21.15.2:0/592113609 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fee4c018bb0 con 0x7fee5c0745c0 2024-09-10T08:10:28.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.508+0000 7fee64cb8740 1 -- start start 2024-09-10T08:10:28.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.508+0000 7fee64cb8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b3491a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.508+0000 7fee64cb8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 0x556b34924ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.508+0000 7fee64cb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34925020 0x556b34927410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.508+0000 7fee64cb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556b348803e0 con 0x556b34925020 2024-09-10T08:10:28.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.508+0000 7fee64cb8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556b34827d80 con 0x556b3491a760 2024-09-10T08:10:28.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.508+0000 7fee64cb8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556b3487c3a0 con 0x556b34919e40 2024-09-10T08:10:28.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.508+0000 7fee5b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 0x556b34924ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.508+0000 7fee5be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b3491a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.508+0000 7fee5be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b3491a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:41998/0 (socket says 172.21.15.2:41998) 2024-09-10T08:10:28.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.508+0000 7fee5be00640 1 -- 172.21.15.2:0/3099113100 learned_addr learned my addr 172.21.15.2:0/3099113100 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:28.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.509+0000 7fee5aa00640 1 --2- 172.21.15.2:0/3099113100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34925020 0x556b34927410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.509+0000 7fee5be00640 1 -- 172.21.15.2:0/3099113100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 msgr2=0x556b34924ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.509+0000 7fee5be00640 1 --2- 172.21.15.2:0/3099113100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 0x556b34924ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.509+0000 7fee5be00640 1 -- 172.21.15.2:0/3099113100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34925020 msgr2=0x556b34927410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.509+0000 7fee5be00640 1 --2- 172.21.15.2:0/3099113100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34925020 0x556b34927410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.509+0000 7fee5be00640 1 -- 172.21.15.2:0/3099113100 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b34846ae0 con 0x556b34919e40 2024-09-10T08:10:28.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.509+0000 7fee5be00640 1 --2- 172.21.15.2:0/3099113100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b3491a220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fee54009ca0 tx=0x7fee540093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4866eb759865bd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:28.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.510+0000 7fee53e00640 1 --2- 172.21.15.2:0/592113609 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fee400a2c00 0x7fee400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.510+0000 7fee47e00640 1 -- 172.21.15.2:0/3099113100 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54048070 con 0x556b34919e40 2024-09-10T08:10:28.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.510+0000 7fee47e00640 1 -- 172.21.15.2:0/3099113100 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5402f940 con 0x556b34919e40 2024-09-10T08:10:28.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.510+0000 7fee47e00640 1 -- 172.21.15.2:0/3099113100 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54034040 con 0x556b34919e40 2024-09-10T08:10:28.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.510+0000 7fee53e00640 1 -- 172.21.15.2:0/592113609 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fee4c0cf670 con 0x7fee5c0745c0 2024-09-10T08:10:28.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.510+0000 7fee5b400640 1 --2- 172.21.15.2:0/592113609 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fee400a2c00 0x7fee400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.511+0000 7fee5b400640 1 --2- 172.21.15.2:0/592113609 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fee400a2c00 0x7fee400a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fee4c097f90 tx=0x7fee4c0d5110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:28.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.511+0000 7fee64cb8740 1 -- 172.21.15.2:0/3099113100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 msgr2=0x556b3491a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.511+0000 7fee64cb8740 1 --2- 172.21.15.2:0/3099113100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b3491a220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fee54009ca0 tx=0x7fee540093e0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.511+0000 7fee64cb8740 1 -- 172.21.15.2:0/3099113100 shutdown_connections 2024-09-10T08:10:28.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.511+0000 7fee64cb8740 1 --2- 172.21.15.2:0/3099113100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34925020 0x556b34927410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.511+0000 7fee64cb8740 1 --2- 172.21.15.2:0/3099113100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 0x556b34924ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.511+0000 7fee64cb8740 1 --2- 172.21.15.2:0/3099113100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b3491a220 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.511+0000 7fee64cb8740 1 -- 172.21.15.2:0/3099113100 >> 172.21.15.2:0/3099113100 conn(0x556b349163d0 msgr2=0x556b349167b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:28.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.511+0000 7fee64cb8740 1 -- 172.21.15.2:0/3099113100 shutdown_connections 2024-09-10T08:10:28.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.511+0000 7fee64cb8740 1 -- 172.21.15.2:0/3099113100 wait complete. 2024-09-10T08:10:28.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.512+0000 7fee64cb8740 1 -- start start 2024-09-10T08:10:28.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.512+0000 7fee64cb8740 1 Processor -- start 2024-09-10T08:10:28.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.513+0000 7fee64cb8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b34921ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.513+0000 7fee64cb8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 0x556b34922420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.513+0000 7fee64cb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34925020 0x556b34922960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.513+0000 7fee64cb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556b34927bf0 con 0x556b34925020 2024-09-10T08:10:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.513+0000 7fee64cb8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556b3483b0f0 con 0x556b3491a760 2024-09-10T08:10:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.513+0000 7fee64cb8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556b34927d70 con 0x556b34919e40 2024-09-10T08:10:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.513+0000 7fee64cb8740 10 client.0 _ll_register_callbacks cb 0x556b34822bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.513+0000 7fee5be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b34921ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.514+0000 7fee5be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b34921ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42000/0 (socket says 172.21.15.2:42000) 2024-09-10T08:10:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.514+0000 7fee5be00640 1 -- 172.21.15.2:0/333098285 learned_addr learned my addr 172.21.15.2:0/333098285 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.514+0000 7fee5be00640 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34925020 0x556b34922960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.514+0000 7fee5aa00640 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 0x556b34922420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.514+0000 7fee5aa00640 1 -- 172.21.15.2:0/333098285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 msgr2=0x556b34921ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.514+0000 7fee5aa00640 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b34921ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.514+0000 7fee5aa00640 1 -- 172.21.15.2:0/333098285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34925020 msgr2=0x556b34922960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.514+0000 7fee5aa00640 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34925020 0x556b34922960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.514+0000 7fee5aa00640 1 -- 172.21.15.2:0/333098285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b345f13c0 con 0x556b3491a760 2024-09-10T08:10:28.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.514+0000 7fee5be00640 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b34921ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:28.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.515+0000 7fee5aa00640 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 0x556b34922420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fee480126d0 tx=0x7fee48012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15a343e1ccee9d2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:28.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.515+0000 7fee44c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:28.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.515+0000 7fee44c00640 1 -- 172.21.15.2:0/333098285 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48004030 con 0x556b3491a760 2024-09-10T08:10:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.515+0000 7fee44c00640 1 -- 172.21.15.2:0/333098285 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fee480041d0 con 0x556b3491a760 2024-09-10T08:10:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.515+0000 7fee64cb8740 1 -- 172.21.15.2:0/333098285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee5402f940 con 0x556b3491a760 2024-09-10T08:10:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.515+0000 7fee44c00640 1 -- 172.21.15.2:0/333098285 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee480043d0 con 0x556b3491a760 2024-09-10T08:10:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.515+0000 7fee64cb8740 10 client.25913 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.515+0000 7fee64cb8740 1 -- 172.21.15.2:0/333098285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b34933d20 con 0x556b3491a760 2024-09-10T08:10:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.516+0000 7fee44c00640 1 -- 172.21.15.2:0/333098285 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5402f940 con 0x556b3491a760 2024-09-10T08:10:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.516+0000 7fee44c00640 1 -- 172.21.15.2:0/333098285 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fee48003ac0 con 0x556b3491a760 2024-09-10T08:10:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.516+0000 7fee64cb8740 10 client.25913 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.516+0000 7fee64cb8740 10 client.25913 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:10:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.516+0000 7fee64cb8740 20 client.25913 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.516+0000 7fee64cb8740 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x556b34935280 0x556b349376a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.516+0000 7fee64cb8740 4 client.25913 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:28.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.516+0000 7fee64cb8740 1 -- 172.21.15.2:0/333098285 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b3488d950 con 0x556b34935280 2024-09-10T08:10:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.517+0000 7fee5be00640 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x556b34935280 0x556b349376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.517+0000 7fee44c00640 10 client.25913 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.517+0000 7fee5be00640 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x556b34935280 0x556b349376a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ac7364c2e5f34bc server_cookie=197e3f3df8db5db9 in_seq=0 out_seq=0 2024-09-10T08:10:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.518+0000 7fee44c00640 1 -- 172.21.15.2:0/333098285 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b3488d950 con 0x556b34935280 2024-09-10T08:10:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.518+0000 7fee44c00640 10 client.25913 handle_command_reply: tid=0 2024-09-10T08:10:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.518+0000 7fee64cb8740 1 client.25913 shutdown 2024-09-10T08:10:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.518+0000 7fee64cb8740 1 -- 172.21.15.2:0/333098285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 msgr2=0x556b34922420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.518+0000 7fee64cb8740 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 0x556b34922420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fee480126d0 tx=0x7fee48012b90 comp rx=0 tx=0).stop 2024-09-10T08:10:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.518+0000 7fee64cb8740 1 -- 172.21.15.2:0/333098285 shutdown_connections 2024-09-10T08:10:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.518+0000 7fee64cb8740 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x556b34935280 0x556b349376a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.518+0000 7fee64cb8740 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34925020 0x556b34922960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.518+0000 7fee64cb8740 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 0x556b34922420 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.518+0000 7fee64cb8740 1 --2- 172.21.15.2:0/333098285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b34921ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.518+0000 7fee44c00640 0 client.25913 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.518+0000 7fee64cb8740 1 -- 172.21.15.2:0/333098285 >> 172.21.15.2:0/333098285 conn(0x556b349163d0 msgr2=0x556b349154b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.519+0000 7fee64cb8740 1 -- 172.21.15.2:0/333098285 shutdown_connections 2024-09-10T08:10:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.519+0000 7fee64cb8740 1 -- 172.21.15.2:0/333098285 wait complete. 2024-09-10T08:10:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.519+0000 7fee64cb8740 20 client.25913 trim_cache size 0 max 16384 2024-09-10T08:10:28.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.533+0000 7fee64cb8740 1 Processor -- start 2024-09-10T08:10:28.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.533+0000 7fee64cb8740 1 -- start start 2024-09-10T08:10:28.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.533+0000 7fee64cb8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b349217a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.533+0000 7fee64cb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b3491a760 0x556b3493cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.533+0000 7fee64cb8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b34925020 0x556b3493d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.533+0000 7fee64cb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c0125d0 con 0x556b3491a760 2024-09-10T08:10:28.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.533+0000 7fee64cb8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c00fea0 con 0x556b34925020 2024-09-10T08:10:28.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.533+0000 7fee64cb8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c00dcf0 con 0x556b34919e40 2024-09-10T08:10:28.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.534+0000 7fee5be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b349217a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.534+0000 7fee5be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b349217a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42014/0 (socket says 172.21.15.2:42014) 2024-09-10T08:10:28.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.534+0000 7fee5be00640 1 -- 172.21.15.2:0/906979459 learned_addr learned my addr 172.21.15.2:0/906979459 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:28.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.534+0000 7fee5aa00640 1 --2- 172.21.15.2:0/906979459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b3491a760 0x556b3493cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.534+0000 7fee5be00640 1 --2- 172.21.15.2:0/906979459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b34925020 0x556b3493d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.534+0000 7fee5be00640 1 -- 172.21.15.2:0/906979459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b34925020 msgr2=0x556b3493d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.534+0000 7fee5be00640 1 --2- 172.21.15.2:0/906979459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b34925020 0x556b3493d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.534+0000 7fee5be00640 1 -- 172.21.15.2:0/906979459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b3491a760 msgr2=0x556b3493cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.534+0000 7fee5be00640 1 --2- 172.21.15.2:0/906979459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b3491a760 0x556b3493cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.534+0000 7fee5be00640 1 -- 172.21.15.2:0/906979459 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b34919c70 con 0x556b34919e40 2024-09-10T08:10:28.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.534+0000 7fee5be00640 1 --2- 172.21.15.2:0/906979459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b349217a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fee54009ca0 tx=0x7fee5402a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=506311b46eb3d4bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:28.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.535+0000 7fee47400640 1 -- 172.21.15.2:0/906979459 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54048040 con 0x556b34919e40 2024-09-10T08:10:28.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.535+0000 7fee47400640 1 -- 172.21.15.2:0/906979459 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee54049040 con 0x556b34919e40 2024-09-10T08:10:28.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.535+0000 7fee47400640 1 -- 172.21.15.2:0/906979459 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54030440 con 0x556b34919e40 2024-09-10T08:10:28.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.535+0000 7fee64cb8740 1 -- 172.21.15.2:0/906979459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 msgr2=0x556b349217a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.535+0000 7fee64cb8740 1 --2- 172.21.15.2:0/906979459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b349217a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fee54009ca0 tx=0x7fee5402a950 comp rx=0 tx=0).stop 2024-09-10T08:10:28.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.536+0000 7fee64cb8740 1 -- 172.21.15.2:0/906979459 shutdown_connections 2024-09-10T08:10:28.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.536+0000 7fee64cb8740 1 --2- 172.21.15.2:0/906979459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b34925020 0x556b3493d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.536+0000 7fee64cb8740 1 --2- 172.21.15.2:0/906979459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b3491a760 0x556b3493cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.536+0000 7fee64cb8740 1 --2- 172.21.15.2:0/906979459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34919e40 0x556b349217a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.536+0000 7fee64cb8740 1 -- 172.21.15.2:0/906979459 >> 172.21.15.2:0/906979459 conn(0x556b349163d0 msgr2=0x556b349154b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:28.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.536+0000 7fee64cb8740 1 -- 172.21.15.2:0/906979459 shutdown_connections 2024-09-10T08:10:28.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.536+0000 7fee64cb8740 1 -- 172.21.15.2:0/906979459 wait complete. 2024-09-10T08:10:28.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.536+0000 7fee64cb8740 1 -- start start 2024-09-10T08:10:28.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.536+0000 7fee64cb8740 1 Processor -- start 2024-09-10T08:10:28.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.537+0000 7fee64cb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34919e40 0x556b3492d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.537+0000 7fee64cb8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 0x556b3491e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.537+0000 7fee64cb8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34925020 0x556b3491ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.537+0000 7fee64cb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c00e360 con 0x556b34919e40 2024-09-10T08:10:28.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.537+0000 7fee64cb8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c00f7d0 con 0x556b3491a760 2024-09-10T08:10:28.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.537+0000 7fee64cb8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c00ea30 con 0x556b34925020 2024-09-10T08:10:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.537+0000 7fee64cb8740 10 client.0 _ll_register_callbacks cb 0x7fee48009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.537+0000 7fee5be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34919e40 0x556b3492d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.537+0000 7fee5be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34919e40 0x556b3492d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49824/0 (socket says 172.21.15.2:49824) 2024-09-10T08:10:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.537+0000 7fee5be00640 1 -- 172.21.15.2:0/3310397658 learned_addr learned my addr 172.21.15.2:0/3310397658 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.537+0000 7fee5aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 0x556b3491e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.537+0000 7fee5be00640 1 --2- 172.21.15.2:0/3310397658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34925020 0x556b3491ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.538+0000 7fee5be00640 1 -- 172.21.15.2:0/3310397658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34925020 msgr2=0x556b3491ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.538+0000 7fee5be00640 1 --2- 172.21.15.2:0/3310397658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34925020 0x556b3491ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.538+0000 7fee5be00640 1 -- 172.21.15.2:0/3310397658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 msgr2=0x556b3491e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.538+0000 7fee5be00640 1 --2- 172.21.15.2:0/3310397658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 0x556b3491e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.538+0000 7fee5be00640 1 -- 172.21.15.2:0/3310397658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3491f490 con 0x556b34919e40 2024-09-10T08:10:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.538+0000 7fee5be00640 1 --2- 172.21.15.2:0/3310397658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34919e40 0x556b3492d4d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fee54006010 tx=0x7fee54004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5c39786414b8db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.538+0000 7fee44c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.538+0000 7fee44c00640 1 -- 172.21.15.2:0/3310397658 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54048050 con 0x556b34919e40 2024-09-10T08:10:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.538+0000 7fee64cb8740 1 -- 172.21.15.2:0/3310397658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee54049040 con 0x556b34919e40 2024-09-10T08:10:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.538+0000 7fee44c00640 1 -- 172.21.15.2:0/3310397658 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fee540046f0 con 0x556b34919e40 2024-09-10T08:10:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.539+0000 7fee44c00640 1 -- 172.21.15.2:0/3310397658 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee5402c020 con 0x556b34919e40 2024-09-10T08:10:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.539+0000 7fee64cb8740 10 client.15885 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.539+0000 7fee64cb8740 1 -- 172.21.15.2:0/3310397658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b3499fcc0 con 0x556b34919e40 2024-09-10T08:10:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.539+0000 7fee44c00640 1 -- 172.21.15.2:0/3310397658 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee54049040 con 0x556b34919e40 2024-09-10T08:10:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.539+0000 7fee44c00640 1 -- 172.21.15.2:0/3310397658 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fee54030710 con 0x556b34919e40 2024-09-10T08:10:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.539+0000 7fee64cb8740 10 client.15885 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.539+0000 7fee64cb8740 10 client.15885 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:10:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.539+0000 7fee64cb8740 20 client.15885 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.539+0000 7fee64cb8740 1 --2- 172.21.15.2:0/3310397658 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x556b3499ff60 0x556b349a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.539+0000 7fee64cb8740 4 client.15885 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:10:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.539+0000 7fee64cb8740 1 -- 172.21.15.2:0/3310397658 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b34887e60 con 0x556b3499ff60 2024-09-10T08:10:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.540+0000 7fee5aa00640 1 --2- 172.21.15.2:0/3310397658 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x556b3499ff60 0x556b349a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.540+0000 7fee5aa00640 1 --2- 172.21.15.2:0/3310397658 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x556b3499ff60 0x556b349a2350 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78e3a750706e5304 server_cookie=a7725e4f6d54aa6e in_seq=0 out_seq=0 2024-09-10T08:10:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.540+0000 7fee44c00640 10 client.15885 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.540+0000 7fee44c00640 1 -- 172.21.15.2:0/3310397658 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b34887e60 con 0x556b3499ff60 2024-09-10T08:10:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.540+0000 7fee44c00640 10 client.15885 handle_command_reply: tid=0 2024-09-10T08:10:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.540+0000 7fee64cb8740 1 client.15885 shutdown 2024-09-10T08:10:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.541+0000 7fee64cb8740 1 -- 172.21.15.2:0/3310397658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34919e40 msgr2=0x556b3492d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.541+0000 7fee64cb8740 1 --2- 172.21.15.2:0/3310397658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34919e40 0x556b3492d4d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fee54006010 tx=0x7fee54004610 comp rx=0 tx=0).stop 2024-09-10T08:10:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.541+0000 7fee64cb8740 1 -- 172.21.15.2:0/3310397658 shutdown_connections 2024-09-10T08:10:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.541+0000 7fee64cb8740 1 --2- 172.21.15.2:0/3310397658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556b34925020 0x556b3491ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.541+0000 7fee64cb8740 1 --2- 172.21.15.2:0/3310397658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b3491a760 0x556b3491e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.541+0000 7fee64cb8740 1 --2- 172.21.15.2:0/3310397658 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x556b3499ff60 0x556b349a2350 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.541+0000 7fee64cb8740 1 --2- 172.21.15.2:0/3310397658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b34919e40 0x556b3492d4d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.541+0000 7fee64cb8740 1 -- 172.21.15.2:0/3310397658 >> 172.21.15.2:0/3310397658 conn(0x556b349163d0 msgr2=0x556b349207b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:28.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.541+0000 7fee44c00640 0 client.15885 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:28.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.541+0000 7fee64cb8740 1 -- 172.21.15.2:0/3310397658 shutdown_connections 2024-09-10T08:10:28.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.541+0000 7fee64cb8740 1 -- 172.21.15.2:0/3310397658 wait complete. 2024-09-10T08:10:28.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.541+0000 7fee64cb8740 20 client.15885 trim_cache size 0 max 16384 2024-09-10T08:10:28.545 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:10:28.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.542+0000 7fee60c00640 1 -- 172.21.15.2:0/592113609 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fee400a2c00 msgr2=0x7fee400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.542+0000 7fee60c00640 1 --2- 172.21.15.2:0/592113609 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fee400a2c00 0x7fee400a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fee4c097f90 tx=0x7fee4c0d5110 comp rx=0 tx=0).stop 2024-09-10T08:10:28.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.542+0000 7fee60c00640 1 -- 172.21.15.2:0/592113609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c0745c0 msgr2=0x7fee5c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.542+0000 7fee60c00640 1 --2- 172.21.15.2:0/592113609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c0745c0 0x7fee5c19a470 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00e9f0 tx=0x7fee4c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.542+0000 7fee60c00640 1 -- 172.21.15.2:0/592113609 shutdown_connections 2024-09-10T08:10:28.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.542+0000 7fee60c00640 1 --2- 172.21.15.2:0/592113609 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fee400a2c00 0x7fee400a50c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.542+0000 7fee60c00640 1 --2- 172.21.15.2:0/592113609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee5c10e260 0x7fee5c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.542+0000 7fee60c00640 1 --2- 172.21.15.2:0/592113609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee5c074ee0 0x7fee5c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.542+0000 7fee60c00640 1 --2- 172.21.15.2:0/592113609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c0745c0 0x7fee5c19a470 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.542+0000 7fee60c00640 1 -- 172.21.15.2:0/592113609 >> 172.21.15.2:0/592113609 conn(0x7fee5c0fd740 msgr2=0x7fee5c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:28.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.542+0000 7fee60c00640 1 -- 172.21.15.2:0/592113609 shutdown_connections 2024-09-10T08:10:28.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:28.542+0000 7fee60c00640 1 -- 172.21.15.2:0/592113609 wait complete. 2024-09-10T08:10:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.609+0000 7f5d3f600640 1 -- 172.21.15.2:0/399592516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d38104f50 msgr2=0x7f5d381053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.609+0000 7f5d3f600640 1 --2- 172.21.15.2:0/399592516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d38104f50 0x7f5d381053d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009860 tx=0x7f5d2402a340 comp rx=0 tx=0).stop 2024-09-10T08:10:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.609+0000 7f5d3f600640 1 -- 172.21.15.2:0/399592516 shutdown_connections 2024-09-10T08:10:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.609+0000 7f5d3f600640 1 --2- 172.21.15.2:0/399592516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d38105910 0x7f5d3810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.609+0000 7f5d3f600640 1 --2- 172.21.15.2:0/399592516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d38104f50 0x7f5d381053d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.609+0000 7f5d3f600640 1 --2- 172.21.15.2:0/399592516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d38103d60 0x7f5d38104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.609+0000 7f5d3f600640 1 -- 172.21.15.2:0/399592516 >> 172.21.15.2:0/399592516 conn(0x7f5d380ff4e0 msgr2=0x7f5d38101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.609+0000 7f5d3f600640 1 -- 172.21.15.2:0/399592516 shutdown_connections 2024-09-10T08:10:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.609+0000 7f5d3f600640 1 -- 172.21.15.2:0/399592516 wait complete. 2024-09-10T08:10:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.610+0000 7f5d3f600640 1 Processor -- start 2024-09-10T08:10:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.610+0000 7f5d3f600640 1 -- start start 2024-09-10T08:10:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.610+0000 7f5d3f600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d38103d60 0x7f5d38199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.610+0000 7f5d3f600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d38104f50 0x7f5d3819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.610+0000 7f5d3f600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d38105910 0x7f5d381a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.610+0000 7f5d3f600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3810f9c0 con 0x7f5d38105910 2024-09-10T08:10:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.610+0000 7f5d3f600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3810f840 con 0x7f5d38103d60 2024-09-10T08:10:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.610+0000 7f5d3f600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3810fb40 con 0x7f5d38104f50 2024-09-10T08:10:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.611+0000 7f5d3cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d38103d60 0x7f5d38199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.611+0000 7f5d3cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d38103d60 0x7f5d38199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60080/0 (socket says 172.21.15.2:60080) 2024-09-10T08:10:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.611+0000 7f5d3cc00640 1 -- 172.21.15.2:0/1201336997 learned_addr learned my addr 172.21.15.2:0/1201336997 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.611+0000 7f5d3d600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d38105910 0x7f5d381a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.611+0000 7f5d37e00640 1 --2- 172.21.15.2:0/1201336997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d38104f50 0x7f5d3819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.611+0000 7f5d3cc00640 1 -- 172.21.15.2:0/1201336997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d38104f50 msgr2=0x7f5d3819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.611+0000 7f5d3cc00640 1 --2- 172.21.15.2:0/1201336997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d38104f50 0x7f5d3819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.611+0000 7f5d3cc00640 1 -- 172.21.15.2:0/1201336997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d38105910 msgr2=0x7f5d381a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.611+0000 7f5d3cc00640 1 --2- 172.21.15.2:0/1201336997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d38105910 0x7f5d381a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.611+0000 7f5d3cc00640 1 -- 172.21.15.2:0/1201336997 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d381a1bd0 con 0x7f5d38103d60 2024-09-10T08:10:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.611+0000 7f5d3cc00640 1 --2- 172.21.15.2:0/1201336997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d38103d60 0x7f5d38199fb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00d950 tx=0x7f5d2c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.612+0000 7f5d35600640 1 -- 172.21.15.2:0/1201336997 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c017020 con 0x7f5d38103d60 2024-09-10T08:10:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.612+0000 7f5d35600640 1 -- 172.21.15.2:0/1201336997 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d2c009c30 con 0x7f5d38103d60 2024-09-10T08:10:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.612+0000 7f5d35600640 1 -- 172.21.15.2:0/1201336997 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c00f720 con 0x7f5d38103d60 2024-09-10T08:10:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.612+0000 7f5d3f600640 1 -- 172.21.15.2:0/1201336997 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d381a1ec0 con 0x7f5d38103d60 2024-09-10T08:10:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.612+0000 7f5d3f600640 1 -- 172.21.15.2:0/1201336997 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d381a22d0 con 0x7f5d38103d60 2024-09-10T08:10:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d425d0740 1 Processor -- start 2024-09-10T08:10:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d425d0740 1 -- start start 2024-09-10T08:10:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d35600640 1 -- 172.21.15.2:0/1201336997 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5d2c01a020 con 0x7f5d38103d60 2024-09-10T08:10:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d425d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be8090 0x55e4a2be8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d425d0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be89d0 0x55e4a2bed490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d425d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2beda60 0x55e4a2beff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d425d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e4a2b4d100 con 0x55e4a2be8090 2024-09-10T08:10:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d425d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e4a2af51b0 con 0x55e4a2beda60 2024-09-10T08:10:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d425d0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e4a2b490c0 con 0x55e4a2be89d0 2024-09-10T08:10:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d3d600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be8090 0x55e4a2be8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d3d600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be8090 0x55e4a2be8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49842/0 (socket says 172.21.15.2:49842) 2024-09-10T08:10:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d3d600640 1 -- 172.21.15.2:0/1411359213 learned_addr learned my addr 172.21.15.2:0/1411359213 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d3cc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be89d0 0x55e4a2bed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.615+0000 7f5d37e00640 1 --2- 172.21.15.2:0/1411359213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2beda60 0x55e4a2beff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.616+0000 7f5d3d600640 1 -- 172.21.15.2:0/1411359213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be89d0 msgr2=0x55e4a2bed490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.616+0000 7f5d3d600640 1 --2- 172.21.15.2:0/1411359213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be89d0 0x55e4a2bed490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.616+0000 7f5d3d600640 1 -- 172.21.15.2:0/1411359213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2beda60 msgr2=0x55e4a2beff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.616+0000 7f5d3d600640 1 --2- 172.21.15.2:0/1411359213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2beda60 0x55e4a2beff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.616+0000 7f5d3d600640 1 -- 172.21.15.2:0/1411359213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a2860210 con 0x55e4a2be8090 2024-09-10T08:10:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.616+0000 7f5d37e00640 1 --2- 172.21.15.2:0/1411359213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2beda60 0x55e4a2beff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.616+0000 7f5d3d600640 1 --2- 172.21.15.2:0/1411359213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be8090 0x55e4a2be8490 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5d300099f0 tx=0x7f5d300095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318c50279f94180a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.617+0000 7f5d29c00640 1 -- 172.21.15.2:0/1411359213 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3001d070 con 0x55e4a2be8090 2024-09-10T08:10:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.617+0000 7f5d29c00640 1 -- 172.21.15.2:0/1411359213 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d30003e40 con 0x55e4a2be8090 2024-09-10T08:10:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.617+0000 7f5d29c00640 1 -- 172.21.15.2:0/1411359213 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3000db30 con 0x55e4a2be8090 2024-09-10T08:10:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.617+0000 7f5d35600640 1 --2- 172.21.15.2:0/1201336997 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5d200a2cb0 0x7f5d200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.617+0000 7f5d35600640 1 -- 172.21.15.2:0/1201336997 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5d2c0cf370 con 0x7f5d38103d60 2024-09-10T08:10:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.617+0000 7f5d3cc00640 1 --2- 172.21.15.2:0/1201336997 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5d200a2cb0 0x7f5d200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.618+0000 7f5d425d0740 1 -- 172.21.15.2:0/1411359213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be8090 msgr2=0x55e4a2be8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.618+0000 7f5d425d0740 1 --2- 172.21.15.2:0/1411359213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be8090 0x55e4a2be8490 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5d300099f0 tx=0x7f5d300095c0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.618+0000 7f5d425d0740 1 -- 172.21.15.2:0/1411359213 shutdown_connections 2024-09-10T08:10:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.618+0000 7f5d425d0740 1 --2- 172.21.15.2:0/1411359213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2beda60 0x55e4a2beff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.618+0000 7f5d425d0740 1 --2- 172.21.15.2:0/1411359213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be89d0 0x55e4a2bed490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.618+0000 7f5d425d0740 1 --2- 172.21.15.2:0/1411359213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be8090 0x55e4a2be8490 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.618+0000 7f5d425d0740 1 -- 172.21.15.2:0/1411359213 >> 172.21.15.2:0/1411359213 conn(0x55e4a2be3700 msgr2=0x55e4a2be3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.618+0000 7f5d425d0740 1 -- 172.21.15.2:0/1411359213 shutdown_connections 2024-09-10T08:10:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.618+0000 7f5d425d0740 1 -- 172.21.15.2:0/1411359213 wait complete. 2024-09-10T08:10:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.618+0000 7f5d3cc00640 1 --2- 172.21.15.2:0/1201336997 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5d200a2cb0 0x7f5d200a5170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c09b710 tx=0x7f5d2c0d2db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.618+0000 7f5d425d0740 1 -- start start 2024-09-10T08:10:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.618+0000 7f5d425d0740 1 Processor -- start 2024-09-10T08:10:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.619+0000 7f5d425d0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be8090 0x55e4a2bf2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.619+0000 7f5d425d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.619+0000 7f5d425d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 0x55e4a2bf9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.619+0000 7f5d425d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e4a2bf0a30 con 0x55e4a2beda60 2024-09-10T08:10:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.619+0000 7f5d425d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e4a2ad6090 con 0x55e4a2be89d0 2024-09-10T08:10:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.619+0000 7f5d425d0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e4a2bf0bb0 con 0x55e4a2be8090 2024-09-10T08:10:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.619+0000 7f5d425d0740 10 client.0 _ll_register_callbacks cb 0x55e4a2b2af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.619+0000 7f5d3d600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 0x55e4a2bf9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.619+0000 7f5d3d600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 0x55e4a2bf9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49850/0 (socket says 172.21.15.2:49850) 2024-09-10T08:10:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.619+0000 7f5d3d600640 1 -- 172.21.15.2:0/201362930 learned_addr learned my addr 172.21.15.2:0/201362930 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.619+0000 7f5d37e00640 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.619+0000 7f5d3d600640 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be8090 0x55e4a2bf2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d37e00640 1 -- 172.21.15.2:0/201362930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be8090 msgr2=0x55e4a2bf2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d37e00640 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be8090 0x55e4a2bf2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d37e00640 1 -- 172.21.15.2:0/201362930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 msgr2=0x55e4a2bf9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d37e00640 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 0x55e4a2bf9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d37e00640 1 -- 172.21.15.2:0/201362930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a292c620 con 0x55e4a2be89d0 2024-09-10T08:10:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d3d600640 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 0x55e4a2bf9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d37e00640 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf27c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5d24035040 tx=0x7f5d24030880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60e73e6cd7cb3c81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d1e000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d1e000640 1 -- 172.21.15.2:0/201362930 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24037040 con 0x55e4a2be89d0 2024-09-10T08:10:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d1e000640 1 -- 172.21.15.2:0/201362930 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5d24004240 con 0x55e4a2be89d0 2024-09-10T08:10:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d425d0740 1 -- 172.21.15.2:0/201362930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e4a2b5a830 con 0x55e4a2be89d0 2024-09-10T08:10:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d1e000640 1 -- 172.21.15.2:0/201362930 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5d2404a050 con 0x55e4a2be89d0 2024-09-10T08:10:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d425d0740 10 client.25931 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.620+0000 7f5d425d0740 1 -- 172.21.15.2:0/201362930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e4a2bfc5f0 con 0x55e4a2be89d0 2024-09-10T08:10:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.621+0000 7f5d1e000640 1 -- 172.21.15.2:0/201362930 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e4a2b5a830 con 0x55e4a2be89d0 2024-09-10T08:10:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.621+0000 7f5d1e000640 1 -- 172.21.15.2:0/201362930 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5d24045030 con 0x55e4a2be89d0 2024-09-10T08:10:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.621+0000 7f5d425d0740 10 client.25931 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.621+0000 7f5d425d0740 10 client.25931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.621+0000 7f5d425d0740 20 client.25931 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.621+0000 7f5d425d0740 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4a2bfdd10 0x55e4a2c00100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.621+0000 7f5d425d0740 4 client.25931 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.621+0000 7f5d425d0740 1 -- 172.21.15.2:0/201362930 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4a2b54b80 con 0x55e4a2bfdd10 2024-09-10T08:10:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.621+0000 7f5d3d600640 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4a2bfdd10 0x55e4a2c00100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.622+0000 7f5d3d600640 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4a2bfdd10 0x55e4a2c00100 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e949c7fe315a1a server_cookie=e10ec10b9b1068d0 in_seq=0 out_seq=0 2024-09-10T08:10:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.622+0000 7f5d1e000640 10 client.25931 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.622+0000 7f5d1e000640 1 -- 172.21.15.2:0/201362930 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4a2b54b80 con 0x55e4a2bfdd10 2024-09-10T08:10:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.622+0000 7f5d1e000640 10 client.25931 handle_command_reply: tid=0 2024-09-10T08:10:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.623+0000 7f5d425d0740 1 client.25931 shutdown 2024-09-10T08:10:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.623+0000 7f5d425d0740 1 -- 172.21.15.2:0/201362930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2be89d0 msgr2=0x55e4a2bf27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.623+0000 7f5d425d0740 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf27c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5d24035040 tx=0x7f5d24030880 comp rx=0 tx=0).stop 2024-09-10T08:10:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.623+0000 7f5d425d0740 1 -- 172.21.15.2:0/201362930 shutdown_connections 2024-09-10T08:10:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.623+0000 7f5d425d0740 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4a2bfdd10 0x55e4a2c00100 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.623+0000 7f5d425d0740 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 0x55e4a2bf9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.623+0000 7f5d425d0740 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf27c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.623+0000 7f5d425d0740 1 --2- 172.21.15.2:0/201362930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be8090 0x55e4a2bf2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.623+0000 7f5d1e000640 0 client.25931 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.623+0000 7f5d425d0740 1 -- 172.21.15.2:0/201362930 >> 172.21.15.2:0/201362930 conn(0x55e4a2be3700 msgr2=0x55e4a2be4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.623+0000 7f5d425d0740 1 -- 172.21.15.2:0/201362930 shutdown_connections 2024-09-10T08:10:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.623+0000 7f5d425d0740 1 -- 172.21.15.2:0/201362930 wait complete. 2024-09-10T08:10:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.623+0000 7f5d425d0740 20 client.25931 trim_cache size 0 max 16384 2024-09-10T08:10:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.638+0000 7f5d425d0740 1 Processor -- start 2024-09-10T08:10:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.638+0000 7f5d425d0740 1 -- start start 2024-09-10T08:10:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d425d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2be8090 0x55e4a2be3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d425d0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d425d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 0x55e4a2be4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d425d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5d381af410 con 0x55e4a2beda60 2024-09-10T08:10:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d425d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d381acce0 con 0x55e4a2be8090 2024-09-10T08:10:28.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d425d0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5d381aab30 con 0x55e4a2be89d0 2024-09-10T08:10:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d3d600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 0x55e4a2be4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d3d600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 0x55e4a2be4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49856/0 (socket says 172.21.15.2:49856) 2024-09-10T08:10:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d3d600640 1 -- 172.21.15.2:0/3866196477 learned_addr learned my addr 172.21.15.2:0/3866196477 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d37e00640 1 --2- 172.21.15.2:0/3866196477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d3d600640 1 --2- 172.21.15.2:0/3866196477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2be8090 0x55e4a2be3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d3d600640 1 -- 172.21.15.2:0/3866196477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be89d0 msgr2=0x55e4a2bf9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d3d600640 1 --2- 172.21.15.2:0/3866196477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d3d600640 1 -- 172.21.15.2:0/3866196477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2be8090 msgr2=0x55e4a2be3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d3d600640 1 --2- 172.21.15.2:0/3866196477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2be8090 0x55e4a2be3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.639+0000 7f5d3d600640 1 -- 172.21.15.2:0/3866196477 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a2be3520 con 0x55e4a2beda60 2024-09-10T08:10:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.640+0000 7f5d3d600640 1 --2- 172.21.15.2:0/3866196477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 0x55e4a2be4020 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55e4a2bfa090 tx=0x7f5d30010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dd9482530f86a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.640+0000 7f5d29200640 1 -- 172.21.15.2:0/3866196477 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3001d070 con 0x55e4a2beda60 2024-09-10T08:10:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.640+0000 7f5d29200640 1 -- 172.21.15.2:0/3866196477 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d3001e040 con 0x55e4a2beda60 2024-09-10T08:10:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.640+0000 7f5d29200640 1 -- 172.21.15.2:0/3866196477 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30004050 con 0x55e4a2beda60 2024-09-10T08:10:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.640+0000 7f5d425d0740 1 -- 172.21.15.2:0/3866196477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 msgr2=0x55e4a2be4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.640+0000 7f5d425d0740 1 --2- 172.21.15.2:0/3866196477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 0x55e4a2be4020 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55e4a2bfa090 tx=0x7f5d30010040 comp rx=0 tx=0).stop 2024-09-10T08:10:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.640+0000 7f5d425d0740 1 -- 172.21.15.2:0/3866196477 shutdown_connections 2024-09-10T08:10:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.640+0000 7f5d425d0740 1 --2- 172.21.15.2:0/3866196477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2beda60 0x55e4a2be4020 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.640+0000 7f5d425d0740 1 --2- 172.21.15.2:0/3866196477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.640+0000 7f5d425d0740 1 --2- 172.21.15.2:0/3866196477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2be8090 0x55e4a2be3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.640+0000 7f5d425d0740 1 -- 172.21.15.2:0/3866196477 >> 172.21.15.2:0/3866196477 conn(0x55e4a2be3700 msgr2=0x55e4a2bf2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.641+0000 7f5d425d0740 1 -- 172.21.15.2:0/3866196477 shutdown_connections 2024-09-10T08:10:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.641+0000 7f5d425d0740 1 -- 172.21.15.2:0/3866196477 wait complete. 2024-09-10T08:10:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.641+0000 7f5d425d0740 1 -- start start 2024-09-10T08:10:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.641+0000 7f5d425d0740 1 Processor -- start 2024-09-10T08:10:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.641+0000 7f5d425d0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be8090 0x55e4a2bf6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.641+0000 7f5d425d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.641+0000 7f5d425d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2beda60 0x55e4a2c669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.641+0000 7f5d425d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5d381ab1a0 con 0x55e4a2be89d0 2024-09-10T08:10:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.641+0000 7f5d425d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d381ac610 con 0x55e4a2beda60 2024-09-10T08:10:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.641+0000 7f5d425d0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5d381ab870 con 0x55e4a2be8090 2024-09-10T08:10:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.642+0000 7f5d425d0740 10 client.0 _ll_register_callbacks cb 0x7f5d30009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.642+0000 7f5d37e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.642+0000 7f5d37e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49858/0 (socket says 172.21.15.2:49858) 2024-09-10T08:10:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.642+0000 7f5d37e00640 1 -- 172.21.15.2:0/756864891 learned_addr learned my addr 172.21.15.2:0/756864891 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.642+0000 7f5d3d600640 1 --2- 172.21.15.2:0/756864891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be8090 0x55e4a2bf6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.642+0000 7f5d3d600640 1 --2- 172.21.15.2:0/756864891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2beda60 0x55e4a2c669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.642+0000 7f5d37e00640 1 -- 172.21.15.2:0/756864891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be8090 msgr2=0x55e4a2bf6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.642+0000 7f5d37e00640 1 --2- 172.21.15.2:0/756864891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be8090 0x55e4a2bf6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.642+0000 7f5d37e00640 1 -- 172.21.15.2:0/756864891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2beda60 msgr2=0x55e4a2c669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.642+0000 7f5d37e00640 1 --2- 172.21.15.2:0/756864891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2beda60 0x55e4a2c669d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.642+0000 7f5d37e00640 1 -- 172.21.15.2:0/756864891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a2c66fc0 con 0x55e4a2be89d0 2024-09-10T08:10:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.642+0000 7f5d37e00640 1 --2- 172.21.15.2:0/756864891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf6d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5d240049b0 tx=0x7f5d24005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02c30c169bc92de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.642+0000 7f5d1e000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.643+0000 7f5d1e000640 1 -- 172.21.15.2:0/756864891 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24037040 con 0x55e4a2be89d0 2024-09-10T08:10:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.643+0000 7f5d425d0740 1 -- 172.21.15.2:0/756864891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d3001e040 con 0x55e4a2be89d0 2024-09-10T08:10:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.643+0000 7f5d1e000640 1 -- 172.21.15.2:0/756864891 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5d2403d680 con 0x55e4a2be89d0 2024-09-10T08:10:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.643+0000 7f5d1e000640 1 -- 172.21.15.2:0/756864891 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5d2405e050 con 0x55e4a2be89d0 2024-09-10T08:10:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.643+0000 7f5d425d0740 10 client.15903 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.643+0000 7f5d425d0740 1 -- 172.21.15.2:0/756864891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e4a2c68370 con 0x55e4a2be89d0 2024-09-10T08:10:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.643+0000 7f5d1e000640 1 -- 172.21.15.2:0/756864891 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d3001e040 con 0x55e4a2be89d0 2024-09-10T08:10:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.644+0000 7f5d1e000640 1 -- 172.21.15.2:0/756864891 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5d24045040 con 0x55e4a2be89d0 2024-09-10T08:10:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.644+0000 7f5d425d0740 10 client.15903 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.644+0000 7f5d425d0740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.644+0000 7f5d425d0740 20 client.15903 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.644+0000 7f5d425d0740 1 --2- 172.21.15.2:0/756864891 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4a2c68da0 0x55e4a2c6b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.644+0000 7f5d425d0740 4 client.15903 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:10:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.644+0000 7f5d425d0740 1 -- 172.21.15.2:0/756864891 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4a2ad17c0 con 0x55e4a2c68da0 2024-09-10T08:10:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.644+0000 7f5d3d600640 1 --2- 172.21.15.2:0/756864891 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4a2c68da0 0x55e4a2c6b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.644+0000 7f5d3d600640 1 --2- 172.21.15.2:0/756864891 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4a2c68da0 0x55e4a2c6b1c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce1938807b3ff0f server_cookie=3c6e1736cdb49c2f in_seq=0 out_seq=0 2024-09-10T08:10:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.644+0000 7f5d1e000640 10 client.15903 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.655+0000 7f5d1e000640 1 -- 172.21.15.2:0/756864891 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e4a2ad17c0 con 0x55e4a2c68da0 2024-09-10T08:10:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.655+0000 7f5d1e000640 10 client.15903 handle_command_reply: tid=0 2024-09-10T08:10:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.655+0000 7f5d425d0740 1 client.15903 shutdown 2024-09-10T08:10:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.656+0000 7f5d425d0740 1 -- 172.21.15.2:0/756864891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be89d0 msgr2=0x55e4a2bf6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.656+0000 7f5d425d0740 1 --2- 172.21.15.2:0/756864891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf6d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5d240049b0 tx=0x7f5d24005b20 comp rx=0 tx=0).stop 2024-09-10T08:10:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.656+0000 7f5d425d0740 1 -- 172.21.15.2:0/756864891 shutdown_connections 2024-09-10T08:10:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.656+0000 7f5d425d0740 1 --2- 172.21.15.2:0/756864891 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4a2c68da0 0x55e4a2c6b1c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.656+0000 7f5d425d0740 1 --2- 172.21.15.2:0/756864891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4a2beda60 0x55e4a2c669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.656+0000 7f5d425d0740 1 --2- 172.21.15.2:0/756864891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4a2be89d0 0x55e4a2bf6d40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.656+0000 7f5d425d0740 1 --2- 172.21.15.2:0/756864891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4a2be8090 0x55e4a2bf6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.656+0000 7f5d1e000640 0 client.15903 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.656+0000 7f5d425d0740 1 -- 172.21.15.2:0/756864891 >> 172.21.15.2:0/756864891 conn(0x55e4a2be3700 msgr2=0x55e4a2be2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.656+0000 7f5d425d0740 1 -- 172.21.15.2:0/756864891 shutdown_connections 2024-09-10T08:10:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.656+0000 7f5d425d0740 1 -- 172.21.15.2:0/756864891 wait complete. 2024-09-10T08:10:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.656+0000 7f5d425d0740 20 client.15903 trim_cache size 0 max 16384 2024-09-10T08:10:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:10:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:10:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:10:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:10:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.657+0000 7f5d3f600640 1 -- 172.21.15.2:0/1201336997 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5d200a2cb0 msgr2=0x7f5d200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.657+0000 7f5d3f600640 1 --2- 172.21.15.2:0/1201336997 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5d200a2cb0 0x7f5d200a5170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c09b710 tx=0x7f5d2c0d2db0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.657+0000 7f5d3f600640 1 -- 172.21.15.2:0/1201336997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d38103d60 msgr2=0x7f5d38199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.657+0000 7f5d3f600640 1 --2- 172.21.15.2:0/1201336997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d38103d60 0x7f5d38199fb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00d950 tx=0x7f5d2c00de10 comp rx=0 tx=0).stop 2024-09-10T08:10:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.657+0000 7f5d3f600640 1 -- 172.21.15.2:0/1201336997 shutdown_connections 2024-09-10T08:10:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.657+0000 7f5d3f600640 1 --2- 172.21.15.2:0/1201336997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d38105910 0x7f5d381a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.657+0000 7f5d3f600640 1 --2- 172.21.15.2:0/1201336997 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5d200a2cb0 0x7f5d200a5170 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.657+0000 7f5d3f600640 1 --2- 172.21.15.2:0/1201336997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d38104f50 0x7f5d3819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.657+0000 7f5d3f600640 1 --2- 172.21.15.2:0/1201336997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d38103d60 0x7f5d38199fb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.657+0000 7f5d3f600640 1 -- 172.21.15.2:0/1201336997 >> 172.21.15.2:0/1201336997 conn(0x7f5d380ff4e0 msgr2=0x7f5d38108270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.657+0000 7f5d3f600640 1 -- 172.21.15.2:0/1201336997 shutdown_connections 2024-09-10T08:10:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:28.657+0000 7f5d3f600640 1 -- 172.21.15.2:0/1201336997 wait complete. 2024-09-10T08:10:29.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:28 smithi155 ceph-mon[35405]: pgmap v457: 129 pgs: 129 active+clean; 767 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-09-10T08:10:29.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:28 smithi107 ceph-mon[35048]: pgmap v457: 129 pgs: 129 active+clean; 767 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-09-10T08:10:29.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:28 smithi002 ceph-mon[30563]: pgmap v457: 129 pgs: 129 active+clean; 767 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-09-10T08:10:29.345 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:10:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:30.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.274+0000 7f7bfdc00640 1 -- 172.21.15.2:0/2461416908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf8104010 msgr2=0x7f7bf81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:30.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.274+0000 7f7bfdc00640 1 --2- 172.21.15.2:0/2461416908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf8104010 0x7f7bf81043f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009900 tx=0x7f7be802a400 comp rx=0 tx=0).stop 2024-09-10T08:10:30.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.274+0000 7f7bfdc00640 1 -- 172.21.15.2:0/2461416908 shutdown_connections 2024-09-10T08:10:30.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.274+0000 7f7bfdc00640 1 --2- 172.21.15.2:0/2461416908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf8109140 0x7f7bf8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.274+0000 7f7bfdc00640 1 --2- 172.21.15.2:0/2461416908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf81049c0 0x7f7bf8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.274+0000 7f7bfdc00640 1 --2- 172.21.15.2:0/2461416908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf8104010 0x7f7bf81043f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.274+0000 7f7bfdc00640 1 -- 172.21.15.2:0/2461416908 >> 172.21.15.2:0/2461416908 conn(0x7f7bf80ffa50 msgr2=0x7f7bf8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:30.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.274+0000 7f7bfdc00640 1 -- 172.21.15.2:0/2461416908 shutdown_connections 2024-09-10T08:10:30.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.274+0000 7f7bfdc00640 1 -- 172.21.15.2:0/2461416908 wait complete. 2024-09-10T08:10:30.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.275+0000 7f7bfdc00640 1 Processor -- start 2024-09-10T08:10:30.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bfdc00640 1 -- start start 2024-09-10T08:10:30.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bfdc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf8104010 0x7f7bf819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bfdc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf81049c0 0x7f7bf819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bfdc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf8109140 0x7f7bf819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bfdc00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81141c0 con 0x7f7bf8109140 2024-09-10T08:10:30.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bfdc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf8114040 con 0x7f7bf81049c0 2024-09-10T08:10:30.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bfdc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf8114340 con 0x7f7bf8104010 2024-09-10T08:10:30.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bf7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf8109140 0x7f7bf819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bf6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf8104010 0x7f7bf819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bf6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf81049c0 0x7f7bf819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bf6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf81049c0 0x7f7bf819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60144/0 (socket says 172.21.15.2:60144) 2024-09-10T08:10:30.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bf6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf8104010 0x7f7bf819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42088/0 (socket says 172.21.15.2:42088) 2024-09-10T08:10:30.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bf6a00640 1 -- 172.21.15.2:0/1151921409 learned_addr learned my addr 172.21.15.2:0/1151921409 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:30.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.276+0000 7f7bf7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf8109140 0x7f7bf819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49874/0 (socket says 172.21.15.2:49874) 2024-09-10T08:10:30.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.277+0000 7f7bf6000640 1 -- 172.21.15.2:0/1151921409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf8104010 msgr2=0x7f7bf819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:30.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.277+0000 7f7bf6000640 1 --2- 172.21.15.2:0/1151921409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf8104010 0x7f7bf819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.277+0000 7f7bf6000640 1 -- 172.21.15.2:0/1151921409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf8109140 msgr2=0x7f7bf819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:30.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.277+0000 7f7bf6000640 1 --2- 172.21.15.2:0/1151921409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf8109140 0x7f7bf819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.277+0000 7f7bf6000640 1 -- 172.21.15.2:0/1151921409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf819f5e0 con 0x7f7bf81049c0 2024-09-10T08:10:30.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.277+0000 7f7bf7400640 1 --2- 172.21.15.2:0/1151921409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf8109140 0x7f7bf819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:30.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.277+0000 7f7bf6000640 1 --2- 172.21.15.2:0/1151921409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf81049c0 0x7f7bf819ab70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7be400e9f0 tx=0x7f7be400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:30.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.277+0000 7f7bef400640 1 -- 172.21.15.2:0/1151921409 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be400ccc0 con 0x7f7bf81049c0 2024-09-10T08:10:30.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.277+0000 7f7bef400640 1 -- 172.21.15.2:0/1151921409 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be4010910 con 0x7f7bf81049c0 2024-09-10T08:10:30.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.277+0000 7f7bfdc00640 1 -- 172.21.15.2:0/1151921409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf819f8d0 con 0x7f7bf81049c0 2024-09-10T08:10:30.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.277+0000 7f7bef400640 1 -- 172.21.15.2:0/1151921409 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4018960 con 0x7f7bf81049c0 2024-09-10T08:10:30.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.277+0000 7f7bfdc00640 1 -- 172.21.15.2:0/1151921409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf81a71b0 con 0x7f7bf81049c0 2024-09-10T08:10:30.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.280+0000 7f7c005f1740 1 Processor -- start 2024-09-10T08:10:30.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.281+0000 7f7c005f1740 1 -- start start 2024-09-10T08:10:30.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.281+0000 7f7bef400640 1 -- 172.21.15.2:0/1151921409 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7be4010ac0 con 0x7f7bf81049c0 2024-09-10T08:10:30.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.281+0000 7f7c005f1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e802c70 0x555e5e803050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.281+0000 7f7c005f1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e803590 0x555e5e80d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.281+0000 7f7c005f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e810240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.281+0000 7f7c005f1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555e5e7693e0 con 0x555e5e802c70 2024-09-10T08:10:30.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.281+0000 7f7c005f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555e5e710d80 con 0x555e5e80de50 2024-09-10T08:10:30.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.281+0000 7f7c005f1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555e5e7653a0 con 0x555e5e803590 2024-09-10T08:10:30.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.281+0000 7f7bf7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e802c70 0x555e5e803050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.281+0000 7f7bf7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e802c70 0x555e5e803050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49880/0 (socket says 172.21.15.2:49880) 2024-09-10T08:10:30.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.281+0000 7f7bf7400640 1 -- 172.21.15.2:0/3205175881 learned_addr learned my addr 172.21.15.2:0/3205175881 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:30.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.281+0000 7f7bf6000640 1 --2- 172.21.15.2:0/3205175881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e810240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.281+0000 7f7bf6a00640 1 --2- 172.21.15.2:0/3205175881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e803590 0x555e5e80d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.282+0000 7f7bf7400640 1 -- 172.21.15.2:0/3205175881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e803590 msgr2=0x555e5e80d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:30.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.282+0000 7f7bf7400640 1 --2- 172.21.15.2:0/3205175881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e803590 0x555e5e80d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.282+0000 7f7bf7400640 1 -- 172.21.15.2:0/3205175881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 msgr2=0x555e5e810240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:30.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.282+0000 7f7bf7400640 1 --2- 172.21.15.2:0/3205175881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e810240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.282+0000 7f7bf7400640 1 -- 172.21.15.2:0/3205175881 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5e72fae0 con 0x555e5e802c70 2024-09-10T08:10:30.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.282+0000 7f7bf7400640 1 --2- 172.21.15.2:0/3205175881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e802c70 0x555e5e803050 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0010710 tx=0x7f7bf0010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ab6162ec629621 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.282+0000 7f7bef400640 1 --2- 172.21.15.2:0/1151921409 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7bdc0a2c00 0x7f7bdc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.282+0000 7f7be3e00640 1 -- 172.21.15.2:0/3205175881 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf000e8d0 con 0x555e5e802c70 2024-09-10T08:10:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.282+0000 7f7be3e00640 1 -- 172.21.15.2:0/3205175881 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf0012450 con 0x555e5e802c70 2024-09-10T08:10:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.283+0000 7f7bef400640 1 -- 172.21.15.2:0/1151921409 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7be40cf780 con 0x7f7bf81049c0 2024-09-10T08:10:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.283+0000 7f7be3e00640 1 -- 172.21.15.2:0/3205175881 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf001c650 con 0x555e5e802c70 2024-09-10T08:10:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.283+0000 7f7bf6a00640 1 --2- 172.21.15.2:0/1151921409 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7bdc0a2c00 0x7f7bdc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.283+0000 7f7c005f1740 1 -- 172.21.15.2:0/3205175881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e802c70 msgr2=0x555e5e803050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.283+0000 7f7c005f1740 1 --2- 172.21.15.2:0/3205175881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e802c70 0x555e5e803050 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0010710 tx=0x7f7bf0010bd0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.283+0000 7f7bf6a00640 1 --2- 172.21.15.2:0/1151921409 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7bdc0a2c00 0x7f7bdc0a50c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7be80096c0 tx=0x7f7be80046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.283+0000 7f7c005f1740 1 -- 172.21.15.2:0/3205175881 shutdown_connections 2024-09-10T08:10:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.283+0000 7f7c005f1740 1 --2- 172.21.15.2:0/3205175881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e810240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.283+0000 7f7c005f1740 1 --2- 172.21.15.2:0/3205175881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e803590 0x555e5e80d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.283+0000 7f7c005f1740 1 --2- 172.21.15.2:0/3205175881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e802c70 0x555e5e803050 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.284+0000 7f7c005f1740 1 -- 172.21.15.2:0/3205175881 >> 172.21.15.2:0/3205175881 conn(0x555e5e7ff200 msgr2=0x555e5e7ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:30.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.284+0000 7f7c005f1740 1 -- 172.21.15.2:0/3205175881 shutdown_connections 2024-09-10T08:10:30.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.284+0000 7f7c005f1740 1 -- 172.21.15.2:0/3205175881 wait complete. 2024-09-10T08:10:30.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.284+0000 7f7c005f1740 1 -- start start 2024-09-10T08:10:30.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.284+0000 7f7c005f1740 1 Processor -- start 2024-09-10T08:10:30.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.285+0000 7f7c005f1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 0x555e5e80ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.285+0000 7f7c005f1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e80b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.285+0000 7f7c005f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e80b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.285+0000 7f7c005f1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555e5e810a20 con 0x555e5e803590 2024-09-10T08:10:30.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.285+0000 7f7c005f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555e5e7240f0 con 0x555e5e80de50 2024-09-10T08:10:30.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.285+0000 7f7c005f1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555e5e810ba0 con 0x555e5e802c70 2024-09-10T08:10:30.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.285+0000 7f7c005f1740 10 client.0 _ll_register_callbacks cb 0x555e5e70bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:30.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.285+0000 7f7bf6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e80b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.285+0000 7f7bf7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e80b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.285+0000 7f7bf6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e80b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49892/0 (socket says 172.21.15.2:49892) 2024-09-10T08:10:30.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.285+0000 7f7bf6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 0x555e5e80ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.285+0000 7f7bf6000640 1 -- 172.21.15.2:0/4269065456 learned_addr learned my addr 172.21.15.2:0/4269065456 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:30.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.286+0000 7f7bf7400640 1 -- 172.21.15.2:0/4269065456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 msgr2=0x555e5e80ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:30.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.286+0000 7f7bf7400640 1 --2- 172.21.15.2:0/4269065456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 0x555e5e80ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.286+0000 7f7bf7400640 1 -- 172.21.15.2:0/4269065456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 msgr2=0x555e5e80b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:30.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.286+0000 7f7bf7400640 1 --2- 172.21.15.2:0/4269065456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e80b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.286+0000 7f7bf7400640 1 -- 172.21.15.2:0/4269065456 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e5e4da3c0 con 0x555e5e80de50 2024-09-10T08:10:30.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.286+0000 7f7bf6000640 1 --2- 172.21.15.2:0/4269065456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e80b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:30.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.286+0000 7f7bf7400640 1 --2- 172.21.15.2:0/4269065456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e80b7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0010710 tx=0x7f7bf0005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8acef899090c8127 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:30.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.286+0000 7f7be0c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:30.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.287+0000 7f7be0c00640 1 -- 172.21.15.2:0/4269065456 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf002b060 con 0x555e5e80de50 2024-09-10T08:10:30.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.287+0000 7f7be0c00640 1 -- 172.21.15.2:0/4269065456 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7bf000eae0 con 0x555e5e80de50 2024-09-10T08:10:30.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.287+0000 7f7c005f1740 1 -- 172.21.15.2:0/4269065456 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bf0012450 con 0x555e5e80de50 2024-09-10T08:10:30.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.287+0000 7f7be0c00640 1 -- 172.21.15.2:0/4269065456 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7bf0032050 con 0x555e5e80de50 2024-09-10T08:10:30.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.287+0000 7f7c005f1740 10 client.25949 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:30.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.287+0000 7f7c005f1740 1 -- 172.21.15.2:0/4269065456 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e5e81ca70 con 0x555e5e80de50 2024-09-10T08:10:30.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.288+0000 7f7be0c00640 1 -- 172.21.15.2:0/4269065456 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf0012450 con 0x555e5e80de50 2024-09-10T08:10:30.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.288+0000 7f7be0c00640 1 -- 172.21.15.2:0/4269065456 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7bf000f040 con 0x555e5e80de50 2024-09-10T08:10:30.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.288+0000 7f7c005f1740 10 client.25949 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:30.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.288+0000 7f7c005f1740 10 client.25949 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:10:30.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.288+0000 7f7c005f1740 20 client.25949 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:30.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.288+0000 7f7c005f1740 1 --2- 172.21.15.2:0/4269065456 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555e5e81ded0 0x555e5e8202c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.288+0000 7f7c005f1740 4 client.25949 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:30.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.288+0000 7f7c005f1740 1 -- 172.21.15.2:0/4269065456 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e5e776950 con 0x555e5e81ded0 2024-09-10T08:10:30.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.288+0000 7f7bf6a00640 1 --2- 172.21.15.2:0/4269065456 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555e5e81ded0 0x555e5e8202c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.289+0000 7f7bf6a00640 1 --2- 172.21.15.2:0/4269065456 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555e5e81ded0 0x555e5e8202c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a1ddc127440c9f1 server_cookie=490880f176722bd2 in_seq=0 out_seq=0 2024-09-10T08:10:30.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.289+0000 7f7be0c00640 10 client.25949 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:30.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.290+0000 7f7be0c00640 1 -- 172.21.15.2:0/4269065456 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e5e776950 con 0x555e5e81ded0 2024-09-10T08:10:30.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.290+0000 7f7be0c00640 10 client.25949 handle_command_reply: tid=0 2024-09-10T08:10:30.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.290+0000 7f7c005f1740 1 client.25949 shutdown 2024-09-10T08:10:30.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.290+0000 7f7c005f1740 1 -- 172.21.15.2:0/4269065456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 msgr2=0x555e5e80b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:30.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.290+0000 7f7c005f1740 1 --2- 172.21.15.2:0/4269065456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e80b7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0010710 tx=0x7f7bf0005c70 comp rx=0 tx=0).stop 2024-09-10T08:10:30.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.290+0000 7f7c005f1740 1 -- 172.21.15.2:0/4269065456 shutdown_connections 2024-09-10T08:10:30.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.290+0000 7f7c005f1740 1 --2- 172.21.15.2:0/4269065456 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555e5e81ded0 0x555e5e8202c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.290+0000 7f7c005f1740 1 --2- 172.21.15.2:0/4269065456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e80b7a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.290+0000 7f7c005f1740 1 --2- 172.21.15.2:0/4269065456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e80b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.290+0000 7f7c005f1740 1 --2- 172.21.15.2:0/4269065456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 0x555e5e80ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.290+0000 7f7be0c00640 0 client.25949 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:30.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.290+0000 7f7c005f1740 1 -- 172.21.15.2:0/4269065456 >> 172.21.15.2:0/4269065456 conn(0x555e5e7ff200 msgr2=0x555e5e7fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:30.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.290+0000 7f7c005f1740 1 -- 172.21.15.2:0/4269065456 shutdown_connections 2024-09-10T08:10:30.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.291+0000 7f7c005f1740 1 -- 172.21.15.2:0/4269065456 wait complete. 2024-09-10T08:10:30.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.291+0000 7f7c005f1740 20 client.25949 trim_cache size 0 max 16384 2024-09-10T08:10:30.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.304+0000 7f7c005f1740 1 Processor -- start 2024-09-10T08:10:30.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.304+0000 7f7c005f1740 1 -- start start 2024-09-10T08:10:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.305+0000 7f7c005f1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 0x555e5e80d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.305+0000 7f7c005f1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e8056d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.305+0000 7f7c005f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e825860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.305+0000 7f7c005f1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81b3870 con 0x555e5e803590 2024-09-10T08:10:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.305+0000 7f7c005f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81b1140 con 0x555e5e80de50 2024-09-10T08:10:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.305+0000 7f7c005f1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81aef90 con 0x555e5e802c70 2024-09-10T08:10:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.305+0000 7f7bf6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e8056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.305+0000 7f7bf7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e825860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.305+0000 7f7bf6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e8056d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49894/0 (socket says 172.21.15.2:49894) 2024-09-10T08:10:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.305+0000 7f7bf7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e825860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60172/0 (socket says 172.21.15.2:60172) 2024-09-10T08:10:30.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.305+0000 7f7bf6000640 1 -- 172.21.15.2:0/2000672236 learned_addr learned my addr 172.21.15.2:0/2000672236 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:30.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.305+0000 7f7bf6a00640 1 --2- 172.21.15.2:0/2000672236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 0x555e5e80d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.306+0000 7f7bf7400640 1 -- 172.21.15.2:0/2000672236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 msgr2=0x555e5e80d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:30.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.306+0000 7f7bf7400640 1 --2- 172.21.15.2:0/2000672236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 0x555e5e80d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.306+0000 7f7bf7400640 1 -- 172.21.15.2:0/2000672236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 msgr2=0x555e5e8056d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:30.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.306+0000 7f7bf7400640 1 --2- 172.21.15.2:0/2000672236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e8056d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.306+0000 7f7bf7400640 1 -- 172.21.15.2:0/2000672236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5e80d690 con 0x555e5e80de50 2024-09-10T08:10:30.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.306+0000 7f7bf7400640 1 --2- 172.21.15.2:0/2000672236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e825860 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0006010 tx=0x7f7bf0009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3820d24d08217 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:30.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.306+0000 7f7be3400640 1 -- 172.21.15.2:0/2000672236 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf002b070 con 0x555e5e80de50 2024-09-10T08:10:30.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.306+0000 7f7be3400640 1 -- 172.21.15.2:0/2000672236 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf0012c20 con 0x555e5e80de50 2024-09-10T08:10:30.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.306+0000 7f7be3400640 1 -- 172.21.15.2:0/2000672236 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0004c30 con 0x555e5e80de50 2024-09-10T08:10:30.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.307+0000 7f7c005f1740 1 -- 172.21.15.2:0/2000672236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 msgr2=0x555e5e825860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:30.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.307+0000 7f7c005f1740 1 --2- 172.21.15.2:0/2000672236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e825860 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0006010 tx=0x7f7bf0009600 comp rx=0 tx=0).stop 2024-09-10T08:10:30.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.307+0000 7f7c005f1740 1 -- 172.21.15.2:0/2000672236 shutdown_connections 2024-09-10T08:10:30.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.307+0000 7f7c005f1740 1 --2- 172.21.15.2:0/2000672236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e825860 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.307+0000 7f7c005f1740 1 --2- 172.21.15.2:0/2000672236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e8056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.307+0000 7f7c005f1740 1 --2- 172.21.15.2:0/2000672236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 0x555e5e80d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.307+0000 7f7c005f1740 1 -- 172.21.15.2:0/2000672236 >> 172.21.15.2:0/2000672236 conn(0x555e5e7ff200 msgr2=0x555e5e809ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:30.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.307+0000 7f7c005f1740 1 -- 172.21.15.2:0/2000672236 shutdown_connections 2024-09-10T08:10:30.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.307+0000 7f7c005f1740 1 -- 172.21.15.2:0/2000672236 wait complete. 2024-09-10T08:10:30.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.307+0000 7f7c005f1740 1 -- start start 2024-09-10T08:10:30.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.308+0000 7f7c005f1740 1 Processor -- start 2024-09-10T08:10:30.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.308+0000 7f7c005f1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 0x555e5e816350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.308+0000 7f7c005f1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e811680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.308+0000 7f7c005f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e811bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.308+0000 7f7c005f1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81af600 con 0x555e5e803590 2024-09-10T08:10:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.308+0000 7f7c005f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81b0a70 con 0x555e5e80de50 2024-09-10T08:10:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.308+0000 7f7c005f1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81afcd0 con 0x555e5e802c70 2024-09-10T08:10:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.308+0000 7f7c005f1740 10 client.0 _ll_register_callbacks cb 0x7f7bf0016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.308+0000 7f7bf6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e811680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.308+0000 7f7bf6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e811680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49910/0 (socket says 172.21.15.2:49910) 2024-09-10T08:10:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.308+0000 7f7bf6000640 1 -- 172.21.15.2:0/4151911254 learned_addr learned my addr 172.21.15.2:0/4151911254 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.308+0000 7f7bf7400640 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e811bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.308+0000 7f7bf6a00640 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 0x555e5e816350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.309+0000 7f7bf6000640 1 -- 172.21.15.2:0/4151911254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 msgr2=0x555e5e816350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.309+0000 7f7bf6000640 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 0x555e5e816350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.309+0000 7f7bf6000640 1 -- 172.21.15.2:0/4151911254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 msgr2=0x555e5e811bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:30.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.309+0000 7f7bf6000640 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e811bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.309+0000 7f7bf6000640 1 -- 172.21.15.2:0/4151911254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e5e8121b0 con 0x555e5e803590 2024-09-10T08:10:30.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.309+0000 7f7bf7400640 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e811bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:30.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.309+0000 7f7bf6000640 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e811680 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7be40980a0 tx=0x7f7be400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0e8965ac845eec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:30.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.309+0000 7f7be0c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:30.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.309+0000 7f7be0c00640 1 -- 172.21.15.2:0/4151911254 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be409b5a0 con 0x555e5e803590 2024-09-10T08:10:30.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.309+0000 7f7c005f1740 1 -- 172.21.15.2:0/4151911254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bf0012c20 con 0x555e5e803590 2024-09-10T08:10:30.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.309+0000 7f7be0c00640 1 -- 172.21.15.2:0/4151911254 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7be40cf400 con 0x555e5e803590 2024-09-10T08:10:30.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.310+0000 7f7be0c00640 1 -- 172.21.15.2:0/4151911254 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7be409bc40 con 0x555e5e803590 2024-09-10T08:10:30.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.310+0000 7f7c005f1740 10 client.15921 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:30.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.310+0000 7f7c005f1740 1 -- 172.21.15.2:0/4151911254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e5e890e80 con 0x555e5e803590 2024-09-10T08:10:30.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.310+0000 7f7be0c00640 1 -- 172.21.15.2:0/4151911254 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf0012c20 con 0x555e5e803590 2024-09-10T08:10:30.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.310+0000 7f7be0c00640 1 -- 172.21.15.2:0/4151911254 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7be40ce040 con 0x555e5e803590 2024-09-10T08:10:30.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.310+0000 7f7c005f1740 10 client.15921 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:30.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.310+0000 7f7c005f1740 10 client.15921 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:10:30.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.310+0000 7f7c005f1740 20 client.15921 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:30.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.310+0000 7f7c005f1740 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555e5e8915a0 0x555e5e8939c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:30.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.310+0000 7f7c005f1740 4 client.15921 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:10:30.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.310+0000 7f7c005f1740 1 -- 172.21.15.2:0/4151911254 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e5e770e60 con 0x555e5e8915a0 2024-09-10T08:10:30.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.311+0000 7f7bf6a00640 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555e5e8915a0 0x555e5e8939c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:30.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.311+0000 7f7bf6a00640 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555e5e8915a0 0x555e5e8939c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f3d203a0f2a0494 server_cookie=ae03ee98d03e02ea in_seq=0 out_seq=0 2024-09-10T08:10:30.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.311+0000 7f7be0c00640 10 client.15921 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:30.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.311+0000 7f7be0c00640 1 -- 172.21.15.2:0/4151911254 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e5e770e60 con 0x555e5e8915a0 2024-09-10T08:10:30.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.311+0000 7f7be0c00640 10 client.15921 handle_command_reply: tid=0 2024-09-10T08:10:30.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.311+0000 7f7c005f1740 1 client.15921 shutdown 2024-09-10T08:10:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.312+0000 7f7c005f1740 1 -- 172.21.15.2:0/4151911254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 msgr2=0x555e5e811680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.312+0000 7f7c005f1740 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e811680 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7be40980a0 tx=0x7f7be400cf60 comp rx=0 tx=0).stop 2024-09-10T08:10:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.312+0000 7f7c005f1740 1 -- 172.21.15.2:0/4151911254 shutdown_connections 2024-09-10T08:10:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.312+0000 7f7c005f1740 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555e5e8915a0 0x555e5e8939c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.312+0000 7f7c005f1740 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e5e80de50 0x555e5e811bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.312+0000 7f7c005f1740 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e5e803590 0x555e5e811680 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.312+0000 7f7c005f1740 1 --2- 172.21.15.2:0/4151911254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e5e802c70 0x555e5e816350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.312+0000 7f7c005f1740 1 -- 172.21.15.2:0/4151911254 >> 172.21.15.2:0/4151911254 conn(0x555e5e7ff200 msgr2=0x555e5e8094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.312+0000 7f7be0c00640 0 client.15921 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.312+0000 7f7c005f1740 1 -- 172.21.15.2:0/4151911254 shutdown_connections 2024-09-10T08:10:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.312+0000 7f7c005f1740 1 -- 172.21.15.2:0/4151911254 wait complete. 2024-09-10T08:10:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.312+0000 7f7c005f1740 20 client.15921 trim_cache size 0 max 16384 2024-09-10T08:10:30.316 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:10:30.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.313+0000 7f7bfdc00640 1 -- 172.21.15.2:0/1151921409 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7bdc0a2c00 msgr2=0x7f7bdc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:30.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.313+0000 7f7bfdc00640 1 --2- 172.21.15.2:0/1151921409 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7bdc0a2c00 0x7f7bdc0a50c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7be80096c0 tx=0x7f7be80046c0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.313+0000 7f7bfdc00640 1 -- 172.21.15.2:0/1151921409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf81049c0 msgr2=0x7f7bf819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:30.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.313+0000 7f7bfdc00640 1 --2- 172.21.15.2:0/1151921409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf81049c0 0x7f7bf819ab70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7be400e9f0 tx=0x7f7be400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.313+0000 7f7bfdc00640 1 -- 172.21.15.2:0/1151921409 shutdown_connections 2024-09-10T08:10:30.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.313+0000 7f7bfdc00640 1 --2- 172.21.15.2:0/1151921409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bf8109140 0x7f7bf819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.313+0000 7f7bfdc00640 1 --2- 172.21.15.2:0/1151921409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf81049c0 0x7f7bf819ab70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.313+0000 7f7bfdc00640 1 --2- 172.21.15.2:0/1151921409 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7bdc0a2c00 0x7f7bdc0a50c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.313+0000 7f7bfdc00640 1 --2- 172.21.15.2:0/1151921409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf8104010 0x7f7bf819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:30.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.313+0000 7f7bfdc00640 1 -- 172.21.15.2:0/1151921409 >> 172.21.15.2:0/1151921409 conn(0x7f7bf80ffa50 msgr2=0x7f7bf8100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:30.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.313+0000 7f7bfdc00640 1 -- 172.21.15.2:0/1151921409 shutdown_connections 2024-09-10T08:10:30.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:30.313+0000 7f7bfdc00640 1 -- 172.21.15.2:0/1151921409 wait complete. 2024-09-10T08:10:31.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:30 smithi002 ceph-mon[30563]: pgmap v458: 129 pgs: 129 active+clean; 767 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 KiB/s wr, 10 op/s 2024-09-10T08:10:31.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:30 smithi155 ceph-mon[35405]: pgmap v458: 129 pgs: 129 active+clean; 767 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 KiB/s wr, 10 op/s 2024-09-10T08:10:31.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:30 smithi107 ceph-mon[35048]: pgmap v458: 129 pgs: 129 active+clean; 767 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 KiB/s wr, 10 op/s 2024-09-10T08:10:32.819 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:10:33.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:32 smithi155 ceph-mon[35405]: pgmap v459: 129 pgs: 129 active+clean; 832 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-10T08:10:33.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:32 smithi107 ceph-mon[35048]: pgmap v459: 129 pgs: 129 active+clean; 832 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-10T08:10:33.206 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:33.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:32 smithi002 ceph-mon[30563]: pgmap v459: 129 pgs: 129 active+clean; 832 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-10T08:10:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.766+0000 7f5d0b200640 1 -- 172.21.15.2:0/3815101579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d0c1079c0 msgr2=0x7f5d0c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.766+0000 7f5d0b200640 1 --2- 172.21.15.2:0/3815101579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d0c1079c0 0x7f5d0c107da0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009930 tx=0x7f5cfc02a420 comp rx=0 tx=0).stop 2024-09-10T08:10:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.766+0000 7f5d0b200640 1 -- 172.21.15.2:0/3815101579 shutdown_connections 2024-09-10T08:10:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.766+0000 7f5d0b200640 1 --2- 172.21.15.2:0/3815101579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c102360 0x7f5d0c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.766+0000 7f5d0b200640 1 --2- 172.21.15.2:0/3815101579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d0c1019c0 0x7f5d0c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.766+0000 7f5d0b200640 1 --2- 172.21.15.2:0/3815101579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d0c1079c0 0x7f5d0c107da0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.766+0000 7f5d0b200640 1 -- 172.21.15.2:0/3815101579 >> 172.21.15.2:0/3815101579 conn(0x7f5d0c0fd740 msgr2=0x7f5d0c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.766+0000 7f5d0b200640 1 -- 172.21.15.2:0/3815101579 shutdown_connections 2024-09-10T08:10:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.766+0000 7f5d0b200640 1 -- 172.21.15.2:0/3815101579 wait complete. 2024-09-10T08:10:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.767+0000 7f5d0b200640 1 Processor -- start 2024-09-10T08:10:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.767+0000 7f5d0b200640 1 -- start start 2024-09-10T08:10:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d0b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c1019c0 0x7f5d0c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d0b200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d0c102360 0x7f5d0c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d0b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d0c1079c0 0x7f5d0c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d0b200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c111dc0 con 0x7f5d0c102360 2024-09-10T08:10:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d0b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c111c40 con 0x7f5d0c1079c0 2024-09-10T08:10:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d0b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c111f40 con 0x7f5d0c1019c0 2024-09-10T08:10:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d09400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d0c102360 0x7f5d0c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d09e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c1019c0 0x7f5d0c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d09e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c1019c0 0x7f5d0c1982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42164/0 (socket says 172.21.15.2:42164) 2024-09-10T08:10:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d09e00640 1 -- 172.21.15.2:0/3211382423 learned_addr learned my addr 172.21.15.2:0/3211382423 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d0a800640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d0c1079c0 0x7f5d0c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d09400640 1 -- 172.21.15.2:0/3211382423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c1019c0 msgr2=0x7f5d0c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d09400640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c1019c0 0x7f5d0c1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d09400640 1 -- 172.21.15.2:0/3211382423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d0c1079c0 msgr2=0x7f5d0c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d09400640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d0c1079c0 0x7f5d0c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d09400640 1 -- 172.21.15.2:0/3211382423 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c19d270 con 0x7f5d0c102360 2024-09-10T08:10:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d09e00640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c1019c0 0x7f5d0c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.768+0000 7f5d0a800640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d0c1079c0 0x7f5d0c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.769+0000 7f5d09400640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d0c102360 0x7f5d0c198800 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8005fd0 tx=0x7f5cf800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.769+0000 7f5d02a00640 1 -- 172.21.15.2:0/3211382423 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8009c30 con 0x7f5d0c102360 2024-09-10T08:10:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.769+0000 7f5d0b200640 1 -- 172.21.15.2:0/3211382423 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0c19d560 con 0x7f5d0c102360 2024-09-10T08:10:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.769+0000 7f5d02a00640 1 -- 172.21.15.2:0/3211382423 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf8009dd0 con 0x7f5d0c102360 2024-09-10T08:10:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.769+0000 7f5d02a00640 1 -- 172.21.15.2:0/3211382423 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8010460 con 0x7f5d0c102360 2024-09-10T08:10:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.769+0000 7f5d0b200640 1 -- 172.21.15.2:0/3211382423 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0c1a4e40 con 0x7f5d0c102360 2024-09-10T08:10:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.772+0000 7f5d02a00640 1 -- 172.21.15.2:0/3211382423 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5cf8003ef0 con 0x7f5d0c102360 2024-09-10T08:10:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.772+0000 7f5d13e54740 1 Processor -- start 2024-09-10T08:10:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.772+0000 7f5d13e54740 1 -- start start 2024-09-10T08:10:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.773+0000 7f5d13e54740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2abd40 0x5624bc2ac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.773+0000 7f5d13e54740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2ac660 0x5624bc2baef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.773+0000 7f5d13e54740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2bb430 0x5624bc2bd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.773+0000 7f5d13e54740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5624bc2123e0 con 0x5624bc2bb430 2024-09-10T08:10:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.773+0000 7f5d13e54740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5624bc1b9d80 con 0x5624bc2ac660 2024-09-10T08:10:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.773+0000 7f5d13e54740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5624bc20e3a0 con 0x5624bc2abd40 2024-09-10T08:10:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.773+0000 7f5d09400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2bb430 0x5624bc2bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.773+0000 7f5d0a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2abd40 0x5624bc2ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.773+0000 7f5d0a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2abd40 0x5624bc2ac120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42166/0 (socket says 172.21.15.2:42166) 2024-09-10T08:10:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.773+0000 7f5d09400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2bb430 0x5624bc2bd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49940/0 (socket says 172.21.15.2:49940) 2024-09-10T08:10:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.773+0000 7f5d0a800640 1 -- 172.21.15.2:0/3349231472 learned_addr learned my addr 172.21.15.2:0/3349231472 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.773+0000 7f5d09e00640 1 --2- 172.21.15.2:0/3349231472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2ac660 0x5624bc2baef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.774+0000 7f5d0a800640 1 -- 172.21.15.2:0/3349231472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2ac660 msgr2=0x5624bc2baef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.774+0000 7f5d0a800640 1 --2- 172.21.15.2:0/3349231472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2ac660 0x5624bc2baef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.774+0000 7f5d0a800640 1 -- 172.21.15.2:0/3349231472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2bb430 msgr2=0x5624bc2bd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.774+0000 7f5d0a800640 1 --2- 172.21.15.2:0/3349231472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2bb430 0x5624bc2bd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.774+0000 7f5d0a800640 1 -- 172.21.15.2:0/3349231472 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624bc1d8ae0 con 0x5624bc2abd40 2024-09-10T08:10:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.774+0000 7f5d0a800640 1 --2- 172.21.15.2:0/3349231472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2abd40 0x5624bc2ac120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d04010710 tx=0x7f5d04010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b77863c4a3dd6f55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.774+0000 7f5d02a00640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5cf00a2c00 0x7f5cf00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.774+0000 7f5cf7400640 1 -- 172.21.15.2:0/3349231472 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400e8d0 con 0x5624bc2abd40 2024-09-10T08:10:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.774+0000 7f5cf7400640 1 -- 172.21.15.2:0/3349231472 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d04012450 con 0x5624bc2abd40 2024-09-10T08:10:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.774+0000 7f5cf7400640 1 -- 172.21.15.2:0/3349231472 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0401c610 con 0x5624bc2abd40 2024-09-10T08:10:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.774+0000 7f5d02a00640 1 -- 172.21.15.2:0/3211382423 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5cf80cef20 con 0x7f5d0c102360 2024-09-10T08:10:33.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.775+0000 7f5d09e00640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5cf00a2c00 0x7f5cf00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.775+0000 7f5d13e54740 1 -- 172.21.15.2:0/3349231472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2abd40 msgr2=0x5624bc2ac120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:33.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.775+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3349231472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2abd40 0x5624bc2ac120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d04010710 tx=0x7f5d04010bd0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.775+0000 7f5d13e54740 1 -- 172.21.15.2:0/3349231472 shutdown_connections 2024-09-10T08:10:33.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.775+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3349231472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2bb430 0x5624bc2bd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.775+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3349231472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2ac660 0x5624bc2baef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.775+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3349231472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2abd40 0x5624bc2ac120 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.775+0000 7f5d13e54740 1 -- 172.21.15.2:0/3349231472 >> 172.21.15.2:0/3349231472 conn(0x5624bc2a83d0 msgr2=0x5624bc2a87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:33.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.775+0000 7f5d13e54740 1 -- 172.21.15.2:0/3349231472 shutdown_connections 2024-09-10T08:10:33.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.775+0000 7f5d13e54740 1 -- 172.21.15.2:0/3349231472 wait complete. 2024-09-10T08:10:33.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.775+0000 7f5d09e00640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5cf00a2c00 0x7f5cf00a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0314b0 tx=0x7f5cfc005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:33.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.776+0000 7f5d13e54740 1 -- start start 2024-09-10T08:10:33.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.776+0000 7f5d13e54740 1 Processor -- start 2024-09-10T08:10:33.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.776+0000 7f5d13e54740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2abd40 0x5624bc2b8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.776+0000 7f5d13e54740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2ac660 0x5624bc2b8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.776+0000 7f5d13e54740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2bb430 0x5624bc2b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.776+0000 7f5d13e54740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5624bc2be060 con 0x5624bc2bb430 2024-09-10T08:10:33.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.776+0000 7f5d13e54740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5624bc1cd0f0 con 0x5624bc2abd40 2024-09-10T08:10:33.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.776+0000 7f5d13e54740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5624bc2be1e0 con 0x5624bc2ac660 2024-09-10T08:10:33.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.777+0000 7f5d13e54740 10 client.0 _ll_register_callbacks cb 0x5624bc1b4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:33.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.777+0000 7f5d0a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2bb430 0x5624bc2b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.777+0000 7f5d0a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2bb430 0x5624bc2b8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49950/0 (socket says 172.21.15.2:49950) 2024-09-10T08:10:33.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.777+0000 7f5d0a800640 1 -- 172.21.15.2:0/466150318 learned_addr learned my addr 172.21.15.2:0/466150318 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.777+0000 7f5d09400640 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2ac660 0x5624bc2b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.777+0000 7f5d09e00640 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2abd40 0x5624bc2b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.777+0000 7f5d0a800640 1 -- 172.21.15.2:0/466150318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2ac660 msgr2=0x5624bc2b8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.777+0000 7f5d0a800640 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2ac660 0x5624bc2b8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.777+0000 7f5d0a800640 1 -- 172.21.15.2:0/466150318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2abd40 msgr2=0x5624bc2b8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.777+0000 7f5d0a800640 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2abd40 0x5624bc2b8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.777+0000 7f5d0a800640 1 -- 172.21.15.2:0/466150318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624bbf833c0 con 0x5624bc2bb430 2024-09-10T08:10:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.777+0000 7f5d09400640 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2ac660 0x5624bc2b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.778+0000 7f5d0a800640 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2bb430 0x5624bc2b8dd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400e6c0 tx=0x7f5d04005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e94813b5537719 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.778+0000 7f5cebe00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:33.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.778+0000 7f5cebe00640 1 -- 172.21.15.2:0/466150318 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04012a30 con 0x5624bc2bb430 2024-09-10T08:10:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.778+0000 7f5d13e54740 1 -- 172.21.15.2:0/466150318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d04012450 con 0x5624bc2bb430 2024-09-10T08:10:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.778+0000 7f5cebe00640 1 -- 172.21.15.2:0/466150318 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5d04004a40 con 0x5624bc2bb430 2024-09-10T08:10:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.778+0000 7f5cebe00640 1 -- 172.21.15.2:0/466150318 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5d04004c40 con 0x5624bc2bb430 2024-09-10T08:10:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.778+0000 7f5d13e54740 10 client.15933 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.778+0000 7f5d13e54740 1 -- 172.21.15.2:0/466150318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5624bc2ca110 con 0x5624bc2bb430 2024-09-10T08:10:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.778+0000 7f5cebe00640 1 -- 172.21.15.2:0/466150318 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d04012450 con 0x5624bc2bb430 2024-09-10T08:10:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.779+0000 7f5cebe00640 1 -- 172.21.15.2:0/466150318 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5d0400f040 con 0x5624bc2bb430 2024-09-10T08:10:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.779+0000 7f5d13e54740 10 client.15933 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.779+0000 7f5d13e54740 10 client.15933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.779+0000 7f5d13e54740 20 client.15933 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.779+0000 7f5d13e54740 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5624bc2cb510 0x5624bc2cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.779+0000 7f5d13e54740 4 client.15933 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.779+0000 7f5d13e54740 1 -- 172.21.15.2:0/466150318 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624bc21f950 con 0x5624bc2cb510 2024-09-10T08:10:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.779+0000 7f5d09e00640 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5624bc2cb510 0x5624bc2cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.780+0000 7f5d09e00640 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5624bc2cb510 0x5624bc2cd900 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b7acdb3a90171b server_cookie=379c96e034793abd in_seq=0 out_seq=0 2024-09-10T08:10:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.780+0000 7f5cebe00640 10 client.15933 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.780+0000 7f5cebe00640 1 -- 172.21.15.2:0/466150318 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624bc21f950 con 0x5624bc2cb510 2024-09-10T08:10:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5cebe00640 10 client.15933 handle_command_reply: tid=0 2024-09-10T08:10:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5d13e54740 1 client.15933 shutdown 2024-09-10T08:10:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5d13e54740 1 -- 172.21.15.2:0/466150318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2bb430 msgr2=0x5624bc2b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5d13e54740 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2bb430 0x5624bc2b8dd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400e6c0 tx=0x7f5d04005ab0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5d13e54740 1 -- 172.21.15.2:0/466150318 shutdown_connections 2024-09-10T08:10:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5d13e54740 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5624bc2cb510 0x5624bc2cd900 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5d13e54740 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2bb430 0x5624bc2b8dd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5d13e54740 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2ac660 0x5624bc2b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5d13e54740 1 --2- 172.21.15.2:0/466150318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2abd40 0x5624bc2b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5d13e54740 1 -- 172.21.15.2:0/466150318 >> 172.21.15.2:0/466150318 conn(0x5624bc2a83d0 msgr2=0x5624bc2ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5cebe00640 0 client.15933 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5d13e54740 1 -- 172.21.15.2:0/466150318 shutdown_connections 2024-09-10T08:10:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5d13e54740 1 -- 172.21.15.2:0/466150318 wait complete. 2024-09-10T08:10:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.781+0000 7f5d13e54740 20 client.15933 trim_cache size 0 max 16384 2024-09-10T08:10:33.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.795+0000 7f5d13e54740 1 Processor -- start 2024-09-10T08:10:33.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.796+0000 7f5d13e54740 1 -- start start 2024-09-10T08:10:33.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.796+0000 7f5d13e54740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2abd40 0x5624bc2b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.796+0000 7f5d13e54740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2ac660 0x5624bc2d3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.796+0000 7f5d13e54740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2bb430 0x5624bc2d3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.796+0000 7f5d13e54740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1af410 con 0x5624bc2abd40 2024-09-10T08:10:33.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.796+0000 7f5d13e54740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1acce0 con 0x5624bc2bb430 2024-09-10T08:10:33.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.796+0000 7f5d13e54740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1aab30 con 0x5624bc2ac660 2024-09-10T08:10:33.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.796+0000 7f5d09e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2abd40 0x5624bc2b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.796+0000 7f5d09e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2abd40 0x5624bc2b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49956/0 (socket says 172.21.15.2:49956) 2024-09-10T08:10:33.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.796+0000 7f5d09e00640 1 -- 172.21.15.2:0/3008276722 learned_addr learned my addr 172.21.15.2:0/3008276722 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:33.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.796+0000 7f5d09400640 1 --2- 172.21.15.2:0/3008276722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2ac660 0x5624bc2d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.796+0000 7f5d0a800640 1 --2- 172.21.15.2:0/3008276722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2bb430 0x5624bc2d3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.797+0000 7f5d09e00640 1 -- 172.21.15.2:0/3008276722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2ac660 msgr2=0x5624bc2d3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:33.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.797+0000 7f5d09e00640 1 --2- 172.21.15.2:0/3008276722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2ac660 0x5624bc2d3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.797+0000 7f5d09e00640 1 -- 172.21.15.2:0/3008276722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2bb430 msgr2=0x5624bc2d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:33.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.797+0000 7f5d09e00640 1 --2- 172.21.15.2:0/3008276722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2bb430 0x5624bc2d3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.797+0000 7f5d09e00640 1 -- 172.21.15.2:0/3008276722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624bc2bacc0 con 0x5624bc2abd40 2024-09-10T08:10:33.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.797+0000 7f5d09e00640 1 --2- 172.21.15.2:0/3008276722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2abd40 0x5624bc2b36b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009a50 tx=0x7f5cfc03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f791f2753f46068c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:33.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.797+0000 7f5cf6a00640 1 -- 172.21.15.2:0/3008276722 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc043070 con 0x5624bc2abd40 2024-09-10T08:10:33.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.797+0000 7f5cf6a00640 1 -- 172.21.15.2:0/3008276722 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cfc003fa0 con 0x5624bc2abd40 2024-09-10T08:10:33.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.797+0000 7f5cf6a00640 1 -- 172.21.15.2:0/3008276722 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc035040 con 0x5624bc2abd40 2024-09-10T08:10:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.798+0000 7f5d13e54740 1 -- 172.21.15.2:0/3008276722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2abd40 msgr2=0x5624bc2b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.798+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3008276722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2abd40 0x5624bc2b36b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009a50 tx=0x7f5cfc03f000 comp rx=0 tx=0).stop 2024-09-10T08:10:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.798+0000 7f5d13e54740 1 -- 172.21.15.2:0/3008276722 shutdown_connections 2024-09-10T08:10:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.798+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3008276722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2bb430 0x5624bc2d3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.798+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3008276722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2ac660 0x5624bc2d3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.798+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3008276722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2abd40 0x5624bc2b36b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.798+0000 7f5d13e54740 1 -- 172.21.15.2:0/3008276722 >> 172.21.15.2:0/3008276722 conn(0x5624bc2a83d0 msgr2=0x5624bc2b7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.798+0000 7f5d13e54740 1 -- 172.21.15.2:0/3008276722 shutdown_connections 2024-09-10T08:10:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.798+0000 7f5d13e54740 1 -- 172.21.15.2:0/3008276722 wait complete. 2024-09-10T08:10:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.799+0000 7f5d13e54740 1 -- start start 2024-09-10T08:10:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.799+0000 7f5d13e54740 1 Processor -- start 2024-09-10T08:10:33.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.799+0000 7f5d13e54740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2abd40 0x5624bc2be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.799+0000 7f5d13e54740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2ac660 0x5624bc2bef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.799+0000 7f5d13e54740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2bb430 0x5624bc333fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.799+0000 7f5d13e54740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1ab1a0 con 0x5624bc2ac660 2024-09-10T08:10:33.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.799+0000 7f5d13e54740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1ac610 con 0x5624bc2abd40 2024-09-10T08:10:33.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.799+0000 7f5d13e54740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1ab870 con 0x5624bc2bb430 2024-09-10T08:10:33.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.799+0000 7f5d13e54740 10 client.0 _ll_register_callbacks cb 0x7f5d04016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:33.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5d0a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2bb430 0x5624bc333fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5d09400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2ac660 0x5624bc2bef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5d0a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2bb430 0x5624bc333fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42184/0 (socket says 172.21.15.2:42184) 2024-09-10T08:10:33.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5d09e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2abd40 0x5624bc2be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5d0a800640 1 -- 172.21.15.2:0/3273087338 learned_addr learned my addr 172.21.15.2:0/3273087338 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5d09e00640 1 -- 172.21.15.2:0/3273087338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2bb430 msgr2=0x5624bc333fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5d09e00640 1 --2- 172.21.15.2:0/3273087338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2bb430 0x5624bc333fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5d09e00640 1 -- 172.21.15.2:0/3273087338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2ac660 msgr2=0x5624bc2bef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5d09e00640 1 --2- 172.21.15.2:0/3273087338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2ac660 0x5624bc2bef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5d09e00640 1 -- 172.21.15.2:0/3273087338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624bc334520 con 0x5624bc2abd40 2024-09-10T08:10:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5d0a800640 1 --2- 172.21.15.2:0/3273087338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2bb430 0x5624bc333fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5d09400640 1 --2- 172.21.15.2:0/3273087338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2ac660 0x5624bc2bef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5d09e00640 1 --2- 172.21.15.2:0/3273087338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2abd40 0x5624bc2be9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc000c00 tx=0x7f5cfc0092c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b995d1d54fd5dc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.800+0000 7f5cebe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.801+0000 7f5cebe00640 1 -- 172.21.15.2:0/3273087338 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc003cc0 con 0x5624bc2abd40 2024-09-10T08:10:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.801+0000 7f5d13e54740 1 -- 172.21.15.2:0/3273087338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cfc003fa0 con 0x5624bc2abd40 2024-09-10T08:10:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.801+0000 7f5cebe00640 1 -- 172.21.15.2:0/3273087338 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5cfc03d450 con 0x5624bc2abd40 2024-09-10T08:10:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.801+0000 7f5cebe00640 1 -- 172.21.15.2:0/3273087338 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5cfc06d050 con 0x5624bc2abd40 2024-09-10T08:10:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.801+0000 7f5d13e54740 10 client.25973 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.801+0000 7f5d13e54740 1 -- 172.21.15.2:0/3273087338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5624bc335840 con 0x5624bc2abd40 2024-09-10T08:10:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.802+0000 7f5cebe00640 1 -- 172.21.15.2:0/3273087338 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cfc003fa0 con 0x5624bc2abd40 2024-09-10T08:10:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.802+0000 7f5cebe00640 1 -- 172.21.15.2:0/3273087338 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5cfc049620 con 0x5624bc2abd40 2024-09-10T08:10:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.802+0000 7f5d13e54740 10 client.25973 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.802+0000 7f5d13e54740 10 client.25973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.802+0000 7f5d13e54740 20 client.25973 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.802+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3273087338 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5624bc335f70 0x5624bc338390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.802+0000 7f5d13e54740 4 client.25973 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:10:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.802+0000 7f5d13e54740 1 -- 172.21.15.2:0/3273087338 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624bc219e60 con 0x5624bc335f70 2024-09-10T08:10:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.802+0000 7f5d09400640 1 --2- 172.21.15.2:0/3273087338 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5624bc335f70 0x5624bc338390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5d09400640 1 --2- 172.21.15.2:0/3273087338 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5624bc335f70 0x5624bc338390 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=113476e1520014fc server_cookie=6e532543f01c4b11 in_seq=0 out_seq=0 2024-09-10T08:10:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5cebe00640 10 client.25973 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5cebe00640 1 -- 172.21.15.2:0/3273087338 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624bc219e60 con 0x5624bc335f70 2024-09-10T08:10:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5cebe00640 10 client.25973 handle_command_reply: tid=0 2024-09-10T08:10:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5d13e54740 1 client.25973 shutdown 2024-09-10T08:10:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5d13e54740 1 -- 172.21.15.2:0/3273087338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2abd40 msgr2=0x5624bc2be9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3273087338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2abd40 0x5624bc2be9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc000c00 tx=0x7f5cfc0092c0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5d13e54740 1 -- 172.21.15.2:0/3273087338 shutdown_connections 2024-09-10T08:10:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3273087338 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5624bc335f70 0x5624bc338390 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3273087338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624bc2bb430 0x5624bc333fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3273087338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624bc2ac660 0x5624bc2bef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5d13e54740 1 --2- 172.21.15.2:0/3273087338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624bc2abd40 0x5624bc2be9f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5cebe00640 0 client.25973 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.803+0000 7f5d13e54740 1 -- 172.21.15.2:0/3273087338 >> 172.21.15.2:0/3273087338 conn(0x5624bc2a83d0 msgr2=0x5624bc2b9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.804+0000 7f5d13e54740 1 -- 172.21.15.2:0/3273087338 shutdown_connections 2024-09-10T08:10:33.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.804+0000 7f5d13e54740 1 -- 172.21.15.2:0/3273087338 wait complete. 2024-09-10T08:10:33.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.804+0000 7f5d13e54740 20 client.25973 trim_cache size 0 max 16384 2024-09-10T08:10:33.810 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:10:33.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.804+0000 7f5d0b200640 1 -- 172.21.15.2:0/3211382423 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5cf00a2c00 msgr2=0x7f5cf00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:33.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.804+0000 7f5d0b200640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5cf00a2c00 0x7f5cf00a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0314b0 tx=0x7f5cfc005d90 comp rx=0 tx=0).stop 2024-09-10T08:10:33.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.804+0000 7f5d0b200640 1 -- 172.21.15.2:0/3211382423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d0c102360 msgr2=0x7f5d0c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:33.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.804+0000 7f5d0b200640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d0c102360 0x7f5d0c198800 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8005fd0 tx=0x7f5cf800dde0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.804+0000 7f5d0b200640 1 -- 172.21.15.2:0/3211382423 shutdown_connections 2024-09-10T08:10:33.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.804+0000 7f5d0b200640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d0c1079c0 0x7f5d0c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.804+0000 7f5d0b200640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d0c102360 0x7f5d0c198800 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.804+0000 7f5d0b200640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5cf00a2c00 0x7f5cf00a50c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.804+0000 7f5d0b200640 1 --2- 172.21.15.2:0/3211382423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c1019c0 0x7f5d0c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:33.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.804+0000 7f5d0b200640 1 -- 172.21.15.2:0/3211382423 >> 172.21.15.2:0/3211382423 conn(0x7f5d0c0fd740 msgr2=0x7f5d0c0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:33.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.805+0000 7f5d0b200640 1 -- 172.21.15.2:0/3211382423 shutdown_connections 2024-09-10T08:10:33.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:33.805+0000 7f5d0b200640 1 -- 172.21.15.2:0/3211382423 wait complete. 2024-09-10T08:10:35.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:34 smithi002 ceph-mon[30563]: pgmap v460: 129 pgs: 129 active+clean; 832 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:10:35.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:34 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:10:35.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:34 smithi155 ceph-mon[35405]: pgmap v460: 129 pgs: 129 active+clean; 832 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:10:35.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:34 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:10:35.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:34 smithi107 ceph-mon[35048]: pgmap v460: 129 pgs: 129 active+clean; 832 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:10:35.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:34 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:10:35.723 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:10:35.723 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:10:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:36.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:35 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:10:36.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:35 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:10:36.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:35 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:10:36.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:35 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:10:36.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:35 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:10:36.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:35 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:10:36.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:35 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:10:36.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:35 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:10:36.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:35 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:10:36.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.670+0000 7fb4b9200640 1 -- 172.21.15.2:0/233737749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b41049c0 msgr2=0x7fb4b4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:36.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.670+0000 7fb4b9200640 1 --2- 172.21.15.2:0/233737749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b41049c0 0x7fb4b4108a10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009900 tx=0x7fb49c02a410 comp rx=0 tx=0).stop 2024-09-10T08:10:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.670+0000 7fb4b9200640 1 -- 172.21.15.2:0/233737749 shutdown_connections 2024-09-10T08:10:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.670+0000 7fb4b9200640 1 --2- 172.21.15.2:0/233737749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4b4109140 0x7fb4b4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.670+0000 7fb4b9200640 1 --2- 172.21.15.2:0/233737749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b41049c0 0x7fb4b4108a10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.670+0000 7fb4b9200640 1 --2- 172.21.15.2:0/233737749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b4104010 0x7fb4b41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.670+0000 7fb4b9200640 1 -- 172.21.15.2:0/233737749 >> 172.21.15.2:0/233737749 conn(0x7fb4b40ffa50 msgr2=0x7fb4b4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.670+0000 7fb4b9200640 1 -- 172.21.15.2:0/233737749 shutdown_connections 2024-09-10T08:10:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.670+0000 7fb4b9200640 1 -- 172.21.15.2:0/233737749 wait complete. 2024-09-10T08:10:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.671+0000 7fb4b9200640 1 Processor -- start 2024-09-10T08:10:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.671+0000 7fb4b9200640 1 -- start start 2024-09-10T08:10:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.671+0000 7fb4b9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b4104010 0x7fb4b419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.671+0000 7fb4b9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4b41049c0 0x7fb4b419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.671+0000 7fb4b9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b4109140 0x7fb4b419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.671+0000 7fb4b9200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4114160 con 0x7fb4b4109140 2024-09-10T08:10:36.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.671+0000 7fb4b9200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4113fe0 con 0x7fb4b4104010 2024-09-10T08:10:36.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.671+0000 7fb4b9200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41142e0 con 0x7fb4b41049c0 2024-09-10T08:10:36.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.672+0000 7fb4b1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b4104010 0x7fb4b419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.672+0000 7fb4b2800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b4109140 0x7fb4b419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.672+0000 7fb4b1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b4104010 0x7fb4b419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60268/0 (socket says 172.21.15.2:60268) 2024-09-10T08:10:36.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.672+0000 7fb4b1e00640 1 -- 172.21.15.2:0/3358261342 learned_addr learned my addr 172.21.15.2:0/3358261342 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:36.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.672+0000 7fb4a9400640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4b41049c0 0x7fb4b419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.672+0000 7fb4b1e00640 1 -- 172.21.15.2:0/3358261342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4b41049c0 msgr2=0x7fb4b419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:36.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.672+0000 7fb4b1e00640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4b41049c0 0x7fb4b419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.672+0000 7fb4b1e00640 1 -- 172.21.15.2:0/3358261342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b4109140 msgr2=0x7fb4b419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:36.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.672+0000 7fb4b1e00640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b4109140 0x7fb4b419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.672+0000 7fb4b1e00640 1 -- 172.21.15.2:0/3358261342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b419f610 con 0x7fb4b4104010 2024-09-10T08:10:36.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.672+0000 7fb4b2800640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b4109140 0x7fb4b419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:36.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.672+0000 7fb4b1e00640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b4104010 0x7fb4b419a660 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400d950 tx=0x7fb4a400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:36.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.673+0000 7fb4ab400640 1 -- 172.21.15.2:0/3358261342 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4017030 con 0x7fb4b4104010 2024-09-10T08:10:36.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.673+0000 7fb4b9200640 1 -- 172.21.15.2:0/3358261342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b419f900 con 0x7fb4b4104010 2024-09-10T08:10:36.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.673+0000 7fb4ab400640 1 -- 172.21.15.2:0/3358261342 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a4004020 con 0x7fb4b4104010 2024-09-10T08:10:36.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.673+0000 7fb4ab400640 1 -- 172.21.15.2:0/3358261342 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a40047a0 con 0x7fb4b4104010 2024-09-10T08:10:36.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.673+0000 7fb4b9200640 1 -- 172.21.15.2:0/3358261342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b41a71e0 con 0x7fb4b4104010 2024-09-10T08:10:36.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.677+0000 7fb4bbe3e740 1 Processor -- start 2024-09-10T08:10:36.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.677+0000 7fb4bbe3e740 1 -- start start 2024-09-10T08:10:36.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.677+0000 7fb4bbe3e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 0x56156853dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.677+0000 7fb4bbe3e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853e3e0 0x561568548760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.677+0000 7fb4bbe3e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561568548ca0 0x56156854b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.677+0000 7fb4bbe3e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5615684a4290 con 0x56156853dac0 2024-09-10T08:10:36.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.677+0000 7fb4bbe3e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56156844bb40 con 0x561568548ca0 2024-09-10T08:10:36.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.677+0000 7fb4bbe3e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5615684a0250 con 0x56156853e3e0 2024-09-10T08:10:36.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.678+0000 7fb4b2800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 0x56156853dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.678+0000 7fb4b2800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 0x56156853dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49990/0 (socket says 172.21.15.2:49990) 2024-09-10T08:10:36.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.678+0000 7fb4b2800640 1 -- 172.21.15.2:0/1197114955 learned_addr learned my addr 172.21.15.2:0/1197114955 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:36.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.678+0000 7fb4ab400640 1 -- 172.21.15.2:0/3358261342 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb4a40041d0 con 0x7fb4b4104010 2024-09-10T08:10:36.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.678+0000 7fb4a9400640 1 --2- 172.21.15.2:0/1197114955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561568548ca0 0x56156854b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.678+0000 7fb4b2800640 1 -- 172.21.15.2:0/1197114955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853e3e0 msgr2=0x561568548760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:36.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.678+0000 7fb4b2800640 1 --2- 172.21.15.2:0/1197114955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853e3e0 0x561568548760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.678+0000 7fb4b2800640 1 -- 172.21.15.2:0/1197114955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561568548ca0 msgr2=0x56156854b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.678+0000 7fb4b2800640 1 --2- 172.21.15.2:0/1197114955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561568548ca0 0x56156854b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.678+0000 7fb4b2800640 1 -- 172.21.15.2:0/1197114955 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615682156a0 con 0x56156853dac0 2024-09-10T08:10:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.679+0000 7fb4b2800640 1 --2- 172.21.15.2:0/1197114955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 0x56156853dea0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac011790 tx=0x7fb4ac011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b22829400cde574 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.679+0000 7fb4a3400640 1 -- 172.21.15.2:0/1197114955 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac003c30 con 0x56156853dac0 2024-09-10T08:10:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.679+0000 7fb4a3400640 1 -- 172.21.15.2:0/1197114955 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4ac00fe80 con 0x56156853dac0 2024-09-10T08:10:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.679+0000 7fb4a3400640 1 -- 172.21.15.2:0/1197114955 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0193e0 con 0x56156853dac0 2024-09-10T08:10:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.679+0000 7fb4ab400640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb4980a2b80 0x7fb4980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.679+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/1197114955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 msgr2=0x56156853dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.679+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/1197114955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 0x56156853dea0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac011790 tx=0x7fb4ac011c50 comp rx=0 tx=0).stop 2024-09-10T08:10:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.680+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/1197114955 shutdown_connections 2024-09-10T08:10:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.680+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/1197114955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561568548ca0 0x56156854b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.680+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/1197114955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853e3e0 0x561568548760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.680+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/1197114955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 0x56156853dea0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.680+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/1197114955 >> 172.21.15.2:0/1197114955 conn(0x56156853a050 msgr2=0x56156853a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.680+0000 7fb4ab400640 1 -- 172.21.15.2:0/3358261342 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb4a40ced60 con 0x7fb4b4104010 2024-09-10T08:10:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.680+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/1197114955 shutdown_connections 2024-09-10T08:10:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.680+0000 7fb4b1e00640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb4980a2b80 0x7fb4980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.680+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/1197114955 wait complete. 2024-09-10T08:10:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.680+0000 7fb4bbe3e740 1 -- start start 2024-09-10T08:10:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4b1e00640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb4980a2b80 0x7fb4980a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4097630 tx=0x7fb4a4005cd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4bbe3e740 1 Processor -- start 2024-09-10T08:10:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4bbe3e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853dac0 0x5615685485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4bbe3e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x561568545880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4bbe3e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561568548ca0 0x561568545dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4bbe3e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56156854b910 con 0x561568548ca0 2024-09-10T08:10:36.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4bbe3e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56156845efa0 con 0x56156853e3e0 2024-09-10T08:10:36.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4bbe3e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56156854ba90 con 0x56156853dac0 2024-09-10T08:10:36.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4bbe3e740 10 client.0 _ll_register_callbacks cb 0x5615681b2ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:36.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4b2800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561568548ca0 0x561568545dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4b2800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561568548ca0 0x561568545dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50000/0 (socket says 172.21.15.2:50000) 2024-09-10T08:10:36.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4b2800640 1 -- 172.21.15.2:0/3696279961 learned_addr learned my addr 172.21.15.2:0/3696279961 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:36.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4a9400640 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x561568545880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.681+0000 7fb4b2800640 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853dac0 0x5615685485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.682+0000 7fb4a9400640 1 -- 172.21.15.2:0/3696279961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853dac0 msgr2=0x5615685485d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:36.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.682+0000 7fb4a9400640 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853dac0 0x5615685485d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.682+0000 7fb4a9400640 1 -- 172.21.15.2:0/3696279961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561568548ca0 msgr2=0x561568545dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:36.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.682+0000 7fb4a9400640 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561568548ca0 0x561568545dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.682+0000 7fb4a9400640 1 -- 172.21.15.2:0/3696279961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615685465d0 con 0x56156853e3e0 2024-09-10T08:10:36.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.682+0000 7fb4b2800640 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561568548ca0 0x561568545dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:36.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.682+0000 7fb4a9400640 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x561568545880 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb49c005940 tx=0x7fb49c032ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50fe70b39d39c810 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:36.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.682+0000 7fb497e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:36.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.683+0000 7fb497e00640 1 -- 172.21.15.2:0/3696279961 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c035040 con 0x56156853e3e0 2024-09-10T08:10:36.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.683+0000 7fb497e00640 1 -- 172.21.15.2:0/3696279961 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb49c0318b0 con 0x56156853e3e0 2024-09-10T08:10:36.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.683+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/3696279961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5615684b1800 con 0x56156853e3e0 2024-09-10T08:10:36.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.683+0000 7fb497e00640 1 -- 172.21.15.2:0/3696279961 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb49c04f050 con 0x56156853e3e0 2024-09-10T08:10:36.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.683+0000 7fb4bbe3e740 10 client.25985 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:36.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.683+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/3696279961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561568557800 con 0x56156853e3e0 2024-09-10T08:10:36.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.683+0000 7fb497e00640 1 -- 172.21.15.2:0/3696279961 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5615684b1800 con 0x56156853e3e0 2024-09-10T08:10:36.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.683+0000 7fb497e00640 1 -- 172.21.15.2:0/3696279961 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb49c004750 con 0x56156853e3e0 2024-09-10T08:10:36.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.684+0000 7fb4bbe3e740 10 client.25985 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:36.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.684+0000 7fb4bbe3e740 10 client.25985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:36.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.684+0000 7fb4bbe3e740 20 client.25985 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:36.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.684+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561568558cc0 0x56156855b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.684+0000 7fb4bbe3e740 4 client.25985 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:36.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.684+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/3696279961 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615684abd10 con 0x561568558cc0 2024-09-10T08:10:36.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.684+0000 7fb4b2800640 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561568558cc0 0x56156855b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.684+0000 7fb4b2800640 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561568558cc0 0x56156855b0b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34326aaf632b5e12 server_cookie=4a48228f628abec6 in_seq=0 out_seq=0 2024-09-10T08:10:36.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.684+0000 7fb497e00640 10 client.25985 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:36.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.685+0000 7fb497e00640 1 -- 172.21.15.2:0/3696279961 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615684abd10 con 0x561568558cc0 2024-09-10T08:10:36.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.685+0000 7fb497e00640 10 client.25985 handle_command_reply: tid=0 2024-09-10T08:10:36.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.685+0000 7fb4bbe3e740 1 client.25985 shutdown 2024-09-10T08:10:36.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.685+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/3696279961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 msgr2=0x561568545880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:36.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.685+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x561568545880 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb49c005940 tx=0x7fb49c032ed0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.686+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/3696279961 shutdown_connections 2024-09-10T08:10:36.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.686+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561568558cc0 0x56156855b0b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.686+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561568548ca0 0x561568545dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.686+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x561568545880 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.686+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/3696279961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853dac0 0x5615685485d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.686+0000 7fb497e00640 0 client.25985 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:36.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.686+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/3696279961 >> 172.21.15.2:0/3696279961 conn(0x56156853a050 msgr2=0x56156853fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:36.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.686+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/3696279961 shutdown_connections 2024-09-10T08:10:36.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.686+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/3696279961 wait complete. 2024-09-10T08:10:36.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.686+0000 7fb4bbe3e740 20 client.25985 trim_cache size 0 max 16384 2024-09-10T08:10:36.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.704+0000 7fb4bbe3e740 1 Processor -- start 2024-09-10T08:10:36.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.705+0000 7fb4bbe3e740 1 -- start start 2024-09-10T08:10:36.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.705+0000 7fb4bbe3e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 0x561568560980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.705+0000 7fb4bbe3e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x561568560ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.705+0000 7fb4bbe3e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561568548ca0 0x561568550cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.705+0000 7fb4bbe3e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41b3990 con 0x56156853dac0 2024-09-10T08:10:36.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.705+0000 7fb4bbe3e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41b1260 con 0x56156853e3e0 2024-09-10T08:10:36.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.705+0000 7fb4bbe3e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41af0b0 con 0x561568548ca0 2024-09-10T08:10:36.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.705+0000 7fb4b2800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 0x561568560980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.705+0000 7fb4b2800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 0x561568560980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50016/0 (socket says 172.21.15.2:50016) 2024-09-10T08:10:36.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.705+0000 7fb4a9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x561568560ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.705+0000 7fb4b2800640 1 -- 172.21.15.2:0/2788285531 learned_addr learned my addr 172.21.15.2:0/2788285531 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:36.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.705+0000 7fb4b2800640 1 --2- 172.21.15.2:0/2788285531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561568548ca0 0x561568550cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.706+0000 7fb4b2800640 1 -- 172.21.15.2:0/2788285531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561568548ca0 msgr2=0x561568550cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:36.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.706+0000 7fb4b2800640 1 --2- 172.21.15.2:0/2788285531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561568548ca0 0x561568550cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.706+0000 7fb4b2800640 1 -- 172.21.15.2:0/2788285531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 msgr2=0x561568560ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:36.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.706+0000 7fb4b2800640 1 --2- 172.21.15.2:0/2788285531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x561568560ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.706+0000 7fb4b2800640 1 -- 172.21.15.2:0/2788285531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561568554b70 con 0x56156853dac0 2024-09-10T08:10:36.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.706+0000 7fb4b2800640 1 --2- 172.21.15.2:0/2788285531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 0x561568560980 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac005f10 tx=0x7fb4ac009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92309a9c212ce697 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:36.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.706+0000 7fb4a2a00640 1 -- 172.21.15.2:0/2788285531 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac02e020 con 0x56156853dac0 2024-09-10T08:10:36.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.706+0000 7fb4a2a00640 1 -- 172.21.15.2:0/2788285531 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4ac011030 con 0x56156853dac0 2024-09-10T08:10:36.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.706+0000 7fb4a2a00640 1 -- 172.21.15.2:0/2788285531 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac005050 con 0x56156853dac0 2024-09-10T08:10:36.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.707+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/2788285531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 msgr2=0x561568560980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:36.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.707+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/2788285531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 0x561568560980 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac005f10 tx=0x7fb4ac009500 comp rx=0 tx=0).stop 2024-09-10T08:10:36.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.707+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/2788285531 shutdown_connections 2024-09-10T08:10:36.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.707+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/2788285531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561568548ca0 0x561568550cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.707+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/2788285531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x561568560ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.707+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/2788285531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56156853dac0 0x561568560980 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.707+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/2788285531 >> 172.21.15.2:0/2788285531 conn(0x56156853a050 msgr2=0x561568547900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:36.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.707+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/2788285531 shutdown_connections 2024-09-10T08:10:36.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.707+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/2788285531 wait complete. 2024-09-10T08:10:36.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.708+0000 7fb4bbe3e740 1 -- start start 2024-09-10T08:10:36.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.708+0000 7fb4bbe3e740 1 Processor -- start 2024-09-10T08:10:36.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.708+0000 7fb4bbe3e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853dac0 0x561568551420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.708+0000 7fb4bbe3e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x56156854e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.708+0000 7fb4bbe3e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561568548ca0 0x56156854ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.708+0000 7fb4bbe3e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41af720 con 0x561568548ca0 2024-09-10T08:10:36.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.708+0000 7fb4bbe3e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41b0b90 con 0x56156853e3e0 2024-09-10T08:10:36.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.708+0000 7fb4bbe3e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41afdf0 con 0x56156853dac0 2024-09-10T08:10:36.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.708+0000 7fb4bbe3e740 10 client.0 _ll_register_callbacks cb 0x7fb49c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:36.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.709+0000 7fb4b2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853dac0 0x561568551420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.709+0000 7fb4a9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x56156854e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.709+0000 7fb4b2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853dac0 0x561568551420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:42248/0 (socket says 172.21.15.2:42248) 2024-09-10T08:10:36.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.709+0000 7fb4b2800640 1 -- 172.21.15.2:0/1099068531 learned_addr learned my addr 172.21.15.2:0/1099068531 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:36.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.709+0000 7fb4a9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x56156854e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60292/0 (socket says 172.21.15.2:60292) 2024-09-10T08:10:36.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.709+0000 7fb4b2800640 1 --2- 172.21.15.2:0/1099068531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561568548ca0 0x56156854ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.709+0000 7fb4a9400640 1 -- 172.21.15.2:0/1099068531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853dac0 msgr2=0x561568551420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.709+0000 7fb4a9400640 1 --2- 172.21.15.2:0/1099068531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853dac0 0x561568551420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.709+0000 7fb4a9400640 1 -- 172.21.15.2:0/1099068531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561568548ca0 msgr2=0x56156854ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.709+0000 7fb4a9400640 1 --2- 172.21.15.2:0/1099068531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561568548ca0 0x56156854ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.709+0000 7fb4a9400640 1 -- 172.21.15.2:0/1099068531 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56156854f4e0 con 0x56156853e3e0 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.709+0000 7fb4a9400640 1 --2- 172.21.15.2:0/1099068531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x56156854e770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009900 tx=0x7fb49c0048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b4f80307395335e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.709+0000 7fb497e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.710+0000 7fb497e00640 1 -- 172.21.15.2:0/1099068531 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c035070 con 0x56156853e3e0 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.710+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/1099068531 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4ac011030 con 0x56156853e3e0 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.710+0000 7fb497e00640 1 -- 172.21.15.2:0/1099068531 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb49c004fa0 con 0x56156853e3e0 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.710+0000 7fb497e00640 1 -- 172.21.15.2:0/1099068531 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb49c060050 con 0x56156853e3e0 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.710+0000 7fb4bbe3e740 10 client.25991 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.710+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/1099068531 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5615685c3990 con 0x56156853e3e0 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.711+0000 7fb497e00640 1 -- 172.21.15.2:0/1099068531 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4ac011030 con 0x56156853e3e0 2024-09-10T08:10:36.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.711+0000 7fb497e00640 1 -- 172.21.15.2:0/1099068531 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb49c05b030 con 0x56156853e3e0 2024-09-10T08:10:36.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.711+0000 7fb4bbe3e740 10 client.25991 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:36.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.711+0000 7fb4bbe3e740 10 client.25991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:36.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.711+0000 7fb4bbe3e740 20 client.25991 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:36.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.711+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/1099068531 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5615685c4100 0x5615685c6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:36.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.711+0000 7fb4bbe3e740 4 client.25991 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:10:36.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.711+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/1099068531 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561568427fb0 con 0x5615685c4100 2024-09-10T08:10:36.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.711+0000 7fb4b2800640 1 --2- 172.21.15.2:0/1099068531 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5615685c4100 0x5615685c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:36.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.712+0000 7fb4b2800640 1 --2- 172.21.15.2:0/1099068531 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5615685c4100 0x5615685c6520 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98cd7b51d18303fc server_cookie=6c8240b004c27a6c in_seq=0 out_seq=0 2024-09-10T08:10:36.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.712+0000 7fb497e00640 10 client.25991 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:36.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb497e00640 1 -- 172.21.15.2:0/1099068531 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561568427fb0 con 0x5615685c4100 2024-09-10T08:10:36.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb497e00640 10 client.25991 handle_command_reply: tid=0 2024-09-10T08:10:36.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb4bbe3e740 1 client.25991 shutdown 2024-09-10T08:10:36.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/1099068531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 msgr2=0x56156854e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:36.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/1099068531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x56156854e770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009900 tx=0x7fb49c0048f0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/1099068531 shutdown_connections 2024-09-10T08:10:36.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/1099068531 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5615685c4100 0x5615685c6520 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/1099068531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561568548ca0 0x56156854ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/1099068531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56156853e3e0 0x56156854e770 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb4bbe3e740 1 --2- 172.21.15.2:0/1099068531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56156853dac0 0x561568551420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/1099068531 >> 172.21.15.2:0/1099068531 conn(0x56156853a050 msgr2=0x561568553b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:36.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb497e00640 0 client.25991 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:36.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/1099068531 shutdown_connections 2024-09-10T08:10:36.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.713+0000 7fb4bbe3e740 1 -- 172.21.15.2:0/1099068531 wait complete. 2024-09-10T08:10:36.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.714+0000 7fb4bbe3e740 20 client.25991 trim_cache size 0 max 16384 2024-09-10T08:10:36.718 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:10:36.718 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:10:36.718 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "c1e99283-f251-4178-9944-e04f9dead25a", 2024-09-10T08:10:36.718 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:10:36.718 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:10:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.714+0000 7fb4b9200640 1 -- 172.21.15.2:0/3358261342 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb4980a2b80 msgr2=0x7fb4980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.714+0000 7fb4b9200640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb4980a2b80 0x7fb4980a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4097630 tx=0x7fb4a4005cd0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.714+0000 7fb4b9200640 1 -- 172.21.15.2:0/3358261342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b4104010 msgr2=0x7fb4b419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.714+0000 7fb4b9200640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b4104010 0x7fb4b419a660 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400d950 tx=0x7fb4a400de10 comp rx=0 tx=0).stop 2024-09-10T08:10:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.714+0000 7fb4b9200640 1 -- 172.21.15.2:0/3358261342 shutdown_connections 2024-09-10T08:10:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.714+0000 7fb4b9200640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b4109140 0x7fb4b419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.714+0000 7fb4b9200640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb4980a2b80 0x7fb4980a5040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.714+0000 7fb4b9200640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4b41049c0 0x7fb4b419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.714+0000 7fb4b9200640 1 --2- 172.21.15.2:0/3358261342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b4104010 0x7fb4b419a660 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.714+0000 7fb4b9200640 1 -- 172.21.15.2:0/3358261342 >> 172.21.15.2:0/3358261342 conn(0x7fb4b40ffa50 msgr2=0x7fb4b4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.715+0000 7fb4b9200640 1 -- 172.21.15.2:0/3358261342 shutdown_connections 2024-09-10T08:10:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:36.715+0000 7fb4b9200640 1 -- 172.21.15.2:0/3358261342 wait complete. 2024-09-10T08:10:37.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:37 smithi002 ceph-mon[30563]: pgmap v461: 129 pgs: 129 active+clean; 887 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:10:37.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:37 smithi155 ceph-mon[35405]: pgmap v461: 129 pgs: 129 active+clean; 887 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:10:37.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:37 smithi107 ceph-mon[35048]: pgmap v461: 129 pgs: 129 active+clean; 887 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:10:37.688 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:10:38.070 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:38.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:38 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:10:38.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:38 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:38.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:38 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:10:38.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:38 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:38.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.654+0000 7fd853400640 1 -- 172.21.15.2:0/418572662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd84c0745c0 msgr2=0x7fd84c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:38.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.654+0000 7fd853400640 1 --2- 172.21.15.2:0/418572662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd84c0745c0 0x7fd84c0749a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd840009900 tx=0x7fd84002a480 comp rx=0 tx=0).stop 2024-09-10T08:10:38.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.654+0000 7fd853400640 1 -- 172.21.15.2:0/418572662 shutdown_connections 2024-09-10T08:10:38.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.654+0000 7fd853400640 1 --2- 172.21.15.2:0/418572662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd84c10e430 0x7fd84c110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.654+0000 7fd853400640 1 --2- 172.21.15.2:0/418572662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd84c074ee0 0x7fd84c10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.654+0000 7fd853400640 1 --2- 172.21.15.2:0/418572662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd84c0745c0 0x7fd84c0749a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.654+0000 7fd853400640 1 -- 172.21.15.2:0/418572662 >> 172.21.15.2:0/418572662 conn(0x7fd84c0fd8b0 msgr2=0x7fd84c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:38.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.654+0000 7fd853400640 1 -- 172.21.15.2:0/418572662 shutdown_connections 2024-09-10T08:10:38.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.654+0000 7fd853400640 1 -- 172.21.15.2:0/418572662 wait complete. 2024-09-10T08:10:38.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.655+0000 7fd853400640 1 Processor -- start 2024-09-10T08:10:38.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.655+0000 7fd853400640 1 -- start start 2024-09-10T08:10:38.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.655+0000 7fd853400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd84c0745c0 0x7fd84c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.655+0000 7fd853400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd84c074ee0 0x7fd84c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.655+0000 7fd853400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd84c10e430 0x7fd84c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.655+0000 7fd853400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1141d0 con 0x7fd84c0745c0 2024-09-10T08:10:38.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.655+0000 7fd853400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c114050 con 0x7fd84c10e430 2024-09-10T08:10:38.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.655+0000 7fd853400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c114350 con 0x7fd84c074ee0 2024-09-10T08:10:38.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.656+0000 7fd850a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd84c0745c0 0x7fd84c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.656+0000 7fd850a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd84c0745c0 0x7fd84c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40714/0 (socket says 172.21.15.2:40714) 2024-09-10T08:10:38.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.656+0000 7fd850a00640 1 -- 172.21.15.2:0/1656181612 learned_addr learned my addr 172.21.15.2:0/1656181612 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:38.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.656+0000 7fd851400640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd84c10e430 0x7fd84c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.656+0000 7fd84be00640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd84c074ee0 0x7fd84c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.656+0000 7fd850a00640 1 -- 172.21.15.2:0/1656181612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd84c074ee0 msgr2=0x7fd84c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:38.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.656+0000 7fd850a00640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd84c074ee0 0x7fd84c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.656+0000 7fd850a00640 1 -- 172.21.15.2:0/1656181612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd84c10e430 msgr2=0x7fd84c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:38.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.656+0000 7fd850a00640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd84c10e430 0x7fd84c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.656+0000 7fd850a00640 1 -- 172.21.15.2:0/1656181612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84c19f5f0 con 0x7fd84c0745c0 2024-09-10T08:10:38.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.656+0000 7fd850a00640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd84c0745c0 0x7fd84c19a640 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd840004770 tx=0x7fd840003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:38.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:38 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:10:38.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:38 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:38.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:38 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:10:38.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:38 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:38.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.657+0000 7fd849600640 1 -- 172.21.15.2:0/1656181612 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84003f070 con 0x7fd84c0745c0 2024-09-10T08:10:38.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.657+0000 7fd849600640 1 -- 172.21.15.2:0/1656181612 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd840031ca0 con 0x7fd84c0745c0 2024-09-10T08:10:38.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.657+0000 7fd849600640 1 -- 172.21.15.2:0/1656181612 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840035030 con 0x7fd84c0745c0 2024-09-10T08:10:38.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.657+0000 7fd853400640 1 -- 172.21.15.2:0/1656181612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84c19f880 con 0x7fd84c0745c0 2024-09-10T08:10:38.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.657+0000 7fd853400640 1 -- 172.21.15.2:0/1656181612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84c19fd10 con 0x7fd84c0745c0 2024-09-10T08:10:38.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.661+0000 7fd855e1a740 1 Processor -- start 2024-09-10T08:10:38.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.661+0000 7fd855e1a740 1 -- start start 2024-09-10T08:10:38.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.662+0000 7fd849600640 1 -- 172.21.15.2:0/1656181612 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd840042070 con 0x7fd84c0745c0 2024-09-10T08:10:38.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.662+0000 7fd855e1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c1b90 0x55cc573c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.662+0000 7fd855e1a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573c24b0 0x55cc573cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.662+0000 7fd855e1a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573ccd70 0x55cc573cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.662+0000 7fd855e1a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cc573283e0 con 0x55cc573ccd70 2024-09-10T08:10:38.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.662+0000 7fd855e1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cc572cfd80 con 0x55cc573c1b90 2024-09-10T08:10:38.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.662+0000 7fd855e1a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cc573243a0 con 0x55cc573c24b0 2024-09-10T08:10:38.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.662+0000 7fd850a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573c24b0 0x55cc573cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.662+0000 7fd850a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573c24b0 0x55cc573cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36562/0 (socket says 172.21.15.2:36562) 2024-09-10T08:10:38.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.662+0000 7fd850a00640 1 -- 172.21.15.2:0/307705921 learned_addr learned my addr 172.21.15.2:0/307705921 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:38.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.662+0000 7fd851400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c1b90 0x55cc573c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.662+0000 7fd84be00640 1 --2- 172.21.15.2:0/307705921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573ccd70 0x55cc573cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.663+0000 7fd850a00640 1 -- 172.21.15.2:0/307705921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c1b90 msgr2=0x55cc573c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.663+0000 7fd850a00640 1 --2- 172.21.15.2:0/307705921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c1b90 0x55cc573c1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.663+0000 7fd850a00640 1 -- 172.21.15.2:0/307705921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573ccd70 msgr2=0x55cc573cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.663+0000 7fd850a00640 1 --2- 172.21.15.2:0/307705921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573ccd70 0x55cc573cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.663+0000 7fd850a00640 1 -- 172.21.15.2:0/307705921 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc572eeae0 con 0x55cc573c24b0 2024-09-10T08:10:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.663+0000 7fd851400640 1 --2- 172.21.15.2:0/307705921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c1b90 0x55cc573c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.663+0000 7fd84be00640 1 --2- 172.21.15.2:0/307705921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573ccd70 0x55cc573cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.663+0000 7fd850a00640 1 --2- 172.21.15.2:0/307705921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573c24b0 0x55cc573cc830 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd8400f3c50 tx=0x7fd8400f3d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5a73c5f563fbdc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.663+0000 7fd849600640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8340a2c00 0x7fd8340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.663+0000 7fd83dc00640 1 -- 172.21.15.2:0/307705921 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8400f8ba0 con 0x55cc573c24b0 2024-09-10T08:10:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.663+0000 7fd83dc00640 1 -- 172.21.15.2:0/307705921 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8400bc620 con 0x55cc573c24b0 2024-09-10T08:10:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.663+0000 7fd83dc00640 1 -- 172.21.15.2:0/307705921 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8400bb880 con 0x55cc573c24b0 2024-09-10T08:10:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.664+0000 7fd849600640 1 -- 172.21.15.2:0/1656181612 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd8400efab0 con 0x7fd84c0745c0 2024-09-10T08:10:38.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.664+0000 7fd851400640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8340a2c00 0x7fd8340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.664+0000 7fd855e1a740 1 -- 172.21.15.2:0/307705921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573c24b0 msgr2=0x55cc573cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:38.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.664+0000 7fd851400640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8340a2c00 0x7fd8340a50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd844009fd0 tx=0x7fd844009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:38.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.664+0000 7fd855e1a740 1 --2- 172.21.15.2:0/307705921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573c24b0 0x55cc573cc830 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd8400f3c50 tx=0x7fd8400f3d40 comp rx=0 tx=0).stop 2024-09-10T08:10:38.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.664+0000 7fd855e1a740 1 -- 172.21.15.2:0/307705921 shutdown_connections 2024-09-10T08:10:38.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.664+0000 7fd855e1a740 1 --2- 172.21.15.2:0/307705921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573ccd70 0x55cc573cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.664+0000 7fd855e1a740 1 --2- 172.21.15.2:0/307705921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573c24b0 0x55cc573cc830 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.665+0000 7fd855e1a740 1 --2- 172.21.15.2:0/307705921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c1b90 0x55cc573c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.665+0000 7fd855e1a740 1 -- 172.21.15.2:0/307705921 >> 172.21.15.2:0/307705921 conn(0x55cc573be120 msgr2=0x55cc573be500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:38.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.665+0000 7fd855e1a740 1 -- 172.21.15.2:0/307705921 shutdown_connections 2024-09-10T08:10:38.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.665+0000 7fd855e1a740 1 -- 172.21.15.2:0/307705921 wait complete. 2024-09-10T08:10:38.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.665+0000 7fd855e1a740 1 -- start start 2024-09-10T08:10:38.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.665+0000 7fd855e1a740 1 Processor -- start 2024-09-10T08:10:38.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd855e1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c1b90 0x55cc573c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd855e1a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573c24b0 0x55cc573ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd855e1a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573ccd70 0x55cc573ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd855e1a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cc573cf700 con 0x55cc573c24b0 2024-09-10T08:10:38.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd855e1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cc572e30f0 con 0x55cc573c1b90 2024-09-10T08:10:38.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd855e1a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cc573cf880 con 0x55cc573ccd70 2024-09-10T08:10:38.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd855e1a740 10 client.0 _ll_register_callbacks cb 0x55cc572d49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:38.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd850a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c1b90 0x55cc573c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd84be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573c24b0 0x55cc573ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd851400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573ccd70 0x55cc573ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd850a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c1b90 0x55cc573c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37320/0 (socket says 172.21.15.2:37320) 2024-09-10T08:10:38.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd850a00640 1 -- 172.21.15.2:0/2243889588 learned_addr learned my addr 172.21.15.2:0/2243889588 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:38.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd850a00640 1 -- 172.21.15.2:0/2243889588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573ccd70 msgr2=0x55cc573ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:38.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd850a00640 1 --2- 172.21.15.2:0/2243889588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573ccd70 0x55cc573ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd850a00640 1 -- 172.21.15.2:0/2243889588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573c24b0 msgr2=0x55cc573ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:38.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd850a00640 1 --2- 172.21.15.2:0/2243889588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573c24b0 0x55cc573ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd850a00640 1 -- 172.21.15.2:0/2243889588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc570993c0 con 0x55cc573c1b90 2024-09-10T08:10:38.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.666+0000 7fd851400640 1 --2- 172.21.15.2:0/2243889588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573ccd70 0x55cc573ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:38.669 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:38 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:10:38.669 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:38 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:38.669 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:38 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:10:38.669 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:38 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:38.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.667+0000 7fd850a00640 1 --2- 172.21.15.2:0/2243889588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c1b90 0x55cc573c9b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd8400bc7d0 tx=0x7fd84003b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6fe9ea5b0cb96e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:38.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.667+0000 7fd832000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:38.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.667+0000 7fd832000640 1 -- 172.21.15.2:0/2243889588 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840047120 con 0x55cc573c1b90 2024-09-10T08:10:38.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.667+0000 7fd832000640 1 -- 172.21.15.2:0/2243889588 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd8400f37d0 con 0x55cc573c1b90 2024-09-10T08:10:38.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.667+0000 7fd855e1a740 1 -- 172.21.15.2:0/2243889588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8400bc620 con 0x55cc573c1b90 2024-09-10T08:10:38.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.667+0000 7fd832000640 1 -- 172.21.15.2:0/2243889588 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd840047310 con 0x55cc573c1b90 2024-09-10T08:10:38.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.667+0000 7fd855e1a740 10 client.26003 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:38.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.667+0000 7fd855e1a740 1 -- 172.21.15.2:0/2243889588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cc573cf700 con 0x55cc573c1b90 2024-09-10T08:10:38.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.668+0000 7fd832000640 1 -- 172.21.15.2:0/2243889588 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8400bc620 con 0x55cc573c1b90 2024-09-10T08:10:38.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.668+0000 7fd832000640 1 -- 172.21.15.2:0/2243889588 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd8400bf020 con 0x55cc573c1b90 2024-09-10T08:10:38.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.668+0000 7fd855e1a740 10 client.26003 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:38.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.668+0000 7fd855e1a740 10 client.26003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:38.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.668+0000 7fd855e1a740 20 client.26003 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:38.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.668+0000 7fd855e1a740 1 --2- 172.21.15.2:0/2243889588 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cc573dc5b0 0x55cc573de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.668+0000 7fd855e1a740 4 client.26003 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:38.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.668+0000 7fd855e1a740 1 -- 172.21.15.2:0/2243889588 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc57335950 con 0x55cc573dc5b0 2024-09-10T08:10:38.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.668+0000 7fd84be00640 1 --2- 172.21.15.2:0/2243889588 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cc573dc5b0 0x55cc573de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.669+0000 7fd84be00640 1 --2- 172.21.15.2:0/2243889588 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cc573dc5b0 0x55cc573de9e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1fecc3d962dea2b server_cookie=3c5e4befb41e8499 in_seq=0 out_seq=0 2024-09-10T08:10:38.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.669+0000 7fd832000640 10 client.26003 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:38.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd832000640 1 -- 172.21.15.2:0/2243889588 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc57335950 con 0x55cc573dc5b0 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd832000640 10 client.26003 handle_command_reply: tid=0 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd855e1a740 1 client.26003 shutdown 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd855e1a740 1 -- 172.21.15.2:0/2243889588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c1b90 msgr2=0x55cc573c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd855e1a740 1 --2- 172.21.15.2:0/2243889588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c1b90 0x55cc573c9b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd8400bc7d0 tx=0x7fd84003b890 comp rx=0 tx=0).stop 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd855e1a740 1 -- 172.21.15.2:0/2243889588 shutdown_connections 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd855e1a740 1 --2- 172.21.15.2:0/2243889588 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cc573dc5b0 0x55cc573de9e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd855e1a740 1 --2- 172.21.15.2:0/2243889588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573ccd70 0x55cc573ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd855e1a740 1 --2- 172.21.15.2:0/2243889588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573c24b0 0x55cc573ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd855e1a740 1 --2- 172.21.15.2:0/2243889588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c1b90 0x55cc573c9b80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd832000640 0 client.26003 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd855e1a740 1 -- 172.21.15.2:0/2243889588 >> 172.21.15.2:0/2243889588 conn(0x55cc573be120 msgr2=0x55cc573c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd855e1a740 1 -- 172.21.15.2:0/2243889588 shutdown_connections 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd855e1a740 1 -- 172.21.15.2:0/2243889588 wait complete. 2024-09-10T08:10:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.670+0000 7fd855e1a740 20 client.26003 trim_cache size 0 max 16384 2024-09-10T08:10:38.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.688+0000 7fd855e1a740 1 Processor -- start 2024-09-10T08:10:38.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.688+0000 7fd855e1a740 1 -- start start 2024-09-10T08:10:38.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd855e1a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573c1b90 0x55cc573c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd855e1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c24b0 0x55cc573e42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd855e1a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573ccd70 0x55cc573e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd855e1a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1b3980 con 0x55cc573c1b90 2024-09-10T08:10:38.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd855e1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1b1250 con 0x55cc573c24b0 2024-09-10T08:10:38.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd855e1a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1af0a0 con 0x55cc573ccd70 2024-09-10T08:10:38.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd850a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573c1b90 0x55cc573c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd850a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573c1b90 0x55cc573c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40740/0 (socket says 172.21.15.2:40740) 2024-09-10T08:10:38.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd850a00640 1 -- 172.21.15.2:0/3815521250 learned_addr learned my addr 172.21.15.2:0/3815521250 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:38.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd84be00640 1 --2- 172.21.15.2:0/3815521250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c24b0 0x55cc573e42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd851400640 1 --2- 172.21.15.2:0/3815521250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573ccd70 0x55cc573e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd850a00640 1 -- 172.21.15.2:0/3815521250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573ccd70 msgr2=0x55cc573e47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:38.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd850a00640 1 --2- 172.21.15.2:0/3815521250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573ccd70 0x55cc573e47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd850a00640 1 -- 172.21.15.2:0/3815521250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c24b0 msgr2=0x55cc573e42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:38.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd850a00640 1 --2- 172.21.15.2:0/3815521250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c24b0 0x55cc573e42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd850a00640 1 -- 172.21.15.2:0/3815521250 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc573bdf50 con 0x55cc573c1b90 2024-09-10T08:10:38.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.689+0000 7fd84be00640 1 --2- 172.21.15.2:0/3815521250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c24b0 0x55cc573e42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:38.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.690+0000 7fd850a00640 1 --2- 172.21.15.2:0/3815521250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573c1b90 0x55cc573c45f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd84003e830 tx=0x7fd8400ba430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42163839f9333927 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:38.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.690+0000 7fd83d200640 1 -- 172.21.15.2:0/3815521250 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840049020 con 0x55cc573c1b90 2024-09-10T08:10:38.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.690+0000 7fd83d200640 1 -- 172.21.15.2:0/3815521250 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8400bc620 con 0x55cc573c1b90 2024-09-10T08:10:38.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.690+0000 7fd83d200640 1 -- 172.21.15.2:0/3815521250 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840047510 con 0x55cc573c1b90 2024-09-10T08:10:38.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.691+0000 7fd855e1a740 1 -- 172.21.15.2:0/3815521250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573c1b90 msgr2=0x55cc573c45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:38.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.691+0000 7fd855e1a740 1 --2- 172.21.15.2:0/3815521250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573c1b90 0x55cc573c45f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd84003e830 tx=0x7fd8400ba430 comp rx=0 tx=0).stop 2024-09-10T08:10:38.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.691+0000 7fd855e1a740 1 -- 172.21.15.2:0/3815521250 shutdown_connections 2024-09-10T08:10:38.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.691+0000 7fd855e1a740 1 --2- 172.21.15.2:0/3815521250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573ccd70 0x55cc573e47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.691+0000 7fd855e1a740 1 --2- 172.21.15.2:0/3815521250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c24b0 0x55cc573e42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.691+0000 7fd855e1a740 1 --2- 172.21.15.2:0/3815521250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573c1b90 0x55cc573c45f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.691+0000 7fd855e1a740 1 -- 172.21.15.2:0/3815521250 >> 172.21.15.2:0/3815521250 conn(0x55cc573be120 msgr2=0x55cc573cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:38.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.691+0000 7fd855e1a740 1 -- 172.21.15.2:0/3815521250 shutdown_connections 2024-09-10T08:10:38.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.691+0000 7fd855e1a740 1 -- 172.21.15.2:0/3815521250 wait complete. 2024-09-10T08:10:38.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.691+0000 7fd855e1a740 1 -- start start 2024-09-10T08:10:38.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.692+0000 7fd855e1a740 1 Processor -- start 2024-09-10T08:10:38.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.692+0000 7fd855e1a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573c1b90 0x55cc5743f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.692+0000 7fd855e1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c24b0 0x55cc5743fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.692+0000 7fd855e1a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573ccd70 0x55cc573d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.692+0000 7fd855e1a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1af710 con 0x55cc573ccd70 2024-09-10T08:10:38.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.692+0000 7fd855e1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1b0b80 con 0x55cc573c24b0 2024-09-10T08:10:38.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.692+0000 7fd855e1a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1afde0 con 0x55cc573c1b90 2024-09-10T08:10:38.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.692+0000 7fd855e1a740 10 client.0 _ll_register_callbacks cb 0x7fd8400f3f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:38.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.692+0000 7fd851400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573ccd70 0x55cc573d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.692+0000 7fd851400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573ccd70 0x55cc573d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40744/0 (socket says 172.21.15.2:40744) 2024-09-10T08:10:38.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.692+0000 7fd851400640 1 -- 172.21.15.2:0/2392653465 learned_addr learned my addr 172.21.15.2:0/2392653465 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:38.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.692+0000 7fd850a00640 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573c1b90 0x55cc5743f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.692+0000 7fd84be00640 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c24b0 0x55cc5743fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.693+0000 7fd851400640 1 -- 172.21.15.2:0/2392653465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573c1b90 msgr2=0x55cc5743f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:38.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.693+0000 7fd851400640 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573c1b90 0x55cc5743f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.693+0000 7fd851400640 1 -- 172.21.15.2:0/2392653465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c24b0 msgr2=0x55cc5743fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:38.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.693+0000 7fd851400640 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c24b0 0x55cc5743fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.693+0000 7fd851400640 1 -- 172.21.15.2:0/2392653465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc573d0900 con 0x55cc573ccd70 2024-09-10T08:10:38.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.693+0000 7fd84be00640 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c24b0 0x55cc5743fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:38.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.693+0000 7fd850a00640 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573c1b90 0x55cc5743f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:38.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.693+0000 7fd851400640 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573ccd70 0x55cc573d00d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd84400fbf0 tx=0x7fd844004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acab4f4b931b4ca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:38.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.693+0000 7fd832000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:38.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.694+0000 7fd832000640 1 -- 172.21.15.2:0/2392653465 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84402e060 con 0x55cc573ccd70 2024-09-10T08:10:38.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.694+0000 7fd855e1a740 1 -- 172.21.15.2:0/2392653465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8400bc620 con 0x55cc573ccd70 2024-09-10T08:10:38.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.694+0000 7fd832000640 1 -- 172.21.15.2:0/2392653465 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd844010bc0 con 0x55cc573ccd70 2024-09-10T08:10:38.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.694+0000 7fd832000640 1 -- 172.21.15.2:0/2392653465 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd844035050 con 0x55cc573ccd70 2024-09-10T08:10:38.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.694+0000 7fd855e1a740 10 client.15975 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:38.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.694+0000 7fd855e1a740 1 -- 172.21.15.2:0/2392653465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cc57446e40 con 0x55cc573ccd70 2024-09-10T08:10:38.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.694+0000 7fd832000640 1 -- 172.21.15.2:0/2392653465 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8400bc620 con 0x55cc573ccd70 2024-09-10T08:10:38.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.694+0000 7fd832000640 1 -- 172.21.15.2:0/2392653465 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd84401d410 con 0x55cc573ccd70 2024-09-10T08:10:38.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.694+0000 7fd855e1a740 10 client.15975 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:38.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.695+0000 7fd855e1a740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:38.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.695+0000 7fd855e1a740 20 client.15975 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:38.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.695+0000 7fd855e1a740 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cc57447510 0x55cc57449900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:38.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.695+0000 7fd855e1a740 4 client.15975 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:10:38.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.695+0000 7fd855e1a740 1 -- 172.21.15.2:0/2392653465 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cc5732fe60 con 0x55cc57447510 2024-09-10T08:10:38.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.695+0000 7fd850a00640 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cc57447510 0x55cc57449900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:38.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.695+0000 7fd850a00640 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cc57447510 0x55cc57449900 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d1f6d2b3ff39312 server_cookie=1dd8a492fcd72bc2 in_seq=0 out_seq=0 2024-09-10T08:10:38.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.695+0000 7fd832000640 10 client.15975 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:38.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd832000640 1 -- 172.21.15.2:0/2392653465 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cc5732fe60 con 0x55cc57447510 2024-09-10T08:10:38.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd832000640 10 client.15975 handle_command_reply: tid=0 2024-09-10T08:10:38.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd855e1a740 1 client.15975 shutdown 2024-09-10T08:10:38.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd855e1a740 1 -- 172.21.15.2:0/2392653465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573ccd70 msgr2=0x55cc573d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:38.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd855e1a740 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573ccd70 0x55cc573d00d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd84400fbf0 tx=0x7fd844004a40 comp rx=0 tx=0).stop 2024-09-10T08:10:38.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd855e1a740 1 -- 172.21.15.2:0/2392653465 shutdown_connections 2024-09-10T08:10:38.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd855e1a740 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cc57447510 0x55cc57449900 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd855e1a740 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc573ccd70 0x55cc573d00d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd855e1a740 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc573c24b0 0x55cc5743fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd855e1a740 1 --2- 172.21.15.2:0/2392653465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc573c1b90 0x55cc5743f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd832000640 0 client.15975 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:38.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd855e1a740 1 -- 172.21.15.2:0/2392653465 >> 172.21.15.2:0/2392653465 conn(0x55cc573be120 msgr2=0x55cc573c8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:38.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd855e1a740 1 -- 172.21.15.2:0/2392653465 shutdown_connections 2024-09-10T08:10:38.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd855e1a740 1 -- 172.21.15.2:0/2392653465 wait complete. 2024-09-10T08:10:38.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.696+0000 7fd855e1a740 20 client.15975 trim_cache size 0 max 16384 2024-09-10T08:10:38.701 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:10:38.701 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:10:38.702 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:10:38.702 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:10:38.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.697+0000 7fd853400640 1 -- 172.21.15.2:0/1656181612 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8340a2c00 msgr2=0x7fd8340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:38.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.697+0000 7fd853400640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8340a2c00 0x7fd8340a50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd844009fd0 tx=0x7fd844009500 comp rx=0 tx=0).stop 2024-09-10T08:10:38.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.697+0000 7fd853400640 1 -- 172.21.15.2:0/1656181612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd84c0745c0 msgr2=0x7fd84c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.697+0000 7fd853400640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd84c0745c0 0x7fd84c19a640 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd840004770 tx=0x7fd840003c10 comp rx=0 tx=0).stop 2024-09-10T08:10:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.697+0000 7fd853400640 1 -- 172.21.15.2:0/1656181612 shutdown_connections 2024-09-10T08:10:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.697+0000 7fd853400640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd84c10e430 0x7fd84c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.697+0000 7fd853400640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8340a2c00 0x7fd8340a50c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.697+0000 7fd853400640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd84c074ee0 0x7fd84c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.697+0000 7fd853400640 1 --2- 172.21.15.2:0/1656181612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd84c0745c0 0x7fd84c19a640 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.697+0000 7fd853400640 1 -- 172.21.15.2:0/1656181612 >> 172.21.15.2:0/1656181612 conn(0x7fd84c0fd8b0 msgr2=0x7fd84c0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.697+0000 7fd853400640 1 -- 172.21.15.2:0/1656181612 shutdown_connections 2024-09-10T08:10:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:38.697+0000 7fd853400640 1 -- 172.21.15.2:0/1656181612 wait complete. 2024-09-10T08:10:39.501 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:39 smithi002 ceph-mon[30563]: pgmap v462: 129 pgs: 129 active+clean; 887 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-10T08:10:39.502 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:39 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:10:39.539 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:10:39.539 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:10:39.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:39 smithi155 ceph-mon[35405]: pgmap v462: 129 pgs: 129 active+clean; 887 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-10T08:10:39.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:39 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:10:39.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:39 smithi107 ceph-mon[35048]: pgmap v462: 129 pgs: 129 active+clean; 887 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-10T08:10:39.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:39 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:10:39.919 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:40.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.501+0000 7f37c0800640 1 -- 172.21.15.2:0/3192710942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37b8101df0 msgr2=0x7f37b81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:40.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.501+0000 7f37c0800640 1 --2- 172.21.15.2:0/3192710942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37b8101df0 0x7f37b81021d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009900 tx=0x7f37ac02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:10:40.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.502+0000 7f37c0800640 1 -- 172.21.15.2:0/3192710942 shutdown_connections 2024-09-10T08:10:40.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.502+0000 7f37c0800640 1 --2- 172.21.15.2:0/3192710942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37b8106f20 0x7f37b810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:40.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.502+0000 7f37c0800640 1 --2- 172.21.15.2:0/3192710942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b81027a0 0x7f37b81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:40.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.502+0000 7f37c0800640 1 --2- 172.21.15.2:0/3192710942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37b8101df0 0x7f37b81021d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:40.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.502+0000 7f37c0800640 1 -- 172.21.15.2:0/3192710942 >> 172.21.15.2:0/3192710942 conn(0x7f37b80fd8b0 msgr2=0x7f37b80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:40.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.502+0000 7f37c0800640 1 -- 172.21.15.2:0/3192710942 shutdown_connections 2024-09-10T08:10:40.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.502+0000 7f37c0800640 1 -- 172.21.15.2:0/3192710942 wait complete. 2024-09-10T08:10:40.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.503+0000 7f37c0800640 1 Processor -- start 2024-09-10T08:10:40.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.503+0000 7f37c0800640 1 -- start start 2024-09-10T08:10:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.503+0000 7f37c0800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37b8101df0 0x7f37b8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.503+0000 7f37c0800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b81027a0 0x7f37b8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.503+0000 7f37c0800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37b8106f20 0x7f37b819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.503+0000 7f37c0800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8111f80 con 0x7f37b8101df0 2024-09-10T08:10:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.503+0000 7f37c0800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8111e00 con 0x7f37b81027a0 2024-09-10T08:10:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.503+0000 7f37c0800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8112100 con 0x7f37b8106f20 2024-09-10T08:10:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.503+0000 7f37bde00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37b8101df0 0x7f37b8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.503+0000 7f37bde00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37b8101df0 0x7f37b8198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40770/0 (socket says 172.21.15.2:40770) 2024-09-10T08:10:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.503+0000 7f37bde00640 1 -- 172.21.15.2:0/3922685381 learned_addr learned my addr 172.21.15.2:0/3922685381 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.504+0000 7f37bd400640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b81027a0 0x7f37b8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.504+0000 7f37be800640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37b8106f20 0x7f37b819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.504+0000 7f37bde00640 1 -- 172.21.15.2:0/3922685381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37b8106f20 msgr2=0x7f37b819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.504+0000 7f37bde00640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37b8106f20 0x7f37b819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.504+0000 7f37bde00640 1 -- 172.21.15.2:0/3922685381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b81027a0 msgr2=0x7f37b8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.504+0000 7f37bde00640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b81027a0 0x7f37b8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.504+0000 7f37bde00640 1 -- 172.21.15.2:0/3922685381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b819d3b0 con 0x7f37b8101df0 2024-09-10T08:10:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.504+0000 7f37bd400640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b81027a0 0x7f37b8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.504+0000 7f37bde00640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37b8101df0 0x7f37b8198400 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f37ac006010 tx=0x7f37ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.505+0000 7f37b2a00640 1 -- 172.21.15.2:0/3922685381 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac03f030 con 0x7f37b8101df0 2024-09-10T08:10:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.505+0000 7f37b2a00640 1 -- 172.21.15.2:0/3922685381 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ac003ce0 con 0x7f37b8101df0 2024-09-10T08:10:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.505+0000 7f37c0800640 1 -- 172.21.15.2:0/3922685381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b819d640 con 0x7f37b8101df0 2024-09-10T08:10:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.505+0000 7f37b2a00640 1 -- 172.21.15.2:0/3922685381 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac035070 con 0x7f37b8101df0 2024-09-10T08:10:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.505+0000 7f37c0800640 1 -- 172.21.15.2:0/3922685381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b81a4f80 con 0x7f37b8101df0 2024-09-10T08:10:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.506+0000 7f37c0800640 1 -- 172.21.15.2:0/3922685381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b81032d0 con 0x7f37b8101df0 2024-09-10T08:10:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.508+0000 7f37b2a00640 1 -- 172.21.15.2:0/3922685381 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f37ac042030 con 0x7f37b8101df0 2024-09-10T08:10:40.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.510+0000 7f37b2a00640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f37a00a2b80 0x7f37a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:40.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.510+0000 7f37b2a00640 1 -- 172.21.15.2:0/3922685381 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f37ac0efa70 con 0x7f37b8101df0 2024-09-10T08:10:40.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.510+0000 7f37bd400640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f37a00a2b80 0x7f37a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:40.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.511+0000 7f37bd400640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f37a00a2b80 0x7f37a00a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f37b80fddb0 tx=0x7f37a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:40.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.514+0000 7f37b2a00640 1 -- 172.21.15.2:0/3922685381 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37ac0bc640 con 0x7f37b8101df0 2024-09-10T08:10:40.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.739+0000 7f37c0800640 1 -- 172.21.15.2:0/3922685381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37b810a100 con 0x7f37b8101df0 2024-09-10T08:10:40.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.740+0000 7f37b2a00640 1 -- 172.21.15.2:0/3922685381 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f37ac0bbf40 con 0x7f37b8101df0 2024-09-10T08:10:40.740 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:10:40.740 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:10:40.740 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:10:40.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.742+0000 7f37c0800640 1 -- 172.21.15.2:0/3922685381 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f37a00a2b80 msgr2=0x7f37a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:40.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.742+0000 7f37c0800640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f37a00a2b80 0x7f37a00a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f37b80fddb0 tx=0x7f37a8005f90 comp rx=0 tx=0).stop 2024-09-10T08:10:40.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.742+0000 7f37c0800640 1 -- 172.21.15.2:0/3922685381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37b8101df0 msgr2=0x7f37b8198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:40.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.742+0000 7f37c0800640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37b8101df0 0x7f37b8198400 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f37ac006010 tx=0x7f37ac0042f0 comp rx=0 tx=0).stop 2024-09-10T08:10:40.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.742+0000 7f37c0800640 1 -- 172.21.15.2:0/3922685381 shutdown_connections 2024-09-10T08:10:40.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.742+0000 7f37c0800640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f37a00a2b80 0x7f37a00a5040 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:40.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.742+0000 7f37c0800640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37b8106f20 0x7f37b819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:40.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.742+0000 7f37c0800640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b81027a0 0x7f37b8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:40.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.742+0000 7f37c0800640 1 --2- 172.21.15.2:0/3922685381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37b8101df0 0x7f37b8198400 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:40.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.742+0000 7f37c0800640 1 -- 172.21.15.2:0/3922685381 >> 172.21.15.2:0/3922685381 conn(0x7f37b80fd8b0 msgr2=0x7f37b80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:40.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.743+0000 7f37c0800640 1 -- 172.21.15.2:0/3922685381 shutdown_connections 2024-09-10T08:10:40.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:40.743+0000 7f37c0800640 1 -- 172.21.15.2:0/3922685381 wait complete. 2024-09-10T08:10:41.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:41 smithi002 ceph-mon[30563]: pgmap v463: 129 pgs: 129 active+clean; 887 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:10:41.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:41 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3922685381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:10:41.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:41 smithi155 ceph-mon[35405]: pgmap v463: 129 pgs: 129 active+clean; 887 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:10:41.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:41 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3922685381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:10:41.661 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:10:41.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:41 smithi107 ceph-mon[35048]: pgmap v463: 129 pgs: 129 active+clean; 887 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:10:41.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:41 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3922685381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:10:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.774+0000 7fdfabc00640 1 -- 172.21.15.2:0/4026809638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfa4101b60 msgr2=0x7fdfa4101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.774+0000 7fdfabc00640 1 --2- 172.21.15.2:0/4026809638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfa4101b60 0x7fdfa4101f60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009900 tx=0x7fdf9802a410 comp rx=0 tx=0).stop 2024-09-10T08:10:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.774+0000 7fdfabc00640 1 -- 172.21.15.2:0/4026809638 shutdown_connections 2024-09-10T08:10:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.774+0000 7fdfabc00640 1 --2- 172.21.15.2:0/4026809638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa4103710 0x7fdfa4109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.774+0000 7fdfabc00640 1 --2- 172.21.15.2:0/4026809638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa4102d50 0x7fdfa41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.774+0000 7fdfabc00640 1 --2- 172.21.15.2:0/4026809638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfa4101b60 0x7fdfa4101f60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.774+0000 7fdfabc00640 1 -- 172.21.15.2:0/4026809638 >> 172.21.15.2:0/4026809638 conn(0x7fdfa40fd2e0 msgr2=0x7fdfa40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.774+0000 7fdfabc00640 1 -- 172.21.15.2:0/4026809638 shutdown_connections 2024-09-10T08:10:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.774+0000 7fdfabc00640 1 -- 172.21.15.2:0/4026809638 wait complete. 2024-09-10T08:10:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.775+0000 7fdfabc00640 1 Processor -- start 2024-09-10T08:10:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.775+0000 7fdfabc00640 1 -- start start 2024-09-10T08:10:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfabc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfa4101b60 0x7fdfa419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfabc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa4102d50 0x7fdfa419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfabc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa4103710 0x7fdfa41a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfabc00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa410d830 con 0x7fdfa4101b60 2024-09-10T08:10:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfabc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa410d6b0 con 0x7fdfa4102d50 2024-09-10T08:10:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfabc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa410d9b0 con 0x7fdfa4103710 2024-09-10T08:10:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfa9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfa4101b60 0x7fdfa419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfa9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfa4101b60 0x7fdfa419a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40776/0 (socket says 172.21.15.2:40776) 2024-09-10T08:10:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfa9200640 1 -- 172.21.15.2:0/3591106231 learned_addr learned my addr 172.21.15.2:0/3591106231 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfa3e00640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa4102d50 0x7fdfa419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa4103710 0x7fdfa41a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfa9200640 1 -- 172.21.15.2:0/3591106231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa4103710 msgr2=0x7fdfa41a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfa9200640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa4103710 0x7fdfa41a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfa9200640 1 -- 172.21.15.2:0/3591106231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa4102d50 msgr2=0x7fdfa419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfa9200640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa4102d50 0x7fdfa419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfa9200640 1 -- 172.21.15.2:0/3591106231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa41a1c40 con 0x7fdfa4101b60 2024-09-10T08:10:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.776+0000 7fdfa3e00640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa4102d50 0x7fdfa419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.777+0000 7fdfa9200640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfa4101b60 0x7fdfa419a020 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdf98031440 tx=0x7fdf98032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.777+0000 7fdfa1600640 1 -- 172.21.15.2:0/3591106231 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9803f030 con 0x7fdfa4101b60 2024-09-10T08:10:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.777+0000 7fdfa1600640 1 -- 172.21.15.2:0/3591106231 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf98003ce0 con 0x7fdfa4101b60 2024-09-10T08:10:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.777+0000 7fdfabc00640 1 -- 172.21.15.2:0/3591106231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa41a1ed0 con 0x7fdfa4101b60 2024-09-10T08:10:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.777+0000 7fdfa1600640 1 -- 172.21.15.2:0/3591106231 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98035070 con 0x7fdfa4101b60 2024-09-10T08:10:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.777+0000 7fdfabc00640 1 -- 172.21.15.2:0/3591106231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa41a2330 con 0x7fdfa4101b60 2024-09-10T08:10:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.780+0000 7fdfae9f9740 1 Processor -- start 2024-09-10T08:10:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.780+0000 7fdfae9f9740 1 -- start start 2024-09-10T08:10:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfa1600640 1 -- 172.21.15.2:0/3591106231 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fdf98042030 con 0x7fdfa4101b60 2024-09-10T08:10:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfae9f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 0x5650a9b34490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfae9f9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b349d0 0x5650a9b39490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfae9f9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b39a60 0x5650a9b3bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfae9f9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5650a9a99100 con 0x5650a9b349d0 2024-09-10T08:10:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfae9f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5650a9a411b0 con 0x5650a9b34090 2024-09-10T08:10:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfae9f9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5650a9a950c0 con 0x5650a9b39a60 2024-09-10T08:10:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfa9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b349d0 0x5650a9b39490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfa9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 0x5650a9b34490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfa9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 0x5650a9b34490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37380/0 (socket says 172.21.15.2:37380) 2024-09-10T08:10:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfa9c00640 1 -- 172.21.15.2:0/1530511284 learned_addr learned my addr 172.21.15.2:0/1530511284 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfa3e00640 1 --2- 172.21.15.2:0/1530511284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b39a60 0x5650a9b3bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfa9c00640 1 -- 172.21.15.2:0/1530511284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b39a60 msgr2=0x5650a9b3bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/1530511284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b39a60 0x5650a9b3bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfa9c00640 1 -- 172.21.15.2:0/1530511284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b349d0 msgr2=0x5650a9b39490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.781+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/1530511284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b349d0 0x5650a9b39490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.782+0000 7fdfa9c00640 1 -- 172.21.15.2:0/1530511284 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a97ac210 con 0x5650a9b34090 2024-09-10T08:10:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.782+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/1530511284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 0x5650a9b34490 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009fc0 tx=0x7fdf9c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1432d0f1d79c9253 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.782+0000 7fdfa1600640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdf8c0a2cb0 0x7fdf8c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.782+0000 7fdf96000640 1 -- 172.21.15.2:0/1530511284 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c003b70 con 0x5650a9b34090 2024-09-10T08:10:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.782+0000 7fdf96000640 1 -- 172.21.15.2:0/1530511284 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf9c003d10 con 0x5650a9b34090 2024-09-10T08:10:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.782+0000 7fdf96000640 1 -- 172.21.15.2:0/1530511284 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c011aa0 con 0x5650a9b34090 2024-09-10T08:10:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.782+0000 7fdfa1600640 1 -- 172.21.15.2:0/3591106231 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fdf980ef6d0 con 0x7fdfa4101b60 2024-09-10T08:10:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.783+0000 7fdfa9200640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdf8c0a2cb0 0x7fdf8c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.783+0000 7fdfae9f9740 1 -- 172.21.15.2:0/1530511284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 msgr2=0x5650a9b34490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.783+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/1530511284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 0x5650a9b34490 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009fc0 tx=0x7fdf9c010860 comp rx=0 tx=0).stop 2024-09-10T08:10:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.783+0000 7fdfa9200640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdf8c0a2cb0 0x7fdf8c0a5170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdf980bba70 tx=0x7fdf980f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.783+0000 7fdfae9f9740 1 -- 172.21.15.2:0/1530511284 shutdown_connections 2024-09-10T08:10:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.783+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/1530511284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b39a60 0x5650a9b3bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.783+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/1530511284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b349d0 0x5650a9b39490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.783+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/1530511284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 0x5650a9b34490 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.783+0000 7fdfae9f9740 1 -- 172.21.15.2:0/1530511284 >> 172.21.15.2:0/1530511284 conn(0x5650a9b2f700 msgr2=0x5650a9b2fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.783+0000 7fdfae9f9740 1 -- 172.21.15.2:0/1530511284 shutdown_connections 2024-09-10T08:10:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.783+0000 7fdfae9f9740 1 -- 172.21.15.2:0/1530511284 wait complete. 2024-09-10T08:10:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.784+0000 7fdfae9f9740 1 -- start start 2024-09-10T08:10:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.784+0000 7fdfae9f9740 1 Processor -- start 2024-09-10T08:10:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.784+0000 7fdfae9f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 0x5650a9b3e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.784+0000 7fdfae9f9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b349d0 0x5650a9b3e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.784+0000 7fdfae9f9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b39a60 0x5650a9b45810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.784+0000 7fdfae9f9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5650a9b3ca30 con 0x5650a9b39a60 2024-09-10T08:10:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.784+0000 7fdfae9f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5650a9a22090 con 0x5650a9b34090 2024-09-10T08:10:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.784+0000 7fdfae9f9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5650a9b3cbb0 con 0x5650a9b349d0 2024-09-10T08:10:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdfae9f9740 10 client.0 _ll_register_callbacks cb 0x5650a9a76f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdfa9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 0x5650a9b3e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdfa9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 0x5650a9b3e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37392/0 (socket says 172.21.15.2:37392) 2024-09-10T08:10:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdfa9c00640 1 -- 172.21.15.2:0/625947457 learned_addr learned my addr 172.21.15.2:0/625947457 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdfa3e00640 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b349d0 0x5650a9b3e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b39a60 0x5650a9b45810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdfa9c00640 1 -- 172.21.15.2:0/625947457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b349d0 msgr2=0x5650a9b3e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b349d0 0x5650a9b3e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdfa9c00640 1 -- 172.21.15.2:0/625947457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b39a60 msgr2=0x5650a9b45810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b39a60 0x5650a9b45810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdfa9c00640 1 -- 172.21.15.2:0/625947457 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a9878620 con 0x5650a9b34090 2024-09-10T08:10:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdfa3e00640 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b349d0 0x5650a9b3e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 0x5650a9b3e300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c02bea0 tx=0x7fdf9c011500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38913946aa4aa0d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.785+0000 7fdf8aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.786+0000 7fdf8aa00640 1 -- 172.21.15.2:0/625947457 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c004a40 con 0x5650a9b34090 2024-09-10T08:10:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.786+0000 7fdfae9f9740 1 -- 172.21.15.2:0/625947457 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf9c003d10 con 0x5650a9b34090 2024-09-10T08:10:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.786+0000 7fdf8aa00640 1 -- 172.21.15.2:0/625947457 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fdf9c004be0 con 0x5650a9b34090 2024-09-10T08:10:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.786+0000 7fdf8aa00640 1 -- 172.21.15.2:0/625947457 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf9c039050 con 0x5650a9b34090 2024-09-10T08:10:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.786+0000 7fdfae9f9740 10 client.26027 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.786+0000 7fdfae9f9740 1 -- 172.21.15.2:0/625947457 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5650a9b48600 con 0x5650a9b34090 2024-09-10T08:10:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.787+0000 7fdf8aa00640 1 -- 172.21.15.2:0/625947457 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf9c003d10 con 0x5650a9b34090 2024-09-10T08:10:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.787+0000 7fdf8aa00640 1 -- 172.21.15.2:0/625947457 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdf9c02b1f0 con 0x5650a9b34090 2024-09-10T08:10:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.787+0000 7fdfae9f9740 10 client.26027 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.787+0000 7fdfae9f9740 10 client.26027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.787+0000 7fdfae9f9740 20 client.26027 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.787+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5650a9b49a70 0x5650a9b4be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.787+0000 7fdfae9f9740 4 client.26027 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.787+0000 7fdfae9f9740 1 -- 172.21.15.2:0/625947457 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650a9aa6830 con 0x5650a9b49a70 2024-09-10T08:10:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.787+0000 7fdfa3e00640 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5650a9b49a70 0x5650a9b4be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.788+0000 7fdfa3e00640 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5650a9b49a70 0x5650a9b4be60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aedae2f80d6d443c server_cookie=83cb328a139b8c9f in_seq=0 out_seq=0 2024-09-10T08:10:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.788+0000 7fdf8aa00640 10 client.26027 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdf8aa00640 1 -- 172.21.15.2:0/625947457 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650a9aa6830 con 0x5650a9b49a70 2024-09-10T08:10:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdf8aa00640 10 client.26027 handle_command_reply: tid=0 2024-09-10T08:10:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdfae9f9740 1 client.26027 shutdown 2024-09-10T08:10:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdfae9f9740 1 -- 172.21.15.2:0/625947457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 msgr2=0x5650a9b3e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 0x5650a9b3e300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c02bea0 tx=0x7fdf9c011500 comp rx=0 tx=0).stop 2024-09-10T08:10:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdfae9f9740 1 -- 172.21.15.2:0/625947457 shutdown_connections 2024-09-10T08:10:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5650a9b49a70 0x5650a9b4be60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b39a60 0x5650a9b45810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b349d0 0x5650a9b3e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdf8aa00640 0 client.26027 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/625947457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b34090 0x5650a9b3e300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdfae9f9740 1 -- 172.21.15.2:0/625947457 >> 172.21.15.2:0/625947457 conn(0x5650a9b2f700 msgr2=0x5650a9b30f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdfae9f9740 1 -- 172.21.15.2:0/625947457 shutdown_connections 2024-09-10T08:10:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdfae9f9740 1 -- 172.21.15.2:0/625947457 wait complete. 2024-09-10T08:10:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.789+0000 7fdfae9f9740 20 client.26027 trim_cache size 0 max 16384 2024-09-10T08:10:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.804+0000 7fdfae9f9740 1 Processor -- start 2024-09-10T08:10:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.805+0000 7fdfae9f9740 1 -- start start 2024-09-10T08:10:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.805+0000 7fdfae9f9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b34090 0x5650a9b3e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.805+0000 7fdfae9f9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b349d0 0x5650a9b457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.805+0000 7fdfae9f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b39a60 0x5650a9b2e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.805+0000 7fdfae9f9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41aaf50 con 0x5650a9b34090 2024-09-10T08:10:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.805+0000 7fdfae9f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41a8820 con 0x5650a9b39a60 2024-09-10T08:10:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.805+0000 7fdfae9f9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41a6670 con 0x5650a9b349d0 2024-09-10T08:10:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.805+0000 7fdfa9c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b34090 0x5650a9b3e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.805+0000 7fdfa9c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b34090 0x5650a9b3e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40802/0 (socket says 172.21.15.2:40802) 2024-09-10T08:10:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.805+0000 7fdfa9c00640 1 -- 172.21.15.2:0/2888735296 learned_addr learned my addr 172.21.15.2:0/2888735296 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.805+0000 7fdfa3e00640 1 --2- 172.21.15.2:0/2888735296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b349d0 0x5650a9b457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.805+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/2888735296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b39a60 0x5650a9b2e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.805+0000 7fdfa9c00640 1 -- 172.21.15.2:0/2888735296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b349d0 msgr2=0x5650a9b457d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.806+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/2888735296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b349d0 0x5650a9b457d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.806+0000 7fdfa9c00640 1 -- 172.21.15.2:0/2888735296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b39a60 msgr2=0x5650a9b2e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.806+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/2888735296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b39a60 0x5650a9b2e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.806+0000 7fdfa9c00640 1 -- 172.21.15.2:0/2888735296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a9b4c1d0 con 0x5650a9b34090 2024-09-10T08:10:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.806+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/2888735296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b34090 0x5650a9b3e8d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c006010 tx=0x7fdf9c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee928251ef1e13c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.806+0000 7fdf95600640 1 -- 172.21.15.2:0/2888735296 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c010d60 con 0x5650a9b34090 2024-09-10T08:10:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.806+0000 7fdf95600640 1 -- 172.21.15.2:0/2888735296 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf9c029e40 con 0x5650a9b34090 2024-09-10T08:10:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.806+0000 7fdf95600640 1 -- 172.21.15.2:0/2888735296 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c005380 con 0x5650a9b34090 2024-09-10T08:10:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.807+0000 7fdfae9f9740 1 -- 172.21.15.2:0/2888735296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b34090 msgr2=0x5650a9b3e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.807+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/2888735296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b34090 0x5650a9b3e8d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c006010 tx=0x7fdf9c0021d0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.807+0000 7fdfae9f9740 1 -- 172.21.15.2:0/2888735296 shutdown_connections 2024-09-10T08:10:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.807+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/2888735296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b39a60 0x5650a9b2e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.807+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/2888735296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b349d0 0x5650a9b457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.807+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/2888735296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b34090 0x5650a9b3e8d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.807+0000 7fdfae9f9740 1 -- 172.21.15.2:0/2888735296 >> 172.21.15.2:0/2888735296 conn(0x5650a9b2f700 msgr2=0x5650a9b2e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.807+0000 7fdfae9f9740 1 -- 172.21.15.2:0/2888735296 shutdown_connections 2024-09-10T08:10:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.807+0000 7fdfae9f9740 1 -- 172.21.15.2:0/2888735296 wait complete. 2024-09-10T08:10:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.807+0000 7fdfae9f9740 1 -- start start 2024-09-10T08:10:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.808+0000 7fdfae9f9740 1 Processor -- start 2024-09-10T08:10:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.808+0000 7fdfae9f9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b34090 0x5650a9b45810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.808+0000 7fdfae9f9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b349d0 0x5650a9b422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.808+0000 7fdfae9f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b39a60 0x5650a9b42910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.808+0000 7fdfae9f9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41a6ce0 con 0x5650a9b349d0 2024-09-10T08:10:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.808+0000 7fdfae9f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41a8150 con 0x5650a9b39a60 2024-09-10T08:10:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.808+0000 7fdfae9f9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41a73b0 con 0x5650a9b34090 2024-09-10T08:10:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.808+0000 7fdfae9f9740 10 client.0 _ll_register_callbacks cb 0x7fdf980f5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.808+0000 7fdfa3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b349d0 0x5650a9b422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.808+0000 7fdfa9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b34090 0x5650a9b45810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.808+0000 7fdfa3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b349d0 0x5650a9b422b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40806/0 (socket says 172.21.15.2:40806) 2024-09-10T08:10:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.808+0000 7fdfa3e00640 1 -- 172.21.15.2:0/3579089053 learned_addr learned my addr 172.21.15.2:0/3579089053 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.809+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/3579089053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b39a60 0x5650a9b42910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.809+0000 7fdfa3e00640 1 -- 172.21.15.2:0/3579089053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b34090 msgr2=0x5650a9b45810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.809+0000 7fdfa3e00640 1 --2- 172.21.15.2:0/3579089053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b34090 0x5650a9b45810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.809+0000 7fdfa3e00640 1 -- 172.21.15.2:0/3579089053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b39a60 msgr2=0x5650a9b42910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.809+0000 7fdfa3e00640 1 --2- 172.21.15.2:0/3579089053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b39a60 0x5650a9b42910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.809+0000 7fdfa3e00640 1 -- 172.21.15.2:0/3579089053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a9bb2710 con 0x5650a9b349d0 2024-09-10T08:10:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.809+0000 7fdfa3e00640 1 --2- 172.21.15.2:0/3579089053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b349d0 0x5650a9b422b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009460 tx=0x7fdf90009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f0f08eccbfc40e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.809+0000 7fdf8aa00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.809+0000 7fdf8aa00640 1 -- 172.21.15.2:0/3579089053 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90024070 con 0x5650a9b349d0 2024-09-10T08:10:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.810+0000 7fdf8aa00640 1 -- 172.21.15.2:0/3579089053 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fdf900037f0 con 0x5650a9b349d0 2024-09-10T08:10:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.810+0000 7fdfae9f9740 1 -- 172.21.15.2:0/3579089053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5650a9aa0b80 con 0x5650a9b349d0 2024-09-10T08:10:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.810+0000 7fdf8aa00640 1 -- 172.21.15.2:0/3579089053 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf900039f0 con 0x5650a9b349d0 2024-09-10T08:10:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.810+0000 7fdfae9f9740 10 client.15999 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.810+0000 7fdfae9f9740 1 -- 172.21.15.2:0/3579089053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5650a9bb3a60 con 0x5650a9b349d0 2024-09-10T08:10:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.810+0000 7fdf8aa00640 1 -- 172.21.15.2:0/3579089053 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5650a9aa0b80 con 0x5650a9b349d0 2024-09-10T08:10:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.810+0000 7fdf8aa00640 1 -- 172.21.15.2:0/3579089053 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdf90011440 con 0x5650a9b349d0 2024-09-10T08:10:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.810+0000 7fdfae9f9740 10 client.15999 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.810+0000 7fdfae9f9740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.810+0000 7fdfae9f9740 20 client.15999 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.810+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/3579089053 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5650a9bb4460 0x5650a9bb6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.810+0000 7fdfae9f9740 4 client.15999 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:10:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.810+0000 7fdfae9f9740 1 -- 172.21.15.2:0/3579089053 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650a9a1d7c0 con 0x5650a9bb4460 2024-09-10T08:10:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.811+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/3579089053 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5650a9bb4460 0x5650a9bb6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.811+0000 7fdfa9c00640 1 --2- 172.21.15.2:0/3579089053 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5650a9bb4460 0x5650a9bb6880 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e42eabb9d7d6c0 server_cookie=13f0f0568ea2129c in_seq=0 out_seq=0 2024-09-10T08:10:41.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.811+0000 7fdf8aa00640 10 client.15999 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.823+0000 7fdf8aa00640 1 -- 172.21.15.2:0/3579089053 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650a9a1d7c0 con 0x5650a9bb4460 2024-09-10T08:10:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.823+0000 7fdf8aa00640 10 client.15999 handle_command_reply: tid=0 2024-09-10T08:10:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.823+0000 7fdfae9f9740 1 client.15999 shutdown 2024-09-10T08:10:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.823+0000 7fdfae9f9740 1 -- 172.21.15.2:0/3579089053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b349d0 msgr2=0x5650a9b422b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.823+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/3579089053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b349d0 0x5650a9b422b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009460 tx=0x7fdf90009310 comp rx=0 tx=0).stop 2024-09-10T08:10:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.823+0000 7fdfae9f9740 1 -- 172.21.15.2:0/3579089053 shutdown_connections 2024-09-10T08:10:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.823+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/3579089053 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5650a9bb4460 0x5650a9bb6880 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.824+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/3579089053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650a9b39a60 0x5650a9b42910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.824+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/3579089053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5650a9b349d0 0x5650a9b422b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.824+0000 7fdfae9f9740 1 --2- 172.21.15.2:0/3579089053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650a9b34090 0x5650a9b45810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.824+0000 7fdf8aa00640 0 client.15999 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.824+0000 7fdfae9f9740 1 -- 172.21.15.2:0/3579089053 >> 172.21.15.2:0/3579089053 conn(0x5650a9b2f700 msgr2=0x5650a9b3e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.824+0000 7fdfae9f9740 1 -- 172.21.15.2:0/3579089053 shutdown_connections 2024-09-10T08:10:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.824+0000 7fdfae9f9740 1 -- 172.21.15.2:0/3579089053 wait complete. 2024-09-10T08:10:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.824+0000 7fdfae9f9740 20 client.15999 trim_cache size 0 max 16384 2024-09-10T08:10:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:10:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:10:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:10:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:10:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.824+0000 7fdfabc00640 1 -- 172.21.15.2:0/3591106231 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdf8c0a2cb0 msgr2=0x7fdf8c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.824+0000 7fdfabc00640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdf8c0a2cb0 0x7fdf8c0a5170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdf980bba70 tx=0x7fdf980f5530 comp rx=0 tx=0).stop 2024-09-10T08:10:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.824+0000 7fdfabc00640 1 -- 172.21.15.2:0/3591106231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfa4101b60 msgr2=0x7fdfa419a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.825+0000 7fdfabc00640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfa4101b60 0x7fdfa419a020 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdf98031440 tx=0x7fdf98032ce0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.825+0000 7fdfabc00640 1 -- 172.21.15.2:0/3591106231 shutdown_connections 2024-09-10T08:10:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.825+0000 7fdfabc00640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdf8c0a2cb0 0x7fdf8c0a5170 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.825+0000 7fdfabc00640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa4103710 0x7fdfa41a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.825+0000 7fdfabc00640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa4102d50 0x7fdfa419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.825+0000 7fdfabc00640 1 --2- 172.21.15.2:0/3591106231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfa4101b60 0x7fdfa419a020 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.825+0000 7fdfabc00640 1 -- 172.21.15.2:0/3591106231 >> 172.21.15.2:0/3591106231 conn(0x7fdfa40fd2e0 msgr2=0x7fdfa4074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.825+0000 7fdfabc00640 1 -- 172.21.15.2:0/3591106231 shutdown_connections 2024-09-10T08:10:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:41.825+0000 7fdfabc00640 1 -- 172.21.15.2:0/3591106231 wait complete. 2024-09-10T08:10:42.049 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:42.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.627+0000 7f081f200640 1 -- 172.21.15.2:0/2092171006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0820102b90 msgr2=0x7f0820102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:42.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.627+0000 7f081f200640 1 --2- 172.21.15.2:0/2092171006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0820102b90 0x7f0820102f70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0810009860 tx=0x7f081002a3d0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.627+0000 7f081f200640 1 -- 172.21.15.2:0/2092171006 shutdown_connections 2024-09-10T08:10:42.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.627+0000 7f081f200640 1 --2- 172.21.15.2:0/2092171006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082010bfc0 0x7f082010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.627+0000 7f081f200640 1 --2- 172.21.15.2:0/2092171006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08201034b0 0x7f082010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.627+0000 7f081f200640 1 --2- 172.21.15.2:0/2092171006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0820102b90 0x7f0820102f70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.627+0000 7f081f200640 1 -- 172.21.15.2:0/2092171006 >> 172.21.15.2:0/2092171006 conn(0x7f08200fc610 msgr2=0x7f08200fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:42.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.627+0000 7f081f200640 1 -- 172.21.15.2:0/2092171006 shutdown_connections 2024-09-10T08:10:42.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.627+0000 7f081f200640 1 -- 172.21.15.2:0/2092171006 wait complete. 2024-09-10T08:10:42.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.628+0000 7f081f200640 1 Processor -- start 2024-09-10T08:10:42.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.628+0000 7f081f200640 1 -- start start 2024-09-10T08:10:42.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.628+0000 7f081f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0820102b90 0x7f082019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.628+0000 7f081f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08201034b0 0x7f082019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.628+0000 7f081f200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082010bfc0 0x7f082019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f081f200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0820111dc0 con 0x7f082010bfc0 2024-09-10T08:10:42.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f081f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0820111c40 con 0x7f0820102b90 2024-09-10T08:10:42.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f081f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0820111f40 con 0x7f08201034b0 2024-09-10T08:10:42.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f081de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0820102b90 0x7f082019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f081de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0820102b90 0x7f082019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37418/0 (socket says 172.21.15.2:37418) 2024-09-10T08:10:42.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f081de00640 1 -- 172.21.15.2:0/1688994905 learned_addr learned my addr 172.21.15.2:0/1688994905 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:42.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f081e800640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082010bfc0 0x7f082019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f0817e00640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08201034b0 0x7f082019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f081de00640 1 -- 172.21.15.2:0/1688994905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08201034b0 msgr2=0x7f082019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:42.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f081de00640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08201034b0 0x7f082019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f081de00640 1 -- 172.21.15.2:0/1688994905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082010bfc0 msgr2=0x7f082019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:42.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f081de00640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082010bfc0 0x7f082019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f081de00640 1 -- 172.21.15.2:0/1688994905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082019f450 con 0x7f0820102b90 2024-09-10T08:10:42.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.629+0000 7f0817e00640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08201034b0 0x7f082019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:42.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.630+0000 7f081de00640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0820102b90 0x7f082019a4a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f081002a8f0 tx=0x7f0810030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:42.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.630+0000 7f0816a00640 1 -- 172.21.15.2:0/1688994905 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810040020 con 0x7f0820102b90 2024-09-10T08:10:42.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.630+0000 7f081f200640 1 -- 172.21.15.2:0/1688994905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082019f6e0 con 0x7f0820102b90 2024-09-10T08:10:42.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.630+0000 7f0816a00640 1 -- 172.21.15.2:0/1688994905 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0810004460 con 0x7f0820102b90 2024-09-10T08:10:42.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.630+0000 7f0816a00640 1 -- 172.21.15.2:0/1688994905 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08100046c0 con 0x7f0820102b90 2024-09-10T08:10:42.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.630+0000 7f081f200640 1 -- 172.21.15.2:0/1688994905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08201a7020 con 0x7f0820102b90 2024-09-10T08:10:42.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.633+0000 7f082781e740 1 Processor -- start 2024-09-10T08:10:42.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.633+0000 7f082781e740 1 -- start start 2024-09-10T08:10:42.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.634+0000 7f082781e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258ae40 0x5576c258b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.634+0000 7f082781e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 0x5576c2595ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.634+0000 7f082781e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c2596020 0x5576c2598410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.634+0000 7f082781e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5576c24f13e0 con 0x5576c2596020 2024-09-10T08:10:42.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.634+0000 7f082781e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5576c2498d80 con 0x5576c258ae40 2024-09-10T08:10:42.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.634+0000 7f082781e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5576c24ed3a0 con 0x5576c258b760 2024-09-10T08:10:42.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.634+0000 7f0817e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c2596020 0x5576c2598410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.634+0000 7f081e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258ae40 0x5576c258b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.634+0000 7f0817e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c2596020 0x5576c2598410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40848/0 (socket says 172.21.15.2:40848) 2024-09-10T08:10:42.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.634+0000 7f0817e00640 1 -- 172.21.15.2:0/3894955046 learned_addr learned my addr 172.21.15.2:0/3894955046 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:42.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.634+0000 7f081e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258ae40 0x5576c258b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37426/0 (socket says 172.21.15.2:37426) 2024-09-10T08:10:42.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.635+0000 7f081e800640 1 -- 172.21.15.2:0/3894955046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 msgr2=0x5576c2595ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:10:42.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.635+0000 7f081e800640 1 --2- 172.21.15.2:0/3894955046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 0x5576c2595ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.635+0000 7f081e800640 1 -- 172.21.15.2:0/3894955046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c2596020 msgr2=0x5576c2598410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:42.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.635+0000 7f081e800640 1 --2- 172.21.15.2:0/3894955046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c2596020 0x5576c2598410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.635+0000 7f081e800640 1 -- 172.21.15.2:0/3894955046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576c24b7ae0 con 0x5576c258ae40 2024-09-10T08:10:42.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.635+0000 7f0816a00640 1 -- 172.21.15.2:0/1688994905 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0810043020 con 0x7f0820102b90 2024-09-10T08:10:42.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.635+0000 7f081e800640 1 --2- 172.21.15.2:0/3894955046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258ae40 0x5576c258b220 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0818009800 tx=0x7f08180095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f960a38a09a2b5d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:42.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.635+0000 7f080f000640 1 -- 172.21.15.2:0/3894955046 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818003970 con 0x5576c258ae40 2024-09-10T08:10:42.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.635+0000 7f080f000640 1 -- 172.21.15.2:0/3894955046 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f081800d6b0 con 0x5576c258ae40 2024-09-10T08:10:42.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.635+0000 7f080f000640 1 -- 172.21.15.2:0/3894955046 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818004370 con 0x5576c258ae40 2024-09-10T08:10:42.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.636+0000 7f082781e740 1 -- 172.21.15.2:0/3894955046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258ae40 msgr2=0x5576c258b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:42.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.636+0000 7f082781e740 1 --2- 172.21.15.2:0/3894955046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258ae40 0x5576c258b220 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0818009800 tx=0x7f08180095c0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.636+0000 7f0816a00640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f08040a2b80 0x7f08040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.637+0000 7f0816a00640 1 -- 172.21.15.2:0/1688994905 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f08100f0400 con 0x7f0820102b90 2024-09-10T08:10:42.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.637+0000 7f082781e740 1 -- 172.21.15.2:0/3894955046 shutdown_connections 2024-09-10T08:10:42.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.637+0000 7f082781e740 1 --2- 172.21.15.2:0/3894955046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c2596020 0x5576c2598410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.637+0000 7f082781e740 1 --2- 172.21.15.2:0/3894955046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 0x5576c2595ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.637+0000 7f082781e740 1 --2- 172.21.15.2:0/3894955046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258ae40 0x5576c258b220 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.637+0000 7f082781e740 1 -- 172.21.15.2:0/3894955046 >> 172.21.15.2:0/3894955046 conn(0x5576c25873d0 msgr2=0x5576c25877b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:42.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.637+0000 7f081de00640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f08040a2b80 0x7f08040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.637+0000 7f082781e740 1 -- 172.21.15.2:0/3894955046 shutdown_connections 2024-09-10T08:10:42.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.637+0000 7f082781e740 1 -- 172.21.15.2:0/3894955046 wait complete. 2024-09-10T08:10:42.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.638+0000 7f081de00640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f08040a2b80 0x7f08040a5040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f08100bc570 tx=0x7f0810004610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:42.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.638+0000 7f082781e740 1 -- start start 2024-09-10T08:10:42.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.638+0000 7f082781e740 1 Processor -- start 2024-09-10T08:10:42.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.638+0000 7f082781e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 0x5576c2592f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.638+0000 7f082781e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 0x5576c2593480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.638+0000 7f082781e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c2596020 0x5576c25939c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f082781e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5576c2598bf0 con 0x5576c258ae40 2024-09-10T08:10:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f082781e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5576c24ac0f0 con 0x5576c2596020 2024-09-10T08:10:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f082781e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5576c2598d70 con 0x5576c258b760 2024-09-10T08:10:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f082781e740 10 client.0 _ll_register_callbacks cb 0x5576c2493bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f081e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 0x5576c2592f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f0817e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 0x5576c2593480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f081e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 0x5576c2592f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40862/0 (socket says 172.21.15.2:40862) 2024-09-10T08:10:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f081e800640 1 -- 172.21.15.2:0/2959909593 learned_addr learned my addr 172.21.15.2:0/2959909593 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f081e800640 1 --2- 172.21.15.2:0/2959909593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c2596020 0x5576c25939c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f081e800640 1 -- 172.21.15.2:0/2959909593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 msgr2=0x5576c2593480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f081e800640 1 --2- 172.21.15.2:0/2959909593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 0x5576c2593480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f081e800640 1 -- 172.21.15.2:0/2959909593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c2596020 msgr2=0x5576c25939c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f081e800640 1 --2- 172.21.15.2:0/2959909593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c2596020 0x5576c25939c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.639+0000 7f081e800640 1 -- 172.21.15.2:0/2959909593 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576c22623c0 con 0x5576c258ae40 2024-09-10T08:10:42.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.640+0000 7f081e800640 1 --2- 172.21.15.2:0/2959909593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 0x5576c2592f40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f081800de10 tx=0x7f081800dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814afc1ebeb2cc7b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:42.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.640+0000 7f0803400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:42.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.640+0000 7f0803400640 1 -- 172.21.15.2:0/2959909593 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081802b020 con 0x5576c258ae40 2024-09-10T08:10:42.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.640+0000 7f0803400640 1 -- 172.21.15.2:0/2959909593 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0818003e30 con 0x5576c258ae40 2024-09-10T08:10:42.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.640+0000 7f082781e740 1 -- 172.21.15.2:0/2959909593 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f081800d6b0 con 0x5576c258ae40 2024-09-10T08:10:42.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.641+0000 7f0803400640 1 -- 172.21.15.2:0/2959909593 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0818033050 con 0x5576c258ae40 2024-09-10T08:10:42.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.641+0000 7f082781e740 10 client.16011 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:42.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.641+0000 7f082781e740 1 -- 172.21.15.2:0/2959909593 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576c25a4c60 con 0x5576c258ae40 2024-09-10T08:10:42.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.641+0000 7f0803400640 1 -- 172.21.15.2:0/2959909593 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f081800d6b0 con 0x5576c258ae40 2024-09-10T08:10:42.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.642+0000 7f0803400640 1 -- 172.21.15.2:0/2959909593 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f081800b040 con 0x5576c258ae40 2024-09-10T08:10:42.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.642+0000 7f082781e740 10 client.16011 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:42.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.642+0000 7f082781e740 10 client.16011 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:10:42.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.642+0000 7f082781e740 20 client.16011 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:42.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.642+0000 7f082781e740 1 --2- 172.21.15.2:0/2959909593 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5576c25a6010 0x5576c25a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.642+0000 7f082781e740 4 client.16011 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:42.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.642+0000 7f082781e740 1 -- 172.21.15.2:0/2959909593 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576c24fe950 con 0x5576c25a6010 2024-09-10T08:10:42.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.642+0000 7f0817e00640 1 --2- 172.21.15.2:0/2959909593 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5576c25a6010 0x5576c25a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.643+0000 7f0817e00640 1 --2- 172.21.15.2:0/2959909593 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5576c25a6010 0x5576c25a8400 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=146379ca4c142a9a server_cookie=cd54092e5b00d60c in_seq=0 out_seq=0 2024-09-10T08:10:42.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.643+0000 7f0803400640 10 client.16011 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:42.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.644+0000 7f0803400640 1 -- 172.21.15.2:0/2959909593 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576c24fe950 con 0x5576c25a6010 2024-09-10T08:10:42.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.644+0000 7f0803400640 10 client.16011 handle_command_reply: tid=0 2024-09-10T08:10:42.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.644+0000 7f082781e740 1 client.16011 shutdown 2024-09-10T08:10:42.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.644+0000 7f082781e740 1 -- 172.21.15.2:0/2959909593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 msgr2=0x5576c2592f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:42.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.644+0000 7f082781e740 1 --2- 172.21.15.2:0/2959909593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 0x5576c2592f40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f081800de10 tx=0x7f081800dd80 comp rx=0 tx=0).stop 2024-09-10T08:10:42.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.644+0000 7f082781e740 1 -- 172.21.15.2:0/2959909593 shutdown_connections 2024-09-10T08:10:42.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.644+0000 7f082781e740 1 --2- 172.21.15.2:0/2959909593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c2596020 0x5576c25939c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.644+0000 7f082781e740 1 --2- 172.21.15.2:0/2959909593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 0x5576c2593480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.644+0000 7f082781e740 1 --2- 172.21.15.2:0/2959909593 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5576c25a6010 0x5576c25a8400 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.644+0000 7f082781e740 1 --2- 172.21.15.2:0/2959909593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 0x5576c2592f40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.644+0000 7f082781e740 1 -- 172.21.15.2:0/2959909593 >> 172.21.15.2:0/2959909593 conn(0x5576c25873d0 msgr2=0x5576c258d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:42.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.644+0000 7f0803400640 0 client.16011 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:42.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.645+0000 7f082781e740 1 -- 172.21.15.2:0/2959909593 shutdown_connections 2024-09-10T08:10:42.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.645+0000 7f082781e740 1 -- 172.21.15.2:0/2959909593 wait complete. 2024-09-10T08:10:42.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.645+0000 7f082781e740 20 client.16011 trim_cache size 0 max 16384 2024-09-10T08:10:42.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.664+0000 7f082781e740 1 Processor -- start 2024-09-10T08:10:42.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.664+0000 7f082781e740 1 -- start start 2024-09-10T08:10:42.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.664+0000 7f082781e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258ae40 0x5576c25ab420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.664+0000 7f082781e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 0x5576c25ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.664+0000 7f082781e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c2596020 0x5576c2590200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.664+0000 7f082781e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f08201b3870 con 0x5576c2596020 2024-09-10T08:10:42.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.664+0000 7f082781e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f08201b1140 con 0x5576c258ae40 2024-09-10T08:10:42.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.664+0000 7f082781e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f08201aef90 con 0x5576c258b760 2024-09-10T08:10:42.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.664+0000 7f0817e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 0x5576c25ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.665+0000 7f0817e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 0x5576c25ab960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36674/0 (socket says 172.21.15.2:36674) 2024-09-10T08:10:42.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.665+0000 7f0817e00640 1 -- 172.21.15.2:0/2476698766 learned_addr learned my addr 172.21.15.2:0/2476698766 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:42.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.665+0000 7f081e800640 1 --2- 172.21.15.2:0/2476698766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258ae40 0x5576c25ab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.665+0000 7f081e800640 1 --2- 172.21.15.2:0/2476698766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c2596020 0x5576c2590200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.665+0000 7f0817e00640 1 -- 172.21.15.2:0/2476698766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258ae40 msgr2=0x5576c25ab420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.665+0000 7f0817e00640 1 --2- 172.21.15.2:0/2476698766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258ae40 0x5576c25ab420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.665+0000 7f0817e00640 1 -- 172.21.15.2:0/2476698766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c2596020 msgr2=0x5576c2590200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.665+0000 7f0817e00640 1 --2- 172.21.15.2:0/2476698766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c2596020 0x5576c2590200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.665+0000 7f0817e00640 1 -- 172.21.15.2:0/2476698766 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576c25914d0 con 0x5576c258b760 2024-09-10T08:10:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.665+0000 7f081e800640 1 --2- 172.21.15.2:0/2476698766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258ae40 0x5576c25ab420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.665+0000 7f0817e00640 1 --2- 172.21.15.2:0/2476698766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 0x5576c25ab960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5576c25927d0 tx=0x7f0808005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c694bccb83a3f77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.665+0000 7f080e600640 1 -- 172.21.15.2:0/2476698766 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080801e6e0 con 0x5576c258b760 2024-09-10T08:10:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.666+0000 7f080e600640 1 -- 172.21.15.2:0/2476698766 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f080801ed60 con 0x5576c258b760 2024-09-10T08:10:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.666+0000 7f080e600640 1 -- 172.21.15.2:0/2476698766 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08080263d0 con 0x5576c258b760 2024-09-10T08:10:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.666+0000 7f082781e740 1 -- 172.21.15.2:0/2476698766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 msgr2=0x5576c25ab960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.666+0000 7f082781e740 1 --2- 172.21.15.2:0/2476698766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 0x5576c25ab960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5576c25927d0 tx=0x7f0808005f90 comp rx=0 tx=0).stop 2024-09-10T08:10:42.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.667+0000 7f082781e740 1 -- 172.21.15.2:0/2476698766 shutdown_connections 2024-09-10T08:10:42.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.667+0000 7f082781e740 1 --2- 172.21.15.2:0/2476698766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c2596020 0x5576c2590200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.667+0000 7f082781e740 1 --2- 172.21.15.2:0/2476698766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c258b760 0x5576c25ab960 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.667+0000 7f082781e740 1 --2- 172.21.15.2:0/2476698766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258ae40 0x5576c25ab420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.667+0000 7f082781e740 1 -- 172.21.15.2:0/2476698766 >> 172.21.15.2:0/2476698766 conn(0x5576c25873d0 msgr2=0x5576c2594fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:42.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.667+0000 7f082781e740 1 -- 172.21.15.2:0/2476698766 shutdown_connections 2024-09-10T08:10:42.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.667+0000 7f082781e740 1 -- 172.21.15.2:0/2476698766 wait complete. 2024-09-10T08:10:42.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.667+0000 7f082781e740 1 -- start start 2024-09-10T08:10:42.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.667+0000 7f082781e740 1 Processor -- start 2024-09-10T08:10:42.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.668+0000 7f082781e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 0x5576c25abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.668+0000 7f082781e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258b760 0x5576c25a9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.668+0000 7f082781e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c2596020 0x5576c25a9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.668+0000 7f082781e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f08201af600 con 0x5576c258ae40 2024-09-10T08:10:42.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.668+0000 7f082781e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f08201b0a70 con 0x5576c258b760 2024-09-10T08:10:42.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.668+0000 7f082781e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f08201afcd0 con 0x5576c2596020 2024-09-10T08:10:42.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.668+0000 7f082781e740 10 client.0 _ll_register_callbacks cb 0x7f082006afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.668+0000 7f081e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 0x5576c25abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.668+0000 7f081e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 0x5576c25abcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40880/0 (socket says 172.21.15.2:40880) 2024-09-10T08:10:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.668+0000 7f081e800640 1 -- 172.21.15.2:0/3810138409 learned_addr learned my addr 172.21.15.2:0/3810138409 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.668+0000 7f0817e00640 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258b760 0x5576c25a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.668+0000 7f081e800640 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c2596020 0x5576c25a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.669+0000 7f081e800640 1 -- 172.21.15.2:0/3810138409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c2596020 msgr2=0x5576c25a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.669+0000 7f081e800640 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c2596020 0x5576c25a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.669+0000 7f081e800640 1 -- 172.21.15.2:0/3810138409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258b760 msgr2=0x5576c25a9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.669+0000 7f081e800640 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258b760 0x5576c25a9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.669+0000 7f081e800640 1 -- 172.21.15.2:0/3810138409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576c25a9e90 con 0x5576c258ae40 2024-09-10T08:10:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.669+0000 7f0817e00640 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258b760 0x5576c25a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.669+0000 7f081e800640 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 0x5576c25abcb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f08180178e0 tx=0x7f081800d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a65423672286a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.669+0000 7f0803400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.669+0000 7f0803400640 1 -- 172.21.15.2:0/3810138409 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081802b040 con 0x5576c258ae40 2024-09-10T08:10:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.669+0000 7f082781e740 1 -- 172.21.15.2:0/3810138409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f080801ed60 con 0x5576c258ae40 2024-09-10T08:10:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.669+0000 7f0803400640 1 -- 172.21.15.2:0/3810138409 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f081800de40 con 0x5576c258ae40 2024-09-10T08:10:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.669+0000 7f0803400640 1 -- 172.21.15.2:0/3810138409 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f08180097e0 con 0x5576c258ae40 2024-09-10T08:10:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.670+0000 7f082781e740 10 client.16017 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.670+0000 7f082781e740 1 -- 172.21.15.2:0/3810138409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576c2610a80 con 0x5576c258ae40 2024-09-10T08:10:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.670+0000 7f0803400640 1 -- 172.21.15.2:0/3810138409 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f080801ed60 con 0x5576c258ae40 2024-09-10T08:10:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.670+0000 7f0803400640 1 -- 172.21.15.2:0/3810138409 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f081800b040 con 0x5576c258ae40 2024-09-10T08:10:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.670+0000 7f082781e740 10 client.16017 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.670+0000 7f082781e740 10 client.16017 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:10:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.670+0000 7f082781e740 20 client.16017 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.670+0000 7f082781e740 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5576c2611080 0x5576c2613470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.670+0000 7f082781e740 4 client.16017 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:10:42.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.670+0000 7f082781e740 1 -- 172.21.15.2:0/3810138409 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576c24f8e60 con 0x5576c2611080 2024-09-10T08:10:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.671+0000 7f0817e00640 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5576c2611080 0x5576c2613470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.671+0000 7f0817e00640 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5576c2611080 0x5576c2613470 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=275aa0f840921b5a server_cookie=816090f550199612 in_seq=0 out_seq=0 2024-09-10T08:10:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.671+0000 7f0803400640 10 client.16017 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.671+0000 7f0803400640 1 -- 172.21.15.2:0/3810138409 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576c24f8e60 con 0x5576c2611080 2024-09-10T08:10:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.671+0000 7f0803400640 10 client.16017 handle_command_reply: tid=0 2024-09-10T08:10:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.671+0000 7f082781e740 1 client.16017 shutdown 2024-09-10T08:10:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.672+0000 7f082781e740 1 -- 172.21.15.2:0/3810138409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 msgr2=0x5576c25abcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.672+0000 7f082781e740 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 0x5576c25abcb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f08180178e0 tx=0x7f081800d8e0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.672+0000 7f082781e740 1 -- 172.21.15.2:0/3810138409 shutdown_connections 2024-09-10T08:10:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.672+0000 7f082781e740 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576c2596020 0x5576c25a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.672+0000 7f082781e740 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576c258b760 0x5576c25a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.672+0000 7f082781e740 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5576c2611080 0x5576c2613470 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.672+0000 7f082781e740 1 --2- 172.21.15.2:0/3810138409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576c258ae40 0x5576c25abcb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.672+0000 7f082781e740 1 -- 172.21.15.2:0/3810138409 >> 172.21.15.2:0/3810138409 conn(0x5576c25873d0 msgr2=0x5576c25a0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:42.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.672+0000 7f0803400640 0 client.16017 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:10:42.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.672+0000 7f082781e740 1 -- 172.21.15.2:0/3810138409 shutdown_connections 2024-09-10T08:10:42.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.672+0000 7f082781e740 1 -- 172.21.15.2:0/3810138409 wait complete. 2024-09-10T08:10:42.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.672+0000 7f082781e740 20 client.16017 trim_cache size 0 max 16384 2024-09-10T08:10:42.675 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:10:42.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.673+0000 7f081f200640 1 -- 172.21.15.2:0/1688994905 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f08040a2b80 msgr2=0x7f08040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:42.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.673+0000 7f081f200640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f08040a2b80 0x7f08040a5040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f08100bc570 tx=0x7f0810004610 comp rx=0 tx=0).stop 2024-09-10T08:10:42.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.673+0000 7f081f200640 1 -- 172.21.15.2:0/1688994905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0820102b90 msgr2=0x7f082019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:42.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.673+0000 7f081f200640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0820102b90 0x7f082019a4a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f081002a8f0 tx=0x7f0810030650 comp rx=0 tx=0).stop 2024-09-10T08:10:42.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.673+0000 7f081f200640 1 -- 172.21.15.2:0/1688994905 shutdown_connections 2024-09-10T08:10:42.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.673+0000 7f081f200640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082010bfc0 0x7f082019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.673+0000 7f081f200640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f08040a2b80 0x7f08040a5040 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.673+0000 7f081f200640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08201034b0 0x7f082019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.673+0000 7f081f200640 1 --2- 172.21.15.2:0/1688994905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0820102b90 0x7f082019a4a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:42.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.673+0000 7f081f200640 1 -- 172.21.15.2:0/1688994905 >> 172.21.15.2:0/1688994905 conn(0x7f08200fc610 msgr2=0x7f08200fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:42.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.673+0000 7f081f200640 1 -- 172.21.15.2:0/1688994905 shutdown_connections 2024-09-10T08:10:42.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:42.673+0000 7f081f200640 1 -- 172.21.15.2:0/1688994905 wait complete. 2024-09-10T08:10:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:43 smithi002 ceph-mon[30563]: pgmap v464: 129 pgs: 129 active+clean; 949 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:10:43.621 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:10:43.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:43 smithi155 ceph-mon[35405]: pgmap v464: 129 pgs: 129 active+clean; 949 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:10:43.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:43 smithi107 ceph-mon[35048]: pgmap v464: 129 pgs: 129 active+clean; 949 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:10:44.052 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:44.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.815+0000 7f398ee00640 1 -- 172.21.15.2:0/3904605440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3988074f40 msgr2=0x7f398810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:44.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.815+0000 7f398ee00640 1 --2- 172.21.15.2:0/3904605440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3988074f40 0x7f398810de80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3970009900 tx=0x7f397002a480 comp rx=0 tx=0).stop 2024-09-10T08:10:44.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.815+0000 7f398ee00640 1 -- 172.21.15.2:0/3904605440 shutdown_connections 2024-09-10T08:10:44.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.815+0000 7f398ee00640 1 --2- 172.21.15.2:0/3904605440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f398810e3c0 0x7f3988110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.815+0000 7f398ee00640 1 --2- 172.21.15.2:0/3904605440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3988074f40 0x7f398810de80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.815+0000 7f398ee00640 1 --2- 172.21.15.2:0/3904605440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3988074620 0x7f3988074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.815+0000 7f398ee00640 1 -- 172.21.15.2:0/3904605440 >> 172.21.15.2:0/3904605440 conn(0x7f39880fd8c0 msgr2=0x7f39880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:44.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.815+0000 7f398ee00640 1 -- 172.21.15.2:0/3904605440 shutdown_connections 2024-09-10T08:10:44.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.815+0000 7f398ee00640 1 -- 172.21.15.2:0/3904605440 wait complete. 2024-09-10T08:10:44.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.816+0000 7f398ee00640 1 Processor -- start 2024-09-10T08:10:44.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.817+0000 7f398ee00640 1 -- start start 2024-09-10T08:10:44.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.817+0000 7f398ee00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3988074620 0x7f398819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.817+0000 7f398ee00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3988074f40 0x7f398819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.817+0000 7f398ee00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f398810e3c0 0x7f398819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.817+0000 7f398ee00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3988114160 con 0x7f3988074f40 2024-09-10T08:10:44.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.817+0000 7f398ee00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3988113fe0 con 0x7f398810e3c0 2024-09-10T08:10:44.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.817+0000 7f398ee00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39881142e0 con 0x7f3988074620 2024-09-10T08:10:44.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.817+0000 7f397f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3988074f40 0x7f398819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.817+0000 7f3987e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3988074620 0x7f398819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.818+0000 7f3987e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3988074620 0x7f398819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36692/0 (socket says 172.21.15.2:36692) 2024-09-10T08:10:44.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.818+0000 7f3987e00640 1 -- 172.21.15.2:0/2676714304 learned_addr learned my addr 172.21.15.2:0/2676714304 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:44.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.818+0000 7f398ce00640 1 --2- 172.21.15.2:0/2676714304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f398810e3c0 0x7f398819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.818+0000 7f3987e00640 1 -- 172.21.15.2:0/2676714304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f398810e3c0 msgr2=0x7f398819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:44.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.818+0000 7f3987e00640 1 --2- 172.21.15.2:0/2676714304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f398810e3c0 0x7f398819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.818+0000 7f3987e00640 1 -- 172.21.15.2:0/2676714304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3988074f40 msgr2=0x7f398819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:44.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.818+0000 7f3987e00640 1 --2- 172.21.15.2:0/2676714304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3988074f40 0x7f398819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.818+0000 7f3987e00640 1 -- 172.21.15.2:0/2676714304 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398819f660 con 0x7f3988074620 2024-09-10T08:10:44.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.818+0000 7f3987e00640 1 --2- 172.21.15.2:0/2676714304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3988074620 0x7f398819a6b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f397800e9f0 tx=0x7f397800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:44.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.819+0000 7f3985600640 1 -- 172.21.15.2:0/2676714304 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397800cd80 con 0x7f3988074620 2024-09-10T08:10:44.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.819+0000 7f3985600640 1 -- 172.21.15.2:0/2676714304 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3978005500 con 0x7f3988074620 2024-09-10T08:10:44.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.819+0000 7f3985600640 1 -- 172.21.15.2:0/2676714304 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978010690 con 0x7f3988074620 2024-09-10T08:10:44.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.819+0000 7f398ee00640 1 -- 172.21.15.2:0/2676714304 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398819f950 con 0x7f3988074620 2024-09-10T08:10:44.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.819+0000 7f398ee00640 1 -- 172.21.15.2:0/2676714304 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39881a7230 con 0x7f3988074620 2024-09-10T08:10:44.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.822+0000 7f3991a68740 1 Processor -- start 2024-09-10T08:10:44.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.822+0000 7f3991a68740 1 -- start start 2024-09-10T08:10:44.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.822+0000 7f3985600640 1 -- 172.21.15.2:0/2676714304 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f39780046c0 con 0x7f3988074620 2024-09-10T08:10:44.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.822+0000 7f3991a68740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199080c70 0x558199081050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.822+0000 7f3991a68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199081590 0x55819908b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.822+0000 7f3991a68740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x55819908e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.822+0000 7f3991a68740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558198fe73e0 con 0x558199080c70 2024-09-10T08:10:44.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.822+0000 7f3991a68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558198f8ed80 con 0x558199081590 2024-09-10T08:10:44.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.822+0000 7f3991a68740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558198fe33a0 con 0x55819908be50 2024-09-10T08:10:44.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.822+0000 7f398ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199080c70 0x558199081050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.823+0000 7f398ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199080c70 0x558199081050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40906/0 (socket says 172.21.15.2:40906) 2024-09-10T08:10:44.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.823+0000 7f398ce00640 1 -- 172.21.15.2:0/1760870961 learned_addr learned my addr 172.21.15.2:0/1760870961 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:44.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.823+0000 7f3987e00640 1 --2- 172.21.15.2:0/1760870961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199081590 0x55819908b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.823+0000 7f397f400640 1 --2- 172.21.15.2:0/1760870961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x55819908e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.823+0000 7f398ce00640 1 -- 172.21.15.2:0/1760870961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 msgr2=0x55819908e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:44.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.823+0000 7f398ce00640 1 --2- 172.21.15.2:0/1760870961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x55819908e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.823+0000 7f398ce00640 1 -- 172.21.15.2:0/1760870961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199081590 msgr2=0x55819908b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:44.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.823+0000 7f398ce00640 1 --2- 172.21.15.2:0/1760870961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199081590 0x55819908b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.823+0000 7f398ce00640 1 -- 172.21.15.2:0/1760870961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558198fadae0 con 0x558199080c70 2024-09-10T08:10:44.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.823+0000 7f3987e00640 1 --2- 172.21.15.2:0/1760870961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199081590 0x55819908b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:44.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.823+0000 7f398ce00640 1 --2- 172.21.15.2:0/1760870961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199080c70 0x558199081050 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3980009f50 tx=0x7f39800118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c9889e812d008e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:44.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.824+0000 7f397d600640 1 -- 172.21.15.2:0/1760870961 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980004d20 con 0x558199080c70 2024-09-10T08:10:44.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.824+0000 7f397d600640 1 -- 172.21.15.2:0/1760870961 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3980004050 con 0x558199080c70 2024-09-10T08:10:44.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.824+0000 7f397d600640 1 -- 172.21.15.2:0/1760870961 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980005260 con 0x558199080c70 2024-09-10T08:10:44.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.824+0000 7f3985600640 1 --2- 172.21.15.2:0/2676714304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f396c0a2c00 0x7f396c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.824+0000 7f3991a68740 1 -- 172.21.15.2:0/1760870961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199080c70 msgr2=0x558199081050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:44.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.824+0000 7f3991a68740 1 --2- 172.21.15.2:0/1760870961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199080c70 0x558199081050 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3980009f50 tx=0x7f39800118d0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.824+0000 7f3985600640 1 -- 172.21.15.2:0/2676714304 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f39780cf810 con 0x7f3988074620 2024-09-10T08:10:44.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.825+0000 7f3987e00640 1 --2- 172.21.15.2:0/2676714304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f396c0a2c00 0x7f396c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.825+0000 7f3991a68740 1 -- 172.21.15.2:0/1760870961 shutdown_connections 2024-09-10T08:10:44.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.825+0000 7f3991a68740 1 --2- 172.21.15.2:0/1760870961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x55819908e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.825+0000 7f3991a68740 1 --2- 172.21.15.2:0/1760870961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199081590 0x55819908b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.825+0000 7f3991a68740 1 --2- 172.21.15.2:0/1760870961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199080c70 0x558199081050 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.825+0000 7f3991a68740 1 -- 172.21.15.2:0/1760870961 >> 172.21.15.2:0/1760870961 conn(0x55819907d200 msgr2=0x55819907d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:44.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.825+0000 7f3991a68740 1 -- 172.21.15.2:0/1760870961 shutdown_connections 2024-09-10T08:10:44.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.825+0000 7f3991a68740 1 -- 172.21.15.2:0/1760870961 wait complete. 2024-09-10T08:10:44.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.825+0000 7f3987e00640 1 --2- 172.21.15.2:0/2676714304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f396c0a2c00 0x7f396c0a50c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f397809bbb0 tx=0x7f3978004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:44.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.825+0000 7f3991a68740 1 -- start start 2024-09-10T08:10:44.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.826+0000 7f3991a68740 1 Processor -- start 2024-09-10T08:10:44.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.826+0000 7f3991a68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199080c70 0x558199088ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.826+0000 7f3991a68740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 0x558199089220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.826+0000 7f3991a68740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x558199089760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.826+0000 7f3991a68740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55819908ea20 con 0x558199081590 2024-09-10T08:10:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.826+0000 7f3991a68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558198fa20f0 con 0x558199080c70 2024-09-10T08:10:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.826+0000 7f3991a68740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55819908eba0 con 0x55819908be50 2024-09-10T08:10:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.826+0000 7f3991a68740 10 client.0 _ll_register_callbacks cb 0x558198f89bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.826+0000 7f397f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 0x558199089220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.826+0000 7f397f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 0x558199089220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40910/0 (socket says 172.21.15.2:40910) 2024-09-10T08:10:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.826+0000 7f397f400640 1 -- 172.21.15.2:0/2784536510 learned_addr learned my addr 172.21.15.2:0/2784536510 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.827+0000 7f398ce00640 1 --2- 172.21.15.2:0/2784536510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199080c70 0x558199088ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.827+0000 7f398ce00640 1 --2- 172.21.15.2:0/2784536510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x558199089760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.827+0000 7f397f400640 1 -- 172.21.15.2:0/2784536510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 msgr2=0x558199089760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.827+0000 7f397f400640 1 --2- 172.21.15.2:0/2784536510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x558199089760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.827+0000 7f397f400640 1 -- 172.21.15.2:0/2784536510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199080c70 msgr2=0x558199088ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:44.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.827+0000 7f397f400640 1 --2- 172.21.15.2:0/2784536510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199080c70 0x558199088ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.827+0000 7f397f400640 1 -- 172.21.15.2:0/2784536510 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558198d583c0 con 0x558199081590 2024-09-10T08:10:44.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.827+0000 7f397f400640 1 --2- 172.21.15.2:0/2784536510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 0x558199089220 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f397002a9a0 tx=0x7f39700317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fdf55b79f914f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:44.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.827+0000 7f3976000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:44.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.828+0000 7f3976000640 1 -- 172.21.15.2:0/2784536510 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970035040 con 0x558199081590 2024-09-10T08:10:44.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.828+0000 7f3976000640 1 -- 172.21.15.2:0/2784536510 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3970004240 con 0x558199081590 2024-09-10T08:10:44.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.828+0000 7f3991a68740 1 -- 172.21.15.2:0/2784536510 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3980004050 con 0x558199081590 2024-09-10T08:10:44.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.828+0000 7f3976000640 1 -- 172.21.15.2:0/2784536510 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3970004440 con 0x558199081590 2024-09-10T08:10:44.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.828+0000 7f3991a68740 10 client.16029 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:44.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.828+0000 7f3991a68740 1 -- 172.21.15.2:0/2784536510 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55819909a9f0 con 0x558199081590 2024-09-10T08:10:44.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.828+0000 7f3976000640 1 -- 172.21.15.2:0/2784536510 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3980004050 con 0x558199081590 2024-09-10T08:10:44.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.828+0000 7f3976000640 1 -- 172.21.15.2:0/2784536510 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f397003d570 con 0x558199081590 2024-09-10T08:10:44.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.829+0000 7f3991a68740 10 client.16029 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.829+0000 7f3991a68740 10 client.16029 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:10:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.829+0000 7f3991a68740 20 client.16029 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.829+0000 7f3991a68740 1 --2- 172.21.15.2:0/2784536510 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55819909beb0 0x55819909e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.829+0000 7f3991a68740 4 client.16029 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.829+0000 7f3991a68740 1 -- 172.21.15.2:0/2784536510 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558198ff4950 con 0x55819909beb0 2024-09-10T08:10:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.829+0000 7f398ce00640 1 --2- 172.21.15.2:0/2784536510 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55819909beb0 0x55819909e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.830+0000 7f398ce00640 1 --2- 172.21.15.2:0/2784536510 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55819909beb0 0x55819909e2e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2eb420f3ad21dc3 server_cookie=5b5e66a437d13d5f in_seq=0 out_seq=0 2024-09-10T08:10:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.830+0000 7f3976000640 10 client.16029 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:44.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.830+0000 7f3976000640 1 -- 172.21.15.2:0/2784536510 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558198ff4950 con 0x55819909beb0 2024-09-10T08:10:44.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.830+0000 7f3976000640 10 client.16029 handle_command_reply: tid=0 2024-09-10T08:10:44.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.830+0000 7f3991a68740 1 client.16029 shutdown 2024-09-10T08:10:44.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.831+0000 7f3991a68740 1 -- 172.21.15.2:0/2784536510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 msgr2=0x558199089220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:44.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.831+0000 7f3991a68740 1 --2- 172.21.15.2:0/2784536510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 0x558199089220 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f397002a9a0 tx=0x7f39700317e0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.831+0000 7f3991a68740 1 -- 172.21.15.2:0/2784536510 shutdown_connections 2024-09-10T08:10:44.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.831+0000 7f3991a68740 1 --2- 172.21.15.2:0/2784536510 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55819909beb0 0x55819909e2e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.831+0000 7f3991a68740 1 --2- 172.21.15.2:0/2784536510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x558199089760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.831+0000 7f3991a68740 1 --2- 172.21.15.2:0/2784536510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 0x558199089220 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.831+0000 7f3991a68740 1 --2- 172.21.15.2:0/2784536510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199080c70 0x558199088ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.831+0000 7f3976000640 0 client.16029 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:44.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.831+0000 7f3991a68740 1 -- 172.21.15.2:0/2784536510 >> 172.21.15.2:0/2784536510 conn(0x55819907d200 msgr2=0x55819907c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:44.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.831+0000 7f3991a68740 1 -- 172.21.15.2:0/2784536510 shutdown_connections 2024-09-10T08:10:44.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.831+0000 7f3991a68740 1 -- 172.21.15.2:0/2784536510 wait complete. 2024-09-10T08:10:44.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.831+0000 7f3991a68740 20 client.16029 trim_cache size 0 max 16384 2024-09-10T08:10:44.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.845+0000 7f3991a68740 1 Processor -- start 2024-09-10T08:10:44.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.845+0000 7f3991a68740 1 -- start start 2024-09-10T08:10:44.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.845+0000 7f3991a68740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199080c70 0x55819908ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.845+0000 7f3991a68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199081590 0x5581990a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.845+0000 7f3991a68740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x5581990a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.845+0000 7f3991a68740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f39881b3940 con 0x558199080c70 2024-09-10T08:10:44.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.845+0000 7f3991a68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39881b1210 con 0x558199081590 2024-09-10T08:10:44.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.845+0000 7f3991a68740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39881af060 con 0x55819908be50 2024-09-10T08:10:44.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.846+0000 7f398ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199080c70 0x55819908ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.846+0000 7f397f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199081590 0x5581990a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.846+0000 7f397f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199081590 0x5581990a3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37516/0 (socket says 172.21.15.2:37516) 2024-09-10T08:10:44.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.846+0000 7f397f400640 1 -- 172.21.15.2:0/3297605881 learned_addr learned my addr 172.21.15.2:0/3297605881 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:44.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.846+0000 7f398ce00640 1 --2- 172.21.15.2:0/3297605881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x5581990a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.846+0000 7f397f400640 1 -- 172.21.15.2:0/3297605881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 msgr2=0x5581990a40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:44.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.846+0000 7f397f400640 1 --2- 172.21.15.2:0/3297605881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x5581990a40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.846+0000 7f397f400640 1 -- 172.21.15.2:0/3297605881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199080c70 msgr2=0x55819908ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:44.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.846+0000 7f397f400640 1 --2- 172.21.15.2:0/3297605881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199080c70 0x55819908ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.846+0000 7f397f400640 1 -- 172.21.15.2:0/3297605881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55819907e2b0 con 0x558199081590 2024-09-10T08:10:44.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.846+0000 7f398ce00640 1 --2- 172.21.15.2:0/3297605881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199080c70 0x55819908ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:44.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.847+0000 7f397f400640 1 --2- 172.21.15.2:0/3297605881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199081590 0x5581990a3bb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f397003db30 tx=0x7f3970009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6138f8fe41e39f64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:44.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.847+0000 7f397cc00640 1 -- 172.21.15.2:0/3297605881 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397005b070 con 0x558199081590 2024-09-10T08:10:44.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.847+0000 7f397cc00640 1 -- 172.21.15.2:0/3297605881 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3970005520 con 0x558199081590 2024-09-10T08:10:44.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.847+0000 7f397cc00640 1 -- 172.21.15.2:0/3297605881 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970035030 con 0x558199081590 2024-09-10T08:10:44.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.847+0000 7f3991a68740 1 -- 172.21.15.2:0/3297605881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199081590 msgr2=0x5581990a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:44.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.847+0000 7f3991a68740 1 --2- 172.21.15.2:0/3297605881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199081590 0x5581990a3bb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f397003db30 tx=0x7f3970009680 comp rx=0 tx=0).stop 2024-09-10T08:10:44.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.847+0000 7f3991a68740 1 -- 172.21.15.2:0/3297605881 shutdown_connections 2024-09-10T08:10:44.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.847+0000 7f3991a68740 1 --2- 172.21.15.2:0/3297605881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x5581990a40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.847+0000 7f3991a68740 1 --2- 172.21.15.2:0/3297605881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199081590 0x5581990a3bb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.847+0000 7f3991a68740 1 --2- 172.21.15.2:0/3297605881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199080c70 0x55819908ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.847+0000 7f3991a68740 1 -- 172.21.15.2:0/3297605881 >> 172.21.15.2:0/3297605881 conn(0x55819907d200 msgr2=0x55819908a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.848+0000 7f3991a68740 1 -- 172.21.15.2:0/3297605881 shutdown_connections 2024-09-10T08:10:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.848+0000 7f3991a68740 1 -- 172.21.15.2:0/3297605881 wait complete. 2024-09-10T08:10:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.848+0000 7f3991a68740 1 -- start start 2024-09-10T08:10:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.848+0000 7f3991a68740 1 Processor -- start 2024-09-10T08:10:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f3991a68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199080c70 0x55819908fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f3991a68740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 0x5581990901e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f3991a68740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x558199090720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f3991a68740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f39881af6d0 con 0x558199081590 2024-09-10T08:10:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f3991a68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39881b0b40 con 0x558199080c70 2024-09-10T08:10:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f3991a68740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39881afda0 con 0x55819908be50 2024-09-10T08:10:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f3991a68740 10 client.0 _ll_register_callbacks cb 0x7f3970009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f397f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 0x5581990901e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f398ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199080c70 0x55819908fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f398ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199080c70 0x55819908fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37530/0 (socket says 172.21.15.2:37530) 2024-09-10T08:10:44.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f398ce00640 1 -- 172.21.15.2:0/1876004910 learned_addr learned my addr 172.21.15.2:0/1876004910 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:44.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f397f400640 1 --2- 172.21.15.2:0/1876004910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 0x5581990901e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40916/0 (socket says 172.21.15.2:40916) 2024-09-10T08:10:44.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f398ce00640 1 --2- 172.21.15.2:0/1876004910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x558199090720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f397f400640 1 -- 172.21.15.2:0/1876004910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 msgr2=0x558199090720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:44.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f397f400640 1 --2- 172.21.15.2:0/1876004910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x558199090720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f397f400640 1 -- 172.21.15.2:0/1876004910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199080c70 msgr2=0x55819908fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:44.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.849+0000 7f397f400640 1 --2- 172.21.15.2:0/1876004910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199080c70 0x55819908fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.850+0000 7f397f400640 1 -- 172.21.15.2:0/1876004910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55819907de90 con 0x558199081590 2024-09-10T08:10:44.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.850+0000 7f397f400640 1 --2- 172.21.15.2:0/1876004910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 0x5581990901e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f397003db30 tx=0x7f397003d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d9a0b1462504d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:44.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.850+0000 7f3976000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:44.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.850+0000 7f3976000640 1 -- 172.21.15.2:0/1876004910 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397005b070 con 0x558199081590 2024-09-10T08:10:44.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.850+0000 7f3976000640 1 -- 172.21.15.2:0/1876004910 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f397003ddc0 con 0x558199081590 2024-09-10T08:10:44.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.850+0000 7f3991a68740 1 -- 172.21.15.2:0/1876004910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3970005520 con 0x558199081590 2024-09-10T08:10:44.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.851+0000 7f3976000640 1 -- 172.21.15.2:0/1876004910 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f397006d050 con 0x558199081590 2024-09-10T08:10:44.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.851+0000 7f3991a68740 10 client.16035 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:44.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.851+0000 7f3991a68740 1 -- 172.21.15.2:0/1876004910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558199078770 con 0x558199081590 2024-09-10T08:10:44.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.851+0000 7f3976000640 1 -- 172.21.15.2:0/1876004910 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3970005520 con 0x558199081590 2024-09-10T08:10:44.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.851+0000 7f3976000640 1 -- 172.21.15.2:0/1876004910 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3970004850 con 0x558199081590 2024-09-10T08:10:44.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.851+0000 7f3991a68740 10 client.16035 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:44.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.851+0000 7f3991a68740 10 client.16035 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:10:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.851+0000 7f3991a68740 20 client.16035 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.851+0000 7f3991a68740 1 --2- 172.21.15.2:0/1876004910 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x558199107080 0x558199109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.851+0000 7f3991a68740 4 client.16035 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:10:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.851+0000 7f3991a68740 1 -- 172.21.15.2:0/1876004910 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558198feee60 con 0x558199107080 2024-09-10T08:10:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.852+0000 7f398ce00640 1 --2- 172.21.15.2:0/1876004910 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x558199107080 0x558199109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.852+0000 7f398ce00640 1 --2- 172.21.15.2:0/1876004910 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x558199107080 0x558199109470 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c64910c8b4ca812 server_cookie=f60c4a70e9a9262f in_seq=0 out_seq=0 2024-09-10T08:10:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.852+0000 7f3976000640 10 client.16035 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.852+0000 7f3976000640 1 -- 172.21.15.2:0/1876004910 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558198feee60 con 0x558199107080 2024-09-10T08:10:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.852+0000 7f3976000640 10 client.16035 handle_command_reply: tid=0 2024-09-10T08:10:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.852+0000 7f3991a68740 1 client.16035 shutdown 2024-09-10T08:10:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.853+0000 7f3991a68740 1 -- 172.21.15.2:0/1876004910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 msgr2=0x5581990901e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.853+0000 7f3991a68740 1 --2- 172.21.15.2:0/1876004910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 0x5581990901e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f397003db30 tx=0x7f397003d790 comp rx=0 tx=0).stop 2024-09-10T08:10:44.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.853+0000 7f3991a68740 1 -- 172.21.15.2:0/1876004910 shutdown_connections 2024-09-10T08:10:44.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.853+0000 7f3991a68740 1 --2- 172.21.15.2:0/1876004910 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x558199107080 0x558199109470 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.853+0000 7f3991a68740 1 --2- 172.21.15.2:0/1876004910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55819908be50 0x558199090720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.853+0000 7f3991a68740 1 --2- 172.21.15.2:0/1876004910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558199081590 0x5581990901e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.853+0000 7f3991a68740 1 --2- 172.21.15.2:0/1876004910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558199080c70 0x55819908fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.853+0000 7f3976000640 0 client.16035 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:10:44.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.853+0000 7f3991a68740 1 -- 172.21.15.2:0/1876004910 >> 172.21.15.2:0/1876004910 conn(0x55819907d200 msgr2=0x558199082ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:44.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.853+0000 7f3991a68740 1 -- 172.21.15.2:0/1876004910 shutdown_connections 2024-09-10T08:10:44.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.853+0000 7f3991a68740 1 -- 172.21.15.2:0/1876004910 wait complete. 2024-09-10T08:10:44.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.853+0000 7f3991a68740 20 client.16035 trim_cache size 0 max 16384 2024-09-10T08:10:44.856 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:10:44.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.854+0000 7f398ee00640 1 -- 172.21.15.2:0/2676714304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f396c0a2c00 msgr2=0x7f396c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:44.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.854+0000 7f398ee00640 1 --2- 172.21.15.2:0/2676714304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f396c0a2c00 0x7f396c0a50c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f397809bbb0 tx=0x7f3978004a40 comp rx=0 tx=0).stop 2024-09-10T08:10:44.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.854+0000 7f398ee00640 1 -- 172.21.15.2:0/2676714304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3988074620 msgr2=0x7f398819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:44.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.854+0000 7f398ee00640 1 --2- 172.21.15.2:0/2676714304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3988074620 0x7f398819a6b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f397800e9f0 tx=0x7f397800edf0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.854+0000 7f398ee00640 1 -- 172.21.15.2:0/2676714304 shutdown_connections 2024-09-10T08:10:44.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.854+0000 7f398ee00640 1 --2- 172.21.15.2:0/2676714304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f398810e3c0 0x7f398819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.854+0000 7f398ee00640 1 --2- 172.21.15.2:0/2676714304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3988074f40 0x7f398819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.854+0000 7f398ee00640 1 --2- 172.21.15.2:0/2676714304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f396c0a2c00 0x7f396c0a50c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.854+0000 7f398ee00640 1 --2- 172.21.15.2:0/2676714304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3988074620 0x7f398819a6b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:44.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.854+0000 7f398ee00640 1 -- 172.21.15.2:0/2676714304 >> 172.21.15.2:0/2676714304 conn(0x7f39880fd8c0 msgr2=0x7f39880fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:44.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.854+0000 7f398ee00640 1 -- 172.21.15.2:0/2676714304 shutdown_connections 2024-09-10T08:10:44.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:44.854+0000 7f398ee00640 1 -- 172.21.15.2:0/2676714304 wait complete. 2024-09-10T08:10:45.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:45 smithi002 ceph-mon[30563]: pgmap v465: 129 pgs: 129 active+clean; 949 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:10:45.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:45 smithi155 ceph-mon[35405]: pgmap v465: 129 pgs: 129 active+clean; 949 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:10:45.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:45 smithi107 ceph-mon[35048]: pgmap v465: 129 pgs: 129 active+clean; 949 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:10:45.837 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:10:46.217 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:46.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.793+0000 7f031a200640 1 -- 172.21.15.2:0/3925603022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0314077040 msgr2=0x7f0314112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:46.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.793+0000 7f031a200640 1 --2- 172.21.15.2:0/3925603022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0314077040 0x7f0314112510 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0300009900 tx=0x7f030002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.793+0000 7f031a200640 1 -- 172.21.15.2:0/3925603022 shutdown_connections 2024-09-10T08:10:46.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.793+0000 7f031a200640 1 --2- 172.21.15.2:0/3925603022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0314112a50 0x7f0314114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.794+0000 7f031a200640 1 --2- 172.21.15.2:0/3925603022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0314077040 0x7f0314112510 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.794+0000 7f031a200640 1 --2- 172.21.15.2:0/3925603022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0314076720 0x7f0314076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.794+0000 7f031a200640 1 -- 172.21.15.2:0/3925603022 >> 172.21.15.2:0/3925603022 conn(0x7f03140ffa50 msgr2=0x7f0314101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:46.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.794+0000 7f031a200640 1 -- 172.21.15.2:0/3925603022 shutdown_connections 2024-09-10T08:10:46.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.794+0000 7f031a200640 1 -- 172.21.15.2:0/3925603022 wait complete. 2024-09-10T08:10:46.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.795+0000 7f031a200640 1 Processor -- start 2024-09-10T08:10:46.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.795+0000 7f031a200640 1 -- start start 2024-09-10T08:10:46.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.795+0000 7f031a200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0314076720 0x7f03141a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.795+0000 7f031a200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0314077040 0x7f03141a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.795+0000 7f031a200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0314112a50 0x7f031419ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.795+0000 7f031a200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0314118630 con 0x7f0314076720 2024-09-10T08:10:46.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.795+0000 7f031a200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03141184b0 con 0x7f0314077040 2024-09-10T08:10:46.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.795+0000 7f031a200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03141187b0 con 0x7f0314112a50 2024-09-10T08:10:46.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.795+0000 7f0313400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0314076720 0x7f03141a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.796+0000 7f0313400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0314076720 0x7f03141a4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40932/0 (socket says 172.21.15.2:40932) 2024-09-10T08:10:46.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.796+0000 7f0313400640 1 -- 172.21.15.2:0/2632138125 learned_addr learned my addr 172.21.15.2:0/2632138125 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:46.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.796+0000 7f0312a00640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0314077040 0x7f03141a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.796+0000 7f0313e00640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0314112a50 0x7f031419ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.796+0000 7f0313400640 1 -- 172.21.15.2:0/2632138125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0314112a50 msgr2=0x7f031419ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:46.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.796+0000 7f0313400640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0314112a50 0x7f031419ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.796+0000 7f0313400640 1 -- 172.21.15.2:0/2632138125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0314077040 msgr2=0x7f03141a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:46.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.796+0000 7f0313400640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0314077040 0x7f03141a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.796+0000 7f0313400640 1 -- 172.21.15.2:0/2632138125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031419f150 con 0x7f0314076720 2024-09-10T08:10:46.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.796+0000 7f0313400640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0314076720 0x7f03141a4860 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f030400e9f0 tx=0x7f030400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:46.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.797+0000 7f030be00640 1 -- 172.21.15.2:0/2632138125 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030400cd80 con 0x7f0314076720 2024-09-10T08:10:46.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.797+0000 7f030be00640 1 -- 172.21.15.2:0/2632138125 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0304005500 con 0x7f0314076720 2024-09-10T08:10:46.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.797+0000 7f030be00640 1 -- 172.21.15.2:0/2632138125 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304010690 con 0x7f0314076720 2024-09-10T08:10:46.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.797+0000 7f031a200640 1 -- 172.21.15.2:0/2632138125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031419f440 con 0x7f0314076720 2024-09-10T08:10:46.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.797+0000 7f031a200640 1 -- 172.21.15.2:0/2632138125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03141ab730 con 0x7f0314076720 2024-09-10T08:10:46.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.800+0000 7f031cc38740 1 Processor -- start 2024-09-10T08:10:46.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.800+0000 7f031cc38740 1 -- start start 2024-09-10T08:10:46.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.800+0000 7f030be00640 1 -- 172.21.15.2:0/2632138125 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f03040046c0 con 0x7f0314076720 2024-09-10T08:10:46.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.800+0000 7f031cc38740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768aac70 0x55e4768ab050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.800+0000 7f031cc38740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768ab590 0x55e4768b5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f031cc38740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768b5e50 0x55e4768b8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f031cc38740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e4768113e0 con 0x55e4768aac70 2024-09-10T08:10:46.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f031cc38740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e4767b8d80 con 0x55e4768b5e50 2024-09-10T08:10:46.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f031cc38740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e47680d3a0 con 0x55e4768ab590 2024-09-10T08:10:46.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f0313e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768aac70 0x55e4768ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f0313400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768ab590 0x55e4768b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f0313e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768aac70 0x55e4768ab050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40948/0 (socket says 172.21.15.2:40948) 2024-09-10T08:10:46.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f0313e00640 1 -- 172.21.15.2:0/1151177615 learned_addr learned my addr 172.21.15.2:0/1151177615 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:46.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f0312a00640 1 --2- 172.21.15.2:0/1151177615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768b5e50 0x55e4768b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f0313400640 1 -- 172.21.15.2:0/1151177615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768b5e50 msgr2=0x55e4768b8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:46.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f0313400640 1 --2- 172.21.15.2:0/1151177615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768b5e50 0x55e4768b8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f0313400640 1 -- 172.21.15.2:0/1151177615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768aac70 msgr2=0x55e4768ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:46.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f0313400640 1 --2- 172.21.15.2:0/1151177615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768aac70 0x55e4768ab050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f0313400640 1 -- 172.21.15.2:0/1151177615 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4767d7ae0 con 0x55e4768ab590 2024-09-10T08:10:46.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.801+0000 7f0313e00640 1 --2- 172.21.15.2:0/1151177615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768aac70 0x55e4768ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:46.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.802+0000 7f0313400640 1 --2- 172.21.15.2:0/1151177615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768ab590 0x55e4768b5910 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f03040ccec0 tx=0x7f03040d5b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ff641bbc0d39d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:46.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.802+0000 7f02ffe00640 1 -- 172.21.15.2:0/1151177615 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040de070 con 0x55e4768ab590 2024-09-10T08:10:46.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.802+0000 7f02ffe00640 1 -- 172.21.15.2:0/1151177615 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03040d10d0 con 0x55e4768ab590 2024-09-10T08:10:46.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.802+0000 7f02ffe00640 1 -- 172.21.15.2:0/1151177615 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040da680 con 0x55e4768ab590 2024-09-10T08:10:46.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.802+0000 7f030be00640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02f80a2b80 0x7f02f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.802+0000 7f030be00640 1 -- 172.21.15.2:0/2632138125 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f03040cf870 con 0x7f0314076720 2024-09-10T08:10:46.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.803+0000 7f0313e00640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02f80a2b80 0x7f02f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.803+0000 7f031cc38740 1 -- 172.21.15.2:0/1151177615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768ab590 msgr2=0x55e4768b5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:46.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.803+0000 7f031cc38740 1 --2- 172.21.15.2:0/1151177615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768ab590 0x55e4768b5910 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f03040ccec0 tx=0x7f03040d5b20 comp rx=0 tx=0).stop 2024-09-10T08:10:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.803+0000 7f031cc38740 1 -- 172.21.15.2:0/1151177615 shutdown_connections 2024-09-10T08:10:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.803+0000 7f031cc38740 1 --2- 172.21.15.2:0/1151177615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768b5e50 0x55e4768b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.803+0000 7f031cc38740 1 --2- 172.21.15.2:0/1151177615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768ab590 0x55e4768b5910 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.803+0000 7f031cc38740 1 --2- 172.21.15.2:0/1151177615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768aac70 0x55e4768ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.803+0000 7f031cc38740 1 -- 172.21.15.2:0/1151177615 >> 172.21.15.2:0/1151177615 conn(0x55e4768a7200 msgr2=0x55e4768a75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.803+0000 7f0313e00640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02f80a2b80 0x7f02f80a5040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f030c009fd0 tx=0x7f030c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.803+0000 7f031cc38740 1 -- 172.21.15.2:0/1151177615 shutdown_connections 2024-09-10T08:10:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.803+0000 7f031cc38740 1 -- 172.21.15.2:0/1151177615 wait complete. 2024-09-10T08:10:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.804+0000 7f031cc38740 1 -- start start 2024-09-10T08:10:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.804+0000 7f031cc38740 1 Processor -- start 2024-09-10T08:10:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.804+0000 7f031cc38740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768aac70 0x55e4768b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.804+0000 7f031cc38740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768ab590 0x55e4768b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.804+0000 7f031cc38740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768b5e50 0x55e4768b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.804+0000 7f031cc38740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e4768b8a20 con 0x55e4768b5e50 2024-09-10T08:10:46.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.804+0000 7f031cc38740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e4767cc0f0 con 0x55e4768ab590 2024-09-10T08:10:46.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.804+0000 7f031cc38740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e4768b8ba0 con 0x55e4768aac70 2024-09-10T08:10:46.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.805+0000 7f031cc38740 10 client.0 _ll_register_callbacks cb 0x55e4767b3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.805+0000 7f0313400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768aac70 0x55e4768b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.805+0000 7f0312a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768ab590 0x55e4768b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.805+0000 7f0313e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768b5e50 0x55e4768b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.805+0000 7f0313400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768aac70 0x55e4768b2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36750/0 (socket says 172.21.15.2:36750) 2024-09-10T08:10:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.805+0000 7f0313400640 1 -- 172.21.15.2:0/230447345 learned_addr learned my addr 172.21.15.2:0/230447345 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.805+0000 7f0312a00640 1 -- 172.21.15.2:0/230447345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768aac70 msgr2=0x55e4768b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.805+0000 7f0312a00640 1 --2- 172.21.15.2:0/230447345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768aac70 0x55e4768b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.805+0000 7f0312a00640 1 -- 172.21.15.2:0/230447345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768b5e50 msgr2=0x55e4768b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.805+0000 7f0312a00640 1 --2- 172.21.15.2:0/230447345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768b5e50 0x55e4768b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.805+0000 7f0312a00640 1 -- 172.21.15.2:0/230447345 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4765823c0 con 0x55e4768ab590 2024-09-10T08:10:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.806+0000 7f0312a00640 1 --2- 172.21.15.2:0/230447345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768ab590 0x55e4768b3240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f030002a910 tx=0x7f0300004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8648cf0c13464f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.806+0000 7f02fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:46.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.806+0000 7f02fcc00640 1 -- 172.21.15.2:0/230447345 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300035040 con 0x55e4768ab590 2024-09-10T08:10:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.806+0000 7f031cc38740 1 -- 172.21.15.2:0/230447345 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03040d10d0 con 0x55e4768ab590 2024-09-10T08:10:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.806+0000 7f02fcc00640 1 -- 172.21.15.2:0/230447345 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0300005450 con 0x55e4768ab590 2024-09-10T08:10:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.806+0000 7f02fcc00640 1 -- 172.21.15.2:0/230447345 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f030004e050 con 0x55e4768ab590 2024-09-10T08:10:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.806+0000 7f031cc38740 10 client.26081 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.806+0000 7f031cc38740 1 -- 172.21.15.2:0/230447345 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e4768c4a80 con 0x55e4768ab590 2024-09-10T08:10:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.807+0000 7f02fcc00640 1 -- 172.21.15.2:0/230447345 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03040d10d0 con 0x55e4768ab590 2024-09-10T08:10:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.807+0000 7f02fcc00640 1 -- 172.21.15.2:0/230447345 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0300031950 con 0x55e4768ab590 2024-09-10T08:10:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.807+0000 7f031cc38740 10 client.26081 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.807+0000 7f031cc38740 10 client.26081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.807+0000 7f031cc38740 20 client.26081 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.807+0000 7f031cc38740 1 --2- 172.21.15.2:0/230447345 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4768c5ee0 0x55e4768c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.807+0000 7f031cc38740 4 client.26081 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:46.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.807+0000 7f031cc38740 1 -- 172.21.15.2:0/230447345 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e47681e950 con 0x55e4768c5ee0 2024-09-10T08:10:46.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.807+0000 7f0313400640 1 --2- 172.21.15.2:0/230447345 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4768c5ee0 0x55e4768c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.808+0000 7f0313400640 1 --2- 172.21.15.2:0/230447345 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4768c5ee0 0x55e4768c8320 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d62a6153987aaf server_cookie=f53e344eb8c35988 in_seq=0 out_seq=0 2024-09-10T08:10:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.808+0000 7f02fcc00640 10 client.26081 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f02fcc00640 1 -- 172.21.15.2:0/230447345 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e47681e950 con 0x55e4768c5ee0 2024-09-10T08:10:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f02fcc00640 10 client.26081 handle_command_reply: tid=0 2024-09-10T08:10:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f031cc38740 1 client.26081 shutdown 2024-09-10T08:10:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f031cc38740 1 -- 172.21.15.2:0/230447345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768ab590 msgr2=0x55e4768b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f031cc38740 1 --2- 172.21.15.2:0/230447345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768ab590 0x55e4768b3240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f030002a910 tx=0x7f0300004440 comp rx=0 tx=0).stop 2024-09-10T08:10:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f031cc38740 1 -- 172.21.15.2:0/230447345 shutdown_connections 2024-09-10T08:10:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f031cc38740 1 --2- 172.21.15.2:0/230447345 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4768c5ee0 0x55e4768c8320 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f031cc38740 1 --2- 172.21.15.2:0/230447345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768b5e50 0x55e4768b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f031cc38740 1 --2- 172.21.15.2:0/230447345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768ab590 0x55e4768b3240 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f031cc38740 1 --2- 172.21.15.2:0/230447345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768aac70 0x55e4768b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f031cc38740 1 -- 172.21.15.2:0/230447345 >> 172.21.15.2:0/230447345 conn(0x55e4768a7200 msgr2=0x55e4768a6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:46.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f02fcc00640 0 client.26081 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:46.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f031cc38740 1 -- 172.21.15.2:0/230447345 shutdown_connections 2024-09-10T08:10:46.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f031cc38740 1 -- 172.21.15.2:0/230447345 wait complete. 2024-09-10T08:10:46.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.809+0000 7f031cc38740 20 client.26081 trim_cache size 0 max 16384 2024-09-10T08:10:46.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.823+0000 7f031cc38740 1 Processor -- start 2024-09-10T08:10:46.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.823+0000 7f031cc38740 1 -- start start 2024-09-10T08:10:46.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.823+0000 7f031cc38740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768aac70 0x55e4768b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.823+0000 7f031cc38740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768ab590 0x55e4768b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f031cc38740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768b5e50 0x55e4768cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f031cc38740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03141b7da0 con 0x55e4768aac70 2024-09-10T08:10:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f031cc38740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03141b5670 con 0x55e4768b5e50 2024-09-10T08:10:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f031cc38740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03141b34c0 con 0x55e4768ab590 2024-09-10T08:10:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f0313e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768b5e50 0x55e4768cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f0313400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768aac70 0x55e4768b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f0312a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768ab590 0x55e4768b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f0313e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768b5e50 0x55e4768cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:56216/0 (socket says 172.21.15.2:56216) 2024-09-10T08:10:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f0313e00640 1 -- 172.21.15.2:0/2508294501 learned_addr learned my addr 172.21.15.2:0/2508294501 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:46.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f0313e00640 1 -- 172.21.15.2:0/2508294501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768ab590 msgr2=0x55e4768b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:46.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f0313e00640 1 --2- 172.21.15.2:0/2508294501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768ab590 0x55e4768b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f0313e00640 1 -- 172.21.15.2:0/2508294501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768aac70 msgr2=0x55e4768b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:46.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f0313e00640 1 --2- 172.21.15.2:0/2508294501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768aac70 0x55e4768b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f0313e00640 1 -- 172.21.15.2:0/2508294501 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4768a2770 con 0x55e4768b5e50 2024-09-10T08:10:46.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.824+0000 7f0313400640 1 --2- 172.21.15.2:0/2508294501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768aac70 0x55e4768b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:46.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.825+0000 7f0313e00640 1 --2- 172.21.15.2:0/2508294501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768b5e50 0x55e4768cdbf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f030c00f660 tx=0x7f030c019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62e3aa97961c83f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:46.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.825+0000 7f02ff400640 1 -- 172.21.15.2:0/2508294501 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c003a30 con 0x55e4768b5e50 2024-09-10T08:10:46.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.825+0000 7f02ff400640 1 -- 172.21.15.2:0/2508294501 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030c003bd0 con 0x55e4768b5e50 2024-09-10T08:10:46.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.825+0000 7f02ff400640 1 -- 172.21.15.2:0/2508294501 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c010b30 con 0x55e4768b5e50 2024-09-10T08:10:46.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.826+0000 7f031cc38740 1 -- 172.21.15.2:0/2508294501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768b5e50 msgr2=0x55e4768cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:46.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.826+0000 7f031cc38740 1 --2- 172.21.15.2:0/2508294501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768b5e50 0x55e4768cdbf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f030c00f660 tx=0x7f030c019000 comp rx=0 tx=0).stop 2024-09-10T08:10:46.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.826+0000 7f031cc38740 1 -- 172.21.15.2:0/2508294501 shutdown_connections 2024-09-10T08:10:46.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.826+0000 7f031cc38740 1 --2- 172.21.15.2:0/2508294501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768b5e50 0x55e4768cdbf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.826+0000 7f031cc38740 1 --2- 172.21.15.2:0/2508294501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768ab590 0x55e4768b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.826+0000 7f031cc38740 1 --2- 172.21.15.2:0/2508294501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768aac70 0x55e4768b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.826+0000 7f031cc38740 1 -- 172.21.15.2:0/2508294501 >> 172.21.15.2:0/2508294501 conn(0x55e4768a7200 msgr2=0x55e4768b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:46.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.826+0000 7f031cc38740 1 -- 172.21.15.2:0/2508294501 shutdown_connections 2024-09-10T08:10:46.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.826+0000 7f031cc38740 1 -- 172.21.15.2:0/2508294501 wait complete. 2024-09-10T08:10:46.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.826+0000 7f031cc38740 1 -- start start 2024-09-10T08:10:46.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.826+0000 7f031cc38740 1 Processor -- start 2024-09-10T08:10:46.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.827+0000 7f031cc38740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768aac70 0x55e4768b9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.827+0000 7f031cc38740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768ab590 0x55e4768ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.827+0000 7f031cc38740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768b5e50 0x55e4768ba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.827+0000 7f031cc38740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03141b3b30 con 0x55e4768b5e50 2024-09-10T08:10:46.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.827+0000 7f031cc38740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03141b4fa0 con 0x55e4768ab590 2024-09-10T08:10:46.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.827+0000 7f031cc38740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03141b4200 con 0x55e4768aac70 2024-09-10T08:10:46.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.827+0000 7f031cc38740 10 client.0 _ll_register_callbacks cb 0x7f0300009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:46.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.827+0000 7f0313e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768b5e50 0x55e4768ba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.827+0000 7f0312a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768ab590 0x55e4768ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.827+0000 7f0313e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768b5e50 0x55e4768ba5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44506/0 (socket says 172.21.15.2:44506) 2024-09-10T08:10:46.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.827+0000 7f0313e00640 1 -- 172.21.15.2:0/3314195444 learned_addr learned my addr 172.21.15.2:0/3314195444 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:46.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.827+0000 7f0313400640 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768aac70 0x55e4768b9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.828+0000 7f0312a00640 1 -- 172.21.15.2:0/3314195444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768aac70 msgr2=0x55e4768b9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:46.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.828+0000 7f0312a00640 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768aac70 0x55e4768b9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.828+0000 7f0312a00640 1 -- 172.21.15.2:0/3314195444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768b5e50 msgr2=0x55e4768ba5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:46.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.828+0000 7f0312a00640 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768b5e50 0x55e4768ba5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.828+0000 7f0312a00640 1 -- 172.21.15.2:0/3314195444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47692fbc0 con 0x55e4768ab590 2024-09-10T08:10:46.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.828+0000 7f0313400640 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768aac70 0x55e4768b9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:46.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.828+0000 7f0313e00640 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768b5e50 0x55e4768ba5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:46.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.828+0000 7f0312a00640 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768ab590 0x55e4768ba090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0300009900 tx=0x7f0300005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54dc4e9543e6d71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:46.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.828+0000 7f02fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:46.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.828+0000 7f02fcc00640 1 -- 172.21.15.2:0/3314195444 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300035040 con 0x55e4768ab590 2024-09-10T08:10:46.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.828+0000 7f031cc38740 1 -- 172.21.15.2:0/3314195444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f030c003bd0 con 0x55e4768ab590 2024-09-10T08:10:46.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.828+0000 7f02fcc00640 1 -- 172.21.15.2:0/3314195444 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0300005450 con 0x55e4768ab590 2024-09-10T08:10:46.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.829+0000 7f02fcc00640 1 -- 172.21.15.2:0/3314195444 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0300060050 con 0x55e4768ab590 2024-09-10T08:10:46.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.829+0000 7f031cc38740 10 client.26087 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:46.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.829+0000 7f031cc38740 1 -- 172.21.15.2:0/3314195444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e476930ed0 con 0x55e4768ab590 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.829+0000 7f02fcc00640 1 -- 172.21.15.2:0/3314195444 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030c003bd0 con 0x55e4768ab590 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.829+0000 7f02fcc00640 1 -- 172.21.15.2:0/3314195444 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f03000316b0 con 0x55e4768ab590 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.829+0000 7f031cc38740 10 client.26087 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.829+0000 7f031cc38740 10 client.26087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.829+0000 7f031cc38740 20 client.26087 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.829+0000 7f031cc38740 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4769316a0 0x55e476933ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.829+0000 7f031cc38740 4 client.26087 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.829+0000 7f031cc38740 1 -- 172.21.15.2:0/3314195444 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e476818e60 con 0x55e4769316a0 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.830+0000 7f0313400640 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4769316a0 0x55e476933ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.830+0000 7f0313400640 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4769316a0 0x55e476933ac0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0c79c269ec2348 server_cookie=be9932f5fd5f6dce in_seq=0 out_seq=0 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.830+0000 7f02fcc00640 10 client.26087 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.830+0000 7f02fcc00640 1 -- 172.21.15.2:0/3314195444 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e476818e60 con 0x55e4769316a0 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.830+0000 7f02fcc00640 10 client.26087 handle_command_reply: tid=0 2024-09-10T08:10:46.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.830+0000 7f031cc38740 1 client.26087 shutdown 2024-09-10T08:10:46.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.831+0000 7f031cc38740 1 -- 172.21.15.2:0/3314195444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768ab590 msgr2=0x55e4768ba090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:46.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.831+0000 7f031cc38740 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768ab590 0x55e4768ba090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0300009900 tx=0x7f0300005de0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.831+0000 7f031cc38740 1 -- 172.21.15.2:0/3314195444 shutdown_connections 2024-09-10T08:10:46.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.831+0000 7f031cc38740 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4769316a0 0x55e476933ac0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.831+0000 7f031cc38740 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4768b5e50 0x55e4768ba5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.831+0000 7f031cc38740 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4768ab590 0x55e4768ba090 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.831+0000 7f02fcc00640 0 client.26087 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:46.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.831+0000 7f031cc38740 1 --2- 172.21.15.2:0/3314195444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4768aac70 0x55e4768b9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.831+0000 7f031cc38740 1 -- 172.21.15.2:0/3314195444 >> 172.21.15.2:0/3314195444 conn(0x55e4768a7200 msgr2=0x55e4768ac6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:46.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.831+0000 7f031cc38740 1 -- 172.21.15.2:0/3314195444 shutdown_connections 2024-09-10T08:10:46.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.831+0000 7f031cc38740 1 -- 172.21.15.2:0/3314195444 wait complete. 2024-09-10T08:10:46.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.831+0000 7f031cc38740 20 client.26087 trim_cache size 0 max 16384 2024-09-10T08:10:46.836 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:10:46.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.832+0000 7f031a200640 1 -- 172.21.15.2:0/2632138125 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02f80a2b80 msgr2=0x7f02f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:46.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.832+0000 7f031a200640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02f80a2b80 0x7f02f80a5040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f030c009fd0 tx=0x7f030c009500 comp rx=0 tx=0).stop 2024-09-10T08:10:46.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.832+0000 7f031a200640 1 -- 172.21.15.2:0/2632138125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0314076720 msgr2=0x7f03141a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:46.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.832+0000 7f031a200640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0314076720 0x7f03141a4860 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f030400e9f0 tx=0x7f030400edf0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.832+0000 7f031a200640 1 -- 172.21.15.2:0/2632138125 shutdown_connections 2024-09-10T08:10:46.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.832+0000 7f031a200640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02f80a2b80 0x7f02f80a5040 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.832+0000 7f031a200640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0314112a50 0x7f031419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.832+0000 7f031a200640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0314077040 0x7f03141a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.832+0000 7f031a200640 1 --2- 172.21.15.2:0/2632138125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0314076720 0x7f03141a4860 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:46.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.832+0000 7f031a200640 1 -- 172.21.15.2:0/2632138125 >> 172.21.15.2:0/2632138125 conn(0x7f03140ffa50 msgr2=0x7f0314101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:46.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.832+0000 7f031a200640 1 -- 172.21.15.2:0/2632138125 shutdown_connections 2024-09-10T08:10:46.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:46.832+0000 7f031a200640 1 -- 172.21.15.2:0/2632138125 wait complete. 2024-09-10T08:10:47.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:47 smithi002 ceph-mon[30563]: pgmap v466: 129 pgs: 129 active+clean; 1019 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:10:47.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:47 smithi155 ceph-mon[35405]: pgmap v466: 129 pgs: 129 active+clean; 1019 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:10:47.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:47 smithi107 ceph-mon[35048]: pgmap v466: 129 pgs: 129 active+clean; 1019 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:10:49.112 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:10:49.113 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:10:49.508 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:49.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:49 smithi002 ceph-mon[30563]: pgmap v467: 129 pgs: 129 active+clean; 1019 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:10:49.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:49 smithi155 ceph-mon[35405]: pgmap v467: 129 pgs: 129 active+clean; 1019 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:10:49.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:49 smithi107 ceph-mon[35048]: pgmap v467: 129 pgs: 129 active+clean; 1019 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:10:50.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.064+0000 7f9709000640 1 -- 172.21.15.2:0/3393267838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9704100780 msgr2=0x7f9704109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:50.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.064+0000 7f9709000640 1 --2- 172.21.15.2:0/3393267838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9704100780 0x7f9704109be0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009930 tx=0x7f96f002a4d0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.064+0000 7f9709000640 1 -- 172.21.15.2:0/3393267838 shutdown_connections 2024-09-10T08:10:50.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.064+0000 7f9709000640 1 --2- 172.21.15.2:0/3393267838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f970410a2f0 0x7f9704110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.064+0000 7f9709000640 1 --2- 172.21.15.2:0/3393267838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9704100780 0x7f9704109be0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.064+0000 7f9709000640 1 --2- 172.21.15.2:0/3393267838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97040ffe60 0x7f9704100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.064+0000 7f9709000640 1 -- 172.21.15.2:0/3393267838 >> 172.21.15.2:0/3393267838 conn(0x7f97040fba90 msgr2=0x7f97040fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:50.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.064+0000 7f9709000640 1 -- 172.21.15.2:0/3393267838 shutdown_connections 2024-09-10T08:10:50.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.065+0000 7f9709000640 1 -- 172.21.15.2:0/3393267838 wait complete. 2024-09-10T08:10:50.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.065+0000 7f9709000640 1 Processor -- start 2024-09-10T08:10:50.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.066+0000 7f9709000640 1 -- start start 2024-09-10T08:10:50.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.066+0000 7f9709000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97040ffe60 0x7f970419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.066+0000 7f9709000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9704100780 0x7f970419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.066+0000 7f9709000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970410a2f0 0x7f970419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.066+0000 7f9709000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9704114120 con 0x7f970410a2f0 2024-09-10T08:10:50.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.066+0000 7f9709000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9704113fa0 con 0x7f97040ffe60 2024-09-10T08:10:50.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.066+0000 7f9709000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f97041142a0 con 0x7f9704100780 2024-09-10T08:10:50.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.066+0000 7f9702800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970410a2f0 0x7f970419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.066+0000 7f9702800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970410a2f0 0x7f970419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44518/0 (socket says 172.21.15.2:44518) 2024-09-10T08:10:50.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.066+0000 7f9702800640 1 -- 172.21.15.2:0/3411677790 learned_addr learned my addr 172.21.15.2:0/3411677790 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:50.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.066+0000 7f9701e00640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97040ffe60 0x7f970419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.066+0000 7f9701400640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9704100780 0x7f970419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.067+0000 7f9702800640 1 -- 172.21.15.2:0/3411677790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9704100780 msgr2=0x7f970419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:50.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.067+0000 7f9702800640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9704100780 0x7f970419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.067+0000 7f9702800640 1 -- 172.21.15.2:0/3411677790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97040ffe60 msgr2=0x7f970419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:50.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.067+0000 7f9702800640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97040ffe60 0x7f970419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.067+0000 7f9702800640 1 -- 172.21.15.2:0/3411677790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970419f630 con 0x7f970410a2f0 2024-09-10T08:10:50.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.067+0000 7f9702800640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970410a2f0 0x7f970419ef50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00eb10 tx=0x7f96fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:50.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.067+0000 7f96faa00640 1 -- 172.21.15.2:0/3411677790 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc004b20 con 0x7f970410a2f0 2024-09-10T08:10:50.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.067+0000 7f96faa00640 1 -- 172.21.15.2:0/3411677790 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96fc004cc0 con 0x7f970410a2f0 2024-09-10T08:10:50.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.067+0000 7f9709000640 1 -- 172.21.15.2:0/3411677790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970419f920 con 0x7f970410a2f0 2024-09-10T08:10:50.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.067+0000 7f96faa00640 1 -- 172.21.15.2:0/3411677790 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc010760 con 0x7f970410a2f0 2024-09-10T08:10:50.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.068+0000 7f9709000640 1 -- 172.21.15.2:0/3411677790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97041a7200 con 0x7f970410a2f0 2024-09-10T08:10:50.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.071+0000 7f970ba96740 1 Processor -- start 2024-09-10T08:10:50.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.071+0000 7f96faa00640 1 -- 172.21.15.2:0/3411677790 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f96fc005020 con 0x7f970410a2f0 2024-09-10T08:10:50.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.071+0000 7f970ba96740 1 -- start start 2024-09-10T08:10:50.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.071+0000 7f970ba96740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1cac0 0x55681cf1cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.071+0000 7f970ba96740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1d3e0 0x55681cf27760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.071+0000 7f970ba96740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf27ca0 0x55681cf2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.071+0000 7f970ba96740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55681ce83290 con 0x55681cf1cac0 2024-09-10T08:10:50.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.071+0000 7f970ba96740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55681ce2ab40 con 0x55681cf27ca0 2024-09-10T08:10:50.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.071+0000 7f970ba96740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55681ce7f250 con 0x55681cf1d3e0 2024-09-10T08:10:50.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.072+0000 7f9701400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1cac0 0x55681cf1cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.072+0000 7f9701e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1d3e0 0x55681cf27760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.072+0000 7f9701e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1d3e0 0x55681cf27760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49554/0 (socket says 172.21.15.2:49554) 2024-09-10T08:10:50.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.072+0000 7f9701e00640 1 -- 172.21.15.2:0/3985338756 learned_addr learned my addr 172.21.15.2:0/3985338756 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:50.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.072+0000 7f9701400640 1 --2- 172.21.15.2:0/3985338756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf27ca0 0x55681cf2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.072+0000 7f9701e00640 1 -- 172.21.15.2:0/3985338756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf27ca0 msgr2=0x55681cf2a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:50.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.072+0000 7f9701e00640 1 --2- 172.21.15.2:0/3985338756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf27ca0 0x55681cf2a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.072+0000 7f9701e00640 1 -- 172.21.15.2:0/3985338756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1cac0 msgr2=0x55681cf1cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.072+0000 7f9701e00640 1 --2- 172.21.15.2:0/3985338756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1cac0 0x55681cf1cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.072+0000 7f9701e00640 1 -- 172.21.15.2:0/3985338756 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55681cbf46a0 con 0x55681cf1d3e0 2024-09-10T08:10:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.072+0000 7f9701400640 1 --2- 172.21.15.2:0/3985338756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf27ca0 0x55681cf2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.073+0000 7f96faa00640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f96e80a2c00 0x7f96e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.073+0000 7f9701e00640 1 --2- 172.21.15.2:0/3985338756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1d3e0 0x55681cf27760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009cf0 tx=0x7f96f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66ea15ba88ca6938 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.073+0000 7f96faa00640 1 -- 172.21.15.2:0/3411677790 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f96fc0cfa70 con 0x7f970410a2f0 2024-09-10T08:10:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.073+0000 7f9701400640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f96e80a2c00 0x7f96e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.073+0000 7f96ef400640 1 -- 172.21.15.2:0/3985338756 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f401e070 con 0x55681cf1d3e0 2024-09-10T08:10:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.073+0000 7f96ef400640 1 -- 172.21.15.2:0/3985338756 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f4019b10 con 0x55681cf1d3e0 2024-09-10T08:10:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.073+0000 7f96ef400640 1 -- 172.21.15.2:0/3985338756 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f400ea60 con 0x55681cf1d3e0 2024-09-10T08:10:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.074+0000 7f970ba96740 1 -- 172.21.15.2:0/3985338756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1d3e0 msgr2=0x55681cf27760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.074+0000 7f9701400640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f96e80a2c00 0x7f96e80a50c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009900 tx=0x7f96f0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.074+0000 7f970ba96740 1 --2- 172.21.15.2:0/3985338756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1d3e0 0x55681cf27760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009cf0 tx=0x7f96f4011810 comp rx=0 tx=0).stop 2024-09-10T08:10:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.074+0000 7f970ba96740 1 -- 172.21.15.2:0/3985338756 shutdown_connections 2024-09-10T08:10:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.074+0000 7f970ba96740 1 --2- 172.21.15.2:0/3985338756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf27ca0 0x55681cf2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.074+0000 7f970ba96740 1 --2- 172.21.15.2:0/3985338756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1d3e0 0x55681cf27760 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.074+0000 7f970ba96740 1 --2- 172.21.15.2:0/3985338756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1cac0 0x55681cf1cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.074+0000 7f970ba96740 1 -- 172.21.15.2:0/3985338756 >> 172.21.15.2:0/3985338756 conn(0x55681cf19050 msgr2=0x55681cf19430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.074+0000 7f970ba96740 1 -- 172.21.15.2:0/3985338756 shutdown_connections 2024-09-10T08:10:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.074+0000 7f970ba96740 1 -- 172.21.15.2:0/3985338756 wait complete. 2024-09-10T08:10:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.075+0000 7f970ba96740 1 -- start start 2024-09-10T08:10:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.075+0000 7f970ba96740 1 Processor -- start 2024-09-10T08:10:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.075+0000 7f970ba96740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf1cac0 0x55681cf27690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.075+0000 7f970ba96740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1d3e0 0x55681cf24940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.075+0000 7f970ba96740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf27ca0 0x55681cf24f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.075+0000 7f970ba96740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55681cf2a910 con 0x55681cf1d3e0 2024-09-10T08:10:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.075+0000 7f970ba96740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55681ce3dfa0 con 0x55681cf1cac0 2024-09-10T08:10:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.075+0000 7f970ba96740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55681cf2aa90 con 0x55681cf27ca0 2024-09-10T08:10:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.075+0000 7f970ba96740 10 client.0 _ll_register_callbacks cb 0x55681cb91ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.076+0000 7f9702800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf27ca0 0x55681cf24f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.076+0000 7f9701400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1d3e0 0x55681cf24940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.076+0000 7f9701e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf1cac0 0x55681cf27690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.076+0000 7f9702800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf27ca0 0x55681cf24f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49564/0 (socket says 172.21.15.2:49564) 2024-09-10T08:10:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.076+0000 7f9702800640 1 -- 172.21.15.2:0/1586754914 learned_addr learned my addr 172.21.15.2:0/1586754914 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.076+0000 7f9701e00640 1 -- 172.21.15.2:0/1586754914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf27ca0 msgr2=0x55681cf24f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.076+0000 7f9701e00640 1 --2- 172.21.15.2:0/1586754914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf27ca0 0x55681cf24f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.076+0000 7f9701e00640 1 -- 172.21.15.2:0/1586754914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1d3e0 msgr2=0x55681cf24940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.076+0000 7f9701e00640 1 --2- 172.21.15.2:0/1586754914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1d3e0 0x55681cf24940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.076+0000 7f9701e00640 1 -- 172.21.15.2:0/1586754914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55681cf25760 con 0x55681cf1cac0 2024-09-10T08:10:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.077+0000 7f9701e00640 1 --2- 172.21.15.2:0/1586754914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf1cac0 0x55681cf27690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f96f401c040 tx=0x7f96f4005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4504fd6e0121d931 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.077+0000 7f96e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:10:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.077+0000 7f96e3e00640 1 -- 172.21.15.2:0/1586754914 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f401e050 con 0x55681cf1cac0 2024-09-10T08:10:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.077+0000 7f96e3e00640 1 -- 172.21.15.2:0/1586754914 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f96f4005040 con 0x55681cf1cac0 2024-09-10T08:10:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.077+0000 7f970ba96740 1 -- 172.21.15.2:0/1586754914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f4019b10 con 0x55681cf1cac0 2024-09-10T08:10:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.077+0000 7f96e3e00640 1 -- 172.21.15.2:0/1586754914 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96f4032050 con 0x55681cf1cac0 2024-09-10T08:10:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.077+0000 7f970ba96740 10 client.26099 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.077+0000 7f970ba96740 1 -- 172.21.15.2:0/1586754914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55681cf368d0 con 0x55681cf1cac0 2024-09-10T08:10:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.078+0000 7f96e3e00640 1 -- 172.21.15.2:0/1586754914 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f4019b10 con 0x55681cf1cac0 2024-09-10T08:10:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.078+0000 7f96e3e00640 1 -- 172.21.15.2:0/1586754914 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f96f400eae0 con 0x55681cf1cac0 2024-09-10T08:10:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.078+0000 7f970ba96740 10 client.26099 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.078+0000 7f970ba96740 10 client.26099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.078+0000 7f970ba96740 20 client.26099 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.078+0000 7f970ba96740 1 --2- 172.21.15.2:0/1586754914 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55681cf37d30 0x55681cf3a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.078+0000 7f970ba96740 4 client.26099 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.078+0000 7f970ba96740 1 -- 172.21.15.2:0/1586754914 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55681ce90800 con 0x55681cf37d30 2024-09-10T08:10:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.078+0000 7f9701400640 1 --2- 172.21.15.2:0/1586754914 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55681cf37d30 0x55681cf3a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.079+0000 7f9701400640 1 --2- 172.21.15.2:0/1586754914 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55681cf37d30 0x55681cf3a120 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b89a43384d002261 server_cookie=6f6323ffee162280 in_seq=0 out_seq=0 2024-09-10T08:10:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.079+0000 7f96e3e00640 10 client.26099 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f96e3e00640 1 -- 172.21.15.2:0/1586754914 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55681ce90800 con 0x55681cf37d30 2024-09-10T08:10:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f96e3e00640 10 client.26099 handle_command_reply: tid=0 2024-09-10T08:10:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f970ba96740 1 client.26099 shutdown 2024-09-10T08:10:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f970ba96740 1 -- 172.21.15.2:0/1586754914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf1cac0 msgr2=0x55681cf27690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f970ba96740 1 --2- 172.21.15.2:0/1586754914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf1cac0 0x55681cf27690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f96f401c040 tx=0x7f96f4005dc0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f970ba96740 1 -- 172.21.15.2:0/1586754914 shutdown_connections 2024-09-10T08:10:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f970ba96740 1 --2- 172.21.15.2:0/1586754914 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55681cf37d30 0x55681cf3a120 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f970ba96740 1 --2- 172.21.15.2:0/1586754914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf27ca0 0x55681cf24f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f96e3e00640 0 client.26099 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:50.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f970ba96740 1 --2- 172.21.15.2:0/1586754914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1d3e0 0x55681cf24940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f970ba96740 1 --2- 172.21.15.2:0/1586754914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf1cac0 0x55681cf27690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f970ba96740 1 -- 172.21.15.2:0/1586754914 >> 172.21.15.2:0/1586754914 conn(0x55681cf19050 msgr2=0x55681cf1ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:50.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f970ba96740 1 -- 172.21.15.2:0/1586754914 shutdown_connections 2024-09-10T08:10:50.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f970ba96740 1 -- 172.21.15.2:0/1586754914 wait complete. 2024-09-10T08:10:50.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.080+0000 7f970ba96740 20 client.26099 trim_cache size 0 max 16384 2024-09-10T08:10:50.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.099+0000 7f970ba96740 1 Processor -- start 2024-09-10T08:10:50.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.099+0000 7f970ba96740 1 -- start start 2024-09-10T08:10:50.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.099+0000 7f970ba96740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf19050 0x55681cf240f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.099+0000 7f970ba96740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1cac0 0x55681cf180b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.099+0000 7f970ba96740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1d3e0 0x55681cf3f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.099+0000 7f970ba96740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f97041b3a40 con 0x55681cf1d3e0 2024-09-10T08:10:50.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.099+0000 7f970ba96740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f97041b1310 con 0x55681cf19050 2024-09-10T08:10:50.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.099+0000 7f970ba96740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f97041af160 con 0x55681cf1cac0 2024-09-10T08:10:50.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.100+0000 7f9702800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1d3e0 0x55681cf3f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.100+0000 7f9702800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1d3e0 0x55681cf3f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44538/0 (socket says 172.21.15.2:44538) 2024-09-10T08:10:50.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.100+0000 7f9702800640 1 -- 172.21.15.2:0/3229579847 learned_addr learned my addr 172.21.15.2:0/3229579847 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:50.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.100+0000 7f9701e00640 1 --2- 172.21.15.2:0/3229579847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf19050 0x55681cf240f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.100+0000 7f9701400640 1 --2- 172.21.15.2:0/3229579847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1cac0 0x55681cf180b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.100+0000 7f9701e00640 1 -- 172.21.15.2:0/3229579847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1cac0 msgr2=0x55681cf180b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:50.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.100+0000 7f9701e00640 1 --2- 172.21.15.2:0/3229579847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1cac0 0x55681cf180b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.100+0000 7f9701e00640 1 -- 172.21.15.2:0/3229579847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1d3e0 msgr2=0x55681cf3f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:50.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.100+0000 7f9701e00640 1 --2- 172.21.15.2:0/3229579847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1d3e0 0x55681cf3f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.100+0000 7f9701e00640 1 -- 172.21.15.2:0/3229579847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55681cf25cc0 con 0x55681cf19050 2024-09-10T08:10:50.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.100+0000 7f9701e00640 1 --2- 172.21.15.2:0/3229579847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf19050 0x55681cf240f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f96f4001eb0 tx=0x7f96f4009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b90f007c2de947 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:50.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.101+0000 7f96eea00640 1 -- 172.21.15.2:0/3229579847 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f401e040 con 0x55681cf19050 2024-09-10T08:10:50.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.101+0000 7f96eea00640 1 -- 172.21.15.2:0/3229579847 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f401f040 con 0x55681cf19050 2024-09-10T08:10:50.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.101+0000 7f96eea00640 1 -- 172.21.15.2:0/3229579847 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f400e3b0 con 0x55681cf19050 2024-09-10T08:10:50.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.101+0000 7f970ba96740 1 -- 172.21.15.2:0/3229579847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf19050 msgr2=0x55681cf240f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:50.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.101+0000 7f970ba96740 1 --2- 172.21.15.2:0/3229579847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf19050 0x55681cf240f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f96f4001eb0 tx=0x7f96f4009bf0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.101+0000 7f970ba96740 1 -- 172.21.15.2:0/3229579847 shutdown_connections 2024-09-10T08:10:50.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.101+0000 7f970ba96740 1 --2- 172.21.15.2:0/3229579847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf1d3e0 0x55681cf3f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.101+0000 7f970ba96740 1 --2- 172.21.15.2:0/3229579847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1cac0 0x55681cf180b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.101+0000 7f970ba96740 1 --2- 172.21.15.2:0/3229579847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf19050 0x55681cf240f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.101+0000 7f970ba96740 1 -- 172.21.15.2:0/3229579847 >> 172.21.15.2:0/3229579847 conn(0x55681cf20330 msgr2=0x55681cf179a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:50.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.101+0000 7f970ba96740 1 -- 172.21.15.2:0/3229579847 shutdown_connections 2024-09-10T08:10:50.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.102+0000 7f970ba96740 1 -- 172.21.15.2:0/3229579847 wait complete. 2024-09-10T08:10:50.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.102+0000 7f970ba96740 1 -- start start 2024-09-10T08:10:50.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.102+0000 7f970ba96740 1 Processor -- start 2024-09-10T08:10:50.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f970ba96740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf19050 0x55681cf2c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f970ba96740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1cac0 0x55681cf2ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f970ba96740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf1d3e0 0x55681cf2fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f970ba96740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f97041af7d0 con 0x55681cf19050 2024-09-10T08:10:50.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f970ba96740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f97041b0c40 con 0x55681cf1d3e0 2024-09-10T08:10:50.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f970ba96740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f97041afea0 con 0x55681cf1cac0 2024-09-10T08:10:50.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f970ba96740 10 client.0 _ll_register_callbacks cb 0x7f96f401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:50.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f9701e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf19050 0x55681cf2c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f9701e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf19050 0x55681cf2c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44546/0 (socket says 172.21.15.2:44546) 2024-09-10T08:10:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f9701e00640 1 -- 172.21.15.2:0/2918076987 learned_addr learned my addr 172.21.15.2:0/2918076987 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f9701400640 1 --2- 172.21.15.2:0/2918076987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1cac0 0x55681cf2ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f9702800640 1 --2- 172.21.15.2:0/2918076987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf1d3e0 0x55681cf2fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f9701e00640 1 -- 172.21.15.2:0/2918076987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1cac0 msgr2=0x55681cf2ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f9701e00640 1 --2- 172.21.15.2:0/2918076987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1cac0 0x55681cf2ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f9701e00640 1 -- 172.21.15.2:0/2918076987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf1d3e0 msgr2=0x55681cf2fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:50.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f9701e00640 1 --2- 172.21.15.2:0/2918076987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf1d3e0 0x55681cf2fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.103+0000 7f9701e00640 1 -- 172.21.15.2:0/2918076987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55681cf301d0 con 0x55681cf19050 2024-09-10T08:10:50.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.104+0000 7f9701e00640 1 --2- 172.21.15.2:0/2918076987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf19050 0x55681cf2c530 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f96f401abb0 tx=0x7f96f4004850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e149e803f140057 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:50.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.104+0000 7f96e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:50.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.104+0000 7f96e3e00640 1 -- 172.21.15.2:0/2918076987 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f401e070 con 0x55681cf19050 2024-09-10T08:10:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.104+0000 7f96e3e00640 1 -- 172.21.15.2:0/2918076987 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f96f4004030 con 0x55681cf19050 2024-09-10T08:10:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.104+0000 7f970ba96740 1 -- 172.21.15.2:0/2918076987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f401f040 con 0x55681cf19050 2024-09-10T08:10:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.105+0000 7f96e3e00640 1 -- 172.21.15.2:0/2918076987 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96f4011d10 con 0x55681cf19050 2024-09-10T08:10:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.105+0000 7f970ba96740 10 client.16071 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.105+0000 7f970ba96740 1 -- 172.21.15.2:0/2918076987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55681cfa3410 con 0x55681cf19050 2024-09-10T08:10:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.105+0000 7f96e3e00640 1 -- 172.21.15.2:0/2918076987 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f401f040 con 0x55681cf19050 2024-09-10T08:10:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.105+0000 7f96e3e00640 1 -- 172.21.15.2:0/2918076987 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f96f4040030 con 0x55681cf19050 2024-09-10T08:10:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.105+0000 7f970ba96740 10 client.16071 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.105+0000 7f970ba96740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.105+0000 7f970ba96740 20 client.16071 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:50.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.105+0000 7f970ba96740 1 --2- 172.21.15.2:0/2918076987 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55681cf27ca0 0x55681cfa5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:50.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.105+0000 7f970ba96740 4 client.16071 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:10:50.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.105+0000 7f970ba96740 1 -- 172.21.15.2:0/2918076987 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55681ce8ad10 con 0x55681cf27ca0 2024-09-10T08:10:50.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.106+0000 7f9701400640 1 --2- 172.21.15.2:0/2918076987 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55681cf27ca0 0x55681cfa5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:50.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.106+0000 7f9701400640 1 --2- 172.21.15.2:0/2918076987 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55681cf27ca0 0x55681cfa5ae0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b92878653f902 server_cookie=892ee54426e269ea in_seq=0 out_seq=0 2024-09-10T08:10:50.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.106+0000 7f96e3e00640 10 client.16071 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:50.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.107+0000 7f96e3e00640 1 -- 172.21.15.2:0/2918076987 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55681ce8ad10 con 0x55681cf27ca0 2024-09-10T08:10:50.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.107+0000 7f96e3e00640 10 client.16071 handle_command_reply: tid=0 2024-09-10T08:10:50.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.108+0000 7f970ba96740 1 client.16071 shutdown 2024-09-10T08:10:50.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.108+0000 7f970ba96740 1 -- 172.21.15.2:0/2918076987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf19050 msgr2=0x55681cf2c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:50.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.108+0000 7f970ba96740 1 --2- 172.21.15.2:0/2918076987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf19050 0x55681cf2c530 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f96f401abb0 tx=0x7f96f4004850 comp rx=0 tx=0).stop 2024-09-10T08:10:50.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.108+0000 7f970ba96740 1 -- 172.21.15.2:0/2918076987 shutdown_connections 2024-09-10T08:10:50.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.108+0000 7f970ba96740 1 --2- 172.21.15.2:0/2918076987 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55681cf27ca0 0x55681cfa5ae0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.108+0000 7f970ba96740 1 --2- 172.21.15.2:0/2918076987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55681cf1d3e0 0x55681cf2fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.108+0000 7f970ba96740 1 --2- 172.21.15.2:0/2918076987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55681cf1cac0 0x55681cf2ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.108+0000 7f970ba96740 1 --2- 172.21.15.2:0/2918076987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55681cf19050 0x55681cf2c530 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.108+0000 7f970ba96740 1 -- 172.21.15.2:0/2918076987 >> 172.21.15.2:0/2918076987 conn(0x55681cf20330 msgr2=0x55681cf1ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:50.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.108+0000 7f96e3e00640 0 client.16071 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:50.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.108+0000 7f970ba96740 1 -- 172.21.15.2:0/2918076987 shutdown_connections 2024-09-10T08:10:50.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.108+0000 7f970ba96740 1 -- 172.21.15.2:0/2918076987 wait complete. 2024-09-10T08:10:50.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.108+0000 7f970ba96740 20 client.16071 trim_cache size 0 max 16384 2024-09-10T08:10:50.124 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:10:50.124 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:10:50.124 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "363d276b-2069-48d6-b0f7-acb2ffbdb07e", 2024-09-10T08:10:50.125 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:10:50.125 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:10:50.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.109+0000 7f9709000640 1 -- 172.21.15.2:0/3411677790 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f96e80a2c00 msgr2=0x7f96e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:50.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.109+0000 7f9709000640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f96e80a2c00 0x7f96e80a50c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009900 tx=0x7f96f0005fd0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.109+0000 7f9709000640 1 -- 172.21.15.2:0/3411677790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970410a2f0 msgr2=0x7f970419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:50.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.109+0000 7f9709000640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970410a2f0 0x7f970419ef50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00eb10 tx=0x7f96fc00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.109+0000 7f9709000640 1 -- 172.21.15.2:0/3411677790 shutdown_connections 2024-09-10T08:10:50.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.109+0000 7f9709000640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970410a2f0 0x7f970419ef50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.109+0000 7f9709000640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f96e80a2c00 0x7f96e80a50c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.109+0000 7f9709000640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9704100780 0x7f970419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.109+0000 7f9709000640 1 --2- 172.21.15.2:0/3411677790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97040ffe60 0x7f970419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:50.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.109+0000 7f9709000640 1 -- 172.21.15.2:0/3411677790 >> 172.21.15.2:0/3411677790 conn(0x7f97040fba90 msgr2=0x7f9704107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:50.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.109+0000 7f9709000640 1 -- 172.21.15.2:0/3411677790 shutdown_connections 2024-09-10T08:10:50.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:50.109+0000 7f9709000640 1 -- 172.21.15.2:0/3411677790 wait complete. 2024-09-10T08:10:51.103 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:10:51.484 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:10:51.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:51 smithi002 ceph-mon[30563]: pgmap v468: 129 pgs: 129 active+clean; 1019 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:10:51.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:51 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:10:51.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:51 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:51.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:51 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:10:51.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:51 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:51.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:51 smithi155 ceph-mon[35405]: pgmap v468: 129 pgs: 129 active+clean; 1019 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:10:51.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:51 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:10:51.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:51 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:51.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:51 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:10:51.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:51 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:51.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:51 smithi107 ceph-mon[35048]: pgmap v468: 129 pgs: 129 active+clean; 1019 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:10:51.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:51 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:10:51.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:51 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:51.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:51 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:10:51.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:51 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:10:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.058+0000 7f393fa00640 1 -- 172.21.15.2:0/2919235561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3938074f40 msgr2=0x7f393810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.058+0000 7f393fa00640 1 --2- 172.21.15.2:0/2919235561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3938074f40 0x7f393810de80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3924009900 tx=0x7f392402a480 comp rx=0 tx=0).stop 2024-09-10T08:10:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.059+0000 7f393fa00640 1 -- 172.21.15.2:0/2919235561 shutdown_connections 2024-09-10T08:10:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.059+0000 7f393fa00640 1 --2- 172.21.15.2:0/2919235561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f393810e3c0 0x7f3938110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.059+0000 7f393fa00640 1 --2- 172.21.15.2:0/2919235561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3938074f40 0x7f393810de80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.059+0000 7f393fa00640 1 --2- 172.21.15.2:0/2919235561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3938074620 0x7f3938074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.059+0000 7f393fa00640 1 -- 172.21.15.2:0/2919235561 >> 172.21.15.2:0/2919235561 conn(0x7f39380fd8c0 msgr2=0x7f39380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.059+0000 7f393fa00640 1 -- 172.21.15.2:0/2919235561 shutdown_connections 2024-09-10T08:10:52.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.059+0000 7f393fa00640 1 -- 172.21.15.2:0/2919235561 wait complete. 2024-09-10T08:10:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.060+0000 7f393fa00640 1 Processor -- start 2024-09-10T08:10:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.060+0000 7f393fa00640 1 -- start start 2024-09-10T08:10:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.061+0000 7f393fa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3938074620 0x7f393819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.061+0000 7f393fa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3938074f40 0x7f393819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.061+0000 7f393fa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f393810e3c0 0x7f393819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.061+0000 7f393fa00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3938114190 con 0x7f3938074f40 2024-09-10T08:10:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.061+0000 7f393fa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3938114010 con 0x7f393810e3c0 2024-09-10T08:10:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.061+0000 7f393fa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3938114310 con 0x7f3938074620 2024-09-10T08:10:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.061+0000 7f3937e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3938074f40 0x7f393819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.061+0000 7f393d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3938074620 0x7f393819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.061+0000 7f3937e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3938074f40 0x7f393819abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44566/0 (socket says 172.21.15.2:44566) 2024-09-10T08:10:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.061+0000 7f3937e00640 1 -- 172.21.15.2:0/164390635 learned_addr learned my addr 172.21.15.2:0/164390635 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.061+0000 7f393da00640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f393810e3c0 0x7f393819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.062+0000 7f393d000640 1 -- 172.21.15.2:0/164390635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f393810e3c0 msgr2=0x7f393819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.062+0000 7f393d000640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f393810e3c0 0x7f393819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.062+0000 7f393d000640 1 -- 172.21.15.2:0/164390635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3938074f40 msgr2=0x7f393819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.062+0000 7f393d000640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3938074f40 0x7f393819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.062+0000 7f393d000640 1 -- 172.21.15.2:0/164390635 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393819f660 con 0x7f3938074620 2024-09-10T08:10:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.062+0000 7f3937e00640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3938074f40 0x7f393819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.062+0000 7f393d000640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3938074620 0x7f393819a6b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f392c00e9f0 tx=0x7f392c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.062+0000 7f3935600640 1 -- 172.21.15.2:0/164390635 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c00ccc0 con 0x7f3938074620 2024-09-10T08:10:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.062+0000 7f393fa00640 1 -- 172.21.15.2:0/164390635 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393819f950 con 0x7f3938074620 2024-09-10T08:10:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.062+0000 7f3935600640 1 -- 172.21.15.2:0/164390635 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392c010910 con 0x7f3938074620 2024-09-10T08:10:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.062+0000 7f3935600640 1 -- 172.21.15.2:0/164390635 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c018960 con 0x7f3938074620 2024-09-10T08:10:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.063+0000 7f393fa00640 1 -- 172.21.15.2:0/164390635 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39381a7230 con 0x7f3938074620 2024-09-10T08:10:52.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.065+0000 7f39424ba740 1 Processor -- start 2024-09-10T08:10:52.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.066+0000 7f39424ba740 1 -- start start 2024-09-10T08:10:52.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.066+0000 7f3935600640 1 -- 172.21.15.2:0/164390635 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f392c010ac0 con 0x7f3938074620 2024-09-10T08:10:52.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.066+0000 7f39424ba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8e3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.066+0000 7f39424ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8ee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.066+0000 7f39424ba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8eed70 0x55ab2e8f1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.066+0000 7f39424ba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ab2e84a3e0 con 0x55ab2e8eed70 2024-09-10T08:10:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.066+0000 7f39424ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ab2e7f1d80 con 0x55ab2e8e44b0 2024-09-10T08:10:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.066+0000 7f39424ba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ab2e8463a0 con 0x55ab2e8e3b90 2024-09-10T08:10:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.066+0000 7f393da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8e3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.066+0000 7f3937e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8eed70 0x55ab2e8f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.066+0000 7f393da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8e3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49620/0 (socket says 172.21.15.2:49620) 2024-09-10T08:10:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.066+0000 7f393da00640 1 -- 172.21.15.2:0/1525955807 learned_addr learned my addr 172.21.15.2:0/1525955807 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.066+0000 7f393d000640 1 --2- 172.21.15.2:0/1525955807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8ee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.067+0000 7f393da00640 1 -- 172.21.15.2:0/1525955807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e44b0 msgr2=0x55ab2e8ee830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.067+0000 7f393da00640 1 --2- 172.21.15.2:0/1525955807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8ee830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.067+0000 7f393da00640 1 -- 172.21.15.2:0/1525955807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8eed70 msgr2=0x55ab2e8f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.067+0000 7f393da00640 1 --2- 172.21.15.2:0/1525955807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8eed70 0x55ab2e8f1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.067+0000 7f393da00640 1 -- 172.21.15.2:0/1525955807 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab2e810ae0 con 0x55ab2e8e3b90 2024-09-10T08:10:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.067+0000 7f393da00640 1 --2- 172.21.15.2:0/1525955807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8e3f70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3930009f90 tx=0x7f3930010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12674a352cd4e6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.067+0000 7f392a000640 1 -- 172.21.15.2:0/1525955807 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930011700 con 0x55ab2e8e3b90 2024-09-10T08:10:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.068+0000 7f3935600640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f39200a2b80 0x7f39200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.068+0000 7f392a000640 1 -- 172.21.15.2:0/1525955807 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3930011d80 con 0x55ab2e8e3b90 2024-09-10T08:10:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.068+0000 7f392a000640 1 -- 172.21.15.2:0/1525955807 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930019e40 con 0x55ab2e8e3b90 2024-09-10T08:10:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.068+0000 7f3935600640 1 -- 172.21.15.2:0/164390635 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f392c0cf820 con 0x7f3938074620 2024-09-10T08:10:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.068+0000 7f393d000640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f39200a2b80 0x7f39200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.068+0000 7f39424ba740 1 -- 172.21.15.2:0/1525955807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8e3b90 msgr2=0x55ab2e8e3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.068+0000 7f39424ba740 1 --2- 172.21.15.2:0/1525955807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8e3f70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3930009f90 tx=0x7f3930010860 comp rx=0 tx=0).stop 2024-09-10T08:10:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.068+0000 7f39424ba740 1 -- 172.21.15.2:0/1525955807 shutdown_connections 2024-09-10T08:10:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.068+0000 7f39424ba740 1 --2- 172.21.15.2:0/1525955807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8eed70 0x55ab2e8f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.068+0000 7f39424ba740 1 --2- 172.21.15.2:0/1525955807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8ee830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.068+0000 7f39424ba740 1 --2- 172.21.15.2:0/1525955807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8e3f70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.068+0000 7f39424ba740 1 -- 172.21.15.2:0/1525955807 >> 172.21.15.2:0/1525955807 conn(0x55ab2e8e0120 msgr2=0x55ab2e8e0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.069+0000 7f39424ba740 1 -- 172.21.15.2:0/1525955807 shutdown_connections 2024-09-10T08:10:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.069+0000 7f393d000640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f39200a2b80 0x7f39200a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f392c0986b0 tx=0x7f392c0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.069+0000 7f39424ba740 1 -- 172.21.15.2:0/1525955807 wait complete. 2024-09-10T08:10:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.069+0000 7f39424ba740 1 -- start start 2024-09-10T08:10:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.069+0000 7f39424ba740 1 Processor -- start 2024-09-10T08:10:52.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.070+0000 7f39424ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8ebcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.070+0000 7f39424ba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8ec230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.070+0000 7f39424ba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8eed70 0x55ab2e8ec770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.070+0000 7f39424ba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ab2e8f19e0 con 0x55ab2e8e44b0 2024-09-10T08:10:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.070+0000 7f39424ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ab2e8050f0 con 0x55ab2e8e3b90 2024-09-10T08:10:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.070+0000 7f39424ba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ab2e8f1b60 con 0x55ab2e8eed70 2024-09-10T08:10:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.070+0000 7f39424ba740 10 client.0 _ll_register_callbacks cb 0x55ab2e7f69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.070+0000 7f3937e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8ec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.070+0000 7f393da00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8ebcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.070+0000 7f3937e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8ec230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44584/0 (socket says 172.21.15.2:44584) 2024-09-10T08:10:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.070+0000 7f3937e00640 1 -- 172.21.15.2:0/2011764776 learned_addr learned my addr 172.21.15.2:0/2011764776 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.070+0000 7f393da00640 1 --2- 172.21.15.2:0/2011764776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8eed70 0x55ab2e8ec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.071+0000 7f3937e00640 1 -- 172.21.15.2:0/2011764776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8eed70 msgr2=0x55ab2e8ec770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.071+0000 7f3937e00640 1 --2- 172.21.15.2:0/2011764776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8eed70 0x55ab2e8ec770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.071+0000 7f3937e00640 1 -- 172.21.15.2:0/2011764776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 msgr2=0x55ab2e8ebcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.071+0000 7f3937e00640 1 --2- 172.21.15.2:0/2011764776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8ebcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.071+0000 7f3937e00640 1 -- 172.21.15.2:0/2011764776 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab2e5bb3c0 con 0x55ab2e8e44b0 2024-09-10T08:10:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.071+0000 7f393da00640 1 --2- 172.21.15.2:0/2011764776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8ebcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.071+0000 7f3937e00640 1 --2- 172.21.15.2:0/2011764776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8ec230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f392402aa70 tx=0x7f3924003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edd28abb7bf3e09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.071+0000 7f391ea00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.071+0000 7f391ea00640 1 -- 172.21.15.2:0/2011764776 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924035040 con 0x55ab2e8e44b0 2024-09-10T08:10:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.071+0000 7f391ea00640 1 -- 172.21.15.2:0/2011764776 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3924003e20 con 0x55ab2e8e44b0 2024-09-10T08:10:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.071+0000 7f39424ba740 1 -- 172.21.15.2:0/2011764776 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3930011d80 con 0x55ab2e8e44b0 2024-09-10T08:10:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.072+0000 7f391ea00640 1 -- 172.21.15.2:0/2011764776 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f392404e050 con 0x55ab2e8e44b0 2024-09-10T08:10:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.072+0000 7f39424ba740 10 client.16083 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.072+0000 7f39424ba740 1 -- 172.21.15.2:0/2011764776 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ab2e8fd740 con 0x55ab2e8e44b0 2024-09-10T08:10:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.072+0000 7f391ea00640 1 -- 172.21.15.2:0/2011764776 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3930011d80 con 0x55ab2e8e44b0 2024-09-10T08:10:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.072+0000 7f391ea00640 1 -- 172.21.15.2:0/2011764776 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f392403c6e0 con 0x55ab2e8e44b0 2024-09-10T08:10:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.072+0000 7f39424ba740 10 client.16083 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.072+0000 7f39424ba740 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.072+0000 7f39424ba740 20 client.16083 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.072+0000 7f39424ba740 1 --2- 172.21.15.2:0/2011764776 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ab2e8fec00 0x55ab2e900ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.072+0000 7f39424ba740 4 client.16083 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.072+0000 7f39424ba740 1 -- 172.21.15.2:0/2011764776 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab2e857950 con 0x55ab2e8fec00 2024-09-10T08:10:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.073+0000 7f393da00640 1 --2- 172.21.15.2:0/2011764776 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ab2e8fec00 0x55ab2e900ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.073+0000 7f393da00640 1 --2- 172.21.15.2:0/2011764776 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ab2e8fec00 0x55ab2e900ff0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df11b3d48a31b1d server_cookie=197c3d3cf140466d in_seq=0 out_seq=0 2024-09-10T08:10:52.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.073+0000 7f391ea00640 10 client.16083 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:52.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.074+0000 7f391ea00640 1 -- 172.21.15.2:0/2011764776 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab2e857950 con 0x55ab2e8fec00 2024-09-10T08:10:52.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.074+0000 7f391ea00640 10 client.16083 handle_command_reply: tid=0 2024-09-10T08:10:52.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.074+0000 7f39424ba740 1 client.16083 shutdown 2024-09-10T08:10:52.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.074+0000 7f39424ba740 1 -- 172.21.15.2:0/2011764776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 msgr2=0x55ab2e8ec230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:52.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.074+0000 7f39424ba740 1 --2- 172.21.15.2:0/2011764776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8ec230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f392402aa70 tx=0x7f3924003820 comp rx=0 tx=0).stop 2024-09-10T08:10:52.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.074+0000 7f39424ba740 1 -- 172.21.15.2:0/2011764776 shutdown_connections 2024-09-10T08:10:52.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.074+0000 7f39424ba740 1 --2- 172.21.15.2:0/2011764776 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ab2e8fec00 0x55ab2e900ff0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.074+0000 7f39424ba740 1 --2- 172.21.15.2:0/2011764776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8eed70 0x55ab2e8ec770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.074+0000 7f39424ba740 1 --2- 172.21.15.2:0/2011764776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8ec230 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.074+0000 7f39424ba740 1 --2- 172.21.15.2:0/2011764776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8ebcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.074+0000 7f39424ba740 1 -- 172.21.15.2:0/2011764776 >> 172.21.15.2:0/2011764776 conn(0x55ab2e8e0120 msgr2=0x55ab2e8e5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:52.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.074+0000 7f391ea00640 0 client.16083 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:52.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.075+0000 7f39424ba740 1 -- 172.21.15.2:0/2011764776 shutdown_connections 2024-09-10T08:10:52.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.075+0000 7f39424ba740 1 -- 172.21.15.2:0/2011764776 wait complete. 2024-09-10T08:10:52.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.075+0000 7f39424ba740 20 client.16083 trim_cache size 0 max 16384 2024-09-10T08:10:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.093+0000 7f39424ba740 1 Processor -- start 2024-09-10T08:10:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.093+0000 7f39424ba740 1 -- start start 2024-09-10T08:10:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.093+0000 7f39424ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8e65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.093+0000 7f39424ba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8e44b0 0x55ab2e906900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.093+0000 7f39424ba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8eed70 0x55ab2e906e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.093+0000 7f39424ba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f39381b3960 con 0x55ab2e8eed70 2024-09-10T08:10:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.093+0000 7f39424ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39381b1230 con 0x55ab2e8e3b90 2024-09-10T08:10:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.093+0000 7f39424ba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39381af080 con 0x55ab2e8e44b0 2024-09-10T08:10:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.093+0000 7f393da00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8eed70 0x55ab2e906e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.093+0000 7f393da00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8eed70 0x55ab2e906e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44598/0 (socket says 172.21.15.2:44598) 2024-09-10T08:10:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.093+0000 7f393da00640 1 -- 172.21.15.2:0/1113414347 learned_addr learned my addr 172.21.15.2:0/1113414347 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.094+0000 7f393da00640 1 --2- 172.21.15.2:0/1113414347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8e65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.094+0000 7f3937e00640 1 --2- 172.21.15.2:0/1113414347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8e44b0 0x55ab2e906900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.094+0000 7f393da00640 1 -- 172.21.15.2:0/1113414347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8e44b0 msgr2=0x55ab2e906900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.094+0000 7f393da00640 1 --2- 172.21.15.2:0/1113414347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8e44b0 0x55ab2e906900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.094+0000 7f393da00640 1 -- 172.21.15.2:0/1113414347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 msgr2=0x55ab2e8e65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.094+0000 7f393da00640 1 --2- 172.21.15.2:0/1113414347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8e65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.094+0000 7f393da00640 1 -- 172.21.15.2:0/1113414347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab2e8e11d0 con 0x55ab2e8eed70 2024-09-10T08:10:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.094+0000 7f393da00640 1 --2- 172.21.15.2:0/1113414347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8eed70 0x55ab2e906e40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3930009d50 tx=0x7f39300096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f584cdb1f031e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.094+0000 7f3929600640 1 -- 172.21.15.2:0/1113414347 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930019d50 con 0x55ab2e8eed70 2024-09-10T08:10:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.094+0000 7f3929600640 1 -- 172.21.15.2:0/1113414347 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3930020040 con 0x55ab2e8eed70 2024-09-10T08:10:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.094+0000 7f3929600640 1 -- 172.21.15.2:0/1113414347 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930005520 con 0x55ab2e8eed70 2024-09-10T08:10:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.095+0000 7f39424ba740 1 -- 172.21.15.2:0/1113414347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8eed70 msgr2=0x55ab2e906e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.095+0000 7f39424ba740 1 --2- 172.21.15.2:0/1113414347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8eed70 0x55ab2e906e40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3930009d50 tx=0x7f39300096a0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.095+0000 7f39424ba740 1 -- 172.21.15.2:0/1113414347 shutdown_connections 2024-09-10T08:10:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.095+0000 7f39424ba740 1 --2- 172.21.15.2:0/1113414347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8eed70 0x55ab2e906e40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.095+0000 7f39424ba740 1 --2- 172.21.15.2:0/1113414347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8e44b0 0x55ab2e906900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.095+0000 7f39424ba740 1 --2- 172.21.15.2:0/1113414347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8e65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.095+0000 7f39424ba740 1 -- 172.21.15.2:0/1113414347 >> 172.21.15.2:0/1113414347 conn(0x55ab2e8e0120 msgr2=0x55ab2e8edd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.095+0000 7f39424ba740 1 -- 172.21.15.2:0/1113414347 shutdown_connections 2024-09-10T08:10:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.095+0000 7f39424ba740 1 -- 172.21.15.2:0/1113414347 wait complete. 2024-09-10T08:10:52.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.096+0000 7f39424ba740 1 -- start start 2024-09-10T08:10:52.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.096+0000 7f39424ba740 1 Processor -- start 2024-09-10T08:10:52.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.096+0000 7f39424ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8f7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.096+0000 7f39424ba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.096+0000 7f39424ba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8eed70 0x55ab2e8f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.096+0000 7f39424ba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f39381af6f0 con 0x55ab2e8e44b0 2024-09-10T08:10:52.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.096+0000 7f39424ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39381b0b60 con 0x55ab2e8e3b90 2024-09-10T08:10:52.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.096+0000 7f39424ba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39381afdc0 con 0x55ab2e8eed70 2024-09-10T08:10:52.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.096+0000 7f39424ba740 10 client.0 _ll_register_callbacks cb 0x7f39240043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:52.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.097+0000 7f3937e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.097+0000 7f3937e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8f2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44604/0 (socket says 172.21.15.2:44604) 2024-09-10T08:10:52.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.097+0000 7f3937e00640 1 -- 172.21.15.2:0/1767449950 learned_addr learned my addr 172.21.15.2:0/1767449950 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:52.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.097+0000 7f393da00640 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.097+0000 7f393da00640 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8eed70 0x55ab2e8f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.097+0000 7f3937e00640 1 -- 172.21.15.2:0/1767449950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8eed70 msgr2=0x55ab2e8f31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.097+0000 7f3937e00640 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8eed70 0x55ab2e8f31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.097+0000 7f3937e00640 1 -- 172.21.15.2:0/1767449950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 msgr2=0x55ab2e8f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.097+0000 7f3937e00640 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.097+0000 7f3937e00640 1 -- 172.21.15.2:0/1767449950 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab2e9688d0 con 0x55ab2e8e44b0 2024-09-10T08:10:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.097+0000 7f393da00640 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8eed70 0x55ab2e8f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:10:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.097+0000 7f3937e00640 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8f2a40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3924004650 tx=0x7f3924003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad2cfa26133ffe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.097+0000 7f391ea00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.098+0000 7f391ea00640 1 -- 172.21.15.2:0/1767449950 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392404b040 con 0x55ab2e8e44b0 2024-09-10T08:10:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.098+0000 7f391ea00640 1 -- 172.21.15.2:0/1767449950 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f392403c850 con 0x55ab2e8e44b0 2024-09-10T08:10:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.098+0000 7f39424ba740 1 -- 172.21.15.2:0/1767449950 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3930020040 con 0x55ab2e8e44b0 2024-09-10T08:10:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.098+0000 7f391ea00640 1 -- 172.21.15.2:0/1767449950 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3924035040 con 0x55ab2e8e44b0 2024-09-10T08:10:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.098+0000 7f39424ba740 10 client.16089 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.098+0000 7f39424ba740 1 -- 172.21.15.2:0/1767449950 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ab2e969af0 con 0x55ab2e8e44b0 2024-09-10T08:10:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.098+0000 7f391ea00640 1 -- 172.21.15.2:0/1767449950 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3930020040 con 0x55ab2e8e44b0 2024-09-10T08:10:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.098+0000 7f391ea00640 1 -- 172.21.15.2:0/1767449950 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3924005200 con 0x55ab2e8e44b0 2024-09-10T08:10:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.098+0000 7f39424ba740 10 client.16089 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.098+0000 7f39424ba740 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.098+0000 7f39424ba740 20 client.16089 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.099+0000 7f39424ba740 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ab2e96a2c0 0x55ab2e96c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.099+0000 7f39424ba740 4 client.16089 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:10:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.099+0000 7f39424ba740 1 -- 172.21.15.2:0/1767449950 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab2e851e60 con 0x55ab2e96a2c0 2024-09-10T08:10:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.099+0000 7f393da00640 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ab2e96a2c0 0x55ab2e96c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:52.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.099+0000 7f393da00640 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ab2e96a2c0 0x55ab2e96c6e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a70d9d664181847 server_cookie=fdad7b119a239f8d in_seq=0 out_seq=0 2024-09-10T08:10:52.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.099+0000 7f391ea00640 10 client.16089 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:52.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f391ea00640 1 -- 172.21.15.2:0/1767449950 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ab2e851e60 con 0x55ab2e96a2c0 2024-09-10T08:10:52.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f391ea00640 10 client.16089 handle_command_reply: tid=0 2024-09-10T08:10:52.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f39424ba740 1 client.16089 shutdown 2024-09-10T08:10:52.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f39424ba740 1 -- 172.21.15.2:0/1767449950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 msgr2=0x55ab2e8f2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:52.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f39424ba740 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8f2a40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3924004650 tx=0x7f3924003e50 comp rx=0 tx=0).stop 2024-09-10T08:10:52.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f39424ba740 1 -- 172.21.15.2:0/1767449950 shutdown_connections 2024-09-10T08:10:52.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f39424ba740 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ab2e96a2c0 0x55ab2e96c6e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f39424ba740 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab2e8eed70 0x55ab2e8f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f39424ba740 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab2e8e44b0 0x55ab2e8f2a40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f39424ba740 1 --2- 172.21.15.2:0/1767449950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab2e8e3b90 0x55ab2e8f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f391ea00640 0 client.16089 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:52.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f39424ba740 1 -- 172.21.15.2:0/1767449950 >> 172.21.15.2:0/1767449950 conn(0x55ab2e8e0120 msgr2=0x55ab2e8edd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:52.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f39424ba740 1 -- 172.21.15.2:0/1767449950 shutdown_connections 2024-09-10T08:10:52.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f39424ba740 1 -- 172.21.15.2:0/1767449950 wait complete. 2024-09-10T08:10:52.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.100+0000 7f39424ba740 20 client.16089 trim_cache size 0 max 16384 2024-09-10T08:10:52.103 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:10:52.103 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:10:52.103 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:10:52.103 INFO:teuthology.orchestra.run.smithi002.stdout: "363d276b-2069-48d6-b0f7-acb2ffbdb07e": { 2024-09-10T08:10:52.103 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:10:52.103 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "363d276b-2069-48d6-b0f7-acb2ffbdb07e", 2024-09-10T08:10:52.103 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:10:52.103 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:10:52.103 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:10:52.103 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:10:52.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.101+0000 7f393fa00640 1 -- 172.21.15.2:0/164390635 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f39200a2b80 msgr2=0x7f39200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:52.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.101+0000 7f393fa00640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f39200a2b80 0x7f39200a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f392c0986b0 tx=0x7f392c0d5530 comp rx=0 tx=0).stop 2024-09-10T08:10:52.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.101+0000 7f393fa00640 1 -- 172.21.15.2:0/164390635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3938074620 msgr2=0x7f393819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:52.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.101+0000 7f393fa00640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3938074620 0x7f393819a6b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f392c00e9f0 tx=0x7f392c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.101+0000 7f393fa00640 1 -- 172.21.15.2:0/164390635 shutdown_connections 2024-09-10T08:10:52.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.101+0000 7f393fa00640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f393810e3c0 0x7f393819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.101+0000 7f393fa00640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3938074f40 0x7f393819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.101+0000 7f393fa00640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f39200a2b80 0x7f39200a5040 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.101+0000 7f393fa00640 1 --2- 172.21.15.2:0/164390635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3938074620 0x7f393819a6b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:52.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.101+0000 7f393fa00640 1 -- 172.21.15.2:0/164390635 >> 172.21.15.2:0/164390635 conn(0x7f39380fd8c0 msgr2=0x7f39380fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:52.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.101+0000 7f393fa00640 1 -- 172.21.15.2:0/164390635 shutdown_connections 2024-09-10T08:10:52.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:10:52.101+0000 7f393fa00640 1 -- 172.21.15.2:0/164390635 wait complete. 2024-09-10T08:10:53.151 INFO:tasks.cephfs.filesystem:scrub status for tag:363d276b-2069-48d6-b0f7-acb2ffbdb07e - {'path': '/', 'tag': '363d276b-2069-48d6-b0f7-acb2ffbdb07e', 'options': 'recursive,force'} 2024-09-10T08:10:53.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:53 smithi002 ceph-mon[30563]: pgmap v469: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:10:53.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:53 smithi155 ceph-mon[35405]: pgmap v469: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:10:53.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:53 smithi107 ceph-mon[35048]: pgmap v469: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:10:54.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:54 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:10:54.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:54 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:10:54.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:54 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:10:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.953+0000 7f3972e00640 1 -- 172.21.15.2:0/2630895242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f396c076bb0 msgr2=0x7f396c076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.953+0000 7f3972e00640 1 --2- 172.21.15.2:0/2630895242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f396c076bb0 0x7f396c076fb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f395c009900 tx=0x7f395c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.953+0000 7f3972e00640 1 -- 172.21.15.2:0/2630895242 shutdown_connections 2024-09-10T08:10:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.953+0000 7f3972e00640 1 --2- 172.21.15.2:0/2630895242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396c078790 0x7f396c079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.953+0000 7f3972e00640 1 --2- 172.21.15.2:0/2630895242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f396c077df0 0x7f396c078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.953+0000 7f3972e00640 1 --2- 172.21.15.2:0/2630895242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f396c076bb0 0x7f396c076fb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.953+0000 7f3972e00640 1 -- 172.21.15.2:0/2630895242 >> 172.21.15.2:0/2630895242 conn(0x7f396c0ff4e0 msgr2=0x7f396c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.953+0000 7f3972e00640 1 -- 172.21.15.2:0/2630895242 shutdown_connections 2024-09-10T08:10:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.953+0000 7f3972e00640 1 -- 172.21.15.2:0/2630895242 wait complete. 2024-09-10T08:10:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.954+0000 7f3972e00640 1 Processor -- start 2024-09-10T08:10:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.954+0000 7f3972e00640 1 -- start start 2024-09-10T08:10:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.955+0000 7f3972e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f396c076bb0 0x7f396c19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.955+0000 7f3972e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396c077df0 0x7f396c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.955+0000 7f3972e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f396c078790 0x7f396c1a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.955+0000 7f3972e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f396c113e40 con 0x7f396c078790 2024-09-10T08:10:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.955+0000 7f3972e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f396c113cc0 con 0x7f396c076bb0 2024-09-10T08:10:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.955+0000 7f3972e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f396c113fc0 con 0x7f396c077df0 2024-09-10T08:10:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.955+0000 7f3970e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f396c078790 0x7f396c1a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.955+0000 7f396be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f396c076bb0 0x7f396c19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.955+0000 7f396be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f396c076bb0 0x7f396c19e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:56336/0 (socket says 172.21.15.2:56336) 2024-09-10T08:10:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.955+0000 7f396be00640 1 -- 172.21.15.2:0/1727772952 learned_addr learned my addr 172.21.15.2:0/1727772952 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.955+0000 7f3970e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f396c078790 0x7f396c1a5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44622/0 (socket says 172.21.15.2:44622) 2024-09-10T08:10:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.955+0000 7f3963e00640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396c077df0 0x7f396c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.956+0000 7f396be00640 1 -- 172.21.15.2:0/1727772952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396c077df0 msgr2=0x7f396c19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.956+0000 7f396be00640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396c077df0 0x7f396c19e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.956+0000 7f396be00640 1 -- 172.21.15.2:0/1727772952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f396c078790 msgr2=0x7f396c1a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.956+0000 7f396be00640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f396c078790 0x7f396c1a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.956+0000 7f396be00640 1 -- 172.21.15.2:0/1727772952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396c1a5ff0 con 0x7f396c076bb0 2024-09-10T08:10:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.956+0000 7f3970e00640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f396c078790 0x7f396c1a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.956+0000 7f396be00640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f396c076bb0 0x7f396c19e3d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f395c031440 tx=0x7f395c032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.956+0000 7f3969600640 1 -- 172.21.15.2:0/1727772952 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c03f030 con 0x7f396c076bb0 2024-09-10T08:10:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.956+0000 7f3969600640 1 -- 172.21.15.2:0/1727772952 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f395c003ce0 con 0x7f396c076bb0 2024-09-10T08:10:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.956+0000 7f3969600640 1 -- 172.21.15.2:0/1727772952 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c035070 con 0x7f396c076bb0 2024-09-10T08:10:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.956+0000 7f3972e00640 1 -- 172.21.15.2:0/1727772952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396c1a6280 con 0x7f396c076bb0 2024-09-10T08:10:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.957+0000 7f3972e00640 1 -- 172.21.15.2:0/1727772952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f396c1a6720 con 0x7f396c076bb0 2024-09-10T08:10:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.959+0000 7f3975c27740 1 Processor -- start 2024-09-10T08:10:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.959+0000 7f3975c27740 1 -- start start 2024-09-10T08:10:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.960+0000 7f3975c27740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a090 0x55f14b61a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.960+0000 7f3975c27740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a9d0 0x55f14b61f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.960+0000 7f3975c27740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61fa60 0x55f14b621f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.960+0000 7f3975c27740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f14b57f100 con 0x55f14b61a090 2024-09-10T08:10:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.960+0000 7f3975c27740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f14b5271b0 con 0x55f14b61fa60 2024-09-10T08:10:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.960+0000 7f3975c27740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f14b57b0c0 con 0x55f14b61a9d0 2024-09-10T08:10:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.960+0000 7f3969600640 1 -- 172.21.15.2:0/1727772952 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f395c042030 con 0x7f396c076bb0 2024-09-10T08:10:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.960+0000 7f3970e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a090 0x55f14b61a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.960+0000 7f3970e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a090 0x55f14b61a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44634/0 (socket says 172.21.15.2:44634) 2024-09-10T08:10:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.960+0000 7f3970e00640 1 -- 172.21.15.2:0/2389212474 learned_addr learned my addr 172.21.15.2:0/2389212474 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.960+0000 7f3963e00640 1 --2- 172.21.15.2:0/2389212474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61fa60 0x55f14b621f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.960+0000 7f396be00640 1 --2- 172.21.15.2:0/2389212474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a9d0 0x55f14b61f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.961+0000 7f3970e00640 1 -- 172.21.15.2:0/2389212474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a9d0 msgr2=0x55f14b61f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.961+0000 7f3970e00640 1 --2- 172.21.15.2:0/2389212474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a9d0 0x55f14b61f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.961+0000 7f3970e00640 1 -- 172.21.15.2:0/2389212474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61fa60 msgr2=0x55f14b621f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.961+0000 7f3970e00640 1 --2- 172.21.15.2:0/2389212474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61fa60 0x55f14b621f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.961+0000 7f3970e00640 1 -- 172.21.15.2:0/2389212474 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f14b292210 con 0x55f14b61a090 2024-09-10T08:10:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.961+0000 7f3970e00640 1 --2- 172.21.15.2:0/2389212474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a090 0x55f14b61a490 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f39640117e0 tx=0x7f3964011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f86d8473da78d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.961+0000 7f3961600640 1 -- 172.21.15.2:0/2389212474 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964011310 con 0x55f14b61a090 2024-09-10T08:10:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.961+0000 7f3961600640 1 -- 172.21.15.2:0/2389212474 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3964003e50 con 0x55f14b61a090 2024-09-10T08:10:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.961+0000 7f3961600640 1 -- 172.21.15.2:0/2389212474 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39640045b0 con 0x55f14b61a090 2024-09-10T08:10:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.962+0000 7f3969600640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f39500a2cb0 0x7f39500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.962+0000 7f3975c27740 1 -- 172.21.15.2:0/2389212474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a090 msgr2=0x55f14b61a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.962+0000 7f3975c27740 1 --2- 172.21.15.2:0/2389212474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a090 0x55f14b61a490 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f39640117e0 tx=0x7f3964011ca0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.962+0000 7f3975c27740 1 -- 172.21.15.2:0/2389212474 shutdown_connections 2024-09-10T08:10:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.962+0000 7f3975c27740 1 --2- 172.21.15.2:0/2389212474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61fa60 0x55f14b621f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.962+0000 7f3975c27740 1 --2- 172.21.15.2:0/2389212474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a9d0 0x55f14b61f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.962+0000 7f3975c27740 1 --2- 172.21.15.2:0/2389212474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a090 0x55f14b61a490 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.962+0000 7f3969600640 1 -- 172.21.15.2:0/1727772952 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f395c0ef860 con 0x7f396c076bb0 2024-09-10T08:10:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.962+0000 7f3975c27740 1 -- 172.21.15.2:0/2389212474 >> 172.21.15.2:0/2389212474 conn(0x55f14b615700 msgr2=0x55f14b615b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.962+0000 7f396be00640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f39500a2cb0 0x7f39500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.962+0000 7f3975c27740 1 -- 172.21.15.2:0/2389212474 shutdown_connections 2024-09-10T08:10:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.963+0000 7f3975c27740 1 -- 172.21.15.2:0/2389212474 wait complete. 2024-09-10T08:10:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.963+0000 7f396be00640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f39500a2cb0 0x7f39500a5170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f395c000c00 tx=0x7f395c02a3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.963+0000 7f3975c27740 1 -- start start 2024-09-10T08:10:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.963+0000 7f3975c27740 1 Processor -- start 2024-09-10T08:10:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3975c27740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a090 0x55f14b6242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3975c27740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 0x55f14b6247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3975c27740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61fa60 0x55f14b62b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3975c27740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f14b622a30 con 0x55f14b61a9d0 2024-09-10T08:10:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3975c27740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f14b508090 con 0x55f14b61fa60 2024-09-10T08:10:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3975c27740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f14b622bb0 con 0x55f14b61a090 2024-09-10T08:10:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3975c27740 10 client.0 _ll_register_callbacks cb 0x55f14b55cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3963e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 0x55f14b6247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3970e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a090 0x55f14b6242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3963e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 0x55f14b6247f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44644/0 (socket says 172.21.15.2:44644) 2024-09-10T08:10:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3963e00640 1 -- 172.21.15.2:0/4171157308 learned_addr learned my addr 172.21.15.2:0/4171157308 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3970e00640 1 --2- 172.21.15.2:0/4171157308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61fa60 0x55f14b62b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3963e00640 1 -- 172.21.15.2:0/4171157308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a090 msgr2=0x55f14b6242b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3963e00640 1 --2- 172.21.15.2:0/4171157308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a090 0x55f14b6242b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3963e00640 1 -- 172.21.15.2:0/4171157308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61fa60 msgr2=0x55f14b62b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3963e00640 1 --2- 172.21.15.2:0/4171157308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61fa60 0x55f14b62b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3963e00640 1 -- 172.21.15.2:0/4171157308 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f14b35e620 con 0x55f14b61a9d0 2024-09-10T08:10:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.964+0000 7f3970e00640 1 --2- 172.21.15.2:0/4171157308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a090 0x55f14b6242b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.965+0000 7f3963e00640 1 --2- 172.21.15.2:0/4171157308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 0x55f14b6247f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3954012700 tx=0x7f3954012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3100c79c105f3ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.965+0000 7f395a000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.965+0000 7f395a000640 1 -- 172.21.15.2:0/4171157308 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395400f8c0 con 0x55f14b61a9d0 2024-09-10T08:10:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.965+0000 7f3975c27740 1 -- 172.21.15.2:0/4171157308 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3964003e50 con 0x55f14b61a9d0 2024-09-10T08:10:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.965+0000 7f395a000640 1 -- 172.21.15.2:0/4171157308 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f395400fa60 con 0x55f14b61a9d0 2024-09-10T08:10:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.965+0000 7f395a000640 1 -- 172.21.15.2:0/4171157308 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f395400fc60 con 0x55f14b61a9d0 2024-09-10T08:10:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.965+0000 7f3975c27740 10 client.16101 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.965+0000 7f3975c27740 1 -- 172.21.15.2:0/4171157308 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f14b62e6d0 con 0x55f14b61a9d0 2024-09-10T08:10:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.966+0000 7f395a000640 1 -- 172.21.15.2:0/4171157308 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3964003e50 con 0x55f14b61a9d0 2024-09-10T08:10:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.966+0000 7f395a000640 1 -- 172.21.15.2:0/4171157308 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f395401c420 con 0x55f14b61a9d0 2024-09-10T08:10:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.966+0000 7f3975c27740 10 client.16101 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.966+0000 7f3975c27740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.966+0000 7f3975c27740 20 client.16101 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.966+0000 7f3975c27740 1 --2- 172.21.15.2:0/4171157308 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f14b62fdc0 0x55f14b632200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.966+0000 7f3975c27740 4 client.16101 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:10:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.966+0000 7f3975c27740 1 -- 172.21.15.2:0/4171157308 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f14b58c830 con 0x55f14b62fdc0 2024-09-10T08:10:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.966+0000 7f3970e00640 1 --2- 172.21.15.2:0/4171157308 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f14b62fdc0 0x55f14b632200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.967+0000 7f3970e00640 1 --2- 172.21.15.2:0/4171157308 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f14b62fdc0 0x55f14b632200 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b19e53185c4c51f7 server_cookie=a43bc1ce8d7566d8 in_seq=0 out_seq=0 2024-09-10T08:10:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.967+0000 7f395a000640 10 client.16101 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f395a000640 1 -- 172.21.15.2:0/4171157308 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f14b58c830 con 0x55f14b62fdc0 2024-09-10T08:10:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f395a000640 10 client.16101 handle_command_reply: tid=0 2024-09-10T08:10:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f3975c27740 1 client.16101 shutdown 2024-09-10T08:10:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f3975c27740 1 -- 172.21.15.2:0/4171157308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 msgr2=0x55f14b6247f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f3975c27740 1 --2- 172.21.15.2:0/4171157308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 0x55f14b6247f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3954012700 tx=0x7f3954012bc0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f3975c27740 1 -- 172.21.15.2:0/4171157308 shutdown_connections 2024-09-10T08:10:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f3975c27740 1 --2- 172.21.15.2:0/4171157308 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f14b62fdc0 0x55f14b632200 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f3975c27740 1 --2- 172.21.15.2:0/4171157308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61fa60 0x55f14b62b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f3975c27740 1 --2- 172.21.15.2:0/4171157308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 0x55f14b6247f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f3975c27740 1 --2- 172.21.15.2:0/4171157308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a090 0x55f14b6242b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f3975c27740 1 -- 172.21.15.2:0/4171157308 >> 172.21.15.2:0/4171157308 conn(0x55f14b615700 msgr2=0x55f14b616f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f395a000640 0 client.16101 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f3975c27740 1 -- 172.21.15.2:0/4171157308 shutdown_connections 2024-09-10T08:10:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f3975c27740 1 -- 172.21.15.2:0/4171157308 wait complete. 2024-09-10T08:10:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.968+0000 7f3975c27740 20 client.16101 trim_cache size 0 max 16384 2024-09-10T08:10:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.984+0000 7f3975c27740 1 Processor -- start 2024-09-10T08:10:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.984+0000 7f3975c27740 1 -- start start 2024-09-10T08:10:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.984+0000 7f3975c27740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a090 0x55f14b614770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.984+0000 7f3975c27740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61a9d0 0x55f14b62b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.984+0000 7f3975c27740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61fa60 0x55f14b636b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.984+0000 7f3975c27740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f396c1b36d0 con 0x55f14b61fa60 2024-09-10T08:10:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.984+0000 7f3975c27740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f396c1b0fa0 con 0x55f14b61a9d0 2024-09-10T08:10:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.984+0000 7f3975c27740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f396c1aedf0 con 0x55f14b61a090 2024-09-10T08:10:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.984+0000 7f3970e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61fa60 0x55f14b636b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.984+0000 7f3970e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61fa60 0x55f14b636b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44652/0 (socket says 172.21.15.2:44652) 2024-09-10T08:10:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.984+0000 7f3970e00640 1 -- 172.21.15.2:0/3468371125 learned_addr learned my addr 172.21.15.2:0/3468371125 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.984+0000 7f3963e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61a9d0 0x55f14b62b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.984+0000 7f3970e00640 1 --2- 172.21.15.2:0/3468371125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a090 0x55f14b614770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.985+0000 7f3963e00640 1 -- 172.21.15.2:0/3468371125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a090 msgr2=0x55f14b614770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.985+0000 7f3963e00640 1 --2- 172.21.15.2:0/3468371125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a090 0x55f14b614770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.985+0000 7f3963e00640 1 -- 172.21.15.2:0/3468371125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61fa60 msgr2=0x55f14b636b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.985+0000 7f3963e00640 1 --2- 172.21.15.2:0/3468371125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61fa60 0x55f14b636b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.985+0000 7f3963e00640 1 -- 172.21.15.2:0/3468371125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f14b615520 con 0x55f14b61a9d0 2024-09-10T08:10:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.985+0000 7f3970e00640 1 --2- 172.21.15.2:0/3468371125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61fa60 0x55f14b636b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:10:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.985+0000 7f3963e00640 1 --2- 172.21.15.2:0/3468371125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61a9d0 0x55f14b62b780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f395401c890 tx=0x7f395400ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=947fc81750564129 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.985+0000 7f3960c00640 1 -- 172.21.15.2:0/3468371125 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395401c420 con 0x55f14b61a9d0 2024-09-10T08:10:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.985+0000 7f3960c00640 1 -- 172.21.15.2:0/3468371125 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f395401c5c0 con 0x55f14b61a9d0 2024-09-10T08:10:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.985+0000 7f3960c00640 1 -- 172.21.15.2:0/3468371125 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954003850 con 0x55f14b61a9d0 2024-09-10T08:10:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.986+0000 7f3975c27740 1 -- 172.21.15.2:0/3468371125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61a9d0 msgr2=0x55f14b62b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.986+0000 7f3975c27740 1 --2- 172.21.15.2:0/3468371125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61a9d0 0x55f14b62b780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f395401c890 tx=0x7f395400ff90 comp rx=0 tx=0).stop 2024-09-10T08:10:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.986+0000 7f3975c27740 1 -- 172.21.15.2:0/3468371125 shutdown_connections 2024-09-10T08:10:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.986+0000 7f3975c27740 1 --2- 172.21.15.2:0/3468371125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61fa60 0x55f14b636b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.986+0000 7f3975c27740 1 --2- 172.21.15.2:0/3468371125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61a9d0 0x55f14b62b780 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.986+0000 7f3975c27740 1 --2- 172.21.15.2:0/3468371125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61a090 0x55f14b614770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.986+0000 7f3975c27740 1 -- 172.21.15.2:0/3468371125 >> 172.21.15.2:0/3468371125 conn(0x55f14b615700 msgr2=0x55f14b614060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.986+0000 7f3975c27740 1 -- 172.21.15.2:0/3468371125 shutdown_connections 2024-09-10T08:10:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.986+0000 7f3975c27740 1 -- 172.21.15.2:0/3468371125 wait complete. 2024-09-10T08:10:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.986+0000 7f3975c27740 1 -- start start 2024-09-10T08:10:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.986+0000 7f3975c27740 1 Processor -- start 2024-09-10T08:10:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.987+0000 7f3975c27740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61a090 0x55f14b628f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.987+0000 7f3975c27740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 0x55f14b629490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.987+0000 7f3975c27740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61fa60 0x55f14b62a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.987+0000 7f3975c27740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f396c1af460 con 0x55f14b61a9d0 2024-09-10T08:10:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.987+0000 7f3975c27740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f396c1b08d0 con 0x55f14b61a090 2024-09-10T08:10:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.987+0000 7f3975c27740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f396c1afb30 con 0x55f14b61fa60 2024-09-10T08:10:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.987+0000 7f3975c27740 10 client.0 _ll_register_callbacks cb 0x7f3954009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:10:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.987+0000 7f3963e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 0x55f14b629490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.987+0000 7f3963e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 0x55f14b629490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44668/0 (socket says 172.21.15.2:44668) 2024-09-10T08:10:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.987+0000 7f3963e00640 1 -- 172.21.15.2:0/2856821128 learned_addr learned my addr 172.21.15.2:0/2856821128 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:10:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.987+0000 7f3970e00640 1 --2- 172.21.15.2:0/2856821128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61fa60 0x55f14b62a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.987+0000 7f3970e00640 1 --2- 172.21.15.2:0/2856821128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61a090 0x55f14b628f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.988+0000 7f3963e00640 1 -- 172.21.15.2:0/2856821128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61fa60 msgr2=0x55f14b62a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.988+0000 7f3963e00640 1 --2- 172.21.15.2:0/2856821128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61fa60 0x55f14b62a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.988+0000 7f3963e00640 1 -- 172.21.15.2:0/2856821128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61a090 msgr2=0x55f14b628f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:10:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.988+0000 7f3963e00640 1 --2- 172.21.15.2:0/2856821128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61a090 0x55f14b628f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.988+0000 7f3963e00640 1 -- 172.21.15.2:0/2856821128 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f14b62aa60 con 0x55f14b61a9d0 2024-09-10T08:10:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.988+0000 7f3963e00640 1 --2- 172.21.15.2:0/2856821128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 0x55f14b629490 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3954009f70 tx=0x7f3954005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19613aac9721970c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:10:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.988+0000 7f395a000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:10:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.988+0000 7f395a000640 1 -- 172.21.15.2:0/2856821128 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954039060 con 0x55f14b61a9d0 2024-09-10T08:10:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.988+0000 7f3975c27740 1 -- 172.21.15.2:0/2856821128 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f395401c5c0 con 0x55f14b61a9d0 2024-09-10T08:10:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.988+0000 7f395a000640 1 -- 172.21.15.2:0/2856821128 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3954018040 con 0x55f14b61a9d0 2024-09-10T08:10:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.988+0000 7f395a000640 1 -- 172.21.15.2:0/2856821128 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3954040050 con 0x55f14b61a9d0 2024-09-10T08:10:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.988+0000 7f3975c27740 10 client.16107 fetch_fsmap learned FSMap version 26 2024-09-10T08:10:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.988+0000 7f3975c27740 1 -- 172.21.15.2:0/2856821128 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f14b69aae0 con 0x55f14b61a9d0 2024-09-10T08:10:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.989+0000 7f395a000640 1 -- 172.21.15.2:0/2856821128 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f395401c5c0 con 0x55f14b61a9d0 2024-09-10T08:10:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.989+0000 7f395a000640 1 -- 172.21.15.2:0/2856821128 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3954034450 con 0x55f14b61a9d0 2024-09-10T08:10:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.989+0000 7f3975c27740 10 client.16107 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:10:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.989+0000 7f3975c27740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:10:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.989+0000 7f3975c27740 20 client.16107 populate_metadata read hostname 'smithi002' 2024-09-10T08:10:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.989+0000 7f3975c27740 1 --2- 172.21.15.2:0/2856821128 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f14b69b370 0x55f14b69d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:10:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.989+0000 7f3975c27740 4 client.16107 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:10:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.989+0000 7f3975c27740 1 -- 172.21.15.2:0/2856821128 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f14b586b80 con 0x55f14b69b370 2024-09-10T08:10:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.989+0000 7f3970e00640 1 --2- 172.21.15.2:0/2856821128 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f14b69b370 0x55f14b69d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:10:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.990+0000 7f3970e00640 1 --2- 172.21.15.2:0/2856821128 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f14b69b370 0x55f14b69d760 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5443168cb02d112 server_cookie=d672b5fc304c3cf3 in_seq=0 out_seq=0 2024-09-10T08:10:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:54.990+0000 7f395a000640 10 client.16107 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.002+0000 7f395a000640 1 -- 172.21.15.2:0/2856821128 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f14b586b80 con 0x55f14b69b370 2024-09-10T08:10:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.002+0000 7f395a000640 10 client.16107 handle_command_reply: tid=0 2024-09-10T08:10:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.002+0000 7f3975c27740 1 client.16107 shutdown 2024-09-10T08:10:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.003+0000 7f3975c27740 1 -- 172.21.15.2:0/2856821128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 msgr2=0x55f14b629490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.003+0000 7f3975c27740 1 --2- 172.21.15.2:0/2856821128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 0x55f14b629490 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3954009f70 tx=0x7f3954005750 comp rx=0 tx=0).stop 2024-09-10T08:10:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.003+0000 7f3975c27740 1 -- 172.21.15.2:0/2856821128 shutdown_connections 2024-09-10T08:10:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.003+0000 7f3975c27740 1 --2- 172.21.15.2:0/2856821128 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f14b69b370 0x55f14b69d760 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.003+0000 7f3975c27740 1 --2- 172.21.15.2:0/2856821128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14b61fa60 0x55f14b62a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.003+0000 7f3975c27740 1 --2- 172.21.15.2:0/2856821128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f14b61a9d0 0x55f14b629490 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.003+0000 7f3975c27740 1 --2- 172.21.15.2:0/2856821128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f14b61a090 0x55f14b628f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.003+0000 7f395a000640 0 client.16107 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:10:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.003+0000 7f3975c27740 1 -- 172.21.15.2:0/2856821128 >> 172.21.15.2:0/2856821128 conn(0x55f14b615700 msgr2=0x55f14b614880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.003+0000 7f3975c27740 1 -- 172.21.15.2:0/2856821128 shutdown_connections 2024-09-10T08:10:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.003+0000 7f3975c27740 1 -- 172.21.15.2:0/2856821128 wait complete. 2024-09-10T08:10:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.003+0000 7f3975c27740 20 client.16107 trim_cache size 0 max 16384 2024-09-10T08:10:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:10:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:10:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:10:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:10:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.004+0000 7f3972e00640 1 -- 172.21.15.2:0/1727772952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f39500a2cb0 msgr2=0x7f39500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.004+0000 7f3972e00640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f39500a2cb0 0x7f39500a5170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f395c000c00 tx=0x7f395c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:10:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.004+0000 7f3972e00640 1 -- 172.21.15.2:0/1727772952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f396c076bb0 msgr2=0x7f396c19e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:10:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.004+0000 7f3972e00640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f396c076bb0 0x7f396c19e3d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f395c031440 tx=0x7f395c032ce0 comp rx=0 tx=0).stop 2024-09-10T08:10:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.004+0000 7f3972e00640 1 -- 172.21.15.2:0/1727772952 shutdown_connections 2024-09-10T08:10:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.004+0000 7f3972e00640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f396c078790 0x7f396c1a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.004+0000 7f3972e00640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f39500a2cb0 0x7f39500a5170 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.004+0000 7f3972e00640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396c077df0 0x7f396c19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.004+0000 7f3972e00640 1 --2- 172.21.15.2:0/1727772952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f396c076bb0 0x7f396c19e3d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:10:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.004+0000 7f3972e00640 1 -- 172.21.15.2:0/1727772952 >> 172.21.15.2:0/1727772952 conn(0x7f396c0ff4e0 msgr2=0x7f396c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:10:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.004+0000 7f3972e00640 1 -- 172.21.15.2:0/1727772952 shutdown_connections 2024-09-10T08:10:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:10:55.004+0000 7f3972e00640 1 -- 172.21.15.2:0/1727772952 wait complete. 2024-09-10T08:10:55.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:55 smithi002 ceph-mon[30563]: pgmap v470: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:10:55.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:55 smithi155 ceph-mon[35405]: pgmap v470: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:10:55.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:55 smithi107 ceph-mon[35048]: pgmap v470: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:10:57.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:57 smithi002 ceph-mon[30563]: pgmap v471: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-10T08:10:57.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:57 smithi155 ceph-mon[35405]: pgmap v471: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-10T08:10:57.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:57 smithi107 ceph-mon[35048]: pgmap v471: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-10T08:10:59.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:10:59 smithi002 ceph-mon[30563]: pgmap v472: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:10:59.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:10:59 smithi155 ceph-mon[35405]: pgmap v472: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:10:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:10:59 smithi107 ceph-mon[35048]: pgmap v472: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:11:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:01 smithi002 ceph-mon[30563]: pgmap v473: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:11:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:01 smithi155 ceph-mon[35405]: pgmap v473: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:11:01.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:01 smithi107 ceph-mon[35048]: pgmap v473: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:11:03.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:03 smithi155 ceph-mon[35405]: pgmap v474: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:11:03.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:03 smithi107 ceph-mon[35048]: pgmap v474: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:11:03.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:03 smithi002 ceph-mon[30563]: pgmap v474: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:11:05.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:05 smithi155 ceph-mon[35405]: pgmap v475: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:11:05.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:05 smithi107 ceph-mon[35048]: pgmap v475: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:11:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:05 smithi002 ceph-mon[30563]: pgmap v475: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:11:07.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:07 smithi155 ceph-mon[35405]: pgmap v476: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:07.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:07 smithi107 ceph-mon[35048]: pgmap v476: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:07.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:07 smithi002 ceph-mon[30563]: pgmap v476: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.132+0000 7fc3f1e00640 1 -- 172.21.15.2:0/1432292015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ec077df0 msgr2=0x7fc3ec078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.132+0000 7fc3f1e00640 1 --2- 172.21.15.2:0/1432292015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ec077df0 0x7fc3ec078250 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009900 tx=0x7fc3d802a410 comp rx=0 tx=0).stop 2024-09-10T08:11:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.132+0000 7fc3f1e00640 1 -- 172.21.15.2:0/1432292015 shutdown_connections 2024-09-10T08:11:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.132+0000 7fc3f1e00640 1 --2- 172.21.15.2:0/1432292015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3ec078790 0x7fc3ec079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.132+0000 7fc3f1e00640 1 --2- 172.21.15.2:0/1432292015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ec077df0 0x7fc3ec078250 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.132+0000 7fc3f1e00640 1 --2- 172.21.15.2:0/1432292015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ec076bb0 0x7fc3ec076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.132+0000 7fc3f1e00640 1 -- 172.21.15.2:0/1432292015 >> 172.21.15.2:0/1432292015 conn(0x7fc3ec0ff2f0 msgr2=0x7fc3ec101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.132+0000 7fc3f1e00640 1 -- 172.21.15.2:0/1432292015 shutdown_connections 2024-09-10T08:11:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.132+0000 7fc3f1e00640 1 -- 172.21.15.2:0/1432292015 wait complete. 2024-09-10T08:11:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.133+0000 7fc3f1e00640 1 Processor -- start 2024-09-10T08:11:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.133+0000 7fc3f1e00640 1 -- start start 2024-09-10T08:11:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.133+0000 7fc3f1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3ec076bb0 0x7fc3ec19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.133+0000 7fc3f1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ec077df0 0x7fc3ec19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.133+0000 7fc3f1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ec078790 0x7fc3ec1a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.133+0000 7fc3f1e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec113c50 con 0x7fc3ec078790 2024-09-10T08:11:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.133+0000 7fc3f1e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec113ad0 con 0x7fc3ec076bb0 2024-09-10T08:11:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.133+0000 7fc3f1e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec113dd0 con 0x7fc3ec077df0 2024-09-10T08:11:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.133+0000 7fc3f1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ec078790 0x7fc3ec1a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.134+0000 7fc3f0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3ec076bb0 0x7fc3ec19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.134+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ec077df0 0x7fc3ec19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.134+0000 7fc3f1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ec078790 0x7fc3ec1a5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53556/0 (socket says 172.21.15.2:53556) 2024-09-10T08:11:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.134+0000 7fc3f1400640 1 -- 172.21.15.2:0/3588446182 learned_addr learned my addr 172.21.15.2:0/3588446182 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.134+0000 7fc3f1400640 1 -- 172.21.15.2:0/3588446182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ec077df0 msgr2=0x7fc3ec19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.134+0000 7fc3f1400640 1 --2- 172.21.15.2:0/3588446182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ec077df0 0x7fc3ec19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.134+0000 7fc3f1400640 1 -- 172.21.15.2:0/3588446182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3ec076bb0 msgr2=0x7fc3ec19e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.134+0000 7fc3f1400640 1 --2- 172.21.15.2:0/3588446182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3ec076bb0 0x7fc3ec19e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.134+0000 7fc3f1400640 1 -- 172.21.15.2:0/3588446182 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec1a5eb0 con 0x7fc3ec078790 2024-09-10T08:11:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.134+0000 7fc3f0a00640 1 --2- 172.21.15.2:0/3588446182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3ec076bb0 0x7fc3ec19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.134+0000 7fc3f1400640 1 --2- 172.21.15.2:0/3588446182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ec078790 0x7fc3ec1a5840 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400eb10 tx=0x7fc3e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.134+0000 7fc3e9600640 1 -- 172.21.15.2:0/3588446182 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4004b20 con 0x7fc3ec078790 2024-09-10T08:11:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.135+0000 7fc3e9600640 1 -- 172.21.15.2:0/3588446182 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e4004cc0 con 0x7fc3ec078790 2024-09-10T08:11:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.135+0000 7fc3e9600640 1 -- 172.21.15.2:0/3588446182 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4010760 con 0x7fc3ec078790 2024-09-10T08:11:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.135+0000 7fc3f1e00640 1 -- 172.21.15.2:0/3588446182 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec1a61a0 con 0x7fc3ec078790 2024-09-10T08:11:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.135+0000 7fc3f1e00640 1 -- 172.21.15.2:0/3588446182 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec1a66c0 con 0x7fc3ec078790 2024-09-10T08:11:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.138+0000 7fc3f6441740 1 Processor -- start 2024-09-10T08:11:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.138+0000 7fc3e9600640 1 -- 172.21.15.2:0/3588446182 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc3e4005020 con 0x7fc3ec078790 2024-09-10T08:11:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.138+0000 7fc3f6441740 1 -- start start 2024-09-10T08:11:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.138+0000 7fc3f6441740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92a260 0x563f4d92a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.138+0000 7fc3f6441740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d92aba0 0x563f4d92f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.138+0000 7fc3f6441740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92fc30 0x563f4d9320f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.138+0000 7fc3f6441740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563f4d88f100 con 0x563f4d92fc30 2024-09-10T08:11:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.138+0000 7fc3f6441740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f4d8371b0 con 0x563f4d92aba0 2024-09-10T08:11:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.138+0000 7fc3f6441740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563f4d88b0c0 con 0x563f4d92a260 2024-09-10T08:11:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.139+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92a260 0x563f4d92a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.139+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92a260 0x563f4d92a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:43912/0 (socket says 172.21.15.2:43912) 2024-09-10T08:11:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.139+0000 7fc3ebe00640 1 -- 172.21.15.2:0/3293384030 learned_addr learned my addr 172.21.15.2:0/3293384030 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.139+0000 7fc3f0a00640 1 --2- 172.21.15.2:0/3293384030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d92aba0 0x563f4d92f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.139+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/3293384030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92fc30 0x563f4d9320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.139+0000 7fc3ebe00640 1 -- 172.21.15.2:0/3293384030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d92aba0 msgr2=0x563f4d92f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.139+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/3293384030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d92aba0 0x563f4d92f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.139+0000 7fc3ebe00640 1 -- 172.21.15.2:0/3293384030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92fc30 msgr2=0x563f4d9320f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.139+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/3293384030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92fc30 0x563f4d9320f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.139+0000 7fc3ebe00640 1 -- 172.21.15.2:0/3293384030 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f4d5a2210 con 0x563f4d92a260 2024-09-10T08:11:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.139+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/3293384030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92a260 0x563f4d92a660 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80314e0 tx=0x7fc3d8005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f26f0afbe6bbeba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.140+0000 7fc3e9600640 1 --2- 172.21.15.2:0/3588446182 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3d40a2c00 0x7fc3d40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.140+0000 7fc3e9600640 1 -- 172.21.15.2:0/3588446182 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc3e40cf9c0 con 0x7fc3ec078790 2024-09-10T08:11:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.140+0000 7fc3f0a00640 1 --2- 172.21.15.2:0/3588446182 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3d40a2c00 0x7fc3d40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.140+0000 7fc3ddc00640 1 -- 172.21.15.2:0/3293384030 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8035070 con 0x563f4d92a260 2024-09-10T08:11:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.140+0000 7fc3ddc00640 1 -- 172.21.15.2:0/3293384030 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d8032e50 con 0x563f4d92a260 2024-09-10T08:11:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.140+0000 7fc3ddc00640 1 -- 172.21.15.2:0/3293384030 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d803d450 con 0x563f4d92a260 2024-09-10T08:11:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.141+0000 7fc3f6441740 1 -- 172.21.15.2:0/3293384030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92a260 msgr2=0x563f4d92a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.141+0000 7fc3f6441740 1 --2- 172.21.15.2:0/3293384030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92a260 0x563f4d92a660 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80314e0 tx=0x7fc3d8005780 comp rx=0 tx=0).stop 2024-09-10T08:11:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.141+0000 7fc3f0a00640 1 --2- 172.21.15.2:0/3588446182 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3d40a2c00 0x7fc3d40a50c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x563f4d92b5d0 tx=0x7fc3e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.141+0000 7fc3f6441740 1 -- 172.21.15.2:0/3293384030 shutdown_connections 2024-09-10T08:11:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.141+0000 7fc3f6441740 1 --2- 172.21.15.2:0/3293384030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92fc30 0x563f4d9320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.141+0000 7fc3f6441740 1 --2- 172.21.15.2:0/3293384030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d92aba0 0x563f4d92f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.141+0000 7fc3f6441740 1 --2- 172.21.15.2:0/3293384030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92a260 0x563f4d92a660 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.141+0000 7fc3f6441740 1 -- 172.21.15.2:0/3293384030 >> 172.21.15.2:0/3293384030 conn(0x563f4d9258d0 msgr2=0x563f4d925cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.141+0000 7fc3f6441740 1 -- 172.21.15.2:0/3293384030 shutdown_connections 2024-09-10T08:11:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.141+0000 7fc3f6441740 1 -- 172.21.15.2:0/3293384030 wait complete. 2024-09-10T08:11:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.141+0000 7fc3f6441740 1 -- start start 2024-09-10T08:11:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.141+0000 7fc3f6441740 1 Processor -- start 2024-09-10T08:11:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.142+0000 7fc3f6441740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d9343e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.142+0000 7fc3f6441740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d92aba0 0x563f4d934920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.142+0000 7fc3f6441740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92fc30 0x563f4d93b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.142+0000 7fc3f6441740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563f4d932ba0 con 0x563f4d92a260 2024-09-10T08:11:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.142+0000 7fc3f6441740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f4d818090 con 0x563f4d92aba0 2024-09-10T08:11:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.142+0000 7fc3f6441740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563f4d932d20 con 0x563f4d92fc30 2024-09-10T08:11:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.142+0000 7fc3f6441740 10 client.0 _ll_register_callbacks cb 0x563f4d86cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.142+0000 7fc3f0a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d9343e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.142+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d92aba0 0x563f4d934920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.142+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d92aba0 0x563f4d934920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54328/0 (socket says 172.21.15.2:54328) 2024-09-10T08:11:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.142+0000 7fc3ebe00640 1 -- 172.21.15.2:0/2314355354 learned_addr learned my addr 172.21.15.2:0/2314355354 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.142+0000 7fc3f1400640 1 --2- 172.21.15.2:0/2314355354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92fc30 0x563f4d93b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.143+0000 7fc3ebe00640 1 -- 172.21.15.2:0/2314355354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92fc30 msgr2=0x563f4d93b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.143+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/2314355354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92fc30 0x563f4d93b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.143+0000 7fc3ebe00640 1 -- 172.21.15.2:0/2314355354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 msgr2=0x563f4d9343e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.143+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/2314355354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d9343e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.143+0000 7fc3ebe00640 1 -- 172.21.15.2:0/2314355354 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f4d66e620 con 0x563f4d92aba0 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.143+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/2314355354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d92aba0 0x563f4d934920 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8006010 tx=0x7fc3d802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef7be1f29e722f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.143+0000 7fc3d2000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.143+0000 7fc3d2000640 1 -- 172.21.15.2:0/2314355354 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d804f070 con 0x563f4d92aba0 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.143+0000 7fc3f6441740 1 -- 172.21.15.2:0/2314355354 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d8032e50 con 0x563f4d92aba0 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.143+0000 7fc3d2000640 1 -- 172.21.15.2:0/2314355354 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc3d8032770 con 0x563f4d92aba0 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.143+0000 7fc3d2000640 1 -- 172.21.15.2:0/2314355354 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc3d8035040 con 0x563f4d92aba0 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.144+0000 7fc3f6441740 10 client.26153 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.144+0000 7fc3f6441740 1 -- 172.21.15.2:0/2314355354 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563f4d93e750 con 0x563f4d92aba0 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.144+0000 7fc3d2000640 1 -- 172.21.15.2:0/2314355354 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d8032e50 con 0x563f4d92aba0 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.144+0000 7fc3d2000640 1 -- 172.21.15.2:0/2314355354 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc3d8049070 con 0x563f4d92aba0 2024-09-10T08:11:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.144+0000 7fc3f6441740 10 client.26153 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.144+0000 7fc3f6441740 10 client.26153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.144+0000 7fc3f6441740 20 client.26153 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.144+0000 7fc3f6441740 1 --2- 172.21.15.2:0/2314355354 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563f4d93faa0 0x563f4d941e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.144+0000 7fc3f6441740 4 client.26153 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:11:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.144+0000 7fc3f6441740 1 -- 172.21.15.2:0/2314355354 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f4d89c830 con 0x563f4d93faa0 2024-09-10T08:11:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.145+0000 7fc3f0a00640 1 --2- 172.21.15.2:0/2314355354 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563f4d93faa0 0x563f4d941e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.145+0000 7fc3f0a00640 1 --2- 172.21.15.2:0/2314355354 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563f4d93faa0 0x563f4d941e90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad670775d8d0998 server_cookie=72dc969514d38422 in_seq=0 out_seq=0 2024-09-10T08:11:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.145+0000 7fc3d2000640 10 client.26153 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3d2000640 1 -- 172.21.15.2:0/2314355354 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f4d89c830 con 0x563f4d93faa0 2024-09-10T08:11:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3d2000640 10 client.26153 handle_command_reply: tid=0 2024-09-10T08:11:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3f6441740 1 client.26153 shutdown 2024-09-10T08:11:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3f6441740 1 -- 172.21.15.2:0/2314355354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d92aba0 msgr2=0x563f4d934920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3f6441740 1 --2- 172.21.15.2:0/2314355354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d92aba0 0x563f4d934920 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8006010 tx=0x7fc3d802a410 comp rx=0 tx=0).stop 2024-09-10T08:11:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3f6441740 1 -- 172.21.15.2:0/2314355354 shutdown_connections 2024-09-10T08:11:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3f6441740 1 --2- 172.21.15.2:0/2314355354 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563f4d93faa0 0x563f4d941e90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3f6441740 1 --2- 172.21.15.2:0/2314355354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92fc30 0x563f4d93b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3f6441740 1 --2- 172.21.15.2:0/2314355354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d92aba0 0x563f4d934920 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3f6441740 1 --2- 172.21.15.2:0/2314355354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d9343e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3f6441740 1 -- 172.21.15.2:0/2314355354 >> 172.21.15.2:0/2314355354 conn(0x563f4d9258d0 msgr2=0x563f4d9270e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3d2000640 0 client.26153 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3f6441740 1 -- 172.21.15.2:0/2314355354 shutdown_connections 2024-09-10T08:11:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.146+0000 7fc3f6441740 1 -- 172.21.15.2:0/2314355354 wait complete. 2024-09-10T08:11:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.147+0000 7fc3f6441740 20 client.26153 trim_cache size 0 max 16384 2024-09-10T08:11:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.162+0000 7fc3f6441740 1 Processor -- start 2024-09-10T08:11:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.162+0000 7fc3f6441740 1 -- start start 2024-09-10T08:11:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.162+0000 7fc3f6441740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d9258d0 0x563f4d924900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.162+0000 7fc3f6441740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d93b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.162+0000 7fc3f6441740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92aba0 0x563f4d9465a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.162+0000 7fc3f6441740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b35d0 con 0x563f4d92a260 2024-09-10T08:11:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.162+0000 7fc3f6441740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b0ea0 con 0x563f4d9258d0 2024-09-10T08:11:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.162+0000 7fc3f6441740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1aecf0 con 0x563f4d92aba0 2024-09-10T08:11:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.162+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d93b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.162+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d93b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53588/0 (socket says 172.21.15.2:53588) 2024-09-10T08:11:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.162+0000 7fc3ebe00640 1 -- 172.21.15.2:0/1286107244 learned_addr learned my addr 172.21.15.2:0/1286107244 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.162+0000 7fc3f0a00640 1 --2- 172.21.15.2:0/1286107244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d9258d0 0x563f4d924900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.162+0000 7fc3f1400640 1 --2- 172.21.15.2:0/1286107244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92aba0 0x563f4d9465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.163+0000 7fc3ebe00640 1 -- 172.21.15.2:0/1286107244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92aba0 msgr2=0x563f4d9465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.163+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/1286107244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92aba0 0x563f4d9465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.163+0000 7fc3ebe00640 1 -- 172.21.15.2:0/1286107244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d9258d0 msgr2=0x563f4d924900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.163+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/1286107244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d9258d0 0x563f4d924900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.163+0000 7fc3ebe00640 1 -- 172.21.15.2:0/1286107244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f4d933f80 con 0x563f4d92a260 2024-09-10T08:11:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.163+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/1286107244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d93b920 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80314e0 tx=0x7fc3d8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3775a650bc1e429a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.163+0000 7fc3dd200640 1 -- 172.21.15.2:0/1286107244 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d804f040 con 0x563f4d92a260 2024-09-10T08:11:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.163+0000 7fc3dd200640 1 -- 172.21.15.2:0/1286107244 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d8003f10 con 0x563f4d92a260 2024-09-10T08:11:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.163+0000 7fc3dd200640 1 -- 172.21.15.2:0/1286107244 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8031a20 con 0x563f4d92a260 2024-09-10T08:11:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.164+0000 7fc3f6441740 1 -- 172.21.15.2:0/1286107244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 msgr2=0x563f4d93b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.164+0000 7fc3f6441740 1 --2- 172.21.15.2:0/1286107244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d93b920 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80314e0 tx=0x7fc3d8001e70 comp rx=0 tx=0).stop 2024-09-10T08:11:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.164+0000 7fc3f6441740 1 -- 172.21.15.2:0/1286107244 shutdown_connections 2024-09-10T08:11:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.164+0000 7fc3f6441740 1 --2- 172.21.15.2:0/1286107244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92aba0 0x563f4d9465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.164+0000 7fc3f6441740 1 --2- 172.21.15.2:0/1286107244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d93b920 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.164+0000 7fc3f6441740 1 --2- 172.21.15.2:0/1286107244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d9258d0 0x563f4d924900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.164+0000 7fc3f6441740 1 -- 172.21.15.2:0/1286107244 >> 172.21.15.2:0/1286107244 conn(0x563f4d92f780 msgr2=0x563f4d9241f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.164+0000 7fc3f6441740 1 -- 172.21.15.2:0/1286107244 shutdown_connections 2024-09-10T08:11:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.164+0000 7fc3f6441740 1 -- 172.21.15.2:0/1286107244 wait complete. 2024-09-10T08:11:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.164+0000 7fc3f6441740 1 -- start start 2024-09-10T08:11:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.165+0000 7fc3f6441740 1 Processor -- start 2024-09-10T08:11:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.165+0000 7fc3f6441740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d9258d0 0x563f4d9394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.165+0000 7fc3f6441740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d939a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.165+0000 7fc3f6441740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92aba0 0x563f4d93a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.165+0000 7fc3f6441740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1af360 con 0x563f4d92a260 2024-09-10T08:11:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.165+0000 7fc3f6441740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b07d0 con 0x563f4d9258d0 2024-09-10T08:11:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.165+0000 7fc3f6441740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1afa30 con 0x563f4d92aba0 2024-09-10T08:11:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.165+0000 7fc3f6441740 10 client.0 _ll_register_callbacks cb 0x7fc3d8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.165+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d939a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.165+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d939a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53600/0 (socket says 172.21.15.2:53600) 2024-09-10T08:11:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.165+0000 7fc3f1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92aba0 0x563f4d93a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.165+0000 7fc3ebe00640 1 -- 172.21.15.2:0/4083465296 learned_addr learned my addr 172.21.15.2:0/4083465296 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.165+0000 7fc3f0a00640 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d9258d0 0x563f4d9394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.166+0000 7fc3ebe00640 1 -- 172.21.15.2:0/4083465296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92aba0 msgr2=0x563f4d93a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.166+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92aba0 0x563f4d93a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.166+0000 7fc3ebe00640 1 -- 172.21.15.2:0/4083465296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d9258d0 msgr2=0x563f4d9394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.166+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d9258d0 0x563f4d9394c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.166+0000 7fc3ebe00640 1 -- 172.21.15.2:0/4083465296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f4d93afb0 con 0x563f4d92a260 2024-09-10T08:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.166+0000 7fc3f1400640 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92aba0 0x563f4d93a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.166+0000 7fc3f0a00640 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d9258d0 0x563f4d9394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.166+0000 7fc3ebe00640 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d939a00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc3d804bbf0 tx=0x7fc3d8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35abfdda86d89df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.166+0000 7fc3d2000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.166+0000 7fc3d2000640 1 -- 172.21.15.2:0/4083465296 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d804f040 con 0x563f4d92a260 2024-09-10T08:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.166+0000 7fc3d2000640 1 -- 172.21.15.2:0/4083465296 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc3d8005040 con 0x563f4d92a260 2024-09-10T08:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.166+0000 7fc3f6441740 1 -- 172.21.15.2:0/4083465296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d8003f10 con 0x563f4d92a260 2024-09-10T08:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.167+0000 7fc3d2000640 1 -- 172.21.15.2:0/4083465296 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc3d806e050 con 0x563f4d92a260 2024-09-10T08:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.167+0000 7fc3f6441740 10 client.16125 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.167+0000 7fc3f6441740 1 -- 172.21.15.2:0/4083465296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563f4d9aa9f0 con 0x563f4d92a260 2024-09-10T08:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.167+0000 7fc3d2000640 1 -- 172.21.15.2:0/4083465296 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d8003f10 con 0x563f4d92a260 2024-09-10T08:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.167+0000 7fc3d2000640 1 -- 172.21.15.2:0/4083465296 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc3d8060030 con 0x563f4d92a260 2024-09-10T08:11:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.167+0000 7fc3f6441740 10 client.16125 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.167+0000 7fc3f6441740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.167+0000 7fc3f6441740 20 client.16125 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.167+0000 7fc3f6441740 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563f4d92fc30 0x563f4d9ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.167+0000 7fc3f6441740 4 client.16125 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:11:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.167+0000 7fc3f6441740 1 -- 172.21.15.2:0/4083465296 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f4d896b80 con 0x563f4d92fc30 2024-09-10T08:11:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.168+0000 7fc3f0a00640 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563f4d92fc30 0x563f4d9ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.168+0000 7fc3f0a00640 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563f4d92fc30 0x563f4d9ad220 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93f5d76b924eaae3 server_cookie=1f4033d76316a480 in_seq=0 out_seq=0 2024-09-10T08:11:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.168+0000 7fc3d2000640 10 client.16125 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.180+0000 7fc3d2000640 1 -- 172.21.15.2:0/4083465296 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f4d896b80 con 0x563f4d92fc30 2024-09-10T08:11:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.180+0000 7fc3d2000640 10 client.16125 handle_command_reply: tid=0 2024-09-10T08:11:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.180+0000 7fc3f6441740 1 client.16125 shutdown 2024-09-10T08:11:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.181+0000 7fc3f6441740 1 -- 172.21.15.2:0/4083465296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 msgr2=0x563f4d939a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.181+0000 7fc3f6441740 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d939a00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc3d804bbf0 tx=0x7fc3d8031400 comp rx=0 tx=0).stop 2024-09-10T08:11:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.181+0000 7fc3f6441740 1 -- 172.21.15.2:0/4083465296 shutdown_connections 2024-09-10T08:11:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.181+0000 7fc3f6441740 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563f4d92fc30 0x563f4d9ad220 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.181+0000 7fc3f6441740 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f4d92aba0 0x563f4d93a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.181+0000 7fc3f6441740 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f4d92a260 0x563f4d939a00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.181+0000 7fc3f6441740 1 --2- 172.21.15.2:0/4083465296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4d9258d0 0x563f4d9394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.181+0000 7fc3d2000640 0 client.16125 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.181+0000 7fc3f6441740 1 -- 172.21.15.2:0/4083465296 >> 172.21.15.2:0/4083465296 conn(0x563f4d92f780 msgr2=0x563f4d926fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.181+0000 7fc3f6441740 1 -- 172.21.15.2:0/4083465296 shutdown_connections 2024-09-10T08:11:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.181+0000 7fc3f6441740 1 -- 172.21.15.2:0/4083465296 wait complete. 2024-09-10T08:11:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.181+0000 7fc3f6441740 20 client.16125 trim_cache size 0 max 16384 2024-09-10T08:11:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:11:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:11:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:11:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:11:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.182+0000 7fc3f1e00640 1 -- 172.21.15.2:0/3588446182 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3d40a2c00 msgr2=0x7fc3d40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.182+0000 7fc3f1e00640 1 --2- 172.21.15.2:0/3588446182 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3d40a2c00 0x7fc3d40a50c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x563f4d92b5d0 tx=0x7fc3e0005f90 comp rx=0 tx=0).stop 2024-09-10T08:11:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.182+0000 7fc3f1e00640 1 -- 172.21.15.2:0/3588446182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ec078790 msgr2=0x7fc3ec1a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.182+0000 7fc3f1e00640 1 --2- 172.21.15.2:0/3588446182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ec078790 0x7fc3ec1a5840 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400eb10 tx=0x7fc3e400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.182+0000 7fc3f1e00640 1 -- 172.21.15.2:0/3588446182 shutdown_connections 2024-09-10T08:11:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.182+0000 7fc3f1e00640 1 --2- 172.21.15.2:0/3588446182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3ec078790 0x7fc3ec1a5840 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.182+0000 7fc3f1e00640 1 --2- 172.21.15.2:0/3588446182 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3d40a2c00 0x7fc3d40a50c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.182+0000 7fc3f1e00640 1 --2- 172.21.15.2:0/3588446182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ec077df0 0x7fc3ec19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.182+0000 7fc3f1e00640 1 --2- 172.21.15.2:0/3588446182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3ec076bb0 0x7fc3ec19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.182+0000 7fc3f1e00640 1 -- 172.21.15.2:0/3588446182 >> 172.21.15.2:0/3588446182 conn(0x7fc3ec0ff2f0 msgr2=0x7fc3ec100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.182+0000 7fc3f1e00640 1 -- 172.21.15.2:0/3588446182 shutdown_connections 2024-09-10T08:11:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:08.182+0000 7fc3f1e00640 1 -- 172.21.15.2:0/3588446182 wait complete. 2024-09-10T08:11:09.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:09 smithi155 ceph-mon[35405]: pgmap v477: 129 pgs: 129 active+clean; 1.2 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:11:09.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:09 smithi107 ceph-mon[35048]: pgmap v477: 129 pgs: 129 active+clean; 1.2 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:11:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:09 smithi002 ceph-mon[30563]: pgmap v477: 129 pgs: 129 active+clean; 1.2 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:11:11.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:11 smithi155 ceph-mon[35405]: pgmap v478: 129 pgs: 129 active+clean; 1.2 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-10T08:11:11.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:11 smithi107 ceph-mon[35048]: pgmap v478: 129 pgs: 129 active+clean; 1.2 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-10T08:11:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:11 smithi002 ceph-mon[30563]: pgmap v478: 129 pgs: 129 active+clean; 1.2 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-10T08:11:13.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:13 smithi155 ceph-mon[35405]: pgmap v479: 129 pgs: 129 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:13.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:13 smithi107 ceph-mon[35048]: pgmap v479: 129 pgs: 129 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:13.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:13 smithi002 ceph-mon[30563]: pgmap v479: 129 pgs: 129 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:15.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:15 smithi155 ceph-mon[35405]: pgmap v480: 129 pgs: 129 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:11:15.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:15 smithi107 ceph-mon[35048]: pgmap v480: 129 pgs: 129 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:11:15.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:15 smithi002 ceph-mon[30563]: pgmap v480: 129 pgs: 129 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:11:17.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:17 smithi155 ceph-mon[35405]: pgmap v481: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:17.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:17 smithi107 ceph-mon[35048]: pgmap v481: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:17.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:17 smithi002 ceph-mon[30563]: pgmap v481: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:19.572 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:19 smithi002 ceph-mon[30563]: pgmap v482: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:11:19.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:19 smithi155 ceph-mon[35405]: pgmap v482: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:11:19.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:19 smithi107 ceph-mon[35048]: pgmap v482: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:11:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.301+0000 7f8897e00640 1 -- 172.21.15.2:0/1274515447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8898101f40 msgr2=0x7f8898109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.301+0000 7f8897e00640 1 --2- 172.21.15.2:0/1274515447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8898101f40 0x7f8898109db0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8890009ca0 tx=0x7f889002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.301+0000 7f8897e00640 1 -- 172.21.15.2:0/1274515447 shutdown_connections 2024-09-10T08:11:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.301+0000 7f8897e00640 1 --2- 172.21.15.2:0/1274515447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8898101f40 0x7f8898109db0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.301+0000 7f8897e00640 1 --2- 172.21.15.2:0/1274515447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8898101580 0x7f8898101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.301+0000 7f8897e00640 1 --2- 172.21.15.2:0/1274515447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8898103050 0x7f8898103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.301+0000 7f8897e00640 1 -- 172.21.15.2:0/1274515447 >> 172.21.15.2:0/1274515447 conn(0x7f88980fd0d0 msgr2=0x7f88980ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.301+0000 7f8897e00640 1 -- 172.21.15.2:0/1274515447 shutdown_connections 2024-09-10T08:11:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.301+0000 7f8897e00640 1 -- 172.21.15.2:0/1274515447 wait complete. 2024-09-10T08:11:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.303+0000 7f8897e00640 1 Processor -- start 2024-09-10T08:11:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.303+0000 7f8897e00640 1 -- start start 2024-09-10T08:11:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.303+0000 7f8897e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8898101580 0x7f8898199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.303+0000 7f8897e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8898101f40 0x7f889819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.303+0000 7f8897e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8898103050 0x7f88981a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.303+0000 7f8897e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f889810d740 con 0x7f8898101580 2024-09-10T08:11:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.303+0000 7f8897e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f889810d5c0 con 0x7f8898101f40 2024-09-10T08:11:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.303+0000 7f8897e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f889810d8c0 con 0x7f8898103050 2024-09-10T08:11:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.304+0000 7f8896000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8898101f40 0x7f889819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.304+0000 7f8896a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8898101580 0x7f8898199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.304+0000 7f8897400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8898103050 0x7f88981a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.304+0000 7f8896000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8898101f40 0x7f889819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:55148/0 (socket says 172.21.15.2:55148) 2024-09-10T08:11:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.304+0000 7f8896000640 1 -- 172.21.15.2:0/2614296078 learned_addr learned my addr 172.21.15.2:0/2614296078 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.304+0000 7f8896a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8898101580 0x7f8898199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58328/0 (socket says 172.21.15.2:58328) 2024-09-10T08:11:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.304+0000 7f8897400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8898103050 0x7f88981a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59240/0 (socket says 172.21.15.2:59240) 2024-09-10T08:11:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.304+0000 7f8896000640 1 -- 172.21.15.2:0/2614296078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8898103050 msgr2=0x7f88981a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.304+0000 7f8896000640 1 --2- 172.21.15.2:0/2614296078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8898103050 0x7f88981a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.304+0000 7f8896000640 1 -- 172.21.15.2:0/2614296078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8898101580 msgr2=0x7f8898199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.304+0000 7f8896000640 1 --2- 172.21.15.2:0/2614296078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8898101580 0x7f8898199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.304+0000 7f8896000640 1 -- 172.21.15.2:0/2614296078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88981a1bf0 con 0x7f8898101f40 2024-09-10T08:11:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.304+0000 7f8896000640 1 --2- 172.21.15.2:0/2614296078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8898101f40 0x7f889819a510 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f888400e9f0 tx=0x7f888400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.305+0000 7f888f400640 1 -- 172.21.15.2:0/2614296078 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888400cd80 con 0x7f8898101f40 2024-09-10T08:11:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.305+0000 7f888f400640 1 -- 172.21.15.2:0/2614296078 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8884005500 con 0x7f8898101f40 2024-09-10T08:11:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.305+0000 7f888f400640 1 -- 172.21.15.2:0/2614296078 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884010690 con 0x7f8898101f40 2024-09-10T08:11:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.305+0000 7f8897e00640 1 -- 172.21.15.2:0/2614296078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88981a1ee0 con 0x7f8898101f40 2024-09-10T08:11:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.305+0000 7f8897e00640 1 -- 172.21.15.2:0/2614296078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88981a2350 con 0x7f8898101f40 2024-09-10T08:11:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.309+0000 7f88a0ac6740 1 Processor -- start 2024-09-10T08:11:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.309+0000 7f888f400640 1 -- 172.21.15.2:0/2614296078 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f88840046c0 con 0x7f8898101f40 2024-09-10T08:11:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.309+0000 7f88a0ac6740 1 -- start start 2024-09-10T08:11:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.309+0000 7f88a0ac6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77aff260 0x55af77aff660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.309+0000 7f88a0ac6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b04660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.309+0000 7f88a0ac6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77b04c30 0x55af77b070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.309+0000 7f88a0ac6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55af77a64100 con 0x55af77affba0 2024-09-10T08:11:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.309+0000 7f88a0ac6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55af77a0c1b0 con 0x55af77aff260 2024-09-10T08:11:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.309+0000 7f88a0ac6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55af77a600c0 con 0x55af77b04c30 2024-09-10T08:11:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.309+0000 7f8896a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b04660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.310+0000 7f8896a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b04660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58342/0 (socket says 172.21.15.2:58342) 2024-09-10T08:11:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.310+0000 7f8896a00640 1 -- 172.21.15.2:0/1783182870 learned_addr learned my addr 172.21.15.2:0/1783182870 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.310+0000 7f8897400640 1 --2- 172.21.15.2:0/1783182870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77aff260 0x55af77aff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.310+0000 7f8896000640 1 --2- 172.21.15.2:0/1783182870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77b04c30 0x55af77b070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.310+0000 7f8896a00640 1 -- 172.21.15.2:0/1783182870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77b04c30 msgr2=0x55af77b070f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.310+0000 7f8896a00640 1 --2- 172.21.15.2:0/1783182870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77b04c30 0x55af77b070f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.310+0000 7f8896a00640 1 -- 172.21.15.2:0/1783182870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77aff260 msgr2=0x55af77aff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.310+0000 7f8896a00640 1 --2- 172.21.15.2:0/1783182870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77aff260 0x55af77aff660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.310+0000 7f8896a00640 1 -- 172.21.15.2:0/1783182870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af77777210 con 0x55af77affba0 2024-09-10T08:11:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.310+0000 7f8896000640 1 --2- 172.21.15.2:0/1783182870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77b04c30 0x55af77b070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:11:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.310+0000 7f8896a00640 1 --2- 172.21.15.2:0/1783182870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b04660 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8888009ca0 tx=0x7f8888011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372b8fbebd27061a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.310+0000 7f888f400640 1 --2- 172.21.15.2:0/2614296078 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f887c0a2c00 0x7f887c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.311+0000 7f8883e00640 1 -- 172.21.15.2:0/1783182870 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888004220 con 0x55af77affba0 2024-09-10T08:11:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.311+0000 7f8883e00640 1 -- 172.21.15.2:0/1783182870 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88880043c0 con 0x55af77affba0 2024-09-10T08:11:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.311+0000 7f888f400640 1 -- 172.21.15.2:0/2614296078 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f88840cf7e0 con 0x7f8898101f40 2024-09-10T08:11:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.311+0000 7f8883e00640 1 -- 172.21.15.2:0/1783182870 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888019930 con 0x55af77affba0 2024-09-10T08:11:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.311+0000 7f8897400640 1 --2- 172.21.15.2:0/2614296078 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f887c0a2c00 0x7f887c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.311+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1783182870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 msgr2=0x55af77b04660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.311+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1783182870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b04660 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8888009ca0 tx=0x7f8888011810 comp rx=0 tx=0).stop 2024-09-10T08:11:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.312+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1783182870 shutdown_connections 2024-09-10T08:11:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.312+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1783182870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77b04c30 0x55af77b070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.312+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1783182870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b04660 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.312+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1783182870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77aff260 0x55af77aff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.312+0000 7f8897400640 1 --2- 172.21.15.2:0/2614296078 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f887c0a2c00 0x7f887c0a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8890005e30 tx=0x7f8890031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.312+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1783182870 >> 172.21.15.2:0/1783182870 conn(0x55af77afa8d0 msgr2=0x55af77afacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.312+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1783182870 shutdown_connections 2024-09-10T08:11:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.312+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1783182870 wait complete. 2024-09-10T08:11:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.312+0000 7f88a0ac6740 1 -- start start 2024-09-10T08:11:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.313+0000 7f88a0ac6740 1 Processor -- start 2024-09-10T08:11:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.313+0000 7f88a0ac6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77aff260 0x55af77b09410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.313+0000 7f88a0ac6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b09950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.313+0000 7f88a0ac6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77b04c30 0x55af77b10990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.313+0000 7f88a0ac6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55af77b07c00 con 0x55af77affba0 2024-09-10T08:11:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.313+0000 7f88a0ac6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55af779ed090 con 0x55af77aff260 2024-09-10T08:11:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.313+0000 7f88a0ac6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55af77b07d80 con 0x55af77b04c30 2024-09-10T08:11:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.313+0000 7f88a0ac6740 10 client.0 _ll_register_callbacks cb 0x55af77a41f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.314+0000 7f8896000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b09950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.314+0000 7f8897400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77b04c30 0x55af77b10990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.314+0000 7f8896a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77aff260 0x55af77b09410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.314+0000 7f8897400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77b04c30 0x55af77b10990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59258/0 (socket says 172.21.15.2:59258) 2024-09-10T08:11:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.314+0000 7f8896000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b09950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58350/0 (socket says 172.21.15.2:58350) 2024-09-10T08:11:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.314+0000 7f8897400640 1 -- 172.21.15.2:0/1615333090 learned_addr learned my addr 172.21.15.2:0/1615333090 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.314+0000 7f8897400640 1 -- 172.21.15.2:0/1615333090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77aff260 msgr2=0x55af77b09410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.314+0000 7f8897400640 1 --2- 172.21.15.2:0/1615333090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77aff260 0x55af77b09410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.314+0000 7f8897400640 1 -- 172.21.15.2:0/1615333090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 msgr2=0x55af77b09950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.314+0000 7f8897400640 1 --2- 172.21.15.2:0/1615333090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b09950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.314+0000 7f8897400640 1 -- 172.21.15.2:0/1615333090 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af77843620 con 0x55af77b04c30 2024-09-10T08:11:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.314+0000 7f8896000640 1 --2- 172.21.15.2:0/1615333090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b09950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.315+0000 7f8897400640 1 --2- 172.21.15.2:0/1615333090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77b04c30 0x55af77b10990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8890035000 tx=0x7f889002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d763578a189d6db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.315+0000 7f8880c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:11:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.315+0000 7f8880c00640 1 -- 172.21.15.2:0/1615333090 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889002f3d0 con 0x55af77b04c30 2024-09-10T08:11:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.315+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1615333090 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88880043c0 con 0x55af77b04c30 2024-09-10T08:11:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.315+0000 7f8880c00640 1 -- 172.21.15.2:0/1615333090 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8890004250 con 0x55af77b04c30 2024-09-10T08:11:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.315+0000 7f8880c00640 1 -- 172.21.15.2:0/1615333090 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8890004450 con 0x55af77b04c30 2024-09-10T08:11:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.315+0000 7f88a0ac6740 10 client.26137 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.315+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1615333090 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af77b13780 con 0x55af77b04c30 2024-09-10T08:11:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.315+0000 7f8880c00640 1 -- 172.21.15.2:0/1615333090 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88880043c0 con 0x55af77b04c30 2024-09-10T08:11:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.316+0000 7f8880c00640 1 -- 172.21.15.2:0/1615333090 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8890051070 con 0x55af77b04c30 2024-09-10T08:11:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.316+0000 7f88a0ac6740 10 client.26137 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.316+0000 7f88a0ac6740 10 client.26137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.316+0000 7f88a0ac6740 20 client.26137 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.316+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1615333090 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55af77b14ec0 0x55af77b172d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.316+0000 7f88a0ac6740 4 client.26137 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:11:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.316+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1615333090 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af77a71830 con 0x55af77b14ec0 2024-09-10T08:11:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.316+0000 7f8896a00640 1 --2- 172.21.15.2:0/1615333090 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55af77b14ec0 0x55af77b172d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.317+0000 7f8896a00640 1 --2- 172.21.15.2:0/1615333090 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55af77b14ec0 0x55af77b172d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d0649c81b934ce2 server_cookie=474610f4f4ef9587 in_seq=0 out_seq=0 2024-09-10T08:11:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.317+0000 7f8880c00640 10 client.26137 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.317+0000 7f8880c00640 1 -- 172.21.15.2:0/1615333090 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af77a71830 con 0x55af77b14ec0 2024-09-10T08:11:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.317+0000 7f8880c00640 10 client.26137 handle_command_reply: tid=0 2024-09-10T08:11:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.318+0000 7f88a0ac6740 1 client.26137 shutdown 2024-09-10T08:11:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.318+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1615333090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77b04c30 msgr2=0x55af77b10990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.318+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1615333090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77b04c30 0x55af77b10990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8890035000 tx=0x7f889002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.318+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1615333090 shutdown_connections 2024-09-10T08:11:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.318+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1615333090 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55af77b14ec0 0x55af77b172d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.318+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1615333090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77b04c30 0x55af77b10990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.318+0000 7f8880c00640 0 client.26137 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.318+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1615333090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b09950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.318+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1615333090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77aff260 0x55af77b09410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.318+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1615333090 >> 172.21.15.2:0/1615333090 conn(0x55af77afa8d0 msgr2=0x55af77afc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.318+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1615333090 shutdown_connections 2024-09-10T08:11:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.318+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1615333090 wait complete. 2024-09-10T08:11:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.318+0000 7f88a0ac6740 20 client.26137 trim_cache size 0 max 16384 2024-09-10T08:11:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.347+0000 7f88a0ac6740 1 Processor -- start 2024-09-10T08:11:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.347+0000 7f88a0ac6740 1 -- start start 2024-09-10T08:11:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.347+0000 7f88a0ac6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77aff260 0x55af77b09410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.347+0000 7f88a0ac6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77affba0 0x55af77b10950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f88a0ac6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77b04c30 0x55af77b09950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f88a0ac6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f88981af280 con 0x55af77b04c30 2024-09-10T08:11:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f88a0ac6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88981acb50 con 0x55af77affba0 2024-09-10T08:11:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f88a0ac6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f88981aa9a0 con 0x55af77aff260 2024-09-10T08:11:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f8897400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77b04c30 0x55af77b09950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f8896a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77aff260 0x55af77b09410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f8897400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77b04c30 0x55af77b09950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58352/0 (socket says 172.21.15.2:58352) 2024-09-10T08:11:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f8897400640 1 -- 172.21.15.2:0/1358307787 learned_addr learned my addr 172.21.15.2:0/1358307787 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f8896000640 1 --2- 172.21.15.2:0/1358307787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77affba0 0x55af77b10950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f8896000640 1 -- 172.21.15.2:0/1358307787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77aff260 msgr2=0x55af77b09410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f8896000640 1 --2- 172.21.15.2:0/1358307787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77aff260 0x55af77b09410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f8896000640 1 -- 172.21.15.2:0/1358307787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77b04c30 msgr2=0x55af77b09950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f8896000640 1 --2- 172.21.15.2:0/1358307787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77b04c30 0x55af77b09950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.348+0000 7f8896000640 1 -- 172.21.15.2:0/1358307787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af77b00080 con 0x55af77affba0 2024-09-10T08:11:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.349+0000 7f8896000640 1 --2- 172.21.15.2:0/1358307787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77affba0 0x55af77b10950 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f888409b8f0 tx=0x7f888409ba80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4454c87724e538 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.349+0000 7f8883400640 1 -- 172.21.15.2:0/1358307787 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88840d5020 con 0x55af77affba0 2024-09-10T08:11:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.349+0000 7f8883400640 1 -- 172.21.15.2:0/1358307787 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8884086040 con 0x55af77affba0 2024-09-10T08:11:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.349+0000 7f8883400640 1 -- 172.21.15.2:0/1358307787 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88840cf460 con 0x55af77affba0 2024-09-10T08:11:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.350+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1358307787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77affba0 msgr2=0x55af77b10950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.350+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1358307787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77affba0 0x55af77b10950 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f888409b8f0 tx=0x7f888409ba80 comp rx=0 tx=0).stop 2024-09-10T08:11:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.350+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1358307787 shutdown_connections 2024-09-10T08:11:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.350+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1358307787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77b04c30 0x55af77b09950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.350+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1358307787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77affba0 0x55af77b10950 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.350+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/1358307787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77aff260 0x55af77b09410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.350+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1358307787 >> 172.21.15.2:0/1358307787 conn(0x55af77afa8d0 msgr2=0x55af77af56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.350+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1358307787 shutdown_connections 2024-09-10T08:11:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.350+0000 7f88a0ac6740 1 -- 172.21.15.2:0/1358307787 wait complete. 2024-09-10T08:11:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.350+0000 7f88a0ac6740 1 -- start start 2024-09-10T08:11:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.350+0000 7f88a0ac6740 1 Processor -- start 2024-09-10T08:11:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f88a0ac6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77aff260 0x55af77b0dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f88a0ac6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b0e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f88a0ac6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77b04c30 0x55af77b0f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f88a0ac6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f88981ab010 con 0x55af77affba0 2024-09-10T08:11:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f88a0ac6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88981ac480 con 0x55af77b04c30 2024-09-10T08:11:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f88a0ac6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f88981ab6e0 con 0x55af77aff260 2024-09-10T08:11:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f88a0ac6740 10 client.0 _ll_register_callbacks cb 0x7f8890009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f8896a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77aff260 0x55af77b0dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f8896000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b0e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f8896000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b0e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58366/0 (socket says 172.21.15.2:58366) 2024-09-10T08:11:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f8896a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77aff260 0x55af77b0dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59274/0 (socket says 172.21.15.2:59274) 2024-09-10T08:11:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f8896000640 1 -- 172.21.15.2:0/3226649239 learned_addr learned my addr 172.21.15.2:0/3226649239 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f8897400640 1 --2- 172.21.15.2:0/3226649239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77b04c30 0x55af77b0f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f8896000640 1 -- 172.21.15.2:0/3226649239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77aff260 msgr2=0x55af77b0dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f8896000640 1 --2- 172.21.15.2:0/3226649239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77aff260 0x55af77b0dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f8896000640 1 -- 172.21.15.2:0/3226649239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77b04c30 msgr2=0x55af77b0f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.351+0000 7f8896000640 1 --2- 172.21.15.2:0/3226649239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77b04c30 0x55af77b0f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.352+0000 7f8896000640 1 -- 172.21.15.2:0/3226649239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af77b0f7f0 con 0x55af77affba0 2024-09-10T08:11:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.352+0000 7f8896000640 1 --2- 172.21.15.2:0/3226649239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b0e1d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f888400c6a0 tx=0x7f888409baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90161d79777c10f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.352+0000 7f8880c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:11:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.352+0000 7f8880c00640 1 -- 172.21.15.2:0/3226649239 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88840d5070 con 0x55af77affba0 2024-09-10T08:11:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.352+0000 7f8880c00640 1 -- 172.21.15.2:0/3226649239 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f88840cfdd0 con 0x55af77affba0 2024-09-10T08:11:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.352+0000 7f88a0ac6740 1 -- 172.21.15.2:0/3226649239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8884086040 con 0x55af77affba0 2024-09-10T08:11:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.352+0000 7f8880c00640 1 -- 172.21.15.2:0/3226649239 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f888409ffa0 con 0x55af77affba0 2024-09-10T08:11:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.352+0000 7f88a0ac6740 10 client.16143 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.352+0000 7f88a0ac6740 1 -- 172.21.15.2:0/3226649239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af77b10b00 con 0x55af77affba0 2024-09-10T08:11:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.352+0000 7f8880c00640 1 -- 172.21.15.2:0/3226649239 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8884086040 con 0x55af77affba0 2024-09-10T08:11:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.353+0000 7f8880c00640 1 -- 172.21.15.2:0/3226649239 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f88840b4020 con 0x55af77affba0 2024-09-10T08:11:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.353+0000 7f88a0ac6740 10 client.16143 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.353+0000 7f88a0ac6740 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.353+0000 7f88a0ac6740 20 client.16143 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.353+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/3226649239 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55af77b801d0 0x55af77b825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.353+0000 7f88a0ac6740 4 client.16143 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:11:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.353+0000 7f88a0ac6740 1 -- 172.21.15.2:0/3226649239 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af77a6bb80 con 0x55af77b801d0 2024-09-10T08:11:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.353+0000 7f8896a00640 1 --2- 172.21.15.2:0/3226649239 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55af77b801d0 0x55af77b825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.354+0000 7f8896a00640 1 --2- 172.21.15.2:0/3226649239 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55af77b801d0 0x55af77b825f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3b394d0b7126ae0 server_cookie=b436287e4e63361c in_seq=0 out_seq=0 2024-09-10T08:11:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.354+0000 7f8880c00640 10 client.16143 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.366+0000 7f8880c00640 1 -- 172.21.15.2:0/3226649239 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af77a6bb80 con 0x55af77b801d0 2024-09-10T08:11:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.366+0000 7f8880c00640 10 client.16143 handle_command_reply: tid=0 2024-09-10T08:11:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.366+0000 7f88a0ac6740 1 client.16143 shutdown 2024-09-10T08:11:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.367+0000 7f88a0ac6740 1 -- 172.21.15.2:0/3226649239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 msgr2=0x55af77b0e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.367+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/3226649239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b0e1d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f888400c6a0 tx=0x7f888409baf0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.367+0000 7f88a0ac6740 1 -- 172.21.15.2:0/3226649239 shutdown_connections 2024-09-10T08:11:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.367+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/3226649239 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55af77b801d0 0x55af77b825f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.367+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/3226649239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af77b04c30 0x55af77b0f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.367+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/3226649239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af77affba0 0x55af77b0e1d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.367+0000 7f88a0ac6740 1 --2- 172.21.15.2:0/3226649239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af77aff260 0x55af77b0dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.367+0000 7f8880c00640 0 client.16143 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.367+0000 7f88a0ac6740 1 -- 172.21.15.2:0/3226649239 >> 172.21.15.2:0/3226649239 conn(0x55af77afa8d0 msgr2=0x55af77b09410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.367+0000 7f88a0ac6740 1 -- 172.21.15.2:0/3226649239 shutdown_connections 2024-09-10T08:11:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.367+0000 7f88a0ac6740 1 -- 172.21.15.2:0/3226649239 wait complete. 2024-09-10T08:11:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.367+0000 7f88a0ac6740 20 client.16143 trim_cache size 0 max 16384 2024-09-10T08:11:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:11:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:11:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:11:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:11:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.368+0000 7f8897e00640 1 -- 172.21.15.2:0/2614296078 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f887c0a2c00 msgr2=0x7f887c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.368+0000 7f8897e00640 1 --2- 172.21.15.2:0/2614296078 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f887c0a2c00 0x7f887c0a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8890005e30 tx=0x7f8890031bb0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.368+0000 7f8897e00640 1 -- 172.21.15.2:0/2614296078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8898101f40 msgr2=0x7f889819a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.368+0000 7f8897e00640 1 --2- 172.21.15.2:0/2614296078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8898101f40 0x7f889819a510 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f888400e9f0 tx=0x7f888400edf0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.368+0000 7f8897e00640 1 -- 172.21.15.2:0/2614296078 shutdown_connections 2024-09-10T08:11:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.368+0000 7f8897e00640 1 --2- 172.21.15.2:0/2614296078 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f887c0a2c00 0x7f887c0a50c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.368+0000 7f8897e00640 1 --2- 172.21.15.2:0/2614296078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8898103050 0x7f88981a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.368+0000 7f8897e00640 1 --2- 172.21.15.2:0/2614296078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8898101f40 0x7f889819a510 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.368+0000 7f8897e00640 1 --2- 172.21.15.2:0/2614296078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8898101580 0x7f8898199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.368+0000 7f8897e00640 1 -- 172.21.15.2:0/2614296078 >> 172.21.15.2:0/2614296078 conn(0x7f88980fd0d0 msgr2=0x7f8898074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.368+0000 7f8897e00640 1 -- 172.21.15.2:0/2614296078 shutdown_connections 2024-09-10T08:11:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:21.368+0000 7f8897e00640 1 -- 172.21.15.2:0/2614296078 wait complete. 2024-09-10T08:11:21.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:21 smithi002 ceph-mon[30563]: pgmap v483: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:11:21.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:21 smithi155 ceph-mon[35405]: pgmap v483: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:11:21.669 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:21 smithi107 ceph-mon[35048]: pgmap v483: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:11:21.858 INFO:tasks.workunit.client.0.smithi002.stdout:1/600: open fd = 3 2024-09-10T08:11:21.858 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.858 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.858 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.858 INFO:tasks.workunit.client.0.smithi002.stdout:2/600: open fd = 3 2024-09-10T08:11:21.858 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.858 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.858 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.858 INFO:tasks.workunit.client.0.smithi002.stdout:3/600: open fd = 3 2024-09-10T08:11:21.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.859 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.859 INFO:tasks.workunit.client.0.smithi002.stdout:4/600: open fd = 3 2024-09-10T08:11:21.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.859 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.859 INFO:tasks.workunit.client.0.smithi002.stdout:5/600: open fd = 3 2024-09-10T08:11:21.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.859 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.859 INFO:tasks.workunit.client.0.smithi002.stdout:6/600: open fd = 3 2024-09-10T08:11:21.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.860 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.860 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.860 INFO:tasks.workunit.client.0.smithi002.stdout:7/600: open fd = 3 2024-09-10T08:11:21.860 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.860 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.860 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.860 INFO:tasks.workunit.client.0.smithi002.stdout:8/600: open fd = 3 2024-09-10T08:11:21.860 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.860 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.860 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.860 INFO:tasks.workunit.client.0.smithi002.stdout:9/600: open fd = 3 2024-09-10T08:11:21.860 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.860 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.861 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.861 INFO:tasks.workunit.client.0.smithi002.stdout:10/600: open fd = 3 2024-09-10T08:11:21.861 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.861 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.861 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.861 INFO:tasks.workunit.client.0.smithi002.stdout:11/600: open fd = 3 2024-09-10T08:11:21.861 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.861 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.861 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.861 INFO:tasks.workunit.client.0.smithi002.stdout:12/600: open fd = 3 2024-09-10T08:11:21.861 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.861 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.861 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:13/600: open fd = 3 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:14/600: open fd = 3 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:15/600: open fd = 3 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:16/600: open fd = 3 2024-09-10T08:11:21.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.863 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.863 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.863 INFO:tasks.workunit.client.0.smithi002.stdout:17/600: open fd = 3 2024-09-10T08:11:21.863 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.863 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.863 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.863 INFO:tasks.workunit.client.0.smithi002.stdout:18/600: open fd = 3 2024-09-10T08:11:21.863 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.863 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.863 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.863 INFO:tasks.workunit.client.0.smithi002.stdout:19/600: open fd = 3 2024-09-10T08:11:21.863 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.863 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.864 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.864 INFO:tasks.workunit.client.0.smithi002.stdout:20/600: open fd = 3 2024-09-10T08:11:21.864 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.864 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.864 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.864 INFO:tasks.workunit.client.0.smithi002.stdout:21/600: open fd = 3 2024-09-10T08:11:21.864 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.864 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.864 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.864 INFO:tasks.workunit.client.0.smithi002.stdout:22/600: open fd = 3 2024-09-10T08:11:21.864 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.864 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.865 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.865 INFO:tasks.workunit.client.0.smithi002.stdout:23/600: open fd = 3 2024-09-10T08:11:21.865 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.865 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.865 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.865 INFO:tasks.workunit.client.0.smithi002.stdout:24/600: open fd = 3 2024-09-10T08:11:21.865 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.865 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.865 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.865 INFO:tasks.workunit.client.0.smithi002.stdout:25/600: open fd = 3 2024-09-10T08:11:21.865 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.865 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.865 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.866 INFO:tasks.workunit.client.0.smithi002.stdout:26/600: open fd = 3 2024-09-10T08:11:21.866 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.866 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.866 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.866 INFO:tasks.workunit.client.0.smithi002.stdout:27/600: open fd = 3 2024-09-10T08:11:21.866 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.866 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.866 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.866 INFO:tasks.workunit.client.0.smithi002.stdout:28/600: open fd = 3 2024-09-10T08:11:21.866 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.866 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.866 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.866 INFO:tasks.workunit.client.0.smithi002.stdout:29/600: open fd = 3 2024-09-10T08:11:21.867 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.867 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.867 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.867 INFO:tasks.workunit.client.0.smithi002.stdout:30/600: open fd = 3 2024-09-10T08:11:21.867 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.867 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.867 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.867 INFO:tasks.workunit.client.0.smithi002.stdout:31/600: open fd = 3 2024-09-10T08:11:21.867 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.867 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.867 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.867 INFO:tasks.workunit.client.0.smithi002.stdout:32/600: open fd = 3 2024-09-10T08:11:21.868 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.868 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.868 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.868 INFO:tasks.workunit.client.0.smithi002.stdout:33/600: open fd = 3 2024-09-10T08:11:21.868 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.868 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.868 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.868 INFO:tasks.workunit.client.0.smithi002.stdout:34/600: open fd = 3 2024-09-10T08:11:21.868 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.868 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.868 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.868 INFO:tasks.workunit.client.0.smithi002.stdout:35/600: open fd = 3 2024-09-10T08:11:21.868 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:36/600: open fd = 3 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:37/600: open fd = 3 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:38/600: open fd = 3 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.869 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.870 INFO:tasks.workunit.client.0.smithi002.stdout:39/600: open fd = 3 2024-09-10T08:11:21.870 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.870 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.870 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.870 INFO:tasks.workunit.client.0.smithi002.stdout:40/600: open fd = 3 2024-09-10T08:11:21.870 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.870 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.870 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.870 INFO:tasks.workunit.client.0.smithi002.stdout:41/600: open fd = 3 2024-09-10T08:11:21.870 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.870 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.870 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.870 INFO:tasks.workunit.client.0.smithi002.stdout:42/600: open fd = 3 2024-09-10T08:11:21.871 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.871 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.871 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.871 INFO:tasks.workunit.client.0.smithi002.stdout:43/600: open fd = 3 2024-09-10T08:11:21.871 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.871 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.871 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.871 INFO:tasks.workunit.client.0.smithi002.stdout:44/600: open fd = 3 2024-09-10T08:11:21.871 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.871 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.871 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.871 INFO:tasks.workunit.client.0.smithi002.stdout:45/600: open fd = 3 2024-09-10T08:11:21.872 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.872 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.872 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.872 INFO:tasks.workunit.client.0.smithi002.stdout:46/600: open fd = 3 2024-09-10T08:11:21.872 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.872 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.872 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.872 INFO:tasks.workunit.client.0.smithi002.stdout:47/600: open fd = 3 2024-09-10T08:11:21.872 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.872 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.872 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.872 INFO:tasks.workunit.client.0.smithi002.stdout:48/600: open fd = 3 2024-09-10T08:11:21.872 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.873 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.873 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.873 INFO:tasks.workunit.client.0.smithi002.stdout:49/600: open fd = 3 2024-09-10T08:11:21.873 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.873 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.873 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.873 INFO:tasks.workunit.client.0.smithi002.stdout:50/600: open fd = 3 2024-09-10T08:11:21.873 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.873 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.873 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.873 INFO:tasks.workunit.client.0.smithi002.stdout:51/600: open fd = 3 2024-09-10T08:11:21.873 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.873 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.874 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.874 INFO:tasks.workunit.client.0.smithi002.stdout:52/600: open fd = 3 2024-09-10T08:11:21.874 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.874 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.874 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.874 INFO:tasks.workunit.client.0.smithi002.stdout:53/600: open fd = 3 2024-09-10T08:11:21.874 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.874 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.874 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.874 INFO:tasks.workunit.client.0.smithi002.stdout:54/600: open fd = 3 2024-09-10T08:11:21.874 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.874 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:55/600: open fd = 3 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:56/600: open fd = 3 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:57/600: open fd = 3 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.875 INFO:tasks.workunit.client.0.smithi002.stdout:58/600: open fd = 3 2024-09-10T08:11:21.876 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.876 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.876 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.876 INFO:tasks.workunit.client.0.smithi002.stdout:59/600: open fd = 3 2024-09-10T08:11:21.876 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.876 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.876 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.876 INFO:tasks.workunit.client.0.smithi002.stdout:60/600: open fd = 3 2024-09-10T08:11:21.876 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.876 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.876 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.876 INFO:tasks.workunit.client.0.smithi002.stdout:61/600: open fd = 3 2024-09-10T08:11:21.877 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.877 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.877 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.877 INFO:tasks.workunit.client.0.smithi002.stdout:62/600: open fd = 3 2024-09-10T08:11:21.877 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.877 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.877 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:21.877 INFO:tasks.workunit.client.0.smithi002.stdout:63/600: open fd = 3 2024-09-10T08:11:21.877 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.877 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:11:21.877 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:11:23.153 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:11:23.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:23 smithi002 ceph-mon[30563]: pgmap v484: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:23.551 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:11:23.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:23 smithi155 ceph-mon[35405]: pgmap v484: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:23.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:23 smithi107 ceph-mon[35048]: pgmap v484: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:24.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.115+0000 7f5c2de00640 1 -- 172.21.15.2:0/784391600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c281079c0 msgr2=0x7f5c28107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:24.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.115+0000 7f5c2de00640 1 --2- 172.21.15.2:0/784391600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c281079c0 0x7f5c28107da0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009890 tx=0x7f5c1802a360 comp rx=0 tx=0).stop 2024-09-10T08:11:24.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.115+0000 7f5c2de00640 1 -- 172.21.15.2:0/784391600 shutdown_connections 2024-09-10T08:11:24.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.115+0000 7f5c2de00640 1 --2- 172.21.15.2:0/784391600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c28102360 0x7f5c2810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.115+0000 7f5c2de00640 1 --2- 172.21.15.2:0/784391600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c281019c0 0x7f5c28101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.115+0000 7f5c2de00640 1 --2- 172.21.15.2:0/784391600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c281079c0 0x7f5c28107da0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.115+0000 7f5c2de00640 1 -- 172.21.15.2:0/784391600 >> 172.21.15.2:0/784391600 conn(0x7f5c280fd740 msgr2=0x7f5c280ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:24.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.115+0000 7f5c2de00640 1 -- 172.21.15.2:0/784391600 shutdown_connections 2024-09-10T08:11:24.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.115+0000 7f5c2de00640 1 -- 172.21.15.2:0/784391600 wait complete. 2024-09-10T08:11:24.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.117+0000 7f5c2de00640 1 Processor -- start 2024-09-10T08:11:24.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.117+0000 7f5c2de00640 1 -- start start 2024-09-10T08:11:24.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.117+0000 7f5c2de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c281019c0 0x7f5c280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.117+0000 7f5c2de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c28102360 0x7f5c28074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.117+0000 7f5c2de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c281079c0 0x7f5c28074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2de00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5c28111dc0 con 0x7f5c281079c0 2024-09-10T08:11:24.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2de00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c28111c40 con 0x7f5c28102360 2024-09-10T08:11:24.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2de00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5c28111f40 con 0x7f5c281019c0 2024-09-10T08:11:24.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c281079c0 0x7f5c28074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c281019c0 0x7f5c280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c281079c0 0x7f5c28074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58394/0 (socket says 172.21.15.2:58394) 2024-09-10T08:11:24.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c28102360 0x7f5c28074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2d400640 1 -- 172.21.15.2:0/143029108 learned_addr learned my addr 172.21.15.2:0/143029108 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:24.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c281019c0 0x7f5c280744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59286/0 (socket says 172.21.15.2:59286) 2024-09-10T08:11:24.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2ca00640 1 -- 172.21.15.2:0/143029108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c28102360 msgr2=0x7f5c28074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:24.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2ca00640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c28102360 0x7f5c28074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2ca00640 1 -- 172.21.15.2:0/143029108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c281079c0 msgr2=0x7f5c28074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:24.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2ca00640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c281079c0 0x7f5c28074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2ca00640 1 -- 172.21.15.2:0/143029108 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c281a6780 con 0x7f5c281019c0 2024-09-10T08:11:24.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c2d400640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c281079c0 0x7f5c28074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:24.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.118+0000 7f5c1fe00640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c28102360 0x7f5c28074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:24.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.119+0000 7f5c2ca00640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c281019c0 0x7f5c280744f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5c1802a950 tx=0x7f5c18033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:24.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.119+0000 7f5c26000640 1 -- 172.21.15.2:0/143029108 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1803f020 con 0x7f5c281019c0 2024-09-10T08:11:24.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.119+0000 7f5c2de00640 1 -- 172.21.15.2:0/143029108 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c281a6a10 con 0x7f5c281019c0 2024-09-10T08:11:24.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.119+0000 7f5c26000640 1 -- 172.21.15.2:0/143029108 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c18030580 con 0x7f5c281019c0 2024-09-10T08:11:24.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.119+0000 7f5c26000640 1 -- 172.21.15.2:0/143029108 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c18037040 con 0x7f5c281019c0 2024-09-10T08:11:24.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.119+0000 7f5c2de00640 1 -- 172.21.15.2:0/143029108 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c281a6f40 con 0x7f5c281019c0 2024-09-10T08:11:24.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.123+0000 7f5c32042740 1 Processor -- start 2024-09-10T08:11:24.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.123+0000 7f5c26000640 1 -- 172.21.15.2:0/143029108 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5c18042020 con 0x7f5c281019c0 2024-09-10T08:11:24.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.123+0000 7f5c32042740 1 -- start start 2024-09-10T08:11:24.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.123+0000 7f5c32042740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfb9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.123+0000 7f5c32042740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb9670 0x55fc5bfc39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.123+0000 7f5c32042740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfc6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.123+0000 7f5c32042740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fc5bf1f3e0 con 0x55fc5bfc3f30 2024-09-10T08:11:24.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.123+0000 7f5c32042740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fc5bec6d80 con 0x55fc5bfb8d50 2024-09-10T08:11:24.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.123+0000 7f5c32042740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fc5bf1b3a0 con 0x55fc5bfb9670 2024-09-10T08:11:24.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.123+0000 7f5c2d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfb9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.123+0000 7f5c2d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfb9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:55214/0 (socket says 172.21.15.2:55214) 2024-09-10T08:11:24.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.123+0000 7f5c2d400640 1 -- 172.21.15.2:0/3794735412 learned_addr learned my addr 172.21.15.2:0/3794735412 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:24.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.124+0000 7f5c2ca00640 1 --2- 172.21.15.2:0/3794735412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb9670 0x55fc5bfc39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.124+0000 7f5c1fe00640 1 --2- 172.21.15.2:0/3794735412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfc6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.124+0000 7f5c2d400640 1 -- 172.21.15.2:0/3794735412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb9670 msgr2=0x55fc5bfc39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:24.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.124+0000 7f5c2d400640 1 --2- 172.21.15.2:0/3794735412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb9670 0x55fc5bfc39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.124+0000 7f5c2d400640 1 -- 172.21.15.2:0/3794735412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 msgr2=0x55fc5bfc6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.124+0000 7f5c2d400640 1 --2- 172.21.15.2:0/3794735412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfc6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.124+0000 7f5c2d400640 1 -- 172.21.15.2:0/3794735412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5bee5ae0 con 0x55fc5bfb8d50 2024-09-10T08:11:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.124+0000 7f5c26000640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5c0c0a2c00 0x7f5c0c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.124+0000 7f5c2d400640 1 --2- 172.21.15.2:0/3794735412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfb9130 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000e930 tx=0x7f5c2000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ab106dbedaf3eaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.124+0000 7f5c26000640 1 -- 172.21.15.2:0/143029108 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5c180eee30 con 0x7f5c281019c0 2024-09-10T08:11:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.124+0000 7f5c2ca00640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5c0c0a2c00 0x7f5c0c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.125+0000 7f5c1dc00640 1 -- 172.21.15.2:0/3794735412 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2000cda0 con 0x55fc5bfb8d50 2024-09-10T08:11:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.125+0000 7f5c1dc00640 1 -- 172.21.15.2:0/3794735412 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c2001d070 con 0x55fc5bfb8d50 2024-09-10T08:11:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.125+0000 7f5c1dc00640 1 -- 172.21.15.2:0/3794735412 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2000b6e0 con 0x55fc5bfb8d50 2024-09-10T08:11:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.125+0000 7f5c32042740 1 -- 172.21.15.2:0/3794735412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 msgr2=0x55fc5bfb9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.125+0000 7f5c32042740 1 --2- 172.21.15.2:0/3794735412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfb9130 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000e930 tx=0x7f5c2000edf0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.125+0000 7f5c2ca00640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5c0c0a2c00 0x7f5c0c0a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5c180b7c10 tx=0x7f5c18005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:24.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.125+0000 7f5c32042740 1 -- 172.21.15.2:0/3794735412 shutdown_connections 2024-09-10T08:11:24.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.125+0000 7f5c32042740 1 --2- 172.21.15.2:0/3794735412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfc6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.125+0000 7f5c32042740 1 --2- 172.21.15.2:0/3794735412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb9670 0x55fc5bfc39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.125+0000 7f5c32042740 1 --2- 172.21.15.2:0/3794735412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfb9130 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.125+0000 7f5c32042740 1 -- 172.21.15.2:0/3794735412 >> 172.21.15.2:0/3794735412 conn(0x55fc5bfb52e0 msgr2=0x55fc5bfb56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:24.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.126+0000 7f5c32042740 1 -- 172.21.15.2:0/3794735412 shutdown_connections 2024-09-10T08:11:24.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.126+0000 7f5c32042740 1 -- 172.21.15.2:0/3794735412 wait complete. 2024-09-10T08:11:24.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.126+0000 7f5c32042740 1 -- start start 2024-09-10T08:11:24.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.126+0000 7f5c32042740 1 Processor -- start 2024-09-10T08:11:24.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c32042740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfc0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c32042740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfb9670 0x55fc5bfc13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c32042740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfc18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c32042740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fc5bfc6ba0 con 0x55fc5bfb9670 2024-09-10T08:11:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c32042740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fc5beda0f0 con 0x55fc5bfb8d50 2024-09-10T08:11:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c32042740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fc5bfc6d20 con 0x55fc5bfc3f30 2024-09-10T08:11:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c32042740 10 client.0 _ll_register_callbacks cb 0x55fc5becb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfb9670 0x55fc5bfc13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfb9670 0x55fc5bfc13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58404/0 (socket says 172.21.15.2:58404) 2024-09-10T08:11:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c2d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfc0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c1fe00640 1 -- 172.21.15.2:0/2400782608 learned_addr learned my addr 172.21.15.2:0/2400782608 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c2d400640 1 --2- 172.21.15.2:0/2400782608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfc18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c1fe00640 1 -- 172.21.15.2:0/2400782608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfc3f30 msgr2=0x55fc5bfc18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c1fe00640 1 --2- 172.21.15.2:0/2400782608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfc18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c1fe00640 1 -- 172.21.15.2:0/2400782608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 msgr2=0x55fc5bfc0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c1fe00640 1 --2- 172.21.15.2:0/2400782608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfc0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.127+0000 7f5c1fe00640 1 -- 172.21.15.2:0/2400782608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5bc903c0 con 0x55fc5bfb9670 2024-09-10T08:11:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.128+0000 7f5c1fe00640 1 --2- 172.21.15.2:0/2400782608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfb9670 0x55fc5bfc13b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5c10012d20 tx=0x7f5c1000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f7c0bea1409158 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.128+0000 7f5c16000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:11:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.128+0000 7f5c16000640 1 -- 172.21.15.2:0/2400782608 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c100120d0 con 0x55fc5bfb9670 2024-09-10T08:11:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.128+0000 7f5c16000640 1 -- 172.21.15.2:0/2400782608 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5c10005020 con 0x55fc5bfb9670 2024-09-10T08:11:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.128+0000 7f5c32042740 1 -- 172.21.15.2:0/2400782608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c2001d070 con 0x55fc5bfb9670 2024-09-10T08:11:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.128+0000 7f5c16000640 1 -- 172.21.15.2:0/2400782608 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5c10025050 con 0x55fc5bfb9670 2024-09-10T08:11:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.128+0000 7f5c32042740 10 client.16155 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.128+0000 7f5c32042740 1 -- 172.21.15.2:0/2400782608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc5bfd2980 con 0x55fc5bfb9670 2024-09-10T08:11:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.129+0000 7f5c16000640 1 -- 172.21.15.2:0/2400782608 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c2001d070 con 0x55fc5bfb9670 2024-09-10T08:11:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.129+0000 7f5c16000640 1 -- 172.21.15.2:0/2400782608 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5c1001c470 con 0x55fc5bfb9670 2024-09-10T08:11:24.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.129+0000 7f5c32042740 10 client.16155 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.129+0000 7f5c32042740 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.129+0000 7f5c32042740 20 client.16155 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.129+0000 7f5c32042740 1 --2- 172.21.15.2:0/2400782608 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc5bfd3e40 0x55fc5bfd6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.129+0000 7f5c32042740 4 client.16155 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.129+0000 7f5c32042740 1 -- 172.21.15.2:0/2400782608 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc5bf2c950 con 0x55fc5bfd3e40 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.129+0000 7f5c2d400640 1 --2- 172.21.15.2:0/2400782608 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc5bfd3e40 0x55fc5bfd6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.130+0000 7f5c2d400640 1 --2- 172.21.15.2:0/2400782608 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc5bfd3e40 0x55fc5bfd6230 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ac9169304cc906 server_cookie=1294bbc8fb1677d0 in_seq=0 out_seq=0 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.130+0000 7f5c16000640 10 client.16155 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c16000640 1 -- 172.21.15.2:0/2400782608 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc5bf2c950 con 0x55fc5bfd3e40 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c16000640 10 client.16155 handle_command_reply: tid=0 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c32042740 1 client.16155 shutdown 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c32042740 1 -- 172.21.15.2:0/2400782608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfb9670 msgr2=0x55fc5bfc13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c32042740 1 --2- 172.21.15.2:0/2400782608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfb9670 0x55fc5bfc13b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5c10012d20 tx=0x7f5c1000f790 comp rx=0 tx=0).stop 2024-09-10T08:11:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c32042740 1 -- 172.21.15.2:0/2400782608 shutdown_connections 2024-09-10T08:11:24.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c32042740 1 --2- 172.21.15.2:0/2400782608 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc5bfd3e40 0x55fc5bfd6230 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c32042740 1 --2- 172.21.15.2:0/2400782608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfc18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c32042740 1 --2- 172.21.15.2:0/2400782608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfb9670 0x55fc5bfc13b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c32042740 1 --2- 172.21.15.2:0/2400782608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfc0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c16000640 0 client.16155 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:24.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c32042740 1 -- 172.21.15.2:0/2400782608 >> 172.21.15.2:0/2400782608 conn(0x55fc5bfb52e0 msgr2=0x55fc5bfbafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:24.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c32042740 1 -- 172.21.15.2:0/2400782608 shutdown_connections 2024-09-10T08:11:24.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c32042740 1 -- 172.21.15.2:0/2400782608 wait complete. 2024-09-10T08:11:24.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.131+0000 7f5c32042740 20 client.16155 trim_cache size 0 max 16384 2024-09-10T08:11:24.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.150+0000 7f5c32042740 1 Processor -- start 2024-09-10T08:11:24.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.150+0000 7f5c32042740 1 -- start start 2024-09-10T08:11:24.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.150+0000 7f5c32042740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfdbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.150+0000 7f5c32042740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb9670 0x55fc5bfdc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.150+0000 7f5c32042740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfbe110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.150+0000 7f5c32042740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281b3760 con 0x55fc5bfc3f30 2024-09-10T08:11:24.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.150+0000 7f5c32042740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281b1030 con 0x55fc5bfb8d50 2024-09-10T08:11:24.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.150+0000 7f5c32042740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281aee80 con 0x55fc5bfb9670 2024-09-10T08:11:24.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.151+0000 7f5c2d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfdbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.151+0000 7f5c2d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfbe110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.151+0000 7f5c2d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfdbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:55232/0 (socket says 172.21.15.2:55232) 2024-09-10T08:11:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.151+0000 7f5c2d400640 1 -- 172.21.15.2:0/3659295655 learned_addr learned my addr 172.21.15.2:0/3659295655 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.151+0000 7f5c1fe00640 1 --2- 172.21.15.2:0/3659295655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb9670 0x55fc5bfdc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.151+0000 7f5c2d400640 1 -- 172.21.15.2:0/3659295655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb9670 msgr2=0x55fc5bfdc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.151+0000 7f5c2d400640 1 --2- 172.21.15.2:0/3659295655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb9670 0x55fc5bfdc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.151+0000 7f5c2d400640 1 -- 172.21.15.2:0/3659295655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 msgr2=0x55fc5bfbe110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.151+0000 7f5c2d400640 1 --2- 172.21.15.2:0/3659295655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfbe110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.151+0000 7f5c2d400640 1 -- 172.21.15.2:0/3659295655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5bfbf3e0 con 0x55fc5bfb8d50 2024-09-10T08:11:24.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.151+0000 7f5c2d400640 1 --2- 172.21.15.2:0/3659295655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfdbb40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c20018f80 tx=0x7f5c20010040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=365dd54f3b48128c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:24.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.152+0000 7f5c1d200640 1 -- 172.21.15.2:0/3659295655 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c20004a00 con 0x55fc5bfb8d50 2024-09-10T08:11:24.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.152+0000 7f5c1d200640 1 -- 172.21.15.2:0/3659295655 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c2001d070 con 0x55fc5bfb8d50 2024-09-10T08:11:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.152+0000 7f5c1d200640 1 -- 172.21.15.2:0/3659295655 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c200186e0 con 0x55fc5bfb8d50 2024-09-10T08:11:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.152+0000 7f5c32042740 1 -- 172.21.15.2:0/3659295655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 msgr2=0x55fc5bfdbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.152+0000 7f5c32042740 1 --2- 172.21.15.2:0/3659295655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfdbb40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c20018f80 tx=0x7f5c20010040 comp rx=0 tx=0).stop 2024-09-10T08:11:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.152+0000 7f5c32042740 1 -- 172.21.15.2:0/3659295655 shutdown_connections 2024-09-10T08:11:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.152+0000 7f5c32042740 1 --2- 172.21.15.2:0/3659295655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfbe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.152+0000 7f5c32042740 1 --2- 172.21.15.2:0/3659295655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb9670 0x55fc5bfdc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.152+0000 7f5c32042740 1 --2- 172.21.15.2:0/3659295655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfdbb40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.152+0000 7f5c32042740 1 -- 172.21.15.2:0/3659295655 >> 172.21.15.2:0/3659295655 conn(0x55fc5bfb52e0 msgr2=0x55fc5bfbafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.152+0000 7f5c32042740 1 -- 172.21.15.2:0/3659295655 shutdown_connections 2024-09-10T08:11:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.153+0000 7f5c32042740 1 -- 172.21.15.2:0/3659295655 wait complete. 2024-09-10T08:11:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.153+0000 7f5c32042740 1 -- start start 2024-09-10T08:11:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.153+0000 7f5c32042740 1 Processor -- start 2024-09-10T08:11:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c32042740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfcc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c32042740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb9670 0x55fc5bfcc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c32042740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfc8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c32042740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281af4f0 con 0x55fc5bfc3f30 2024-09-10T08:11:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c32042740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281b0960 con 0x55fc5bfb9670 2024-09-10T08:11:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c32042740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281afbc0 con 0x55fc5bfb8d50 2024-09-10T08:11:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c32042740 10 client.0 _ll_register_callbacks cb 0x7f5c10012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb9670 0x55fc5bfcc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb9670 0x55fc5bfcc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:55244/0 (socket says 172.21.15.2:55244) 2024-09-10T08:11:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c1fe00640 1 -- 172.21.15.2:0/244647565 learned_addr learned my addr 172.21.15.2:0/244647565 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c2d400640 1 --2- 172.21.15.2:0/244647565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfc8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c2d400640 1 --2- 172.21.15.2:0/244647565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfcc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c1fe00640 1 -- 172.21.15.2:0/244647565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb8d50 msgr2=0x55fc5bfcc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:24.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c1fe00640 1 --2- 172.21.15.2:0/244647565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfcc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c1fe00640 1 -- 172.21.15.2:0/244647565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 msgr2=0x55fc5bfc8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:24.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c1fe00640 1 --2- 172.21.15.2:0/244647565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfc8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.154+0000 7f5c1fe00640 1 -- 172.21.15.2:0/244647565 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5bfc9680 con 0x55fc5bfb9670 2024-09-10T08:11:24.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.155+0000 7f5c1fe00640 1 --2- 172.21.15.2:0/244647565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb9670 0x55fc5bfcc6c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55fc5bfc24b0 tx=0x7f5c1001c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f8fc325d06b8f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:24.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.155+0000 7f5c16000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:11:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.155+0000 7f5c16000640 1 -- 172.21.15.2:0/244647565 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1001cab0 con 0x55fc5bfb9670 2024-09-10T08:11:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.155+0000 7f5c32042740 1 -- 172.21.15.2:0/244647565 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c2001d070 con 0x55fc5bfb9670 2024-09-10T08:11:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.155+0000 7f5c16000640 1 -- 172.21.15.2:0/244647565 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5c10018070 con 0x55fc5bfb9670 2024-09-10T08:11:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.155+0000 7f5c16000640 1 -- 172.21.15.2:0/244647565 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5c10037050 con 0x55fc5bfb9670 2024-09-10T08:11:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.155+0000 7f5c32042740 10 client.26195 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.155+0000 7f5c32042740 1 -- 172.21.15.2:0/244647565 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc5c03f560 con 0x55fc5bfb9670 2024-09-10T08:11:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.156+0000 7f5c16000640 1 -- 172.21.15.2:0/244647565 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c2001d070 con 0x55fc5bfb9670 2024-09-10T08:11:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.156+0000 7f5c16000640 1 -- 172.21.15.2:0/244647565 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5c100035f0 con 0x55fc5bfb9670 2024-09-10T08:11:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.156+0000 7f5c32042740 10 client.26195 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.156+0000 7f5c32042740 10 client.26195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.156+0000 7f5c32042740 20 client.26195 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.156+0000 7f5c32042740 1 --2- 172.21.15.2:0/244647565 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc5c03fd00 0x55fc5c042120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.156+0000 7f5c32042740 4 client.26195 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:11:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.156+0000 7f5c32042740 1 -- 172.21.15.2:0/244647565 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fc5bf26e60 con 0x55fc5c03fd00 2024-09-10T08:11:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.156+0000 7f5c2d400640 1 --2- 172.21.15.2:0/244647565 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc5c03fd00 0x55fc5c042120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c2d400640 1 --2- 172.21.15.2:0/244647565 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc5c03fd00 0x55fc5c042120 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3abfa3df0c413b05 server_cookie=95346604f15c2f7 in_seq=0 out_seq=0 2024-09-10T08:11:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c16000640 10 client.26195 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c16000640 1 -- 172.21.15.2:0/244647565 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fc5bf26e60 con 0x55fc5c03fd00 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c16000640 10 client.26195 handle_command_reply: tid=0 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c32042740 1 client.26195 shutdown 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c32042740 1 -- 172.21.15.2:0/244647565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb9670 msgr2=0x55fc5bfcc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c32042740 1 --2- 172.21.15.2:0/244647565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb9670 0x55fc5bfcc6c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55fc5bfc24b0 tx=0x7f5c1001c720 comp rx=0 tx=0).stop 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c32042740 1 -- 172.21.15.2:0/244647565 shutdown_connections 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c32042740 1 --2- 172.21.15.2:0/244647565 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc5c03fd00 0x55fc5c042120 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c32042740 1 --2- 172.21.15.2:0/244647565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5bfc3f30 0x55fc5bfc8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c32042740 1 --2- 172.21.15.2:0/244647565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc5bfb9670 0x55fc5bfcc6c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c32042740 1 --2- 172.21.15.2:0/244647565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc5bfb8d50 0x55fc5bfcc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c16000640 0 client.26195 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.157+0000 7f5c32042740 1 -- 172.21.15.2:0/244647565 >> 172.21.15.2:0/244647565 conn(0x55fc5bfb52e0 msgr2=0x55fc5bfb3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.158+0000 7f5c32042740 1 -- 172.21.15.2:0/244647565 shutdown_connections 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.158+0000 7f5c32042740 1 -- 172.21.15.2:0/244647565 wait complete. 2024-09-10T08:11:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.158+0000 7f5c32042740 20 client.26195 trim_cache size 0 max 16384 2024-09-10T08:11:24.162 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:11:24.162 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:11:24.162 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:11:24.162 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:11:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.158+0000 7f5c2de00640 1 -- 172.21.15.2:0/143029108 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5c0c0a2c00 msgr2=0x7f5c0c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.158+0000 7f5c2de00640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5c0c0a2c00 0x7f5c0c0a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5c180b7c10 tx=0x7f5c18005f90 comp rx=0 tx=0).stop 2024-09-10T08:11:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.158+0000 7f5c2de00640 1 -- 172.21.15.2:0/143029108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c281019c0 msgr2=0x7f5c280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.158+0000 7f5c2de00640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c281019c0 0x7f5c280744f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5c1802a950 tx=0x7f5c18033b60 comp rx=0 tx=0).stop 2024-09-10T08:11:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.159+0000 7f5c2de00640 1 -- 172.21.15.2:0/143029108 shutdown_connections 2024-09-10T08:11:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.159+0000 7f5c2de00640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c281079c0 0x7f5c28074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.159+0000 7f5c2de00640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c28102360 0x7f5c28074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.159+0000 7f5c2de00640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5c0c0a2c00 0x7f5c0c0a50c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.159+0000 7f5c2de00640 1 --2- 172.21.15.2:0/143029108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c281019c0 0x7f5c280744f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.159+0000 7f5c2de00640 1 -- 172.21.15.2:0/143029108 >> 172.21.15.2:0/143029108 conn(0x7f5c280fd740 msgr2=0x7f5c280fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.159+0000 7f5c2de00640 1 -- 172.21.15.2:0/143029108 shutdown_connections 2024-09-10T08:11:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:24.159+0000 7f5c2de00640 1 -- 172.21.15.2:0/143029108 wait complete. 2024-09-10T08:11:25.069 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:11:25.069 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:11:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:11:25.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:25 smithi002 ceph-mon[30563]: pgmap v485: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:11:25.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:25 smithi155 ceph-mon[35405]: pgmap v485: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:11:25.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:25 smithi107 ceph-mon[35048]: pgmap v485: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:11:26.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.031+0000 7f65ede00640 1 -- 172.21.15.2:0/3238354706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65e8077040 msgr2=0x7f65e8112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:26.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.031+0000 7f65ede00640 1 --2- 172.21.15.2:0/3238354706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65e8077040 0x7f65e8112510 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009900 tx=0x7f65d002a410 comp rx=0 tx=0).stop 2024-09-10T08:11:26.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.031+0000 7f65ede00640 1 -- 172.21.15.2:0/3238354706 shutdown_connections 2024-09-10T08:11:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.031+0000 7f65ede00640 1 --2- 172.21.15.2:0/3238354706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65e8112a50 0x7f65e8114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.031+0000 7f65ede00640 1 --2- 172.21.15.2:0/3238354706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65e8077040 0x7f65e8112510 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.031+0000 7f65ede00640 1 --2- 172.21.15.2:0/3238354706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e8076720 0x7f65e8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.031+0000 7f65ede00640 1 -- 172.21.15.2:0/3238354706 >> 172.21.15.2:0/3238354706 conn(0x7f65e80ffa50 msgr2=0x7f65e8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.031+0000 7f65ede00640 1 -- 172.21.15.2:0/3238354706 shutdown_connections 2024-09-10T08:11:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.031+0000 7f65ede00640 1 -- 172.21.15.2:0/3238354706 wait complete. 2024-09-10T08:11:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.032+0000 7f65ede00640 1 Processor -- start 2024-09-10T08:11:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.033+0000 7f65ede00640 1 -- start start 2024-09-10T08:11:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.033+0000 7f65ede00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e8076720 0x7f65e81a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.033+0000 7f65ede00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65e8077040 0x7f65e81a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.033+0000 7f65ede00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65e8112a50 0x7f65e819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.033+0000 7f65ede00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f65e8118620 con 0x7f65e8112a50 2024-09-10T08:11:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.033+0000 7f65ede00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f65e81184a0 con 0x7f65e8076720 2024-09-10T08:11:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.033+0000 7f65ede00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f65e81187a0 con 0x7f65e8077040 2024-09-10T08:11:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.033+0000 7f65e7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65e8112a50 0x7f65e819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.033+0000 7f65e7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65e8112a50 0x7f65e819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58434/0 (socket says 172.21.15.2:58434) 2024-09-10T08:11:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.033+0000 7f65e7400640 1 -- 172.21.15.2:0/1545532926 learned_addr learned my addr 172.21.15.2:0/1545532926 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.033+0000 7f65e6a00640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e8076720 0x7f65e81a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.033+0000 7f65de000640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65e8077040 0x7f65e81a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.034+0000 7f65e7400640 1 -- 172.21.15.2:0/1545532926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65e8077040 msgr2=0x7f65e81a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.034+0000 7f65e7400640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65e8077040 0x7f65e81a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.034+0000 7f65e7400640 1 -- 172.21.15.2:0/1545532926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e8076720 msgr2=0x7f65e81a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.034+0000 7f65e7400640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e8076720 0x7f65e81a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.034+0000 7f65e7400640 1 -- 172.21.15.2:0/1545532926 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e819f020 con 0x7f65e8112a50 2024-09-10T08:11:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.034+0000 7f65e7400640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65e8112a50 0x7f65e819e8f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f65e000eb10 tx=0x7f65e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.034+0000 7f65eca00640 1 -- 172.21.15.2:0/1545532926 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0004b20 con 0x7f65e8112a50 2024-09-10T08:11:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.034+0000 7f65eca00640 1 -- 172.21.15.2:0/1545532926 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e0004cc0 con 0x7f65e8112a50 2024-09-10T08:11:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.034+0000 7f65eca00640 1 -- 172.21.15.2:0/1545532926 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0010790 con 0x7f65e8112a50 2024-09-10T08:11:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.034+0000 7f65ede00640 1 -- 172.21.15.2:0/1545532926 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e819f310 con 0x7f65e8112a50 2024-09-10T08:11:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.034+0000 7f65ede00640 1 -- 172.21.15.2:0/1545532926 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e81ab600 con 0x7f65e8112a50 2024-09-10T08:11:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.036+0000 7f65ede00640 1 -- 172.21.15.2:0/1545532926 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e8076b00 con 0x7f65e8112a50 2024-09-10T08:11:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.038+0000 7f65eca00640 1 -- 172.21.15.2:0/1545532926 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f65e0005020 con 0x7f65e8112a50 2024-09-10T08:11:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.040+0000 7f65eca00640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f65cc0a2c00 0x7f65cc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.040+0000 7f65eca00640 1 -- 172.21.15.2:0/1545532926 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f65e00cfae0 con 0x7f65e8112a50 2024-09-10T08:11:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.040+0000 7f65e6a00640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f65cc0a2c00 0x7f65cc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.041+0000 7f65e6a00640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f65cc0a2c00 0x7f65cc0a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f65e819fa30 tx=0x7f65d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.044+0000 7f65eca00640 1 -- 172.21.15.2:0/1545532926 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65e009c5f0 con 0x7f65e8112a50 2024-09-10T08:11:26.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.265+0000 7f65ede00640 1 -- 172.21.15.2:0/1545532926 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65e81ab970 con 0x7f65e8112a50 2024-09-10T08:11:26.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.266+0000 7f65eca00640 1 -- 172.21.15.2:0/1545532926 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f65e009bef0 con 0x7f65e8112a50 2024-09-10T08:11:26.266 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:11:26.266 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:11:26.266 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:11:26.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.269+0000 7f65ede00640 1 -- 172.21.15.2:0/1545532926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f65cc0a2c00 msgr2=0x7f65cc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.269+0000 7f65ede00640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f65cc0a2c00 0x7f65cc0a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f65e819fa30 tx=0x7f65d8005f90 comp rx=0 tx=0).stop 2024-09-10T08:11:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.269+0000 7f65ede00640 1 -- 172.21.15.2:0/1545532926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65e8112a50 msgr2=0x7f65e819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.269+0000 7f65ede00640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65e8112a50 0x7f65e819e8f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f65e000eb10 tx=0x7f65e000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:11:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.269+0000 7f65ede00640 1 -- 172.21.15.2:0/1545532926 shutdown_connections 2024-09-10T08:11:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.269+0000 7f65ede00640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65e8112a50 0x7f65e819e8f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.269+0000 7f65ede00640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f65cc0a2c00 0x7f65cc0a50c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.269+0000 7f65ede00640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65e8077040 0x7f65e81a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.269+0000 7f65ede00640 1 --2- 172.21.15.2:0/1545532926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e8076720 0x7f65e81a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.269+0000 7f65ede00640 1 -- 172.21.15.2:0/1545532926 >> 172.21.15.2:0/1545532926 conn(0x7f65e80ffa50 msgr2=0x7f65e8101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.269+0000 7f65ede00640 1 -- 172.21.15.2:0/1545532926 shutdown_connections 2024-09-10T08:11:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:26.269+0000 7f65ede00640 1 -- 172.21.15.2:0/1545532926 wait complete. 2024-09-10T08:11:26.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:26 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1545532926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:11:26.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:26 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1545532926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:11:26.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:26 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1545532926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:11:27.080 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:11:27.459 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:11:27.598 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:27 smithi002 ceph-mon[30563]: pgmap v486: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T08:11:27.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:27 smithi155 ceph-mon[35405]: pgmap v486: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T08:11:27.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:27 smithi107 ceph-mon[35048]: pgmap v486: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T08:11:28.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.043+0000 7f5992e00640 1 -- 172.21.15.2:0/3197569818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f598c102790 msgr2=0x7f598c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:28.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.043+0000 7f5992e00640 1 --2- 172.21.15.2:0/3197569818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f598c102790 0x7f598c1067e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5974009900 tx=0x7f597402a410 comp rx=0 tx=0).stop 2024-09-10T08:11:28.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.044+0000 7f5992e00640 1 -- 172.21.15.2:0/3197569818 shutdown_connections 2024-09-10T08:11:28.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.044+0000 7f5992e00640 1 --2- 172.21.15.2:0/3197569818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f598c106f10 0x7f598c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.044+0000 7f5992e00640 1 --2- 172.21.15.2:0/3197569818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f598c102790 0x7f598c1067e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.044+0000 7f5992e00640 1 --2- 172.21.15.2:0/3197569818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f598c101de0 0x7f598c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.044+0000 7f5992e00640 1 -- 172.21.15.2:0/3197569818 >> 172.21.15.2:0/3197569818 conn(0x7f598c0fd8c0 msgr2=0x7f598c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:28.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.044+0000 7f5992e00640 1 -- 172.21.15.2:0/3197569818 shutdown_connections 2024-09-10T08:11:28.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.044+0000 7f5992e00640 1 -- 172.21.15.2:0/3197569818 wait complete. 2024-09-10T08:11:28.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.045+0000 7f5992e00640 1 Processor -- start 2024-09-10T08:11:28.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.045+0000 7f5992e00640 1 -- start start 2024-09-10T08:11:28.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.045+0000 7f5992e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f598c101de0 0x7f598c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.045+0000 7f5992e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f598c102790 0x7f598c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.045+0000 7f5992e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f598c106f10 0x7f598c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.045+0000 7f5992e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f598c111fc0 con 0x7f598c106f10 2024-09-10T08:11:28.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.045+0000 7f5992e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f598c111e40 con 0x7f598c101de0 2024-09-10T08:11:28.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.045+0000 7f5992e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f598c112140 con 0x7f598c102790 2024-09-10T08:11:28.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.045+0000 7f5990e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f598c106f10 0x7f598c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.045+0000 7f5990e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f598c106f10 0x7f598c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56644/0 (socket says 172.21.15.2:56644) 2024-09-10T08:11:28.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.045+0000 7f5990e00640 1 -- 172.21.15.2:0/1240218922 learned_addr learned my addr 172.21.15.2:0/1240218922 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:28.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.045+0000 7f598be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f598c101de0 0x7f598c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.046+0000 7f5983400640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f598c102790 0x7f598c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.046+0000 7f598be00640 1 -- 172.21.15.2:0/1240218922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f598c102790 msgr2=0x7f598c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:28.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.046+0000 7f598be00640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f598c102790 0x7f598c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.046+0000 7f598be00640 1 -- 172.21.15.2:0/1240218922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f598c106f10 msgr2=0x7f598c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:28.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.046+0000 7f598be00640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f598c106f10 0x7f598c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.046+0000 7f598be00640 1 -- 172.21.15.2:0/1240218922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598c1a6920 con 0x7f598c101de0 2024-09-10T08:11:28.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.046+0000 7f5990e00640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f598c106f10 0x7f598c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:28.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.046+0000 7f598be00640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f598c101de0 0x7f598c0744f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f597c00e9f0 tx=0x7f597c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:28.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.047+0000 7f5989600640 1 -- 172.21.15.2:0/1240218922 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c00cd80 con 0x7f598c101de0 2024-09-10T08:11:28.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.047+0000 7f5989600640 1 -- 172.21.15.2:0/1240218922 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f597c0035f0 con 0x7f598c101de0 2024-09-10T08:11:28.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.047+0000 7f5992e00640 1 -- 172.21.15.2:0/1240218922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f598c1a6c10 con 0x7f598c101de0 2024-09-10T08:11:28.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.047+0000 7f5989600640 1 -- 172.21.15.2:0/1240218922 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c010690 con 0x7f598c101de0 2024-09-10T08:11:28.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.047+0000 7f5992e00640 1 -- 172.21.15.2:0/1240218922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f598c1a7170 con 0x7f598c101de0 2024-09-10T08:11:28.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.050+0000 7f59959f9740 1 Processor -- start 2024-09-10T08:11:28.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.050+0000 7f59959f9740 1 -- start start 2024-09-10T08:11:28.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.050+0000 7f5989600640 1 -- 172.21.15.2:0/1240218922 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f597c003b30 con 0x7f598c101de0 2024-09-10T08:11:28.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.050+0000 7f59959f9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 0x564567943050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.050+0000 7f59959f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567943590 0x56456794d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.050+0000 7f59959f9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56456794de50 0x564567950240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.050+0000 7f59959f9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5645678a93e0 con 0x564567942c70 2024-09-10T08:11:28.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.050+0000 7f59959f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564567850d80 con 0x564567943590 2024-09-10T08:11:28.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.050+0000 7f59959f9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5645678a53a0 con 0x56456794de50 2024-09-10T08:11:28.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.050+0000 7f5990e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 0x564567943050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.050+0000 7f5990e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 0x564567943050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56660/0 (socket says 172.21.15.2:56660) 2024-09-10T08:11:28.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.051+0000 7f5990e00640 1 -- 172.21.15.2:0/2665988371 learned_addr learned my addr 172.21.15.2:0/2665988371 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:28.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.051+0000 7f598be00640 1 --2- 172.21.15.2:0/2665988371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567943590 0x56456794d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.051+0000 7f5983400640 1 --2- 172.21.15.2:0/2665988371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56456794de50 0x564567950240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.051+0000 7f5983400640 1 -- 172.21.15.2:0/2665988371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567943590 msgr2=0x56456794d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:28.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.051+0000 7f5983400640 1 --2- 172.21.15.2:0/2665988371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567943590 0x56456794d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.051+0000 7f5983400640 1 -- 172.21.15.2:0/2665988371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 msgr2=0x564567943050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:28.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.051+0000 7f5983400640 1 --2- 172.21.15.2:0/2665988371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 0x564567943050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.051+0000 7f5983400640 1 -- 172.21.15.2:0/2665988371 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56456786fae0 con 0x56456794de50 2024-09-10T08:11:28.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.051+0000 7f5990e00640 1 --2- 172.21.15.2:0/2665988371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 0x564567943050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:28.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.051+0000 7f5983400640 1 --2- 172.21.15.2:0/2665988371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56456794de50 0x564567950240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f597402aa00 tx=0x7f5974005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b971b58ccaeddde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:28.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.052+0000 7f5981600640 1 -- 172.21.15.2:0/2665988371 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974035040 con 0x56456794de50 2024-09-10T08:11:28.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.052+0000 7f5981600640 1 -- 172.21.15.2:0/2665988371 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5974005520 con 0x56456794de50 2024-09-10T08:11:28.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.052+0000 7f5989600640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59700a2c00 0x7f59700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.052+0000 7f5981600640 1 -- 172.21.15.2:0/2665988371 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974004e10 con 0x56456794de50 2024-09-10T08:11:28.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.052+0000 7f5989600640 1 -- 172.21.15.2:0/1240218922 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f597c0cf6e0 con 0x7f598c101de0 2024-09-10T08:11:28.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.052+0000 7f5990e00640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59700a2c00 0x7f59700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.052+0000 7f59959f9740 1 -- 172.21.15.2:0/2665988371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56456794de50 msgr2=0x564567950240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:28.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.052+0000 7f59959f9740 1 --2- 172.21.15.2:0/2665988371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56456794de50 0x564567950240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f597402aa00 tx=0x7f5974005de0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.053+0000 7f59959f9740 1 -- 172.21.15.2:0/2665988371 shutdown_connections 2024-09-10T08:11:28.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.053+0000 7f59959f9740 1 --2- 172.21.15.2:0/2665988371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56456794de50 0x564567950240 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.053+0000 7f59959f9740 1 --2- 172.21.15.2:0/2665988371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567943590 0x56456794d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.053+0000 7f59959f9740 1 --2- 172.21.15.2:0/2665988371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 0x564567943050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.053+0000 7f59959f9740 1 -- 172.21.15.2:0/2665988371 >> 172.21.15.2:0/2665988371 conn(0x56456793f200 msgr2=0x56456793f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:28.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.053+0000 7f59959f9740 1 -- 172.21.15.2:0/2665988371 shutdown_connections 2024-09-10T08:11:28.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.053+0000 7f5990e00640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59700a2c00 0x7f59700a50c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5984011880 tx=0x7f5984009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:28.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.053+0000 7f59959f9740 1 -- 172.21.15.2:0/2665988371 wait complete. 2024-09-10T08:11:28.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.053+0000 7f59959f9740 1 -- start start 2024-09-10T08:11:28.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.054+0000 7f59959f9740 1 Processor -- start 2024-09-10T08:11:28.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.054+0000 7f59959f9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 0x56456794ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.054+0000 7f59959f9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564567943590 0x56456794b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.054+0000 7f59959f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56456794de50 0x56456794b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.054+0000 7f59959f9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564567950a20 con 0x564567942c70 2024-09-10T08:11:28.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.054+0000 7f59959f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5645678640f0 con 0x56456794de50 2024-09-10T08:11:28.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.054+0000 7f59959f9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564567950ba0 con 0x564567943590 2024-09-10T08:11:28.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.054+0000 7f59959f9740 10 client.0 _ll_register_callbacks cb 0x56456784bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:28.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.054+0000 7f598be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 0x56456794ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.054+0000 7f598be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 0x56456794ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56666/0 (socket says 172.21.15.2:56666) 2024-09-10T08:11:28.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.054+0000 7f598be00640 1 -- 172.21.15.2:0/1655389518 learned_addr learned my addr 172.21.15.2:0/1655389518 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:28.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.054+0000 7f5983400640 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564567943590 0x56456794b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.055+0000 7f5990e00640 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56456794de50 0x56456794b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.055+0000 7f598be00640 1 -- 172.21.15.2:0/1655389518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564567943590 msgr2=0x56456794b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:28.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.055+0000 7f598be00640 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564567943590 0x56456794b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.055+0000 7f598be00640 1 -- 172.21.15.2:0/1655389518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56456794de50 msgr2=0x56456794b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:28.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.055+0000 7f598be00640 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56456794de50 0x56456794b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.055+0000 7f598be00640 1 -- 172.21.15.2:0/1655389518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56456761a3c0 con 0x564567942c70 2024-09-10T08:11:28.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.055+0000 7f5990e00640 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56456794de50 0x56456794b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:11:28.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.055+0000 7f598be00640 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 0x56456794ace0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f597c09ba80 tx=0x7f597c09b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e423fc6d5c967aa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:28.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.055+0000 7f597a000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:11:28.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.055+0000 7f597a000640 1 -- 172.21.15.2:0/1655389518 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c09c1f0 con 0x564567942c70 2024-09-10T08:11:28.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.055+0000 7f597a000640 1 -- 172.21.15.2:0/1655389518 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f597c098160 con 0x564567942c70 2024-09-10T08:11:28.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.055+0000 7f59959f9740 1 -- 172.21.15.2:0/1655389518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5974005520 con 0x564567942c70 2024-09-10T08:11:28.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.056+0000 7f597a000640 1 -- 172.21.15.2:0/1655389518 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f597c09baf0 con 0x564567942c70 2024-09-10T08:11:28.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.056+0000 7f59959f9740 10 client.16179 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:28.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.056+0000 7f59959f9740 1 -- 172.21.15.2:0/1655389518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56456795ca20 con 0x564567942c70 2024-09-10T08:11:28.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.056+0000 7f597a000640 1 -- 172.21.15.2:0/1655389518 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5974005520 con 0x564567942c70 2024-09-10T08:11:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.056+0000 7f597a000640 1 -- 172.21.15.2:0/1655389518 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f597c0cf020 con 0x564567942c70 2024-09-10T08:11:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.056+0000 7f59959f9740 10 client.16179 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.056+0000 7f59959f9740 10 client.16179 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:11:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.056+0000 7f59959f9740 20 client.16179 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.056+0000 7f59959f9740 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56456795de30 0x5645679602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.056+0000 7f59959f9740 4 client.16179 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:11:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.056+0000 7f59959f9740 1 -- 172.21.15.2:0/1655389518 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645678b6950 con 0x56456795de30 2024-09-10T08:11:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.057+0000 7f5983400640 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56456795de30 0x5645679602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.057+0000 7f5983400640 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56456795de30 0x5645679602a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df96883f293f56b1 server_cookie=1294f2879729c67 in_seq=0 out_seq=0 2024-09-10T08:11:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.057+0000 7f597a000640 10 client.16179 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:11:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.058+0000 7f597a000640 1 -- 172.21.15.2:0/1655389518 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645678b6950 con 0x56456795de30 2024-09-10T08:11:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.058+0000 7f597a000640 10 client.16179 handle_command_reply: tid=0 2024-09-10T08:11:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.058+0000 7f59959f9740 1 client.16179 shutdown 2024-09-10T08:11:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.058+0000 7f59959f9740 1 -- 172.21.15.2:0/1655389518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 msgr2=0x56456794ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.058+0000 7f59959f9740 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 0x56456794ace0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f597c09ba80 tx=0x7f597c09b980 comp rx=0 tx=0).stop 2024-09-10T08:11:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.058+0000 7f59959f9740 1 -- 172.21.15.2:0/1655389518 shutdown_connections 2024-09-10T08:11:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.058+0000 7f59959f9740 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56456794de50 0x56456794b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.058+0000 7f59959f9740 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564567943590 0x56456794b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.058+0000 7f59959f9740 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56456795de30 0x5645679602a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.058+0000 7f59959f9740 1 --2- 172.21.15.2:0/1655389518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567942c70 0x56456794ace0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.058+0000 7f59959f9740 1 -- 172.21.15.2:0/1655389518 >> 172.21.15.2:0/1655389518 conn(0x56456793f200 msgr2=0x564567944f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.058+0000 7f597a000640 0 client.16179 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:11:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.059+0000 7f59959f9740 1 -- 172.21.15.2:0/1655389518 shutdown_connections 2024-09-10T08:11:28.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.059+0000 7f59959f9740 1 -- 172.21.15.2:0/1655389518 wait complete. 2024-09-10T08:11:28.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.059+0000 7f59959f9740 20 client.16179 trim_cache size 0 max 16384 2024-09-10T08:11:28.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.073+0000 7f59959f9740 1 Processor -- start 2024-09-10T08:11:28.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.073+0000 7f59959f9740 1 -- start start 2024-09-10T08:11:28.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.073+0000 7f59959f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567942c70 0x564567945720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.073+0000 7f59959f9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567943590 0x56456794cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.073+0000 7f59959f9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56456794de50 0x564567965b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.073+0000 7f59959f9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f598c1b38c0 con 0x564567943590 2024-09-10T08:11:28.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.073+0000 7f59959f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f598c1b1190 con 0x564567942c70 2024-09-10T08:11:28.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.073+0000 7f59959f9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f598c1aefe0 con 0x56456794de50 2024-09-10T08:11:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.073+0000 7f5983400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567943590 0x56456794cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.073+0000 7f598be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567942c70 0x564567945720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.073+0000 7f5983400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567943590 0x56456794cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56668/0 (socket says 172.21.15.2:56668) 2024-09-10T08:11:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.073+0000 7f5983400640 1 -- 172.21.15.2:0/1324034721 learned_addr learned my addr 172.21.15.2:0/1324034721 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.073+0000 7f5990e00640 1 --2- 172.21.15.2:0/1324034721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56456794de50 0x564567965b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.074+0000 7f5983400640 1 -- 172.21.15.2:0/1324034721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56456794de50 msgr2=0x564567965b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.074+0000 7f5983400640 1 --2- 172.21.15.2:0/1324034721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56456794de50 0x564567965b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.074+0000 7f5983400640 1 -- 172.21.15.2:0/1324034721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567942c70 msgr2=0x564567945720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.074+0000 7f5983400640 1 --2- 172.21.15.2:0/1324034721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567942c70 0x564567945720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.074+0000 7f5983400640 1 -- 172.21.15.2:0/1324034721 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56456793f030 con 0x564567943590 2024-09-10T08:11:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.074+0000 7f598be00640 1 --2- 172.21.15.2:0/1324034721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567942c70 0x564567945720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.074+0000 7f5983400640 1 --2- 172.21.15.2:0/1324034721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567943590 0x56456794cc50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5974005c40 tx=0x7f597402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be34b185f215b307 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.074+0000 7f5980c00640 1 -- 172.21.15.2:0/1324034721 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974035040 con 0x564567943590 2024-09-10T08:11:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.074+0000 7f5980c00640 1 -- 172.21.15.2:0/1324034721 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59740042c0 con 0x564567943590 2024-09-10T08:11:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.074+0000 7f5980c00640 1 -- 172.21.15.2:0/1324034721 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974031810 con 0x564567943590 2024-09-10T08:11:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.075+0000 7f59959f9740 1 -- 172.21.15.2:0/1324034721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567943590 msgr2=0x56456794cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.075+0000 7f59959f9740 1 --2- 172.21.15.2:0/1324034721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567943590 0x56456794cc50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5974005c40 tx=0x7f597402a410 comp rx=0 tx=0).stop 2024-09-10T08:11:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.075+0000 7f59959f9740 1 -- 172.21.15.2:0/1324034721 shutdown_connections 2024-09-10T08:11:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.075+0000 7f59959f9740 1 --2- 172.21.15.2:0/1324034721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56456794de50 0x564567965b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.075+0000 7f59959f9740 1 --2- 172.21.15.2:0/1324034721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564567943590 0x56456794cc50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.075+0000 7f59959f9740 1 --2- 172.21.15.2:0/1324034721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567942c70 0x564567945720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.075+0000 7f59959f9740 1 -- 172.21.15.2:0/1324034721 >> 172.21.15.2:0/1324034721 conn(0x56456793f200 msgr2=0x56456794c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.075+0000 7f59959f9740 1 -- 172.21.15.2:0/1324034721 shutdown_connections 2024-09-10T08:11:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.075+0000 7f59959f9740 1 -- 172.21.15.2:0/1324034721 wait complete. 2024-09-10T08:11:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.076+0000 7f59959f9740 1 -- start start 2024-09-10T08:11:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.076+0000 7f59959f9740 1 Processor -- start 2024-09-10T08:11:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.076+0000 7f59959f9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564567942c70 0x5645679561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.076+0000 7f59959f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567943590 0x564567956720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.076+0000 7f59959f9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56456794de50 0x5645679478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.076+0000 7f59959f9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f598c1af650 con 0x56456794de50 2024-09-10T08:11:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.076+0000 7f59959f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f598c1b0ac0 con 0x564567943590 2024-09-10T08:11:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.076+0000 7f59959f9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f598c1afd20 con 0x564567942c70 2024-09-10T08:11:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.076+0000 7f59959f9740 10 client.0 _ll_register_callbacks cb 0x7f597c0d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:28.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.077+0000 7f5983400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567943590 0x564567956720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.077+0000 7f5983400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567943590 0x564567956720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41204/0 (socket says 172.21.15.2:41204) 2024-09-10T08:11:28.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.077+0000 7f5983400640 1 -- 172.21.15.2:0/3633112759 learned_addr learned my addr 172.21.15.2:0/3633112759 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:28.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.077+0000 7f5990e00640 1 --2- 172.21.15.2:0/3633112759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56456794de50 0x5645679478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.077+0000 7f598be00640 1 --2- 172.21.15.2:0/3633112759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564567942c70 0x5645679561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.077+0000 7f5983400640 1 -- 172.21.15.2:0/3633112759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564567942c70 msgr2=0x5645679561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:28.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.077+0000 7f5983400640 1 --2- 172.21.15.2:0/3633112759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564567942c70 0x5645679561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.077+0000 7f5983400640 1 -- 172.21.15.2:0/3633112759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56456794de50 msgr2=0x5645679478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:28.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.077+0000 7f5983400640 1 --2- 172.21.15.2:0/3633112759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56456794de50 0x5645679478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.077+0000 7f5983400640 1 -- 172.21.15.2:0/3633112759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56456793a770 con 0x564567943590 2024-09-10T08:11:28.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.077+0000 7f5983400640 1 --2- 172.21.15.2:0/3633112759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567943590 0x564567956720 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f597402a930 tx=0x7f5974009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=233bee38da3a6d45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:28.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.077+0000 7f597a000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:11:28.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.078+0000 7f597a000640 1 -- 172.21.15.2:0/3633112759 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59740037f0 con 0x564567943590 2024-09-10T08:11:28.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.078+0000 7f597a000640 1 -- 172.21.15.2:0/3633112759 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5974003990 con 0x564567943590 2024-09-10T08:11:28.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.078+0000 7f59959f9740 1 -- 172.21.15.2:0/3633112759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59740042c0 con 0x564567943590 2024-09-10T08:11:28.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.078+0000 7f597a000640 1 -- 172.21.15.2:0/3633112759 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5974003b90 con 0x564567943590 2024-09-10T08:11:28.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.078+0000 7f59959f9740 10 client.26219 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.078+0000 7f59959f9740 1 -- 172.21.15.2:0/3633112759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5645679c8660 con 0x564567943590 2024-09-10T08:11:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.078+0000 7f597a000640 1 -- 172.21.15.2:0/3633112759 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59740042c0 con 0x564567943590 2024-09-10T08:11:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.079+0000 7f597a000640 1 -- 172.21.15.2:0/3633112759 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5974061030 con 0x564567943590 2024-09-10T08:11:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.079+0000 7f59959f9740 10 client.26219 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.079+0000 7f59959f9740 10 client.26219 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:11:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.079+0000 7f59959f9740 20 client.26219 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.079+0000 7f59959f9740 1 --2- 172.21.15.2:0/3633112759 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5645679c8950 0x5645679cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.079+0000 7f59959f9740 4 client.26219 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:11:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.079+0000 7f59959f9740 1 -- 172.21.15.2:0/3633112759 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645678b0e60 con 0x5645679c8950 2024-09-10T08:11:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.079+0000 7f598be00640 1 --2- 172.21.15.2:0/3633112759 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5645679c8950 0x5645679cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f598be00640 1 --2- 172.21.15.2:0/3633112759 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5645679c8950 0x5645679cad70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e79a4bdae76ebe02 server_cookie=c407754cfb31a8d3 in_seq=0 out_seq=0 2024-09-10T08:11:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f597a000640 10 client.26219 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:11:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f597a000640 1 -- 172.21.15.2:0/3633112759 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645678b0e60 con 0x5645679c8950 2024-09-10T08:11:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f597a000640 10 client.26219 handle_command_reply: tid=0 2024-09-10T08:11:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f59959f9740 1 client.26219 shutdown 2024-09-10T08:11:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f59959f9740 1 -- 172.21.15.2:0/3633112759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567943590 msgr2=0x564567956720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f59959f9740 1 --2- 172.21.15.2:0/3633112759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567943590 0x564567956720 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f597402a930 tx=0x7f5974009520 comp rx=0 tx=0).stop 2024-09-10T08:11:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f59959f9740 1 -- 172.21.15.2:0/3633112759 shutdown_connections 2024-09-10T08:11:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f59959f9740 1 --2- 172.21.15.2:0/3633112759 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5645679c8950 0x5645679cad70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f59959f9740 1 --2- 172.21.15.2:0/3633112759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56456794de50 0x5645679478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f59959f9740 1 --2- 172.21.15.2:0/3633112759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564567943590 0x564567956720 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f59959f9740 1 --2- 172.21.15.2:0/3633112759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564567942c70 0x5645679561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f597a000640 0 client.26219 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:11:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f59959f9740 1 -- 172.21.15.2:0/3633112759 >> 172.21.15.2:0/3633112759 conn(0x56456793f200 msgr2=0x56456794cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f59959f9740 1 -- 172.21.15.2:0/3633112759 shutdown_connections 2024-09-10T08:11:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.080+0000 7f59959f9740 1 -- 172.21.15.2:0/3633112759 wait complete. 2024-09-10T08:11:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.081+0000 7f59959f9740 20 client.26219 trim_cache size 0 max 16384 2024-09-10T08:11:28.087 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:11:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.081+0000 7f5992e00640 1 -- 172.21.15.2:0/1240218922 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59700a2c00 msgr2=0x7f59700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.081+0000 7f5992e00640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59700a2c00 0x7f59700a50c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5984011880 tx=0x7f5984009500 comp rx=0 tx=0).stop 2024-09-10T08:11:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.081+0000 7f5992e00640 1 -- 172.21.15.2:0/1240218922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f598c101de0 msgr2=0x7f598c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.081+0000 7f5992e00640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f598c101de0 0x7f598c0744f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f597c00e9f0 tx=0x7f597c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.082+0000 7f5992e00640 1 -- 172.21.15.2:0/1240218922 shutdown_connections 2024-09-10T08:11:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.082+0000 7f5992e00640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f598c106f10 0x7f598c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.082+0000 7f5992e00640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f59700a2c00 0x7f59700a50c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.082+0000 7f5992e00640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f598c102790 0x7f598c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.082+0000 7f5992e00640 1 --2- 172.21.15.2:0/1240218922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f598c101de0 0x7f598c0744f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.082+0000 7f5992e00640 1 -- 172.21.15.2:0/1240218922 >> 172.21.15.2:0/1240218922 conn(0x7f598c0fd8c0 msgr2=0x7f598c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.082+0000 7f5992e00640 1 -- 172.21.15.2:0/1240218922 shutdown_connections 2024-09-10T08:11:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:28.082+0000 7f5992e00640 1 -- 172.21.15.2:0/1240218922 wait complete. 2024-09-10T08:11:28.992 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:11:29.371 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:11:29.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:29 smithi155 ceph-mon[35405]: pgmap v487: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:11:29.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:29 smithi107 ceph-mon[35048]: pgmap v487: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:11:29.688 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:29 smithi002 ceph-mon[30563]: pgmap v487: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:11:29.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.945+0000 7f3acb600640 1 -- 172.21.15.2:0/3524199653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ac4109140 msgr2=0x7f3ac4110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:29.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.945+0000 7f3acb600640 1 --2- 172.21.15.2:0/3524199653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ac4109140 0x7f3ac4110cc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009ca0 tx=0x7f3abc02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.945+0000 7f3acb600640 1 -- 172.21.15.2:0/3524199653 shutdown_connections 2024-09-10T08:11:29.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.945+0000 7f3acb600640 1 --2- 172.21.15.2:0/3524199653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ac4109140 0x7f3ac4110cc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.945+0000 7f3acb600640 1 --2- 172.21.15.2:0/3524199653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac41049c0 0x7f3ac4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.945+0000 7f3acb600640 1 --2- 172.21.15.2:0/3524199653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ac4104010 0x7f3ac41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.945+0000 7f3acb600640 1 -- 172.21.15.2:0/3524199653 >> 172.21.15.2:0/3524199653 conn(0x7f3ac40ffa50 msgr2=0x7f3ac4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:29.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.945+0000 7f3acb600640 1 -- 172.21.15.2:0/3524199653 shutdown_connections 2024-09-10T08:11:29.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.946+0000 7f3acb600640 1 -- 172.21.15.2:0/3524199653 wait complete. 2024-09-10T08:11:29.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.946+0000 7f3acb600640 1 Processor -- start 2024-09-10T08:11:29.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.947+0000 7f3acb600640 1 -- start start 2024-09-10T08:11:29.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.947+0000 7f3acb600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ac4104010 0x7f3ac4076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.947+0000 7f3acb600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ac41049c0 0x7f3ac4076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.947+0000 7f3acb600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac4109140 0x7f3ac4077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.947+0000 7f3acb600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41141a0 con 0x7f3ac41049c0 2024-09-10T08:11:29.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.947+0000 7f3acb600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4114020 con 0x7f3ac4104010 2024-09-10T08:11:29.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.947+0000 7f3acb600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4114320 con 0x7f3ac4109140 2024-09-10T08:11:29.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.947+0000 7f3ac3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ac41049c0 0x7f3ac4076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.947+0000 7f3ac3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ac41049c0 0x7f3ac4076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56694/0 (socket says 172.21.15.2:56694) 2024-09-10T08:11:29.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.947+0000 7f3ac3e00640 1 -- 172.21.15.2:0/2041418283 learned_addr learned my addr 172.21.15.2:0/2041418283 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:29.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.947+0000 7f3ac8c00640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ac4104010 0x7f3ac4076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.947+0000 7f3ac9600640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac4109140 0x7f3ac4077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.948+0000 7f3ac3e00640 1 -- 172.21.15.2:0/2041418283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac4109140 msgr2=0x7f3ac4077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:29.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.948+0000 7f3ac3e00640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac4109140 0x7f3ac4077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.948+0000 7f3ac3e00640 1 -- 172.21.15.2:0/2041418283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ac4104010 msgr2=0x7f3ac4076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:29.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.948+0000 7f3ac3e00640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ac4104010 0x7f3ac4076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.948+0000 7f3ac3e00640 1 -- 172.21.15.2:0/2041418283 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac41aad20 con 0x7f3ac41049c0 2024-09-10T08:11:29.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.948+0000 7f3ac3e00640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ac41049c0 0x7f3ac4076c40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000ea20 tx=0x7f3ab000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:29.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.948+0000 7f3ac1600640 1 -- 172.21.15.2:0/2041418283 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab000cd80 con 0x7f3ac41049c0 2024-09-10T08:11:29.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.948+0000 7f3ac1600640 1 -- 172.21.15.2:0/2041418283 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab0010910 con 0x7f3ac41049c0 2024-09-10T08:11:29.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.948+0000 7f3ac1600640 1 -- 172.21.15.2:0/2041418283 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0018920 con 0x7f3ac41049c0 2024-09-10T08:11:29.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.948+0000 7f3acb600640 1 -- 172.21.15.2:0/2041418283 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac41aafb0 con 0x7f3ac41049c0 2024-09-10T08:11:29.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.948+0000 7f3acb600640 1 -- 172.21.15.2:0/2041418283 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac41ab4b0 con 0x7f3ac41049c0 2024-09-10T08:11:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.952+0000 7f3acdfcb740 1 Processor -- start 2024-09-10T08:11:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.952+0000 7f3acdfcb740 1 -- start start 2024-09-10T08:11:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.952+0000 7f3ac1600640 1 -- 172.21.15.2:0/2041418283 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3ab0010ac0 con 0x7f3ac41049c0 2024-09-10T08:11:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.952+0000 7f3acdfcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b75c70 0x564710b76050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.952+0000 7f3acdfcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b76590 0x564710b80910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.952+0000 7f3acdfcb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b80e50 0x564710b83240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.952+0000 7f3acdfcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564710adc3e0 con 0x564710b76590 2024-09-10T08:11:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.952+0000 7f3acdfcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564710a83d80 con 0x564710b75c70 2024-09-10T08:11:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.952+0000 7f3acdfcb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564710ad83a0 con 0x564710b80e50 2024-09-10T08:11:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.953+0000 7f3ac9600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b75c70 0x564710b76050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.953+0000 7f3ac8c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b76590 0x564710b80910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.953+0000 7f3ac3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b80e50 0x564710b83240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.953+0000 7f3ac9600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b75c70 0x564710b76050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41228/0 (socket says 172.21.15.2:41228) 2024-09-10T08:11:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.953+0000 7f3ac9600640 1 -- 172.21.15.2:0/595653441 learned_addr learned my addr 172.21.15.2:0/595653441 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.953+0000 7f3ac9600640 1 -- 172.21.15.2:0/595653441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b80e50 msgr2=0x564710b83240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.953+0000 7f3ac9600640 1 --2- 172.21.15.2:0/595653441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b80e50 0x564710b83240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.953+0000 7f3ac9600640 1 -- 172.21.15.2:0/595653441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b76590 msgr2=0x564710b80910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.953+0000 7f3ac9600640 1 --2- 172.21.15.2:0/595653441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b76590 0x564710b80910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.953+0000 7f3ac9600640 1 -- 172.21.15.2:0/595653441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564710aa2ae0 con 0x564710b75c70 2024-09-10T08:11:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.953+0000 7f3ac9600640 1 --2- 172.21.15.2:0/595653441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b75c70 0x564710b76050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009ca0 tx=0x7f3abc0093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6a8f269e0e56b42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.954+0000 7f3ab5c00640 1 -- 172.21.15.2:0/595653441 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc031db0 con 0x564710b75c70 2024-09-10T08:11:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.954+0000 7f3ab5c00640 1 -- 172.21.15.2:0/595653441 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3abc036030 con 0x564710b75c70 2024-09-10T08:11:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.954+0000 7f3ab5c00640 1 -- 172.21.15.2:0/595653441 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0042d0 con 0x564710b75c70 2024-09-10T08:11:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.954+0000 7f3ac1600640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3aac0a2b80 0x7f3aac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.954+0000 7f3ac1600640 1 -- 172.21.15.2:0/2041418283 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3ab00cf780 con 0x7f3ac41049c0 2024-09-10T08:11:29.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.954+0000 7f3ac8c00640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3aac0a2b80 0x7f3aac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.954+0000 7f3acdfcb740 1 -- 172.21.15.2:0/595653441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b75c70 msgr2=0x564710b76050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:29.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.954+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/595653441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b75c70 0x564710b76050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009ca0 tx=0x7f3abc0093e0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.955+0000 7f3acdfcb740 1 -- 172.21.15.2:0/595653441 shutdown_connections 2024-09-10T08:11:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.955+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/595653441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b80e50 0x564710b83240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.955+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/595653441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b76590 0x564710b80910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.955+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/595653441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b75c70 0x564710b76050 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.955+0000 7f3acdfcb740 1 -- 172.21.15.2:0/595653441 >> 172.21.15.2:0/595653441 conn(0x564710b72200 msgr2=0x564710b725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.955+0000 7f3acdfcb740 1 -- 172.21.15.2:0/595653441 shutdown_connections 2024-09-10T08:11:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.955+0000 7f3acdfcb740 1 -- 172.21.15.2:0/595653441 wait complete. 2024-09-10T08:11:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.955+0000 7f3ac8c00640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3aac0a2b80 0x7f3aac0a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x564710b73060 tx=0x7f3ab8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.955+0000 7f3acdfcb740 1 -- start start 2024-09-10T08:11:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.956+0000 7f3acdfcb740 1 Processor -- start 2024-09-10T08:11:29.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.956+0000 7f3acdfcb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b7dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.956+0000 7f3acdfcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b7e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.956+0000 7f3acdfcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b7e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.956+0000 7f3acdfcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564710b838a0 con 0x564710b80e50 2024-09-10T08:11:29.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.956+0000 7f3acdfcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564710a970f0 con 0x564710b76590 2024-09-10T08:11:29.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.956+0000 7f3acdfcb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564710b83a20 con 0x564710b75c70 2024-09-10T08:11:29.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.956+0000 7f3acdfcb740 10 client.0 _ll_register_callbacks cb 0x564710a7ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:29.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.956+0000 7f3ac8c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b7dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.957+0000 7f3ac3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b7e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.957+0000 7f3ac8c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b7dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56380/0 (socket says 172.21.15.2:56380) 2024-09-10T08:11:29.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.957+0000 7f3ac9600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b7e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.957+0000 7f3ac8c00640 1 -- 172.21.15.2:0/4062452097 learned_addr learned my addr 172.21.15.2:0/4062452097 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:29.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.957+0000 7f3ac3e00640 1 -- 172.21.15.2:0/4062452097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 msgr2=0x564710b7dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:29.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.957+0000 7f3ac3e00640 1 --2- 172.21.15.2:0/4062452097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b7dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.957+0000 7f3ac3e00640 1 -- 172.21.15.2:0/4062452097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 msgr2=0x564710b7e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:29.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.957+0000 7f3ac3e00640 1 --2- 172.21.15.2:0/4062452097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b7e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.957+0000 7f3ac3e00640 1 -- 172.21.15.2:0/4062452097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56471084d3c0 con 0x564710b76590 2024-09-10T08:11:29.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.957+0000 7f3ac3e00640 1 --2- 172.21.15.2:0/4062452097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b7e1b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x564710b73580 tx=0x7f3ab009b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c632dc17516a3cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:29.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.957+0000 7f3aaa000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:11:29.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3aaa000640 1 -- 172.21.15.2:0/4062452097 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab00cf3f0 con 0x564710b76590 2024-09-10T08:11:29.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3aaa000640 1 -- 172.21.15.2:0/4062452097 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3ab00982b0 con 0x564710b76590 2024-09-10T08:11:29.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3acdfcb740 1 -- 172.21.15.2:0/4062452097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3abc036030 con 0x564710b76590 2024-09-10T08:11:29.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3aaa000640 1 -- 172.21.15.2:0/4062452097 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ab00cfb70 con 0x564710b76590 2024-09-10T08:11:29.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3acdfcb740 10 client.26231 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:29.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3acdfcb740 1 -- 172.21.15.2:0/4062452097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564710b8f8a0 con 0x564710b76590 2024-09-10T08:11:29.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3aaa000640 1 -- 172.21.15.2:0/4062452097 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3abc036030 con 0x564710b76590 2024-09-10T08:11:29.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3aaa000640 1 -- 172.21.15.2:0/4062452097 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ab00ce030 con 0x564710b76590 2024-09-10T08:11:29.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3acdfcb740 10 client.26231 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:29.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3acdfcb740 10 client.26231 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:11:29.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3acdfcb740 20 client.26231 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:29.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/4062452097 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x564710b90d60 0x564710b93150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3acdfcb740 4 client.26231 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:11:29.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.958+0000 7f3acdfcb740 1 -- 172.21.15.2:0/4062452097 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564710ae9950 con 0x564710b90d60 2024-09-10T08:11:29.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.959+0000 7f3ac8c00640 1 --2- 172.21.15.2:0/4062452097 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x564710b90d60 0x564710b93150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.959+0000 7f3ac8c00640 1 --2- 172.21.15.2:0/4062452097 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x564710b90d60 0x564710b93150 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9782e69bc7dae40d server_cookie=f712861965fd9f71 in_seq=0 out_seq=0 2024-09-10T08:11:29.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.959+0000 7f3aaa000640 10 client.26231 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:11:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.960+0000 7f3aaa000640 1 -- 172.21.15.2:0/4062452097 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564710ae9950 con 0x564710b90d60 2024-09-10T08:11:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.960+0000 7f3aaa000640 10 client.26231 handle_command_reply: tid=0 2024-09-10T08:11:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.960+0000 7f3acdfcb740 1 client.26231 shutdown 2024-09-10T08:11:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.960+0000 7f3acdfcb740 1 -- 172.21.15.2:0/4062452097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 msgr2=0x564710b7e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.960+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/4062452097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b7e1b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x564710b73580 tx=0x7f3ab009b660 comp rx=0 tx=0).stop 2024-09-10T08:11:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.961+0000 7f3acdfcb740 1 -- 172.21.15.2:0/4062452097 shutdown_connections 2024-09-10T08:11:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.961+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/4062452097 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x564710b90d60 0x564710b93150 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.961+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/4062452097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b7e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.961+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/4062452097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b7e1b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.961+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/4062452097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b7dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.961+0000 7f3aaa000640 0 client.26231 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:11:29.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.961+0000 7f3acdfcb740 1 -- 172.21.15.2:0/4062452097 >> 172.21.15.2:0/4062452097 conn(0x564710b72200 msgr2=0x564710b71380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:29.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.961+0000 7f3acdfcb740 1 -- 172.21.15.2:0/4062452097 shutdown_connections 2024-09-10T08:11:29.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.961+0000 7f3acdfcb740 1 -- 172.21.15.2:0/4062452097 wait complete. 2024-09-10T08:11:29.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.961+0000 7f3acdfcb740 20 client.26231 trim_cache size 0 max 16384 2024-09-10T08:11:29.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.974+0000 7f3acdfcb740 1 Processor -- start 2024-09-10T08:11:29.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.975+0000 7f3acdfcb740 1 -- start start 2024-09-10T08:11:29.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.975+0000 7f3acdfcb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b98a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.975+0000 7f3acdfcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b98f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.975+0000 7f3acdfcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b7aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.975+0000 7f3acdfcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b7d90 con 0x564710b80e50 2024-09-10T08:11:29.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.975+0000 7f3acdfcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b5660 con 0x564710b76590 2024-09-10T08:11:29.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.975+0000 7f3acdfcb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b34b0 con 0x564710b75c70 2024-09-10T08:11:29.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.975+0000 7f3ac9600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b7aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.975+0000 7f3ac9600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b7aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56734/0 (socket says 172.21.15.2:56734) 2024-09-10T08:11:29.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.975+0000 7f3ac9600640 1 -- 172.21.15.2:0/2934184488 learned_addr learned my addr 172.21.15.2:0/2934184488 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:29.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.975+0000 7f3ac8c00640 1 --2- 172.21.15.2:0/2934184488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b98a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.975+0000 7f3ac3e00640 1 --2- 172.21.15.2:0/2934184488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b98f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.976+0000 7f3ac9600640 1 -- 172.21.15.2:0/2934184488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 msgr2=0x564710b98a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:29.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.976+0000 7f3ac9600640 1 --2- 172.21.15.2:0/2934184488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b98a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.976+0000 7f3ac9600640 1 -- 172.21.15.2:0/2934184488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 msgr2=0x564710b98f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:29.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.976+0000 7f3ac9600640 1 --2- 172.21.15.2:0/2934184488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b98f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.976+0000 7f3ac9600640 1 -- 172.21.15.2:0/2934184488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564710b72030 con 0x564710b80e50 2024-09-10T08:11:29.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.976+0000 7f3ac8c00640 1 --2- 172.21.15.2:0/2934184488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b98a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:29.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.976+0000 7f3ac9600640 1 --2- 172.21.15.2:0/2934184488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b7aed0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3abc006010 tx=0x7f3abc031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c704c488f7f74de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:29.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.976+0000 7f3ab5200640 1 -- 172.21.15.2:0/2934184488 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc055070 con 0x564710b80e50 2024-09-10T08:11:29.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.976+0000 7f3ab5200640 1 -- 172.21.15.2:0/2934184488 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3abc036040 con 0x564710b80e50 2024-09-10T08:11:29.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.976+0000 7f3ab5200640 1 -- 172.21.15.2:0/2934184488 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc04a040 con 0x564710b80e50 2024-09-10T08:11:29.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.977+0000 7f3acdfcb740 1 -- 172.21.15.2:0/2934184488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 msgr2=0x564710b7aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:29.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.977+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/2934184488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b7aed0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3abc006010 tx=0x7f3abc031a20 comp rx=0 tx=0).stop 2024-09-10T08:11:29.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.977+0000 7f3acdfcb740 1 -- 172.21.15.2:0/2934184488 shutdown_connections 2024-09-10T08:11:29.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.977+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/2934184488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b7aed0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.977+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/2934184488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b98f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.977+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/2934184488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b98a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.977+0000 7f3acdfcb740 1 -- 172.21.15.2:0/2934184488 >> 172.21.15.2:0/2934184488 conn(0x564710b72200 msgr2=0x564710b7fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:29.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.977+0000 7f3acdfcb740 1 -- 172.21.15.2:0/2934184488 shutdown_connections 2024-09-10T08:11:29.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.978+0000 7f3acdfcb740 1 -- 172.21.15.2:0/2934184488 wait complete. 2024-09-10T08:11:29.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.978+0000 7f3acdfcb740 1 -- start start 2024-09-10T08:11:29.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.978+0000 7f3acdfcb740 1 Processor -- start 2024-09-10T08:11:29.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3acdfcb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b84aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3acdfcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b84fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3acdfcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b85520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3acdfcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b3b20 con 0x564710b80e50 2024-09-10T08:11:29.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3acdfcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b4f90 con 0x564710b76590 2024-09-10T08:11:29.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3acdfcb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b41f0 con 0x564710b75c70 2024-09-10T08:11:29.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3acdfcb740 10 client.0 _ll_register_callbacks cb 0x7f3ab000ef50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:29.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3ac3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b84fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3ac3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b84fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41270/0 (socket says 172.21.15.2:41270) 2024-09-10T08:11:29.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3ac3e00640 1 -- 172.21.15.2:0/2412161149 learned_addr learned my addr 172.21.15.2:0/2412161149 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:29.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3ac9600640 1 --2- 172.21.15.2:0/2412161149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b85520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3ac8c00640 1 --2- 172.21.15.2:0/2412161149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b84aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3ac3e00640 1 -- 172.21.15.2:0/2412161149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 msgr2=0x564710b84aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:29.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3ac3e00640 1 --2- 172.21.15.2:0/2412161149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b84aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3ac3e00640 1 -- 172.21.15.2:0/2412161149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 msgr2=0x564710b85520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:29.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3ac3e00640 1 --2- 172.21.15.2:0/2412161149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b85520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.979+0000 7f3ac3e00640 1 -- 172.21.15.2:0/2412161149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564710b732b0 con 0x564710b76590 2024-09-10T08:11:29.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3ac3e00640 1 --2- 172.21.15.2:0/2412161149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b84fe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00cfb40 tx=0x7f3ab000c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2c71d66319d09c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:29.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3aaa000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:11:29.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3aaa000640 1 -- 172.21.15.2:0/2412161149 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab009b780 con 0x564710b76590 2024-09-10T08:11:29.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3aaa000640 1 -- 172.21.15.2:0/2412161149 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3ab0010430 con 0x564710b76590 2024-09-10T08:11:29.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3acdfcb740 1 -- 172.21.15.2:0/2412161149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3abc036040 con 0x564710b76590 2024-09-10T08:11:29.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3aaa000640 1 -- 172.21.15.2:0/2412161149 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ab00b8050 con 0x564710b76590 2024-09-10T08:11:29.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3acdfcb740 10 client.26237 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:29.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3acdfcb740 1 -- 172.21.15.2:0/2412161149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564710bfbbc0 con 0x564710b76590 2024-09-10T08:11:29.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3aaa000640 1 -- 172.21.15.2:0/2412161149 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3abc036040 con 0x564710b76590 2024-09-10T08:11:29.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3aaa000640 1 -- 172.21.15.2:0/2412161149 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ab00ce040 con 0x564710b76590 2024-09-10T08:11:29.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3acdfcb740 10 client.26237 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:29.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3acdfcb740 10 client.26237 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:11:29.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3acdfcb740 20 client.26237 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:29.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/2412161149 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x564710bfc1b0 0x564710bfe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:29.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3acdfcb740 4 client.26237 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:11:29.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.980+0000 7f3acdfcb740 1 -- 172.21.15.2:0/2412161149 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564710ae3e60 con 0x564710bfc1b0 2024-09-10T08:11:29.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.981+0000 7f3ac8c00640 1 --2- 172.21.15.2:0/2412161149 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x564710bfc1b0 0x564710bfe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:29.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.981+0000 7f3ac8c00640 1 --2- 172.21.15.2:0/2412161149 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x564710bfc1b0 0x564710bfe5d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41f77e1a37130bc7 server_cookie=e0d9134713a0a04c in_seq=0 out_seq=0 2024-09-10T08:11:29.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.981+0000 7f3aaa000640 10 client.26237 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:11:29.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3aaa000640 1 -- 172.21.15.2:0/2412161149 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564710ae3e60 con 0x564710bfc1b0 2024-09-10T08:11:29.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3aaa000640 10 client.26237 handle_command_reply: tid=0 2024-09-10T08:11:29.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3acdfcb740 1 client.26237 shutdown 2024-09-10T08:11:29.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3acdfcb740 1 -- 172.21.15.2:0/2412161149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 msgr2=0x564710b84fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:29.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/2412161149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b84fe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00cfb40 tx=0x7f3ab000c840 comp rx=0 tx=0).stop 2024-09-10T08:11:29.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3acdfcb740 1 -- 172.21.15.2:0/2412161149 shutdown_connections 2024-09-10T08:11:29.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/2412161149 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x564710bfc1b0 0x564710bfe5d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/2412161149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564710b80e50 0x564710b85520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/2412161149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564710b76590 0x564710b84fe0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3aaa000640 0 client.26237 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:11:29.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3acdfcb740 1 --2- 172.21.15.2:0/2412161149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564710b75c70 0x564710b84aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3acdfcb740 1 -- 172.21.15.2:0/2412161149 >> 172.21.15.2:0/2412161149 conn(0x564710b72200 msgr2=0x564710b70b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:29.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3acdfcb740 1 -- 172.21.15.2:0/2412161149 shutdown_connections 2024-09-10T08:11:29.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3acdfcb740 1 -- 172.21.15.2:0/2412161149 wait complete. 2024-09-10T08:11:29.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.982+0000 7f3acdfcb740 20 client.26237 trim_cache size 0 max 16384 2024-09-10T08:11:29.986 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:11:29.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.983+0000 7f3acb600640 1 -- 172.21.15.2:0/2041418283 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3aac0a2b80 msgr2=0x7f3aac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:29.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.983+0000 7f3acb600640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3aac0a2b80 0x7f3aac0a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x564710b73060 tx=0x7f3ab8005f90 comp rx=0 tx=0).stop 2024-09-10T08:11:29.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.983+0000 7f3acb600640 1 -- 172.21.15.2:0/2041418283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ac41049c0 msgr2=0x7f3ac4076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:29.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.983+0000 7f3acb600640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ac41049c0 0x7f3ac4076c40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000ea20 tx=0x7f3ab000eee0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.983+0000 7f3acb600640 1 -- 172.21.15.2:0/2041418283 shutdown_connections 2024-09-10T08:11:29.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.983+0000 7f3acb600640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3aac0a2b80 0x7f3aac0a5040 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.983+0000 7f3acb600640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac4109140 0x7f3ac4077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.983+0000 7f3acb600640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ac41049c0 0x7f3ac4076c40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.983+0000 7f3acb600640 1 --2- 172.21.15.2:0/2041418283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ac4104010 0x7f3ac4076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:29.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.983+0000 7f3acb600640 1 -- 172.21.15.2:0/2041418283 >> 172.21.15.2:0/2041418283 conn(0x7f3ac40ffa50 msgr2=0x7f3ac4100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:29.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.983+0000 7f3acb600640 1 -- 172.21.15.2:0/2041418283 shutdown_connections 2024-09-10T08:11:29.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:29.983+0000 7f3acb600640 1 -- 172.21.15.2:0/2041418283 wait complete. 2024-09-10T08:11:30.817 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:11:31.195 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:11:31.588 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:31 smithi002 ceph-mon[30563]: pgmap v488: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:11:31.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:31 smithi155 ceph-mon[35405]: pgmap v488: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:11:31.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:31 smithi107 ceph-mon[35048]: pgmap v488: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:11:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.746+0000 7f13dd200640 1 -- 172.21.15.2:0/4244456511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13d81047e0 msgr2=0x7f13d8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.746+0000 7f13dd200640 1 --2- 172.21.15.2:0/4244456511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13d81047e0 0x7f13d8108830 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009900 tx=0x7f13c402a410 comp rx=0 tx=0).stop 2024-09-10T08:11:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.746+0000 7f13dd200640 1 -- 172.21.15.2:0/4244456511 shutdown_connections 2024-09-10T08:11:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.746+0000 7f13dd200640 1 --2- 172.21.15.2:0/4244456511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13d8108f60 0x7f13d8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.746+0000 7f13dd200640 1 --2- 172.21.15.2:0/4244456511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13d81047e0 0x7f13d8108830 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.746+0000 7f13dd200640 1 --2- 172.21.15.2:0/4244456511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d8103e30 0x7f13d8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.746+0000 7f13dd200640 1 -- 172.21.15.2:0/4244456511 >> 172.21.15.2:0/4244456511 conn(0x7f13d80ff910 msgr2=0x7f13d8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.746+0000 7f13dd200640 1 -- 172.21.15.2:0/4244456511 shutdown_connections 2024-09-10T08:11:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.746+0000 7f13dd200640 1 -- 172.21.15.2:0/4244456511 wait complete. 2024-09-10T08:11:31.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.747+0000 7f13dd200640 1 Processor -- start 2024-09-10T08:11:31.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.747+0000 7f13dd200640 1 -- start start 2024-09-10T08:11:31.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.747+0000 7f13dd200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13d8103e30 0x7f13d819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.747+0000 7f13dd200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d81047e0 0x7f13d819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.747+0000 7f13dd200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13d8108f60 0x7f13d819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.747+0000 7f13dd200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13d8113fc0 con 0x7f13d8103e30 2024-09-10T08:11:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.747+0000 7f13dd200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13d8113e40 con 0x7f13d81047e0 2024-09-10T08:11:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.747+0000 7f13dd200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13d8114140 con 0x7f13d8108f60 2024-09-10T08:11:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.748+0000 7f13d7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13d8103e30 0x7f13d819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.748+0000 7f13d7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13d8108f60 0x7f13d819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.748+0000 7f13d6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d81047e0 0x7f13d819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.748+0000 7f13d7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13d8108f60 0x7f13d819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56420/0 (socket says 172.21.15.2:56420) 2024-09-10T08:11:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.748+0000 7f13d7e00640 1 -- 172.21.15.2:0/3204301195 learned_addr learned my addr 172.21.15.2:0/3204301195 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.748+0000 7f13d7e00640 1 -- 172.21.15.2:0/3204301195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d81047e0 msgr2=0x7f13d819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.748+0000 7f13d7e00640 1 --2- 172.21.15.2:0/3204301195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d81047e0 0x7f13d819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.748+0000 7f13d7e00640 1 -- 172.21.15.2:0/3204301195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13d8103e30 msgr2=0x7f13d819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.748+0000 7f13d7e00640 1 --2- 172.21.15.2:0/3204301195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13d8103e30 0x7f13d819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.748+0000 7f13d7e00640 1 -- 172.21.15.2:0/3204301195 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d819f470 con 0x7f13d8108f60 2024-09-10T08:11:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.748+0000 7f13d7e00640 1 --2- 172.21.15.2:0/3204301195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13d8108f60 0x7f13d819ed90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f13d000eb10 tx=0x7f13d000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.749+0000 7f13cfe00640 1 -- 172.21.15.2:0/3204301195 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0004b20 con 0x7f13d8108f60 2024-09-10T08:11:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.749+0000 7f13cfe00640 1 -- 172.21.15.2:0/3204301195 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d0004cc0 con 0x7f13d8108f60 2024-09-10T08:11:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.749+0000 7f13cfe00640 1 -- 172.21.15.2:0/3204301195 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0010760 con 0x7f13d8108f60 2024-09-10T08:11:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.749+0000 7f13dd200640 1 -- 172.21.15.2:0/3204301195 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d819f760 con 0x7f13d8108f60 2024-09-10T08:11:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.749+0000 7f13dd200640 1 -- 172.21.15.2:0/3204301195 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d81a7040 con 0x7f13d8108f60 2024-09-10T08:11:31.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.753+0000 7f13e1465740 1 Processor -- start 2024-09-10T08:11:31.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.753+0000 7f13cfe00640 1 -- 172.21.15.2:0/3204301195 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f13d0005020 con 0x7f13d8108f60 2024-09-10T08:11:31.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.753+0000 7f13e1465740 1 -- start start 2024-09-10T08:11:31.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.754+0000 7f13e1465740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03be40 0x55897b03c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.754+0000 7f13e1465740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b046ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.754+0000 7f13e1465740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b047020 0x55897b049410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.754+0000 7f13e1465740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55897afa23e0 con 0x55897b03be40 2024-09-10T08:11:31.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.754+0000 7f13e1465740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55897af49d80 con 0x55897b047020 2024-09-10T08:11:31.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.754+0000 7f13e1465740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55897af9e3a0 con 0x55897b03c760 2024-09-10T08:11:31.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.754+0000 7f13d6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03be40 0x55897b03c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.754+0000 7f13d6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03be40 0x55897b03c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56770/0 (socket says 172.21.15.2:56770) 2024-09-10T08:11:31.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.754+0000 7f13d6a00640 1 -- 172.21.15.2:0/3629500534 learned_addr learned my addr 172.21.15.2:0/3629500534 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:31.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.754+0000 7f13d7400640 1 --2- 172.21.15.2:0/3629500534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b046ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.754+0000 7f13d6a00640 1 --2- 172.21.15.2:0/3629500534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b047020 0x55897b049410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.755+0000 7f13d6a00640 1 -- 172.21.15.2:0/3629500534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 msgr2=0x55897b046ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:31.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.755+0000 7f13d6a00640 1 --2- 172.21.15.2:0/3629500534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b046ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.755+0000 7f13d6a00640 1 -- 172.21.15.2:0/3629500534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b047020 msgr2=0x55897b049410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.755+0000 7f13d6a00640 1 --2- 172.21.15.2:0/3629500534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b047020 0x55897b049410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.755+0000 7f13d6a00640 1 -- 172.21.15.2:0/3629500534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55897af68ae0 con 0x55897b03be40 2024-09-10T08:11:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.755+0000 7f13d7400640 1 --2- 172.21.15.2:0/3629500534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b046ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.755+0000 7f13cfe00640 1 --2- 172.21.15.2:0/3204301195 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13bc0a2c00 0x7f13bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.755+0000 7f13d6a00640 1 --2- 172.21.15.2:0/3629500534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03be40 0x55897b03c220 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55897b039750 tx=0x7f13c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd64a0557d258ccc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.755+0000 7f13cfe00640 1 -- 172.21.15.2:0/3204301195 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f13d00cf920 con 0x7f13d8108f60 2024-09-10T08:11:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.755+0000 7f13ccc00640 1 -- 172.21.15.2:0/3629500534 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4044030 con 0x55897b03be40 2024-09-10T08:11:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.755+0000 7f13d7400640 1 --2- 172.21.15.2:0/3204301195 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13bc0a2c00 0x7f13bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.755+0000 7f13ccc00640 1 -- 172.21.15.2:0/3629500534 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c40319d0 con 0x55897b03be40 2024-09-10T08:11:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.755+0000 7f13ccc00640 1 -- 172.21.15.2:0/3629500534 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4035040 con 0x55897b03be40 2024-09-10T08:11:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.756+0000 7f13e1465740 1 -- 172.21.15.2:0/3629500534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03be40 msgr2=0x55897b03c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.756+0000 7f13e1465740 1 --2- 172.21.15.2:0/3629500534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03be40 0x55897b03c220 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55897b039750 tx=0x7f13c4000bc0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.756+0000 7f13d7400640 1 --2- 172.21.15.2:0/3204301195 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13bc0a2c00 0x7f13bc0a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55897b039230 tx=0x7f13c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:31.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.756+0000 7f13e1465740 1 -- 172.21.15.2:0/3629500534 shutdown_connections 2024-09-10T08:11:31.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.756+0000 7f13e1465740 1 --2- 172.21.15.2:0/3629500534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b047020 0x55897b049410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.756+0000 7f13e1465740 1 --2- 172.21.15.2:0/3629500534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b046ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.756+0000 7f13e1465740 1 --2- 172.21.15.2:0/3629500534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03be40 0x55897b03c220 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.756+0000 7f13e1465740 1 -- 172.21.15.2:0/3629500534 >> 172.21.15.2:0/3629500534 conn(0x55897b0383d0 msgr2=0x55897b0387b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:31.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.756+0000 7f13e1465740 1 -- 172.21.15.2:0/3629500534 shutdown_connections 2024-09-10T08:11:31.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.757+0000 7f13e1465740 1 -- 172.21.15.2:0/3629500534 wait complete. 2024-09-10T08:11:31.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.757+0000 7f13e1465740 1 -- start start 2024-09-10T08:11:31.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.757+0000 7f13e1465740 1 Processor -- start 2024-09-10T08:11:31.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.758+0000 7f13e1465740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b043fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.758+0000 7f13e1465740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b044510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.758+0000 7f13e1465740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b047020 0x55897b044a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.758+0000 7f13e1465740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55897b049bf0 con 0x55897b047020 2024-09-10T08:11:31.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.758+0000 7f13e1465740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55897af5d0f0 con 0x55897b03be40 2024-09-10T08:11:31.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.758+0000 7f13e1465740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55897b049d70 con 0x55897b03c760 2024-09-10T08:11:31.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.758+0000 7f13e1465740 10 client.0 _ll_register_callbacks cb 0x55897af44bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:31.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.758+0000 7f13d7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b047020 0x55897b044a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.758+0000 7f13d7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b047020 0x55897b044a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56776/0 (socket says 172.21.15.2:56776) 2024-09-10T08:11:31.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.758+0000 7f13d7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b043fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.758+0000 7f13d7e00640 1 -- 172.21.15.2:0/2910425690 learned_addr learned my addr 172.21.15.2:0/2910425690 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.758+0000 7f13d6a00640 1 --2- 172.21.15.2:0/2910425690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b044510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.759+0000 7f13d7400640 1 -- 172.21.15.2:0/2910425690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 msgr2=0x55897b044510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.759+0000 7f13d7400640 1 --2- 172.21.15.2:0/2910425690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b044510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.759+0000 7f13d7400640 1 -- 172.21.15.2:0/2910425690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b047020 msgr2=0x55897b044a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.759+0000 7f13d7400640 1 --2- 172.21.15.2:0/2910425690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b047020 0x55897b044a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.759+0000 7f13d7400640 1 -- 172.21.15.2:0/2910425690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897ad133c0 con 0x55897b03be40 2024-09-10T08:11:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.759+0000 7f13d6a00640 1 --2- 172.21.15.2:0/2910425690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b044510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.759+0000 7f13d7400640 1 --2- 172.21.15.2:0/2910425690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b043fd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13c8012cb0 tx=0x7f13c8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256515ba89576111 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.759+0000 7f13c1600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:11:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.759+0000 7f13c1600640 1 -- 172.21.15.2:0/2910425690 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c801ede0 con 0x55897b03be40 2024-09-10T08:11:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.759+0000 7f13e1465740 1 -- 172.21.15.2:0/2910425690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13c40319d0 con 0x55897b03be40 2024-09-10T08:11:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.759+0000 7f13c1600640 1 -- 172.21.15.2:0/2910425690 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f13c8018030 con 0x55897b03be40 2024-09-10T08:11:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.760+0000 7f13c1600640 1 -- 172.21.15.2:0/2910425690 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f13c8034050 con 0x55897b03be40 2024-09-10T08:11:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.760+0000 7f13e1465740 10 client.26249 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.760+0000 7f13e1465740 1 -- 172.21.15.2:0/2910425690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55897b055da0 con 0x55897b03be40 2024-09-10T08:11:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.760+0000 7f13c1600640 1 -- 172.21.15.2:0/2910425690 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c40319d0 con 0x55897b03be40 2024-09-10T08:11:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.760+0000 7f13c1600640 1 -- 172.21.15.2:0/2910425690 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f13c800f560 con 0x55897b03be40 2024-09-10T08:11:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.760+0000 7f13e1465740 10 client.26249 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.760+0000 7f13e1465740 10 client.26249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.760+0000 7f13e1465740 20 client.26249 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.760+0000 7f13e1465740 1 --2- 172.21.15.2:0/2910425690 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55897b0571b0 0x55897b059630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.760+0000 7f13e1465740 4 client.26249 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:11:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.760+0000 7f13e1465740 1 -- 172.21.15.2:0/2910425690 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55897afaf950 con 0x55897b0571b0 2024-09-10T08:11:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.761+0000 7f13d6a00640 1 --2- 172.21.15.2:0/2910425690 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55897b0571b0 0x55897b059630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.761+0000 7f13d6a00640 1 --2- 172.21.15.2:0/2910425690 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55897b0571b0 0x55897b059630 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fc335e55303d023 server_cookie=20e6d43a2153abd0 in_seq=0 out_seq=0 2024-09-10T08:11:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.761+0000 7f13c1600640 10 client.26249 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13c1600640 1 -- 172.21.15.2:0/2910425690 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55897afaf950 con 0x55897b0571b0 2024-09-10T08:11:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13c1600640 10 client.26249 handle_command_reply: tid=0 2024-09-10T08:11:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13e1465740 1 client.26249 shutdown 2024-09-10T08:11:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13e1465740 1 -- 172.21.15.2:0/2910425690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 msgr2=0x55897b043fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13e1465740 1 --2- 172.21.15.2:0/2910425690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b043fd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13c8012cb0 tx=0x7f13c8004730 comp rx=0 tx=0).stop 2024-09-10T08:11:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13e1465740 1 -- 172.21.15.2:0/2910425690 shutdown_connections 2024-09-10T08:11:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13e1465740 1 --2- 172.21.15.2:0/2910425690 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55897b0571b0 0x55897b059630 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13e1465740 1 --2- 172.21.15.2:0/2910425690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b047020 0x55897b044a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13e1465740 1 --2- 172.21.15.2:0/2910425690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b044510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13c1600640 0 client.26249 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13e1465740 1 --2- 172.21.15.2:0/2910425690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b043fd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13e1465740 1 -- 172.21.15.2:0/2910425690 >> 172.21.15.2:0/2910425690 conn(0x55897b0383d0 msgr2=0x55897b03e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13e1465740 1 -- 172.21.15.2:0/2910425690 shutdown_connections 2024-09-10T08:11:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.762+0000 7f13e1465740 1 -- 172.21.15.2:0/2910425690 wait complete. 2024-09-10T08:11:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.763+0000 7f13e1465740 20 client.26249 trim_cache size 0 max 16384 2024-09-10T08:11:31.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.776+0000 7f13e1465740 1 Processor -- start 2024-09-10T08:11:31.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.776+0000 7f13e1465740 1 -- start start 2024-09-10T08:11:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13e1465740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b03e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13e1465740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03c760 0x55897b05ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13e1465740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b047020 0x55897b05f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13e1465740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81b3770 con 0x55897b03c760 2024-09-10T08:11:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13e1465740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81b1040 con 0x55897b03be40 2024-09-10T08:11:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13e1465740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81aee90 con 0x55897b047020 2024-09-10T08:11:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13d6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03c760 0x55897b05ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13d6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03c760 0x55897b05ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56780/0 (socket says 172.21.15.2:56780) 2024-09-10T08:11:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13d6a00640 1 -- 172.21.15.2:0/4188314520 learned_addr learned my addr 172.21.15.2:0/4188314520 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13d7400640 1 --2- 172.21.15.2:0/4188314520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b03e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13d7e00640 1 --2- 172.21.15.2:0/4188314520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b047020 0x55897b05f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13d6a00640 1 -- 172.21.15.2:0/4188314520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b047020 msgr2=0x55897b05f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13d6a00640 1 --2- 172.21.15.2:0/4188314520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b047020 0x55897b05f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13d6a00640 1 -- 172.21.15.2:0/4188314520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 msgr2=0x55897b03e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13d6a00640 1 --2- 172.21.15.2:0/4188314520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b03e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.777+0000 7f13d6a00640 1 -- 172.21.15.2:0/4188314520 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55897b039060 con 0x55897b03c760 2024-09-10T08:11:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.778+0000 7f13d6a00640 1 --2- 172.21.15.2:0/4188314520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03c760 0x55897b05ef00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009900 tx=0x7f13c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd3388d0100daba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.778+0000 7f13c3e00640 1 -- 172.21.15.2:0/4188314520 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4057070 con 0x55897b03c760 2024-09-10T08:11:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.778+0000 7f13c3e00640 1 -- 172.21.15.2:0/4188314520 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c40315b0 con 0x55897b03c760 2024-09-10T08:11:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.778+0000 7f13c3e00640 1 -- 172.21.15.2:0/4188314520 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4035040 con 0x55897b03c760 2024-09-10T08:11:31.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.779+0000 7f13e1465740 1 -- 172.21.15.2:0/4188314520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03c760 msgr2=0x55897b05ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:31.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.779+0000 7f13e1465740 1 --2- 172.21.15.2:0/4188314520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03c760 0x55897b05ef00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009900 tx=0x7f13c4005fd0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.779+0000 7f13e1465740 1 -- 172.21.15.2:0/4188314520 shutdown_connections 2024-09-10T08:11:31.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.779+0000 7f13e1465740 1 --2- 172.21.15.2:0/4188314520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b047020 0x55897b05f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.779+0000 7f13e1465740 1 --2- 172.21.15.2:0/4188314520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b03c760 0x55897b05ef00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.779+0000 7f13e1465740 1 --2- 172.21.15.2:0/4188314520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b03e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.779+0000 7f13e1465740 1 -- 172.21.15.2:0/4188314520 >> 172.21.15.2:0/4188314520 conn(0x55897b0383d0 msgr2=0x55897b0447b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:31.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.779+0000 7f13e1465740 1 -- 172.21.15.2:0/4188314520 shutdown_connections 2024-09-10T08:11:31.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.779+0000 7f13e1465740 1 -- 172.21.15.2:0/4188314520 wait complete. 2024-09-10T08:11:31.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.779+0000 7f13e1465740 1 -- start start 2024-09-10T08:11:31.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.780+0000 7f13e1465740 1 Processor -- start 2024-09-10T08:11:31.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.780+0000 7f13e1465740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b04ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.780+0000 7f13e1465740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b04b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.780+0000 7f13e1465740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b047020 0x55897b04b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.780+0000 7f13e1465740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81af500 con 0x55897b047020 2024-09-10T08:11:31.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.780+0000 7f13e1465740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81b0970 con 0x55897b03be40 2024-09-10T08:11:31.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.780+0000 7f13e1465740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81afbd0 con 0x55897b03c760 2024-09-10T08:11:31.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.780+0000 7f13e1465740 10 client.0 _ll_register_callbacks cb 0x7f13c8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:31.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.780+0000 7f13d7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b047020 0x55897b04b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.780+0000 7f13d7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b04ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.781+0000 7f13d6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b04b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.781+0000 7f13d7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b04ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41326/0 (socket says 172.21.15.2:41326) 2024-09-10T08:11:31.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.781+0000 7f13d7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b047020 0x55897b04b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56782/0 (socket says 172.21.15.2:56782) 2024-09-10T08:11:31.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.781+0000 7f13d7400640 1 -- 172.21.15.2:0/4233844925 learned_addr learned my addr 172.21.15.2:0/4233844925 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:31.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.781+0000 7f13d6a00640 1 -- 172.21.15.2:0/4233844925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 msgr2=0x55897b04ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:31.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.781+0000 7f13d6a00640 1 --2- 172.21.15.2:0/4233844925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b04ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.781+0000 7f13d6a00640 1 -- 172.21.15.2:0/4233844925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b047020 msgr2=0x55897b04b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:31.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.781+0000 7f13d6a00640 1 --2- 172.21.15.2:0/4233844925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b047020 0x55897b04b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.781+0000 7f13d6a00640 1 -- 172.21.15.2:0/4233844925 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897b0c0ed0 con 0x55897b03c760 2024-09-10T08:11:31.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.781+0000 7f13d7400640 1 --2- 172.21.15.2:0/4233844925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b04ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:31.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.781+0000 7f13d7e00640 1 --2- 172.21.15.2:0/4233844925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b047020 0x55897b04b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:31.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.781+0000 7f13d6a00640 1 --2- 172.21.15.2:0/4233844925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b04b190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55897b033a00 tx=0x7f13c402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebbdd0a746a78938 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:31.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.781+0000 7f13c1600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:11:31.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.782+0000 7f13c1600640 1 -- 172.21.15.2:0/4233844925 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4057070 con 0x55897b03c760 2024-09-10T08:11:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.782+0000 7f13c1600640 1 -- 172.21.15.2:0/4233844925 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f13c40035f0 con 0x55897b03c760 2024-09-10T08:11:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.782+0000 7f13e1465740 1 -- 172.21.15.2:0/4233844925 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13c40315b0 con 0x55897b03c760 2024-09-10T08:11:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.782+0000 7f13c1600640 1 -- 172.21.15.2:0/4233844925 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f13c4035070 con 0x55897b03c760 2024-09-10T08:11:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.782+0000 7f13e1465740 10 client.26221 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.782+0000 7f13e1465740 1 -- 172.21.15.2:0/4233844925 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55897b0c20f0 con 0x55897b03c760 2024-09-10T08:11:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.782+0000 7f13c1600640 1 -- 172.21.15.2:0/4233844925 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c40315b0 con 0x55897b03c760 2024-09-10T08:11:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.783+0000 7f13c1600640 1 -- 172.21.15.2:0/4233844925 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f13c4049440 con 0x55897b03c760 2024-09-10T08:11:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.783+0000 7f13e1465740 10 client.26221 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.783+0000 7f13e1465740 10 client.26221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.783+0000 7f13e1465740 20 client.26221 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.783+0000 7f13e1465740 1 --2- 172.21.15.2:0/4233844925 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55897b0c2320 0x55897b0c4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.783+0000 7f13e1465740 4 client.26221 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:11:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.783+0000 7f13e1465740 1 -- 172.21.15.2:0/4233844925 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55897afa9e60 con 0x55897b0c2320 2024-09-10T08:11:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.783+0000 7f13d7400640 1 --2- 172.21.15.2:0/4233844925 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55897b0c2320 0x55897b0c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.783+0000 7f13d7400640 1 --2- 172.21.15.2:0/4233844925 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55897b0c2320 0x55897b0c4710 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfd392dbd7526d43 server_cookie=a94adf9bc013f04a in_seq=0 out_seq=0 2024-09-10T08:11:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.783+0000 7f13c1600640 10 client.26221 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13c1600640 1 -- 172.21.15.2:0/4233844925 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55897afa9e60 con 0x55897b0c2320 2024-09-10T08:11:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13c1600640 10 client.26221 handle_command_reply: tid=0 2024-09-10T08:11:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13e1465740 1 client.26221 shutdown 2024-09-10T08:11:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13e1465740 1 -- 172.21.15.2:0/4233844925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 msgr2=0x55897b04b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13e1465740 1 --2- 172.21.15.2:0/4233844925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b04b190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55897b033a00 tx=0x7f13c402a410 comp rx=0 tx=0).stop 2024-09-10T08:11:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13e1465740 1 -- 172.21.15.2:0/4233844925 shutdown_connections 2024-09-10T08:11:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13e1465740 1 --2- 172.21.15.2:0/4233844925 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55897b0c2320 0x55897b0c4710 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13e1465740 1 --2- 172.21.15.2:0/4233844925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55897b047020 0x55897b04b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13e1465740 1 --2- 172.21.15.2:0/4233844925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55897b03c760 0x55897b04b190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13c1600640 0 client.26221 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:31.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13e1465740 1 --2- 172.21.15.2:0/4233844925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55897b03be40 0x55897b04ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13e1465740 1 -- 172.21.15.2:0/4233844925 >> 172.21.15.2:0/4233844925 conn(0x55897b0383d0 msgr2=0x55897b036c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:31.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13e1465740 1 -- 172.21.15.2:0/4233844925 shutdown_connections 2024-09-10T08:11:31.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13e1465740 1 -- 172.21.15.2:0/4233844925 wait complete. 2024-09-10T08:11:31.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.784+0000 7f13e1465740 20 client.26221 trim_cache size 0 max 16384 2024-09-10T08:11:31.791 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:11:31.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.785+0000 7f13dd200640 1 -- 172.21.15.2:0/3204301195 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13bc0a2c00 msgr2=0x7f13bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:31.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.785+0000 7f13dd200640 1 --2- 172.21.15.2:0/3204301195 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13bc0a2c00 0x7f13bc0a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55897b039230 tx=0x7f13c8005f90 comp rx=0 tx=0).stop 2024-09-10T08:11:31.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.785+0000 7f13dd200640 1 -- 172.21.15.2:0/3204301195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13d8108f60 msgr2=0x7f13d819ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:31.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.785+0000 7f13dd200640 1 --2- 172.21.15.2:0/3204301195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13d8108f60 0x7f13d819ed90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f13d000eb10 tx=0x7f13d000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.785+0000 7f13dd200640 1 -- 172.21.15.2:0/3204301195 shutdown_connections 2024-09-10T08:11:31.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.785+0000 7f13dd200640 1 --2- 172.21.15.2:0/3204301195 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13bc0a2c00 0x7f13bc0a50c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.785+0000 7f13dd200640 1 --2- 172.21.15.2:0/3204301195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13d8108f60 0x7f13d819ed90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.785+0000 7f13dd200640 1 --2- 172.21.15.2:0/3204301195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13d81047e0 0x7f13d819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.785+0000 7f13dd200640 1 --2- 172.21.15.2:0/3204301195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13d8103e30 0x7f13d819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:31.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.785+0000 7f13dd200640 1 -- 172.21.15.2:0/3204301195 >> 172.21.15.2:0/3204301195 conn(0x7f13d80ff910 msgr2=0x7f13d8100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:31.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.785+0000 7f13dd200640 1 -- 172.21.15.2:0/3204301195 shutdown_connections 2024-09-10T08:11:31.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:31.785+0000 7f13dd200640 1 -- 172.21.15.2:0/3204301195 wait complete. 2024-09-10T08:11:33.604 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:11:33.605 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:11:33.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:33 smithi155 ceph-mon[35405]: pgmap v489: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:11:33.659 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:33 smithi002 ceph-mon[30563]: pgmap v489: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:11:33.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:33 smithi107 ceph-mon[35048]: pgmap v489: 129 pgs: 129 active+clean; 1.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:11:33.998 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:11:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.489+0000 7fd3ebe00640 1 -- 172.21.15.2:0/1970340692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3ec104d60 msgr2=0x7fd3ec1051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.489+0000 7fd3ebe00640 1 --2- 172.21.15.2:0/1970340692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3ec104d60 0x7fd3ec1051e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009900 tx=0x7fd3d802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.489+0000 7fd3ebe00640 1 -- 172.21.15.2:0/1970340692 shutdown_connections 2024-09-10T08:11:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.489+0000 7fd3ebe00640 1 --2- 172.21.15.2:0/1970340692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3ec105720 0x7fd3ec10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.489+0000 7fd3ebe00640 1 --2- 172.21.15.2:0/1970340692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3ec104d60 0x7fd3ec1051e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.489+0000 7fd3ebe00640 1 --2- 172.21.15.2:0/1970340692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3ec103b70 0x7fd3ec103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.489+0000 7fd3ebe00640 1 -- 172.21.15.2:0/1970340692 >> 172.21.15.2:0/1970340692 conn(0x7fd3ec0ff310 msgr2=0x7fd3ec101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.489+0000 7fd3ebe00640 1 -- 172.21.15.2:0/1970340692 shutdown_connections 2024-09-10T08:11:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.489+0000 7fd3ebe00640 1 -- 172.21.15.2:0/1970340692 wait complete. 2024-09-10T08:11:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.490+0000 7fd3ebe00640 1 Processor -- start 2024-09-10T08:11:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.490+0000 7fd3ebe00640 1 -- start start 2024-09-10T08:11:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.490+0000 7fd3ebe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3ec103b70 0x7fd3ec199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.490+0000 7fd3ebe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3ec104d60 0x7fd3ec19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.490+0000 7fd3ebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3ec105720 0x7fd3ec1a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.490+0000 7fd3ebe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec10f7d0 con 0x7fd3ec104d60 2024-09-10T08:11:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.490+0000 7fd3ebe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec10f650 con 0x7fd3ec103b70 2024-09-10T08:11:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.490+0000 7fd3ebe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec10f950 con 0x7fd3ec105720 2024-09-10T08:11:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.491+0000 7fd3eaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3ec103b70 0x7fd3ec199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.491+0000 7fd3eaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3ec103b70 0x7fd3ec199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41346/0 (socket says 172.21.15.2:41346) 2024-09-10T08:11:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.491+0000 7fd3eaa00640 1 -- 172.21.15.2:0/265015531 learned_addr learned my addr 172.21.15.2:0/265015531 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.491+0000 7fd3ea000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3ec104d60 0x7fd3ec19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.491+0000 7fd3eb400640 1 --2- 172.21.15.2:0/265015531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3ec105720 0x7fd3ec1a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.491+0000 7fd3eaa00640 1 -- 172.21.15.2:0/265015531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3ec105720 msgr2=0x7fd3ec1a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.491+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/265015531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3ec105720 0x7fd3ec1a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.491+0000 7fd3eaa00640 1 -- 172.21.15.2:0/265015531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3ec104d60 msgr2=0x7fd3ec19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.491+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/265015531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3ec104d60 0x7fd3ec19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.491+0000 7fd3eaa00640 1 -- 172.21.15.2:0/265015531 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ec1a1aa0 con 0x7fd3ec103b70 2024-09-10T08:11:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.492+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/265015531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3ec103b70 0x7fd3ec199e80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00e9c0 tx=0x7fd3dc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.492+0000 7fd3e3400640 1 -- 172.21.15.2:0/265015531 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc00cde0 con 0x7fd3ec103b70 2024-09-10T08:11:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.492+0000 7fd3ebe00640 1 -- 172.21.15.2:0/265015531 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ec1a1d90 con 0x7fd3ec103b70 2024-09-10T08:11:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.492+0000 7fd3e3400640 1 -- 172.21.15.2:0/265015531 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3dc010910 con 0x7fd3ec103b70 2024-09-10T08:11:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.492+0000 7fd3e3400640 1 -- 172.21.15.2:0/265015531 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc018a10 con 0x7fd3ec103b70 2024-09-10T08:11:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.492+0000 7fd3ebe00640 1 -- 172.21.15.2:0/265015531 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ec1a2250 con 0x7fd3ec103b70 2024-09-10T08:11:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.495+0000 7fd3f4a75740 1 Processor -- start 2024-09-10T08:11:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.495+0000 7fd3f4a75740 1 -- start start 2024-09-10T08:11:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.495+0000 7fd3e3400640 1 -- 172.21.15.2:0/265015531 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd3dc01f020 con 0x7fd3ec103b70 2024-09-10T08:11:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.495+0000 7fd3f4a75740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 0x55e1359d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.495+0000 7fd3f4a75740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359d5ba0 0x55e1359da660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.495+0000 7fd3f4a75740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359dac30 0x55e1359dd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.495+0000 7fd3f4a75740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e13593a100 con 0x55e1359d5260 2024-09-10T08:11:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.495+0000 7fd3f4a75740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e1358e21b0 con 0x55e1359dac30 2024-09-10T08:11:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.495+0000 7fd3f4a75740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e1359360c0 con 0x55e1359d5ba0 2024-09-10T08:11:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.496+0000 7fd3eb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 0x55e1359d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.496+0000 7fd3eaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359d5ba0 0x55e1359da660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.496+0000 7fd3eb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 0x55e1359d5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56806/0 (socket says 172.21.15.2:56806) 2024-09-10T08:11:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.496+0000 7fd3eb400640 1 -- 172.21.15.2:0/1879621091 learned_addr learned my addr 172.21.15.2:0/1879621091 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.496+0000 7fd3ea000640 1 --2- 172.21.15.2:0/1879621091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359dac30 0x55e1359dd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.496+0000 7fd3eaa00640 1 -- 172.21.15.2:0/1879621091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359dac30 msgr2=0x55e1359dd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.496+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/1879621091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359dac30 0x55e1359dd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.496+0000 7fd3eaa00640 1 -- 172.21.15.2:0/1879621091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 msgr2=0x55e1359d5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.496+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/1879621091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 0x55e1359d5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.496+0000 7fd3eaa00640 1 -- 172.21.15.2:0/1879621091 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e13564d210 con 0x55e1359d5ba0 2024-09-10T08:11:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.496+0000 7fd3eb400640 1 --2- 172.21.15.2:0/1879621091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 0x55e1359d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.497+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/1879621091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359d5ba0 0x55e1359da660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0cccf0 tx=0x7fd3dc0d5cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af869a199dc6591b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.497+0000 7fd3e3400640 1 --2- 172.21.15.2:0/265015531 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd3d00a2b80 0x7fd3d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.497+0000 7fd3e3400640 1 -- 172.21.15.2:0/265015531 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd3dc0cf6a0 con 0x7fd3ec103b70 2024-09-10T08:11:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.497+0000 7fd3d7e00640 1 -- 172.21.15.2:0/1879621091 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0de070 con 0x55e1359d5ba0 2024-09-10T08:11:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.497+0000 7fd3d7e00640 1 -- 172.21.15.2:0/1879621091 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3dc09c1b0 con 0x55e1359d5ba0 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.497+0000 7fd3d7e00640 1 -- 172.21.15.2:0/1879621091 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc09b880 con 0x55e1359d5ba0 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.497+0000 7fd3eb400640 1 --2- 172.21.15.2:0/265015531 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd3d00a2b80 0x7fd3d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.498+0000 7fd3f4a75740 1 -- 172.21.15.2:0/1879621091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359d5ba0 msgr2=0x55e1359da660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.498+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/1879621091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359d5ba0 0x55e1359da660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0cccf0 tx=0x7fd3dc0d5cb0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.498+0000 7fd3eb400640 1 --2- 172.21.15.2:0/265015531 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd3d00a2b80 0x7fd3d00a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4011890 tx=0x7fd3e4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.498+0000 7fd3f4a75740 1 -- 172.21.15.2:0/1879621091 shutdown_connections 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.498+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/1879621091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359dac30 0x55e1359dd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.498+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/1879621091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359d5ba0 0x55e1359da660 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.498+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/1879621091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 0x55e1359d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.498+0000 7fd3f4a75740 1 -- 172.21.15.2:0/1879621091 >> 172.21.15.2:0/1879621091 conn(0x55e1359d08d0 msgr2=0x55e1359d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.498+0000 7fd3f4a75740 1 -- 172.21.15.2:0/1879621091 shutdown_connections 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.498+0000 7fd3f4a75740 1 -- 172.21.15.2:0/1879621091 wait complete. 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.498+0000 7fd3f4a75740 1 -- start start 2024-09-10T08:11:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.498+0000 7fd3f4a75740 1 Processor -- start 2024-09-10T08:11:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.499+0000 7fd3f4a75740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 0x55e1359df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.499+0000 7fd3f4a75740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5ba0 0x55e1359df920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.499+0000 7fd3f4a75740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359dac30 0x55e1359e6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.499+0000 7fd3f4a75740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e1359ddc00 con 0x55e1359d5260 2024-09-10T08:11:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.499+0000 7fd3f4a75740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e1358c3090 con 0x55e1359d5ba0 2024-09-10T08:11:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.499+0000 7fd3f4a75740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e1359ddd80 con 0x55e1359dac30 2024-09-10T08:11:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.499+0000 7fd3f4a75740 10 client.0 _ll_register_callbacks cb 0x55e135917f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.499+0000 7fd3eaa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 0x55e1359df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.499+0000 7fd3eaa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 0x55e1359df3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56818/0 (socket says 172.21.15.2:56818) 2024-09-10T08:11:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.499+0000 7fd3eaa00640 1 -- 172.21.15.2:0/1151057554 learned_addr learned my addr 172.21.15.2:0/1151057554 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.499+0000 7fd3ea000640 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5ba0 0x55e1359df920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.499+0000 7fd3eb400640 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359dac30 0x55e1359e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3eaa00640 1 -- 172.21.15.2:0/1151057554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359dac30 msgr2=0x55e1359e6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359dac30 0x55e1359e6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3eaa00640 1 -- 172.21.15.2:0/1151057554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5ba0 msgr2=0x55e1359df920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5ba0 0x55e1359df920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3eaa00640 1 -- 172.21.15.2:0/1151057554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e135719620 con 0x55e1359d5260 2024-09-10T08:11:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3eb400640 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359dac30 0x55e1359e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:11:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 0x55e1359df3e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0a41f0 tx=0x7fd3dc0d5390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d855e1207b23033 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:11:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3d4c00640 1 -- 172.21.15.2:0/1151057554 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0de070 con 0x55e1359d5260 2024-09-10T08:11:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3d4c00640 1 -- 172.21.15.2:0/1151057554 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd3dc018bb0 con 0x55e1359d5260 2024-09-10T08:11:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3f4a75740 1 -- 172.21.15.2:0/1151057554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3dc09c1b0 con 0x55e1359d5260 2024-09-10T08:11:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3d4c00640 1 -- 172.21.15.2:0/1151057554 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd3dc0b4050 con 0x55e1359d5260 2024-09-10T08:11:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3f4a75740 10 client.16233 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.500+0000 7fd3f4a75740 1 -- 172.21.15.2:0/1151057554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e1359e9750 con 0x55e1359d5260 2024-09-10T08:11:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.501+0000 7fd3d4c00640 1 -- 172.21.15.2:0/1151057554 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3dc09c1b0 con 0x55e1359d5260 2024-09-10T08:11:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.501+0000 7fd3d4c00640 1 -- 172.21.15.2:0/1151057554 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd3dc0cf310 con 0x55e1359d5260 2024-09-10T08:11:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.501+0000 7fd3f4a75740 10 client.16233 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.501+0000 7fd3f4a75740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.501+0000 7fd3f4a75740 20 client.16233 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.501+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e1359eaaa0 0x55e1359ece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.501+0000 7fd3f4a75740 4 client.16233 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:11:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.501+0000 7fd3f4a75740 1 -- 172.21.15.2:0/1151057554 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e135947830 con 0x55e1359eaaa0 2024-09-10T08:11:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.501+0000 7fd3ea000640 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e1359eaaa0 0x55e1359ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.502+0000 7fd3ea000640 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e1359eaaa0 0x55e1359ece90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd4f1250af3e9462 server_cookie=79a64d8fc3c4e0a7 in_seq=0 out_seq=0 2024-09-10T08:11:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.502+0000 7fd3d4c00640 10 client.16233 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.502+0000 7fd3d4c00640 1 -- 172.21.15.2:0/1151057554 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e135947830 con 0x55e1359eaaa0 2024-09-10T08:11:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3d4c00640 10 client.16233 handle_command_reply: tid=0 2024-09-10T08:11:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3f4a75740 1 client.16233 shutdown 2024-09-10T08:11:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3f4a75740 1 -- 172.21.15.2:0/1151057554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 msgr2=0x55e1359df3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 0x55e1359df3e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0a41f0 tx=0x7fd3dc0d5390 comp rx=0 tx=0).stop 2024-09-10T08:11:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3f4a75740 1 -- 172.21.15.2:0/1151057554 shutdown_connections 2024-09-10T08:11:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e1359eaaa0 0x55e1359ece90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359dac30 0x55e1359e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5ba0 0x55e1359df920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3d4c00640 0 client.16233 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/1151057554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5260 0x55e1359df3e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3f4a75740 1 -- 172.21.15.2:0/1151057554 >> 172.21.15.2:0/1151057554 conn(0x55e1359d08d0 msgr2=0x55e1359d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3f4a75740 1 -- 172.21.15.2:0/1151057554 shutdown_connections 2024-09-10T08:11:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3f4a75740 1 -- 172.21.15.2:0/1151057554 wait complete. 2024-09-10T08:11:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.503+0000 7fd3f4a75740 20 client.16233 trim_cache size 0 max 16384 2024-09-10T08:11:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.518+0000 7fd3f4a75740 1 Processor -- start 2024-09-10T08:11:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.518+0000 7fd3f4a75740 1 -- start start 2024-09-10T08:11:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.518+0000 7fd3f4a75740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359d5260 0x55e1359df7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.518+0000 7fd3f4a75740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5ba0 0x55e1359e6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.518+0000 7fd3f4a75740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359dac30 0x55e1359cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.518+0000 7fd3f4a75740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec1af140 con 0x55e1359dac30 2024-09-10T08:11:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.518+0000 7fd3f4a75740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec1aca10 con 0x55e1359d5ba0 2024-09-10T08:11:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.518+0000 7fd3f4a75740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec1aa860 con 0x55e1359d5260 2024-09-10T08:11:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.519+0000 7fd3eb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359dac30 0x55e1359cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.519+0000 7fd3eb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359dac30 0x55e1359cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56822/0 (socket says 172.21.15.2:56822) 2024-09-10T08:11:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.519+0000 7fd3eaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359d5260 0x55e1359df7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.519+0000 7fd3eb400640 1 -- 172.21.15.2:0/2956488740 learned_addr learned my addr 172.21.15.2:0/2956488740 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.519+0000 7fd3ea000640 1 --2- 172.21.15.2:0/2956488740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5ba0 0x55e1359e6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.519+0000 7fd3eaa00640 1 -- 172.21.15.2:0/2956488740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5ba0 msgr2=0x55e1359e6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.519+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/2956488740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5ba0 0x55e1359e6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.519+0000 7fd3eaa00640 1 -- 172.21.15.2:0/2956488740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359dac30 msgr2=0x55e1359cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.519+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/2956488740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359dac30 0x55e1359cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.519+0000 7fd3eaa00640 1 -- 172.21.15.2:0/2956488740 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1359ed420 con 0x55e1359d5260 2024-09-10T08:11:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.519+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/2956488740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359d5260 0x55e1359df7e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc098480 tx=0x7fd3dc0983b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3045da6332f27e6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.519+0000 7fd3d7400640 1 -- 172.21.15.2:0/2956488740 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0de070 con 0x55e1359d5260 2024-09-10T08:11:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.520+0000 7fd3d7400640 1 -- 172.21.15.2:0/2956488740 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3dc0d5820 con 0x55e1359d5260 2024-09-10T08:11:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.520+0000 7fd3d7400640 1 -- 172.21.15.2:0/2956488740 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc09b410 con 0x55e1359d5260 2024-09-10T08:11:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.520+0000 7fd3f4a75740 1 -- 172.21.15.2:0/2956488740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359d5260 msgr2=0x55e1359df7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.520+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/2956488740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359d5260 0x55e1359df7e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc098480 tx=0x7fd3dc0983b0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.520+0000 7fd3f4a75740 1 -- 172.21.15.2:0/2956488740 shutdown_connections 2024-09-10T08:11:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.520+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/2956488740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359dac30 0x55e1359cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.520+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/2956488740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5ba0 0x55e1359e6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.520+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/2956488740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359d5260 0x55e1359df7e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.520+0000 7fd3f4a75740 1 -- 172.21.15.2:0/2956488740 >> 172.21.15.2:0/2956488740 conn(0x55e1359d08d0 msgr2=0x55e1359cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.520+0000 7fd3f4a75740 1 -- 172.21.15.2:0/2956488740 shutdown_connections 2024-09-10T08:11:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.520+0000 7fd3f4a75740 1 -- 172.21.15.2:0/2956488740 wait complete. 2024-09-10T08:11:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.521+0000 7fd3f4a75740 1 -- start start 2024-09-10T08:11:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.521+0000 7fd3f4a75740 1 Processor -- start 2024-09-10T08:11:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.521+0000 7fd3f4a75740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5260 0x55e1359e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.521+0000 7fd3f4a75740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5ba0 0x55e1359e3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.521+0000 7fd3f4a75740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359dac30 0x55e135a53610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.521+0000 7fd3f4a75740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec1aaed0 con 0x55e1359d5ba0 2024-09-10T08:11:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.521+0000 7fd3f4a75740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec1ac340 con 0x55e1359d5260 2024-09-10T08:11:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.521+0000 7fd3f4a75740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec1ab5a0 con 0x55e1359dac30 2024-09-10T08:11:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.521+0000 7fd3f4a75740 10 client.0 _ll_register_callbacks cb 0x7fd3e4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.521+0000 7fd3ea000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5ba0 0x55e1359e3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.521+0000 7fd3eb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359dac30 0x55e135a53610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.521+0000 7fd3ea000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5ba0 0x55e1359e3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56830/0 (socket says 172.21.15.2:56830) 2024-09-10T08:11:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.521+0000 7fd3ea000640 1 -- 172.21.15.2:0/2953248096 learned_addr learned my addr 172.21.15.2:0/2953248096 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.522+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/2953248096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5260 0x55e1359e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.522+0000 7fd3ea000640 1 -- 172.21.15.2:0/2953248096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359dac30 msgr2=0x55e135a53610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.522+0000 7fd3ea000640 1 --2- 172.21.15.2:0/2953248096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359dac30 0x55e135a53610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.522+0000 7fd3ea000640 1 -- 172.21.15.2:0/2953248096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5260 msgr2=0x55e1359e38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.522+0000 7fd3ea000640 1 --2- 172.21.15.2:0/2953248096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5260 0x55e1359e38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.522+0000 7fd3ea000640 1 -- 172.21.15.2:0/2953248096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e135a53bd0 con 0x55e1359d5ba0 2024-09-10T08:11:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.522+0000 7fd3ea000640 1 --2- 172.21.15.2:0/2953248096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5ba0 0x55e1359e3e00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd3d802a910 tx=0x7fd3d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e701fe1ecbaa0ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.522+0000 7fd3d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:11:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.522+0000 7fd3d4c00640 1 -- 172.21.15.2:0/2953248096 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d804b060 con 0x55e1359d5ba0 2024-09-10T08:11:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.522+0000 7fd3d4c00640 1 -- 172.21.15.2:0/2953248096 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd3d8004450 con 0x55e1359d5ba0 2024-09-10T08:11:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.522+0000 7fd3f4a75740 1 -- 172.21.15.2:0/2953248096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e135941b80 con 0x55e1359d5ba0 2024-09-10T08:11:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.523+0000 7fd3d4c00640 1 -- 172.21.15.2:0/2953248096 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd3d8052050 con 0x55e1359d5ba0 2024-09-10T08:11:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.523+0000 7fd3f4a75740 10 client.16239 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.523+0000 7fd3f4a75740 1 -- 172.21.15.2:0/2953248096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e135a54fb0 con 0x55e1359d5ba0 2024-09-10T08:11:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.523+0000 7fd3d4c00640 1 -- 172.21.15.2:0/2953248096 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e135941b80 con 0x55e1359d5ba0 2024-09-10T08:11:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.523+0000 7fd3d4c00640 1 -- 172.21.15.2:0/2953248096 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd3d80316b0 con 0x55e1359d5ba0 2024-09-10T08:11:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.523+0000 7fd3f4a75740 10 client.16239 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.523+0000 7fd3f4a75740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.523+0000 7fd3f4a75740 20 client.16239 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.523+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/2953248096 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e135a55940 0x55e135a57d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.523+0000 7fd3f4a75740 4 client.16239 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:11:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.523+0000 7fd3f4a75740 1 -- 172.21.15.2:0/2953248096 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1358be7c0 con 0x55e135a55940 2024-09-10T08:11:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.524+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/2953248096 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e135a55940 0x55e135a57d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.524+0000 7fd3eaa00640 1 --2- 172.21.15.2:0/2953248096 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e135a55940 0x55e135a57d60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb75a3a802c3a7c0 server_cookie=19baf9649dab223e in_seq=0 out_seq=0 2024-09-10T08:11:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.524+0000 7fd3d4c00640 10 client.16239 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.538+0000 7fd3d4c00640 1 -- 172.21.15.2:0/2953248096 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1358be7c0 con 0x55e135a55940 2024-09-10T08:11:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.538+0000 7fd3d4c00640 10 client.16239 handle_command_reply: tid=0 2024-09-10T08:11:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.538+0000 7fd3f4a75740 1 client.16239 shutdown 2024-09-10T08:11:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.538+0000 7fd3f4a75740 1 -- 172.21.15.2:0/2953248096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5ba0 msgr2=0x55e1359e3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.538+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/2953248096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5ba0 0x55e1359e3e00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd3d802a910 tx=0x7fd3d80046c0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.538+0000 7fd3f4a75740 1 -- 172.21.15.2:0/2953248096 shutdown_connections 2024-09-10T08:11:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.538+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/2953248096 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e135a55940 0x55e135a57d60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.538+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/2953248096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e1359dac30 0x55e135a53610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.538+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/2953248096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e1359d5ba0 0x55e1359e3e00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.538+0000 7fd3f4a75740 1 --2- 172.21.15.2:0/2953248096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1359d5260 0x55e1359e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.538+0000 7fd3f4a75740 1 -- 172.21.15.2:0/2953248096 >> 172.21.15.2:0/2953248096 conn(0x55e1359d08d0 msgr2=0x55e1359cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.538+0000 7fd3d4c00640 0 client.16239 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.539+0000 7fd3f4a75740 1 -- 172.21.15.2:0/2953248096 shutdown_connections 2024-09-10T08:11:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.539+0000 7fd3f4a75740 1 -- 172.21.15.2:0/2953248096 wait complete. 2024-09-10T08:11:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.539+0000 7fd3f4a75740 20 client.16239 trim_cache size 0 max 16384 2024-09-10T08:11:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:11:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:11:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:11:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:11:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.540+0000 7fd3ebe00640 1 -- 172.21.15.2:0/265015531 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd3d00a2b80 msgr2=0x7fd3d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.540+0000 7fd3ebe00640 1 --2- 172.21.15.2:0/265015531 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd3d00a2b80 0x7fd3d00a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4011890 tx=0x7fd3e4009500 comp rx=0 tx=0).stop 2024-09-10T08:11:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.540+0000 7fd3ebe00640 1 -- 172.21.15.2:0/265015531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3ec103b70 msgr2=0x7fd3ec199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.540+0000 7fd3ebe00640 1 --2- 172.21.15.2:0/265015531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3ec103b70 0x7fd3ec199e80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00e9c0 tx=0x7fd3dc00ee80 comp rx=0 tx=0).stop 2024-09-10T08:11:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.540+0000 7fd3ebe00640 1 -- 172.21.15.2:0/265015531 shutdown_connections 2024-09-10T08:11:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.540+0000 7fd3ebe00640 1 --2- 172.21.15.2:0/265015531 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd3d00a2b80 0x7fd3d00a5040 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.540+0000 7fd3ebe00640 1 --2- 172.21.15.2:0/265015531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3ec105720 0x7fd3ec1a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.540+0000 7fd3ebe00640 1 --2- 172.21.15.2:0/265015531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3ec104d60 0x7fd3ec19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.540+0000 7fd3ebe00640 1 --2- 172.21.15.2:0/265015531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3ec103b70 0x7fd3ec199e80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.540+0000 7fd3ebe00640 1 -- 172.21.15.2:0/265015531 >> 172.21.15.2:0/265015531 conn(0x7fd3ec0ff310 msgr2=0x7fd3ec108130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.540+0000 7fd3ebe00640 1 -- 172.21.15.2:0/265015531 shutdown_connections 2024-09-10T08:11:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:34.540+0000 7fd3ebe00640 1 -- 172.21.15.2:0/265015531 wait complete. 2024-09-10T08:11:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.651+0000 7faf06400640 1 -- 172.21.15.2:0/1453619083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf001049c0 msgr2=0x7faf00108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.651+0000 7faf06400640 1 --2- 172.21.15.2:0/1453619083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf001049c0 0x7faf00108a10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faee8009900 tx=0x7faee802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.651+0000 7faf06400640 1 -- 172.21.15.2:0/1453619083 shutdown_connections 2024-09-10T08:11:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.651+0000 7faf06400640 1 --2- 172.21.15.2:0/1453619083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf00109140 0x7faf00110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.651+0000 7faf06400640 1 --2- 172.21.15.2:0/1453619083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf001049c0 0x7faf00108a10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.651+0000 7faf06400640 1 --2- 172.21.15.2:0/1453619083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf00104010 0x7faf001043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.651+0000 7faf06400640 1 -- 172.21.15.2:0/1453619083 >> 172.21.15.2:0/1453619083 conn(0x7faf000ffa50 msgr2=0x7faf00101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.651+0000 7faf06400640 1 -- 172.21.15.2:0/1453619083 shutdown_connections 2024-09-10T08:11:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.651+0000 7faf06400640 1 -- 172.21.15.2:0/1453619083 wait complete. 2024-09-10T08:11:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.652+0000 7faf06400640 1 Processor -- start 2024-09-10T08:11:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.653+0000 7faf06400640 1 -- start start 2024-09-10T08:11:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.653+0000 7faf06400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf00104010 0x7faf001a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.653+0000 7faf06400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf001049c0 0x7faf001a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.653+0000 7faf06400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf00109140 0x7faf0019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.653+0000 7faf06400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf001141b0 con 0x7faf00104010 2024-09-10T08:11:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.653+0000 7faf06400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf00114030 con 0x7faf00109140 2024-09-10T08:11:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.653+0000 7faf06400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf00114330 con 0x7faf001049c0 2024-09-10T08:11:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.653+0000 7faeff400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf00104010 0x7faf001a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.653+0000 7faeff400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf00104010 0x7faf001a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56848/0 (socket says 172.21.15.2:56848) 2024-09-10T08:11:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.653+0000 7faeff400640 1 -- 172.21.15.2:0/102763769 learned_addr learned my addr 172.21.15.2:0/102763769 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.654+0000 7faef6a00640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf001049c0 0x7faf001a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.654+0000 7faeffe00640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf00109140 0x7faf0019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.654+0000 7faeff400640 1 -- 172.21.15.2:0/102763769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf001049c0 msgr2=0x7faf001a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.654+0000 7faeff400640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf001049c0 0x7faf001a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.654+0000 7faeff400640 1 -- 172.21.15.2:0/102763769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf00109140 msgr2=0x7faf0019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.654+0000 7faeff400640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf00109140 0x7faf0019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.654+0000 7faeff400640 1 -- 172.21.15.2:0/102763769 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0019acf0 con 0x7faf00104010 2024-09-10T08:11:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.654+0000 7faeff400640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf00104010 0x7faf001a0400 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7faef000e9f0 tx=0x7faef000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.654+0000 7faefcc00640 1 -- 172.21.15.2:0/102763769 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef000cd80 con 0x7faf00104010 2024-09-10T08:11:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.654+0000 7faefcc00640 1 -- 172.21.15.2:0/102763769 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef0010910 con 0x7faf00104010 2024-09-10T08:11:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.654+0000 7faf06400640 1 -- 172.21.15.2:0/102763769 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0019afe0 con 0x7faf00104010 2024-09-10T08:11:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.654+0000 7faefcc00640 1 -- 172.21.15.2:0/102763769 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0018990 con 0x7faf00104010 2024-09-10T08:11:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.655+0000 7faf06400640 1 -- 172.21.15.2:0/102763769 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf001a72d0 con 0x7faf00104010 2024-09-10T08:11:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.657+0000 7faefcc00640 1 -- 172.21.15.2:0/102763769 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7faef0010ac0 con 0x7faf00104010 2024-09-10T08:11:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.658+0000 7faf08feb740 1 Processor -- start 2024-09-10T08:11:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.658+0000 7faf08feb740 1 -- start start 2024-09-10T08:11:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.659+0000 7faf08feb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f7804ac0 0x5604f7804ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.659+0000 7faf08feb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 0x5604f780f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.659+0000 7faf08feb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 0x5604f7812090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.659+0000 7faf08feb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5604f776b290 con 0x5604f7804ac0 2024-09-10T08:11:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.659+0000 7faf08feb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5604f7712b40 con 0x5604f78053e0 2024-09-10T08:11:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.659+0000 7faf08feb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5604f7767250 con 0x5604f780fca0 2024-09-10T08:11:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.659+0000 7faeffe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f7804ac0 0x5604f7804ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.659+0000 7faeff400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 0x5604f780f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.659+0000 7faeffe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f7804ac0 0x5604f7804ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56850/0 (socket says 172.21.15.2:56850) 2024-09-10T08:11:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.659+0000 7faeff400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 0x5604f780f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41402/0 (socket says 172.21.15.2:41402) 2024-09-10T08:11:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.659+0000 7faeffe00640 1 -- 172.21.15.2:0/517937134 learned_addr learned my addr 172.21.15.2:0/517937134 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.659+0000 7faef6a00640 1 --2- 172.21.15.2:0/517937134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 0x5604f7812090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.660+0000 7faeffe00640 1 -- 172.21.15.2:0/517937134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 msgr2=0x5604f7812090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.660+0000 7faeffe00640 1 --2- 172.21.15.2:0/517937134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 0x5604f7812090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.660+0000 7faeffe00640 1 -- 172.21.15.2:0/517937134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 msgr2=0x5604f780f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.660+0000 7faeffe00640 1 --2- 172.21.15.2:0/517937134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 0x5604f780f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.660+0000 7faeffe00640 1 -- 172.21.15.2:0/517937134 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f74dc6a0 con 0x5604f7804ac0 2024-09-10T08:11:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.660+0000 7faeff400640 1 --2- 172.21.15.2:0/517937134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 0x5604f780f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.660+0000 7faefcc00640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faee40a2b80 0x7faee40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.660+0000 7faeffe00640 1 --2- 172.21.15.2:0/517937134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f7804ac0 0x5604f7804ea0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7faef8009f50 tx=0x7faef80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f7920c9d4f0489 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.660+0000 7faefcc00640 1 -- 172.21.15.2:0/102763769 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faef00cf7a0 con 0x7faf00104010 2024-09-10T08:11:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.660+0000 7faeff400640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faee40a2b80 0x7faee40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.661+0000 7faef4c00640 1 -- 172.21.15.2:0/517937134 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8004d20 con 0x5604f7804ac0 2024-09-10T08:11:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.661+0000 7faef4c00640 1 -- 172.21.15.2:0/517937134 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef8004050 con 0x5604f7804ac0 2024-09-10T08:11:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.661+0000 7faef4c00640 1 -- 172.21.15.2:0/517937134 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8005280 con 0x5604f7804ac0 2024-09-10T08:11:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.661+0000 7faf08feb740 1 -- 172.21.15.2:0/517937134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f7804ac0 msgr2=0x5604f7804ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.661+0000 7faeff400640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faee40a2b80 0x7faee40a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faef009b960 tx=0x7faef00d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.661+0000 7faf08feb740 1 --2- 172.21.15.2:0/517937134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f7804ac0 0x5604f7804ea0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7faef8009f50 tx=0x7faef80118d0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.662+0000 7faf08feb740 1 -- 172.21.15.2:0/517937134 shutdown_connections 2024-09-10T08:11:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.662+0000 7faf08feb740 1 --2- 172.21.15.2:0/517937134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 0x5604f7812090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.662+0000 7faf08feb740 1 --2- 172.21.15.2:0/517937134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 0x5604f780f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.662+0000 7faf08feb740 1 --2- 172.21.15.2:0/517937134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f7804ac0 0x5604f7804ea0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.662+0000 7faf08feb740 1 -- 172.21.15.2:0/517937134 >> 172.21.15.2:0/517937134 conn(0x5604f7801050 msgr2=0x5604f7801430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.662+0000 7faf08feb740 1 -- 172.21.15.2:0/517937134 shutdown_connections 2024-09-10T08:11:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.662+0000 7faf08feb740 1 -- 172.21.15.2:0/517937134 wait complete. 2024-09-10T08:11:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.662+0000 7faf08feb740 1 -- start start 2024-09-10T08:11:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.662+0000 7faf08feb740 1 Processor -- start 2024-09-10T08:11:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.663+0000 7faf08feb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f7804ac0 0x5604f780f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.663+0000 7faf08feb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f78053e0 0x5604f780c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.663+0000 7faf08feb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 0x5604f780cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.663+0000 7faf08feb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5604f7812960 con 0x5604f78053e0 2024-09-10T08:11:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.663+0000 7faf08feb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5604f7725fa0 con 0x5604f7804ac0 2024-09-10T08:11:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.663+0000 7faf08feb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5604f7812ae0 con 0x5604f780fca0 2024-09-10T08:11:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.663+0000 7faf08feb740 10 client.0 _ll_register_callbacks cb 0x5604f7479ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.663+0000 7faef6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f78053e0 0x5604f780c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.663+0000 7faef6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f78053e0 0x5604f780c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56858/0 (socket says 172.21.15.2:56858) 2024-09-10T08:11:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.663+0000 7faef6a00640 1 -- 172.21.15.2:0/4228013117 learned_addr learned my addr 172.21.15.2:0/4228013117 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.663+0000 7faeffe00640 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f7804ac0 0x5604f780f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.663+0000 7faeffe00640 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 0x5604f780cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.663+0000 7faef6a00640 1 -- 172.21.15.2:0/4228013117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 msgr2=0x5604f780cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.664+0000 7faef6a00640 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 0x5604f780cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.664+0000 7faef6a00640 1 -- 172.21.15.2:0/4228013117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f7804ac0 msgr2=0x5604f780f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.664+0000 7faef6a00640 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f7804ac0 0x5604f780f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.664+0000 7faef6a00640 1 -- 172.21.15.2:0/4228013117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f780d590 con 0x5604f78053e0 2024-09-10T08:11:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.664+0000 7faeffe00640 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f7804ac0 0x5604f780f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:11:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.664+0000 7faef6a00640 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f78053e0 0x5604f780c820 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faee8009900 tx=0x7faee8004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19857b1d2e960adc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.664+0000 7faeed600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:11:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.664+0000 7faeed600640 1 -- 172.21.15.2:0/4228013117 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8035040 con 0x5604f78053e0 2024-09-10T08:11:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.664+0000 7faeed600640 1 -- 172.21.15.2:0/4228013117 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faee8005450 con 0x5604f78053e0 2024-09-10T08:11:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.664+0000 7faf08feb740 1 -- 172.21.15.2:0/4228013117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faef8004050 con 0x5604f78053e0 2024-09-10T08:11:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.664+0000 7faeed600640 1 -- 172.21.15.2:0/4228013117 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faee8005650 con 0x5604f78053e0 2024-09-10T08:11:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.665+0000 7faf08feb740 10 client.16251 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.665+0000 7faf08feb740 1 -- 172.21.15.2:0/4228013117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5604f781e850 con 0x5604f78053e0 2024-09-10T08:11:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.665+0000 7faeed600640 1 -- 172.21.15.2:0/4228013117 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef8004050 con 0x5604f78053e0 2024-09-10T08:11:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.665+0000 7faeed600640 1 -- 172.21.15.2:0/4228013117 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faee8003a60 con 0x5604f78053e0 2024-09-10T08:11:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.665+0000 7faf08feb740 10 client.16251 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.665+0000 7faf08feb740 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.665+0000 7faf08feb740 20 client.16251 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.665+0000 7faf08feb740 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5604f781fd10 0x5604f7822190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.665+0000 7faf08feb740 4 client.16251 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:11:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.665+0000 7faf08feb740 1 -- 172.21.15.2:0/4228013117 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604f7778800 con 0x5604f781fd10 2024-09-10T08:11:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.666+0000 7faeffe00640 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5604f781fd10 0x5604f7822190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.666+0000 7faeffe00640 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5604f781fd10 0x5604f7822190 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43414e43dbee7598 server_cookie=da0d401e4fa79479 in_seq=0 out_seq=0 2024-09-10T08:11:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.666+0000 7faeed600640 10 client.16251 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faeed600640 1 -- 172.21.15.2:0/4228013117 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604f7778800 con 0x5604f781fd10 2024-09-10T08:11:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faeed600640 10 client.16251 handle_command_reply: tid=0 2024-09-10T08:11:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faf08feb740 1 client.16251 shutdown 2024-09-10T08:11:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faf08feb740 1 -- 172.21.15.2:0/4228013117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f78053e0 msgr2=0x5604f780c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faf08feb740 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f78053e0 0x5604f780c820 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faee8009900 tx=0x7faee8004390 comp rx=0 tx=0).stop 2024-09-10T08:11:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faf08feb740 1 -- 172.21.15.2:0/4228013117 shutdown_connections 2024-09-10T08:11:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faf08feb740 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5604f781fd10 0x5604f7822190 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faf08feb740 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 0x5604f780cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faf08feb740 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f78053e0 0x5604f780c820 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faf08feb740 1 --2- 172.21.15.2:0/4228013117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f7804ac0 0x5604f780f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faeed600640 0 client.16251 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:34.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faf08feb740 1 -- 172.21.15.2:0/4228013117 >> 172.21.15.2:0/4228013117 conn(0x5604f7801050 msgr2=0x5604f78001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:34.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faf08feb740 1 -- 172.21.15.2:0/4228013117 shutdown_connections 2024-09-10T08:11:34.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.667+0000 7faf08feb740 1 -- 172.21.15.2:0/4228013117 wait complete. 2024-09-10T08:11:34.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.668+0000 7faf08feb740 20 client.16251 trim_cache size 0 max 16384 2024-09-10T08:11:34.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.686+0000 7faf08feb740 1 Processor -- start 2024-09-10T08:11:34.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.686+0000 7faf08feb740 1 -- start start 2024-09-10T08:11:34.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.686+0000 7faf08feb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f7804ac0 0x5604f780e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.686+0000 7faf08feb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f78053e0 0x5604f7827a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.687+0000 7faf08feb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 0x5604f7827fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.687+0000 7faf08feb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf001b39b0 con 0x5604f78053e0 2024-09-10T08:11:34.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.687+0000 7faf08feb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf001b1280 con 0x5604f7804ac0 2024-09-10T08:11:34.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.687+0000 7faf08feb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf001af0d0 con 0x5604f780fca0 2024-09-10T08:11:34.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.687+0000 7faef6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f78053e0 0x5604f7827a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.687+0000 7faef6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f78053e0 0x5604f7827a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56866/0 (socket says 172.21.15.2:56866) 2024-09-10T08:11:34.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.687+0000 7faef6a00640 1 -- 172.21.15.2:0/1010969038 learned_addr learned my addr 172.21.15.2:0/1010969038 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:34.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.687+0000 7faeffe00640 1 --2- 172.21.15.2:0/1010969038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f7804ac0 0x5604f780e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.687+0000 7faeffe00640 1 --2- 172.21.15.2:0/1010969038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 0x5604f7827fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.687+0000 7faeffe00640 1 -- 172.21.15.2:0/1010969038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 msgr2=0x5604f7827fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.687+0000 7faeffe00640 1 --2- 172.21.15.2:0/1010969038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 0x5604f7827fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.687+0000 7faeffe00640 1 -- 172.21.15.2:0/1010969038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f78053e0 msgr2=0x5604f7827a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.687+0000 7faeffe00640 1 --2- 172.21.15.2:0/1010969038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f78053e0 0x5604f7827a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.688+0000 7faeffe00640 1 -- 172.21.15.2:0/1010969038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f7802100 con 0x5604f7804ac0 2024-09-10T08:11:34.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.688+0000 7faeffe00640 1 --2- 172.21.15.2:0/1010969038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f7804ac0 0x5604f780e790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5604f7801e10 tx=0x7faef8009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a1e66143b0772e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:34.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.688+0000 7faeefe00640 1 -- 172.21.15.2:0/1010969038 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8003dc0 con 0x5604f7804ac0 2024-09-10T08:11:34.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.688+0000 7faeefe00640 1 -- 172.21.15.2:0/1010969038 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef8004530 con 0x5604f7804ac0 2024-09-10T08:11:34.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.688+0000 7faeefe00640 1 -- 172.21.15.2:0/1010969038 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8028b10 con 0x5604f7804ac0 2024-09-10T08:11:34.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.689+0000 7faf08feb740 1 -- 172.21.15.2:0/1010969038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f7804ac0 msgr2=0x5604f780e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.689+0000 7faf08feb740 1 --2- 172.21.15.2:0/1010969038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f7804ac0 0x5604f780e790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5604f7801e10 tx=0x7faef8009500 comp rx=0 tx=0).stop 2024-09-10T08:11:34.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.689+0000 7faf08feb740 1 -- 172.21.15.2:0/1010969038 shutdown_connections 2024-09-10T08:11:34.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.689+0000 7faf08feb740 1 --2- 172.21.15.2:0/1010969038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f780fca0 0x5604f7827fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.689+0000 7faf08feb740 1 --2- 172.21.15.2:0/1010969038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f78053e0 0x5604f7827a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.689+0000 7faf08feb740 1 --2- 172.21.15.2:0/1010969038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f7804ac0 0x5604f780e790 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.689+0000 7faf08feb740 1 -- 172.21.15.2:0/1010969038 >> 172.21.15.2:0/1010969038 conn(0x5604f7801050 msgr2=0x5604f780e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:34.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.689+0000 7faf08feb740 1 -- 172.21.15.2:0/1010969038 shutdown_connections 2024-09-10T08:11:34.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.689+0000 7faf08feb740 1 -- 172.21.15.2:0/1010969038 wait complete. 2024-09-10T08:11:34.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.690+0000 7faf08feb740 1 -- start start 2024-09-10T08:11:34.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.690+0000 7faf08feb740 1 Processor -- start 2024-09-10T08:11:34.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.690+0000 7faf08feb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f7804ac0 0x5604f7809830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.690+0000 7faf08feb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 0x5604f7809d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.690+0000 7faf08feb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f780fca0 0x5604f780a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.690+0000 7faf08feb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf001af740 con 0x5604f780fca0 2024-09-10T08:11:34.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faf08feb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf001b0bb0 con 0x5604f78053e0 2024-09-10T08:11:34.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faf08feb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf001afe10 con 0x5604f7804ac0 2024-09-10T08:11:34.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faf08feb740 10 client.0 _ll_register_callbacks cb 0x7faee8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:34.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faef6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 0x5604f7809d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faef6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 0x5604f7809d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41418/0 (socket says 172.21.15.2:41418) 2024-09-10T08:11:34.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faef6a00640 1 -- 172.21.15.2:0/281620091 learned_addr learned my addr 172.21.15.2:0/281620091 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:34.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faeffe00640 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f7804ac0 0x5604f7809830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faeffe00640 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f780fca0 0x5604f780a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faef6a00640 1 -- 172.21.15.2:0/281620091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f7804ac0 msgr2=0x5604f7809830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faef6a00640 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f7804ac0 0x5604f7809830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faef6a00640 1 -- 172.21.15.2:0/281620091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f780fca0 msgr2=0x5604f780a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:34.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faef6a00640 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f780fca0 0x5604f780a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faef6a00640 1 -- 172.21.15.2:0/281620091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f7889a30 con 0x5604f78053e0 2024-09-10T08:11:34.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.691+0000 7faeffe00640 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f780fca0 0x5604f780a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:11:34.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.692+0000 7faef6a00640 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 0x5604f7809d70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faee8009900 tx=0x7faee8032c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4b88449490af260 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:34.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.692+0000 7faeed600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.692+0000 7faeed600640 1 -- 172.21.15.2:0/281620091 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8035040 con 0x5604f78053e0 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.692+0000 7faf08feb740 1 -- 172.21.15.2:0/281620091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faef8004530 con 0x5604f78053e0 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.692+0000 7faeed600640 1 -- 172.21.15.2:0/281620091 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faee80037f0 con 0x5604f78053e0 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.692+0000 7faeed600640 1 -- 172.21.15.2:0/281620091 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faee8061050 con 0x5604f78053e0 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.692+0000 7faf08feb740 10 client.26291 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.692+0000 7faf08feb740 1 -- 172.21.15.2:0/281620091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5604f788ad30 con 0x5604f78053e0 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.693+0000 7faeed600640 1 -- 172.21.15.2:0/281620091 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef8004530 con 0x5604f78053e0 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.693+0000 7faeed600640 1 -- 172.21.15.2:0/281620091 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faee805c030 con 0x5604f78053e0 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.693+0000 7faf08feb740 10 client.26291 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.693+0000 7faf08feb740 10 client.26291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.693+0000 7faf08feb740 20 client.26291 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.693+0000 7faf08feb740 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5604f788b4d0 0x5604f788d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.693+0000 7faf08feb740 4 client.26291 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:11:34.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.693+0000 7faf08feb740 1 -- 172.21.15.2:0/281620091 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5604f7772d10 con 0x5604f788b4d0 2024-09-10T08:11:34.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.693+0000 7faeffe00640 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5604f788b4d0 0x5604f788d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:34.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.694+0000 7faeffe00640 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5604f788b4d0 0x5604f788d8f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aa03a63b0337077 server_cookie=dc8b2e42f8eb1b14 in_seq=0 out_seq=0 2024-09-10T08:11:34.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.694+0000 7faeed600640 10 client.26291 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:34.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.695+0000 7faeed600640 1 -- 172.21.15.2:0/281620091 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5604f7772d10 con 0x5604f788b4d0 2024-09-10T08:11:34.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.695+0000 7faeed600640 10 client.26291 handle_command_reply: tid=0 2024-09-10T08:11:34.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.695+0000 7faf08feb740 1 client.26291 shutdown 2024-09-10T08:11:34.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.695+0000 7faf08feb740 1 -- 172.21.15.2:0/281620091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 msgr2=0x5604f7809d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.695+0000 7faf08feb740 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 0x5604f7809d70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faee8009900 tx=0x7faee8032c10 comp rx=0 tx=0).stop 2024-09-10T08:11:34.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.696+0000 7faf08feb740 1 -- 172.21.15.2:0/281620091 shutdown_connections 2024-09-10T08:11:34.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.696+0000 7faf08feb740 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5604f788b4d0 0x5604f788d8f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.696+0000 7faf08feb740 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604f780fca0 0x5604f780a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.696+0000 7faf08feb740 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5604f78053e0 0x5604f7809d70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.696+0000 7faf08feb740 1 --2- 172.21.15.2:0/281620091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604f7804ac0 0x5604f7809830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.696+0000 7faeed600640 0 client.26291 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:34.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.696+0000 7faf08feb740 1 -- 172.21.15.2:0/281620091 >> 172.21.15.2:0/281620091 conn(0x5604f7801050 msgr2=0x5604f780e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:34.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.696+0000 7faf08feb740 1 -- 172.21.15.2:0/281620091 shutdown_connections 2024-09-10T08:11:34.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.696+0000 7faf08feb740 1 -- 172.21.15.2:0/281620091 wait complete. 2024-09-10T08:11:34.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.696+0000 7faf08feb740 20 client.26291 trim_cache size 0 max 16384 2024-09-10T08:11:34.704 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:11:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:11:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "5d4ee328-996c-4774-be09-36894908d1b8", 2024-09-10T08:11:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:11:34.705 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:11:34.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.697+0000 7faf06400640 1 -- 172.21.15.2:0/102763769 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faee40a2b80 msgr2=0x7faee40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.697+0000 7faf06400640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faee40a2b80 0x7faee40a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faef009b960 tx=0x7faef00d5530 comp rx=0 tx=0).stop 2024-09-10T08:11:34.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.697+0000 7faf06400640 1 -- 172.21.15.2:0/102763769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf00104010 msgr2=0x7faf001a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:34.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.697+0000 7faf06400640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf00104010 0x7faf001a0400 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7faef000e9f0 tx=0x7faef000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.697+0000 7faf06400640 1 -- 172.21.15.2:0/102763769 shutdown_connections 2024-09-10T08:11:34.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.697+0000 7faf06400640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf00109140 0x7faf0019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.697+0000 7faf06400640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faee40a2b80 0x7faee40a5040 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.697+0000 7faf06400640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf001049c0 0x7faf001a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.697+0000 7faf06400640 1 --2- 172.21.15.2:0/102763769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf00104010 0x7faf001a0400 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:34.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.697+0000 7faf06400640 1 -- 172.21.15.2:0/102763769 >> 172.21.15.2:0/102763769 conn(0x7faf000ffa50 msgr2=0x7faf00100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:34.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.697+0000 7faf06400640 1 -- 172.21.15.2:0/102763769 shutdown_connections 2024-09-10T08:11:34.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:34.697+0000 7faf06400640 1 -- 172.21.15.2:0/102763769 wait complete. 2024-09-10T08:11:35.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:35 smithi002 ceph-mon[30563]: pgmap v490: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:11:35.582 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:11:35.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:35 smithi155 ceph-mon[35405]: pgmap v490: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:11:35.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:35 smithi107 ceph-mon[35048]: pgmap v490: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:11:35.990 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:11:36.450 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:36 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:11:36.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:36 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:11:36.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:36 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:11:36.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:36 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:11:36.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:36 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:11:36.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:36 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:11:36.452 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:36 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:11:36.452 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:36 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:11:36.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.566+0000 7fbe92e00640 1 -- 172.21.15.2:0/1142922175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe8c102790 msgr2=0x7fbe8c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:36.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.566+0000 7fbe92e00640 1 --2- 172.21.15.2:0/1142922175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe8c102790 0x7fbe8c1067e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009900 tx=0x7fbe7402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.566+0000 7fbe92e00640 1 -- 172.21.15.2:0/1142922175 shutdown_connections 2024-09-10T08:11:36.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.566+0000 7fbe92e00640 1 --2- 172.21.15.2:0/1142922175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8c106f10 0x7fbe8c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.566+0000 7fbe92e00640 1 --2- 172.21.15.2:0/1142922175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe8c102790 0x7fbe8c1067e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.566+0000 7fbe92e00640 1 --2- 172.21.15.2:0/1142922175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe8c101de0 0x7fbe8c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.566+0000 7fbe92e00640 1 -- 172.21.15.2:0/1142922175 >> 172.21.15.2:0/1142922175 conn(0x7fbe8c0fd8c0 msgr2=0x7fbe8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.566+0000 7fbe92e00640 1 -- 172.21.15.2:0/1142922175 shutdown_connections 2024-09-10T08:11:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.566+0000 7fbe92e00640 1 -- 172.21.15.2:0/1142922175 wait complete. 2024-09-10T08:11:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.567+0000 7fbe92e00640 1 Processor -- start 2024-09-10T08:11:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.567+0000 7fbe92e00640 1 -- start start 2024-09-10T08:11:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.568+0000 7fbe92e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8c101de0 0x7fbe8c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.568+0000 7fbe92e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe8c102790 0x7fbe8c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.568+0000 7fbe92e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe8c106f10 0x7fbe8c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.568+0000 7fbe92e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c111f70 con 0x7fbe8c106f10 2024-09-10T08:11:36.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.568+0000 7fbe92e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c111df0 con 0x7fbe8c101de0 2024-09-10T08:11:36.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.568+0000 7fbe92e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1120f0 con 0x7fbe8c102790 2024-09-10T08:11:36.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.568+0000 7fbe90e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe8c106f10 0x7fbe8c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.568+0000 7fbe8be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8c101de0 0x7fbe8c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.568+0000 7fbe90e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe8c106f10 0x7fbe8c19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56898/0 (socket says 172.21.15.2:56898) 2024-09-10T08:11:36.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.568+0000 7fbe90e00640 1 -- 172.21.15.2:0/2129899025 learned_addr learned my addr 172.21.15.2:0/2129899025 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:36.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.568+0000 7fbe83400640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe8c102790 0x7fbe8c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.569+0000 7fbe8be00640 1 -- 172.21.15.2:0/2129899025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe8c102790 msgr2=0x7fbe8c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:36.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.569+0000 7fbe8be00640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe8c102790 0x7fbe8c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.569+0000 7fbe8be00640 1 -- 172.21.15.2:0/2129899025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe8c106f10 msgr2=0x7fbe8c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:36.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.569+0000 7fbe8be00640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe8c106f10 0x7fbe8c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.569+0000 7fbe8be00640 1 -- 172.21.15.2:0/2129899025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c19d3e0 con 0x7fbe8c101de0 2024-09-10T08:11:36.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.569+0000 7fbe90e00640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe8c106f10 0x7fbe8c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:36.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.569+0000 7fbe8be00640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8c101de0 0x7fbe8c198430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00e9f0 tx=0x7fbe7c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:36.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.569+0000 7fbe89600640 1 -- 172.21.15.2:0/2129899025 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c00ccc0 con 0x7fbe8c101de0 2024-09-10T08:11:36.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.569+0000 7fbe89600640 1 -- 172.21.15.2:0/2129899025 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe7c005020 con 0x7fbe8c101de0 2024-09-10T08:11:36.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.569+0000 7fbe89600640 1 -- 172.21.15.2:0/2129899025 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c0049e0 con 0x7fbe8c101de0 2024-09-10T08:11:36.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.569+0000 7fbe92e00640 1 -- 172.21.15.2:0/2129899025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8c19d6d0 con 0x7fbe8c101de0 2024-09-10T08:11:36.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.569+0000 7fbe92e00640 1 -- 172.21.15.2:0/2129899025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8c1a4fb0 con 0x7fbe8c101de0 2024-09-10T08:11:36.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.572+0000 7fbe89600640 1 -- 172.21.15.2:0/2129899025 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbe7c005560 con 0x7fbe8c101de0 2024-09-10T08:11:36.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.572+0000 7fbe95894740 1 Processor -- start 2024-09-10T08:11:36.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.573+0000 7fbe95894740 1 -- start start 2024-09-10T08:11:36.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.573+0000 7fbe95894740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e24e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.573+0000 7fbe95894740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e33c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.573+0000 7fbe95894740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 0x55f3a9e36570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.573+0000 7fbe95894740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f3a9d8b3e0 con 0x55f3a9e24a90 2024-09-10T08:11:36.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.573+0000 7fbe95894740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f3a9d32d80 con 0x55f3a9e34180 2024-09-10T08:11:36.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.573+0000 7fbe95894740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f3a9d873a0 con 0x55f3a9e253b0 2024-09-10T08:11:36.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.573+0000 7fbe90e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e24e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.573+0000 7fbe90e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e24e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56902/0 (socket says 172.21.15.2:56902) 2024-09-10T08:11:36.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.573+0000 7fbe90e00640 1 -- 172.21.15.2:0/1430719528 learned_addr learned my addr 172.21.15.2:0/1430719528 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:36.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.573+0000 7fbe83400640 1 --2- 172.21.15.2:0/1430719528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 0x55f3a9e36570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.573+0000 7fbe8be00640 1 --2- 172.21.15.2:0/1430719528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e33c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.574+0000 7fbe90e00640 1 -- 172.21.15.2:0/1430719528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 msgr2=0x55f3a9e33c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:36.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.574+0000 7fbe90e00640 1 --2- 172.21.15.2:0/1430719528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e33c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.574+0000 7fbe90e00640 1 -- 172.21.15.2:0/1430719528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 msgr2=0x55f3a9e36570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:36.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.574+0000 7fbe90e00640 1 --2- 172.21.15.2:0/1430719528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 0x55f3a9e36570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.574+0000 7fbe90e00640 1 -- 172.21.15.2:0/1430719528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a9d51ae0 con 0x55f3a9e24a90 2024-09-10T08:11:36.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.574+0000 7fbe90e00640 1 --2- 172.21.15.2:0/1430719528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e24e70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbe84011850 tx=0x7fbe84011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a59b903879da8c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:36.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.574+0000 7fbe89600640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbe700a2c00 0x7fbe700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.574+0000 7fbe81600640 1 -- 172.21.15.2:0/1430719528 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84011330 con 0x55f3a9e24a90 2024-09-10T08:11:36.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.574+0000 7fbe81600640 1 -- 172.21.15.2:0/1430719528 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8400fe80 con 0x55f3a9e24a90 2024-09-10T08:11:36.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.574+0000 7fbe81600640 1 -- 172.21.15.2:0/1430719528 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe840193e0 con 0x55f3a9e24a90 2024-09-10T08:11:36.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.574+0000 7fbe89600640 1 -- 172.21.15.2:0/2129899025 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbe7c0cf3a0 con 0x7fbe8c101de0 2024-09-10T08:11:36.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.575+0000 7fbe8be00640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbe700a2c00 0x7fbe700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.575+0000 7fbe8be00640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbe700a2c00 0x7fbe700a50c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c097c10 tx=0x7fbe7c0096e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:36.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.575+0000 7fbe95894740 1 -- 172.21.15.2:0/1430719528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 msgr2=0x55f3a9e24e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:36.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.575+0000 7fbe95894740 1 --2- 172.21.15.2:0/1430719528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e24e70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbe84011850 tx=0x7fbe84011d10 comp rx=0 tx=0).stop 2024-09-10T08:11:36.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.575+0000 7fbe95894740 1 -- 172.21.15.2:0/1430719528 shutdown_connections 2024-09-10T08:11:36.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.575+0000 7fbe95894740 1 --2- 172.21.15.2:0/1430719528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 0x55f3a9e36570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.575+0000 7fbe95894740 1 --2- 172.21.15.2:0/1430719528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e33c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.575+0000 7fbe95894740 1 --2- 172.21.15.2:0/1430719528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e24e70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.575+0000 7fbe95894740 1 -- 172.21.15.2:0/1430719528 >> 172.21.15.2:0/1430719528 conn(0x55f3a9e21120 msgr2=0x55f3a9e21500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:36.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.576+0000 7fbe95894740 1 -- 172.21.15.2:0/1430719528 shutdown_connections 2024-09-10T08:11:36.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.576+0000 7fbe95894740 1 -- 172.21.15.2:0/1430719528 wait complete. 2024-09-10T08:11:36.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.576+0000 7fbe95894740 1 -- start start 2024-09-10T08:11:36.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.576+0000 7fbe95894740 1 Processor -- start 2024-09-10T08:11:36.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.577+0000 7fbe95894740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e30f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.577+0000 7fbe95894740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.577+0000 7fbe95894740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 0x55f3a9e319e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.577+0000 7fbe95894740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f3a9e36ea0 con 0x55f3a9e24a90 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.577+0000 7fbe95894740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f3a9d460f0 con 0x55f3a9e34180 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.577+0000 7fbe95894740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f3a9e37020 con 0x55f3a9e253b0 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.577+0000 7fbe95894740 10 client.0 _ll_register_callbacks cb 0x55f3a9d379e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.577+0000 7fbe90e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e30f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.577+0000 7fbe90e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e30f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56912/0 (socket says 172.21.15.2:56912) 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.577+0000 7fbe90e00640 1 -- 172.21.15.2:0/3513129940 learned_addr learned my addr 172.21.15.2:0/3513129940 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.577+0000 7fbe83400640 1 --2- 172.21.15.2:0/3513129940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.577+0000 7fbe90e00640 1 --2- 172.21.15.2:0/3513129940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 0x55f3a9e319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.578+0000 7fbe83400640 1 -- 172.21.15.2:0/3513129940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 msgr2=0x55f3a9e319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.578+0000 7fbe83400640 1 --2- 172.21.15.2:0/3513129940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 0x55f3a9e319e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.578+0000 7fbe83400640 1 -- 172.21.15.2:0/3513129940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 msgr2=0x55f3a9e30f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.578+0000 7fbe83400640 1 --2- 172.21.15.2:0/3513129940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e30f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.578+0000 7fbe83400640 1 -- 172.21.15.2:0/3513129940 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a9afc3c0 con 0x55f3a9e253b0 2024-09-10T08:11:36.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.578+0000 7fbe83400640 1 --2- 172.21.15.2:0/3513129940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e314a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbe74005940 tx=0x7fbe740317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25e144be394bc313 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:36.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.578+0000 7fbe7a000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:11:36.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.578+0000 7fbe7a000640 1 -- 172.21.15.2:0/3513129940 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe74035040 con 0x55f3a9e253b0 2024-09-10T08:11:36.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.579+0000 7fbe95894740 1 -- 172.21.15.2:0/3513129940 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f3a9d98950 con 0x55f3a9e253b0 2024-09-10T08:11:36.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.579+0000 7fbe7a000640 1 -- 172.21.15.2:0/3513129940 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbe74004040 con 0x55f3a9e253b0 2024-09-10T08:11:36.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.579+0000 7fbe7a000640 1 -- 172.21.15.2:0/3513129940 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe7404f050 con 0x55f3a9e253b0 2024-09-10T08:11:36.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.579+0000 7fbe95894740 10 client.26269 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:36.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.579+0000 7fbe95894740 1 -- 172.21.15.2:0/3513129940 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3a9e42ae0 con 0x55f3a9e253b0 2024-09-10T08:11:36.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.579+0000 7fbe7a000640 1 -- 172.21.15.2:0/3513129940 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f3a9d98950 con 0x55f3a9e253b0 2024-09-10T08:11:36.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.580+0000 7fbe7a000640 1 -- 172.21.15.2:0/3513129940 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbe74004930 con 0x55f3a9e253b0 2024-09-10T08:11:36.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.580+0000 7fbe95894740 10 client.26269 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:36.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.580+0000 7fbe95894740 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:36.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.580+0000 7fbe95894740 20 client.26269 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:36.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.580+0000 7fbe95894740 1 --2- 172.21.15.2:0/3513129940 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f3a9e43e90 0x55f3a9e462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.580+0000 7fbe95894740 4 client.26269 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:11:36.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.580+0000 7fbe95894740 1 -- 172.21.15.2:0/3513129940 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3a9d92e60 con 0x55f3a9e43e90 2024-09-10T08:11:36.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.580+0000 7fbe90e00640 1 --2- 172.21.15.2:0/3513129940 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f3a9e43e90 0x55f3a9e462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.580+0000 7fbe90e00640 1 --2- 172.21.15.2:0/3513129940 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f3a9e43e90 0x55f3a9e462c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=950bdeeda8845d94 server_cookie=ae0a9298b815294b in_seq=0 out_seq=0 2024-09-10T08:11:36.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.580+0000 7fbe7a000640 10 client.26269 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:36.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.581+0000 7fbe7a000640 1 -- 172.21.15.2:0/3513129940 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3a9d92e60 con 0x55f3a9e43e90 2024-09-10T08:11:36.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.581+0000 7fbe7a000640 10 client.26269 handle_command_reply: tid=0 2024-09-10T08:11:36.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.581+0000 7fbe95894740 1 client.26269 shutdown 2024-09-10T08:11:36.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.581+0000 7fbe95894740 1 -- 172.21.15.2:0/3513129940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 msgr2=0x55f3a9e314a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:36.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.581+0000 7fbe95894740 1 --2- 172.21.15.2:0/3513129940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e314a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbe74005940 tx=0x7fbe740317e0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.581+0000 7fbe95894740 1 -- 172.21.15.2:0/3513129940 shutdown_connections 2024-09-10T08:11:36.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.581+0000 7fbe95894740 1 --2- 172.21.15.2:0/3513129940 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f3a9e43e90 0x55f3a9e462c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.581+0000 7fbe95894740 1 --2- 172.21.15.2:0/3513129940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 0x55f3a9e319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.581+0000 7fbe7a000640 0 client.26269 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.581+0000 7fbe95894740 1 --2- 172.21.15.2:0/3513129940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e314a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.581+0000 7fbe95894740 1 --2- 172.21.15.2:0/3513129940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e30f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.582+0000 7fbe95894740 1 -- 172.21.15.2:0/3513129940 >> 172.21.15.2:0/3513129940 conn(0x55f3a9e21120 msgr2=0x55f3a9e26de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.582+0000 7fbe95894740 1 -- 172.21.15.2:0/3513129940 shutdown_connections 2024-09-10T08:11:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.582+0000 7fbe95894740 1 -- 172.21.15.2:0/3513129940 wait complete. 2024-09-10T08:11:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.582+0000 7fbe95894740 20 client.26269 trim_cache size 0 max 16384 2024-09-10T08:11:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.600+0000 7fbe95894740 1 Processor -- start 2024-09-10T08:11:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.600+0000 7fbe95894740 1 -- start start 2024-09-10T08:11:36.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.600+0000 7fbe95894740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e4bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.600+0000 7fbe95894740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e253b0 0x55f3a9e4c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.600+0000 7fbe95894740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e34180 0x55f3a9e2e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.600+0000 7fbe95894740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1af4b0 con 0x55f3a9e24a90 2024-09-10T08:11:36.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.600+0000 7fbe95894740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1acd80 con 0x55f3a9e253b0 2024-09-10T08:11:36.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.600+0000 7fbe95894740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1aabd0 con 0x55f3a9e34180 2024-09-10T08:11:36.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.601+0000 7fbe90e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e4bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.601+0000 7fbe90e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e4bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56922/0 (socket says 172.21.15.2:56922) 2024-09-10T08:11:36.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.601+0000 7fbe90e00640 1 -- 172.21.15.2:0/390573963 learned_addr learned my addr 172.21.15.2:0/390573963 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:36.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.601+0000 7fbe83400640 1 --2- 172.21.15.2:0/390573963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e253b0 0x55f3a9e4c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.601+0000 7fbe90e00640 1 --2- 172.21.15.2:0/390573963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e34180 0x55f3a9e2e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.601+0000 7fbe90e00640 1 -- 172.21.15.2:0/390573963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e34180 msgr2=0x55f3a9e2e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:36.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.601+0000 7fbe90e00640 1 --2- 172.21.15.2:0/390573963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e34180 0x55f3a9e2e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.601+0000 7fbe90e00640 1 -- 172.21.15.2:0/390573963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e253b0 msgr2=0x55f3a9e4c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:36.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.601+0000 7fbe90e00640 1 --2- 172.21.15.2:0/390573963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e253b0 0x55f3a9e4c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.601+0000 7fbe90e00640 1 -- 172.21.15.2:0/390573963 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a9e251e0 con 0x55f3a9e24a90 2024-09-10T08:11:36.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.601+0000 7fbe83400640 1 --2- 172.21.15.2:0/390573963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e253b0 0x55f3a9e4c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:11:36.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.601+0000 7fbe90e00640 1 --2- 172.21.15.2:0/390573963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e4bb90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbe84011850 tx=0x7fbe84011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c367fa56312bb7c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:36.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.602+0000 7fbe80c00640 1 -- 172.21.15.2:0/390573963 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84031030 con 0x55f3a9e24a90 2024-09-10T08:11:36.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.602+0000 7fbe80c00640 1 -- 172.21.15.2:0/390573963 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8400fae0 con 0x55f3a9e24a90 2024-09-10T08:11:36.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.602+0000 7fbe80c00640 1 -- 172.21.15.2:0/390573963 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8401f040 con 0x55f3a9e24a90 2024-09-10T08:11:36.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.602+0000 7fbe95894740 1 -- 172.21.15.2:0/390573963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 msgr2=0x55f3a9e4bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:36.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.602+0000 7fbe95894740 1 --2- 172.21.15.2:0/390573963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e4bb90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbe84011850 tx=0x7fbe84011620 comp rx=0 tx=0).stop 2024-09-10T08:11:36.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.602+0000 7fbe95894740 1 -- 172.21.15.2:0/390573963 shutdown_connections 2024-09-10T08:11:36.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.602+0000 7fbe95894740 1 --2- 172.21.15.2:0/390573963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e34180 0x55f3a9e2e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.602+0000 7fbe95894740 1 --2- 172.21.15.2:0/390573963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e253b0 0x55f3a9e4c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.602+0000 7fbe95894740 1 --2- 172.21.15.2:0/390573963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e4bb90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.602+0000 7fbe95894740 1 -- 172.21.15.2:0/390573963 >> 172.21.15.2:0/390573963 conn(0x55f3a9e21120 msgr2=0x55f3a9e32fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:36.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.603+0000 7fbe95894740 1 -- 172.21.15.2:0/390573963 shutdown_connections 2024-09-10T08:11:36.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.603+0000 7fbe95894740 1 -- 172.21.15.2:0/390573963 wait complete. 2024-09-10T08:11:36.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.603+0000 7fbe95894740 1 -- start start 2024-09-10T08:11:36.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.603+0000 7fbe95894740 1 Processor -- start 2024-09-10T08:11:36.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe95894740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e3c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe95894740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e3c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe95894740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 0x55f3a9e37270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe95894740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1ab240 con 0x55f3a9e24a90 2024-09-10T08:11:36.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe95894740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1ac6b0 con 0x55f3a9e34180 2024-09-10T08:11:36.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe95894740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1ab910 con 0x55f3a9e253b0 2024-09-10T08:11:36.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe95894740 10 client.0 _ll_register_callbacks cb 0x7fbe74009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:36.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe90e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e3c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe90e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e3c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56938/0 (socket says 172.21.15.2:56938) 2024-09-10T08:11:36.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe90e00640 1 -- 172.21.15.2:0/806550572 learned_addr learned my addr 172.21.15.2:0/806550572 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:36.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe83400640 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e3c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe90e00640 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 0x55f3a9e37270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe83400640 1 -- 172.21.15.2:0/806550572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 msgr2=0x55f3a9e37270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:36.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe83400640 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 0x55f3a9e37270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe83400640 1 -- 172.21.15.2:0/806550572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 msgr2=0x55f3a9e3c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:36.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.604+0000 7fbe83400640 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e3c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.605+0000 7fbe83400640 1 -- 172.21.15.2:0/806550572 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a9e37aa0 con 0x55f3a9e253b0 2024-09-10T08:11:36.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.605+0000 7fbe90e00640 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e3c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:36.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.605+0000 7fbe83400640 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e3c9e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbe74004ef0 tx=0x7fbe74005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2098a14186a3c62b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:36.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.605+0000 7fbe7a000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:11:36.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.605+0000 7fbe7a000640 1 -- 172.21.15.2:0/806550572 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe74057070 con 0x55f3a9e253b0 2024-09-10T08:11:36.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.605+0000 7fbe95894740 1 -- 172.21.15.2:0/806550572 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe8400fae0 con 0x55f3a9e253b0 2024-09-10T08:11:36.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.605+0000 7fbe7a000640 1 -- 172.21.15.2:0/806550572 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbe740048c0 con 0x55f3a9e253b0 2024-09-10T08:11:36.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.605+0000 7fbe7a000640 1 -- 172.21.15.2:0/806550572 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe74063050 con 0x55f3a9e253b0 2024-09-10T08:11:36.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.605+0000 7fbe95894740 10 client.26275 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:36.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.605+0000 7fbe95894740 1 -- 172.21.15.2:0/806550572 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3a9eaf650 con 0x55f3a9e253b0 2024-09-10T08:11:36.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.606+0000 7fbe7a000640 1 -- 172.21.15.2:0/806550572 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8400fae0 con 0x55f3a9e253b0 2024-09-10T08:11:36.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.606+0000 7fbe7a000640 1 -- 172.21.15.2:0/806550572 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbe740465a0 con 0x55f3a9e253b0 2024-09-10T08:11:36.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.606+0000 7fbe95894740 10 client.26275 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:36.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.606+0000 7fbe95894740 10 client.26275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:36.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.606+0000 7fbe95894740 20 client.26275 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:36.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.606+0000 7fbe95894740 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f3a9eafda0 0x55f3a9eb21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:36.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.606+0000 7fbe95894740 4 client.26275 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:11:36.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.606+0000 7fbe95894740 1 -- 172.21.15.2:0/806550572 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3a9d0f0c0 con 0x55f3a9eafda0 2024-09-10T08:11:36.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.606+0000 7fbe90e00640 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f3a9eafda0 0x55f3a9eb21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:36.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe90e00640 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f3a9eafda0 0x55f3a9eb21c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71584ab6f1c2fa68 server_cookie=e2d898bae8c7503b in_seq=0 out_seq=0 2024-09-10T08:11:36.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe7a000640 10 client.26275 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:36.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe7a000640 1 -- 172.21.15.2:0/806550572 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f3a9d0f0c0 con 0x55f3a9eafda0 2024-09-10T08:11:36.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe7a000640 10 client.26275 handle_command_reply: tid=0 2024-09-10T08:11:36.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe95894740 1 client.26275 shutdown 2024-09-10T08:11:36.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe95894740 1 -- 172.21.15.2:0/806550572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 msgr2=0x55f3a9e3c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:36.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe95894740 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e3c9e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbe74004ef0 tx=0x7fbe74005e70 comp rx=0 tx=0).stop 2024-09-10T08:11:36.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe95894740 1 -- 172.21.15.2:0/806550572 shutdown_connections 2024-09-10T08:11:36.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe95894740 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f3a9eafda0 0x55f3a9eb21c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe95894740 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3a9e34180 0x55f3a9e37270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe95894740 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a9e253b0 0x55f3a9e3c9e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe95894740 1 --2- 172.21.15.2:0/806550572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3a9e24a90 0x55f3a9e3c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe7a000640 0 client.26275 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:36.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe95894740 1 -- 172.21.15.2:0/806550572 >> 172.21.15.2:0/806550572 conn(0x55f3a9e21120 msgr2=0x55f3a9e265c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:36.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe95894740 1 -- 172.21.15.2:0/806550572 shutdown_connections 2024-09-10T08:11:36.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.607+0000 7fbe95894740 1 -- 172.21.15.2:0/806550572 wait complete. 2024-09-10T08:11:36.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.608+0000 7fbe95894740 20 client.26275 trim_cache size 0 max 16384 2024-09-10T08:11:36.625 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:11:36.625 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:11:36.625 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:11:36.625 INFO:teuthology.orchestra.run.smithi002.stdout: "5d4ee328-996c-4774-be09-36894908d1b8": { 2024-09-10T08:11:36.626 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:11:36.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "5d4ee328-996c-4774-be09-36894908d1b8", 2024-09-10T08:11:36.626 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:11:36.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:11:36.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:11:36.626 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:11:36.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.608+0000 7fbe92e00640 1 -- 172.21.15.2:0/2129899025 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbe700a2c00 msgr2=0x7fbe700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:36.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.608+0000 7fbe92e00640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbe700a2c00 0x7fbe700a50c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c097c10 tx=0x7fbe7c0096e0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.608+0000 7fbe92e00640 1 -- 172.21.15.2:0/2129899025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8c101de0 msgr2=0x7fbe8c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:36.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.608+0000 7fbe92e00640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8c101de0 0x7fbe8c198430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00e9f0 tx=0x7fbe7c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.608+0000 7fbe92e00640 1 -- 172.21.15.2:0/2129899025 shutdown_connections 2024-09-10T08:11:36.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.608+0000 7fbe92e00640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe8c106f10 0x7fbe8c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.608+0000 7fbe92e00640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbe700a2c00 0x7fbe700a50c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.608+0000 7fbe92e00640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe8c102790 0x7fbe8c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.608+0000 7fbe92e00640 1 --2- 172.21.15.2:0/2129899025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8c101de0 0x7fbe8c198430 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:36.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.608+0000 7fbe92e00640 1 -- 172.21.15.2:0/2129899025 >> 172.21.15.2:0/2129899025 conn(0x7fbe8c0fd8c0 msgr2=0x7fbe8c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:36.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.609+0000 7fbe92e00640 1 -- 172.21.15.2:0/2129899025 shutdown_connections 2024-09-10T08:11:36.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:11:36.609+0000 7fbe92e00640 1 -- 172.21.15.2:0/2129899025 wait complete. 2024-09-10T08:11:36.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:36 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:11:36.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:36 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:11:36.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:36 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:11:36.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:36 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:11:36.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:36 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:11:36.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:36 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:11:36.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:36 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:11:36.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:36 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:11:36.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:36 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:11:36.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:36 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:11:36.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:36 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:11:36.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:36 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:11:36.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:36 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:11:36.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:36 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:11:36.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:36 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:11:36.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:36 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:11:37.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:37 smithi002 ceph-mon[30563]: pgmap v491: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-10T08:11:37.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:37 smithi155 ceph-mon[35405]: pgmap v491: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-10T08:11:37.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:37 smithi107 ceph-mon[35048]: pgmap v491: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-10T08:11:37.752 INFO:tasks.cephfs.filesystem:scrub status for tag:5d4ee328-996c-4774-be09-36894908d1b8 - {'path': '/', 'tag': '5d4ee328-996c-4774-be09-36894908d1b8', 'options': 'recursive,force'} 2024-09-10T08:11:39.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:39 smithi155 ceph-mon[35405]: pgmap v492: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:11:39.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:39 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:11:39.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:39 smithi107 ceph-mon[35048]: pgmap v492: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:11:39.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:39 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:11:39.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:39 smithi002 ceph-mon[30563]: pgmap v492: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:11:39.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:39 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:11:41.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:41 smithi155 ceph-mon[35405]: pgmap v493: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:11:41.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:41 smithi107 ceph-mon[35048]: pgmap v493: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:11:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:41 smithi002 ceph-mon[30563]: pgmap v493: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:11:43.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:43 smithi155 ceph-mon[35405]: pgmap v494: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:11:43.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:43 smithi107 ceph-mon[35048]: pgmap v494: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:11:43.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:43 smithi002 ceph-mon[30563]: pgmap v494: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:11:45.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:45 smithi155 ceph-mon[35405]: pgmap v495: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:11:45.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:45 smithi107 ceph-mon[35048]: pgmap v495: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:11:45.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:45 smithi002 ceph-mon[30563]: pgmap v495: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:11:47.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:47 smithi155 ceph-mon[35405]: pgmap v496: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.666+0000 7f3444c00640 1 -- 172.21.15.2:0/2591759387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3440103050 msgr2=0x7f3440103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.666+0000 7f3444c00640 1 --2- 172.21.15.2:0/2591759387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3440103050 0x7f3440103450 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3430009900 tx=0x7f343002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.666+0000 7f3444c00640 1 -- 172.21.15.2:0/2591759387 shutdown_connections 2024-09-10T08:11:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.666+0000 7f3444c00640 1 --2- 172.21.15.2:0/2591759387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3440101f40 0x7f3440109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.666+0000 7f3444c00640 1 --2- 172.21.15.2:0/2591759387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3440101580 0x7f3440101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.666+0000 7f3444c00640 1 --2- 172.21.15.2:0/2591759387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3440103050 0x7f3440103450 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.666+0000 7f3444c00640 1 -- 172.21.15.2:0/2591759387 >> 172.21.15.2:0/2591759387 conn(0x7f34400fd0d0 msgr2=0x7f34400ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.666+0000 7f3444c00640 1 -- 172.21.15.2:0/2591759387 shutdown_connections 2024-09-10T08:11:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.666+0000 7f3444c00640 1 -- 172.21.15.2:0/2591759387 wait complete. 2024-09-10T08:11:47.668 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:47 smithi107 ceph-mon[35048]: pgmap v496: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:11:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.667+0000 7f3444c00640 1 Processor -- start 2024-09-10T08:11:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.668+0000 7f3444c00640 1 -- start start 2024-09-10T08:11:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.668+0000 7f3444c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3440101580 0x7f344019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.668+0000 7f3444c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3440101f40 0x7f344019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.668+0000 7f3444c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3440103050 0x7f34401a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.668+0000 7f3444c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f344010d740 con 0x7f3440101580 2024-09-10T08:11:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.668+0000 7f3444c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f344010d5c0 con 0x7f3440103050 2024-09-10T08:11:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.668+0000 7f3444c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f344010d8c0 con 0x7f3440101f40 2024-09-10T08:11:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.668+0000 7f343f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3440101580 0x7f344019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.668+0000 7f343f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3440101580 0x7f344019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39980/0 (socket says 172.21.15.2:39980) 2024-09-10T08:11:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.668+0000 7f343f400640 1 -- 172.21.15.2:0/284552073 learned_addr learned my addr 172.21.15.2:0/284552073 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.668+0000 7f343fe00640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3440103050 0x7f34401a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.668+0000 7f3437e00640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3440101f40 0x7f344019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.669+0000 7f343f400640 1 -- 172.21.15.2:0/284552073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3440101f40 msgr2=0x7f344019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.669+0000 7f343f400640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3440101f40 0x7f344019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.669+0000 7f343f400640 1 -- 172.21.15.2:0/284552073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3440103050 msgr2=0x7f34401a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.669+0000 7f343f400640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3440103050 0x7f34401a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.669+0000 7f343f400640 1 -- 172.21.15.2:0/284552073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34401a1c60 con 0x7f3440101580 2024-09-10T08:11:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.669+0000 7f343f400640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3440101580 0x7f344019a040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3430031440 tx=0x7f3430032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.669+0000 7f343cc00640 1 -- 172.21.15.2:0/284552073 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343003f030 con 0x7f3440101580 2024-09-10T08:11:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.669+0000 7f343cc00640 1 -- 172.21.15.2:0/284552073 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3430003ce0 con 0x7f3440101580 2024-09-10T08:11:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.669+0000 7f343cc00640 1 -- 172.21.15.2:0/284552073 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430035070 con 0x7f3440101580 2024-09-10T08:11:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.669+0000 7f3444c00640 1 -- 172.21.15.2:0/284552073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34401a1ef0 con 0x7f3440101580 2024-09-10T08:11:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.670+0000 7f3444c00640 1 -- 172.21.15.2:0/284552073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34401a2300 con 0x7f3440101580 2024-09-10T08:11:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.672+0000 7f343cc00640 1 -- 172.21.15.2:0/284552073 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3430042030 con 0x7f3440101580 2024-09-10T08:11:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.672+0000 7f34491eb740 1 Processor -- start 2024-09-10T08:11:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.673+0000 7f34491eb740 1 -- start start 2024-09-10T08:11:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.673+0000 7f34491eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d931e260 0x5643d931e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.673+0000 7f34491eb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931eba0 0x5643d9323660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.673+0000 7f34491eb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d9323c30 0x5643d93260f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.673+0000 7f34491eb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5643d9283100 con 0x5643d9323c30 2024-09-10T08:11:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.673+0000 7f34491eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5643d922b1b0 con 0x5643d931e260 2024-09-10T08:11:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.673+0000 7f34491eb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5643d927f0c0 con 0x5643d931eba0 2024-09-10T08:11:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.673+0000 7f3437e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d9323c30 0x5643d93260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.673+0000 7f343fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d931e260 0x5643d931e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.673+0000 7f3437e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d9323c30 0x5643d93260f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39990/0 (socket says 172.21.15.2:39990) 2024-09-10T08:11:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.673+0000 7f3437e00640 1 -- 172.21.15.2:0/3491904657 learned_addr learned my addr 172.21.15.2:0/3491904657 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.673+0000 7f343f400640 1 --2- 172.21.15.2:0/3491904657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931eba0 0x5643d9323660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.674+0000 7f3437e00640 1 -- 172.21.15.2:0/3491904657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931eba0 msgr2=0x5643d9323660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.674+0000 7f3437e00640 1 --2- 172.21.15.2:0/3491904657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931eba0 0x5643d9323660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.674+0000 7f3437e00640 1 -- 172.21.15.2:0/3491904657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d931e260 msgr2=0x5643d931e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.674+0000 7f3437e00640 1 --2- 172.21.15.2:0/3491904657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d931e260 0x5643d931e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.674+0000 7f3437e00640 1 -- 172.21.15.2:0/3491904657 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d8f96210 con 0x5643d9323c30 2024-09-10T08:11:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.674+0000 7f343fe00640 1 --2- 172.21.15.2:0/3491904657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d931e260 0x5643d931e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.674+0000 7f343cc00640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f34240a2c00 0x7f34240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.674+0000 7f3437e00640 1 --2- 172.21.15.2:0/3491904657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d9323c30 0x5643d93260f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3428009ca0 tx=0x7f3428011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339a7dd8790ee250 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.674+0000 7f343cc00640 1 -- 172.21.15.2:0/284552073 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f34300ef890 con 0x7f3440101580 2024-09-10T08:11:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.674+0000 7f343fe00640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f34240a2c00 0x7f34240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.675+0000 7f342fe00640 1 -- 172.21.15.2:0/3491904657 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342801e070 con 0x5643d9323c30 2024-09-10T08:11:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.675+0000 7f342fe00640 1 -- 172.21.15.2:0/3491904657 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3428019b10 con 0x5643d9323c30 2024-09-10T08:11:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.675+0000 7f342fe00640 1 -- 172.21.15.2:0/3491904657 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342800eae0 con 0x5643d9323c30 2024-09-10T08:11:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.675+0000 7f34491eb740 1 -- 172.21.15.2:0/3491904657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d9323c30 msgr2=0x5643d93260f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.675+0000 7f34491eb740 1 --2- 172.21.15.2:0/3491904657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d9323c30 0x5643d93260f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3428009ca0 tx=0x7f3428011810 comp rx=0 tx=0).stop 2024-09-10T08:11:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.675+0000 7f343fe00640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f34240a2c00 0x7f34240a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3438005fd0 tx=0x7f3438009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.675+0000 7f34491eb740 1 -- 172.21.15.2:0/3491904657 shutdown_connections 2024-09-10T08:11:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.675+0000 7f34491eb740 1 --2- 172.21.15.2:0/3491904657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d9323c30 0x5643d93260f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.675+0000 7f34491eb740 1 --2- 172.21.15.2:0/3491904657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931eba0 0x5643d9323660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.675+0000 7f34491eb740 1 --2- 172.21.15.2:0/3491904657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d931e260 0x5643d931e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.675+0000 7f34491eb740 1 -- 172.21.15.2:0/3491904657 >> 172.21.15.2:0/3491904657 conn(0x5643d93198d0 msgr2=0x5643d9319cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.676+0000 7f34491eb740 1 -- 172.21.15.2:0/3491904657 shutdown_connections 2024-09-10T08:11:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.676+0000 7f34491eb740 1 -- 172.21.15.2:0/3491904657 wait complete. 2024-09-10T08:11:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.676+0000 7f34491eb740 1 -- start start 2024-09-10T08:11:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.676+0000 7f34491eb740 1 Processor -- start 2024-09-10T08:11:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f34491eb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931e260 0x5643d9328420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f34491eb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931eba0 0x5643d9328960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f34491eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d9323c30 0x5643d932f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f34491eb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5643d9326c00 con 0x5643d931eba0 2024-09-10T08:11:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f34491eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5643d920c090 con 0x5643d9323c30 2024-09-10T08:11:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f34491eb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5643d9326d80 con 0x5643d931e260 2024-09-10T08:11:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f34491eb740 10 client.0 _ll_register_callbacks cb 0x5643d9260f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f343f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931e260 0x5643d9328420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f343f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931e260 0x5643d9328420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:41752/0 (socket says 172.21.15.2:41752) 2024-09-10T08:11:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f343f400640 1 -- 172.21.15.2:0/1994262915 learned_addr learned my addr 172.21.15.2:0/1994262915 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f3437e00640 1 --2- 172.21.15.2:0/1994262915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931eba0 0x5643d9328960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f343fe00640 1 --2- 172.21.15.2:0/1994262915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d9323c30 0x5643d932f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f343f400640 1 -- 172.21.15.2:0/1994262915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d9323c30 msgr2=0x5643d932f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f343f400640 1 --2- 172.21.15.2:0/1994262915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d9323c30 0x5643d932f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f343f400640 1 -- 172.21.15.2:0/1994262915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931eba0 msgr2=0x5643d9328960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f343f400640 1 --2- 172.21.15.2:0/1994262915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931eba0 0x5643d9328960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.677+0000 7f343f400640 1 -- 172.21.15.2:0/1994262915 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643d9062620 con 0x5643d931e260 2024-09-10T08:11:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.678+0000 7f343f400640 1 --2- 172.21.15.2:0/1994262915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931e260 0x5643d9328420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3430006010 tx=0x7f3430003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f2bedca7d61bda9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.678+0000 7f342cc00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:11:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.678+0000 7f342cc00640 1 -- 172.21.15.2:0/1994262915 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430049070 con 0x5643d931e260 2024-09-10T08:11:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.678+0000 7f34491eb740 1 -- 172.21.15.2:0/1994262915 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3428019b10 con 0x5643d931e260 2024-09-10T08:11:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.678+0000 7f342cc00640 1 -- 172.21.15.2:0/1994262915 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3430003800 con 0x5643d931e260 2024-09-10T08:11:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.678+0000 7f342cc00640 1 -- 172.21.15.2:0/1994262915 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f34300bba70 con 0x5643d931e260 2024-09-10T08:11:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.678+0000 7f34491eb740 10 client.26287 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.678+0000 7f34491eb740 1 -- 172.21.15.2:0/1994262915 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643d9326cc0 con 0x5643d931e260 2024-09-10T08:11:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.678+0000 7f342cc00640 1 -- 172.21.15.2:0/1994262915 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3428019b10 con 0x5643d931e260 2024-09-10T08:11:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.679+0000 7f342cc00640 1 -- 172.21.15.2:0/1994262915 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f34300b8310 con 0x5643d931e260 2024-09-10T08:11:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.679+0000 7f34491eb740 10 client.26287 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.679+0000 7f34491eb740 10 client.26287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.679+0000 7f34491eb740 20 client.26287 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.679+0000 7f34491eb740 1 --2- 172.21.15.2:0/1994262915 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5643d9333ef0 0x5643d9336340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.679+0000 7f34491eb740 4 client.26287 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:11:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.679+0000 7f34491eb740 1 -- 172.21.15.2:0/1994262915 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643d9290830 con 0x5643d9333ef0 2024-09-10T08:11:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.679+0000 7f3437e00640 1 --2- 172.21.15.2:0/1994262915 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5643d9333ef0 0x5643d9336340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.680+0000 7f3437e00640 1 --2- 172.21.15.2:0/1994262915 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5643d9333ef0 0x5643d9336340 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a84317dff89f4089 server_cookie=ece27147d9705bef in_seq=0 out_seq=0 2024-09-10T08:11:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.680+0000 7f342cc00640 10 client.26287 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f342cc00640 1 -- 172.21.15.2:0/1994262915 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643d9290830 con 0x5643d9333ef0 2024-09-10T08:11:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f342cc00640 10 client.26287 handle_command_reply: tid=0 2024-09-10T08:11:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f34491eb740 1 client.26287 shutdown 2024-09-10T08:11:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f34491eb740 1 -- 172.21.15.2:0/1994262915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931e260 msgr2=0x5643d9328420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f34491eb740 1 --2- 172.21.15.2:0/1994262915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931e260 0x5643d9328420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3430006010 tx=0x7f3430003f50 comp rx=0 tx=0).stop 2024-09-10T08:11:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f34491eb740 1 -- 172.21.15.2:0/1994262915 shutdown_connections 2024-09-10T08:11:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f34491eb740 1 --2- 172.21.15.2:0/1994262915 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5643d9333ef0 0x5643d9336340 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f34491eb740 1 --2- 172.21.15.2:0/1994262915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d9323c30 0x5643d932f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f342cc00640 0 client.26287 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f34491eb740 1 --2- 172.21.15.2:0/1994262915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931eba0 0x5643d9328960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f34491eb740 1 --2- 172.21.15.2:0/1994262915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931e260 0x5643d9328420 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f34491eb740 1 -- 172.21.15.2:0/1994262915 >> 172.21.15.2:0/1994262915 conn(0x5643d93198d0 msgr2=0x5643d931b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f34491eb740 1 -- 172.21.15.2:0/1994262915 shutdown_connections 2024-09-10T08:11:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f34491eb740 1 -- 172.21.15.2:0/1994262915 wait complete. 2024-09-10T08:11:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.681+0000 7f34491eb740 20 client.26287 trim_cache size 0 max 16384 2024-09-10T08:11:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.697+0000 7f34491eb740 1 Processor -- start 2024-09-10T08:11:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.697+0000 7f34491eb740 1 -- start start 2024-09-10T08:11:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.697+0000 7f34491eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d931e260 0x5643d93281d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.697+0000 7f34491eb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931eba0 0x5643d932f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.697+0000 7f34491eb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d9323c30 0x5643d9328710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.697+0000 7f34491eb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f34401af1d0 con 0x5643d931eba0 2024-09-10T08:11:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.697+0000 7f34491eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f34401acaa0 con 0x5643d931e260 2024-09-10T08:11:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.697+0000 7f34491eb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f34401aa8f0 con 0x5643d9323c30 2024-09-10T08:11:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.697+0000 7f3437e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931eba0 0x5643d932f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.697+0000 7f3437e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931eba0 0x5643d932f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40014/0 (socket says 172.21.15.2:40014) 2024-09-10T08:11:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.697+0000 7f3437e00640 1 -- 172.21.15.2:0/1973474162 learned_addr learned my addr 172.21.15.2:0/1973474162 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.697+0000 7f343f400640 1 --2- 172.21.15.2:0/1973474162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d931e260 0x5643d93281d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.697+0000 7f343fe00640 1 --2- 172.21.15.2:0/1973474162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d9323c30 0x5643d9328710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.698+0000 7f3437e00640 1 -- 172.21.15.2:0/1973474162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d9323c30 msgr2=0x5643d9328710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.698+0000 7f3437e00640 1 --2- 172.21.15.2:0/1973474162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d9323c30 0x5643d9328710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.698+0000 7f3437e00640 1 -- 172.21.15.2:0/1973474162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d931e260 msgr2=0x5643d93281d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.698+0000 7f3437e00640 1 --2- 172.21.15.2:0/1973474162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d931e260 0x5643d93281d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.698+0000 7f3437e00640 1 -- 172.21.15.2:0/1973474162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d93196f0 con 0x5643d931eba0 2024-09-10T08:11:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.698+0000 7f343fe00640 1 --2- 172.21.15.2:0/1973474162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d9323c30 0x5643d9328710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:11:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.698+0000 7f3437e00640 1 --2- 172.21.15.2:0/1973474162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931eba0 0x5643d932f960 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f34280058c0 tx=0x7f3428005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf28212a2ac1f72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.698+0000 7f342f400640 1 -- 172.21.15.2:0/1973474162 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342801e040 con 0x5643d931eba0 2024-09-10T08:11:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.698+0000 7f342f400640 1 -- 172.21.15.2:0/1973474162 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f342801f040 con 0x5643d931eba0 2024-09-10T08:11:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.698+0000 7f342f400640 1 -- 172.21.15.2:0/1973474162 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34280048e0 con 0x5643d931eba0 2024-09-10T08:11:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.699+0000 7f34491eb740 1 -- 172.21.15.2:0/1973474162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931eba0 msgr2=0x5643d932f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.699+0000 7f34491eb740 1 --2- 172.21.15.2:0/1973474162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931eba0 0x5643d932f960 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f34280058c0 tx=0x7f3428005850 comp rx=0 tx=0).stop 2024-09-10T08:11:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.699+0000 7f34491eb740 1 -- 172.21.15.2:0/1973474162 shutdown_connections 2024-09-10T08:11:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.699+0000 7f34491eb740 1 --2- 172.21.15.2:0/1973474162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d9323c30 0x5643d9328710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.699+0000 7f34491eb740 1 --2- 172.21.15.2:0/1973474162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931eba0 0x5643d932f960 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.699+0000 7f34491eb740 1 --2- 172.21.15.2:0/1973474162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d931e260 0x5643d93281d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.699+0000 7f34491eb740 1 -- 172.21.15.2:0/1973474162 >> 172.21.15.2:0/1973474162 conn(0x5643d93198d0 msgr2=0x5643d93233d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.699+0000 7f34491eb740 1 -- 172.21.15.2:0/1973474162 shutdown_connections 2024-09-10T08:11:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.699+0000 7f34491eb740 1 -- 172.21.15.2:0/1973474162 wait complete. 2024-09-10T08:11:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.699+0000 7f34491eb740 1 -- start start 2024-09-10T08:11:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.700+0000 7f34491eb740 1 Processor -- start 2024-09-10T08:11:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.700+0000 7f34491eb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931e260 0x5643d932c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.700+0000 7f34491eb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931eba0 0x5643d932ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.700+0000 7f34491eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d9323c30 0x5643d939cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.700+0000 7f34491eb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f34401aaf60 con 0x5643d931e260 2024-09-10T08:11:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.700+0000 7f34491eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f34401ac3d0 con 0x5643d9323c30 2024-09-10T08:11:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.700+0000 7f34491eb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f34401ab630 con 0x5643d931eba0 2024-09-10T08:11:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.700+0000 7f34491eb740 10 client.0 _ll_register_callbacks cb 0x7f3430005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:11:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.700+0000 7f343f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931e260 0x5643d932c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.700+0000 7f343f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931e260 0x5643d932c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40020/0 (socket says 172.21.15.2:40020) 2024-09-10T08:11:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.700+0000 7f343f400640 1 -- 172.21.15.2:0/1620850289 learned_addr learned my addr 172.21.15.2:0/1620850289 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:11:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.700+0000 7f3437e00640 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931eba0 0x5643d932ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.700+0000 7f343fe00640 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d9323c30 0x5643d939cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f343f400640 1 -- 172.21.15.2:0/1620850289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931eba0 msgr2=0x5643d932ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f343f400640 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931eba0 0x5643d932ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f343f400640 1 -- 172.21.15.2:0/1620850289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d9323c30 msgr2=0x5643d939cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:11:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f343f400640 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d9323c30 0x5643d939cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f343f400640 1 -- 172.21.15.2:0/1620850289 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643d932fb70 con 0x5643d931e260 2024-09-10T08:11:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f3437e00640 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931eba0 0x5643d932ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:11:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f343f400640 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931e260 0x5643d932c540 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f34300efc50 tx=0x7f34300bc2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adac0a40bcfcb77a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:11:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f342cc00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:11:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f342cc00640 1 -- 172.21.15.2:0/1620850289 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430049040 con 0x5643d931e260 2024-09-10T08:11:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f342cc00640 1 -- 172.21.15.2:0/1620850289 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f34300c4680 con 0x5643d931e260 2024-09-10T08:11:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f34491eb740 1 -- 172.21.15.2:0/1620850289 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f342801f040 con 0x5643d931e260 2024-09-10T08:11:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f342cc00640 1 -- 172.21.15.2:0/1620850289 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3430005d50 con 0x5643d931e260 2024-09-10T08:11:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f34491eb740 10 client.16293 fetch_fsmap learned FSMap version 26 2024-09-10T08:11:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.701+0000 7f34491eb740 1 -- 172.21.15.2:0/1620850289 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643d939e110 con 0x5643d931e260 2024-09-10T08:11:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.702+0000 7f342cc00640 1 -- 172.21.15.2:0/1620850289 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f342801f040 con 0x5643d931e260 2024-09-10T08:11:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.702+0000 7f342cc00640 1 -- 172.21.15.2:0/1620850289 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f34300c4b60 con 0x5643d931e260 2024-09-10T08:11:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.702+0000 7f34491eb740 10 client.16293 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:11:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.702+0000 7f34491eb740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:11:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.702+0000 7f34491eb740 20 client.16293 populate_metadata read hostname 'smithi002' 2024-09-10T08:11:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.702+0000 7f34491eb740 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5643d9333ef0 0x5643d93a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:11:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.702+0000 7f34491eb740 4 client.16293 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:11:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.702+0000 7f34491eb740 1 -- 172.21.15.2:0/1620850289 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643d931f890 con 0x5643d9333ef0 2024-09-10T08:11:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.702+0000 7f3437e00640 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5643d9333ef0 0x5643d93a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:11:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.703+0000 7f3437e00640 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5643d9333ef0 0x5643d93a09c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aac55aa65a96cfc server_cookie=6e05be55be107e50 in_seq=0 out_seq=0 2024-09-10T08:11:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.703+0000 7f342cc00640 10 client.16293 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f342cc00640 1 -- 172.21.15.2:0/1620850289 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643d931f890 con 0x5643d9333ef0 2024-09-10T08:11:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f342cc00640 10 client.16293 handle_command_reply: tid=0 2024-09-10T08:11:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f34491eb740 1 client.16293 shutdown 2024-09-10T08:11:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f34491eb740 1 -- 172.21.15.2:0/1620850289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931e260 msgr2=0x5643d932c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f34491eb740 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931e260 0x5643d932c540 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f34300efc50 tx=0x7f34300bc2c0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f34491eb740 1 -- 172.21.15.2:0/1620850289 shutdown_connections 2024-09-10T08:11:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f34491eb740 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5643d9333ef0 0x5643d93a09c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f34491eb740 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5643d9323c30 0x5643d939cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f34491eb740 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5643d931eba0 0x5643d932ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f34491eb740 1 --2- 172.21.15.2:0/1620850289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5643d931e260 0x5643d932c540 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f342cc00640 0 client.16293 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:11:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f34491eb740 1 -- 172.21.15.2:0/1620850289 >> 172.21.15.2:0/1620850289 conn(0x5643d93198d0 msgr2=0x5643d9327f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f34491eb740 1 -- 172.21.15.2:0/1620850289 shutdown_connections 2024-09-10T08:11:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.716+0000 7f34491eb740 1 -- 172.21.15.2:0/1620850289 wait complete. 2024-09-10T08:11:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.717+0000 7f34491eb740 20 client.16293 trim_cache size 0 max 16384 2024-09-10T08:11:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:11:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:11:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:11:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:11:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.717+0000 7f3444c00640 1 -- 172.21.15.2:0/284552073 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f34240a2c00 msgr2=0x7f34240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.717+0000 7f3444c00640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f34240a2c00 0x7f34240a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3438005fd0 tx=0x7f3438009500 comp rx=0 tx=0).stop 2024-09-10T08:11:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.717+0000 7f3444c00640 1 -- 172.21.15.2:0/284552073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3440101580 msgr2=0x7f344019a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:11:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.717+0000 7f3444c00640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3440101580 0x7f344019a040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3430031440 tx=0x7f3430032ce0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.717+0000 7f3444c00640 1 -- 172.21.15.2:0/284552073 shutdown_connections 2024-09-10T08:11:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.717+0000 7f3444c00640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3440103050 0x7f34401a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.717+0000 7f3444c00640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f34240a2c00 0x7f34240a50c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.717+0000 7f3444c00640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3440101f40 0x7f344019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.717+0000 7f3444c00640 1 --2- 172.21.15.2:0/284552073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3440101580 0x7f344019a040 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:11:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.717+0000 7f3444c00640 1 -- 172.21.15.2:0/284552073 >> 172.21.15.2:0/284552073 conn(0x7f34400fd0d0 msgr2=0x7f3440108330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:11:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.718+0000 7f3444c00640 1 -- 172.21.15.2:0/284552073 shutdown_connections 2024-09-10T08:11:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:11:47.718+0000 7f3444c00640 1 -- 172.21.15.2:0/284552073 wait complete. 2024-09-10T08:11:47.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:47 smithi002 ceph-mon[30563]: pgmap v496: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:11:49.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:49 smithi155 ceph-mon[35405]: pgmap v497: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:11:49.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:49 smithi107 ceph-mon[35048]: pgmap v497: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:11:49.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:49 smithi002 ceph-mon[30563]: pgmap v497: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:11:51.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:51 smithi002 ceph-mon[30563]: pgmap v498: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-10T08:11:51.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:51 smithi155 ceph-mon[35405]: pgmap v498: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-10T08:11:51.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:51 smithi107 ceph-mon[35048]: pgmap v498: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-10T08:11:53.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:53 smithi155 ceph-mon[35405]: pgmap v499: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:53.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:53 smithi107 ceph-mon[35048]: pgmap v499: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:53.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:53 smithi002 ceph-mon[30563]: pgmap v499: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:11:55.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:55 smithi155 ceph-mon[35405]: pgmap v500: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:11:55.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:55 smithi107 ceph-mon[35048]: pgmap v500: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:11:55.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:55 smithi002 ceph-mon[30563]: pgmap v500: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:11:57.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:57 smithi155 ceph-mon[35405]: pgmap v501: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:11:57.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:57 smithi107 ceph-mon[35048]: pgmap v501: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:11:57.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:57 smithi002 ceph-mon[30563]: pgmap v501: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:11:59.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:11:59 smithi155 ceph-mon[35405]: pgmap v502: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:11:59.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:11:59 smithi107 ceph-mon[35048]: pgmap v502: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:11:59.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:11:59 smithi002 ceph-mon[30563]: pgmap v502: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.847+0000 7ff933c00640 1 -- 172.21.15.2:0/1875857032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c103d60 msgr2=0x7ff92c104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.847+0000 7ff933c00640 1 --2- 172.21.15.2:0/1875857032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c103d60 0x7ff92c104160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009860 tx=0x7ff91c02a340 comp rx=0 tx=0).stop 2024-09-10T08:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.847+0000 7ff933c00640 1 -- 172.21.15.2:0/1875857032 shutdown_connections 2024-09-10T08:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.847+0000 7ff933c00640 1 --2- 172.21.15.2:0/1875857032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff92c105910 0x7ff92c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.847+0000 7ff933c00640 1 --2- 172.21.15.2:0/1875857032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff92c104f50 0x7ff92c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.847+0000 7ff933c00640 1 --2- 172.21.15.2:0/1875857032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c103d60 0x7ff92c104160 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.847+0000 7ff933c00640 1 -- 172.21.15.2:0/1875857032 >> 172.21.15.2:0/1875857032 conn(0x7ff92c0ff4e0 msgr2=0x7ff92c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.847+0000 7ff933c00640 1 -- 172.21.15.2:0/1875857032 shutdown_connections 2024-09-10T08:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.847+0000 7ff933c00640 1 -- 172.21.15.2:0/1875857032 wait complete. 2024-09-10T08:12:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.848+0000 7ff933c00640 1 Processor -- start 2024-09-10T08:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.848+0000 7ff933c00640 1 -- start start 2024-09-10T08:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.848+0000 7ff933c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff92c103d60 0x7ff92c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.848+0000 7ff933c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c104f50 0x7ff92c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.848+0000 7ff933c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff92c105910 0x7ff92c1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.848+0000 7ff933c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c10f9e0 con 0x7ff92c103d60 2024-09-10T08:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.848+0000 7ff933c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c10f860 con 0x7ff92c105910 2024-09-10T08:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.848+0000 7ff933c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c10fb60 con 0x7ff92c104f50 2024-09-10T08:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.849+0000 7ff931200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff92c103d60 0x7ff92c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.849+0000 7ff931c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff92c105910 0x7ff92c1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.849+0000 7ff931200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff92c103d60 0x7ff92c19a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51570/0 (socket says 172.21.15.2:51570) 2024-09-10T08:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.849+0000 7ff931200640 1 -- 172.21.15.2:0/3268959080 learned_addr learned my addr 172.21.15.2:0/3268959080 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.849+0000 7ff923e00640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c104f50 0x7ff92c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.849+0000 7ff931c00640 1 -- 172.21.15.2:0/3268959080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c104f50 msgr2=0x7ff92c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.849+0000 7ff931c00640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c104f50 0x7ff92c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.849+0000 7ff931c00640 1 -- 172.21.15.2:0/3268959080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff92c103d60 msgr2=0x7ff92c19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.849+0000 7ff931c00640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff92c103d60 0x7ff92c19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.849+0000 7ff931c00640 1 -- 172.21.15.2:0/3268959080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92c1a1c20 con 0x7ff92c105910 2024-09-10T08:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.849+0000 7ff931200640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff92c103d60 0x7ff92c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.849+0000 7ff931c00640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff92c105910 0x7ff92c1a15b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff92400ec50 tx=0x7ff92400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.850+0000 7ff92a000640 1 -- 172.21.15.2:0/3268959080 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924003b70 con 0x7ff92c105910 2024-09-10T08:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.850+0000 7ff92a000640 1 -- 172.21.15.2:0/3268959080 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff92400ee90 con 0x7ff92c105910 2024-09-10T08:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.850+0000 7ff933c00640 1 -- 172.21.15.2:0/3268959080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92c1a1f10 con 0x7ff92c105910 2024-09-10T08:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.850+0000 7ff92a000640 1 -- 172.21.15.2:0/3268959080 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9240107a0 con 0x7ff92c105910 2024-09-10T08:12:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.850+0000 7ff933c00640 1 -- 172.21.15.2:0/3268959080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92c1a2380 con 0x7ff92c105910 2024-09-10T08:12:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.853+0000 7ff936c08740 1 Processor -- start 2024-09-10T08:12:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.853+0000 7ff92a000640 1 -- 172.21.15.2:0/3268959080 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff924004020 con 0x7ff92c105910 2024-09-10T08:12:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.853+0000 7ff936c08740 1 -- start start 2024-09-10T08:12:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.853+0000 7ff936c08740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c090 0x556fe135c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff936c08740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe135c9d0 0x556fe1361490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff936c08740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe1361a60 0x556fe1363f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff936c08740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556fe12c1100 con 0x556fe135c090 2024-09-10T08:12:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff936c08740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556fe12691b0 con 0x556fe1361a60 2024-09-10T08:12:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff936c08740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556fe12bd0c0 con 0x556fe135c9d0 2024-09-10T08:12:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff923e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c090 0x556fe135c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff923e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c090 0x556fe135c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51576/0 (socket says 172.21.15.2:51576) 2024-09-10T08:12:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff923e00640 1 -- 172.21.15.2:0/140346532 learned_addr learned my addr 172.21.15.2:0/140346532 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff931200640 1 --2- 172.21.15.2:0/140346532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe135c9d0 0x556fe1361490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff923e00640 1 --2- 172.21.15.2:0/140346532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe1361a60 0x556fe1363f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff923e00640 1 -- 172.21.15.2:0/140346532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe135c9d0 msgr2=0x556fe1361490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff923e00640 1 --2- 172.21.15.2:0/140346532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe135c9d0 0x556fe1361490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff923e00640 1 -- 172.21.15.2:0/140346532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe1361a60 msgr2=0x556fe1363f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff923e00640 1 --2- 172.21.15.2:0/140346532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe1361a60 0x556fe1363f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.854+0000 7ff923e00640 1 -- 172.21.15.2:0/140346532 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe0fd4210 con 0x556fe135c090 2024-09-10T08:12:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.855+0000 7ff92a000640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff9100a2c30 0x7ff9100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.855+0000 7ff923e00640 1 --2- 172.21.15.2:0/140346532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c090 0x556fe135c490 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff914013840 tx=0x7ff914009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6726fe28c6956a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.855+0000 7ff92a000640 1 -- 172.21.15.2:0/3268959080 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff9240cf9e0 con 0x7ff92c105910 2024-09-10T08:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.855+0000 7ff922000640 1 -- 172.21.15.2:0/140346532 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914013130 con 0x556fe135c090 2024-09-10T08:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.855+0000 7ff931200640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff9100a2c30 0x7ff9100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.855+0000 7ff922000640 1 -- 172.21.15.2:0/140346532 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9140132d0 con 0x556fe135c090 2024-09-10T08:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.855+0000 7ff922000640 1 -- 172.21.15.2:0/140346532 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914004de0 con 0x556fe135c090 2024-09-10T08:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.856+0000 7ff936c08740 1 -- 172.21.15.2:0/140346532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c090 msgr2=0x556fe135c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.856+0000 7ff931200640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff9100a2c30 0x7ff9100a50f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff91c031000 tx=0x7ff91c030550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.856+0000 7ff936c08740 1 --2- 172.21.15.2:0/140346532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c090 0x556fe135c490 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff914013840 tx=0x7ff914009870 comp rx=0 tx=0).stop 2024-09-10T08:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.856+0000 7ff936c08740 1 -- 172.21.15.2:0/140346532 shutdown_connections 2024-09-10T08:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.856+0000 7ff936c08740 1 --2- 172.21.15.2:0/140346532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe1361a60 0x556fe1363f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.856+0000 7ff936c08740 1 --2- 172.21.15.2:0/140346532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe135c9d0 0x556fe1361490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.856+0000 7ff936c08740 1 --2- 172.21.15.2:0/140346532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c090 0x556fe135c490 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.856+0000 7ff936c08740 1 -- 172.21.15.2:0/140346532 >> 172.21.15.2:0/140346532 conn(0x556fe1357700 msgr2=0x556fe1357b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.856+0000 7ff936c08740 1 -- 172.21.15.2:0/140346532 shutdown_connections 2024-09-10T08:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.856+0000 7ff936c08740 1 -- 172.21.15.2:0/140346532 wait complete. 2024-09-10T08:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.857+0000 7ff936c08740 1 -- start start 2024-09-10T08:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.857+0000 7ff936c08740 1 Processor -- start 2024-09-10T08:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.857+0000 7ff936c08740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe135c090 0x556fe13661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.857+0000 7ff936c08740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe13666f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.857+0000 7ff936c08740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe1361a60 0x556fe136d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.857+0000 7ff936c08740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556fe13649d0 con 0x556fe135c9d0 2024-09-10T08:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.857+0000 7ff936c08740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556fe124a090 con 0x556fe135c090 2024-09-10T08:12:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.857+0000 7ff936c08740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556fe1364b50 con 0x556fe1361a60 2024-09-10T08:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.857+0000 7ff936c08740 10 client.0 _ll_register_callbacks cb 0x556fe129ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.857+0000 7ff923e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe13666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.857+0000 7ff923e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe13666f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51592/0 (socket says 172.21.15.2:51592) 2024-09-10T08:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.858+0000 7ff923e00640 1 -- 172.21.15.2:0/1664751278 learned_addr learned my addr 172.21.15.2:0/1664751278 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.858+0000 7ff931c00640 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe1361a60 0x556fe136d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.858+0000 7ff931200640 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe135c090 0x556fe13661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.858+0000 7ff923e00640 1 -- 172.21.15.2:0/1664751278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe1361a60 msgr2=0x556fe136d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.858+0000 7ff923e00640 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe1361a60 0x556fe136d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.858+0000 7ff923e00640 1 -- 172.21.15.2:0/1664751278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe135c090 msgr2=0x556fe13661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.858+0000 7ff923e00640 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe135c090 0x556fe13661b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.858+0000 7ff923e00640 1 -- 172.21.15.2:0/1664751278 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe10a0620 con 0x556fe135c9d0 2024-09-10T08:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.858+0000 7ff931c00640 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe1361a60 0x556fe136d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.858+0000 7ff923e00640 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe13666f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff914013810 tx=0x7ff914009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabf2da3efa17ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.858+0000 7ff91aa00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.858+0000 7ff91aa00640 1 -- 172.21.15.2:0/1664751278 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914005430 con 0x556fe135c9d0 2024-09-10T08:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.859+0000 7ff91aa00640 1 -- 172.21.15.2:0/1664751278 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff9140055d0 con 0x556fe135c9d0 2024-09-10T08:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.859+0000 7ff936c08740 1 -- 172.21.15.2:0/1664751278 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9140132d0 con 0x556fe135c9d0 2024-09-10T08:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.859+0000 7ff91aa00640 1 -- 172.21.15.2:0/1664751278 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff914032050 con 0x556fe135c9d0 2024-09-10T08:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.859+0000 7ff936c08740 10 client.16305 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.859+0000 7ff936c08740 1 -- 172.21.15.2:0/1664751278 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556fe1370520 con 0x556fe135c9d0 2024-09-10T08:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.859+0000 7ff91aa00640 1 -- 172.21.15.2:0/1664751278 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9140132d0 con 0x556fe135c9d0 2024-09-10T08:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.859+0000 7ff91aa00640 1 -- 172.21.15.2:0/1664751278 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff9140207c0 con 0x556fe135c9d0 2024-09-10T08:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.859+0000 7ff936c08740 10 client.16305 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.859+0000 7ff936c08740 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.859+0000 7ff936c08740 20 client.16305 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.859+0000 7ff936c08740 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556fe1371a60 0x556fe1373e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.859+0000 7ff936c08740 4 client.16305 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.859+0000 7ff936c08740 1 -- 172.21.15.2:0/1664751278 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fe12ce830 con 0x556fe1371a60 2024-09-10T08:12:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.860+0000 7ff931200640 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556fe1371a60 0x556fe1373e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.860+0000 7ff931200640 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556fe1371a60 0x556fe1373e50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3822126629448202 server_cookie=2cba82285e885fe in_seq=0 out_seq=0 2024-09-10T08:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.860+0000 7ff91aa00640 10 client.16305 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.861+0000 7ff91aa00640 1 -- 172.21.15.2:0/1664751278 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fe12ce830 con 0x556fe1371a60 2024-09-10T08:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.861+0000 7ff91aa00640 10 client.16305 handle_command_reply: tid=0 2024-09-10T08:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.861+0000 7ff936c08740 1 client.16305 shutdown 2024-09-10T08:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.861+0000 7ff936c08740 1 -- 172.21.15.2:0/1664751278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 msgr2=0x556fe13666f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.861+0000 7ff936c08740 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe13666f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff914013810 tx=0x7ff914009350 comp rx=0 tx=0).stop 2024-09-10T08:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.861+0000 7ff936c08740 1 -- 172.21.15.2:0/1664751278 shutdown_connections 2024-09-10T08:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.861+0000 7ff936c08740 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556fe1371a60 0x556fe1373e50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.861+0000 7ff936c08740 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe1361a60 0x556fe136d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.861+0000 7ff936c08740 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe13666f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.861+0000 7ff936c08740 1 --2- 172.21.15.2:0/1664751278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe135c090 0x556fe13661b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.861+0000 7ff91aa00640 0 client.16305 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.862+0000 7ff936c08740 1 -- 172.21.15.2:0/1664751278 >> 172.21.15.2:0/1664751278 conn(0x556fe1357700 msgr2=0x556fe1358f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.862+0000 7ff936c08740 1 -- 172.21.15.2:0/1664751278 shutdown_connections 2024-09-10T08:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.862+0000 7ff936c08740 1 -- 172.21.15.2:0/1664751278 wait complete. 2024-09-10T08:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.862+0000 7ff936c08740 20 client.16305 trim_cache size 0 max 16384 2024-09-10T08:12:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.877+0000 7ff936c08740 1 Processor -- start 2024-09-10T08:12:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.877+0000 7ff936c08740 1 -- start start 2024-09-10T08:12:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.877+0000 7ff936c08740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe1357700 0x556fe1357ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.877+0000 7ff936c08740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe135c090 0x556fe136d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.877+0000 7ff936c08740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe1358020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.878+0000 7ff936c08740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c1af3c0 con 0x556fe135c9d0 2024-09-10T08:12:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.878+0000 7ff936c08740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c1acc90 con 0x556fe135c090 2024-09-10T08:12:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.878+0000 7ff936c08740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c1aaae0 con 0x556fe1357700 2024-09-10T08:12:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.878+0000 7ff931c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe1358020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.878+0000 7ff923e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe135c090 0x556fe136d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.878+0000 7ff931c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe1358020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51600/0 (socket says 172.21.15.2:51600) 2024-09-10T08:12:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.878+0000 7ff931200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe1357700 0x556fe1357ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.878+0000 7ff931c00640 1 -- 172.21.15.2:0/497636631 learned_addr learned my addr 172.21.15.2:0/497636631 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.878+0000 7ff923e00640 1 -- 172.21.15.2:0/497636631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe1357700 msgr2=0x556fe1357ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.878+0000 7ff923e00640 1 --2- 172.21.15.2:0/497636631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe1357700 0x556fe1357ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.878+0000 7ff923e00640 1 -- 172.21.15.2:0/497636631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 msgr2=0x556fe1358020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.878+0000 7ff923e00640 1 --2- 172.21.15.2:0/497636631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe1358020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.878+0000 7ff923e00640 1 -- 172.21.15.2:0/497636631 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe1357520 con 0x556fe135c090 2024-09-10T08:12:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.879+0000 7ff923e00640 1 --2- 172.21.15.2:0/497636631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe135c090 0x556fe136d6f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff914020be0 tx=0x7ff914013690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec98437e4e9102e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.879+0000 7ff921600640 1 -- 172.21.15.2:0/497636631 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91402f070 con 0x556fe135c090 2024-09-10T08:12:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.879+0000 7ff921600640 1 -- 172.21.15.2:0/497636631 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff914025030 con 0x556fe135c090 2024-09-10T08:12:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.879+0000 7ff921600640 1 -- 172.21.15.2:0/497636631 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914009af0 con 0x556fe135c090 2024-09-10T08:12:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.879+0000 7ff936c08740 1 -- 172.21.15.2:0/497636631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe135c090 msgr2=0x556fe136d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.879+0000 7ff936c08740 1 --2- 172.21.15.2:0/497636631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe135c090 0x556fe136d6f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff914020be0 tx=0x7ff914013690 comp rx=0 tx=0).stop 2024-09-10T08:12:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.879+0000 7ff936c08740 1 -- 172.21.15.2:0/497636631 shutdown_connections 2024-09-10T08:12:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.879+0000 7ff936c08740 1 --2- 172.21.15.2:0/497636631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe1358020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.879+0000 7ff936c08740 1 --2- 172.21.15.2:0/497636631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe135c090 0x556fe136d6f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.879+0000 7ff936c08740 1 --2- 172.21.15.2:0/497636631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe1357700 0x556fe1357ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.879+0000 7ff936c08740 1 -- 172.21.15.2:0/497636631 >> 172.21.15.2:0/497636631 conn(0x556fe1361580 msgr2=0x556fe135db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.880+0000 7ff936c08740 1 -- 172.21.15.2:0/497636631 shutdown_connections 2024-09-10T08:12:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.880+0000 7ff936c08740 1 -- 172.21.15.2:0/497636631 wait complete. 2024-09-10T08:12:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.880+0000 7ff936c08740 1 -- start start 2024-09-10T08:12:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.880+0000 7ff936c08740 1 Processor -- start 2024-09-10T08:12:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.880+0000 7ff936c08740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe1357700 0x556fe136a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.880+0000 7ff936c08740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe135c090 0x556fe136aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.880+0000 7ff936c08740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe13da690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.880+0000 7ff936c08740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c1ab150 con 0x556fe135c9d0 2024-09-10T08:12:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.880+0000 7ff936c08740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c1ac5c0 con 0x556fe1357700 2024-09-10T08:12:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.880+0000 7ff936c08740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c1ab820 con 0x556fe135c090 2024-09-10T08:12:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.881+0000 7ff936c08740 10 client.0 _ll_register_callbacks cb 0x7ff914009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.881+0000 7ff923e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe135c090 0x556fe136aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.881+0000 7ff931c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe13da690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.881+0000 7ff931200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe1357700 0x556fe136a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.881+0000 7ff931200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe1357700 0x556fe136a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:45660/0 (socket says 172.21.15.2:45660) 2024-09-10T08:12:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.881+0000 7ff931200640 1 -- 172.21.15.2:0/444518313 learned_addr learned my addr 172.21.15.2:0/444518313 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.881+0000 7ff923e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe135c090 0x556fe136aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59694/0 (socket says 172.21.15.2:59694) 2024-09-10T08:12:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.881+0000 7ff931200640 1 -- 172.21.15.2:0/444518313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe135c090 msgr2=0x556fe136aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.881+0000 7ff931200640 1 --2- 172.21.15.2:0/444518313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe135c090 0x556fe136aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.881+0000 7ff931200640 1 -- 172.21.15.2:0/444518313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 msgr2=0x556fe13da690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.881+0000 7ff931200640 1 --2- 172.21.15.2:0/444518313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe13da690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.881+0000 7ff931200640 1 -- 172.21.15.2:0/444518313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe13dabd0 con 0x556fe1357700 2024-09-10T08:12:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.881+0000 7ff923e00640 1 --2- 172.21.15.2:0/444518313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe135c090 0x556fe136aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.882+0000 7ff931200640 1 --2- 172.21.15.2:0/444518313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe1357700 0x556fe136a4e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff91c030410 tx=0x7ff91c02a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91455458783818e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.882+0000 7ff91aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.882+0000 7ff91aa00640 1 -- 172.21.15.2:0/444518313 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c058060 con 0x556fe1357700 2024-09-10T08:12:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.882+0000 7ff936c08740 1 -- 172.21.15.2:0/444518313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff914025030 con 0x556fe1357700 2024-09-10T08:12:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.882+0000 7ff91aa00640 1 -- 172.21.15.2:0/444518313 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff91c0046c0 con 0x556fe1357700 2024-09-10T08:12:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.882+0000 7ff91aa00640 1 -- 172.21.15.2:0/444518313 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff91c05f050 con 0x556fe1357700 2024-09-10T08:12:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.882+0000 7ff936c08740 10 client.26345 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.882+0000 7ff936c08740 1 -- 172.21.15.2:0/444518313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556fe13dbdf0 con 0x556fe1357700 2024-09-10T08:12:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.882+0000 7ff91aa00640 1 -- 172.21.15.2:0/444518313 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff914025030 con 0x556fe1357700 2024-09-10T08:12:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.883+0000 7ff91aa00640 1 -- 172.21.15.2:0/444518313 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff91c048040 con 0x556fe1357700 2024-09-10T08:12:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.883+0000 7ff936c08740 10 client.26345 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.883+0000 7ff936c08740 10 client.26345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.883+0000 7ff936c08740 20 client.26345 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.883+0000 7ff936c08740 1 --2- 172.21.15.2:0/444518313 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556fe1361a60 0x556fe13de7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.883+0000 7ff936c08740 4 client.26345 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:12:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.883+0000 7ff936c08740 1 -- 172.21.15.2:0/444518313 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556fe1352d80 con 0x556fe1361a60 2024-09-10T08:12:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.883+0000 7ff923e00640 1 --2- 172.21.15.2:0/444518313 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556fe1361a60 0x556fe13de7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.883+0000 7ff923e00640 1 --2- 172.21.15.2:0/444518313 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556fe1361a60 0x556fe13de7c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3724493581f445d3 server_cookie=6bacab587b9c185 in_seq=0 out_seq=0 2024-09-10T08:12:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.883+0000 7ff91aa00640 10 client.26345 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff91aa00640 1 -- 172.21.15.2:0/444518313 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556fe1352d80 con 0x556fe1361a60 2024-09-10T08:12:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff91aa00640 10 client.26345 handle_command_reply: tid=0 2024-09-10T08:12:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff936c08740 1 client.26345 shutdown 2024-09-10T08:12:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff936c08740 1 -- 172.21.15.2:0/444518313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe1357700 msgr2=0x556fe136a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff936c08740 1 --2- 172.21.15.2:0/444518313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe1357700 0x556fe136a4e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff91c030410 tx=0x7ff91c02a340 comp rx=0 tx=0).stop 2024-09-10T08:12:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff936c08740 1 -- 172.21.15.2:0/444518313 shutdown_connections 2024-09-10T08:12:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff936c08740 1 --2- 172.21.15.2:0/444518313 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556fe1361a60 0x556fe13de7c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff936c08740 1 --2- 172.21.15.2:0/444518313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556fe135c9d0 0x556fe13da690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff936c08740 1 --2- 172.21.15.2:0/444518313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fe135c090 0x556fe136aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff936c08740 1 --2- 172.21.15.2:0/444518313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fe1357700 0x556fe136a4e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff91aa00640 0 client.26345 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff936c08740 1 -- 172.21.15.2:0/444518313 >> 172.21.15.2:0/444518313 conn(0x556fe1361580 msgr2=0x556fe1356060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff936c08740 1 -- 172.21.15.2:0/444518313 shutdown_connections 2024-09-10T08:12:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff936c08740 1 -- 172.21.15.2:0/444518313 wait complete. 2024-09-10T08:12:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.897+0000 7ff936c08740 20 client.26345 trim_cache size 0 max 16384 2024-09-10T08:12:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:12:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:12:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:12:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:12:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.898+0000 7ff933c00640 1 -- 172.21.15.2:0/3268959080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff9100a2c30 msgr2=0x7ff9100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.898+0000 7ff933c00640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff9100a2c30 0x7ff9100a50f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff91c031000 tx=0x7ff91c030550 comp rx=0 tx=0).stop 2024-09-10T08:12:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.898+0000 7ff933c00640 1 -- 172.21.15.2:0/3268959080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff92c105910 msgr2=0x7ff92c1a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.898+0000 7ff933c00640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff92c105910 0x7ff92c1a15b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff92400ec50 tx=0x7ff92400c6a0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.898+0000 7ff933c00640 1 -- 172.21.15.2:0/3268959080 shutdown_connections 2024-09-10T08:12:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.898+0000 7ff933c00640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff92c105910 0x7ff92c1a15b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.898+0000 7ff933c00640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff9100a2c30 0x7ff9100a50f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.898+0000 7ff933c00640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c104f50 0x7ff92c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.898+0000 7ff933c00640 1 --2- 172.21.15.2:0/3268959080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff92c103d60 0x7ff92c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.898+0000 7ff933c00640 1 -- 172.21.15.2:0/3268959080 >> 172.21.15.2:0/3268959080 conn(0x7ff92c0ff4e0 msgr2=0x7ff92c1082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.898+0000 7ff933c00640 1 -- 172.21.15.2:0/3268959080 shutdown_connections 2024-09-10T08:12:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:00.898+0000 7ff933c00640 1 -- 172.21.15.2:0/3268959080 wait complete. 2024-09-10T08:12:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:01 smithi155 ceph-mon[35405]: pgmap v503: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:12:01.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:01 smithi107 ceph-mon[35048]: pgmap v503: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:12:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:01 smithi002 ceph-mon[30563]: pgmap v503: 129 pgs: 129 active+clean; 1.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:12:03.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:03 smithi155 ceph-mon[35405]: pgmap v504: 129 pgs: 129 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:12:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:03 smithi107 ceph-mon[35048]: pgmap v504: 129 pgs: 129 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:12:03.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:03 smithi002 ceph-mon[30563]: pgmap v504: 129 pgs: 129 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:12:05.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:05 smithi155 ceph-mon[35405]: pgmap v505: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:12:05.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:05 smithi107 ceph-mon[35048]: pgmap v505: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:12:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:05 smithi002 ceph-mon[30563]: pgmap v505: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:12:07.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:07 smithi155 ceph-mon[35405]: pgmap v506: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:12:07.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:07 smithi107 ceph-mon[35048]: pgmap v506: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:12:07.754 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:12:07.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:07 smithi002 ceph-mon[30563]: pgmap v506: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:12:08.153 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.803+0000 7f967d000640 1 -- 172.21.15.2:0/2231927839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9678112860 msgr2=0x7f9678114c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.803+0000 7f967d000640 1 --2- 172.21.15.2:0/2231927839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9678112860 0x7f9678114c50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9670009ca0 tx=0x7f967002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.803+0000 7f967d000640 1 -- 172.21.15.2:0/2231927839 shutdown_connections 2024-09-10T08:12:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.803+0000 7f967d000640 1 --2- 172.21.15.2:0/2231927839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9678112860 0x7f9678114c50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.803+0000 7f967d000640 1 --2- 172.21.15.2:0/2231927839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9678077040 0x7f9678112320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.803+0000 7f967d000640 1 --2- 172.21.15.2:0/2231927839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9678076720 0x7f9678076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.804+0000 7f967d000640 1 -- 172.21.15.2:0/2231927839 >> 172.21.15.2:0/2231927839 conn(0x7f96780ff880 msgr2=0x7f9678101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.804+0000 7f967d000640 1 -- 172.21.15.2:0/2231927839 shutdown_connections 2024-09-10T08:12:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.804+0000 7f967d000640 1 -- 172.21.15.2:0/2231927839 wait complete. 2024-09-10T08:12:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.804+0000 7f967d000640 1 Processor -- start 2024-09-10T08:12:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.805+0000 7f967d000640 1 -- start start 2024-09-10T08:12:08.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.805+0000 7f967d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9678076720 0x7f96781a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.805+0000 7f967d000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9678077040 0x7f96781a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.805+0000 7f967d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9678112860 0x7f967819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.805+0000 7f967d000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9678118410 con 0x7f9678077040 2024-09-10T08:12:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.805+0000 7f967d000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9678118290 con 0x7f9678112860 2024-09-10T08:12:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.805+0000 7f967d000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9678118590 con 0x7f9678076720 2024-09-10T08:12:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.805+0000 7f9677e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9678112860 0x7f967819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.805+0000 7f9676a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9678077040 0x7f96781a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.805+0000 7f9677e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9678112860 0x7f967819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46072/0 (socket says 172.21.15.2:46072) 2024-09-10T08:12:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.805+0000 7f9677e00640 1 -- 172.21.15.2:0/3630835169 learned_addr learned my addr 172.21.15.2:0/3630835169 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.805+0000 7f9677400640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9678076720 0x7f96781a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.806+0000 7f9676a00640 1 -- 172.21.15.2:0/3630835169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9678076720 msgr2=0x7f96781a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.806+0000 7f9676a00640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9678076720 0x7f96781a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.806+0000 7f9676a00640 1 -- 172.21.15.2:0/3630835169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9678112860 msgr2=0x7f967819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.806+0000 7f9676a00640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9678112860 0x7f967819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.806+0000 7f9676a00640 1 -- 172.21.15.2:0/3630835169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f967819efb0 con 0x7f9678077040 2024-09-10T08:12:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.806+0000 7f9677e00640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9678112860 0x7f967819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.806+0000 7f9676a00640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9678077040 0x7f96781a4bb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f966400e9f0 tx=0x7f966400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.806+0000 7f966fe00640 1 -- 172.21.15.2:0/3630835169 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966400ccc0 con 0x7f9678077040 2024-09-10T08:12:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.807+0000 7f966fe00640 1 -- 172.21.15.2:0/3630835169 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9664005020 con 0x7f9678077040 2024-09-10T08:12:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.807+0000 7f967d000640 1 -- 172.21.15.2:0/3630835169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f967819f2a0 con 0x7f9678077040 2024-09-10T08:12:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.807+0000 7f966fe00640 1 -- 172.21.15.2:0/3630835169 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96640049e0 con 0x7f9678077040 2024-09-10T08:12:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.807+0000 7f967d000640 1 -- 172.21.15.2:0/3630835169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96781ab5f0 con 0x7f9678077040 2024-09-10T08:12:08.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.811+0000 7f9681019740 1 Processor -- start 2024-09-10T08:12:08.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.811+0000 7f9681019740 1 -- start start 2024-09-10T08:12:08.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.811+0000 7f9681019740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dabd50 0x560484dac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.811+0000 7f9681019740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 0x560484db69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.811+0000 7f9681019740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484db9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.812+0000 7f9681019740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560484d123e0 con 0x560484db6f30 2024-09-10T08:12:08.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.812+0000 7f9681019740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560484cb9d80 con 0x560484dac670 2024-09-10T08:12:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.812+0000 7f9681019740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560484d0e3a0 con 0x560484dabd50 2024-09-10T08:12:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.812+0000 7f966fe00640 1 -- 172.21.15.2:0/3630835169 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9664005560 con 0x7f9678077040 2024-09-10T08:12:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.812+0000 7f9677e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dabd50 0x560484dac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.812+0000 7f9677400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 0x560484db69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.812+0000 7f9677400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 0x560484db69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46082/0 (socket says 172.21.15.2:46082) 2024-09-10T08:12:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.812+0000 7f9677e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dabd50 0x560484dac130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34798/0 (socket says 172.21.15.2:34798) 2024-09-10T08:12:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.812+0000 7f9677400640 1 -- 172.21.15.2:0/158383890 learned_addr learned my addr 172.21.15.2:0/158383890 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.812+0000 7f9677400640 1 -- 172.21.15.2:0/158383890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dabd50 msgr2=0x560484dac130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.812+0000 7f9677400640 1 --2- 172.21.15.2:0/158383890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dabd50 0x560484dac130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.812+0000 7f9677400640 1 -- 172.21.15.2:0/158383890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 msgr2=0x560484db9320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:12:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.812+0000 7f9676a00640 1 --2- 172.21.15.2:0/158383890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484db9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.813+0000 7f9677400640 1 --2- 172.21.15.2:0/158383890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484db9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.813+0000 7f9677400640 1 -- 172.21.15.2:0/158383890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560484cd8ae0 con 0x560484dac670 2024-09-10T08:12:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.813+0000 7f9676a00640 1 --2- 172.21.15.2:0/158383890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484db9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:12:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.813+0000 7f9677400640 1 --2- 172.21.15.2:0/158383890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 0x560484db69f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f966800bd10 tx=0x7f96680107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d134650448ff16f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.813+0000 7f966cc00640 1 -- 172.21.15.2:0/158383890 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668004030 con 0x560484dac670 2024-09-10T08:12:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.813+0000 7f966cc00640 1 -- 172.21.15.2:0/158383890 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96680041d0 con 0x560484dac670 2024-09-10T08:12:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.813+0000 7f966cc00640 1 -- 172.21.15.2:0/158383890 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668011820 con 0x560484dac670 2024-09-10T08:12:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.813+0000 7f966fe00640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f965c0a2c00 0x7f965c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.814+0000 7f966fe00640 1 -- 172.21.15.2:0/3630835169 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f96640cf2e0 con 0x7f9678077040 2024-09-10T08:12:08.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.814+0000 7f9677e00640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f965c0a2c00 0x7f965c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.814+0000 7f9681019740 1 -- 172.21.15.2:0/158383890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 msgr2=0x560484db69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:08.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.814+0000 7f9681019740 1 --2- 172.21.15.2:0/158383890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 0x560484db69f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f966800bd10 tx=0x7f96680107a0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.814+0000 7f9681019740 1 -- 172.21.15.2:0/158383890 shutdown_connections 2024-09-10T08:12:08.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.814+0000 7f9681019740 1 --2- 172.21.15.2:0/158383890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484db9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.814+0000 7f9681019740 1 --2- 172.21.15.2:0/158383890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 0x560484db69f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.814+0000 7f9681019740 1 --2- 172.21.15.2:0/158383890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dabd50 0x560484dac130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.814+0000 7f9681019740 1 -- 172.21.15.2:0/158383890 >> 172.21.15.2:0/158383890 conn(0x560484da82e0 msgr2=0x560484da86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:08.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.815+0000 7f9681019740 1 -- 172.21.15.2:0/158383890 shutdown_connections 2024-09-10T08:12:08.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.815+0000 7f9677e00640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f965c0a2c00 0x7f965c0a50c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9670006010 tx=0x7f9670005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:08.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.815+0000 7f9681019740 1 -- 172.21.15.2:0/158383890 wait complete. 2024-09-10T08:12:08.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.815+0000 7f9681019740 1 -- start start 2024-09-10T08:12:08.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.815+0000 7f9681019740 1 Processor -- start 2024-09-10T08:12:08.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.816+0000 7f9681019740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dabd50 0x560484db3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.816+0000 7f9681019740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484dac670 0x560484db42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.816+0000 7f9681019740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484db6f30 0x560484db47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.816+0000 7f9681019740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560484db9ba0 con 0x560484dac670 2024-09-10T08:12:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.816+0000 7f9681019740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560484ccd0f0 con 0x560484dabd50 2024-09-10T08:12:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.816+0000 7f9681019740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560484db9d20 con 0x560484db6f30 2024-09-10T08:12:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.816+0000 7f9681019740 10 client.0 _ll_register_callbacks cb 0x560484cbe9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.816+0000 7f9677400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dabd50 0x560484db3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.816+0000 7f9677400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dabd50 0x560484db3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46086/0 (socket says 172.21.15.2:46086) 2024-09-10T08:12:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.816+0000 7f9677400640 1 -- 172.21.15.2:0/3078428134 learned_addr learned my addr 172.21.15.2:0/3078428134 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.816+0000 7f9676a00640 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484dac670 0x560484db42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.816+0000 7f9677e00640 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484db6f30 0x560484db47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.817+0000 7f9677400640 1 -- 172.21.15.2:0/3078428134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484db6f30 msgr2=0x560484db47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.817+0000 7f9677400640 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484db6f30 0x560484db47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.817+0000 7f9677400640 1 -- 172.21.15.2:0/3078428134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484dac670 msgr2=0x560484db42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.817+0000 7f9677400640 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484dac670 0x560484db42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.817+0000 7f9677400640 1 -- 172.21.15.2:0/3078428134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560484a833c0 con 0x560484dabd50 2024-09-10T08:12:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.817+0000 7f9676a00640 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484dac670 0x560484db42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.817+0000 7f9677400640 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dabd50 0x560484db3d60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9668010ca0 tx=0x7f9668005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=994f1a5f4b5ee6e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.817+0000 7f9661600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.817+0000 7f9661600640 1 -- 172.21.15.2:0/3078428134 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668029060 con 0x560484dabd50 2024-09-10T08:12:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.817+0000 7f9681019740 1 -- 172.21.15.2:0/3078428134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96680041d0 con 0x560484dabd50 2024-09-10T08:12:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.817+0000 7f9661600640 1 -- 172.21.15.2:0/3078428134 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f96680035f0 con 0x560484dabd50 2024-09-10T08:12:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.818+0000 7f9661600640 1 -- 172.21.15.2:0/3078428134 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9668030050 con 0x560484dabd50 2024-09-10T08:12:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.818+0000 7f9681019740 10 client.26357 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.818+0000 7f9681019740 1 -- 172.21.15.2:0/3078428134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560484dc5740 con 0x560484dabd50 2024-09-10T08:12:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.818+0000 7f9661600640 1 -- 172.21.15.2:0/3078428134 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96680041d0 con 0x560484dabd50 2024-09-10T08:12:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.818+0000 7f9661600640 1 -- 172.21.15.2:0/3078428134 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f96680046c0 con 0x560484dabd50 2024-09-10T08:12:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.819+0000 7f9681019740 10 client.26357 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.819+0000 7f9681019740 10 client.26357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.819+0000 7f9681019740 20 client.26357 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.819+0000 7f9681019740 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560484dc6b50 0x560484dc8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.819+0000 7f9681019740 4 client.26357 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.819+0000 7f9681019740 1 -- 172.21.15.2:0/3078428134 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560484d1f950 con 0x560484dc6b50 2024-09-10T08:12:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.819+0000 7f9676a00640 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560484dc6b50 0x560484dc8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.819+0000 7f9676a00640 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560484dc6b50 0x560484dc8f40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b400e7447ea3931 server_cookie=da2ee33d8fd99891 in_seq=0 out_seq=0 2024-09-10T08:12:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.819+0000 7f9661600640 10 client.26357 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.820+0000 7f9661600640 1 -- 172.21.15.2:0/3078428134 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560484d1f950 con 0x560484dc6b50 2024-09-10T08:12:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.820+0000 7f9661600640 10 client.26357 handle_command_reply: tid=0 2024-09-10T08:12:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.820+0000 7f9681019740 1 client.26357 shutdown 2024-09-10T08:12:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.820+0000 7f9681019740 1 -- 172.21.15.2:0/3078428134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dabd50 msgr2=0x560484db3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.820+0000 7f9681019740 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dabd50 0x560484db3d60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9668010ca0 tx=0x7f9668005cc0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.820+0000 7f9681019740 1 -- 172.21.15.2:0/3078428134 shutdown_connections 2024-09-10T08:12:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.820+0000 7f9681019740 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560484dc6b50 0x560484dc8f40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.820+0000 7f9681019740 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484db6f30 0x560484db47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.820+0000 7f9681019740 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484dac670 0x560484db42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.820+0000 7f9681019740 1 --2- 172.21.15.2:0/3078428134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dabd50 0x560484db3d60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.820+0000 7f9681019740 1 -- 172.21.15.2:0/3078428134 >> 172.21.15.2:0/3078428134 conn(0x560484da82e0 msgr2=0x560484dadfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.821+0000 7f9661600640 0 client.26357 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.821+0000 7f9681019740 1 -- 172.21.15.2:0/3078428134 shutdown_connections 2024-09-10T08:12:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.821+0000 7f9681019740 1 -- 172.21.15.2:0/3078428134 wait complete. 2024-09-10T08:12:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.821+0000 7f9681019740 20 client.26357 trim_cache size 0 max 16384 2024-09-10T08:12:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.839+0000 7f9681019740 1 Processor -- start 2024-09-10T08:12:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.839+0000 7f9681019740 1 -- start start 2024-09-10T08:12:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.839+0000 7f9681019740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dabd50 0x560484db2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.839+0000 7f9681019740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dac670 0x560484dce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.839+0000 7f9681019740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484dceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.839+0000 7f9681019740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96781b7b90 con 0x560484db6f30 2024-09-10T08:12:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.839+0000 7f9681019740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96781b5460 con 0x560484dabd50 2024-09-10T08:12:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.839+0000 7f9681019740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96781b32b0 con 0x560484dac670 2024-09-10T08:12:08.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.839+0000 7f9677e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484dceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.839+0000 7f9677400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dabd50 0x560484db2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.839+0000 7f9677e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484dceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34992/0 (socket says 172.21.15.2:34992) 2024-09-10T08:12:08.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.840+0000 7f9677e00640 1 -- 172.21.15.2:0/1527387208 learned_addr learned my addr 172.21.15.2:0/1527387208 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:08.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.840+0000 7f9676a00640 1 --2- 172.21.15.2:0/1527387208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dac670 0x560484dce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.840+0000 7f9677400640 1 -- 172.21.15.2:0/1527387208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dac670 msgr2=0x560484dce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:08.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.840+0000 7f9677400640 1 --2- 172.21.15.2:0/1527387208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dac670 0x560484dce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.840+0000 7f9677400640 1 -- 172.21.15.2:0/1527387208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 msgr2=0x560484dceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:08.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.840+0000 7f9677400640 1 --2- 172.21.15.2:0/1527387208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484dceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.840+0000 7f9677400640 1 -- 172.21.15.2:0/1527387208 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560484da9470 con 0x560484dabd50 2024-09-10T08:12:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.840+0000 7f9677400640 1 --2- 172.21.15.2:0/1527387208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dabd50 0x560484db2fe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9668004ac0 tx=0x7f9668010e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76286ed9d86b5a76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.840+0000 7f9663e00640 1 -- 172.21.15.2:0/1527387208 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668029040 con 0x560484dabd50 2024-09-10T08:12:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.840+0000 7f9663e00640 1 -- 172.21.15.2:0/1527387208 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96680041d0 con 0x560484dabd50 2024-09-10T08:12:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.840+0000 7f9663e00640 1 -- 172.21.15.2:0/1527387208 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668004f20 con 0x560484dabd50 2024-09-10T08:12:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.841+0000 7f9681019740 1 -- 172.21.15.2:0/1527387208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dabd50 msgr2=0x560484db2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.841+0000 7f9681019740 1 --2- 172.21.15.2:0/1527387208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dabd50 0x560484db2fe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9668004ac0 tx=0x7f9668010e10 comp rx=0 tx=0).stop 2024-09-10T08:12:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.841+0000 7f9681019740 1 -- 172.21.15.2:0/1527387208 shutdown_connections 2024-09-10T08:12:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.841+0000 7f9681019740 1 --2- 172.21.15.2:0/1527387208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484dceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.841+0000 7f9681019740 1 --2- 172.21.15.2:0/1527387208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dac670 0x560484dce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.841+0000 7f9681019740 1 --2- 172.21.15.2:0/1527387208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dabd50 0x560484db2fe0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.841+0000 7f9681019740 1 -- 172.21.15.2:0/1527387208 >> 172.21.15.2:0/1527387208 conn(0x560484da82e0 msgr2=0x560484db5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.841+0000 7f9681019740 1 -- 172.21.15.2:0/1527387208 shutdown_connections 2024-09-10T08:12:08.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.841+0000 7f9681019740 1 -- 172.21.15.2:0/1527387208 wait complete. 2024-09-10T08:12:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.842+0000 7f9681019740 1 -- start start 2024-09-10T08:12:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.842+0000 7f9681019740 1 Processor -- start 2024-09-10T08:12:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.842+0000 7f9681019740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dabd50 0x560484dbefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.842+0000 7f9681019740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 0x560484dbf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.842+0000 7f9681019740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484db0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.842+0000 7f9681019740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96781b3920 con 0x560484db6f30 2024-09-10T08:12:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.842+0000 7f9681019740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96781b4d90 con 0x560484dac670 2024-09-10T08:12:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.842+0000 7f9681019740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96781b3ff0 con 0x560484dabd50 2024-09-10T08:12:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.842+0000 7f9681019740 10 client.0 _ll_register_callbacks cb 0x7f966800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:08.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9676a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 0x560484dbf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9677400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dabd50 0x560484dbefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9676a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 0x560484dbf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46104/0 (socket says 172.21.15.2:46104) 2024-09-10T08:12:08.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9676a00640 1 -- 172.21.15.2:0/227104321 learned_addr learned my addr 172.21.15.2:0/227104321 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:08.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9677e00640 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484db0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9676a00640 1 -- 172.21.15.2:0/227104321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dabd50 msgr2=0x560484dbefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:08.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9676a00640 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dabd50 0x560484dbefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9676a00640 1 -- 172.21.15.2:0/227104321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 msgr2=0x560484db0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:08.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9676a00640 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484db0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9676a00640 1 -- 172.21.15.2:0/227104321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560484db06d0 con 0x560484dac670 2024-09-10T08:12:08.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9677e00640 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484db0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:08.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9677400640 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dabd50 0x560484dbefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:08.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9676a00640 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 0x560484dbf4e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f966400c6a0 tx=0x7f9664086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38db78e7be5068af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:08.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9661600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:08.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.843+0000 7f9661600640 1 -- 172.21.15.2:0/227104321 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96640b3070 con 0x560484dac670 2024-09-10T08:12:08.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.844+0000 7f9661600640 1 -- 172.21.15.2:0/227104321 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f96640cca10 con 0x560484dac670 2024-09-10T08:12:08.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.844+0000 7f9681019740 1 -- 172.21.15.2:0/227104321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96680041d0 con 0x560484dac670 2024-09-10T08:12:08.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.844+0000 7f9661600640 1 -- 172.21.15.2:0/227104321 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96640cde60 con 0x560484dac670 2024-09-10T08:12:08.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.844+0000 7f9681019740 10 client.26363 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:08.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.844+0000 7f9681019740 1 -- 172.21.15.2:0/227104321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560484e30c90 con 0x560484dac670 2024-09-10T08:12:08.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.844+0000 7f9661600640 1 -- 172.21.15.2:0/227104321 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96680041d0 con 0x560484dac670 2024-09-10T08:12:08.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.844+0000 7f9661600640 1 -- 172.21.15.2:0/227104321 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9664097d60 con 0x560484dac670 2024-09-10T08:12:08.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.844+0000 7f9681019740 10 client.26363 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:08.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.844+0000 7f9681019740 10 client.26363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:08.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.844+0000 7f9681019740 20 client.26363 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:08.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.844+0000 7f9681019740 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560484e313c0 0x560484e337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:08.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.844+0000 7f9681019740 4 client.26363 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:12:08.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.844+0000 7f9681019740 1 -- 172.21.15.2:0/227104321 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560484d19e60 con 0x560484e313c0 2024-09-10T08:12:08.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.845+0000 7f9677400640 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560484e313c0 0x560484e337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:08.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.845+0000 7f9677400640 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560484e313c0 0x560484e337e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf088d58b8d089ef server_cookie=3bd51179d3893bf0 in_seq=0 out_seq=0 2024-09-10T08:12:08.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.845+0000 7f9661600640 10 client.26363 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:08.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.845+0000 7f9661600640 1 -- 172.21.15.2:0/227104321 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560484d19e60 con 0x560484e313c0 2024-09-10T08:12:08.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.845+0000 7f9661600640 10 client.26363 handle_command_reply: tid=0 2024-09-10T08:12:08.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.845+0000 7f9681019740 1 client.26363 shutdown 2024-09-10T08:12:08.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f9681019740 1 -- 172.21.15.2:0/227104321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 msgr2=0x560484dbf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:08.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f9681019740 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 0x560484dbf4e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f966400c6a0 tx=0x7f9664086040 comp rx=0 tx=0).stop 2024-09-10T08:12:08.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f9681019740 1 -- 172.21.15.2:0/227104321 shutdown_connections 2024-09-10T08:12:08.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f9681019740 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560484e313c0 0x560484e337e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f9681019740 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560484db6f30 0x560484db0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f9681019740 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560484dac670 0x560484dbf4e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f9681019740 1 --2- 172.21.15.2:0/227104321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560484dabd50 0x560484dbefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f9681019740 1 -- 172.21.15.2:0/227104321 >> 172.21.15.2:0/227104321 conn(0x560484da82e0 msgr2=0x560484da6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:08.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f9661600640 0 client.26363 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:08.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f9681019740 1 -- 172.21.15.2:0/227104321 shutdown_connections 2024-09-10T08:12:08.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f9681019740 1 -- 172.21.15.2:0/227104321 wait complete. 2024-09-10T08:12:08.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f9681019740 20 client.26363 trim_cache size 0 max 16384 2024-09-10T08:12:08.850 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:12:08.850 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:12:08.851 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:12:08.851 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:12:08.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f967d000640 1 -- 172.21.15.2:0/3630835169 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f965c0a2c00 msgr2=0x7f965c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:08.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.846+0000 7f967d000640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f965c0a2c00 0x7f965c0a50c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9670006010 tx=0x7f9670005d80 comp rx=0 tx=0).stop 2024-09-10T08:12:08.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.847+0000 7f967d000640 1 -- 172.21.15.2:0/3630835169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9678077040 msgr2=0x7f96781a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:08.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.847+0000 7f967d000640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9678077040 0x7f96781a4bb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f966400e9f0 tx=0x7f966400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.847+0000 7f967d000640 1 -- 172.21.15.2:0/3630835169 shutdown_connections 2024-09-10T08:12:08.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.847+0000 7f967d000640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9678112860 0x7f967819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.847+0000 7f967d000640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9678077040 0x7f96781a4bb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.847+0000 7f967d000640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f965c0a2c00 0x7f965c0a50c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.847+0000 7f967d000640 1 --2- 172.21.15.2:0/3630835169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9678076720 0x7f96781a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:08.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.847+0000 7f967d000640 1 -- 172.21.15.2:0/3630835169 >> 172.21.15.2:0/3630835169 conn(0x7f96780ff880 msgr2=0x7f9678113e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:08.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.847+0000 7f967d000640 1 -- 172.21.15.2:0/3630835169 shutdown_connections 2024-09-10T08:12:08.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:08.847+0000 7f967d000640 1 -- 172.21.15.2:0/3630835169 wait complete. 2024-09-10T08:12:09.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:09 smithi002 ceph-mon[30563]: pgmap v507: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:12:09.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:09 smithi155 ceph-mon[35405]: pgmap v507: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:12:09.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:09 smithi107 ceph-mon[35048]: pgmap v507: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:12:09.718 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:12:09.719 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:12:10.143 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:10.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.753+0000 7f141e000640 1 -- 172.21.15.2:0/214341763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f141810c170 msgr2=0x7f141810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:10.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.753+0000 7f141e000640 1 --2- 172.21.15.2:0/214341763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f141810c170 0x7f141810e610 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1410009ca0 tx=0x7f141002a950 comp rx=0 tx=0).stop 2024-09-10T08:12:10.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.753+0000 7f141e000640 1 -- 172.21.15.2:0/214341763 shutdown_connections 2024-09-10T08:12:10.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.753+0000 7f141e000640 1 --2- 172.21.15.2:0/214341763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f141810c170 0x7f141810e610 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:10.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.753+0000 7f141e000640 1 --2- 172.21.15.2:0/214341763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1418103660 0x7f141810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:10.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.753+0000 7f141e000640 1 --2- 172.21.15.2:0/214341763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1418102d40 0x7f1418103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:10.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.753+0000 7f141e000640 1 -- 172.21.15.2:0/214341763 >> 172.21.15.2:0/214341763 conn(0x7f14180fc7a0 msgr2=0x7f14180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:10.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.753+0000 7f141e000640 1 -- 172.21.15.2:0/214341763 shutdown_connections 2024-09-10T08:12:10.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.753+0000 7f141e000640 1 -- 172.21.15.2:0/214341763 wait complete. 2024-09-10T08:12:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.754+0000 7f141e000640 1 Processor -- start 2024-09-10T08:12:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.754+0000 7f141e000640 1 -- start start 2024-09-10T08:12:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.755+0000 7f141e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1418102d40 0x7f141819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.755+0000 7f141e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1418103660 0x7f141819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.755+0000 7f141e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f141810c170 0x7f141819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.755+0000 7f141e000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1418111f80 con 0x7f141810c170 2024-09-10T08:12:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.755+0000 7f141e000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1418111e00 con 0x7f1418102d40 2024-09-10T08:12:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.755+0000 7f141e000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1418112100 con 0x7f1418103660 2024-09-10T08:12:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.755+0000 7f1417400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1418102d40 0x7f141819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:10.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.755+0000 7f1417e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f141810c170 0x7f141819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:10.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.755+0000 7f1417400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1418102d40 0x7f141819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46116/0 (socket says 172.21.15.2:46116) 2024-09-10T08:12:10.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.755+0000 7f1417e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f141810c170 0x7f141819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35016/0 (socket says 172.21.15.2:35016) 2024-09-10T08:12:10.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.755+0000 7f1417400640 1 -- 172.21.15.2:0/2867778080 learned_addr learned my addr 172.21.15.2:0/2867778080 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:10.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.755+0000 7f140fe00640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1418103660 0x7f141819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.756+0000 7f1417400640 1 -- 172.21.15.2:0/2867778080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1418103660 msgr2=0x7f141819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.756+0000 7f1417400640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1418103660 0x7f141819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.756+0000 7f1417400640 1 -- 172.21.15.2:0/2867778080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f141810c170 msgr2=0x7f141819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.756+0000 7f1417400640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f141810c170 0x7f141819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.756+0000 7f1417400640 1 -- 172.21.15.2:0/2867778080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141819f5a0 con 0x7f1418102d40 2024-09-10T08:12:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.756+0000 7f1417e00640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f141810c170 0x7f141819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.756+0000 7f140fe00640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1418103660 0x7f141819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.756+0000 7f1417400640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1418102d40 0x7f141819a5f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f140800e9f0 tx=0x7f140800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.756+0000 7f1414c00640 1 -- 172.21.15.2:0/2867778080 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140800cd80 con 0x7f1418102d40 2024-09-10T08:12:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.756+0000 7f141e000640 1 -- 172.21.15.2:0/2867778080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141819f890 con 0x7f1418102d40 2024-09-10T08:12:10.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.756+0000 7f1414c00640 1 -- 172.21.15.2:0/2867778080 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1408010910 con 0x7f1418102d40 2024-09-10T08:12:10.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.756+0000 7f1414c00640 1 -- 172.21.15.2:0/2867778080 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14080188f0 con 0x7f1418102d40 2024-09-10T08:12:10.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.756+0000 7f141e000640 1 -- 172.21.15.2:0/2867778080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14181a7170 con 0x7f1418102d40 2024-09-10T08:12:10.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.757+0000 7f141e000640 1 -- 172.21.15.2:0/2867778080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1418103120 con 0x7f1418102d40 2024-09-10T08:12:10.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.759+0000 7f1414c00640 1 -- 172.21.15.2:0/2867778080 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1408010ac0 con 0x7f1418102d40 2024-09-10T08:12:10.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.761+0000 7f1414c00640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13fc0a2b80 0x7f13fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:10.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.761+0000 7f1414c00640 1 -- 172.21.15.2:0/2867778080 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f14080cfd80 con 0x7f1418102d40 2024-09-10T08:12:10.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.761+0000 7f140fe00640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13fc0a2b80 0x7f13fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:10.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.762+0000 7f140fe00640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13fc0a2b80 0x7f13fc0a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f141810ead0 tx=0x7f140000b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:10.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.763+0000 7f1414c00640 1 -- 172.21.15.2:0/2867778080 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f140809c240 con 0x7f1418102d40 2024-09-10T08:12:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.988+0000 7f141e000640 1 -- 172.21.15.2:0/2867778080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14181a7370 con 0x7f1418102d40 2024-09-10T08:12:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.989+0000 7f1414c00640 1 -- 172.21.15.2:0/2867778080 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f14080a0540 con 0x7f1418102d40 2024-09-10T08:12:10.990 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:12:10.990 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:12:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:12:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.993+0000 7f141e000640 1 -- 172.21.15.2:0/2867778080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13fc0a2b80 msgr2=0x7f13fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.993+0000 7f141e000640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13fc0a2b80 0x7f13fc0a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f141810ead0 tx=0x7f140000b2e0 comp rx=0 tx=0).stop 2024-09-10T08:12:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.993+0000 7f141e000640 1 -- 172.21.15.2:0/2867778080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1418102d40 msgr2=0x7f141819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.993+0000 7f141e000640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1418102d40 0x7f141819a5f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f140800e9f0 tx=0x7f140800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:12:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.993+0000 7f141e000640 1 -- 172.21.15.2:0/2867778080 shutdown_connections 2024-09-10T08:12:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.993+0000 7f141e000640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f141810c170 0x7f141819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.993+0000 7f141e000640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13fc0a2b80 0x7f13fc0a5040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.993+0000 7f141e000640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1418103660 0x7f141819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.993+0000 7f141e000640 1 --2- 172.21.15.2:0/2867778080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1418102d40 0x7f141819a5f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.993+0000 7f141e000640 1 -- 172.21.15.2:0/2867778080 >> 172.21.15.2:0/2867778080 conn(0x7f14180fc7a0 msgr2=0x7f141810d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.993+0000 7f141e000640 1 -- 172.21.15.2:0/2867778080 shutdown_connections 2024-09-10T08:12:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:10.993+0000 7f141e000640 1 -- 172.21.15.2:0/2867778080 wait complete. 2024-09-10T08:12:11.086 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T08:12:11.086 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T08:12:11.087 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:12:11.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:11 smithi155 ceph-mon[35405]: pgmap v508: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:12:11.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:11 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2867778080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:11.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:11 smithi107 ceph-mon[35048]: pgmap v508: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:12:11.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:11 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2867778080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:11 smithi002 ceph-mon[30563]: pgmap v508: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:12:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:11 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2867778080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:11.956 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:12:12.005 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:12.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.665+0000 7f1d49c00640 1 -- 172.21.15.2:0/3068612980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d441079c0 msgr2=0x7f1d44107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:12.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.665+0000 7f1d49c00640 1 --2- 172.21.15.2:0/3068612980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d441079c0 0x7f1d44107da0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009930 tx=0x7f1d3402a420 comp rx=0 tx=0).stop 2024-09-10T08:12:12.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.665+0000 7f1d49c00640 1 -- 172.21.15.2:0/3068612980 shutdown_connections 2024-09-10T08:12:12.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.665+0000 7f1d49c00640 1 --2- 172.21.15.2:0/3068612980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d44102360 0x7f1d4410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:12.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.665+0000 7f1d49c00640 1 --2- 172.21.15.2:0/3068612980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d441019c0 0x7f1d44101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:12.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.665+0000 7f1d49c00640 1 --2- 172.21.15.2:0/3068612980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d441079c0 0x7f1d44107da0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:12.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.666+0000 7f1d49c00640 1 -- 172.21.15.2:0/3068612980 >> 172.21.15.2:0/3068612980 conn(0x7f1d440fd740 msgr2=0x7f1d440ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:12.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.666+0000 7f1d49c00640 1 -- 172.21.15.2:0/3068612980 shutdown_connections 2024-09-10T08:12:12.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.666+0000 7f1d49c00640 1 -- 172.21.15.2:0/3068612980 wait complete. 2024-09-10T08:12:12.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.667+0000 7f1d49c00640 1 Processor -- start 2024-09-10T08:12:12.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.667+0000 7f1d49c00640 1 -- start start 2024-09-10T08:12:12.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.668+0000 7f1d49c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d441019c0 0x7f1d4419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:12.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.668+0000 7f1d49c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d44102360 0x7f1d4419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:12.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.668+0000 7f1d49c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d441079c0 0x7f1d4419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:12.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.668+0000 7f1d49c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44111e10 con 0x7f1d441019c0 2024-09-10T08:12:12.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.668+0000 7f1d49c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44111c90 con 0x7f1d441079c0 2024-09-10T08:12:12.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.668+0000 7f1d49c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44111f90 con 0x7f1d44102360 2024-09-10T08:12:12.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.668+0000 7f1d43e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d441019c0 0x7f1d4419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:12.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.668+0000 7f1d43e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d441019c0 0x7f1d4419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35044/0 (socket says 172.21.15.2:35044) 2024-09-10T08:12:12.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.668+0000 7f1d43e00640 1 -- 172.21.15.2:0/2385169766 learned_addr learned my addr 172.21.15.2:0/2385169766 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:12.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.668+0000 7f1d49200640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d441079c0 0x7f1d4419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:12.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.669+0000 7f1d43e00640 1 -- 172.21.15.2:0/2385169766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d44102360 msgr2=0x7f1d4419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:12.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.669+0000 7f1d43e00640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d44102360 0x7f1d4419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:12.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.669+0000 7f1d43e00640 1 -- 172.21.15.2:0/2385169766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d441079c0 msgr2=0x7f1d4419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:12.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.669+0000 7f1d43e00640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d441079c0 0x7f1d4419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:12.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.669+0000 7f1d43e00640 1 -- 172.21.15.2:0/2385169766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4419f4a0 con 0x7f1d441019c0 2024-09-10T08:12:12.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.669+0000 7f1d49200640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d441079c0 0x7f1d4419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:12.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.669+0000 7f1d43e00640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d441019c0 0x7f1d4419a4f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1d3402a940 tx=0x7f1d34003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:12.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.669+0000 7f1d40c00640 1 -- 172.21.15.2:0/2385169766 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3403f070 con 0x7f1d441019c0 2024-09-10T08:12:12.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.669+0000 7f1d40c00640 1 -- 172.21.15.2:0/2385169766 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d34031c90 con 0x7f1d441019c0 2024-09-10T08:12:12.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.669+0000 7f1d49c00640 1 -- 172.21.15.2:0/2385169766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4419f730 con 0x7f1d441019c0 2024-09-10T08:12:12.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.670+0000 7f1d40c00640 1 -- 172.21.15.2:0/2385169766 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34035030 con 0x7f1d441019c0 2024-09-10T08:12:12.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.670+0000 7f1d49c00640 1 -- 172.21.15.2:0/2385169766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d441a7070 con 0x7f1d441019c0 2024-09-10T08:12:12.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.671+0000 7f1d49c00640 1 -- 172.21.15.2:0/2385169766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d44101e20 con 0x7f1d441019c0 2024-09-10T08:12:12.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.673+0000 7f1d40c00640 1 -- 172.21.15.2:0/2385169766 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1d34042070 con 0x7f1d441019c0 2024-09-10T08:12:12.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.674+0000 7f1d40c00640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d280a2c00 0x7f1d280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:12.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.674+0000 7f1d40c00640 1 -- 172.21.15.2:0/2385169766 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1d340efb50 con 0x7f1d441019c0 2024-09-10T08:12:12.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.674+0000 7f1d43400640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d280a2c00 0x7f1d280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:12.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.675+0000 7f1d43400640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d280a2c00 0x7f1d280a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1d300058c0 tx=0x7f1d30005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:12.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.680+0000 7f1d40c00640 1 -- 172.21.15.2:0/2385169766 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d340bc660 con 0x7f1d441019c0 2024-09-10T08:12:12.727 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:12.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.927+0000 7f1d49c00640 1 -- 172.21.15.2:0/2385169766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d441a7240 con 0x7f1d441019c0 2024-09-10T08:12:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.928+0000 7f1d40c00640 1 -- 172.21.15.2:0/2385169766 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f1d340bbf60 con 0x7f1d441019c0 2024-09-10T08:12:12.928 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:12:12.929 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:12:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:12:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.930+0000 7f1d49c00640 1 -- 172.21.15.2:0/2385169766 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d280a2c00 msgr2=0x7f1d280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.930+0000 7f1d49c00640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d280a2c00 0x7f1d280a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1d300058c0 tx=0x7f1d30005850 comp rx=0 tx=0).stop 2024-09-10T08:12:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.930+0000 7f1d49c00640 1 -- 172.21.15.2:0/2385169766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d441019c0 msgr2=0x7f1d4419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.930+0000 7f1d49c00640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d441019c0 0x7f1d4419a4f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1d3402a940 tx=0x7f1d34003b90 comp rx=0 tx=0).stop 2024-09-10T08:12:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.931+0000 7f1d49c00640 1 -- 172.21.15.2:0/2385169766 shutdown_connections 2024-09-10T08:12:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.931+0000 7f1d49c00640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d441079c0 0x7f1d4419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.931+0000 7f1d49c00640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d280a2c00 0x7f1d280a50c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.931+0000 7f1d49c00640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d44102360 0x7f1d4419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.931+0000 7f1d49c00640 1 --2- 172.21.15.2:0/2385169766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d441019c0 0x7f1d4419a4f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.931+0000 7f1d49c00640 1 -- 172.21.15.2:0/2385169766 >> 172.21.15.2:0/2385169766 conn(0x7f1d440fd740 msgr2=0x7f1d440fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.931+0000 7f1d49c00640 1 -- 172.21.15.2:0/2385169766 shutdown_connections 2024-09-10T08:12:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:12.931+0000 7f1d49c00640 1 -- 172.21.15.2:0/2385169766 wait complete. 2024-09-10T08:12:13.538 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:13 smithi002 ceph-mon[30563]: pgmap v509: 129 pgs: 129 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:12:13.538 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:13 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2385169766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:13.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.557+0000 7f047b200640 1 -- 172.21.15.2:0/2153154884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f047c1047e0 msgr2=0x7f047c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:13.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.557+0000 7f047b200640 1 --2- 172.21.15.2:0/2153154884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f047c1047e0 0x7f047c108830 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0468009860 tx=0x7f046802a340 comp rx=0 tx=0).stop 2024-09-10T08:12:13.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.557+0000 7f047b200640 1 -- 172.21.15.2:0/2153154884 shutdown_connections 2024-09-10T08:12:13.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.557+0000 7f047b200640 1 --2- 172.21.15.2:0/2153154884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c108f60 0x7f047c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.557+0000 7f047b200640 1 --2- 172.21.15.2:0/2153154884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f047c1047e0 0x7f047c108830 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.557+0000 7f047b200640 1 --2- 172.21.15.2:0/2153154884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f047c103e30 0x7f047c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.557+0000 7f047b200640 1 -- 172.21.15.2:0/2153154884 >> 172.21.15.2:0/2153154884 conn(0x7f047c0ff8f0 msgr2=0x7f047c101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:13.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.557+0000 7f047b200640 1 -- 172.21.15.2:0/2153154884 shutdown_connections 2024-09-10T08:12:13.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.557+0000 7f047b200640 1 -- 172.21.15.2:0/2153154884 wait complete. 2024-09-10T08:12:13.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.558+0000 7f047b200640 1 Processor -- start 2024-09-10T08:12:13.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.558+0000 7f047b200640 1 -- start start 2024-09-10T08:12:13.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.559+0000 7f047b200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c103e30 0x7f047c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.559+0000 7f047b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f047c1047e0 0x7f047c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.559+0000 7f047b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f047c108f60 0x7f047c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.559+0000 7f047b200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f047c113fc0 con 0x7f047c103e30 2024-09-10T08:12:13.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.559+0000 7f047b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f047c113e40 con 0x7f047c108f60 2024-09-10T08:12:13.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.559+0000 7f047b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f047c114140 con 0x7f047c1047e0 2024-09-10T08:12:13.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.559+0000 7f0479e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c103e30 0x7f047c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.559+0000 7f0479e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c103e30 0x7f047c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35070/0 (socket says 172.21.15.2:35070) 2024-09-10T08:12:13.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.559+0000 7f0479e00640 1 -- 172.21.15.2:0/2849705348 learned_addr learned my addr 172.21.15.2:0/2849705348 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:13.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.559+0000 7f047a800640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f047c108f60 0x7f047c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.559+0000 7f0479400640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f047c1047e0 0x7f047c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.559+0000 7f0479e00640 1 -- 172.21.15.2:0/2849705348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f047c1047e0 msgr2=0x7f047c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:13.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.559+0000 7f0479e00640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f047c1047e0 0x7f047c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.560+0000 7f0479e00640 1 -- 172.21.15.2:0/2849705348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f047c108f60 msgr2=0x7f047c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:13.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.560+0000 7f0479e00640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f047c108f60 0x7f047c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.560+0000 7f0479e00640 1 -- 172.21.15.2:0/2849705348 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c19f450 con 0x7f047c103e30 2024-09-10T08:12:13.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.560+0000 7f047a800640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f047c108f60 0x7f047c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:13.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.560+0000 7f0479e00640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c103e30 0x7f047c19a4a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f046c00e9f0 tx=0x7f046c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:13.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.560+0000 7f0472a00640 1 -- 172.21.15.2:0/2849705348 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c00cd80 con 0x7f047c103e30 2024-09-10T08:12:13.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.560+0000 7f0472a00640 1 -- 172.21.15.2:0/2849705348 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f046c010910 con 0x7f047c103e30 2024-09-10T08:12:13.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.560+0000 7f047b200640 1 -- 172.21.15.2:0/2849705348 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047c19f740 con 0x7f047c103e30 2024-09-10T08:12:13.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.560+0000 7f0472a00640 1 -- 172.21.15.2:0/2849705348 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c018990 con 0x7f047c103e30 2024-09-10T08:12:13.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.561+0000 7f047b200640 1 -- 172.21.15.2:0/2849705348 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f047c1a7020 con 0x7f047c103e30 2024-09-10T08:12:13.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.564+0000 7f0483dc5740 1 Processor -- start 2024-09-10T08:12:13.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.564+0000 7f0472a00640 1 -- 172.21.15.2:0/2849705348 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f046c010ac0 con 0x7f047c103e30 2024-09-10T08:12:13.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.564+0000 7f0483dc5740 1 -- start start 2024-09-10T08:12:13.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.564+0000 7f0483dc5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2d5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.564+0000 7f0483dc5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2dfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.564+0000 7f0483dc5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 0x560f5a2e2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.564+0000 7f0483dc5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560f5a23b3e0 con 0x560f5a2d4e40 2024-09-10T08:12:13.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.564+0000 7f0483dc5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f5a1e2d80 con 0x560f5a2d5760 2024-09-10T08:12:13.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.564+0000 7f0483dc5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560f5a2373a0 con 0x560f5a2e0020 2024-09-10T08:12:13.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.565+0000 7f047a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2d5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.565+0000 7f0479e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2dfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.565+0000 7f0479400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 0x560f5a2e2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.565+0000 7f047a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2d5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35076/0 (socket says 172.21.15.2:35076) 2024-09-10T08:12:13.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.565+0000 7f047a800640 1 -- 172.21.15.2:0/417040029 learned_addr learned my addr 172.21.15.2:0/417040029 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:13.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.565+0000 7f0479e00640 1 -- 172.21.15.2:0/417040029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 msgr2=0x560f5a2e2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:13.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.565+0000 7f0479e00640 1 --2- 172.21.15.2:0/417040029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 0x560f5a2e2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.565+0000 7f0479e00640 1 -- 172.21.15.2:0/417040029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 msgr2=0x560f5a2d5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:13.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.565+0000 7f0479e00640 1 --2- 172.21.15.2:0/417040029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2d5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.565+0000 7f0479e00640 1 -- 172.21.15.2:0/417040029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f5a201ae0 con 0x560f5a2d5760 2024-09-10T08:12:13.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.565+0000 7f047a800640 1 --2- 172.21.15.2:0/417040029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2d5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:13.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.565+0000 7f0479e00640 1 --2- 172.21.15.2:0/417040029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2dfae0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f046c0ccd80 tx=0x7f046c0d7690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f384308f092b5ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:13.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.566+0000 7f0472a00640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04600a2c00 0x7f04600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.566+0000 7f047a800640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04600a2c00 0x7f04600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.566+0000 7f0467400640 1 -- 172.21.15.2:0/417040029 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c0de070 con 0x560f5a2d5760 2024-09-10T08:12:13.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.566+0000 7f0472a00640 1 -- 172.21.15.2:0/2849705348 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f046c0cf6a0 con 0x7f047c103e30 2024-09-10T08:12:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.566+0000 7f0467400640 1 -- 172.21.15.2:0/417040029 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f046c09fec0 con 0x560f5a2d5760 2024-09-10T08:12:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.566+0000 7f0467400640 1 -- 172.21.15.2:0/417040029 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c09b990 con 0x560f5a2d5760 2024-09-10T08:12:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.567+0000 7f0483dc5740 1 -- 172.21.15.2:0/417040029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 msgr2=0x560f5a2dfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.567+0000 7f047a800640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04600a2c00 0x7f04600a50c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0474005c40 tx=0x7f04740095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.567+0000 7f0483dc5740 1 --2- 172.21.15.2:0/417040029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2dfae0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f046c0ccd80 tx=0x7f046c0d7690 comp rx=0 tx=0).stop 2024-09-10T08:12:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.567+0000 7f0483dc5740 1 -- 172.21.15.2:0/417040029 shutdown_connections 2024-09-10T08:12:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.567+0000 7f0483dc5740 1 --2- 172.21.15.2:0/417040029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 0x560f5a2e2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.567+0000 7f0483dc5740 1 --2- 172.21.15.2:0/417040029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2dfae0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.567+0000 7f0483dc5740 1 --2- 172.21.15.2:0/417040029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2d5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.567+0000 7f0483dc5740 1 -- 172.21.15.2:0/417040029 >> 172.21.15.2:0/417040029 conn(0x560f5a2d13d0 msgr2=0x560f5a2d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.567+0000 7f0483dc5740 1 -- 172.21.15.2:0/417040029 shutdown_connections 2024-09-10T08:12:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.567+0000 7f0483dc5740 1 -- 172.21.15.2:0/417040029 wait complete. 2024-09-10T08:12:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.568+0000 7f0483dc5740 1 -- start start 2024-09-10T08:12:13.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.568+0000 7f0483dc5740 1 Processor -- start 2024-09-10T08:12:13.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.568+0000 7f0483dc5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.568+0000 7f0483dc5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2dd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.568+0000 7f0483dc5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 0x560f5a2dd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.568+0000 7f0483dc5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560f5a2e2bf0 con 0x560f5a2d4e40 2024-09-10T08:12:13.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.568+0000 7f0483dc5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f5a1f60f0 con 0x560f5a2d5760 2024-09-10T08:12:13.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.568+0000 7f0483dc5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560f5a2e2d70 con 0x560f5a2e0020 2024-09-10T08:12:13.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.568+0000 7f0483dc5740 10 client.0 _ll_register_callbacks cb 0x560f5a1ddbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:13.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.569+0000 7f0479400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2dd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.569+0000 7f0479e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.569+0000 7f0479400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2dd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46188/0 (socket says 172.21.15.2:46188) 2024-09-10T08:12:13.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.569+0000 7f0479400640 1 -- 172.21.15.2:0/1985816309 learned_addr learned my addr 172.21.15.2:0/1985816309 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:13.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.569+0000 7f047a800640 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 0x560f5a2dd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.569+0000 7f0479400640 1 -- 172.21.15.2:0/1985816309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 msgr2=0x560f5a2dd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:13.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.569+0000 7f0479400640 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 0x560f5a2dd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.569+0000 7f0479400640 1 -- 172.21.15.2:0/1985816309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 msgr2=0x560f5a2dcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:13.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.569+0000 7f0479400640 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2dcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.569+0000 7f0479400640 1 -- 172.21.15.2:0/1985816309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f59fac3c0 con 0x560f5a2d5760 2024-09-10T08:12:13.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.569+0000 7f0479e00640 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2dcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:13.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.569+0000 7f047a800640 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 0x560f5a2dd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:13.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.570+0000 7f0479400640 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2dd490 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0468009780 tx=0x7f0468030340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1af568ed5fd99e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:13.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.570+0000 7f045be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:13.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.570+0000 7f045be00640 1 -- 172.21.15.2:0/1985816309 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468037040 con 0x560f5a2d5760 2024-09-10T08:12:13.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.570+0000 7f0483dc5740 1 -- 172.21.15.2:0/1985816309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f046c09fec0 con 0x560f5a2d5760 2024-09-10T08:12:13.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.570+0000 7f045be00640 1 -- 172.21.15.2:0/1985816309 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0468030550 con 0x560f5a2d5760 2024-09-10T08:12:13.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.570+0000 7f045be00640 1 -- 172.21.15.2:0/1985816309 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f046804c050 con 0x560f5a2d5760 2024-09-10T08:12:13.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.570+0000 7f0483dc5740 10 client.26387 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:13.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.570+0000 7f0483dc5740 1 -- 172.21.15.2:0/1985816309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f5a2eecd0 con 0x560f5a2d5760 2024-09-10T08:12:13.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.570+0000 7f045be00640 1 -- 172.21.15.2:0/1985816309 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f046c09fec0 con 0x560f5a2d5760 2024-09-10T08:12:13.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.571+0000 7f045be00640 1 -- 172.21.15.2:0/1985816309 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f046803d450 con 0x560f5a2d5760 2024-09-10T08:12:13.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.571+0000 7f0483dc5740 10 client.26387 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:13.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.571+0000 7f0483dc5740 10 client.26387 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:12:13.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.571+0000 7f0483dc5740 20 client.26387 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:13.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.571+0000 7f0483dc5740 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x560f5a2f0190 0x560f5a2f2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.571+0000 7f0483dc5740 4 client.26387 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:13.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.571+0000 7f0483dc5740 1 -- 172.21.15.2:0/1985816309 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f5a248950 con 0x560f5a2f0190 2024-09-10T08:12:13.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.571+0000 7f0479e00640 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x560f5a2f0190 0x560f5a2f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.572+0000 7f0479e00640 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x560f5a2f0190 0x560f5a2f2610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79ab6f5fe58e16cc server_cookie=1ee26ddfcbadc415 in_seq=0 out_seq=0 2024-09-10T08:12:13.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.572+0000 7f045be00640 10 client.26387 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:12:13.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.572+0000 7f045be00640 1 -- 172.21.15.2:0/1985816309 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f5a248950 con 0x560f5a2f0190 2024-09-10T08:12:13.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.572+0000 7f045be00640 10 client.26387 handle_command_reply: tid=0 2024-09-10T08:12:13.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.573+0000 7f0483dc5740 1 client.26387 shutdown 2024-09-10T08:12:13.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.573+0000 7f0483dc5740 1 -- 172.21.15.2:0/1985816309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 msgr2=0x560f5a2dd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:13.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.573+0000 7f0483dc5740 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2dd490 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0468009780 tx=0x7f0468030340 comp rx=0 tx=0).stop 2024-09-10T08:12:13.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.573+0000 7f0483dc5740 1 -- 172.21.15.2:0/1985816309 shutdown_connections 2024-09-10T08:12:13.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.573+0000 7f0483dc5740 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 0x560f5a2dd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.573+0000 7f0483dc5740 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2dd490 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.573+0000 7f0483dc5740 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x560f5a2f0190 0x560f5a2f2610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.573+0000 7f0483dc5740 1 --2- 172.21.15.2:0/1985816309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2dcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.573+0000 7f045be00640 0 client.26387 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:12:13.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.573+0000 7f0483dc5740 1 -- 172.21.15.2:0/1985816309 >> 172.21.15.2:0/1985816309 conn(0x560f5a2d13d0 msgr2=0x560f5a2d7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:13.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.573+0000 7f0483dc5740 1 -- 172.21.15.2:0/1985816309 shutdown_connections 2024-09-10T08:12:13.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.573+0000 7f0483dc5740 1 -- 172.21.15.2:0/1985816309 wait complete. 2024-09-10T08:12:13.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.573+0000 7f0483dc5740 20 client.26387 trim_cache size 0 max 16384 2024-09-10T08:12:13.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.587+0000 7f0483dc5740 1 Processor -- start 2024-09-10T08:12:13.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.587+0000 7f0483dc5740 1 -- start start 2024-09-10T08:12:13.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.587+0000 7f0483dc5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.587+0000 7f0483dc5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2f8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.587+0000 7f0483dc5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 0x560f5a2da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.587+0000 7f0483dc5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b36e0 con 0x560f5a2d4e40 2024-09-10T08:12:13.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.587+0000 7f0483dc5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b0fb0 con 0x560f5a2d5760 2024-09-10T08:12:13.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.587+0000 7f0483dc5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1aee00 con 0x560f5a2e0020 2024-09-10T08:12:13.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.588+0000 7f0479e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.588+0000 7f047a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 0x560f5a2da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.588+0000 7f0479e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2f7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35090/0 (socket says 172.21.15.2:35090) 2024-09-10T08:12:13.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.588+0000 7f0479e00640 1 -- 172.21.15.2:0/2633870108 learned_addr learned my addr 172.21.15.2:0/2633870108 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:13.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.588+0000 7f0479400640 1 --2- 172.21.15.2:0/2633870108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.588+0000 7f0479e00640 1 -- 172.21.15.2:0/2633870108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 msgr2=0x560f5a2da1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:13.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.588+0000 7f0479e00640 1 --2- 172.21.15.2:0/2633870108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 0x560f5a2da1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.588+0000 7f0479e00640 1 -- 172.21.15.2:0/2633870108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 msgr2=0x560f5a2f8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:13.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.588+0000 7f0479e00640 1 --2- 172.21.15.2:0/2633870108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2f8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.588+0000 7f0479e00640 1 -- 172.21.15.2:0/2633870108 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f5a2d5590 con 0x560f5a2d4e40 2024-09-10T08:12:13.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.588+0000 7f0479e00640 1 --2- 172.21.15.2:0/2633870108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2f7ee0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f046c098540 tx=0x7f046c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb85f463c0ca9e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:13.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.589+0000 7f0466a00640 1 -- 172.21.15.2:0/2633870108 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c0de040 con 0x560f5a2d4e40 2024-09-10T08:12:13.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.589+0000 7f0466a00640 1 -- 172.21.15.2:0/2633870108 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f046c0d7e90 con 0x560f5a2d4e40 2024-09-10T08:12:13.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.589+0000 7f0466a00640 1 -- 172.21.15.2:0/2633870108 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c0046c0 con 0x560f5a2d4e40 2024-09-10T08:12:13.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.589+0000 7f0483dc5740 1 -- 172.21.15.2:0/2633870108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 msgr2=0x560f5a2f7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:13.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.589+0000 7f0483dc5740 1 --2- 172.21.15.2:0/2633870108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2f7ee0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f046c098540 tx=0x7f046c000bc0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.589+0000 7f0483dc5740 1 -- 172.21.15.2:0/2633870108 shutdown_connections 2024-09-10T08:12:13.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.589+0000 7f0483dc5740 1 --2- 172.21.15.2:0/2633870108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2e0020 0x560f5a2da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.589+0000 7f0483dc5740 1 --2- 172.21.15.2:0/2633870108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.589+0000 7f0483dc5740 1 --2- 172.21.15.2:0/2633870108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2d4e40 0x560f5a2f7ee0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.589+0000 7f0483dc5740 1 -- 172.21.15.2:0/2633870108 >> 172.21.15.2:0/2633870108 conn(0x560f5a2d13d0 msgr2=0x560f5a2defd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:13.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.589+0000 7f0483dc5740 1 -- 172.21.15.2:0/2633870108 shutdown_connections 2024-09-10T08:12:13.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.589+0000 7f0483dc5740 1 -- 172.21.15.2:0/2633870108 wait complete. 2024-09-10T08:12:13.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.590+0000 7f0483dc5740 1 -- start start 2024-09-10T08:12:13.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.590+0000 7f0483dc5740 1 Processor -- start 2024-09-10T08:12:13.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.590+0000 7f0483dc5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2d4e40 0x560f5a2e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0483dc5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2d9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0483dc5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2e0020 0x560f5a2da2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0483dc5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1af470 con 0x560f5a2e0020 2024-09-10T08:12:13.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0483dc5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b08e0 con 0x560f5a2d5760 2024-09-10T08:12:13.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0483dc5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1afb40 con 0x560f5a2d4e40 2024-09-10T08:12:13.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0483dc5740 10 client.0 _ll_register_callbacks cb 0x7f046c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:13.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f047a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2e0020 0x560f5a2da2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0479e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2d4e40 0x560f5a2e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0479e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2d4e40 0x560f5a2e87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34914/0 (socket says 172.21.15.2:34914) 2024-09-10T08:12:13.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0479e00640 1 -- 172.21.15.2:0/4287950293 learned_addr learned my addr 172.21.15.2:0/4287950293 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:13.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0479400640 1 --2- 172.21.15.2:0/4287950293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2d9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0479e00640 1 -- 172.21.15.2:0/4287950293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 msgr2=0x560f5a2d9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:13.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0479e00640 1 --2- 172.21.15.2:0/4287950293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2d9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0479e00640 1 -- 172.21.15.2:0/4287950293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2e0020 msgr2=0x560f5a2da2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:13.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0479e00640 1 --2- 172.21.15.2:0/4287950293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2e0020 0x560f5a2da2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.591+0000 7f0479e00640 1 -- 172.21.15.2:0/4287950293 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f5a2daae0 con 0x560f5a2d4e40 2024-09-10T08:12:13.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.592+0000 7f0479e00640 1 --2- 172.21.15.2:0/4287950293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2d4e40 0x560f5a2e87b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f046c098020 tx=0x7f046c0d7c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd4d5b346e0b9dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:13.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.592+0000 7f045be00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:13.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.592+0000 7f045be00640 1 -- 172.21.15.2:0/4287950293 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c0b9060 con 0x560f5a2d4e40 2024-09-10T08:12:13.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.592+0000 7f045be00640 1 -- 172.21.15.2:0/4287950293 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f046c099ec0 con 0x560f5a2d4e40 2024-09-10T08:12:13.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.592+0000 7f0483dc5740 1 -- 172.21.15.2:0/4287950293 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f046c0d7e90 con 0x560f5a2d4e40 2024-09-10T08:12:13.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.592+0000 7f045be00640 1 -- 172.21.15.2:0/4287950293 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f046c0b3070 con 0x560f5a2d4e40 2024-09-10T08:12:13.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.592+0000 7f0483dc5740 10 client.26356 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:13.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.592+0000 7f0483dc5740 1 -- 172.21.15.2:0/4287950293 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f5a35af50 con 0x560f5a2d4e40 2024-09-10T08:12:13.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.592+0000 7f045be00640 1 -- 172.21.15.2:0/4287950293 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f046c0d7e90 con 0x560f5a2d4e40 2024-09-10T08:12:13.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.593+0000 7f045be00640 1 -- 172.21.15.2:0/4287950293 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f046c09be20 con 0x560f5a2d4e40 2024-09-10T08:12:13.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.593+0000 7f0483dc5740 10 client.26356 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:13.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.593+0000 7f0483dc5740 10 client.26356 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:12:13.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.593+0000 7f0483dc5740 20 client.26356 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:13.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.593+0000 7f0483dc5740 1 --2- 172.21.15.2:0/4287950293 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x560f5a35b1d0 0x560f5a35d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:13.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.593+0000 7f0483dc5740 4 client.26356 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:12:13.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.593+0000 7f0483dc5740 1 -- 172.21.15.2:0/4287950293 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f5a242e60 con 0x560f5a35b1d0 2024-09-10T08:12:13.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.593+0000 7f0479400640 1 --2- 172.21.15.2:0/4287950293 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x560f5a35b1d0 0x560f5a35d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:13.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f0479400640 1 --2- 172.21.15.2:0/4287950293 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x560f5a35b1d0 0x560f5a35d5c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6b5032c90f60cad server_cookie=22939d21881436f3 in_seq=0 out_seq=0 2024-09-10T08:12:13.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f045be00640 10 client.26356 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:12:13.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f045be00640 1 -- 172.21.15.2:0/4287950293 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f5a242e60 con 0x560f5a35b1d0 2024-09-10T08:12:13.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f045be00640 10 client.26356 handle_command_reply: tid=0 2024-09-10T08:12:13.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f0483dc5740 1 client.26356 shutdown 2024-09-10T08:12:13.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f0483dc5740 1 -- 172.21.15.2:0/4287950293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2d4e40 msgr2=0x560f5a2e87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:13.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f0483dc5740 1 --2- 172.21.15.2:0/4287950293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2d4e40 0x560f5a2e87b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f046c098020 tx=0x7f046c0d7c90 comp rx=0 tx=0).stop 2024-09-10T08:12:13.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f0483dc5740 1 -- 172.21.15.2:0/4287950293 shutdown_connections 2024-09-10T08:12:13.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f0483dc5740 1 --2- 172.21.15.2:0/4287950293 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x560f5a35b1d0 0x560f5a35d5c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f0483dc5740 1 --2- 172.21.15.2:0/4287950293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560f5a2e0020 0x560f5a2da2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f0483dc5740 1 --2- 172.21.15.2:0/4287950293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5a2d5760 0x560f5a2d9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f0483dc5740 1 --2- 172.21.15.2:0/4287950293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560f5a2d4e40 0x560f5a2e87b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f045be00640 0 client.26356 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:12:13.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f0483dc5740 1 -- 172.21.15.2:0/4287950293 >> 172.21.15.2:0/4287950293 conn(0x560f5a2d13d0 msgr2=0x560f5a2cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:13.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.594+0000 7f0483dc5740 1 -- 172.21.15.2:0/4287950293 shutdown_connections 2024-09-10T08:12:13.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.595+0000 7f0483dc5740 1 -- 172.21.15.2:0/4287950293 wait complete. 2024-09-10T08:12:13.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.595+0000 7f0483dc5740 20 client.26356 trim_cache size 0 max 16384 2024-09-10T08:12:13.599 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:12:13.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.595+0000 7f047b200640 1 -- 172.21.15.2:0/2849705348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04600a2c00 msgr2=0x7f04600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:13.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.595+0000 7f047b200640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04600a2c00 0x7f04600a50c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0474005c40 tx=0x7f04740095c0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.595+0000 7f047b200640 1 -- 172.21.15.2:0/2849705348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c103e30 msgr2=0x7f047c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:13.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.595+0000 7f047b200640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c103e30 0x7f047c19a4a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f046c00e9f0 tx=0x7f046c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.595+0000 7f047b200640 1 -- 172.21.15.2:0/2849705348 shutdown_connections 2024-09-10T08:12:13.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.595+0000 7f047b200640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f047c108f60 0x7f047c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.595+0000 7f047b200640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04600a2c00 0x7f04600a50c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.595+0000 7f047b200640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f047c1047e0 0x7f047c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.595+0000 7f047b200640 1 --2- 172.21.15.2:0/2849705348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c103e30 0x7f047c19a4a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:13.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.595+0000 7f047b200640 1 -- 172.21.15.2:0/2849705348 >> 172.21.15.2:0/2849705348 conn(0x7f047c0ff8f0 msgr2=0x7f047c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:13.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.596+0000 7f047b200640 1 -- 172.21.15.2:0/2849705348 shutdown_connections 2024-09-10T08:12:13.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:13.596+0000 7f047b200640 1 -- 172.21.15.2:0/2849705348 wait complete. 2024-09-10T08:12:13.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:13 smithi155 ceph-mon[35405]: pgmap v509: 129 pgs: 129 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:12:13.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:13 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2385169766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:13.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:13 smithi107 ceph-mon[35048]: pgmap v509: 129 pgs: 129 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:12:13.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:13 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2385169766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:14.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.026+0000 7f74f9600640 1 -- 172.21.15.2:0/249559117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74f4103050 msgr2=0x7f74f4103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.026+0000 7f74f9600640 1 --2- 172.21.15.2:0/249559117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74f4103050 0x7f74f4103450 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009860 tx=0x7f74e402a340 comp rx=0 tx=0).stop 2024-09-10T08:12:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.026+0000 7f74f9600640 1 -- 172.21.15.2:0/249559117 shutdown_connections 2024-09-10T08:12:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.026+0000 7f74f9600640 1 --2- 172.21.15.2:0/249559117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74f4101f40 0x7f74f4109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.026+0000 7f74f9600640 1 --2- 172.21.15.2:0/249559117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74f4101580 0x7f74f4101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.026+0000 7f74f9600640 1 --2- 172.21.15.2:0/249559117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74f4103050 0x7f74f4103450 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.027+0000 7f74f9600640 1 -- 172.21.15.2:0/249559117 >> 172.21.15.2:0/249559117 conn(0x7f74f40fd0d0 msgr2=0x7f74f40ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.027+0000 7f74f9600640 1 -- 172.21.15.2:0/249559117 shutdown_connections 2024-09-10T08:12:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.027+0000 7f74f9600640 1 -- 172.21.15.2:0/249559117 wait complete. 2024-09-10T08:12:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.028+0000 7f74f9600640 1 Processor -- start 2024-09-10T08:12:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.028+0000 7f74f9600640 1 -- start start 2024-09-10T08:12:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.028+0000 7f74f9600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74f4101580 0x7f74f4197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.028+0000 7f74f9600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74f4101f40 0x7f74f4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.028+0000 7f74f9600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74f4103050 0x7f74f419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.028+0000 7f74f9600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f74f410d6f0 con 0x7f74f4101f40 2024-09-10T08:12:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.028+0000 7f74f9600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74f410d570 con 0x7f74f4103050 2024-09-10T08:12:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.028+0000 7f74f9600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f74f410d870 con 0x7f74f4101580 2024-09-10T08:12:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.028+0000 7f74f3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74f4101580 0x7f74f4197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.028+0000 7f74f3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74f4101f40 0x7f74f4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.028+0000 7f74f3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74f4101580 0x7f74f4197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34932/0 (socket says 172.21.15.2:34932) 2024-09-10T08:12:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.028+0000 7f74f8c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74f4103050 0x7f74f419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.028+0000 7f74f3e00640 1 -- 172.21.15.2:0/444625955 learned_addr learned my addr 172.21.15.2:0/444625955 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.029+0000 7f74f3e00640 1 -- 172.21.15.2:0/444625955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74f4103050 msgr2=0x7f74f419f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.029+0000 7f74f3e00640 1 --2- 172.21.15.2:0/444625955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74f4103050 0x7f74f419f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.029+0000 7f74f3e00640 1 -- 172.21.15.2:0/444625955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74f4101f40 msgr2=0x7f74f4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.029+0000 7f74f3e00640 1 --2- 172.21.15.2:0/444625955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74f4101f40 0x7f74f4198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.029+0000 7f74f3e00640 1 -- 172.21.15.2:0/444625955 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f419fa00 con 0x7f74f4101580 2024-09-10T08:12:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.029+0000 7f74f3e00640 1 --2- 172.21.15.2:0/444625955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74f4101580 0x7f74f4197de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f74e4035040 tx=0x7f74e4033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.029+0000 7f74f0c00640 1 -- 172.21.15.2:0/444625955 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4037040 con 0x7f74f4101580 2024-09-10T08:12:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.030+0000 7f74f0c00640 1 -- 172.21.15.2:0/444625955 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74e4030850 con 0x7f74f4101580 2024-09-10T08:12:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.030+0000 7f74f0c00640 1 -- 172.21.15.2:0/444625955 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4004920 con 0x7f74f4101580 2024-09-10T08:12:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.030+0000 7f74f9600640 1 -- 172.21.15.2:0/444625955 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f419fc90 con 0x7f74f4101580 2024-09-10T08:12:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.030+0000 7f74f9600640 1 -- 172.21.15.2:0/444625955 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f41a0130 con 0x7f74f4101580 2024-09-10T08:12:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.033+0000 7f74fd9f2740 1 Processor -- start 2024-09-10T08:12:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.033+0000 7f74fd9f2740 1 -- start start 2024-09-10T08:12:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.033+0000 7f74f0c00640 1 -- 172.21.15.2:0/444625955 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f74e4004050 con 0x7f74f4101580 2024-09-10T08:12:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74fd9f2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccbda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74fd9f2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccc2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74fd9f2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccc3040 0x55fc4ccc5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74fd9f2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fc4cc1e100 con 0x55fc4ccbd670 2024-09-10T08:12:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74fd9f2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fc4cbc61b0 con 0x55fc4ccc3040 2024-09-10T08:12:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74fd9f2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fc4cc1a0c0 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74f8c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccbda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74f8c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccbda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35116/0 (socket says 172.21.15.2:35116) 2024-09-10T08:12:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74f8c00640 1 -- 172.21.15.2:0/881821969 learned_addr learned my addr 172.21.15.2:0/881821969 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74f3e00640 1 --2- 172.21.15.2:0/881821969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccc2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74f3400640 1 --2- 172.21.15.2:0/881821969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccc3040 0x55fc4ccc5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74f8c00640 1 -- 172.21.15.2:0/881821969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccbdfb0 msgr2=0x55fc4ccc2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74f8c00640 1 --2- 172.21.15.2:0/881821969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccc2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74f8c00640 1 -- 172.21.15.2:0/881821969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccc3040 msgr2=0x55fc4ccc5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74f8c00640 1 --2- 172.21.15.2:0/881821969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccc3040 0x55fc4ccc5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74f8c00640 1 -- 172.21.15.2:0/881821969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4c931210 con 0x55fc4ccbd670 2024-09-10T08:12:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.034+0000 7f74f3e00640 1 --2- 172.21.15.2:0/881821969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccc2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.035+0000 7f74f8c00640 1 --2- 172.21.15.2:0/881821969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccbda70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00e580 tx=0x7f74ec0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc45b956da028a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.035+0000 7f74e9200640 1 -- 172.21.15.2:0/881821969 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec003970 con 0x55fc4ccbd670 2024-09-10T08:12:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.035+0000 7f74e9200640 1 -- 172.21.15.2:0/881821969 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ec00d830 con 0x55fc4ccbd670 2024-09-10T08:12:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.035+0000 7f74e9200640 1 -- 172.21.15.2:0/881821969 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec0045a0 con 0x55fc4ccbd670 2024-09-10T08:12:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.035+0000 7f74f0c00640 1 --2- 172.21.15.2:0/444625955 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f74d80a2c00 0x7f74d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.035+0000 7f74f0c00640 1 -- 172.21.15.2:0/444625955 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f74e40ef060 con 0x7f74f4101580 2024-09-10T08:12:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.035+0000 7f74f3e00640 1 --2- 172.21.15.2:0/444625955 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f74d80a2c00 0x7f74d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.036+0000 7f74fd9f2740 1 -- 172.21.15.2:0/881821969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 msgr2=0x55fc4ccbda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.036+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/881821969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccbda70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00e580 tx=0x7f74ec0095c0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.036+0000 7f74fd9f2740 1 -- 172.21.15.2:0/881821969 shutdown_connections 2024-09-10T08:12:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.036+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/881821969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccc3040 0x55fc4ccc5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.036+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/881821969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccc2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.036+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/881821969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccbda70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.036+0000 7f74fd9f2740 1 -- 172.21.15.2:0/881821969 >> 172.21.15.2:0/881821969 conn(0x55fc4ccb48d0 msgr2=0x55fc4ccb4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.036+0000 7f74f3e00640 1 --2- 172.21.15.2:0/444625955 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f74d80a2c00 0x7f74d80a50c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009830 tx=0x7f74e40b7040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.036+0000 7f74fd9f2740 1 -- 172.21.15.2:0/881821969 shutdown_connections 2024-09-10T08:12:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.036+0000 7f74fd9f2740 1 -- 172.21.15.2:0/881821969 wait complete. 2024-09-10T08:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.037+0000 7f74fd9f2740 1 -- start start 2024-09-10T08:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.037+0000 7f74fd9f2740 1 Processor -- start 2024-09-10T08:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.037+0000 7f74fd9f2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccc77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.037+0000 7f74fd9f2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccc7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.037+0000 7f74fd9f2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccc3040 0x55fc4ccced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.037+0000 7f74fd9f2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fc4ccc5fb0 con 0x55fc4ccbd670 2024-09-10T08:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.037+0000 7f74fd9f2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fc4cba7090 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.037+0000 7f74fd9f2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fc4ccc6130 con 0x55fc4ccc3040 2024-09-10T08:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.037+0000 7f74fd9f2740 10 client.0 _ll_register_callbacks cb 0x55fc4cbfbf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.037+0000 7f74f3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccc7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.037+0000 7f74f8c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccc77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74f3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccc7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46254/0 (socket says 172.21.15.2:46254) 2024-09-10T08:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74f8c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccc77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35130/0 (socket says 172.21.15.2:35130) 2024-09-10T08:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74f3400640 1 -- 172.21.15.2:0/3159531866 learned_addr learned my addr 172.21.15.2:0/3159531866 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74f8c00640 1 --2- 172.21.15.2:0/3159531866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccc3040 0x55fc4ccced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74f3400640 1 -- 172.21.15.2:0/3159531866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccc3040 msgr2=0x55fc4ccced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74f3400640 1 --2- 172.21.15.2:0/3159531866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccc3040 0x55fc4ccced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74f3400640 1 -- 172.21.15.2:0/3159531866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 msgr2=0x55fc4ccc77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74f3400640 1 --2- 172.21.15.2:0/3159531866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccc77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74f3400640 1 -- 172.21.15.2:0/3159531866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc4c9fd620 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74f8c00640 1 --2- 172.21.15.2:0/3159531866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccc3040 0x55fc4ccced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:12:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74f3400640 1 --2- 172.21.15.2:0/3159531866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccc7d00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f74e00126e0 tx=0x7f74e0012ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=569bcce2c2e2c20e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74dd600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74dd600640 1 -- 172.21.15.2:0/3159531866 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e000f920 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74fd9f2740 1 -- 172.21.15.2:0/3159531866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74ec00d830 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.038+0000 7f74dd600640 1 -- 172.21.15.2:0/3159531866 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f74e000fac0 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.039+0000 7f74dd600640 1 -- 172.21.15.2:0/3159531866 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f74e000fcc0 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.039+0000 7f74fd9f2740 10 client.26405 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.039+0000 7f74fd9f2740 1 -- 172.21.15.2:0/3159531866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc4ccd1c50 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.039+0000 7f74dd600640 1 -- 172.21.15.2:0/3159531866 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ec00d830 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.039+0000 7f74dd600640 1 -- 172.21.15.2:0/3159531866 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f74e001c420 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.039+0000 7f74fd9f2740 10 client.26405 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.039+0000 7f74fd9f2740 10 client.26405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.039+0000 7f74fd9f2740 20 client.26405 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.039+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/3159531866 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc4ccd32f0 0x55fc4ccd5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.039+0000 7f74fd9f2740 4 client.26405 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.039+0000 7f74fd9f2740 1 -- 172.21.15.2:0/3159531866 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc4cc2b830 con 0x55fc4ccd32f0 2024-09-10T08:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.040+0000 7f74f8c00640 1 --2- 172.21.15.2:0/3159531866 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc4ccd32f0 0x55fc4ccd5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.040+0000 7f74f8c00640 1 --2- 172.21.15.2:0/3159531866 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc4ccd32f0 0x55fc4ccd5740 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b143b7f8e7756a server_cookie=eb7d1dc31869270e in_seq=0 out_seq=0 2024-09-10T08:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.040+0000 7f74dd600640 10 client.26405 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74dd600640 1 -- 172.21.15.2:0/3159531866 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc4cc2b830 con 0x55fc4ccd32f0 2024-09-10T08:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74dd600640 10 client.26405 handle_command_reply: tid=0 2024-09-10T08:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74fd9f2740 1 client.26405 shutdown 2024-09-10T08:12:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74fd9f2740 1 -- 172.21.15.2:0/3159531866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbdfb0 msgr2=0x55fc4ccc7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/3159531866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccc7d00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f74e00126e0 tx=0x7f74e0012ba0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74fd9f2740 1 -- 172.21.15.2:0/3159531866 shutdown_connections 2024-09-10T08:12:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/3159531866 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc4ccd32f0 0x55fc4ccd5740 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/3159531866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccc3040 0x55fc4ccced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/3159531866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccc7d00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/3159531866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccc77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74dd600640 0 client.26405 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74fd9f2740 1 -- 172.21.15.2:0/3159531866 >> 172.21.15.2:0/3159531866 conn(0x55fc4ccb48d0 msgr2=0x55fc4ccb60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74fd9f2740 1 -- 172.21.15.2:0/3159531866 shutdown_connections 2024-09-10T08:12:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74fd9f2740 1 -- 172.21.15.2:0/3159531866 wait complete. 2024-09-10T08:12:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.041+0000 7f74fd9f2740 20 client.26405 trim_cache size 0 max 16384 2024-09-10T08:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.056+0000 7f74fd9f2740 1 Processor -- start 2024-09-10T08:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.056+0000 7f74fd9f2740 1 -- start start 2024-09-10T08:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.056+0000 7f74fd9f2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccaf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.056+0000 7f74fd9f2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.056+0000 7f74fd9f2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccc3040 0x55fc4ccb3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.056+0000 7f74fd9f2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f74f41aaed0 con 0x55fc4ccbd670 2024-09-10T08:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.056+0000 7f74fd9f2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74f41a87a0 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.056+0000 7f74fd9f2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f74f41a65f0 con 0x55fc4ccc3040 2024-09-10T08:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.057+0000 7f74f8c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccaf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.057+0000 7f74f8c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccaf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35144/0 (socket says 172.21.15.2:35144) 2024-09-10T08:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.057+0000 7f74f8c00640 1 -- 172.21.15.2:0/1083876650 learned_addr learned my addr 172.21.15.2:0/1083876650 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.057+0000 7f74f3400640 1 --2- 172.21.15.2:0/1083876650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.057+0000 7f74f8c00640 1 --2- 172.21.15.2:0/1083876650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccc3040 0x55fc4ccb3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.057+0000 7f74f8c00640 1 -- 172.21.15.2:0/1083876650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccc3040 msgr2=0x55fc4ccb3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.057+0000 7f74f8c00640 1 --2- 172.21.15.2:0/1083876650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccc3040 0x55fc4ccb3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.057+0000 7f74f8c00640 1 -- 172.21.15.2:0/1083876650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbdfb0 msgr2=0x55fc4ccced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.057+0000 7f74f8c00640 1 --2- 172.21.15.2:0/1083876650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.057+0000 7f74f8c00640 1 -- 172.21.15.2:0/1083876650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4ccb2ff0 con 0x55fc4ccbd670 2024-09-10T08:12:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.057+0000 7f74f3400640 1 --2- 172.21.15.2:0/1083876650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.057+0000 7f74f8c00640 1 --2- 172.21.15.2:0/1083876650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccaf8d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f74ec006010 tx=0x7f74ec009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8982d251b172d8a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.058+0000 7f74dfe00640 1 -- 172.21.15.2:0/1083876650 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec003970 con 0x55fc4ccbd670 2024-09-10T08:12:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.058+0000 7f74dfe00640 1 -- 172.21.15.2:0/1083876650 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ec0050d0 con 0x55fc4ccbd670 2024-09-10T08:12:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.058+0000 7f74dfe00640 1 -- 172.21.15.2:0/1083876650 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec017680 con 0x55fc4ccbd670 2024-09-10T08:12:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.058+0000 7f74fd9f2740 1 -- 172.21.15.2:0/1083876650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 msgr2=0x55fc4ccaf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.058+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/1083876650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccaf8d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f74ec006010 tx=0x7f74ec009b30 comp rx=0 tx=0).stop 2024-09-10T08:12:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.058+0000 7f74fd9f2740 1 -- 172.21.15.2:0/1083876650 shutdown_connections 2024-09-10T08:12:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.058+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/1083876650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccc3040 0x55fc4ccb3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.058+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/1083876650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbdfb0 0x55fc4ccced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.058+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/1083876650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccbd670 0x55fc4ccaf8d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.058+0000 7f74fd9f2740 1 -- 172.21.15.2:0/1083876650 >> 172.21.15.2:0/1083876650 conn(0x55fc4ccb48d0 msgr2=0x55fc4ccb31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.058+0000 7f74fd9f2740 1 -- 172.21.15.2:0/1083876650 shutdown_connections 2024-09-10T08:12:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.058+0000 7f74fd9f2740 1 -- 172.21.15.2:0/1083876650 wait complete. 2024-09-10T08:12:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.059+0000 7f74fd9f2740 1 -- start start 2024-09-10T08:12:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.059+0000 7f74fd9f2740 1 Processor -- start 2024-09-10T08:12:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.059+0000 7f74fd9f2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbd670 0x55fc4cccc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.059+0000 7f74fd9f2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccbdfb0 0x55fc4cccc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.059+0000 7f74fd9f2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccc3040 0x55fc4cccd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.059+0000 7f74fd9f2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f74f41a6c60 con 0x55fc4ccc3040 2024-09-10T08:12:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.059+0000 7f74fd9f2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74f41a80d0 con 0x55fc4ccbd670 2024-09-10T08:12:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.059+0000 7f74fd9f2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f74f41a7330 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.059+0000 7f74fd9f2740 10 client.0 _ll_register_callbacks cb 0x7f74e0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.060+0000 7f74f8c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbd670 0x55fc4cccc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.060+0000 7f74f3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccbdfb0 0x55fc4cccc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.060+0000 7f74f8c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbd670 0x55fc4cccc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46272/0 (socket says 172.21.15.2:46272) 2024-09-10T08:12:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.060+0000 7f74f8c00640 1 -- 172.21.15.2:0/2262773473 learned_addr learned my addr 172.21.15.2:0/2262773473 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.060+0000 7f74f8c00640 1 --2- 172.21.15.2:0/2262773473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccc3040 0x55fc4cccd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.060+0000 7f74f3400640 1 -- 172.21.15.2:0/2262773473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbd670 msgr2=0x55fc4cccc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.060+0000 7f74f3400640 1 --2- 172.21.15.2:0/2262773473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbd670 0x55fc4cccc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.060+0000 7f74f3400640 1 -- 172.21.15.2:0/2262773473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccc3040 msgr2=0x55fc4cccd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.060+0000 7f74f3400640 1 --2- 172.21.15.2:0/2262773473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccc3040 0x55fc4cccd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.060+0000 7f74f3400640 1 -- 172.21.15.2:0/2262773473 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc4cccddf0 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.060+0000 7f74f3400640 1 --2- 172.21.15.2:0/2262773473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccbdfb0 0x55fc4cccc820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f74e001c890 tx=0x7f74e001cb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8e6ef717f926 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.060+0000 7f74dd600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.061+0000 7f74dd600640 1 -- 172.21.15.2:0/2262773473 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e001c630 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.061+0000 7f74dd600640 1 -- 172.21.15.2:0/2262773473 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f74e0005040 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.061+0000 7f74fd9f2740 1 -- 172.21.15.2:0/2262773473 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74ec0050d0 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.061+0000 7f74dd600640 1 -- 172.21.15.2:0/2262773473 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f74e0005240 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.061+0000 7f74fd9f2740 10 client.26371 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.061+0000 7f74fd9f2740 1 -- 172.21.15.2:0/2262773473 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc4cd3e020 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.061+0000 7f74dd600640 1 -- 172.21.15.2:0/2262773473 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ec0050d0 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.061+0000 7f74dd600640 1 -- 172.21.15.2:0/2262773473 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f74e0003c60 con 0x55fc4ccbdfb0 2024-09-10T08:12:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.061+0000 7f74fd9f2740 10 client.26371 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.061+0000 7f74fd9f2740 10 client.26371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.061+0000 7f74fd9f2740 20 client.26371 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.062+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/2262773473 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc4cd3e950 0x55fc4cd40d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.062+0000 7f74fd9f2740 4 client.26371 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:12:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.062+0000 7f74fd9f2740 1 -- 172.21.15.2:0/2262773473 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc4cc25b80 con 0x55fc4cd3e950 2024-09-10T08:12:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.062+0000 7f74f8c00640 1 --2- 172.21.15.2:0/2262773473 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc4cd3e950 0x55fc4cd40d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.062+0000 7f74f8c00640 1 --2- 172.21.15.2:0/2262773473 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc4cd3e950 0x55fc4cd40d70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b37a539f7a298fe5 server_cookie=595436296a1046b9 in_seq=0 out_seq=0 2024-09-10T08:12:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.062+0000 7f74dd600640 10 client.26371 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74dd600640 1 -- 172.21.15.2:0/2262773473 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc4cc25b80 con 0x55fc4cd3e950 2024-09-10T08:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74dd600640 10 client.26371 handle_command_reply: tid=0 2024-09-10T08:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74fd9f2740 1 client.26371 shutdown 2024-09-10T08:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74fd9f2740 1 -- 172.21.15.2:0/2262773473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccbdfb0 msgr2=0x55fc4cccc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/2262773473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccbdfb0 0x55fc4cccc820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f74e001c890 tx=0x7f74e001cb00 comp rx=0 tx=0).stop 2024-09-10T08:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74fd9f2740 1 -- 172.21.15.2:0/2262773473 shutdown_connections 2024-09-10T08:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/2262773473 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fc4cd3e950 0x55fc4cd40d70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/2262773473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc4ccc3040 0x55fc4cccd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/2262773473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc4ccbdfb0 0x55fc4cccc820 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74fd9f2740 1 --2- 172.21.15.2:0/2262773473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fc4ccbd670 0x55fc4cccc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74dd600640 0 client.26371 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74fd9f2740 1 -- 172.21.15.2:0/2262773473 >> 172.21.15.2:0/2262773473 conn(0x55fc4ccb48d0 msgr2=0x55fc4ccb3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74fd9f2740 1 -- 172.21.15.2:0/2262773473 shutdown_connections 2024-09-10T08:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.076+0000 7f74fd9f2740 1 -- 172.21.15.2:0/2262773473 wait complete. 2024-09-10T08:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.077+0000 7f74fd9f2740 20 client.26371 trim_cache size 0 max 16384 2024-09-10T08:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.077+0000 7f74f9600640 1 -- 172.21.15.2:0/444625955 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f74d80a2c00 msgr2=0x7f74d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.077+0000 7f74f9600640 1 --2- 172.21.15.2:0/444625955 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f74d80a2c00 0x7f74d80a50c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009830 tx=0x7f74e40b7040 comp rx=0 tx=0).stop 2024-09-10T08:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.077+0000 7f74f9600640 1 -- 172.21.15.2:0/444625955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74f4101580 msgr2=0x7f74f4197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.077+0000 7f74f9600640 1 --2- 172.21.15.2:0/444625955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74f4101580 0x7f74f4197de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f74e4035040 tx=0x7f74e4033d90 comp rx=0 tx=0).stop 2024-09-10T08:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.077+0000 7f74f9600640 1 -- 172.21.15.2:0/444625955 shutdown_connections 2024-09-10T08:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.077+0000 7f74f9600640 1 --2- 172.21.15.2:0/444625955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74f4103050 0x7f74f419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.077+0000 7f74f9600640 1 --2- 172.21.15.2:0/444625955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74f4101f40 0x7f74f4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.077+0000 7f74f9600640 1 --2- 172.21.15.2:0/444625955 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f74d80a2c00 0x7f74d80a50c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.077+0000 7f74f9600640 1 --2- 172.21.15.2:0/444625955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74f4101580 0x7f74f4197de0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.077+0000 7f74f9600640 1 -- 172.21.15.2:0/444625955 >> 172.21.15.2:0/444625955 conn(0x7f74f40fd0d0 msgr2=0x7f74f4106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.077+0000 7f74f9600640 1 -- 172.21.15.2:0/444625955 shutdown_connections 2024-09-10T08:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:14.078+0000 7f74f9600640 1 -- 172.21.15.2:0/444625955 wait complete. 2024-09-10T08:12:14.445 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:12:15.017 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T08:12:15.017 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T08:12:15.159 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:15.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:15 smithi155 ceph-mon[35405]: pgmap v510: 129 pgs: 129 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:12:15.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:15 smithi107 ceph-mon[35048]: pgmap v510: 129 pgs: 129 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:12:15.668 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:15 smithi002 ceph-mon[30563]: pgmap v510: 129 pgs: 129 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:12:15.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.783+0000 7efce4a00640 1 -- 172.21.15.2:0/393864642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce0102790 msgr2=0x7efce01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:15.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.783+0000 7efce4a00640 1 --2- 172.21.15.2:0/393864642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce0102790 0x7efce01067e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efccc009900 tx=0x7efccc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.783+0000 7efce4a00640 1 -- 172.21.15.2:0/393864642 shutdown_connections 2024-09-10T08:12:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.783+0000 7efce4a00640 1 --2- 172.21.15.2:0/393864642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce0106f10 0x7efce010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.783+0000 7efce4a00640 1 --2- 172.21.15.2:0/393864642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce0102790 0x7efce01067e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.783+0000 7efce4a00640 1 --2- 172.21.15.2:0/393864642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efce0101de0 0x7efce01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.783+0000 7efce4a00640 1 -- 172.21.15.2:0/393864642 >> 172.21.15.2:0/393864642 conn(0x7efce00fd8c0 msgr2=0x7efce00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.783+0000 7efce4a00640 1 -- 172.21.15.2:0/393864642 shutdown_connections 2024-09-10T08:12:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.783+0000 7efce4a00640 1 -- 172.21.15.2:0/393864642 wait complete. 2024-09-10T08:12:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.784+0000 7efce4a00640 1 Processor -- start 2024-09-10T08:12:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.784+0000 7efce4a00640 1 -- start start 2024-09-10T08:12:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.785+0000 7efce4a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efce0101de0 0x7efce019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.785+0000 7efce4a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce0102790 0x7efce019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.785+0000 7efce4a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce0106f10 0x7efce019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.785+0000 7efce4a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efce0111f70 con 0x7efce0106f10 2024-09-10T08:12:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.785+0000 7efce4a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efce0111df0 con 0x7efce0101de0 2024-09-10T08:12:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.785+0000 7efce4a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efce01120f0 con 0x7efce0102790 2024-09-10T08:12:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.785+0000 7efcde800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce0106f10 0x7efce019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.785+0000 7efcdde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efce0101de0 0x7efce019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.785+0000 7efcde800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce0106f10 0x7efce019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35176/0 (socket says 172.21.15.2:35176) 2024-09-10T08:12:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.785+0000 7efcde800640 1 -- 172.21.15.2:0/2806114629 learned_addr learned my addr 172.21.15.2:0/2806114629 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.785+0000 7efcdd400640 1 --2- 172.21.15.2:0/2806114629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce0102790 0x7efce019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.786+0000 7efcdde00640 1 -- 172.21.15.2:0/2806114629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce0102790 msgr2=0x7efce019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:15.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.786+0000 7efcdde00640 1 --2- 172.21.15.2:0/2806114629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce0102790 0x7efce019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.786+0000 7efcdde00640 1 -- 172.21.15.2:0/2806114629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce0106f10 msgr2=0x7efce019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:15.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.786+0000 7efcdde00640 1 --2- 172.21.15.2:0/2806114629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce0106f10 0x7efce019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.786+0000 7efcdde00640 1 -- 172.21.15.2:0/2806114629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce019f5e0 con 0x7efce0101de0 2024-09-10T08:12:15.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.786+0000 7efcdde00640 1 --2- 172.21.15.2:0/2806114629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efce0101de0 0x7efce019a630 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efcd000e9f0 tx=0x7efcd000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:15.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.786+0000 7efcd6a00640 1 -- 172.21.15.2:0/2806114629 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd000ccc0 con 0x7efce0101de0 2024-09-10T08:12:15.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.786+0000 7efcd6a00640 1 -- 172.21.15.2:0/2806114629 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd0010910 con 0x7efce0101de0 2024-09-10T08:12:15.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.786+0000 7efcd6a00640 1 -- 172.21.15.2:0/2806114629 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0018960 con 0x7efce0101de0 2024-09-10T08:12:15.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.786+0000 7efce4a00640 1 -- 172.21.15.2:0/2806114629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce019f8d0 con 0x7efce0101de0 2024-09-10T08:12:15.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.786+0000 7efce4a00640 1 -- 172.21.15.2:0/2806114629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce01a71b0 con 0x7efce0101de0 2024-09-10T08:12:15.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efce73bc740 1 Processor -- start 2024-09-10T08:12:15.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efcd6a00640 1 -- 172.21.15.2:0/2806114629 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7efcd0010ac0 con 0x7efce0101de0 2024-09-10T08:12:15.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efce73bc740 1 -- start start 2024-09-10T08:12:15.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efce73bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70d5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efce73bc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70d5590 0x563ae70df910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efce73bc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70dfe50 0x563ae70e2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efce73bc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563ae703b3e0 con 0x563ae70d5590 2024-09-10T08:12:15.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efce73bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563ae6fe2d80 con 0x563ae70d4c70 2024-09-10T08:12:15.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efce73bc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563ae70373a0 con 0x563ae70dfe50 2024-09-10T08:12:15.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efcdde00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70d5590 0x563ae70df910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efcdd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70dfe50 0x563ae70e2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efcde800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70d5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efcdd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70dfe50 0x563ae70e2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34982/0 (socket says 172.21.15.2:34982) 2024-09-10T08:12:15.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.791+0000 7efcdd400640 1 -- 172.21.15.2:0/1571239223 learned_addr learned my addr 172.21.15.2:0/1571239223 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:15.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.790+0000 7efcdde00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70d5590 0x563ae70df910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35186/0 (socket says 172.21.15.2:35186) 2024-09-10T08:12:15.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.791+0000 7efcdd400640 1 -- 172.21.15.2:0/1571239223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 msgr2=0x563ae70d5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:15.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.791+0000 7efcdd400640 1 --2- 172.21.15.2:0/1571239223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70d5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.791+0000 7efcdd400640 1 -- 172.21.15.2:0/1571239223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70d5590 msgr2=0x563ae70df910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:15.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.791+0000 7efcdd400640 1 --2- 172.21.15.2:0/1571239223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70d5590 0x563ae70df910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.791+0000 7efcdd400640 1 -- 172.21.15.2:0/1571239223 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae7001ae0 con 0x563ae70dfe50 2024-09-10T08:12:15.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.791+0000 7efcdd400640 1 --2- 172.21.15.2:0/1571239223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70dfe50 0x563ae70e2240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efccc009900 tx=0x7efccc005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9b40abbaf8bc7f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:15.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.791+0000 7efccb000640 1 -- 172.21.15.2:0/1571239223 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc040070 con 0x563ae70dfe50 2024-09-10T08:12:15.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.792+0000 7efccb000640 1 -- 172.21.15.2:0/1571239223 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efccc0317b0 con 0x563ae70dfe50 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.792+0000 7efccb000640 1 -- 172.21.15.2:0/1571239223 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc035040 con 0x563ae70dfe50 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.792+0000 7efcd6a00640 1 --2- 172.21.15.2:0/2806114629 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efcc40a2c00 0x7efcc40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.792+0000 7efcd6a00640 1 -- 172.21.15.2:0/2806114629 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7efcd00cf780 con 0x7efce0101de0 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.792+0000 7efcde800640 1 --2- 172.21.15.2:0/2806114629 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efcc40a2c00 0x7efcc40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.792+0000 7efce73bc740 1 -- 172.21.15.2:0/1571239223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70dfe50 msgr2=0x563ae70e2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.792+0000 7efce73bc740 1 --2- 172.21.15.2:0/1571239223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70dfe50 0x563ae70e2240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efccc009900 tx=0x7efccc005be0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.792+0000 7efce73bc740 1 -- 172.21.15.2:0/1571239223 shutdown_connections 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.792+0000 7efce73bc740 1 --2- 172.21.15.2:0/1571239223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70dfe50 0x563ae70e2240 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.792+0000 7efce73bc740 1 --2- 172.21.15.2:0/1571239223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70d5590 0x563ae70df910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.792+0000 7efce73bc740 1 --2- 172.21.15.2:0/1571239223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70d5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.792+0000 7efce73bc740 1 -- 172.21.15.2:0/1571239223 >> 172.21.15.2:0/1571239223 conn(0x563ae70d1200 msgr2=0x563ae70d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.793+0000 7efce73bc740 1 -- 172.21.15.2:0/1571239223 shutdown_connections 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.793+0000 7efce73bc740 1 -- 172.21.15.2:0/1571239223 wait complete. 2024-09-10T08:12:15.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.793+0000 7efcde800640 1 --2- 172.21.15.2:0/2806114629 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efcc40a2c00 0x7efcc40a50c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efcd80049d0 tx=0x7efcd8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:15.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.793+0000 7efce73bc740 1 -- start start 2024-09-10T08:12:15.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.793+0000 7efce73bc740 1 Processor -- start 2024-09-10T08:12:15.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.794+0000 7efce73bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.794+0000 7efce73bc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d5590 0x563ae70dd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.794+0000 7efce73bc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 0x563ae70dd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.794+0000 7efce73bc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563ae70e2a20 con 0x563ae70dfe50 2024-09-10T08:12:15.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.794+0000 7efce73bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563ae6ff60f0 con 0x563ae70d4c70 2024-09-10T08:12:15.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.794+0000 7efce73bc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563ae70e2ba0 con 0x563ae70d5590 2024-09-10T08:12:15.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.794+0000 7efce73bc740 10 client.0 _ll_register_callbacks cb 0x563ae6fddbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:15.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.794+0000 7efcdde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.794+0000 7efcde800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 0x563ae70dd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.794+0000 7efcdde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70dccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46310/0 (socket says 172.21.15.2:46310) 2024-09-10T08:12:15.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.794+0000 7efcdde00640 1 -- 172.21.15.2:0/1780646245 learned_addr learned my addr 172.21.15.2:0/1780646245 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:15.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.794+0000 7efcdd400640 1 --2- 172.21.15.2:0/1780646245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d5590 0x563ae70dd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.795+0000 7efcdde00640 1 -- 172.21.15.2:0/1780646245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d5590 msgr2=0x563ae70dd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:15.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.795+0000 7efcdde00640 1 --2- 172.21.15.2:0/1780646245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d5590 0x563ae70dd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.795+0000 7efcdde00640 1 -- 172.21.15.2:0/1780646245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 msgr2=0x563ae70dd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:15.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.795+0000 7efcdde00640 1 --2- 172.21.15.2:0/1780646245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 0x563ae70dd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.795+0000 7efcdde00640 1 -- 172.21.15.2:0/1780646245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae6dac3c0 con 0x563ae70d4c70 2024-09-10T08:12:15.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.795+0000 7efcdde00640 1 --2- 172.21.15.2:0/1780646245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70dccb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efcd0098610 tx=0x7efcd0010ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23750e6c263816e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:15.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.795+0000 7efcbf400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:15.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.795+0000 7efcbf400640 1 -- 172.21.15.2:0/1780646245 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd009bc40 con 0x563ae70d4c70 2024-09-10T08:12:15.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.795+0000 7efce73bc740 1 -- 172.21.15.2:0/1780646245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efccc0317b0 con 0x563ae70d4c70 2024-09-10T08:12:15.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.795+0000 7efcbf400640 1 -- 172.21.15.2:0/1780646245 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7efcd00cf3f0 con 0x563ae70d4c70 2024-09-10T08:12:15.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.795+0000 7efcbf400640 1 -- 172.21.15.2:0/1780646245 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efcd009ff40 con 0x563ae70d4c70 2024-09-10T08:12:15.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.796+0000 7efce73bc740 10 client.26423 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:15.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.796+0000 7efce73bc740 1 -- 172.21.15.2:0/1780646245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ae70eea60 con 0x563ae70d4c70 2024-09-10T08:12:15.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.796+0000 7efcbf400640 1 -- 172.21.15.2:0/1780646245 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd00b3020 con 0x563ae70d4c70 2024-09-10T08:12:15.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.796+0000 7efcbf400640 1 -- 172.21.15.2:0/1780646245 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efcd00ce030 con 0x563ae70d4c70 2024-09-10T08:12:15.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.796+0000 7efce73bc740 10 client.26423 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:15.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.796+0000 7efce73bc740 10 client.26423 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:12:15.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.796+0000 7efce73bc740 20 client.26423 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:15.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.796+0000 7efce73bc740 1 --2- 172.21.15.2:0/1780646245 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563ae70efe10 0x563ae70f2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.796+0000 7efce73bc740 4 client.26423 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:15.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.796+0000 7efce73bc740 1 -- 172.21.15.2:0/1780646245 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ae7048950 con 0x563ae70efe10 2024-09-10T08:12:15.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.797+0000 7efcdd400640 1 --2- 172.21.15.2:0/1780646245 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563ae70efe10 0x563ae70f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.797+0000 7efcdd400640 1 --2- 172.21.15.2:0/1780646245 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563ae70efe10 0x563ae70f2280 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd74f16e9b5d6d35 server_cookie=968db64909407213 in_seq=0 out_seq=0 2024-09-10T08:12:15.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.797+0000 7efcbf400640 10 client.26423 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:12:15.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.798+0000 7efcbf400640 1 -- 172.21.15.2:0/1780646245 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ae7048950 con 0x563ae70efe10 2024-09-10T08:12:15.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.798+0000 7efcbf400640 10 client.26423 handle_command_reply: tid=0 2024-09-10T08:12:15.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.798+0000 7efce73bc740 1 client.26423 shutdown 2024-09-10T08:12:15.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.798+0000 7efce73bc740 1 -- 172.21.15.2:0/1780646245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 msgr2=0x563ae70dccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:15.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.798+0000 7efce73bc740 1 --2- 172.21.15.2:0/1780646245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70dccb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efcd0098610 tx=0x7efcd0010ee0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.798+0000 7efce73bc740 1 -- 172.21.15.2:0/1780646245 shutdown_connections 2024-09-10T08:12:15.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.798+0000 7efce73bc740 1 --2- 172.21.15.2:0/1780646245 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563ae70efe10 0x563ae70f2280 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.798+0000 7efce73bc740 1 --2- 172.21.15.2:0/1780646245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 0x563ae70dd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.798+0000 7efce73bc740 1 --2- 172.21.15.2:0/1780646245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d5590 0x563ae70dd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.798+0000 7efce73bc740 1 --2- 172.21.15.2:0/1780646245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70dccb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.798+0000 7efcbf400640 0 client.26423 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:12:15.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.798+0000 7efce73bc740 1 -- 172.21.15.2:0/1780646245 >> 172.21.15.2:0/1780646245 conn(0x563ae70d1200 msgr2=0x563ae70d6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:15.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.799+0000 7efce73bc740 1 -- 172.21.15.2:0/1780646245 shutdown_connections 2024-09-10T08:12:15.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.799+0000 7efce73bc740 1 -- 172.21.15.2:0/1780646245 wait complete. 2024-09-10T08:12:15.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.799+0000 7efce73bc740 20 client.26423 trim_cache size 0 max 16384 2024-09-10T08:12:15.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.813+0000 7efce73bc740 1 Processor -- start 2024-09-10T08:12:15.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.813+0000 7efce73bc740 1 -- start start 2024-09-10T08:12:15.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.813+0000 7efce73bc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d4c70 0x563ae70dec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.813+0000 7efce73bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d5590 0x563ae70f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.813+0000 7efce73bc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 0x563ae70f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.813+0000 7efce73bc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efce01b3950 con 0x563ae70dfe50 2024-09-10T08:12:15.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.813+0000 7efce73bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efce01b1220 con 0x563ae70d5590 2024-09-10T08:12:15.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.813+0000 7efce73bc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efce01af070 con 0x563ae70d4c70 2024-09-10T08:12:15.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.814+0000 7efcdde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d4c70 0x563ae70dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.814+0000 7efcdde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d4c70 0x563ae70dec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34998/0 (socket says 172.21.15.2:34998) 2024-09-10T08:12:15.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.814+0000 7efcdde00640 1 -- 172.21.15.2:0/2914011899 learned_addr learned my addr 172.21.15.2:0/2914011899 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:15.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.814+0000 7efcdd400640 1 --2- 172.21.15.2:0/2914011899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d5590 0x563ae70f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.814+0000 7efcde800640 1 --2- 172.21.15.2:0/2914011899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 0x563ae70f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.814+0000 7efcdde00640 1 -- 172.21.15.2:0/2914011899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d5590 msgr2=0x563ae70f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:15.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.814+0000 7efcdde00640 1 --2- 172.21.15.2:0/2914011899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d5590 0x563ae70f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.814+0000 7efcdde00640 1 -- 172.21.15.2:0/2914011899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 msgr2=0x563ae70f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:15.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.814+0000 7efcdde00640 1 --2- 172.21.15.2:0/2914011899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 0x563ae70f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.814+0000 7efcdde00640 1 -- 172.21.15.2:0/2914011899 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae70d1030 con 0x563ae70d4c70 2024-09-10T08:12:15.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.814+0000 7efcdd400640 1 --2- 172.21.15.2:0/2914011899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d5590 0x563ae70f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:15.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.814+0000 7efcdde00640 1 --2- 172.21.15.2:0/2914011899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d4c70 0x563ae70dec20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efcd00a42d0 tx=0x7efcd0097ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9393d00b4c6a502 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:15.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.815+0000 7efcca600640 1 -- 172.21.15.2:0/2914011899 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd009b090 con 0x563ae70d4c70 2024-09-10T08:12:15.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.815+0000 7efcca600640 1 -- 172.21.15.2:0/2914011899 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd00b3070 con 0x563ae70d4c70 2024-09-10T08:12:15.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.815+0000 7efcca600640 1 -- 172.21.15.2:0/2914011899 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0097990 con 0x563ae70d4c70 2024-09-10T08:12:15.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.815+0000 7efce73bc740 1 -- 172.21.15.2:0/2914011899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d4c70 msgr2=0x563ae70dec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:15.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.815+0000 7efce73bc740 1 --2- 172.21.15.2:0/2914011899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d4c70 0x563ae70dec20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efcd00a42d0 tx=0x7efcd0097ee0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.815+0000 7efce73bc740 1 -- 172.21.15.2:0/2914011899 shutdown_connections 2024-09-10T08:12:15.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.815+0000 7efce73bc740 1 --2- 172.21.15.2:0/2914011899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 0x563ae70f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.815+0000 7efce73bc740 1 --2- 172.21.15.2:0/2914011899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d5590 0x563ae70f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.815+0000 7efce73bc740 1 --2- 172.21.15.2:0/2914011899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d4c70 0x563ae70dec20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.815+0000 7efce73bc740 1 -- 172.21.15.2:0/2914011899 >> 172.21.15.2:0/2914011899 conn(0x563ae70d1200 msgr2=0x563ae70de510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:15.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.815+0000 7efce73bc740 1 -- 172.21.15.2:0/2914011899 shutdown_connections 2024-09-10T08:12:15.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.815+0000 7efce73bc740 1 -- 172.21.15.2:0/2914011899 wait complete. 2024-09-10T08:12:15.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.816+0000 7efce73bc740 1 -- start start 2024-09-10T08:12:15.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.816+0000 7efce73bc740 1 Processor -- start 2024-09-10T08:12:15.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.816+0000 7efce73bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70d9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.816+0000 7efce73bc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d5590 0x563ae70da190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.816+0000 7efce73bc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 0x563ae70da6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.816+0000 7efce73bc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efce01af6e0 con 0x563ae70dfe50 2024-09-10T08:12:15.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.816+0000 7efce73bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efce01b0b50 con 0x563ae70d4c70 2024-09-10T08:12:15.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.816+0000 7efce73bc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efce01afdb0 con 0x563ae70d5590 2024-09-10T08:12:15.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.816+0000 7efce73bc740 10 client.0 _ll_register_callbacks cb 0x7efcd00d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:15.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.817+0000 7efcdde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70d9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.817+0000 7efcdde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70d9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46332/0 (socket says 172.21.15.2:46332) 2024-09-10T08:12:15.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.817+0000 7efcdde00640 1 -- 172.21.15.2:0/2369938472 learned_addr learned my addr 172.21.15.2:0/2369938472 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:15.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.817+0000 7efcde800640 1 --2- 172.21.15.2:0/2369938472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 0x563ae70da6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.817+0000 7efcdd400640 1 --2- 172.21.15.2:0/2369938472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d5590 0x563ae70da190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.817+0000 7efcdde00640 1 -- 172.21.15.2:0/2369938472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d5590 msgr2=0x563ae70da190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:15.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.817+0000 7efcdde00640 1 --2- 172.21.15.2:0/2369938472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d5590 0x563ae70da190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.817+0000 7efcdde00640 1 -- 172.21.15.2:0/2369938472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 msgr2=0x563ae70da6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:15.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.817+0000 7efcdde00640 1 --2- 172.21.15.2:0/2369938472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 0x563ae70da6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.817+0000 7efcdde00640 1 -- 172.21.15.2:0/2369938472 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae70cc770 con 0x563ae70d4c70 2024-09-10T08:12:15.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.817+0000 7efcdde00640 1 --2- 172.21.15.2:0/2369938472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70d9c50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efcd009be00 tx=0x7efcd00cf3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbe3cb54e0e8137d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:15.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.817+0000 7efcbf400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:15.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efcbf400640 1 -- 172.21.15.2:0/2369938472 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd00a4800 con 0x563ae70d4c70 2024-09-10T08:12:15.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efcbf400640 1 -- 172.21.15.2:0/2369938472 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7efcd0010430 con 0x563ae70d4c70 2024-09-10T08:12:15.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efce73bc740 1 -- 172.21.15.2:0/2369938472 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcd00b3070 con 0x563ae70d4c70 2024-09-10T08:12:15.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efcbf400640 1 -- 172.21.15.2:0/2369938472 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efcd0010630 con 0x563ae70d4c70 2024-09-10T08:12:15.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efce73bc740 10 client.26429 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:15.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efce73bc740 1 -- 172.21.15.2:0/2369938472 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ae715ac30 con 0x563ae70d4c70 2024-09-10T08:12:15.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efcbf400640 1 -- 172.21.15.2:0/2369938472 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd00b3070 con 0x563ae70d4c70 2024-09-10T08:12:15.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efcbf400640 1 -- 172.21.15.2:0/2369938472 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efcd00ce070 con 0x563ae70d4c70 2024-09-10T08:12:15.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efce73bc740 10 client.26429 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:15.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efce73bc740 10 client.26429 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:12:15.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efce73bc740 20 client.26429 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:15.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efce73bc740 1 --2- 172.21.15.2:0/2369938472 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563ae715b010 0x563ae715d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:15.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efce73bc740 4 client.26429 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:12:15.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.818+0000 7efce73bc740 1 -- 172.21.15.2:0/2369938472 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ae7042e60 con 0x563ae715b010 2024-09-10T08:12:15.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.819+0000 7efcdd400640 1 --2- 172.21.15.2:0/2369938472 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563ae715b010 0x563ae715d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:15.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.819+0000 7efcdd400640 1 --2- 172.21.15.2:0/2369938472 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563ae715b010 0x563ae715d400 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5bdde05a17f28e7e server_cookie=470cbf788fab51cf in_seq=0 out_seq=0 2024-09-10T08:12:15.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.819+0000 7efcbf400640 10 client.26429 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:12:15.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efcbf400640 1 -- 172.21.15.2:0/2369938472 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ae7042e60 con 0x563ae715b010 2024-09-10T08:12:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efcbf400640 10 client.26429 handle_command_reply: tid=0 2024-09-10T08:12:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efce73bc740 1 client.26429 shutdown 2024-09-10T08:12:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efce73bc740 1 -- 172.21.15.2:0/2369938472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 msgr2=0x563ae70d9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efce73bc740 1 --2- 172.21.15.2:0/2369938472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70d9c50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efcd009be00 tx=0x7efcd00cf3f0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efce73bc740 1 -- 172.21.15.2:0/2369938472 shutdown_connections 2024-09-10T08:12:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efce73bc740 1 --2- 172.21.15.2:0/2369938472 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563ae715b010 0x563ae715d400 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efce73bc740 1 --2- 172.21.15.2:0/2369938472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ae70dfe50 0x563ae70da6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efce73bc740 1 --2- 172.21.15.2:0/2369938472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ae70d5590 0x563ae70da190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efce73bc740 1 --2- 172.21.15.2:0/2369938472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ae70d4c70 0x563ae70d9c50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efcbf400640 0 client.26429 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:12:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efce73bc740 1 -- 172.21.15.2:0/2369938472 >> 172.21.15.2:0/2369938472 conn(0x563ae70d1200 msgr2=0x563ae70ded30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efce73bc740 1 -- 172.21.15.2:0/2369938472 shutdown_connections 2024-09-10T08:12:15.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efce73bc740 1 -- 172.21.15.2:0/2369938472 wait complete. 2024-09-10T08:12:15.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.820+0000 7efce73bc740 20 client.26429 trim_cache size 0 max 16384 2024-09-10T08:12:15.825 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:12:15.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.821+0000 7efce4a00640 1 -- 172.21.15.2:0/2806114629 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efcc40a2c00 msgr2=0x7efcc40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.821+0000 7efce4a00640 1 --2- 172.21.15.2:0/2806114629 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efcc40a2c00 0x7efcc40a50c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efcd80049d0 tx=0x7efcd8009500 comp rx=0 tx=0).stop 2024-09-10T08:12:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.821+0000 7efce4a00640 1 -- 172.21.15.2:0/2806114629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efce0101de0 msgr2=0x7efce019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.821+0000 7efce4a00640 1 --2- 172.21.15.2:0/2806114629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efce0101de0 0x7efce019a630 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efcd000e9f0 tx=0x7efcd000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.821+0000 7efce4a00640 1 -- 172.21.15.2:0/2806114629 shutdown_connections 2024-09-10T08:12:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.821+0000 7efce4a00640 1 --2- 172.21.15.2:0/2806114629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce0106f10 0x7efce019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.821+0000 7efce4a00640 1 --2- 172.21.15.2:0/2806114629 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efcc40a2c00 0x7efcc40a50c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.821+0000 7efce4a00640 1 --2- 172.21.15.2:0/2806114629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce0102790 0x7efce019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.821+0000 7efce4a00640 1 --2- 172.21.15.2:0/2806114629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efce0101de0 0x7efce019a630 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.821+0000 7efce4a00640 1 -- 172.21.15.2:0/2806114629 >> 172.21.15.2:0/2806114629 conn(0x7efce00fd8c0 msgr2=0x7efce00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.821+0000 7efce4a00640 1 -- 172.21.15.2:0/2806114629 shutdown_connections 2024-09-10T08:12:15.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:15.821+0000 7efce4a00640 1 -- 172.21.15.2:0/2806114629 wait complete. 2024-09-10T08:12:16.898 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:12:17.037 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:17.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:17 smithi002 ceph-mon[30563]: pgmap v511: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-10T08:12:17.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:17 smithi155 ceph-mon[35405]: pgmap v511: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-10T08:12:17.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:17 smithi107 ceph-mon[35048]: pgmap v511: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-10T08:12:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.687+0000 7fd456600640 1 -- 172.21.15.2:0/3668861845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd450076720 msgr2=0x7fd450076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.687+0000 7fd456600640 1 --2- 172.21.15.2:0/3668861845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd450076720 0x7fd450076b00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd444009900 tx=0x7fd44402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.688+0000 7fd456600640 1 -- 172.21.15.2:0/3668861845 shutdown_connections 2024-09-10T08:12:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.688+0000 7fd456600640 1 --2- 172.21.15.2:0/3668861845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd450112870 0x7fd450114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.688+0000 7fd456600640 1 --2- 172.21.15.2:0/3668861845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd450077040 0x7fd450112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.688+0000 7fd456600640 1 --2- 172.21.15.2:0/3668861845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd450076720 0x7fd450076b00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.688+0000 7fd456600640 1 -- 172.21.15.2:0/3668861845 >> 172.21.15.2:0/3668861845 conn(0x7fd4500ff910 msgr2=0x7fd450101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.688+0000 7fd456600640 1 -- 172.21.15.2:0/3668861845 shutdown_connections 2024-09-10T08:12:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.688+0000 7fd456600640 1 -- 172.21.15.2:0/3668861845 wait complete. 2024-09-10T08:12:17.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.689+0000 7fd456600640 1 Processor -- start 2024-09-10T08:12:17.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.689+0000 7fd456600640 1 -- start start 2024-09-10T08:12:17.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.689+0000 7fd456600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd450076720 0x7fd4501a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.689+0000 7fd456600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd450077040 0x7fd4501a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.689+0000 7fd456600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd450112870 0x7fd45019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.689+0000 7fd456600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd450118450 con 0x7fd450076720 2024-09-10T08:12:17.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.689+0000 7fd456600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501182d0 con 0x7fd450077040 2024-09-10T08:12:17.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.689+0000 7fd456600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501185d0 con 0x7fd450112870 2024-09-10T08:12:17.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.689+0000 7fd455200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd450076720 0x7fd4501a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.690+0000 7fd455200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd450076720 0x7fd4501a4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60360/0 (socket says 172.21.15.2:60360) 2024-09-10T08:12:17.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.690+0000 7fd455200640 1 -- 172.21.15.2:0/362131271 learned_addr learned my addr 172.21.15.2:0/362131271 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:17.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.690+0000 7fd455c00640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd450112870 0x7fd45019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.690+0000 7fd44fe00640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd450077040 0x7fd4501a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.690+0000 7fd455200640 1 -- 172.21.15.2:0/362131271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd450112870 msgr2=0x7fd45019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:17.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.690+0000 7fd455200640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd450112870 0x7fd45019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.690+0000 7fd455200640 1 -- 172.21.15.2:0/362131271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd450077040 msgr2=0x7fd4501a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:17.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.690+0000 7fd455200640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd450077040 0x7fd4501a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.690+0000 7fd455200640 1 -- 172.21.15.2:0/362131271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45019efb0 con 0x7fd450076720 2024-09-10T08:12:17.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.690+0000 7fd455c00640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd450112870 0x7fd45019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:17.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.690+0000 7fd44fe00640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd450077040 0x7fd4501a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:17.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.690+0000 7fd455200640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd450076720 0x7fd4501a4670 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd444004770 tx=0x7fd444003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:17.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.691+0000 7fd44d600640 1 -- 172.21.15.2:0/362131271 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44403f070 con 0x7fd450076720 2024-09-10T08:12:17.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.691+0000 7fd44d600640 1 -- 172.21.15.2:0/362131271 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd444031ca0 con 0x7fd450076720 2024-09-10T08:12:17.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.691+0000 7fd44d600640 1 -- 172.21.15.2:0/362131271 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444035030 con 0x7fd450076720 2024-09-10T08:12:17.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.691+0000 7fd456600640 1 -- 172.21.15.2:0/362131271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45019f240 con 0x7fd450076720 2024-09-10T08:12:17.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.691+0000 7fd456600640 1 -- 172.21.15.2:0/362131271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4501ab420 con 0x7fd450076720 2024-09-10T08:12:17.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.694+0000 7fd45a7d5740 1 Processor -- start 2024-09-10T08:12:17.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.694+0000 7fd44d600640 1 -- 172.21.15.2:0/362131271 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd444042070 con 0x7fd450076720 2024-09-10T08:12:17.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.694+0000 7fd45a7d5740 1 -- start start 2024-09-10T08:12:17.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.694+0000 7fd45a7d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 0x55f7d4e35aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd45a7d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e40360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd45a7d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e408a0 0x55f7d4e42c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd45a7d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f7d4d9a260 con 0x55f7d4e356c0 2024-09-10T08:12:17.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd45a7d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f7d4d41500 con 0x55f7d4e408a0 2024-09-10T08:12:17.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd45a7d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f7d4d96220 con 0x55f7d4e35fe0 2024-09-10T08:12:17.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd455c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 0x55f7d4e35aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd455c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 0x55f7d4e35aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60370/0 (socket says 172.21.15.2:60370) 2024-09-10T08:12:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd455c00640 1 -- 172.21.15.2:0/1765339319 learned_addr learned my addr 172.21.15.2:0/1765339319 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd44fe00640 1 --2- 172.21.15.2:0/1765339319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e408a0 0x55f7d4e42c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd455200640 1 --2- 172.21.15.2:0/1765339319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e40360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd455c00640 1 -- 172.21.15.2:0/1765339319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 msgr2=0x55f7d4e40360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd455c00640 1 --2- 172.21.15.2:0/1765339319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e40360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd455c00640 1 -- 172.21.15.2:0/1765339319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e408a0 msgr2=0x55f7d4e42c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd455c00640 1 --2- 172.21.15.2:0/1765339319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e408a0 0x55f7d4e42c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.695+0000 7fd455c00640 1 -- 172.21.15.2:0/1765339319 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7d4b0bba0 con 0x55f7d4e356c0 2024-09-10T08:12:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.696+0000 7fd455c00640 1 --2- 172.21.15.2:0/1765339319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 0x55f7d4e35aa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd448011790 tx=0x7fd448011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a0f91487fe3bd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.696+0000 7fd44d600640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4380a2c00 0x7fd4380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.696+0000 7fd442000640 1 -- 172.21.15.2:0/1765339319 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448003c30 con 0x55f7d4e356c0 2024-09-10T08:12:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.696+0000 7fd442000640 1 -- 172.21.15.2:0/1765339319 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd448003dd0 con 0x55f7d4e356c0 2024-09-10T08:12:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.696+0000 7fd442000640 1 -- 172.21.15.2:0/1765339319 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480193e0 con 0x55f7d4e356c0 2024-09-10T08:12:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.696+0000 7fd44d600640 1 -- 172.21.15.2:0/362131271 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd4440ef9a0 con 0x7fd450076720 2024-09-10T08:12:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.696+0000 7fd455200640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4380a2c00 0x7fd4380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.697+0000 7fd45a7d5740 1 -- 172.21.15.2:0/1765339319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 msgr2=0x55f7d4e35aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.697+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/1765339319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 0x55f7d4e35aa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd448011790 tx=0x7fd448011c50 comp rx=0 tx=0).stop 2024-09-10T08:12:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.697+0000 7fd455200640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4380a2c00 0x7fd4380a50c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd4440b8380 tx=0x7fd444009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.697+0000 7fd45a7d5740 1 -- 172.21.15.2:0/1765339319 shutdown_connections 2024-09-10T08:12:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.697+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/1765339319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e408a0 0x55f7d4e42c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.697+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/1765339319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e40360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.697+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/1765339319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 0x55f7d4e35aa0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.697+0000 7fd45a7d5740 1 -- 172.21.15.2:0/1765339319 >> 172.21.15.2:0/1765339319 conn(0x55f7d4e31c50 msgr2=0x55f7d4e32030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.697+0000 7fd45a7d5740 1 -- 172.21.15.2:0/1765339319 shutdown_connections 2024-09-10T08:12:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.697+0000 7fd45a7d5740 1 -- 172.21.15.2:0/1765339319 wait complete. 2024-09-10T08:12:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.698+0000 7fd45a7d5740 1 -- start start 2024-09-10T08:12:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.698+0000 7fd45a7d5740 1 Processor -- start 2024-09-10T08:12:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.698+0000 7fd45a7d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 0x55f7d4e3d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.698+0000 7fd45a7d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e3dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.698+0000 7fd45a7d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e408a0 0x55f7d4e3e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.698+0000 7fd45a7d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f7d4e43470 con 0x55f7d4e356c0 2024-09-10T08:12:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.698+0000 7fd45a7d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f7d4d54bd0 con 0x55f7d4e408a0 2024-09-10T08:12:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.698+0000 7fd45a7d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f7d4e435f0 con 0x55f7d4e35fe0 2024-09-10T08:12:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.698+0000 7fd45a7d5740 10 client.0 _ll_register_callbacks cb 0x55f7d4aa90a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.699+0000 7fd455c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 0x55f7d4e3d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.699+0000 7fd455c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 0x55f7d4e3d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60386/0 (socket says 172.21.15.2:60386) 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.699+0000 7fd455c00640 1 -- 172.21.15.2:0/28294518 learned_addr learned my addr 172.21.15.2:0/28294518 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.699+0000 7fd455c00640 1 --2- 172.21.15.2:0/28294518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e408a0 0x55f7d4e3e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.699+0000 7fd455c00640 1 -- 172.21.15.2:0/28294518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 msgr2=0x55f7d4e3dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.699+0000 7fd455c00640 1 --2- 172.21.15.2:0/28294518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e3dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.699+0000 7fd455c00640 1 -- 172.21.15.2:0/28294518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e408a0 msgr2=0x55f7d4e3e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.699+0000 7fd455c00640 1 --2- 172.21.15.2:0/28294518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e408a0 0x55f7d4e3e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.699+0000 7fd455c00640 1 -- 172.21.15.2:0/28294518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7d4e4c910 con 0x55f7d4e356c0 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.699+0000 7fd455c00640 1 --2- 172.21.15.2:0/28294518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 0x55f7d4e3d740 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd44800a010 tx=0x7fd448004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fe3451ba427589 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.699+0000 7fd436a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.700+0000 7fd436a00640 1 -- 172.21.15.2:0/28294518 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44801f040 con 0x55f7d4e356c0 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.700+0000 7fd436a00640 1 -- 172.21.15.2:0/28294518 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd4480112c0 con 0x55f7d4e356c0 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.700+0000 7fd45a7d5740 1 -- 172.21.15.2:0/28294518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd448003dd0 con 0x55f7d4e356c0 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.700+0000 7fd436a00640 1 -- 172.21.15.2:0/28294518 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd448004a40 con 0x55f7d4e356c0 2024-09-10T08:12:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.700+0000 7fd45a7d5740 10 client.16404 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.700+0000 7fd45a7d5740 1 -- 172.21.15.2:0/28294518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f7d4e4f240 con 0x55f7d4e356c0 2024-09-10T08:12:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.700+0000 7fd436a00640 1 -- 172.21.15.2:0/28294518 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd448003dd0 con 0x55f7d4e356c0 2024-09-10T08:12:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.701+0000 7fd436a00640 1 -- 172.21.15.2:0/28294518 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd448018070 con 0x55f7d4e356c0 2024-09-10T08:12:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.701+0000 7fd45a7d5740 10 client.16404 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.701+0000 7fd45a7d5740 10 client.16404 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:12:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.701+0000 7fd45a7d5740 20 client.16404 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.701+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/28294518 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f7d4e50700 0x55f7d4e52af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.701+0000 7fd45a7d5740 4 client.16404 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.701+0000 7fd45a7d5740 1 -- 172.21.15.2:0/28294518 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7d4da7990 con 0x55f7d4e50700 2024-09-10T08:12:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.701+0000 7fd44fe00640 1 --2- 172.21.15.2:0/28294518 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f7d4e50700 0x55f7d4e52af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.702+0000 7fd44fe00640 1 --2- 172.21.15.2:0/28294518 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f7d4e50700 0x55f7d4e52af0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c476de299e30668 server_cookie=eb6789a50760fc4c in_seq=0 out_seq=0 2024-09-10T08:12:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.702+0000 7fd436a00640 10 client.16404 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.702+0000 7fd436a00640 1 -- 172.21.15.2:0/28294518 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7d4da7990 con 0x55f7d4e50700 2024-09-10T08:12:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.702+0000 7fd436a00640 10 client.16404 handle_command_reply: tid=0 2024-09-10T08:12:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.702+0000 7fd45a7d5740 1 client.16404 shutdown 2024-09-10T08:12:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.703+0000 7fd45a7d5740 1 -- 172.21.15.2:0/28294518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 msgr2=0x55f7d4e3d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.703+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/28294518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 0x55f7d4e3d740 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd44800a010 tx=0x7fd448004560 comp rx=0 tx=0).stop 2024-09-10T08:12:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.703+0000 7fd45a7d5740 1 -- 172.21.15.2:0/28294518 shutdown_connections 2024-09-10T08:12:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.703+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/28294518 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f7d4e50700 0x55f7d4e52af0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.703+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/28294518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e408a0 0x55f7d4e3e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.703+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/28294518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e3dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.703+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/28294518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e356c0 0x55f7d4e3d740 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.703+0000 7fd45a7d5740 1 -- 172.21.15.2:0/28294518 >> 172.21.15.2:0/28294518 conn(0x55f7d4e31c50 msgr2=0x55f7d4e30e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.703+0000 7fd436a00640 0 client.16404 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.703+0000 7fd45a7d5740 1 -- 172.21.15.2:0/28294518 shutdown_connections 2024-09-10T08:12:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.703+0000 7fd45a7d5740 1 -- 172.21.15.2:0/28294518 wait complete. 2024-09-10T08:12:17.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.703+0000 7fd45a7d5740 20 client.16404 trim_cache size 0 max 16384 2024-09-10T08:12:17.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.721+0000 7fd45a7d5740 1 Processor -- start 2024-09-10T08:12:17.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.721+0000 7fd45a7d5740 1 -- start start 2024-09-10T08:12:17.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.721+0000 7fd45a7d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e356c0 0x55f7d4e32030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.721+0000 7fd45a7d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.721+0000 7fd45a7d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e408a0 0x55f7d4e587f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.721+0000 7fd45a7d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501b7b90 con 0x55f7d4e408a0 2024-09-10T08:12:17.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.721+0000 7fd45a7d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501b5460 con 0x55f7d4e356c0 2024-09-10T08:12:17.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.721+0000 7fd45a7d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501b32b0 con 0x55f7d4e35fe0 2024-09-10T08:12:17.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.722+0000 7fd455c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e356c0 0x55f7d4e32030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.722+0000 7fd455c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e408a0 0x55f7d4e587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.722+0000 7fd455c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e408a0 0x55f7d4e587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60388/0 (socket says 172.21.15.2:60388) 2024-09-10T08:12:17.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.722+0000 7fd455c00640 1 -- 172.21.15.2:0/3461134952 learned_addr learned my addr 172.21.15.2:0/3461134952 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:17.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.722+0000 7fd44fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.722+0000 7fd455c00640 1 -- 172.21.15.2:0/3461134952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 msgr2=0x55f7d4e582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.722+0000 7fd455c00640 1 --2- 172.21.15.2:0/3461134952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.722+0000 7fd455c00640 1 -- 172.21.15.2:0/3461134952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e356c0 msgr2=0x55f7d4e32030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.722+0000 7fd455c00640 1 --2- 172.21.15.2:0/3461134952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e356c0 0x55f7d4e32030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.722+0000 7fd455c00640 1 -- 172.21.15.2:0/3461134952 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7d4e52df0 con 0x55f7d4e408a0 2024-09-10T08:12:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.722+0000 7fd455c00640 1 --2- 172.21.15.2:0/3461134952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e408a0 0x55f7d4e587f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd448005140 tx=0x7fd448005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef72fda50f0f5aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.723+0000 7fd441600640 1 -- 172.21.15.2:0/3461134952 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480053b0 con 0x55f7d4e408a0 2024-09-10T08:12:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.723+0000 7fd441600640 1 -- 172.21.15.2:0/3461134952 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd448009580 con 0x55f7d4e408a0 2024-09-10T08:12:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.723+0000 7fd441600640 1 -- 172.21.15.2:0/3461134952 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480113b0 con 0x55f7d4e408a0 2024-09-10T08:12:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.723+0000 7fd45a7d5740 1 -- 172.21.15.2:0/3461134952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e408a0 msgr2=0x55f7d4e587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.723+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/3461134952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e408a0 0x55f7d4e587f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd448005140 tx=0x7fd448005fd0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.723+0000 7fd45a7d5740 1 -- 172.21.15.2:0/3461134952 shutdown_connections 2024-09-10T08:12:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.723+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/3461134952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e408a0 0x55f7d4e587f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.723+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/3461134952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.723+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/3461134952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e356c0 0x55f7d4e32030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.723+0000 7fd45a7d5740 1 -- 172.21.15.2:0/3461134952 >> 172.21.15.2:0/3461134952 conn(0x55f7d4e31c50 msgr2=0x55f7d4e3c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.724+0000 7fd45a7d5740 1 -- 172.21.15.2:0/3461134952 shutdown_connections 2024-09-10T08:12:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.724+0000 7fd45a7d5740 1 -- 172.21.15.2:0/3461134952 wait complete. 2024-09-10T08:12:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.724+0000 7fd45a7d5740 1 -- start start 2024-09-10T08:12:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.724+0000 7fd45a7d5740 1 Processor -- start 2024-09-10T08:12:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd45a7d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e356c0 0x55f7d4e43dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd45a7d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e44310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd45a7d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e408a0 0x55f7d4eb9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd45a7d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501b3920 con 0x55f7d4e408a0 2024-09-10T08:12:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd45a7d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501b4d90 con 0x55f7d4e35fe0 2024-09-10T08:12:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd45a7d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501b3ff0 con 0x55f7d4e356c0 2024-09-10T08:12:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd45a7d5740 10 client.0 _ll_register_callbacks cb 0x7fd43c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd44fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e44310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd455c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e356c0 0x55f7d4e43dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd455c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e408a0 0x55f7d4eb9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd44fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e44310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52784/0 (socket says 172.21.15.2:52784) 2024-09-10T08:12:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd44fe00640 1 -- 172.21.15.2:0/1168294628 learned_addr learned my addr 172.21.15.2:0/1168294628 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd44fe00640 1 -- 172.21.15.2:0/1168294628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e356c0 msgr2=0x55f7d4e43dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd44fe00640 1 --2- 172.21.15.2:0/1168294628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e356c0 0x55f7d4e43dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd44fe00640 1 -- 172.21.15.2:0/1168294628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e408a0 msgr2=0x55f7d4eb9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd44fe00640 1 --2- 172.21.15.2:0/1168294628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e408a0 0x55f7d4eb9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.725+0000 7fd44fe00640 1 -- 172.21.15.2:0/1168294628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7d4eb97b0 con 0x55f7d4e35fe0 2024-09-10T08:12:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.726+0000 7fd44fe00640 1 --2- 172.21.15.2:0/1168294628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e44310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55f7d4e32fd0 tx=0x7fd43c013980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a139ec1fbe95122c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.726+0000 7fd436a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.726+0000 7fd436a00640 1 -- 172.21.15.2:0/1168294628 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c004460 con 0x55f7d4e35fe0 2024-09-10T08:12:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.726+0000 7fd45a7d5740 1 -- 172.21.15.2:0/1168294628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f7d4da1ce0 con 0x55f7d4e35fe0 2024-09-10T08:12:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.726+0000 7fd436a00640 1 -- 172.21.15.2:0/1168294628 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd43c0035f0 con 0x55f7d4e35fe0 2024-09-10T08:12:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.726+0000 7fd436a00640 1 -- 172.21.15.2:0/1168294628 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd43c030050 con 0x55f7d4e35fe0 2024-09-10T08:12:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.726+0000 7fd45a7d5740 10 client.26447 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:17.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.726+0000 7fd45a7d5740 1 -- 172.21.15.2:0/1168294628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f7d4ebaac0 con 0x55f7d4e35fe0 2024-09-10T08:12:17.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.727+0000 7fd436a00640 1 -- 172.21.15.2:0/1168294628 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f7d4da1ce0 con 0x55f7d4e35fe0 2024-09-10T08:12:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.727+0000 7fd436a00640 1 -- 172.21.15.2:0/1168294628 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd43c026030 con 0x55f7d4e35fe0 2024-09-10T08:12:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.727+0000 7fd45a7d5740 10 client.26447 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.727+0000 7fd45a7d5740 10 client.26447 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:12:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.727+0000 7fd45a7d5740 20 client.26447 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.727+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/1168294628 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f7d4ebb290 0x55f7d4ebd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.727+0000 7fd45a7d5740 4 client.26447 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T08:12:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.727+0000 7fd45a7d5740 1 -- 172.21.15.2:0/1168294628 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x55f7d4d1d730 con 0x55f7d4ebb290 2024-09-10T08:12:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.728+0000 7fd455c00640 1 --2- 172.21.15.2:0/1168294628 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f7d4ebb290 0x55f7d4ebd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.728+0000 7fd455c00640 1 --2- 172.21.15.2:0/1168294628 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f7d4ebb290 0x55f7d4ebd6b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4bdb6a7d3518b26 server_cookie=e60332a99fdd1f91 in_seq=0 out_seq=0 2024-09-10T08:12:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.728+0000 7fd436a00640 10 client.26447 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd436a00640 1 -- 172.21.15.2:0/1168294628 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f7d4d1d730 con 0x55f7d4ebb290 2024-09-10T08:12:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd436a00640 10 client.26447 handle_command_reply: tid=0 2024-09-10T08:12:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd45a7d5740 1 client.26447 shutdown 2024-09-10T08:12:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd45a7d5740 1 -- 172.21.15.2:0/1168294628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e35fe0 msgr2=0x55f7d4e44310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/1168294628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e44310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55f7d4e32fd0 tx=0x7fd43c013980 comp rx=0 tx=0).stop 2024-09-10T08:12:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd45a7d5740 1 -- 172.21.15.2:0/1168294628 shutdown_connections 2024-09-10T08:12:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/1168294628 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f7d4ebb290 0x55f7d4ebd6b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/1168294628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7d4e408a0 0x55f7d4eb9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/1168294628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7d4e35fe0 0x55f7d4e44310 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd436a00640 0 client.26447 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd45a7d5740 1 --2- 172.21.15.2:0/1168294628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d4e356c0 0x55f7d4e43dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd45a7d5740 1 -- 172.21.15.2:0/1168294628 >> 172.21.15.2:0/1168294628 conn(0x55f7d4e31c50 msgr2=0x55f7d4e3d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd45a7d5740 1 -- 172.21.15.2:0/1168294628 shutdown_connections 2024-09-10T08:12:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.740+0000 7fd45a7d5740 1 -- 172.21.15.2:0/1168294628 wait complete. 2024-09-10T08:12:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.741+0000 7fd45a7d5740 20 client.26447 trim_cache size 0 max 16384 2024-09-10T08:12:17.741 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:12:17.741 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:12:17.741 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:12:17.741 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 10, 2024-09-10T08:12:17.741 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:12:17.741 INFO:teuthology.orchestra.run.smithi002.stdout: "43e5efd8": { 2024-09-10T08:12:17.741 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 10, 2024-09-10T08:12:17.741 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:12:17.741 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:12:17.741 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:12:17.741 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:12:17.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:12:17.742 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:12:17.742 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 142.8, 2024-09-10T08:12:17.742 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:12:17.742 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:12:17.742 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:12:17.742 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:12:17.742 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:12:17.742 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:12:17.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:17.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:17.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:17.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:17.743 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:17.743 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:12:17.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.741+0000 7fd456600640 1 -- 172.21.15.2:0/362131271 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4380a2c00 msgr2=0x7fd4380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:17.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.741+0000 7fd456600640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4380a2c00 0x7fd4380a50c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd4440b8380 tx=0x7fd444009680 comp rx=0 tx=0).stop 2024-09-10T08:12:17.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.741+0000 7fd456600640 1 -- 172.21.15.2:0/362131271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd450076720 msgr2=0x7fd4501a4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:17.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.741+0000 7fd456600640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd450076720 0x7fd4501a4670 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd444004770 tx=0x7fd444003c10 comp rx=0 tx=0).stop 2024-09-10T08:12:17.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.741+0000 7fd456600640 1 -- 172.21.15.2:0/362131271 shutdown_connections 2024-09-10T08:12:17.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.741+0000 7fd456600640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd4380a2c00 0x7fd4380a50c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.741+0000 7fd456600640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd450112870 0x7fd45019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.741+0000 7fd456600640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd450077040 0x7fd4501a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.741+0000 7fd456600640 1 --2- 172.21.15.2:0/362131271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd450076720 0x7fd4501a4670 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:17.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.741+0000 7fd456600640 1 -- 172.21.15.2:0/362131271 >> 172.21.15.2:0/362131271 conn(0x7fd4500ff910 msgr2=0x7fd450101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:17.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.742+0000 7fd456600640 1 -- 172.21.15.2:0/362131271 shutdown_connections 2024-09-10T08:12:17.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:17.742+0000 7fd456600640 1 -- 172.21.15.2:0/362131271 wait complete. 2024-09-10T08:12:18.741 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:18.769 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '43e5efd8', quiesce took 7.7 seconds. Will release after: 15.05 2024-09-10T08:12:19.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.296+0000 7f0063e00640 1 -- 172.21.15.2:0/514259914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0064103e30 msgr2=0x7f0064104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:19.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.296+0000 7f0063e00640 1 --2- 172.21.15.2:0/514259914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0064103e30 0x7f0064104210 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0054009900 tx=0x7f005402a410 comp rx=0 tx=0).stop 2024-09-10T08:12:19.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.296+0000 7f0063e00640 1 -- 172.21.15.2:0/514259914 shutdown_connections 2024-09-10T08:12:19.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.296+0000 7f0063e00640 1 --2- 172.21.15.2:0/514259914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0064108f60 0x7f0064110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.296+0000 7f0063e00640 1 --2- 172.21.15.2:0/514259914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00641047e0 0x7f0064108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.296+0000 7f0063e00640 1 --2- 172.21.15.2:0/514259914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0064103e30 0x7f0064104210 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.296+0000 7f0063e00640 1 -- 172.21.15.2:0/514259914 >> 172.21.15.2:0/514259914 conn(0x7f00640ff8d0 msgr2=0x7f0064101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:19.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.296+0000 7f0063e00640 1 -- 172.21.15.2:0/514259914 shutdown_connections 2024-09-10T08:12:19.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.296+0000 7f0063e00640 1 -- 172.21.15.2:0/514259914 wait complete. 2024-09-10T08:12:19.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.297+0000 7f0063e00640 1 Processor -- start 2024-09-10T08:12:19.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.297+0000 7f0063e00640 1 -- start start 2024-09-10T08:12:19.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.298+0000 7f0063e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0064103e30 0x7f006419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.298+0000 7f0063e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00641047e0 0x7f006419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.298+0000 7f0063e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0064108f60 0x7f006419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.298+0000 7f0063e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0064114010 con 0x7f0064103e30 2024-09-10T08:12:19.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.298+0000 7f0063e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0064113e90 con 0x7f0064108f60 2024-09-10T08:12:19.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.298+0000 7f0063e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0064114190 con 0x7f00641047e0 2024-09-10T08:12:19.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.298+0000 7f0062a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0064103e30 0x7f006419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.298+0000 7f0062a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0064103e30 0x7f006419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60408/0 (socket says 172.21.15.2:60408) 2024-09-10T08:12:19.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.298+0000 7f0062a00640 1 -- 172.21.15.2:0/546222682 learned_addr learned my addr 172.21.15.2:0/546222682 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:19.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.298+0000 7f005a000640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00641047e0 0x7f006419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.298+0000 7f0063400640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0064108f60 0x7f006419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.299+0000 7f0062a00640 1 -- 172.21.15.2:0/546222682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00641047e0 msgr2=0x7f006419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:19.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.299+0000 7f0062a00640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00641047e0 0x7f006419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.299+0000 7f0062a00640 1 -- 172.21.15.2:0/546222682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0064108f60 msgr2=0x7f006419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:19.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.299+0000 7f0062a00640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0064108f60 0x7f006419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.299+0000 7f0062a00640 1 -- 172.21.15.2:0/546222682 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006419f400 con 0x7f0064103e30 2024-09-10T08:12:19.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.299+0000 7f0062a00640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0064103e30 0x7f006419a450 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0054004770 tx=0x7f0054003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:19.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.299+0000 7f005be00640 1 -- 172.21.15.2:0/546222682 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005403f070 con 0x7f0064103e30 2024-09-10T08:12:19.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.299+0000 7f005be00640 1 -- 172.21.15.2:0/546222682 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0054031ca0 con 0x7f0064103e30 2024-09-10T08:12:19.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.299+0000 7f0063e00640 1 -- 172.21.15.2:0/546222682 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006419f690 con 0x7f0064103e30 2024-09-10T08:12:19.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.299+0000 7f005be00640 1 -- 172.21.15.2:0/546222682 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054035030 con 0x7f0064103e30 2024-09-10T08:12:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.300+0000 7f0063e00640 1 -- 172.21.15.2:0/546222682 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00641a6fd0 con 0x7f0064103e30 2024-09-10T08:12:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.303+0000 7f006bfe8740 1 Processor -- start 2024-09-10T08:12:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.303+0000 7f005be00640 1 -- 172.21.15.2:0/546222682 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0054042070 con 0x7f0064103e30 2024-09-10T08:12:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.303+0000 7f006bfe8740 1 -- start start 2024-09-10T08:12:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.303+0000 7f006bfe8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e00769e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.303+0000 7f006bfe8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e00769e760 0x55e0076a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.303+0000 7f006bfe8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0076a9020 0x55e0076ab410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.303+0000 7f006bfe8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e0076043e0 con 0x55e0076a9020 2024-09-10T08:12:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.303+0000 7f006bfe8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e0075abd80 con 0x55e00769e760 2024-09-10T08:12:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.303+0000 7f006bfe8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e0076003a0 con 0x55e00769de40 2024-09-10T08:12:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.304+0000 7f0063400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e00769e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.304+0000 7f0063400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e00769e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:44390/0 (socket says 172.21.15.2:44390) 2024-09-10T08:12:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.304+0000 7f0063400640 1 -- 172.21.15.2:0/2744087326 learned_addr learned my addr 172.21.15.2:0/2744087326 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.304+0000 7f0062a00640 1 --2- 172.21.15.2:0/2744087326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e00769e760 0x55e0076a8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.304+0000 7f005a000640 1 --2- 172.21.15.2:0/2744087326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0076a9020 0x55e0076ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.304+0000 7f0063400640 1 -- 172.21.15.2:0/2744087326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e00769e760 msgr2=0x55e0076a8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.304+0000 7f0063400640 1 --2- 172.21.15.2:0/2744087326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e00769e760 0x55e0076a8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.304+0000 7f0063400640 1 -- 172.21.15.2:0/2744087326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0076a9020 msgr2=0x55e0076ab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.304+0000 7f0063400640 1 --2- 172.21.15.2:0/2744087326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0076a9020 0x55e0076ab410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.304+0000 7f0063400640 1 -- 172.21.15.2:0/2744087326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0075caae0 con 0x55e00769de40 2024-09-10T08:12:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.304+0000 7f0062a00640 1 --2- 172.21.15.2:0/2744087326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e00769e760 0x55e0076a8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.304+0000 7f0063400640 1 --2- 172.21.15.2:0/2744087326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e00769e220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f005c009f50 tx=0x7f005c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bcdf5f6ee66dc92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.305+0000 7f005be00640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00480a2b80 0x7f00480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.305+0000 7f0053400640 1 -- 172.21.15.2:0/2744087326 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c004d20 con 0x55e00769de40 2024-09-10T08:12:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.305+0000 7f005be00640 1 -- 172.21.15.2:0/546222682 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f00540efa50 con 0x7f0064103e30 2024-09-10T08:12:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.305+0000 7f0062a00640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00480a2b80 0x7f00480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.305+0000 7f0053400640 1 -- 172.21.15.2:0/2744087326 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005c004050 con 0x55e00769de40 2024-09-10T08:12:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.305+0000 7f0053400640 1 -- 172.21.15.2:0/2744087326 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c005280 con 0x55e00769de40 2024-09-10T08:12:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.306+0000 7f006bfe8740 1 -- 172.21.15.2:0/2744087326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 msgr2=0x55e00769e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.306+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2744087326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e00769e220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f005c009f50 tx=0x7f005c0118d0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.306+0000 7f0062a00640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00480a2b80 0x7f00480a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f00540bbf60 tx=0x7f00540f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.306+0000 7f006bfe8740 1 -- 172.21.15.2:0/2744087326 shutdown_connections 2024-09-10T08:12:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.306+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2744087326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0076a9020 0x55e0076ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.306+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2744087326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e00769e760 0x55e0076a8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.306+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2744087326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e00769e220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.306+0000 7f006bfe8740 1 -- 172.21.15.2:0/2744087326 >> 172.21.15.2:0/2744087326 conn(0x55e00769a3d0 msgr2=0x55e00769a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.306+0000 7f006bfe8740 1 -- 172.21.15.2:0/2744087326 shutdown_connections 2024-09-10T08:12:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.306+0000 7f006bfe8740 1 -- 172.21.15.2:0/2744087326 wait complete. 2024-09-10T08:12:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.307+0000 7f006bfe8740 1 -- start start 2024-09-10T08:12:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.307+0000 7f006bfe8740 1 Processor -- start 2024-09-10T08:12:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.308+0000 7f006bfe8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e00769de40 0x55e0076a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.308+0000 7f006bfe8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769e760 0x55e0076a6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.308+0000 7f006bfe8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0076a9020 0x55e0076a6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.308+0000 7f006bfe8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e0076abbf0 con 0x55e00769de40 2024-09-10T08:12:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.308+0000 7f006bfe8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e0075bf0f0 con 0x55e0076a9020 2024-09-10T08:12:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.308+0000 7f006bfe8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e0076abd70 con 0x55e00769e760 2024-09-10T08:12:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.308+0000 7f006bfe8740 10 client.0 _ll_register_callbacks cb 0x55e0075a6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.308+0000 7f0063400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e00769de40 0x55e0076a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.308+0000 7f0063400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e00769de40 0x55e0076a5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60424/0 (socket says 172.21.15.2:60424) 2024-09-10T08:12:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.308+0000 7f0063400640 1 -- 172.21.15.2:0/2252054522 learned_addr learned my addr 172.21.15.2:0/2252054522 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.308+0000 7f005a000640 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769e760 0x55e0076a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.308+0000 7f0063400640 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0076a9020 0x55e0076a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.309+0000 7f005a000640 1 -- 172.21.15.2:0/2252054522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0076a9020 msgr2=0x55e0076a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.309+0000 7f005a000640 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0076a9020 0x55e0076a6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.309+0000 7f005a000640 1 -- 172.21.15.2:0/2252054522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e00769de40 msgr2=0x55e0076a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.309+0000 7f005a000640 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e00769de40 0x55e0076a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.309+0000 7f005a000640 1 -- 172.21.15.2:0/2252054522 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0073753c0 con 0x55e00769e760 2024-09-10T08:12:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.309+0000 7f0063400640 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e00769de40 0x55e0076a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.309+0000 7f005a000640 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769e760 0x55e0076a6440 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f004c0126d0 tx=0x7f004c012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fff8da8c55e3964 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.309+0000 7f0047e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:19.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.309+0000 7f0047e00640 1 -- 172.21.15.2:0/2252054522 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c004020 con 0x55e00769e760 2024-09-10T08:12:19.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.309+0000 7f0047e00640 1 -- 172.21.15.2:0/2252054522 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f004c0041c0 con 0x55e00769e760 2024-09-10T08:12:19.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.309+0000 7f006bfe8740 1 -- 172.21.15.2:0/2252054522 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f005c004050 con 0x55e00769e760 2024-09-10T08:12:19.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.310+0000 7f0047e00640 1 -- 172.21.15.2:0/2252054522 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f004c025050 con 0x55e00769e760 2024-09-10T08:12:19.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.310+0000 7f006bfe8740 10 client.26416 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:19.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.310+0000 7f006bfe8740 1 -- 172.21.15.2:0/2252054522 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e0076b7c90 con 0x55e00769e760 2024-09-10T08:12:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.310+0000 7f0047e00640 1 -- 172.21.15.2:0/2252054522 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005c004050 con 0x55e00769e760 2024-09-10T08:12:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.310+0000 7f0047e00640 1 -- 172.21.15.2:0/2252054522 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f004c003a20 con 0x55e00769e760 2024-09-10T08:12:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.311+0000 7f006bfe8740 10 client.26416 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.311+0000 7f006bfe8740 10 client.26416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.311+0000 7f006bfe8740 20 client.26416 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.311+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e0076b9150 0x55e0076bb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.311+0000 7f006bfe8740 4 client.26416 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.311+0000 7f006bfe8740 1 -- 172.21.15.2:0/2252054522 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e007611950 con 0x55e0076b9150 2024-09-10T08:12:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.311+0000 7f0063400640 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e0076b9150 0x55e0076bb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.312+0000 7f0063400640 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e0076b9150 0x55e0076bb5b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=524560f648ad85be server_cookie=e7a8ff80f19c9e0d in_seq=0 out_seq=0 2024-09-10T08:12:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.312+0000 7f0047e00640 10 client.26416 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f0047e00640 1 -- 172.21.15.2:0/2252054522 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e007611950 con 0x55e0076b9150 2024-09-10T08:12:19.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f0047e00640 10 client.26416 handle_command_reply: tid=0 2024-09-10T08:12:19.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f006bfe8740 1 client.26416 shutdown 2024-09-10T08:12:19.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f006bfe8740 1 -- 172.21.15.2:0/2252054522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769e760 msgr2=0x55e0076a6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:19.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769e760 0x55e0076a6440 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f004c0126d0 tx=0x7f004c012b90 comp rx=0 tx=0).stop 2024-09-10T08:12:19.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f006bfe8740 1 -- 172.21.15.2:0/2252054522 shutdown_connections 2024-09-10T08:12:19.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e0076b9150 0x55e0076bb5b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0076a9020 0x55e0076a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769e760 0x55e0076a6440 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2252054522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e00769de40 0x55e0076a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f006bfe8740 1 -- 172.21.15.2:0/2252054522 >> 172.21.15.2:0/2252054522 conn(0x55e00769a3d0 msgr2=0x55e0076a00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:19.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f0047e00640 0 client.26416 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:19.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f006bfe8740 1 -- 172.21.15.2:0/2252054522 shutdown_connections 2024-09-10T08:12:19.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f006bfe8740 1 -- 172.21.15.2:0/2252054522 wait complete. 2024-09-10T08:12:19.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.313+0000 7f006bfe8740 20 client.26416 trim_cache size 0 max 16384 2024-09-10T08:12:19.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.329+0000 7f006bfe8740 1 Processor -- start 2024-09-10T08:12:19.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.329+0000 7f006bfe8740 1 -- start start 2024-09-10T08:12:19.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f006bfe8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e00769efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f006bfe8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e00769e760 0x55e0076c0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f006bfe8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0076a9020 0x55e0076c1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f006bfe8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f00641b3710 con 0x55e0076a9020 2024-09-10T08:12:19.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f006bfe8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00641b0fe0 con 0x55e00769e760 2024-09-10T08:12:19.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f006bfe8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f00641aee30 con 0x55e00769de40 2024-09-10T08:12:19.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f005a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e00769e760 0x55e0076c0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f0063400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0076a9020 0x55e0076c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f005a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e00769e760 0x55e0076c0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52830/0 (socket says 172.21.15.2:52830) 2024-09-10T08:12:19.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f005a000640 1 -- 172.21.15.2:0/2925971637 learned_addr learned my addr 172.21.15.2:0/2925971637 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:19.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f0063400640 1 --2- 172.21.15.2:0/2925971637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e00769efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f005a000640 1 -- 172.21.15.2:0/2925971637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 msgr2=0x55e00769efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:19.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f005a000640 1 --2- 172.21.15.2:0/2925971637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e00769efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f005a000640 1 -- 172.21.15.2:0/2925971637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0076a9020 msgr2=0x55e0076c1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:19.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f005a000640 1 --2- 172.21.15.2:0/2925971637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0076a9020 0x55e0076c1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.330+0000 7f005a000640 1 -- 172.21.15.2:0/2925971637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0076a4450 con 0x55e00769e760 2024-09-10T08:12:19.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.331+0000 7f005a000640 1 --2- 172.21.15.2:0/2925971637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e00769e760 0x55e0076c0ec0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f004c000c00 tx=0x7f004c0096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e567ab6e5cd7b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:19.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.331+0000 7f0052a00640 1 -- 172.21.15.2:0/2925971637 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c02e070 con 0x55e00769e760 2024-09-10T08:12:19.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.331+0000 7f0052a00640 1 -- 172.21.15.2:0/2925971637 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c0044c0 con 0x55e00769e760 2024-09-10T08:12:19.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.331+0000 7f0052a00640 1 -- 172.21.15.2:0/2925971637 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c022040 con 0x55e00769e760 2024-09-10T08:12:19.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.331+0000 7f006bfe8740 1 -- 172.21.15.2:0/2925971637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e00769e760 msgr2=0x55e0076c0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:19.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.331+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2925971637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e00769e760 0x55e0076c0ec0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f004c000c00 tx=0x7f004c0096a0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.332+0000 7f006bfe8740 1 -- 172.21.15.2:0/2925971637 shutdown_connections 2024-09-10T08:12:19.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.332+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2925971637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e0076a9020 0x55e0076c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.332+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2925971637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e00769e760 0x55e0076c0ec0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.332+0000 7f006bfe8740 1 --2- 172.21.15.2:0/2925971637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e00769efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.332+0000 7f006bfe8740 1 -- 172.21.15.2:0/2925971637 >> 172.21.15.2:0/2925971637 conn(0x55e00769a3d0 msgr2=0x55e0076a47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:19.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.332+0000 7f006bfe8740 1 -- 172.21.15.2:0/2925971637 shutdown_connections 2024-09-10T08:12:19.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.332+0000 7f006bfe8740 1 -- 172.21.15.2:0/2925971637 wait complete. 2024-09-10T08:12:19.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.332+0000 7f006bfe8740 1 -- start start 2024-09-10T08:12:19.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.332+0000 7f006bfe8740 1 Processor -- start 2024-09-10T08:12:19.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.333+0000 7f006bfe8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e0076b1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.333+0000 7f006bfe8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e00769e760 0x55e0076ac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.333+0000 7f006bfe8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0076a9020 0x55e0076accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.333+0000 7f006bfe8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f00641af4a0 con 0x55e00769e760 2024-09-10T08:12:19.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.333+0000 7f006bfe8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00641b0910 con 0x55e0076a9020 2024-09-10T08:12:19.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.333+0000 7f006bfe8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f00641afb70 con 0x55e00769de40 2024-09-10T08:12:19.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.333+0000 7f006bfe8740 10 client.0 _ll_register_callbacks cb 0x7f004c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:19.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.333+0000 7f0063400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e0076b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.333+0000 7f005a000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e00769e760 0x55e0076ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.333+0000 7f0063400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e0076b1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:44418/0 (socket says 172.21.15.2:44418) 2024-09-10T08:12:19.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.333+0000 7f0063400640 1 -- 172.21.15.2:0/900286672 learned_addr learned my addr 172.21.15.2:0/900286672 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:19.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.333+0000 7f0063400640 1 --2- 172.21.15.2:0/900286672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0076a9020 0x55e0076accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.334+0000 7f0063400640 1 -- 172.21.15.2:0/900286672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0076a9020 msgr2=0x55e0076accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:19.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.334+0000 7f0063400640 1 --2- 172.21.15.2:0/900286672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0076a9020 0x55e0076accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.334+0000 7f0063400640 1 -- 172.21.15.2:0/900286672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e00769e760 msgr2=0x55e0076ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:19.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.334+0000 7f0063400640 1 --2- 172.21.15.2:0/900286672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e00769e760 0x55e0076ac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.334+0000 7f0063400640 1 -- 172.21.15.2:0/900286672 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0076ad1f0 con 0x55e00769de40 2024-09-10T08:12:19.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.334+0000 7f0063400640 1 --2- 172.21.15.2:0/900286672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e0076b1440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f005c006010 tx=0x7f005c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47dd107d65584d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:19.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.334+0000 7f0047e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:19.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.334+0000 7f0047e00640 1 -- 172.21.15.2:0/900286672 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c003e40 con 0x55e00769de40 2024-09-10T08:12:19.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.334+0000 7f0047e00640 1 -- 172.21.15.2:0/900286672 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f005c027440 con 0x55e00769de40 2024-09-10T08:12:19.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.334+0000 7f006bfe8740 1 -- 172.21.15.2:0/900286672 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e00760be60 con 0x55e00769de40 2024-09-10T08:12:19.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.334+0000 7f0047e00640 1 -- 172.21.15.2:0/900286672 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f005c027640 con 0x55e00769de40 2024-09-10T08:12:19.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.334+0000 7f006bfe8740 10 client.26422 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:19.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.334+0000 7f006bfe8740 1 -- 172.21.15.2:0/900286672 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e007723bc0 con 0x55e00769de40 2024-09-10T08:12:19.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.335+0000 7f0047e00640 1 -- 172.21.15.2:0/900286672 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e00760be60 con 0x55e00769de40 2024-09-10T08:12:19.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.335+0000 7f0047e00640 1 -- 172.21.15.2:0/900286672 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f005c00f750 con 0x55e00769de40 2024-09-10T08:12:19.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.335+0000 7f006bfe8740 10 client.26422 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:19.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.335+0000 7f006bfe8740 10 client.26422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:19.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.335+0000 7f006bfe8740 20 client.26422 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:19.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.335+0000 7f006bfe8740 1 --2- 172.21.15.2:0/900286672 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e007724150 0x55e007726540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:19.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.335+0000 7f006bfe8740 4 client.26422 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:12:19.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.335+0000 7f006bfe8740 1 -- 172.21.15.2:0/900286672 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e007588010 con 0x55e007724150 2024-09-10T08:12:19.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.335+0000 7f005a000640 1 --2- 172.21.15.2:0/900286672 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e007724150 0x55e007726540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:19.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f005a000640 1 --2- 172.21.15.2:0/900286672 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e007724150 0x55e007726540 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9c3cdcc2528529 server_cookie=321fa318ad9b654b in_seq=0 out_seq=0 2024-09-10T08:12:19.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f0047e00640 10 client.26422 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:19.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f0047e00640 1 -- 172.21.15.2:0/900286672 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e007588010 con 0x55e007724150 2024-09-10T08:12:19.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f0047e00640 10 client.26422 handle_command_reply: tid=0 2024-09-10T08:12:19.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f006bfe8740 1 client.26422 shutdown 2024-09-10T08:12:19.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f006bfe8740 1 -- 172.21.15.2:0/900286672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 msgr2=0x55e0076b1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:19.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f006bfe8740 1 --2- 172.21.15.2:0/900286672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e0076b1440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f005c006010 tx=0x7f005c0118d0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f006bfe8740 1 -- 172.21.15.2:0/900286672 shutdown_connections 2024-09-10T08:12:19.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f006bfe8740 1 --2- 172.21.15.2:0/900286672 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e007724150 0x55e007726540 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f006bfe8740 1 --2- 172.21.15.2:0/900286672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0076a9020 0x55e0076accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f006bfe8740 1 --2- 172.21.15.2:0/900286672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e00769e760 0x55e0076ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f006bfe8740 1 --2- 172.21.15.2:0/900286672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e00769de40 0x55e0076b1440 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f0047e00640 0 client.26422 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:19.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f006bfe8740 1 -- 172.21.15.2:0/900286672 >> 172.21.15.2:0/900286672 conn(0x55e00769a3d0 msgr2=0x55e00769f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:19.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.336+0000 7f006bfe8740 1 -- 172.21.15.2:0/900286672 shutdown_connections 2024-09-10T08:12:19.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.337+0000 7f006bfe8740 1 -- 172.21.15.2:0/900286672 wait complete. 2024-09-10T08:12:19.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.337+0000 7f006bfe8740 20 client.26422 trim_cache size 0 max 16384 2024-09-10T08:12:19.340 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:12:19.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.337+0000 7f0063e00640 1 -- 172.21.15.2:0/546222682 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00480a2b80 msgr2=0x7f00480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:19.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.337+0000 7f0063e00640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00480a2b80 0x7f00480a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f00540bbf60 tx=0x7f00540f5530 comp rx=0 tx=0).stop 2024-09-10T08:12:19.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.337+0000 7f0063e00640 1 -- 172.21.15.2:0/546222682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0064103e30 msgr2=0x7f006419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:19.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.337+0000 7f0063e00640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0064103e30 0x7f006419a450 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0054004770 tx=0x7f0054003c10 comp rx=0 tx=0).stop 2024-09-10T08:12:19.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.337+0000 7f0063e00640 1 -- 172.21.15.2:0/546222682 shutdown_connections 2024-09-10T08:12:19.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.337+0000 7f0063e00640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0064108f60 0x7f006419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.337+0000 7f0063e00640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f00480a2b80 0x7f00480a5040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.337+0000 7f0063e00640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00641047e0 0x7f006419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.337+0000 7f0063e00640 1 --2- 172.21.15.2:0/546222682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0064103e30 0x7f006419a450 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:19.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.337+0000 7f0063e00640 1 -- 172.21.15.2:0/546222682 >> 172.21.15.2:0/546222682 conn(0x7f00640ff8d0 msgr2=0x7f0064100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:19.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.338+0000 7f0063e00640 1 -- 172.21.15.2:0/546222682 shutdown_connections 2024-09-10T08:12:19.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:19.338+0000 7f0063e00640 1 -- 172.21.15.2:0/546222682 wait complete. 2024-09-10T08:12:19.659 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:19 smithi002 ceph-mon[30563]: pgmap v512: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:12:19.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:19 smithi107 ceph-mon[35048]: pgmap v512: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:12:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:19 smithi155 ceph-mon[35405]: pgmap v512: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:12:21.360 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:12:21.361 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:12:21.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:21 smithi107 ceph-mon[35048]: pgmap v513: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:12:21.709 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:21 smithi155 ceph-mon[35405]: pgmap v513: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:12:21.759 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:21 smithi002 ceph-mon[30563]: pgmap v513: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:12:22.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.274+0000 7f0f72400640 1 -- 172.21.15.2:0/4291153664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c109120 msgr2=0x7f0f6c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:22.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.274+0000 7f0f72400640 1 --2- 172.21.15.2:0/4291153664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c109120 0x7f0f6c110ca0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009ca0 tx=0x7f0f6402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.274+0000 7f0f72400640 1 -- 172.21.15.2:0/4291153664 shutdown_connections 2024-09-10T08:12:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.274+0000 7f0f72400640 1 --2- 172.21.15.2:0/4291153664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c109120 0x7f0f6c110ca0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.274+0000 7f0f72400640 1 --2- 172.21.15.2:0/4291153664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f6c1049a0 0x7f0f6c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.274+0000 7f0f72400640 1 --2- 172.21.15.2:0/4291153664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f6c103ff0 0x7f0f6c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.274+0000 7f0f72400640 1 -- 172.21.15.2:0/4291153664 >> 172.21.15.2:0/4291153664 conn(0x7f0f6c0ffa90 msgr2=0x7f0f6c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.274+0000 7f0f72400640 1 -- 172.21.15.2:0/4291153664 shutdown_connections 2024-09-10T08:12:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.274+0000 7f0f72400640 1 -- 172.21.15.2:0/4291153664 wait complete. 2024-09-10T08:12:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.275+0000 7f0f72400640 1 Processor -- start 2024-09-10T08:12:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.275+0000 7f0f72400640 1 -- start start 2024-09-10T08:12:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f72400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f6c103ff0 0x7f0f6c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f72400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f6c1049a0 0x7f0f6c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f72400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c109120 0x7f0f6c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f72400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c1141a0 con 0x7f0f6c109120 2024-09-10T08:12:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f72400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c114020 con 0x7f0f6c103ff0 2024-09-10T08:12:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f72400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c114320 con 0x7f0f6c1049a0 2024-09-10T08:12:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f6aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f6c1049a0 0x7f0f6c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f6b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f6c103ff0 0x7f0f6c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f6be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c109120 0x7f0f6c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f6aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f6c1049a0 0x7f0f6c19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:44438/0 (socket says 172.21.15.2:44438) 2024-09-10T08:12:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f6aa00640 1 -- 172.21.15.2:0/1831150873 learned_addr learned my addr 172.21.15.2:0/1831150873 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f6aa00640 1 -- 172.21.15.2:0/1831150873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f6c103ff0 msgr2=0x7f0f6c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f6aa00640 1 --2- 172.21.15.2:0/1831150873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f6c103ff0 0x7f0f6c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f6aa00640 1 -- 172.21.15.2:0/1831150873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c109120 msgr2=0x7f0f6c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f6aa00640 1 --2- 172.21.15.2:0/1831150873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c109120 0x7f0f6c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.276+0000 7f0f6aa00640 1 -- 172.21.15.2:0/1831150873 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c19f590 con 0x7f0f6c1049a0 2024-09-10T08:12:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.277+0000 7f0f6aa00640 1 --2- 172.21.15.2:0/1831150873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f6c1049a0 0x7f0f6c19ab20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800e9f0 tx=0x7f0f5800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.277+0000 7f0f63e00640 1 -- 172.21.15.2:0/1831150873 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5800cd80 con 0x7f0f6c1049a0 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.277+0000 7f0f63e00640 1 -- 172.21.15.2:0/1831150873 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f58005500 con 0x7f0f6c1049a0 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.277+0000 7f0f72400640 1 -- 172.21.15.2:0/1831150873 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6c19f880 con 0x7f0f6c1049a0 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.277+0000 7f0f63e00640 1 -- 172.21.15.2:0/1831150873 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58010690 con 0x7f0f6c1049a0 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.277+0000 7f0f72400640 1 -- 172.21.15.2:0/1831150873 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f6c1a7160 con 0x7f0f6c1049a0 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.280+0000 7f0f74e0b740 1 Processor -- start 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.280+0000 7f0f74e0b740 1 -- start start 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.280+0000 7f0f63e00640 1 -- 172.21.15.2:0/1831150873 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0f580046c0 con 0x7f0f6c1049a0 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f74e0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 0x55a45d2d5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f74e0b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2d63e0 0x55a45d2e0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f74e0b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2e0ca0 0x55a45d2e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f74e0b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a45d23c290 con 0x55a45d2d63e0 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f74e0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a45d1e3b40 con 0x55a45d2d5ac0 2024-09-10T08:12:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f74e0b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a45d238250 con 0x55a45d2e0ca0 2024-09-10T08:12:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f6be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 0x55a45d2d5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f6b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2d63e0 0x55a45d2e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f6be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 0x55a45d2d5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52878/0 (socket says 172.21.15.2:52878) 2024-09-10T08:12:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f6be00640 1 -- 172.21.15.2:0/1628402575 learned_addr learned my addr 172.21.15.2:0/1628402575 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f6aa00640 1 --2- 172.21.15.2:0/1628402575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2e0ca0 0x55a45d2e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f6be00640 1 -- 172.21.15.2:0/1628402575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2e0ca0 msgr2=0x55a45d2e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f6be00640 1 --2- 172.21.15.2:0/1628402575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2e0ca0 0x55a45d2e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f6be00640 1 -- 172.21.15.2:0/1628402575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2d63e0 msgr2=0x55a45d2e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f6be00640 1 --2- 172.21.15.2:0/1628402575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2d63e0 0x55a45d2e0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f6be00640 1 -- 172.21.15.2:0/1628402575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a45cfad6a0 con 0x55a45d2d5ac0 2024-09-10T08:12:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.281+0000 7f0f6b400640 1 --2- 172.21.15.2:0/1628402575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2d63e0 0x55a45d2e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.282+0000 7f0f6be00640 1 --2- 172.21.15.2:0/1628402575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 0x55a45d2d5ea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009ca0 tx=0x7f0f640093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42d65145fd8b9b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.282+0000 7f0f63e00640 1 --2- 172.21.15.2:0/1831150873 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0f500a2b80 0x7f0f500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.282+0000 7f0f60c00640 1 -- 172.21.15.2:0/1628402575 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64031db0 con 0x55a45d2d5ac0 2024-09-10T08:12:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.282+0000 7f0f63e00640 1 -- 172.21.15.2:0/1831150873 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0f580cf7d0 con 0x7f0f6c1049a0 2024-09-10T08:12:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.282+0000 7f0f60c00640 1 -- 172.21.15.2:0/1628402575 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f64036030 con 0x55a45d2d5ac0 2024-09-10T08:12:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.282+0000 7f0f60c00640 1 -- 172.21.15.2:0/1628402575 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f640042a0 con 0x55a45d2d5ac0 2024-09-10T08:12:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.282+0000 7f0f6b400640 1 --2- 172.21.15.2:0/1831150873 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0f500a2b80 0x7f0f500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.283+0000 7f0f74e0b740 1 -- 172.21.15.2:0/1628402575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 msgr2=0x55a45d2d5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.283+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/1628402575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 0x55a45d2d5ea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009ca0 tx=0x7f0f640093e0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.283+0000 7f0f6b400640 1 --2- 172.21.15.2:0/1831150873 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0f500a2b80 0x7f0f500a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55a45d2d2eb0 tx=0x7f0f5c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.283+0000 7f0f74e0b740 1 -- 172.21.15.2:0/1628402575 shutdown_connections 2024-09-10T08:12:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.283+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/1628402575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2e0ca0 0x55a45d2e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.283+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/1628402575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2d63e0 0x55a45d2e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.283+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/1628402575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 0x55a45d2d5ea0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.283+0000 7f0f74e0b740 1 -- 172.21.15.2:0/1628402575 >> 172.21.15.2:0/1628402575 conn(0x55a45d2d2050 msgr2=0x55a45d2d2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.283+0000 7f0f74e0b740 1 -- 172.21.15.2:0/1628402575 shutdown_connections 2024-09-10T08:12:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.283+0000 7f0f74e0b740 1 -- 172.21.15.2:0/1628402575 wait complete. 2024-09-10T08:12:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.284+0000 7f0f74e0b740 1 -- start start 2024-09-10T08:12:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.284+0000 7f0f74e0b740 1 Processor -- start 2024-09-10T08:12:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.284+0000 7f0f74e0b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2d5ac0 0x55a45d2e0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.284+0000 7f0f74e0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 0x55a45d2dd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.284+0000 7f0f74e0b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2e0ca0 0x55a45d2dde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f74e0b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a45d2e3790 con 0x55a45d2e0ca0 2024-09-10T08:12:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f74e0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a45d1f6fa0 con 0x55a45d2d63e0 2024-09-10T08:12:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f74e0b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a45d2e3910 con 0x55a45d2d5ac0 2024-09-10T08:12:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f74e0b740 10 client.0 _ll_register_callbacks cb 0x55a45cf4aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f6aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 0x55a45d2dd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f6be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2e0ca0 0x55a45d2dde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f6aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 0x55a45d2dd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52880/0 (socket says 172.21.15.2:52880) 2024-09-10T08:12:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f6aa00640 1 -- 172.21.15.2:0/2761887606 learned_addr learned my addr 172.21.15.2:0/2761887606 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f6b400640 1 --2- 172.21.15.2:0/2761887606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2d5ac0 0x55a45d2e0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f6aa00640 1 -- 172.21.15.2:0/2761887606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2d5ac0 msgr2=0x55a45d2e0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f6aa00640 1 --2- 172.21.15.2:0/2761887606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2d5ac0 0x55a45d2e0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f6aa00640 1 -- 172.21.15.2:0/2761887606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2e0ca0 msgr2=0x55a45d2dde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f6aa00640 1 --2- 172.21.15.2:0/2761887606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2e0ca0 0x55a45d2dde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f6aa00640 1 -- 172.21.15.2:0/2761887606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a45d2de680 con 0x55a45d2d63e0 2024-09-10T08:12:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.285+0000 7f0f6be00640 1 --2- 172.21.15.2:0/2761887606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2e0ca0 0x55a45d2dde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.286+0000 7f0f6aa00640 1 --2- 172.21.15.2:0/2761887606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 0x55a45d2dd8d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800e9c0 tx=0x7f0f58004ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=551fc250db9efc54 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.286+0000 7f0f55600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.286+0000 7f0f55600640 1 -- 172.21.15.2:0/2761887606 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5809b5a0 con 0x55a45d2d63e0 2024-09-10T08:12:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.286+0000 7f0f55600640 1 -- 172.21.15.2:0/2761887606 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0f580a4420 con 0x55a45d2d63e0 2024-09-10T08:12:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.286+0000 7f0f74e0b740 1 -- 172.21.15.2:0/2761887606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f64036030 con 0x55a45d2d63e0 2024-09-10T08:12:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.286+0000 7f0f55600640 1 -- 172.21.15.2:0/2761887606 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f5809c340 con 0x55a45d2d63e0 2024-09-10T08:12:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.286+0000 7f0f74e0b740 10 client.26477 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.286+0000 7f0f74e0b740 1 -- 172.21.15.2:0/2761887606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a45d2ef790 con 0x55a45d2d63e0 2024-09-10T08:12:22.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.286+0000 7f0f55600640 1 -- 172.21.15.2:0/2761887606 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f64036030 con 0x55a45d2d63e0 2024-09-10T08:12:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.286+0000 7f0f55600640 1 -- 172.21.15.2:0/2761887606 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0f5809b090 con 0x55a45d2d63e0 2024-09-10T08:12:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.287+0000 7f0f74e0b740 10 client.26477 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.287+0000 7f0f74e0b740 10 client.26477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.287+0000 7f0f74e0b740 20 client.26477 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.287+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/2761887606 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a45d2f0ba0 0x55a45d2f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.287+0000 7f0f74e0b740 4 client.26477 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.287+0000 7f0f74e0b740 1 -- 172.21.15.2:0/2761887606 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a45d249800 con 0x55a45d2f0ba0 2024-09-10T08:12:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.287+0000 7f0f6b400640 1 --2- 172.21.15.2:0/2761887606 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a45d2f0ba0 0x55a45d2f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.287+0000 7f0f6b400640 1 --2- 172.21.15.2:0/2761887606 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a45d2f0ba0 0x55a45d2f2f90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a61c22566b29fa server_cookie=4b8089d919b337c5 in_seq=0 out_seq=0 2024-09-10T08:12:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.287+0000 7f0f55600640 10 client.26477 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.288+0000 7f0f55600640 1 -- 172.21.15.2:0/2761887606 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a45d249800 con 0x55a45d2f0ba0 2024-09-10T08:12:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.288+0000 7f0f55600640 10 client.26477 handle_command_reply: tid=0 2024-09-10T08:12:22.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.288+0000 7f0f74e0b740 1 client.26477 shutdown 2024-09-10T08:12:22.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.289+0000 7f0f74e0b740 1 -- 172.21.15.2:0/2761887606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 msgr2=0x55a45d2dd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:22.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.289+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/2761887606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 0x55a45d2dd8d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800e9c0 tx=0x7f0f58004ae0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.289+0000 7f0f74e0b740 1 -- 172.21.15.2:0/2761887606 shutdown_connections 2024-09-10T08:12:22.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.289+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/2761887606 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a45d2f0ba0 0x55a45d2f2f90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.289+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/2761887606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2e0ca0 0x55a45d2dde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.289+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/2761887606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 0x55a45d2dd8d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.289+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/2761887606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2d5ac0 0x55a45d2e0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.289+0000 7f0f55600640 0 client.26477 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:22.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.289+0000 7f0f74e0b740 1 -- 172.21.15.2:0/2761887606 >> 172.21.15.2:0/2761887606 conn(0x55a45d2d2050 msgr2=0x55a45d2d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:22.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.289+0000 7f0f74e0b740 1 -- 172.21.15.2:0/2761887606 shutdown_connections 2024-09-10T08:12:22.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.289+0000 7f0f74e0b740 1 -- 172.21.15.2:0/2761887606 wait complete. 2024-09-10T08:12:22.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.289+0000 7f0f74e0b740 20 client.26477 trim_cache size 0 max 16384 2024-09-10T08:12:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.307+0000 7f0f74e0b740 1 Processor -- start 2024-09-10T08:12:22.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.307+0000 7f0f74e0b740 1 -- start start 2024-09-10T08:12:22.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f74e0b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2d5ac0 0x55a45d2d8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f74e0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 0x55a45d2f8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f74e0b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2e0ca0 0x55a45d2f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f74e0b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c1b3800 con 0x55a45d2e0ca0 2024-09-10T08:12:22.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f74e0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c1b10d0 con 0x55a45d2d63e0 2024-09-10T08:12:22.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f74e0b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c1aef20 con 0x55a45d2d5ac0 2024-09-10T08:12:22.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f6aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 0x55a45d2f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f6be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2e0ca0 0x55a45d2f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f6aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 0x55a45d2f8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52894/0 (socket says 172.21.15.2:52894) 2024-09-10T08:12:22.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f6aa00640 1 -- 172.21.15.2:0/3955763165 learned_addr learned my addr 172.21.15.2:0/3955763165 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f6b400640 1 --2- 172.21.15.2:0/3955763165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2d5ac0 0x55a45d2d8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f6aa00640 1 -- 172.21.15.2:0/3955763165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2d5ac0 msgr2=0x55a45d2d8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f6aa00640 1 --2- 172.21.15.2:0/3955763165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2d5ac0 0x55a45d2d8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f6aa00640 1 -- 172.21.15.2:0/3955763165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2e0ca0 msgr2=0x55a45d2f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f6aa00640 1 --2- 172.21.15.2:0/3955763165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2e0ca0 0x55a45d2f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f6aa00640 1 -- 172.21.15.2:0/3955763165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a45d2d5840 con 0x55a45d2d63e0 2024-09-10T08:12:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.308+0000 7f0f6be00640 1 --2- 172.21.15.2:0/3955763165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2e0ca0 0x55a45d2f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.309+0000 7f0f6aa00640 1 --2- 172.21.15.2:0/3955763165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 0x55a45d2f8860 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0f580ccf30 tx=0x7f0f5809a150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e0005460ec5d6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.309+0000 7f0f57e00640 1 -- 172.21.15.2:0/3955763165 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5809b5a0 con 0x55a45d2d63e0 2024-09-10T08:12:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.309+0000 7f0f57e00640 1 -- 172.21.15.2:0/3955763165 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f5809fff0 con 0x55a45d2d63e0 2024-09-10T08:12:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.309+0000 7f0f57e00640 1 -- 172.21.15.2:0/3955763165 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58097ee0 con 0x55a45d2d63e0 2024-09-10T08:12:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.310+0000 7f0f74e0b740 1 -- 172.21.15.2:0/3955763165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 msgr2=0x55a45d2f8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.310+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/3955763165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 0x55a45d2f8860 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0f580ccf30 tx=0x7f0f5809a150 comp rx=0 tx=0).stop 2024-09-10T08:12:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.310+0000 7f0f74e0b740 1 -- 172.21.15.2:0/3955763165 shutdown_connections 2024-09-10T08:12:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.310+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/3955763165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2e0ca0 0x55a45d2f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.310+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/3955763165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d63e0 0x55a45d2f8860 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.310+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/3955763165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2d5ac0 0x55a45d2d8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.310+0000 7f0f74e0b740 1 -- 172.21.15.2:0/3955763165 >> 172.21.15.2:0/3955763165 conn(0x55a45d2d2050 msgr2=0x55a45d2d8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.310+0000 7f0f74e0b740 1 -- 172.21.15.2:0/3955763165 shutdown_connections 2024-09-10T08:12:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.310+0000 7f0f74e0b740 1 -- 172.21.15.2:0/3955763165 wait complete. 2024-09-10T08:12:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.310+0000 7f0f74e0b740 1 -- start start 2024-09-10T08:12:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.310+0000 7f0f74e0b740 1 Processor -- start 2024-09-10T08:12:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.311+0000 7f0f74e0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 0x55a45d2e4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.311+0000 7f0f74e0b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2d63e0 0x55a45d2e5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.311+0000 7f0f74e0b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2e0ca0 0x55a45d35a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.311+0000 7f0f74e0b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c1af590 con 0x55a45d2d63e0 2024-09-10T08:12:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.311+0000 7f0f74e0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c1b0a00 con 0x55a45d2d5ac0 2024-09-10T08:12:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.311+0000 7f0f74e0b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c1afc60 con 0x55a45d2e0ca0 2024-09-10T08:12:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.311+0000 7f0f74e0b740 10 client.0 _ll_register_callbacks cb 0x7f0f580d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.311+0000 7f0f6b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 0x55a45d2e4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.311+0000 7f0f6b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 0x55a45d2e4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52904/0 (socket says 172.21.15.2:52904) 2024-09-10T08:12:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.311+0000 7f0f6b400640 1 -- 172.21.15.2:0/1610195103 learned_addr learned my addr 172.21.15.2:0/1610195103 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.311+0000 7f0f6aa00640 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2d63e0 0x55a45d2e5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.311+0000 7f0f6be00640 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2e0ca0 0x55a45d35a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.312+0000 7f0f6b400640 1 -- 172.21.15.2:0/1610195103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2e0ca0 msgr2=0x55a45d35a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.312+0000 7f0f6b400640 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2e0ca0 0x55a45d35a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.312+0000 7f0f6b400640 1 -- 172.21.15.2:0/1610195103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2d63e0 msgr2=0x55a45d2e5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:22.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.312+0000 7f0f6b400640 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2d63e0 0x55a45d2e5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.312+0000 7f0f6b400640 1 -- 172.21.15.2:0/1610195103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a45d2e5a50 con 0x55a45d2d5ac0 2024-09-10T08:12:22.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.312+0000 7f0f6be00640 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2e0ca0 0x55a45d35a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:22.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.312+0000 7f0f6b400640 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 0x55a45d2e4c30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55a45d2f37d0 tx=0x7f0f5c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3be9bfa6eea49c07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:22.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.312+0000 7f0f55600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:22.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.312+0000 7f0f55600640 1 -- 172.21.15.2:0/1610195103 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c003c40 con 0x55a45d2d5ac0 2024-09-10T08:12:22.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.312+0000 7f0f55600640 1 -- 172.21.15.2:0/1610195103 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0f5c003de0 con 0x55a45d2d5ac0 2024-09-10T08:12:22.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.312+0000 7f0f74e0b740 1 -- 172.21.15.2:0/1610195103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f5809fff0 con 0x55a45d2d5ac0 2024-09-10T08:12:22.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.313+0000 7f0f55600640 1 -- 172.21.15.2:0/1610195103 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f5c005020 con 0x55a45d2d5ac0 2024-09-10T08:12:22.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.313+0000 7f0f74e0b740 10 client.26483 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:22.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.313+0000 7f0f74e0b740 1 -- 172.21.15.2:0/1610195103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a45d35bf20 con 0x55a45d2d5ac0 2024-09-10T08:12:22.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.313+0000 7f0f55600640 1 -- 172.21.15.2:0/1610195103 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f5809fff0 con 0x55a45d2d5ac0 2024-09-10T08:12:22.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.313+0000 7f0f55600640 1 -- 172.21.15.2:0/1610195103 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0f5c02b380 con 0x55a45d2d5ac0 2024-09-10T08:12:22.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.313+0000 7f0f74e0b740 10 client.26483 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:22.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.313+0000 7f0f74e0b740 10 client.26483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:22.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.313+0000 7f0f74e0b740 20 client.26483 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:22.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.313+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a45d35c640 0x55a45d35ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:22.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.313+0000 7f0f74e0b740 4 client.26483 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:12:22.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.313+0000 7f0f74e0b740 1 -- 172.21.15.2:0/1610195103 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a45d243d10 con 0x55a45d35c640 2024-09-10T08:12:22.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.313+0000 7f0f6aa00640 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a45d35c640 0x55a45d35ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:22.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.314+0000 7f0f6aa00640 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a45d35c640 0x55a45d35ea60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a781f8a06dceb8e8 server_cookie=c12dbdbb3f52dfb2 in_seq=0 out_seq=0 2024-09-10T08:12:22.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.314+0000 7f0f55600640 10 client.26483 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:22.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.315+0000 7f0f55600640 1 -- 172.21.15.2:0/1610195103 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a45d243d10 con 0x55a45d35c640 2024-09-10T08:12:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.315+0000 7f0f55600640 10 client.26483 handle_command_reply: tid=0 2024-09-10T08:12:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.315+0000 7f0f74e0b740 1 client.26483 shutdown 2024-09-10T08:12:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.315+0000 7f0f74e0b740 1 -- 172.21.15.2:0/1610195103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 msgr2=0x55a45d2e4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.315+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 0x55a45d2e4c30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55a45d2f37d0 tx=0x7f0f5c0046c0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.315+0000 7f0f74e0b740 1 -- 172.21.15.2:0/1610195103 shutdown_connections 2024-09-10T08:12:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.315+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a45d35c640 0x55a45d35ea60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.315+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a45d2e0ca0 0x55a45d35a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.315+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a45d2d63e0 0x55a45d2e5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.315+0000 7f0f74e0b740 1 --2- 172.21.15.2:0/1610195103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a45d2d5ac0 0x55a45d2e4c30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.315+0000 7f0f74e0b740 1 -- 172.21.15.2:0/1610195103 >> 172.21.15.2:0/1610195103 conn(0x55a45d2d2050 msgr2=0x55a45d2df950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.315+0000 7f0f55600640 0 client.26483 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.316+0000 7f0f74e0b740 1 -- 172.21.15.2:0/1610195103 shutdown_connections 2024-09-10T08:12:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.316+0000 7f0f74e0b740 1 -- 172.21.15.2:0/1610195103 wait complete. 2024-09-10T08:12:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.316+0000 7f0f74e0b740 20 client.26483 trim_cache size 0 max 16384 2024-09-10T08:12:22.321 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:12:22.321 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:12:22.321 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "cd043db2-bedf-46c7-96e5-284df7ff82c0", 2024-09-10T08:12:22.321 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:12:22.321 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:12:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.316+0000 7f0f72400640 1 -- 172.21.15.2:0/1831150873 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0f500a2b80 msgr2=0x7f0f500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.316+0000 7f0f72400640 1 --2- 172.21.15.2:0/1831150873 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0f500a2b80 0x7f0f500a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55a45d2d2eb0 tx=0x7f0f5c005f90 comp rx=0 tx=0).stop 2024-09-10T08:12:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.316+0000 7f0f72400640 1 -- 172.21.15.2:0/1831150873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f6c1049a0 msgr2=0x7f0f6c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.316+0000 7f0f72400640 1 --2- 172.21.15.2:0/1831150873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f6c1049a0 0x7f0f6c19ab20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800e9f0 tx=0x7f0f5800edf0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.317+0000 7f0f72400640 1 -- 172.21.15.2:0/1831150873 shutdown_connections 2024-09-10T08:12:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.317+0000 7f0f72400640 1 --2- 172.21.15.2:0/1831150873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c109120 0x7f0f6c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.317+0000 7f0f72400640 1 --2- 172.21.15.2:0/1831150873 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0f500a2b80 0x7f0f500a5040 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.317+0000 7f0f72400640 1 --2- 172.21.15.2:0/1831150873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f6c1049a0 0x7f0f6c19ab20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.317+0000 7f0f72400640 1 --2- 172.21.15.2:0/1831150873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f6c103ff0 0x7f0f6c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.317+0000 7f0f72400640 1 -- 172.21.15.2:0/1831150873 >> 172.21.15.2:0/1831150873 conn(0x7f0f6c0ffa90 msgr2=0x7f0f6c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.317+0000 7f0f72400640 1 -- 172.21.15.2:0/1831150873 shutdown_connections 2024-09-10T08:12:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:22.317+0000 7f0f72400640 1 -- 172.21.15.2:0/1831150873 wait complete. 2024-09-10T08:12:23.200 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:12:23.581 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:23.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:23 smithi107 ceph-mon[35048]: pgmap v514: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-10T08:12:23.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:23 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:12:23.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:23 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:23.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:23 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:12:23.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:23 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:23.721 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:23 smithi002 ceph-mon[30563]: pgmap v514: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-10T08:12:23.721 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:23 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:12:23.721 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:23 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:23.721 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:23 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:12:23.721 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:23 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:23 smithi155 ceph-mon[35405]: pgmap v514: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-10T08:12:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:23 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:12:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:23 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:23.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:23 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:12:23.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:23 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:24.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.131+0000 7f3aed200640 1 -- 172.21.15.2:0/244674518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ae8077040 msgr2=0x7f3ae81124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:24.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.131+0000 7f3aed200640 1 --2- 172.21.15.2:0/244674518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ae8077040 0x7f3ae81124f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009900 tx=0x7f3ad002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.131+0000 7f3aed200640 1 -- 172.21.15.2:0/244674518 shutdown_connections 2024-09-10T08:12:24.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.131+0000 7f3aed200640 1 --2- 172.21.15.2:0/244674518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8112a30 0x7f3ae8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.131+0000 7f3aed200640 1 --2- 172.21.15.2:0/244674518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ae8077040 0x7f3ae81124f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.131+0000 7f3aed200640 1 --2- 172.21.15.2:0/244674518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ae8076720 0x7f3ae8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.131+0000 7f3aed200640 1 -- 172.21.15.2:0/244674518 >> 172.21.15.2:0/244674518 conn(0x7f3ae80ffa90 msgr2=0x7f3ae8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.132+0000 7f3aed200640 1 -- 172.21.15.2:0/244674518 shutdown_connections 2024-09-10T08:12:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.132+0000 7f3aed200640 1 -- 172.21.15.2:0/244674518 wait complete. 2024-09-10T08:12:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.132+0000 7f3aed200640 1 Processor -- start 2024-09-10T08:12:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.133+0000 7f3aed200640 1 -- start start 2024-09-10T08:12:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.133+0000 7f3aed200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ae8076720 0x7f3ae81a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.133+0000 7f3aed200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ae8077040 0x7f3ae81a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.133+0000 7f3aed200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8112a30 0x7f3ae819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.133+0000 7f3aed200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8118610 con 0x7f3ae8076720 2024-09-10T08:12:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.133+0000 7f3aed200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8118490 con 0x7f3ae8112a30 2024-09-10T08:12:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.133+0000 7f3aed200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8118790 con 0x7f3ae8077040 2024-09-10T08:12:24.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.133+0000 7f3ae5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ae8076720 0x7f3ae81a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.133+0000 7f3ae5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ae8076720 0x7f3ae81a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60496/0 (socket says 172.21.15.2:60496) 2024-09-10T08:12:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.133+0000 7f3ae5e00640 1 -- 172.21.15.2:0/3162540177 learned_addr learned my addr 172.21.15.2:0/3162540177 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.133+0000 7f3ae6800640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8112a30 0x7f3ae819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.133+0000 7f3adfe00640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ae8077040 0x7f3ae81a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.134+0000 7f3ae5e00640 1 -- 172.21.15.2:0/3162540177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ae8077040 msgr2=0x7f3ae81a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.134+0000 7f3ae5e00640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ae8077040 0x7f3ae81a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.134+0000 7f3ae5e00640 1 -- 172.21.15.2:0/3162540177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8112a30 msgr2=0x7f3ae819e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.134+0000 7f3ae5e00640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8112a30 0x7f3ae819e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.134+0000 7f3ae5e00640 1 -- 172.21.15.2:0/3162540177 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae819f020 con 0x7f3ae8076720 2024-09-10T08:12:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.134+0000 7f3ae5e00640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ae8076720 0x7f3ae81a4730 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800e9f0 tx=0x7f3ad800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.134+0000 7f3adea00640 1 -- 172.21.15.2:0/3162540177 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad800cd80 con 0x7f3ae8076720 2024-09-10T08:12:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.134+0000 7f3adea00640 1 -- 172.21.15.2:0/3162540177 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ad8005500 con 0x7f3ae8076720 2024-09-10T08:12:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.134+0000 7f3adea00640 1 -- 172.21.15.2:0/3162540177 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8010690 con 0x7f3ae8076720 2024-09-10T08:12:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.134+0000 7f3aed200640 1 -- 172.21.15.2:0/3162540177 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae819f310 con 0x7f3ae8076720 2024-09-10T08:12:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.134+0000 7f3aed200640 1 -- 172.21.15.2:0/3162540177 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae81ab600 con 0x7f3ae8076720 2024-09-10T08:12:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.137+0000 7f3aefe01740 1 Processor -- start 2024-09-10T08:12:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.137+0000 7f3adea00640 1 -- 172.21.15.2:0/3162540177 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3ad80046c0 con 0x7f3ae8076720 2024-09-10T08:12:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.138+0000 7f3aefe01740 1 -- start start 2024-09-10T08:12:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.138+0000 7f3aefe01740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f6ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.138+0000 7f3aefe01740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f6f4b0 0x556f62f79830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.138+0000 7f3aefe01740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f79d70 0x556f62f7c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.138+0000 7f3aefe01740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556f62ed53e0 con 0x556f62f6eb90 2024-09-10T08:12:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.138+0000 7f3aefe01740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556f62e7cd80 con 0x556f62f6f4b0 2024-09-10T08:12:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.138+0000 7f3aefe01740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556f62ed13a0 con 0x556f62f79d70 2024-09-10T08:12:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.138+0000 7f3ae6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f6ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.138+0000 7f3ae6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f6ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60504/0 (socket says 172.21.15.2:60504) 2024-09-10T08:12:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.138+0000 7f3ae6800640 1 -- 172.21.15.2:0/2099574519 learned_addr learned my addr 172.21.15.2:0/2099574519 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.138+0000 7f3ae5e00640 1 --2- 172.21.15.2:0/2099574519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f6f4b0 0x556f62f79830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.138+0000 7f3adfe00640 1 --2- 172.21.15.2:0/2099574519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f79d70 0x556f62f7c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.139+0000 7f3ae6800640 1 -- 172.21.15.2:0/2099574519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f79d70 msgr2=0x556f62f7c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.139+0000 7f3ae6800640 1 --2- 172.21.15.2:0/2099574519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f79d70 0x556f62f7c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.139+0000 7f3ae6800640 1 -- 172.21.15.2:0/2099574519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f6f4b0 msgr2=0x556f62f79830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.139+0000 7f3ae6800640 1 --2- 172.21.15.2:0/2099574519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f6f4b0 0x556f62f79830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.139+0000 7f3ae6800640 1 -- 172.21.15.2:0/2099574519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f62e9bae0 con 0x556f62f6eb90 2024-09-10T08:12:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.139+0000 7f3adfe00640 1 --2- 172.21.15.2:0/2099574519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f79d70 0x556f62f7c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.139+0000 7f3ae5e00640 1 --2- 172.21.15.2:0/2099574519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f6f4b0 0x556f62f79830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.139+0000 7f3ae6800640 1 --2- 172.21.15.2:0/2099574519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f6ef70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009f10 tx=0x7f3ae0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c4243aab40026 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.139+0000 7f3adea00640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3acc0a2c00 0x7f3acc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.139+0000 7f3ad7400640 1 -- 172.21.15.2:0/2099574519 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0016070 con 0x556f62f6eb90 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.140+0000 7f3ad7400640 1 -- 172.21.15.2:0/2099574519 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae001bb40 con 0x556f62f6eb90 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.140+0000 7f3adea00640 1 -- 172.21.15.2:0/3162540177 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3ad80cf810 con 0x7f3ae8076720 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.140+0000 7f3ae5e00640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3acc0a2c00 0x7f3acc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.140+0000 7f3ad7400640 1 -- 172.21.15.2:0/2099574519 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0012c50 con 0x556f62f6eb90 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.140+0000 7f3aefe01740 1 -- 172.21.15.2:0/2099574519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 msgr2=0x556f62f6ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.140+0000 7f3ae5e00640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3acc0a2c00 0x7f3acc0a50c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3ad809bbb0 tx=0x7f3ad80d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.140+0000 7f3aefe01740 1 --2- 172.21.15.2:0/2099574519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f6ef70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009f10 tx=0x7f3ae0010740 comp rx=0 tx=0).stop 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.141+0000 7f3aefe01740 1 -- 172.21.15.2:0/2099574519 shutdown_connections 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.141+0000 7f3aefe01740 1 --2- 172.21.15.2:0/2099574519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f79d70 0x556f62f7c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.141+0000 7f3aefe01740 1 --2- 172.21.15.2:0/2099574519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f6f4b0 0x556f62f79830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.141+0000 7f3aefe01740 1 --2- 172.21.15.2:0/2099574519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f6ef70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.141+0000 7f3aefe01740 1 -- 172.21.15.2:0/2099574519 >> 172.21.15.2:0/2099574519 conn(0x556f62f6b120 msgr2=0x556f62f6b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.141+0000 7f3aefe01740 1 -- 172.21.15.2:0/2099574519 shutdown_connections 2024-09-10T08:12:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.141+0000 7f3aefe01740 1 -- 172.21.15.2:0/2099574519 wait complete. 2024-09-10T08:12:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.141+0000 7f3aefe01740 1 -- start start 2024-09-10T08:12:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.142+0000 7f3aefe01740 1 Processor -- start 2024-09-10T08:12:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.142+0000 7f3aefe01740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f6eb90 0x556f62f76ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.142+0000 7f3aefe01740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.142+0000 7f3aefe01740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f79d70 0x556f62f77720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.142+0000 7f3aefe01740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556f62f7ca30 con 0x556f62f79d70 2024-09-10T08:12:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.142+0000 7f3aefe01740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556f62e900f0 con 0x556f62f6eb90 2024-09-10T08:12:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.142+0000 7f3aefe01740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556f62f7cbb0 con 0x556f62f6f4b0 2024-09-10T08:12:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.142+0000 7f3aefe01740 10 client.0 _ll_register_callbacks cb 0x556f62e819e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:24.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.142+0000 7f3ae6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f79d70 0x556f62f77720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.142+0000 7f3adfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.142+0000 7f3ae6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f79d70 0x556f62f77720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60516/0 (socket says 172.21.15.2:60516) 2024-09-10T08:12:24.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.142+0000 7f3ae6800640 1 -- 172.21.15.2:0/3365959604 learned_addr learned my addr 172.21.15.2:0/3365959604 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:24.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.142+0000 7f3adfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f771e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:44504/0 (socket says 172.21.15.2:44504) 2024-09-10T08:12:24.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.143+0000 7f3ae6800640 1 --2- 172.21.15.2:0/3365959604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f6eb90 0x556f62f76ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.143+0000 7f3ae6800640 1 -- 172.21.15.2:0/3365959604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 msgr2=0x556f62f771e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:24.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.143+0000 7f3ae6800640 1 --2- 172.21.15.2:0/3365959604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f771e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.143+0000 7f3ae6800640 1 -- 172.21.15.2:0/3365959604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f6eb90 msgr2=0x556f62f76ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:24.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.143+0000 7f3ae6800640 1 --2- 172.21.15.2:0/3365959604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f6eb90 0x556f62f76ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.143+0000 7f3ae6800640 1 -- 172.21.15.2:0/3365959604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f62c463c0 con 0x556f62f79d70 2024-09-10T08:12:24.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.143+0000 7f3ae6800640 1 --2- 172.21.15.2:0/3365959604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f79d70 0x556f62f77720 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009880 tx=0x7f3ae0012810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1690b0d59050b703 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:24.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.143+0000 7f3acbe00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:12:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.143+0000 7f3acbe00640 1 -- 172.21.15.2:0/3365959604 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0016040 con 0x556f62f79d70 2024-09-10T08:12:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.143+0000 7f3acbe00640 1 -- 172.21.15.2:0/3365959604 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3ae0004a40 con 0x556f62f79d70 2024-09-10T08:12:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.143+0000 7f3aefe01740 1 -- 172.21.15.2:0/3365959604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae001bb40 con 0x556f62f79d70 2024-09-10T08:12:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.144+0000 7f3acbe00640 1 -- 172.21.15.2:0/3365959604 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ae002f050 con 0x556f62f79d70 2024-09-10T08:12:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.144+0000 7f3aefe01740 10 client.16458 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.144+0000 7f3aefe01740 1 -- 172.21.15.2:0/3365959604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556f62f88730 con 0x556f62f79d70 2024-09-10T08:12:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.144+0000 7f3acbe00640 1 -- 172.21.15.2:0/3365959604 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae001bb40 con 0x556f62f79d70 2024-09-10T08:12:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.144+0000 7f3acbe00640 1 -- 172.21.15.2:0/3365959604 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ae000f040 con 0x556f62f79d70 2024-09-10T08:12:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.144+0000 7f3aefe01740 10 client.16458 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.144+0000 7f3aefe01740 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.144+0000 7f3aefe01740 20 client.16458 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.144+0000 7f3aefe01740 1 --2- 172.21.15.2:0/3365959604 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556f62f89bf0 0x556f62f8bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.144+0000 7f3aefe01740 4 client.16458 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.144+0000 7f3aefe01740 1 -- 172.21.15.2:0/3365959604 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f62ee2950 con 0x556f62f89bf0 2024-09-10T08:12:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.145+0000 7f3adfe00640 1 --2- 172.21.15.2:0/3365959604 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556f62f89bf0 0x556f62f8bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.145+0000 7f3adfe00640 1 --2- 172.21.15.2:0/3365959604 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556f62f89bf0 0x556f62f8bfe0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=371c51dd6c14be5b server_cookie=f17a5a4be996c7cd in_seq=0 out_seq=0 2024-09-10T08:12:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.145+0000 7f3acbe00640 10 client.16458 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3acbe00640 1 -- 172.21.15.2:0/3365959604 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f62ee2950 con 0x556f62f89bf0 2024-09-10T08:12:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3acbe00640 10 client.16458 handle_command_reply: tid=0 2024-09-10T08:12:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3aefe01740 1 client.16458 shutdown 2024-09-10T08:12:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3aefe01740 1 -- 172.21.15.2:0/3365959604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f79d70 msgr2=0x556f62f77720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3aefe01740 1 --2- 172.21.15.2:0/3365959604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f79d70 0x556f62f77720 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009880 tx=0x7f3ae0012810 comp rx=0 tx=0).stop 2024-09-10T08:12:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3aefe01740 1 -- 172.21.15.2:0/3365959604 shutdown_connections 2024-09-10T08:12:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3aefe01740 1 --2- 172.21.15.2:0/3365959604 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556f62f89bf0 0x556f62f8bfe0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3aefe01740 1 --2- 172.21.15.2:0/3365959604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f79d70 0x556f62f77720 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3aefe01740 1 --2- 172.21.15.2:0/3365959604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3aefe01740 1 --2- 172.21.15.2:0/3365959604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f6eb90 0x556f62f76ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3aefe01740 1 -- 172.21.15.2:0/3365959604 >> 172.21.15.2:0/3365959604 conn(0x556f62f6b120 msgr2=0x556f62f70e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:24.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3acbe00640 0 client.16458 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:24.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3aefe01740 1 -- 172.21.15.2:0/3365959604 shutdown_connections 2024-09-10T08:12:24.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.146+0000 7f3aefe01740 1 -- 172.21.15.2:0/3365959604 wait complete. 2024-09-10T08:12:24.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.147+0000 7f3aefe01740 20 client.16458 trim_cache size 0 max 16384 2024-09-10T08:12:24.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.165+0000 7f3aefe01740 1 Processor -- start 2024-09-10T08:12:24.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.165+0000 7f3aefe01740 1 -- start start 2024-09-10T08:12:24.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.165+0000 7f3aefe01740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f78fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3aefe01740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f76580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3aefe01740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f79d70 0x556f62f91690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3aefe01740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81b7dc0 con 0x556f62f6eb90 2024-09-10T08:12:24.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3aefe01740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81b5690 con 0x556f62f79d70 2024-09-10T08:12:24.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3aefe01740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81b34e0 con 0x556f62f6f4b0 2024-09-10T08:12:24.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3ae6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f78fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3ae6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f78fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60526/0 (socket says 172.21.15.2:60526) 2024-09-10T08:12:24.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3ae6800640 1 -- 172.21.15.2:0/66666222 learned_addr learned my addr 172.21.15.2:0/66666222 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:24.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3adfe00640 1 --2- 172.21.15.2:0/66666222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f76580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3ae6800640 1 --2- 172.21.15.2:0/66666222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f79d70 0x556f62f91690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3ae6800640 1 -- 172.21.15.2:0/66666222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 msgr2=0x556f62f76580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:24.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3ae6800640 1 --2- 172.21.15.2:0/66666222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f76580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3ae6800640 1 -- 172.21.15.2:0/66666222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f79d70 msgr2=0x556f62f91690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:24.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3ae6800640 1 --2- 172.21.15.2:0/66666222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f79d70 0x556f62f91690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.166+0000 7f3ae6800640 1 -- 172.21.15.2:0/66666222 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f62f6fa20 con 0x556f62f6eb90 2024-09-10T08:12:24.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.167+0000 7f3ae6800640 1 --2- 172.21.15.2:0/66666222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f78fd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00044d0 tx=0x7f3ae0009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8691c21b7d08a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:24.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.167+0000 7f3ad6a00640 1 -- 172.21.15.2:0/66666222 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0016070 con 0x556f62f6eb90 2024-09-10T08:12:24.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.167+0000 7f3ad6a00640 1 -- 172.21.15.2:0/66666222 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae001bb40 con 0x556f62f6eb90 2024-09-10T08:12:24.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.167+0000 7f3ad6a00640 1 -- 172.21.15.2:0/66666222 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0003af0 con 0x556f62f6eb90 2024-09-10T08:12:24.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.167+0000 7f3aefe01740 1 -- 172.21.15.2:0/66666222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 msgr2=0x556f62f78fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:24.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.168+0000 7f3aefe01740 1 --2- 172.21.15.2:0/66666222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f78fd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00044d0 tx=0x7f3ae0009600 comp rx=0 tx=0).stop 2024-09-10T08:12:24.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.168+0000 7f3aefe01740 1 -- 172.21.15.2:0/66666222 shutdown_connections 2024-09-10T08:12:24.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.168+0000 7f3aefe01740 1 --2- 172.21.15.2:0/66666222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f79d70 0x556f62f91690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.168+0000 7f3aefe01740 1 --2- 172.21.15.2:0/66666222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f76580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.168+0000 7f3aefe01740 1 --2- 172.21.15.2:0/66666222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f78fd0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.168+0000 7f3aefe01740 1 -- 172.21.15.2:0/66666222 >> 172.21.15.2:0/66666222 conn(0x556f62f6b120 msgr2=0x556f62f77480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:24.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.168+0000 7f3aefe01740 1 -- 172.21.15.2:0/66666222 shutdown_connections 2024-09-10T08:12:24.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.168+0000 7f3aefe01740 1 -- 172.21.15.2:0/66666222 wait complete. 2024-09-10T08:12:24.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.168+0000 7f3aefe01740 1 -- start start 2024-09-10T08:12:24.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.168+0000 7f3aefe01740 1 Processor -- start 2024-09-10T08:12:24.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.169+0000 7f3aefe01740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f820e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.169+0000 7f3aefe01740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f82620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.169+0000 7f3aefe01740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f79d70 0x556f62f736e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.169+0000 7f3aefe01740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81b3b50 con 0x556f62f6eb90 2024-09-10T08:12:24.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.169+0000 7f3aefe01740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81b4fc0 con 0x556f62f79d70 2024-09-10T08:12:24.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.169+0000 7f3aefe01740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81b4220 con 0x556f62f6f4b0 2024-09-10T08:12:24.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.169+0000 7f3aefe01740 10 client.0 _ll_register_callbacks cb 0x7f3ae0013e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:24.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.169+0000 7f3ae6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f820e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.169+0000 7f3adfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f82620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.169+0000 7f3ae6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f820e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60540/0 (socket says 172.21.15.2:60540) 2024-09-10T08:12:24.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.169+0000 7f3ae6800640 1 -- 172.21.15.2:0/2906377991 learned_addr learned my addr 172.21.15.2:0/2906377991 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:24.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.169+0000 7f3ae6800640 1 --2- 172.21.15.2:0/2906377991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f79d70 0x556f62f736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.170+0000 7f3adfe00640 1 -- 172.21.15.2:0/2906377991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f79d70 msgr2=0x556f62f736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:24.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.170+0000 7f3adfe00640 1 --2- 172.21.15.2:0/2906377991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f79d70 0x556f62f736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.170+0000 7f3adfe00640 1 -- 172.21.15.2:0/2906377991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 msgr2=0x556f62f820e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:24.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.170+0000 7f3adfe00640 1 --2- 172.21.15.2:0/2906377991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f820e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.170+0000 7f3adfe00640 1 -- 172.21.15.2:0/2906377991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f62f73f10 con 0x556f62f6f4b0 2024-09-10T08:12:24.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.170+0000 7f3ae6800640 1 --2- 172.21.15.2:0/2906377991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f820e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:24.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.170+0000 7f3adfe00640 1 --2- 172.21.15.2:0/2906377991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f82620 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3ad002a9e0 tx=0x7f3ad00043d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a1481d6486010f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:24.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.170+0000 7f3acbe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:24.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.170+0000 7f3acbe00640 1 -- 172.21.15.2:0/2906377991 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0035040 con 0x556f62f6f4b0 2024-09-10T08:12:24.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.170+0000 7f3aefe01740 1 -- 172.21.15.2:0/2906377991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae001bb40 con 0x556f62f6f4b0 2024-09-10T08:12:24.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.170+0000 7f3acbe00640 1 -- 172.21.15.2:0/2906377991 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3ad0005450 con 0x556f62f6f4b0 2024-09-10T08:12:24.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.171+0000 7f3acbe00640 1 -- 172.21.15.2:0/2906377991 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ad0054050 con 0x556f62f6f4b0 2024-09-10T08:12:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.171+0000 7f3aefe01740 10 client.26458 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.171+0000 7f3aefe01740 1 -- 172.21.15.2:0/2906377991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556f62ff42f0 con 0x556f62f6f4b0 2024-09-10T08:12:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.171+0000 7f3acbe00640 1 -- 172.21.15.2:0/2906377991 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae001bb40 con 0x556f62f6f4b0 2024-09-10T08:12:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.171+0000 7f3acbe00640 1 -- 172.21.15.2:0/2906377991 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ad0003860 con 0x556f62f6f4b0 2024-09-10T08:12:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.171+0000 7f3aefe01740 10 client.26458 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.171+0000 7f3aefe01740 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:24.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.171+0000 7f3aefe01740 20 client.26458 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:24.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.171+0000 7f3aefe01740 1 --2- 172.21.15.2:0/2906377991 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556f62ff4aa0 0x556f62ff6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:24.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.171+0000 7f3aefe01740 4 client.26458 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:12:24.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.171+0000 7f3aefe01740 1 -- 172.21.15.2:0/2906377991 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556f62edce60 con 0x556f62ff4aa0 2024-09-10T08:12:24.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.172+0000 7f3ae6800640 1 --2- 172.21.15.2:0/2906377991 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556f62ff4aa0 0x556f62ff6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:24.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.172+0000 7f3ae6800640 1 --2- 172.21.15.2:0/2906377991 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556f62ff4aa0 0x556f62ff6ec0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e612e5442ccbcfca server_cookie=3c1e17d546e5e52e in_seq=0 out_seq=0 2024-09-10T08:12:24.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.172+0000 7f3acbe00640 10 client.26458 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:24.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.172+0000 7f3acbe00640 1 -- 172.21.15.2:0/2906377991 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556f62edce60 con 0x556f62ff4aa0 2024-09-10T08:12:24.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.172+0000 7f3acbe00640 10 client.26458 handle_command_reply: tid=0 2024-09-10T08:12:24.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.172+0000 7f3aefe01740 1 client.26458 shutdown 2024-09-10T08:12:24.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.173+0000 7f3aefe01740 1 -- 172.21.15.2:0/2906377991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 msgr2=0x556f62f82620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:24.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.173+0000 7f3aefe01740 1 --2- 172.21.15.2:0/2906377991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f82620 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3ad002a9e0 tx=0x7f3ad00043d0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.173+0000 7f3aefe01740 1 -- 172.21.15.2:0/2906377991 shutdown_connections 2024-09-10T08:12:24.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.173+0000 7f3aefe01740 1 --2- 172.21.15.2:0/2906377991 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556f62ff4aa0 0x556f62ff6ec0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.173+0000 7f3aefe01740 1 --2- 172.21.15.2:0/2906377991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f62f79d70 0x556f62f736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.173+0000 7f3aefe01740 1 --2- 172.21.15.2:0/2906377991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f62f6f4b0 0x556f62f82620 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.173+0000 7f3aefe01740 1 --2- 172.21.15.2:0/2906377991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f62f6eb90 0x556f62f820e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.173+0000 7f3acbe00640 0 client.26458 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:24.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.173+0000 7f3aefe01740 1 -- 172.21.15.2:0/2906377991 >> 172.21.15.2:0/2906377991 conn(0x556f62f6b120 msgr2=0x556f62f76c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:24.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.173+0000 7f3aefe01740 1 -- 172.21.15.2:0/2906377991 shutdown_connections 2024-09-10T08:12:24.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.173+0000 7f3aefe01740 1 -- 172.21.15.2:0/2906377991 wait complete. 2024-09-10T08:12:24.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.173+0000 7f3aefe01740 20 client.26458 trim_cache size 0 max 16384 2024-09-10T08:12:24.188 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:12:24.188 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:12:24.188 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:12:24.189 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:12:24.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.174+0000 7f3aed200640 1 -- 172.21.15.2:0/3162540177 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3acc0a2c00 msgr2=0x7f3acc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:24.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.174+0000 7f3aed200640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3acc0a2c00 0x7f3acc0a50c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3ad809bbb0 tx=0x7f3ad80d5530 comp rx=0 tx=0).stop 2024-09-10T08:12:24.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.174+0000 7f3aed200640 1 -- 172.21.15.2:0/3162540177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ae8076720 msgr2=0x7f3ae81a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:24.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.174+0000 7f3aed200640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ae8076720 0x7f3ae81a4730 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800e9f0 tx=0x7f3ad800edf0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.174+0000 7f3aed200640 1 -- 172.21.15.2:0/3162540177 shutdown_connections 2024-09-10T08:12:24.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.174+0000 7f3aed200640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8112a30 0x7f3ae819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.174+0000 7f3aed200640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3acc0a2c00 0x7f3acc0a50c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.174+0000 7f3aed200640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ae8077040 0x7f3ae81a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.174+0000 7f3aed200640 1 --2- 172.21.15.2:0/3162540177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ae8076720 0x7f3ae81a4730 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:24.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.174+0000 7f3aed200640 1 -- 172.21.15.2:0/3162540177 >> 172.21.15.2:0/3162540177 conn(0x7f3ae80ffa90 msgr2=0x7f3ae8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:24.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.174+0000 7f3aed200640 1 -- 172.21.15.2:0/3162540177 shutdown_connections 2024-09-10T08:12:24.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:24.174+0000 7f3aed200640 1 -- 172.21.15.2:0/3162540177 wait complete. 2024-09-10T08:12:24.680 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:24 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:12:24.909 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:24 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:12:24.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:24 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:12:25.177 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:12:25.178 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:12:25.558 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:25.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:25 smithi002 ceph-mon[30563]: pgmap v515: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-10T08:12:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:25 smithi155 ceph-mon[35405]: pgmap v515: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-10T08:12:25.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:25 smithi107 ceph-mon[35048]: pgmap v515: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-10T08:12:26.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.208+0000 7fca82a00640 1 -- 172.21.15.2:0/2898666045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca7c112870 msgr2=0x7fca7c114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.208+0000 7fca82a00640 1 --2- 172.21.15.2:0/2898666045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca7c112870 0x7fca7c114c60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fca74009ca0 tx=0x7fca7402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:12:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.208+0000 7fca82a00640 1 -- 172.21.15.2:0/2898666045 shutdown_connections 2024-09-10T08:12:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.208+0000 7fca82a00640 1 --2- 172.21.15.2:0/2898666045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca7c112870 0x7fca7c114c60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.208+0000 7fca82a00640 1 --2- 172.21.15.2:0/2898666045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c077040 0x7fca7c112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.208+0000 7fca82a00640 1 --2- 172.21.15.2:0/2898666045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca7c076720 0x7fca7c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.208+0000 7fca82a00640 1 -- 172.21.15.2:0/2898666045 >> 172.21.15.2:0/2898666045 conn(0x7fca7c0ff8d0 msgr2=0x7fca7c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.208+0000 7fca82a00640 1 -- 172.21.15.2:0/2898666045 shutdown_connections 2024-09-10T08:12:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.208+0000 7fca82a00640 1 -- 172.21.15.2:0/2898666045 wait complete. 2024-09-10T08:12:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.209+0000 7fca82a00640 1 Processor -- start 2024-09-10T08:12:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.209+0000 7fca82a00640 1 -- start start 2024-09-10T08:12:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.210+0000 7fca82a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca7c076720 0x7fca7c1a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.210+0000 7fca82a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca7c077040 0x7fca7c1a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.210+0000 7fca82a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c112870 0x7fca7c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.210+0000 7fca82a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1183b0 con 0x7fca7c077040 2024-09-10T08:12:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.210+0000 7fca82a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c118230 con 0x7fca7c112870 2024-09-10T08:12:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.210+0000 7fca82a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c118530 con 0x7fca7c076720 2024-09-10T08:12:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.210+0000 7fca80c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca7c077040 0x7fca7c1a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.210+0000 7fca80c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca7c077040 0x7fca7c1a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60562/0 (socket says 172.21.15.2:60562) 2024-09-10T08:12:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.210+0000 7fca80c00640 1 -- 172.21.15.2:0/3342277282 learned_addr learned my addr 172.21.15.2:0/3342277282 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.210+0000 7fca81600640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca7c076720 0x7fca7c1a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.210+0000 7fca82000640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c112870 0x7fca7c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:26.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.211+0000 7fca80c00640 1 -- 172.21.15.2:0/3342277282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca7c076720 msgr2=0x7fca7c1a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:26.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.211+0000 7fca80c00640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca7c076720 0x7fca7c1a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:26.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.211+0000 7fca80c00640 1 -- 172.21.15.2:0/3342277282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c112870 msgr2=0x7fca7c19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:26.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.211+0000 7fca80c00640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c112870 0x7fca7c19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:26.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.211+0000 7fca80c00640 1 -- 172.21.15.2:0/3342277282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c19ef00 con 0x7fca7c077040 2024-09-10T08:12:26.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.211+0000 7fca80c00640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca7c077040 0x7fca7c1a4b00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fca6800d950 tx=0x7fca6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:26.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.211+0000 7fca7a000640 1 -- 172.21.15.2:0/3342277282 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68017070 con 0x7fca7c077040 2024-09-10T08:12:26.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.211+0000 7fca7a000640 1 -- 172.21.15.2:0/3342277282 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca68009c30 con 0x7fca7c077040 2024-09-10T08:12:26.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.211+0000 7fca7a000640 1 -- 172.21.15.2:0/3342277282 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6800f6b0 con 0x7fca7c077040 2024-09-10T08:12:26.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.211+0000 7fca82a00640 1 -- 172.21.15.2:0/3342277282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7c19f1f0 con 0x7fca7c077040 2024-09-10T08:12:26.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.212+0000 7fca82a00640 1 -- 172.21.15.2:0/3342277282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7c1ab540 con 0x7fca7c077040 2024-09-10T08:12:26.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.213+0000 7fca82a00640 1 -- 172.21.15.2:0/3342277282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca7c077e60 con 0x7fca7c077040 2024-09-10T08:12:26.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.214+0000 7fca7a000640 1 -- 172.21.15.2:0/3342277282 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fca680109d0 con 0x7fca7c077040 2024-09-10T08:12:26.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.217+0000 7fca7a000640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca640a2c00 0x7fca640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:26.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.217+0000 7fca7a000640 1 -- 172.21.15.2:0/3342277282 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fca680cef30 con 0x7fca7c077040 2024-09-10T08:12:26.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.217+0000 7fca81600640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca640a2c00 0x7fca640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:26.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.217+0000 7fca81600640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca640a2c00 0x7fca640a50c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fca70005d00 tx=0x7fca7000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:26.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.221+0000 7fca7a000640 1 -- 172.21.15.2:0/3342277282 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca68009de0 con 0x7fca7c077040 2024-09-10T08:12:26.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.444+0000 7fca82a00640 1 -- 172.21.15.2:0/3342277282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca7c10da00 con 0x7fca7c077040 2024-09-10T08:12:26.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.446+0000 7fca7a000640 1 -- 172.21.15.2:0/3342277282 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fca6809b3f0 con 0x7fca7c077040 2024-09-10T08:12:26.446 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:12:26.446 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:12:26.446 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:12:26.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.448+0000 7fca82a00640 1 -- 172.21.15.2:0/3342277282 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca640a2c00 msgr2=0x7fca640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:26.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.448+0000 7fca82a00640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca640a2c00 0x7fca640a50c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fca70005d00 tx=0x7fca7000b410 comp rx=0 tx=0).stop 2024-09-10T08:12:26.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.448+0000 7fca82a00640 1 -- 172.21.15.2:0/3342277282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca7c077040 msgr2=0x7fca7c1a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:26.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.448+0000 7fca82a00640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca7c077040 0x7fca7c1a4b00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fca6800d950 tx=0x7fca6800de10 comp rx=0 tx=0).stop 2024-09-10T08:12:26.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.448+0000 7fca82a00640 1 -- 172.21.15.2:0/3342277282 shutdown_connections 2024-09-10T08:12:26.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.448+0000 7fca82a00640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c112870 0x7fca7c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:26.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.448+0000 7fca82a00640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca7c077040 0x7fca7c1a4b00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:26.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.448+0000 7fca82a00640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca640a2c00 0x7fca640a50c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:26.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.448+0000 7fca82a00640 1 --2- 172.21.15.2:0/3342277282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca7c076720 0x7fca7c1a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:26.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.448+0000 7fca82a00640 1 -- 172.21.15.2:0/3342277282 >> 172.21.15.2:0/3342277282 conn(0x7fca7c0ff8d0 msgr2=0x7fca7c101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:26.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.449+0000 7fca82a00640 1 -- 172.21.15.2:0/3342277282 shutdown_connections 2024-09-10T08:12:26.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:26.449+0000 7fca82a00640 1 -- 172.21.15.2:0/3342277282 wait complete. 2024-09-10T08:12:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.194+0000 7f3116c00640 1 -- 172.21.15.2:0/1398689740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3110101580 msgr2=0x7f3110101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.194+0000 7f3116c00640 1 --2- 172.21.15.2:0/1398689740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3110101580 0x7f3110101a00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009900 tx=0x7f30fc02a410 comp rx=0 tx=0).stop 2024-09-10T08:12:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.194+0000 7f3116c00640 1 -- 172.21.15.2:0/1398689740 shutdown_connections 2024-09-10T08:12:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.194+0000 7f3116c00640 1 --2- 172.21.15.2:0/1398689740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3110101f40 0x7f3110109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.194+0000 7f3116c00640 1 --2- 172.21.15.2:0/1398689740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3110101580 0x7f3110101a00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.194+0000 7f3116c00640 1 --2- 172.21.15.2:0/1398689740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3110103050 0x7f3110103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.194+0000 7f3116c00640 1 -- 172.21.15.2:0/1398689740 >> 172.21.15.2:0/1398689740 conn(0x7f31100fd0d0 msgr2=0x7f31100ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.194+0000 7f3116c00640 1 -- 172.21.15.2:0/1398689740 shutdown_connections 2024-09-10T08:12:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.194+0000 7f3116c00640 1 -- 172.21.15.2:0/1398689740 wait complete. 2024-09-10T08:12:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.195+0000 7f3116c00640 1 Processor -- start 2024-09-10T08:12:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.195+0000 7f3116c00640 1 -- start start 2024-09-10T08:12:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.195+0000 7f3116c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3110101580 0x7f3110075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3116c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3110101f40 0x7f31100745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3116c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3110103050 0x7f3110074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3116c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f311010d740 con 0x7f3110101580 2024-09-10T08:12:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3116c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f311010d5c0 con 0x7f3110101f40 2024-09-10T08:12:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3116c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f311010d8c0 con 0x7f3110103050 2024-09-10T08:12:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3115800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3110101580 0x7f3110075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3114e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3110101f40 0x7f31100745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3114e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3110101f40 0x7f31100745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:59856/0 (socket says 172.21.15.2:59856) 2024-09-10T08:12:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3114e00640 1 -- 172.21.15.2:0/1906926683 learned_addr learned my addr 172.21.15.2:0/1906926683 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3116200640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3110103050 0x7f3110074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3115800640 1 -- 172.21.15.2:0/1906926683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3110103050 msgr2=0x7f3110074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3115800640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3110103050 0x7f3110074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3115800640 1 -- 172.21.15.2:0/1906926683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3110101f40 msgr2=0x7f31100745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3115800640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3110101f40 0x7f31100745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3115800640 1 -- 172.21.15.2:0/1906926683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31100753b0 con 0x7f3110101580 2024-09-10T08:12:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.196+0000 7f3116200640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3110103050 0x7f3110074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.197+0000 7f3115800640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3110101580 0x7f3110075fa0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f310400e9c0 tx=0x7f310400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.197+0000 7f310e000640 1 -- 172.21.15.2:0/1906926683 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310400ccc0 con 0x7f3110101580 2024-09-10T08:12:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.197+0000 7f310e000640 1 -- 172.21.15.2:0/1906926683 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31040035f0 con 0x7f3110101580 2024-09-10T08:12:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.197+0000 7f310e000640 1 -- 172.21.15.2:0/1906926683 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310400ce60 con 0x7f3110101580 2024-09-10T08:12:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.197+0000 7f3116c00640 1 -- 172.21.15.2:0/1906926683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31101a1f10 con 0x7f3110101580 2024-09-10T08:12:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.197+0000 7f3116c00640 1 -- 172.21.15.2:0/1906926683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31101a2370 con 0x7f3110101580 2024-09-10T08:12:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.200+0000 7f311b232740 1 Processor -- start 2024-09-10T08:12:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.200+0000 7f310e000640 1 -- 172.21.15.2:0/1906926683 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f31040037a0 con 0x7f3110101580 2024-09-10T08:12:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.200+0000 7f311b232740 1 -- start start 2024-09-10T08:12:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.200+0000 7f311b232740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff0260 0x558c87ff0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.200+0000 7f311b232740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 0x558c87ff5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.200+0000 7f311b232740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff5c30 0x558c87ff80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f311b232740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558c87f55100 con 0x558c87ff5c30 2024-09-10T08:12:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f311b232740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558c87efd1b0 con 0x558c87ff0ba0 2024-09-10T08:12:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f311b232740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558c87f510c0 con 0x558c87ff0260 2024-09-10T08:12:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f3114e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff5c30 0x558c87ff80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f3114e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff5c30 0x558c87ff80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34400/0 (socket says 172.21.15.2:34400) 2024-09-10T08:12:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f3114e00640 1 -- 172.21.15.2:0/370848119 learned_addr learned my addr 172.21.15.2:0/370848119 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f3116200640 1 --2- 172.21.15.2:0/370848119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff0260 0x558c87ff0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f3115800640 1 --2- 172.21.15.2:0/370848119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 0x558c87ff5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f3114e00640 1 -- 172.21.15.2:0/370848119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff0260 msgr2=0x558c87ff0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f3114e00640 1 --2- 172.21.15.2:0/370848119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff0260 0x558c87ff0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f3114e00640 1 -- 172.21.15.2:0/370848119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 msgr2=0x558c87ff5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f3114e00640 1 --2- 172.21.15.2:0/370848119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 0x558c87ff5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f3114e00640 1 -- 172.21.15.2:0/370848119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c87c68210 con 0x558c87ff5c30 2024-09-10T08:12:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.201+0000 7f3116200640 1 --2- 172.21.15.2:0/370848119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff0260 0x558c87ff0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.202+0000 7f3114e00640 1 --2- 172.21.15.2:0/370848119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff5c30 0x558c87ff80f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f30fc02aa00 tx=0x7f30fc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4af9c1337a430bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.202+0000 7f310e000640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f30f80a2c00 0x7f30f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.202+0000 7f3102a00640 1 -- 172.21.15.2:0/370848119 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc040070 con 0x558c87ff5c30 2024-09-10T08:12:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.202+0000 7f3102a00640 1 -- 172.21.15.2:0/370848119 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30fc005500 con 0x558c87ff5c30 2024-09-10T08:12:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.202+0000 7f3102a00640 1 -- 172.21.15.2:0/370848119 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc035040 con 0x558c87ff5c30 2024-09-10T08:12:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.202+0000 7f310e000640 1 -- 172.21.15.2:0/1906926683 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f31040cf8e0 con 0x7f3110101580 2024-09-10T08:12:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.202+0000 7f3116200640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f30f80a2c00 0x7f30f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.203+0000 7f311b232740 1 -- 172.21.15.2:0/370848119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff5c30 msgr2=0x558c87ff80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.203+0000 7f311b232740 1 --2- 172.21.15.2:0/370848119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff5c30 0x558c87ff80f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f30fc02aa00 tx=0x7f30fc004040 comp rx=0 tx=0).stop 2024-09-10T08:12:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.203+0000 7f3116200640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f30f80a2c00 0x7f30f80a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3108009fd0 tx=0x7f3108009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.203+0000 7f311b232740 1 -- 172.21.15.2:0/370848119 shutdown_connections 2024-09-10T08:12:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.203+0000 7f311b232740 1 --2- 172.21.15.2:0/370848119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff5c30 0x558c87ff80f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.203+0000 7f311b232740 1 --2- 172.21.15.2:0/370848119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 0x558c87ff5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.203+0000 7f311b232740 1 --2- 172.21.15.2:0/370848119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff0260 0x558c87ff0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.203+0000 7f311b232740 1 -- 172.21.15.2:0/370848119 >> 172.21.15.2:0/370848119 conn(0x558c87feb8d0 msgr2=0x558c87febcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.203+0000 7f311b232740 1 -- 172.21.15.2:0/370848119 shutdown_connections 2024-09-10T08:12:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.203+0000 7f311b232740 1 -- 172.21.15.2:0/370848119 wait complete. 2024-09-10T08:12:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.203+0000 7f311b232740 1 -- start start 2024-09-10T08:12:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.204+0000 7f311b232740 1 Processor -- start 2024-09-10T08:12:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.204+0000 7f311b232740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 0x558c87ffa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.204+0000 7f311b232740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 0x558c87ffa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.204+0000 7f311b232740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff5c30 0x558c88001930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.204+0000 7f311b232740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558c87ff8ba0 con 0x558c87ff0260 2024-09-10T08:12:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.204+0000 7f311b232740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558c87ede090 con 0x558c87ff0ba0 2024-09-10T08:12:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.204+0000 7f311b232740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558c87ff8d20 con 0x558c87ff5c30 2024-09-10T08:12:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.204+0000 7f311b232740 10 client.0 _ll_register_callbacks cb 0x558c87f32f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.204+0000 7f3115800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 0x558c87ffa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.204+0000 7f3115800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 0x558c87ffa3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34402/0 (socket says 172.21.15.2:34402) 2024-09-10T08:12:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.204+0000 7f3115800640 1 -- 172.21.15.2:0/1951601130 learned_addr learned my addr 172.21.15.2:0/1951601130 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.204+0000 7f3114e00640 1 --2- 172.21.15.2:0/1951601130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 0x558c87ffa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.204+0000 7f3116200640 1 --2- 172.21.15.2:0/1951601130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff5c30 0x558c88001930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.205+0000 7f3115800640 1 -- 172.21.15.2:0/1951601130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff5c30 msgr2=0x558c88001930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.205+0000 7f3115800640 1 --2- 172.21.15.2:0/1951601130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff5c30 0x558c88001930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.205+0000 7f3115800640 1 -- 172.21.15.2:0/1951601130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 msgr2=0x558c87ffa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.205+0000 7f3115800640 1 --2- 172.21.15.2:0/1951601130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 0x558c87ffa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.205+0000 7f3115800640 1 -- 172.21.15.2:0/1951601130 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c87d34620 con 0x558c87ff0260 2024-09-10T08:12:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.205+0000 7f3115800640 1 --2- 172.21.15.2:0/1951601130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 0x558c87ffa3b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3104009770 tx=0x7f3104098050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9b798b0823f649 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.205+0000 7f30f7400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:12:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.205+0000 7f30f7400640 1 -- 172.21.15.2:0/1951601130 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310409c3f0 con 0x558c87ff0260 2024-09-10T08:12:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.205+0000 7f30f7400640 1 -- 172.21.15.2:0/1951601130 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f310409b140 con 0x558c87ff0260 2024-09-10T08:12:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.205+0000 7f311b232740 1 -- 172.21.15.2:0/1951601130 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30fc005500 con 0x558c87ff0260 2024-09-10T08:12:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.205+0000 7f30f7400640 1 -- 172.21.15.2:0/1951601130 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f310409bcf0 con 0x558c87ff0260 2024-09-10T08:12:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.205+0000 7f311b232740 10 client.16482 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.205+0000 7f311b232740 1 -- 172.21.15.2:0/1951601130 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c88004840 con 0x558c87ff0260 2024-09-10T08:12:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.206+0000 7f30f7400640 1 -- 172.21.15.2:0/1951601130 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30fc005500 con 0x558c87ff0260 2024-09-10T08:12:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.206+0000 7f30f7400640 1 -- 172.21.15.2:0/1951601130 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f31040b3030 con 0x558c87ff0260 2024-09-10T08:12:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.206+0000 7f311b232740 10 client.16482 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.206+0000 7f311b232740 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.206+0000 7f311b232740 20 client.16482 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.206+0000 7f311b232740 1 --2- 172.21.15.2:0/1951601130 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558c88005ec0 0x558c880082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.206+0000 7f311b232740 4 client.16482 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.206+0000 7f311b232740 1 -- 172.21.15.2:0/1951601130 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c87f62830 con 0x558c88005ec0 2024-09-10T08:12:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.206+0000 7f3114e00640 1 --2- 172.21.15.2:0/1951601130 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558c88005ec0 0x558c880082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.207+0000 7f3114e00640 1 --2- 172.21.15.2:0/1951601130 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558c88005ec0 0x558c880082b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67bf554be5f63ea2 server_cookie=1548a8e7556fa0e7 in_seq=0 out_seq=0 2024-09-10T08:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.207+0000 7f30f7400640 10 client.16482 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f30f7400640 1 -- 172.21.15.2:0/1951601130 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c87f62830 con 0x558c88005ec0 2024-09-10T08:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f30f7400640 10 client.16482 handle_command_reply: tid=0 2024-09-10T08:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f311b232740 1 client.16482 shutdown 2024-09-10T08:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f311b232740 1 -- 172.21.15.2:0/1951601130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 msgr2=0x558c87ffa3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f311b232740 1 --2- 172.21.15.2:0/1951601130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 0x558c87ffa3b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3104009770 tx=0x7f3104098050 comp rx=0 tx=0).stop 2024-09-10T08:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f311b232740 1 -- 172.21.15.2:0/1951601130 shutdown_connections 2024-09-10T08:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f311b232740 1 --2- 172.21.15.2:0/1951601130 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558c88005ec0 0x558c880082b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f311b232740 1 --2- 172.21.15.2:0/1951601130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff5c30 0x558c88001930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f311b232740 1 --2- 172.21.15.2:0/1951601130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 0x558c87ffa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f311b232740 1 --2- 172.21.15.2:0/1951601130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 0x558c87ffa3b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f30f7400640 0 client.16482 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f311b232740 1 -- 172.21.15.2:0/1951601130 >> 172.21.15.2:0/1951601130 conn(0x558c87feb8d0 msgr2=0x558c87fed0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f311b232740 1 -- 172.21.15.2:0/1951601130 shutdown_connections 2024-09-10T08:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f311b232740 1 -- 172.21.15.2:0/1951601130 wait complete. 2024-09-10T08:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.208+0000 7f311b232740 20 client.16482 trim_cache size 0 max 16384 2024-09-10T08:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.223+0000 7f311b232740 1 Processor -- start 2024-09-10T08:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.223+0000 7f311b232740 1 -- start start 2024-09-10T08:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.223+0000 7f311b232740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0260 0x558c87fea900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.223+0000 7f311b232740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff0ba0 0x558c87feae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f311b232740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff5c30 0x558c880018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f311b232740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f31101af190 con 0x558c87ff5c30 2024-09-10T08:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f311b232740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f31101aca60 con 0x558c87ff0260 2024-09-10T08:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f311b232740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f31101aa8b0 con 0x558c87ff0ba0 2024-09-10T08:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f3116200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff5c30 0x558c880018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f3115800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0260 0x558c87fea900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f3116200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff5c30 0x558c880018f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34416/0 (socket says 172.21.15.2:34416) 2024-09-10T08:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f3115800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0260 0x558c87fea900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:59884/0 (socket says 172.21.15.2:59884) 2024-09-10T08:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f3116200640 1 -- 172.21.15.2:0/158457890 learned_addr learned my addr 172.21.15.2:0/158457890 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f3114e00640 1 --2- 172.21.15.2:0/158457890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff0ba0 0x558c87feae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f3115800640 1 -- 172.21.15.2:0/158457890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff0ba0 msgr2=0x558c87feae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f3115800640 1 --2- 172.21.15.2:0/158457890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff0ba0 0x558c87feae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f3115800640 1 -- 172.21.15.2:0/158457890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff5c30 msgr2=0x558c880018f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f3115800640 1 --2- 172.21.15.2:0/158457890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff5c30 0x558c880018f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f3115800640 1 -- 172.21.15.2:0/158457890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c87feb6f0 con 0x558c87ff0260 2024-09-10T08:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.224+0000 7f3115800640 1 --2- 172.21.15.2:0/158457890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0260 0x558c87fea900 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3104098150 tx=0x7f3104003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fea02e0591e085f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.225+0000 7f3102000640 1 -- 172.21.15.2:0/158457890 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310409b830 con 0x558c87ff0260 2024-09-10T08:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.225+0000 7f3102000640 1 -- 172.21.15.2:0/158457890 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31040d2e50 con 0x558c87ff0260 2024-09-10T08:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.225+0000 7f3102000640 1 -- 172.21.15.2:0/158457890 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310409b380 con 0x558c87ff0260 2024-09-10T08:12:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.225+0000 7f311b232740 1 -- 172.21.15.2:0/158457890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0260 msgr2=0x558c87fea900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.225+0000 7f311b232740 1 --2- 172.21.15.2:0/158457890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0260 0x558c87fea900 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3104098150 tx=0x7f3104003e10 comp rx=0 tx=0).stop 2024-09-10T08:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.225+0000 7f311b232740 1 -- 172.21.15.2:0/158457890 shutdown_connections 2024-09-10T08:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.225+0000 7f311b232740 1 --2- 172.21.15.2:0/158457890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff5c30 0x558c880018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.225+0000 7f311b232740 1 --2- 172.21.15.2:0/158457890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff0ba0 0x558c87feae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.225+0000 7f311b232740 1 --2- 172.21.15.2:0/158457890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0260 0x558c87fea900 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.225+0000 7f311b232740 1 -- 172.21.15.2:0/158457890 >> 172.21.15.2:0/158457890 conn(0x558c87feb8d0 msgr2=0x558c87fea1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.225+0000 7f311b232740 1 -- 172.21.15.2:0/158457890 shutdown_connections 2024-09-10T08:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.225+0000 7f311b232740 1 -- 172.21.15.2:0/158457890 wait complete. 2024-09-10T08:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.226+0000 7f311b232740 1 -- start start 2024-09-10T08:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.226+0000 7f311b232740 1 Processor -- start 2024-09-10T08:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.226+0000 7f311b232740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 0x558c87ffeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.226+0000 7f311b232740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 0x558c87fff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.226+0000 7f311b232740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff5c30 0x558c880003f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.226+0000 7f311b232740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f31101aaf20 con 0x558c87ff0260 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.226+0000 7f311b232740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f31101ac390 con 0x558c87ff0ba0 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.226+0000 7f311b232740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f31101ab5f0 con 0x558c87ff5c30 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.226+0000 7f311b232740 10 client.0 _ll_register_callbacks cb 0x7f31040d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.226+0000 7f3115800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 0x558c87ffeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.226+0000 7f3114e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 0x558c87fff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.226+0000 7f3115800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 0x558c87ffeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34418/0 (socket says 172.21.15.2:34418) 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.226+0000 7f3115800640 1 -- 172.21.15.2:0/2732006826 learned_addr learned my addr 172.21.15.2:0/2732006826 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f3116200640 1 --2- 172.21.15.2:0/2732006826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff5c30 0x558c880003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f3115800640 1 -- 172.21.15.2:0/2732006826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff5c30 msgr2=0x558c880003f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f3115800640 1 --2- 172.21.15.2:0/2732006826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff5c30 0x558c880003f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f3115800640 1 -- 172.21.15.2:0/2732006826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 msgr2=0x558c87fff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f3115800640 1 --2- 172.21.15.2:0/2732006826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 0x558c87fff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f3115800640 1 -- 172.21.15.2:0/2732006826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c88000a00 con 0x558c87ff0260 2024-09-10T08:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f3115800640 1 --2- 172.21.15.2:0/2732006826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 0x558c87ffeef0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3104012020 tx=0x7f310409b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785b781118e32b9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f30f7400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f30f7400640 1 -- 172.21.15.2:0/2732006826 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310407e020 con 0x558c87ff0260 2024-09-10T08:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f30f7400640 1 -- 172.21.15.2:0/2732006826 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3104004000 con 0x558c87ff0260 2024-09-10T08:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f311b232740 1 -- 172.21.15.2:0/2732006826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31040d2e50 con 0x558c87ff0260 2024-09-10T08:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f30f7400640 1 -- 172.21.15.2:0/2732006826 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f310409b830 con 0x558c87ff0260 2024-09-10T08:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f311b232740 10 client.16488 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.227+0000 7f311b232740 1 -- 172.21.15.2:0/2732006826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c88070bb0 con 0x558c87ff0260 2024-09-10T08:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.228+0000 7f30f7400640 1 -- 172.21.15.2:0/2732006826 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31040d2e50 con 0x558c87ff0260 2024-09-10T08:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.228+0000 7f30f7400640 1 -- 172.21.15.2:0/2732006826 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f31040b3070 con 0x558c87ff0260 2024-09-10T08:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.228+0000 7f311b232740 10 client.16488 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.228+0000 7f311b232740 10 client.16488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.228+0000 7f311b232740 20 client.16488 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.228+0000 7f311b232740 1 --2- 172.21.15.2:0/2732006826 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558c880713c0 0x558c880737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.228+0000 7f311b232740 4 client.16488 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:12:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.228+0000 7f311b232740 1 -- 172.21.15.2:0/2732006826 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c87f5cb80 con 0x558c880713c0 2024-09-10T08:12:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.228+0000 7f3114e00640 1 --2- 172.21.15.2:0/2732006826 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558c880713c0 0x558c880737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.229+0000 7f3114e00640 1 --2- 172.21.15.2:0/2732006826 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558c880713c0 0x558c880737e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6b9e5e25261d189 server_cookie=ff627bd7b7db2c89 in_seq=0 out_seq=0 2024-09-10T08:12:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.229+0000 7f30f7400640 10 client.16488 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.241+0000 7f30f7400640 1 -- 172.21.15.2:0/2732006826 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c87f5cb80 con 0x558c880713c0 2024-09-10T08:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.241+0000 7f30f7400640 10 client.16488 handle_command_reply: tid=0 2024-09-10T08:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.241+0000 7f311b232740 1 client.16488 shutdown 2024-09-10T08:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f311b232740 1 -- 172.21.15.2:0/2732006826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 msgr2=0x558c87ffeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f311b232740 1 --2- 172.21.15.2:0/2732006826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 0x558c87ffeef0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3104012020 tx=0x7f310409b590 comp rx=0 tx=0).stop 2024-09-10T08:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f311b232740 1 -- 172.21.15.2:0/2732006826 shutdown_connections 2024-09-10T08:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f311b232740 1 --2- 172.21.15.2:0/2732006826 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558c880713c0 0x558c880737e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f311b232740 1 --2- 172.21.15.2:0/2732006826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c87ff5c30 0x558c880003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f311b232740 1 --2- 172.21.15.2:0/2732006826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c87ff0ba0 0x558c87fff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f311b232740 1 --2- 172.21.15.2:0/2732006826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c87ff0260 0x558c87ffeef0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f30f7400640 0 client.16488 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f311b232740 1 -- 172.21.15.2:0/2732006826 >> 172.21.15.2:0/2732006826 conn(0x558c87feb8d0 msgr2=0x558c87feaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f311b232740 1 -- 172.21.15.2:0/2732006826 shutdown_connections 2024-09-10T08:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f311b232740 1 -- 172.21.15.2:0/2732006826 wait complete. 2024-09-10T08:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f311b232740 20 client.16488 trim_cache size 0 max 16384 2024-09-10T08:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f3116c00640 1 -- 172.21.15.2:0/1906926683 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f30f80a2c00 msgr2=0x7f30f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.242+0000 7f3116c00640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f30f80a2c00 0x7f30f80a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3108009fd0 tx=0x7f3108009500 comp rx=0 tx=0).stop 2024-09-10T08:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.243+0000 7f3116c00640 1 -- 172.21.15.2:0/1906926683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3110101580 msgr2=0x7f3110075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.243+0000 7f3116c00640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3110101580 0x7f3110075fa0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f310400e9c0 tx=0x7f310400ee80 comp rx=0 tx=0).stop 2024-09-10T08:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.243+0000 7f3116c00640 1 -- 172.21.15.2:0/1906926683 shutdown_connections 2024-09-10T08:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.243+0000 7f3116c00640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f30f80a2c00 0x7f30f80a50c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.243+0000 7f3116c00640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3110103050 0x7f3110074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.243+0000 7f3116c00640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3110101f40 0x7f31100745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.243+0000 7f3116c00640 1 --2- 172.21.15.2:0/1906926683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3110101580 0x7f3110075fa0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.243+0000 7f3116c00640 1 -- 172.21.15.2:0/1906926683 >> 172.21.15.2:0/1906926683 conn(0x7f31100fd0d0 msgr2=0x7f3110105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.243+0000 7f3116c00640 1 -- 172.21.15.2:0/1906926683 shutdown_connections 2024-09-10T08:12:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:27.243+0000 7f3116c00640 1 -- 172.21.15.2:0/1906926683 wait complete. 2024-09-10T08:12:27.402 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:12:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:27 smithi002 ceph-mon[30563]: pgmap v516: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-10T08:12:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:27 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3342277282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:27.781 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:27.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:27 smithi155 ceph-mon[35405]: pgmap v516: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-10T08:12:27.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:27 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3342277282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:27.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:27 smithi107 ceph-mon[35048]: pgmap v516: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-10T08:12:27.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:27 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3342277282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:28.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.344+0000 7f35b9c00640 1 -- 172.21.15.2:0/1001983766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b410bfc0 msgr2=0x7f35b410e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:28.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.344+0000 7f35b9c00640 1 --2- 172.21.15.2:0/1001983766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b410bfc0 0x7f35b410e460 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009be0 tx=0x7f35ac02a750 comp rx=0 tx=0).stop 2024-09-10T08:12:28.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.344+0000 7f35b9c00640 1 -- 172.21.15.2:0/1001983766 shutdown_connections 2024-09-10T08:12:28.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.344+0000 7f35b9c00640 1 --2- 172.21.15.2:0/1001983766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b410bfc0 0x7f35b410e460 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.344+0000 7f35b9c00640 1 --2- 172.21.15.2:0/1001983766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35b41034b0 0x7f35b410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.344+0000 7f35b9c00640 1 --2- 172.21.15.2:0/1001983766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b4102b90 0x7f35b4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.344+0000 7f35b9c00640 1 -- 172.21.15.2:0/1001983766 >> 172.21.15.2:0/1001983766 conn(0x7f35b40fc630 msgr2=0x7f35b40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:28.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.344+0000 7f35b9c00640 1 -- 172.21.15.2:0/1001983766 shutdown_connections 2024-09-10T08:12:28.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.344+0000 7f35b9c00640 1 -- 172.21.15.2:0/1001983766 wait complete. 2024-09-10T08:12:28.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.345+0000 7f35b9c00640 1 Processor -- start 2024-09-10T08:12:28.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.345+0000 7f35b9c00640 1 -- start start 2024-09-10T08:12:28.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.345+0000 7f35b9c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b4102b90 0x7f35b419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.345+0000 7f35b9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b41034b0 0x7f35b419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.345+0000 7f35b9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35b410bfc0 0x7f35b419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.345+0000 7f35b9c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4111d60 con 0x7f35b4102b90 2024-09-10T08:12:28.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.345+0000 7f35b9c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4111be0 con 0x7f35b41034b0 2024-09-10T08:12:28.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.345+0000 7f35b9c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4111ee0 con 0x7f35b410bfc0 2024-09-10T08:12:28.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.346+0000 7f35ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b41034b0 0x7f35b419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.346+0000 7f35b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b4102b90 0x7f35b419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.346+0000 7f35ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b41034b0 0x7f35b419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:59904/0 (socket says 172.21.15.2:59904) 2024-09-10T08:12:28.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.346+0000 7f35ab400640 1 -- 172.21.15.2:0/66495202 learned_addr learned my addr 172.21.15.2:0/66495202 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:28.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.346+0000 7f35b9200640 1 --2- 172.21.15.2:0/66495202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35b410bfc0 0x7f35b419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.346+0000 7f35ab400640 1 -- 172.21.15.2:0/66495202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35b410bfc0 msgr2=0x7f35b419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:28.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.346+0000 7f35ab400640 1 --2- 172.21.15.2:0/66495202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35b410bfc0 0x7f35b419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.346+0000 7f35ab400640 1 -- 172.21.15.2:0/66495202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b4102b90 msgr2=0x7f35b419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:28.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.346+0000 7f35ab400640 1 --2- 172.21.15.2:0/66495202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b4102b90 0x7f35b419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.346+0000 7f35ab400640 1 -- 172.21.15.2:0/66495202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b419f4a0 con 0x7f35b41034b0 2024-09-10T08:12:28.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.346+0000 7f35ab400640 1 --2- 172.21.15.2:0/66495202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b41034b0 0x7f35b419aa30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f359c00e9f0 tx=0x7f359c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:28.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.347+0000 7f35b1600640 1 -- 172.21.15.2:0/66495202 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c00ccc0 con 0x7f35b41034b0 2024-09-10T08:12:28.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.347+0000 7f35b9c00640 1 -- 172.21.15.2:0/66495202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b419f790 con 0x7f35b41034b0 2024-09-10T08:12:28.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.347+0000 7f35b1600640 1 -- 172.21.15.2:0/66495202 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f359c00ce60 con 0x7f35b41034b0 2024-09-10T08:12:28.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.347+0000 7f35b1600640 1 -- 172.21.15.2:0/66495202 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c010690 con 0x7f35b41034b0 2024-09-10T08:12:28.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.347+0000 7f35b9c00640 1 -- 172.21.15.2:0/66495202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b41a7070 con 0x7f35b41034b0 2024-09-10T08:12:28.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.350+0000 7f35bde49740 1 Processor -- start 2024-09-10T08:12:28.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.350+0000 7f35b1600640 1 -- 172.21.15.2:0/66495202 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f359c0046c0 con 0x7f35b41034b0 2024-09-10T08:12:28.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.350+0000 7f35bde49740 1 -- start start 2024-09-10T08:12:28.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35bde49740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x559872034220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35bde49740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x55987203eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35bde49740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 0x559872041410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35bde49740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559871f9a3e0 con 0x559872033e40 2024-09-10T08:12:28.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35bde49740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559871f41d80 con 0x559872034760 2024-09-10T08:12:28.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35bde49740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559871f963a0 con 0x55987203f020 2024-09-10T08:12:28.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35b9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x559872034220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35b9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x559872034220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34450/0 (socket says 172.21.15.2:34450) 2024-09-10T08:12:28.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35b9200640 1 -- 172.21.15.2:0/942022895 learned_addr learned my addr 172.21.15.2:0/942022895 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:28.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35b3e00640 1 --2- 172.21.15.2:0/942022895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x55987203eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35ab400640 1 --2- 172.21.15.2:0/942022895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 0x559872041410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35b3e00640 1 -- 172.21.15.2:0/942022895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 msgr2=0x559872041410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:28.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35b3e00640 1 --2- 172.21.15.2:0/942022895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 0x559872041410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35b3e00640 1 -- 172.21.15.2:0/942022895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 msgr2=0x559872034220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:28.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35b3e00640 1 --2- 172.21.15.2:0/942022895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x559872034220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.351+0000 7f35b3e00640 1 -- 172.21.15.2:0/942022895 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559871f60ae0 con 0x559872034760 2024-09-10T08:12:28.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.352+0000 7f35b3e00640 1 --2- 172.21.15.2:0/942022895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x55987203eae0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f35a4005fd0 tx=0x7f35a4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9aff1ccb59b9e83 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:28.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.352+0000 7f35b1600640 1 --2- 172.21.15.2:0/66495202 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f35980a2b80 0x7f35980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.352+0000 7f35a9600640 1 -- 172.21.15.2:0/942022895 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4004020 con 0x559872034760 2024-09-10T08:12:28.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.352+0000 7f35b1600640 1 -- 172.21.15.2:0/66495202 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f359c0cf740 con 0x7f35b41034b0 2024-09-10T08:12:28.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.352+0000 7f35a9600640 1 -- 172.21.15.2:0/942022895 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a40041c0 con 0x559872034760 2024-09-10T08:12:28.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.352+0000 7f35a9600640 1 -- 172.21.15.2:0/942022895 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4019890 con 0x559872034760 2024-09-10T08:12:28.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.352+0000 7f35b9200640 1 --2- 172.21.15.2:0/66495202 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f35980a2b80 0x7f35980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.353+0000 7f35bde49740 1 -- 172.21.15.2:0/942022895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 msgr2=0x55987203eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:28.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.353+0000 7f35bde49740 1 --2- 172.21.15.2:0/942022895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x55987203eae0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f35a4005fd0 tx=0x7f35a4011810 comp rx=0 tx=0).stop 2024-09-10T08:12:28.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.353+0000 7f35bde49740 1 -- 172.21.15.2:0/942022895 shutdown_connections 2024-09-10T08:12:28.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.353+0000 7f35bde49740 1 --2- 172.21.15.2:0/942022895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 0x559872041410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.353+0000 7f35bde49740 1 --2- 172.21.15.2:0/942022895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x55987203eae0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.353+0000 7f35bde49740 1 --2- 172.21.15.2:0/942022895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x559872034220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.353+0000 7f35bde49740 1 -- 172.21.15.2:0/942022895 >> 172.21.15.2:0/942022895 conn(0x5598720303d0 msgr2=0x5598720307b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:28.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.353+0000 7f35bde49740 1 -- 172.21.15.2:0/942022895 shutdown_connections 2024-09-10T08:12:28.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.353+0000 7f35b9200640 1 --2- 172.21.15.2:0/66495202 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f35980a2b80 0x7f35980a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f35ac033000 tx=0x7f35ac009840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:28.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.353+0000 7f35bde49740 1 -- 172.21.15.2:0/942022895 wait complete. 2024-09-10T08:12:28.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.354+0000 7f35bde49740 1 -- start start 2024-09-10T08:12:28.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.354+0000 7f35bde49740 1 Processor -- start 2024-09-10T08:12:28.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.354+0000 7f35bde49740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x55987203bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.354+0000 7f35bde49740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x55987203c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.354+0000 7f35bde49740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 0x55987203c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.354+0000 7f35bde49740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559872041bf0 con 0x559872033e40 2024-09-10T08:12:28.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.354+0000 7f35bde49740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559871f550f0 con 0x559872034760 2024-09-10T08:12:28.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.354+0000 7f35bde49740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559872041d70 con 0x55987203f020 2024-09-10T08:12:28.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.355+0000 7f35bde49740 10 client.0 _ll_register_callbacks cb 0x559871f3cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:28.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.355+0000 7f35b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x55987203bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.355+0000 7f35b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x55987203bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34460/0 (socket says 172.21.15.2:34460) 2024-09-10T08:12:28.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.355+0000 7f35b3e00640 1 -- 172.21.15.2:0/3822337102 learned_addr learned my addr 172.21.15.2:0/3822337102 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:28.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.355+0000 7f35ab400640 1 --2- 172.21.15.2:0/3822337102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x55987203c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.355+0000 7f35b9200640 1 --2- 172.21.15.2:0/3822337102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 0x55987203c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.355+0000 7f35b3e00640 1 -- 172.21.15.2:0/3822337102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 msgr2=0x55987203c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:28.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.355+0000 7f35b3e00640 1 --2- 172.21.15.2:0/3822337102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 0x55987203c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.355+0000 7f35b3e00640 1 -- 172.21.15.2:0/3822337102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 msgr2=0x55987203c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:28.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.355+0000 7f35b3e00640 1 --2- 172.21.15.2:0/3822337102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x55987203c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.355+0000 7f35b3e00640 1 -- 172.21.15.2:0/3822337102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559871d0b3c0 con 0x559872033e40 2024-09-10T08:12:28.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.356+0000 7f35b3e00640 1 --2- 172.21.15.2:0/3822337102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x55987203bf00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f35a4011d10 tx=0x7f35a4005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd095f3cf0a16f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:28.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.356+0000 7f35a2000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:12:28.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.356+0000 7f35a2000640 1 -- 172.21.15.2:0/3822337102 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4029020 con 0x559872033e40 2024-09-10T08:12:28.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.356+0000 7f35a2000640 1 -- 172.21.15.2:0/3822337102 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f35a4005040 con 0x559872033e40 2024-09-10T08:12:28.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.356+0000 7f35bde49740 1 -- 172.21.15.2:0/3822337102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35a40041c0 con 0x559872033e40 2024-09-10T08:12:28.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.356+0000 7f35a2000640 1 -- 172.21.15.2:0/3822337102 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f35a4034050 con 0x559872033e40 2024-09-10T08:12:28.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.356+0000 7f35bde49740 10 client.16500 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:28.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.356+0000 7f35bde49740 1 -- 172.21.15.2:0/3822337102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55987204dc60 con 0x559872033e40 2024-09-10T08:12:28.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.357+0000 7f35a2000640 1 -- 172.21.15.2:0/3822337102 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a40041c0 con 0x559872033e40 2024-09-10T08:12:28.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.357+0000 7f35a2000640 1 -- 172.21.15.2:0/3822337102 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f35a4004c90 con 0x559872033e40 2024-09-10T08:12:28.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.357+0000 7f35bde49740 10 client.16500 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:28.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.357+0000 7f35bde49740 10 client.16500 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:12:28.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.357+0000 7f35bde49740 20 client.16500 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:28.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.357+0000 7f35bde49740 1 --2- 172.21.15.2:0/3822337102 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55987204f0c0 0x5598720514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.357+0000 7f35bde49740 4 client.16500 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:28.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.357+0000 7f35bde49740 1 -- 172.21.15.2:0/3822337102 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559871fa7950 con 0x55987204f0c0 2024-09-10T08:12:28.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.357+0000 7f35ab400640 1 --2- 172.21.15.2:0/3822337102 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55987204f0c0 0x5598720514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.358+0000 7f35ab400640 1 --2- 172.21.15.2:0/3822337102 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55987204f0c0 0x5598720514b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f930d7440669e58 server_cookie=1a7d33d339b69276 in_seq=0 out_seq=0 2024-09-10T08:12:28.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.358+0000 7f35a2000640 10 client.16500 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35a2000640 1 -- 172.21.15.2:0/3822337102 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559871fa7950 con 0x55987204f0c0 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35a2000640 10 client.16500 handle_command_reply: tid=0 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35bde49740 1 client.16500 shutdown 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35bde49740 1 -- 172.21.15.2:0/3822337102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 msgr2=0x55987203bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35bde49740 1 --2- 172.21.15.2:0/3822337102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x55987203bf00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f35a4011d10 tx=0x7f35a4005df0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35bde49740 1 -- 172.21.15.2:0/3822337102 shutdown_connections 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35bde49740 1 --2- 172.21.15.2:0/3822337102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 0x55987203c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35bde49740 1 --2- 172.21.15.2:0/3822337102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x55987203c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35bde49740 1 --2- 172.21.15.2:0/3822337102 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55987204f0c0 0x5598720514b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35bde49740 1 --2- 172.21.15.2:0/3822337102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x55987203bf00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35bde49740 1 -- 172.21.15.2:0/3822337102 >> 172.21.15.2:0/3822337102 conn(0x5598720303d0 msgr2=0x559872036100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35a2000640 0 client.16500 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35bde49740 1 -- 172.21.15.2:0/3822337102 shutdown_connections 2024-09-10T08:12:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35bde49740 1 -- 172.21.15.2:0/3822337102 wait complete. 2024-09-10T08:12:28.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.359+0000 7f35bde49740 20 client.16500 trim_cache size 0 max 16384 2024-09-10T08:12:28.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.373+0000 7f35bde49740 1 Processor -- start 2024-09-10T08:12:28.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.373+0000 7f35bde49740 1 -- start start 2024-09-10T08:12:28.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.373+0000 7f35bde49740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x55987203b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.373+0000 7f35bde49740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x559872056c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35bde49740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 0x559872057180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35bde49740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41b37e0 con 0x559872033e40 2024-09-10T08:12:28.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35bde49740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41b10b0 con 0x559872034760 2024-09-10T08:12:28.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35bde49740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41aef00 con 0x55987203f020 2024-09-10T08:12:28.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x55987203b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x55987203b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34470/0 (socket says 172.21.15.2:34470) 2024-09-10T08:12:28.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35ab400640 1 --2- 172.21.15.2:0/2540647190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x559872056c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35b3e00640 1 -- 172.21.15.2:0/2540647190 learned_addr learned my addr 172.21.15.2:0/2540647190 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:28.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35b9200640 1 --2- 172.21.15.2:0/2540647190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 0x559872057180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35ab400640 1 -- 172.21.15.2:0/2540647190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 msgr2=0x559872057180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:28.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35ab400640 1 --2- 172.21.15.2:0/2540647190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 0x559872057180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35ab400640 1 -- 172.21.15.2:0/2540647190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 msgr2=0x55987203b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:28.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35ab400640 1 --2- 172.21.15.2:0/2540647190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x55987203b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35ab400640 1 -- 172.21.15.2:0/2540647190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559872051a10 con 0x559872034760 2024-09-10T08:12:28.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35b9200640 1 --2- 172.21.15.2:0/2540647190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 0x559872057180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:28.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.374+0000 7f35b3e00640 1 --2- 172.21.15.2:0/2540647190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x55987203b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:28.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.375+0000 7f35ab400640 1 --2- 172.21.15.2:0/2540647190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x559872056c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f359c0ccea0 tx=0x7f359c0ccd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a69723947ee7fe7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:28.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.375+0000 7f35a8c00640 1 -- 172.21.15.2:0/2540647190 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c09c2b0 con 0x559872034760 2024-09-10T08:12:28.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.375+0000 7f35a8c00640 1 -- 172.21.15.2:0/2540647190 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f359c09bbb0 con 0x559872034760 2024-09-10T08:12:28.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.375+0000 7f35a8c00640 1 -- 172.21.15.2:0/2540647190 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c098220 con 0x559872034760 2024-09-10T08:12:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.375+0000 7f35bde49740 1 -- 172.21.15.2:0/2540647190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 msgr2=0x559872056c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.375+0000 7f35bde49740 1 --2- 172.21.15.2:0/2540647190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x559872056c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f359c0ccea0 tx=0x7f359c0ccd20 comp rx=0 tx=0).stop 2024-09-10T08:12:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.375+0000 7f35bde49740 1 -- 172.21.15.2:0/2540647190 shutdown_connections 2024-09-10T08:12:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.375+0000 7f35bde49740 1 --2- 172.21.15.2:0/2540647190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55987203f020 0x559872057180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.375+0000 7f35bde49740 1 --2- 172.21.15.2:0/2540647190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x559872056c40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.375+0000 7f35bde49740 1 --2- 172.21.15.2:0/2540647190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559872033e40 0x55987203b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.375+0000 7f35bde49740 1 -- 172.21.15.2:0/2540647190 >> 172.21.15.2:0/2540647190 conn(0x5598720303d0 msgr2=0x5598720360b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.376+0000 7f35bde49740 1 -- 172.21.15.2:0/2540647190 shutdown_connections 2024-09-10T08:12:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.376+0000 7f35bde49740 1 -- 172.21.15.2:0/2540647190 wait complete. 2024-09-10T08:12:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.376+0000 7f35bde49740 1 -- start start 2024-09-10T08:12:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.376+0000 7f35bde49740 1 Processor -- start 2024-09-10T08:12:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.376+0000 7f35bde49740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872033e40 0x5598720472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35bde49740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x559872047800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35bde49740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55987203f020 0x559872042770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35bde49740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41af570 con 0x55987203f020 2024-09-10T08:12:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35bde49740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41b09e0 con 0x559872034760 2024-09-10T08:12:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35bde49740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35b41afc40 con 0x559872033e40 2024-09-10T08:12:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35bde49740 10 client.0 _ll_register_callbacks cb 0x7f35a4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35b9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55987203f020 0x559872042770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35b9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55987203f020 0x559872042770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34486/0 (socket says 172.21.15.2:34486) 2024-09-10T08:12:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35b9200640 1 -- 172.21.15.2:0/680837874 learned_addr learned my addr 172.21.15.2:0/680837874 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35b3e00640 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872033e40 0x5598720472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35ab400640 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x559872047800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35b3e00640 1 -- 172.21.15.2:0/680837874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 msgr2=0x559872047800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35b3e00640 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x559872047800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35b3e00640 1 -- 172.21.15.2:0/680837874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55987203f020 msgr2=0x559872042770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35b3e00640 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55987203f020 0x559872042770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35b3e00640 1 -- 172.21.15.2:0/680837874 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559872042fa0 con 0x559872033e40 2024-09-10T08:12:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.377+0000 7f35b9200640 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55987203f020 0x559872042770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.378+0000 7f35b3e00640 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872033e40 0x5598720472c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f35a4011850 tx=0x7f35a40041c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e6b2bf52279b652 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.378+0000 7f35a2000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.378+0000 7f35a2000640 1 -- 172.21.15.2:0/680837874 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a403d060 con 0x559872033e40 2024-09-10T08:12:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.378+0000 7f35bde49740 1 -- 172.21.15.2:0/680837874 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f359c09bbb0 con 0x559872033e40 2024-09-10T08:12:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.378+0000 7f35a2000640 1 -- 172.21.15.2:0/680837874 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f35a4003900 con 0x559872033e40 2024-09-10T08:12:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.378+0000 7f35a2000640 1 -- 172.21.15.2:0/680837874 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f35a4029040 con 0x559872033e40 2024-09-10T08:12:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.379+0000 7f35bde49740 10 client.26500 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.379+0000 7f35bde49740 1 -- 172.21.15.2:0/680837874 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5598720b9790 con 0x559872033e40 2024-09-10T08:12:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.379+0000 7f35a2000640 1 -- 172.21.15.2:0/680837874 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f359c09bbb0 con 0x559872033e40 2024-09-10T08:12:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.379+0000 7f35a2000640 1 -- 172.21.15.2:0/680837874 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f35a4003b00 con 0x559872033e40 2024-09-10T08:12:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.379+0000 7f35bde49740 10 client.26500 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.379+0000 7f35bde49740 10 client.26500 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:12:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.379+0000 7f35bde49740 20 client.26500 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.379+0000 7f35bde49740 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5598720b9e50 0x5598720bc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.379+0000 7f35bde49740 4 client.26500 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:12:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.379+0000 7f35bde49740 1 -- 172.21.15.2:0/680837874 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559871fa1e60 con 0x5598720b9e50 2024-09-10T08:12:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.380+0000 7f35ab400640 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5598720b9e50 0x5598720bc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:28.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.380+0000 7f35ab400640 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5598720b9e50 0x5598720bc270 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4270a7beda684350 server_cookie=b383485311f82651 in_seq=0 out_seq=0 2024-09-10T08:12:28.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.380+0000 7f35a2000640 10 client.26500 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:12:28.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.380+0000 7f35a2000640 1 -- 172.21.15.2:0/680837874 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559871fa1e60 con 0x5598720b9e50 2024-09-10T08:12:28.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.380+0000 7f35a2000640 10 client.26500 handle_command_reply: tid=0 2024-09-10T08:12:28.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.380+0000 7f35bde49740 1 client.26500 shutdown 2024-09-10T08:12:28.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.380+0000 7f35bde49740 1 -- 172.21.15.2:0/680837874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872033e40 msgr2=0x5598720472c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.380+0000 7f35bde49740 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872033e40 0x5598720472c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f35a4011850 tx=0x7f35a40041c0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.381+0000 7f35bde49740 1 -- 172.21.15.2:0/680837874 shutdown_connections 2024-09-10T08:12:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.381+0000 7f35bde49740 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5598720b9e50 0x5598720bc270 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.381+0000 7f35bde49740 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55987203f020 0x559872042770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.381+0000 7f35bde49740 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559872034760 0x559872047800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.381+0000 7f35a2000640 0 client.26500 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:12:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.381+0000 7f35bde49740 1 --2- 172.21.15.2:0/680837874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872033e40 0x5598720472c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.381+0000 7f35bde49740 1 -- 172.21.15.2:0/680837874 >> 172.21.15.2:0/680837874 conn(0x5598720303d0 msgr2=0x55987203be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.381+0000 7f35bde49740 1 -- 172.21.15.2:0/680837874 shutdown_connections 2024-09-10T08:12:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.381+0000 7f35bde49740 1 -- 172.21.15.2:0/680837874 wait complete. 2024-09-10T08:12:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.381+0000 7f35bde49740 20 client.26500 trim_cache size 0 max 16384 2024-09-10T08:12:28.387 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:12:28.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.382+0000 7f35b9c00640 1 -- 172.21.15.2:0/66495202 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f35980a2b80 msgr2=0x7f35980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:28.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.382+0000 7f35b9c00640 1 --2- 172.21.15.2:0/66495202 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f35980a2b80 0x7f35980a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f35ac033000 tx=0x7f35ac009840 comp rx=0 tx=0).stop 2024-09-10T08:12:28.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.382+0000 7f35b9c00640 1 -- 172.21.15.2:0/66495202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b41034b0 msgr2=0x7f35b419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:28.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.382+0000 7f35b9c00640 1 --2- 172.21.15.2:0/66495202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b41034b0 0x7f35b419aa30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f359c00e9f0 tx=0x7f359c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.382+0000 7f35b9c00640 1 -- 172.21.15.2:0/66495202 shutdown_connections 2024-09-10T08:12:28.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.382+0000 7f35b9c00640 1 --2- 172.21.15.2:0/66495202 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f35980a2b80 0x7f35980a5040 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.382+0000 7f35b9c00640 1 --2- 172.21.15.2:0/66495202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35b410bfc0 0x7f35b419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.382+0000 7f35b9c00640 1 --2- 172.21.15.2:0/66495202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b41034b0 0x7f35b419aa30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.382+0000 7f35b9c00640 1 --2- 172.21.15.2:0/66495202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b4102b90 0x7f35b419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:28.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.382+0000 7f35b9c00640 1 -- 172.21.15.2:0/66495202 >> 172.21.15.2:0/66495202 conn(0x7f35b40fc630 msgr2=0x7f35b40fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:28.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.382+0000 7f35b9c00640 1 -- 172.21.15.2:0/66495202 shutdown_connections 2024-09-10T08:12:28.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:28.382+0000 7f35b9c00640 1 -- 172.21.15.2:0/66495202 wait complete. 2024-09-10T08:12:29.463 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:12:29.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:29 smithi002 ceph-mon[30563]: pgmap v517: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-09-10T08:12:29.844 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:29 smithi155 ceph-mon[35405]: pgmap v517: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-09-10T08:12:29.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:29 smithi107 ceph-mon[35048]: pgmap v517: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-09-10T08:12:30.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.580+0000 7f600ec00640 1 -- 172.21.15.2:0/4090786305 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60081047e0 msgr2=0x7f6008108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:30.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.580+0000 7f600ec00640 1 --2- 172.21.15.2:0/4090786305 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60081047e0 0x7f6008108830 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009900 tx=0x7f5ff402a410 comp rx=0 tx=0).stop 2024-09-10T08:12:30.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.580+0000 7f600ec00640 1 -- 172.21.15.2:0/4090786305 shutdown_connections 2024-09-10T08:12:30.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.580+0000 7f600ec00640 1 --2- 172.21.15.2:0/4090786305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6008108f60 0x7f6008110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.580+0000 7f600ec00640 1 --2- 172.21.15.2:0/4090786305 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60081047e0 0x7f6008108830 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.580+0000 7f600ec00640 1 --2- 172.21.15.2:0/4090786305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6008103e30 0x7f6008104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.580+0000 7f600ec00640 1 -- 172.21.15.2:0/4090786305 >> 172.21.15.2:0/4090786305 conn(0x7f60080ff910 msgr2=0x7f6008101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:30.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.580+0000 7f600ec00640 1 -- 172.21.15.2:0/4090786305 shutdown_connections 2024-09-10T08:12:30.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.580+0000 7f600ec00640 1 -- 172.21.15.2:0/4090786305 wait complete. 2024-09-10T08:12:30.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.581+0000 7f600ec00640 1 Processor -- start 2024-09-10T08:12:30.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.582+0000 7f600ec00640 1 -- start start 2024-09-10T08:12:30.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.582+0000 7f600ec00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6008103e30 0x7f600819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.582+0000 7f600ec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60081047e0 0x7f600819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.582+0000 7f600ec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6008108f60 0x7f600819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.582+0000 7f600ec00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6008114010 con 0x7f6008103e30 2024-09-10T08:12:30.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.582+0000 7f600ec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6008113e90 con 0x7f6008108f60 2024-09-10T08:12:30.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.582+0000 7f600ec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6008114190 con 0x7f60081047e0 2024-09-10T08:12:30.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.582+0000 7f600d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6008103e30 0x7f600819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.583+0000 7f600d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6008103e30 0x7f600819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34500/0 (socket says 172.21.15.2:34500) 2024-09-10T08:12:30.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.583+0000 7f600d800640 1 -- 172.21.15.2:0/1999026634 learned_addr learned my addr 172.21.15.2:0/1999026634 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:30.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.583+0000 7f600e200640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6008108f60 0x7f600819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.583+0000 7f600ce00640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60081047e0 0x7f600819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.583+0000 7f600d800640 1 -- 172.21.15.2:0/1999026634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60081047e0 msgr2=0x7f600819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:30.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.583+0000 7f600d800640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60081047e0 0x7f600819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.583+0000 7f600d800640 1 -- 172.21.15.2:0/1999026634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6008108f60 msgr2=0x7f600819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:30.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.583+0000 7f600d800640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6008108f60 0x7f600819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.583+0000 7f600d800640 1 -- 172.21.15.2:0/1999026634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600819f480 con 0x7f6008103e30 2024-09-10T08:12:30.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.583+0000 7f600e200640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6008108f60 0x7f600819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:30.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.583+0000 7f600d800640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6008103e30 0x7f600819a4d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00e9f0 tx=0x7f5ffc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:30.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.583+0000 7f6006000640 1 -- 172.21.15.2:0/1999026634 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc00cd80 con 0x7f6008103e30 2024-09-10T08:12:30.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.584+0000 7f6006000640 1 -- 172.21.15.2:0/1999026634 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ffc005500 con 0x7f6008103e30 2024-09-10T08:12:30.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.584+0000 7f6006000640 1 -- 172.21.15.2:0/1999026634 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc010690 con 0x7f6008103e30 2024-09-10T08:12:30.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.584+0000 7f600ec00640 1 -- 172.21.15.2:0/1999026634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600819f770 con 0x7f6008103e30 2024-09-10T08:12:30.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.584+0000 7f600ec00640 1 -- 172.21.15.2:0/1999026634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60081a7050 con 0x7f6008103e30 2024-09-10T08:12:30.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f6012bd4740 1 Processor -- start 2024-09-10T08:12:30.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f6006000640 1 -- 172.21.15.2:0/1999026634 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5ffc0046c0 con 0x7f6008103e30 2024-09-10T08:12:30.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f6012bd4740 1 -- start start 2024-09-10T08:12:30.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f6012bd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c4375220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f6012bd4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f6012bd4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4380020 0x5629c4382410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f6012bd4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5629c42db3e0 con 0x5629c4375760 2024-09-10T08:12:30.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f6012bd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5629c4282d80 con 0x5629c4374e40 2024-09-10T08:12:30.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f6012bd4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5629c42d73a0 con 0x5629c4380020 2024-09-10T08:12:30.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f600d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f600ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4380020 0x5629c4382410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f600e200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c4375220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f600ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4380020 0x5629c4382410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35772/0 (socket says 172.21.15.2:35772) 2024-09-10T08:12:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.587+0000 7f600ce00640 1 -- 172.21.15.2:0/4090192101 learned_addr learned my addr 172.21.15.2:0/4090192101 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.588+0000 7f600ce00640 1 -- 172.21.15.2:0/4090192101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 msgr2=0x5629c4375220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.588+0000 7f600ce00640 1 --2- 172.21.15.2:0/4090192101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c4375220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.588+0000 7f600ce00640 1 -- 172.21.15.2:0/4090192101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 msgr2=0x5629c437fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.588+0000 7f600ce00640 1 --2- 172.21.15.2:0/4090192101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.588+0000 7f600ce00640 1 -- 172.21.15.2:0/4090192101 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c42a1ae0 con 0x5629c4380020 2024-09-10T08:12:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.588+0000 7f600ce00640 1 --2- 172.21.15.2:0/4090192101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4380020 0x5629c4382410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009900 tx=0x7f5ff4005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c942a5ec482f1926 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.588+0000 7f6006000640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5ff00a2b80 0x7f5ff00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.589+0000 7f5ffaa00640 1 -- 172.21.15.2:0/4090192101 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4035040 con 0x5629c4380020 2024-09-10T08:12:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.589+0000 7f6006000640 1 -- 172.21.15.2:0/1999026634 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5ffc0cf870 con 0x7f6008103e30 2024-09-10T08:12:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.589+0000 7f5ffaa00640 1 -- 172.21.15.2:0/4090192101 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ff4004400 con 0x5629c4380020 2024-09-10T08:12:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.589+0000 7f5ffaa00640 1 -- 172.21.15.2:0/4090192101 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff40035f0 con 0x5629c4380020 2024-09-10T08:12:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.589+0000 7f600e200640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5ff00a2b80 0x7f5ff00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.589+0000 7f6012bd4740 1 -- 172.21.15.2:0/4090192101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4380020 msgr2=0x5629c4382410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.589+0000 7f6012bd4740 1 --2- 172.21.15.2:0/4090192101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4380020 0x5629c4382410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009900 tx=0x7f5ff4005de0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.589+0000 7f600e200640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5ff00a2b80 0x7f5ff00a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6000009fd0 tx=0x7f6000009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.589+0000 7f6012bd4740 1 -- 172.21.15.2:0/4090192101 shutdown_connections 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.589+0000 7f6012bd4740 1 --2- 172.21.15.2:0/4090192101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4380020 0x5629c4382410 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.589+0000 7f6012bd4740 1 --2- 172.21.15.2:0/4090192101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.589+0000 7f6012bd4740 1 --2- 172.21.15.2:0/4090192101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c4375220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.589+0000 7f6012bd4740 1 -- 172.21.15.2:0/4090192101 >> 172.21.15.2:0/4090192101 conn(0x5629c43713d0 msgr2=0x5629c43717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.590+0000 7f6012bd4740 1 -- 172.21.15.2:0/4090192101 shutdown_connections 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.590+0000 7f6012bd4740 1 -- 172.21.15.2:0/4090192101 wait complete. 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.590+0000 7f6012bd4740 1 -- start start 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.590+0000 7f6012bd4740 1 Processor -- start 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f6012bd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c437ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f6012bd4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f6012bd4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4380020 0x5629c437d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f6012bd4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5629c4382bf0 con 0x5629c4375760 2024-09-10T08:12:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f6012bd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5629c42960f0 con 0x5629c4374e40 2024-09-10T08:12:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f6012bd4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5629c4382d70 con 0x5629c4380020 2024-09-10T08:12:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f6012bd4740 10 client.0 _ll_register_callbacks cb 0x5629c427dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f600ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f600d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c437ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f600ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34514/0 (socket says 172.21.15.2:34514) 2024-09-10T08:12:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f600ce00640 1 -- 172.21.15.2:0/629778797 learned_addr learned my addr 172.21.15.2:0/629778797 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f600e200640 1 --2- 172.21.15.2:0/629778797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4380020 0x5629c437d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f600d800640 1 -- 172.21.15.2:0/629778797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4380020 msgr2=0x5629c437d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f600d800640 1 --2- 172.21.15.2:0/629778797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4380020 0x5629c437d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f600d800640 1 -- 172.21.15.2:0/629778797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 msgr2=0x5629c437d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f600d800640 1 --2- 172.21.15.2:0/629778797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f600d800640 1 -- 172.21.15.2:0/629778797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c404c3c0 con 0x5629c4374e40 2024-09-10T08:12:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.591+0000 7f600ce00640 1 --2- 172.21.15.2:0/629778797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.592+0000 7f600d800640 1 --2- 172.21.15.2:0/629778797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c437ced0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0ccef0 tx=0x7f5ffc097fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33555305805daa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.592+0000 7f5fef400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.592+0000 7f5fef400640 1 -- 172.21.15.2:0/629778797 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc0a4660 con 0x5629c4374e40 2024-09-10T08:12:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.592+0000 7f5fef400640 1 -- 172.21.15.2:0/629778797 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5ffc09b0d0 con 0x5629c4374e40 2024-09-10T08:12:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.592+0000 7f6012bd4740 1 -- 172.21.15.2:0/629778797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ff4004400 con 0x5629c4374e40 2024-09-10T08:12:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.592+0000 7f5fef400640 1 -- 172.21.15.2:0/629778797 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ffc09c380 con 0x5629c4374e40 2024-09-10T08:12:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.592+0000 7f6012bd4740 10 client.26555 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.592+0000 7f6012bd4740 1 -- 172.21.15.2:0/629778797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629c438ec60 con 0x5629c4374e40 2024-09-10T08:12:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.592+0000 7f5fef400640 1 -- 172.21.15.2:0/629778797 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ff4004400 con 0x5629c4374e40 2024-09-10T08:12:30.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.593+0000 7f5fef400640 1 -- 172.21.15.2:0/629778797 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5ffc0b4080 con 0x5629c4374e40 2024-09-10T08:12:30.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.593+0000 7f6012bd4740 10 client.26555 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:30.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.593+0000 7f6012bd4740 10 client.26555 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:12:30.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.593+0000 7f6012bd4740 20 client.26555 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:30.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.593+0000 7f6012bd4740 1 --2- 172.21.15.2:0/629778797 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5629c4390120 0x5629c4392590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.593+0000 7f6012bd4740 4 client.26555 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:30.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.593+0000 7f6012bd4740 1 -- 172.21.15.2:0/629778797 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629c42e8950 con 0x5629c4390120 2024-09-10T08:12:30.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.593+0000 7f600ce00640 1 --2- 172.21.15.2:0/629778797 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5629c4390120 0x5629c4392590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.594+0000 7f600ce00640 1 --2- 172.21.15.2:0/629778797 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5629c4390120 0x5629c4392590 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e52587e4f1673461 server_cookie=10b5a09d45b158d2 in_seq=0 out_seq=0 2024-09-10T08:12:30.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.594+0000 7f5fef400640 10 client.26555 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:12:30.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f5fef400640 1 -- 172.21.15.2:0/629778797 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629c42e8950 con 0x5629c4390120 2024-09-10T08:12:30.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f5fef400640 10 client.26555 handle_command_reply: tid=0 2024-09-10T08:12:30.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f6012bd4740 1 client.26555 shutdown 2024-09-10T08:12:30.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f6012bd4740 1 -- 172.21.15.2:0/629778797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 msgr2=0x5629c437ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:30.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f6012bd4740 1 --2- 172.21.15.2:0/629778797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c437ced0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0ccef0 tx=0x7f5ffc097fe0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f6012bd4740 1 -- 172.21.15.2:0/629778797 shutdown_connections 2024-09-10T08:12:30.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f6012bd4740 1 --2- 172.21.15.2:0/629778797 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5629c4390120 0x5629c4392590 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f6012bd4740 1 --2- 172.21.15.2:0/629778797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4380020 0x5629c437d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f6012bd4740 1 --2- 172.21.15.2:0/629778797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f6012bd4740 1 --2- 172.21.15.2:0/629778797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c437ced0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f5fef400640 0 client.26555 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:12:30.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f6012bd4740 1 -- 172.21.15.2:0/629778797 >> 172.21.15.2:0/629778797 conn(0x5629c43713d0 msgr2=0x5629c4377100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:30.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f6012bd4740 1 -- 172.21.15.2:0/629778797 shutdown_connections 2024-09-10T08:12:30.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.595+0000 7f6012bd4740 1 -- 172.21.15.2:0/629778797 wait complete. 2024-09-10T08:12:30.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.596+0000 7f6012bd4740 20 client.26555 trim_cache size 0 max 16384 2024-09-10T08:12:30.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.610+0000 7f6012bd4740 1 Processor -- start 2024-09-10T08:12:30.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.610+0000 7f6012bd4740 1 -- start start 2024-09-10T08:12:30.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.610+0000 7f6012bd4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4374e40 0x5629c43778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.610+0000 7f6012bd4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.610+0000 7f6012bd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4380020 0x5629c4397e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.610+0000 7f6012bd4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f60081b3780 con 0x5629c4375760 2024-09-10T08:12:30.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.610+0000 7f6012bd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f60081b1050 con 0x5629c4380020 2024-09-10T08:12:30.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.610+0000 7f6012bd4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f60081aeea0 con 0x5629c4374e40 2024-09-10T08:12:30.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.610+0000 7f600ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.610+0000 7f600ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34524/0 (socket says 172.21.15.2:34524) 2024-09-10T08:12:30.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.610+0000 7f600ce00640 1 -- 172.21.15.2:0/1040897335 learned_addr learned my addr 172.21.15.2:0/1040897335 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:30.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.610+0000 7f600d800640 1 --2- 172.21.15.2:0/1040897335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4374e40 0x5629c43778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.610+0000 7f600e200640 1 --2- 172.21.15.2:0/1040897335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4380020 0x5629c4397e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.611+0000 7f600ce00640 1 -- 172.21.15.2:0/1040897335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4374e40 msgr2=0x5629c43778f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:30.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.611+0000 7f600ce00640 1 --2- 172.21.15.2:0/1040897335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4374e40 0x5629c43778f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.611+0000 7f600ce00640 1 -- 172.21.15.2:0/1040897335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4380020 msgr2=0x5629c4397e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:30.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.611+0000 7f600ce00640 1 --2- 172.21.15.2:0/1040897335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4380020 0x5629c4397e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.611+0000 7f600ce00640 1 -- 172.21.15.2:0/1040897335 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c4371200 con 0x5629c4375760 2024-09-10T08:12:30.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.611+0000 7f600d800640 1 --2- 172.21.15.2:0/1040897335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4374e40 0x5629c43778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:30.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.611+0000 7f600ce00640 1 --2- 172.21.15.2:0/1040897335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437ee40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009900 tx=0x7f5ff4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a457d780f4ae040e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:30.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.611+0000 7f5ffa000640 1 -- 172.21.15.2:0/1040897335 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4035040 con 0x5629c4375760 2024-09-10T08:12:30.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.611+0000 7f5ffa000640 1 -- 172.21.15.2:0/1040897335 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ff40046c0 con 0x5629c4375760 2024-09-10T08:12:30.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.611+0000 7f5ffa000640 1 -- 172.21.15.2:0/1040897335 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4003850 con 0x5629c4375760 2024-09-10T08:12:30.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.612+0000 7f6012bd4740 1 -- 172.21.15.2:0/1040897335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 msgr2=0x5629c437ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:30.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.612+0000 7f6012bd4740 1 --2- 172.21.15.2:0/1040897335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437ee40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009900 tx=0x7f5ff4004b40 comp rx=0 tx=0).stop 2024-09-10T08:12:30.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.612+0000 7f6012bd4740 1 -- 172.21.15.2:0/1040897335 shutdown_connections 2024-09-10T08:12:30.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.612+0000 7f6012bd4740 1 --2- 172.21.15.2:0/1040897335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4380020 0x5629c4397e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.612+0000 7f6012bd4740 1 --2- 172.21.15.2:0/1040897335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4375760 0x5629c437ee40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.612+0000 7f6012bd4740 1 --2- 172.21.15.2:0/1040897335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4374e40 0x5629c43778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.612+0000 7f6012bd4740 1 -- 172.21.15.2:0/1040897335 >> 172.21.15.2:0/1040897335 conn(0x5629c43713d0 msgr2=0x5629c437e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:30.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.612+0000 7f6012bd4740 1 -- 172.21.15.2:0/1040897335 shutdown_connections 2024-09-10T08:12:30.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.612+0000 7f6012bd4740 1 -- 172.21.15.2:0/1040897335 wait complete. 2024-09-10T08:12:30.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.613+0000 7f6012bd4740 1 -- start start 2024-09-10T08:12:30.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.613+0000 7f6012bd4740 1 Processor -- start 2024-09-10T08:12:30.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.613+0000 7f6012bd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c4388850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.613+0000 7f6012bd4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4375760 0x5629c4379d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.613+0000 7f6012bd4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4380020 0x5629c437a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.613+0000 7f6012bd4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f60081af510 con 0x5629c4380020 2024-09-10T08:12:30.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.613+0000 7f6012bd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f60081b0980 con 0x5629c4374e40 2024-09-10T08:12:30.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.613+0000 7f6012bd4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f60081afbe0 con 0x5629c4375760 2024-09-10T08:12:30.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.613+0000 7f6012bd4740 10 client.0 _ll_register_callbacks cb 0x7f5ffc0d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:30.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.614+0000 7f600e200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4380020 0x5629c437a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.614+0000 7f600d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c4388850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.614+0000 7f600e200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4380020 0x5629c437a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34538/0 (socket says 172.21.15.2:34538) 2024-09-10T08:12:30.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.614+0000 7f600e200640 1 -- 172.21.15.2:0/3593633060 learned_addr learned my addr 172.21.15.2:0/3593633060 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:30.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.614+0000 7f600d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c4388850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60004/0 (socket says 172.21.15.2:60004) 2024-09-10T08:12:30.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.614+0000 7f600ce00640 1 --2- 172.21.15.2:0/3593633060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4375760 0x5629c4379d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.614+0000 7f600e200640 1 -- 172.21.15.2:0/3593633060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4375760 msgr2=0x5629c4379d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:30.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.614+0000 7f600e200640 1 --2- 172.21.15.2:0/3593633060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4375760 0x5629c4379d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.614+0000 7f600e200640 1 -- 172.21.15.2:0/3593633060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 msgr2=0x5629c4388850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:30.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.614+0000 7f600e200640 1 --2- 172.21.15.2:0/3593633060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c4388850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.614+0000 7f600e200640 1 -- 172.21.15.2:0/3593633060 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c4372060 con 0x5629c4380020 2024-09-10T08:12:30.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.614+0000 7f600e200640 1 --2- 172.21.15.2:0/3593633060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4380020 0x5629c437a390 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6000003fd0 tx=0x7f6000004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81f389792fe9bef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:30.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.614+0000 7f5fef400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:12:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.615+0000 7f5fef400640 1 -- 172.21.15.2:0/3593633060 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60000047d0 con 0x5629c4380020 2024-09-10T08:12:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.615+0000 7f5fef400640 1 -- 172.21.15.2:0/3593633060 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6000012020 con 0x5629c4380020 2024-09-10T08:12:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.615+0000 7f6012bd4740 1 -- 172.21.15.2:0/3593633060 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ff40046c0 con 0x5629c4380020 2024-09-10T08:12:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.615+0000 7f5fef400640 1 -- 172.21.15.2:0/3593633060 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6000035050 con 0x5629c4380020 2024-09-10T08:12:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.615+0000 7f6012bd4740 10 client.16524 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.615+0000 7f6012bd4740 1 -- 172.21.15.2:0/3593633060 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629c43fadb0 con 0x5629c4380020 2024-09-10T08:12:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.615+0000 7f5fef400640 1 -- 172.21.15.2:0/3593633060 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ff40046c0 con 0x5629c4380020 2024-09-10T08:12:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.615+0000 7f5fef400640 1 -- 172.21.15.2:0/3593633060 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f600001f430 con 0x5629c4380020 2024-09-10T08:12:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.616+0000 7f6012bd4740 10 client.16524 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.616+0000 7f6012bd4740 10 client.16524 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:12:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.616+0000 7f6012bd4740 20 client.16524 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.616+0000 7f6012bd4740 1 --2- 172.21.15.2:0/3593633060 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5629c43fb580 0x5629c43fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.616+0000 7f6012bd4740 4 client.16524 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.616+0000 7f6012bd4740 1 -- 172.21.15.2:0/3593633060 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629c42e2e60 con 0x5629c43fb580 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.616+0000 7f600d800640 1 --2- 172.21.15.2:0/3593633060 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5629c43fb580 0x5629c43fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f600d800640 1 --2- 172.21.15.2:0/3593633060 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5629c43fb580 0x5629c43fd9a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fbdbb9f5bdb07797 server_cookie=f7180a00ba15fbe in_seq=0 out_seq=0 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f5fef400640 10 client.16524 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f5fef400640 1 -- 172.21.15.2:0/3593633060 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629c42e2e60 con 0x5629c43fb580 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f5fef400640 10 client.16524 handle_command_reply: tid=0 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f6012bd4740 1 client.16524 shutdown 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f6012bd4740 1 -- 172.21.15.2:0/3593633060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4380020 msgr2=0x5629c437a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f6012bd4740 1 --2- 172.21.15.2:0/3593633060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4380020 0x5629c437a390 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6000003fd0 tx=0x7f6000004a40 comp rx=0 tx=0).stop 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f6012bd4740 1 -- 172.21.15.2:0/3593633060 shutdown_connections 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f6012bd4740 1 --2- 172.21.15.2:0/3593633060 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5629c43fb580 0x5629c43fd9a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f6012bd4740 1 --2- 172.21.15.2:0/3593633060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629c4380020 0x5629c437a390 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f5fef400640 0 client.16524 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:12:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f6012bd4740 1 --2- 172.21.15.2:0/3593633060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629c4375760 0x5629c4379d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f6012bd4740 1 --2- 172.21.15.2:0/3593633060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629c4374e40 0x5629c4388850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f6012bd4740 1 -- 172.21.15.2:0/3593633060 >> 172.21.15.2:0/3593633060 conn(0x5629c43713d0 msgr2=0x5629c437ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:30.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f6012bd4740 1 -- 172.21.15.2:0/3593633060 shutdown_connections 2024-09-10T08:12:30.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.617+0000 7f6012bd4740 1 -- 172.21.15.2:0/3593633060 wait complete. 2024-09-10T08:12:30.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.618+0000 7f6012bd4740 20 client.16524 trim_cache size 0 max 16384 2024-09-10T08:12:30.627 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:12:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.618+0000 7f600ec00640 1 -- 172.21.15.2:0/1999026634 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5ff00a2b80 msgr2=0x7f5ff00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.618+0000 7f600ec00640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5ff00a2b80 0x7f5ff00a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6000009fd0 tx=0x7f6000009500 comp rx=0 tx=0).stop 2024-09-10T08:12:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.618+0000 7f600ec00640 1 -- 172.21.15.2:0/1999026634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6008103e30 msgr2=0x7f600819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.618+0000 7f600ec00640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6008103e30 0x7f600819a4d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00e9f0 tx=0x7f5ffc00edf0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.618+0000 7f600ec00640 1 -- 172.21.15.2:0/1999026634 shutdown_connections 2024-09-10T08:12:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.618+0000 7f600ec00640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6008108f60 0x7f600819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.618+0000 7f600ec00640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5ff00a2b80 0x7f5ff00a5040 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.618+0000 7f600ec00640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60081047e0 0x7f600819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.618+0000 7f600ec00640 1 --2- 172.21.15.2:0/1999026634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6008103e30 0x7f600819a4d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:30.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.618+0000 7f600ec00640 1 -- 172.21.15.2:0/1999026634 >> 172.21.15.2:0/1999026634 conn(0x7f60080ff910 msgr2=0x7f6008100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:30.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.619+0000 7f600ec00640 1 -- 172.21.15.2:0/1999026634 shutdown_connections 2024-09-10T08:12:30.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:30.619+0000 7f600ec00640 1 -- 172.21.15.2:0/1999026634 wait complete. 2024-09-10T08:12:31.515 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:12:31.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:31 smithi002 ceph-mon[30563]: pgmap v518: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-10T08:12:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:31.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:31 smithi155 ceph-mon[35405]: pgmap v518: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-10T08:12:31.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:31 smithi107 ceph-mon[35048]: pgmap v518: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-10T08:12:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.472+0000 7f4780e00640 1 -- 172.21.15.2:0/4266498624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c112b90 msgr2=0x7f477c114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.472+0000 7f4780e00640 1 --2- 172.21.15.2:0/4266498624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c112b90 0x7f477c114f80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4774009ca0 tx=0x7f477402a950 comp rx=0 tx=0).stop 2024-09-10T08:12:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.472+0000 7f4780e00640 1 -- 172.21.15.2:0/4266498624 shutdown_connections 2024-09-10T08:12:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.472+0000 7f4780e00640 1 --2- 172.21.15.2:0/4266498624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c112b90 0x7f477c114f80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.472+0000 7f4780e00640 1 --2- 172.21.15.2:0/4266498624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f477c074eb0 0x7f477c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.472+0000 7f4780e00640 1 --2- 172.21.15.2:0/4266498624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477c074590 0x7f477c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.472+0000 7f4780e00640 1 -- 172.21.15.2:0/4266498624 >> 172.21.15.2:0/4266498624 conn(0x7f477c0fd8c0 msgr2=0x7f477c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.472+0000 7f4780e00640 1 -- 172.21.15.2:0/4266498624 shutdown_connections 2024-09-10T08:12:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.473+0000 7f4780e00640 1 -- 172.21.15.2:0/4266498624 wait complete. 2024-09-10T08:12:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.473+0000 7f4780e00640 1 Processor -- start 2024-09-10T08:12:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.474+0000 7f4780e00640 1 -- start start 2024-09-10T08:12:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.474+0000 7f4780e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c074590 0x7f477c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.474+0000 7f4780e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477c074eb0 0x7f477c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.474+0000 7f4780e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f477c112b90 0x7f477c1a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.474+0000 7f4780e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1185c0 con 0x7f477c074590 2024-09-10T08:12:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.474+0000 7f4780e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f477c118440 con 0x7f477c074eb0 2024-09-10T08:12:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.474+0000 7f4780e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f477c118740 con 0x7f477c112b90 2024-09-10T08:12:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.474+0000 7f4779e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c074590 0x7f477c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.474+0000 7f4779e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c074590 0x7f477c19eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34566/0 (socket says 172.21.15.2:34566) 2024-09-10T08:12:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.474+0000 7f4779e00640 1 -- 172.21.15.2:0/2979297155 learned_addr learned my addr 172.21.15.2:0/2979297155 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.474+0000 7f477a800640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f477c112b90 0x7f477c1a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.474+0000 7f4779400640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477c074eb0 0x7f477c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.475+0000 7f4779e00640 1 -- 172.21.15.2:0/2979297155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f477c112b90 msgr2=0x7f477c1a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.475+0000 7f4779e00640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f477c112b90 0x7f477c1a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.475+0000 7f4779e00640 1 -- 172.21.15.2:0/2979297155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477c074eb0 msgr2=0x7f477c19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.475+0000 7f4779e00640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477c074eb0 0x7f477c19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.475+0000 7f4779e00640 1 -- 172.21.15.2:0/2979297155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477c1a3b40 con 0x7f477c074590 2024-09-10T08:12:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.475+0000 7f4779e00640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c074590 0x7f477c19eb10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f476c00d950 tx=0x7f476c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.475+0000 7f4772a00640 1 -- 172.21.15.2:0/2979297155 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c017030 con 0x7f477c074590 2024-09-10T08:12:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.475+0000 7f4772a00640 1 -- 172.21.15.2:0/2979297155 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f476c004020 con 0x7f477c074590 2024-09-10T08:12:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.475+0000 7f4772a00640 1 -- 172.21.15.2:0/2979297155 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c0047a0 con 0x7f477c074590 2024-09-10T08:12:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.475+0000 7f4780e00640 1 -- 172.21.15.2:0/2979297155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477c1a3e30 con 0x7f477c074590 2024-09-10T08:12:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.476+0000 7f4780e00640 1 -- 172.21.15.2:0/2979297155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477c1ab6c0 con 0x7f477c074590 2024-09-10T08:12:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.479+0000 7f4772a00640 1 -- 172.21.15.2:0/2979297155 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f476c0041d0 con 0x7f477c074590 2024-09-10T08:12:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.479+0000 7f4783894740 1 Processor -- start 2024-09-10T08:12:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.479+0000 7f4783894740 1 -- start start 2024-09-10T08:12:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.479+0000 7f4783894740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737ec70 0x557d3737f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.479+0000 7f4783894740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737f590 0x557d37389910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.479+0000 7f4783894740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d37389e50 0x557d3738c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.479+0000 7f4783894740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557d372e53e0 con 0x557d3737ec70 2024-09-10T08:12:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.479+0000 7f4783894740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557d3728cd80 con 0x557d37389e50 2024-09-10T08:12:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.479+0000 7f4783894740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557d372e13a0 con 0x557d3737f590 2024-09-10T08:12:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.480+0000 7f477a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737ec70 0x557d3737f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.480+0000 7f477a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737ec70 0x557d3737f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34568/0 (socket says 172.21.15.2:34568) 2024-09-10T08:12:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.480+0000 7f477a800640 1 -- 172.21.15.2:0/2467720218 learned_addr learned my addr 172.21.15.2:0/2467720218 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.480+0000 7f4779e00640 1 --2- 172.21.15.2:0/2467720218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737f590 0x557d37389910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.480+0000 7f4779400640 1 --2- 172.21.15.2:0/2467720218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d37389e50 0x557d3738c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.480+0000 7f477a800640 1 -- 172.21.15.2:0/2467720218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737f590 msgr2=0x557d37389910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.480+0000 7f477a800640 1 --2- 172.21.15.2:0/2467720218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737f590 0x557d37389910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.480+0000 7f477a800640 1 -- 172.21.15.2:0/2467720218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d37389e50 msgr2=0x557d3738c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.480+0000 7f477a800640 1 --2- 172.21.15.2:0/2467720218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d37389e50 0x557d3738c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.480+0000 7f4772a00640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47600a2c00 0x7f47600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.480+0000 7f477a800640 1 -- 172.21.15.2:0/2467720218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d372abae0 con 0x557d3737ec70 2024-09-10T08:12:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.480+0000 7f4779400640 1 --2- 172.21.15.2:0/2467720218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d37389e50 0x557d3738c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.480+0000 7f4772a00640 1 -- 172.21.15.2:0/2979297155 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f476c0ced50 con 0x7f477c074590 2024-09-10T08:12:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.481+0000 7f477a800640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47600a2c00 0x7f47600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.481+0000 7f477a800640 1 --2- 172.21.15.2:0/2467720218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737ec70 0x557d3737f050 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4774009ca0 tx=0x7f477402f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2784a319cbcdddb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.481+0000 7f4767400640 1 -- 172.21.15.2:0/2467720218 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774034040 con 0x557d3737ec70 2024-09-10T08:12:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.481+0000 7f4767400640 1 -- 172.21.15.2:0/2467720218 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4774004040 con 0x557d3737ec70 2024-09-10T08:12:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.481+0000 7f4767400640 1 -- 172.21.15.2:0/2467720218 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774030660 con 0x557d3737ec70 2024-09-10T08:12:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.482+0000 7f4783894740 1 -- 172.21.15.2:0/2467720218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737ec70 msgr2=0x557d3737f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.482+0000 7f477a800640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47600a2c00 0x7f47600a50c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f477402f8e0 tx=0x7f477404d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.482+0000 7f4783894740 1 --2- 172.21.15.2:0/2467720218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737ec70 0x557d3737f050 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4774009ca0 tx=0x7f477402f450 comp rx=0 tx=0).stop 2024-09-10T08:12:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.482+0000 7f4783894740 1 -- 172.21.15.2:0/2467720218 shutdown_connections 2024-09-10T08:12:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.482+0000 7f4783894740 1 --2- 172.21.15.2:0/2467720218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d37389e50 0x557d3738c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.482+0000 7f4783894740 1 --2- 172.21.15.2:0/2467720218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737f590 0x557d37389910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.482+0000 7f4783894740 1 --2- 172.21.15.2:0/2467720218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737ec70 0x557d3737f050 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.482+0000 7f4783894740 1 -- 172.21.15.2:0/2467720218 >> 172.21.15.2:0/2467720218 conn(0x557d3737b200 msgr2=0x557d3737b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.482+0000 7f4783894740 1 -- 172.21.15.2:0/2467720218 shutdown_connections 2024-09-10T08:12:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.482+0000 7f4783894740 1 -- 172.21.15.2:0/2467720218 wait complete. 2024-09-10T08:12:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.483+0000 7f4783894740 1 -- start start 2024-09-10T08:12:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.483+0000 7f4783894740 1 Processor -- start 2024-09-10T08:12:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.483+0000 7f4783894740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3737ec70 0x557d37386d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.483+0000 7f4783894740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 0x557d373872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.483+0000 7f4783894740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d37389e50 0x557d37387800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4783894740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557d3738ca20 con 0x557d3737f590 2024-09-10T08:12:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4783894740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557d372a00f0 con 0x557d3737ec70 2024-09-10T08:12:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4783894740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557d3738cba0 con 0x557d37389e50 2024-09-10T08:12:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4783894740 10 client.0 _ll_register_callbacks cb 0x557d37287bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4779400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 0x557d373872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4779e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3737ec70 0x557d37386d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4779400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 0x557d373872c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34570/0 (socket says 172.21.15.2:34570) 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4779400640 1 -- 172.21.15.2:0/2929693957 learned_addr learned my addr 172.21.15.2:0/2929693957 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f477a800640 1 --2- 172.21.15.2:0/2929693957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d37389e50 0x557d37387800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4779e00640 1 -- 172.21.15.2:0/2929693957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d37389e50 msgr2=0x557d37387800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4779e00640 1 --2- 172.21.15.2:0/2929693957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d37389e50 0x557d37387800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4779e00640 1 -- 172.21.15.2:0/2929693957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 msgr2=0x557d373872c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4779e00640 1 --2- 172.21.15.2:0/2929693957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 0x557d373872c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4779e00640 1 -- 172.21.15.2:0/2929693957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d370563c0 con 0x557d3737ec70 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.484+0000 7f4779400640 1 --2- 172.21.15.2:0/2929693957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 0x557d373872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.485+0000 7f4779e00640 1 --2- 172.21.15.2:0/2929693957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3737ec70 0x557d37386d80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x557d3737c060 tx=0x7f476c0051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a90373371ef72d74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.485+0000 7f475be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.485+0000 7f475be00640 1 -- 172.21.15.2:0/2929693957 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c09aac0 con 0x557d3737ec70 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.485+0000 7f4783894740 1 -- 172.21.15.2:0/2929693957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4774004040 con 0x557d3737ec70 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.485+0000 7f475be00640 1 -- 172.21.15.2:0/2929693957 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f476c0bb0c0 con 0x557d3737ec70 2024-09-10T08:12:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.485+0000 7f475be00640 1 -- 172.21.15.2:0/2929693957 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f476c0b4050 con 0x557d3737ec70 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.485+0000 7f4783894740 10 client.26573 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.485+0000 7f4783894740 1 -- 172.21.15.2:0/2929693957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557d37398b30 con 0x557d3737ec70 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.485+0000 7f475be00640 1 -- 172.21.15.2:0/2929693957 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4774004040 con 0x557d3737ec70 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.485+0000 7f475be00640 1 -- 172.21.15.2:0/2929693957 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f476c09f110 con 0x557d3737ec70 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.486+0000 7f4783894740 10 client.26573 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.486+0000 7f4783894740 10 client.26573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.486+0000 7f4783894740 20 client.26573 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.486+0000 7f4783894740 1 --2- 172.21.15.2:0/2929693957 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d37399f40 0x557d3739c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.486+0000 7f4783894740 4 client.26573 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.486+0000 7f4783894740 1 -- 172.21.15.2:0/2929693957 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d372f2950 con 0x557d37399f40 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.486+0000 7f4779400640 1 --2- 172.21.15.2:0/2929693957 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d37399f40 0x557d3739c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.486+0000 7f4779400640 1 --2- 172.21.15.2:0/2929693957 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d37399f40 0x557d3739c3b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a04d26b3d139afb server_cookie=202a5536f5d86b4 in_seq=0 out_seq=0 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.487+0000 7f475be00640 10 client.26573 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.487+0000 7f475be00640 1 -- 172.21.15.2:0/2929693957 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d372f2950 con 0x557d37399f40 2024-09-10T08:12:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.487+0000 7f475be00640 10 client.26573 handle_command_reply: tid=0 2024-09-10T08:12:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.487+0000 7f4783894740 1 client.26573 shutdown 2024-09-10T08:12:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.487+0000 7f4783894740 1 -- 172.21.15.2:0/2929693957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3737ec70 msgr2=0x557d37386d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.487+0000 7f4783894740 1 --2- 172.21.15.2:0/2929693957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3737ec70 0x557d37386d80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x557d3737c060 tx=0x7f476c0051f0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.488+0000 7f4783894740 1 -- 172.21.15.2:0/2929693957 shutdown_connections 2024-09-10T08:12:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.488+0000 7f4783894740 1 --2- 172.21.15.2:0/2929693957 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d37399f40 0x557d3739c3b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.488+0000 7f4783894740 1 --2- 172.21.15.2:0/2929693957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d37389e50 0x557d37387800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.488+0000 7f4783894740 1 --2- 172.21.15.2:0/2929693957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 0x557d373872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.488+0000 7f4783894740 1 --2- 172.21.15.2:0/2929693957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3737ec70 0x557d37386d80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.488+0000 7f475be00640 0 client.26573 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.488+0000 7f4783894740 1 -- 172.21.15.2:0/2929693957 >> 172.21.15.2:0/2929693957 conn(0x557d3737b200 msgr2=0x557d3737a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.488+0000 7f4783894740 1 -- 172.21.15.2:0/2929693957 shutdown_connections 2024-09-10T08:12:32.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.488+0000 7f4783894740 1 -- 172.21.15.2:0/2929693957 wait complete. 2024-09-10T08:12:32.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.488+0000 7f4783894740 20 client.26573 trim_cache size 0 max 16384 2024-09-10T08:12:32.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.502+0000 7f4783894740 1 Processor -- start 2024-09-10T08:12:32.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.502+0000 7f4783894740 1 -- start start 2024-09-10T08:12:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.502+0000 7f4783894740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737ec70 0x557d373a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.502+0000 7f4783894740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 0x557d373a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.502+0000 7f4783894740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d37389e50 0x557d37384020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.502+0000 7f4783894740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1b7de0 con 0x557d3737f590 2024-09-10T08:12:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.502+0000 7f4783894740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1b56b0 con 0x557d37389e50 2024-09-10T08:12:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.502+0000 7f4783894740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1b3500 con 0x557d3737ec70 2024-09-10T08:12:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.502+0000 7f4779400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 0x557d373a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.502+0000 7f4779400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 0x557d373a21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34574/0 (socket says 172.21.15.2:34574) 2024-09-10T08:12:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.502+0000 7f4779400640 1 -- 172.21.15.2:0/2720827951 learned_addr learned my addr 172.21.15.2:0/2720827951 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.502+0000 7f4779e00640 1 --2- 172.21.15.2:0/2720827951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737ec70 0x557d373a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.503+0000 7f477a800640 1 --2- 172.21.15.2:0/2720827951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d37389e50 0x557d37384020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.503+0000 7f4779400640 1 -- 172.21.15.2:0/2720827951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737ec70 msgr2=0x557d373a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.503+0000 7f4779400640 1 --2- 172.21.15.2:0/2720827951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737ec70 0x557d373a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.503+0000 7f4779400640 1 -- 172.21.15.2:0/2720827951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d37389e50 msgr2=0x557d37384020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:32.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.503+0000 7f4779400640 1 --2- 172.21.15.2:0/2720827951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d37389e50 0x557d37384020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.503+0000 7f4779400640 1 -- 172.21.15.2:0/2720827951 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d3737be90 con 0x557d3737f590 2024-09-10T08:12:32.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.503+0000 7f4779e00640 1 --2- 172.21.15.2:0/2720827951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737ec70 0x557d373a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:32.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.503+0000 7f4779400640 1 --2- 172.21.15.2:0/2720827951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 0x557d373a21c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x557d37386610 tx=0x7f47680039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c041a8baac558b1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:32.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.503+0000 7f4766a00640 1 -- 172.21.15.2:0/2720827951 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768005020 con 0x557d3737f590 2024-09-10T08:12:32.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.503+0000 7f4766a00640 1 -- 172.21.15.2:0/2720827951 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47680047a0 con 0x557d3737f590 2024-09-10T08:12:32.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.503+0000 7f4766a00640 1 -- 172.21.15.2:0/2720827951 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47680116a0 con 0x557d3737f590 2024-09-10T08:12:32.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.504+0000 7f4783894740 1 -- 172.21.15.2:0/2720827951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 msgr2=0x557d373a21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:32.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.504+0000 7f4783894740 1 --2- 172.21.15.2:0/2720827951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 0x557d373a21c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x557d37386610 tx=0x7f47680039c0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.504+0000 7f4783894740 1 -- 172.21.15.2:0/2720827951 shutdown_connections 2024-09-10T08:12:32.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.504+0000 7f4783894740 1 --2- 172.21.15.2:0/2720827951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d37389e50 0x557d37384020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.504+0000 7f4783894740 1 --2- 172.21.15.2:0/2720827951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3737f590 0x557d373a21c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.504+0000 7f4783894740 1 --2- 172.21.15.2:0/2720827951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737ec70 0x557d373a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.504+0000 7f4783894740 1 -- 172.21.15.2:0/2720827951 >> 172.21.15.2:0/2720827951 conn(0x557d3737b200 msgr2=0x557d37388e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:32.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.504+0000 7f4783894740 1 -- 172.21.15.2:0/2720827951 shutdown_connections 2024-09-10T08:12:32.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.504+0000 7f4783894740 1 -- 172.21.15.2:0/2720827951 wait complete. 2024-09-10T08:12:32.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.505+0000 7f4783894740 1 -- start start 2024-09-10T08:12:32.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.505+0000 7f4783894740 1 Processor -- start 2024-09-10T08:12:32.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.505+0000 7f4783894740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737ec70 0x557d37392570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.505+0000 7f4783894740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3737f590 0x557d37383bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.505+0000 7f4783894740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d37389e50 0x557d373840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.505+0000 7f4783894740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1b3b70 con 0x557d37389e50 2024-09-10T08:12:32.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.505+0000 7f4783894740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1b4fe0 con 0x557d3737f590 2024-09-10T08:12:32.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.505+0000 7f4783894740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1b4240 con 0x557d3737ec70 2024-09-10T08:12:32.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.505+0000 7f4783894740 10 client.0 _ll_register_callbacks cb 0x7f476c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:32.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.506+0000 7f477a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d37389e50 0x557d373840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.506+0000 7f4779400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3737f590 0x557d37383bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.506+0000 7f477a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d37389e50 0x557d373840f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34588/0 (socket says 172.21.15.2:34588) 2024-09-10T08:12:32.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.506+0000 7f477a800640 1 -- 172.21.15.2:0/2574214981 learned_addr learned my addr 172.21.15.2:0/2574214981 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:32.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.506+0000 7f4779e00640 1 --2- 172.21.15.2:0/2574214981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737ec70 0x557d37392570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.506+0000 7f4779400640 1 -- 172.21.15.2:0/2574214981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737ec70 msgr2=0x557d37392570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:32.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.506+0000 7f4779400640 1 --2- 172.21.15.2:0/2574214981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737ec70 0x557d37392570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.506+0000 7f4779400640 1 -- 172.21.15.2:0/2574214981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d37389e50 msgr2=0x557d373840f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:32.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.506+0000 7f4779400640 1 --2- 172.21.15.2:0/2574214981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d37389e50 0x557d373840f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.506+0000 7f4779400640 1 -- 172.21.15.2:0/2574214981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d37384920 con 0x557d3737f590 2024-09-10T08:12:32.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.506+0000 7f477a800640 1 --2- 172.21.15.2:0/2574214981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d37389e50 0x557d373840f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:32.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.507+0000 7f4779400640 1 --2- 172.21.15.2:0/2574214981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3737f590 0x557d37383bb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x557d3737ac50 tx=0x7f476800b970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad44250d74ab5d52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.507+0000 7f475be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.507+0000 7f475be00640 1 -- 172.21.15.2:0/2574214981 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476800bd90 con 0x557d3737f590 2024-09-10T08:12:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.507+0000 7f4783894740 1 -- 172.21.15.2:0/2574214981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47680047a0 con 0x557d3737f590 2024-09-10T08:12:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.507+0000 7f475be00640 1 -- 172.21.15.2:0/2574214981 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4768011440 con 0x557d3737f590 2024-09-10T08:12:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.507+0000 7f475be00640 1 -- 172.21.15.2:0/2574214981 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4768011640 con 0x557d3737f590 2024-09-10T08:12:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.507+0000 7f4783894740 10 client.26579 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.507+0000 7f4783894740 1 -- 172.21.15.2:0/2574214981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557d37404cb0 con 0x557d3737f590 2024-09-10T08:12:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.508+0000 7f475be00640 1 -- 172.21.15.2:0/2574214981 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47680047a0 con 0x557d3737f590 2024-09-10T08:12:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.508+0000 7f475be00640 1 -- 172.21.15.2:0/2574214981 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4768030b60 con 0x557d3737f590 2024-09-10T08:12:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.508+0000 7f4783894740 10 client.26579 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.508+0000 7f4783894740 10 client.26579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.508+0000 7f4783894740 20 client.26579 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.508+0000 7f4783894740 1 --2- 172.21.15.2:0/2574214981 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d37405310 0x557d37407700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.508+0000 7f4783894740 4 client.26579 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:12:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.508+0000 7f4783894740 1 -- 172.21.15.2:0/2574214981 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d372ece60 con 0x557d37405310 2024-09-10T08:12:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.508+0000 7f4779e00640 1 --2- 172.21.15.2:0/2574214981 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d37405310 0x557d37407700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f4779e00640 1 --2- 172.21.15.2:0/2574214981 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d37405310 0x557d37407700 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=195a9b38b19f6523 server_cookie=fb879c3d9a2ab811 in_seq=0 out_seq=0 2024-09-10T08:12:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f475be00640 10 client.26579 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f475be00640 1 -- 172.21.15.2:0/2574214981 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d372ece60 con 0x557d37405310 2024-09-10T08:12:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f475be00640 10 client.26579 handle_command_reply: tid=0 2024-09-10T08:12:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f4783894740 1 client.26579 shutdown 2024-09-10T08:12:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f4783894740 1 -- 172.21.15.2:0/2574214981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3737f590 msgr2=0x557d37383bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f4783894740 1 --2- 172.21.15.2:0/2574214981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3737f590 0x557d37383bb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x557d3737ac50 tx=0x7f476800b970 comp rx=0 tx=0).stop 2024-09-10T08:12:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f4783894740 1 -- 172.21.15.2:0/2574214981 shutdown_connections 2024-09-10T08:12:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f4783894740 1 --2- 172.21.15.2:0/2574214981 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d37405310 0x557d37407700 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f4783894740 1 --2- 172.21.15.2:0/2574214981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d37389e50 0x557d373840f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f4783894740 1 --2- 172.21.15.2:0/2574214981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3737f590 0x557d37383bb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f4783894740 1 --2- 172.21.15.2:0/2574214981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3737ec70 0x557d37392570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f475be00640 0 client.26579 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.509+0000 7f4783894740 1 -- 172.21.15.2:0/2574214981 >> 172.21.15.2:0/2574214981 conn(0x557d3737b200 msgr2=0x557d37379b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.510+0000 7f4783894740 1 -- 172.21.15.2:0/2574214981 shutdown_connections 2024-09-10T08:12:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.510+0000 7f4783894740 1 -- 172.21.15.2:0/2574214981 wait complete. 2024-09-10T08:12:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.510+0000 7f4783894740 20 client.26579 trim_cache size 0 max 16384 2024-09-10T08:12:32.520 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:12:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.510+0000 7f4780e00640 1 -- 172.21.15.2:0/2979297155 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47600a2c00 msgr2=0x7f47600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.510+0000 7f4780e00640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47600a2c00 0x7f47600a50c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f477402f8e0 tx=0x7f477404d000 comp rx=0 tx=0).stop 2024-09-10T08:12:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.510+0000 7f4780e00640 1 -- 172.21.15.2:0/2979297155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c074590 msgr2=0x7f477c19eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.510+0000 7f4780e00640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c074590 0x7f477c19eb10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f476c00d950 tx=0x7f476c00de10 comp rx=0 tx=0).stop 2024-09-10T08:12:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.511+0000 7f4780e00640 1 -- 172.21.15.2:0/2979297155 shutdown_connections 2024-09-10T08:12:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.511+0000 7f4780e00640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47600a2c00 0x7f47600a50c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.511+0000 7f4780e00640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f477c112b90 0x7f477c1a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.511+0000 7f4780e00640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477c074eb0 0x7f477c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.511+0000 7f4780e00640 1 --2- 172.21.15.2:0/2979297155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f477c074590 0x7f477c19eb10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.511+0000 7f4780e00640 1 -- 172.21.15.2:0/2979297155 >> 172.21.15.2:0/2979297155 conn(0x7f477c0fd8c0 msgr2=0x7f477c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.511+0000 7f4780e00640 1 -- 172.21.15.2:0/2979297155 shutdown_connections 2024-09-10T08:12:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:32.511+0000 7f4780e00640 1 -- 172.21.15.2:0/2979297155 wait complete. 2024-09-10T08:12:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:33 smithi002 ceph-mon[30563]: pgmap v519: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-10T08:12:33.820 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:12:33.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:33 smithi155 ceph-mon[35405]: pgmap v519: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-10T08:12:33.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:33 smithi107 ceph-mon[35048]: pgmap v519: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-10T08:12:34.214 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:34.432 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:12:34.432 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:12:34.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.973+0000 7f89d7000640 1 -- 172.21.15.2:0/2806917069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0103ff0 msgr2=0x7f89d01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:34.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.973+0000 7f89d7000640 1 --2- 172.21.15.2:0/2806917069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0103ff0 0x7f89d01043d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009860 tx=0x7f89c002a340 comp rx=0 tx=0).stop 2024-09-10T08:12:34.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.973+0000 7f89d7000640 1 -- 172.21.15.2:0/2806917069 shutdown_connections 2024-09-10T08:12:34.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.973+0000 7f89d7000640 1 --2- 172.21.15.2:0/2806917069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89d0109120 0x7f89d0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:34.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.973+0000 7f89d7000640 1 --2- 172.21.15.2:0/2806917069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89d01049a0 0x7f89d01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:34.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.973+0000 7f89d7000640 1 --2- 172.21.15.2:0/2806917069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0103ff0 0x7f89d01043d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:34.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.973+0000 7f89d7000640 1 -- 172.21.15.2:0/2806917069 >> 172.21.15.2:0/2806917069 conn(0x7f89d00ffa90 msgr2=0x7f89d0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:34.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.973+0000 7f89d7000640 1 -- 172.21.15.2:0/2806917069 shutdown_connections 2024-09-10T08:12:34.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.973+0000 7f89d7000640 1 -- 172.21.15.2:0/2806917069 wait complete. 2024-09-10T08:12:34.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.974+0000 7f89d7000640 1 Processor -- start 2024-09-10T08:12:34.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.974+0000 7f89d7000640 1 -- start start 2024-09-10T08:12:34.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.975+0000 7f89d7000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0103ff0 0x7f89d019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:34.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.975+0000 7f89d7000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89d01049a0 0x7f89d019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.975+0000 7f89d7000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89d0109120 0x7f89d019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.975+0000 7f89d7000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0114140 con 0x7f89d0109120 2024-09-10T08:12:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.975+0000 7f89d7000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0113fc0 con 0x7f89d0103ff0 2024-09-10T08:12:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.975+0000 7f89d7000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01142c0 con 0x7f89d01049a0 2024-09-10T08:12:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.975+0000 7f89d5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89d0109120 0x7f89d019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.975+0000 7f89c7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89d01049a0 0x7f89d019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.975+0000 7f89d5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89d0109120 0x7f89d019eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34598/0 (socket says 172.21.15.2:34598) 2024-09-10T08:12:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.975+0000 7f89d5000640 1 -- 172.21.15.2:0/63710146 learned_addr learned my addr 172.21.15.2:0/63710146 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.975+0000 7f89cfe00640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0103ff0 0x7f89d019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.976+0000 7f89d5000640 1 -- 172.21.15.2:0/63710146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89d01049a0 msgr2=0x7f89d019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.976+0000 7f89d5000640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89d01049a0 0x7f89d019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.976+0000 7f89d5000640 1 -- 172.21.15.2:0/63710146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0103ff0 msgr2=0x7f89d019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.976+0000 7f89d5000640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0103ff0 0x7f89d019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.976+0000 7f89d5000640 1 -- 172.21.15.2:0/63710146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d019f6d0 con 0x7f89d0109120 2024-09-10T08:12:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.976+0000 7f89cfe00640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0103ff0 0x7f89d019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.976+0000 7f89c7400640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89d01049a0 0x7f89d019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.976+0000 7f89d5000640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89d0109120 0x7f89d019eff0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009fd0 tx=0x7f89c800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.976+0000 7f89cd600640 1 -- 172.21.15.2:0/63710146 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8004b20 con 0x7f89d0109120 2024-09-10T08:12:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.976+0000 7f89d7000640 1 -- 172.21.15.2:0/63710146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d019f9c0 con 0x7f89d0109120 2024-09-10T08:12:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.976+0000 7f89cd600640 1 -- 172.21.15.2:0/63710146 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c8003e50 con 0x7f89d0109120 2024-09-10T08:12:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.976+0000 7f89cd600640 1 -- 172.21.15.2:0/63710146 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c800b630 con 0x7f89d0109120 2024-09-10T08:12:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.976+0000 7f89d7000640 1 -- 172.21.15.2:0/63710146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d01a72a0 con 0x7f89d0109120 2024-09-10T08:12:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.977+0000 7f89d7000640 1 -- 172.21.15.2:0/63710146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d01043d0 con 0x7f89d0109120 2024-09-10T08:12:34.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.979+0000 7f89cd600640 1 -- 172.21.15.2:0/63710146 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f89c801d020 con 0x7f89d0109120 2024-09-10T08:12:34.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.981+0000 7f89cd600640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f89b40a2c00 0x7f89b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:34.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.981+0000 7f89cd600640 1 -- 172.21.15.2:0/63710146 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f89c80cf360 con 0x7f89d0109120 2024-09-10T08:12:34.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.981+0000 7f89cfe00640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f89b40a2c00 0x7f89b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:34.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.982+0000 7f89cfe00640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f89b40a2c00 0x7f89b40a50c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f89c002a930 tx=0x7f89c00056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:34.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:34.985+0000 7f89cd600640 1 -- 172.21.15.2:0/63710146 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89c809be70 con 0x7f89d0109120 2024-09-10T08:12:35.156 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:35.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.217+0000 7f89d7000640 1 -- 172.21.15.2:0/63710146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89d010bd70 con 0x7f89d0109120 2024-09-10T08:12:35.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.218+0000 7f89cd600640 1 -- 172.21.15.2:0/63710146 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f89c809b770 con 0x7f89d0109120 2024-09-10T08:12:35.218 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:12:35.218 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:12:35.219 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:12:35.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.220+0000 7f89d7000640 1 -- 172.21.15.2:0/63710146 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f89b40a2c00 msgr2=0x7f89b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:35.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.220+0000 7f89d7000640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f89b40a2c00 0x7f89b40a50c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f89c002a930 tx=0x7f89c00056d0 comp rx=0 tx=0).stop 2024-09-10T08:12:35.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.220+0000 7f89d7000640 1 -- 172.21.15.2:0/63710146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89d0109120 msgr2=0x7f89d019eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:35.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.220+0000 7f89d7000640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89d0109120 0x7f89d019eff0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009fd0 tx=0x7f89c800c800 comp rx=0 tx=0).stop 2024-09-10T08:12:35.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.220+0000 7f89d7000640 1 -- 172.21.15.2:0/63710146 shutdown_connections 2024-09-10T08:12:35.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.220+0000 7f89d7000640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89d0109120 0x7f89d019eff0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:35.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.220+0000 7f89d7000640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f89b40a2c00 0x7f89b40a50c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:35.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.220+0000 7f89d7000640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89d01049a0 0x7f89d019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:35.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.220+0000 7f89d7000640 1 --2- 172.21.15.2:0/63710146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d0103ff0 0x7f89d019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:35.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.220+0000 7f89d7000640 1 -- 172.21.15.2:0/63710146 >> 172.21.15.2:0/63710146 conn(0x7f89d00ffa90 msgr2=0x7f89d0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:35.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.221+0000 7f89d7000640 1 -- 172.21.15.2:0/63710146 shutdown_connections 2024-09-10T08:12:35.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:35.221+0000 7f89d7000640 1 -- 172.21.15.2:0/63710146 wait complete. 2024-09-10T08:12:35.478 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:35 smithi002 ceph-mon[30563]: pgmap v520: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-10T08:12:35.478 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:35 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/63710146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:35 smithi155 ceph-mon[35405]: pgmap v520: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-10T08:12:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:35 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/63710146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:35.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:35 smithi107 ceph-mon[35048]: pgmap v520: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-10T08:12:35.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:35 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/63710146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.040+0000 7f4f80e00640 1 -- 172.21.15.2:0/2802185254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f7c1019c0 msgr2=0x7f4f7c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.040+0000 7f4f80e00640 1 --2- 172.21.15.2:0/2802185254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f7c1019c0 0x7f4f7c101e20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009930 tx=0x7f4f6402a420 comp rx=0 tx=0).stop 2024-09-10T08:12:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.040+0000 7f4f80e00640 1 -- 172.21.15.2:0/2802185254 shutdown_connections 2024-09-10T08:12:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.040+0000 7f4f80e00640 1 --2- 172.21.15.2:0/2802185254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c102360 0x7f4f7c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.040+0000 7f4f80e00640 1 --2- 172.21.15.2:0/2802185254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f7c1019c0 0x7f4f7c101e20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.040+0000 7f4f80e00640 1 --2- 172.21.15.2:0/2802185254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f7c1079c0 0x7f4f7c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.040+0000 7f4f80e00640 1 -- 172.21.15.2:0/2802185254 >> 172.21.15.2:0/2802185254 conn(0x7f4f7c0fd740 msgr2=0x7f4f7c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.040+0000 7f4f80e00640 1 -- 172.21.15.2:0/2802185254 shutdown_connections 2024-09-10T08:12:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.040+0000 7f4f80e00640 1 -- 172.21.15.2:0/2802185254 wait complete. 2024-09-10T08:12:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.041+0000 7f4f80e00640 1 Processor -- start 2024-09-10T08:12:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.041+0000 7f4f80e00640 1 -- start start 2024-09-10T08:12:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f80e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f7c1019c0 0x7f4f7c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f80e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f7c102360 0x7f4f7c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f80e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c1079c0 0x7f4f7c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f80e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c111e10 con 0x7f4f7c1019c0 2024-09-10T08:12:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f80e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c111c90 con 0x7f4f7c1079c0 2024-09-10T08:12:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f80e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c111f90 con 0x7f4f7c102360 2024-09-10T08:12:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f7b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f7c1019c0 0x7f4f7c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f7b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f7c1019c0 0x7f4f7c1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34630/0 (socket says 172.21.15.2:34630) 2024-09-10T08:12:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f7b400640 1 -- 172.21.15.2:0/879617404 learned_addr learned my addr 172.21.15.2:0/879617404 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f7be00640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c1079c0 0x7f4f7c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f72a00640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f7c102360 0x7f4f7c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f7b400640 1 -- 172.21.15.2:0/879617404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f7c102360 msgr2=0x7f4f7c1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f7b400640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f7c102360 0x7f4f7c1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f7b400640 1 -- 172.21.15.2:0/879617404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c1079c0 msgr2=0x7f4f7c19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.042+0000 7f4f7b400640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c1079c0 0x7f4f7c19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.043+0000 7f4f7b400640 1 -- 172.21.15.2:0/879617404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c19d260 con 0x7f4f7c1019c0 2024-09-10T08:12:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.043+0000 7f4f72a00640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f7c102360 0x7f4f7c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.043+0000 7f4f7b400640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f7c1019c0 0x7f4f7c1982b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00e9f0 tx=0x7f4f6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.043+0000 7f4f78c00640 1 -- 172.21.15.2:0/879617404 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c00cd80 con 0x7f4f7c1019c0 2024-09-10T08:12:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.043+0000 7f4f78c00640 1 -- 172.21.15.2:0/879617404 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f6c005500 con 0x7f4f7c1019c0 2024-09-10T08:12:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.043+0000 7f4f78c00640 1 -- 172.21.15.2:0/879617404 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c010690 con 0x7f4f7c1019c0 2024-09-10T08:12:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.043+0000 7f4f80e00640 1 -- 172.21.15.2:0/879617404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7c19d550 con 0x7f4f7c1019c0 2024-09-10T08:12:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.043+0000 7f4f80e00640 1 -- 172.21.15.2:0/879617404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f7c1a4e30 con 0x7f4f7c1019c0 2024-09-10T08:12:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.046+0000 7f4f78c00640 1 -- 172.21.15.2:0/879617404 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4f6c0046c0 con 0x7f4f7c1019c0 2024-09-10T08:12:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.046+0000 7f4f84e9a740 1 Processor -- start 2024-09-10T08:12:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.047+0000 7f4f84e9a740 1 -- start start 2024-09-10T08:12:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.047+0000 7f4f84e9a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b20c0 0x559f6f5b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.047+0000 7f4f84e9a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5b29e0 0x559f6f5bcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.047+0000 7f4f84e9a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5bd2a0 0x559f6f5bf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.047+0000 7f4f84e9a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559f6f514290 con 0x559f6f5b20c0 2024-09-10T08:12:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.047+0000 7f4f84e9a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559f6f4bbb40 con 0x559f6f5b29e0 2024-09-10T08:12:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.047+0000 7f4f84e9a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559f6f510250 con 0x559f6f5bd2a0 2024-09-10T08:12:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.047+0000 7f4f7be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b20c0 0x559f6f5b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.047+0000 7f4f7be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b20c0 0x559f6f5b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34632/0 (socket says 172.21.15.2:34632) 2024-09-10T08:12:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.047+0000 7f4f7be00640 1 -- 172.21.15.2:0/3636562318 learned_addr learned my addr 172.21.15.2:0/3636562318 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.047+0000 7f4f72a00640 1 --2- 172.21.15.2:0/3636562318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5bd2a0 0x559f6f5bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.047+0000 7f4f7b400640 1 --2- 172.21.15.2:0/3636562318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5b29e0 0x559f6f5bcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.048+0000 7f4f72a00640 1 -- 172.21.15.2:0/3636562318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5b29e0 msgr2=0x559f6f5bcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.048+0000 7f4f72a00640 1 --2- 172.21.15.2:0/3636562318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5b29e0 0x559f6f5bcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.048+0000 7f4f72a00640 1 -- 172.21.15.2:0/3636562318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b20c0 msgr2=0x559f6f5b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.048+0000 7f4f72a00640 1 --2- 172.21.15.2:0/3636562318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b20c0 0x559f6f5b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.048+0000 7f4f72a00640 1 -- 172.21.15.2:0/3636562318 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6f2856a0 con 0x559f6f5bd2a0 2024-09-10T08:12:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.048+0000 7f4f78c00640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4f600a2c00 0x7f4f600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.048+0000 7f4f7be00640 1 --2- 172.21.15.2:0/3636562318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b20c0 0x559f6f5b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.048+0000 7f4f78c00640 1 -- 172.21.15.2:0/879617404 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4f6c0cf8e0 con 0x7f4f7c1019c0 2024-09-10T08:12:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.048+0000 7f4f72a00640 1 --2- 172.21.15.2:0/3636562318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5bd2a0 0x559f6f5bf690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4f6402a940 tx=0x7f4f64004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c694da7312d35a6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.048+0000 7f4f7be00640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4f600a2c00 0x7f4f600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.049+0000 7f4f70c00640 1 -- 172.21.15.2:0/3636562318 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64035030 con 0x559f6f5bd2a0 2024-09-10T08:12:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.049+0000 7f4f70c00640 1 -- 172.21.15.2:0/3636562318 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f64005520 con 0x559f6f5bd2a0 2024-09-10T08:12:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.049+0000 7f4f70c00640 1 -- 172.21.15.2:0/3636562318 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f640035f0 con 0x559f6f5bd2a0 2024-09-10T08:12:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.049+0000 7f4f84e9a740 1 -- 172.21.15.2:0/3636562318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5bd2a0 msgr2=0x559f6f5bf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.049+0000 7f4f7be00640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4f600a2c00 0x7f4f600a50c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4f74011810 tx=0x7f4f74009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.049+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/3636562318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5bd2a0 0x559f6f5bf690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4f6402a940 tx=0x7f4f64004040 comp rx=0 tx=0).stop 2024-09-10T08:12:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.050+0000 7f4f84e9a740 1 -- 172.21.15.2:0/3636562318 shutdown_connections 2024-09-10T08:12:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.050+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/3636562318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5bd2a0 0x559f6f5bf690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.050+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/3636562318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5b29e0 0x559f6f5bcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.050+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/3636562318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b20c0 0x559f6f5b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.050+0000 7f4f84e9a740 1 -- 172.21.15.2:0/3636562318 >> 172.21.15.2:0/3636562318 conn(0x559f6f5ae650 msgr2=0x559f6f5aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.050+0000 7f4f84e9a740 1 -- 172.21.15.2:0/3636562318 shutdown_connections 2024-09-10T08:12:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.050+0000 7f4f84e9a740 1 -- 172.21.15.2:0/3636562318 wait complete. 2024-09-10T08:12:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.050+0000 7f4f84e9a740 1 -- start start 2024-09-10T08:12:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.051+0000 7f4f84e9a740 1 Processor -- start 2024-09-10T08:12:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.051+0000 7f4f84e9a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b20c0 0x559f6f5bcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.051+0000 7f4f84e9a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5b29e0 0x559f6f5b9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.051+0000 7f4f84e9a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5bd2a0 0x559f6f5ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.051+0000 7f4f84e9a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559f6f5bff10 con 0x559f6f5b20c0 2024-09-10T08:12:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.051+0000 7f4f84e9a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559f6f4cefa0 con 0x559f6f5b29e0 2024-09-10T08:12:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.051+0000 7f4f84e9a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559f6f5c0090 con 0x559f6f5bd2a0 2024-09-10T08:12:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.051+0000 7f4f84e9a740 10 client.0 _ll_register_callbacks cb 0x559f6f222ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.051+0000 7f4f7b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b20c0 0x559f6f5bcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.052+0000 7f4f7b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b20c0 0x559f6f5bcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34634/0 (socket says 172.21.15.2:34634) 2024-09-10T08:12:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.052+0000 7f4f7b400640 1 -- 172.21.15.2:0/1375736110 learned_addr learned my addr 172.21.15.2:0/1375736110 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.052+0000 7f4f72a00640 1 --2- 172.21.15.2:0/1375736110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5b29e0 0x559f6f5b9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.052+0000 7f4f7be00640 1 --2- 172.21.15.2:0/1375736110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5bd2a0 0x559f6f5ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.052+0000 7f4f72a00640 1 -- 172.21.15.2:0/1375736110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5bd2a0 msgr2=0x559f6f5ba450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.052+0000 7f4f72a00640 1 --2- 172.21.15.2:0/1375736110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5bd2a0 0x559f6f5ba450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.052+0000 7f4f72a00640 1 -- 172.21.15.2:0/1375736110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b20c0 msgr2=0x559f6f5bcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.052+0000 7f4f72a00640 1 --2- 172.21.15.2:0/1375736110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b20c0 0x559f6f5bcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.052+0000 7f4f72a00640 1 -- 172.21.15.2:0/1375736110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6f5bac80 con 0x559f6f5b29e0 2024-09-10T08:12:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.052+0000 7f4f72a00640 1 --2- 172.21.15.2:0/1375736110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5b29e0 0x559f6f5b9ed0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4f64006010 tx=0x7f4f640048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90a00fec366cbe1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.052+0000 7f4f69600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.053+0000 7f4f69600640 1 -- 172.21.15.2:0/1375736110 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64051060 con 0x559f6f5b29e0 2024-09-10T08:12:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.053+0000 7f4f84e9a740 1 -- 172.21.15.2:0/1375736110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f64005520 con 0x559f6f5b29e0 2024-09-10T08:12:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.053+0000 7f4f69600640 1 -- 172.21.15.2:0/1375736110 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4f64004fb0 con 0x559f6f5b29e0 2024-09-10T08:12:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.053+0000 7f4f69600640 1 -- 172.21.15.2:0/1375736110 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4f64058050 con 0x559f6f5b29e0 2024-09-10T08:12:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.053+0000 7f4f84e9a740 10 client.26597 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.053+0000 7f4f84e9a740 1 -- 172.21.15.2:0/1375736110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f6f5cbe60 con 0x559f6f5b29e0 2024-09-10T08:12:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.053+0000 7f4f69600640 1 -- 172.21.15.2:0/1375736110 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f64005520 con 0x559f6f5b29e0 2024-09-10T08:12:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.054+0000 7f4f69600640 1 -- 172.21.15.2:0/1375736110 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4f640315b0 con 0x559f6f5b29e0 2024-09-10T08:12:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.054+0000 7f4f84e9a740 10 client.26597 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.054+0000 7f4f84e9a740 10 client.26597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.054+0000 7f4f84e9a740 20 client.26597 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.054+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/1375736110 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f6f5cd1b0 0x559f6f5cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.054+0000 7f4f84e9a740 4 client.26597 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.054+0000 7f4f84e9a740 1 -- 172.21.15.2:0/1375736110 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f6f521800 con 0x559f6f5cd1b0 2024-09-10T08:12:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.054+0000 7f4f7b400640 1 --2- 172.21.15.2:0/1375736110 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f6f5cd1b0 0x559f6f5cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.054+0000 7f4f7b400640 1 --2- 172.21.15.2:0/1375736110 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f6f5cd1b0 0x559f6f5cf5a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8e1a77449fde78 server_cookie=887ed84cdd5c032a in_seq=0 out_seq=0 2024-09-10T08:12:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.054+0000 7f4f69600640 10 client.26597 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.055+0000 7f4f69600640 1 -- 172.21.15.2:0/1375736110 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f6f521800 con 0x559f6f5cd1b0 2024-09-10T08:12:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.055+0000 7f4f69600640 10 client.26597 handle_command_reply: tid=0 2024-09-10T08:12:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.055+0000 7f4f84e9a740 1 client.26597 shutdown 2024-09-10T08:12:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.055+0000 7f4f84e9a740 1 -- 172.21.15.2:0/1375736110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5b29e0 msgr2=0x559f6f5b9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.055+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/1375736110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5b29e0 0x559f6f5b9ed0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4f64006010 tx=0x7f4f640048f0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.056+0000 7f4f84e9a740 1 -- 172.21.15.2:0/1375736110 shutdown_connections 2024-09-10T08:12:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.056+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/1375736110 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f6f5cd1b0 0x559f6f5cf5a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.056+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/1375736110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5bd2a0 0x559f6f5ba450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.056+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/1375736110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5b29e0 0x559f6f5b9ed0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.056+0000 7f4f69600640 0 client.26597 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.056+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/1375736110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b20c0 0x559f6f5bcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.056+0000 7f4f84e9a740 1 -- 172.21.15.2:0/1375736110 >> 172.21.15.2:0/1375736110 conn(0x559f6f5ae650 msgr2=0x559f6f5b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.056+0000 7f4f84e9a740 1 -- 172.21.15.2:0/1375736110 shutdown_connections 2024-09-10T08:12:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.056+0000 7f4f84e9a740 1 -- 172.21.15.2:0/1375736110 wait complete. 2024-09-10T08:12:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.056+0000 7f4f84e9a740 20 client.26597 trim_cache size 0 max 16384 2024-09-10T08:12:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.074+0000 7f4f84e9a740 1 Processor -- start 2024-09-10T08:12:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.074+0000 7f4f84e9a740 1 -- start start 2024-09-10T08:12:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f84e9a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5b20c0 0x559f6f5b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f84e9a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b29e0 0x559f6f5d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f84e9a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5bd2a0 0x559f6f5d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f84e9a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c1af330 con 0x559f6f5b29e0 2024-09-10T08:12:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f84e9a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c1acc00 con 0x559f6f5bd2a0 2024-09-10T08:12:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f84e9a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c1aaa50 con 0x559f6f5b20c0 2024-09-10T08:12:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f72a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b29e0 0x559f6f5d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f72a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b29e0 0x559f6f5d25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34648/0 (socket says 172.21.15.2:34648) 2024-09-10T08:12:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f72a00640 1 -- 172.21.15.2:0/1924760288 learned_addr learned my addr 172.21.15.2:0/1924760288 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f7b400640 1 --2- 172.21.15.2:0/1924760288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5b20c0 0x559f6f5b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f7be00640 1 --2- 172.21.15.2:0/1924760288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5bd2a0 0x559f6f5d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f72a00640 1 -- 172.21.15.2:0/1924760288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5b20c0 msgr2=0x559f6f5b8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f72a00640 1 --2- 172.21.15.2:0/1924760288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5b20c0 0x559f6f5b8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f72a00640 1 -- 172.21.15.2:0/1924760288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5bd2a0 msgr2=0x559f6f5d2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f72a00640 1 --2- 172.21.15.2:0/1924760288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5bd2a0 0x559f6f5d2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.075+0000 7f4f72a00640 1 -- 172.21.15.2:0/1924760288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6f5b1ed0 con 0x559f6f5b29e0 2024-09-10T08:12:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.076+0000 7f4f72a00640 1 --2- 172.21.15.2:0/1924760288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b29e0 0x559f6f5d25c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4f6402aa10 tx=0x7f4f64004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be62848172685cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.076+0000 7f4f6be00640 1 -- 172.21.15.2:0/1924760288 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64051030 con 0x559f6f5b29e0 2024-09-10T08:12:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.076+0000 7f4f6be00640 1 -- 172.21.15.2:0/1924760288 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f64005490 con 0x559f6f5b29e0 2024-09-10T08:12:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.076+0000 7f4f6be00640 1 -- 172.21.15.2:0/1924760288 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64035070 con 0x559f6f5b29e0 2024-09-10T08:12:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.077+0000 7f4f84e9a740 1 -- 172.21.15.2:0/1924760288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b29e0 msgr2=0x559f6f5d25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.077+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/1924760288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b29e0 0x559f6f5d25c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4f6402aa10 tx=0x7f4f64004b90 comp rx=0 tx=0).stop 2024-09-10T08:12:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.077+0000 7f4f84e9a740 1 -- 172.21.15.2:0/1924760288 shutdown_connections 2024-09-10T08:12:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.077+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/1924760288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f5bd2a0 0x559f6f5d2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.077+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/1924760288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5b29e0 0x559f6f5d25c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.077+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/1924760288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5b20c0 0x559f6f5b8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.077+0000 7f4f84e9a740 1 -- 172.21.15.2:0/1924760288 >> 172.21.15.2:0/1924760288 conn(0x559f6f5ae650 msgr2=0x559f6f5b8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.077+0000 7f4f84e9a740 1 -- 172.21.15.2:0/1924760288 shutdown_connections 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.077+0000 7f4f84e9a740 1 -- 172.21.15.2:0/1924760288 wait complete. 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.077+0000 7f4f84e9a740 1 -- start start 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.078+0000 7f4f84e9a740 1 Processor -- start 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.078+0000 7f4f84e9a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5b20c0 0x559f6f5d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.078+0000 7f4f84e9a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5bd2a0 0x559f6f5d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.078+0000 7f4f84e9a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f634ae0 0x559f6f636ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.078+0000 7f4f84e9a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c1ab0c0 con 0x559f6f5bd2a0 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.078+0000 7f4f84e9a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c1ac530 con 0x559f6f634ae0 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.078+0000 7f4f84e9a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c1ab790 con 0x559f6f5b20c0 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.078+0000 7f4f84e9a740 10 client.0 _ll_register_callbacks cb 0x7f4f74009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.079+0000 7f4f7b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5b20c0 0x559f6f5d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.079+0000 7f4f7be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f634ae0 0x559f6f636ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.079+0000 7f4f7be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f634ae0 0x559f6f636ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60104/0 (socket says 172.21.15.2:60104) 2024-09-10T08:12:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.079+0000 7f4f7be00640 1 -- 172.21.15.2:0/261580228 learned_addr learned my addr 172.21.15.2:0/261580228 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.079+0000 7f4f7b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5b20c0 0x559f6f5d0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35906/0 (socket says 172.21.15.2:35906) 2024-09-10T08:12:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.079+0000 7f4f72a00640 1 --2- 172.21.15.2:0/261580228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5bd2a0 0x559f6f5d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.079+0000 7f4f7b400640 1 -- 172.21.15.2:0/261580228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f634ae0 msgr2=0x559f6f636ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.079+0000 7f4f7b400640 1 --2- 172.21.15.2:0/261580228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f634ae0 0x559f6f636ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.079+0000 7f4f7b400640 1 -- 172.21.15.2:0/261580228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5bd2a0 msgr2=0x559f6f5d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.079+0000 7f4f7b400640 1 --2- 172.21.15.2:0/261580228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5bd2a0 0x559f6f5d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.079+0000 7f4f7b400640 1 -- 172.21.15.2:0/261580228 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6f637410 con 0x559f6f5b20c0 2024-09-10T08:12:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.079+0000 7f4f7b400640 1 --2- 172.21.15.2:0/261580228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5b20c0 0x559f6f5d0970 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c09b9f0 tx=0x7f4f6c0cce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0151f43351d142c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.079+0000 7f4f69600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.080+0000 7f4f69600640 1 -- 172.21.15.2:0/261580228 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c09b830 con 0x559f6f5b20c0 2024-09-10T08:12:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.080+0000 7f4f69600640 1 -- 172.21.15.2:0/261580228 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4f6c0cd0a0 con 0x559f6f5b20c0 2024-09-10T08:12:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.080+0000 7f4f84e9a740 1 -- 172.21.15.2:0/261580228 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f64005490 con 0x559f6f5b20c0 2024-09-10T08:12:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.080+0000 7f4f69600640 1 -- 172.21.15.2:0/261580228 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4f6c09b650 con 0x559f6f5b20c0 2024-09-10T08:12:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.080+0000 7f4f84e9a740 10 client.26560 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.080+0000 7f4f84e9a740 1 -- 172.21.15.2:0/261580228 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f6f6386f0 con 0x559f6f5b20c0 2024-09-10T08:12:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.080+0000 7f4f69600640 1 -- 172.21.15.2:0/261580228 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f64005490 con 0x559f6f5b20c0 2024-09-10T08:12:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.080+0000 7f4f69600640 1 -- 172.21.15.2:0/261580228 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4f6c098360 con 0x559f6f5b20c0 2024-09-10T08:12:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.080+0000 7f4f84e9a740 10 client.26560 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.080+0000 7f4f84e9a740 10 client.26560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.080+0000 7f4f84e9a740 20 client.26560 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.080+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/261580228 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f6f638dc0 0x559f6f63b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.081+0000 7f4f84e9a740 4 client.26560 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:12:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.081+0000 7f4f84e9a740 1 -- 172.21.15.2:0/261580228 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f6f51bd10 con 0x559f6f638dc0 2024-09-10T08:12:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.081+0000 7f4f72a00640 1 --2- 172.21.15.2:0/261580228 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f6f638dc0 0x559f6f63b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.081+0000 7f4f72a00640 1 --2- 172.21.15.2:0/261580228 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f6f638dc0 0x559f6f63b210 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16f9f3ec8c12e115 server_cookie=b52e1bb277832022 in_seq=0 out_seq=0 2024-09-10T08:12:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.081+0000 7f4f69600640 10 client.26560 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.082+0000 7f4f69600640 1 -- 172.21.15.2:0/261580228 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f6f51bd10 con 0x559f6f638dc0 2024-09-10T08:12:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.082+0000 7f4f69600640 10 client.26560 handle_command_reply: tid=0 2024-09-10T08:12:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.082+0000 7f4f84e9a740 1 client.26560 shutdown 2024-09-10T08:12:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f84e9a740 1 -- 172.21.15.2:0/261580228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5b20c0 msgr2=0x559f6f5d0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/261580228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5b20c0 0x559f6f5d0970 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c09b9f0 tx=0x7f4f6c0cce60 comp rx=0 tx=0).stop 2024-09-10T08:12:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f84e9a740 1 -- 172.21.15.2:0/261580228 shutdown_connections 2024-09-10T08:12:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/261580228 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f6f638dc0 0x559f6f63b210 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/261580228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6f634ae0 0x559f6f636ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/261580228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6f5bd2a0 0x559f6f5d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f84e9a740 1 --2- 172.21.15.2:0/261580228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6f5b20c0 0x559f6f5d0970 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f69600640 0 client.26560 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:36.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f84e9a740 1 -- 172.21.15.2:0/261580228 >> 172.21.15.2:0/261580228 conn(0x559f6f5ae650 msgr2=0x559f6f5a9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:36.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f84e9a740 1 -- 172.21.15.2:0/261580228 shutdown_connections 2024-09-10T08:12:36.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f84e9a740 1 -- 172.21.15.2:0/261580228 wait complete. 2024-09-10T08:12:36.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f84e9a740 20 client.26560 trim_cache size 0 max 16384 2024-09-10T08:12:36.119 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:12:36.119 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:12:36.119 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "9293f4f1-901f-4823-ba5d-0075d94a27de", 2024-09-10T08:12:36.119 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:12:36.119 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:12:36.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f80e00640 1 -- 172.21.15.2:0/879617404 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4f600a2c00 msgr2=0x7f4f600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:36.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f80e00640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4f600a2c00 0x7f4f600a50c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4f74011810 tx=0x7f4f74009500 comp rx=0 tx=0).stop 2024-09-10T08:12:36.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.083+0000 7f4f80e00640 1 -- 172.21.15.2:0/879617404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f7c1019c0 msgr2=0x7f4f7c1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:36.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.084+0000 7f4f80e00640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f7c1019c0 0x7f4f7c1982b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00e9f0 tx=0x7f4f6c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.084+0000 7f4f80e00640 1 -- 172.21.15.2:0/879617404 shutdown_connections 2024-09-10T08:12:36.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.084+0000 7f4f80e00640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f7c1079c0 0x7f4f7c19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.084+0000 7f4f80e00640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4f600a2c00 0x7f4f600a50c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.084+0000 7f4f80e00640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f7c102360 0x7f4f7c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.084+0000 7f4f80e00640 1 --2- 172.21.15.2:0/879617404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f7c1019c0 0x7f4f7c1982b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:36.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.084+0000 7f4f80e00640 1 -- 172.21.15.2:0/879617404 >> 172.21.15.2:0/879617404 conn(0x7f4f7c0fd740 msgr2=0x7f4f7c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:36.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.084+0000 7f4f80e00640 1 -- 172.21.15.2:0/879617404 shutdown_connections 2024-09-10T08:12:36.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:36.084+0000 7f4f80e00640 1 -- 172.21.15.2:0/879617404 wait complete. 2024-09-10T08:12:36.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:36 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:12:36.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:36 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:12:36.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:36 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:12:37.050 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id 43e5efd8 --release --await' 2024-09-10T08:12:37.051 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id 43e5efd8 --release --await 2024-09-10T08:12:37.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:37 smithi002 ceph-mon[30563]: pgmap v521: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-10T08:12:37.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:37 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:12:37.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:37 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:37.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:37 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:12:37.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:37 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:37.664 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:12:37.789 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:37 smithi155 ceph-mon[35405]: pgmap v521: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-10T08:12:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:37 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:12:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:37 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:37 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:12:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:37 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:37.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:37 smithi107 ceph-mon[35048]: pgmap v521: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-10T08:12:37.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:37 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:12:37.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:37 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:37.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:37 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:12:37.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:37 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.368+0000 7f5ed0200640 1 -- 172.21.15.2:0/3231595805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8077040 msgr2=0x7f5ec81124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.368+0000 7f5ed0200640 1 --2- 172.21.15.2:0/3231595805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8077040 0x7f5ec81124f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009900 tx=0x7f5eb402a410 comp rx=0 tx=0).stop 2024-09-10T08:12:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.368+0000 7f5ed0200640 1 -- 172.21.15.2:0/3231595805 shutdown_connections 2024-09-10T08:12:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.369+0000 7f5ed0200640 1 --2- 172.21.15.2:0/3231595805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ec8112a30 0x7f5ec8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.369+0000 7f5ed0200640 1 --2- 172.21.15.2:0/3231595805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8077040 0x7f5ec81124f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.369+0000 7f5ed0200640 1 --2- 172.21.15.2:0/3231595805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ec8076720 0x7f5ec8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.369+0000 7f5ed0200640 1 -- 172.21.15.2:0/3231595805 >> 172.21.15.2:0/3231595805 conn(0x7f5ec80ffa90 msgr2=0x7f5ec8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.369+0000 7f5ed0200640 1 -- 172.21.15.2:0/3231595805 shutdown_connections 2024-09-10T08:12:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.369+0000 7f5ed0200640 1 -- 172.21.15.2:0/3231595805 wait complete. 2024-09-10T08:12:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.369+0000 7f5ed0200640 1 Processor -- start 2024-09-10T08:12:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.370+0000 7f5ed0200640 1 -- start start 2024-09-10T08:12:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.370+0000 7f5ed0200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ec8076720 0x7f5ec81a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.370+0000 7f5ed0200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8077040 0x7f5ec81a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.370+0000 7f5ed0200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ec8112a30 0x7f5ec819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.370+0000 7f5ed0200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec8118600 con 0x7f5ec8077040 2024-09-10T08:12:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.370+0000 7f5ed0200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec8118480 con 0x7f5ec8112a30 2024-09-10T08:12:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.370+0000 7f5ed0200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec8118780 con 0x7f5ec8076720 2024-09-10T08:12:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.370+0000 7f5ecce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8077040 0x7f5ec81a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.370+0000 7f5ecce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8077040 0x7f5ec81a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36174/0 (socket says 172.21.15.2:36174) 2024-09-10T08:12:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.370+0000 7f5ecce00640 1 -- 172.21.15.2:0/1997014041 learned_addr learned my addr 172.21.15.2:0/1997014041 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.370+0000 7f5ecd800640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ec8076720 0x7f5ec81a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.370+0000 7f5ece200640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ec8112a30 0x7f5ec819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.371+0000 7f5ecd800640 1 -- 172.21.15.2:0/1997014041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ec8112a30 msgr2=0x7f5ec819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.371+0000 7f5ecd800640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ec8112a30 0x7f5ec819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.371+0000 7f5ecd800640 1 -- 172.21.15.2:0/1997014041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8077040 msgr2=0x7f5ec81a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.371+0000 7f5ecd800640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8077040 0x7f5ec81a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.371+0000 7f5ecd800640 1 -- 172.21.15.2:0/1997014041 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec819efc0 con 0x7f5ec8076720 2024-09-10T08:12:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.371+0000 7f5ecce00640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8077040 0x7f5ec81a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.371+0000 7f5ecd800640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ec8076720 0x7f5ec81a46d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00e9f0 tx=0x7f5ebc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.371+0000 7f5ec6000640 1 -- 172.21.15.2:0/1997014041 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc00ccc0 con 0x7f5ec8076720 2024-09-10T08:12:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.371+0000 7f5ed0200640 1 -- 172.21.15.2:0/1997014041 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec819f2b0 con 0x7f5ec8076720 2024-09-10T08:12:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.371+0000 7f5ec6000640 1 -- 172.21.15.2:0/1997014041 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ebc010910 con 0x7f5ec8076720 2024-09-10T08:12:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.372+0000 7f5ec6000640 1 -- 172.21.15.2:0/1997014041 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc018960 con 0x7f5ec8076720 2024-09-10T08:12:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.372+0000 7f5ed0200640 1 -- 172.21.15.2:0/1997014041 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec81ab5a0 con 0x7f5ec8076720 2024-09-10T08:12:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.374+0000 7f5ed2e7d740 1 Processor -- start 2024-09-10T08:12:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.375+0000 7f5ed2e7d740 1 -- start start 2024-09-10T08:12:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.375+0000 7f5ed2e7d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723c4310 0x5592723c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.375+0000 7f5ed2e7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4c30 0x5592723cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.375+0000 7f5ed2e7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723cf4f0 0x5592723d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.375+0000 7f5ed2e7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55927232a650 con 0x5592723cf4f0 2024-09-10T08:12:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.375+0000 7f5ed2e7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5592722d1f10 con 0x5592723c4c30 2024-09-10T08:12:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.375+0000 7f5ed2e7d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559272326610 con 0x5592723c4310 2024-09-10T08:12:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.375+0000 7f5ece200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723c4310 0x5592723c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.375+0000 7f5ecce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723cf4f0 0x5592723d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.375+0000 7f5ece200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723c4310 0x5592723c46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:48000/0 (socket says 172.21.15.2:48000) 2024-09-10T08:12:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.375+0000 7f5ece200640 1 -- 172.21.15.2:0/2488981999 learned_addr learned my addr 172.21.15.2:0/2488981999 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.375+0000 7f5ecce00640 1 --2- 172.21.15.2:0/2488981999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723cf4f0 0x5592723d18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36182/0 (socket says 172.21.15.2:36182) 2024-09-10T08:12:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.375+0000 7f5ec6000640 1 -- 172.21.15.2:0/1997014041 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5ebc010ac0 con 0x7f5ec8076720 2024-09-10T08:12:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.376+0000 7f5ece200640 1 -- 172.21.15.2:0/2488981999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4c30 msgr2=0x5592723cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.376+0000 7f5ece200640 1 --2- 172.21.15.2:0/2488981999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4c30 0x5592723cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.376+0000 7f5ece200640 1 -- 172.21.15.2:0/2488981999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723cf4f0 msgr2=0x5592723d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.376+0000 7f5ece200640 1 --2- 172.21.15.2:0/2488981999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723cf4f0 0x5592723d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.376+0000 7f5ece200640 1 -- 172.21.15.2:0/2488981999 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559272039580 con 0x5592723c4310 2024-09-10T08:12:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.376+0000 7f5ecce00640 1 --2- 172.21.15.2:0/2488981999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723cf4f0 0x5592723d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.376+0000 7f5ece200640 1 --2- 172.21.15.2:0/2488981999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723c4310 0x5592723c46f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009f30 tx=0x7f5ec00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69933982db2bc13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.376+0000 7f5ebaa00640 1 -- 172.21.15.2:0/2488981999 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004d20 con 0x5592723c4310 2024-09-10T08:12:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.376+0000 7f5ebaa00640 1 -- 172.21.15.2:0/2488981999 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec0004050 con 0x5592723c4310 2024-09-10T08:12:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.376+0000 7f5ebaa00640 1 -- 172.21.15.2:0/2488981999 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0005260 con 0x5592723c4310 2024-09-10T08:12:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.377+0000 7f5ec6000640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5eb00a2b80 0x7f5eb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.377+0000 7f5ec6000640 1 -- 172.21.15.2:0/1997014041 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5ebc0cf820 con 0x7f5ec8076720 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.377+0000 7f5ecd800640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5eb00a2b80 0x7f5eb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.377+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2488981999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723c4310 msgr2=0x5592723c46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.377+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2488981999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723c4310 0x5592723c46f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009f30 tx=0x7f5ec00118d0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.377+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2488981999 shutdown_connections 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.377+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2488981999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723cf4f0 0x5592723d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.377+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2488981999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4c30 0x5592723cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.377+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2488981999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723c4310 0x5592723c46f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.377+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2488981999 >> 172.21.15.2:0/2488981999 conn(0x5592723c08a0 msgr2=0x5592723c0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.377+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2488981999 shutdown_connections 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.377+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2488981999 wait complete. 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.378+0000 7f5ecd800640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5eb00a2b80 0x7f5eb00a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0986b0 tx=0x7f5ebc098010 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.378+0000 7f5ed2e7d740 1 -- start start 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.378+0000 7f5ed2e7d740 1 Processor -- start 2024-09-10T08:12:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ed2e7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4310 0x5592723cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ed2e7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4c30 0x5592723cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ed2e7d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723cf4f0 0x5592723ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ed2e7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5592723d20c0 con 0x5592723c4c30 2024-09-10T08:12:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ed2e7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5592722e5350 con 0x5592723c4310 2024-09-10T08:12:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ed2e7d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5592723d2240 con 0x5592723cf4f0 2024-09-10T08:12:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ed2e7d740 10 client.0 _ll_register_callbacks cb 0x559272308290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ece200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4310 0x5592723cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ece200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4310 0x5592723cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50524/0 (socket says 172.21.15.2:50524) 2024-09-10T08:12:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ece200640 1 -- 172.21.15.2:0/2201636548 learned_addr learned my addr 172.21.15.2:0/2201636548 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ecce00640 1 --2- 172.21.15.2:0/2201636548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4c30 0x5592723cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ece200640 1 --2- 172.21.15.2:0/2201636548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723cf4f0 0x5592723ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ecce00640 1 -- 172.21.15.2:0/2201636548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723cf4f0 msgr2=0x5592723ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.379+0000 7f5ecce00640 1 --2- 172.21.15.2:0/2201636548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723cf4f0 0x5592723ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.380+0000 7f5ecce00640 1 -- 172.21.15.2:0/2201636548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4310 msgr2=0x5592723cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.380+0000 7f5ecce00640 1 --2- 172.21.15.2:0/2201636548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4310 0x5592723cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.380+0000 7f5ecce00640 1 -- 172.21.15.2:0/2201636548 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559272103960 con 0x5592723c4c30 2024-09-10T08:12:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.380+0000 7f5ecce00640 1 --2- 172.21.15.2:0/2201636548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4c30 0x5592723cc8a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4000c00 tx=0x7f5eb40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a85ad0b724ad92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.380+0000 7f5eaf400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:12:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.380+0000 7f5eaf400640 1 -- 172.21.15.2:0/2201636548 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4035040 con 0x5592723c4c30 2024-09-10T08:12:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.380+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2201636548 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec0004050 con 0x5592723c4c30 2024-09-10T08:12:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.380+0000 7f5eaf400640 1 -- 172.21.15.2:0/2201636548 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5eb4003cb0 con 0x5592723c4c30 2024-09-10T08:12:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.381+0000 7f5eaf400640 1 -- 172.21.15.2:0/2201636548 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5eb4005020 con 0x5592723c4c30 2024-09-10T08:12:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.381+0000 7f5ed2e7d740 10 client.16578 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.381+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2201636548 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5592723ddc50 con 0x5592723c4c30 2024-09-10T08:12:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.381+0000 7f5eaf400640 1 -- 172.21.15.2:0/2201636548 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec0004050 con 0x5592723c4c30 2024-09-10T08:12:38.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.381+0000 7f5eaf400640 1 -- 172.21.15.2:0/2201636548 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5eb40315b0 con 0x5592723c4c30 2024-09-10T08:12:38.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.381+0000 7f5ed2e7d740 10 client.16578 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:38.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.381+0000 7f5ed2e7d740 10 client.16578 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:12:38.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.381+0000 7f5ed2e7d740 20 client.16578 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:38.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.381+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2201636548 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5592723df110 0x5592723e1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.381+0000 7f5ed2e7d740 4 client.16578 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:38.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.381+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2201636548 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559272337bc0 con 0x5592723df110 2024-09-10T08:12:38.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.382+0000 7f5ece200640 1 --2- 172.21.15.2:0/2201636548 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5592723df110 0x5592723e1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.382+0000 7f5ece200640 1 --2- 172.21.15.2:0/2201636548 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5592723df110 0x5592723e1500 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37299665b9fc675e server_cookie=d7102d0a71bcd1a0 in_seq=0 out_seq=0 2024-09-10T08:12:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.382+0000 7f5eaf400640 10 client.16578 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.383+0000 7f5eaf400640 1 -- 172.21.15.2:0/2201636548 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559272337bc0 con 0x5592723df110 2024-09-10T08:12:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.383+0000 7f5eaf400640 10 client.16578 handle_command_reply: tid=0 2024-09-10T08:12:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.383+0000 7f5ed2e7d740 1 client.16578 shutdown 2024-09-10T08:12:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.383+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2201636548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4c30 msgr2=0x5592723cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.383+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2201636548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4c30 0x5592723cc8a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4000c00 tx=0x7f5eb40035f0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.383+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2201636548 shutdown_connections 2024-09-10T08:12:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.383+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2201636548 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5592723df110 0x5592723e1500 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.383+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2201636548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723cf4f0 0x5592723ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.384+0000 7f5eaf400640 0 client.16578 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.383+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2201636548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4c30 0x5592723cc8a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.384+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2201636548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4310 0x5592723cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.384+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2201636548 >> 172.21.15.2:0/2201636548 conn(0x5592723c08a0 msgr2=0x5592723bfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.384+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2201636548 shutdown_connections 2024-09-10T08:12:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.384+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2201636548 wait complete. 2024-09-10T08:12:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.384+0000 7f5ed2e7d740 20 client.16578 trim_cache size 0 max 16384 2024-09-10T08:12:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.407+0000 7f5ed2e7d740 1 Processor -- start 2024-09-10T08:12:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.407+0000 7f5ed2e7d740 1 -- start start 2024-09-10T08:12:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.407+0000 7f5ed2e7d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723c4310 0x5592723bf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.407+0000 7f5ed2e7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4c30 0x5592723e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.407+0000 7f5ed2e7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723cf4f0 0x5592723e71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.407+0000 7f5ed2e7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81b7d60 con 0x5592723c4c30 2024-09-10T08:12:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.407+0000 7f5ed2e7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81b5630 con 0x5592723cf4f0 2024-09-10T08:12:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.407+0000 7f5ed2e7d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81b3480 con 0x5592723c4310 2024-09-10T08:12:38.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.407+0000 7f5ecce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4c30 0x5592723e6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.407+0000 7f5ecce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4c30 0x5592723e6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36204/0 (socket says 172.21.15.2:36204) 2024-09-10T08:12:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.407+0000 7f5ecce00640 1 -- 172.21.15.2:0/3499166177 learned_addr learned my addr 172.21.15.2:0/3499166177 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.408+0000 7f5ece200640 1 --2- 172.21.15.2:0/3499166177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723cf4f0 0x5592723e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.408+0000 7f5ece200640 1 --2- 172.21.15.2:0/3499166177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723c4310 0x5592723bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.408+0000 7f5ecce00640 1 -- 172.21.15.2:0/3499166177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723c4310 msgr2=0x5592723bf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.408+0000 7f5ecce00640 1 --2- 172.21.15.2:0/3499166177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723c4310 0x5592723bf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.408+0000 7f5ecce00640 1 -- 172.21.15.2:0/3499166177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723cf4f0 msgr2=0x5592723e71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.408+0000 7f5ecce00640 1 --2- 172.21.15.2:0/3499166177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723cf4f0 0x5592723e71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.408+0000 7f5ecce00640 1 -- 172.21.15.2:0/3499166177 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592723bc260 con 0x5592723c4c30 2024-09-10T08:12:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.408+0000 7f5ece200640 1 --2- 172.21.15.2:0/3499166177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723c4310 0x5592723bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.408+0000 7f5ecce00640 1 --2- 172.21.15.2:0/3499166177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4c30 0x5592723e6c70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4006010 tx=0x7f5eb40095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df753b571544637f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.409+0000 7f5eba000640 1 -- 172.21.15.2:0/3499166177 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4000e60 con 0x5592723c4c30 2024-09-10T08:12:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.409+0000 7f5eba000640 1 -- 172.21.15.2:0/3499166177 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb4005850 con 0x5592723c4c30 2024-09-10T08:12:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.409+0000 7f5eba000640 1 -- 172.21.15.2:0/3499166177 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4035070 con 0x5592723c4c30 2024-09-10T08:12:38.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.409+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/3499166177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4c30 msgr2=0x5592723e6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:38.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.409+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/3499166177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4c30 0x5592723e6c70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4006010 tx=0x7f5eb40095f0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.410+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/3499166177 shutdown_connections 2024-09-10T08:12:38.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.410+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/3499166177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723cf4f0 0x5592723e71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.410+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/3499166177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4c30 0x5592723e6c70 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.410+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/3499166177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723c4310 0x5592723bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.410+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/3499166177 >> 172.21.15.2:0/3499166177 conn(0x5592723c08a0 msgr2=0x5592723bf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.410+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/3499166177 shutdown_connections 2024-09-10T08:12:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.410+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/3499166177 wait complete. 2024-09-10T08:12:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.410+0000 7f5ed2e7d740 1 -- start start 2024-09-10T08:12:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.410+0000 7f5ed2e7d740 1 Processor -- start 2024-09-10T08:12:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.411+0000 7f5ed2e7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4310 0x5592723d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.411+0000 7f5ed2e7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4c30 0x5592723d39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.411+0000 7f5ed2e7d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723cf4f0 0x5592723d3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.411+0000 7f5ed2e7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81b3af0 con 0x5592723c4310 2024-09-10T08:12:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.411+0000 7f5ed2e7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81b4f60 con 0x5592723c4c30 2024-09-10T08:12:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.411+0000 7f5ed2e7d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81b41c0 con 0x5592723cf4f0 2024-09-10T08:12:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.411+0000 7f5ed2e7d740 10 client.0 _ll_register_callbacks cb 0x7f5eb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.411+0000 7f5ecce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4c30 0x5592723d39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.411+0000 7f5ecce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4c30 0x5592723d39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50538/0 (socket says 172.21.15.2:50538) 2024-09-10T08:12:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.411+0000 7f5ecce00640 1 -- 172.21.15.2:0/2308194929 learned_addr learned my addr 172.21.15.2:0/2308194929 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.411+0000 7f5ece200640 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4310 0x5592723d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.411+0000 7f5ece200640 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723cf4f0 0x5592723d3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.412+0000 7f5ecce00640 1 -- 172.21.15.2:0/2308194929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723cf4f0 msgr2=0x5592723d3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.412+0000 7f5ecce00640 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723cf4f0 0x5592723d3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.412+0000 7f5ecce00640 1 -- 172.21.15.2:0/2308194929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4310 msgr2=0x5592723d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.412+0000 7f5ecce00640 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4310 0x5592723d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.412+0000 7f5ecce00640 1 -- 172.21.15.2:0/2308194929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559272448ee0 con 0x5592723c4c30 2024-09-10T08:12:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.412+0000 7f5ece200640 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723cf4f0 0x5592723d3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.412+0000 7f5ecce00640 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4c30 0x5592723d39d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5eb402a450 tx=0x7f5eb4005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db3bc1a9736328dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.412+0000 7f5eaf400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.412+0000 7f5eaf400640 1 -- 172.21.15.2:0/2308194929 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4035050 con 0x5592723c4c30 2024-09-10T08:12:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.412+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2308194929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eb4005850 con 0x5592723c4c30 2024-09-10T08:12:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.412+0000 7f5eaf400640 1 -- 172.21.15.2:0/2308194929 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5eb40051c0 con 0x5592723c4c30 2024-09-10T08:12:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.413+0000 7f5eaf400640 1 -- 172.21.15.2:0/2308194929 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5eb4060050 con 0x5592723c4c30 2024-09-10T08:12:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.413+0000 7f5ed2e7d740 10 client.26618 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.413+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2308194929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55927244a200 con 0x5592723c4c30 2024-09-10T08:12:38.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.413+0000 7f5eaf400640 1 -- 172.21.15.2:0/2308194929 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb4005850 con 0x5592723c4c30 2024-09-10T08:12:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.413+0000 7f5eaf400640 1 -- 172.21.15.2:0/2308194929 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5eb40524f0 con 0x5592723c4c30 2024-09-10T08:12:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.413+0000 7f5ed2e7d740 10 client.26618 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.413+0000 7f5ed2e7d740 10 client.26618 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:12:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.413+0000 7f5ed2e7d740 20 client.26618 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.413+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55927244a860 0x55927244cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.413+0000 7f5ed2e7d740 4 client.26618 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "43e5efd8", "release": true, "await": true}] 2024-09-10T08:12:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.413+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2308194929 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "43e5efd8", "release": true, "await": true}) -- 0x5592723320d0 con 0x55927244a860 2024-09-10T08:12:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.414+0000 7f5ece200640 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55927244a860 0x55927244cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.414+0000 7f5ece200640 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55927244a860 0x55927244cc80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=190f439a0e01b3df server_cookie=9391d9764a72e2ab in_seq=0 out_seq=0 2024-09-10T08:12:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.414+0000 7f5eaf400640 10 client.26618 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.416+0000 7f5eaf400640 1 -- 172.21.15.2:0/2308194929 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5592723320d0 con 0x55927244a860 2024-09-10T08:12:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.416+0000 7f5eaf400640 10 client.26618 handle_command_reply: tid=0 2024-09-10T08:12:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.416+0000 7f5ed2e7d740 1 client.26618 shutdown 2024-09-10T08:12:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.416+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2308194929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4c30 msgr2=0x5592723d39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.416+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4c30 0x5592723d39d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5eb402a450 tx=0x7f5eb4005bd0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.416+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2308194929 shutdown_connections 2024-09-10T08:12:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.416+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55927244a860 0x55927244cc80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.416+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592723cf4f0 0x5592723d3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.416+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5592723c4c30 0x5592723d39d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.416+0000 7f5ed2e7d740 1 --2- 172.21.15.2:0/2308194929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592723c4310 0x5592723d3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.416+0000 7f5eaf400640 0 client.26618 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.416+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2308194929 >> 172.21.15.2:0/2308194929 conn(0x5592723c08a0 msgr2=0x5592723bfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.417+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2308194929 shutdown_connections 2024-09-10T08:12:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.417+0000 7f5ed2e7d740 1 -- 172.21.15.2:0/2308194929 wait complete. 2024-09-10T08:12:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.417+0000 7f5ed2e7d740 20 client.26618 trim_cache size 0 max 16384 2024-09-10T08:12:38.419 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:12:38.419 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:12:38.419 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:12:38.419 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 12, 2024-09-10T08:12:38.419 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:12:38.419 INFO:teuthology.orchestra.run.smithi002.stdout: "43e5efd8": { 2024-09-10T08:12:38.419 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 12, 2024-09-10T08:12:38.419 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:12:38.419 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:12:38.419 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:12:38.419 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:12:38.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:12:38.420 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:12:38.420 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 142.8, 2024-09-10T08:12:38.420 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:12:38.420 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:12:38.420 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:12:38.420 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:12:38.420 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:12:38.420 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:12:38.420 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:38.420 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:38.420 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:38.420 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:38.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:38.421 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:12:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.418+0000 7f5ed0200640 1 -- 172.21.15.2:0/1997014041 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5eb00a2b80 msgr2=0x7f5eb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.418+0000 7f5ed0200640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5eb00a2b80 0x7f5eb00a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0986b0 tx=0x7f5ebc098010 comp rx=0 tx=0).stop 2024-09-10T08:12:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.418+0000 7f5ed0200640 1 -- 172.21.15.2:0/1997014041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ec8076720 msgr2=0x7f5ec81a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.418+0000 7f5ed0200640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ec8076720 0x7f5ec81a46d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00e9f0 tx=0x7f5ebc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.418+0000 7f5ed0200640 1 -- 172.21.15.2:0/1997014041 shutdown_connections 2024-09-10T08:12:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.418+0000 7f5ed0200640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ec8112a30 0x7f5ec819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.418+0000 7f5ed0200640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec8077040 0x7f5ec81a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.418+0000 7f5ed0200640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5eb00a2b80 0x7f5eb00a5040 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.418+0000 7f5ed0200640 1 --2- 172.21.15.2:0/1997014041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ec8076720 0x7f5ec81a46d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.418+0000 7f5ed0200640 1 -- 172.21.15.2:0/1997014041 >> 172.21.15.2:0/1997014041 conn(0x7f5ec80ffa90 msgr2=0x7f5ec8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.418+0000 7f5ed0200640 1 -- 172.21.15.2:0/1997014041 shutdown_connections 2024-09-10T08:12:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:38.418+0000 7f5ed0200640 1 -- 172.21.15.2:0/1997014041 wait complete. 2024-09-10T08:12:38.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:12:38.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:12:38.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:12:38.901 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:38.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:12:38.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:12:38.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:12:38.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:12:38.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:12:38.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:12:39.592 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '43e5efd8', total seconds elapsed: 28.5 2024-09-10T08:12:39.592 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T08:12:39.592 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:12:39.843 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:39 smithi002 ceph-mon[30563]: pgmap v522: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-10T08:12:39.843 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:39 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:12:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:39 smithi155 ceph-mon[35405]: pgmap v522: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-10T08:12:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:39 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:12:39.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:39 smithi107 ceph-mon[35048]: pgmap v522: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-10T08:12:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:39 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:12:40.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.123+0000 7f07f6e00640 1 -- 172.21.15.2:0/2844580926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f01047e0 msgr2=0x7f07f0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.123+0000 7f07f6e00640 1 --2- 172.21.15.2:0/2844580926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f01047e0 0x7f07f0108830 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009900 tx=0x7f07dc02a410 comp rx=0 tx=0).stop 2024-09-10T08:12:40.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.123+0000 7f07f6e00640 1 -- 172.21.15.2:0/2844580926 shutdown_connections 2024-09-10T08:12:40.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.123+0000 7f07f6e00640 1 --2- 172.21.15.2:0/2844580926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0108f60 0x7f07f0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.123+0000 7f07f6e00640 1 --2- 172.21.15.2:0/2844580926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f01047e0 0x7f07f0108830 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.123+0000 7f07f6e00640 1 --2- 172.21.15.2:0/2844580926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07f0103e30 0x7f07f0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.123+0000 7f07f6e00640 1 -- 172.21.15.2:0/2844580926 >> 172.21.15.2:0/2844580926 conn(0x7f07f00ff8d0 msgr2=0x7f07f0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:40.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.124+0000 7f07f6e00640 1 -- 172.21.15.2:0/2844580926 shutdown_connections 2024-09-10T08:12:40.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.124+0000 7f07f6e00640 1 -- 172.21.15.2:0/2844580926 wait complete. 2024-09-10T08:12:40.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.125+0000 7f07f6e00640 1 Processor -- start 2024-09-10T08:12:40.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.125+0000 7f07f6e00640 1 -- start start 2024-09-10T08:12:40.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.125+0000 7f07f6e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103e30 0x7f07f019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.125+0000 7f07f6e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f01047e0 0x7f07f019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f6e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07f0108f60 0x7f07f019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f6e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f07f0113fc0 con 0x7f07f01047e0 2024-09-10T08:12:40.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f6e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f07f0113e40 con 0x7f07f0103e30 2024-09-10T08:12:40.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f6e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07f0114140 con 0x7f07f0108f60 2024-09-10T08:12:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103e30 0x7f07f019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f01047e0 0x7f07f019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103e30 0x7f07f019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50554/0 (socket says 172.21.15.2:50554) 2024-09-10T08:12:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f5a00640 1 -- 172.21.15.2:0/1264578671 learned_addr learned my addr 172.21.15.2:0/1264578671 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f6400640 1 --2- 172.21.15.2:0/1264578671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07f0108f60 0x7f07f019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f5a00640 1 -- 172.21.15.2:0/1264578671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07f0108f60 msgr2=0x7f07f019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f5a00640 1 --2- 172.21.15.2:0/1264578671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07f0108f60 0x7f07f019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f5a00640 1 -- 172.21.15.2:0/1264578671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f01047e0 msgr2=0x7f07f019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f5a00640 1 --2- 172.21.15.2:0/1264578671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f01047e0 0x7f07f019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.126+0000 7f07f5a00640 1 -- 172.21.15.2:0/1264578671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f019f430 con 0x7f07f0103e30 2024-09-10T08:12:40.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.127+0000 7f07f5a00640 1 --2- 172.21.15.2:0/1264578671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103e30 0x7f07f019a480 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f07e400e9f0 tx=0x7f07e400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:40.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.127+0000 7f07ee000640 1 -- 172.21.15.2:0/1264578671 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e400cd80 con 0x7f07f0103e30 2024-09-10T08:12:40.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.127+0000 7f07ee000640 1 -- 172.21.15.2:0/1264578671 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07e4005500 con 0x7f07f0103e30 2024-09-10T08:12:40.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.127+0000 7f07f6e00640 1 -- 172.21.15.2:0/1264578671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f019f720 con 0x7f07f0103e30 2024-09-10T08:12:40.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.127+0000 7f07ee000640 1 -- 172.21.15.2:0/1264578671 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4010690 con 0x7f07f0103e30 2024-09-10T08:12:40.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.127+0000 7f07f6e00640 1 -- 172.21.15.2:0/1264578671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f01a7000 con 0x7f07f0103e30 2024-09-10T08:12:40.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.130+0000 7f07fafec740 1 Processor -- start 2024-09-10T08:12:40.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.130+0000 7f07fafec740 1 -- start start 2024-09-10T08:12:40.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.131+0000 7f07ee000640 1 -- 172.21.15.2:0/1264578671 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f07e40046c0 con 0x7f07f0103e30 2024-09-10T08:12:40.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.131+0000 7f07fafec740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a005d50 0x55fb4a006130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.131+0000 7f07fafec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a0109f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.131+0000 7f07fafec740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a010f30 0x55fb4a013320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.131+0000 7f07fafec740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fb49f6c3e0 con 0x55fb4a005d50 2024-09-10T08:12:40.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.131+0000 7f07fafec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fb49f13d80 con 0x55fb4a006670 2024-09-10T08:12:40.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.131+0000 7f07fafec740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fb49f683a0 con 0x55fb4a010f30 2024-09-10T08:12:40.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.131+0000 7f07f5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a0109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.131+0000 7f07f6400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a005d50 0x55fb4a006130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.131+0000 7f07f5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a0109f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50568/0 (socket says 172.21.15.2:50568) 2024-09-10T08:12:40.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.131+0000 7f07f5a00640 1 -- 172.21.15.2:0/264016679 learned_addr learned my addr 172.21.15.2:0/264016679 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:40.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.131+0000 7f07f5000640 1 --2- 172.21.15.2:0/264016679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a010f30 0x55fb4a013320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.132+0000 7f07f5a00640 1 -- 172.21.15.2:0/264016679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a010f30 msgr2=0x55fb4a013320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.132+0000 7f07f5a00640 1 --2- 172.21.15.2:0/264016679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a010f30 0x55fb4a013320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.132+0000 7f07f5a00640 1 -- 172.21.15.2:0/264016679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a005d50 msgr2=0x55fb4a006130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.132+0000 7f07f5a00640 1 --2- 172.21.15.2:0/264016679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a005d50 0x55fb4a006130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.132+0000 7f07f5a00640 1 -- 172.21.15.2:0/264016679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb49f32ae0 con 0x55fb4a006670 2024-09-10T08:12:40.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.132+0000 7f07f6400640 1 --2- 172.21.15.2:0/264016679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a005d50 0x55fb4a006130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:40.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.132+0000 7f07f5a00640 1 --2- 172.21.15.2:0/264016679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a0109f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f07e40d5a90 tx=0x7f07e40d5b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e336c7c336124291 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:40.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.133+0000 7f07ee000640 1 --2- 172.21.15.2:0/1264578671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f07d80a2b80 0x7f07d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.133+0000 7f07ee000640 1 -- 172.21.15.2:0/1264578671 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f07e40cf870 con 0x7f07f0103e30 2024-09-10T08:12:40.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.133+0000 7f07e2a00640 1 -- 172.21.15.2:0/264016679 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e40de070 con 0x55fb4a006670 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.133+0000 7f07f6400640 1 --2- 172.21.15.2:0/1264578671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f07d80a2b80 0x7f07d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.133+0000 7f07e2a00640 1 -- 172.21.15.2:0/264016679 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07e409c380 con 0x55fb4a006670 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.133+0000 7f07e2a00640 1 -- 172.21.15.2:0/264016679 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e409b7c0 con 0x55fb4a006670 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.134+0000 7f07fafec740 1 -- 172.21.15.2:0/264016679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 msgr2=0x55fb4a0109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.134+0000 7f07f6400640 1 --2- 172.21.15.2:0/1264578671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f07d80a2b80 0x7f07d80a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009fd0 tx=0x7f07e8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.134+0000 7f07fafec740 1 --2- 172.21.15.2:0/264016679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a0109f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f07e40d5a90 tx=0x7f07e40d5b80 comp rx=0 tx=0).stop 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.134+0000 7f07fafec740 1 -- 172.21.15.2:0/264016679 shutdown_connections 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.134+0000 7f07fafec740 1 --2- 172.21.15.2:0/264016679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a010f30 0x55fb4a013320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.134+0000 7f07fafec740 1 --2- 172.21.15.2:0/264016679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a0109f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.134+0000 7f07fafec740 1 --2- 172.21.15.2:0/264016679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a005d50 0x55fb4a006130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.134+0000 7f07fafec740 1 -- 172.21.15.2:0/264016679 >> 172.21.15.2:0/264016679 conn(0x55fb4a0022e0 msgr2=0x55fb4a0026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.134+0000 7f07fafec740 1 -- 172.21.15.2:0/264016679 shutdown_connections 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.134+0000 7f07fafec740 1 -- 172.21.15.2:0/264016679 wait complete. 2024-09-10T08:12:40.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.135+0000 7f07fafec740 1 -- start start 2024-09-10T08:12:40.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.135+0000 7f07fafec740 1 Processor -- start 2024-09-10T08:12:40.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.135+0000 7f07fafec740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a005d50 0x55fb4a00dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.135+0000 7f07fafec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a00e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.135+0000 7f07fafec740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a010f30 0x55fb4a00e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.135+0000 7f07fafec740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fb4a013ba0 con 0x55fb4a010f30 2024-09-10T08:12:40.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.135+0000 7f07fafec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fb49f270f0 con 0x55fb4a006670 2024-09-10T08:12:40.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.135+0000 7f07fafec740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fb4a013d20 con 0x55fb4a005d50 2024-09-10T08:12:40.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.136+0000 7f07fafec740 10 client.0 _ll_register_callbacks cb 0x55fb49f189e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:40.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.136+0000 7f07f6400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a010f30 0x55fb4a00e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.136+0000 7f07f5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a00e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.136+0000 7f07f5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a00e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50570/0 (socket says 172.21.15.2:50570) 2024-09-10T08:12:40.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.136+0000 7f07f5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a005d50 0x55fb4a00dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.136+0000 7f07f5000640 1 -- 172.21.15.2:0/2196083467 learned_addr learned my addr 172.21.15.2:0/2196083467 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:40.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.136+0000 7f07f5000640 1 -- 172.21.15.2:0/2196083467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a005d50 msgr2=0x55fb4a00dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.136+0000 7f07f5000640 1 --2- 172.21.15.2:0/2196083467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a005d50 0x55fb4a00dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.136+0000 7f07f5000640 1 -- 172.21.15.2:0/2196083467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a010f30 msgr2=0x55fb4a00e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.136+0000 7f07f5000640 1 --2- 172.21.15.2:0/2196083467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a010f30 0x55fb4a00e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.136+0000 7f07f5000640 1 -- 172.21.15.2:0/2196083467 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb49cdd3c0 con 0x55fb4a006670 2024-09-10T08:12:40.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.137+0000 7f07f5000640 1 --2- 172.21.15.2:0/2196083467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a00e320 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f07dc033000 tx=0x7f07dc031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a003de56bbe739d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:40.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.137+0000 7f07d7400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:40.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.137+0000 7f07d7400640 1 -- 172.21.15.2:0/2196083467 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc035040 con 0x55fb4a006670 2024-09-10T08:12:40.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.137+0000 7f07d7400640 1 -- 172.21.15.2:0/2196083467 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f07dc004460 con 0x55fb4a006670 2024-09-10T08:12:40.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.137+0000 7f07fafec740 1 -- 172.21.15.2:0/2196083467 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07e409c380 con 0x55fb4a006670 2024-09-10T08:12:40.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.137+0000 7f07d7400640 1 -- 172.21.15.2:0/2196083467 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f07dc04e050 con 0x55fb4a006670 2024-09-10T08:12:40.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.137+0000 7f07fafec740 10 client.26630 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:40.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.137+0000 7f07fafec740 1 -- 172.21.15.2:0/2196083467 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fb4a01f820 con 0x55fb4a006670 2024-09-10T08:12:40.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.138+0000 7f07d7400640 1 -- 172.21.15.2:0/2196083467 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07e409c380 con 0x55fb4a006670 2024-09-10T08:12:40.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.138+0000 7f07d7400640 1 -- 172.21.15.2:0/2196083467 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f07dc03d760 con 0x55fb4a006670 2024-09-10T08:12:40.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.138+0000 7f07fafec740 10 client.26630 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:40.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.138+0000 7f07fafec740 10 client.26630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:40.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.138+0000 7f07fafec740 20 client.26630 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:40.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.138+0000 7f07fafec740 1 --2- 172.21.15.2:0/2196083467 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb4a020d80 0x55fb4a0231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.138+0000 7f07fafec740 4 client.26630 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:40.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.138+0000 7f07fafec740 1 -- 172.21.15.2:0/2196083467 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb49f79950 con 0x55fb4a020d80 2024-09-10T08:12:40.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.139+0000 7f07f5a00640 1 --2- 172.21.15.2:0/2196083467 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb4a020d80 0x55fb4a0231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.139+0000 7f07f5a00640 1 --2- 172.21.15.2:0/2196083467 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb4a020d80 0x55fb4a0231a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6eb4cd9485a4c31 server_cookie=4c046980bb449537 in_seq=0 out_seq=0 2024-09-10T08:12:40.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.139+0000 7f07d7400640 10 client.26630 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:40.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.140+0000 7f07d7400640 1 -- 172.21.15.2:0/2196083467 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb49f79950 con 0x55fb4a020d80 2024-09-10T08:12:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.140+0000 7f07d7400640 10 client.26630 handle_command_reply: tid=0 2024-09-10T08:12:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.140+0000 7f07fafec740 1 client.26630 shutdown 2024-09-10T08:12:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.140+0000 7f07fafec740 1 -- 172.21.15.2:0/2196083467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 msgr2=0x55fb4a00e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.140+0000 7f07fafec740 1 --2- 172.21.15.2:0/2196083467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a00e320 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f07dc033000 tx=0x7f07dc031850 comp rx=0 tx=0).stop 2024-09-10T08:12:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.140+0000 7f07fafec740 1 -- 172.21.15.2:0/2196083467 shutdown_connections 2024-09-10T08:12:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.140+0000 7f07fafec740 1 --2- 172.21.15.2:0/2196083467 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb4a020d80 0x55fb4a0231a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.140+0000 7f07fafec740 1 --2- 172.21.15.2:0/2196083467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a010f30 0x55fb4a00e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.140+0000 7f07fafec740 1 --2- 172.21.15.2:0/2196083467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a00e320 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.140+0000 7f07d7400640 0 client.26630 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.140+0000 7f07fafec740 1 --2- 172.21.15.2:0/2196083467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a005d50 0x55fb4a00dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.140+0000 7f07fafec740 1 -- 172.21.15.2:0/2196083467 >> 172.21.15.2:0/2196083467 conn(0x55fb4a0022e0 msgr2=0x55fb4a008010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.140+0000 7f07fafec740 1 -- 172.21.15.2:0/2196083467 shutdown_connections 2024-09-10T08:12:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.141+0000 7f07fafec740 1 -- 172.21.15.2:0/2196083467 wait complete. 2024-09-10T08:12:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.141+0000 7f07fafec740 20 client.26630 trim_cache size 0 max 16384 2024-09-10T08:12:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.167+0000 7f07fafec740 1 Processor -- start 2024-09-10T08:12:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.167+0000 7f07fafec740 1 -- start start 2024-09-10T08:12:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.167+0000 7f07fafec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a005d50 0x55fb4a00e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.167+0000 7f07fafec740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a006670 0x55fb4a007f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.167+0000 7f07fafec740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a010f30 0x55fb4a028a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.167+0000 7f07fafec740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f07f01b37a0 con 0x55fb4a010f30 2024-09-10T08:12:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.167+0000 7f07fafec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f07f01b1070 con 0x55fb4a005d50 2024-09-10T08:12:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.167+0000 7f07fafec740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07f01aeec0 con 0x55fb4a006670 2024-09-10T08:12:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.168+0000 7f07f6400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a010f30 0x55fb4a028a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.168+0000 7f07f5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a005d50 0x55fb4a00e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.168+0000 7f07f5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a005d50 0x55fb4a00e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50582/0 (socket says 172.21.15.2:50582) 2024-09-10T08:12:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.168+0000 7f07f5a00640 1 -- 172.21.15.2:0/249298566 learned_addr learned my addr 172.21.15.2:0/249298566 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.168+0000 7f07f5000640 1 --2- 172.21.15.2:0/249298566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a006670 0x55fb4a007f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.168+0000 7f07f5a00640 1 -- 172.21.15.2:0/249298566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a006670 msgr2=0x55fb4a007f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.168+0000 7f07f5a00640 1 --2- 172.21.15.2:0/249298566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a006670 0x55fb4a007f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.168+0000 7f07f5a00640 1 -- 172.21.15.2:0/249298566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a010f30 msgr2=0x55fb4a028a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.168+0000 7f07f5a00640 1 --2- 172.21.15.2:0/249298566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a010f30 0x55fb4a028a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.168+0000 7f07f5a00640 1 -- 172.21.15.2:0/249298566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb4a002110 con 0x55fb4a005d50 2024-09-10T08:12:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.169+0000 7f07f5a00640 1 --2- 172.21.15.2:0/249298566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a005d50 0x55fb4a00e6b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f07e409bc10 tx=0x7f07e4009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7af426b27bfc18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.169+0000 7f07e2000640 1 -- 172.21.15.2:0/249298566 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e40de040 con 0x55fb4a005d50 2024-09-10T08:12:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.169+0000 7f07e2000640 1 -- 172.21.15.2:0/249298566 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07e409c380 con 0x55fb4a005d50 2024-09-10T08:12:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.169+0000 7f07e2000640 1 -- 172.21.15.2:0/249298566 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e409b5e0 con 0x55fb4a005d50 2024-09-10T08:12:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.170+0000 7f07fafec740 1 -- 172.21.15.2:0/249298566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a005d50 msgr2=0x55fb4a00e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.170+0000 7f07fafec740 1 --2- 172.21.15.2:0/249298566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a005d50 0x55fb4a00e6b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f07e409bc10 tx=0x7f07e4009330 comp rx=0 tx=0).stop 2024-09-10T08:12:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.170+0000 7f07fafec740 1 -- 172.21.15.2:0/249298566 shutdown_connections 2024-09-10T08:12:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.170+0000 7f07fafec740 1 --2- 172.21.15.2:0/249298566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a010f30 0x55fb4a028a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.170+0000 7f07fafec740 1 --2- 172.21.15.2:0/249298566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a006670 0x55fb4a007f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.170+0000 7f07fafec740 1 --2- 172.21.15.2:0/249298566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a005d50 0x55fb4a00e6b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.170+0000 7f07fafec740 1 -- 172.21.15.2:0/249298566 >> 172.21.15.2:0/249298566 conn(0x55fb4a0022e0 msgr2=0x55fb4a0077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.170+0000 7f07fafec740 1 -- 172.21.15.2:0/249298566 shutdown_connections 2024-09-10T08:12:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.170+0000 7f07fafec740 1 -- 172.21.15.2:0/249298566 wait complete. 2024-09-10T08:12:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.171+0000 7f07fafec740 1 -- start start 2024-09-10T08:12:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.171+0000 7f07fafec740 1 Processor -- start 2024-09-10T08:12:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.171+0000 7f07fafec740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a005d50 0x55fb4a014fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.171+0000 7f07fafec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a0154e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.171+0000 7f07fafec740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a010f30 0x55fb4a015a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.171+0000 7f07fafec740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f07f01af530 con 0x55fb4a005d50 2024-09-10T08:12:40.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.171+0000 7f07fafec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f07f01b09a0 con 0x55fb4a006670 2024-09-10T08:12:40.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.171+0000 7f07fafec740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07f01afc00 con 0x55fb4a010f30 2024-09-10T08:12:40.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.171+0000 7f07fafec740 10 client.0 _ll_register_callbacks cb 0x7f07dc005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:40.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.172+0000 7f07f5a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a005d50 0x55fb4a014fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.172+0000 7f07f5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a0154e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.172+0000 7f07f5a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a005d50 0x55fb4a014fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36264/0 (socket says 172.21.15.2:36264) 2024-09-10T08:12:40.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.172+0000 7f07f5a00640 1 -- 172.21.15.2:0/2085127092 learned_addr learned my addr 172.21.15.2:0/2085127092 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:40.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.172+0000 7f07f6400640 1 --2- 172.21.15.2:0/2085127092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a010f30 0x55fb4a015a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.172+0000 7f07f5000640 1 -- 172.21.15.2:0/2085127092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a010f30 msgr2=0x55fb4a015a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.172+0000 7f07f5000640 1 --2- 172.21.15.2:0/2085127092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a010f30 0x55fb4a015a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.172+0000 7f07f5000640 1 -- 172.21.15.2:0/2085127092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a005d50 msgr2=0x55fb4a014fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.172+0000 7f07f5000640 1 --2- 172.21.15.2:0/2085127092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a005d50 0x55fb4a014fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.172+0000 7f07f5000640 1 -- 172.21.15.2:0/2085127092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb4a08ad30 con 0x55fb4a006670 2024-09-10T08:12:40.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.172+0000 7f07f5000640 1 --2- 172.21.15.2:0/2085127092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a0154e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f07dc006010 tx=0x7f07dc031b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f67c4662d8bea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:40.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.172+0000 7f07d7400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.173+0000 7f07d7400640 1 -- 172.21.15.2:0/2085127092 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc035040 con 0x55fb4a006670 2024-09-10T08:12:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.173+0000 7f07d7400640 1 -- 172.21.15.2:0/2085127092 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f07dc03d450 con 0x55fb4a006670 2024-09-10T08:12:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.173+0000 7f07fafec740 1 -- 172.21.15.2:0/2085127092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07e409c380 con 0x55fb4a006670 2024-09-10T08:12:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.173+0000 7f07d7400640 1 -- 172.21.15.2:0/2085127092 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f07dc061050 con 0x55fb4a006670 2024-09-10T08:12:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.173+0000 7f07fafec740 10 client.26636 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.174+0000 7f07fafec740 1 -- 172.21.15.2:0/2085127092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fb4a08bfc0 con 0x55fb4a006670 2024-09-10T08:12:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.174+0000 7f07d7400640 1 -- 172.21.15.2:0/2085127092 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07e409c380 con 0x55fb4a006670 2024-09-10T08:12:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.174+0000 7f07d7400640 1 -- 172.21.15.2:0/2085127092 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f07dc0046c0 con 0x55fb4a006670 2024-09-10T08:12:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.174+0000 7f07fafec740 10 client.26636 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.174+0000 7f07fafec740 10 client.26636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.174+0000 7f07fafec740 20 client.26636 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.174+0000 7f07fafec740 1 --2- 172.21.15.2:0/2085127092 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb4a08c730 0x55fb4a08eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.174+0000 7f07fafec740 4 client.26636 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:12:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.174+0000 7f07fafec740 1 -- 172.21.15.2:0/2085127092 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb49f73e60 con 0x55fb4a08c730 2024-09-10T08:12:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.175+0000 7f07f5a00640 1 --2- 172.21.15.2:0/2085127092 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb4a08c730 0x55fb4a08eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.175+0000 7f07f5a00640 1 --2- 172.21.15.2:0/2085127092 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb4a08c730 0x55fb4a08eb50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70404330d4f73a8d server_cookie=260975fc561cb928 in_seq=0 out_seq=0 2024-09-10T08:12:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.175+0000 7f07d7400640 10 client.26636 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.175+0000 7f07d7400640 1 -- 172.21.15.2:0/2085127092 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fb49f73e60 con 0x55fb4a08c730 2024-09-10T08:12:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.175+0000 7f07d7400640 10 client.26636 handle_command_reply: tid=0 2024-09-10T08:12:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.175+0000 7f07fafec740 1 client.26636 shutdown 2024-09-10T08:12:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.176+0000 7f07fafec740 1 -- 172.21.15.2:0/2085127092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 msgr2=0x55fb4a0154e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.176+0000 7f07fafec740 1 --2- 172.21.15.2:0/2085127092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a0154e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f07dc006010 tx=0x7f07dc031b30 comp rx=0 tx=0).stop 2024-09-10T08:12:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.176+0000 7f07fafec740 1 -- 172.21.15.2:0/2085127092 shutdown_connections 2024-09-10T08:12:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.176+0000 7f07fafec740 1 --2- 172.21.15.2:0/2085127092 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb4a08c730 0x55fb4a08eb50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.176+0000 7f07fafec740 1 --2- 172.21.15.2:0/2085127092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4a010f30 0x55fb4a015a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.176+0000 7f07fafec740 1 --2- 172.21.15.2:0/2085127092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb4a006670 0x55fb4a0154e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.176+0000 7f07d7400640 0 client.26636 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:40.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.176+0000 7f07fafec740 1 --2- 172.21.15.2:0/2085127092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb4a005d50 0x55fb4a014fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.176+0000 7f07fafec740 1 -- 172.21.15.2:0/2085127092 >> 172.21.15.2:0/2085127092 conn(0x55fb4a0022e0 msgr2=0x55fb4a001380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:40.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.176+0000 7f07fafec740 1 -- 172.21.15.2:0/2085127092 shutdown_connections 2024-09-10T08:12:40.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.176+0000 7f07fafec740 1 -- 172.21.15.2:0/2085127092 wait complete. 2024-09-10T08:12:40.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.176+0000 7f07fafec740 20 client.26636 trim_cache size 0 max 16384 2024-09-10T08:12:40.178 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:12:40.179 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:12:40.179 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:12:40.179 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:12:40.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.177+0000 7f07f6e00640 1 -- 172.21.15.2:0/1264578671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f07d80a2b80 msgr2=0x7f07d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.177+0000 7f07f6e00640 1 --2- 172.21.15.2:0/1264578671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f07d80a2b80 0x7f07d80a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009fd0 tx=0x7f07e8009500 comp rx=0 tx=0).stop 2024-09-10T08:12:40.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.177+0000 7f07f6e00640 1 -- 172.21.15.2:0/1264578671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103e30 msgr2=0x7f07f019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.177+0000 7f07f6e00640 1 --2- 172.21.15.2:0/1264578671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103e30 0x7f07f019a480 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f07e400e9f0 tx=0x7f07e400edf0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.177+0000 7f07f6e00640 1 -- 172.21.15.2:0/1264578671 shutdown_connections 2024-09-10T08:12:40.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.177+0000 7f07f6e00640 1 --2- 172.21.15.2:0/1264578671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f07d80a2b80 0x7f07d80a5040 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.177+0000 7f07f6e00640 1 --2- 172.21.15.2:0/1264578671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07f0108f60 0x7f07f019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.177+0000 7f07f6e00640 1 --2- 172.21.15.2:0/1264578671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07f01047e0 0x7f07f019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.177+0000 7f07f6e00640 1 --2- 172.21.15.2:0/1264578671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07f0103e30 0x7f07f019a480 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.177+0000 7f07f6e00640 1 -- 172.21.15.2:0/1264578671 >> 172.21.15.2:0/1264578671 conn(0x7f07f00ff8d0 msgr2=0x7f07f0100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:40.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.177+0000 7f07f6e00640 1 -- 172.21.15.2:0/1264578671 shutdown_connections 2024-09-10T08:12:40.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:40.177+0000 7f07f6e00640 1 -- 172.21.15.2:0/1264578671 wait complete. 2024-09-10T08:12:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.362+0000 7f0d87400640 1 -- 172.21.15.2:0/329292187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d80102d50 msgr2=0x7f0d801031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.362+0000 7f0d87400640 1 --2- 172.21.15.2:0/329292187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d80102d50 0x7f0d801031d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009900 tx=0x7f0d6c02a410 comp rx=0 tx=0).stop 2024-09-10T08:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.362+0000 7f0d87400640 1 -- 172.21.15.2:0/329292187 shutdown_connections 2024-09-10T08:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.362+0000 7f0d87400640 1 --2- 172.21.15.2:0/329292187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d80103710 0x7f0d80109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.362+0000 7f0d87400640 1 --2- 172.21.15.2:0/329292187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d80102d50 0x7f0d801031d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.362+0000 7f0d87400640 1 --2- 172.21.15.2:0/329292187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80101b60 0x7f0d80101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.362+0000 7f0d87400640 1 -- 172.21.15.2:0/329292187 >> 172.21.15.2:0/329292187 conn(0x7f0d800fd2e0 msgr2=0x7f0d800ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.362+0000 7f0d87400640 1 -- 172.21.15.2:0/329292187 shutdown_connections 2024-09-10T08:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.362+0000 7f0d87400640 1 -- 172.21.15.2:0/329292187 wait complete. 2024-09-10T08:12:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.363+0000 7f0d87400640 1 Processor -- start 2024-09-10T08:12:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.363+0000 7f0d87400640 1 -- start start 2024-09-10T08:12:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d87400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d80101b60 0x7f0d80199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d87400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80102d50 0x7f0d8019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d87400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d80103710 0x7f0d801a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d87400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010d810 con 0x7f0d80101b60 2024-09-10T08:12:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d87400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010d690 con 0x7f0d80103710 2024-09-10T08:12:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d87400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010d990 con 0x7f0d80102d50 2024-09-10T08:12:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d84a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d80101b60 0x7f0d80199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d84a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d80101b60 0x7f0d80199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36286/0 (socket says 172.21.15.2:36286) 2024-09-10T08:12:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d84a00640 1 -- 172.21.15.2:0/1230074350 learned_addr learned my addr 172.21.15.2:0/1230074350 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d85400640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d80103710 0x7f0d801a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d7fe00640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80102d50 0x7f0d8019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d84a00640 1 -- 172.21.15.2:0/1230074350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80102d50 msgr2=0x7f0d8019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d84a00640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80102d50 0x7f0d8019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d84a00640 1 -- 172.21.15.2:0/1230074350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d80103710 msgr2=0x7f0d801a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d84a00640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d80103710 0x7f0d801a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d84a00640 1 -- 172.21.15.2:0/1230074350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d801a1c10 con 0x7f0d80101b60 2024-09-10T08:12:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.364+0000 7f0d7fe00640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80102d50 0x7f0d8019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.365+0000 7f0d84a00640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d80101b60 0x7f0d80199ff0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400e9c0 tx=0x7f0d7400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.365+0000 7f0d7d600640 1 -- 172.21.15.2:0/1230074350 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400cd80 con 0x7f0d80101b60 2024-09-10T08:12:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.365+0000 7f0d7d600640 1 -- 172.21.15.2:0/1230074350 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d74010910 con 0x7f0d80101b60 2024-09-10T08:12:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.365+0000 7f0d87400640 1 -- 172.21.15.2:0/1230074350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d801a1f00 con 0x7f0d80101b60 2024-09-10T08:12:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.365+0000 7f0d7d600640 1 -- 172.21.15.2:0/1230074350 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74018950 con 0x7f0d80101b60 2024-09-10T08:12:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.365+0000 7f0d87400640 1 -- 172.21.15.2:0/1230074350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d801a2370 con 0x7f0d80101b60 2024-09-10T08:12:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.368+0000 7f0d8a44c740 1 Processor -- start 2024-09-10T08:12:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.368+0000 7f0d8a44c740 1 -- start start 2024-09-10T08:12:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d8a44c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fd8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d8a44c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 0x556884fdd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d8a44c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 0x556884fdff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d8a44c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556884f3d100 con 0x556884fd89d0 2024-09-10T08:12:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d8a44c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556884ee51b0 con 0x556884fd8090 2024-09-10T08:12:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d8a44c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556884f390c0 con 0x556884fdda60 2024-09-10T08:12:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d85400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fd8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d85400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fd8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50614/0 (socket says 172.21.15.2:50614) 2024-09-10T08:12:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d85400640 1 -- 172.21.15.2:0/60848968 learned_addr learned my addr 172.21.15.2:0/60848968 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d7fe00640 1 --2- 172.21.15.2:0/60848968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 0x556884fdff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d85400640 1 -- 172.21.15.2:0/60848968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 msgr2=0x556884fdff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d85400640 1 --2- 172.21.15.2:0/60848968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 0x556884fdff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d85400640 1 -- 172.21.15.2:0/60848968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 msgr2=0x556884fdd490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:12:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d85400640 1 --2- 172.21.15.2:0/60848968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 0x556884fdd490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.369+0000 7f0d85400640 1 -- 172.21.15.2:0/60848968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556884c50210 con 0x556884fd8090 2024-09-10T08:12:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.370+0000 7f0d85400640 1 --2- 172.21.15.2:0/60848968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fd8490 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0d78011440 tx=0x7f0d78011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a29144f3544fa61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.370+0000 7f0d71c00640 1 -- 172.21.15.2:0/60848968 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78003c30 con 0x556884fd8090 2024-09-10T08:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.370+0000 7f0d7d600640 1 -- 172.21.15.2:0/1230074350 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0d7401f020 con 0x7f0d80101b60 2024-09-10T08:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.370+0000 7f0d71c00640 1 -- 172.21.15.2:0/60848968 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d78003dd0 con 0x556884fd8090 2024-09-10T08:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.370+0000 7f0d71c00640 1 -- 172.21.15.2:0/60848968 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800fb00 con 0x556884fd8090 2024-09-10T08:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.370+0000 7f0d8a44c740 1 -- 172.21.15.2:0/60848968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 msgr2=0x556884fd8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.370+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/60848968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fd8490 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0d78011440 tx=0x7f0d78011900 comp rx=0 tx=0).stop 2024-09-10T08:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.371+0000 7f0d8a44c740 1 -- 172.21.15.2:0/60848968 shutdown_connections 2024-09-10T08:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.371+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/60848968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 0x556884fdff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.371+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/60848968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 0x556884fdd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.371+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/60848968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fd8490 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.371+0000 7f0d8a44c740 1 -- 172.21.15.2:0/60848968 >> 172.21.15.2:0/60848968 conn(0x556884fd3700 msgr2=0x556884fd3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.371+0000 7f0d8a44c740 1 -- 172.21.15.2:0/60848968 shutdown_connections 2024-09-10T08:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.371+0000 7f0d8a44c740 1 -- 172.21.15.2:0/60848968 wait complete. 2024-09-10T08:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.371+0000 7f0d8a44c740 1 -- start start 2024-09-10T08:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.371+0000 7f0d8a44c740 1 Processor -- start 2024-09-10T08:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d7d600640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d680a2bc0 0x7f0d680a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d7d600640 1 -- 172.21.15.2:0/1230074350 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0d740cf520 con 0x7f0d80101b60 2024-09-10T08:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d8a44c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d8a44c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fd89d0 0x556884fe2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d8a44c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fdda60 0x556884fe9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d8a44c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556884fe0a30 con 0x556884fdda60 2024-09-10T08:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d8a44c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556884ec6090 con 0x556884fd8090 2024-09-10T08:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d8a44c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556884fe0bb0 con 0x556884fd89d0 2024-09-10T08:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d8a44c740 10 client.0 _ll_register_callbacks cb 0x556884f1af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d85400640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d680a2bc0 0x7f0d680a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d84a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d7fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fd89d0 0x556884fe2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d84a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50628/0 (socket says 172.21.15.2:50628) 2024-09-10T08:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d7fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fd89d0 0x556884fe2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:48126/0 (socket says 172.21.15.2:48126) 2024-09-10T08:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.372+0000 7f0d84a00640 1 -- 172.21.15.2:0/2680072659 learned_addr learned my addr 172.21.15.2:0/2680072659 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.373+0000 7f0d85400640 1 --2- 172.21.15.2:0/2680072659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fdda60 0x556884fe9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.373+0000 7f0d84a00640 1 -- 172.21.15.2:0/2680072659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fd89d0 msgr2=0x556884fe2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.373+0000 7f0d84a00640 1 --2- 172.21.15.2:0/2680072659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fd89d0 0x556884fe2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.373+0000 7f0d84a00640 1 -- 172.21.15.2:0/2680072659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fdda60 msgr2=0x556884fe9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.373+0000 7f0d84a00640 1 --2- 172.21.15.2:0/2680072659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fdda60 0x556884fe9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.373+0000 7f0d84a00640 1 -- 172.21.15.2:0/2680072659 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556884d1c620 con 0x556884fd8090 2024-09-10T08:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.373+0000 7f0d84a00640 1 --2- 172.21.15.2:0/2680072659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe22d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0d74098360 tx=0x7f0d7409b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26c12c280da4ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.373+0000 7f0d66000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.373+0000 7f0d85400640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d680a2bc0 0x7f0d680a5080 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0d7801b040 tx=0x7f0d78003c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.373+0000 7f0d66000640 1 -- 172.21.15.2:0/2680072659 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7409b990 con 0x556884fd8090 2024-09-10T08:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.373+0000 7f0d8a44c740 1 -- 172.21.15.2:0/2680072659 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d78003dd0 con 0x556884fd8090 2024-09-10T08:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.373+0000 7f0d66000640 1 -- 172.21.15.2:0/2680072659 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0d74098000 con 0x556884fd8090 2024-09-10T08:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.374+0000 7f0d66000640 1 -- 172.21.15.2:0/2680072659 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0d7409fd40 con 0x556884fd8090 2024-09-10T08:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.374+0000 7f0d8a44c740 10 client.26648 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.374+0000 7f0d8a44c740 1 -- 172.21.15.2:0/2680072659 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556884fec6b0 con 0x556884fd8090 2024-09-10T08:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.374+0000 7f0d66000640 1 -- 172.21.15.2:0/2680072659 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d78003dd0 con 0x556884fd8090 2024-09-10T08:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.374+0000 7f0d66000640 1 -- 172.21.15.2:0/2680072659 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0d74010ac0 con 0x556884fd8090 2024-09-10T08:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.374+0000 7f0d8a44c740 10 client.26648 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.374+0000 7f0d8a44c740 10 client.26648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.374+0000 7f0d8a44c740 20 client.26648 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.374+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/2680072659 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556884fedd80 0x556884ff0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.374+0000 7f0d8a44c740 4 client.26648 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.374+0000 7f0d8a44c740 1 -- 172.21.15.2:0/2680072659 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556884f4a830 con 0x556884fedd80 2024-09-10T08:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.375+0000 7f0d7fe00640 1 --2- 172.21.15.2:0/2680072659 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556884fedd80 0x556884ff0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.375+0000 7f0d7fe00640 1 --2- 172.21.15.2:0/2680072659 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556884fedd80 0x556884ff0170 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a23a12629f0b6d server_cookie=ede3accfc541e8a1 in_seq=0 out_seq=0 2024-09-10T08:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.375+0000 7f0d66000640 10 client.26648 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.376+0000 7f0d66000640 1 -- 172.21.15.2:0/2680072659 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556884f4a830 con 0x556884fedd80 2024-09-10T08:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.376+0000 7f0d66000640 10 client.26648 handle_command_reply: tid=0 2024-09-10T08:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.376+0000 7f0d8a44c740 1 client.26648 shutdown 2024-09-10T08:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.376+0000 7f0d8a44c740 1 -- 172.21.15.2:0/2680072659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 msgr2=0x556884fe22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.376+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/2680072659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe22d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0d74098360 tx=0x7f0d7409b3b0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.376+0000 7f0d8a44c740 1 -- 172.21.15.2:0/2680072659 shutdown_connections 2024-09-10T08:12:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.376+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/2680072659 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556884fedd80 0x556884ff0170 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.376+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/2680072659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fdda60 0x556884fe9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.376+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/2680072659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fd89d0 0x556884fe2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.376+0000 7f0d66000640 0 client.26648 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.376+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/2680072659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe22d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.376+0000 7f0d8a44c740 1 -- 172.21.15.2:0/2680072659 >> 172.21.15.2:0/2680072659 conn(0x556884fd3700 msgr2=0x556884fd4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.377+0000 7f0d8a44c740 1 -- 172.21.15.2:0/2680072659 shutdown_connections 2024-09-10T08:12:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.377+0000 7f0d8a44c740 1 -- 172.21.15.2:0/2680072659 wait complete. 2024-09-10T08:12:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.377+0000 7f0d8a44c740 20 client.26648 trim_cache size 0 max 16384 2024-09-10T08:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.392+0000 7f0d8a44c740 1 Processor -- start 2024-09-10T08:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.392+0000 7f0d8a44c740 1 -- start start 2024-09-10T08:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.392+0000 7f0d8a44c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.392+0000 7f0d8a44c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 0x556884fe9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.392+0000 7f0d8a44c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 0x556884fd2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.392+0000 7f0d8a44c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801af380 con 0x556884fd89d0 2024-09-10T08:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.392+0000 7f0d8a44c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801acc50 con 0x556884fd8090 2024-09-10T08:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.392+0000 7f0d8a44c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801aaaa0 con 0x556884fdda60 2024-09-10T08:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.393+0000 7f0d7fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 0x556884fe9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.393+0000 7f0d84a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.393+0000 7f0d7fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 0x556884fe9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36306/0 (socket says 172.21.15.2:36306) 2024-09-10T08:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.393+0000 7f0d7fe00640 1 -- 172.21.15.2:0/3598569684 learned_addr learned my addr 172.21.15.2:0/3598569684 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.393+0000 7f0d85400640 1 --2- 172.21.15.2:0/3598569684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 0x556884fd2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.393+0000 7f0d84a00640 1 -- 172.21.15.2:0/3598569684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 msgr2=0x556884fd2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.393+0000 7f0d84a00640 1 --2- 172.21.15.2:0/3598569684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 0x556884fd2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.393+0000 7f0d84a00640 1 -- 172.21.15.2:0/3598569684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 msgr2=0x556884fe9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.393+0000 7f0d84a00640 1 --2- 172.21.15.2:0/3598569684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 0x556884fe9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.393+0000 7f0d84a00640 1 -- 172.21.15.2:0/3598569684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556884fd3520 con 0x556884fd8090 2024-09-10T08:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.393+0000 7f0d85400640 1 --2- 172.21.15.2:0/3598569684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 0x556884fd2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.393+0000 7f0d84a00640 1 --2- 172.21.15.2:0/3598569684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe28a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c6a0 tx=0x7f0d74099ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4955a6948701763f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.394+0000 7f0d71200640 1 -- 172.21.15.2:0/3598569684 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7409b990 con 0x556884fd8090 2024-09-10T08:12:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.394+0000 7f0d71200640 1 -- 172.21.15.2:0/3598569684 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d740b4b90 con 0x556884fd8090 2024-09-10T08:12:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.394+0000 7f0d71200640 1 -- 172.21.15.2:0/3598569684 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d740cf200 con 0x556884fd8090 2024-09-10T08:12:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.394+0000 7f0d8a44c740 1 -- 172.21.15.2:0/3598569684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 msgr2=0x556884fe28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.394+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/3598569684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe28a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c6a0 tx=0x7f0d74099ea0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.394+0000 7f0d8a44c740 1 -- 172.21.15.2:0/3598569684 shutdown_connections 2024-09-10T08:12:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.394+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/3598569684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 0x556884fd2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.394+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/3598569684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 0x556884fe9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.394+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/3598569684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe28a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.394+0000 7f0d8a44c740 1 -- 172.21.15.2:0/3598569684 >> 172.21.15.2:0/3598569684 conn(0x556884fd3700 msgr2=0x556884fd2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.395+0000 7f0d8a44c740 1 -- 172.21.15.2:0/3598569684 shutdown_connections 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.395+0000 7f0d8a44c740 1 -- 172.21.15.2:0/3598569684 wait complete. 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.395+0000 7f0d8a44c740 1 -- start start 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.395+0000 7f0d8a44c740 1 Processor -- start 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.395+0000 7f0d8a44c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.395+0000 7f0d8a44c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 0x556884fe6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.395+0000 7f0d8a44c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 0x5568850569f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.395+0000 7f0d8a44c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801ab110 con 0x556884fd89d0 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.395+0000 7f0d8a44c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801ac580 con 0x556884fd8090 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.395+0000 7f0d8a44c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801ab7e0 con 0x556884fdda60 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d8a44c740 10 client.0 _ll_register_callbacks cb 0x7f0d740097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d84a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d7fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 0x556884fe6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d84a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50642/0 (socket says 172.21.15.2:50642) 2024-09-10T08:12:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d84a00640 1 -- 172.21.15.2:0/3555345954 learned_addr learned my addr 172.21.15.2:0/3555345954 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d85400640 1 --2- 172.21.15.2:0/3555345954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 0x5568850569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d84a00640 1 -- 172.21.15.2:0/3555345954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 msgr2=0x5568850569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d84a00640 1 --2- 172.21.15.2:0/3555345954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 0x5568850569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d84a00640 1 -- 172.21.15.2:0/3555345954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 msgr2=0x556884fe6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d84a00640 1 --2- 172.21.15.2:0/3555345954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 0x556884fe6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d84a00640 1 -- 172.21.15.2:0/3555345954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556885056f60 con 0x556884fd8090 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d7fe00640 1 --2- 172.21.15.2:0/3555345954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 0x556884fe6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d84a00640 1 --2- 172.21.15.2:0/3555345954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe6820 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0d74097df0 tx=0x7f0d7409b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb3fd8af6c59d234 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.396+0000 7f0d66000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d66000640 1 -- 172.21.15.2:0/3555345954 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7409b990 con 0x556884fd8090 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d66000640 1 -- 172.21.15.2:0/3555345954 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0d7409ade0 con 0x556884fd8090 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d8a44c740 1 -- 172.21.15.2:0/3555345954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d740b4b90 con 0x556884fd8090 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d66000640 1 -- 172.21.15.2:0/3555345954 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0d7409afe0 con 0x556884fd8090 2024-09-10T08:12:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d8a44c740 10 client.26654 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d8a44c740 1 -- 172.21.15.2:0/3555345954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556885058250 con 0x556884fd8090 2024-09-10T08:12:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d66000640 1 -- 172.21.15.2:0/3555345954 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d740b4b90 con 0x556884fd8090 2024-09-10T08:12:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d66000640 1 -- 172.21.15.2:0/3555345954 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0d7409a850 con 0x556884fd8090 2024-09-10T08:12:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d8a44c740 10 client.26654 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d8a44c740 10 client.26654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d8a44c740 20 client.26654 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/3555345954 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556885058a00 0x55688505adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d8a44c740 4 client.26654 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:12:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.397+0000 7f0d8a44c740 1 -- 172.21.15.2:0/3555345954 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556884fdd820 con 0x556885058a00 2024-09-10T08:12:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.398+0000 7f0d7fe00640 1 --2- 172.21.15.2:0/3555345954 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556885058a00 0x55688505adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.398+0000 7f0d7fe00640 1 --2- 172.21.15.2:0/3555345954 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556885058a00 0x55688505adf0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5667203bbf7765c server_cookie=fe1fbae4b1d30c00 in_seq=0 out_seq=0 2024-09-10T08:12:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.398+0000 7f0d66000640 10 client.26654 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.409+0000 7f0d66000640 1 -- 172.21.15.2:0/3555345954 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556884fdd820 con 0x556885058a00 2024-09-10T08:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.409+0000 7f0d66000640 10 client.26654 handle_command_reply: tid=0 2024-09-10T08:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.409+0000 7f0d8a44c740 1 client.26654 shutdown 2024-09-10T08:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.410+0000 7f0d8a44c740 1 -- 172.21.15.2:0/3555345954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 msgr2=0x556884fe6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.410+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/3555345954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe6820 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0d74097df0 tx=0x7f0d7409b3b0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.410+0000 7f0d8a44c740 1 -- 172.21.15.2:0/3555345954 shutdown_connections 2024-09-10T08:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.410+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/3555345954 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556885058a00 0x55688505adf0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.410+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/3555345954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556884fdda60 0x5568850569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.410+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/3555345954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556884fd89d0 0x556884fe6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.410+0000 7f0d8a44c740 1 --2- 172.21.15.2:0/3555345954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556884fd8090 0x556884fe6820 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.410+0000 7f0d66000640 0 client.26654 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.410+0000 7f0d8a44c740 1 -- 172.21.15.2:0/3555345954 >> 172.21.15.2:0/3555345954 conn(0x556884fd3700 msgr2=0x556884fd2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.410+0000 7f0d8a44c740 1 -- 172.21.15.2:0/3555345954 shutdown_connections 2024-09-10T08:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.410+0000 7f0d8a44c740 1 -- 172.21.15.2:0/3555345954 wait complete. 2024-09-10T08:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.410+0000 7f0d8a44c740 20 client.26654 trim_cache size 0 max 16384 2024-09-10T08:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:12:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.411+0000 7f0d87400640 1 -- 172.21.15.2:0/1230074350 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d680a2bc0 msgr2=0x7f0d680a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.411+0000 7f0d87400640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d680a2bc0 0x7f0d680a5080 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0d7801b040 tx=0x7f0d78003c30 comp rx=0 tx=0).stop 2024-09-10T08:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.411+0000 7f0d87400640 1 -- 172.21.15.2:0/1230074350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d80101b60 msgr2=0x7f0d80199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.411+0000 7f0d87400640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d80101b60 0x7f0d80199ff0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400e9c0 tx=0x7f0d7400edc0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.411+0000 7f0d87400640 1 -- 172.21.15.2:0/1230074350 shutdown_connections 2024-09-10T08:12:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.411+0000 7f0d87400640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d80103710 0x7f0d801a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.411+0000 7f0d87400640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d680a2bc0 0x7f0d680a5080 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.411+0000 7f0d87400640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80102d50 0x7f0d8019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.411+0000 7f0d87400640 1 --2- 172.21.15.2:0/1230074350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d80101b60 0x7f0d80199ff0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.411+0000 7f0d87400640 1 -- 172.21.15.2:0/1230074350 >> 172.21.15.2:0/1230074350 conn(0x7f0d800fd2e0 msgr2=0x7f0d80074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.411+0000 7f0d87400640 1 -- 172.21.15.2:0/1230074350 shutdown_connections 2024-09-10T08:12:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:40.411+0000 7f0d87400640 1 -- 172.21.15.2:0/1230074350 wait complete. 2024-09-10T08:12:40.432 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:41.195 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:12:41.195 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:12:41.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:41 smithi002 ceph-mon[30563]: pgmap v523: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-10T08:12:41.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.760+0000 7fd283e00640 1 -- 172.21.15.2:0/751719478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c1049a0 msgr2=0x7fd27c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.760+0000 7fd283e00640 1 --2- 172.21.15.2:0/751719478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c1049a0 0x7fd27c1089f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd268009900 tx=0x7fd26802a410 comp rx=0 tx=0).stop 2024-09-10T08:12:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.761+0000 7fd283e00640 1 -- 172.21.15.2:0/751719478 shutdown_connections 2024-09-10T08:12:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.761+0000 7fd283e00640 1 --2- 172.21.15.2:0/751719478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd27c109120 0x7fd27c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.761+0000 7fd283e00640 1 --2- 172.21.15.2:0/751719478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c1049a0 0x7fd27c1089f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.761+0000 7fd283e00640 1 --2- 172.21.15.2:0/751719478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd27c103ff0 0x7fd27c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.761+0000 7fd283e00640 1 -- 172.21.15.2:0/751719478 >> 172.21.15.2:0/751719478 conn(0x7fd27c0ffa90 msgr2=0x7fd27c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.761+0000 7fd283e00640 1 -- 172.21.15.2:0/751719478 shutdown_connections 2024-09-10T08:12:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.761+0000 7fd283e00640 1 -- 172.21.15.2:0/751719478 wait complete. 2024-09-10T08:12:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.762+0000 7fd283e00640 1 Processor -- start 2024-09-10T08:12:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.762+0000 7fd283e00640 1 -- start start 2024-09-10T08:12:41.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.762+0000 7fd283e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd27c103ff0 0x7fd27c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.762+0000 7fd283e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c1049a0 0x7fd27c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.762+0000 7fd283e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd27c109120 0x7fd27c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.762+0000 7fd283e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c114190 con 0x7fd27c1049a0 2024-09-10T08:12:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.762+0000 7fd283e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c114010 con 0x7fd27c109120 2024-09-10T08:12:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.762+0000 7fd283e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c114310 con 0x7fd27c103ff0 2024-09-10T08:12:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.763+0000 7fd280a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c1049a0 0x7fd27c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.763+0000 7fd280a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c1049a0 0x7fd27c19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36338/0 (socket says 172.21.15.2:36338) 2024-09-10T08:12:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.763+0000 7fd280a00640 1 -- 172.21.15.2:0/3618164680 learned_addr learned my addr 172.21.15.2:0/3618164680 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:41.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.763+0000 7fd281e00640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd27c109120 0x7fd27c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.763+0000 7fd281400640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd27c103ff0 0x7fd27c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.763+0000 7fd280a00640 1 -- 172.21.15.2:0/3618164680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd27c103ff0 msgr2=0x7fd27c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.763+0000 7fd280a00640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd27c103ff0 0x7fd27c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.763+0000 7fd280a00640 1 -- 172.21.15.2:0/3618164680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd27c109120 msgr2=0x7fd27c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.763+0000 7fd280a00640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd27c109120 0x7fd27c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.763+0000 7fd280a00640 1 -- 172.21.15.2:0/3618164680 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c19f5c0 con 0x7fd27c1049a0 2024-09-10T08:12:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.763+0000 7fd281e00640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd27c109120 0x7fd27c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.764+0000 7fd280a00640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c1049a0 0x7fd27c19ab50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd268004770 tx=0x7fd268003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.764+0000 7fd27a000640 1 -- 172.21.15.2:0/3618164680 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26803f070 con 0x7fd27c1049a0 2024-09-10T08:12:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.764+0000 7fd27a000640 1 -- 172.21.15.2:0/3618164680 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd268031ca0 con 0x7fd27c1049a0 2024-09-10T08:12:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.764+0000 7fd27a000640 1 -- 172.21.15.2:0/3618164680 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268035030 con 0x7fd27c1049a0 2024-09-10T08:12:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.764+0000 7fd283e00640 1 -- 172.21.15.2:0/3618164680 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27c19f850 con 0x7fd27c1049a0 2024-09-10T08:12:41.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.764+0000 7fd283e00640 1 -- 172.21.15.2:0/3618164680 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27c19fce0 con 0x7fd27c1049a0 2024-09-10T08:12:41.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.766+0000 7fd283e00640 1 -- 172.21.15.2:0/3618164680 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27c1043d0 con 0x7fd27c1049a0 2024-09-10T08:12:41.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.767+0000 7fd27a000640 1 -- 172.21.15.2:0/3618164680 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd268042070 con 0x7fd27c1049a0 2024-09-10T08:12:41.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.769+0000 7fd27a000640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd2640a2c00 0x7fd2640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:41.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.769+0000 7fd27a000640 1 -- 172.21.15.2:0/3618164680 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd2680ef940 con 0x7fd27c1049a0 2024-09-10T08:12:41.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.769+0000 7fd281400640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd2640a2c00 0x7fd2640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:41.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.770+0000 7fd281400640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd2640a2c00 0x7fd2640a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd270009bd0 tx=0x7fd270005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:41.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:41.773+0000 7fd27a000640 1 -- 172.21.15.2:0/3618164680 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2680bbf20 con 0x7fd27c1049a0 2024-09-10T08:12:41.853 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:41 smithi155 ceph-mon[35405]: pgmap v523: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-10T08:12:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:41 smithi107 ceph-mon[35048]: pgmap v523: 129 pgs: 129 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-10T08:12:42.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.008+0000 7fd283e00640 1 -- 172.21.15.2:0/3618164680 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd27c02d7c0 con 0x7fd27c1049a0 2024-09-10T08:12:42.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.008+0000 7fd27a000640 1 -- 172.21.15.2:0/3618164680 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fd2680c0220 con 0x7fd27c1049a0 2024-09-10T08:12:42.009 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:12:42.009 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:12:42.009 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:12:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.011+0000 7fd283e00640 1 -- 172.21.15.2:0/3618164680 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd2640a2c00 msgr2=0x7fd2640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.011+0000 7fd283e00640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd2640a2c00 0x7fd2640a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd270009bd0 tx=0x7fd270005f90 comp rx=0 tx=0).stop 2024-09-10T08:12:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.011+0000 7fd283e00640 1 -- 172.21.15.2:0/3618164680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c1049a0 msgr2=0x7fd27c19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.011+0000 7fd283e00640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c1049a0 0x7fd27c19ab50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd268004770 tx=0x7fd268003c10 comp rx=0 tx=0).stop 2024-09-10T08:12:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.011+0000 7fd283e00640 1 -- 172.21.15.2:0/3618164680 shutdown_connections 2024-09-10T08:12:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.011+0000 7fd283e00640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd27c109120 0x7fd27c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.011+0000 7fd283e00640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c1049a0 0x7fd27c19ab50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.011+0000 7fd283e00640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd2640a2c00 0x7fd2640a50c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.011+0000 7fd283e00640 1 --2- 172.21.15.2:0/3618164680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd27c103ff0 0x7fd27c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.011+0000 7fd283e00640 1 -- 172.21.15.2:0/3618164680 >> 172.21.15.2:0/3618164680 conn(0x7fd27c0ffa90 msgr2=0x7fd27c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.011+0000 7fd283e00640 1 -- 172.21.15.2:0/3618164680 shutdown_connections 2024-09-10T08:12:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:42.012+0000 7fd283e00640 1 -- 172.21.15.2:0/3618164680 wait complete. 2024-09-10T08:12:42.773 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:42 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3618164680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:42.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:42 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3618164680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:42.913 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T08:12:42.913 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T08:12:42.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:42 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3618164680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:43.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.401+0000 7efc55000640 1 -- 172.21.15.2:0/4075835644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc50108f60 msgr2=0x7efc50110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:43.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.401+0000 7efc55000640 1 --2- 172.21.15.2:0/4075835644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc50108f60 0x7efc50110ae0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efc48009be0 tx=0x7efc4802a6c0 comp rx=0 tx=0).stop 2024-09-10T08:12:43.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.401+0000 7efc55000640 1 -- 172.21.15.2:0/4075835644 shutdown_connections 2024-09-10T08:12:43.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.401+0000 7efc55000640 1 --2- 172.21.15.2:0/4075835644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc50108f60 0x7efc50110ae0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.401+0000 7efc55000640 1 --2- 172.21.15.2:0/4075835644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc501047e0 0x7efc50108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.401+0000 7efc55000640 1 --2- 172.21.15.2:0/4075835644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc50103e30 0x7efc50104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.401+0000 7efc55000640 1 -- 172.21.15.2:0/4075835644 >> 172.21.15.2:0/4075835644 conn(0x7efc500ff910 msgr2=0x7efc50101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.401+0000 7efc55000640 1 -- 172.21.15.2:0/4075835644 shutdown_connections 2024-09-10T08:12:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.401+0000 7efc55000640 1 -- 172.21.15.2:0/4075835644 wait complete. 2024-09-10T08:12:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.402+0000 7efc55000640 1 Processor -- start 2024-09-10T08:12:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.402+0000 7efc55000640 1 -- start start 2024-09-10T08:12:43.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.403+0000 7efc55000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc50103e30 0x7efc5019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:43.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.403+0000 7efc55000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc501047e0 0x7efc5019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:43.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.403+0000 7efc55000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc50108f60 0x7efc5019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:43.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.403+0000 7efc55000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efc50113fc0 con 0x7efc50108f60 2024-09-10T08:12:43.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.403+0000 7efc55000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efc50113e40 con 0x7efc50103e30 2024-09-10T08:12:43.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.403+0000 7efc55000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efc50114140 con 0x7efc501047e0 2024-09-10T08:12:43.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.403+0000 7efc4fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc50108f60 0x7efc5019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:43.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.403+0000 7efc4fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc50108f60 0x7efc5019ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36358/0 (socket says 172.21.15.2:36358) 2024-09-10T08:12:43.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.403+0000 7efc4fe00640 1 -- 172.21.15.2:0/1523389458 learned_addr learned my addr 172.21.15.2:0/1523389458 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.403+0000 7efc4f400640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc50103e30 0x7efc5019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.403+0000 7efc4ea00640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc501047e0 0x7efc5019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.403+0000 7efc4f400640 1 -- 172.21.15.2:0/1523389458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc501047e0 msgr2=0x7efc5019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.404+0000 7efc4f400640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc501047e0 0x7efc5019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.404+0000 7efc4f400640 1 -- 172.21.15.2:0/1523389458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc50108f60 msgr2=0x7efc5019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.404+0000 7efc4f400640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc50108f60 0x7efc5019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:43.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.404+0000 7efc4f400640 1 -- 172.21.15.2:0/1523389458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5019f3d0 con 0x7efc50103e30 2024-09-10T08:12:43.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.404+0000 7efc4ea00640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc501047e0 0x7efc5019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:43.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.404+0000 7efc4f400640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc50103e30 0x7efc5019a420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efc4000e790 tx=0x7efc4000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:43.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.404+0000 7efc47e00640 1 -- 172.21.15.2:0/1523389458 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4000eea0 con 0x7efc50103e30 2024-09-10T08:12:43.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.404+0000 7efc47e00640 1 -- 172.21.15.2:0/1523389458 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc4000ccc0 con 0x7efc50103e30 2024-09-10T08:12:43.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.404+0000 7efc55000640 1 -- 172.21.15.2:0/1523389458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5019f6c0 con 0x7efc50103e30 2024-09-10T08:12:43.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.404+0000 7efc47e00640 1 -- 172.21.15.2:0/1523389458 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40018780 con 0x7efc50103e30 2024-09-10T08:12:43.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.405+0000 7efc55000640 1 -- 172.21.15.2:0/1523389458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc501a6fa0 con 0x7efc50103e30 2024-09-10T08:12:43.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.406+0000 7efc55000640 1 -- 172.21.15.2:0/1523389458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc50105310 con 0x7efc50103e30 2024-09-10T08:12:43.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.408+0000 7efc47e00640 1 -- 172.21.15.2:0/1523389458 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7efc40010430 con 0x7efc50103e30 2024-09-10T08:12:43.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.409+0000 7efc47e00640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efc340a2c00 0x7efc340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:43.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.409+0000 7efc47e00640 1 -- 172.21.15.2:0/1523389458 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7efc400cfa30 con 0x7efc50103e30 2024-09-10T08:12:43.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.409+0000 7efc4ea00640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efc340a2c00 0x7efc340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:43.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.410+0000 7efc4ea00640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efc340a2c00 0x7efc340a50c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efc500ffd30 tx=0x7efc3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:43.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.411+0000 7efc47e00640 1 -- 172.21.15.2:0/1523389458 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc4009bef0 con 0x7efc50103e30 2024-09-10T08:12:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:43.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.645+0000 7efc55000640 1 -- 172.21.15.2:0/1523389458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc50104210 con 0x7efc50103e30 2024-09-10T08:12:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.646+0000 7efc47e00640 1 -- 172.21.15.2:0/1523389458 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7efc400a01f0 con 0x7efc50103e30 2024-09-10T08:12:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:12:43.646 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:12:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:12:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.649+0000 7efc55000640 1 -- 172.21.15.2:0/1523389458 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efc340a2c00 msgr2=0x7efc340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.649+0000 7efc55000640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efc340a2c00 0x7efc340a50c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efc500ffd30 tx=0x7efc3c005f90 comp rx=0 tx=0).stop 2024-09-10T08:12:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.649+0000 7efc55000640 1 -- 172.21.15.2:0/1523389458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc50103e30 msgr2=0x7efc5019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.649+0000 7efc55000640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc50103e30 0x7efc5019a420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efc4000e790 tx=0x7efc4000ec50 comp rx=0 tx=0).stop 2024-09-10T08:12:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.649+0000 7efc55000640 1 -- 172.21.15.2:0/1523389458 shutdown_connections 2024-09-10T08:12:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.649+0000 7efc55000640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc50108f60 0x7efc5019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.649+0000 7efc55000640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efc340a2c00 0x7efc340a50c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.649+0000 7efc55000640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc501047e0 0x7efc5019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.649+0000 7efc55000640 1 --2- 172.21.15.2:0/1523389458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc50103e30 0x7efc5019a420 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.649+0000 7efc55000640 1 -- 172.21.15.2:0/1523389458 >> 172.21.15.2:0/1523389458 conn(0x7efc500ff910 msgr2=0x7efc50100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.649+0000 7efc55000640 1 -- 172.21.15.2:0/1523389458 shutdown_connections 2024-09-10T08:12:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:43.649+0000 7efc55000640 1 -- 172.21.15.2:0/1523389458 wait complete. 2024-09-10T08:12:43.662 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:43 smithi002 ceph-mon[30563]: pgmap v524: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-09-10T08:12:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:43 smithi155 ceph-mon[35405]: pgmap v524: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-09-10T08:12:43.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:43 smithi107 ceph-mon[35048]: pgmap v524: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-09-10T08:12:44.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.525+0000 7f1566000640 1 -- 172.21.15.2:0/3372685815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560074e30 msgr2=0x7f156010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:44.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.525+0000 7f1566000640 1 --2- 172.21.15.2:0/3372685815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560074e30 0x7f156010dd40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f154c009900 tx=0x7f154c02a480 comp rx=0 tx=0).stop 2024-09-10T08:12:44.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.525+0000 7f1566000640 1 -- 172.21.15.2:0/3372685815 shutdown_connections 2024-09-10T08:12:44.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.525+0000 7f1566000640 1 --2- 172.21.15.2:0/3372685815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f156010e280 0x7f1560110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.525+0000 7f1566000640 1 --2- 172.21.15.2:0/3372685815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560074e30 0x7f156010dd40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.525+0000 7f1566000640 1 --2- 172.21.15.2:0/3372685815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1560074510 0x7f15600748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.525+0000 7f1566000640 1 -- 172.21.15.2:0/3372685815 >> 172.21.15.2:0/3372685815 conn(0x7f15600fd6e0 msgr2=0x7f15600ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:44.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.525+0000 7f1566000640 1 -- 172.21.15.2:0/3372685815 shutdown_connections 2024-09-10T08:12:44.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.525+0000 7f1566000640 1 -- 172.21.15.2:0/3372685815 wait complete. 2024-09-10T08:12:44.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.526+0000 7f1566000640 1 Processor -- start 2024-09-10T08:12:44.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.526+0000 7f1566000640 1 -- start start 2024-09-10T08:12:44.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1566000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560074510 0x7f15601019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1566000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1560074e30 0x7f1560101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1566000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f156010e280 0x7f1560102420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1566000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1560113f70 con 0x7f1560074510 2024-09-10T08:12:44.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1566000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1560113df0 con 0x7f1560074e30 2024-09-10T08:12:44.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1566000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15601140f0 con 0x7f156010e280 2024-09-10T08:12:44.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1564c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560074510 0x7f15601019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1564c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560074510 0x7f15601019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36378/0 (socket says 172.21.15.2:36378) 2024-09-10T08:12:44.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1564c00640 1 -- 172.21.15.2:0/1800513348 learned_addr learned my addr 172.21.15.2:0/1800513348 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:44.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f155fe00640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1560074e30 0x7f1560101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1565600640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f156010e280 0x7f1560102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1564c00640 1 -- 172.21.15.2:0/1800513348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f156010e280 msgr2=0x7f1560102420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:44.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1564c00640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f156010e280 0x7f1560102420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1564c00640 1 -- 172.21.15.2:0/1800513348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1560074e30 msgr2=0x7f1560101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:44.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1564c00640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1560074e30 0x7f1560101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.527+0000 7f1564c00640 1 -- 172.21.15.2:0/1800513348 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15601aabc0 con 0x7f1560074510 2024-09-10T08:12:44.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.528+0000 7f1564c00640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560074510 0x7f15601019a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f155400ea90 tx=0x7f155400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:44.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.528+0000 7f155d600640 1 -- 172.21.15.2:0/1800513348 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155400ce40 con 0x7f1560074510 2024-09-10T08:12:44.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.528+0000 7f155d600640 1 -- 172.21.15.2:0/1800513348 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15540035f0 con 0x7f1560074510 2024-09-10T08:12:44.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.528+0000 7f155d600640 1 -- 172.21.15.2:0/1800513348 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554010690 con 0x7f1560074510 2024-09-10T08:12:44.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.528+0000 7f1566000640 1 -- 172.21.15.2:0/1800513348 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15601aaeb0 con 0x7f1560074510 2024-09-10T08:12:44.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.528+0000 7f1566000640 1 -- 172.21.15.2:0/1800513348 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15601ab2b0 con 0x7f1560074510 2024-09-10T08:12:44.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.532+0000 7f156a08a740 1 Processor -- start 2024-09-10T08:12:44.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.533+0000 7f156a08a740 1 -- start start 2024-09-10T08:12:44.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.533+0000 7f155d600640 1 -- 172.21.15.2:0/1800513348 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f15540037a0 con 0x7f1560074510 2024-09-10T08:12:44.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.533+0000 7f156a08a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5066626c0 0x55a506662aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.533+0000 7f156a08a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a506662fe0 0x55a50666d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.533+0000 7f156a08a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50666d8a0 0x55a50666fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.533+0000 7f156a08a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a5065c7260 con 0x55a506662fe0 2024-09-10T08:12:44.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.533+0000 7f156a08a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a50656e500 con 0x55a5066626c0 2024-09-10T08:12:44.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.533+0000 7f156a08a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a5065c3220 con 0x55a50666d8a0 2024-09-10T08:12:44.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.533+0000 7f1565600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5066626c0 0x55a506662aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.533+0000 7f1565600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5066626c0 0x55a506662aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50722/0 (socket says 172.21.15.2:50722) 2024-09-10T08:12:44.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.533+0000 7f1565600640 1 -- 172.21.15.2:0/4085478334 learned_addr learned my addr 172.21.15.2:0/4085478334 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:44.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.533+0000 7f1564c00640 1 --2- 172.21.15.2:0/4085478334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a506662fe0 0x55a50666d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.533+0000 7f155fe00640 1 --2- 172.21.15.2:0/4085478334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50666d8a0 0x55a50666fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.534+0000 7f1565600640 1 -- 172.21.15.2:0/4085478334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50666d8a0 msgr2=0x55a50666fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:44.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.534+0000 7f1565600640 1 --2- 172.21.15.2:0/4085478334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50666d8a0 0x55a50666fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.534+0000 7f1565600640 1 -- 172.21.15.2:0/4085478334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a506662fe0 msgr2=0x55a50666d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:44.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.534+0000 7f1565600640 1 --2- 172.21.15.2:0/4085478334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a506662fe0 0x55a50666d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.534+0000 7f1565600640 1 -- 172.21.15.2:0/4085478334 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a506338ba0 con 0x55a5066626c0 2024-09-10T08:12:44.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.534+0000 7f1564c00640 1 --2- 172.21.15.2:0/4085478334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a506662fe0 0x55a50666d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:44.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.534+0000 7f1565600640 1 --2- 172.21.15.2:0/4085478334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5066626c0 0x55a506662aa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1558009f00 tx=0x7f15580107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98380682ab89528d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:44.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.535+0000 7f155d600640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f15480a2c00 0x7f15480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.535+0000 7f155d600640 1 -- 172.21.15.2:0/1800513348 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f15540cf7b0 con 0x7f1560074510 2024-09-10T08:12:44.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.535+0000 7f1564c00640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f15480a2c00 0x7f15480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.536+0000 7f1564c00640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f15480a2c00 0x7f15480a50c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f15540cd230 tx=0x7f1554003c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:44.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.537+0000 7f1551c00640 1 -- 172.21.15.2:0/4085478334 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558004d20 con 0x55a5066626c0 2024-09-10T08:12:44.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.537+0000 7f1551c00640 1 -- 172.21.15.2:0/4085478334 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1558015070 con 0x55a5066626c0 2024-09-10T08:12:44.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.537+0000 7f1551c00640 1 -- 172.21.15.2:0/4085478334 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15580118e0 con 0x55a5066626c0 2024-09-10T08:12:44.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.538+0000 7f156a08a740 1 -- 172.21.15.2:0/4085478334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5066626c0 msgr2=0x55a506662aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:44.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.538+0000 7f156a08a740 1 --2- 172.21.15.2:0/4085478334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5066626c0 0x55a506662aa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1558009f00 tx=0x7f15580107a0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.539+0000 7f156a08a740 1 -- 172.21.15.2:0/4085478334 shutdown_connections 2024-09-10T08:12:44.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.539+0000 7f156a08a740 1 --2- 172.21.15.2:0/4085478334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50666d8a0 0x55a50666fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.539+0000 7f156a08a740 1 --2- 172.21.15.2:0/4085478334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a506662fe0 0x55a50666d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.539+0000 7f156a08a740 1 --2- 172.21.15.2:0/4085478334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5066626c0 0x55a506662aa0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.539+0000 7f156a08a740 1 -- 172.21.15.2:0/4085478334 >> 172.21.15.2:0/4085478334 conn(0x55a50665ec50 msgr2=0x55a50665f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:44.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.539+0000 7f156a08a740 1 -- 172.21.15.2:0/4085478334 shutdown_connections 2024-09-10T08:12:44.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.539+0000 7f156a08a740 1 -- 172.21.15.2:0/4085478334 wait complete. 2024-09-10T08:12:44.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.539+0000 7f156a08a740 1 -- start start 2024-09-10T08:12:44.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.540+0000 7f156a08a740 1 Processor -- start 2024-09-10T08:12:44.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.540+0000 7f156a08a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a5066626c0 0x55a50666a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.540+0000 7f156a08a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a506662fe0 0x55a50666ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.540+0000 7f156a08a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50666d8a0 0x55a50666b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.540+0000 7f156a08a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a506670470 con 0x55a5066626c0 2024-09-10T08:12:44.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.540+0000 7f156a08a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a506581bd0 con 0x55a506662fe0 2024-09-10T08:12:44.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.540+0000 7f156a08a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a5066705f0 con 0x55a50666d8a0 2024-09-10T08:12:44.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.540+0000 7f156a08a740 10 client.0 _ll_register_callbacks cb 0x55a5062d60a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:44.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.541+0000 7f1565600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a5066626c0 0x55a50666a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.541+0000 7f155fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a506662fe0 0x55a50666ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.541+0000 7f155fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a506662fe0 0x55a50666ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50726/0 (socket says 172.21.15.2:50726) 2024-09-10T08:12:44.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.541+0000 7f1565600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a5066626c0 0x55a50666a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36396/0 (socket says 172.21.15.2:36396) 2024-09-10T08:12:44.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.541+0000 7f155fe00640 1 -- 172.21.15.2:0/3899043704 learned_addr learned my addr 172.21.15.2:0/3899043704 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:44.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.541+0000 7f1565600640 1 --2- 172.21.15.2:0/3899043704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50666d8a0 0x55a50666b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.541+0000 7f155fe00640 1 -- 172.21.15.2:0/3899043704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50666d8a0 msgr2=0x55a50666b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:44.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.541+0000 7f155fe00640 1 --2- 172.21.15.2:0/3899043704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50666d8a0 0x55a50666b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.541+0000 7f155fe00640 1 -- 172.21.15.2:0/3899043704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a5066626c0 msgr2=0x55a50666a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:44.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.541+0000 7f155fe00640 1 --2- 172.21.15.2:0/3899043704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a5066626c0 0x55a50666a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.541+0000 7f155fe00640 1 -- 172.21.15.2:0/3899043704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5066798e0 con 0x55a506662fe0 2024-09-10T08:12:44.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.542+0000 7f155fe00640 1 --2- 172.21.15.2:0/3899043704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a506662fe0 0x55a50666ace0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f154c009900 tx=0x7f154c031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd7dcd84668ba97c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:44.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.542+0000 7f1546000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:44.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.543+0000 7f1546000640 1 -- 172.21.15.2:0/3899043704 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c035040 con 0x55a506662fe0 2024-09-10T08:12:44.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.543+0000 7f156a08a740 1 -- 172.21.15.2:0/3899043704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1558015070 con 0x55a506662fe0 2024-09-10T08:12:44.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.543+0000 7f1546000640 1 -- 172.21.15.2:0/3899043704 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f154c004460 con 0x55a506662fe0 2024-09-10T08:12:44.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.544+0000 7f1546000640 1 -- 172.21.15.2:0/3899043704 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f154c0035f0 con 0x55a506662fe0 2024-09-10T08:12:44.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.544+0000 7f156a08a740 10 client.26678 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:44.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.544+0000 7f156a08a740 1 -- 172.21.15.2:0/3899043704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a50667c2c0 con 0x55a506662fe0 2024-09-10T08:12:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.544+0000 7f1546000640 1 -- 172.21.15.2:0/3899043704 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1558015070 con 0x55a506662fe0 2024-09-10T08:12:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.544+0000 7f1546000640 1 -- 172.21.15.2:0/3899043704 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f154c0468f0 con 0x55a506662fe0 2024-09-10T08:12:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.544+0000 7f156a08a740 10 client.26678 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.544+0000 7f156a08a740 10 client.26678 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:12:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.544+0000 7f156a08a740 20 client.26678 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.544+0000 7f156a08a740 1 --2- 172.21.15.2:0/3899043704 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a50667d670 0x55a50667faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.545+0000 7f156a08a740 4 client.26678 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:44.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.545+0000 7f156a08a740 1 -- 172.21.15.2:0/3899043704 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5065d4990 con 0x55a50667d670 2024-09-10T08:12:44.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.545+0000 7f1565600640 1 --2- 172.21.15.2:0/3899043704 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a50667d670 0x55a50667faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.545+0000 7f1565600640 1 --2- 172.21.15.2:0/3899043704 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a50667d670 0x55a50667faf0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7eebda0360f704a server_cookie=d3009c54ce30187f in_seq=0 out_seq=0 2024-09-10T08:12:44.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.545+0000 7f1546000640 10 client.26678 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.546+0000 7f1546000640 1 -- 172.21.15.2:0/3899043704 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5065d4990 con 0x55a50667d670 2024-09-10T08:12:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.546+0000 7f1546000640 10 client.26678 handle_command_reply: tid=0 2024-09-10T08:12:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.546+0000 7f156a08a740 1 client.26678 shutdown 2024-09-10T08:12:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.546+0000 7f156a08a740 1 -- 172.21.15.2:0/3899043704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a506662fe0 msgr2=0x55a50666ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.546+0000 7f156a08a740 1 --2- 172.21.15.2:0/3899043704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a506662fe0 0x55a50666ace0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f154c009900 tx=0x7f154c031850 comp rx=0 tx=0).stop 2024-09-10T08:12:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.546+0000 7f156a08a740 1 -- 172.21.15.2:0/3899043704 shutdown_connections 2024-09-10T08:12:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.546+0000 7f156a08a740 1 --2- 172.21.15.2:0/3899043704 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a50667d670 0x55a50667faf0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.546+0000 7f156a08a740 1 --2- 172.21.15.2:0/3899043704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50666d8a0 0x55a50666b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.546+0000 7f156a08a740 1 --2- 172.21.15.2:0/3899043704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a506662fe0 0x55a50666ace0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.546+0000 7f156a08a740 1 --2- 172.21.15.2:0/3899043704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a5066626c0 0x55a50666a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.546+0000 7f1546000640 0 client.26678 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.546+0000 7f156a08a740 1 -- 172.21.15.2:0/3899043704 >> 172.21.15.2:0/3899043704 conn(0x55a50665ec50 msgr2=0x55a506664980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.547+0000 7f156a08a740 1 -- 172.21.15.2:0/3899043704 shutdown_connections 2024-09-10T08:12:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.547+0000 7f156a08a740 1 -- 172.21.15.2:0/3899043704 wait complete. 2024-09-10T08:12:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.547+0000 7f156a08a740 20 client.26678 trim_cache size 0 max 16384 2024-09-10T08:12:44.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.565+0000 7f156a08a740 1 Processor -- start 2024-09-10T08:12:44.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.565+0000 7f156a08a740 1 -- start start 2024-09-10T08:12:44.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.565+0000 7f156a08a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5066626c0 0x55a506665170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.565+0000 7f156a08a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a506662fe0 0x55a50665dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.565+0000 7f156a08a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a50666d8a0 0x55a5066853c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.565+0000 7f156a08a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f15601b7b40 con 0x55a50666d8a0 2024-09-10T08:12:44.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.565+0000 7f156a08a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15601b5410 con 0x55a5066626c0 2024-09-10T08:12:44.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.565+0000 7f156a08a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15601b3260 con 0x55a506662fe0 2024-09-10T08:12:44.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.566+0000 7f1565600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5066626c0 0x55a506665170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.566+0000 7f155fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a506662fe0 0x55a50665dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.566+0000 7f1565600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a50666d8a0 0x55a5066853c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.566+0000 7f155fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a506662fe0 0x55a50665dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:48256/0 (socket says 172.21.15.2:48256) 2024-09-10T08:12:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.566+0000 7f1565600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a50666d8a0 0x55a5066853c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36408/0 (socket says 172.21.15.2:36408) 2024-09-10T08:12:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.566+0000 7f155fe00640 1 -- 172.21.15.2:0/721951374 learned_addr learned my addr 172.21.15.2:0/721951374 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.566+0000 7f155fe00640 1 -- 172.21.15.2:0/721951374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5066626c0 msgr2=0x55a506665170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.566+0000 7f155fe00640 1 --2- 172.21.15.2:0/721951374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5066626c0 0x55a506665170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.566+0000 7f155fe00640 1 -- 172.21.15.2:0/721951374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a50666d8a0 msgr2=0x55a5066853c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.566+0000 7f155fe00640 1 --2- 172.21.15.2:0/721951374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a50666d8a0 0x55a5066853c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.566+0000 7f155fe00640 1 -- 172.21.15.2:0/721951374 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50665fd00 con 0x55a506662fe0 2024-09-10T08:12:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.566+0000 7f1565600640 1 --2- 172.21.15.2:0/721951374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a50666d8a0 0x55a5066853c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:44.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.566+0000 7f155fe00640 1 --2- 172.21.15.2:0/721951374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a506662fe0 0x55a50665dd60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f154c005940 tx=0x7f154c031b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2681e5c19a542e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:44.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.567+0000 7f1551200640 1 -- 172.21.15.2:0/721951374 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c035070 con 0x55a506662fe0 2024-09-10T08:12:44.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.567+0000 7f1551200640 1 -- 172.21.15.2:0/721951374 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154c041070 con 0x55a506662fe0 2024-09-10T08:12:44.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.567+0000 7f1551200640 1 -- 172.21.15.2:0/721951374 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c000e60 con 0x55a506662fe0 2024-09-10T08:12:44.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.567+0000 7f156a08a740 1 -- 172.21.15.2:0/721951374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a506662fe0 msgr2=0x55a50665dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:44.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.567+0000 7f156a08a740 1 --2- 172.21.15.2:0/721951374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a506662fe0 0x55a50665dd60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f154c005940 tx=0x7f154c031b00 comp rx=0 tx=0).stop 2024-09-10T08:12:44.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.568+0000 7f156a08a740 1 -- 172.21.15.2:0/721951374 shutdown_connections 2024-09-10T08:12:44.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.568+0000 7f156a08a740 1 --2- 172.21.15.2:0/721951374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a50666d8a0 0x55a5066853c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.568+0000 7f156a08a740 1 --2- 172.21.15.2:0/721951374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a506662fe0 0x55a50665dd60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.568+0000 7f156a08a740 1 --2- 172.21.15.2:0/721951374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a5066626c0 0x55a506665170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.568+0000 7f156a08a740 1 -- 172.21.15.2:0/721951374 >> 172.21.15.2:0/721951374 conn(0x55a50665ec50 msgr2=0x55a50665d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:44.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.568+0000 7f156a08a740 1 -- 172.21.15.2:0/721951374 shutdown_connections 2024-09-10T08:12:44.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.568+0000 7f156a08a740 1 -- 172.21.15.2:0/721951374 wait complete. 2024-09-10T08:12:44.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.568+0000 7f156a08a740 1 -- start start 2024-09-10T08:12:44.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.568+0000 7f156a08a740 1 Processor -- start 2024-09-10T08:12:44.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.569+0000 7f156a08a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a5066626c0 0x55a506675840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.569+0000 7f156a08a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a506662fe0 0x55a506675d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.569+0000 7f156a08a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a50666d8a0 0x55a506672810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.569+0000 7f156a08a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f15601b38d0 con 0x55a5066626c0 2024-09-10T08:12:44.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.569+0000 7f156a08a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15601b4d40 con 0x55a50666d8a0 2024-09-10T08:12:44.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.569+0000 7f156a08a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15601b3fa0 con 0x55a506662fe0 2024-09-10T08:12:44.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.569+0000 7f156a08a740 10 client.0 _ll_register_callbacks cb 0x7f154c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:44.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.569+0000 7f1565600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a5066626c0 0x55a506675840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.569+0000 7f1565600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a5066626c0 0x55a506675840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36412/0 (socket says 172.21.15.2:36412) 2024-09-10T08:12:44.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.569+0000 7f1565600640 1 -- 172.21.15.2:0/57520358 learned_addr learned my addr 172.21.15.2:0/57520358 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:44.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.569+0000 7f155fe00640 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a506662fe0 0x55a506675d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.569+0000 7f1565600640 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a50666d8a0 0x55a506672810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f155fe00640 1 -- 172.21.15.2:0/57520358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a50666d8a0 msgr2=0x55a506672810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:44.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f155fe00640 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a50666d8a0 0x55a506672810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f155fe00640 1 -- 172.21.15.2:0/57520358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a5066626c0 msgr2=0x55a506675840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f155fe00640 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a5066626c0 0x55a506675840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f155fe00640 1 -- 172.21.15.2:0/57520358 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a506672d50 con 0x55a506662fe0 2024-09-10T08:12:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f1565600640 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a5066626c0 0x55a506675840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f155fe00640 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a506662fe0 0x55a506675d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f154c046d60 tx=0x7f154c0095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb990fde0d610462 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f1546000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f1546000640 1 -- 172.21.15.2:0/57520358 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c060060 con 0x55a506662fe0 2024-09-10T08:12:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f156a08a740 1 -- 172.21.15.2:0/57520358 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f154c041070 con 0x55a506662fe0 2024-09-10T08:12:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f1546000640 1 -- 172.21.15.2:0/57520358 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f154c0055e0 con 0x55a506662fe0 2024-09-10T08:12:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f1546000640 1 -- 172.21.15.2:0/57520358 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f154c035040 con 0x55a506662fe0 2024-09-10T08:12:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f156a08a740 10 client.26644 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.570+0000 7f156a08a740 1 -- 172.21.15.2:0/57520358 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a5066e8a30 con 0x55a506662fe0 2024-09-10T08:12:44.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.571+0000 7f1546000640 1 -- 172.21.15.2:0/57520358 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154c041070 con 0x55a506662fe0 2024-09-10T08:12:44.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.571+0000 7f1546000640 1 -- 172.21.15.2:0/57520358 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f154c046470 con 0x55a506662fe0 2024-09-10T08:12:44.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.571+0000 7f156a08a740 10 client.26644 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:44.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.571+0000 7f156a08a740 10 client.26644 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:12:44.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.571+0000 7f156a08a740 20 client.26644 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:44.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.571+0000 7f156a08a740 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a5066e9030 0x55a5066eb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:44.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.571+0000 7f156a08a740 4 client.26644 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T08:12:44.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.571+0000 7f156a08a740 1 -- 172.21.15.2:0/57520358 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x55a5065cece0 con 0x55a5066e9030 2024-09-10T08:12:44.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.572+0000 7f1565600640 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a5066e9030 0x55a5066eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:44.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.572+0000 7f1565600640 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a5066e9030 0x55a5066eb450 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3659f855c01685e9 server_cookie=52ed08c0440064ce in_seq=0 out_seq=0 2024-09-10T08:12:44.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.572+0000 7f1546000640 10 client.26644 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:44.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.584+0000 7f1546000640 1 -- 172.21.15.2:0/57520358 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a5065cece0 con 0x55a5066e9030 2024-09-10T08:12:44.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f1546000640 10 client.26644 handle_command_reply: tid=0 2024-09-10T08:12:44.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f156a08a740 1 client.26644 shutdown 2024-09-10T08:12:44.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f156a08a740 1 -- 172.21.15.2:0/57520358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a506662fe0 msgr2=0x55a506675d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:44.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f156a08a740 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a506662fe0 0x55a506675d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f154c046d60 tx=0x7f154c0095a0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f156a08a740 1 -- 172.21.15.2:0/57520358 shutdown_connections 2024-09-10T08:12:44.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f156a08a740 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a5066e9030 0x55a5066eb450 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f156a08a740 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a50666d8a0 0x55a506672810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f1546000640 0 client.26644 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:44.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f156a08a740 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a506662fe0 0x55a506675d80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f156a08a740 1 --2- 172.21.15.2:0/57520358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a5066626c0 0x55a506675840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f156a08a740 1 -- 172.21.15.2:0/57520358 >> 172.21.15.2:0/57520358 conn(0x55a50665ec50 msgr2=0x55a50665de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:44.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f156a08a740 1 -- 172.21.15.2:0/57520358 shutdown_connections 2024-09-10T08:12:44.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f156a08a740 1 -- 172.21.15.2:0/57520358 wait complete. 2024-09-10T08:12:44.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.585+0000 7f156a08a740 20 client.26644 trim_cache size 0 max 16384 2024-09-10T08:12:44.587 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:12:44.587 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:12:44.587 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:12:44.587 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 14, 2024-09-10T08:12:44.587 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:12:44.587 INFO:teuthology.orchestra.run.smithi002.stdout: "232ae216": { 2024-09-10T08:12:44.587 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 14, 2024-09-10T08:12:44.587 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:12:44.588 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:12:44.588 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:12:44.588 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:12:44.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:12:44.588 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:12:44.588 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 142.8, 2024-09-10T08:12:44.588 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:12:44.588 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:12:44.588 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:12:44.588 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:12:44.588 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:12:44.588 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:12:44.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:44.589 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:44.589 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:44.589 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:44.589 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:44.589 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:12:44.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.586+0000 7f1566000640 1 -- 172.21.15.2:0/1800513348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f15480a2c00 msgr2=0x7f15480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:44.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.586+0000 7f1566000640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f15480a2c00 0x7f15480a50c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f15540cd230 tx=0x7f1554003c90 comp rx=0 tx=0).stop 2024-09-10T08:12:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.586+0000 7f1566000640 1 -- 172.21.15.2:0/1800513348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560074510 msgr2=0x7f15601019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.586+0000 7f1566000640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560074510 0x7f15601019a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f155400ea90 tx=0x7f155400ef50 comp rx=0 tx=0).stop 2024-09-10T08:12:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.586+0000 7f1566000640 1 -- 172.21.15.2:0/1800513348 shutdown_connections 2024-09-10T08:12:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.586+0000 7f1566000640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f15480a2c00 0x7f15480a50c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.586+0000 7f1566000640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f156010e280 0x7f1560102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.586+0000 7f1566000640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1560074e30 0x7f1560101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.586+0000 7f1566000640 1 --2- 172.21.15.2:0/1800513348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1560074510 0x7f15601019a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.586+0000 7f1566000640 1 -- 172.21.15.2:0/1800513348 >> 172.21.15.2:0/1800513348 conn(0x7f15600fd6e0 msgr2=0x7f15600fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.586+0000 7f1566000640 1 -- 172.21.15.2:0/1800513348 shutdown_connections 2024-09-10T08:12:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:44.586+0000 7f1566000640 1 -- 172.21.15.2:0/1800513348 wait complete. 2024-09-10T08:12:44.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:44 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1523389458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:44.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:44 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1523389458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:44.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:44 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1523389458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:12:45.035 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:12:45.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:45 smithi002 ceph-mon[30563]: pgmap v525: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-10T08:12:45.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:45 smithi155 ceph-mon[35405]: pgmap v525: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-10T08:12:45.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:45 smithi107 ceph-mon[35048]: pgmap v525: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-10T08:12:45.950 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '232ae216', quiesce took 6.4 seconds. Will release after: 16.35 2024-09-10T08:12:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:46.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.614+0000 7f1a44000640 1 -- 172.21.15.2:0/3390867919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a3c102790 msgr2=0x7f1a3c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:46.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.614+0000 7f1a44000640 1 --2- 172.21.15.2:0/3390867919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a3c102790 0x7f1a3c1067e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009900 tx=0x7f1a2802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.614+0000 7f1a44000640 1 -- 172.21.15.2:0/3390867919 shutdown_connections 2024-09-10T08:12:46.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.614+0000 7f1a44000640 1 --2- 172.21.15.2:0/3390867919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a3c106f10 0x7f1a3c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.614+0000 7f1a44000640 1 --2- 172.21.15.2:0/3390867919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a3c102790 0x7f1a3c1067e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.614+0000 7f1a44000640 1 --2- 172.21.15.2:0/3390867919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a3c101de0 0x7f1a3c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.614+0000 7f1a44000640 1 -- 172.21.15.2:0/3390867919 >> 172.21.15.2:0/3390867919 conn(0x7f1a3c0fd8c0 msgr2=0x7f1a3c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:46.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.614+0000 7f1a44000640 1 -- 172.21.15.2:0/3390867919 shutdown_connections 2024-09-10T08:12:46.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.614+0000 7f1a44000640 1 -- 172.21.15.2:0/3390867919 wait complete. 2024-09-10T08:12:46.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.615+0000 7f1a44000640 1 Processor -- start 2024-09-10T08:12:46.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.615+0000 7f1a44000640 1 -- start start 2024-09-10T08:12:46.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a44000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a3c101de0 0x7f1a3c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a44000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a3c102790 0x7f1a3c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a44000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a3c106f10 0x7f1a3c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a44000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c111f70 con 0x7f1a3c102790 2024-09-10T08:12:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a44000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c111df0 con 0x7f1a3c101de0 2024-09-10T08:12:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a44000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1120f0 con 0x7f1a3c106f10 2024-09-10T08:12:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a40c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a3c102790 0x7f1a3c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a41600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a3c101de0 0x7f1a3c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a40c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a3c102790 0x7f1a3c19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36424/0 (socket says 172.21.15.2:36424) 2024-09-10T08:12:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a40c00640 1 -- 172.21.15.2:0/1223307009 learned_addr learned my addr 172.21.15.2:0/1223307009 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a42000640 1 --2- 172.21.15.2:0/1223307009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a3c106f10 0x7f1a3c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a40c00640 1 -- 172.21.15.2:0/1223307009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a3c106f10 msgr2=0x7f1a3c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a40c00640 1 --2- 172.21.15.2:0/1223307009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a3c106f10 0x7f1a3c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a40c00640 1 -- 172.21.15.2:0/1223307009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a3c101de0 msgr2=0x7f1a3c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a40c00640 1 --2- 172.21.15.2:0/1223307009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a3c101de0 0x7f1a3c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.616+0000 7f1a40c00640 1 -- 172.21.15.2:0/1223307009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3c19f5e0 con 0x7f1a3c102790 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.617+0000 7f1a40c00640 1 --2- 172.21.15.2:0/1223307009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a3c102790 0x7f1a3c19ab70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1a28006010 tx=0x7f1a28004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.617+0000 7f1a3a000640 1 -- 172.21.15.2:0/1223307009 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2803f030 con 0x7f1a3c102790 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.617+0000 7f1a3a000640 1 -- 172.21.15.2:0/1223307009 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a28004440 con 0x7f1a3c102790 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.617+0000 7f1a44000640 1 -- 172.21.15.2:0/1223307009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3c19f870 con 0x7f1a3c102790 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.617+0000 7f1a3a000640 1 -- 172.21.15.2:0/1223307009 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28035070 con 0x7f1a3c102790 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.617+0000 7f1a44000640 1 -- 172.21.15.2:0/1223307009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3c1a71b0 con 0x7f1a3c102790 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.620+0000 7f1a3a000640 1 -- 172.21.15.2:0/1223307009 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1a28042030 con 0x7f1a3c102790 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.621+0000 7f1a46a09740 1 Processor -- start 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.621+0000 7f1a46a09740 1 -- start start 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.621+0000 7f1a46a09740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593ac70 0x55c96593b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.621+0000 7f1a46a09740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c96593b590 0x55c965945910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.621+0000 7f1a46a09740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c965945e50 0x55c965948240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.621+0000 7f1a46a09740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c9658a13e0 con 0x55c96593ac70 2024-09-10T08:12:46.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.621+0000 7f1a46a09740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c965848d80 con 0x55c96593b590 2024-09-10T08:12:46.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.621+0000 7f1a46a09740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c96589d3a0 con 0x55c965945e50 2024-09-10T08:12:46.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.621+0000 7f1a42000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593ac70 0x55c96593b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.621+0000 7f1a42000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593ac70 0x55c96593b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36440/0 (socket says 172.21.15.2:36440) 2024-09-10T08:12:46.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.621+0000 7f1a42000640 1 -- 172.21.15.2:0/3077244467 learned_addr learned my addr 172.21.15.2:0/3077244467 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:46.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.622+0000 7f1a41600640 1 --2- 172.21.15.2:0/3077244467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c96593b590 0x55c965945910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.622+0000 7f1a40c00640 1 --2- 172.21.15.2:0/3077244467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c965945e50 0x55c965948240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.622+0000 7f1a42000640 1 -- 172.21.15.2:0/3077244467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c965945e50 msgr2=0x55c965948240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.622+0000 7f1a42000640 1 --2- 172.21.15.2:0/3077244467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c965945e50 0x55c965948240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.622+0000 7f1a42000640 1 -- 172.21.15.2:0/3077244467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c96593b590 msgr2=0x55c965945910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.622+0000 7f1a42000640 1 --2- 172.21.15.2:0/3077244467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c96593b590 0x55c965945910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.622+0000 7f1a42000640 1 -- 172.21.15.2:0/3077244467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c965867ae0 con 0x55c96593ac70 2024-09-10T08:12:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.622+0000 7f1a3a000640 1 --2- 172.21.15.2:0/1223307009 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1a240a2b80 0x7f1a240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.622+0000 7f1a42000640 1 --2- 172.21.15.2:0/3077244467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593ac70 0x55c96593b050 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009f40 tx=0x7f1a34010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fd1dc7d768ad77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.622+0000 7f1a3a000640 1 -- 172.21.15.2:0/1223307009 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1a280ef890 con 0x7f1a3c102790 2024-09-10T08:12:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.623+0000 7f1a2e600640 1 -- 172.21.15.2:0/3077244467 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34003b70 con 0x55c96593ac70 2024-09-10T08:12:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.623+0000 7f1a41600640 1 --2- 172.21.15.2:0/1223307009 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1a240a2b80 0x7f1a240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.623+0000 7f1a2e600640 1 -- 172.21.15.2:0/3077244467 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a34003d10 con 0x55c96593ac70 2024-09-10T08:12:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.623+0000 7f1a2e600640 1 -- 172.21.15.2:0/3077244467 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34011a90 con 0x55c96593ac70 2024-09-10T08:12:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.623+0000 7f1a46a09740 1 -- 172.21.15.2:0/3077244467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593ac70 msgr2=0x55c96593b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.623+0000 7f1a41600640 1 --2- 172.21.15.2:0/1223307009 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1a240a2b80 0x7f1a240a5040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55c965938060 tx=0x7f1a30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.623+0000 7f1a46a09740 1 --2- 172.21.15.2:0/3077244467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593ac70 0x55c96593b050 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009f40 tx=0x7f1a34010860 comp rx=0 tx=0).stop 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.623+0000 7f1a46a09740 1 -- 172.21.15.2:0/3077244467 shutdown_connections 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.623+0000 7f1a46a09740 1 --2- 172.21.15.2:0/3077244467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c965945e50 0x55c965948240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.623+0000 7f1a46a09740 1 --2- 172.21.15.2:0/3077244467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c96593b590 0x55c965945910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.623+0000 7f1a46a09740 1 --2- 172.21.15.2:0/3077244467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593ac70 0x55c96593b050 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.623+0000 7f1a46a09740 1 -- 172.21.15.2:0/3077244467 >> 172.21.15.2:0/3077244467 conn(0x55c965937200 msgr2=0x55c9659375e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.624+0000 7f1a46a09740 1 -- 172.21.15.2:0/3077244467 shutdown_connections 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.624+0000 7f1a46a09740 1 -- 172.21.15.2:0/3077244467 wait complete. 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.624+0000 7f1a46a09740 1 -- start start 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.624+0000 7f1a46a09740 1 Processor -- start 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.625+0000 7f1a46a09740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593ac70 0x55c965942d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.625+0000 7f1a46a09740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593b590 0x55c965943260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.625+0000 7f1a46a09740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c965945e50 0x55c9659437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.625+0000 7f1a46a09740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c965948a20 con 0x55c96593b590 2024-09-10T08:12:46.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.625+0000 7f1a46a09740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c96585c0f0 con 0x55c965945e50 2024-09-10T08:12:46.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.625+0000 7f1a46a09740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c965948ba0 con 0x55c96593ac70 2024-09-10T08:12:46.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.625+0000 7f1a46a09740 10 client.0 _ll_register_callbacks cb 0x55c965843bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:46.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.625+0000 7f1a40c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593b590 0x55c965943260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.625+0000 7f1a40c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593b590 0x55c965943260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36456/0 (socket says 172.21.15.2:36456) 2024-09-10T08:12:46.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.625+0000 7f1a40c00640 1 -- 172.21.15.2:0/1918034541 learned_addr learned my addr 172.21.15.2:0/1918034541 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:46.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.625+0000 7f1a41600640 1 --2- 172.21.15.2:0/1918034541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593ac70 0x55c965942d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.625+0000 7f1a42000640 1 --2- 172.21.15.2:0/1918034541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c965945e50 0x55c9659437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.625+0000 7f1a40c00640 1 -- 172.21.15.2:0/1918034541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593ac70 msgr2=0x55c965942d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:46.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.626+0000 7f1a40c00640 1 --2- 172.21.15.2:0/1918034541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593ac70 0x55c965942d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.626+0000 7f1a40c00640 1 -- 172.21.15.2:0/1918034541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c965945e50 msgr2=0x55c9659437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.626+0000 7f1a40c00640 1 --2- 172.21.15.2:0/1918034541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c965945e50 0x55c9659437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.626+0000 7f1a40c00640 1 -- 172.21.15.2:0/1918034541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9656123c0 con 0x55c96593b590 2024-09-10T08:12:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.626+0000 7f1a40c00640 1 --2- 172.21.15.2:0/1918034541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593b590 0x55c965943260 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1a280c43e0 tx=0x7f1a280b7f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d83d168fc7f636 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.626+0000 7f1a22a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:12:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.626+0000 7f1a22a00640 1 -- 172.21.15.2:0/1918034541 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28049070 con 0x55c96593b590 2024-09-10T08:12:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.626+0000 7f1a46a09740 1 -- 172.21.15.2:0/1918034541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a34003d10 con 0x55c96593b590 2024-09-10T08:12:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.626+0000 7f1a22a00640 1 -- 172.21.15.2:0/1918034541 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1a28003cd0 con 0x55c96593b590 2024-09-10T08:12:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.626+0000 7f1a22a00640 1 -- 172.21.15.2:0/1918034541 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1a280bb600 con 0x55c96593b590 2024-09-10T08:12:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.626+0000 7f1a46a09740 10 client.16659 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.626+0000 7f1a46a09740 1 -- 172.21.15.2:0/1918034541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c965954a40 con 0x55c96593b590 2024-09-10T08:12:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.627+0000 7f1a22a00640 1 -- 172.21.15.2:0/1918034541 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a34003d10 con 0x55c96593b590 2024-09-10T08:12:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.627+0000 7f1a22a00640 1 -- 172.21.15.2:0/1918034541 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1a280b8310 con 0x55c96593b590 2024-09-10T08:12:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.627+0000 7f1a46a09740 10 client.16659 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.627+0000 7f1a46a09740 10 client.16659 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:12:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.627+0000 7f1a46a09740 20 client.16659 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.627+0000 7f1a46a09740 1 --2- 172.21.15.2:0/1918034541 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c965955e50 0x55c9659582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.627+0000 7f1a46a09740 4 client.16659 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.627+0000 7f1a46a09740 1 -- 172.21.15.2:0/1918034541 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9658ae950 con 0x55c965955e50 2024-09-10T08:12:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.628+0000 7f1a41600640 1 --2- 172.21.15.2:0/1918034541 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c965955e50 0x55c9659582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.628+0000 7f1a41600640 1 --2- 172.21.15.2:0/1918034541 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c965955e50 0x55c9659582b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1e4f7e7970ee9ad server_cookie=15ab6539f85848a5 in_seq=0 out_seq=0 2024-09-10T08:12:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.628+0000 7f1a22a00640 10 client.16659 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:12:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a22a00640 1 -- 172.21.15.2:0/1918034541 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9658ae950 con 0x55c965955e50 2024-09-10T08:12:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a22a00640 10 client.16659 handle_command_reply: tid=0 2024-09-10T08:12:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a46a09740 1 client.16659 shutdown 2024-09-10T08:12:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a46a09740 1 -- 172.21.15.2:0/1918034541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593b590 msgr2=0x55c965943260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a46a09740 1 --2- 172.21.15.2:0/1918034541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593b590 0x55c965943260 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1a280c43e0 tx=0x7f1a280b7f40 comp rx=0 tx=0).stop 2024-09-10T08:12:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a46a09740 1 -- 172.21.15.2:0/1918034541 shutdown_connections 2024-09-10T08:12:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a46a09740 1 --2- 172.21.15.2:0/1918034541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c965945e50 0x55c9659437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a46a09740 1 --2- 172.21.15.2:0/1918034541 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c965955e50 0x55c9659582b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a46a09740 1 --2- 172.21.15.2:0/1918034541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593b590 0x55c965943260 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a46a09740 1 --2- 172.21.15.2:0/1918034541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593ac70 0x55c965942d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a46a09740 1 -- 172.21.15.2:0/1918034541 >> 172.21.15.2:0/1918034541 conn(0x55c965937200 msgr2=0x55c96593cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a22a00640 0 client.16659 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:12:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a46a09740 1 -- 172.21.15.2:0/1918034541 shutdown_connections 2024-09-10T08:12:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a46a09740 1 -- 172.21.15.2:0/1918034541 wait complete. 2024-09-10T08:12:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.629+0000 7f1a46a09740 20 client.16659 trim_cache size 0 max 16384 2024-09-10T08:12:46.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.644+0000 7f1a46a09740 1 Processor -- start 2024-09-10T08:12:46.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.644+0000 7f1a46a09740 1 -- start start 2024-09-10T08:12:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.644+0000 7f1a46a09740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c965937200 0x55c96593d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.644+0000 7f1a46a09740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c96593ac70 0x55c96593e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.644+0000 7f1a46a09740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593b590 0x55c96595db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.644+0000 7f1a46a09740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1b38b0 con 0x55c965937200 2024-09-10T08:12:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.644+0000 7f1a46a09740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1b1180 con 0x55c96593ac70 2024-09-10T08:12:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.644+0000 7f1a46a09740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1aefd0 con 0x55c96593b590 2024-09-10T08:12:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.645+0000 7f1a41600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c965937200 0x55c96593d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.645+0000 7f1a41600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c965937200 0x55c96593d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36468/0 (socket says 172.21.15.2:36468) 2024-09-10T08:12:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.645+0000 7f1a41600640 1 -- 172.21.15.2:0/2146254940 learned_addr learned my addr 172.21.15.2:0/2146254940 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.645+0000 7f1a40c00640 1 --2- 172.21.15.2:0/2146254940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c96593ac70 0x55c96593e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.645+0000 7f1a42000640 1 --2- 172.21.15.2:0/2146254940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593b590 0x55c96595db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.645+0000 7f1a41600640 1 -- 172.21.15.2:0/2146254940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593b590 msgr2=0x55c96595db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:46.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.645+0000 7f1a41600640 1 --2- 172.21.15.2:0/2146254940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593b590 0x55c96595db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.645+0000 7f1a41600640 1 -- 172.21.15.2:0/2146254940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c96593ac70 msgr2=0x55c96593e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:46.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.645+0000 7f1a41600640 1 --2- 172.21.15.2:0/2146254940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c96593ac70 0x55c96593e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.645+0000 7f1a41600640 1 -- 172.21.15.2:0/2146254940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96593b3c0 con 0x55c965937200 2024-09-10T08:12:46.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.645+0000 7f1a40c00640 1 --2- 172.21.15.2:0/2146254940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c96593ac70 0x55c96593e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:46.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.645+0000 7f1a41600640 1 --2- 172.21.15.2:0/2146254940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c965937200 0x55c96593d720 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1a30012cb0 tx=0x7f1a3001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6868d8c7d578eb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:46.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.646+0000 7f1a2dc00640 1 -- 172.21.15.2:0/2146254940 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3001fe40 con 0x55c965937200 2024-09-10T08:12:46.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.646+0000 7f1a2dc00640 1 -- 172.21.15.2:0/2146254940 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a30005500 con 0x55c965937200 2024-09-10T08:12:46.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.646+0000 7f1a2dc00640 1 -- 172.21.15.2:0/2146254940 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3002b640 con 0x55c965937200 2024-09-10T08:12:46.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.646+0000 7f1a46a09740 1 -- 172.21.15.2:0/2146254940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c965937200 msgr2=0x55c96593d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:46.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.646+0000 7f1a46a09740 1 --2- 172.21.15.2:0/2146254940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c965937200 0x55c96593d720 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1a30012cb0 tx=0x7f1a3001b000 comp rx=0 tx=0).stop 2024-09-10T08:12:46.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.646+0000 7f1a46a09740 1 -- 172.21.15.2:0/2146254940 shutdown_connections 2024-09-10T08:12:46.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.646+0000 7f1a46a09740 1 --2- 172.21.15.2:0/2146254940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593b590 0x55c96595db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.646+0000 7f1a46a09740 1 --2- 172.21.15.2:0/2146254940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c96593ac70 0x55c96593e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.646+0000 7f1a46a09740 1 --2- 172.21.15.2:0/2146254940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c965937200 0x55c96593d720 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.646+0000 7f1a46a09740 1 -- 172.21.15.2:0/2146254940 >> 172.21.15.2:0/2146254940 conn(0x55c96593e710 msgr2=0x55c965944da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:46.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.646+0000 7f1a46a09740 1 -- 172.21.15.2:0/2146254940 shutdown_connections 2024-09-10T08:12:46.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.647+0000 7f1a46a09740 1 -- 172.21.15.2:0/2146254940 wait complete. 2024-09-10T08:12:46.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.647+0000 7f1a46a09740 1 -- start start 2024-09-10T08:12:46.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.647+0000 7f1a46a09740 1 Processor -- start 2024-09-10T08:12:46.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a46a09740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c965937200 0x55c96594e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a46a09740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593ac70 0x55c9659499b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a46a09740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593b590 0x55c965949fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a46a09740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1af640 con 0x55c96593b590 2024-09-10T08:12:46.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a46a09740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1b0ab0 con 0x55c965937200 2024-09-10T08:12:46.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a46a09740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1afd10 con 0x55c96593ac70 2024-09-10T08:12:46.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a46a09740 10 client.0 _ll_register_callbacks cb 0x7f1a280b7820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:46.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a41600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c965937200 0x55c96594e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a40c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593ac70 0x55c9659499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a40c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593ac70 0x55c9659499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:48326/0 (socket says 172.21.15.2:48326) 2024-09-10T08:12:46.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a40c00640 1 -- 172.21.15.2:0/2495686209 learned_addr learned my addr 172.21.15.2:0/2495686209 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:46.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a41600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c965937200 0x55c96594e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50784/0 (socket says 172.21.15.2:50784) 2024-09-10T08:12:46.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a42000640 1 --2- 172.21.15.2:0/2495686209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593b590 0x55c965949fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a40c00640 1 -- 172.21.15.2:0/2495686209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c965937200 msgr2=0x55c96594e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a40c00640 1 --2- 172.21.15.2:0/2495686209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c965937200 0x55c96594e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a40c00640 1 -- 172.21.15.2:0/2495686209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593b590 msgr2=0x55c965949fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a40c00640 1 --2- 172.21.15.2:0/2495686209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593b590 0x55c965949fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a40c00640 1 -- 172.21.15.2:0/2495686209 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96594a7f0 con 0x55c96593ac70 2024-09-10T08:12:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.648+0000 7f1a41600640 1 --2- 172.21.15.2:0/2495686209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c965937200 0x55c96594e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.649+0000 7f1a40c00640 1 --2- 172.21.15.2:0/2495686209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593ac70 0x55c9659499b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a280bb9a0 tx=0x7f1a280c4780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cace533a343ea0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.649+0000 7f1a22a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.649+0000 7f1a22a00640 1 -- 172.21.15.2:0/2495686209 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a280d9070 con 0x55c96593ac70 2024-09-10T08:12:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.649+0000 7f1a46a09740 1 -- 172.21.15.2:0/2495686209 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a30005500 con 0x55c96593ac70 2024-09-10T08:12:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.649+0000 7f1a22a00640 1 -- 172.21.15.2:0/2495686209 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1a280bb0f0 con 0x55c96593ac70 2024-09-10T08:12:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.649+0000 7f1a22a00640 1 -- 172.21.15.2:0/2495686209 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1a28049040 con 0x55c96593ac70 2024-09-10T08:12:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.649+0000 7f1a46a09740 10 client.26662 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.649+0000 7f1a46a09740 1 -- 172.21.15.2:0/2495686209 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9659c0c70 con 0x55c96593ac70 2024-09-10T08:12:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.649+0000 7f1a22a00640 1 -- 172.21.15.2:0/2495686209 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a30005500 con 0x55c96593ac70 2024-09-10T08:12:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.650+0000 7f1a22a00640 1 -- 172.21.15.2:0/2495686209 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1a280c4b60 con 0x55c96593ac70 2024-09-10T08:12:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.650+0000 7f1a46a09740 10 client.26662 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.650+0000 7f1a46a09740 10 client.26662 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:12:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.650+0000 7f1a46a09740 20 client.26662 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.650+0000 7f1a46a09740 1 --2- 172.21.15.2:0/2495686209 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c965945e50 0x55c9659c3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.650+0000 7f1a46a09740 4 client.26662 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:12:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.650+0000 7f1a46a09740 1 -- 172.21.15.2:0/2495686209 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9658a8e60 con 0x55c965945e50 2024-09-10T08:12:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.650+0000 7f1a41600640 1 --2- 172.21.15.2:0/2495686209 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c965945e50 0x55c9659c3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a41600640 1 --2- 172.21.15.2:0/2495686209 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c965945e50 0x55c9659c3260 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a992b5571736ab47 server_cookie=237d40cf5a5f5089 in_seq=0 out_seq=0 2024-09-10T08:12:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a22a00640 10 client.26662 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:12:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a22a00640 1 -- 172.21.15.2:0/2495686209 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9658a8e60 con 0x55c965945e50 2024-09-10T08:12:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a22a00640 10 client.26662 handle_command_reply: tid=0 2024-09-10T08:12:46.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a46a09740 1 client.26662 shutdown 2024-09-10T08:12:46.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a46a09740 1 -- 172.21.15.2:0/2495686209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593ac70 msgr2=0x55c9659499b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:46.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a46a09740 1 --2- 172.21.15.2:0/2495686209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593ac70 0x55c9659499b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a280bb9a0 tx=0x7f1a280c4780 comp rx=0 tx=0).stop 2024-09-10T08:12:46.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a46a09740 1 -- 172.21.15.2:0/2495686209 shutdown_connections 2024-09-10T08:12:46.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a46a09740 1 --2- 172.21.15.2:0/2495686209 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55c965945e50 0x55c9659c3260 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a46a09740 1 --2- 172.21.15.2:0/2495686209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c96593b590 0x55c965949fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a46a09740 1 --2- 172.21.15.2:0/2495686209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c96593ac70 0x55c9659499b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a46a09740 1 --2- 172.21.15.2:0/2495686209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c965937200 0x55c96594e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a46a09740 1 -- 172.21.15.2:0/2495686209 >> 172.21.15.2:0/2495686209 conn(0x55c96593e710 msgr2=0x55c965942ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:46.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a22a00640 0 client.26662 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:12:46.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.651+0000 7f1a46a09740 1 -- 172.21.15.2:0/2495686209 shutdown_connections 2024-09-10T08:12:46.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.652+0000 7f1a46a09740 1 -- 172.21.15.2:0/2495686209 wait complete. 2024-09-10T08:12:46.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.652+0000 7f1a46a09740 20 client.26662 trim_cache size 0 max 16384 2024-09-10T08:12:46.657 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:12:46.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.652+0000 7f1a44000640 1 -- 172.21.15.2:0/1223307009 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1a240a2b80 msgr2=0x7f1a240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:46.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.652+0000 7f1a44000640 1 --2- 172.21.15.2:0/1223307009 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1a240a2b80 0x7f1a240a5040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55c965938060 tx=0x7f1a30005f90 comp rx=0 tx=0).stop 2024-09-10T08:12:46.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.652+0000 7f1a44000640 1 -- 172.21.15.2:0/1223307009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a3c102790 msgr2=0x7f1a3c19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:46.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.652+0000 7f1a44000640 1 --2- 172.21.15.2:0/1223307009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a3c102790 0x7f1a3c19ab70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1a28006010 tx=0x7f1a28004040 comp rx=0 tx=0).stop 2024-09-10T08:12:46.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.652+0000 7f1a44000640 1 -- 172.21.15.2:0/1223307009 shutdown_connections 2024-09-10T08:12:46.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.652+0000 7f1a44000640 1 --2- 172.21.15.2:0/1223307009 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1a240a2b80 0x7f1a240a5040 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.652+0000 7f1a44000640 1 --2- 172.21.15.2:0/1223307009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a3c106f10 0x7f1a3c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.652+0000 7f1a44000640 1 --2- 172.21.15.2:0/1223307009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a3c102790 0x7f1a3c19ab70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.652+0000 7f1a44000640 1 --2- 172.21.15.2:0/1223307009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a3c101de0 0x7f1a3c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:46.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.652+0000 7f1a44000640 1 -- 172.21.15.2:0/1223307009 >> 172.21.15.2:0/1223307009 conn(0x7f1a3c0fd8c0 msgr2=0x7f1a3c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:46.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.653+0000 7f1a44000640 1 -- 172.21.15.2:0/1223307009 shutdown_connections 2024-09-10T08:12:46.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:46.653+0000 7f1a44000640 1 -- 172.21.15.2:0/1223307009 wait complete. 2024-09-10T08:12:47.611 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:12:47.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:47 smithi155 ceph-mon[35405]: pgmap v526: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 12 op/s 2024-09-10T08:12:47.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:47 smithi107 ceph-mon[35048]: pgmap v526: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 12 op/s 2024-09-10T08:12:47.990 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:48.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:47 smithi002 ceph-mon[30563]: pgmap v526: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 12 op/s 2024-09-10T08:12:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.581+0000 7f53f4200640 1 -- 172.21.15.2:0/1180640314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53ec102790 msgr2=0x7f53ec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.581+0000 7f53f4200640 1 --2- 172.21.15.2:0/1180640314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53ec102790 0x7f53ec1067e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009900 tx=0x7f53d802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.581+0000 7f53f4200640 1 -- 172.21.15.2:0/1180640314 shutdown_connections 2024-09-10T08:12:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.581+0000 7f53f4200640 1 --2- 172.21.15.2:0/1180640314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53ec106f10 0x7f53ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.581+0000 7f53f4200640 1 --2- 172.21.15.2:0/1180640314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53ec102790 0x7f53ec1067e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.581+0000 7f53f4200640 1 --2- 172.21.15.2:0/1180640314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53ec101de0 0x7f53ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.581+0000 7f53f4200640 1 -- 172.21.15.2:0/1180640314 >> 172.21.15.2:0/1180640314 conn(0x7f53ec0fd8c0 msgr2=0x7f53ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.581+0000 7f53f4200640 1 -- 172.21.15.2:0/1180640314 shutdown_connections 2024-09-10T08:12:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.582+0000 7f53f4200640 1 -- 172.21.15.2:0/1180640314 wait complete. 2024-09-10T08:12:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.582+0000 7f53f4200640 1 Processor -- start 2024-09-10T08:12:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.583+0000 7f53f4200640 1 -- start start 2024-09-10T08:12:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.583+0000 7f53f4200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53ec101de0 0x7f53ec19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.583+0000 7f53f4200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53ec102790 0x7f53ec19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.583+0000 7f53f4200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53ec106f10 0x7f53ec19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.583+0000 7f53f4200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec111fc0 con 0x7f53ec106f10 2024-09-10T08:12:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.583+0000 7f53f4200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec111e40 con 0x7f53ec101de0 2024-09-10T08:12:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.583+0000 7f53f4200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec112140 con 0x7f53ec102790 2024-09-10T08:12:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.583+0000 7f53f2200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53ec106f10 0x7f53ec19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.583+0000 7f53f1800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53ec101de0 0x7f53ec19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.583+0000 7f53f0e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53ec102790 0x7f53ec19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.583+0000 7f53f2200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53ec106f10 0x7f53ec19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34496/0 (socket says 172.21.15.2:34496) 2024-09-10T08:12:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.583+0000 7f53f2200640 1 -- 172.21.15.2:0/2363971223 learned_addr learned my addr 172.21.15.2:0/2363971223 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.584+0000 7f53f1800640 1 -- 172.21.15.2:0/2363971223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53ec102790 msgr2=0x7f53ec19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.584+0000 7f53f1800640 1 --2- 172.21.15.2:0/2363971223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53ec102790 0x7f53ec19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.584+0000 7f53f1800640 1 -- 172.21.15.2:0/2363971223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53ec106f10 msgr2=0x7f53ec19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.584+0000 7f53f1800640 1 --2- 172.21.15.2:0/2363971223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53ec106f10 0x7f53ec19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.584+0000 7f53f1800640 1 -- 172.21.15.2:0/2363971223 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec19f680 con 0x7f53ec101de0 2024-09-10T08:12:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.584+0000 7f53f2200640 1 --2- 172.21.15.2:0/2363971223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53ec106f10 0x7f53ec19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.584+0000 7f53f1800640 1 --2- 172.21.15.2:0/2363971223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53ec101de0 0x7f53ec19a6d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f53e000e9f0 tx=0x7f53e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.584+0000 7f53ea000640 1 -- 172.21.15.2:0/2363971223 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e000ccc0 con 0x7f53ec101de0 2024-09-10T08:12:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.584+0000 7f53ea000640 1 -- 172.21.15.2:0/2363971223 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e0005020 con 0x7f53ec101de0 2024-09-10T08:12:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.584+0000 7f53ea000640 1 -- 172.21.15.2:0/2363971223 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00049e0 con 0x7f53ec101de0 2024-09-10T08:12:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.584+0000 7f53f4200640 1 -- 172.21.15.2:0/2363971223 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ec19f970 con 0x7f53ec101de0 2024-09-10T08:12:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.584+0000 7f53f4200640 1 -- 172.21.15.2:0/2363971223 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ec1a7250 con 0x7f53ec101de0 2024-09-10T08:12:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.587+0000 7f53f6cbb740 1 Processor -- start 2024-09-10T08:12:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.587+0000 7f53ea000640 1 -- 172.21.15.2:0/2363971223 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f53e0005560 con 0x7f53ec101de0 2024-09-10T08:12:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.588+0000 7f53f6cbb740 1 -- start start 2024-09-10T08:12:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.588+0000 7f53f6cbb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06ecc3c70 0x55b06ecc4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.588+0000 7f53f6cbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc4590 0x55b06ecce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.588+0000 7f53f6cbb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06eccee50 0x55b06ecd1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.588+0000 7f53f6cbb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b06ec2a3e0 con 0x55b06ecc3c70 2024-09-10T08:12:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.588+0000 7f53f6cbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b06ebd1d80 con 0x55b06ecc4590 2024-09-10T08:12:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.588+0000 7f53f6cbb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b06ec263a0 con 0x55b06eccee50 2024-09-10T08:12:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.588+0000 7f53f2200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06ecc3c70 0x55b06ecc4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.588+0000 7f53f2200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06ecc3c70 0x55b06ecc4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34498/0 (socket says 172.21.15.2:34498) 2024-09-10T08:12:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.588+0000 7f53f2200640 1 -- 172.21.15.2:0/4124889976 learned_addr learned my addr 172.21.15.2:0/4124889976 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.588+0000 7f53f0e00640 1 --2- 172.21.15.2:0/4124889976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06eccee50 0x55b06ecd1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.588+0000 7f53f1800640 1 --2- 172.21.15.2:0/4124889976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc4590 0x55b06ecce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.589+0000 7f53f2200640 1 -- 172.21.15.2:0/4124889976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06eccee50 msgr2=0x55b06ecd1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.589+0000 7f53f2200640 1 --2- 172.21.15.2:0/4124889976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06eccee50 0x55b06ecd1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.589+0000 7f53f2200640 1 -- 172.21.15.2:0/4124889976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc4590 msgr2=0x55b06ecce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.589+0000 7f53f2200640 1 --2- 172.21.15.2:0/4124889976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc4590 0x55b06ecce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.589+0000 7f53f2200640 1 -- 172.21.15.2:0/4124889976 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b06ebf0ae0 con 0x55b06ecc3c70 2024-09-10T08:12:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.589+0000 7f53f0e00640 1 --2- 172.21.15.2:0/4124889976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06eccee50 0x55b06ecd1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.589+0000 7f53f2200640 1 --2- 172.21.15.2:0/4124889976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06ecc3c70 0x55b06ecc4050 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f53e4010810 tx=0x7f53e4010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d2a911896c29d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.589+0000 7f53ea000640 1 --2- 172.21.15.2:0/2363971223 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f53d40a2b80 0x7f53d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.589+0000 7f53dea00640 1 -- 172.21.15.2:0/4124889976 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e400eb30 con 0x55b06ecc3c70 2024-09-10T08:12:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.590+0000 7f53dea00640 1 -- 172.21.15.2:0/4124889976 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e4015070 con 0x55b06ecc3c70 2024-09-10T08:12:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.590+0000 7f53dea00640 1 -- 172.21.15.2:0/4124889976 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e40116a0 con 0x55b06ecc3c70 2024-09-10T08:12:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.590+0000 7f53ea000640 1 -- 172.21.15.2:0/2363971223 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f53e00cf3b0 con 0x7f53ec101de0 2024-09-10T08:12:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.590+0000 7f53f1800640 1 --2- 172.21.15.2:0/2363971223 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f53d40a2b80 0x7f53d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.590+0000 7f53f6cbb740 1 -- 172.21.15.2:0/4124889976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06ecc3c70 msgr2=0x55b06ecc4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.590+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/4124889976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06ecc3c70 0x55b06ecc4050 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f53e4010810 tx=0x7f53e4010cd0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.590+0000 7f53f1800640 1 --2- 172.21.15.2:0/2363971223 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f53d40a2b80 0x7f53d40a5040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55b06ecc1060 tx=0x7f53e00d2e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.590+0000 7f53f6cbb740 1 -- 172.21.15.2:0/4124889976 shutdown_connections 2024-09-10T08:12:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.590+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/4124889976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06eccee50 0x55b06ecd1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.590+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/4124889976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc4590 0x55b06ecce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.590+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/4124889976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06ecc3c70 0x55b06ecc4050 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.590+0000 7f53f6cbb740 1 -- 172.21.15.2:0/4124889976 >> 172.21.15.2:0/4124889976 conn(0x55b06ecc0200 msgr2=0x55b06ecc05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.590+0000 7f53f6cbb740 1 -- 172.21.15.2:0/4124889976 shutdown_connections 2024-09-10T08:12:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.591+0000 7f53f6cbb740 1 -- 172.21.15.2:0/4124889976 wait complete. 2024-09-10T08:12:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.591+0000 7f53f6cbb740 1 -- start start 2024-09-10T08:12:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.591+0000 7f53f6cbb740 1 Processor -- start 2024-09-10T08:12:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.592+0000 7f53f6cbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc3c70 0x55b06eccbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.592+0000 7f53f6cbb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06ecc4590 0x55b06eccc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.592+0000 7f53f6cbb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06eccee50 0x55b06eccc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.592+0000 7f53f6cbb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b06ecd1a20 con 0x55b06eccee50 2024-09-10T08:12:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.592+0000 7f53f6cbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b06ebe50f0 con 0x55b06ecc3c70 2024-09-10T08:12:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.592+0000 7f53f6cbb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b06ecd1ba0 con 0x55b06ecc4590 2024-09-10T08:12:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.592+0000 7f53f6cbb740 10 client.0 _ll_register_callbacks cb 0x55b06ebccbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.592+0000 7f53f2200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06eccee50 0x55b06eccc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.592+0000 7f53f2200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06eccee50 0x55b06eccc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34502/0 (socket says 172.21.15.2:34502) 2024-09-10T08:12:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.592+0000 7f53f2200640 1 -- 172.21.15.2:0/2673311604 learned_addr learned my addr 172.21.15.2:0/2673311604 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.592+0000 7f53f2200640 1 --2- 172.21.15.2:0/2673311604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc3c70 0x55b06eccbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.592+0000 7f53f0e00640 1 --2- 172.21.15.2:0/2673311604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06ecc4590 0x55b06eccc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.593+0000 7f53f2200640 1 -- 172.21.15.2:0/2673311604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06ecc4590 msgr2=0x55b06eccc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.593+0000 7f53f2200640 1 --2- 172.21.15.2:0/2673311604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06ecc4590 0x55b06eccc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.593+0000 7f53f2200640 1 -- 172.21.15.2:0/2673311604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc3c70 msgr2=0x55b06eccbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.593+0000 7f53f2200640 1 --2- 172.21.15.2:0/2673311604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc3c70 0x55b06eccbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.593+0000 7f53f2200640 1 -- 172.21.15.2:0/2673311604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b06e99b3c0 con 0x55b06eccee50 2024-09-10T08:12:48.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.593+0000 7f53f2200640 1 --2- 172.21.15.2:0/2673311604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06eccee50 0x55b06eccc7b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f53e400ebf0 tx=0x7f53e401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfca058bd1704cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:48.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.593+0000 7f53d3400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:12:48.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.593+0000 7f53d3400640 1 -- 172.21.15.2:0/2673311604 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4004c40 con 0x55b06eccee50 2024-09-10T08:12:48.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.593+0000 7f53d3400640 1 -- 172.21.15.2:0/2673311604 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f53e4004de0 con 0x55b06eccee50 2024-09-10T08:12:48.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.593+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2673311604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e4015070 con 0x55b06eccee50 2024-09-10T08:12:48.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.593+0000 7f53d3400640 1 -- 172.21.15.2:0/2673311604 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53e4030050 con 0x55b06eccee50 2024-09-10T08:12:48.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.593+0000 7f53f6cbb740 10 client.16677 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:48.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.593+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2673311604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b06ecdda60 con 0x55b06eccee50 2024-09-10T08:12:48.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.594+0000 7f53d3400640 1 -- 172.21.15.2:0/2673311604 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e4015070 con 0x55b06eccee50 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.594+0000 7f53d3400640 1 -- 172.21.15.2:0/2673311604 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f53e4011af0 con 0x55b06eccee50 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.594+0000 7f53f6cbb740 10 client.16677 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.594+0000 7f53f6cbb740 10 client.16677 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.594+0000 7f53f6cbb740 20 client.16677 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.594+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2673311604 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b06ecdee70 0x55b06ece1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.594+0000 7f53f6cbb740 4 client.16677 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.594+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2673311604 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b06ec37950 con 0x55b06ecdee70 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.595+0000 7f53f0e00640 1 --2- 172.21.15.2:0/2673311604 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b06ecdee70 0x55b06ece1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.596+0000 7f53f0e00640 1 --2- 172.21.15.2:0/2673311604 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b06ecdee70 0x55b06ece1260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd4eb1de1dc76741 server_cookie=4808a7a3404a2e9a in_seq=0 out_seq=0 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.596+0000 7f53d3400640 10 client.16677 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.596+0000 7f53d3400640 1 -- 172.21.15.2:0/2673311604 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b06ec37950 con 0x55b06ecdee70 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.596+0000 7f53d3400640 10 client.16677 handle_command_reply: tid=0 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.597+0000 7f53f6cbb740 1 client.16677 shutdown 2024-09-10T08:12:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.597+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2673311604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06eccee50 msgr2=0x55b06eccc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.597+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2673311604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06eccee50 0x55b06eccc7b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f53e400ebf0 tx=0x7f53e401ace0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.597+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2673311604 shutdown_connections 2024-09-10T08:12:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.597+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2673311604 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b06ecdee70 0x55b06ece1260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.597+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2673311604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06eccee50 0x55b06eccc7b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.597+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2673311604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06ecc4590 0x55b06eccc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.597+0000 7f53d3400640 0 client.16677 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:12:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.597+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2673311604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc3c70 0x55b06eccbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.597+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2673311604 >> 172.21.15.2:0/2673311604 conn(0x55b06ecc0200 msgr2=0x55b06ecc5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.597+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2673311604 shutdown_connections 2024-09-10T08:12:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.597+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2673311604 wait complete. 2024-09-10T08:12:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.597+0000 7f53f6cbb740 20 client.16677 trim_cache size 0 max 16384 2024-09-10T08:12:48.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.611+0000 7f53f6cbb740 1 Processor -- start 2024-09-10T08:12:48.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.611+0000 7f53f6cbb740 1 -- start start 2024-09-10T08:12:48.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f6cbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc3c70 0x55b06ecc66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f6cbb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06ecc4590 0x55b06eccb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f6cbb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06eccee50 0x55b06ece4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f6cbb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1b3980 con 0x55b06eccee50 2024-09-10T08:12:48.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f6cbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1b1250 con 0x55b06ecc3c70 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f6cbb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1af0a0 con 0x55b06ecc4590 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f2200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06eccee50 0x55b06ece4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f2200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc3c70 0x55b06ecc66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f2200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc3c70 0x55b06ecc66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:45842/0 (socket says 172.21.15.2:45842) 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f2200640 1 -- 172.21.15.2:0/2865836568 learned_addr learned my addr 172.21.15.2:0/2865836568 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f0e00640 1 --2- 172.21.15.2:0/2865836568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06ecc4590 0x55b06eccb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f2200640 1 -- 172.21.15.2:0/2865836568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06ecc4590 msgr2=0x55b06eccb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f2200640 1 --2- 172.21.15.2:0/2865836568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06ecc4590 0x55b06eccb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f2200640 1 -- 172.21.15.2:0/2865836568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06eccee50 msgr2=0x55b06ece4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f2200640 1 --2- 172.21.15.2:0/2865836568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06eccee50 0x55b06ece4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.612+0000 7f53f2200640 1 -- 172.21.15.2:0/2865836568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b06ecbb770 con 0x55b06ecc3c70 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.613+0000 7f53f2200640 1 --2- 172.21.15.2:0/2865836568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc3c70 0x55b06ecc66d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f53e401acb0 tx=0x7f53e4000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0e0eb9b39983505 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.613+0000 7f53de000640 1 -- 172.21.15.2:0/2865836568 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4005300 con 0x55b06ecc3c70 2024-09-10T08:12:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.613+0000 7f53de000640 1 -- 172.21.15.2:0/2865836568 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e4015070 con 0x55b06ecc3c70 2024-09-10T08:12:48.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.613+0000 7f53de000640 1 -- 172.21.15.2:0/2865836568 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e401a6d0 con 0x55b06ecc3c70 2024-09-10T08:12:48.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.614+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2865836568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc3c70 msgr2=0x55b06ecc66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:48.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.614+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2865836568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc3c70 0x55b06ecc66d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f53e401acb0 tx=0x7f53e4000f40 comp rx=0 tx=0).stop 2024-09-10T08:12:48.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.614+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2865836568 shutdown_connections 2024-09-10T08:12:48.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.614+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2865836568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06eccee50 0x55b06ece4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.614+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2865836568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06ecc4590 0x55b06eccb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.614+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2865836568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc3c70 0x55b06ecc66d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.614+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2865836568 >> 172.21.15.2:0/2865836568 conn(0x55b06ecc0200 msgr2=0x55b06ecda610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:48.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.614+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2865836568 shutdown_connections 2024-09-10T08:12:48.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.614+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2865836568 wait complete. 2024-09-10T08:12:48.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.614+0000 7f53f6cbb740 1 -- start start 2024-09-10T08:12:48.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.614+0000 7f53f6cbb740 1 Processor -- start 2024-09-10T08:12:48.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.615+0000 7f53f6cbb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06ecc3c70 0x55b06ece46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.615+0000 7f53f6cbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc4590 0x55b06ece1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.615+0000 7f53f6cbb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06eccee50 0x55b06ece1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.615+0000 7f53f6cbb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1af710 con 0x55b06ecc3c70 2024-09-10T08:12:48.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.615+0000 7f53f6cbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1b0b80 con 0x55b06ecc4590 2024-09-10T08:12:48.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.615+0000 7f53f6cbb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1afde0 con 0x55b06eccee50 2024-09-10T08:12:48.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.615+0000 7f53f6cbb740 10 client.0 _ll_register_callbacks cb 0x7f53e00d2d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:48.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.615+0000 7f53f2200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06ecc3c70 0x55b06ece46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.615+0000 7f53f2200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06ecc3c70 0x55b06ece46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34514/0 (socket says 172.21.15.2:34514) 2024-09-10T08:12:48.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.615+0000 7f53f2200640 1 -- 172.21.15.2:0/2790712906 learned_addr learned my addr 172.21.15.2:0/2790712906 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:48.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.615+0000 7f53f0e00640 1 --2- 172.21.15.2:0/2790712906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc4590 0x55b06ece1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.615+0000 7f53f2200640 1 --2- 172.21.15.2:0/2790712906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06eccee50 0x55b06ece1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.616+0000 7f53f0e00640 1 -- 172.21.15.2:0/2790712906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06eccee50 msgr2=0x55b06ece1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:48.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.616+0000 7f53f0e00640 1 --2- 172.21.15.2:0/2790712906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06eccee50 0x55b06ece1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.616+0000 7f53f0e00640 1 -- 172.21.15.2:0/2790712906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06ecc3c70 msgr2=0x55b06ece46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:48.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.616+0000 7f53f0e00640 1 --2- 172.21.15.2:0/2790712906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06ecc3c70 0x55b06ece46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.616+0000 7f53f0e00640 1 -- 172.21.15.2:0/2790712906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b06ece24a0 con 0x55b06ecc4590 2024-09-10T08:12:48.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.616+0000 7f53f0e00640 1 --2- 172.21.15.2:0/2790712906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc4590 0x55b06ece1a20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f53d802a9e0 tx=0x7f53d802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96312a1e12c5ec46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.616+0000 7f53d3400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.616+0000 7f53d3400640 1 -- 172.21.15.2:0/2790712906 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8035040 con 0x55b06ecc4590 2024-09-10T08:12:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.616+0000 7f53d3400640 1 -- 172.21.15.2:0/2790712906 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f53d8005450 con 0x55b06ecc4590 2024-09-10T08:12:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.616+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2790712906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e4015070 con 0x55b06ecc4590 2024-09-10T08:12:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.617+0000 7f53d3400640 1 -- 172.21.15.2:0/2790712906 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53d8056050 con 0x55b06ecc4590 2024-09-10T08:12:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.617+0000 7f53f6cbb740 10 client.26720 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.617+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2790712906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b06ed494a0 con 0x55b06ecc4590 2024-09-10T08:12:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.617+0000 7f53d3400640 1 -- 172.21.15.2:0/2790712906 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e4015070 con 0x55b06ecc4590 2024-09-10T08:12:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.617+0000 7f53d3400640 1 -- 172.21.15.2:0/2790712906 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f53d8031880 con 0x55b06ecc4590 2024-09-10T08:12:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.617+0000 7f53f6cbb740 10 client.26720 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.617+0000 7f53f6cbb740 10 client.26720 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:12:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.617+0000 7f53f6cbb740 20 client.26720 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.617+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2790712906 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b06ed49bf0 0x55b06ed4c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:48.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.617+0000 7f53f6cbb740 4 client.26720 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:12:48.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.617+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2790712906 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b06ec31e60 con 0x55b06ed49bf0 2024-09-10T08:12:48.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.618+0000 7f53f2200640 1 --2- 172.21.15.2:0/2790712906 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b06ed49bf0 0x55b06ed4c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:48.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.618+0000 7f53f2200640 1 --2- 172.21.15.2:0/2790712906 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b06ed49bf0 0x55b06ed4c010 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40dfdc2fce633d5f server_cookie=d59d23dd28ccf45 in_seq=0 out_seq=0 2024-09-10T08:12:48.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.618+0000 7f53d3400640 10 client.26720 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:12:48.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.618+0000 7f53d3400640 1 -- 172.21.15.2:0/2790712906 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b06ec31e60 con 0x55b06ed49bf0 2024-09-10T08:12:48.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.618+0000 7f53d3400640 10 client.26720 handle_command_reply: tid=0 2024-09-10T08:12:48.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.618+0000 7f53f6cbb740 1 client.26720 shutdown 2024-09-10T08:12:48.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.619+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2790712906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc4590 msgr2=0x55b06ece1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:48.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.619+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2790712906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc4590 0x55b06ece1a20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f53d802a9e0 tx=0x7f53d802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.619+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2790712906 shutdown_connections 2024-09-10T08:12:48.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.619+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2790712906 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55b06ed49bf0 0x55b06ed4c010 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.619+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2790712906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b06eccee50 0x55b06ece1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.619+0000 7f53d3400640 0 client.26720 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:12:48.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.619+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2790712906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b06ecc4590 0x55b06ece1a20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.619+0000 7f53f6cbb740 1 --2- 172.21.15.2:0/2790712906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b06ecc3c70 0x55b06ece46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.619+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2790712906 >> 172.21.15.2:0/2790712906 conn(0x55b06ecc0200 msgr2=0x55b06eccbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:48.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.619+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2790712906 shutdown_connections 2024-09-10T08:12:48.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.619+0000 7f53f6cbb740 1 -- 172.21.15.2:0/2790712906 wait complete. 2024-09-10T08:12:48.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.619+0000 7f53f6cbb740 20 client.26720 trim_cache size 0 max 16384 2024-09-10T08:12:48.623 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:12:48.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.620+0000 7f53f4200640 1 -- 172.21.15.2:0/2363971223 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f53d40a2b80 msgr2=0x7f53d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:48.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.620+0000 7f53f4200640 1 --2- 172.21.15.2:0/2363971223 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f53d40a2b80 0x7f53d40a5040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55b06ecc1060 tx=0x7f53e00d2e80 comp rx=0 tx=0).stop 2024-09-10T08:12:48.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.620+0000 7f53f4200640 1 -- 172.21.15.2:0/2363971223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53ec101de0 msgr2=0x7f53ec19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:48.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.620+0000 7f53f4200640 1 --2- 172.21.15.2:0/2363971223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53ec101de0 0x7f53ec19a6d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f53e000e9f0 tx=0x7f53e000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.620+0000 7f53f4200640 1 -- 172.21.15.2:0/2363971223 shutdown_connections 2024-09-10T08:12:48.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.620+0000 7f53f4200640 1 --2- 172.21.15.2:0/2363971223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53ec106f10 0x7f53ec19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.620+0000 7f53f4200640 1 --2- 172.21.15.2:0/2363971223 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f53d40a2b80 0x7f53d40a5040 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.620+0000 7f53f4200640 1 --2- 172.21.15.2:0/2363971223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53ec102790 0x7f53ec19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.620+0000 7f53f4200640 1 --2- 172.21.15.2:0/2363971223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53ec101de0 0x7f53ec19a6d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:48.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.620+0000 7f53f4200640 1 -- 172.21.15.2:0/2363971223 >> 172.21.15.2:0/2363971223 conn(0x7f53ec0fd8c0 msgr2=0x7f53ec0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:48.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.620+0000 7f53f4200640 1 -- 172.21.15.2:0/2363971223 shutdown_connections 2024-09-10T08:12:48.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:48.620+0000 7f53f4200640 1 -- 172.21.15.2:0/2363971223 wait complete. 2024-09-10T08:12:49.556 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:12:49.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:49 smithi002 ceph-mon[30563]: pgmap v527: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-09-10T08:12:49.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:49 smithi155 ceph-mon[35405]: pgmap v527: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-09-10T08:12:49.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:49 smithi107 ceph-mon[35048]: pgmap v527: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-09-10T08:12:49.937 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:50.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.658+0000 7f7920600640 1 -- 172.21.15.2:0/953193668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7918112a30 msgr2=0x7f7918114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:50.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.658+0000 7f7920600640 1 --2- 172.21.15.2:0/953193668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7918112a30 0x7f7918114e20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7910009ca0 tx=0x7f791002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.658+0000 7f7920600640 1 -- 172.21.15.2:0/953193668 shutdown_connections 2024-09-10T08:12:50.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.658+0000 7f7920600640 1 --2- 172.21.15.2:0/953193668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7918112a30 0x7f7918114e20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.658+0000 7f7920600640 1 --2- 172.21.15.2:0/953193668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7918077040 0x7f79181124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.658+0000 7f7920600640 1 --2- 172.21.15.2:0/953193668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7918076720 0x7f7918076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.658+0000 7f7920600640 1 -- 172.21.15.2:0/953193668 >> 172.21.15.2:0/953193668 conn(0x7f79180ffa90 msgr2=0x7f7918101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:50.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.659+0000 7f7920600640 1 -- 172.21.15.2:0/953193668 shutdown_connections 2024-09-10T08:12:50.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.659+0000 7f7920600640 1 -- 172.21.15.2:0/953193668 wait complete. 2024-09-10T08:12:50.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.659+0000 7f7920600640 1 Processor -- start 2024-09-10T08:12:50.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.660+0000 7f7920600640 1 -- start start 2024-09-10T08:12:50.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.660+0000 7f7920600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7918076720 0x7f79181a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.660+0000 7f7920600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7918077040 0x7f79181a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.660+0000 7f7920600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7918112a30 0x7f791819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.660+0000 7f7920600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f79181185e0 con 0x7f7918076720 2024-09-10T08:12:50.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.660+0000 7f7920600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7918118460 con 0x7f7918077040 2024-09-10T08:12:50.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.660+0000 7f7920600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7918118760 con 0x7f7918112a30 2024-09-10T08:12:50.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.660+0000 7f791dc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7918076720 0x7f79181a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.660+0000 7f791dc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7918076720 0x7f79181a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34528/0 (socket says 172.21.15.2:34528) 2024-09-10T08:12:50.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.660+0000 7f791dc00640 1 -- 172.21.15.2:0/1863586768 learned_addr learned my addr 172.21.15.2:0/1863586768 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:50.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.660+0000 7f791d200640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7918077040 0x7f79181a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.660+0000 7f791e600640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7918112a30 0x7f791819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.661+0000 7f791d200640 1 -- 172.21.15.2:0/1863586768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7918112a30 msgr2=0x7f791819e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:50.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.661+0000 7f791d200640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7918112a30 0x7f791819e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.661+0000 7f791d200640 1 -- 172.21.15.2:0/1863586768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7918076720 msgr2=0x7f79181a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:50.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.661+0000 7f791d200640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7918076720 0x7f79181a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.661+0000 7f791d200640 1 -- 172.21.15.2:0/1863586768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791819f040 con 0x7f7918077040 2024-09-10T08:12:50.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.661+0000 7f791d200640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7918077040 0x7f79181a4c90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f790400e9f0 tx=0x7f790400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:50.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.661+0000 7f7916000640 1 -- 172.21.15.2:0/1863586768 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790400ccc0 con 0x7f7918077040 2024-09-10T08:12:50.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.661+0000 7f7916000640 1 -- 172.21.15.2:0/1863586768 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7904005020 con 0x7f7918077040 2024-09-10T08:12:50.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.661+0000 7f7920600640 1 -- 172.21.15.2:0/1863586768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791819f330 con 0x7f7918077040 2024-09-10T08:12:50.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.661+0000 7f7916000640 1 -- 172.21.15.2:0/1863586768 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79040049e0 con 0x7f7918077040 2024-09-10T08:12:50.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.662+0000 7f7920600640 1 -- 172.21.15.2:0/1863586768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79181ab620 con 0x7f7918077040 2024-09-10T08:12:50.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.665+0000 7f792305f740 1 Processor -- start 2024-09-10T08:12:50.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.665+0000 7f7916000640 1 -- 172.21.15.2:0/1863586768 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7904005560 con 0x7f7918077040 2024-09-10T08:12:50.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.665+0000 7f792305f740 1 -- start start 2024-09-10T08:12:50.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.665+0000 7f792305f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f3833050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.665+0000 7f792305f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f3833590 0x55f2f383d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.665+0000 7f792305f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f383de50 0x55f2f3840240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.665+0000 7f792305f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f2f37993e0 con 0x55f2f3832c70 2024-09-10T08:12:50.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.665+0000 7f792305f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f2f3740d80 con 0x55f2f3833590 2024-09-10T08:12:50.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.665+0000 7f792305f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f2f37953a0 con 0x55f2f383de50 2024-09-10T08:12:50.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.665+0000 7f791e600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f3833050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.665+0000 7f791e600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f3833050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34544/0 (socket says 172.21.15.2:34544) 2024-09-10T08:12:50.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.665+0000 7f791e600640 1 -- 172.21.15.2:0/1510646069 learned_addr learned my addr 172.21.15.2:0/1510646069 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:50.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.666+0000 7f791d200640 1 --2- 172.21.15.2:0/1510646069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f383de50 0x55f2f3840240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.666+0000 7f791dc00640 1 --2- 172.21.15.2:0/1510646069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f3833590 0x55f2f383d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.666+0000 7f791e600640 1 -- 172.21.15.2:0/1510646069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f383de50 msgr2=0x55f2f3840240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:50.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.666+0000 7f791e600640 1 --2- 172.21.15.2:0/1510646069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f383de50 0x55f2f3840240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.666+0000 7f791e600640 1 -- 172.21.15.2:0/1510646069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f3833590 msgr2=0x55f2f383d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:50.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.666+0000 7f791e600640 1 --2- 172.21.15.2:0/1510646069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f3833590 0x55f2f383d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.666+0000 7f791e600640 1 -- 172.21.15.2:0/1510646069 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2f375fae0 con 0x55f2f3832c70 2024-09-10T08:12:50.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.666+0000 7f791dc00640 1 --2- 172.21.15.2:0/1510646069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f3833590 0x55f2f383d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:50.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.666+0000 7f791e600640 1 --2- 172.21.15.2:0/1510646069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f3833050 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f791807aa10 tx=0x7f7910005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0f93b7eb1b9e70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:50.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.667+0000 7f7916000640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79000a2b80 0x7f79000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.667+0000 7f790aa00640 1 -- 172.21.15.2:0/1510646069 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910031db0 con 0x55f2f3832c70 2024-09-10T08:12:50.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.667+0000 7f790aa00640 1 -- 172.21.15.2:0/1510646069 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7910036030 con 0x55f2f3832c70 2024-09-10T08:12:50.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.667+0000 7f7916000640 1 -- 172.21.15.2:0/1863586768 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f79040cf380 con 0x7f7918077040 2024-09-10T08:12:50.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.667+0000 7f790aa00640 1 -- 172.21.15.2:0/1510646069 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100042a0 con 0x55f2f3832c70 2024-09-10T08:12:50.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.667+0000 7f791dc00640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79000a2b80 0x7f79000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.667+0000 7f792305f740 1 -- 172.21.15.2:0/1510646069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 msgr2=0x55f2f3833050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:50.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.667+0000 7f792305f740 1 --2- 172.21.15.2:0/1510646069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f3833050 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f791807aa10 tx=0x7f7910005f80 comp rx=0 tx=0).stop 2024-09-10T08:12:50.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.668+0000 7f791dc00640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79000a2b80 0x7f79000a5040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55f2f3830060 tx=0x7f790c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:50.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.668+0000 7f792305f740 1 -- 172.21.15.2:0/1510646069 shutdown_connections 2024-09-10T08:12:50.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.668+0000 7f792305f740 1 --2- 172.21.15.2:0/1510646069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f383de50 0x55f2f3840240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.668+0000 7f792305f740 1 --2- 172.21.15.2:0/1510646069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f3833590 0x55f2f383d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.668+0000 7f792305f740 1 --2- 172.21.15.2:0/1510646069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f3833050 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.668+0000 7f792305f740 1 -- 172.21.15.2:0/1510646069 >> 172.21.15.2:0/1510646069 conn(0x55f2f382f200 msgr2=0x55f2f382f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:50.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.668+0000 7f792305f740 1 -- 172.21.15.2:0/1510646069 shutdown_connections 2024-09-10T08:12:50.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.668+0000 7f792305f740 1 -- 172.21.15.2:0/1510646069 wait complete. 2024-09-10T08:12:50.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.668+0000 7f792305f740 1 -- start start 2024-09-10T08:12:50.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.669+0000 7f792305f740 1 Processor -- start 2024-09-10T08:12:50.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.669+0000 7f792305f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f383add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.669+0000 7f792305f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f383b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.669+0000 7f792305f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f383de50 0x55f2f383b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.669+0000 7f792305f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f2f38408a0 con 0x55f2f3832c70 2024-09-10T08:12:50.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.669+0000 7f792305f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f2f37540f0 con 0x55f2f383de50 2024-09-10T08:12:50.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.669+0000 7f792305f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f2f3840a20 con 0x55f2f3833590 2024-09-10T08:12:50.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.669+0000 7f792305f740 10 client.0 _ll_register_callbacks cb 0x55f2f373bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:50.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.669+0000 7f791dc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f383add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.669+0000 7f791d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f383b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.669+0000 7f791dc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f383add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34556/0 (socket says 172.21.15.2:34556) 2024-09-10T08:12:50.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.669+0000 7f791d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f383b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59976/0 (socket says 172.21.15.2:59976) 2024-09-10T08:12:50.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.669+0000 7f791dc00640 1 -- 172.21.15.2:0/3006838187 learned_addr learned my addr 172.21.15.2:0/3006838187 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:50.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.670+0000 7f791e600640 1 --2- 172.21.15.2:0/3006838187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f383de50 0x55f2f383b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.670+0000 7f791d200640 1 -- 172.21.15.2:0/3006838187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f383de50 msgr2=0x55f2f383b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:50.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.670+0000 7f791d200640 1 --2- 172.21.15.2:0/3006838187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f383de50 0x55f2f383b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.670+0000 7f791d200640 1 -- 172.21.15.2:0/3006838187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 msgr2=0x55f2f383add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:50.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.670+0000 7f791d200640 1 --2- 172.21.15.2:0/3006838187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f383add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.670+0000 7f791d200640 1 -- 172.21.15.2:0/3006838187 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2f350a3c0 con 0x55f2f3833590 2024-09-10T08:12:50.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.670+0000 7f791d200640 1 --2- 172.21.15.2:0/3006838187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f383b310 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55f2f3830580 tx=0x7f7904097430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeb430f23525c2f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:50.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.670+0000 7f78ff400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:50.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.670+0000 7f78ff400640 1 -- 172.21.15.2:0/3006838187 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790401c070 con 0x55f2f3833590 2024-09-10T08:12:50.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.670+0000 7f78ff400640 1 -- 172.21.15.2:0/3006838187 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7904097e00 con 0x55f2f3833590 2024-09-10T08:12:50.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.670+0000 7f792305f740 1 -- 172.21.15.2:0/3006838187 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7910036030 con 0x55f2f3833590 2024-09-10T08:12:50.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.671+0000 7f78ff400640 1 -- 172.21.15.2:0/3006838187 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f790409b0f0 con 0x55f2f3833590 2024-09-10T08:12:50.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.671+0000 7f792305f740 10 client.26692 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:50.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.671+0000 7f792305f740 1 -- 172.21.15.2:0/3006838187 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f2f384c960 con 0x55f2f3833590 2024-09-10T08:12:50.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.671+0000 7f78ff400640 1 -- 172.21.15.2:0/3006838187 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7910036030 con 0x55f2f3833590 2024-09-10T08:12:50.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.671+0000 7f78ff400640 1 -- 172.21.15.2:0/3006838187 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f79040a3f70 con 0x55f2f3833590 2024-09-10T08:12:50.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.671+0000 7f792305f740 10 client.26692 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:50.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.671+0000 7f792305f740 10 client.26692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:50.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.671+0000 7f792305f740 20 client.26692 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:50.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.671+0000 7f792305f740 1 --2- 172.21.15.2:0/3006838187 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f2f384dd70 0x55f2f3850180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.671+0000 7f792305f740 4 client.26692 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:50.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.671+0000 7f792305f740 1 -- 172.21.15.2:0/3006838187 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2f37a6950 con 0x55f2f384dd70 2024-09-10T08:12:50.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.672+0000 7f791dc00640 1 --2- 172.21.15.2:0/3006838187 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f2f384dd70 0x55f2f3850180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.672+0000 7f791dc00640 1 --2- 172.21.15.2:0/3006838187 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f2f384dd70 0x55f2f3850180 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ecf8646f9df0ea server_cookie=3e81604cc5ae00ac in_seq=0 out_seq=0 2024-09-10T08:12:50.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.672+0000 7f78ff400640 10 client.26692 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:50.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.673+0000 7f78ff400640 1 -- 172.21.15.2:0/3006838187 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2f37a6950 con 0x55f2f384dd70 2024-09-10T08:12:50.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.673+0000 7f78ff400640 10 client.26692 handle_command_reply: tid=0 2024-09-10T08:12:50.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.673+0000 7f792305f740 1 client.26692 shutdown 2024-09-10T08:12:50.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.673+0000 7f792305f740 1 -- 172.21.15.2:0/3006838187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 msgr2=0x55f2f383b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:50.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.673+0000 7f792305f740 1 --2- 172.21.15.2:0/3006838187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f383b310 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55f2f3830580 tx=0x7f7904097430 comp rx=0 tx=0).stop 2024-09-10T08:12:50.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.674+0000 7f792305f740 1 -- 172.21.15.2:0/3006838187 shutdown_connections 2024-09-10T08:12:50.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.674+0000 7f792305f740 1 --2- 172.21.15.2:0/3006838187 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f2f384dd70 0x55f2f3850180 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.674+0000 7f792305f740 1 --2- 172.21.15.2:0/3006838187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f383de50 0x55f2f383b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.674+0000 7f792305f740 1 --2- 172.21.15.2:0/3006838187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f383b310 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.674+0000 7f792305f740 1 --2- 172.21.15.2:0/3006838187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f383add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.674+0000 7f78ff400640 0 client.26692 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:50.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.674+0000 7f792305f740 1 -- 172.21.15.2:0/3006838187 >> 172.21.15.2:0/3006838187 conn(0x55f2f382f200 msgr2=0x55f2f382e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:50.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.674+0000 7f792305f740 1 -- 172.21.15.2:0/3006838187 shutdown_connections 2024-09-10T08:12:50.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.674+0000 7f792305f740 1 -- 172.21.15.2:0/3006838187 wait complete. 2024-09-10T08:12:50.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.674+0000 7f792305f740 20 client.26692 trim_cache size 0 max 16384 2024-09-10T08:12:50.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.688+0000 7f792305f740 1 Processor -- start 2024-09-10T08:12:50.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.688+0000 7f792305f740 1 -- start start 2024-09-10T08:12:50.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.688+0000 7f792305f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f3835720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.688+0000 7f792305f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f3855a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.688+0000 7f792305f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f383de50 0x55f2f3855f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.688+0000 7f792305f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f79181b7cf0 con 0x55f2f3832c70 2024-09-10T08:12:50.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.688+0000 7f792305f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f79181b55c0 con 0x55f2f383de50 2024-09-10T08:12:50.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.688+0000 7f792305f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79181b3410 con 0x55f2f3833590 2024-09-10T08:12:50.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.689+0000 7f791d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f3855a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.689+0000 7f791e600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f383de50 0x55f2f3855f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.689+0000 7f791dc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f3835720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.689+0000 7f791e600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f383de50 0x55f2f3855f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:45900/0 (socket says 172.21.15.2:45900) 2024-09-10T08:12:50.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.689+0000 7f791e600640 1 -- 172.21.15.2:0/1213873570 learned_addr learned my addr 172.21.15.2:0/1213873570 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:50.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.689+0000 7f791d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f3855a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59980/0 (socket says 172.21.15.2:59980) 2024-09-10T08:12:50.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.689+0000 7f791e600640 1 -- 172.21.15.2:0/1213873570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 msgr2=0x55f2f3855a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:50.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.689+0000 7f791e600640 1 --2- 172.21.15.2:0/1213873570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f3855a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.689+0000 7f791e600640 1 -- 172.21.15.2:0/1213873570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 msgr2=0x55f2f3835720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:50.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.689+0000 7f791e600640 1 --2- 172.21.15.2:0/1213873570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f3835720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.689+0000 7f791e600640 1 -- 172.21.15.2:0/1213873570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2f3832aa0 con 0x55f2f383de50 2024-09-10T08:12:50.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.689+0000 7f791e600640 1 --2- 172.21.15.2:0/1213873570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f383de50 0x55f2f3855f90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7910004af0 tx=0x7f7910031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e6b1f923c28413 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:50.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.690+0000 7f790a000640 1 -- 172.21.15.2:0/1213873570 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791004a040 con 0x55f2f383de50 2024-09-10T08:12:50.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.690+0000 7f790a000640 1 -- 172.21.15.2:0/1213873570 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7910055070 con 0x55f2f383de50 2024-09-10T08:12:50.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.690+0000 7f790a000640 1 -- 172.21.15.2:0/1213873570 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100042a0 con 0x55f2f383de50 2024-09-10T08:12:50.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.690+0000 7f792305f740 1 -- 172.21.15.2:0/1213873570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f383de50 msgr2=0x55f2f3855f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:50.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.690+0000 7f792305f740 1 --2- 172.21.15.2:0/1213873570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f383de50 0x55f2f3855f90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7910004af0 tx=0x7f7910031a20 comp rx=0 tx=0).stop 2024-09-10T08:12:50.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.690+0000 7f792305f740 1 -- 172.21.15.2:0/1213873570 shutdown_connections 2024-09-10T08:12:50.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.690+0000 7f792305f740 1 --2- 172.21.15.2:0/1213873570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f383de50 0x55f2f3855f90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.690+0000 7f792305f740 1 --2- 172.21.15.2:0/1213873570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f3855a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.690+0000 7f792305f740 1 --2- 172.21.15.2:0/1213873570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f3832c70 0x55f2f3835720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.690+0000 7f792305f740 1 -- 172.21.15.2:0/1213873570 >> 172.21.15.2:0/1213873570 conn(0x55f2f382f200 msgr2=0x55f2f383ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:50.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.690+0000 7f792305f740 1 -- 172.21.15.2:0/1213873570 shutdown_connections 2024-09-10T08:12:50.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.690+0000 7f792305f740 1 -- 172.21.15.2:0/1213873570 wait complete. 2024-09-10T08:12:50.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.691+0000 7f792305f740 1 -- start start 2024-09-10T08:12:50.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.691+0000 7f792305f740 1 Processor -- start 2024-09-10T08:12:50.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.691+0000 7f792305f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f3832c70 0x55f2f3842280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.691+0000 7f792305f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f38427c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.691+0000 7f792305f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f383de50 0x55f2f38b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f792305f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f79181b3a80 con 0x55f2f383de50 2024-09-10T08:12:50.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f792305f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f79181b4ef0 con 0x55f2f3832c70 2024-09-10T08:12:50.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f792305f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79181b4150 con 0x55f2f3833590 2024-09-10T08:12:50.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f792305f740 10 client.0 _ll_register_callbacks cb 0x7f79040d2ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:50.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f791e600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f383de50 0x55f2f38b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f791d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f38427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f791dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f3832c70 0x55f2f3842280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f791d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f38427c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59994/0 (socket says 172.21.15.2:59994) 2024-09-10T08:12:50.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f791d200640 1 -- 172.21.15.2:0/3023296203 learned_addr learned my addr 172.21.15.2:0/3023296203 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:50.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f791e600640 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f383de50 0x55f2f38b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34570/0 (socket says 172.21.15.2:34570) 2024-09-10T08:12:50.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f791e600640 1 -- 172.21.15.2:0/3023296203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 msgr2=0x55f2f38427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:50.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f791e600640 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f38427c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f791e600640 1 -- 172.21.15.2:0/3023296203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f3832c70 msgr2=0x55f2f3842280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:50.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f791e600640 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f3832c70 0x55f2f3842280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f791e600640 1 -- 172.21.15.2:0/3023296203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2f382fe90 con 0x55f2f383de50 2024-09-10T08:12:50.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f791d200640 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f38427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:50.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.692+0000 7f791dc00640 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f3832c70 0x55f2f3842280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:50.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.693+0000 7f791e600640 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f383de50 0x55f2f38b19f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7910052040 tx=0x7f791002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34a44aff4a143af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:50.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.693+0000 7f78ff400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:12:50.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.693+0000 7f78ff400640 1 -- 172.21.15.2:0/3023296203 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100315d0 con 0x55f2f383de50 2024-09-10T08:12:50.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.693+0000 7f78ff400640 1 -- 172.21.15.2:0/3023296203 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7910031770 con 0x55f2f383de50 2024-09-10T08:12:50.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.693+0000 7f792305f740 1 -- 172.21.15.2:0/3023296203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7910055070 con 0x55f2f383de50 2024-09-10T08:12:50.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.693+0000 7f78ff400640 1 -- 172.21.15.2:0/3023296203 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7910068050 con 0x55f2f383de50 2024-09-10T08:12:50.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.693+0000 7f792305f740 10 client.16701 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:50.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.693+0000 7f792305f740 1 -- 172.21.15.2:0/3023296203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f2f38302b0 con 0x55f2f383de50 2024-09-10T08:12:50.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.694+0000 7f78ff400640 1 -- 172.21.15.2:0/3023296203 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7910055070 con 0x55f2f383de50 2024-09-10T08:12:50.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.694+0000 7f78ff400640 1 -- 172.21.15.2:0/3023296203 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7910063030 con 0x55f2f383de50 2024-09-10T08:12:50.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.694+0000 7f792305f740 10 client.16701 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:50.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.694+0000 7f792305f740 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:50.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.694+0000 7f792305f740 20 client.16701 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:50.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.694+0000 7f792305f740 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f2f38b93d0 0x55f2f38bb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:50.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.694+0000 7f792305f740 4 client.16701 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:12:50.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.694+0000 7f792305f740 1 -- 172.21.15.2:0/3023296203 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2f37a0e60 con 0x55f2f38b93d0 2024-09-10T08:12:50.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.694+0000 7f791dc00640 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f2f38b93d0 0x55f2f38bb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:50.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f791dc00640 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f2f38b93d0 0x55f2f38bb7c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8ec5e76e8acfe46 server_cookie=6f5bc3b525adebca in_seq=0 out_seq=0 2024-09-10T08:12:50.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f78ff400640 10 client.16701 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:50.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f78ff400640 1 -- 172.21.15.2:0/3023296203 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2f37a0e60 con 0x55f2f38b93d0 2024-09-10T08:12:50.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f78ff400640 10 client.16701 handle_command_reply: tid=0 2024-09-10T08:12:50.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f792305f740 1 client.16701 shutdown 2024-09-10T08:12:50.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f792305f740 1 -- 172.21.15.2:0/3023296203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f383de50 msgr2=0x55f2f38b19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:50.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f792305f740 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f383de50 0x55f2f38b19f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7910052040 tx=0x7f791002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f792305f740 1 -- 172.21.15.2:0/3023296203 shutdown_connections 2024-09-10T08:12:50.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f792305f740 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f2f38b93d0 0x55f2f38bb7c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f792305f740 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2f383de50 0x55f2f38b19f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f792305f740 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2f3833590 0x55f2f38427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f792305f740 1 --2- 172.21.15.2:0/3023296203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2f3832c70 0x55f2f3842280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f792305f740 1 -- 172.21.15.2:0/3023296203 >> 172.21.15.2:0/3023296203 conn(0x55f2f382f200 msgr2=0x55f2f383ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:50.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.695+0000 7f78ff400640 0 client.16701 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:50.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.696+0000 7f792305f740 1 -- 172.21.15.2:0/3023296203 shutdown_connections 2024-09-10T08:12:50.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.696+0000 7f792305f740 1 -- 172.21.15.2:0/3023296203 wait complete. 2024-09-10T08:12:50.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.696+0000 7f792305f740 20 client.16701 trim_cache size 0 max 16384 2024-09-10T08:12:50.701 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:12:50.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.697+0000 7f7920600640 1 -- 172.21.15.2:0/1863586768 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79000a2b80 msgr2=0x7f79000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:50.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.697+0000 7f7920600640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79000a2b80 0x7f79000a5040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55f2f3830060 tx=0x7f790c005f90 comp rx=0 tx=0).stop 2024-09-10T08:12:50.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.697+0000 7f7920600640 1 -- 172.21.15.2:0/1863586768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7918077040 msgr2=0x7f79181a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:50.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.697+0000 7f7920600640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7918077040 0x7f79181a4c90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f790400e9f0 tx=0x7f790400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.697+0000 7f7920600640 1 -- 172.21.15.2:0/1863586768 shutdown_connections 2024-09-10T08:12:50.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.697+0000 7f7920600640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f79000a2b80 0x7f79000a5040 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.697+0000 7f7920600640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7918112a30 0x7f791819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.697+0000 7f7920600640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7918077040 0x7f79181a4c90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.697+0000 7f7920600640 1 --2- 172.21.15.2:0/1863586768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7918076720 0x7f79181a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:50.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.697+0000 7f7920600640 1 -- 172.21.15.2:0/1863586768 >> 172.21.15.2:0/1863586768 conn(0x7f79180ffa90 msgr2=0x7f7918113630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:50.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.697+0000 7f7920600640 1 -- 172.21.15.2:0/1863586768 shutdown_connections 2024-09-10T08:12:50.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:50.697+0000 7f7920600640 1 -- 172.21.15.2:0/1863586768 wait complete. 2024-09-10T08:12:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:51 smithi155 ceph-mon[35405]: pgmap v528: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-09-10T08:12:51.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:51 smithi107 ceph-mon[35048]: pgmap v528: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-09-10T08:12:52.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:51 smithi002 ceph-mon[30563]: pgmap v528: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-09-10T08:12:52.593 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:12:52.593 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:12:52.984 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.536+0000 7f94d3200640 1 -- 172.21.15.2:0/598141069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94d4101580 msgr2=0x7f94d4101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.536+0000 7f94d3200640 1 --2- 172.21.15.2:0/598141069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94d4101580 0x7f94d4101a00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009900 tx=0x7f94bc02a410 comp rx=0 tx=0).stop 2024-09-10T08:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.536+0000 7f94d3200640 1 -- 172.21.15.2:0/598141069 shutdown_connections 2024-09-10T08:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.536+0000 7f94d3200640 1 --2- 172.21.15.2:0/598141069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d4101f40 0x7f94d4109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.536+0000 7f94d3200640 1 --2- 172.21.15.2:0/598141069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94d4101580 0x7f94d4101a00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.536+0000 7f94d3200640 1 --2- 172.21.15.2:0/598141069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4103050 0x7f94d4103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.536+0000 7f94d3200640 1 -- 172.21.15.2:0/598141069 >> 172.21.15.2:0/598141069 conn(0x7f94d40fd0d0 msgr2=0x7f94d40ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.537+0000 7f94d3200640 1 -- 172.21.15.2:0/598141069 shutdown_connections 2024-09-10T08:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.537+0000 7f94d3200640 1 -- 172.21.15.2:0/598141069 wait complete. 2024-09-10T08:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.538+0000 7f94d3200640 1 Processor -- start 2024-09-10T08:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.538+0000 7f94d3200640 1 -- start start 2024-09-10T08:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.538+0000 7f94d3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d4101580 0x7f94d4199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.538+0000 7f94d3200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4101f40 0x7f94d419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.538+0000 7f94d3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94d4103050 0x7f94d41a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.538+0000 7f94d3200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f94d410d740 con 0x7f94d4101f40 2024-09-10T08:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.538+0000 7f94d3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f94d410d5c0 con 0x7f94d4101580 2024-09-10T08:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.538+0000 7f94d3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94d410d8c0 con 0x7f94d4103050 2024-09-10T08:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.538+0000 7f94cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4101f40 0x7f94d419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.538+0000 7f94cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4101f40 0x7f94d419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34600/0 (socket says 172.21.15.2:34600) 2024-09-10T08:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.538+0000 7f94cbe00640 1 -- 172.21.15.2:0/1392892163 learned_addr learned my addr 172.21.15.2:0/1392892163 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.538+0000 7f94d1e00640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d4101580 0x7f94d4199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.539+0000 7f94d2800640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94d4103050 0x7f94d41a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.539+0000 7f94d1e00640 1 -- 172.21.15.2:0/1392892163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94d4103050 msgr2=0x7f94d41a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.539+0000 7f94d1e00640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94d4103050 0x7f94d41a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.539+0000 7f94d1e00640 1 -- 172.21.15.2:0/1392892163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4101f40 msgr2=0x7f94d419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.539+0000 7f94d1e00640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4101f40 0x7f94d419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.539+0000 7f94d1e00640 1 -- 172.21.15.2:0/1392892163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d41a1c10 con 0x7f94d4101580 2024-09-10T08:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.539+0000 7f94cbe00640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4101f40 0x7f94d419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.539+0000 7f94d1e00640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d4101580 0x7f94d4199ff0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f94c400e9c0 tx=0x7f94c400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.539+0000 7f94caa00640 1 -- 172.21.15.2:0/1392892163 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c400ccc0 con 0x7f94d4101580 2024-09-10T08:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.540+0000 7f94caa00640 1 -- 172.21.15.2:0/1392892163 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c40035f0 con 0x7f94d4101580 2024-09-10T08:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.540+0000 7f94d3200640 1 -- 172.21.15.2:0/1392892163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d41a1f00 con 0x7f94d4101580 2024-09-10T08:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.540+0000 7f94caa00640 1 -- 172.21.15.2:0/1392892163 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c400ce60 con 0x7f94d4101580 2024-09-10T08:12:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.540+0000 7f94d3200640 1 -- 172.21.15.2:0/1392892163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d41a2360 con 0x7f94d4101580 2024-09-10T08:12:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94dbc21740 1 Processor -- start 2024-09-10T08:12:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94dbc21740 1 -- start start 2024-09-10T08:12:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94dbc21740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f08f7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94caa00640 1 -- 172.21.15.2:0/1392892163 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f94c40037a0 con 0x7f94d4101580 2024-09-10T08:12:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94dbc21740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08f7ba0 0x55c6f08fc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94dbc21740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08fcc30 0x55c6f08ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94dbc21740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c6f085c100 con 0x55c6f08fcc30 2024-09-10T08:12:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94dbc21740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c6f08041b0 con 0x55c6f08f7ba0 2024-09-10T08:12:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94dbc21740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c6f08580c0 con 0x55c6f08f7260 2024-09-10T08:12:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94d2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f08f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08fcc30 0x55c6f08ff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94d2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f08f7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60018/0 (socket says 172.21.15.2:60018) 2024-09-10T08:12:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94d2800640 1 -- 172.21.15.2:0/253919799 learned_addr learned my addr 172.21.15.2:0/253919799 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.543+0000 7f94d1e00640 1 --2- 172.21.15.2:0/253919799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08f7ba0 0x55c6f08fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.544+0000 7f94d2800640 1 -- 172.21.15.2:0/253919799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08f7ba0 msgr2=0x55c6f08fc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.544+0000 7f94d2800640 1 --2- 172.21.15.2:0/253919799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08f7ba0 0x55c6f08fc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.544+0000 7f94d2800640 1 -- 172.21.15.2:0/253919799 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08fcc30 msgr2=0x55c6f08ff0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.544+0000 7f94d2800640 1 --2- 172.21.15.2:0/253919799 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08fcc30 0x55c6f08ff0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.544+0000 7f94d2800640 1 -- 172.21.15.2:0/253919799 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f056f210 con 0x55c6f08f7260 2024-09-10T08:12:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.544+0000 7f94d2800640 1 --2- 172.21.15.2:0/253919799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f08f7660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009fd0 tx=0x7f94cc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a82d6165d612bea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.544+0000 7f94c3000640 1 -- 172.21.15.2:0/253919799 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc004d20 con 0x55c6f08f7260 2024-09-10T08:12:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.544+0000 7f94c3000640 1 -- 172.21.15.2:0/253919799 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94cc004050 con 0x55c6f08f7260 2024-09-10T08:12:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.544+0000 7f94c3000640 1 -- 172.21.15.2:0/253919799 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc005260 con 0x55c6f08f7260 2024-09-10T08:12:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.545+0000 7f94caa00640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94b80a2b80 0x7f94b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.545+0000 7f94caa00640 1 -- 172.21.15.2:0/1392892163 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f94c40cf830 con 0x7f94d4101580 2024-09-10T08:12:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.545+0000 7f94d1e00640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94b80a2b80 0x7f94b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.545+0000 7f94dbc21740 1 -- 172.21.15.2:0/253919799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 msgr2=0x55c6f08f7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.545+0000 7f94dbc21740 1 --2- 172.21.15.2:0/253919799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f08f7660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009fd0 tx=0x7f94cc0118d0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.545+0000 7f94dbc21740 1 -- 172.21.15.2:0/253919799 shutdown_connections 2024-09-10T08:12:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.545+0000 7f94dbc21740 1 --2- 172.21.15.2:0/253919799 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08fcc30 0x55c6f08ff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.545+0000 7f94dbc21740 1 --2- 172.21.15.2:0/253919799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08f7ba0 0x55c6f08fc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.545+0000 7f94dbc21740 1 --2- 172.21.15.2:0/253919799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f08f7660 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.545+0000 7f94dbc21740 1 -- 172.21.15.2:0/253919799 >> 172.21.15.2:0/253919799 conn(0x55c6f08f28d0 msgr2=0x55c6f08f2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.546+0000 7f94dbc21740 1 -- 172.21.15.2:0/253919799 shutdown_connections 2024-09-10T08:12:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.546+0000 7f94dbc21740 1 -- 172.21.15.2:0/253919799 wait complete. 2024-09-10T08:12:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.546+0000 7f94d1e00640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94b80a2b80 0x7f94b80a5040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f94c40a43e0 tx=0x7f94c40d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.546+0000 7f94dbc21740 1 -- start start 2024-09-10T08:12:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.546+0000 7f94dbc21740 1 Processor -- start 2024-09-10T08:12:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94dbc21740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f09013f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94dbc21740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08f7ba0 0x55c6f0901930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94dbc21740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08fcc30 0x55c6f0908970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94dbc21740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c6f08ffc00 con 0x55c6f08f7ba0 2024-09-10T08:12:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94dbc21740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c6f07e5090 con 0x55c6f08fcc30 2024-09-10T08:12:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94dbc21740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c6f08ffd80 con 0x55c6f08f7260 2024-09-10T08:12:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94dbc21740 10 client.0 _ll_register_callbacks cb 0x55c6f0839f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94d2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f09013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08f7ba0 0x55c6f0901930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94d2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f09013f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60034/0 (socket says 172.21.15.2:60034) 2024-09-10T08:12:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94d2800640 1 -- 172.21.15.2:0/2198770784 learned_addr learned my addr 172.21.15.2:0/2198770784 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94d2800640 1 --2- 172.21.15.2:0/2198770784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08fcc30 0x55c6f0908970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94d2800640 1 -- 172.21.15.2:0/2198770784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08fcc30 msgr2=0x55c6f0908970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94d2800640 1 --2- 172.21.15.2:0/2198770784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08fcc30 0x55c6f0908970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94d2800640 1 -- 172.21.15.2:0/2198770784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08f7ba0 msgr2=0x55c6f0901930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94d2800640 1 --2- 172.21.15.2:0/2198770784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08f7ba0 0x55c6f0901930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.547+0000 7f94d2800640 1 -- 172.21.15.2:0/2198770784 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f063b620 con 0x55c6f08f7260 2024-09-10T08:12:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.548+0000 7f94d2800640 1 --2- 172.21.15.2:0/2198770784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f09013f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009fa0 tx=0x7f94cc009c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf1bbe08706fe5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.548+0000 7f94b7400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.548+0000 7f94b7400640 1 -- 172.21.15.2:0/2198770784 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc004610 con 0x55c6f08f7260 2024-09-10T08:12:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.548+0000 7f94dbc21740 1 -- 172.21.15.2:0/2198770784 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94cc004050 con 0x55c6f08f7260 2024-09-10T08:12:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.548+0000 7f94b7400640 1 -- 172.21.15.2:0/2198770784 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f94cc00f4e0 con 0x55c6f08f7260 2024-09-10T08:12:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.548+0000 7f94b7400640 1 -- 172.21.15.2:0/2198770784 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f94cc036050 con 0x55c6f08f7260 2024-09-10T08:12:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.548+0000 7f94dbc21740 10 client.26710 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.548+0000 7f94dbc21740 1 -- 172.21.15.2:0/2198770784 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c6f090b7b0 con 0x55c6f08f7260 2024-09-10T08:12:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.548+0000 7f94b7400640 1 -- 172.21.15.2:0/2198770784 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94cc004050 con 0x55c6f08f7260 2024-09-10T08:12:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.549+0000 7f94b7400640 1 -- 172.21.15.2:0/2198770784 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f94cc028660 con 0x55c6f08f7260 2024-09-10T08:12:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.549+0000 7f94dbc21740 10 client.26710 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.549+0000 7f94dbc21740 10 client.26710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.549+0000 7f94dbc21740 20 client.26710 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.549+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2198770784 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c6f090cd00 0x55c6f090f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.549+0000 7f94dbc21740 4 client.26710 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.549+0000 7f94dbc21740 1 -- 172.21.15.2:0/2198770784 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6f0869830 con 0x55c6f090cd00 2024-09-10T08:12:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.549+0000 7f94cbe00640 1 --2- 172.21.15.2:0/2198770784 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c6f090cd00 0x55c6f090f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.550+0000 7f94cbe00640 1 --2- 172.21.15.2:0/2198770784 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c6f090cd00 0x55c6f090f0f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d51c48fd1538cd server_cookie=18994ee70fb15ea8 in_seq=0 out_seq=0 2024-09-10T08:12:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.550+0000 7f94b7400640 10 client.26710 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.550+0000 7f94b7400640 1 -- 172.21.15.2:0/2198770784 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6f0869830 con 0x55c6f090cd00 2024-09-10T08:12:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.550+0000 7f94b7400640 10 client.26710 handle_command_reply: tid=0 2024-09-10T08:12:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.550+0000 7f94dbc21740 1 client.26710 shutdown 2024-09-10T08:12:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.551+0000 7f94dbc21740 1 -- 172.21.15.2:0/2198770784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 msgr2=0x55c6f09013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.551+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2198770784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f09013f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009fa0 tx=0x7f94cc009c80 comp rx=0 tx=0).stop 2024-09-10T08:12:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.551+0000 7f94dbc21740 1 -- 172.21.15.2:0/2198770784 shutdown_connections 2024-09-10T08:12:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.551+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2198770784 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c6f090cd00 0x55c6f090f0f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.551+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2198770784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08fcc30 0x55c6f0908970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.551+0000 7f94b7400640 0 client.26710 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.551+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2198770784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08f7ba0 0x55c6f0901930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.551+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2198770784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f09013f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.551+0000 7f94dbc21740 1 -- 172.21.15.2:0/2198770784 >> 172.21.15.2:0/2198770784 conn(0x55c6f08f28d0 msgr2=0x55c6f08f40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.551+0000 7f94dbc21740 1 -- 172.21.15.2:0/2198770784 shutdown_connections 2024-09-10T08:12:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.551+0000 7f94dbc21740 1 -- 172.21.15.2:0/2198770784 wait complete. 2024-09-10T08:12:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.551+0000 7f94dbc21740 20 client.26710 trim_cache size 0 max 16384 2024-09-10T08:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.567+0000 7f94dbc21740 1 Processor -- start 2024-09-10T08:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.567+0000 7f94dbc21740 1 -- start start 2024-09-10T08:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.567+0000 7f94dbc21740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08f7260 0x55c6f09019c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.567+0000 7f94dbc21740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7ba0 0x55c6f0908890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.567+0000 7f94dbc21740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08fcc30 0x55c6f08f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.567+0000 7f94dbc21740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41af220 con 0x55c6f08fcc30 2024-09-10T08:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.567+0000 7f94dbc21740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41acaf0 con 0x55c6f08f7260 2024-09-10T08:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.567+0000 7f94dbc21740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41aa940 con 0x55c6f08f7ba0 2024-09-10T08:12:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.567+0000 7f94cbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7ba0 0x55c6f0908890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.567+0000 7f94d2800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08fcc30 0x55c6f08f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.567+0000 7f94cbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7ba0 0x55c6f0908890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60046/0 (socket says 172.21.15.2:60046) 2024-09-10T08:12:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.568+0000 7f94cbe00640 1 -- 172.21.15.2:0/2994960325 learned_addr learned my addr 172.21.15.2:0/2994960325 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.568+0000 7f94d2800640 1 --2- 172.21.15.2:0/2994960325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08f7260 0x55c6f09019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.568+0000 7f94cbe00640 1 -- 172.21.15.2:0/2994960325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08f7260 msgr2=0x55c6f09019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.568+0000 7f94cbe00640 1 --2- 172.21.15.2:0/2994960325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08f7260 0x55c6f09019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.568+0000 7f94cbe00640 1 -- 172.21.15.2:0/2994960325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08fcc30 msgr2=0x55c6f08f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.568+0000 7f94cbe00640 1 --2- 172.21.15.2:0/2994960325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08fcc30 0x55c6f08f2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.568+0000 7f94cbe00640 1 -- 172.21.15.2:0/2994960325 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f08f26f0 con 0x55c6f08f7ba0 2024-09-10T08:12:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.568+0000 7f94d2800640 1 --2- 172.21.15.2:0/2994960325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08f7260 0x55c6f09019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.568+0000 7f94cbe00640 1 --2- 172.21.15.2:0/2994960325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7ba0 0x55c6f0908890 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f94bc031440 tx=0x7f94bc031980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d974d7bda632970 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.568+0000 7f94c2600640 1 -- 172.21.15.2:0/2994960325 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc035040 con 0x55c6f08f7ba0 2024-09-10T08:12:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.568+0000 7f94c2600640 1 -- 172.21.15.2:0/2994960325 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94bc005500 con 0x55c6f08f7ba0 2024-09-10T08:12:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.569+0000 7f94c2600640 1 -- 172.21.15.2:0/2994960325 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc004ba0 con 0x55c6f08f7ba0 2024-09-10T08:12:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.569+0000 7f94dbc21740 1 -- 172.21.15.2:0/2994960325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7ba0 msgr2=0x55c6f0908890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.569+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2994960325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7ba0 0x55c6f0908890 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f94bc031440 tx=0x7f94bc031980 comp rx=0 tx=0).stop 2024-09-10T08:12:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.569+0000 7f94dbc21740 1 -- 172.21.15.2:0/2994960325 shutdown_connections 2024-09-10T08:12:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.569+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2994960325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08fcc30 0x55c6f08f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.569+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2994960325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7ba0 0x55c6f0908890 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.569+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2994960325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08f7260 0x55c6f09019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.569+0000 7f94dbc21740 1 -- 172.21.15.2:0/2994960325 >> 172.21.15.2:0/2994960325 conn(0x55c6f08f28d0 msgr2=0x55c6f08fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.569+0000 7f94dbc21740 1 -- 172.21.15.2:0/2994960325 shutdown_connections 2024-09-10T08:12:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.569+0000 7f94dbc21740 1 -- 172.21.15.2:0/2994960325 wait complete. 2024-09-10T08:12:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.570+0000 7f94dbc21740 1 -- start start 2024-09-10T08:12:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.570+0000 7f94dbc21740 1 Processor -- start 2024-09-10T08:12:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.570+0000 7f94dbc21740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f0905760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.570+0000 7f94dbc21740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08f7ba0 0x55c6f0905ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.570+0000 7f94dbc21740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08fcc30 0x55c6f09759a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.570+0000 7f94dbc21740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41aafb0 con 0x55c6f08f7ba0 2024-09-10T08:12:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.570+0000 7f94dbc21740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41ac420 con 0x55c6f08fcc30 2024-09-10T08:12:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.570+0000 7f94dbc21740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41ab680 con 0x55c6f08f7260 2024-09-10T08:12:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.570+0000 7f94dbc21740 10 client.0 _ll_register_callbacks cb 0x7f94cc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.571+0000 7f94d2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f0905760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.571+0000 7f94cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08f7ba0 0x55c6f0905ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.571+0000 7f94d2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f0905760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60054/0 (socket says 172.21.15.2:60054) 2024-09-10T08:12:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.571+0000 7f94d2800640 1 -- 172.21.15.2:0/2145092638 learned_addr learned my addr 172.21.15.2:0/2145092638 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.571+0000 7f94d2800640 1 --2- 172.21.15.2:0/2145092638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08fcc30 0x55c6f09759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.571+0000 7f94d2800640 1 -- 172.21.15.2:0/2145092638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08fcc30 msgr2=0x55c6f09759a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.571+0000 7f94d2800640 1 --2- 172.21.15.2:0/2145092638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08fcc30 0x55c6f09759a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.571+0000 7f94d2800640 1 -- 172.21.15.2:0/2145092638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08f7ba0 msgr2=0x55c6f0905ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.571+0000 7f94d2800640 1 --2- 172.21.15.2:0/2145092638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08f7ba0 0x55c6f0905ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.571+0000 7f94d2800640 1 -- 172.21.15.2:0/2145092638 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f0975ee0 con 0x55c6f08f7260 2024-09-10T08:12:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.571+0000 7f94d2800640 1 --2- 172.21.15.2:0/2145092638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f0905760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f94cc028ab0 tx=0x7f94cc009b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e541cd86f14d97f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.571+0000 7f94b7400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94b7400640 1 -- 172.21.15.2:0/2145092638 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc04b020 con 0x55c6f08f7260 2024-09-10T08:12:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94dbc21740 1 -- 172.21.15.2:0/2145092638 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94bc005500 con 0x55c6f08f7260 2024-09-10T08:12:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94b7400640 1 -- 172.21.15.2:0/2145092638 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f94cc028690 con 0x55c6f08f7260 2024-09-10T08:12:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94b7400640 1 -- 172.21.15.2:0/2145092638 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f94cc053050 con 0x55c6f08f7260 2024-09-10T08:12:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94dbc21740 10 client.26716 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94dbc21740 1 -- 172.21.15.2:0/2145092638 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c6f0977170 con 0x55c6f08f7260 2024-09-10T08:12:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94b7400640 1 -- 172.21.15.2:0/2145092638 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94bc005500 con 0x55c6f08f7260 2024-09-10T08:12:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94b7400640 1 -- 172.21.15.2:0/2145092638 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f94cc0198c0 con 0x55c6f08f7260 2024-09-10T08:12:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94dbc21740 10 client.26716 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94dbc21740 10 client.26716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94dbc21740 20 client.26716 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2145092638 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c6f0977a60 0x55c6f0979e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94dbc21740 4 client.26716 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:12:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.572+0000 7f94dbc21740 1 -- 172.21.15.2:0/2145092638 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6f0863b80 con 0x55c6f0977a60 2024-09-10T08:12:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.573+0000 7f94cbe00640 1 --2- 172.21.15.2:0/2145092638 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c6f0977a60 0x55c6f0979e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.573+0000 7f94cbe00640 1 --2- 172.21.15.2:0/2145092638 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c6f0977a60 0x55c6f0979e80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a70bbd394b54b79 server_cookie=650a33cc6d764e80 in_seq=0 out_seq=0 2024-09-10T08:12:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.573+0000 7f94b7400640 10 client.26716 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.585+0000 7f94b7400640 1 -- 172.21.15.2:0/2145092638 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6f0863b80 con 0x55c6f0977a60 2024-09-10T08:12:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.585+0000 7f94b7400640 10 client.26716 handle_command_reply: tid=0 2024-09-10T08:12:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.585+0000 7f94dbc21740 1 client.26716 shutdown 2024-09-10T08:12:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.586+0000 7f94dbc21740 1 -- 172.21.15.2:0/2145092638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 msgr2=0x55c6f0905760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.586+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2145092638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f0905760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f94cc028ab0 tx=0x7f94cc009b70 comp rx=0 tx=0).stop 2024-09-10T08:12:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.586+0000 7f94dbc21740 1 -- 172.21.15.2:0/2145092638 shutdown_connections 2024-09-10T08:12:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.586+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2145092638 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c6f0977a60 0x55c6f0979e80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.586+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2145092638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6f08fcc30 0x55c6f09759a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.586+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2145092638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6f08f7ba0 0x55c6f0905ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.586+0000 7f94b7400640 0 client.26716 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.586+0000 7f94dbc21740 1 --2- 172.21.15.2:0/2145092638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6f08f7260 0x55c6f0905760 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.586+0000 7f94dbc21740 1 -- 172.21.15.2:0/2145092638 >> 172.21.15.2:0/2145092638 conn(0x55c6f08f28d0 msgr2=0x55c6f09017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.586+0000 7f94dbc21740 1 -- 172.21.15.2:0/2145092638 shutdown_connections 2024-09-10T08:12:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.586+0000 7f94dbc21740 1 -- 172.21.15.2:0/2145092638 wait complete. 2024-09-10T08:12:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.586+0000 7f94dbc21740 20 client.26716 trim_cache size 0 max 16384 2024-09-10T08:12:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:12:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:12:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:12:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:12:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.587+0000 7f94d3200640 1 -- 172.21.15.2:0/1392892163 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94b80a2b80 msgr2=0x7f94b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.587+0000 7f94d3200640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94b80a2b80 0x7f94b80a5040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f94c40a43e0 tx=0x7f94c40d5550 comp rx=0 tx=0).stop 2024-09-10T08:12:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.587+0000 7f94d3200640 1 -- 172.21.15.2:0/1392892163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d4101580 msgr2=0x7f94d4199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.587+0000 7f94d3200640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d4101580 0x7f94d4199ff0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f94c400e9c0 tx=0x7f94c400ee80 comp rx=0 tx=0).stop 2024-09-10T08:12:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.587+0000 7f94d3200640 1 -- 172.21.15.2:0/1392892163 shutdown_connections 2024-09-10T08:12:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.587+0000 7f94d3200640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94b80a2b80 0x7f94b80a5040 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.587+0000 7f94d3200640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94d4103050 0x7f94d41a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.587+0000 7f94d3200640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4101f40 0x7f94d419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.587+0000 7f94d3200640 1 --2- 172.21.15.2:0/1392892163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d4101580 0x7f94d4199ff0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.587+0000 7f94d3200640 1 -- 172.21.15.2:0/1392892163 >> 172.21.15.2:0/1392892163 conn(0x7f94d40fd0d0 msgr2=0x7f94d4108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.587+0000 7f94d3200640 1 -- 172.21.15.2:0/1392892163 shutdown_connections 2024-09-10T08:12:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:12:53.587+0000 7f94d3200640 1 -- 172.21.15.2:0/1392892163 wait complete. 2024-09-10T08:12:53.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.593+0000 7f7d2fc00640 1 -- 172.21.15.2:0/3751866280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28102d30 msgr2=0x7f7d28103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.593+0000 7f7d2fc00640 1 --2- 172.21.15.2:0/3751866280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28102d30 0x7f7d28103110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009860 tx=0x7f7d1c02a3d0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.593+0000 7f7d2fc00640 1 -- 172.21.15.2:0/3751866280 shutdown_connections 2024-09-10T08:12:53.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.593+0000 7f7d2fc00640 1 --2- 172.21.15.2:0/3751866280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d2810c160 0x7f7d2810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.593+0000 7f7d2fc00640 1 --2- 172.21.15.2:0/3751866280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d28103650 0x7f7d2810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.593+0000 7f7d2fc00640 1 --2- 172.21.15.2:0/3751866280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28102d30 0x7f7d28103110 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.593+0000 7f7d2fc00640 1 -- 172.21.15.2:0/3751866280 >> 172.21.15.2:0/3751866280 conn(0x7f7d280fc7d0 msgr2=0x7f7d280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:53.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.593+0000 7f7d2fc00640 1 -- 172.21.15.2:0/3751866280 shutdown_connections 2024-09-10T08:12:53.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.593+0000 7f7d2fc00640 1 -- 172.21.15.2:0/3751866280 wait complete. 2024-09-10T08:12:53.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.595+0000 7f7d2fc00640 1 Processor -- start 2024-09-10T08:12:53.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.595+0000 7f7d2fc00640 1 -- start start 2024-09-10T08:12:53.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.595+0000 7f7d2fc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28102d30 0x7f7d2819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.595+0000 7f7d2fc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d28103650 0x7f7d2819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.595+0000 7f7d2fc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d2810c160 0x7f7d2819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.595+0000 7f7d2fc00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7d28111fb0 con 0x7f7d2810c160 2024-09-10T08:12:53.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.595+0000 7f7d2fc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7d28111e30 con 0x7f7d28103650 2024-09-10T08:12:53.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.595+0000 7f7d2fc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d28112130 con 0x7f7d28102d30 2024-09-10T08:12:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.596+0000 7f7d27e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d28103650 0x7f7d2819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.596+0000 7f7d2d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28102d30 0x7f7d2819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.596+0000 7f7d27e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d28103650 0x7f7d2819ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:45988/0 (socket says 172.21.15.2:45988) 2024-09-10T08:12:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.596+0000 7f7d2d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28102d30 0x7f7d2819a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60080/0 (socket says 172.21.15.2:60080) 2024-09-10T08:12:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.596+0000 7f7d27e00640 1 -- 172.21.15.2:0/2764322936 learned_addr learned my addr 172.21.15.2:0/2764322936 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.596+0000 7f7d2dc00640 1 --2- 172.21.15.2:0/2764322936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d2810c160 0x7f7d2819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.596+0000 7f7d27e00640 1 -- 172.21.15.2:0/2764322936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28102d30 msgr2=0x7f7d2819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.596+0000 7f7d27e00640 1 --2- 172.21.15.2:0/2764322936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28102d30 0x7f7d2819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.596+0000 7f7d27e00640 1 -- 172.21.15.2:0/2764322936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d2810c160 msgr2=0x7f7d2819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.596+0000 7f7d27e00640 1 --2- 172.21.15.2:0/2764322936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d2810c160 0x7f7d2819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.596+0000 7f7d27e00640 1 -- 172.21.15.2:0/2764322936 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2819f680 con 0x7f7d28103650 2024-09-10T08:12:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.597+0000 7f7d27e00640 1 --2- 172.21.15.2:0/2764322936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d28103650 0x7f7d2819ac10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400e910 tx=0x7f7d1400edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.597+0000 7f7d25600640 1 -- 172.21.15.2:0/2764322936 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1400cd80 con 0x7f7d28103650 2024-09-10T08:12:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.597+0000 7f7d25600640 1 -- 172.21.15.2:0/2764322936 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d140035f0 con 0x7f7d28103650 2024-09-10T08:12:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.597+0000 7f7d2fc00640 1 -- 172.21.15.2:0/2764322936 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2819f970 con 0x7f7d28103650 2024-09-10T08:12:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.597+0000 7f7d25600640 1 -- 172.21.15.2:0/2764322936 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14010670 con 0x7f7d28103650 2024-09-10T08:12:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.597+0000 7f7d2fc00640 1 -- 172.21.15.2:0/2764322936 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d281a7250 con 0x7f7d28103650 2024-09-10T08:12:53.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.600+0000 7f7d3264a740 1 Processor -- start 2024-09-10T08:12:53.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.600+0000 7f7d3264a740 1 -- start start 2024-09-10T08:12:53.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.601+0000 7f7d25600640 1 -- 172.21.15.2:0/2764322936 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7d140037a0 con 0x7f7d28103650 2024-09-10T08:12:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.601+0000 7f7d3264a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684a6ac0 0x55cd684a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.601+0000 7f7d3264a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a73e0 0x55cd684b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.601+0000 7f7d3264a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684b1ca0 0x55cd684b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.601+0000 7f7d3264a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cd6840d290 con 0x55cd684b1ca0 2024-09-10T08:12:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.601+0000 7f7d3264a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cd683b4b40 con 0x55cd684a73e0 2024-09-10T08:12:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.601+0000 7f7d3264a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cd68409250 con 0x55cd684a6ac0 2024-09-10T08:12:53.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.601+0000 7f7d27e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684b1ca0 0x55cd684b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.601+0000 7f7d2d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a73e0 0x55cd684b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.601+0000 7f7d27e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684b1ca0 0x55cd684b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34666/0 (socket says 172.21.15.2:34666) 2024-09-10T08:12:53.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.601+0000 7f7d27e00640 1 -- 172.21.15.2:0/587741931 learned_addr learned my addr 172.21.15.2:0/587741931 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:53.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.601+0000 7f7d2d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a73e0 0x55cd684b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:45990/0 (socket says 172.21.15.2:45990) 2024-09-10T08:12:53.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.601+0000 7f7d2dc00640 1 --2- 172.21.15.2:0/587741931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684a6ac0 0x55cd684a6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.602+0000 7f7d27e00640 1 -- 172.21.15.2:0/587741931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684a6ac0 msgr2=0x55cd684a6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.602+0000 7f7d27e00640 1 --2- 172.21.15.2:0/587741931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684a6ac0 0x55cd684a6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.602+0000 7f7d27e00640 1 -- 172.21.15.2:0/587741931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a73e0 msgr2=0x55cd684b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.602+0000 7f7d27e00640 1 --2- 172.21.15.2:0/587741931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a73e0 0x55cd684b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.602+0000 7f7d27e00640 1 -- 172.21.15.2:0/587741931 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd6817e6a0 con 0x55cd684b1ca0 2024-09-10T08:12:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.602+0000 7f7d27e00640 1 --2- 172.21.15.2:0/587741931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684b1ca0 0x55cd684b4090 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7d140d5a80 tx=0x7f7d140d5f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6621c5cece398bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.602+0000 7f7d25600640 1 --2- 172.21.15.2:0/2764322936 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d100a2c00 0x7f7d100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.603+0000 7f7d25600640 1 -- 172.21.15.2:0/2764322936 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7d140cf650 con 0x7f7d28103650 2024-09-10T08:12:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.603+0000 7f7d2dc00640 1 --2- 172.21.15.2:0/2764322936 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d100a2c00 0x7f7d100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.603+0000 7f7d1a000640 1 -- 172.21.15.2:0/587741931 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d140de070 con 0x55cd684b1ca0 2024-09-10T08:12:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.603+0000 7f7d1a000640 1 -- 172.21.15.2:0/587741931 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1409c1c0 con 0x55cd684b1ca0 2024-09-10T08:12:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.603+0000 7f7d1a000640 1 -- 172.21.15.2:0/587741931 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1409b420 con 0x55cd684b1ca0 2024-09-10T08:12:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.604+0000 7f7d3264a740 1 -- 172.21.15.2:0/587741931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684b1ca0 msgr2=0x55cd684b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.604+0000 7f7d3264a740 1 --2- 172.21.15.2:0/587741931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684b1ca0 0x55cd684b4090 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7d140d5a80 tx=0x7f7d140d5f40 comp rx=0 tx=0).stop 2024-09-10T08:12:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.604+0000 7f7d2dc00640 1 --2- 172.21.15.2:0/2764322936 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d100a2c00 0x7f7d100a50c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7d20005fd0 tx=0x7f7d20009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.604+0000 7f7d3264a740 1 -- 172.21.15.2:0/587741931 shutdown_connections 2024-09-10T08:12:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.604+0000 7f7d3264a740 1 --2- 172.21.15.2:0/587741931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684b1ca0 0x55cd684b4090 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.604+0000 7f7d3264a740 1 --2- 172.21.15.2:0/587741931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a73e0 0x55cd684b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.604+0000 7f7d3264a740 1 --2- 172.21.15.2:0/587741931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684a6ac0 0x55cd684a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.604+0000 7f7d3264a740 1 -- 172.21.15.2:0/587741931 >> 172.21.15.2:0/587741931 conn(0x55cd684a3050 msgr2=0x55cd684a3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.604+0000 7f7d3264a740 1 -- 172.21.15.2:0/587741931 shutdown_connections 2024-09-10T08:12:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.604+0000 7f7d3264a740 1 -- 172.21.15.2:0/587741931 wait complete. 2024-09-10T08:12:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.605+0000 7f7d3264a740 1 -- start start 2024-09-10T08:12:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.605+0000 7f7d3264a740 1 Processor -- start 2024-09-10T08:12:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.605+0000 7f7d3264a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684a6ac0 0x55cd684b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.605+0000 7f7d3264a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a73e0 0x55cd684ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.605+0000 7f7d3264a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684b1ca0 0x55cd684aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.605+0000 7f7d3264a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cd684b4910 con 0x55cd684a73e0 2024-09-10T08:12:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.605+0000 7f7d3264a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cd683c7fa0 con 0x55cd684b1ca0 2024-09-10T08:12:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.605+0000 7f7d3264a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cd684b4a90 con 0x55cd684a6ac0 2024-09-10T08:12:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.606+0000 7f7d3264a740 10 client.0 _ll_register_callbacks cb 0x55cd6811bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.606+0000 7f7d2d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684a6ac0 0x55cd684b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.606+0000 7f7d27e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a73e0 0x55cd684ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.606+0000 7f7d2d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684a6ac0 0x55cd684b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60098/0 (socket says 172.21.15.2:60098) 2024-09-10T08:12:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.606+0000 7f7d2d200640 1 -- 172.21.15.2:0/779019843 learned_addr learned my addr 172.21.15.2:0/779019843 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.606+0000 7f7d2dc00640 1 --2- 172.21.15.2:0/779019843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684b1ca0 0x55cd684aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.606+0000 7f7d2d200640 1 -- 172.21.15.2:0/779019843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684b1ca0 msgr2=0x55cd684aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.606+0000 7f7d2d200640 1 --2- 172.21.15.2:0/779019843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684b1ca0 0x55cd684aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.606+0000 7f7d2d200640 1 -- 172.21.15.2:0/779019843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a73e0 msgr2=0x55cd684ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.606+0000 7f7d2d200640 1 --2- 172.21.15.2:0/779019843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a73e0 0x55cd684ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.606+0000 7f7d2d200640 1 -- 172.21.15.2:0/779019843 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd684af690 con 0x55cd684a6ac0 2024-09-10T08:12:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.606+0000 7f7d27e00640 1 --2- 172.21.15.2:0/779019843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a73e0 0x55cd684ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:12:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.607+0000 7f7d2d200640 1 --2- 172.21.15.2:0/779019843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684a6ac0 0x55cd684b1630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009860 tx=0x7f7d1c0305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb66eb69d63a180f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.607+0000 7f7d0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.607+0000 7f7d0ea00640 1 -- 172.21.15.2:0/779019843 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c004050 con 0x55cd684a6ac0 2024-09-10T08:12:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.607+0000 7f7d0ea00640 1 -- 172.21.15.2:0/779019843 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7d1c0041f0 con 0x55cd684a6ac0 2024-09-10T08:12:53.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.607+0000 7f7d3264a740 1 -- 172.21.15.2:0/779019843 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d1409c1c0 con 0x55cd684a6ac0 2024-09-10T08:12:53.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.607+0000 7f7d0ea00640 1 -- 172.21.15.2:0/779019843 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d1c04f050 con 0x55cd684a6ac0 2024-09-10T08:12:53.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.607+0000 7f7d3264a740 10 client.26728 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:53.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.607+0000 7f7d3264a740 1 -- 172.21.15.2:0/779019843 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cd684c0890 con 0x55cd684a6ac0 2024-09-10T08:12:53.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.608+0000 7f7d0ea00640 1 -- 172.21.15.2:0/779019843 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1409c1c0 con 0x55cd684a6ac0 2024-09-10T08:12:53.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.608+0000 7f7d0ea00640 1 -- 172.21.15.2:0/779019843 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7d1c004930 con 0x55cd684a6ac0 2024-09-10T08:12:53.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.608+0000 7f7d3264a740 10 client.26728 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:53.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.608+0000 7f7d3264a740 10 client.26728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:53.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.608+0000 7f7d3264a740 20 client.26728 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.608+0000 7f7d3264a740 1 --2- 172.21.15.2:0/779019843 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd684c1d50 0x55cd684c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.608+0000 7f7d3264a740 4 client.26728 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.608+0000 7f7d3264a740 1 -- 172.21.15.2:0/779019843 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd6841a800 con 0x55cd684c1d50 2024-09-10T08:12:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.609+0000 7f7d27e00640 1 --2- 172.21.15.2:0/779019843 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd684c1d50 0x55cd684c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.609+0000 7f7d27e00640 1 --2- 172.21.15.2:0/779019843 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd684c1d50 0x55cd684c41b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af13b5de31c5ecf2 server_cookie=36aaf265c774fedc in_seq=0 out_seq=0 2024-09-10T08:12:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.609+0000 7f7d0ea00640 10 client.26728 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d0ea00640 1 -- 172.21.15.2:0/779019843 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd6841a800 con 0x55cd684c1d50 2024-09-10T08:12:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d0ea00640 10 client.26728 handle_command_reply: tid=0 2024-09-10T08:12:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d3264a740 1 client.26728 shutdown 2024-09-10T08:12:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d3264a740 1 -- 172.21.15.2:0/779019843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684a6ac0 msgr2=0x55cd684b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d3264a740 1 --2- 172.21.15.2:0/779019843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684a6ac0 0x55cd684b1630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009860 tx=0x7f7d1c0305d0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d3264a740 1 -- 172.21.15.2:0/779019843 shutdown_connections 2024-09-10T08:12:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d3264a740 1 --2- 172.21.15.2:0/779019843 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd684c1d50 0x55cd684c41b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d3264a740 1 --2- 172.21.15.2:0/779019843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684b1ca0 0x55cd684aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d3264a740 1 --2- 172.21.15.2:0/779019843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a73e0 0x55cd684ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d3264a740 1 --2- 172.21.15.2:0/779019843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684a6ac0 0x55cd684b1630 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d3264a740 1 -- 172.21.15.2:0/779019843 >> 172.21.15.2:0/779019843 conn(0x55cd684a3050 msgr2=0x55cd684a8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d0ea00640 0 client.26728 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d3264a740 1 -- 172.21.15.2:0/779019843 shutdown_connections 2024-09-10T08:12:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.610+0000 7f7d3264a740 1 -- 172.21.15.2:0/779019843 wait complete. 2024-09-10T08:12:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.611+0000 7f7d3264a740 20 client.26728 trim_cache size 0 max 16384 2024-09-10T08:12:53.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.629+0000 7f7d3264a740 1 Processor -- start 2024-09-10T08:12:53.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.629+0000 7f7d3264a740 1 -- start start 2024-09-10T08:12:53.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.629+0000 7f7d3264a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a6ac0 0x55cd684a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.629+0000 7f7d3264a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a73e0 0x55cd684c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.629+0000 7f7d3264a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684b1ca0 0x55cd684ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.629+0000 7f7d3264a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281b3a30 con 0x55cd684a6ac0 2024-09-10T08:12:53.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.629+0000 7f7d3264a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281b1300 con 0x55cd684a73e0 2024-09-10T08:12:53.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.629+0000 7f7d3264a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281af150 con 0x55cd684b1ca0 2024-09-10T08:12:53.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.630+0000 7f7d2d200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a6ac0 0x55cd684a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.630+0000 7f7d2d200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a6ac0 0x55cd684a9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34690/0 (socket says 172.21.15.2:34690) 2024-09-10T08:12:53.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.630+0000 7f7d2d200640 1 -- 172.21.15.2:0/4054158556 learned_addr learned my addr 172.21.15.2:0/4054158556 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:53.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.630+0000 7f7d27e00640 1 --2- 172.21.15.2:0/4054158556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a73e0 0x55cd684c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.630+0000 7f7d2dc00640 1 --2- 172.21.15.2:0/4054158556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684b1ca0 0x55cd684ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.630+0000 7f7d2d200640 1 -- 172.21.15.2:0/4054158556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684b1ca0 msgr2=0x55cd684ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.630+0000 7f7d2d200640 1 --2- 172.21.15.2:0/4054158556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684b1ca0 0x55cd684ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.630+0000 7f7d2d200640 1 -- 172.21.15.2:0/4054158556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a73e0 msgr2=0x55cd684c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.630+0000 7f7d2d200640 1 --2- 172.21.15.2:0/4054158556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a73e0 0x55cd684c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.630+0000 7f7d2d200640 1 -- 172.21.15.2:0/4054158556 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd684a4130 con 0x55cd684a6ac0 2024-09-10T08:12:53.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.630+0000 7f7d27e00640 1 --2- 172.21.15.2:0/4054158556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a73e0 0x55cd684c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:53.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.630+0000 7f7d2d200640 1 --2- 172.21.15.2:0/4054158556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a6ac0 0x55cd684a9dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c04d040 tx=0x7f7d1c003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c280268d4cc8a3c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:53.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.631+0000 7f7d19600640 1 -- 172.21.15.2:0/4054158556 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c058070 con 0x55cd684a6ac0 2024-09-10T08:12:53.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.631+0000 7f7d19600640 1 -- 172.21.15.2:0/4054158556 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1c004f20 con 0x55cd684a6ac0 2024-09-10T08:12:53.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.631+0000 7f7d19600640 1 -- 172.21.15.2:0/4054158556 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c053670 con 0x55cd684a6ac0 2024-09-10T08:12:53.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.631+0000 7f7d3264a740 1 -- 172.21.15.2:0/4054158556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a6ac0 msgr2=0x55cd684a9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.631+0000 7f7d3264a740 1 --2- 172.21.15.2:0/4054158556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a6ac0 0x55cd684a9dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c04d040 tx=0x7f7d1c003780 comp rx=0 tx=0).stop 2024-09-10T08:12:53.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.631+0000 7f7d3264a740 1 -- 172.21.15.2:0/4054158556 shutdown_connections 2024-09-10T08:12:53.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.631+0000 7f7d3264a740 1 --2- 172.21.15.2:0/4054158556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684b1ca0 0x55cd684ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.631+0000 7f7d3264a740 1 --2- 172.21.15.2:0/4054158556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a73e0 0x55cd684c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.631+0000 7f7d3264a740 1 --2- 172.21.15.2:0/4054158556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a6ac0 0x55cd684a9dc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.631+0000 7f7d3264a740 1 -- 172.21.15.2:0/4054158556 >> 172.21.15.2:0/4054158556 conn(0x55cd684a3050 msgr2=0x55cd684a96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:53.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.631+0000 7f7d3264a740 1 -- 172.21.15.2:0/4054158556 shutdown_connections 2024-09-10T08:12:53.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.632+0000 7f7d3264a740 1 -- 172.21.15.2:0/4054158556 wait complete. 2024-09-10T08:12:53.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.632+0000 7f7d3264a740 1 -- start start 2024-09-10T08:12:53.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.632+0000 7f7d3264a740 1 Processor -- start 2024-09-10T08:12:53.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.632+0000 7f7d3264a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a6ac0 0x55cd684b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.632+0000 7f7d3264a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a73e0 0x55cd684b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.632+0000 7f7d3264a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684b1ca0 0x55cd684b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.632+0000 7f7d3264a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281af7c0 con 0x55cd684a73e0 2024-09-10T08:12:53.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.632+0000 7f7d3264a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281b0c30 con 0x55cd684a6ac0 2024-09-10T08:12:53.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.632+0000 7f7d3264a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281afe90 con 0x55cd684b1ca0 2024-09-10T08:12:53.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.633+0000 7f7d3264a740 10 client.0 _ll_register_callbacks cb 0x7f7d1c0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:53.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.633+0000 7f7d2d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a6ac0 0x55cd684b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.633+0000 7f7d27e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a73e0 0x55cd684b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.633+0000 7f7d2d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a6ac0 0x55cd684b5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46014/0 (socket says 172.21.15.2:46014) 2024-09-10T08:12:53.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.633+0000 7f7d2d200640 1 -- 172.21.15.2:0/1914026594 learned_addr learned my addr 172.21.15.2:0/1914026594 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:53.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.633+0000 7f7d2dc00640 1 --2- 172.21.15.2:0/1914026594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684b1ca0 0x55cd684b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.633+0000 7f7d2d200640 1 -- 172.21.15.2:0/1914026594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684b1ca0 msgr2=0x55cd684b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.633+0000 7f7d2d200640 1 --2- 172.21.15.2:0/1914026594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684b1ca0 0x55cd684b67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.633+0000 7f7d2d200640 1 -- 172.21.15.2:0/1914026594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a73e0 msgr2=0x55cd684b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:53.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.633+0000 7f7d2d200640 1 --2- 172.21.15.2:0/1914026594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a73e0 0x55cd684b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.633+0000 7f7d2d200640 1 -- 172.21.15.2:0/1914026594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd6852bcf0 con 0x55cd684a6ac0 2024-09-10T08:12:53.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.633+0000 7f7d2d200640 1 --2- 172.21.15.2:0/1914026594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a6ac0 0x55cd684b5d20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c004ea0 tx=0x7f7d1c0048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=def8e8ed5328525b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:53.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.633+0000 7f7d0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:12:53.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.634+0000 7f7d0ea00640 1 -- 172.21.15.2:0/1914026594 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c058070 con 0x55cd684a6ac0 2024-09-10T08:12:53.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.634+0000 7f7d0ea00640 1 -- 172.21.15.2:0/1914026594 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7d1c004ae0 con 0x55cd684a6ac0 2024-09-10T08:12:53.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.634+0000 7f7d3264a740 1 -- 172.21.15.2:0/1914026594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d1c004f20 con 0x55cd684a6ac0 2024-09-10T08:12:53.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.634+0000 7f7d0ea00640 1 -- 172.21.15.2:0/1914026594 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d1c0052e0 con 0x55cd684a6ac0 2024-09-10T08:12:53.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.634+0000 7f7d3264a740 10 client.26774 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:53.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.634+0000 7f7d3264a740 1 -- 172.21.15.2:0/1914026594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cd6852d030 con 0x55cd684a6ac0 2024-09-10T08:12:53.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.634+0000 7f7d0ea00640 1 -- 172.21.15.2:0/1914026594 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1c004f20 con 0x55cd684a6ac0 2024-09-10T08:12:53.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.634+0000 7f7d0ea00640 1 -- 172.21.15.2:0/1914026594 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7d1c053b20 con 0x55cd684a6ac0 2024-09-10T08:12:53.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.634+0000 7f7d3264a740 10 client.26774 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:53.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.634+0000 7f7d3264a740 10 client.26774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:53.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.635+0000 7f7d3264a740 20 client.26774 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:53.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.635+0000 7f7d3264a740 1 --2- 172.21.15.2:0/1914026594 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd6852d630 0x55cd6852fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:53.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.635+0000 7f7d3264a740 4 client.26774 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:12:53.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.635+0000 7f7d3264a740 1 -- 172.21.15.2:0/1914026594 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cd68414d10 con 0x55cd6852d630 2024-09-10T08:12:53.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.635+0000 7f7d27e00640 1 --2- 172.21.15.2:0/1914026594 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd6852d630 0x55cd6852fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:53.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.635+0000 7f7d27e00640 1 --2- 172.21.15.2:0/1914026594 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd6852d630 0x55cd6852fa40 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cf7ed5d7e22375d server_cookie=2b455550fcb6dff in_seq=0 out_seq=0 2024-09-10T08:12:53.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.635+0000 7f7d0ea00640 10 client.26774 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:53.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.636+0000 7f7d0ea00640 1 -- 172.21.15.2:0/1914026594 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cd68414d10 con 0x55cd6852d630 2024-09-10T08:12:53.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.636+0000 7f7d0ea00640 10 client.26774 handle_command_reply: tid=0 2024-09-10T08:12:53.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.636+0000 7f7d3264a740 1 client.26774 shutdown 2024-09-10T08:12:53.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.637+0000 7f7d3264a740 1 -- 172.21.15.2:0/1914026594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a6ac0 msgr2=0x55cd684b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.637+0000 7f7d3264a740 1 --2- 172.21.15.2:0/1914026594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a6ac0 0x55cd684b5d20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c004ea0 tx=0x7f7d1c0048f0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.637+0000 7f7d3264a740 1 -- 172.21.15.2:0/1914026594 shutdown_connections 2024-09-10T08:12:53.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.637+0000 7f7d3264a740 1 --2- 172.21.15.2:0/1914026594 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd6852d630 0x55cd6852fa40 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.637+0000 7f7d3264a740 1 --2- 172.21.15.2:0/1914026594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd684b1ca0 0x55cd684b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.637+0000 7f7d3264a740 1 --2- 172.21.15.2:0/1914026594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd684a73e0 0x55cd684b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.637+0000 7f7d0ea00640 0 client.26774 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:53.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.637+0000 7f7d3264a740 1 --2- 172.21.15.2:0/1914026594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd684a6ac0 0x55cd684b5d20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.637+0000 7f7d3264a740 1 -- 172.21.15.2:0/1914026594 >> 172.21.15.2:0/1914026594 conn(0x55cd684a3050 msgr2=0x55cd684a21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:53.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.637+0000 7f7d3264a740 1 -- 172.21.15.2:0/1914026594 shutdown_connections 2024-09-10T08:12:53.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.637+0000 7f7d3264a740 1 -- 172.21.15.2:0/1914026594 wait complete. 2024-09-10T08:12:53.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.637+0000 7f7d3264a740 20 client.26774 trim_cache size 0 max 16384 2024-09-10T08:12:53.643 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:12:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:12:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "5753f598-fa05-4827-bb77-d4073b04b243", 2024-09-10T08:12:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:12:53.643 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:12:53.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.638+0000 7f7d2fc00640 1 -- 172.21.15.2:0/2764322936 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d100a2c00 msgr2=0x7f7d100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.638+0000 7f7d2fc00640 1 --2- 172.21.15.2:0/2764322936 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d100a2c00 0x7f7d100a50c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7d20005fd0 tx=0x7f7d20009500 comp rx=0 tx=0).stop 2024-09-10T08:12:53.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.638+0000 7f7d2fc00640 1 -- 172.21.15.2:0/2764322936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d28103650 msgr2=0x7f7d2819ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:53.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.638+0000 7f7d2fc00640 1 --2- 172.21.15.2:0/2764322936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d28103650 0x7f7d2819ac10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400e910 tx=0x7f7d1400edd0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.638+0000 7f7d2fc00640 1 -- 172.21.15.2:0/2764322936 shutdown_connections 2024-09-10T08:12:53.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.638+0000 7f7d2fc00640 1 --2- 172.21.15.2:0/2764322936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d2810c160 0x7f7d2819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.638+0000 7f7d2fc00640 1 --2- 172.21.15.2:0/2764322936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d28103650 0x7f7d2819ac10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.638+0000 7f7d2fc00640 1 --2- 172.21.15.2:0/2764322936 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d100a2c00 0x7f7d100a50c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.638+0000 7f7d2fc00640 1 --2- 172.21.15.2:0/2764322936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d28102d30 0x7f7d2819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:53.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.638+0000 7f7d2fc00640 1 -- 172.21.15.2:0/2764322936 >> 172.21.15.2:0/2764322936 conn(0x7f7d280fc7d0 msgr2=0x7f7d2810d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:53.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.638+0000 7f7d2fc00640 1 -- 172.21.15.2:0/2764322936 shutdown_connections 2024-09-10T08:12:53.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:53.638+0000 7f7d2fc00640 1 -- 172.21.15.2:0/2764322936 wait complete. 2024-09-10T08:12:53.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:53 smithi002 ceph-mon[30563]: pgmap v529: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-10T08:12:53.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:53 smithi155 ceph-mon[35405]: pgmap v529: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-10T08:12:53.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:53 smithi107 ceph-mon[35048]: pgmap v529: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-10T08:12:54.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:54 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:12:54.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:54 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:54.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:54 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:12:54.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:54 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:54.823 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:12:54.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:54 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:12:54.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:54 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:54.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:54 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:12:54.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:54 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:54.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:54 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:12:54.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:54 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:54.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:54 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:12:54.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:54 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:12:55.248 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:12:55.749 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:55 smithi002 ceph-mon[30563]: pgmap v530: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-10T08:12:55.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.865+0000 7fa8d2a00640 1 -- 172.21.15.2:0/3883381804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8cc074e30 msgr2=0x7fa8cc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:55.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.865+0000 7fa8d2a00640 1 --2- 172.21.15.2:0/3883381804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8cc074e30 0x7fa8cc10dd40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009900 tx=0x7fa8b802a480 comp rx=0 tx=0).stop 2024-09-10T08:12:55.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.865+0000 7fa8d2a00640 1 -- 172.21.15.2:0/3883381804 shutdown_connections 2024-09-10T08:12:55.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.865+0000 7fa8d2a00640 1 --2- 172.21.15.2:0/3883381804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8cc10e280 0x7fa8cc110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.865+0000 7fa8d2a00640 1 --2- 172.21.15.2:0/3883381804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8cc074e30 0x7fa8cc10dd40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.865+0000 7fa8d2a00640 1 --2- 172.21.15.2:0/3883381804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc074510 0x7fa8cc0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.865+0000 7fa8d2a00640 1 -- 172.21.15.2:0/3883381804 >> 172.21.15.2:0/3883381804 conn(0x7fa8cc0fd740 msgr2=0x7fa8cc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:55.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.865+0000 7fa8d2a00640 1 -- 172.21.15.2:0/3883381804 shutdown_connections 2024-09-10T08:12:55.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.865+0000 7fa8d2a00640 1 -- 172.21.15.2:0/3883381804 wait complete. 2024-09-10T08:12:55.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.866+0000 7fa8d2a00640 1 Processor -- start 2024-09-10T08:12:55.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.866+0000 7fa8d2a00640 1 -- start start 2024-09-10T08:12:55.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.867+0000 7fa8d2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8cc074510 0x7fa8cc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.867+0000 7fa8d2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8cc074e30 0x7fa8cc19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.867+0000 7fa8d2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc10e280 0x7fa8cc19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.867+0000 7fa8d2a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc113fb0 con 0x7fa8cc074510 2024-09-10T08:12:55.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.867+0000 7fa8d2a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc113e30 con 0x7fa8cc10e280 2024-09-10T08:12:55.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.867+0000 7fa8d2a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc114130 con 0x7fa8cc074e30 2024-09-10T08:12:55.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.867+0000 7fa8d1600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8cc074510 0x7fa8cc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.867+0000 7fa8d1600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8cc074510 0x7fa8cc19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34704/0 (socket says 172.21.15.2:34704) 2024-09-10T08:12:55.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.867+0000 7fa8d0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8cc074e30 0x7fa8cc19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.867+0000 7fa8d1600640 1 -- 172.21.15.2:0/4093298897 learned_addr learned my addr 172.21.15.2:0/4093298897 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:55.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.867+0000 7fa8d2000640 1 --2- 172.21.15.2:0/4093298897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc10e280 0x7fa8cc19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.867+0000 7fa8d1600640 1 -- 172.21.15.2:0/4093298897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8cc074e30 msgr2=0x7fa8cc19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:55.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.867+0000 7fa8d1600640 1 --2- 172.21.15.2:0/4093298897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8cc074e30 0x7fa8cc19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.868+0000 7fa8d1600640 1 -- 172.21.15.2:0/4093298897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc10e280 msgr2=0x7fa8cc19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:55.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.868+0000 7fa8d1600640 1 --2- 172.21.15.2:0/4093298897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc10e280 0x7fa8cc19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.868+0000 7fa8d1600640 1 -- 172.21.15.2:0/4093298897 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc19f4a0 con 0x7fa8cc074510 2024-09-10T08:12:55.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.868+0000 7fa8d1600640 1 --2- 172.21.15.2:0/4093298897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8cc074510 0x7fa8cc19a4f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009d50 tx=0x7fa8c000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:55.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.868+0000 7fa8ca000640 1 -- 172.21.15.2:0/4093298897 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0003b70 con 0x7fa8cc074510 2024-09-10T08:12:55.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.868+0000 7fa8ca000640 1 -- 172.21.15.2:0/4093298897 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c0003d10 con 0x7fa8cc074510 2024-09-10T08:12:55.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.868+0000 7fa8ca000640 1 -- 172.21.15.2:0/4093298897 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0010a20 con 0x7fa8cc074510 2024-09-10T08:12:55.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.868+0000 7fa8d2a00640 1 -- 172.21.15.2:0/4093298897 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc19f790 con 0x7fa8cc074510 2024-09-10T08:12:55.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.868+0000 7fa8d2a00640 1 -- 172.21.15.2:0/4093298897 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc1a7070 con 0x7fa8cc074510 2024-09-10T08:12:55.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.871+0000 7fa8d6a91740 1 Processor -- start 2024-09-10T08:12:55.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.872+0000 7fa8d6a91740 1 -- start start 2024-09-10T08:12:55.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.872+0000 7fa8ca000640 1 -- 172.21.15.2:0/4093298897 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa8c0003ec0 con 0x7fa8cc074510 2024-09-10T08:12:55.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.872+0000 7fa8d6a91740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf94f8d50 0x558cf94f9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.872+0000 7fa8d6a91740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf95039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.872+0000 7fa8d6a91740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf9503f30 0x558cf9506320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.872+0000 7fa8d6a91740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558cf945f3e0 con 0x558cf9503f30 2024-09-10T08:12:55.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.872+0000 7fa8d6a91740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558cf9406d80 con 0x558cf94f8d50 2024-09-10T08:12:55.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.872+0000 7fa8d6a91740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558cf945b3a0 con 0x558cf94f9670 2024-09-10T08:12:55.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.872+0000 7fa8d0c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf9503f30 0x558cf9506320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.872+0000 7fa8d1600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf95039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.872+0000 7fa8d0c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf9503f30 0x558cf9506320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34716/0 (socket says 172.21.15.2:34716) 2024-09-10T08:12:55.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.872+0000 7fa8d0c00640 1 -- 172.21.15.2:0/3381520018 learned_addr learned my addr 172.21.15.2:0/3381520018 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:55.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.872+0000 7fa8d2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf94f8d50 0x558cf94f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.873+0000 7fa8d2000640 1 -- 172.21.15.2:0/3381520018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 msgr2=0x558cf95039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:55.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.873+0000 7fa8d2000640 1 --2- 172.21.15.2:0/3381520018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf95039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.873+0000 7fa8d2000640 1 -- 172.21.15.2:0/3381520018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf9503f30 msgr2=0x558cf9506320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:55.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.873+0000 7fa8d2000640 1 --2- 172.21.15.2:0/3381520018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf9503f30 0x558cf9506320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.873+0000 7fa8d2000640 1 -- 172.21.15.2:0/3381520018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cf9425ae0 con 0x558cf94f8d50 2024-09-10T08:12:55.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.873+0000 7fa8d2000640 1 --2- 172.21.15.2:0/3381520018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf94f8d50 0x558cf94f9130 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009b50 tx=0x7fa8c4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=876d730a2783b559 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:55.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.873+0000 7fa8be600640 1 -- 172.21.15.2:0/3381520018 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4004050 con 0x558cf94f8d50 2024-09-10T08:12:55.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.873+0000 7fa8be600640 1 -- 172.21.15.2:0/3381520018 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c4005500 con 0x558cf94f8d50 2024-09-10T08:12:55.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.873+0000 7fa8be600640 1 -- 172.21.15.2:0/3381520018 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4004c30 con 0x558cf94f8d50 2024-09-10T08:12:55.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.874+0000 7fa8ca000640 1 --2- 172.21.15.2:0/4093298897 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa8b40a2c00 0x7fa8b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.874+0000 7fa8ca000640 1 -- 172.21.15.2:0/4093298897 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa8c00cfac0 con 0x7fa8cc074510 2024-09-10T08:12:55.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.874+0000 7fa8d1600640 1 --2- 172.21.15.2:0/4093298897 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa8b40a2c00 0x7fa8b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.874+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3381520018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf94f8d50 msgr2=0x558cf94f9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:55.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.874+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3381520018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf94f8d50 0x558cf94f9130 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009b50 tx=0x7fa8c4011810 comp rx=0 tx=0).stop 2024-09-10T08:12:55.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.874+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3381520018 shutdown_connections 2024-09-10T08:12:55.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.874+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3381520018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf9503f30 0x558cf9506320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.874+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3381520018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf95039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.874+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3381520018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf94f8d50 0x558cf94f9130 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.874+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3381520018 >> 172.21.15.2:0/3381520018 conn(0x558cf94f52e0 msgr2=0x558cf94f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:55.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.875+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3381520018 shutdown_connections 2024-09-10T08:12:55.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.875+0000 7fa8d1600640 1 --2- 172.21.15.2:0/4093298897 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa8b40a2c00 0x7fa8b40a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00cd170 tx=0x7fa8c00d34a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:55.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.875+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3381520018 wait complete. 2024-09-10T08:12:55.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.875+0000 7fa8d6a91740 1 -- start start 2024-09-10T08:12:55.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.876+0000 7fa8d6a91740 1 Processor -- start 2024-09-10T08:12:55.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.876+0000 7fa8d6a91740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf94f8d50 0x558cf9500e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.876+0000 7fa8d6a91740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf9501380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.876+0000 7fa8d6a91740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf9503f30 0x558cf95018c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.876+0000 7fa8d6a91740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558cf9506ba0 con 0x558cf94f8d50 2024-09-10T08:12:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.876+0000 7fa8d6a91740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558cf941a0f0 con 0x558cf9503f30 2024-09-10T08:12:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.876+0000 7fa8d6a91740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558cf9506d20 con 0x558cf94f9670 2024-09-10T08:12:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.876+0000 7fa8d6a91740 10 client.0 _ll_register_callbacks cb 0x558cf940b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.876+0000 7fa8d0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf9501380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.876+0000 7fa8d0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf9501380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60154/0 (socket says 172.21.15.2:60154) 2024-09-10T08:12:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.876+0000 7fa8d0c00640 1 -- 172.21.15.2:0/3533699435 learned_addr learned my addr 172.21.15.2:0/3533699435 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.876+0000 7fa8d2000640 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf94f8d50 0x558cf9500e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.877+0000 7fa8d2000640 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf9503f30 0x558cf95018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.877+0000 7fa8d0c00640 1 -- 172.21.15.2:0/3533699435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf9503f30 msgr2=0x558cf95018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:55.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.877+0000 7fa8d0c00640 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf9503f30 0x558cf95018c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.877+0000 7fa8d0c00640 1 -- 172.21.15.2:0/3533699435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf94f8d50 msgr2=0x558cf9500e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:55.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.877+0000 7fa8d0c00640 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf94f8d50 0x558cf9500e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.877+0000 7fa8d0c00640 1 -- 172.21.15.2:0/3533699435 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cf91d03c0 con 0x558cf94f9670 2024-09-10T08:12:55.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.877+0000 7fa8d2000640 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf94f8d50 0x558cf9500e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:12:55.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.877+0000 7fa8d0c00640 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf9501380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8033000 tx=0x7fa8b8031850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e91ef6803b5e78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:55.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.877+0000 7fa8b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:55.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.877+0000 7fa8b2a00640 1 -- 172.21.15.2:0/3533699435 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8035040 con 0x558cf94f9670 2024-09-10T08:12:55.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.877+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3533699435 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c4005500 con 0x558cf94f9670 2024-09-10T08:12:55.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.878+0000 7fa8b2a00640 1 -- 172.21.15.2:0/3533699435 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa8b8004460 con 0x558cf94f9670 2024-09-10T08:12:55.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.878+0000 7fa8b2a00640 1 -- 172.21.15.2:0/3533699435 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa8b804e050 con 0x558cf94f9670 2024-09-10T08:12:55.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.878+0000 7fa8d6a91740 10 client.26746 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:55.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.878+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3533699435 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558cf951b0e0 con 0x558cf94f9670 2024-09-10T08:12:55.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.878+0000 7fa8b2a00640 1 -- 172.21.15.2:0/3533699435 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c4005500 con 0x558cf94f9670 2024-09-10T08:12:55.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.878+0000 7fa8b2a00640 1 -- 172.21.15.2:0/3533699435 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa8b8004ce0 con 0x558cf94f9670 2024-09-10T08:12:55.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.878+0000 7fa8d6a91740 10 client.26746 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:55.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.878+0000 7fa8d6a91740 10 client.26746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:55.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.878+0000 7fa8d6a91740 20 client.26746 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:55.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.878+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558cf951c4f0 0x558cf951e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.878+0000 7fa8d6a91740 4 client.26746 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:12:55.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.878+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3533699435 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cf946c950 con 0x558cf951c4f0 2024-09-10T08:12:55.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.879+0000 7fa8d2000640 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558cf951c4f0 0x558cf951e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.879+0000 7fa8d2000640 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558cf951c4f0 0x558cf951e940 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0e841355ca3fa5 server_cookie=465b68f01d207424 in_seq=0 out_seq=0 2024-09-10T08:12:55.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.879+0000 7fa8b2a00640 10 client.26746 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:55.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.880+0000 7fa8b2a00640 1 -- 172.21.15.2:0/3533699435 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558cf946c950 con 0x558cf951c4f0 2024-09-10T08:12:55.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.880+0000 7fa8b2a00640 10 client.26746 handle_command_reply: tid=0 2024-09-10T08:12:55.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.880+0000 7fa8d6a91740 1 client.26746 shutdown 2024-09-10T08:12:55.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.881+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3533699435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 msgr2=0x558cf9501380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:55.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.881+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf9501380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8033000 tx=0x7fa8b8031850 comp rx=0 tx=0).stop 2024-09-10T08:12:55.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.881+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3533699435 shutdown_connections 2024-09-10T08:12:55.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.881+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558cf951c4f0 0x558cf951e940 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.881+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf9503f30 0x558cf95018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.881+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf9501380 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.881+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3533699435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf94f8d50 0x558cf9500e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.881+0000 7fa8b2a00640 0 client.26746 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:55.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.881+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3533699435 >> 172.21.15.2:0/3533699435 conn(0x558cf94f52e0 msgr2=0x558cf94fafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:55.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.881+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3533699435 shutdown_connections 2024-09-10T08:12:55.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.881+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3533699435 wait complete. 2024-09-10T08:12:55.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.881+0000 7fa8d6a91740 20 client.26746 trim_cache size 0 max 16384 2024-09-10T08:12:55.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.899+0000 7fa8d6a91740 1 Processor -- start 2024-09-10T08:12:55.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.899+0000 7fa8d6a91740 1 -- start start 2024-09-10T08:12:55.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.899+0000 7fa8d6a91740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f8d50 0x558cf94fb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d6a91740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf94f9670 0x558cf9524210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d6a91740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf9503f30 0x558cf9524750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d6a91740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1af480 con 0x558cf94f9670 2024-09-10T08:12:55.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d6a91740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1acd50 con 0x558cf9503f30 2024-09-10T08:12:55.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d6a91740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1aaba0 con 0x558cf94f8d50 2024-09-10T08:12:55.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d0c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf94f9670 0x558cf9524210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d0c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf94f9670 0x558cf9524210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34740/0 (socket says 172.21.15.2:34740) 2024-09-10T08:12:55.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d0c00640 1 -- 172.21.15.2:0/1218758511 learned_addr learned my addr 172.21.15.2:0/1218758511 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:55.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d2000640 1 --2- 172.21.15.2:0/1218758511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f8d50 0x558cf94fb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d2000640 1 --2- 172.21.15.2:0/1218758511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf9503f30 0x558cf9524750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d0c00640 1 -- 172.21.15.2:0/1218758511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f8d50 msgr2=0x558cf94fb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:55.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d0c00640 1 --2- 172.21.15.2:0/1218758511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f8d50 0x558cf94fb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d0c00640 1 -- 172.21.15.2:0/1218758511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf9503f30 msgr2=0x558cf9524750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:55.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d0c00640 1 --2- 172.21.15.2:0/1218758511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf9503f30 0x558cf9524750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d0c00640 1 -- 172.21.15.2:0/1218758511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cf94f9cc0 con 0x558cf94f9670 2024-09-10T08:12:55.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.900+0000 7fa8d2000640 1 --2- 172.21.15.2:0/1218758511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf9503f30 0x558cf9524750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:12:55.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.901+0000 7fa8d0c00640 1 --2- 172.21.15.2:0/1218758511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf94f9670 0x558cf9524210 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8003690 tx=0x7fa8b802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0599c6174f02cd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.901+0000 7fa8bdc00640 1 -- 172.21.15.2:0/1218758511 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8057070 con 0x558cf94f9670 2024-09-10T08:12:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.901+0000 7fa8bdc00640 1 -- 172.21.15.2:0/1218758511 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b804b070 con 0x558cf94f9670 2024-09-10T08:12:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.901+0000 7fa8bdc00640 1 -- 172.21.15.2:0/1218758511 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8035030 con 0x558cf94f9670 2024-09-10T08:12:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.902+0000 7fa8d6a91740 1 -- 172.21.15.2:0/1218758511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf94f9670 msgr2=0x558cf9524210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.902+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/1218758511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf94f9670 0x558cf9524210 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8003690 tx=0x7fa8b802a480 comp rx=0 tx=0).stop 2024-09-10T08:12:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.902+0000 7fa8d6a91740 1 -- 172.21.15.2:0/1218758511 shutdown_connections 2024-09-10T08:12:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.902+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/1218758511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf9503f30 0x558cf9524750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.902+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/1218758511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf94f9670 0x558cf9524210 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.902+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/1218758511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f8d50 0x558cf94fb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.902+0000 7fa8d6a91740 1 -- 172.21.15.2:0/1218758511 >> 172.21.15.2:0/1218758511 conn(0x558cf94f52e0 msgr2=0x558cf94ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.902+0000 7fa8d6a91740 1 -- 172.21.15.2:0/1218758511 shutdown_connections 2024-09-10T08:12:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.902+0000 7fa8d6a91740 1 -- 172.21.15.2:0/1218758511 wait complete. 2024-09-10T08:12:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.902+0000 7fa8d6a91740 1 -- start start 2024-09-10T08:12:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.903+0000 7fa8d6a91740 1 Processor -- start 2024-09-10T08:12:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.903+0000 7fa8d6a91740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf94f8d50 0x558cf9510790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.903+0000 7fa8d6a91740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf9510cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.903+0000 7fa8d6a91740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf9503f30 0x558cf9511210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.903+0000 7fa8d6a91740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1ab210 con 0x558cf9503f30 2024-09-10T08:12:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.903+0000 7fa8d6a91740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1ac680 con 0x558cf94f8d50 2024-09-10T08:12:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.903+0000 7fa8d6a91740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1ab8e0 con 0x558cf94f9670 2024-09-10T08:12:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.903+0000 7fa8d6a91740 10 client.0 _ll_register_callbacks cb 0x7fa8b8005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:12:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.903+0000 7fa8d0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf9510cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.903+0000 7fa8d2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf94f8d50 0x558cf9510790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.903+0000 7fa8d0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf9510cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60184/0 (socket says 172.21.15.2:60184) 2024-09-10T08:12:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.903+0000 7fa8d0c00640 1 -- 172.21.15.2:0/3544135027 learned_addr learned my addr 172.21.15.2:0/3544135027 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:12:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.903+0000 7fa8d2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf94f8d50 0x558cf9510790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:46072/0 (socket says 172.21.15.2:46072) 2024-09-10T08:12:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.904+0000 7fa8d2000640 1 --2- 172.21.15.2:0/3544135027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf9503f30 0x558cf9511210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.904+0000 7fa8d0c00640 1 -- 172.21.15.2:0/3544135027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf94f8d50 msgr2=0x558cf9510790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.904+0000 7fa8d0c00640 1 --2- 172.21.15.2:0/3544135027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf94f8d50 0x558cf9510790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.904+0000 7fa8d0c00640 1 -- 172.21.15.2:0/3544135027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf9503f30 msgr2=0x558cf9511210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:12:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.904+0000 7fa8d0c00640 1 --2- 172.21.15.2:0/3544135027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf9503f30 0x558cf9511210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.904+0000 7fa8d0c00640 1 -- 172.21.15.2:0/3544135027 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cf9586430 con 0x558cf94f9670 2024-09-10T08:12:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.904+0000 7fa8d0c00640 1 --2- 172.21.15.2:0/3544135027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf9510cd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8006010 tx=0x7fa8b8003760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1b2f2d8b00263ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:12:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.904+0000 7fa8b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:12:55.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:55 smithi155 ceph-mon[35405]: pgmap v530: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-10T08:12:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.904+0000 7fa8b2a00640 1 -- 172.21.15.2:0/3544135027 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8035070 con 0x558cf94f9670 2024-09-10T08:12:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.904+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3544135027 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8b804b070 con 0x558cf94f9670 2024-09-10T08:12:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.905+0000 7fa8b2a00640 1 -- 172.21.15.2:0/3544135027 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa8b8004260 con 0x558cf94f9670 2024-09-10T08:12:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.905+0000 7fa8b2a00640 1 -- 172.21.15.2:0/3544135027 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa8b8057070 con 0x558cf94f9670 2024-09-10T08:12:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.905+0000 7fa8d6a91740 10 client.26752 fetch_fsmap learned FSMap version 26 2024-09-10T08:12:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.905+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3544135027 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558cf9587670 con 0x558cf94f9670 2024-09-10T08:12:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.905+0000 7fa8b2a00640 1 -- 172.21.15.2:0/3544135027 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b804b070 con 0x558cf94f9670 2024-09-10T08:12:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.905+0000 7fa8b2a00640 1 -- 172.21.15.2:0/3544135027 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa8b8004960 con 0x558cf94f9670 2024-09-10T08:12:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.905+0000 7fa8d6a91740 10 client.26752 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:12:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.905+0000 7fa8d6a91740 10 client.26752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:12:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.905+0000 7fa8d6a91740 20 client.26752 populate_metadata read hostname 'smithi002' 2024-09-10T08:12:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.905+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3544135027 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558cf9587d60 0x558cf958a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:12:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.905+0000 7fa8d6a91740 4 client.26752 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:12:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.905+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3544135027 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558cf9466e60 con 0x558cf9587d60 2024-09-10T08:12:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.906+0000 7fa8d2000640 1 --2- 172.21.15.2:0/3544135027 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558cf9587d60 0x558cf958a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:12:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.906+0000 7fa8d2000640 1 --2- 172.21.15.2:0/3544135027 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558cf9587d60 0x558cf958a180 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a303ea8c203faf6 server_cookie=48261187d5fab845 in_seq=0 out_seq=0 2024-09-10T08:12:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.906+0000 7fa8b2a00640 10 client.26752 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.906+0000 7fa8b2a00640 1 -- 172.21.15.2:0/3544135027 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558cf9466e60 con 0x558cf9587d60 2024-09-10T08:12:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.906+0000 7fa8b2a00640 10 client.26752 handle_command_reply: tid=0 2024-09-10T08:12:55.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.906+0000 7fa8d6a91740 1 client.26752 shutdown 2024-09-10T08:12:55.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.907+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3544135027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 msgr2=0x558cf9510cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:55.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.907+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3544135027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf9510cd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8006010 tx=0x7fa8b8003760 comp rx=0 tx=0).stop 2024-09-10T08:12:55.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.907+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3544135027 shutdown_connections 2024-09-10T08:12:55.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.907+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3544135027 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558cf9587d60 0x558cf958a180 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.907+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3544135027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558cf9503f30 0x558cf9511210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.907+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3544135027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cf94f9670 0x558cf9510cd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.907+0000 7fa8d6a91740 1 --2- 172.21.15.2:0/3544135027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558cf94f8d50 0x558cf9510790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.907+0000 7fa8b2a00640 0 client.26752 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:12:55.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.907+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3544135027 >> 172.21.15.2:0/3544135027 conn(0x558cf94f52e0 msgr2=0x558cf9500db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:55.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.907+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3544135027 shutdown_connections 2024-09-10T08:12:55.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.907+0000 7fa8d6a91740 1 -- 172.21.15.2:0/3544135027 wait complete. 2024-09-10T08:12:55.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.907+0000 7fa8d6a91740 20 client.26752 trim_cache size 0 max 16384 2024-09-10T08:12:55.912 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:12:55.912 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:12:55.912 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:12:55.912 INFO:teuthology.orchestra.run.smithi002.stdout: "5753f598-fa05-4827-bb77-d4073b04b243": { 2024-09-10T08:12:55.912 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:12:55.912 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "5753f598-fa05-4827-bb77-d4073b04b243", 2024-09-10T08:12:55.912 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:12:55.912 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:55.912 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:12:55.912 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:12:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.908+0000 7fa8d2a00640 1 -- 172.21.15.2:0/4093298897 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa8b40a2c00 msgr2=0x7fa8b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.908+0000 7fa8d2a00640 1 --2- 172.21.15.2:0/4093298897 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa8b40a2c00 0x7fa8b40a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00cd170 tx=0x7fa8c00d34a0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.908+0000 7fa8d2a00640 1 -- 172.21.15.2:0/4093298897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8cc074510 msgr2=0x7fa8cc19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:12:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.908+0000 7fa8d2a00640 1 --2- 172.21.15.2:0/4093298897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8cc074510 0x7fa8cc19a4f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009d50 tx=0x7fa8c000c6a0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.908+0000 7fa8d2a00640 1 -- 172.21.15.2:0/4093298897 shutdown_connections 2024-09-10T08:12:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.908+0000 7fa8d2a00640 1 --2- 172.21.15.2:0/4093298897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8cc10e280 0x7fa8cc19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.908+0000 7fa8d2a00640 1 --2- 172.21.15.2:0/4093298897 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa8b40a2c00 0x7fa8b40a50c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.908+0000 7fa8d2a00640 1 --2- 172.21.15.2:0/4093298897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8cc074e30 0x7fa8cc19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.908+0000 7fa8d2a00640 1 --2- 172.21.15.2:0/4093298897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8cc074510 0x7fa8cc19a4f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:12:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.908+0000 7fa8d2a00640 1 -- 172.21.15.2:0/4093298897 >> 172.21.15.2:0/4093298897 conn(0x7fa8cc0fd740 msgr2=0x7fa8cc0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:12:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.908+0000 7fa8d2a00640 1 -- 172.21.15.2:0/4093298897 shutdown_connections 2024-09-10T08:12:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:12:55.908+0000 7fa8d2a00640 1 -- 172.21.15.2:0/4093298897 wait complete. 2024-09-10T08:12:55.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:55 smithi107 ceph-mon[35048]: pgmap v530: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-10T08:12:56.809 INFO:tasks.cephfs.filesystem:scrub status for tag:5753f598-fa05-4827-bb77-d4073b04b243 - {'path': '/', 'tag': '5753f598-fa05-4827-bb77-d4073b04b243', 'options': 'recursive,force'} 2024-09-10T08:12:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:57 smithi155 ceph-mon[35405]: pgmap v531: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 12 op/s 2024-09-10T08:12:57.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:57 smithi107 ceph-mon[35048]: pgmap v531: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 12 op/s 2024-09-10T08:12:58.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:57 smithi002 ceph-mon[30563]: pgmap v531: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 12 op/s 2024-09-10T08:12:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:59 smithi155 ceph-mon[35405]: pgmap v532: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-10T08:12:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:12:59 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:12:59.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:59 smithi107 ceph-mon[35048]: pgmap v532: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-10T08:12:59.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:12:59 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:13:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:59 smithi002 ceph-mon[30563]: pgmap v532: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-10T08:13:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:12:59 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:13:01.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:01 smithi155 ceph-mon[35405]: pgmap v533: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-10T08:13:01.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:01 smithi107 ceph-mon[35048]: pgmap v533: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-10T08:13:02.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:01 smithi002 ceph-mon[30563]: pgmap v533: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-10T08:13:02.302 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:13:02.691 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:13:03.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.289+0000 7f23b9800640 1 -- 172.21.15.2:0/1770283030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b4103ff0 msgr2=0x7f23b41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:03.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.289+0000 7f23b9800640 1 --2- 172.21.15.2:0/1770283030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b4103ff0 0x7f23b41043d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009900 tx=0x7f23a402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:13:03.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.289+0000 7f23b9800640 1 -- 172.21.15.2:0/1770283030 shutdown_connections 2024-09-10T08:13:03.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.289+0000 7f23b9800640 1 --2- 172.21.15.2:0/1770283030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b4109120 0x7f23b4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:03.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.289+0000 7f23b9800640 1 --2- 172.21.15.2:0/1770283030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b41049a0 0x7f23b41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:03.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.289+0000 7f23b9800640 1 --2- 172.21.15.2:0/1770283030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b4103ff0 0x7f23b41043d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:03.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.289+0000 7f23b9800640 1 -- 172.21.15.2:0/1770283030 >> 172.21.15.2:0/1770283030 conn(0x7f23b40ffa90 msgr2=0x7f23b4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:03.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.290+0000 7f23b9800640 1 -- 172.21.15.2:0/1770283030 shutdown_connections 2024-09-10T08:13:03.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.290+0000 7f23b9800640 1 -- 172.21.15.2:0/1770283030 wait complete. 2024-09-10T08:13:03.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.290+0000 7f23b9800640 1 Processor -- start 2024-09-10T08:13:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.291+0000 7f23b9800640 1 -- start start 2024-09-10T08:13:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.291+0000 7f23b9800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b4103ff0 0x7f23b419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.291+0000 7f23b9800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b41049a0 0x7f23b419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.291+0000 7f23b9800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b4109120 0x7f23b419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.291+0000 7f23b9800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f23b41141a0 con 0x7f23b4103ff0 2024-09-10T08:13:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.291+0000 7f23b9800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23b4114020 con 0x7f23b41049a0 2024-09-10T08:13:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.291+0000 7f23b9800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f23b4114320 con 0x7f23b4109120 2024-09-10T08:13:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.291+0000 7f23b2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b4103ff0 0x7f23b419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.291+0000 7f23b2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b4103ff0 0x7f23b419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34056/0 (socket says 172.21.15.2:34056) 2024-09-10T08:13:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.291+0000 7f23b2a00640 1 -- 172.21.15.2:0/1572908457 learned_addr learned my addr 172.21.15.2:0/1572908457 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.291+0000 7f23b3400640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b4109120 0x7f23b419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.291+0000 7f23b2000640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b41049a0 0x7f23b419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.292+0000 7f23b2a00640 1 -- 172.21.15.2:0/1572908457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b4109120 msgr2=0x7f23b419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.292+0000 7f23b2a00640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b4109120 0x7f23b419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.292+0000 7f23b2a00640 1 -- 172.21.15.2:0/1572908457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b41049a0 msgr2=0x7f23b419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.292+0000 7f23b2a00640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b41049a0 0x7f23b419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.292+0000 7f23b2a00640 1 -- 172.21.15.2:0/1572908457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b419f5d0 con 0x7f23b4103ff0 2024-09-10T08:13:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.292+0000 7f23b2000640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b41049a0 0x7f23b419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:13:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.292+0000 7f23b2a00640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b4103ff0 0x7f23b419a620 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f23a4004770 tx=0x7f23a4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.292+0000 7f23ab400640 1 -- 172.21.15.2:0/1572908457 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a403f070 con 0x7f23b4103ff0 2024-09-10T08:13:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.292+0000 7f23b9800640 1 -- 172.21.15.2:0/1572908457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b419f860 con 0x7f23b4103ff0 2024-09-10T08:13:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.292+0000 7f23ab400640 1 -- 172.21.15.2:0/1572908457 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23a4031ca0 con 0x7f23b4103ff0 2024-09-10T08:13:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.292+0000 7f23ab400640 1 -- 172.21.15.2:0/1572908457 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4035030 con 0x7f23b4103ff0 2024-09-10T08:13:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.292+0000 7f23b9800640 1 -- 172.21.15.2:0/1572908457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b41a71a0 con 0x7f23b4103ff0 2024-09-10T08:13:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.293+0000 7f23b9800640 1 -- 172.21.15.2:0/1572908457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b41043d0 con 0x7f23b4103ff0 2024-09-10T08:13:03.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.295+0000 7f23ab400640 1 -- 172.21.15.2:0/1572908457 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f23a4042070 con 0x7f23b4103ff0 2024-09-10T08:13:03.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.296+0000 7f23ab400640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23980a2b80 0x7f23980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:03.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.297+0000 7f23ab400640 1 -- 172.21.15.2:0/1572908457 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f23a40ef850 con 0x7f23b4103ff0 2024-09-10T08:13:03.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.297+0000 7f23b2000640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23980a2b80 0x7f23980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:03.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.297+0000 7f23b2000640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23980a2b80 0x7f23980a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f23b40fff20 tx=0x7f23a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:03.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.301+0000 7f23ab400640 1 -- 172.21.15.2:0/1572908457 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23a40bc420 con 0x7f23b4103ff0 2024-09-10T08:13:03.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.522+0000 7f23b9800640 1 -- 172.21.15.2:0/1572908457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23b410c300 con 0x7f23b4103ff0 2024-09-10T08:13:03.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.523+0000 7f23ab400640 1 -- 172.21.15.2:0/1572908457 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f23a40bbd20 con 0x7f23b4103ff0 2024-09-10T08:13:03.522 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:13:03.522 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:13:03.523 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:13:03.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.525+0000 7f23b9800640 1 -- 172.21.15.2:0/1572908457 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23980a2b80 msgr2=0x7f23980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:03.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.525+0000 7f23b9800640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23980a2b80 0x7f23980a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f23b40fff20 tx=0x7f23a0005f90 comp rx=0 tx=0).stop 2024-09-10T08:13:03.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.525+0000 7f23b9800640 1 -- 172.21.15.2:0/1572908457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b4103ff0 msgr2=0x7f23b419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:03.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.525+0000 7f23b9800640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b4103ff0 0x7f23b419a620 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f23a4004770 tx=0x7f23a4003c10 comp rx=0 tx=0).stop 2024-09-10T08:13:03.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.526+0000 7f23b9800640 1 -- 172.21.15.2:0/1572908457 shutdown_connections 2024-09-10T08:13:03.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.526+0000 7f23b9800640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23980a2b80 0x7f23980a5040 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:03.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.526+0000 7f23b9800640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b4109120 0x7f23b419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:03.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.526+0000 7f23b9800640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b41049a0 0x7f23b419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:03.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.526+0000 7f23b9800640 1 --2- 172.21.15.2:0/1572908457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b4103ff0 0x7f23b419a620 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:03.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.526+0000 7f23b9800640 1 -- 172.21.15.2:0/1572908457 >> 172.21.15.2:0/1572908457 conn(0x7f23b40ffa90 msgr2=0x7f23b4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:03.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.526+0000 7f23b9800640 1 -- 172.21.15.2:0/1572908457 shutdown_connections 2024-09-10T08:13:03.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:03.526+0000 7f23b9800640 1 -- 172.21.15.2:0/1572908457 wait complete. 2024-09-10T08:13:03.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:03 smithi155 ceph-mon[35405]: pgmap v534: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-10T08:13:03.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:03 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1572908457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:13:03.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:03 smithi107 ceph-mon[35048]: pgmap v534: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-10T08:13:03.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:03 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1572908457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:13:03.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:03 smithi002 ceph-mon[30563]: pgmap v534: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-10T08:13:03.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:03 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1572908457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:13:04.403 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id 232ae216 --release --await' 2024-09-10T08:13:04.403 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id 232ae216 --release --await 2024-09-10T08:13:04.782 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:13:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.350+0000 7f139da00640 1 -- 172.21.15.2:0/3092804817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13981047e0 msgr2=0x7f1398108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.350+0000 7f139da00640 1 --2- 172.21.15.2:0/3092804817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13981047e0 0x7f1398108830 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1380009900 tx=0x7f138002a410 comp rx=0 tx=0).stop 2024-09-10T08:13:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.351+0000 7f139da00640 1 -- 172.21.15.2:0/3092804817 shutdown_connections 2024-09-10T08:13:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.351+0000 7f139da00640 1 --2- 172.21.15.2:0/3092804817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1398108f60 0x7f1398110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.351+0000 7f139da00640 1 --2- 172.21.15.2:0/3092804817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13981047e0 0x7f1398108830 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.351+0000 7f139da00640 1 --2- 172.21.15.2:0/3092804817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1398103e30 0x7f1398104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.351+0000 7f139da00640 1 -- 172.21.15.2:0/3092804817 >> 172.21.15.2:0/3092804817 conn(0x7f13980ff8d0 msgr2=0x7f1398101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.351+0000 7f139da00640 1 -- 172.21.15.2:0/3092804817 shutdown_connections 2024-09-10T08:13:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.351+0000 7f139da00640 1 -- 172.21.15.2:0/3092804817 wait complete. 2024-09-10T08:13:05.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.352+0000 7f139da00640 1 Processor -- start 2024-09-10T08:13:05.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.352+0000 7f139da00640 1 -- start start 2024-09-10T08:13:05.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.352+0000 7f139da00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1398103e30 0x7f139819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.352+0000 7f139da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13981047e0 0x7f139819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.352+0000 7f139da00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1398108f60 0x7f139819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.352+0000 7f139da00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1398114010 con 0x7f1398103e30 2024-09-10T08:13:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.352+0000 7f139da00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1398113e90 con 0x7f1398108f60 2024-09-10T08:13:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.352+0000 7f139da00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1398114190 con 0x7f13981047e0 2024-09-10T08:13:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.353+0000 7f1397e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1398103e30 0x7f139819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.353+0000 7f1397e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1398103e30 0x7f139819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34078/0 (socket says 172.21.15.2:34078) 2024-09-10T08:13:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.353+0000 7f1397e00640 1 -- 172.21.15.2:0/4179859007 learned_addr learned my addr 172.21.15.2:0/4179859007 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.353+0000 7f138f400640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13981047e0 0x7f139819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.353+0000 7f139d000640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1398108f60 0x7f139819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.353+0000 7f1397e00640 1 -- 172.21.15.2:0/4179859007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13981047e0 msgr2=0x7f139819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:05.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.353+0000 7f1397e00640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13981047e0 0x7f139819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.353+0000 7f1397e00640 1 -- 172.21.15.2:0/4179859007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1398108f60 msgr2=0x7f139819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:05.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.353+0000 7f1397e00640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1398108f60 0x7f139819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.353+0000 7f1397e00640 1 -- 172.21.15.2:0/4179859007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139819f430 con 0x7f1398103e30 2024-09-10T08:13:05.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.353+0000 7f1397e00640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1398103e30 0x7f139819a480 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f138800e9f0 tx=0x7f138800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:05.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.354+0000 7f1395600640 1 -- 172.21.15.2:0/4179859007 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138800cd80 con 0x7f1398103e30 2024-09-10T08:13:05.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.354+0000 7f1395600640 1 -- 172.21.15.2:0/4179859007 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1388010910 con 0x7f1398103e30 2024-09-10T08:13:05.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.354+0000 7f1395600640 1 -- 172.21.15.2:0/4179859007 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388018990 con 0x7f1398103e30 2024-09-10T08:13:05.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.354+0000 7f139da00640 1 -- 172.21.15.2:0/4179859007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139819f720 con 0x7f1398103e30 2024-09-10T08:13:05.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.354+0000 7f139da00640 1 -- 172.21.15.2:0/4179859007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13981a7000 con 0x7f1398103e30 2024-09-10T08:13:05.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.357+0000 7f13a1c3b740 1 Processor -- start 2024-09-10T08:13:05.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.357+0000 7f1395600640 1 -- 172.21.15.2:0/4179859007 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1388010ac0 con 0x7f1398103e30 2024-09-10T08:13:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.357+0000 7f13a1c3b740 1 -- start start 2024-09-10T08:13:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.357+0000 7f13a1c3b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd17900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.357+0000 7f13a1c3b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd17e40 0x5602acd221c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.358+0000 7f13a1c3b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd22700 0x5602acd24af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.358+0000 7f13a1c3b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5602acc7d650 con 0x5602acd22700 2024-09-10T08:13:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.358+0000 7f13a1c3b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5602acc24f10 con 0x5602acd17e40 2024-09-10T08:13:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.358+0000 7f13a1c3b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5602acc79610 con 0x5602acd17520 2024-09-10T08:13:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.358+0000 7f1397e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd17e40 0x5602acd221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.358+0000 7f139d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd17900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.358+0000 7f138f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd22700 0x5602acd24af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.358+0000 7f139d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd17900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49608/0 (socket says 172.21.15.2:49608) 2024-09-10T08:13:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.358+0000 7f139d000640 1 -- 172.21.15.2:0/2981561611 learned_addr learned my addr 172.21.15.2:0/2981561611 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:05.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.358+0000 7f1397e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd17e40 0x5602acd221c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52020/0 (socket says 172.21.15.2:52020) 2024-09-10T08:13:05.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.359+0000 7f139d000640 1 -- 172.21.15.2:0/2981561611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd17e40 msgr2=0x5602acd221c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:05.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.359+0000 7f139d000640 1 --2- 172.21.15.2:0/2981561611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd17e40 0x5602acd221c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.359+0000 7f139d000640 1 -- 172.21.15.2:0/2981561611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd22700 msgr2=0x5602acd24af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:05.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.359+0000 7f139d000640 1 --2- 172.21.15.2:0/2981561611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd22700 0x5602acd24af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.359+0000 7f139d000640 1 -- 172.21.15.2:0/2981561611 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602ac98c580 con 0x5602acd17520 2024-09-10T08:13:05.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.359+0000 7f1397e00640 1 --2- 172.21.15.2:0/2981561611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd17e40 0x5602acd221c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.359+0000 7f1395600640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f137c0a2c00 0x7f137c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.359+0000 7f139d000640 1 --2- 172.21.15.2:0/2981561611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd17900 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1390009f90 tx=0x7f1390010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a27e8f6bbc181f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.359+0000 7f1395600640 1 -- 172.21.15.2:0/4179859007 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f13880cf740 con 0x7f1398103e30 2024-09-10T08:13:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.359+0000 7f1397e00640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f137c0a2c00 0x7f137c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.359+0000 7f138d600640 1 -- 172.21.15.2:0/2981561611 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390011700 con 0x5602acd17520 2024-09-10T08:13:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.359+0000 7f138d600640 1 -- 172.21.15.2:0/2981561611 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1390011d80 con 0x5602acd17520 2024-09-10T08:13:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.359+0000 7f138d600640 1 -- 172.21.15.2:0/2981561611 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390019e40 con 0x5602acd17520 2024-09-10T08:13:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.360+0000 7f13a1c3b740 1 -- 172.21.15.2:0/2981561611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 msgr2=0x5602acd17900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.360+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/2981561611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd17900 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1390009f90 tx=0x7f1390010860 comp rx=0 tx=0).stop 2024-09-10T08:13:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.360+0000 7f1397e00640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f137c0a2c00 0x7f137c0a50c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f138800e9c0 tx=0x7f13880d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.360+0000 7f13a1c3b740 1 -- 172.21.15.2:0/2981561611 shutdown_connections 2024-09-10T08:13:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.360+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/2981561611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd22700 0x5602acd24af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.360+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/2981561611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd17e40 0x5602acd221c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.360+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/2981561611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd17900 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.360+0000 7f13a1c3b740 1 -- 172.21.15.2:0/2981561611 >> 172.21.15.2:0/2981561611 conn(0x5602acd13ab0 msgr2=0x5602acd13e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.360+0000 7f13a1c3b740 1 -- 172.21.15.2:0/2981561611 shutdown_connections 2024-09-10T08:13:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.360+0000 7f13a1c3b740 1 -- 172.21.15.2:0/2981561611 wait complete. 2024-09-10T08:13:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.361+0000 7f13a1c3b740 1 -- start start 2024-09-10T08:13:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.361+0000 7f13a1c3b740 1 Processor -- start 2024-09-10T08:13:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.362+0000 7f13a1c3b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd1f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.362+0000 7f13a1c3b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd17e40 0x5602acd1fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.362+0000 7f13a1c3b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd22700 0x5602acd200c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.362+0000 7f13a1c3b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5602acd252d0 con 0x5602acd22700 2024-09-10T08:13:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.362+0000 7f13a1c3b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5602acc38350 con 0x5602acd17e40 2024-09-10T08:13:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.362+0000 7f13a1c3b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5602acd25450 con 0x5602acd17520 2024-09-10T08:13:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.362+0000 7f13a1c3b740 10 client.0 _ll_register_callbacks cb 0x5602acc5b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.362+0000 7f139d000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd22700 0x5602acd200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.362+0000 7f138f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd17e40 0x5602acd1fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.362+0000 7f139d000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd22700 0x5602acd200c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34102/0 (socket says 172.21.15.2:34102) 2024-09-10T08:13:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.362+0000 7f139d000640 1 -- 172.21.15.2:0/312507865 learned_addr learned my addr 172.21.15.2:0/312507865 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.362+0000 7f139d000640 1 --2- 172.21.15.2:0/312507865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd1f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.363+0000 7f138f400640 1 -- 172.21.15.2:0/312507865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 msgr2=0x5602acd1f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.363+0000 7f138f400640 1 --2- 172.21.15.2:0/312507865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd1f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.363+0000 7f138f400640 1 -- 172.21.15.2:0/312507865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd22700 msgr2=0x5602acd200c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.363+0000 7f138f400640 1 --2- 172.21.15.2:0/312507865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd22700 0x5602acd200c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.363+0000 7f138f400640 1 -- 172.21.15.2:0/312507865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602aca56960 con 0x5602acd17e40 2024-09-10T08:13:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.363+0000 7f139d000640 1 --2- 172.21.15.2:0/312507865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd22700 0x5602acd200c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.363+0000 7f138f400640 1 --2- 172.21.15.2:0/312507865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd17e40 0x5602acd1fb80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f138002aa00 tx=0x7f1380004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8b566c9180b4b58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.363+0000 7f1386000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:13:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.363+0000 7f1386000640 1 -- 172.21.15.2:0/312507865 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380035040 con 0x5602acd17e40 2024-09-10T08:13:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.363+0000 7f13a1c3b740 1 -- 172.21.15.2:0/312507865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1390011d80 con 0x5602acd17e40 2024-09-10T08:13:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.363+0000 7f1386000640 1 -- 172.21.15.2:0/312507865 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1380005450 con 0x5602acd17e40 2024-09-10T08:13:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.364+0000 7f1386000640 1 -- 172.21.15.2:0/312507865 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f138004f050 con 0x5602acd17e40 2024-09-10T08:13:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.364+0000 7f13a1c3b740 10 client.26807 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.364+0000 7f13a1c3b740 1 -- 172.21.15.2:0/312507865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5602acd30f80 con 0x5602acd17e40 2024-09-10T08:13:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.364+0000 7f1386000640 1 -- 172.21.15.2:0/312507865 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1390011d80 con 0x5602acd17e40 2024-09-10T08:13:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.364+0000 7f1386000640 1 -- 172.21.15.2:0/312507865 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1380031850 con 0x5602acd17e40 2024-09-10T08:13:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.364+0000 7f13a1c3b740 10 client.26807 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.364+0000 7f13a1c3b740 10 client.26807 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:13:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.364+0000 7f13a1c3b740 20 client.26807 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.364+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/312507865 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5602acd32390 0x5602acd34780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.364+0000 7f13a1c3b740 4 client.26807 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:13:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.364+0000 7f13a1c3b740 1 -- 172.21.15.2:0/312507865 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602acc8abc0 con 0x5602acd32390 2024-09-10T08:13:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.365+0000 7f139d000640 1 --2- 172.21.15.2:0/312507865 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5602acd32390 0x5602acd34780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.365+0000 7f139d000640 1 --2- 172.21.15.2:0/312507865 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5602acd32390 0x5602acd34780 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95072d2ec03480ff server_cookie=4bdbd48ceb50ec2b in_seq=0 out_seq=0 2024-09-10T08:13:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.365+0000 7f1386000640 10 client.26807 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f1386000640 1 -- 172.21.15.2:0/312507865 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602acc8abc0 con 0x5602acd32390 2024-09-10T08:13:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f1386000640 10 client.26807 handle_command_reply: tid=0 2024-09-10T08:13:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f13a1c3b740 1 client.26807 shutdown 2024-09-10T08:13:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f13a1c3b740 1 -- 172.21.15.2:0/312507865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd17e40 msgr2=0x5602acd1fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/312507865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd17e40 0x5602acd1fb80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f138002aa00 tx=0x7f1380004420 comp rx=0 tx=0).stop 2024-09-10T08:13:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f13a1c3b740 1 -- 172.21.15.2:0/312507865 shutdown_connections 2024-09-10T08:13:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/312507865 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5602acd32390 0x5602acd34780 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/312507865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd22700 0x5602acd200c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/312507865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd17e40 0x5602acd1fb80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f1386000640 0 client.26807 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:05.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/312507865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd1f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f13a1c3b740 1 -- 172.21.15.2:0/312507865 >> 172.21.15.2:0/312507865 conn(0x5602acd13ab0 msgr2=0x5602acd12cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:05.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f13a1c3b740 1 -- 172.21.15.2:0/312507865 shutdown_connections 2024-09-10T08:13:05.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.366+0000 7f13a1c3b740 1 -- 172.21.15.2:0/312507865 wait complete. 2024-09-10T08:13:05.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.367+0000 7f13a1c3b740 20 client.26807 trim_cache size 0 max 16384 2024-09-10T08:13:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.384+0000 7f13a1c3b740 1 Processor -- start 2024-09-10T08:13:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.384+0000 7f13a1c3b740 1 -- start start 2024-09-10T08:13:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.385+0000 7f13a1c3b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd1abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.385+0000 7f13a1c3b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd17e40 0x5602acd1e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.385+0000 7f13a1c3b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd22700 0x5602acd3a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.385+0000 7f13a1c3b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13981b37a0 con 0x5602acd17e40 2024-09-10T08:13:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.385+0000 7f13a1c3b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13981b1070 con 0x5602acd22700 2024-09-10T08:13:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.385+0000 7f13a1c3b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13981aeec0 con 0x5602acd17520 2024-09-10T08:13:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.385+0000 7f138f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd17e40 0x5602acd1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.385+0000 7f138f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd17e40 0x5602acd1e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34104/0 (socket says 172.21.15.2:34104) 2024-09-10T08:13:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.385+0000 7f138f400640 1 -- 172.21.15.2:0/2719147291 learned_addr learned my addr 172.21.15.2:0/2719147291 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.385+0000 7f139d000640 1 --2- 172.21.15.2:0/2719147291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd22700 0x5602acd3a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.385+0000 7f139d000640 1 --2- 172.21.15.2:0/2719147291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd1abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.385+0000 7f138f400640 1 -- 172.21.15.2:0/2719147291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 msgr2=0x5602acd1abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.385+0000 7f138f400640 1 --2- 172.21.15.2:0/2719147291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd1abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.386+0000 7f138f400640 1 -- 172.21.15.2:0/2719147291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd22700 msgr2=0x5602acd3a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.386+0000 7f138f400640 1 --2- 172.21.15.2:0/2719147291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd22700 0x5602acd3a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.386+0000 7f138f400640 1 -- 172.21.15.2:0/2719147291 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602acd17350 con 0x5602acd17e40 2024-09-10T08:13:05.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.386+0000 7f139d000640 1 --2- 172.21.15.2:0/2719147291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd22700 0x5602acd3a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:05.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.386+0000 7f138f400640 1 --2- 172.21.15.2:0/2719147291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd17e40 0x5602acd1e580 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1380031c50 tx=0x7f1380005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410199801c042847 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:05.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.386+0000 7f138cc00640 1 -- 172.21.15.2:0/2719147291 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380058070 con 0x5602acd17e40 2024-09-10T08:13:05.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.386+0000 7f138cc00640 1 -- 172.21.15.2:0/2719147291 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1380041070 con 0x5602acd17e40 2024-09-10T08:13:05.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.386+0000 7f138cc00640 1 -- 172.21.15.2:0/2719147291 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380035030 con 0x5602acd17e40 2024-09-10T08:13:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.387+0000 7f13a1c3b740 1 -- 172.21.15.2:0/2719147291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd17e40 msgr2=0x5602acd1e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.387+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/2719147291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd17e40 0x5602acd1e580 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1380031c50 tx=0x7f1380005a40 comp rx=0 tx=0).stop 2024-09-10T08:13:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.387+0000 7f13a1c3b740 1 -- 172.21.15.2:0/2719147291 shutdown_connections 2024-09-10T08:13:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.387+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/2719147291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd22700 0x5602acd3a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.387+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/2719147291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd17e40 0x5602acd1e580 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.387+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/2719147291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd1abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.387+0000 7f13a1c3b740 1 -- 172.21.15.2:0/2719147291 >> 172.21.15.2:0/2719147291 conn(0x5602acd13ab0 msgr2=0x5602acd12cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.387+0000 7f13a1c3b740 1 -- 172.21.15.2:0/2719147291 shutdown_connections 2024-09-10T08:13:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.387+0000 7f13a1c3b740 1 -- 172.21.15.2:0/2719147291 wait complete. 2024-09-10T08:13:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.387+0000 7f13a1c3b740 1 -- start start 2024-09-10T08:13:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.388+0000 7f13a1c3b740 1 Processor -- start 2024-09-10T08:13:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.388+0000 7f13a1c3b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd264f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.388+0000 7f13a1c3b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd17e40 0x5602acd26a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.388+0000 7f13a1c3b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd22700 0x5602acd26f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.388+0000 7f13a1c3b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13981af530 con 0x5602acd17e40 2024-09-10T08:13:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.388+0000 7f13a1c3b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13981b09a0 con 0x5602acd22700 2024-09-10T08:13:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.388+0000 7f13a1c3b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13981afc00 con 0x5602acd17520 2024-09-10T08:13:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.388+0000 7f13a1c3b740 10 client.0 _ll_register_callbacks cb 0x7f1380009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.388+0000 7f139d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd264f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.388+0000 7f138f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd17e40 0x5602acd26a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.388+0000 7f139d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd264f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49640/0 (socket says 172.21.15.2:49640) 2024-09-10T08:13:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.388+0000 7f139d000640 1 -- 172.21.15.2:0/3484441522 learned_addr learned my addr 172.21.15.2:0/3484441522 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.388+0000 7f139d000640 1 --2- 172.21.15.2:0/3484441522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd22700 0x5602acd26f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.389+0000 7f139d000640 1 -- 172.21.15.2:0/3484441522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd22700 msgr2=0x5602acd26f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.389+0000 7f139d000640 1 --2- 172.21.15.2:0/3484441522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd22700 0x5602acd26f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.389+0000 7f139d000640 1 -- 172.21.15.2:0/3484441522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd17e40 msgr2=0x5602acd26a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.389+0000 7f139d000640 1 --2- 172.21.15.2:0/3484441522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd17e40 0x5602acd26a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.389+0000 7f139d000640 1 -- 172.21.15.2:0/3484441522 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602acd14740 con 0x5602acd17520 2024-09-10T08:13:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.389+0000 7f138f400640 1 --2- 172.21.15.2:0/3484441522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd17e40 0x5602acd26a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.389+0000 7f139d000640 1 --2- 172.21.15.2:0/3484441522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd264f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1390019970 tx=0x7f1380004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51d38c1d67daf38f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.389+0000 7f1386000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:13:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.389+0000 7f1386000640 1 -- 172.21.15.2:0/3484441522 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390019da0 con 0x5602acd17520 2024-09-10T08:13:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.389+0000 7f1386000640 1 -- 172.21.15.2:0/3484441522 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1390003970 con 0x5602acd17520 2024-09-10T08:13:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.389+0000 7f13a1c3b740 1 -- 172.21.15.2:0/3484441522 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1380041070 con 0x5602acd17520 2024-09-10T08:13:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.390+0000 7f1386000640 1 -- 172.21.15.2:0/3484441522 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1390003b70 con 0x5602acd17520 2024-09-10T08:13:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.390+0000 7f13a1c3b740 10 client.26776 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.390+0000 7f13a1c3b740 1 -- 172.21.15.2:0/3484441522 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5602acd9d270 con 0x5602acd17520 2024-09-10T08:13:05.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.390+0000 7f1386000640 1 -- 172.21.15.2:0/3484441522 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1380041070 con 0x5602acd17520 2024-09-10T08:13:05.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.390+0000 7f1386000640 1 -- 172.21.15.2:0/3484441522 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1390011500 con 0x5602acd17520 2024-09-10T08:13:05.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.390+0000 7f13a1c3b740 10 client.26776 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:05.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.390+0000 7f13a1c3b740 10 client.26776 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:13:05.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.390+0000 7f13a1c3b740 20 client.26776 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:05.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.390+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/3484441522 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5602acd9d6c0 0x5602acd9fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:05.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.390+0000 7f13a1c3b740 4 client.26776 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "232ae216", "release": true, "await": true}] 2024-09-10T08:13:05.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.390+0000 7f13a1c3b740 1 -- 172.21.15.2:0/3484441522 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "232ae216", "release": true, "await": true}) -- 0x5602acc850d0 con 0x5602acd9d6c0 2024-09-10T08:13:05.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.390+0000 7f138f400640 1 --2- 172.21.15.2:0/3484441522 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5602acd9d6c0 0x5602acd9fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:05.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.391+0000 7f138f400640 1 --2- 172.21.15.2:0/3484441522 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5602acd9d6c0 0x5602acd9fab0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3767460ddcb4262d server_cookie=8bb1fb57f5130a46 in_seq=0 out_seq=0 2024-09-10T08:13:05.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.391+0000 7f1386000640 10 client.26776 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:05.393 INFO:tasks.workunit.client.0.smithi002.stdout:64/600: open fd = 3 2024-09-10T08:13:05.394 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.394 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.394 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.394 INFO:tasks.workunit.client.0.smithi002.stdout:65/600: open fd = 3 2024-09-10T08:13:05.394 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.394 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.394 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.394 INFO:tasks.workunit.client.0.smithi002.stdout:66/600: open fd = 3 2024-09-10T08:13:05.394 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.394 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.394 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.394 INFO:tasks.workunit.client.0.smithi002.stdout:67/600: open fd = 3 2024-09-10T08:13:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:68/600: open fd = 3 2024-09-10T08:13:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:69/600: open fd = 3 2024-09-10T08:13:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:70/600: open fd = 3 2024-09-10T08:13:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.396 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.396 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.396 INFO:tasks.workunit.client.0.smithi002.stdout:71/600: open fd = 3 2024-09-10T08:13:05.396 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.396 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.396 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.396 INFO:tasks.workunit.client.0.smithi002.stdout:72/600: open fd = 3 2024-09-10T08:13:05.396 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.396 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.396 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.396 INFO:tasks.workunit.client.0.smithi002.stdout:73/600: open fd = 3 2024-09-10T08:13:05.396 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.397 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.397 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.397 INFO:tasks.workunit.client.0.smithi002.stdout:74/600: open fd = 3 2024-09-10T08:13:05.397 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.397 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.397 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.397 INFO:tasks.workunit.client.0.smithi002.stdout:75/600: open fd = 3 2024-09-10T08:13:05.397 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.397 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.397 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.397 INFO:tasks.workunit.client.0.smithi002.stdout:76/600: open fd = 3 2024-09-10T08:13:05.397 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.398 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.398 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.398 INFO:tasks.workunit.client.0.smithi002.stdout:77/600: open fd = 3 2024-09-10T08:13:05.398 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.398 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.398 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.398 INFO:tasks.workunit.client.0.smithi002.stdout:78/600: open fd = 3 2024-09-10T08:13:05.398 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.398 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.398 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.398 INFO:tasks.workunit.client.0.smithi002.stdout:79/600: open fd = 3 2024-09-10T08:13:05.398 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.399 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.399 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.399 INFO:tasks.workunit.client.0.smithi002.stdout:80/600: open fd = 3 2024-09-10T08:13:05.399 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.399 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.399 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.399 INFO:tasks.workunit.client.0.smithi002.stdout:81/600: open fd = 3 2024-09-10T08:13:05.399 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.399 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.399 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.399 INFO:tasks.workunit.client.0.smithi002.stdout:82/600: open fd = 3 2024-09-10T08:13:05.399 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.399 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.400 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.400 INFO:tasks.workunit.client.0.smithi002.stdout:83/600: open fd = 3 2024-09-10T08:13:05.400 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.400 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.400 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.400 INFO:tasks.workunit.client.0.smithi002.stdout:84/600: open fd = 3 2024-09-10T08:13:05.400 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.400 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.400 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.400 INFO:tasks.workunit.client.0.smithi002.stdout:85/600: open fd = 3 2024-09-10T08:13:05.400 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.400 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.401 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.401 INFO:tasks.workunit.client.0.smithi002.stdout:86/600: open fd = 3 2024-09-10T08:13:05.401 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.401 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.401 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.401 INFO:tasks.workunit.client.0.smithi002.stdout:87/600: open fd = 3 2024-09-10T08:13:05.401 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.401 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.401 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.401 INFO:tasks.workunit.client.0.smithi002.stdout:88/600: open fd = 3 2024-09-10T08:13:05.401 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.401 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.401 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.402 INFO:tasks.workunit.client.0.smithi002.stdout:89/600: open fd = 3 2024-09-10T08:13:05.402 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.402 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.402 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.402 INFO:tasks.workunit.client.0.smithi002.stdout:90/600: open fd = 3 2024-09-10T08:13:05.402 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.402 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.402 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.403 INFO:tasks.workunit.client.0.smithi002.stdout:91/600: open fd = 3 2024-09-10T08:13:05.403 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.403 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.403 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.403 INFO:tasks.workunit.client.0.smithi002.stdout:92/600: open fd = 3 2024-09-10T08:13:05.403 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.403 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.403 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.403 INFO:tasks.workunit.client.0.smithi002.stdout:93/600: open fd = 3 2024-09-10T08:13:05.403 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.403 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.403 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.404 INFO:tasks.workunit.client.0.smithi002.stdout:94/600: open fd = 3 2024-09-10T08:13:05.404 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.404 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.404 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.404 INFO:tasks.workunit.client.0.smithi002.stdout:95/600: open fd = 3 2024-09-10T08:13:05.404 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.404 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.404 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.404 INFO:tasks.workunit.client.0.smithi002.stdout:96/600: open fd = 3 2024-09-10T08:13:05.405 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.405 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.405 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.405 INFO:tasks.workunit.client.0.smithi002.stdout:97/600: open fd = 3 2024-09-10T08:13:05.405 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.405 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.405 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.405 INFO:tasks.workunit.client.0.smithi002.stdout:98/600: open fd = 3 2024-09-10T08:13:05.405 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.406 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.406 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.406 INFO:tasks.workunit.client.0.smithi002.stdout:99/600: open fd = 3 2024-09-10T08:13:05.406 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.406 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.406 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.406 INFO:tasks.workunit.client.0.smithi002.stdout:100/600: open fd = 3 2024-09-10T08:13:05.406 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.406 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.406 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.406 INFO:tasks.workunit.client.0.smithi002.stdout:101/600: open fd = 3 2024-09-10T08:13:05.406 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.406 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.407 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.407 INFO:tasks.workunit.client.0.smithi002.stdout:102/600: open fd = 3 2024-09-10T08:13:05.407 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.407 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.407 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.407 INFO:tasks.workunit.client.0.smithi002.stdout:103/600: open fd = 3 2024-09-10T08:13:05.407 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.407 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.407 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.407 INFO:tasks.workunit.client.0.smithi002.stdout:104/600: open fd = 3 2024-09-10T08:13:05.407 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.408 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.408 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.408 INFO:tasks.workunit.client.0.smithi002.stdout:105/600: open fd = 3 2024-09-10T08:13:05.408 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.408 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.408 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.408 INFO:tasks.workunit.client.0.smithi002.stdout:106/600: open fd = 3 2024-09-10T08:13:05.408 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.408 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.408 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.408 INFO:tasks.workunit.client.0.smithi002.stdout:107/600: open fd = 3 2024-09-10T08:13:05.408 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.409 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.409 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.409 INFO:tasks.workunit.client.0.smithi002.stdout:108/600: open fd = 3 2024-09-10T08:13:05.409 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.409 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.409 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.409 INFO:tasks.workunit.client.0.smithi002.stdout:109/600: open fd = 3 2024-09-10T08:13:05.409 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.409 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.409 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.409 INFO:tasks.workunit.client.0.smithi002.stdout:110/600: open fd = 3 2024-09-10T08:13:05.409 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.409 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:111/600: open fd = 3 2024-09-10T08:13:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:112/600: open fd = 3 2024-09-10T08:13:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:113/600: open fd = 3 2024-09-10T08:13:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:114/600: open fd = 3 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:115/600: open fd = 3 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:116/600: open fd = 3 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:117/600: open fd = 3 2024-09-10T08:13:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:118/600: open fd = 3 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:119/600: open fd = 3 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:120/600: open fd = 3 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.412 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:121/600: open fd = 3 2024-09-10T08:13:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:122/600: open fd = 3 2024-09-10T08:13:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:123/600: open fd = 3 2024-09-10T08:13:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:124/600: open fd = 3 2024-09-10T08:13:05.414 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.414 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.414 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.414 INFO:tasks.workunit.client.0.smithi002.stdout:125/600: open fd = 3 2024-09-10T08:13:05.414 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.414 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.414 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:13:05.414 INFO:tasks.workunit.client.0.smithi002.stdout:126/600: open fd = 3 2024-09-10T08:13:05.414 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:13:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.393+0000 7f1386000640 1 -- 172.21.15.2:0/3484441522 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5602acc850d0 con 0x5602acd9d6c0 2024-09-10T08:13:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.393+0000 7f1386000640 10 client.26776 handle_command_reply: tid=0 2024-09-10T08:13:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.393+0000 7f13a1c3b740 1 client.26776 shutdown 2024-09-10T08:13:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.393+0000 7f13a1c3b740 1 -- 172.21.15.2:0/3484441522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 msgr2=0x5602acd264f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.393+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/3484441522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd264f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1390019970 tx=0x7f1380004040 comp rx=0 tx=0).stop 2024-09-10T08:13:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.393+0000 7f13a1c3b740 1 -- 172.21.15.2:0/3484441522 shutdown_connections 2024-09-10T08:13:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.393+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/3484441522 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5602acd9d6c0 0x5602acd9fab0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.394+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/3484441522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602acd22700 0x5602acd26f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.394+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/3484441522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5602acd17e40 0x5602acd26a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.394+0000 7f1386000640 0 client.26776 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:05.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.394+0000 7f13a1c3b740 1 --2- 172.21.15.2:0/3484441522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602acd17520 0x5602acd264f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.394+0000 7f13a1c3b740 1 -- 172.21.15.2:0/3484441522 >> 172.21.15.2:0/3484441522 conn(0x5602acd13ab0 msgr2=0x5602acd18f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:05.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.394+0000 7f13a1c3b740 1 -- 172.21.15.2:0/3484441522 shutdown_connections 2024-09-10T08:13:05.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.394+0000 7f13a1c3b740 1 -- 172.21.15.2:0/3484441522 wait complete. 2024-09-10T08:13:05.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.394+0000 7f13a1c3b740 20 client.26776 trim_cache size 0 max 16384 2024-09-10T08:13:05.416 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:13:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:13:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:13:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 16, 2024-09-10T08:13:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:13:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: "232ae216": { 2024-09-10T08:13:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 16, 2024-09-10T08:13:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:13:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:13:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:13:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:13:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:13:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:13:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 142.8, 2024-09-10T08:13:05.418 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:13:05.418 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:13:05.418 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:13:05.418 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:13:05.418 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:13:05.418 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:13:05.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:13:05.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:13:05.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:13:05.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:13:05.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:13:05.418 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:13:05.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.394+0000 7f139da00640 1 -- 172.21.15.2:0/4179859007 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f137c0a2c00 msgr2=0x7f137c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:05.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.394+0000 7f139da00640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f137c0a2c00 0x7f137c0a50c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f138800e9c0 tx=0x7f13880d5530 comp rx=0 tx=0).stop 2024-09-10T08:13:05.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.394+0000 7f139da00640 1 -- 172.21.15.2:0/4179859007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1398103e30 msgr2=0x7f139819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:05.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.394+0000 7f139da00640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1398103e30 0x7f139819a480 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f138800e9f0 tx=0x7f138800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.395+0000 7f139da00640 1 -- 172.21.15.2:0/4179859007 shutdown_connections 2024-09-10T08:13:05.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.395+0000 7f139da00640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1398108f60 0x7f139819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.395+0000 7f139da00640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f137c0a2c00 0x7f137c0a50c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.395+0000 7f139da00640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13981047e0 0x7f139819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.395+0000 7f139da00640 1 --2- 172.21.15.2:0/4179859007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1398103e30 0x7f139819a480 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:05.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.395+0000 7f139da00640 1 -- 172.21.15.2:0/4179859007 >> 172.21.15.2:0/4179859007 conn(0x7f13980ff8d0 msgr2=0x7f1398100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:05.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.395+0000 7f139da00640 1 -- 172.21.15.2:0/4179859007 shutdown_connections 2024-09-10T08:13:05.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:05.395+0000 7f139da00640 1 -- 172.21.15.2:0/4179859007 wait complete. 2024-09-10T08:13:05.905 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:05 smithi002 ceph-mon[30563]: pgmap v535: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-10T08:13:05.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:05 smithi155 ceph-mon[35405]: pgmap v535: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-10T08:13:05.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:05 smithi107 ceph-mon[35048]: pgmap v535: 129 pgs: 129 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-10T08:13:06.456 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '232ae216', total seconds elapsed: 26.9 2024-09-10T08:13:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.711+0000 7f327ca00640 1 -- 172.21.15.2:0/4163589111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3278076bb0 msgr2=0x7f3278076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.711+0000 7f327ca00640 1 --2- 172.21.15.2:0/4163589111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3278076bb0 0x7f3278076fb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3268009900 tx=0x7f326802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.711+0000 7f327ca00640 1 -- 172.21.15.2:0/4163589111 shutdown_connections 2024-09-10T08:13:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.711+0000 7f327ca00640 1 --2- 172.21.15.2:0/4163589111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3278078790 0x7f3278079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.711+0000 7f327ca00640 1 --2- 172.21.15.2:0/4163589111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3278077df0 0x7f3278078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.711+0000 7f327ca00640 1 --2- 172.21.15.2:0/4163589111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3278076bb0 0x7f3278076fb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.711+0000 7f327ca00640 1 -- 172.21.15.2:0/4163589111 >> 172.21.15.2:0/4163589111 conn(0x7f32780ff4e0 msgr2=0x7f3278101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.711+0000 7f327ca00640 1 -- 172.21.15.2:0/4163589111 shutdown_connections 2024-09-10T08:13:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.711+0000 7f327ca00640 1 -- 172.21.15.2:0/4163589111 wait complete. 2024-09-10T08:13:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.712+0000 7f327ca00640 1 Processor -- start 2024-09-10T08:13:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.712+0000 7f327ca00640 1 -- start start 2024-09-10T08:13:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.712+0000 7f327ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3278076bb0 0x7f327819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.712+0000 7f327ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3278077df0 0x7f327819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.712+0000 7f327ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3278078790 0x7f32781a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.712+0000 7f327ca00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3278113df0 con 0x7f3278076bb0 2024-09-10T08:13:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.712+0000 7f327ca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3278113c70 con 0x7f3278077df0 2024-09-10T08:13:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.712+0000 7f327ca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3278113f70 con 0x7f3278078790 2024-09-10T08:13:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.712+0000 7f3275e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3278076bb0 0x7f327819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.712+0000 7f3275e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3278076bb0 0x7f327819e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34120/0 (socket says 172.21.15.2:34120) 2024-09-10T08:13:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.712+0000 7f3275e00640 1 -- 172.21.15.2:0/1465583876 learned_addr learned my addr 172.21.15.2:0/1465583876 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.713+0000 7f326fe00640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3278077df0 0x7f327819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.713+0000 7f3276800640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3278078790 0x7f32781a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.713+0000 7f3275e00640 1 -- 172.21.15.2:0/1465583876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3278078790 msgr2=0x7f32781a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.713+0000 7f3275e00640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3278078790 0x7f32781a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.713+0000 7f3275e00640 1 -- 172.21.15.2:0/1465583876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3278077df0 msgr2=0x7f327819e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.713+0000 7f3275e00640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3278077df0 0x7f327819e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.713+0000 7f3275e00640 1 -- 172.21.15.2:0/1465583876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32781a5ff0 con 0x7f3278076bb0 2024-09-10T08:13:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.713+0000 7f3275e00640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3278076bb0 0x7f327819e3d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3268009230 tx=0x7f32680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.713+0000 7f326ea00640 1 -- 172.21.15.2:0/1465583876 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326803f030 con 0x7f3278076bb0 2024-09-10T08:13:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.713+0000 7f326ea00640 1 -- 172.21.15.2:0/1465583876 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3268003ce0 con 0x7f3278076bb0 2024-09-10T08:13:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.713+0000 7f327ca00640 1 -- 172.21.15.2:0/1465583876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32781a6280 con 0x7f3278076bb0 2024-09-10T08:13:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.714+0000 7f326ea00640 1 -- 172.21.15.2:0/1465583876 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268035070 con 0x7f3278076bb0 2024-09-10T08:13:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.714+0000 7f327ca00640 1 -- 172.21.15.2:0/1465583876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32781a67c0 con 0x7f3278076bb0 2024-09-10T08:13:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.717+0000 7f327f882740 1 Processor -- start 2024-09-10T08:13:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.718+0000 7f327f882740 1 -- start start 2024-09-10T08:13:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.718+0000 7f327f882740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf090 0x5618aa1bf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.718+0000 7f327f882740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf9d0 0x5618aa1c4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.718+0000 7f326ea00640 1 -- 172.21.15.2:0/1465583876 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3268042030 con 0x7f3278076bb0 2024-09-10T08:13:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.718+0000 7f327f882740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.718+0000 7f327f882740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5618aa124100 con 0x5618aa1bf090 2024-09-10T08:13:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.718+0000 7f327f882740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5618aa0cc1b0 con 0x5618aa1bf9d0 2024-09-10T08:13:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.718+0000 7f327f882740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5618aa1200c0 con 0x5618aa1c4a60 2024-09-10T08:13:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.718+0000 7f326fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.718+0000 7f3276800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf090 0x5618aa1bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.718+0000 7f3275e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf9d0 0x5618aa1c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.718+0000 7f326fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49662/0 (socket says 172.21.15.2:49662) 2024-09-10T08:13:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.718+0000 7f326fe00640 1 -- 172.21.15.2:0/3335406912 learned_addr learned my addr 172.21.15.2:0/3335406912 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.719+0000 7f326fe00640 1 -- 172.21.15.2:0/3335406912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf9d0 msgr2=0x5618aa1c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.719+0000 7f326fe00640 1 --2- 172.21.15.2:0/3335406912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf9d0 0x5618aa1c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.719+0000 7f326fe00640 1 -- 172.21.15.2:0/3335406912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf090 msgr2=0x5618aa1bf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.719+0000 7f326fe00640 1 --2- 172.21.15.2:0/3335406912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf090 0x5618aa1bf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.719+0000 7f326fe00640 1 -- 172.21.15.2:0/3335406912 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618a9e37210 con 0x5618aa1c4a60 2024-09-10T08:13:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.719+0000 7f3276800640 1 --2- 172.21.15.2:0/3335406912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf090 0x5618aa1bf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.719+0000 7f326fe00640 1 --2- 172.21.15.2:0/3335406912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1c6f20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3260009ca0 tx=0x7f3260011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2c546ed8054bd4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.719+0000 7f3267000640 1 -- 172.21.15.2:0/3335406912 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326001e070 con 0x5618aa1c4a60 2024-09-10T08:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.719+0000 7f3267000640 1 -- 172.21.15.2:0/3335406912 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3260019b10 con 0x5618aa1c4a60 2024-09-10T08:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.720+0000 7f3267000640 1 -- 172.21.15.2:0/3335406912 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326000eac0 con 0x5618aa1c4a60 2024-09-10T08:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.720+0000 7f326ea00640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f325c0a2cb0 0x7f325c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.720+0000 7f326ea00640 1 -- 172.21.15.2:0/1465583876 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f32680ef860 con 0x7f3278076bb0 2024-09-10T08:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.720+0000 7f3276800640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f325c0a2cb0 0x7f325c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.720+0000 7f327f882740 1 -- 172.21.15.2:0/3335406912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 msgr2=0x5618aa1c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.720+0000 7f327f882740 1 --2- 172.21.15.2:0/3335406912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1c6f20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3260009ca0 tx=0x7f3260011810 comp rx=0 tx=0).stop 2024-09-10T08:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.720+0000 7f327f882740 1 -- 172.21.15.2:0/3335406912 shutdown_connections 2024-09-10T08:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.720+0000 7f327f882740 1 --2- 172.21.15.2:0/3335406912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1c6f20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.720+0000 7f327f882740 1 --2- 172.21.15.2:0/3335406912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf9d0 0x5618aa1c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.720+0000 7f327f882740 1 --2- 172.21.15.2:0/3335406912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf090 0x5618aa1bf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.720+0000 7f327f882740 1 -- 172.21.15.2:0/3335406912 >> 172.21.15.2:0/3335406912 conn(0x5618aa1ba700 msgr2=0x5618aa1bab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.720+0000 7f327f882740 1 -- 172.21.15.2:0/3335406912 shutdown_connections 2024-09-10T08:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.721+0000 7f327f882740 1 -- 172.21.15.2:0/3335406912 wait complete. 2024-09-10T08:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.721+0000 7f3276800640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f325c0a2cb0 0x7f325c0a5170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3270009fd0 tx=0x7f3270009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.721+0000 7f327f882740 1 -- start start 2024-09-10T08:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.721+0000 7f327f882740 1 Processor -- start 2024-09-10T08:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.722+0000 7f327f882740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf090 0x5618aa1c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.722+0000 7f327f882740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 0x5618aa1c9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.722+0000 7f327f882740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.722+0000 7f327f882740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5618aa1c7a30 con 0x5618aa1bf9d0 2024-09-10T08:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.722+0000 7f327f882740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5618aa0ad090 con 0x5618aa1bf090 2024-09-10T08:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.722+0000 7f327f882740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5618aa1c7bb0 con 0x5618aa1c4a60 2024-09-10T08:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.722+0000 7f327f882740 10 client.0 _ll_register_callbacks cb 0x5618aa101f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.722+0000 7f326fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 0x5618aa1c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.722+0000 7f326fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 0x5618aa1c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34138/0 (socket says 172.21.15.2:34138) 2024-09-10T08:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.722+0000 7f326fe00640 1 -- 172.21.15.2:0/796457774 learned_addr learned my addr 172.21.15.2:0/796457774 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.722+0000 7f3276800640 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.722+0000 7f3275e00640 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf090 0x5618aa1c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.723+0000 7f3275e00640 1 -- 172.21.15.2:0/796457774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 msgr2=0x5618aa1d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.723+0000 7f3275e00640 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1d0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.723+0000 7f3275e00640 1 -- 172.21.15.2:0/796457774 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 msgr2=0x5618aa1c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.723+0000 7f3275e00640 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 0x5618aa1c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.723+0000 7f3275e00640 1 -- 172.21.15.2:0/796457774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618a9f03620 con 0x5618aa1bf090 2024-09-10T08:13:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.723+0000 7f326fe00640 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 0x5618aa1c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.723+0000 7f3275e00640 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf090 0x5618aa1c9210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x5618aa1c0400 tx=0x7f3268003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395d99e17acd9dd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.723+0000 7f325b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.723+0000 7f325b400640 1 -- 172.21.15.2:0/796457774 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268049070 con 0x5618aa1bf090 2024-09-10T08:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.723+0000 7f325b400640 1 -- 172.21.15.2:0/796457774 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3268003800 con 0x5618aa1bf090 2024-09-10T08:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.723+0000 7f327f882740 1 -- 172.21.15.2:0/796457774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3260019b10 con 0x5618aa1bf090 2024-09-10T08:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.723+0000 7f325b400640 1 -- 172.21.15.2:0/796457774 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f32680bb860 con 0x5618aa1bf090 2024-09-10T08:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.723+0000 7f327f882740 10 client.26825 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.724+0000 7f327f882740 1 -- 172.21.15.2:0/796457774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5618aa1d3580 con 0x5618aa1bf090 2024-09-10T08:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.724+0000 7f325b400640 1 -- 172.21.15.2:0/796457774 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3260019b10 con 0x5618aa1bf090 2024-09-10T08:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.724+0000 7f325b400640 1 -- 172.21.15.2:0/796457774 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f32680bbf80 con 0x5618aa1bf090 2024-09-10T08:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.724+0000 7f327f882740 10 client.26825 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.724+0000 7f327f882740 10 client.26825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.724+0000 7f327f882740 20 client.26825 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.724+0000 7f327f882740 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618aa1d4d00 0x5618aa1d7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.724+0000 7f327f882740 4 client.26825 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.724+0000 7f327f882740 1 -- 172.21.15.2:0/796457774 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618aa131830 con 0x5618aa1d4d00 2024-09-10T08:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.724+0000 7f326fe00640 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618aa1d4d00 0x5618aa1d7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.725+0000 7f326fe00640 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618aa1d4d00 0x5618aa1d7130 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c22e6a9de3aa4b server_cookie=707a11c8fa64bab8 in_seq=0 out_seq=0 2024-09-10T08:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.725+0000 7f325b400640 10 client.26825 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f325b400640 1 -- 172.21.15.2:0/796457774 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618aa131830 con 0x5618aa1d4d00 2024-09-10T08:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f325b400640 10 client.26825 handle_command_reply: tid=0 2024-09-10T08:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f327f882740 1 client.26825 shutdown 2024-09-10T08:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f327f882740 1 -- 172.21.15.2:0/796457774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf090 msgr2=0x5618aa1c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f327f882740 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf090 0x5618aa1c9210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x5618aa1c0400 tx=0x7f3268003f50 comp rx=0 tx=0).stop 2024-09-10T08:13:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f327f882740 1 -- 172.21.15.2:0/796457774 shutdown_connections 2024-09-10T08:13:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f327f882740 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618aa1d4d00 0x5618aa1d7130 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f327f882740 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1d0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f327f882740 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 0x5618aa1c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f325b400640 0 client.26825 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f327f882740 1 --2- 172.21.15.2:0/796457774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf090 0x5618aa1c9210 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f327f882740 1 -- 172.21.15.2:0/796457774 >> 172.21.15.2:0/796457774 conn(0x5618aa1ba700 msgr2=0x5618aa1bbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f327f882740 1 -- 172.21.15.2:0/796457774 shutdown_connections 2024-09-10T08:13:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.726+0000 7f327f882740 1 -- 172.21.15.2:0/796457774 wait complete. 2024-09-10T08:13:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.727+0000 7f327f882740 20 client.26825 trim_cache size 0 max 16384 2024-09-10T08:13:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.741+0000 7f327f882740 1 Processor -- start 2024-09-10T08:13:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.741+0000 7f327f882740 1 -- start start 2024-09-10T08:13:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f327f882740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf090 0x5618aa1baae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f327f882740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf9d0 0x5618aa1d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f327f882740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1bb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f327f882740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f32781b3650 con 0x5618aa1bf090 2024-09-10T08:13:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f327f882740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f32781b0f20 con 0x5618aa1bf9d0 2024-09-10T08:13:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f327f882740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f32781aed70 con 0x5618aa1c4a60 2024-09-10T08:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f3275e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf090 0x5618aa1baae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f3275e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf090 0x5618aa1baae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34148/0 (socket says 172.21.15.2:34148) 2024-09-10T08:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f3275e00640 1 -- 172.21.15.2:0/1393226631 learned_addr learned my addr 172.21.15.2:0/1393226631 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f3276800640 1 --2- 172.21.15.2:0/1393226631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1bb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f326fe00640 1 --2- 172.21.15.2:0/1393226631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf9d0 0x5618aa1d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f3275e00640 1 -- 172.21.15.2:0/1393226631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 msgr2=0x5618aa1bb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f3275e00640 1 --2- 172.21.15.2:0/1393226631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1bb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f3275e00640 1 -- 172.21.15.2:0/1393226631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf9d0 msgr2=0x5618aa1d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f3275e00640 1 --2- 172.21.15.2:0/1393226631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf9d0 0x5618aa1d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.742+0000 7f3275e00640 1 -- 172.21.15.2:0/1393226631 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618aa1ba520 con 0x5618aa1bf090 2024-09-10T08:13:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.743+0000 7f3275e00640 1 --2- 172.21.15.2:0/1393226631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf090 0x5618aa1baae0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3268009a20 tx=0x7f3268031ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3409a60c7ac31dca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.743+0000 7f3266600640 1 -- 172.21.15.2:0/1393226631 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32680d8070 con 0x5618aa1bf090 2024-09-10T08:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.743+0000 7f3266600640 1 -- 172.21.15.2:0/1393226631 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32680bbd20 con 0x5618aa1bf090 2024-09-10T08:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.743+0000 7f3266600640 1 -- 172.21.15.2:0/1393226631 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268049070 con 0x5618aa1bf090 2024-09-10T08:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.743+0000 7f327f882740 1 -- 172.21.15.2:0/1393226631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf090 msgr2=0x5618aa1baae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.744+0000 7f327f882740 1 --2- 172.21.15.2:0/1393226631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf090 0x5618aa1baae0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3268009a20 tx=0x7f3268031ad0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.744+0000 7f327f882740 1 -- 172.21.15.2:0/1393226631 shutdown_connections 2024-09-10T08:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.744+0000 7f327f882740 1 --2- 172.21.15.2:0/1393226631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1c4a60 0x5618aa1bb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.744+0000 7f327f882740 1 --2- 172.21.15.2:0/1393226631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1bf9d0 0x5618aa1d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.744+0000 7f327f882740 1 --2- 172.21.15.2:0/1393226631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf090 0x5618aa1baae0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.744+0000 7f327f882740 1 -- 172.21.15.2:0/1393226631 >> 172.21.15.2:0/1393226631 conn(0x5618aa1ba700 msgr2=0x5618aa1c41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.744+0000 7f327f882740 1 -- 172.21.15.2:0/1393226631 shutdown_connections 2024-09-10T08:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.744+0000 7f327f882740 1 -- 172.21.15.2:0/1393226631 wait complete. 2024-09-10T08:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.744+0000 7f327f882740 1 -- start start 2024-09-10T08:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.744+0000 7f327f882740 1 Processor -- start 2024-09-10T08:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f327f882740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1bf090 0x5618aa1cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f327f882740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 0x5618aa1cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f327f882740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1c4a60 0x5618aa23d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f327f882740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f32781af3e0 con 0x5618aa1bf9d0 2024-09-10T08:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f327f882740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f32781b0850 con 0x5618aa1c4a60 2024-09-10T08:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f327f882740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f32781afab0 con 0x5618aa1bf090 2024-09-10T08:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f327f882740 10 client.0 _ll_register_callbacks cb 0x7f32680f5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f326fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 0x5618aa1cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f326fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 0x5618aa1cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34156/0 (socket says 172.21.15.2:34156) 2024-09-10T08:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f326fe00640 1 -- 172.21.15.2:0/3451022918 learned_addr learned my addr 172.21.15.2:0/3451022918 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f3276800640 1 --2- 172.21.15.2:0/3451022918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1c4a60 0x5618aa23d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f3275e00640 1 --2- 172.21.15.2:0/3451022918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1bf090 0x5618aa1cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f326fe00640 1 -- 172.21.15.2:0/3451022918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1bf090 msgr2=0x5618aa1cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f326fe00640 1 --2- 172.21.15.2:0/3451022918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1bf090 0x5618aa1cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f326fe00640 1 -- 172.21.15.2:0/3451022918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1c4a60 msgr2=0x5618aa23d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f326fe00640 1 --2- 172.21.15.2:0/3451022918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1c4a60 0x5618aa23d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.745+0000 7f326fe00640 1 -- 172.21.15.2:0/3451022918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618aa23df10 con 0x5618aa1bf9d0 2024-09-10T08:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.746+0000 7f326fe00640 1 --2- 172.21.15.2:0/3451022918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 0x5618aa1cdb30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3260009250 tx=0x7f32600197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41f747429b1cb7b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.746+0000 7f325b400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.746+0000 7f325b400640 1 -- 172.21.15.2:0/3451022918 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326001e040 con 0x5618aa1bf9d0 2024-09-10T08:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.746+0000 7f325b400640 1 -- 172.21.15.2:0/3451022918 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3260004220 con 0x5618aa1bf9d0 2024-09-10T08:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.746+0000 7f327f882740 1 -- 172.21.15.2:0/3451022918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32680bbd20 con 0x5618aa1bf9d0 2024-09-10T08:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.746+0000 7f325b400640 1 -- 172.21.15.2:0/3451022918 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f32600098a0 con 0x5618aa1bf9d0 2024-09-10T08:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.746+0000 7f327f882740 10 client.16797 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.746+0000 7f327f882740 1 -- 172.21.15.2:0/3451022918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5618aa23f120 con 0x5618aa1bf9d0 2024-09-10T08:13:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.746+0000 7f325b400640 1 -- 172.21.15.2:0/3451022918 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32680bbd20 con 0x5618aa1bf9d0 2024-09-10T08:13:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.747+0000 7f325b400640 1 -- 172.21.15.2:0/3451022918 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3260004920 con 0x5618aa1bf9d0 2024-09-10T08:13:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.747+0000 7f327f882740 10 client.16797 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.747+0000 7f327f882740 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.747+0000 7f327f882740 20 client.16797 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.747+0000 7f327f882740 1 --2- 172.21.15.2:0/3451022918 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618aa23f930 0x5618aa241d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.747+0000 7f327f882740 4 client.16797 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:13:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.747+0000 7f327f882740 1 -- 172.21.15.2:0/3451022918 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618aa12bb80 con 0x5618aa23f930 2024-09-10T08:13:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.747+0000 7f3275e00640 1 --2- 172.21.15.2:0/3451022918 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618aa23f930 0x5618aa241d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:06.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.747+0000 7f3275e00640 1 --2- 172.21.15.2:0/3451022918 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618aa23f930 0x5618aa241d50 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a71f42054f2033 server_cookie=f8e343393d1bac8e in_seq=0 out_seq=0 2024-09-10T08:13:06.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.747+0000 7f325b400640 10 client.16797 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f325b400640 1 -- 172.21.15.2:0/3451022918 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5618aa12bb80 con 0x5618aa23f930 2024-09-10T08:13:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f325b400640 10 client.16797 handle_command_reply: tid=0 2024-09-10T08:13:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f327f882740 1 client.16797 shutdown 2024-09-10T08:13:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f327f882740 1 -- 172.21.15.2:0/3451022918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 msgr2=0x5618aa1cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f327f882740 1 --2- 172.21.15.2:0/3451022918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 0x5618aa1cdb30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3260009250 tx=0x7f32600197d0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f327f882740 1 -- 172.21.15.2:0/3451022918 shutdown_connections 2024-09-10T08:13:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f327f882740 1 --2- 172.21.15.2:0/3451022918 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618aa23f930 0x5618aa241d50 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f327f882740 1 --2- 172.21.15.2:0/3451022918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618aa1c4a60 0x5618aa23d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f327f882740 1 --2- 172.21.15.2:0/3451022918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618aa1bf9d0 0x5618aa1cdb30 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f325b400640 0 client.16797 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f327f882740 1 --2- 172.21.15.2:0/3451022918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618aa1bf090 0x5618aa1cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f327f882740 1 -- 172.21.15.2:0/3451022918 >> 172.21.15.2:0/3451022918 conn(0x5618aa1ba700 msgr2=0x5618aa1b9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f327f882740 1 -- 172.21.15.2:0/3451022918 shutdown_connections 2024-09-10T08:13:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f327f882740 1 -- 172.21.15.2:0/3451022918 wait complete. 2024-09-10T08:13:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.758+0000 7f327f882740 20 client.16797 trim_cache size 0 max 16384 2024-09-10T08:13:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:13:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:13:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:13:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:13:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.759+0000 7f327ca00640 1 -- 172.21.15.2:0/1465583876 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f325c0a2cb0 msgr2=0x7f325c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.759+0000 7f327ca00640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f325c0a2cb0 0x7f325c0a5170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3270009fd0 tx=0x7f3270009500 comp rx=0 tx=0).stop 2024-09-10T08:13:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.759+0000 7f327ca00640 1 -- 172.21.15.2:0/1465583876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3278076bb0 msgr2=0x7f327819e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.759+0000 7f327ca00640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3278076bb0 0x7f327819e3d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3268009230 tx=0x7f32680042f0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.759+0000 7f327ca00640 1 -- 172.21.15.2:0/1465583876 shutdown_connections 2024-09-10T08:13:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.759+0000 7f327ca00640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f325c0a2cb0 0x7f325c0a5170 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.759+0000 7f327ca00640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3278078790 0x7f32781a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.759+0000 7f327ca00640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3278077df0 0x7f327819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.759+0000 7f327ca00640 1 --2- 172.21.15.2:0/1465583876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3278076bb0 0x7f327819e3d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.759+0000 7f327ca00640 1 -- 172.21.15.2:0/1465583876 >> 172.21.15.2:0/1465583876 conn(0x7f32780ff4e0 msgr2=0x7f3278100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.759+0000 7f327ca00640 1 -- 172.21.15.2:0/1465583876 shutdown_connections 2024-09-10T08:13:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:06.759+0000 7f327ca00640 1 -- 172.21.15.2:0/1465583876 wait complete. 2024-09-10T08:13:07.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:07 smithi155 ceph-mon[35405]: pgmap v536: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-10T08:13:07.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:07 smithi107 ceph-mon[35048]: pgmap v536: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-10T08:13:08.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:07 smithi002 ceph-mon[30563]: pgmap v536: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-10T08:13:09.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:09 smithi155 ceph-mon[35405]: pgmap v537: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:13:09.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:09 smithi107 ceph-mon[35048]: pgmap v537: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:13:10.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:09 smithi002 ceph-mon[30563]: pgmap v537: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:13:11.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:11 smithi155 ceph-mon[35405]: pgmap v538: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T08:13:11.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:11 smithi107 ceph-mon[35048]: pgmap v538: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T08:13:12.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:11 smithi002 ceph-mon[30563]: pgmap v538: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T08:13:13.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:13 smithi155 ceph-mon[35405]: pgmap v539: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.6 KiB/s wr, 11 op/s 2024-09-10T08:13:13.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:13 smithi107 ceph-mon[35048]: pgmap v539: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.6 KiB/s wr, 11 op/s 2024-09-10T08:13:14.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:13 smithi002 ceph-mon[30563]: pgmap v539: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.6 KiB/s wr, 11 op/s 2024-09-10T08:13:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:15 smithi155 ceph-mon[35405]: pgmap v540: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-09-10T08:13:15.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:15 smithi107 ceph-mon[35048]: pgmap v540: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-09-10T08:13:16.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:15 smithi002 ceph-mon[30563]: pgmap v540: 129 pgs: 129 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-09-10T08:13:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:17 smithi155 ceph-mon[35405]: pgmap v541: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:13:17.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:17 smithi107 ceph-mon[35048]: pgmap v541: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:13:18.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:17 smithi002 ceph-mon[30563]: pgmap v541: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:13:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.886+0000 7ff6b1800640 1 -- 172.21.15.2:0/564091537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ac104f50 msgr2=0x7ff6ac1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.886+0000 7ff6b1800640 1 --2- 172.21.15.2:0/564091537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ac104f50 0x7ff6ac1053d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff694009860 tx=0x7ff69402a340 comp rx=0 tx=0).stop 2024-09-10T08:13:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.886+0000 7ff6b1800640 1 -- 172.21.15.2:0/564091537 shutdown_connections 2024-09-10T08:13:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.886+0000 7ff6b1800640 1 --2- 172.21.15.2:0/564091537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6ac105910 0x7ff6ac10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.886+0000 7ff6b1800640 1 --2- 172.21.15.2:0/564091537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ac104f50 0x7ff6ac1053d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.886+0000 7ff6b1800640 1 --2- 172.21.15.2:0/564091537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6ac103d60 0x7ff6ac104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.886+0000 7ff6b1800640 1 -- 172.21.15.2:0/564091537 >> 172.21.15.2:0/564091537 conn(0x7ff6ac0ff4e0 msgr2=0x7ff6ac101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.887+0000 7ff6b1800640 1 -- 172.21.15.2:0/564091537 shutdown_connections 2024-09-10T08:13:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.887+0000 7ff6b1800640 1 -- 172.21.15.2:0/564091537 wait complete. 2024-09-10T08:13:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.887+0000 7ff6b1800640 1 Processor -- start 2024-09-10T08:13:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6b1800640 1 -- start start 2024-09-10T08:13:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6b1800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ac103d60 0x7ff6ac19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6b1800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6ac104f50 0x7ff6ac19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6b1800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6ac105910 0x7ff6ac1a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6b1800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac10fa40 con 0x7ff6ac105910 2024-09-10T08:13:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6b1800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac10f8c0 con 0x7ff6ac103d60 2024-09-10T08:13:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6b1800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac10fbc0 con 0x7ff6ac104f50 2024-09-10T08:13:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6aaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ac103d60 0x7ff6ac19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6ab400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6ac105910 0x7ff6ac1a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6aaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ac103d60 0x7ff6ac19a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:43562/0 (socket says 172.21.15.2:43562) 2024-09-10T08:13:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6aaa00640 1 -- 172.21.15.2:0/1768594267 learned_addr learned my addr 172.21.15.2:0/1768594267 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6a3e00640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6ac104f50 0x7ff6ac19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6aaa00640 1 -- 172.21.15.2:0/1768594267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6ac104f50 msgr2=0x7ff6ac19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6aaa00640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6ac104f50 0x7ff6ac19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6aaa00640 1 -- 172.21.15.2:0/1768594267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6ac105910 msgr2=0x7ff6ac1a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.888+0000 7ff6aaa00640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6ac105910 0x7ff6ac1a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.889+0000 7ff6aaa00640 1 -- 172.21.15.2:0/1768594267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac1a1c60 con 0x7ff6ac103d60 2024-09-10T08:13:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.889+0000 7ff6a3e00640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6ac104f50 0x7ff6ac19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:13:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.889+0000 7ff6aaa00640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ac103d60 0x7ff6ac19a040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00e9a0 tx=0x7ff69c00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.889+0000 7ff6a3400640 1 -- 172.21.15.2:0/1768594267 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00cd80 con 0x7ff6ac103d60 2024-09-10T08:13:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.889+0000 7ff6b1800640 1 -- 172.21.15.2:0/1768594267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ac1a1f50 con 0x7ff6ac103d60 2024-09-10T08:13:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.889+0000 7ff6a3400640 1 -- 172.21.15.2:0/1768594267 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69c010910 con 0x7ff6ac103d60 2024-09-10T08:13:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.889+0000 7ff6a3400640 1 -- 172.21.15.2:0/1768594267 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c018a40 con 0x7ff6ac103d60 2024-09-10T08:13:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.889+0000 7ff6b1800640 1 -- 172.21.15.2:0/1768594267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ac1a2440 con 0x7ff6ac103d60 2024-09-10T08:13:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.892+0000 7ff6b4610740 1 Processor -- start 2024-09-10T08:13:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.892+0000 7ff6b4610740 1 -- start start 2024-09-10T08:13:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.892+0000 7ff6a3400640 1 -- 172.21.15.2:0/1768594267 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff69c010ac0 con 0x7ff6ac103d60 2024-09-10T08:13:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6b4610740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf090 0x55fb3afbf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6b4610740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf9d0 0x55fb3afc4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6b4610740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afc4a60 0x55fb3afc6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6b4610740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fb3af24100 con 0x55fb3afbf9d0 2024-09-10T08:13:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6b4610740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fb3aecc1b0 con 0x55fb3afc4a60 2024-09-10T08:13:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6b4610740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fb3af200c0 con 0x55fb3afbf090 2024-09-10T08:13:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6aaa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf9d0 0x55fb3afc4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6aaa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf9d0 0x55fb3afc4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34716/0 (socket says 172.21.15.2:34716) 2024-09-10T08:13:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6aaa00640 1 -- 172.21.15.2:0/565340340 learned_addr learned my addr 172.21.15.2:0/565340340 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6ab400640 1 --2- 172.21.15.2:0/565340340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf090 0x55fb3afbf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6a3e00640 1 --2- 172.21.15.2:0/565340340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afc4a60 0x55fb3afc6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6ab400640 1 -- 172.21.15.2:0/565340340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afc4a60 msgr2=0x55fb3afc6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6ab400640 1 --2- 172.21.15.2:0/565340340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afc4a60 0x55fb3afc6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6ab400640 1 -- 172.21.15.2:0/565340340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf9d0 msgr2=0x55fb3afc4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6ab400640 1 --2- 172.21.15.2:0/565340340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf9d0 0x55fb3afc4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6ab400640 1 -- 172.21.15.2:0/565340340 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb3ac37210 con 0x55fb3afbf090 2024-09-10T08:13:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.893+0000 7ff6aaa00640 1 --2- 172.21.15.2:0/565340340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf9d0 0x55fb3afc4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.894+0000 7ff6ab400640 1 --2- 172.21.15.2:0/565340340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf090 0x55fb3afbf490 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009f70 tx=0x7ff6a4009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e70421d3e4895e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.894+0000 7ff69be00640 1 -- 172.21.15.2:0/565340340 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4013850 con 0x55fb3afbf090 2024-09-10T08:13:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.894+0000 7ff69be00640 1 -- 172.21.15.2:0/565340340 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6a4019070 con 0x55fb3afbf090 2024-09-10T08:13:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.894+0000 7ff69be00640 1 -- 172.21.15.2:0/565340340 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a40122e0 con 0x55fb3afbf090 2024-09-10T08:13:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.894+0000 7ff6a3400640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff6900a2c30 0x7ff6900a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.894+0000 7ff6a3400640 1 -- 172.21.15.2:0/1768594267 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff69c0cf870 con 0x7ff6ac103d60 2024-09-10T08:13:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.894+0000 7ff6aaa00640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff6900a2c30 0x7ff6900a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.895+0000 7ff6b4610740 1 -- 172.21.15.2:0/565340340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf090 msgr2=0x55fb3afbf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.895+0000 7ff6b4610740 1 --2- 172.21.15.2:0/565340340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf090 0x55fb3afbf490 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009f70 tx=0x7ff6a4009650 comp rx=0 tx=0).stop 2024-09-10T08:13:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.895+0000 7ff6b4610740 1 -- 172.21.15.2:0/565340340 shutdown_connections 2024-09-10T08:13:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.895+0000 7ff6b4610740 1 --2- 172.21.15.2:0/565340340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afc4a60 0x55fb3afc6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.895+0000 7ff6b4610740 1 --2- 172.21.15.2:0/565340340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf9d0 0x55fb3afc4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.895+0000 7ff6b4610740 1 --2- 172.21.15.2:0/565340340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf090 0x55fb3afbf490 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.895+0000 7ff6b4610740 1 -- 172.21.15.2:0/565340340 >> 172.21.15.2:0/565340340 conn(0x55fb3afba700 msgr2=0x55fb3afbab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.895+0000 7ff6b4610740 1 -- 172.21.15.2:0/565340340 shutdown_connections 2024-09-10T08:13:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.895+0000 7ff6b4610740 1 -- 172.21.15.2:0/565340340 wait complete. 2024-09-10T08:13:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.895+0000 7ff6aaa00640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff6900a2c30 0x7ff6900a50f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0cfb10 tx=0x7ff69c0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.896+0000 7ff6b4610740 1 -- start start 2024-09-10T08:13:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.896+0000 7ff6b4610740 1 Processor -- start 2024-09-10T08:13:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.896+0000 7ff6b4610740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 0x55fb3afc91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.896+0000 7ff6b4610740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afbf9d0 0x55fb3afc9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.896+0000 7ff6b4610740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afc4a60 0x55fb3afd0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.896+0000 7ff6b4610740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fb3afc7a30 con 0x55fb3afbf090 2024-09-10T08:13:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.896+0000 7ff6b4610740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fb3aead090 con 0x55fb3afbf9d0 2024-09-10T08:13:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.896+0000 7ff6b4610740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fb3afc7bb0 con 0x55fb3afc4a60 2024-09-10T08:13:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.896+0000 7ff6b4610740 10 client.0 _ll_register_callbacks cb 0x55fb3af01f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.896+0000 7ff6ab400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 0x55fb3afc91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.897+0000 7ff6a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afbf9d0 0x55fb3afc9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.897+0000 7ff6ab400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 0x55fb3afc91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34718/0 (socket says 172.21.15.2:34718) 2024-09-10T08:13:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.897+0000 7ff6ab400640 1 -- 172.21.15.2:0/1455683616 learned_addr learned my addr 172.21.15.2:0/1455683616 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.897+0000 7ff6ab400640 1 --2- 172.21.15.2:0/1455683616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afc4a60 0x55fb3afd0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.897+0000 7ff6a3e00640 1 -- 172.21.15.2:0/1455683616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afc4a60 msgr2=0x55fb3afd0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.897+0000 7ff6a3e00640 1 --2- 172.21.15.2:0/1455683616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afc4a60 0x55fb3afd0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.897+0000 7ff6a3e00640 1 -- 172.21.15.2:0/1455683616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 msgr2=0x55fb3afc91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.897+0000 7ff6a3e00640 1 --2- 172.21.15.2:0/1455683616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 0x55fb3afc91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.897+0000 7ff6a3e00640 1 -- 172.21.15.2:0/1455683616 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb3ad03620 con 0x55fb3afbf9d0 2024-09-10T08:13:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.897+0000 7ff6ab400640 1 --2- 172.21.15.2:0/1455683616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 0x55fb3afc91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.897+0000 7ff6a3e00640 1 --2- 172.21.15.2:0/1455683616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afbf9d0 0x55fb3afc9700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff694035040 tx=0x7ff6940307b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f68e0367f0d7798 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.897+0000 7ff698c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:13:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.898+0000 7ff698c00640 1 -- 172.21.15.2:0/1455683616 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694037040 con 0x55fb3afbf9d0 2024-09-10T08:13:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.898+0000 7ff6b4610740 1 -- 172.21.15.2:0/1455683616 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6a4019070 con 0x55fb3afbf9d0 2024-09-10T08:13:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.898+0000 7ff698c00640 1 -- 172.21.15.2:0/1455683616 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff694004450 con 0x55fb3afbf9d0 2024-09-10T08:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.898+0000 7ff698c00640 1 -- 172.21.15.2:0/1455683616 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff69404a050 con 0x55fb3afbf9d0 2024-09-10T08:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.898+0000 7ff6b4610740 10 client.26843 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.898+0000 7ff6b4610740 1 -- 172.21.15.2:0/1455683616 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fb3afd35a0 con 0x55fb3afbf9d0 2024-09-10T08:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.898+0000 7ff698c00640 1 -- 172.21.15.2:0/1455683616 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6a4019070 con 0x55fb3afbf9d0 2024-09-10T08:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.899+0000 7ff698c00640 1 -- 172.21.15.2:0/1455683616 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff694045030 con 0x55fb3afbf9d0 2024-09-10T08:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.899+0000 7ff6b4610740 10 client.26843 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.899+0000 7ff6b4610740 10 client.26843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.899+0000 7ff6b4610740 20 client.26843 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.899+0000 7ff6b4610740 1 --2- 172.21.15.2:0/1455683616 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb3afd4c70 0x55fb3afd7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.899+0000 7ff6b4610740 4 client.26843 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.899+0000 7ff6b4610740 1 -- 172.21.15.2:0/1455683616 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb3af31830 con 0x55fb3afd4c70 2024-09-10T08:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.899+0000 7ff6ab400640 1 --2- 172.21.15.2:0/1455683616 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb3afd4c70 0x55fb3afd7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.900+0000 7ff6ab400640 1 --2- 172.21.15.2:0/1455683616 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb3afd4c70 0x55fb3afd7080 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd1c691eb40515ba server_cookie=57eaebd620985049 in_seq=0 out_seq=0 2024-09-10T08:13:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.900+0000 7ff698c00640 10 client.26843 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.900+0000 7ff698c00640 1 -- 172.21.15.2:0/1455683616 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb3af31830 con 0x55fb3afd4c70 2024-09-10T08:13:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.900+0000 7ff698c00640 10 client.26843 handle_command_reply: tid=0 2024-09-10T08:13:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.900+0000 7ff6b4610740 1 client.26843 shutdown 2024-09-10T08:13:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.901+0000 7ff6b4610740 1 -- 172.21.15.2:0/1455683616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afbf9d0 msgr2=0x55fb3afc9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.901+0000 7ff6b4610740 1 --2- 172.21.15.2:0/1455683616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afbf9d0 0x55fb3afc9700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff694035040 tx=0x7ff6940307b0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.901+0000 7ff6b4610740 1 -- 172.21.15.2:0/1455683616 shutdown_connections 2024-09-10T08:13:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.901+0000 7ff6b4610740 1 --2- 172.21.15.2:0/1455683616 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb3afd4c70 0x55fb3afd7080 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.901+0000 7ff6b4610740 1 --2- 172.21.15.2:0/1455683616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afc4a60 0x55fb3afd0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.901+0000 7ff6b4610740 1 --2- 172.21.15.2:0/1455683616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afbf9d0 0x55fb3afc9700 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.901+0000 7ff6b4610740 1 --2- 172.21.15.2:0/1455683616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 0x55fb3afc91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.901+0000 7ff698c00640 0 client.26843 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.901+0000 7ff6b4610740 1 -- 172.21.15.2:0/1455683616 >> 172.21.15.2:0/1455683616 conn(0x55fb3afba700 msgr2=0x55fb3afbbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.901+0000 7ff6b4610740 1 -- 172.21.15.2:0/1455683616 shutdown_connections 2024-09-10T08:13:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.901+0000 7ff6b4610740 1 -- 172.21.15.2:0/1455683616 wait complete. 2024-09-10T08:13:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.901+0000 7ff6b4610740 20 client.26843 trim_cache size 0 max 16384 2024-09-10T08:13:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:19 smithi155 ceph-mon[35405]: pgmap v542: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:13:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.916+0000 7ff6b4610740 1 Processor -- start 2024-09-10T08:13:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.916+0000 7ff6b4610740 1 -- start start 2024-09-10T08:13:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6b4610740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afbf090 0x55fb3afbaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6b4610740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf9d0 0x55fb3afd0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6b4610740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afc4a60 0x55fb3afbb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6b4610740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1af420 con 0x55fb3afc4a60 2024-09-10T08:13:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6b4610740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1accf0 con 0x55fb3afbf090 2024-09-10T08:13:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6b4610740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1aab40 con 0x55fb3afbf9d0 2024-09-10T08:13:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afbf090 0x55fb3afbaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6a3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf9d0 0x55fb3afd0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afbf090 0x55fb3afbaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:43590/0 (socket says 172.21.15.2:43590) 2024-09-10T08:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6ab400640 1 -- 172.21.15.2:0/746532307 learned_addr learned my addr 172.21.15.2:0/746532307 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6ab400640 1 --2- 172.21.15.2:0/746532307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afc4a60 0x55fb3afbb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6a3e00640 1 -- 172.21.15.2:0/746532307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afbf090 msgr2=0x55fb3afbaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6a3e00640 1 --2- 172.21.15.2:0/746532307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afbf090 0x55fb3afbaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6a3e00640 1 -- 172.21.15.2:0/746532307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afc4a60 msgr2=0x55fb3afbb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6a3e00640 1 --2- 172.21.15.2:0/746532307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afc4a60 0x55fb3afbb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.917+0000 7ff6a3e00640 1 -- 172.21.15.2:0/746532307 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb3afc0660 con 0x55fb3afbf9d0 2024-09-10T08:13:19.919 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:19 smithi107 ceph-mon[35048]: pgmap v542: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:13:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.918+0000 7ff6a3e00640 1 --2- 172.21.15.2:0/746532307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf9d0 0x55fb3afd0700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff694004e30 tx=0x7ff6940096b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bba1d7c58e353d3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.918+0000 7ff69b400640 1 -- 172.21.15.2:0/746532307 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694037030 con 0x55fb3afbf9d0 2024-09-10T08:13:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.918+0000 7ff69b400640 1 -- 172.21.15.2:0/746532307 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff694004250 con 0x55fb3afbf9d0 2024-09-10T08:13:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.918+0000 7ff69b400640 1 -- 172.21.15.2:0/746532307 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694003e40 con 0x55fb3afbf9d0 2024-09-10T08:13:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.919+0000 7ff6b4610740 1 -- 172.21.15.2:0/746532307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf9d0 msgr2=0x55fb3afd0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.919+0000 7ff6b4610740 1 --2- 172.21.15.2:0/746532307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf9d0 0x55fb3afd0700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff694004e30 tx=0x7ff6940096b0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.919+0000 7ff6b4610740 1 -- 172.21.15.2:0/746532307 shutdown_connections 2024-09-10T08:13:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.919+0000 7ff6b4610740 1 --2- 172.21.15.2:0/746532307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afc4a60 0x55fb3afbb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.919+0000 7ff6b4610740 1 --2- 172.21.15.2:0/746532307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf9d0 0x55fb3afd0700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.919+0000 7ff6b4610740 1 --2- 172.21.15.2:0/746532307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afbf090 0x55fb3afbaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.919+0000 7ff6b4610740 1 -- 172.21.15.2:0/746532307 >> 172.21.15.2:0/746532307 conn(0x55fb3afba700 msgr2=0x55fb3afc91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.919+0000 7ff6b4610740 1 -- 172.21.15.2:0/746532307 shutdown_connections 2024-09-10T08:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.919+0000 7ff6b4610740 1 -- 172.21.15.2:0/746532307 wait complete. 2024-09-10T08:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.919+0000 7ff6b4610740 1 -- start start 2024-09-10T08:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.919+0000 7ff6b4610740 1 Processor -- start 2024-09-10T08:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.920+0000 7ff6b4610740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 0x55fb3afd05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.920+0000 7ff6b4610740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf9d0 0x55fb3afcd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.920+0000 7ff6b4610740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afc4a60 0x55fb3afcd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.920+0000 7ff6b4610740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1ab1b0 con 0x55fb3afbf090 2024-09-10T08:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.920+0000 7ff6b4610740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1ac620 con 0x55fb3afc4a60 2024-09-10T08:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.920+0000 7ff6b4610740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1ab880 con 0x55fb3afbf9d0 2024-09-10T08:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.920+0000 7ff6b4610740 10 client.0 _ll_register_callbacks cb 0x7ff69c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.920+0000 7ff6ab400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 0x55fb3afd05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.920+0000 7ff6ab400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 0x55fb3afd05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34738/0 (socket says 172.21.15.2:34738) 2024-09-10T08:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.920+0000 7ff6ab400640 1 -- 172.21.15.2:0/1774045971 learned_addr learned my addr 172.21.15.2:0/1774045971 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.920+0000 7ff6a3e00640 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf9d0 0x55fb3afcd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.920+0000 7ff6ab400640 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afc4a60 0x55fb3afcd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff6ab400640 1 -- 172.21.15.2:0/1774045971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf9d0 msgr2=0x55fb3afcd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff6ab400640 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf9d0 0x55fb3afcd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff6ab400640 1 -- 172.21.15.2:0/1774045971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afc4a60 msgr2=0x55fb3afcd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff6ab400640 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afc4a60 0x55fb3afcd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff6ab400640 1 -- 172.21.15.2:0/1774045971 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb3b03d950 con 0x55fb3afbf090 2024-09-10T08:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff6a3e00640 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf9d0 0x55fb3afcd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff6ab400640 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 0x55fb3afd05d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009a10 tx=0x7ff6a4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d39c826e4212616 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff698c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff698c00640 1 -- 172.21.15.2:0/1774045971 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a402a040 con 0x55fb3afbf090 2024-09-10T08:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff6b4610740 1 -- 172.21.15.2:0/1774045971 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff694004250 con 0x55fb3afbf090 2024-09-10T08:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff698c00640 1 -- 172.21.15.2:0/1774045971 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff6a4013910 con 0x55fb3afbf090 2024-09-10T08:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff698c00640 1 -- 172.21.15.2:0/1774045971 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6a4013b10 con 0x55fb3afbf090 2024-09-10T08:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff6b4610740 10 client.16815 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.921+0000 7ff6b4610740 1 -- 172.21.15.2:0/1774045971 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fb3b03ec70 con 0x55fb3afbf090 2024-09-10T08:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.922+0000 7ff698c00640 1 -- 172.21.15.2:0/1774045971 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff694004250 con 0x55fb3afbf090 2024-09-10T08:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.922+0000 7ff698c00640 1 -- 172.21.15.2:0/1774045971 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff6a402b280 con 0x55fb3afbf090 2024-09-10T08:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.922+0000 7ff6b4610740 10 client.16815 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.922+0000 7ff6b4610740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.922+0000 7ff6b4610740 20 client.16815 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.922+0000 7ff6b4610740 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb3b03f2f0 0x55fb3b0416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.922+0000 7ff6b4610740 4 client.16815 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:13:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.922+0000 7ff6b4610740 1 -- 172.21.15.2:0/1774045971 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb3af2bb80 con 0x55fb3b03f2f0 2024-09-10T08:13:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.922+0000 7ff6a3e00640 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb3b03f2f0 0x55fb3b0416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.923+0000 7ff6a3e00640 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb3b03f2f0 0x55fb3b0416e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea009fe3f603a612 server_cookie=a92afab064b7833c in_seq=0 out_seq=0 2024-09-10T08:13:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.923+0000 7ff698c00640 10 client.16815 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff698c00640 1 -- 172.21.15.2:0/1774045971 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb3af2bb80 con 0x55fb3b03f2f0 2024-09-10T08:13:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff698c00640 10 client.16815 handle_command_reply: tid=0 2024-09-10T08:13:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff6b4610740 1 client.16815 shutdown 2024-09-10T08:13:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff6b4610740 1 -- 172.21.15.2:0/1774045971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 msgr2=0x55fb3afd05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff6b4610740 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 0x55fb3afd05d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009a10 tx=0x7ff6a4009650 comp rx=0 tx=0).stop 2024-09-10T08:13:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff6b4610740 1 -- 172.21.15.2:0/1774045971 shutdown_connections 2024-09-10T08:13:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff6b4610740 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb3b03f2f0 0x55fb3b0416e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff6b4610740 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb3afc4a60 0x55fb3afcd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff6b4610740 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb3afbf9d0 0x55fb3afcd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff698c00640 0 client.16815 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff6b4610740 1 --2- 172.21.15.2:0/1774045971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb3afbf090 0x55fb3afd05d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff6b4610740 1 -- 172.21.15.2:0/1774045971 >> 172.21.15.2:0/1774045971 conn(0x55fb3afba700 msgr2=0x55fb3afb9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff6b4610740 1 -- 172.21.15.2:0/1774045971 shutdown_connections 2024-09-10T08:13:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.937+0000 7ff6b4610740 1 -- 172.21.15.2:0/1774045971 wait complete. 2024-09-10T08:13:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.938+0000 7ff6b4610740 20 client.16815 trim_cache size 0 max 16384 2024-09-10T08:13:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:13:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:13:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:13:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:13:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.938+0000 7ff6b1800640 1 -- 172.21.15.2:0/1768594267 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff6900a2c30 msgr2=0x7ff6900a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.938+0000 7ff6b1800640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff6900a2c30 0x7ff6900a50f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0cfb10 tx=0x7ff69c0d5530 comp rx=0 tx=0).stop 2024-09-10T08:13:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.938+0000 7ff6b1800640 1 -- 172.21.15.2:0/1768594267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ac103d60 msgr2=0x7ff6ac19a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.938+0000 7ff6b1800640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ac103d60 0x7ff6ac19a040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00e9a0 tx=0x7ff69c00ee60 comp rx=0 tx=0).stop 2024-09-10T08:13:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.938+0000 7ff6b1800640 1 -- 172.21.15.2:0/1768594267 shutdown_connections 2024-09-10T08:13:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.938+0000 7ff6b1800640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6ac105910 0x7ff6ac1a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.938+0000 7ff6b1800640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff6900a2c30 0x7ff6900a50f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.938+0000 7ff6b1800640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6ac104f50 0x7ff6ac19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.938+0000 7ff6b1800640 1 --2- 172.21.15.2:0/1768594267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ac103d60 0x7ff6ac19a040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.938+0000 7ff6b1800640 1 -- 172.21.15.2:0/1768594267 >> 172.21.15.2:0/1768594267 conn(0x7ff6ac0ff4e0 msgr2=0x7ff6ac108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.938+0000 7ff6b1800640 1 -- 172.21.15.2:0/1768594267 shutdown_connections 2024-09-10T08:13:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:19.938+0000 7ff6b1800640 1 -- 172.21.15.2:0/1768594267 wait complete. 2024-09-10T08:13:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:19 smithi002 ceph-mon[30563]: pgmap v542: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:13:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:21 smithi002 ceph-mon[30563]: pgmap v543: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:13:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:21 smithi155 ceph-mon[35405]: pgmap v543: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:13:21.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:21 smithi107 ceph-mon[35048]: pgmap v543: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:13:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:23 smithi155 ceph-mon[35405]: pgmap v544: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:13:23.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:23 smithi107 ceph-mon[35048]: pgmap v544: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:13:24.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:23 smithi002 ceph-mon[30563]: pgmap v544: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:13:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:25 smithi155 ceph-mon[35405]: pgmap v545: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:13:25.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:25 smithi107 ceph-mon[35048]: pgmap v545: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:13:26.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:25 smithi002 ceph-mon[30563]: pgmap v545: 129 pgs: 129 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:13:26.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:13:27.207 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:13:27.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.787+0000 7f8f93200640 1 -- 172.21.15.2:0/3205292596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f941019c0 msgr2=0x7f8f94101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:27.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.787+0000 7f8f93200640 1 --2- 172.21.15.2:0/3205292596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f941019c0 0x7f8f94101e20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009930 tx=0x7f8f7c02a420 comp rx=0 tx=0).stop 2024-09-10T08:13:27.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.787+0000 7f8f93200640 1 -- 172.21.15.2:0/3205292596 shutdown_connections 2024-09-10T08:13:27.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.787+0000 7f8f93200640 1 --2- 172.21.15.2:0/3205292596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f94102360 0x7f8f9410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.787+0000 7f8f93200640 1 --2- 172.21.15.2:0/3205292596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f941019c0 0x7f8f94101e20 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.787+0000 7f8f93200640 1 --2- 172.21.15.2:0/3205292596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f941079c0 0x7f8f94107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.787+0000 7f8f93200640 1 -- 172.21.15.2:0/3205292596 >> 172.21.15.2:0/3205292596 conn(0x7f8f940fd740 msgr2=0x7f8f940ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.787+0000 7f8f93200640 1 -- 172.21.15.2:0/3205292596 shutdown_connections 2024-09-10T08:13:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.787+0000 7f8f93200640 1 -- 172.21.15.2:0/3205292596 wait complete. 2024-09-10T08:13:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.788+0000 7f8f93200640 1 Processor -- start 2024-09-10T08:13:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.788+0000 7f8f93200640 1 -- start start 2024-09-10T08:13:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f93200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f941019c0 0x7f8f9419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f93200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f94102360 0x7f8f9419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f93200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f941079c0 0x7f8f9419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f93200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94111e10 con 0x7f8f941019c0 2024-09-10T08:13:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f93200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94111c90 con 0x7f8f941079c0 2024-09-10T08:13:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f93200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94111f90 con 0x7f8f94102360 2024-09-10T08:13:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f91e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f941019c0 0x7f8f9419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f91e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f941019c0 0x7f8f9419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52640/0 (socket says 172.21.15.2:52640) 2024-09-10T08:13:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f91e00640 1 -- 172.21.15.2:0/2997575012 learned_addr learned my addr 172.21.15.2:0/2997575012 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f92800640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f941079c0 0x7f8f9419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f8be00640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f94102360 0x7f8f9419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f91e00640 1 -- 172.21.15.2:0/2997575012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f94102360 msgr2=0x7f8f9419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f91e00640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f94102360 0x7f8f9419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f91e00640 1 -- 172.21.15.2:0/2997575012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f941079c0 msgr2=0x7f8f9419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f91e00640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f941079c0 0x7f8f9419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.789+0000 7f8f91e00640 1 -- 172.21.15.2:0/2997575012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9419f4a0 con 0x7f8f941019c0 2024-09-10T08:13:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.790+0000 7f8f91e00640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f941019c0 0x7f8f9419a4f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400d770 tx=0x7f8f8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.790+0000 7f8f8aa00640 1 -- 172.21.15.2:0/2997575012 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84017070 con 0x7f8f941019c0 2024-09-10T08:13:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.790+0000 7f8f8aa00640 1 -- 172.21.15.2:0/2997575012 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f84005500 con 0x7f8f941019c0 2024-09-10T08:13:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.790+0000 7f8f8aa00640 1 -- 172.21.15.2:0/2997575012 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84004e00 con 0x7f8f941019c0 2024-09-10T08:13:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.790+0000 7f8f93200640 1 -- 172.21.15.2:0/2997575012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9419f790 con 0x7f8f941019c0 2024-09-10T08:13:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.790+0000 7f8f93200640 1 -- 172.21.15.2:0/2997575012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f941a7070 con 0x7f8f941019c0 2024-09-10T08:13:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.794+0000 7f8f9ba71740 1 Processor -- start 2024-09-10T08:13:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.795+0000 7f8f9ba71740 1 -- start start 2024-09-10T08:13:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.795+0000 7f8f9ba71740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32ead50 0x5639c32eb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.795+0000 7f8f9ba71740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32eb670 0x5639c32f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.795+0000 7f8f9ba71740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32f5f30 0x5639c32f8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.795+0000 7f8f8aa00640 1 -- 172.21.15.2:0/2997575012 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8f84004fa0 con 0x7f8f941019c0 2024-09-10T08:13:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.795+0000 7f8f9ba71740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5639c32513e0 con 0x5639c32eb670 2024-09-10T08:13:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.795+0000 7f8f9ba71740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5639c31f8d80 con 0x5639c32f5f30 2024-09-10T08:13:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.795+0000 7f8f9ba71740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5639c324d3a0 con 0x5639c32ead50 2024-09-10T08:13:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.796+0000 7f8f92800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32ead50 0x5639c32eb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.796+0000 7f8f8be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32f5f30 0x5639c32f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.796+0000 7f8f8be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32f5f30 0x5639c32f8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54062/0 (socket says 172.21.15.2:54062) 2024-09-10T08:13:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.796+0000 7f8f8be00640 1 -- 172.21.15.2:0/1912637421 learned_addr learned my addr 172.21.15.2:0/1912637421 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.796+0000 7f8f92800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32ead50 0x5639c32eb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:45042/0 (socket says 172.21.15.2:45042) 2024-09-10T08:13:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.796+0000 7f8f91e00640 1 --2- 172.21.15.2:0/1912637421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32eb670 0x5639c32f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.796+0000 7f8f8be00640 1 -- 172.21.15.2:0/1912637421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32ead50 msgr2=0x5639c32eb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.796+0000 7f8f8be00640 1 --2- 172.21.15.2:0/1912637421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32ead50 0x5639c32eb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.796+0000 7f8f8be00640 1 -- 172.21.15.2:0/1912637421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32eb670 msgr2=0x5639c32f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.796+0000 7f8f8be00640 1 --2- 172.21.15.2:0/1912637421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32eb670 0x5639c32f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.796+0000 7f8f8be00640 1 -- 172.21.15.2:0/1912637421 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639c3217ae0 con 0x5639c32f5f30 2024-09-10T08:13:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.796+0000 7f8f8be00640 1 --2- 172.21.15.2:0/1912637421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32f5f30 0x5639c32f8320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009930 tx=0x7f8f7c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f75b3cc13086302 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:27.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.797+0000 7f8f83400640 1 -- 172.21.15.2:0/1912637421 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c035040 con 0x5639c32f5f30 2024-09-10T08:13:27.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.797+0000 7f8f83400640 1 -- 172.21.15.2:0/1912637421 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f7c005520 con 0x5639c32f5f30 2024-09-10T08:13:27.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.797+0000 7f8f83400640 1 -- 172.21.15.2:0/1912637421 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c003850 con 0x5639c32f5f30 2024-09-10T08:13:27.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.797+0000 7f8f8aa00640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8f780a2b80 0x7f8f780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.797+0000 7f8f8aa00640 1 -- 172.21.15.2:0/2997575012 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8f840cee00 con 0x7f8f941019c0 2024-09-10T08:13:27.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.797+0000 7f8f92800640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8f780a2b80 0x7f8f780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.798+0000 7f8f9ba71740 1 -- 172.21.15.2:0/1912637421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32f5f30 msgr2=0x5639c32f8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:27.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.798+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/1912637421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32f5f30 0x5639c32f8320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009930 tx=0x7f8f7c005c00 comp rx=0 tx=0).stop 2024-09-10T08:13:27.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.798+0000 7f8f9ba71740 1 -- 172.21.15.2:0/1912637421 shutdown_connections 2024-09-10T08:13:27.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.798+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/1912637421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32f5f30 0x5639c32f8320 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.798+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/1912637421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32eb670 0x5639c32f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.798+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/1912637421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32ead50 0x5639c32eb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.798+0000 7f8f9ba71740 1 -- 172.21.15.2:0/1912637421 >> 172.21.15.2:0/1912637421 conn(0x5639c32e72e0 msgr2=0x5639c32e76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.798+0000 7f8f9ba71740 1 -- 172.21.15.2:0/1912637421 shutdown_connections 2024-09-10T08:13:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.798+0000 7f8f9ba71740 1 -- 172.21.15.2:0/1912637421 wait complete. 2024-09-10T08:13:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.798+0000 7f8f92800640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8f780a2b80 0x7f8f780a5040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c005fd0 tx=0x7f8f8c0096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.799+0000 7f8f9ba71740 1 -- start start 2024-09-10T08:13:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.799+0000 7f8f9ba71740 1 Processor -- start 2024-09-10T08:13:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.799+0000 7f8f9ba71740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32ead50 0x5639c32f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.799+0000 7f8f9ba71740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32eb670 0x5639c32f3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.799+0000 7f8f9ba71740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32f5f30 0x5639c32f3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.799+0000 7f8f9ba71740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5639c32f8ba0 con 0x5639c32ead50 2024-09-10T08:13:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.799+0000 7f8f9ba71740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5639c320c0f0 con 0x5639c32f5f30 2024-09-10T08:13:27.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.799+0000 7f8f9ba71740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5639c32f8d20 con 0x5639c32eb670 2024-09-10T08:13:27.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.799+0000 7f8f9ba71740 10 client.0 _ll_register_callbacks cb 0x5639c31fd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:27.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.800+0000 7f8f91e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32ead50 0x5639c32f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.800+0000 7f8f91e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32ead50 0x5639c32f2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52652/0 (socket says 172.21.15.2:52652) 2024-09-10T08:13:27.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.800+0000 7f8f91e00640 1 -- 172.21.15.2:0/208019099 learned_addr learned my addr 172.21.15.2:0/208019099 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:27.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.800+0000 7f8f8be00640 1 --2- 172.21.15.2:0/208019099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32eb670 0x5639c32f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.800+0000 7f8f92800640 1 --2- 172.21.15.2:0/208019099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32f5f30 0x5639c32f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.800+0000 7f8f91e00640 1 -- 172.21.15.2:0/208019099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32eb670 msgr2=0x5639c32f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:27.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.800+0000 7f8f91e00640 1 --2- 172.21.15.2:0/208019099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32eb670 0x5639c32f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.800+0000 7f8f91e00640 1 -- 172.21.15.2:0/208019099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32f5f30 msgr2=0x5639c32f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:27.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.800+0000 7f8f91e00640 1 --2- 172.21.15.2:0/208019099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32f5f30 0x5639c32f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.800+0000 7f8f91e00640 1 -- 172.21.15.2:0/208019099 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639c2fc23c0 con 0x5639c32ead50 2024-09-10T08:13:27.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.800+0000 7f8f91e00640 1 --2- 172.21.15.2:0/208019099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32ead50 0x5639c32f2e00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400d740 tx=0x7f8f84005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7aee6d038c078e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:27.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.800+0000 7f8f77e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:13:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.801+0000 7f8f77e00640 1 -- 172.21.15.2:0/208019099 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84020070 con 0x5639c32ead50 2024-09-10T08:13:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.801+0000 7f8f77e00640 1 -- 172.21.15.2:0/208019099 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8f84009cc0 con 0x5639c32ead50 2024-09-10T08:13:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.801+0000 7f8f9ba71740 1 -- 172.21.15.2:0/208019099 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f7c005520 con 0x5639c32ead50 2024-09-10T08:13:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.801+0000 7f8f77e00640 1 -- 172.21.15.2:0/208019099 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f8409ac20 con 0x5639c32ead50 2024-09-10T08:13:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.801+0000 7f8f9ba71740 10 client.16827 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.801+0000 7f8f9ba71740 1 -- 172.21.15.2:0/208019099 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5639c3304830 con 0x5639c32ead50 2024-09-10T08:13:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.801+0000 7f8f77e00640 1 -- 172.21.15.2:0/208019099 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f7c005520 con 0x5639c32ead50 2024-09-10T08:13:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.801+0000 7f8f77e00640 1 -- 172.21.15.2:0/208019099 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8f84097930 con 0x5639c32ead50 2024-09-10T08:13:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.802+0000 7f8f9ba71740 10 client.16827 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.802+0000 7f8f9ba71740 10 client.16827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.802+0000 7f8f9ba71740 20 client.16827 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.802+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/208019099 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5639c3305c40 0x5639c3308050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.802+0000 7f8f9ba71740 4 client.16827 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:13:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.802+0000 7f8f9ba71740 1 -- 172.21.15.2:0/208019099 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639c325e950 con 0x5639c3305c40 2024-09-10T08:13:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.802+0000 7f8f8be00640 1 --2- 172.21.15.2:0/208019099 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5639c3305c40 0x5639c3308050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.802+0000 7f8f8be00640 1 --2- 172.21.15.2:0/208019099 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5639c3305c40 0x5639c3308050 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e502c1b41beed713 server_cookie=ecba44e4bf1c9c82 in_seq=0 out_seq=0 2024-09-10T08:13:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.802+0000 7f8f77e00640 10 client.16827 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.803+0000 7f8f77e00640 1 -- 172.21.15.2:0/208019099 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639c325e950 con 0x5639c3305c40 2024-09-10T08:13:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.803+0000 7f8f77e00640 10 client.16827 handle_command_reply: tid=0 2024-09-10T08:13:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.803+0000 7f8f9ba71740 1 client.16827 shutdown 2024-09-10T08:13:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.803+0000 7f8f9ba71740 1 -- 172.21.15.2:0/208019099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32ead50 msgr2=0x5639c32f2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.803+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/208019099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32ead50 0x5639c32f2e00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400d740 tx=0x7f8f84005d30 comp rx=0 tx=0).stop 2024-09-10T08:13:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.803+0000 7f8f9ba71740 1 -- 172.21.15.2:0/208019099 shutdown_connections 2024-09-10T08:13:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.803+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/208019099 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5639c3305c40 0x5639c3308050 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.804+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/208019099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32f5f30 0x5639c32f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.804+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/208019099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32eb670 0x5639c32f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.804+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/208019099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32ead50 0x5639c32f2e00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.804+0000 7f8f9ba71740 1 -- 172.21.15.2:0/208019099 >> 172.21.15.2:0/208019099 conn(0x5639c32e72e0 msgr2=0x5639c32ed010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.804+0000 7f8f77e00640 0 client.16827 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.804+0000 7f8f9ba71740 1 -- 172.21.15.2:0/208019099 shutdown_connections 2024-09-10T08:13:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.804+0000 7f8f9ba71740 1 -- 172.21.15.2:0/208019099 wait complete. 2024-09-10T08:13:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.804+0000 7f8f9ba71740 20 client.16827 trim_cache size 0 max 16384 2024-09-10T08:13:27.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.822+0000 7f8f9ba71740 1 Processor -- start 2024-09-10T08:13:27.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.822+0000 7f8f9ba71740 1 -- start start 2024-09-10T08:13:27.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.822+0000 7f8f9ba71740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32ead50 0x5639c32ed7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.822+0000 7f8f9ba71740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32eb670 0x5639c32e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.822+0000 7f8f9ba71740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32f5f30 0x5639c32f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.822+0000 7f8f9ba71740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8f941b37a0 con 0x5639c32f5f30 2024-09-10T08:13:27.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.822+0000 7f8f9ba71740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f941b1070 con 0x5639c32eb670 2024-09-10T08:13:27.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.822+0000 7f8f9ba71740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f941aeec0 con 0x5639c32ead50 2024-09-10T08:13:27.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.822+0000 7f8f92800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32f5f30 0x5639c32f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.822+0000 7f8f92800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32f5f30 0x5639c32f4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52660/0 (socket says 172.21.15.2:52660) 2024-09-10T08:13:27.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.822+0000 7f8f92800640 1 -- 172.21.15.2:0/3889962339 learned_addr learned my addr 172.21.15.2:0/3889962339 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:27.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.822+0000 7f8f91e00640 1 --2- 172.21.15.2:0/3889962339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32ead50 0x5639c32ed7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.822+0000 7f8f8be00640 1 --2- 172.21.15.2:0/3889962339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32eb670 0x5639c32e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.823+0000 7f8f91e00640 1 -- 172.21.15.2:0/3889962339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32eb670 msgr2=0x5639c32e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:27.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.823+0000 7f8f91e00640 1 --2- 172.21.15.2:0/3889962339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32eb670 0x5639c32e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.823+0000 7f8f91e00640 1 -- 172.21.15.2:0/3889962339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32f5f30 msgr2=0x5639c32f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:27.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.823+0000 7f8f91e00640 1 --2- 172.21.15.2:0/3889962339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32f5f30 0x5639c32f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.823+0000 7f8f91e00640 1 -- 172.21.15.2:0/3889962339 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639c32f1dd0 con 0x5639c32ead50 2024-09-10T08:13:27.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.823+0000 7f8f8be00640 1 --2- 172.21.15.2:0/3889962339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32eb670 0x5639c32e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:27.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.823+0000 7f8f92800640 1 --2- 172.21.15.2:0/3889962339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32f5f30 0x5639c32f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:27.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.823+0000 7f8f91e00640 1 --2- 172.21.15.2:0/3889962339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32ead50 0x5639c32ed7b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f84099570 tx=0x7f8f840a39f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc4bbd1b4fad1e7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:27.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.823+0000 7f8f82a00640 1 -- 172.21.15.2:0/3889962339 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84020070 con 0x5639c32ead50 2024-09-10T08:13:27.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.824+0000 7f8f82a00640 1 -- 172.21.15.2:0/3889962339 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f840b9e50 con 0x5639c32ead50 2024-09-10T08:13:27.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.824+0000 7f8f82a00640 1 -- 172.21.15.2:0/3889962339 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f840cc530 con 0x5639c32ead50 2024-09-10T08:13:27.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.824+0000 7f8f9ba71740 1 -- 172.21.15.2:0/3889962339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32ead50 msgr2=0x5639c32ed7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:27.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.824+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/3889962339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32ead50 0x5639c32ed7b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f84099570 tx=0x7f8f840a39f0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.824+0000 7f8f9ba71740 1 -- 172.21.15.2:0/3889962339 shutdown_connections 2024-09-10T08:13:27.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.824+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/3889962339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32f5f30 0x5639c32f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.824+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/3889962339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32eb670 0x5639c32e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.824+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/3889962339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32ead50 0x5639c32ed7b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.824+0000 7f8f9ba71740 1 -- 172.21.15.2:0/3889962339 >> 172.21.15.2:0/3889962339 conn(0x5639c32e72e0 msgr2=0x5639c32f4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:27.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.824+0000 7f8f9ba71740 1 -- 172.21.15.2:0/3889962339 shutdown_connections 2024-09-10T08:13:27.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.824+0000 7f8f9ba71740 1 -- 172.21.15.2:0/3889962339 wait complete. 2024-09-10T08:13:27.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.825+0000 7f8f9ba71740 1 -- start start 2024-09-10T08:13:27.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.825+0000 7f8f9ba71740 1 Processor -- start 2024-09-10T08:13:27.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.825+0000 7f8f9ba71740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32ead50 0x5639c32fe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.825+0000 7f8f9ba71740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32eb670 0x5639c32ef930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.825+0000 7f8f9ba71740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32f5f30 0x5639c32efe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.825+0000 7f8f9ba71740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8f941af530 con 0x5639c32eb670 2024-09-10T08:13:27.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.825+0000 7f8f9ba71740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f941b09a0 con 0x5639c32ead50 2024-09-10T08:13:27.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.825+0000 7f8f9ba71740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f941afc00 con 0x5639c32f5f30 2024-09-10T08:13:27.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.825+0000 7f8f9ba71740 10 client.0 _ll_register_callbacks cb 0x7f8f8400de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:27.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.826+0000 7f8f91e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32ead50 0x5639c32fe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.826+0000 7f8f91e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32ead50 0x5639c32fe4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54096/0 (socket says 172.21.15.2:54096) 2024-09-10T08:13:27.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.826+0000 7f8f91e00640 1 -- 172.21.15.2:0/472068673 learned_addr learned my addr 172.21.15.2:0/472068673 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:27.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.826+0000 7f8f8be00640 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32eb670 0x5639c32ef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.826+0000 7f8f92800640 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32f5f30 0x5639c32efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.826+0000 7f8f91e00640 1 -- 172.21.15.2:0/472068673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32f5f30 msgr2=0x5639c32efe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:27.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.826+0000 7f8f91e00640 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32f5f30 0x5639c32efe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.826+0000 7f8f91e00640 1 -- 172.21.15.2:0/472068673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32eb670 msgr2=0x5639c32ef930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:27.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.826+0000 7f8f91e00640 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32eb670 0x5639c32ef930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.826+0000 7f8f91e00640 1 -- 172.21.15.2:0/472068673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639c32f0460 con 0x5639c32ead50 2024-09-10T08:13:27.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.826+0000 7f8f8be00640 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32eb670 0x5639c32ef930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:27.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.826+0000 7f8f91e00640 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32ead50 0x5639c32fe4f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f840a3950 tx=0x7f8f8409aec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=808ebd40b41d058e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:27.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.826+0000 7f8f77e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:13:27.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f77e00640 1 -- 172.21.15.2:0/472068673 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84014070 con 0x5639c32ead50 2024-09-10T08:13:27.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f9ba71740 1 -- 172.21.15.2:0/472068673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f840b9e50 con 0x5639c32ead50 2024-09-10T08:13:27.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f77e00640 1 -- 172.21.15.2:0/472068673 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8f8409f180 con 0x5639c32ead50 2024-09-10T08:13:27.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f77e00640 1 -- 172.21.15.2:0/472068673 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f84020070 con 0x5639c32ead50 2024-09-10T08:13:27.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f9ba71740 10 client.26867 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:27.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f9ba71740 1 -- 172.21.15.2:0/472068673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5639c32f0460 con 0x5639c32ead50 2024-09-10T08:13:27.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f77e00640 1 -- 172.21.15.2:0/472068673 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f840b9e50 con 0x5639c32ead50 2024-09-10T08:13:27.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f77e00640 1 -- 172.21.15.2:0/472068673 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8f8409a710 con 0x5639c32ead50 2024-09-10T08:13:27.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f9ba71740 10 client.26867 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:27.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f9ba71740 10 client.26867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:27.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f9ba71740 20 client.26867 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:27.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5639c3370900 0x5639c3372cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:27.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f9ba71740 4 client.26867 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:13:27.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.827+0000 7f8f9ba71740 1 -- 172.21.15.2:0/472068673 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5639c3258e60 con 0x5639c3370900 2024-09-10T08:13:27.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.828+0000 7f8f8be00640 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5639c3370900 0x5639c3372cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:27.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.828+0000 7f8f8be00640 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5639c3370900 0x5639c3372cf0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef3400acc18c41 server_cookie=c65707fede42e700 in_seq=0 out_seq=0 2024-09-10T08:13:27.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.828+0000 7f8f77e00640 10 client.26867 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.829+0000 7f8f77e00640 1 -- 172.21.15.2:0/472068673 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5639c3258e60 con 0x5639c3370900 2024-09-10T08:13:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.829+0000 7f8f77e00640 10 client.26867 handle_command_reply: tid=0 2024-09-10T08:13:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.829+0000 7f8f9ba71740 1 client.26867 shutdown 2024-09-10T08:13:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.829+0000 7f8f9ba71740 1 -- 172.21.15.2:0/472068673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32ead50 msgr2=0x5639c32fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.829+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32ead50 0x5639c32fe4f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f840a3950 tx=0x7f8f8409aec0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.829+0000 7f8f9ba71740 1 -- 172.21.15.2:0/472068673 shutdown_connections 2024-09-10T08:13:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.829+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5639c3370900 0x5639c3372cf0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.829+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639c32f5f30 0x5639c32efe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.829+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5639c32eb670 0x5639c32ef930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.829+0000 7f8f9ba71740 1 --2- 172.21.15.2:0/472068673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639c32ead50 0x5639c32fe4f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.829+0000 7f8f77e00640 0 client.26867 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:27.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.829+0000 7f8f9ba71740 1 -- 172.21.15.2:0/472068673 >> 172.21.15.2:0/472068673 conn(0x5639c32e72e0 msgr2=0x5639c32e6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:27.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.830+0000 7f8f9ba71740 1 -- 172.21.15.2:0/472068673 shutdown_connections 2024-09-10T08:13:27.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.830+0000 7f8f9ba71740 1 -- 172.21.15.2:0/472068673 wait complete. 2024-09-10T08:13:27.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.830+0000 7f8f9ba71740 20 client.26867 trim_cache size 0 max 16384 2024-09-10T08:13:27.834 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:13:27.834 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:13:27.835 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:13:27.835 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:13:27.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.830+0000 7f8f93200640 1 -- 172.21.15.2:0/2997575012 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8f780a2b80 msgr2=0x7f8f780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:27.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.830+0000 7f8f93200640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8f780a2b80 0x7f8f780a5040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c005fd0 tx=0x7f8f8c0096a0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.830+0000 7f8f93200640 1 -- 172.21.15.2:0/2997575012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f941019c0 msgr2=0x7f8f9419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:27.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.830+0000 7f8f93200640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f941019c0 0x7f8f9419a4f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400d770 tx=0x7f8f8400dc30 comp rx=0 tx=0).stop 2024-09-10T08:13:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.830+0000 7f8f93200640 1 -- 172.21.15.2:0/2997575012 shutdown_connections 2024-09-10T08:13:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.830+0000 7f8f93200640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f941079c0 0x7f8f9419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.830+0000 7f8f93200640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8f780a2b80 0x7f8f780a5040 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.830+0000 7f8f93200640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f94102360 0x7f8f9419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.830+0000 7f8f93200640 1 --2- 172.21.15.2:0/2997575012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f941019c0 0x7f8f9419a4f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.830+0000 7f8f93200640 1 -- 172.21.15.2:0/2997575012 >> 172.21.15.2:0/2997575012 conn(0x7f8f940fd740 msgr2=0x7f8f9410d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.831+0000 7f8f93200640 1 -- 172.21.15.2:0/2997575012 shutdown_connections 2024-09-10T08:13:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:27.831+0000 7f8f93200640 1 -- 172.21.15.2:0/2997575012 wait complete. 2024-09-10T08:13:27.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:27 smithi155 ceph-mon[35405]: pgmap v546: 129 pgs: 129 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:13:27.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:27 smithi107 ceph-mon[35048]: pgmap v546: 129 pgs: 129 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:13:28.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:27 smithi002 ceph-mon[30563]: pgmap v546: 129 pgs: 129 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:13:28.706 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:13:28.707 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:13:29.090 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:13:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.728+0000 7f7eec600640 1 -- 172.21.15.2:0/391978414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ee4101de0 msgr2=0x7f7ee41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.728+0000 7f7eec600640 1 --2- 172.21.15.2:0/391978414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ee4101de0 0x7f7ee41021c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009900 tx=0x7f7ed802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:13:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.728+0000 7f7eec600640 1 -- 172.21.15.2:0/391978414 shutdown_connections 2024-09-10T08:13:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.728+0000 7f7eec600640 1 --2- 172.21.15.2:0/391978414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ee4106f10 0x7f7ee410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.728+0000 7f7eec600640 1 --2- 172.21.15.2:0/391978414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ee4102790 0x7f7ee41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.728+0000 7f7eec600640 1 --2- 172.21.15.2:0/391978414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ee4101de0 0x7f7ee41021c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.728+0000 7f7eec600640 1 -- 172.21.15.2:0/391978414 >> 172.21.15.2:0/391978414 conn(0x7f7ee40fd8c0 msgr2=0x7f7ee40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.729+0000 7f7eec600640 1 -- 172.21.15.2:0/391978414 shutdown_connections 2024-09-10T08:13:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.729+0000 7f7eec600640 1 -- 172.21.15.2:0/391978414 wait complete. 2024-09-10T08:13:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.729+0000 7f7eec600640 1 Processor -- start 2024-09-10T08:13:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.730+0000 7f7eec600640 1 -- start start 2024-09-10T08:13:29.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.730+0000 7f7eec600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ee4101de0 0x7f7ee4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:29.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.730+0000 7f7eec600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ee4102790 0x7f7ee4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:29.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.730+0000 7f7eec600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ee4106f10 0x7f7ee419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:29.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.730+0000 7f7eec600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee4111f70 con 0x7f7ee4106f10 2024-09-10T08:13:29.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.730+0000 7f7eec600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee4111df0 con 0x7f7ee4102790 2024-09-10T08:13:29.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.730+0000 7f7eec600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee41120f0 con 0x7f7ee4101de0 2024-09-10T08:13:29.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.730+0000 7f7eea600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ee4106f10 0x7f7ee419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:29.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.730+0000 7f7eea600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ee4106f10 0x7f7ee419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52704/0 (socket says 172.21.15.2:52704) 2024-09-10T08:13:29.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.730+0000 7f7eea600640 1 -- 172.21.15.2:0/2777216558 learned_addr learned my addr 172.21.15.2:0/2777216558 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:29.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.730+0000 7f7ee9200640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ee4102790 0x7f7ee4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:29.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.730+0000 7f7ee9c00640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ee4101de0 0x7f7ee4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:29.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.731+0000 7f7eea600640 1 -- 172.21.15.2:0/2777216558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ee4101de0 msgr2=0x7f7ee4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:29.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.731+0000 7f7eea600640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ee4101de0 0x7f7ee4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:29.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.731+0000 7f7eea600640 1 -- 172.21.15.2:0/2777216558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ee4102790 msgr2=0x7f7ee4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:29.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.731+0000 7f7eea600640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ee4102790 0x7f7ee4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:29.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.731+0000 7f7eea600640 1 -- 172.21.15.2:0/2777216558 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee419d3c0 con 0x7f7ee4106f10 2024-09-10T08:13:29.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.731+0000 7f7eea600640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ee4106f10 0x7f7ee419cce0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00eb10 tx=0x7f7edc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:29.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.731+0000 7f7ee2000640 1 -- 172.21.15.2:0/2777216558 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc004b20 con 0x7f7ee4106f10 2024-09-10T08:13:29.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.731+0000 7f7ee2000640 1 -- 172.21.15.2:0/2777216558 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7edc004cc0 con 0x7f7ee4106f10 2024-09-10T08:13:29.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.731+0000 7f7eec600640 1 -- 172.21.15.2:0/2777216558 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee419d6b0 con 0x7f7ee4106f10 2024-09-10T08:13:29.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.731+0000 7f7ee2000640 1 -- 172.21.15.2:0/2777216558 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc010760 con 0x7f7ee4106f10 2024-09-10T08:13:29.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.732+0000 7f7eec600640 1 -- 172.21.15.2:0/2777216558 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee41a4f90 con 0x7f7ee4106f10 2024-09-10T08:13:29.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.733+0000 7f7eec600640 1 -- 172.21.15.2:0/2777216558 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee41021c0 con 0x7f7ee4106f10 2024-09-10T08:13:29.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.734+0000 7f7ee2000640 1 -- 172.21.15.2:0/2777216558 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7edc005020 con 0x7f7ee4106f10 2024-09-10T08:13:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.737+0000 7f7ee2000640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7ecc0a2c00 0x7f7ecc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.737+0000 7f7ee2000640 1 -- 172.21.15.2:0/2777216558 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7edc0cfa60 con 0x7f7ee4106f10 2024-09-10T08:13:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.737+0000 7f7ee9c00640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7ecc0a2c00 0x7f7ecc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:29.737 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:29 smithi002 ceph-mon[30563]: pgmap v547: 129 pgs: 129 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:13:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.738+0000 7f7ee9c00640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7ecc0a2c00 0x7f7ecc0a50c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009230 tx=0x7f7ed80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.744+0000 7f7ee2000640 1 -- 172.21.15.2:0/2777216558 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7edc09c570 con 0x7f7ee4106f10 2024-09-10T08:13:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:29 smithi155 ceph-mon[35405]: pgmap v547: 129 pgs: 129 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:13:29.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:29 smithi107 ceph-mon[35048]: pgmap v547: 129 pgs: 129 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:13:29.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.970+0000 7f7eec600640 1 -- 172.21.15.2:0/2777216558 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ee402d7c0 con 0x7f7ee4106f10 2024-09-10T08:13:29.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.971+0000 7f7ee2000640 1 -- 172.21.15.2:0/2777216558 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f7edc09be70 con 0x7f7ee4106f10 2024-09-10T08:13:29.971 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:13:29.972 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:13:29.972 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:13:29.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.974+0000 7f7eec600640 1 -- 172.21.15.2:0/2777216558 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7ecc0a2c00 msgr2=0x7f7ecc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:29.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.974+0000 7f7eec600640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7ecc0a2c00 0x7f7ecc0a50c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009230 tx=0x7f7ed80058d0 comp rx=0 tx=0).stop 2024-09-10T08:13:29.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.974+0000 7f7eec600640 1 -- 172.21.15.2:0/2777216558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ee4106f10 msgr2=0x7f7ee419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:29.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.974+0000 7f7eec600640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ee4106f10 0x7f7ee419cce0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00eb10 tx=0x7f7edc00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:13:29.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.974+0000 7f7eec600640 1 -- 172.21.15.2:0/2777216558 shutdown_connections 2024-09-10T08:13:29.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.974+0000 7f7eec600640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ee4106f10 0x7f7ee419cce0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:29.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.974+0000 7f7eec600640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ee4102790 0x7f7ee4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:29.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.974+0000 7f7eec600640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7ecc0a2c00 0x7f7ecc0a50c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:29.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.974+0000 7f7eec600640 1 --2- 172.21.15.2:0/2777216558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ee4101de0 0x7f7ee4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:29.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.974+0000 7f7eec600640 1 -- 172.21.15.2:0/2777216558 >> 172.21.15.2:0/2777216558 conn(0x7f7ee40fd8c0 msgr2=0x7f7ee40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:29.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.974+0000 7f7eec600640 1 -- 172.21.15.2:0/2777216558 shutdown_connections 2024-09-10T08:13:29.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:29.974+0000 7f7eec600640 1 -- 172.21.15.2:0/2777216558 wait complete. 2024-09-10T08:13:30.849 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:13:30.900 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:30 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2777216558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:13:30.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:30 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2777216558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:13:30.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:30 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2777216558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:13:31.229 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:13:31.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.805+0000 7f4da2600640 1 -- 172.21.15.2:0/3360451393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d9c102790 msgr2=0x7f4d9c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:31.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.805+0000 7f4da2600640 1 --2- 172.21.15.2:0/3360451393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d9c102790 0x7f4d9c1067e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009900 tx=0x7f4d8802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.805+0000 7f4da2600640 1 -- 172.21.15.2:0/3360451393 shutdown_connections 2024-09-10T08:13:31.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.805+0000 7f4da2600640 1 --2- 172.21.15.2:0/3360451393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d9c106f10 0x7f4d9c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.805+0000 7f4da2600640 1 --2- 172.21.15.2:0/3360451393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d9c102790 0x7f4d9c1067e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.805+0000 7f4da2600640 1 --2- 172.21.15.2:0/3360451393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d9c101de0 0x7f4d9c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.805+0000 7f4da2600640 1 -- 172.21.15.2:0/3360451393 >> 172.21.15.2:0/3360451393 conn(0x7f4d9c0fd8c0 msgr2=0x7f4d9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:31.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.805+0000 7f4da2600640 1 -- 172.21.15.2:0/3360451393 shutdown_connections 2024-09-10T08:13:31.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.805+0000 7f4da2600640 1 -- 172.21.15.2:0/3360451393 wait complete. 2024-09-10T08:13:31.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.806+0000 7f4da2600640 1 Processor -- start 2024-09-10T08:13:31.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.806+0000 7f4da2600640 1 -- start start 2024-09-10T08:13:31.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.806+0000 7f4da2600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d9c101de0 0x7f4d9c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.806+0000 7f4da2600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d9c102790 0x7f4d9c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4da2600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d9c106f10 0x7f4d9c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4da2600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c111f70 con 0x7f4d9c102790 2024-09-10T08:13:31.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4da2600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c111df0 con 0x7f4d9c106f10 2024-09-10T08:13:31.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4da2600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c1120f0 con 0x7f4d9c101de0 2024-09-10T08:13:31.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4d9aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d9c102790 0x7f4d9c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4d9aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d9c102790 0x7f4d9c1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52718/0 (socket says 172.21.15.2:52718) 2024-09-10T08:13:31.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4d9aa00640 1 -- 172.21.15.2:0/513841243 learned_addr learned my addr 172.21.15.2:0/513841243 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:31.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4d9b400640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d9c101de0 0x7f4d9c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4d9be00640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d9c106f10 0x7f4d9c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4d9b400640 1 -- 172.21.15.2:0/513841243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d9c106f10 msgr2=0x7f4d9c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:31.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4d9b400640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d9c106f10 0x7f4d9c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4d9b400640 1 -- 172.21.15.2:0/513841243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d9c102790 msgr2=0x7f4d9c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:31.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4d9b400640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d9c102790 0x7f4d9c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.807+0000 7f4d9b400640 1 -- 172.21.15.2:0/513841243 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d9c19d420 con 0x7f4d9c101de0 2024-09-10T08:13:31.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.808+0000 7f4d9b400640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d9c101de0 0x7f4d9c198470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00e9f0 tx=0x7f4d8c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:31.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.808+0000 7f4d93e00640 1 -- 172.21.15.2:0/513841243 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00ccc0 con 0x7f4d9c101de0 2024-09-10T08:13:31.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.808+0000 7f4da2600640 1 -- 172.21.15.2:0/513841243 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d9c19d710 con 0x7f4d9c101de0 2024-09-10T08:13:31.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.808+0000 7f4d93e00640 1 -- 172.21.15.2:0/513841243 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d8c005020 con 0x7f4d9c101de0 2024-09-10T08:13:31.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.808+0000 7f4d93e00640 1 -- 172.21.15.2:0/513841243 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c004a00 con 0x7f4d9c101de0 2024-09-10T08:13:31.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.808+0000 7f4da2600640 1 -- 172.21.15.2:0/513841243 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d9c1a4ff0 con 0x7f4d9c101de0 2024-09-10T08:13:31.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.811+0000 7f4da51ba740 1 Processor -- start 2024-09-10T08:13:31.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.811+0000 7f4d93e00640 1 -- 172.21.15.2:0/513841243 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4d8c0051d0 con 0x7f4d9c101de0 2024-09-10T08:13:31.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.812+0000 7f4da51ba740 1 -- start start 2024-09-10T08:13:31.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.812+0000 7f4da51ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841db70 0x563d7841df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.812+0000 7f4da51ba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7841e490 0x563d7842cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.812+0000 7f4da51ba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7842d260 0x563d7842f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.812+0000 7f4da51ba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563d783843e0 con 0x563d7842d260 2024-09-10T08:13:31.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.812+0000 7f4da51ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563d7832bd80 con 0x563d7841db70 2024-09-10T08:13:31.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.812+0000 7f4da51ba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563d783803a0 con 0x563d7841e490 2024-09-10T08:13:31.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.812+0000 7f4d9aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7842d260 0x563d7842f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.812+0000 7f4d9be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841db70 0x563d7841df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.812+0000 7f4d9b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7841e490 0x563d7842cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.812+0000 7f4d9be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841db70 0x563d7841df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54132/0 (socket says 172.21.15.2:54132) 2024-09-10T08:13:31.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.812+0000 7f4d9be00640 1 -- 172.21.15.2:0/1153732559 learned_addr learned my addr 172.21.15.2:0/1153732559 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:31.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.813+0000 7f4d9be00640 1 -- 172.21.15.2:0/1153732559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7841e490 msgr2=0x563d7842cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:31.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.813+0000 7f4d9be00640 1 --2- 172.21.15.2:0/1153732559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7841e490 0x563d7842cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.813+0000 7f4d9be00640 1 -- 172.21.15.2:0/1153732559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7842d260 msgr2=0x563d7842f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:31.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.813+0000 7f4d9be00640 1 --2- 172.21.15.2:0/1153732559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7842d260 0x563d7842f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.813+0000 7f4d9be00640 1 -- 172.21.15.2:0/1153732559 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7834aae0 con 0x563d7841db70 2024-09-10T08:13:31.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.813+0000 7f4d9be00640 1 --2- 172.21.15.2:0/1153732559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841db70 0x563d7841df50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009f60 tx=0x7f4d94009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1be5f69abde4ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:31.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.813+0000 7f4d93e00640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d800a2b80 0x7f4d800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.813+0000 7f4d90c00640 1 -- 172.21.15.2:0/1153732559 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9401e070 con 0x563d7841db70 2024-09-10T08:13:31.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.813+0000 7f4d90c00640 1 -- 172.21.15.2:0/1153732559 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d94003c30 con 0x563d7841db70 2024-09-10T08:13:31.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.813+0000 7f4d93e00640 1 -- 172.21.15.2:0/513841243 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4d8c0cf320 con 0x7f4d9c101de0 2024-09-10T08:13:31.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.813+0000 7f4d90c00640 1 -- 172.21.15.2:0/1153732559 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9400fa20 con 0x563d7841db70 2024-09-10T08:13:31.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.814+0000 7f4d9b400640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d800a2b80 0x7f4d800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.814+0000 7f4da51ba740 1 -- 172.21.15.2:0/1153732559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841db70 msgr2=0x563d7841df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:31.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.814+0000 7f4da51ba740 1 --2- 172.21.15.2:0/1153732559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841db70 0x563d7841df50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009f60 tx=0x7f4d94009f90 comp rx=0 tx=0).stop 2024-09-10T08:13:31.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.814+0000 7f4d9b400640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d800a2b80 0x7f4d800a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0ccda0 tx=0x7f4d8c0d2e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:31.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.814+0000 7f4da51ba740 1 -- 172.21.15.2:0/1153732559 shutdown_connections 2024-09-10T08:13:31.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.814+0000 7f4da51ba740 1 --2- 172.21.15.2:0/1153732559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7842d260 0x563d7842f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.814+0000 7f4da51ba740 1 --2- 172.21.15.2:0/1153732559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7841e490 0x563d7842cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.814+0000 7f4da51ba740 1 --2- 172.21.15.2:0/1153732559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841db70 0x563d7841df50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.814+0000 7f4da51ba740 1 -- 172.21.15.2:0/1153732559 >> 172.21.15.2:0/1153732559 conn(0x563d7841a200 msgr2=0x563d7841a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:31.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.814+0000 7f4da51ba740 1 -- 172.21.15.2:0/1153732559 shutdown_connections 2024-09-10T08:13:31.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.815+0000 7f4da51ba740 1 -- 172.21.15.2:0/1153732559 wait complete. 2024-09-10T08:13:31.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.815+0000 7f4da51ba740 1 -- start start 2024-09-10T08:13:31.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.815+0000 7f4da51ba740 1 Processor -- start 2024-09-10T08:13:31.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.815+0000 7f4da51ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841db70 0x563d7842a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4da51ba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7841e490 0x563d7842a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4da51ba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7842d260 0x563d7842abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4da51ba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563d7842fe90 con 0x563d7842d260 2024-09-10T08:13:31.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4da51ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563d7833f0f0 con 0x563d7841db70 2024-09-10T08:13:31.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4da51ba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563d78430010 con 0x563d7841e490 2024-09-10T08:13:31.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4da51ba740 10 client.0 _ll_register_callbacks cb 0x563d78326bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:31.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4d9be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841db70 0x563d7842a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4d9be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841db70 0x563d7842a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54148/0 (socket says 172.21.15.2:54148) 2024-09-10T08:13:31.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4d9be00640 1 -- 172.21.15.2:0/1462131270 learned_addr learned my addr 172.21.15.2:0/1462131270 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:31.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4d9be00640 1 --2- 172.21.15.2:0/1462131270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7842d260 0x563d7842abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4d9be00640 1 -- 172.21.15.2:0/1462131270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7841e490 msgr2=0x563d7842a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:31.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4d9be00640 1 --2- 172.21.15.2:0/1462131270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7841e490 0x563d7842a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4d9be00640 1 -- 172.21.15.2:0/1462131270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841db70 msgr2=0x563d7842a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:31.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.816+0000 7f4d9be00640 1 --2- 172.21.15.2:0/1462131270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841db70 0x563d7842a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.817+0000 7f4d9be00640 1 -- 172.21.15.2:0/1462131270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d780f53c0 con 0x563d7842d260 2024-09-10T08:13:31.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.817+0000 7f4d9be00640 1 --2- 172.21.15.2:0/1462131270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7842d260 0x563d7842abf0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009f30 tx=0x7f4d94005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bacabf09c11dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:31.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.817+0000 7f4d85600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:13:31.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.817+0000 7f4d85600640 1 -- 172.21.15.2:0/1462131270 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9401e070 con 0x563d7842d260 2024-09-10T08:13:31.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.817+0000 7f4da51ba740 1 -- 172.21.15.2:0/1462131270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d94003c30 con 0x563d7842d260 2024-09-10T08:13:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.817+0000 7f4d85600640 1 -- 172.21.15.2:0/1462131270 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4d94004250 con 0x563d7842d260 2024-09-10T08:13:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.817+0000 7f4d85600640 1 -- 172.21.15.2:0/1462131270 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d94004660 con 0x563d7842d260 2024-09-10T08:13:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.818+0000 7f4da51ba740 10 client.16851 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.818+0000 7f4da51ba740 1 -- 172.21.15.2:0/1462131270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563d7843bf40 con 0x563d7842d260 2024-09-10T08:13:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.818+0000 7f4d85600640 1 -- 172.21.15.2:0/1462131270 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d94003c30 con 0x563d7842d260 2024-09-10T08:13:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.818+0000 7f4d85600640 1 -- 172.21.15.2:0/1462131270 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4d94005430 con 0x563d7842d260 2024-09-10T08:13:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.818+0000 7f4da51ba740 10 client.16851 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.818+0000 7f4da51ba740 10 client.16851 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:13:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.818+0000 7f4da51ba740 20 client.16851 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.818+0000 7f4da51ba740 1 --2- 172.21.15.2:0/1462131270 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x563d7843d290 0x563d7843f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.818+0000 7f4da51ba740 4 client.16851 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:13:31.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.818+0000 7f4da51ba740 1 -- 172.21.15.2:0/1462131270 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d78391950 con 0x563d7843d290 2024-09-10T08:13:31.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.819+0000 7f4d9aa00640 1 --2- 172.21.15.2:0/1462131270 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x563d7843d290 0x563d7843f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.819+0000 7f4d9aa00640 1 --2- 172.21.15.2:0/1462131270 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x563d7843d290 0x563d7843f680 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a865932527ab319e server_cookie=d15d790259f4a978 in_seq=0 out_seq=0 2024-09-10T08:13:31.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.819+0000 7f4d85600640 10 client.16851 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:13:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.820+0000 7f4d85600640 1 -- 172.21.15.2:0/1462131270 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d78391950 con 0x563d7843d290 2024-09-10T08:13:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.820+0000 7f4d85600640 10 client.16851 handle_command_reply: tid=0 2024-09-10T08:13:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.820+0000 7f4da51ba740 1 client.16851 shutdown 2024-09-10T08:13:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.820+0000 7f4da51ba740 1 -- 172.21.15.2:0/1462131270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7842d260 msgr2=0x563d7842abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.820+0000 7f4da51ba740 1 --2- 172.21.15.2:0/1462131270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7842d260 0x563d7842abf0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009f30 tx=0x7f4d94005fd0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.820+0000 7f4da51ba740 1 -- 172.21.15.2:0/1462131270 shutdown_connections 2024-09-10T08:13:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.820+0000 7f4da51ba740 1 --2- 172.21.15.2:0/1462131270 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x563d7843d290 0x563d7843f680 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.820+0000 7f4da51ba740 1 --2- 172.21.15.2:0/1462131270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7842d260 0x563d7842abf0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.820+0000 7f4da51ba740 1 --2- 172.21.15.2:0/1462131270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7841e490 0x563d7842a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.820+0000 7f4da51ba740 1 --2- 172.21.15.2:0/1462131270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841db70 0x563d7842a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.820+0000 7f4da51ba740 1 -- 172.21.15.2:0/1462131270 >> 172.21.15.2:0/1462131270 conn(0x563d7841a200 msgr2=0x563d7841fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:31.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.820+0000 7f4d85600640 0 client.16851 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:13:31.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.821+0000 7f4da51ba740 1 -- 172.21.15.2:0/1462131270 shutdown_connections 2024-09-10T08:13:31.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.821+0000 7f4da51ba740 1 -- 172.21.15.2:0/1462131270 wait complete. 2024-09-10T08:13:31.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.821+0000 7f4da51ba740 20 client.16851 trim_cache size 0 max 16384 2024-09-10T08:13:31.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.834+0000 7f4da51ba740 1 Processor -- start 2024-09-10T08:13:31.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.834+0000 7f4da51ba740 1 -- start start 2024-09-10T08:13:31.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.835+0000 7f4da51ba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7841db70 0x563d78444df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.835+0000 7f4da51ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841e490 0x563d78445330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.835+0000 7f4da51ba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7842d260 0x563d7842b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.835+0000 7f4da51ba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c1af570 con 0x563d7841db70 2024-09-10T08:13:31.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.835+0000 7f4da51ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c1ace40 con 0x563d7841e490 2024-09-10T08:13:31.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.835+0000 7f4da51ba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c1aac90 con 0x563d7842d260 2024-09-10T08:13:31.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.835+0000 7f4d9be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7841db70 0x563d78444df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.835+0000 7f4d9aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841e490 0x563d78445330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.835+0000 7f4d9be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7841db70 0x563d78444df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52756/0 (socket says 172.21.15.2:52756) 2024-09-10T08:13:31.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.835+0000 7f4d9be00640 1 -- 172.21.15.2:0/3921629815 learned_addr learned my addr 172.21.15.2:0/3921629815 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:31.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.835+0000 7f4d9be00640 1 --2- 172.21.15.2:0/3921629815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7842d260 0x563d7842b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.836+0000 7f4d9aa00640 1 -- 172.21.15.2:0/3921629815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7842d260 msgr2=0x563d7842b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:31.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.836+0000 7f4d9aa00640 1 --2- 172.21.15.2:0/3921629815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7842d260 0x563d7842b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.836+0000 7f4d9aa00640 1 -- 172.21.15.2:0/3921629815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7841db70 msgr2=0x563d78444df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:31.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.836+0000 7f4d9aa00640 1 --2- 172.21.15.2:0/3921629815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7841db70 0x563d78444df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.836+0000 7f4d9aa00640 1 -- 172.21.15.2:0/3921629815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d78415770 con 0x563d7841e490 2024-09-10T08:13:31.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.836+0000 7f4d9be00640 1 --2- 172.21.15.2:0/3921629815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7841db70 0x563d78444df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:31.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.836+0000 7f4d9aa00640 1 --2- 172.21.15.2:0/3921629815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841e490 0x563d78445330 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009230 tx=0x7f4d88009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f3056cb6065b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:31.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.836+0000 7f4d87e00640 1 -- 172.21.15.2:0/3921629815 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8804a070 con 0x563d7841e490 2024-09-10T08:13:31.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.836+0000 7f4d87e00640 1 -- 172.21.15.2:0/3921629815 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d88005030 con 0x563d7841e490 2024-09-10T08:13:31.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.836+0000 7f4d87e00640 1 -- 172.21.15.2:0/3921629815 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88035040 con 0x563d7841e490 2024-09-10T08:13:31.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.837+0000 7f4da51ba740 1 -- 172.21.15.2:0/3921629815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841e490 msgr2=0x563d78445330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:31.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.837+0000 7f4da51ba740 1 --2- 172.21.15.2:0/3921629815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841e490 0x563d78445330 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009230 tx=0x7f4d88009560 comp rx=0 tx=0).stop 2024-09-10T08:13:31.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.837+0000 7f4da51ba740 1 -- 172.21.15.2:0/3921629815 shutdown_connections 2024-09-10T08:13:31.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.837+0000 7f4da51ba740 1 --2- 172.21.15.2:0/3921629815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7842d260 0x563d7842b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.837+0000 7f4da51ba740 1 --2- 172.21.15.2:0/3921629815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841e490 0x563d78445330 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.837+0000 7f4da51ba740 1 --2- 172.21.15.2:0/3921629815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7841db70 0x563d78444df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.837+0000 7f4da51ba740 1 -- 172.21.15.2:0/3921629815 >> 172.21.15.2:0/3921629815 conn(0x563d7841a200 msgr2=0x563d78429260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:31.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.837+0000 7f4da51ba740 1 -- 172.21.15.2:0/3921629815 shutdown_connections 2024-09-10T08:13:31.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.837+0000 7f4da51ba740 1 -- 172.21.15.2:0/3921629815 wait complete. 2024-09-10T08:13:31.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.837+0000 7f4da51ba740 1 -- start start 2024-09-10T08:13:31.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.837+0000 7f4da51ba740 1 Processor -- start 2024-09-10T08:13:31.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.838+0000 7f4da51ba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7841db70 0x563d78421330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.838+0000 7f4da51ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841e490 0x563d78421870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.838+0000 7f4da51ba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7842d260 0x563d78434290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.838+0000 7f4da51ba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c1ab300 con 0x563d7841db70 2024-09-10T08:13:31.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.838+0000 7f4da51ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c1ac770 con 0x563d7841e490 2024-09-10T08:13:31.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.838+0000 7f4da51ba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c1ab9d0 con 0x563d7842d260 2024-09-10T08:13:31.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.838+0000 7f4da51ba740 10 client.0 _ll_register_callbacks cb 0x7f4d8c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.838+0000 7f4d9be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7841db70 0x563d78421330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.838+0000 7f4d9aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841e490 0x563d78421870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.838+0000 7f4d9aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841e490 0x563d78421870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54176/0 (socket says 172.21.15.2:54176) 2024-09-10T08:13:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.838+0000 7f4d9aa00640 1 -- 172.21.15.2:0/4016443526 learned_addr learned my addr 172.21.15.2:0/4016443526 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.838+0000 7f4d9be00640 1 --2- 172.21.15.2:0/4016443526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7842d260 0x563d78434290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.839+0000 7f4d9aa00640 1 -- 172.21.15.2:0/4016443526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7842d260 msgr2=0x563d78434290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.839+0000 7f4d9aa00640 1 --2- 172.21.15.2:0/4016443526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7842d260 0x563d78434290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.839+0000 7f4d9aa00640 1 -- 172.21.15.2:0/4016443526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7841db70 msgr2=0x563d78421330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.839+0000 7f4d9aa00640 1 --2- 172.21.15.2:0/4016443526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7841db70 0x563d78421330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.839+0000 7f4d9aa00640 1 -- 172.21.15.2:0/4016443526 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d78434850 con 0x563d7841e490 2024-09-10T08:13:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.839+0000 7f4d9be00640 1 --2- 172.21.15.2:0/4016443526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7841db70 0x563d78421330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:13:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.839+0000 7f4d9aa00640 1 --2- 172.21.15.2:0/4016443526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841e490 0x563d78421870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4d88006010 tx=0x7f4d88005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6c40c45e6f6cb05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.839+0000 7f4d85600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:13:31.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.839+0000 7f4d85600640 1 -- 172.21.15.2:0/4016443526 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88035050 con 0x563d7841e490 2024-09-10T08:13:31.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.839+0000 7f4da51ba740 1 -- 172.21.15.2:0/4016443526 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d88005030 con 0x563d7841e490 2024-09-10T08:13:31.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.839+0000 7f4d85600640 1 -- 172.21.15.2:0/4016443526 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4d88005570 con 0x563d7841e490 2024-09-10T08:13:31.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.840+0000 7f4d85600640 1 -- 172.21.15.2:0/4016443526 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d8805b050 con 0x563d7841e490 2024-09-10T08:13:31.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.840+0000 7f4da51ba740 10 client.26891 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:31.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.840+0000 7f4da51ba740 1 -- 172.21.15.2:0/4016443526 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563d78435b10 con 0x563d7841e490 2024-09-10T08:13:31.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.840+0000 7f4d85600640 1 -- 172.21.15.2:0/4016443526 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d88005030 con 0x563d7841e490 2024-09-10T08:13:31.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.840+0000 7f4d85600640 1 -- 172.21.15.2:0/4016443526 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4d88031be0 con 0x563d7841e490 2024-09-10T08:13:31.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.840+0000 7f4da51ba740 10 client.26891 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:31.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.840+0000 7f4da51ba740 10 client.26891 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:13:31.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.840+0000 7f4da51ba740 20 client.26891 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:31.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.840+0000 7f4da51ba740 1 --2- 172.21.15.2:0/4016443526 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x563d784ac670 0x563d784aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:31.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.840+0000 7f4da51ba740 4 client.26891 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:13:31.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.840+0000 7f4da51ba740 1 -- 172.21.15.2:0/4016443526 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d7838be60 con 0x563d784ac670 2024-09-10T08:13:31.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.840+0000 7f4d9be00640 1 --2- 172.21.15.2:0/4016443526 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x563d784ac670 0x563d784aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4d9be00640 1 --2- 172.21.15.2:0/4016443526 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x563d784ac670 0x563d784aea90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3684f2a5ca99d135 server_cookie=381baa7c603dba57 in_seq=0 out_seq=0 2024-09-10T08:13:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4d85600640 10 client.26891 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:13:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4d85600640 1 -- 172.21.15.2:0/4016443526 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d7838be60 con 0x563d784ac670 2024-09-10T08:13:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4d85600640 10 client.26891 handle_command_reply: tid=0 2024-09-10T08:13:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4da51ba740 1 client.26891 shutdown 2024-09-10T08:13:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4da51ba740 1 -- 172.21.15.2:0/4016443526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841e490 msgr2=0x563d78421870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4da51ba740 1 --2- 172.21.15.2:0/4016443526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841e490 0x563d78421870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4d88006010 tx=0x7f4d88005270 comp rx=0 tx=0).stop 2024-09-10T08:13:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4da51ba740 1 -- 172.21.15.2:0/4016443526 shutdown_connections 2024-09-10T08:13:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4da51ba740 1 --2- 172.21.15.2:0/4016443526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d7842d260 0x563d78434290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4da51ba740 1 --2- 172.21.15.2:0/4016443526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d7841e490 0x563d78421870 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4da51ba740 1 --2- 172.21.15.2:0/4016443526 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x563d784ac670 0x563d784aea90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4da51ba740 1 --2- 172.21.15.2:0/4016443526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d7841db70 0x563d78421330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4da51ba740 1 -- 172.21.15.2:0/4016443526 >> 172.21.15.2:0/4016443526 conn(0x563d7841a200 msgr2=0x563d7842a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:31.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.841+0000 7f4d85600640 0 client.26891 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:13:31.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.842+0000 7f4da51ba740 1 -- 172.21.15.2:0/4016443526 shutdown_connections 2024-09-10T08:13:31.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.842+0000 7f4da51ba740 1 -- 172.21.15.2:0/4016443526 wait complete. 2024-09-10T08:13:31.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.842+0000 7f4da51ba740 20 client.26891 trim_cache size 0 max 16384 2024-09-10T08:13:31.846 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:13:31.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.842+0000 7f4da2600640 1 -- 172.21.15.2:0/513841243 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d800a2b80 msgr2=0x7f4d800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:31.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.842+0000 7f4da2600640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d800a2b80 0x7f4d800a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0ccda0 tx=0x7f4d8c0d2e00 comp rx=0 tx=0).stop 2024-09-10T08:13:31.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.842+0000 7f4da2600640 1 -- 172.21.15.2:0/513841243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d9c101de0 msgr2=0x7f4d9c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:31.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.842+0000 7f4da2600640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d9c101de0 0x7f4d9c198470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00e9f0 tx=0x7f4d8c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.843+0000 7f4da2600640 1 -- 172.21.15.2:0/513841243 shutdown_connections 2024-09-10T08:13:31.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.843+0000 7f4da2600640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d9c106f10 0x7f4d9c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.843+0000 7f4da2600640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d9c102790 0x7f4d9c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.843+0000 7f4da2600640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d800a2b80 0x7f4d800a5040 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.843+0000 7f4da2600640 1 --2- 172.21.15.2:0/513841243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d9c101de0 0x7f4d9c198470 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:31.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.843+0000 7f4da2600640 1 -- 172.21.15.2:0/513841243 >> 172.21.15.2:0/513841243 conn(0x7f4d9c0fd8c0 msgr2=0x7f4d9c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:31.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.843+0000 7f4da2600640 1 -- 172.21.15.2:0/513841243 shutdown_connections 2024-09-10T08:13:31.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:31.843+0000 7f4da2600640 1 -- 172.21.15.2:0/513841243 wait complete. 2024-09-10T08:13:31.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:31 smithi155 ceph-mon[35405]: pgmap v548: 129 pgs: 129 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:13:31.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:31 smithi107 ceph-mon[35048]: pgmap v548: 129 pgs: 129 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:13:32.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:31 smithi002 ceph-mon[30563]: pgmap v548: 129 pgs: 129 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:13:32.777 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:13:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.057+0000 7f58bbe00640 1 -- 172.21.15.2:0/1473410850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58bc107620 msgr2=0x7f58bc109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.057+0000 7f58bbe00640 1 --2- 172.21.15.2:0/1473410850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58bc107620 0x7f58bc109a90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009ca0 tx=0x7f58b402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.057+0000 7f58bbe00640 1 -- 172.21.15.2:0/1473410850 shutdown_connections 2024-09-10T08:13:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.057+0000 7f58bbe00640 1 --2- 172.21.15.2:0/1473410850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58bc107620 0x7f58bc109a90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.057+0000 7f58bbe00640 1 --2- 172.21.15.2:0/1473410850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58bc104cf0 0x7f58bc1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.057+0000 7f58bbe00640 1 --2- 172.21.15.2:0/1473410850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58bc1023c0 0x7f58bc1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.057+0000 7f58bbe00640 1 -- 172.21.15.2:0/1473410850 >> 172.21.15.2:0/1473410850 conn(0x7f58bc0fbfa0 msgr2=0x7f58bc0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.057+0000 7f58bbe00640 1 -- 172.21.15.2:0/1473410850 shutdown_connections 2024-09-10T08:13:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.057+0000 7f58bbe00640 1 -- 172.21.15.2:0/1473410850 wait complete. 2024-09-10T08:13:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.058+0000 7f58bbe00640 1 Processor -- start 2024-09-10T08:13:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.058+0000 7f58bbe00640 1 -- start start 2024-09-10T08:13:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.058+0000 7f58bbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58bc1023c0 0x7f58bc199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.058+0000 7f58bbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58bc104cf0 0x7f58bc19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.058+0000 7f58bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58bc107620 0x7f58bc1a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.058+0000 7f58bbe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc10d630 con 0x7f58bc1023c0 2024-09-10T08:13:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.058+0000 7f58bbe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc10d4b0 con 0x7f58bc107620 2024-09-10T08:13:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.058+0000 7f58bbe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc10d7b0 con 0x7f58bc104cf0 2024-09-10T08:13:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.059+0000 7f58baa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58bc1023c0 0x7f58bc199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.059+0000 7f58bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58bc107620 0x7f58bc1a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.059+0000 7f58baa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58bc1023c0 0x7f58bc199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52768/0 (socket says 172.21.15.2:52768) 2024-09-10T08:13:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.059+0000 7f58baa00640 1 -- 172.21.15.2:0/3138404047 learned_addr learned my addr 172.21.15.2:0/3138404047 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.059+0000 7f58b3e00640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58bc104cf0 0x7f58bc19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.059+0000 7f58baa00640 1 -- 172.21.15.2:0/3138404047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58bc104cf0 msgr2=0x7f58bc19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.059+0000 7f58baa00640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58bc104cf0 0x7f58bc19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.059+0000 7f58baa00640 1 -- 172.21.15.2:0/3138404047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58bc107620 msgr2=0x7f58bc1a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.059+0000 7f58baa00640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58bc107620 0x7f58bc1a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.059+0000 7f58baa00640 1 -- 172.21.15.2:0/3138404047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc1a1ab0 con 0x7f58bc1023c0 2024-09-10T08:13:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.059+0000 7f58bb400640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58bc107620 0x7f58bc1a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.059+0000 7f58baa00640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58bc1023c0 0x7f58bc199e90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00e9f0 tx=0x7f58ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.060+0000 7f58b3400640 1 -- 172.21.15.2:0/3138404047 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00cd80 con 0x7f58bc1023c0 2024-09-10T08:13:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.060+0000 7f58b3400640 1 -- 172.21.15.2:0/3138404047 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ac010910 con 0x7f58bc1023c0 2024-09-10T08:13:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.060+0000 7f58bbe00640 1 -- 172.21.15.2:0/3138404047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58bc1a1da0 con 0x7f58bc1023c0 2024-09-10T08:13:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.060+0000 7f58b3400640 1 -- 172.21.15.2:0/3138404047 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac018990 con 0x7f58bc1023c0 2024-09-10T08:13:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.060+0000 7f58bbe00640 1 -- 172.21.15.2:0/3138404047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58bc1a2260 con 0x7f58bc1023c0 2024-09-10T08:13:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.063+0000 7f58c47e0740 1 Processor -- start 2024-09-10T08:13:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.063+0000 7f58b3400640 1 -- 172.21.15.2:0/3138404047 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f58ac010ac0 con 0x7f58bc1023c0 2024-09-10T08:13:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.063+0000 7f58c47e0740 1 -- start start 2024-09-10T08:13:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.064+0000 7f58c47e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511da70 0x556e3511de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.064+0000 7f58c47e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e35122e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.064+0000 7f58c47e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e35123440 0x556e35125900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.064+0000 7f58c47e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556e3507a100 con 0x556e3511da70 2024-09-10T08:13:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.064+0000 7f58c47e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556e350221b0 con 0x556e35123440 2024-09-10T08:13:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.064+0000 7f58c47e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556e350760c0 con 0x556e3511e3b0 2024-09-10T08:13:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.064+0000 7f58bb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511da70 0x556e3511de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.064+0000 7f58bb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511da70 0x556e3511de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52782/0 (socket says 172.21.15.2:52782) 2024-09-10T08:13:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.064+0000 7f58bb400640 1 -- 172.21.15.2:0/1851175395 learned_addr learned my addr 172.21.15.2:0/1851175395 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.064+0000 7f58baa00640 1 --2- 172.21.15.2:0/1851175395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e35122e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.064+0000 7f58b3e00640 1 --2- 172.21.15.2:0/1851175395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e35123440 0x556e35125900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.065+0000 7f58bb400640 1 -- 172.21.15.2:0/1851175395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 msgr2=0x556e35122e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.065+0000 7f58bb400640 1 --2- 172.21.15.2:0/1851175395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e35122e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.065+0000 7f58bb400640 1 -- 172.21.15.2:0/1851175395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e35123440 msgr2=0x556e35125900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.065+0000 7f58bb400640 1 --2- 172.21.15.2:0/1851175395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e35123440 0x556e35125900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.065+0000 7f58bb400640 1 -- 172.21.15.2:0/1851175395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e34d8d210 con 0x556e3511da70 2024-09-10T08:13:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.065+0000 7f58baa00640 1 --2- 172.21.15.2:0/1851175395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e35122e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.065+0000 7f58b3400640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f58a00a2c00 0x7f58a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.065+0000 7f58b3400640 1 -- 172.21.15.2:0/3138404047 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f58ac0cf700 con 0x7f58bc1023c0 2024-09-10T08:13:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.065+0000 7f58bb400640 1 --2- 172.21.15.2:0/1851175395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511da70 0x556e3511de70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f58bc06a980 tx=0x7f58b4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6d9562d72193a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.065+0000 7f58baa00640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f58a00a2c00 0x7f58a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.065+0000 7f58abe00640 1 -- 172.21.15.2:0/1851175395 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4031650 con 0x556e3511da70 2024-09-10T08:13:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.065+0000 7f58abe00640 1 -- 172.21.15.2:0/1851175395 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58b4036040 con 0x556e3511da70 2024-09-10T08:13:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.065+0000 7f58abe00640 1 -- 172.21.15.2:0/1851175395 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4004700 con 0x556e3511da70 2024-09-10T08:13:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.066+0000 7f58c47e0740 1 -- 172.21.15.2:0/1851175395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511da70 msgr2=0x556e3511de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.066+0000 7f58c47e0740 1 --2- 172.21.15.2:0/1851175395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511da70 0x556e3511de70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f58bc06a980 tx=0x7f58b4005ef0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.066+0000 7f58baa00640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f58a00a2c00 0x7f58a00a50c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00c6a0 tx=0x7f58ac0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.066+0000 7f58c47e0740 1 -- 172.21.15.2:0/1851175395 shutdown_connections 2024-09-10T08:13:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.066+0000 7f58c47e0740 1 --2- 172.21.15.2:0/1851175395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e35123440 0x556e35125900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.066+0000 7f58c47e0740 1 --2- 172.21.15.2:0/1851175395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e35122e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.066+0000 7f58c47e0740 1 --2- 172.21.15.2:0/1851175395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511da70 0x556e3511de70 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.066+0000 7f58c47e0740 1 -- 172.21.15.2:0/1851175395 >> 172.21.15.2:0/1851175395 conn(0x556e351108d0 msgr2=0x556e35110cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.067+0000 7f58c47e0740 1 -- 172.21.15.2:0/1851175395 shutdown_connections 2024-09-10T08:13:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.067+0000 7f58c47e0740 1 -- 172.21.15.2:0/1851175395 wait complete. 2024-09-10T08:13:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.067+0000 7f58c47e0740 1 -- start start 2024-09-10T08:13:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.067+0000 7f58c47e0740 1 Processor -- start 2024-09-10T08:13:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58c47e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e351279e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58c47e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511e3b0 0x556e35127f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58c47e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e35123440 0x556e3512ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58c47e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556e35126230 con 0x556e3511e3b0 2024-09-10T08:13:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58c47e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556e35003090 con 0x556e3511da70 2024-09-10T08:13:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58c47e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556e351263b0 con 0x556e35123440 2024-09-10T08:13:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58c47e0740 10 client.0 _ll_register_callbacks cb 0x556e35057f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511e3b0 0x556e35127f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511e3b0 0x556e35127f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52788/0 (socket says 172.21.15.2:52788) 2024-09-10T08:13:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58b3e00640 1 -- 172.21.15.2:0/3101853451 learned_addr learned my addr 172.21.15.2:0/3101853451 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58bb400640 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e351279e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58bb400640 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e35123440 0x556e3512ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58b3e00640 1 -- 172.21.15.2:0/3101853451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e35123440 msgr2=0x556e3512ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58b3e00640 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e35123440 0x556e3512ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58b3e00640 1 -- 172.21.15.2:0/3101853451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 msgr2=0x556e351279e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58b3e00640 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e351279e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58b3e00640 1 -- 172.21.15.2:0/3101853451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e34e59620 con 0x556e3511e3b0 2024-09-10T08:13:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.068+0000 7f58bb400640 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e351279e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:13:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.069+0000 7f58b3e00640 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511e3b0 0x556e35127f20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f58a40126e0 tx=0x7f58a4012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82de05a9714e9f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.069+0000 7f58a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:13:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.069+0000 7f58a8c00640 1 -- 172.21.15.2:0/3101853451 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a40120d0 con 0x556e3511e3b0 2024-09-10T08:13:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.069+0000 7f58a8c00640 1 -- 172.21.15.2:0/3101853451 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f58a400f920 con 0x556e3511e3b0 2024-09-10T08:13:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.069+0000 7f58c47e0740 1 -- 172.21.15.2:0/3101853451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58b4036040 con 0x556e3511e3b0 2024-09-10T08:13:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.069+0000 7f58a8c00640 1 -- 172.21.15.2:0/3101853451 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f58a4012270 con 0x556e3511e3b0 2024-09-10T08:13:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.069+0000 7f58c47e0740 10 client.16869 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.069+0000 7f58c47e0740 1 -- 172.21.15.2:0/3101853451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e35131e70 con 0x556e3511e3b0 2024-09-10T08:13:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.069+0000 7f58a8c00640 1 -- 172.21.15.2:0/3101853451 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58b4036040 con 0x556e3511e3b0 2024-09-10T08:13:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.070+0000 7f58a8c00640 1 -- 172.21.15.2:0/3101853451 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f58a401c420 con 0x556e3511e3b0 2024-09-10T08:13:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.070+0000 7f58c47e0740 10 client.16869 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.070+0000 7f58c47e0740 10 client.16869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.070+0000 7f58c47e0740 20 client.16869 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.070+0000 7f58c47e0740 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556e35133560 0x556e351359b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.070+0000 7f58c47e0740 4 client.16869 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:13:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.070+0000 7f58c47e0740 1 -- 172.21.15.2:0/3101853451 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e35087830 con 0x556e35133560 2024-09-10T08:13:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.070+0000 7f58bb400640 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556e35133560 0x556e351359b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.071+0000 7f58bb400640 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556e35133560 0x556e351359b0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4ac9726ab3fbedb server_cookie=cf43582ddf9a6c63 in_seq=0 out_seq=0 2024-09-10T08:13:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.071+0000 7f58a8c00640 10 client.16869 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.071+0000 7f58a8c00640 1 -- 172.21.15.2:0/3101853451 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e35087830 con 0x556e35133560 2024-09-10T08:13:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.071+0000 7f58a8c00640 10 client.16869 handle_command_reply: tid=0 2024-09-10T08:13:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.072+0000 7f58c47e0740 1 client.16869 shutdown 2024-09-10T08:13:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.072+0000 7f58c47e0740 1 -- 172.21.15.2:0/3101853451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511e3b0 msgr2=0x556e35127f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.072+0000 7f58c47e0740 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511e3b0 0x556e35127f20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f58a40126e0 tx=0x7f58a4012ba0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.072+0000 7f58c47e0740 1 -- 172.21.15.2:0/3101853451 shutdown_connections 2024-09-10T08:13:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.072+0000 7f58c47e0740 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556e35133560 0x556e351359b0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.072+0000 7f58c47e0740 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e35123440 0x556e3512ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.072+0000 7f58c47e0740 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3511e3b0 0x556e35127f20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.072+0000 7f58c47e0740 1 --2- 172.21.15.2:0/3101853451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e351279e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.072+0000 7f58a8c00640 0 client.16869 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.072+0000 7f58c47e0740 1 -- 172.21.15.2:0/3101853451 >> 172.21.15.2:0/3101853451 conn(0x556e351108d0 msgr2=0x556e351120e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.072+0000 7f58c47e0740 1 -- 172.21.15.2:0/3101853451 shutdown_connections 2024-09-10T08:13:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.072+0000 7f58c47e0740 1 -- 172.21.15.2:0/3101853451 wait complete. 2024-09-10T08:13:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.072+0000 7f58c47e0740 20 client.16869 trim_cache size 0 max 16384 2024-09-10T08:13:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.089+0000 7f58c47e0740 1 Processor -- start 2024-09-10T08:13:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.089+0000 7f58c47e0740 1 -- start start 2024-09-10T08:13:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.089+0000 7f58c47e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e3510f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.089+0000 7f58c47e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e3512ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.089+0000 7f58c47e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e35123440 0x556e3510f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.089+0000 7f58c47e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc1aad30 con 0x556e35123440 2024-09-10T08:13:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.089+0000 7f58c47e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc1a8600 con 0x556e3511da70 2024-09-10T08:13:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.089+0000 7f58c47e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc1a6450 con 0x556e3511e3b0 2024-09-10T08:13:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.090+0000 7f58bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e3510f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.090+0000 7f58b3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e3512ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.090+0000 7f58bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e3510f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54212/0 (socket says 172.21.15.2:54212) 2024-09-10T08:13:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.090+0000 7f58bb400640 1 -- 172.21.15.2:0/2614137747 learned_addr learned my addr 172.21.15.2:0/2614137747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.090+0000 7f58bb400640 1 --2- 172.21.15.2:0/2614137747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e35123440 0x556e3510f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.090+0000 7f58b3e00640 1 -- 172.21.15.2:0/2614137747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 msgr2=0x556e3510f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.090+0000 7f58b3e00640 1 --2- 172.21.15.2:0/2614137747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e3510f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.090+0000 7f58b3e00640 1 -- 172.21.15.2:0/2614137747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e35123440 msgr2=0x556e3510f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.090+0000 7f58b3e00640 1 --2- 172.21.15.2:0/2614137747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e35123440 0x556e3510f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.090+0000 7f58b3e00640 1 -- 172.21.15.2:0/2614137747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e351231e0 con 0x556e3511e3b0 2024-09-10T08:13:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.090+0000 7f58b3e00640 1 --2- 172.21.15.2:0/2614137747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e3512ef20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f58a401ca80 tx=0x7f58a400f810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9ff830f63ef8c8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.091+0000 7f58ab400640 1 -- 172.21.15.2:0/2614137747 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a401cb80 con 0x556e3511e3b0 2024-09-10T08:13:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.091+0000 7f58ab400640 1 -- 172.21.15.2:0/2614137747 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a400fe00 con 0x556e3511e3b0 2024-09-10T08:13:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.091+0000 7f58ab400640 1 -- 172.21.15.2:0/2614137747 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a40056d0 con 0x556e3511e3b0 2024-09-10T08:13:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.091+0000 7f58c47e0740 1 -- 172.21.15.2:0/2614137747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 msgr2=0x556e3512ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.091+0000 7f58c47e0740 1 --2- 172.21.15.2:0/2614137747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e3512ef20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f58a401ca80 tx=0x7f58a400f810 comp rx=0 tx=0).stop 2024-09-10T08:13:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.091+0000 7f58c47e0740 1 -- 172.21.15.2:0/2614137747 shutdown_connections 2024-09-10T08:13:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.091+0000 7f58c47e0740 1 --2- 172.21.15.2:0/2614137747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e35123440 0x556e3510f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.091+0000 7f58c47e0740 1 --2- 172.21.15.2:0/2614137747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e3512ef20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.091+0000 7f58c47e0740 1 --2- 172.21.15.2:0/2614137747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e3510f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.091+0000 7f58c47e0740 1 -- 172.21.15.2:0/2614137747 >> 172.21.15.2:0/2614137747 conn(0x556e351108d0 msgr2=0x556e35110cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.092+0000 7f58c47e0740 1 -- 172.21.15.2:0/2614137747 shutdown_connections 2024-09-10T08:13:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.092+0000 7f58c47e0740 1 -- 172.21.15.2:0/2614137747 wait complete. 2024-09-10T08:13:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.092+0000 7f58c47e0740 1 -- start start 2024-09-10T08:13:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.092+0000 7f58c47e0740 1 Processor -- start 2024-09-10T08:13:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.092+0000 7f58c47e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e3512c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.092+0000 7f58c47e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e3512c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.092+0000 7f58c47e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e35123440 0x556e3512d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.092+0000 7f58c47e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc1a6ac0 con 0x556e35123440 2024-09-10T08:13:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.092+0000 7f58c47e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc1a7f30 con 0x556e3511da70 2024-09-10T08:13:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.092+0000 7f58c47e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc1a7190 con 0x556e3511e3b0 2024-09-10T08:13:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.092+0000 7f58c47e0740 10 client.0 _ll_register_callbacks cb 0x7f58a4009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.093+0000 7f58bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e3512c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.093+0000 7f58bb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e35123440 0x556e3512d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.093+0000 7f58b3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e3512c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.093+0000 7f58b3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e3512c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:45200/0 (socket says 172.21.15.2:45200) 2024-09-10T08:13:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.093+0000 7f58bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e3512c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54228/0 (socket says 172.21.15.2:54228) 2024-09-10T08:13:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.093+0000 7f58b3e00640 1 -- 172.21.15.2:0/60301413 learned_addr learned my addr 172.21.15.2:0/60301413 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.093+0000 7f58bb400640 1 -- 172.21.15.2:0/60301413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 msgr2=0x556e3512c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.093+0000 7f58bb400640 1 --2- 172.21.15.2:0/60301413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e3512c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.093+0000 7f58bb400640 1 -- 172.21.15.2:0/60301413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e35123440 msgr2=0x556e3512d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.093+0000 7f58bb400640 1 --2- 172.21.15.2:0/60301413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e35123440 0x556e3512d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.093+0000 7f58bb400640 1 -- 172.21.15.2:0/60301413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e3512dbd0 con 0x556e3511da70 2024-09-10T08:13:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.094+0000 7f58bb400640 1 --2- 172.21.15.2:0/60301413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e3512c070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009a60 tx=0x7f58b403e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36e9f136160f173c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.094+0000 7f58a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:13:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.094+0000 7f58a8c00640 1 -- 172.21.15.2:0/60301413 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b40042c0 con 0x556e3511da70 2024-09-10T08:13:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.094+0000 7f58c47e0740 1 -- 172.21.15.2:0/60301413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58a400fe00 con 0x556e3511da70 2024-09-10T08:13:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.094+0000 7f58a8c00640 1 -- 172.21.15.2:0/60301413 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f58b4031450 con 0x556e3511da70 2024-09-10T08:13:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.094+0000 7f58a8c00640 1 -- 172.21.15.2:0/60301413 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f58b4031650 con 0x556e3511da70 2024-09-10T08:13:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.094+0000 7f58c47e0740 10 client.26909 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.094+0000 7f58c47e0740 1 -- 172.21.15.2:0/60301413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e3512ef90 con 0x556e3511da70 2024-09-10T08:13:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.094+0000 7f58a8c00640 1 -- 172.21.15.2:0/60301413 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a400fe00 con 0x556e3511da70 2024-09-10T08:13:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.095+0000 7f58a8c00640 1 -- 172.21.15.2:0/60301413 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f58b405b070 con 0x556e3511da70 2024-09-10T08:13:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.095+0000 7f58c47e0740 10 client.26909 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.095+0000 7f58c47e0740 10 client.26909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.095+0000 7f58c47e0740 20 client.26909 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.095+0000 7f58c47e0740 1 --2- 172.21.15.2:0/60301413 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556e3519e3c0 0x556e351a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.095+0000 7f58c47e0740 4 client.26909 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:13:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.095+0000 7f58c47e0740 1 -- 172.21.15.2:0/60301413 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e35081b80 con 0x556e3519e3c0 2024-09-10T08:13:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.095+0000 7f58b3e00640 1 --2- 172.21.15.2:0/60301413 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556e3519e3c0 0x556e351a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.096+0000 7f58b3e00640 1 --2- 172.21.15.2:0/60301413 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556e3519e3c0 0x556e351a07b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beeb016df1ac84d2 server_cookie=b868f38702f7445f in_seq=0 out_seq=0 2024-09-10T08:13:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.096+0000 7f58a8c00640 10 client.26909 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58a8c00640 1 -- 172.21.15.2:0/60301413 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e35081b80 con 0x556e3519e3c0 2024-09-10T08:13:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58a8c00640 10 client.26909 handle_command_reply: tid=0 2024-09-10T08:13:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58c47e0740 1 client.26909 shutdown 2024-09-10T08:13:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58c47e0740 1 -- 172.21.15.2:0/60301413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 msgr2=0x556e3512c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58c47e0740 1 --2- 172.21.15.2:0/60301413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e3512c070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009a60 tx=0x7f58b403e040 comp rx=0 tx=0).stop 2024-09-10T08:13:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58c47e0740 1 -- 172.21.15.2:0/60301413 shutdown_connections 2024-09-10T08:13:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58c47e0740 1 --2- 172.21.15.2:0/60301413 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556e3519e3c0 0x556e351a07b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58c47e0740 1 --2- 172.21.15.2:0/60301413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e35123440 0x556e3512d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58c47e0740 1 --2- 172.21.15.2:0/60301413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e3511e3b0 0x556e3512c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58c47e0740 1 --2- 172.21.15.2:0/60301413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e3511da70 0x556e3512c070 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58a8c00640 0 client.26909 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58c47e0740 1 -- 172.21.15.2:0/60301413 >> 172.21.15.2:0/60301413 conn(0x556e351108d0 msgr2=0x556e3510fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58c47e0740 1 -- 172.21.15.2:0/60301413 shutdown_connections 2024-09-10T08:13:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58c47e0740 1 -- 172.21.15.2:0/60301413 wait complete. 2024-09-10T08:13:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.110+0000 7f58c47e0740 20 client.26909 trim_cache size 0 max 16384 2024-09-10T08:13:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:13:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:13:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:13:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:13:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.111+0000 7f58bbe00640 1 -- 172.21.15.2:0/3138404047 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f58a00a2c00 msgr2=0x7f58a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.111+0000 7f58bbe00640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f58a00a2c00 0x7f58a00a50c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00c6a0 tx=0x7f58ac0d5530 comp rx=0 tx=0).stop 2024-09-10T08:13:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.111+0000 7f58bbe00640 1 -- 172.21.15.2:0/3138404047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58bc1023c0 msgr2=0x7f58bc199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.111+0000 7f58bbe00640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58bc1023c0 0x7f58bc199e90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00e9f0 tx=0x7f58ac00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.111+0000 7f58bbe00640 1 -- 172.21.15.2:0/3138404047 shutdown_connections 2024-09-10T08:13:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.111+0000 7f58bbe00640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58bc107620 0x7f58bc1a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.111+0000 7f58bbe00640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f58a00a2c00 0x7f58a00a50c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.111+0000 7f58bbe00640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58bc104cf0 0x7f58bc19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.111+0000 7f58bbe00640 1 --2- 172.21.15.2:0/3138404047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58bc1023c0 0x7f58bc199e90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.111+0000 7f58bbe00640 1 -- 172.21.15.2:0/3138404047 >> 172.21.15.2:0/3138404047 conn(0x7f58bc0fbfa0 msgr2=0x7f58bc101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.111+0000 7f58bbe00640 1 -- 172.21.15.2:0/3138404047 shutdown_connections 2024-09-10T08:13:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:33.111+0000 7f58bbe00640 1 -- 172.21.15.2:0/3138404047 wait complete. 2024-09-10T08:13:33.165 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:13:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.776+0000 7f2d4e400640 1 -- 172.21.15.2:0/1525158127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d481047e0 msgr2=0x7f2d48108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.776+0000 7f2d4e400640 1 --2- 172.21.15.2:0/1525158127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d481047e0 0x7f2d48108830 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009900 tx=0x7f2d3002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.776+0000 7f2d4e400640 1 -- 172.21.15.2:0/1525158127 shutdown_connections 2024-09-10T08:13:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.776+0000 7f2d4e400640 1 --2- 172.21.15.2:0/1525158127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d48108f60 0x7f2d48110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.776+0000 7f2d4e400640 1 --2- 172.21.15.2:0/1525158127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d481047e0 0x7f2d48108830 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.776+0000 7f2d4e400640 1 --2- 172.21.15.2:0/1525158127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d48103e30 0x7f2d48104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.776+0000 7f2d4e400640 1 -- 172.21.15.2:0/1525158127 >> 172.21.15.2:0/1525158127 conn(0x7f2d480ff8d0 msgr2=0x7f2d48101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.776+0000 7f2d4e400640 1 -- 172.21.15.2:0/1525158127 shutdown_connections 2024-09-10T08:13:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.776+0000 7f2d4e400640 1 -- 172.21.15.2:0/1525158127 wait complete. 2024-09-10T08:13:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.777+0000 7f2d4e400640 1 Processor -- start 2024-09-10T08:13:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.777+0000 7f2d4e400640 1 -- start start 2024-09-10T08:13:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.777+0000 7f2d4e400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d48103e30 0x7f2d4819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.777+0000 7f2d4e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d481047e0 0x7f2d4819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.777+0000 7f2d4e400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d48108f60 0x7f2d4819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.777+0000 7f2d4e400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2d48113fc0 con 0x7f2d48108f60 2024-09-10T08:13:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.777+0000 7f2d4e400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d48113e40 con 0x7f2d481047e0 2024-09-10T08:13:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.777+0000 7f2d4e400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d48114140 con 0x7f2d48103e30 2024-09-10T08:13:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.778+0000 7f2d4da00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d48108f60 0x7f2d4819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.778+0000 7f2d4d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d48103e30 0x7f2d4819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.778+0000 7f2d4da00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d48108f60 0x7f2d4819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52834/0 (socket says 172.21.15.2:52834) 2024-09-10T08:13:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.778+0000 7f2d4da00640 1 -- 172.21.15.2:0/1419424739 learned_addr learned my addr 172.21.15.2:0/1419424739 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.778+0000 7f2d3fe00640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d481047e0 0x7f2d4819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.778+0000 7f2d4d000640 1 -- 172.21.15.2:0/1419424739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d481047e0 msgr2=0x7f2d4819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.778+0000 7f2d4d000640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d481047e0 0x7f2d4819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.778+0000 7f2d4d000640 1 -- 172.21.15.2:0/1419424739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d48108f60 msgr2=0x7f2d4819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.778+0000 7f2d4d000640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d48108f60 0x7f2d4819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.778+0000 7f2d4d000640 1 -- 172.21.15.2:0/1419424739 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4819f430 con 0x7f2d48103e30 2024-09-10T08:13:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.778+0000 7f2d4da00640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d48108f60 0x7f2d4819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.778+0000 7f2d3fe00640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d481047e0 0x7f2d4819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.779+0000 7f2d4d000640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d48103e30 0x7f2d4819a480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800e9f0 tx=0x7f2d3800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.779+0000 7f2d46000640 1 -- 172.21.15.2:0/1419424739 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800ccc0 con 0x7f2d48103e30 2024-09-10T08:13:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.779+0000 7f2d46000640 1 -- 172.21.15.2:0/1419424739 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d38005020 con 0x7f2d48103e30 2024-09-10T08:13:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.779+0000 7f2d4e400640 1 -- 172.21.15.2:0/1419424739 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4819f720 con 0x7f2d48103e30 2024-09-10T08:13:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.779+0000 7f2d46000640 1 -- 172.21.15.2:0/1419424739 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d380049e0 con 0x7f2d48103e30 2024-09-10T08:13:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.779+0000 7f2d4e400640 1 -- 172.21.15.2:0/1419424739 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d481a7000 con 0x7f2d48103e30 2024-09-10T08:13:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.782+0000 7f2d524cd740 1 Processor -- start 2024-09-10T08:13:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.782+0000 7f2d524cd740 1 -- start start 2024-09-10T08:13:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.782+0000 7f2d46000640 1 -- 172.21.15.2:0/1419424739 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2d38005560 con 0x7f2d48103e30 2024-09-10T08:13:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.782+0000 7f2d524cd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d752220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.782+0000 7f2d524cd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d752760 0x55da1d75cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.782+0000 7f2d524cd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d75d020 0x55da1d75f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.782+0000 7f2d524cd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55da1d6b83e0 con 0x55da1d752760 2024-09-10T08:13:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.782+0000 7f2d524cd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55da1d65fd80 con 0x55da1d75d020 2024-09-10T08:13:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.782+0000 7f2d524cd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55da1d6b43a0 con 0x55da1d751e40 2024-09-10T08:13:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.783+0000 7f2d4d000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d752760 0x55da1d75cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.783+0000 7f2d4da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d752220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.783+0000 7f2d4da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d752220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:45232/0 (socket says 172.21.15.2:45232) 2024-09-10T08:13:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.783+0000 7f2d4da00640 1 -- 172.21.15.2:0/1977777739 learned_addr learned my addr 172.21.15.2:0/1977777739 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.783+0000 7f2d3fe00640 1 --2- 172.21.15.2:0/1977777739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d75d020 0x55da1d75f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.783+0000 7f2d4da00640 1 -- 172.21.15.2:0/1977777739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d75d020 msgr2=0x55da1d75f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.783+0000 7f2d4da00640 1 --2- 172.21.15.2:0/1977777739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d75d020 0x55da1d75f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.783+0000 7f2d4da00640 1 -- 172.21.15.2:0/1977777739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d752760 msgr2=0x55da1d75cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.783+0000 7f2d4da00640 1 --2- 172.21.15.2:0/1977777739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d752760 0x55da1d75cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.783+0000 7f2d4da00640 1 -- 172.21.15.2:0/1977777739 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da1d67eae0 con 0x55da1d751e40 2024-09-10T08:13:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.783+0000 7f2d4da00640 1 --2- 172.21.15.2:0/1977777739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d752220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2d40011850 tx=0x7f2d40011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6702223c5e244b3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.784+0000 7f2d3e000640 1 -- 172.21.15.2:0/1977777739 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40011330 con 0x55da1d751e40 2024-09-10T08:13:33.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.784+0000 7f2d3e000640 1 -- 172.21.15.2:0/1977777739 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d4000fe80 con 0x55da1d751e40 2024-09-10T08:13:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.784+0000 7f2d3e000640 1 -- 172.21.15.2:0/1977777739 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d400193e0 con 0x55da1d751e40 2024-09-10T08:13:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.784+0000 7f2d46000640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d2c0a2c00 0x7f2d2c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.784+0000 7f2d46000640 1 -- 172.21.15.2:0/1419424739 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2d380cf360 con 0x7f2d48103e30 2024-09-10T08:13:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.784+0000 7f2d4d000640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d2c0a2c00 0x7f2d2c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.784+0000 7f2d524cd740 1 -- 172.21.15.2:0/1977777739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 msgr2=0x55da1d752220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.784+0000 7f2d524cd740 1 --2- 172.21.15.2:0/1977777739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d752220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2d40011850 tx=0x7f2d40011d10 comp rx=0 tx=0).stop 2024-09-10T08:13:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.785+0000 7f2d524cd740 1 -- 172.21.15.2:0/1977777739 shutdown_connections 2024-09-10T08:13:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.785+0000 7f2d524cd740 1 --2- 172.21.15.2:0/1977777739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d75d020 0x55da1d75f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.785+0000 7f2d524cd740 1 --2- 172.21.15.2:0/1977777739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d752760 0x55da1d75cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.785+0000 7f2d524cd740 1 --2- 172.21.15.2:0/1977777739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d752220 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.785+0000 7f2d524cd740 1 -- 172.21.15.2:0/1977777739 >> 172.21.15.2:0/1977777739 conn(0x55da1d74e3d0 msgr2=0x55da1d74e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.785+0000 7f2d524cd740 1 -- 172.21.15.2:0/1977777739 shutdown_connections 2024-09-10T08:13:33.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.785+0000 7f2d524cd740 1 -- 172.21.15.2:0/1977777739 wait complete. 2024-09-10T08:13:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.785+0000 7f2d4d000640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d2c0a2c00 0x7f2d2c0a50c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2d380cca10 tx=0x7f2d380d2da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.786+0000 7f2d524cd740 1 -- start start 2024-09-10T08:13:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.786+0000 7f2d524cd740 1 Processor -- start 2024-09-10T08:13:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.786+0000 7f2d524cd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d759ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.786+0000 7f2d524cd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d752760 0x55da1d75a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.786+0000 7f2d524cd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d75d020 0x55da1d75a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.786+0000 7f2d524cd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55da1d75fbf0 con 0x55da1d75d020 2024-09-10T08:13:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.786+0000 7f2d524cd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55da1d6730f0 con 0x55da1d752760 2024-09-10T08:13:33.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.786+0000 7f2d524cd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55da1d75fd70 con 0x55da1d751e40 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.786+0000 7f2d524cd740 10 client.0 _ll_register_callbacks cb 0x55da1d65abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.787+0000 7f2d3fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d752760 0x55da1d75a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.787+0000 7f2d3fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d752760 0x55da1d75a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54266/0 (socket says 172.21.15.2:54266) 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.787+0000 7f2d3fe00640 1 -- 172.21.15.2:0/2568495002 learned_addr learned my addr 172.21.15.2:0/2568495002 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.787+0000 7f2d4da00640 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d759ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.787+0000 7f2d4da00640 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d75d020 0x55da1d75a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.787+0000 7f2d3fe00640 1 -- 172.21.15.2:0/2568495002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 msgr2=0x55da1d759ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.787+0000 7f2d3fe00640 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d759ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.787+0000 7f2d3fe00640 1 -- 172.21.15.2:0/2568495002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d75d020 msgr2=0x55da1d75a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.787+0000 7f2d3fe00640 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d75d020 0x55da1d75a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.787+0000 7f2d3fe00640 1 -- 172.21.15.2:0/2568495002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da1d4293c0 con 0x55da1d752760 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.787+0000 7f2d4da00640 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d75d020 0x55da1d75a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.787+0000 7f2d3fe00640 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d752760 0x55da1d75a420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2d30033000 tx=0x7f2d300317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94559df1b094e1a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.787+0000 7f2d36a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:13:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.788+0000 7f2d36a00640 1 -- 172.21.15.2:0/2568495002 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30035040 con 0x55da1d752760 2024-09-10T08:13:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.788+0000 7f2d524cd740 1 -- 172.21.15.2:0/2568495002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da1d6c5950 con 0x55da1d752760 2024-09-10T08:13:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.788+0000 7f2d36a00640 1 -- 172.21.15.2:0/2568495002 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2d30004250 con 0x55da1d752760 2024-09-10T08:13:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.788+0000 7f2d36a00640 1 -- 172.21.15.2:0/2568495002 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d3004f050 con 0x55da1d752760 2024-09-10T08:13:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.788+0000 7f2d524cd740 10 client.26921 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.788+0000 7f2d524cd740 1 -- 172.21.15.2:0/2568495002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da1d76bc50 con 0x55da1d752760 2024-09-10T08:13:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.789+0000 7f2d36a00640 1 -- 172.21.15.2:0/2568495002 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55da1d6c5950 con 0x55da1d752760 2024-09-10T08:13:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.789+0000 7f2d36a00640 1 -- 172.21.15.2:0/2568495002 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2d30046510 con 0x55da1d752760 2024-09-10T08:13:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.789+0000 7f2d524cd740 10 client.26921 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.789+0000 7f2d524cd740 10 client.26921 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:13:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.789+0000 7f2d524cd740 20 client.26921 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:33.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.789+0000 7f2d524cd740 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55da1d76d060 0x55da1d76f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.789+0000 7f2d524cd740 4 client.26921 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:13:33.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.789+0000 7f2d524cd740 1 -- 172.21.15.2:0/2568495002 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da1d6bfe60 con 0x55da1d76d060 2024-09-10T08:13:33.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.789+0000 7f2d4da00640 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55da1d76d060 0x55da1d76f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.790+0000 7f2d4da00640 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55da1d76d060 0x55da1d76f4a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3853eecebb77b1ea server_cookie=6f293b183b2a1734 in_seq=0 out_seq=0 2024-09-10T08:13:33.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.790+0000 7f2d36a00640 10 client.26921 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:13:33.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d36a00640 1 -- 172.21.15.2:0/2568495002 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da1d6bfe60 con 0x55da1d76d060 2024-09-10T08:13:33.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d36a00640 10 client.26921 handle_command_reply: tid=0 2024-09-10T08:13:33.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d524cd740 1 client.26921 shutdown 2024-09-10T08:13:33.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d524cd740 1 -- 172.21.15.2:0/2568495002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d752760 msgr2=0x55da1d75a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d524cd740 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d752760 0x55da1d75a420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2d30033000 tx=0x7f2d300317e0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d524cd740 1 -- 172.21.15.2:0/2568495002 shutdown_connections 2024-09-10T08:13:33.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d524cd740 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55da1d76d060 0x55da1d76f4a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d524cd740 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d75d020 0x55da1d75a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d524cd740 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d752760 0x55da1d75a420 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d524cd740 1 --2- 172.21.15.2:0/2568495002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d759ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d36a00640 0 client.26921 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:13:33.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d524cd740 1 -- 172.21.15.2:0/2568495002 >> 172.21.15.2:0/2568495002 conn(0x55da1d74e3d0 msgr2=0x55da1d754100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:33.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d524cd740 1 -- 172.21.15.2:0/2568495002 shutdown_connections 2024-09-10T08:13:33.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d524cd740 1 -- 172.21.15.2:0/2568495002 wait complete. 2024-09-10T08:13:33.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.791+0000 7f2d524cd740 20 client.26921 trim_cache size 0 max 16384 2024-09-10T08:13:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.805+0000 7f2d524cd740 1 Processor -- start 2024-09-10T08:13:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.805+0000 7f2d524cd740 1 -- start start 2024-09-10T08:13:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d524cd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d751e40 0x55da1d7587b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d524cd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d752760 0x55da1d75a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d524cd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d75d020 0x55da1d774d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d524cd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481b3760 con 0x55da1d751e40 2024-09-10T08:13:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d524cd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481b1030 con 0x55da1d752760 2024-09-10T08:13:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d524cd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481aee80 con 0x55da1d75d020 2024-09-10T08:13:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d4da00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d751e40 0x55da1d7587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d4da00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d751e40 0x55da1d7587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52868/0 (socket says 172.21.15.2:52868) 2024-09-10T08:13:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d4da00640 1 -- 172.21.15.2:0/3272392583 learned_addr learned my addr 172.21.15.2:0/3272392583 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d3fe00640 1 --2- 172.21.15.2:0/3272392583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d752760 0x55da1d75a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d4da00640 1 --2- 172.21.15.2:0/3272392583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d75d020 0x55da1d774d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d4da00640 1 -- 172.21.15.2:0/3272392583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d75d020 msgr2=0x55da1d774d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d4da00640 1 --2- 172.21.15.2:0/3272392583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d75d020 0x55da1d774d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d4da00640 1 -- 172.21.15.2:0/3272392583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d752760 msgr2=0x55da1d75a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d4da00640 1 --2- 172.21.15.2:0/3272392583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d752760 0x55da1d75a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.806+0000 7f2d4da00640 1 -- 172.21.15.2:0/3272392583 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da1d75c850 con 0x55da1d751e40 2024-09-10T08:13:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.807+0000 7f2d4da00640 1 --2- 172.21.15.2:0/3272392583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d751e40 0x55da1d7587b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2d40011850 tx=0x7f2d40009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e049ff5ab594f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.807+0000 7f2d3d600640 1 -- 172.21.15.2:0/3272392583 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40005bd0 con 0x55da1d751e40 2024-09-10T08:13:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.807+0000 7f2d3d600640 1 -- 172.21.15.2:0/3272392583 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d4000fc00 con 0x55da1d751e40 2024-09-10T08:13:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.807+0000 7f2d3d600640 1 -- 172.21.15.2:0/3272392583 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40010690 con 0x55da1d751e40 2024-09-10T08:13:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.808+0000 7f2d524cd740 1 -- 172.21.15.2:0/3272392583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d751e40 msgr2=0x55da1d7587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.808+0000 7f2d524cd740 1 --2- 172.21.15.2:0/3272392583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d751e40 0x55da1d7587b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2d40011850 tx=0x7f2d40009500 comp rx=0 tx=0).stop 2024-09-10T08:13:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.808+0000 7f2d524cd740 1 -- 172.21.15.2:0/3272392583 shutdown_connections 2024-09-10T08:13:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.808+0000 7f2d524cd740 1 --2- 172.21.15.2:0/3272392583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d75d020 0x55da1d774d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.808+0000 7f2d524cd740 1 --2- 172.21.15.2:0/3272392583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d752760 0x55da1d75a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.808+0000 7f2d524cd740 1 --2- 172.21.15.2:0/3272392583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d751e40 0x55da1d7587b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.808+0000 7f2d524cd740 1 -- 172.21.15.2:0/3272392583 >> 172.21.15.2:0/3272392583 conn(0x55da1d74e3d0 msgr2=0x55da1d75bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.808+0000 7f2d524cd740 1 -- 172.21.15.2:0/3272392583 shutdown_connections 2024-09-10T08:13:33.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.808+0000 7f2d524cd740 1 -- 172.21.15.2:0/3272392583 wait complete. 2024-09-10T08:13:33.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.808+0000 7f2d524cd740 1 -- start start 2024-09-10T08:13:33.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.808+0000 7f2d524cd740 1 Processor -- start 2024-09-10T08:13:33.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.809+0000 7f2d524cd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d756e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.809+0000 7f2d524cd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d752760 0x55da1d757380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.809+0000 7f2d524cd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d75d020 0x55da1d7578c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.809+0000 7f2d524cd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481af4f0 con 0x55da1d752760 2024-09-10T08:13:33.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.809+0000 7f2d524cd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481b0960 con 0x55da1d75d020 2024-09-10T08:13:33.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.809+0000 7f2d524cd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481afbc0 con 0x55da1d751e40 2024-09-10T08:13:33.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.809+0000 7f2d524cd740 10 client.0 _ll_register_callbacks cb 0x7f2d30005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:33.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.809+0000 7f2d3fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d752760 0x55da1d757380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.809+0000 7f2d4da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d756e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.809+0000 7f2d4da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d756e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:45256/0 (socket says 172.21.15.2:45256) 2024-09-10T08:13:33.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.809+0000 7f2d3fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d752760 0x55da1d757380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52876/0 (socket says 172.21.15.2:52876) 2024-09-10T08:13:33.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.809+0000 7f2d4da00640 1 -- 172.21.15.2:0/3778574896 learned_addr learned my addr 172.21.15.2:0/3778574896 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:33.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.809+0000 7f2d4da00640 1 --2- 172.21.15.2:0/3778574896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d75d020 0x55da1d7578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.810+0000 7f2d3fe00640 1 -- 172.21.15.2:0/3778574896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 msgr2=0x55da1d756e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.810+0000 7f2d3fe00640 1 --2- 172.21.15.2:0/3778574896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d756e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.810+0000 7f2d3fe00640 1 -- 172.21.15.2:0/3778574896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d75d020 msgr2=0x55da1d7578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.810+0000 7f2d3fe00640 1 --2- 172.21.15.2:0/3778574896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d75d020 0x55da1d7578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.810+0000 7f2d3fe00640 1 -- 172.21.15.2:0/3778574896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da1d7d6d40 con 0x55da1d752760 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.810+0000 7f2d3fe00640 1 --2- 172.21.15.2:0/3778574896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d752760 0x55da1d757380 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2d30046a50 tx=0x7f2d30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9834412c3c770408 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.810+0000 7f2d36a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.810+0000 7f2d36a00640 1 -- 172.21.15.2:0/3778574896 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30035050 con 0x55da1d752760 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.810+0000 7f2d36a00640 1 -- 172.21.15.2:0/3778574896 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2d30046690 con 0x55da1d752760 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.810+0000 7f2d524cd740 1 -- 172.21.15.2:0/3778574896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d4000fc00 con 0x55da1d752760 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.811+0000 7f2d36a00640 1 -- 172.21.15.2:0/3778574896 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d30004040 con 0x55da1d752760 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.811+0000 7f2d524cd740 10 client.16893 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.811+0000 7f2d524cd740 1 -- 172.21.15.2:0/3778574896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da1d7d7f80 con 0x55da1d752760 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.811+0000 7f2d36a00640 1 -- 172.21.15.2:0/3778574896 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d4000fc00 con 0x55da1d752760 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.811+0000 7f2d36a00640 1 -- 172.21.15.2:0/3778574896 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2d30004300 con 0x55da1d752760 2024-09-10T08:13:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.811+0000 7f2d524cd740 10 client.16893 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:33.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.811+0000 7f2d524cd740 10 client.16893 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:13:33.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.811+0000 7f2d524cd740 20 client.16893 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:33.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.811+0000 7f2d524cd740 1 --2- 172.21.15.2:0/3778574896 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55da1d7d8720 0x55da1d7dab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:33.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.811+0000 7f2d524cd740 4 client.16893 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:13:33.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.811+0000 7f2d524cd740 1 -- 172.21.15.2:0/3778574896 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da1d7499a0 con 0x55da1d7d8720 2024-09-10T08:13:33.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.812+0000 7f2d4da00640 1 --2- 172.21.15.2:0/3778574896 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55da1d7d8720 0x55da1d7dab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:33.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.812+0000 7f2d4da00640 1 --2- 172.21.15.2:0/3778574896 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55da1d7d8720 0x55da1d7dab40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfc70f6e4c72d359 server_cookie=b198d6d40c1f58a4 in_seq=0 out_seq=0 2024-09-10T08:13:33.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.812+0000 7f2d36a00640 10 client.16893 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:13:33.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.812+0000 7f2d36a00640 1 -- 172.21.15.2:0/3778574896 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da1d7499a0 con 0x55da1d7d8720 2024-09-10T08:13:33.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.812+0000 7f2d36a00640 10 client.16893 handle_command_reply: tid=0 2024-09-10T08:13:33.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.812+0000 7f2d524cd740 1 client.16893 shutdown 2024-09-10T08:13:33.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.813+0000 7f2d524cd740 1 -- 172.21.15.2:0/3778574896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d752760 msgr2=0x55da1d757380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.813+0000 7f2d524cd740 1 --2- 172.21.15.2:0/3778574896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d752760 0x55da1d757380 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2d30046a50 tx=0x7f2d30000bc0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.813+0000 7f2d524cd740 1 -- 172.21.15.2:0/3778574896 shutdown_connections 2024-09-10T08:13:33.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.813+0000 7f2d524cd740 1 --2- 172.21.15.2:0/3778574896 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55da1d7d8720 0x55da1d7dab40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.813+0000 7f2d524cd740 1 --2- 172.21.15.2:0/3778574896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1d75d020 0x55da1d7578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.813+0000 7f2d524cd740 1 --2- 172.21.15.2:0/3778574896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da1d752760 0x55da1d757380 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.813+0000 7f2d524cd740 1 --2- 172.21.15.2:0/3778574896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da1d751e40 0x55da1d756e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.813+0000 7f2d524cd740 1 -- 172.21.15.2:0/3778574896 >> 172.21.15.2:0/3778574896 conn(0x55da1d74e3d0 msgr2=0x55da1d74cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:33.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.813+0000 7f2d36a00640 0 client.16893 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:13:33.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.813+0000 7f2d524cd740 1 -- 172.21.15.2:0/3778574896 shutdown_connections 2024-09-10T08:13:33.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.813+0000 7f2d524cd740 1 -- 172.21.15.2:0/3778574896 wait complete. 2024-09-10T08:13:33.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.813+0000 7f2d524cd740 20 client.16893 trim_cache size 0 max 16384 2024-09-10T08:13:33.818 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:13:33.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.814+0000 7f2d4e400640 1 -- 172.21.15.2:0/1419424739 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d2c0a2c00 msgr2=0x7f2d2c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.814+0000 7f2d4e400640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d2c0a2c00 0x7f2d2c0a50c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2d380cca10 tx=0x7f2d380d2da0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.814+0000 7f2d4e400640 1 -- 172.21.15.2:0/1419424739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d48103e30 msgr2=0x7f2d4819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:33.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.814+0000 7f2d4e400640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d48103e30 0x7f2d4819a480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800e9f0 tx=0x7f2d3800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.814+0000 7f2d4e400640 1 -- 172.21.15.2:0/1419424739 shutdown_connections 2024-09-10T08:13:33.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.814+0000 7f2d4e400640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d48108f60 0x7f2d4819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.814+0000 7f2d4e400640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d481047e0 0x7f2d4819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.814+0000 7f2d4e400640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d2c0a2c00 0x7f2d2c0a50c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.814+0000 7f2d4e400640 1 --2- 172.21.15.2:0/1419424739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d48103e30 0x7f2d4819a480 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:33.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.814+0000 7f2d4e400640 1 -- 172.21.15.2:0/1419424739 >> 172.21.15.2:0/1419424739 conn(0x7f2d480ff8d0 msgr2=0x7f2d48100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:33.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.814+0000 7f2d4e400640 1 -- 172.21.15.2:0/1419424739 shutdown_connections 2024-09-10T08:13:33.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:33.814+0000 7f2d4e400640 1 -- 172.21.15.2:0/1419424739 wait complete. 2024-09-10T08:13:33.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:33 smithi155 ceph-mon[35405]: pgmap v549: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:13:33.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:33 smithi107 ceph-mon[35048]: pgmap v549: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:13:34.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:33 smithi002 ceph-mon[30563]: pgmap v549: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:13:34.720 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:13:35.099 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:13:35.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.693+0000 7fd5d7e00640 1 -- 172.21.15.2:0/705841967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5d8108f60 msgr2=0x7fd5d8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:35.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.693+0000 7fd5d7e00640 1 --2- 172.21.15.2:0/705841967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5d8108f60 0x7fd5d8110ae0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009ca0 tx=0x7fd5d002a8b0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.693+0000 7fd5d7e00640 1 -- 172.21.15.2:0/705841967 shutdown_connections 2024-09-10T08:13:35.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.693+0000 7fd5d7e00640 1 --2- 172.21.15.2:0/705841967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5d8108f60 0x7fd5d8110ae0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.693+0000 7fd5d7e00640 1 --2- 172.21.15.2:0/705841967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d81047e0 0x7fd5d8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.693+0000 7fd5d7e00640 1 --2- 172.21.15.2:0/705841967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5d8103e30 0x7fd5d8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.693+0000 7fd5d7e00640 1 -- 172.21.15.2:0/705841967 >> 172.21.15.2:0/705841967 conn(0x7fd5d80ff910 msgr2=0x7fd5d8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:35.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.693+0000 7fd5d7e00640 1 -- 172.21.15.2:0/705841967 shutdown_connections 2024-09-10T08:13:35.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.693+0000 7fd5d7e00640 1 -- 172.21.15.2:0/705841967 wait complete. 2024-09-10T08:13:35.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.694+0000 7fd5d7e00640 1 Processor -- start 2024-09-10T08:13:35.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.694+0000 7fd5d7e00640 1 -- start start 2024-09-10T08:13:35.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.695+0000 7fd5d7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d8103e30 0x7fd5d819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.695+0000 7fd5d7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5d81047e0 0x7fd5d819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.695+0000 7fd5d7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5d8108f60 0x7fd5d819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.695+0000 7fd5d7e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d8114010 con 0x7fd5d8108f60 2024-09-10T08:13:35.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.695+0000 7fd5d7e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d8113e90 con 0x7fd5d81047e0 2024-09-10T08:13:35.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.695+0000 7fd5d7e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d8114190 con 0x7fd5d8103e30 2024-09-10T08:13:35.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.695+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5d8108f60 0x7fd5d819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.695+0000 7fd5d6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5d81047e0 0x7fd5d819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.695+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5d8108f60 0x7fd5d819ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52894/0 (socket says 172.21.15.2:52894) 2024-09-10T08:13:35.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.695+0000 7fd5d7400640 1 -- 172.21.15.2:0/2486951357 learned_addr learned my addr 172.21.15.2:0/2486951357 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:35.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.695+0000 7fd5d6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5d81047e0 0x7fd5d819a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54306/0 (socket says 172.21.15.2:54306) 2024-09-10T08:13:35.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.695+0000 7fd5d6a00640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d8103e30 0x7fd5d819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.696+0000 7fd5d6a00640 1 -- 172.21.15.2:0/2486951357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5d81047e0 msgr2=0x7fd5d819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:35.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.696+0000 7fd5d6a00640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5d81047e0 0x7fd5d819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.696+0000 7fd5d6a00640 1 -- 172.21.15.2:0/2486951357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5d8108f60 msgr2=0x7fd5d819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:35.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.696+0000 7fd5d6a00640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5d8108f60 0x7fd5d819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.696+0000 7fd5d6a00640 1 -- 172.21.15.2:0/2486951357 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d819f3c0 con 0x7fd5d8103e30 2024-09-10T08:13:35.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.696+0000 7fd5d7400640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5d8108f60 0x7fd5d819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:35.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.696+0000 7fd5d6a00640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d8103e30 0x7fd5d819a410 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e9f0 tx=0x7fd5c800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:35.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.696+0000 7fd5cf400640 1 -- 172.21.15.2:0/2486951357 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800ccc0 con 0x7fd5d8103e30 2024-09-10T08:13:35.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.696+0000 7fd5d7e00640 1 -- 172.21.15.2:0/2486951357 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d819f6b0 con 0x7fd5d8103e30 2024-09-10T08:13:35.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.696+0000 7fd5cf400640 1 -- 172.21.15.2:0/2486951357 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5c8010910 con 0x7fd5d8103e30 2024-09-10T08:13:35.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.696+0000 7fd5cf400640 1 -- 172.21.15.2:0/2486951357 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8018960 con 0x7fd5d8103e30 2024-09-10T08:13:35.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.697+0000 7fd5d7e00640 1 -- 172.21.15.2:0/2486951357 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d81a6f90 con 0x7fd5d8103e30 2024-09-10T08:13:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.701+0000 7fd5e041e740 1 Processor -- start 2024-09-10T08:13:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.701+0000 7fd5e041e740 1 -- start start 2024-09-10T08:13:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.701+0000 7fd5cf400640 1 -- 172.21.15.2:0/2486951357 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd5c8010ac0 con 0x7fd5d8103e30 2024-09-10T08:13:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.701+0000 7fd5e041e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba53e40 0x557caba54220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.701+0000 7fd5e041e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba54760 0x557caba5eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.701+0000 7fd5e041e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 0x557caba61410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.701+0000 7fd5e041e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557cab9ba3e0 con 0x557caba5f020 2024-09-10T08:13:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.701+0000 7fd5e041e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557cab961d80 con 0x557caba54760 2024-09-10T08:13:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.701+0000 7fd5e041e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557cab9b63a0 con 0x557caba53e40 2024-09-10T08:13:35.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.702+0000 7fd5d6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba54760 0x557caba5eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.702+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba53e40 0x557caba54220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.702+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba53e40 0x557caba54220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:45276/0 (socket says 172.21.15.2:45276) 2024-09-10T08:13:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.702+0000 7fd5d6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba54760 0x557caba5eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54316/0 (socket says 172.21.15.2:54316) 2024-09-10T08:13:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.702+0000 7fd5d7400640 1 -- 172.21.15.2:0/1565014442 learned_addr learned my addr 172.21.15.2:0/1565014442 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.702+0000 7fd5d6000640 1 --2- 172.21.15.2:0/1565014442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 0x557caba61410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.702+0000 7fd5d7400640 1 -- 172.21.15.2:0/1565014442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba54760 msgr2=0x557caba5eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:35.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.702+0000 7fd5d7400640 1 --2- 172.21.15.2:0/1565014442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba54760 0x557caba5eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.702+0000 7fd5d7400640 1 -- 172.21.15.2:0/1565014442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 msgr2=0x557caba61410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:35.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.702+0000 7fd5d7400640 1 --2- 172.21.15.2:0/1565014442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 0x557caba61410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.702+0000 7fd5d7400640 1 -- 172.21.15.2:0/1565014442 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cab980ae0 con 0x557caba53e40 2024-09-10T08:13:35.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.702+0000 7fd5d7400640 1 --2- 172.21.15.2:0/1565014442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba53e40 0x557caba54220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0005fd0 tx=0x7fd5d0009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6057a9c6b57553b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.703+0000 7fd5c3e00640 1 -- 172.21.15.2:0/1565014442 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0003bf0 con 0x557caba53e40 2024-09-10T08:13:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.703+0000 7fd5c3e00640 1 -- 172.21.15.2:0/1565014442 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d0036030 con 0x557caba53e40 2024-09-10T08:13:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.703+0000 7fd5c3e00640 1 -- 172.21.15.2:0/1565014442 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00046e0 con 0x557caba53e40 2024-09-10T08:13:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.703+0000 7fd5cf400640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5bc0a2c00 0x7fd5bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.703+0000 7fd5cf400640 1 -- 172.21.15.2:0/2486951357 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd5c80cf740 con 0x7fd5d8103e30 2024-09-10T08:13:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.703+0000 7fd5d6a00640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5bc0a2c00 0x7fd5bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.704+0000 7fd5e041e740 1 -- 172.21.15.2:0/1565014442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba53e40 msgr2=0x557caba54220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.704+0000 7fd5e041e740 1 --2- 172.21.15.2:0/1565014442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba53e40 0x557caba54220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0005fd0 tx=0x7fd5d0009880 comp rx=0 tx=0).stop 2024-09-10T08:13:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.704+0000 7fd5e041e740 1 -- 172.21.15.2:0/1565014442 shutdown_connections 2024-09-10T08:13:35.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.704+0000 7fd5e041e740 1 --2- 172.21.15.2:0/1565014442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 0x557caba61410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.704+0000 7fd5e041e740 1 --2- 172.21.15.2:0/1565014442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba54760 0x557caba5eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.704+0000 7fd5e041e740 1 --2- 172.21.15.2:0/1565014442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba53e40 0x557caba54220 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.704+0000 7fd5e041e740 1 -- 172.21.15.2:0/1565014442 >> 172.21.15.2:0/1565014442 conn(0x557caba503d0 msgr2=0x557caba507b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.704+0000 7fd5e041e740 1 -- 172.21.15.2:0/1565014442 shutdown_connections 2024-09-10T08:13:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.704+0000 7fd5e041e740 1 -- 172.21.15.2:0/1565014442 wait complete. 2024-09-10T08:13:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.704+0000 7fd5d6a00640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5bc0a2c00 0x7fd5bc0a50c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80cd1c0 tx=0x7fd5c80d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.704+0000 7fd5e041e740 1 -- start start 2024-09-10T08:13:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.705+0000 7fd5e041e740 1 Processor -- start 2024-09-10T08:13:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.705+0000 7fd5e041e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba5bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.705+0000 7fd5e041e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba54760 0x557caba5c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.705+0000 7fd5e041e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 0x557caba5c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.705+0000 7fd5e041e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557caba61bf0 con 0x557caba5f020 2024-09-10T08:13:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.705+0000 7fd5e041e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557cab9750f0 con 0x557caba53e40 2024-09-10T08:13:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.705+0000 7fd5e041e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557caba61d70 con 0x557caba54760 2024-09-10T08:13:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.705+0000 7fd5e041e740 10 client.0 _ll_register_callbacks cb 0x557cab95cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.705+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba5bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.705+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba5bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54326/0 (socket says 172.21.15.2:54326) 2024-09-10T08:13:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.705+0000 7fd5d7400640 1 -- 172.21.15.2:0/2666574049 learned_addr learned my addr 172.21.15.2:0/2666574049 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.705+0000 7fd5d6000640 1 --2- 172.21.15.2:0/2666574049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba54760 0x557caba5c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.706+0000 7fd5d7400640 1 --2- 172.21.15.2:0/2666574049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 0x557caba5c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.706+0000 7fd5d6000640 1 -- 172.21.15.2:0/2666574049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 msgr2=0x557caba5bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.706+0000 7fd5d6000640 1 --2- 172.21.15.2:0/2666574049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba5bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.706+0000 7fd5d6000640 1 -- 172.21.15.2:0/2666574049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 msgr2=0x557caba5c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.706+0000 7fd5d6000640 1 --2- 172.21.15.2:0/2666574049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 0x557caba5c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.706+0000 7fd5d6000640 1 -- 172.21.15.2:0/2666574049 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cab72b3c0 con 0x557caba54760 2024-09-10T08:13:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.706+0000 7fd5d6000640 1 --2- 172.21.15.2:0/2666574049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba54760 0x557caba5c470 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40126d0 tx=0x7fd5c4012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c65d49399372f315 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.706+0000 7fd5c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:13:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.706+0000 7fd5c0c00640 1 -- 172.21.15.2:0/2666574049 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4004030 con 0x557caba54760 2024-09-10T08:13:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.707+0000 7fd5e041e740 1 -- 172.21.15.2:0/2666574049 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5d0036030 con 0x557caba54760 2024-09-10T08:13:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.707+0000 7fd5c0c00640 1 -- 172.21.15.2:0/2666574049 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd5c40041d0 con 0x557caba54760 2024-09-10T08:13:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.707+0000 7fd5c0c00640 1 -- 172.21.15.2:0/2666574049 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd5c40043d0 con 0x557caba54760 2024-09-10T08:13:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.707+0000 7fd5e041e740 10 client.26896 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.707+0000 7fd5e041e740 1 -- 172.21.15.2:0/2666574049 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557caba6dd10 con 0x557caba54760 2024-09-10T08:13:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.707+0000 7fd5c0c00640 1 -- 172.21.15.2:0/2666574049 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d0036030 con 0x557caba54760 2024-09-10T08:13:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.708+0000 7fd5c0c00640 1 -- 172.21.15.2:0/2666574049 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd5c4003ad0 con 0x557caba54760 2024-09-10T08:13:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.708+0000 7fd5e041e740 10 client.26896 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.708+0000 7fd5e041e740 10 client.26896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.708+0000 7fd5e041e740 20 client.26896 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.708+0000 7fd5e041e740 1 --2- 172.21.15.2:0/2666574049 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557caba6f220 0x557caba71640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.708+0000 7fd5e041e740 4 client.26896 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:13:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.708+0000 7fd5e041e740 1 -- 172.21.15.2:0/2666574049 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cab9c7950 con 0x557caba6f220 2024-09-10T08:13:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.708+0000 7fd5d7400640 1 --2- 172.21.15.2:0/2666574049 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557caba6f220 0x557caba71640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.709+0000 7fd5d7400640 1 --2- 172.21.15.2:0/2666574049 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557caba6f220 0x557caba71640 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fd7ee06b339cbef server_cookie=184c2a8d7fcb0ffa in_seq=0 out_seq=0 2024-09-10T08:13:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.709+0000 7fd5c0c00640 10 client.26896 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.709+0000 7fd5c0c00640 1 -- 172.21.15.2:0/2666574049 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cab9c7950 con 0x557caba6f220 2024-09-10T08:13:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.709+0000 7fd5c0c00640 10 client.26896 handle_command_reply: tid=0 2024-09-10T08:13:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.709+0000 7fd5e041e740 1 client.26896 shutdown 2024-09-10T08:13:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.710+0000 7fd5e041e740 1 -- 172.21.15.2:0/2666574049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba54760 msgr2=0x557caba5c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.710+0000 7fd5e041e740 1 --2- 172.21.15.2:0/2666574049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba54760 0x557caba5c470 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40126d0 tx=0x7fd5c4012b90 comp rx=0 tx=0).stop 2024-09-10T08:13:35.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.710+0000 7fd5e041e740 1 -- 172.21.15.2:0/2666574049 shutdown_connections 2024-09-10T08:13:35.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.710+0000 7fd5e041e740 1 --2- 172.21.15.2:0/2666574049 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557caba6f220 0x557caba71640 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.710+0000 7fd5e041e740 1 --2- 172.21.15.2:0/2666574049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 0x557caba5c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.710+0000 7fd5e041e740 1 --2- 172.21.15.2:0/2666574049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba54760 0x557caba5c470 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.710+0000 7fd5e041e740 1 --2- 172.21.15.2:0/2666574049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba5bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.710+0000 7fd5c0c00640 0 client.26896 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:35.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.710+0000 7fd5e041e740 1 -- 172.21.15.2:0/2666574049 >> 172.21.15.2:0/2666574049 conn(0x557caba503d0 msgr2=0x557caba560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:35.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.710+0000 7fd5e041e740 1 -- 172.21.15.2:0/2666574049 shutdown_connections 2024-09-10T08:13:35.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.710+0000 7fd5e041e740 1 -- 172.21.15.2:0/2666574049 wait complete. 2024-09-10T08:13:35.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.710+0000 7fd5e041e740 20 client.26896 trim_cache size 0 max 16384 2024-09-10T08:13:35.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.724+0000 7fd5e041e740 1 Processor -- start 2024-09-10T08:13:35.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.724+0000 7fd5e041e740 1 -- start start 2024-09-10T08:13:35.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.724+0000 7fd5e041e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba76ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.724+0000 7fd5e041e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba54760 0x557caba77430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.724+0000 7fd5e041e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba5f020 0x557caba59190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.724+0000 7fd5e041e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d81b36a0 con 0x557caba54760 2024-09-10T08:13:35.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.724+0000 7fd5e041e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d81b0f70 con 0x557caba53e40 2024-09-10T08:13:35.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.724+0000 7fd5e041e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d81aedc0 con 0x557caba5f020 2024-09-10T08:13:35.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.724+0000 7fd5d6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba54760 0x557caba77430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.724+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba76ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.724+0000 7fd5d6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba54760 0x557caba77430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52906/0 (socket says 172.21.15.2:52906) 2024-09-10T08:13:35.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.724+0000 7fd5d6000640 1 -- 172.21.15.2:0/825268216 learned_addr learned my addr 172.21.15.2:0/825268216 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:35.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.724+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba76ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54328/0 (socket says 172.21.15.2:54328) 2024-09-10T08:13:35.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.725+0000 7fd5d7400640 1 --2- 172.21.15.2:0/825268216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba5f020 0x557caba59190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.725+0000 7fd5d7400640 1 -- 172.21.15.2:0/825268216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba5f020 msgr2=0x557caba59190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:35.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.725+0000 7fd5d7400640 1 --2- 172.21.15.2:0/825268216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba5f020 0x557caba59190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.725+0000 7fd5d7400640 1 -- 172.21.15.2:0/825268216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba54760 msgr2=0x557caba77430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:35.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.725+0000 7fd5d7400640 1 --2- 172.21.15.2:0/825268216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba54760 0x557caba77430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.725+0000 7fd5d7400640 1 -- 172.21.15.2:0/825268216 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557caba5a480 con 0x557caba53e40 2024-09-10T08:13:35.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.725+0000 7fd5d7400640 1 --2- 172.21.15.2:0/825268216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba76ef0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00098c0 tx=0x7fd5d0004260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24ba11a6d4d1f699 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:35.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.725+0000 7fd5c3400640 1 -- 172.21.15.2:0/825268216 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d005f070 con 0x557caba53e40 2024-09-10T08:13:35.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.725+0000 7fd5c3400640 1 -- 172.21.15.2:0/825268216 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d0036040 con 0x557caba53e40 2024-09-10T08:13:35.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.725+0000 7fd5c3400640 1 -- 172.21.15.2:0/825268216 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0045440 con 0x557caba53e40 2024-09-10T08:13:35.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.726+0000 7fd5e041e740 1 -- 172.21.15.2:0/825268216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 msgr2=0x557caba76ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:35.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.726+0000 7fd5e041e740 1 --2- 172.21.15.2:0/825268216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba76ef0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00098c0 tx=0x7fd5d0004260 comp rx=0 tx=0).stop 2024-09-10T08:13:35.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.726+0000 7fd5e041e740 1 -- 172.21.15.2:0/825268216 shutdown_connections 2024-09-10T08:13:35.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.726+0000 7fd5e041e740 1 --2- 172.21.15.2:0/825268216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba5f020 0x557caba59190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.726+0000 7fd5e041e740 1 --2- 172.21.15.2:0/825268216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba54760 0x557caba77430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.726+0000 7fd5e041e740 1 --2- 172.21.15.2:0/825268216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba76ef0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.726+0000 7fd5e041e740 1 -- 172.21.15.2:0/825268216 >> 172.21.15.2:0/825268216 conn(0x557caba503d0 msgr2=0x557caba5dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:35.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.726+0000 7fd5e041e740 1 -- 172.21.15.2:0/825268216 shutdown_connections 2024-09-10T08:13:35.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.726+0000 7fd5e041e740 1 -- 172.21.15.2:0/825268216 wait complete. 2024-09-10T08:13:35.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.727+0000 7fd5e041e740 1 -- start start 2024-09-10T08:13:35.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.727+0000 7fd5e041e740 1 Processor -- start 2024-09-10T08:13:35.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.727+0000 7fd5e041e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba62e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.727+0000 7fd5e041e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba54760 0x557caba633d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.727+0000 7fd5e041e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 0x557caba63910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.727+0000 7fd5e041e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d81af430 con 0x557caba5f020 2024-09-10T08:13:35.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.727+0000 7fd5e041e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d81b08a0 con 0x557caba53e40 2024-09-10T08:13:35.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.727+0000 7fd5e041e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d81afb00 con 0x557caba54760 2024-09-10T08:13:35.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.727+0000 7fd5e041e740 10 client.0 _ll_register_callbacks cb 0x7fd5c4009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:35.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.728+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba62e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.728+0000 7fd5d6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba54760 0x557caba633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.728+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba62e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54330/0 (socket says 172.21.15.2:54330) 2024-09-10T08:13:35.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.728+0000 7fd5d6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba54760 0x557caba633d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:45298/0 (socket says 172.21.15.2:45298) 2024-09-10T08:13:35.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.728+0000 7fd5d7400640 1 -- 172.21.15.2:0/3060445718 learned_addr learned my addr 172.21.15.2:0/3060445718 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:35.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.728+0000 7fd5d7400640 1 --2- 172.21.15.2:0/3060445718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 0x557caba63910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.728+0000 7fd5d6000640 1 -- 172.21.15.2:0/3060445718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 msgr2=0x557caba62e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:35.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.728+0000 7fd5d6000640 1 --2- 172.21.15.2:0/3060445718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba62e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.728+0000 7fd5d6000640 1 -- 172.21.15.2:0/3060445718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 msgr2=0x557caba63910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:35.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.728+0000 7fd5d6000640 1 --2- 172.21.15.2:0/3060445718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 0x557caba63910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.728+0000 7fd5d6000640 1 -- 172.21.15.2:0/3060445718 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cabad9020 con 0x557caba54760 2024-09-10T08:13:35.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.728+0000 7fd5d6000640 1 --2- 172.21.15.2:0/3060445718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba54760 0x557caba633d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4001eb0 tx=0x7fd5c4004230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78d253a60d9be429 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:35.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.728+0000 7fd5c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:13:35.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.729+0000 7fd5c0c00640 1 -- 172.21.15.2:0/3060445718 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400fb30 con 0x557caba54760 2024-09-10T08:13:35.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.729+0000 7fd5e041e740 1 -- 172.21.15.2:0/3060445718 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5d0036040 con 0x557caba54760 2024-09-10T08:13:35.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.729+0000 7fd5c0c00640 1 -- 172.21.15.2:0/3060445718 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd5c400fcd0 con 0x557caba54760 2024-09-10T08:13:35.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.729+0000 7fd5c0c00640 1 -- 172.21.15.2:0/3060445718 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd5c40035f0 con 0x557caba54760 2024-09-10T08:13:35.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.729+0000 7fd5e041e740 10 client.26902 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:35.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.729+0000 7fd5e041e740 1 -- 172.21.15.2:0/3060445718 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557cabada2c0 con 0x557caba54760 2024-09-10T08:13:35.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.729+0000 7fd5c0c00640 1 -- 172.21.15.2:0/3060445718 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d0036040 con 0x557caba54760 2024-09-10T08:13:35.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.730+0000 7fd5c0c00640 1 -- 172.21.15.2:0/3060445718 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd5c401c990 con 0x557caba54760 2024-09-10T08:13:35.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.730+0000 7fd5e041e740 10 client.26902 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:35.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.730+0000 7fd5e041e740 10 client.26902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:35.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.730+0000 7fd5e041e740 20 client.26902 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:35.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.730+0000 7fd5e041e740 1 --2- 172.21.15.2:0/3060445718 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557cabadaac0 0x557cabadcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:35.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.730+0000 7fd5e041e740 4 client.26902 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:13:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.730+0000 7fd5e041e740 1 -- 172.21.15.2:0/3060445718 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557cab9c1e60 con 0x557cabadaac0 2024-09-10T08:13:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.730+0000 7fd5d7400640 1 --2- 172.21.15.2:0/3060445718 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557cabadaac0 0x557cabadcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5d7400640 1 --2- 172.21.15.2:0/3060445718 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557cabadaac0 0x557cabadcee0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e5db9b79d22f31e server_cookie=430577faabfb2941 in_seq=0 out_seq=0 2024-09-10T08:13:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5c0c00640 10 client.26902 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5c0c00640 1 -- 172.21.15.2:0/3060445718 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557cab9c1e60 con 0x557cabadaac0 2024-09-10T08:13:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5c0c00640 10 client.26902 handle_command_reply: tid=0 2024-09-10T08:13:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5e041e740 1 client.26902 shutdown 2024-09-10T08:13:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5e041e740 1 -- 172.21.15.2:0/3060445718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba54760 msgr2=0x557caba633d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5e041e740 1 --2- 172.21.15.2:0/3060445718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba54760 0x557caba633d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4001eb0 tx=0x7fd5c4004230 comp rx=0 tx=0).stop 2024-09-10T08:13:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5e041e740 1 -- 172.21.15.2:0/3060445718 shutdown_connections 2024-09-10T08:13:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5e041e740 1 --2- 172.21.15.2:0/3060445718 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557cabadaac0 0x557cabadcee0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5e041e740 1 --2- 172.21.15.2:0/3060445718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557caba5f020 0x557caba63910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5e041e740 1 --2- 172.21.15.2:0/3060445718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557caba54760 0x557caba633d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5e041e740 1 --2- 172.21.15.2:0/3060445718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557caba53e40 0x557caba62e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5e041e740 1 -- 172.21.15.2:0/3060445718 >> 172.21.15.2:0/3060445718 conn(0x557caba503d0 msgr2=0x557caba4ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:35.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5c0c00640 0 client.26902 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:35.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5e041e740 1 -- 172.21.15.2:0/3060445718 shutdown_connections 2024-09-10T08:13:35.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5e041e740 1 -- 172.21.15.2:0/3060445718 wait complete. 2024-09-10T08:13:35.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.731+0000 7fd5e041e740 20 client.26902 trim_cache size 0 max 16384 2024-09-10T08:13:35.735 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:13:35.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.732+0000 7fd5d7e00640 1 -- 172.21.15.2:0/2486951357 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5bc0a2c00 msgr2=0x7fd5bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:35.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.732+0000 7fd5d7e00640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5bc0a2c00 0x7fd5bc0a50c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80cd1c0 tx=0x7fd5c80d5550 comp rx=0 tx=0).stop 2024-09-10T08:13:35.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.732+0000 7fd5d7e00640 1 -- 172.21.15.2:0/2486951357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d8103e30 msgr2=0x7fd5d819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:35.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.732+0000 7fd5d7e00640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d8103e30 0x7fd5d819a410 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e9f0 tx=0x7fd5c800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.732+0000 7fd5d7e00640 1 -- 172.21.15.2:0/2486951357 shutdown_connections 2024-09-10T08:13:35.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.732+0000 7fd5d7e00640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5d8108f60 0x7fd5d819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.732+0000 7fd5d7e00640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5d81047e0 0x7fd5d819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.732+0000 7fd5d7e00640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd5bc0a2c00 0x7fd5bc0a50c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.732+0000 7fd5d7e00640 1 --2- 172.21.15.2:0/2486951357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d8103e30 0x7fd5d819a410 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:35.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.732+0000 7fd5d7e00640 1 -- 172.21.15.2:0/2486951357 >> 172.21.15.2:0/2486951357 conn(0x7fd5d80ff910 msgr2=0x7fd5d8100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:35.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.732+0000 7fd5d7e00640 1 -- 172.21.15.2:0/2486951357 shutdown_connections 2024-09-10T08:13:35.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:35.732+0000 7fd5d7e00640 1 -- 172.21.15.2:0/2486951357 wait complete. 2024-09-10T08:13:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:35 smithi155 ceph-mon[35405]: pgmap v550: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:13:35.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:35 smithi107 ceph-mon[35048]: pgmap v550: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:13:36.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:35 smithi002 ceph-mon[30563]: pgmap v550: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:13:37.576 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:13:37.576 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:13:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:37 smithi155 ceph-mon[35405]: pgmap v551: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:13:37.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:37 smithi107 ceph-mon[35048]: pgmap v551: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:13:37.983 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:13:38.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:37 smithi002 ceph-mon[30563]: pgmap v551: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:13:38.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.605+0000 7fc8c7600640 1 -- 172.21.15.2:0/2678206571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8c0102790 msgr2=0x7fc8c01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:38.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.605+0000 7fc8c7600640 1 --2- 172.21.15.2:0/2678206571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8c0102790 0x7fc8c01067e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009900 tx=0x7fc8a802a410 comp rx=0 tx=0).stop 2024-09-10T08:13:38.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.605+0000 7fc8c7600640 1 -- 172.21.15.2:0/2678206571 shutdown_connections 2024-09-10T08:13:38.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.605+0000 7fc8c7600640 1 --2- 172.21.15.2:0/2678206571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c0106f10 0x7fc8c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.605+0000 7fc8c7600640 1 --2- 172.21.15.2:0/2678206571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8c0102790 0x7fc8c01067e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.605+0000 7fc8c7600640 1 --2- 172.21.15.2:0/2678206571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c0101de0 0x7fc8c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.605+0000 7fc8c7600640 1 -- 172.21.15.2:0/2678206571 >> 172.21.15.2:0/2678206571 conn(0x7fc8c00fd8c0 msgr2=0x7fc8c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:38.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.605+0000 7fc8c7600640 1 -- 172.21.15.2:0/2678206571 shutdown_connections 2024-09-10T08:13:38.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.605+0000 7fc8c7600640 1 -- 172.21.15.2:0/2678206571 wait complete. 2024-09-10T08:13:38.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.606+0000 7fc8c7600640 1 Processor -- start 2024-09-10T08:13:38.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.606+0000 7fc8c7600640 1 -- start start 2024-09-10T08:13:38.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.606+0000 7fc8c7600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c0101de0 0x7fc8c00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.606+0000 7fc8c7600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8c0102790 0x7fc8c0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.606+0000 7fc8c7600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c0106f10 0x7fc8c0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.606+0000 7fc8c7600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c0111f30 con 0x7fc8c0102790 2024-09-10T08:13:38.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.606+0000 7fc8c7600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c0111db0 con 0x7fc8c0106f10 2024-09-10T08:13:38.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.606+0000 7fc8c7600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01120b0 con 0x7fc8c0101de0 2024-09-10T08:13:38.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.607+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c0101de0 0x7fc8c00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.607+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8c0102790 0x7fc8c0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.607+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c0101de0 0x7fc8c00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:58088/0 (socket says 172.21.15.2:58088) 2024-09-10T08:13:38.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.607+0000 7fc8c4c00640 1 -- 172.21.15.2:0/3117455205 learned_addr learned my addr 172.21.15.2:0/3117455205 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:38.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.607+0000 7fc8c5600640 1 --2- 172.21.15.2:0/3117455205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c0106f10 0x7fc8c0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.607+0000 7fc8c4c00640 1 -- 172.21.15.2:0/3117455205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c0106f10 msgr2=0x7fc8c0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:38.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.607+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/3117455205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c0106f10 0x7fc8c0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.607+0000 7fc8c4c00640 1 -- 172.21.15.2:0/3117455205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8c0102790 msgr2=0x7fc8c0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:38.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.607+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/3117455205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8c0102790 0x7fc8c0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.607+0000 7fc8c4c00640 1 -- 172.21.15.2:0/3117455205 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c01a6910 con 0x7fc8c0101de0 2024-09-10T08:13:38.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.607+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/3117455205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c0101de0 0x7fc8c00744f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ea60 tx=0x7fc8b000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:38.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.607+0000 7fc8be000640 1 -- 172.21.15.2:0/3117455205 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000cde0 con 0x7fc8c0101de0 2024-09-10T08:13:38.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.608+0000 7fc8c7600640 1 -- 172.21.15.2:0/3117455205 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c01a6c00 con 0x7fc8c0101de0 2024-09-10T08:13:38.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.608+0000 7fc8be000640 1 -- 172.21.15.2:0/3117455205 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8b00035f0 con 0x7fc8c0101de0 2024-09-10T08:13:38.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.608+0000 7fc8be000640 1 -- 172.21.15.2:0/3117455205 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0010670 con 0x7fc8c0101de0 2024-09-10T08:13:38.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.608+0000 7fc8c7600640 1 -- 172.21.15.2:0/3117455205 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c01a7110 con 0x7fc8c0101de0 2024-09-10T08:13:38.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.611+0000 7fc8ca060740 1 Processor -- start 2024-09-10T08:13:38.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.611+0000 7fc8be000640 1 -- 172.21.15.2:0/3117455205 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc8b00037a0 con 0x7fc8c0101de0 2024-09-10T08:13:38.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.611+0000 7fc8ca060740 1 -- start start 2024-09-10T08:13:38.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.611+0000 7fc8ca060740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee40ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.611+0000 7fc8ca060740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee4b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.611+0000 7fc8ca060740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee4bca0 0x55e33ee4e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.611+0000 7fc8ca060740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e33eda7290 con 0x55e33ee4bca0 2024-09-10T08:13:38.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.611+0000 7fc8ca060740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e33ed4eb40 con 0x55e33ee413e0 2024-09-10T08:13:38.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.611+0000 7fc8ca060740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e33eda3250 con 0x55e33ee40ac0 2024-09-10T08:13:38.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.612+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee4b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.612+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee4b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60342/0 (socket says 172.21.15.2:60342) 2024-09-10T08:13:38.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.612+0000 7fc8c4c00640 1 -- 172.21.15.2:0/2884973518 learned_addr learned my addr 172.21.15.2:0/2884973518 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:38.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.612+0000 7fc8c5600640 1 --2- 172.21.15.2:0/2884973518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee40ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.612+0000 7fc8b7e00640 1 --2- 172.21.15.2:0/2884973518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee4bca0 0x55e33ee4e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.612+0000 7fc8c4c00640 1 -- 172.21.15.2:0/2884973518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 msgr2=0x55e33ee40ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:38.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.612+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/2884973518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee40ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.612+0000 7fc8c4c00640 1 -- 172.21.15.2:0/2884973518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee4bca0 msgr2=0x55e33ee4e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:38.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.612+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/2884973518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee4bca0 0x55e33ee4e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.612+0000 7fc8c4c00640 1 -- 172.21.15.2:0/2884973518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e33eb186a0 con 0x55e33ee413e0 2024-09-10T08:13:38.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.612+0000 7fc8b7e00640 1 --2- 172.21.15.2:0/2884973518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee4bca0 0x55e33ee4e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:13:38.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.612+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/2884973518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee4b760 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00d5a90 tx=0x7fc8b00d5f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41862b4a18f0ab33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:38.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.613+0000 7fc8be000640 1 --2- 172.21.15.2:0/3117455205 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc8a40a2b80 0x7fc8a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.613+0000 7fc8b5c00640 1 -- 172.21.15.2:0/2884973518 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00de070 con 0x55e33ee413e0 2024-09-10T08:13:38.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.613+0000 7fc8b5c00640 1 -- 172.21.15.2:0/2884973518 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8b009ff00 con 0x55e33ee413e0 2024-09-10T08:13:38.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.613+0000 7fc8b5c00640 1 -- 172.21.15.2:0/2884973518 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b009e360 con 0x55e33ee413e0 2024-09-10T08:13:38.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.613+0000 7fc8be000640 1 -- 172.21.15.2:0/3117455205 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc8b00cf740 con 0x7fc8c0101de0 2024-09-10T08:13:38.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.613+0000 7fc8c5600640 1 --2- 172.21.15.2:0/3117455205 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc8a40a2b80 0x7fc8a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.614+0000 7fc8ca060740 1 -- 172.21.15.2:0/2884973518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 msgr2=0x55e33ee4b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:38.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.614+0000 7fc8ca060740 1 --2- 172.21.15.2:0/2884973518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee4b760 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00d5a90 tx=0x7fc8b00d5f50 comp rx=0 tx=0).stop 2024-09-10T08:13:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.614+0000 7fc8ca060740 1 -- 172.21.15.2:0/2884973518 shutdown_connections 2024-09-10T08:13:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.614+0000 7fc8ca060740 1 --2- 172.21.15.2:0/2884973518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee4bca0 0x55e33ee4e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.614+0000 7fc8ca060740 1 --2- 172.21.15.2:0/2884973518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee4b760 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.614+0000 7fc8ca060740 1 --2- 172.21.15.2:0/2884973518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee40ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.614+0000 7fc8ca060740 1 -- 172.21.15.2:0/2884973518 >> 172.21.15.2:0/2884973518 conn(0x55e33ee3d050 msgr2=0x55e33ee3d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.614+0000 7fc8ca060740 1 -- 172.21.15.2:0/2884973518 shutdown_connections 2024-09-10T08:13:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.614+0000 7fc8c5600640 1 --2- 172.21.15.2:0/3117455205 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc8a40a2b80 0x7fc8a40a5040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8005fd0 tx=0x7fc8b8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.614+0000 7fc8ca060740 1 -- 172.21.15.2:0/2884973518 wait complete. 2024-09-10T08:13:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.614+0000 7fc8ca060740 1 -- start start 2024-09-10T08:13:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.615+0000 7fc8ca060740 1 Processor -- start 2024-09-10T08:13:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.615+0000 7fc8ca060740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee40ac0 0x55e33ee4b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.615+0000 7fc8ca060740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee48800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.615+0000 7fc8ca060740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee4bca0 0x55e33ee48d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.615+0000 7fc8ca060740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e33ee4e910 con 0x55e33ee40ac0 2024-09-10T08:13:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.615+0000 7fc8ca060740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e33ed61fa0 con 0x55e33ee413e0 2024-09-10T08:13:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.615+0000 7fc8ca060740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e33ee4ea90 con 0x55e33ee4bca0 2024-09-10T08:13:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.615+0000 7fc8ca060740 10 client.0 _ll_register_callbacks cb 0x55e33eab5ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.615+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee40ac0 0x55e33ee4b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.615+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee40ac0 0x55e33ee4b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41432/0 (socket says 172.21.15.2:41432) 2024-09-10T08:13:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.615+0000 7fc8c4c00640 1 -- 172.21.15.2:0/699462327 learned_addr learned my addr 172.21.15.2:0/699462327 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.616+0000 7fc8b7e00640 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee48800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.616+0000 7fc8c5600640 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee4bca0 0x55e33ee48d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.616+0000 7fc8c4c00640 1 -- 172.21.15.2:0/699462327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee4bca0 msgr2=0x55e33ee48d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.616+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee4bca0 0x55e33ee48d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.616+0000 7fc8c4c00640 1 -- 172.21.15.2:0/699462327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 msgr2=0x55e33ee48800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.616+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee48800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.616+0000 7fc8c4c00640 1 -- 172.21.15.2:0/699462327 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e33ee49540 con 0x55e33ee40ac0 2024-09-10T08:13:38.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.616+0000 7fc8b7e00640 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee48800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:38.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.616+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee40ac0 0x55e33ee4b500 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00d5a60 tx=0x7fc8b009d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d16720e24e0885 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:38.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.616+0000 7fc8ae000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:13:38.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.616+0000 7fc8ae000640 1 -- 172.21.15.2:0/699462327 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b009d2e0 con 0x55e33ee40ac0 2024-09-10T08:13:38.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.616+0000 7fc8ae000640 1 -- 172.21.15.2:0/699462327 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc8b00ccf00 con 0x55e33ee40ac0 2024-09-10T08:13:38.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.616+0000 7fc8ca060740 1 -- 172.21.15.2:0/699462327 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8b009ff00 con 0x55e33ee40ac0 2024-09-10T08:13:38.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.617+0000 7fc8ae000640 1 -- 172.21.15.2:0/699462327 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc8b00de030 con 0x55e33ee40ac0 2024-09-10T08:13:38.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.617+0000 7fc8ca060740 10 client.16920 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:38.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.617+0000 7fc8ca060740 1 -- 172.21.15.2:0/699462327 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e33ee5a700 con 0x55e33ee40ac0 2024-09-10T08:13:38.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.617+0000 7fc8ae000640 1 -- 172.21.15.2:0/699462327 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8b009ff00 con 0x55e33ee40ac0 2024-09-10T08:13:38.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.617+0000 7fc8ae000640 1 -- 172.21.15.2:0/699462327 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc8b00981c0 con 0x55e33ee40ac0 2024-09-10T08:13:38.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.617+0000 7fc8ca060740 10 client.16920 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:38.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.617+0000 7fc8ca060740 10 client.16920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:38.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.617+0000 7fc8ca060740 20 client.16920 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:38.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.617+0000 7fc8ca060740 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e33ee5ba50 0x55e33ee5de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.617+0000 7fc8ca060740 4 client.16920 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:13:38.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.617+0000 7fc8ca060740 1 -- 172.21.15.2:0/699462327 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e33edb4800 con 0x55e33ee5ba50 2024-09-10T08:13:38.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.618+0000 7fc8b7e00640 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e33ee5ba50 0x55e33ee5de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.618+0000 7fc8b7e00640 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e33ee5ba50 0x55e33ee5de40 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcacba6a2d55d522 server_cookie=de95dd9e07bda020 in_seq=0 out_seq=0 2024-09-10T08:13:38.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.618+0000 7fc8ae000640 10 client.16920 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:38.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.619+0000 7fc8ae000640 1 -- 172.21.15.2:0/699462327 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e33edb4800 con 0x55e33ee5ba50 2024-09-10T08:13:38.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.619+0000 7fc8ae000640 10 client.16920 handle_command_reply: tid=0 2024-09-10T08:13:38.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.619+0000 7fc8ca060740 1 client.16920 shutdown 2024-09-10T08:13:38.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.619+0000 7fc8ca060740 1 -- 172.21.15.2:0/699462327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee40ac0 msgr2=0x55e33ee4b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:38.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.619+0000 7fc8ca060740 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee40ac0 0x55e33ee4b500 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00d5a60 tx=0x7fc8b009d930 comp rx=0 tx=0).stop 2024-09-10T08:13:38.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.619+0000 7fc8ca060740 1 -- 172.21.15.2:0/699462327 shutdown_connections 2024-09-10T08:13:38.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.619+0000 7fc8ca060740 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e33ee5ba50 0x55e33ee5de40 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.619+0000 7fc8ca060740 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee4bca0 0x55e33ee48d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.619+0000 7fc8ca060740 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee48800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.619+0000 7fc8ca060740 1 --2- 172.21.15.2:0/699462327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee40ac0 0x55e33ee4b500 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.619+0000 7fc8ca060740 1 -- 172.21.15.2:0/699462327 >> 172.21.15.2:0/699462327 conn(0x55e33ee3d050 msgr2=0x55e33ee3c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:38.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.619+0000 7fc8ae000640 0 client.16920 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:38.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.620+0000 7fc8ca060740 1 -- 172.21.15.2:0/699462327 shutdown_connections 2024-09-10T08:13:38.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.620+0000 7fc8ca060740 1 -- 172.21.15.2:0/699462327 wait complete. 2024-09-10T08:13:38.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.620+0000 7fc8ca060740 20 client.16920 trim_cache size 0 max 16384 2024-09-10T08:13:38.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.638+0000 7fc8ca060740 1 Processor -- start 2024-09-10T08:13:38.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.638+0000 7fc8ca060740 1 -- start start 2024-09-10T08:13:38.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.639+0000 7fc8ca060740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee60e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.639+0000 7fc8ca060740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.639+0000 7fc8ca060740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee4bca0 0x55e33ee45a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.639+0000 7fc8ca060740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01b3930 con 0x55e33ee4bca0 2024-09-10T08:13:38.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.639+0000 7fc8ca060740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01b1200 con 0x55e33ee413e0 2024-09-10T08:13:38.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.639+0000 7fc8ca060740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01af050 con 0x55e33ee40ac0 2024-09-10T08:13:38.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.639+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee60e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.639+0000 7fc8c5600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee4bca0 0x55e33ee45a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.639+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.639+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee60e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:58120/0 (socket says 172.21.15.2:58120) 2024-09-10T08:13:38.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.639+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee613a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60358/0 (socket says 172.21.15.2:60358) 2024-09-10T08:13:38.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.639+0000 7fc8c4c00640 1 -- 172.21.15.2:0/3054775642 learned_addr learned my addr 172.21.15.2:0/3054775642 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.640+0000 7fc8c4c00640 1 -- 172.21.15.2:0/3054775642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 msgr2=0x55e33ee613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.640+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/3054775642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.640+0000 7fc8c4c00640 1 -- 172.21.15.2:0/3054775642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee4bca0 msgr2=0x55e33ee45a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.640+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/3054775642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee4bca0 0x55e33ee45a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.640+0000 7fc8c4c00640 1 -- 172.21.15.2:0/3054775642 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e33ee3b5d0 con 0x55e33ee40ac0 2024-09-10T08:13:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.640+0000 7fc8b7e00640 1 --2- 172.21.15.2:0/3054775642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.640+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/3054775642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee60e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0097fb0 tx=0x7fc8b00d5750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e68af764c0c7af2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.641+0000 7fc8b5200640 1 -- 172.21.15.2:0/3054775642 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b009d540 con 0x55e33ee40ac0 2024-09-10T08:13:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.641+0000 7fc8b5200640 1 -- 172.21.15.2:0/3054775642 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8b00d5300 con 0x55e33ee40ac0 2024-09-10T08:13:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.641+0000 7fc8b5200640 1 -- 172.21.15.2:0/3054775642 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b009d790 con 0x55e33ee40ac0 2024-09-10T08:13:38.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.641+0000 7fc8ca060740 1 -- 172.21.15.2:0/3054775642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 msgr2=0x55e33ee60e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:38.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.641+0000 7fc8ca060740 1 --2- 172.21.15.2:0/3054775642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee60e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0097fb0 tx=0x7fc8b00d5750 comp rx=0 tx=0).stop 2024-09-10T08:13:38.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.642+0000 7fc8ca060740 1 -- 172.21.15.2:0/3054775642 shutdown_connections 2024-09-10T08:13:38.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.642+0000 7fc8ca060740 1 --2- 172.21.15.2:0/3054775642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee4bca0 0x55e33ee45a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.642+0000 7fc8ca060740 1 --2- 172.21.15.2:0/3054775642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee413e0 0x55e33ee613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.642+0000 7fc8ca060740 1 --2- 172.21.15.2:0/3054775642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee60e60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.642+0000 7fc8ca060740 1 -- 172.21.15.2:0/3054775642 >> 172.21.15.2:0/3054775642 conn(0x55e33ee3d050 msgr2=0x55e33ee43e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:38.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.642+0000 7fc8ca060740 1 -- 172.21.15.2:0/3054775642 shutdown_connections 2024-09-10T08:13:38.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.642+0000 7fc8ca060740 1 -- 172.21.15.2:0/3054775642 wait complete. 2024-09-10T08:13:38.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.642+0000 7fc8ca060740 1 -- start start 2024-09-10T08:13:38.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.642+0000 7fc8ca060740 1 Processor -- start 2024-09-10T08:13:38.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.643+0000 7fc8ca060740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee5ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.643+0000 7fc8ca060740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee413e0 0x55e33ee5f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.643+0000 7fc8ca060740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee4bca0 0x55e33ee5f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.643+0000 7fc8ca060740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01af6c0 con 0x55e33ee413e0 2024-09-10T08:13:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.643+0000 7fc8ca060740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01b0b30 con 0x55e33ee4bca0 2024-09-10T08:13:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.643+0000 7fc8ca060740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01afd90 con 0x55e33ee40ac0 2024-09-10T08:13:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.643+0000 7fc8ca060740 10 client.0 _ll_register_callbacks cb 0x7fc8b8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.643+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee5ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.643+0000 7fc8c4c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee5ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:58122/0 (socket says 172.21.15.2:58122) 2024-09-10T08:13:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.643+0000 7fc8c4c00640 1 -- 172.21.15.2:0/3593422070 learned_addr learned my addr 172.21.15.2:0/3593422070 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.643+0000 7fc8b7e00640 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee413e0 0x55e33ee5f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.643+0000 7fc8c5600640 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee4bca0 0x55e33ee5f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.644+0000 7fc8c4c00640 1 -- 172.21.15.2:0/3593422070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee4bca0 msgr2=0x55e33ee5f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.644+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee4bca0 0x55e33ee5f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.644+0000 7fc8c4c00640 1 -- 172.21.15.2:0/3593422070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee413e0 msgr2=0x55e33ee5f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.644+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee413e0 0x55e33ee5f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.644+0000 7fc8c4c00640 1 -- 172.21.15.2:0/3593422070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e33eec5440 con 0x55e33ee40ac0 2024-09-10T08:13:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.644+0000 7fc8c5600640 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee4bca0 0x55e33ee5f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:13:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.644+0000 7fc8c4c00640 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee5ece0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ea30 tx=0x7fc8b009b530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca384d79c3eef189 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.644+0000 7fc8ae000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:13:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.644+0000 7fc8ae000640 1 -- 172.21.15.2:0/3593422070 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b009d9f0 con 0x55e33ee40ac0 2024-09-10T08:13:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.644+0000 7fc8ae000640 1 -- 172.21.15.2:0/3593422070 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc8b0003b30 con 0x55e33ee40ac0 2024-09-10T08:13:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.644+0000 7fc8ca060740 1 -- 172.21.15.2:0/3593422070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8b00d5300 con 0x55e33ee40ac0 2024-09-10T08:13:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.645+0000 7fc8ae000640 1 -- 172.21.15.2:0/3593422070 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc8b00ae050 con 0x55e33ee40ac0 2024-09-10T08:13:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.645+0000 7fc8ca060740 10 client.26920 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.645+0000 7fc8ca060740 1 -- 172.21.15.2:0/3593422070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e33eec5440 con 0x55e33ee40ac0 2024-09-10T08:13:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.645+0000 7fc8ae000640 1 -- 172.21.15.2:0/3593422070 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8b00d5300 con 0x55e33ee40ac0 2024-09-10T08:13:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.645+0000 7fc8ae000640 1 -- 172.21.15.2:0/3593422070 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc8b00981c0 con 0x55e33ee40ac0 2024-09-10T08:13:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.645+0000 7fc8ca060740 10 client.26920 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.645+0000 7fc8ca060740 10 client.26920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:38.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.645+0000 7fc8ca060740 20 client.26920 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:38.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.645+0000 7fc8ca060740 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e33eec6960 0x55e33eec8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:38.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.645+0000 7fc8ca060740 4 client.26920 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:13:38.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.645+0000 7fc8ca060740 1 -- 172.21.15.2:0/3593422070 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e33edaed10 con 0x55e33eec6960 2024-09-10T08:13:38.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.646+0000 7fc8b7e00640 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e33eec6960 0x55e33eec8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:38.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.646+0000 7fc8b7e00640 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e33eec6960 0x55e33eec8d50 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a2d1253e16a0dd9 server_cookie=860306a4c9486e0e in_seq=0 out_seq=0 2024-09-10T08:13:38.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.646+0000 7fc8ae000640 10 client.26920 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:38.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.648+0000 7fc8ae000640 1 -- 172.21.15.2:0/3593422070 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e33edaed10 con 0x55e33eec6960 2024-09-10T08:13:38.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.648+0000 7fc8ae000640 10 client.26920 handle_command_reply: tid=0 2024-09-10T08:13:38.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.648+0000 7fc8ca060740 1 client.26920 shutdown 2024-09-10T08:13:38.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.648+0000 7fc8ca060740 1 -- 172.21.15.2:0/3593422070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 msgr2=0x55e33ee5ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:38.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.648+0000 7fc8ca060740 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee5ece0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ea30 tx=0x7fc8b009b530 comp rx=0 tx=0).stop 2024-09-10T08:13:38.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.649+0000 7fc8ca060740 1 -- 172.21.15.2:0/3593422070 shutdown_connections 2024-09-10T08:13:38.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.649+0000 7fc8ca060740 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e33eec6960 0x55e33eec8d50 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.649+0000 7fc8ca060740 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e33ee4bca0 0x55e33ee5f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.649+0000 7fc8ca060740 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e33ee413e0 0x55e33ee5f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.649+0000 7fc8ca060740 1 --2- 172.21.15.2:0/3593422070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e33ee40ac0 0x55e33ee5ece0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.649+0000 7fc8ca060740 1 -- 172.21.15.2:0/3593422070 >> 172.21.15.2:0/3593422070 conn(0x55e33ee3d050 msgr2=0x55e33ee470d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:38.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.649+0000 7fc8ae000640 0 client.26920 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:38.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.649+0000 7fc8ca060740 1 -- 172.21.15.2:0/3593422070 shutdown_connections 2024-09-10T08:13:38.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.649+0000 7fc8ca060740 1 -- 172.21.15.2:0/3593422070 wait complete. 2024-09-10T08:13:38.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.649+0000 7fc8ca060740 20 client.26920 trim_cache size 0 max 16384 2024-09-10T08:13:38.651 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:13:38.651 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:13:38.651 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "48fe7890-cf48-49ad-a911-b77ed47f74f6", 2024-09-10T08:13:38.651 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:13:38.651 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:13:38.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.650+0000 7fc8c7600640 1 -- 172.21.15.2:0/3117455205 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc8a40a2b80 msgr2=0x7fc8a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:38.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.650+0000 7fc8c7600640 1 --2- 172.21.15.2:0/3117455205 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc8a40a2b80 0x7fc8a40a5040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8005fd0 tx=0x7fc8b8009500 comp rx=0 tx=0).stop 2024-09-10T08:13:38.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.650+0000 7fc8c7600640 1 -- 172.21.15.2:0/3117455205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c0101de0 msgr2=0x7fc8c00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:38.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.650+0000 7fc8c7600640 1 --2- 172.21.15.2:0/3117455205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c0101de0 0x7fc8c00744f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ea60 tx=0x7fc8b000ef20 comp rx=0 tx=0).stop 2024-09-10T08:13:38.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.650+0000 7fc8c7600640 1 -- 172.21.15.2:0/3117455205 shutdown_connections 2024-09-10T08:13:38.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.650+0000 7fc8c7600640 1 --2- 172.21.15.2:0/3117455205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c0106f10 0x7fc8c0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.650+0000 7fc8c7600640 1 --2- 172.21.15.2:0/3117455205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8c0102790 0x7fc8c0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.650+0000 7fc8c7600640 1 --2- 172.21.15.2:0/3117455205 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc8a40a2b80 0x7fc8a40a5040 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.650+0000 7fc8c7600640 1 --2- 172.21.15.2:0/3117455205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c0101de0 0x7fc8c00744f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:38.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.650+0000 7fc8c7600640 1 -- 172.21.15.2:0/3117455205 >> 172.21.15.2:0/3117455205 conn(0x7fc8c00fd8c0 msgr2=0x7fc8c00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:38.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.650+0000 7fc8c7600640 1 -- 172.21.15.2:0/3117455205 shutdown_connections 2024-09-10T08:13:38.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:38.650+0000 7fc8c7600640 1 -- 172.21.15.2:0/3117455205 wait complete. 2024-09-10T08:13:38.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:38 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:13:38.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:38 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:13:38.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:38 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:13:39.676 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:13:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:39 smithi155 ceph-mon[35405]: pgmap v552: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:13:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:39 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:13:39.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:39 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:13:39.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:39 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:13:39.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:39 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:13:39.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:13:39.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:13:39.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:13:39.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:13:39.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:13:39.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:13:39.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:13:39.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:39 smithi107 ceph-mon[35048]: pgmap v552: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:13:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:39 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:13:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:39 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:13:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:39 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:13:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:39 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:13:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:13:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:13:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:13:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:13:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:13:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:13:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:13:40.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:39 smithi002 ceph-mon[30563]: pgmap v552: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:13:40.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:39 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:13:40.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:39 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:13:40.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:39 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:13:40.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:39 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:13:40.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:13:40.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:13:40.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:13:40.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:13:40.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:13:40.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:13:40.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:13:40.055 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:13:40.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.637+0000 7ff5b2600640 1 -- 172.21.15.2:0/1626376188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5ac1047e0 msgr2=0x7ff5ac108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:40.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.637+0000 7ff5b2600640 1 --2- 172.21.15.2:0/1626376188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5ac1047e0 0x7ff5ac108830 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff594009900 tx=0x7ff59402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.637+0000 7ff5b2600640 1 -- 172.21.15.2:0/1626376188 shutdown_connections 2024-09-10T08:13:40.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.637+0000 7ff5b2600640 1 --2- 172.21.15.2:0/1626376188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ac108f60 0x7ff5ac110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.637+0000 7ff5b2600640 1 --2- 172.21.15.2:0/1626376188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5ac1047e0 0x7ff5ac108830 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.637+0000 7ff5b2600640 1 --2- 172.21.15.2:0/1626376188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5ac103e30 0x7ff5ac104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.637+0000 7ff5b2600640 1 -- 172.21.15.2:0/1626376188 >> 172.21.15.2:0/1626376188 conn(0x7ff5ac0ff8d0 msgr2=0x7ff5ac101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:40.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.637+0000 7ff5b2600640 1 -- 172.21.15.2:0/1626376188 shutdown_connections 2024-09-10T08:13:40.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.637+0000 7ff5b2600640 1 -- 172.21.15.2:0/1626376188 wait complete. 2024-09-10T08:13:40.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.638+0000 7ff5b2600640 1 Processor -- start 2024-09-10T08:13:40.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.638+0000 7ff5b2600640 1 -- start start 2024-09-10T08:13:40.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.638+0000 7ff5b2600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5ac103e30 0x7ff5ac19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.638+0000 7ff5b2600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5ac1047e0 0x7ff5ac19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.638+0000 7ff5b2600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ac108f60 0x7ff5ac19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.638+0000 7ff5b2600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac114010 con 0x7ff5ac1047e0 2024-09-10T08:13:40.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.638+0000 7ff5b2600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac113e90 con 0x7ff5ac103e30 2024-09-10T08:13:40.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.638+0000 7ff5b2600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac114190 con 0x7ff5ac108f60 2024-09-10T08:13:40.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.639+0000 7ff5b1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5ac103e30 0x7ff5ac19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.639+0000 7ff5b1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5ac103e30 0x7ff5ac19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60376/0 (socket says 172.21.15.2:60376) 2024-09-10T08:13:40.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.639+0000 7ff5a3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5ac1047e0 0x7ff5ac19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.639+0000 7ff5b1200640 1 -- 172.21.15.2:0/2073600393 learned_addr learned my addr 172.21.15.2:0/2073600393 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:40.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.639+0000 7ff5b1c00640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ac108f60 0x7ff5ac19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.639+0000 7ff5b1200640 1 -- 172.21.15.2:0/2073600393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ac108f60 msgr2=0x7ff5ac19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:40.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.639+0000 7ff5b1200640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ac108f60 0x7ff5ac19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.639+0000 7ff5b1200640 1 -- 172.21.15.2:0/2073600393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5ac1047e0 msgr2=0x7ff5ac19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:40.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.639+0000 7ff5b1200640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5ac1047e0 0x7ff5ac19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.639+0000 7ff5b1200640 1 -- 172.21.15.2:0/2073600393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ac19f400 con 0x7ff5ac103e30 2024-09-10T08:13:40.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.639+0000 7ff5b1c00640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ac108f60 0x7ff5ac19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:13:40.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.639+0000 7ff5b1200640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5ac103e30 0x7ff5ac19a450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00e9f0 tx=0x7ff59c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:40.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.640+0000 7ff5aa000640 1 -- 172.21.15.2:0/2073600393 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c00cd80 con 0x7ff5ac103e30 2024-09-10T08:13:40.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.640+0000 7ff5b2600640 1 -- 172.21.15.2:0/2073600393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ac19f6f0 con 0x7ff5ac103e30 2024-09-10T08:13:40.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.640+0000 7ff5aa000640 1 -- 172.21.15.2:0/2073600393 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff59c010910 con 0x7ff5ac103e30 2024-09-10T08:13:40.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.640+0000 7ff5aa000640 1 -- 172.21.15.2:0/2073600393 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0188f0 con 0x7ff5ac103e30 2024-09-10T08:13:40.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.640+0000 7ff5b2600640 1 -- 172.21.15.2:0/2073600393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ac1a6fd0 con 0x7ff5ac103e30 2024-09-10T08:13:40.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.643+0000 7ff5b65d6740 1 Processor -- start 2024-09-10T08:13:40.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.643+0000 7ff5b65d6740 1 -- start start 2024-09-10T08:13:40.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.643+0000 7ff5aa000640 1 -- 172.21.15.2:0/2073600393 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff59c010ac0 con 0x7ff5ac103e30 2024-09-10T08:13:40.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.643+0000 7ff5b65d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8ca98d50 0x55ce8ca99130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.643+0000 7ff5b65d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca99670 0x55ce8caa39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.643+0000 7ff5b65d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8caa3f30 0x55ce8caa6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.643+0000 7ff5b65d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ce8c9ff3e0 con 0x55ce8caa3f30 2024-09-10T08:13:40.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.643+0000 7ff5b65d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ce8c9a6d80 con 0x55ce8ca99670 2024-09-10T08:13:40.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.643+0000 7ff5b65d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ce8c9fb3a0 con 0x55ce8ca98d50 2024-09-10T08:13:40.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.644+0000 7ff5b1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca99670 0x55ce8caa39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.644+0000 7ff5b1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca99670 0x55ce8caa39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60380/0 (socket says 172.21.15.2:60380) 2024-09-10T08:13:40.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.644+0000 7ff5b1200640 1 -- 172.21.15.2:0/4159586052 learned_addr learned my addr 172.21.15.2:0/4159586052 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:40.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.644+0000 7ff5a3e00640 1 --2- 172.21.15.2:0/4159586052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8caa3f30 0x55ce8caa6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.644+0000 7ff5b1c00640 1 --2- 172.21.15.2:0/4159586052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8ca98d50 0x55ce8ca99130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.644+0000 7ff5b1200640 1 -- 172.21.15.2:0/4159586052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8ca98d50 msgr2=0x55ce8ca99130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:40.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.644+0000 7ff5b1200640 1 --2- 172.21.15.2:0/4159586052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8ca98d50 0x55ce8ca99130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.644+0000 7ff5b1200640 1 -- 172.21.15.2:0/4159586052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8caa3f30 msgr2=0x55ce8caa6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:40.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.644+0000 7ff5b1200640 1 --2- 172.21.15.2:0/4159586052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8caa3f30 0x55ce8caa6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.644+0000 7ff5b1200640 1 -- 172.21.15.2:0/4159586052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce8c9c5ae0 con 0x55ce8ca99670 2024-09-10T08:13:40.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.644+0000 7ff5b1c00640 1 --2- 172.21.15.2:0/4159586052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8ca98d50 0x55ce8ca99130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:13:40.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.645+0000 7ff5b1200640 1 --2- 172.21.15.2:0/4159586052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca99670 0x55ce8caa39f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0d5c20 tx=0x7ff59c0d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c38b14f6b20b725 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:40.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.645+0000 7ff5a2000640 1 -- 172.21.15.2:0/4159586052 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0de070 con 0x55ce8ca99670 2024-09-10T08:13:40.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.645+0000 7ff5a2000640 1 -- 172.21.15.2:0/4159586052 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff59c0dab00 con 0x55ce8ca99670 2024-09-10T08:13:40.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.645+0000 7ff5a2000640 1 -- 172.21.15.2:0/4159586052 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0d2e60 con 0x55ce8ca99670 2024-09-10T08:13:40.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.645+0000 7ff5aa000640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5900a2b80 0x7ff5900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.645+0000 7ff5aa000640 1 -- 172.21.15.2:0/2073600393 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff59c0cf8f0 con 0x7ff5ac103e30 2024-09-10T08:13:40.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.646+0000 7ff5b1c00640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5900a2b80 0x7ff5900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.646+0000 7ff5b65d6740 1 -- 172.21.15.2:0/4159586052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca99670 msgr2=0x55ce8caa39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:40.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.646+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/4159586052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca99670 0x55ce8caa39f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0d5c20 tx=0x7ff59c0d7670 comp rx=0 tx=0).stop 2024-09-10T08:13:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.646+0000 7ff5b65d6740 1 -- 172.21.15.2:0/4159586052 shutdown_connections 2024-09-10T08:13:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.646+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/4159586052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8caa3f30 0x55ce8caa6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.646+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/4159586052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca99670 0x55ce8caa39f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.646+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/4159586052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8ca98d50 0x55ce8ca99130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.646+0000 7ff5b65d6740 1 -- 172.21.15.2:0/4159586052 >> 172.21.15.2:0/4159586052 conn(0x55ce8ca952e0 msgr2=0x55ce8ca956c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.646+0000 7ff5b65d6740 1 -- 172.21.15.2:0/4159586052 shutdown_connections 2024-09-10T08:13:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.646+0000 7ff5b65d6740 1 -- 172.21.15.2:0/4159586052 wait complete. 2024-09-10T08:13:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.647+0000 7ff5b1c00640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5900a2b80 0x7ff5900a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4005fd0 tx=0x7ff5a40094a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.647+0000 7ff5b65d6740 1 -- start start 2024-09-10T08:13:40.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.647+0000 7ff5b65d6740 1 Processor -- start 2024-09-10T08:13:40.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.647+0000 7ff5b65d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 0x55ce8caa0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.647+0000 7ff5b65d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8ca99670 0x55ce8caa1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.647+0000 7ff5b65d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8caa3f30 0x55ce8caa1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.647+0000 7ff5b65d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ce8caa6ba0 con 0x55ce8ca99670 2024-09-10T08:13:40.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.647+0000 7ff5b65d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ce8c9ba0f0 con 0x55ce8ca98d50 2024-09-10T08:13:40.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.647+0000 7ff5b65d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ce8caa6d20 con 0x55ce8caa3f30 2024-09-10T08:13:40.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.648+0000 7ff5b65d6740 10 client.0 _ll_register_callbacks cb 0x55ce8c9ab9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:40.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.648+0000 7ff5b1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 0x55ce8caa0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.648+0000 7ff5a3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8ca99670 0x55ce8caa1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.648+0000 7ff5a3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8ca99670 0x55ce8caa1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41484/0 (socket says 172.21.15.2:41484) 2024-09-10T08:13:40.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.648+0000 7ff5a3e00640 1 -- 172.21.15.2:0/3715552544 learned_addr learned my addr 172.21.15.2:0/3715552544 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:40.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.648+0000 7ff5b1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 0x55ce8caa0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60396/0 (socket says 172.21.15.2:60396) 2024-09-10T08:13:40.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.648+0000 7ff5b1c00640 1 --2- 172.21.15.2:0/3715552544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8caa3f30 0x55ce8caa1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.648+0000 7ff5b1200640 1 -- 172.21.15.2:0/3715552544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8caa3f30 msgr2=0x55ce8caa1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:40.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.648+0000 7ff5b1200640 1 --2- 172.21.15.2:0/3715552544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8caa3f30 0x55ce8caa1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.648+0000 7ff5b1200640 1 -- 172.21.15.2:0/3715552544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8ca99670 msgr2=0x55ce8caa1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:40.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.648+0000 7ff5b1200640 1 --2- 172.21.15.2:0/3715552544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8ca99670 0x55ce8caa1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.648+0000 7ff5b1200640 1 -- 172.21.15.2:0/3715552544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce8c7703c0 con 0x55ce8ca98d50 2024-09-10T08:13:40.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.649+0000 7ff5b1200640 1 --2- 172.21.15.2:0/3715552544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 0x55ce8caa0df0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c6a0 tx=0x7ff59c0d7ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa767f4f69693786 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:40.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.649+0000 7ff59aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:13:40.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.649+0000 7ff59aa00640 1 -- 172.21.15.2:0/3715552544 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0de070 con 0x55ce8ca98d50 2024-09-10T08:13:40.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.649+0000 7ff59aa00640 1 -- 172.21.15.2:0/3715552544 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff59c09a000 con 0x55ce8ca98d50 2024-09-10T08:13:40.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.649+0000 7ff5b65d6740 1 -- 172.21.15.2:0/3715552544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff59c0dab00 con 0x55ce8ca98d50 2024-09-10T08:13:40.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.653+0000 7ff59aa00640 1 -- 172.21.15.2:0/3715552544 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff59c097fa0 con 0x55ce8ca98d50 2024-09-10T08:13:40.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.653+0000 7ff5b65d6740 10 client.26975 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:40.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.653+0000 7ff5b65d6740 1 -- 172.21.15.2:0/3715552544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ce8cab2790 con 0x55ce8ca98d50 2024-09-10T08:13:40.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.653+0000 7ff59aa00640 1 -- 172.21.15.2:0/3715552544 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff59c0dab00 con 0x55ce8ca98d50 2024-09-10T08:13:40.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.653+0000 7ff59aa00640 1 -- 172.21.15.2:0/3715552544 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff59c0cfce0 con 0x55ce8ca98d50 2024-09-10T08:13:40.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.654+0000 7ff5b65d6740 10 client.26975 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:40.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.654+0000 7ff5b65d6740 10 client.26975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:40.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.654+0000 7ff5b65d6740 20 client.26975 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:40.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.654+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/3715552544 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ce8cab3ae0 0x55ce8cab5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.654+0000 7ff5b65d6740 4 client.26975 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:13:40.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.654+0000 7ff5b65d6740 1 -- 172.21.15.2:0/3715552544 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce8ca0c950 con 0x55ce8cab3ae0 2024-09-10T08:13:40.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.654+0000 7ff5a3e00640 1 --2- 172.21.15.2:0/3715552544 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ce8cab3ae0 0x55ce8cab5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.654+0000 7ff5a3e00640 1 --2- 172.21.15.2:0/3715552544 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ce8cab3ae0 0x55ce8cab5ed0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa9f1c943da0bafc server_cookie=a4d9221dc33bab4f in_seq=0 out_seq=0 2024-09-10T08:13:40.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.654+0000 7ff59aa00640 10 client.26975 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:40.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.655+0000 7ff59aa00640 1 -- 172.21.15.2:0/3715552544 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce8ca0c950 con 0x55ce8cab3ae0 2024-09-10T08:13:40.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.655+0000 7ff59aa00640 10 client.26975 handle_command_reply: tid=0 2024-09-10T08:13:40.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.655+0000 7ff5b65d6740 1 client.26975 shutdown 2024-09-10T08:13:40.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.655+0000 7ff5b65d6740 1 -- 172.21.15.2:0/3715552544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 msgr2=0x55ce8caa0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:40.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.655+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/3715552544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 0x55ce8caa0df0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c6a0 tx=0x7ff59c0d7ea0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.656+0000 7ff5b65d6740 1 -- 172.21.15.2:0/3715552544 shutdown_connections 2024-09-10T08:13:40.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.656+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/3715552544 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ce8cab3ae0 0x55ce8cab5ed0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.656+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/3715552544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8caa3f30 0x55ce8caa1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.656+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/3715552544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8ca99670 0x55ce8caa1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.656+0000 7ff59aa00640 0 client.26975 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:40.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.656+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/3715552544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 0x55ce8caa0df0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.656+0000 7ff5b65d6740 1 -- 172.21.15.2:0/3715552544 >> 172.21.15.2:0/3715552544 conn(0x55ce8ca952e0 msgr2=0x55ce8ca9afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:40.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.656+0000 7ff5b65d6740 1 -- 172.21.15.2:0/3715552544 shutdown_connections 2024-09-10T08:13:40.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.656+0000 7ff5b65d6740 1 -- 172.21.15.2:0/3715552544 wait complete. 2024-09-10T08:13:40.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.656+0000 7ff5b65d6740 20 client.26975 trim_cache size 0 max 16384 2024-09-10T08:13:40.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.674+0000 7ff5b65d6740 1 Processor -- start 2024-09-10T08:13:40.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.674+0000 7ff5b65d6740 1 -- start start 2024-09-10T08:13:40.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.675+0000 7ff5b65d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8ca98d50 0x55ce8ca9b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.675+0000 7ff5b65d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca99670 0x55ce8ca90060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.675+0000 7ff5b65d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8caa3f30 0x55ce8caaf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.675+0000 7ff5b65d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1b3830 con 0x55ce8caa3f30 2024-09-10T08:13:40.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.675+0000 7ff5b65d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1b1100 con 0x55ce8ca99670 2024-09-10T08:13:40.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.675+0000 7ff5b65d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1aef50 con 0x55ce8ca98d50 2024-09-10T08:13:40.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.675+0000 7ff5b1c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8caa3f30 0x55ce8caaf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.675+0000 7ff5b1c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8caa3f30 0x55ce8caaf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41488/0 (socket says 172.21.15.2:41488) 2024-09-10T08:13:40.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.675+0000 7ff5b1c00640 1 -- 172.21.15.2:0/4026272874 learned_addr learned my addr 172.21.15.2:0/4026272874 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:40.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.675+0000 7ff5b1200640 1 --2- 172.21.15.2:0/4026272874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8ca98d50 0x55ce8ca9b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.675+0000 7ff5a3e00640 1 --2- 172.21.15.2:0/4026272874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca99670 0x55ce8ca90060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.676+0000 7ff5b1c00640 1 -- 172.21.15.2:0/4026272874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8ca98d50 msgr2=0x55ce8ca9b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:40.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.676+0000 7ff5b1c00640 1 --2- 172.21.15.2:0/4026272874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8ca98d50 0x55ce8ca9b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.676+0000 7ff5b1c00640 1 -- 172.21.15.2:0/4026272874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca99670 msgr2=0x55ce8ca90060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:40.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.676+0000 7ff5b1c00640 1 --2- 172.21.15.2:0/4026272874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca99670 0x55ce8ca90060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.676+0000 7ff5b1c00640 1 -- 172.21.15.2:0/4026272874 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce8ca9fda0 con 0x55ce8caa3f30 2024-09-10T08:13:40.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.676+0000 7ff5b1200640 1 --2- 172.21.15.2:0/4026272874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8ca98d50 0x55ce8ca9b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:40.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.676+0000 7ff5a3e00640 1 --2- 172.21.15.2:0/4026272874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca99670 0x55ce8ca90060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:40.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.676+0000 7ff5b1c00640 1 --2- 172.21.15.2:0/4026272874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8caa3f30 0x55ce8caaf680 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40207c0 tx=0x7ff5a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8fa8e158f4a3df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:40.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.676+0000 7ff5a1600640 1 -- 172.21.15.2:0/4026272874 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4020220 con 0x55ce8caa3f30 2024-09-10T08:13:40.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.676+0000 7ff5a1600640 1 -- 172.21.15.2:0/4026272874 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a4012970 con 0x55ce8caa3f30 2024-09-10T08:13:40.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.676+0000 7ff5a1600640 1 -- 172.21.15.2:0/4026272874 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40056c0 con 0x55ce8caa3f30 2024-09-10T08:13:40.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.677+0000 7ff5b65d6740 1 -- 172.21.15.2:0/4026272874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8caa3f30 msgr2=0x55ce8caaf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:40.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.677+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/4026272874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8caa3f30 0x55ce8caaf680 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40207c0 tx=0x7ff5a4014000 comp rx=0 tx=0).stop 2024-09-10T08:13:40.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.677+0000 7ff5b65d6740 1 -- 172.21.15.2:0/4026272874 shutdown_connections 2024-09-10T08:13:40.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.677+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/4026272874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8caa3f30 0x55ce8caaf680 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.677+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/4026272874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca99670 0x55ce8ca90060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.677+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/4026272874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8ca98d50 0x55ce8ca9b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.677+0000 7ff5b65d6740 1 -- 172.21.15.2:0/4026272874 >> 172.21.15.2:0/4026272874 conn(0x55ce8ca952e0 msgr2=0x55ce8caaef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:40.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.677+0000 7ff5b65d6740 1 -- 172.21.15.2:0/4026272874 shutdown_connections 2024-09-10T08:13:40.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.677+0000 7ff5b65d6740 1 -- 172.21.15.2:0/4026272874 wait complete. 2024-09-10T08:13:40.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.678+0000 7ff5b65d6740 1 -- start start 2024-09-10T08:13:40.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.678+0000 7ff5b65d6740 1 Processor -- start 2024-09-10T08:13:40.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.678+0000 7ff5b65d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 0x55ce8cab6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.678+0000 7ff5b65d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8ca99670 0x55ce8cab7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.678+0000 7ff5b65d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8caa3f30 0x55ce8cab79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.678+0000 7ff5b65d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1af5c0 con 0x55ce8ca99670 2024-09-10T08:13:40.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.678+0000 7ff5b65d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1b0a30 con 0x55ce8ca98d50 2024-09-10T08:13:40.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.678+0000 7ff5b65d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1afc90 con 0x55ce8caa3f30 2024-09-10T08:13:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.678+0000 7ff5b65d6740 10 client.0 _ll_register_callbacks cb 0x7ff59c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.678+0000 7ff5a3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8ca99670 0x55ce8cab7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.678+0000 7ff5b1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 0x55ce8cab6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.679+0000 7ff5b1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 0x55ce8cab6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60412/0 (socket says 172.21.15.2:60412) 2024-09-10T08:13:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.679+0000 7ff5b1200640 1 -- 172.21.15.2:0/2440891246 learned_addr learned my addr 172.21.15.2:0/2440891246 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.679+0000 7ff5b1c00640 1 --2- 172.21.15.2:0/2440891246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8caa3f30 0x55ce8cab79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.679+0000 7ff5b1200640 1 -- 172.21.15.2:0/2440891246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8caa3f30 msgr2=0x55ce8cab79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.679+0000 7ff5b1200640 1 --2- 172.21.15.2:0/2440891246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8caa3f30 0x55ce8cab79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.679+0000 7ff5b1200640 1 -- 172.21.15.2:0/2440891246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8ca99670 msgr2=0x55ce8cab7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.679+0000 7ff5b1200640 1 --2- 172.21.15.2:0/2440891246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8ca99670 0x55ce8cab7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.679+0000 7ff5b1200640 1 -- 172.21.15.2:0/2440891246 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce8cb25f20 con 0x55ce8ca98d50 2024-09-10T08:13:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.679+0000 7ff5b1200640 1 --2- 172.21.15.2:0/2440891246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 0x55ce8cab6f50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff59c09bc90 tx=0x7ff59c0d7c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb71a0c65421e0ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.679+0000 7ff59aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:13:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.679+0000 7ff59aa00640 1 -- 172.21.15.2:0/2440891246 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0de070 con 0x55ce8ca98d50 2024-09-10T08:13:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.680+0000 7ff59aa00640 1 -- 172.21.15.2:0/2440891246 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff59c0ba5e0 con 0x55ce8ca98d50 2024-09-10T08:13:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.680+0000 7ff5b65d6740 1 -- 172.21.15.2:0/2440891246 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a4012970 con 0x55ce8ca98d50 2024-09-10T08:13:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.680+0000 7ff59aa00640 1 -- 172.21.15.2:0/2440891246 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff59c09b660 con 0x55ce8ca98d50 2024-09-10T08:13:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.680+0000 7ff5b65d6740 10 client.26981 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.680+0000 7ff5b65d6740 1 -- 172.21.15.2:0/2440891246 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ce8cb271b0 con 0x55ce8ca98d50 2024-09-10T08:13:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.680+0000 7ff59aa00640 1 -- 172.21.15.2:0/2440891246 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a4012970 con 0x55ce8ca98d50 2024-09-10T08:13:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.680+0000 7ff59aa00640 1 -- 172.21.15.2:0/2440891246 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff59c0ba2b0 con 0x55ce8ca98d50 2024-09-10T08:13:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.680+0000 7ff5b65d6740 10 client.26981 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.680+0000 7ff5b65d6740 10 client.26981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.680+0000 7ff5b65d6740 20 client.26981 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.680+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/2440891246 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ce8cb27740 0x55ce8cb29b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:40.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.680+0000 7ff5b65d6740 4 client.26981 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:13:40.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.680+0000 7ff5b65d6740 1 -- 172.21.15.2:0/2440891246 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce8ca06e60 con 0x55ce8cb27740 2024-09-10T08:13:40.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.681+0000 7ff5a3e00640 1 --2- 172.21.15.2:0/2440891246 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ce8cb27740 0x55ce8cb29b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:40.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.681+0000 7ff5a3e00640 1 --2- 172.21.15.2:0/2440891246 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ce8cb27740 0x55ce8cb29b30 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5f241ec4fc67fcf server_cookie=517e5707350f1736 in_seq=0 out_seq=0 2024-09-10T08:13:40.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.681+0000 7ff59aa00640 10 client.26981 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:40.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.681+0000 7ff59aa00640 1 -- 172.21.15.2:0/2440891246 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ce8ca06e60 con 0x55ce8cb27740 2024-09-10T08:13:40.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.681+0000 7ff59aa00640 10 client.26981 handle_command_reply: tid=0 2024-09-10T08:13:40.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.681+0000 7ff5b65d6740 1 client.26981 shutdown 2024-09-10T08:13:40.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff5b65d6740 1 -- 172.21.15.2:0/2440891246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 msgr2=0x55ce8cab6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:40.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/2440891246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 0x55ce8cab6f50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff59c09bc90 tx=0x7ff59c0d7c20 comp rx=0 tx=0).stop 2024-09-10T08:13:40.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff5b65d6740 1 -- 172.21.15.2:0/2440891246 shutdown_connections 2024-09-10T08:13:40.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/2440891246 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ce8cb27740 0x55ce8cb29b30 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/2440891246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ce8caa3f30 0x55ce8cab79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/2440891246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ce8ca99670 0x55ce8cab7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff5b65d6740 1 --2- 172.21.15.2:0/2440891246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce8ca98d50 0x55ce8cab6f50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff59aa00640 0 client.26981 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:40.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff5b65d6740 1 -- 172.21.15.2:0/2440891246 >> 172.21.15.2:0/2440891246 conn(0x55ce8ca952e0 msgr2=0x55ce8caaf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:40.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff5b65d6740 1 -- 172.21.15.2:0/2440891246 shutdown_connections 2024-09-10T08:13:40.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff5b65d6740 1 -- 172.21.15.2:0/2440891246 wait complete. 2024-09-10T08:13:40.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff5b65d6740 20 client.26981 trim_cache size 0 max 16384 2024-09-10T08:13:40.687 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:13:40.687 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:13:40.687 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:13:40.687 INFO:teuthology.orchestra.run.smithi002.stdout: "48fe7890-cf48-49ad-a911-b77ed47f74f6": { 2024-09-10T08:13:40.687 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:13:40.687 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "48fe7890-cf48-49ad-a911-b77ed47f74f6", 2024-09-10T08:13:40.687 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:13:40.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:13:40.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:13:40.687 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:13:40.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff5b2600640 1 -- 172.21.15.2:0/2073600393 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5900a2b80 msgr2=0x7ff5900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:40.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.682+0000 7ff5b2600640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5900a2b80 0x7ff5900a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4005fd0 tx=0x7ff5a40094a0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.683+0000 7ff5b2600640 1 -- 172.21.15.2:0/2073600393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5ac103e30 msgr2=0x7ff5ac19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:40.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.683+0000 7ff5b2600640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5ac103e30 0x7ff5ac19a450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00e9f0 tx=0x7ff59c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.683+0000 7ff5b2600640 1 -- 172.21.15.2:0/2073600393 shutdown_connections 2024-09-10T08:13:40.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.683+0000 7ff5b2600640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5900a2b80 0x7ff5900a5040 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.683+0000 7ff5b2600640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ac108f60 0x7ff5ac19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.683+0000 7ff5b2600640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5ac1047e0 0x7ff5ac19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.683+0000 7ff5b2600640 1 --2- 172.21.15.2:0/2073600393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5ac103e30 0x7ff5ac19a450 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:40.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.683+0000 7ff5b2600640 1 -- 172.21.15.2:0/2073600393 >> 172.21.15.2:0/2073600393 conn(0x7ff5ac0ff8d0 msgr2=0x7ff5ac100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:40.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.683+0000 7ff5b2600640 1 -- 172.21.15.2:0/2073600393 shutdown_connections 2024-09-10T08:13:40.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:13:40.683+0000 7ff5b2600640 1 -- 172.21.15.2:0/2073600393 wait complete. 2024-09-10T08:13:40.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:40 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi002 to 177.5M 2024-09-10T08:13:40.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:40 smithi155 ceph-mon[35405]: Unable to set osd_memory_target on smithi002 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T08:13:40.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:40 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi002 to 177.5M 2024-09-10T08:13:40.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:40 smithi107 ceph-mon[35048]: Unable to set osd_memory_target on smithi002 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T08:13:41.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:40 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi002 to 177.5M 2024-09-10T08:13:41.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:40 smithi002 ceph-mon[30563]: Unable to set osd_memory_target on smithi002 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T08:13:41.592 INFO:tasks.cephfs.filesystem:scrub status for tag:48fe7890-cf48-49ad-a911-b77ed47f74f6 - {'path': '/', 'tag': '48fe7890-cf48-49ad-a911-b77ed47f74f6', 'options': 'recursive,force'} 2024-09-10T08:13:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:41 smithi155 ceph-mon[35405]: pgmap v553: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:13:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:41 smithi107 ceph-mon[35048]: pgmap v553: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:13:42.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:41 smithi002 ceph-mon[30563]: pgmap v553: 129 pgs: 129 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:13:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:43 smithi155 ceph-mon[35405]: pgmap v554: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:13:43.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:43 smithi107 ceph-mon[35048]: pgmap v554: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:13:44.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:43 smithi002 ceph-mon[30563]: pgmap v554: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:13:44.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:44 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:13:45.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:44 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:13:45.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:44 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:13:45.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:45 smithi107 ceph-mon[35048]: pgmap v555: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:13:46.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:45 smithi002 ceph-mon[30563]: pgmap v555: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:13:46.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:45 smithi155 ceph-mon[35405]: pgmap v555: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:13:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.241+0000 7f35bcc00640 1 -- 172.21.15.2:0/207552422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8105910 msgr2=0x7f35b810c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.241+0000 7f35bcc00640 1 --2- 172.21.15.2:0/207552422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8105910 0x7f35b810c1a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009900 tx=0x7f35b002a400 comp rx=0 tx=0).stop 2024-09-10T08:13:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.241+0000 7f35bcc00640 1 -- 172.21.15.2:0/207552422 shutdown_connections 2024-09-10T08:13:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.241+0000 7f35bcc00640 1 --2- 172.21.15.2:0/207552422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8105910 0x7f35b810c1a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.241+0000 7f35bcc00640 1 --2- 172.21.15.2:0/207552422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35b8104f50 0x7f35b81053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.241+0000 7f35bcc00640 1 --2- 172.21.15.2:0/207552422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b8103d60 0x7f35b8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.241+0000 7f35bcc00640 1 -- 172.21.15.2:0/207552422 >> 172.21.15.2:0/207552422 conn(0x7f35b80ff4e0 msgr2=0x7f35b8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.241+0000 7f35bcc00640 1 -- 172.21.15.2:0/207552422 shutdown_connections 2024-09-10T08:13:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.241+0000 7f35bcc00640 1 -- 172.21.15.2:0/207552422 wait complete. 2024-09-10T08:13:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.242+0000 7f35bcc00640 1 Processor -- start 2024-09-10T08:13:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.242+0000 7f35bcc00640 1 -- start start 2024-09-10T08:13:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35bcc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b8103d60 0x7f35b8199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35bcc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35b8104f50 0x7f35b819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35bcc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8105910 0x7f35b81a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35bcc00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f35b810fa30 con 0x7f35b8105910 2024-09-10T08:13:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35bcc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f35b810f8b0 con 0x7f35b8103d60 2024-09-10T08:13:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35bcc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35b810fbb0 con 0x7f35b8104f50 2024-09-10T08:13:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35b6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8105910 0x7f35b81a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35ad400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35b8104f50 0x7f35b819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35b6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8105910 0x7f35b81a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41530/0 (socket says 172.21.15.2:41530) 2024-09-10T08:13:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35b6800640 1 -- 172.21.15.2:0/3061875873 learned_addr learned my addr 172.21.15.2:0/3061875873 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35b5e00640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b8103d60 0x7f35b8199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35b6800640 1 -- 172.21.15.2:0/3061875873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35b8104f50 msgr2=0x7f35b819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35b6800640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35b8104f50 0x7f35b819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35b6800640 1 -- 172.21.15.2:0/3061875873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b8103d60 msgr2=0x7f35b8199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35b6800640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b8103d60 0x7f35b8199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35b6800640 1 -- 172.21.15.2:0/3061875873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b81a1b70 con 0x7f35b8105910 2024-09-10T08:13:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.243+0000 7f35b5e00640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b8103d60 0x7f35b8199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.244+0000 7f35b6800640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8105910 0x7f35b81a1500 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f35b002a920 tx=0x7f35b0032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.244+0000 7f35af400640 1 -- 172.21.15.2:0/3061875873 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b003f030 con 0x7f35b8105910 2024-09-10T08:13:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.244+0000 7f35af400640 1 -- 172.21.15.2:0/3061875873 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b0032e50 con 0x7f35b8105910 2024-09-10T08:13:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.244+0000 7f35af400640 1 -- 172.21.15.2:0/3061875873 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0035070 con 0x7f35b8105910 2024-09-10T08:13:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.244+0000 7f35bcc00640 1 -- 172.21.15.2:0/3061875873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b81a1e00 con 0x7f35b8105910 2024-09-10T08:13:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.244+0000 7f35bcc00640 1 -- 172.21.15.2:0/3061875873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b81a2210 con 0x7f35b8105910 2024-09-10T08:13:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.247+0000 7f35bf9f2740 1 Processor -- start 2024-09-10T08:13:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.247+0000 7f35af400640 1 -- 172.21.15.2:0/3061875873 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f35b0042030 con 0x7f35b8105910 2024-09-10T08:13:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.247+0000 7f35bf9f2740 1 -- start start 2024-09-10T08:13:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.248+0000 7f35bf9f2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc61a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.248+0000 7f35bf9f2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a9d0 0x5649cc61f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.248+0000 7f35bf9f2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61fa60 0x5649cc621f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.248+0000 7f35bf9f2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5649cc57f100 con 0x5649cc61fa60 2024-09-10T08:13:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.248+0000 7f35bf9f2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5649cc5271b0 con 0x5649cc61a090 2024-09-10T08:13:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.248+0000 7f35bf9f2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5649cc57b0c0 con 0x5649cc61a9d0 2024-09-10T08:13:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.248+0000 7f35ad400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61fa60 0x5649cc621f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.248+0000 7f35ad400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc61a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.248+0000 7f35b5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a9d0 0x5649cc61f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.248+0000 7f35ad400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc61a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60430/0 (socket says 172.21.15.2:60430) 2024-09-10T08:13:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.248+0000 7f35ad400640 1 -- 172.21.15.2:0/753457492 learned_addr learned my addr 172.21.15.2:0/753457492 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.249+0000 7f35b5e00640 1 -- 172.21.15.2:0/753457492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 msgr2=0x5649cc61a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.249+0000 7f35b5e00640 1 --2- 172.21.15.2:0/753457492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc61a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.249+0000 7f35b5e00640 1 -- 172.21.15.2:0/753457492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61fa60 msgr2=0x5649cc621f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.249+0000 7f35b5e00640 1 --2- 172.21.15.2:0/753457492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61fa60 0x5649cc621f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.249+0000 7f35b5e00640 1 -- 172.21.15.2:0/753457492 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649cc292210 con 0x5649cc61a9d0 2024-09-10T08:13:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.249+0000 7f35ad400640 1 --2- 172.21.15.2:0/753457492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc61a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.249+0000 7f35b5e00640 1 --2- 172.21.15.2:0/753457492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a9d0 0x5649cc61f490 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f35b8104db0 tx=0x7f35a8011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9efb51cb8f837797 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.249+0000 7f35af400640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f359c0a2c30 0x7f359c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.249+0000 7f35a7000640 1 -- 172.21.15.2:0/753457492 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8003c30 con 0x5649cc61a9d0 2024-09-10T08:13:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.249+0000 7f35af400640 1 -- 172.21.15.2:0/3061875873 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f35b00ef890 con 0x7f35b8105910 2024-09-10T08:13:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.249+0000 7f35ad400640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f359c0a2c30 0x7f359c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.249+0000 7f35a7000640 1 -- 172.21.15.2:0/753457492 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a800fe50 con 0x5649cc61a9d0 2024-09-10T08:13:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.249+0000 7f35a7000640 1 -- 172.21.15.2:0/753457492 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a80193e0 con 0x5649cc61a9d0 2024-09-10T08:13:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.250+0000 7f35bf9f2740 1 -- 172.21.15.2:0/753457492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a9d0 msgr2=0x5649cc61f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.250+0000 7f35ad400640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f359c0a2c30 0x7f359c0a50f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f35a0010950 tx=0x7f35a0004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.250+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/753457492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a9d0 0x5649cc61f490 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f35b8104db0 tx=0x7f35a8011c70 comp rx=0 tx=0).stop 2024-09-10T08:13:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.250+0000 7f35bf9f2740 1 -- 172.21.15.2:0/753457492 shutdown_connections 2024-09-10T08:13:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.250+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/753457492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61fa60 0x5649cc621f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.250+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/753457492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a9d0 0x5649cc61f490 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.250+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/753457492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc61a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.251+0000 7f35bf9f2740 1 -- 172.21.15.2:0/753457492 >> 172.21.15.2:0/753457492 conn(0x5649cc615700 msgr2=0x5649cc615b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.251+0000 7f35bf9f2740 1 -- 172.21.15.2:0/753457492 shutdown_connections 2024-09-10T08:13:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.251+0000 7f35bf9f2740 1 -- 172.21.15.2:0/753457492 wait complete. 2024-09-10T08:13:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.251+0000 7f35bf9f2740 1 -- start start 2024-09-10T08:13:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.251+0000 7f35bf9f2740 1 Processor -- start 2024-09-10T08:13:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35bf9f2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a090 0x5649cc6241e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35bf9f2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61a9d0 0x5649cc624720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35bf9f2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61fa60 0x5649cc62b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35bf9f2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5649cc622a30 con 0x5649cc61a9d0 2024-09-10T08:13:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35bf9f2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5649cc508090 con 0x5649cc61fa60 2024-09-10T08:13:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35bf9f2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5649cc622bb0 con 0x5649cc61a090 2024-09-10T08:13:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35bf9f2740 10 client.0 _ll_register_callbacks cb 0x5649cc55cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35ad400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61a9d0 0x5649cc624720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35b6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61fa60 0x5649cc62b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35b5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a090 0x5649cc6241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35b5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a090 0x5649cc6241e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:58206/0 (socket says 172.21.15.2:58206) 2024-09-10T08:13:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35b5e00640 1 -- 172.21.15.2:0/2120799785 learned_addr learned my addr 172.21.15.2:0/2120799785 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35b6800640 1 --2- 172.21.15.2:0/2120799785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61fa60 0x5649cc62b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60440/0 (socket says 172.21.15.2:60440) 2024-09-10T08:13:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35b5e00640 1 -- 172.21.15.2:0/2120799785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61fa60 msgr2=0x5649cc62b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35b5e00640 1 --2- 172.21.15.2:0/2120799785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61fa60 0x5649cc62b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35b5e00640 1 -- 172.21.15.2:0/2120799785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61a9d0 msgr2=0x5649cc624720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35b5e00640 1 --2- 172.21.15.2:0/2120799785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61a9d0 0x5649cc624720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35b5e00640 1 -- 172.21.15.2:0/2120799785 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649cc35e620 con 0x5649cc61a090 2024-09-10T08:13:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.252+0000 7f35b6800640 1 --2- 172.21.15.2:0/2120799785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61fa60 0x5649cc62b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.253+0000 7f35b5e00640 1 --2- 172.21.15.2:0/2120799785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a090 0x5649cc6241e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f35a8000f80 tx=0x7f35a8004050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22c64c6186689fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.253+0000 7f359b400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:13:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.253+0000 7f359b400640 1 -- 172.21.15.2:0/2120799785 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8028020 con 0x5649cc61a090 2024-09-10T08:13:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.253+0000 7f35bf9f2740 1 -- 172.21.15.2:0/2120799785 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35a800fe50 con 0x5649cc61a090 2024-09-10T08:13:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.253+0000 7f359b400640 1 -- 172.21.15.2:0/2120799785 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f35a800fb80 con 0x5649cc61a090 2024-09-10T08:13:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.253+0000 7f359b400640 1 -- 172.21.15.2:0/2120799785 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f35a8033050 con 0x5649cc61a090 2024-09-10T08:13:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.253+0000 7f35bf9f2740 10 client.26947 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.253+0000 7f35bf9f2740 1 -- 172.21.15.2:0/2120799785 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5649cc62e5a0 con 0x5649cc61a090 2024-09-10T08:13:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.254+0000 7f359b400640 1 -- 172.21.15.2:0/2120799785 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a800fe50 con 0x5649cc61a090 2024-09-10T08:13:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.254+0000 7f359b400640 1 -- 172.21.15.2:0/2120799785 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f35a8018070 con 0x5649cc61a090 2024-09-10T08:13:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.254+0000 7f35bf9f2740 10 client.26947 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.254+0000 7f35bf9f2740 10 client.26947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.254+0000 7f35bf9f2740 20 client.26947 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.254+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/2120799785 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5649cc62fb30 0x5649cc631f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.254+0000 7f35bf9f2740 4 client.26947 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:13:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.254+0000 7f35bf9f2740 1 -- 172.21.15.2:0/2120799785 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649cc58c830 con 0x5649cc62fb30 2024-09-10T08:13:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.254+0000 7f35ad400640 1 --2- 172.21.15.2:0/2120799785 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5649cc62fb30 0x5649cc631f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.255+0000 7f35ad400640 1 --2- 172.21.15.2:0/2120799785 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5649cc62fb30 0x5649cc631f20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f469bce8c608b9 server_cookie=2a83670087b1b68d in_seq=0 out_seq=0 2024-09-10T08:13:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.255+0000 7f359b400640 10 client.26947 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f359b400640 1 -- 172.21.15.2:0/2120799785 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649cc58c830 con 0x5649cc62fb30 2024-09-10T08:13:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f359b400640 10 client.26947 handle_command_reply: tid=0 2024-09-10T08:13:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f35bf9f2740 1 client.26947 shutdown 2024-09-10T08:13:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f35bf9f2740 1 -- 172.21.15.2:0/2120799785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a090 msgr2=0x5649cc6241e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/2120799785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a090 0x5649cc6241e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f35a8000f80 tx=0x7f35a8004050 comp rx=0 tx=0).stop 2024-09-10T08:13:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f35bf9f2740 1 -- 172.21.15.2:0/2120799785 shutdown_connections 2024-09-10T08:13:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/2120799785 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5649cc62fb30 0x5649cc631f20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/2120799785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61fa60 0x5649cc62b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/2120799785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61a9d0 0x5649cc624720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/2120799785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a090 0x5649cc6241e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f359b400640 0 client.26947 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f35bf9f2740 1 -- 172.21.15.2:0/2120799785 >> 172.21.15.2:0/2120799785 conn(0x5649cc615700 msgr2=0x5649cc616f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f35bf9f2740 1 -- 172.21.15.2:0/2120799785 shutdown_connections 2024-09-10T08:13:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f35bf9f2740 1 -- 172.21.15.2:0/2120799785 wait complete. 2024-09-10T08:13:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.256+0000 7f35bf9f2740 20 client.26947 trim_cache size 0 max 16384 2024-09-10T08:13:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.271+0000 7f35bf9f2740 1 Processor -- start 2024-09-10T08:13:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.271+0000 7f35bf9f2740 1 -- start start 2024-09-10T08:13:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35bf9f2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc624600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35bf9f2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a9d0 0x5649cc62b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35bf9f2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61fa60 0x5649cc616af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35bf9f2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81af300 con 0x5649cc61fa60 2024-09-10T08:13:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35bf9f2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81acbd0 con 0x5649cc61a090 2024-09-10T08:13:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35bf9f2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81aaa20 con 0x5649cc61a9d0 2024-09-10T08:13:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35b6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61fa60 0x5649cc616af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35b6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61fa60 0x5649cc616af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41566/0 (socket says 172.21.15.2:41566) 2024-09-10T08:13:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35b6800640 1 -- 172.21.15.2:0/2643767262 learned_addr learned my addr 172.21.15.2:0/2643767262 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35b5e00640 1 --2- 172.21.15.2:0/2643767262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc624600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35ad400640 1 --2- 172.21.15.2:0/2643767262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a9d0 0x5649cc62b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35b6800640 1 -- 172.21.15.2:0/2643767262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a9d0 msgr2=0x5649cc62b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35b6800640 1 --2- 172.21.15.2:0/2643767262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a9d0 0x5649cc62b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35b6800640 1 -- 172.21.15.2:0/2643767262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 msgr2=0x5649cc624600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35b6800640 1 --2- 172.21.15.2:0/2643767262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc624600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35b6800640 1 -- 172.21.15.2:0/2643767262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649cc615520 con 0x5649cc61fa60 2024-09-10T08:13:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.272+0000 7f35b5e00640 1 --2- 172.21.15.2:0/2643767262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc624600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.273+0000 7f35b6800640 1 --2- 172.21.15.2:0/2643767262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61fa60 0x5649cc616af0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f35b00b8720 tx=0x7f35b00bb770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83f0333e0dd42cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.273+0000 7f35a6600640 1 -- 172.21.15.2:0/2643767262 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0049040 con 0x5649cc61fa60 2024-09-10T08:13:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.273+0000 7f35a6600640 1 -- 172.21.15.2:0/2643767262 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b00c4910 con 0x5649cc61fa60 2024-09-10T08:13:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.273+0000 7f35a6600640 1 -- 172.21.15.2:0/2643767262 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b00c4480 con 0x5649cc61fa60 2024-09-10T08:13:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.273+0000 7f35bf9f2740 1 -- 172.21.15.2:0/2643767262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61fa60 msgr2=0x5649cc616af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.273+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/2643767262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61fa60 0x5649cc616af0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f35b00b8720 tx=0x7f35b00bb770 comp rx=0 tx=0).stop 2024-09-10T08:13:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.273+0000 7f35bf9f2740 1 -- 172.21.15.2:0/2643767262 shutdown_connections 2024-09-10T08:13:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.273+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/2643767262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61fa60 0x5649cc616af0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.273+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/2643767262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61a9d0 0x5649cc62b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.273+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/2643767262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc624600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.273+0000 7f35bf9f2740 1 -- 172.21.15.2:0/2643767262 >> 172.21.15.2:0/2643767262 conn(0x5649cc615700 msgr2=0x5649cc614880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.274+0000 7f35bf9f2740 1 -- 172.21.15.2:0/2643767262 shutdown_connections 2024-09-10T08:13:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.274+0000 7f35bf9f2740 1 -- 172.21.15.2:0/2643767262 wait complete. 2024-09-10T08:13:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.274+0000 7f35bf9f2740 1 -- start start 2024-09-10T08:13:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.274+0000 7f35bf9f2740 1 Processor -- start 2024-09-10T08:13:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.274+0000 7f35bf9f2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc628e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.274+0000 7f35bf9f2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61a9d0 0x5649cc6293b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.274+0000 7f35bf9f2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61fa60 0x5649cc62a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.274+0000 7f35bf9f2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81ab090 con 0x5649cc61a9d0 2024-09-10T08:13:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.274+0000 7f35bf9f2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81ac500 con 0x5649cc61a090 2024-09-10T08:13:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.274+0000 7f35bf9f2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81ab760 con 0x5649cc61fa60 2024-09-10T08:13:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f35bf9f2740 10 client.0 _ll_register_callbacks cb 0x7f35a0009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f35ad400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61a9d0 0x5649cc6293b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f35ad400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61a9d0 0x5649cc6293b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41578/0 (socket says 172.21.15.2:41578) 2024-09-10T08:13:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f35ad400640 1 -- 172.21.15.2:0/1696017174 learned_addr learned my addr 172.21.15.2:0/1696017174 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f35b5e00640 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc628e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f35b6800640 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61fa60 0x5649cc62a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f35ad400640 1 -- 172.21.15.2:0/1696017174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61fa60 msgr2=0x5649cc62a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f35ad400640 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61fa60 0x5649cc62a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f35ad400640 1 -- 172.21.15.2:0/1696017174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 msgr2=0x5649cc628e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f35ad400640 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc628e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f35ad400640 1 -- 172.21.15.2:0/1696017174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649cc62a9d0 con 0x5649cc61a9d0 2024-09-10T08:13:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f35b5e00640 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc628e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f35ad400640 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61a9d0 0x5649cc6293b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x5649cc632790 tx=0x7f35a0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49331b144fd9c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.275+0000 7f359b400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:13:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.276+0000 7f359b400640 1 -- 172.21.15.2:0/1696017174 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0032070 con 0x5649cc61a9d0 2024-09-10T08:13:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.276+0000 7f35bf9f2740 1 -- 172.21.15.2:0/1696017174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b00c4910 con 0x5649cc61a9d0 2024-09-10T08:13:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.276+0000 7f359b400640 1 -- 172.21.15.2:0/1696017174 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f35a0020d40 con 0x5649cc61a9d0 2024-09-10T08:13:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.276+0000 7f359b400640 1 -- 172.21.15.2:0/1696017174 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f35a003e050 con 0x5649cc61a9d0 2024-09-10T08:13:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.276+0000 7f35bf9f2740 10 client.16962 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.276+0000 7f35bf9f2740 1 -- 172.21.15.2:0/1696017174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5649cc69a950 con 0x5649cc61a9d0 2024-09-10T08:13:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.276+0000 7f359b400640 1 -- 172.21.15.2:0/1696017174 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b00c4910 con 0x5649cc61a9d0 2024-09-10T08:13:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.276+0000 7f359b400640 1 -- 172.21.15.2:0/1696017174 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f35a0004be0 con 0x5649cc61a9d0 2024-09-10T08:13:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.277+0000 7f35bf9f2740 10 client.16962 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.277+0000 7f35bf9f2740 10 client.16962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.277+0000 7f35bf9f2740 20 client.16962 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.277+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5649cc69b310 0x5649cc69d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.277+0000 7f35bf9f2740 4 client.16962 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:13:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.277+0000 7f35bf9f2740 1 -- 172.21.15.2:0/1696017174 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5649cc586b80 con 0x5649cc69b310 2024-09-10T08:13:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.277+0000 7f35b5e00640 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5649cc69b310 0x5649cc69d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.277+0000 7f35b5e00640 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5649cc69b310 0x5649cc69d730 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db64e33d0b75582a server_cookie=24c960cbaebec65b in_seq=0 out_seq=0 2024-09-10T08:13:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.277+0000 7f359b400640 10 client.16962 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.291+0000 7f359b400640 1 -- 172.21.15.2:0/1696017174 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5649cc586b80 con 0x5649cc69b310 2024-09-10T08:13:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.291+0000 7f359b400640 10 client.16962 handle_command_reply: tid=0 2024-09-10T08:13:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.291+0000 7f35bf9f2740 1 client.16962 shutdown 2024-09-10T08:13:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.291+0000 7f35bf9f2740 1 -- 172.21.15.2:0/1696017174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61a9d0 msgr2=0x5649cc6293b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.291+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61a9d0 0x5649cc6293b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x5649cc632790 tx=0x7f35a0005f90 comp rx=0 tx=0).stop 2024-09-10T08:13:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.292+0000 7f35bf9f2740 1 -- 172.21.15.2:0/1696017174 shutdown_connections 2024-09-10T08:13:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.292+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5649cc69b310 0x5649cc69d730 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.292+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649cc61fa60 0x5649cc62a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.292+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649cc61a9d0 0x5649cc6293b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.292+0000 7f35bf9f2740 1 --2- 172.21.15.2:0/1696017174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5649cc61a090 0x5649cc628e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.292+0000 7f359b400640 0 client.16962 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.292+0000 7f35bf9f2740 1 -- 172.21.15.2:0/1696017174 >> 172.21.15.2:0/1696017174 conn(0x5649cc615700 msgr2=0x5649cc614880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.292+0000 7f35bf9f2740 1 -- 172.21.15.2:0/1696017174 shutdown_connections 2024-09-10T08:13:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.292+0000 7f35bf9f2740 1 -- 172.21.15.2:0/1696017174 wait complete. 2024-09-10T08:13:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.292+0000 7f35bf9f2740 20 client.16962 trim_cache size 0 max 16384 2024-09-10T08:13:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:13:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:13:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:13:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:13:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.293+0000 7f35bcc00640 1 -- 172.21.15.2:0/3061875873 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f359c0a2c30 msgr2=0x7f359c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.293+0000 7f35bcc00640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f359c0a2c30 0x7f359c0a50f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f35a0010950 tx=0x7f35a0004730 comp rx=0 tx=0).stop 2024-09-10T08:13:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.293+0000 7f35bcc00640 1 -- 172.21.15.2:0/3061875873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8105910 msgr2=0x7f35b81a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.293+0000 7f35bcc00640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8105910 0x7f35b81a1500 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f35b002a920 tx=0x7f35b0032c60 comp rx=0 tx=0).stop 2024-09-10T08:13:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.293+0000 7f35bcc00640 1 -- 172.21.15.2:0/3061875873 shutdown_connections 2024-09-10T08:13:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.293+0000 7f35bcc00640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8105910 0x7f35b81a1500 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.293+0000 7f35bcc00640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f359c0a2c30 0x7f359c0a50f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.293+0000 7f35bcc00640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35b8104f50 0x7f35b819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.293+0000 7f35bcc00640 1 --2- 172.21.15.2:0/3061875873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35b8103d60 0x7f35b8199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.293+0000 7f35bcc00640 1 -- 172.21.15.2:0/3061875873 >> 172.21.15.2:0/3061875873 conn(0x7f35b80ff4e0 msgr2=0x7f35b81081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.293+0000 7f35bcc00640 1 -- 172.21.15.2:0/3061875873 shutdown_connections 2024-09-10T08:13:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:46.293+0000 7f35bcc00640 1 -- 172.21.15.2:0/3061875873 wait complete. 2024-09-10T08:13:47.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:47 smithi107 ceph-mon[35048]: pgmap v556: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:13:48.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:47 smithi002 ceph-mon[30563]: pgmap v556: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:13:48.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:47 smithi155 ceph-mon[35405]: pgmap v556: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:13:49.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:48 smithi002 ceph-mon[30563]: pgmap v557: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:13:49.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:48 smithi155 ceph-mon[35405]: pgmap v557: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:13:49.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:48 smithi107 ceph-mon[35048]: pgmap v557: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:13:51.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:50 smithi155 ceph-mon[35405]: pgmap v558: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:13:51.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:50 smithi107 ceph-mon[35048]: pgmap v558: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:13:51.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:50 smithi002 ceph-mon[30563]: pgmap v558: 129 pgs: 129 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:13:52.935 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:52 smithi002 ceph-mon[30563]: pgmap v559: 129 pgs: 129 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:13:53.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:52 smithi155 ceph-mon[35405]: pgmap v559: 129 pgs: 129 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:13:53.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:52 smithi107 ceph-mon[35048]: pgmap v559: 129 pgs: 129 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:13:55.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:54 smithi155 ceph-mon[35405]: pgmap v560: 129 pgs: 129 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:13:55.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:54 smithi107 ceph-mon[35048]: pgmap v560: 129 pgs: 129 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:13:55.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:54 smithi002 ceph-mon[30563]: pgmap v560: 129 pgs: 129 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:13:57.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:56 smithi155 ceph-mon[35405]: pgmap v561: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:13:57.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:56 smithi107 ceph-mon[35048]: pgmap v561: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:13:57.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:56 smithi002 ceph-mon[30563]: pgmap v561: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:13:59.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:13:58 smithi155 ceph-mon[35405]: pgmap v562: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:13:59.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:13:58 smithi107 ceph-mon[35048]: pgmap v562: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:13:59.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:13:58 smithi002 ceph-mon[30563]: pgmap v562: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:13:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.420+0000 7f9b78000640 1 -- 172.21.15.2:0/136835556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70103d60 msgr2=0x7f9b70104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.420+0000 7f9b78000640 1 --2- 172.21.15.2:0/136835556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70103d60 0x7f9b70104160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009900 tx=0x7f9b6402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.420+0000 7f9b78000640 1 -- 172.21.15.2:0/136835556 shutdown_connections 2024-09-10T08:13:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.420+0000 7f9b78000640 1 --2- 172.21.15.2:0/136835556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b70105910 0x7f9b7010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.420+0000 7f9b78000640 1 --2- 172.21.15.2:0/136835556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b70104f50 0x7f9b701053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.420+0000 7f9b78000640 1 --2- 172.21.15.2:0/136835556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70103d60 0x7f9b70104160 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.420+0000 7f9b78000640 1 -- 172.21.15.2:0/136835556 >> 172.21.15.2:0/136835556 conn(0x7f9b700ff4e0 msgr2=0x7f9b70101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.420+0000 7f9b78000640 1 -- 172.21.15.2:0/136835556 shutdown_connections 2024-09-10T08:13:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.420+0000 7f9b78000640 1 -- 172.21.15.2:0/136835556 wait complete. 2024-09-10T08:13:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.421+0000 7f9b78000640 1 Processor -- start 2024-09-10T08:13:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.421+0000 7f9b78000640 1 -- start start 2024-09-10T08:13:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b78000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b70103d60 0x7f9b70199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b78000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70104f50 0x7f9b7019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b78000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b70105910 0x7f9b701a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b78000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7010f9c0 con 0x7f9b70105910 2024-09-10T08:13:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b78000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7010f840 con 0x7f9b70104f50 2024-09-10T08:13:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b78000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7010fb40 con 0x7f9b70103d60 2024-09-10T08:13:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b75600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b70103d60 0x7f9b70199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b76000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b70105910 0x7f9b701a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b75600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b70103d60 0x7f9b70199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50866/0 (socket says 172.21.15.2:50866) 2024-09-10T08:13:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b75600640 1 -- 172.21.15.2:0/3866849259 learned_addr learned my addr 172.21.15.2:0/3866849259 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b74c00640 1 --2- 172.21.15.2:0/3866849259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70104f50 0x7f9b7019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b76000640 1 -- 172.21.15.2:0/3866849259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b70103d60 msgr2=0x7f9b70199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b76000640 1 --2- 172.21.15.2:0/3866849259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b70103d60 0x7f9b70199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b76000640 1 -- 172.21.15.2:0/3866849259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70104f50 msgr2=0x7f9b7019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b76000640 1 --2- 172.21.15.2:0/3866849259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70104f50 0x7f9b7019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.422+0000 7f9b76000640 1 -- 172.21.15.2:0/3866849259 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b701a1c00 con 0x7f9b70105910 2024-09-10T08:13:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.423+0000 7f9b76000640 1 --2- 172.21.15.2:0/3866849259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b70105910 0x7f9b701a1590 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800da10 tx=0x7f9b6800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.423+0000 7f9b6e000640 1 -- 172.21.15.2:0/3866849259 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6800bd70 con 0x7f9b70105910 2024-09-10T08:13:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.423+0000 7f9b6e000640 1 -- 172.21.15.2:0/3866849259 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b68003e50 con 0x7f9b70105910 2024-09-10T08:13:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.423+0000 7f9b78000640 1 -- 172.21.15.2:0/3866849259 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b701a1ef0 con 0x7f9b70105910 2024-09-10T08:13:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.423+0000 7f9b6e000640 1 -- 172.21.15.2:0/3866849259 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b680106a0 con 0x7f9b70105910 2024-09-10T08:13:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.423+0000 7f9b78000640 1 -- 172.21.15.2:0/3866849259 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b701a2360 con 0x7f9b70105910 2024-09-10T08:13:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.426+0000 7f9b7ae0b740 1 Processor -- start 2024-09-10T08:13:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.426+0000 7f9b6e000640 1 -- 172.21.15.2:0/3866849259 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9b68003970 con 0x7f9b70105910 2024-09-10T08:13:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.427+0000 7f9b7ae0b740 1 -- start start 2024-09-10T08:13:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.427+0000 7f9b7ae0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 0x5586fa4e0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.427+0000 7f9b7ae0b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 0x5586fa4e5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.427+0000 7f9b7ae0b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.427+0000 7f9b7ae0b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5586fa445100 con 0x5586fa4e5a60 2024-09-10T08:13:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.427+0000 7f9b7ae0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5586fa3ed1b0 con 0x5586fa4e0090 2024-09-10T08:13:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.427+0000 7f9b7ae0b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5586fa4410c0 con 0x5586fa4e09d0 2024-09-10T08:13:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.427+0000 7f9b74c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.427+0000 7f9b74c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4e7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35728/0 (socket says 172.21.15.2:35728) 2024-09-10T08:13:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.427+0000 7f9b74c00640 1 -- 172.21.15.2:0/3972020622 learned_addr learned my addr 172.21.15.2:0/3972020622 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.427+0000 7f9b75600640 1 --2- 172.21.15.2:0/3972020622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 0x5586fa4e5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.427+0000 7f9b74c00640 1 --2- 172.21.15.2:0/3972020622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 0x5586fa4e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.428+0000 7f9b74c00640 1 -- 172.21.15.2:0/3972020622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 msgr2=0x5586fa4e5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.428+0000 7f9b74c00640 1 --2- 172.21.15.2:0/3972020622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 0x5586fa4e5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.428+0000 7f9b74c00640 1 -- 172.21.15.2:0/3972020622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 msgr2=0x5586fa4e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.428+0000 7f9b74c00640 1 --2- 172.21.15.2:0/3972020622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 0x5586fa4e0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.428+0000 7f9b74c00640 1 -- 172.21.15.2:0/3972020622 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586fa158210 con 0x5586fa4e5a60 2024-09-10T08:13:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.428+0000 7f9b6e000640 1 --2- 172.21.15.2:0/3866849259 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9b580a2c30 0x7f9b580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.428+0000 7f9b74c00640 1 --2- 172.21.15.2:0/3972020622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4e7f20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0058c0 tx=0x7f9b5c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8e060cf82a3cfa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.428+0000 7f9b6e000640 1 -- 172.21.15.2:0/3866849259 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9b680cf840 con 0x7f9b70105910 2024-09-10T08:13:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.428+0000 7f9b75600640 1 --2- 172.21.15.2:0/3866849259 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9b580a2c30 0x7f9b580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.428+0000 7f9b62600640 1 -- 172.21.15.2:0/3972020622 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c004030 con 0x5586fa4e5a60 2024-09-10T08:13:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.428+0000 7f9b62600640 1 -- 172.21.15.2:0/3972020622 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b5c00fca0 con 0x5586fa4e5a60 2024-09-10T08:13:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.428+0000 7f9b62600640 1 -- 172.21.15.2:0/3972020622 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c003a10 con 0x5586fa4e5a60 2024-09-10T08:13:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.429+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/3972020622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 msgr2=0x5586fa4e7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.429+0000 7f9b75600640 1 --2- 172.21.15.2:0/3866849259 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9b580a2c30 0x7f9b580a50f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9b64000c00 tx=0x7f9b640046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.429+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/3972020622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4e7f20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0058c0 tx=0x7f9b5c005850 comp rx=0 tx=0).stop 2024-09-10T08:13:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.429+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/3972020622 shutdown_connections 2024-09-10T08:13:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.429+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/3972020622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4e7f20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.429+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/3972020622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 0x5586fa4e5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.429+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/3972020622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 0x5586fa4e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.429+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/3972020622 >> 172.21.15.2:0/3972020622 conn(0x5586fa4db700 msgr2=0x5586fa4dbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.429+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/3972020622 shutdown_connections 2024-09-10T08:13:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.429+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/3972020622 wait complete. 2024-09-10T08:13:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.430+0000 7f9b7ae0b740 1 -- start start 2024-09-10T08:13:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.430+0000 7f9b7ae0b740 1 Processor -- start 2024-09-10T08:13:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b7ae0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 0x5586fa4ea280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b7ae0b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 0x5586fa4ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b7ae0b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4f1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b7ae0b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5586fa4e8a30 con 0x5586fa4e5a60 2024-09-10T08:13:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b7ae0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5586fa3ce090 con 0x5586fa4e0090 2024-09-10T08:13:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b7ae0b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5586fa4e8bb0 con 0x5586fa4e09d0 2024-09-10T08:13:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b7ae0b740 10 client.0 _ll_register_callbacks cb 0x5586fa422f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b76000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b76000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4f1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35734/0 (socket says 172.21.15.2:35734) 2024-09-10T08:13:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b76000640 1 -- 172.21.15.2:0/1732554819 learned_addr learned my addr 172.21.15.2:0/1732554819 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b75600640 1 --2- 172.21.15.2:0/1732554819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 0x5586fa4ea280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b74c00640 1 --2- 172.21.15.2:0/1732554819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 0x5586fa4ea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b76000640 1 -- 172.21.15.2:0/1732554819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 msgr2=0x5586fa4ea7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b76000640 1 --2- 172.21.15.2:0/1732554819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 0x5586fa4ea7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b76000640 1 -- 172.21.15.2:0/1732554819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 msgr2=0x5586fa4ea280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b76000640 1 --2- 172.21.15.2:0/1732554819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 0x5586fa4ea280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.431+0000 7f9b76000640 1 -- 172.21.15.2:0/1732554819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586fa224620 con 0x5586fa4e5a60 2024-09-10T08:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.432+0000 7f9b76000640 1 --2- 172.21.15.2:0/1732554819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4f1800 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9b6809bbe0 tx=0x7f9b6809c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1d6d06594f3a44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.432+0000 7f9b56a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.432+0000 7f9b56a00640 1 -- 172.21.15.2:0/1732554819 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b680b1070 con 0x5586fa4e5a60 2024-09-10T08:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.432+0000 7f9b56a00640 1 -- 172.21.15.2:0/1732554819 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9b68004000 con 0x5586fa4e5a60 2024-09-10T08:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.432+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/1732554819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b5c00fca0 con 0x5586fa4e5a60 2024-09-10T08:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.432+0000 7f9b56a00640 1 -- 172.21.15.2:0/1732554819 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9b680a48c0 con 0x5586fa4e5a60 2024-09-10T08:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.432+0000 7f9b7ae0b740 10 client.16974 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.432+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/1732554819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5586fa4f46c0 con 0x5586fa4e5a60 2024-09-10T08:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.432+0000 7f9b56a00640 1 -- 172.21.15.2:0/1732554819 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b5c00fca0 con 0x5586fa4e5a60 2024-09-10T08:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.433+0000 7f9b56a00640 1 -- 172.21.15.2:0/1732554819 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9b68024060 con 0x5586fa4e5a60 2024-09-10T08:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.433+0000 7f9b7ae0b740 10 client.16974 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.433+0000 7f9b7ae0b740 10 client.16974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.433+0000 7f9b7ae0b740 20 client.16974 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.433+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/1732554819 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586fa4f5d90 0x5586fa4f8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.433+0000 7f9b7ae0b740 4 client.16974 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.433+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/1732554819 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586fa452830 con 0x5586fa4f5d90 2024-09-10T08:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.433+0000 7f9b75600640 1 --2- 172.21.15.2:0/1732554819 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586fa4f5d90 0x5586fa4f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.434+0000 7f9b75600640 1 --2- 172.21.15.2:0/1732554819 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586fa4f5d90 0x5586fa4f8180 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7317596bb783db85 server_cookie=4f331c978c692221 in_seq=0 out_seq=0 2024-09-10T08:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.434+0000 7f9b56a00640 10 client.16974 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b56a00640 1 -- 172.21.15.2:0/1732554819 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586fa452830 con 0x5586fa4f5d90 2024-09-10T08:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b56a00640 10 client.16974 handle_command_reply: tid=0 2024-09-10T08:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b7ae0b740 1 client.16974 shutdown 2024-09-10T08:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/1732554819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 msgr2=0x5586fa4f1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/1732554819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4f1800 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9b6809bbe0 tx=0x7f9b6809c410 comp rx=0 tx=0).stop 2024-09-10T08:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/1732554819 shutdown_connections 2024-09-10T08:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/1732554819 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586fa4f5d90 0x5586fa4f8180 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/1732554819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4f1800 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/1732554819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 0x5586fa4ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/1732554819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 0x5586fa4ea280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b56a00640 0 client.16974 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/1732554819 >> 172.21.15.2:0/1732554819 conn(0x5586fa4db700 msgr2=0x5586fa4dcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/1732554819 shutdown_connections 2024-09-10T08:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/1732554819 wait complete. 2024-09-10T08:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.435+0000 7f9b7ae0b740 20 client.16974 trim_cache size 0 max 16384 2024-09-10T08:13:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.450+0000 7f9b7ae0b740 1 Processor -- start 2024-09-10T08:13:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.451+0000 7f9b7ae0b740 1 -- start start 2024-09-10T08:13:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.451+0000 7f9b7ae0b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e0090 0x5586fa4da770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.451+0000 7f9b7ae0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e09d0 0x5586fa4f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.451+0000 7f9b7ae0b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4dcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.451+0000 7f9b7ae0b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9b701af370 con 0x5586fa4e5a60 2024-09-10T08:13:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.451+0000 7f9b7ae0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9b701acc40 con 0x5586fa4e09d0 2024-09-10T08:13:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.451+0000 7f9b7ae0b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b701aaa90 con 0x5586fa4e0090 2024-09-10T08:13:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.451+0000 7f9b74c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e09d0 0x5586fa4f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.451+0000 7f9b76000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4dcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.451+0000 7f9b74c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e09d0 0x5586fa4f17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:57714/0 (socket says 172.21.15.2:57714) 2024-09-10T08:13:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.451+0000 7f9b74c00640 1 -- 172.21.15.2:0/1545848571 learned_addr learned my addr 172.21.15.2:0/1545848571 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.451+0000 7f9b75600640 1 --2- 172.21.15.2:0/1545848571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e0090 0x5586fa4da770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.452+0000 7f9b76000640 1 -- 172.21.15.2:0/1545848571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e0090 msgr2=0x5586fa4da770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.452+0000 7f9b76000640 1 --2- 172.21.15.2:0/1545848571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e0090 0x5586fa4da770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.452+0000 7f9b76000640 1 -- 172.21.15.2:0/1545848571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e09d0 msgr2=0x5586fa4f17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.452+0000 7f9b76000640 1 --2- 172.21.15.2:0/1545848571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e09d0 0x5586fa4f17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.452+0000 7f9b76000640 1 -- 172.21.15.2:0/1545848571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586fa4ea960 con 0x5586fa4e5a60 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.452+0000 7f9b74c00640 1 --2- 172.21.15.2:0/1545848571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e09d0 0x5586fa4f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.452+0000 7f9b76000640 1 --2- 172.21.15.2:0/1545848571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4dcaf0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9b680a4a80 tx=0x7f9b680a0180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8fe4d281b1a770 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.452+0000 7f9b61c00640 1 -- 172.21.15.2:0/1545848571 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b680b1050 con 0x5586fa4e5a60 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.452+0000 7f9b61c00640 1 -- 172.21.15.2:0/1545848571 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b680b4e50 con 0x5586fa4e5a60 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.452+0000 7f9b61c00640 1 -- 172.21.15.2:0/1545848571 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6809a020 con 0x5586fa4e5a60 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.453+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/1545848571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 msgr2=0x5586fa4dcaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.453+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/1545848571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4dcaf0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9b680a4a80 tx=0x7f9b680a0180 comp rx=0 tx=0).stop 2024-09-10T08:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.453+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/1545848571 shutdown_connections 2024-09-10T08:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.453+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/1545848571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4dcaf0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.453+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/1545848571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e09d0 0x5586fa4f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.453+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/1545848571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e0090 0x5586fa4da770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.453+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/1545848571 >> 172.21.15.2:0/1545848571 conn(0x5586fa4db700 msgr2=0x5586fa4da060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.453+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/1545848571 shutdown_connections 2024-09-10T08:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.453+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/1545848571 wait complete. 2024-09-10T08:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.453+0000 7f9b7ae0b740 1 -- start start 2024-09-10T08:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.453+0000 7f9b7ae0b740 1 Processor -- start 2024-09-10T08:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b7ae0b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 0x5586fa4eed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b7ae0b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 0x5586fa4ef290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b7ae0b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4f0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b7ae0b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9b701ab100 con 0x5586fa4e5a60 2024-09-10T08:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b7ae0b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9b701ac570 con 0x5586fa4e0090 2024-09-10T08:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b7ae0b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b701ab7d0 con 0x5586fa4e09d0 2024-09-10T08:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b7ae0b740 10 client.0 _ll_register_callbacks cb 0x7f9b680977d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b76000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4f0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b74c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 0x5586fa4ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b75600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 0x5586fa4eed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b74c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 0x5586fa4ef290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50902/0 (socket says 172.21.15.2:50902) 2024-09-10T08:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b76000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4f0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35756/0 (socket says 172.21.15.2:35756) 2024-09-10T08:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b74c00640 1 -- 172.21.15.2:0/2145622803 learned_addr learned my addr 172.21.15.2:0/2145622803 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b76000640 1 -- 172.21.15.2:0/2145622803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 msgr2=0x5586fa4ef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b76000640 1 --2- 172.21.15.2:0/2145622803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 0x5586fa4ef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b76000640 1 -- 172.21.15.2:0/2145622803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 msgr2=0x5586fa4eed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:13:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b76000640 1 --2- 172.21.15.2:0/2145622803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 0x5586fa4eed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.454+0000 7f9b76000640 1 -- 172.21.15.2:0/2145622803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586fa4f0840 con 0x5586fa4e5a60 2024-09-10T08:13:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.455+0000 7f9b76000640 1 --2- 172.21.15.2:0/2145622803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4f0230 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800b630 tx=0x7f9b6809b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3634773c842f27e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:13:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.455+0000 7f9b56a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:13:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.455+0000 7f9b56a00640 1 -- 172.21.15.2:0/2145622803 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b680aa060 con 0x5586fa4e5a60 2024-09-10T08:13:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.455+0000 7f9b56a00640 1 -- 172.21.15.2:0/2145622803 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9b680042e0 con 0x5586fa4e5a60 2024-09-10T08:13:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.455+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/2145622803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b680b4e50 con 0x5586fa4e5a60 2024-09-10T08:13:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.455+0000 7f9b56a00640 1 -- 172.21.15.2:0/2145622803 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9b680b1070 con 0x5586fa4e5a60 2024-09-10T08:13:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.455+0000 7f9b7ae0b740 10 client.16980 fetch_fsmap learned FSMap version 26 2024-09-10T08:13:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.455+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/2145622803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5586fa4f0840 con 0x5586fa4e5a60 2024-09-10T08:13:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.455+0000 7f9b56a00640 1 -- 172.21.15.2:0/2145622803 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b680b4e50 con 0x5586fa4e5a60 2024-09-10T08:13:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.456+0000 7f9b56a00640 1 -- 172.21.15.2:0/2145622803 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9b6807aac0 con 0x5586fa4e5a60 2024-09-10T08:13:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.456+0000 7f9b7ae0b740 10 client.16980 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:13:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.456+0000 7f9b7ae0b740 10 client.16980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:13:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.456+0000 7f9b7ae0b740 20 client.16980 populate_metadata read hostname 'smithi002' 2024-09-10T08:13:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.456+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/2145622803 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586fa560e20 0x5586fa563240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:13:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.456+0000 7f9b7ae0b740 4 client.16980 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:13:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.456+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/2145622803 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5586fa44cb80 con 0x5586fa560e20 2024-09-10T08:13:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.456+0000 7f9b75600640 1 --2- 172.21.15.2:0/2145622803 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586fa560e20 0x5586fa563240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:13:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.456+0000 7f9b56a00640 10 client.16980 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.456+0000 7f9b75600640 1 --2- 172.21.15.2:0/2145622803 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586fa560e20 0x5586fa563240 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9998a4d382b7511 server_cookie=f71cdb8f32b5b598 in_seq=0 out_seq=0 2024-09-10T08:13:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.471+0000 7f9b56a00640 1 -- 172.21.15.2:0/2145622803 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5586fa44cb80 con 0x5586fa560e20 2024-09-10T08:13:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.471+0000 7f9b56a00640 10 client.16980 handle_command_reply: tid=0 2024-09-10T08:13:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.471+0000 7f9b7ae0b740 1 client.16980 shutdown 2024-09-10T08:13:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.471+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/2145622803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 msgr2=0x5586fa4f0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.471+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/2145622803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4f0230 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800b630 tx=0x7f9b6809b4d0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.471+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/2145622803 shutdown_connections 2024-09-10T08:13:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.471+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/2145622803 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586fa560e20 0x5586fa563240 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.471+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/2145622803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586fa4e5a60 0x5586fa4f0230 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.471+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/2145622803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586fa4e09d0 0x5586fa4ef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.471+0000 7f9b7ae0b740 1 --2- 172.21.15.2:0/2145622803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586fa4e0090 0x5586fa4eed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.471+0000 7f9b56a00640 0 client.16980 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:13:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.471+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/2145622803 >> 172.21.15.2:0/2145622803 conn(0x5586fa4db700 msgr2=0x5586fa4da880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.472+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/2145622803 shutdown_connections 2024-09-10T08:13:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.472+0000 7f9b7ae0b740 1 -- 172.21.15.2:0/2145622803 wait complete. 2024-09-10T08:13:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.472+0000 7f9b7ae0b740 20 client.16980 trim_cache size 0 max 16384 2024-09-10T08:13:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:13:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:13:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:13:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:13:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.472+0000 7f9b78000640 1 -- 172.21.15.2:0/3866849259 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9b580a2c30 msgr2=0x7f9b580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.472+0000 7f9b78000640 1 --2- 172.21.15.2:0/3866849259 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9b580a2c30 0x7f9b580a50f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9b64000c00 tx=0x7f9b640046c0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.472+0000 7f9b78000640 1 -- 172.21.15.2:0/3866849259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b70105910 msgr2=0x7f9b701a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:13:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.472+0000 7f9b78000640 1 --2- 172.21.15.2:0/3866849259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b70105910 0x7f9b701a1590 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800da10 tx=0x7f9b6800ded0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.473+0000 7f9b78000640 1 -- 172.21.15.2:0/3866849259 shutdown_connections 2024-09-10T08:13:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.473+0000 7f9b78000640 1 --2- 172.21.15.2:0/3866849259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b70105910 0x7f9b701a1590 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.473+0000 7f9b78000640 1 --2- 172.21.15.2:0/3866849259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70104f50 0x7f9b7019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.473+0000 7f9b78000640 1 --2- 172.21.15.2:0/3866849259 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9b580a2c30 0x7f9b580a50f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.473+0000 7f9b78000640 1 --2- 172.21.15.2:0/3866849259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b70103d60 0x7f9b70199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:13:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.473+0000 7f9b78000640 1 -- 172.21.15.2:0/3866849259 >> 172.21.15.2:0/3866849259 conn(0x7f9b700ff4e0 msgr2=0x7f9b701082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:13:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.473+0000 7f9b78000640 1 -- 172.21.15.2:0/3866849259 shutdown_connections 2024-09-10T08:13:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:13:59.473+0000 7f9b78000640 1 -- 172.21.15.2:0/3866849259 wait complete. 2024-09-10T08:14:01.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:00 smithi155 ceph-mon[35405]: pgmap v563: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-10T08:14:01.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:00 smithi107 ceph-mon[35048]: pgmap v563: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-10T08:14:01.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:00 smithi002 ceph-mon[30563]: pgmap v563: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-10T08:14:03.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:02 smithi155 ceph-mon[35405]: pgmap v564: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-10T08:14:03.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:02 smithi107 ceph-mon[35048]: pgmap v564: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-10T08:14:03.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:02 smithi002 ceph-mon[30563]: pgmap v564: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-10T08:14:05.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:04 smithi155 ceph-mon[35405]: pgmap v565: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:14:05.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:04 smithi107 ceph-mon[35048]: pgmap v565: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:14:05.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:04 smithi002 ceph-mon[30563]: pgmap v565: 129 pgs: 129 active+clean; 2.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:14:07.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:06 smithi155 ceph-mon[35405]: pgmap v566: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:14:07.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:06 smithi107 ceph-mon[35048]: pgmap v566: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:14:07.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:06 smithi002 ceph-mon[30563]: pgmap v566: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:14:07.873 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.873 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.873 INFO:tasks.workunit.client.0.smithi002.stdout:127/600: open fd = 3 2024-09-10T08:14:07.874 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.874 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.874 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.874 INFO:tasks.workunit.client.0.smithi002.stdout:128/600: open fd = 3 2024-09-10T08:14:07.874 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.874 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.874 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.874 INFO:tasks.workunit.client.0.smithi002.stdout:129/600: open fd = 3 2024-09-10T08:14:07.874 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.874 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.874 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.875 INFO:tasks.workunit.client.0.smithi002.stdout:130/600: open fd = 3 2024-09-10T08:14:07.875 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.875 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.875 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.875 INFO:tasks.workunit.client.0.smithi002.stdout:131/600: open fd = 3 2024-09-10T08:14:07.875 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.875 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.875 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.875 INFO:tasks.workunit.client.0.smithi002.stdout:132/600: open fd = 3 2024-09-10T08:14:07.875 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.875 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.875 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.875 INFO:tasks.workunit.client.0.smithi002.stdout:133/600: open fd = 3 2024-09-10T08:14:07.876 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.876 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.876 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.876 INFO:tasks.workunit.client.0.smithi002.stdout:134/600: open fd = 3 2024-09-10T08:14:07.876 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.876 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.876 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.876 INFO:tasks.workunit.client.0.smithi002.stdout:135/600: open fd = 3 2024-09-10T08:14:07.876 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.876 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.877 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.877 INFO:tasks.workunit.client.0.smithi002.stdout:136/600: open fd = 3 2024-09-10T08:14:07.877 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.877 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.877 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.877 INFO:tasks.workunit.client.0.smithi002.stdout:137/600: open fd = 3 2024-09-10T08:14:07.877 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.877 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.878 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.878 INFO:tasks.workunit.client.0.smithi002.stdout:138/600: open fd = 3 2024-09-10T08:14:07.878 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.878 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.878 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.878 INFO:tasks.workunit.client.0.smithi002.stdout:139/600: open fd = 3 2024-09-10T08:14:07.878 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.878 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.878 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.878 INFO:tasks.workunit.client.0.smithi002.stdout:140/600: open fd = 3 2024-09-10T08:14:07.878 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.879 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.879 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.879 INFO:tasks.workunit.client.0.smithi002.stdout:141/600: open fd = 3 2024-09-10T08:14:07.879 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.879 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.879 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.879 INFO:tasks.workunit.client.0.smithi002.stdout:142/600: open fd = 3 2024-09-10T08:14:07.879 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.879 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.879 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.879 INFO:tasks.workunit.client.0.smithi002.stdout:143/600: open fd = 3 2024-09-10T08:14:07.880 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.880 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.880 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.880 INFO:tasks.workunit.client.0.smithi002.stdout:144/600: open fd = 3 2024-09-10T08:14:07.880 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.880 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.880 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.880 INFO:tasks.workunit.client.0.smithi002.stdout:145/600: open fd = 3 2024-09-10T08:14:07.880 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.880 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.880 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.880 INFO:tasks.workunit.client.0.smithi002.stdout:146/600: open fd = 3 2024-09-10T08:14:07.881 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.881 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.881 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.881 INFO:tasks.workunit.client.0.smithi002.stdout:147/600: open fd = 3 2024-09-10T08:14:07.881 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.881 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.881 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.881 INFO:tasks.workunit.client.0.smithi002.stdout:148/600: open fd = 3 2024-09-10T08:14:07.881 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.881 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.881 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.882 INFO:tasks.workunit.client.0.smithi002.stdout:149/600: open fd = 3 2024-09-10T08:14:07.882 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.882 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.882 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.882 INFO:tasks.workunit.client.0.smithi002.stdout:150/600: open fd = 3 2024-09-10T08:14:07.882 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.882 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.882 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.882 INFO:tasks.workunit.client.0.smithi002.stdout:151/600: open fd = 3 2024-09-10T08:14:07.882 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.882 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.883 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.883 INFO:tasks.workunit.client.0.smithi002.stdout:152/600: open fd = 3 2024-09-10T08:14:07.883 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.883 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.883 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.883 INFO:tasks.workunit.client.0.smithi002.stdout:153/600: open fd = 3 2024-09-10T08:14:07.883 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.883 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.883 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.883 INFO:tasks.workunit.client.0.smithi002.stdout:154/600: open fd = 3 2024-09-10T08:14:07.883 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.884 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.884 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.884 INFO:tasks.workunit.client.0.smithi002.stdout:155/600: open fd = 3 2024-09-10T08:14:07.884 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.884 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.884 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.884 INFO:tasks.workunit.client.0.smithi002.stdout:156/600: open fd = 3 2024-09-10T08:14:07.884 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.884 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.884 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.884 INFO:tasks.workunit.client.0.smithi002.stdout:157/600: open fd = 3 2024-09-10T08:14:07.884 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.885 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.885 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.885 INFO:tasks.workunit.client.0.smithi002.stdout:158/600: open fd = 3 2024-09-10T08:14:07.885 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.885 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.885 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.885 INFO:tasks.workunit.client.0.smithi002.stdout:159/600: open fd = 3 2024-09-10T08:14:07.885 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.885 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.885 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.885 INFO:tasks.workunit.client.0.smithi002.stdout:160/600: open fd = 3 2024-09-10T08:14:07.885 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.885 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.886 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.886 INFO:tasks.workunit.client.0.smithi002.stdout:161/600: open fd = 3 2024-09-10T08:14:07.886 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.886 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.886 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.886 INFO:tasks.workunit.client.0.smithi002.stdout:162/600: open fd = 3 2024-09-10T08:14:07.886 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.886 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.886 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.886 INFO:tasks.workunit.client.0.smithi002.stdout:163/600: open fd = 3 2024-09-10T08:14:07.886 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.886 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.887 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.887 INFO:tasks.workunit.client.0.smithi002.stdout:164/600: open fd = 3 2024-09-10T08:14:07.887 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.887 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.887 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.887 INFO:tasks.workunit.client.0.smithi002.stdout:165/600: open fd = 3 2024-09-10T08:14:07.887 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.887 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.887 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.887 INFO:tasks.workunit.client.0.smithi002.stdout:166/600: open fd = 3 2024-09-10T08:14:07.887 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.887 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.888 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.888 INFO:tasks.workunit.client.0.smithi002.stdout:167/600: open fd = 3 2024-09-10T08:14:07.888 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.888 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.888 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.888 INFO:tasks.workunit.client.0.smithi002.stdout:168/600: open fd = 3 2024-09-10T08:14:07.888 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.888 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.888 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.888 INFO:tasks.workunit.client.0.smithi002.stdout:169/600: open fd = 3 2024-09-10T08:14:07.888 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.889 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.889 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.889 INFO:tasks.workunit.client.0.smithi002.stdout:170/600: open fd = 3 2024-09-10T08:14:07.889 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.889 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.889 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.889 INFO:tasks.workunit.client.0.smithi002.stdout:171/600: open fd = 3 2024-09-10T08:14:07.889 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.889 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.889 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.890 INFO:tasks.workunit.client.0.smithi002.stdout:172/600: open fd = 3 2024-09-10T08:14:07.890 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.890 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.890 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.890 INFO:tasks.workunit.client.0.smithi002.stdout:173/600: open fd = 3 2024-09-10T08:14:07.890 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.890 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.890 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.890 INFO:tasks.workunit.client.0.smithi002.stdout:174/600: open fd = 3 2024-09-10T08:14:07.890 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.890 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.890 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.891 INFO:tasks.workunit.client.0.smithi002.stdout:175/600: open fd = 3 2024-09-10T08:14:07.891 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.891 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.891 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.891 INFO:tasks.workunit.client.0.smithi002.stdout:176/600: open fd = 3 2024-09-10T08:14:07.891 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.891 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.891 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.891 INFO:tasks.workunit.client.0.smithi002.stdout:177/600: open fd = 3 2024-09-10T08:14:07.891 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.891 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.891 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.892 INFO:tasks.workunit.client.0.smithi002.stdout:178/600: open fd = 3 2024-09-10T08:14:07.892 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.892 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.892 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.892 INFO:tasks.workunit.client.0.smithi002.stdout:179/600: open fd = 3 2024-09-10T08:14:07.892 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.892 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.892 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.892 INFO:tasks.workunit.client.0.smithi002.stdout:180/600: open fd = 3 2024-09-10T08:14:07.892 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.893 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.893 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.893 INFO:tasks.workunit.client.0.smithi002.stdout:181/600: open fd = 3 2024-09-10T08:14:07.893 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.893 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.893 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.893 INFO:tasks.workunit.client.0.smithi002.stdout:182/600: open fd = 3 2024-09-10T08:14:07.893 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.893 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.893 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.894 INFO:tasks.workunit.client.0.smithi002.stdout:183/600: open fd = 3 2024-09-10T08:14:07.894 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.894 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.894 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.894 INFO:tasks.workunit.client.0.smithi002.stdout:184/600: open fd = 3 2024-09-10T08:14:07.894 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.894 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.894 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.894 INFO:tasks.workunit.client.0.smithi002.stdout:185/600: open fd = 3 2024-09-10T08:14:07.894 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.894 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.895 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.895 INFO:tasks.workunit.client.0.smithi002.stdout:186/600: open fd = 3 2024-09-10T08:14:07.895 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.895 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.895 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.895 INFO:tasks.workunit.client.0.smithi002.stdout:187/600: open fd = 3 2024-09-10T08:14:07.895 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.895 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.895 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:14:07.895 INFO:tasks.workunit.client.0.smithi002.stdout:188/600: open fd = 3 2024-09-10T08:14:07.895 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:07.895 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:14:09.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:08 smithi155 ceph-mon[35405]: pgmap v567: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:14:09.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:08 smithi107 ceph-mon[35048]: pgmap v567: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:14:09.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:08 smithi002 ceph-mon[30563]: pgmap v567: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:14:11.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:10 smithi155 ceph-mon[35405]: pgmap v568: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:14:11.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:10 smithi107 ceph-mon[35048]: pgmap v568: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:14:11.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:10 smithi002 ceph-mon[30563]: pgmap v568: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:14:11.593 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:14:11.990 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.592+0000 7f16c3200640 1 -- 172.21.15.2:0/3211533794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16bc104f50 msgr2=0x7f16bc1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.592+0000 7f16c3200640 1 --2- 172.21.15.2:0/3211533794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16bc104f50 0x7f16bc1053d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009900 tx=0x7f16a802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.592+0000 7f16c3200640 1 -- 172.21.15.2:0/3211533794 shutdown_connections 2024-09-10T08:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.592+0000 7f16c3200640 1 --2- 172.21.15.2:0/3211533794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16bc105910 0x7f16bc10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.592+0000 7f16c3200640 1 --2- 172.21.15.2:0/3211533794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16bc104f50 0x7f16bc1053d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.592+0000 7f16c3200640 1 --2- 172.21.15.2:0/3211533794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc103d60 0x7f16bc104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.592+0000 7f16c3200640 1 -- 172.21.15.2:0/3211533794 >> 172.21.15.2:0/3211533794 conn(0x7f16bc0ff4e0 msgr2=0x7f16bc101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.592+0000 7f16c3200640 1 -- 172.21.15.2:0/3211533794 shutdown_connections 2024-09-10T08:14:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.592+0000 7f16c3200640 1 -- 172.21.15.2:0/3211533794 wait complete. 2024-09-10T08:14:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.593+0000 7f16c3200640 1 Processor -- start 2024-09-10T08:14:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.593+0000 7f16c3200640 1 -- start start 2024-09-10T08:14:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.593+0000 7f16c3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16bc103d60 0x7f16bc19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.593+0000 7f16c3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc104f50 0x7f16bc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.593+0000 7f16c3200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16bc105910 0x7f16bc1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.593+0000 7f16c3200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc10fa90 con 0x7f16bc105910 2024-09-10T08:14:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.593+0000 7f16c3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc10f910 con 0x7f16bc104f50 2024-09-10T08:14:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.593+0000 7f16c3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc10fc10 con 0x7f16bc103d60 2024-09-10T08:14:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.594+0000 7f16bbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16bc103d60 0x7f16bc19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.594+0000 7f16bbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16bc103d60 0x7f16bc19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:57374/0 (socket says 172.21.15.2:57374) 2024-09-10T08:14:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.594+0000 7f16bbe00640 1 -- 172.21.15.2:0/1335682304 learned_addr learned my addr 172.21.15.2:0/1335682304 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.594+0000 7f16c1200640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16bc105910 0x7f16bc1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.594+0000 7f16bb400640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc104f50 0x7f16bc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.594+0000 7f16bbe00640 1 -- 172.21.15.2:0/1335682304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc104f50 msgr2=0x7f16bc19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.594+0000 7f16bbe00640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc104f50 0x7f16bc19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.594+0000 7f16bbe00640 1 -- 172.21.15.2:0/1335682304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16bc105910 msgr2=0x7f16bc1a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.594+0000 7f16bbe00640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16bc105910 0x7f16bc1a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.594+0000 7f16bbe00640 1 -- 172.21.15.2:0/1335682304 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc1a1ca0 con 0x7f16bc103d60 2024-09-10T08:14:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.594+0000 7f16bbe00640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16bc103d60 0x7f16bc19a080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00d950 tx=0x7f16ac00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.595+0000 7f16b8c00640 1 -- 172.21.15.2:0/1335682304 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac017020 con 0x7f16bc103d60 2024-09-10T08:14:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.595+0000 7f16b8c00640 1 -- 172.21.15.2:0/1335682304 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16ac009c30 con 0x7f16bc103d60 2024-09-10T08:14:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.595+0000 7f16b8c00640 1 -- 172.21.15.2:0/1335682304 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac00f720 con 0x7f16bc103d60 2024-09-10T08:14:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.595+0000 7f16c3200640 1 -- 172.21.15.2:0/1335682304 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc1a1f90 con 0x7f16bc103d60 2024-09-10T08:14:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.595+0000 7f16c3200640 1 -- 172.21.15.2:0/1335682304 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc1a2340 con 0x7f16bc103d60 2024-09-10T08:14:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.598+0000 7f16c6071740 1 Processor -- start 2024-09-10T08:14:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.598+0000 7f16c6071740 1 -- start start 2024-09-10T08:14:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.598+0000 7f16b8c00640 1 -- 172.21.15.2:0/1335682304 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f16ac01a020 con 0x7f16bc103d60 2024-09-10T08:14:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.598+0000 7f16c6071740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db090 0x55cd508db490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.598+0000 7f16c6071740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db9d0 0x55cd508e0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.598+0000 7f16c6071740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.598+0000 7f16c6071740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cd50840100 con 0x55cd508e0a60 2024-09-10T08:14:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.599+0000 7f16c6071740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cd507e81b0 con 0x55cd508db9d0 2024-09-10T08:14:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.599+0000 7f16c6071740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cd5083c0c0 con 0x55cd508db090 2024-09-10T08:14:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.599+0000 7f16c1200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db090 0x55cd508db490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.599+0000 7f16c1200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db090 0x55cd508db490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:57382/0 (socket says 172.21.15.2:57382) 2024-09-10T08:14:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.599+0000 7f16c1200640 1 -- 172.21.15.2:0/2555842747 learned_addr learned my addr 172.21.15.2:0/2555842747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.599+0000 7f16bbe00640 1 --2- 172.21.15.2:0/2555842747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db9d0 0x55cd508e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.599+0000 7f16bb400640 1 --2- 172.21.15.2:0/2555842747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.599+0000 7f16bb400640 1 -- 172.21.15.2:0/2555842747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db090 msgr2=0x55cd508db490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.599+0000 7f16bb400640 1 --2- 172.21.15.2:0/2555842747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db090 0x55cd508db490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.599+0000 7f16bb400640 1 -- 172.21.15.2:0/2555842747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db9d0 msgr2=0x55cd508e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.599+0000 7f16bb400640 1 --2- 172.21.15.2:0/2555842747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db9d0 0x55cd508e0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.599+0000 7f16bb400640 1 -- 172.21.15.2:0/2555842747 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd50553210 con 0x55cd508e0a60 2024-09-10T08:14:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.599+0000 7f16bbe00640 1 --2- 172.21.15.2:0/2555842747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db9d0 0x55cd508e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.600+0000 7f16bb400640 1 --2- 172.21.15.2:0/2555842747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508e2f20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f16a802a9e0 tx=0x7f16a8004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7815bd23d640bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.600+0000 7f16b8c00640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16a00a2c30 0x7f16a00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.600+0000 7f16b1600640 1 -- 172.21.15.2:0/2555842747 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8035040 con 0x55cd508e0a60 2024-09-10T08:14:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.600+0000 7f16b8c00640 1 -- 172.21.15.2:0/1335682304 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f16ac0cf2e0 con 0x7f16bc103d60 2024-09-10T08:14:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.600+0000 7f16b1600640 1 -- 172.21.15.2:0/2555842747 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a8005520 con 0x55cd508e0a60 2024-09-10T08:14:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.600+0000 7f16b1600640 1 -- 172.21.15.2:0/2555842747 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8031910 con 0x55cd508e0a60 2024-09-10T08:14:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.600+0000 7f16c1200640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16a00a2c30 0x7f16a00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.601+0000 7f16c6071740 1 -- 172.21.15.2:0/2555842747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 msgr2=0x55cd508e2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.601+0000 7f16c1200640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16a00a2c30 0x7f16a00a50f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f16b400a510 tx=0x7f16b400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.601+0000 7f16c6071740 1 --2- 172.21.15.2:0/2555842747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508e2f20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f16a802a9e0 tx=0x7f16a8004170 comp rx=0 tx=0).stop 2024-09-10T08:14:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.601+0000 7f16c6071740 1 -- 172.21.15.2:0/2555842747 shutdown_connections 2024-09-10T08:14:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.601+0000 7f16c6071740 1 --2- 172.21.15.2:0/2555842747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508e2f20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.601+0000 7f16c6071740 1 --2- 172.21.15.2:0/2555842747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db9d0 0x55cd508e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.601+0000 7f16c6071740 1 --2- 172.21.15.2:0/2555842747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db090 0x55cd508db490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.601+0000 7f16c6071740 1 -- 172.21.15.2:0/2555842747 >> 172.21.15.2:0/2555842747 conn(0x55cd508d6700 msgr2=0x55cd508d6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.601+0000 7f16c6071740 1 -- 172.21.15.2:0/2555842747 shutdown_connections 2024-09-10T08:14:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.601+0000 7f16c6071740 1 -- 172.21.15.2:0/2555842747 wait complete. 2024-09-10T08:14:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.602+0000 7f16c6071740 1 -- start start 2024-09-10T08:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.602+0000 7f16c6071740 1 Processor -- start 2024-09-10T08:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.602+0000 7f16c6071740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.602+0000 7f16c6071740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db9d0 0x55cd508e5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.602+0000 7f16c6071740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.602+0000 7f16c6071740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cd508e39d0 con 0x55cd508e0a60 2024-09-10T08:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.602+0000 7f16c6071740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cd507c9090 con 0x55cd508db090 2024-09-10T08:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.602+0000 7f16c6071740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cd508e3b50 con 0x55cd508db9d0 2024-09-10T08:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.602+0000 7f16c6071740 10 client.0 _ll_register_callbacks cb 0x55cd5081df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.603+0000 7f16c1200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.603+0000 7f16c1200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508ec760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45004/0 (socket says 172.21.15.2:45004) 2024-09-10T08:14:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.603+0000 7f16c1200640 1 -- 172.21.15.2:0/3802593486 learned_addr learned my addr 172.21.15.2:0/3802593486 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.603+0000 7f16bbe00640 1 --2- 172.21.15.2:0/3802593486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.603+0000 7f16bb400640 1 --2- 172.21.15.2:0/3802593486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db9d0 0x55cd508e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.603+0000 7f16c1200640 1 -- 172.21.15.2:0/3802593486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db9d0 msgr2=0x55cd508e5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.603+0000 7f16c1200640 1 --2- 172.21.15.2:0/3802593486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db9d0 0x55cd508e5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.603+0000 7f16c1200640 1 -- 172.21.15.2:0/3802593486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 msgr2=0x55cd508e51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.603+0000 7f16c1200640 1 --2- 172.21.15.2:0/3802593486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508e51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.603+0000 7f16c1200640 1 -- 172.21.15.2:0/3802593486 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd5061f620 con 0x55cd508e0a60 2024-09-10T08:14:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.603+0000 7f16c1200640 1 --2- 172.21.15.2:0/3802593486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508ec760 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f16b4018de0 tx=0x7f16b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a8b9c8b9d7f762 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.603+0000 7f16a6000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.604+0000 7f16a6000640 1 -- 172.21.15.2:0/3802593486 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4028060 con 0x55cd508e0a60 2024-09-10T08:14:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.604+0000 7f16a6000640 1 -- 172.21.15.2:0/3802593486 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f16b4023670 con 0x55cd508e0a60 2024-09-10T08:14:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.604+0000 7f16c6071740 1 -- 172.21.15.2:0/3802593486 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a8005520 con 0x55cd508e0a60 2024-09-10T08:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.604+0000 7f16a6000640 1 -- 172.21.15.2:0/3802593486 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16b402f050 con 0x55cd508e0a60 2024-09-10T08:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.604+0000 7f16c6071740 10 client.16992 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.604+0000 7f16c6071740 1 -- 172.21.15.2:0/3802593486 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cd508ef620 con 0x55cd508e0a60 2024-09-10T08:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.604+0000 7f16a6000640 1 -- 172.21.15.2:0/3802593486 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a8005520 con 0x55cd508e0a60 2024-09-10T08:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.605+0000 7f16a6000640 1 -- 172.21.15.2:0/3802593486 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f16b4017830 con 0x55cd508e0a60 2024-09-10T08:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.605+0000 7f16c6071740 10 client.16992 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.605+0000 7f16c6071740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.605+0000 7f16c6071740 20 client.16992 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.605+0000 7f16c6071740 1 --2- 172.21.15.2:0/3802593486 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd508f0cf0 0x55cd508f3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.605+0000 7f16c6071740 4 client.16992 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.605+0000 7f16c6071740 1 -- 172.21.15.2:0/3802593486 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd5084d830 con 0x55cd508f0cf0 2024-09-10T08:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.605+0000 7f16bbe00640 1 --2- 172.21.15.2:0/3802593486 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd508f0cf0 0x55cd508f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.605+0000 7f16bbe00640 1 --2- 172.21.15.2:0/3802593486 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd508f0cf0 0x55cd508f3120 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ee9faa80b0c6fe server_cookie=ed397aea7c874cc1 in_seq=0 out_seq=0 2024-09-10T08:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.605+0000 7f16a6000640 10 client.16992 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.606+0000 7f16a6000640 1 -- 172.21.15.2:0/3802593486 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd5084d830 con 0x55cd508f0cf0 2024-09-10T08:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.606+0000 7f16a6000640 10 client.16992 handle_command_reply: tid=0 2024-09-10T08:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.606+0000 7f16c6071740 1 client.16992 shutdown 2024-09-10T08:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.606+0000 7f16c6071740 1 -- 172.21.15.2:0/3802593486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 msgr2=0x55cd508ec760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.606+0000 7f16c6071740 1 --2- 172.21.15.2:0/3802593486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508ec760 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f16b4018de0 tx=0x7f16b4004a40 comp rx=0 tx=0).stop 2024-09-10T08:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.607+0000 7f16c6071740 1 -- 172.21.15.2:0/3802593486 shutdown_connections 2024-09-10T08:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.607+0000 7f16c6071740 1 --2- 172.21.15.2:0/3802593486 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd508f0cf0 0x55cd508f3120 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.607+0000 7f16c6071740 1 --2- 172.21.15.2:0/3802593486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508ec760 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.607+0000 7f16c6071740 1 --2- 172.21.15.2:0/3802593486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db9d0 0x55cd508e5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.607+0000 7f16c6071740 1 --2- 172.21.15.2:0/3802593486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508e51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.607+0000 7f16c6071740 1 -- 172.21.15.2:0/3802593486 >> 172.21.15.2:0/3802593486 conn(0x55cd508d6700 msgr2=0x55cd508d7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.607+0000 7f16a6000640 0 client.16992 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.607+0000 7f16c6071740 1 -- 172.21.15.2:0/3802593486 shutdown_connections 2024-09-10T08:14:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.607+0000 7f16c6071740 1 -- 172.21.15.2:0/3802593486 wait complete. 2024-09-10T08:14:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.607+0000 7f16c6071740 20 client.16992 trim_cache size 0 max 16384 2024-09-10T08:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.622+0000 7f16c6071740 1 Processor -- start 2024-09-10T08:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.622+0000 7f16c6071740 1 -- start start 2024-09-10T08:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.622+0000 7f16c6071740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508d5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.622+0000 7f16c6071740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508db9d0 0x55cd508ec720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.622+0000 7f16c6071740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508e0a60 0x55cd508f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.622+0000 7f16c6071740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1af3f0 con 0x55cd508db9d0 2024-09-10T08:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.622+0000 7f16c6071740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1accc0 con 0x55cd508db090 2024-09-10T08:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.622+0000 7f16c6071740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1aab10 con 0x55cd508e0a60 2024-09-10T08:14:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.623+0000 7f16bb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508db9d0 0x55cd508ec720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.623+0000 7f16bb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508db9d0 0x55cd508ec720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45016/0 (socket says 172.21.15.2:45016) 2024-09-10T08:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.623+0000 7f16c1200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508e0a60 0x55cd508f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.623+0000 7f16bb400640 1 -- 172.21.15.2:0/163379304 learned_addr learned my addr 172.21.15.2:0/163379304 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.623+0000 7f16bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508d5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.623+0000 7f16bbe00640 1 -- 172.21.15.2:0/163379304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508e0a60 msgr2=0x55cd508f7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.623+0000 7f16bbe00640 1 --2- 172.21.15.2:0/163379304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508e0a60 0x55cd508f7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.623+0000 7f16bbe00640 1 -- 172.21.15.2:0/163379304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508db9d0 msgr2=0x55cd508ec720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.623+0000 7f16bbe00640 1 --2- 172.21.15.2:0/163379304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508db9d0 0x55cd508ec720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.623+0000 7f16bbe00640 1 -- 172.21.15.2:0/163379304 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd508d6520 con 0x55cd508db090 2024-09-10T08:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.623+0000 7f16c1200640 1 --2- 172.21.15.2:0/163379304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508e0a60 0x55cd508f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.623+0000 7f16bbe00640 1 --2- 172.21.15.2:0/163379304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508d5770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0cca40 tx=0x7f16ac0d17c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1436e1c5ed65eb9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.624+0000 7f16b0c00640 1 -- 172.21.15.2:0/163379304 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac09b0b0 con 0x55cd508db090 2024-09-10T08:14:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.624+0000 7f16b0c00640 1 -- 172.21.15.2:0/163379304 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16ac0cf120 con 0x55cd508db090 2024-09-10T08:14:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.624+0000 7f16b0c00640 1 -- 172.21.15.2:0/163379304 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0ccb00 con 0x55cd508db090 2024-09-10T08:14:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.624+0000 7f16c6071740 1 -- 172.21.15.2:0/163379304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 msgr2=0x55cd508d5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.624+0000 7f16c6071740 1 --2- 172.21.15.2:0/163379304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508d5770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0cca40 tx=0x7f16ac0d17c0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.625+0000 7f16c6071740 1 -- 172.21.15.2:0/163379304 shutdown_connections 2024-09-10T08:14:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.625+0000 7f16c6071740 1 --2- 172.21.15.2:0/163379304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508e0a60 0x55cd508f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.625+0000 7f16c6071740 1 --2- 172.21.15.2:0/163379304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508db9d0 0x55cd508ec720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.625+0000 7f16c6071740 1 --2- 172.21.15.2:0/163379304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508d5770 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.625+0000 7f16c6071740 1 -- 172.21.15.2:0/163379304 >> 172.21.15.2:0/163379304 conn(0x55cd508d6700 msgr2=0x55cd508d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.625+0000 7f16c6071740 1 -- 172.21.15.2:0/163379304 shutdown_connections 2024-09-10T08:14:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.625+0000 7f16c6071740 1 -- 172.21.15.2:0/163379304 wait complete. 2024-09-10T08:14:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.625+0000 7f16c6071740 1 -- start start 2024-09-10T08:14:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.625+0000 7f16c6071740 1 Processor -- start 2024-09-10T08:14:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16c6071740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508e9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16c6071740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db9d0 0x55cd508e9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16c6071740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508eaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16c6071740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1ab180 con 0x55cd508e0a60 2024-09-10T08:14:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16c6071740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1ac5f0 con 0x55cd508db090 2024-09-10T08:14:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16c6071740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1ab850 con 0x55cd508db9d0 2024-09-10T08:14:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16c6071740 10 client.0 _ll_register_callbacks cb 0x7f16b4009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16c1200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508eaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508e9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508e9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42244/0 (socket says 172.21.15.2:42244) 2024-09-10T08:14:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16bbe00640 1 -- 172.21.15.2:0/2421738005 learned_addr learned my addr 172.21.15.2:0/2421738005 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16c1200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508eaf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45028/0 (socket says 172.21.15.2:45028) 2024-09-10T08:14:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16bb400640 1 --2- 172.21.15.2:0/2421738005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db9d0 0x55cd508e9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16bbe00640 1 -- 172.21.15.2:0/2421738005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db9d0 msgr2=0x55cd508e9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16bbe00640 1 --2- 172.21.15.2:0/2421738005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db9d0 0x55cd508e9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16bbe00640 1 -- 172.21.15.2:0/2421738005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 msgr2=0x55cd508eaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16bbe00640 1 --2- 172.21.15.2:0/2421738005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508eaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.626+0000 7f16bbe00640 1 -- 172.21.15.2:0/2421738005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd508eb560 con 0x55cd508db090 2024-09-10T08:14:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.627+0000 7f16bbe00640 1 --2- 172.21.15.2:0/2421738005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508e9a00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f16ac09b6e0 tx=0x7f16ac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fad88e515387bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.627+0000 7f16a6000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:14:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.627+0000 7f16a6000640 1 -- 172.21.15.2:0/2421738005 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0b3020 con 0x55cd508db090 2024-09-10T08:14:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.627+0000 7f16c6071740 1 -- 172.21.15.2:0/2421738005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cd50847b80 con 0x55cd508db090 2024-09-10T08:14:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.627+0000 7f16a6000640 1 -- 172.21.15.2:0/2421738005 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f16ac0860b0 con 0x55cd508db090 2024-09-10T08:14:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.627+0000 7f16a6000640 1 -- 172.21.15.2:0/2421738005 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16ac09b0b0 con 0x55cd508db090 2024-09-10T08:14:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.627+0000 7f16c6071740 10 client.27035 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.627+0000 7f16c6071740 1 -- 172.21.15.2:0/2421738005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cd508ec870 con 0x55cd508db090 2024-09-10T08:14:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.627+0000 7f16a6000640 1 -- 172.21.15.2:0/2421738005 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cd50847b80 con 0x55cd508db090 2024-09-10T08:14:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.627+0000 7f16a6000640 1 -- 172.21.15.2:0/2421738005 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f16ac084ce0 con 0x55cd508db090 2024-09-10T08:14:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.628+0000 7f16c6071740 10 client.27035 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.628+0000 7f16c6071740 10 client.27035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.628+0000 7f16c6071740 20 client.27035 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.628+0000 7f16c6071740 1 --2- 172.21.15.2:0/2421738005 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd5095be80 0x55cd5095e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.628+0000 7f16c6071740 4 client.27035 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:14:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.628+0000 7f16c6071740 1 -- 172.21.15.2:0/2421738005 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd507c47c0 con 0x55cd5095be80 2024-09-10T08:14:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.628+0000 7f16bb400640 1 --2- 172.21.15.2:0/2421738005 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd5095be80 0x55cd5095e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.629+0000 7f16a6000640 10 client.27035 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.629+0000 7f16bb400640 1 --2- 172.21.15.2:0/2421738005 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd5095be80 0x55cd5095e270 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf13f87773edca3b server_cookie=5464ab47e8c0b7fa in_seq=0 out_seq=0 2024-09-10T08:14:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16a6000640 1 -- 172.21.15.2:0/2421738005 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd507c47c0 con 0x55cd5095be80 2024-09-10T08:14:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16a6000640 10 client.27035 handle_command_reply: tid=0 2024-09-10T08:14:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16c6071740 1 client.27035 shutdown 2024-09-10T08:14:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16c6071740 1 -- 172.21.15.2:0/2421738005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 msgr2=0x55cd508e9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16c6071740 1 --2- 172.21.15.2:0/2421738005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508e9a00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f16ac09b6e0 tx=0x7f16ac0046c0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16c6071740 1 -- 172.21.15.2:0/2421738005 shutdown_connections 2024-09-10T08:14:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16c6071740 1 --2- 172.21.15.2:0/2421738005 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd5095be80 0x55cd5095e270 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16c6071740 1 --2- 172.21.15.2:0/2421738005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd508e0a60 0x55cd508eaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16c6071740 1 --2- 172.21.15.2:0/2421738005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd508db9d0 0x55cd508e9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16a6000640 0 client.27035 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16c6071740 1 --2- 172.21.15.2:0/2421738005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd508db090 0x55cd508e9a00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16c6071740 1 -- 172.21.15.2:0/2421738005 >> 172.21.15.2:0/2421738005 conn(0x55cd508d6700 msgr2=0x55cd508e5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16c6071740 1 -- 172.21.15.2:0/2421738005 shutdown_connections 2024-09-10T08:14:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16c6071740 1 -- 172.21.15.2:0/2421738005 wait complete. 2024-09-10T08:14:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.644+0000 7f16c6071740 20 client.27035 trim_cache size 0 max 16384 2024-09-10T08:14:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:14:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:14:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:14:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:14:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.645+0000 7f16c3200640 1 -- 172.21.15.2:0/1335682304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16a00a2c30 msgr2=0x7f16a00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.645+0000 7f16c3200640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16a00a2c30 0x7f16a00a50f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f16b400a510 tx=0x7f16b400d040 comp rx=0 tx=0).stop 2024-09-10T08:14:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.645+0000 7f16c3200640 1 -- 172.21.15.2:0/1335682304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16bc103d60 msgr2=0x7f16bc19a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.645+0000 7f16c3200640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16bc103d60 0x7f16bc19a080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00d950 tx=0x7f16ac00de10 comp rx=0 tx=0).stop 2024-09-10T08:14:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.645+0000 7f16c3200640 1 -- 172.21.15.2:0/1335682304 shutdown_connections 2024-09-10T08:14:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.645+0000 7f16c3200640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16bc105910 0x7f16bc1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.645+0000 7f16c3200640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16bc104f50 0x7f16bc19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.645+0000 7f16c3200640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16a00a2c30 0x7f16a00a50f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.645+0000 7f16c3200640 1 --2- 172.21.15.2:0/1335682304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16bc103d60 0x7f16bc19a080 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.645+0000 7f16c3200640 1 -- 172.21.15.2:0/1335682304 >> 172.21.15.2:0/1335682304 conn(0x7f16bc0ff4e0 msgr2=0x7f16bc1081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.645+0000 7f16c3200640 1 -- 172.21.15.2:0/1335682304 shutdown_connections 2024-09-10T08:14:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:12.645+0000 7f16c3200640 1 -- 172.21.15.2:0/1335682304 wait complete. 2024-09-10T08:14:12.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.690+0000 7fc143c00640 1 -- 172.21.15.2:0/309885413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc13c112a50 msgr2=0x7fc13c114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.690+0000 7fc143c00640 1 --2- 172.21.15.2:0/309885413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc13c112a50 0x7fc13c114e40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc134009ca0 tx=0x7fc13402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.690+0000 7fc143c00640 1 -- 172.21.15.2:0/309885413 shutdown_connections 2024-09-10T08:14:12.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.690+0000 7fc143c00640 1 --2- 172.21.15.2:0/309885413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc13c112a50 0x7fc13c114e40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.690+0000 7fc143c00640 1 --2- 172.21.15.2:0/309885413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc13c077040 0x7fc13c112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.690+0000 7fc143c00640 1 --2- 172.21.15.2:0/309885413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc13c076720 0x7fc13c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.690+0000 7fc143c00640 1 -- 172.21.15.2:0/309885413 >> 172.21.15.2:0/309885413 conn(0x7fc13c0ffa50 msgr2=0x7fc13c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:12.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.690+0000 7fc143c00640 1 -- 172.21.15.2:0/309885413 shutdown_connections 2024-09-10T08:14:12.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.690+0000 7fc143c00640 1 -- 172.21.15.2:0/309885413 wait complete. 2024-09-10T08:14:12.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.691+0000 7fc143c00640 1 Processor -- start 2024-09-10T08:14:12.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.691+0000 7fc143c00640 1 -- start start 2024-09-10T08:14:12.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc143c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc13c076720 0x7fc13c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc143c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc13c077040 0x7fc13c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc143c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc13c112a50 0x7fc13c1a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc143c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c118600 con 0x7fc13c076720 2024-09-10T08:14:12.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc143c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c118480 con 0x7fc13c077040 2024-09-10T08:14:12.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc143c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c118780 con 0x7fc13c112a50 2024-09-10T08:14:12.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc141200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc13c076720 0x7fc13c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc141200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc13c076720 0x7fc13c19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45052/0 (socket says 172.21.15.2:45052) 2024-09-10T08:14:12.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc141200640 1 -- 172.21.15.2:0/2017770891 learned_addr learned my addr 172.21.15.2:0/2017770891 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:12.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc133e00640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc13c077040 0x7fc13c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc141c00640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc13c112a50 0x7fc13c1a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc141200640 1 -- 172.21.15.2:0/2017770891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc13c112a50 msgr2=0x7fc13c1a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc141200640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc13c112a50 0x7fc13c1a3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc141200640 1 -- 172.21.15.2:0/2017770891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc13c077040 msgr2=0x7fc13c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc141200640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc13c077040 0x7fc13c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.692+0000 7fc141200640 1 -- 172.21.15.2:0/2017770891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13c1a3b90 con 0x7fc13c076720 2024-09-10T08:14:12.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.693+0000 7fc141200640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc13c076720 0x7fc13c19eb60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00e9f0 tx=0x7fc12c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:12.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.693+0000 7fc13a000640 1 -- 172.21.15.2:0/2017770891 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c00cd80 con 0x7fc13c076720 2024-09-10T08:14:12.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.693+0000 7fc13a000640 1 -- 172.21.15.2:0/2017770891 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc12c010910 con 0x7fc13c076720 2024-09-10T08:14:12.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.693+0000 7fc13a000640 1 -- 172.21.15.2:0/2017770891 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c018990 con 0x7fc13c076720 2024-09-10T08:14:12.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.693+0000 7fc143c00640 1 -- 172.21.15.2:0/2017770891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13c1a3e80 con 0x7fc13c076720 2024-09-10T08:14:12.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.693+0000 7fc143c00640 1 -- 172.21.15.2:0/2017770891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc13c1ab710 con 0x7fc13c076720 2024-09-10T08:14:12.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.696+0000 7fc146671740 1 Processor -- start 2024-09-10T08:14:12.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.696+0000 7fc13a000640 1 -- 172.21.15.2:0/2017770891 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc12c010ac0 con 0x7fc13c076720 2024-09-10T08:14:12.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.696+0000 7fc146671740 1 -- start start 2024-09-10T08:14:12.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.697+0000 7fc146671740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562927396b90 0x562927396f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.697+0000 7fc146671740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273974b0 0x5629273a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.697+0000 7fc146671740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273a1d70 0x5629273a4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.697+0000 7fc146671740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5629272fd3e0 con 0x5629273a1d70 2024-09-10T08:14:12.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.697+0000 7fc146671740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5629272a4d80 con 0x562927396b90 2024-09-10T08:14:12.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.697+0000 7fc146671740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5629272f93a0 con 0x5629273974b0 2024-09-10T08:14:12.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.697+0000 7fc141c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562927396b90 0x562927396f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.697+0000 7fc141c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562927396b90 0x562927396f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42274/0 (socket says 172.21.15.2:42274) 2024-09-10T08:14:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.697+0000 7fc141c00640 1 -- 172.21.15.2:0/4278088046 learned_addr learned my addr 172.21.15.2:0/4278088046 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.697+0000 7fc141200640 1 --2- 172.21.15.2:0/4278088046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273974b0 0x5629273a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.697+0000 7fc133e00640 1 --2- 172.21.15.2:0/4278088046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273a1d70 0x5629273a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.698+0000 7fc141200640 1 -- 172.21.15.2:0/4278088046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562927396b90 msgr2=0x562927396f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.698+0000 7fc141200640 1 --2- 172.21.15.2:0/4278088046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562927396b90 0x562927396f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.698+0000 7fc141200640 1 -- 172.21.15.2:0/4278088046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273a1d70 msgr2=0x5629273a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.698+0000 7fc141200640 1 --2- 172.21.15.2:0/4278088046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273a1d70 0x5629273a4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.698+0000 7fc141200640 1 -- 172.21.15.2:0/4278088046 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629272c3ae0 con 0x5629273974b0 2024-09-10T08:14:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.698+0000 7fc141200640 1 --2- 172.21.15.2:0/4278088046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273974b0 0x5629273a1830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0cce00 tx=0x7fc12c0d7670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565a97fed1f8cbe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.698+0000 7fc13a000640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc1200a2c00 0x7fc1200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.698+0000 7fc13a000640 1 -- 172.21.15.2:0/2017770891 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc12c0cf7b0 con 0x7fc13c076720 2024-09-10T08:14:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.698+0000 7fc132000640 1 -- 172.21.15.2:0/4278088046 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c0de070 con 0x5629273974b0 2024-09-10T08:14:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.699+0000 7fc141c00640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc1200a2c00 0x7fc1200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.699+0000 7fc132000640 1 -- 172.21.15.2:0/4278088046 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc12c09c2c0 con 0x5629273974b0 2024-09-10T08:14:12.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.699+0000 7fc132000640 1 -- 172.21.15.2:0/4278088046 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c09b700 con 0x5629273974b0 2024-09-10T08:14:12.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.699+0000 7fc146671740 1 -- 172.21.15.2:0/4278088046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273974b0 msgr2=0x5629273a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.699+0000 7fc146671740 1 --2- 172.21.15.2:0/4278088046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273974b0 0x5629273a1830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0cce00 tx=0x7fc12c0d7670 comp rx=0 tx=0).stop 2024-09-10T08:14:12.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.699+0000 7fc141c00640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc1200a2c00 0x7fc1200a50c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc13402f540 tx=0x7fc13402f3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:12.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.700+0000 7fc146671740 1 -- 172.21.15.2:0/4278088046 shutdown_connections 2024-09-10T08:14:12.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.700+0000 7fc146671740 1 --2- 172.21.15.2:0/4278088046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273a1d70 0x5629273a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.700+0000 7fc146671740 1 --2- 172.21.15.2:0/4278088046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273974b0 0x5629273a1830 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.700+0000 7fc146671740 1 --2- 172.21.15.2:0/4278088046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562927396b90 0x562927396f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.700+0000 7fc146671740 1 -- 172.21.15.2:0/4278088046 >> 172.21.15.2:0/4278088046 conn(0x562927393120 msgr2=0x562927393500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:12.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.700+0000 7fc146671740 1 -- 172.21.15.2:0/4278088046 shutdown_connections 2024-09-10T08:14:12.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.700+0000 7fc146671740 1 -- 172.21.15.2:0/4278088046 wait complete. 2024-09-10T08:14:12.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.700+0000 7fc146671740 1 -- start start 2024-09-10T08:14:12.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.700+0000 7fc146671740 1 Processor -- start 2024-09-10T08:14:12.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.701+0000 7fc146671740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927396b90 0x56292739ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.701+0000 7fc146671740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273974b0 0x56292739f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.701+0000 7fc146671740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629273a1d70 0x56292739f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.701+0000 7fc146671740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5629273a49e0 con 0x562927396b90 2024-09-10T08:14:12.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.701+0000 7fc146671740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5629272b80f0 con 0x5629273a1d70 2024-09-10T08:14:12.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.701+0000 7fc146671740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5629273a4b60 con 0x5629273974b0 2024-09-10T08:14:12.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.701+0000 7fc146671740 10 client.0 _ll_register_callbacks cb 0x5629272a99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:12.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.701+0000 7fc141200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927396b90 0x56292739ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.701+0000 7fc141200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927396b90 0x56292739ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45068/0 (socket says 172.21.15.2:45068) 2024-09-10T08:14:12.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.701+0000 7fc141200640 1 -- 172.21.15.2:0/91204278 learned_addr learned my addr 172.21.15.2:0/91204278 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:12.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.701+0000 7fc133e00640 1 --2- 172.21.15.2:0/91204278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273974b0 0x56292739f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.701+0000 7fc141c00640 1 --2- 172.21.15.2:0/91204278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629273a1d70 0x56292739f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.702+0000 7fc141200640 1 -- 172.21.15.2:0/91204278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273974b0 msgr2=0x56292739f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.702+0000 7fc141200640 1 --2- 172.21.15.2:0/91204278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273974b0 0x56292739f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.702+0000 7fc141200640 1 -- 172.21.15.2:0/91204278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629273a1d70 msgr2=0x56292739f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.702+0000 7fc141200640 1 --2- 172.21.15.2:0/91204278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629273a1d70 0x56292739f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.702+0000 7fc141200640 1 -- 172.21.15.2:0/91204278 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56292706e3c0 con 0x562927396b90 2024-09-10T08:14:12.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.702+0000 7fc141200640 1 --2- 172.21.15.2:0/91204278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927396b90 0x56292739ec30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc12c09bb00 tx=0x7fc12c09b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bf3ceb8f76d6ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:12.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.702+0000 7fc12aa00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:12.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.702+0000 7fc12aa00640 1 -- 172.21.15.2:0/91204278 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c0de070 con 0x562927396b90 2024-09-10T08:14:12.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.702+0000 7fc12aa00640 1 -- 172.21.15.2:0/91204278 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc12c0d5730 con 0x562927396b90 2024-09-10T08:14:12.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.702+0000 7fc146671740 1 -- 172.21.15.2:0/91204278 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc12c09c2c0 con 0x562927396b90 2024-09-10T08:14:12.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.702+0000 7fc12aa00640 1 -- 172.21.15.2:0/91204278 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc12c09b520 con 0x562927396b90 2024-09-10T08:14:12.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.702+0000 7fc146671740 10 client.17010 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:12.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.702+0000 7fc146671740 1 -- 172.21.15.2:0/91204278 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629273b0690 con 0x562927396b90 2024-09-10T08:14:12.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.703+0000 7fc12aa00640 1 -- 172.21.15.2:0/91204278 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc12c0b2020 con 0x562927396b90 2024-09-10T08:14:12.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.703+0000 7fc12aa00640 1 -- 172.21.15.2:0/91204278 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc12c0cf420 con 0x562927396b90 2024-09-10T08:14:12.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.703+0000 7fc146671740 10 client.17010 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:12.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.703+0000 7fc146671740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.703+0000 7fc146671740 20 client.17010 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.703+0000 7fc146671740 1 --2- 172.21.15.2:0/91204278 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5629273b19e0 0x5629273b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.703+0000 7fc146671740 4 client.17010 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.703+0000 7fc146671740 1 -- 172.21.15.2:0/91204278 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56292730a950 con 0x5629273b19e0 2024-09-10T08:14:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.704+0000 7fc133e00640 1 --2- 172.21.15.2:0/91204278 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5629273b19e0 0x5629273b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.704+0000 7fc12aa00640 10 client.17010 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.704+0000 7fc133e00640 1 --2- 172.21.15.2:0/91204278 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5629273b19e0 0x5629273b3dd0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e66f083d6feac0 server_cookie=9e15841034cf0bf8 in_seq=0 out_seq=0 2024-09-10T08:14:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.705+0000 7fc12aa00640 1 -- 172.21.15.2:0/91204278 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56292730a950 con 0x5629273b19e0 2024-09-10T08:14:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.705+0000 7fc12aa00640 10 client.17010 handle_command_reply: tid=0 2024-09-10T08:14:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.705+0000 7fc146671740 1 client.17010 shutdown 2024-09-10T08:14:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.705+0000 7fc146671740 1 -- 172.21.15.2:0/91204278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927396b90 msgr2=0x56292739ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.705+0000 7fc146671740 1 --2- 172.21.15.2:0/91204278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927396b90 0x56292739ec30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc12c09bb00 tx=0x7fc12c09b990 comp rx=0 tx=0).stop 2024-09-10T08:14:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.705+0000 7fc146671740 1 -- 172.21.15.2:0/91204278 shutdown_connections 2024-09-10T08:14:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.705+0000 7fc146671740 1 --2- 172.21.15.2:0/91204278 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5629273b19e0 0x5629273b3dd0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.705+0000 7fc146671740 1 --2- 172.21.15.2:0/91204278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629273a1d70 0x56292739f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.705+0000 7fc146671740 1 --2- 172.21.15.2:0/91204278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273974b0 0x56292739f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.705+0000 7fc146671740 1 --2- 172.21.15.2:0/91204278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927396b90 0x56292739ec30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.705+0000 7fc146671740 1 -- 172.21.15.2:0/91204278 >> 172.21.15.2:0/91204278 conn(0x562927393120 msgr2=0x562927398e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.705+0000 7fc12aa00640 0 client.17010 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.706+0000 7fc146671740 1 -- 172.21.15.2:0/91204278 shutdown_connections 2024-09-10T08:14:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.706+0000 7fc146671740 1 -- 172.21.15.2:0/91204278 wait complete. 2024-09-10T08:14:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.706+0000 7fc146671740 20 client.17010 trim_cache size 0 max 16384 2024-09-10T08:14:12.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.724+0000 7fc146671740 1 Processor -- start 2024-09-10T08:14:12.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.724+0000 7fc146671740 1 -- start start 2024-09-10T08:14:12.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc146671740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927396b90 0x56292739f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc146671740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 0x562927398cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc146671740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629273a1d70 0x5629273b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc146671740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1b7e00 con 0x5629273974b0 2024-09-10T08:14:12.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc146671740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1b56d0 con 0x5629273a1d70 2024-09-10T08:14:12.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc146671740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1b3520 con 0x562927396b90 2024-09-10T08:14:12.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc133e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 0x562927398cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc141c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629273a1d70 0x5629273b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc133e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 0x562927398cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45070/0 (socket says 172.21.15.2:45070) 2024-09-10T08:14:12.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc133e00640 1 -- 172.21.15.2:0/660785364 learned_addr learned my addr 172.21.15.2:0/660785364 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:12.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc141200640 1 --2- 172.21.15.2:0/660785364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927396b90 0x56292739f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc133e00640 1 -- 172.21.15.2:0/660785364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927396b90 msgr2=0x56292739f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc133e00640 1 --2- 172.21.15.2:0/660785364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927396b90 0x56292739f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc133e00640 1 -- 172.21.15.2:0/660785364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629273a1d70 msgr2=0x5629273b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc133e00640 1 --2- 172.21.15.2:0/660785364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629273a1d70 0x5629273b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.725+0000 7fc133e00640 1 -- 172.21.15.2:0/660785364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562927396910 con 0x5629273974b0 2024-09-10T08:14:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.726+0000 7fc133e00640 1 --2- 172.21.15.2:0/660785364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 0x562927398cf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc13c1034b0 tx=0x7fc124005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dda6d26a6ff4505 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.726+0000 7fc131600640 1 -- 172.21.15.2:0/660785364 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124012110 con 0x5629273974b0 2024-09-10T08:14:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.726+0000 7fc131600640 1 -- 172.21.15.2:0/660785364 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1240035f0 con 0x5629273974b0 2024-09-10T08:14:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.726+0000 7fc131600640 1 -- 172.21.15.2:0/660785364 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1240122b0 con 0x5629273974b0 2024-09-10T08:14:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.727+0000 7fc146671740 1 -- 172.21.15.2:0/660785364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 msgr2=0x562927398cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.727+0000 7fc146671740 1 --2- 172.21.15.2:0/660785364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 0x562927398cf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc13c1034b0 tx=0x7fc124005f90 comp rx=0 tx=0).stop 2024-09-10T08:14:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.727+0000 7fc146671740 1 -- 172.21.15.2:0/660785364 shutdown_connections 2024-09-10T08:14:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.727+0000 7fc146671740 1 --2- 172.21.15.2:0/660785364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629273a1d70 0x5629273b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.727+0000 7fc146671740 1 --2- 172.21.15.2:0/660785364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 0x562927398cf0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.727+0000 7fc146671740 1 --2- 172.21.15.2:0/660785364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927396b90 0x56292739f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.727+0000 7fc146671740 1 -- 172.21.15.2:0/660785364 >> 172.21.15.2:0/660785364 conn(0x562927393120 msgr2=0x5629273985e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.727+0000 7fc146671740 1 -- 172.21.15.2:0/660785364 shutdown_connections 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.727+0000 7fc146671740 1 -- 172.21.15.2:0/660785364 wait complete. 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.727+0000 7fc146671740 1 -- start start 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.727+0000 7fc146671740 1 Processor -- start 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.728+0000 7fc146671740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562927396b90 0x5629273b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.728+0000 7fc146671740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 0x5629273b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.728+0000 7fc146671740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273a1d70 0x5629274142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.728+0000 7fc146671740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1b3b90 con 0x5629273974b0 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.728+0000 7fc146671740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1b5000 con 0x562927396b90 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.728+0000 7fc146671740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1b4260 con 0x5629273a1d70 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.728+0000 7fc146671740 10 client.0 _ll_register_callbacks cb 0x7fc12c0d5a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.728+0000 7fc133e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 0x5629273b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.728+0000 7fc133e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 0x5629273b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45074/0 (socket says 172.21.15.2:45074) 2024-09-10T08:14:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.728+0000 7fc133e00640 1 -- 172.21.15.2:0/667715138 learned_addr learned my addr 172.21.15.2:0/667715138 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.728+0000 7fc141200640 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562927396b90 0x5629273b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.728+0000 7fc141c00640 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273a1d70 0x5629274142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.729+0000 7fc133e00640 1 -- 172.21.15.2:0/667715138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273a1d70 msgr2=0x5629274142e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.729+0000 7fc133e00640 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273a1d70 0x5629274142e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.729+0000 7fc133e00640 1 -- 172.21.15.2:0/667715138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562927396b90 msgr2=0x5629273b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.729+0000 7fc133e00640 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562927396b90 0x5629273b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.729+0000 7fc133e00640 1 -- 172.21.15.2:0/667715138 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562927414980 con 0x5629273974b0 2024-09-10T08:14:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.729+0000 7fc141200640 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562927396b90 0x5629273b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.729+0000 7fc141c00640 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273a1d70 0x5629274142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:14:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.729+0000 7fc133e00640 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 0x5629273b4650 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc124003f40 tx=0x7fc124005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7c87307f30fd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.729+0000 7fc12aa00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.729+0000 7fc12aa00640 1 -- 172.21.15.2:0/667715138 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124012110 con 0x5629273974b0 2024-09-10T08:14:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.729+0000 7fc12aa00640 1 -- 172.21.15.2:0/667715138 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc124004050 con 0x5629273974b0 2024-09-10T08:14:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.729+0000 7fc146671740 1 -- 172.21.15.2:0/667715138 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1240035f0 con 0x5629273974b0 2024-09-10T08:14:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.730+0000 7fc12aa00640 1 -- 172.21.15.2:0/667715138 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc1240122b0 con 0x5629273974b0 2024-09-10T08:14:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.730+0000 7fc146671740 10 client.17016 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.730+0000 7fc146671740 1 -- 172.21.15.2:0/667715138 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56292741bd70 con 0x5629273974b0 2024-09-10T08:14:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.730+0000 7fc12aa00640 1 -- 172.21.15.2:0/667715138 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1240035f0 con 0x5629273974b0 2024-09-10T08:14:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.730+0000 7fc12aa00640 1 -- 172.21.15.2:0/667715138 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc124010990 con 0x5629273974b0 2024-09-10T08:14:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.730+0000 7fc146671740 10 client.17016 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.730+0000 7fc146671740 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.730+0000 7fc146671740 20 client.17016 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.730+0000 7fc146671740 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56292741c4c0 0x56292741e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.730+0000 7fc146671740 4 client.17016 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:14:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.730+0000 7fc146671740 1 -- 172.21.15.2:0/667715138 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562927304e60 con 0x56292741c4c0 2024-09-10T08:14:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.731+0000 7fc141200640 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56292741c4c0 0x56292741e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.731+0000 7fc141200640 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56292741c4c0 0x56292741e8e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd8b4bbc5223bad server_cookie=d6b96038edd66838 in_seq=0 out_seq=0 2024-09-10T08:14:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.731+0000 7fc12aa00640 10 client.17016 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.731+0000 7fc12aa00640 1 -- 172.21.15.2:0/667715138 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562927304e60 con 0x56292741c4c0 2024-09-10T08:14:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.731+0000 7fc12aa00640 10 client.17016 handle_command_reply: tid=0 2024-09-10T08:14:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.731+0000 7fc146671740 1 client.17016 shutdown 2024-09-10T08:14:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.732+0000 7fc146671740 1 -- 172.21.15.2:0/667715138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 msgr2=0x5629273b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.732+0000 7fc146671740 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 0x5629273b4650 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc124003f40 tx=0x7fc124005c70 comp rx=0 tx=0).stop 2024-09-10T08:14:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.732+0000 7fc146671740 1 -- 172.21.15.2:0/667715138 shutdown_connections 2024-09-10T08:14:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.732+0000 7fc146671740 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56292741c4c0 0x56292741e8e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.732+0000 7fc146671740 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629273a1d70 0x5629274142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.732+0000 7fc146671740 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629273974b0 0x5629273b4650 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.732+0000 7fc146671740 1 --2- 172.21.15.2:0/667715138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562927396b90 0x5629273b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.732+0000 7fc12aa00640 0 client.17016 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.732+0000 7fc146671740 1 -- 172.21.15.2:0/667715138 >> 172.21.15.2:0/667715138 conn(0x562927393120 msgr2=0x5629273ad600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.732+0000 7fc146671740 1 -- 172.21.15.2:0/667715138 shutdown_connections 2024-09-10T08:14:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.732+0000 7fc146671740 1 -- 172.21.15.2:0/667715138 wait complete. 2024-09-10T08:14:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.732+0000 7fc146671740 20 client.17016 trim_cache size 0 max 16384 2024-09-10T08:14:12.736 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:14:12.736 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:14:12.736 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:14:12.736 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:14:12.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.733+0000 7fc143c00640 1 -- 172.21.15.2:0/2017770891 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc1200a2c00 msgr2=0x7fc1200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.733+0000 7fc143c00640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc1200a2c00 0x7fc1200a50c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc13402f540 tx=0x7fc13402f3b0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.733+0000 7fc143c00640 1 -- 172.21.15.2:0/2017770891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc13c076720 msgr2=0x7fc13c19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.733+0000 7fc143c00640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc13c076720 0x7fc13c19eb60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00e9f0 tx=0x7fc12c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.733+0000 7fc143c00640 1 -- 172.21.15.2:0/2017770891 shutdown_connections 2024-09-10T08:14:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.733+0000 7fc143c00640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc1200a2c00 0x7fc1200a50c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.733+0000 7fc143c00640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc13c112a50 0x7fc13c1a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.733+0000 7fc143c00640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc13c077040 0x7fc13c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.733+0000 7fc143c00640 1 --2- 172.21.15.2:0/2017770891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc13c076720 0x7fc13c19eb60 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.733+0000 7fc143c00640 1 -- 172.21.15.2:0/2017770891 >> 172.21.15.2:0/2017770891 conn(0x7fc13c0ffa50 msgr2=0x7fc13c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.733+0000 7fc143c00640 1 -- 172.21.15.2:0/2017770891 shutdown_connections 2024-09-10T08:14:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:12.733+0000 7fc143c00640 1 -- 172.21.15.2:0/2017770891 wait complete. 2024-09-10T08:14:13.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:12 smithi002 ceph-mon[30563]: pgmap v569: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:14:13.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:12 smithi155 ceph-mon[35405]: pgmap v569: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:14:13.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:12 smithi107 ceph-mon[35048]: pgmap v569: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:14:13.650 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:14:13.650 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:14:14.027 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:14.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.642+0000 7f9cd9800640 1 -- 172.21.15.2:0/3194969052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cd40745c0 msgr2=0x7f9cd40749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:14.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.642+0000 7f9cd9800640 1 --2- 172.21.15.2:0/3194969052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cd40745c0 0x7f9cd40749a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009900 tx=0x7f9cc402a480 comp rx=0 tx=0).stop 2024-09-10T08:14:14.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.642+0000 7f9cd9800640 1 -- 172.21.15.2:0/3194969052 shutdown_connections 2024-09-10T08:14:14.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.642+0000 7f9cd9800640 1 --2- 172.21.15.2:0/3194969052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd410e410 0x7f9cd4110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:14.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.642+0000 7f9cd9800640 1 --2- 172.21.15.2:0/3194969052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cd4074ee0 0x7f9cd410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:14.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.642+0000 7f9cd9800640 1 --2- 172.21.15.2:0/3194969052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cd40745c0 0x7f9cd40749a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:14.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.642+0000 7f9cd9800640 1 -- 172.21.15.2:0/3194969052 >> 172.21.15.2:0/3194969052 conn(0x7f9cd40fd890 msgr2=0x7f9cd40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:14.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.642+0000 7f9cd9800640 1 -- 172.21.15.2:0/3194969052 shutdown_connections 2024-09-10T08:14:14.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.642+0000 7f9cd9800640 1 -- 172.21.15.2:0/3194969052 wait complete. 2024-09-10T08:14:14.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.644+0000 7f9cd9800640 1 Processor -- start 2024-09-10T08:14:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.644+0000 7f9cd9800640 1 -- start start 2024-09-10T08:14:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.644+0000 7f9cd9800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd40745c0 0x7f9cd419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.644+0000 7f9cd9800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cd4074ee0 0x7f9cd419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.644+0000 7f9cd9800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cd410e410 0x7f9cd419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.644+0000 7f9cd9800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd41141d0 con 0x7f9cd4074ee0 2024-09-10T08:14:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.644+0000 7f9cd9800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd4114050 con 0x7f9cd40745c0 2024-09-10T08:14:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.644+0000 7f9cd9800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd4114350 con 0x7f9cd410e410 2024-09-10T08:14:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.645+0000 7f9cd2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd40745c0 0x7f9cd419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.645+0000 7f9cd2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd40745c0 0x7f9cd419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42320/0 (socket says 172.21.15.2:42320) 2024-09-10T08:14:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.645+0000 7f9cd2a00640 1 -- 172.21.15.2:0/406474270 learned_addr learned my addr 172.21.15.2:0/406474270 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.645+0000 7f9cd2000640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cd4074ee0 0x7f9cd419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.645+0000 7f9cd3400640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cd410e410 0x7f9cd419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:14.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.645+0000 7f9cd2a00640 1 -- 172.21.15.2:0/406474270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cd410e410 msgr2=0x7f9cd419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:14.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.645+0000 7f9cd2a00640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cd410e410 0x7f9cd419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:14.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.645+0000 7f9cd2a00640 1 -- 172.21.15.2:0/406474270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cd4074ee0 msgr2=0x7f9cd419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:14.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.645+0000 7f9cd2a00640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cd4074ee0 0x7f9cd419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:14.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.645+0000 7f9cd2a00640 1 -- 172.21.15.2:0/406474270 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd419f660 con 0x7f9cd40745c0 2024-09-10T08:14:14.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.646+0000 7f9cd2a00640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd40745c0 0x7f9cd419a6b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4004770 tx=0x7f9cc4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:14.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.646+0000 7f9ccb400640 1 -- 172.21.15.2:0/406474270 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc403f070 con 0x7f9cd40745c0 2024-09-10T08:14:14.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.646+0000 7f9cd9800640 1 -- 172.21.15.2:0/406474270 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd419f8f0 con 0x7f9cd40745c0 2024-09-10T08:14:14.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.646+0000 7f9ccb400640 1 -- 172.21.15.2:0/406474270 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc4031ca0 con 0x7f9cd40745c0 2024-09-10T08:14:14.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.646+0000 7f9ccb400640 1 -- 172.21.15.2:0/406474270 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4035030 con 0x7f9cd40745c0 2024-09-10T08:14:14.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.646+0000 7f9cd9800640 1 -- 172.21.15.2:0/406474270 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd41a7230 con 0x7f9cd40745c0 2024-09-10T08:14:14.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.648+0000 7f9cd9800640 1 -- 172.21.15.2:0/406474270 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd40749a0 con 0x7f9cd40745c0 2024-09-10T08:14:14.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.649+0000 7f9ccb400640 1 -- 172.21.15.2:0/406474270 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9cc4042070 con 0x7f9cd40745c0 2024-09-10T08:14:14.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.651+0000 7f9ccb400640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9cb80a2b80 0x7f9cb80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:14.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.651+0000 7f9ccb400640 1 -- 172.21.15.2:0/406474270 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9cc40efa70 con 0x7f9cd40745c0 2024-09-10T08:14:14.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.651+0000 7f9cd2000640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9cb80a2b80 0x7f9cb80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:14.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.652+0000 7f9cd2000640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9cb80a2b80 0x7f9cb80a5040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40ffe20 tx=0x7f9cc0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:14.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.655+0000 7f9ccb400640 1 -- 172.21.15.2:0/406474270 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9cc40bbff0 con 0x7f9cd40745c0 2024-09-10T08:14:14.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.895+0000 7f9cd9800640 1 -- 172.21.15.2:0/406474270 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cd4109590 con 0x7f9cd40745c0 2024-09-10T08:14:14.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.897+0000 7f9ccb400640 1 -- 172.21.15.2:0/406474270 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f9cc40c02f0 con 0x7f9cd40745c0 2024-09-10T08:14:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:14:14.897 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:14:14.897 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:14:14.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.899+0000 7f9cd9800640 1 -- 172.21.15.2:0/406474270 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9cb80a2b80 msgr2=0x7f9cb80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:14.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.899+0000 7f9cd9800640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9cb80a2b80 0x7f9cb80a5040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40ffe20 tx=0x7f9cc0005f90 comp rx=0 tx=0).stop 2024-09-10T08:14:14.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.899+0000 7f9cd9800640 1 -- 172.21.15.2:0/406474270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd40745c0 msgr2=0x7f9cd419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:14.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.899+0000 7f9cd9800640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd40745c0 0x7f9cd419a6b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4004770 tx=0x7f9cc4003c10 comp rx=0 tx=0).stop 2024-09-10T08:14:14.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.899+0000 7f9cd9800640 1 -- 172.21.15.2:0/406474270 shutdown_connections 2024-09-10T08:14:14.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.899+0000 7f9cd9800640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9cb80a2b80 0x7f9cb80a5040 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:14.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.899+0000 7f9cd9800640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cd410e410 0x7f9cd419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:14.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.899+0000 7f9cd9800640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cd4074ee0 0x7f9cd419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:14.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.899+0000 7f9cd9800640 1 --2- 172.21.15.2:0/406474270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd40745c0 0x7f9cd419a6b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:14.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.899+0000 7f9cd9800640 1 -- 172.21.15.2:0/406474270 >> 172.21.15.2:0/406474270 conn(0x7f9cd40fd890 msgr2=0x7f9cd40fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:14.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.900+0000 7f9cd9800640 1 -- 172.21.15.2:0/406474270 shutdown_connections 2024-09-10T08:14:14.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:14.900+0000 7f9cd9800640 1 -- 172.21.15.2:0/406474270 wait complete. 2024-09-10T08:14:15.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:14 smithi155 ceph-mon[35405]: pgmap v570: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:14:15.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:14 smithi107 ceph-mon[35048]: pgmap v570: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:14:15.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:14 smithi002 ceph-mon[30563]: pgmap v570: 129 pgs: 129 active+clean; 3.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:14:15.780 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:14:16.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:15 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/406474270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:14:16.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:15 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/406474270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:14:16.164 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:16.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:15 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/406474270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:14:16.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.777+0000 7f52bbe00640 1 -- 172.21.15.2:0/3431283957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52bc1047e0 msgr2=0x7f52bc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:16.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.777+0000 7f52bbe00640 1 --2- 172.21.15.2:0/3431283957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52bc1047e0 0x7f52bc108830 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009900 tx=0x7f52a402a410 comp rx=0 tx=0).stop 2024-09-10T08:14:16.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.777+0000 7f52bbe00640 1 -- 172.21.15.2:0/3431283957 shutdown_connections 2024-09-10T08:14:16.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.777+0000 7f52bbe00640 1 --2- 172.21.15.2:0/3431283957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc108f60 0x7f52bc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.777+0000 7f52bbe00640 1 --2- 172.21.15.2:0/3431283957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52bc1047e0 0x7f52bc108830 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.777+0000 7f52bbe00640 1 --2- 172.21.15.2:0/3431283957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52bc103e30 0x7f52bc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.777+0000 7f52bbe00640 1 -- 172.21.15.2:0/3431283957 >> 172.21.15.2:0/3431283957 conn(0x7f52bc0ff910 msgr2=0x7f52bc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:16.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.777+0000 7f52bbe00640 1 -- 172.21.15.2:0/3431283957 shutdown_connections 2024-09-10T08:14:16.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.777+0000 7f52bbe00640 1 -- 172.21.15.2:0/3431283957 wait complete. 2024-09-10T08:14:16.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.778+0000 7f52bbe00640 1 Processor -- start 2024-09-10T08:14:16.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.778+0000 7f52bbe00640 1 -- start start 2024-09-10T08:14:16.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.779+0000 7f52bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc103e30 0x7f52bc19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.779+0000 7f52bbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52bc1047e0 0x7f52bc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.779+0000 7f52bbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52bc108f60 0x7f52bc19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.779+0000 7f52bbe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc114010 con 0x7f52bc108f60 2024-09-10T08:14:16.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.779+0000 7f52bbe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc113e90 con 0x7f52bc103e30 2024-09-10T08:14:16.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.779+0000 7f52bbe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc114190 con 0x7f52bc1047e0 2024-09-10T08:14:16.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.779+0000 7f52bb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52bc108f60 0x7f52bc19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.779+0000 7f52bb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52bc108f60 0x7f52bc19ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45114/0 (socket says 172.21.15.2:45114) 2024-09-10T08:14:16.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.779+0000 7f52bb400640 1 -- 172.21.15.2:0/1250686499 learned_addr learned my addr 172.21.15.2:0/1250686499 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:16.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.779+0000 7f52baa00640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc103e30 0x7f52bc19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.779+0000 7f52b2000640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52bc1047e0 0x7f52bc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.779+0000 7f52bb400640 1 -- 172.21.15.2:0/1250686499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52bc1047e0 msgr2=0x7f52bc19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:16.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.780+0000 7f52bb400640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52bc1047e0 0x7f52bc19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.780+0000 7f52bb400640 1 -- 172.21.15.2:0/1250686499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc103e30 msgr2=0x7f52bc19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:16.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.780+0000 7f52bb400640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc103e30 0x7f52bc19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.780+0000 7f52bb400640 1 -- 172.21.15.2:0/1250686499 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc19f4c0 con 0x7f52bc108f60 2024-09-10T08:14:16.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.780+0000 7f52baa00640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc103e30 0x7f52bc19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:14:16.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.780+0000 7f52bb400640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52bc108f60 0x7f52bc19ede0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f52b400d950 tx=0x7f52b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:16.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.780+0000 7f52b3e00640 1 -- 172.21.15.2:0/1250686499 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4017030 con 0x7f52bc108f60 2024-09-10T08:14:16.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.780+0000 7f52bbe00640 1 -- 172.21.15.2:0/1250686499 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52bc19f7b0 con 0x7f52bc108f60 2024-09-10T08:14:16.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.780+0000 7f52b3e00640 1 -- 172.21.15.2:0/1250686499 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b4004020 con 0x7f52bc108f60 2024-09-10T08:14:16.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.780+0000 7f52b3e00640 1 -- 172.21.15.2:0/1250686499 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4003c00 con 0x7f52bc108f60 2024-09-10T08:14:16.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.780+0000 7f52bbe00640 1 -- 172.21.15.2:0/1250686499 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52bc1a7090 con 0x7f52bc108f60 2024-09-10T08:14:16.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.784+0000 7f52c4665740 1 Processor -- start 2024-09-10T08:14:16.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.785+0000 7f52b3e00640 1 -- 172.21.15.2:0/1250686499 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f52b401a070 con 0x7f52bc108f60 2024-09-10T08:14:16.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.785+0000 7f52c4665740 1 -- start start 2024-09-10T08:14:16.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.785+0000 7f52c4665740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa8e40 0x557e34aa9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.785+0000 7f52c4665740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa9760 0x557e34ab3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.785+0000 7f52c4665740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34ab4020 0x557e34ab6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.785+0000 7f52c4665740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557e34a0f3e0 con 0x557e34ab4020 2024-09-10T08:14:16.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.785+0000 7f52c4665740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557e349b6d80 con 0x557e34aa8e40 2024-09-10T08:14:16.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.785+0000 7f52c4665740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557e34a0b3a0 con 0x557e34aa9760 2024-09-10T08:14:16.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.785+0000 7f52b2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34ab4020 0x557e34ab6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.785+0000 7f52b2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34ab4020 0x557e34ab6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45130/0 (socket says 172.21.15.2:45130) 2024-09-10T08:14:16.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.786+0000 7f52b2000640 1 -- 172.21.15.2:0/3571428409 learned_addr learned my addr 172.21.15.2:0/3571428409 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:16.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.786+0000 7f52baa00640 1 --2- 172.21.15.2:0/3571428409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa9760 0x557e34ab3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.786+0000 7f52b2000640 1 --2- 172.21.15.2:0/3571428409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa8e40 0x557e34aa9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.786+0000 7f52b2000640 1 -- 172.21.15.2:0/3571428409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa9760 msgr2=0x557e34ab3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:16.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.786+0000 7f52b2000640 1 --2- 172.21.15.2:0/3571428409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa9760 0x557e34ab3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.786+0000 7f52b2000640 1 -- 172.21.15.2:0/3571428409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa8e40 msgr2=0x557e34aa9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:16.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.786+0000 7f52b2000640 1 --2- 172.21.15.2:0/3571428409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa8e40 0x557e34aa9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.786+0000 7f52b2000640 1 -- 172.21.15.2:0/3571428409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e349d5ae0 con 0x557e34ab4020 2024-09-10T08:14:16.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.786+0000 7f52b3e00640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52a00a2c00 0x7f52a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.786+0000 7f52b2000640 1 --2- 172.21.15.2:0/3571428409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34ab4020 0x557e34ab6410 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f52a4005ab0 tx=0x7f52a40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da99ce5ee513b52e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:16.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.786+0000 7f52b3e00640 1 -- 172.21.15.2:0/1250686499 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f52b40ced20 con 0x7f52bc108f60 2024-09-10T08:14:16.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.787+0000 7f52abe00640 1 -- 172.21.15.2:0/3571428409 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4035040 con 0x557e34ab4020 2024-09-10T08:14:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.787+0000 7f52abe00640 1 -- 172.21.15.2:0/3571428409 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a4031c90 con 0x557e34ab4020 2024-09-10T08:14:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.787+0000 7f52abe00640 1 -- 172.21.15.2:0/3571428409 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4003ec0 con 0x557e34ab4020 2024-09-10T08:14:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.787+0000 7f52baa00640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52a00a2c00 0x7f52a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.787+0000 7f52c4665740 1 -- 172.21.15.2:0/3571428409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34ab4020 msgr2=0x557e34ab6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.787+0000 7f52c4665740 1 --2- 172.21.15.2:0/3571428409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34ab4020 0x557e34ab6410 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f52a4005ab0 tx=0x7f52a40056d0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.787+0000 7f52baa00640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52a00a2c00 0x7f52a00a50c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x557e34aa6230 tx=0x7f52ac009880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.787+0000 7f52c4665740 1 -- 172.21.15.2:0/3571428409 shutdown_connections 2024-09-10T08:14:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.787+0000 7f52c4665740 1 --2- 172.21.15.2:0/3571428409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34ab4020 0x557e34ab6410 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.787+0000 7f52c4665740 1 --2- 172.21.15.2:0/3571428409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa9760 0x557e34ab3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.787+0000 7f52c4665740 1 --2- 172.21.15.2:0/3571428409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa8e40 0x557e34aa9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.787+0000 7f52c4665740 1 -- 172.21.15.2:0/3571428409 >> 172.21.15.2:0/3571428409 conn(0x557e34aa53d0 msgr2=0x557e34aa57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.788+0000 7f52c4665740 1 -- 172.21.15.2:0/3571428409 shutdown_connections 2024-09-10T08:14:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.788+0000 7f52c4665740 1 -- 172.21.15.2:0/3571428409 wait complete. 2024-09-10T08:14:16.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.788+0000 7f52c4665740 1 -- start start 2024-09-10T08:14:16.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.788+0000 7f52c4665740 1 Processor -- start 2024-09-10T08:14:16.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.789+0000 7f52c4665740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa8e40 0x557e34ab0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.789+0000 7f52c4665740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 0x557e34ab14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.789+0000 7f52c4665740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34ab4020 0x557e34ab19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.789+0000 7f52c4665740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557e34ab6bf0 con 0x557e34aa9760 2024-09-10T08:14:16.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.789+0000 7f52c4665740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557e349ca0f0 con 0x557e34ab4020 2024-09-10T08:14:16.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.789+0000 7f52c4665740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557e34ab6d70 con 0x557e34aa8e40 2024-09-10T08:14:16.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.789+0000 7f52c4665740 10 client.0 _ll_register_callbacks cb 0x557e349b1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:16.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.789+0000 7f52b2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 0x557e34ab14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.789+0000 7f52b2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 0x557e34ab14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45144/0 (socket says 172.21.15.2:45144) 2024-09-10T08:14:16.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.789+0000 7f52b2000640 1 -- 172.21.15.2:0/3337174567 learned_addr learned my addr 172.21.15.2:0/3337174567 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.789+0000 7f52bb400640 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34ab4020 0x557e34ab19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.789+0000 7f52baa00640 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa8e40 0x557e34ab0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.790+0000 7f52b2000640 1 -- 172.21.15.2:0/3337174567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa8e40 msgr2=0x557e34ab0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.790+0000 7f52b2000640 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa8e40 0x557e34ab0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.790+0000 7f52b2000640 1 -- 172.21.15.2:0/3337174567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34ab4020 msgr2=0x557e34ab19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.790+0000 7f52b2000640 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34ab4020 0x557e34ab19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.790+0000 7f52b2000640 1 -- 172.21.15.2:0/3337174567 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e347803c0 con 0x557e34aa9760 2024-09-10T08:14:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.790+0000 7f52bb400640 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34ab4020 0x557e34ab19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.790+0000 7f52baa00640 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa8e40 0x557e34ab0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.790+0000 7f52b2000640 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 0x557e34ab14b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f52a40096c0 tx=0x7f52a4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b6ca192e44e6a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.790+0000 7f52a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.790+0000 7f52a8c00640 1 -- 172.21.15.2:0/3337174567 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4035070 con 0x557e34aa9760 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.790+0000 7f52c4665740 1 -- 172.21.15.2:0/3337174567 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a4031c90 con 0x557e34aa9760 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.790+0000 7f52a8c00640 1 -- 172.21.15.2:0/3337174567 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f52a40315b0 con 0x557e34aa9760 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.790+0000 7f52a8c00640 1 -- 172.21.15.2:0/3337174567 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f52a405a050 con 0x557e34aa9760 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.791+0000 7f52c4665740 10 client.17034 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.791+0000 7f52c4665740 1 -- 172.21.15.2:0/3337174567 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557e34ac2c90 con 0x557e34aa9760 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.791+0000 7f52a8c00640 1 -- 172.21.15.2:0/3337174567 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a4031c90 con 0x557e34aa9760 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.791+0000 7f52a8c00640 1 -- 172.21.15.2:0/3337174567 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f52a403a040 con 0x557e34aa9760 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.791+0000 7f52c4665740 10 client.17034 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.791+0000 7f52c4665740 10 client.17034 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.791+0000 7f52c4665740 20 client.17034 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.791+0000 7f52c4665740 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557e34ac3fe0 0x557e34ac63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.791+0000 7f52c4665740 4 client.17034 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.791+0000 7f52c4665740 1 -- 172.21.15.2:0/3337174567 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e34a1c950 con 0x557e34ac3fe0 2024-09-10T08:14:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.792+0000 7f52baa00640 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557e34ac3fe0 0x557e34ac63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.792+0000 7f52baa00640 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557e34ac3fe0 0x557e34ac63d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6747704769510db server_cookie=5ad647d2e8b31fc4 in_seq=0 out_seq=0 2024-09-10T08:14:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.792+0000 7f52a8c00640 10 client.17034 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:14:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.793+0000 7f52a8c00640 1 -- 172.21.15.2:0/3337174567 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e34a1c950 con 0x557e34ac3fe0 2024-09-10T08:14:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.793+0000 7f52a8c00640 10 client.17034 handle_command_reply: tid=0 2024-09-10T08:14:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.793+0000 7f52c4665740 1 client.17034 shutdown 2024-09-10T08:14:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.793+0000 7f52c4665740 1 -- 172.21.15.2:0/3337174567 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 msgr2=0x557e34ab14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.793+0000 7f52c4665740 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 0x557e34ab14b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f52a40096c0 tx=0x7f52a4005c90 comp rx=0 tx=0).stop 2024-09-10T08:14:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.793+0000 7f52c4665740 1 -- 172.21.15.2:0/3337174567 shutdown_connections 2024-09-10T08:14:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.793+0000 7f52c4665740 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34ab4020 0x557e34ab19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.793+0000 7f52c4665740 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557e34ac3fe0 0x557e34ac63d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.793+0000 7f52c4665740 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 0x557e34ab14b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.793+0000 7f52c4665740 1 --2- 172.21.15.2:0/3337174567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa8e40 0x557e34ab0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.793+0000 7f52c4665740 1 -- 172.21.15.2:0/3337174567 >> 172.21.15.2:0/3337174567 conn(0x557e34aa53d0 msgr2=0x557e34aab0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:16.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.793+0000 7f52a8c00640 0 client.17034 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:14:16.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.794+0000 7f52c4665740 1 -- 172.21.15.2:0/3337174567 shutdown_connections 2024-09-10T08:14:16.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.794+0000 7f52c4665740 1 -- 172.21.15.2:0/3337174567 wait complete. 2024-09-10T08:14:16.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.794+0000 7f52c4665740 20 client.17034 trim_cache size 0 max 16384 2024-09-10T08:14:16.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.807+0000 7f52c4665740 1 Processor -- start 2024-09-10T08:14:16.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.808+0000 7f52c4665740 1 -- start start 2024-09-10T08:14:16.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.808+0000 7f52c4665740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa8e40 0x557e34aae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.808+0000 7f52c4665740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 0x557e34aae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.808+0000 7f52c4665740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34ab4020 0x557e34aa3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.808+0000 7f52c4665740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1b3800 con 0x557e34aa9760 2024-09-10T08:14:16.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.808+0000 7f52c4665740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1b10d0 con 0x557e34aa8e40 2024-09-10T08:14:16.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.808+0000 7f52c4665740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1aef20 con 0x557e34ab4020 2024-09-10T08:14:16.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.808+0000 7f52b2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 0x557e34aae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.808+0000 7f52b2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 0x557e34aae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45152/0 (socket says 172.21.15.2:45152) 2024-09-10T08:14:16.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.808+0000 7f52b2000640 1 -- 172.21.15.2:0/1781166635 learned_addr learned my addr 172.21.15.2:0/1781166635 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:16.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.808+0000 7f52baa00640 1 --2- 172.21.15.2:0/1781166635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa8e40 0x557e34aae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.808+0000 7f52bb400640 1 --2- 172.21.15.2:0/1781166635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34ab4020 0x557e34aa3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.809+0000 7f52b2000640 1 -- 172.21.15.2:0/1781166635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34ab4020 msgr2=0x557e34aa3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.809+0000 7f52b2000640 1 --2- 172.21.15.2:0/1781166635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34ab4020 0x557e34aa3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.809+0000 7f52b2000640 1 -- 172.21.15.2:0/1781166635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa8e40 msgr2=0x557e34aae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.809+0000 7f52b2000640 1 --2- 172.21.15.2:0/1781166635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa8e40 0x557e34aae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.809+0000 7f52b2000640 1 -- 172.21.15.2:0/1781166635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e34aa8c70 con 0x557e34aa9760 2024-09-10T08:14:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.809+0000 7f52b2000640 1 --2- 172.21.15.2:0/1781166635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 0x557e34aae770 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f52a402a930 tx=0x7f52a40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1deb47aa0be42ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.809+0000 7f52ab400640 1 -- 172.21.15.2:0/1781166635 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4035070 con 0x557e34aa9760 2024-09-10T08:14:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.809+0000 7f52ab400640 1 -- 172.21.15.2:0/1781166635 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a40043c0 con 0x557e34aa9760 2024-09-10T08:14:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.809+0000 7f52ab400640 1 -- 172.21.15.2:0/1781166635 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4031a80 con 0x557e34aa9760 2024-09-10T08:14:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.810+0000 7f52c4665740 1 -- 172.21.15.2:0/1781166635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 msgr2=0x557e34aae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.810+0000 7f52c4665740 1 --2- 172.21.15.2:0/1781166635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 0x557e34aae770 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f52a402a930 tx=0x7f52a40056d0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.810+0000 7f52c4665740 1 -- 172.21.15.2:0/1781166635 shutdown_connections 2024-09-10T08:14:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.810+0000 7f52c4665740 1 --2- 172.21.15.2:0/1781166635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34ab4020 0x557e34aa3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.810+0000 7f52c4665740 1 --2- 172.21.15.2:0/1781166635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34aa9760 0x557e34aae770 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.810+0000 7f52c4665740 1 --2- 172.21.15.2:0/1781166635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa8e40 0x557e34aae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.810+0000 7f52c4665740 1 -- 172.21.15.2:0/1781166635 >> 172.21.15.2:0/1781166635 conn(0x557e34aa53d0 msgr2=0x557e34ab00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.810+0000 7f52c4665740 1 -- 172.21.15.2:0/1781166635 shutdown_connections 2024-09-10T08:14:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.810+0000 7f52c4665740 1 -- 172.21.15.2:0/1781166635 wait complete. 2024-09-10T08:14:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.811+0000 7f52c4665740 1 -- start start 2024-09-10T08:14:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.811+0000 7f52c4665740 1 Processor -- start 2024-09-10T08:14:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.811+0000 7f52c4665740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa8e40 0x557e34ac60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.811+0000 7f52c4665740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa9760 0x557e34ac65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.811+0000 7f52c4665740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34ab4020 0x557e34b2c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.811+0000 7f52c4665740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1af590 con 0x557e34ab4020 2024-09-10T08:14:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.811+0000 7f52c4665740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1b0a00 con 0x557e34aa9760 2024-09-10T08:14:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.811+0000 7f52c4665740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1afc60 con 0x557e34aa8e40 2024-09-10T08:14:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.811+0000 7f52c4665740 10 client.0 _ll_register_callbacks cb 0x7f52a4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.812+0000 7f52b2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa9760 0x557e34ac65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.812+0000 7f52b2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa9760 0x557e34ac65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42382/0 (socket says 172.21.15.2:42382) 2024-09-10T08:14:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.812+0000 7f52b2000640 1 -- 172.21.15.2:0/2865218098 learned_addr learned my addr 172.21.15.2:0/2865218098 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.812+0000 7f52baa00640 1 --2- 172.21.15.2:0/2865218098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa8e40 0x557e34ac60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.812+0000 7f52bb400640 1 --2- 172.21.15.2:0/2865218098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34ab4020 0x557e34b2c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.812+0000 7f52b2000640 1 -- 172.21.15.2:0/2865218098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa8e40 msgr2=0x557e34ac60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.812+0000 7f52b2000640 1 --2- 172.21.15.2:0/2865218098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa8e40 0x557e34ac60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.812+0000 7f52b2000640 1 -- 172.21.15.2:0/2865218098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34ab4020 msgr2=0x557e34b2c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.812+0000 7f52b2000640 1 --2- 172.21.15.2:0/2865218098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34ab4020 0x557e34b2c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.812+0000 7f52b2000640 1 -- 172.21.15.2:0/2865218098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e34ac6e80 con 0x557e34aa9760 2024-09-10T08:14:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.812+0000 7f52b2000640 1 --2- 172.21.15.2:0/2865218098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa9760 0x557e34ac65f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f52a40096c0 tx=0x7f52a40315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8d63c116eae1287 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.812+0000 7f52a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:14:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.812+0000 7f52a8c00640 1 -- 172.21.15.2:0/2865218098 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4035040 con 0x557e34aa9760 2024-09-10T08:14:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.813+0000 7f52c4665740 1 -- 172.21.15.2:0/2865218098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a40043c0 con 0x557e34aa9760 2024-09-10T08:14:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.813+0000 7f52a8c00640 1 -- 172.21.15.2:0/2865218098 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f52a403d640 con 0x557e34aa9760 2024-09-10T08:14:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.813+0000 7f52a8c00640 1 -- 172.21.15.2:0/2865218098 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f52a403dc20 con 0x557e34aa9760 2024-09-10T08:14:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.813+0000 7f52c4665740 10 client.27077 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.813+0000 7f52c4665740 1 -- 172.21.15.2:0/2865218098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557e34b2df20 con 0x557e34aa9760 2024-09-10T08:14:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.813+0000 7f52a8c00640 1 -- 172.21.15.2:0/2865218098 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a40043c0 con 0x557e34aa9760 2024-09-10T08:14:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.814+0000 7f52a8c00640 1 -- 172.21.15.2:0/2865218098 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f52a40496b0 con 0x557e34aa9760 2024-09-10T08:14:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.814+0000 7f52c4665740 10 client.27077 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.814+0000 7f52c4665740 10 client.27077 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:14:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.814+0000 7f52c4665740 20 client.27077 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.814+0000 7f52c4665740 1 --2- 172.21.15.2:0/2865218098 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557e34b2e3b0 0x557e34b307d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.814+0000 7f52c4665740 4 client.27077 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:14:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.814+0000 7f52c4665740 1 -- 172.21.15.2:0/2865218098 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e34a16e60 con 0x557e34b2e3b0 2024-09-10T08:14:16.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.814+0000 7f52baa00640 1 --2- 172.21.15.2:0/2865218098 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557e34b2e3b0 0x557e34b307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:16.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52baa00640 1 --2- 172.21.15.2:0/2865218098 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557e34b2e3b0 0x557e34b307d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c829b052314a4b82 server_cookie=d253db8f359b0c91 in_seq=0 out_seq=0 2024-09-10T08:14:16.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52a8c00640 10 client.27077 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:14:16.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52a8c00640 1 -- 172.21.15.2:0/2865218098 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e34a16e60 con 0x557e34b2e3b0 2024-09-10T08:14:16.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52a8c00640 10 client.27077 handle_command_reply: tid=0 2024-09-10T08:14:16.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52c4665740 1 client.27077 shutdown 2024-09-10T08:14:16.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52c4665740 1 -- 172.21.15.2:0/2865218098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa9760 msgr2=0x557e34ac65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:16.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52c4665740 1 --2- 172.21.15.2:0/2865218098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa9760 0x557e34ac65f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f52a40096c0 tx=0x7f52a40315b0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52c4665740 1 -- 172.21.15.2:0/2865218098 shutdown_connections 2024-09-10T08:14:16.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52c4665740 1 --2- 172.21.15.2:0/2865218098 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557e34b2e3b0 0x557e34b307d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52c4665740 1 --2- 172.21.15.2:0/2865218098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e34ab4020 0x557e34b2c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52c4665740 1 --2- 172.21.15.2:0/2865218098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557e34aa9760 0x557e34ac65f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52c4665740 1 --2- 172.21.15.2:0/2865218098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e34aa8e40 0x557e34ac60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52a8c00640 0 client.27077 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:14:16.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52c4665740 1 -- 172.21.15.2:0/2865218098 >> 172.21.15.2:0/2865218098 conn(0x557e34aa53d0 msgr2=0x557e34ab27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:16.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52c4665740 1 -- 172.21.15.2:0/2865218098 shutdown_connections 2024-09-10T08:14:16.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52c4665740 1 -- 172.21.15.2:0/2865218098 wait complete. 2024-09-10T08:14:16.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.815+0000 7f52c4665740 20 client.27077 trim_cache size 0 max 16384 2024-09-10T08:14:16.819 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:14:16.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.816+0000 7f52bbe00640 1 -- 172.21.15.2:0/1250686499 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52a00a2c00 msgr2=0x7f52a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:16.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.816+0000 7f52bbe00640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52a00a2c00 0x7f52a00a50c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x557e34aa6230 tx=0x7f52ac009880 comp rx=0 tx=0).stop 2024-09-10T08:14:16.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.816+0000 7f52bbe00640 1 -- 172.21.15.2:0/1250686499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52bc108f60 msgr2=0x7f52bc19ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:16.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.816+0000 7f52bbe00640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52bc108f60 0x7f52bc19ede0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f52b400d950 tx=0x7f52b400de10 comp rx=0 tx=0).stop 2024-09-10T08:14:16.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.816+0000 7f52bbe00640 1 -- 172.21.15.2:0/1250686499 shutdown_connections 2024-09-10T08:14:16.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.816+0000 7f52bbe00640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52bc108f60 0x7f52bc19ede0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.816+0000 7f52bbe00640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52a00a2c00 0x7f52a00a50c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.816+0000 7f52bbe00640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52bc1047e0 0x7f52bc19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.816+0000 7f52bbe00640 1 --2- 172.21.15.2:0/1250686499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc103e30 0x7f52bc19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:16.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.816+0000 7f52bbe00640 1 -- 172.21.15.2:0/1250686499 >> 172.21.15.2:0/1250686499 conn(0x7f52bc0ff910 msgr2=0x7f52bc100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:16.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.816+0000 7f52bbe00640 1 -- 172.21.15.2:0/1250686499 shutdown_connections 2024-09-10T08:14:16.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:16.816+0000 7f52bbe00640 1 -- 172.21.15.2:0/1250686499 wait complete. 2024-09-10T08:14:17.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:16 smithi002 ceph-mon[30563]: pgmap v571: 129 pgs: 129 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:14:17.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:16 smithi155 ceph-mon[35405]: pgmap v571: 129 pgs: 129 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:14:17.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:16 smithi107 ceph-mon[35048]: pgmap v571: 129 pgs: 129 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:14:17.807 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:14:18.189 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.819+0000 7f2e48600640 1 -- 172.21.15.2:0/1457829300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e401049a0 msgr2=0x7f2e401089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.819+0000 7f2e48600640 1 --2- 172.21.15.2:0/1457829300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e401049a0 0x7f2e401089f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009900 tx=0x7f2e2c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.819+0000 7f2e48600640 1 -- 172.21.15.2:0/1457829300 shutdown_connections 2024-09-10T08:14:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.819+0000 7f2e48600640 1 --2- 172.21.15.2:0/1457829300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e40109120 0x7f2e40110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.819+0000 7f2e48600640 1 --2- 172.21.15.2:0/1457829300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e401049a0 0x7f2e401089f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.819+0000 7f2e48600640 1 --2- 172.21.15.2:0/1457829300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e40103ff0 0x7f2e401043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.819+0000 7f2e48600640 1 -- 172.21.15.2:0/1457829300 >> 172.21.15.2:0/1457829300 conn(0x7f2e400ffa90 msgr2=0x7f2e40101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.819+0000 7f2e48600640 1 -- 172.21.15.2:0/1457829300 shutdown_connections 2024-09-10T08:14:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.819+0000 7f2e48600640 1 -- 172.21.15.2:0/1457829300 wait complete. 2024-09-10T08:14:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.820+0000 7f2e48600640 1 Processor -- start 2024-09-10T08:14:18.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.820+0000 7f2e48600640 1 -- start start 2024-09-10T08:14:18.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.821+0000 7f2e48600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e40103ff0 0x7f2e4019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.821+0000 7f2e48600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e401049a0 0x7f2e4019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.821+0000 7f2e48600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e40109120 0x7f2e4019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.821+0000 7f2e48600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40114170 con 0x7f2e401049a0 2024-09-10T08:14:18.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.821+0000 7f2e48600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40113ff0 con 0x7f2e40109120 2024-09-10T08:14:18.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.821+0000 7f2e48600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401142f0 con 0x7f2e40103ff0 2024-09-10T08:14:18.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.821+0000 7f2e45200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e401049a0 0x7f2e4019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.821+0000 7f2e45200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e401049a0 0x7f2e4019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57730/0 (socket says 172.21.15.2:57730) 2024-09-10T08:14:18.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.821+0000 7f2e45200640 1 -- 172.21.15.2:0/378503517 learned_addr learned my addr 172.21.15.2:0/378503517 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.821+0000 7f2e46600640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e40109120 0x7f2e4019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.821+0000 7f2e45c00640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e40103ff0 0x7f2e4019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.822+0000 7f2e45200640 1 -- 172.21.15.2:0/378503517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e40103ff0 msgr2=0x7f2e4019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.822+0000 7f2e45200640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e40103ff0 0x7f2e4019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.822+0000 7f2e45200640 1 -- 172.21.15.2:0/378503517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e40109120 msgr2=0x7f2e4019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.822+0000 7f2e45200640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e40109120 0x7f2e4019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.822+0000 7f2e45200640 1 -- 172.21.15.2:0/378503517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4019f600 con 0x7f2e401049a0 2024-09-10T08:14:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.822+0000 7f2e45200640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e401049a0 0x7f2e4019ab90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c004770 tx=0x7f2e2c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.822+0000 7f2e3e000640 1 -- 172.21.15.2:0/378503517 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c03f070 con 0x7f2e401049a0 2024-09-10T08:14:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.823+0000 7f2e3e000640 1 -- 172.21.15.2:0/378503517 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e2c031ca0 con 0x7f2e401049a0 2024-09-10T08:14:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.823+0000 7f2e3e000640 1 -- 172.21.15.2:0/378503517 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c035030 con 0x7f2e401049a0 2024-09-10T08:14:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.823+0000 7f2e48600640 1 -- 172.21.15.2:0/378503517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4019f890 con 0x7f2e401049a0 2024-09-10T08:14:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.823+0000 7f2e48600640 1 -- 172.21.15.2:0/378503517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4019fd20 con 0x7f2e401049a0 2024-09-10T08:14:18.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.827+0000 7f2e4b223740 1 Processor -- start 2024-09-10T08:14:18.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.827+0000 7f2e4b223740 1 -- start start 2024-09-10T08:14:18.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.827+0000 7f2e3e000640 1 -- 172.21.15.2:0/378503517 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2e2c042070 con 0x7f2e401049a0 2024-09-10T08:14:18.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.827+0000 7f2e4b223740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af57c70 0x56496af58050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.827+0000 7f2e4b223740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af58590 0x56496af62910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.827+0000 7f2e4b223740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af62e50 0x56496af65240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.827+0000 7f2e4b223740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56496aebe3e0 con 0x56496af57c70 2024-09-10T08:14:18.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.827+0000 7f2e4b223740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56496ae65d80 con 0x56496af62e50 2024-09-10T08:14:18.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.827+0000 7f2e4b223740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56496aeba3a0 con 0x56496af58590 2024-09-10T08:14:18.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.828+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af58590 0x56496af62910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.828+0000 7f2e46600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af57c70 0x56496af58050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.828+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af58590 0x56496af62910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60994/0 (socket says 172.21.15.2:60994) 2024-09-10T08:14:18.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.828+0000 7f2e45c00640 1 -- 172.21.15.2:0/2424359845 learned_addr learned my addr 172.21.15.2:0/2424359845 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:18.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.828+0000 7f2e45200640 1 --2- 172.21.15.2:0/2424359845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af62e50 0x56496af65240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.828+0000 7f2e45c00640 1 -- 172.21.15.2:0/2424359845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af62e50 msgr2=0x56496af65240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:18.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.828+0000 7f2e45c00640 1 --2- 172.21.15.2:0/2424359845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af62e50 0x56496af65240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.828+0000 7f2e45c00640 1 -- 172.21.15.2:0/2424359845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af57c70 msgr2=0x56496af58050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:18.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.828+0000 7f2e45c00640 1 --2- 172.21.15.2:0/2424359845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af57c70 0x56496af58050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.828+0000 7f2e45c00640 1 -- 172.21.15.2:0/2424359845 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56496ae84ae0 con 0x56496af58590 2024-09-10T08:14:18.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.828+0000 7f2e46600640 1 --2- 172.21.15.2:0/2424359845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af57c70 0x56496af58050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:14:18.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.829+0000 7f2e45c00640 1 --2- 172.21.15.2:0/2424359845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af58590 0x56496af62910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009cf0 tx=0x7f2e34011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb3fbba63d753ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:18.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.829+0000 7f2e32a00640 1 -- 172.21.15.2:0/2424359845 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34004230 con 0x56496af58590 2024-09-10T08:14:18.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.829+0000 7f2e32a00640 1 -- 172.21.15.2:0/2424359845 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e340043d0 con 0x56496af58590 2024-09-10T08:14:18.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.829+0000 7f2e32a00640 1 -- 172.21.15.2:0/2424359845 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34019960 con 0x56496af58590 2024-09-10T08:14:18.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.829+0000 7f2e3e000640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2e280a2c00 0x7f2e280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.829+0000 7f2e3e000640 1 -- 172.21.15.2:0/378503517 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2e2c0ef970 con 0x7f2e401049a0 2024-09-10T08:14:18.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.829+0000 7f2e46600640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2e280a2c00 0x7f2e280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.830+0000 7f2e4b223740 1 -- 172.21.15.2:0/2424359845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af58590 msgr2=0x56496af62910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:18.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.830+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2424359845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af58590 0x56496af62910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009cf0 tx=0x7f2e34011810 comp rx=0 tx=0).stop 2024-09-10T08:14:18.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.830+0000 7f2e4b223740 1 -- 172.21.15.2:0/2424359845 shutdown_connections 2024-09-10T08:14:18.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.830+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2424359845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af62e50 0x56496af65240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.830+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2424359845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af58590 0x56496af62910 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.830+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2424359845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af57c70 0x56496af58050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.830+0000 7f2e4b223740 1 -- 172.21.15.2:0/2424359845 >> 172.21.15.2:0/2424359845 conn(0x56496af54200 msgr2=0x56496af545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:18.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.830+0000 7f2e4b223740 1 -- 172.21.15.2:0/2424359845 shutdown_connections 2024-09-10T08:14:18.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.830+0000 7f2e4b223740 1 -- 172.21.15.2:0/2424359845 wait complete. 2024-09-10T08:14:18.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.830+0000 7f2e46600640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2e280a2c00 0x7f2e280a50c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005fd0 tx=0x7f2e38009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:18.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.831+0000 7f2e4b223740 1 -- start start 2024-09-10T08:14:18.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.831+0000 7f2e4b223740 1 Processor -- start 2024-09-10T08:14:18.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.831+0000 7f2e4b223740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af57c70 0x56496af5fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.831+0000 7f2e4b223740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af58590 0x56496af60230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.831+0000 7f2e4b223740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af62e50 0x56496af60770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.831+0000 7f2e4b223740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56496af65a20 con 0x56496af57c70 2024-09-10T08:14:18.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.831+0000 7f2e4b223740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56496ae790f0 con 0x56496af58590 2024-09-10T08:14:18.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.831+0000 7f2e4b223740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56496af65ba0 con 0x56496af62e50 2024-09-10T08:14:18.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.831+0000 7f2e4b223740 10 client.0 _ll_register_callbacks cb 0x56496ae60bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:18.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.832+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af57c70 0x56496af5fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.832+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af57c70 0x56496af5fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57746/0 (socket says 172.21.15.2:57746) 2024-09-10T08:14:18.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.832+0000 7f2e45c00640 1 -- 172.21.15.2:0/2763199797 learned_addr learned my addr 172.21.15.2:0/2763199797 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:18.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.832+0000 7f2e46600640 1 --2- 172.21.15.2:0/2763199797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af62e50 0x56496af60770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.832+0000 7f2e45200640 1 --2- 172.21.15.2:0/2763199797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af58590 0x56496af60230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.832+0000 7f2e45200640 1 -- 172.21.15.2:0/2763199797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af62e50 msgr2=0x56496af60770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:18.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.832+0000 7f2e45200640 1 --2- 172.21.15.2:0/2763199797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af62e50 0x56496af60770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.832+0000 7f2e45200640 1 -- 172.21.15.2:0/2763199797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af57c70 msgr2=0x56496af5fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:18.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.832+0000 7f2e45200640 1 --2- 172.21.15.2:0/2763199797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af57c70 0x56496af5fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.832+0000 7f2e45200640 1 -- 172.21.15.2:0/2763199797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56496ac2f3c0 con 0x56496af58590 2024-09-10T08:14:18.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.833+0000 7f2e45200640 1 --2- 172.21.15.2:0/2763199797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af58590 0x56496af60230 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0ed0d0 tx=0x7f2e2c02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af71a4fe5e4aae38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:18.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.833+0000 7f2e27400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:14:18.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.833+0000 7f2e27400640 1 -- 172.21.15.2:0/2763199797 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c049070 con 0x56496af58590 2024-09-10T08:14:18.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.833+0000 7f2e4b223740 1 -- 172.21.15.2:0/2763199797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e340043d0 con 0x56496af58590 2024-09-10T08:14:18.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.833+0000 7f2e27400640 1 -- 172.21.15.2:0/2763199797 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2e2c0ba250 con 0x56496af58590 2024-09-10T08:14:18.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.833+0000 7f2e27400640 1 -- 172.21.15.2:0/2763199797 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2e2c005d50 con 0x56496af58590 2024-09-10T08:14:18.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.833+0000 7f2e4b223740 10 client.27089 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:18.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.833+0000 7f2e4b223740 1 -- 172.21.15.2:0/2763199797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56496af71a10 con 0x56496af58590 2024-09-10T08:14:18.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.833+0000 7f2e27400640 1 -- 172.21.15.2:0/2763199797 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e340043d0 con 0x56496af58590 2024-09-10T08:14:18.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.834+0000 7f2e27400640 1 -- 172.21.15.2:0/2763199797 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2e2c03b4f0 con 0x56496af58590 2024-09-10T08:14:18.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.834+0000 7f2e4b223740 10 client.27089 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:18.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.834+0000 7f2e4b223740 10 client.27089 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:14:18.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.834+0000 7f2e4b223740 20 client.27089 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:18.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.834+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2763199797 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x56496af72e70 0x56496af752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.834+0000 7f2e4b223740 4 client.27089 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:18.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.834+0000 7f2e4b223740 1 -- 172.21.15.2:0/2763199797 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56496aecb950 con 0x56496af72e70 2024-09-10T08:14:18.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.834+0000 7f2e45c00640 1 --2- 172.21.15.2:0/2763199797 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x56496af72e70 0x56496af752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.835+0000 7f2e45c00640 1 --2- 172.21.15.2:0/2763199797 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x56496af72e70 0x56496af752c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c7792650a7ab11b server_cookie=1580b69014a0cd55 in_seq=0 out_seq=0 2024-09-10T08:14:18.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.835+0000 7f2e27400640 10 client.27089 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:14:18.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e27400640 1 -- 172.21.15.2:0/2763199797 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56496aecb950 con 0x56496af72e70 2024-09-10T08:14:18.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e27400640 10 client.27089 handle_command_reply: tid=0 2024-09-10T08:14:18.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e4b223740 1 client.27089 shutdown 2024-09-10T08:14:18.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e4b223740 1 -- 172.21.15.2:0/2763199797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af58590 msgr2=0x56496af60230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:18.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2763199797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af58590 0x56496af60230 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0ed0d0 tx=0x7f2e2c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e4b223740 1 -- 172.21.15.2:0/2763199797 shutdown_connections 2024-09-10T08:14:18.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2763199797 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x56496af72e70 0x56496af752c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2763199797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af62e50 0x56496af60770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2763199797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af58590 0x56496af60230 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e27400640 0 client.27089 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:14:18.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2763199797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af57c70 0x56496af5fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e4b223740 1 -- 172.21.15.2:0/2763199797 >> 172.21.15.2:0/2763199797 conn(0x56496af54200 msgr2=0x56496af59f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:18.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e4b223740 1 -- 172.21.15.2:0/2763199797 shutdown_connections 2024-09-10T08:14:18.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e4b223740 1 -- 172.21.15.2:0/2763199797 wait complete. 2024-09-10T08:14:18.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.836+0000 7f2e4b223740 20 client.27089 trim_cache size 0 max 16384 2024-09-10T08:14:18.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.850+0000 7f2e4b223740 1 Processor -- start 2024-09-10T08:14:18.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.850+0000 7f2e4b223740 1 -- start start 2024-09-10T08:14:18.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e4b223740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af57c70 0x56496af5a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e4b223740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af58590 0x56496af61c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e4b223740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af62e50 0x56496af7aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e4b223740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401b3900 con 0x56496af58590 2024-09-10T08:14:18.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e4b223740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401b11d0 con 0x56496af62e50 2024-09-10T08:14:18.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e4b223740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401af020 con 0x56496af57c70 2024-09-10T08:14:18.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e45200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af58590 0x56496af61c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e45200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af58590 0x56496af61c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57756/0 (socket says 172.21.15.2:57756) 2024-09-10T08:14:18.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e45200640 1 -- 172.21.15.2:0/1266536176 learned_addr learned my addr 172.21.15.2:0/1266536176 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:18.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e45c00640 1 --2- 172.21.15.2:0/1266536176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af57c70 0x56496af5a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e46600640 1 --2- 172.21.15.2:0/1266536176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af62e50 0x56496af7aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e45200640 1 -- 172.21.15.2:0/1266536176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af57c70 msgr2=0x56496af5a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:18.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e45200640 1 --2- 172.21.15.2:0/1266536176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af57c70 0x56496af5a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e45200640 1 -- 172.21.15.2:0/1266536176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af62e50 msgr2=0x56496af7aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e45200640 1 --2- 172.21.15.2:0/1266536176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af62e50 0x56496af7aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e45200640 1 -- 172.21.15.2:0/1266536176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56496af54030 con 0x56496af58590 2024-09-10T08:14:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.851+0000 7f2e46600640 1 --2- 172.21.15.2:0/1266536176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af62e50 0x56496af7aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:14:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.852+0000 7f2e45200640 1 --2- 172.21.15.2:0/1266536176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af58590 0x56496af61c60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0bbc50 tx=0x7f2e2c0c4680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c64c393c56ed33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.852+0000 7f2e32000640 1 -- 172.21.15.2:0/1266536176 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c049070 con 0x56496af58590 2024-09-10T08:14:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.852+0000 7f2e32000640 1 -- 172.21.15.2:0/1266536176 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e2c0efc70 con 0x56496af58590 2024-09-10T08:14:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.852+0000 7f2e32000640 1 -- 172.21.15.2:0/1266536176 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c0ba320 con 0x56496af58590 2024-09-10T08:14:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.853+0000 7f2e4b223740 1 -- 172.21.15.2:0/1266536176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af58590 msgr2=0x56496af61c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.853+0000 7f2e4b223740 1 --2- 172.21.15.2:0/1266536176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af58590 0x56496af61c60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0bbc50 tx=0x7f2e2c0c4680 comp rx=0 tx=0).stop 2024-09-10T08:14:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.853+0000 7f2e4b223740 1 -- 172.21.15.2:0/1266536176 shutdown_connections 2024-09-10T08:14:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.853+0000 7f2e4b223740 1 --2- 172.21.15.2:0/1266536176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af62e50 0x56496af7aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.853+0000 7f2e4b223740 1 --2- 172.21.15.2:0/1266536176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af58590 0x56496af61c60 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.853+0000 7f2e4b223740 1 --2- 172.21.15.2:0/1266536176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af57c70 0x56496af5a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.853+0000 7f2e4b223740 1 -- 172.21.15.2:0/1266536176 >> 172.21.15.2:0/1266536176 conn(0x56496af54200 msgr2=0x56496af61550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:18.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.853+0000 7f2e4b223740 1 -- 172.21.15.2:0/1266536176 shutdown_connections 2024-09-10T08:14:18.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.853+0000 7f2e4b223740 1 -- 172.21.15.2:0/1266536176 wait complete. 2024-09-10T08:14:18.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.853+0000 7f2e4b223740 1 -- start start 2024-09-10T08:14:18.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.853+0000 7f2e4b223740 1 Processor -- start 2024-09-10T08:14:18.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.854+0000 7f2e4b223740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af57c70 0x56496af6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.854+0000 7f2e4b223740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af58590 0x56496af668a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.854+0000 7f2e4b223740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af62e50 0x56496af66de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.854+0000 7f2e4b223740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401af690 con 0x56496af62e50 2024-09-10T08:14:18.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.854+0000 7f2e4b223740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401b0b00 con 0x56496af58590 2024-09-10T08:14:18.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.854+0000 7f2e4b223740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401afd60 con 0x56496af57c70 2024-09-10T08:14:18.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.854+0000 7f2e4b223740 10 client.0 _ll_register_callbacks cb 0x7f2e2c009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:18.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.854+0000 7f2e46600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af62e50 0x56496af66de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.854+0000 7f2e46600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af62e50 0x56496af66de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57760/0 (socket says 172.21.15.2:57760) 2024-09-10T08:14:18.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.854+0000 7f2e46600640 1 -- 172.21.15.2:0/2232127061 learned_addr learned my addr 172.21.15.2:0/2232127061 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:18.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.854+0000 7f2e45c00640 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af57c70 0x56496af6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.854+0000 7f2e45200640 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af58590 0x56496af668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.855+0000 7f2e46600640 1 -- 172.21.15.2:0/2232127061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af57c70 msgr2=0x56496af6b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:18.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.855+0000 7f2e46600640 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af57c70 0x56496af6b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.855+0000 7f2e46600640 1 -- 172.21.15.2:0/2232127061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af58590 msgr2=0x56496af668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:18.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.855+0000 7f2e46600640 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af58590 0x56496af668a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.855+0000 7f2e46600640 1 -- 172.21.15.2:0/2232127061 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56496af4f770 con 0x56496af62e50 2024-09-10T08:14:18.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.855+0000 7f2e45c00640 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af57c70 0x56496af6b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:18.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.855+0000 7f2e46600640 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af62e50 0x56496af66de0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2e38004310 tx=0x7f2e38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdcf1387d4ef83f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:18.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.855+0000 7f2e27400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:18.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.855+0000 7f2e27400640 1 -- 172.21.15.2:0/2232127061 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3802e060 con 0x56496af62e50 2024-09-10T08:14:18.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.855+0000 7f2e27400640 1 -- 172.21.15.2:0/2232127061 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2e3801fbc0 con 0x56496af62e50 2024-09-10T08:14:18.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.855+0000 7f2e4b223740 1 -- 172.21.15.2:0/2232127061 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e2c0efc70 con 0x56496af62e50 2024-09-10T08:14:18.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.856+0000 7f2e27400640 1 -- 172.21.15.2:0/2232127061 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2e38035050 con 0x56496af62e50 2024-09-10T08:14:18.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.856+0000 7f2e4b223740 10 client.17058 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:18.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.856+0000 7f2e4b223740 1 -- 172.21.15.2:0/2232127061 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56496afdda30 con 0x56496af62e50 2024-09-10T08:14:18.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.856+0000 7f2e27400640 1 -- 172.21.15.2:0/2232127061 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e2c0efc70 con 0x56496af62e50 2024-09-10T08:14:18.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.856+0000 7f2e27400640 1 -- 172.21.15.2:0/2232127061 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2e3801e420 con 0x56496af62e50 2024-09-10T08:14:18.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.856+0000 7f2e4b223740 10 client.17058 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:18.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.856+0000 7f2e4b223740 10 client.17058 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:14:18.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.856+0000 7f2e4b223740 20 client.17058 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:18.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.856+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x56496afde1b0 0x56496afe05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:18.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.856+0000 7f2e4b223740 4 client.17058 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:14:18.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.856+0000 7f2e4b223740 1 -- 172.21.15.2:0/2232127061 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56496aec5e60 con 0x56496afde1b0 2024-09-10T08:14:18.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.857+0000 7f2e45c00640 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x56496afde1b0 0x56496afe05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:18.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.857+0000 7f2e45c00640 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x56496afde1b0 0x56496afe05d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91bd1c26661437aa server_cookie=764eba73fc2d0321 in_seq=0 out_seq=0 2024-09-10T08:14:18.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.857+0000 7f2e27400640 10 client.17058 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:14:18.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.857+0000 7f2e27400640 1 -- 172.21.15.2:0/2232127061 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56496aec5e60 con 0x56496afde1b0 2024-09-10T08:14:18.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.857+0000 7f2e27400640 10 client.17058 handle_command_reply: tid=0 2024-09-10T08:14:18.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.857+0000 7f2e4b223740 1 client.17058 shutdown 2024-09-10T08:14:18.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.857+0000 7f2e4b223740 1 -- 172.21.15.2:0/2232127061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af62e50 msgr2=0x56496af66de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:18.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.857+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af62e50 0x56496af66de0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2e38004310 tx=0x7f2e38004a40 comp rx=0 tx=0).stop 2024-09-10T08:14:18.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.858+0000 7f2e4b223740 1 -- 172.21.15.2:0/2232127061 shutdown_connections 2024-09-10T08:14:18.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.858+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x56496afde1b0 0x56496afe05d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.858+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56496af62e50 0x56496af66de0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.858+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56496af58590 0x56496af668a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.858+0000 7f2e4b223740 1 --2- 172.21.15.2:0/2232127061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56496af57c70 0x56496af6b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.858+0000 7f2e4b223740 1 -- 172.21.15.2:0/2232127061 >> 172.21.15.2:0/2232127061 conn(0x56496af54200 msgr2=0x56496af53380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:18.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.858+0000 7f2e27400640 0 client.17058 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:14:18.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.858+0000 7f2e4b223740 1 -- 172.21.15.2:0/2232127061 shutdown_connections 2024-09-10T08:14:18.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.858+0000 7f2e4b223740 1 -- 172.21.15.2:0/2232127061 wait complete. 2024-09-10T08:14:18.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.858+0000 7f2e4b223740 20 client.17058 trim_cache size 0 max 16384 2024-09-10T08:14:18.862 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:14:18.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.859+0000 7f2e48600640 1 -- 172.21.15.2:0/378503517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2e280a2c00 msgr2=0x7f2e280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:18.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.859+0000 7f2e48600640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2e280a2c00 0x7f2e280a50c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005fd0 tx=0x7f2e38009500 comp rx=0 tx=0).stop 2024-09-10T08:14:18.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.859+0000 7f2e48600640 1 -- 172.21.15.2:0/378503517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e401049a0 msgr2=0x7f2e4019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:18.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.859+0000 7f2e48600640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e401049a0 0x7f2e4019ab90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c004770 tx=0x7f2e2c003c10 comp rx=0 tx=0).stop 2024-09-10T08:14:18.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.859+0000 7f2e48600640 1 -- 172.21.15.2:0/378503517 shutdown_connections 2024-09-10T08:14:18.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.859+0000 7f2e48600640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e40109120 0x7f2e4019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.859+0000 7f2e48600640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e401049a0 0x7f2e4019ab90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.859+0000 7f2e48600640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2e280a2c00 0x7f2e280a50c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.859+0000 7f2e48600640 1 --2- 172.21.15.2:0/378503517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e40103ff0 0x7f2e4019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:18.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.859+0000 7f2e48600640 1 -- 172.21.15.2:0/378503517 >> 172.21.15.2:0/378503517 conn(0x7f2e400ffa90 msgr2=0x7f2e40100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:18.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.859+0000 7f2e48600640 1 -- 172.21.15.2:0/378503517 shutdown_connections 2024-09-10T08:14:18.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:18.859+0000 7f2e48600640 1 -- 172.21.15.2:0/378503517 wait complete. 2024-09-10T08:14:19.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:18 smithi002 ceph-mon[30563]: pgmap v572: 129 pgs: 129 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:14:19.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:18 smithi155 ceph-mon[35405]: pgmap v572: 129 pgs: 129 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:14:19.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:18 smithi107 ceph-mon[35048]: pgmap v572: 129 pgs: 129 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:14:19.917 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:14:20.299 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:20.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.886+0000 7f4b20c00640 1 -- 172.21.15.2:0/4269693825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b1c1049c0 msgr2=0x7f4b1c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:20.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.886+0000 7f4b20c00640 1 --2- 172.21.15.2:0/4269693825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b1c1049c0 0x7f4b1c108a10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009900 tx=0x7f4b0802a410 comp rx=0 tx=0).stop 2024-09-10T08:14:20.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.886+0000 7f4b20c00640 1 -- 172.21.15.2:0/4269693825 shutdown_connections 2024-09-10T08:14:20.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.886+0000 7f4b20c00640 1 --2- 172.21.15.2:0/4269693825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b1c109140 0x7f4b1c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.886+0000 7f4b20c00640 1 --2- 172.21.15.2:0/4269693825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b1c1049c0 0x7f4b1c108a10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.886+0000 7f4b20c00640 1 --2- 172.21.15.2:0/4269693825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b1c104010 0x7f4b1c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.886+0000 7f4b20c00640 1 -- 172.21.15.2:0/4269693825 >> 172.21.15.2:0/4269693825 conn(0x7f4b1c0ffa50 msgr2=0x7f4b1c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.886+0000 7f4b20c00640 1 -- 172.21.15.2:0/4269693825 shutdown_connections 2024-09-10T08:14:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.886+0000 7f4b20c00640 1 -- 172.21.15.2:0/4269693825 wait complete. 2024-09-10T08:14:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.887+0000 7f4b20c00640 1 Processor -- start 2024-09-10T08:14:20.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.887+0000 7f4b20c00640 1 -- start start 2024-09-10T08:14:20.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.888+0000 7f4b20c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b1c104010 0x7f4b1c076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.888+0000 7f4b20c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b1c1049c0 0x7f4b1c076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.888+0000 7f4b20c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b1c109140 0x7f4b1c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.888+0000 7f4b20c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c114140 con 0x7f4b1c1049c0 2024-09-10T08:14:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.888+0000 7f4b20c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c113fc0 con 0x7f4b1c104010 2024-09-10T08:14:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.888+0000 7f4b20c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1142c0 con 0x7f4b1c109140 2024-09-10T08:14:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.888+0000 7f4b19400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b1c1049c0 0x7f4b1c076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.888+0000 7f4b19400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b1c1049c0 0x7f4b1c076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57790/0 (socket says 172.21.15.2:57790) 2024-09-10T08:14:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.888+0000 7f4b19400640 1 -- 172.21.15.2:0/480142260 learned_addr learned my addr 172.21.15.2:0/480142260 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.888+0000 7f4b19e00640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b1c104010 0x7f4b1c076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.888+0000 7f4b1a800640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b1c109140 0x7f4b1c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.889+0000 7f4b19400640 1 -- 172.21.15.2:0/480142260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b1c109140 msgr2=0x7f4b1c077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.889+0000 7f4b19400640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b1c109140 0x7f4b1c077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.889+0000 7f4b19400640 1 -- 172.21.15.2:0/480142260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b1c104010 msgr2=0x7f4b1c076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.889+0000 7f4b19400640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b1c104010 0x7f4b1c076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.889+0000 7f4b19400640 1 -- 172.21.15.2:0/480142260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c1aacc0 con 0x7f4b1c1049c0 2024-09-10T08:14:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.889+0000 7f4b19400640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b1c1049c0 0x7f4b1c076c40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4b08000c00 tx=0x7f4b080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.889+0000 7f4b12a00640 1 -- 172.21.15.2:0/480142260 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0803f030 con 0x7f4b1c1049c0 2024-09-10T08:14:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.889+0000 7f4b12a00640 1 -- 172.21.15.2:0/480142260 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b08003ce0 con 0x7f4b1c1049c0 2024-09-10T08:14:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.889+0000 7f4b20c00640 1 -- 172.21.15.2:0/480142260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1c1aaf50 con 0x7f4b1c1049c0 2024-09-10T08:14:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.889+0000 7f4b12a00640 1 -- 172.21.15.2:0/480142260 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08035070 con 0x7f4b1c1049c0 2024-09-10T08:14:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.889+0000 7f4b20c00640 1 -- 172.21.15.2:0/480142260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b1c1ab3a0 con 0x7f4b1c1049c0 2024-09-10T08:14:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.894+0000 7f4b237d5740 1 Processor -- start 2024-09-10T08:14:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.894+0000 7f4b237d5740 1 -- start start 2024-09-10T08:14:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.894+0000 7f4b12a00640 1 -- 172.21.15.2:0/480142260 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4b08042030 con 0x7f4b1c1049c0 2024-09-10T08:14:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.894+0000 7f4b237d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6b1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.894+0000 7f4b237d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6b1590 0x559b7a6bb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.894+0000 7f4b237d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6bbe50 0x559b7a6be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.894+0000 7f4b237d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559b7a6173e0 con 0x559b7a6b0c70 2024-09-10T08:14:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.894+0000 7f4b237d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559b7a5bed80 con 0x559b7a6b1590 2024-09-10T08:14:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.894+0000 7f4b237d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559b7a6133a0 con 0x559b7a6bbe50 2024-09-10T08:14:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.895+0000 7f4b1a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.895+0000 7f4b1a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57804/0 (socket says 172.21.15.2:57804) 2024-09-10T08:14:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.895+0000 7f4b19e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6b1590 0x559b7a6bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.895+0000 7f4b1a800640 1 -- 172.21.15.2:0/2495622212 learned_addr learned my addr 172.21.15.2:0/2495622212 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.895+0000 7f4b19400640 1 --2- 172.21.15.2:0/2495622212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6bbe50 0x559b7a6be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.895+0000 7f4b19e00640 1 -- 172.21.15.2:0/2495622212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6bbe50 msgr2=0x559b7a6be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:20.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.895+0000 7f4b19e00640 1 --2- 172.21.15.2:0/2495622212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6bbe50 0x559b7a6be240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.895+0000 7f4b19e00640 1 -- 172.21.15.2:0/2495622212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 msgr2=0x559b7a6b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:20.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.895+0000 7f4b19e00640 1 --2- 172.21.15.2:0/2495622212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.895+0000 7f4b19e00640 1 -- 172.21.15.2:0/2495622212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b7a5ddae0 con 0x559b7a6b1590 2024-09-10T08:14:20.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.896+0000 7f4b19e00640 1 --2- 172.21.15.2:0/2495622212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6b1590 0x559b7a6bb910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c005fd0 tx=0x7f4b0c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1319195e5d3d060f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.896+0000 7f4b12a00640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b000a2b80 0x7f4b000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.896+0000 7f4b07000640 1 -- 172.21.15.2:0/2495622212 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c01c480 con 0x559b7a6b1590 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.896+0000 7f4b12a00640 1 -- 172.21.15.2:0/480142260 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4b080ef780 con 0x7f4b1c1049c0 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.896+0000 7f4b07000640 1 -- 172.21.15.2:0/2495622212 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b0c01cb00 con 0x559b7a6b1590 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.896+0000 7f4b07000640 1 -- 172.21.15.2:0/2495622212 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c011b80 con 0x559b7a6b1590 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.896+0000 7f4b1a800640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b000a2b80 0x7f4b000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.897+0000 7f4b237d5740 1 -- 172.21.15.2:0/2495622212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6b1590 msgr2=0x559b7a6bb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.897+0000 7f4b237d5740 1 --2- 172.21.15.2:0/2495622212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6b1590 0x559b7a6bb910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c005fd0 tx=0x7f4b0c0107a0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.897+0000 7f4b1a800640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b000a2b80 0x7f4b000a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4b14011630 tx=0x7f4b14009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.897+0000 7f4b237d5740 1 -- 172.21.15.2:0/2495622212 shutdown_connections 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.897+0000 7f4b237d5740 1 --2- 172.21.15.2:0/2495622212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6bbe50 0x559b7a6be240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.897+0000 7f4b237d5740 1 --2- 172.21.15.2:0/2495622212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6b1590 0x559b7a6bb910 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.897+0000 7f4b237d5740 1 --2- 172.21.15.2:0/2495622212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.897+0000 7f4b237d5740 1 -- 172.21.15.2:0/2495622212 >> 172.21.15.2:0/2495622212 conn(0x559b7a6ad200 msgr2=0x559b7a6ad5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:20.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.897+0000 7f4b237d5740 1 -- 172.21.15.2:0/2495622212 shutdown_connections 2024-09-10T08:14:20.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.897+0000 7f4b237d5740 1 -- 172.21.15.2:0/2495622212 wait complete. 2024-09-10T08:14:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.898+0000 7f4b237d5740 1 -- start start 2024-09-10T08:14:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.898+0000 7f4b237d5740 1 Processor -- start 2024-09-10T08:14:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b237d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b0c70 0x559b7a6b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b237d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b1590 0x559b7a6b9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b237d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6b9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b237d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559b7a6bea20 con 0x559b7a6b1590 2024-09-10T08:14:20.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b237d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559b7a5d20f0 con 0x559b7a6bbe50 2024-09-10T08:14:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b237d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559b7a6beba0 con 0x559b7a6b0c70 2024-09-10T08:14:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b237d5740 10 client.0 _ll_register_callbacks cb 0x559b7a5b9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b19e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b0c70 0x559b7a6b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b19e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b0c70 0x559b7a6b8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:32812/0 (socket says 172.21.15.2:32812) 2024-09-10T08:14:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b19e00640 1 -- 172.21.15.2:0/89570665 learned_addr learned my addr 172.21.15.2:0/89570665 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b1a800640 1 --2- 172.21.15.2:0/89570665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6b9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b19400640 1 --2- 172.21.15.2:0/89570665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b1590 0x559b7a6b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b19e00640 1 -- 172.21.15.2:0/89570665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 msgr2=0x559b7a6b9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b19e00640 1 --2- 172.21.15.2:0/89570665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6b9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b19e00640 1 -- 172.21.15.2:0/89570665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b1590 msgr2=0x559b7a6b9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b19e00640 1 --2- 172.21.15.2:0/89570665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b1590 0x559b7a6b9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.899+0000 7f4b19e00640 1 -- 172.21.15.2:0/89570665 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b7a3883c0 con 0x559b7a6b0c70 2024-09-10T08:14:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.900+0000 7f4b19e00640 1 --2- 172.21.15.2:0/89570665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b0c70 0x559b7a6b8d10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c005890 tx=0x7f4b0c011660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9f3b95d78548ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:20.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.900+0000 7f4afb400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:14:20.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.902+0000 7f4afb400640 1 -- 172.21.15.2:0/89570665 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c01c690 con 0x559b7a6b0c70 2024-09-10T08:14:20.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.902+0000 7f4b237d5740 1 -- 172.21.15.2:0/89570665 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b0c01cb00 con 0x559b7a6b0c70 2024-09-10T08:14:20.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.902+0000 7f4afb400640 1 -- 172.21.15.2:0/89570665 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4b0c01c830 con 0x559b7a6b0c70 2024-09-10T08:14:20.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.903+0000 7f4afb400640 1 -- 172.21.15.2:0/89570665 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4b0c031050 con 0x559b7a6b0c70 2024-09-10T08:14:20.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.903+0000 7f4b237d5740 10 client.27058 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:20.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.903+0000 7f4b237d5740 1 -- 172.21.15.2:0/89570665 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b7a6caa60 con 0x559b7a6b0c70 2024-09-10T08:14:20.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.903+0000 7f4afb400640 1 -- 172.21.15.2:0/89570665 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b0c01cb00 con 0x559b7a6b0c70 2024-09-10T08:14:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.903+0000 7f4afb400640 1 -- 172.21.15.2:0/89570665 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4b0c026990 con 0x559b7a6b0c70 2024-09-10T08:14:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.903+0000 7f4b237d5740 10 client.27058 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.903+0000 7f4b237d5740 10 client.27058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.903+0000 7f4b237d5740 20 client.27058 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.903+0000 7f4b237d5740 1 --2- 172.21.15.2:0/89570665 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559b7a6cbec0 0x559b7a6ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.903+0000 7f4b237d5740 4 client.27058 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.903+0000 7f4b237d5740 1 -- 172.21.15.2:0/89570665 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b7a624950 con 0x559b7a6cbec0 2024-09-10T08:14:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.904+0000 7f4b19400640 1 --2- 172.21.15.2:0/89570665 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559b7a6cbec0 0x559b7a6ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.904+0000 7f4b19400640 1 --2- 172.21.15.2:0/89570665 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559b7a6cbec0 0x559b7a6ce2b0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa5a9724398b30fd server_cookie=7e5d63b6d10741c3 in_seq=0 out_seq=0 2024-09-10T08:14:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.904+0000 7f4afb400640 10 client.27058 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.905+0000 7f4afb400640 1 -- 172.21.15.2:0/89570665 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b7a624950 con 0x559b7a6cbec0 2024-09-10T08:14:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.905+0000 7f4afb400640 10 client.27058 handle_command_reply: tid=0 2024-09-10T08:14:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.905+0000 7f4b237d5740 1 client.27058 shutdown 2024-09-10T08:14:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.905+0000 7f4b237d5740 1 -- 172.21.15.2:0/89570665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b0c70 msgr2=0x559b7a6b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.905+0000 7f4b237d5740 1 --2- 172.21.15.2:0/89570665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b0c70 0x559b7a6b8d10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c005890 tx=0x7f4b0c011660 comp rx=0 tx=0).stop 2024-09-10T08:14:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.906+0000 7f4b237d5740 1 -- 172.21.15.2:0/89570665 shutdown_connections 2024-09-10T08:14:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.906+0000 7f4b237d5740 1 --2- 172.21.15.2:0/89570665 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559b7a6cbec0 0x559b7a6ce2b0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.906+0000 7f4b237d5740 1 --2- 172.21.15.2:0/89570665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6b9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.906+0000 7f4b237d5740 1 --2- 172.21.15.2:0/89570665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b1590 0x559b7a6b9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.906+0000 7f4b237d5740 1 --2- 172.21.15.2:0/89570665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b0c70 0x559b7a6b8d10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.906+0000 7f4afb400640 0 client.27058 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:20.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.906+0000 7f4b237d5740 1 -- 172.21.15.2:0/89570665 >> 172.21.15.2:0/89570665 conn(0x559b7a6ad200 msgr2=0x559b7a6ac380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:20.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.906+0000 7f4b237d5740 1 -- 172.21.15.2:0/89570665 shutdown_connections 2024-09-10T08:14:20.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.906+0000 7f4b237d5740 1 -- 172.21.15.2:0/89570665 wait complete. 2024-09-10T08:14:20.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.906+0000 7f4b237d5740 20 client.27058 trim_cache size 0 max 16384 2024-09-10T08:14:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.919+0000 7f4b237d5740 1 Processor -- start 2024-09-10T08:14:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.920+0000 7f4b237d5740 1 -- start start 2024-09-10T08:14:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.920+0000 7f4b237d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.920+0000 7f4b237d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b1590 0x559b7a6d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.920+0000 7f4b237d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.920+0000 7f4b237d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1b7d50 con 0x559b7a6b0c70 2024-09-10T08:14:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.920+0000 7f4b237d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1b5620 con 0x559b7a6bbe50 2024-09-10T08:14:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.920+0000 7f4b237d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1b3470 con 0x559b7a6b1590 2024-09-10T08:14:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.920+0000 7f4b19400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b1590 0x559b7a6d3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.920+0000 7f4b1a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.920+0000 7f4b19400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b1590 0x559b7a6d3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:32822/0 (socket says 172.21.15.2:32822) 2024-09-10T08:14:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.920+0000 7f4b19400640 1 -- 172.21.15.2:0/3453312358 learned_addr learned my addr 172.21.15.2:0/3453312358 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.920+0000 7f4b19e00640 1 --2- 172.21.15.2:0/3453312358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.921+0000 7f4b19400640 1 -- 172.21.15.2:0/3453312358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 msgr2=0x559b7a6b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.921+0000 7f4b19400640 1 --2- 172.21.15.2:0/3453312358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.921+0000 7f4b19400640 1 -- 172.21.15.2:0/3453312358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 msgr2=0x559b7a6d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.921+0000 7f4b19400640 1 --2- 172.21.15.2:0/3453312358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.921+0000 7f4b19400640 1 -- 172.21.15.2:0/3453312358 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b7a6ad030 con 0x559b7a6b1590 2024-09-10T08:14:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.921+0000 7f4b1a800640 1 --2- 172.21.15.2:0/3453312358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.921+0000 7f4b19400640 1 --2- 172.21.15.2:0/3453312358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b1590 0x559b7a6d3e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4b080ece90 tx=0x7f4b0803b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=154d16099a8f92bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.921+0000 7f4b06600640 1 -- 172.21.15.2:0/3453312358 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08049040 con 0x559b7a6b1590 2024-09-10T08:14:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.921+0000 7f4b06600640 1 -- 172.21.15.2:0/3453312358 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b08003e90 con 0x559b7a6b1590 2024-09-10T08:14:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.921+0000 7f4b06600640 1 -- 172.21.15.2:0/3453312358 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b080c4920 con 0x559b7a6b1590 2024-09-10T08:14:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.922+0000 7f4b237d5740 1 -- 172.21.15.2:0/3453312358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b1590 msgr2=0x559b7a6d3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.922+0000 7f4b237d5740 1 --2- 172.21.15.2:0/3453312358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b1590 0x559b7a6d3e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4b080ece90 tx=0x7f4b0803b460 comp rx=0 tx=0).stop 2024-09-10T08:14:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.922+0000 7f4b237d5740 1 -- 172.21.15.2:0/3453312358 shutdown_connections 2024-09-10T08:14:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.922+0000 7f4b237d5740 1 --2- 172.21.15.2:0/3453312358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.922+0000 7f4b237d5740 1 --2- 172.21.15.2:0/3453312358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b1590 0x559b7a6d3e50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.922+0000 7f4b237d5740 1 --2- 172.21.15.2:0/3453312358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.922+0000 7f4b237d5740 1 -- 172.21.15.2:0/3453312358 >> 172.21.15.2:0/3453312358 conn(0x559b7a6ad200 msgr2=0x559b7a6bad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.922+0000 7f4b237d5740 1 -- 172.21.15.2:0/3453312358 shutdown_connections 2024-09-10T08:14:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.922+0000 7f4b237d5740 1 -- 172.21.15.2:0/3453312358 wait complete. 2024-09-10T08:14:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.923+0000 7f4b237d5740 1 -- start start 2024-09-10T08:14:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.923+0000 7f4b237d5740 1 Processor -- start 2024-09-10T08:14:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.923+0000 7f4b237d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6bf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.923+0000 7f4b237d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b1590 0x559b7a6bfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.923+0000 7f4b237d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.923+0000 7f4b237d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1b3ae0 con 0x559b7a6b0c70 2024-09-10T08:14:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.923+0000 7f4b237d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1b4f50 con 0x559b7a6bbe50 2024-09-10T08:14:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.923+0000 7f4b237d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1b41b0 con 0x559b7a6b1590 2024-09-10T08:14:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.923+0000 7f4b237d5740 10 client.0 _ll_register_callbacks cb 0x7f4b14009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.924+0000 7f4b19e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.924+0000 7f4b1a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.924+0000 7f4b1a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:43486/0 (socket says 172.21.15.2:43486) 2024-09-10T08:14:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.924+0000 7f4b1a800640 1 -- 172.21.15.2:0/1850781622 learned_addr learned my addr 172.21.15.2:0/1850781622 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.924+0000 7f4b19400640 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b1590 0x559b7a6bfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.924+0000 7f4b1a800640 1 -- 172.21.15.2:0/1850781622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b1590 msgr2=0x559b7a6bfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.924+0000 7f4b1a800640 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b1590 0x559b7a6bfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.924+0000 7f4b1a800640 1 -- 172.21.15.2:0/1850781622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 msgr2=0x559b7a6bf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.924+0000 7f4b1a800640 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6bf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.924+0000 7f4b1a800640 1 -- 172.21.15.2:0/1850781622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b7a6c4180 con 0x559b7a6bbe50 2024-09-10T08:14:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.924+0000 7f4b19400640 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b1590 0x559b7a6bfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.924+0000 7f4b19e00640 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6bf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.925+0000 7f4b1a800640 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6c3ae0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4b14004040 tx=0x7f4b14004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf9030f828c02305 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.925+0000 7f4afb400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:14:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.925+0000 7f4afb400640 1 -- 172.21.15.2:0/1850781622 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1402e060 con 0x559b7a6bbe50 2024-09-10T08:14:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.925+0000 7f4b237d5740 1 -- 172.21.15.2:0/1850781622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b08003e90 con 0x559b7a6bbe50 2024-09-10T08:14:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.925+0000 7f4afb400640 1 -- 172.21.15.2:0/1850781622 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4b1400f9d0 con 0x559b7a6bbe50 2024-09-10T08:14:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.925+0000 7f4afb400640 1 -- 172.21.15.2:0/1850781622 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4b14035050 con 0x559b7a6bbe50 2024-09-10T08:14:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.925+0000 7f4b237d5740 10 client.27113 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.925+0000 7f4b237d5740 1 -- 172.21.15.2:0/1850781622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b7a7363e0 con 0x559b7a6bbe50 2024-09-10T08:14:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.925+0000 7f4afb400640 1 -- 172.21.15.2:0/1850781622 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b08003e90 con 0x559b7a6bbe50 2024-09-10T08:14:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.926+0000 7f4afb400640 1 -- 172.21.15.2:0/1850781622 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4b1401d420 con 0x559b7a6bbe50 2024-09-10T08:14:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.926+0000 7f4b237d5740 10 client.27113 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.926+0000 7f4b237d5740 10 client.27113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.926+0000 7f4b237d5740 20 client.27113 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.926+0000 7f4b237d5740 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559b7a736b40 0x559b7a738f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.926+0000 7f4b237d5740 4 client.27113 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:14:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.926+0000 7f4b237d5740 1 -- 172.21.15.2:0/1850781622 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b7a61ee60 con 0x559b7a736b40 2024-09-10T08:14:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.926+0000 7f4b19e00640 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559b7a736b40 0x559b7a738f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.926+0000 7f4b19e00640 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559b7a736b40 0x559b7a738f60 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d0bf5dac47e558 server_cookie=9540092ce93712a5 in_seq=0 out_seq=0 2024-09-10T08:14:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.926+0000 7f4afb400640 10 client.27113 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4afb400640 1 -- 172.21.15.2:0/1850781622 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b7a61ee60 con 0x559b7a736b40 2024-09-10T08:14:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4afb400640 10 client.27113 handle_command_reply: tid=0 2024-09-10T08:14:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4b237d5740 1 client.27113 shutdown 2024-09-10T08:14:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4b237d5740 1 -- 172.21.15.2:0/1850781622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 msgr2=0x559b7a6c3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4b237d5740 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6c3ae0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4b14004040 tx=0x7f4b14004a40 comp rx=0 tx=0).stop 2024-09-10T08:14:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4b237d5740 1 -- 172.21.15.2:0/1850781622 shutdown_connections 2024-09-10T08:14:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4b237d5740 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559b7a736b40 0x559b7a738f60 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4b237d5740 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b7a6bbe50 0x559b7a6c3ae0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4b237d5740 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b7a6b1590 0x559b7a6bfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4b237d5740 1 --2- 172.21.15.2:0/1850781622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b7a6b0c70 0x559b7a6bf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4afb400640 0 client.27113 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4b237d5740 1 -- 172.21.15.2:0/1850781622 >> 172.21.15.2:0/1850781622 conn(0x559b7a6ad200 msgr2=0x559b7a6abb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4b237d5740 1 -- 172.21.15.2:0/1850781622 shutdown_connections 2024-09-10T08:14:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4b237d5740 1 -- 172.21.15.2:0/1850781622 wait complete. 2024-09-10T08:14:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.927+0000 7f4b237d5740 20 client.27113 trim_cache size 0 max 16384 2024-09-10T08:14:20.932 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:14:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.928+0000 7f4b20c00640 1 -- 172.21.15.2:0/480142260 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b000a2b80 msgr2=0x7f4b000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.928+0000 7f4b20c00640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b000a2b80 0x7f4b000a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4b14011630 tx=0x7f4b14009500 comp rx=0 tx=0).stop 2024-09-10T08:14:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.928+0000 7f4b20c00640 1 -- 172.21.15.2:0/480142260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b1c1049c0 msgr2=0x7f4b1c076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.928+0000 7f4b20c00640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b1c1049c0 0x7f4b1c076c40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4b08000c00 tx=0x7f4b080042f0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.928+0000 7f4b20c00640 1 -- 172.21.15.2:0/480142260 shutdown_connections 2024-09-10T08:14:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.928+0000 7f4b20c00640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b000a2b80 0x7f4b000a5040 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.928+0000 7f4b20c00640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b1c109140 0x7f4b1c077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.928+0000 7f4b20c00640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b1c1049c0 0x7f4b1c076c40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.928+0000 7f4b20c00640 1 --2- 172.21.15.2:0/480142260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b1c104010 0x7f4b1c076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.928+0000 7f4b20c00640 1 -- 172.21.15.2:0/480142260 >> 172.21.15.2:0/480142260 conn(0x7f4b1c0ffa50 msgr2=0x7f4b1c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.928+0000 7f4b20c00640 1 -- 172.21.15.2:0/480142260 shutdown_connections 2024-09-10T08:14:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:20.928+0000 7f4b20c00640 1 -- 172.21.15.2:0/480142260 wait complete. 2024-09-10T08:14:21.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:20 smithi155 ceph-mon[35405]: pgmap v573: 129 pgs: 129 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:14:21.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:20 smithi107 ceph-mon[35048]: pgmap v573: 129 pgs: 129 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:14:21.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:20 smithi002 ceph-mon[30563]: pgmap v573: 129 pgs: 129 active+clean; 3.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:14:22.860 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:14:22.861 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:14:23.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:22 smithi155 ceph-mon[35405]: pgmap v574: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:14:23.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:22 smithi107 ceph-mon[35048]: pgmap v574: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:14:23.254 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:23.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:22 smithi002 ceph-mon[30563]: pgmap v574: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:14:23.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.888+0000 7f0da4c00640 1 -- 172.21.15.2:0/2218957149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0da0109120 msgr2=0x7f0da0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:23.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.888+0000 7f0da4c00640 1 --2- 172.21.15.2:0/2218957149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0da0109120 0x7f0da0110ca0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009ca0 tx=0x7f0d9802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.888+0000 7f0da4c00640 1 -- 172.21.15.2:0/2218957149 shutdown_connections 2024-09-10T08:14:23.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.888+0000 7f0da4c00640 1 --2- 172.21.15.2:0/2218957149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0da0109120 0x7f0da0110ca0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.888+0000 7f0da4c00640 1 --2- 172.21.15.2:0/2218957149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da01049a0 0x7f0da01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.888+0000 7f0da4c00640 1 --2- 172.21.15.2:0/2218957149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0da0103ff0 0x7f0da01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.888+0000 7f0da4c00640 1 -- 172.21.15.2:0/2218957149 >> 172.21.15.2:0/2218957149 conn(0x7f0da00ffa90 msgr2=0x7f0da0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:23.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.888+0000 7f0da4c00640 1 -- 172.21.15.2:0/2218957149 shutdown_connections 2024-09-10T08:14:23.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.888+0000 7f0da4c00640 1 -- 172.21.15.2:0/2218957149 wait complete. 2024-09-10T08:14:23.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.889+0000 7f0da4c00640 1 Processor -- start 2024-09-10T08:14:23.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.890+0000 7f0da4c00640 1 -- start start 2024-09-10T08:14:23.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.890+0000 7f0da4c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0da0103ff0 0x7f0da019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.890+0000 7f0da4c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da01049a0 0x7f0da019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.890+0000 7f0da4c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0da0109120 0x7f0da019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.890+0000 7f0da4c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0114130 con 0x7f0da0109120 2024-09-10T08:14:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.890+0000 7f0da4c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0113fb0 con 0x7f0da01049a0 2024-09-10T08:14:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.890+0000 7f0da4c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0da01142b0 con 0x7f0da0103ff0 2024-09-10T08:14:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.890+0000 7f0d9e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0da0109120 0x7f0da019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.890+0000 7f0d95400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da01049a0 0x7f0da019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.890+0000 7f0d9e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0da0109120 0x7f0da019ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57858/0 (socket says 172.21.15.2:57858) 2024-09-10T08:14:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.890+0000 7f0d9e800640 1 -- 172.21.15.2:0/2663526956 learned_addr learned my addr 172.21.15.2:0/2663526956 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.890+0000 7f0d9de00640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0da0103ff0 0x7f0da019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.891+0000 7f0d95400640 1 -- 172.21.15.2:0/2663526956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0da0103ff0 msgr2=0x7f0da019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:23.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.891+0000 7f0d95400640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0da0103ff0 0x7f0da019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.891+0000 7f0d95400640 1 -- 172.21.15.2:0/2663526956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0da0109120 msgr2=0x7f0da019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.891+0000 7f0d95400640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0da0109120 0x7f0da019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.891+0000 7f0d95400640 1 -- 172.21.15.2:0/2663526956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da019f540 con 0x7f0da01049a0 2024-09-10T08:14:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.891+0000 7f0d9e800640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0da0109120 0x7f0da019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.891+0000 7f0d95400640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da01049a0 0x7f0da019aad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800e9f0 tx=0x7f0d8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.891+0000 7f0d97400640 1 -- 172.21.15.2:0/2663526956 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800ccc0 con 0x7f0da01049a0 2024-09-10T08:14:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.891+0000 7f0da4c00640 1 -- 172.21.15.2:0/2663526956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da019f830 con 0x7f0da01049a0 2024-09-10T08:14:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.891+0000 7f0d97400640 1 -- 172.21.15.2:0/2663526956 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d88010910 con 0x7f0da01049a0 2024-09-10T08:14:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.892+0000 7f0d97400640 1 -- 172.21.15.2:0/2663526956 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88018960 con 0x7f0da01049a0 2024-09-10T08:14:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.892+0000 7f0da4c00640 1 -- 172.21.15.2:0/2663526956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da01a7110 con 0x7f0da01049a0 2024-09-10T08:14:23.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.894+0000 7f0da780e740 1 Processor -- start 2024-09-10T08:14:23.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.895+0000 7f0da780e740 1 -- start start 2024-09-10T08:14:23.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.895+0000 7f0da780e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da0fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.895+0000 7f0da780e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da1a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.895+0000 7f0da780e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 0x563e9da1d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.895+0000 7f0d97400640 1 -- 172.21.15.2:0/2663526956 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0d88010ac0 con 0x7f0da01049a0 2024-09-10T08:14:23.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.895+0000 7f0da780e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563e9d976290 con 0x563e9da0fac0 2024-09-10T08:14:23.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.895+0000 7f0da780e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563e9d91db40 con 0x563e9da103e0 2024-09-10T08:14:23.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.895+0000 7f0da780e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563e9d972250 con 0x563e9da1aca0 2024-09-10T08:14:23.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.895+0000 7f0d9de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da1a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.895+0000 7f0d9e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da0fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.895+0000 7f0d9de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da1a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:43504/0 (socket says 172.21.15.2:43504) 2024-09-10T08:14:23.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.895+0000 7f0d9de00640 1 -- 172.21.15.2:0/3919619144 learned_addr learned my addr 172.21.15.2:0/3919619144 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:23.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.896+0000 7f0d95400640 1 --2- 172.21.15.2:0/3919619144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 0x563e9da1d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.896+0000 7f0d9de00640 1 -- 172.21.15.2:0/3919619144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 msgr2=0x563e9da1d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:23.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.896+0000 7f0d9de00640 1 --2- 172.21.15.2:0/3919619144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 0x563e9da1d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.896+0000 7f0d9de00640 1 -- 172.21.15.2:0/3919619144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 msgr2=0x563e9da0fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:23.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.896+0000 7f0d9de00640 1 --2- 172.21.15.2:0/3919619144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da0fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.896+0000 7f0d9de00640 1 -- 172.21.15.2:0/3919619144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e9d6e76a0 con 0x563e9da103e0 2024-09-10T08:14:23.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.896+0000 7f0d9de00640 1 --2- 172.21.15.2:0/3919619144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da1a760 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009cf0 tx=0x7f0d90011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbaedbe57b1572ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:23.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.896+0000 7f0d8f000640 1 -- 172.21.15.2:0/3919619144 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9001e070 con 0x563e9da103e0 2024-09-10T08:14:23.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.896+0000 7f0d8f000640 1 -- 172.21.15.2:0/3919619144 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d90019b10 con 0x563e9da103e0 2024-09-10T08:14:23.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.896+0000 7f0d8f000640 1 -- 172.21.15.2:0/3919619144 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9000ea60 con 0x563e9da103e0 2024-09-10T08:14:23.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.897+0000 7f0d97400640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d840a2b80 0x7f0d840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.897+0000 7f0d97400640 1 -- 172.21.15.2:0/2663526956 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0d880cf7b0 con 0x7f0da01049a0 2024-09-10T08:14:23.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.897+0000 7f0da780e740 1 -- 172.21.15.2:0/3919619144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 msgr2=0x563e9da1a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:23.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.897+0000 7f0da780e740 1 --2- 172.21.15.2:0/3919619144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da1a760 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009cf0 tx=0x7f0d90011810 comp rx=0 tx=0).stop 2024-09-10T08:14:23.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.897+0000 7f0d9e800640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d840a2b80 0x7f0d840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.897+0000 7f0da780e740 1 -- 172.21.15.2:0/3919619144 shutdown_connections 2024-09-10T08:14:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.897+0000 7f0da780e740 1 --2- 172.21.15.2:0/3919619144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 0x563e9da1d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.897+0000 7f0da780e740 1 --2- 172.21.15.2:0/3919619144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da1a760 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.897+0000 7f0da780e740 1 --2- 172.21.15.2:0/3919619144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da0fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.897+0000 7f0da780e740 1 -- 172.21.15.2:0/3919619144 >> 172.21.15.2:0/3919619144 conn(0x563e9da0c050 msgr2=0x563e9da0c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.897+0000 7f0da780e740 1 -- 172.21.15.2:0/3919619144 shutdown_connections 2024-09-10T08:14:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.898+0000 7f0da780e740 1 -- 172.21.15.2:0/3919619144 wait complete. 2024-09-10T08:14:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.898+0000 7f0d9e800640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d840a2b80 0x7f0d840a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0d98031f80 tx=0x7f0d98004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.898+0000 7f0da780e740 1 -- start start 2024-09-10T08:14:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.898+0000 7f0da780e740 1 Processor -- start 2024-09-10T08:14:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0da780e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da0fac0 0x563e9da1a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0da780e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da103e0 0x563e9da178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0da780e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da1aca0 0x563e9da17e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0da780e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563e9da1d910 con 0x563e9da103e0 2024-09-10T08:14:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0da780e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563e9d930fa0 con 0x563e9da1aca0 2024-09-10T08:14:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0da780e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563e9da1da90 con 0x563e9da0fac0 2024-09-10T08:14:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0da780e740 10 client.0 _ll_register_callbacks cb 0x563e9d684ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0d9de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da0fac0 0x563e9da1a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0d9de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da0fac0 0x563e9da1a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:32880/0 (socket says 172.21.15.2:32880) 2024-09-10T08:14:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0d9de00640 1 -- 172.21.15.2:0/1969509606 learned_addr learned my addr 172.21.15.2:0/1969509606 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0d95400640 1 --2- 172.21.15.2:0/1969509606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da103e0 0x563e9da178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0d9e800640 1 --2- 172.21.15.2:0/1969509606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da1aca0 0x563e9da17e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0d9de00640 1 -- 172.21.15.2:0/1969509606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da1aca0 msgr2=0x563e9da17e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0d9de00640 1 --2- 172.21.15.2:0/1969509606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da1aca0 0x563e9da17e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0d9de00640 1 -- 172.21.15.2:0/1969509606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da103e0 msgr2=0x563e9da178a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:23.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0d9de00640 1 --2- 172.21.15.2:0/1969509606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da103e0 0x563e9da178a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.899+0000 7f0d9de00640 1 -- 172.21.15.2:0/1969509606 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e9da18630 con 0x563e9da0fac0 2024-09-10T08:14:23.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.900+0000 7f0d9de00640 1 --2- 172.21.15.2:0/1969509606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da0fac0 0x563e9da1a5f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0d900097b0 tx=0x7f0d90019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=857a83fac6f5aa5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:23.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.900+0000 7f0d83400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:14:23.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.900+0000 7f0d83400640 1 -- 172.21.15.2:0/1969509606 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9001e050 con 0x563e9da0fac0 2024-09-10T08:14:23.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.900+0000 7f0da780e740 1 -- 172.21.15.2:0/1969509606 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d90019b10 con 0x563e9da0fac0 2024-09-10T08:14:23.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.900+0000 7f0d83400640 1 -- 172.21.15.2:0/1969509606 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0d90005240 con 0x563e9da0fac0 2024-09-10T08:14:23.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.900+0000 7f0d83400640 1 -- 172.21.15.2:0/1969509606 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0d90033050 con 0x563e9da0fac0 2024-09-10T08:14:23.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.900+0000 7f0da780e740 10 client.27076 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:23.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.900+0000 7f0da780e740 1 -- 172.21.15.2:0/1969509606 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e9da29850 con 0x563e9da0fac0 2024-09-10T08:14:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.901+0000 7f0d83400640 1 -- 172.21.15.2:0/1969509606 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d90019b10 con 0x563e9da0fac0 2024-09-10T08:14:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.901+0000 7f0d83400640 1 -- 172.21.15.2:0/1969509606 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0d900054d0 con 0x563e9da0fac0 2024-09-10T08:14:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.901+0000 7f0da780e740 10 client.27076 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.901+0000 7f0da780e740 10 client.27076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.901+0000 7f0da780e740 20 client.27076 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.901+0000 7f0da780e740 1 --2- 172.21.15.2:0/1969509606 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563e9da2acb0 0x563e9da2d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.901+0000 7f0da780e740 4 client.27076 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.901+0000 7f0da780e740 1 -- 172.21.15.2:0/1969509606 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e9d983800 con 0x563e9da2acb0 2024-09-10T08:14:23.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.901+0000 7f0d95400640 1 --2- 172.21.15.2:0/1969509606 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563e9da2acb0 0x563e9da2d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.902+0000 7f0d95400640 1 --2- 172.21.15.2:0/1969509606 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563e9da2acb0 0x563e9da2d0a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fcfa36b92a6fa82 server_cookie=7be612168bc48556 in_seq=0 out_seq=0 2024-09-10T08:14:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.902+0000 7f0d83400640 10 client.27076 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0d83400640 1 -- 172.21.15.2:0/1969509606 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e9d983800 con 0x563e9da2acb0 2024-09-10T08:14:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0d83400640 10 client.27076 handle_command_reply: tid=0 2024-09-10T08:14:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0da780e740 1 client.27076 shutdown 2024-09-10T08:14:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0da780e740 1 -- 172.21.15.2:0/1969509606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da0fac0 msgr2=0x563e9da1a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0da780e740 1 --2- 172.21.15.2:0/1969509606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da0fac0 0x563e9da1a5f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0d900097b0 tx=0x7f0d90019660 comp rx=0 tx=0).stop 2024-09-10T08:14:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0da780e740 1 -- 172.21.15.2:0/1969509606 shutdown_connections 2024-09-10T08:14:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0da780e740 1 --2- 172.21.15.2:0/1969509606 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563e9da2acb0 0x563e9da2d0a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0da780e740 1 --2- 172.21.15.2:0/1969509606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da1aca0 0x563e9da17e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0da780e740 1 --2- 172.21.15.2:0/1969509606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da103e0 0x563e9da178a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0da780e740 1 --2- 172.21.15.2:0/1969509606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da0fac0 0x563e9da1a5f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0d83400640 0 client.27076 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:23.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0da780e740 1 -- 172.21.15.2:0/1969509606 >> 172.21.15.2:0/1969509606 conn(0x563e9da0c050 msgr2=0x563e9da0b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:23.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0da780e740 1 -- 172.21.15.2:0/1969509606 shutdown_connections 2024-09-10T08:14:23.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0da780e740 1 -- 172.21.15.2:0/1969509606 wait complete. 2024-09-10T08:14:23.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.903+0000 7f0da780e740 20 client.27076 trim_cache size 0 max 16384 2024-09-10T08:14:23.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.922+0000 7f0da780e740 1 Processor -- start 2024-09-10T08:14:23.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.922+0000 7f0da780e740 1 -- start start 2024-09-10T08:14:23.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.922+0000 7f0da780e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da32830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.922+0000 7f0da780e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da32d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.922+0000 7f0da780e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 0x563e9da14b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.922+0000 7f0da780e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0da01b3950 con 0x563e9da0fac0 2024-09-10T08:14:23.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.922+0000 7f0da780e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0da01b1220 con 0x563e9da103e0 2024-09-10T08:14:23.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.922+0000 7f0da780e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0da01af070 con 0x563e9da1aca0 2024-09-10T08:14:23.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.923+0000 7f0d9de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da32830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.923+0000 7f0d9de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da32830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57876/0 (socket says 172.21.15.2:57876) 2024-09-10T08:14:23.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.923+0000 7f0d9de00640 1 -- 172.21.15.2:0/3928538816 learned_addr learned my addr 172.21.15.2:0/3928538816 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:23.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.923+0000 7f0d95400640 1 --2- 172.21.15.2:0/3928538816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da32d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.923+0000 7f0d9e800640 1 --2- 172.21.15.2:0/3928538816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 0x563e9da14b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.923+0000 7f0d9de00640 1 -- 172.21.15.2:0/3928538816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 msgr2=0x563e9da14b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:23.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.923+0000 7f0d9de00640 1 --2- 172.21.15.2:0/3928538816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 0x563e9da14b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.923+0000 7f0d9de00640 1 -- 172.21.15.2:0/3928538816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 msgr2=0x563e9da32d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:23.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.923+0000 7f0d9de00640 1 --2- 172.21.15.2:0/3928538816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da32d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.923+0000 7f0d9de00640 1 -- 172.21.15.2:0/3928538816 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e9da0d1e0 con 0x563e9da0fac0 2024-09-10T08:14:23.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.923+0000 7f0d9de00640 1 --2- 172.21.15.2:0/3928538816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da32830 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0d90011850 tx=0x7f0d90019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994b5d2e607a23ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:23.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.924+0000 7f0d8e600640 1 -- 172.21.15.2:0/3928538816 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9001e070 con 0x563e9da0fac0 2024-09-10T08:14:23.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.924+0000 7f0d8e600640 1 -- 172.21.15.2:0/3928538816 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d9002a070 con 0x563e9da0fac0 2024-09-10T08:14:23.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.924+0000 7f0d8e600640 1 -- 172.21.15.2:0/3928538816 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90004280 con 0x563e9da0fac0 2024-09-10T08:14:23.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.924+0000 7f0da780e740 1 -- 172.21.15.2:0/3928538816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 msgr2=0x563e9da32830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:23.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.924+0000 7f0da780e740 1 --2- 172.21.15.2:0/3928538816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da32830 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0d90011850 tx=0x7f0d90019750 comp rx=0 tx=0).stop 2024-09-10T08:14:23.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.924+0000 7f0da780e740 1 -- 172.21.15.2:0/3928538816 shutdown_connections 2024-09-10T08:14:23.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.924+0000 7f0da780e740 1 --2- 172.21.15.2:0/3928538816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 0x563e9da14b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.924+0000 7f0da780e740 1 --2- 172.21.15.2:0/3928538816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da32d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.924+0000 7f0da780e740 1 --2- 172.21.15.2:0/3928538816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da32830 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.924+0000 7f0da780e740 1 -- 172.21.15.2:0/3928538816 >> 172.21.15.2:0/3928538816 conn(0x563e9da0c050 msgr2=0x563e9da12f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:23.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.925+0000 7f0da780e740 1 -- 172.21.15.2:0/3928538816 shutdown_connections 2024-09-10T08:14:23.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.925+0000 7f0da780e740 1 -- 172.21.15.2:0/3928538816 wait complete. 2024-09-10T08:14:23.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.925+0000 7f0da780e740 1 -- start start 2024-09-10T08:14:23.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.925+0000 7f0da780e740 1 Processor -- start 2024-09-10T08:14:23.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.926+0000 7f0da780e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da22fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.926+0000 7f0da780e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da234f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.926+0000 7f0da780e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 0x563e9da1e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.926+0000 7f0da780e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0da01af6e0 con 0x563e9da0fac0 2024-09-10T08:14:23.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.926+0000 7f0da780e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0da01b0b50 con 0x563e9da103e0 2024-09-10T08:14:23.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.926+0000 7f0da780e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0da01afdb0 con 0x563e9da1aca0 2024-09-10T08:14:23.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.926+0000 7f0da780e740 10 client.0 _ll_register_callbacks cb 0x563e9da18800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:23.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.926+0000 7f0d9de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da22fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.926+0000 7f0d9de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da22fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57882/0 (socket says 172.21.15.2:57882) 2024-09-10T08:14:23.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.926+0000 7f0d9de00640 1 -- 172.21.15.2:0/1566995640 learned_addr learned my addr 172.21.15.2:0/1566995640 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:23.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.926+0000 7f0d95400640 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da234f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.926+0000 7f0d9e800640 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 0x563e9da1e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0d95400640 1 -- 172.21.15.2:0/1566995640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 msgr2=0x563e9da1e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:23.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0d95400640 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 0x563e9da1e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0d95400640 1 -- 172.21.15.2:0/1566995640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 msgr2=0x563e9da22fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0d95400640 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da22fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0d95400640 1 -- 172.21.15.2:0/1566995640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e9da1eb70 con 0x563e9da103e0 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0d9de00640 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da22fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0d95400640 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da234f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x563e9da0d3d0 tx=0x7f0d88009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3b6e70b2b1cc36c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0d83400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0d83400640 1 -- 172.21.15.2:0/1566995640 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d880982e0 con 0x563e9da103e0 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0d83400640 1 -- 172.21.15.2:0/1566995640 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0d880cf400 con 0x563e9da103e0 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0da780e740 1 -- 172.21.15.2:0/1566995640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d9002a070 con 0x563e9da103e0 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0d83400640 1 -- 172.21.15.2:0/1566995640 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0d88098480 con 0x563e9da103e0 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0da780e740 10 client.27131 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.927+0000 7f0da780e740 1 -- 172.21.15.2:0/1566995640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e9da953c0 con 0x563e9da103e0 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.928+0000 7f0d83400640 1 -- 172.21.15.2:0/1566995640 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d9002a070 con 0x563e9da103e0 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.928+0000 7f0d83400640 1 -- 172.21.15.2:0/1566995640 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0d880ba020 con 0x563e9da103e0 2024-09-10T08:14:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.928+0000 7f0da780e740 10 client.27131 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:23.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.928+0000 7f0da780e740 10 client.27131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:23.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.928+0000 7f0da780e740 20 client.27131 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:23.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.928+0000 7f0da780e740 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563e9da95af0 0x563e9da97f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:23.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.928+0000 7f0da780e740 4 client.27131 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:14:23.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.928+0000 7f0da780e740 1 -- 172.21.15.2:0/1566995640 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563e9d97dd10 con 0x563e9da95af0 2024-09-10T08:14:23.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.928+0000 7f0d9de00640 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563e9da95af0 0x563e9da97f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:23.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.929+0000 7f0d9de00640 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563e9da95af0 0x563e9da97f10 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d828fffb7a40b6 server_cookie=7ab7b00f7835acf in_seq=0 out_seq=0 2024-09-10T08:14:23.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.929+0000 7f0d83400640 10 client.27131 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:23.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0d83400640 1 -- 172.21.15.2:0/1566995640 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563e9d97dd10 con 0x563e9da95af0 2024-09-10T08:14:23.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0d83400640 10 client.27131 handle_command_reply: tid=0 2024-09-10T08:14:23.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0da780e740 1 client.27131 shutdown 2024-09-10T08:14:23.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0da780e740 1 -- 172.21.15.2:0/1566995640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 msgr2=0x563e9da234f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:23.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0da780e740 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da234f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x563e9da0d3d0 tx=0x7f0d88009700 comp rx=0 tx=0).stop 2024-09-10T08:14:23.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0da780e740 1 -- 172.21.15.2:0/1566995640 shutdown_connections 2024-09-10T08:14:23.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0da780e740 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563e9da95af0 0x563e9da97f10 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0da780e740 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9da1aca0 0x563e9da1e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0da780e740 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e9da103e0 0x563e9da234f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0da780e740 1 --2- 172.21.15.2:0/1566995640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e9da0fac0 0x563e9da22fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0d83400640 0 client.27131 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:23.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0da780e740 1 -- 172.21.15.2:0/1566995640 >> 172.21.15.2:0/1566995640 conn(0x563e9da0c050 msgr2=0x563e9da16170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:23.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0da780e740 1 -- 172.21.15.2:0/1566995640 shutdown_connections 2024-09-10T08:14:23.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.930+0000 7f0da780e740 1 -- 172.21.15.2:0/1566995640 wait complete. 2024-09-10T08:14:23.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.931+0000 7f0da780e740 20 client.27131 trim_cache size 0 max 16384 2024-09-10T08:14:23.933 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:14:23.933 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:14:23.933 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "6ceb7d3d-b060-43ee-b343-22d00267936e", 2024-09-10T08:14:23.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:14:23.934 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:14:23.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.931+0000 7f0da4c00640 1 -- 172.21.15.2:0/2663526956 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d840a2b80 msgr2=0x7f0d840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:23.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.931+0000 7f0da4c00640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d840a2b80 0x7f0d840a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0d98031f80 tx=0x7f0d98004a60 comp rx=0 tx=0).stop 2024-09-10T08:14:23.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.931+0000 7f0da4c00640 1 -- 172.21.15.2:0/2663526956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da01049a0 msgr2=0x7f0da019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:23.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.931+0000 7f0da4c00640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da01049a0 0x7f0da019aad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800e9f0 tx=0x7f0d8800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.931+0000 7f0da4c00640 1 -- 172.21.15.2:0/2663526956 shutdown_connections 2024-09-10T08:14:23.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.931+0000 7f0da4c00640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0da0109120 0x7f0da019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.931+0000 7f0da4c00640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da01049a0 0x7f0da019aad0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.931+0000 7f0da4c00640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0d840a2b80 0x7f0d840a5040 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.931+0000 7f0da4c00640 1 --2- 172.21.15.2:0/2663526956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0da0103ff0 0x7f0da019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:23.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.931+0000 7f0da4c00640 1 -- 172.21.15.2:0/2663526956 >> 172.21.15.2:0/2663526956 conn(0x7f0da00ffa90 msgr2=0x7f0da0101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:23.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.932+0000 7f0da4c00640 1 -- 172.21.15.2:0/2663526956 shutdown_connections 2024-09-10T08:14:23.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:23.932+0000 7f0da4c00640 1 -- 172.21.15.2:0/2663526956 wait complete. 2024-09-10T08:14:24.938 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:14:25.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:24 smithi155 ceph-mon[35405]: pgmap v575: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:14:25.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:24 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:14:25.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:24 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:14:25.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:24 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:14:25.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:24 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:14:25.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:24 smithi107 ceph-mon[35048]: pgmap v575: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:14:25.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:24 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:14:25.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:24 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:14:25.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:24 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:14:25.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:24 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:14:25.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:24 smithi002 ceph-mon[30563]: pgmap v575: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:14:25.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:24 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:14:25.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:24 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:14:25.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:24 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:14:25.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:24 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:14:25.315 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.762+0000 7f409b200640 1 -- 172.21.15.2:0/1081535673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f409c101580 msgr2=0x7f409c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.762+0000 7f409b200640 1 --2- 172.21.15.2:0/1081535673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f409c101580 0x7f409c101a00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4088009900 tx=0x7f408802a400 comp rx=0 tx=0).stop 2024-09-10T08:14:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.762+0000 7f409b200640 1 -- 172.21.15.2:0/1081535673 shutdown_connections 2024-09-10T08:14:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.762+0000 7f409b200640 1 --2- 172.21.15.2:0/1081535673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f409c101f40 0x7f409c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.762+0000 7f409b200640 1 --2- 172.21.15.2:0/1081535673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f409c101580 0x7f409c101a00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.762+0000 7f409b200640 1 --2- 172.21.15.2:0/1081535673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409c103050 0x7f409c103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.762+0000 7f409b200640 1 -- 172.21.15.2:0/1081535673 >> 172.21.15.2:0/1081535673 conn(0x7f409c0fd0d0 msgr2=0x7f409c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.762+0000 7f409b200640 1 -- 172.21.15.2:0/1081535673 shutdown_connections 2024-09-10T08:14:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.762+0000 7f409b200640 1 -- 172.21.15.2:0/1081535673 wait complete. 2024-09-10T08:14:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.763+0000 7f409b200640 1 Processor -- start 2024-09-10T08:14:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.763+0000 7f409b200640 1 -- start start 2024-09-10T08:14:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f409b200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f409c101580 0x7f409c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f409b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f409c101f40 0x7f409c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f409b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409c103050 0x7f409c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f409b200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f409c10d720 con 0x7f409c101580 2024-09-10T08:14:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f409b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f409c10d5a0 con 0x7f409c101f40 2024-09-10T08:14:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f409b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f409c10d8a0 con 0x7f409c103050 2024-09-10T08:14:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f4099e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f409c101580 0x7f409c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f4099e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f409c101580 0x7f409c197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57890/0 (socket says 172.21.15.2:57890) 2024-09-10T08:14:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f4099e00640 1 -- 172.21.15.2:0/1899579189 learned_addr learned my addr 172.21.15.2:0/1899579189 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f4099400640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f409c101f40 0x7f409c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f409a800640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409c103050 0x7f409c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f4099e00640 1 -- 172.21.15.2:0/1899579189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409c103050 msgr2=0x7f409c19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f4099e00640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409c103050 0x7f409c19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f4099e00640 1 -- 172.21.15.2:0/1899579189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f409c101f40 msgr2=0x7f409c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f4099e00640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f409c101f40 0x7f409c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.764+0000 7f4099e00640 1 -- 172.21.15.2:0/1899579189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c19f9f0 con 0x7f409c101580 2024-09-10T08:14:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.765+0000 7f4099e00640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f409c101580 0x7f409c197dd0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f408c00e9c0 tx=0x7f408c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.765+0000 7f4092a00640 1 -- 172.21.15.2:0/1899579189 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c00cd80 con 0x7f409c101580 2024-09-10T08:14:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.765+0000 7f4092a00640 1 -- 172.21.15.2:0/1899579189 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f408c0035f0 con 0x7f409c101580 2024-09-10T08:14:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.765+0000 7f409b200640 1 -- 172.21.15.2:0/1899579189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c19fce0 con 0x7f409c101580 2024-09-10T08:14:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.765+0000 7f4092a00640 1 -- 172.21.15.2:0/1899579189 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c010690 con 0x7f409c101580 2024-09-10T08:14:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.765+0000 7f409b200640 1 -- 172.21.15.2:0/1899579189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c1a01b0 con 0x7f409c101580 2024-09-10T08:14:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.768+0000 7f4092a00640 1 -- 172.21.15.2:0/1899579189 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f408c0037a0 con 0x7f409c101580 2024-09-10T08:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.768+0000 7f40a4064740 1 Processor -- start 2024-09-10T08:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.768+0000 7f40a4064740 1 -- start start 2024-09-10T08:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.769+0000 7f40a4064740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 0x55e560121660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.769+0000 7f40a4064740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e560126660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.769+0000 7f40a4064740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 0x55e5601290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.769+0000 7f40a4064740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e560086100 con 0x55e560121260 2024-09-10T08:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.769+0000 7f40a4064740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e56002e1b0 con 0x55e560121ba0 2024-09-10T08:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.769+0000 7f40a4064740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e5600820c0 con 0x55e560126c30 2024-09-10T08:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.769+0000 7f409a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 0x55e560121660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.769+0000 7f409a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 0x55e560121660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57904/0 (socket says 172.21.15.2:57904) 2024-09-10T08:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.769+0000 7f409a800640 1 -- 172.21.15.2:0/2210405550 learned_addr learned my addr 172.21.15.2:0/2210405550 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.769+0000 7f4099e00640 1 --2- 172.21.15.2:0/2210405550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e560126660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.769+0000 7f4099400640 1 --2- 172.21.15.2:0/2210405550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 0x55e5601290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.770+0000 7f409a800640 1 -- 172.21.15.2:0/2210405550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 msgr2=0x55e5601290f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.770+0000 7f409a800640 1 --2- 172.21.15.2:0/2210405550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 0x55e5601290f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.770+0000 7f409a800640 1 -- 172.21.15.2:0/2210405550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 msgr2=0x55e560126660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.770+0000 7f409a800640 1 --2- 172.21.15.2:0/2210405550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e560126660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.770+0000 7f409a800640 1 -- 172.21.15.2:0/2210405550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55fd99210 con 0x55e560121260 2024-09-10T08:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.770+0000 7f4099e00640 1 --2- 172.21.15.2:0/2210405550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e560126660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.770+0000 7f4092a00640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f40800a2b80 0x7f40800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.770+0000 7f409a800640 1 --2- 172.21.15.2:0/2210405550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 0x55e560121660 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4094009fd0 tx=0x7f40940118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85f3689a4717e44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.770+0000 7f4092a00640 1 -- 172.21.15.2:0/1899579189 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f408c0cf870 con 0x7f409c101580 2024-09-10T08:14:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.770+0000 7f4099e00640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f40800a2b80 0x7f40800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.770+0000 7f4087400640 1 -- 172.21.15.2:0/2210405550 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094004d20 con 0x55e560121260 2024-09-10T08:14:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.770+0000 7f4087400640 1 -- 172.21.15.2:0/2210405550 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4094004050 con 0x55e560121260 2024-09-10T08:14:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.770+0000 7f4087400640 1 -- 172.21.15.2:0/2210405550 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094005260 con 0x55e560121260 2024-09-10T08:14:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.771+0000 7f40a4064740 1 -- 172.21.15.2:0/2210405550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 msgr2=0x55e560121660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.771+0000 7f4099e00640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f40800a2b80 0x7f40800a5040 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f408c0980e0 tx=0x7f408c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.771+0000 7f40a4064740 1 --2- 172.21.15.2:0/2210405550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 0x55e560121660 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4094009fd0 tx=0x7f40940118d0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.771+0000 7f40a4064740 1 -- 172.21.15.2:0/2210405550 shutdown_connections 2024-09-10T08:14:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.771+0000 7f40a4064740 1 --2- 172.21.15.2:0/2210405550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 0x55e5601290f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.771+0000 7f40a4064740 1 --2- 172.21.15.2:0/2210405550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e560126660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.771+0000 7f40a4064740 1 --2- 172.21.15.2:0/2210405550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 0x55e560121660 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.771+0000 7f40a4064740 1 -- 172.21.15.2:0/2210405550 >> 172.21.15.2:0/2210405550 conn(0x55e56011c8d0 msgr2=0x55e56011ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.771+0000 7f40a4064740 1 -- 172.21.15.2:0/2210405550 shutdown_connections 2024-09-10T08:14:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.771+0000 7f40a4064740 1 -- 172.21.15.2:0/2210405550 wait complete. 2024-09-10T08:14:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.772+0000 7f40a4064740 1 -- start start 2024-09-10T08:14:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.772+0000 7f40a4064740 1 Processor -- start 2024-09-10T08:14:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.772+0000 7f40a4064740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 0x55e56012f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.772+0000 7f40a4064740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e56012fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.772+0000 7f40a4064740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 0x55e560136d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.772+0000 7f40a4064740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e560129c00 con 0x55e560121260 2024-09-10T08:14:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.772+0000 7f40a4064740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e56000f090 con 0x55e560121ba0 2024-09-10T08:14:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f40a4064740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e560129d80 con 0x55e560126c30 2024-09-10T08:14:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f40a4064740 10 client.0 _ll_register_callbacks cb 0x55e560063f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f409a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 0x55e56012f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f409a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 0x55e56012f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57906/0 (socket says 172.21.15.2:57906) 2024-09-10T08:14:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f409a800640 1 -- 172.21.15.2:0/3125361362 learned_addr learned my addr 172.21.15.2:0/3125361362 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f4099400640 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e56012fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f409a800640 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 0x55e560136d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f409a800640 1 -- 172.21.15.2:0/3125361362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 msgr2=0x55e560136d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f409a800640 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 0x55e560136d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f409a800640 1 -- 172.21.15.2:0/3125361362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 msgr2=0x55e56012fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f409a800640 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e56012fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f409a800640 1 -- 172.21.15.2:0/3125361362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55fe65620 con 0x55e560121260 2024-09-10T08:14:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f4099400640 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e56012fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f409a800640 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 0x55e56012f7d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4094011dd0 tx=0x7f4094003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280e873403635d6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.773+0000 7f407be00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.774+0000 7f407be00640 1 -- 172.21.15.2:0/3125361362 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094004690 con 0x55e560121260 2024-09-10T08:14:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.774+0000 7f407be00640 1 -- 172.21.15.2:0/3125361362 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4094005040 con 0x55e560121260 2024-09-10T08:14:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.774+0000 7f40a4064740 1 -- 172.21.15.2:0/3125361362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4094004050 con 0x55e560121260 2024-09-10T08:14:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.774+0000 7f407be00640 1 -- 172.21.15.2:0/3125361362 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4094035050 con 0x55e560121260 2024-09-10T08:14:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.774+0000 7f40a4064740 10 client.17106 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.774+0000 7f40a4064740 1 -- 172.21.15.2:0/3125361362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e560139b40 con 0x55e560121260 2024-09-10T08:14:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.774+0000 7f407be00640 1 -- 172.21.15.2:0/3125361362 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4094004050 con 0x55e560121260 2024-09-10T08:14:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.774+0000 7f407be00640 1 -- 172.21.15.2:0/3125361362 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f409400f900 con 0x55e560121260 2024-09-10T08:14:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.775+0000 7f40a4064740 10 client.17106 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.775+0000 7f40a4064740 10 client.17106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.775+0000 7f40a4064740 20 client.17106 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.775+0000 7f40a4064740 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e56013b090 0x55e56013d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.775+0000 7f40a4064740 4 client.17106 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.775+0000 7f40a4064740 1 -- 172.21.15.2:0/3125361362 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e560093830 con 0x55e56013b090 2024-09-10T08:14:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.775+0000 7f4099400640 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e56013b090 0x55e56013d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.776+0000 7f4099400640 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e56013b090 0x55e56013d480 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee31f6f1ebd2fe6b server_cookie=af621d3a67542380 in_seq=0 out_seq=0 2024-09-10T08:14:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.776+0000 7f407be00640 10 client.17106 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f407be00640 1 -- 172.21.15.2:0/3125361362 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e560093830 con 0x55e56013b090 2024-09-10T08:14:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f407be00640 10 client.17106 handle_command_reply: tid=0 2024-09-10T08:14:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f40a4064740 1 client.17106 shutdown 2024-09-10T08:14:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f40a4064740 1 -- 172.21.15.2:0/3125361362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 msgr2=0x55e56012f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f40a4064740 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 0x55e56012f7d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4094011dd0 tx=0x7f4094003a30 comp rx=0 tx=0).stop 2024-09-10T08:14:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f40a4064740 1 -- 172.21.15.2:0/3125361362 shutdown_connections 2024-09-10T08:14:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f40a4064740 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e56013b090 0x55e56013d480 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f40a4064740 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 0x55e560136d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f40a4064740 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e56012fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f407be00640 0 client.17106 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f40a4064740 1 --2- 172.21.15.2:0/3125361362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121260 0x55e56012f7d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f40a4064740 1 -- 172.21.15.2:0/3125361362 >> 172.21.15.2:0/3125361362 conn(0x55e56011c8d0 msgr2=0x55e56011e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f40a4064740 1 -- 172.21.15.2:0/3125361362 shutdown_connections 2024-09-10T08:14:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f40a4064740 1 -- 172.21.15.2:0/3125361362 wait complete. 2024-09-10T08:14:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.777+0000 7f40a4064740 20 client.17106 trim_cache size 0 max 16384 2024-09-10T08:14:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.792+0000 7f40a4064740 1 Processor -- start 2024-09-10T08:14:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.793+0000 7f40a4064740 1 -- start start 2024-09-10T08:14:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.793+0000 7f40a4064740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121260 0x55e5601178d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.793+0000 7f40a4064740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121ba0 0x55e560136d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.793+0000 7f40a4064740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 0x55e56011ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.793+0000 7f40a4064740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1aae90 con 0x55e560121ba0 2024-09-10T08:14:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.793+0000 7f40a4064740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1a8760 con 0x55e560121260 2024-09-10T08:14:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.793+0000 7f40a4064740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1a65b0 con 0x55e560126c30 2024-09-10T08:14:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.793+0000 7f4099400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121ba0 0x55e560136d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.793+0000 7f4099400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121ba0 0x55e560136d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57920/0 (socket says 172.21.15.2:57920) 2024-09-10T08:14:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.793+0000 7f4099400640 1 -- 172.21.15.2:0/856217263 learned_addr learned my addr 172.21.15.2:0/856217263 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.793+0000 7f409a800640 1 --2- 172.21.15.2:0/856217263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 0x55e56011ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.793+0000 7f409a800640 1 --2- 172.21.15.2:0/856217263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121260 0x55e5601178d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.794+0000 7f409a800640 1 -- 172.21.15.2:0/856217263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121260 msgr2=0x55e5601178d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.794+0000 7f409a800640 1 --2- 172.21.15.2:0/856217263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121260 0x55e5601178d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.794+0000 7f409a800640 1 -- 172.21.15.2:0/856217263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121ba0 msgr2=0x55e560136d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.794+0000 7f409a800640 1 --2- 172.21.15.2:0/856217263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121ba0 0x55e560136d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.794+0000 7f409a800640 1 -- 172.21.15.2:0/856217263 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e56011c6f0 con 0x55e560126c30 2024-09-10T08:14:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.794+0000 7f409a800640 1 --2- 172.21.15.2:0/856217263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 0x55e56011ccb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4094000f80 tx=0x7f4094014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a724fb10bd2cbce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.794+0000 7f4086a00640 1 -- 172.21.15.2:0/856217263 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094009ea0 con 0x55e560126c30 2024-09-10T08:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.794+0000 7f4086a00640 1 -- 172.21.15.2:0/856217263 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f409402c070 con 0x55e560126c30 2024-09-10T08:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.794+0000 7f4086a00640 1 -- 172.21.15.2:0/856217263 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40940046f0 con 0x55e560126c30 2024-09-10T08:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.795+0000 7f40a4064740 1 -- 172.21.15.2:0/856217263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 msgr2=0x55e56011ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.795+0000 7f40a4064740 1 --2- 172.21.15.2:0/856217263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 0x55e56011ccb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4094000f80 tx=0x7f4094014040 comp rx=0 tx=0).stop 2024-09-10T08:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.795+0000 7f40a4064740 1 -- 172.21.15.2:0/856217263 shutdown_connections 2024-09-10T08:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.795+0000 7f40a4064740 1 --2- 172.21.15.2:0/856217263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560126c30 0x55e56011ccb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.795+0000 7f40a4064740 1 --2- 172.21.15.2:0/856217263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560121ba0 0x55e560136d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.795+0000 7f40a4064740 1 --2- 172.21.15.2:0/856217263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121260 0x55e5601178d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.795+0000 7f40a4064740 1 -- 172.21.15.2:0/856217263 >> 172.21.15.2:0/856217263 conn(0x55e56011c8d0 msgr2=0x55e56012f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.795+0000 7f40a4064740 1 -- 172.21.15.2:0/856217263 shutdown_connections 2024-09-10T08:14:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.795+0000 7f40a4064740 1 -- 172.21.15.2:0/856217263 wait complete. 2024-09-10T08:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.795+0000 7f40a4064740 1 -- start start 2024-09-10T08:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.795+0000 7f40a4064740 1 Processor -- start 2024-09-10T08:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.796+0000 7f40a4064740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560121260 0x55e560133a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.796+0000 7f40a4064740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e560133fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.796+0000 7f40a4064740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560126c30 0x55e5601a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.796+0000 7f40a4064740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1a6c20 con 0x55e560126c30 2024-09-10T08:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.796+0000 7f40a4064740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1a8090 con 0x55e560121ba0 2024-09-10T08:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.796+0000 7f40a4064740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1a72f0 con 0x55e560121260 2024-09-10T08:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.796+0000 7f40a4064740 10 client.0 _ll_register_callbacks cb 0x7f408c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.796+0000 7f409a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560126c30 0x55e5601a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.796+0000 7f409a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560126c30 0x55e5601a3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57922/0 (socket says 172.21.15.2:57922) 2024-09-10T08:14:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.796+0000 7f409a800640 1 -- 172.21.15.2:0/2935296309 learned_addr learned my addr 172.21.15.2:0/2935296309 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.796+0000 7f409a800640 1 --2- 172.21.15.2:0/2935296309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560121260 0x55e560133a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.796+0000 7f4099400640 1 --2- 172.21.15.2:0/2935296309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e560133fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.797+0000 7f409a800640 1 -- 172.21.15.2:0/2935296309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560121260 msgr2=0x55e560133a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.797+0000 7f409a800640 1 --2- 172.21.15.2:0/2935296309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560121260 0x55e560133a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.797+0000 7f409a800640 1 -- 172.21.15.2:0/2935296309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 msgr2=0x55e560133fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.797+0000 7f409a800640 1 --2- 172.21.15.2:0/2935296309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e560133fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.797+0000 7f409a800640 1 -- 172.21.15.2:0/2935296309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e560134620 con 0x55e560126c30 2024-09-10T08:14:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.797+0000 7f409a800640 1 --2- 172.21.15.2:0/2935296309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560126c30 0x55e5601a3d30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x55e560133040 tx=0x7f4094003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03769346add5aaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.797+0000 7f407be00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.797+0000 7f407be00640 1 -- 172.21.15.2:0/2935296309 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094011dd0 con 0x55e560126c30 2024-09-10T08:14:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.797+0000 7f407be00640 1 -- 172.21.15.2:0/2935296309 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4094004490 con 0x55e560126c30 2024-09-10T08:14:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.797+0000 7f40a4064740 1 -- 172.21.15.2:0/2935296309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f409402c070 con 0x55e560126c30 2024-09-10T08:14:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.797+0000 7f407be00640 1 -- 172.21.15.2:0/2935296309 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4094009b30 con 0x55e560126c30 2024-09-10T08:14:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.797+0000 7f40a4064740 10 client.17112 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.797+0000 7f40a4064740 1 -- 172.21.15.2:0/2935296309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5601a5380 con 0x55e560126c30 2024-09-10T08:14:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.798+0000 7f407be00640 1 -- 172.21.15.2:0/2935296309 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f409402c070 con 0x55e560126c30 2024-09-10T08:14:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.798+0000 7f407be00640 1 -- 172.21.15.2:0/2935296309 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4094019b70 con 0x55e560126c30 2024-09-10T08:14:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.798+0000 7f40a4064740 10 client.17112 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.798+0000 7f40a4064740 10 client.17112 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.798+0000 7f40a4064740 20 client.17112 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.798+0000 7f40a4064740 1 --2- 172.21.15.2:0/2935296309 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e5601a5c90 0x55e5601a80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.798+0000 7f40a4064740 4 client.17112 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:14:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.798+0000 7f40a4064740 1 -- 172.21.15.2:0/2935296309 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e56008db80 con 0x55e5601a5c90 2024-09-10T08:14:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.798+0000 7f4099400640 1 --2- 172.21.15.2:0/2935296309 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e5601a5c90 0x55e5601a80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.799+0000 7f4099400640 1 --2- 172.21.15.2:0/2935296309 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e5601a5c90 0x55e5601a80b0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3727b002408ae419 server_cookie=32038903c4235dc2 in_seq=0 out_seq=0 2024-09-10T08:14:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.799+0000 7f407be00640 10 client.17112 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f407be00640 1 -- 172.21.15.2:0/2935296309 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e56008db80 con 0x55e5601a5c90 2024-09-10T08:14:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f407be00640 10 client.17112 handle_command_reply: tid=0 2024-09-10T08:14:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f40a4064740 1 client.17112 shutdown 2024-09-10T08:14:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f40a4064740 1 -- 172.21.15.2:0/2935296309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560126c30 msgr2=0x55e5601a3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f40a4064740 1 --2- 172.21.15.2:0/2935296309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560126c30 0x55e5601a3d30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x55e560133040 tx=0x7f4094003a30 comp rx=0 tx=0).stop 2024-09-10T08:14:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f40a4064740 1 -- 172.21.15.2:0/2935296309 shutdown_connections 2024-09-10T08:14:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f40a4064740 1 --2- 172.21.15.2:0/2935296309 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e5601a5c90 0x55e5601a80b0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f40a4064740 1 --2- 172.21.15.2:0/2935296309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e560126c30 0x55e5601a3d30 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f40a4064740 1 --2- 172.21.15.2:0/2935296309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e560121ba0 0x55e560133fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f40a4064740 1 --2- 172.21.15.2:0/2935296309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e560121260 0x55e560133a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f40a4064740 1 -- 172.21.15.2:0/2935296309 >> 172.21.15.2:0/2935296309 conn(0x55e56011c8d0 msgr2=0x55e56011b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f407be00640 0 client.17112 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f40a4064740 1 -- 172.21.15.2:0/2935296309 shutdown_connections 2024-09-10T08:14:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f40a4064740 1 -- 172.21.15.2:0/2935296309 wait complete. 2024-09-10T08:14:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.813+0000 7f40a4064740 20 client.17112 trim_cache size 0 max 16384 2024-09-10T08:14:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:14:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:14:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:14:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:14:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.814+0000 7f409b200640 1 -- 172.21.15.2:0/1899579189 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f40800a2b80 msgr2=0x7f40800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.814+0000 7f409b200640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f40800a2b80 0x7f40800a5040 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f408c0980e0 tx=0x7f408c000bc0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.814+0000 7f409b200640 1 -- 172.21.15.2:0/1899579189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f409c101580 msgr2=0x7f409c197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.814+0000 7f409b200640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f409c101580 0x7f409c197dd0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f408c00e9c0 tx=0x7f408c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:14:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.814+0000 7f409b200640 1 -- 172.21.15.2:0/1899579189 shutdown_connections 2024-09-10T08:14:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.814+0000 7f409b200640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f40800a2b80 0x7f40800a5040 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.814+0000 7f409b200640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409c103050 0x7f409c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.814+0000 7f409b200640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f409c101f40 0x7f409c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.814+0000 7f409b200640 1 --2- 172.21.15.2:0/1899579189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f409c101580 0x7f409c197dd0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.814+0000 7f409b200640 1 -- 172.21.15.2:0/1899579189 >> 172.21.15.2:0/1899579189 conn(0x7f409c0fd0d0 msgr2=0x7f409c105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.814+0000 7f409b200640 1 -- 172.21.15.2:0/1899579189 shutdown_connections 2024-09-10T08:14:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:25.814+0000 7f409b200640 1 -- 172.21.15.2:0/1899579189 wait complete. 2024-09-10T08:14:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.947+0000 7f0b0aa00640 1 -- 172.21.15.2:0/3473741441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b04074620 msgr2=0x7f0b04074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.947+0000 7f0b0aa00640 1 --2- 172.21.15.2:0/3473741441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b04074620 0x7f0b04074a00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009900 tx=0x7f0af402a480 comp rx=0 tx=0).stop 2024-09-10T08:14:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.948+0000 7f0b0aa00640 1 -- 172.21.15.2:0/3473741441 shutdown_connections 2024-09-10T08:14:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.948+0000 7f0b0aa00640 1 --2- 172.21.15.2:0/3473741441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b0410e3c0 0x7f0b04110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.948+0000 7f0b0aa00640 1 --2- 172.21.15.2:0/3473741441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b04074f40 0x7f0b0410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.948+0000 7f0b0aa00640 1 --2- 172.21.15.2:0/3473741441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b04074620 0x7f0b04074a00 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.948+0000 7f0b0aa00640 1 -- 172.21.15.2:0/3473741441 >> 172.21.15.2:0/3473741441 conn(0x7f0b040fd8c0 msgr2=0x7f0b040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.948+0000 7f0b0aa00640 1 -- 172.21.15.2:0/3473741441 shutdown_connections 2024-09-10T08:14:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.948+0000 7f0b0aa00640 1 -- 172.21.15.2:0/3473741441 wait complete. 2024-09-10T08:14:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.949+0000 7f0b0aa00640 1 Processor -- start 2024-09-10T08:14:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.949+0000 7f0b0aa00640 1 -- start start 2024-09-10T08:14:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.949+0000 7f0b0aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b04074620 0x7f0b0419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.949+0000 7f0b0aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b04074f40 0x7f0b0419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.949+0000 7f0b0aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b0410e3c0 0x7f0b0419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.949+0000 7f0b0aa00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041141c0 con 0x7f0b04074620 2024-09-10T08:14:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.949+0000 7f0b0aa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b04114040 con 0x7f0b0410e3c0 2024-09-10T08:14:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.949+0000 7f0b0aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0b04114340 con 0x7f0b04074f40 2024-09-10T08:14:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.949+0000 7f0b03e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b04074620 0x7f0b0419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.950+0000 7f0afb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b04074f40 0x7f0b0419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.950+0000 7f0b08a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b0410e3c0 0x7f0b0419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.950+0000 7f0afb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b04074f40 0x7f0b0419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:32988/0 (socket says 172.21.15.2:32988) 2024-09-10T08:14:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.950+0000 7f0b03e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b04074620 0x7f0b0419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57946/0 (socket says 172.21.15.2:57946) 2024-09-10T08:14:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.950+0000 7f0afb400640 1 -- 172.21.15.2:0/3086761788 learned_addr learned my addr 172.21.15.2:0/3086761788 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.950+0000 7f0afb400640 1 -- 172.21.15.2:0/3086761788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b0410e3c0 msgr2=0x7f0b0419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.950+0000 7f0afb400640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b0410e3c0 0x7f0b0419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.950+0000 7f0afb400640 1 -- 172.21.15.2:0/3086761788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b04074620 msgr2=0x7f0b0419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.950+0000 7f0afb400640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b04074620 0x7f0b0419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.950+0000 7f0afb400640 1 -- 172.21.15.2:0/3086761788 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0419f610 con 0x7f0b04074f40 2024-09-10T08:14:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.950+0000 7f0b08a00640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b0410e3c0 0x7f0b0419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:25.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.950+0000 7f0b03e00640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b04074620 0x7f0b0419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:25.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.950+0000 7f0afb400640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b04074f40 0x7f0b0419aba0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00d950 tx=0x7f0aec00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:25.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.951+0000 7f0b01600640 1 -- 172.21.15.2:0/3086761788 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec017030 con 0x7f0b04074f40 2024-09-10T08:14:25.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.951+0000 7f0b01600640 1 -- 172.21.15.2:0/3086761788 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aec004020 con 0x7f0b04074f40 2024-09-10T08:14:25.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.951+0000 7f0b01600640 1 -- 172.21.15.2:0/3086761788 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec0047a0 con 0x7f0b04074f40 2024-09-10T08:14:25.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.951+0000 7f0b0aa00640 1 -- 172.21.15.2:0/3086761788 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0419f900 con 0x7f0b04074f40 2024-09-10T08:14:25.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.951+0000 7f0b0aa00640 1 -- 172.21.15.2:0/3086761788 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b041a71e0 con 0x7f0b04074f40 2024-09-10T08:14:25.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.955+0000 7f0b0d415740 1 Processor -- start 2024-09-10T08:14:25.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.955+0000 7f0b0d415740 1 -- start start 2024-09-10T08:14:25.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.955+0000 7f0b0d415740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79dab90 0x55b2e79daf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.955+0000 7f0b0d415740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79e5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b0d415740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79e5d70 0x55b2e79e8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b0d415740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b2e79413e0 con 0x55b2e79db4b0 2024-09-10T08:14:25.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b0d415740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b2e78e8d80 con 0x55b2e79dab90 2024-09-10T08:14:25.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b0d415740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b2e793d3a0 con 0x55b2e79e5d70 2024-09-10T08:14:25.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b03e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79e5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b08a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79dab90 0x55b2e79daf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b03e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79e5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57960/0 (socket says 172.21.15.2:57960) 2024-09-10T08:14:25.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b08a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79dab90 0x55b2e79daf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:43614/0 (socket says 172.21.15.2:43614) 2024-09-10T08:14:25.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b03e00640 1 -- 172.21.15.2:0/2270013478 learned_addr learned my addr 172.21.15.2:0/2270013478 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:25.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b01600640 1 -- 172.21.15.2:0/3086761788 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0aec0041d0 con 0x7f0b04074f40 2024-09-10T08:14:25.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b08a00640 1 -- 172.21.15.2:0/2270013478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79e5d70 msgr2=0x55b2e79e8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b08a00640 1 --2- 172.21.15.2:0/2270013478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79e5d70 0x55b2e79e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b08a00640 1 -- 172.21.15.2:0/2270013478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 msgr2=0x55b2e79e5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b08a00640 1 --2- 172.21.15.2:0/2270013478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79e5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.956+0000 7f0b08a00640 1 -- 172.21.15.2:0/2270013478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e7907ae0 con 0x55b2e79dab90 2024-09-10T08:14:25.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.957+0000 7f0b08a00640 1 --2- 172.21.15.2:0/2270013478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79dab90 0x55b2e79daf70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0afc010710 tx=0x7f0afc010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5119e0afaf761b04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:25.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.957+0000 7f0af9200640 1 -- 172.21.15.2:0/2270013478 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc004b20 con 0x55b2e79dab90 2024-09-10T08:14:25.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.957+0000 7f0af9200640 1 -- 172.21.15.2:0/2270013478 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0afc00edb0 con 0x55b2e79dab90 2024-09-10T08:14:25.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.957+0000 7f0af9200640 1 -- 172.21.15.2:0/2270013478 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc005670 con 0x55b2e79dab90 2024-09-10T08:14:25.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.957+0000 7f0b01600640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0ae80a2c00 0x7f0ae80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.958+0000 7f0b0d415740 1 -- 172.21.15.2:0/2270013478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79dab90 msgr2=0x55b2e79daf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.958+0000 7f0b0d415740 1 --2- 172.21.15.2:0/2270013478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79dab90 0x55b2e79daf70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0afc010710 tx=0x7f0afc010bd0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.958+0000 7f0b01600640 1 -- 172.21.15.2:0/3086761788 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0aec0ced50 con 0x7f0b04074f40 2024-09-10T08:14:25.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.958+0000 7f0b0d415740 1 -- 172.21.15.2:0/2270013478 shutdown_connections 2024-09-10T08:14:25.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.958+0000 7f0b0d415740 1 --2- 172.21.15.2:0/2270013478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79e5d70 0x55b2e79e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.958+0000 7f0b0d415740 1 --2- 172.21.15.2:0/2270013478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79e5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.958+0000 7f0b0d415740 1 --2- 172.21.15.2:0/2270013478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79dab90 0x55b2e79daf70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.958+0000 7f0b0d415740 1 -- 172.21.15.2:0/2270013478 >> 172.21.15.2:0/2270013478 conn(0x55b2e79d7120 msgr2=0x55b2e79d7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:25.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.958+0000 7f0b03e00640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0ae80a2c00 0x7f0ae80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.958+0000 7f0b0d415740 1 -- 172.21.15.2:0/2270013478 shutdown_connections 2024-09-10T08:14:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.958+0000 7f0b0d415740 1 -- 172.21.15.2:0/2270013478 wait complete. 2024-09-10T08:14:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.959+0000 7f0b0d415740 1 -- start start 2024-09-10T08:14:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.959+0000 7f0b03e00640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0ae80a2c00 0x7f0ae80a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55b2e79d7f80 tx=0x7f0af4031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.959+0000 7f0b0d415740 1 Processor -- start 2024-09-10T08:14:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.959+0000 7f0b0d415740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79dab90 0x55b2e79e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.959+0000 7f0b0d415740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79e3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.959+0000 7f0b0d415740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79e5d70 0x55b2e79e3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.959+0000 7f0b0d415740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b2e79e89e0 con 0x55b2e79db4b0 2024-09-10T08:14:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.959+0000 7f0b0d415740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b2e78fc0f0 con 0x55b2e79e5d70 2024-09-10T08:14:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.959+0000 7f0b0d415740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b2e79e8b60 con 0x55b2e79dab90 2024-09-10T08:14:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.959+0000 7f0b0d415740 10 client.0 _ll_register_callbacks cb 0x55b2e78ed9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.960+0000 7f0afb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.960+0000 7f0afb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79e3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57962/0 (socket says 172.21.15.2:57962) 2024-09-10T08:14:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.960+0000 7f0afb400640 1 -- 172.21.15.2:0/733522038 learned_addr learned my addr 172.21.15.2:0/733522038 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.960+0000 7f0b03e00640 1 --2- 172.21.15.2:0/733522038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79dab90 0x55b2e79e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.960+0000 7f0b08a00640 1 --2- 172.21.15.2:0/733522038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79e5d70 0x55b2e79e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.960+0000 7f0afb400640 1 -- 172.21.15.2:0/733522038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79dab90 msgr2=0x55b2e79e2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.960+0000 7f0afb400640 1 --2- 172.21.15.2:0/733522038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79dab90 0x55b2e79e2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.960+0000 7f0afb400640 1 -- 172.21.15.2:0/733522038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79e5d70 msgr2=0x55b2e79e3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.960+0000 7f0afb400640 1 --2- 172.21.15.2:0/733522038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79e5d70 0x55b2e79e3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.960+0000 7f0afb400640 1 -- 172.21.15.2:0/733522038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e76b23c0 con 0x55b2e79db4b0 2024-09-10T08:14:25.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.960+0000 7f0afb400640 1 --2- 172.21.15.2:0/733522038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79e3200 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0a38a0 tx=0x7f0aec09ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e952579494bbb4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:25.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.960+0000 7f0af1600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:25.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.961+0000 7f0af1600640 1 -- 172.21.15.2:0/733522038 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec09f510 con 0x55b2e79db4b0 2024-09-10T08:14:25.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.961+0000 7f0af1600640 1 -- 172.21.15.2:0/733522038 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0aec0bb070 con 0x55b2e79db4b0 2024-09-10T08:14:25.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.961+0000 7f0b0d415740 1 -- 172.21.15.2:0/733522038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0afc00edb0 con 0x55b2e79db4b0 2024-09-10T08:14:25.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.961+0000 7f0af1600640 1 -- 172.21.15.2:0/733522038 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0aec0a3dd0 con 0x55b2e79db4b0 2024-09-10T08:14:25.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.961+0000 7f0b0d415740 10 client.17124 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:25.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.961+0000 7f0b0d415740 1 -- 172.21.15.2:0/733522038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b2e79f4740 con 0x55b2e79db4b0 2024-09-10T08:14:25.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.961+0000 7f0af1600640 1 -- 172.21.15.2:0/733522038 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0afc00edb0 con 0x55b2e79db4b0 2024-09-10T08:14:25.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.961+0000 7f0af1600640 1 -- 172.21.15.2:0/733522038 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0aec09a5b0 con 0x55b2e79db4b0 2024-09-10T08:14:25.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.962+0000 7f0b0d415740 10 client.17124 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:25.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.962+0000 7f0b0d415740 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:25.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.962+0000 7f0b0d415740 20 client.17124 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:25.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.962+0000 7f0b0d415740 1 --2- 172.21.15.2:0/733522038 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b2e79f5c00 0x55b2e79f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.962+0000 7f0b0d415740 4 client.17124 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:25.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.962+0000 7f0b0d415740 1 -- 172.21.15.2:0/733522038 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2e794e950 con 0x55b2e79f5c00 2024-09-10T08:14:25.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.962+0000 7f0b03e00640 1 --2- 172.21.15.2:0/733522038 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b2e79f5c00 0x55b2e79f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.963+0000 7f0b03e00640 1 --2- 172.21.15.2:0/733522038 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b2e79f5c00 0x55b2e79f7ff0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b9b57ddb8dfc735 server_cookie=3d6bec96fb446af3 in_seq=0 out_seq=0 2024-09-10T08:14:25.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.963+0000 7f0af1600640 10 client.17124 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:25.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.963+0000 7f0af1600640 1 -- 172.21.15.2:0/733522038 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2e794e950 con 0x55b2e79f5c00 2024-09-10T08:14:25.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.963+0000 7f0af1600640 10 client.17124 handle_command_reply: tid=0 2024-09-10T08:14:25.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.963+0000 7f0b0d415740 1 client.17124 shutdown 2024-09-10T08:14:25.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.964+0000 7f0b0d415740 1 -- 172.21.15.2:0/733522038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 msgr2=0x55b2e79e3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.964+0000 7f0b0d415740 1 --2- 172.21.15.2:0/733522038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79e3200 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0a38a0 tx=0x7f0aec09ad60 comp rx=0 tx=0).stop 2024-09-10T08:14:25.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.964+0000 7f0b0d415740 1 -- 172.21.15.2:0/733522038 shutdown_connections 2024-09-10T08:14:25.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.964+0000 7f0b0d415740 1 --2- 172.21.15.2:0/733522038 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b2e79f5c00 0x55b2e79f7ff0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.964+0000 7f0b0d415740 1 --2- 172.21.15.2:0/733522038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79e5d70 0x55b2e79e3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.964+0000 7f0b0d415740 1 --2- 172.21.15.2:0/733522038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79e3200 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.964+0000 7f0b0d415740 1 --2- 172.21.15.2:0/733522038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79dab90 0x55b2e79e2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.964+0000 7f0af1600640 0 client.17124 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:25.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.964+0000 7f0b0d415740 1 -- 172.21.15.2:0/733522038 >> 172.21.15.2:0/733522038 conn(0x55b2e79d7120 msgr2=0x55b2e79dce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:25.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.964+0000 7f0b0d415740 1 -- 172.21.15.2:0/733522038 shutdown_connections 2024-09-10T08:14:25.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.964+0000 7f0b0d415740 1 -- 172.21.15.2:0/733522038 wait complete. 2024-09-10T08:14:25.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.964+0000 7f0b0d415740 20 client.17124 trim_cache size 0 max 16384 2024-09-10T08:14:25.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.982+0000 7f0b0d415740 1 Processor -- start 2024-09-10T08:14:25.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.982+0000 7f0b0d415740 1 -- start start 2024-09-10T08:14:25.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b0d415740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79dab90 0x55b2e79dccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b0d415740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79db4b0 0x55b2e79fd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b0d415740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79e5d70 0x55b2e79fddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b0d415740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041b3890 con 0x55b2e79dab90 2024-09-10T08:14:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b0d415740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041b1160 con 0x55b2e79e5d70 2024-09-10T08:14:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b0d415740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041aefb0 con 0x55b2e79db4b0 2024-09-10T08:14:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b03e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79dab90 0x55b2e79dccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b03e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79dab90 0x55b2e79dccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57974/0 (socket says 172.21.15.2:57974) 2024-09-10T08:14:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b03e00640 1 -- 172.21.15.2:0/1274169571 learned_addr learned my addr 172.21.15.2:0/1274169571 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b08a00640 1 --2- 172.21.15.2:0/1274169571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79e5d70 0x55b2e79fddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0afb400640 1 --2- 172.21.15.2:0/1274169571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79db4b0 0x55b2e79fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b03e00640 1 -- 172.21.15.2:0/1274169571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79db4b0 msgr2=0x55b2e79fd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b03e00640 1 --2- 172.21.15.2:0/1274169571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79db4b0 0x55b2e79fd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b03e00640 1 -- 172.21.15.2:0/1274169571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79e5d70 msgr2=0x55b2e79fddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b03e00640 1 --2- 172.21.15.2:0/1274169571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79e5d70 0x55b2e79fddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.983+0000 7f0b03e00640 1 -- 172.21.15.2:0/1274169571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e79d81d0 con 0x55b2e79dab90 2024-09-10T08:14:25.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.984+0000 7f0b03e00640 1 --2- 172.21.15.2:0/1274169571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79dab90 0x55b2e79dccf0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0af402a9a0 tx=0x7f0af403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12856e2aa1f1ec6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:25.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.984+0000 7f0af3e00640 1 -- 172.21.15.2:0/1274169571 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4035040 con 0x55b2e79dab90 2024-09-10T08:14:25.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.984+0000 7f0af3e00640 1 -- 172.21.15.2:0/1274169571 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af4031e40 con 0x55b2e79dab90 2024-09-10T08:14:25.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.984+0000 7f0af3e00640 1 -- 172.21.15.2:0/1274169571 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af40518f0 con 0x55b2e79dab90 2024-09-10T08:14:25.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.985+0000 7f0b0d415740 1 -- 172.21.15.2:0/1274169571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79dab90 msgr2=0x55b2e79dccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.985+0000 7f0b0d415740 1 --2- 172.21.15.2:0/1274169571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79dab90 0x55b2e79dccf0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0af402a9a0 tx=0x7f0af403f040 comp rx=0 tx=0).stop 2024-09-10T08:14:25.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.985+0000 7f0b0d415740 1 -- 172.21.15.2:0/1274169571 shutdown_connections 2024-09-10T08:14:25.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.985+0000 7f0b0d415740 1 --2- 172.21.15.2:0/1274169571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79e5d70 0x55b2e79fddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.985+0000 7f0b0d415740 1 --2- 172.21.15.2:0/1274169571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79db4b0 0x55b2e79fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.985+0000 7f0b0d415740 1 --2- 172.21.15.2:0/1274169571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79dab90 0x55b2e79dccf0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.985+0000 7f0b0d415740 1 -- 172.21.15.2:0/1274169571 >> 172.21.15.2:0/1274169571 conn(0x55b2e79d7120 msgr2=0x55b2e79dc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:25.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.985+0000 7f0b0d415740 1 -- 172.21.15.2:0/1274169571 shutdown_connections 2024-09-10T08:14:25.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.985+0000 7f0b0d415740 1 -- 172.21.15.2:0/1274169571 wait complete. 2024-09-10T08:14:25.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.985+0000 7f0b0d415740 1 -- start start 2024-09-10T08:14:25.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.986+0000 7f0b0d415740 1 Processor -- start 2024-09-10T08:14:25.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.986+0000 7f0b0d415740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79dab90 0x55b2e79ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.986+0000 7f0b0d415740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79ea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.986+0000 7f0b0d415740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79e5d70 0x55b2e7a5f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.986+0000 7f0b0d415740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041af620 con 0x55b2e79db4b0 2024-09-10T08:14:25.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.986+0000 7f0b0d415740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041b0a90 con 0x55b2e79dab90 2024-09-10T08:14:25.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.986+0000 7f0b0d415740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041afcf0 con 0x55b2e79e5d70 2024-09-10T08:14:25.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.986+0000 7f0b0d415740 10 client.0 _ll_register_callbacks cb 0x7f0aec097400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:25.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.986+0000 7f0afb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.986+0000 7f0b08a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79e5d70 0x55b2e7a5f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.986+0000 7f0b03e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79dab90 0x55b2e79ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.986+0000 7f0b08a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79e5d70 0x55b2e7a5f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:33018/0 (socket says 172.21.15.2:33018) 2024-09-10T08:14:25.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.986+0000 7f0b08a00640 1 -- 172.21.15.2:0/4160373789 learned_addr learned my addr 172.21.15.2:0/4160373789 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:25.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.987+0000 7f0afb400640 1 -- 172.21.15.2:0/4160373789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79e5d70 msgr2=0x55b2e7a5f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.987+0000 7f0afb400640 1 --2- 172.21.15.2:0/4160373789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79e5d70 0x55b2e7a5f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.987+0000 7f0afb400640 1 -- 172.21.15.2:0/4160373789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79dab90 msgr2=0x55b2e79ea2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:25.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.987+0000 7f0afb400640 1 --2- 172.21.15.2:0/4160373789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79dab90 0x55b2e79ea2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.987+0000 7f0afb400640 1 -- 172.21.15.2:0/4160373789 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e7a5fe50 con 0x55b2e79db4b0 2024-09-10T08:14:25.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.987+0000 7f0b03e00640 1 --2- 172.21.15.2:0/4160373789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79dab90 0x55b2e79ea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:25.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.987+0000 7f0afb400640 1 --2- 172.21.15.2:0/4160373789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79ea810 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x55b2e79e4270 tx=0x7f0aec09add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabce096e189a182 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:25.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.987+0000 7f0af1600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:25.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.987+0000 7f0af1600640 1 -- 172.21.15.2:0/4160373789 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec09a7b0 con 0x55b2e79db4b0 2024-09-10T08:14:25.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.987+0000 7f0af1600640 1 -- 172.21.15.2:0/4160373789 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0aec096eb0 con 0x55b2e79db4b0 2024-09-10T08:14:25.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.987+0000 7f0b0d415740 1 -- 172.21.15.2:0/4160373789 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b2e7948e60 con 0x55b2e79db4b0 2024-09-10T08:14:25.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.988+0000 7f0af1600640 1 -- 172.21.15.2:0/4160373789 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0aec09f510 con 0x55b2e79db4b0 2024-09-10T08:14:25.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.988+0000 7f0b0d415740 10 client.17130 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:25.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.988+0000 7f0b0d415740 1 -- 172.21.15.2:0/4160373789 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b2e7a61120 con 0x55b2e79db4b0 2024-09-10T08:14:25.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.988+0000 7f0af1600640 1 -- 172.21.15.2:0/4160373789 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b2e7948e60 con 0x55b2e79db4b0 2024-09-10T08:14:25.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.988+0000 7f0af1600640 1 -- 172.21.15.2:0/4160373789 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0aec0970b0 con 0x55b2e79db4b0 2024-09-10T08:14:25.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.988+0000 7f0b0d415740 10 client.17130 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:25.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.988+0000 7f0b0d415740 10 client.17130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:25.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.988+0000 7f0b0d415740 20 client.17130 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:25.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.988+0000 7f0b0d415740 1 --2- 172.21.15.2:0/4160373789 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b2e7a61710 0x55b2e7a63b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:25.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.988+0000 7f0b0d415740 4 client.17130 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:14:25.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.988+0000 7f0b0d415740 1 -- 172.21.15.2:0/4160373789 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2e78c50c0 con 0x55b2e7a61710 2024-09-10T08:14:25.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.989+0000 7f0b03e00640 1 --2- 172.21.15.2:0/4160373789 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b2e7a61710 0x55b2e7a63b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:25.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.989+0000 7f0b03e00640 1 --2- 172.21.15.2:0/4160373789 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b2e7a61710 0x55b2e7a63b30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b696e93e0439bf server_cookie=6282c184e2d46b17 in_seq=0 out_seq=0 2024-09-10T08:14:25.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.989+0000 7f0af1600640 10 client.17130 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:25.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.989+0000 7f0af1600640 1 -- 172.21.15.2:0/4160373789 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b2e78c50c0 con 0x55b2e7a61710 2024-09-10T08:14:25.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.989+0000 7f0af1600640 10 client.17130 handle_command_reply: tid=0 2024-09-10T08:14:25.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.989+0000 7f0b0d415740 1 client.17130 shutdown 2024-09-10T08:14:25.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.990+0000 7f0b0d415740 1 -- 172.21.15.2:0/4160373789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 msgr2=0x55b2e79ea810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.990+0000 7f0b0d415740 1 --2- 172.21.15.2:0/4160373789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79ea810 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x55b2e79e4270 tx=0x7f0aec09add0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.990+0000 7f0b0d415740 1 -- 172.21.15.2:0/4160373789 shutdown_connections 2024-09-10T08:14:25.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.990+0000 7f0b0d415740 1 --2- 172.21.15.2:0/4160373789 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b2e7a61710 0x55b2e7a63b30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.990+0000 7f0b0d415740 1 --2- 172.21.15.2:0/4160373789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b2e79e5d70 0x55b2e7a5f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.990+0000 7f0b0d415740 1 --2- 172.21.15.2:0/4160373789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2e79db4b0 0x55b2e79ea810 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.990+0000 7f0b0d415740 1 --2- 172.21.15.2:0/4160373789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2e79dab90 0x55b2e79ea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.990+0000 7f0b0d415740 1 -- 172.21.15.2:0/4160373789 >> 172.21.15.2:0/4160373789 conn(0x55b2e79d7120 msgr2=0x55b2e79e34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:25.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.990+0000 7f0af1600640 0 client.17130 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:25.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.990+0000 7f0b0d415740 1 -- 172.21.15.2:0/4160373789 shutdown_connections 2024-09-10T08:14:25.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.990+0000 7f0b0d415740 1 -- 172.21.15.2:0/4160373789 wait complete. 2024-09-10T08:14:25.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.990+0000 7f0b0d415740 20 client.17130 trim_cache size 0 max 16384 2024-09-10T08:14:25.996 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:14:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:14:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:14:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "6ceb7d3d-b060-43ee-b343-22d00267936e": { 2024-09-10T08:14:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:14:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "6ceb7d3d-b060-43ee-b343-22d00267936e", 2024-09-10T08:14:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:14:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:25.997 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:14:25.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.991+0000 7f0b0aa00640 1 -- 172.21.15.2:0/3086761788 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0ae80a2c00 msgr2=0x7f0ae80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.991+0000 7f0b0aa00640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0ae80a2c00 0x7f0ae80a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55b2e79d7f80 tx=0x7f0af4031510 comp rx=0 tx=0).stop 2024-09-10T08:14:25.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.991+0000 7f0b0aa00640 1 -- 172.21.15.2:0/3086761788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b04074f40 msgr2=0x7f0b0419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:25.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.991+0000 7f0b0aa00640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b04074f40 0x7f0b0419aba0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00d950 tx=0x7f0aec00de10 comp rx=0 tx=0).stop 2024-09-10T08:14:25.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.991+0000 7f0b0aa00640 1 -- 172.21.15.2:0/3086761788 shutdown_connections 2024-09-10T08:14:25.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.991+0000 7f0b0aa00640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b0410e3c0 0x7f0b0419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.991+0000 7f0b0aa00640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0ae80a2c00 0x7f0ae80a50c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.991+0000 7f0b0aa00640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b04074f40 0x7f0b0419aba0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.991+0000 7f0b0aa00640 1 --2- 172.21.15.2:0/3086761788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b04074620 0x7f0b0419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:25.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.991+0000 7f0b0aa00640 1 -- 172.21.15.2:0/3086761788 >> 172.21.15.2:0/3086761788 conn(0x7f0b040fd8c0 msgr2=0x7f0b040fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:25.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.991+0000 7f0b0aa00640 1 -- 172.21.15.2:0/3086761788 shutdown_connections 2024-09-10T08:14:25.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:25.991+0000 7f0b0aa00640 1 -- 172.21.15.2:0/3086761788 wait complete. 2024-09-10T08:14:26.792 INFO:tasks.cephfs.filesystem:scrub status for tag:6ceb7d3d-b060-43ee-b343-22d00267936e - {'path': '/', 'tag': '6ceb7d3d-b060-43ee-b343-22d00267936e', 'options': 'recursive,force'} 2024-09-10T08:14:27.121 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:26 smithi002 ceph-mon[30563]: pgmap v576: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:14:27.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:26 smithi107 ceph-mon[35048]: pgmap v576: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:14:27.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:26 smithi155 ceph-mon[35405]: pgmap v576: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:14:29.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:28 smithi107 ceph-mon[35048]: pgmap v577: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:14:29.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:28 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:14:29.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:28 smithi002 ceph-mon[30563]: pgmap v577: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:14:29.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:28 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:14:29.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:28 smithi155 ceph-mon[35405]: pgmap v577: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:14:29.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:28 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:14:30.957 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T08:14:30.957 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T08:14:30.957 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:14:31.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:30 smithi107 ceph-mon[35048]: pgmap v578: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:14:31.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:30 smithi002 ceph-mon[30563]: pgmap v578: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:14:31.347 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:31.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:30 smithi155 ceph-mon[35405]: pgmap v578: 129 pgs: 129 active+clean; 3.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:14:31.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.923+0000 7f1322c00640 1 -- 172.21.15.2:0/1293347269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f131c1049a0 msgr2=0x7f131c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:31.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.923+0000 7f1322c00640 1 --2- 172.21.15.2:0/1293347269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f131c1049a0 0x7f131c1089f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1304009900 tx=0x7f130402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:14:31.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.923+0000 7f1322c00640 1 -- 172.21.15.2:0/1293347269 shutdown_connections 2024-09-10T08:14:31.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.923+0000 7f1322c00640 1 --2- 172.21.15.2:0/1293347269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f131c109120 0x7f131c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:31.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.923+0000 7f1322c00640 1 --2- 172.21.15.2:0/1293347269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f131c1049a0 0x7f131c1089f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:31.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.923+0000 7f1322c00640 1 --2- 172.21.15.2:0/1293347269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f131c103ff0 0x7f131c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:31.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.923+0000 7f1322c00640 1 -- 172.21.15.2:0/1293347269 >> 172.21.15.2:0/1293347269 conn(0x7f131c0ffa90 msgr2=0x7f131c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:31.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.924+0000 7f1322c00640 1 -- 172.21.15.2:0/1293347269 shutdown_connections 2024-09-10T08:14:31.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.924+0000 7f1322c00640 1 -- 172.21.15.2:0/1293347269 wait complete. 2024-09-10T08:14:31.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.924+0000 7f1322c00640 1 Processor -- start 2024-09-10T08:14:31.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.925+0000 7f1322c00640 1 -- start start 2024-09-10T08:14:31.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.925+0000 7f1322c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f131c103ff0 0x7f131c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:31.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.925+0000 7f1322c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f131c1049a0 0x7f131c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:31.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.925+0000 7f1322c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f131c109120 0x7f131c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:31.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.925+0000 7f1322c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f131c114190 con 0x7f131c1049a0 2024-09-10T08:14:31.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.925+0000 7f1322c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f131c114010 con 0x7f131c109120 2024-09-10T08:14:31.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.925+0000 7f1322c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f131c114310 con 0x7f131c103ff0 2024-09-10T08:14:31.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.925+0000 7f1313400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f131c1049a0 0x7f131c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:31.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.925+0000 7f1313400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f131c1049a0 0x7f131c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58938/0 (socket says 172.21.15.2:58938) 2024-09-10T08:14:31.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.925+0000 7f1313400640 1 -- 172.21.15.2:0/1317640250 learned_addr learned my addr 172.21.15.2:0/1317640250 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:31.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.925+0000 7f131be00640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f131c103ff0 0x7f131c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:31.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.925+0000 7f1320c00640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f131c109120 0x7f131c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:31.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.926+0000 7f131be00640 1 -- 172.21.15.2:0/1317640250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f131c109120 msgr2=0x7f131c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:31.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.926+0000 7f131be00640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f131c109120 0x7f131c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:31.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.926+0000 7f131be00640 1 -- 172.21.15.2:0/1317640250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f131c1049a0 msgr2=0x7f131c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:31.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.926+0000 7f131be00640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f131c1049a0 0x7f131c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:31.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.926+0000 7f131be00640 1 -- 172.21.15.2:0/1317640250 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131c19f5a0 con 0x7f131c103ff0 2024-09-10T08:14:31.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.926+0000 7f131be00640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f131c103ff0 0x7f131c19a5f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f130c00eb00 tx=0x7f130c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:31.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.926+0000 7f1319600640 1 -- 172.21.15.2:0/1317640250 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c010430 con 0x7f131c103ff0 2024-09-10T08:14:31.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.927+0000 7f1322c00640 1 -- 172.21.15.2:0/1317640250 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131c19f890 con 0x7f131c103ff0 2024-09-10T08:14:31.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.927+0000 7f1319600640 1 -- 172.21.15.2:0/1317640250 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130c010ab0 con 0x7f131c103ff0 2024-09-10T08:14:31.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.927+0000 7f1319600640 1 -- 172.21.15.2:0/1317640250 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c017bb0 con 0x7f131c103ff0 2024-09-10T08:14:31.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.927+0000 7f1322c00640 1 -- 172.21.15.2:0/1317640250 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f131c1a7170 con 0x7f131c103ff0 2024-09-10T08:14:31.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.928+0000 7f1322c00640 1 -- 172.21.15.2:0/1317640250 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f131c1043d0 con 0x7f131c103ff0 2024-09-10T08:14:31.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.929+0000 7f1319600640 1 -- 172.21.15.2:0/1317640250 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f130c010c60 con 0x7f131c103ff0 2024-09-10T08:14:31.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.931+0000 7f1319600640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13000a2b80 0x7f13000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:31.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.931+0000 7f1319600640 1 -- 172.21.15.2:0/1317640250 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f130c0cfe40 con 0x7f131c103ff0 2024-09-10T08:14:31.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.931+0000 7f1313400640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13000a2b80 0x7f13000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:31.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.932+0000 7f1313400640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13000a2b80 0x7f13000a5040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f13040096c0 tx=0x7f13040046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:31.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:31.935+0000 7f1319600640 1 -- 172.21.15.2:0/1317640250 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f130c09c950 con 0x7f131c103ff0 2024-09-10T08:14:32.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.157+0000 7f1322c00640 1 -- 172.21.15.2:0/1317640250 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f131c1a73d0 con 0x7f131c103ff0 2024-09-10T08:14:32.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.158+0000 7f1319600640 1 -- 172.21.15.2:0/1317640250 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f130c09c250 con 0x7f131c103ff0 2024-09-10T08:14:32.158 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:14:32.158 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:14:32.158 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:14:32.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.161+0000 7f1322c00640 1 -- 172.21.15.2:0/1317640250 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13000a2b80 msgr2=0x7f13000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:32.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.161+0000 7f1322c00640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13000a2b80 0x7f13000a5040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f13040096c0 tx=0x7f13040046c0 comp rx=0 tx=0).stop 2024-09-10T08:14:32.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.161+0000 7f1322c00640 1 -- 172.21.15.2:0/1317640250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f131c103ff0 msgr2=0x7f131c19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:32.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.161+0000 7f1322c00640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f131c103ff0 0x7f131c19a5f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f130c00eb00 tx=0x7f130c00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:14:32.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.161+0000 7f1322c00640 1 -- 172.21.15.2:0/1317640250 shutdown_connections 2024-09-10T08:14:32.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.161+0000 7f1322c00640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f131c109120 0x7f131c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:32.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.161+0000 7f1322c00640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f131c1049a0 0x7f131c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:32.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.161+0000 7f1322c00640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13000a2b80 0x7f13000a5040 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:32.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.161+0000 7f1322c00640 1 --2- 172.21.15.2:0/1317640250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f131c103ff0 0x7f131c19a5f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:32.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.161+0000 7f1322c00640 1 -- 172.21.15.2:0/1317640250 >> 172.21.15.2:0/1317640250 conn(0x7f131c0ffa90 msgr2=0x7f131c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:32.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.162+0000 7f1322c00640 1 -- 172.21.15.2:0/1317640250 shutdown_connections 2024-09-10T08:14:32.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:32.162+0000 7f1322c00640 1 -- 172.21.15.2:0/1317640250 wait complete. 2024-09-10T08:14:33.222 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:32 smithi002 ceph-mon[30563]: pgmap v579: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:14:33.222 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:32 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1317640250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:14:33.259 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T08:14:33.260 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T08:14:33.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:32 smithi155 ceph-mon[35405]: pgmap v579: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:14:33.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:32 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1317640250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:14:33.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:32 smithi107 ceph-mon[35048]: pgmap v579: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:14:33.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:32 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1317640250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:14:33.638 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:34.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.186+0000 7f17d3600640 1 -- 172.21.15.2:0/2978352124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc1049c0 msgr2=0x7f17cc108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:34.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.186+0000 7f17d3600640 1 --2- 172.21.15.2:0/2978352124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc1049c0 0x7f17cc108a10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009900 tx=0x7f17b802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.186+0000 7f17d3600640 1 -- 172.21.15.2:0/2978352124 shutdown_connections 2024-09-10T08:14:34.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.186+0000 7f17d3600640 1 --2- 172.21.15.2:0/2978352124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17cc109140 0x7f17cc110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.186+0000 7f17d3600640 1 --2- 172.21.15.2:0/2978352124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc1049c0 0x7f17cc108a10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.186+0000 7f17d3600640 1 --2- 172.21.15.2:0/2978352124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17cc104010 0x7f17cc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.186+0000 7f17d3600640 1 -- 172.21.15.2:0/2978352124 >> 172.21.15.2:0/2978352124 conn(0x7f17cc0ffa50 msgr2=0x7f17cc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:34.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.186+0000 7f17d3600640 1 -- 172.21.15.2:0/2978352124 shutdown_connections 2024-09-10T08:14:34.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.186+0000 7f17d3600640 1 -- 172.21.15.2:0/2978352124 wait complete. 2024-09-10T08:14:34.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.187+0000 7f17d3600640 1 Processor -- start 2024-09-10T08:14:34.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.187+0000 7f17d3600640 1 -- start start 2024-09-10T08:14:34.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.187+0000 7f17d3600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17cc104010 0x7f17cc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.187+0000 7f17d3600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc1049c0 0x7f17cc19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.187+0000 7f17d3600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17cc109140 0x7f17cc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.187+0000 7f17d3600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc114140 con 0x7f17cc1049c0 2024-09-10T08:14:34.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.187+0000 7f17d3600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc113fc0 con 0x7f17cc104010 2024-09-10T08:14:34.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.187+0000 7f17d3600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1142c0 con 0x7f17cc109140 2024-09-10T08:14:34.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.188+0000 7f17cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc1049c0 0x7f17cc19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.188+0000 7f17d0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17cc104010 0x7f17cc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.188+0000 7f17d0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17cc104010 0x7f17cc19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50446/0 (socket says 172.21.15.2:50446) 2024-09-10T08:14:34.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.188+0000 7f17d0c00640 1 -- 172.21.15.2:0/1619354953 learned_addr learned my addr 172.21.15.2:0/1619354953 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:34.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.188+0000 7f17cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc1049c0 0x7f17cc19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58944/0 (socket says 172.21.15.2:58944) 2024-09-10T08:14:34.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.188+0000 7f17d1600640 1 --2- 172.21.15.2:0/1619354953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17cc109140 0x7f17cc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.188+0000 7f17d0c00640 1 -- 172.21.15.2:0/1619354953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17cc109140 msgr2=0x7f17cc19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:34.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.188+0000 7f17d0c00640 1 --2- 172.21.15.2:0/1619354953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17cc109140 0x7f17cc19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.188+0000 7f17d0c00640 1 -- 172.21.15.2:0/1619354953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc1049c0 msgr2=0x7f17cc19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:34.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.188+0000 7f17d0c00640 1 --2- 172.21.15.2:0/1619354953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc1049c0 0x7f17cc19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.188+0000 7f17d0c00640 1 -- 172.21.15.2:0/1619354953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc19f610 con 0x7f17cc104010 2024-09-10T08:14:34.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.188+0000 7f17d0c00640 1 --2- 172.21.15.2:0/1619354953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17cc104010 0x7f17cc19a660 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f17c000e9f0 tx=0x7f17c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:34.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.189+0000 7f17c9600640 1 -- 172.21.15.2:0/1619354953 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c000cd80 con 0x7f17cc104010 2024-09-10T08:14:34.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.189+0000 7f17d3600640 1 -- 172.21.15.2:0/1619354953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc19f900 con 0x7f17cc104010 2024-09-10T08:14:34.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.189+0000 7f17c9600640 1 -- 172.21.15.2:0/1619354953 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c0010910 con 0x7f17cc104010 2024-09-10T08:14:34.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.189+0000 7f17c9600640 1 -- 172.21.15.2:0/1619354953 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c00188f0 con 0x7f17cc104010 2024-09-10T08:14:34.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.189+0000 7f17d3600640 1 -- 172.21.15.2:0/1619354953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc1a71e0 con 0x7f17cc104010 2024-09-10T08:14:34.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.192+0000 7f17d6087740 1 Processor -- start 2024-09-10T08:14:34.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.192+0000 7f17d6087740 1 -- start start 2024-09-10T08:14:34.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.192+0000 7f17d6087740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f22790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.192+0000 7f17d6087740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 0x5618e2f2d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.192+0000 7f17d6087740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 0x5618e2f2f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.192+0000 7f17d6087740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5618e2e88530 con 0x5618e2f22cd0 2024-09-10T08:14:34.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.192+0000 7f17d6087740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5618e2e2f860 con 0x5618e2f223b0 2024-09-10T08:14:34.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.192+0000 7f17d6087740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5618e2e844f0 con 0x5618e2f2d590 2024-09-10T08:14:34.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.193+0000 7f17d1600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f22790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.193+0000 7f17d1600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f22790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50462/0 (socket says 172.21.15.2:50462) 2024-09-10T08:14:34.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.193+0000 7f17d1600640 1 -- 172.21.15.2:0/731358495 learned_addr learned my addr 172.21.15.2:0/731358495 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:34.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.193+0000 7f17cbe00640 1 --2- 172.21.15.2:0/731358495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 0x5618e2f2f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.193+0000 7f17c9600640 1 -- 172.21.15.2:0/1619354953 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f17c0010ac0 con 0x7f17cc104010 2024-09-10T08:14:34.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.193+0000 7f17d1600640 1 -- 172.21.15.2:0/731358495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 msgr2=0x5618e2f2f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:34.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.193+0000 7f17d1600640 1 --2- 172.21.15.2:0/731358495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 0x5618e2f2f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.193+0000 7f17d1600640 1 -- 172.21.15.2:0/731358495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 msgr2=0x5618e2f2d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:34.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.193+0000 7f17d1600640 1 --2- 172.21.15.2:0/731358495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 0x5618e2f2d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.193+0000 7f17d1600640 1 -- 172.21.15.2:0/731358495 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618e2bf9b90 con 0x5618e2f223b0 2024-09-10T08:14:34.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.193+0000 7f17cbe00640 1 --2- 172.21.15.2:0/731358495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 0x5618e2f2f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:14:34.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.193+0000 7f17d1600640 1 --2- 172.21.15.2:0/731358495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f22790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009f90 tx=0x7f17c4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46b727d0aa5282e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:34.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.194+0000 7f17bdc00640 1 -- 172.21.15.2:0/731358495 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4011700 con 0x5618e2f223b0 2024-09-10T08:14:34.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.194+0000 7f17bdc00640 1 -- 172.21.15.2:0/731358495 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c4011d80 con 0x5618e2f223b0 2024-09-10T08:14:34.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.194+0000 7f17bdc00640 1 -- 172.21.15.2:0/731358495 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4018380 con 0x5618e2f223b0 2024-09-10T08:14:34.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.194+0000 7f17d6087740 1 -- 172.21.15.2:0/731358495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 msgr2=0x5618e2f22790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:34.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.194+0000 7f17d6087740 1 --2- 172.21.15.2:0/731358495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f22790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009f90 tx=0x7f17c4010860 comp rx=0 tx=0).stop 2024-09-10T08:14:34.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.194+0000 7f17d6087740 1 -- 172.21.15.2:0/731358495 shutdown_connections 2024-09-10T08:14:34.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.194+0000 7f17d6087740 1 --2- 172.21.15.2:0/731358495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 0x5618e2f2f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.194+0000 7f17d6087740 1 --2- 172.21.15.2:0/731358495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 0x5618e2f2d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.194+0000 7f17d6087740 1 --2- 172.21.15.2:0/731358495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f22790 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.194+0000 7f17c9600640 1 --2- 172.21.15.2:0/1619354953 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f17b40a2b80 0x7f17b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.194+0000 7f17d6087740 1 -- 172.21.15.2:0/731358495 >> 172.21.15.2:0/731358495 conn(0x5618e2f1e940 msgr2=0x5618e2f1ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:34.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.195+0000 7f17c9600640 1 -- 172.21.15.2:0/1619354953 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f17c00cf890 con 0x7f17cc104010 2024-09-10T08:14:34.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.195+0000 7f17d6087740 1 -- 172.21.15.2:0/731358495 shutdown_connections 2024-09-10T08:14:34.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.195+0000 7f17d6087740 1 -- 172.21.15.2:0/731358495 wait complete. 2024-09-10T08:14:34.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.195+0000 7f17d1600640 1 --2- 172.21.15.2:0/1619354953 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f17b40a2b80 0x7f17b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.195+0000 7f17d6087740 1 -- start start 2024-09-10T08:14:34.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.196+0000 7f17d1600640 1 --2- 172.21.15.2:0/1619354953 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f17b40a2b80 0x7f17b40a5040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f17c4019930 tx=0x7f17c4010fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:34.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.196+0000 7f17d6087740 1 Processor -- start 2024-09-10T08:14:34.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.196+0000 7f17d6087740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f2a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.196+0000 7f17d6087740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 0x5618e2f2aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.196+0000 7f17d6087740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 0x5618e2f2b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.196+0000 7f17d6087740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5618e2f2ffb0 con 0x5618e2f22cd0 2024-09-10T08:14:34.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.196+0000 7f17d6087740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5618e2e42e60 con 0x5618e2f223b0 2024-09-10T08:14:34.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.196+0000 7f17d6087740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5618e2f30130 con 0x5618e2f2d590 2024-09-10T08:14:34.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.196+0000 7f17d6087740 10 client.0 _ll_register_callbacks cb 0x5618e2e47cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:34.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 0x5618e2f2aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17d0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f2a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 0x5618e2f2aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58974/0 (socket says 172.21.15.2:58974) 2024-09-10T08:14:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17d0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f2a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50464/0 (socket says 172.21.15.2:50464) 2024-09-10T08:14:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17cbe00640 1 -- 172.21.15.2:0/4220476128 learned_addr learned my addr 172.21.15.2:0/4220476128 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17d1600640 1 --2- 172.21.15.2:0/4220476128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 0x5618e2f2b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17d0c00640 1 -- 172.21.15.2:0/4220476128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 msgr2=0x5618e2f2b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17d0c00640 1 --2- 172.21.15.2:0/4220476128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 0x5618e2f2b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17d0c00640 1 -- 172.21.15.2:0/4220476128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 msgr2=0x5618e2f2aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17d0c00640 1 --2- 172.21.15.2:0/4220476128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 0x5618e2f2aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17d0c00640 1 -- 172.21.15.2:0/4220476128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618e2e4e990 con 0x5618e2f223b0 2024-09-10T08:14:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17d0c00640 1 --2- 172.21.15.2:0/4220476128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f2a660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x5618e2f1f7a0 tx=0x7f17c009be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66a194e4d9e7a911 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17b2000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:14:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.197+0000 7f17b2000640 1 -- 172.21.15.2:0/4220476128 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c009b1a0 con 0x5618e2f223b0 2024-09-10T08:14:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.198+0000 7f17b2000640 1 -- 172.21.15.2:0/4220476128 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f17c009b340 con 0x5618e2f223b0 2024-09-10T08:14:34.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.198+0000 7f17d6087740 1 -- 172.21.15.2:0/4220476128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c4011d80 con 0x5618e2f223b0 2024-09-10T08:14:34.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.198+0000 7f17b2000640 1 -- 172.21.15.2:0/4220476128 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17c0010430 con 0x5618e2f223b0 2024-09-10T08:14:34.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.198+0000 7f17d6087740 10 client.27185 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:34.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.198+0000 7f17d6087740 1 -- 172.21.15.2:0/4220476128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5618e2f3bd60 con 0x5618e2f223b0 2024-09-10T08:14:34.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.198+0000 7f17b2000640 1 -- 172.21.15.2:0/4220476128 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c4011d80 con 0x5618e2f223b0 2024-09-10T08:14:34.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.199+0000 7f17b2000640 1 -- 172.21.15.2:0/4220476128 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f17c009ac10 con 0x5618e2f223b0 2024-09-10T08:14:34.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.199+0000 7f17d6087740 10 client.27185 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:34.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.199+0000 7f17d6087740 10 client.27185 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:14:34.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.199+0000 7f17d6087740 20 client.27185 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:34.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.199+0000 7f17d6087740 1 --2- 172.21.15.2:0/4220476128 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618e2f3d1c0 0x5618e2f3f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.199+0000 7f17d6087740 4 client.27185 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:34.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.199+0000 7f17d6087740 1 -- 172.21.15.2:0/4220476128 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618e2e2b310 con 0x5618e2f3d1c0 2024-09-10T08:14:34.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.199+0000 7f17cbe00640 1 --2- 172.21.15.2:0/4220476128 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618e2f3d1c0 0x5618e2f3f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.200+0000 7f17cbe00640 1 --2- 172.21.15.2:0/4220476128 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618e2f3d1c0 0x5618e2f3f600 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5f67535173ab7bf server_cookie=9ccd2c87511aa20d in_seq=0 out_seq=0 2024-09-10T08:14:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.200+0000 7f17b2000640 10 client.27185 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.201+0000 7f17b2000640 1 -- 172.21.15.2:0/4220476128 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618e2e2b310 con 0x5618e2f3d1c0 2024-09-10T08:14:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.201+0000 7f17b2000640 10 client.27185 handle_command_reply: tid=0 2024-09-10T08:14:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.201+0000 7f17d6087740 1 client.27185 shutdown 2024-09-10T08:14:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.201+0000 7f17d6087740 1 -- 172.21.15.2:0/4220476128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 msgr2=0x5618e2f2a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.201+0000 7f17d6087740 1 --2- 172.21.15.2:0/4220476128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f2a660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x5618e2f1f7a0 tx=0x7f17c009be10 comp rx=0 tx=0).stop 2024-09-10T08:14:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.201+0000 7f17d6087740 1 -- 172.21.15.2:0/4220476128 shutdown_connections 2024-09-10T08:14:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.201+0000 7f17d6087740 1 --2- 172.21.15.2:0/4220476128 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618e2f3d1c0 0x5618e2f3f600 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.201+0000 7f17d6087740 1 --2- 172.21.15.2:0/4220476128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 0x5618e2f2b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.201+0000 7f17d6087740 1 --2- 172.21.15.2:0/4220476128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 0x5618e2f2aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.201+0000 7f17d6087740 1 --2- 172.21.15.2:0/4220476128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f2a660 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.201+0000 7f17b2000640 0 client.27185 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:34.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.201+0000 7f17d6087740 1 -- 172.21.15.2:0/4220476128 >> 172.21.15.2:0/4220476128 conn(0x5618e2f1e940 msgr2=0x5618e2f1dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:34.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.202+0000 7f17d6087740 1 -- 172.21.15.2:0/4220476128 shutdown_connections 2024-09-10T08:14:34.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.202+0000 7f17d6087740 1 -- 172.21.15.2:0/4220476128 wait complete. 2024-09-10T08:14:34.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.202+0000 7f17d6087740 20 client.27185 trim_cache size 0 max 16384 2024-09-10T08:14:34.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.220+0000 7f17d6087740 1 Processor -- start 2024-09-10T08:14:34.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.220+0000 7f17d6087740 1 -- start start 2024-09-10T08:14:34.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.220+0000 7f17d6087740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f223b0 0x5618e2f44ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.220+0000 7f17d6087740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f22cd0 0x5618e2f45410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.220+0000 7f17d6087740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f2d590 0x5618e2f35370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.220+0000 7f17d6087740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1b3a90 con 0x5618e2f2d590 2024-09-10T08:14:34.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.220+0000 7f17d6087740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1b1360 con 0x5618e2f22cd0 2024-09-10T08:14:34.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.220+0000 7f17d6087740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1af1b0 con 0x5618e2f223b0 2024-09-10T08:14:34.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.220+0000 7f17d0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f223b0 0x5618e2f44ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.220+0000 7f17d1600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f2d590 0x5618e2f35370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.220+0000 7f17d0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f223b0 0x5618e2f44ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:52796/0 (socket says 172.21.15.2:52796) 2024-09-10T08:14:34.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.221+0000 7f17d0c00640 1 -- 172.21.15.2:0/959114373 learned_addr learned my addr 172.21.15.2:0/959114373 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:34.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.221+0000 7f17cbe00640 1 --2- 172.21.15.2:0/959114373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f22cd0 0x5618e2f45410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.221+0000 7f17d0c00640 1 -- 172.21.15.2:0/959114373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f22cd0 msgr2=0x5618e2f45410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:34.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.221+0000 7f17d0c00640 1 --2- 172.21.15.2:0/959114373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f22cd0 0x5618e2f45410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.221+0000 7f17d0c00640 1 -- 172.21.15.2:0/959114373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f2d590 msgr2=0x5618e2f35370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:34.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.221+0000 7f17d0c00640 1 --2- 172.21.15.2:0/959114373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f2d590 0x5618e2f35370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.221+0000 7f17d0c00640 1 -- 172.21.15.2:0/959114373 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618e2f39210 con 0x5618e2f223b0 2024-09-10T08:14:34.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.221+0000 7f17d0c00640 1 --2- 172.21.15.2:0/959114373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f223b0 0x5618e2f44ed0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f17c009bce0 tx=0x7f17c000e980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deebe20224da1160 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:34.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.221+0000 7f17bd200640 1 -- 172.21.15.2:0/959114373 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c00a0050 con 0x5618e2f223b0 2024-09-10T08:14:34.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.221+0000 7f17bd200640 1 -- 172.21.15.2:0/959114373 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c009ac10 con 0x5618e2f223b0 2024-09-10T08:14:34.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.221+0000 7f17bd200640 1 -- 172.21.15.2:0/959114373 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0097d00 con 0x5618e2f223b0 2024-09-10T08:14:34.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.222+0000 7f17d6087740 1 -- 172.21.15.2:0/959114373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f223b0 msgr2=0x5618e2f44ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:34.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.222+0000 7f17d6087740 1 --2- 172.21.15.2:0/959114373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f223b0 0x5618e2f44ed0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f17c009bce0 tx=0x7f17c000e980 comp rx=0 tx=0).stop 2024-09-10T08:14:34.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.222+0000 7f17d6087740 1 -- 172.21.15.2:0/959114373 shutdown_connections 2024-09-10T08:14:34.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.222+0000 7f17d6087740 1 --2- 172.21.15.2:0/959114373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f2d590 0x5618e2f35370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.222+0000 7f17d6087740 1 --2- 172.21.15.2:0/959114373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f22cd0 0x5618e2f45410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.222+0000 7f17d6087740 1 --2- 172.21.15.2:0/959114373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f223b0 0x5618e2f44ed0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.222+0000 7f17d6087740 1 -- 172.21.15.2:0/959114373 >> 172.21.15.2:0/959114373 conn(0x5618e2f1e940 msgr2=0x5618e2f2c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:34.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.222+0000 7f17d6087740 1 -- 172.21.15.2:0/959114373 shutdown_connections 2024-09-10T08:14:34.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.222+0000 7f17d6087740 1 -- 172.21.15.2:0/959114373 wait complete. 2024-09-10T08:14:34.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.223+0000 7f17d6087740 1 -- start start 2024-09-10T08:14:34.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.223+0000 7f17d6087740 1 Processor -- start 2024-09-10T08:14:34.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.223+0000 7f17d6087740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f32af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.223+0000 7f17d6087740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 0x5618e2f33030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.223+0000 7f17d6087740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 0x5618e2fa1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.223+0000 7f17d6087740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1af820 con 0x5618e2f22cd0 2024-09-10T08:14:34.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.223+0000 7f17d6087740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1b0c90 con 0x5618e2f223b0 2024-09-10T08:14:34.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.223+0000 7f17d6087740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1afef0 con 0x5618e2f2d590 2024-09-10T08:14:34.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.224+0000 7f17d6087740 10 client.0 _ll_register_callbacks cb 0x7f17c0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:34.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.224+0000 7f17cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 0x5618e2f33030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.224+0000 7f17d0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f32af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.224+0000 7f17d0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f32af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50480/0 (socket says 172.21.15.2:50480) 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.224+0000 7f17d0c00640 1 -- 172.21.15.2:0/3302491521 learned_addr learned my addr 172.21.15.2:0/3302491521 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.224+0000 7f17d1600640 1 --2- 172.21.15.2:0/3302491521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 0x5618e2fa1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.224+0000 7f17d0c00640 1 -- 172.21.15.2:0/3302491521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 msgr2=0x5618e2fa1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.224+0000 7f17d0c00640 1 --2- 172.21.15.2:0/3302491521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 0x5618e2fa1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.224+0000 7f17d0c00640 1 -- 172.21.15.2:0/3302491521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 msgr2=0x5618e2f33030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.224+0000 7f17d0c00640 1 --2- 172.21.15.2:0/3302491521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 0x5618e2f33030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.224+0000 7f17d0c00640 1 -- 172.21.15.2:0/3302491521 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618e2fa2520 con 0x5618e2f223b0 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.224+0000 7f17d0c00640 1 --2- 172.21.15.2:0/3302491521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f32af0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f17c009a000 tx=0x7f17c009b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15bf8261c9142e4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.224+0000 7f17b2000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17b2000640 1 -- 172.21.15.2:0/3302491521 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c00a4d60 con 0x5618e2f223b0 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17d6087740 1 -- 172.21.15.2:0/3302491521 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c009ac10 con 0x5618e2f223b0 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17b2000640 1 -- 172.21.15.2:0/3302491521 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f17c009b1a0 con 0x5618e2f223b0 2024-09-10T08:14:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17b2000640 1 -- 172.21.15.2:0/3302491521 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17c008b050 con 0x5618e2f223b0 2024-09-10T08:14:34.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17d6087740 10 client.27191 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:34.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17d6087740 1 -- 172.21.15.2:0/3302491521 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5618e2fa2520 con 0x5618e2f223b0 2024-09-10T08:14:34.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17b2000640 1 -- 172.21.15.2:0/3302491521 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c009ac10 con 0x5618e2f223b0 2024-09-10T08:14:34.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17b2000640 1 -- 172.21.15.2:0/3302491521 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f17c00a11e0 con 0x5618e2f223b0 2024-09-10T08:14:34.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17d6087740 10 client.27191 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:34.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17d6087740 10 client.27191 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:14:34.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17d6087740 20 client.27191 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:34.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17d6087740 1 --2- 172.21.15.2:0/3302491521 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618e2fa9c60 0x5618e2fac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:34.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17d6087740 4 client.27191 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T08:14:34.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.225+0000 7f17d6087740 1 -- 172.21.15.2:0/3302491521 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x5618e2f24cb0 con 0x5618e2fa9c60 2024-09-10T08:14:34.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.226+0000 7f17cbe00640 1 --2- 172.21.15.2:0/3302491521 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618e2fa9c60 0x5618e2fac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:34.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.226+0000 7f17cbe00640 1 --2- 172.21.15.2:0/3302491521 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618e2fa9c60 0x5618e2fac050 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0a3e3572cb8a2a6 server_cookie=b1b00849f12306d8 in_seq=0 out_seq=0 2024-09-10T08:14:34.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.226+0000 7f17b2000640 10 client.27191 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:34.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.237+0000 7f17b2000640 1 -- 172.21.15.2:0/3302491521 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5618e2f24cb0 con 0x5618e2fa9c60 2024-09-10T08:14:34.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.237+0000 7f17b2000640 10 client.27191 handle_command_reply: tid=0 2024-09-10T08:14:34.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.237+0000 7f17d6087740 1 client.27191 shutdown 2024-09-10T08:14:34.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.237+0000 7f17d6087740 1 -- 172.21.15.2:0/3302491521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 msgr2=0x5618e2f32af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:34.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.237+0000 7f17d6087740 1 --2- 172.21.15.2:0/3302491521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f32af0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f17c009a000 tx=0x7f17c009b9c0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.237+0000 7f17d6087740 1 -- 172.21.15.2:0/3302491521 shutdown_connections 2024-09-10T08:14:34.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.237+0000 7f17d6087740 1 --2- 172.21.15.2:0/3302491521 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5618e2fa9c60 0x5618e2fac050 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.237+0000 7f17d6087740 1 --2- 172.21.15.2:0/3302491521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e2f2d590 0x5618e2fa1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.237+0000 7f17d6087740 1 --2- 172.21.15.2:0/3302491521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5618e2f22cd0 0x5618e2f33030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.237+0000 7f17b2000640 0 client.27191 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:34.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.237+0000 7f17d6087740 1 --2- 172.21.15.2:0/3302491521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618e2f223b0 0x5618e2f32af0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.237+0000 7f17d6087740 1 -- 172.21.15.2:0/3302491521 >> 172.21.15.2:0/3302491521 conn(0x5618e2f1e940 msgr2=0x5618e2f38200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:34.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.238+0000 7f17d6087740 1 -- 172.21.15.2:0/3302491521 shutdown_connections 2024-09-10T08:14:34.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.238+0000 7f17d6087740 1 -- 172.21.15.2:0/3302491521 wait complete. 2024-09-10T08:14:34.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.238+0000 7f17d6087740 20 client.27191 trim_cache size 0 max 16384 2024-09-10T08:14:34.238 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:14:34.238 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:14:34.238 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:14:34.238 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 18, 2024-09-10T08:14:34.238 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:14:34.238 INFO:teuthology.orchestra.run.smithi002.stdout: "f78d3e3b": { 2024-09-10T08:14:34.238 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 18, 2024-09-10T08:14:34.238 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:14:34.238 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:14:34.239 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:14:34.239 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:14:34.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:14:34.239 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:14:34.239 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 140.1, 2024-09-10T08:14:34.239 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:14:34.239 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:14:34.239 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:14:34.239 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:14:34.239 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:14:34.239 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:14:34.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:34.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:34.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:34.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:34.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:34.240 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:14:34.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.239+0000 7f17d3600640 1 -- 172.21.15.2:0/1619354953 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f17b40a2b80 msgr2=0x7f17b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:34.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.239+0000 7f17d3600640 1 --2- 172.21.15.2:0/1619354953 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f17b40a2b80 0x7f17b40a5040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f17c4019930 tx=0x7f17c4010fd0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.239+0000 7f17d3600640 1 -- 172.21.15.2:0/1619354953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17cc104010 msgr2=0x7f17cc19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:34.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.239+0000 7f17d3600640 1 --2- 172.21.15.2:0/1619354953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17cc104010 0x7f17cc19a660 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f17c000e9f0 tx=0x7f17c000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.239+0000 7f17d3600640 1 -- 172.21.15.2:0/1619354953 shutdown_connections 2024-09-10T08:14:34.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.239+0000 7f17d3600640 1 --2- 172.21.15.2:0/1619354953 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f17b40a2b80 0x7f17b40a5040 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.239+0000 7f17d3600640 1 --2- 172.21.15.2:0/1619354953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17cc109140 0x7f17cc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.239+0000 7f17d3600640 1 --2- 172.21.15.2:0/1619354953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc1049c0 0x7f17cc19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.239+0000 7f17d3600640 1 --2- 172.21.15.2:0/1619354953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17cc104010 0x7f17cc19a660 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:34.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.239+0000 7f17d3600640 1 -- 172.21.15.2:0/1619354953 >> 172.21.15.2:0/1619354953 conn(0x7f17cc0ffa50 msgr2=0x7f17cc100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:34.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.239+0000 7f17d3600640 1 -- 172.21.15.2:0/1619354953 shutdown_connections 2024-09-10T08:14:34.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:34.239+0000 7f17d3600640 1 -- 172.21.15.2:0/1619354953 wait complete. 2024-09-10T08:14:35.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:34 smithi002 ceph-mon[30563]: pgmap v580: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:14:35.089 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f78d3e3b', quiesce took 4.1 seconds. Will release after: 16.0 2024-09-10T08:14:35.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:34 smithi155 ceph-mon[35405]: pgmap v580: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:14:35.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:34 smithi107 ceph-mon[35048]: pgmap v580: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:14:37.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:36 smithi002 ceph-mon[30563]: pgmap v581: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:14:37.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:36 smithi155 ceph-mon[35405]: pgmap v581: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:14:37.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:36 smithi107 ceph-mon[35048]: pgmap v581: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:14:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.942+0000 7f8f9a400640 1 -- 172.21.15.2:0/3128355840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f94074bf0 msgr2=0x7f8f94075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.942+0000 7f8f9a400640 1 --2- 172.21.15.2:0/3128355840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f94074bf0 0x7f8f94075050 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009900 tx=0x7f8f8002a480 comp rx=0 tx=0).stop 2024-09-10T08:14:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.943+0000 7f8f9a400640 1 -- 172.21.15.2:0/3128355840 shutdown_connections 2024-09-10T08:14:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.943+0000 7f8f9a400640 1 --2- 172.21.15.2:0/3128355840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f941099d0 0x7f8f9410be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.943+0000 7f8f9a400640 1 --2- 172.21.15.2:0/3128355840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f94074bf0 0x7f8f94075050 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.943+0000 7f8f9a400640 1 --2- 172.21.15.2:0/3128355840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f940761d0 0x7f8f94074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.943+0000 7f8f9a400640 1 -- 172.21.15.2:0/3128355840 >> 172.21.15.2:0/3128355840 conn(0x7f8f940fd2e0 msgr2=0x7f8f940ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.943+0000 7f8f9a400640 1 -- 172.21.15.2:0/3128355840 shutdown_connections 2024-09-10T08:14:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.943+0000 7f8f9a400640 1 -- 172.21.15.2:0/3128355840 wait complete. 2024-09-10T08:14:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.943+0000 7f8f9a400640 1 Processor -- start 2024-09-10T08:14:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.944+0000 7f8f9a400640 1 -- start start 2024-09-10T08:14:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.944+0000 7f8f9a400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f94074bf0 0x7f8f94199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.944+0000 7f8f9a400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f940761d0 0x7f8f9419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.944+0000 7f8f9a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f941099d0 0x7f8f941a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.944+0000 7f8f9a400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9410f9d0 con 0x7f8f94074bf0 2024-09-10T08:14:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.944+0000 7f8f9a400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9410f850 con 0x7f8f941099d0 2024-09-10T08:14:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.944+0000 7f8f9a400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9410fb50 con 0x7f8f940761d0 2024-09-10T08:14:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.944+0000 7f8f93400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f94074bf0 0x7f8f94199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.944+0000 7f8f93400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f94074bf0 0x7f8f94199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50880/0 (socket says 172.21.15.2:50880) 2024-09-10T08:14:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.944+0000 7f8f93400640 1 -- 172.21.15.2:0/410928938 learned_addr learned my addr 172.21.15.2:0/410928938 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.944+0000 7f8f93e00640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f941099d0 0x7f8f941a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.944+0000 7f8f92a00640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f940761d0 0x7f8f9419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.945+0000 7f8f93400640 1 -- 172.21.15.2:0/410928938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f940761d0 msgr2=0x7f8f9419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.945+0000 7f8f93400640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f940761d0 0x7f8f9419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.945+0000 7f8f93400640 1 -- 172.21.15.2:0/410928938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f941099d0 msgr2=0x7f8f941a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.945+0000 7f8f93400640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f941099d0 0x7f8f941a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.945+0000 7f8f93400640 1 -- 172.21.15.2:0/410928938 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f941a1bf0 con 0x7f8f94074bf0 2024-09-10T08:14:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.945+0000 7f8f92a00640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f940761d0 0x7f8f9419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:14:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.945+0000 7f8f93400640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f94074bf0 0x7f8f94199fd0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400e9f0 tx=0x7f8f8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.945+0000 7f8f8be00640 1 -- 172.21.15.2:0/410928938 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8400cd80 con 0x7f8f94074bf0 2024-09-10T08:14:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.945+0000 7f8f8be00640 1 -- 172.21.15.2:0/410928938 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f84005500 con 0x7f8f94074bf0 2024-09-10T08:14:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.945+0000 7f8f9a400640 1 -- 172.21.15.2:0/410928938 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f941a1ee0 con 0x7f8f94074bf0 2024-09-10T08:14:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.945+0000 7f8f8be00640 1 -- 172.21.15.2:0/410928938 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84010690 con 0x7f8f94074bf0 2024-09-10T08:14:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.945+0000 7f8f9a400640 1 -- 172.21.15.2:0/410928938 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f941a2340 con 0x7f8f94074bf0 2024-09-10T08:14:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.948+0000 7f8f9d2b1740 1 Processor -- start 2024-09-10T08:14:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.948+0000 7f8f8be00640 1 -- 172.21.15.2:0/410928938 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8f840046c0 con 0x7f8f94074bf0 2024-09-10T08:14:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.948+0000 7f8f9d2b1740 1 -- start start 2024-09-10T08:14:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.949+0000 7f8f9d2b1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f090 0x55ea29b9f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.949+0000 7f8f9d2b1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29b9f9d0 0x55ea29ba4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.949+0000 7f8f9d2b1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29ba4a60 0x55ea29ba6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.949+0000 7f8f9d2b1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ea29b04100 con 0x55ea29ba4a60 2024-09-10T08:14:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.949+0000 7f8f9d2b1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ea29aac1b0 con 0x55ea29b9f090 2024-09-10T08:14:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.949+0000 7f8f9d2b1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ea29b000c0 con 0x55ea29b9f9d0 2024-09-10T08:14:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.949+0000 7f8f92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29ba4a60 0x55ea29ba6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.949+0000 7f8f92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29ba4a60 0x55ea29ba6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50894/0 (socket says 172.21.15.2:50894) 2024-09-10T08:14:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.949+0000 7f8f92a00640 1 -- 172.21.15.2:0/2158270378 learned_addr learned my addr 172.21.15.2:0/2158270378 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.949+0000 7f8f93e00640 1 --2- 172.21.15.2:0/2158270378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f090 0x55ea29b9f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.949+0000 7f8f93400640 1 --2- 172.21.15.2:0/2158270378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29b9f9d0 0x55ea29ba4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.950+0000 7f8f92a00640 1 -- 172.21.15.2:0/2158270378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29b9f9d0 msgr2=0x55ea29ba4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.950+0000 7f8f92a00640 1 --2- 172.21.15.2:0/2158270378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29b9f9d0 0x55ea29ba4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.950+0000 7f8f92a00640 1 -- 172.21.15.2:0/2158270378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f090 msgr2=0x55ea29b9f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.950+0000 7f8f92a00640 1 --2- 172.21.15.2:0/2158270378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f090 0x55ea29b9f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.950+0000 7f8f92a00640 1 -- 172.21.15.2:0/2158270378 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea29817210 con 0x55ea29ba4a60 2024-09-10T08:14:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.950+0000 7f8f93e00640 1 --2- 172.21.15.2:0/2158270378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f090 0x55ea29b9f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.950+0000 7f8f8be00640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8f780a2cb0 0x7f8f780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.950+0000 7f8f92a00640 1 --2- 172.21.15.2:0/2158270378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29ba4a60 0x55ea29ba6f20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8f80031610 tx=0x7f8f80032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb9d85d5d733147 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.950+0000 7f8f8be00640 1 -- 172.21.15.2:0/410928938 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8f840cf8c0 con 0x7f8f94074bf0 2024-09-10T08:14:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.951+0000 7f8f93e00640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8f780a2cb0 0x7f8f780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.951+0000 7f8f88c00640 1 -- 172.21.15.2:0/2158270378 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8003f070 con 0x55ea29ba4a60 2024-09-10T08:14:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.951+0000 7f8f88c00640 1 -- 172.21.15.2:0/2158270378 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f800318b0 con 0x55ea29ba4a60 2024-09-10T08:14:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.951+0000 7f8f88c00640 1 -- 172.21.15.2:0/2158270378 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80035070 con 0x55ea29ba4a60 2024-09-10T08:14:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.952+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/2158270378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29ba4a60 msgr2=0x55ea29ba6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.952+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/2158270378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29ba4a60 0x55ea29ba6f20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8f80031610 tx=0x7f8f80032cb0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.952+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/2158270378 shutdown_connections 2024-09-10T08:14:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.952+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/2158270378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29ba4a60 0x55ea29ba6f20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.952+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/2158270378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29b9f9d0 0x55ea29ba4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.952+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/2158270378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f090 0x55ea29b9f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.952+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/2158270378 >> 172.21.15.2:0/2158270378 conn(0x55ea29b9a700 msgr2=0x55ea29b9ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.952+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/2158270378 shutdown_connections 2024-09-10T08:14:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.952+0000 7f8f93e00640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8f780a2cb0 0x7f8f780a5170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009fd0 tx=0x7f8f8c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.952+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/2158270378 wait complete. 2024-09-10T08:14:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.952+0000 7f8f9d2b1740 1 -- start start 2024-09-10T08:14:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.953+0000 7f8f9d2b1740 1 Processor -- start 2024-09-10T08:14:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.953+0000 7f8f9d2b1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f090 0x55ea29ba9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.953+0000 7f8f9d2b1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f9d0 0x55ea29ba97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.953+0000 7f8f9d2b1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29ba4a60 0x55ea29bb07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.953+0000 7f8f9d2b1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ea29ba79d0 con 0x55ea29b9f090 2024-09-10T08:14:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.953+0000 7f8f9d2b1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ea29a8d090 con 0x55ea29b9f9d0 2024-09-10T08:14:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.953+0000 7f8f9d2b1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ea29ba7b50 con 0x55ea29ba4a60 2024-09-10T08:14:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.953+0000 7f8f9d2b1740 10 client.0 _ll_register_callbacks cb 0x55ea29ae1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.953+0000 7f8f93400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f090 0x55ea29ba9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.953+0000 7f8f93400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f090 0x55ea29ba9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50900/0 (socket says 172.21.15.2:50900) 2024-09-10T08:14:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.953+0000 7f8f93400640 1 -- 172.21.15.2:0/1458087253 learned_addr learned my addr 172.21.15.2:0/1458087253 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.953+0000 7f8f92a00640 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f9d0 0x55ea29ba97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.953+0000 7f8f93e00640 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29ba4a60 0x55ea29bb07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.954+0000 7f8f93400640 1 -- 172.21.15.2:0/1458087253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29ba4a60 msgr2=0x55ea29bb07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.954+0000 7f8f93400640 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29ba4a60 0x55ea29bb07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.954+0000 7f8f93400640 1 -- 172.21.15.2:0/1458087253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f9d0 msgr2=0x55ea29ba97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.954+0000 7f8f93400640 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f9d0 0x55ea29ba97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.954+0000 7f8f93400640 1 -- 172.21.15.2:0/1458087253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea298e3620 con 0x55ea29b9f090 2024-09-10T08:14:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.954+0000 7f8f92a00640 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f9d0 0x55ea29ba97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.954+0000 7f8f93400640 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f090 0x55ea29ba9270 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8f8409b9d0 tx=0x7f8f84098030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e178a593e9d806 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.954+0000 7f8f7d600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.954+0000 7f8f7d600640 1 -- 172.21.15.2:0/1458087253 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f840a46b0 con 0x55ea29b9f090 2024-09-10T08:14:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.954+0000 7f8f7d600640 1 -- 172.21.15.2:0/1458087253 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8f8409b120 con 0x55ea29b9f090 2024-09-10T08:14:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.954+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/1458087253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f800318b0 con 0x55ea29b9f090 2024-09-10T08:14:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.954+0000 7f8f7d600640 1 -- 172.21.15.2:0/1458087253 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f8409c3d0 con 0x55ea29b9f090 2024-09-10T08:14:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.955+0000 7f8f9d2b1740 10 client.17163 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.955+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/1458087253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ea29bb36a0 con 0x55ea29b9f090 2024-09-10T08:14:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.955+0000 7f8f7d600640 1 -- 172.21.15.2:0/1458087253 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f800318b0 con 0x55ea29b9f090 2024-09-10T08:14:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.955+0000 7f8f7d600640 1 -- 172.21.15.2:0/1458087253 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8f8409ab90 con 0x55ea29b9f090 2024-09-10T08:14:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.955+0000 7f8f9d2b1740 10 client.17163 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.955+0000 7f8f9d2b1740 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.955+0000 7f8f9d2b1740 20 client.17163 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.955+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea29bb4d20 0x55ea29bb7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.955+0000 7f8f9d2b1740 4 client.17163 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.955+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/1458087253 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea29b11830 con 0x55ea29bb4d20 2024-09-10T08:14:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.955+0000 7f8f92a00640 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea29bb4d20 0x55ea29bb7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.956+0000 7f8f92a00640 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea29bb4d20 0x55ea29bb7110 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90d7c810c9c2e808 server_cookie=59d071f4209270e7 in_seq=0 out_seq=0 2024-09-10T08:14:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.956+0000 7f8f7d600640 10 client.17163 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f7d600640 1 -- 172.21.15.2:0/1458087253 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea29b11830 con 0x55ea29bb4d20 2024-09-10T08:14:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f7d600640 10 client.17163 handle_command_reply: tid=0 2024-09-10T08:14:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f9d2b1740 1 client.17163 shutdown 2024-09-10T08:14:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/1458087253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f090 msgr2=0x55ea29ba9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f090 0x55ea29ba9270 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8f8409b9d0 tx=0x7f8f84098030 comp rx=0 tx=0).stop 2024-09-10T08:14:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/1458087253 shutdown_connections 2024-09-10T08:14:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea29bb4d20 0x55ea29bb7110 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29ba4a60 0x55ea29bb07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f9d0 0x55ea29ba97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f7d600640 0 client.17163 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/1458087253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f090 0x55ea29ba9270 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/1458087253 >> 172.21.15.2:0/1458087253 conn(0x55ea29b9a700 msgr2=0x55ea29b9bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/1458087253 shutdown_connections 2024-09-10T08:14:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/1458087253 wait complete. 2024-09-10T08:14:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.957+0000 7f8f9d2b1740 20 client.17163 trim_cache size 0 max 16384 2024-09-10T08:14:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.972+0000 7f8f9d2b1740 1 Processor -- start 2024-09-10T08:14:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f9d2b1740 1 -- start start 2024-09-10T08:14:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f9d2b1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f090 0x55ea29ba9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f9d2b1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 0x55ea29bb07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f9d2b1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29ba4a60 0x55ea29b956a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f9d2b1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8f941af270 con 0x55ea29b9f9d0 2024-09-10T08:14:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f9d2b1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f941acb40 con 0x55ea29b9f090 2024-09-10T08:14:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f9d2b1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f941aa990 con 0x55ea29ba4a60 2024-09-10T08:14:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 0x55ea29bb07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 0x55ea29bb07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50910/0 (socket says 172.21.15.2:50910) 2024-09-10T08:14:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f92a00640 1 -- 172.21.15.2:0/28049195 learned_addr learned my addr 172.21.15.2:0/28049195 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f93400640 1 --2- 172.21.15.2:0/28049195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f090 0x55ea29ba9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f93e00640 1 --2- 172.21.15.2:0/28049195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29ba4a60 0x55ea29b956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f92a00640 1 -- 172.21.15.2:0/28049195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29ba4a60 msgr2=0x55ea29b956a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f92a00640 1 --2- 172.21.15.2:0/28049195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29ba4a60 0x55ea29b956a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f92a00640 1 -- 172.21.15.2:0/28049195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f090 msgr2=0x55ea29ba9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f92a00640 1 --2- 172.21.15.2:0/28049195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f090 0x55ea29ba9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.973+0000 7f8f92a00640 1 -- 172.21.15.2:0/28049195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea29b9a520 con 0x55ea29b9f9d0 2024-09-10T08:14:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.974+0000 7f8f92a00640 1 --2- 172.21.15.2:0/28049195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 0x55ea29bb07b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8f80031440 tx=0x7f8f80032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7727ba7445e745a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.974+0000 7f8f7fe00640 1 -- 172.21.15.2:0/28049195 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8003f040 con 0x55ea29b9f9d0 2024-09-10T08:14:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.974+0000 7f8f7fe00640 1 -- 172.21.15.2:0/28049195 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f800319d0 con 0x55ea29b9f9d0 2024-09-10T08:14:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.974+0000 7f8f7fe00640 1 -- 172.21.15.2:0/28049195 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80035040 con 0x55ea29b9f9d0 2024-09-10T08:14:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.974+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/28049195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 msgr2=0x55ea29bb07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.974+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/28049195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 0x55ea29bb07b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8f80031440 tx=0x7f8f80032f60 comp rx=0 tx=0).stop 2024-09-10T08:14:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.975+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/28049195 shutdown_connections 2024-09-10T08:14:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.975+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/28049195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29ba4a60 0x55ea29b956a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.975+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/28049195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 0x55ea29bb07b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.975+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/28049195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29b9f090 0x55ea29ba9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.975+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/28049195 >> 172.21.15.2:0/28049195 conn(0x55ea29b9a700 msgr2=0x55ea29ba9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.975+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/28049195 shutdown_connections 2024-09-10T08:14:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.975+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/28049195 wait complete. 2024-09-10T08:14:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.975+0000 7f8f9d2b1740 1 -- start start 2024-09-10T08:14:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.975+0000 7f8f9d2b1740 1 Processor -- start 2024-09-10T08:14:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f9d2b1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29b9f090 0x55ea29bad630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f9d2b1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 0x55ea29badb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f9d2b1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29ba4a60 0x55ea29c1d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f9d2b1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8f941ab000 con 0x55ea29b9f9d0 2024-09-10T08:14:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f9d2b1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f941ac470 con 0x55ea29ba4a60 2024-09-10T08:14:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f9d2b1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f941ab6d0 con 0x55ea29b9f090 2024-09-10T08:14:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f9d2b1740 10 client.0 _ll_register_callbacks cb 0x7f8f840d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 0x55ea29badb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f92a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 0x55ea29badb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50924/0 (socket says 172.21.15.2:50924) 2024-09-10T08:14:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f92a00640 1 -- 172.21.15.2:0/959521052 learned_addr learned my addr 172.21.15.2:0/959521052 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f93400640 1 --2- 172.21.15.2:0/959521052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29b9f090 0x55ea29bad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f93e00640 1 --2- 172.21.15.2:0/959521052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29ba4a60 0x55ea29c1d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f92a00640 1 -- 172.21.15.2:0/959521052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29b9f090 msgr2=0x55ea29bad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f92a00640 1 --2- 172.21.15.2:0/959521052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29b9f090 0x55ea29bad630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f92a00640 1 -- 172.21.15.2:0/959521052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29ba4a60 msgr2=0x55ea29c1d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f92a00640 1 --2- 172.21.15.2:0/959521052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29ba4a60 0x55ea29c1d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.976+0000 7f8f92a00640 1 -- 172.21.15.2:0/959521052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea29c1df00 con 0x55ea29b9f9d0 2024-09-10T08:14:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.977+0000 7f8f92a00640 1 --2- 172.21.15.2:0/959521052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 0x55ea29badb70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8f8002a9a0 tx=0x7f8f80031910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365018b9afcf0ec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.977+0000 7f8f7d600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.977+0000 7f8f7d600640 1 -- 172.21.15.2:0/959521052 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8003f070 con 0x55ea29b9f9d0 2024-09-10T08:14:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.977+0000 7f8f7d600640 1 -- 172.21.15.2:0/959521052 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8f800037f0 con 0x55ea29b9f9d0 2024-09-10T08:14:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.977+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/959521052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f800319d0 con 0x55ea29b9f9d0 2024-09-10T08:14:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.977+0000 7f8f7d600640 1 -- 172.21.15.2:0/959521052 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f80035070 con 0x55ea29b9f9d0 2024-09-10T08:14:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.977+0000 7f8f9d2b1740 10 client.17169 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.977+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/959521052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ea29c1f0f0 con 0x55ea29b9f9d0 2024-09-10T08:14:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.977+0000 7f8f7d600640 1 -- 172.21.15.2:0/959521052 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f800319d0 con 0x55ea29b9f9d0 2024-09-10T08:14:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.978+0000 7f8f7d600640 1 -- 172.21.15.2:0/959521052 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8f8004dad0 con 0x55ea29b9f9d0 2024-09-10T08:14:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.978+0000 7f8f9d2b1740 10 client.17169 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.978+0000 7f8f9d2b1740 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.978+0000 7f8f9d2b1740 20 client.17169 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.978+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/959521052 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea29c1f540 0x55ea29c21960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.978+0000 7f8f9d2b1740 4 client.17169 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:14:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.978+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/959521052 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea29b95d80 con 0x55ea29c1f540 2024-09-10T08:14:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.978+0000 7f8f93400640 1 --2- 172.21.15.2:0/959521052 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea29c1f540 0x55ea29c21960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.979+0000 7f8f93400640 1 --2- 172.21.15.2:0/959521052 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea29c1f540 0x55ea29c21960 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4078c828715ab02 server_cookie=575933c46547e47f in_seq=0 out_seq=0 2024-09-10T08:14:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.979+0000 7f8f7d600640 10 client.17169 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f7d600640 1 -- 172.21.15.2:0/959521052 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ea29b95d80 con 0x55ea29c1f540 2024-09-10T08:14:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f7d600640 10 client.17169 handle_command_reply: tid=0 2024-09-10T08:14:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f9d2b1740 1 client.17169 shutdown 2024-09-10T08:14:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/959521052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 msgr2=0x55ea29badb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/959521052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 0x55ea29badb70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8f8002a9a0 tx=0x7f8f80031910 comp rx=0 tx=0).stop 2024-09-10T08:14:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/959521052 shutdown_connections 2024-09-10T08:14:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/959521052 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ea29c1f540 0x55ea29c21960 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/959521052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea29ba4a60 0x55ea29c1d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/959521052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea29b9f9d0 0x55ea29badb70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f9d2b1740 1 --2- 172.21.15.2:0/959521052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea29b9f090 0x55ea29bad630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f7d600640 0 client.17169 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/959521052 >> 172.21.15.2:0/959521052 conn(0x55ea29b9a700 msgr2=0x55ea29ba9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/959521052 shutdown_connections 2024-09-10T08:14:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f9d2b1740 1 -- 172.21.15.2:0/959521052 wait complete. 2024-09-10T08:14:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.994+0000 7f8f9d2b1740 20 client.17169 trim_cache size 0 max 16384 2024-09-10T08:14:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:14:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:14:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:14:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:14:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.995+0000 7f8f9a400640 1 -- 172.21.15.2:0/410928938 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8f780a2cb0 msgr2=0x7f8f780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.995+0000 7f8f9a400640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8f780a2cb0 0x7f8f780a5170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009fd0 tx=0x7f8f8c009500 comp rx=0 tx=0).stop 2024-09-10T08:14:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.995+0000 7f8f9a400640 1 -- 172.21.15.2:0/410928938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f94074bf0 msgr2=0x7f8f94199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.995+0000 7f8f9a400640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f94074bf0 0x7f8f94199fd0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400e9f0 tx=0x7f8f8400edf0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.995+0000 7f8f9a400640 1 -- 172.21.15.2:0/410928938 shutdown_connections 2024-09-10T08:14:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.995+0000 7f8f9a400640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f941099d0 0x7f8f941a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.995+0000 7f8f9a400640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8f780a2cb0 0x7f8f780a5170 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.995+0000 7f8f9a400640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f940761d0 0x7f8f9419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.995+0000 7f8f9a400640 1 --2- 172.21.15.2:0/410928938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f94074bf0 0x7f8f94199fd0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.995+0000 7f8f9a400640 1 -- 172.21.15.2:0/410928938 >> 172.21.15.2:0/410928938 conn(0x7f8f940fd2e0 msgr2=0x7f8f94105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.995+0000 7f8f9a400640 1 -- 172.21.15.2:0/410928938 shutdown_connections 2024-09-10T08:14:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:38.995+0000 7f8f9a400640 1 -- 172.21.15.2:0/410928938 wait complete. 2024-09-10T08:14:39.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:38 smithi002 ceph-mon[30563]: pgmap v582: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:14:39.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:38 smithi155 ceph-mon[35405]: pgmap v582: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:14:39.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:38 smithi107 ceph-mon[35048]: pgmap v582: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:14:40.272 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:39 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:14:40.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:39 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:14:40.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:39 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:14:41.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: pgmap v583: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:14:41.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi155 to 176.0M 2024-09-10T08:14:41.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi107 to 175.9M 2024-09-10T08:14:41.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: Unable to set osd_memory_target on smithi155 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T08:14:41.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: Unable to set osd_memory_target on smithi107 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-10T08:14:41.272 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:14:41.272 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:14:41.272 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:40 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:14:41.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: pgmap v583: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:14:41.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi155 to 176.0M 2024-09-10T08:14:41.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi107 to 175.9M 2024-09-10T08:14:41.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: Unable to set osd_memory_target on smithi155 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T08:14:41.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: Unable to set osd_memory_target on smithi107 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-10T08:14:41.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:14:41.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:14:41.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:40 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:14:41.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: pgmap v583: 129 pgs: 129 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:14:41.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:14:41.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi155 to 176.0M 2024-09-10T08:14:41.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi107 to 175.9M 2024-09-10T08:14:41.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: Unable to set osd_memory_target on smithi155 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T08:14:41.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: Unable to set osd_memory_target on smithi107 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-10T08:14:41.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:14:41.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:14:41.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:40 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:14:43.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:42 smithi002 ceph-mon[30563]: pgmap v584: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-10T08:14:43.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:42 smithi155 ceph-mon[35405]: pgmap v584: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-10T08:14:43.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:42 smithi107 ceph-mon[35048]: pgmap v584: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-10T08:14:45.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:44 smithi002 ceph-mon[30563]: pgmap v585: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-09-10T08:14:45.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:44 smithi155 ceph-mon[35405]: pgmap v585: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-09-10T08:14:45.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:44 smithi107 ceph-mon[35048]: pgmap v585: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-09-10T08:14:47.219 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:46 smithi107 ceph-mon[35048]: pgmap v586: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-09-10T08:14:47.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:46 smithi002 ceph-mon[30563]: pgmap v586: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-09-10T08:14:47.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:46 smithi155 ceph-mon[35405]: pgmap v586: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-09-10T08:14:49.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:48 smithi002 ceph-mon[30563]: pgmap v587: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-09-10T08:14:49.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:48 smithi155 ceph-mon[35405]: pgmap v587: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-09-10T08:14:49.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:48 smithi107 ceph-mon[35048]: pgmap v587: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-09-10T08:14:51.090 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:14:51.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:50 smithi002 ceph-mon[30563]: pgmap v588: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-10T08:14:51.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:50 smithi155 ceph-mon[35405]: pgmap v588: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-10T08:14:51.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:50 smithi107 ceph-mon[35048]: pgmap v588: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-10T08:14:51.494 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.116+0000 7f42f8400640 1 -- 172.21.15.2:0/418029992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42f01004e0 msgr2=0x7f42f01008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.116+0000 7f42f8400640 1 --2- 172.21.15.2:0/418029992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42f01004e0 0x7f42f01008e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f42e400aa40 tx=0x7f42e402b750 comp rx=0 tx=0).stop 2024-09-10T08:14:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.116+0000 7f42f8400640 1 -- 172.21.15.2:0/418029992 shutdown_connections 2024-09-10T08:14:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.116+0000 7f42f8400640 1 --2- 172.21.15.2:0/418029992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0101d40 0x7f42f01085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.116+0000 7f42f8400640 1 --2- 172.21.15.2:0/418029992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f0101380 0x7f42f0101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.116+0000 7f42f8400640 1 --2- 172.21.15.2:0/418029992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42f01004e0 0x7f42f01008e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.116+0000 7f42f8400640 1 -- 172.21.15.2:0/418029992 >> 172.21.15.2:0/418029992 conn(0x7f42f00fb500 msgr2=0x7f42f00fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.116+0000 7f42f8400640 1 -- 172.21.15.2:0/418029992 shutdown_connections 2024-09-10T08:14:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.116+0000 7f42f8400640 1 -- 172.21.15.2:0/418029992 wait complete. 2024-09-10T08:14:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.117+0000 7f42f8400640 1 Processor -- start 2024-09-10T08:14:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.117+0000 7f42f8400640 1 -- start start 2024-09-10T08:14:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.118+0000 7f42f8400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42f01004e0 0x7f42f019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.118+0000 7f42f8400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0101380 0x7f42f019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.118+0000 7f42f8400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f0101d40 0x7f42f01a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.118+0000 7f42f8400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f42f010b640 con 0x7f42f0101380 2024-09-10T08:14:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.118+0000 7f42f8400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f42f010b4c0 con 0x7f42f01004e0 2024-09-10T08:14:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.118+0000 7f42f8400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f42f010b7c0 con 0x7f42f0101d40 2024-09-10T08:14:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.118+0000 7f42f5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0101380 0x7f42f019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.118+0000 7f42f5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0101380 0x7f42f019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60014/0 (socket says 172.21.15.2:60014) 2024-09-10T08:14:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.118+0000 7f42f5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42f01004e0 0x7f42f019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.118+0000 7f42f5000640 1 -- 172.21.15.2:0/603736081 learned_addr learned my addr 172.21.15.2:0/603736081 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.119+0000 7f42f5000640 1 -- 172.21.15.2:0/603736081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f0101d40 msgr2=0x7f42f01a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.119+0000 7f42f5000640 1 --2- 172.21.15.2:0/603736081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f0101d40 0x7f42f01a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.119+0000 7f42f5000640 1 -- 172.21.15.2:0/603736081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42f01004e0 msgr2=0x7f42f019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.119+0000 7f42f5000640 1 --2- 172.21.15.2:0/603736081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42f01004e0 0x7f42f019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.119+0000 7f42f5000640 1 -- 172.21.15.2:0/603736081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f01a1ca0 con 0x7f42f0101380 2024-09-10T08:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.119+0000 7f42f5a00640 1 --2- 172.21.15.2:0/603736081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42f01004e0 0x7f42f019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.119+0000 7f42f5000640 1 --2- 172.21.15.2:0/603736081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0101380 0x7f42f019a5c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00e9f0 tx=0x7f42dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.119+0000 7f42ee000640 1 -- 172.21.15.2:0/603736081 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc00cd80 con 0x7f42f0101380 2024-09-10T08:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.119+0000 7f42ee000640 1 -- 172.21.15.2:0/603736081 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42dc005500 con 0x7f42f0101380 2024-09-10T08:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.119+0000 7f42ee000640 1 -- 172.21.15.2:0/603736081 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc010690 con 0x7f42f0101380 2024-09-10T08:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.120+0000 7f42f8400640 1 -- 172.21.15.2:0/603736081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f01a1f90 con 0x7f42f0101380 2024-09-10T08:14:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.120+0000 7f42f8400640 1 -- 172.21.15.2:0/603736081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f01a2400 con 0x7f42f0101380 2024-09-10T08:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.124+0000 7f42fb45e740 1 Processor -- start 2024-09-10T08:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.124+0000 7f42fb45e740 1 -- start start 2024-09-10T08:14:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42fb45e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf21090 0x5653cbf21490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42fb45e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf219d0 0x5653cbf26490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42fb45e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf28f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42fb45e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5653cbe86100 con 0x5653cbf26a60 2024-09-10T08:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42fb45e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5653cbe2e1b0 con 0x5653cbf219d0 2024-09-10T08:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42fb45e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5653cbe820c0 con 0x5653cbf21090 2024-09-10T08:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42f6400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf21090 0x5653cbf21490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42f6400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf21090 0x5653cbf21490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:46096/0 (socket says 172.21.15.2:46096) 2024-09-10T08:14:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42f6400640 1 -- 172.21.15.2:0/450757860 learned_addr learned my addr 172.21.15.2:0/450757860 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42f5a00640 1 --2- 172.21.15.2:0/450757860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf219d0 0x5653cbf26490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42f6400640 1 -- 172.21.15.2:0/450757860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf219d0 msgr2=0x5653cbf26490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42f6400640 1 --2- 172.21.15.2:0/450757860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf219d0 0x5653cbf26490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42f6400640 1 -- 172.21.15.2:0/450757860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 msgr2=0x5653cbf28f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:14:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42f6400640 1 --2- 172.21.15.2:0/450757860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf28f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.125+0000 7f42f6400640 1 -- 172.21.15.2:0/450757860 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653cbb99210 con 0x5653cbf21090 2024-09-10T08:14:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.126+0000 7f42ee000640 1 -- 172.21.15.2:0/603736081 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f42dc0046c0 con 0x7f42f0101380 2024-09-10T08:14:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.126+0000 7f42f6400640 1 --2- 172.21.15.2:0/450757860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf21090 0x5653cbf21490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009ea0 tx=0x7f42e8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3f16e91cf621618 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.126+0000 7f42e2a00640 1 -- 172.21.15.2:0/450757860 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8004d20 con 0x5653cbf21090 2024-09-10T08:14:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.126+0000 7f42e2a00640 1 -- 172.21.15.2:0/450757860 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42e800ee60 con 0x5653cbf21090 2024-09-10T08:14:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.126+0000 7f42e2a00640 1 -- 172.21.15.2:0/450757860 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e80108a0 con 0x5653cbf21090 2024-09-10T08:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.127+0000 7f42fb45e740 1 -- 172.21.15.2:0/450757860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf21090 msgr2=0x5653cbf21490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.127+0000 7f42fb45e740 1 --2- 172.21.15.2:0/450757860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf21090 0x5653cbf21490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009ea0 tx=0x7f42e8009f90 comp rx=0 tx=0).stop 2024-09-10T08:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.127+0000 7f42fb45e740 1 -- 172.21.15.2:0/450757860 shutdown_connections 2024-09-10T08:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.127+0000 7f42fb45e740 1 --2- 172.21.15.2:0/450757860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf28f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.127+0000 7f42fb45e740 1 --2- 172.21.15.2:0/450757860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf219d0 0x5653cbf26490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.127+0000 7f42fb45e740 1 --2- 172.21.15.2:0/450757860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf21090 0x5653cbf21490 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.127+0000 7f42fb45e740 1 -- 172.21.15.2:0/450757860 >> 172.21.15.2:0/450757860 conn(0x5653cbf1c700 msgr2=0x5653cbf1cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.127+0000 7f42fb45e740 1 -- 172.21.15.2:0/450757860 shutdown_connections 2024-09-10T08:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.127+0000 7f42fb45e740 1 -- 172.21.15.2:0/450757860 wait complete. 2024-09-10T08:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.128+0000 7f42fb45e740 1 -- start start 2024-09-10T08:14:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.128+0000 7f42fb45e740 1 Processor -- start 2024-09-10T08:14:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.128+0000 7f42ee000640 1 --2- 172.21.15.2:0/603736081 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f42d80a2bc0 0x7f42d80a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.128+0000 7f42ee000640 1 -- 172.21.15.2:0/603736081 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f42dc0cf8c0 con 0x7f42f0101380 2024-09-10T08:14:52.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.129+0000 7f42fb45e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf21090 0x5653cbf33a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.129+0000 7f42f6400640 1 --2- 172.21.15.2:0/603736081 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f42d80a2bc0 0x7f42d80a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.129+0000 7f42fb45e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf219d0 0x5653cbf33fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.129+0000 7f42fb45e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf3b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.129+0000 7f42fb45e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5653cbf29a30 con 0x5653cbf26a60 2024-09-10T08:14:52.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.129+0000 7f42fb45e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5653cbe0f090 con 0x5653cbf21090 2024-09-10T08:14:52.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.129+0000 7f42fb45e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5653cbf29bb0 con 0x5653cbf219d0 2024-09-10T08:14:52.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.129+0000 7f42fb45e740 10 client.0 _ll_register_callbacks cb 0x5653cbe63f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:52.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.129+0000 7f42f5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf21090 0x5653cbf33a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.129+0000 7f42f6400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf3b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.129+0000 7f42f5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf21090 0x5653cbf33a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38848/0 (socket says 172.21.15.2:38848) 2024-09-10T08:14:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.129+0000 7f42f5a00640 1 -- 172.21.15.2:0/1654480918 learned_addr learned my addr 172.21.15.2:0/1654480918 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.129+0000 7f42f5000640 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf219d0 0x5653cbf33fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.130+0000 7f42f5a00640 1 -- 172.21.15.2:0/1654480918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf219d0 msgr2=0x5653cbf33fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.130+0000 7f42f5a00640 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf219d0 0x5653cbf33fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.130+0000 7f42f5a00640 1 -- 172.21.15.2:0/1654480918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 msgr2=0x5653cbf3b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.130+0000 7f42f5a00640 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf3b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.130+0000 7f42f5a00640 1 -- 172.21.15.2:0/1654480918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653cbc65620 con 0x5653cbf21090 2024-09-10T08:14:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.130+0000 7f42f5000640 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf219d0 0x5653cbf33fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.130+0000 7f42f6400640 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf3b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.130+0000 7f42f5a00640 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf21090 0x5653cbf33a90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f42e4030720 tx=0x7f42e400a6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fdc9c3d79bb0163 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.130+0000 7f42d7400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:14:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.130+0000 7f42f6400640 1 --2- 172.21.15.2:0/603736081 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f42d80a2bc0 0x7f42d80a5080 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009ce0 tx=0x7f42e8009ae0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.131+0000 7f42d7400640 1 -- 172.21.15.2:0/1654480918 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4036040 con 0x5653cbf21090 2024-09-10T08:14:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.131+0000 7f42fb45e740 1 -- 172.21.15.2:0/1654480918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42e800ee60 con 0x5653cbf21090 2024-09-10T08:14:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.131+0000 7f42d7400640 1 -- 172.21.15.2:0/1654480918 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f42e4004b90 con 0x5653cbf21090 2024-09-10T08:14:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.131+0000 7f42d7400640 1 -- 172.21.15.2:0/1654480918 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42e404f050 con 0x5653cbf21090 2024-09-10T08:14:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.131+0000 7f42fb45e740 10 client.27227 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.131+0000 7f42fb45e740 1 -- 172.21.15.2:0/1654480918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5653cbf3dea0 con 0x5653cbf21090 2024-09-10T08:14:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.131+0000 7f42d7400640 1 -- 172.21.15.2:0/1654480918 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42e800ee60 con 0x5653cbf21090 2024-09-10T08:14:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.132+0000 7f42d7400640 1 -- 172.21.15.2:0/1654480918 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f42e403b040 con 0x5653cbf21090 2024-09-10T08:14:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.132+0000 7f42fb45e740 10 client.27227 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.132+0000 7f42fb45e740 10 client.27227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.132+0000 7f42fb45e740 20 client.27227 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.132+0000 7f42fb45e740 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5653cbf3f500 0x5653cbf418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.132+0000 7f42fb45e740 4 client.27227 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.132+0000 7f42fb45e740 1 -- 172.21.15.2:0/1654480918 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653cbe93830 con 0x5653cbf3f500 2024-09-10T08:14:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.132+0000 7f42f5000640 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5653cbf3f500 0x5653cbf418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.133+0000 7f42f5000640 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5653cbf3f500 0x5653cbf418f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c525f18967a0967 server_cookie=ee4bff8cc5c08aae in_seq=0 out_seq=0 2024-09-10T08:14:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.133+0000 7f42d7400640 10 client.27227 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42d7400640 1 -- 172.21.15.2:0/1654480918 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653cbe93830 con 0x5653cbf3f500 2024-09-10T08:14:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42d7400640 10 client.27227 handle_command_reply: tid=0 2024-09-10T08:14:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42fb45e740 1 client.27227 shutdown 2024-09-10T08:14:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42fb45e740 1 -- 172.21.15.2:0/1654480918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf21090 msgr2=0x5653cbf33a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42fb45e740 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf21090 0x5653cbf33a90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f42e4030720 tx=0x7f42e400a6b0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42fb45e740 1 -- 172.21.15.2:0/1654480918 shutdown_connections 2024-09-10T08:14:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42fb45e740 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5653cbf3f500 0x5653cbf418f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42fb45e740 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf3b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42fb45e740 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf219d0 0x5653cbf33fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42d7400640 0 client.27227 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42fb45e740 1 --2- 172.21.15.2:0/1654480918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf21090 0x5653cbf33a90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42fb45e740 1 -- 172.21.15.2:0/1654480918 >> 172.21.15.2:0/1654480918 conn(0x5653cbf1c700 msgr2=0x5653cbf1df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42fb45e740 1 -- 172.21.15.2:0/1654480918 shutdown_connections 2024-09-10T08:14:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.134+0000 7f42fb45e740 1 -- 172.21.15.2:0/1654480918 wait complete. 2024-09-10T08:14:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.135+0000 7f42fb45e740 20 client.27227 trim_cache size 0 max 16384 2024-09-10T08:14:52.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.120+0000 7efdf0a00640 1 -- 172.21.15.2:0/3394418943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdec1019c0 msgr2=0x7efdec101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.120+0000 7efdf0a00640 1 --2- 172.21.15.2:0/3394418943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdec1019c0 0x7efdec101e20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009930 tx=0x7efdd802a420 comp rx=0 tx=0).stop 2024-09-10T08:14:52.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.121+0000 7efdf0a00640 1 -- 172.21.15.2:0/3394418943 shutdown_connections 2024-09-10T08:14:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.121+0000 7efdf0a00640 1 --2- 172.21.15.2:0/3394418943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdec102360 0x7efdec10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.121+0000 7efdf0a00640 1 --2- 172.21.15.2:0/3394418943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdec1019c0 0x7efdec101e20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.121+0000 7efdf0a00640 1 --2- 172.21.15.2:0/3394418943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdec1079c0 0x7efdec107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.121+0000 7efdf0a00640 1 -- 172.21.15.2:0/3394418943 >> 172.21.15.2:0/3394418943 conn(0x7efdec0fd740 msgr2=0x7efdec0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.121+0000 7efdf0a00640 1 -- 172.21.15.2:0/3394418943 shutdown_connections 2024-09-10T08:14:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.121+0000 7efdf0a00640 1 -- 172.21.15.2:0/3394418943 wait complete. 2024-09-10T08:14:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.122+0000 7efdf0a00640 1 Processor -- start 2024-09-10T08:14:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.122+0000 7efdf0a00640 1 -- start start 2024-09-10T08:14:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.123+0000 7efdf0a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdec1019c0 0x7efdec19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.123+0000 7efdf0a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdec102360 0x7efdec19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.123+0000 7efdf0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdec1079c0 0x7efdec19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.123+0000 7efdf0a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efdec111d90 con 0x7efdec1019c0 2024-09-10T08:14:52.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.123+0000 7efdf0a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efdec111c10 con 0x7efdec1079c0 2024-09-10T08:14:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.123+0000 7efdf0a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efdec111f10 con 0x7efdec102360 2024-09-10T08:14:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.123+0000 7efdeaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdec102360 0x7efdec19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.123+0000 7efdeaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdec102360 0x7efdec19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:46080/0 (socket says 172.21.15.2:46080) 2024-09-10T08:14:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.123+0000 7efdeaa00640 1 -- 172.21.15.2:0/3246919295 learned_addr learned my addr 172.21.15.2:0/3246919295 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.123+0000 7efdeb400640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdec1019c0 0x7efdec19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.123+0000 7efdebe00640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdec1079c0 0x7efdec19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.124+0000 7efdeaa00640 1 -- 172.21.15.2:0/3246919295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdec1079c0 msgr2=0x7efdec19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.124+0000 7efdeaa00640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdec1079c0 0x7efdec19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.124+0000 7efdeaa00640 1 -- 172.21.15.2:0/3246919295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdec1019c0 msgr2=0x7efdec19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.124+0000 7efdeaa00640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdec1019c0 0x7efdec19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.124+0000 7efdeaa00640 1 -- 172.21.15.2:0/3246919295 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec19f450 con 0x7efdec102360 2024-09-10T08:14:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.124+0000 7efdeaa00640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdec102360 0x7efdec19a9e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009230 tx=0x7efdd8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:52.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.124+0000 7efde3e00640 1 -- 172.21.15.2:0/3246919295 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd803f030 con 0x7efdec102360 2024-09-10T08:14:52.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.125+0000 7efdf0a00640 1 -- 172.21.15.2:0/3246919295 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdec19f6e0 con 0x7efdec102360 2024-09-10T08:14:52.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.125+0000 7efdf0a00640 1 -- 172.21.15.2:0/3246919295 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdec1a7020 con 0x7efdec102360 2024-09-10T08:14:52.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.125+0000 7efde3e00640 1 -- 172.21.15.2:0/3246919295 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd8004440 con 0x7efdec102360 2024-09-10T08:14:52.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.125+0000 7efde3e00640 1 -- 172.21.15.2:0/3246919295 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8035070 con 0x7efdec102360 2024-09-10T08:14:52.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.126+0000 7efdf0a00640 1 -- 172.21.15.2:0/3246919295 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdcc005290 con 0x7efdec102360 2024-09-10T08:14:52.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.129+0000 7efde3e00640 1 -- 172.21.15.2:0/3246919295 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7efdd8042030 con 0x7efdec102360 2024-09-10T08:14:52.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.131+0000 7efde3e00640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efdd00a2c00 0x7efdd00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.131+0000 7efde3e00640 1 -- 172.21.15.2:0/3246919295 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7efdd80ef250 con 0x7efdec102360 2024-09-10T08:14:52.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.131+0000 7efdeb400640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efdd00a2c00 0x7efdd00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.132+0000 7efdeb400640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efdd00a2c00 0x7efdd00a50c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efddc0095f0 tx=0x7efddc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:52.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.135+0000 7efde3e00640 1 -- 172.21.15.2:0/3246919295 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efdd80bb710 con 0x7efdec102360 2024-09-10T08:14:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.150+0000 7f42fb45e740 1 Processor -- start 2024-09-10T08:14:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.150+0000 7f42fb45e740 1 -- start start 2024-09-10T08:14:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.150+0000 7f42fb45e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf21090 0x5653cbf21470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42fb45e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf219d0 0x5653cbf33eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42fb45e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf3afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42fb45e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01ab090 con 0x5653cbf26a60 2024-09-10T08:14:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42fb45e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01a8960 con 0x5653cbf219d0 2024-09-10T08:14:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42fb45e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01a67b0 con 0x5653cbf21090 2024-09-10T08:14:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42f6400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf3afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42f6400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf3afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60030/0 (socket says 172.21.15.2:60030) 2024-09-10T08:14:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42f6400640 1 -- 172.21.15.2:0/3435359031 learned_addr learned my addr 172.21.15.2:0/3435359031 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42f5000640 1 --2- 172.21.15.2:0/3435359031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf219d0 0x5653cbf33eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42f6400640 1 -- 172.21.15.2:0/3435359031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf21090 msgr2=0x5653cbf21470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42f6400640 1 --2- 172.21.15.2:0/3435359031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf21090 0x5653cbf21470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42f6400640 1 -- 172.21.15.2:0/3435359031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf219d0 msgr2=0x5653cbf33eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42f6400640 1 --2- 172.21.15.2:0/3435359031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf219d0 0x5653cbf33eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.151+0000 7f42f6400640 1 -- 172.21.15.2:0/3435359031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653cbf17d60 con 0x5653cbf26a60 2024-09-10T08:14:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.152+0000 7f42f6400640 1 --2- 172.21.15.2:0/3435359031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf3afd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f42e8010480 tx=0x7f42e8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c824464c6ca1fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.152+0000 7f42e2000640 1 -- 172.21.15.2:0/3435359031 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8038070 con 0x5653cbf26a60 2024-09-10T08:14:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.152+0000 7f42e2000640 1 -- 172.21.15.2:0/3435359031 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42e8004b20 con 0x5653cbf26a60 2024-09-10T08:14:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.152+0000 7f42e2000640 1 -- 172.21.15.2:0/3435359031 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e801d040 con 0x5653cbf26a60 2024-09-10T08:14:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.153+0000 7f42fb45e740 1 -- 172.21.15.2:0/3435359031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 msgr2=0x5653cbf3afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.153+0000 7f42fb45e740 1 --2- 172.21.15.2:0/3435359031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf3afd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f42e8010480 tx=0x7f42e8009f90 comp rx=0 tx=0).stop 2024-09-10T08:14:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.153+0000 7f42fb45e740 1 -- 172.21.15.2:0/3435359031 shutdown_connections 2024-09-10T08:14:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.153+0000 7f42fb45e740 1 --2- 172.21.15.2:0/3435359031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf26a60 0x5653cbf3afd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.153+0000 7f42fb45e740 1 --2- 172.21.15.2:0/3435359031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf219d0 0x5653cbf33eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.153+0000 7f42fb45e740 1 --2- 172.21.15.2:0/3435359031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf21090 0x5653cbf21470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.153+0000 7f42fb45e740 1 -- 172.21.15.2:0/3435359031 >> 172.21.15.2:0/3435359031 conn(0x5653cbf1c700 msgr2=0x5653cbf1b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.153+0000 7f42fb45e740 1 -- 172.21.15.2:0/3435359031 shutdown_connections 2024-09-10T08:14:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.153+0000 7f42fb45e740 1 -- 172.21.15.2:0/3435359031 wait complete. 2024-09-10T08:14:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.154+0000 7f42fb45e740 1 -- start start 2024-09-10T08:14:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.154+0000 7f42fb45e740 1 Processor -- start 2024-09-10T08:14:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.155+0000 7f42fb45e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf21090 0x5653cbf38450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.155+0000 7f42fb45e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf219d0 0x5653cbf38990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.155+0000 7f42fb45e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf26a60 0x5653cbf399a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.155+0000 7f42fb45e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01a6e20 con 0x5653cbf219d0 2024-09-10T08:14:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.155+0000 7f42fb45e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01a8290 con 0x5653cbf21090 2024-09-10T08:14:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.155+0000 7f42fb45e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f42f01a74f0 con 0x5653cbf26a60 2024-09-10T08:14:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.155+0000 7f42fb45e740 10 client.0 _ll_register_callbacks cb 0x7f42dc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.155+0000 7f42f5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf219d0 0x5653cbf38990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.155+0000 7f42f5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf219d0 0x5653cbf38990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60044/0 (socket says 172.21.15.2:60044) 2024-09-10T08:14:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.155+0000 7f42f5000640 1 -- 172.21.15.2:0/2496908445 learned_addr learned my addr 172.21.15.2:0/2496908445 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.155+0000 7f42f5a00640 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf21090 0x5653cbf38450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.155+0000 7f42f6400640 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf26a60 0x5653cbf399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.156+0000 7f42f5a00640 1 -- 172.21.15.2:0/2496908445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf26a60 msgr2=0x5653cbf399a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.156+0000 7f42f5a00640 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf26a60 0x5653cbf399a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.156+0000 7f42f5a00640 1 -- 172.21.15.2:0/2496908445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf219d0 msgr2=0x5653cbf38990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.156+0000 7f42f5a00640 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf219d0 0x5653cbf38990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.156+0000 7f42f5a00640 1 -- 172.21.15.2:0/2496908445 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653cbf39fb0 con 0x5653cbf21090 2024-09-10T08:14:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.156+0000 7f42f5000640 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf219d0 0x5653cbf38990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.156+0000 7f42f5a00640 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf21090 0x5653cbf38450 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f42e40305f0 tx=0x7f42e4003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc2ff65f70f2e454 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.156+0000 7f42d7400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:14:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.156+0000 7f42d7400640 1 -- 172.21.15.2:0/2496908445 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4058060 con 0x5653cbf21090 2024-09-10T08:14:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.156+0000 7f42d7400640 1 -- 172.21.15.2:0/2496908445 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f42e4004da0 con 0x5653cbf21090 2024-09-10T08:14:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.156+0000 7f42fb45e740 1 -- 172.21.15.2:0/2496908445 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42e8004b20 con 0x5653cbf21090 2024-09-10T08:14:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.157+0000 7f42d7400640 1 -- 172.21.15.2:0/2496908445 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42e405f050 con 0x5653cbf21090 2024-09-10T08:14:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.157+0000 7f42fb45e740 10 client.27233 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.157+0000 7f42fb45e740 1 -- 172.21.15.2:0/2496908445 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5653cbf3b2c0 con 0x5653cbf21090 2024-09-10T08:14:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.157+0000 7f42d7400640 1 -- 172.21.15.2:0/2496908445 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42e8004b20 con 0x5653cbf21090 2024-09-10T08:14:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.157+0000 7f42d7400640 1 -- 172.21.15.2:0/2496908445 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f42e400b040 con 0x5653cbf21090 2024-09-10T08:14:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.157+0000 7f42fb45e740 10 client.27233 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.157+0000 7f42fb45e740 10 client.27233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.157+0000 7f42fb45e740 20 client.27233 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.157+0000 7f42fb45e740 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5653cbfaa840 0x5653cbfacc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.157+0000 7f42fb45e740 4 client.27233 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:14:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.157+0000 7f42fb45e740 1 -- 172.21.15.2:0/2496908445 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653cbe8db80 con 0x5653cbfaa840 2024-09-10T08:14:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.158+0000 7f42f5000640 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5653cbfaa840 0x5653cbfacc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.158+0000 7f42f5000640 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5653cbfaa840 0x5653cbfacc60 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6be15e298290a34 server_cookie=f38eacdae9b5d445 in_seq=0 out_seq=0 2024-09-10T08:14:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.158+0000 7f42d7400640 10 client.27233 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.171+0000 7f42d7400640 1 -- 172.21.15.2:0/2496908445 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653cbe8db80 con 0x5653cbfaa840 2024-09-10T08:14:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.171+0000 7f42d7400640 10 client.27233 handle_command_reply: tid=0 2024-09-10T08:14:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.172+0000 7f42fb45e740 1 client.27233 shutdown 2024-09-10T08:14:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.172+0000 7f42fb45e740 1 -- 172.21.15.2:0/2496908445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf21090 msgr2=0x5653cbf38450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.172+0000 7f42fb45e740 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf21090 0x5653cbf38450 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f42e40305f0 tx=0x7f42e4003f90 comp rx=0 tx=0).stop 2024-09-10T08:14:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.172+0000 7f42fb45e740 1 -- 172.21.15.2:0/2496908445 shutdown_connections 2024-09-10T08:14:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.172+0000 7f42fb45e740 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5653cbfaa840 0x5653cbfacc60 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.172+0000 7f42fb45e740 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653cbf26a60 0x5653cbf399a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.172+0000 7f42fb45e740 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653cbf219d0 0x5653cbf38990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.172+0000 7f42d7400640 0 client.27233 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.172+0000 7f42fb45e740 1 --2- 172.21.15.2:0/2496908445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653cbf21090 0x5653cbf38450 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.172+0000 7f42fb45e740 1 -- 172.21.15.2:0/2496908445 >> 172.21.15.2:0/2496908445 conn(0x5653cbf1c700 msgr2=0x5653cbf1cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.172+0000 7f42fb45e740 1 -- 172.21.15.2:0/2496908445 shutdown_connections 2024-09-10T08:14:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.172+0000 7f42fb45e740 1 -- 172.21.15.2:0/2496908445 wait complete. 2024-09-10T08:14:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.172+0000 7f42fb45e740 20 client.27233 trim_cache size 0 max 16384 2024-09-10T08:14:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:14:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:14:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:14:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:14:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.173+0000 7f42f8400640 1 -- 172.21.15.2:0/603736081 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f42d80a2bc0 msgr2=0x7f42d80a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.173+0000 7f42f8400640 1 --2- 172.21.15.2:0/603736081 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f42d80a2bc0 0x7f42d80a5080 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009ce0 tx=0x7f42e8009ae0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.173+0000 7f42f8400640 1 -- 172.21.15.2:0/603736081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0101380 msgr2=0x7f42f019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.173+0000 7f42f8400640 1 --2- 172.21.15.2:0/603736081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0101380 0x7f42f019a5c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00e9f0 tx=0x7f42dc00edf0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.173+0000 7f42f8400640 1 -- 172.21.15.2:0/603736081 shutdown_connections 2024-09-10T08:14:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.173+0000 7f42f8400640 1 --2- 172.21.15.2:0/603736081 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f42d80a2bc0 0x7f42d80a5080 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.173+0000 7f42f8400640 1 --2- 172.21.15.2:0/603736081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f0101d40 0x7f42f01a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.173+0000 7f42f8400640 1 --2- 172.21.15.2:0/603736081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0101380 0x7f42f019a5c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.173+0000 7f42f8400640 1 --2- 172.21.15.2:0/603736081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42f01004e0 0x7f42f019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.173+0000 7f42f8400640 1 -- 172.21.15.2:0/603736081 >> 172.21.15.2:0/603736081 conn(0x7f42f00fb500 msgr2=0x7f42f0104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.173+0000 7f42f8400640 1 -- 172.21.15.2:0/603736081 shutdown_connections 2024-09-10T08:14:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:14:52.174+0000 7f42f8400640 1 -- 172.21.15.2:0/603736081 wait complete. 2024-09-10T08:14:52.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.374+0000 7efdf0a00640 1 -- 172.21.15.2:0/3246919295 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdcc005580 con 0x7efdec102360 2024-09-10T08:14:52.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.375+0000 7efde3e00640 1 -- 172.21.15.2:0/3246919295 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7efdd80bfa10 con 0x7efdec102360 2024-09-10T08:14:52.376 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:14:52.376 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:14:52.376 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:14:52.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.378+0000 7efdf0a00640 1 -- 172.21.15.2:0/3246919295 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efdd00a2c00 msgr2=0x7efdd00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.378+0000 7efdf0a00640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efdd00a2c00 0x7efdd00a50c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efddc0095f0 tx=0x7efddc005f90 comp rx=0 tx=0).stop 2024-09-10T08:14:52.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.378+0000 7efdf0a00640 1 -- 172.21.15.2:0/3246919295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdec102360 msgr2=0x7efdec19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:52.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.378+0000 7efdf0a00640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdec102360 0x7efdec19a9e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009230 tx=0x7efdd8004040 comp rx=0 tx=0).stop 2024-09-10T08:14:52.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.378+0000 7efdf0a00640 1 -- 172.21.15.2:0/3246919295 shutdown_connections 2024-09-10T08:14:52.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.378+0000 7efdf0a00640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdec1079c0 0x7efdec19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.378+0000 7efdf0a00640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efdd00a2c00 0x7efdd00a50c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.378+0000 7efdf0a00640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdec102360 0x7efdec19a9e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.378+0000 7efdf0a00640 1 --2- 172.21.15.2:0/3246919295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdec1019c0 0x7efdec19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:52.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.378+0000 7efdf0a00640 1 -- 172.21.15.2:0/3246919295 >> 172.21.15.2:0/3246919295 conn(0x7efdec0fd740 msgr2=0x7efdec10cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:52.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.378+0000 7efdf0a00640 1 -- 172.21.15.2:0/3246919295 shutdown_connections 2024-09-10T08:14:52.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:52.378+0000 7efdf0a00640 1 -- 172.21.15.2:0/3246919295 wait complete. 2024-09-10T08:14:53.208 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:52 smithi155 ceph-mon[35405]: pgmap v589: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-10T08:14:53.209 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:52 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3246919295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:14:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:52 smithi002 ceph-mon[30563]: pgmap v589: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-10T08:14:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:52 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3246919295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:14:53.314 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id f78d3e3b --release --await' 2024-09-10T08:14:53.314 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id f78d3e3b --release --await 2024-09-10T08:14:53.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:52 smithi107 ceph-mon[35048]: pgmap v589: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-10T08:14:53.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:52 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3246919295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:14:53.740 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:54.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.301+0000 7fb80f000640 1 -- 172.21.15.2:0/1472284703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb808102360 msgr2=0x7fb80810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:54.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.301+0000 7fb80f000640 1 --2- 172.21.15.2:0/1472284703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb808102360 0x7fb80810e890 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb800009cd0 tx=0x7fb80002a8f0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.301+0000 7fb80f000640 1 -- 172.21.15.2:0/1472284703 shutdown_connections 2024-09-10T08:14:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.301+0000 7fb80f000640 1 --2- 172.21.15.2:0/1472284703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb808102360 0x7fb80810e890 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.301+0000 7fb80f000640 1 --2- 172.21.15.2:0/1472284703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8081019c0 0x7fb808101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.301+0000 7fb80f000640 1 --2- 172.21.15.2:0/1472284703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8081079c0 0x7fb808107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.301+0000 7fb80f000640 1 -- 172.21.15.2:0/1472284703 >> 172.21.15.2:0/1472284703 conn(0x7fb8080fd740 msgr2=0x7fb8080ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.301+0000 7fb80f000640 1 -- 172.21.15.2:0/1472284703 shutdown_connections 2024-09-10T08:14:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.301+0000 7fb80f000640 1 -- 172.21.15.2:0/1472284703 wait complete. 2024-09-10T08:14:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.302+0000 7fb80f000640 1 Processor -- start 2024-09-10T08:14:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.302+0000 7fb80f000640 1 -- start start 2024-09-10T08:14:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.303+0000 7fb80f000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8081019c0 0x7fb80819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.303+0000 7fb80f000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb808102360 0x7fb80819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.303+0000 7fb80f000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8081079c0 0x7fb80819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.303+0000 7fb80f000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb808111e10 con 0x7fb8081079c0 2024-09-10T08:14:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.303+0000 7fb80f000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb808111c90 con 0x7fb8081019c0 2024-09-10T08:14:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.303+0000 7fb80f000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb808111f90 con 0x7fb808102360 2024-09-10T08:14:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.303+0000 7fb80e600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8081079c0 0x7fb80819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.303+0000 7fb80e600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8081079c0 0x7fb80819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60060/0 (socket says 172.21.15.2:60060) 2024-09-10T08:14:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.303+0000 7fb80e600640 1 -- 172.21.15.2:0/1954204730 learned_addr learned my addr 172.21.15.2:0/1954204730 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.303+0000 7fb80dc00640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8081019c0 0x7fb80819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.303+0000 7fb80d200640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb808102360 0x7fb80819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.304+0000 7fb80d200640 1 -- 172.21.15.2:0/1954204730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8081019c0 msgr2=0x7fb80819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.304+0000 7fb80d200640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8081019c0 0x7fb80819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.304+0000 7fb80d200640 1 -- 172.21.15.2:0/1954204730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8081079c0 msgr2=0x7fb80819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.304+0000 7fb80d200640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8081079c0 0x7fb80819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.304+0000 7fb80d200640 1 -- 172.21.15.2:0/1954204730 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80819f450 con 0x7fb808102360 2024-09-10T08:14:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.304+0000 7fb80dc00640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8081019c0 0x7fb80819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.304+0000 7fb80e600640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8081079c0 0x7fb80819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.304+0000 7fb80d200640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb808102360 0x7fb80819a9e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400e9f0 tx=0x7fb7f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.304+0000 7fb806000640 1 -- 172.21.15.2:0/1954204730 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400ccc0 con 0x7fb808102360 2024-09-10T08:14:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.305+0000 7fb80f000640 1 -- 172.21.15.2:0/1954204730 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80819f740 con 0x7fb808102360 2024-09-10T08:14:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.305+0000 7fb806000640 1 -- 172.21.15.2:0/1954204730 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f4005020 con 0x7fb808102360 2024-09-10T08:14:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.305+0000 7fb806000640 1 -- 172.21.15.2:0/1954204730 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40049e0 con 0x7fb808102360 2024-09-10T08:14:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.305+0000 7fb80f000640 1 -- 172.21.15.2:0/1954204730 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8081a7020 con 0x7fb808102360 2024-09-10T08:14:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.308+0000 7fb812fd9740 1 Processor -- start 2024-09-10T08:14:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.308+0000 7fb812fd9740 1 -- start start 2024-09-10T08:14:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.308+0000 7fb812fd9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad59900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.308+0000 7fb812fd9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.308+0000 7fb812fd9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 0x55c9bad66af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.308+0000 7fb806000640 1 -- 172.21.15.2:0/1954204730 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb7f4005560 con 0x7fb808102360 2024-09-10T08:14:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.308+0000 7fb812fd9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c9bacbf650 con 0x55c9bad59e40 2024-09-10T08:14:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.308+0000 7fb812fd9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c9bac66f10 con 0x55c9bad59520 2024-09-10T08:14:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.308+0000 7fb812fd9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c9bacbb610 con 0x55c9bad64700 2024-09-10T08:14:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.309+0000 7fb80dc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.309+0000 7fb80e600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad59900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.309+0000 7fb80e600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad59900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38888/0 (socket says 172.21.15.2:38888) 2024-09-10T08:14:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.309+0000 7fb80e600640 1 -- 172.21.15.2:0/4279546699 learned_addr learned my addr 172.21.15.2:0/4279546699 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.309+0000 7fb80d200640 1 --2- 172.21.15.2:0/4279546699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 0x55c9bad66af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.309+0000 7fb80e600640 1 -- 172.21.15.2:0/4279546699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 msgr2=0x55c9bad66af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.309+0000 7fb80e600640 1 --2- 172.21.15.2:0/4279546699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 0x55c9bad66af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.309+0000 7fb80e600640 1 -- 172.21.15.2:0/4279546699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 msgr2=0x55c9bad641c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.309+0000 7fb80e600640 1 --2- 172.21.15.2:0/4279546699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad641c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.309+0000 7fb80e600640 1 -- 172.21.15.2:0/4279546699 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ba9ce580 con 0x55c9bad59520 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.309+0000 7fb80e600640 1 --2- 172.21.15.2:0/4279546699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad59900 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb800005fb0 tx=0x7fb8000098a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f1ae743421ec592 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.310+0000 7fb7faa00640 1 -- 172.21.15.2:0/4279546699 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800003b70 con 0x55c9bad59520 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.310+0000 7fb7faa00640 1 -- 172.21.15.2:0/4279546699 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb800036030 con 0x55c9bad59520 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.310+0000 7fb7faa00640 1 -- 172.21.15.2:0/4279546699 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800004700 con 0x55c9bad59520 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.310+0000 7fb806000640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7f00a2b80 0x7fb7f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.310+0000 7fb806000640 1 -- 172.21.15.2:0/1954204730 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb7f40cf400 con 0x7fb808102360 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.311+0000 7fb812fd9740 1 -- 172.21.15.2:0/4279546699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 msgr2=0x55c9bad59900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.311+0000 7fb812fd9740 1 --2- 172.21.15.2:0/4279546699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad59900 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb800005fb0 tx=0x7fb8000098a0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.311+0000 7fb80dc00640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7f00a2b80 0x7fb7f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.311+0000 7fb812fd9740 1 -- 172.21.15.2:0/4279546699 shutdown_connections 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.311+0000 7fb812fd9740 1 --2- 172.21.15.2:0/4279546699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 0x55c9bad66af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.311+0000 7fb812fd9740 1 --2- 172.21.15.2:0/4279546699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad641c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.311+0000 7fb812fd9740 1 --2- 172.21.15.2:0/4279546699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad59900 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.311+0000 7fb812fd9740 1 -- 172.21.15.2:0/4279546699 >> 172.21.15.2:0/4279546699 conn(0x55c9bad55ab0 msgr2=0x55c9bad55e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.311+0000 7fb812fd9740 1 -- 172.21.15.2:0/4279546699 shutdown_connections 2024-09-10T08:14:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.311+0000 7fb812fd9740 1 -- 172.21.15.2:0/4279546699 wait complete. 2024-09-10T08:14:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.311+0000 7fb80dc00640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7f00a2b80 0x7fb7f00a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55c9bad56910 tx=0x7fb7fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.311+0000 7fb812fd9740 1 -- start start 2024-09-10T08:14:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.312+0000 7fb812fd9740 1 Processor -- start 2024-09-10T08:14:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.312+0000 7fb812fd9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad61630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.312+0000 7fb812fd9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad61b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.312+0000 7fb812fd9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 0x55c9bad620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.312+0000 7fb812fd9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c9bad672d0 con 0x55c9bad59e40 2024-09-10T08:14:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.312+0000 7fb812fd9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c9bac7a350 con 0x55c9bad59520 2024-09-10T08:14:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.312+0000 7fb812fd9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c9bad67450 con 0x55c9bad64700 2024-09-10T08:14:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.312+0000 7fb812fd9740 10 client.0 _ll_register_callbacks cb 0x55c9bac9d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.312+0000 7fb80dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad61630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb80d200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad61b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb80dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad61630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38904/0 (socket says 172.21.15.2:38904) 2024-09-10T08:14:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb80dc00640 1 -- 172.21.15.2:0/4167106539 learned_addr learned my addr 172.21.15.2:0/4167106539 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb80e600640 1 --2- 172.21.15.2:0/4167106539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 0x55c9bad620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb80d200640 1 -- 172.21.15.2:0/4167106539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 msgr2=0x55c9bad620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb80d200640 1 --2- 172.21.15.2:0/4167106539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 0x55c9bad620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb80d200640 1 -- 172.21.15.2:0/4167106539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 msgr2=0x55c9bad61630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb80d200640 1 --2- 172.21.15.2:0/4167106539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad61630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb80d200640 1 -- 172.21.15.2:0/4167106539 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9baa98960 con 0x55c9bad59e40 2024-09-10T08:14:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb80d200640 1 --2- 172.21.15.2:0/4167106539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad61b70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55c9bad56e30 tx=0x7fb7f409b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf3cc91cc2aa529 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb7ef400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb7ef400640 1 -- 172.21.15.2:0/4167106539 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f401c070 con 0x55c9bad59e40 2024-09-10T08:14:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb7ef400640 1 -- 172.21.15.2:0/4167106539 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb7f40860c0 con 0x55c9bad59e40 2024-09-10T08:14:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.313+0000 7fb812fd9740 1 -- 172.21.15.2:0/4167106539 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb800036030 con 0x55c9bad59e40 2024-09-10T08:14:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.314+0000 7fb7ef400640 1 -- 172.21.15.2:0/4167106539 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb7f40051d0 con 0x55c9bad59e40 2024-09-10T08:14:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.314+0000 7fb812fd9740 10 client.17202 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.314+0000 7fb812fd9740 1 -- 172.21.15.2:0/4167106539 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9bad72fc0 con 0x55c9bad59e40 2024-09-10T08:14:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.314+0000 7fb7ef400640 1 -- 172.21.15.2:0/4167106539 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb800036030 con 0x55c9bad59e40 2024-09-10T08:14:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.314+0000 7fb7ef400640 1 -- 172.21.15.2:0/4167106539 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb7f40a3ff0 con 0x55c9bad59e40 2024-09-10T08:14:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.314+0000 7fb812fd9740 10 client.17202 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.314+0000 7fb812fd9740 10 client.17202 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:14:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.314+0000 7fb812fd9740 20 client.17202 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.314+0000 7fb812fd9740 1 --2- 172.21.15.2:0/4167106539 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c9bad743d0 0x55c9bad767e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.314+0000 7fb812fd9740 4 client.17202 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.314+0000 7fb812fd9740 1 -- 172.21.15.2:0/4167106539 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9bacccbc0 con 0x55c9bad743d0 2024-09-10T08:14:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.315+0000 7fb80dc00640 1 --2- 172.21.15.2:0/4167106539 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c9bad743d0 0x55c9bad767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.315+0000 7fb80dc00640 1 --2- 172.21.15.2:0/4167106539 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c9bad743d0 0x55c9bad767e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39bd99fbb69b355f server_cookie=c9b60d5a7778ccc in_seq=0 out_seq=0 2024-09-10T08:14:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.315+0000 7fb7ef400640 10 client.17202 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.316+0000 7fb7ef400640 1 -- 172.21.15.2:0/4167106539 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9bacccbc0 con 0x55c9bad743d0 2024-09-10T08:14:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.316+0000 7fb7ef400640 10 client.17202 handle_command_reply: tid=0 2024-09-10T08:14:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.316+0000 7fb812fd9740 1 client.17202 shutdown 2024-09-10T08:14:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.316+0000 7fb812fd9740 1 -- 172.21.15.2:0/4167106539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 msgr2=0x55c9bad61b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.316+0000 7fb812fd9740 1 --2- 172.21.15.2:0/4167106539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad61b70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55c9bad56e30 tx=0x7fb7f409b5e0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.316+0000 7fb812fd9740 1 -- 172.21.15.2:0/4167106539 shutdown_connections 2024-09-10T08:14:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.316+0000 7fb812fd9740 1 --2- 172.21.15.2:0/4167106539 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c9bad743d0 0x55c9bad767e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.316+0000 7fb812fd9740 1 --2- 172.21.15.2:0/4167106539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 0x55c9bad620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.316+0000 7fb812fd9740 1 --2- 172.21.15.2:0/4167106539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad61b70 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.316+0000 7fb812fd9740 1 --2- 172.21.15.2:0/4167106539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad61630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.316+0000 7fb7ef400640 0 client.17202 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.316+0000 7fb812fd9740 1 -- 172.21.15.2:0/4167106539 >> 172.21.15.2:0/4167106539 conn(0x55c9bad55ab0 msgr2=0x55c9bad54cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.317+0000 7fb812fd9740 1 -- 172.21.15.2:0/4167106539 shutdown_connections 2024-09-10T08:14:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.317+0000 7fb812fd9740 1 -- 172.21.15.2:0/4167106539 wait complete. 2024-09-10T08:14:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.317+0000 7fb812fd9740 20 client.17202 trim_cache size 0 max 16384 2024-09-10T08:14:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.334+0000 7fb812fd9740 1 Processor -- start 2024-09-10T08:14:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.334+0000 7fb812fd9740 1 -- start start 2024-09-10T08:14:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb812fd9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59520 0x55c9bad63490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb812fd9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad59e40 0x55c9bad7c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb812fd9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad64700 0x55c9bad7c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb812fd9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081b36a0 con 0x55c9bad59520 2024-09-10T08:14:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb812fd9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081b0f70 con 0x55c9bad64700 2024-09-10T08:14:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb812fd9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081aedc0 con 0x55c9bad59e40 2024-09-10T08:14:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb80dc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59520 0x55c9bad63490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb80dc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59520 0x55c9bad63490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60090/0 (socket says 172.21.15.2:60090) 2024-09-10T08:14:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb80dc00640 1 -- 172.21.15.2:0/1985941747 learned_addr learned my addr 172.21.15.2:0/1985941747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb80e600640 1 --2- 172.21.15.2:0/1985941747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad64700 0x55c9bad7c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb80d200640 1 --2- 172.21.15.2:0/1985941747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad59e40 0x55c9bad7c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb80dc00640 1 -- 172.21.15.2:0/1985941747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad59e40 msgr2=0x55c9bad7c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb80dc00640 1 --2- 172.21.15.2:0/1985941747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad59e40 0x55c9bad7c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb80dc00640 1 -- 172.21.15.2:0/1985941747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad64700 msgr2=0x55c9bad7c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb80dc00640 1 --2- 172.21.15.2:0/1985941747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad64700 0x55c9bad7c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.335+0000 7fb80dc00640 1 -- 172.21.15.2:0/1985941747 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9bad56b60 con 0x55c9bad59520 2024-09-10T08:14:54.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.336+0000 7fb80dc00640 1 --2- 172.21.15.2:0/1985941747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59520 0x55c9bad63490 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc012ea0 tx=0x7fb7fc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d450e494c80d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:54.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.336+0000 7fb7fa000640 1 -- 172.21.15.2:0/1985941747 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc005020 con 0x55c9bad59520 2024-09-10T08:14:54.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.336+0000 7fb7fa000640 1 -- 172.21.15.2:0/1985941747 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7fc0047a0 con 0x55c9bad59520 2024-09-10T08:14:54.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.336+0000 7fb7fa000640 1 -- 172.21.15.2:0/1985941747 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc02b820 con 0x55c9bad59520 2024-09-10T08:14:54.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.336+0000 7fb812fd9740 1 -- 172.21.15.2:0/1985941747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59520 msgr2=0x55c9bad63490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:54.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.336+0000 7fb812fd9740 1 --2- 172.21.15.2:0/1985941747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59520 0x55c9bad63490 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc012ea0 tx=0x7fb7fc01b000 comp rx=0 tx=0).stop 2024-09-10T08:14:54.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.337+0000 7fb812fd9740 1 -- 172.21.15.2:0/1985941747 shutdown_connections 2024-09-10T08:14:54.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.337+0000 7fb812fd9740 1 --2- 172.21.15.2:0/1985941747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad64700 0x55c9bad7c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.337+0000 7fb812fd9740 1 --2- 172.21.15.2:0/1985941747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad59e40 0x55c9bad7c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.337+0000 7fb812fd9740 1 --2- 172.21.15.2:0/1985941747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59520 0x55c9bad63490 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.337+0000 7fb812fd9740 1 -- 172.21.15.2:0/1985941747 >> 172.21.15.2:0/1985941747 conn(0x55c9bad55ab0 msgr2=0x55c9bad62d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.337+0000 7fb812fd9740 1 -- 172.21.15.2:0/1985941747 shutdown_connections 2024-09-10T08:14:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.337+0000 7fb812fd9740 1 -- 172.21.15.2:0/1985941747 wait complete. 2024-09-10T08:14:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.337+0000 7fb812fd9740 1 -- start start 2024-09-10T08:14:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.337+0000 7fb812fd9740 1 Processor -- start 2024-09-10T08:14:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.338+0000 7fb812fd9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad68cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.338+0000 7fb812fd9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad69230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.338+0000 7fb812fd9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 0x55c9badd80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.338+0000 7fb812fd9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081af430 con 0x55c9bad59e40 2024-09-10T08:14:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.338+0000 7fb812fd9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081b08a0 con 0x55c9bad59520 2024-09-10T08:14:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.338+0000 7fb812fd9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081afb00 con 0x55c9bad64700 2024-09-10T08:14:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.338+0000 7fb812fd9740 10 client.0 _ll_register_callbacks cb 0x7fb7f40053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.338+0000 7fb80d200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad69230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.338+0000 7fb80dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad68cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.338+0000 7fb80d200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad69230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60094/0 (socket says 172.21.15.2:60094) 2024-09-10T08:14:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.338+0000 7fb80dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad68cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38928/0 (socket says 172.21.15.2:38928) 2024-09-10T08:14:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.338+0000 7fb80d200640 1 -- 172.21.15.2:0/2772065563 learned_addr learned my addr 172.21.15.2:0/2772065563 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.338+0000 7fb80e600640 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 0x55c9badd80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb80d200640 1 -- 172.21.15.2:0/2772065563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 msgr2=0x55c9badd80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb80d200640 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 0x55c9badd80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb80d200640 1 -- 172.21.15.2:0/2772065563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 msgr2=0x55c9bad68cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb80d200640 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad68cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb80d200640 1 -- 172.21.15.2:0/2772065563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9bad56740 con 0x55c9bad59e40 2024-09-10T08:14:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb80e600640 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 0x55c9badd80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:14:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb80dc00640 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad68cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb80d200640 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad69230 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4098190 tx=0x7fb7f4004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352e970c36c3c8f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb7ef400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb7ef400640 1 -- 172.21.15.2:0/2772065563 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f401c070 con 0x55c9bad59e40 2024-09-10T08:14:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb7ef400640 1 -- 172.21.15.2:0/2772065563 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb7f409cd40 con 0x55c9bad59e40 2024-09-10T08:14:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb812fd9740 1 -- 172.21.15.2:0/2772065563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7fc0047a0 con 0x55c9bad59e40 2024-09-10T08:14:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb7ef400640 1 -- 172.21.15.2:0/2772065563 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb7f4004d30 con 0x55c9bad59e40 2024-09-10T08:14:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb812fd9740 10 client.17208 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.339+0000 7fb812fd9740 1 -- 172.21.15.2:0/2772065563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9baddf890 con 0x55c9bad59e40 2024-09-10T08:14:54.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.340+0000 7fb7ef400640 1 -- 172.21.15.2:0/2772065563 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7fc0047a0 con 0x55c9bad59e40 2024-09-10T08:14:54.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.340+0000 7fb7ef400640 1 -- 172.21.15.2:0/2772065563 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb7f40d2b30 con 0x55c9bad59e40 2024-09-10T08:14:54.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.340+0000 7fb812fd9740 10 client.17208 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:54.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.340+0000 7fb812fd9740 10 client.17208 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:14:54.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.340+0000 7fb812fd9740 20 client.17208 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:54.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.340+0000 7fb812fd9740 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c9baddfd50 0x55c9bade2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:54.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.340+0000 7fb812fd9740 4 client.17208 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f78d3e3b", "release": true, "await": true}] 2024-09-10T08:14:54.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.340+0000 7fb812fd9740 1 -- 172.21.15.2:0/2772065563 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f78d3e3b", "release": true, "await": true}) -- 0x55c9bacc70d0 con 0x55c9baddfd50 2024-09-10T08:14:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.340+0000 7fb80dc00640 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c9baddfd50 0x55c9bade2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.341+0000 7fb80dc00640 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c9baddfd50 0x55c9bade2170 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a953b476c7b12dd server_cookie=d7457d2a49265b17 in_seq=0 out_seq=0 2024-09-10T08:14:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.341+0000 7fb7ef400640 10 client.17208 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb7ef400640 1 -- 172.21.15.2:0/2772065563 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c9bacc70d0 con 0x55c9baddfd50 2024-09-10T08:14:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb7ef400640 10 client.17208 handle_command_reply: tid=0 2024-09-10T08:14:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb812fd9740 1 client.17208 shutdown 2024-09-10T08:14:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb812fd9740 1 -- 172.21.15.2:0/2772065563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 msgr2=0x55c9bad69230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb812fd9740 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad69230 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4098190 tx=0x7fb7f4004f90 comp rx=0 tx=0).stop 2024-09-10T08:14:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb812fd9740 1 -- 172.21.15.2:0/2772065563 shutdown_connections 2024-09-10T08:14:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb812fd9740 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c9baddfd50 0x55c9bade2170 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb812fd9740 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9bad64700 0x55c9badd80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb812fd9740 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c9bad59e40 0x55c9bad69230 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb812fd9740 1 --2- 172.21.15.2:0/2772065563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9bad59520 0x55c9bad68cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb7ef400640 0 client.17208 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb812fd9740 1 -- 172.21.15.2:0/2772065563 >> 172.21.15.2:0/2772065563 conn(0x55c9bad55ab0 msgr2=0x55c9bad5b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:54.423 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:14:54.423 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:14:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:14:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 20, 2024-09-10T08:14:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:14:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "f78d3e3b": { 2024-09-10T08:14:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 20, 2024-09-10T08:14:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:14:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:14:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:14:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:14:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:14:54.424 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:14:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 140.1, 2024-09-10T08:14:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:14:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:14:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:14:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:14:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:14:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:14:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:54.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:54.426 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:14:54.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb812fd9740 1 -- 172.21.15.2:0/2772065563 shutdown_connections 2024-09-10T08:14:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb812fd9740 1 -- 172.21.15.2:0/2772065563 wait complete. 2024-09-10T08:14:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.343+0000 7fb812fd9740 20 client.17208 trim_cache size 0 max 16384 2024-09-10T08:14:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.344+0000 7fb80f000640 1 -- 172.21.15.2:0/1954204730 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7f00a2b80 msgr2=0x7fb7f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.344+0000 7fb80f000640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7f00a2b80 0x7fb7f00a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55c9bad56910 tx=0x7fb7fc005f90 comp rx=0 tx=0).stop 2024-09-10T08:14:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.344+0000 7fb80f000640 1 -- 172.21.15.2:0/1954204730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb808102360 msgr2=0x7fb80819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.344+0000 7fb80f000640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb808102360 0x7fb80819a9e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400e9f0 tx=0x7fb7f400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.344+0000 7fb80f000640 1 -- 172.21.15.2:0/1954204730 shutdown_connections 2024-09-10T08:14:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.344+0000 7fb80f000640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8081079c0 0x7fb80819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.344+0000 7fb80f000640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7f00a2b80 0x7fb7f00a5040 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.344+0000 7fb80f000640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb808102360 0x7fb80819a9e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.344+0000 7fb80f000640 1 --2- 172.21.15.2:0/1954204730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8081019c0 0x7fb80819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.344+0000 7fb80f000640 1 -- 172.21.15.2:0/1954204730 >> 172.21.15.2:0/1954204730 conn(0x7fb8080fd740 msgr2=0x7fb8080fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.344+0000 7fb80f000640 1 -- 172.21.15.2:0/1954204730 shutdown_connections 2024-09-10T08:14:54.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:54.344+0000 7fb80f000640 1 -- 172.21.15.2:0/1954204730 wait complete. 2024-09-10T08:14:55.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:54 smithi002 ceph-mon[30563]: pgmap v590: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-10T08:14:55.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:54 smithi155 ceph-mon[35405]: pgmap v590: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-10T08:14:55.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:54 smithi107 ceph-mon[35048]: pgmap v590: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-10T08:14:55.424 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f78d3e3b', total seconds elapsed: 24.5 2024-09-10T08:14:55.424 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T08:14:55.424 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:14:55.804 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:56.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.385+0000 7f65f9000640 1 -- 172.21.15.2:0/588331648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65f4108f60 msgr2=0x7f65f4110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:56.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.385+0000 7f65f9000640 1 --2- 172.21.15.2:0/588331648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65f4108f60 0x7f65f4110ae0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009ca0 tx=0x7f65ec02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:14:56.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.385+0000 7f65f9000640 1 -- 172.21.15.2:0/588331648 shutdown_connections 2024-09-10T08:14:56.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.385+0000 7f65f9000640 1 --2- 172.21.15.2:0/588331648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65f4108f60 0x7f65f4110ae0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:56.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.385+0000 7f65f9000640 1 --2- 172.21.15.2:0/588331648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65f41047e0 0x7f65f4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:56.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.385+0000 7f65f9000640 1 --2- 172.21.15.2:0/588331648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f4103e30 0x7f65f4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:56.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.385+0000 7f65f9000640 1 -- 172.21.15.2:0/588331648 >> 172.21.15.2:0/588331648 conn(0x7f65f40ff8d0 msgr2=0x7f65f4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:56.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.385+0000 7f65f9000640 1 -- 172.21.15.2:0/588331648 shutdown_connections 2024-09-10T08:14:56.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.385+0000 7f65f9000640 1 -- 172.21.15.2:0/588331648 wait complete. 2024-09-10T08:14:56.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.386+0000 7f65f9000640 1 Processor -- start 2024-09-10T08:14:56.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.387+0000 7f65f9000640 1 -- start start 2024-09-10T08:14:56.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.387+0000 7f65f9000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65f4103e30 0x7f65f419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:56.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.387+0000 7f65f9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65f41047e0 0x7f65f419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:56.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.387+0000 7f65f9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f4108f60 0x7f65f419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:56.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.387+0000 7f65f9000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4113fc0 con 0x7f65f4103e30 2024-09-10T08:14:56.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.387+0000 7f65f9000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4113e40 con 0x7f65f41047e0 2024-09-10T08:14:56.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.387+0000 7f65f9000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4114140 con 0x7f65f4108f60 2024-09-10T08:14:56.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.387+0000 7f65f3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65f4103e30 0x7f65f419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:56.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.387+0000 7f65eaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65f41047e0 0x7f65f419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:56.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.387+0000 7f65f3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65f4103e30 0x7f65f419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60104/0 (socket says 172.21.15.2:60104) 2024-09-10T08:14:56.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.387+0000 7f65f3400640 1 -- 172.21.15.2:0/4103718723 learned_addr learned my addr 172.21.15.2:0/4103718723 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:56.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.388+0000 7f65f3e00640 1 --2- 172.21.15.2:0/4103718723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f4108f60 0x7f65f419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:56.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.388+0000 7f65f3400640 1 -- 172.21.15.2:0/4103718723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f4108f60 msgr2=0x7f65f419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:56.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.388+0000 7f65f3400640 1 --2- 172.21.15.2:0/4103718723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f4108f60 0x7f65f419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:56.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.388+0000 7f65f3400640 1 -- 172.21.15.2:0/4103718723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65f41047e0 msgr2=0x7f65f419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:56.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.388+0000 7f65f3400640 1 --2- 172.21.15.2:0/4103718723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65f41047e0 0x7f65f419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:56.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.388+0000 7f65f3400640 1 -- 172.21.15.2:0/4103718723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f419f3b0 con 0x7f65f4103e30 2024-09-10T08:14:56.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.388+0000 7f65f3400640 1 --2- 172.21.15.2:0/4103718723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65f4103e30 0x7f65f419a400 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f65e400e9f0 tx=0x7f65e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:56.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.388+0000 7f65f0c00640 1 -- 172.21.15.2:0/4103718723 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e400cd80 con 0x7f65f4103e30 2024-09-10T08:14:56.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.388+0000 7f65f0c00640 1 -- 172.21.15.2:0/4103718723 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e4005500 con 0x7f65f4103e30 2024-09-10T08:14:56.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.389+0000 7f65f9000640 1 -- 172.21.15.2:0/4103718723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f419f6a0 con 0x7f65f4103e30 2024-09-10T08:14:56.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.389+0000 7f65f0c00640 1 -- 172.21.15.2:0/4103718723 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4010690 con 0x7f65f4103e30 2024-09-10T08:14:56.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.389+0000 7f65f9000640 1 -- 172.21.15.2:0/4103718723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f41a6f80 con 0x7f65f4103e30 2024-09-10T08:14:56.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.390+0000 7f65f9000640 1 -- 172.21.15.2:0/4103718723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f4104210 con 0x7f65f4103e30 2024-09-10T08:14:56.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.391+0000 7f65f0c00640 1 -- 172.21.15.2:0/4103718723 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f65e40046c0 con 0x7f65f4103e30 2024-09-10T08:14:56.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.393+0000 7f65f0c00640 1 --2- 172.21.15.2:0/4103718723 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f65d80a2b80 0x7f65d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:56.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.393+0000 7f65f0c00640 1 -- 172.21.15.2:0/4103718723 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f65e40cf870 con 0x7f65f4103e30 2024-09-10T08:14:56.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.393+0000 7f65eaa00640 1 --2- 172.21.15.2:0/4103718723 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f65d80a2b80 0x7f65d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:56.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.394+0000 7f65eaa00640 1 --2- 172.21.15.2:0/4103718723 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f65d80a2b80 0x7f65d80a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f65f410a930 tx=0x7f65dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:56.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.397+0000 7f65f0c00640 1 -- 172.21.15.2:0/4103718723 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65e409c380 con 0x7f65f4103e30 2024-09-10T08:14:56.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.620+0000 7f65f9000640 1 -- 172.21.15.2:0/4103718723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65f410c200 con 0x7f65f4103e30 2024-09-10T08:14:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.621+0000 7f65f0c00640 1 -- 172.21.15.2:0/4103718723 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f65e409bc80 con 0x7f65f4103e30 2024-09-10T08:14:56.621 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:14:56.621 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:14:56.622 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:14:56.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.623+0000 7f65f9000640 1 -- 172.21.15.2:0/4103718723 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f65d80a2b80 msgr2=0x7f65d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:56.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.623+0000 7f65f9000640 1 --2- 172.21.15.2:0/4103718723 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f65d80a2b80 0x7f65d80a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f65f410a930 tx=0x7f65dc005f90 comp rx=0 tx=0).stop 2024-09-10T08:14:56.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.623+0000 7f65f9000640 1 -- 172.21.15.2:0/4103718723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65f4103e30 msgr2=0x7f65f419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:56.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.623+0000 7f65f9000640 1 --2- 172.21.15.2:0/4103718723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65f4103e30 0x7f65f419a400 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f65e400e9f0 tx=0x7f65e400edf0 comp rx=0 tx=0).stop 2024-09-10T08:14:56.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.623+0000 7f65f9000640 1 -- 172.21.15.2:0/4103718723 shutdown_connections 2024-09-10T08:14:56.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.623+0000 7f65f9000640 1 --2- 172.21.15.2:0/4103718723 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f65d80a2b80 0x7f65d80a5040 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:56.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.623+0000 7f65f9000640 1 --2- 172.21.15.2:0/4103718723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f4108f60 0x7f65f419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:56.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.623+0000 7f65f9000640 1 --2- 172.21.15.2:0/4103718723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65f41047e0 0x7f65f419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:56.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.623+0000 7f65f9000640 1 --2- 172.21.15.2:0/4103718723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65f4103e30 0x7f65f419a400 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:56.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.623+0000 7f65f9000640 1 -- 172.21.15.2:0/4103718723 >> 172.21.15.2:0/4103718723 conn(0x7f65f40ff8d0 msgr2=0x7f65f4100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:56.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.624+0000 7f65f9000640 1 -- 172.21.15.2:0/4103718723 shutdown_connections 2024-09-10T08:14:56.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:56.624+0000 7f65f9000640 1 -- 172.21.15.2:0/4103718723 wait complete. 2024-09-10T08:14:57.121 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:14:57.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:56 smithi002 ceph-mon[30563]: pgmap v591: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-09-10T08:14:57.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:56 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/4103718723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:14:57.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:56 smithi155 ceph-mon[35405]: pgmap v591: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-09-10T08:14:57.436 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:56 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/4103718723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:14:57.437 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:56 smithi107 ceph-mon[35048]: pgmap v591: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-09-10T08:14:57.437 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:56 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/4103718723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:14:57.452 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T08:14:57.453 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T08:14:57.541 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:58.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.159+0000 7f8b40200640 1 -- 172.21.15.2:0/1733361513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b381027a0 msgr2=0x7f8b381067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:58.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.159+0000 7f8b40200640 1 --2- 172.21.15.2:0/1733361513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b381027a0 0x7f8b381067f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009900 tx=0x7f8b2402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.160+0000 7f8b40200640 1 -- 172.21.15.2:0/1733361513 shutdown_connections 2024-09-10T08:14:58.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.160+0000 7f8b40200640 1 --2- 172.21.15.2:0/1733361513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b38106f20 0x7f8b3810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.160+0000 7f8b40200640 1 --2- 172.21.15.2:0/1733361513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b381027a0 0x7f8b381067f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.160+0000 7f8b40200640 1 --2- 172.21.15.2:0/1733361513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b38101df0 0x7f8b381021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.160+0000 7f8b40200640 1 -- 172.21.15.2:0/1733361513 >> 172.21.15.2:0/1733361513 conn(0x7f8b380fd8b0 msgr2=0x7f8b380ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:58.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.160+0000 7f8b40200640 1 -- 172.21.15.2:0/1733361513 shutdown_connections 2024-09-10T08:14:58.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.160+0000 7f8b40200640 1 -- 172.21.15.2:0/1733361513 wait complete. 2024-09-10T08:14:58.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.161+0000 7f8b40200640 1 Processor -- start 2024-09-10T08:14:58.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.161+0000 7f8b40200640 1 -- start start 2024-09-10T08:14:58.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.161+0000 7f8b40200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b38101df0 0x7f8b3819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.161+0000 7f8b40200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b381027a0 0x7f8b3819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.161+0000 7f8b40200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b38106f20 0x7f8b3819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.161+0000 7f8b40200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38111fd0 con 0x7f8b381027a0 2024-09-10T08:14:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.161+0000 7f8b40200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38111e50 con 0x7f8b38106f20 2024-09-10T08:14:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.161+0000 7f8b40200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38112150 con 0x7f8b38101df0 2024-09-10T08:14:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.162+0000 7f8b3d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b38101df0 0x7f8b3819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.162+0000 7f8b3ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b381027a0 0x7f8b3819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.162+0000 7f8b3e200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b38106f20 0x7f8b3819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.162+0000 7f8b3d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b38101df0 0x7f8b3819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:44782/0 (socket says 172.21.15.2:44782) 2024-09-10T08:14:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.162+0000 7f8b3d800640 1 -- 172.21.15.2:0/475878140 learned_addr learned my addr 172.21.15.2:0/475878140 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.162+0000 7f8b3d800640 1 -- 172.21.15.2:0/475878140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b38106f20 msgr2=0x7f8b3819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.162+0000 7f8b3d800640 1 --2- 172.21.15.2:0/475878140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b38106f20 0x7f8b3819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.162+0000 7f8b3d800640 1 -- 172.21.15.2:0/475878140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b381027a0 msgr2=0x7f8b3819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.162+0000 7f8b3d800640 1 --2- 172.21.15.2:0/475878140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b381027a0 0x7f8b3819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.162+0000 7f8b3d800640 1 -- 172.21.15.2:0/475878140 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3819f650 con 0x7f8b38101df0 2024-09-10T08:14:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.162+0000 7f8b3e200640 1 --2- 172.21.15.2:0/475878140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b38106f20 0x7f8b3819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.163+0000 7f8b3d800640 1 --2- 172.21.15.2:0/475878140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b38101df0 0x7f8b3819a6a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00ea60 tx=0x7f8b2c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.163+0000 7f8b36000640 1 -- 172.21.15.2:0/475878140 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00cde0 con 0x7f8b38101df0 2024-09-10T08:14:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.163+0000 7f8b36000640 1 -- 172.21.15.2:0/475878140 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2c0035f0 con 0x7f8b38101df0 2024-09-10T08:14:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.163+0000 7f8b36000640 1 -- 172.21.15.2:0/475878140 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c010670 con 0x7f8b38101df0 2024-09-10T08:14:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.163+0000 7f8b40200640 1 -- 172.21.15.2:0/475878140 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3819f940 con 0x7f8b38101df0 2024-09-10T08:14:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.163+0000 7f8b40200640 1 -- 172.21.15.2:0/475878140 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b381a7220 con 0x7f8b38101df0 2024-09-10T08:14:58.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.166+0000 7f8b42c7f740 1 Processor -- start 2024-09-10T08:14:58.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.166+0000 7f8b42c7f740 1 -- start start 2024-09-10T08:14:58.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.166+0000 7f8b36000640 1 -- 172.21.15.2:0/475878140 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8b2c0037a0 con 0x7f8b38101df0 2024-09-10T08:14:58.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.167+0000 7f8b42c7f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559921571b90 0x559921571f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.167+0000 7f8b42c7f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5599215724b0 0x55992157c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.167+0000 7f8b42c7f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55992157cd70 0x55992157f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.167+0000 7f8b42c7f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5599214d83e0 con 0x55992157cd70 2024-09-10T08:14:58.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.167+0000 7f8b42c7f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55992147fd80 con 0x5599215724b0 2024-09-10T08:14:58.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.167+0000 7f8b42c7f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5599214d43a0 con 0x559921571b90 2024-09-10T08:14:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.167+0000 7f8b3d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5599215724b0 0x55992157c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.167+0000 7f8b3ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55992157cd70 0x55992157f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.167+0000 7f8b3d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5599215724b0 0x55992157c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:34962/0 (socket says 172.21.15.2:34962) 2024-09-10T08:14:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.167+0000 7f8b3d800640 1 -- 172.21.15.2:0/3824810195 learned_addr learned my addr 172.21.15.2:0/3824810195 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.167+0000 7f8b3e200640 1 --2- 172.21.15.2:0/3824810195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559921571b90 0x559921571f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.168+0000 7f8b3d800640 1 -- 172.21.15.2:0/3824810195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559921571b90 msgr2=0x559921571f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.168+0000 7f8b3d800640 1 --2- 172.21.15.2:0/3824810195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559921571b90 0x559921571f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.168+0000 7f8b3d800640 1 -- 172.21.15.2:0/3824810195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55992157cd70 msgr2=0x55992157f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.168+0000 7f8b3d800640 1 --2- 172.21.15.2:0/3824810195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55992157cd70 0x55992157f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.168+0000 7f8b3d800640 1 -- 172.21.15.2:0/3824810195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55992149eae0 con 0x5599215724b0 2024-09-10T08:14:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.168+0000 7f8b3d800640 1 --2- 172.21.15.2:0/3824810195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5599215724b0 0x55992157c830 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0d5a90 tx=0x7f8b2c0d5f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81ea0896500d8e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.168+0000 7f8b36000640 1 --2- 172.21.15.2:0/475878140 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8b200a2b80 0x7f8b200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.168+0000 7f8b2aa00640 1 -- 172.21.15.2:0/3824810195 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0de070 con 0x5599215724b0 2024-09-10T08:14:58.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.169+0000 7f8b2aa00640 1 -- 172.21.15.2:0/3824810195 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2c087de0 con 0x5599215724b0 2024-09-10T08:14:58.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.169+0000 7f8b2aa00640 1 -- 172.21.15.2:0/3824810195 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c09b4d0 con 0x5599215724b0 2024-09-10T08:14:58.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.169+0000 7f8b3e200640 1 --2- 172.21.15.2:0/475878140 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8b200a2b80 0x7f8b200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.169+0000 7f8b36000640 1 -- 172.21.15.2:0/475878140 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8b2c0cf760 con 0x7f8b38101df0 2024-09-10T08:14:58.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.169+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3824810195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5599215724b0 msgr2=0x55992157c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:58.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.169+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3824810195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5599215724b0 0x55992157c830 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0d5a90 tx=0x7f8b2c0d5f50 comp rx=0 tx=0).stop 2024-09-10T08:14:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.169+0000 7f8b3e200640 1 --2- 172.21.15.2:0/475878140 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8b200a2b80 0x7f8b200a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009fd0 tx=0x7f8b30009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.170+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3824810195 shutdown_connections 2024-09-10T08:14:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.170+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3824810195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55992157cd70 0x55992157f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.170+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3824810195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5599215724b0 0x55992157c830 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.170+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3824810195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559921571b90 0x559921571f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.170+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3824810195 >> 172.21.15.2:0/3824810195 conn(0x55992156e120 msgr2=0x55992156e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.170+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3824810195 shutdown_connections 2024-09-10T08:14:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.170+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3824810195 wait complete. 2024-09-10T08:14:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.170+0000 7f8b42c7f740 1 -- start start 2024-09-10T08:14:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.170+0000 7f8b42c7f740 1 Processor -- start 2024-09-10T08:14:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.171+0000 7f8b42c7f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559921571b90 0x559921579bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.171+0000 7f8b42c7f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599215724b0 0x55992157a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.171+0000 7f8b42c7f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55992157cd70 0x55992157a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.171+0000 7f8b42c7f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55992157f9e0 con 0x559921571b90 2024-09-10T08:14:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.171+0000 7f8b42c7f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5599214930f0 con 0x55992157cd70 2024-09-10T08:14:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.171+0000 7f8b42c7f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55992157fb60 con 0x5599215724b0 2024-09-10T08:14:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.171+0000 7f8b42c7f740 10 client.0 _ll_register_callbacks cb 0x5599214849e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.171+0000 7f8b3d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559921571b90 0x559921579bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.171+0000 7f8b3d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559921571b90 0x559921579bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50194/0 (socket says 172.21.15.2:50194) 2024-09-10T08:14:58.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.171+0000 7f8b3d800640 1 -- 172.21.15.2:0/3517873960 learned_addr learned my addr 172.21.15.2:0/3517873960 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:58.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.171+0000 7f8b3e200640 1 --2- 172.21.15.2:0/3517873960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55992157cd70 0x55992157a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.171+0000 7f8b3ce00640 1 --2- 172.21.15.2:0/3517873960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599215724b0 0x55992157a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.172+0000 7f8b3ce00640 1 -- 172.21.15.2:0/3517873960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55992157cd70 msgr2=0x55992157a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:58.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.172+0000 7f8b3ce00640 1 --2- 172.21.15.2:0/3517873960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55992157cd70 0x55992157a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.172+0000 7f8b3ce00640 1 -- 172.21.15.2:0/3517873960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559921571b90 msgr2=0x559921579bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:58.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.172+0000 7f8b3ce00640 1 --2- 172.21.15.2:0/3517873960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559921571b90 0x559921579bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.172+0000 7f8b3ce00640 1 -- 172.21.15.2:0/3517873960 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599212493c0 con 0x5599215724b0 2024-09-10T08:14:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.172+0000 7f8b3ce00640 1 --2- 172.21.15.2:0/3517873960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599215724b0 0x55992157a0f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8b2402a9e0 tx=0x7f8b24004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=492c05eb2dc24895 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.172+0000 7f8b1f400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:14:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.173+0000 7f8b1f400640 1 -- 172.21.15.2:0/3517873960 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24035040 con 0x5599215724b0 2024-09-10T08:14:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.173+0000 7f8b1f400640 1 -- 172.21.15.2:0/3517873960 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8b24005450 con 0x5599215724b0 2024-09-10T08:14:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.173+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3517873960 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b2c087de0 con 0x5599215724b0 2024-09-10T08:14:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.173+0000 7f8b1f400640 1 -- 172.21.15.2:0/3517873960 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8b2404f050 con 0x5599215724b0 2024-09-10T08:14:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.173+0000 7f8b42c7f740 10 client.27211 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.173+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3517873960 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55992158b5e0 con 0x5599215724b0 2024-09-10T08:14:58.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.174+0000 7f8b1f400640 1 -- 172.21.15.2:0/3517873960 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2c087de0 con 0x5599215724b0 2024-09-10T08:14:58.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.174+0000 7f8b1f400640 1 -- 172.21.15.2:0/3517873960 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8b240039f0 con 0x5599215724b0 2024-09-10T08:14:58.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.174+0000 7f8b42c7f740 10 client.27211 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:58.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.174+0000 7f8b42c7f740 10 client.27211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:58.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.174+0000 7f8b42c7f740 20 client.27211 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:58.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.174+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3517873960 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55992158caa0 0x55992158ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.174+0000 7f8b42c7f740 4 client.27211 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:58.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.174+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3517873960 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599214e5950 con 0x55992158caa0 2024-09-10T08:14:58.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.174+0000 7f8b3d800640 1 --2- 172.21.15.2:0/3517873960 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55992158caa0 0x55992158ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.175+0000 7f8b3d800640 1 --2- 172.21.15.2:0/3517873960 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55992158caa0 0x55992158ee90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11cb086309fb6985 server_cookie=99a08d2f10aa45b6 in_seq=0 out_seq=0 2024-09-10T08:14:58.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.175+0000 7f8b1f400640 10 client.27211 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:58.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b1f400640 1 -- 172.21.15.2:0/3517873960 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599214e5950 con 0x55992158caa0 2024-09-10T08:14:58.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b1f400640 10 client.27211 handle_command_reply: tid=0 2024-09-10T08:14:58.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b42c7f740 1 client.27211 shutdown 2024-09-10T08:14:58.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3517873960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599215724b0 msgr2=0x55992157a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:58.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3517873960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599215724b0 0x55992157a0f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8b2402a9e0 tx=0x7f8b24004440 comp rx=0 tx=0).stop 2024-09-10T08:14:58.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3517873960 shutdown_connections 2024-09-10T08:14:58.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3517873960 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55992158caa0 0x55992158ee90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3517873960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55992157cd70 0x55992157a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3517873960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599215724b0 0x55992157a0f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3517873960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559921571b90 0x559921579bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3517873960 >> 172.21.15.2:0/3517873960 conn(0x55992156e120 msgr2=0x559921573e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:58.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b1f400640 0 client.27211 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:58.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3517873960 shutdown_connections 2024-09-10T08:14:58.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.176+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3517873960 wait complete. 2024-09-10T08:14:58.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.177+0000 7f8b42c7f740 20 client.27211 trim_cache size 0 max 16384 2024-09-10T08:14:58.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.195+0000 7f8b42c7f740 1 Processor -- start 2024-09-10T08:14:58.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.195+0000 7f8b42c7f740 1 -- start start 2024-09-10T08:14:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b42c7f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559921571b90 0x559921594740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b42c7f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599215724b0 0x559921594c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b42c7f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55992157cd70 0x55992156c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b42c7f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381b39a0 con 0x559921571b90 2024-09-10T08:14:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b42c7f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381b1270 con 0x55992157cd70 2024-09-10T08:14:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b42c7f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381af0c0 con 0x5599215724b0 2024-09-10T08:14:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b3d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559921571b90 0x559921594740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b3d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559921571b90 0x559921594740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50206/0 (socket says 172.21.15.2:50206) 2024-09-10T08:14:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b3d800640 1 -- 172.21.15.2:0/3550300718 learned_addr learned my addr 172.21.15.2:0/3550300718 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:58.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b3e200640 1 --2- 172.21.15.2:0/3550300718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55992157cd70 0x55992156c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b3d800640 1 -- 172.21.15.2:0/3550300718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599215724b0 msgr2=0x559921594c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:14:58.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b3d800640 1 --2- 172.21.15.2:0/3550300718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599215724b0 0x559921594c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b3d800640 1 -- 172.21.15.2:0/3550300718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55992157cd70 msgr2=0x55992156c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:58.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.196+0000 7f8b3d800640 1 --2- 172.21.15.2:0/3550300718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55992157cd70 0x55992156c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.197+0000 7f8b3d800640 1 -- 172.21.15.2:0/3550300718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55992156df50 con 0x559921571b90 2024-09-10T08:14:58.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.197+0000 7f8b3d800640 1 --2- 172.21.15.2:0/3550300718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559921571b90 0x559921594740 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0d5a90 tx=0x7f8b2c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d39278291e127a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:58.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.197+0000 7f8b2a000640 1 -- 172.21.15.2:0/3550300718 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c016c80 con 0x559921571b90 2024-09-10T08:14:58.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.197+0000 7f8b2a000640 1 -- 172.21.15.2:0/3550300718 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2c0d5300 con 0x559921571b90 2024-09-10T08:14:58.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.197+0000 7f8b2a000640 1 -- 172.21.15.2:0/3550300718 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00b2c0 con 0x559921571b90 2024-09-10T08:14:58.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.198+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3550300718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559921571b90 msgr2=0x559921594740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:58.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.198+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3550300718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559921571b90 0x559921594740 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0d5a90 tx=0x7f8b2c00ef90 comp rx=0 tx=0).stop 2024-09-10T08:14:58.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.198+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3550300718 shutdown_connections 2024-09-10T08:14:58.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.198+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3550300718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55992157cd70 0x55992156c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.198+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3550300718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599215724b0 0x559921594c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.198+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/3550300718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559921571b90 0x559921594740 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.198+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3550300718 >> 172.21.15.2:0/3550300718 conn(0x55992156e120 msgr2=0x559921578ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:58.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.198+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3550300718 shutdown_connections 2024-09-10T08:14:58.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.198+0000 7f8b42c7f740 1 -- 172.21.15.2:0/3550300718 wait complete. 2024-09-10T08:14:58.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.199+0000 7f8b42c7f740 1 -- start start 2024-09-10T08:14:58.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.199+0000 7f8b42c7f740 1 Processor -- start 2024-09-10T08:14:58.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b42c7f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599215724b0 0x559921585540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b42c7f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55992157cd70 0x5599215807d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b42c7f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559921580d10 0x559921581170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b42c7f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381af730 con 0x5599215724b0 2024-09-10T08:14:58.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b42c7f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381b0ba0 con 0x559921580d10 2024-09-10T08:14:58.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b42c7f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381afe00 con 0x55992157cd70 2024-09-10T08:14:58.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b42c7f740 10 client.0 _ll_register_callbacks cb 0x7f8b240040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:58.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b3d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599215724b0 0x559921585540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b3e200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559921580d10 0x559921581170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b3d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599215724b0 0x559921585540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50208/0 (socket says 172.21.15.2:50208) 2024-09-10T08:14:58.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b3d800640 1 -- 172.21.15.2:0/1292762098 learned_addr learned my addr 172.21.15.2:0/1292762098 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:58.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b3ce00640 1 --2- 172.21.15.2:0/1292762098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55992157cd70 0x5599215807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b3e200640 1 -- 172.21.15.2:0/1292762098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55992157cd70 msgr2=0x5599215807d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:58.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b3e200640 1 --2- 172.21.15.2:0/1292762098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55992157cd70 0x5599215807d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.200+0000 7f8b3e200640 1 -- 172.21.15.2:0/1292762098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599215724b0 msgr2=0x559921585540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:58.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.201+0000 7f8b3e200640 1 --2- 172.21.15.2:0/1292762098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599215724b0 0x559921585540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.201+0000 7f8b3e200640 1 -- 172.21.15.2:0/1292762098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599215ff070 con 0x559921580d10 2024-09-10T08:14:58.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.201+0000 7f8b3d800640 1 --2- 172.21.15.2:0/1292762098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599215724b0 0x559921585540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:58.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.201+0000 7f8b3e200640 1 --2- 172.21.15.2:0/1292762098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559921580d10 0x559921581170 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55992157fc50 tx=0x7f8b30004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4f441924c2bb2a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:58.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.201+0000 7f8b1f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:14:58.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.201+0000 7f8b1f400640 1 -- 172.21.15.2:0/1292762098 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30004860 con 0x559921580d10 2024-09-10T08:14:58.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.201+0000 7f8b1f400640 1 -- 172.21.15.2:0/1292762098 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8b30004b20 con 0x559921580d10 2024-09-10T08:14:58.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.201+0000 7f8b42c7f740 1 -- 172.21.15.2:0/1292762098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b2c0d5300 con 0x559921580d10 2024-09-10T08:14:58.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.202+0000 7f8b1f400640 1 -- 172.21.15.2:0/1292762098 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8b30035050 con 0x559921580d10 2024-09-10T08:14:58.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.202+0000 7f8b42c7f740 10 client.27275 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:58.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.202+0000 7f8b42c7f740 1 -- 172.21.15.2:0/1292762098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5599216003c0 con 0x559921580d10 2024-09-10T08:14:58.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.202+0000 7f8b1f400640 1 -- 172.21.15.2:0/1292762098 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2c0d5300 con 0x559921580d10 2024-09-10T08:14:58.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.202+0000 7f8b1f400640 1 -- 172.21.15.2:0/1292762098 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8b3001f430 con 0x559921580d10 2024-09-10T08:14:58.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.202+0000 7f8b42c7f740 10 client.27275 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:58.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.202+0000 7f8b42c7f740 10 client.27275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:14:58.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.202+0000 7f8b42c7f740 20 client.27275 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:58.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.202+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/1292762098 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559921600b60 0x559921602f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:58.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.202+0000 7f8b42c7f740 4 client.27275 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:14:58.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.202+0000 7f8b42c7f740 1 -- 172.21.15.2:0/1292762098 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5599214dfe60 con 0x559921600b60 2024-09-10T08:14:58.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.203+0000 7f8b3d800640 1 --2- 172.21.15.2:0/1292762098 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559921600b60 0x559921602f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:58.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.203+0000 7f8b3d800640 1 --2- 172.21.15.2:0/1292762098 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559921600b60 0x559921602f80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c064981496e9c943 server_cookie=2b23fab1e1b03d01 in_seq=0 out_seq=0 2024-09-10T08:14:58.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.203+0000 7f8b1f400640 10 client.27275 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:58.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.203+0000 7f8b1f400640 1 -- 172.21.15.2:0/1292762098 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5599214dfe60 con 0x559921600b60 2024-09-10T08:14:58.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.203+0000 7f8b1f400640 10 client.27275 handle_command_reply: tid=0 2024-09-10T08:14:58.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.204+0000 7f8b42c7f740 1 client.27275 shutdown 2024-09-10T08:14:58.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.204+0000 7f8b42c7f740 1 -- 172.21.15.2:0/1292762098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559921580d10 msgr2=0x559921581170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:58.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.204+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/1292762098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559921580d10 0x559921581170 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55992157fc50 tx=0x7f8b30004a40 comp rx=0 tx=0).stop 2024-09-10T08:14:58.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.204+0000 7f8b42c7f740 1 -- 172.21.15.2:0/1292762098 shutdown_connections 2024-09-10T08:14:58.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.204+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/1292762098 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559921600b60 0x559921602f80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.204+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/1292762098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559921580d10 0x559921581170 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.204+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/1292762098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55992157cd70 0x5599215807d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.204+0000 7f8b42c7f740 1 --2- 172.21.15.2:0/1292762098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599215724b0 0x559921585540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.204+0000 7f8b42c7f740 1 -- 172.21.15.2:0/1292762098 >> 172.21.15.2:0/1292762098 conn(0x55992156e120 msgr2=0x55992156c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:58.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.204+0000 7f8b1f400640 0 client.27275 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:58.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.204+0000 7f8b42c7f740 1 -- 172.21.15.2:0/1292762098 shutdown_connections 2024-09-10T08:14:58.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.204+0000 7f8b42c7f740 1 -- 172.21.15.2:0/1292762098 wait complete. 2024-09-10T08:14:58.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.204+0000 7f8b42c7f740 20 client.27275 trim_cache size 0 max 16384 2024-09-10T08:14:58.208 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:14:58.208 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:14:58.208 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:14:58.208 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:14:58.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.205+0000 7f8b40200640 1 -- 172.21.15.2:0/475878140 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8b200a2b80 msgr2=0x7f8b200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:58.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.205+0000 7f8b40200640 1 --2- 172.21.15.2:0/475878140 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8b200a2b80 0x7f8b200a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009fd0 tx=0x7f8b30009500 comp rx=0 tx=0).stop 2024-09-10T08:14:58.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.205+0000 7f8b40200640 1 -- 172.21.15.2:0/475878140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b38101df0 msgr2=0x7f8b3819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:58.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.205+0000 7f8b40200640 1 --2- 172.21.15.2:0/475878140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b38101df0 0x7f8b3819a6a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00ea60 tx=0x7f8b2c00ef20 comp rx=0 tx=0).stop 2024-09-10T08:14:58.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.205+0000 7f8b40200640 1 -- 172.21.15.2:0/475878140 shutdown_connections 2024-09-10T08:14:58.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.205+0000 7f8b40200640 1 --2- 172.21.15.2:0/475878140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b38106f20 0x7f8b3819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.205+0000 7f8b40200640 1 --2- 172.21.15.2:0/475878140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b381027a0 0x7f8b3819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.205+0000 7f8b40200640 1 --2- 172.21.15.2:0/475878140 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8b200a2b80 0x7f8b200a5040 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.205+0000 7f8b40200640 1 --2- 172.21.15.2:0/475878140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b38101df0 0x7f8b3819a6a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:58.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.205+0000 7f8b40200640 1 -- 172.21.15.2:0/475878140 >> 172.21.15.2:0/475878140 conn(0x7f8b380fd8b0 msgr2=0x7f8b380ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:58.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.206+0000 7f8b40200640 1 -- 172.21.15.2:0/475878140 shutdown_connections 2024-09-10T08:14:58.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:58.206+0000 7f8b40200640 1 -- 172.21.15.2:0/475878140 wait complete. 2024-09-10T08:14:58.217 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:14:59.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:14:58 smithi002 ceph-mon[30563]: pgmap v592: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-10T08:14:59.403 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:14:59.404 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:14:59.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:14:58 smithi155 ceph-mon[35405]: pgmap v592: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-10T08:14:59.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:14:58 smithi107 ceph-mon[35048]: pgmap v592: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-10T08:14:59.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.644+0000 7fc04d400640 1 -- 172.21.15.2:0/4147790718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0481047e0 msgr2=0x7fc048108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:59.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.644+0000 7fc04d400640 1 --2- 172.21.15.2:0/4147790718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0481047e0 0x7fc048108830 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc034009900 tx=0x7fc03402a410 comp rx=0 tx=0).stop 2024-09-10T08:14:59.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.644+0000 7fc04d400640 1 -- 172.21.15.2:0/4147790718 shutdown_connections 2024-09-10T08:14:59.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.644+0000 7fc04d400640 1 --2- 172.21.15.2:0/4147790718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048108f60 0x7fc048110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.644+0000 7fc04d400640 1 --2- 172.21.15.2:0/4147790718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0481047e0 0x7fc048108830 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.644+0000 7fc04d400640 1 --2- 172.21.15.2:0/4147790718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc048103e30 0x7fc048104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.644+0000 7fc04d400640 1 -- 172.21.15.2:0/4147790718 >> 172.21.15.2:0/4147790718 conn(0x7fc0480ff8d0 msgr2=0x7fc048101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:59.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.645+0000 7fc04d400640 1 -- 172.21.15.2:0/4147790718 shutdown_connections 2024-09-10T08:14:59.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.645+0000 7fc04d400640 1 -- 172.21.15.2:0/4147790718 wait complete. 2024-09-10T08:14:59.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.646+0000 7fc04d400640 1 Processor -- start 2024-09-10T08:14:59.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.646+0000 7fc04d400640 1 -- start start 2024-09-10T08:14:59.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.646+0000 7fc04d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048103e30 0x7fc04819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.646+0000 7fc04d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0481047e0 0x7fc04819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.646+0000 7fc04d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc048108f60 0x7fc04819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.646+0000 7fc04d400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc048113fc0 con 0x7fc0481047e0 2024-09-10T08:14:59.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.646+0000 7fc04d400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc048113e40 con 0x7fc048103e30 2024-09-10T08:14:59.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.646+0000 7fc04d400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc048114140 con 0x7fc048108f60 2024-09-10T08:14:59.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.647+0000 7fc047e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048103e30 0x7fc04819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.647+0000 7fc047400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0481047e0 0x7fc04819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.647+0000 7fc047e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048103e30 0x7fc04819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35006/0 (socket says 172.21.15.2:35006) 2024-09-10T08:14:59.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.647+0000 7fc047e00640 1 -- 172.21.15.2:0/1728067948 learned_addr learned my addr 172.21.15.2:0/1728067948 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:59.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.647+0000 7fc04ca00640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc048108f60 0x7fc04819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.647+0000 7fc047e00640 1 -- 172.21.15.2:0/1728067948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc048108f60 msgr2=0x7fc04819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:59.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.647+0000 7fc047e00640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc048108f60 0x7fc04819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.647+0000 7fc047e00640 1 -- 172.21.15.2:0/1728067948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0481047e0 msgr2=0x7fc04819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:59.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.647+0000 7fc047e00640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0481047e0 0x7fc04819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.647+0000 7fc047e00640 1 -- 172.21.15.2:0/1728067948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04819f450 con 0x7fc048103e30 2024-09-10T08:14:59.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.647+0000 7fc04ca00640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc048108f60 0x7fc04819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:59.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.648+0000 7fc047e00640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048103e30 0x7fc04819a4a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc03800e9f0 tx=0x7fc03800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:59.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.648+0000 7fc044c00640 1 -- 172.21.15.2:0/1728067948 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03800ccc0 con 0x7fc048103e30 2024-09-10T08:14:59.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.648+0000 7fc04d400640 1 -- 172.21.15.2:0/1728067948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04819f740 con 0x7fc048103e30 2024-09-10T08:14:59.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.648+0000 7fc044c00640 1 -- 172.21.15.2:0/1728067948 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc03800ce60 con 0x7fc048103e30 2024-09-10T08:14:59.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.648+0000 7fc044c00640 1 -- 172.21.15.2:0/1728067948 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038010690 con 0x7fc048103e30 2024-09-10T08:14:59.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.648+0000 7fc04d400640 1 -- 172.21.15.2:0/1728067948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0481a7020 con 0x7fc048103e30 2024-09-10T08:14:59.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.651+0000 7fc0514a4740 1 Processor -- start 2024-09-10T08:14:59.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.651+0000 7fc0514a4740 1 -- start start 2024-09-10T08:14:59.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.652+0000 7fc0514a4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1560 0x55c0722b1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.652+0000 7fc0514a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1e80 0x55c0722bc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.652+0000 7fc0514a4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722bc740 0x55c0722beb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.652+0000 7fc0514a4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c072217530 con 0x55c0722b1560 2024-09-10T08:14:59.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.652+0000 7fc0514a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c0721be860 con 0x55c0722b1e80 2024-09-10T08:14:59.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.652+0000 7fc0514a4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c0722134f0 con 0x55c0722bc740 2024-09-10T08:14:59.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.652+0000 7fc04ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1560 0x55c0722b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.652+0000 7fc04ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1560 0x55c0722b1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50246/0 (socket says 172.21.15.2:50246) 2024-09-10T08:14:59.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.652+0000 7fc04ca00640 1 -- 172.21.15.2:0/798972649 learned_addr learned my addr 172.21.15.2:0/798972649 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:59.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.652+0000 7fc047400640 1 --2- 172.21.15.2:0/798972649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722bc740 0x55c0722beb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.653+0000 7fc04ca00640 1 -- 172.21.15.2:0/798972649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722bc740 msgr2=0x55c0722beb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:59.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.653+0000 7fc04ca00640 1 --2- 172.21.15.2:0/798972649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722bc740 0x55c0722beb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.653+0000 7fc04ca00640 1 -- 172.21.15.2:0/798972649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1e80 msgr2=0x55c0722bc200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:14:59.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.653+0000 7fc04ca00640 1 --2- 172.21.15.2:0/798972649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1e80 0x55c0722bc200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.653+0000 7fc04ca00640 1 -- 172.21.15.2:0/798972649 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c071f88b90 con 0x55c0722b1560 2024-09-10T08:14:59.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.653+0000 7fc04ca00640 1 --2- 172.21.15.2:0/798972649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1560 0x55c0722b1940 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc040011790 tx=0x7fc040011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7d488bc25a2090 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:59.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.653+0000 7fc03d200640 1 -- 172.21.15.2:0/798972649 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040004d20 con 0x55c0722b1560 2024-09-10T08:14:59.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.653+0000 7fc03d200640 1 -- 172.21.15.2:0/798972649 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc040003970 con 0x55c0722b1560 2024-09-10T08:14:59.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.653+0000 7fc03d200640 1 -- 172.21.15.2:0/798972649 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0400044e0 con 0x55c0722b1560 2024-09-10T08:14:59.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.654+0000 7fc044c00640 1 -- 172.21.15.2:0/1728067948 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc0380046c0 con 0x7fc048103e30 2024-09-10T08:14:59.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.654+0000 7fc0514a4740 1 -- 172.21.15.2:0/798972649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1560 msgr2=0x55c0722b1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:59.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.654+0000 7fc0514a4740 1 --2- 172.21.15.2:0/798972649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1560 0x55c0722b1940 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc040011790 tx=0x7fc040011c50 comp rx=0 tx=0).stop 2024-09-10T08:14:59.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.655+0000 7fc0514a4740 1 -- 172.21.15.2:0/798972649 shutdown_connections 2024-09-10T08:14:59.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.655+0000 7fc0514a4740 1 --2- 172.21.15.2:0/798972649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722bc740 0x55c0722beb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.655+0000 7fc0514a4740 1 --2- 172.21.15.2:0/798972649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1e80 0x55c0722bc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.655+0000 7fc0514a4740 1 --2- 172.21.15.2:0/798972649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1560 0x55c0722b1940 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.655+0000 7fc0514a4740 1 -- 172.21.15.2:0/798972649 >> 172.21.15.2:0/798972649 conn(0x55c0722adaf0 msgr2=0x55c0722aded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:59.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.655+0000 7fc0514a4740 1 -- 172.21.15.2:0/798972649 shutdown_connections 2024-09-10T08:14:59.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.655+0000 7fc0514a4740 1 -- 172.21.15.2:0/798972649 wait complete. 2024-09-10T08:14:59.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.655+0000 7fc0514a4740 1 -- start start 2024-09-10T08:14:59.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.656+0000 7fc0514a4740 1 Processor -- start 2024-09-10T08:14:59.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.656+0000 7fc044c00640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc02c0a2b10 0x7fc02c0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.656+0000 7fc044c00640 1 -- 172.21.15.2:0/1728067948 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc0380cf7f0 con 0x7fc048103e30 2024-09-10T08:14:59.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.656+0000 7fc0514a4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722b1560 0x55c0722b9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.656+0000 7fc04ca00640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc02c0a2b10 0x7fc02c0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.656+0000 7fc0514a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1e80 0x55c0722b9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.656+0000 7fc0514a4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722bc740 0x55c0722ba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.656+0000 7fc0514a4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c0722bf310 con 0x55c0722bc740 2024-09-10T08:14:59.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.656+0000 7fc0514a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c0721d1e60 con 0x55c0722b1e80 2024-09-10T08:14:59.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.656+0000 7fc0514a4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c0722bf490 con 0x55c0722b1560 2024-09-10T08:14:59.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.656+0000 7fc0514a4740 10 client.0 _ll_register_callbacks cb 0x55c0721d6cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:59.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.657+0000 7fc047e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722b1560 0x55c0722b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.657+0000 7fc047e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722b1560 0x55c0722b9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:44868/0 (socket says 172.21.15.2:44868) 2024-09-10T08:14:59.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.657+0000 7fc047e00640 1 -- 172.21.15.2:0/1672909783 learned_addr learned my addr 172.21.15.2:0/1672909783 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:59.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.657+0000 7fc04ca00640 1 --2- 172.21.15.2:0/1672909783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722bc740 0x55c0722ba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.657+0000 7fc047e00640 1 -- 172.21.15.2:0/1672909783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1e80 msgr2=0x55c0722b9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:59.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.657+0000 7fc047e00640 1 --2- 172.21.15.2:0/1672909783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1e80 0x55c0722b9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.657+0000 7fc047e00640 1 -- 172.21.15.2:0/1672909783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722bc740 msgr2=0x55c0722ba0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:59.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.657+0000 7fc047e00640 1 --2- 172.21.15.2:0/1672909783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722bc740 0x55c0722ba0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.657+0000 7fc047e00640 1 -- 172.21.15.2:0/1672909783 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0721dd990 con 0x55c0722b1560 2024-09-10T08:14:59.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.657+0000 7fc047e00640 1 --2- 172.21.15.2:0/1672909783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722b1560 0x55c0722b9620 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc03800c6a0 tx=0x7fc03809c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9867e82939d8c05f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:59.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.657+0000 7fc031600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:14:59.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.658+0000 7fc04ca00640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc02c0a2b10 0x7fc02c0a4fd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc040011760 tx=0x7fc040004160 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:59.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.658+0000 7fc031600640 1 -- 172.21.15.2:0/1672909783 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0380a4870 con 0x55c0722b1560 2024-09-10T08:14:59.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.658+0000 7fc0514a4740 1 -- 172.21.15.2:0/1672909783 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc040003970 con 0x55c0722b1560 2024-09-10T08:14:59.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.658+0000 7fc031600640 1 -- 172.21.15.2:0/1672909783 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc0380a43e0 con 0x55c0722b1560 2024-09-10T08:14:59.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.658+0000 7fc031600640 1 -- 172.21.15.2:0/1672909783 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc0380a4620 con 0x55c0722b1560 2024-09-10T08:14:59.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.658+0000 7fc0514a4740 10 client.27226 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:59.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.658+0000 7fc0514a4740 1 -- 172.21.15.2:0/1672909783 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c0722d37b0 con 0x55c0722b1560 2024-09-10T08:14:59.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.658+0000 7fc031600640 1 -- 172.21.15.2:0/1672909783 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc040003970 con 0x55c0722b1560 2024-09-10T08:14:59.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.659+0000 7fc031600640 1 -- 172.21.15.2:0/1672909783 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc0380a1140 con 0x55c0722b1560 2024-09-10T08:14:59.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.659+0000 7fc0514a4740 10 client.27226 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:59.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.659+0000 7fc0514a4740 10 client.27226 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:14:59.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.659+0000 7fc0514a4740 20 client.27226 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:59.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.659+0000 7fc0514a4740 1 --2- 172.21.15.2:0/1672909783 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c0722d4bc0 0x55c0722d6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.659+0000 7fc0514a4740 4 client.27226 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:14:59.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.659+0000 7fc0514a4740 1 -- 172.21.15.2:0/1672909783 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0721ba310 con 0x55c0722d4bc0 2024-09-10T08:14:59.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.659+0000 7fc047400640 1 --2- 172.21.15.2:0/1672909783 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c0722d4bc0 0x55c0722d6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.660+0000 7fc047400640 1 --2- 172.21.15.2:0/1672909783 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c0722d4bc0 0x55c0722d6fb0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19713e43cd992768 server_cookie=347da971fa090a18 in_seq=0 out_seq=0 2024-09-10T08:14:59.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.660+0000 7fc031600640 10 client.27226 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:59.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.661+0000 7fc031600640 1 -- 172.21.15.2:0/1672909783 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0721ba310 con 0x55c0722d4bc0 2024-09-10T08:14:59.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.661+0000 7fc031600640 10 client.27226 handle_command_reply: tid=0 2024-09-10T08:14:59.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.661+0000 7fc0514a4740 1 client.27226 shutdown 2024-09-10T08:14:59.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.661+0000 7fc0514a4740 1 -- 172.21.15.2:0/1672909783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722b1560 msgr2=0x55c0722b9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:59.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.661+0000 7fc0514a4740 1 --2- 172.21.15.2:0/1672909783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722b1560 0x55c0722b9620 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc03800c6a0 tx=0x7fc03809c3c0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.661+0000 7fc0514a4740 1 -- 172.21.15.2:0/1672909783 shutdown_connections 2024-09-10T08:14:59.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.661+0000 7fc0514a4740 1 --2- 172.21.15.2:0/1672909783 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c0722d4bc0 0x55c0722d6fb0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.661+0000 7fc0514a4740 1 --2- 172.21.15.2:0/1672909783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722bc740 0x55c0722ba0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.661+0000 7fc0514a4740 1 --2- 172.21.15.2:0/1672909783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1e80 0x55c0722b9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.661+0000 7fc0514a4740 1 --2- 172.21.15.2:0/1672909783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722b1560 0x55c0722b9620 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.661+0000 7fc0514a4740 1 -- 172.21.15.2:0/1672909783 >> 172.21.15.2:0/1672909783 conn(0x55c0722adaf0 msgr2=0x55c0722b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:59.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.661+0000 7fc031600640 0 client.27226 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:59.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.662+0000 7fc0514a4740 1 -- 172.21.15.2:0/1672909783 shutdown_connections 2024-09-10T08:14:59.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.662+0000 7fc0514a4740 1 -- 172.21.15.2:0/1672909783 wait complete. 2024-09-10T08:14:59.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.662+0000 7fc0514a4740 20 client.27226 trim_cache size 0 max 16384 2024-09-10T08:14:59.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.686+0000 7fc0514a4740 1 Processor -- start 2024-09-10T08:14:59.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.686+0000 7fc0514a4740 1 -- start start 2024-09-10T08:14:59.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc0514a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1560 0x55c0722b8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc0514a4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 0x55c0722b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc0514a4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722bc740 0x55c0722dc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc0514a4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0481af2d0 con 0x55c0722b1e80 2024-09-10T08:14:59.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc0514a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0481acba0 con 0x55c0722b1560 2024-09-10T08:14:59.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc0514a4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0481aa9f0 con 0x55c0722bc740 2024-09-10T08:14:59.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc047400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 0x55c0722b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc047400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 0x55c0722b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50272/0 (socket says 172.21.15.2:50272) 2024-09-10T08:14:59.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc047400640 1 -- 172.21.15.2:0/994643609 learned_addr learned my addr 172.21.15.2:0/994643609 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:59.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc047e00640 1 --2- 172.21.15.2:0/994643609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1560 0x55c0722b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc047400640 1 -- 172.21.15.2:0/994643609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722bc740 msgr2=0x55c0722dc7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:14:59.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc047400640 1 --2- 172.21.15.2:0/994643609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722bc740 0x55c0722dc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc047400640 1 -- 172.21.15.2:0/994643609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1560 msgr2=0x55c0722b8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:59.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc047400640 1 --2- 172.21.15.2:0/994643609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1560 0x55c0722b8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc047400640 1 -- 172.21.15.2:0/994643609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0722ad920 con 0x55c0722b1e80 2024-09-10T08:14:59.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.687+0000 7fc047e00640 1 --2- 172.21.15.2:0/994643609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1560 0x55c0722b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:59.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.688+0000 7fc047400640 1 --2- 172.21.15.2:0/994643609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 0x55c0722b3fc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc03402a930 tx=0x7fc034005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81184cdec05e63c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:59.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.688+0000 7fc033e00640 1 -- 172.21.15.2:0/994643609 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034048030 con 0x55c0722b1e80 2024-09-10T08:14:59.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.688+0000 7fc033e00640 1 -- 172.21.15.2:0/994643609 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc034031750 con 0x55c0722b1e80 2024-09-10T08:14:59.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.688+0000 7fc033e00640 1 -- 172.21.15.2:0/994643609 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034035040 con 0x55c0722b1e80 2024-09-10T08:14:59.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.689+0000 7fc0514a4740 1 -- 172.21.15.2:0/994643609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 msgr2=0x55c0722b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:59.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.689+0000 7fc0514a4740 1 --2- 172.21.15.2:0/994643609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 0x55c0722b3fc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc03402a930 tx=0x7fc034005b50 comp rx=0 tx=0).stop 2024-09-10T08:14:59.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.689+0000 7fc0514a4740 1 -- 172.21.15.2:0/994643609 shutdown_connections 2024-09-10T08:14:59.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.689+0000 7fc0514a4740 1 --2- 172.21.15.2:0/994643609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722bc740 0x55c0722dc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.689+0000 7fc0514a4740 1 --2- 172.21.15.2:0/994643609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 0x55c0722b3fc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.689+0000 7fc0514a4740 1 --2- 172.21.15.2:0/994643609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722b1560 0x55c0722b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.689+0000 7fc0514a4740 1 -- 172.21.15.2:0/994643609 >> 172.21.15.2:0/994643609 conn(0x55c0722adaf0 msgr2=0x55c0722acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:59.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.689+0000 7fc0514a4740 1 -- 172.21.15.2:0/994643609 shutdown_connections 2024-09-10T08:14:59.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.689+0000 7fc0514a4740 1 -- 172.21.15.2:0/994643609 wait complete. 2024-09-10T08:14:59.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.690+0000 7fc0514a4740 1 -- start start 2024-09-10T08:14:59.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.690+0000 7fc0514a4740 1 Processor -- start 2024-09-10T08:14:59.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.690+0000 7fc0514a4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722b1560 0x55c0722c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.690+0000 7fc0514a4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 0x55c0722c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.690+0000 7fc0514a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722bc740 0x55c07233e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.690+0000 7fc0514a4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0481ab060 con 0x55c0722b1e80 2024-09-10T08:14:59.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.690+0000 7fc0514a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0481ac4d0 con 0x55c0722bc740 2024-09-10T08:14:59.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.690+0000 7fc0514a4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0481ab730 con 0x55c0722b1560 2024-09-10T08:14:59.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.690+0000 7fc0514a4740 10 client.0 _ll_register_callbacks cb 0x7fc038009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:14:59.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.691+0000 7fc047400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 0x55c0722c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.691+0000 7fc047e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722b1560 0x55c0722c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.691+0000 7fc047400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 0x55c0722c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50282/0 (socket says 172.21.15.2:50282) 2024-09-10T08:14:59.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.691+0000 7fc047400640 1 -- 172.21.15.2:0/3918742044 learned_addr learned my addr 172.21.15.2:0/3918742044 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:14:59.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.691+0000 7fc04ca00640 1 --2- 172.21.15.2:0/3918742044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722bc740 0x55c07233e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.691+0000 7fc047400640 1 -- 172.21.15.2:0/3918742044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722b1560 msgr2=0x55c0722c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:59.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.691+0000 7fc047400640 1 --2- 172.21.15.2:0/3918742044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722b1560 0x55c0722c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.691+0000 7fc047400640 1 -- 172.21.15.2:0/3918742044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722bc740 msgr2=0x55c07233e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:14:59.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.691+0000 7fc047400640 1 --2- 172.21.15.2:0/3918742044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722bc740 0x55c07233e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.691+0000 7fc047400640 1 -- 172.21.15.2:0/3918742044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0722ad380 con 0x55c0722b1e80 2024-09-10T08:14:59.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.691+0000 7fc047e00640 1 --2- 172.21.15.2:0/3918742044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722b1560 0x55c0722c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:14:59.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.692+0000 7fc047400640 1 --2- 172.21.15.2:0/3918742044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 0x55c0722c95a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc034000c00 tx=0x7fc03402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939c530e29b28f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:14:59.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.692+0000 7fc031600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:14:59.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.692+0000 7fc031600640 1 -- 172.21.15.2:0/3918742044 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034050030 con 0x55c0722b1e80 2024-09-10T08:14:59.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.692+0000 7fc031600640 1 -- 172.21.15.2:0/3918742044 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc034044040 con 0x55c0722b1e80 2024-09-10T08:14:59.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.692+0000 7fc0514a4740 1 -- 172.21.15.2:0/3918742044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc034031750 con 0x55c0722b1e80 2024-09-10T08:14:59.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.692+0000 7fc031600640 1 -- 172.21.15.2:0/3918742044 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc03405d050 con 0x55c0722b1e80 2024-09-10T08:14:59.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.692+0000 7fc0514a4740 10 client.17250 fetch_fsmap learned FSMap version 26 2024-09-10T08:14:59.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.692+0000 7fc0514a4740 1 -- 172.21.15.2:0/3918742044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c0722a9310 con 0x55c0722b1e80 2024-09-10T08:14:59.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.693+0000 7fc031600640 1 -- 172.21.15.2:0/3918742044 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc034031750 con 0x55c0722b1e80 2024-09-10T08:14:59.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.693+0000 7fc031600640 1 -- 172.21.15.2:0/3918742044 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc03403d5f0 con 0x55c0722b1e80 2024-09-10T08:14:59.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.693+0000 7fc0514a4740 10 client.17250 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:14:59.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.693+0000 7fc0514a4740 10 client.17250 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:14:59.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.693+0000 7fc0514a4740 20 client.17250 populate_metadata read hostname 'smithi002' 2024-09-10T08:14:59.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.693+0000 7fc0514a4740 1 --2- 172.21.15.2:0/3918742044 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c072340130 0x55c072342550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:14:59.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.693+0000 7fc0514a4740 4 client.17250 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T08:14:59.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.693+0000 7fc0514a4740 1 -- 172.21.15.2:0/3918742044 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x55c072224c60 con 0x55c072340130 2024-09-10T08:14:59.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.694+0000 7fc047e00640 1 --2- 172.21.15.2:0/3918742044 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c072340130 0x55c072342550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:14:59.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.694+0000 7fc047e00640 1 --2- 172.21.15.2:0/3918742044 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c072340130 0x55c072342550 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6024007336a4a51a server_cookie=81bbee71ab5951ed in_seq=0 out_seq=0 2024-09-10T08:14:59.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.694+0000 7fc031600640 10 client.17250 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:59.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc031600640 1 -- 172.21.15.2:0/3918742044 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c072224c60 con 0x55c072340130 2024-09-10T08:14:59.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc031600640 10 client.17250 handle_command_reply: tid=0 2024-09-10T08:14:59.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc0514a4740 1 client.17250 shutdown 2024-09-10T08:14:59.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc0514a4740 1 -- 172.21.15.2:0/3918742044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 msgr2=0x55c0722c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:59.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc0514a4740 1 --2- 172.21.15.2:0/3918742044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 0x55c0722c95a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc034000c00 tx=0x7fc03402a410 comp rx=0 tx=0).stop 2024-09-10T08:14:59.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc0514a4740 1 -- 172.21.15.2:0/3918742044 shutdown_connections 2024-09-10T08:14:59.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc0514a4740 1 --2- 172.21.15.2:0/3918742044 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c072340130 0x55c072342550 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc0514a4740 1 --2- 172.21.15.2:0/3918742044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0722bc740 0x55c07233e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc0514a4740 1 --2- 172.21.15.2:0/3918742044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0722b1e80 0x55c0722c95a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc0514a4740 1 --2- 172.21.15.2:0/3918742044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0722b1560 0x55c0722c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc0514a4740 1 -- 172.21.15.2:0/3918742044 >> 172.21.15.2:0/3918742044 conn(0x55c0722adaf0 msgr2=0x55c0722cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:59.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc031600640 0 client.17250 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:14:59.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc0514a4740 1 -- 172.21.15.2:0/3918742044 shutdown_connections 2024-09-10T08:14:59.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc0514a4740 1 -- 172.21.15.2:0/3918742044 wait complete. 2024-09-10T08:14:59.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.703+0000 7fc0514a4740 20 client.17250 trim_cache size 0 max 16384 2024-09-10T08:14:59.704 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:14:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:14:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:14:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 22, 2024-09-10T08:14:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:14:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "7dd6451a": { 2024-09-10T08:14:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 22, 2024-09-10T08:14:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:14:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:14:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:14:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:14:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:14:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:14:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 140.1, 2024-09-10T08:14:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:14:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:14:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:14:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:14:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:14:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:14:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:14:59.706 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:14:59.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.704+0000 7fc04d400640 1 -- 172.21.15.2:0/1728067948 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc02c0a2b10 msgr2=0x7fc02c0a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:59.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.704+0000 7fc04d400640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc02c0a2b10 0x7fc02c0a4fd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc040011760 tx=0x7fc040004160 comp rx=0 tx=0).stop 2024-09-10T08:14:59.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.704+0000 7fc04d400640 1 -- 172.21.15.2:0/1728067948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048103e30 msgr2=0x7fc04819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:14:59.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.704+0000 7fc04d400640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048103e30 0x7fc04819a4a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc03800e9f0 tx=0x7fc03800edf0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.704+0000 7fc04d400640 1 -- 172.21.15.2:0/1728067948 shutdown_connections 2024-09-10T08:14:59.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.704+0000 7fc04d400640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc02c0a2b10 0x7fc02c0a4fd0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.704+0000 7fc04d400640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc048108f60 0x7fc04819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.704+0000 7fc04d400640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0481047e0 0x7fc04819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.704+0000 7fc04d400640 1 --2- 172.21.15.2:0/1728067948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc048103e30 0x7fc04819a4a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:14:59.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.704+0000 7fc04d400640 1 -- 172.21.15.2:0/1728067948 >> 172.21.15.2:0/1728067948 conn(0x7fc0480ff8d0 msgr2=0x7fc048100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:14:59.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.705+0000 7fc04d400640 1 -- 172.21.15.2:0/1728067948 shutdown_connections 2024-09-10T08:14:59.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:14:59.705+0000 7fc04d400640 1 -- 172.21.15.2:0/1728067948 wait complete. 2024-09-10T08:15:00.586 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:00.617 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7dd6451a', quiesce took 5.2 seconds. Will release after: 14.900000000000002 2024-09-10T08:15:01.194 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:00 smithi002 ceph-mon[30563]: pgmap v593: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-10T08:15:01.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.216+0000 7f46dba00640 1 -- 172.21.15.2:0/3490573344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46d4074590 msgr2=0x7f46d4074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:01.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.216+0000 7f46dba00640 1 --2- 172.21.15.2:0/3490573344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46d4074590 0x7f46d4074970 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009900 tx=0x7f46c802a480 comp rx=0 tx=0).stop 2024-09-10T08:15:01.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.216+0000 7f46dba00640 1 -- 172.21.15.2:0/3490573344 shutdown_connections 2024-09-10T08:15:01.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.216+0000 7f46dba00640 1 --2- 172.21.15.2:0/3490573344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4112b90 0x7f46d4114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:01.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.216+0000 7f46dba00640 1 --2- 172.21.15.2:0/3490573344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46d4074eb0 0x7f46d4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:01.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.216+0000 7f46dba00640 1 --2- 172.21.15.2:0/3490573344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46d4074590 0x7f46d4074970 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:01.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.216+0000 7f46dba00640 1 -- 172.21.15.2:0/3490573344 >> 172.21.15.2:0/3490573344 conn(0x7f46d40fd8c0 msgr2=0x7f46d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:01.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.216+0000 7f46dba00640 1 -- 172.21.15.2:0/3490573344 shutdown_connections 2024-09-10T08:15:01.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.216+0000 7f46dba00640 1 -- 172.21.15.2:0/3490573344 wait complete. 2024-09-10T08:15:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.217+0000 7f46dba00640 1 Processor -- start 2024-09-10T08:15:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.218+0000 7f46dba00640 1 -- start start 2024-09-10T08:15:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.218+0000 7f46dba00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46d4074590 0x7f46d41a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.218+0000 7f46dba00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4074eb0 0x7f46d41a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.218+0000 7f46dba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46d4112b90 0x7f46d419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.218+0000 7f46dba00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4118550 con 0x7f46d4074590 2024-09-10T08:15:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.218+0000 7f46dba00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46d41183d0 con 0x7f46d4074eb0 2024-09-10T08:15:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.218+0000 7f46dba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f46d41186d0 con 0x7f46d4112b90 2024-09-10T08:15:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.218+0000 7f46d9000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46d4074590 0x7f46d41a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.218+0000 7f46d9000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46d4074590 0x7f46d41a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50298/0 (socket says 172.21.15.2:50298) 2024-09-10T08:15:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.218+0000 7f46d9000640 1 -- 172.21.15.2:0/3709356899 learned_addr learned my addr 172.21.15.2:0/3709356899 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.218+0000 7f46d9a00640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46d4112b90 0x7f46d419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.218+0000 7f46d3e00640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4074eb0 0x7f46d41a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.219+0000 7f46d9000640 1 -- 172.21.15.2:0/3709356899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46d4112b90 msgr2=0x7f46d419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.219+0000 7f46d9000640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46d4112b90 0x7f46d419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.219+0000 7f46d9000640 1 -- 172.21.15.2:0/3709356899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4074eb0 msgr2=0x7f46d41a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.219+0000 7f46d9000640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4074eb0 0x7f46d41a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.219+0000 7f46d9000640 1 -- 172.21.15.2:0/3709356899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d419f010 con 0x7f46d4074590 2024-09-10T08:15:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.219+0000 7f46d9a00640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46d4112b90 0x7f46d419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.219+0000 7f46d3e00640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4074eb0 0x7f46d41a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:01.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.219+0000 7f46d9000640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46d4074590 0x7f46d41a4720 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f46c8006010 tx=0x7f46c8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.220+0000 7f46d1600640 1 -- 172.21.15.2:0/3709356899 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c803f030 con 0x7f46d4074590 2024-09-10T08:15:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.220+0000 7f46d1600640 1 -- 172.21.15.2:0/3709356899 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c8004440 con 0x7f46d4074590 2024-09-10T08:15:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.220+0000 7f46d1600640 1 -- 172.21.15.2:0/3709356899 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8035070 con 0x7f46d4074590 2024-09-10T08:15:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.220+0000 7f46dba00640 1 -- 172.21.15.2:0/3709356899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d419f2a0 con 0x7f46d4074590 2024-09-10T08:15:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.220+0000 7f46dba00640 1 -- 172.21.15.2:0/3709356899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d41ab5f0 con 0x7f46d4074590 2024-09-10T08:15:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.221+0000 7f46dba00640 1 -- 172.21.15.2:0/3709356899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d4074970 con 0x7f46d4074590 2024-09-10T08:15:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.223+0000 7f46d1600640 1 -- 172.21.15.2:0/3709356899 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f46c8042030 con 0x7f46d4074590 2024-09-10T08:15:01.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.224+0000 7f46d1600640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46bc0a2c00 0x7f46bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:01.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.224+0000 7f46d1600640 1 -- 172.21.15.2:0/3709356899 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f46c80ef760 con 0x7f46d4074590 2024-09-10T08:15:01.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.224+0000 7f46d3e00640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46bc0a2c00 0x7f46bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:01.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.225+0000 7f46d3e00640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46bc0a2c00 0x7f46bc0a50c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f46d40fff30 tx=0x7f46c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.229+0000 7f46d1600640 1 -- 172.21.15.2:0/3709356899 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46c80bbc20 con 0x7f46d4074590 2024-09-10T08:15:01.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:00 smithi155 ceph-mon[35405]: pgmap v593: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-10T08:15:01.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:00 smithi107 ceph-mon[35048]: pgmap v593: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-10T08:15:01.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.454+0000 7f46dba00640 1 -- 172.21.15.2:0/3709356899 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46d419fe10 con 0x7f46d4074590 2024-09-10T08:15:01.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.455+0000 7f46d1600640 1 -- 172.21.15.2:0/3709356899 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f46c80bff20 con 0x7f46d4074590 2024-09-10T08:15:01.455 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:15:01.455 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:15:01.456 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:15:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.457+0000 7f46dba00640 1 -- 172.21.15.2:0/3709356899 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46bc0a2c00 msgr2=0x7f46bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.457+0000 7f46dba00640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46bc0a2c00 0x7f46bc0a50c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f46d40fff30 tx=0x7f46c0005f90 comp rx=0 tx=0).stop 2024-09-10T08:15:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.457+0000 7f46dba00640 1 -- 172.21.15.2:0/3709356899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46d4074590 msgr2=0x7f46d41a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.457+0000 7f46dba00640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46d4074590 0x7f46d41a4720 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f46c8006010 tx=0x7f46c8004040 comp rx=0 tx=0).stop 2024-09-10T08:15:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.458+0000 7f46dba00640 1 -- 172.21.15.2:0/3709356899 shutdown_connections 2024-09-10T08:15:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.458+0000 7f46dba00640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46bc0a2c00 0x7f46bc0a50c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.458+0000 7f46dba00640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46d4112b90 0x7f46d419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.458+0000 7f46dba00640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4074eb0 0x7f46d41a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.458+0000 7f46dba00640 1 --2- 172.21.15.2:0/3709356899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46d4074590 0x7f46d41a4720 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.458+0000 7f46dba00640 1 -- 172.21.15.2:0/3709356899 >> 172.21.15.2:0/3709356899 conn(0x7f46d40fd8c0 msgr2=0x7f46d40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.458+0000 7f46dba00640 1 -- 172.21.15.2:0/3709356899 shutdown_connections 2024-09-10T08:15:01.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:01.458+0000 7f46dba00640 1 -- 172.21.15.2:0/3709356899 wait complete. 2024-09-10T08:15:02.110 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:01 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3709356899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:15:02.244 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:15:02.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:01 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3709356899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:15:02.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:01 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3709356899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:15:02.625 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:03.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:02 smithi002 ceph-mon[30563]: pgmap v594: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-09-10T08:15:03.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.218+0000 7ff33c600640 1 -- 172.21.15.2:0/2748537829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff334103ff0 msgr2=0x7ff3341043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:03.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.218+0000 7ff33c600640 1 --2- 172.21.15.2:0/2748537829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff334103ff0 0x7ff3341043d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff328009900 tx=0x7ff32802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.219+0000 7ff33c600640 1 -- 172.21.15.2:0/2748537829 shutdown_connections 2024-09-10T08:15:03.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.219+0000 7ff33c600640 1 --2- 172.21.15.2:0/2748537829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff334109120 0x7ff334110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.219+0000 7ff33c600640 1 --2- 172.21.15.2:0/2748537829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3341049a0 0x7ff3341089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.219+0000 7ff33c600640 1 --2- 172.21.15.2:0/2748537829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff334103ff0 0x7ff3341043d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.219+0000 7ff33c600640 1 -- 172.21.15.2:0/2748537829 >> 172.21.15.2:0/2748537829 conn(0x7ff3340ffa90 msgr2=0x7ff334101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:03.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.219+0000 7ff33c600640 1 -- 172.21.15.2:0/2748537829 shutdown_connections 2024-09-10T08:15:03.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.219+0000 7ff33c600640 1 -- 172.21.15.2:0/2748537829 wait complete. 2024-09-10T08:15:03.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.220+0000 7ff33c600640 1 Processor -- start 2024-09-10T08:15:03.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.220+0000 7ff33c600640 1 -- start start 2024-09-10T08:15:03.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.221+0000 7ff33c600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff334103ff0 0x7ff33419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.221+0000 7ff33c600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3341049a0 0x7ff33419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.221+0000 7ff33c600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff334109120 0x7ff33419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.221+0000 7ff33c600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff334114120 con 0x7ff334109120 2024-09-10T08:15:03.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.221+0000 7ff33c600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff334113fa0 con 0x7ff334103ff0 2024-09-10T08:15:03.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.221+0000 7ff33c600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff3341142a0 con 0x7ff3341049a0 2024-09-10T08:15:03.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.221+0000 7ff33a600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff334109120 0x7ff33419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.221+0000 7ff339c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff334103ff0 0x7ff33419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.221+0000 7ff339c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff334103ff0 0x7ff33419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35076/0 (socket says 172.21.15.2:35076) 2024-09-10T08:15:03.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.221+0000 7ff33a600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff334109120 0x7ff33419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50314/0 (socket says 172.21.15.2:50314) 2024-09-10T08:15:03.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.221+0000 7ff339c00640 1 -- 172.21.15.2:0/1895299810 learned_addr learned my addr 172.21.15.2:0/1895299810 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:03.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.221+0000 7ff339200640 1 --2- 172.21.15.2:0/1895299810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3341049a0 0x7ff33419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.222+0000 7ff339c00640 1 -- 172.21.15.2:0/1895299810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3341049a0 msgr2=0x7ff33419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.222+0000 7ff339c00640 1 --2- 172.21.15.2:0/1895299810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3341049a0 0x7ff33419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.222+0000 7ff339c00640 1 -- 172.21.15.2:0/1895299810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff334109120 msgr2=0x7ff33419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.222+0000 7ff339c00640 1 --2- 172.21.15.2:0/1895299810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff334109120 0x7ff33419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.222+0000 7ff339c00640 1 -- 172.21.15.2:0/1895299810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33419f590 con 0x7ff334103ff0 2024-09-10T08:15:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.222+0000 7ff339c00640 1 --2- 172.21.15.2:0/1895299810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff334103ff0 0x7ff33419a5e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff328006010 tx=0x7ff3280042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.222+0000 7ff332000640 1 -- 172.21.15.2:0/1895299810 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32803f030 con 0x7ff334103ff0 2024-09-10T08:15:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.222+0000 7ff332000640 1 -- 172.21.15.2:0/1895299810 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff328003ce0 con 0x7ff334103ff0 2024-09-10T08:15:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.222+0000 7ff33c600640 1 -- 172.21.15.2:0/1895299810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33419f820 con 0x7ff334103ff0 2024-09-10T08:15:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.222+0000 7ff332000640 1 -- 172.21.15.2:0/1895299810 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328035070 con 0x7ff334103ff0 2024-09-10T08:15:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.222+0000 7ff33c600640 1 -- 172.21.15.2:0/1895299810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3341a7160 con 0x7ff334103ff0 2024-09-10T08:15:03.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.226+0000 7ff33f200740 1 Processor -- start 2024-09-10T08:15:03.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.226+0000 7ff332000640 1 -- 172.21.15.2:0/1895299810 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff328042030 con 0x7ff334103ff0 2024-09-10T08:15:03.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.226+0000 7ff33f200740 1 -- start start 2024-09-10T08:15:03.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.226+0000 7ff33f200740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a8c70 0x55d9389a9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.226+0000 7ff33f200740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.226+0000 7ff33f200740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d9389b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.226+0000 7ff33f200740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d93890f3e0 con 0x55d9389a9590 2024-09-10T08:15:03.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.226+0000 7ff33f200740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d9388b6d80 con 0x55d9389b3e50 2024-09-10T08:15:03.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.226+0000 7ff33f200740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d93890b3a0 con 0x55d9389a8c70 2024-09-10T08:15:03.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.226+0000 7ff339c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.226+0000 7ff339200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d9389b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.227+0000 7ff339200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d9389b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35090/0 (socket says 172.21.15.2:35090) 2024-09-10T08:15:03.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.227+0000 7ff339c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50316/0 (socket says 172.21.15.2:50316) 2024-09-10T08:15:03.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.227+0000 7ff339200640 1 -- 172.21.15.2:0/2327938314 learned_addr learned my addr 172.21.15.2:0/2327938314 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:03.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.227+0000 7ff33a600640 1 --2- 172.21.15.2:0/2327938314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a8c70 0x55d9389a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.227+0000 7ff339200640 1 -- 172.21.15.2:0/2327938314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a8c70 msgr2=0x55d9389a9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:03.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.227+0000 7ff339200640 1 --2- 172.21.15.2:0/2327938314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a8c70 0x55d9389a9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.227+0000 7ff339200640 1 -- 172.21.15.2:0/2327938314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 msgr2=0x55d9389b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:03.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.227+0000 7ff339200640 1 --2- 172.21.15.2:0/2327938314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.227+0000 7ff339200640 1 -- 172.21.15.2:0/2327938314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9388d5ae0 con 0x55d9389b3e50 2024-09-10T08:15:03.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.227+0000 7ff332000640 1 --2- 172.21.15.2:0/1895299810 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff31c0a2c00 0x7ff31c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.227+0000 7ff339200640 1 --2- 172.21.15.2:0/2327938314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d9389b6240 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff320009c90 tx=0x7ff320011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e102249f2383511 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:03.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.228+0000 7ff332000640 1 -- 172.21.15.2:0/1895299810 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff3280ef8c0 con 0x7ff334103ff0 2024-09-10T08:15:03.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.228+0000 7ff33a600640 1 --2- 172.21.15.2:0/1895299810 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff31c0a2c00 0x7ff31c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.228+0000 7ff326a00640 1 -- 172.21.15.2:0/2327938314 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32001e070 con 0x55d9389b3e50 2024-09-10T08:15:03.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.228+0000 7ff326a00640 1 -- 172.21.15.2:0/2327938314 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff320019b10 con 0x55d9389b3e50 2024-09-10T08:15:03.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.228+0000 7ff326a00640 1 -- 172.21.15.2:0/2327938314 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32000eac0 con 0x55d9389b3e50 2024-09-10T08:15:03.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.228+0000 7ff33f200740 1 -- 172.21.15.2:0/2327938314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 msgr2=0x55d9389b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:03.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.228+0000 7ff33a600640 1 --2- 172.21.15.2:0/1895299810 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff31c0a2c00 0x7ff31c0a50c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff32c005fd0 tx=0x7ff32c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:03.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.228+0000 7ff33f200740 1 --2- 172.21.15.2:0/2327938314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d9389b6240 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff320009c90 tx=0x7ff320011810 comp rx=0 tx=0).stop 2024-09-10T08:15:03.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.229+0000 7ff33f200740 1 -- 172.21.15.2:0/2327938314 shutdown_connections 2024-09-10T08:15:03.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.229+0000 7ff33f200740 1 --2- 172.21.15.2:0/2327938314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d9389b6240 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.229+0000 7ff33f200740 1 --2- 172.21.15.2:0/2327938314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.229+0000 7ff33f200740 1 --2- 172.21.15.2:0/2327938314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a8c70 0x55d9389a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.229+0000 7ff33f200740 1 -- 172.21.15.2:0/2327938314 >> 172.21.15.2:0/2327938314 conn(0x55d9389a5200 msgr2=0x55d9389a55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:03.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.229+0000 7ff33f200740 1 -- 172.21.15.2:0/2327938314 shutdown_connections 2024-09-10T08:15:03.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.229+0000 7ff33f200740 1 -- 172.21.15.2:0/2327938314 wait complete. 2024-09-10T08:15:03.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.229+0000 7ff33f200740 1 -- start start 2024-09-10T08:15:03.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.229+0000 7ff33f200740 1 Processor -- start 2024-09-10T08:15:03.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.230+0000 7ff33f200740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a8c70 0x55d9389b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.230+0000 7ff33f200740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.230+0000 7ff33f200740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d9389b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.230+0000 7ff33f200740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d9389b6a20 con 0x55d9389a9590 2024-09-10T08:15:03.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.230+0000 7ff33f200740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d9388ca0f0 con 0x55d9389b3e50 2024-09-10T08:15:03.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.230+0000 7ff33f200740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d9389b6ba0 con 0x55d9389a8c70 2024-09-10T08:15:03.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.230+0000 7ff33f200740 10 client.0 _ll_register_callbacks cb 0x55d9388b1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:03.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.230+0000 7ff339200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.230+0000 7ff339c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a8c70 0x55d9389b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.230+0000 7ff339200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50320/0 (socket says 172.21.15.2:50320) 2024-09-10T08:15:03.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.230+0000 7ff339200640 1 -- 172.21.15.2:0/3292772034 learned_addr learned my addr 172.21.15.2:0/3292772034 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:03.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.230+0000 7ff33a600640 1 --2- 172.21.15.2:0/3292772034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d9389b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.231+0000 7ff339200640 1 -- 172.21.15.2:0/3292772034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a8c70 msgr2=0x55d9389b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:03.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.231+0000 7ff339200640 1 --2- 172.21.15.2:0/3292772034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a8c70 0x55d9389b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.231+0000 7ff339200640 1 -- 172.21.15.2:0/3292772034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 msgr2=0x55d9389b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:03.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.231+0000 7ff339200640 1 --2- 172.21.15.2:0/3292772034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d9389b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.231+0000 7ff339200640 1 -- 172.21.15.2:0/3292772034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9386803c0 con 0x55d9389a9590 2024-09-10T08:15:03.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.231+0000 7ff339c00640 1 --2- 172.21.15.2:0/3292772034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a8c70 0x55d9389b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:03.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.231+0000 7ff339200640 1 --2- 172.21.15.2:0/3292772034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389b1260 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff32001c040 tx=0x7ff32000e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243dc9f81836f229 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:03.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.231+0000 7ff31b400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:03.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.231+0000 7ff31b400640 1 -- 172.21.15.2:0/3292772034 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320029020 con 0x55d9389a9590 2024-09-10T08:15:03.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.231+0000 7ff31b400640 1 -- 172.21.15.2:0/3292772034 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff320005040 con 0x55d9389a9590 2024-09-10T08:15:03.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.231+0000 7ff33f200740 1 -- 172.21.15.2:0/3292772034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff320019b10 con 0x55d9389a9590 2024-09-10T08:15:03.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.232+0000 7ff31b400640 1 -- 172.21.15.2:0/3292772034 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff320034050 con 0x55d9389a9590 2024-09-10T08:15:03.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.232+0000 7ff33f200740 10 client.17268 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:03.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.232+0000 7ff33f200740 1 -- 172.21.15.2:0/3292772034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9389c2a40 con 0x55d9389a9590 2024-09-10T08:15:03.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.232+0000 7ff31b400640 1 -- 172.21.15.2:0/3292772034 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff320019b10 con 0x55d9389a9590 2024-09-10T08:15:03.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.232+0000 7ff31b400640 1 -- 172.21.15.2:0/3292772034 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff32000ec50 con 0x55d9389a9590 2024-09-10T08:15:03.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.232+0000 7ff33f200740 10 client.17268 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:03.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.232+0000 7ff33f200740 10 client.17268 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:15:03.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.232+0000 7ff33f200740 20 client.17268 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:03.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.232+0000 7ff33f200740 1 --2- 172.21.15.2:0/3292772034 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d9389c3e50 0x55d9389c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.232+0000 7ff33f200740 4 client.17268 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:03.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.232+0000 7ff33f200740 1 -- 172.21.15.2:0/3292772034 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d93891c950 con 0x55d9389c3e50 2024-09-10T08:15:03.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.233+0000 7ff339c00640 1 --2- 172.21.15.2:0/3292772034 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d9389c3e50 0x55d9389c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.233+0000 7ff339c00640 1 --2- 172.21.15.2:0/3292772034 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d9389c3e50 0x55d9389c6240 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e89eabc26fe1a08 server_cookie=f6554505d6fbd48c in_seq=0 out_seq=0 2024-09-10T08:15:03.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.233+0000 7ff31b400640 10 client.17268 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:15:03.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.234+0000 7ff31b400640 1 -- 172.21.15.2:0/3292772034 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d93891c950 con 0x55d9389c3e50 2024-09-10T08:15:03.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.234+0000 7ff31b400640 10 client.17268 handle_command_reply: tid=0 2024-09-10T08:15:03.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.234+0000 7ff33f200740 1 client.17268 shutdown 2024-09-10T08:15:03.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.234+0000 7ff33f200740 1 -- 172.21.15.2:0/3292772034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 msgr2=0x55d9389b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:03.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.234+0000 7ff33f200740 1 --2- 172.21.15.2:0/3292772034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389b1260 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff32001c040 tx=0x7ff32000e5a0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.234+0000 7ff33f200740 1 -- 172.21.15.2:0/3292772034 shutdown_connections 2024-09-10T08:15:03.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.234+0000 7ff33f200740 1 --2- 172.21.15.2:0/3292772034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d9389b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.234+0000 7ff33f200740 1 --2- 172.21.15.2:0/3292772034 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d9389c3e50 0x55d9389c6240 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.234+0000 7ff33f200740 1 --2- 172.21.15.2:0/3292772034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389b1260 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.234+0000 7ff33f200740 1 --2- 172.21.15.2:0/3292772034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a8c70 0x55d9389b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.234+0000 7ff33f200740 1 -- 172.21.15.2:0/3292772034 >> 172.21.15.2:0/3292772034 conn(0x55d9389a5200 msgr2=0x55d9389aaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:03.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.234+0000 7ff31b400640 0 client.17268 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:15:03.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.235+0000 7ff33f200740 1 -- 172.21.15.2:0/3292772034 shutdown_connections 2024-09-10T08:15:03.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.235+0000 7ff33f200740 1 -- 172.21.15.2:0/3292772034 wait complete. 2024-09-10T08:15:03.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.235+0000 7ff33f200740 20 client.17268 trim_cache size 0 max 16384 2024-09-10T08:15:03.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.248+0000 7ff33f200740 1 Processor -- start 2024-09-10T08:15:03.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.249+0000 7ff33f200740 1 -- start start 2024-09-10T08:15:03.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.249+0000 7ff33f200740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389a8c70 0x55d9389cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.249+0000 7ff33f200740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.249+0000 7ff33f200740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389b3e50 0x55d9389a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.249+0000 7ff33f200740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff3341b3950 con 0x55d9389a9590 2024-09-10T08:15:03.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.249+0000 7ff33f200740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff3341b1220 con 0x55d9389a8c70 2024-09-10T08:15:03.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.249+0000 7ff33f200740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff3341af070 con 0x55d9389b3e50 2024-09-10T08:15:03.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.249+0000 7ff339200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.249+0000 7ff339200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50334/0 (socket says 172.21.15.2:50334) 2024-09-10T08:15:03.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.249+0000 7ff339200640 1 -- 172.21.15.2:0/855877331 learned_addr learned my addr 172.21.15.2:0/855877331 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:03.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.249+0000 7ff33a600640 1 --2- 172.21.15.2:0/855877331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389b3e50 0x55d9389a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.249+0000 7ff339c00640 1 --2- 172.21.15.2:0/855877331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389a8c70 0x55d9389cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.250+0000 7ff339200640 1 -- 172.21.15.2:0/855877331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389b3e50 msgr2=0x55d9389a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:03.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.250+0000 7ff339200640 1 --2- 172.21.15.2:0/855877331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389b3e50 0x55d9389a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.250+0000 7ff339200640 1 -- 172.21.15.2:0/855877331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389a8c70 msgr2=0x55d9389cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:03.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.250+0000 7ff339200640 1 --2- 172.21.15.2:0/855877331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389a8c70 0x55d9389cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.250+0000 7ff339200640 1 -- 172.21.15.2:0/855877331 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9389a0770 con 0x55d9389a9590 2024-09-10T08:15:03.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.250+0000 7ff339200640 1 --2- 172.21.15.2:0/855877331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389cbef0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff320004890 tx=0x7ff32000e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0949868f52daae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:03.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.250+0000 7ff326000640 1 -- 172.21.15.2:0/855877331 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320029040 con 0x55d9389a9590 2024-09-10T08:15:03.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.250+0000 7ff326000640 1 -- 172.21.15.2:0/855877331 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff320004040 con 0x55d9389a9590 2024-09-10T08:15:03.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.250+0000 7ff326000640 1 -- 172.21.15.2:0/855877331 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32001e040 con 0x55d9389a9590 2024-09-10T08:15:03.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.251+0000 7ff33f200740 1 -- 172.21.15.2:0/855877331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 msgr2=0x55d9389cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:03.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.251+0000 7ff33f200740 1 --2- 172.21.15.2:0/855877331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389cbef0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff320004890 tx=0x7ff32000e610 comp rx=0 tx=0).stop 2024-09-10T08:15:03.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.251+0000 7ff33f200740 1 -- 172.21.15.2:0/855877331 shutdown_connections 2024-09-10T08:15:03.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.251+0000 7ff33f200740 1 --2- 172.21.15.2:0/855877331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389b3e50 0x55d9389a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.251+0000 7ff33f200740 1 --2- 172.21.15.2:0/855877331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a9590 0x55d9389cbef0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.251+0000 7ff33f200740 1 --2- 172.21.15.2:0/855877331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389a8c70 0x55d9389cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.251+0000 7ff33f200740 1 -- 172.21.15.2:0/855877331 >> 172.21.15.2:0/855877331 conn(0x55d9389a5200 msgr2=0x55d9389abff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.251+0000 7ff33f200740 1 -- 172.21.15.2:0/855877331 shutdown_connections 2024-09-10T08:15:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.251+0000 7ff33f200740 1 -- 172.21.15.2:0/855877331 wait complete. 2024-09-10T08:15:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.251+0000 7ff33f200740 1 -- start start 2024-09-10T08:15:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.252+0000 7ff33f200740 1 Processor -- start 2024-09-10T08:15:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.252+0000 7ff33f200740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a8c70 0x55d9389b7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.252+0000 7ff33f200740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a9590 0x55d9389b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.252+0000 7ff33f200740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d938a2c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.252+0000 7ff33f200740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff3341af6e0 con 0x55d9389a8c70 2024-09-10T08:15:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.252+0000 7ff33f200740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff3341b0b50 con 0x55d9389b3e50 2024-09-10T08:15:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.252+0000 7ff33f200740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff3341afdb0 con 0x55d9389a9590 2024-09-10T08:15:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.252+0000 7ff33f200740 10 client.0 _ll_register_callbacks cb 0x7ff32001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:03.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.252+0000 7ff339c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a8c70 0x55d9389b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.252+0000 7ff339c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a8c70 0x55d9389b7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50348/0 (socket says 172.21.15.2:50348) 2024-09-10T08:15:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.252+0000 7ff339c00640 1 -- 172.21.15.2:0/1442364889 learned_addr learned my addr 172.21.15.2:0/1442364889 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.252+0000 7ff339200640 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a9590 0x55d9389b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.253+0000 7ff33a600640 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d938a2c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.253+0000 7ff339c00640 1 -- 172.21.15.2:0/1442364889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a9590 msgr2=0x55d9389b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.253+0000 7ff339c00640 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a9590 0x55d9389b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.253+0000 7ff339c00640 1 -- 172.21.15.2:0/1442364889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 msgr2=0x55d938a2c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.253+0000 7ff339c00640 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d938a2c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.253+0000 7ff339c00640 1 -- 172.21.15.2:0/1442364889 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9389abae0 con 0x55d9389a8c70 2024-09-10T08:15:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.253+0000 7ff33a600640 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d938a2c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.253+0000 7ff339200640 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a9590 0x55d9389b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.253+0000 7ff339c00640 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a8c70 0x55d9389b7310 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff328004650 tx=0x7ff3280045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8373933e23c39c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.253+0000 7ff31b400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:03.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.253+0000 7ff31b400640 1 -- 172.21.15.2:0/1442364889 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328049040 con 0x55d9389a8c70 2024-09-10T08:15:03.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.253+0000 7ff31b400640 1 -- 172.21.15.2:0/1442364889 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff3280da5e0 con 0x55d9389a8c70 2024-09-10T08:15:03.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.253+0000 7ff33f200740 1 -- 172.21.15.2:0/1442364889 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff320004040 con 0x55d9389a8c70 2024-09-10T08:15:03.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.254+0000 7ff31b400640 1 -- 172.21.15.2:0/1442364889 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff3280bbc10 con 0x55d9389a8c70 2024-09-10T08:15:03.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.254+0000 7ff33f200740 10 client.17274 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:03.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.254+0000 7ff33f200740 1 -- 172.21.15.2:0/1442364889 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9389a5e90 con 0x55d9389a8c70 2024-09-10T08:15:03.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.254+0000 7ff31b400640 1 -- 172.21.15.2:0/1442364889 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff320004040 con 0x55d9389a8c70 2024-09-10T08:15:03.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.254+0000 7ff31b400640 1 -- 172.21.15.2:0/1442364889 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff3280b84b0 con 0x55d9389a8c70 2024-09-10T08:15:03.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.254+0000 7ff33f200740 10 client.17274 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:03.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.254+0000 7ff33f200740 10 client.17274 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:15:03.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.254+0000 7ff33f200740 20 client.17274 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:03.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.254+0000 7ff33f200740 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d938a2e570 0x55d938a30960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:03.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.254+0000 7ff33f200740 4 client.17274 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:15:03.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.254+0000 7ff33f200740 1 -- 172.21.15.2:0/1442364889 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d938916e60 con 0x55d938a2e570 2024-09-10T08:15:03.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.255+0000 7ff339200640 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d938a2e570 0x55d938a30960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:03.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.255+0000 7ff339200640 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d938a2e570 0x55d938a30960 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d4974c10dfb2421 server_cookie=3e052f1494981dfd in_seq=0 out_seq=0 2024-09-10T08:15:03.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.255+0000 7ff31b400640 10 client.17274 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:15:03.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.255+0000 7ff31b400640 1 -- 172.21.15.2:0/1442364889 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d938916e60 con 0x55d938a2e570 2024-09-10T08:15:03.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.255+0000 7ff31b400640 10 client.17274 handle_command_reply: tid=0 2024-09-10T08:15:03.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.255+0000 7ff33f200740 1 client.17274 shutdown 2024-09-10T08:15:03.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.255+0000 7ff33f200740 1 -- 172.21.15.2:0/1442364889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a8c70 msgr2=0x55d9389b7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:03.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.255+0000 7ff33f200740 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a8c70 0x55d9389b7310 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff328004650 tx=0x7ff3280045b0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.256+0000 7ff33f200740 1 -- 172.21.15.2:0/1442364889 shutdown_connections 2024-09-10T08:15:03.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.256+0000 7ff33f200740 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9389b3e50 0x55d938a2c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.256+0000 7ff33f200740 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9389a9590 0x55d9389b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.256+0000 7ff33f200740 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d938a2e570 0x55d938a30960 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.256+0000 7ff33f200740 1 --2- 172.21.15.2:0/1442364889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d9389a8c70 0x55d9389b7310 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.256+0000 7ff33f200740 1 -- 172.21.15.2:0/1442364889 >> 172.21.15.2:0/1442364889 conn(0x55d9389a5200 msgr2=0x55d9389b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:03.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.256+0000 7ff31b400640 0 client.17274 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:15:03.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.256+0000 7ff33f200740 1 -- 172.21.15.2:0/1442364889 shutdown_connections 2024-09-10T08:15:03.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.256+0000 7ff33f200740 1 -- 172.21.15.2:0/1442364889 wait complete. 2024-09-10T08:15:03.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.256+0000 7ff33f200740 20 client.17274 trim_cache size 0 max 16384 2024-09-10T08:15:03.260 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:15:03.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.256+0000 7ff33c600640 1 -- 172.21.15.2:0/1895299810 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff31c0a2c00 msgr2=0x7ff31c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:03.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.256+0000 7ff33c600640 1 --2- 172.21.15.2:0/1895299810 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff31c0a2c00 0x7ff31c0a50c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff32c005fd0 tx=0x7ff32c009600 comp rx=0 tx=0).stop 2024-09-10T08:15:03.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.257+0000 7ff33c600640 1 -- 172.21.15.2:0/1895299810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff334103ff0 msgr2=0x7ff33419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:03.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.257+0000 7ff33c600640 1 --2- 172.21.15.2:0/1895299810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff334103ff0 0x7ff33419a5e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff328006010 tx=0x7ff3280042f0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.257+0000 7ff33c600640 1 -- 172.21.15.2:0/1895299810 shutdown_connections 2024-09-10T08:15:03.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.257+0000 7ff33c600640 1 --2- 172.21.15.2:0/1895299810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff334109120 0x7ff33419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.257+0000 7ff33c600640 1 --2- 172.21.15.2:0/1895299810 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff31c0a2c00 0x7ff31c0a50c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.257+0000 7ff33c600640 1 --2- 172.21.15.2:0/1895299810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3341049a0 0x7ff33419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.257+0000 7ff33c600640 1 --2- 172.21.15.2:0/1895299810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff334103ff0 0x7ff33419a5e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:03.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.257+0000 7ff33c600640 1 -- 172.21.15.2:0/1895299810 >> 172.21.15.2:0/1895299810 conn(0x7ff3340ffa90 msgr2=0x7ff334100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:03.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.257+0000 7ff33c600640 1 -- 172.21.15.2:0/1895299810 shutdown_connections 2024-09-10T08:15:03.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:03.257+0000 7ff33c600640 1 -- 172.21.15.2:0/1895299810 wait complete. 2024-09-10T08:15:03.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:02 smithi155 ceph-mon[35405]: pgmap v594: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-09-10T08:15:03.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:02 smithi107 ceph-mon[35048]: pgmap v594: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-09-10T08:15:04.274 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:15:04.653 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:05.098 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:04 smithi002 ceph-mon[30563]: pgmap v595: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-09-10T08:15:05.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.214+0000 7f062d400640 1 -- 172.21.15.2:0/3357047253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0628101de0 msgr2=0x7f06281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.214+0000 7f062d400640 1 --2- 172.21.15.2:0/3357047253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0628101de0 0x7f06281021c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0618009900 tx=0x7f061802a410 comp rx=0 tx=0).stop 2024-09-10T08:15:05.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.214+0000 7f062d400640 1 -- 172.21.15.2:0/3357047253 shutdown_connections 2024-09-10T08:15:05.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.214+0000 7f062d400640 1 --2- 172.21.15.2:0/3357047253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0628106f10 0x7f062810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.214+0000 7f062d400640 1 --2- 172.21.15.2:0/3357047253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0628102790 0x7f06281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.214+0000 7f062d400640 1 --2- 172.21.15.2:0/3357047253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0628101de0 0x7f06281021c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.214+0000 7f062d400640 1 -- 172.21.15.2:0/3357047253 >> 172.21.15.2:0/3357047253 conn(0x7f06280fd8c0 msgr2=0x7f06280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:05.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.214+0000 7f062d400640 1 -- 172.21.15.2:0/3357047253 shutdown_connections 2024-09-10T08:15:05.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.214+0000 7f062d400640 1 -- 172.21.15.2:0/3357047253 wait complete. 2024-09-10T08:15:05.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.215+0000 7f062d400640 1 Processor -- start 2024-09-10T08:15:05.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.215+0000 7f062d400640 1 -- start start 2024-09-10T08:15:05.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.215+0000 7f062d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0628101de0 0x7f062819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f062d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0628102790 0x7f062819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f062d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0628106f10 0x7f062819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f062d400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0628111f80 con 0x7f0628101de0 2024-09-10T08:15:05.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f062d400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0628111e00 con 0x7f0628106f10 2024-09-10T08:15:05.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f062d400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0628112100 con 0x7f0628102790 2024-09-10T08:15:05.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f0626800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0628101de0 0x7f062819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f0626800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0628101de0 0x7f062819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50362/0 (socket says 172.21.15.2:50362) 2024-09-10T08:15:05.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f0626800640 1 -- 172.21.15.2:0/3684664430 learned_addr learned my addr 172.21.15.2:0/3684664430 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:05.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f061fe00640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0628102790 0x7f062819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f0627200640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0628106f10 0x7f062819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f0626800640 1 -- 172.21.15.2:0/3684664430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0628102790 msgr2=0x7f062819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f0626800640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0628102790 0x7f062819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f0626800640 1 -- 172.21.15.2:0/3684664430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0628106f10 msgr2=0x7f062819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f0626800640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0628106f10 0x7f062819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.216+0000 7f0626800640 1 -- 172.21.15.2:0/3684664430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062819f670 con 0x7f0628101de0 2024-09-10T08:15:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.217+0000 7f0626800640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0628101de0 0x7f062819a6c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0618004770 tx=0x7f0618003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.217+0000 7f061f400640 1 -- 172.21.15.2:0/3684664430 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061803f070 con 0x7f0628101de0 2024-09-10T08:15:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.217+0000 7f061f400640 1 -- 172.21.15.2:0/3684664430 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0618031ca0 con 0x7f0628101de0 2024-09-10T08:15:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.217+0000 7f062d400640 1 -- 172.21.15.2:0/3684664430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062819f900 con 0x7f0628101de0 2024-09-10T08:15:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.217+0000 7f061f400640 1 -- 172.21.15.2:0/3684664430 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618035030 con 0x7f0628101de0 2024-09-10T08:15:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.217+0000 7f062d400640 1 -- 172.21.15.2:0/3684664430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06281a7240 con 0x7f0628101de0 2024-09-10T08:15:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.221+0000 7f061f400640 1 -- 172.21.15.2:0/3684664430 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0618042070 con 0x7f0628101de0 2024-09-10T08:15:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.221+0000 7f062ffe8740 1 Processor -- start 2024-09-10T08:15:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.221+0000 7f062ffe8740 1 -- start start 2024-09-10T08:15:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.221+0000 7f062ffe8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224febc70 0x560224fec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.221+0000 7f062ffe8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224fec590 0x560224ff6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.221+0000 7f062ffe8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224ff6e50 0x560224ff9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.221+0000 7f062ffe8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560224f523e0 con 0x560224ff6e50 2024-09-10T08:15:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.221+0000 7f062ffe8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560224ef9d80 con 0x560224febc70 2024-09-10T08:15:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.221+0000 7f062ffe8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560224f4e3a0 con 0x560224fec590 2024-09-10T08:15:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.222+0000 7f0627200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224febc70 0x560224fec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.222+0000 7f061fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224ff6e50 0x560224ff9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.222+0000 7f0627200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224febc70 0x560224fec050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35140/0 (socket says 172.21.15.2:35140) 2024-09-10T08:15:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.222+0000 7f0627200640 1 -- 172.21.15.2:0/2401894533 learned_addr learned my addr 172.21.15.2:0/2401894533 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.222+0000 7f0626800640 1 --2- 172.21.15.2:0/2401894533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224fec590 0x560224ff6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.222+0000 7f0627200640 1 -- 172.21.15.2:0/2401894533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224fec590 msgr2=0x560224ff6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.222+0000 7f0627200640 1 --2- 172.21.15.2:0/2401894533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224fec590 0x560224ff6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.222+0000 7f0627200640 1 -- 172.21.15.2:0/2401894533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224ff6e50 msgr2=0x560224ff9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.222+0000 7f0627200640 1 --2- 172.21.15.2:0/2401894533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224ff6e50 0x560224ff9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.222+0000 7f0627200640 1 -- 172.21.15.2:0/2401894533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560224f18ae0 con 0x560224febc70 2024-09-10T08:15:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.223+0000 7f0627200640 1 --2- 172.21.15.2:0/2401894533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224febc70 0x560224fec050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0620009f50 tx=0x7f06200118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eddcd3ac4229f9d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.223+0000 7f061f400640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f060c0a2c00 0x7f060c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.223+0000 7f0617400640 1 -- 172.21.15.2:0/2401894533 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620004d20 con 0x560224febc70 2024-09-10T08:15:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.223+0000 7f0617400640 1 -- 172.21.15.2:0/2401894533 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0620004050 con 0x560224febc70 2024-09-10T08:15:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.223+0000 7f0617400640 1 -- 172.21.15.2:0/2401894533 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620005280 con 0x560224febc70 2024-09-10T08:15:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.223+0000 7f061f400640 1 -- 172.21.15.2:0/3684664430 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f06180ef9a0 con 0x7f0628101de0 2024-09-10T08:15:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.223+0000 7f0626800640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f060c0a2c00 0x7f060c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.224+0000 7f062ffe8740 1 -- 172.21.15.2:0/2401894533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224febc70 msgr2=0x560224fec050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.224+0000 7f062ffe8740 1 --2- 172.21.15.2:0/2401894533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224febc70 0x560224fec050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0620009f50 tx=0x7f06200118d0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.224+0000 7f062ffe8740 1 -- 172.21.15.2:0/2401894533 shutdown_connections 2024-09-10T08:15:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.224+0000 7f062ffe8740 1 --2- 172.21.15.2:0/2401894533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224ff6e50 0x560224ff9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.224+0000 7f062ffe8740 1 --2- 172.21.15.2:0/2401894533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224fec590 0x560224ff6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.224+0000 7f062ffe8740 1 --2- 172.21.15.2:0/2401894533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224febc70 0x560224fec050 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.224+0000 7f062ffe8740 1 -- 172.21.15.2:0/2401894533 >> 172.21.15.2:0/2401894533 conn(0x560224fe8200 msgr2=0x560224fe85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.224+0000 7f0626800640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f060c0a2c00 0x7f060c0a50c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f06180b8380 tx=0x7f0618009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.224+0000 7f062ffe8740 1 -- 172.21.15.2:0/2401894533 shutdown_connections 2024-09-10T08:15:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.224+0000 7f062ffe8740 1 -- 172.21.15.2:0/2401894533 wait complete. 2024-09-10T08:15:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.225+0000 7f062ffe8740 1 -- start start 2024-09-10T08:15:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.225+0000 7f062ffe8740 1 Processor -- start 2024-09-10T08:15:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.226+0000 7f062ffe8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224febc70 0x560224ff3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.226+0000 7f062ffe8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224fec590 0x560224ff4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.226+0000 7f062ffe8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224ff6e50 0x560224ff47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.226+0000 7f062ffe8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560224ff9a20 con 0x560224febc70 2024-09-10T08:15:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.226+0000 7f062ffe8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560224f0d0f0 con 0x560224fec590 2024-09-10T08:15:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.226+0000 7f062ffe8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560224ff9ba0 con 0x560224ff6e50 2024-09-10T08:15:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.226+0000 7f062ffe8740 10 client.0 _ll_register_callbacks cb 0x560224ef4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.226+0000 7f0627200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224febc70 0x560224ff3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.226+0000 7f061fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224fec590 0x560224ff4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.226+0000 7f0627200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224febc70 0x560224ff3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50390/0 (socket says 172.21.15.2:50390) 2024-09-10T08:15:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.226+0000 7f0627200640 1 -- 172.21.15.2:0/326147016 learned_addr learned my addr 172.21.15.2:0/326147016 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.226+0000 7f0627200640 1 --2- 172.21.15.2:0/326147016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224ff6e50 0x560224ff47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.227+0000 7f061fe00640 1 -- 172.21.15.2:0/326147016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224ff6e50 msgr2=0x560224ff47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.227+0000 7f061fe00640 1 --2- 172.21.15.2:0/326147016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224ff6e50 0x560224ff47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.227+0000 7f061fe00640 1 -- 172.21.15.2:0/326147016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224febc70 msgr2=0x560224ff3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.227+0000 7f061fe00640 1 --2- 172.21.15.2:0/326147016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224febc70 0x560224ff3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.227+0000 7f061fe00640 1 -- 172.21.15.2:0/326147016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560224cc33c0 con 0x560224fec590 2024-09-10T08:15:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.227+0000 7f061fe00640 1 --2- 172.21.15.2:0/326147016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224fec590 0x560224ff4270 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f06280fddc0 tx=0x7f0610012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7ac72b2d68e90ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.227+0000 7f060be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:15:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.227+0000 7f060be00640 1 -- 172.21.15.2:0/326147016 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610004050 con 0x560224fec590 2024-09-10T08:15:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.227+0000 7f062ffe8740 1 -- 172.21.15.2:0/326147016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0620004050 con 0x560224fec590 2024-09-10T08:15:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.227+0000 7f060be00640 1 -- 172.21.15.2:0/326147016 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f06100041f0 con 0x560224fec590 2024-09-10T08:15:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.228+0000 7f060be00640 1 -- 172.21.15.2:0/326147016 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0610025050 con 0x560224fec590 2024-09-10T08:15:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.228+0000 7f062ffe8740 10 client.27323 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.228+0000 7f062ffe8740 1 -- 172.21.15.2:0/326147016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560225005b10 con 0x560224fec590 2024-09-10T08:15:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.228+0000 7f060be00640 1 -- 172.21.15.2:0/326147016 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0620004050 con 0x560224fec590 2024-09-10T08:15:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.228+0000 7f060be00640 1 -- 172.21.15.2:0/326147016 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f06100039d0 con 0x560224fec590 2024-09-10T08:15:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.228+0000 7f062ffe8740 10 client.27323 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.228+0000 7f062ffe8740 10 client.27323 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:15:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.228+0000 7f062ffe8740 20 client.27323 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.228+0000 7f062ffe8740 1 --2- 172.21.15.2:0/326147016 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560225006f20 0x560225009380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.228+0000 7f062ffe8740 4 client.27323 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.228+0000 7f062ffe8740 1 -- 172.21.15.2:0/326147016 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560224f5f950 con 0x560225006f20 2024-09-10T08:15:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.229+0000 7f0627200640 1 --2- 172.21.15.2:0/326147016 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560225006f20 0x560225009380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.229+0000 7f0627200640 1 --2- 172.21.15.2:0/326147016 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560225006f20 0x560225009380 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfb9c538fba1ff54 server_cookie=421dfc9af4ef5e93 in_seq=0 out_seq=0 2024-09-10T08:15:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.229+0000 7f060be00640 10 client.27323 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:15:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.230+0000 7f060be00640 1 -- 172.21.15.2:0/326147016 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560224f5f950 con 0x560225006f20 2024-09-10T08:15:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.230+0000 7f060be00640 10 client.27323 handle_command_reply: tid=0 2024-09-10T08:15:05.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.230+0000 7f062ffe8740 1 client.27323 shutdown 2024-09-10T08:15:05.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.230+0000 7f062ffe8740 1 -- 172.21.15.2:0/326147016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224fec590 msgr2=0x560224ff4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.230+0000 7f062ffe8740 1 --2- 172.21.15.2:0/326147016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224fec590 0x560224ff4270 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f06280fddc0 tx=0x7f0610012b60 comp rx=0 tx=0).stop 2024-09-10T08:15:05.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.230+0000 7f062ffe8740 1 -- 172.21.15.2:0/326147016 shutdown_connections 2024-09-10T08:15:05.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.230+0000 7f062ffe8740 1 --2- 172.21.15.2:0/326147016 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560225006f20 0x560225009380 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.231+0000 7f062ffe8740 1 --2- 172.21.15.2:0/326147016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224ff6e50 0x560224ff47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.231+0000 7f062ffe8740 1 --2- 172.21.15.2:0/326147016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224fec590 0x560224ff4270 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.231+0000 7f062ffe8740 1 --2- 172.21.15.2:0/326147016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224febc70 0x560224ff3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.231+0000 7f060be00640 0 client.27323 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:15:05.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.231+0000 7f062ffe8740 1 -- 172.21.15.2:0/326147016 >> 172.21.15.2:0/326147016 conn(0x560224fe8200 msgr2=0x560224fe7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:05.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.231+0000 7f062ffe8740 1 -- 172.21.15.2:0/326147016 shutdown_connections 2024-09-10T08:15:05.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.231+0000 7f062ffe8740 1 -- 172.21.15.2:0/326147016 wait complete. 2024-09-10T08:15:05.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.231+0000 7f062ffe8740 20 client.27323 trim_cache size 0 max 16384 2024-09-10T08:15:05.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.245+0000 7f062ffe8740 1 Processor -- start 2024-09-10T08:15:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.245+0000 7f062ffe8740 1 -- start start 2024-09-10T08:15:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.245+0000 7f062ffe8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224febc70 0x560224fee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.245+0000 7f062ffe8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224fec590 0x560224ff5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.245+0000 7f062ffe8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224ff6e50 0x56022500ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.245+0000 7f062ffe8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f06281b3950 con 0x560224fec590 2024-09-10T08:15:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.246+0000 7f062ffe8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06281b1220 con 0x560224ff6e50 2024-09-10T08:15:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.246+0000 7f062ffe8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f06281af070 con 0x560224febc70 2024-09-10T08:15:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.246+0000 7f061fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224fec590 0x560224ff5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.246+0000 7f061fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224fec590 0x560224ff5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50406/0 (socket says 172.21.15.2:50406) 2024-09-10T08:15:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.246+0000 7f061fe00640 1 -- 172.21.15.2:0/4100915136 learned_addr learned my addr 172.21.15.2:0/4100915136 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.246+0000 7f0627200640 1 --2- 172.21.15.2:0/4100915136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224febc70 0x560224fee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.246+0000 7f0627200640 1 --2- 172.21.15.2:0/4100915136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224ff6e50 0x56022500ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.246+0000 7f061fe00640 1 -- 172.21.15.2:0/4100915136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224febc70 msgr2=0x560224fee6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.246+0000 7f061fe00640 1 --2- 172.21.15.2:0/4100915136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224febc70 0x560224fee6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.246+0000 7f061fe00640 1 -- 172.21.15.2:0/4100915136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224ff6e50 msgr2=0x56022500ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.246+0000 7f061fe00640 1 --2- 172.21.15.2:0/4100915136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224ff6e50 0x56022500ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.246+0000 7f061fe00640 1 -- 172.21.15.2:0/4100915136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560224fe3770 con 0x560224fec590 2024-09-10T08:15:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.246+0000 7f0627200640 1 --2- 172.21.15.2:0/4100915136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224febc70 0x560224fee6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.247+0000 7f061fe00640 1 --2- 172.21.15.2:0/4100915136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224fec590 0x560224ff5ca0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0610003da0 tx=0x7f0610010690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf8b9a5c6e60bc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.247+0000 7f0616a00640 1 -- 172.21.15.2:0/4100915136 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061000f920 con 0x560224fec590 2024-09-10T08:15:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.247+0000 7f0616a00640 1 -- 172.21.15.2:0/4100915136 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061002e070 con 0x560224fec590 2024-09-10T08:15:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.247+0000 7f0616a00640 1 -- 172.21.15.2:0/4100915136 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06100055b0 con 0x560224fec590 2024-09-10T08:15:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.248+0000 7f062ffe8740 1 -- 172.21.15.2:0/4100915136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224fec590 msgr2=0x560224ff5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.248+0000 7f062ffe8740 1 --2- 172.21.15.2:0/4100915136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224fec590 0x560224ff5ca0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0610003da0 tx=0x7f0610010690 comp rx=0 tx=0).stop 2024-09-10T08:15:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.248+0000 7f062ffe8740 1 -- 172.21.15.2:0/4100915136 shutdown_connections 2024-09-10T08:15:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.248+0000 7f062ffe8740 1 --2- 172.21.15.2:0/4100915136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224ff6e50 0x56022500ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.248+0000 7f062ffe8740 1 --2- 172.21.15.2:0/4100915136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224fec590 0x560224ff5ca0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.248+0000 7f062ffe8740 1 --2- 172.21.15.2:0/4100915136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224febc70 0x560224fee6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.248+0000 7f062ffe8740 1 -- 172.21.15.2:0/4100915136 >> 172.21.15.2:0/4100915136 conn(0x560224fe8200 msgr2=0x560224ff5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.248+0000 7f062ffe8740 1 -- 172.21.15.2:0/4100915136 shutdown_connections 2024-09-10T08:15:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.248+0000 7f062ffe8740 1 -- 172.21.15.2:0/4100915136 wait complete. 2024-09-10T08:15:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.249+0000 7f062ffe8740 1 -- start start 2024-09-10T08:15:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.249+0000 7f062ffe8740 1 Processor -- start 2024-09-10T08:15:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.249+0000 7f062ffe8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224febc70 0x560224fff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.249+0000 7f062ffe8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224fec590 0x560224ffa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.249+0000 7f062ffe8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224ff6e50 0x560224ffac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.249+0000 7f062ffe8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f06281af6e0 con 0x560224febc70 2024-09-10T08:15:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.249+0000 7f062ffe8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06281b0b50 con 0x560224ff6e50 2024-09-10T08:15:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.249+0000 7f062ffe8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f06281afdb0 con 0x560224fec590 2024-09-10T08:15:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.249+0000 7f062ffe8740 10 client.0 _ll_register_callbacks cb 0x7f06100097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.250+0000 7f0627200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224febc70 0x560224fff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.250+0000 7f0627200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224febc70 0x560224fff390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50412/0 (socket says 172.21.15.2:50412) 2024-09-10T08:15:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.250+0000 7f0627200640 1 -- 172.21.15.2:0/3359748132 learned_addr learned my addr 172.21.15.2:0/3359748132 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.250+0000 7f061fe00640 1 --2- 172.21.15.2:0/3359748132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224fec590 0x560224ffa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.250+0000 7f0627200640 1 --2- 172.21.15.2:0/3359748132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224ff6e50 0x560224ffac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.250+0000 7f0627200640 1 -- 172.21.15.2:0/3359748132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224fec590 msgr2=0x560224ffa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.250+0000 7f0627200640 1 --2- 172.21.15.2:0/3359748132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224fec590 0x560224ffa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.250+0000 7f0627200640 1 -- 172.21.15.2:0/3359748132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224ff6e50 msgr2=0x560224ffac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.250+0000 7f0627200640 1 --2- 172.21.15.2:0/3359748132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224ff6e50 0x560224ffac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.250+0000 7f0627200640 1 -- 172.21.15.2:0/3359748132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560224ffb240 con 0x560224febc70 2024-09-10T08:15:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.251+0000 7f0627200640 1 --2- 172.21.15.2:0/3359748132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224febc70 0x560224fff390 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0620006010 tx=0x7f0610009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0aa947fbb629c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.251+0000 7f060be00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:05.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.251+0000 7f060be00640 1 -- 172.21.15.2:0/3359748132 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620003f00 con 0x560224febc70 2024-09-10T08:15:05.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.251+0000 7f060be00640 1 -- 172.21.15.2:0/3359748132 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0620027440 con 0x560224febc70 2024-09-10T08:15:05.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.251+0000 7f062ffe8740 1 -- 172.21.15.2:0/3359748132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f061002e070 con 0x560224febc70 2024-09-10T08:15:05.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.251+0000 7f060be00640 1 -- 172.21.15.2:0/3359748132 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f06200040a0 con 0x560224febc70 2024-09-10T08:15:05.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.251+0000 7f062ffe8740 10 client.17292 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:05.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.251+0000 7f062ffe8740 1 -- 172.21.15.2:0/3359748132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560225071ab0 con 0x560224febc70 2024-09-10T08:15:05.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.252+0000 7f060be00640 1 -- 172.21.15.2:0/3359748132 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061002e070 con 0x560224febc70 2024-09-10T08:15:05.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.252+0000 7f060be00640 1 -- 172.21.15.2:0/3359748132 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f062000f7b0 con 0x560224febc70 2024-09-10T08:15:05.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.252+0000 7f062ffe8740 10 client.17292 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:05.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.252+0000 7f062ffe8740 10 client.17292 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:15:05.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.252+0000 7f062ffe8740 20 client.17292 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:05.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.252+0000 7f062ffe8740 1 --2- 172.21.15.2:0/3359748132 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560225072040 0x560225074430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.252+0000 7f062ffe8740 4 client.17292 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:15:05.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.252+0000 7f062ffe8740 1 -- 172.21.15.2:0/3359748132 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560224f59e60 con 0x560225072040 2024-09-10T08:15:05.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.252+0000 7f061fe00640 1 --2- 172.21.15.2:0/3359748132 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560225072040 0x560225074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f061fe00640 1 --2- 172.21.15.2:0/3359748132 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560225072040 0x560225074430 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fcc722e6786fdf8d server_cookie=26f92444c1943493 in_seq=0 out_seq=0 2024-09-10T08:15:05.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f060be00640 10 client.17292 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:15:05.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f060be00640 1 -- 172.21.15.2:0/3359748132 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560224f59e60 con 0x560225072040 2024-09-10T08:15:05.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f060be00640 10 client.17292 handle_command_reply: tid=0 2024-09-10T08:15:05.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f062ffe8740 1 client.17292 shutdown 2024-09-10T08:15:05.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f062ffe8740 1 -- 172.21.15.2:0/3359748132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224febc70 msgr2=0x560224fff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f062ffe8740 1 --2- 172.21.15.2:0/3359748132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224febc70 0x560224fff390 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0620006010 tx=0x7f0610009ca0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f062ffe8740 1 -- 172.21.15.2:0/3359748132 shutdown_connections 2024-09-10T08:15:05.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f062ffe8740 1 --2- 172.21.15.2:0/3359748132 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560225072040 0x560225074430 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f062ffe8740 1 --2- 172.21.15.2:0/3359748132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560224ff6e50 0x560224ffac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f062ffe8740 1 --2- 172.21.15.2:0/3359748132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560224fec590 0x560224ffa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f062ffe8740 1 --2- 172.21.15.2:0/3359748132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560224febc70 0x560224fff390 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f062ffe8740 1 -- 172.21.15.2:0/3359748132 >> 172.21.15.2:0/3359748132 conn(0x560224fe8200 msgr2=0x560224ff4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:05.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f060be00640 0 client.17292 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:15:05.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.253+0000 7f062ffe8740 1 -- 172.21.15.2:0/3359748132 shutdown_connections 2024-09-10T08:15:05.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.254+0000 7f062ffe8740 1 -- 172.21.15.2:0/3359748132 wait complete. 2024-09-10T08:15:05.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.254+0000 7f062ffe8740 20 client.17292 trim_cache size 0 max 16384 2024-09-10T08:15:05.264 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:15:05.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.254+0000 7f062d400640 1 -- 172.21.15.2:0/3684664430 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f060c0a2c00 msgr2=0x7f060c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.254+0000 7f062d400640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f060c0a2c00 0x7f060c0a50c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f06180b8380 tx=0x7f0618009680 comp rx=0 tx=0).stop 2024-09-10T08:15:05.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.254+0000 7f062d400640 1 -- 172.21.15.2:0/3684664430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0628101de0 msgr2=0x7f062819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.254+0000 7f062d400640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0628101de0 0x7f062819a6c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0618004770 tx=0x7f0618003c10 comp rx=0 tx=0).stop 2024-09-10T08:15:05.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.255+0000 7f062d400640 1 -- 172.21.15.2:0/3684664430 shutdown_connections 2024-09-10T08:15:05.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.255+0000 7f062d400640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0628106f10 0x7f062819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.255+0000 7f062d400640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f060c0a2c00 0x7f060c0a50c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.255+0000 7f062d400640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0628102790 0x7f062819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.255+0000 7f062d400640 1 --2- 172.21.15.2:0/3684664430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0628101de0 0x7f062819a6c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.255+0000 7f062d400640 1 -- 172.21.15.2:0/3684664430 >> 172.21.15.2:0/3684664430 conn(0x7f06280fd8c0 msgr2=0x7f06280febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:05.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.255+0000 7f062d400640 1 -- 172.21.15.2:0/3684664430 shutdown_connections 2024-09-10T08:15:05.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:05.255+0000 7f062d400640 1 -- 172.21.15.2:0/3684664430 wait complete. 2024-09-10T08:15:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.305+0000 7f7d6fe00640 1 -- 172.21.15.2:0/137370820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d70104cf0 msgr2=0x7f7d701070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.305+0000 7f7d6fe00640 1 --2- 172.21.15.2:0/137370820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d70104cf0 0x7f7d701070e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009900 tx=0x7f7d5c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.305+0000 7f7d6fe00640 1 -- 172.21.15.2:0/137370820 shutdown_connections 2024-09-10T08:15:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.305+0000 7f7d6fe00640 1 --2- 172.21.15.2:0/137370820 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d70107620 0x7f7d70109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.305+0000 7f7d6fe00640 1 --2- 172.21.15.2:0/137370820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d70104cf0 0x7f7d701070e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.305+0000 7f7d6fe00640 1 --2- 172.21.15.2:0/137370820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d701023c0 0x7f7d701047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.305+0000 7f7d6fe00640 1 -- 172.21.15.2:0/137370820 >> 172.21.15.2:0/137370820 conn(0x7f7d700fbfa0 msgr2=0x7f7d700fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.305+0000 7f7d6fe00640 1 -- 172.21.15.2:0/137370820 shutdown_connections 2024-09-10T08:15:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.305+0000 7f7d6fe00640 1 -- 172.21.15.2:0/137370820 wait complete. 2024-09-10T08:15:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.306+0000 7f7d6fe00640 1 Processor -- start 2024-09-10T08:15:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.306+0000 7f7d6fe00640 1 -- start start 2024-09-10T08:15:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.306+0000 7f7d6fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d701023c0 0x7f7d70070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.306+0000 7f7d6fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d70104cf0 0x7f7d70071160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.306+0000 7f7d6fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d70107620 0x7f7d701a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.306+0000 7f7d6fe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7010d640 con 0x7f7d70104cf0 2024-09-10T08:15:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.306+0000 7f7d6fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7010d4c0 con 0x7f7d70107620 2024-09-10T08:15:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.306+0000 7f7d6fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7010d7c0 con 0x7f7d701023c0 2024-09-10T08:15:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.307+0000 7f7d6f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d70107620 0x7f7d701a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.307+0000 7f7d6e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d70104cf0 0x7f7d70071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.307+0000 7f7d6f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d70107620 0x7f7d701a7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35182/0 (socket says 172.21.15.2:35182) 2024-09-10T08:15:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.307+0000 7f7d6ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d701023c0 0x7f7d70070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.307+0000 7f7d6f400640 1 -- 172.21.15.2:0/4176366056 learned_addr learned my addr 172.21.15.2:0/4176366056 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.307+0000 7f7d6f400640 1 -- 172.21.15.2:0/4176366056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d701023c0 msgr2=0x7f7d70070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.307+0000 7f7d6f400640 1 --2- 172.21.15.2:0/4176366056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d701023c0 0x7f7d70070c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.307+0000 7f7d6f400640 1 -- 172.21.15.2:0/4176366056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d70104cf0 msgr2=0x7f7d70071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.307+0000 7f7d6f400640 1 --2- 172.21.15.2:0/4176366056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d70104cf0 0x7f7d70071160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.307+0000 7f7d6f400640 1 -- 172.21.15.2:0/4176366056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d701a8290 con 0x7f7d70107620 2024-09-10T08:15:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.307+0000 7f7d6f400640 1 --2- 172.21.15.2:0/4176366056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d70107620 0x7f7d701a7d50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800eb40 tx=0x7f7d6800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.308+0000 7f7d67400640 1 -- 172.21.15.2:0/4176366056 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68004b20 con 0x7f7d70107620 2024-09-10T08:15:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.308+0000 7f7d67400640 1 -- 172.21.15.2:0/4176366056 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d68004cc0 con 0x7f7d70107620 2024-09-10T08:15:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.308+0000 7f7d67400640 1 -- 172.21.15.2:0/4176366056 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68010850 con 0x7f7d70107620 2024-09-10T08:15:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.308+0000 7f7d6fe00640 1 -- 172.21.15.2:0/4176366056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d701a8460 con 0x7f7d70107620 2024-09-10T08:15:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.308+0000 7f7d6fe00640 1 -- 172.21.15.2:0/4176366056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d701a8930 con 0x7f7d70107620 2024-09-10T08:15:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.311+0000 7f7d67400640 1 -- 172.21.15.2:0/4176366056 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7d68005020 con 0x7f7d70107620 2024-09-10T08:15:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.311+0000 7f7d7841b740 1 Processor -- start 2024-09-10T08:15:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.311+0000 7f7d7841b740 1 -- start start 2024-09-10T08:15:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.311+0000 7f7d7841b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be13660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d7841b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be13ba0 0x56335be22f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d7841b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be234a0 0x56335be258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d7841b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56335bd7a100 con 0x56335be13260 2024-09-10T08:15:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d7841b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56335bd221b0 con 0x56335be13ba0 2024-09-10T08:15:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d7841b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56335bd760c0 con 0x56335be234a0 2024-09-10T08:15:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d6e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be13660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d6e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be13660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50436/0 (socket says 172.21.15.2:50436) 2024-09-10T08:15:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d6ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be13ba0 0x56335be22f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d6e000640 1 -- 172.21.15.2:0/4013751535 learned_addr learned my addr 172.21.15.2:0/4013751535 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d6e000640 1 --2- 172.21.15.2:0/4013751535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be234a0 0x56335be258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d6ea00640 1 -- 172.21.15.2:0/4013751535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be234a0 msgr2=0x56335be258f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d6ea00640 1 --2- 172.21.15.2:0/4013751535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be234a0 0x56335be258f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d6ea00640 1 -- 172.21.15.2:0/4013751535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 msgr2=0x56335be13660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d6ea00640 1 --2- 172.21.15.2:0/4013751535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be13660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d6ea00640 1 -- 172.21.15.2:0/4013751535 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335ba8d210 con 0x56335be13ba0 2024-09-10T08:15:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.312+0000 7f7d6e000640 1 --2- 172.21.15.2:0/4013751535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be234a0 0x56335be258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.313+0000 7f7d6ea00640 1 --2- 172.21.15.2:0/4013751535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be13ba0 0x56335be22f60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7d60004630 tx=0x7f7d6000bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4798397a976b9cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.313+0000 7f7d67400640 1 --2- 172.21.15.2:0/4176366056 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d540a2c00 0x7f7d540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.313+0000 7f7d5b400640 1 -- 172.21.15.2:0/4013751535 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60003bd0 con 0x56335be13ba0 2024-09-10T08:15:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.313+0000 7f7d5b400640 1 -- 172.21.15.2:0/4013751535 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d60015070 con 0x56335be13ba0 2024-09-10T08:15:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.313+0000 7f7d5b400640 1 -- 172.21.15.2:0/4013751535 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60004d30 con 0x56335be13ba0 2024-09-10T08:15:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.313+0000 7f7d67400640 1 -- 172.21.15.2:0/4176366056 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7d680cfbc0 con 0x7f7d70107620 2024-09-10T08:15:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.313+0000 7f7d6e000640 1 --2- 172.21.15.2:0/4176366056 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d540a2c00 0x7f7d540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.313+0000 7f7d7841b740 1 -- 172.21.15.2:0/4013751535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be13ba0 msgr2=0x56335be22f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.313+0000 7f7d7841b740 1 --2- 172.21.15.2:0/4013751535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be13ba0 0x56335be22f60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7d60004630 tx=0x7f7d6000bb20 comp rx=0 tx=0).stop 2024-09-10T08:15:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.314+0000 7f7d7841b740 1 -- 172.21.15.2:0/4013751535 shutdown_connections 2024-09-10T08:15:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.314+0000 7f7d7841b740 1 --2- 172.21.15.2:0/4013751535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be234a0 0x56335be258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.314+0000 7f7d7841b740 1 --2- 172.21.15.2:0/4013751535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be13ba0 0x56335be22f60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.314+0000 7f7d7841b740 1 --2- 172.21.15.2:0/4013751535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be13660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.314+0000 7f7d7841b740 1 -- 172.21.15.2:0/4013751535 >> 172.21.15.2:0/4013751535 conn(0x56335be10850 msgr2=0x56335be10c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.314+0000 7f7d7841b740 1 -- 172.21.15.2:0/4013751535 shutdown_connections 2024-09-10T08:15:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.314+0000 7f7d6e000640 1 --2- 172.21.15.2:0/4176366056 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d540a2c00 0x7f7d540a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c000c00 tx=0x7f7d5c005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.314+0000 7f7d7841b740 1 -- 172.21.15.2:0/4013751535 wait complete. 2024-09-10T08:15:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.314+0000 7f7d7841b740 1 -- start start 2024-09-10T08:15:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.314+0000 7f7d7841b740 1 Processor -- start 2024-09-10T08:15:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.315+0000 7f7d7841b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be27bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.315+0000 7f7d7841b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be13ba0 0x56335be28100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.315+0000 7f7d7841b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be234a0 0x56335be2f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.315+0000 7f7d7841b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56335be26400 con 0x56335be13260 2024-09-10T08:15:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.315+0000 7f7d7841b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56335bd03090 con 0x56335be234a0 2024-09-10T08:15:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.315+0000 7f7d7841b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56335be26580 con 0x56335be13ba0 2024-09-10T08:15:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.315+0000 7f7d7841b740 10 client.0 _ll_register_callbacks cb 0x56335bd57f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.315+0000 7f7d6ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be27bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.315+0000 7f7d6ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be27bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50452/0 (socket says 172.21.15.2:50452) 2024-09-10T08:15:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.315+0000 7f7d6ea00640 1 -- 172.21.15.2:0/3275442847 learned_addr learned my addr 172.21.15.2:0/3275442847 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.315+0000 7f7d6e000640 1 --2- 172.21.15.2:0/3275442847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be13ba0 0x56335be28100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.315+0000 7f7d6f400640 1 --2- 172.21.15.2:0/3275442847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be234a0 0x56335be2f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.316+0000 7f7d6ea00640 1 -- 172.21.15.2:0/3275442847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be13ba0 msgr2=0x56335be28100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.316+0000 7f7d6ea00640 1 --2- 172.21.15.2:0/3275442847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be13ba0 0x56335be28100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.316+0000 7f7d6ea00640 1 -- 172.21.15.2:0/3275442847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be234a0 msgr2=0x56335be2f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.316+0000 7f7d6ea00640 1 --2- 172.21.15.2:0/3275442847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be234a0 0x56335be2f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.316+0000 7f7d6ea00640 1 -- 172.21.15.2:0/3275442847 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335bb59620 con 0x56335be13260 2024-09-10T08:15:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.316+0000 7f7d6ea00640 1 --2- 172.21.15.2:0/3275442847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be27bc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000bfa0 tx=0x7f7d6000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96791054725c5bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.316+0000 7f7d4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.316+0000 7f7d4fe00640 1 -- 172.21.15.2:0/3275442847 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60028060 con 0x56335be13260 2024-09-10T08:15:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.316+0000 7f7d7841b740 1 -- 172.21.15.2:0/3275442847 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d60015070 con 0x56335be13260 2024-09-10T08:15:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.316+0000 7f7d4fe00640 1 -- 172.21.15.2:0/3275442847 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7d60011640 con 0x56335be13260 2024-09-10T08:15:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.316+0000 7f7d4fe00640 1 -- 172.21.15.2:0/3275442847 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d6002f050 con 0x56335be13260 2024-09-10T08:15:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.316+0000 7f7d7841b740 10 client.17304 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.316+0000 7f7d7841b740 1 -- 172.21.15.2:0/3275442847 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56335be31f30 con 0x56335be13260 2024-09-10T08:15:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.317+0000 7f7d4fe00640 1 -- 172.21.15.2:0/3275442847 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d60015070 con 0x56335be13260 2024-09-10T08:15:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.317+0000 7f7d4fe00640 1 -- 172.21.15.2:0/3275442847 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7d60022070 con 0x56335be13260 2024-09-10T08:15:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.317+0000 7f7d7841b740 10 client.17304 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.317+0000 7f7d7841b740 10 client.17304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.317+0000 7f7d7841b740 20 client.17304 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.317+0000 7f7d7841b740 1 --2- 172.21.15.2:0/3275442847 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56335be33390 0x56335be35780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.317+0000 7f7d7841b740 4 client.17304 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.317+0000 7f7d7841b740 1 -- 172.21.15.2:0/3275442847 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56335bd87830 con 0x56335be33390 2024-09-10T08:15:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.317+0000 7f7d6e000640 1 --2- 172.21.15.2:0/3275442847 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56335be33390 0x56335be35780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.318+0000 7f7d6e000640 1 --2- 172.21.15.2:0/3275442847 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56335be33390 0x56335be35780 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d7c06aafd5cae4 server_cookie=f42dd9dd8bc11f21 in_seq=0 out_seq=0 2024-09-10T08:15:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.318+0000 7f7d4fe00640 10 client.17304 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d4fe00640 1 -- 172.21.15.2:0/3275442847 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56335bd87830 con 0x56335be33390 2024-09-10T08:15:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d4fe00640 10 client.17304 handle_command_reply: tid=0 2024-09-10T08:15:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d7841b740 1 client.17304 shutdown 2024-09-10T08:15:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d7841b740 1 -- 172.21.15.2:0/3275442847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 msgr2=0x56335be27bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d7841b740 1 --2- 172.21.15.2:0/3275442847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be27bc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000bfa0 tx=0x7f7d6000bfd0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d7841b740 1 -- 172.21.15.2:0/3275442847 shutdown_connections 2024-09-10T08:15:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d7841b740 1 --2- 172.21.15.2:0/3275442847 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56335be33390 0x56335be35780 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d7841b740 1 --2- 172.21.15.2:0/3275442847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be234a0 0x56335be2f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d7841b740 1 --2- 172.21.15.2:0/3275442847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be13ba0 0x56335be28100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d7841b740 1 --2- 172.21.15.2:0/3275442847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be27bc0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d7841b740 1 -- 172.21.15.2:0/3275442847 >> 172.21.15.2:0/3275442847 conn(0x56335be10850 msgr2=0x56335be12060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d4fe00640 0 client.17304 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d7841b740 1 -- 172.21.15.2:0/3275442847 shutdown_connections 2024-09-10T08:15:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d7841b740 1 -- 172.21.15.2:0/3275442847 wait complete. 2024-09-10T08:15:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.319+0000 7f7d7841b740 20 client.17304 trim_cache size 0 max 16384 2024-09-10T08:15:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.334+0000 7f7d7841b740 1 Processor -- start 2024-09-10T08:15:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.334+0000 7f7d7841b740 1 -- start start 2024-09-10T08:15:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.334+0000 7f7d7841b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be28080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.334+0000 7f7d7841b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be13ba0 0x56335be2f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.334+0000 7f7d7841b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be234a0 0x56335be1cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d7841b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701b3660 con 0x56335be13260 2024-09-10T08:15:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d7841b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701b0f30 con 0x56335be234a0 2024-09-10T08:15:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d7841b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701aed80 con 0x56335be13ba0 2024-09-10T08:15:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d6ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be28080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d6ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be28080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50462/0 (socket says 172.21.15.2:50462) 2024-09-10T08:15:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d6ea00640 1 -- 172.21.15.2:0/1490373727 learned_addr learned my addr 172.21.15.2:0/1490373727 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d6e000640 1 --2- 172.21.15.2:0/1490373727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be13ba0 0x56335be2f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d6f400640 1 --2- 172.21.15.2:0/1490373727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be234a0 0x56335be1cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d6ea00640 1 -- 172.21.15.2:0/1490373727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be13ba0 msgr2=0x56335be2f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d6ea00640 1 --2- 172.21.15.2:0/1490373727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be13ba0 0x56335be2f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d6ea00640 1 -- 172.21.15.2:0/1490373727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be234a0 msgr2=0x56335be1cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d6ea00640 1 --2- 172.21.15.2:0/1490373727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be234a0 0x56335be1cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d6ea00640 1 -- 172.21.15.2:0/1490373727 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335be10670 con 0x56335be13260 2024-09-10T08:15:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.335+0000 7f7d6ea00640 1 --2- 172.21.15.2:0/1490373727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be28080 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7d60019bb0 tx=0x7f7d60005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9823f30e5fca292 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.336+0000 7f7d5aa00640 1 -- 172.21.15.2:0/1490373727 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60028050 con 0x56335be13260 2024-09-10T08:15:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.336+0000 7f7d5aa00640 1 -- 172.21.15.2:0/1490373727 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d60015070 con 0x56335be13260 2024-09-10T08:15:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.336+0000 7f7d5aa00640 1 -- 172.21.15.2:0/1490373727 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60010c80 con 0x56335be13260 2024-09-10T08:15:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.336+0000 7f7d7841b740 1 -- 172.21.15.2:0/1490373727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 msgr2=0x56335be28080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.336+0000 7f7d7841b740 1 --2- 172.21.15.2:0/1490373727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be28080 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7d60019bb0 tx=0x7f7d60005a70 comp rx=0 tx=0).stop 2024-09-10T08:15:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.336+0000 7f7d7841b740 1 -- 172.21.15.2:0/1490373727 shutdown_connections 2024-09-10T08:15:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.336+0000 7f7d7841b740 1 --2- 172.21.15.2:0/1490373727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be234a0 0x56335be1cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.336+0000 7f7d7841b740 1 --2- 172.21.15.2:0/1490373727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be13ba0 0x56335be2f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.336+0000 7f7d7841b740 1 --2- 172.21.15.2:0/1490373727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be28080 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.336+0000 7f7d7841b740 1 -- 172.21.15.2:0/1490373727 >> 172.21.15.2:0/1490373727 conn(0x56335be10850 msgr2=0x56335be27970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.336+0000 7f7d7841b740 1 -- 172.21.15.2:0/1490373727 shutdown_connections 2024-09-10T08:15:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.336+0000 7f7d7841b740 1 -- 172.21.15.2:0/1490373727 wait complete. 2024-09-10T08:15:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.337+0000 7f7d7841b740 1 -- start start 2024-09-10T08:15:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.337+0000 7f7d7841b740 1 Processor -- start 2024-09-10T08:15:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.337+0000 7f7d7841b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be2a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.337+0000 7f7d7841b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be13ba0 0x56335be2a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.337+0000 7f7d7841b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be234a0 0x56335be9bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.337+0000 7f7d7841b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701af3f0 con 0x56335be13260 2024-09-10T08:15:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.337+0000 7f7d7841b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701b0860 con 0x56335be13ba0 2024-09-10T08:15:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.337+0000 7f7d7841b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701afac0 con 0x56335be234a0 2024-09-10T08:15:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.337+0000 7f7d7841b740 10 client.0 _ll_register_callbacks cb 0x7f7d5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d6e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be13ba0 0x56335be2a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d6e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be13ba0 0x56335be2a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35226/0 (socket says 172.21.15.2:35226) 2024-09-10T08:15:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d6e000640 1 -- 172.21.15.2:0/4062202056 learned_addr learned my addr 172.21.15.2:0/4062202056 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d6ea00640 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be2a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d6f400640 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be234a0 0x56335be9bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d6f400640 1 -- 172.21.15.2:0/4062202056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be13ba0 msgr2=0x56335be2a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d6f400640 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be13ba0 0x56335be2a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d6f400640 1 -- 172.21.15.2:0/4062202056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 msgr2=0x56335be2a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d6f400640 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be2a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d6f400640 1 -- 172.21.15.2:0/4062202056 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335be9c480 con 0x56335be234a0 2024-09-10T08:15:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d6ea00640 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be2a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d6f400640 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be234a0 0x56335be9bf40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800eb10 tx=0x7f7d6809b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725fa0ed7effcab7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:15:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.338+0000 7f7d4fe00640 1 -- 172.21.15.2:0/4062202056 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6809c6d0 con 0x56335be234a0 2024-09-10T08:15:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.339+0000 7f7d7841b740 1 -- 172.21.15.2:0/4062202056 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d60015070 con 0x56335be234a0 2024-09-10T08:15:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.339+0000 7f7d4fe00640 1 -- 172.21.15.2:0/4062202056 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7d68005310 con 0x56335be234a0 2024-09-10T08:15:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.339+0000 7f7d4fe00640 1 -- 172.21.15.2:0/4062202056 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d6809bfd0 con 0x56335be234a0 2024-09-10T08:15:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.339+0000 7f7d7841b740 10 client.27289 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.339+0000 7f7d7841b740 1 -- 172.21.15.2:0/4062202056 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56335be9d880 con 0x56335be234a0 2024-09-10T08:15:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.339+0000 7f7d4fe00640 1 -- 172.21.15.2:0/4062202056 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d60015070 con 0x56335be234a0 2024-09-10T08:15:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.339+0000 7f7d4fe00640 1 -- 172.21.15.2:0/4062202056 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7d68096b40 con 0x56335be234a0 2024-09-10T08:15:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.339+0000 7f7d7841b740 10 client.27289 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.339+0000 7f7d7841b740 10 client.27289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.339+0000 7f7d7841b740 20 client.27289 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.339+0000 7f7d7841b740 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56335be9e130 0x56335bea0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.339+0000 7f7d7841b740 4 client.27289 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:15:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.339+0000 7f7d7841b740 1 -- 172.21.15.2:0/4062202056 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56335bd81b80 con 0x56335be9e130 2024-09-10T08:15:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.340+0000 7f7d6ea00640 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56335be9e130 0x56335bea0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.340+0000 7f7d6ea00640 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56335be9e130 0x56335bea0550 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fecf85d2d97b7f80 server_cookie=88941da245fcce77 in_seq=0 out_seq=0 2024-09-10T08:15:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.340+0000 7f7d4fe00640 10 client.27289 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d4fe00640 1 -- 172.21.15.2:0/4062202056 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56335bd81b80 con 0x56335be9e130 2024-09-10T08:15:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d4fe00640 10 client.27289 handle_command_reply: tid=0 2024-09-10T08:15:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d7841b740 1 client.27289 shutdown 2024-09-10T08:15:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d7841b740 1 -- 172.21.15.2:0/4062202056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be234a0 msgr2=0x56335be9bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d7841b740 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be234a0 0x56335be9bf40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800eb10 tx=0x7f7d6809b9f0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d7841b740 1 -- 172.21.15.2:0/4062202056 shutdown_connections 2024-09-10T08:15:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d7841b740 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56335be9e130 0x56335bea0550 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d7841b740 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56335be234a0 0x56335be9bf40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d7841b740 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56335be13ba0 0x56335be2a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d7841b740 1 --2- 172.21.15.2:0/4062202056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335be13260 0x56335be2a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d4fe00640 0 client.27289 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d7841b740 1 -- 172.21.15.2:0/4062202056 >> 172.21.15.2:0/4062202056 conn(0x56335be10850 msgr2=0x56335be27e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d7841b740 1 -- 172.21.15.2:0/4062202056 shutdown_connections 2024-09-10T08:15:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.351+0000 7f7d7841b740 1 -- 172.21.15.2:0/4062202056 wait complete. 2024-09-10T08:15:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.352+0000 7f7d7841b740 20 client.27289 trim_cache size 0 max 16384 2024-09-10T08:15:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:15:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:15:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:15:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:15:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.352+0000 7f7d6fe00640 1 -- 172.21.15.2:0/4176366056 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d540a2c00 msgr2=0x7f7d540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.352+0000 7f7d6fe00640 1 --2- 172.21.15.2:0/4176366056 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d540a2c00 0x7f7d540a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c000c00 tx=0x7f7d5c005d50 comp rx=0 tx=0).stop 2024-09-10T08:15:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.352+0000 7f7d6fe00640 1 -- 172.21.15.2:0/4176366056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d70107620 msgr2=0x7f7d701a7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.352+0000 7f7d6fe00640 1 --2- 172.21.15.2:0/4176366056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d70107620 0x7f7d701a7d50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800eb40 tx=0x7f7d6800c2d0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.352+0000 7f7d6fe00640 1 -- 172.21.15.2:0/4176366056 shutdown_connections 2024-09-10T08:15:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.352+0000 7f7d6fe00640 1 --2- 172.21.15.2:0/4176366056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d70107620 0x7f7d701a7d50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.352+0000 7f7d6fe00640 1 --2- 172.21.15.2:0/4176366056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d70104cf0 0x7f7d70071160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.352+0000 7f7d6fe00640 1 --2- 172.21.15.2:0/4176366056 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7d540a2c00 0x7f7d540a50c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.352+0000 7f7d6fe00640 1 --2- 172.21.15.2:0/4176366056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d701023c0 0x7f7d70070c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.352+0000 7f7d6fe00640 1 -- 172.21.15.2:0/4176366056 >> 172.21.15.2:0/4176366056 conn(0x7f7d700fbfa0 msgr2=0x7f7d70102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.352+0000 7f7d6fe00640 1 -- 172.21.15.2:0/4176366056 shutdown_connections 2024-09-10T08:15:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:05.352+0000 7f7d6fe00640 1 -- 172.21.15.2:0/4176366056 wait complete. 2024-09-10T08:15:05.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:04 smithi155 ceph-mon[35405]: pgmap v595: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-09-10T08:15:05.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:04 smithi107 ceph-mon[35048]: pgmap v595: 129 pgs: 129 active+clean; 3.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-09-10T08:15:06.201 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:15:06.577 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:07.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.172+0000 7f50ca200640 1 -- 172.21.15.2:0/3569657817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c4103ff0 msgr2=0x7f50c41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:07.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.172+0000 7f50ca200640 1 --2- 172.21.15.2:0/3569657817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c4103ff0 0x7f50c41043d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009860 tx=0x7f50b402a340 comp rx=0 tx=0).stop 2024-09-10T08:15:07.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.172+0000 7f50ca200640 1 -- 172.21.15.2:0/3569657817 shutdown_connections 2024-09-10T08:15:07.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.172+0000 7f50ca200640 1 --2- 172.21.15.2:0/3569657817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50c4109120 0x7f50c4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.172+0000 7f50ca200640 1 --2- 172.21.15.2:0/3569657817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c41049a0 0x7f50c41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.172+0000 7f50ca200640 1 --2- 172.21.15.2:0/3569657817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c4103ff0 0x7f50c41043d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.172+0000 7f50ca200640 1 -- 172.21.15.2:0/3569657817 >> 172.21.15.2:0/3569657817 conn(0x7f50c40ffa90 msgr2=0x7f50c4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:07.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.172+0000 7f50ca200640 1 -- 172.21.15.2:0/3569657817 shutdown_connections 2024-09-10T08:15:07.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.172+0000 7f50ca200640 1 -- 172.21.15.2:0/3569657817 wait complete. 2024-09-10T08:15:07.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.173+0000 7f50ca200640 1 Processor -- start 2024-09-10T08:15:07.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.173+0000 7f50ca200640 1 -- start start 2024-09-10T08:15:07.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.174+0000 7f50ca200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4103ff0 0x7f50c419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.174+0000 7f50ca200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50c41049a0 0x7f50c419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.174+0000 7f50ca200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c4109120 0x7f50c419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.174+0000 7f50ca200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f50c4114120 con 0x7f50c4103ff0 2024-09-10T08:15:07.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.174+0000 7f50ca200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50c4113fa0 con 0x7f50c4109120 2024-09-10T08:15:07.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.174+0000 7f50ca200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41142a0 con 0x7f50c41049a0 2024-09-10T08:15:07.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.174+0000 7f50c3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4103ff0 0x7f50c419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.174+0000 7f50c3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4103ff0 0x7f50c419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42852/0 (socket says 172.21.15.2:42852) 2024-09-10T08:15:07.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.174+0000 7f50c3400640 1 -- 172.21.15.2:0/1874839330 learned_addr learned my addr 172.21.15.2:0/1874839330 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:07.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.174+0000 7f50c3e00640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c4109120 0x7f50c419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.174+0000 7f50baa00640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50c41049a0 0x7f50c419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.175+0000 7f50c3400640 1 -- 172.21.15.2:0/1874839330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50c41049a0 msgr2=0x7f50c419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:07.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.175+0000 7f50c3400640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50c41049a0 0x7f50c419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.175+0000 7f50c3400640 1 -- 172.21.15.2:0/1874839330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c4109120 msgr2=0x7f50c419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:07.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.175+0000 7f50c3400640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c4109120 0x7f50c419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.175+0000 7f50c3400640 1 -- 172.21.15.2:0/1874839330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c419f590 con 0x7f50c4103ff0 2024-09-10T08:15:07.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.175+0000 7f50c3400640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4103ff0 0x7f50c419a5e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f50b4035040 tx=0x7f50b4033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:07.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.175+0000 7f50c0c00640 1 -- 172.21.15.2:0/1874839330 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b403f020 con 0x7f50c4103ff0 2024-09-10T08:15:07.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.175+0000 7f50ca200640 1 -- 172.21.15.2:0/1874839330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c419f820 con 0x7f50c4103ff0 2024-09-10T08:15:07.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.175+0000 7f50c0c00640 1 -- 172.21.15.2:0/1874839330 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50b4032770 con 0x7f50c4103ff0 2024-09-10T08:15:07.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.175+0000 7f50c0c00640 1 -- 172.21.15.2:0/1874839330 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4037040 con 0x7f50c4103ff0 2024-09-10T08:15:07.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.175+0000 7f50ca200640 1 -- 172.21.15.2:0/1874839330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c41a7160 con 0x7f50c4103ff0 2024-09-10T08:15:07.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.178+0000 7f50cce28740 1 Processor -- start 2024-09-10T08:15:07.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.178+0000 7f50c0c00640 1 -- 172.21.15.2:0/1874839330 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f50b4042020 con 0x7f50c4103ff0 2024-09-10T08:15:07.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.179+0000 7f50cce28740 1 -- start start 2024-09-10T08:15:07.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.179+0000 7f50cce28740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad93c70 0x55d56ad94050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.179+0000 7f50cce28740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad94590 0x55d56ad9e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.179+0000 7f50cce28740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 0x55d56ada1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.179+0000 7f50cce28740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d56acfa3e0 con 0x55d56ad94590 2024-09-10T08:15:07.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.179+0000 7f50cce28740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d56aca1d80 con 0x55d56ad93c70 2024-09-10T08:15:07.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.179+0000 7f50cce28740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d56acf63a0 con 0x55d56ad9ee50 2024-09-10T08:15:07.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.179+0000 7f50c3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad94590 0x55d56ad9e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.179+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad93c70 0x55d56ad94050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.179+0000 7f50c3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad94590 0x55d56ad9e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42868/0 (socket says 172.21.15.2:42868) 2024-09-10T08:15:07.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.179+0000 7f50c3400640 1 -- 172.21.15.2:0/3414678488 learned_addr learned my addr 172.21.15.2:0/3414678488 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:07.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.179+0000 7f50baa00640 1 --2- 172.21.15.2:0/3414678488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 0x55d56ada1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.180+0000 7f50c3e00640 1 -- 172.21.15.2:0/3414678488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 msgr2=0x55d56ada1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:07.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.180+0000 7f50c3e00640 1 --2- 172.21.15.2:0/3414678488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 0x55d56ada1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.180+0000 7f50c3e00640 1 -- 172.21.15.2:0/3414678488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad94590 msgr2=0x55d56ad9e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:07.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.180+0000 7f50c3e00640 1 --2- 172.21.15.2:0/3414678488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad94590 0x55d56ad9e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.180+0000 7f50c3e00640 1 -- 172.21.15.2:0/3414678488 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d56acc0ae0 con 0x55d56ad93c70 2024-09-10T08:15:07.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.180+0000 7f50c3e00640 1 --2- 172.21.15.2:0/3414678488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad93c70 0x55d56ad94050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009f00 tx=0x7f50bc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54d0b7a3d24f5f01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:07.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.180+0000 7f50c0c00640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f50a80a2c00 0x7f50a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.180+0000 7f50b3e00640 1 -- 172.21.15.2:0/3414678488 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc004d20 con 0x55d56ad93c70 2024-09-10T08:15:07.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.180+0000 7f50b3e00640 1 -- 172.21.15.2:0/3414678488 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50bc004050 con 0x55d56ad93c70 2024-09-10T08:15:07.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.181+0000 7f50b3e00640 1 -- 172.21.15.2:0/3414678488 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc005280 con 0x55d56ad93c70 2024-09-10T08:15:07.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.181+0000 7f50c0c00640 1 -- 172.21.15.2:0/1874839330 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f50b40ef540 con 0x7f50c4103ff0 2024-09-10T08:15:07.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.181+0000 7f50c3400640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f50a80a2c00 0x7f50a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.181+0000 7f50cce28740 1 -- 172.21.15.2:0/3414678488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad93c70 msgr2=0x55d56ad94050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:07.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.181+0000 7f50cce28740 1 --2- 172.21.15.2:0/3414678488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad93c70 0x55d56ad94050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009f00 tx=0x7f50bc011810 comp rx=0 tx=0).stop 2024-09-10T08:15:07.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.181+0000 7f50c3400640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f50a80a2c00 0x7f50a80a50c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f50b40bb6b0 tx=0x7f50b40f50e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:07.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.182+0000 7f50cce28740 1 -- 172.21.15.2:0/3414678488 shutdown_connections 2024-09-10T08:15:07.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.182+0000 7f50cce28740 1 --2- 172.21.15.2:0/3414678488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 0x55d56ada1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.182+0000 7f50cce28740 1 --2- 172.21.15.2:0/3414678488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad94590 0x55d56ad9e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.182+0000 7f50cce28740 1 --2- 172.21.15.2:0/3414678488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad93c70 0x55d56ad94050 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.182+0000 7f50cce28740 1 -- 172.21.15.2:0/3414678488 >> 172.21.15.2:0/3414678488 conn(0x55d56ad90200 msgr2=0x55d56ad905e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:07.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.182+0000 7f50cce28740 1 -- 172.21.15.2:0/3414678488 shutdown_connections 2024-09-10T08:15:07.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.182+0000 7f50cce28740 1 -- 172.21.15.2:0/3414678488 wait complete. 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.182+0000 7f50cce28740 1 -- start start 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.182+0000 7f50cce28740 1 Processor -- start 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.183+0000 7f50cce28740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad93c70 0x55d56ad9bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.183+0000 7f50cce28740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad94590 0x55d56ad9c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.183+0000 7f50cce28740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 0x55d56ad9c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.183+0000 7f50cce28740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d56ada1a20 con 0x55d56ad93c70 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.183+0000 7f50cce28740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d56acb50f0 con 0x55d56ad94590 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.183+0000 7f50cce28740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d56ada1ba0 con 0x55d56ad9ee50 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.183+0000 7f50cce28740 10 client.0 _ll_register_callbacks cb 0x55d56ac9cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.183+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad93c70 0x55d56ad9bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.183+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad93c70 0x55d56ad9bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42870/0 (socket says 172.21.15.2:42870) 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.183+0000 7f50c3e00640 1 -- 172.21.15.2:0/1302402146 learned_addr learned my addr 172.21.15.2:0/1302402146 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.183+0000 7f50baa00640 1 --2- 172.21.15.2:0/1302402146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad94590 0x55d56ad9c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.184+0000 7f50c3e00640 1 --2- 172.21.15.2:0/1302402146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 0x55d56ad9c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.184+0000 7f50c3e00640 1 -- 172.21.15.2:0/1302402146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 msgr2=0x55d56ad9c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:07.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.184+0000 7f50c3e00640 1 --2- 172.21.15.2:0/1302402146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 0x55d56ad9c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.184+0000 7f50c3e00640 1 -- 172.21.15.2:0/1302402146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad94590 msgr2=0x55d56ad9c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:07.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.184+0000 7f50c3e00640 1 --2- 172.21.15.2:0/1302402146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad94590 0x55d56ad9c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.184+0000 7f50c3e00640 1 -- 172.21.15.2:0/1302402146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d56aa6b3c0 con 0x55d56ad93c70 2024-09-10T08:15:07.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.184+0000 7f50c3e00640 1 --2- 172.21.15.2:0/1302402146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad93c70 0x55d56ad9bd20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f50bc01b040 tx=0x7f50bc004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e9089688c4dc8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:07.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.184+0000 7f50b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:07.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.184+0000 7f50b0c00640 1 -- 172.21.15.2:0/1302402146 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc003d20 con 0x55d56ad93c70 2024-09-10T08:15:07.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.184+0000 7f50b0c00640 1 -- 172.21.15.2:0/1302402146 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f50bc028480 con 0x55d56ad93c70 2024-09-10T08:15:07.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.184+0000 7f50cce28740 1 -- 172.21.15.2:0/1302402146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50bc004050 con 0x55d56ad93c70 2024-09-10T08:15:07.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.184+0000 7f50b0c00640 1 -- 172.21.15.2:0/1302402146 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f50bc036050 con 0x55d56ad93c70 2024-09-10T08:15:07.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.185+0000 7f50cce28740 10 client.17322 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:07.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.185+0000 7f50cce28740 1 -- 172.21.15.2:0/1302402146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d56adada40 con 0x55d56ad93c70 2024-09-10T08:15:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.185+0000 7f50b0c00640 1 -- 172.21.15.2:0/1302402146 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50bc004050 con 0x55d56ad93c70 2024-09-10T08:15:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.185+0000 7f50b0c00640 1 -- 172.21.15.2:0/1302402146 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f50bc026680 con 0x55d56ad93c70 2024-09-10T08:15:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.185+0000 7f50cce28740 10 client.17322 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.185+0000 7f50cce28740 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.185+0000 7f50cce28740 20 client.17322 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.185+0000 7f50cce28740 1 --2- 172.21.15.2:0/1302402146 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d56adaeea0 0x55d56adb1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.185+0000 7f50cce28740 4 client.17322 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.185+0000 7f50cce28740 1 -- 172.21.15.2:0/1302402146 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d56ad07950 con 0x55d56adaeea0 2024-09-10T08:15:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.186+0000 7f50baa00640 1 --2- 172.21.15.2:0/1302402146 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d56adaeea0 0x55d56adb1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.186+0000 7f50baa00640 1 --2- 172.21.15.2:0/1302402146 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d56adaeea0 0x55d56adb1290 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd99f49896aed57f server_cookie=71f9e5b3d2c93f30 in_seq=0 out_seq=0 2024-09-10T08:15:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.186+0000 7f50b0c00640 10 client.17322 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.187+0000 7f50b0c00640 1 -- 172.21.15.2:0/1302402146 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d56ad07950 con 0x55d56adaeea0 2024-09-10T08:15:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.187+0000 7f50b0c00640 10 client.17322 handle_command_reply: tid=0 2024-09-10T08:15:07.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.187+0000 7f50cce28740 1 client.17322 shutdown 2024-09-10T08:15:07.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.187+0000 7f50cce28740 1 -- 172.21.15.2:0/1302402146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad93c70 msgr2=0x55d56ad9bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:07.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.187+0000 7f50cce28740 1 --2- 172.21.15.2:0/1302402146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad93c70 0x55d56ad9bd20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f50bc01b040 tx=0x7f50bc004890 comp rx=0 tx=0).stop 2024-09-10T08:15:07.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.187+0000 7f50cce28740 1 -- 172.21.15.2:0/1302402146 shutdown_connections 2024-09-10T08:15:07.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.187+0000 7f50cce28740 1 --2- 172.21.15.2:0/1302402146 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d56adaeea0 0x55d56adb1290 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.187+0000 7f50cce28740 1 --2- 172.21.15.2:0/1302402146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 0x55d56ad9c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.187+0000 7f50cce28740 1 --2- 172.21.15.2:0/1302402146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad94590 0x55d56ad9c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.187+0000 7f50cce28740 1 --2- 172.21.15.2:0/1302402146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad93c70 0x55d56ad9bd20 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.187+0000 7f50b0c00640 0 client.17322 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:07.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.187+0000 7f50cce28740 1 -- 172.21.15.2:0/1302402146 >> 172.21.15.2:0/1302402146 conn(0x55d56ad90200 msgr2=0x55d56ad8f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:07.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.188+0000 7f50cce28740 1 -- 172.21.15.2:0/1302402146 shutdown_connections 2024-09-10T08:15:07.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.188+0000 7f50cce28740 1 -- 172.21.15.2:0/1302402146 wait complete. 2024-09-10T08:15:07.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.188+0000 7f50cce28740 20 client.17322 trim_cache size 0 max 16384 2024-09-10T08:15:07.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.201+0000 7f50cce28740 1 Processor -- start 2024-09-10T08:15:07.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.201+0000 7f50cce28740 1 -- start start 2024-09-10T08:15:07.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50cce28740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad93c70 0x55d56adb6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50cce28740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad94590 0x55d56adb6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50cce28740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 0x55d56ad98f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50cce28740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41b3910 con 0x55d56ad93c70 2024-09-10T08:15:07.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50cce28740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41b11e0 con 0x55d56ad94590 2024-09-10T08:15:07.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50cce28740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41af030 con 0x55d56ad9ee50 2024-09-10T08:15:07.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad93c70 0x55d56adb6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad94590 0x55d56adb6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad93c70 0x55d56adb6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42880/0 (socket says 172.21.15.2:42880) 2024-09-10T08:15:07.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad94590 0x55d56adb6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39534/0 (socket says 172.21.15.2:39534) 2024-09-10T08:15:07.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50c3e00640 1 -- 172.21.15.2:0/1860999561 learned_addr learned my addr 172.21.15.2:0/1860999561 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:07.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50c3e00640 1 --2- 172.21.15.2:0/1860999561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 0x55d56ad98f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50baa00640 1 -- 172.21.15.2:0/1860999561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 msgr2=0x55d56ad98f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:07.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50baa00640 1 --2- 172.21.15.2:0/1860999561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 0x55d56ad98f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50baa00640 1 -- 172.21.15.2:0/1860999561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad93c70 msgr2=0x55d56adb6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:07.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50baa00640 1 --2- 172.21.15.2:0/1860999561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad93c70 0x55d56adb6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.202+0000 7f50baa00640 1 -- 172.21.15.2:0/1860999561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d56adb1820 con 0x55d56ad94590 2024-09-10T08:15:07.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.203+0000 7f50c3e00640 1 --2- 172.21.15.2:0/1860999561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 0x55d56ad98f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:15:07.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.203+0000 7f50baa00640 1 --2- 172.21.15.2:0/1860999561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad94590 0x55d56adb6f90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55d56ad9b5b0 tx=0x7f50ac009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d45704c3e8ba61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:07.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.203+0000 7f50b3400640 1 -- 172.21.15.2:0/1860999561 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac005220 con 0x55d56ad94590 2024-09-10T08:15:07.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.203+0000 7f50b3400640 1 -- 172.21.15.2:0/1860999561 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50ac004c80 con 0x55d56ad94590 2024-09-10T08:15:07.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.203+0000 7f50b3400640 1 -- 172.21.15.2:0/1860999561 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac01db40 con 0x55d56ad94590 2024-09-10T08:15:07.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.204+0000 7f50cce28740 1 -- 172.21.15.2:0/1860999561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad94590 msgr2=0x55d56adb6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:07.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.204+0000 7f50cce28740 1 --2- 172.21.15.2:0/1860999561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad94590 0x55d56adb6f90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55d56ad9b5b0 tx=0x7f50ac009260 comp rx=0 tx=0).stop 2024-09-10T08:15:07.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.204+0000 7f50cce28740 1 -- 172.21.15.2:0/1860999561 shutdown_connections 2024-09-10T08:15:07.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.204+0000 7f50cce28740 1 --2- 172.21.15.2:0/1860999561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad9ee50 0x55d56ad98f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.204+0000 7f50cce28740 1 --2- 172.21.15.2:0/1860999561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad94590 0x55d56adb6f90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.204+0000 7f50cce28740 1 --2- 172.21.15.2:0/1860999561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad93c70 0x55d56adb6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.204+0000 7f50cce28740 1 -- 172.21.15.2:0/1860999561 >> 172.21.15.2:0/1860999561 conn(0x55d56ad90200 msgr2=0x55d56ad95ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:07.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.204+0000 7f50cce28740 1 -- 172.21.15.2:0/1860999561 shutdown_connections 2024-09-10T08:15:07.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.204+0000 7f50cce28740 1 -- 172.21.15.2:0/1860999561 wait complete. 2024-09-10T08:15:07.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.205+0000 7f50cce28740 1 -- start start 2024-09-10T08:15:07.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.205+0000 7f50cce28740 1 Processor -- start 2024-09-10T08:15:07.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.205+0000 7f50cce28740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad93c70 0x55d56ada7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.205+0000 7f50cce28740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad94590 0x55d56ada28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.205+0000 7f50cce28740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad9ee50 0x55d56ada2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.205+0000 7f50cce28740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41af6a0 con 0x55d56ad9ee50 2024-09-10T08:15:07.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.205+0000 7f50cce28740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41b0b10 con 0x55d56ad93c70 2024-09-10T08:15:07.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.205+0000 7f50cce28740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41afd70 con 0x55d56ad94590 2024-09-10T08:15:07.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.205+0000 7f50cce28740 10 client.0 _ll_register_callbacks cb 0x7f50b40f2ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:07.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.206+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad93c70 0x55d56ada7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.206+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad93c70 0x55d56ada7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39538/0 (socket says 172.21.15.2:39538) 2024-09-10T08:15:07.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.206+0000 7f50c3e00640 1 -- 172.21.15.2:0/2879620930 learned_addr learned my addr 172.21.15.2:0/2879620930 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:07.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.206+0000 7f50c3e00640 1 --2- 172.21.15.2:0/2879620930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad9ee50 0x55d56ada2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.206+0000 7f50baa00640 1 --2- 172.21.15.2:0/2879620930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad94590 0x55d56ada28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.206+0000 7f50c3e00640 1 -- 172.21.15.2:0/2879620930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad94590 msgr2=0x55d56ada28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:07.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.206+0000 7f50c3e00640 1 --2- 172.21.15.2:0/2879620930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad94590 0x55d56ada28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.206+0000 7f50c3e00640 1 -- 172.21.15.2:0/2879620930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad9ee50 msgr2=0x55d56ada2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:07.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.206+0000 7f50c3e00640 1 --2- 172.21.15.2:0/2879620930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad9ee50 0x55d56ada2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.206+0000 7f50c3e00640 1 -- 172.21.15.2:0/2879620930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d56ad8b770 con 0x55d56ad93c70 2024-09-10T08:15:07.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.206+0000 7f50c3e00640 1 --2- 172.21.15.2:0/2879620930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad93c70 0x55d56ada7580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f50bc005c40 tx=0x7f50bc009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac99ec0fa84ce937 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:07.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.206+0000 7f50b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:15:07.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50b0c00640 1 -- 172.21.15.2:0/2879620930 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc004b50 con 0x55d56ad93c70 2024-09-10T08:15:07.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50cce28740 1 -- 172.21.15.2:0/2879620930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50ac004c80 con 0x55d56ad93c70 2024-09-10T08:15:07.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50b0c00640 1 -- 172.21.15.2:0/2879620930 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f50bc004cf0 con 0x55d56ad93c70 2024-09-10T08:15:07.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50b0c00640 1 -- 172.21.15.2:0/2879620930 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f50bc026450 con 0x55d56ad93c70 2024-09-10T08:15:07.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50cce28740 10 client.27365 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:07.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50cce28740 1 -- 172.21.15.2:0/2879620930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d56ae197f0 con 0x55d56ad93c70 2024-09-10T08:15:07.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50b0c00640 1 -- 172.21.15.2:0/2879620930 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50ac004c80 con 0x55d56ad93c70 2024-09-10T08:15:07.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50b0c00640 1 -- 172.21.15.2:0/2879620930 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f50bc028a80 con 0x55d56ad93c70 2024-09-10T08:15:07.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50cce28740 10 client.27365 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:07.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50cce28740 10 client.27365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:07.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50cce28740 20 client.27365 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:07.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50cce28740 1 --2- 172.21.15.2:0/2879620930 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d56ae19e50 0x55d56ae1c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:07.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50cce28740 4 client.27365 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:15:07.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.207+0000 7f50cce28740 1 -- 172.21.15.2:0/2879620930 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d56ad01e60 con 0x55d56ae19e50 2024-09-10T08:15:07.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.208+0000 7f50baa00640 1 --2- 172.21.15.2:0/2879620930 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d56ae19e50 0x55d56ae1c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:07.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.208+0000 7f50baa00640 1 --2- 172.21.15.2:0/2879620930 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d56ae19e50 0x55d56ae1c240 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bd0089ade8b222b server_cookie=febecf23c72b23d9 in_seq=0 out_seq=0 2024-09-10T08:15:07.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.208+0000 7f50b0c00640 10 client.27365 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:07.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.208+0000 7f50b0c00640 1 -- 172.21.15.2:0/2879620930 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d56ad01e60 con 0x55d56ae19e50 2024-09-10T08:15:07.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.208+0000 7f50b0c00640 10 client.27365 handle_command_reply: tid=0 2024-09-10T08:15:07.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.209+0000 7f50cce28740 1 client.27365 shutdown 2024-09-10T08:15:07.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.209+0000 7f50cce28740 1 -- 172.21.15.2:0/2879620930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad93c70 msgr2=0x55d56ada7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:07.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.209+0000 7f50cce28740 1 --2- 172.21.15.2:0/2879620930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad93c70 0x55d56ada7580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f50bc005c40 tx=0x7f50bc009500 comp rx=0 tx=0).stop 2024-09-10T08:15:07.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.209+0000 7f50cce28740 1 -- 172.21.15.2:0/2879620930 shutdown_connections 2024-09-10T08:15:07.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.209+0000 7f50cce28740 1 --2- 172.21.15.2:0/2879620930 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d56ae19e50 0x55d56ae1c240 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.209+0000 7f50cce28740 1 --2- 172.21.15.2:0/2879620930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d56ad9ee50 0x55d56ada2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.209+0000 7f50cce28740 1 --2- 172.21.15.2:0/2879620930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d56ad94590 0x55d56ada28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.209+0000 7f50cce28740 1 --2- 172.21.15.2:0/2879620930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d56ad93c70 0x55d56ada7580 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.209+0000 7f50b0c00640 0 client.27365 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:07.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.209+0000 7f50cce28740 1 -- 172.21.15.2:0/2879620930 >> 172.21.15.2:0/2879620930 conn(0x55d56ad90200 msgr2=0x55d56ada9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:07.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.209+0000 7f50cce28740 1 -- 172.21.15.2:0/2879620930 shutdown_connections 2024-09-10T08:15:07.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.209+0000 7f50cce28740 1 -- 172.21.15.2:0/2879620930 wait complete. 2024-09-10T08:15:07.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.209+0000 7f50cce28740 20 client.27365 trim_cache size 0 max 16384 2024-09-10T08:15:07.214 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:15:07.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.210+0000 7f50ca200640 1 -- 172.21.15.2:0/1874839330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f50a80a2c00 msgr2=0x7f50a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:07.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.210+0000 7f50ca200640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f50a80a2c00 0x7f50a80a50c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f50b40bb6b0 tx=0x7f50b40f50e0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.210+0000 7f50ca200640 1 -- 172.21.15.2:0/1874839330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4103ff0 msgr2=0x7f50c419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:07.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.210+0000 7f50ca200640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4103ff0 0x7f50c419a5e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f50b4035040 tx=0x7f50b4033d90 comp rx=0 tx=0).stop 2024-09-10T08:15:07.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.210+0000 7f50ca200640 1 -- 172.21.15.2:0/1874839330 shutdown_connections 2024-09-10T08:15:07.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.210+0000 7f50ca200640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c4109120 0x7f50c419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.210+0000 7f50ca200640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f50a80a2c00 0x7f50a80a50c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.210+0000 7f50ca200640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50c41049a0 0x7f50c419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.210+0000 7f50ca200640 1 --2- 172.21.15.2:0/1874839330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50c4103ff0 0x7f50c419a5e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:07.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.210+0000 7f50ca200640 1 -- 172.21.15.2:0/1874839330 >> 172.21.15.2:0/1874839330 conn(0x7f50c40ffa90 msgr2=0x7f50c4100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:07.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.210+0000 7f50ca200640 1 -- 172.21.15.2:0/1874839330 shutdown_connections 2024-09-10T08:15:07.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:07.210+0000 7f50ca200640 1 -- 172.21.15.2:0/1874839330 wait complete. 2024-09-10T08:15:07.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:06 smithi002 ceph-mon[30563]: pgmap v596: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-10T08:15:07.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:06 smithi155 ceph-mon[35405]: pgmap v596: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-10T08:15:07.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:06 smithi107 ceph-mon[35048]: pgmap v596: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-10T08:15:09.213 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:15:09.214 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:15:09.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:08 smithi002 ceph-mon[30563]: pgmap v597: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-09-10T08:15:09.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:08 smithi155 ceph-mon[35405]: pgmap v597: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-09-10T08:15:09.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:08 smithi107 ceph-mon[35048]: pgmap v597: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-09-10T08:15:09.608 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:10.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.218+0000 7f279e200640 1 -- 172.21.15.2:0/423798929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798101de0 msgr2=0x7f27981021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:10.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.218+0000 7f279e200640 1 --2- 172.21.15.2:0/423798929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798101de0 0x7f27981021c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2788009900 tx=0x7f278802a410 comp rx=0 tx=0).stop 2024-09-10T08:15:10.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.218+0000 7f279e200640 1 -- 172.21.15.2:0/423798929 shutdown_connections 2024-09-10T08:15:10.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.218+0000 7f279e200640 1 --2- 172.21.15.2:0/423798929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2798106f10 0x7f279810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.218+0000 7f279e200640 1 --2- 172.21.15.2:0/423798929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2798102790 0x7f27981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.218+0000 7f279e200640 1 --2- 172.21.15.2:0/423798929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798101de0 0x7f27981021c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.218+0000 7f279e200640 1 -- 172.21.15.2:0/423798929 >> 172.21.15.2:0/423798929 conn(0x7f27980fd8c0 msgr2=0x7f27980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:10.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.218+0000 7f279e200640 1 -- 172.21.15.2:0/423798929 shutdown_connections 2024-09-10T08:15:10.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.218+0000 7f279e200640 1 -- 172.21.15.2:0/423798929 wait complete. 2024-09-10T08:15:10.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.219+0000 7f279e200640 1 Processor -- start 2024-09-10T08:15:10.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.219+0000 7f279e200640 1 -- start start 2024-09-10T08:15:10.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.219+0000 7f279e200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2798101de0 0x7f279819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f279e200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2798102790 0x7f279819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f279e200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798106f10 0x7f279819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f279e200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2798111f80 con 0x7f2798106f10 2024-09-10T08:15:10.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f279e200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2798111e00 con 0x7f2798102790 2024-09-10T08:15:10.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f279e200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2798112100 con 0x7f2798101de0 2024-09-10T08:15:10.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f278ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2798102790 0x7f279819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f2797e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798106f10 0x7f279819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f278ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2798102790 0x7f279819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39562/0 (socket says 172.21.15.2:39562) 2024-09-10T08:15:10.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f278ea00640 1 -- 172.21.15.2:0/3826266897 learned_addr learned my addr 172.21.15.2:0/3826266897 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:10.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f2797400640 1 --2- 172.21.15.2:0/3826266897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2798101de0 0x7f279819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f278ea00640 1 -- 172.21.15.2:0/3826266897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2798101de0 msgr2=0x7f279819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:10.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f278ea00640 1 --2- 172.21.15.2:0/3826266897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2798101de0 0x7f279819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f278ea00640 1 -- 172.21.15.2:0/3826266897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798106f10 msgr2=0x7f279819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:10.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f278ea00640 1 --2- 172.21.15.2:0/3826266897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798106f10 0x7f279819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.220+0000 7f278ea00640 1 -- 172.21.15.2:0/3826266897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279819f620 con 0x7f2798102790 2024-09-10T08:15:10.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.221+0000 7f278ea00640 1 --2- 172.21.15.2:0/3826266897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2798102790 0x7f279819abb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f278000d950 tx=0x7f278000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:10.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.221+0000 7f2794c00640 1 -- 172.21.15.2:0/3826266897 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780017030 con 0x7f2798102790 2024-09-10T08:15:10.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.221+0000 7f279e200640 1 -- 172.21.15.2:0/3826266897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279819f910 con 0x7f2798102790 2024-09-10T08:15:10.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.221+0000 7f2794c00640 1 -- 172.21.15.2:0/3826266897 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2780004020 con 0x7f2798102790 2024-09-10T08:15:10.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.221+0000 7f2794c00640 1 -- 172.21.15.2:0/3826266897 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800047a0 con 0x7f2798102790 2024-09-10T08:15:10.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.221+0000 7f279e200640 1 -- 172.21.15.2:0/3826266897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27981a71f0 con 0x7f2798102790 2024-09-10T08:15:10.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.224+0000 7f27a0cb0740 1 Processor -- start 2024-09-10T08:15:10.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.224+0000 7f27a0cb0740 1 -- start start 2024-09-10T08:15:10.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.224+0000 7f2794c00640 1 -- 172.21.15.2:0/3826266897 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f27800041d0 con 0x7f2798102790 2024-09-10T08:15:10.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.225+0000 7f27a0cb0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd0fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.225+0000 7f27a0cb0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd103e0 0x55ecdfd1a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.225+0000 7f27a0cb0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd1d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.225+0000 7f27a0cb0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ecdfc76290 con 0x55ecdfd103e0 2024-09-10T08:15:10.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.225+0000 7f27a0cb0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ecdfc1db40 con 0x55ecdfd1aca0 2024-09-10T08:15:10.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.225+0000 7f27a0cb0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ecdfc72250 con 0x55ecdfd0fac0 2024-09-10T08:15:10.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.225+0000 7f2797e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd0fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.225+0000 7f2797e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd0fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:32908/0 (socket says 172.21.15.2:32908) 2024-09-10T08:15:10.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.225+0000 7f2797e00640 1 -- 172.21.15.2:0/3212193326 learned_addr learned my addr 172.21.15.2:0/3212193326 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:10.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.225+0000 7f2797400640 1 --2- 172.21.15.2:0/3212193326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd103e0 0x55ecdfd1a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.225+0000 7f278ea00640 1 --2- 172.21.15.2:0/3212193326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd1d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.226+0000 7f2797e00640 1 -- 172.21.15.2:0/3212193326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd1aca0 msgr2=0x55ecdfd1d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:10.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.226+0000 7f2797e00640 1 --2- 172.21.15.2:0/3212193326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd1d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.226+0000 7f2797e00640 1 -- 172.21.15.2:0/3212193326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd103e0 msgr2=0x55ecdfd1a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:10.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.226+0000 7f2797e00640 1 --2- 172.21.15.2:0/3212193326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd103e0 0x55ecdfd1a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.226+0000 7f2797e00640 1 -- 172.21.15.2:0/3212193326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecdf9e76a0 con 0x55ecdfd0fac0 2024-09-10T08:15:10.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.226+0000 7f278ea00640 1 --2- 172.21.15.2:0/3212193326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd1d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:10.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.226+0000 7f2797e00640 1 --2- 172.21.15.2:0/3212193326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd0fea0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2790009f50 tx=0x7f27900118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7719e0aaab7434d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:10.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.226+0000 7f2794c00640 1 --2- 172.21.15.2:0/3826266897 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f277c0a2b80 0x7f277c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.226+0000 7f2787e00640 1 -- 172.21.15.2:0/3212193326 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790004d20 con 0x55ecdfd0fac0 2024-09-10T08:15:10.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.226+0000 7f2787e00640 1 -- 172.21.15.2:0/3212193326 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2790004050 con 0x55ecdfd0fac0 2024-09-10T08:15:10.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.226+0000 7f2787e00640 1 -- 172.21.15.2:0/3212193326 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790005280 con 0x55ecdfd0fac0 2024-09-10T08:15:10.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.226+0000 7f2794c00640 1 -- 172.21.15.2:0/3826266897 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f27800cede0 con 0x7f2798102790 2024-09-10T08:15:10.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.226+0000 7f2797400640 1 --2- 172.21.15.2:0/3826266897 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f277c0a2b80 0x7f277c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.227+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3212193326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 msgr2=0x55ecdfd0fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:10.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.227+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3212193326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd0fea0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2790009f50 tx=0x7f27900118d0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.227+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3212193326 shutdown_connections 2024-09-10T08:15:10.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.227+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3212193326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd1d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.227+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3212193326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd103e0 0x55ecdfd1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.227+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3212193326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd0fea0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.227+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3212193326 >> 172.21.15.2:0/3212193326 conn(0x55ecdfd0c050 msgr2=0x55ecdfd0c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:10.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.227+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3212193326 shutdown_connections 2024-09-10T08:15:10.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.227+0000 7f2797400640 1 --2- 172.21.15.2:0/3826266897 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f277c0a2b80 0x7f277c0a5040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2788000c00 tx=0x7f2788005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:10.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.227+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3212193326 wait complete. 2024-09-10T08:15:10.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.228+0000 7f27a0cb0740 1 -- start start 2024-09-10T08:15:10.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.228+0000 7f27a0cb0740 1 Processor -- start 2024-09-10T08:15:10.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.228+0000 7f27a0cb0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd1a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.228+0000 7f27a0cb0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd178c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.228+0000 7f27a0cb0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd17e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.228+0000 7f27a0cb0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ecdfd1d910 con 0x55ecdfd0fac0 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.228+0000 7f27a0cb0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ecdfc30fa0 con 0x55ecdfd103e0 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.228+0000 7f27a0cb0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ecdfd1da90 con 0x55ecdfd1aca0 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.229+0000 7f27a0cb0740 10 client.0 _ll_register_callbacks cb 0x55ecdf984ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.229+0000 7f2797400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd1a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.229+0000 7f2797e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd17e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.229+0000 7f278ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.229+0000 7f2797400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd1a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42914/0 (socket says 172.21.15.2:42914) 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.229+0000 7f2797400640 1 -- 172.21.15.2:0/3818546848 learned_addr learned my addr 172.21.15.2:0/3818546848 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.229+0000 7f2797e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd17e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:32918/0 (socket says 172.21.15.2:32918) 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.229+0000 7f2797400640 1 -- 172.21.15.2:0/3818546848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd1aca0 msgr2=0x55ecdfd17e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.229+0000 7f2797400640 1 --2- 172.21.15.2:0/3818546848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd17e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.229+0000 7f2797400640 1 -- 172.21.15.2:0/3818546848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 msgr2=0x55ecdfd178c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:10.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.229+0000 7f2797400640 1 --2- 172.21.15.2:0/3818546848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd178c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.229+0000 7f2797400640 1 -- 172.21.15.2:0/3818546848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecdfd18650 con 0x55ecdfd0fac0 2024-09-10T08:15:10.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.229+0000 7f2797e00640 1 --2- 172.21.15.2:0/3818546848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd17e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:10.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.230+0000 7f2797400640 1 --2- 172.21.15.2:0/3818546848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd1a610 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2788009900 tx=0x7f2788009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d072f5cc1505d746 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:10.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.230+0000 7f2784c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:10.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.230+0000 7f2784c00640 1 -- 172.21.15.2:0/3818546848 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278804e070 con 0x55ecdfd0fac0 2024-09-10T08:15:10.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.230+0000 7f2784c00640 1 -- 172.21.15.2:0/3818546848 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f27880048c0 con 0x55ecdfd0fac0 2024-09-10T08:15:10.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.230+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3818546848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2790004050 con 0x55ecdfd0fac0 2024-09-10T08:15:10.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.230+0000 7f2784c00640 1 -- 172.21.15.2:0/3818546848 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f278805a050 con 0x55ecdfd0fac0 2024-09-10T08:15:10.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.230+0000 7f27a0cb0740 10 client.17340 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:10.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.230+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3818546848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ecdfd29850 con 0x55ecdfd0fac0 2024-09-10T08:15:10.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.231+0000 7f2784c00640 1 -- 172.21.15.2:0/3818546848 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2790004050 con 0x55ecdfd0fac0 2024-09-10T08:15:10.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.231+0000 7f2784c00640 1 -- 172.21.15.2:0/3818546848 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f27880035f0 con 0x55ecdfd0fac0 2024-09-10T08:15:10.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.231+0000 7f27a0cb0740 10 client.17340 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:10.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.231+0000 7f27a0cb0740 10 client.17340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:10.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.231+0000 7f27a0cb0740 20 client.17340 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:10.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.231+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3818546848 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ecdfd2ac60 0x55ecdfd2d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.231+0000 7f27a0cb0740 4 client.17340 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:10.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.231+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3818546848 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecdfc83800 con 0x55ecdfd2ac60 2024-09-10T08:15:10.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.231+0000 7f278ea00640 1 --2- 172.21.15.2:0/3818546848 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ecdfd2ac60 0x55ecdfd2d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.232+0000 7f278ea00640 1 --2- 172.21.15.2:0/3818546848 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ecdfd2ac60 0x55ecdfd2d090 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be42447ce83b313 server_cookie=ca14a453fa11dc0e in_seq=0 out_seq=0 2024-09-10T08:15:10.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.232+0000 7f2784c00640 10 client.17340 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:10.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f2784c00640 1 -- 172.21.15.2:0/3818546848 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ecdfc83800 con 0x55ecdfd2ac60 2024-09-10T08:15:10.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f2784c00640 10 client.17340 handle_command_reply: tid=0 2024-09-10T08:15:10.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f27a0cb0740 1 client.17340 shutdown 2024-09-10T08:15:10.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3818546848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 msgr2=0x55ecdfd1a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:10.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3818546848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd1a610 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2788009900 tx=0x7f2788009260 comp rx=0 tx=0).stop 2024-09-10T08:15:10.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3818546848 shutdown_connections 2024-09-10T08:15:10.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3818546848 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ecdfd2ac60 0x55ecdfd2d090 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3818546848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd17e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3818546848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd178c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3818546848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd1a610 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f2784c00640 0 client.17340 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:10.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3818546848 >> 172.21.15.2:0/3818546848 conn(0x55ecdfd0c050 msgr2=0x55ecdfd0b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:10.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3818546848 shutdown_connections 2024-09-10T08:15:10.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3818546848 wait complete. 2024-09-10T08:15:10.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.233+0000 7f27a0cb0740 20 client.17340 trim_cache size 0 max 16384 2024-09-10T08:15:10.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.251+0000 7f27a0cb0740 1 Processor -- start 2024-09-10T08:15:10.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.252+0000 7f27a0cb0740 1 -- start start 2024-09-10T08:15:10.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.252+0000 7f27a0cb0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd329b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.252+0000 7f27a0cb0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd32ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.252+0000 7f27a0cb0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd19120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.252+0000 7f27a0cb0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f27981b3990 con 0x55ecdfd0fac0 2024-09-10T08:15:10.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.252+0000 7f27a0cb0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f27981b1260 con 0x55ecdfd103e0 2024-09-10T08:15:10.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.252+0000 7f27a0cb0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f27981af0b0 con 0x55ecdfd1aca0 2024-09-10T08:15:10.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.252+0000 7f2797400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd329b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.252+0000 7f2797400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd329b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42918/0 (socket says 172.21.15.2:42918) 2024-09-10T08:15:10.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.252+0000 7f2797400640 1 -- 172.21.15.2:0/3746713886 learned_addr learned my addr 172.21.15.2:0/3746713886 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:10.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.252+0000 7f2797e00640 1 --2- 172.21.15.2:0/3746713886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd19120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.252+0000 7f278ea00640 1 --2- 172.21.15.2:0/3746713886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd32ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.253+0000 7f2797400640 1 -- 172.21.15.2:0/3746713886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd1aca0 msgr2=0x55ecdfd19120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:10.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.253+0000 7f2797400640 1 --2- 172.21.15.2:0/3746713886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd19120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.253+0000 7f2797400640 1 -- 172.21.15.2:0/3746713886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 msgr2=0x55ecdfd32ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:10.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.253+0000 7f2797400640 1 --2- 172.21.15.2:0/3746713886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd32ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.253+0000 7f2797400640 1 -- 172.21.15.2:0/3746713886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecdfd2d710 con 0x55ecdfd0fac0 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.253+0000 7f2797400640 1 --2- 172.21.15.2:0/3746713886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd329b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f27880057c0 tx=0x7f2788004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae54ba95c03b0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.253+0000 7f2787400640 1 -- 172.21.15.2:0/3746713886 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278804e070 con 0x55ecdfd0fac0 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.253+0000 7f2787400640 1 -- 172.21.15.2:0/3746713886 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27880035f0 con 0x55ecdfd0fac0 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.253+0000 7f2787400640 1 -- 172.21.15.2:0/3746713886 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788035040 con 0x55ecdfd0fac0 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.254+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3746713886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 msgr2=0x55ecdfd329b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.254+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3746713886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd329b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f27880057c0 tx=0x7f2788004490 comp rx=0 tx=0).stop 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.254+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3746713886 shutdown_connections 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.254+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3746713886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd19120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.254+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3746713886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd32ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.254+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3746713886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd329b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.254+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3746713886 >> 172.21.15.2:0/3746713886 conn(0x55ecdfd0c050 msgr2=0x55ecdfd16190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.254+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3746713886 shutdown_connections 2024-09-10T08:15:10.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.254+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3746713886 wait complete. 2024-09-10T08:15:10.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.254+0000 7f27a0cb0740 1 -- start start 2024-09-10T08:15:10.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.255+0000 7f27a0cb0740 1 Processor -- start 2024-09-10T08:15:10.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.255+0000 7f27a0cb0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd23050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.255+0000 7f27a0cb0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd23590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.255+0000 7f27a0cb0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd1e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.255+0000 7f27a0cb0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f27981af720 con 0x55ecdfd1aca0 2024-09-10T08:15:10.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.255+0000 7f27a0cb0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f27981b0b90 con 0x55ecdfd103e0 2024-09-10T08:15:10.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.255+0000 7f27a0cb0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f27981afdf0 con 0x55ecdfd0fac0 2024-09-10T08:15:10.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.255+0000 7f27a0cb0740 10 client.0 _ll_register_callbacks cb 0x55ecdfd18820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:10.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f2797400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd23050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f278ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd23590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f278ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd23590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39590/0 (socket says 172.21.15.2:39590) 2024-09-10T08:15:10.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f2797e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd1e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f278ea00640 1 -- 172.21.15.2:0/3585342414 learned_addr learned my addr 172.21.15.2:0/3585342414 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:10.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f2797400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd23050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:32922/0 (socket says 172.21.15.2:32922) 2024-09-10T08:15:10.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f278ea00640 1 -- 172.21.15.2:0/3585342414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 msgr2=0x55ecdfd23050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:10.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f278ea00640 1 --2- 172.21.15.2:0/3585342414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd23050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f278ea00640 1 -- 172.21.15.2:0/3585342414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd1aca0 msgr2=0x55ecdfd1e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:10.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f278ea00640 1 --2- 172.21.15.2:0/3585342414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd1e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f278ea00640 1 -- 172.21.15.2:0/3585342414 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecdfd1ee00 con 0x55ecdfd103e0 2024-09-10T08:15:10.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f2797400640 1 --2- 172.21.15.2:0/3585342414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd23050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:10.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f278ea00640 1 --2- 172.21.15.2:0/3585342414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd23590 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2780009610 tx=0x7f2780000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ef07bf100f0880d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:10.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.256+0000 7f2784c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:15:10.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f2784c00640 1 -- 172.21.15.2:0/3585342414 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278009b1f0 con 0x55ecdfd103e0 2024-09-10T08:15:10.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f2784c00640 1 -- 172.21.15.2:0/3585342414 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f27800844e0 con 0x55ecdfd103e0 2024-09-10T08:15:10.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3585342414 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27880035f0 con 0x55ecdfd103e0 2024-09-10T08:15:10.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f2784c00640 1 -- 172.21.15.2:0/3585342414 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f278009f5a0 con 0x55ecdfd103e0 2024-09-10T08:15:10.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f27a0cb0740 10 client.27383 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:10.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3585342414 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ecdfd95600 con 0x55ecdfd103e0 2024-09-10T08:15:10.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f2784c00640 1 -- 172.21.15.2:0/3585342414 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27880035f0 con 0x55ecdfd103e0 2024-09-10T08:15:10.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f2784c00640 1 -- 172.21.15.2:0/3585342414 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f278009f170 con 0x55ecdfd103e0 2024-09-10T08:15:10.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f27a0cb0740 10 client.27383 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:10.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f27a0cb0740 10 client.27383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:10.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f27a0cb0740 20 client.27383 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:10.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3585342414 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ecdfd95d30 0x55ecdfd98150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:10.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f27a0cb0740 4 client.27383 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:15:10.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.257+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3585342414 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ecdfc7dd10 con 0x55ecdfd95d30 2024-09-10T08:15:10.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.258+0000 7f2797400640 1 --2- 172.21.15.2:0/3585342414 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ecdfd95d30 0x55ecdfd98150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:10.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.258+0000 7f2797400640 1 --2- 172.21.15.2:0/3585342414 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ecdfd95d30 0x55ecdfd98150 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d27ac41ba9ef73b server_cookie=64a9891cda2ce73c in_seq=0 out_seq=0 2024-09-10T08:15:10.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.258+0000 7f2784c00640 10 client.27383 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:10.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.259+0000 7f2784c00640 1 -- 172.21.15.2:0/3585342414 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ecdfc7dd10 con 0x55ecdfd95d30 2024-09-10T08:15:10.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.259+0000 7f2784c00640 10 client.27383 handle_command_reply: tid=0 2024-09-10T08:15:10.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.259+0000 7f27a0cb0740 1 client.27383 shutdown 2024-09-10T08:15:10.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.259+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3585342414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 msgr2=0x55ecdfd23590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:10.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.259+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3585342414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd23590 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2780009610 tx=0x7f2780000bc0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.260+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3585342414 shutdown_connections 2024-09-10T08:15:10.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.260+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3585342414 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ecdfd95d30 0x55ecdfd98150 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.260+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3585342414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ecdfd1aca0 0x55ecdfd1e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.260+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3585342414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecdfd103e0 0x55ecdfd23590 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.260+0000 7f2784c00640 0 client.27383 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:10.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.260+0000 7f27a0cb0740 1 --2- 172.21.15.2:0/3585342414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecdfd0fac0 0x55ecdfd23050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.260+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3585342414 >> 172.21.15.2:0/3585342414 conn(0x55ecdfd0c050 msgr2=0x55ecdfd19940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:10.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.260+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3585342414 shutdown_connections 2024-09-10T08:15:10.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.260+0000 7f27a0cb0740 1 -- 172.21.15.2:0/3585342414 wait complete. 2024-09-10T08:15:10.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.260+0000 7f27a0cb0740 20 client.27383 trim_cache size 0 max 16384 2024-09-10T08:15:10.263 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:15:10.263 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:15:10.263 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "c4f72832-eda3-4d14-9f77-5d75fd3cadb0", 2024-09-10T08:15:10.263 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:15:10.263 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:15:10.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.261+0000 7f279e200640 1 -- 172.21.15.2:0/3826266897 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f277c0a2b80 msgr2=0x7f277c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:10.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.261+0000 7f279e200640 1 --2- 172.21.15.2:0/3826266897 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f277c0a2b80 0x7f277c0a5040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2788000c00 tx=0x7f2788005b50 comp rx=0 tx=0).stop 2024-09-10T08:15:10.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.261+0000 7f279e200640 1 -- 172.21.15.2:0/3826266897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2798102790 msgr2=0x7f279819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:10.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.261+0000 7f279e200640 1 --2- 172.21.15.2:0/3826266897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2798102790 0x7f279819abb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f278000d950 tx=0x7f278000de10 comp rx=0 tx=0).stop 2024-09-10T08:15:10.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.261+0000 7f279e200640 1 -- 172.21.15.2:0/3826266897 shutdown_connections 2024-09-10T08:15:10.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.261+0000 7f279e200640 1 --2- 172.21.15.2:0/3826266897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798106f10 0x7f279819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.261+0000 7f279e200640 1 --2- 172.21.15.2:0/3826266897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2798102790 0x7f279819abb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.261+0000 7f279e200640 1 --2- 172.21.15.2:0/3826266897 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f277c0a2b80 0x7f277c0a5040 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.261+0000 7f279e200640 1 --2- 172.21.15.2:0/3826266897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2798101de0 0x7f279819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:10.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.261+0000 7f279e200640 1 -- 172.21.15.2:0/3826266897 >> 172.21.15.2:0/3826266897 conn(0x7f27980fd8c0 msgr2=0x7f27980feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:10.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.261+0000 7f279e200640 1 -- 172.21.15.2:0/3826266897 shutdown_connections 2024-09-10T08:15:10.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:10.261+0000 7f279e200640 1 -- 172.21.15.2:0/3826266897 wait complete. 2024-09-10T08:15:11.161 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:10 smithi002 ceph-mon[30563]: pgmap v598: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.6 KiB/s wr, 9 op/s 2024-09-10T08:15:11.161 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:10 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:15:11.161 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:10 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:15:11.161 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:10 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:15:11.161 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:10 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:15:11.199 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:15:11.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:10 smithi155 ceph-mon[35405]: pgmap v598: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.6 KiB/s wr, 9 op/s 2024-09-10T08:15:11.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:10 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:15:11.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:10 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:15:11.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:10 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:15:11.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:10 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:15:11.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:10 smithi107 ceph-mon[35048]: pgmap v598: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.6 KiB/s wr, 9 op/s 2024-09-10T08:15:11.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:10 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:15:11.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:10 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:15:11.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:10 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:15:11.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:10 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:15:11.580 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:12.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.151+0000 7fc766e00640 1 -- 172.21.15.2:0/2757077792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc760076720 msgr2=0x7fc760076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:12.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.151+0000 7fc766e00640 1 --2- 172.21.15.2:0/2757077792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc760076720 0x7fc760076b00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc750009900 tx=0x7fc75002a410 comp rx=0 tx=0).stop 2024-09-10T08:15:12.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.151+0000 7fc766e00640 1 -- 172.21.15.2:0/2757077792 shutdown_connections 2024-09-10T08:15:12.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.151+0000 7fc766e00640 1 --2- 172.21.15.2:0/2757077792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc760112a30 0x7fc760114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.151+0000 7fc766e00640 1 --2- 172.21.15.2:0/2757077792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc760077040 0x7fc7601124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.151+0000 7fc766e00640 1 --2- 172.21.15.2:0/2757077792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc760076720 0x7fc760076b00 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.151+0000 7fc766e00640 1 -- 172.21.15.2:0/2757077792 >> 172.21.15.2:0/2757077792 conn(0x7fc7600ffa90 msgr2=0x7fc760101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:12.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.151+0000 7fc766e00640 1 -- 172.21.15.2:0/2757077792 shutdown_connections 2024-09-10T08:15:12.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.152+0000 7fc766e00640 1 -- 172.21.15.2:0/2757077792 wait complete. 2024-09-10T08:15:12.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.152+0000 7fc766e00640 1 Processor -- start 2024-09-10T08:15:12.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.153+0000 7fc766e00640 1 -- start start 2024-09-10T08:15:12.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.153+0000 7fc766e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc760076720 0x7fc7601a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.153+0000 7fc766e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc760077040 0x7fc7601a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.153+0000 7fc766e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc760112a30 0x7fc76019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.153+0000 7fc766e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc7601185c0 con 0x7fc760076720 2024-09-10T08:15:12.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.153+0000 7fc766e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc760118440 con 0x7fc760077040 2024-09-10T08:15:12.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.153+0000 7fc766e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc760118740 con 0x7fc760112a30 2024-09-10T08:15:12.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.153+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc760076720 0x7fc7601a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.153+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc760076720 0x7fc7601a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42938/0 (socket says 172.21.15.2:42938) 2024-09-10T08:15:12.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.153+0000 7fc75fe00640 1 -- 172.21.15.2:0/2306644061 learned_addr learned my addr 172.21.15.2:0/2306644061 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:12.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.153+0000 7fc764e00640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc760112a30 0x7fc76019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.153+0000 7fc757400640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc760077040 0x7fc7601a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.154+0000 7fc75fe00640 1 -- 172.21.15.2:0/2306644061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc760112a30 msgr2=0x7fc76019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:12.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.154+0000 7fc75fe00640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc760112a30 0x7fc76019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.154+0000 7fc75fe00640 1 -- 172.21.15.2:0/2306644061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc760077040 msgr2=0x7fc7601a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:12.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.154+0000 7fc75fe00640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc760077040 0x7fc7601a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.154+0000 7fc75fe00640 1 -- 172.21.15.2:0/2306644061 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76019efc0 con 0x7fc760076720 2024-09-10T08:15:12.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.154+0000 7fc764e00640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc760112a30 0x7fc76019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:12.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.154+0000 7fc75fe00640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc760076720 0x7fc7601a46d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc750006010 tx=0x7fc750004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:12.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.154+0000 7fc75d600640 1 -- 172.21.15.2:0/2306644061 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75003f030 con 0x7fc760076720 2024-09-10T08:15:12.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.154+0000 7fc75d600640 1 -- 172.21.15.2:0/2306644061 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc750004440 con 0x7fc760076720 2024-09-10T08:15:12.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.154+0000 7fc766e00640 1 -- 172.21.15.2:0/2306644061 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76019f250 con 0x7fc760076720 2024-09-10T08:15:12.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.155+0000 7fc75d600640 1 -- 172.21.15.2:0/2306644061 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750035070 con 0x7fc760076720 2024-09-10T08:15:12.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.155+0000 7fc766e00640 1 -- 172.21.15.2:0/2306644061 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7601ab5a0 con 0x7fc760076720 2024-09-10T08:15:12.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.158+0000 7fc769817740 1 Processor -- start 2024-09-10T08:15:12.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.158+0000 7fc75d600640 1 -- 172.21.15.2:0/2306644061 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc750042030 con 0x7fc760076720 2024-09-10T08:15:12.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.158+0000 7fc769817740 1 -- start start 2024-09-10T08:15:12.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.158+0000 7fc769817740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bbb90 0x55fca83bbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.158+0000 7fc769817740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.158+0000 7fc769817740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83c6d70 0x55fca83c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.158+0000 7fc769817740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fca83223e0 con 0x55fca83c6d70 2024-09-10T08:15:12.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.158+0000 7fc769817740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fca82c9d80 con 0x55fca83bc4b0 2024-09-10T08:15:12.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.158+0000 7fc769817740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fca831e3a0 con 0x55fca83bbb90 2024-09-10T08:15:12.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.159+0000 7fc764e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bbb90 0x55fca83bbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.159+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.159+0000 7fc764e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bbb90 0x55fca83bbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:32936/0 (socket says 172.21.15.2:32936) 2024-09-10T08:15:12.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.159+0000 7fc757400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83c6d70 0x55fca83c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.159+0000 7fc764e00640 1 -- 172.21.15.2:0/2873751568 learned_addr learned my addr 172.21.15.2:0/2873751568 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:12.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.159+0000 7fc764e00640 1 -- 172.21.15.2:0/2873751568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 msgr2=0x55fca83c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:12.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.159+0000 7fc764e00640 1 --2- 172.21.15.2:0/2873751568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.159+0000 7fc764e00640 1 -- 172.21.15.2:0/2873751568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83c6d70 msgr2=0x55fca83c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.159+0000 7fc764e00640 1 --2- 172.21.15.2:0/2873751568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83c6d70 0x55fca83c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.159+0000 7fc764e00640 1 -- 172.21.15.2:0/2873751568 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca82e8ae0 con 0x55fca83bbb90 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.160+0000 7fc764e00640 1 --2- 172.21.15.2:0/2873751568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bbb90 0x55fca83bbf70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc758011790 tx=0x7fc758011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba5c33920d0d6a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.160+0000 7fc755600640 1 -- 172.21.15.2:0/2873751568 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758003c30 con 0x55fca83bbb90 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.160+0000 7fc755600640 1 -- 172.21.15.2:0/2873751568 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc758003dd0 con 0x55fca83bbb90 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.160+0000 7fc755600640 1 -- 172.21.15.2:0/2873751568 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7580193e0 con 0x55fca83bbb90 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.160+0000 7fc75d600640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7440a2b80 0x7fc7440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.160+0000 7fc75d600640 1 -- 172.21.15.2:0/2306644061 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc7500ef7e0 con 0x7fc760076720 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.160+0000 7fc75fe00640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7440a2b80 0x7fc7440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.161+0000 7fc769817740 1 -- 172.21.15.2:0/2873751568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bbb90 msgr2=0x55fca83bbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.161+0000 7fc769817740 1 --2- 172.21.15.2:0/2873751568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bbb90 0x55fca83bbf70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc758011790 tx=0x7fc758011c50 comp rx=0 tx=0).stop 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.161+0000 7fc769817740 1 -- 172.21.15.2:0/2873751568 shutdown_connections 2024-09-10T08:15:12.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.161+0000 7fc769817740 1 --2- 172.21.15.2:0/2873751568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83c6d70 0x55fca83c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.161+0000 7fc769817740 1 --2- 172.21.15.2:0/2873751568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.161+0000 7fc769817740 1 --2- 172.21.15.2:0/2873751568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bbb90 0x55fca83bbf70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.161+0000 7fc769817740 1 -- 172.21.15.2:0/2873751568 >> 172.21.15.2:0/2873751568 conn(0x55fca83b8120 msgr2=0x55fca83b8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:12.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.161+0000 7fc769817740 1 -- 172.21.15.2:0/2873751568 shutdown_connections 2024-09-10T08:15:12.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.161+0000 7fc769817740 1 -- 172.21.15.2:0/2873751568 wait complete. 2024-09-10T08:15:12.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.161+0000 7fc75fe00640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7440a2b80 0x7fc7440a5040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc7500efa80 tx=0x7fc7500f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:12.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.161+0000 7fc769817740 1 -- start start 2024-09-10T08:15:12.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.162+0000 7fc769817740 1 Processor -- start 2024-09-10T08:15:12.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.162+0000 7fc769817740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bbb90 0x55fca83c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.162+0000 7fc769817740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.162+0000 7fc769817740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83c6d70 0x55fca83c4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.162+0000 7fc769817740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fca83c99e0 con 0x55fca83c6d70 2024-09-10T08:15:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.162+0000 7fc769817740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fca82dd0f0 con 0x55fca83bc4b0 2024-09-10T08:15:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.162+0000 7fc769817740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fca83c9b60 con 0x55fca83bbb90 2024-09-10T08:15:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.162+0000 7fc769817740 10 client.0 _ll_register_callbacks cb 0x55fca82ce9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.162+0000 7fc757400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc764e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bbb90 0x55fca83c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc757400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39630/0 (socket says 172.21.15.2:39630) 2024-09-10T08:15:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc757400640 1 -- 172.21.15.2:0/912037709 learned_addr learned my addr 172.21.15.2:0/912037709 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc764e00640 1 --2- 172.21.15.2:0/912037709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83c6d70 0x55fca83c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc757400640 1 -- 172.21.15.2:0/912037709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bbb90 msgr2=0x55fca83c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc757400640 1 --2- 172.21.15.2:0/912037709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bbb90 0x55fca83c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc757400640 1 -- 172.21.15.2:0/912037709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83c6d70 msgr2=0x55fca83c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc757400640 1 --2- 172.21.15.2:0/912037709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83c6d70 0x55fca83c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc757400640 1 -- 172.21.15.2:0/912037709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca80933c0 con 0x55fca83bc4b0 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc764e00640 1 --2- 172.21.15.2:0/912037709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bbb90 0x55fca83c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc757400640 1 --2- 172.21.15.2:0/912037709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c41d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc748012730 tx=0x7fc748012bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=212dabf4f6327781 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc74e000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc74e000640 1 -- 172.21.15.2:0/912037709 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74800f920 con 0x55fca83bc4b0 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.163+0000 7fc769817740 1 -- 172.21.15.2:0/912037709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc758003dd0 con 0x55fca83bc4b0 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.164+0000 7fc74e000640 1 -- 172.21.15.2:0/912037709 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc74800fac0 con 0x55fca83bc4b0 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.164+0000 7fc74e000640 1 -- 172.21.15.2:0/912037709 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc74800fcc0 con 0x55fca83bc4b0 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.164+0000 7fc769817740 10 client.27395 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.164+0000 7fc769817740 1 -- 172.21.15.2:0/912037709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fca83d57a0 con 0x55fca83bc4b0 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.164+0000 7fc74e000640 1 -- 172.21.15.2:0/912037709 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc758003dd0 con 0x55fca83bc4b0 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.164+0000 7fc74e000640 1 -- 172.21.15.2:0/912037709 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc74801c420 con 0x55fca83bc4b0 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.164+0000 7fc769817740 10 client.27395 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.164+0000 7fc769817740 10 client.27395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:12.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.164+0000 7fc769817740 20 client.27395 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:12.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.164+0000 7fc769817740 1 --2- 172.21.15.2:0/912037709 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fca83d6c50 0x55fca83d9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.164+0000 7fc769817740 4 client.27395 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:12.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.164+0000 7fc769817740 1 -- 172.21.15.2:0/912037709 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fca832f950 con 0x55fca83d6c50 2024-09-10T08:15:12.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.165+0000 7fc764e00640 1 --2- 172.21.15.2:0/912037709 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fca83d6c50 0x55fca83d9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.165+0000 7fc764e00640 1 --2- 172.21.15.2:0/912037709 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fca83d6c50 0x55fca83d9070 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7db144a80b96af0 server_cookie=2d0ea86e066dab50 in_seq=0 out_seq=0 2024-09-10T08:15:12.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.165+0000 7fc74e000640 10 client.27395 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:12.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.166+0000 7fc74e000640 1 -- 172.21.15.2:0/912037709 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fca832f950 con 0x55fca83d6c50 2024-09-10T08:15:12.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.166+0000 7fc74e000640 10 client.27395 handle_command_reply: tid=0 2024-09-10T08:15:12.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.166+0000 7fc769817740 1 client.27395 shutdown 2024-09-10T08:15:12.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.166+0000 7fc769817740 1 -- 172.21.15.2:0/912037709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 msgr2=0x55fca83c41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:12.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.166+0000 7fc769817740 1 --2- 172.21.15.2:0/912037709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c41d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc748012730 tx=0x7fc748012bf0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.166+0000 7fc769817740 1 -- 172.21.15.2:0/912037709 shutdown_connections 2024-09-10T08:15:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.166+0000 7fc769817740 1 --2- 172.21.15.2:0/912037709 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fca83d6c50 0x55fca83d9070 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.166+0000 7fc769817740 1 --2- 172.21.15.2:0/912037709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83c6d70 0x55fca83c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.166+0000 7fc769817740 1 --2- 172.21.15.2:0/912037709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c41d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.166+0000 7fc769817740 1 --2- 172.21.15.2:0/912037709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bbb90 0x55fca83c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.166+0000 7fc74e000640 0 client.27395 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.166+0000 7fc769817740 1 -- 172.21.15.2:0/912037709 >> 172.21.15.2:0/912037709 conn(0x55fca83b8120 msgr2=0x55fca83bde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.167+0000 7fc769817740 1 -- 172.21.15.2:0/912037709 shutdown_connections 2024-09-10T08:15:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.167+0000 7fc769817740 1 -- 172.21.15.2:0/912037709 wait complete. 2024-09-10T08:15:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.167+0000 7fc769817740 20 client.27395 trim_cache size 0 max 16384 2024-09-10T08:15:12.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.185+0000 7fc769817740 1 Processor -- start 2024-09-10T08:15:12.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.185+0000 7fc769817740 1 -- start start 2024-09-10T08:15:12.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.185+0000 7fc769817740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 0x55fca83de920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.185+0000 7fc769817740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bc4b0 0x55fca83dee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.185+0000 7fc769817740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83c6d70 0x55fca83c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.185+0000 7fc769817740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc7601b7ce0 con 0x55fca83bbb90 2024-09-10T08:15:12.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.185+0000 7fc769817740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc7601b55b0 con 0x55fca83c6d70 2024-09-10T08:15:12.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.185+0000 7fc769817740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7601b3400 con 0x55fca83bc4b0 2024-09-10T08:15:12.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.185+0000 7fc764e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 0x55fca83de920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.185+0000 7fc764e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 0x55fca83de920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42954/0 (socket says 172.21.15.2:42954) 2024-09-10T08:15:12.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.185+0000 7fc764e00640 1 -- 172.21.15.2:0/1223298250 learned_addr learned my addr 172.21.15.2:0/1223298250 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:12.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.185+0000 7fc757400640 1 --2- 172.21.15.2:0/1223298250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bc4b0 0x55fca83dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.186+0000 7fc764e00640 1 --2- 172.21.15.2:0/1223298250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83c6d70 0x55fca83c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.186+0000 7fc764e00640 1 -- 172.21.15.2:0/1223298250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bc4b0 msgr2=0x55fca83dee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:12.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.186+0000 7fc764e00640 1 --2- 172.21.15.2:0/1223298250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bc4b0 0x55fca83dee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.186+0000 7fc764e00640 1 -- 172.21.15.2:0/1223298250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83c6d70 msgr2=0x55fca83c0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:12.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.186+0000 7fc764e00640 1 --2- 172.21.15.2:0/1223298250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83c6d70 0x55fca83c0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.186+0000 7fc764e00640 1 -- 172.21.15.2:0/1223298250 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca83b8db0 con 0x55fca83bbb90 2024-09-10T08:15:12.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.186+0000 7fc764e00640 1 --2- 172.21.15.2:0/1223298250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 0x55fca83de920 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc758011730 tx=0x7fc758009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd940cbeaf548ded server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:12.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.186+0000 7fc754c00640 1 -- 172.21.15.2:0/1223298250 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758004270 con 0x55fca83bbb90 2024-09-10T08:15:12.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.186+0000 7fc754c00640 1 -- 172.21.15.2:0/1223298250 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75800f920 con 0x55fca83bbb90 2024-09-10T08:15:12.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.186+0000 7fc754c00640 1 -- 172.21.15.2:0/1223298250 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758004a10 con 0x55fca83bbb90 2024-09-10T08:15:12.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.187+0000 7fc769817740 1 -- 172.21.15.2:0/1223298250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 msgr2=0x55fca83de920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:12.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.187+0000 7fc769817740 1 --2- 172.21.15.2:0/1223298250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 0x55fca83de920 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc758011730 tx=0x7fc758009500 comp rx=0 tx=0).stop 2024-09-10T08:15:12.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.187+0000 7fc769817740 1 -- 172.21.15.2:0/1223298250 shutdown_connections 2024-09-10T08:15:12.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.187+0000 7fc769817740 1 --2- 172.21.15.2:0/1223298250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83c6d70 0x55fca83c0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.187+0000 7fc769817740 1 --2- 172.21.15.2:0/1223298250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83bc4b0 0x55fca83dee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.187+0000 7fc769817740 1 --2- 172.21.15.2:0/1223298250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 0x55fca83de920 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.187+0000 7fc769817740 1 -- 172.21.15.2:0/1223298250 >> 172.21.15.2:0/1223298250 conn(0x55fca83b8120 msgr2=0x55fca83bde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:12.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.187+0000 7fc769817740 1 -- 172.21.15.2:0/1223298250 shutdown_connections 2024-09-10T08:15:12.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.187+0000 7fc769817740 1 -- 172.21.15.2:0/1223298250 wait complete. 2024-09-10T08:15:12.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.188+0000 7fc769817740 1 -- start start 2024-09-10T08:15:12.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.188+0000 7fc769817740 1 Processor -- start 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.188+0000 7fc769817740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 0x55fca83cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.188+0000 7fc769817740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.188+0000 7fc769817740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83c6d70 0x55fca83c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.188+0000 7fc769817740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc7601b3a70 con 0x55fca83bbb90 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.188+0000 7fc769817740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc7601b4ee0 con 0x55fca83bc4b0 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.188+0000 7fc769817740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7601b4140 con 0x55fca83c6d70 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.188+0000 7fc769817740 10 client.0 _ll_register_callbacks cb 0x7fc748009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.189+0000 7fc764e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 0x55fca83cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.189+0000 7fc764e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 0x55fca83cf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42968/0 (socket says 172.21.15.2:42968) 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.189+0000 7fc764e00640 1 -- 172.21.15.2:0/43404441 learned_addr learned my addr 172.21.15.2:0/43404441 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.189+0000 7fc757400640 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.189+0000 7fc764e00640 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83c6d70 0x55fca83c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.189+0000 7fc764e00640 1 -- 172.21.15.2:0/43404441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83c6d70 msgr2=0x55fca83c0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:12.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.189+0000 7fc764e00640 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83c6d70 0x55fca83c0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.189+0000 7fc764e00640 1 -- 172.21.15.2:0/43404441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 msgr2=0x55fca83c0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:12.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.189+0000 7fc764e00640 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.189+0000 7fc764e00640 1 -- 172.21.15.2:0/43404441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca83c1430 con 0x55fca83bbb90 2024-09-10T08:15:12.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.189+0000 7fc757400640 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:12.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.190+0000 7fc764e00640 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 0x55fca83cf3b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc758011c90 tx=0x7fc758004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20d8d8bc75c41f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:12.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.190+0000 7fc74e000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:12.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.190+0000 7fc74e000640 1 -- 172.21.15.2:0/43404441 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75800fb40 con 0x55fca83bbb90 2024-09-10T08:15:12.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.190+0000 7fc74e000640 1 -- 172.21.15.2:0/43404441 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc75800fce0 con 0x55fca83bbb90 2024-09-10T08:15:12.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.190+0000 7fc769817740 1 -- 172.21.15.2:0/43404441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc75800f920 con 0x55fca83bbb90 2024-09-10T08:15:12.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.190+0000 7fc74e000640 1 -- 172.21.15.2:0/43404441 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc758041050 con 0x55fca83bbb90 2024-09-10T08:15:12.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.190+0000 7fc769817740 10 client.17364 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:12.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.190+0000 7fc769817740 1 -- 172.21.15.2:0/43404441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fca8441790 con 0x55fca83bbb90 2024-09-10T08:15:12.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.190+0000 7fc74e000640 1 -- 172.21.15.2:0/43404441 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75800f920 con 0x55fca83bbb90 2024-09-10T08:15:12.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.191+0000 7fc74e000640 1 -- 172.21.15.2:0/43404441 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc758018070 con 0x55fca83bbb90 2024-09-10T08:15:12.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.191+0000 7fc769817740 10 client.17364 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:12.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.191+0000 7fc769817740 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:12.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.191+0000 7fc769817740 20 client.17364 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:12.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.191+0000 7fc769817740 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fca8441c10 0x55fca8444030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:12.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.191+0000 7fc769817740 4 client.17364 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:15:12.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.191+0000 7fc769817740 1 -- 172.21.15.2:0/43404441 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fca8329e60 con 0x55fca8441c10 2024-09-10T08:15:12.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.191+0000 7fc757400640 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fca8441c10 0x55fca8444030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:12.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc757400640 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fca8441c10 0x55fca8444030 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db184c374409dfc5 server_cookie=ad192b08ea4eca59 in_seq=0 out_seq=0 2024-09-10T08:15:12.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc74e000640 10 client.17364 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc74e000640 1 -- 172.21.15.2:0/43404441 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fca8329e60 con 0x55fca8441c10 2024-09-10T08:15:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc74e000640 10 client.17364 handle_command_reply: tid=0 2024-09-10T08:15:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc769817740 1 client.17364 shutdown 2024-09-10T08:15:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc769817740 1 -- 172.21.15.2:0/43404441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 msgr2=0x55fca83cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc769817740 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 0x55fca83cf3b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc758011c90 tx=0x7fc758004050 comp rx=0 tx=0).stop 2024-09-10T08:15:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc769817740 1 -- 172.21.15.2:0/43404441 shutdown_connections 2024-09-10T08:15:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc769817740 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fca8441c10 0x55fca8444030 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc769817740 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fca83c6d70 0x55fca83c0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc769817740 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fca83bc4b0 0x55fca83c0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc769817740 1 --2- 172.21.15.2:0/43404441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fca83bbb90 0x55fca83cf3b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc769817740 1 -- 172.21.15.2:0/43404441 >> 172.21.15.2:0/43404441 conn(0x55fca83b8120 msgr2=0x55fca83c2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc74e000640 0 client.17364 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc769817740 1 -- 172.21.15.2:0/43404441 shutdown_connections 2024-09-10T08:15:12.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.192+0000 7fc769817740 1 -- 172.21.15.2:0/43404441 wait complete. 2024-09-10T08:15:12.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.193+0000 7fc769817740 20 client.17364 trim_cache size 0 max 16384 2024-09-10T08:15:12.196 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:15:12.196 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:15:12.196 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:15:12.196 INFO:teuthology.orchestra.run.smithi002.stdout: "c4f72832-eda3-4d14-9f77-5d75fd3cadb0": { 2024-09-10T08:15:12.196 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:15:12.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "c4f72832-eda3-4d14-9f77-5d75fd3cadb0", 2024-09-10T08:15:12.196 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:15:12.196 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:15:12.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:15:12.197 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:15:12.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.193+0000 7fc766e00640 1 -- 172.21.15.2:0/2306644061 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7440a2b80 msgr2=0x7fc7440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:12.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.193+0000 7fc766e00640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7440a2b80 0x7fc7440a5040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc7500efa80 tx=0x7fc7500f5530 comp rx=0 tx=0).stop 2024-09-10T08:15:12.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.193+0000 7fc766e00640 1 -- 172.21.15.2:0/2306644061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc760076720 msgr2=0x7fc7601a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:12.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.193+0000 7fc766e00640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc760076720 0x7fc7601a46d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc750006010 tx=0x7fc750004040 comp rx=0 tx=0).stop 2024-09-10T08:15:12.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.193+0000 7fc766e00640 1 -- 172.21.15.2:0/2306644061 shutdown_connections 2024-09-10T08:15:12.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.193+0000 7fc766e00640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7440a2b80 0x7fc7440a5040 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.193+0000 7fc766e00640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc760112a30 0x7fc76019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.193+0000 7fc766e00640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc760077040 0x7fc7601a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.193+0000 7fc766e00640 1 --2- 172.21.15.2:0/2306644061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc760076720 0x7fc7601a46d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:12.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.193+0000 7fc766e00640 1 -- 172.21.15.2:0/2306644061 >> 172.21.15.2:0/2306644061 conn(0x7fc7600ffa90 msgr2=0x7fc760101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:12.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.194+0000 7fc766e00640 1 -- 172.21.15.2:0/2306644061 shutdown_connections 2024-09-10T08:15:12.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:12.194+0000 7fc766e00640 1 -- 172.21.15.2:0/2306644061 wait complete. 2024-09-10T08:15:13.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:12 smithi002 ceph-mon[30563]: pgmap v599: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-09-10T08:15:13.260 INFO:tasks.cephfs.filesystem:scrub status for tag:c4f72832-eda3-4d14-9f77-5d75fd3cadb0 - {'path': '/', 'tag': 'c4f72832-eda3-4d14-9f77-5d75fd3cadb0', 'options': 'recursive,force'} 2024-09-10T08:15:13.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:12 smithi155 ceph-mon[35405]: pgmap v599: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-09-10T08:15:13.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:12 smithi107 ceph-mon[35048]: pgmap v599: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-09-10T08:15:14.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:13 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:15:14.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:13 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:15:14.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:13 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:15:15.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:15 smithi002 ceph-mon[30563]: pgmap v600: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T08:15:15.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:15 smithi155 ceph-mon[35405]: pgmap v600: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T08:15:15.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:15 smithi107 ceph-mon[35048]: pgmap v600: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T08:15:15.518 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:15:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:16.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.436+0000 7f8e55600640 1 -- 172.21.15.2:0/1465861588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e50103ff0 msgr2=0x7f8e501043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:16.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.436+0000 7f8e55600640 1 --2- 172.21.15.2:0/1465861588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e50103ff0 0x7f8e501043d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009900 tx=0x7f8e4002a410 comp rx=0 tx=0).stop 2024-09-10T08:15:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.436+0000 7f8e55600640 1 -- 172.21.15.2:0/1465861588 shutdown_connections 2024-09-10T08:15:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.436+0000 7f8e55600640 1 --2- 172.21.15.2:0/1465861588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e50109120 0x7f8e50110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.436+0000 7f8e55600640 1 --2- 172.21.15.2:0/1465861588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e501049a0 0x7f8e501089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.436+0000 7f8e55600640 1 --2- 172.21.15.2:0/1465861588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e50103ff0 0x7f8e501043d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.436+0000 7f8e55600640 1 -- 172.21.15.2:0/1465861588 >> 172.21.15.2:0/1465861588 conn(0x7f8e500ffa90 msgr2=0x7f8e50101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.436+0000 7f8e55600640 1 -- 172.21.15.2:0/1465861588 shutdown_connections 2024-09-10T08:15:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.436+0000 7f8e55600640 1 -- 172.21.15.2:0/1465861588 wait complete. 2024-09-10T08:15:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.438+0000 7f8e55600640 1 Processor -- start 2024-09-10T08:15:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.438+0000 7f8e55600640 1 -- start start 2024-09-10T08:15:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.438+0000 7f8e55600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e50103ff0 0x7f8e5019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.438+0000 7f8e55600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e501049a0 0x7f8e5019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.438+0000 7f8e55600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e50109120 0x7f8e5019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.438+0000 7f8e55600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8e50114150 con 0x7f8e50109120 2024-09-10T08:15:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.438+0000 7f8e55600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8e50113fd0 con 0x7f8e50103ff0 2024-09-10T08:15:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.438+0000 7f8e55600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501142d0 con 0x7f8e501049a0 2024-09-10T08:15:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.439+0000 7f8e4e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e501049a0 0x7f8e5019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.439+0000 7f8e4f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e50109120 0x7f8e5019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.439+0000 7f8e4e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e501049a0 0x7f8e5019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:32960/0 (socket says 172.21.15.2:32960) 2024-09-10T08:15:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.439+0000 7f8e4e000640 1 -- 172.21.15.2:0/4113695083 learned_addr learned my addr 172.21.15.2:0/4113695083 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.439+0000 7f8e4ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e50103ff0 0x7f8e5019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.439+0000 7f8e4e000640 1 -- 172.21.15.2:0/4113695083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e50103ff0 msgr2=0x7f8e5019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.439+0000 7f8e4e000640 1 --2- 172.21.15.2:0/4113695083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e50103ff0 0x7f8e5019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:16.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.439+0000 7f8e4e000640 1 -- 172.21.15.2:0/4113695083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e50109120 msgr2=0x7f8e5019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.439+0000 7f8e4e000640 1 --2- 172.21.15.2:0/4113695083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e50109120 0x7f8e5019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.439+0000 7f8e4e000640 1 -- 172.21.15.2:0/4113695083 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5019f5a0 con 0x7f8e501049a0 2024-09-10T08:15:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.439+0000 7f8e4f400640 1 --2- 172.21.15.2:0/4113695083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e50109120 0x7f8e5019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.439+0000 7f8e4e000640 1 --2- 172.21.15.2:0/4113695083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e501049a0 0x7f8e5019ab30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00e9f0 tx=0x7f8e3c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.440+0000 7f8e47400640 1 -- 172.21.15.2:0/4113695083 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c00cd80 con 0x7f8e501049a0 2024-09-10T08:15:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.440+0000 7f8e47400640 1 -- 172.21.15.2:0/4113695083 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e3c010910 con 0x7f8e501049a0 2024-09-10T08:15:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.440+0000 7f8e55600640 1 -- 172.21.15.2:0/4113695083 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5019f890 con 0x7f8e501049a0 2024-09-10T08:15:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.440+0000 7f8e47400640 1 -- 172.21.15.2:0/4113695083 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c018990 con 0x7f8e501049a0 2024-09-10T08:15:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.440+0000 7f8e55600640 1 -- 172.21.15.2:0/4113695083 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e501a7170 con 0x7f8e501049a0 2024-09-10T08:15:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.441+0000 7f8e55600640 1 -- 172.21.15.2:0/4113695083 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e501043d0 con 0x7f8e501049a0 2024-09-10T08:15:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.444+0000 7f8e47400640 1 -- 172.21.15.2:0/4113695083 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8e3c010ac0 con 0x7f8e501049a0 2024-09-10T08:15:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.445+0000 7f8e47400640 1 --2- 172.21.15.2:0/4113695083 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8e340a2c00 0x7f8e340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.445+0000 7f8e47400640 1 -- 172.21.15.2:0/4113695083 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8e3c0cfb20 con 0x7f8e501049a0 2024-09-10T08:15:16.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.445+0000 7f8e4ea00640 1 --2- 172.21.15.2:0/4113695083 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8e340a2c00 0x7f8e340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:16.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.446+0000 7f8e4ea00640 1 --2- 172.21.15.2:0/4113695083 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8e340a2c00 0x7f8e340a50c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009230 tx=0x7f8e400058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:16.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.447+0000 7f8e47400640 1 -- 172.21.15.2:0/4113695083 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e3c09bfe0 con 0x7f8e501049a0 2024-09-10T08:15:16.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.668+0000 7f8e55600640 1 -- 172.21.15.2:0/4113695083 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e5019b950 con 0x7f8e501049a0 2024-09-10T08:15:16.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.669+0000 7f8e47400640 1 -- 172.21.15.2:0/4113695083 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f8e3c0a02e0 con 0x7f8e501049a0 2024-09-10T08:15:16.668 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:15:16.668 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:15:16.669 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:15:16.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.671+0000 7f8e55600640 1 -- 172.21.15.2:0/4113695083 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8e340a2c00 msgr2=0x7f8e340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:16.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.671+0000 7f8e55600640 1 --2- 172.21.15.2:0/4113695083 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8e340a2c00 0x7f8e340a50c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009230 tx=0x7f8e400058d0 comp rx=0 tx=0).stop 2024-09-10T08:15:16.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.671+0000 7f8e55600640 1 -- 172.21.15.2:0/4113695083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e501049a0 msgr2=0x7f8e5019ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:16.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.671+0000 7f8e55600640 1 --2- 172.21.15.2:0/4113695083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e501049a0 0x7f8e5019ab30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00e9f0 tx=0x7f8e3c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:15:16.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.672+0000 7f8e55600640 1 -- 172.21.15.2:0/4113695083 shutdown_connections 2024-09-10T08:15:16.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.672+0000 7f8e55600640 1 --2- 172.21.15.2:0/4113695083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e50109120 0x7f8e5019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:16.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.672+0000 7f8e55600640 1 --2- 172.21.15.2:0/4113695083 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8e340a2c00 0x7f8e340a50c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:16.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.672+0000 7f8e55600640 1 --2- 172.21.15.2:0/4113695083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e501049a0 0x7f8e5019ab30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:16.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.672+0000 7f8e55600640 1 --2- 172.21.15.2:0/4113695083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e50103ff0 0x7f8e5019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:16.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.672+0000 7f8e55600640 1 -- 172.21.15.2:0/4113695083 >> 172.21.15.2:0/4113695083 conn(0x7f8e500ffa90 msgr2=0x7f8e50100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:16.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.672+0000 7f8e55600640 1 -- 172.21.15.2:0/4113695083 shutdown_connections 2024-09-10T08:15:16.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:16.672+0000 7f8e55600640 1 -- 172.21.15.2:0/4113695083 wait complete. 2024-09-10T08:15:17.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:17 smithi002 ceph-mon[30563]: pgmap v601: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-10T08:15:17.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:17 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/4113695083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:15:17.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:17 smithi155 ceph-mon[35405]: pgmap v601: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-10T08:15:17.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:17 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/4113695083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:15:17.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:17 smithi107 ceph-mon[35048]: pgmap v601: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-10T08:15:17.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:17 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/4113695083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:15:17.556 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id 7dd6451a --release --await' 2024-09-10T08:15:17.556 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id 7dd6451a --release --await 2024-09-10T08:15:17.934 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.473+0000 7fc269000640 1 -- 172.21.15.2:0/2180986452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2640761d0 msgr2=0x7fc264074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.473+0000 7fc269000640 1 --2- 172.21.15.2:0/2180986452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2640761d0 0x7fc264074620 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc254009900 tx=0x7fc25402a480 comp rx=0 tx=0).stop 2024-09-10T08:15:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.473+0000 7fc269000640 1 -- 172.21.15.2:0/2180986452 shutdown_connections 2024-09-10T08:15:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.473+0000 7fc269000640 1 --2- 172.21.15.2:0/2180986452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2641099f0 0x7fc26410beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.473+0000 7fc269000640 1 --2- 172.21.15.2:0/2180986452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc264074bf0 0x7fc264075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.473+0000 7fc269000640 1 --2- 172.21.15.2:0/2180986452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2640761d0 0x7fc264074620 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.473+0000 7fc269000640 1 -- 172.21.15.2:0/2180986452 >> 172.21.15.2:0/2180986452 conn(0x7fc2640fd2e0 msgr2=0x7fc2640ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.474+0000 7fc269000640 1 -- 172.21.15.2:0/2180986452 shutdown_connections 2024-09-10T08:15:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.474+0000 7fc269000640 1 -- 172.21.15.2:0/2180986452 wait complete. 2024-09-10T08:15:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.475+0000 7fc269000640 1 Processor -- start 2024-09-10T08:15:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.475+0000 7fc269000640 1 -- start start 2024-09-10T08:15:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.475+0000 7fc269000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc264074bf0 0x7fc26419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.475+0000 7fc269000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2640761d0 0x7fc26419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.475+0000 7fc269000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2641099f0 0x7fc2641a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.475+0000 7fc269000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc26410fa20 con 0x7fc2640761d0 2024-09-10T08:15:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.475+0000 7fc269000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc26410f8a0 con 0x7fc2641099f0 2024-09-10T08:15:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.475+0000 7fc269000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc26410fba0 con 0x7fc264074bf0 2024-09-10T08:15:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.475+0000 7fc261400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2640761d0 0x7fc26419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.475+0000 7fc261400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2640761d0 0x7fc26419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37676/0 (socket says 172.21.15.2:37676) 2024-09-10T08:15:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.475+0000 7fc261400640 1 -- 172.21.15.2:0/1024833136 learned_addr learned my addr 172.21.15.2:0/1024833136 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.475+0000 7fc261e00640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc264074bf0 0x7fc26419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.476+0000 7fc262800640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2641099f0 0x7fc2641a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.476+0000 7fc261400640 1 -- 172.21.15.2:0/1024833136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc264074bf0 msgr2=0x7fc26419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.476+0000 7fc261400640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc264074bf0 0x7fc26419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.476+0000 7fc261400640 1 -- 172.21.15.2:0/1024833136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2641099f0 msgr2=0x7fc2641a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.476+0000 7fc261400640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2641099f0 0x7fc2641a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.476+0000 7fc261400640 1 -- 172.21.15.2:0/1024833136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2641a1c40 con 0x7fc2640761d0 2024-09-10T08:15:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.476+0000 7fc262800640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2641099f0 0x7fc2641a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.476+0000 7fc261400640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2640761d0 0x7fc26419a560 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc25000e9f0 tx=0x7fc25000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.476+0000 7fc25aa00640 1 -- 172.21.15.2:0/1024833136 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25000cd80 con 0x7fc2640761d0 2024-09-10T08:15:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.476+0000 7fc25aa00640 1 -- 172.21.15.2:0/1024833136 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc250010910 con 0x7fc2640761d0 2024-09-10T08:15:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.476+0000 7fc269000640 1 -- 172.21.15.2:0/1024833136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2641a1f30 con 0x7fc2640761d0 2024-09-10T08:15:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.476+0000 7fc25aa00640 1 -- 172.21.15.2:0/1024833136 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250018990 con 0x7fc2640761d0 2024-09-10T08:15:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.477+0000 7fc269000640 1 -- 172.21.15.2:0/1024833136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2641a23f0 con 0x7fc2640761d0 2024-09-10T08:15:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.480+0000 7fc26c009740 1 Processor -- start 2024-09-10T08:15:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.480+0000 7fc26c009740 1 -- start start 2024-09-10T08:15:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.480+0000 7fc26c009740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12d7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.480+0000 7fc26c009740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12dc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.480+0000 7fc26c009740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12def20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.480+0000 7fc26c009740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5634c123c100 con 0x5634c12d7090 2024-09-10T08:15:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.480+0000 7fc26c009740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5634c11e41b0 con 0x5634c12dca60 2024-09-10T08:15:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.480+0000 7fc26c009740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5634c12380c0 con 0x5634c12d79d0 2024-09-10T08:15:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.480+0000 7fc25aa00640 1 -- 172.21.15.2:0/1024833136 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc250010ac0 con 0x7fc2640761d0 2024-09-10T08:15:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.481+0000 7fc261e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.481+0000 7fc262800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12d7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.481+0000 7fc261e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12dc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:39538/0 (socket says 172.21.15.2:39538) 2024-09-10T08:15:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.481+0000 7fc261e00640 1 -- 172.21.15.2:0/2442975201 learned_addr learned my addr 172.21.15.2:0/2442975201 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.481+0000 7fc261400640 1 --2- 172.21.15.2:0/2442975201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12def20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.481+0000 7fc261e00640 1 -- 172.21.15.2:0/2442975201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 msgr2=0x5634c12def20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.481+0000 7fc261e00640 1 --2- 172.21.15.2:0/2442975201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12def20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.481+0000 7fc261e00640 1 -- 172.21.15.2:0/2442975201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 msgr2=0x5634c12d7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.481+0000 7fc261e00640 1 --2- 172.21.15.2:0/2442975201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12d7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.481+0000 7fc261e00640 1 -- 172.21.15.2:0/2442975201 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c0f4f210 con 0x5634c12d79d0 2024-09-10T08:15:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.481+0000 7fc261400640 1 --2- 172.21.15.2:0/2442975201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12def20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.482+0000 7fc261e00640 1 --2- 172.21.15.2:0/2442975201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12dc490 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc254000c00 tx=0x7fc254004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c0c2be4d5fc4436 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.482+0000 7fc24f400640 1 -- 172.21.15.2:0/2442975201 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254035040 con 0x5634c12d79d0 2024-09-10T08:15:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.482+0000 7fc24f400640 1 -- 172.21.15.2:0/2442975201 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc254005520 con 0x5634c12d79d0 2024-09-10T08:15:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.482+0000 7fc24f400640 1 -- 172.21.15.2:0/2442975201 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254031910 con 0x5634c12d79d0 2024-09-10T08:15:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.482+0000 7fc25aa00640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc2480a2c30 0x7fc2480a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.482+0000 7fc25aa00640 1 -- 172.21.15.2:0/1024833136 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc2500cf800 con 0x7fc2640761d0 2024-09-10T08:15:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.483+0000 7fc262800640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc2480a2c30 0x7fc2480a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.483+0000 7fc26c009740 1 -- 172.21.15.2:0/2442975201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 msgr2=0x5634c12dc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.483+0000 7fc26c009740 1 --2- 172.21.15.2:0/2442975201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12dc490 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc254000c00 tx=0x7fc254004040 comp rx=0 tx=0).stop 2024-09-10T08:15:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.483+0000 7fc26c009740 1 -- 172.21.15.2:0/2442975201 shutdown_connections 2024-09-10T08:15:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.483+0000 7fc26c009740 1 --2- 172.21.15.2:0/2442975201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12def20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.483+0000 7fc26c009740 1 --2- 172.21.15.2:0/2442975201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12dc490 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.483+0000 7fc26c009740 1 --2- 172.21.15.2:0/2442975201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12d7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.483+0000 7fc26c009740 1 -- 172.21.15.2:0/2442975201 >> 172.21.15.2:0/2442975201 conn(0x5634c12d2700 msgr2=0x5634c12d2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.483+0000 7fc26c009740 1 -- 172.21.15.2:0/2442975201 shutdown_connections 2024-09-10T08:15:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.483+0000 7fc26c009740 1 -- 172.21.15.2:0/2442975201 wait complete. 2024-09-10T08:15:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.483+0000 7fc26c009740 1 -- start start 2024-09-10T08:15:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.483+0000 7fc262800640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc2480a2c30 0x7fc2480a50f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0099a0 tx=0x7fc25c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.484+0000 7fc26c009740 1 Processor -- start 2024-09-10T08:15:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.484+0000 7fc26c009740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.484+0000 7fc26c009740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12e1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.484+0000 7fc26c009740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.484+0000 7fc26c009740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5634c12df9d0 con 0x5634c12d7090 2024-09-10T08:15:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.484+0000 7fc26c009740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5634c11c5090 con 0x5634c12dca60 2024-09-10T08:15:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.484+0000 7fc26c009740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5634c12dfb50 con 0x5634c12d79d0 2024-09-10T08:15:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.484+0000 7fc26c009740 10 client.0 _ll_register_callbacks cb 0x5634c1219f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.484+0000 7fc261e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.484+0000 7fc261e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12e11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37694/0 (socket says 172.21.15.2:37694) 2024-09-10T08:15:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.484+0000 7fc261e00640 1 -- 172.21.15.2:0/2861412720 learned_addr learned my addr 172.21.15.2:0/2861412720 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.484+0000 7fc261400640 1 --2- 172.21.15.2:0/2861412720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.484+0000 7fc262800640 1 --2- 172.21.15.2:0/2861412720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.485+0000 7fc261e00640 1 -- 172.21.15.2:0/2861412720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 msgr2=0x5634c12e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.485+0000 7fc261e00640 1 --2- 172.21.15.2:0/2861412720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12e1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.485+0000 7fc261e00640 1 -- 172.21.15.2:0/2861412720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 msgr2=0x5634c12e8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.485+0000 7fc261e00640 1 --2- 172.21.15.2:0/2861412720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12e8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.485+0000 7fc261e00640 1 -- 172.21.15.2:0/2861412720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c101b620 con 0x5634c12d7090 2024-09-10T08:15:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.485+0000 7fc261e00640 1 --2- 172.21.15.2:0/2861412720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12e11e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc2540314e0 tx=0x7fc254004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67699f44f131478 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.485+0000 7fc243e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.485+0000 7fc243e00640 1 -- 172.21.15.2:0/2861412720 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254052060 con 0x5634c12d7090 2024-09-10T08:15:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.485+0000 7fc26c009740 1 -- 172.21.15.2:0/2861412720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc254005520 con 0x5634c12d7090 2024-09-10T08:15:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.485+0000 7fc243e00640 1 -- 172.21.15.2:0/2861412720 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc254004980 con 0x5634c12d7090 2024-09-10T08:15:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.486+0000 7fc243e00640 1 -- 172.21.15.2:0/2861412720 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc254035070 con 0x5634c12d7090 2024-09-10T08:15:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.486+0000 7fc26c009740 10 client.17382 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.486+0000 7fc26c009740 1 -- 172.21.15.2:0/2861412720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5634c12eb550 con 0x5634c12d7090 2024-09-10T08:15:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.486+0000 7fc243e00640 1 -- 172.21.15.2:0/2861412720 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc254005520 con 0x5634c12d7090 2024-09-10T08:15:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.486+0000 7fc243e00640 1 -- 172.21.15.2:0/2861412720 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc254003aa0 con 0x5634c12d7090 2024-09-10T08:15:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.486+0000 7fc26c009740 10 client.17382 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.486+0000 7fc26c009740 10 client.17382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.486+0000 7fc26c009740 20 client.17382 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.486+0000 7fc26c009740 1 --2- 172.21.15.2:0/2861412720 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5634c12ec8a0 0x5634c12eece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.486+0000 7fc26c009740 4 client.17382 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.486+0000 7fc26c009740 1 -- 172.21.15.2:0/2861412720 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634c1249830 con 0x5634c12ec8a0 2024-09-10T08:15:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.487+0000 7fc261400640 1 --2- 172.21.15.2:0/2861412720 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5634c12ec8a0 0x5634c12eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.487+0000 7fc261400640 1 --2- 172.21.15.2:0/2861412720 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5634c12ec8a0 0x5634c12eece0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c199519b6b1c5fe server_cookie=c587e09da13db35 in_seq=0 out_seq=0 2024-09-10T08:15:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.487+0000 7fc243e00640 10 client.17382 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.488+0000 7fc243e00640 1 -- 172.21.15.2:0/2861412720 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634c1249830 con 0x5634c12ec8a0 2024-09-10T08:15:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.488+0000 7fc243e00640 10 client.17382 handle_command_reply: tid=0 2024-09-10T08:15:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.488+0000 7fc26c009740 1 client.17382 shutdown 2024-09-10T08:15:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.488+0000 7fc26c009740 1 -- 172.21.15.2:0/2861412720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 msgr2=0x5634c12e11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.488+0000 7fc26c009740 1 --2- 172.21.15.2:0/2861412720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12e11e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc2540314e0 tx=0x7fc254004500 comp rx=0 tx=0).stop 2024-09-10T08:15:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.488+0000 7fc26c009740 1 -- 172.21.15.2:0/2861412720 shutdown_connections 2024-09-10T08:15:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.488+0000 7fc26c009740 1 --2- 172.21.15.2:0/2861412720 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5634c12ec8a0 0x5634c12eece0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.488+0000 7fc26c009740 1 --2- 172.21.15.2:0/2861412720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.488+0000 7fc26c009740 1 --2- 172.21.15.2:0/2861412720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.488+0000 7fc26c009740 1 --2- 172.21.15.2:0/2861412720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12e11e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.488+0000 7fc243e00640 0 client.17382 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.488+0000 7fc26c009740 1 -- 172.21.15.2:0/2861412720 >> 172.21.15.2:0/2861412720 conn(0x5634c12d2700 msgr2=0x5634c12d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.489+0000 7fc26c009740 1 -- 172.21.15.2:0/2861412720 shutdown_connections 2024-09-10T08:15:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.489+0000 7fc26c009740 1 -- 172.21.15.2:0/2861412720 wait complete. 2024-09-10T08:15:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.489+0000 7fc26c009740 20 client.17382 trim_cache size 0 max 16384 2024-09-10T08:15:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.503+0000 7fc26c009740 1 Processor -- start 2024-09-10T08:15:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.504+0000 7fc26c009740 1 -- start start 2024-09-10T08:15:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.504+0000 7fc26c009740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d7090 0x5634c12d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.504+0000 7fc26c009740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d79d0 0x5634c12e8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.504+0000 7fc26c009740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12d3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.504+0000 7fc26c009740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641af350 con 0x5634c12d79d0 2024-09-10T08:15:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.504+0000 7fc26c009740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641acc20 con 0x5634c12dca60 2024-09-10T08:15:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.504+0000 7fc26c009740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641aaa70 con 0x5634c12d7090 2024-09-10T08:15:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.504+0000 7fc261400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d79d0 0x5634c12e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.504+0000 7fc261400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d79d0 0x5634c12e8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37696/0 (socket says 172.21.15.2:37696) 2024-09-10T08:15:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.504+0000 7fc261400640 1 -- 172.21.15.2:0/1969044623 learned_addr learned my addr 172.21.15.2:0/1969044623 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.504+0000 7fc262800640 1 --2- 172.21.15.2:0/1969044623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.504+0000 7fc261e00640 1 --2- 172.21.15.2:0/1969044623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d7090 0x5634c12d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.505+0000 7fc261400640 1 -- 172.21.15.2:0/1969044623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d7090 msgr2=0x5634c12d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.505+0000 7fc261400640 1 --2- 172.21.15.2:0/1969044623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d7090 0x5634c12d2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.505+0000 7fc261400640 1 -- 172.21.15.2:0/1969044623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 msgr2=0x5634c12d3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.505+0000 7fc261400640 1 --2- 172.21.15.2:0/1969044623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12d3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.505+0000 7fc261400640 1 -- 172.21.15.2:0/1969044623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c12ef360 con 0x5634c12d79d0 2024-09-10T08:15:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.505+0000 7fc261400640 1 --2- 172.21.15.2:0/1969044623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d79d0 0x5634c12e8680 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc25000e9c0 tx=0x7fc250000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bb278541563b47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.505+0000 7fc24ea00640 1 -- 172.21.15.2:0/1969044623 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250099f70 con 0x5634c12d79d0 2024-09-10T08:15:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.505+0000 7fc24ea00640 1 -- 172.21.15.2:0/1969044623 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2500cd020 con 0x5634c12d79d0 2024-09-10T08:15:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.505+0000 7fc24ea00640 1 -- 172.21.15.2:0/1969044623 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250018e20 con 0x5634c12d79d0 2024-09-10T08:15:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.506+0000 7fc26c009740 1 -- 172.21.15.2:0/1969044623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d79d0 msgr2=0x5634c12e8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.506+0000 7fc26c009740 1 --2- 172.21.15.2:0/1969044623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d79d0 0x5634c12e8680 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc25000e9c0 tx=0x7fc250000bc0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.506+0000 7fc26c009740 1 -- 172.21.15.2:0/1969044623 shutdown_connections 2024-09-10T08:15:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.506+0000 7fc26c009740 1 --2- 172.21.15.2:0/1969044623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12d3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.506+0000 7fc26c009740 1 --2- 172.21.15.2:0/1969044623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d79d0 0x5634c12e8680 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.506+0000 7fc26c009740 1 --2- 172.21.15.2:0/1969044623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d7090 0x5634c12d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.506+0000 7fc26c009740 1 -- 172.21.15.2:0/1969044623 >> 172.21.15.2:0/1969044623 conn(0x5634c12d2700 msgr2=0x5634c12e0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.506+0000 7fc26c009740 1 -- 172.21.15.2:0/1969044623 shutdown_connections 2024-09-10T08:15:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.506+0000 7fc26c009740 1 -- 172.21.15.2:0/1969044623 wait complete. 2024-09-10T08:15:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.507+0000 7fc26c009740 1 -- start start 2024-09-10T08:15:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.507+0000 7fc26c009740 1 Processor -- start 2024-09-10T08:15:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.507+0000 7fc26c009740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12e5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.507+0000 7fc26c009740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12e5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.507+0000 7fc26c009740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12e6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.507+0000 7fc26c009740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641ab0e0 con 0x5634c12d7090 2024-09-10T08:15:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.507+0000 7fc26c009740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641ac550 con 0x5634c12dca60 2024-09-10T08:15:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.507+0000 7fc26c009740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641ab7b0 con 0x5634c12d79d0 2024-09-10T08:15:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.507+0000 7fc26c009740 10 client.0 _ll_register_callbacks cb 0x7fc25c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.507+0000 7fc261e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.507+0000 7fc261e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12e5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37700/0 (socket says 172.21.15.2:37700) 2024-09-10T08:15:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.507+0000 7fc261e00640 1 -- 172.21.15.2:0/3245914604 learned_addr learned my addr 172.21.15.2:0/3245914604 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.508+0000 7fc261400640 1 --2- 172.21.15.2:0/3245914604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12e5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.508+0000 7fc262800640 1 --2- 172.21.15.2:0/3245914604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12e6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.508+0000 7fc261e00640 1 -- 172.21.15.2:0/3245914604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 msgr2=0x5634c12e5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.508+0000 7fc261e00640 1 --2- 172.21.15.2:0/3245914604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12e5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.508+0000 7fc261e00640 1 -- 172.21.15.2:0/3245914604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 msgr2=0x5634c12e6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.508+0000 7fc261e00640 1 --2- 172.21.15.2:0/3245914604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12e6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.508+0000 7fc261e00640 1 -- 172.21.15.2:0/3245914604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c12e7380 con 0x5634c12d7090 2024-09-10T08:15:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.508+0000 7fc261e00640 1 --2- 172.21.15.2:0/3245914604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12e5820 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc254009900 tx=0x7fc254003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecac5f6e9ec68b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.508+0000 7fc243e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.508+0000 7fc243e00640 1 -- 172.21.15.2:0/3245914604 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254062060 con 0x5634c12d7090 2024-09-10T08:15:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.508+0000 7fc243e00640 1 -- 172.21.15.2:0/3245914604 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc2540318c0 con 0x5634c12d7090 2024-09-10T08:15:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.508+0000 7fc26c009740 1 -- 172.21.15.2:0/3245914604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2500cd020 con 0x5634c12d7090 2024-09-10T08:15:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.509+0000 7fc243e00640 1 -- 172.21.15.2:0/3245914604 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc254069050 con 0x5634c12d7090 2024-09-10T08:15:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.509+0000 7fc26c009740 10 client.17388 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.509+0000 7fc26c009740 1 -- 172.21.15.2:0/3245914604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5634c12e8700 con 0x5634c12d7090 2024-09-10T08:15:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.509+0000 7fc243e00640 1 -- 172.21.15.2:0/3245914604 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2500cd020 con 0x5634c12d7090 2024-09-10T08:15:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.509+0000 7fc243e00640 1 -- 172.21.15.2:0/3245914604 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc25404e8e0 con 0x5634c12d7090 2024-09-10T08:15:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.509+0000 7fc26c009740 10 client.17388 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.509+0000 7fc26c009740 10 client.17388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.509+0000 7fc26c009740 20 client.17388 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.509+0000 7fc26c009740 1 --2- 172.21.15.2:0/3245914604 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5634c1357a60 0x5634c1359e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.509+0000 7fc26c009740 4 client.17388 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:15:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.509+0000 7fc26c009740 1 -- 172.21.15.2:0/3245914604 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634c1243b80 con 0x5634c1357a60 2024-09-10T08:15:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.510+0000 7fc261400640 1 --2- 172.21.15.2:0/3245914604 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5634c1357a60 0x5634c1359e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.510+0000 7fc261400640 1 --2- 172.21.15.2:0/3245914604 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5634c1357a60 0x5634c1359e80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fdf2eb20b6a5073 server_cookie=fe877a96e6e88bde in_seq=0 out_seq=0 2024-09-10T08:15:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.510+0000 7fc243e00640 10 client.17388 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc243e00640 1 -- 172.21.15.2:0/3245914604 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634c1243b80 con 0x5634c1357a60 2024-09-10T08:15:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc243e00640 10 client.17388 handle_command_reply: tid=0 2024-09-10T08:15:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc26c009740 1 client.17388 shutdown 2024-09-10T08:15:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc26c009740 1 -- 172.21.15.2:0/3245914604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 msgr2=0x5634c12e5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc26c009740 1 --2- 172.21.15.2:0/3245914604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12e5820 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc254009900 tx=0x7fc254003d50 comp rx=0 tx=0).stop 2024-09-10T08:15:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc26c009740 1 -- 172.21.15.2:0/3245914604 shutdown_connections 2024-09-10T08:15:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc26c009740 1 --2- 172.21.15.2:0/3245914604 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5634c1357a60 0x5634c1359e80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc26c009740 1 --2- 172.21.15.2:0/3245914604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634c12dca60 0x5634c12e6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc26c009740 1 --2- 172.21.15.2:0/3245914604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c12d79d0 0x5634c12e5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc26c009740 1 --2- 172.21.15.2:0/3245914604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634c12d7090 0x5634c12e5820 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc26c009740 1 -- 172.21.15.2:0/3245914604 >> 172.21.15.2:0/3245914604 conn(0x5634c12d2700 msgr2=0x5634c12d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc243e00640 0 client.17388 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc26c009740 1 -- 172.21.15.2:0/3245914604 shutdown_connections 2024-09-10T08:15:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc26c009740 1 -- 172.21.15.2:0/3245914604 wait complete. 2024-09-10T08:15:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.519+0000 7fc26c009740 20 client.17388 trim_cache size 0 max 16384 2024-09-10T08:15:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:15:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:15:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:15:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:15:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.520+0000 7fc269000640 1 -- 172.21.15.2:0/1024833136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc2480a2c30 msgr2=0x7fc2480a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.520+0000 7fc269000640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc2480a2c30 0x7fc2480a50f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0099a0 tx=0x7fc25c009500 comp rx=0 tx=0).stop 2024-09-10T08:15:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.520+0000 7fc269000640 1 -- 172.21.15.2:0/1024833136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2640761d0 msgr2=0x7fc26419a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.520+0000 7fc269000640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2640761d0 0x7fc26419a560 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc25000e9f0 tx=0x7fc25000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.520+0000 7fc269000640 1 -- 172.21.15.2:0/1024833136 shutdown_connections 2024-09-10T08:15:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.520+0000 7fc269000640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2641099f0 0x7fc2641a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.520+0000 7fc269000640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2640761d0 0x7fc26419a560 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.520+0000 7fc269000640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc2480a2c30 0x7fc2480a50f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.520+0000 7fc269000640 1 --2- 172.21.15.2:0/1024833136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc264074bf0 0x7fc26419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.520+0000 7fc269000640 1 -- 172.21.15.2:0/1024833136 >> 172.21.15.2:0/1024833136 conn(0x7fc2640fd2e0 msgr2=0x7fc264105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.521+0000 7fc269000640 1 -- 172.21.15.2:0/1024833136 shutdown_connections 2024-09-10T08:15:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:18.521+0000 7fc269000640 1 -- 172.21.15.2:0/1024833136 wait complete. 2024-09-10T08:15:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.519+0000 7f3624c00640 1 -- 172.21.15.2:0/428410491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3620106f00 msgr2=0x7f362010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.519+0000 7f3624c00640 1 --2- 172.21.15.2:0/428410491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3620106f00 0x7f362010ea80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3618009be0 tx=0x7f361802a6c0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.520+0000 7f3624c00640 1 -- 172.21.15.2:0/428410491 shutdown_connections 2024-09-10T08:15:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.520+0000 7f3624c00640 1 --2- 172.21.15.2:0/428410491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3620106f00 0x7f362010ea80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.520+0000 7f3624c00640 1 --2- 172.21.15.2:0/428410491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3620102780 0x7f36201067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.520+0000 7f3624c00640 1 --2- 172.21.15.2:0/428410491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101dd0 0x7f36201021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.520+0000 7f3624c00640 1 -- 172.21.15.2:0/428410491 >> 172.21.15.2:0/428410491 conn(0x7f36200fd890 msgr2=0x7f36200ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.520+0000 7f3624c00640 1 -- 172.21.15.2:0/428410491 shutdown_connections 2024-09-10T08:15:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.520+0000 7f3624c00640 1 -- 172.21.15.2:0/428410491 wait complete. 2024-09-10T08:15:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.521+0000 7f3624c00640 1 Processor -- start 2024-09-10T08:15:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.521+0000 7f3624c00640 1 -- start start 2024-09-10T08:15:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.521+0000 7f3624c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3620101dd0 0x7f362019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.521+0000 7f3624c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3620102780 0x7f362019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.521+0000 7f3624c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620106f00 0x7f362019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.521+0000 7f3624c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3620111f60 con 0x7f3620101dd0 2024-09-10T08:15:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.521+0000 7f3624c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3620111de0 con 0x7f3620106f00 2024-09-10T08:15:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.521+0000 7f3624c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f36201120e0 con 0x7f3620102780 2024-09-10T08:15:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.522+0000 7f361e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620106f00 0x7f362019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.522+0000 7f361de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3620101dd0 0x7f362019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.522+0000 7f361e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620106f00 0x7f362019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35634/0 (socket says 172.21.15.2:35634) 2024-09-10T08:15:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.522+0000 7f361e800640 1 -- 172.21.15.2:0/608466920 learned_addr learned my addr 172.21.15.2:0/608466920 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.522+0000 7f361d400640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3620102780 0x7f362019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.522+0000 7f361d400640 1 -- 172.21.15.2:0/608466920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620106f00 msgr2=0x7f362019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.522+0000 7f361d400640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620106f00 0x7f362019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.522+0000 7f361d400640 1 -- 172.21.15.2:0/608466920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3620101dd0 msgr2=0x7f362019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.522+0000 7f361d400640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3620101dd0 0x7f362019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.522+0000 7f361d400640 1 -- 172.21.15.2:0/608466920 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362019f5a0 con 0x7f3620102780 2024-09-10T08:15:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.522+0000 7f361e800640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620106f00 0x7f362019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.522+0000 7f361de00640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3620101dd0 0x7f362019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.523+0000 7f361d400640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3620102780 0x7f362019ab30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f360c005fd0 tx=0x7f360c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.523+0000 7f3616a00640 1 -- 172.21.15.2:0/608466920 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c00cd00 con 0x7f3620102780 2024-09-10T08:15:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.523+0000 7f3624c00640 1 -- 172.21.15.2:0/608466920 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362019f890 con 0x7f3620102780 2024-09-10T08:15:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.523+0000 7f3616a00640 1 -- 172.21.15.2:0/608466920 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f360c0035f0 con 0x7f3620102780 2024-09-10T08:15:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.523+0000 7f3616a00640 1 -- 172.21.15.2:0/608466920 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c010690 con 0x7f3620102780 2024-09-10T08:15:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.523+0000 7f3624c00640 1 -- 172.21.15.2:0/608466920 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36201a7170 con 0x7f3620102780 2024-09-10T08:15:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.526+0000 7f3627824740 1 Processor -- start 2024-09-10T08:15:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.526+0000 7f3627824740 1 -- start start 2024-09-10T08:15:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.526+0000 7f3616a00640 1 -- 172.21.15.2:0/608466920 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f360c0037a0 con 0x7f3620102780 2024-09-10T08:15:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.527+0000 7f3627824740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1310 0x55cf35aa16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.527+0000 7f3627824740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aa1c30 0x55cf35aabfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.527+0000 7f3627824740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aac4f0 0x55cf35aae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.527+0000 7f3627824740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cf35a07650 con 0x55cf35aac4f0 2024-09-10T08:15:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.527+0000 7f3627824740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cf359aef10 con 0x55cf35aa1c30 2024-09-10T08:15:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.527+0000 7f3627824740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf35a03610 con 0x55cf35aa1310 2024-09-10T08:15:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.527+0000 7f361d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aac4f0 0x55cf35aae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.527+0000 7f361e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1310 0x55cf35aa16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.527+0000 7f361e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1310 0x55cf35aa16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:39574/0 (socket says 172.21.15.2:39574) 2024-09-10T08:15:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.527+0000 7f361e800640 1 -- 172.21.15.2:0/449959121 learned_addr learned my addr 172.21.15.2:0/449959121 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.527+0000 7f361de00640 1 --2- 172.21.15.2:0/449959121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aa1c30 0x55cf35aabfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.527+0000 7f361d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aac4f0 0x55cf35aae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37720/0 (socket says 172.21.15.2:37720) 2024-09-10T08:15:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.528+0000 7f361e800640 1 -- 172.21.15.2:0/449959121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aa1c30 msgr2=0x55cf35aabfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.528+0000 7f361e800640 1 --2- 172.21.15.2:0/449959121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aa1c30 0x55cf35aabfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.528+0000 7f361e800640 1 -- 172.21.15.2:0/449959121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aac4f0 msgr2=0x55cf35aae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.528+0000 7f361e800640 1 --2- 172.21.15.2:0/449959121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aac4f0 0x55cf35aae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.528+0000 7f361e800640 1 -- 172.21.15.2:0/449959121 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf35716580 con 0x55cf35aa1310 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.528+0000 7f361de00640 1 --2- 172.21.15.2:0/449959121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aa1c30 0x55cf35aabfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.528+0000 7f361d400640 1 --2- 172.21.15.2:0/449959121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aac4f0 0x55cf35aae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.528+0000 7f361e800640 1 --2- 172.21.15.2:0/449959121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1310 0x55cf35aa16f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3618030f10 tx=0x7f3618030f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2094578c53ae972f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.528+0000 7f3616a00640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36040a2b80 0x7f36040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.528+0000 7f360b000640 1 -- 172.21.15.2:0/449959121 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618041030 con 0x55cf35aa1310 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.529+0000 7f360b000640 1 -- 172.21.15.2:0/449959121 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3618003970 con 0x55cf35aa1310 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.529+0000 7f360b000640 1 -- 172.21.15.2:0/449959121 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618036040 con 0x55cf35aa1310 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.529+0000 7f3616a00640 1 -- 172.21.15.2:0/608466920 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f360c0cf5d0 con 0x7f3620102780 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.529+0000 7f361de00640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36040a2b80 0x7f36040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.529+0000 7f3627824740 1 -- 172.21.15.2:0/449959121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1310 msgr2=0x55cf35aa16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.529+0000 7f3627824740 1 --2- 172.21.15.2:0/449959121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1310 0x55cf35aa16f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3618030f10 tx=0x7f3618030f40 comp rx=0 tx=0).stop 2024-09-10T08:15:18.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.530+0000 7f361de00640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36040a2b80 0x7f36040a5040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3610009fd0 tx=0x7f3610005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:18.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.530+0000 7f3627824740 1 -- 172.21.15.2:0/449959121 shutdown_connections 2024-09-10T08:15:18.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.530+0000 7f3627824740 1 --2- 172.21.15.2:0/449959121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aac4f0 0x55cf35aae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.530+0000 7f3627824740 1 --2- 172.21.15.2:0/449959121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aa1c30 0x55cf35aabfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.530+0000 7f3627824740 1 --2- 172.21.15.2:0/449959121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1310 0x55cf35aa16f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.530+0000 7f3627824740 1 -- 172.21.15.2:0/449959121 >> 172.21.15.2:0/449959121 conn(0x55cf35a9d8a0 msgr2=0x55cf35a9dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:18.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.530+0000 7f3627824740 1 -- 172.21.15.2:0/449959121 shutdown_connections 2024-09-10T08:15:18.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.530+0000 7f3627824740 1 -- 172.21.15.2:0/449959121 wait complete. 2024-09-10T08:15:18.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.530+0000 7f3627824740 1 -- start start 2024-09-10T08:15:18.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.531+0000 7f3627824740 1 Processor -- start 2024-09-10T08:15:18.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.531+0000 7f3627824740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1310 0x55cf35aa9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.531+0000 7f3627824740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1c30 0x55cf35aa9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.531+0000 7f3627824740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35aa9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.531+0000 7f3627824740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cf35aaf110 con 0x55cf35aa1310 2024-09-10T08:15:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.531+0000 7f3627824740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cf359c2350 con 0x55cf35aac4f0 2024-09-10T08:15:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.531+0000 7f3627824740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf35aaf290 con 0x55cf35aa1c30 2024-09-10T08:15:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.531+0000 7f3627824740 10 client.0 _ll_register_callbacks cb 0x55cf359e5290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.531+0000 7f361de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1310 0x55cf35aa9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.531+0000 7f361de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1310 0x55cf35aa9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37730/0 (socket says 172.21.15.2:37730) 2024-09-10T08:15:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.531+0000 7f361de00640 1 -- 172.21.15.2:0/2814944644 learned_addr learned my addr 172.21.15.2:0/2814944644 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.531+0000 7f361e800640 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35aa9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.531+0000 7f361d400640 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1c30 0x55cf35aa9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.532+0000 7f361d400640 1 -- 172.21.15.2:0/2814944644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 msgr2=0x55cf35aa9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.532+0000 7f361d400640 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35aa9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.532+0000 7f361d400640 1 -- 172.21.15.2:0/2814944644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1310 msgr2=0x55cf35aa9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.532+0000 7f361d400640 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1310 0x55cf35aa9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.532+0000 7f361d400640 1 -- 172.21.15.2:0/2814944644 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf357e0960 con 0x55cf35aa1c30 2024-09-10T08:15:18.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.532+0000 7f361de00640 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1310 0x55cf35aa9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:18.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.532+0000 7f361e800640 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35aa9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:18.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.532+0000 7f361d400640 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1c30 0x55cf35aa9890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55cf35a9ec20 tx=0x7f360c009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9f461decebd39fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:18.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.532+0000 7f35ff400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:15:18.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.532+0000 7f35ff400640 1 -- 172.21.15.2:0/2814944644 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c009510 con 0x55cf35aa1c30 2024-09-10T08:15:18.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.532+0000 7f3627824740 1 -- 172.21.15.2:0/2814944644 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3618003970 con 0x55cf35aa1c30 2024-09-10T08:15:18.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.533+0000 7f35ff400640 1 -- 172.21.15.2:0/2814944644 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f360c0cf220 con 0x55cf35aa1c30 2024-09-10T08:15:18.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.533+0000 7f35ff400640 1 -- 172.21.15.2:0/2814944644 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f360c09c0e0 con 0x55cf35aa1c30 2024-09-10T08:15:18.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.533+0000 7f3627824740 10 client.27373 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:18.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.533+0000 7f3627824740 1 -- 172.21.15.2:0/2814944644 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cf35abad50 con 0x55cf35aa1c30 2024-09-10T08:15:18.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.533+0000 7f35ff400640 1 -- 172.21.15.2:0/2814944644 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3618003970 con 0x55cf35aa1c30 2024-09-10T08:15:18.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.533+0000 7f35ff400640 1 -- 172.21.15.2:0/2814944644 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f360c09af50 con 0x55cf35aa1c30 2024-09-10T08:15:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.533+0000 7f3627824740 10 client.27373 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.533+0000 7f3627824740 10 client.27373 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:15:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.533+0000 7f3627824740 20 client.27373 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.533+0000 7f3627824740 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf35abc160 0x55cf35abe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.533+0000 7f3627824740 4 client.27373 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.533+0000 7f3627824740 1 -- 172.21.15.2:0/2814944644 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf35a14bc0 con 0x55cf35abc160 2024-09-10T08:15:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.534+0000 7f361de00640 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf35abc160 0x55cf35abe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.534+0000 7f361de00640 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf35abc160 0x55cf35abe550 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3de8c51a025addf server_cookie=c45a2a1be59f9858 in_seq=0 out_seq=0 2024-09-10T08:15:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.534+0000 7f35ff400640 10 client.27373 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.535+0000 7f35ff400640 1 -- 172.21.15.2:0/2814944644 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf35a14bc0 con 0x55cf35abc160 2024-09-10T08:15:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.535+0000 7f35ff400640 10 client.27373 handle_command_reply: tid=0 2024-09-10T08:15:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.535+0000 7f3627824740 1 client.27373 shutdown 2024-09-10T08:15:18.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.535+0000 7f3627824740 1 -- 172.21.15.2:0/2814944644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1c30 msgr2=0x55cf35aa9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.535+0000 7f3627824740 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1c30 0x55cf35aa9890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55cf35a9ec20 tx=0x7f360c009f70 comp rx=0 tx=0).stop 2024-09-10T08:15:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.536+0000 7f3627824740 1 -- 172.21.15.2:0/2814944644 shutdown_connections 2024-09-10T08:15:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.536+0000 7f3627824740 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf35abc160 0x55cf35abe550 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.536+0000 7f3627824740 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35aa9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.536+0000 7f3627824740 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1c30 0x55cf35aa9890 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.536+0000 7f3627824740 1 --2- 172.21.15.2:0/2814944644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1310 0x55cf35aa9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.536+0000 7f3627824740 1 -- 172.21.15.2:0/2814944644 >> 172.21.15.2:0/2814944644 conn(0x55cf35a9d8a0 msgr2=0x55cf35a9caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.536+0000 7f35ff400640 0 client.27373 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.536+0000 7f3627824740 1 -- 172.21.15.2:0/2814944644 shutdown_connections 2024-09-10T08:15:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.536+0000 7f3627824740 1 -- 172.21.15.2:0/2814944644 wait complete. 2024-09-10T08:15:18.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.536+0000 7f3627824740 20 client.27373 trim_cache size 0 max 16384 2024-09-10T08:15:18.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.555+0000 7f3627824740 1 Processor -- start 2024-09-10T08:15:18.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.555+0000 7f3627824740 1 -- start start 2024-09-10T08:15:18.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.555+0000 7f3627824740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1310 0x55cf35ac3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.555+0000 7f3627824740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1c30 0x55cf35ac4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.555+0000 7f3627824740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35aa65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.555+0000 7f3627824740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f36201b38a0 con 0x55cf35aa1c30 2024-09-10T08:15:18.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.555+0000 7f3627824740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36201b1170 con 0x55cf35aac4f0 2024-09-10T08:15:18.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.555+0000 7f3627824740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f36201aefc0 con 0x55cf35aa1310 2024-09-10T08:15:18.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.555+0000 7f361d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1c30 0x55cf35ac4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.555+0000 7f361d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1c30 0x55cf35ac4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37734/0 (socket says 172.21.15.2:37734) 2024-09-10T08:15:18.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.555+0000 7f361e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35aa65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.555+0000 7f361d400640 1 -- 172.21.15.2:0/1290890976 learned_addr learned my addr 172.21.15.2:0/1290890976 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:18.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.556+0000 7f361de00640 1 --2- 172.21.15.2:0/1290890976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1310 0x55cf35ac3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.556+0000 7f361de00640 1 -- 172.21.15.2:0/1290890976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 msgr2=0x55cf35aa65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.556+0000 7f361de00640 1 --2- 172.21.15.2:0/1290890976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35aa65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.556+0000 7f361de00640 1 -- 172.21.15.2:0/1290890976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1c30 msgr2=0x55cf35ac4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.556+0000 7f361de00640 1 --2- 172.21.15.2:0/1290890976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1c30 0x55cf35ac4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.556+0000 7f361de00640 1 -- 172.21.15.2:0/1290890976 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf35aa1a60 con 0x55cf35aa1310 2024-09-10T08:15:18.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.556+0000 7f361de00640 1 --2- 172.21.15.2:0/1290890976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1310 0x55cf35ac3e20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55cf35aac440 tx=0x7f361001a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f36b4e0046a515 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:18.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.556+0000 7f360a600640 1 -- 172.21.15.2:0/1290890976 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36100040f0 con 0x55cf35aa1310 2024-09-10T08:15:18.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.557+0000 7f360a600640 1 -- 172.21.15.2:0/1290890976 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3610004290 con 0x55cf35aa1310 2024-09-10T08:15:18.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.557+0000 7f360a600640 1 -- 172.21.15.2:0/1290890976 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361002b8b0 con 0x55cf35aa1310 2024-09-10T08:15:18.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.557+0000 7f3627824740 1 -- 172.21.15.2:0/1290890976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1310 msgr2=0x55cf35ac3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.557+0000 7f3627824740 1 --2- 172.21.15.2:0/1290890976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1310 0x55cf35ac3e20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55cf35aac440 tx=0x7f361001a000 comp rx=0 tx=0).stop 2024-09-10T08:15:18.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.557+0000 7f3627824740 1 -- 172.21.15.2:0/1290890976 shutdown_connections 2024-09-10T08:15:18.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.557+0000 7f3627824740 1 --2- 172.21.15.2:0/1290890976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35aa65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.557+0000 7f3627824740 1 --2- 172.21.15.2:0/1290890976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1c30 0x55cf35ac4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.557+0000 7f3627824740 1 --2- 172.21.15.2:0/1290890976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1310 0x55cf35ac3e20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.557+0000 7f3627824740 1 -- 172.21.15.2:0/1290890976 >> 172.21.15.2:0/1290890976 conn(0x55cf35a9d8a0 msgr2=0x55cf35a9caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:18.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.557+0000 7f3627824740 1 -- 172.21.15.2:0/1290890976 shutdown_connections 2024-09-10T08:15:18.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.557+0000 7f3627824740 1 -- 172.21.15.2:0/1290890976 wait complete. 2024-09-10T08:15:18.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.558+0000 7f3627824740 1 -- start start 2024-09-10T08:15:18.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.558+0000 7f3627824740 1 Processor -- start 2024-09-10T08:15:18.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.558+0000 7f3627824740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1310 0x55cf35ab03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.558+0000 7f3627824740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1c30 0x55cf35ab0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.558+0000 7f3627824740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35ab0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.558+0000 7f3627824740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f36201af630 con 0x55cf35aa1310 2024-09-10T08:15:18.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.558+0000 7f3627824740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36201b0aa0 con 0x55cf35aac4f0 2024-09-10T08:15:18.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.558+0000 7f3627824740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f36201afd00 con 0x55cf35aa1c30 2024-09-10T08:15:18.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.559+0000 7f3627824740 10 client.0 _ll_register_callbacks cb 0x7f360c0d2d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:18.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.559+0000 7f361e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35ab0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.559+0000 7f361de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1310 0x55cf35ab03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.559+0000 7f361d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1c30 0x55cf35ab0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.559+0000 7f361de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1310 0x55cf35ab03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37738/0 (socket says 172.21.15.2:37738) 2024-09-10T08:15:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.559+0000 7f361de00640 1 -- 172.21.15.2:0/2803559179 learned_addr learned my addr 172.21.15.2:0/2803559179 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.559+0000 7f361e800640 1 -- 172.21.15.2:0/2803559179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1c30 msgr2=0x55cf35ab0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.559+0000 7f361e800640 1 --2- 172.21.15.2:0/2803559179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1c30 0x55cf35ab0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.559+0000 7f361e800640 1 -- 172.21.15.2:0/2803559179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1310 msgr2=0x55cf35ab03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.559+0000 7f361e800640 1 --2- 172.21.15.2:0/2803559179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1310 0x55cf35ab03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.559+0000 7f361e800640 1 -- 172.21.15.2:0/2803559179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf35b2e7a0 con 0x55cf35aac4f0 2024-09-10T08:15:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.560+0000 7f361e800640 1 --2- 172.21.15.2:0/2803559179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35ab0e50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3618034040 tx=0x7f3618005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e77415734e8b8a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.560+0000 7f35ff400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:15:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.560+0000 7f35ff400640 1 -- 172.21.15.2:0/2803559179 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618036040 con 0x55cf35aac4f0 2024-09-10T08:15:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.560+0000 7f3627824740 1 -- 172.21.15.2:0/2803559179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3610004290 con 0x55cf35aac4f0 2024-09-10T08:15:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.560+0000 7f35ff400640 1 -- 172.21.15.2:0/2803559179 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f36180303a0 con 0x55cf35aac4f0 2024-09-10T08:15:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.560+0000 7f35ff400640 1 -- 172.21.15.2:0/2803559179 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3618059050 con 0x55cf35aac4f0 2024-09-10T08:15:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.560+0000 7f3627824740 10 client.27443 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.560+0000 7f3627824740 1 -- 172.21.15.2:0/2803559179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cf35b2fb10 con 0x55cf35aac4f0 2024-09-10T08:15:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.560+0000 7f35ff400640 1 -- 172.21.15.2:0/2803559179 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3610004290 con 0x55cf35aac4f0 2024-09-10T08:15:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.561+0000 7f35ff400640 1 -- 172.21.15.2:0/2803559179 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f361803d460 con 0x55cf35aac4f0 2024-09-10T08:15:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.561+0000 7f3627824740 10 client.27443 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.561+0000 7f3627824740 10 client.27443 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:15:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.561+0000 7f3627824740 20 client.27443 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.561+0000 7f3627824740 1 --2- 172.21.15.2:0/2803559179 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf35b2fe00 0x55cf35b321f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.561+0000 7f3627824740 4 client.27443 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7dd6451a", "release": true, "await": true}] 2024-09-10T08:15:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.561+0000 7f3627824740 1 -- 172.21.15.2:0/2803559179 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7dd6451a", "release": true, "await": true}) -- 0x55cf35a0f0d0 con 0x55cf35b2fe00 2024-09-10T08:15:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.561+0000 7f361de00640 1 --2- 172.21.15.2:0/2803559179 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf35b2fe00 0x55cf35b321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.562+0000 7f361de00640 1 --2- 172.21.15.2:0/2803559179 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf35b2fe00 0x55cf35b321f0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25707fab63dc1f6 server_cookie=a6546d2f48495463 in_seq=0 out_seq=0 2024-09-10T08:15:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.562+0000 7f35ff400640 10 client.27443 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.564+0000 7f35ff400640 1 -- 172.21.15.2:0/2803559179 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cf35a0f0d0 con 0x55cf35b2fe00 2024-09-10T08:15:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.564+0000 7f35ff400640 10 client.27443 handle_command_reply: tid=0 2024-09-10T08:15:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.564+0000 7f3627824740 1 client.27443 shutdown 2024-09-10T08:15:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.564+0000 7f3627824740 1 -- 172.21.15.2:0/2803559179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 msgr2=0x55cf35ab0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.564+0000 7f3627824740 1 --2- 172.21.15.2:0/2803559179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35ab0e50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3618034040 tx=0x7f3618005bd0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.564+0000 7f3627824740 1 -- 172.21.15.2:0/2803559179 shutdown_connections 2024-09-10T08:15:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.564+0000 7f3627824740 1 --2- 172.21.15.2:0/2803559179 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf35b2fe00 0x55cf35b321f0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.564+0000 7f3627824740 1 --2- 172.21.15.2:0/2803559179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf35aac4f0 0x55cf35ab0e50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.564+0000 7f3627824740 1 --2- 172.21.15.2:0/2803559179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf35aa1c30 0x55cf35ab0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.564+0000 7f35ff400640 0 client.27443 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.564+0000 7f3627824740 1 --2- 172.21.15.2:0/2803559179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf35aa1310 0x55cf35ab03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.564+0000 7f3627824740 1 -- 172.21.15.2:0/2803559179 >> 172.21.15.2:0/2803559179 conn(0x55cf35a9d8a0 msgr2=0x55cf35aa9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.565+0000 7f3627824740 1 -- 172.21.15.2:0/2803559179 shutdown_connections 2024-09-10T08:15:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.565+0000 7f3627824740 1 -- 172.21.15.2:0/2803559179 wait complete. 2024-09-10T08:15:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.565+0000 7f3627824740 20 client.27443 trim_cache size 0 max 16384 2024-09-10T08:15:18.566 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:15:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:15:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:15:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 24, 2024-09-10T08:15:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:15:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "7dd6451a": { 2024-09-10T08:15:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 24, 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 140.1, 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:15:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:15:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:15:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:15:18.568 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:15:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.565+0000 7f3624c00640 1 -- 172.21.15.2:0/608466920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36040a2b80 msgr2=0x7f36040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.565+0000 7f3624c00640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36040a2b80 0x7f36040a5040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3610009fd0 tx=0x7f3610005f90 comp rx=0 tx=0).stop 2024-09-10T08:15:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.565+0000 7f3624c00640 1 -- 172.21.15.2:0/608466920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3620102780 msgr2=0x7f362019ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.565+0000 7f3624c00640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3620102780 0x7f362019ab30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f360c005fd0 tx=0x7f360c00ee50 comp rx=0 tx=0).stop 2024-09-10T08:15:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.565+0000 7f3624c00640 1 -- 172.21.15.2:0/608466920 shutdown_connections 2024-09-10T08:15:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.565+0000 7f3624c00640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620106f00 0x7f362019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.565+0000 7f3624c00640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f36040a2b80 0x7f36040a5040 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.565+0000 7f3624c00640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3620102780 0x7f362019ab30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.565+0000 7f3624c00640 1 --2- 172.21.15.2:0/608466920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3620101dd0 0x7f362019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.565+0000 7f3624c00640 1 -- 172.21.15.2:0/608466920 >> 172.21.15.2:0/608466920 conn(0x7f36200fd890 msgr2=0x7f36200fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.566+0000 7f3624c00640 1 -- 172.21.15.2:0/608466920 shutdown_connections 2024-09-10T08:15:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:18.566+0000 7f3624c00640 1 -- 172.21.15.2:0/608466920 wait complete. 2024-09-10T08:15:19.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:19 smithi002 ceph-mon[30563]: pgmap v602: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T08:15:19.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:19 smithi155 ceph-mon[35405]: pgmap v602: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T08:15:19.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:19 smithi107 ceph-mon[35048]: pgmap v602: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T08:15:19.538 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7dd6451a', total seconds elapsed: 24.1 2024-09-10T08:15:21.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:21 smithi002 ceph-mon[30563]: pgmap v603: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-10T08:15:21.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:21 smithi155 ceph-mon[35405]: pgmap v603: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-10T08:15:21.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:21 smithi107 ceph-mon[35048]: pgmap v603: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-10T08:15:23.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:23 smithi155 ceph-mon[35405]: pgmap v604: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T08:15:23.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:23 smithi107 ceph-mon[35048]: pgmap v604: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T08:15:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:23 smithi002 ceph-mon[30563]: pgmap v604: 129 pgs: 129 active+clean; 3.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T08:15:25.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:25 smithi155 ceph-mon[35405]: pgmap v605: 129 pgs: 129 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.9 KiB/s wr, 9 op/s 2024-09-10T08:15:25.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:25 smithi107 ceph-mon[35048]: pgmap v605: 129 pgs: 129 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.9 KiB/s wr, 9 op/s 2024-09-10T08:15:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:25 smithi002 ceph-mon[30563]: pgmap v605: 129 pgs: 129 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.9 KiB/s wr, 9 op/s 2024-09-10T08:15:27.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:27 smithi155 ceph-mon[35405]: pgmap v606: 129 pgs: 129 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-09-10T08:15:27.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:27 smithi107 ceph-mon[35048]: pgmap v606: 129 pgs: 129 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-09-10T08:15:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:27 smithi002 ceph-mon[30563]: pgmap v606: 129 pgs: 129 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-09-10T08:15:29.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:29 smithi155 ceph-mon[35405]: pgmap v607: 129 pgs: 129 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-09-10T08:15:29.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:29 smithi107 ceph-mon[35048]: pgmap v607: 129 pgs: 129 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-09-10T08:15:29.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:29 smithi002 ceph-mon[30563]: pgmap v607: 129 pgs: 129 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-09-10T08:15:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.649+0000 7f0380000640 1 -- 172.21.15.2:0/2921091095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0378076230 msgr2=0x7f0378074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.649+0000 7f0380000640 1 --2- 172.21.15.2:0/2921091095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0378076230 0x7f0378074680 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f036c009900 tx=0x7f036c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.649+0000 7f0380000640 1 -- 172.21.15.2:0/2921091095 shutdown_connections 2024-09-10T08:15:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.649+0000 7f0380000640 1 --2- 172.21.15.2:0/2921091095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03781099d0 0x7f037810be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.649+0000 7f0380000640 1 --2- 172.21.15.2:0/2921091095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0378074c50 0x7f03780750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.649+0000 7f0380000640 1 --2- 172.21.15.2:0/2921091095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0378076230 0x7f0378074680 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.649+0000 7f0380000640 1 -- 172.21.15.2:0/2921091095 >> 172.21.15.2:0/2921091095 conn(0x7f03780fd2e0 msgr2=0x7f03780ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.649+0000 7f0380000640 1 -- 172.21.15.2:0/2921091095 shutdown_connections 2024-09-10T08:15:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.649+0000 7f0380000640 1 -- 172.21.15.2:0/2921091095 wait complete. 2024-09-10T08:15:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.650+0000 7f0380000640 1 Processor -- start 2024-09-10T08:15:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.650+0000 7f0380000640 1 -- start start 2024-09-10T08:15:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.650+0000 7f0380000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0378074c50 0x7f0378199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.650+0000 7f0380000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0378076230 0x7f037819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.650+0000 7f0380000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03781099d0 0x7f03781a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.650+0000 7f0380000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f037810fa20 con 0x7f0378076230 2024-09-10T08:15:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.650+0000 7f0380000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f037810f8a0 con 0x7f0378074c50 2024-09-10T08:15:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.650+0000 7f0380000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f037810fba0 con 0x7f03781099d0 2024-09-10T08:15:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.651+0000 7f037cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0378076230 0x7f037819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.651+0000 7f037e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03781099d0 0x7f03781a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.651+0000 7f037d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0378074c50 0x7f0378199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.651+0000 7f037e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03781099d0 0x7f03781a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36660/0 (socket says 172.21.15.2:36660) 2024-09-10T08:15:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.651+0000 7f037e000640 1 -- 172.21.15.2:0/4035240778 learned_addr learned my addr 172.21.15.2:0/4035240778 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.651+0000 7f037e000640 1 -- 172.21.15.2:0/4035240778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0378074c50 msgr2=0x7f0378199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.651+0000 7f037e000640 1 --2- 172.21.15.2:0/4035240778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0378074c50 0x7f0378199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.651+0000 7f037e000640 1 -- 172.21.15.2:0/4035240778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0378076230 msgr2=0x7f037819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.651+0000 7f037e000640 1 --2- 172.21.15.2:0/4035240778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0378076230 0x7f037819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.651+0000 7f037e000640 1 -- 172.21.15.2:0/4035240778 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03781a1bd0 con 0x7f03781099d0 2024-09-10T08:15:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.651+0000 7f037e000640 1 --2- 172.21.15.2:0/4035240778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03781099d0 0x7f03781a1560 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f037000eb10 tx=0x7f037000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.652+0000 7f0376000640 1 -- 172.21.15.2:0/4035240778 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370004b20 con 0x7f03781099d0 2024-09-10T08:15:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.652+0000 7f0376000640 1 -- 172.21.15.2:0/4035240778 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0370004cc0 con 0x7f03781099d0 2024-09-10T08:15:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.652+0000 7f0380000640 1 -- 172.21.15.2:0/4035240778 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03781a1ec0 con 0x7f03781099d0 2024-09-10T08:15:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.652+0000 7f0376000640 1 -- 172.21.15.2:0/4035240778 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370010760 con 0x7f03781099d0 2024-09-10T08:15:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.652+0000 7f0380000640 1 -- 172.21.15.2:0/4035240778 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03781a2380 con 0x7f03781099d0 2024-09-10T08:15:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.655+0000 7f0382ea9740 1 Processor -- start 2024-09-10T08:15:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.655+0000 7f0376000640 1 -- 172.21.15.2:0/4035240778 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0370005020 con 0x7f03781099d0 2024-09-10T08:15:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.655+0000 7f0382ea9740 1 -- start start 2024-09-10T08:15:31.658 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:31 smithi155 ceph-mon[35405]: pgmap v608: 129 pgs: 129 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-09-10T08:15:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.655+0000 7f0382ea9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f64090 0x55a832f64490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.655+0000 7f0382ea9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f649d0 0x55a832f69490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f0382ea9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 0x55a832f6bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f0382ea9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a832ec9100 con 0x55a832f69a60 2024-09-10T08:15:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f0382ea9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a832e711b0 con 0x55a832f64090 2024-09-10T08:15:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f0382ea9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a832ec50c0 con 0x55a832f649d0 2024-09-10T08:15:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f037cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 0x55a832f6bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f037cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 0x55a832f6bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47790/0 (socket says 172.21.15.2:47790) 2024-09-10T08:15:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f037cc00640 1 -- 172.21.15.2:0/401074206 learned_addr learned my addr 172.21.15.2:0/401074206 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f037d600640 1 --2- 172.21.15.2:0/401074206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f649d0 0x55a832f69490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f037cc00640 1 --2- 172.21.15.2:0/401074206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f64090 0x55a832f64490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f037d600640 1 -- 172.21.15.2:0/401074206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f64090 msgr2=0x55a832f64490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f037d600640 1 --2- 172.21.15.2:0/401074206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f64090 0x55a832f64490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f037d600640 1 -- 172.21.15.2:0/401074206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 msgr2=0x55a832f6bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f0376000640 1 --2- 172.21.15.2:0/4035240778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f03600a2c30 0x7f03600a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f037d600640 1 --2- 172.21.15.2:0/401074206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 0x55a832f6bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f037d600640 1 -- 172.21.15.2:0/401074206 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a832bdc210 con 0x55a832f649d0 2024-09-10T08:15:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.656+0000 7f037cc00640 1 --2- 172.21.15.2:0/401074206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 0x55a832f6bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.657+0000 7f0376000640 1 -- 172.21.15.2:0/4035240778 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f03700cfb30 con 0x7f03781099d0 2024-09-10T08:15:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.657+0000 7f037e000640 1 --2- 172.21.15.2:0/4035240778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f03600a2c30 0x7f03600a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.657+0000 7f037d600640 1 --2- 172.21.15.2:0/401074206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f649d0 0x55a832f69490 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f036c02a910 tx=0x7f036c005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4ab8931e0cdb980 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.657+0000 7f036a600640 1 -- 172.21.15.2:0/401074206 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c035040 con 0x55a832f649d0 2024-09-10T08:15:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.657+0000 7f036a600640 1 -- 172.21.15.2:0/401074206 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c005520 con 0x55a832f649d0 2024-09-10T08:15:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.657+0000 7f036a600640 1 -- 172.21.15.2:0/401074206 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c0316b0 con 0x55a832f649d0 2024-09-10T08:15:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.658+0000 7f0382ea9740 1 -- 172.21.15.2:0/401074206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f649d0 msgr2=0x55a832f69490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.658+0000 7f0382ea9740 1 --2- 172.21.15.2:0/401074206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f649d0 0x55a832f69490 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f036c02a910 tx=0x7f036c005c00 comp rx=0 tx=0).stop 2024-09-10T08:15:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.658+0000 7f037e000640 1 --2- 172.21.15.2:0/4035240778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f03600a2c30 0x7f03600a50f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f03700a4680 tx=0x7f037000efd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.658+0000 7f0382ea9740 1 -- 172.21.15.2:0/401074206 shutdown_connections 2024-09-10T08:15:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.658+0000 7f0382ea9740 1 --2- 172.21.15.2:0/401074206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 0x55a832f6bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.658+0000 7f0382ea9740 1 --2- 172.21.15.2:0/401074206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f649d0 0x55a832f69490 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.658+0000 7f0382ea9740 1 --2- 172.21.15.2:0/401074206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f64090 0x55a832f64490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.658+0000 7f0382ea9740 1 -- 172.21.15.2:0/401074206 >> 172.21.15.2:0/401074206 conn(0x55a832f5f700 msgr2=0x55a832f5fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.658+0000 7f0382ea9740 1 -- 172.21.15.2:0/401074206 shutdown_connections 2024-09-10T08:15:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.658+0000 7f0382ea9740 1 -- 172.21.15.2:0/401074206 wait complete. 2024-09-10T08:15:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.658+0000 7f0382ea9740 1 -- start start 2024-09-10T08:15:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.659+0000 7f0382ea9740 1 Processor -- start 2024-09-10T08:15:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.659+0000 7f0382ea9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f64090 0x55a832f6e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.659+0000 7f0382ea9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f649d0 0x55a832f6e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.659+0000 7f0382ea9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f69a60 0x55a832f75790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.659+0000 7f0382ea9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a832f6c9d0 con 0x55a832f649d0 2024-09-10T08:15:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.659+0000 7f0382ea9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a832e52090 con 0x55a832f69a60 2024-09-10T08:15:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.659+0000 7f0382ea9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a832f6cb50 con 0x55a832f64090 2024-09-10T08:15:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.659+0000 7f0382ea9740 10 client.0 _ll_register_callbacks cb 0x55a832ea6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.659+0000 7f037d600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f649d0 0x55a832f6e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.659+0000 7f037d600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f649d0 0x55a832f6e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47804/0 (socket says 172.21.15.2:47804) 2024-09-10T08:15:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.659+0000 7f037d600640 1 -- 172.21.15.2:0/150909307 learned_addr learned my addr 172.21.15.2:0/150909307 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.660+0000 7f037cc00640 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f64090 0x55a832f6e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.660+0000 7f037cc00640 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f69a60 0x55a832f75790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.660+0000 7f037d600640 1 -- 172.21.15.2:0/150909307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f64090 msgr2=0x55a832f6e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.660+0000 7f037d600640 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f64090 0x55a832f6e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.660+0000 7f037d600640 1 -- 172.21.15.2:0/150909307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f69a60 msgr2=0x55a832f75790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.660+0000 7f037d600640 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f69a60 0x55a832f75790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.660+0000 7f037d600640 1 -- 172.21.15.2:0/150909307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a832ca8620 con 0x55a832f649d0 2024-09-10T08:15:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.660+0000 7f037cc00640 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f69a60 0x55a832f75790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.660+0000 7f037d600640 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f649d0 0x55a832f6e750 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f036c0314e0 tx=0x7f036c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6637cddd1a7b49fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.660+0000 7f035ea00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.660+0000 7f035ea00640 1 -- 172.21.15.2:0/150909307 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c052060 con 0x55a832f649d0 2024-09-10T08:15:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.660+0000 7f0382ea9740 1 -- 172.21.15.2:0/150909307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f036c005520 con 0x55a832f649d0 2024-09-10T08:15:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.660+0000 7f035ea00640 1 -- 172.21.15.2:0/150909307 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f036c004e50 con 0x55a832f649d0 2024-09-10T08:15:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.661+0000 7f035ea00640 1 -- 172.21.15.2:0/150909307 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f036c035070 con 0x55a832f649d0 2024-09-10T08:15:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.661+0000 7f0382ea9740 10 client.17418 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.661+0000 7f0382ea9740 1 -- 172.21.15.2:0/150909307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a832f78580 con 0x55a832f649d0 2024-09-10T08:15:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.661+0000 7f035ea00640 1 -- 172.21.15.2:0/150909307 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c005520 con 0x55a832f649d0 2024-09-10T08:15:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.661+0000 7f035ea00640 1 -- 172.21.15.2:0/150909307 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f036c0051f0 con 0x55a832f649d0 2024-09-10T08:15:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.661+0000 7f0382ea9740 10 client.17418 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.661+0000 7f0382ea9740 10 client.17418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.661+0000 7f0382ea9740 20 client.17418 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.661+0000 7f0382ea9740 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a832f798d0 0x55a832f7bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.661+0000 7f0382ea9740 4 client.17418 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.661+0000 7f0382ea9740 1 -- 172.21.15.2:0/150909307 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a832ed6830 con 0x55a832f798d0 2024-09-10T08:15:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.662+0000 7f037cc00640 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a832f798d0 0x55a832f7bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.662+0000 7f037cc00640 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a832f798d0 0x55a832f7bcc0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e4ddb2299a4a1db server_cookie=2411d3708c5a5851 in_seq=0 out_seq=0 2024-09-10T08:15:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.662+0000 7f035ea00640 10 client.17418 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:31.669 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:31 smithi107 ceph-mon[35048]: pgmap v608: 129 pgs: 129 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-09-10T08:15:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.663+0000 7f035ea00640 1 -- 172.21.15.2:0/150909307 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a832ed6830 con 0x55a832f798d0 2024-09-10T08:15:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.663+0000 7f035ea00640 10 client.17418 handle_command_reply: tid=0 2024-09-10T08:15:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.663+0000 7f0382ea9740 1 client.17418 shutdown 2024-09-10T08:15:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.663+0000 7f0382ea9740 1 -- 172.21.15.2:0/150909307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f649d0 msgr2=0x55a832f6e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.663+0000 7f0382ea9740 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f649d0 0x55a832f6e750 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f036c0314e0 tx=0x7f036c004530 comp rx=0 tx=0).stop 2024-09-10T08:15:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.663+0000 7f0382ea9740 1 -- 172.21.15.2:0/150909307 shutdown_connections 2024-09-10T08:15:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.663+0000 7f0382ea9740 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a832f798d0 0x55a832f7bcc0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.663+0000 7f0382ea9740 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f69a60 0x55a832f75790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.663+0000 7f0382ea9740 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f649d0 0x55a832f6e750 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.663+0000 7f0382ea9740 1 --2- 172.21.15.2:0/150909307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f64090 0x55a832f6e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.663+0000 7f0382ea9740 1 -- 172.21.15.2:0/150909307 >> 172.21.15.2:0/150909307 conn(0x55a832f5f700 msgr2=0x55a832f60f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.663+0000 7f035ea00640 0 client.17418 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.664+0000 7f0382ea9740 1 -- 172.21.15.2:0/150909307 shutdown_connections 2024-09-10T08:15:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.664+0000 7f0382ea9740 1 -- 172.21.15.2:0/150909307 wait complete. 2024-09-10T08:15:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.664+0000 7f0382ea9740 20 client.17418 trim_cache size 0 max 16384 2024-09-10T08:15:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.679+0000 7f0382ea9740 1 Processor -- start 2024-09-10T08:15:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.679+0000 7f0382ea9740 1 -- start start 2024-09-10T08:15:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.679+0000 7f0382ea9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f64090 0x55a832f60580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f0382ea9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f649d0 0x55a832f75750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f0382ea9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f69a60 0x55a832f60ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f0382ea9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03781af310 con 0x55a832f64090 2024-09-10T08:15:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f0382ea9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03781acbe0 con 0x55a832f69a60 2024-09-10T08:15:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f0382ea9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03781aaa30 con 0x55a832f649d0 2024-09-10T08:15:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f037cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f64090 0x55a832f60580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f037d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f649d0 0x55a832f75750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f037d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f649d0 0x55a832f75750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36684/0 (socket says 172.21.15.2:36684) 2024-09-10T08:15:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f037d600640 1 -- 172.21.15.2:0/1583804165 learned_addr learned my addr 172.21.15.2:0/1583804165 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f037cc00640 1 --2- 172.21.15.2:0/1583804165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f69a60 0x55a832f60ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f037d600640 1 -- 172.21.15.2:0/1583804165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f69a60 msgr2=0x55a832f60ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f037d600640 1 --2- 172.21.15.2:0/1583804165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f69a60 0x55a832f60ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f037d600640 1 -- 172.21.15.2:0/1583804165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f64090 msgr2=0x55a832f60580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f037d600640 1 --2- 172.21.15.2:0/1583804165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f64090 0x55a832f60580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f037d600640 1 -- 172.21.15.2:0/1583804165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a832f6dfc0 con 0x55a832f649d0 2024-09-10T08:15:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f037cc00640 1 --2- 172.21.15.2:0/1583804165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f64090 0x55a832f60580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.680+0000 7f037d600640 1 --2- 172.21.15.2:0/1583804165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f649d0 0x55a832f75750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f036c02a910 tx=0x7f036c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=539468ed1a997b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.681+0000 7f0369c00640 1 -- 172.21.15.2:0/1583804165 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c052070 con 0x55a832f649d0 2024-09-10T08:15:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.681+0000 7f0369c00640 1 -- 172.21.15.2:0/1583804165 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c0038b0 con 0x55a832f649d0 2024-09-10T08:15:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.681+0000 7f0369c00640 1 -- 172.21.15.2:0/1583804165 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c005310 con 0x55a832f649d0 2024-09-10T08:15:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.681+0000 7f0382ea9740 1 -- 172.21.15.2:0/1583804165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f649d0 msgr2=0x55a832f75750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.681+0000 7f0382ea9740 1 --2- 172.21.15.2:0/1583804165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f649d0 0x55a832f75750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f036c02a910 tx=0x7f036c0046c0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.681+0000 7f0382ea9740 1 -- 172.21.15.2:0/1583804165 shutdown_connections 2024-09-10T08:15:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.681+0000 7f0382ea9740 1 --2- 172.21.15.2:0/1583804165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f69a60 0x55a832f60ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.681+0000 7f0382ea9740 1 --2- 172.21.15.2:0/1583804165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f649d0 0x55a832f75750 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.681+0000 7f0382ea9740 1 --2- 172.21.15.2:0/1583804165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f64090 0x55a832f60580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.681+0000 7f0382ea9740 1 -- 172.21.15.2:0/1583804165 >> 172.21.15.2:0/1583804165 conn(0x55a832f5f700 msgr2=0x55a832f6d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.681+0000 7f0382ea9740 1 -- 172.21.15.2:0/1583804165 shutdown_connections 2024-09-10T08:15:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.681+0000 7f0382ea9740 1 -- 172.21.15.2:0/1583804165 wait complete. 2024-09-10T08:15:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.682+0000 7f0382ea9740 1 -- start start 2024-09-10T08:15:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.682+0000 7f0382ea9740 1 Processor -- start 2024-09-10T08:15:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.682+0000 7f0382ea9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f64090 0x55a832f72c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.682+0000 7f0382ea9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f649d0 0x55a832f731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.682+0000 7f0382ea9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 0x55a832f741d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.682+0000 7f0382ea9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03781ab0a0 con 0x55a832f69a60 2024-09-10T08:15:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.682+0000 7f0382ea9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03781ac510 con 0x55a832f649d0 2024-09-10T08:15:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.682+0000 7f0382ea9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03781ab770 con 0x55a832f64090 2024-09-10T08:15:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f0382ea9740 10 client.0 _ll_register_callbacks cb 0x7f0364009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f037cc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f64090 0x55a832f72c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f037cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 0x55a832f741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f037d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f649d0 0x55a832f731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f037cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 0x55a832f741d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47820/0 (socket says 172.21.15.2:47820) 2024-09-10T08:15:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f037d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f649d0 0x55a832f731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:57338/0 (socket says 172.21.15.2:57338) 2024-09-10T08:15:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f037cc00640 1 -- 172.21.15.2:0/2360993586 learned_addr learned my addr 172.21.15.2:0/2360993586 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f037cc00640 1 -- 172.21.15.2:0/2360993586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f64090 msgr2=0x55a832f72c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f037cc00640 1 --2- 172.21.15.2:0/2360993586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f64090 0x55a832f72c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f037cc00640 1 -- 172.21.15.2:0/2360993586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f649d0 msgr2=0x55a832f731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f037cc00640 1 --2- 172.21.15.2:0/2360993586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f649d0 0x55a832f731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f037cc00640 1 -- 172.21.15.2:0/2360993586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a832f747e0 con 0x55a832f69a60 2024-09-10T08:15:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f037cc00640 1 --2- 172.21.15.2:0/2360993586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 0x55a832f741d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0364009fa0 tx=0x7f0364013720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64835876c43de519 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.683+0000 7f035ea00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.684+0000 7f035ea00640 1 -- 172.21.15.2:0/2360993586 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364013cf0 con 0x55a832f69a60 2024-09-10T08:15:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.684+0000 7f035ea00640 1 -- 172.21.15.2:0/2360993586 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f036402d450 con 0x55a832f69a60 2024-09-10T08:15:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.684+0000 7f0382ea9740 1 -- 172.21.15.2:0/2360993586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f036c0038b0 con 0x55a832f69a60 2024-09-10T08:15:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.684+0000 7f035ea00640 1 -- 172.21.15.2:0/2360993586 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f036402d650 con 0x55a832f69a60 2024-09-10T08:15:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.684+0000 7f0382ea9740 10 client.17424 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.684+0000 7f0382ea9740 1 -- 172.21.15.2:0/2360993586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a832fe44c0 con 0x55a832f69a60 2024-09-10T08:15:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.684+0000 7f035ea00640 1 -- 172.21.15.2:0/2360993586 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c0038b0 con 0x55a832f69a60 2024-09-10T08:15:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.684+0000 7f035ea00640 1 -- 172.21.15.2:0/2360993586 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0364022750 con 0x55a832f69a60 2024-09-10T08:15:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.685+0000 7f0382ea9740 10 client.17424 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.685+0000 7f0382ea9740 10 client.17424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.685+0000 7f0382ea9740 20 client.17424 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.685+0000 7f0382ea9740 1 --2- 172.21.15.2:0/2360993586 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a832fe4d50 0x55a832fe7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.685+0000 7f0382ea9740 4 client.17424 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:15:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.685+0000 7f0382ea9740 1 -- 172.21.15.2:0/2360993586 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a832ed0b80 con 0x55a832fe4d50 2024-09-10T08:15:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.685+0000 7f037d600640 1 --2- 172.21.15.2:0/2360993586 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a832fe4d50 0x55a832fe7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.685+0000 7f037d600640 1 --2- 172.21.15.2:0/2360993586 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a832fe4d50 0x55a832fe7170 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6753b4c133b2b505 server_cookie=9e901df71500be3c in_seq=0 out_seq=0 2024-09-10T08:15:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.685+0000 7f035ea00640 10 client.17424 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f035ea00640 1 -- 172.21.15.2:0/2360993586 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a832ed0b80 con 0x55a832fe4d50 2024-09-10T08:15:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f035ea00640 10 client.17424 handle_command_reply: tid=0 2024-09-10T08:15:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f0382ea9740 1 client.17424 shutdown 2024-09-10T08:15:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f0382ea9740 1 -- 172.21.15.2:0/2360993586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 msgr2=0x55a832f741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f0382ea9740 1 --2- 172.21.15.2:0/2360993586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 0x55a832f741d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0364009fa0 tx=0x7f0364013720 comp rx=0 tx=0).stop 2024-09-10T08:15:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f0382ea9740 1 -- 172.21.15.2:0/2360993586 shutdown_connections 2024-09-10T08:15:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f0382ea9740 1 --2- 172.21.15.2:0/2360993586 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a832fe4d50 0x55a832fe7170 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f0382ea9740 1 --2- 172.21.15.2:0/2360993586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a832f69a60 0x55a832f741d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f0382ea9740 1 --2- 172.21.15.2:0/2360993586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a832f649d0 0x55a832f731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f0382ea9740 1 --2- 172.21.15.2:0/2360993586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a832f64090 0x55a832f72c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f035ea00640 0 client.17424 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f0382ea9740 1 -- 172.21.15.2:0/2360993586 >> 172.21.15.2:0/2360993586 conn(0x55a832f5f700 msgr2=0x55a832f5e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f0382ea9740 1 -- 172.21.15.2:0/2360993586 shutdown_connections 2024-09-10T08:15:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f0382ea9740 1 -- 172.21.15.2:0/2360993586 wait complete. 2024-09-10T08:15:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.697+0000 7f0382ea9740 20 client.17424 trim_cache size 0 max 16384 2024-09-10T08:15:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:15:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:15:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:15:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:15:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.698+0000 7f0380000640 1 -- 172.21.15.2:0/4035240778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f03600a2c30 msgr2=0x7f03600a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.698+0000 7f0380000640 1 --2- 172.21.15.2:0/4035240778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f03600a2c30 0x7f03600a50f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f03700a4680 tx=0x7f037000efd0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.698+0000 7f0380000640 1 -- 172.21.15.2:0/4035240778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03781099d0 msgr2=0x7f03781a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.698+0000 7f0380000640 1 --2- 172.21.15.2:0/4035240778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03781099d0 0x7f03781a1560 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f037000eb10 tx=0x7f037000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.698+0000 7f0380000640 1 -- 172.21.15.2:0/4035240778 shutdown_connections 2024-09-10T08:15:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.698+0000 7f0380000640 1 --2- 172.21.15.2:0/4035240778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f03600a2c30 0x7f03600a50f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.698+0000 7f0380000640 1 --2- 172.21.15.2:0/4035240778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03781099d0 0x7f03781a1560 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.698+0000 7f0380000640 1 --2- 172.21.15.2:0/4035240778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0378076230 0x7f037819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.698+0000 7f0380000640 1 --2- 172.21.15.2:0/4035240778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0378074c50 0x7f0378199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.698+0000 7f0380000640 1 -- 172.21.15.2:0/4035240778 >> 172.21.15.2:0/4035240778 conn(0x7f03780fd2e0 msgr2=0x7f0378105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.698+0000 7f0380000640 1 -- 172.21.15.2:0/4035240778 shutdown_connections 2024-09-10T08:15:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:31.698+0000 7f0380000640 1 -- 172.21.15.2:0/4035240778 wait complete. 2024-09-10T08:15:31.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:31 smithi002 ceph-mon[30563]: pgmap v608: 129 pgs: 129 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-09-10T08:15:33.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:33 smithi107 ceph-mon[35048]: pgmap v609: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:15:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:33 smithi002 ceph-mon[30563]: pgmap v609: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:15:33.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:33 smithi155 ceph-mon[35405]: pgmap v609: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:15:35.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:35 smithi107 ceph-mon[35048]: pgmap v610: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T08:15:35.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:35 smithi002 ceph-mon[30563]: pgmap v610: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T08:15:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:35 smithi155 ceph-mon[35405]: pgmap v610: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T08:15:37.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:37 smithi107 ceph-mon[35048]: pgmap v611: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-10T08:15:37.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:37 smithi002 ceph-mon[30563]: pgmap v611: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-10T08:15:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:37 smithi155 ceph-mon[35405]: pgmap v611: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-10T08:15:39.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:39 smithi002 ceph-mon[30563]: pgmap v612: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:15:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:39 smithi155 ceph-mon[35405]: pgmap v612: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:15:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:39 smithi107 ceph-mon[35048]: pgmap v612: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:15:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:41 smithi002 ceph-mon[30563]: pgmap v613: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:15:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:41 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:15:41.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:41 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:15:41.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:41 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:15:41.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:41 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:15:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:41 smithi155 ceph-mon[35405]: pgmap v613: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:15:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:41 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:15:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:41 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:15:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:41 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:15:41.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:41 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:15:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:41 smithi107 ceph-mon[35048]: pgmap v613: 129 pgs: 129 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:15:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:41 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:15:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:41 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:15:41.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:41 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:15:41.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:41 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:15:43.262 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:15:43.655 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:43.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:43 smithi002 ceph-mon[30563]: pgmap v614: 129 pgs: 129 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:15:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:43 smithi155 ceph-mon[35405]: pgmap v614: 129 pgs: 129 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:15:43.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:43 smithi107 ceph-mon[35048]: pgmap v614: 129 pgs: 129 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:15:44.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.231+0000 7f4d21600640 1 -- 172.21.15.2:0/591134876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d1c102790 msgr2=0x7f4d1c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.231+0000 7f4d21600640 1 --2- 172.21.15.2:0/591134876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d1c102790 0x7f4d1c1067e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009900 tx=0x7f4d0402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.231+0000 7f4d21600640 1 -- 172.21.15.2:0/591134876 shutdown_connections 2024-09-10T08:15:44.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.231+0000 7f4d21600640 1 --2- 172.21.15.2:0/591134876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c106f10 0x7f4d1c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.231+0000 7f4d21600640 1 --2- 172.21.15.2:0/591134876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d1c102790 0x7f4d1c1067e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.231+0000 7f4d21600640 1 --2- 172.21.15.2:0/591134876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d1c101de0 0x7f4d1c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.231+0000 7f4d21600640 1 -- 172.21.15.2:0/591134876 >> 172.21.15.2:0/591134876 conn(0x7f4d1c0fd8c0 msgr2=0x7f4d1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:44.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.231+0000 7f4d21600640 1 -- 172.21.15.2:0/591134876 shutdown_connections 2024-09-10T08:15:44.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.231+0000 7f4d21600640 1 -- 172.21.15.2:0/591134876 wait complete. 2024-09-10T08:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.232+0000 7f4d21600640 1 Processor -- start 2024-09-10T08:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.232+0000 7f4d21600640 1 -- start start 2024-09-10T08:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.232+0000 7f4d21600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d1c101de0 0x7f4d1c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.232+0000 7f4d21600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c102790 0x7f4d1c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.232+0000 7f4d21600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d1c106f10 0x7f4d1c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.232+0000 7f4d21600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c111fc0 con 0x7f4d1c101de0 2024-09-10T08:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.232+0000 7f4d21600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c111e40 con 0x7f4d1c102790 2024-09-10T08:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.232+0000 7f4d21600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c112140 con 0x7f4d1c106f10 2024-09-10T08:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.233+0000 7f4d1aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d1c101de0 0x7f4d1c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.233+0000 7f4d1aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d1c101de0 0x7f4d1c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53710/0 (socket says 172.21.15.2:53710) 2024-09-10T08:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.233+0000 7f4d1aa00640 1 -- 172.21.15.2:0/3570566100 learned_addr learned my addr 172.21.15.2:0/3570566100 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.233+0000 7f4d13e00640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c102790 0x7f4d1c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.233+0000 7f4d1b400640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d1c106f10 0x7f4d1c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.233+0000 7f4d1aa00640 1 -- 172.21.15.2:0/3570566100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d1c106f10 msgr2=0x7f4d1c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.233+0000 7f4d1aa00640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d1c106f10 0x7f4d1c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.233+0000 7f4d1aa00640 1 -- 172.21.15.2:0/3570566100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c102790 msgr2=0x7f4d1c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.233+0000 7f4d1aa00640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c102790 0x7f4d1c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.233+0000 7f4d1aa00640 1 -- 172.21.15.2:0/3570566100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c1a6920 con 0x7f4d1c101de0 2024-09-10T08:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.234+0000 7f4d1aa00640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d1c101de0 0x7f4d1c0744f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00e9f0 tx=0x7f4d0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.234+0000 7f4d13400640 1 -- 172.21.15.2:0/3570566100 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00cd80 con 0x7f4d1c101de0 2024-09-10T08:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.234+0000 7f4d13400640 1 -- 172.21.15.2:0/3570566100 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d0c005500 con 0x7f4d1c101de0 2024-09-10T08:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.234+0000 7f4d21600640 1 -- 172.21.15.2:0/3570566100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1c1a6c10 con 0x7f4d1c101de0 2024-09-10T08:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.234+0000 7f4d13400640 1 -- 172.21.15.2:0/3570566100 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c010690 con 0x7f4d1c101de0 2024-09-10T08:15:44.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.234+0000 7f4d21600640 1 -- 172.21.15.2:0/3570566100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1c1a70d0 con 0x7f4d1c101de0 2024-09-10T08:15:44.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.237+0000 7f4d13400640 1 -- 172.21.15.2:0/3570566100 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4d0c0046c0 con 0x7f4d1c101de0 2024-09-10T08:15:44.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.237+0000 7f4d2408b740 1 Processor -- start 2024-09-10T08:15:44.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.237+0000 7f4d2408b740 1 -- start start 2024-09-10T08:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.238+0000 7f4d2408b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddccb90 0x5595fddccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.238+0000 7f4d2408b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddcd4b0 0x5595fddd7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.238+0000 7f4d2408b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 0x5595fddda160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.238+0000 7f4d2408b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5595fdd333e0 con 0x5595fddcd4b0 2024-09-10T08:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.238+0000 7f4d2408b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5595fdcdad80 con 0x5595fddccb90 2024-09-10T08:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.238+0000 7f4d2408b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5595fdd2f3a0 con 0x5595fddd7d70 2024-09-10T08:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.238+0000 7f4d1aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddcd4b0 0x5595fddd7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.238+0000 7f4d1aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddcd4b0 0x5595fddd7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53712/0 (socket says 172.21.15.2:53712) 2024-09-10T08:15:44.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.238+0000 7f4d1aa00640 1 -- 172.21.15.2:0/3581538327 learned_addr learned my addr 172.21.15.2:0/3581538327 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:44.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.238+0000 7f4d1b400640 1 --2- 172.21.15.2:0/3581538327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddccb90 0x5595fddccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.238+0000 7f4d13e00640 1 --2- 172.21.15.2:0/3581538327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 0x5595fddda160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.239+0000 7f4d1b400640 1 -- 172.21.15.2:0/3581538327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 msgr2=0x5595fddda160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.239+0000 7f4d1b400640 1 --2- 172.21.15.2:0/3581538327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 0x5595fddda160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.239+0000 7f4d1b400640 1 -- 172.21.15.2:0/3581538327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddcd4b0 msgr2=0x5595fddd7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.239+0000 7f4d1b400640 1 --2- 172.21.15.2:0/3581538327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddcd4b0 0x5595fddd7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.239+0000 7f4d1b400640 1 -- 172.21.15.2:0/3581538327 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595fdcf9ae0 con 0x5595fddccb90 2024-09-10T08:15:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.239+0000 7f4d1aa00640 1 --2- 172.21.15.2:0/3581538327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddcd4b0 0x5595fddd7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.239+0000 7f4d13400640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d000a2b80 0x7f4d000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.239+0000 7f4d1b400640 1 --2- 172.21.15.2:0/3581538327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddccb90 0x5595fddccf70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009eb0 tx=0x7f4d140107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0e3a06a7ce59082 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.239+0000 7f4d13400640 1 -- 172.21.15.2:0/3570566100 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4d0c0cf8b0 con 0x7f4d1c101de0 2024-09-10T08:15:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.239+0000 7f4d1aa00640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d000a2b80 0x7f4d000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.239+0000 7f4d0b400640 1 -- 172.21.15.2:0/3581538327 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14003b70 con 0x5595fddccb90 2024-09-10T08:15:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.240+0000 7f4d0b400640 1 -- 172.21.15.2:0/3581538327 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d14015070 con 0x5595fddccb90 2024-09-10T08:15:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.240+0000 7f4d0b400640 1 -- 172.21.15.2:0/3581538327 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1401a830 con 0x5595fddccb90 2024-09-10T08:15:44.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.240+0000 7f4d2408b740 1 -- 172.21.15.2:0/3581538327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddccb90 msgr2=0x5595fddccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.240+0000 7f4d1aa00640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d000a2b80 0x7f4d000a5040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0049d0 tx=0x7f4d0c0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.240+0000 7f4d2408b740 1 --2- 172.21.15.2:0/3581538327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddccb90 0x5595fddccf70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009eb0 tx=0x7f4d140107a0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.240+0000 7f4d2408b740 1 -- 172.21.15.2:0/3581538327 shutdown_connections 2024-09-10T08:15:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.240+0000 7f4d2408b740 1 --2- 172.21.15.2:0/3581538327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 0x5595fddda160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.240+0000 7f4d2408b740 1 --2- 172.21.15.2:0/3581538327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddcd4b0 0x5595fddd7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.240+0000 7f4d2408b740 1 --2- 172.21.15.2:0/3581538327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddccb90 0x5595fddccf70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.240+0000 7f4d2408b740 1 -- 172.21.15.2:0/3581538327 >> 172.21.15.2:0/3581538327 conn(0x5595fddc9120 msgr2=0x5595fddc9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.241+0000 7f4d2408b740 1 -- 172.21.15.2:0/3581538327 shutdown_connections 2024-09-10T08:15:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.241+0000 7f4d2408b740 1 -- 172.21.15.2:0/3581538327 wait complete. 2024-09-10T08:15:44.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.241+0000 7f4d2408b740 1 -- start start 2024-09-10T08:15:44.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.241+0000 7f4d2408b740 1 Processor -- start 2024-09-10T08:15:44.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.242+0000 7f4d2408b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddccb90 0x5595fddd4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.242+0000 7f4d2408b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fddd5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.242+0000 7f4d2408b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 0x5595fddd56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.242+0000 7f4d2408b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5595fddda9e0 con 0x5595fddccb90 2024-09-10T08:15:44.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.242+0000 7f4d2408b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5595fdcee0f0 con 0x5595fddcd4b0 2024-09-10T08:15:44.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.242+0000 7f4d2408b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5595fdddab60 con 0x5595fddd7d70 2024-09-10T08:15:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.242+0000 7f4d2408b740 10 client.0 _ll_register_callbacks cb 0x5595fdcdf9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.242+0000 7f4d1b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddccb90 0x5595fddd4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.242+0000 7f4d1b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddccb90 0x5595fddd4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53716/0 (socket says 172.21.15.2:53716) 2024-09-10T08:15:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.242+0000 7f4d1b400640 1 -- 172.21.15.2:0/461787863 learned_addr learned my addr 172.21.15.2:0/461787863 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.242+0000 7f4d13e00640 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fddd5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.242+0000 7f4d1b400640 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 0x5595fddd56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.243+0000 7f4d13e00640 1 -- 172.21.15.2:0/461787863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 msgr2=0x5595fddd56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.243+0000 7f4d13e00640 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 0x5595fddd56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.243+0000 7f4d13e00640 1 -- 172.21.15.2:0/461787863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddccb90 msgr2=0x5595fddd4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.243+0000 7f4d13e00640 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddccb90 0x5595fddd4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.243+0000 7f4d13e00640 1 -- 172.21.15.2:0/461787863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595fdaa43c0 con 0x5595fddcd4b0 2024-09-10T08:15:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.243+0000 7f4d1b400640 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddccb90 0x5595fddd4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.243+0000 7f4d13e00640 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fddd5190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4d0402a9e0 tx=0x7f4d04004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af82f7f240bf432c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.243+0000 7f4cffe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:15:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.243+0000 7f4cffe00640 1 -- 172.21.15.2:0/461787863 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04035040 con 0x5595fddcd4b0 2024-09-10T08:15:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.243+0000 7f4d2408b740 1 -- 172.21.15.2:0/461787863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d14015070 con 0x5595fddcd4b0 2024-09-10T08:15:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.243+0000 7f4cffe00640 1 -- 172.21.15.2:0/461787863 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4d04005450 con 0x5595fddcd4b0 2024-09-10T08:15:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.243+0000 7f4cffe00640 1 -- 172.21.15.2:0/461787863 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d0404f050 con 0x5595fddcd4b0 2024-09-10T08:15:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.244+0000 7f4d2408b740 10 client.27473 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.244+0000 7f4d2408b740 1 -- 172.21.15.2:0/461787863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595fdde6690 con 0x5595fddcd4b0 2024-09-10T08:15:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.244+0000 7f4cffe00640 1 -- 172.21.15.2:0/461787863 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d14015070 con 0x5595fddcd4b0 2024-09-10T08:15:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.244+0000 7f4cffe00640 1 -- 172.21.15.2:0/461787863 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4d04003920 con 0x5595fddcd4b0 2024-09-10T08:15:44.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.244+0000 7f4d2408b740 10 client.27473 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:44.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.244+0000 7f4d2408b740 10 client.27473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:44.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.244+0000 7f4d2408b740 20 client.27473 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:44.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.244+0000 7f4d2408b740 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5595fdde7b50 0x5595fdde9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.244+0000 7f4d2408b740 4 client.27473 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:44.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.244+0000 7f4d2408b740 1 -- 172.21.15.2:0/461787863 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595fdd40950 con 0x5595fdde7b50 2024-09-10T08:15:44.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.244+0000 7f4d1b400640 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5595fdde7b50 0x5595fdde9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.245+0000 7f4d1b400640 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5595fdde7b50 0x5595fdde9f40 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4444d814767a400d server_cookie=28aac69324fd59c in_seq=0 out_seq=0 2024-09-10T08:15:44.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.245+0000 7f4cffe00640 10 client.27473 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:44.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4cffe00640 1 -- 172.21.15.2:0/461787863 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595fdd40950 con 0x5595fdde7b50 2024-09-10T08:15:44.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4cffe00640 10 client.27473 handle_command_reply: tid=0 2024-09-10T08:15:44.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4d2408b740 1 client.27473 shutdown 2024-09-10T08:15:44.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4d2408b740 1 -- 172.21.15.2:0/461787863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 msgr2=0x5595fddd5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4d2408b740 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fddd5190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4d0402a9e0 tx=0x7f4d04004420 comp rx=0 tx=0).stop 2024-09-10T08:15:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4d2408b740 1 -- 172.21.15.2:0/461787863 shutdown_connections 2024-09-10T08:15:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4d2408b740 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5595fdde7b50 0x5595fdde9f40 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4d2408b740 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 0x5595fddd56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4d2408b740 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fddd5190 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4d2408b740 1 --2- 172.21.15.2:0/461787863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddccb90 0x5595fddd4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4cffe00640 0 client.27473 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4d2408b740 1 -- 172.21.15.2:0/461787863 >> 172.21.15.2:0/461787863 conn(0x5595fddc9120 msgr2=0x5595fddcee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4d2408b740 1 -- 172.21.15.2:0/461787863 shutdown_connections 2024-09-10T08:15:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4d2408b740 1 -- 172.21.15.2:0/461787863 wait complete. 2024-09-10T08:15:44.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.246+0000 7f4d2408b740 20 client.27473 trim_cache size 0 max 16384 2024-09-10T08:15:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.264+0000 7f4d2408b740 1 Processor -- start 2024-09-10T08:15:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.264+0000 7f4d2408b740 1 -- start start 2024-09-10T08:15:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d2408b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddccb90 0x5595fddcf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d2408b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fddd6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d2408b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddd7d70 0x5595fddef810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d2408b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1b3930 con 0x5595fddd7d70 2024-09-10T08:15:44.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d2408b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1b1200 con 0x5595fddcd4b0 2024-09-10T08:15:44.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d2408b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1af050 con 0x5595fddccb90 2024-09-10T08:15:44.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d1b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddccb90 0x5595fddcf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d1b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddccb90 0x5595fddcf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55186/0 (socket says 172.21.15.2:55186) 2024-09-10T08:15:44.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d1b400640 1 -- 172.21.15.2:0/1018420349 learned_addr learned my addr 172.21.15.2:0/1018420349 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:44.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d13e00640 1 --2- 172.21.15.2:0/1018420349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fddd6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d1b400640 1 --2- 172.21.15.2:0/1018420349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddd7d70 0x5595fddef810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d1b400640 1 -- 172.21.15.2:0/1018420349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 msgr2=0x5595fddd6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d1b400640 1 --2- 172.21.15.2:0/1018420349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fddd6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.265+0000 7f4d1b400640 1 -- 172.21.15.2:0/1018420349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddd7d70 msgr2=0x5595fddef810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.266+0000 7f4d1b400640 1 --2- 172.21.15.2:0/1018420349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddd7d70 0x5595fddef810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.266+0000 7f4d1b400640 1 -- 172.21.15.2:0/1018420349 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595fddc8f50 con 0x5595fddccb90 2024-09-10T08:15:44.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.266+0000 7f4d13e00640 1 --2- 172.21.15.2:0/1018420349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fddd6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:44.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.266+0000 7f4d1b400640 1 --2- 172.21.15.2:0/1018420349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddccb90 0x5595fddcf5f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d14005c60 tx=0x7f4d1401a470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc6c24738ded1679 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:44.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.266+0000 7f4d0aa00640 1 -- 172.21.15.2:0/1018420349 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14025040 con 0x5595fddccb90 2024-09-10T08:15:44.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.266+0000 7f4d0aa00640 1 -- 172.21.15.2:0/1018420349 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d14015030 con 0x5595fddccb90 2024-09-10T08:15:44.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.266+0000 7f4d0aa00640 1 -- 172.21.15.2:0/1018420349 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14005020 con 0x5595fddccb90 2024-09-10T08:15:44.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.267+0000 7f4d2408b740 1 -- 172.21.15.2:0/1018420349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddccb90 msgr2=0x5595fddcf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.267+0000 7f4d2408b740 1 --2- 172.21.15.2:0/1018420349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddccb90 0x5595fddcf5f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d14005c60 tx=0x7f4d1401a470 comp rx=0 tx=0).stop 2024-09-10T08:15:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.267+0000 7f4d2408b740 1 -- 172.21.15.2:0/1018420349 shutdown_connections 2024-09-10T08:15:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.267+0000 7f4d2408b740 1 --2- 172.21.15.2:0/1018420349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddd7d70 0x5595fddef810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.267+0000 7f4d2408b740 1 --2- 172.21.15.2:0/1018420349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fddd6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.267+0000 7f4d2408b740 1 --2- 172.21.15.2:0/1018420349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddccb90 0x5595fddcf5f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.267+0000 7f4d2408b740 1 -- 172.21.15.2:0/1018420349 >> 172.21.15.2:0/1018420349 conn(0x5595fddc9120 msgr2=0x5595fddd64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.267+0000 7f4d2408b740 1 -- 172.21.15.2:0/1018420349 shutdown_connections 2024-09-10T08:15:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.267+0000 7f4d2408b740 1 -- 172.21.15.2:0/1018420349 wait complete. 2024-09-10T08:15:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.267+0000 7f4d2408b740 1 -- start start 2024-09-10T08:15:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.268+0000 7f4d2408b740 1 Processor -- start 2024-09-10T08:15:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.268+0000 7f4d2408b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddccb90 0x5595fdde06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.268+0000 7f4d2408b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fdddb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.268+0000 7f4d2408b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 0x5595fdddc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.268+0000 7f4d2408b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1af6c0 con 0x5595fddccb90 2024-09-10T08:15:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.268+0000 7f4d2408b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1b0b30 con 0x5595fddcd4b0 2024-09-10T08:15:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.268+0000 7f4d2408b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1afd90 con 0x5595fddd7d70 2024-09-10T08:15:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.268+0000 7f4d2408b740 10 client.0 _ll_register_callbacks cb 0x7f4d040040b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.268+0000 7f4d1b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddccb90 0x5595fdde06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.268+0000 7f4d13e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fdddb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.269+0000 7f4d1b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 0x5595fdddc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.269+0000 7f4d13e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fdddb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38486/0 (socket says 172.21.15.2:38486) 2024-09-10T08:15:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.269+0000 7f4d13e00640 1 -- 172.21.15.2:0/912492747 learned_addr learned my addr 172.21.15.2:0/912492747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.269+0000 7f4d13e00640 1 -- 172.21.15.2:0/912492747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 msgr2=0x5595fdddc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.269+0000 7f4d13e00640 1 --2- 172.21.15.2:0/912492747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 0x5595fdddc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.269+0000 7f4d13e00640 1 -- 172.21.15.2:0/912492747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddccb90 msgr2=0x5595fdde06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.269+0000 7f4d13e00640 1 --2- 172.21.15.2:0/912492747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddccb90 0x5595fdde06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.269+0000 7f4d13e00640 1 -- 172.21.15.2:0/912492747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595fde517e0 con 0x5595fddcd4b0 2024-09-10T08:15:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.269+0000 7f4d13e00640 1 --2- 172.21.15.2:0/912492747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fdddb9d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4d04004650 tx=0x7f4d04005480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f488816589b024ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.269+0000 7f4cffe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:15:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.269+0000 7f4cffe00640 1 -- 172.21.15.2:0/912492747 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04035070 con 0x5595fddcd4b0 2024-09-10T08:15:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.270+0000 7f4d2408b740 1 -- 172.21.15.2:0/912492747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d14015030 con 0x5595fddcd4b0 2024-09-10T08:15:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.270+0000 7f4cffe00640 1 -- 172.21.15.2:0/912492747 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4d04005670 con 0x5595fddcd4b0 2024-09-10T08:15:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.270+0000 7f4cffe00640 1 -- 172.21.15.2:0/912492747 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d04003800 con 0x5595fddcd4b0 2024-09-10T08:15:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.270+0000 7f4d2408b740 10 client.27479 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.270+0000 7f4d2408b740 1 -- 172.21.15.2:0/912492747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595fde52990 con 0x5595fddcd4b0 2024-09-10T08:15:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.270+0000 7f4cffe00640 1 -- 172.21.15.2:0/912492747 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d14015030 con 0x5595fddcd4b0 2024-09-10T08:15:44.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.270+0000 7f4cffe00640 1 -- 172.21.15.2:0/912492747 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4d040046c0 con 0x5595fddcd4b0 2024-09-10T08:15:44.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.270+0000 7f4d2408b740 10 client.27479 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:44.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.270+0000 7f4d2408b740 10 client.27479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:44.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.270+0000 7f4d2408b740 20 client.27479 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:44.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.270+0000 7f4d2408b740 1 --2- 172.21.15.2:0/912492747 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5595fde53160 0x5595fde55580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.270+0000 7f4d2408b740 4 client.27479 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:15:44.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.270+0000 7f4d2408b740 1 -- 172.21.15.2:0/912492747 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5595fdd3ae60 con 0x5595fde53160 2024-09-10T08:15:44.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.271+0000 7f4d1b400640 1 --2- 172.21.15.2:0/912492747 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5595fde53160 0x5595fde55580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.271+0000 7f4d1b400640 1 --2- 172.21.15.2:0/912492747 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5595fde53160 0x5595fde55580 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa1c9a346f31435d server_cookie=24b92e6ce04999d4 in_seq=0 out_seq=0 2024-09-10T08:15:44.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.271+0000 7f4cffe00640 10 client.27479 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:44.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.271+0000 7f4cffe00640 1 -- 172.21.15.2:0/912492747 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5595fdd3ae60 con 0x5595fde53160 2024-09-10T08:15:44.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.271+0000 7f4cffe00640 10 client.27479 handle_command_reply: tid=0 2024-09-10T08:15:44.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.271+0000 7f4d2408b740 1 client.27479 shutdown 2024-09-10T08:15:44.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.272+0000 7f4d2408b740 1 -- 172.21.15.2:0/912492747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 msgr2=0x5595fdddb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.272+0000 7f4d2408b740 1 --2- 172.21.15.2:0/912492747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fdddb9d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4d04004650 tx=0x7f4d04005480 comp rx=0 tx=0).stop 2024-09-10T08:15:44.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.272+0000 7f4d2408b740 1 -- 172.21.15.2:0/912492747 shutdown_connections 2024-09-10T08:15:44.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.272+0000 7f4d2408b740 1 --2- 172.21.15.2:0/912492747 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5595fde53160 0x5595fde55580 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.272+0000 7f4d2408b740 1 --2- 172.21.15.2:0/912492747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595fddd7d70 0x5595fdddc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.272+0000 7f4d2408b740 1 --2- 172.21.15.2:0/912492747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5595fddcd4b0 0x5595fdddb9d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.272+0000 7f4d2408b740 1 --2- 172.21.15.2:0/912492747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595fddccb90 0x5595fdde06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.272+0000 7f4cffe00640 0 client.27479 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:44.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.272+0000 7f4d2408b740 1 -- 172.21.15.2:0/912492747 >> 172.21.15.2:0/912492747 conn(0x5595fddc9120 msgr2=0x5595fddd6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:44.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.272+0000 7f4d2408b740 1 -- 172.21.15.2:0/912492747 shutdown_connections 2024-09-10T08:15:44.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.272+0000 7f4d2408b740 1 -- 172.21.15.2:0/912492747 wait complete. 2024-09-10T08:15:44.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.272+0000 7f4d2408b740 20 client.27479 trim_cache size 0 max 16384 2024-09-10T08:15:44.276 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:15:44.276 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:15:44.276 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:15:44.276 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:15:44.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.273+0000 7f4d21600640 1 -- 172.21.15.2:0/3570566100 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d000a2b80 msgr2=0x7f4d000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.273+0000 7f4d21600640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d000a2b80 0x7f4d000a5040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0049d0 tx=0x7f4d0c0d5530 comp rx=0 tx=0).stop 2024-09-10T08:15:44.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.273+0000 7f4d21600640 1 -- 172.21.15.2:0/3570566100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d1c101de0 msgr2=0x7f4d1c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.273+0000 7f4d21600640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d1c101de0 0x7f4d1c0744f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00e9f0 tx=0x7f4d0c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.273+0000 7f4d21600640 1 -- 172.21.15.2:0/3570566100 shutdown_connections 2024-09-10T08:15:44.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.273+0000 7f4d21600640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4d000a2b80 0x7f4d000a5040 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.273+0000 7f4d21600640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d1c106f10 0x7f4d1c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.273+0000 7f4d21600640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c102790 0x7f4d1c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.273+0000 7f4d21600640 1 --2- 172.21.15.2:0/3570566100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d1c101de0 0x7f4d1c0744f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.273+0000 7f4d21600640 1 -- 172.21.15.2:0/3570566100 >> 172.21.15.2:0/3570566100 conn(0x7f4d1c0fd8c0 msgr2=0x7f4d1c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:44.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.273+0000 7f4d21600640 1 -- 172.21.15.2:0/3570566100 shutdown_connections 2024-09-10T08:15:44.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:44.273+0000 7f4d21600640 1 -- 172.21.15.2:0/3570566100 wait complete. 2024-09-10T08:15:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.817+0000 7f322c800640 1 -- 172.21.15.2:0/4139170928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3224105910 msgr2=0x7f322410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.817+0000 7f322c800640 1 --2- 172.21.15.2:0/4139170928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3224105910 0x7f322410c1a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3220009ca0 tx=0x7f322002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.817+0000 7f322c800640 1 -- 172.21.15.2:0/4139170928 shutdown_connections 2024-09-10T08:15:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.817+0000 7f322c800640 1 --2- 172.21.15.2:0/4139170928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3224105910 0x7f322410c1a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.817+0000 7f322c800640 1 --2- 172.21.15.2:0/4139170928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224104f50 0x7f32241053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.817+0000 7f322c800640 1 --2- 172.21.15.2:0/4139170928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3224103d60 0x7f3224104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.817+0000 7f322c800640 1 -- 172.21.15.2:0/4139170928 >> 172.21.15.2:0/4139170928 conn(0x7f32240ff4e0 msgr2=0x7f3224101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.817+0000 7f322c800640 1 -- 172.21.15.2:0/4139170928 shutdown_connections 2024-09-10T08:15:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.817+0000 7f322c800640 1 -- 172.21.15.2:0/4139170928 wait complete. 2024-09-10T08:15:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.818+0000 7f322c800640 1 Processor -- start 2024-09-10T08:15:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.818+0000 7f322c800640 1 -- start start 2024-09-10T08:15:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.818+0000 7f322c800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224103d60 0x7f3224199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.818+0000 7f322c800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3224104f50 0x7f322419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.818+0000 7f322c800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3224105910 0x7f32241a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.818+0000 7f322c800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f322410fa30 con 0x7f3224105910 2024-09-10T08:15:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.818+0000 7f322c800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f322410f8b0 con 0x7f3224104f50 2024-09-10T08:15:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.818+0000 7f322c800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f322410fbb0 con 0x7f3224103d60 2024-09-10T08:15:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.819+0000 7f3229400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3224104f50 0x7f322419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.819+0000 7f3229e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224103d60 0x7f3224199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.819+0000 7f3229400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3224104f50 0x7f322419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38502/0 (socket says 172.21.15.2:38502) 2024-09-10T08:15:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.819+0000 7f322a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3224105910 0x7f32241a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.819+0000 7f3229e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224103d60 0x7f3224199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55208/0 (socket says 172.21.15.2:55208) 2024-09-10T08:15:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.819+0000 7f3229400640 1 -- 172.21.15.2:0/1298134151 learned_addr learned my addr 172.21.15.2:0/1298134151 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.819+0000 7f3229400640 1 -- 172.21.15.2:0/1298134151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224103d60 msgr2=0x7f3224199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.819+0000 7f3229400640 1 --2- 172.21.15.2:0/1298134151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224103d60 0x7f3224199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.819+0000 7f3229400640 1 -- 172.21.15.2:0/1298134151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3224105910 msgr2=0x7f32241a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.819+0000 7f3229400640 1 --2- 172.21.15.2:0/1298134151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3224105910 0x7f32241a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.819+0000 7f3229400640 1 -- 172.21.15.2:0/1298134151 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32241a1b80 con 0x7f3224104f50 2024-09-10T08:15:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.819+0000 7f3229400640 1 --2- 172.21.15.2:0/1298134151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3224104f50 0x7f322419a4a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f321400e9f0 tx=0x7f321400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.820+0000 7f321ea00640 1 -- 172.21.15.2:0/1298134151 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321400cd80 con 0x7f3224104f50 2024-09-10T08:15:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.820+0000 7f321ea00640 1 -- 172.21.15.2:0/1298134151 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3214005500 con 0x7f3224104f50 2024-09-10T08:15:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.820+0000 7f322c800640 1 -- 172.21.15.2:0/1298134151 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32241a1e70 con 0x7f3224104f50 2024-09-10T08:15:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.820+0000 7f321ea00640 1 -- 172.21.15.2:0/1298134151 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214010690 con 0x7f3224104f50 2024-09-10T08:15:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.820+0000 7f322c800640 1 -- 172.21.15.2:0/1298134151 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32241a22e0 con 0x7f3224104f50 2024-09-10T08:15:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.823+0000 7f322f6aa740 1 Processor -- start 2024-09-10T08:15:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.823+0000 7f322f6aa740 1 -- start start 2024-09-10T08:15:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.823+0000 7f321ea00640 1 -- 172.21.15.2:0/1298134151 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f32140046c0 con 0x7f3224104f50 2024-09-10T08:15:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.823+0000 7f322f6aa740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db090 0x56277e5db490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.823+0000 7f322f6aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db9d0 0x56277e5e0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.823+0000 7f322f6aa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e5e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.823+0000 7f322f6aa740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56277e540100 con 0x56277e5db090 2024-09-10T08:15:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.823+0000 7f322f6aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56277e4e81b0 con 0x56277e5db9d0 2024-09-10T08:15:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.823+0000 7f322f6aa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56277e53c0c0 con 0x56277e5e0a60 2024-09-10T08:15:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.823+0000 7f322a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db090 0x56277e5db490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.823+0000 7f322a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db090 0x56277e5db490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53760/0 (socket says 172.21.15.2:53760) 2024-09-10T08:15:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.823+0000 7f322a800640 1 -- 172.21.15.2:0/2024620550 learned_addr learned my addr 172.21.15.2:0/2024620550 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.823+0000 7f3229400640 1 --2- 172.21.15.2:0/2024620550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e5e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.824+0000 7f3229e00640 1 --2- 172.21.15.2:0/2024620550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db9d0 0x56277e5e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.824+0000 7f322a800640 1 -- 172.21.15.2:0/2024620550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 msgr2=0x56277e5e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.824+0000 7f322a800640 1 --2- 172.21.15.2:0/2024620550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e5e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.824+0000 7f322a800640 1 -- 172.21.15.2:0/2024620550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db9d0 msgr2=0x56277e5e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.824+0000 7f322a800640 1 --2- 172.21.15.2:0/2024620550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db9d0 0x56277e5e0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.824+0000 7f322a800640 1 -- 172.21.15.2:0/2024620550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56277e253210 con 0x56277e5db090 2024-09-10T08:15:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.824+0000 7f3229e00640 1 --2- 172.21.15.2:0/2024620550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db9d0 0x56277e5e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.824+0000 7f322a800640 1 --2- 172.21.15.2:0/2024620550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db090 0x56277e5db490 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3220009ca0 tx=0x7f32200093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a696a4df3def0e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.824+0000 7f3213000640 1 -- 172.21.15.2:0/2024620550 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220031650 con 0x56277e5db090 2024-09-10T08:15:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f3213000640 1 -- 172.21.15.2:0/2024620550 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3220036030 con 0x56277e5db090 2024-09-10T08:15:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f3213000640 1 -- 172.21.15.2:0/2024620550 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220004720 con 0x56277e5db090 2024-09-10T08:15:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f321ea00640 1 --2- 172.21.15.2:0/1298134151 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f320c0a2c30 0x7f320c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f322f6aa740 1 -- 172.21.15.2:0/2024620550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db090 msgr2=0x56277e5db490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f322f6aa740 1 --2- 172.21.15.2:0/2024620550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db090 0x56277e5db490 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3220009ca0 tx=0x7f32200093c0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f321ea00640 1 -- 172.21.15.2:0/1298134151 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f32140cf800 con 0x7f3224104f50 2024-09-10T08:15:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f322f6aa740 1 -- 172.21.15.2:0/2024620550 shutdown_connections 2024-09-10T08:15:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f322f6aa740 1 --2- 172.21.15.2:0/2024620550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e5e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f322f6aa740 1 --2- 172.21.15.2:0/2024620550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db9d0 0x56277e5e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f322f6aa740 1 --2- 172.21.15.2:0/2024620550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db090 0x56277e5db490 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f322f6aa740 1 -- 172.21.15.2:0/2024620550 >> 172.21.15.2:0/2024620550 conn(0x56277e5d6700 msgr2=0x56277e5d6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f3229e00640 1 --2- 172.21.15.2:0/1298134151 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f320c0a2c30 0x7f320c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f322f6aa740 1 -- 172.21.15.2:0/2024620550 shutdown_connections 2024-09-10T08:15:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.825+0000 7f322f6aa740 1 -- 172.21.15.2:0/2024620550 wait complete. 2024-09-10T08:15:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.826+0000 7f322f6aa740 1 -- start start 2024-09-10T08:15:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.826+0000 7f3229e00640 1 --2- 172.21.15.2:0/1298134151 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f320c0a2c30 0x7f320c0a50f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x56277e5d7580 tx=0x7f3218005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.826+0000 7f322f6aa740 1 Processor -- start 2024-09-10T08:15:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f322f6aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5e5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f322f6aa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5db9d0 0x56277e5e5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f322f6aa740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5e0a60 0x56277e5ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f322f6aa740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56277e5e3850 con 0x56277e5e0a60 2024-09-10T08:15:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f322f6aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56277e4c9090 con 0x56277e5db090 2024-09-10T08:15:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f322f6aa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56277e5e39d0 con 0x56277e5db9d0 2024-09-10T08:15:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f322f6aa740 10 client.0 _ll_register_callbacks cb 0x56277e51df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f322a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5e0a60 0x56277e5ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f3229e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5e5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f3229400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5db9d0 0x56277e5e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f3229400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5db9d0 0x56277e5e5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55230/0 (socket says 172.21.15.2:55230) 2024-09-10T08:15:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f3229400640 1 -- 172.21.15.2:0/1013190188 learned_addr learned my addr 172.21.15.2:0/1013190188 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f322a800640 1 -- 172.21.15.2:0/1013190188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5db9d0 msgr2=0x56277e5e5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f322a800640 1 --2- 172.21.15.2:0/1013190188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5db9d0 0x56277e5e5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f322a800640 1 -- 172.21.15.2:0/1013190188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 msgr2=0x56277e5e5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f322a800640 1 --2- 172.21.15.2:0/1013190188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5e5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f322a800640 1 -- 172.21.15.2:0/1013190188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56277e31f620 con 0x56277e5e0a60 2024-09-10T08:15:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.827+0000 7f3229400640 1 --2- 172.21.15.2:0/1013190188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5db9d0 0x56277e5e5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.828+0000 7f322a800640 1 --2- 172.21.15.2:0/1013190188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5e0a60 0x56277e5ec5b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3220006010 tx=0x7f3220031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79e3131a6e9ea0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.828+0000 7f3207400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.828+0000 7f3207400640 1 -- 172.21.15.2:0/1013190188 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322004a040 con 0x56277e5e0a60 2024-09-10T08:15:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.828+0000 7f322f6aa740 1 -- 172.21.15.2:0/1013190188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3220036030 con 0x56277e5e0a60 2024-09-10T08:15:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.828+0000 7f3207400640 1 -- 172.21.15.2:0/1013190188 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3220031870 con 0x56277e5e0a60 2024-09-10T08:15:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.828+0000 7f3207400640 1 -- 172.21.15.2:0/1013190188 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f322005d050 con 0x56277e5e0a60 2024-09-10T08:15:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.828+0000 7f322f6aa740 10 client.17454 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.828+0000 7f322f6aa740 1 -- 172.21.15.2:0/1013190188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56277e5e3850 con 0x56277e5e0a60 2024-09-10T08:15:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.829+0000 7f3207400640 1 -- 172.21.15.2:0/1013190188 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3220036030 con 0x56277e5e0a60 2024-09-10T08:15:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.829+0000 7f3207400640 1 -- 172.21.15.2:0/1013190188 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f322004d030 con 0x56277e5e0a60 2024-09-10T08:15:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.829+0000 7f322f6aa740 10 client.17454 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.829+0000 7f322f6aa740 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.829+0000 7f322f6aa740 20 client.17454 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.829+0000 7f322f6aa740 1 --2- 172.21.15.2:0/1013190188 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56277e5f0890 0x56277e5f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.829+0000 7f322f6aa740 4 client.17454 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.829+0000 7f322f6aa740 1 -- 172.21.15.2:0/1013190188 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56277e54d830 con 0x56277e5f0890 2024-09-10T08:15:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.829+0000 7f3229e00640 1 --2- 172.21.15.2:0/1013190188 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56277e5f0890 0x56277e5f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.830+0000 7f3229e00640 1 --2- 172.21.15.2:0/1013190188 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56277e5f0890 0x56277e5f2c80 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b563493a3d35ce3 server_cookie=d2bffe48aaa2031f in_seq=0 out_seq=0 2024-09-10T08:15:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.830+0000 7f3207400640 10 client.17454 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f3207400640 1 -- 172.21.15.2:0/1013190188 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56277e54d830 con 0x56277e5f0890 2024-09-10T08:15:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f3207400640 10 client.17454 handle_command_reply: tid=0 2024-09-10T08:15:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f322f6aa740 1 client.17454 shutdown 2024-09-10T08:15:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f322f6aa740 1 -- 172.21.15.2:0/1013190188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5e0a60 msgr2=0x56277e5ec5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f322f6aa740 1 --2- 172.21.15.2:0/1013190188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5e0a60 0x56277e5ec5b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3220006010 tx=0x7f3220031680 comp rx=0 tx=0).stop 2024-09-10T08:15:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f322f6aa740 1 -- 172.21.15.2:0/1013190188 shutdown_connections 2024-09-10T08:15:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f322f6aa740 1 --2- 172.21.15.2:0/1013190188 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56277e5f0890 0x56277e5f2c80 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f322f6aa740 1 --2- 172.21.15.2:0/1013190188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5e0a60 0x56277e5ec5b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f322f6aa740 1 --2- 172.21.15.2:0/1013190188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5db9d0 0x56277e5e5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f322f6aa740 1 --2- 172.21.15.2:0/1013190188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5e5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f322f6aa740 1 -- 172.21.15.2:0/1013190188 >> 172.21.15.2:0/1013190188 conn(0x56277e5d6700 msgr2=0x56277e5d7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f3207400640 0 client.17454 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f322f6aa740 1 -- 172.21.15.2:0/1013190188 shutdown_connections 2024-09-10T08:15:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.831+0000 7f322f6aa740 1 -- 172.21.15.2:0/1013190188 wait complete. 2024-09-10T08:15:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.832+0000 7f322f6aa740 20 client.17454 trim_cache size 0 max 16384 2024-09-10T08:15:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.847+0000 7f322f6aa740 1 Processor -- start 2024-09-10T08:15:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.847+0000 7f322f6aa740 1 -- start start 2024-09-10T08:15:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.847+0000 7f322f6aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5d18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.847+0000 7f322f6aa740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 0x56277e5ec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.847+0000 7f322f6aa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e5d75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.847+0000 7f322f6aa740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f32241af130 con 0x56277e5db9d0 2024-09-10T08:15:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.847+0000 7f322f6aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f32241aca00 con 0x56277e5db090 2024-09-10T08:15:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.847+0000 7f322f6aa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f32241aa850 con 0x56277e5e0a60 2024-09-10T08:15:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.848+0000 7f3229400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 0x56277e5ec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.848+0000 7f3229400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 0x56277e5ec570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53776/0 (socket says 172.21.15.2:53776) 2024-09-10T08:15:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.848+0000 7f3229e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5d18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.848+0000 7f3229400640 1 -- 172.21.15.2:0/4160482528 learned_addr learned my addr 172.21.15.2:0/4160482528 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.848+0000 7f322a800640 1 --2- 172.21.15.2:0/4160482528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e5d75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.848+0000 7f3229400640 1 -- 172.21.15.2:0/4160482528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 msgr2=0x56277e5d75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.848+0000 7f3229400640 1 --2- 172.21.15.2:0/4160482528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e5d75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.848+0000 7f3229400640 1 -- 172.21.15.2:0/4160482528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 msgr2=0x56277e5d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.848+0000 7f3229400640 1 --2- 172.21.15.2:0/4160482528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5d18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.848+0000 7f3229400640 1 -- 172.21.15.2:0/4160482528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56277e5dfe60 con 0x56277e5db9d0 2024-09-10T08:15:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.848+0000 7f322a800640 1 --2- 172.21.15.2:0/4160482528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e5d75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.848+0000 7f3229e00640 1 --2- 172.21.15.2:0/4160482528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5d18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.848+0000 7f3229400640 1 --2- 172.21.15.2:0/4160482528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 0x56277e5ec570 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f32140ccf60 tx=0x7f3214009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638c9e42bba37514 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.849+0000 7f3212600640 1 -- 172.21.15.2:0/4160482528 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214097f10 con 0x56277e5db9d0 2024-09-10T08:15:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.849+0000 7f3212600640 1 -- 172.21.15.2:0/4160482528 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321409c370 con 0x56277e5db9d0 2024-09-10T08:15:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.849+0000 7f3212600640 1 -- 172.21.15.2:0/4160482528 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321409b5d0 con 0x56277e5db9d0 2024-09-10T08:15:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.849+0000 7f322f6aa740 1 -- 172.21.15.2:0/4160482528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 msgr2=0x56277e5ec570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.849+0000 7f322f6aa740 1 --2- 172.21.15.2:0/4160482528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 0x56277e5ec570 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f32140ccf60 tx=0x7f3214009310 comp rx=0 tx=0).stop 2024-09-10T08:15:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.849+0000 7f322f6aa740 1 -- 172.21.15.2:0/4160482528 shutdown_connections 2024-09-10T08:15:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.849+0000 7f322f6aa740 1 --2- 172.21.15.2:0/4160482528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e5d75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.849+0000 7f322f6aa740 1 --2- 172.21.15.2:0/4160482528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 0x56277e5ec570 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.849+0000 7f322f6aa740 1 --2- 172.21.15.2:0/4160482528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5d18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.849+0000 7f322f6aa740 1 -- 172.21.15.2:0/4160482528 >> 172.21.15.2:0/4160482528 conn(0x56277e5d6700 msgr2=0x56277e5e5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.850+0000 7f322f6aa740 1 -- 172.21.15.2:0/4160482528 shutdown_connections 2024-09-10T08:15:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.850+0000 7f322f6aa740 1 -- 172.21.15.2:0/4160482528 wait complete. 2024-09-10T08:15:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.850+0000 7f322f6aa740 1 -- start start 2024-09-10T08:15:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.850+0000 7f322f6aa740 1 Processor -- start 2024-09-10T08:15:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.850+0000 7f322f6aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5e93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.850+0000 7f322f6aa740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 0x56277e5e9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.850+0000 7f322f6aa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e659550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.850+0000 7f322f6aa740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f32241aaec0 con 0x56277e5db9d0 2024-09-10T08:15:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.850+0000 7f322f6aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f32241ac330 con 0x56277e5db090 2024-09-10T08:15:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.850+0000 7f322f6aa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f32241ab590 con 0x56277e5e0a60 2024-09-10T08:15:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.850+0000 7f322f6aa740 10 client.0 _ll_register_callbacks cb 0x7f32140d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f3229400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 0x56277e5e9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f322a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e659550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f3229400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 0x56277e5e9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53792/0 (socket says 172.21.15.2:53792) 2024-09-10T08:15:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f3229e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5e93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f322a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e659550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55242/0 (socket says 172.21.15.2:55242) 2024-09-10T08:15:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f3229400640 1 -- 172.21.15.2:0/3698324441 learned_addr learned my addr 172.21.15.2:0/3698324441 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f3229400640 1 -- 172.21.15.2:0/3698324441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 msgr2=0x56277e659550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f3229400640 1 --2- 172.21.15.2:0/3698324441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e659550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f3229400640 1 -- 172.21.15.2:0/3698324441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 msgr2=0x56277e5e93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f3229400640 1 --2- 172.21.15.2:0/3698324441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5e93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f3229400640 1 -- 172.21.15.2:0/3698324441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56277e659a90 con 0x56277e5db9d0 2024-09-10T08:15:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f3229e00640 1 --2- 172.21.15.2:0/3698324441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5e93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f3229400640 1 --2- 172.21.15.2:0/3698324441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 0x56277e5e9930 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f32140980d0 tx=0x7f3214005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3804f9673d10cb2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.851+0000 7f3207400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f3207400640 1 -- 172.21.15.2:0/3698324441 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32140a9060 con 0x56277e5db9d0 2024-09-10T08:15:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f3207400640 1 -- 172.21.15.2:0/3698324441 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f32140a4d30 con 0x56277e5db9d0 2024-09-10T08:15:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f322f6aa740 1 -- 172.21.15.2:0/3698324441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f321409c370 con 0x56277e5db9d0 2024-09-10T08:15:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f3207400640 1 -- 172.21.15.2:0/3698324441 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f32140ba070 con 0x56277e5db9d0 2024-09-10T08:15:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f322f6aa740 10 client.17460 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f322f6aa740 1 -- 172.21.15.2:0/3698324441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56277e65ac20 con 0x56277e5db9d0 2024-09-10T08:15:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f3207400640 1 -- 172.21.15.2:0/3698324441 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321409c370 con 0x56277e5db9d0 2024-09-10T08:15:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f3207400640 1 -- 172.21.15.2:0/3698324441 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f321409ab30 con 0x56277e5db9d0 2024-09-10T08:15:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f322f6aa740 10 client.17460 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f322f6aa740 10 client.17460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f322f6aa740 20 client.17460 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f322f6aa740 1 --2- 172.21.15.2:0/3698324441 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56277e65b4f0 0x56277e65d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f322f6aa740 4 client.17460 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:15:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.852+0000 7f322f6aa740 1 -- 172.21.15.2:0/3698324441 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56277e547b80 con 0x56277e65b4f0 2024-09-10T08:15:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.853+0000 7f3229e00640 1 --2- 172.21.15.2:0/3698324441 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56277e65b4f0 0x56277e65d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.853+0000 7f3229e00640 1 --2- 172.21.15.2:0/3698324441 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56277e65b4f0 0x56277e65d910 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c490ac6ada1619a server_cookie=dc3f8b7aaec1b4f7 in_seq=0 out_seq=0 2024-09-10T08:15:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.853+0000 7f3207400640 10 client.17460 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.867+0000 7f3207400640 1 -- 172.21.15.2:0/3698324441 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56277e547b80 con 0x56277e65b4f0 2024-09-10T08:15:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.867+0000 7f3207400640 10 client.17460 handle_command_reply: tid=0 2024-09-10T08:15:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.867+0000 7f322f6aa740 1 client.17460 shutdown 2024-09-10T08:15:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.867+0000 7f322f6aa740 1 -- 172.21.15.2:0/3698324441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 msgr2=0x56277e5e9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.867+0000 7f322f6aa740 1 --2- 172.21.15.2:0/3698324441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 0x56277e5e9930 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f32140980d0 tx=0x7f3214005c10 comp rx=0 tx=0).stop 2024-09-10T08:15:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.868+0000 7f322f6aa740 1 -- 172.21.15.2:0/3698324441 shutdown_connections 2024-09-10T08:15:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.868+0000 7f322f6aa740 1 --2- 172.21.15.2:0/3698324441 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56277e65b4f0 0x56277e65d910 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.868+0000 7f322f6aa740 1 --2- 172.21.15.2:0/3698324441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56277e5e0a60 0x56277e659550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.868+0000 7f322f6aa740 1 --2- 172.21.15.2:0/3698324441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56277e5db9d0 0x56277e5e9930 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.868+0000 7f3207400640 0 client.17460 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.868+0000 7f322f6aa740 1 --2- 172.21.15.2:0/3698324441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56277e5db090 0x56277e5e93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.868+0000 7f322f6aa740 1 -- 172.21.15.2:0/3698324441 >> 172.21.15.2:0/3698324441 conn(0x56277e5d6700 msgr2=0x56277e5e3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.868+0000 7f322f6aa740 1 -- 172.21.15.2:0/3698324441 shutdown_connections 2024-09-10T08:15:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.868+0000 7f322f6aa740 1 -- 172.21.15.2:0/3698324441 wait complete. 2024-09-10T08:15:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.868+0000 7f322f6aa740 20 client.17460 trim_cache size 0 max 16384 2024-09-10T08:15:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:15:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:15:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:15:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:15:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.868+0000 7f322c800640 1 -- 172.21.15.2:0/1298134151 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f320c0a2c30 msgr2=0x7f320c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.868+0000 7f322c800640 1 --2- 172.21.15.2:0/1298134151 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f320c0a2c30 0x7f320c0a50f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x56277e5d7580 tx=0x7f3218005f90 comp rx=0 tx=0).stop 2024-09-10T08:15:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.869+0000 7f322c800640 1 -- 172.21.15.2:0/1298134151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3224104f50 msgr2=0x7f322419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.869+0000 7f322c800640 1 --2- 172.21.15.2:0/1298134151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3224104f50 0x7f322419a4a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f321400e9f0 tx=0x7f321400edf0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.869+0000 7f322c800640 1 -- 172.21.15.2:0/1298134151 shutdown_connections 2024-09-10T08:15:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.869+0000 7f322c800640 1 --2- 172.21.15.2:0/1298134151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3224105910 0x7f32241a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.869+0000 7f322c800640 1 --2- 172.21.15.2:0/1298134151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3224104f50 0x7f322419a4a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.869+0000 7f322c800640 1 --2- 172.21.15.2:0/1298134151 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f320c0a2c30 0x7f320c0a50f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.869+0000 7f322c800640 1 --2- 172.21.15.2:0/1298134151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224103d60 0x7f3224199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.869+0000 7f322c800640 1 -- 172.21.15.2:0/1298134151 >> 172.21.15.2:0/1298134151 conn(0x7f32240ff4e0 msgr2=0x7f3224108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.869+0000 7f322c800640 1 -- 172.21.15.2:0/1298134151 shutdown_connections 2024-09-10T08:15:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:44.869+0000 7f322c800640 1 -- 172.21.15.2:0/1298134151 wait complete. 2024-09-10T08:15:45.337 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:15:45.337 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:15:45.719 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:45.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:45 smithi002 ceph-mon[30563]: pgmap v615: 129 pgs: 129 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:15:45.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:45 smithi155 ceph-mon[35405]: pgmap v615: 129 pgs: 129 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:15:45.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:45 smithi107 ceph-mon[35048]: pgmap v615: 129 pgs: 129 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:15:46.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.450+0000 7fa0c5600640 1 -- 172.21.15.2:0/627755390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c010e430 msgr2=0x7fa0c0110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:46.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.450+0000 7fa0c5600640 1 --2- 172.21.15.2:0/627755390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c010e430 0x7fa0c0110820 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009ca0 tx=0x7fa0b802a950 comp rx=0 tx=0).stop 2024-09-10T08:15:46.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.450+0000 7fa0c5600640 1 -- 172.21.15.2:0/627755390 shutdown_connections 2024-09-10T08:15:46.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.450+0000 7fa0c5600640 1 --2- 172.21.15.2:0/627755390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c010e430 0x7fa0c0110820 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:46.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.450+0000 7fa0c5600640 1 --2- 172.21.15.2:0/627755390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c0074ee0 0x7fa0c010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:46.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.450+0000 7fa0c5600640 1 --2- 172.21.15.2:0/627755390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0c00745c0 0x7fa0c00749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:46.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.450+0000 7fa0c5600640 1 -- 172.21.15.2:0/627755390 >> 172.21.15.2:0/627755390 conn(0x7fa0c00fd8b0 msgr2=0x7fa0c00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:46.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.450+0000 7fa0c5600640 1 -- 172.21.15.2:0/627755390 shutdown_connections 2024-09-10T08:15:46.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.450+0000 7fa0c5600640 1 -- 172.21.15.2:0/627755390 wait complete. 2024-09-10T08:15:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.451+0000 7fa0c5600640 1 Processor -- start 2024-09-10T08:15:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.451+0000 7fa0c5600640 1 -- start start 2024-09-10T08:15:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.451+0000 7fa0c5600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0c00745c0 0x7fa0c019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0c5600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c0074ee0 0x7fa0c019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0c5600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c010e430 0x7fa0c019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0c5600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c01141d0 con 0x7fa0c0074ee0 2024-09-10T08:15:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0c5600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c0114050 con 0x7fa0c00745c0 2024-09-10T08:15:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0c5600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c0114350 con 0x7fa0c010e430 2024-09-10T08:15:46.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0bea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0c00745c0 0x7fa0c019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:46.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0b7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c0074ee0 0x7fa0c019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:46.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0bea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0c00745c0 0x7fa0c019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:38544/0 (socket says 172.21.15.2:38544) 2024-09-10T08:15:46.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0bea00640 1 -- 172.21.15.2:0/1664468893 learned_addr learned my addr 172.21.15.2:0/1664468893 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:46.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0bf400640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c010e430 0x7fa0c019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:46.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0bea00640 1 -- 172.21.15.2:0/1664468893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c010e430 msgr2=0x7fa0c019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:46.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0bea00640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c010e430 0x7fa0c019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:46.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0bea00640 1 -- 172.21.15.2:0/1664468893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c0074ee0 msgr2=0x7fa0c019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:46.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0bea00640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c0074ee0 0x7fa0c019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:46.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0bea00640 1 -- 172.21.15.2:0/1664468893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c019f5f0 con 0x7fa0c00745c0 2024-09-10T08:15:46.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.452+0000 7fa0b7e00640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c0074ee0 0x7fa0c019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:46.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.453+0000 7fa0bea00640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0c00745c0 0x7fa0c019a640 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000ea60 tx=0x7fa0b000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:46.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.453+0000 7fa0b7400640 1 -- 172.21.15.2:0/1664468893 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000cde0 con 0x7fa0c00745c0 2024-09-10T08:15:46.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.453+0000 7fa0b7400640 1 -- 172.21.15.2:0/1664468893 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b00035f0 con 0x7fa0c00745c0 2024-09-10T08:15:46.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.453+0000 7fa0c5600640 1 -- 172.21.15.2:0/1664468893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c019f8e0 con 0x7fa0c00745c0 2024-09-10T08:15:46.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.453+0000 7fa0b7400640 1 -- 172.21.15.2:0/1664468893 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0010670 con 0x7fa0c00745c0 2024-09-10T08:15:46.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.453+0000 7fa0c5600640 1 -- 172.21.15.2:0/1664468893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c01a71c0 con 0x7fa0c00745c0 2024-09-10T08:15:46.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.454+0000 7fa0c5600640 1 -- 172.21.15.2:0/1664468893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c00749a0 con 0x7fa0c00745c0 2024-09-10T08:15:46.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.456+0000 7fa0b7400640 1 -- 172.21.15.2:0/1664468893 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa0b00037a0 con 0x7fa0c00745c0 2024-09-10T08:15:46.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.457+0000 7fa0b7400640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa0a40a2c00 0x7fa0a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:46.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.457+0000 7fa0b7400640 1 -- 172.21.15.2:0/1664468893 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa0b00cf660 con 0x7fa0c00745c0 2024-09-10T08:15:46.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.458+0000 7fa0b7e00640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa0a40a2c00 0x7fa0a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:46.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.458+0000 7fa0b7e00640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa0a40a2c00 0x7fa0a40a50c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa0c010f750 tx=0x7fa0a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:46.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.459+0000 7fa0b7400640 1 -- 172.21.15.2:0/1664468893 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0b009c170 con 0x7fa0c00745c0 2024-09-10T08:15:46.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.682+0000 7fa0c5600640 1 -- 172.21.15.2:0/1664468893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0c0109610 con 0x7fa0c00745c0 2024-09-10T08:15:46.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.683+0000 7fa0b7400640 1 -- 172.21.15.2:0/1664468893 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fa0b009ba70 con 0x7fa0c00745c0 2024-09-10T08:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:15:46.684 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:15:46.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.686+0000 7fa0c5600640 1 -- 172.21.15.2:0/1664468893 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa0a40a2c00 msgr2=0x7fa0a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:46.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.686+0000 7fa0c5600640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa0a40a2c00 0x7fa0a40a50c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa0c010f750 tx=0x7fa0a8005f90 comp rx=0 tx=0).stop 2024-09-10T08:15:46.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.686+0000 7fa0c5600640 1 -- 172.21.15.2:0/1664468893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0c00745c0 msgr2=0x7fa0c019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:46.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.686+0000 7fa0c5600640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0c00745c0 0x7fa0c019a640 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000ea60 tx=0x7fa0b000ef20 comp rx=0 tx=0).stop 2024-09-10T08:15:46.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.686+0000 7fa0c5600640 1 -- 172.21.15.2:0/1664468893 shutdown_connections 2024-09-10T08:15:46.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.686+0000 7fa0c5600640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa0a40a2c00 0x7fa0a40a50c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:46.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.686+0000 7fa0c5600640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c010e430 0x7fa0c019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:46.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.686+0000 7fa0c5600640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c0074ee0 0x7fa0c019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:46.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.686+0000 7fa0c5600640 1 --2- 172.21.15.2:0/1664468893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0c00745c0 0x7fa0c019a640 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:46.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.686+0000 7fa0c5600640 1 -- 172.21.15.2:0/1664468893 >> 172.21.15.2:0/1664468893 conn(0x7fa0c00fd8b0 msgr2=0x7fa0c00fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:46.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.686+0000 7fa0c5600640 1 -- 172.21.15.2:0/1664468893 shutdown_connections 2024-09-10T08:15:46.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:46.686+0000 7fa0c5600640 1 -- 172.21.15.2:0/1664468893 wait complete. 2024-09-10T08:15:47.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:47 smithi002 ceph-mon[30563]: pgmap v616: 129 pgs: 129 active+clean; 3.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-10T08:15:47.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:47 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1664468893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:15:47.719 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:47 smithi107 ceph-mon[35048]: pgmap v616: 129 pgs: 129 active+clean; 3.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-10T08:15:47.719 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:47 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1664468893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:15:47.755 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:15:47.791 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.791 INFO:tasks.workunit.client.0.smithi002.stdout:189/600: open fd = 3 2024-09-10T08:15:47.791 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.791 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.791 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.791 INFO:tasks.workunit.client.0.smithi002.stdout:190/600: open fd = 3 2024-09-10T08:15:47.791 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.791 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.791 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.792 INFO:tasks.workunit.client.0.smithi002.stdout:191/600: open fd = 3 2024-09-10T08:15:47.792 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.792 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.792 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.792 INFO:tasks.workunit.client.0.smithi002.stdout:192/600: open fd = 3 2024-09-10T08:15:47.792 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.792 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.792 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.792 INFO:tasks.workunit.client.0.smithi002.stdout:193/600: open fd = 3 2024-09-10T08:15:47.792 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.792 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.792 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.793 INFO:tasks.workunit.client.0.smithi002.stdout:194/600: open fd = 3 2024-09-10T08:15:47.793 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.793 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.793 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.793 INFO:tasks.workunit.client.0.smithi002.stdout:195/600: open fd = 3 2024-09-10T08:15:47.793 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.793 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.793 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.793 INFO:tasks.workunit.client.0.smithi002.stdout:196/600: open fd = 3 2024-09-10T08:15:47.793 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.793 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.793 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.794 INFO:tasks.workunit.client.0.smithi002.stdout:197/600: open fd = 3 2024-09-10T08:15:47.794 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.794 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.794 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.794 INFO:tasks.workunit.client.0.smithi002.stdout:198/600: open fd = 3 2024-09-10T08:15:47.794 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.794 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.794 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.794 INFO:tasks.workunit.client.0.smithi002.stdout:199/600: open fd = 3 2024-09-10T08:15:47.794 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.794 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.794 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.794 INFO:tasks.workunit.client.0.smithi002.stdout:200/600: open fd = 3 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:201/600: open fd = 3 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:202/600: open fd = 3 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:203/600: open fd = 3 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.795 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.796 INFO:tasks.workunit.client.0.smithi002.stdout:204/600: open fd = 3 2024-09-10T08:15:47.796 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.796 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.796 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.796 INFO:tasks.workunit.client.0.smithi002.stdout:205/600: open fd = 3 2024-09-10T08:15:47.796 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.796 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.796 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.796 INFO:tasks.workunit.client.0.smithi002.stdout:206/600: open fd = 3 2024-09-10T08:15:47.796 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.796 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.796 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.796 INFO:tasks.workunit.client.0.smithi002.stdout:207/600: open fd = 3 2024-09-10T08:15:47.797 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.797 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.797 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.797 INFO:tasks.workunit.client.0.smithi002.stdout:208/600: open fd = 3 2024-09-10T08:15:47.797 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.797 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.797 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.797 INFO:tasks.workunit.client.0.smithi002.stdout:209/600: open fd = 3 2024-09-10T08:15:47.797 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.797 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.797 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.797 INFO:tasks.workunit.client.0.smithi002.stdout:210/600: open fd = 3 2024-09-10T08:15:47.797 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:211/600: open fd = 3 2024-09-10T08:15:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:212/600: open fd = 3 2024-09-10T08:15:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:213/600: open fd = 3 2024-09-10T08:15:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:214/600: open fd = 3 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:215/600: open fd = 3 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:216/600: open fd = 3 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:217/600: open fd = 3 2024-09-10T08:15:47.799 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:218/600: open fd = 3 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:219/600: open fd = 3 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:220/600: open fd = 3 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.800 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.801 INFO:tasks.workunit.client.0.smithi002.stdout:221/600: open fd = 3 2024-09-10T08:15:47.801 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.801 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.801 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.801 INFO:tasks.workunit.client.0.smithi002.stdout:222/600: open fd = 3 2024-09-10T08:15:47.801 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.801 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.801 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.801 INFO:tasks.workunit.client.0.smithi002.stdout:223/600: open fd = 3 2024-09-10T08:15:47.801 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.801 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.801 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:224/600: open fd = 3 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:225/600: open fd = 3 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:226/600: open fd = 3 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:227/600: open fd = 3 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.802 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:228/600: open fd = 3 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:229/600: open fd = 3 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:230/600: open fd = 3 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:231/600: open fd = 3 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:232/600: open fd = 3 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:233/600: open fd = 3 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:234/600: open fd = 3 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.805 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.805 INFO:tasks.workunit.client.0.smithi002.stdout:235/600: open fd = 3 2024-09-10T08:15:47.805 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.805 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.805 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.805 INFO:tasks.workunit.client.0.smithi002.stdout:236/600: open fd = 3 2024-09-10T08:15:47.805 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.805 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.805 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.805 INFO:tasks.workunit.client.0.smithi002.stdout:237/600: open fd = 3 2024-09-10T08:15:47.805 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.805 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:238/600: open fd = 3 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:239/600: open fd = 3 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:240/600: open fd = 3 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:241/600: open fd = 3 2024-09-10T08:15:47.806 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.807 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.807 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.807 INFO:tasks.workunit.client.0.smithi002.stdout:242/600: open fd = 3 2024-09-10T08:15:47.807 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.807 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.807 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.807 INFO:tasks.workunit.client.0.smithi002.stdout:243/600: open fd = 3 2024-09-10T08:15:47.807 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.807 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.807 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.808 INFO:tasks.workunit.client.0.smithi002.stdout:244/600: open fd = 3 2024-09-10T08:15:47.808 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.808 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.808 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.808 INFO:tasks.workunit.client.0.smithi002.stdout:245/600: open fd = 3 2024-09-10T08:15:47.808 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.808 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.808 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.808 INFO:tasks.workunit.client.0.smithi002.stdout:246/600: open fd = 3 2024-09-10T08:15:47.808 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.808 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.809 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.809 INFO:tasks.workunit.client.0.smithi002.stdout:247/600: open fd = 3 2024-09-10T08:15:47.809 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.809 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.809 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.809 INFO:tasks.workunit.client.0.smithi002.stdout:248/600: open fd = 3 2024-09-10T08:15:47.809 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.809 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.809 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.809 INFO:tasks.workunit.client.0.smithi002.stdout:249/600: open fd = 3 2024-09-10T08:15:47.809 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.809 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.810 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:15:47.810 INFO:tasks.workunit.client.0.smithi002.stdout:250/600: open fd = 3 2024-09-10T08:15:47.810 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.810 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:15:47.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:47 smithi155 ceph-mon[35405]: pgmap v616: 129 pgs: 129 active+clean; 3.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-10T08:15:47.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:47 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1664468893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:15:48.138 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:48.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.724+0000 7fba09000640 1 -- 172.21.15.2:0/1336805131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041027a0 msgr2=0x7fba041067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:48.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.724+0000 7fba09000640 1 --2- 172.21.15.2:0/1336805131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041027a0 0x7fba041067f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009900 tx=0x7fb9ec02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.725+0000 7fba09000640 1 -- 172.21.15.2:0/1336805131 shutdown_connections 2024-09-10T08:15:48.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.725+0000 7fba09000640 1 --2- 172.21.15.2:0/1336805131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba04106f20 0x7fba0410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.725+0000 7fba09000640 1 --2- 172.21.15.2:0/1336805131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041027a0 0x7fba041067f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.725+0000 7fba09000640 1 --2- 172.21.15.2:0/1336805131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba04101df0 0x7fba041021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.725+0000 7fba09000640 1 -- 172.21.15.2:0/1336805131 >> 172.21.15.2:0/1336805131 conn(0x7fba040fd8b0 msgr2=0x7fba040ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:48.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.725+0000 7fba09000640 1 -- 172.21.15.2:0/1336805131 shutdown_connections 2024-09-10T08:15:48.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.725+0000 7fba09000640 1 -- 172.21.15.2:0/1336805131 wait complete. 2024-09-10T08:15:48.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.726+0000 7fba09000640 1 Processor -- start 2024-09-10T08:15:48.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.726+0000 7fba09000640 1 -- start start 2024-09-10T08:15:48.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.726+0000 7fba09000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba04101df0 0x7fba0419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.726+0000 7fba09000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041027a0 0x7fba0419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.726+0000 7fba09000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba04106f20 0x7fba0419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.726+0000 7fba09000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fba04111f80 con 0x7fba041027a0 2024-09-10T08:15:48.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.726+0000 7fba09000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fba04111e00 con 0x7fba04101df0 2024-09-10T08:15:48.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.726+0000 7fba09000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fba04112100 con 0x7fba04106f20 2024-09-10T08:15:48.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.726+0000 7fb9f9400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041027a0 0x7fba0419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.726+0000 7fba01e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba04101df0 0x7fba0419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.726+0000 7fb9f9400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041027a0 0x7fba0419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36596/0 (socket says 172.21.15.2:36596) 2024-09-10T08:15:48.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.726+0000 7fb9f9400640 1 -- 172.21.15.2:0/3075178007 learned_addr learned my addr 172.21.15.2:0/3075178007 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:48.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.727+0000 7fba02800640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba04106f20 0x7fba0419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.727+0000 7fba01e00640 1 -- 172.21.15.2:0/3075178007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba04106f20 msgr2=0x7fba0419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:48.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.727+0000 7fba01e00640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba04106f20 0x7fba0419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.727+0000 7fba01e00640 1 -- 172.21.15.2:0/3075178007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041027a0 msgr2=0x7fba0419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:48.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.727+0000 7fba01e00640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041027a0 0x7fba0419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.727+0000 7fba01e00640 1 -- 172.21.15.2:0/3075178007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0419f5c0 con 0x7fba04101df0 2024-09-10T08:15:48.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.727+0000 7fb9f9400640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041027a0 0x7fba0419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:48.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.727+0000 7fba01e00640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba04101df0 0x7fba0419a610 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400e9f0 tx=0x7fb9f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:48.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.727+0000 7fb9fb400640 1 -- 172.21.15.2:0/3075178007 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f400ccc0 con 0x7fba04101df0 2024-09-10T08:15:48.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.728+0000 7fba09000640 1 -- 172.21.15.2:0/3075178007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0419f8b0 con 0x7fba04101df0 2024-09-10T08:15:48.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.728+0000 7fb9fb400640 1 -- 172.21.15.2:0/3075178007 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f4005020 con 0x7fba04101df0 2024-09-10T08:15:48.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.728+0000 7fb9fb400640 1 -- 172.21.15.2:0/3075178007 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f40049e0 con 0x7fba04101df0 2024-09-10T08:15:48.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.728+0000 7fba09000640 1 -- 172.21.15.2:0/3075178007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba041a7190 con 0x7fba04101df0 2024-09-10T08:15:48.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.731+0000 7fb9fb400640 1 -- 172.21.15.2:0/3075178007 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb9f4005560 con 0x7fba04101df0 2024-09-10T08:15:48.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.731+0000 7fba0b9c5740 1 Processor -- start 2024-09-10T08:15:48.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.731+0000 7fba0b9c5740 1 -- start start 2024-09-10T08:15:48.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.731+0000 7fba0b9c5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e64cc70 0x56251e64d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.731+0000 7fba0b9c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 0x56251e657910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.731+0000 7fba0b9c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e657e50 0x56251e65a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.731+0000 7fba0b9c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56251e5b33e0 con 0x56251e657e50 2024-09-10T08:15:48.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.731+0000 7fba0b9c5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56251e55ad80 con 0x56251e64cc70 2024-09-10T08:15:48.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.731+0000 7fba0b9c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56251e5af3a0 con 0x56251e64d590 2024-09-10T08:15:48.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.732+0000 7fba02800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e64cc70 0x56251e64d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.732+0000 7fba01e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 0x56251e657910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.732+0000 7fb9f9400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e657e50 0x56251e65a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.732+0000 7fba02800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e64cc70 0x56251e64d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54438/0 (socket says 172.21.15.2:54438) 2024-09-10T08:15:48.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.732+0000 7fba02800640 1 -- 172.21.15.2:0/3251749929 learned_addr learned my addr 172.21.15.2:0/3251749929 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:48.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.732+0000 7fba02800640 1 -- 172.21.15.2:0/3251749929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 msgr2=0x56251e657910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:48.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.732+0000 7fba02800640 1 --2- 172.21.15.2:0/3251749929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 0x56251e657910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.732+0000 7fba02800640 1 -- 172.21.15.2:0/3251749929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e657e50 msgr2=0x56251e65a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.732+0000 7fba02800640 1 --2- 172.21.15.2:0/3251749929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e657e50 0x56251e65a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.732+0000 7fba02800640 1 -- 172.21.15.2:0/3251749929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56251e579ae0 con 0x56251e64cc70 2024-09-10T08:15:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.732+0000 7fb9fb400640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9e80a2c00 0x7fb9e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.733+0000 7fba02800640 1 --2- 172.21.15.2:0/3251749929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e64cc70 0x56251e64d050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009f50 tx=0x7fb9fc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba87a93456cf9333 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.732+0000 7fb9fb400640 1 -- 172.21.15.2:0/3075178007 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb9f40cf360 con 0x7fba04101df0 2024-09-10T08:15:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.733+0000 7fba01e00640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9e80a2c00 0x7fb9e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.733+0000 7fb9f3400640 1 -- 172.21.15.2:0/3251749929 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc004d20 con 0x56251e64cc70 2024-09-10T08:15:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.733+0000 7fb9f3400640 1 -- 172.21.15.2:0/3251749929 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9fc004050 con 0x56251e64cc70 2024-09-10T08:15:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.733+0000 7fb9f3400640 1 -- 172.21.15.2:0/3251749929 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc005280 con 0x56251e64cc70 2024-09-10T08:15:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.733+0000 7fba0b9c5740 1 -- 172.21.15.2:0/3251749929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e64cc70 msgr2=0x56251e64d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.733+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/3251749929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e64cc70 0x56251e64d050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009f50 tx=0x7fb9fc0118d0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.733+0000 7fba01e00640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9e80a2c00 0x7fb9e80a50c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4098140 tx=0x7fb9f40d2da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.734+0000 7fba0b9c5740 1 -- 172.21.15.2:0/3251749929 shutdown_connections 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.734+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/3251749929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e657e50 0x56251e65a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.734+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/3251749929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 0x56251e657910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.734+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/3251749929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e64cc70 0x56251e64d050 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.734+0000 7fba0b9c5740 1 -- 172.21.15.2:0/3251749929 >> 172.21.15.2:0/3251749929 conn(0x56251e649200 msgr2=0x56251e6495e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.734+0000 7fba0b9c5740 1 -- 172.21.15.2:0/3251749929 shutdown_connections 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.734+0000 7fba0b9c5740 1 -- 172.21.15.2:0/3251749929 wait complete. 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.734+0000 7fba0b9c5740 1 -- start start 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.734+0000 7fba0b9c5740 1 Processor -- start 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba0b9c5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e64cc70 0x56251e654d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba0b9c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 0x56251e655270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba0b9c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e657e50 0x56251e6557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba0b9c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56251e65aa20 con 0x56251e657e50 2024-09-10T08:15:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba0b9c5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56251e56e0f0 con 0x56251e64cc70 2024-09-10T08:15:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba0b9c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56251e65aba0 con 0x56251e64d590 2024-09-10T08:15:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba0b9c5740 10 client.0 _ll_register_callbacks cb 0x56251e555bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba02800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e657e50 0x56251e6557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba02800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e657e50 0x56251e6557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36608/0 (socket says 172.21.15.2:36608) 2024-09-10T08:15:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba02800640 1 -- 172.21.15.2:0/2767055823 learned_addr learned my addr 172.21.15.2:0/2767055823 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba02800640 1 --2- 172.21.15.2:0/2767055823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e64cc70 0x56251e654d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fb9f9400640 1 --2- 172.21.15.2:0/2767055823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 0x56251e655270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba02800640 1 -- 172.21.15.2:0/2767055823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 msgr2=0x56251e655270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba02800640 1 --2- 172.21.15.2:0/2767055823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 0x56251e655270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba02800640 1 -- 172.21.15.2:0/2767055823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e64cc70 msgr2=0x56251e654d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba02800640 1 --2- 172.21.15.2:0/2767055823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e64cc70 0x56251e654d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.735+0000 7fba02800640 1 -- 172.21.15.2:0/2767055823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56251e3243c0 con 0x56251e657e50 2024-09-10T08:15:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.736+0000 7fba02800640 1 --2- 172.21.15.2:0/2767055823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e657e50 0x56251e6557b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc004690 tx=0x7fb9fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe9e200884350af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.736+0000 7fb9e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.736+0000 7fb9e7e00640 1 -- 172.21.15.2:0/2767055823 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc02b020 con 0x56251e657e50 2024-09-10T08:15:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.736+0000 7fb9e7e00640 1 -- 172.21.15.2:0/2767055823 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb9fc027440 con 0x56251e657e50 2024-09-10T08:15:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.736+0000 7fba0b9c5740 1 -- 172.21.15.2:0/2767055823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9fc004050 con 0x56251e657e50 2024-09-10T08:15:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.736+0000 7fb9e7e00640 1 -- 172.21.15.2:0/2767055823 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb9fc036050 con 0x56251e657e50 2024-09-10T08:15:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.736+0000 7fba0b9c5740 10 client.17478 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.736+0000 7fba0b9c5740 1 -- 172.21.15.2:0/2767055823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56251e666a60 con 0x56251e657e50 2024-09-10T08:15:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.737+0000 7fb9e7e00640 1 -- 172.21.15.2:0/2767055823 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9fc004050 con 0x56251e657e50 2024-09-10T08:15:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.737+0000 7fb9e7e00640 1 -- 172.21.15.2:0/2767055823 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb9fc00f6d0 con 0x56251e657e50 2024-09-10T08:15:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.737+0000 7fba0b9c5740 10 client.17478 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.737+0000 7fba0b9c5740 10 client.17478 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:15:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.737+0000 7fba0b9c5740 20 client.17478 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.737+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/2767055823 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56251e667ec0 0x56251e66a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.737+0000 7fba0b9c5740 4 client.17478 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.737+0000 7fba0b9c5740 1 -- 172.21.15.2:0/2767055823 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56251e5c0950 con 0x56251e667ec0 2024-09-10T08:15:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.737+0000 7fb9f9400640 1 --2- 172.21.15.2:0/2767055823 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56251e667ec0 0x56251e66a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.738+0000 7fb9f9400640 1 --2- 172.21.15.2:0/2767055823 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56251e667ec0 0x56251e66a2b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4257cc579af32271 server_cookie=659e941d05a357dd in_seq=0 out_seq=0 2024-09-10T08:15:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.738+0000 7fb9e7e00640 10 client.17478 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:15:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fb9e7e00640 1 -- 172.21.15.2:0/2767055823 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56251e5c0950 con 0x56251e667ec0 2024-09-10T08:15:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fb9e7e00640 10 client.17478 handle_command_reply: tid=0 2024-09-10T08:15:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fba0b9c5740 1 client.17478 shutdown 2024-09-10T08:15:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fba0b9c5740 1 -- 172.21.15.2:0/2767055823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e657e50 msgr2=0x56251e6557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/2767055823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e657e50 0x56251e6557b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc004690 tx=0x7fb9fc0047b0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fba0b9c5740 1 -- 172.21.15.2:0/2767055823 shutdown_connections 2024-09-10T08:15:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/2767055823 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56251e667ec0 0x56251e66a2b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/2767055823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e657e50 0x56251e6557b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/2767055823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 0x56251e655270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/2767055823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e64cc70 0x56251e654d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fba0b9c5740 1 -- 172.21.15.2:0/2767055823 >> 172.21.15.2:0/2767055823 conn(0x56251e649200 msgr2=0x56251e648380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fb9e7e00640 0 client.17478 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:15:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fba0b9c5740 1 -- 172.21.15.2:0/2767055823 shutdown_connections 2024-09-10T08:15:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fba0b9c5740 1 -- 172.21.15.2:0/2767055823 wait complete. 2024-09-10T08:15:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.739+0000 7fba0b9c5740 20 client.17478 trim_cache size 0 max 16384 2024-09-10T08:15:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.753+0000 7fba0b9c5740 1 Processor -- start 2024-09-10T08:15:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.753+0000 7fba0b9c5740 1 -- start start 2024-09-10T08:15:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.753+0000 7fba0b9c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64cc70 0x56251e6544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.753+0000 7fba0b9c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e64d590 0x56251e66f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.753+0000 7fba0b9c5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e657e50 0x56251e66ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.754+0000 7fba0b9c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fba041b39b0 con 0x56251e64d590 2024-09-10T08:15:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.754+0000 7fba0b9c5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fba041b1280 con 0x56251e657e50 2024-09-10T08:15:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.754+0000 7fba0b9c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fba041af0d0 con 0x56251e64cc70 2024-09-10T08:15:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.754+0000 7fba02800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64cc70 0x56251e6544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.754+0000 7fba02800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64cc70 0x56251e6544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56640/0 (socket says 172.21.15.2:56640) 2024-09-10T08:15:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.754+0000 7fba02800640 1 -- 172.21.15.2:0/340048012 learned_addr learned my addr 172.21.15.2:0/340048012 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.754+0000 7fba02800640 1 --2- 172.21.15.2:0/340048012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e657e50 0x56251e66ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.754+0000 7fb9f9400640 1 --2- 172.21.15.2:0/340048012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e64d590 0x56251e66f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.755+0000 7fba02800640 1 -- 172.21.15.2:0/340048012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e657e50 msgr2=0x56251e66ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.755+0000 7fba02800640 1 --2- 172.21.15.2:0/340048012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e657e50 0x56251e66ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.755+0000 7fba02800640 1 -- 172.21.15.2:0/340048012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e64d590 msgr2=0x56251e66f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.755+0000 7fba02800640 1 --2- 172.21.15.2:0/340048012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e64d590 0x56251e66f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.755+0000 7fba02800640 1 -- 172.21.15.2:0/340048012 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56251e644770 con 0x56251e64cc70 2024-09-10T08:15:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.755+0000 7fba02800640 1 --2- 172.21.15.2:0/340048012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64cc70 0x56251e6544e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009560 tx=0x7fb9fc009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb3d43e37976de1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.755+0000 7fb9f2a00640 1 -- 172.21.15.2:0/340048012 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc02b050 con 0x56251e64cc70 2024-09-10T08:15:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.755+0000 7fb9f2a00640 1 -- 172.21.15.2:0/340048012 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9fc009e20 con 0x56251e64cc70 2024-09-10T08:15:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.755+0000 7fb9f2a00640 1 -- 172.21.15.2:0/340048012 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc0276a0 con 0x56251e64cc70 2024-09-10T08:15:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.756+0000 7fba0b9c5740 1 -- 172.21.15.2:0/340048012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64cc70 msgr2=0x56251e6544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.756+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/340048012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64cc70 0x56251e6544e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009560 tx=0x7fb9fc009230 comp rx=0 tx=0).stop 2024-09-10T08:15:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.756+0000 7fba0b9c5740 1 -- 172.21.15.2:0/340048012 shutdown_connections 2024-09-10T08:15:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.756+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/340048012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e657e50 0x56251e66ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.756+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/340048012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e64d590 0x56251e66f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.756+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/340048012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64cc70 0x56251e6544e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.756+0000 7fba0b9c5740 1 -- 172.21.15.2:0/340048012 >> 172.21.15.2:0/340048012 conn(0x56251e649200 msgr2=0x56251e656db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.756+0000 7fba0b9c5740 1 -- 172.21.15.2:0/340048012 shutdown_connections 2024-09-10T08:15:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.756+0000 7fba0b9c5740 1 -- 172.21.15.2:0/340048012 wait complete. 2024-09-10T08:15:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.757+0000 7fba0b9c5740 1 -- start start 2024-09-10T08:15:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.757+0000 7fba0b9c5740 1 Processor -- start 2024-09-10T08:15:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.757+0000 7fba0b9c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e64cc70 0x56251e65adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.757+0000 7fba0b9c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 0x56251e65b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.757+0000 7fba0b9c5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e657e50 0x56251e65b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.757+0000 7fba0b9c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fba041af740 con 0x56251e64cc70 2024-09-10T08:15:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.757+0000 7fba0b9c5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fba041b0bb0 con 0x56251e657e50 2024-09-10T08:15:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.757+0000 7fba0b9c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fba041afe10 con 0x56251e64d590 2024-09-10T08:15:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.757+0000 7fba0b9c5740 10 client.0 _ll_register_callbacks cb 0x7fb9f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.757+0000 7fba02800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e64cc70 0x56251e65adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.757+0000 7fba02800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e64cc70 0x56251e65adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36626/0 (socket says 172.21.15.2:36626) 2024-09-10T08:15:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.757+0000 7fba02800640 1 -- 172.21.15.2:0/194489049 learned_addr learned my addr 172.21.15.2:0/194489049 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.757+0000 7fb9f9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 0x56251e65b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.758+0000 7fba02800640 1 --2- 172.21.15.2:0/194489049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e657e50 0x56251e65b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.758+0000 7fba02800640 1 -- 172.21.15.2:0/194489049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 msgr2=0x56251e65b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.758+0000 7fba02800640 1 --2- 172.21.15.2:0/194489049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 0x56251e65b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.758+0000 7fba02800640 1 -- 172.21.15.2:0/194489049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e657e50 msgr2=0x56251e65b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.758+0000 7fba02800640 1 --2- 172.21.15.2:0/194489049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e657e50 0x56251e65b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.758+0000 7fba02800640 1 -- 172.21.15.2:0/194489049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56251e6d09b0 con 0x56251e64cc70 2024-09-10T08:15:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.758+0000 7fba02800640 1 --2- 172.21.15.2:0/194489049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e64cc70 0x56251e65adb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc004740 tx=0x7fb9fc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2a532da0cd585d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.758+0000 7fb9e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.758+0000 7fb9e7e00640 1 -- 172.21.15.2:0/194489049 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc02b040 con 0x56251e64cc70 2024-09-10T08:15:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.758+0000 7fb9e7e00640 1 -- 172.21.15.2:0/194489049 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb9fc003c70 con 0x56251e64cc70 2024-09-10T08:15:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.758+0000 7fba0b9c5740 1 -- 172.21.15.2:0/194489049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9fc009e20 con 0x56251e64cc70 2024-09-10T08:15:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.759+0000 7fb9e7e00640 1 -- 172.21.15.2:0/194489049 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb9fc003e70 con 0x56251e64cc70 2024-09-10T08:15:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.759+0000 7fba0b9c5740 10 client.17484 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.759+0000 7fba0b9c5740 1 -- 172.21.15.2:0/194489049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56251e6d1bc0 con 0x56251e64cc70 2024-09-10T08:15:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.759+0000 7fb9e7e00640 1 -- 172.21.15.2:0/194489049 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9fc009e20 con 0x56251e64cc70 2024-09-10T08:15:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.759+0000 7fb9e7e00640 1 -- 172.21.15.2:0/194489049 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb9fc046420 con 0x56251e64cc70 2024-09-10T08:15:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.759+0000 7fba0b9c5740 10 client.17484 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.759+0000 7fba0b9c5740 10 client.17484 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:15:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.759+0000 7fba0b9c5740 20 client.17484 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.759+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/194489049 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56251e6d22e0 0x56251e6d46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.759+0000 7fba0b9c5740 4 client.17484 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:15:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.759+0000 7fba0b9c5740 1 -- 172.21.15.2:0/194489049 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56251e5bae60 con 0x56251e6d22e0 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.759+0000 7fb9f9400640 1 --2- 172.21.15.2:0/194489049 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56251e6d22e0 0x56251e6d46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fb9f9400640 1 --2- 172.21.15.2:0/194489049 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56251e6d22e0 0x56251e6d46d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5018cb89d753cca6 server_cookie=ee3df9b594b5d57a in_seq=0 out_seq=0 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fb9e7e00640 10 client.17484 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fb9e7e00640 1 -- 172.21.15.2:0/194489049 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56251e5bae60 con 0x56251e6d22e0 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fb9e7e00640 10 client.17484 handle_command_reply: tid=0 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fba0b9c5740 1 client.17484 shutdown 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fba0b9c5740 1 -- 172.21.15.2:0/194489049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e64cc70 msgr2=0x56251e65adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/194489049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e64cc70 0x56251e65adb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc004740 tx=0x7fb9fc0046d0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fba0b9c5740 1 -- 172.21.15.2:0/194489049 shutdown_connections 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/194489049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56251e657e50 0x56251e65b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/194489049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56251e64d590 0x56251e65b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/194489049 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56251e6d22e0 0x56251e6d46d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fba0b9c5740 1 --2- 172.21.15.2:0/194489049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56251e64cc70 0x56251e65adb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fb9e7e00640 0 client.17484 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:15:48.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.760+0000 7fba0b9c5740 1 -- 172.21.15.2:0/194489049 >> 172.21.15.2:0/194489049 conn(0x56251e649200 msgr2=0x56251e656590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:48.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.761+0000 7fba0b9c5740 1 -- 172.21.15.2:0/194489049 shutdown_connections 2024-09-10T08:15:48.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.761+0000 7fba0b9c5740 1 -- 172.21.15.2:0/194489049 wait complete. 2024-09-10T08:15:48.895 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:15:48.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.761+0000 7fba0b9c5740 20 client.17484 trim_cache size 0 max 16384 2024-09-10T08:15:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.761+0000 7fba09000640 1 -- 172.21.15.2:0/3075178007 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9e80a2c00 msgr2=0x7fb9e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.761+0000 7fba09000640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9e80a2c00 0x7fb9e80a50c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4098140 tx=0x7fb9f40d2da0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.761+0000 7fba09000640 1 -- 172.21.15.2:0/3075178007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba04101df0 msgr2=0x7fba0419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.761+0000 7fba09000640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba04101df0 0x7fba0419a610 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400e9f0 tx=0x7fb9f400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.761+0000 7fba09000640 1 -- 172.21.15.2:0/3075178007 shutdown_connections 2024-09-10T08:15:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.761+0000 7fba09000640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb9e80a2c00 0x7fb9e80a50c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.761+0000 7fba09000640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba04106f20 0x7fba0419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.761+0000 7fba09000640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041027a0 0x7fba0419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.761+0000 7fba09000640 1 --2- 172.21.15.2:0/3075178007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba04101df0 0x7fba0419a610 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.761+0000 7fba09000640 1 -- 172.21.15.2:0/3075178007 >> 172.21.15.2:0/3075178007 conn(0x7fba040fd8b0 msgr2=0x7fba040feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.762+0000 7fba09000640 1 -- 172.21.15.2:0/3075178007 shutdown_connections 2024-09-10T08:15:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:48.762+0000 7fba09000640 1 -- 172.21.15.2:0/3075178007 wait complete. 2024-09-10T08:15:49.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:49 smithi002 ceph-mon[30563]: pgmap v617: 129 pgs: 129 active+clean; 3.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:15:49.736 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:15:49.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:49 smithi155 ceph-mon[35405]: pgmap v617: 129 pgs: 129 active+clean; 3.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:15:49.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:49 smithi107 ceph-mon[35048]: pgmap v617: 129 pgs: 129 active+clean; 3.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:15:50.160 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:50.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.937+0000 7f8317a00640 1 -- 172.21.15.2:0/484395848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83101049a0 msgr2=0x7f83101089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:50.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.937+0000 7f8317a00640 1 --2- 172.21.15.2:0/484395848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83101049a0 0x7f83101089f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009860 tx=0x7f82fc02a340 comp rx=0 tx=0).stop 2024-09-10T08:15:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.937+0000 7f8317a00640 1 -- 172.21.15.2:0/484395848 shutdown_connections 2024-09-10T08:15:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.937+0000 7f8317a00640 1 --2- 172.21.15.2:0/484395848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8310109120 0x7f8310110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.937+0000 7f8317a00640 1 --2- 172.21.15.2:0/484395848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83101049a0 0x7f83101089f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.937+0000 7f8317a00640 1 --2- 172.21.15.2:0/484395848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8310103ff0 0x7f83101043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.937+0000 7f8317a00640 1 -- 172.21.15.2:0/484395848 >> 172.21.15.2:0/484395848 conn(0x7f83100ffa90 msgr2=0x7f8310101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.937+0000 7f8317a00640 1 -- 172.21.15.2:0/484395848 shutdown_connections 2024-09-10T08:15:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.937+0000 7f8317a00640 1 -- 172.21.15.2:0/484395848 wait complete. 2024-09-10T08:15:50.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.938+0000 7f8317a00640 1 Processor -- start 2024-09-10T08:15:50.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.938+0000 7f8317a00640 1 -- start start 2024-09-10T08:15:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.939+0000 7f8317a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8310103ff0 0x7f831019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.939+0000 7f8317a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83101049a0 0x7f831019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.939+0000 7f8317a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8310109120 0x7f831019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.939+0000 7f8317a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8310114140 con 0x7f8310103ff0 2024-09-10T08:15:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.939+0000 7f8317a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8310113fc0 con 0x7f8310109120 2024-09-10T08:15:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.939+0000 7f8317a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f83101142c0 con 0x7f83101049a0 2024-09-10T08:15:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.939+0000 7f8315000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8310103ff0 0x7f831019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.939+0000 7f8315000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8310103ff0 0x7f831019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36654/0 (socket says 172.21.15.2:36654) 2024-09-10T08:15:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.939+0000 7f8315000640 1 -- 172.21.15.2:0/2338714595 learned_addr learned my addr 172.21.15.2:0/2338714595 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.939+0000 7f8315a00640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8310109120 0x7f831019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.939+0000 7f830fe00640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83101049a0 0x7f831019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.940+0000 7f8315000640 1 -- 172.21.15.2:0/2338714595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83101049a0 msgr2=0x7f831019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:50.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.940+0000 7f8315000640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83101049a0 0x7f831019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.940+0000 7f8315000640 1 -- 172.21.15.2:0/2338714595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8310109120 msgr2=0x7f831019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:50.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.940+0000 7f8315000640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8310109120 0x7f831019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.940+0000 7f8315000640 1 -- 172.21.15.2:0/2338714595 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831019f650 con 0x7f8310103ff0 2024-09-10T08:15:50.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.940+0000 7f8315a00640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8310109120 0x7f831019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:50.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.940+0000 7f8315000640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8310103ff0 0x7f831019a6a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f830400e9f0 tx=0x7f830400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.940+0000 7f830d600640 1 -- 172.21.15.2:0/2338714595 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830400cd80 con 0x7f8310103ff0 2024-09-10T08:15:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.940+0000 7f8317a00640 1 -- 172.21.15.2:0/2338714595 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831019f940 con 0x7f8310103ff0 2024-09-10T08:15:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.940+0000 7f830d600640 1 -- 172.21.15.2:0/2338714595 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8304010910 con 0x7f8310103ff0 2024-09-10T08:15:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.940+0000 7f830d600640 1 -- 172.21.15.2:0/2338714595 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304018990 con 0x7f8310103ff0 2024-09-10T08:15:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.940+0000 7f8317a00640 1 -- 172.21.15.2:0/2338714595 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83101a7220 con 0x7f8310103ff0 2024-09-10T08:15:50.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.943+0000 7f830d600640 1 -- 172.21.15.2:0/2338714595 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8304010ac0 con 0x7f8310103ff0 2024-09-10T08:15:50.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.944+0000 7f831a648740 1 Processor -- start 2024-09-10T08:15:50.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.944+0000 7f831a648740 1 -- start start 2024-09-10T08:15:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.944+0000 7f831a648740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b0ffc70 0x557d4b100050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.944+0000 7f831a648740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b100590 0x557d4b10a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.944+0000 7f831a648740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b10ae50 0x557d4b10d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.944+0000 7f831a648740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557d4b0663e0 con 0x557d4b0ffc70 2024-09-10T08:15:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.944+0000 7f831a648740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557d4b00dd80 con 0x557d4b10ae50 2024-09-10T08:15:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.944+0000 7f831a648740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557d4b0623a0 con 0x557d4b100590 2024-09-10T08:15:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.945+0000 7f8315a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b0ffc70 0x557d4b100050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.945+0000 7f8315000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b100590 0x557d4b10a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.945+0000 7f830fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b10ae50 0x557d4b10d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.945+0000 7f830fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b10ae50 0x557d4b10d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54486/0 (socket says 172.21.15.2:54486) 2024-09-10T08:15:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.945+0000 7f8315a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b0ffc70 0x557d4b100050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36662/0 (socket says 172.21.15.2:36662) 2024-09-10T08:15:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.945+0000 7f830fe00640 1 -- 172.21.15.2:0/649193842 learned_addr learned my addr 172.21.15.2:0/649193842 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.945+0000 7f8315000640 1 -- 172.21.15.2:0/649193842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b10ae50 msgr2=0x557d4b10d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.945+0000 7f8315000640 1 --2- 172.21.15.2:0/649193842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b10ae50 0x557d4b10d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.945+0000 7f8315000640 1 -- 172.21.15.2:0/649193842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b0ffc70 msgr2=0x557d4b100050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.945+0000 7f8315000640 1 --2- 172.21.15.2:0/649193842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b0ffc70 0x557d4b100050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.945+0000 7f8315000640 1 -- 172.21.15.2:0/649193842 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4b02cae0 con 0x557d4b100590 2024-09-10T08:15:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.946+0000 7f8315000640 1 --2- 172.21.15.2:0/649193842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b100590 0x557d4b10a910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f83040ccdf0 tx=0x7f83040d7670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28ad35f6e4afab6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.946+0000 7f830d600640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82f80a2b80 0x7f82f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.946+0000 7f8302000640 1 -- 172.21.15.2:0/649193842 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83040de070 con 0x557d4b100590 2024-09-10T08:15:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.946+0000 7f830d600640 1 -- 172.21.15.2:0/2338714595 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f83040cf7a0 con 0x7f8310103ff0 2024-09-10T08:15:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.946+0000 7f8302000640 1 -- 172.21.15.2:0/649193842 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830401ce60 con 0x557d4b100590 2024-09-10T08:15:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.946+0000 7f8302000640 1 -- 172.21.15.2:0/649193842 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830409ba30 con 0x557d4b100590 2024-09-10T08:15:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.946+0000 7f8315a00640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82f80a2b80 0x7f82f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.947+0000 7f831a648740 1 -- 172.21.15.2:0/649193842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b100590 msgr2=0x557d4b10a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.947+0000 7f831a648740 1 --2- 172.21.15.2:0/649193842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b100590 0x557d4b10a910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f83040ccdf0 tx=0x7f83040d7670 comp rx=0 tx=0).stop 2024-09-10T08:15:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.947+0000 7f8315a00640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82f80a2b80 0x7f82f80a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8308005fd0 tx=0x7f8308010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:50.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.947+0000 7f831a648740 1 -- 172.21.15.2:0/649193842 shutdown_connections 2024-09-10T08:15:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.947+0000 7f831a648740 1 --2- 172.21.15.2:0/649193842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b10ae50 0x557d4b10d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.947+0000 7f831a648740 1 --2- 172.21.15.2:0/649193842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b100590 0x557d4b10a910 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.947+0000 7f831a648740 1 --2- 172.21.15.2:0/649193842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b0ffc70 0x557d4b100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.947+0000 7f831a648740 1 -- 172.21.15.2:0/649193842 >> 172.21.15.2:0/649193842 conn(0x557d4b0fc200 msgr2=0x557d4b0fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.947+0000 7f831a648740 1 -- 172.21.15.2:0/649193842 shutdown_connections 2024-09-10T08:15:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.947+0000 7f831a648740 1 -- 172.21.15.2:0/649193842 wait complete. 2024-09-10T08:15:50.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.948+0000 7f831a648740 1 -- start start 2024-09-10T08:15:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.948+0000 7f831a648740 1 Processor -- start 2024-09-10T08:15:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.948+0000 7f831a648740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b0ffc70 0x557d4b107d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.948+0000 7f831a648740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b100590 0x557d4b108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f831a648740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b10ae50 0x557d4b1087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f831a648740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557d4b10da20 con 0x557d4b10ae50 2024-09-10T08:15:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f831a648740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557d4b0210f0 con 0x557d4b100590 2024-09-10T08:15:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f831a648740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557d4b10dba0 con 0x557d4b0ffc70 2024-09-10T08:15:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f831a648740 10 client.0 _ll_register_callbacks cb 0x557d4b008bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f8315000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b0ffc70 0x557d4b107d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f8315000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b0ffc70 0x557d4b107d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56690/0 (socket says 172.21.15.2:56690) 2024-09-10T08:15:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f8315000640 1 -- 172.21.15.2:0/1374885286 learned_addr learned my addr 172.21.15.2:0/1374885286 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f8315a00640 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b10ae50 0x557d4b1087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f830fe00640 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b100590 0x557d4b108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f8315a00640 1 -- 172.21.15.2:0/1374885286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b0ffc70 msgr2=0x557d4b107d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:50.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f8315a00640 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b0ffc70 0x557d4b107d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f8315a00640 1 -- 172.21.15.2:0/1374885286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b100590 msgr2=0x557d4b108270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:50.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f8315a00640 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b100590 0x557d4b108270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f8315a00640 1 -- 172.21.15.2:0/1374885286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4add73c0 con 0x557d4b10ae50 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f8315000640 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b0ffc70 0x557d4b107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.949+0000 7f830fe00640 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b100590 0x557d4b108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.950+0000 7f8315a00640 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b10ae50 0x557d4b1087b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8308003bf0 tx=0x7f8308004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a203e68ed421164 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.950+0000 7f82f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.950+0000 7f82f6a00640 1 -- 172.21.15.2:0/1374885286 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308020040 con 0x557d4b10ae50 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.950+0000 7f831a648740 1 -- 172.21.15.2:0/1374885286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557d4b073950 con 0x557d4b10ae50 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.950+0000 7f82f6a00640 1 -- 172.21.15.2:0/1374885286 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8308004800 con 0x557d4b10ae50 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.950+0000 7f82f6a00640 1 -- 172.21.15.2:0/1374885286 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8308031050 con 0x557d4b10ae50 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.950+0000 7f831a648740 10 client.17496 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.950+0000 7f831a648740 1 -- 172.21.15.2:0/1374885286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557d4b119b10 con 0x557d4b10ae50 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.951+0000 7f82f6a00640 1 -- 172.21.15.2:0/1374885286 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557d4b073950 con 0x557d4b10ae50 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.951+0000 7f82f6a00640 1 -- 172.21.15.2:0/1374885286 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f830800b040 con 0x557d4b10ae50 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.951+0000 7f831a648740 10 client.17496 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:50.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.951+0000 7f831a648740 10 client.17496 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.951+0000 7f831a648740 20 client.17496 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.951+0000 7f831a648740 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557d4b11b020 0x557d4b11d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.951+0000 7f831a648740 4 client.17496 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.951+0000 7f831a648740 1 -- 172.21.15.2:0/1374885286 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d4b06de60 con 0x557d4b11b020 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.952+0000 7f8315000640 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557d4b11b020 0x557d4b11d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.952+0000 7f8315000640 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557d4b11b020 0x557d4b11d430 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33ff6b31bbb04b17 server_cookie=48afb8edee6e8ff4 in_seq=0 out_seq=0 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.952+0000 7f82f6a00640 10 client.17496 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f82f6a00640 1 -- 172.21.15.2:0/1374885286 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d4b06de60 con 0x557d4b11b020 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f82f6a00640 10 client.17496 handle_command_reply: tid=0 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f831a648740 1 client.17496 shutdown 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f831a648740 1 -- 172.21.15.2:0/1374885286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b10ae50 msgr2=0x557d4b1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f831a648740 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b10ae50 0x557d4b1087b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8308003bf0 tx=0x7f8308004ab0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f831a648740 1 -- 172.21.15.2:0/1374885286 shutdown_connections 2024-09-10T08:15:50.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f831a648740 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557d4b11b020 0x557d4b11d430 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f831a648740 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b10ae50 0x557d4b1087b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f831a648740 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b100590 0x557d4b108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f831a648740 1 --2- 172.21.15.2:0/1374885286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b0ffc70 0x557d4b107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f831a648740 1 -- 172.21.15.2:0/1374885286 >> 172.21.15.2:0/1374885286 conn(0x557d4b0fc200 msgr2=0x557d4b0fb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f82f6a00640 0 client.17496 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:15:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f831a648740 1 -- 172.21.15.2:0/1374885286 shutdown_connections 2024-09-10T08:15:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f831a648740 1 -- 172.21.15.2:0/1374885286 wait complete. 2024-09-10T08:15:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.953+0000 7f831a648740 20 client.17496 trim_cache size 0 max 16384 2024-09-10T08:15:50.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.967+0000 7f831a648740 1 Processor -- start 2024-09-10T08:15:50.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.967+0000 7f831a648740 1 -- start start 2024-09-10T08:15:50.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.967+0000 7f831a648740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b0ffc70 0x557d4b1026d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.967+0000 7f831a648740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b100590 0x557d4b122d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.967+0000 7f831a648740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b10ae50 0x557d4b123240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.967+0000 7f831a648740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83101b3940 con 0x557d4b100590 2024-09-10T08:15:50.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.967+0000 7f831a648740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f83101b1210 con 0x557d4b0ffc70 2024-09-10T08:15:50.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.967+0000 7f831a648740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f83101af060 con 0x557d4b10ae50 2024-09-10T08:15:50.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.968+0000 7f830fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b100590 0x557d4b122d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.968+0000 7f8315000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b0ffc70 0x557d4b1026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.968+0000 7f8315000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b0ffc70 0x557d4b1026d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54506/0 (socket says 172.21.15.2:54506) 2024-09-10T08:15:50.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.968+0000 7f8315000640 1 -- 172.21.15.2:0/825185097 learned_addr learned my addr 172.21.15.2:0/825185097 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:50.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.968+0000 7f8315a00640 1 --2- 172.21.15.2:0/825185097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b10ae50 0x557d4b123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.968+0000 7f830fe00640 1 -- 172.21.15.2:0/825185097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b10ae50 msgr2=0x557d4b123240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:50.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.968+0000 7f830fe00640 1 --2- 172.21.15.2:0/825185097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b10ae50 0x557d4b123240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.968+0000 7f830fe00640 1 -- 172.21.15.2:0/825185097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b0ffc70 msgr2=0x557d4b1026d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:50.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.968+0000 7f830fe00640 1 --2- 172.21.15.2:0/825185097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b0ffc70 0x557d4b1026d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.968+0000 7f830fe00640 1 -- 172.21.15.2:0/825185097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4b0ffaa0 con 0x557d4b100590 2024-09-10T08:15:50.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.968+0000 7f8315a00640 1 --2- 172.21.15.2:0/825185097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b10ae50 0x557d4b123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:50.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.968+0000 7f8315000640 1 --2- 172.21.15.2:0/825185097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b0ffc70 0x557d4b1026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:50.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.969+0000 7f830fe00640 1 --2- 172.21.15.2:0/825185097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b100590 0x557d4b122d00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x557d4b0fd580 tx=0x7f82fc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f754cbe3e2dc6bbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:50.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.969+0000 7f8301600640 1 -- 172.21.15.2:0/825185097 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc042070 con 0x557d4b100590 2024-09-10T08:15:50.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.969+0000 7f8301600640 1 -- 172.21.15.2:0/825185097 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82fc004230 con 0x557d4b100590 2024-09-10T08:15:50.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.969+0000 7f8301600640 1 -- 172.21.15.2:0/825185097 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc037040 con 0x557d4b100590 2024-09-10T08:15:50.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.969+0000 7f831a648740 1 -- 172.21.15.2:0/825185097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b100590 msgr2=0x557d4b122d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:50.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.969+0000 7f831a648740 1 --2- 172.21.15.2:0/825185097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b100590 0x557d4b122d00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x557d4b0fd580 tx=0x7f82fc0056d0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.970+0000 7f831a648740 1 -- 172.21.15.2:0/825185097 shutdown_connections 2024-09-10T08:15:50.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.970+0000 7f831a648740 1 --2- 172.21.15.2:0/825185097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b10ae50 0x557d4b123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.970+0000 7f831a648740 1 --2- 172.21.15.2:0/825185097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b100590 0x557d4b122d00 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.970+0000 7f831a648740 1 --2- 172.21.15.2:0/825185097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b0ffc70 0x557d4b1026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.970+0000 7f831a648740 1 -- 172.21.15.2:0/825185097 >> 172.21.15.2:0/825185097 conn(0x557d4b0fc200 msgr2=0x557d4b0fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:50.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.970+0000 7f831a648740 1 -- 172.21.15.2:0/825185097 shutdown_connections 2024-09-10T08:15:50.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.970+0000 7f831a648740 1 -- 172.21.15.2:0/825185097 wait complete. 2024-09-10T08:15:50.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.970+0000 7f831a648740 1 -- start start 2024-09-10T08:15:50.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.970+0000 7f831a648740 1 Processor -- start 2024-09-10T08:15:50.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f831a648740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b0ffc70 0x557d4b10e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f831a648740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b100590 0x557d4b10e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f831a648740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b10ae50 0x557d4b183cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f831a648740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83101af6d0 con 0x557d4b10ae50 2024-09-10T08:15:50.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f831a648740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f83101b0b40 con 0x557d4b100590 2024-09-10T08:15:50.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f831a648740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f83101afda0 con 0x557d4b0ffc70 2024-09-10T08:15:50.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f831a648740 10 client.0 _ll_register_callbacks cb 0x7f830801d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:50.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f8315a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b10ae50 0x557d4b183cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f8315a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b10ae50 0x557d4b183cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36668/0 (socket says 172.21.15.2:36668) 2024-09-10T08:15:50.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f8315a00640 1 -- 172.21.15.2:0/2039284086 learned_addr learned my addr 172.21.15.2:0/2039284086 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:50.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f8315000640 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b0ffc70 0x557d4b10e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f830fe00640 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b100590 0x557d4b10e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f8315a00640 1 -- 172.21.15.2:0/2039284086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b0ffc70 msgr2=0x557d4b10e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:50.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f8315a00640 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b0ffc70 0x557d4b10e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.971+0000 7f8315a00640 1 -- 172.21.15.2:0/2039284086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b100590 msgr2=0x557d4b10e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:50.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.972+0000 7f8315a00640 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b100590 0x557d4b10e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.972+0000 7f8315a00640 1 -- 172.21.15.2:0/2039284086 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4b0f7770 con 0x557d4b10ae50 2024-09-10T08:15:50.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.972+0000 7f830fe00640 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b100590 0x557d4b10e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:50.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.972+0000 7f8315a00640 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b10ae50 0x557d4b183cc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f830800d7b0 tx=0x7f8308005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10731ddba9c7b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:50.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.972+0000 7f82f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:50.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.972+0000 7f82f6a00640 1 -- 172.21.15.2:0/2039284086 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308039070 con 0x557d4b10ae50 2024-09-10T08:15:50.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.972+0000 7f82f6a00640 1 -- 172.21.15.2:0/2039284086 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8308024890 con 0x557d4b10ae50 2024-09-10T08:15:50.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.972+0000 7f831a648740 1 -- 172.21.15.2:0/2039284086 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82fc004230 con 0x557d4b10ae50 2024-09-10T08:15:50.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.972+0000 7f82f6a00640 1 -- 172.21.15.2:0/2039284086 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8308045050 con 0x557d4b10ae50 2024-09-10T08:15:50.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.972+0000 7f831a648740 10 client.17502 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:50.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.972+0000 7f831a648740 1 -- 172.21.15.2:0/2039284086 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557d4b185310 con 0x557d4b10ae50 2024-09-10T08:15:50.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.973+0000 7f82f6a00640 1 -- 172.21.15.2:0/2039284086 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82fc004230 con 0x557d4b10ae50 2024-09-10T08:15:50.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.973+0000 7f82f6a00640 1 -- 172.21.15.2:0/2039284086 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8308035550 con 0x557d4b10ae50 2024-09-10T08:15:50.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.973+0000 7f831a648740 10 client.17502 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:50.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.973+0000 7f831a648740 10 client.17502 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:15:50.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.973+0000 7f831a648740 20 client.17502 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:50.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.973+0000 7f831a648740 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557d4b1859d0 0x557d4b187dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:50.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.973+0000 7f831a648740 4 client.17502 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:15:50.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.973+0000 7f831a648740 1 -- 172.21.15.2:0/2039284086 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d4afea010 con 0x557d4b1859d0 2024-09-10T08:15:50.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.973+0000 7f8315000640 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557d4b1859d0 0x557d4b187dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:50.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f8315000640 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557d4b1859d0 0x557d4b187dc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ff489c45f30c953 server_cookie=97b11b817a479f13 in_seq=0 out_seq=0 2024-09-10T08:15:50.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f82f6a00640 10 client.17502 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:15:50.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f82f6a00640 1 -- 172.21.15.2:0/2039284086 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d4afea010 con 0x557d4b1859d0 2024-09-10T08:15:50.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f82f6a00640 10 client.17502 handle_command_reply: tid=0 2024-09-10T08:15:50.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f831a648740 1 client.17502 shutdown 2024-09-10T08:15:50.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f831a648740 1 -- 172.21.15.2:0/2039284086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b10ae50 msgr2=0x557d4b183cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:50.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f831a648740 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b10ae50 0x557d4b183cc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f830800d7b0 tx=0x7f8308005bd0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f831a648740 1 -- 172.21.15.2:0/2039284086 shutdown_connections 2024-09-10T08:15:50.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f831a648740 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557d4b1859d0 0x557d4b187dc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f831a648740 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d4b10ae50 0x557d4b183cc0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f831a648740 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d4b100590 0x557d4b10e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f831a648740 1 --2- 172.21.15.2:0/2039284086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d4b0ffc70 0x557d4b10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f831a648740 1 -- 172.21.15.2:0/2039284086 >> 172.21.15.2:0/2039284086 conn(0x557d4b0fc200 msgr2=0x557d4b107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:50.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f82f6a00640 0 client.17502 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:15:50.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.974+0000 7f831a648740 1 -- 172.21.15.2:0/2039284086 shutdown_connections 2024-09-10T08:15:50.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.975+0000 7f831a648740 1 -- 172.21.15.2:0/2039284086 wait complete. 2024-09-10T08:15:50.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.975+0000 7f831a648740 20 client.17502 trim_cache size 0 max 16384 2024-09-10T08:15:50.980 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:15:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.975+0000 7f8317a00640 1 -- 172.21.15.2:0/2338714595 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82f80a2b80 msgr2=0x7f82f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.975+0000 7f8317a00640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82f80a2b80 0x7f82f80a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8308005fd0 tx=0x7f8308010040 comp rx=0 tx=0).stop 2024-09-10T08:15:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.975+0000 7f8317a00640 1 -- 172.21.15.2:0/2338714595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8310103ff0 msgr2=0x7f831019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.975+0000 7f8317a00640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8310103ff0 0x7f831019a6a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f830400e9f0 tx=0x7f830400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.975+0000 7f8317a00640 1 -- 172.21.15.2:0/2338714595 shutdown_connections 2024-09-10T08:15:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.975+0000 7f8317a00640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8310109120 0x7f831019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.975+0000 7f8317a00640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82f80a2b80 0x7f82f80a5040 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.975+0000 7f8317a00640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83101049a0 0x7f831019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.975+0000 7f8317a00640 1 --2- 172.21.15.2:0/2338714595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8310103ff0 0x7f831019a6a0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.975+0000 7f8317a00640 1 -- 172.21.15.2:0/2338714595 >> 172.21.15.2:0/2338714595 conn(0x7f83100ffa90 msgr2=0x7f8310100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.976+0000 7f8317a00640 1 -- 172.21.15.2:0/2338714595 shutdown_connections 2024-09-10T08:15:50.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:50.976+0000 7f8317a00640 1 -- 172.21.15.2:0/2338714595 wait complete. 2024-09-10T08:15:51.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:51 smithi002 ceph-mon[30563]: pgmap v618: 129 pgs: 129 active+clean; 3.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:15:51.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:51 smithi155 ceph-mon[35405]: pgmap v618: 129 pgs: 129 active+clean; 3.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:15:51.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:51 smithi107 ceph-mon[35048]: pgmap v618: 129 pgs: 129 active+clean; 3.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:15:51.949 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:15:52.326 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:52.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.970+0000 7f390dc00640 1 -- 172.21.15.2:0/2533567207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39081049a0 msgr2=0x7f39081089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:52.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.970+0000 7f390dc00640 1 --2- 172.21.15.2:0/2533567207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39081049a0 0x7f39081089f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009900 tx=0x7f38f002a410 comp rx=0 tx=0).stop 2024-09-10T08:15:52.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.970+0000 7f390dc00640 1 -- 172.21.15.2:0/2533567207 shutdown_connections 2024-09-10T08:15:52.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.970+0000 7f390dc00640 1 --2- 172.21.15.2:0/2533567207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3908109120 0x7f3908110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.970+0000 7f390dc00640 1 --2- 172.21.15.2:0/2533567207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39081049a0 0x7f39081089f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.970+0000 7f390dc00640 1 --2- 172.21.15.2:0/2533567207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3908103ff0 0x7f39081043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.970+0000 7f390dc00640 1 -- 172.21.15.2:0/2533567207 >> 172.21.15.2:0/2533567207 conn(0x7f39080ffa90 msgr2=0x7f3908101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:52.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.970+0000 7f390dc00640 1 -- 172.21.15.2:0/2533567207 shutdown_connections 2024-09-10T08:15:52.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.970+0000 7f390dc00640 1 -- 172.21.15.2:0/2533567207 wait complete. 2024-09-10T08:15:52.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.971+0000 7f390dc00640 1 Processor -- start 2024-09-10T08:15:52.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.971+0000 7f390dc00640 1 -- start start 2024-09-10T08:15:52.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.972+0000 7f390dc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3908103ff0 0x7f390819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:52.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.972+0000 7f390dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39081049a0 0x7f390819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:52.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.972+0000 7f390dc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3908109120 0x7f390819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:52.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.972+0000 7f390dc00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3908114190 con 0x7f3908109120 2024-09-10T08:15:52.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.972+0000 7f390dc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3908114010 con 0x7f39081049a0 2024-09-10T08:15:52.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.972+0000 7f390dc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3908114310 con 0x7f3908103ff0 2024-09-10T08:15:52.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.972+0000 7f3907400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3908109120 0x7f390819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:52.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.972+0000 7f3907400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3908109120 0x7f390819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36694/0 (socket says 172.21.15.2:36694) 2024-09-10T08:15:52.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.972+0000 7f3907400640 1 -- 172.21.15.2:0/937047830 learned_addr learned my addr 172.21.15.2:0/937047830 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:52.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.972+0000 7f3906a00640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3908103ff0 0x7f390819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:52.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.972+0000 7f38fe000640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39081049a0 0x7f390819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:52.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.973+0000 7f3907400640 1 -- 172.21.15.2:0/937047830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3908103ff0 msgr2=0x7f390819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:52.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.973+0000 7f3907400640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3908103ff0 0x7f390819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.973+0000 7f3907400640 1 -- 172.21.15.2:0/937047830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39081049a0 msgr2=0x7f390819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:52.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.973+0000 7f3907400640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39081049a0 0x7f390819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.973+0000 7f3907400640 1 -- 172.21.15.2:0/937047830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390819f5a0 con 0x7f3908109120 2024-09-10T08:15:52.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.973+0000 7f3907400640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3908109120 0x7f390819eec0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f390000eaa0 tx=0x7f390000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:52.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.973+0000 7f38ffe00640 1 -- 172.21.15.2:0/937047830 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390000cdb0 con 0x7f3908109120 2024-09-10T08:15:52.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.973+0000 7f38ffe00640 1 -- 172.21.15.2:0/937047830 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3900005020 con 0x7f3908109120 2024-09-10T08:15:52.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.973+0000 7f390dc00640 1 -- 172.21.15.2:0/937047830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390819f890 con 0x7f3908109120 2024-09-10T08:15:52.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.973+0000 7f38ffe00640 1 -- 172.21.15.2:0/937047830 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39000116b0 con 0x7f3908109120 2024-09-10T08:15:52.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.974+0000 7f390dc00640 1 -- 172.21.15.2:0/937047830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39081a7170 con 0x7f3908109120 2024-09-10T08:15:52.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.977+0000 7f38ffe00640 1 -- 172.21.15.2:0/937047830 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3900005560 con 0x7f3908109120 2024-09-10T08:15:52.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.977+0000 7f391060f740 1 Processor -- start 2024-09-10T08:15:52.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.977+0000 7f391060f740 1 -- start start 2024-09-10T08:15:52.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.977+0000 7f391060f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedb0c70 0x5638fedb1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:52.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.977+0000 7f391060f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb1590 0x5638fedbb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:52.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.977+0000 7f391060f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedbbe50 0x5638fedbe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:52.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.977+0000 7f391060f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5638fed173e0 con 0x5638fedbbe50 2024-09-10T08:15:52.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.977+0000 7f391060f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5638fecbed80 con 0x5638fedb0c70 2024-09-10T08:15:52.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.977+0000 7f391060f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638fed133a0 con 0x5638fedb1590 2024-09-10T08:15:52.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.978+0000 7f38fe000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedb0c70 0x5638fedb1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:52.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.978+0000 7f3906a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb1590 0x5638fedbb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:52.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.978+0000 7f38fe000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedb0c70 0x5638fedb1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54542/0 (socket says 172.21.15.2:54542) 2024-09-10T08:15:52.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.978+0000 7f38fe000640 1 -- 172.21.15.2:0/4224859823 learned_addr learned my addr 172.21.15.2:0/4224859823 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:52.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.978+0000 7f38fe000640 1 --2- 172.21.15.2:0/4224859823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedbbe50 0x5638fedbe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:52.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.978+0000 7f3906a00640 1 -- 172.21.15.2:0/4224859823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedb0c70 msgr2=0x5638fedb1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:52.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.978+0000 7f3906a00640 1 --2- 172.21.15.2:0/4224859823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedb0c70 0x5638fedb1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.978+0000 7f3906a00640 1 -- 172.21.15.2:0/4224859823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedbbe50 msgr2=0x5638fedbe240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:52.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.978+0000 7f3906a00640 1 --2- 172.21.15.2:0/4224859823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedbbe50 0x5638fedbe240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.978+0000 7f3906a00640 1 -- 172.21.15.2:0/4224859823 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638fecddae0 con 0x5638fedb1590 2024-09-10T08:15:52.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.978+0000 7f3906a00640 1 --2- 172.21.15.2:0/4224859823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb1590 0x5638fedbb910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009c20 tx=0x7f38f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a09fa2461e316c2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:52.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.979+0000 7f38f7e00640 1 -- 172.21.15.2:0/4224859823 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8004050 con 0x5638fedb1590 2024-09-10T08:15:52.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.979+0000 7f38f7e00640 1 -- 172.21.15.2:0/4224859823 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f8005500 con 0x5638fedb1590 2024-09-10T08:15:52.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.979+0000 7f38f7e00640 1 -- 172.21.15.2:0/4224859823 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8004e00 con 0x5638fedb1590 2024-09-10T08:15:52.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.979+0000 7f38ffe00640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38ec0a2b80 0x7f38ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.979+0000 7f38ffe00640 1 -- 172.21.15.2:0/937047830 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f39000cf590 con 0x7f3908109120 2024-09-10T08:15:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.979+0000 7f38fe000640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38ec0a2b80 0x7f38ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.979+0000 7f391060f740 1 -- 172.21.15.2:0/4224859823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb1590 msgr2=0x5638fedbb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.979+0000 7f391060f740 1 --2- 172.21.15.2:0/4224859823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb1590 0x5638fedbb910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009c20 tx=0x7f38f8011810 comp rx=0 tx=0).stop 2024-09-10T08:15:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.980+0000 7f391060f740 1 -- 172.21.15.2:0/4224859823 shutdown_connections 2024-09-10T08:15:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.980+0000 7f391060f740 1 --2- 172.21.15.2:0/4224859823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedbbe50 0x5638fedbe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.980+0000 7f391060f740 1 --2- 172.21.15.2:0/4224859823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb1590 0x5638fedbb910 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.980+0000 7f391060f740 1 --2- 172.21.15.2:0/4224859823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedb0c70 0x5638fedb1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.980+0000 7f391060f740 1 -- 172.21.15.2:0/4224859823 >> 172.21.15.2:0/4224859823 conn(0x5638fedad200 msgr2=0x5638fedad5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.980+0000 7f391060f740 1 -- 172.21.15.2:0/4224859823 shutdown_connections 2024-09-10T08:15:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.980+0000 7f391060f740 1 -- 172.21.15.2:0/4224859823 wait complete. 2024-09-10T08:15:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.980+0000 7f38fe000640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38ec0a2b80 0x7f38ec0a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f38f0005c40 tx=0x7f38f002a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.980+0000 7f391060f740 1 -- start start 2024-09-10T08:15:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f391060f740 1 Processor -- start 2024-09-10T08:15:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f391060f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb0c70 0x5638fedb8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f391060f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedb1590 0x5638fedb92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f391060f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedbbe50 0x5638fedb9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f391060f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5638fedbea20 con 0x5638fedb0c70 2024-09-10T08:15:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f391060f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5638fecd20f0 con 0x5638fedb1590 2024-09-10T08:15:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f391060f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638fedbeba0 con 0x5638fedbbe50 2024-09-10T08:15:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f391060f740 10 client.0 _ll_register_callbacks cb 0x5638fecb9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f3906a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb0c70 0x5638fedb8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f38fe000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedb1590 0x5638fedb92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f3907400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedbbe50 0x5638fedb9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f3906a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb0c70 0x5638fedb8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36714/0 (socket says 172.21.15.2:36714) 2024-09-10T08:15:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f38fe000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedb1590 0x5638fedb92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54558/0 (socket says 172.21.15.2:54558) 2024-09-10T08:15:52.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.981+0000 7f3906a00640 1 -- 172.21.15.2:0/3264067964 learned_addr learned my addr 172.21.15.2:0/3264067964 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:52.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.982+0000 7f3907400640 1 -- 172.21.15.2:0/3264067964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedb1590 msgr2=0x5638fedb92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:52.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.982+0000 7f3907400640 1 --2- 172.21.15.2:0/3264067964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedb1590 0x5638fedb92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.982+0000 7f3907400640 1 -- 172.21.15.2:0/3264067964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb0c70 msgr2=0x5638fedb8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:52.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.982+0000 7f3907400640 1 --2- 172.21.15.2:0/3264067964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb0c70 0x5638fedb8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.982+0000 7f3907400640 1 -- 172.21.15.2:0/3264067964 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638fea883c0 con 0x5638fedbbe50 2024-09-10T08:15:52.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.982+0000 7f3907400640 1 --2- 172.21.15.2:0/3264067964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedbbe50 0x5638fedb9800 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3900000f80 tx=0x7f390009fe10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d0594e77efb16a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:52.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.982+0000 7f38f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:15:52.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f38f4c00640 1 -- 172.21.15.2:0/3264067964 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390001f040 con 0x5638fedbbe50 2024-09-10T08:15:52.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f391060f740 1 -- 172.21.15.2:0/3264067964 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38f8005500 con 0x5638fedbbe50 2024-09-10T08:15:52.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f38f4c00640 1 -- 172.21.15.2:0/3264067964 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f39000a4180 con 0x5638fedbbe50 2024-09-10T08:15:52.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f38f4c00640 1 -- 172.21.15.2:0/3264067964 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3900099e10 con 0x5638fedbbe50 2024-09-10T08:15:52.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f391060f740 10 client.27487 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:52.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f391060f740 1 -- 172.21.15.2:0/3264067964 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5638fedcab10 con 0x5638fedbbe50 2024-09-10T08:15:52.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f38f4c00640 1 -- 172.21.15.2:0/3264067964 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f8005500 con 0x5638fedbbe50 2024-09-10T08:15:52.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f38f4c00640 1 -- 172.21.15.2:0/3264067964 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f390000f040 con 0x5638fedbbe50 2024-09-10T08:15:52.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f391060f740 10 client.27487 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:52.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f391060f740 10 client.27487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:52.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f391060f740 20 client.27487 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:52.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f391060f740 1 --2- 172.21.15.2:0/3264067964 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5638fedcbfd0 0x5638fedce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:52.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f391060f740 4 client.27487 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:52.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.983+0000 7f391060f740 1 -- 172.21.15.2:0/3264067964 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638fed24950 con 0x5638fedcbfd0 2024-09-10T08:15:52.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.984+0000 7f3906a00640 1 --2- 172.21.15.2:0/3264067964 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5638fedcbfd0 0x5638fedce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:52.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.984+0000 7f3906a00640 1 --2- 172.21.15.2:0/3264067964 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5638fedcbfd0 0x5638fedce420 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30544a979a89cb2a server_cookie=add4a52d9000c726 in_seq=0 out_seq=0 2024-09-10T08:15:52.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.984+0000 7f38f4c00640 10 client.27487 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:52.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.985+0000 7f38f4c00640 1 -- 172.21.15.2:0/3264067964 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638fed24950 con 0x5638fedcbfd0 2024-09-10T08:15:52.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.985+0000 7f38f4c00640 10 client.27487 handle_command_reply: tid=0 2024-09-10T08:15:52.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.985+0000 7f391060f740 1 client.27487 shutdown 2024-09-10T08:15:52.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.985+0000 7f391060f740 1 -- 172.21.15.2:0/3264067964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedbbe50 msgr2=0x5638fedb9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:52.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.985+0000 7f391060f740 1 --2- 172.21.15.2:0/3264067964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedbbe50 0x5638fedb9800 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3900000f80 tx=0x7f390009fe10 comp rx=0 tx=0).stop 2024-09-10T08:15:52.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.986+0000 7f391060f740 1 -- 172.21.15.2:0/3264067964 shutdown_connections 2024-09-10T08:15:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.986+0000 7f391060f740 1 --2- 172.21.15.2:0/3264067964 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5638fedcbfd0 0x5638fedce420 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.986+0000 7f391060f740 1 --2- 172.21.15.2:0/3264067964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedbbe50 0x5638fedb9800 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.986+0000 7f391060f740 1 --2- 172.21.15.2:0/3264067964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedb1590 0x5638fedb92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.986+0000 7f391060f740 1 --2- 172.21.15.2:0/3264067964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb0c70 0x5638fedb8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.986+0000 7f38f4c00640 0 client.27487 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.986+0000 7f391060f740 1 -- 172.21.15.2:0/3264067964 >> 172.21.15.2:0/3264067964 conn(0x5638fedad200 msgr2=0x5638fedb2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.986+0000 7f391060f740 1 -- 172.21.15.2:0/3264067964 shutdown_connections 2024-09-10T08:15:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.986+0000 7f391060f740 1 -- 172.21.15.2:0/3264067964 wait complete. 2024-09-10T08:15:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:52.986+0000 7f391060f740 20 client.27487 trim_cache size 0 max 16384 2024-09-10T08:15:52.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.000+0000 7f391060f740 1 Processor -- start 2024-09-10T08:15:52.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.000+0000 7f391060f740 1 -- start start 2024-09-10T08:15:53.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.000+0000 7f391060f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb0c70 0x5638fedb2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:53.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.000+0000 7f391060f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb1590 0x5638fedd3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:53.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.000+0000 7f391060f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedbbe50 0x5638fedd4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:53.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.000+0000 7f391060f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f39081b38d0 con 0x5638fedb0c70 2024-09-10T08:15:53.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.000+0000 7f391060f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39081b11a0 con 0x5638fedbbe50 2024-09-10T08:15:53.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.000+0000 7f391060f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39081aeff0 con 0x5638fedb1590 2024-09-10T08:15:53.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.000+0000 7f3906a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb0c70 0x5638fedb2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:53.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.000+0000 7f3906a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb0c70 0x5638fedb2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36716/0 (socket says 172.21.15.2:36716) 2024-09-10T08:15:53.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.000+0000 7f3906a00640 1 -- 172.21.15.2:0/3248363159 learned_addr learned my addr 172.21.15.2:0/3248363159 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:53.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.001+0000 7f38fe000640 1 --2- 172.21.15.2:0/3248363159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb1590 0x5638fedd3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:53.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.001+0000 7f3907400640 1 --2- 172.21.15.2:0/3248363159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedbbe50 0x5638fedd4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:53.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.001+0000 7f3906a00640 1 -- 172.21.15.2:0/3248363159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb1590 msgr2=0x5638fedd3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:53.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.001+0000 7f3906a00640 1 --2- 172.21.15.2:0/3248363159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb1590 0x5638fedd3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.001+0000 7f3906a00640 1 -- 172.21.15.2:0/3248363159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedbbe50 msgr2=0x5638fedd4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:53.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.001+0000 7f3906a00640 1 --2- 172.21.15.2:0/3248363159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedbbe50 0x5638fedd4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.001+0000 7f3906a00640 1 -- 172.21.15.2:0/3248363159 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638feda8770 con 0x5638fedb0c70 2024-09-10T08:15:53.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.001+0000 7f3907400640 1 --2- 172.21.15.2:0/3248363159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedbbe50 0x5638fedd4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:53.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.001+0000 7f3906a00640 1 --2- 172.21.15.2:0/3248363159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb0c70 0x5638fedb2dd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f38f80099b0 tx=0x7f38f8005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1882afd9f5f7cc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:53.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.001+0000 7f38f7400640 1 -- 172.21.15.2:0/3248363159 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8004250 con 0x5638fedb0c70 2024-09-10T08:15:53.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.001+0000 7f38f7400640 1 -- 172.21.15.2:0/3248363159 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f80043f0 con 0x5638fedb0c70 2024-09-10T08:15:53.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.001+0000 7f38f7400640 1 -- 172.21.15.2:0/3248363159 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f80035f0 con 0x5638fedb0c70 2024-09-10T08:15:53.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.002+0000 7f391060f740 1 -- 172.21.15.2:0/3248363159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb0c70 msgr2=0x5638fedb2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:53.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.002+0000 7f391060f740 1 --2- 172.21.15.2:0/3248363159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb0c70 0x5638fedb2dd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f38f80099b0 tx=0x7f38f8005d90 comp rx=0 tx=0).stop 2024-09-10T08:15:53.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.002+0000 7f391060f740 1 -- 172.21.15.2:0/3248363159 shutdown_connections 2024-09-10T08:15:53.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.002+0000 7f391060f740 1 --2- 172.21.15.2:0/3248363159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedbbe50 0x5638fedd4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.002+0000 7f391060f740 1 --2- 172.21.15.2:0/3248363159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb1590 0x5638fedd3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.002+0000 7f391060f740 1 --2- 172.21.15.2:0/3248363159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb0c70 0x5638fedb2dd0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.002+0000 7f391060f740 1 -- 172.21.15.2:0/3248363159 >> 172.21.15.2:0/3248363159 conn(0x5638fedad200 msgr2=0x5638fedb26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.002+0000 7f391060f740 1 -- 172.21.15.2:0/3248363159 shutdown_connections 2024-09-10T08:15:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.002+0000 7f391060f740 1 -- 172.21.15.2:0/3248363159 wait complete. 2024-09-10T08:15:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.003+0000 7f391060f740 1 -- start start 2024-09-10T08:15:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.003+0000 7f391060f740 1 Processor -- start 2024-09-10T08:15:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.003+0000 7f391060f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb0c70 0x5638fedb5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.003+0000 7f391060f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb1590 0x5638fedb6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.003+0000 7f391060f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedbbe50 0x5638fedb6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.003+0000 7f391060f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f39081af660 con 0x5638fedb1590 2024-09-10T08:15:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.003+0000 7f391060f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39081b0ad0 con 0x5638fedbbe50 2024-09-10T08:15:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.003+0000 7f391060f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39081afd30 con 0x5638fedb0c70 2024-09-10T08:15:53.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.003+0000 7f391060f740 10 client.0 _ll_register_callbacks cb 0x7f3900097db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:53.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.004+0000 7f38fe000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb1590 0x5638fedb6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:53.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.004+0000 7f3906a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb0c70 0x5638fedb5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:53.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.004+0000 7f38fe000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb1590 0x5638fedb6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36722/0 (socket says 172.21.15.2:36722) 2024-09-10T08:15:53.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.004+0000 7f38fe000640 1 -- 172.21.15.2:0/157697660 learned_addr learned my addr 172.21.15.2:0/157697660 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:53.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.004+0000 7f3907400640 1 --2- 172.21.15.2:0/157697660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedbbe50 0x5638fedb6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:53.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.004+0000 7f3906a00640 1 -- 172.21.15.2:0/157697660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedbbe50 msgr2=0x5638fedb6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:53.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.004+0000 7f3906a00640 1 --2- 172.21.15.2:0/157697660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedbbe50 0x5638fedb6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.004+0000 7f3906a00640 1 -- 172.21.15.2:0/157697660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb1590 msgr2=0x5638fedb6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:53.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.004+0000 7f3906a00640 1 --2- 172.21.15.2:0/157697660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb1590 0x5638fedb6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.004+0000 7f3906a00640 1 -- 172.21.15.2:0/157697660 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638fee35ce0 con 0x5638fedb0c70 2024-09-10T08:15:53.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.004+0000 7f3906a00640 1 --2- 172.21.15.2:0/157697660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb0c70 0x5638fedb5df0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f38f80059f0 tx=0x7f38f80045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d970e1821e0a66c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:53.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.004+0000 7f38f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:15:53.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.005+0000 7f38f4c00640 1 -- 172.21.15.2:0/157697660 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f80047e0 con 0x5638fedb0c70 2024-09-10T08:15:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.005+0000 7f38f4c00640 1 -- 172.21.15.2:0/157697660 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f38f80196c0 con 0x5638fedb0c70 2024-09-10T08:15:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.005+0000 7f391060f740 1 -- 172.21.15.2:0/157697660 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38f80043f0 con 0x5638fedb0c70 2024-09-10T08:15:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.005+0000 7f38f4c00640 1 -- 172.21.15.2:0/157697660 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f38f8004250 con 0x5638fedb0c70 2024-09-10T08:15:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.005+0000 7f391060f740 10 client.27493 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.005+0000 7f391060f740 1 -- 172.21.15.2:0/157697660 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5638fee36fe0 con 0x5638fedb0c70 2024-09-10T08:15:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.006+0000 7f38f4c00640 1 -- 172.21.15.2:0/157697660 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f80043f0 con 0x5638fedb0c70 2024-09-10T08:15:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.006+0000 7f38f4c00640 1 -- 172.21.15.2:0/157697660 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f38f8003bf0 con 0x5638fedb0c70 2024-09-10T08:15:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.006+0000 7f391060f740 10 client.27493 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.006+0000 7f391060f740 10 client.27493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.006+0000 7f391060f740 20 client.27493 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.006+0000 7f391060f740 1 --2- 172.21.15.2:0/157697660 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5638fee37500 0x5638fee398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.006+0000 7f391060f740 4 client.27493 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:15:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.006+0000 7f391060f740 1 -- 172.21.15.2:0/157697660 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638fed1ee60 con 0x5638fee37500 2024-09-10T08:15:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.006+0000 7f38fe000640 1 --2- 172.21.15.2:0/157697660 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5638fee37500 0x5638fee398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f38fe000640 1 --2- 172.21.15.2:0/157697660 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5638fee37500 0x5638fee398f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c08ca2b39fec3862 server_cookie=61f8b59916d375f7 in_seq=0 out_seq=0 2024-09-10T08:15:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f38f4c00640 10 client.27493 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f38f4c00640 1 -- 172.21.15.2:0/157697660 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638fed1ee60 con 0x5638fee37500 2024-09-10T08:15:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f38f4c00640 10 client.27493 handle_command_reply: tid=0 2024-09-10T08:15:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f391060f740 1 client.27493 shutdown 2024-09-10T08:15:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f391060f740 1 -- 172.21.15.2:0/157697660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb0c70 msgr2=0x5638fedb5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f391060f740 1 --2- 172.21.15.2:0/157697660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb0c70 0x5638fedb5df0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f38f80059f0 tx=0x7f38f80045c0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f391060f740 1 -- 172.21.15.2:0/157697660 shutdown_connections 2024-09-10T08:15:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f391060f740 1 --2- 172.21.15.2:0/157697660 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5638fee37500 0x5638fee398f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f391060f740 1 --2- 172.21.15.2:0/157697660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638fedbbe50 0x5638fedb6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f391060f740 1 --2- 172.21.15.2:0/157697660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638fedb1590 0x5638fedb6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f38f4c00640 0 client.27493 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:53.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f391060f740 1 --2- 172.21.15.2:0/157697660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fedb0c70 0x5638fedb5df0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.007+0000 7f391060f740 1 -- 172.21.15.2:0/157697660 >> 172.21.15.2:0/157697660 conn(0x5638fedad200 msgr2=0x5638fedb9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:53.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.008+0000 7f391060f740 1 -- 172.21.15.2:0/157697660 shutdown_connections 2024-09-10T08:15:53.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.008+0000 7f391060f740 1 -- 172.21.15.2:0/157697660 wait complete. 2024-09-10T08:15:53.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.008+0000 7f391060f740 20 client.27493 trim_cache size 0 max 16384 2024-09-10T08:15:53.011 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:15:53.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.008+0000 7f390dc00640 1 -- 172.21.15.2:0/937047830 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38ec0a2b80 msgr2=0x7f38ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:53.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.008+0000 7f390dc00640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38ec0a2b80 0x7f38ec0a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f38f0005c40 tx=0x7f38f002a410 comp rx=0 tx=0).stop 2024-09-10T08:15:53.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.008+0000 7f390dc00640 1 -- 172.21.15.2:0/937047830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3908109120 msgr2=0x7f390819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:53.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.008+0000 7f390dc00640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3908109120 0x7f390819eec0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f390000eaa0 tx=0x7f390000ef60 comp rx=0 tx=0).stop 2024-09-10T08:15:53.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.008+0000 7f390dc00640 1 -- 172.21.15.2:0/937047830 shutdown_connections 2024-09-10T08:15:53.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.008+0000 7f390dc00640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3908109120 0x7f390819eec0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.009+0000 7f390dc00640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39081049a0 0x7f390819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.009+0000 7f390dc00640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f38ec0a2b80 0x7f38ec0a5040 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.009+0000 7f390dc00640 1 --2- 172.21.15.2:0/937047830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3908103ff0 0x7f390819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:53.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.009+0000 7f390dc00640 1 -- 172.21.15.2:0/937047830 >> 172.21.15.2:0/937047830 conn(0x7f39080ffa90 msgr2=0x7f3908101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:53.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.009+0000 7f390dc00640 1 -- 172.21.15.2:0/937047830 shutdown_connections 2024-09-10T08:15:53.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:53.009+0000 7f390dc00640 1 -- 172.21.15.2:0/937047830 wait complete. 2024-09-10T08:15:53.709 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:53 smithi155 ceph-mon[35405]: pgmap v619: 129 pgs: 129 active+clean; 3.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:15:53.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:53 smithi002 ceph-mon[30563]: pgmap v619: 129 pgs: 129 active+clean; 3.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:15:53.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:53 smithi107 ceph-mon[35048]: pgmap v619: 129 pgs: 129 active+clean; 3.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:15:54.943 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:15:54.944 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:15:55.332 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:55.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:55 smithi002 ceph-mon[30563]: pgmap v620: 129 pgs: 129 active+clean; 3.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:15:55.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.887+0000 7fb078e00640 1 -- 172.21.15.2:0/859588336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb074109140 msgr2=0x7fb074110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:55.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.887+0000 7fb078e00640 1 --2- 172.21.15.2:0/859588336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb074109140 0x7fb074110cc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009ca0 tx=0x7fb06c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.888+0000 7fb078e00640 1 -- 172.21.15.2:0/859588336 shutdown_connections 2024-09-10T08:15:55.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.888+0000 7fb078e00640 1 --2- 172.21.15.2:0/859588336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb074109140 0x7fb074110cc0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.888+0000 7fb078e00640 1 --2- 172.21.15.2:0/859588336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0741049c0 0x7fb074108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.888+0000 7fb078e00640 1 --2- 172.21.15.2:0/859588336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb074104010 0x7fb0741043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.888+0000 7fb078e00640 1 -- 172.21.15.2:0/859588336 >> 172.21.15.2:0/859588336 conn(0x7fb0740ffa50 msgr2=0x7fb074101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:55.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.888+0000 7fb078e00640 1 -- 172.21.15.2:0/859588336 shutdown_connections 2024-09-10T08:15:55.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.888+0000 7fb078e00640 1 -- 172.21.15.2:0/859588336 wait complete. 2024-09-10T08:15:55.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.889+0000 7fb078e00640 1 Processor -- start 2024-09-10T08:15:55.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.889+0000 7fb078e00640 1 -- start start 2024-09-10T08:15:55.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.889+0000 7fb078e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb074104010 0x7fb07419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.889+0000 7fb078e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0741049c0 0x7fb07419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.889+0000 7fb078e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074109140 0x7fb07419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.889+0000 7fb078e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb074114170 con 0x7fb0741049c0 2024-09-10T08:15:55.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.889+0000 7fb078e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb074113ff0 con 0x7fb074109140 2024-09-10T08:15:55.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.889+0000 7fb078e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741142f0 con 0x7fb074104010 2024-09-10T08:15:55.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.890+0000 7fb06be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0741049c0 0x7fb07419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.890+0000 7fb071e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb074104010 0x7fb07419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.890+0000 7fb06be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0741049c0 0x7fb07419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36732/0 (socket says 172.21.15.2:36732) 2024-09-10T08:15:55.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.890+0000 7fb071e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb074104010 0x7fb07419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56764/0 (socket says 172.21.15.2:56764) 2024-09-10T08:15:55.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.890+0000 7fb06be00640 1 -- 172.21.15.2:0/3768342275 learned_addr learned my addr 172.21.15.2:0/3768342275 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:55.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.890+0000 7fb072800640 1 --2- 172.21.15.2:0/3768342275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074109140 0x7fb07419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.890+0000 7fb071e00640 1 -- 172.21.15.2:0/3768342275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074109140 msgr2=0x7fb07419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:55.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.890+0000 7fb071e00640 1 --2- 172.21.15.2:0/3768342275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074109140 0x7fb07419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.890+0000 7fb071e00640 1 -- 172.21.15.2:0/3768342275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0741049c0 msgr2=0x7fb07419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:55.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.890+0000 7fb071e00640 1 --2- 172.21.15.2:0/3768342275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0741049c0 0x7fb07419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.890+0000 7fb071e00640 1 -- 172.21.15.2:0/3768342275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07419f5c0 con 0x7fb074104010 2024-09-10T08:15:55.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.890+0000 7fb071e00640 1 --2- 172.21.15.2:0/3768342275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb074104010 0x7fb07419a610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb06400e9f0 tx=0x7fb06400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:55.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.891+0000 7fb06aa00640 1 -- 172.21.15.2:0/3768342275 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06400ccc0 con 0x7fb074104010 2024-09-10T08:15:55.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.891+0000 7fb078e00640 1 -- 172.21.15.2:0/3768342275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07419f8b0 con 0x7fb074104010 2024-09-10T08:15:55.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.891+0000 7fb078e00640 1 -- 172.21.15.2:0/3768342275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0741a7190 con 0x7fb074104010 2024-09-10T08:15:55.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.891+0000 7fb06aa00640 1 -- 172.21.15.2:0/3768342275 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb064005020 con 0x7fb074104010 2024-09-10T08:15:55.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.891+0000 7fb06aa00640 1 -- 172.21.15.2:0/3768342275 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06401c070 con 0x7fb074104010 2024-09-10T08:15:55.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.894+0000 7fb07b8a9740 1 Processor -- start 2024-09-10T08:15:55.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.894+0000 7fb07b8a9740 1 -- start start 2024-09-10T08:15:55.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.894+0000 7fb06aa00640 1 -- 172.21.15.2:0/3768342275 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb064004a00 con 0x7fb074104010 2024-09-10T08:15:55.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.894+0000 7fb07b8a9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cbac0 0x5564cc0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.894+0000 7fb07b8a9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0cc3e0 0x5564cc0d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.894+0000 7fb07b8a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 0x5564cc0d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.894+0000 7fb07b8a9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5564cc032290 con 0x5564cc0cc3e0 2024-09-10T08:15:55.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.894+0000 7fb07b8a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5564cbfd9b40 con 0x5564cc0d6ca0 2024-09-10T08:15:55.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.894+0000 7fb07b8a9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5564cc02e250 con 0x5564cc0cbac0 2024-09-10T08:15:55.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.895+0000 7fb071e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0cc3e0 0x5564cc0d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.895+0000 7fb06be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 0x5564cc0d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.895+0000 7fb072800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cbac0 0x5564cc0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.895+0000 7fb06be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 0x5564cc0d9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54608/0 (socket says 172.21.15.2:54608) 2024-09-10T08:15:55.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.895+0000 7fb06be00640 1 -- 172.21.15.2:0/3482254689 learned_addr learned my addr 172.21.15.2:0/3482254689 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:55.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.895+0000 7fb06be00640 1 -- 172.21.15.2:0/3482254689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cbac0 msgr2=0x5564cc0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:55.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.895+0000 7fb06be00640 1 --2- 172.21.15.2:0/3482254689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cbac0 0x5564cc0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.895+0000 7fb06be00640 1 -- 172.21.15.2:0/3482254689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0cc3e0 msgr2=0x5564cc0d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:55.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.895+0000 7fb06be00640 1 --2- 172.21.15.2:0/3482254689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0cc3e0 0x5564cc0d6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.895+0000 7fb06be00640 1 -- 172.21.15.2:0/3482254689 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564cbda36a0 con 0x5564cc0d6ca0 2024-09-10T08:15:55.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.896+0000 7fb06be00640 1 --2- 172.21.15.2:0/3482254689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 0x5564cc0d9090 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009cf0 tx=0x7fb05c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e080556c90d709b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:55.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.896+0000 7fb063400640 1 -- 172.21.15.2:0/3482254689 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c005220 con 0x5564cc0d6ca0 2024-09-10T08:15:55.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.896+0000 7fb063400640 1 -- 172.21.15.2:0/3482254689 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05c004c80 con 0x5564cc0d6ca0 2024-09-10T08:15:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.896+0000 7fb063400640 1 -- 172.21.15.2:0/3482254689 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c019940 con 0x5564cc0d6ca0 2024-09-10T08:15:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.896+0000 7fb06aa00640 1 --2- 172.21.15.2:0/3768342275 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb0580a2c00 0x7fb0580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.896+0000 7fb06aa00640 1 -- 172.21.15.2:0/3768342275 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb0640cf2b0 con 0x7fb074104010 2024-09-10T08:15:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.896+0000 7fb072800640 1 --2- 172.21.15.2:0/3768342275 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb0580a2c00 0x7fb0580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.897+0000 7fb07b8a9740 1 -- 172.21.15.2:0/3482254689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 msgr2=0x5564cc0d9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.897+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/3482254689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 0x5564cc0d9090 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009cf0 tx=0x7fb05c011810 comp rx=0 tx=0).stop 2024-09-10T08:15:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.897+0000 7fb07b8a9740 1 -- 172.21.15.2:0/3482254689 shutdown_connections 2024-09-10T08:15:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.897+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/3482254689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 0x5564cc0d9090 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.897+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/3482254689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0cc3e0 0x5564cc0d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.897+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/3482254689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cbac0 0x5564cc0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.897+0000 7fb07b8a9740 1 -- 172.21.15.2:0/3482254689 >> 172.21.15.2:0/3482254689 conn(0x5564cc0c8050 msgr2=0x5564cc0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.897+0000 7fb07b8a9740 1 -- 172.21.15.2:0/3482254689 shutdown_connections 2024-09-10T08:15:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.897+0000 7fb07b8a9740 1 -- 172.21.15.2:0/3482254689 wait complete. 2024-09-10T08:15:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.897+0000 7fb072800640 1 --2- 172.21.15.2:0/3768342275 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb0580a2c00 0x7fb0580a50c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb06c004af0 tx=0x7fb06c004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.898+0000 7fb07b8a9740 1 -- start start 2024-09-10T08:15:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.898+0000 7fb07b8a9740 1 Processor -- start 2024-09-10T08:15:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.898+0000 7fb07b8a9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0cbac0 0x5564cc0d64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.898+0000 7fb07b8a9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 0x5564cc0d37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.898+0000 7fb07b8a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 0x5564cc0d3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.898+0000 7fb07b8a9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5564cc0d9910 con 0x5564cc0cbac0 2024-09-10T08:15:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.898+0000 7fb07b8a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5564cbfecfa0 con 0x5564cc0d6ca0 2024-09-10T08:15:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.898+0000 7fb07b8a9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5564cc0d9a90 con 0x5564cc0cc3e0 2024-09-10T08:15:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.898+0000 7fb07b8a9740 10 client.0 _ll_register_callbacks cb 0x5564cbd40ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.899+0000 7fb072800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 0x5564cc0d3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.899+0000 7fb071e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0cbac0 0x5564cc0d64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.899+0000 7fb06be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 0x5564cc0d37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.899+0000 7fb072800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 0x5564cc0d3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:54614/0 (socket says 172.21.15.2:54614) 2024-09-10T08:15:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.899+0000 7fb072800640 1 -- 172.21.15.2:0/1921421988 learned_addr learned my addr 172.21.15.2:0/1921421988 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.899+0000 7fb072800640 1 -- 172.21.15.2:0/1921421988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 msgr2=0x5564cc0d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.899+0000 7fb072800640 1 --2- 172.21.15.2:0/1921421988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 0x5564cc0d37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.899+0000 7fb072800640 1 -- 172.21.15.2:0/1921421988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0cbac0 msgr2=0x5564cc0d64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.899+0000 7fb072800640 1 --2- 172.21.15.2:0/1921421988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0cbac0 0x5564cc0d64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.899+0000 7fb072800640 1 -- 172.21.15.2:0/1921421988 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564cc0d44f0 con 0x5564cc0d6ca0 2024-09-10T08:15:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.899+0000 7fb072800640 1 --2- 172.21.15.2:0/1921421988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 0x5564cc0d3ce0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb06c035000 tx=0x7fb06c02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d974566d63de73b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.899+0000 7fb057e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:15:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.900+0000 7fb057e00640 1 -- 172.21.15.2:0/1921421988 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c004250 con 0x5564cc0d6ca0 2024-09-10T08:15:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.900+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1921421988 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05c004c80 con 0x5564cc0d6ca0 2024-09-10T08:15:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.900+0000 7fb057e00640 1 -- 172.21.15.2:0/1921421988 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb06c0043f0 con 0x5564cc0d6ca0 2024-09-10T08:15:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.900+0000 7fb057e00640 1 -- 172.21.15.2:0/1921421988 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb06c05c050 con 0x5564cc0d6ca0 2024-09-10T08:15:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.900+0000 7fb07b8a9740 10 client.27569 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.900+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1921421988 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5564cc0e57c0 con 0x5564cc0d6ca0 2024-09-10T08:15:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.900+0000 7fb057e00640 1 -- 172.21.15.2:0/1921421988 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05c004c80 con 0x5564cc0d6ca0 2024-09-10T08:15:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.901+0000 7fb057e00640 1 -- 172.21.15.2:0/1921421988 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb06c0305c0 con 0x5564cc0d6ca0 2024-09-10T08:15:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.901+0000 7fb07b8a9740 10 client.27569 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.901+0000 7fb07b8a9740 10 client.27569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.901+0000 7fb07b8a9740 20 client.27569 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.901+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/1921421988 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5564cc0e6c80 0x5564cc0e9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.901+0000 7fb07b8a9740 4 client.27569 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.901+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1921421988 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564cc03f800 con 0x5564cc0e6c80 2024-09-10T08:15:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.901+0000 7fb071e00640 1 --2- 172.21.15.2:0/1921421988 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5564cc0e6c80 0x5564cc0e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.902+0000 7fb071e00640 1 --2- 172.21.15.2:0/1921421988 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5564cc0e6c80 0x5564cc0e9120 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6b1c26f05e46dcc server_cookie=d20608ddcc4b9218 in_seq=0 out_seq=0 2024-09-10T08:15:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.902+0000 7fb057e00640 10 client.27569 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.903+0000 7fb057e00640 1 -- 172.21.15.2:0/1921421988 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564cc03f800 con 0x5564cc0e6c80 2024-09-10T08:15:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.903+0000 7fb057e00640 10 client.27569 handle_command_reply: tid=0 2024-09-10T08:15:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.903+0000 7fb07b8a9740 1 client.27569 shutdown 2024-09-10T08:15:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.903+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1921421988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 msgr2=0x5564cc0d3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.903+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/1921421988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 0x5564cc0d3ce0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb06c035000 tx=0x7fb06c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.903+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1921421988 shutdown_connections 2024-09-10T08:15:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.903+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/1921421988 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5564cc0e6c80 0x5564cc0e9120 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.903+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/1921421988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0d6ca0 0x5564cc0d3ce0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.903+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/1921421988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 0x5564cc0d37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.903+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/1921421988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0cbac0 0x5564cc0d64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.903+0000 7fb057e00640 0 client.27569 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.903+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1921421988 >> 172.21.15.2:0/1921421988 conn(0x5564cc0c8050 msgr2=0x5564cc0cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.903+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1921421988 shutdown_connections 2024-09-10T08:15:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.904+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1921421988 wait complete. 2024-09-10T08:15:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.904+0000 7fb07b8a9740 20 client.27569 trim_cache size 0 max 16384 2024-09-10T08:15:55.909 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:55 smithi155 ceph-mon[35405]: pgmap v620: 129 pgs: 129 active+clean; 3.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:15:55.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:55 smithi107 ceph-mon[35048]: pgmap v620: 129 pgs: 129 active+clean; 3.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:15:55.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.921+0000 7fb07b8a9740 1 Processor -- start 2024-09-10T08:15:55.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.922+0000 7fb07b8a9740 1 -- start start 2024-09-10T08:15:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.922+0000 7fb07b8a9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 0x5564cc0ee9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.922+0000 7fb07b8a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0cbac0 0x5564cc0eef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.922+0000 7fb07b8a9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 0x5564cc0d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.922+0000 7fb07b8a9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb0500125e0 con 0x5564cc0c8050 2024-09-10T08:15:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.922+0000 7fb07b8a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb05000feb0 con 0x5564cc0cbac0 2024-09-10T08:15:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.922+0000 7fb07b8a9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb05000dd00 con 0x5564cc0cc3e0 2024-09-10T08:15:55.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.922+0000 7fb071e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 0x5564cc0ee9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.922+0000 7fb071e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 0x5564cc0ee9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36764/0 (socket says 172.21.15.2:36764) 2024-09-10T08:15:55.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.922+0000 7fb071e00640 1 -- 172.21.15.2:0/3778085264 learned_addr learned my addr 172.21.15.2:0/3778085264 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:55.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.922+0000 7fb06be00640 1 --2- 172.21.15.2:0/3778085264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0cbac0 0x5564cc0eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.922+0000 7fb072800640 1 --2- 172.21.15.2:0/3778085264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 0x5564cc0d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.923+0000 7fb071e00640 1 -- 172.21.15.2:0/3778085264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 msgr2=0x5564cc0d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:55.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.923+0000 7fb071e00640 1 --2- 172.21.15.2:0/3778085264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 0x5564cc0d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.923+0000 7fb071e00640 1 -- 172.21.15.2:0/3778085264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0cbac0 msgr2=0x5564cc0eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:55.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.923+0000 7fb071e00640 1 --2- 172.21.15.2:0/3778085264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0cbac0 0x5564cc0eef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.923+0000 7fb071e00640 1 -- 172.21.15.2:0/3778085264 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564cc0c7e80 con 0x5564cc0c8050 2024-09-10T08:15:55.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.923+0000 7fb071e00640 1 --2- 172.21.15.2:0/3778085264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 0x5564cc0ee9f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb06400c6a0 tx=0x7fb0640cc830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483114727ba7d13f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:55.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.923+0000 7fb062a00640 1 -- 172.21.15.2:0/3778085264 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06409b200 con 0x5564cc0c8050 2024-09-10T08:15:55.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.923+0000 7fb062a00640 1 -- 172.21.15.2:0/3778085264 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb06409bdc0 con 0x5564cc0c8050 2024-09-10T08:15:55.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.923+0000 7fb062a00640 1 -- 172.21.15.2:0/3778085264 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06409b020 con 0x5564cc0c8050 2024-09-10T08:15:55.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.924+0000 7fb07b8a9740 1 -- 172.21.15.2:0/3778085264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 msgr2=0x5564cc0ee9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:55.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.924+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/3778085264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 0x5564cc0ee9f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb06400c6a0 tx=0x7fb0640cc830 comp rx=0 tx=0).stop 2024-09-10T08:15:55.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.924+0000 7fb07b8a9740 1 -- 172.21.15.2:0/3778085264 shutdown_connections 2024-09-10T08:15:55.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.924+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/3778085264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 0x5564cc0d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.924+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/3778085264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0cbac0 0x5564cc0eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.924+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/3778085264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 0x5564cc0ee9f0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.924+0000 7fb07b8a9740 1 -- 172.21.15.2:0/3778085264 >> 172.21.15.2:0/3778085264 conn(0x5564cc0d3e40 msgr2=0x5564cc0d5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:55.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.924+0000 7fb07b8a9740 1 -- 172.21.15.2:0/3778085264 shutdown_connections 2024-09-10T08:15:55.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.924+0000 7fb07b8a9740 1 -- 172.21.15.2:0/3778085264 wait complete. 2024-09-10T08:15:55.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.924+0000 7fb07b8a9740 1 -- start start 2024-09-10T08:15:55.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.925+0000 7fb07b8a9740 1 Processor -- start 2024-09-10T08:15:55.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.925+0000 7fb07b8a9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 0x5564cc0df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.925+0000 7fb07b8a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0cbac0 0x5564cc0df650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.925+0000 7fb07b8a9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 0x5564cc0dd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.925+0000 7fb07b8a9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb05000e370 con 0x5564cc0c8050 2024-09-10T08:15:55.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.925+0000 7fb07b8a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb05000f7e0 con 0x5564cc0cbac0 2024-09-10T08:15:55.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.925+0000 7fb07b8a9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb05000ea40 con 0x5564cc0cc3e0 2024-09-10T08:15:55.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.925+0000 7fb07b8a9740 10 client.0 _ll_register_callbacks cb 0x7fb06c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:55.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.925+0000 7fb071e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 0x5564cc0df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.925+0000 7fb071e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 0x5564cc0df110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36774/0 (socket says 172.21.15.2:36774) 2024-09-10T08:15:55.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.925+0000 7fb071e00640 1 -- 172.21.15.2:0/1923694206 learned_addr learned my addr 172.21.15.2:0/1923694206 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:55.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.925+0000 7fb072800640 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 0x5564cc0dd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.926+0000 7fb06be00640 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0cbac0 0x5564cc0df650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.926+0000 7fb071e00640 1 -- 172.21.15.2:0/1923694206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 msgr2=0x5564cc0dd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:55.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.926+0000 7fb071e00640 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 0x5564cc0dd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.926+0000 7fb071e00640 1 -- 172.21.15.2:0/1923694206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0cbac0 msgr2=0x5564cc0df650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:55.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.926+0000 7fb071e00640 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0cbac0 0x5564cc0df650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.926+0000 7fb071e00640 1 -- 172.21.15.2:0/1923694206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564cc0ddec0 con 0x5564cc0c8050 2024-09-10T08:15:55.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.926+0000 7fb06be00640 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0cbac0 0x5564cc0df650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.926+0000 7fb071e00640 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 0x5564cc0df110 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb0640d2a00 tx=0x7fb0640d2ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa30b4561e69209 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.926+0000 7fb057e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.926+0000 7fb057e00640 1 -- 172.21.15.2:0/1923694206 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0640b4060 con 0x5564cc0c8050 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.926+0000 7fb057e00640 1 -- 172.21.15.2:0/1923694206 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb064004cf0 con 0x5564cc0c8050 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.926+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1923694206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb06409bdc0 con 0x5564cc0c8050 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.927+0000 7fb057e00640 1 -- 172.21.15.2:0/1923694206 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb06409b200 con 0x5564cc0c8050 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.927+0000 7fb07b8a9740 10 client.17538 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.927+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1923694206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5564cc152710 con 0x5564cc0c8050 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.927+0000 7fb057e00640 1 -- 172.21.15.2:0/1923694206 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb06409bdc0 con 0x5564cc0c8050 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.927+0000 7fb057e00640 1 -- 172.21.15.2:0/1923694206 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb064097d30 con 0x5564cc0c8050 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.927+0000 7fb07b8a9740 10 client.17538 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.927+0000 7fb07b8a9740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.927+0000 7fb07b8a9740 20 client.17538 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:55.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.927+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5564cc0d6ca0 0x5564cc154db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:55.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.927+0000 7fb07b8a9740 4 client.17538 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:15:55.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.927+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1923694206 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5564cc039d10 con 0x5564cc0d6ca0 2024-09-10T08:15:55.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.928+0000 7fb06be00640 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5564cc0d6ca0 0x5564cc154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:55.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.928+0000 7fb06be00640 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5564cc0d6ca0 0x5564cc154db0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6089684b381831f2 server_cookie=6e0cf34ff041fe41 in_seq=0 out_seq=0 2024-09-10T08:15:55.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.928+0000 7fb057e00640 10 client.17538 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:55.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.929+0000 7fb057e00640 1 -- 172.21.15.2:0/1923694206 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5564cc039d10 con 0x5564cc0d6ca0 2024-09-10T08:15:55.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.929+0000 7fb057e00640 10 client.17538 handle_command_reply: tid=0 2024-09-10T08:15:55.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.929+0000 7fb07b8a9740 1 client.17538 shutdown 2024-09-10T08:15:55.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.929+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1923694206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 msgr2=0x5564cc0df110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:55.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.929+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 0x5564cc0df110 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb0640d2a00 tx=0x7fb0640d2ac0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.929+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1923694206 shutdown_connections 2024-09-10T08:15:55.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.929+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5564cc0d6ca0 0x5564cc154db0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.930+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564cc0cc3e0 0x5564cc0dd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.930+0000 7fb057e00640 0 client.17538 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:55.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.930+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5564cc0cbac0 0x5564cc0df650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.930+0000 7fb07b8a9740 1 --2- 172.21.15.2:0/1923694206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5564cc0c8050 0x5564cc0df110 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.930+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1923694206 >> 172.21.15.2:0/1923694206 conn(0x5564cc0d3e40 msgr2=0x5564cc0d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:55.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.930+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1923694206 shutdown_connections 2024-09-10T08:15:55.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.930+0000 7fb07b8a9740 1 -- 172.21.15.2:0/1923694206 wait complete. 2024-09-10T08:15:55.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.930+0000 7fb07b8a9740 20 client.17538 trim_cache size 0 max 16384 2024-09-10T08:15:55.935 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:15:55.935 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:15:55.935 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "dbeebd8b-f2cb-4d50-8242-cf36223c1a87", 2024-09-10T08:15:55.935 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:15:55.935 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:15:55.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.931+0000 7fb078e00640 1 -- 172.21.15.2:0/3768342275 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb0580a2c00 msgr2=0x7fb0580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:55.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.931+0000 7fb078e00640 1 --2- 172.21.15.2:0/3768342275 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb0580a2c00 0x7fb0580a50c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb06c004af0 tx=0x7fb06c004a80 comp rx=0 tx=0).stop 2024-09-10T08:15:55.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.931+0000 7fb078e00640 1 -- 172.21.15.2:0/3768342275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb074104010 msgr2=0x7fb07419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:55.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.931+0000 7fb078e00640 1 --2- 172.21.15.2:0/3768342275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb074104010 0x7fb07419a610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb06400e9f0 tx=0x7fb06400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.931+0000 7fb078e00640 1 -- 172.21.15.2:0/3768342275 shutdown_connections 2024-09-10T08:15:55.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.931+0000 7fb078e00640 1 --2- 172.21.15.2:0/3768342275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074109140 0x7fb07419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.931+0000 7fb078e00640 1 --2- 172.21.15.2:0/3768342275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0741049c0 0x7fb07419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.931+0000 7fb078e00640 1 --2- 172.21.15.2:0/3768342275 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb0580a2c00 0x7fb0580a50c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.931+0000 7fb078e00640 1 --2- 172.21.15.2:0/3768342275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb074104010 0x7fb07419a610 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:55.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.931+0000 7fb078e00640 1 -- 172.21.15.2:0/3768342275 >> 172.21.15.2:0/3768342275 conn(0x7fb0740ffa50 msgr2=0x7fb074100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:55.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.931+0000 7fb078e00640 1 -- 172.21.15.2:0/3768342275 shutdown_connections 2024-09-10T08:15:55.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:55.931+0000 7fb078e00640 1 -- 172.21.15.2:0/3768342275 wait complete. 2024-09-10T08:15:56.920 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:15:57.299 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:15:57.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:57 smithi002 ceph-mon[30563]: pgmap v621: 129 pgs: 129 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:15:57.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:57 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:15:57.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:57 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:15:57.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:57 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:15:57.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:57 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:15:57.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.863+0000 7ffa2e400640 1 -- 172.21.15.2:0/2795131678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa28077040 msgr2=0x7ffa281124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:57.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.863+0000 7ffa2e400640 1 --2- 172.21.15.2:0/2795131678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa28077040 0x7ffa281124f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009900 tx=0x7ffa1402a410 comp rx=0 tx=0).stop 2024-09-10T08:15:57.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.863+0000 7ffa2e400640 1 -- 172.21.15.2:0/2795131678 shutdown_connections 2024-09-10T08:15:57.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.863+0000 7ffa2e400640 1 --2- 172.21.15.2:0/2795131678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa28112a30 0x7ffa28114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.864+0000 7ffa2e400640 1 --2- 172.21.15.2:0/2795131678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa28077040 0x7ffa281124f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.864+0000 7ffa2e400640 1 --2- 172.21.15.2:0/2795131678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa28076720 0x7ffa28076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.864+0000 7ffa2e400640 1 -- 172.21.15.2:0/2795131678 >> 172.21.15.2:0/2795131678 conn(0x7ffa280ffa90 msgr2=0x7ffa28101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:57.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.864+0000 7ffa2e400640 1 -- 172.21.15.2:0/2795131678 shutdown_connections 2024-09-10T08:15:57.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.864+0000 7ffa2e400640 1 -- 172.21.15.2:0/2795131678 wait complete. 2024-09-10T08:15:57.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.864+0000 7ffa2e400640 1 Processor -- start 2024-09-10T08:15:57.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.865+0000 7ffa2e400640 1 -- start start 2024-09-10T08:15:57.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.865+0000 7ffa2e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa28076720 0x7ffa281a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.865+0000 7ffa2e400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa28077040 0x7ffa281a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.865+0000 7ffa2e400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa28112a30 0x7ffa2819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.865+0000 7ffa2e400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281185f0 con 0x7ffa28112a30 2024-09-10T08:15:57.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.865+0000 7ffa2e400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa28118470 con 0x7ffa28076720 2024-09-10T08:15:57.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.865+0000 7ffa2e400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa28118770 con 0x7ffa28077040 2024-09-10T08:15:57.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.865+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa28112a30 0x7ffa2819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.865+0000 7ffa26a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa28077040 0x7ffa281a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.865+0000 7ffa26a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa28077040 0x7ffa281a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:39122/0 (socket says 172.21.15.2:39122) 2024-09-10T08:15:57.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.865+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa28112a30 0x7ffa2819e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47000/0 (socket says 172.21.15.2:47000) 2024-09-10T08:15:57.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.865+0000 7ffa26a00640 1 -- 172.21.15.2:0/1334710960 learned_addr learned my addr 172.21.15.2:0/1334710960 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:57.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.865+0000 7ffa27400640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa28076720 0x7ffa281a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.866+0000 7ffa27e00640 1 -- 172.21.15.2:0/1334710960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa28077040 msgr2=0x7ffa281a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:57.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.866+0000 7ffa27e00640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa28077040 0x7ffa281a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.866+0000 7ffa27e00640 1 -- 172.21.15.2:0/1334710960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa28076720 msgr2=0x7ffa281a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:57.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.866+0000 7ffa27e00640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa28076720 0x7ffa281a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.866+0000 7ffa27e00640 1 -- 172.21.15.2:0/1334710960 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2819efd0 con 0x7ffa28112a30 2024-09-10T08:15:57.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.866+0000 7ffa26a00640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa28077040 0x7ffa281a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:57.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.866+0000 7ffa27e00640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa28112a30 0x7ffa2819e8a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ffa2807aa10 tx=0x7ffa2000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:57.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.866+0000 7ffa1fe00640 1 -- 172.21.15.2:0/1334710960 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2000cdf0 con 0x7ffa28112a30 2024-09-10T08:15:57.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.866+0000 7ffa1fe00640 1 -- 172.21.15.2:0/1334710960 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa20005020 con 0x7ffa28112a30 2024-09-10T08:15:57.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.866+0000 7ffa1fe00640 1 -- 172.21.15.2:0/1334710960 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20010670 con 0x7ffa28112a30 2024-09-10T08:15:57.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.866+0000 7ffa2e400640 1 -- 172.21.15.2:0/1334710960 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2819f2c0 con 0x7ffa28112a30 2024-09-10T08:15:57.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.867+0000 7ffa2e400640 1 -- 172.21.15.2:0/1334710960 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa281ab5b0 con 0x7ffa28112a30 2024-09-10T08:15:57.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.870+0000 7ffa1fe00640 1 -- 172.21.15.2:0/1334710960 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ffa200054d0 con 0x7ffa28112a30 2024-09-10T08:15:57.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.870+0000 7ffa30e3f740 1 Processor -- start 2024-09-10T08:15:57.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.870+0000 7ffa30e3f740 1 -- start start 2024-09-10T08:15:57.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa30e3f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee83f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa30e3f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee844b0 0x561e2ee8e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa30e3f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee8ed70 0x561e2ee91160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa30e3f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561e2edea3e0 con 0x561e2ee83b90 2024-09-10T08:15:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa30e3f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561e2ed91d80 con 0x561e2ee8ed70 2024-09-10T08:15:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa30e3f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561e2ede63a0 con 0x561e2ee844b0 2024-09-10T08:15:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa26a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee83f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa27400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee844b0 0x561e2ee8e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa27400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee844b0 0x561e2ee8e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:39136/0 (socket says 172.21.15.2:39136) 2024-09-10T08:15:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa27400640 1 -- 172.21.15.2:0/3843315581 learned_addr learned my addr 172.21.15.2:0/3843315581 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa26a00640 1 --2- 172.21.15.2:0/3843315581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee8ed70 0x561e2ee91160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa27400640 1 -- 172.21.15.2:0/3843315581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee8ed70 msgr2=0x561e2ee91160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa27400640 1 --2- 172.21.15.2:0/3843315581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee8ed70 0x561e2ee91160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa27400640 1 -- 172.21.15.2:0/3843315581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 msgr2=0x561e2ee83f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa27400640 1 --2- 172.21.15.2:0/3843315581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee83f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.871+0000 7ffa27400640 1 -- 172.21.15.2:0/3843315581 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e2edb0ae0 con 0x561e2ee844b0 2024-09-10T08:15:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.872+0000 7ffa27400640 1 --2- 172.21.15.2:0/3843315581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee844b0 0x561e2ee8e830 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009c90 tx=0x7ffa18011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b1a996ed2353913 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.872+0000 7ffa1fe00640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa0c0a2b80 0x7ffa0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.872+0000 7ffa1cc00640 1 -- 172.21.15.2:0/3843315581 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18005220 con 0x561e2ee844b0 2024-09-10T08:15:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.872+0000 7ffa1cc00640 1 -- 172.21.15.2:0/3843315581 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa18004c80 con 0x561e2ee844b0 2024-09-10T08:15:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.872+0000 7ffa1cc00640 1 -- 172.21.15.2:0/3843315581 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18019830 con 0x561e2ee844b0 2024-09-10T08:15:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.872+0000 7ffa1fe00640 1 -- 172.21.15.2:0/1334710960 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ffa200cfaf0 con 0x7ffa28112a30 2024-09-10T08:15:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.872+0000 7ffa26a00640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa0c0a2b80 0x7ffa0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.873+0000 7ffa30e3f740 1 -- 172.21.15.2:0/3843315581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee844b0 msgr2=0x561e2ee8e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.873+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/3843315581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee844b0 0x561e2ee8e830 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009c90 tx=0x7ffa18011810 comp rx=0 tx=0).stop 2024-09-10T08:15:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.873+0000 7ffa26a00640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa0c0a2b80 0x7ffa0c0a5040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ffa28079400 tx=0x7ffa14041040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.873+0000 7ffa30e3f740 1 -- 172.21.15.2:0/3843315581 shutdown_connections 2024-09-10T08:15:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.873+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/3843315581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee8ed70 0x561e2ee91160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.873+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/3843315581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee844b0 0x561e2ee8e830 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.873+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/3843315581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee83f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.873+0000 7ffa30e3f740 1 -- 172.21.15.2:0/3843315581 >> 172.21.15.2:0/3843315581 conn(0x561e2ee80120 msgr2=0x561e2ee80500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.873+0000 7ffa30e3f740 1 -- 172.21.15.2:0/3843315581 shutdown_connections 2024-09-10T08:15:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.873+0000 7ffa30e3f740 1 -- 172.21.15.2:0/3843315581 wait complete. 2024-09-10T08:15:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.874+0000 7ffa30e3f740 1 -- start start 2024-09-10T08:15:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.874+0000 7ffa30e3f740 1 Processor -- start 2024-09-10T08:15:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.874+0000 7ffa30e3f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee8bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.874+0000 7ffa30e3f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee844b0 0x561e2ee8c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.874+0000 7ffa30e3f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee8ed70 0x561e2ee8c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.874+0000 7ffa30e3f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561e2ee919e0 con 0x561e2ee83b90 2024-09-10T08:15:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.874+0000 7ffa30e3f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561e2eda50f0 con 0x561e2ee844b0 2024-09-10T08:15:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.874+0000 7ffa30e3f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561e2ee91b60 con 0x561e2ee8ed70 2024-09-10T08:15:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.874+0000 7ffa30e3f740 10 client.0 _ll_register_callbacks cb 0x561e2ed969e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.875+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee8ed70 0x561e2ee8c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.875+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee8ed70 0x561e2ee8c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:39150/0 (socket says 172.21.15.2:39150) 2024-09-10T08:15:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.875+0000 7ffa27e00640 1 -- 172.21.15.2:0/1314003061 learned_addr learned my addr 172.21.15.2:0/1314003061 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.875+0000 7ffa27400640 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee8bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.875+0000 7ffa27e00640 1 -- 172.21.15.2:0/1314003061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee844b0 msgr2=0x561e2ee8c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.875+0000 7ffa26a00640 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee844b0 0x561e2ee8c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.875+0000 7ffa27e00640 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee844b0 0x561e2ee8c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.875+0000 7ffa27e00640 1 -- 172.21.15.2:0/1314003061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 msgr2=0x561e2ee8bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.875+0000 7ffa27e00640 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee8bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.875+0000 7ffa27e00640 1 -- 172.21.15.2:0/1314003061 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e2eb5b3c0 con 0x561e2ee8ed70 2024-09-10T08:15:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.875+0000 7ffa27400640 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee8bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.875+0000 7ffa26a00640 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee844b0 0x561e2ee8c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:15:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.876+0000 7ffa27e00640 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee8ed70 0x561e2ee8c6e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffa200a4640 tx=0x7ffa2009c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bf664fecae9240d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.876+0000 7ffa11600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:15:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.876+0000 7ffa11600640 1 -- 172.21.15.2:0/1314003061 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa200b4020 con 0x561e2ee8ed70 2024-09-10T08:15:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.876+0000 7ffa11600640 1 -- 172.21.15.2:0/1314003061 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ffa200a46e0 con 0x561e2ee8ed70 2024-09-10T08:15:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.876+0000 7ffa30e3f740 1 -- 172.21.15.2:0/1314003061 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa18004c80 con 0x561e2ee8ed70 2024-09-10T08:15:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.876+0000 7ffa11600640 1 -- 172.21.15.2:0/1314003061 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffa200a4b70 con 0x561e2ee8ed70 2024-09-10T08:15:57.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.876+0000 7ffa30e3f740 10 client.27523 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:57.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.876+0000 7ffa30e3f740 1 -- 172.21.15.2:0/1314003061 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561e2ee9d6f0 con 0x561e2ee8ed70 2024-09-10T08:15:57.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.876+0000 7ffa11600640 1 -- 172.21.15.2:0/1314003061 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa18004c80 con 0x561e2ee8ed70 2024-09-10T08:15:57.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.877+0000 7ffa11600640 1 -- 172.21.15.2:0/1314003061 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ffa20081050 con 0x561e2ee8ed70 2024-09-10T08:15:57.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.877+0000 7ffa30e3f740 10 client.27523 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:57.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.877+0000 7ffa30e3f740 10 client.27523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:57.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.877+0000 7ffa30e3f740 20 client.27523 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:57.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.877+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561e2ee9eb00 0x561e2eea0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.877+0000 7ffa30e3f740 4 client.27523 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:57.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.877+0000 7ffa30e3f740 1 -- 172.21.15.2:0/1314003061 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e2edf7950 con 0x561e2ee9eb00 2024-09-10T08:15:57.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.877+0000 7ffa27400640 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561e2ee9eb00 0x561e2eea0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.878+0000 7ffa27400640 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561e2ee9eb00 0x561e2eea0f10 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60cedee0d7daf9f0 server_cookie=ef96d0856e505efc in_seq=0 out_seq=0 2024-09-10T08:15:57.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.878+0000 7ffa11600640 10 client.27523 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:57.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.878+0000 7ffa11600640 1 -- 172.21.15.2:0/1314003061 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e2edf7950 con 0x561e2ee9eb00 2024-09-10T08:15:57.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.878+0000 7ffa11600640 10 client.27523 handle_command_reply: tid=0 2024-09-10T08:15:57.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.878+0000 7ffa30e3f740 1 client.27523 shutdown 2024-09-10T08:15:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.879+0000 7ffa30e3f740 1 -- 172.21.15.2:0/1314003061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee8ed70 msgr2=0x561e2ee8c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.879+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee8ed70 0x561e2ee8c6e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffa200a4640 tx=0x7ffa2009c6c0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.879+0000 7ffa30e3f740 1 -- 172.21.15.2:0/1314003061 shutdown_connections 2024-09-10T08:15:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.879+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561e2ee9eb00 0x561e2eea0f10 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.879+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee8ed70 0x561e2ee8c6e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.879+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee844b0 0x561e2ee8c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.879+0000 7ffa11600640 0 client.27523 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.879+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/1314003061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee8bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.879+0000 7ffa30e3f740 1 -- 172.21.15.2:0/1314003061 >> 172.21.15.2:0/1314003061 conn(0x561e2ee80120 msgr2=0x561e2ee85e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.879+0000 7ffa30e3f740 1 -- 172.21.15.2:0/1314003061 shutdown_connections 2024-09-10T08:15:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.879+0000 7ffa30e3f740 1 -- 172.21.15.2:0/1314003061 wait complete. 2024-09-10T08:15:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.879+0000 7ffa30e3f740 20 client.27523 trim_cache size 0 max 16384 2024-09-10T08:15:57.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.898+0000 7ffa30e3f740 1 Processor -- start 2024-09-10T08:15:57.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.898+0000 7ffa30e3f740 1 -- start start 2024-09-10T08:15:57.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.898+0000 7ffa30e3f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee83b90 0x561e2eea67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.898+0000 7ffa30e3f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee844b0 0x561e2eea6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.898+0000 7ffa30e3f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee8ed70 0x561e2ee88ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.898+0000 7ffa30e3f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b7d40 con 0x561e2ee8ed70 2024-09-10T08:15:57.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.898+0000 7ffa30e3f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b5610 con 0x561e2ee83b90 2024-09-10T08:15:57.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.898+0000 7ffa30e3f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b3460 con 0x561e2ee844b0 2024-09-10T08:15:57.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.898+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee8ed70 0x561e2ee88ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.898+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee8ed70 0x561e2ee88ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47036/0 (socket says 172.21.15.2:47036) 2024-09-10T08:15:57.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.898+0000 7ffa27e00640 1 -- 172.21.15.2:0/1634136833 learned_addr learned my addr 172.21.15.2:0/1634136833 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:57.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.898+0000 7ffa26a00640 1 --2- 172.21.15.2:0/1634136833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee844b0 0x561e2eea6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.898+0000 7ffa27400640 1 --2- 172.21.15.2:0/1634136833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee83b90 0x561e2eea67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.899+0000 7ffa27e00640 1 -- 172.21.15.2:0/1634136833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee844b0 msgr2=0x561e2eea6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:57.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.899+0000 7ffa27e00640 1 --2- 172.21.15.2:0/1634136833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee844b0 0x561e2eea6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.899+0000 7ffa27e00640 1 -- 172.21.15.2:0/1634136833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee83b90 msgr2=0x561e2eea67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:57.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.899+0000 7ffa27e00640 1 --2- 172.21.15.2:0/1634136833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee83b90 0x561e2eea67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.899+0000 7ffa27e00640 1 -- 172.21.15.2:0/1634136833 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e2ee7ff50 con 0x561e2ee8ed70 2024-09-10T08:15:57.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.899+0000 7ffa27e00640 1 --2- 172.21.15.2:0/1634136833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee8ed70 0x561e2ee88ee0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ffa20009fd0 tx=0x7ffa2009b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2b70f39f5c438d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:57.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.899+0000 7ffa13e00640 1 -- 172.21.15.2:0/1634136833 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa200b4070 con 0x561e2ee8ed70 2024-09-10T08:15:57.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.900+0000 7ffa13e00640 1 -- 172.21.15.2:0/1634136833 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa2009b120 con 0x561e2ee8ed70 2024-09-10T08:15:57.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.900+0000 7ffa13e00640 1 -- 172.21.15.2:0/1634136833 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa200a4940 con 0x561e2ee8ed70 2024-09-10T08:15:57.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.900+0000 7ffa30e3f740 1 -- 172.21.15.2:0/1634136833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee8ed70 msgr2=0x561e2ee88ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:57.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.900+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/1634136833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee8ed70 0x561e2ee88ee0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ffa20009fd0 tx=0x7ffa2009b780 comp rx=0 tx=0).stop 2024-09-10T08:15:57.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.900+0000 7ffa30e3f740 1 -- 172.21.15.2:0/1634136833 shutdown_connections 2024-09-10T08:15:57.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.900+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/1634136833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee8ed70 0x561e2ee88ee0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.900+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/1634136833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee844b0 0x561e2eea6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.900+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/1634136833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee83b90 0x561e2eea67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.900+0000 7ffa30e3f740 1 -- 172.21.15.2:0/1634136833 >> 172.21.15.2:0/1634136833 conn(0x561e2ee80120 msgr2=0x561e2ee8a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:57.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.901+0000 7ffa30e3f740 1 -- 172.21.15.2:0/1634136833 shutdown_connections 2024-09-10T08:15:57.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.901+0000 7ffa30e3f740 1 -- 172.21.15.2:0/1634136833 wait complete. 2024-09-10T08:15:57.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.901+0000 7ffa30e3f740 1 -- start start 2024-09-10T08:15:57.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.901+0000 7ffa30e3f740 1 Processor -- start 2024-09-10T08:15:57.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.902+0000 7ffa30e3f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee96db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.902+0000 7ffa30e3f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee844b0 0x561e2ee972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.902+0000 7ffa30e3f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee8ed70 0x561e2ee93d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.902+0000 7ffa30e3f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b3ad0 con 0x561e2ee83b90 2024-09-10T08:15:57.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.902+0000 7ffa30e3f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b4f40 con 0x561e2ee844b0 2024-09-10T08:15:57.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.902+0000 7ffa30e3f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b41a0 con 0x561e2ee8ed70 2024-09-10T08:15:57.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.902+0000 7ffa30e3f740 10 client.0 _ll_register_callbacks cb 0x7ffa2009bcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:57.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.902+0000 7ffa27400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee96db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.902+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee8ed70 0x561e2ee93d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.902+0000 7ffa27400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee96db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47042/0 (socket says 172.21.15.2:47042) 2024-09-10T08:15:57.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.902+0000 7ffa27400640 1 -- 172.21.15.2:0/3503107821 learned_addr learned my addr 172.21.15.2:0/3503107821 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:57.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.902+0000 7ffa26a00640 1 --2- 172.21.15.2:0/3503107821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee844b0 0x561e2ee972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.903+0000 7ffa27e00640 1 -- 172.21.15.2:0/3503107821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee844b0 msgr2=0x561e2ee972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:57.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.903+0000 7ffa27e00640 1 --2- 172.21.15.2:0/3503107821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee844b0 0x561e2ee972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.903+0000 7ffa27e00640 1 -- 172.21.15.2:0/3503107821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 msgr2=0x561e2ee96db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:57.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.903+0000 7ffa27e00640 1 --2- 172.21.15.2:0/3503107821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee96db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.903+0000 7ffa27e00640 1 -- 172.21.15.2:0/3503107821 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e2ee97960 con 0x561e2ee8ed70 2024-09-10T08:15:57.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.903+0000 7ffa27400640 1 --2- 172.21.15.2:0/3503107821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee96db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:57.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.903+0000 7ffa27e00640 1 --2- 172.21.15.2:0/3503107821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee8ed70 0x561e2ee93d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa200988d0 tx=0x7ffa2009b350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff0b9af729ab49cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:57.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.903+0000 7ffa11600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:15:57.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.903+0000 7ffa11600640 1 -- 172.21.15.2:0/3503107821 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa200b4030 con 0x561e2ee8ed70 2024-09-10T08:15:57.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.903+0000 7ffa11600640 1 -- 172.21.15.2:0/3503107821 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ffa200a48e0 con 0x561e2ee8ed70 2024-09-10T08:15:57.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.903+0000 7ffa30e3f740 1 -- 172.21.15.2:0/3503107821 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa2009b120 con 0x561e2ee8ed70 2024-09-10T08:15:57.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.904+0000 7ffa11600640 1 -- 172.21.15.2:0/3503107821 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffa200ae050 con 0x561e2ee8ed70 2024-09-10T08:15:57.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.904+0000 7ffa30e3f740 10 client.27529 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:57.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.904+0000 7ffa30e3f740 1 -- 172.21.15.2:0/3503107821 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561e2ee97960 con 0x561e2ee8ed70 2024-09-10T08:15:57.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.904+0000 7ffa11600640 1 -- 172.21.15.2:0/3503107821 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa2009b120 con 0x561e2ee8ed70 2024-09-10T08:15:57.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.904+0000 7ffa11600640 1 -- 172.21.15.2:0/3503107821 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ffa20081070 con 0x561e2ee8ed70 2024-09-10T08:15:57.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.904+0000 7ffa30e3f740 10 client.27529 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:57.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.904+0000 7ffa30e3f740 10 client.27529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:57.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.904+0000 7ffa30e3f740 20 client.27529 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:57.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.904+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/3503107821 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561e2ef0a040 0x561e2ef0c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.904+0000 7ffa30e3f740 4 client.27529 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:15:57.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.904+0000 7ffa30e3f740 1 -- 172.21.15.2:0/3503107821 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e2edf1e60 con 0x561e2ef0a040 2024-09-10T08:15:57.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.905+0000 7ffa27400640 1 --2- 172.21.15.2:0/3503107821 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561e2ef0a040 0x561e2ef0c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.905+0000 7ffa27400640 1 --2- 172.21.15.2:0/3503107821 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561e2ef0a040 0x561e2ef0c430 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd6711631e1065 server_cookie=935a135f2ff59926 in_seq=0 out_seq=0 2024-09-10T08:15:57.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.905+0000 7ffa11600640 10 client.27529 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:57.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.905+0000 7ffa11600640 1 -- 172.21.15.2:0/3503107821 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561e2edf1e60 con 0x561e2ef0a040 2024-09-10T08:15:57.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.905+0000 7ffa11600640 10 client.27529 handle_command_reply: tid=0 2024-09-10T08:15:57.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.905+0000 7ffa30e3f740 1 client.27529 shutdown 2024-09-10T08:15:57.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.906+0000 7ffa30e3f740 1 -- 172.21.15.2:0/3503107821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee8ed70 msgr2=0x561e2ee93d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:57.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.906+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/3503107821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee8ed70 0x561e2ee93d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa200988d0 tx=0x7ffa2009b350 comp rx=0 tx=0).stop 2024-09-10T08:15:57.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.906+0000 7ffa30e3f740 1 -- 172.21.15.2:0/3503107821 shutdown_connections 2024-09-10T08:15:57.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.906+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/3503107821 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561e2ef0a040 0x561e2ef0c430 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.906+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/3503107821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e2ee8ed70 0x561e2ee93d80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.906+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/3503107821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2ee844b0 0x561e2ee972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.906+0000 7ffa30e3f740 1 --2- 172.21.15.2:0/3503107821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e2ee83b90 0x561e2ee96db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.906+0000 7ffa11600640 0 client.27529 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:57.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.906+0000 7ffa30e3f740 1 -- 172.21.15.2:0/3503107821 >> 172.21.15.2:0/3503107821 conn(0x561e2ee80120 msgr2=0x561e2ee7e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:57.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.906+0000 7ffa30e3f740 1 -- 172.21.15.2:0/3503107821 shutdown_connections 2024-09-10T08:15:57.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.906+0000 7ffa30e3f740 1 -- 172.21.15.2:0/3503107821 wait complete. 2024-09-10T08:15:57.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.906+0000 7ffa30e3f740 20 client.27529 trim_cache size 0 max 16384 2024-09-10T08:15:57.912 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:15:57.912 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:15:57.912 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:15:57.912 INFO:teuthology.orchestra.run.smithi002.stdout: "dbeebd8b-f2cb-4d50-8242-cf36223c1a87": { 2024-09-10T08:15:57.912 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:15:57.913 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "dbeebd8b-f2cb-4d50-8242-cf36223c1a87", 2024-09-10T08:15:57.913 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:15:57.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:15:57.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:15:57.913 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:15:57.913 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:57 smithi155 ceph-mon[35405]: pgmap v621: 129 pgs: 129 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:15:57.913 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:57 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:15:57.914 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:57 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:15:57.914 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:57 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:15:57.914 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:57 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:15:57.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.907+0000 7ffa2e400640 1 -- 172.21.15.2:0/1334710960 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa0c0a2b80 msgr2=0x7ffa0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:57.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.907+0000 7ffa2e400640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa0c0a2b80 0x7ffa0c0a5040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ffa28079400 tx=0x7ffa14041040 comp rx=0 tx=0).stop 2024-09-10T08:15:57.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.907+0000 7ffa2e400640 1 -- 172.21.15.2:0/1334710960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa28112a30 msgr2=0x7ffa2819e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:57.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.907+0000 7ffa2e400640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa28112a30 0x7ffa2819e8a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ffa2807aa10 tx=0x7ffa2000ef10 comp rx=0 tx=0).stop 2024-09-10T08:15:57.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.907+0000 7ffa2e400640 1 -- 172.21.15.2:0/1334710960 shutdown_connections 2024-09-10T08:15:57.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.907+0000 7ffa2e400640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa28112a30 0x7ffa2819e8a0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.907+0000 7ffa2e400640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa0c0a2b80 0x7ffa0c0a5040 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.907+0000 7ffa2e400640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa28077040 0x7ffa281a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.907+0000 7ffa2e400640 1 --2- 172.21.15.2:0/1334710960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa28076720 0x7ffa281a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.907+0000 7ffa2e400640 1 -- 172.21.15.2:0/1334710960 >> 172.21.15.2:0/1334710960 conn(0x7ffa280ffa90 msgr2=0x7ffa28101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:57.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.907+0000 7ffa2e400640 1 -- 172.21.15.2:0/1334710960 shutdown_connections 2024-09-10T08:15:57.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:15:57.907+0000 7ffa2e400640 1 -- 172.21.15.2:0/1334710960 wait complete. 2024-09-10T08:15:57.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:57 smithi107 ceph-mon[35048]: pgmap v621: 129 pgs: 129 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:15:57.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:57 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:15:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:57 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:15:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:57 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:15:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:57 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:15:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.991+0000 7fc0c1000640 1 -- 172.21.15.2:0/116524690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc103d60 msgr2=0x7fc0bc104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.991+0000 7fc0c1000640 1 --2- 172.21.15.2:0/116524690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc103d60 0x7fc0bc104160 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009900 tx=0x7fc0ac02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.991+0000 7fc0c1000640 1 -- 172.21.15.2:0/116524690 shutdown_connections 2024-09-10T08:15:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.991+0000 7fc0c1000640 1 --2- 172.21.15.2:0/116524690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0bc105910 0x7fc0bc10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.991+0000 7fc0c1000640 1 --2- 172.21.15.2:0/116524690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc104f50 0x7fc0bc1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.991+0000 7fc0c1000640 1 --2- 172.21.15.2:0/116524690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc103d60 0x7fc0bc104160 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.991+0000 7fc0c1000640 1 -- 172.21.15.2:0/116524690 >> 172.21.15.2:0/116524690 conn(0x7fc0bc0ff4e0 msgr2=0x7fc0bc101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.991+0000 7fc0c1000640 1 -- 172.21.15.2:0/116524690 shutdown_connections 2024-09-10T08:15:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.991+0000 7fc0c1000640 1 -- 172.21.15.2:0/116524690 wait complete. 2024-09-10T08:15:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.992+0000 7fc0c1000640 1 Processor -- start 2024-09-10T08:15:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.992+0000 7fc0c1000640 1 -- start start 2024-09-10T08:15:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.992+0000 7fc0c1000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc103d60 0x7fc0bc199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.992+0000 7fc0c1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc104f50 0x7fc0bc19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.992+0000 7fc0c1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0bc105910 0x7fc0bc1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.992+0000 7fc0c1000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc10f9e0 con 0x7fc0bc103d60 2024-09-10T08:15:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.992+0000 7fc0c1000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc10f860 con 0x7fc0bc105910 2024-09-10T08:15:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.992+0000 7fc0c1000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc10fb60 con 0x7fc0bc104f50 2024-09-10T08:15:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.993+0000 7fc0b9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc103d60 0x7fc0bc199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.993+0000 7fc0ba800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0bc105910 0x7fc0bc1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.993+0000 7fc0b9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc103d60 0x7fc0bc199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47064/0 (socket says 172.21.15.2:47064) 2024-09-10T08:15:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.993+0000 7fc0b9e00640 1 -- 172.21.15.2:0/2428113519 learned_addr learned my addr 172.21.15.2:0/2428113519 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.993+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc104f50 0x7fc0bc19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.993+0000 7fc0ba800640 1 -- 172.21.15.2:0/2428113519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc104f50 msgr2=0x7fc0bc19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.993+0000 7fc0ba800640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc104f50 0x7fc0bc19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.993+0000 7fc0ba800640 1 -- 172.21.15.2:0/2428113519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc103d60 msgr2=0x7fc0bc199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.993+0000 7fc0ba800640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc103d60 0x7fc0bc199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.993+0000 7fc0ba800640 1 -- 172.21.15.2:0/2428113519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc1a1bf0 con 0x7fc0bc105910 2024-09-10T08:15:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.993+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc104f50 0x7fc0bc19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.993+0000 7fc0b9e00640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc103d60 0x7fc0bc199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.993+0000 7fc0ba800640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0bc105910 0x7fc0bc1a1580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4009fd0 tx=0x7fc0b400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.994+0000 7fc0b2a00640 1 -- 172.21.15.2:0/2428113519 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b400ce20 con 0x7fc0bc105910 2024-09-10T08:15:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.994+0000 7fc0b2a00640 1 -- 172.21.15.2:0/2428113519 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b4005020 con 0x7fc0bc105910 2024-09-10T08:15:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.994+0000 7fc0c1000640 1 -- 172.21.15.2:0/2428113519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0bc1a1ee0 con 0x7fc0bc105910 2024-09-10T08:15:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.994+0000 7fc0b2a00640 1 -- 172.21.15.2:0/2428113519 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4010720 con 0x7fc0bc105910 2024-09-10T08:15:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.994+0000 7fc0c1000640 1 -- 172.21.15.2:0/2428113519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0bc1a23a0 con 0x7fc0bc105910 2024-09-10T08:15:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.997+0000 7fc0c405c740 1 Processor -- start 2024-09-10T08:15:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.997+0000 7fc0b2a00640 1 -- 172.21.15.2:0/2428113519 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc0b4005540 con 0x7fc0bc105910 2024-09-10T08:15:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.997+0000 7fc0c405c740 1 -- start start 2024-09-10T08:15:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0c405c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c364090 0x55f88c364490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0c405c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c369490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0c405c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 0x55f88c36bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0c405c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f88c2c9100 con 0x55f88c364090 2024-09-10T08:15:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0c405c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f88c2711b0 con 0x55f88c369a60 2024-09-10T08:15:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0c405c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f88c2c50c0 con 0x55f88c3649d0 2024-09-10T08:15:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c364090 0x55f88c364490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c364090 0x55f88c364490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47072/0 (socket says 172.21.15.2:47072) 2024-09-10T08:15:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0b3e00640 1 -- 172.21.15.2:0/3162550502 learned_addr learned my addr 172.21.15.2:0/3162550502 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0b9e00640 1 --2- 172.21.15.2:0/3162550502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c369490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/3162550502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 0x55f88c36bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0b3e00640 1 -- 172.21.15.2:0/3162550502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 msgr2=0x55f88c369490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/3162550502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c369490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0b3e00640 1 -- 172.21.15.2:0/3162550502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 msgr2=0x55f88c36bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/3162550502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 0x55f88c36bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.998+0000 7fc0b3e00640 1 -- 172.21.15.2:0/3162550502 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88bfdc210 con 0x55f88c364090 2024-09-10T08:15:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.999+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/3162550502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c364090 0x55f88c364490 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4013610 tx=0x7fc0a40097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e789807c4c3fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.999+0000 7fc0b2a00640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc0a00a2c30 0x7fc0a00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.999+0000 7fc0ab400640 1 -- 172.21.15.2:0/3162550502 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a400f520 con 0x55f88c364090 2024-09-10T08:15:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.999+0000 7fc0ab400640 1 -- 172.21.15.2:0/3162550502 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0a400fba0 con 0x55f88c364090 2024-09-10T08:15:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.999+0000 7fc0ab400640 1 -- 172.21.15.2:0/3162550502 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4010ad0 con 0x55f88c364090 2024-09-10T08:15:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.999+0000 7fc0b2a00640 1 -- 172.21.15.2:0/2428113519 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc0b40cfb50 con 0x7fc0bc105910 2024-09-10T08:15:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:57.999+0000 7fc0b9e00640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc0a00a2c30 0x7fc0a00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.000+0000 7fc0c405c740 1 -- 172.21.15.2:0/3162550502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c364090 msgr2=0x55f88c364490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.000+0000 7fc0c405c740 1 --2- 172.21.15.2:0/3162550502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c364090 0x55f88c364490 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4013610 tx=0x7fc0a40097d0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.000+0000 7fc0b9e00640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc0a00a2c30 0x7fc0a00a50f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac000c00 tx=0x7fc0ac0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.000+0000 7fc0c405c740 1 -- 172.21.15.2:0/3162550502 shutdown_connections 2024-09-10T08:15:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.000+0000 7fc0c405c740 1 --2- 172.21.15.2:0/3162550502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 0x55f88c36bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.000+0000 7fc0c405c740 1 --2- 172.21.15.2:0/3162550502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c369490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.000+0000 7fc0c405c740 1 --2- 172.21.15.2:0/3162550502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c364090 0x55f88c364490 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.000+0000 7fc0c405c740 1 -- 172.21.15.2:0/3162550502 >> 172.21.15.2:0/3162550502 conn(0x55f88c35f700 msgr2=0x55f88c35fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.000+0000 7fc0c405c740 1 -- 172.21.15.2:0/3162550502 shutdown_connections 2024-09-10T08:15:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.000+0000 7fc0c405c740 1 -- 172.21.15.2:0/3162550502 wait complete. 2024-09-10T08:15:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.001+0000 7fc0c405c740 1 -- start start 2024-09-10T08:15:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.001+0000 7fc0c405c740 1 Processor -- start 2024-09-10T08:15:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.001+0000 7fc0c405c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c364090 0x55f88c36e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.001+0000 7fc0c405c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c36e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.001+0000 7fc0c405c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 0x55f88c3757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.001+0000 7fc0c405c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f88c36ca30 con 0x55f88c364090 2024-09-10T08:15:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.001+0000 7fc0c405c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f88c252090 con 0x55f88c369a60 2024-09-10T08:15:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.001+0000 7fc0c405c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f88c36cbb0 con 0x55f88c3649d0 2024-09-10T08:15:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.001+0000 7fc0c405c740 10 client.0 _ll_register_callbacks cb 0x55f88c2a6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.002+0000 7fc0b3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c36e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.002+0000 7fc0b9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c364090 0x55f88c36e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.002+0000 7fc0b3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c36e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:39196/0 (socket says 172.21.15.2:39196) 2024-09-10T08:15:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.002+0000 7fc0b3e00640 1 -- 172.21.15.2:0/1476812152 learned_addr learned my addr 172.21.15.2:0/1476812152 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.002+0000 7fc0ba800640 1 --2- 172.21.15.2:0/1476812152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 0x55f88c3757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.002+0000 7fc0b3e00640 1 -- 172.21.15.2:0/1476812152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 msgr2=0x55f88c3757f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.002+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/1476812152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 0x55f88c3757f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.002+0000 7fc0b3e00640 1 -- 172.21.15.2:0/1476812152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c364090 msgr2=0x55f88c36e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.002+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/1476812152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c364090 0x55f88c36e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.002+0000 7fc0b3e00640 1 -- 172.21.15.2:0/1476812152 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88c0a8620 con 0x55f88c3649d0 2024-09-10T08:15:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.002+0000 7fc0b9e00640 1 --2- 172.21.15.2:0/1476812152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c364090 0x55f88c36e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.002+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/1476812152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c36e7b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4005760 tx=0x7fc0a4010470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=366e00a586219b78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.002+0000 7fc09fe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:15:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.003+0000 7fc09fe00640 1 -- 172.21.15.2:0/1476812152 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4009530 con 0x55f88c3649d0 2024-09-10T08:15:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.003+0000 7fc0c405c740 1 -- 172.21.15.2:0/1476812152 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0a400fba0 con 0x55f88c3649d0 2024-09-10T08:15:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.003+0000 7fc09fe00640 1 -- 172.21.15.2:0/1476812152 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc0a4013110 con 0x55f88c3649d0 2024-09-10T08:15:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.003+0000 7fc09fe00640 1 -- 172.21.15.2:0/1476812152 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc0a4030050 con 0x55f88c3649d0 2024-09-10T08:15:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.003+0000 7fc0c405c740 10 client.27541 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.003+0000 7fc0c405c740 1 -- 172.21.15.2:0/1476812152 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f88c378630 con 0x55f88c3649d0 2024-09-10T08:15:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.003+0000 7fc09fe00640 1 -- 172.21.15.2:0/1476812152 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0a400fba0 con 0x55f88c3649d0 2024-09-10T08:15:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.003+0000 7fc09fe00640 1 -- 172.21.15.2:0/1476812152 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc0a4010850 con 0x55f88c3649d0 2024-09-10T08:15:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.003+0000 7fc0c405c740 10 client.27541 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.003+0000 7fc0c405c740 10 client.27541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.003+0000 7fc0c405c740 20 client.27541 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.004+0000 7fc0c405c740 1 --2- 172.21.15.2:0/1476812152 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f88c379a90 0x55f88c37be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.004+0000 7fc0c405c740 4 client.27541 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:15:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.004+0000 7fc0c405c740 1 -- 172.21.15.2:0/1476812152 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f88c2d6830 con 0x55f88c379a90 2024-09-10T08:15:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.004+0000 7fc0b9e00640 1 --2- 172.21.15.2:0/1476812152 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f88c379a90 0x55f88c37be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.005+0000 7fc0b9e00640 1 --2- 172.21.15.2:0/1476812152 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f88c379a90 0x55f88c37be80 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab1bf534c4a7cad server_cookie=67211cc8c14e9d7c in_seq=0 out_seq=0 2024-09-10T08:15:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.005+0000 7fc09fe00640 10 client.27541 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.005+0000 7fc09fe00640 1 -- 172.21.15.2:0/1476812152 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f88c2d6830 con 0x55f88c379a90 2024-09-10T08:15:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.005+0000 7fc09fe00640 10 client.27541 handle_command_reply: tid=0 2024-09-10T08:15:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.005+0000 7fc0c405c740 1 client.27541 shutdown 2024-09-10T08:15:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.006+0000 7fc0c405c740 1 -- 172.21.15.2:0/1476812152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 msgr2=0x55f88c36e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.006+0000 7fc0c405c740 1 --2- 172.21.15.2:0/1476812152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c36e7b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4005760 tx=0x7fc0a4010470 comp rx=0 tx=0).stop 2024-09-10T08:15:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.006+0000 7fc0c405c740 1 -- 172.21.15.2:0/1476812152 shutdown_connections 2024-09-10T08:15:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.006+0000 7fc0c405c740 1 --2- 172.21.15.2:0/1476812152 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f88c379a90 0x55f88c37be80 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.006+0000 7fc0c405c740 1 --2- 172.21.15.2:0/1476812152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 0x55f88c3757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.006+0000 7fc09fe00640 0 client.27541 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.006+0000 7fc0c405c740 1 --2- 172.21.15.2:0/1476812152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c36e7b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.006+0000 7fc0c405c740 1 --2- 172.21.15.2:0/1476812152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c364090 0x55f88c36e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.006+0000 7fc0c405c740 1 -- 172.21.15.2:0/1476812152 >> 172.21.15.2:0/1476812152 conn(0x55f88c35f700 msgr2=0x55f88c360f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.006+0000 7fc0c405c740 1 -- 172.21.15.2:0/1476812152 shutdown_connections 2024-09-10T08:15:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.006+0000 7fc0c405c740 1 -- 172.21.15.2:0/1476812152 wait complete. 2024-09-10T08:15:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.006+0000 7fc0c405c740 20 client.27541 trim_cache size 0 max 16384 2024-09-10T08:15:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.021+0000 7fc0c405c740 1 Processor -- start 2024-09-10T08:15:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.021+0000 7fc0c405c740 1 -- start start 2024-09-10T08:15:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.021+0000 7fc0c405c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c364090 0x55f88c35fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.021+0000 7fc0c405c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c3757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.021+0000 7fc0c405c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c369a60 0x55f88c360020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.021+0000 7fc0c405c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1af390 con 0x55f88c369a60 2024-09-10T08:15:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.021+0000 7fc0c405c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1acc60 con 0x55f88c364090 2024-09-10T08:15:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.021+0000 7fc0c405c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1aaab0 con 0x55f88c3649d0 2024-09-10T08:15:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.022+0000 7fc0ba800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c369a60 0x55f88c360020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.022+0000 7fc0ba800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c369a60 0x55f88c360020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47090/0 (socket says 172.21.15.2:47090) 2024-09-10T08:15:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.022+0000 7fc0ba800640 1 -- 172.21.15.2:0/762412259 learned_addr learned my addr 172.21.15.2:0/762412259 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.022+0000 7fc0b9e00640 1 --2- 172.21.15.2:0/762412259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c364090 0x55f88c35fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.022+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/762412259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c3757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.022+0000 7fc0ba800640 1 -- 172.21.15.2:0/762412259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 msgr2=0x55f88c3757b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.022+0000 7fc0ba800640 1 --2- 172.21.15.2:0/762412259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c3757b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.022+0000 7fc0ba800640 1 -- 172.21.15.2:0/762412259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c364090 msgr2=0x55f88c35fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.022+0000 7fc0ba800640 1 --2- 172.21.15.2:0/762412259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c364090 0x55f88c35fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.022+0000 7fc0ba800640 1 -- 172.21.15.2:0/762412259 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88c35f520 con 0x55f88c369a60 2024-09-10T08:15:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.022+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/762412259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c3757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.022+0000 7fc0ba800640 1 --2- 172.21.15.2:0/762412259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c369a60 0x55f88c360020 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc0b401f000 tx=0x7fc0b40cd0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee912fef418be8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.023+0000 7fc0aaa00640 1 -- 172.21.15.2:0/762412259 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b409a260 con 0x55f88c369a60 2024-09-10T08:15:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.023+0000 7fc0aaa00640 1 -- 172.21.15.2:0/762412259 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b40051d0 con 0x55f88c369a60 2024-09-10T08:15:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.023+0000 7fc0aaa00640 1 -- 172.21.15.2:0/762412259 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b40baba0 con 0x55f88c369a60 2024-09-10T08:15:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.023+0000 7fc0c405c740 1 -- 172.21.15.2:0/762412259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c369a60 msgr2=0x55f88c360020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.023+0000 7fc0c405c740 1 --2- 172.21.15.2:0/762412259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c369a60 0x55f88c360020 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc0b401f000 tx=0x7fc0b40cd0d0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.023+0000 7fc0c405c740 1 -- 172.21.15.2:0/762412259 shutdown_connections 2024-09-10T08:15:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.023+0000 7fc0c405c740 1 --2- 172.21.15.2:0/762412259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c369a60 0x55f88c360020 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.023+0000 7fc0c405c740 1 --2- 172.21.15.2:0/762412259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c3649d0 0x55f88c3757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.023+0000 7fc0c405c740 1 --2- 172.21.15.2:0/762412259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c364090 0x55f88c35fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.023+0000 7fc0c405c740 1 -- 172.21.15.2:0/762412259 >> 172.21.15.2:0/762412259 conn(0x55f88c35f700 msgr2=0x55f88c365be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.023+0000 7fc0c405c740 1 -- 172.21.15.2:0/762412259 shutdown_connections 2024-09-10T08:15:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.023+0000 7fc0c405c740 1 -- 172.21.15.2:0/762412259 wait complete. 2024-09-10T08:15:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.024+0000 7fc0c405c740 1 -- start start 2024-09-10T08:15:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.024+0000 7fc0c405c740 1 Processor -- start 2024-09-10T08:15:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.024+0000 7fc0c405c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c364090 0x55f88c3756d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.024+0000 7fc0c405c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c3649d0 0x55f88c372170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.024+0000 7fc0c405c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 0x55f88c372710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.024+0000 7fc0c405c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1ab120 con 0x55f88c3649d0 2024-09-10T08:15:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.024+0000 7fc0c405c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1ac590 con 0x55f88c369a60 2024-09-10T08:15:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.024+0000 7fc0c405c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1ab7f0 con 0x55f88c364090 2024-09-10T08:15:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.024+0000 7fc0c405c740 10 client.0 _ll_register_callbacks cb 0x7fc0b40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:15:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c3649d0 0x55f88c372170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0b9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c364090 0x55f88c3756d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c3649d0 0x55f88c372170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47104/0 (socket says 172.21.15.2:47104) 2024-09-10T08:15:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0b9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c364090 0x55f88c3756d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:39206/0 (socket says 172.21.15.2:39206) 2024-09-10T08:15:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0b3e00640 1 -- 172.21.15.2:0/2353486342 learned_addr learned my addr 172.21.15.2:0/2353486342 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:15:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0ba800640 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 0x55f88c372710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0b9e00640 1 -- 172.21.15.2:0/2353486342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 msgr2=0x55f88c372710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0b9e00640 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 0x55f88c372710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0b9e00640 1 -- 172.21.15.2:0/2353486342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c3649d0 msgr2=0x55f88c372170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:15:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0b9e00640 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c3649d0 0x55f88c372170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0b9e00640 1 -- 172.21.15.2:0/2353486342 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88c3e2710 con 0x55f88c364090 2024-09-10T08:15:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0ba800640 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 0x55f88c372710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:15:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c3649d0 0x55f88c372170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:15:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc0b9e00640 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c364090 0x55f88c3756d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac005ab0 tx=0x7fc0ac009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e4867436dc616c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:15:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.025+0000 7fc09fe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.026+0000 7fc09fe00640 1 -- 172.21.15.2:0/2353486342 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac043070 con 0x55f88c364090 2024-09-10T08:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.026+0000 7fc0c405c740 1 -- 172.21.15.2:0/2353486342 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0b40051d0 con 0x55f88c364090 2024-09-10T08:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.026+0000 7fc09fe00640 1 -- 172.21.15.2:0/2353486342 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc0ac003c80 con 0x55f88c364090 2024-09-10T08:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.026+0000 7fc09fe00640 1 -- 172.21.15.2:0/2353486342 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc0ac035040 con 0x55f88c364090 2024-09-10T08:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.026+0000 7fc0c405c740 10 client.27547 fetch_fsmap learned FSMap version 26 2024-09-10T08:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.026+0000 7fc0c405c740 1 -- 172.21.15.2:0/2353486342 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f88c3e3ad0 con 0x55f88c364090 2024-09-10T08:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.026+0000 7fc09fe00640 1 -- 172.21.15.2:0/2353486342 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b40051d0 con 0x55f88c364090 2024-09-10T08:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.026+0000 7fc09fe00640 1 -- 172.21.15.2:0/2353486342 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc0ac004b80 con 0x55f88c364090 2024-09-10T08:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.027+0000 7fc0c405c740 10 client.27547 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.027+0000 7fc0c405c740 10 client.27547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.027+0000 7fc0c405c740 20 client.27547 populate_metadata read hostname 'smithi002' 2024-09-10T08:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.027+0000 7fc0c405c740 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f88c3e43d0 0x55f88c3e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:15:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.027+0000 7fc0c405c740 4 client.27547 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:15:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.027+0000 7fc0c405c740 1 -- 172.21.15.2:0/2353486342 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f88c2d0b80 con 0x55f88c3e43d0 2024-09-10T08:15:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.027+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f88c3e43d0 0x55f88c3e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:15:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.027+0000 7fc0b3e00640 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f88c3e43d0 0x55f88c3e67f0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=880e4d2ba5a1e82d server_cookie=ad739e40cd69c0e6 in_seq=0 out_seq=0 2024-09-10T08:15:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.027+0000 7fc09fe00640 10 client.27547 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc09fe00640 1 -- 172.21.15.2:0/2353486342 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f88c2d0b80 con 0x55f88c3e43d0 2024-09-10T08:15:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc09fe00640 10 client.27547 handle_command_reply: tid=0 2024-09-10T08:15:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc0c405c740 1 client.27547 shutdown 2024-09-10T08:15:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc0c405c740 1 -- 172.21.15.2:0/2353486342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c364090 msgr2=0x55f88c3756d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc0c405c740 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c364090 0x55f88c3756d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac005ab0 tx=0x7fc0ac009260 comp rx=0 tx=0).stop 2024-09-10T08:15:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc0c405c740 1 -- 172.21.15.2:0/2353486342 shutdown_connections 2024-09-10T08:15:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc0c405c740 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f88c3e43d0 0x55f88c3e67f0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc0c405c740 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f88c369a60 0x55f88c372710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc0c405c740 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f88c3649d0 0x55f88c372170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc0c405c740 1 --2- 172.21.15.2:0/2353486342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f88c364090 0x55f88c3756d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc09fe00640 0 client.27547 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:15:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc0c405c740 1 -- 172.21.15.2:0/2353486342 >> 172.21.15.2:0/2353486342 conn(0x55f88c35f700 msgr2=0x55f88c35e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc0c405c740 1 -- 172.21.15.2:0/2353486342 shutdown_connections 2024-09-10T08:15:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc0c405c740 1 -- 172.21.15.2:0/2353486342 wait complete. 2024-09-10T08:15:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.042+0000 7fc0c405c740 20 client.27547 trim_cache size 0 max 16384 2024-09-10T08:15:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:15:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:15:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:15:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:15:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.043+0000 7fc0c1000640 1 -- 172.21.15.2:0/2428113519 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc0a00a2c30 msgr2=0x7fc0a00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.043+0000 7fc0c1000640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc0a00a2c30 0x7fc0a00a50f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac000c00 tx=0x7fc0ac0046c0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.043+0000 7fc0c1000640 1 -- 172.21.15.2:0/2428113519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0bc105910 msgr2=0x7fc0bc1a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:15:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.043+0000 7fc0c1000640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0bc105910 0x7fc0bc1a1580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4009fd0 tx=0x7fc0b400ef10 comp rx=0 tx=0).stop 2024-09-10T08:15:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.043+0000 7fc0c1000640 1 -- 172.21.15.2:0/2428113519 shutdown_connections 2024-09-10T08:15:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.043+0000 7fc0c1000640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0bc105910 0x7fc0bc1a1580 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.043+0000 7fc0c1000640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc0a00a2c30 0x7fc0a00a50f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.043+0000 7fc0c1000640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc104f50 0x7fc0bc19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.043+0000 7fc0c1000640 1 --2- 172.21.15.2:0/2428113519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0bc103d60 0x7fc0bc199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:15:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.043+0000 7fc0c1000640 1 -- 172.21.15.2:0/2428113519 >> 172.21.15.2:0/2428113519 conn(0x7fc0bc0ff4e0 msgr2=0x7fc0bc108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:15:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.043+0000 7fc0c1000640 1 -- 172.21.15.2:0/2428113519 shutdown_connections 2024-09-10T08:15:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:15:58.043+0000 7fc0c1000640 1 -- 172.21.15.2:0/2428113519 wait complete. 2024-09-10T08:15:58.770 INFO:tasks.cephfs.filesystem:scrub status for tag:dbeebd8b-f2cb-4d50-8242-cf36223c1a87 - {'path': '/', 'tag': 'dbeebd8b-f2cb-4d50-8242-cf36223c1a87', 'options': 'recursive,force'} 2024-09-10T08:15:59.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:59 smithi002 ceph-mon[30563]: pgmap v622: 129 pgs: 129 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:15:59.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:15:59 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:15:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:59 smithi155 ceph-mon[35405]: pgmap v622: 129 pgs: 129 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:15:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:15:59 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:15:59.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:59 smithi107 ceph-mon[35048]: pgmap v622: 129 pgs: 129 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:15:59.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:15:59 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:16:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:01 smithi002 ceph-mon[30563]: pgmap v623: 129 pgs: 129 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:16:01.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:01 smithi155 ceph-mon[35405]: pgmap v623: 129 pgs: 129 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:16:01.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:01 smithi107 ceph-mon[35048]: pgmap v623: 129 pgs: 129 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:16:03.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:03 smithi002 ceph-mon[30563]: pgmap v624: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:16:03.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:03 smithi155 ceph-mon[35405]: pgmap v624: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:16:04.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:03 smithi107 ceph-mon[35048]: pgmap v624: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:16:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:05 smithi002 ceph-mon[30563]: pgmap v625: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:16:05.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:05 smithi155 ceph-mon[35405]: pgmap v625: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:16:05.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:05 smithi107 ceph-mon[35048]: pgmap v625: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:16:07.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:07 smithi002 ceph-mon[30563]: pgmap v626: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:16:07.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:07 smithi155 ceph-mon[35405]: pgmap v626: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:16:07.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:07 smithi107 ceph-mon[35048]: pgmap v626: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:16:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:09 smithi002 ceph-mon[30563]: pgmap v627: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:16:09.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:09 smithi155 ceph-mon[35405]: pgmap v627: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:16:09.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:09 smithi107 ceph-mon[35048]: pgmap v627: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:16:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.172+0000 7fc6d1c00640 1 -- 172.21.15.2:0/3570397721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc103d60 msgr2=0x7fc6cc104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.172+0000 7fc6d1c00640 1 --2- 172.21.15.2:0/3570397721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc103d60 0x7fc6cc104160 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009900 tx=0x7fc6bc02a410 comp rx=0 tx=0).stop 2024-09-10T08:16:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.172+0000 7fc6d1c00640 1 -- 172.21.15.2:0/3570397721 shutdown_connections 2024-09-10T08:16:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.172+0000 7fc6d1c00640 1 --2- 172.21.15.2:0/3570397721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6cc105910 0x7fc6cc10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.172+0000 7fc6d1c00640 1 --2- 172.21.15.2:0/3570397721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc104f50 0x7fc6cc1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.172+0000 7fc6d1c00640 1 --2- 172.21.15.2:0/3570397721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc103d60 0x7fc6cc104160 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.172+0000 7fc6d1c00640 1 -- 172.21.15.2:0/3570397721 >> 172.21.15.2:0/3570397721 conn(0x7fc6cc0ff4e0 msgr2=0x7fc6cc101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.172+0000 7fc6d1c00640 1 -- 172.21.15.2:0/3570397721 shutdown_connections 2024-09-10T08:16:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.172+0000 7fc6d1c00640 1 -- 172.21.15.2:0/3570397721 wait complete. 2024-09-10T08:16:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.173+0000 7fc6d1c00640 1 Processor -- start 2024-09-10T08:16:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.173+0000 7fc6d1c00640 1 -- start start 2024-09-10T08:16:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.173+0000 7fc6d1c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6cc103d60 0x7fc6cc199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.173+0000 7fc6d1c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc104f50 0x7fc6cc19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.173+0000 7fc6d1c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc105910 0x7fc6cc1a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.173+0000 7fc6d1c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc10fa30 con 0x7fc6cc103d60 2024-09-10T08:16:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.173+0000 7fc6d1c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc10f8b0 con 0x7fc6cc104f50 2024-09-10T08:16:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.173+0000 7fc6d1c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc10fbb0 con 0x7fc6cc105910 2024-09-10T08:16:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.174+0000 7fc6caa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6cc103d60 0x7fc6cc199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.174+0000 7fc6caa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6cc103d60 0x7fc6cc199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46196/0 (socket says 172.21.15.2:46196) 2024-09-10T08:16:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.174+0000 7fc6caa00640 1 -- 172.21.15.2:0/2305970926 learned_addr learned my addr 172.21.15.2:0/2305970926 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.174+0000 7fc6c3e00640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc104f50 0x7fc6cc19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.174+0000 7fc6cb400640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc105910 0x7fc6cc1a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.174+0000 7fc6caa00640 1 -- 172.21.15.2:0/2305970926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc105910 msgr2=0x7fc6cc1a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.174+0000 7fc6caa00640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc105910 0x7fc6cc1a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.174+0000 7fc6caa00640 1 -- 172.21.15.2:0/2305970926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc104f50 msgr2=0x7fc6cc19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.174+0000 7fc6caa00640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc104f50 0x7fc6cc19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.174+0000 7fc6caa00640 1 -- 172.21.15.2:0/2305970926 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6cc1a1c10 con 0x7fc6cc103d60 2024-09-10T08:16:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.175+0000 7fc6caa00640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6cc103d60 0x7fc6cc199ff0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc031440 tx=0x7fc6bc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.175+0000 7fc6c3400640 1 -- 172.21.15.2:0/2305970926 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc03f030 con 0x7fc6cc103d60 2024-09-10T08:16:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.175+0000 7fc6c3400640 1 -- 172.21.15.2:0/2305970926 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6bc003ce0 con 0x7fc6cc103d60 2024-09-10T08:16:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.175+0000 7fc6d1c00640 1 -- 172.21.15.2:0/2305970926 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6cc1a1ea0 con 0x7fc6cc103d60 2024-09-10T08:16:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.175+0000 7fc6c3400640 1 -- 172.21.15.2:0/2305970926 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc035070 con 0x7fc6cc103d60 2024-09-10T08:16:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.175+0000 7fc6d1c00640 1 -- 172.21.15.2:0/2305970926 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6cc1a22b0 con 0x7fc6cc103d60 2024-09-10T08:16:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.179+0000 7fc6d4a1b740 1 Processor -- start 2024-09-10T08:16:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.179+0000 7fc6d4a1b740 1 -- start start 2024-09-10T08:16:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.180+0000 7fc6c3400640 1 -- 172.21.15.2:0/2305970926 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc6bc042030 con 0x7fc6cc103d60 2024-09-10T08:16:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.180+0000 7fc6d4a1b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.180+0000 7fc6d4a1b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce9d0 0x55fd333d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.180+0000 7fc6d4a1b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333d3a60 0x55fd333d5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.180+0000 7fc6d4a1b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fd33333100 con 0x55fd333ce090 2024-09-10T08:16:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.180+0000 7fc6d4a1b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fd332db1b0 con 0x55fd333ce9d0 2024-09-10T08:16:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.180+0000 7fc6d4a1b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fd3332f0c0 con 0x55fd333d3a60 2024-09-10T08:16:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.180+0000 7fc6cb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.180+0000 7fc6c3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333d3a60 0x55fd333d5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.180+0000 7fc6caa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce9d0 0x55fd333d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.180+0000 7fc6c3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333d3a60 0x55fd333d5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49526/0 (socket says 172.21.15.2:49526) 2024-09-10T08:16:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.180+0000 7fc6cb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46198/0 (socket says 172.21.15.2:46198) 2024-09-10T08:16:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.180+0000 7fc6c3e00640 1 -- 172.21.15.2:0/3439235961 learned_addr learned my addr 172.21.15.2:0/3439235961 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.181+0000 7fc6c3e00640 1 -- 172.21.15.2:0/3439235961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce9d0 msgr2=0x55fd333d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.181+0000 7fc6c3e00640 1 --2- 172.21.15.2:0/3439235961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce9d0 0x55fd333d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.181+0000 7fc6c3e00640 1 -- 172.21.15.2:0/3439235961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 msgr2=0x55fd333ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.181+0000 7fc6c3e00640 1 --2- 172.21.15.2:0/3439235961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.181+0000 7fc6c3e00640 1 -- 172.21.15.2:0/3439235961 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd33046210 con 0x55fd333d3a60 2024-09-10T08:16:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.181+0000 7fc6cb400640 1 --2- 172.21.15.2:0/3439235961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.181+0000 7fc6c3e00640 1 --2- 172.21.15.2:0/3439235961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333d3a60 0x55fd333d5f20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009ca0 tx=0x7fc6b4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34d8d67f3c71f8dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.181+0000 7fc6bbe00640 1 -- 172.21.15.2:0/3439235961 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b401e070 con 0x55fd333d3a60 2024-09-10T08:16:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.181+0000 7fc6c3400640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6b00a2c30 0x7fc6b00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.181+0000 7fc6bbe00640 1 -- 172.21.15.2:0/3439235961 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6b4019b10 con 0x55fd333d3a60 2024-09-10T08:16:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.181+0000 7fc6bbe00640 1 -- 172.21.15.2:0/3439235961 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400eae0 con 0x55fd333d3a60 2024-09-10T08:16:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.181+0000 7fc6c3400640 1 -- 172.21.15.2:0/2305970926 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc6bc0ef8c0 con 0x7fc6cc103d60 2024-09-10T08:16:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.182+0000 7fc6cb400640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6b00a2c30 0x7fc6b00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.182+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/3439235961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333d3a60 msgr2=0x55fd333d5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.182+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/3439235961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333d3a60 0x55fd333d5f20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009ca0 tx=0x7fc6b4011810 comp rx=0 tx=0).stop 2024-09-10T08:16:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.182+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/3439235961 shutdown_connections 2024-09-10T08:16:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.182+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/3439235961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333d3a60 0x55fd333d5f20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.182+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/3439235961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce9d0 0x55fd333d3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.182+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/3439235961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.182+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/3439235961 >> 172.21.15.2:0/3439235961 conn(0x55fd333c9700 msgr2=0x55fd333c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.182+0000 7fc6cb400640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6b00a2c30 0x7fc6b00a50f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009fd0 tx=0x7fc6c4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.182+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/3439235961 shutdown_connections 2024-09-10T08:16:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.182+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/3439235961 wait complete. 2024-09-10T08:16:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.183+0000 7fc6d4a1b740 1 -- start start 2024-09-10T08:16:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.183+0000 7fc6d4a1b740 1 Processor -- start 2024-09-10T08:16:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.183+0000 7fc6d4a1b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333d8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.183+0000 7fc6d4a1b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce9d0 0x55fd333d8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.183+0000 7fc6d4a1b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333d3a60 0x55fd333df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.183+0000 7fc6d4a1b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fd333d6a30 con 0x55fd333ce090 2024-09-10T08:16:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.183+0000 7fc6d4a1b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fd332bc090 con 0x55fd333ce9d0 2024-09-10T08:16:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.183+0000 7fc6d4a1b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fd333d6bb0 con 0x55fd333d3a60 2024-09-10T08:16:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6d4a1b740 10 client.0 _ll_register_callbacks cb 0x55fd33310f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6caa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333d8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6caa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333d8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46208/0 (socket says 172.21.15.2:46208) 2024-09-10T08:16:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6caa00640 1 -- 172.21.15.2:0/2896710339 learned_addr learned my addr 172.21.15.2:0/2896710339 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6c3e00640 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce9d0 0x55fd333d8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6cb400640 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333d3a60 0x55fd333df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6caa00640 1 -- 172.21.15.2:0/2896710339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333d3a60 msgr2=0x55fd333df7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6caa00640 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333d3a60 0x55fd333df7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6caa00640 1 -- 172.21.15.2:0/2896710339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce9d0 msgr2=0x55fd333d8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6caa00640 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce9d0 0x55fd333d8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6caa00640 1 -- 172.21.15.2:0/2896710339 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd33112620 con 0x55fd333ce090 2024-09-10T08:16:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6c3e00640 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce9d0 0x55fd333d8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6caa00640 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333d8240 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc000c00 tx=0x7fc6bc032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8c8b906a0eaea7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.184+0000 7fc6b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:16:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.185+0000 7fc6b8c00640 1 -- 172.21.15.2:0/2896710339 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc049070 con 0x55fd333ce090 2024-09-10T08:16:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.185+0000 7fc6b8c00640 1 -- 172.21.15.2:0/2896710339 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc6bc03bcf0 con 0x55fd333ce090 2024-09-10T08:16:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.185+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/2896710339 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6b4019b10 con 0x55fd333ce090 2024-09-10T08:16:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.185+0000 7fc6b8c00640 1 -- 172.21.15.2:0/2896710339 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc6bc0bb920 con 0x55fd333ce090 2024-09-10T08:16:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.185+0000 7fc6d4a1b740 10 client.17586 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.185+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/2896710339 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fd333e25b0 con 0x55fd333ce090 2024-09-10T08:16:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.185+0000 7fc6b8c00640 1 -- 172.21.15.2:0/2896710339 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6b4019b10 con 0x55fd333ce090 2024-09-10T08:16:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.185+0000 7fc6b8c00640 1 -- 172.21.15.2:0/2896710339 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc6bc0d2080 con 0x55fd333ce090 2024-09-10T08:16:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.186+0000 7fc6d4a1b740 10 client.17586 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.186+0000 7fc6d4a1b740 10 client.17586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.186+0000 7fc6d4a1b740 20 client.17586 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.186+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fd333e3ce0 0x55fd333e60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.186+0000 7fc6d4a1b740 4 client.17586 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.186+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/2896710339 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd33340830 con 0x55fd333e3ce0 2024-09-10T08:16:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.186+0000 7fc6c3e00640 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fd333e3ce0 0x55fd333e60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.187+0000 7fc6c3e00640 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fd333e3ce0 0x55fd333e60f0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fc9d5a8f8aea6b4 server_cookie=fbba20b0028b5738 in_seq=0 out_seq=0 2024-09-10T08:16:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.187+0000 7fc6b8c00640 10 client.17586 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.187+0000 7fc6b8c00640 1 -- 172.21.15.2:0/2896710339 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd33340830 con 0x55fd333e3ce0 2024-09-10T08:16:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.187+0000 7fc6b8c00640 10 client.17586 handle_command_reply: tid=0 2024-09-10T08:16:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.187+0000 7fc6d4a1b740 1 client.17586 shutdown 2024-09-10T08:16:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.188+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/2896710339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 msgr2=0x55fd333d8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.188+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333d8240 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc000c00 tx=0x7fc6bc032f90 comp rx=0 tx=0).stop 2024-09-10T08:16:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.188+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/2896710339 shutdown_connections 2024-09-10T08:16:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.188+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fd333e3ce0 0x55fd333e60f0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.188+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333d3a60 0x55fd333df7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.188+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce9d0 0x55fd333d8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.188+0000 7fc6b8c00640 0 client.17586 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.188+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/2896710339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333d8240 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.188+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/2896710339 >> 172.21.15.2:0/2896710339 conn(0x55fd333c9700 msgr2=0x55fd333caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.188+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/2896710339 shutdown_connections 2024-09-10T08:16:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.188+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/2896710339 wait complete. 2024-09-10T08:16:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.188+0000 7fc6d4a1b740 20 client.17586 trim_cache size 0 max 16384 2024-09-10T08:16:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.203+0000 7fc6d4a1b740 1 Processor -- start 2024-09-10T08:16:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.203+0000 7fc6d4a1b740 1 -- start start 2024-09-10T08:16:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.203+0000 7fc6d4a1b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333d8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.203+0000 7fc6d4a1b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333ce9d0 0x55fd333df780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.203+0000 7fc6d4a1b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333d3a60 0x55fd333d8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.203+0000 7fc6d4a1b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1af2b0 con 0x55fd333ce090 2024-09-10T08:16:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6d4a1b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1acb80 con 0x55fd333d3a60 2024-09-10T08:16:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6d4a1b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1aa9d0 con 0x55fd333ce9d0 2024-09-10T08:16:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6caa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333d8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6caa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333d8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46216/0 (socket says 172.21.15.2:46216) 2024-09-10T08:16:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6caa00640 1 -- 172.21.15.2:0/2111744443 learned_addr learned my addr 172.21.15.2:0/2111744443 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6cb400640 1 --2- 172.21.15.2:0/2111744443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333d3a60 0x55fd333d8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6c3e00640 1 --2- 172.21.15.2:0/2111744443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333ce9d0 0x55fd333df780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6caa00640 1 -- 172.21.15.2:0/2111744443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333ce9d0 msgr2=0x55fd333df780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6caa00640 1 --2- 172.21.15.2:0/2111744443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333ce9d0 0x55fd333df780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6caa00640 1 -- 172.21.15.2:0/2111744443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333d3a60 msgr2=0x55fd333d8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6caa00640 1 --2- 172.21.15.2:0/2111744443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333d3a60 0x55fd333d8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6caa00640 1 -- 172.21.15.2:0/2111744443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd333c9520 con 0x55fd333ce090 2024-09-10T08:16:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6cb400640 1 --2- 172.21.15.2:0/2111744443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333d3a60 0x55fd333d8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:16:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.204+0000 7fc6caa00640 1 --2- 172.21.15.2:0/2111744443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333d8200 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0096c0 tx=0x7fc6bc0ba2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fc0f41c4ac36b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.205+0000 7fc6bb400640 1 -- 172.21.15.2:0/2111744443 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc049070 con 0x55fd333ce090 2024-09-10T08:16:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.205+0000 7fc6bb400640 1 -- 172.21.15.2:0/2111744443 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6bc0c49a0 con 0x55fd333ce090 2024-09-10T08:16:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.205+0000 7fc6bb400640 1 -- 172.21.15.2:0/2111744443 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc005d50 con 0x55fd333ce090 2024-09-10T08:16:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.205+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/2111744443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 msgr2=0x55fd333d8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.205+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/2111744443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333d8200 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0096c0 tx=0x7fc6bc0ba2f0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.205+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/2111744443 shutdown_connections 2024-09-10T08:16:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.205+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/2111744443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333d3a60 0x55fd333d8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.205+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/2111744443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333ce9d0 0x55fd333df780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.205+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/2111744443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333ce090 0x55fd333d8200 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.205+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/2111744443 >> 172.21.15.2:0/2111744443 conn(0x55fd333c9700 msgr2=0x55fd333cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.205+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/2111744443 shutdown_connections 2024-09-10T08:16:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.205+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/2111744443 wait complete. 2024-09-10T08:16:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.206+0000 7fc6d4a1b740 1 -- start start 2024-09-10T08:16:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.206+0000 7fc6d4a1b740 1 Processor -- start 2024-09-10T08:16:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.206+0000 7fc6d4a1b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce090 0x55fd333dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.206+0000 7fc6d4a1b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333ce9d0 0x55fd333dcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.206+0000 7fc6d4a1b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333d3a60 0x55fd3344c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.206+0000 7fc6d4a1b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1ab040 con 0x55fd333d3a60 2024-09-10T08:16:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.206+0000 7fc6d4a1b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1ac4b0 con 0x55fd333ce090 2024-09-10T08:16:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.206+0000 7fc6d4a1b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1ab710 con 0x55fd333ce9d0 2024-09-10T08:16:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.206+0000 7fc6d4a1b740 10 client.0 _ll_register_callbacks cb 0x7fc6bc0f5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.207+0000 7fc6c3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333ce9d0 0x55fd333dcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.207+0000 7fc6c3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333ce9d0 0x55fd333dcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49568/0 (socket says 172.21.15.2:49568) 2024-09-10T08:16:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.207+0000 7fc6c3e00640 1 -- 172.21.15.2:0/4251523735 learned_addr learned my addr 172.21.15.2:0/4251523735 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.207+0000 7fc6cb400640 1 --2- 172.21.15.2:0/4251523735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333d3a60 0x55fd3344c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.207+0000 7fc6caa00640 1 --2- 172.21.15.2:0/4251523735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce090 0x55fd333dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.207+0000 7fc6c3e00640 1 -- 172.21.15.2:0/4251523735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce090 msgr2=0x55fd333dc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.207+0000 7fc6c3e00640 1 --2- 172.21.15.2:0/4251523735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce090 0x55fd333dc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.207+0000 7fc6c3e00640 1 -- 172.21.15.2:0/4251523735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333d3a60 msgr2=0x55fd3344c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.207+0000 7fc6c3e00640 1 --2- 172.21.15.2:0/4251523735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333d3a60 0x55fd3344c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.207+0000 7fc6c3e00640 1 -- 172.21.15.2:0/4251523735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd333dd160 con 0x55fd333ce9d0 2024-09-10T08:16:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.207+0000 7fc6c3e00640 1 --2- 172.21.15.2:0/4251523735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333ce9d0 0x55fd333dcac0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55fd333ca580 tx=0x7fc6b40197b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f322922e78cfd879 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.207+0000 7fc6b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:16:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.208+0000 7fc6b8c00640 1 -- 172.21.15.2:0/4251523735 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b401e040 con 0x55fd333ce9d0 2024-09-10T08:16:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.208+0000 7fc6b8c00640 1 -- 172.21.15.2:0/4251523735 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc6b4005240 con 0x55fd333ce9d0 2024-09-10T08:16:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.208+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/4251523735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6bc0c49a0 con 0x55fd333ce9d0 2024-09-10T08:16:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.208+0000 7fc6b8c00640 1 -- 172.21.15.2:0/4251523735 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc6b403b050 con 0x55fd333ce9d0 2024-09-10T08:16:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.208+0000 7fc6d4a1b740 10 client.27565 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.208+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/4251523735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fd3344e090 con 0x55fd333ce9d0 2024-09-10T08:16:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.208+0000 7fc6b8c00640 1 -- 172.21.15.2:0/4251523735 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6bc0c49a0 con 0x55fd333ce9d0 2024-09-10T08:16:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.209+0000 7fc6b8c00640 1 -- 172.21.15.2:0/4251523735 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc6b4004bf0 con 0x55fd333ce9d0 2024-09-10T08:16:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.209+0000 7fc6d4a1b740 10 client.27565 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.209+0000 7fc6d4a1b740 10 client.27565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.209+0000 7fc6d4a1b740 20 client.27565 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.209+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/4251523735 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fd3344e8e0 0x55fd33450d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.209+0000 7fc6d4a1b740 4 client.27565 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:16:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.209+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/4251523735 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd3333ab80 con 0x55fd3344e8e0 2024-09-10T08:16:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.209+0000 7fc6caa00640 1 --2- 172.21.15.2:0/4251523735 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fd3344e8e0 0x55fd33450d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.210+0000 7fc6caa00640 1 --2- 172.21.15.2:0/4251523735 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fd3344e8e0 0x55fd33450d00 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5bb05eebc1cd248 server_cookie=a791e05a95fbe2bc in_seq=0 out_seq=0 2024-09-10T08:16:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.210+0000 7fc6b8c00640 10 client.27565 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6b8c00640 1 -- 172.21.15.2:0/4251523735 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd3333ab80 con 0x55fd3344e8e0 2024-09-10T08:16:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6b8c00640 10 client.27565 handle_command_reply: tid=0 2024-09-10T08:16:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6d4a1b740 1 client.27565 shutdown 2024-09-10T08:16:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/4251523735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333ce9d0 msgr2=0x55fd333dcac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/4251523735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333ce9d0 0x55fd333dcac0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55fd333ca580 tx=0x7fc6b40197b0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/4251523735 shutdown_connections 2024-09-10T08:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/4251523735 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fd3344e8e0 0x55fd33450d00 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/4251523735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fd333d3a60 0x55fd3344c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/4251523735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd333ce9d0 0x55fd333dcac0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6d4a1b740 1 --2- 172.21.15.2:0/4251523735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fd333ce090 0x55fd333dc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6b8c00640 0 client.27565 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/4251523735 >> 172.21.15.2:0/4251523735 conn(0x55fd333c9700 msgr2=0x55fd333cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/4251523735 shutdown_connections 2024-09-10T08:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6d4a1b740 1 -- 172.21.15.2:0/4251523735 wait complete. 2024-09-10T08:16:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.224+0000 7fc6d4a1b740 20 client.27565 trim_cache size 0 max 16384 2024-09-10T08:16:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:16:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:16:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:16:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:16:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.225+0000 7fc6d1c00640 1 -- 172.21.15.2:0/2305970926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6b00a2c30 msgr2=0x7fc6b00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.225+0000 7fc6d1c00640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6b00a2c30 0x7fc6b00a50f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009fd0 tx=0x7fc6c4009500 comp rx=0 tx=0).stop 2024-09-10T08:16:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.225+0000 7fc6d1c00640 1 -- 172.21.15.2:0/2305970926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6cc103d60 msgr2=0x7fc6cc199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.225+0000 7fc6d1c00640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6cc103d60 0x7fc6cc199ff0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc031440 tx=0x7fc6bc032ce0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.225+0000 7fc6d1c00640 1 -- 172.21.15.2:0/2305970926 shutdown_connections 2024-09-10T08:16:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.225+0000 7fc6d1c00640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6b00a2c30 0x7fc6b00a50f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.225+0000 7fc6d1c00640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc105910 0x7fc6cc1a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.225+0000 7fc6d1c00640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc104f50 0x7fc6cc19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.225+0000 7fc6d1c00640 1 --2- 172.21.15.2:0/2305970926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6cc103d60 0x7fc6cc199ff0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.225+0000 7fc6d1c00640 1 -- 172.21.15.2:0/2305970926 >> 172.21.15.2:0/2305970926 conn(0x7fc6cc0ff4e0 msgr2=0x7fc6cc108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.225+0000 7fc6d1c00640 1 -- 172.21.15.2:0/2305970926 shutdown_connections 2024-09-10T08:16:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:11.225+0000 7fc6d1c00640 1 -- 172.21.15.2:0/2305970926 wait complete. 2024-09-10T08:16:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:11 smithi002 ceph-mon[30563]: pgmap v628: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:16:11.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:11 smithi155 ceph-mon[35405]: pgmap v628: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:16:11.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:11 smithi107 ceph-mon[35048]: pgmap v628: 129 pgs: 129 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:16:13.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:13 smithi002 ceph-mon[30563]: pgmap v629: 129 pgs: 129 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:16:13.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:13 smithi155 ceph-mon[35405]: pgmap v629: 129 pgs: 129 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:16:13.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:13 smithi107 ceph-mon[35048]: pgmap v629: 129 pgs: 129 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:16:15.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:15 smithi002 ceph-mon[30563]: pgmap v630: 129 pgs: 129 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:16:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:15 smithi155 ceph-mon[35405]: pgmap v630: 129 pgs: 129 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:16:15.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:15 smithi107 ceph-mon[35048]: pgmap v630: 129 pgs: 129 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:16:17.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:17 smithi002 ceph-mon[30563]: pgmap v631: 129 pgs: 129 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:16:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:17 smithi155 ceph-mon[35405]: pgmap v631: 129 pgs: 129 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:16:17.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:17 smithi107 ceph-mon[35048]: pgmap v631: 129 pgs: 129 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:16:19.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:19 smithi002 ceph-mon[30563]: pgmap v632: 129 pgs: 129 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:16:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:19 smithi155 ceph-mon[35405]: pgmap v632: 129 pgs: 129 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:16:19.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:19 smithi107 ceph-mon[35048]: pgmap v632: 129 pgs: 129 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:16:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:21 smithi002 ceph-mon[30563]: pgmap v633: 129 pgs: 129 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:16:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:21 smithi155 ceph-mon[35405]: pgmap v633: 129 pgs: 129 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:16:21.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:21 smithi107 ceph-mon[35048]: pgmap v633: 129 pgs: 129 active+clean; 4.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:16:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:23 smithi002 ceph-mon[30563]: pgmap v634: 129 pgs: 129 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:16:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:23 smithi155 ceph-mon[35405]: pgmap v634: 129 pgs: 129 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:16:23.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:23 smithi107 ceph-mon[35048]: pgmap v634: 129 pgs: 129 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:16:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.352+0000 7f57d5800640 1 -- 172.21.15.2:0/2512187485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57d0103d60 msgr2=0x7f57d0104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.352+0000 7f57d5800640 1 --2- 172.21.15.2:0/2512187485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57d0103d60 0x7f57d0104160 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009900 tx=0x7f57c002a410 comp rx=0 tx=0).stop 2024-09-10T08:16:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.352+0000 7f57d5800640 1 -- 172.21.15.2:0/2512187485 shutdown_connections 2024-09-10T08:16:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.352+0000 7f57d5800640 1 --2- 172.21.15.2:0/2512187485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57d0105910 0x7f57d010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.352+0000 7f57d5800640 1 --2- 172.21.15.2:0/2512187485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d0104f50 0x7f57d01053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.352+0000 7f57d5800640 1 --2- 172.21.15.2:0/2512187485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57d0103d60 0x7f57d0104160 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.352+0000 7f57d5800640 1 -- 172.21.15.2:0/2512187485 >> 172.21.15.2:0/2512187485 conn(0x7f57d00ff4e0 msgr2=0x7f57d0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.352+0000 7f57d5800640 1 -- 172.21.15.2:0/2512187485 shutdown_connections 2024-09-10T08:16:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.352+0000 7f57d5800640 1 -- 172.21.15.2:0/2512187485 wait complete. 2024-09-10T08:16:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.353+0000 7f57d5800640 1 Processor -- start 2024-09-10T08:16:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.353+0000 7f57d5800640 1 -- start start 2024-09-10T08:16:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.354+0000 7f57d5800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57d0103d60 0x7f57d019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.354+0000 7f57d5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d0104f50 0x7f57d019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.354+0000 7f57d5800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57d0105910 0x7f57d01a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.354+0000 7f57d5800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f57d010fa30 con 0x7f57d0103d60 2024-09-10T08:16:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.354+0000 7f57d5800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f57d010f8b0 con 0x7f57d0105910 2024-09-10T08:16:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.354+0000 7f57d5800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f57d010fbb0 con 0x7f57d0104f50 2024-09-10T08:16:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.354+0000 7f57cea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57d0103d60 0x7f57d019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.354+0000 7f57c7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d0104f50 0x7f57d019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.354+0000 7f57cea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57d0103d60 0x7f57d019a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56044/0 (socket says 172.21.15.2:56044) 2024-09-10T08:16:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.354+0000 7f57cea00640 1 -- 172.21.15.2:0/3235279814 learned_addr learned my addr 172.21.15.2:0/3235279814 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.354+0000 7f57cf400640 1 --2- 172.21.15.2:0/3235279814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57d0105910 0x7f57d01a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.355+0000 7f57c7e00640 1 -- 172.21.15.2:0/3235279814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57d0105910 msgr2=0x7f57d01a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.355+0000 7f57c7e00640 1 --2- 172.21.15.2:0/3235279814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57d0105910 0x7f57d01a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.355+0000 7f57c7e00640 1 -- 172.21.15.2:0/3235279814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57d0103d60 msgr2=0x7f57d019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.355+0000 7f57c7e00640 1 --2- 172.21.15.2:0/3235279814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57d0103d60 0x7f57d019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.355+0000 7f57c7e00640 1 -- 172.21.15.2:0/3235279814 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d01a1c40 con 0x7f57d0104f50 2024-09-10T08:16:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.355+0000 7f57c7e00640 1 --2- 172.21.15.2:0/3235279814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d0104f50 0x7f57d019a560 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f57b800e9f0 tx=0x7f57b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.355+0000 7f57c7400640 1 -- 172.21.15.2:0/3235279814 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b800ccc0 con 0x7f57d0104f50 2024-09-10T08:16:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.355+0000 7f57c7400640 1 -- 172.21.15.2:0/3235279814 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b8005020 con 0x7f57d0104f50 2024-09-10T08:16:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.355+0000 7f57d5800640 1 -- 172.21.15.2:0/3235279814 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d01a1f30 con 0x7f57d0104f50 2024-09-10T08:16:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.355+0000 7f57c7400640 1 -- 172.21.15.2:0/3235279814 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b80049e0 con 0x7f57d0104f50 2024-09-10T08:16:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.355+0000 7f57d5800640 1 -- 172.21.15.2:0/3235279814 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d01a23f0 con 0x7f57d0104f50 2024-09-10T08:16:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.358+0000 7f57d8681740 1 Processor -- start 2024-09-10T08:16:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.358+0000 7f57d8681740 1 -- start start 2024-09-10T08:16:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57d8681740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c631956090 0x55c631956490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57d8681740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6319569d0 0x55c63195b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57d8681740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c63195ba60 0x55c63195df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57d8681740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c6318bb100 con 0x55c63195ba60 2024-09-10T08:16:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57d8681740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c6318631b0 con 0x55c631956090 2024-09-10T08:16:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57d8681740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c6318b70c0 con 0x55c6319569d0 2024-09-10T08:16:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57c7400640 1 -- 172.21.15.2:0/3235279814 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f57b8005560 con 0x7f57d0104f50 2024-09-10T08:16:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57cf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c631956090 0x55c631956490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57cea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6319569d0 0x55c63195b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57cf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c631956090 0x55c631956490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37246/0 (socket says 172.21.15.2:37246) 2024-09-10T08:16:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57cf400640 1 -- 172.21.15.2:0/1599132518 learned_addr learned my addr 172.21.15.2:0/1599132518 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57c7e00640 1 --2- 172.21.15.2:0/1599132518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c63195ba60 0x55c63195df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57cf400640 1 -- 172.21.15.2:0/1599132518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6319569d0 msgr2=0x55c63195b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57cf400640 1 --2- 172.21.15.2:0/1599132518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6319569d0 0x55c63195b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57cf400640 1 -- 172.21.15.2:0/1599132518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c63195ba60 msgr2=0x55c63195df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.359+0000 7f57cf400640 1 --2- 172.21.15.2:0/1599132518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c63195ba60 0x55c63195df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.360+0000 7f57cf400640 1 -- 172.21.15.2:0/1599132518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6315ce210 con 0x55c631956090 2024-09-10T08:16:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.360+0000 7f57cf400640 1 --2- 172.21.15.2:0/1599132518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c631956090 0x55c631956490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f57c80039f0 tx=0x7f57c8003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd0b97fbb4230403 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.360+0000 7f57bfe00640 1 -- 172.21.15.2:0/1599132518 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c801a470 con 0x55c631956090 2024-09-10T08:16:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.360+0000 7f57bfe00640 1 -- 172.21.15.2:0/1599132518 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c8015070 con 0x55c631956090 2024-09-10T08:16:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.360+0000 7f57bfe00640 1 -- 172.21.15.2:0/1599132518 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8010900 con 0x55c631956090 2024-09-10T08:16:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.360+0000 7f57c7400640 1 --2- 172.21.15.2:0/3235279814 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f57b40a2cb0 0x7f57b40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.361+0000 7f57c7400640 1 -- 172.21.15.2:0/3235279814 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f57b80cf330 con 0x7f57d0104f50 2024-09-10T08:16:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.361+0000 7f57cea00640 1 --2- 172.21.15.2:0/3235279814 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f57b40a2cb0 0x7f57b40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.361+0000 7f57d8681740 1 -- 172.21.15.2:0/1599132518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c631956090 msgr2=0x55c631956490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.361+0000 7f57d8681740 1 --2- 172.21.15.2:0/1599132518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c631956090 0x55c631956490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f57c80039f0 tx=0x7f57c8003eb0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.361+0000 7f57d8681740 1 -- 172.21.15.2:0/1599132518 shutdown_connections 2024-09-10T08:16:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.361+0000 7f57d8681740 1 --2- 172.21.15.2:0/1599132518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c63195ba60 0x55c63195df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.361+0000 7f57d8681740 1 --2- 172.21.15.2:0/1599132518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6319569d0 0x55c63195b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.361+0000 7f57d8681740 1 --2- 172.21.15.2:0/1599132518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c631956090 0x55c631956490 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.361+0000 7f57d8681740 1 -- 172.21.15.2:0/1599132518 >> 172.21.15.2:0/1599132518 conn(0x55c631951700 msgr2=0x55c631951b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.361+0000 7f57d8681740 1 -- 172.21.15.2:0/1599132518 shutdown_connections 2024-09-10T08:16:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.361+0000 7f57d8681740 1 -- 172.21.15.2:0/1599132518 wait complete. 2024-09-10T08:16:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.361+0000 7f57cea00640 1 --2- 172.21.15.2:0/3235279814 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f57b40a2cb0 0x7f57b40a5170 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f57c0000c00 tx=0x7f57c00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.362+0000 7f57d8681740 1 -- start start 2024-09-10T08:16:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.362+0000 7f57d8681740 1 Processor -- start 2024-09-10T08:16:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.362+0000 7f57d8681740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 0x55c631960240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.362+0000 7f57d8681740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6319569d0 0x55c631960780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.362+0000 7f57d8681740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c63195ba60 0x55c631967750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.362+0000 7f57d8681740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c63195ea30 con 0x55c631956090 2024-09-10T08:16:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.362+0000 7f57d8681740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c631844090 con 0x55c6319569d0 2024-09-10T08:16:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.362+0000 7f57d8681740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c63195ebb0 con 0x55c63195ba60 2024-09-10T08:16:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.362+0000 7f57d8681740 10 client.0 _ll_register_callbacks cb 0x55c631898f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.363+0000 7f57cea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 0x55c631960240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.363+0000 7f57c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6319569d0 0x55c631960780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.363+0000 7f57cf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c63195ba60 0x55c631967750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.363+0000 7f57cea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 0x55c631960240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56066/0 (socket says 172.21.15.2:56066) 2024-09-10T08:16:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.363+0000 7f57cea00640 1 -- 172.21.15.2:0/4165885383 learned_addr learned my addr 172.21.15.2:0/4165885383 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.363+0000 7f57cea00640 1 -- 172.21.15.2:0/4165885383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c63195ba60 msgr2=0x55c631967750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.363+0000 7f57cea00640 1 --2- 172.21.15.2:0/4165885383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c63195ba60 0x55c631967750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.363+0000 7f57cea00640 1 -- 172.21.15.2:0/4165885383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6319569d0 msgr2=0x55c631960780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.363+0000 7f57cea00640 1 --2- 172.21.15.2:0/4165885383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6319569d0 0x55c631960780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.363+0000 7f57cea00640 1 -- 172.21.15.2:0/4165885383 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c63169a620 con 0x55c631956090 2024-09-10T08:16:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.363+0000 7f57c7e00640 1 --2- 172.21.15.2:0/4165885383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6319569d0 0x55c631960780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.363+0000 7f57cea00640 1 --2- 172.21.15.2:0/4165885383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 0x55c631960240 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009230 tx=0x7f57c0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372eb842dbcc4845 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.363+0000 7f57bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:16:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.364+0000 7f57bcc00640 1 -- 172.21.15.2:0/4165885383 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c004e070 con 0x55c631956090 2024-09-10T08:16:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.364+0000 7f57bcc00640 1 -- 172.21.15.2:0/4165885383 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f57c00035f0 con 0x55c631956090 2024-09-10T08:16:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.364+0000 7f57d8681740 1 -- 172.21.15.2:0/4165885383 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57c8015070 con 0x55c631956090 2024-09-10T08:16:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.364+0000 7f57bcc00640 1 -- 172.21.15.2:0/4165885383 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f57c005a050 con 0x55c631956090 2024-09-10T08:16:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.364+0000 7f57d8681740 10 client.17604 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.364+0000 7f57d8681740 1 -- 172.21.15.2:0/4165885383 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c63196a5b0 con 0x55c631956090 2024-09-10T08:16:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.364+0000 7f57bcc00640 1 -- 172.21.15.2:0/4165885383 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c8015070 con 0x55c631956090 2024-09-10T08:16:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.365+0000 7f57bcc00640 1 -- 172.21.15.2:0/4165885383 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f57c004a9e0 con 0x55c631956090 2024-09-10T08:16:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.365+0000 7f57d8681740 10 client.17604 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.365+0000 7f57d8681740 10 client.17604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.365+0000 7f57d8681740 20 client.17604 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.365+0000 7f57d8681740 1 --2- 172.21.15.2:0/4165885383 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c63196bc80 0x55c63196e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.365+0000 7f57d8681740 4 client.17604 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.365+0000 7f57d8681740 1 -- 172.21.15.2:0/4165885383 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6318c8830 con 0x55c63196bc80 2024-09-10T08:16:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.365+0000 7f57c7e00640 1 --2- 172.21.15.2:0/4165885383 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c63196bc80 0x55c63196e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.366+0000 7f57c7e00640 1 --2- 172.21.15.2:0/4165885383 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c63196bc80 0x55c63196e0b0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43ae949666810161 server_cookie=9f920b345bf589e6 in_seq=0 out_seq=0 2024-09-10T08:16:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.366+0000 7f57bcc00640 10 client.17604 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.366+0000 7f57bcc00640 1 -- 172.21.15.2:0/4165885383 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6318c8830 con 0x55c63196bc80 2024-09-10T08:16:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.366+0000 7f57bcc00640 10 client.17604 handle_command_reply: tid=0 2024-09-10T08:16:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.366+0000 7f57d8681740 1 client.17604 shutdown 2024-09-10T08:16:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.366+0000 7f57d8681740 1 -- 172.21.15.2:0/4165885383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 msgr2=0x55c631960240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.366+0000 7f57d8681740 1 --2- 172.21.15.2:0/4165885383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 0x55c631960240 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009230 tx=0x7f57c0009260 comp rx=0 tx=0).stop 2024-09-10T08:16:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.367+0000 7f57d8681740 1 -- 172.21.15.2:0/4165885383 shutdown_connections 2024-09-10T08:16:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.367+0000 7f57d8681740 1 --2- 172.21.15.2:0/4165885383 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c63196bc80 0x55c63196e0b0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.367+0000 7f57d8681740 1 --2- 172.21.15.2:0/4165885383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c63195ba60 0x55c631967750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.367+0000 7f57d8681740 1 --2- 172.21.15.2:0/4165885383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6319569d0 0x55c631960780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.367+0000 7f57d8681740 1 --2- 172.21.15.2:0/4165885383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 0x55c631960240 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.367+0000 7f57bcc00640 0 client.17604 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.367+0000 7f57d8681740 1 -- 172.21.15.2:0/4165885383 >> 172.21.15.2:0/4165885383 conn(0x55c631951700 msgr2=0x55c631952f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.367+0000 7f57d8681740 1 -- 172.21.15.2:0/4165885383 shutdown_connections 2024-09-10T08:16:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.367+0000 7f57d8681740 1 -- 172.21.15.2:0/4165885383 wait complete. 2024-09-10T08:16:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.367+0000 7f57d8681740 20 client.17604 trim_cache size 0 max 16384 2024-09-10T08:16:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.382+0000 7f57d8681740 1 Processor -- start 2024-09-10T08:16:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.382+0000 7f57d8681740 1 -- start start 2024-09-10T08:16:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57d8681740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 0x55c631960660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57d8681740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6319569d0 0x55c631967670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57d8681740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c63195ba60 0x55c631952af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57d8681740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f57d01af2b0 con 0x55c631956090 2024-09-10T08:16:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57d8681740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f57d01acb80 con 0x55c63195ba60 2024-09-10T08:16:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57d8681740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f57d01aa9d0 con 0x55c6319569d0 2024-09-10T08:16:24.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57cea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 0x55c631960660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57cea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 0x55c631960660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56076/0 (socket says 172.21.15.2:56076) 2024-09-10T08:16:24.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57cea00640 1 -- 172.21.15.2:0/1680359634 learned_addr learned my addr 172.21.15.2:0/1680359634 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57c7e00640 1 --2- 172.21.15.2:0/1680359634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6319569d0 0x55c631967670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57cf400640 1 --2- 172.21.15.2:0/1680359634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c63195ba60 0x55c631952af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57cea00640 1 -- 172.21.15.2:0/1680359634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6319569d0 msgr2=0x55c631967670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57cea00640 1 --2- 172.21.15.2:0/1680359634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6319569d0 0x55c631967670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57cea00640 1 -- 172.21.15.2:0/1680359634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c63195ba60 msgr2=0x55c631952af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57cea00640 1 --2- 172.21.15.2:0/1680359634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c63195ba60 0x55c631952af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.383+0000 7f57cea00640 1 -- 172.21.15.2:0/1680359634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c631951520 con 0x55c631956090 2024-09-10T08:16:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.384+0000 7f57cea00640 1 --2- 172.21.15.2:0/1680359634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 0x55c631960660 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f57c004ade0 tx=0x7f57c0031a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31709ec0ebf671c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:24.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.384+0000 7f57bf400640 1 -- 172.21.15.2:0/1680359634 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0062070 con 0x55c631956090 2024-09-10T08:16:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.384+0000 7f57bf400640 1 -- 172.21.15.2:0/1680359634 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c004ab30 con 0x55c631956090 2024-09-10T08:16:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.384+0000 7f57bf400640 1 -- 172.21.15.2:0/1680359634 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c004e040 con 0x55c631956090 2024-09-10T08:16:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.385+0000 7f57d8681740 1 -- 172.21.15.2:0/1680359634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 msgr2=0x55c631960660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.385+0000 7f57d8681740 1 --2- 172.21.15.2:0/1680359634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 0x55c631960660 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f57c004ade0 tx=0x7f57c0031a70 comp rx=0 tx=0).stop 2024-09-10T08:16:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.385+0000 7f57d8681740 1 -- 172.21.15.2:0/1680359634 shutdown_connections 2024-09-10T08:16:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.385+0000 7f57d8681740 1 --2- 172.21.15.2:0/1680359634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c63195ba60 0x55c631952af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.385+0000 7f57d8681740 1 --2- 172.21.15.2:0/1680359634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6319569d0 0x55c631967670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.385+0000 7f57d8681740 1 --2- 172.21.15.2:0/1680359634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c631956090 0x55c631960660 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.385+0000 7f57d8681740 1 -- 172.21.15.2:0/1680359634 >> 172.21.15.2:0/1680359634 conn(0x55c631951700 msgr2=0x55c631950880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.385+0000 7f57d8681740 1 -- 172.21.15.2:0/1680359634 shutdown_connections 2024-09-10T08:16:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.385+0000 7f57d8681740 1 -- 172.21.15.2:0/1680359634 wait complete. 2024-09-10T08:16:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.385+0000 7f57d8681740 1 -- start start 2024-09-10T08:16:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.385+0000 7f57d8681740 1 Processor -- start 2024-09-10T08:16:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.386+0000 7f57d8681740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c631956090 0x55c631964bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.386+0000 7f57d8681740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6319569d0 0x55c6319650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.386+0000 7f57d8681740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c63195ba60 0x55c631966100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.386+0000 7f57d8681740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f57d01ab040 con 0x55c6319569d0 2024-09-10T08:16:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.386+0000 7f57d8681740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f57d01ac4b0 con 0x55c63195ba60 2024-09-10T08:16:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.386+0000 7f57d8681740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f57d01ab710 con 0x55c631956090 2024-09-10T08:16:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.386+0000 7f57d8681740 10 client.0 _ll_register_callbacks cb 0x7f57c0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.386+0000 7f57c7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6319569d0 0x55c6319650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.386+0000 7f57cf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c63195ba60 0x55c631966100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.386+0000 7f57cf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c63195ba60 0x55c631966100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37282/0 (socket says 172.21.15.2:37282) 2024-09-10T08:16:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.386+0000 7f57cf400640 1 -- 172.21.15.2:0/2450037179 learned_addr learned my addr 172.21.15.2:0/2450037179 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.386+0000 7f57cea00640 1 --2- 172.21.15.2:0/2450037179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c631956090 0x55c631964bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57c7e00640 1 -- 172.21.15.2:0/2450037179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c631956090 msgr2=0x55c631964bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57c7e00640 1 --2- 172.21.15.2:0/2450037179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c631956090 0x55c631964bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57c7e00640 1 -- 172.21.15.2:0/2450037179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c63195ba60 msgr2=0x55c631966100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57c7e00640 1 --2- 172.21.15.2:0/2450037179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c63195ba60 0x55c631966100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57c7e00640 1 -- 172.21.15.2:0/2450037179 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c631966710 con 0x55c6319569d0 2024-09-10T08:16:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57cf400640 1 --2- 172.21.15.2:0/2450037179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c63195ba60 0x55c631966100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57c7e00640 1 --2- 172.21.15.2:0/2450037179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6319569d0 0x55c6319650f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f57b800e9c0 tx=0x7f57b8086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee741cd258850fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:16:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57bcc00640 1 -- 172.21.15.2:0/2450037179 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b801c040 con 0x55c6319569d0 2024-09-10T08:16:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57bcc00640 1 -- 172.21.15.2:0/2450037179 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f57b80860b0 con 0x55c6319569d0 2024-09-10T08:16:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57d8681740 1 -- 172.21.15.2:0/2450037179 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57c004ab30 con 0x55c6319569d0 2024-09-10T08:16:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57bcc00640 1 -- 172.21.15.2:0/2450037179 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f57b809be40 con 0x55c6319569d0 2024-09-10T08:16:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57d8681740 10 client.17610 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.387+0000 7f57d8681740 1 -- 172.21.15.2:0/2450037179 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c631967af0 con 0x55c6319569d0 2024-09-10T08:16:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.388+0000 7f57bcc00640 1 -- 172.21.15.2:0/2450037179 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c004ab30 con 0x55c6319569d0 2024-09-10T08:16:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.388+0000 7f57bcc00640 1 -- 172.21.15.2:0/2450037179 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f57b80a3f90 con 0x55c6319569d0 2024-09-10T08:16:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.388+0000 7f57d8681740 10 client.17610 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.388+0000 7f57d8681740 10 client.17610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.388+0000 7f57d8681740 20 client.17610 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.388+0000 7f57d8681740 1 --2- 172.21.15.2:0/2450037179 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c6319d7070 0x55c6319d9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.388+0000 7f57d8681740 4 client.17610 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:16:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.388+0000 7f57d8681740 1 -- 172.21.15.2:0/2450037179 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6318c2b80 con 0x55c6319d7070 2024-09-10T08:16:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.388+0000 7f57cea00640 1 --2- 172.21.15.2:0/2450037179 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c6319d7070 0x55c6319d9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.389+0000 7f57cea00640 1 --2- 172.21.15.2:0/2450037179 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c6319d7070 0x55c6319d9490 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3aa3f2e8db1bb4 server_cookie=1fefa528dcca1b43 in_seq=0 out_seq=0 2024-09-10T08:16:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.389+0000 7f57bcc00640 10 client.17610 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.404+0000 7f57bcc00640 1 -- 172.21.15.2:0/2450037179 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6318c2b80 con 0x55c6319d7070 2024-09-10T08:16:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.404+0000 7f57bcc00640 10 client.17610 handle_command_reply: tid=0 2024-09-10T08:16:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.404+0000 7f57d8681740 1 client.17610 shutdown 2024-09-10T08:16:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.405+0000 7f57d8681740 1 -- 172.21.15.2:0/2450037179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6319569d0 msgr2=0x55c6319650f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.405+0000 7f57d8681740 1 --2- 172.21.15.2:0/2450037179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6319569d0 0x55c6319650f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f57b800e9c0 tx=0x7f57b8086040 comp rx=0 tx=0).stop 2024-09-10T08:16:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.405+0000 7f57d8681740 1 -- 172.21.15.2:0/2450037179 shutdown_connections 2024-09-10T08:16:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.405+0000 7f57d8681740 1 --2- 172.21.15.2:0/2450037179 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c6319d7070 0x55c6319d9490 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.405+0000 7f57d8681740 1 --2- 172.21.15.2:0/2450037179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c63195ba60 0x55c631966100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.405+0000 7f57d8681740 1 --2- 172.21.15.2:0/2450037179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6319569d0 0x55c6319650f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.405+0000 7f57d8681740 1 --2- 172.21.15.2:0/2450037179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c631956090 0x55c631964bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.405+0000 7f57bcc00640 0 client.17610 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.405+0000 7f57d8681740 1 -- 172.21.15.2:0/2450037179 >> 172.21.15.2:0/2450037179 conn(0x55c631951700 msgr2=0x55c631951ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.405+0000 7f57d8681740 1 -- 172.21.15.2:0/2450037179 shutdown_connections 2024-09-10T08:16:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.405+0000 7f57d8681740 1 -- 172.21.15.2:0/2450037179 wait complete. 2024-09-10T08:16:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.405+0000 7f57d8681740 20 client.17610 trim_cache size 0 max 16384 2024-09-10T08:16:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:16:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:16:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:16:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:16:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.406+0000 7f57d5800640 1 -- 172.21.15.2:0/3235279814 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f57b40a2cb0 msgr2=0x7f57b40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.406+0000 7f57d5800640 1 --2- 172.21.15.2:0/3235279814 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f57b40a2cb0 0x7f57b40a5170 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f57c0000c00 tx=0x7f57c00315b0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.406+0000 7f57d5800640 1 -- 172.21.15.2:0/3235279814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d0104f50 msgr2=0x7f57d019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.406+0000 7f57d5800640 1 --2- 172.21.15.2:0/3235279814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d0104f50 0x7f57d019a560 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f57b800e9f0 tx=0x7f57b800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.406+0000 7f57d5800640 1 -- 172.21.15.2:0/3235279814 shutdown_connections 2024-09-10T08:16:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.406+0000 7f57d5800640 1 --2- 172.21.15.2:0/3235279814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57d0105910 0x7f57d01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.406+0000 7f57d5800640 1 --2- 172.21.15.2:0/3235279814 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f57b40a2cb0 0x7f57b40a5170 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.406+0000 7f57d5800640 1 --2- 172.21.15.2:0/3235279814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d0104f50 0x7f57d019a560 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.406+0000 7f57d5800640 1 --2- 172.21.15.2:0/3235279814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57d0103d60 0x7f57d019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.406+0000 7f57d5800640 1 -- 172.21.15.2:0/3235279814 >> 172.21.15.2:0/3235279814 conn(0x7f57d00ff4e0 msgr2=0x7f57d0108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.406+0000 7f57d5800640 1 -- 172.21.15.2:0/3235279814 shutdown_connections 2024-09-10T08:16:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:24.406+0000 7f57d5800640 1 -- 172.21.15.2:0/3235279814 wait complete. 2024-09-10T08:16:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:25 smithi002 ceph-mon[30563]: pgmap v635: 129 pgs: 129 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:16:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:25 smithi155 ceph-mon[35405]: pgmap v635: 129 pgs: 129 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:16:25.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:25 smithi107 ceph-mon[35048]: pgmap v635: 129 pgs: 129 active+clean; 4.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:16:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:27 smithi002 ceph-mon[30563]: pgmap v636: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-10T08:16:27.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:27 smithi155 ceph-mon[35405]: pgmap v636: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-10T08:16:27.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:27 smithi107 ceph-mon[35048]: pgmap v636: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-09-10T08:16:28.772 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:16:29.160 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:29.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:29 smithi002 ceph-mon[30563]: pgmap v637: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:16:29.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.781+0000 7fe553c00640 1 -- 172.21.15.2:0/2984074562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe54c10e430 msgr2=0x7fe54c110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:29.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.781+0000 7fe553c00640 1 --2- 172.21.15.2:0/2984074562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe54c10e430 0x7fe54c110820 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe544009ca0 tx=0x7fe54402a950 comp rx=0 tx=0).stop 2024-09-10T08:16:29.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.781+0000 7fe553c00640 1 -- 172.21.15.2:0/2984074562 shutdown_connections 2024-09-10T08:16:29.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.781+0000 7fe553c00640 1 --2- 172.21.15.2:0/2984074562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe54c10e430 0x7fe54c110820 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.781+0000 7fe553c00640 1 --2- 172.21.15.2:0/2984074562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe54c074ee0 0x7fe54c10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.781+0000 7fe553c00640 1 --2- 172.21.15.2:0/2984074562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe54c0745c0 0x7fe54c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.781+0000 7fe553c00640 1 -- 172.21.15.2:0/2984074562 >> 172.21.15.2:0/2984074562 conn(0x7fe54c0fd8b0 msgr2=0x7fe54c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:29.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.781+0000 7fe553c00640 1 -- 172.21.15.2:0/2984074562 shutdown_connections 2024-09-10T08:16:29.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.781+0000 7fe553c00640 1 -- 172.21.15.2:0/2984074562 wait complete. 2024-09-10T08:16:29.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.782+0000 7fe553c00640 1 Processor -- start 2024-09-10T08:16:29.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.782+0000 7fe553c00640 1 -- start start 2024-09-10T08:16:29.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.783+0000 7fe553c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe54c0745c0 0x7fe54c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.783+0000 7fe553c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe54c074ee0 0x7fe54c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.783+0000 7fe553c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe54c10e430 0x7fe54c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.783+0000 7fe553c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c114180 con 0x7fe54c074ee0 2024-09-10T08:16:29.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.783+0000 7fe553c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c114000 con 0x7fe54c0745c0 2024-09-10T08:16:29.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.783+0000 7fe553c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c114300 con 0x7fe54c10e430 2024-09-10T08:16:29.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.783+0000 7fe54be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe54c074ee0 0x7fe54c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.783+0000 7fe551200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe54c0745c0 0x7fe54c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.783+0000 7fe54be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe54c074ee0 0x7fe54c19ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37494/0 (socket says 172.21.15.2:37494) 2024-09-10T08:16:29.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.783+0000 7fe54be00640 1 -- 172.21.15.2:0/3451428623 learned_addr learned my addr 172.21.15.2:0/3451428623 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:29.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.783+0000 7fe551c00640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe54c10e430 0x7fe54c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.784+0000 7fe54be00640 1 -- 172.21.15.2:0/3451428623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe54c10e430 msgr2=0x7fe54c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:29.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.784+0000 7fe54be00640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe54c10e430 0x7fe54c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.784+0000 7fe54be00640 1 -- 172.21.15.2:0/3451428623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe54c0745c0 msgr2=0x7fe54c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:29.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.784+0000 7fe54be00640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe54c0745c0 0x7fe54c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.784+0000 7fe54be00640 1 -- 172.21.15.2:0/3451428623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c19f690 con 0x7fe54c074ee0 2024-09-10T08:16:29.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.784+0000 7fe551200640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe54c0745c0 0x7fe54c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:29.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.784+0000 7fe54be00640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe54c074ee0 0x7fe54c19ac20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe53800e9f0 tx=0x7fe53800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:29.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.784+0000 7fe549600640 1 -- 172.21.15.2:0/3451428623 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53800cd80 con 0x7fe54c074ee0 2024-09-10T08:16:29.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.784+0000 7fe549600640 1 -- 172.21.15.2:0/3451428623 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe538005500 con 0x7fe54c074ee0 2024-09-10T08:16:29.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.784+0000 7fe553c00640 1 -- 172.21.15.2:0/3451428623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c19f980 con 0x7fe54c074ee0 2024-09-10T08:16:29.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.784+0000 7fe549600640 1 -- 172.21.15.2:0/3451428623 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538010690 con 0x7fe54c074ee0 2024-09-10T08:16:29.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.784+0000 7fe553c00640 1 -- 172.21.15.2:0/3451428623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c1a7260 con 0x7fe54c074ee0 2024-09-10T08:16:29.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.788+0000 7fe5567f5740 1 Processor -- start 2024-09-10T08:16:29.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.789+0000 7fe5567f5740 1 -- start start 2024-09-10T08:16:29.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.789+0000 7fe5567f5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fcb90 0x5594fc6fcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.789+0000 7fe5567f5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 0x5594fc707830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.789+0000 7fe549600640 1 -- 172.21.15.2:0/3451428623 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe5380046c0 con 0x7fe54c074ee0 2024-09-10T08:16:29.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.789+0000 7fe5567f5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc70a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.789+0000 7fe5567f5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5594fc6633e0 con 0x5594fc707d70 2024-09-10T08:16:29.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.789+0000 7fe5567f5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5594fc60ad80 con 0x5594fc6fd4b0 2024-09-10T08:16:29.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.789+0000 7fe5567f5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5594fc65f3a0 con 0x5594fc6fcb90 2024-09-10T08:16:29.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.789+0000 7fe551c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fcb90 0x5594fc6fcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.790+0000 7fe551c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fcb90 0x5594fc6fcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36374/0 (socket says 172.21.15.2:36374) 2024-09-10T08:16:29.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.790+0000 7fe551c00640 1 -- 172.21.15.2:0/808629789 learned_addr learned my addr 172.21.15.2:0/808629789 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:29.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.790+0000 7fe54be00640 1 --2- 172.21.15.2:0/808629789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc70a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.790+0000 7fe551200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 0x5594fc707830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.790+0000 7fe551c00640 1 -- 172.21.15.2:0/808629789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 msgr2=0x5594fc707830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:29.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.790+0000 7fe551c00640 1 --2- 172.21.15.2:0/808629789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 0x5594fc707830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.790+0000 7fe551c00640 1 -- 172.21.15.2:0/808629789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 msgr2=0x5594fc70a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:29.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.790+0000 7fe551c00640 1 --2- 172.21.15.2:0/808629789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc70a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.790+0000 7fe551c00640 1 -- 172.21.15.2:0/808629789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594fc629ae0 con 0x5594fc6fcb90 2024-09-10T08:16:29.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.790+0000 7fe551c00640 1 --2- 172.21.15.2:0/808629789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fcb90 0x5594fc6fcf70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe544009ca0 tx=0x7fe5440093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cfa613f0025c5c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:29.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.791+0000 7fe53e000640 1 -- 172.21.15.2:0/808629789 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544048070 con 0x5594fc6fcb90 2024-09-10T08:16:29.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.791+0000 7fe53e000640 1 -- 172.21.15.2:0/808629789 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54402f940 con 0x5594fc6fcb90 2024-09-10T08:16:29.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.791+0000 7fe53e000640 1 -- 172.21.15.2:0/808629789 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544034040 con 0x5594fc6fcb90 2024-09-10T08:16:29.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.791+0000 7fe549600640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5340a2c00 0x7fe5340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.791+0000 7fe549600640 1 -- 172.21.15.2:0/3451428623 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe5380cf7e0 con 0x7fe54c074ee0 2024-09-10T08:16:29.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.791+0000 7fe551200640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5340a2c00 0x7fe5340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.792+0000 7fe5567f5740 1 -- 172.21.15.2:0/808629789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fcb90 msgr2=0x5594fc6fcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:29.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.792+0000 7fe5567f5740 1 --2- 172.21.15.2:0/808629789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fcb90 0x5594fc6fcf70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe544009ca0 tx=0x7fe5440093e0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.792+0000 7fe5567f5740 1 -- 172.21.15.2:0/808629789 shutdown_connections 2024-09-10T08:16:29.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.792+0000 7fe5567f5740 1 --2- 172.21.15.2:0/808629789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc70a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.792+0000 7fe5567f5740 1 --2- 172.21.15.2:0/808629789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 0x5594fc707830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.792+0000 7fe5567f5740 1 --2- 172.21.15.2:0/808629789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fcb90 0x5594fc6fcf70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.792+0000 7fe5567f5740 1 -- 172.21.15.2:0/808629789 >> 172.21.15.2:0/808629789 conn(0x5594fc6f9120 msgr2=0x5594fc6f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:29.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.792+0000 7fe5567f5740 1 -- 172.21.15.2:0/808629789 shutdown_connections 2024-09-10T08:16:29.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.792+0000 7fe5567f5740 1 -- 172.21.15.2:0/808629789 wait complete. 2024-09-10T08:16:29.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.792+0000 7fe551200640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5340a2c00 0x7fe5340a50c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x5594fc6f9f80 tx=0x7fe540005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:29.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.792+0000 7fe5567f5740 1 -- start start 2024-09-10T08:16:29.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.793+0000 7fe5567f5740 1 Processor -- start 2024-09-10T08:16:29.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.793+0000 7fe5567f5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc6fcb90 0x5594fc704ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.793+0000 7fe5567f5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 0x5594fc705220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.793+0000 7fe5567f5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc707d70 0x5594fc705760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.793+0000 7fe5567f5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5594fc70a9e0 con 0x5594fc6fcb90 2024-09-10T08:16:29.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.793+0000 7fe5567f5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5594fc61e0f0 con 0x5594fc6fd4b0 2024-09-10T08:16:29.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.793+0000 7fe5567f5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5594fc70ab60 con 0x5594fc707d70 2024-09-10T08:16:29.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.793+0000 7fe5567f5740 10 client.0 _ll_register_callbacks cb 0x5594fc60f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:29.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.793+0000 7fe551c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc707d70 0x5594fc705760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.793+0000 7fe551200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc6fcb90 0x5594fc704ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.793+0000 7fe551c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc707d70 0x5594fc705760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36380/0 (socket says 172.21.15.2:36380) 2024-09-10T08:16:29.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.794+0000 7fe551c00640 1 -- 172.21.15.2:0/3388969463 learned_addr learned my addr 172.21.15.2:0/3388969463 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:29.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.794+0000 7fe54be00640 1 --2- 172.21.15.2:0/3388969463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 0x5594fc705220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.794+0000 7fe551c00640 1 -- 172.21.15.2:0/3388969463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 msgr2=0x5594fc705220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:29.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.794+0000 7fe551c00640 1 --2- 172.21.15.2:0/3388969463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 0x5594fc705220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.794+0000 7fe551c00640 1 -- 172.21.15.2:0/3388969463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc6fcb90 msgr2=0x5594fc704ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:29.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.794+0000 7fe551c00640 1 --2- 172.21.15.2:0/3388969463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc6fcb90 0x5594fc704ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.794+0000 7fe551c00640 1 -- 172.21.15.2:0/3388969463 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594fc3d43c0 con 0x5594fc707d70 2024-09-10T08:16:29.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.794+0000 7fe551c00640 1 --2- 172.21.15.2:0/3388969463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc707d70 0x5594fc705760 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe544005cc0 tx=0x7fe544003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a5cca447a54d82d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:29.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.794+0000 7fe532a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:16:29.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.795+0000 7fe532a00640 1 -- 172.21.15.2:0/3388969463 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544048030 con 0x5594fc707d70 2024-09-10T08:16:29.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.795+0000 7fe5567f5740 1 -- 172.21.15.2:0/3388969463 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe54402f940 con 0x5594fc707d70 2024-09-10T08:16:29.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.795+0000 7fe532a00640 1 -- 172.21.15.2:0/3388969463 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe544040040 con 0x5594fc707d70 2024-09-10T08:16:29.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.795+0000 7fe532a00640 1 -- 172.21.15.2:0/3388969463 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe544034050 con 0x5594fc707d70 2024-09-10T08:16:29.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.795+0000 7fe5567f5740 10 client.27595 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:29.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.795+0000 7fe5567f5740 1 -- 172.21.15.2:0/3388969463 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594fc70a9e0 con 0x5594fc707d70 2024-09-10T08:16:29.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.795+0000 7fe532a00640 1 -- 172.21.15.2:0/3388969463 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54402f940 con 0x5594fc707d70 2024-09-10T08:16:29.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.795+0000 7fe532a00640 1 -- 172.21.15.2:0/3388969463 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe54403a430 con 0x5594fc707d70 2024-09-10T08:16:29.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.795+0000 7fe5567f5740 10 client.27595 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:29.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.796+0000 7fe5567f5740 10 client.27595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:29.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.796+0000 7fe5567f5740 20 client.27595 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:29.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.796+0000 7fe5567f5740 1 --2- 172.21.15.2:0/3388969463 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594fc7178c0 0x5594fc719cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.796+0000 7fe5567f5740 4 client.27595 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:29.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.796+0000 7fe5567f5740 1 -- 172.21.15.2:0/3388969463 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594fc670950 con 0x5594fc7178c0 2024-09-10T08:16:29.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.796+0000 7fe551200640 1 --2- 172.21.15.2:0/3388969463 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594fc7178c0 0x5594fc719cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.796+0000 7fe551200640 1 --2- 172.21.15.2:0/3388969463 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594fc7178c0 0x5594fc719cb0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10a60812c6206139 server_cookie=de6f1cab27cd3f2 in_seq=0 out_seq=0 2024-09-10T08:16:29.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.796+0000 7fe532a00640 10 client.27595 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:29.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.797+0000 7fe532a00640 1 -- 172.21.15.2:0/3388969463 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594fc670950 con 0x5594fc7178c0 2024-09-10T08:16:29.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.797+0000 7fe532a00640 10 client.27595 handle_command_reply: tid=0 2024-09-10T08:16:29.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.798+0000 7fe5567f5740 1 client.27595 shutdown 2024-09-10T08:16:29.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.798+0000 7fe5567f5740 1 -- 172.21.15.2:0/3388969463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc707d70 msgr2=0x5594fc705760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:29.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.798+0000 7fe5567f5740 1 --2- 172.21.15.2:0/3388969463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc707d70 0x5594fc705760 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe544005cc0 tx=0x7fe544003970 comp rx=0 tx=0).stop 2024-09-10T08:16:29.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.798+0000 7fe5567f5740 1 -- 172.21.15.2:0/3388969463 shutdown_connections 2024-09-10T08:16:29.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.798+0000 7fe5567f5740 1 --2- 172.21.15.2:0/3388969463 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594fc7178c0 0x5594fc719cb0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.798+0000 7fe5567f5740 1 --2- 172.21.15.2:0/3388969463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc707d70 0x5594fc705760 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.798+0000 7fe5567f5740 1 --2- 172.21.15.2:0/3388969463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 0x5594fc705220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.798+0000 7fe532a00640 0 client.27595 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:29.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.798+0000 7fe5567f5740 1 --2- 172.21.15.2:0/3388969463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc6fcb90 0x5594fc704ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.798+0000 7fe5567f5740 1 -- 172.21.15.2:0/3388969463 >> 172.21.15.2:0/3388969463 conn(0x5594fc6f9120 msgr2=0x5594fc6f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:29.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.798+0000 7fe5567f5740 1 -- 172.21.15.2:0/3388969463 shutdown_connections 2024-09-10T08:16:29.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.798+0000 7fe5567f5740 1 -- 172.21.15.2:0/3388969463 wait complete. 2024-09-10T08:16:29.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.798+0000 7fe5567f5740 20 client.27595 trim_cache size 0 max 16384 2024-09-10T08:16:29.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.816+0000 7fe5567f5740 1 Processor -- start 2024-09-10T08:16:29.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.816+0000 7fe5567f5740 1 -- start start 2024-09-10T08:16:29.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.816+0000 7fe5567f5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fcb90 0x5594fc6ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe5567f5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 0x5594fc71f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe5567f5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc71f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe5567f5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1b3760 con 0x5594fc707d70 2024-09-10T08:16:29.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe5567f5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1b1030 con 0x5594fc6fd4b0 2024-09-10T08:16:29.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe5567f5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1aee80 con 0x5594fc6fcb90 2024-09-10T08:16:29.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe551c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc71f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe551c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc71f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37510/0 (socket says 172.21.15.2:37510) 2024-09-10T08:16:29.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe551c00640 1 -- 172.21.15.2:0/1476577961 learned_addr learned my addr 172.21.15.2:0/1476577961 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:29.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe551200640 1 --2- 172.21.15.2:0/1476577961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fcb90 0x5594fc6ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe54be00640 1 --2- 172.21.15.2:0/1476577961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 0x5594fc71f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe551c00640 1 -- 172.21.15.2:0/1476577961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fcb90 msgr2=0x5594fc6ff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:29.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe551c00640 1 --2- 172.21.15.2:0/1476577961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fcb90 0x5594fc6ff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe551c00640 1 -- 172.21.15.2:0/1476577961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 msgr2=0x5594fc71f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:29.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe551c00640 1 --2- 172.21.15.2:0/1476577961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 0x5594fc71f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe551c00640 1 -- 172.21.15.2:0/1476577961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594fc6fc9b0 con 0x5594fc707d70 2024-09-10T08:16:29.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.817+0000 7fe551200640 1 --2- 172.21.15.2:0/1476577961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fcb90 0x5594fc6ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:29.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.818+0000 7fe551c00640 1 --2- 172.21.15.2:0/1476577961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc71f910 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe54403a800 tx=0x7fe544005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ad57c026065619 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:29.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.818+0000 7fe53d600640 1 -- 172.21.15.2:0/1476577961 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544048040 con 0x5594fc707d70 2024-09-10T08:16:29.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.818+0000 7fe53d600640 1 -- 172.21.15.2:0/1476577961 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54404a070 con 0x5594fc707d70 2024-09-10T08:16:29.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.818+0000 7fe53d600640 1 -- 172.21.15.2:0/1476577961 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54403a610 con 0x5594fc707d70 2024-09-10T08:16:29.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.819+0000 7fe5567f5740 1 -- 172.21.15.2:0/1476577961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 msgr2=0x5594fc71f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:29.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.819+0000 7fe5567f5740 1 --2- 172.21.15.2:0/1476577961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc71f910 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe54403a800 tx=0x7fe544005bd0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.819+0000 7fe5567f5740 1 -- 172.21.15.2:0/1476577961 shutdown_connections 2024-09-10T08:16:29.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.819+0000 7fe5567f5740 1 --2- 172.21.15.2:0/1476577961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc71f910 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.819+0000 7fe5567f5740 1 --2- 172.21.15.2:0/1476577961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fd4b0 0x5594fc71f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.819+0000 7fe5567f5740 1 --2- 172.21.15.2:0/1476577961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fcb90 0x5594fc6ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.819+0000 7fe5567f5740 1 -- 172.21.15.2:0/1476577961 >> 172.21.15.2:0/1476577961 conn(0x5594fc6f9120 msgr2=0x5594fc706d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:29.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.819+0000 7fe5567f5740 1 -- 172.21.15.2:0/1476577961 shutdown_connections 2024-09-10T08:16:29.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.819+0000 7fe5567f5740 1 -- 172.21.15.2:0/1476577961 wait complete. 2024-09-10T08:16:29.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.819+0000 7fe5567f5740 1 -- start start 2024-09-10T08:16:29.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.819+0000 7fe5567f5740 1 Processor -- start 2024-09-10T08:16:29.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.820+0000 7fe5567f5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fcb90 0x5594fc7102d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.820+0000 7fe5567f5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fd4b0 0x5594fc701850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.820+0000 7fe5567f5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc701d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.820+0000 7fe5567f5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1af4f0 con 0x5594fc707d70 2024-09-10T08:16:29.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.820+0000 7fe5567f5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1b0960 con 0x5594fc6fcb90 2024-09-10T08:16:29.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.820+0000 7fe5567f5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1afbc0 con 0x5594fc6fd4b0 2024-09-10T08:16:29.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.820+0000 7fe5567f5740 10 client.0 _ll_register_callbacks cb 0x7fe540009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:29.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.820+0000 7fe551c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc701d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.820+0000 7fe551c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc701d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37516/0 (socket says 172.21.15.2:37516) 2024-09-10T08:16:29.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.820+0000 7fe551c00640 1 -- 172.21.15.2:0/3652170124 learned_addr learned my addr 172.21.15.2:0/3652170124 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:29.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.820+0000 7fe551200640 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fcb90 0x5594fc7102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.820+0000 7fe54be00640 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fd4b0 0x5594fc701850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.821+0000 7fe551c00640 1 -- 172.21.15.2:0/3652170124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fd4b0 msgr2=0x5594fc701850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:29.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.821+0000 7fe551c00640 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fd4b0 0x5594fc701850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.821+0000 7fe551c00640 1 -- 172.21.15.2:0/3652170124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fcb90 msgr2=0x5594fc7102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:29.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.821+0000 7fe551c00640 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fcb90 0x5594fc7102d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.821+0000 7fe551c00640 1 -- 172.21.15.2:0/3652170124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594fc702350 con 0x5594fc707d70 2024-09-10T08:16:29.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.821+0000 7fe551200640 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fcb90 0x5594fc7102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:29.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.821+0000 7fe551c00640 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc701d90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe544005e30 tx=0x7fe544009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b945061260120414 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.821+0000 7fe532a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:16:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.821+0000 7fe532a00640 1 -- 172.21.15.2:0/3652170124 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544048040 con 0x5594fc707d70 2024-09-10T08:16:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.821+0000 7fe5567f5740 1 -- 172.21.15.2:0/3652170124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe54404a070 con 0x5594fc707d70 2024-09-10T08:16:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.821+0000 7fe532a00640 1 -- 172.21.15.2:0/3652170124 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe54403a5b0 con 0x5594fc707d70 2024-09-10T08:16:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.822+0000 7fe532a00640 1 -- 172.21.15.2:0/3652170124 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe54406c050 con 0x5594fc707d70 2024-09-10T08:16:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.822+0000 7fe5567f5740 10 client.17628 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.822+0000 7fe5567f5740 1 -- 172.21.15.2:0/3652170124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594fc782180 con 0x5594fc707d70 2024-09-10T08:16:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.822+0000 7fe532a00640 1 -- 172.21.15.2:0/3652170124 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54404a070 con 0x5594fc707d70 2024-09-10T08:16:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.822+0000 7fe532a00640 1 -- 172.21.15.2:0/3652170124 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe54405b070 con 0x5594fc707d70 2024-09-10T08:16:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.822+0000 7fe5567f5740 10 client.17628 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.822+0000 7fe5567f5740 10 client.17628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.822+0000 7fe5567f5740 20 client.17628 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.822+0000 7fe5567f5740 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594fc782790 0x5594fc784bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:29.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.822+0000 7fe5567f5740 4 client.17628 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:16:29.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.822+0000 7fe5567f5740 1 -- 172.21.15.2:0/3652170124 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594fc66ae60 con 0x5594fc782790 2024-09-10T08:16:29.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.823+0000 7fe551200640 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594fc782790 0x5594fc784bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:29.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.823+0000 7fe551200640 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594fc782790 0x5594fc784bb0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9c16059e63757de server_cookie=6161fcddcaba1ca9 in_seq=0 out_seq=0 2024-09-10T08:16:29.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.823+0000 7fe532a00640 10 client.17628 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:29.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.823+0000 7fe532a00640 1 -- 172.21.15.2:0/3652170124 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5594fc66ae60 con 0x5594fc782790 2024-09-10T08:16:29.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.823+0000 7fe532a00640 10 client.17628 handle_command_reply: tid=0 2024-09-10T08:16:29.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.823+0000 7fe5567f5740 1 client.17628 shutdown 2024-09-10T08:16:29.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.823+0000 7fe5567f5740 1 -- 172.21.15.2:0/3652170124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 msgr2=0x5594fc701d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:29.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.823+0000 7fe5567f5740 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc701d90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe544005e30 tx=0x7fe544009800 comp rx=0 tx=0).stop 2024-09-10T08:16:29.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.824+0000 7fe5567f5740 1 -- 172.21.15.2:0/3652170124 shutdown_connections 2024-09-10T08:16:29.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.824+0000 7fe5567f5740 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594fc782790 0x5594fc784bb0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.824+0000 7fe5567f5740 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594fc707d70 0x5594fc701d90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.824+0000 7fe5567f5740 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594fc6fd4b0 0x5594fc701850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.824+0000 7fe5567f5740 1 --2- 172.21.15.2:0/3652170124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594fc6fcb90 0x5594fc7102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.824+0000 7fe5567f5740 1 -- 172.21.15.2:0/3652170124 >> 172.21.15.2:0/3652170124 conn(0x5594fc6f9120 msgr2=0x5594fc704c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:29.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.824+0000 7fe532a00640 0 client.17628 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:29.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.824+0000 7fe5567f5740 1 -- 172.21.15.2:0/3652170124 shutdown_connections 2024-09-10T08:16:29.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.824+0000 7fe5567f5740 1 -- 172.21.15.2:0/3652170124 wait complete. 2024-09-10T08:16:29.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.824+0000 7fe5567f5740 20 client.17628 trim_cache size 0 max 16384 2024-09-10T08:16:29.828 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:16:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:16:29.828 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:16:29.829 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:16:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.825+0000 7fe553c00640 1 -- 172.21.15.2:0/3451428623 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5340a2c00 msgr2=0x7fe5340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.825+0000 7fe553c00640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5340a2c00 0x7fe5340a50c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x5594fc6f9f80 tx=0x7fe540005f90 comp rx=0 tx=0).stop 2024-09-10T08:16:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.825+0000 7fe553c00640 1 -- 172.21.15.2:0/3451428623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe54c074ee0 msgr2=0x7fe54c19ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.825+0000 7fe553c00640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe54c074ee0 0x7fe54c19ac20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe53800e9f0 tx=0x7fe53800edf0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.825+0000 7fe553c00640 1 -- 172.21.15.2:0/3451428623 shutdown_connections 2024-09-10T08:16:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.825+0000 7fe553c00640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe5340a2c00 0x7fe5340a50c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.825+0000 7fe553c00640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe54c10e430 0x7fe54c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.825+0000 7fe553c00640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe54c074ee0 0x7fe54c19ac20 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.825+0000 7fe553c00640 1 --2- 172.21.15.2:0/3451428623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe54c0745c0 0x7fe54c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.825+0000 7fe553c00640 1 -- 172.21.15.2:0/3451428623 >> 172.21.15.2:0/3451428623 conn(0x7fe54c0fd8b0 msgr2=0x7fe54c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.825+0000 7fe553c00640 1 -- 172.21.15.2:0/3451428623 shutdown_connections 2024-09-10T08:16:29.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:29.825+0000 7fe553c00640 1 -- 172.21.15.2:0/3451428623 wait complete. 2024-09-10T08:16:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:29 smithi155 ceph-mon[35405]: pgmap v637: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:16:29.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:29 smithi107 ceph-mon[35048]: pgmap v637: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:16:30.743 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:16:30.743 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:16:31.122 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:31.518 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:31 smithi002 ceph-mon[30563]: pgmap v638: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:16:31.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.775+0000 7f2f16800640 1 -- 172.21.15.2:0/3993904169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f10100eb0 msgr2=0x7f2f10106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:31.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.775+0000 7f2f16800640 1 --2- 172.21.15.2:0/3993904169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f10100eb0 0x7f2f10106840 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009900 tx=0x7f2ef802a410 comp rx=0 tx=0).stop 2024-09-10T08:16:31.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.776+0000 7f2f16800640 1 -- 172.21.15.2:0/3993904169 shutdown_connections 2024-09-10T08:16:31.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.776+0000 7f2f16800640 1 --2- 172.21.15.2:0/3993904169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f10106f70 0x7f2f1010eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:31.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.776+0000 7f2f16800640 1 --2- 172.21.15.2:0/3993904169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f10100eb0 0x7f2f10106840 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:31.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.776+0000 7f2f16800640 1 --2- 172.21.15.2:0/3993904169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f10100500 0x7f2f101008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:31.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.776+0000 7f2f16800640 1 -- 172.21.15.2:0/3993904169 >> 172.21.15.2:0/3993904169 conn(0x7f2f100fc5c0 msgr2=0x7f2f100fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:31.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.776+0000 7f2f16800640 1 -- 172.21.15.2:0/3993904169 shutdown_connections 2024-09-10T08:16:31.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.776+0000 7f2f16800640 1 -- 172.21.15.2:0/3993904169 wait complete. 2024-09-10T08:16:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.777+0000 7f2f16800640 1 Processor -- start 2024-09-10T08:16:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.777+0000 7f2f16800640 1 -- start start 2024-09-10T08:16:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.777+0000 7f2f16800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f10100500 0x7f2f10101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.777+0000 7f2f16800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f10100eb0 0x7f2f101019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.777+0000 7f2f16800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f10106f70 0x7f2f10101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.777+0000 7f2f16800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2f10111fe0 con 0x7f2f10106f70 2024-09-10T08:16:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.777+0000 7f2f16800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2f10111e60 con 0x7f2f10100500 2024-09-10T08:16:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.777+0000 7f2f16800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2f10112160 con 0x7f2f10100eb0 2024-09-10T08:16:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.778+0000 7f2f0f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f10100500 0x7f2f10101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.778+0000 7f2f0f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f10100500 0x7f2f10101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42018/0 (socket says 172.21.15.2:42018) 2024-09-10T08:16:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.778+0000 7f2f0f400640 1 -- 172.21.15.2:0/1603388203 learned_addr learned my addr 172.21.15.2:0/1603388203 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.778+0000 7f2f0fe00640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f10106f70 0x7f2f10101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.778+0000 7f2f06a00640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f10100eb0 0x7f2f101019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.778+0000 7f2f0f400640 1 -- 172.21.15.2:0/1603388203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f10100eb0 msgr2=0x7f2f101019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.778+0000 7f2f0f400640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f10100eb0 0x7f2f101019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.778+0000 7f2f0f400640 1 -- 172.21.15.2:0/1603388203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f10106f70 msgr2=0x7f2f10101f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.778+0000 7f2f0f400640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f10106f70 0x7f2f10101f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.778+0000 7f2f0f400640 1 -- 172.21.15.2:0/1603388203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f101a6950 con 0x7f2f10100500 2024-09-10T08:16:31.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.778+0000 7f2f0f400640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f10100500 0x7f2f10101480 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000e9c0 tx=0x7f2f0000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:31.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.779+0000 7f2f0cc00640 1 -- 172.21.15.2:0/1603388203 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000cd80 con 0x7f2f10100500 2024-09-10T08:16:31.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.779+0000 7f2f16800640 1 -- 172.21.15.2:0/1603388203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f101a6c40 con 0x7f2f10100500 2024-09-10T08:16:31.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.779+0000 7f2f0cc00640 1 -- 172.21.15.2:0/1603388203 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f000035f0 con 0x7f2f10100500 2024-09-10T08:16:31.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.779+0000 7f2f0cc00640 1 -- 172.21.15.2:0/1603388203 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00010690 con 0x7f2f10100500 2024-09-10T08:16:31.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.779+0000 7f2f16800640 1 -- 172.21.15.2:0/1603388203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f101a7100 con 0x7f2f10100500 2024-09-10T08:16:31.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.780+0000 7f2f16800640 1 -- 172.21.15.2:0/1603388203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f101008e0 con 0x7f2f10100500 2024-09-10T08:16:31.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.782+0000 7f2f0cc00640 1 -- 172.21.15.2:0/1603388203 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2f000037a0 con 0x7f2f10100500 2024-09-10T08:16:31.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.783+0000 7f2f0cc00640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2ef40a2b80 0x7f2ef40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:31.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.783+0000 7f2f0cc00640 1 -- 172.21.15.2:0/1603388203 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2f000cf880 con 0x7f2f10100500 2024-09-10T08:16:31.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.784+0000 7f2f06a00640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2ef40a2b80 0x7f2ef40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:31.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.784+0000 7f2f06a00640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2ef40a2b80 0x7f2ef40a5040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2ef802aa00 tx=0x7f2ef8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:31.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:31.785+0000 7f2f0cc00640 1 -- 172.21.15.2:0/1603388203 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f0009c390 con 0x7f2f10100500 2024-09-10T08:16:31.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:31 smithi155 ceph-mon[35405]: pgmap v638: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:16:31.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:31 smithi107 ceph-mon[35048]: pgmap v638: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:16:32.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.009+0000 7f2f16800640 1 -- 172.21.15.2:0/1603388203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f1002d7c0 con 0x7f2f10100500 2024-09-10T08:16:32.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.010+0000 7f2f0cc00640 1 -- 172.21.15.2:0/1603388203 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f2f000b3ea0 con 0x7f2f10100500 2024-09-10T08:16:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:16:32.010 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:16:32.010 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:16:32.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.012+0000 7f2f16800640 1 -- 172.21.15.2:0/1603388203 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2ef40a2b80 msgr2=0x7f2ef40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:32.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.012+0000 7f2f16800640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2ef40a2b80 0x7f2ef40a5040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2ef802aa00 tx=0x7f2ef8000bc0 comp rx=0 tx=0).stop 2024-09-10T08:16:32.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.012+0000 7f2f16800640 1 -- 172.21.15.2:0/1603388203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f10100500 msgr2=0x7f2f10101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:32.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.012+0000 7f2f16800640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f10100500 0x7f2f10101480 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000e9c0 tx=0x7f2f0000ee80 comp rx=0 tx=0).stop 2024-09-10T08:16:32.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.012+0000 7f2f16800640 1 -- 172.21.15.2:0/1603388203 shutdown_connections 2024-09-10T08:16:32.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.012+0000 7f2f16800640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f10106f70 0x7f2f10101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:32.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.012+0000 7f2f16800640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2ef40a2b80 0x7f2ef40a5040 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:32.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.012+0000 7f2f16800640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f10100eb0 0x7f2f101019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:32.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.012+0000 7f2f16800640 1 --2- 172.21.15.2:0/1603388203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f10100500 0x7f2f10101480 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:32.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.012+0000 7f2f16800640 1 -- 172.21.15.2:0/1603388203 >> 172.21.15.2:0/1603388203 conn(0x7f2f100fc5c0 msgr2=0x7f2f101052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:32.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.012+0000 7f2f16800640 1 -- 172.21.15.2:0/1603388203 shutdown_connections 2024-09-10T08:16:32.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:32.012+0000 7f2f16800640 1 -- 172.21.15.2:0/1603388203 wait complete. 2024-09-10T08:16:32.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:32 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1603388203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:16:32.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:32 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1603388203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:16:32.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:32 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1603388203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:16:32.996 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:16:33.374 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:33 smithi002 ceph-mon[30563]: pgmap v639: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:16:33.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:33 smithi155 ceph-mon[35405]: pgmap v639: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:16:33.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:33 smithi107 ceph-mon[35048]: pgmap v639: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:16:34.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.000+0000 7fabd6c00640 1 -- 172.21.15.2:0/754621187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd0103e30 msgr2=0x7fabd0104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:34.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.000+0000 7fabd6c00640 1 --2- 172.21.15.2:0/754621187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd0103e30 0x7fabd0104210 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009900 tx=0x7fabc402a410 comp rx=0 tx=0).stop 2024-09-10T08:16:34.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.000+0000 7fabd6c00640 1 -- 172.21.15.2:0/754621187 shutdown_connections 2024-09-10T08:16:34.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.000+0000 7fabd6c00640 1 --2- 172.21.15.2:0/754621187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd0108f60 0x7fabd0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.000+0000 7fabd6c00640 1 --2- 172.21.15.2:0/754621187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabd01047e0 0x7fabd0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.000+0000 7fabd6c00640 1 --2- 172.21.15.2:0/754621187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd0103e30 0x7fabd0104210 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.000+0000 7fabd6c00640 1 -- 172.21.15.2:0/754621187 >> 172.21.15.2:0/754621187 conn(0x7fabd00ff910 msgr2=0x7fabd0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.000+0000 7fabd6c00640 1 -- 172.21.15.2:0/754621187 shutdown_connections 2024-09-10T08:16:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.001+0000 7fabd6c00640 1 -- 172.21.15.2:0/754621187 wait complete. 2024-09-10T08:16:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.001+0000 7fabd6c00640 1 Processor -- start 2024-09-10T08:16:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.002+0000 7fabd6c00640 1 -- start start 2024-09-10T08:16:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.002+0000 7fabd6c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabd0103e30 0x7fabd019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.002+0000 7fabd6c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd01047e0 0x7fabd019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.002+0000 7fabd6c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd0108f60 0x7fabd019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.002+0000 7fabd6c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fabd0114010 con 0x7fabd0108f60 2024-09-10T08:16:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.002+0000 7fabd6c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fabd0113e90 con 0x7fabd01047e0 2024-09-10T08:16:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.002+0000 7fabd6c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fabd0114190 con 0x7fabd0103e30 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.002+0000 7fabd6200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd0108f60 0x7fabd019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.002+0000 7fabd5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabd0103e30 0x7fabd019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.002+0000 7fabd5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabd0103e30 0x7fabd019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36454/0 (socket says 172.21.15.2:36454) 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.002+0000 7fabd5800640 1 -- 172.21.15.2:0/3504565812 learned_addr learned my addr 172.21.15.2:0/3504565812 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.002+0000 7fabd4e00640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd01047e0 0x7fabd019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.003+0000 7fabd5800640 1 -- 172.21.15.2:0/3504565812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd01047e0 msgr2=0x7fabd019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.003+0000 7fabd5800640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd01047e0 0x7fabd019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.003+0000 7fabd5800640 1 -- 172.21.15.2:0/3504565812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd0108f60 msgr2=0x7fabd019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.003+0000 7fabd5800640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd0108f60 0x7fabd019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.003+0000 7fabd5800640 1 -- 172.21.15.2:0/3504565812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd019f460 con 0x7fabd0103e30 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.003+0000 7fabd6200640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd0108f60 0x7fabd019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.003+0000 7fabd5800640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabd0103e30 0x7fabd019a4b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fabc4004770 tx=0x7fabc4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.003+0000 7fabce000640 1 -- 172.21.15.2:0/3504565812 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc403f070 con 0x7fabd0103e30 2024-09-10T08:16:34.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.003+0000 7fabd6c00640 1 -- 172.21.15.2:0/3504565812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd019f6f0 con 0x7fabd0103e30 2024-09-10T08:16:34.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.003+0000 7fabce000640 1 -- 172.21.15.2:0/3504565812 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc4031ca0 con 0x7fabd0103e30 2024-09-10T08:16:34.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.003+0000 7fabce000640 1 -- 172.21.15.2:0/3504565812 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4035030 con 0x7fabd0103e30 2024-09-10T08:16:34.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.004+0000 7fabd6c00640 1 -- 172.21.15.2:0/3504565812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd01a7030 con 0x7fabd0103e30 2024-09-10T08:16:34.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.006+0000 7fabdae02740 1 Processor -- start 2024-09-10T08:16:34.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.006+0000 7fabdae02740 1 -- start start 2024-09-10T08:16:34.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.007+0000 7fabdae02740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebf2e40 0x55df5ebf3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.007+0000 7fabdae02740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf3760 0x55df5ebfdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.007+0000 7fabdae02740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebfe020 0x55df5ec00410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.007+0000 7fabdae02740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55df5eb593e0 con 0x55df5ebfe020 2024-09-10T08:16:34.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.007+0000 7fabdae02740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55df5eb00d80 con 0x55df5ebf3760 2024-09-10T08:16:34.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.007+0000 7fabdae02740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55df5eb553a0 con 0x55df5ebf2e40 2024-09-10T08:16:34.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.007+0000 7fabce000640 1 -- 172.21.15.2:0/3504565812 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fabc4042070 con 0x7fabd0103e30 2024-09-10T08:16:34.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.007+0000 7fabd6200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebf2e40 0x55df5ebf3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.007+0000 7fabd4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebfe020 0x55df5ec00410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.007+0000 7fabd6200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebf2e40 0x55df5ebf3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36462/0 (socket says 172.21.15.2:36462) 2024-09-10T08:16:34.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.007+0000 7fabd6200640 1 -- 172.21.15.2:0/2933292434 learned_addr learned my addr 172.21.15.2:0/2933292434 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:34.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.007+0000 7fabd5800640 1 --2- 172.21.15.2:0/2933292434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf3760 0x55df5ebfdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.008+0000 7fabd6200640 1 -- 172.21.15.2:0/2933292434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf3760 msgr2=0x55df5ebfdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:34.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.008+0000 7fabd6200640 1 --2- 172.21.15.2:0/2933292434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf3760 0x55df5ebfdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.008+0000 7fabd6200640 1 -- 172.21.15.2:0/2933292434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebfe020 msgr2=0x55df5ec00410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:34.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.008+0000 7fabd6200640 1 --2- 172.21.15.2:0/2933292434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebfe020 0x55df5ec00410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.008+0000 7fabd6200640 1 -- 172.21.15.2:0/2933292434 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df5eb1fae0 con 0x55df5ebf2e40 2024-09-10T08:16:34.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.008+0000 7fabd5800640 1 --2- 172.21.15.2:0/2933292434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf3760 0x55df5ebfdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:16:34.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.008+0000 7fabd4e00640 1 --2- 172.21.15.2:0/2933292434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebfe020 0x55df5ec00410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:34.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.008+0000 7fabd6200640 1 --2- 172.21.15.2:0/2933292434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebf2e40 0x55df5ebf3220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fabc8011790 tx=0x7fabc8011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81e3e5312a536a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:34.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.008+0000 7fabc2a00640 1 -- 172.21.15.2:0/2933292434 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc8004d20 con 0x55df5ebf2e40 2024-09-10T08:16:34.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.008+0000 7fabc2a00640 1 -- 172.21.15.2:0/2933292434 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc8003970 con 0x55df5ebf2e40 2024-09-10T08:16:34.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.008+0000 7fabc2a00640 1 -- 172.21.15.2:0/2933292434 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc8004620 con 0x55df5ebf2e40 2024-09-10T08:16:34.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.009+0000 7fabce000640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fabb80a2b80 0x7fabb80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.009+0000 7fabce000640 1 -- 172.21.15.2:0/3504565812 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fabc40efa50 con 0x7fabd0103e30 2024-09-10T08:16:34.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.009+0000 7fabd5800640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fabb80a2b80 0x7fabb80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.009+0000 7fabdae02740 1 -- 172.21.15.2:0/2933292434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebf2e40 msgr2=0x55df5ebf3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.009+0000 7fabdae02740 1 --2- 172.21.15.2:0/2933292434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebf2e40 0x55df5ebf3220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fabc8011790 tx=0x7fabc8011c50 comp rx=0 tx=0).stop 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.009+0000 7fabdae02740 1 -- 172.21.15.2:0/2933292434 shutdown_connections 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.009+0000 7fabdae02740 1 --2- 172.21.15.2:0/2933292434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebfe020 0x55df5ec00410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.009+0000 7fabdae02740 1 --2- 172.21.15.2:0/2933292434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf3760 0x55df5ebfdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.009+0000 7fabdae02740 1 --2- 172.21.15.2:0/2933292434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebf2e40 0x55df5ebf3220 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.009+0000 7fabdae02740 1 -- 172.21.15.2:0/2933292434 >> 172.21.15.2:0/2933292434 conn(0x55df5ebef3d0 msgr2=0x55df5ebef7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.009+0000 7fabdae02740 1 -- 172.21.15.2:0/2933292434 shutdown_connections 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.009+0000 7fabdae02740 1 -- 172.21.15.2:0/2933292434 wait complete. 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.010+0000 7fabd5800640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fabb80a2b80 0x7fabb80a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fabc40b8830 tx=0x7fabc4005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.010+0000 7fabdae02740 1 -- start start 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.010+0000 7fabdae02740 1 Processor -- start 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabdae02740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf2e40 0x55df5ebfaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabdae02740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf3760 0x55df5ebfb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabdae02740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebfe020 0x55df5ebfb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabdae02740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55df5ec00bf0 con 0x55df5ebf3760 2024-09-10T08:16:34.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabdae02740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55df5eb140f0 con 0x55df5ebf2e40 2024-09-10T08:16:34.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabdae02740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55df5ec00d70 con 0x55df5ebfe020 2024-09-10T08:16:34.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabdae02740 10 client.0 _ll_register_callbacks cb 0x55df5eafbbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:34.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabd6200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf2e40 0x55df5ebfaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabd4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf3760 0x55df5ebfb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabd6200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf2e40 0x55df5ebfaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42048/0 (socket says 172.21.15.2:42048) 2024-09-10T08:16:34.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabd6200640 1 -- 172.21.15.2:0/3521831665 learned_addr learned my addr 172.21.15.2:0/3521831665 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:34.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabd6200640 1 --2- 172.21.15.2:0/3521831665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebfe020 0x55df5ebfb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabd4e00640 1 -- 172.21.15.2:0/3521831665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebfe020 msgr2=0x55df5ebfb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:34.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabd4e00640 1 --2- 172.21.15.2:0/3521831665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebfe020 0x55df5ebfb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabd4e00640 1 -- 172.21.15.2:0/3521831665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf2e40 msgr2=0x55df5ebfaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:34.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabd4e00640 1 --2- 172.21.15.2:0/3521831665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf2e40 0x55df5ebfaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabd4e00640 1 -- 172.21.15.2:0/3521831665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df5e8ca3c0 con 0x55df5ebf3760 2024-09-10T08:16:34.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.011+0000 7fabd6200640 1 --2- 172.21.15.2:0/3521831665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf2e40 0x55df5ebfaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:34.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.012+0000 7fabd4e00640 1 --2- 172.21.15.2:0/3521831665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf3760 0x55df5ebfb420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0126d0 tx=0x7fabbc012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20b5f9ea93b7951 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:34.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.012+0000 7fabb7400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:16:34.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.012+0000 7fabb7400640 1 -- 172.21.15.2:0/3521831665 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc004030 con 0x55df5ebf3760 2024-09-10T08:16:34.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.012+0000 7fabb7400640 1 -- 172.21.15.2:0/3521831665 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fabbc0041d0 con 0x55df5ebf3760 2024-09-10T08:16:34.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.012+0000 7fabdae02740 1 -- 172.21.15.2:0/3521831665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabc8003970 con 0x55df5ebf3760 2024-09-10T08:16:34.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.012+0000 7fabb7400640 1 -- 172.21.15.2:0/3521831665 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fabbc0043d0 con 0x55df5ebf3760 2024-09-10T08:16:34.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.012+0000 7fabdae02740 10 client.17646 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:34.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.012+0000 7fabdae02740 1 -- 172.21.15.2:0/3521831665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55df5ec0cbd0 con 0x55df5ebf3760 2024-09-10T08:16:34.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.013+0000 7fabb7400640 1 -- 172.21.15.2:0/3521831665 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc8003970 con 0x55df5ebf3760 2024-09-10T08:16:34.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.013+0000 7fabb7400640 1 -- 172.21.15.2:0/3521831665 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fabbc003bc0 con 0x55df5ebf3760 2024-09-10T08:16:34.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.013+0000 7fabdae02740 10 client.17646 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.013+0000 7fabdae02740 10 client.17646 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:16:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.013+0000 7fabdae02740 20 client.17646 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.013+0000 7fabdae02740 1 --2- 172.21.15.2:0/3521831665 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55df5ec0e090 0x55df5ec104c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.013+0000 7fabdae02740 4 client.17646 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.013+0000 7fabdae02740 1 -- 172.21.15.2:0/3521831665 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df5eb66950 con 0x55df5ec0e090 2024-09-10T08:16:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.013+0000 7fabd6200640 1 --2- 172.21.15.2:0/3521831665 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55df5ec0e090 0x55df5ec104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.014+0000 7fabd6200640 1 --2- 172.21.15.2:0/3521831665 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55df5ec0e090 0x55df5ec104c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66250322774f407e server_cookie=af03ad1d78a8e3bd in_seq=0 out_seq=0 2024-09-10T08:16:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.014+0000 7fabb7400640 10 client.17646 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:16:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabb7400640 1 -- 172.21.15.2:0/3521831665 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df5eb66950 con 0x55df5ec0e090 2024-09-10T08:16:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabb7400640 10 client.17646 handle_command_reply: tid=0 2024-09-10T08:16:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabdae02740 1 client.17646 shutdown 2024-09-10T08:16:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabdae02740 1 -- 172.21.15.2:0/3521831665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf3760 msgr2=0x55df5ebfb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabdae02740 1 --2- 172.21.15.2:0/3521831665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf3760 0x55df5ebfb420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0126d0 tx=0x7fabbc012b90 comp rx=0 tx=0).stop 2024-09-10T08:16:34.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabdae02740 1 -- 172.21.15.2:0/3521831665 shutdown_connections 2024-09-10T08:16:34.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabdae02740 1 --2- 172.21.15.2:0/3521831665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebfe020 0x55df5ebfb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabdae02740 1 --2- 172.21.15.2:0/3521831665 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55df5ec0e090 0x55df5ec104c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabdae02740 1 --2- 172.21.15.2:0/3521831665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf3760 0x55df5ebfb420 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabdae02740 1 --2- 172.21.15.2:0/3521831665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf2e40 0x55df5ebfaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabb7400640 0 client.17646 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:16:34.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabdae02740 1 -- 172.21.15.2:0/3521831665 >> 172.21.15.2:0/3521831665 conn(0x55df5ebef3d0 msgr2=0x55df5ebf50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:34.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabdae02740 1 -- 172.21.15.2:0/3521831665 shutdown_connections 2024-09-10T08:16:34.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabdae02740 1 -- 172.21.15.2:0/3521831665 wait complete. 2024-09-10T08:16:34.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.015+0000 7fabdae02740 20 client.17646 trim_cache size 0 max 16384 2024-09-10T08:16:34.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.029+0000 7fabdae02740 1 Processor -- start 2024-09-10T08:16:34.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.029+0000 7fabdae02740 1 -- start start 2024-09-10T08:16:34.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.029+0000 7fabdae02740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf2e40 0x55df5ebf4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.029+0000 7fabdae02740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf3760 0x55df5ec15dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.029+0000 7fabdae02740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebfe020 0x55df5ec16310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.029+0000 7fabdae02740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01b37c0 con 0x55df5ebf3760 2024-09-10T08:16:34.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.029+0000 7fabdae02740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01b1090 con 0x55df5ebf2e40 2024-09-10T08:16:34.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.029+0000 7fabdae02740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01aeee0 con 0x55df5ebfe020 2024-09-10T08:16:34.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.030+0000 7fabd4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf3760 0x55df5ec15dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.030+0000 7fabd4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf3760 0x55df5ec15dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37562/0 (socket says 172.21.15.2:37562) 2024-09-10T08:16:34.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.030+0000 7fabd4e00640 1 -- 172.21.15.2:0/3992838193 learned_addr learned my addr 172.21.15.2:0/3992838193 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:34.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.030+0000 7fabd6200640 1 --2- 172.21.15.2:0/3992838193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf2e40 0x55df5ebf4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.030+0000 7fabd6200640 1 --2- 172.21.15.2:0/3992838193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebfe020 0x55df5ec16310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.030+0000 7fabd4e00640 1 -- 172.21.15.2:0/3992838193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebfe020 msgr2=0x55df5ec16310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:34.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.030+0000 7fabd4e00640 1 --2- 172.21.15.2:0/3992838193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebfe020 0x55df5ec16310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.030+0000 7fabd4e00640 1 -- 172.21.15.2:0/3992838193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf2e40 msgr2=0x55df5ebf4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:34.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.030+0000 7fabd4e00640 1 --2- 172.21.15.2:0/3992838193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf2e40 0x55df5ebf4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.030+0000 7fabd4e00640 1 -- 172.21.15.2:0/3992838193 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df5ec10b40 con 0x55df5ebf3760 2024-09-10T08:16:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.030+0000 7fabd6200640 1 --2- 172.21.15.2:0/3992838193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf2e40 0x55df5ebf4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.030+0000 7fabd4e00640 1 --2- 172.21.15.2:0/3992838193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf3760 0x55df5ec15dd0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00a010 tx=0x7fabbc00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27aaf7eff44c275d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.031+0000 7fabc2000640 1 -- 172.21.15.2:0/3992838193 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc02e070 con 0x55df5ebf3760 2024-09-10T08:16:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.031+0000 7fabc2000640 1 -- 172.21.15.2:0/3992838193 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabbc00fe00 con 0x55df5ebf3760 2024-09-10T08:16:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.031+0000 7fabc2000640 1 -- 172.21.15.2:0/3992838193 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc010b10 con 0x55df5ebf3760 2024-09-10T08:16:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.031+0000 7fabdae02740 1 -- 172.21.15.2:0/3992838193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf3760 msgr2=0x55df5ec15dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.031+0000 7fabdae02740 1 --2- 172.21.15.2:0/3992838193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf3760 0x55df5ec15dd0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00a010 tx=0x7fabbc00f810 comp rx=0 tx=0).stop 2024-09-10T08:16:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.031+0000 7fabdae02740 1 -- 172.21.15.2:0/3992838193 shutdown_connections 2024-09-10T08:16:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.031+0000 7fabdae02740 1 --2- 172.21.15.2:0/3992838193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ebfe020 0x55df5ec16310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.031+0000 7fabdae02740 1 --2- 172.21.15.2:0/3992838193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf3760 0x55df5ec15dd0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.031+0000 7fabdae02740 1 --2- 172.21.15.2:0/3992838193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebf2e40 0x55df5ebf4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.031+0000 7fabdae02740 1 -- 172.21.15.2:0/3992838193 >> 172.21.15.2:0/3992838193 conn(0x55df5ebef3d0 msgr2=0x55df5ebf4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.032+0000 7fabdae02740 1 -- 172.21.15.2:0/3992838193 shutdown_connections 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.032+0000 7fabdae02740 1 -- 172.21.15.2:0/3992838193 wait complete. 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.032+0000 7fabdae02740 1 -- start start 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.032+0000 7fabdae02740 1 Processor -- start 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabdae02740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf2e40 0x55df5ec05fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabdae02740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebfe020 0x55df5ec06500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabdae02740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ec02f90 0x55df5ec03410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabdae02740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01af550 con 0x55df5ebf2e40 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabdae02740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01b09c0 con 0x55df5ebfe020 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabdae02740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01afc20 con 0x55df5ec02f90 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabdae02740 10 client.0 _ll_register_callbacks cb 0x7fabbc009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabd6200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf2e40 0x55df5ec05fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabd6200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf2e40 0x55df5ec05fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37576/0 (socket says 172.21.15.2:37576) 2024-09-10T08:16:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabd6200640 1 -- 172.21.15.2:0/2683402562 learned_addr learned my addr 172.21.15.2:0/2683402562 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:34.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabd4e00640 1 --2- 172.21.15.2:0/2683402562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebfe020 0x55df5ec06500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabd6200640 1 --2- 172.21.15.2:0/2683402562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ec02f90 0x55df5ec03410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabd6200640 1 -- 172.21.15.2:0/2683402562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ec02f90 msgr2=0x55df5ec03410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:34.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabd6200640 1 --2- 172.21.15.2:0/2683402562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ec02f90 0x55df5ec03410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabd6200640 1 -- 172.21.15.2:0/2683402562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebfe020 msgr2=0x55df5ec06500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:34.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabd6200640 1 --2- 172.21.15.2:0/2683402562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebfe020 0x55df5ec06500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.033+0000 7fabd6200640 1 -- 172.21.15.2:0/2683402562 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df5ebeec80 con 0x55df5ebf2e40 2024-09-10T08:16:34.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.034+0000 7fabd6200640 1 --2- 172.21.15.2:0/2683402562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf2e40 0x55df5ec05fc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009b80 tx=0x7fabc8011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13209493339bca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:34.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.034+0000 7fabb7400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:16:34.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.034+0000 7fabb7400640 1 -- 172.21.15.2:0/2683402562 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc802b020 con 0x55df5ebf2e40 2024-09-10T08:16:34.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.034+0000 7fabb7400640 1 -- 172.21.15.2:0/2683402562 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fabc8003c30 con 0x55df5ebf2e40 2024-09-10T08:16:34.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.034+0000 7fabdae02740 1 -- 172.21.15.2:0/2683402562 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabbc00fe00 con 0x55df5ebf2e40 2024-09-10T08:16:34.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.034+0000 7fabb7400640 1 -- 172.21.15.2:0/2683402562 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fabc801d040 con 0x55df5ebf2e40 2024-09-10T08:16:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.034+0000 7fabdae02740 10 client.17652 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.034+0000 7fabdae02740 1 -- 172.21.15.2:0/2683402562 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55df5ebf5cb0 con 0x55df5ebf2e40 2024-09-10T08:16:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.035+0000 7fabb7400640 1 -- 172.21.15.2:0/2683402562 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabbc00fe00 con 0x55df5ebf2e40 2024-09-10T08:16:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.035+0000 7fabb7400640 1 -- 172.21.15.2:0/2683402562 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fabc8039030 con 0x55df5ebf2e40 2024-09-10T08:16:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.035+0000 7fabdae02740 10 client.17652 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.035+0000 7fabdae02740 10 client.17652 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:16:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.035+0000 7fabdae02740 20 client.17652 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.035+0000 7fabdae02740 1 --2- 172.21.15.2:0/2683402562 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55df5ec79d00 0x55df5ec7c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.035+0000 7fabdae02740 4 client.17652 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:16:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.035+0000 7fabdae02740 1 -- 172.21.15.2:0/2683402562 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df5eb60e60 con 0x55df5ec79d00 2024-09-10T08:16:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.035+0000 7fabd4e00640 1 --2- 172.21.15.2:0/2683402562 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55df5ec79d00 0x55df5ec7c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:34.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.036+0000 7fabd4e00640 1 --2- 172.21.15.2:0/2683402562 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55df5ec79d00 0x55df5ec7c120 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c90a744ef68d6a8b server_cookie=29a0a8d5bed6bc36 in_seq=0 out_seq=0 2024-09-10T08:16:34.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.036+0000 7fabb7400640 10 client.17652 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:16:34.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.036+0000 7fabb7400640 1 -- 172.21.15.2:0/2683402562 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df5eb60e60 con 0x55df5ec79d00 2024-09-10T08:16:34.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.036+0000 7fabb7400640 10 client.17652 handle_command_reply: tid=0 2024-09-10T08:16:34.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.036+0000 7fabdae02740 1 client.17652 shutdown 2024-09-10T08:16:34.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.036+0000 7fabdae02740 1 -- 172.21.15.2:0/2683402562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf2e40 msgr2=0x55df5ec05fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:34.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.036+0000 7fabdae02740 1 --2- 172.21.15.2:0/2683402562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf2e40 0x55df5ec05fc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009b80 tx=0x7fabc8011c50 comp rx=0 tx=0).stop 2024-09-10T08:16:34.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.036+0000 7fabdae02740 1 -- 172.21.15.2:0/2683402562 shutdown_connections 2024-09-10T08:16:34.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.036+0000 7fabdae02740 1 --2- 172.21.15.2:0/2683402562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df5ec02f90 0x55df5ec03410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.036+0000 7fabdae02740 1 --2- 172.21.15.2:0/2683402562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df5ebfe020 0x55df5ec06500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.036+0000 7fabdae02740 1 --2- 172.21.15.2:0/2683402562 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55df5ec79d00 0x55df5ec7c120 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.036+0000 7fabdae02740 1 --2- 172.21.15.2:0/2683402562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df5ebf2e40 0x55df5ec05fc0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.037+0000 7fabdae02740 1 -- 172.21.15.2:0/2683402562 >> 172.21.15.2:0/2683402562 conn(0x55df5ebef3d0 msgr2=0x55df5ebee490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:34.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.037+0000 7fabb7400640 0 client.17652 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:16:34.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.037+0000 7fabdae02740 1 -- 172.21.15.2:0/2683402562 shutdown_connections 2024-09-10T08:16:34.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.037+0000 7fabdae02740 1 -- 172.21.15.2:0/2683402562 wait complete. 2024-09-10T08:16:34.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.037+0000 7fabdae02740 20 client.17652 trim_cache size 0 max 16384 2024-09-10T08:16:34.043 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:16:34.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.037+0000 7fabd6c00640 1 -- 172.21.15.2:0/3504565812 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fabb80a2b80 msgr2=0x7fabb80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:34.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.037+0000 7fabd6c00640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fabb80a2b80 0x7fabb80a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fabc40b8830 tx=0x7fabc4005d50 comp rx=0 tx=0).stop 2024-09-10T08:16:34.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.037+0000 7fabd6c00640 1 -- 172.21.15.2:0/3504565812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabd0103e30 msgr2=0x7fabd019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:34.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.037+0000 7fabd6c00640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabd0103e30 0x7fabd019a4b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fabc4004770 tx=0x7fabc4003c10 comp rx=0 tx=0).stop 2024-09-10T08:16:34.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.038+0000 7fabd6c00640 1 -- 172.21.15.2:0/3504565812 shutdown_connections 2024-09-10T08:16:34.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.038+0000 7fabd6c00640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd0108f60 0x7fabd019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.038+0000 7fabd6c00640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd01047e0 0x7fabd019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.038+0000 7fabd6c00640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fabb80a2b80 0x7fabb80a5040 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.038+0000 7fabd6c00640 1 --2- 172.21.15.2:0/3504565812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabd0103e30 0x7fabd019a4b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:34.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.038+0000 7fabd6c00640 1 -- 172.21.15.2:0/3504565812 >> 172.21.15.2:0/3504565812 conn(0x7fabd00ff910 msgr2=0x7fabd0100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:34.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.038+0000 7fabd6c00640 1 -- 172.21.15.2:0/3504565812 shutdown_connections 2024-09-10T08:16:34.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:34.038+0000 7fabd6c00640 1 -- 172.21.15.2:0/3504565812 wait complete. 2024-09-10T08:16:34.196 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-10T08:16:34.197 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:16:34.954 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:16:35.048 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:35.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.667+0000 7efffb200640 1 -- 172.21.15.2:0/3853746284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efffc104800 msgr2=0x7efffc108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.667+0000 7efffb200640 1 --2- 172.21.15.2:0/3853746284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efffc104800 0x7efffc108960 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7effe8009900 tx=0x7effe802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:16:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.667+0000 7efffb200640 1 -- 172.21.15.2:0/3853746284 shutdown_connections 2024-09-10T08:16:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.667+0000 7efffb200640 1 --2- 172.21.15.2:0/3853746284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efffc109090 0x7efffc110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.667+0000 7efffb200640 1 --2- 172.21.15.2:0/3853746284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efffc104800 0x7efffc108960 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.667+0000 7efffb200640 1 --2- 172.21.15.2:0/3853746284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efffc103e50 0x7efffc104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.667+0000 7efffb200640 1 -- 172.21.15.2:0/3853746284 >> 172.21.15.2:0/3853746284 conn(0x7efffc0ff8b0 msgr2=0x7efffc101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.667+0000 7efffb200640 1 -- 172.21.15.2:0/3853746284 shutdown_connections 2024-09-10T08:16:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.667+0000 7efffb200640 1 -- 172.21.15.2:0/3853746284 wait complete. 2024-09-10T08:16:35.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.668+0000 7efffb200640 1 Processor -- start 2024-09-10T08:16:35.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.668+0000 7efffb200640 1 -- start start 2024-09-10T08:16:35.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.669+0000 7efffb200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efffc103e50 0x7efffc19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.669+0000 7efffb200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efffc104800 0x7efffc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.669+0000 7efffb200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efffc109090 0x7efffc19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.669+0000 7efffb200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efffc114100 con 0x7efffc104800 2024-09-10T08:16:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.669+0000 7efffb200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efffc113f80 con 0x7efffc109090 2024-09-10T08:16:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.669+0000 7efffb200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efffc114280 con 0x7efffc103e50 2024-09-10T08:16:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.669+0000 7efff9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efffc103e50 0x7efffc19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.669+0000 7efff9400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efffc104800 0x7efffc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.669+0000 7efff9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efffc103e50 0x7efffc19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36514/0 (socket says 172.21.15.2:36514) 2024-09-10T08:16:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.669+0000 7efff9e00640 1 -- 172.21.15.2:0/2675017714 learned_addr learned my addr 172.21.15.2:0/2675017714 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.669+0000 7efffa800640 1 --2- 172.21.15.2:0/2675017714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efffc109090 0x7efffc19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:35.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.670+0000 7efffa800640 1 -- 172.21.15.2:0/2675017714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efffc103e50 msgr2=0x7efffc19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:35.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.670+0000 7efffa800640 1 --2- 172.21.15.2:0/2675017714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efffc103e50 0x7efffc19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:35.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.670+0000 7efffa800640 1 -- 172.21.15.2:0/2675017714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efffc104800 msgr2=0x7efffc19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:35.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.670+0000 7efffa800640 1 --2- 172.21.15.2:0/2675017714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efffc104800 0x7efffc19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:35.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.670+0000 7efffa800640 1 -- 172.21.15.2:0/2675017714 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efffc19f550 con 0x7efffc109090 2024-09-10T08:16:35.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.670+0000 7efffa800640 1 --2- 172.21.15.2:0/2675017714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efffc109090 0x7efffc19ee70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efff400ec50 tx=0x7efff400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:35.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.670+0000 7efff2a00640 1 -- 172.21.15.2:0/2675017714 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4003b70 con 0x7efffc109090 2024-09-10T08:16:35.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.671+0000 7efffb200640 1 -- 172.21.15.2:0/2675017714 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efffc19f840 con 0x7efffc109090 2024-09-10T08:16:35.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.671+0000 7efff2a00640 1 -- 172.21.15.2:0/2675017714 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff4003d10 con 0x7efffc109090 2024-09-10T08:16:35.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.671+0000 7efff2a00640 1 -- 172.21.15.2:0/2675017714 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4010890 con 0x7efffc109090 2024-09-10T08:16:35.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.671+0000 7efffb200640 1 -- 172.21.15.2:0/2675017714 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efffc1a7120 con 0x7efffc109090 2024-09-10T08:16:35.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.672+0000 7efffb200640 1 -- 172.21.15.2:0/2675017714 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efffc105330 con 0x7efffc109090 2024-09-10T08:16:35.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.674+0000 7efff2a00640 1 -- 172.21.15.2:0/2675017714 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7efff4003fe0 con 0x7efffc109090 2024-09-10T08:16:35.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.675+0000 7efff2a00640 1 --2- 172.21.15.2:0/2675017714 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7effe00a2c00 0x7effe00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:35.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.675+0000 7efff2a00640 1 -- 172.21.15.2:0/2675017714 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7efff40cf940 con 0x7efffc109090 2024-09-10T08:16:35.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.675+0000 7efff9e00640 1 --2- 172.21.15.2:0/2675017714 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7effe00a2c00 0x7effe00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:35.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.676+0000 7efff9e00640 1 --2- 172.21.15.2:0/2675017714 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7effe00a2c00 0x7effe00a50c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7effec009cd0 tx=0x7effec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:35.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.679+0000 7efff2a00640 1 -- 172.21.15.2:0/2675017714 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efff409c450 con 0x7efffc109090 2024-09-10T08:16:35.735 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:35.875 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:35 smithi002 ceph-mon[30563]: pgmap v640: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:16:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:35 smithi155 ceph-mon[35405]: pgmap v640: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:16:35.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:35 smithi107 ceph-mon[35048]: pgmap v640: 129 pgs: 129 active+clean; 4.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:16:35.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.929+0000 7efffb200640 1 -- 172.21.15.2:0/2675017714 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efffc10c2d0 con 0x7efffc109090 2024-09-10T08:16:35.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.930+0000 7efff2a00640 1 -- 172.21.15.2:0/2675017714 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7efff409bd50 con 0x7efffc109090 2024-09-10T08:16:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:16:35.930 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:16:35.930 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:16:35.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.932+0000 7efffb200640 1 -- 172.21.15.2:0/2675017714 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7effe00a2c00 msgr2=0x7effe00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:35.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.932+0000 7efffb200640 1 --2- 172.21.15.2:0/2675017714 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7effe00a2c00 0x7effe00a50c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7effec009cd0 tx=0x7effec005f90 comp rx=0 tx=0).stop 2024-09-10T08:16:35.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.932+0000 7efffb200640 1 -- 172.21.15.2:0/2675017714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efffc109090 msgr2=0x7efffc19ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:35.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.932+0000 7efffb200640 1 --2- 172.21.15.2:0/2675017714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efffc109090 0x7efffc19ee70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efff400ec50 tx=0x7efff400c6a0 comp rx=0 tx=0).stop 2024-09-10T08:16:35.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.932+0000 7efffb200640 1 -- 172.21.15.2:0/2675017714 shutdown_connections 2024-09-10T08:16:35.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.932+0000 7efffb200640 1 --2- 172.21.15.2:0/2675017714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efffc109090 0x7efffc19ee70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:35.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.932+0000 7efffb200640 1 --2- 172.21.15.2:0/2675017714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efffc104800 0x7efffc19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:35.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.932+0000 7efffb200640 1 --2- 172.21.15.2:0/2675017714 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7effe00a2c00 0x7effe00a50c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:35.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.932+0000 7efffb200640 1 --2- 172.21.15.2:0/2675017714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efffc103e50 0x7efffc19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:35.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.932+0000 7efffb200640 1 -- 172.21.15.2:0/2675017714 >> 172.21.15.2:0/2675017714 conn(0x7efffc0ff8b0 msgr2=0x7efffc100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:35.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.932+0000 7efffb200640 1 -- 172.21.15.2:0/2675017714 shutdown_connections 2024-09-10T08:16:35.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:35.933+0000 7efffb200640 1 -- 172.21.15.2:0/2675017714 wait complete. 2024-09-10T08:16:36.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.506+0000 7ffa7a800640 1 -- 172.21.15.2:0/2519071457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa74103e30 msgr2=0x7ffa74104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:36.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.506+0000 7ffa7a800640 1 --2- 172.21.15.2:0/2519071457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa74103e30 0x7ffa74104210 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009900 tx=0x7ffa6802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.506+0000 7ffa7a800640 1 -- 172.21.15.2:0/2519071457 shutdown_connections 2024-09-10T08:16:36.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.506+0000 7ffa7a800640 1 --2- 172.21.15.2:0/2519071457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa74108f60 0x7ffa74110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.506+0000 7ffa7a800640 1 --2- 172.21.15.2:0/2519071457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa741047e0 0x7ffa74108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.506+0000 7ffa7a800640 1 --2- 172.21.15.2:0/2519071457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa74103e30 0x7ffa74104210 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.506+0000 7ffa7a800640 1 -- 172.21.15.2:0/2519071457 >> 172.21.15.2:0/2519071457 conn(0x7ffa740ff8d0 msgr2=0x7ffa74101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.506+0000 7ffa7a800640 1 -- 172.21.15.2:0/2519071457 shutdown_connections 2024-09-10T08:16:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.506+0000 7ffa7a800640 1 -- 172.21.15.2:0/2519071457 wait complete. 2024-09-10T08:16:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.507+0000 7ffa7a800640 1 Processor -- start 2024-09-10T08:16:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.507+0000 7ffa7a800640 1 -- start start 2024-09-10T08:16:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.507+0000 7ffa7a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa74103e30 0x7ffa7419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.507+0000 7ffa7a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa741047e0 0x7ffa7419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.507+0000 7ffa7a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa74108f60 0x7ffa7419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.507+0000 7ffa7a800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffa74113f80 con 0x7ffa74103e30 2024-09-10T08:16:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.507+0000 7ffa7a800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa74113e00 con 0x7ffa74108f60 2024-09-10T08:16:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.507+0000 7ffa7a800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa74114100 con 0x7ffa741047e0 2024-09-10T08:16:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.508+0000 7ffa79400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa74103e30 0x7ffa7419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.508+0000 7ffa79e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa74108f60 0x7ffa7419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.508+0000 7ffa79400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa74103e30 0x7ffa7419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37622/0 (socket says 172.21.15.2:37622) 2024-09-10T08:16:36.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.508+0000 7ffa78a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa741047e0 0x7ffa7419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.508+0000 7ffa79400640 1 -- 172.21.15.2:0/3456502783 learned_addr learned my addr 172.21.15.2:0/3456502783 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:36.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.508+0000 7ffa79e00640 1 -- 172.21.15.2:0/3456502783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa741047e0 msgr2=0x7ffa7419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:36.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.508+0000 7ffa79e00640 1 --2- 172.21.15.2:0/3456502783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa741047e0 0x7ffa7419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.508+0000 7ffa79e00640 1 -- 172.21.15.2:0/3456502783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa74103e30 msgr2=0x7ffa7419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:36.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.508+0000 7ffa79e00640 1 --2- 172.21.15.2:0/3456502783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa74103e30 0x7ffa7419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.508+0000 7ffa79e00640 1 -- 172.21.15.2:0/3456502783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7419f450 con 0x7ffa74108f60 2024-09-10T08:16:36.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.509+0000 7ffa79e00640 1 --2- 172.21.15.2:0/3456502783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa74108f60 0x7ffa7419ed70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00da10 tx=0x7ffa6c00ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:36.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.509+0000 7ffa72000640 1 -- 172.21.15.2:0/3456502783 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c00bd70 con 0x7ffa74108f60 2024-09-10T08:16:36.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.509+0000 7ffa7a800640 1 -- 172.21.15.2:0/3456502783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7419f740 con 0x7ffa74108f60 2024-09-10T08:16:36.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.509+0000 7ffa72000640 1 -- 172.21.15.2:0/3456502783 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c003e50 con 0x7ffa74108f60 2024-09-10T08:16:36.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.509+0000 7ffa72000640 1 -- 172.21.15.2:0/3456502783 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0106a0 con 0x7ffa74108f60 2024-09-10T08:16:36.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.509+0000 7ffa7a800640 1 -- 172.21.15.2:0/3456502783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa741a7020 con 0x7ffa74108f60 2024-09-10T08:16:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.512+0000 7ffa7e88a740 1 Processor -- start 2024-09-10T08:16:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.512+0000 7ffa72000640 1 -- 172.21.15.2:0/3456502783 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ffa6c003970 con 0x7ffa74108f60 2024-09-10T08:16:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.512+0000 7ffa7e88a740 1 -- start start 2024-09-10T08:16:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.513+0000 7ffa7e88a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 0x562b88a03220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.513+0000 7ffa7e88a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a0dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.513+0000 7ffa7e88a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a0e020 0x562b88a10410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.513+0000 7ffa7e88a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562b889693e0 con 0x562b88a02e40 2024-09-10T08:16:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.513+0000 7ffa7e88a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562b88910d80 con 0x562b88a0e020 2024-09-10T08:16:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.513+0000 7ffa7e88a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562b889653a0 con 0x562b88a03760 2024-09-10T08:16:36.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.513+0000 7ffa78a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 0x562b88a03220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.513+0000 7ffa78a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 0x562b88a03220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37628/0 (socket says 172.21.15.2:37628) 2024-09-10T08:16:36.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.513+0000 7ffa78a00640 1 -- 172.21.15.2:0/2039934499 learned_addr learned my addr 172.21.15.2:0/2039934499 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:36.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.513+0000 7ffa79400640 1 --2- 172.21.15.2:0/2039934499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a0dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.513+0000 7ffa78a00640 1 --2- 172.21.15.2:0/2039934499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a0e020 0x562b88a10410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.514+0000 7ffa79400640 1 -- 172.21.15.2:0/2039934499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a0e020 msgr2=0x562b88a10410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:36.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.514+0000 7ffa79400640 1 --2- 172.21.15.2:0/2039934499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a0e020 0x562b88a10410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.514+0000 7ffa79400640 1 -- 172.21.15.2:0/2039934499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 msgr2=0x562b88a03220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:36.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.514+0000 7ffa79400640 1 --2- 172.21.15.2:0/2039934499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 0x562b88a03220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.514+0000 7ffa79400640 1 -- 172.21.15.2:0/2039934499 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8892fae0 con 0x562b88a03760 2024-09-10T08:16:36.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.514+0000 7ffa78a00640 1 --2- 172.21.15.2:0/2039934499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 0x562b88a03220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:36.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.514+0000 7ffa72000640 1 --2- 172.21.15.2:0/3456502783 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa5c0a2c00 0x7ffa5c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.514+0000 7ffa79400640 1 --2- 172.21.15.2:0/2039934499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a0dae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009230 tx=0x7ffa68032cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f5a6ee6d6e6b2c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:36.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.514+0000 7ffa72000640 1 -- 172.21.15.2:0/3456502783 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ffa6c0cf780 con 0x7ffa74108f60 2024-09-10T08:16:36.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.514+0000 7ffa78a00640 1 --2- 172.21.15.2:0/3456502783 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa5c0a2c00 0x7ffa5c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.514+0000 7ffa66600640 1 -- 172.21.15.2:0/2039934499 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68040070 con 0x562b88a03760 2024-09-10T08:16:36.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.515+0000 7ffa66600640 1 -- 172.21.15.2:0/2039934499 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa680317b0 con 0x562b88a03760 2024-09-10T08:16:36.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.515+0000 7ffa66600640 1 -- 172.21.15.2:0/2039934499 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68035040 con 0x562b88a03760 2024-09-10T08:16:36.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.515+0000 7ffa7e88a740 1 -- 172.21.15.2:0/2039934499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 msgr2=0x562b88a0dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:36.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.515+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/2039934499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a0dae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009230 tx=0x7ffa68032cb0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.515+0000 7ffa78a00640 1 --2- 172.21.15.2:0/3456502783 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa5c0a2c00 0x7ffa5c0a50c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x562b88a00750 tx=0x7ffa6000b650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:36.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.515+0000 7ffa7e88a740 1 -- 172.21.15.2:0/2039934499 shutdown_connections 2024-09-10T08:16:36.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.515+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/2039934499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a0e020 0x562b88a10410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.515+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/2039934499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a0dae0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.515+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/2039934499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 0x562b88a03220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.515+0000 7ffa7e88a740 1 -- 172.21.15.2:0/2039934499 >> 172.21.15.2:0/2039934499 conn(0x562b889ff3d0 msgr2=0x562b889ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:36.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.516+0000 7ffa7e88a740 1 -- 172.21.15.2:0/2039934499 shutdown_connections 2024-09-10T08:16:36.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.516+0000 7ffa7e88a740 1 -- 172.21.15.2:0/2039934499 wait complete. 2024-09-10T08:16:36.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.516+0000 7ffa7e88a740 1 -- start start 2024-09-10T08:16:36.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.517+0000 7ffa7e88a740 1 Processor -- start 2024-09-10T08:16:36.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.517+0000 7ffa7e88a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a02e40 0x562b88a0afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.517+0000 7ffa7e88a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a0b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.517+0000 7ffa7e88a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a0e020 0x562b88a0ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.517+0000 7ffa7e88a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562b88a10bf0 con 0x562b88a0e020 2024-09-10T08:16:36.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.517+0000 7ffa7e88a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562b889240f0 con 0x562b88a02e40 2024-09-10T08:16:36.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.517+0000 7ffa7e88a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562b88a10d70 con 0x562b88a03760 2024-09-10T08:16:36.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.517+0000 7ffa7e88a740 10 client.0 _ll_register_callbacks cb 0x562b8890bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:36.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.518+0000 7ffa79400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a02e40 0x562b88a0afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.518+0000 7ffa79400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a02e40 0x562b88a0afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42108/0 (socket says 172.21.15.2:42108) 2024-09-10T08:16:36.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.518+0000 7ffa79400640 1 -- 172.21.15.2:0/598788708 learned_addr learned my addr 172.21.15.2:0/598788708 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:36.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.518+0000 7ffa78a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a0b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.518+0000 7ffa79e00640 1 --2- 172.21.15.2:0/598788708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a0e020 0x562b88a0ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.518+0000 7ffa78a00640 1 -- 172.21.15.2:0/598788708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a02e40 msgr2=0x562b88a0afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:36.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.518+0000 7ffa78a00640 1 --2- 172.21.15.2:0/598788708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a02e40 0x562b88a0afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.518+0000 7ffa78a00640 1 -- 172.21.15.2:0/598788708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a0e020 msgr2=0x562b88a0ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:36.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.518+0000 7ffa78a00640 1 --2- 172.21.15.2:0/598788708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a0e020 0x562b88a0ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.518+0000 7ffa78a00640 1 -- 172.21.15.2:0/598788708 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b886da3c0 con 0x562b88a03760 2024-09-10T08:16:36.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.519+0000 7ffa78a00640 1 --2- 172.21.15.2:0/598788708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a0b500 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000b6c0 tx=0x7ffa60004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=775594d849e992bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:36.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.519+0000 7ffa5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:16:36.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.519+0000 7ffa5aa00640 1 -- 172.21.15.2:0/598788708 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60004220 con 0x562b88a03760 2024-09-10T08:16:36.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.519+0000 7ffa7e88a740 1 -- 172.21.15.2:0/598788708 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa680317b0 con 0x562b88a03760 2024-09-10T08:16:36.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.519+0000 7ffa5aa00640 1 -- 172.21.15.2:0/598788708 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ffa600043c0 con 0x562b88a03760 2024-09-10T08:16:36.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.519+0000 7ffa5aa00640 1 -- 172.21.15.2:0/598788708 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffa600119c0 con 0x562b88a03760 2024-09-10T08:16:36.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.519+0000 7ffa7e88a740 10 client.27643 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:36.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.519+0000 7ffa7e88a740 1 -- 172.21.15.2:0/598788708 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b88a1cd80 con 0x562b88a03760 2024-09-10T08:16:36.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.520+0000 7ffa5aa00640 1 -- 172.21.15.2:0/598788708 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa680317b0 con 0x562b88a03760 2024-09-10T08:16:36.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.520+0000 7ffa5aa00640 1 -- 172.21.15.2:0/598788708 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ffa60027070 con 0x562b88a03760 2024-09-10T08:16:36.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.520+0000 7ffa7e88a740 10 client.27643 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:36.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.520+0000 7ffa7e88a740 10 client.27643 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:16:36.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.520+0000 7ffa7e88a740 20 client.27643 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:36.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.520+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/598788708 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562b88a1e1e0 0x562b88a20640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.520+0000 7ffa7e88a740 4 client.27643 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:36.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.520+0000 7ffa7e88a740 1 -- 172.21.15.2:0/598788708 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b88976950 con 0x562b88a1e1e0 2024-09-10T08:16:36.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.520+0000 7ffa79400640 1 --2- 172.21.15.2:0/598788708 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562b88a1e1e0 0x562b88a20640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.521+0000 7ffa79400640 1 --2- 172.21.15.2:0/598788708 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562b88a1e1e0 0x562b88a20640 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c9b4adefc3482ed server_cookie=524f306b50ac5c9a in_seq=0 out_seq=0 2024-09-10T08:16:36.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.521+0000 7ffa5aa00640 10 client.27643 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.522+0000 7ffa5aa00640 1 -- 172.21.15.2:0/598788708 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ffa6804fe80 con 0x562b88a1e1e0 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.522+0000 7ffa5aa00640 10 client.27643 handle_command_reply: tid=0 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.522+0000 7ffa7e88a740 1 client.27643 shutdown 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.522+0000 7ffa7e88a740 1 -- 172.21.15.2:0/598788708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 msgr2=0x562b88a0b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.522+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/598788708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a0b500 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000b6c0 tx=0x7ffa60004730 comp rx=0 tx=0).stop 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.522+0000 7ffa7e88a740 1 -- 172.21.15.2:0/598788708 shutdown_connections 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.522+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/598788708 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562b88a1e1e0 0x562b88a20640 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.522+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/598788708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a0e020 0x562b88a0ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.522+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/598788708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a0b500 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.522+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/598788708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a02e40 0x562b88a0afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.522+0000 7ffa5aa00640 0 client.27643 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.522+0000 7ffa7e88a740 1 -- 172.21.15.2:0/598788708 >> 172.21.15.2:0/598788708 conn(0x562b889ff3d0 msgr2=0x562b88a05100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.523+0000 7ffa7e88a740 1 -- 172.21.15.2:0/598788708 shutdown_connections 2024-09-10T08:16:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.523+0000 7ffa7e88a740 1 -- 172.21.15.2:0/598788708 wait complete. 2024-09-10T08:16:36.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.523+0000 7ffa7e88a740 20 client.27643 trim_cache size 0 max 16384 2024-09-10T08:16:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.538+0000 7ffa7e88a740 1 Processor -- start 2024-09-10T08:16:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.538+0000 7ffa7e88a740 1 -- start start 2024-09-10T08:16:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.538+0000 7ffa7e88a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 0x562b88a058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.538+0000 7ffa7e88a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a03760 0x562b88a25f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.538+0000 7ffa7e88a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a0e020 0x562b88a26450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.538+0000 7ffa7e88a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffa741b3780 con 0x562b88a02e40 2024-09-10T08:16:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.538+0000 7ffa7e88a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa741b1050 con 0x562b88a03760 2024-09-10T08:16:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.538+0000 7ffa7e88a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa741aeea0 con 0x562b88a0e020 2024-09-10T08:16:36.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.538+0000 7ffa79400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 0x562b88a058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.538+0000 7ffa79400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 0x562b88a058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37652/0 (socket says 172.21.15.2:37652) 2024-09-10T08:16:36.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.538+0000 7ffa79400640 1 -- 172.21.15.2:0/3988008260 learned_addr learned my addr 172.21.15.2:0/3988008260 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:36.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.538+0000 7ffa78a00640 1 --2- 172.21.15.2:0/3988008260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a03760 0x562b88a25f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.539+0000 7ffa79e00640 1 --2- 172.21.15.2:0/3988008260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a0e020 0x562b88a26450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.539+0000 7ffa79400640 1 -- 172.21.15.2:0/3988008260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a0e020 msgr2=0x562b88a26450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:36.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.539+0000 7ffa79400640 1 --2- 172.21.15.2:0/3988008260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a0e020 0x562b88a26450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.539+0000 7ffa79400640 1 -- 172.21.15.2:0/3988008260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a03760 msgr2=0x562b88a25f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:36.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.539+0000 7ffa79400640 1 --2- 172.21.15.2:0/3988008260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a03760 0x562b88a25f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.539+0000 7ffa79400640 1 -- 172.21.15.2:0/3988008260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b88a02c70 con 0x562b88a02e40 2024-09-10T08:16:36.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.539+0000 7ffa78a00640 1 --2- 172.21.15.2:0/3988008260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a03760 0x562b88a25f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:36.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.539+0000 7ffa79400640 1 --2- 172.21.15.2:0/3988008260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 0x562b88a058f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009230 tx=0x7ffa68005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8070df84e32fd4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:36.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.539+0000 7ffa65c00640 1 -- 172.21.15.2:0/3988008260 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68035040 con 0x562b88a02e40 2024-09-10T08:16:36.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.539+0000 7ffa65c00640 1 -- 172.21.15.2:0/3988008260 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa68053070 con 0x562b88a02e40 2024-09-10T08:16:36.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.540+0000 7ffa65c00640 1 -- 172.21.15.2:0/3988008260 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68040040 con 0x562b88a02e40 2024-09-10T08:16:36.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.540+0000 7ffa7e88a740 1 -- 172.21.15.2:0/3988008260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 msgr2=0x562b88a058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:36.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.540+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/3988008260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 0x562b88a058f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009230 tx=0x7ffa68005260 comp rx=0 tx=0).stop 2024-09-10T08:16:36.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.540+0000 7ffa7e88a740 1 -- 172.21.15.2:0/3988008260 shutdown_connections 2024-09-10T08:16:36.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.540+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/3988008260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a0e020 0x562b88a26450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.540+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/3988008260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a03760 0x562b88a25f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.540+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/3988008260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a02e40 0x562b88a058f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.540+0000 7ffa7e88a740 1 -- 172.21.15.2:0/3988008260 >> 172.21.15.2:0/3988008260 conn(0x562b889ff3d0 msgr2=0x562b88a0d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:36.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.540+0000 7ffa7e88a740 1 -- 172.21.15.2:0/3988008260 shutdown_connections 2024-09-10T08:16:36.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.540+0000 7ffa7e88a740 1 -- 172.21.15.2:0/3988008260 wait complete. 2024-09-10T08:16:36.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.541+0000 7ffa7e88a740 1 -- start start 2024-09-10T08:16:36.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.541+0000 7ffa7e88a740 1 Processor -- start 2024-09-10T08:16:36.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.541+0000 7ffa7e88a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a165f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.541+0000 7ffa7e88a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a0e020 0x562b88a11880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa7e88a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a11dc0 0x562b88a12240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa7e88a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffa741af510 con 0x562b88a11dc0 2024-09-10T08:16:36.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa7e88a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa741b0980 con 0x562b88a0e020 2024-09-10T08:16:36.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa7e88a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa741afbe0 con 0x562b88a03760 2024-09-10T08:16:36.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa7e88a740 10 client.0 _ll_register_callbacks cb 0x7ffa6000b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:36.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa79e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a11dc0 0x562b88a12240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa79e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a11dc0 0x562b88a12240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37666/0 (socket says 172.21.15.2:37666) 2024-09-10T08:16:36.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa79e00640 1 -- 172.21.15.2:0/2558367188 learned_addr learned my addr 172.21.15.2:0/2558367188 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:36.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa79400640 1 --2- 172.21.15.2:0/2558367188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa78a00640 1 --2- 172.21.15.2:0/2558367188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a0e020 0x562b88a11880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa79e00640 1 -- 172.21.15.2:0/2558367188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 msgr2=0x562b88a165f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:36.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa79e00640 1 --2- 172.21.15.2:0/2558367188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a165f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa79e00640 1 -- 172.21.15.2:0/2558367188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a0e020 msgr2=0x562b88a11880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:36.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa79e00640 1 --2- 172.21.15.2:0/2558367188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a0e020 0x562b88a11880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.542+0000 7ffa79e00640 1 -- 172.21.15.2:0/2558367188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b88a87f10 con 0x562b88a11dc0 2024-09-10T08:16:36.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.543+0000 7ffa79e00640 1 --2- 172.21.15.2:0/2558367188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a11dc0 0x562b88a12240 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c09bb20 tx=0x7ffa6c09c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5750bf040f17a351 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:36.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.543+0000 7ffa5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:16:36.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.543+0000 7ffa5aa00640 1 -- 172.21.15.2:0/2558367188 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0b8070 con 0x562b88a11dc0 2024-09-10T08:16:36.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.543+0000 7ffa5aa00640 1 -- 172.21.15.2:0/2558367188 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ffa6c0042e0 con 0x562b88a11dc0 2024-09-10T08:16:36.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.543+0000 7ffa7e88a740 1 -- 172.21.15.2:0/2558367188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa68053070 con 0x562b88a11dc0 2024-09-10T08:16:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.543+0000 7ffa5aa00640 1 -- 172.21.15.2:0/2558367188 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffa6c0a4800 con 0x562b88a11dc0 2024-09-10T08:16:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.543+0000 7ffa7e88a740 10 client.17676 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.543+0000 7ffa7e88a740 1 -- 172.21.15.2:0/2558367188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b88a891f0 con 0x562b88a11dc0 2024-09-10T08:16:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.543+0000 7ffa5aa00640 1 -- 172.21.15.2:0/2558367188 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa68053070 con 0x562b88a11dc0 2024-09-10T08:16:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.544+0000 7ffa5aa00640 1 -- 172.21.15.2:0/2558367188 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ffa6c0ce040 con 0x562b88a11dc0 2024-09-10T08:16:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.544+0000 7ffa7e88a740 10 client.17676 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.544+0000 7ffa7e88a740 10 client.17676 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:16:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.544+0000 7ffa7e88a740 20 client.17676 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.544+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/2558367188 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562b88a89780 0x562b88a8bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.544+0000 7ffa7e88a740 4 client.17676 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:16:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.544+0000 7ffa7e88a740 1 -- 172.21.15.2:0/2558367188 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b88970e60 con 0x562b88a89780 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.544+0000 7ffa79400640 1 --2- 172.21.15.2:0/2558367188 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562b88a89780 0x562b88a8bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa79400640 1 --2- 172.21.15.2:0/2558367188 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562b88a89780 0x562b88a8bb70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1a2582ecc0ffbed server_cookie=26219f803a574569 in_seq=0 out_seq=0 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa5aa00640 10 client.17676 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa5aa00640 1 -- 172.21.15.2:0/2558367188 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b88970e60 con 0x562b88a89780 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa5aa00640 10 client.17676 handle_command_reply: tid=0 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa7e88a740 1 client.17676 shutdown 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa7e88a740 1 -- 172.21.15.2:0/2558367188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a11dc0 msgr2=0x562b88a12240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/2558367188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a11dc0 0x562b88a12240 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c09bb20 tx=0x7ffa6c09c350 comp rx=0 tx=0).stop 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa7e88a740 1 -- 172.21.15.2:0/2558367188 shutdown_connections 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/2558367188 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x562b88a89780 0x562b88a8bb70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/2558367188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b88a11dc0 0x562b88a12240 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa5aa00640 0 client.17676 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/2558367188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b88a0e020 0x562b88a11880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa7e88a740 1 --2- 172.21.15.2:0/2558367188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b88a03760 0x562b88a165f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.545+0000 7ffa7e88a740 1 -- 172.21.15.2:0/2558367188 >> 172.21.15.2:0/2558367188 conn(0x562b889ff3d0 msgr2=0x562b88a0af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:36.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.546+0000 7ffa7e88a740 1 -- 172.21.15.2:0/2558367188 shutdown_connections 2024-09-10T08:16:36.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.546+0000 7ffa7e88a740 1 -- 172.21.15.2:0/2558367188 wait complete. 2024-09-10T08:16:36.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.546+0000 7ffa7e88a740 20 client.17676 trim_cache size 0 max 16384 2024-09-10T08:16:36.549 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:16:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.547+0000 7ffa7a800640 1 -- 172.21.15.2:0/3456502783 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa5c0a2c00 msgr2=0x7ffa5c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.547+0000 7ffa7a800640 1 --2- 172.21.15.2:0/3456502783 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa5c0a2c00 0x7ffa5c0a50c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x562b88a00750 tx=0x7ffa6000b650 comp rx=0 tx=0).stop 2024-09-10T08:16:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.547+0000 7ffa7a800640 1 -- 172.21.15.2:0/3456502783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa74108f60 msgr2=0x7ffa7419ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.547+0000 7ffa7a800640 1 --2- 172.21.15.2:0/3456502783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa74108f60 0x7ffa7419ed70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00da10 tx=0x7ffa6c00ded0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.547+0000 7ffa7a800640 1 -- 172.21.15.2:0/3456502783 shutdown_connections 2024-09-10T08:16:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.547+0000 7ffa7a800640 1 --2- 172.21.15.2:0/3456502783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa74108f60 0x7ffa7419ed70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.547+0000 7ffa7a800640 1 --2- 172.21.15.2:0/3456502783 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ffa5c0a2c00 0x7ffa5c0a50c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.547+0000 7ffa7a800640 1 --2- 172.21.15.2:0/3456502783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa741047e0 0x7ffa7419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.547+0000 7ffa7a800640 1 --2- 172.21.15.2:0/3456502783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa74103e30 0x7ffa7419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.547+0000 7ffa7a800640 1 -- 172.21.15.2:0/3456502783 >> 172.21.15.2:0/3456502783 conn(0x7ffa740ff8d0 msgr2=0x7ffa74100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.547+0000 7ffa7a800640 1 -- 172.21.15.2:0/3456502783 shutdown_connections 2024-09-10T08:16:36.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:36.547+0000 7ffa7a800640 1 -- 172.21.15.2:0/3456502783 wait complete. 2024-09-10T08:16:36.673 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:36 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2675017714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:16:36.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:36 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2675017714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:16:36.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:36 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2675017714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:16:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.531+0000 7f06cbe00640 1 -- 172.21.15.2:0/3145854021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06cc076bb0 msgr2=0x7f06cc076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.531+0000 7f06cbe00640 1 --2- 172.21.15.2:0/3145854021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06cc076bb0 0x7f06cc076fb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009900 tx=0x7f06bc02a410 comp rx=0 tx=0).stop 2024-09-10T08:16:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.531+0000 7f06cbe00640 1 -- 172.21.15.2:0/3145854021 shutdown_connections 2024-09-10T08:16:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.531+0000 7f06cbe00640 1 --2- 172.21.15.2:0/3145854021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc078790 0x7f06cc079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.531+0000 7f06cbe00640 1 --2- 172.21.15.2:0/3145854021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06cc077df0 0x7f06cc078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.531+0000 7f06cbe00640 1 --2- 172.21.15.2:0/3145854021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06cc076bb0 0x7f06cc076fb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.531+0000 7f06cbe00640 1 -- 172.21.15.2:0/3145854021 >> 172.21.15.2:0/3145854021 conn(0x7f06cc0ff310 msgr2=0x7f06cc101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.531+0000 7f06cbe00640 1 -- 172.21.15.2:0/3145854021 shutdown_connections 2024-09-10T08:16:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.531+0000 7f06cbe00640 1 -- 172.21.15.2:0/3145854021 wait complete. 2024-09-10T08:16:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.532+0000 7f06cbe00640 1 Processor -- start 2024-09-10T08:16:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.532+0000 7f06cbe00640 1 -- start start 2024-09-10T08:16:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.532+0000 7f06cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc076bb0 0x7f06cc19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.532+0000 7f06cbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06cc077df0 0x7f06cc19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.532+0000 7f06cbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06cc078790 0x7f06cc1a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.532+0000 7f06cbe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc113c50 con 0x7f06cc076bb0 2024-09-10T08:16:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.532+0000 7f06cbe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc113ad0 con 0x7f06cc078790 2024-09-10T08:16:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.532+0000 7f06cbe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc113dd0 con 0x7f06cc077df0 2024-09-10T08:16:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.533+0000 7f06caa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc076bb0 0x7f06cc19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.533+0000 7f06caa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc076bb0 0x7f06cc19e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37696/0 (socket says 172.21.15.2:37696) 2024-09-10T08:16:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.533+0000 7f06caa00640 1 -- 172.21.15.2:0/3923471028 learned_addr learned my addr 172.21.15.2:0/3923471028 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.533+0000 7f06cb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06cc078790 0x7f06cc1a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.533+0000 7f06ca000640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06cc077df0 0x7f06cc19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.533+0000 7f06caa00640 1 -- 172.21.15.2:0/3923471028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06cc077df0 msgr2=0x7f06cc19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.533+0000 7f06caa00640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06cc077df0 0x7f06cc19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.533+0000 7f06caa00640 1 -- 172.21.15.2:0/3923471028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06cc078790 msgr2=0x7f06cc1a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.533+0000 7f06caa00640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06cc078790 0x7f06cc1a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.533+0000 7f06caa00640 1 -- 172.21.15.2:0/3923471028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc1a5ea0 con 0x7f06cc076bb0 2024-09-10T08:16:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.533+0000 7f06cb400640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06cc078790 0x7f06cc1a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.533+0000 7f06caa00640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc076bb0 0x7f06cc19e280 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f06bc031440 tx=0x7f06bc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.534+0000 7f06d0a00640 1 -- 172.21.15.2:0/3923471028 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc03f030 con 0x7f06cc076bb0 2024-09-10T08:16:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.534+0000 7f06cbe00640 1 -- 172.21.15.2:0/3923471028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06cc1a6130 con 0x7f06cc076bb0 2024-09-10T08:16:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.534+0000 7f06d0a00640 1 -- 172.21.15.2:0/3923471028 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06bc003ce0 con 0x7f06cc076bb0 2024-09-10T08:16:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.534+0000 7f06d0a00640 1 -- 172.21.15.2:0/3923471028 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc035070 con 0x7f06cc076bb0 2024-09-10T08:16:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.534+0000 7f06cbe00640 1 -- 172.21.15.2:0/3923471028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06cc1a6620 con 0x7f06cc076bb0 2024-09-10T08:16:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.537+0000 7f06d0a00640 1 -- 172.21.15.2:0/3923471028 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f06bc042030 con 0x7f06cc076bb0 2024-09-10T08:16:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.537+0000 7f06d4bbd740 1 Processor -- start 2024-09-10T08:16:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.537+0000 7f06d4bbd740 1 -- start start 2024-09-10T08:16:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.537+0000 7f06d4bbd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bd260 0x5654d98bd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.537+0000 7f06d4bbd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98bdba0 0x5654d98c2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.537+0000 7f06d4bbd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98c2c30 0x5654d98c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.537+0000 7f06d4bbd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5654d9822100 con 0x5654d98c2c30 2024-09-10T08:16:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.537+0000 7f06d4bbd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5654d97ca1b0 con 0x5654d98bd260 2024-09-10T08:16:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.537+0000 7f06d4bbd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5654d981e0c0 con 0x5654d98bdba0 2024-09-10T08:16:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.538+0000 7f06cb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bd260 0x5654d98bd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.538+0000 7f06cb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bd260 0x5654d98bd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:34978/0 (socket says 172.21.15.2:34978) 2024-09-10T08:16:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.538+0000 7f06cb400640 1 -- 172.21.15.2:0/1643479136 learned_addr learned my addr 172.21.15.2:0/1643479136 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.538+0000 7f06caa00640 1 --2- 172.21.15.2:0/1643479136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98bdba0 0x5654d98c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.538+0000 7f06ca000640 1 --2- 172.21.15.2:0/1643479136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98c2c30 0x5654d98c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.538+0000 7f06cb400640 1 -- 172.21.15.2:0/1643479136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98bdba0 msgr2=0x5654d98c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.538+0000 7f06cb400640 1 --2- 172.21.15.2:0/1643479136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98bdba0 0x5654d98c2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.538+0000 7f06cb400640 1 -- 172.21.15.2:0/1643479136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98c2c30 msgr2=0x5654d98c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.538+0000 7f06cb400640 1 --2- 172.21.15.2:0/1643479136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98c2c30 0x5654d98c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.538+0000 7f06cb400640 1 -- 172.21.15.2:0/1643479136 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d9535210 con 0x5654d98bd260 2024-09-10T08:16:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.538+0000 7f06caa00640 1 --2- 172.21.15.2:0/1643479136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98bdba0 0x5654d98c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:16:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.538+0000 7f06cb400640 1 --2- 172.21.15.2:0/1643479136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bd260 0x5654d98bd660 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f06c40117e0 tx=0x7f06c4011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3be10af877ab160d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.538+0000 7f06d0a00640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06b00a2b80 0x7f06b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06d0a00640 1 -- 172.21.15.2:0/3923471028 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f06bc0ef940 con 0x7f06cc076bb0 2024-09-10T08:16:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06b7e00640 1 -- 172.21.15.2:0/1643479136 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4011310 con 0x5654d98bd260 2024-09-10T08:16:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06b7e00640 1 -- 172.21.15.2:0/1643479136 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c4003e50 con 0x5654d98bd260 2024-09-10T08:16:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06b7e00640 1 -- 172.21.15.2:0/1643479136 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4004620 con 0x5654d98bd260 2024-09-10T08:16:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06caa00640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06b00a2b80 0x7f06b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06d4bbd740 1 -- 172.21.15.2:0/1643479136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bd260 msgr2=0x5654d98bd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/1643479136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bd260 0x5654d98bd660 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f06c40117e0 tx=0x7f06c4011ca0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06caa00640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06b00a2b80 0x7f06b00a5040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0bbb00 tx=0x7f06bc02a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06d4bbd740 1 -- 172.21.15.2:0/1643479136 shutdown_connections 2024-09-10T08:16:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/1643479136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98c2c30 0x5654d98c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/1643479136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98bdba0 0x5654d98c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/1643479136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bd260 0x5654d98bd660 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06d4bbd740 1 -- 172.21.15.2:0/1643479136 >> 172.21.15.2:0/1643479136 conn(0x5654d98b88d0 msgr2=0x5654d98b8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06d4bbd740 1 -- 172.21.15.2:0/1643479136 shutdown_connections 2024-09-10T08:16:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.539+0000 7f06d4bbd740 1 -- 172.21.15.2:0/1643479136 wait complete. 2024-09-10T08:16:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.540+0000 7f06d4bbd740 1 -- start start 2024-09-10T08:16:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.540+0000 7f06d4bbd740 1 Processor -- start 2024-09-10T08:16:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.540+0000 7f06d4bbd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98c7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.540+0000 7f06d4bbd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bdba0 0x5654d98c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.540+0000 7f06d4bbd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98c2c30 0x5654d98ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.540+0000 7f06d4bbd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5654d98c5c00 con 0x5654d98bd260 2024-09-10T08:16:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.540+0000 7f06d4bbd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5654d97ab090 con 0x5654d98bdba0 2024-09-10T08:16:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.540+0000 7f06d4bbd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5654d98c5d80 con 0x5654d98c2c30 2024-09-10T08:16:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.540+0000 7f06d4bbd740 10 client.0 _ll_register_callbacks cb 0x5654d97fff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.541+0000 7f06ca000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bdba0 0x5654d98c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.541+0000 7f06ca000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bdba0 0x5654d98c7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:34990/0 (socket says 172.21.15.2:34990) 2024-09-10T08:16:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.541+0000 7f06ca000640 1 -- 172.21.15.2:0/3658423888 learned_addr learned my addr 172.21.15.2:0/3658423888 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.541+0000 7f06cb400640 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.541+0000 7f06cb400640 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98c2c30 0x5654d98ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.541+0000 7f06ca000640 1 -- 172.21.15.2:0/3658423888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98c2c30 msgr2=0x5654d98ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.541+0000 7f06ca000640 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98c2c30 0x5654d98ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.541+0000 7f06ca000640 1 -- 172.21.15.2:0/3658423888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 msgr2=0x5654d98c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.541+0000 7f06ca000640 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.541+0000 7f06ca000640 1 -- 172.21.15.2:0/3658423888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d9601620 con 0x5654d98bdba0 2024-09-10T08:16:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.541+0000 7f06cb400640 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98c2c30 0x5654d98ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:16:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.541+0000 7f06ca000640 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bdba0 0x5654d98c7980 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f06b8012700 tx=0x7f06b8012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=295a736dbd3713de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.541+0000 7f06b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.542+0000 7f06b4c00640 1 -- 172.21.15.2:0/3658423888 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8004050 con 0x5654d98bdba0 2024-09-10T08:16:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.542+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3658423888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06c4003e50 con 0x5654d98bdba0 2024-09-10T08:16:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.542+0000 7f06b4c00640 1 -- 172.21.15.2:0/3658423888 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f06b80041f0 con 0x5654d98bdba0 2024-09-10T08:16:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.542+0000 7f06b4c00640 1 -- 172.21.15.2:0/3658423888 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f06b8025050 con 0x5654d98bdba0 2024-09-10T08:16:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.542+0000 7f06d4bbd740 10 client.27719 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.542+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3658423888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5654d98d18d0 con 0x5654d98bdba0 2024-09-10T08:16:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.542+0000 7f06b4c00640 1 -- 172.21.15.2:0/3658423888 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c4003e50 con 0x5654d98bdba0 2024-09-10T08:16:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.543+0000 7f06b4c00640 1 -- 172.21.15.2:0/3658423888 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f06b8003a30 con 0x5654d98bdba0 2024-09-10T08:16:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.543+0000 7f06d4bbd740 10 client.27719 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.543+0000 7f06d4bbd740 10 client.27719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.543+0000 7f06d4bbd740 20 client.27719 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.543+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654d98d2f50 0x5654d98d5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.543+0000 7f06d4bbd740 4 client.27719 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.543+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3658423888 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654d982f830 con 0x5654d98d2f50 2024-09-10T08:16:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.543+0000 7f06cb400640 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654d98d2f50 0x5654d98d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.544+0000 7f06cb400640 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654d98d2f50 0x5654d98d5360 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cabb244bddaec269 server_cookie=6fab151d5343edd1 in_seq=0 out_seq=0 2024-09-10T08:16:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.544+0000 7f06b4c00640 10 client.27719 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.544+0000 7f06b4c00640 1 -- 172.21.15.2:0/3658423888 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654d982f830 con 0x5654d98d2f50 2024-09-10T08:16:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.544+0000 7f06b4c00640 10 client.27719 handle_command_reply: tid=0 2024-09-10T08:16:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.545+0000 7f06d4bbd740 1 client.27719 shutdown 2024-09-10T08:16:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.545+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3658423888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bdba0 msgr2=0x5654d98c7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.545+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bdba0 0x5654d98c7980 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f06b8012700 tx=0x7f06b8012bc0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.545+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3658423888 shutdown_connections 2024-09-10T08:16:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.545+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654d98d2f50 0x5654d98d5360 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.545+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98c2c30 0x5654d98ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.545+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bdba0 0x5654d98c7980 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.545+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/3658423888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.545+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3658423888 >> 172.21.15.2:0/3658423888 conn(0x5654d98b88d0 msgr2=0x5654d98ba0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.545+0000 7f06b4c00640 0 client.27719 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.545+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3658423888 shutdown_connections 2024-09-10T08:16:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.545+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3658423888 wait complete. 2024-09-10T08:16:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.545+0000 7f06d4bbd740 20 client.27719 trim_cache size 0 max 16384 2024-09-10T08:16:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06d4bbd740 1 Processor -- start 2024-09-10T08:16:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06d4bbd740 1 -- start start 2024-09-10T08:16:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06d4bbd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98bd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06d4bbd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98bdba0 0x5654d98ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06d4bbd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98c2c30 0x5654d98b7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06d4bbd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc1b35a0 con 0x5654d98bd260 2024-09-10T08:16:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06d4bbd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc1b0e70 con 0x5654d98c2c30 2024-09-10T08:16:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06d4bbd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc1aecc0 con 0x5654d98bdba0 2024-09-10T08:16:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06cb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98bd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06ca000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98bdba0 0x5654d98ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06cb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98bd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37740/0 (socket says 172.21.15.2:37740) 2024-09-10T08:16:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06cb400640 1 -- 172.21.15.2:0/1501607183 learned_addr learned my addr 172.21.15.2:0/1501607183 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06ca000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98bdba0 0x5654d98ce980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54280/0 (socket says 172.21.15.2:54280) 2024-09-10T08:16:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06cb400640 1 -- 172.21.15.2:0/1501607183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98bdba0 msgr2=0x5654d98ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06cb400640 1 --2- 172.21.15.2:0/1501607183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98bdba0 0x5654d98ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.561+0000 7f06cb400640 1 -- 172.21.15.2:0/1501607183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98c2c30 msgr2=0x5654d98b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.562+0000 7f06cb400640 1 --2- 172.21.15.2:0/1501607183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98c2c30 0x5654d98b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.562+0000 7f06cb400640 1 -- 172.21.15.2:0/1501607183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d98b86f0 con 0x5654d98bd260 2024-09-10T08:16:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.562+0000 7f06cb400640 1 --2- 172.21.15.2:0/1501607183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98bd640 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f06c4005c40 tx=0x7f06c4003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16b247dfca7b930 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.562+0000 7f06b7400640 1 -- 172.21.15.2:0/1501607183 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4023040 con 0x5654d98bd260 2024-09-10T08:16:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.562+0000 7f06b7400640 1 -- 172.21.15.2:0/1501607183 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c400fe20 con 0x5654d98bd260 2024-09-10T08:16:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.562+0000 7f06b7400640 1 -- 172.21.15.2:0/1501607183 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4029300 con 0x5654d98bd260 2024-09-10T08:16:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.563+0000 7f06d4bbd740 1 -- 172.21.15.2:0/1501607183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 msgr2=0x5654d98bd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.563+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/1501607183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98bd640 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f06c4005c40 tx=0x7f06c4003a90 comp rx=0 tx=0).stop 2024-09-10T08:16:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.563+0000 7f06d4bbd740 1 -- 172.21.15.2:0/1501607183 shutdown_connections 2024-09-10T08:16:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.563+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/1501607183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98c2c30 0x5654d98b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.563+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/1501607183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98bdba0 0x5654d98ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.563+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/1501607183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98bd640 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.563+0000 7f06d4bbd740 1 -- 172.21.15.2:0/1501607183 >> 172.21.15.2:0/1501607183 conn(0x5654d98b88d0 msgr2=0x5654d98b71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.563+0000 7f06d4bbd740 1 -- 172.21.15.2:0/1501607183 shutdown_connections 2024-09-10T08:16:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.563+0000 7f06d4bbd740 1 -- 172.21.15.2:0/1501607183 wait complete. 2024-09-10T08:16:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.563+0000 7f06d4bbd740 1 -- start start 2024-09-10T08:16:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.564+0000 7f06d4bbd740 1 Processor -- start 2024-09-10T08:16:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.564+0000 7f06d4bbd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.564+0000 7f06d4bbd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bdba0 0x5654d98cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.564+0000 7f06d4bbd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98c2c30 0x5654d98cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.564+0000 7f06d4bbd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc1af330 con 0x5654d98bd260 2024-09-10T08:16:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.564+0000 7f06d4bbd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc1b07a0 con 0x5654d98bdba0 2024-09-10T08:16:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.564+0000 7f06d4bbd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc1afa00 con 0x5654d98c2c30 2024-09-10T08:16:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.564+0000 7f06d4bbd740 10 client.0 _ll_register_callbacks cb 0x7f06b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.564+0000 7f06cb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.564+0000 7f06cb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37744/0 (socket says 172.21.15.2:37744) 2024-09-10T08:16:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.564+0000 7f06cb400640 1 -- 172.21.15.2:0/3790390242 learned_addr learned my addr 172.21.15.2:0/3790390242 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.564+0000 7f06ca000640 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bdba0 0x5654d98cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.565+0000 7f06cb400640 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98c2c30 0x5654d98cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.565+0000 7f06ca000640 1 -- 172.21.15.2:0/3790390242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98c2c30 msgr2=0x5654d98cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.565+0000 7f06ca000640 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98c2c30 0x5654d98cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.565+0000 7f06ca000640 1 -- 172.21.15.2:0/3790390242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 msgr2=0x5654d98cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.565+0000 7f06ca000640 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.565+0000 7f06ca000640 1 -- 172.21.15.2:0/3790390242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d98cd920 con 0x5654d98bdba0 2024-09-10T08:16:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.565+0000 7f06cb400640 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.565+0000 7f06ca000640 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bdba0 0x5654d98cc300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f06b8003e50 tx=0x7f06b800f920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad855ee3685b5e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.565+0000 7f06b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.565+0000 7f06b4c00640 1 -- 172.21.15.2:0/3790390242 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b800fde0 con 0x5654d98bdba0 2024-09-10T08:16:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.565+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3790390242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06c400fe20 con 0x5654d98bdba0 2024-09-10T08:16:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.565+0000 7f06b4c00640 1 -- 172.21.15.2:0/3790390242 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f06b8022030 con 0x5654d98bdba0 2024-09-10T08:16:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.566+0000 7f06b4c00640 1 -- 172.21.15.2:0/3790390242 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f06b803a050 con 0x5654d98bdba0 2024-09-10T08:16:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.566+0000 7f06d4bbd740 10 client.27722 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.566+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3790390242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5654d98ceca0 con 0x5654d98bdba0 2024-09-10T08:16:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.566+0000 7f06b4c00640 1 -- 172.21.15.2:0/3790390242 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c400fe20 con 0x5654d98bdba0 2024-09-10T08:16:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.566+0000 7f06b4c00640 1 -- 172.21.15.2:0/3790390242 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f06b8010480 con 0x5654d98bdba0 2024-09-10T08:16:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.566+0000 7f06d4bbd740 10 client.27722 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.566+0000 7f06d4bbd740 10 client.27722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.566+0000 7f06d4bbd740 20 client.27722 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.566+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654d993e300 0x5654d9940720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.566+0000 7f06d4bbd740 4 client.27722 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:16:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.566+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3790390242 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654d9829b80 con 0x5654d993e300 2024-09-10T08:16:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.567+0000 7f06cb400640 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654d993e300 0x5654d9940720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.567+0000 7f06cb400640 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654d993e300 0x5654d9940720 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee759dc61dbfb3f8 server_cookie=8876f557d1f10950 in_seq=0 out_seq=0 2024-09-10T08:16:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.567+0000 7f06b4c00640 10 client.27722 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06b4c00640 1 -- 172.21.15.2:0/3790390242 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654d9829b80 con 0x5654d993e300 2024-09-10T08:16:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06b4c00640 10 client.27722 handle_command_reply: tid=0 2024-09-10T08:16:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06d4bbd740 1 client.27722 shutdown 2024-09-10T08:16:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3790390242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bdba0 msgr2=0x5654d98cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bdba0 0x5654d98cc300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f06b8003e50 tx=0x7f06b800f920 comp rx=0 tx=0).stop 2024-09-10T08:16:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3790390242 shutdown_connections 2024-09-10T08:16:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654d993e300 0x5654d9940720 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654d98c2c30 0x5654d98cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06b4c00640 0 client.27722 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654d98bdba0 0x5654d98cc300 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06d4bbd740 1 --2- 172.21.15.2:0/3790390242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654d98bd260 0x5654d98cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3790390242 >> 172.21.15.2:0/3790390242 conn(0x5654d98b88d0 msgr2=0x5654d98b71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3790390242 shutdown_connections 2024-09-10T08:16:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06d4bbd740 1 -- 172.21.15.2:0/3790390242 wait complete. 2024-09-10T08:16:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.583+0000 7f06d4bbd740 20 client.27722 trim_cache size 0 max 16384 2024-09-10T08:16:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:16:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:16:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:16:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:16:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.584+0000 7f06cbe00640 1 -- 172.21.15.2:0/3923471028 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06b00a2b80 msgr2=0x7f06b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.584+0000 7f06cbe00640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06b00a2b80 0x7f06b00a5040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0bbb00 tx=0x7f06bc02a410 comp rx=0 tx=0).stop 2024-09-10T08:16:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.584+0000 7f06cbe00640 1 -- 172.21.15.2:0/3923471028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc076bb0 msgr2=0x7f06cc19e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.584+0000 7f06cbe00640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc076bb0 0x7f06cc19e280 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f06bc031440 tx=0x7f06bc032ce0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.584+0000 7f06cbe00640 1 -- 172.21.15.2:0/3923471028 shutdown_connections 2024-09-10T08:16:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.584+0000 7f06cbe00640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06cc078790 0x7f06cc1a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.584+0000 7f06cbe00640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06b00a2b80 0x7f06b00a5040 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.584+0000 7f06cbe00640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06cc077df0 0x7f06cc19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.584+0000 7f06cbe00640 1 --2- 172.21.15.2:0/3923471028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc076bb0 0x7f06cc19e280 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.584+0000 7f06cbe00640 1 -- 172.21.15.2:0/3923471028 >> 172.21.15.2:0/3923471028 conn(0x7f06cc0ff310 msgr2=0x7f06cc100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.584+0000 7f06cbe00640 1 -- 172.21.15.2:0/3923471028 shutdown_connections 2024-09-10T08:16:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:37.584+0000 7f06cbe00640 1 -- 172.21.15.2:0/3923471028 wait complete. 2024-09-10T08:16:37.621 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:16:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:37 smithi155 ceph-mon[35405]: pgmap v641: 129 pgs: 129 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:16:37.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:37 smithi107 ceph-mon[35048]: pgmap v641: 129 pgs: 129 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:16:38.003 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:37 smithi002 ceph-mon[30563]: pgmap v641: 129 pgs: 129 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:16:38.046 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-10T08:16:38.046 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-10T08:16:38.139 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:38.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.762+0000 7f9605800640 1 -- 172.21.15.2:0/2996916157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96000745c0 msgr2=0x7f96000749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:38.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.762+0000 7f9605800640 1 --2- 172.21.15.2:0/2996916157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96000745c0 0x7f96000749a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009900 tx=0x7f95f002a480 comp rx=0 tx=0).stop 2024-09-10T08:16:38.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.762+0000 7f9605800640 1 -- 172.21.15.2:0/2996916157 shutdown_connections 2024-09-10T08:16:38.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.762+0000 7f9605800640 1 --2- 172.21.15.2:0/2996916157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960010e260 0x7f9600110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.762+0000 7f9605800640 1 --2- 172.21.15.2:0/2996916157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9600074ee0 0x7f960010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.762+0000 7f9605800640 1 --2- 172.21.15.2:0/2996916157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96000745c0 0x7f96000749a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.762+0000 7f9605800640 1 -- 172.21.15.2:0/2996916157 >> 172.21.15.2:0/2996916157 conn(0x7f96000fd740 msgr2=0x7f96000ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:38.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.762+0000 7f9605800640 1 -- 172.21.15.2:0/2996916157 shutdown_connections 2024-09-10T08:16:38.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.762+0000 7f9605800640 1 -- 172.21.15.2:0/2996916157 wait complete. 2024-09-10T08:16:38.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.763+0000 7f9605800640 1 Processor -- start 2024-09-10T08:16:38.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.763+0000 7f9605800640 1 -- start start 2024-09-10T08:16:38.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.764+0000 7f9605800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96000745c0 0x7f960019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.764+0000 7f9605800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9600074ee0 0x7f960019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.764+0000 7f9605800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f960010e260 0x7f960019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.764+0000 7f9605800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9600114000 con 0x7f9600074ee0 2024-09-10T08:16:38.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.764+0000 7f9605800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9600113e80 con 0x7f96000745c0 2024-09-10T08:16:38.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.764+0000 7f9605800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9600114180 con 0x7f960010e260 2024-09-10T08:16:38.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.764+0000 7f95f7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9600074ee0 0x7f960019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.764+0000 7f9604e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f960010e260 0x7f960019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.764+0000 7f95f7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9600074ee0 0x7f960019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37770/0 (socket says 172.21.15.2:37770) 2024-09-10T08:16:38.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.764+0000 7f95f7e00640 1 -- 172.21.15.2:0/241051882 learned_addr learned my addr 172.21.15.2:0/241051882 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:38.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.764+0000 7f95ffe00640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96000745c0 0x7f960019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.765+0000 7f9604e00640 1 -- 172.21.15.2:0/241051882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96000745c0 msgr2=0x7f960019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:38.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.765+0000 7f9604e00640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96000745c0 0x7f960019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.765+0000 7f9604e00640 1 -- 172.21.15.2:0/241051882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9600074ee0 msgr2=0x7f960019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:38.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.765+0000 7f9604e00640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9600074ee0 0x7f960019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.765+0000 7f9604e00640 1 -- 172.21.15.2:0/241051882 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960019f460 con 0x7f960010e260 2024-09-10T08:16:38.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.765+0000 7f95f7e00640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9600074ee0 0x7f960019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:38.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.765+0000 7f9604e00640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f960010e260 0x7f960019ed80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f95f8005fd0 tx=0x7f95f800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:38.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.765+0000 7f95fd600640 1 -- 172.21.15.2:0/241051882 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f800cdf0 con 0x7f960010e260 2024-09-10T08:16:38.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.765+0000 7f9605800640 1 -- 172.21.15.2:0/241051882 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960019f750 con 0x7f960010e260 2024-09-10T08:16:38.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.765+0000 7f95fd600640 1 -- 172.21.15.2:0/241051882 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95f8005020 con 0x7f960010e260 2024-09-10T08:16:38.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.765+0000 7f95fd600640 1 -- 172.21.15.2:0/241051882 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f80106d0 con 0x7f960010e260 2024-09-10T08:16:38.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.765+0000 7f9605800640 1 -- 172.21.15.2:0/241051882 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96001a7030 con 0x7f960010e260 2024-09-10T08:16:38.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.769+0000 7f96099f1740 1 Processor -- start 2024-09-10T08:16:38.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.770+0000 7f95fd600640 1 -- 172.21.15.2:0/241051882 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f95f80054d0 con 0x7f960010e260 2024-09-10T08:16:38.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.770+0000 7f96099f1740 1 -- start start 2024-09-10T08:16:38.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.771+0000 7f96099f1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcee40 0x55c1a6fcf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.771+0000 7f96099f1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcf760 0x55c1a6fd9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.771+0000 7f96099f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fda020 0x55c1a6fdc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.771+0000 7f96099f1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c1a6f353e0 con 0x55c1a6fcf760 2024-09-10T08:16:38.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.771+0000 7f96099f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c1a6edcd80 con 0x55c1a6fda020 2024-09-10T08:16:38.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.771+0000 7f96099f1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c1a6f313a0 con 0x55c1a6fcee40 2024-09-10T08:16:38.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.771+0000 7f95ffe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcf760 0x55c1a6fd9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.771+0000 7f95f7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcee40 0x55c1a6fcf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.771+0000 7f95f7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcee40 0x55c1a6fcf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54316/0 (socket says 172.21.15.2:54316) 2024-09-10T08:16:38.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.771+0000 7f95f7e00640 1 -- 172.21.15.2:0/359941095 learned_addr learned my addr 172.21.15.2:0/359941095 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:38.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.771+0000 7f95f7e00640 1 --2- 172.21.15.2:0/359941095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fda020 0x55c1a6fdc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.772+0000 7f95f7e00640 1 -- 172.21.15.2:0/359941095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fda020 msgr2=0x55c1a6fdc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:38.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.772+0000 7f95f7e00640 1 --2- 172.21.15.2:0/359941095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fda020 0x55c1a6fdc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.772+0000 7f95f7e00640 1 -- 172.21.15.2:0/359941095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcf760 msgr2=0x55c1a6fd9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:38.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.772+0000 7f95f7e00640 1 --2- 172.21.15.2:0/359941095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcf760 0x55c1a6fd9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.772+0000 7f95f7e00640 1 -- 172.21.15.2:0/359941095 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a6efbae0 con 0x55c1a6fcee40 2024-09-10T08:16:38.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.772+0000 7f95f7e00640 1 --2- 172.21.15.2:0/359941095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcee40 0x55c1a6fcf220 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f95e8011c70 tx=0x7f95e8009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1b908f86f1632c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:38.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.772+0000 7f95fd600640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f95e40a2c00 0x7f95e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.772+0000 7f95fd600640 1 -- 172.21.15.2:0/241051882 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f95f80cfa60 con 0x7f960010e260 2024-09-10T08:16:38.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.772+0000 7f95f5600640 1 -- 172.21.15.2:0/359941095 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e800fc50 con 0x55c1a6fcee40 2024-09-10T08:16:38.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.773+0000 7f95f5600640 1 -- 172.21.15.2:0/359941095 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e800fdf0 con 0x55c1a6fcee40 2024-09-10T08:16:38.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.773+0000 7f95f5600640 1 -- 172.21.15.2:0/359941095 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e800e6f0 con 0x55c1a6fcee40 2024-09-10T08:16:38.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.773+0000 7f95ffe00640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f95e40a2c00 0x7f95e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.773+0000 7f96099f1740 1 -- 172.21.15.2:0/359941095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcee40 msgr2=0x55c1a6fcf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:38.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.773+0000 7f96099f1740 1 --2- 172.21.15.2:0/359941095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcee40 0x55c1a6fcf220 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f95e8011c70 tx=0x7f95e8009c60 comp rx=0 tx=0).stop 2024-09-10T08:16:38.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.773+0000 7f95ffe00640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f95e40a2c00 0x7f95e40a50c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f95f0000c00 tx=0x7f95f0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:38.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.774+0000 7f96099f1740 1 -- 172.21.15.2:0/359941095 shutdown_connections 2024-09-10T08:16:38.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.774+0000 7f96099f1740 1 --2- 172.21.15.2:0/359941095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fda020 0x55c1a6fdc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.774+0000 7f96099f1740 1 --2- 172.21.15.2:0/359941095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcf760 0x55c1a6fd9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.774+0000 7f96099f1740 1 --2- 172.21.15.2:0/359941095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcee40 0x55c1a6fcf220 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.774+0000 7f96099f1740 1 -- 172.21.15.2:0/359941095 >> 172.21.15.2:0/359941095 conn(0x55c1a6fcb3d0 msgr2=0x55c1a6fcb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:38.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.774+0000 7f96099f1740 1 -- 172.21.15.2:0/359941095 shutdown_connections 2024-09-10T08:16:38.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.774+0000 7f96099f1740 1 -- 172.21.15.2:0/359941095 wait complete. 2024-09-10T08:16:38.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.774+0000 7f96099f1740 1 -- start start 2024-09-10T08:16:38.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.775+0000 7f96099f1740 1 Processor -- start 2024-09-10T08:16:38.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.775+0000 7f96099f1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcee40 0x55c1a6fd6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.775+0000 7f96099f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fcf760 0x55c1a6fd74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.775+0000 7f96099f1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fda020 0x55c1a6fd7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.775+0000 7f96099f1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c1a6fdcbf0 con 0x55c1a6fcee40 2024-09-10T08:16:38.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.775+0000 7f96099f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c1a6ef00f0 con 0x55c1a6fcf760 2024-09-10T08:16:38.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.775+0000 7f96099f1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c1a6fdcd70 con 0x55c1a6fda020 2024-09-10T08:16:38.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.775+0000 7f96099f1740 10 client.0 _ll_register_callbacks cb 0x55c1a6ed7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:38.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.775+0000 7f95ffe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcee40 0x55c1a6fd6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.775+0000 7f95ffe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcee40 0x55c1a6fd6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37792/0 (socket says 172.21.15.2:37792) 2024-09-10T08:16:38.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.775+0000 7f95ffe00640 1 -- 172.21.15.2:0/602758830 learned_addr learned my addr 172.21.15.2:0/602758830 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:38.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.775+0000 7f95f7e00640 1 --2- 172.21.15.2:0/602758830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fcf760 0x55c1a6fd74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.776+0000 7f9604e00640 1 --2- 172.21.15.2:0/602758830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fda020 0x55c1a6fd7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.776+0000 7f95f7e00640 1 -- 172.21.15.2:0/602758830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fda020 msgr2=0x55c1a6fd7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:38.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.776+0000 7f95f7e00640 1 --2- 172.21.15.2:0/602758830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fda020 0x55c1a6fd7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.776+0000 7f95f7e00640 1 -- 172.21.15.2:0/602758830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcee40 msgr2=0x55c1a6fd6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:38.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.776+0000 7f95f7e00640 1 --2- 172.21.15.2:0/602758830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcee40 0x55c1a6fd6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.776+0000 7f95f7e00640 1 -- 172.21.15.2:0/602758830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a6ca63c0 con 0x55c1a6fcf760 2024-09-10T08:16:38.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.776+0000 7f95f7e00640 1 --2- 172.21.15.2:0/602758830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fcf760 0x55c1a6fd74e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f95e801e000 tx=0x7f95e8005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9386bfeb192b1dcc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:38.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.776+0000 7f95ee000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:38.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.777+0000 7f95ee000640 1 -- 172.21.15.2:0/602758830 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8005640 con 0x55c1a6fcf760 2024-09-10T08:16:38.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.777+0000 7f95ee000640 1 -- 172.21.15.2:0/602758830 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f95e8004020 con 0x55c1a6fcf760 2024-09-10T08:16:38.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.777+0000 7f96099f1740 1 -- 172.21.15.2:0/602758830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e800fdf0 con 0x55c1a6fcf760 2024-09-10T08:16:38.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.777+0000 7f95ee000640 1 -- 172.21.15.2:0/602758830 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f95e8004220 con 0x55c1a6fcf760 2024-09-10T08:16:38.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.777+0000 7f96099f1740 10 client.27734 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:38.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.777+0000 7f96099f1740 1 -- 172.21.15.2:0/602758830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c1a6fe8cb0 con 0x55c1a6fcf760 2024-09-10T08:16:38.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.777+0000 7f95ee000640 1 -- 172.21.15.2:0/602758830 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e800fdf0 con 0x55c1a6fcf760 2024-09-10T08:16:38.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.778+0000 7f95ee000640 1 -- 172.21.15.2:0/602758830 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f95e801d020 con 0x55c1a6fcf760 2024-09-10T08:16:38.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.778+0000 7f96099f1740 10 client.27734 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:38.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.778+0000 7f96099f1740 10 client.27734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:38.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.778+0000 7f96099f1740 20 client.27734 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:38.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.778+0000 7f96099f1740 1 --2- 172.21.15.2:0/602758830 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c1a6fea000 0x55c1a6fec3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.778+0000 7f96099f1740 4 client.27734 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:38.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.778+0000 7f96099f1740 1 -- 172.21.15.2:0/602758830 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1a6f42950 con 0x55c1a6fea000 2024-09-10T08:16:38.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.778+0000 7f95ffe00640 1 --2- 172.21.15.2:0/602758830 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c1a6fea000 0x55c1a6fec3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.779+0000 7f95ffe00640 1 --2- 172.21.15.2:0/602758830 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c1a6fea000 0x55c1a6fec3f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3c3a61ef4929722 server_cookie=4d928818680408f in_seq=0 out_seq=0 2024-09-10T08:16:38.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.779+0000 7f95ee000640 10 client.27734 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:38.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.779+0000 7f95ee000640 1 -- 172.21.15.2:0/602758830 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1a6f42950 con 0x55c1a6fea000 2024-09-10T08:16:38.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.779+0000 7f95ee000640 10 client.27734 handle_command_reply: tid=0 2024-09-10T08:16:38.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.780+0000 7f96099f1740 1 client.27734 shutdown 2024-09-10T08:16:38.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.780+0000 7f96099f1740 1 -- 172.21.15.2:0/602758830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fcf760 msgr2=0x55c1a6fd74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:38.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.780+0000 7f96099f1740 1 --2- 172.21.15.2:0/602758830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fcf760 0x55c1a6fd74e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f95e801e000 tx=0x7f95e8005970 comp rx=0 tx=0).stop 2024-09-10T08:16:38.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.780+0000 7f96099f1740 1 -- 172.21.15.2:0/602758830 shutdown_connections 2024-09-10T08:16:38.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.780+0000 7f96099f1740 1 --2- 172.21.15.2:0/602758830 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c1a6fea000 0x55c1a6fec3f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.780+0000 7f96099f1740 1 --2- 172.21.15.2:0/602758830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fda020 0x55c1a6fd7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.780+0000 7f96099f1740 1 --2- 172.21.15.2:0/602758830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fcf760 0x55c1a6fd74e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.780+0000 7f96099f1740 1 --2- 172.21.15.2:0/602758830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcee40 0x55c1a6fd6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.780+0000 7f96099f1740 1 -- 172.21.15.2:0/602758830 >> 172.21.15.2:0/602758830 conn(0x55c1a6fcb3d0 msgr2=0x55c1a6fd1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:38.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.780+0000 7f95ee000640 0 client.27734 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:38.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.780+0000 7f96099f1740 1 -- 172.21.15.2:0/602758830 shutdown_connections 2024-09-10T08:16:38.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.780+0000 7f96099f1740 1 -- 172.21.15.2:0/602758830 wait complete. 2024-09-10T08:16:38.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.780+0000 7f96099f1740 20 client.27734 trim_cache size 0 max 16384 2024-09-10T08:16:38.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.796+0000 7f96099f1740 1 Processor -- start 2024-09-10T08:16:38.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.796+0000 7f96099f1740 1 -- start start 2024-09-10T08:16:38.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.797+0000 7f96099f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fcee40 0x55c1a6fd18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.797+0000 7f96099f1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcf760 0x55c1a6ff1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.797+0000 7f96099f1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fda020 0x55c1a6ff1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.797+0000 7f96099f1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96001b3760 con 0x55c1a6fda020 2024-09-10T08:16:38.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.797+0000 7f96099f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96001b1030 con 0x55c1a6fcee40 2024-09-10T08:16:38.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.797+0000 7f96099f1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96001aee80 con 0x55c1a6fcf760 2024-09-10T08:16:38.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.797+0000 7f9604e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fda020 0x55c1a6ff1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.797+0000 7f9604e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fda020 0x55c1a6ff1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37804/0 (socket says 172.21.15.2:37804) 2024-09-10T08:16:38.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.797+0000 7f95ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fcee40 0x55c1a6fd18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.797+0000 7f9604e00640 1 -- 172.21.15.2:0/2560758153 learned_addr learned my addr 172.21.15.2:0/2560758153 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:38.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.797+0000 7f95ffe00640 1 -- 172.21.15.2:0/2560758153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcf760 msgr2=0x55c1a6ff1a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:16:38.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.797+0000 7f95ffe00640 1 --2- 172.21.15.2:0/2560758153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcf760 0x55c1a6ff1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.798+0000 7f95ffe00640 1 -- 172.21.15.2:0/2560758153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fda020 msgr2=0x55c1a6ff1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:38.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.798+0000 7f95ffe00640 1 --2- 172.21.15.2:0/2560758153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fda020 0x55c1a6ff1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.798+0000 7f95ffe00640 1 -- 172.21.15.2:0/2560758153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a6fdc410 con 0x55c1a6fcee40 2024-09-10T08:16:38.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.798+0000 7f9604e00640 1 --2- 172.21.15.2:0/2560758153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fda020 0x55c1a6ff1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:38.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.798+0000 7f95ffe00640 1 --2- 172.21.15.2:0/2560758153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fcee40 0x55c1a6fd18a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55c1a6fec9f0 tx=0x7f95f003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b718e292a92c2b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:38.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.798+0000 7f95f4c00640 1 -- 172.21.15.2:0/2560758153 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0035040 con 0x55c1a6fcee40 2024-09-10T08:16:38.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.798+0000 7f95f4c00640 1 -- 172.21.15.2:0/2560758153 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95f005a070 con 0x55c1a6fcee40 2024-09-10T08:16:38.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.798+0000 7f95f4c00640 1 -- 172.21.15.2:0/2560758153 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f00035f0 con 0x55c1a6fcee40 2024-09-10T08:16:38.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.799+0000 7f96099f1740 1 -- 172.21.15.2:0/2560758153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fcee40 msgr2=0x55c1a6fd18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:38.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.799+0000 7f96099f1740 1 --2- 172.21.15.2:0/2560758153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fcee40 0x55c1a6fd18a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55c1a6fec9f0 tx=0x7f95f003f000 comp rx=0 tx=0).stop 2024-09-10T08:16:38.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.799+0000 7f96099f1740 1 -- 172.21.15.2:0/2560758153 shutdown_connections 2024-09-10T08:16:38.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.799+0000 7f96099f1740 1 --2- 172.21.15.2:0/2560758153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fda020 0x55c1a6ff1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.799+0000 7f96099f1740 1 --2- 172.21.15.2:0/2560758153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcf760 0x55c1a6ff1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.799+0000 7f96099f1740 1 --2- 172.21.15.2:0/2560758153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fcee40 0x55c1a6fd18a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.799+0000 7f96099f1740 1 -- 172.21.15.2:0/2560758153 >> 172.21.15.2:0/2560758153 conn(0x55c1a6fcb3d0 msgr2=0x55c1a6fd9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:38.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.799+0000 7f96099f1740 1 -- 172.21.15.2:0/2560758153 shutdown_connections 2024-09-10T08:16:38.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.799+0000 7f96099f1740 1 -- 172.21.15.2:0/2560758153 wait complete. 2024-09-10T08:16:38.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.800+0000 7f96099f1740 1 -- start start 2024-09-10T08:16:38.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.800+0000 7f96099f1740 1 Processor -- start 2024-09-10T08:16:38.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.800+0000 7f96099f1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcee40 0x55c1a6fe2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.800+0000 7f96099f1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcf760 0x55c1a6fe28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.800+0000 7f96099f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fda020 0x55c1a6fdd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.800+0000 7f96099f1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96001af4f0 con 0x55c1a6fcee40 2024-09-10T08:16:38.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.800+0000 7f96099f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96001b0960 con 0x55c1a6fda020 2024-09-10T08:16:38.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.800+0000 7f96099f1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96001afbc0 con 0x55c1a6fcf760 2024-09-10T08:16:38.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.800+0000 7f96099f1740 10 client.0 _ll_register_callbacks cb 0x7f95e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:38.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.801+0000 7f95f7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcf760 0x55c1a6fe28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.801+0000 7f95f7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcf760 0x55c1a6fe28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54344/0 (socket says 172.21.15.2:54344) 2024-09-10T08:16:38.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.801+0000 7f95f7e00640 1 -- 172.21.15.2:0/3757908589 learned_addr learned my addr 172.21.15.2:0/3757908589 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:38.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.801+0000 7f9604e00640 1 --2- 172.21.15.2:0/3757908589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fda020 0x55c1a6fdd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.801+0000 7f95ffe00640 1 --2- 172.21.15.2:0/3757908589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcee40 0x55c1a6fe2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.801+0000 7f95f7e00640 1 -- 172.21.15.2:0/3757908589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fda020 msgr2=0x55c1a6fdd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:38.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.801+0000 7f95f7e00640 1 --2- 172.21.15.2:0/3757908589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fda020 0x55c1a6fdd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.801+0000 7f95f7e00640 1 -- 172.21.15.2:0/3757908589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcee40 msgr2=0x55c1a6fe2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:38.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.801+0000 7f95f7e00640 1 --2- 172.21.15.2:0/3757908589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcee40 0x55c1a6fe2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.801+0000 7f95f7e00640 1 -- 172.21.15.2:0/3757908589 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a6fde100 con 0x55c1a6fcf760 2024-09-10T08:16:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.802+0000 7f95f7e00640 1 --2- 172.21.15.2:0/3757908589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcf760 0x55c1a6fe28c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f95e80037f0 tx=0x7f95e8003c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69b6bc3d1153b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.802+0000 7f95ee000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:16:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.802+0000 7f95ee000640 1 -- 172.21.15.2:0/3757908589 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8005640 con 0x55c1a6fcf760 2024-09-10T08:16:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.802+0000 7f96099f1740 1 -- 172.21.15.2:0/3757908589 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95f005a070 con 0x55c1a6fcf760 2024-09-10T08:16:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.802+0000 7f95ee000640 1 -- 172.21.15.2:0/3757908589 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f95e8011070 con 0x55c1a6fcf760 2024-09-10T08:16:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.802+0000 7f95ee000640 1 -- 172.21.15.2:0/3757908589 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f95e80112b0 con 0x55c1a6fcf760 2024-09-10T08:16:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.802+0000 7f96099f1740 10 client.27679 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.802+0000 7f96099f1740 1 -- 172.21.15.2:0/3757908589 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c1a705cee0 con 0x55c1a6fcf760 2024-09-10T08:16:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.803+0000 7f95ee000640 1 -- 172.21.15.2:0/3757908589 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95f005a070 con 0x55c1a6fcf760 2024-09-10T08:16:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.803+0000 7f95ee000640 1 -- 172.21.15.2:0/3757908589 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f95e8036060 con 0x55c1a6fcf760 2024-09-10T08:16:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.803+0000 7f96099f1740 10 client.27679 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.803+0000 7f96099f1740 10 client.27679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.803+0000 7f96099f1740 20 client.27679 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.803+0000 7f96099f1740 1 --2- 172.21.15.2:0/3757908589 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c1a705d630 0x55c1a705fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.803+0000 7f96099f1740 4 client.27679 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:16:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.803+0000 7f96099f1740 1 -- 172.21.15.2:0/3757908589 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1a6f3ce60 con 0x55c1a705d630 2024-09-10T08:16:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.804+0000 7f95ffe00640 1 --2- 172.21.15.2:0/3757908589 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c1a705d630 0x55c1a705fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.804+0000 7f95ffe00640 1 --2- 172.21.15.2:0/3757908589 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c1a705d630 0x55c1a705fa50 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da45ff7669f8ec23 server_cookie=3be28195b7c9f816 in_seq=0 out_seq=0 2024-09-10T08:16:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.804+0000 7f95ee000640 10 client.27679 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.805+0000 7f95ee000640 1 -- 172.21.15.2:0/3757908589 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1a6f3ce60 con 0x55c1a705d630 2024-09-10T08:16:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.805+0000 7f95ee000640 10 client.27679 handle_command_reply: tid=0 2024-09-10T08:16:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.805+0000 7f96099f1740 1 client.27679 shutdown 2024-09-10T08:16:38.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.805+0000 7f96099f1740 1 -- 172.21.15.2:0/3757908589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcf760 msgr2=0x55c1a6fe28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:38.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.805+0000 7f96099f1740 1 --2- 172.21.15.2:0/3757908589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcf760 0x55c1a6fe28c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f95e80037f0 tx=0x7f95e8003c90 comp rx=0 tx=0).stop 2024-09-10T08:16:38.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.805+0000 7f96099f1740 1 -- 172.21.15.2:0/3757908589 shutdown_connections 2024-09-10T08:16:38.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.805+0000 7f96099f1740 1 --2- 172.21.15.2:0/3757908589 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c1a705d630 0x55c1a705fa50 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.806+0000 7f96099f1740 1 --2- 172.21.15.2:0/3757908589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c1a6fda020 0x55c1a6fdd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.806+0000 7f96099f1740 1 --2- 172.21.15.2:0/3757908589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1a6fcf760 0x55c1a6fe28c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.806+0000 7f96099f1740 1 --2- 172.21.15.2:0/3757908589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c1a6fcee40 0x55c1a6fe2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.806+0000 7f96099f1740 1 -- 172.21.15.2:0/3757908589 >> 172.21.15.2:0/3757908589 conn(0x55c1a6fcb3d0 msgr2=0x55c1a6fd6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:38.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.806+0000 7f95ee000640 0 client.27679 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:38.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.806+0000 7f96099f1740 1 -- 172.21.15.2:0/3757908589 shutdown_connections 2024-09-10T08:16:38.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.806+0000 7f96099f1740 1 -- 172.21.15.2:0/3757908589 wait complete. 2024-09-10T08:16:38.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.806+0000 7f96099f1740 20 client.27679 trim_cache size 0 max 16384 2024-09-10T08:16:38.808 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:16:38.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.807+0000 7f9605800640 1 -- 172.21.15.2:0/241051882 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f95e40a2c00 msgr2=0x7f95e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:38.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.807+0000 7f9605800640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f95e40a2c00 0x7f95e40a50c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f95f0000c00 tx=0x7f95f0005b50 comp rx=0 tx=0).stop 2024-09-10T08:16:38.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.807+0000 7f9605800640 1 -- 172.21.15.2:0/241051882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f960010e260 msgr2=0x7f960019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:38.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.807+0000 7f9605800640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f960010e260 0x7f960019ed80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f95f8005fd0 tx=0x7f95f800ef10 comp rx=0 tx=0).stop 2024-09-10T08:16:38.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.807+0000 7f9605800640 1 -- 172.21.15.2:0/241051882 shutdown_connections 2024-09-10T08:16:38.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.807+0000 7f9605800640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f95e40a2c00 0x7f95e40a50c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.807+0000 7f9605800640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f960010e260 0x7f960019ed80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.807+0000 7f9605800640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9600074ee0 0x7f960019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.807+0000 7f9605800640 1 --2- 172.21.15.2:0/241051882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96000745c0 0x7f960019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:38.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.807+0000 7f9605800640 1 -- 172.21.15.2:0/241051882 >> 172.21.15.2:0/241051882 conn(0x7f96000fd740 msgr2=0x7f96000fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:38.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.807+0000 7f9605800640 1 -- 172.21.15.2:0/241051882 shutdown_connections 2024-09-10T08:16:38.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:38.807+0000 7f9605800640 1 -- 172.21.15.2:0/241051882 wait complete. 2024-09-10T08:16:38.818 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:39.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:39 smithi002 ceph-mon[30563]: pgmap v642: 129 pgs: 129 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:16:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:39 smithi155 ceph-mon[35405]: pgmap v642: 129 pgs: 129 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:16:39.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:39 smithi107 ceph-mon[35048]: pgmap v642: 129 pgs: 129 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:16:39.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.919+0000 7f5e8ec00640 1 -- 172.21.15.2:0/2921034927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e881049a0 msgr2=0x7f5e881089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:39.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.919+0000 7f5e8ec00640 1 --2- 172.21.15.2:0/2921034927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e881049a0 0x7f5e881089f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009900 tx=0x7f5e7002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.919+0000 7f5e8ec00640 1 -- 172.21.15.2:0/2921034927 shutdown_connections 2024-09-10T08:16:39.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.919+0000 7f5e8ec00640 1 --2- 172.21.15.2:0/2921034927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e88109120 0x7f5e88110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.919+0000 7f5e8ec00640 1 --2- 172.21.15.2:0/2921034927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e881049a0 0x7f5e881089f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.919+0000 7f5e8ec00640 1 --2- 172.21.15.2:0/2921034927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e88103ff0 0x7f5e881043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.919+0000 7f5e8ec00640 1 -- 172.21.15.2:0/2921034927 >> 172.21.15.2:0/2921034927 conn(0x7f5e880ffa90 msgr2=0x7f5e88101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:39.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.919+0000 7f5e8ec00640 1 -- 172.21.15.2:0/2921034927 shutdown_connections 2024-09-10T08:16:39.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.919+0000 7f5e8ec00640 1 -- 172.21.15.2:0/2921034927 wait complete. 2024-09-10T08:16:39.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.920+0000 7f5e8ec00640 1 Processor -- start 2024-09-10T08:16:39.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.920+0000 7f5e8ec00640 1 -- start start 2024-09-10T08:16:39.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.921+0000 7f5e8ec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e88103ff0 0x7f5e8819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.921+0000 7f5e8ec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e881049a0 0x7f5e8819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.921+0000 7f5e8ec00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e88109120 0x7f5e8819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.921+0000 7f5e8ec00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88114140 con 0x7f5e88109120 2024-09-10T08:16:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.921+0000 7f5e8ec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88113fc0 con 0x7f5e881049a0 2024-09-10T08:16:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.921+0000 7f5e8ec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881142c0 con 0x7f5e88103ff0 2024-09-10T08:16:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.921+0000 7f5e8cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e88109120 0x7f5e8819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.921+0000 7f5e8cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e88109120 0x7f5e8819eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37814/0 (socket says 172.21.15.2:37814) 2024-09-10T08:16:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.921+0000 7f5e8cc00640 1 -- 172.21.15.2:0/4014813257 learned_addr learned my addr 172.21.15.2:0/4014813257 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.921+0000 7f5e87e00640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e88103ff0 0x7f5e8819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.921+0000 7f5e7f400640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e881049a0 0x7f5e8819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.922+0000 7f5e8cc00640 1 -- 172.21.15.2:0/4014813257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e88103ff0 msgr2=0x7f5e8819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.922+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e88103ff0 0x7f5e8819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.922+0000 7f5e8cc00640 1 -- 172.21.15.2:0/4014813257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e881049a0 msgr2=0x7f5e8819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.922+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e881049a0 0x7f5e8819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.922+0000 7f5e8cc00640 1 -- 172.21.15.2:0/4014813257 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8819f5d0 con 0x7f5e88109120 2024-09-10T08:16:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.922+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e88109120 0x7f5e8819eef0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000eaa0 tx=0x7f5e8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.922+0000 7f5e85600640 1 -- 172.21.15.2:0/4014813257 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8000cdb0 con 0x7f5e88109120 2024-09-10T08:16:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.922+0000 7f5e85600640 1 -- 172.21.15.2:0/4014813257 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e80005020 con 0x7f5e88109120 2024-09-10T08:16:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.922+0000 7f5e85600640 1 -- 172.21.15.2:0/4014813257 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e800116b0 con 0x7f5e88109120 2024-09-10T08:16:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.922+0000 7f5e8ec00640 1 -- 172.21.15.2:0/4014813257 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8819f8c0 con 0x7f5e88109120 2024-09-10T08:16:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.922+0000 7f5e8ec00640 1 -- 172.21.15.2:0/4014813257 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e881a71a0 con 0x7f5e88109120 2024-09-10T08:16:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.926+0000 7f5e85600640 1 -- 172.21.15.2:0/4014813257 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5e80005560 con 0x7f5e88109120 2024-09-10T08:16:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.927+0000 7f5e91848740 1 Processor -- start 2024-09-10T08:16:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.927+0000 7f5e91848740 1 -- start start 2024-09-10T08:16:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.927+0000 7f5e85600640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5e6c0a2b80 0x7f5e6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.927+0000 7f5e85600640 1 -- 172.21.15.2:0/4014813257 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5e800cf5d0 con 0x7f5e88109120 2024-09-10T08:16:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.927+0000 7f5e91848740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c6c3b0 0x55db43c6c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.927+0000 7f5e7f400640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5e6c0a2b80 0x7f5e6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.927+0000 7f5e91848740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c77050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.927+0000 7f5e91848740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c77590 0x55db43c79980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.927+0000 7f5e91848740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55db43bd2530 con 0x55db43c6c3b0 2024-09-10T08:16:39.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.927+0000 7f5e91848740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55db43b79860 con 0x55db43c77590 2024-09-10T08:16:39.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.927+0000 7f5e91848740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55db43bce4f0 con 0x55db43c6ccd0 2024-09-10T08:16:39.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.928+0000 7f5e7f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c77050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.928+0000 7f5e7f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c77050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54370/0 (socket says 172.21.15.2:54370) 2024-09-10T08:16:39.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.928+0000 7f5e87e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c6c3b0 0x55db43c6c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.928+0000 7f5e7f400640 1 -- 172.21.15.2:0/3419227896 learned_addr learned my addr 172.21.15.2:0/3419227896 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:39.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.928+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/3419227896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c77590 0x55db43c79980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.928+0000 7f5e87e00640 1 -- 172.21.15.2:0/3419227896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 msgr2=0x55db43c77050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:39.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.928+0000 7f5e87e00640 1 --2- 172.21.15.2:0/3419227896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c77050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.928+0000 7f5e87e00640 1 -- 172.21.15.2:0/3419227896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c77590 msgr2=0x55db43c79980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:39.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.928+0000 7f5e87e00640 1 --2- 172.21.15.2:0/3419227896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c77590 0x55db43c79980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.928+0000 7f5e87e00640 1 -- 172.21.15.2:0/3419227896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db43943b90 con 0x55db43c6c3b0 2024-09-10T08:16:39.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.928+0000 7f5e7f400640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5e6c0a2b80 0x7f5e6c0a5040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5e700096c0 tx=0x7f5e70031800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:39.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.929+0000 7f5e87e00640 1 --2- 172.21.15.2:0/3419227896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c6c3b0 0x55db43c6c790 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009c00 tx=0x7f5e78011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398231443ed90997 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:39.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.929+0000 7f5e7d200640 1 -- 172.21.15.2:0/3419227896 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78004050 con 0x55db43c6c3b0 2024-09-10T08:16:39.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.929+0000 7f5e7d200640 1 -- 172.21.15.2:0/3419227896 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e78005500 con 0x55db43c6c3b0 2024-09-10T08:16:39.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.929+0000 7f5e7d200640 1 -- 172.21.15.2:0/3419227896 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78004d70 con 0x55db43c6c3b0 2024-09-10T08:16:39.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.930+0000 7f5e91848740 1 -- 172.21.15.2:0/3419227896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c6c3b0 msgr2=0x55db43c6c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:39.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.930+0000 7f5e91848740 1 --2- 172.21.15.2:0/3419227896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c6c3b0 0x55db43c6c790 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009c00 tx=0x7f5e78011810 comp rx=0 tx=0).stop 2024-09-10T08:16:39.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.930+0000 7f5e91848740 1 -- 172.21.15.2:0/3419227896 shutdown_connections 2024-09-10T08:16:39.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.930+0000 7f5e91848740 1 --2- 172.21.15.2:0/3419227896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c77590 0x55db43c79980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.930+0000 7f5e91848740 1 --2- 172.21.15.2:0/3419227896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c77050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.930+0000 7f5e91848740 1 --2- 172.21.15.2:0/3419227896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c6c3b0 0x55db43c6c790 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.930+0000 7f5e91848740 1 -- 172.21.15.2:0/3419227896 >> 172.21.15.2:0/3419227896 conn(0x55db43c68940 msgr2=0x55db43c68d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.930+0000 7f5e91848740 1 -- 172.21.15.2:0/3419227896 shutdown_connections 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.930+0000 7f5e91848740 1 -- 172.21.15.2:0/3419227896 wait complete. 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.931+0000 7f5e91848740 1 -- start start 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.931+0000 7f5e91848740 1 Processor -- start 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.931+0000 7f5e91848740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6c3b0 0x55db43c74680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.931+0000 7f5e91848740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c6ccd0 0x55db43c74bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.931+0000 7f5e91848740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c77590 0x55db43c75100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.931+0000 7f5e91848740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55db43c79fb0 con 0x55db43c6ccd0 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.931+0000 7f5e91848740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55db43b8ce60 con 0x55db43c77590 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.931+0000 7f5e91848740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55db43c7a130 con 0x55db43c6c3b0 2024-09-10T08:16:39.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.931+0000 7f5e91848740 10 client.0 _ll_register_callbacks cb 0x55db43b91cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:39.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e87e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6c3b0 0x55db43c74680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e7f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c6ccd0 0x55db43c74bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e87e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6c3b0 0x55db43c74680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54382/0 (socket says 172.21.15.2:54382) 2024-09-10T08:16:39.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e87e00640 1 -- 172.21.15.2:0/3706746418 learned_addr learned my addr 172.21.15.2:0/3706746418 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:39.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c77590 0x55db43c75100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e8cc00640 1 -- 172.21.15.2:0/3706746418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6c3b0 msgr2=0x55db43c74680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:39.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6c3b0 0x55db43c74680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e8cc00640 1 -- 172.21.15.2:0/3706746418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c6ccd0 msgr2=0x55db43c74bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:39.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c6ccd0 0x55db43c74bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e8cc00640 1 -- 172.21.15.2:0/3706746418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db43b98990 con 0x55db43c77590 2024-09-10T08:16:39.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e87e00640 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6c3b0 0x55db43c74680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:39.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e7f400640 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c6ccd0 0x55db43c74bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:39.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c77590 0x55db43c75100 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5e800ccc80 tx=0x7f5e8009fe50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b85003be3932000 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.932+0000 7f5e75600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.933+0000 7f5e75600640 1 -- 172.21.15.2:0/3706746418 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8001f030 con 0x55db43c77590 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.933+0000 7f5e91848740 1 -- 172.21.15.2:0/3706746418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e78005500 con 0x55db43c77590 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.933+0000 7f5e75600640 1 -- 172.21.15.2:0/3706746418 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5e800b4be0 con 0x55db43c77590 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.933+0000 7f5e75600640 1 -- 172.21.15.2:0/3706746418 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e800b4de0 con 0x55db43c77590 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.933+0000 7f5e91848740 10 client.27752 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.933+0000 7f5e91848740 1 -- 172.21.15.2:0/3706746418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55db43c85d90 con 0x55db43c77590 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.933+0000 7f5e75600640 1 -- 172.21.15.2:0/3706746418 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e78005500 con 0x55db43c77590 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.933+0000 7f5e75600640 1 -- 172.21.15.2:0/3706746418 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5e8000f040 con 0x55db43c77590 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.933+0000 7f5e91848740 10 client.27752 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.933+0000 7f5e91848740 10 client.27752 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.934+0000 7f5e91848740 20 client.27752 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.934+0000 7f5e91848740 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55db43c87250 0x55db43c89660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.934+0000 7f5e91848740 4 client.27752 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:39.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.934+0000 7f5e91848740 1 -- 172.21.15.2:0/3706746418 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db43b75310 con 0x55db43c87250 2024-09-10T08:16:39.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.934+0000 7f5e87e00640 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55db43c87250 0x55db43c89660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.934+0000 7f5e87e00640 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55db43c87250 0x55db43c89660 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19e43c7846eb808b server_cookie=9d775711960fce45 in_seq=0 out_seq=0 2024-09-10T08:16:39.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.934+0000 7f5e75600640 10 client.27752 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:39.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.935+0000 7f5e75600640 1 -- 172.21.15.2:0/3706746418 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db43b75310 con 0x55db43c87250 2024-09-10T08:16:39.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.935+0000 7f5e75600640 10 client.27752 handle_command_reply: tid=0 2024-09-10T08:16:39.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.935+0000 7f5e91848740 1 client.27752 shutdown 2024-09-10T08:16:39.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.935+0000 7f5e91848740 1 -- 172.21.15.2:0/3706746418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c77590 msgr2=0x55db43c75100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:39.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.935+0000 7f5e91848740 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c77590 0x55db43c75100 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5e800ccc80 tx=0x7f5e8009fe50 comp rx=0 tx=0).stop 2024-09-10T08:16:39.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.936+0000 7f5e91848740 1 -- 172.21.15.2:0/3706746418 shutdown_connections 2024-09-10T08:16:39.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.936+0000 7f5e91848740 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55db43c87250 0x55db43c89660 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.936+0000 7f5e91848740 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c77590 0x55db43c75100 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.936+0000 7f5e91848740 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c6ccd0 0x55db43c74bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.936+0000 7f5e91848740 1 --2- 172.21.15.2:0/3706746418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6c3b0 0x55db43c74680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.936+0000 7f5e75600640 0 client.27752 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:39.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.936+0000 7f5e91848740 1 -- 172.21.15.2:0/3706746418 >> 172.21.15.2:0/3706746418 conn(0x55db43c68940 msgr2=0x55db43c67c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:39.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.936+0000 7f5e91848740 1 -- 172.21.15.2:0/3706746418 shutdown_connections 2024-09-10T08:16:39.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.936+0000 7f5e91848740 1 -- 172.21.15.2:0/3706746418 wait complete. 2024-09-10T08:16:39.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.936+0000 7f5e91848740 20 client.27752 trim_cache size 0 max 16384 2024-09-10T08:16:39.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.954+0000 7f5e91848740 1 Processor -- start 2024-09-10T08:16:39.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.954+0000 7f5e91848740 1 -- start start 2024-09-10T08:16:39.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.955+0000 7f5e91848740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 0x55db43c8ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.955+0000 7f5e91848740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c8f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.955+0000 7f5e91848740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c77590 0x55db43c7f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.955+0000 7f5e91848740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881b3920 con 0x55db43c77590 2024-09-10T08:16:39.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.955+0000 7f5e91848740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881b11f0 con 0x55db43c6c3b0 2024-09-10T08:16:39.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.955+0000 7f5e91848740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881af040 con 0x55db43c6ccd0 2024-09-10T08:16:39.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.955+0000 7f5e8cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c77590 0x55db43c7f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.955+0000 7f5e87e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 0x55db43c8ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.955+0000 7f5e87e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 0x55db43c8ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35120/0 (socket says 172.21.15.2:35120) 2024-09-10T08:16:39.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.955+0000 7f5e8cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c77590 0x55db43c7f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37852/0 (socket says 172.21.15.2:37852) 2024-09-10T08:16:39.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.955+0000 7f5e87e00640 1 -- 172.21.15.2:0/957249491 learned_addr learned my addr 172.21.15.2:0/957249491 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:39.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.955+0000 7f5e7f400640 1 --2- 172.21.15.2:0/957249491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c8f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.956+0000 7f5e8cc00640 1 -- 172.21.15.2:0/957249491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 msgr2=0x55db43c8f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:39.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.956+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/957249491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c8f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.956+0000 7f5e8cc00640 1 -- 172.21.15.2:0/957249491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 msgr2=0x55db43c8ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:39.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.956+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/957249491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 0x55db43c8ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.956+0000 7f5e8cc00640 1 -- 172.21.15.2:0/957249491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db43c79cd0 con 0x55db43c77590 2024-09-10T08:16:39.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.956+0000 7f5e87e00640 1 --2- 172.21.15.2:0/957249491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 0x55db43c8ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:39.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.956+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/957249491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c77590 0x55db43c7f280 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5e80097ef0 tx=0x7f5e800ccd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77cfd829fdb588e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:39.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.956+0000 7f5e77e00640 1 -- 172.21.15.2:0/957249491 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8001f070 con 0x55db43c77590 2024-09-10T08:16:39.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.956+0000 7f5e77e00640 1 -- 172.21.15.2:0/957249491 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e800b4070 con 0x55db43c77590 2024-09-10T08:16:39.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.956+0000 7f5e77e00640 1 -- 172.21.15.2:0/957249491 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e800ba0d0 con 0x55db43c77590 2024-09-10T08:16:39.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.957+0000 7f5e91848740 1 -- 172.21.15.2:0/957249491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c77590 msgr2=0x55db43c7f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:39.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.957+0000 7f5e91848740 1 --2- 172.21.15.2:0/957249491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c77590 0x55db43c7f280 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5e80097ef0 tx=0x7f5e800ccd00 comp rx=0 tx=0).stop 2024-09-10T08:16:39.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.957+0000 7f5e91848740 1 -- 172.21.15.2:0/957249491 shutdown_connections 2024-09-10T08:16:39.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.957+0000 7f5e91848740 1 --2- 172.21.15.2:0/957249491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c77590 0x55db43c7f280 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.957+0000 7f5e91848740 1 --2- 172.21.15.2:0/957249491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c8f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.957+0000 7f5e91848740 1 --2- 172.21.15.2:0/957249491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 0x55db43c8ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.957+0000 7f5e91848740 1 -- 172.21.15.2:0/957249491 >> 172.21.15.2:0/957249491 conn(0x55db43c68940 msgr2=0x55db43c67420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:39.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.957+0000 7f5e91848740 1 -- 172.21.15.2:0/957249491 shutdown_connections 2024-09-10T08:16:39.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.957+0000 7f5e91848740 1 -- 172.21.15.2:0/957249491 wait complete. 2024-09-10T08:16:39.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.957+0000 7f5e91848740 1 -- start start 2024-09-10T08:16:39.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.958+0000 7f5e91848740 1 Processor -- start 2024-09-10T08:16:39.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.958+0000 7f5e91848740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 0x55db43c7bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.958+0000 7f5e91848740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c7c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.958+0000 7f5e91848740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c77590 0x55db43ce9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:39.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.958+0000 7f5e91848740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881af6b0 con 0x55db43c77590 2024-09-10T08:16:39.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.958+0000 7f5e91848740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881b0b20 con 0x55db43c6c3b0 2024-09-10T08:16:39.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.958+0000 7f5e91848740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881afd80 con 0x55db43c6ccd0 2024-09-10T08:16:39.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.958+0000 7f5e91848740 10 client.0 _ll_register_callbacks cb 0x7f5e8009b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:39.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e8cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c77590 0x55db43ce9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e87e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 0x55db43c7bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e7f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c7c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:39.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e7f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c7c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54406/0 (socket says 172.21.15.2:54406) 2024-09-10T08:16:39.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e7f400640 1 -- 172.21.15.2:0/3723751096 learned_addr learned my addr 172.21.15.2:0/3723751096 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:39.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e87e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 0x55db43c7bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35130/0 (socket says 172.21.15.2:35130) 2024-09-10T08:16:39.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e8cc00640 1 -- 172.21.15.2:0/3723751096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 msgr2=0x55db43c7c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:39.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/3723751096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c7c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e8cc00640 1 -- 172.21.15.2:0/3723751096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 msgr2=0x55db43c7bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:39.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/3723751096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 0x55db43c7bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:39.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e8cc00640 1 -- 172.21.15.2:0/3723751096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db43cea6c0 con 0x55db43c77590 2024-09-10T08:16:39.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e87e00640 1 --2- 172.21.15.2:0/3723751096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 0x55db43c7bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:39.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e8cc00640 1 --2- 172.21.15.2:0/3723751096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c77590 0x55db43ce9e90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5e80097ef0 tx=0x7f5e8009b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e031837a249b275e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:39.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e75600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:16:39.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.959+0000 7f5e75600640 1 -- 172.21.15.2:0/3723751096 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8001f070 con 0x55db43c77590 2024-09-10T08:16:40.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.960+0000 7f5e75600640 1 -- 172.21.15.2:0/3723751096 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5e800979f0 con 0x55db43c77590 2024-09-10T08:16:40.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.960+0000 7f5e91848740 1 -- 172.21.15.2:0/3723751096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e800b4070 con 0x55db43c77590 2024-09-10T08:16:40.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.960+0000 7f5e75600640 1 -- 172.21.15.2:0/3723751096 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e8009aee0 con 0x55db43c77590 2024-09-10T08:16:40.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.960+0000 7f5e91848740 10 client.17730 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:40.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.960+0000 7f5e91848740 1 -- 172.21.15.2:0/3723751096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55db43cea6c0 con 0x55db43c77590 2024-09-10T08:16:40.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.960+0000 7f5e75600640 1 -- 172.21.15.2:0/3723751096 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e800b4070 con 0x55db43c77590 2024-09-10T08:16:40.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.960+0000 7f5e75600640 1 -- 172.21.15.2:0/3723751096 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5e80004a40 con 0x55db43c77590 2024-09-10T08:16:40.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.960+0000 7f5e91848740 10 client.17730 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:40.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.960+0000 7f5e91848740 10 client.17730 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:16:40.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.960+0000 7f5e91848740 20 client.17730 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:40.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.960+0000 7f5e91848740 1 --2- 172.21.15.2:0/3723751096 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55db43cf2cd0 0x55db43cf50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:40.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.960+0000 7f5e91848740 4 client.17730 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-10T08:16:40.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.960+0000 7f5e91848740 1 -- 172.21.15.2:0/3723751096 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x55db43bdfc60 con 0x55db43cf2cd0 2024-09-10T08:16:40.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.961+0000 7f5e87e00640 1 --2- 172.21.15.2:0/3723751096 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55db43cf2cd0 0x55db43cf50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:40.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.961+0000 7f5e87e00640 1 --2- 172.21.15.2:0/3723751096 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55db43cf2cd0 0x55db43cf50c0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c8a69cf942a723e server_cookie=fadca7176d4d26c0 in_seq=0 out_seq=0 2024-09-10T08:16:40.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.961+0000 7f5e75600640 10 client.17730 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:40.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e75600640 1 -- 172.21.15.2:0/3723751096 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55db43bdfc60 con 0x55db43cf2cd0 2024-09-10T08:16:40.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e75600640 10 client.17730 handle_command_reply: tid=0 2024-09-10T08:16:40.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e91848740 1 client.17730 shutdown 2024-09-10T08:16:40.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e91848740 1 -- 172.21.15.2:0/3723751096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c77590 msgr2=0x55db43ce9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:40.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e91848740 1 --2- 172.21.15.2:0/3723751096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c77590 0x55db43ce9e90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5e80097ef0 tx=0x7f5e8009b860 comp rx=0 tx=0).stop 2024-09-10T08:16:40.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e91848740 1 -- 172.21.15.2:0/3723751096 shutdown_connections 2024-09-10T08:16:40.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e91848740 1 --2- 172.21.15.2:0/3723751096 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55db43cf2cd0 0x55db43cf50c0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:40.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e91848740 1 --2- 172.21.15.2:0/3723751096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55db43c77590 0x55db43ce9e90 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:40.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e91848740 1 --2- 172.21.15.2:0/3723751096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db43c6ccd0 0x55db43c7c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:40.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e91848740 1 --2- 172.21.15.2:0/3723751096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db43c6c3b0 0x55db43c7bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:40.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e75600640 0 client.17730 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:40.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e91848740 1 -- 172.21.15.2:0/3723751096 >> 172.21.15.2:0/3723751096 conn(0x55db43c68940 msgr2=0x55db43c6de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:40.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e91848740 1 -- 172.21.15.2:0/3723751096 shutdown_connections 2024-09-10T08:16:40.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e91848740 1 -- 172.21.15.2:0/3723751096 wait complete. 2024-09-10T08:16:40.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.976+0000 7f5e91848740 20 client.17730 trim_cache size 0 max 16384 2024-09-10T08:16:40.003 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:16:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:16:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:16:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 26, 2024-09-10T08:16:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:16:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "32b5535c": { 2024-09-10T08:16:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 26, 2024-09-10T08:16:40.003 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:16:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:16:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:16:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:16:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:16:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:16:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 133.4, 2024-09-10T08:16:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:16:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:16:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:16:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:16:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:16:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:16:40.004 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:16:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:16:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:16:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:16:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:16:40.005 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:16:40.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.977+0000 7f5e8ec00640 1 -- 172.21.15.2:0/4014813257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5e6c0a2b80 msgr2=0x7f5e6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:40.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.977+0000 7f5e8ec00640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5e6c0a2b80 0x7f5e6c0a5040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5e700096c0 tx=0x7f5e70031800 comp rx=0 tx=0).stop 2024-09-10T08:16:40.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.977+0000 7f5e8ec00640 1 -- 172.21.15.2:0/4014813257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e88109120 msgr2=0x7f5e8819eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:40.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.977+0000 7f5e8ec00640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e88109120 0x7f5e8819eef0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000eaa0 tx=0x7f5e8000ef60 comp rx=0 tx=0).stop 2024-09-10T08:16:40.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.977+0000 7f5e8ec00640 1 -- 172.21.15.2:0/4014813257 shutdown_connections 2024-09-10T08:16:40.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.977+0000 7f5e8ec00640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e88109120 0x7f5e8819eef0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:40.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.977+0000 7f5e8ec00640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e881049a0 0x7f5e8819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:40.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.977+0000 7f5e8ec00640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5e6c0a2b80 0x7f5e6c0a5040 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:40.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.977+0000 7f5e8ec00640 1 --2- 172.21.15.2:0/4014813257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e88103ff0 0x7f5e8819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:40.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.977+0000 7f5e8ec00640 1 -- 172.21.15.2:0/4014813257 >> 172.21.15.2:0/4014813257 conn(0x7f5e880ffa90 msgr2=0x7f5e88100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:40.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.977+0000 7f5e8ec00640 1 -- 172.21.15.2:0/4014813257 shutdown_connections 2024-09-10T08:16:40.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:39.977+0000 7f5e8ec00640 1 -- 172.21.15.2:0/4014813257 wait complete. 2024-09-10T08:16:40.846 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '32b5535c', quiesce took 6.6 seconds. Will release after: 6.800000000000001 2024-09-10T08:16:40.934 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:16:40.934 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:16:41.313 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:41 smithi002 ceph-mon[30563]: pgmap v643: 129 pgs: 129 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-10T08:16:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:41 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:16:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:41 smithi155 ceph-mon[35405]: pgmap v643: 129 pgs: 129 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-10T08:16:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:41 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:16:41.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:41 smithi107 ceph-mon[35048]: pgmap v643: 129 pgs: 129 active+clean; 4.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-10T08:16:41.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:41 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:16:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.928+0000 7f13c3a00640 1 -- 172.21.15.2:0/562536667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13bc103ff0 msgr2=0x7f13bc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.928+0000 7f13c3a00640 1 --2- 172.21.15.2:0/562536667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13bc103ff0 0x7f13bc1043d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009900 tx=0x7f13b002a410 comp rx=0 tx=0).stop 2024-09-10T08:16:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.928+0000 7f13c3a00640 1 -- 172.21.15.2:0/562536667 shutdown_connections 2024-09-10T08:16:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.928+0000 7f13c3a00640 1 --2- 172.21.15.2:0/562536667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13bc109120 0x7f13bc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.928+0000 7f13c3a00640 1 --2- 172.21.15.2:0/562536667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13bc1049a0 0x7f13bc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.928+0000 7f13c3a00640 1 --2- 172.21.15.2:0/562536667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13bc103ff0 0x7f13bc1043d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.928+0000 7f13c3a00640 1 -- 172.21.15.2:0/562536667 >> 172.21.15.2:0/562536667 conn(0x7f13bc0ffa90 msgr2=0x7f13bc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:41.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.928+0000 7f13c3a00640 1 -- 172.21.15.2:0/562536667 shutdown_connections 2024-09-10T08:16:41.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.928+0000 7f13c3a00640 1 -- 172.21.15.2:0/562536667 wait complete. 2024-09-10T08:16:41.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.929+0000 7f13c3a00640 1 Processor -- start 2024-09-10T08:16:41.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.930+0000 7f13c3a00640 1 -- start start 2024-09-10T08:16:41.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.930+0000 7f13c3a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13bc103ff0 0x7f13bc076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.930+0000 7f13c3a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13bc1049a0 0x7f13bc076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.930+0000 7f13c3a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13bc109120 0x7f13bc077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.930+0000 7f13c3a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc114150 con 0x7f13bc109120 2024-09-10T08:16:41.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.930+0000 7f13c3a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc113fd0 con 0x7f13bc103ff0 2024-09-10T08:16:41.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.930+0000 7f13c3a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1142d0 con 0x7f13bc1049a0 2024-09-10T08:16:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.930+0000 7f13bbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13bc1049a0 0x7f13bc076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.930+0000 7f13c1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13bc103ff0 0x7f13bc076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.930+0000 7f13bbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13bc1049a0 0x7f13bc076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54416/0 (socket says 172.21.15.2:54416) 2024-09-10T08:16:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.930+0000 7f13c1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13bc103ff0 0x7f13bc076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35140/0 (socket says 172.21.15.2:35140) 2024-09-10T08:16:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.930+0000 7f13bbe00640 1 -- 172.21.15.2:0/2059440895 learned_addr learned my addr 172.21.15.2:0/2059440895 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:41.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.930+0000 7f13c1a00640 1 --2- 172.21.15.2:0/2059440895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13bc109120 0x7f13bc077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.931+0000 7f13bbe00640 1 -- 172.21.15.2:0/2059440895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13bc103ff0 msgr2=0x7f13bc076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:41.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.931+0000 7f13bbe00640 1 --2- 172.21.15.2:0/2059440895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13bc103ff0 0x7f13bc076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.931+0000 7f13bbe00640 1 -- 172.21.15.2:0/2059440895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13bc109120 msgr2=0x7f13bc077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:41.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.931+0000 7f13bbe00640 1 --2- 172.21.15.2:0/2059440895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13bc109120 0x7f13bc077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.931+0000 7f13bbe00640 1 -- 172.21.15.2:0/2059440895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc1aad10 con 0x7f13bc1049a0 2024-09-10T08:16:41.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.931+0000 7f13bbe00640 1 --2- 172.21.15.2:0/2059440895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13bc1049a0 0x7f13bc076c40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f13a800ea20 tx=0x7f13a800eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:41.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.931+0000 7f13b9600640 1 -- 172.21.15.2:0/2059440895 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a800cd80 con 0x7f13bc1049a0 2024-09-10T08:16:41.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.931+0000 7f13b9600640 1 -- 172.21.15.2:0/2059440895 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13a8010910 con 0x7f13bc1049a0 2024-09-10T08:16:41.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.931+0000 7f13c3a00640 1 -- 172.21.15.2:0/2059440895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13bc1aafa0 con 0x7f13bc1049a0 2024-09-10T08:16:41.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.931+0000 7f13b9600640 1 -- 172.21.15.2:0/2059440895 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8018920 con 0x7f13bc1049a0 2024-09-10T08:16:41.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.932+0000 7f13c3a00640 1 -- 172.21.15.2:0/2059440895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13bc1ab4b0 con 0x7f13bc1049a0 2024-09-10T08:16:41.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.935+0000 7f13c65d5740 1 Processor -- start 2024-09-10T08:16:41.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.935+0000 7f13c65d5740 1 -- start start 2024-09-10T08:16:41.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.935+0000 7f13c65d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5edac0 0x55cbad5edea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.935+0000 7f13b9600640 1 -- 172.21.15.2:0/2059440895 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f13a8010ac0 con 0x7f13bc1049a0 2024-09-10T08:16:41.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.935+0000 7f13c65d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad5f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.935+0000 7f13c65d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5f8ca0 0x55cbad5fb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.935+0000 7f13c65d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cbad554290 con 0x55cbad5ee3e0 2024-09-10T08:16:41.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.935+0000 7f13c65d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cbad4fbb40 con 0x55cbad5f8ca0 2024-09-10T08:16:41.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.935+0000 7f13c65d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cbad550250 con 0x55cbad5edac0 2024-09-10T08:16:41.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.936+0000 7f13c1000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad5f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.936+0000 7f13c1a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5edac0 0x55cbad5edea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.936+0000 7f13c1a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5edac0 0x55cbad5edea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54420/0 (socket says 172.21.15.2:54420) 2024-09-10T08:16:41.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.936+0000 7f13c1a00640 1 -- 172.21.15.2:0/2345804373 learned_addr learned my addr 172.21.15.2:0/2345804373 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:41.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.936+0000 7f13c1000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad5f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37898/0 (socket says 172.21.15.2:37898) 2024-09-10T08:16:41.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.936+0000 7f13bbe00640 1 --2- 172.21.15.2:0/2345804373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5f8ca0 0x55cbad5fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.936+0000 7f13c1000640 1 -- 172.21.15.2:0/2345804373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5edac0 msgr2=0x55cbad5edea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.936+0000 7f13c1000640 1 --2- 172.21.15.2:0/2345804373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5edac0 0x55cbad5edea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.936+0000 7f13c1000640 1 -- 172.21.15.2:0/2345804373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5f8ca0 msgr2=0x55cbad5fb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.936+0000 7f13c1000640 1 --2- 172.21.15.2:0/2345804373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5f8ca0 0x55cbad5fb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.936+0000 7f13c1000640 1 -- 172.21.15.2:0/2345804373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbad2c56a0 con 0x55cbad5ee3e0 2024-09-10T08:16:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.937+0000 7f13c1000640 1 --2- 172.21.15.2:0/2345804373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad5f8760 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f13b0005940 tx=0x7f13b0005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8256704968742156 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.937+0000 7f13ae000640 1 -- 172.21.15.2:0/2345804373 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0040070 con 0x55cbad5ee3e0 2024-09-10T08:16:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.937+0000 7f13ae000640 1 -- 172.21.15.2:0/2345804373 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13b0031870 con 0x55cbad5ee3e0 2024-09-10T08:16:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.937+0000 7f13ae000640 1 -- 172.21.15.2:0/2345804373 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0035040 con 0x55cbad5ee3e0 2024-09-10T08:16:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.937+0000 7f13b9600640 1 --2- 172.21.15.2:0/2059440895 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13a40a2b80 0x7f13a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.937+0000 7f13b9600640 1 -- 172.21.15.2:0/2059440895 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f13a80cf890 con 0x7f13bc1049a0 2024-09-10T08:16:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.938+0000 7f13c1a00640 1 --2- 172.21.15.2:0/2059440895 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13a40a2b80 0x7f13a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.938+0000 7f13c65d5740 1 -- 172.21.15.2:0/2345804373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 msgr2=0x55cbad5f8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:41.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.938+0000 7f13c65d5740 1 --2- 172.21.15.2:0/2345804373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad5f8760 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f13b0005940 tx=0x7f13b0005be0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.938+0000 7f13c65d5740 1 -- 172.21.15.2:0/2345804373 shutdown_connections 2024-09-10T08:16:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.938+0000 7f13c65d5740 1 --2- 172.21.15.2:0/2345804373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5f8ca0 0x55cbad5fb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.938+0000 7f13c65d5740 1 --2- 172.21.15.2:0/2345804373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad5f8760 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.938+0000 7f13c65d5740 1 --2- 172.21.15.2:0/2345804373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5edac0 0x55cbad5edea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.938+0000 7f13c65d5740 1 -- 172.21.15.2:0/2345804373 >> 172.21.15.2:0/2345804373 conn(0x55cbad5ea050 msgr2=0x55cbad5ea430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.938+0000 7f13c65d5740 1 -- 172.21.15.2:0/2345804373 shutdown_connections 2024-09-10T08:16:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.939+0000 7f13c1a00640 1 --2- 172.21.15.2:0/2059440895 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13a40a2b80 0x7f13a40a5040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009fd0 tx=0x7f13b4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.939+0000 7f13c65d5740 1 -- 172.21.15.2:0/2345804373 wait complete. 2024-09-10T08:16:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.939+0000 7f13c65d5740 1 -- start start 2024-09-10T08:16:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.939+0000 7f13c65d5740 1 Processor -- start 2024-09-10T08:16:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c65d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad5f8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c65d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5ee3e0 0x55cbad5f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c65d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5f8ca0 0x55cbad5f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c65d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cbad5fb910 con 0x55cbad5f8ca0 2024-09-10T08:16:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c65d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cbad50efa0 con 0x55cbad5edac0 2024-09-10T08:16:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c65d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cbad5fba90 con 0x55cbad5ee3e0 2024-09-10T08:16:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c65d5740 10 client.0 _ll_register_callbacks cb 0x55cbad262ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c1a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5f8ca0 0x55cbad5f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad5f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c1a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5f8ca0 0x55cbad5f5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37914/0 (socket says 172.21.15.2:37914) 2024-09-10T08:16:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad5f8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35148/0 (socket says 172.21.15.2:35148) 2024-09-10T08:16:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c1a00640 1 -- 172.21.15.2:0/3818109220 learned_addr learned my addr 172.21.15.2:0/3818109220 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13bbe00640 1 --2- 172.21.15.2:0/3818109220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5ee3e0 0x55cbad5f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c1000640 1 -- 172.21.15.2:0/3818109220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5ee3e0 msgr2=0x55cbad5f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.940+0000 7f13c1000640 1 --2- 172.21.15.2:0/3818109220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5ee3e0 0x55cbad5f57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.941+0000 7f13c1000640 1 -- 172.21.15.2:0/3818109220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5f8ca0 msgr2=0x55cbad5f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.941+0000 7f13c1000640 1 --2- 172.21.15.2:0/3818109220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5f8ca0 0x55cbad5f5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.941+0000 7f13c1000640 1 -- 172.21.15.2:0/3818109220 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbad5f64f0 con 0x55cbad5edac0 2024-09-10T08:16:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.941+0000 7f13c1000640 1 --2- 172.21.15.2:0/3818109220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad5f8520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f13b0000c00 tx=0x7f13b0005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b7dc426adbbbaa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.941+0000 7f13a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.941+0000 7f13a2a00640 1 -- 172.21.15.2:0/3818109220 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0040070 con 0x55cbad5edac0 2024-09-10T08:16:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.941+0000 7f13a2a00640 1 -- 172.21.15.2:0/3818109220 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f13b0031670 con 0x55cbad5edac0 2024-09-10T08:16:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.941+0000 7f13c65d5740 1 -- 172.21.15.2:0/3818109220 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13b0031870 con 0x55cbad5edac0 2024-09-10T08:16:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.942+0000 7f13a2a00640 1 -- 172.21.15.2:0/3818109220 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f13b0035050 con 0x55cbad5edac0 2024-09-10T08:16:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.942+0000 7f13c65d5740 10 client.27770 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.942+0000 7f13c65d5740 1 -- 172.21.15.2:0/3818109220 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cbad6077a0 con 0x55cbad5edac0 2024-09-10T08:16:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.942+0000 7f13a2a00640 1 -- 172.21.15.2:0/3818109220 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13b0031870 con 0x55cbad5edac0 2024-09-10T08:16:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.942+0000 7f13a2a00640 1 -- 172.21.15.2:0/3818109220 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f13b0004240 con 0x55cbad5edac0 2024-09-10T08:16:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.942+0000 7f13c65d5740 10 client.27770 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.942+0000 7f13c65d5740 10 client.27770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.942+0000 7f13c65d5740 20 client.27770 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:41.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.942+0000 7f13c65d5740 1 --2- 172.21.15.2:0/3818109220 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cbad608af0 0x55cbad60aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.942+0000 7f13c65d5740 4 client.27770 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:41.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.942+0000 7f13c65d5740 1 -- 172.21.15.2:0/3818109220 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbad561800 con 0x55cbad608af0 2024-09-10T08:16:41.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.943+0000 7f13bbe00640 1 --2- 172.21.15.2:0/3818109220 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cbad608af0 0x55cbad60aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.943+0000 7f13bbe00640 1 --2- 172.21.15.2:0/3818109220 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cbad608af0 0x55cbad60aee0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cd91a828f55b31 server_cookie=38999b2376cac01e in_seq=0 out_seq=0 2024-09-10T08:16:41.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.943+0000 7f13a2a00640 10 client.27770 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:41.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.944+0000 7f13a2a00640 1 -- 172.21.15.2:0/3818109220 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbad561800 con 0x55cbad608af0 2024-09-10T08:16:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.944+0000 7f13a2a00640 10 client.27770 handle_command_reply: tid=0 2024-09-10T08:16:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.944+0000 7f13c65d5740 1 client.27770 shutdown 2024-09-10T08:16:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.944+0000 7f13c65d5740 1 -- 172.21.15.2:0/3818109220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 msgr2=0x55cbad5f8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.944+0000 7f13c65d5740 1 --2- 172.21.15.2:0/3818109220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad5f8520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f13b0000c00 tx=0x7f13b0005250 comp rx=0 tx=0).stop 2024-09-10T08:16:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.944+0000 7f13c65d5740 1 -- 172.21.15.2:0/3818109220 shutdown_connections 2024-09-10T08:16:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.944+0000 7f13c65d5740 1 --2- 172.21.15.2:0/3818109220 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cbad608af0 0x55cbad60aee0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.944+0000 7f13c65d5740 1 --2- 172.21.15.2:0/3818109220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5f8ca0 0x55cbad5f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.944+0000 7f13c65d5740 1 --2- 172.21.15.2:0/3818109220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5ee3e0 0x55cbad5f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.944+0000 7f13c65d5740 1 --2- 172.21.15.2:0/3818109220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad5f8520 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.944+0000 7f13a2a00640 0 client.27770 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:41.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.944+0000 7f13c65d5740 1 -- 172.21.15.2:0/3818109220 >> 172.21.15.2:0/3818109220 conn(0x55cbad5ea050 msgr2=0x55cbad5efd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:41.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.944+0000 7f13c65d5740 1 -- 172.21.15.2:0/3818109220 shutdown_connections 2024-09-10T08:16:41.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.945+0000 7f13c65d5740 1 -- 172.21.15.2:0/3818109220 wait complete. 2024-09-10T08:16:41.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.945+0000 7f13c65d5740 20 client.27770 trim_cache size 0 max 16384 2024-09-10T08:16:41.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.981+0000 7f13c65d5740 1 Processor -- start 2024-09-10T08:16:41.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.981+0000 7f13c65d5740 1 -- start start 2024-09-10T08:16:41.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.981+0000 7f13c65d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad5f8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.981+0000 7f13c65d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad60df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.981+0000 7f13c65d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5f8ca0 0x55cbad60e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.981+0000 7f13c65d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b7d60 con 0x55cbad5ee3e0 2024-09-10T08:16:41.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.981+0000 7f13c65d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b5630 con 0x55cbad5edac0 2024-09-10T08:16:41.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.981+0000 7f13c65d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b3480 con 0x55cbad5f8ca0 2024-09-10T08:16:41.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.981+0000 7f13bbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad60df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.982+0000 7f13bbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad60df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37928/0 (socket says 172.21.15.2:37928) 2024-09-10T08:16:41.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.982+0000 7f13bbe00640 1 -- 172.21.15.2:0/2714938700 learned_addr learned my addr 172.21.15.2:0/2714938700 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:41.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.982+0000 7f13c1000640 1 --2- 172.21.15.2:0/2714938700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad5f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.982+0000 7f13c1a00640 1 --2- 172.21.15.2:0/2714938700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5f8ca0 0x55cbad60e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.982+0000 7f13c1000640 1 -- 172.21.15.2:0/2714938700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5f8ca0 msgr2=0x55cbad60e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:41.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.982+0000 7f13c1000640 1 --2- 172.21.15.2:0/2714938700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5f8ca0 0x55cbad60e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.982+0000 7f13c1000640 1 -- 172.21.15.2:0/2714938700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 msgr2=0x55cbad60df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:41.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.982+0000 7f13c1000640 1 --2- 172.21.15.2:0/2714938700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad60df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.982+0000 7f13c1000640 1 -- 172.21.15.2:0/2714938700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbad5f6760 con 0x55cbad5edac0 2024-09-10T08:16:41.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.982+0000 7f13bbe00640 1 --2- 172.21.15.2:0/2714938700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad60df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:41.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.982+0000 7f13c1000640 1 --2- 172.21.15.2:0/2714938700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad5f8040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f13b0005500 tx=0x7f13b0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac64a72d56e2133 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:41.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.983+0000 7f13ad600640 1 -- 172.21.15.2:0/2714938700 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0040040 con 0x55cbad5edac0 2024-09-10T08:16:41.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.983+0000 7f13ad600640 1 -- 172.21.15.2:0/2714938700 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13b0003890 con 0x55cbad5edac0 2024-09-10T08:16:41.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.983+0000 7f13ad600640 1 -- 172.21.15.2:0/2714938700 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0003e10 con 0x55cbad5edac0 2024-09-10T08:16:41.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.983+0000 7f13c65d5740 1 -- 172.21.15.2:0/2714938700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 msgr2=0x55cbad5f8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:41.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.983+0000 7f13c65d5740 1 --2- 172.21.15.2:0/2714938700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad5f8040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f13b0005500 tx=0x7f13b0005fd0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.983+0000 7f13c65d5740 1 -- 172.21.15.2:0/2714938700 shutdown_connections 2024-09-10T08:16:41.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.983+0000 7f13c65d5740 1 --2- 172.21.15.2:0/2714938700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5f8ca0 0x55cbad60e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.983+0000 7f13c65d5740 1 --2- 172.21.15.2:0/2714938700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad60df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.983+0000 7f13c65d5740 1 --2- 172.21.15.2:0/2714938700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad5f8040 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.983+0000 7f13c65d5740 1 -- 172.21.15.2:0/2714938700 >> 172.21.15.2:0/2714938700 conn(0x55cbad5ea050 msgr2=0x55cbad604530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:41.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.984+0000 7f13c65d5740 1 -- 172.21.15.2:0/2714938700 shutdown_connections 2024-09-10T08:16:41.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.984+0000 7f13c65d5740 1 -- 172.21.15.2:0/2714938700 wait complete. 2024-09-10T08:16:41.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.984+0000 7f13c65d5740 1 -- start start 2024-09-10T08:16:41.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.984+0000 7f13c65d5740 1 Processor -- start 2024-09-10T08:16:41.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.985+0000 7f13c65d5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad60e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.985+0000 7f13c65d5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad60ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.985+0000 7f13c65d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5f8ca0 0x55cbad60bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.985+0000 7f13c65d5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b3af0 con 0x55cbad5ee3e0 2024-09-10T08:16:41.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.985+0000 7f13c65d5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b4f60 con 0x55cbad5edac0 2024-09-10T08:16:41.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.985+0000 7f13c65d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b41c0 con 0x55cbad5f8ca0 2024-09-10T08:16:41.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.985+0000 7f13c65d5740 10 client.0 _ll_register_callbacks cb 0x7f13a80d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:41.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.985+0000 7f13bbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad60ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.985+0000 7f13bbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad60ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37930/0 (socket says 172.21.15.2:37930) 2024-09-10T08:16:41.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.985+0000 7f13bbe00640 1 -- 172.21.15.2:0/3749039693 learned_addr learned my addr 172.21.15.2:0/3749039693 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:41.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.985+0000 7f13c1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad60e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.985+0000 7f13c1a00640 1 --2- 172.21.15.2:0/3749039693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5f8ca0 0x55cbad60bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.985+0000 7f13c1000640 1 -- 172.21.15.2:0/3749039693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5f8ca0 msgr2=0x55cbad60bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:41.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.986+0000 7f13c1000640 1 --2- 172.21.15.2:0/3749039693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5f8ca0 0x55cbad60bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.986+0000 7f13c1000640 1 -- 172.21.15.2:0/3749039693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 msgr2=0x55cbad60ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:41.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.986+0000 7f13c1000640 1 --2- 172.21.15.2:0/3749039693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad60ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.986+0000 7f13c1000640 1 -- 172.21.15.2:0/3749039693 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbad60c800 con 0x55cbad5edac0 2024-09-10T08:16:41.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.986+0000 7f13c1000640 1 --2- 172.21.15.2:0/3749039693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad60e700 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f13b0003690 tx=0x7f13b0005950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e187f32742bcf7bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:41.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.986+0000 7f13a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:41.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.986+0000 7f13a2a00640 1 -- 172.21.15.2:0/3749039693 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0040040 con 0x55cbad5edac0 2024-09-10T08:16:41.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.986+0000 7f13c65d5740 1 -- 172.21.15.2:0/3749039693 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13b0003890 con 0x55cbad5edac0 2024-09-10T08:16:41.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.987+0000 7f13a2a00640 1 -- 172.21.15.2:0/3749039693 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f13b0004240 con 0x55cbad5edac0 2024-09-10T08:16:41.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.987+0000 7f13a2a00640 1 -- 172.21.15.2:0/3749039693 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f13b0009520 con 0x55cbad5edac0 2024-09-10T08:16:41.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.987+0000 7f13c65d5740 10 client.27776 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:41.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.987+0000 7f13c65d5740 1 -- 172.21.15.2:0/3749039693 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cbad673510 con 0x55cbad5edac0 2024-09-10T08:16:41.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.987+0000 7f13a2a00640 1 -- 172.21.15.2:0/3749039693 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13b0003890 con 0x55cbad5edac0 2024-09-10T08:16:41.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.987+0000 7f13a2a00640 1 -- 172.21.15.2:0/3749039693 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f13b005b030 con 0x55cbad5edac0 2024-09-10T08:16:41.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.987+0000 7f13c65d5740 10 client.27776 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:41.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.987+0000 7f13c65d5740 10 client.27776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:41.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.987+0000 7f13c65d5740 20 client.27776 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:41.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.987+0000 7f13c65d5740 1 --2- 172.21.15.2:0/3749039693 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cbad6739e0 0x55cbad675dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:41.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.988+0000 7f13c65d5740 4 client.27776 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:16:41.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.988+0000 7f13c65d5740 1 -- 172.21.15.2:0/3749039693 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cbad55bd10 con 0x55cbad6739e0 2024-09-10T08:16:41.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.988+0000 7f13bbe00640 1 --2- 172.21.15.2:0/3749039693 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cbad6739e0 0x55cbad675dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:41.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.988+0000 7f13bbe00640 1 --2- 172.21.15.2:0/3749039693 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cbad6739e0 0x55cbad675dd0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a81b993737642886 server_cookie=704cf452d746d46e in_seq=0 out_seq=0 2024-09-10T08:16:41.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.988+0000 7f13a2a00640 10 client.27776 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:41.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.989+0000 7f13a2a00640 1 -- 172.21.15.2:0/3749039693 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cbad55bd10 con 0x55cbad6739e0 2024-09-10T08:16:41.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.989+0000 7f13a2a00640 10 client.27776 handle_command_reply: tid=0 2024-09-10T08:16:41.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.989+0000 7f13c65d5740 1 client.27776 shutdown 2024-09-10T08:16:41.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.989+0000 7f13c65d5740 1 -- 172.21.15.2:0/3749039693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 msgr2=0x55cbad60e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:41.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.989+0000 7f13c65d5740 1 --2- 172.21.15.2:0/3749039693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad60e700 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f13b0003690 tx=0x7f13b0005950 comp rx=0 tx=0).stop 2024-09-10T08:16:41.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.989+0000 7f13c65d5740 1 -- 172.21.15.2:0/3749039693 shutdown_connections 2024-09-10T08:16:41.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.989+0000 7f13c65d5740 1 --2- 172.21.15.2:0/3749039693 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cbad6739e0 0x55cbad675dd0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.990+0000 7f13a2a00640 0 client.27776 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:41.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.990+0000 7f13c65d5740 1 --2- 172.21.15.2:0/3749039693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbad5f8ca0 0x55cbad60bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.990+0000 7f13c65d5740 1 --2- 172.21.15.2:0/3749039693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbad5ee3e0 0x55cbad60ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.990+0000 7f13c65d5740 1 --2- 172.21.15.2:0/3749039693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cbad5edac0 0x55cbad60e700 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.990+0000 7f13c65d5740 1 -- 172.21.15.2:0/3749039693 >> 172.21.15.2:0/3749039693 conn(0x55cbad5ea050 msgr2=0x55cbad5ef480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:41.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.990+0000 7f13c65d5740 1 -- 172.21.15.2:0/3749039693 shutdown_connections 2024-09-10T08:16:41.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.990+0000 7f13c65d5740 1 -- 172.21.15.2:0/3749039693 wait complete. 2024-09-10T08:16:41.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.990+0000 7f13c65d5740 20 client.27776 trim_cache size 0 max 16384 2024-09-10T08:16:41.993 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:16:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:16:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "eaff8a57-c0d6-4691-9f73-c3d64c555e79", 2024-09-10T08:16:41.993 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:16:41.993 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:16:41.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.991+0000 7f13c3a00640 1 -- 172.21.15.2:0/2059440895 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13a40a2b80 msgr2=0x7f13a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:41.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.991+0000 7f13c3a00640 1 --2- 172.21.15.2:0/2059440895 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13a40a2b80 0x7f13a40a5040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009fd0 tx=0x7f13b4009500 comp rx=0 tx=0).stop 2024-09-10T08:16:41.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.991+0000 7f13c3a00640 1 -- 172.21.15.2:0/2059440895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13bc1049a0 msgr2=0x7f13bc076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:41.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.991+0000 7f13c3a00640 1 --2- 172.21.15.2:0/2059440895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13bc1049a0 0x7f13bc076c40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f13a800ea20 tx=0x7f13a800eee0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.991+0000 7f13c3a00640 1 -- 172.21.15.2:0/2059440895 shutdown_connections 2024-09-10T08:16:41.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.991+0000 7f13c3a00640 1 --2- 172.21.15.2:0/2059440895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13bc109120 0x7f13bc077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.991+0000 7f13c3a00640 1 --2- 172.21.15.2:0/2059440895 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f13a40a2b80 0x7f13a40a5040 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.991+0000 7f13c3a00640 1 --2- 172.21.15.2:0/2059440895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13bc1049a0 0x7f13bc076c40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.991+0000 7f13c3a00640 1 --2- 172.21.15.2:0/2059440895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13bc103ff0 0x7f13bc076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:41.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.991+0000 7f13c3a00640 1 -- 172.21.15.2:0/2059440895 >> 172.21.15.2:0/2059440895 conn(0x7f13bc0ffa90 msgr2=0x7f13bc100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:41.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.991+0000 7f13c3a00640 1 -- 172.21.15.2:0/2059440895 shutdown_connections 2024-09-10T08:16:41.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:41.991+0000 7f13c3a00640 1 -- 172.21.15.2:0/2059440895 wait complete. 2024-09-10T08:16:42.747 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:16:42.747 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:16:42.747 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:16:42.785 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:16:42.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:16:42.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:16:42.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:16:42.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:16:42.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:16:42.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:16:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:43.652 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:43 smithi002 ceph-mon[30563]: pgmap v644: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:16:43.652 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:43 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:16:43.652 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:43 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:16:43.652 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:43 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:16:43.652 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:43 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:16:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.767+0000 7fdfd4e00640 1 -- 172.21.15.2:0/534760104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfd0102790 msgr2=0x7fdfd01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.767+0000 7fdfd4e00640 1 --2- 172.21.15.2:0/534760104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfd0102790 0x7fdfd01067e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009900 tx=0x7fdfbc02a410 comp rx=0 tx=0).stop 2024-09-10T08:16:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.767+0000 7fdfd4e00640 1 -- 172.21.15.2:0/534760104 shutdown_connections 2024-09-10T08:16:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.767+0000 7fdfd4e00640 1 --2- 172.21.15.2:0/534760104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd0106f10 0x7fdfd010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.767+0000 7fdfd4e00640 1 --2- 172.21.15.2:0/534760104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfd0102790 0x7fdfd01067e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.767+0000 7fdfd4e00640 1 --2- 172.21.15.2:0/534760104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfd0101de0 0x7fdfd01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.767+0000 7fdfd4e00640 1 -- 172.21.15.2:0/534760104 >> 172.21.15.2:0/534760104 conn(0x7fdfd00fd8c0 msgr2=0x7fdfd00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.767+0000 7fdfd4e00640 1 -- 172.21.15.2:0/534760104 shutdown_connections 2024-09-10T08:16:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.767+0000 7fdfd4e00640 1 -- 172.21.15.2:0/534760104 wait complete. 2024-09-10T08:16:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.768+0000 7fdfd4e00640 1 Processor -- start 2024-09-10T08:16:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.768+0000 7fdfd4e00640 1 -- start start 2024-09-10T08:16:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfd4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfd0101de0 0x7fdfd019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfd4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfd0102790 0x7fdfd019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfd4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd0106f10 0x7fdfd019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfd4e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd0111fc0 con 0x7fdfd0102790 2024-09-10T08:16:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfd4e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd0111e40 con 0x7fdfd0106f10 2024-09-10T08:16:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfd4e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd0112140 con 0x7fdfd0101de0 2024-09-10T08:16:43.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfcd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfd0102790 0x7fdfd019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfcde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfd0101de0 0x7fdfd019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfcd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfd0102790 0x7fdfd019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37946/0 (socket says 172.21.15.2:37946) 2024-09-10T08:16:43.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfcd400640 1 -- 172.21.15.2:0/2921329860 learned_addr learned my addr 172.21.15.2:0/2921329860 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:43.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfce800640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd0106f10 0x7fdfd019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfcd400640 1 -- 172.21.15.2:0/2921329860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfd0101de0 msgr2=0x7fdfd019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:43.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfcd400640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfd0101de0 0x7fdfd019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfcd400640 1 -- 172.21.15.2:0/2921329860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd0106f10 msgr2=0x7fdfd019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:43.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.769+0000 7fdfcd400640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd0106f10 0x7fdfd019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.770+0000 7fdfcd400640 1 -- 172.21.15.2:0/2921329860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd019f600 con 0x7fdfd0102790 2024-09-10T08:16:43.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.770+0000 7fdfcde00640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfd0101de0 0x7fdfd019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:43.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.770+0000 7fdfcd400640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfd0102790 0x7fdfd019ab90 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc004770 tx=0x7fdfbc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:43.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.770+0000 7fdfc6a00640 1 -- 172.21.15.2:0/2921329860 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc03f070 con 0x7fdfd0102790 2024-09-10T08:16:43.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.770+0000 7fdfc6a00640 1 -- 172.21.15.2:0/2921329860 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfbc031ca0 con 0x7fdfd0102790 2024-09-10T08:16:43.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.770+0000 7fdfd4e00640 1 -- 172.21.15.2:0/2921329860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd019f890 con 0x7fdfd0102790 2024-09-10T08:16:43.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.770+0000 7fdfc6a00640 1 -- 172.21.15.2:0/2921329860 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc035030 con 0x7fdfd0102790 2024-09-10T08:16:43.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.770+0000 7fdfd4e00640 1 -- 172.21.15.2:0/2921329860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd01a71d0 con 0x7fdfd0102790 2024-09-10T08:16:43.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.773+0000 7fdfd78bb740 1 Processor -- start 2024-09-10T08:16:43.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.773+0000 7fdfc6a00640 1 -- 172.21.15.2:0/2921329860 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fdfbc042070 con 0x7fdfd0102790 2024-09-10T08:16:43.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.774+0000 7fdfd78bb740 1 -- start start 2024-09-10T08:16:43.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.774+0000 7fdfd78bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.774+0000 7fdfd78bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428bf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.774+0000 7fdfd78bb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 0x5641428c2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.774+0000 7fdfd78bb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56414281b3e0 con 0x5641428bfd70 2024-09-10T08:16:43.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.774+0000 7fdfd78bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5641427c2d80 con 0x5641428b54b0 2024-09-10T08:16:43.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.774+0000 7fdfd78bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5641428173a0 con 0x5641428b4b90 2024-09-10T08:16:43.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.774+0000 7fdfce800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.774+0000 7fdfce800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428b4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54462/0 (socket says 172.21.15.2:54462) 2024-09-10T08:16:43.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.774+0000 7fdfce800640 1 -- 172.21.15.2:0/825650714 learned_addr learned my addr 172.21.15.2:0/825650714 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:43.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.775+0000 7fdfcd400640 1 --2- 172.21.15.2:0/825650714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 0x5641428c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.775+0000 7fdfcde00640 1 --2- 172.21.15.2:0/825650714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428bf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.775+0000 7fdfce800640 1 -- 172.21.15.2:0/825650714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 msgr2=0x5641428bf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:43.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.775+0000 7fdfce800640 1 --2- 172.21.15.2:0/825650714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428bf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.775+0000 7fdfce800640 1 -- 172.21.15.2:0/825650714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 msgr2=0x5641428c2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:43.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.775+0000 7fdfce800640 1 --2- 172.21.15.2:0/825650714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 0x5641428c2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.775+0000 7fdfce800640 1 -- 172.21.15.2:0/825650714 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641427e1ae0 con 0x5641428b4b90 2024-09-10T08:16:43.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.775+0000 7fdfce800640 1 --2- 172.21.15.2:0/825650714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428b4f70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009f90 tx=0x7fdfc8010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee9f44d8fa48b8a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.775+0000 7fdfc6a00640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdfb40a2c00 0x7fdfb40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.775+0000 7fdfbb400640 1 -- 172.21.15.2:0/825650714 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8011700 con 0x5641428b4b90 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.775+0000 7fdfbb400640 1 -- 172.21.15.2:0/825650714 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfc8011d80 con 0x5641428b4b90 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.775+0000 7fdfbb400640 1 -- 172.21.15.2:0/825650714 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8018380 con 0x5641428b4b90 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.776+0000 7fdfc6a00640 1 -- 172.21.15.2:0/2921329860 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fdfbc0efa90 con 0x7fdfd0102790 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.776+0000 7fdfcde00640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdfb40a2c00 0x7fdfb40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.776+0000 7fdfd78bb740 1 -- 172.21.15.2:0/825650714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 msgr2=0x5641428b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.776+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/825650714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428b4f70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009f90 tx=0x7fdfc8010860 comp rx=0 tx=0).stop 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.776+0000 7fdfd78bb740 1 -- 172.21.15.2:0/825650714 shutdown_connections 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.776+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/825650714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 0x5641428c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.776+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/825650714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428bf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.776+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/825650714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428b4f70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.776+0000 7fdfd78bb740 1 -- 172.21.15.2:0/825650714 >> 172.21.15.2:0/825650714 conn(0x5641428b1120 msgr2=0x5641428b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.776+0000 7fdfcde00640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdfb40a2c00 0x7fdfb40a50c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x5641428b1f80 tx=0x7fdfc0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.776+0000 7fdfd78bb740 1 -- 172.21.15.2:0/825650714 shutdown_connections 2024-09-10T08:16:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.776+0000 7fdfd78bb740 1 -- 172.21.15.2:0/825650714 wait complete. 2024-09-10T08:16:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.777+0000 7fdfd78bb740 1 -- start start 2024-09-10T08:16:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.777+0000 7fdfd78bb740 1 Processor -- start 2024-09-10T08:16:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.778+0000 7fdfd78bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428bcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.778+0000 7fdfd78bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428bd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.778+0000 7fdfd78bb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 0x5641428bd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.778+0000 7fdfd78bb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5641428c29e0 con 0x5641428bfd70 2024-09-10T08:16:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.778+0000 7fdfd78bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5641427d60f0 con 0x5641428b54b0 2024-09-10T08:16:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.778+0000 7fdfd78bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5641428c2b60 con 0x5641428b4b90 2024-09-10T08:16:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.778+0000 7fdfd78bb740 10 client.0 _ll_register_callbacks cb 0x5641427c79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.778+0000 7fdfcd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428bd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.778+0000 7fdfce800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 0x5641428bd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.778+0000 7fdfcd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428bd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35186/0 (socket says 172.21.15.2:35186) 2024-09-10T08:16:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.778+0000 7fdfcd400640 1 -- 172.21.15.2:0/2660511703 learned_addr learned my addr 172.21.15.2:0/2660511703 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.778+0000 7fdfcde00640 1 --2- 172.21.15.2:0/2660511703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428bcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.779+0000 7fdfcd400640 1 -- 172.21.15.2:0/2660511703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 msgr2=0x5641428bcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.779+0000 7fdfcd400640 1 --2- 172.21.15.2:0/2660511703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428bcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.779+0000 7fdfcd400640 1 -- 172.21.15.2:0/2660511703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 msgr2=0x5641428bd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.779+0000 7fdfcd400640 1 --2- 172.21.15.2:0/2660511703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 0x5641428bd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.779+0000 7fdfcd400640 1 -- 172.21.15.2:0/2660511703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414258c3c0 con 0x5641428b54b0 2024-09-10T08:16:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.779+0000 7fdfcde00640 1 --2- 172.21.15.2:0/2660511703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428bcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:16:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.779+0000 7fdfcd400640 1 --2- 172.21.15.2:0/2660511703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428bd1b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccdd62738162f284 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.779+0000 7fdfafe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.779+0000 7fdfafe00640 1 -- 172.21.15.2:0/2660511703 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc049070 con 0x5641428b54b0 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.779+0000 7fdfafe00640 1 -- 172.21.15.2:0/2660511703 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fdfbc0ed250 con 0x5641428b54b0 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.779+0000 7fdfd78bb740 1 -- 172.21.15.2:0/2660511703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc8011d80 con 0x5641428b54b0 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.779+0000 7fdfafe00640 1 -- 172.21.15.2:0/2660511703 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdfbc005d50 con 0x5641428b54b0 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.779+0000 7fdfd78bb740 10 client.27788 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.780+0000 7fdfd78bb740 1 -- 172.21.15.2:0/2660511703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5641428ce6d0 con 0x5641428b54b0 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.780+0000 7fdfafe00640 1 -- 172.21.15.2:0/2660511703 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfc8011d80 con 0x5641428b54b0 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.780+0000 7fdfafe00640 1 -- 172.21.15.2:0/2660511703 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdfbc0b85c0 con 0x5641428b54b0 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.780+0000 7fdfd78bb740 10 client.27788 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.780+0000 7fdfd78bb740 10 client.27788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.780+0000 7fdfd78bb740 20 client.27788 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.780+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/2660511703 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5641428cfb30 0x5641428d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.780+0000 7fdfd78bb740 4 client.27788 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.780+0000 7fdfd78bb740 1 -- 172.21.15.2:0/2660511703 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564142828950 con 0x5641428cfb30 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.781+0000 7fdfcde00640 1 --2- 172.21.15.2:0/2660511703 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5641428cfb30 0x5641428d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.781+0000 7fdfcde00640 1 --2- 172.21.15.2:0/2660511703 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5641428cfb30 0x5641428d1f50 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc7a62c89bf037a0 server_cookie=735deb61e89aeee9 in_seq=0 out_seq=0 2024-09-10T08:16:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.781+0000 7fdfafe00640 10 client.27788 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.782+0000 7fdfafe00640 1 -- 172.21.15.2:0/2660511703 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564142828950 con 0x5641428cfb30 2024-09-10T08:16:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.782+0000 7fdfafe00640 10 client.27788 handle_command_reply: tid=0 2024-09-10T08:16:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.782+0000 7fdfd78bb740 1 client.27788 shutdown 2024-09-10T08:16:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.782+0000 7fdfd78bb740 1 -- 172.21.15.2:0/2660511703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 msgr2=0x5641428bd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.782+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/2660511703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428bd1b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc02a410 comp rx=0 tx=0).stop 2024-09-10T08:16:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.782+0000 7fdfd78bb740 1 -- 172.21.15.2:0/2660511703 shutdown_connections 2024-09-10T08:16:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.782+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/2660511703 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5641428cfb30 0x5641428d1f50 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.782+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/2660511703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 0x5641428bd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.782+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/2660511703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428bd1b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.782+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/2660511703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428bcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.782+0000 7fdfafe00640 0 client.27788 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.782+0000 7fdfd78bb740 1 -- 172.21.15.2:0/2660511703 >> 172.21.15.2:0/2660511703 conn(0x5641428b1120 msgr2=0x5641428b6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.783+0000 7fdfd78bb740 1 -- 172.21.15.2:0/2660511703 shutdown_connections 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.783+0000 7fdfd78bb740 1 -- 172.21.15.2:0/2660511703 wait complete. 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.783+0000 7fdfd78bb740 20 client.27788 trim_cache size 0 max 16384 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.801+0000 7fdfd78bb740 1 Processor -- start 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.801+0000 7fdfd78bb740 1 -- start start 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.801+0000 7fdfd78bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b4b90 0x5641428beb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.801+0000 7fdfd78bb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428b54b0 0x5641428d7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.801+0000 7fdfd78bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428bfd70 0x5641428d7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.801+0000 7fdfd78bb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01b39a0 con 0x5641428b54b0 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.801+0000 7fdfd78bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01b1270 con 0x5641428b4b90 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.801+0000 7fdfd78bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01af0c0 con 0x5641428bfd70 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.801+0000 7fdfcd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428b54b0 0x5641428d7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.801+0000 7fdfcd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428b54b0 0x5641428d7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37978/0 (socket says 172.21.15.2:37978) 2024-09-10T08:16:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.801+0000 7fdfcd400640 1 -- 172.21.15.2:0/461476448 learned_addr learned my addr 172.21.15.2:0/461476448 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.801+0000 7fdfcde00640 1 --2- 172.21.15.2:0/461476448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b4b90 0x5641428beb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.802+0000 7fdfce800640 1 --2- 172.21.15.2:0/461476448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428bfd70 0x5641428d7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.802+0000 7fdfcd400640 1 -- 172.21.15.2:0/461476448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428bfd70 msgr2=0x5641428d7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.802+0000 7fdfcd400640 1 --2- 172.21.15.2:0/461476448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428bfd70 0x5641428d7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.802+0000 7fdfcd400640 1 -- 172.21.15.2:0/461476448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b4b90 msgr2=0x5641428beb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.802+0000 7fdfcd400640 1 --2- 172.21.15.2:0/461476448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b4b90 0x5641428beb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.802+0000 7fdfcd400640 1 -- 172.21.15.2:0/461476448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641428d25b0 con 0x5641428b54b0 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.802+0000 7fdfcd400640 1 --2- 172.21.15.2:0/461476448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428b54b0 0x5641428d7840 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0b83b0 tx=0x7fdfbc0ba460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fbc241e331ad2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.803+0000 7fdfbaa00640 1 -- 172.21.15.2:0/461476448 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc049030 con 0x5641428b54b0 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.803+0000 7fdfbaa00640 1 -- 172.21.15.2:0/461476448 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfbc0bc1b0 con 0x5641428b54b0 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.803+0000 7fdfbaa00640 1 -- 172.21.15.2:0/461476448 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0d8080 con 0x5641428b54b0 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.803+0000 7fdfd78bb740 1 -- 172.21.15.2:0/461476448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428b54b0 msgr2=0x5641428d7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.803+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/461476448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428b54b0 0x5641428d7840 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0b83b0 tx=0x7fdfbc0ba460 comp rx=0 tx=0).stop 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.803+0000 7fdfd78bb740 1 -- 172.21.15.2:0/461476448 shutdown_connections 2024-09-10T08:16:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.804+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/461476448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428bfd70 0x5641428d7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.804+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/461476448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428b54b0 0x5641428d7840 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.804+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/461476448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b4b90 0x5641428beb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.804+0000 7fdfd78bb740 1 -- 172.21.15.2:0/461476448 >> 172.21.15.2:0/461476448 conn(0x5641428b1120 msgr2=0x5641428b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:43.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.804+0000 7fdfd78bb740 1 -- 172.21.15.2:0/461476448 shutdown_connections 2024-09-10T08:16:43.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.804+0000 7fdfd78bb740 1 -- 172.21.15.2:0/461476448 wait complete. 2024-09-10T08:16:43.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.804+0000 7fdfd78bb740 1 -- start start 2024-09-10T08:16:43.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.804+0000 7fdfd78bb740 1 Processor -- start 2024-09-10T08:16:43.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfd78bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfd78bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfd78bb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 0x5641428c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfd78bb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01af730 con 0x5641428bfd70 2024-09-10T08:16:43.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfd78bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01b0ba0 con 0x5641428b54b0 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfd78bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01afe00 con 0x5641428b4b90 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfd78bb740 10 client.0 _ll_register_callbacks cb 0x7fdfbc0b82e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfcde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfcde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54498/0 (socket says 172.21.15.2:54498) 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfcde00640 1 -- 172.21.15.2:0/3051261113 learned_addr learned my addr 172.21.15.2:0/3051261113 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfce800640 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 0x5641428c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfcd400640 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfcde00640 1 -- 172.21.15.2:0/3051261113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 msgr2=0x5641428c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfcde00640 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfcde00640 1 -- 172.21.15.2:0/3051261113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 msgr2=0x5641428c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfcde00640 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 0x5641428c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfcde00640 1 -- 172.21.15.2:0/3051261113 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641428b49c0 con 0x5641428b4b90 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.805+0000 7fdfce800640 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 0x5641428c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:16:43.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.806+0000 7fdfcde00640 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428c8350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x5641428d2780 tx=0x7fdfc00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=963a81405d0fc3c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.806+0000 7fdfafe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.806+0000 7fdfafe00640 1 -- 172.21.15.2:0/3051261113 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc002e070 con 0x5641428b4b90 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.806+0000 7fdfd78bb740 1 -- 172.21.15.2:0/3051261113 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfbc0bc1b0 con 0x5641428b4b90 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.806+0000 7fdfafe00640 1 -- 172.21.15.2:0/3051261113 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fdfc002a4f0 con 0x5641428b4b90 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.806+0000 7fdfafe00640 1 -- 172.21.15.2:0/3051261113 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdfc003c050 con 0x5641428b4b90 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.806+0000 7fdfd78bb740 10 client.27733 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.806+0000 7fdfd78bb740 1 -- 172.21.15.2:0/3051261113 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5641428b21d0 con 0x5641428b4b90 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.807+0000 7fdfafe00640 1 -- 172.21.15.2:0/3051261113 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfbc0bc1b0 con 0x5641428b4b90 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.807+0000 7fdfafe00640 1 -- 172.21.15.2:0/3051261113 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdfc0033a80 con 0x5641428b4b90 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.807+0000 7fdfd78bb740 10 client.27733 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.807+0000 7fdfd78bb740 10 client.27733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.807+0000 7fdfd78bb740 20 client.27733 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.807+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56414293ba00 0x56414293de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:43.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.807+0000 7fdfd78bb740 4 client.27733 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:16:43.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.807+0000 7fdfd78bb740 1 -- 172.21.15.2:0/3051261113 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564142822e60 con 0x56414293ba00 2024-09-10T08:16:43.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.807+0000 7fdfcd400640 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56414293ba00 0x56414293de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:43.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfcd400640 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56414293ba00 0x56414293de20 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29929a5009943ad server_cookie=f18fd7d3ef09a933 in_seq=0 out_seq=0 2024-09-10T08:16:43.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfafe00640 10 client.27733 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:43.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfafe00640 1 -- 172.21.15.2:0/3051261113 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564142822e60 con 0x56414293ba00 2024-09-10T08:16:43.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfafe00640 10 client.27733 handle_command_reply: tid=0 2024-09-10T08:16:43.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfd78bb740 1 client.27733 shutdown 2024-09-10T08:16:43.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfd78bb740 1 -- 172.21.15.2:0/3051261113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 msgr2=0x5641428c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:43.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428c8350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x5641428d2780 tx=0x7fdfc00046c0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfd78bb740 1 -- 172.21.15.2:0/3051261113 shutdown_connections 2024-09-10T08:16:43.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56414293ba00 0x56414293de20 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641428bfd70 0x5641428c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5641428b54b0 0x5641428c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfd78bb740 1 --2- 172.21.15.2:0/3051261113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641428b4b90 0x5641428c8350 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfd78bb740 1 -- 172.21.15.2:0/3051261113 >> 172.21.15.2:0/3051261113 conn(0x5641428b1120 msgr2=0x5641428bcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:43.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfafe00640 0 client.27733 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:43.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfd78bb740 1 -- 172.21.15.2:0/3051261113 shutdown_connections 2024-09-10T08:16:43.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfd78bb740 1 -- 172.21.15.2:0/3051261113 wait complete. 2024-09-10T08:16:43.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.808+0000 7fdfd78bb740 20 client.27733 trim_cache size 0 max 16384 2024-09-10T08:16:43.822 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:16:43.822 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:16:43.822 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:16:43.822 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:16:43.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.809+0000 7fdfd4e00640 1 -- 172.21.15.2:0/2921329860 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdfb40a2c00 msgr2=0x7fdfb40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:43.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.809+0000 7fdfd4e00640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdfb40a2c00 0x7fdfb40a50c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x5641428b1f80 tx=0x7fdfc0005f90 comp rx=0 tx=0).stop 2024-09-10T08:16:43.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.809+0000 7fdfd4e00640 1 -- 172.21.15.2:0/2921329860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfd0102790 msgr2=0x7fdfd019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:43.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.809+0000 7fdfd4e00640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfd0102790 0x7fdfd019ab90 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc004770 tx=0x7fdfbc003c10 comp rx=0 tx=0).stop 2024-09-10T08:16:43.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.809+0000 7fdfd4e00640 1 -- 172.21.15.2:0/2921329860 shutdown_connections 2024-09-10T08:16:43.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.809+0000 7fdfd4e00640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd0106f10 0x7fdfd019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.809+0000 7fdfd4e00640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfd0102790 0x7fdfd019ab90 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.809+0000 7fdfd4e00640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fdfb40a2c00 0x7fdfb40a50c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.809+0000 7fdfd4e00640 1 --2- 172.21.15.2:0/2921329860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfd0101de0 0x7fdfd019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:43.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.809+0000 7fdfd4e00640 1 -- 172.21.15.2:0/2921329860 >> 172.21.15.2:0/2921329860 conn(0x7fdfd00fd8c0 msgr2=0x7fdfd00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:43.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.809+0000 7fdfd4e00640 1 -- 172.21.15.2:0/2921329860 shutdown_connections 2024-09-10T08:16:43.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:43.809+0000 7fdfd4e00640 1 -- 172.21.15.2:0/2921329860 wait complete. 2024-09-10T08:16:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:43 smithi155 ceph-mon[35405]: pgmap v644: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:16:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:43 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:16:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:43 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:16:43.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:43 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:16:43.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:43 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:16:43.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:43 smithi107 ceph-mon[35048]: pgmap v644: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:16:43.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:43 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:16:43.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:43 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:16:43.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:43 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:16:43.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:43 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:16:44.588 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:16:44.589 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:16:44.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:44 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:16:44.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:44 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:16:44.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:44 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:16:44.968 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:45.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.519+0000 7f745d000640 1 -- 172.21.15.2:0/3939906504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74581047e0 msgr2=0x7f7458108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.519+0000 7f745d000640 1 --2- 172.21.15.2:0/3939906504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74581047e0 0x7f7458108830 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7444009900 tx=0x7f744402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:16:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.519+0000 7f745d000640 1 -- 172.21.15.2:0/3939906504 shutdown_connections 2024-09-10T08:16:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.519+0000 7f745d000640 1 --2- 172.21.15.2:0/3939906504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7458108f60 0x7f7458110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.519+0000 7f745d000640 1 --2- 172.21.15.2:0/3939906504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74581047e0 0x7f7458108830 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.519+0000 7f745d000640 1 --2- 172.21.15.2:0/3939906504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7458103e30 0x7f7458104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.519+0000 7f745d000640 1 -- 172.21.15.2:0/3939906504 >> 172.21.15.2:0/3939906504 conn(0x7f74580ff910 msgr2=0x7f7458101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.519+0000 7f745d000640 1 -- 172.21.15.2:0/3939906504 shutdown_connections 2024-09-10T08:16:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.519+0000 7f745d000640 1 -- 172.21.15.2:0/3939906504 wait complete. 2024-09-10T08:16:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.520+0000 7f745d000640 1 Processor -- start 2024-09-10T08:16:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.520+0000 7f745d000640 1 -- start start 2024-09-10T08:16:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.521+0000 7f745d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7458103e30 0x7f745819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.521+0000 7f745d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74581047e0 0x7f745819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.521+0000 7f745d000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7458108f60 0x7f745819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.521+0000 7f745d000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7458113fc0 con 0x7f7458108f60 2024-09-10T08:16:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.521+0000 7f745d000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7458113e40 con 0x7f74581047e0 2024-09-10T08:16:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.521+0000 7f745d000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7458114140 con 0x7f7458103e30 2024-09-10T08:16:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.521+0000 7f7457e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7458108f60 0x7f745819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.521+0000 7f7456a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74581047e0 0x7f745819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.521+0000 7f7457e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7458108f60 0x7f745819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37996/0 (socket says 172.21.15.2:37996) 2024-09-10T08:16:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.521+0000 7f7457e00640 1 -- 172.21.15.2:0/1803016610 learned_addr learned my addr 172.21.15.2:0/1803016610 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.521+0000 7f7457400640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7458103e30 0x7f745819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.522+0000 7f7457e00640 1 -- 172.21.15.2:0/1803016610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7458103e30 msgr2=0x7f745819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:45.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.522+0000 7f7457e00640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7458103e30 0x7f745819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:45.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.522+0000 7f7457e00640 1 -- 172.21.15.2:0/1803016610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74581047e0 msgr2=0x7f745819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:45.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.522+0000 7f7457e00640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74581047e0 0x7f745819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:45.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.522+0000 7f7457e00640 1 -- 172.21.15.2:0/1803016610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745819f470 con 0x7f7458108f60 2024-09-10T08:16:45.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.522+0000 7f7457400640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7458103e30 0x7f745819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:45.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.522+0000 7f7457e00640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7458108f60 0x7f745819ed90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f745000efd0 tx=0x7f745000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:45.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.522+0000 7f744fe00640 1 -- 172.21.15.2:0/1803016610 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450003b70 con 0x7f7458108f60 2024-09-10T08:16:45.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.522+0000 7f745d000640 1 -- 172.21.15.2:0/1803016610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745819f760 con 0x7f7458108f60 2024-09-10T08:16:45.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.522+0000 7f744fe00640 1 -- 172.21.15.2:0/1803016610 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7450003d10 con 0x7f7458108f60 2024-09-10T08:16:45.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.522+0000 7f744fe00640 1 -- 172.21.15.2:0/1803016610 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500109f0 con 0x7f7458108f60 2024-09-10T08:16:45.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.523+0000 7f745d000640 1 -- 172.21.15.2:0/1803016610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74581a7040 con 0x7f7458108f60 2024-09-10T08:16:45.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.524+0000 7f745d000640 1 -- 172.21.15.2:0/1803016610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7458105310 con 0x7f7458108f60 2024-09-10T08:16:45.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.526+0000 7f744fe00640 1 -- 172.21.15.2:0/1803016610 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7450010b90 con 0x7f7458108f60 2024-09-10T08:16:45.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.527+0000 7f744fe00640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f743c0a2b80 0x7f743c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:45.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.527+0000 7f744fe00640 1 -- 172.21.15.2:0/1803016610 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f74500cfc40 con 0x7f7458108f60 2024-09-10T08:16:45.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.528+0000 7f7457400640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f743c0a2b80 0x7f743c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:45.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.528+0000 7f7457400640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f743c0a2b80 0x7f743c0a5040 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7448009fc0 tx=0x7f7448005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:45.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.530+0000 7f744fe00640 1 -- 172.21.15.2:0/1803016610 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f745009c750 con 0x7f7458108f60 2024-09-10T08:16:45.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.754+0000 7f745d000640 1 -- 172.21.15.2:0/1803016610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f745810c1a0 con 0x7f7458108f60 2024-09-10T08:16:45.753 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:45 smithi002 ceph-mon[30563]: pgmap v645: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:16:45.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.755+0000 7f744fe00640 1 -- 172.21.15.2:0/1803016610 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f745009c050 con 0x7f7458108f60 2024-09-10T08:16:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:16:45.754 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:16:45.754 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:16:45.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.758+0000 7f745d000640 1 -- 172.21.15.2:0/1803016610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f743c0a2b80 msgr2=0x7f743c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:45.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.758+0000 7f745d000640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f743c0a2b80 0x7f743c0a5040 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7448009fc0 tx=0x7f7448005f90 comp rx=0 tx=0).stop 2024-09-10T08:16:45.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.758+0000 7f745d000640 1 -- 172.21.15.2:0/1803016610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7458108f60 msgr2=0x7f745819ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:45.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.758+0000 7f745d000640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7458108f60 0x7f745819ed90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f745000efd0 tx=0x7f745000c6a0 comp rx=0 tx=0).stop 2024-09-10T08:16:45.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.758+0000 7f745d000640 1 -- 172.21.15.2:0/1803016610 shutdown_connections 2024-09-10T08:16:45.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.758+0000 7f745d000640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7458108f60 0x7f745819ed90 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:45.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.758+0000 7f745d000640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74581047e0 0x7f745819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:45.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.758+0000 7f745d000640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f743c0a2b80 0x7f743c0a5040 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:45.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.758+0000 7f745d000640 1 --2- 172.21.15.2:0/1803016610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7458103e30 0x7f745819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:45.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.758+0000 7f745d000640 1 -- 172.21.15.2:0/1803016610 >> 172.21.15.2:0/1803016610 conn(0x7f74580ff910 msgr2=0x7f7458100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:45.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.758+0000 7f745d000640 1 -- 172.21.15.2:0/1803016610 shutdown_connections 2024-09-10T08:16:45.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:45.758+0000 7f745d000640 1 -- 172.21.15.2:0/1803016610 wait complete. 2024-09-10T08:16:45.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:45 smithi155 ceph-mon[35405]: pgmap v645: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:16:45.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:45 smithi107 ceph-mon[35048]: pgmap v645: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:16:46.506 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:16:46.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:46 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1803016610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:16:46.884 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:46.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:46 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1803016610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:16:46.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:46 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1803016610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:16:47.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.460+0000 7fc814600640 1 -- 172.21.15.2:0/1965820898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc80c103670 msgr2=0x7fc80c10bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:47.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.460+0000 7fc814600640 1 --2- 172.21.15.2:0/1965820898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc80c103670 0x7fc80c10bb60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009860 tx=0x7fc7f802a3d0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.460+0000 7fc814600640 1 -- 172.21.15.2:0/1965820898 shutdown_connections 2024-09-10T08:16:47.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.460+0000 7fc814600640 1 --2- 172.21.15.2:0/1965820898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c10c180 0x7fc80c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.460+0000 7fc814600640 1 --2- 172.21.15.2:0/1965820898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc80c103670 0x7fc80c10bb60 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.460+0000 7fc814600640 1 --2- 172.21.15.2:0/1965820898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc80c102d50 0x7fc80c103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.460+0000 7fc814600640 1 -- 172.21.15.2:0/1965820898 >> 172.21.15.2:0/1965820898 conn(0x7fc80c0fc7d0 msgr2=0x7fc80c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:47.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.460+0000 7fc814600640 1 -- 172.21.15.2:0/1965820898 shutdown_connections 2024-09-10T08:16:47.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.460+0000 7fc814600640 1 -- 172.21.15.2:0/1965820898 wait complete. 2024-09-10T08:16:47.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.462+0000 7fc814600640 1 Processor -- start 2024-09-10T08:16:47.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.462+0000 7fc814600640 1 -- start start 2024-09-10T08:16:47.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.462+0000 7fc814600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc80c102d50 0x7fc80c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.462+0000 7fc814600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c103670 0x7fc80c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.462+0000 7fc814600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc80c10c180 0x7fc80c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.462+0000 7fc814600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c111fd0 con 0x7fc80c102d50 2024-09-10T08:16:47.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.462+0000 7fc814600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c111e50 con 0x7fc80c10c180 2024-09-10T08:16:47.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.462+0000 7fc814600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c112150 con 0x7fc80c103670 2024-09-10T08:16:47.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.463+0000 7fc811c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc80c102d50 0x7fc80c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.463+0000 7fc811200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c103670 0x7fc80c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.463+0000 7fc811c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc80c102d50 0x7fc80c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50154/0 (socket says 172.21.15.2:50154) 2024-09-10T08:16:47.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.463+0000 7fc811c00640 1 -- 172.21.15.2:0/2607324499 learned_addr learned my addr 172.21.15.2:0/2607324499 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:47.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.463+0000 7fc812600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc80c10c180 0x7fc80c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.463+0000 7fc812600640 1 -- 172.21.15.2:0/2607324499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c103670 msgr2=0x7fc80c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:47.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.463+0000 7fc812600640 1 --2- 172.21.15.2:0/2607324499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c103670 0x7fc80c19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.463+0000 7fc812600640 1 -- 172.21.15.2:0/2607324499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc80c102d50 msgr2=0x7fc80c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:47.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.463+0000 7fc812600640 1 --2- 172.21.15.2:0/2607324499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc80c102d50 0x7fc80c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.463+0000 7fc812600640 1 -- 172.21.15.2:0/2607324499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c19f640 con 0x7fc80c10c180 2024-09-10T08:16:47.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.463+0000 7fc811c00640 1 --2- 172.21.15.2:0/2607324499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc80c102d50 0x7fc80c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:47.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.463+0000 7fc812600640 1 --2- 172.21.15.2:0/2607324499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc80c10c180 0x7fc80c19ef60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc80400ef00 tx=0x7fc80400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:47.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.464+0000 7fc80a000640 1 -- 172.21.15.2:0/2607324499 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804004b20 con 0x7fc80c10c180 2024-09-10T08:16:47.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.464+0000 7fc814600640 1 -- 172.21.15.2:0/2607324499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c19f930 con 0x7fc80c10c180 2024-09-10T08:16:47.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.464+0000 7fc80a000640 1 -- 172.21.15.2:0/2607324499 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc804003e50 con 0x7fc80c10c180 2024-09-10T08:16:47.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.464+0000 7fc80a000640 1 -- 172.21.15.2:0/2607324499 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80400b570 con 0x7fc80c10c180 2024-09-10T08:16:47.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.464+0000 7fc814600640 1 -- 172.21.15.2:0/2607324499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c1a7210 con 0x7fc80c10c180 2024-09-10T08:16:47.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.467+0000 7fc81722c740 1 Processor -- start 2024-09-10T08:16:47.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.467+0000 7fc80a000640 1 -- 172.21.15.2:0/2607324499 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc80401d020 con 0x7fc80c10c180 2024-09-10T08:16:47.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.467+0000 7fc81722c740 1 -- start start 2024-09-10T08:16:47.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.468+0000 7fc81722c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.468+0000 7fc81722c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1dcb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.468+0000 7fc81722c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1dce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.468+0000 7fc81722c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55faa1d273e0 con 0x55faa1dc0c70 2024-09-10T08:16:47.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.468+0000 7fc81722c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55faa1cced80 con 0x55faa1dc1590 2024-09-10T08:16:47.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.468+0000 7fc81722c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55faa1d233a0 con 0x55faa1dcbe50 2024-09-10T08:16:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.468+0000 7fc811200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.468+0000 7fc811c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1dcb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.468+0000 7fc811200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50160/0 (socket says 172.21.15.2:50160) 2024-09-10T08:16:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.468+0000 7fc811200640 1 -- 172.21.15.2:0/1789219465 learned_addr learned my addr 172.21.15.2:0/1789219465 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.468+0000 7fc811200640 1 --2- 172.21.15.2:0/1789219465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1dce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.469+0000 7fc811200640 1 -- 172.21.15.2:0/1789219465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 msgr2=0x55faa1dce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.469+0000 7fc811200640 1 --2- 172.21.15.2:0/1789219465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1dce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.469+0000 7fc811200640 1 -- 172.21.15.2:0/1789219465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 msgr2=0x55faa1dcb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:47.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.469+0000 7fc811200640 1 --2- 172.21.15.2:0/1789219465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1dcb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.469+0000 7fc811200640 1 -- 172.21.15.2:0/1789219465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa1cedae0 con 0x55faa1dc0c70 2024-09-10T08:16:47.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.469+0000 7fc811c00640 1 --2- 172.21.15.2:0/1789219465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1dcb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:47.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.469+0000 7fc80a000640 1 --2- 172.21.15.2:0/2607324499 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7f40a2b80 0x7fc7f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.469+0000 7fc811200640 1 --2- 172.21.15.2:0/1789219465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc1050 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8032ff0 tx=0x7fc7f8032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533760cae48b37a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:47.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.469+0000 7fc80a000640 1 -- 172.21.15.2:0/2607324499 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc8040cf1f0 con 0x7fc80c10c180 2024-09-10T08:16:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.469+0000 7fc7fea00640 1 -- 172.21.15.2:0/1789219465 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8004230 con 0x55faa1dc0c70 2024-09-10T08:16:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.469+0000 7fc811c00640 1 --2- 172.21.15.2:0/2607324499 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7f40a2b80 0x7fc7f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.469+0000 7fc7fea00640 1 -- 172.21.15.2:0/1789219465 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7f8047070 con 0x55faa1dc0c70 2024-09-10T08:16:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.469+0000 7fc7fea00640 1 -- 172.21.15.2:0/1789219465 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80046c0 con 0x55faa1dc0c70 2024-09-10T08:16:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.470+0000 7fc81722c740 1 -- 172.21.15.2:0/1789219465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 msgr2=0x55faa1dc1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.470+0000 7fc81722c740 1 --2- 172.21.15.2:0/1789219465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc1050 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8032ff0 tx=0x7fc7f8032f60 comp rx=0 tx=0).stop 2024-09-10T08:16:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.470+0000 7fc811c00640 1 --2- 172.21.15.2:0/2607324499 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7f40a2b80 0x7fc7f40a5040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc800009620 tx=0x7fc800005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.470+0000 7fc81722c740 1 -- 172.21.15.2:0/1789219465 shutdown_connections 2024-09-10T08:16:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.470+0000 7fc81722c740 1 --2- 172.21.15.2:0/1789219465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1dce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.470+0000 7fc81722c740 1 --2- 172.21.15.2:0/1789219465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1dcb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.470+0000 7fc81722c740 1 --2- 172.21.15.2:0/1789219465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc1050 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.470+0000 7fc81722c740 1 -- 172.21.15.2:0/1789219465 >> 172.21.15.2:0/1789219465 conn(0x55faa1dbd200 msgr2=0x55faa1dbd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.470+0000 7fc81722c740 1 -- 172.21.15.2:0/1789219465 shutdown_connections 2024-09-10T08:16:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.470+0000 7fc81722c740 1 -- 172.21.15.2:0/1789219465 wait complete. 2024-09-10T08:16:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.471+0000 7fc81722c740 1 -- start start 2024-09-10T08:16:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.471+0000 7fc81722c740 1 Processor -- start 2024-09-10T08:16:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc81722c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc81722c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1dc9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc81722c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1dc97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc81722c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55faa1dcea20 con 0x55faa1dc0c70 2024-09-10T08:16:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc81722c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55faa1ce20f0 con 0x55faa1dc1590 2024-09-10T08:16:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc81722c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55faa1dceba0 con 0x55faa1dcbe50 2024-09-10T08:16:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc81722c740 10 client.0 _ll_register_callbacks cb 0x55faa1cc9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:47.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc811200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1dc9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc811c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc811200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1dc9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:43914/0 (socket says 172.21.15.2:43914) 2024-09-10T08:16:47.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc811200640 1 -- 172.21.15.2:0/3926229787 learned_addr learned my addr 172.21.15.2:0/3926229787 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:47.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc812600640 1 --2- 172.21.15.2:0/3926229787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1dc97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc811200640 1 -- 172.21.15.2:0/3926229787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 msgr2=0x55faa1dc97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc811200640 1 --2- 172.21.15.2:0/3926229787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1dc97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc811200640 1 -- 172.21.15.2:0/3926229787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 msgr2=0x55faa1dc8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc811200640 1 --2- 172.21.15.2:0/3926229787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.472+0000 7fc811200640 1 -- 172.21.15.2:0/3926229787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa1a983c0 con 0x55faa1dc1590 2024-09-10T08:16:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.473+0000 7fc811200640 1 --2- 172.21.15.2:0/3926229787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1dc9260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8005360 tx=0x7fc7f80305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1385cfd11e31b1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.473+0000 7fc7f3400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.473+0000 7fc7f3400640 1 -- 172.21.15.2:0/3926229787 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8050020 con 0x55faa1dc1590 2024-09-10T08:16:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.473+0000 7fc81722c740 1 -- 172.21.15.2:0/3926229787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f8047070 con 0x55faa1dc1590 2024-09-10T08:16:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.473+0000 7fc7f3400640 1 -- 172.21.15.2:0/3926229787 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc7f80048c0 con 0x55faa1dc1590 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.473+0000 7fc7f3400640 1 -- 172.21.15.2:0/3926229787 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc7f805b050 con 0x55faa1dc1590 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.473+0000 7fc81722c740 10 client.27812 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.473+0000 7fc81722c740 1 -- 172.21.15.2:0/3926229787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55faa1ddaa70 con 0x55faa1dc1590 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.474+0000 7fc7f3400640 1 -- 172.21.15.2:0/3926229787 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7f8047070 con 0x55faa1dc1590 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.474+0000 7fc7f3400640 1 -- 172.21.15.2:0/3926229787 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc7f803c570 con 0x55faa1dc1590 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.474+0000 7fc81722c740 10 client.27812 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.474+0000 7fc81722c740 10 client.27812 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.474+0000 7fc81722c740 20 client.27812 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.474+0000 7fc81722c740 1 --2- 172.21.15.2:0/3926229787 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55faa1ddbdc0 0x55faa1dde1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.474+0000 7fc81722c740 4 client.27812 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.474+0000 7fc81722c740 1 -- 172.21.15.2:0/3926229787 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faa1d34950 con 0x55faa1ddbdc0 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.475+0000 7fc811c00640 1 --2- 172.21.15.2:0/3926229787 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55faa1ddbdc0 0x55faa1dde1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.475+0000 7fc811c00640 1 --2- 172.21.15.2:0/3926229787 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55faa1ddbdc0 0x55faa1dde1b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee41c6d807c7c8f7 server_cookie=fb68f653e268044d in_seq=0 out_seq=0 2024-09-10T08:16:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.475+0000 7fc7f3400640 10 client.27812 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:16:47.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.476+0000 7fc7f3400640 1 -- 172.21.15.2:0/3926229787 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55faa1d34950 con 0x55faa1ddbdc0 2024-09-10T08:16:47.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.476+0000 7fc7f3400640 10 client.27812 handle_command_reply: tid=0 2024-09-10T08:16:47.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.476+0000 7fc81722c740 1 client.27812 shutdown 2024-09-10T08:16:47.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.476+0000 7fc81722c740 1 -- 172.21.15.2:0/3926229787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 msgr2=0x55faa1dc9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:47.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.476+0000 7fc81722c740 1 --2- 172.21.15.2:0/3926229787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1dc9260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8005360 tx=0x7fc7f80305d0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.476+0000 7fc81722c740 1 -- 172.21.15.2:0/3926229787 shutdown_connections 2024-09-10T08:16:47.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.476+0000 7fc81722c740 1 --2- 172.21.15.2:0/3926229787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1dc97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.476+0000 7fc81722c740 1 --2- 172.21.15.2:0/3926229787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1dc9260 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.476+0000 7fc81722c740 1 --2- 172.21.15.2:0/3926229787 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55faa1ddbdc0 0x55faa1dde1b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.477+0000 7fc81722c740 1 --2- 172.21.15.2:0/3926229787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.477+0000 7fc81722c740 1 -- 172.21.15.2:0/3926229787 >> 172.21.15.2:0/3926229787 conn(0x55faa1dbd200 msgr2=0x55faa1dc2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:47.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.477+0000 7fc7f3400640 0 client.27812 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:16:47.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.477+0000 7fc81722c740 1 -- 172.21.15.2:0/3926229787 shutdown_connections 2024-09-10T08:16:47.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.477+0000 7fc81722c740 1 -- 172.21.15.2:0/3926229787 wait complete. 2024-09-10T08:16:47.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.477+0000 7fc81722c740 20 client.27812 trim_cache size 0 max 16384 2024-09-10T08:16:47.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.490+0000 7fc81722c740 1 Processor -- start 2024-09-10T08:16:47.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.491+0000 7fc81722c740 1 -- start start 2024-09-10T08:16:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.491+0000 7fc81722c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.491+0000 7fc81722c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1de11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.491+0000 7fc81722c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1de1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.491+0000 7fc81722c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b3880 con 0x55faa1dc0c70 2024-09-10T08:16:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.491+0000 7fc81722c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b1150 con 0x55faa1dc1590 2024-09-10T08:16:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.491+0000 7fc81722c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1aefa0 con 0x55faa1dcbe50 2024-09-10T08:16:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.491+0000 7fc811c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.491+0000 7fc811c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50192/0 (socket says 172.21.15.2:50192) 2024-09-10T08:16:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.491+0000 7fc811c00640 1 -- 172.21.15.2:0/275278467 learned_addr learned my addr 172.21.15.2:0/275278467 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.491+0000 7fc811200640 1 --2- 172.21.15.2:0/275278467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1de11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.492+0000 7fc812600640 1 --2- 172.21.15.2:0/275278467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1de1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.492+0000 7fc811c00640 1 -- 172.21.15.2:0/275278467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 msgr2=0x55faa1de1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:47.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.492+0000 7fc811c00640 1 --2- 172.21.15.2:0/275278467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1de1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.492+0000 7fc811c00640 1 -- 172.21.15.2:0/275278467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 msgr2=0x55faa1de11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:47.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.492+0000 7fc811c00640 1 --2- 172.21.15.2:0/275278467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1de11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.492+0000 7fc811c00640 1 -- 172.21.15.2:0/275278467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa1dbd030 con 0x55faa1dc0c70 2024-09-10T08:16:47.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.492+0000 7fc811c00640 1 --2- 172.21.15.2:0/275278467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc36d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x55faa1dcecb0 tx=0x7fc80001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1821d6cc4ea3ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:47.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.492+0000 7fc7fe000640 1 -- 172.21.15.2:0/275278467 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8000036b0 con 0x55faa1dc0c70 2024-09-10T08:16:47.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.492+0000 7fc7fe000640 1 -- 172.21.15.2:0/275278467 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc800003d30 con 0x55faa1dc0c70 2024-09-10T08:16:47.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.492+0000 7fc7fe000640 1 -- 172.21.15.2:0/275278467 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80001f400 con 0x55faa1dc0c70 2024-09-10T08:16:47.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.493+0000 7fc81722c740 1 -- 172.21.15.2:0/275278467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 msgr2=0x55faa1dc36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:47.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.493+0000 7fc81722c740 1 --2- 172.21.15.2:0/275278467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc36d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x55faa1dcecb0 tx=0x7fc80001b000 comp rx=0 tx=0).stop 2024-09-10T08:16:47.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.493+0000 7fc81722c740 1 -- 172.21.15.2:0/275278467 shutdown_connections 2024-09-10T08:16:47.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.493+0000 7fc81722c740 1 --2- 172.21.15.2:0/275278467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1de1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.493+0000 7fc81722c740 1 --2- 172.21.15.2:0/275278467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1de11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.493+0000 7fc81722c740 1 --2- 172.21.15.2:0/275278467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1dc36d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.493+0000 7fc81722c740 1 -- 172.21.15.2:0/275278467 >> 172.21.15.2:0/275278467 conn(0x55faa1dbd200 msgr2=0x55faa1dcada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.493+0000 7fc81722c740 1 -- 172.21.15.2:0/275278467 shutdown_connections 2024-09-10T08:16:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.493+0000 7fc81722c740 1 -- 172.21.15.2:0/275278467 wait complete. 2024-09-10T08:16:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.494+0000 7fc81722c740 1 -- start start 2024-09-10T08:16:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.494+0000 7fc81722c740 1 Processor -- start 2024-09-10T08:16:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.494+0000 7fc81722c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1ddef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.494+0000 7fc81722c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1ddf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.494+0000 7fc81722c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1ddf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.494+0000 7fc81722c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1af610 con 0x55faa1dc0c70 2024-09-10T08:16:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.494+0000 7fc81722c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b0a80 con 0x55faa1dc1590 2024-09-10T08:16:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.494+0000 7fc81722c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1afce0 con 0x55faa1dcbe50 2024-09-10T08:16:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.494+0000 7fc81722c740 10 client.0 _ll_register_callbacks cb 0x7fc800009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.494+0000 7fc811c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1ddef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.495+0000 7fc811200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1ddf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.495+0000 7fc811c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1ddef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50202/0 (socket says 172.21.15.2:50202) 2024-09-10T08:16:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.495+0000 7fc811c00640 1 -- 172.21.15.2:0/3972586522 learned_addr learned my addr 172.21.15.2:0/3972586522 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.495+0000 7fc812600640 1 --2- 172.21.15.2:0/3972586522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1ddf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.495+0000 7fc811200640 1 -- 172.21.15.2:0/3972586522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 msgr2=0x55faa1ddf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.495+0000 7fc811200640 1 --2- 172.21.15.2:0/3972586522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1ddf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.495+0000 7fc811200640 1 -- 172.21.15.2:0/3972586522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 msgr2=0x55faa1ddef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.495+0000 7fc811200640 1 --2- 172.21.15.2:0/3972586522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1ddef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.495+0000 7fc811200640 1 -- 172.21.15.2:0/3972586522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa1dbde90 con 0x55faa1dc1590 2024-09-10T08:16:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.495+0000 7fc811c00640 1 --2- 172.21.15.2:0/3972586522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1ddef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.495+0000 7fc811200640 1 --2- 172.21.15.2:0/3972586522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1ddf4a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc7f802a8d0 tx=0x7fc7f8030da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3bcad236125aa42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.495+0000 7fc7f3400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc7f3400640 1 -- 172.21.15.2:0/3972586522 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8050040 con 0x55faa1dc1590 2024-09-10T08:16:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc81722c740 1 -- 172.21.15.2:0/3972586522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc800003d30 con 0x55faa1dc1590 2024-09-10T08:16:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc7f3400640 1 -- 172.21.15.2:0/3972586522 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc7f803c450 con 0x55faa1dc1590 2024-09-10T08:16:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc7f3400640 1 -- 172.21.15.2:0/3972586522 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc7f8005c90 con 0x55faa1dc1590 2024-09-10T08:16:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc81722c740 10 client.27818 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc81722c740 1 -- 172.21.15.2:0/3972586522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55faa1e46810 con 0x55faa1dc1590 2024-09-10T08:16:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc7f3400640 1 -- 172.21.15.2:0/3972586522 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc800003d30 con 0x55faa1dc1590 2024-09-10T08:16:47.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc7f3400640 1 -- 172.21.15.2:0/3972586522 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc7f8004c30 con 0x55faa1dc1590 2024-09-10T08:16:47.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc81722c740 10 client.27818 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:47.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc81722c740 10 client.27818 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:16:47.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc81722c740 20 client.27818 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:47.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc81722c740 1 --2- 172.21.15.2:0/3972586522 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55faa1e46d90 0x55faa1e491b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:47.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc81722c740 4 client.27818 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:16:47.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.496+0000 7fc81722c740 1 -- 172.21.15.2:0/3972586522 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55faa1d2ee60 con 0x55faa1e46d90 2024-09-10T08:16:47.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.497+0000 7fc811c00640 1 --2- 172.21.15.2:0/3972586522 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55faa1e46d90 0x55faa1e491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:47.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc811c00640 1 --2- 172.21.15.2:0/3972586522 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55faa1e46d90 0x55faa1e491b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b07e1c170cd6a5e0 server_cookie=896ef02cfefacc86 in_seq=0 out_seq=0 2024-09-10T08:16:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc7f3400640 10 client.27818 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:16:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc7f3400640 1 -- 172.21.15.2:0/3972586522 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55faa1d2ee60 con 0x55faa1e46d90 2024-09-10T08:16:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc7f3400640 10 client.27818 handle_command_reply: tid=0 2024-09-10T08:16:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc81722c740 1 client.27818 shutdown 2024-09-10T08:16:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc81722c740 1 -- 172.21.15.2:0/3972586522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 msgr2=0x55faa1ddf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc81722c740 1 --2- 172.21.15.2:0/3972586522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1ddf4a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc7f802a8d0 tx=0x7fc7f8030da0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc81722c740 1 -- 172.21.15.2:0/3972586522 shutdown_connections 2024-09-10T08:16:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc81722c740 1 --2- 172.21.15.2:0/3972586522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faa1dcbe50 0x55faa1ddf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc81722c740 1 --2- 172.21.15.2:0/3972586522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faa1dc1590 0x55faa1ddf4a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc81722c740 1 --2- 172.21.15.2:0/3972586522 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55faa1e46d90 0x55faa1e491b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc81722c740 1 --2- 172.21.15.2:0/3972586522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faa1dc0c70 0x55faa1ddef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc81722c740 1 -- 172.21.15.2:0/3972586522 >> 172.21.15.2:0/3972586522 conn(0x55faa1dbd200 msgr2=0x55faa1dcada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:47.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc7f3400640 0 client.27818 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:16:47.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc81722c740 1 -- 172.21.15.2:0/3972586522 shutdown_connections 2024-09-10T08:16:47.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc81722c740 1 -- 172.21.15.2:0/3972586522 wait complete. 2024-09-10T08:16:47.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.498+0000 7fc81722c740 20 client.27818 trim_cache size 0 max 16384 2024-09-10T08:16:47.502 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:16:47.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.499+0000 7fc814600640 1 -- 172.21.15.2:0/2607324499 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7f40a2b80 msgr2=0x7fc7f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:47.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.499+0000 7fc814600640 1 --2- 172.21.15.2:0/2607324499 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7f40a2b80 0x7fc7f40a5040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc800009620 tx=0x7fc800005f90 comp rx=0 tx=0).stop 2024-09-10T08:16:47.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.499+0000 7fc814600640 1 -- 172.21.15.2:0/2607324499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc80c10c180 msgr2=0x7fc80c19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:47.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.499+0000 7fc814600640 1 --2- 172.21.15.2:0/2607324499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc80c10c180 0x7fc80c19ef60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc80400ef00 tx=0x7fc80400c740 comp rx=0 tx=0).stop 2024-09-10T08:16:47.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.499+0000 7fc814600640 1 -- 172.21.15.2:0/2607324499 shutdown_connections 2024-09-10T08:16:47.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.499+0000 7fc814600640 1 --2- 172.21.15.2:0/2607324499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc80c10c180 0x7fc80c19ef60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.499+0000 7fc814600640 1 --2- 172.21.15.2:0/2607324499 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7f40a2b80 0x7fc7f40a5040 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.499+0000 7fc814600640 1 --2- 172.21.15.2:0/2607324499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c103670 0x7fc80c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.499+0000 7fc814600640 1 --2- 172.21.15.2:0/2607324499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc80c102d50 0x7fc80c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:47.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.499+0000 7fc814600640 1 -- 172.21.15.2:0/2607324499 >> 172.21.15.2:0/2607324499 conn(0x7fc80c0fc7d0 msgr2=0x7fc80c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:47.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.499+0000 7fc814600640 1 -- 172.21.15.2:0/2607324499 shutdown_connections 2024-09-10T08:16:47.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:47.499+0000 7fc814600640 1 -- 172.21.15.2:0/2607324499 wait complete. 2024-09-10T08:16:47.647 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:16:47.697 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:47 smithi002 ceph-mon[30563]: pgmap v646: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:16:47.818 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:47 smithi107 ceph-mon[35048]: pgmap v646: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:16:47.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:47 smithi155 ceph-mon[35405]: pgmap v646: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:16:48.335 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:16:48.376 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.011+0000 7f463fe00640 1 -- 172.21.15.2:0/795432204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4638103ff0 msgr2=0x7f46381043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.011+0000 7f463fe00640 1 --2- 172.21.15.2:0/795432204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4638103ff0 0x7f46381043d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f462c009900 tx=0x7f462c02a410 comp rx=0 tx=0).stop 2024-09-10T08:16:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.011+0000 7f463fe00640 1 -- 172.21.15.2:0/795432204 shutdown_connections 2024-09-10T08:16:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.011+0000 7f463fe00640 1 --2- 172.21.15.2:0/795432204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4638109120 0x7f4638110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.011+0000 7f463fe00640 1 --2- 172.21.15.2:0/795432204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46381049a0 0x7f46381089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.011+0000 7f463fe00640 1 --2- 172.21.15.2:0/795432204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4638103ff0 0x7f46381043d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.011+0000 7f463fe00640 1 -- 172.21.15.2:0/795432204 >> 172.21.15.2:0/795432204 conn(0x7f46380ffa90 msgr2=0x7f4638101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.011+0000 7f463fe00640 1 -- 172.21.15.2:0/795432204 shutdown_connections 2024-09-10T08:16:49.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.011+0000 7f463fe00640 1 -- 172.21.15.2:0/795432204 wait complete. 2024-09-10T08:16:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.012+0000 7f463fe00640 1 Processor -- start 2024-09-10T08:16:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.013+0000 7f463fe00640 1 -- start start 2024-09-10T08:16:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.013+0000 7f463fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4638103ff0 0x7f463819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.013+0000 7f463fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46381049a0 0x7f463819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.013+0000 7f463fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4638109120 0x7f463819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.013+0000 7f463fe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4638114150 con 0x7f46381049a0 2024-09-10T08:16:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.013+0000 7f463fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4638113fd0 con 0x7f4638109120 2024-09-10T08:16:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.013+0000 7f463fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f46381142d0 con 0x7f4638103ff0 2024-09-10T08:16:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.013+0000 7f463ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46381049a0 0x7f463819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.013+0000 7f463ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46381049a0 0x7f463819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50220/0 (socket says 172.21.15.2:50220) 2024-09-10T08:16:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.013+0000 7f463ca00640 1 -- 172.21.15.2:0/2644995866 learned_addr learned my addr 172.21.15.2:0/2644995866 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.013+0000 7f463d400640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4638103ff0 0x7f463819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:49.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.014+0000 7f463de00640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4638109120 0x7f463819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.014+0000 7f463ca00640 1 -- 172.21.15.2:0/2644995866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4638103ff0 msgr2=0x7f463819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.014+0000 7f463ca00640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4638103ff0 0x7f463819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.014+0000 7f463ca00640 1 -- 172.21.15.2:0/2644995866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4638109120 msgr2=0x7f463819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.014+0000 7f463ca00640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4638109120 0x7f463819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.014+0000 7f463ca00640 1 -- 172.21.15.2:0/2644995866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463819f5a0 con 0x7f46381049a0 2024-09-10T08:16:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.014+0000 7f463ca00640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46381049a0 0x7f463819ab30 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f462400ea60 tx=0x7f462400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.014+0000 7f4636000640 1 -- 172.21.15.2:0/2644995866 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462400cde0 con 0x7f46381049a0 2024-09-10T08:16:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.015+0000 7f463fe00640 1 -- 172.21.15.2:0/2644995866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463819f890 con 0x7f46381049a0 2024-09-10T08:16:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.015+0000 7f4636000640 1 -- 172.21.15.2:0/2644995866 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46240035f0 con 0x7f46381049a0 2024-09-10T08:16:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.015+0000 7f4636000640 1 -- 172.21.15.2:0/2644995866 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624010670 con 0x7f46381049a0 2024-09-10T08:16:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.015+0000 7f463fe00640 1 -- 172.21.15.2:0/2644995866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46381a7170 con 0x7f46381049a0 2024-09-10T08:16:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.016+0000 7f463fe00640 1 -- 172.21.15.2:0/2644995866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46381043d0 con 0x7f46381049a0 2024-09-10T08:16:49.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.018+0000 7f4636000640 1 -- 172.21.15.2:0/2644995866 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f46240037a0 con 0x7f46381049a0 2024-09-10T08:16:49.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.019+0000 7f4636000640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46200a2b80 0x7f46200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:49.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.019+0000 7f4636000640 1 -- 172.21.15.2:0/2644995866 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f46240cf740 con 0x7f46381049a0 2024-09-10T08:16:49.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.020+0000 7f463d400640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46200a2b80 0x7f46200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:49.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.020+0000 7f463d400640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46200a2b80 0x7f46200a5040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f462c009230 tx=0x7f462c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:49.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.024+0000 7f4636000640 1 -- 172.21.15.2:0/2644995866 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f462409c250 con 0x7f46381049a0 2024-09-10T08:16:49.065 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:49.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.253+0000 7f463fe00640 1 -- 172.21.15.2:0/2644995866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f463819b950 con 0x7f46381049a0 2024-09-10T08:16:49.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.254+0000 7f4636000640 1 -- 172.21.15.2:0/2644995866 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f462409bb50 con 0x7f46381049a0 2024-09-10T08:16:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:16:49.255 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:16:49.255 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:16:49.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.257+0000 7f463fe00640 1 -- 172.21.15.2:0/2644995866 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46200a2b80 msgr2=0x7f46200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:49.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.257+0000 7f463fe00640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46200a2b80 0x7f46200a5040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f462c009230 tx=0x7f462c0058d0 comp rx=0 tx=0).stop 2024-09-10T08:16:49.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.257+0000 7f463fe00640 1 -- 172.21.15.2:0/2644995866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46381049a0 msgr2=0x7f463819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:49.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.257+0000 7f463fe00640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46381049a0 0x7f463819ab30 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f462400ea60 tx=0x7f462400ef20 comp rx=0 tx=0).stop 2024-09-10T08:16:49.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.257+0000 7f463fe00640 1 -- 172.21.15.2:0/2644995866 shutdown_connections 2024-09-10T08:16:49.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.257+0000 7f463fe00640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4638109120 0x7f463819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:49.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.257+0000 7f463fe00640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46381049a0 0x7f463819ab30 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:49.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.257+0000 7f463fe00640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46200a2b80 0x7f46200a5040 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:49.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.257+0000 7f463fe00640 1 --2- 172.21.15.2:0/2644995866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4638103ff0 0x7f463819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:49.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.257+0000 7f463fe00640 1 -- 172.21.15.2:0/2644995866 >> 172.21.15.2:0/2644995866 conn(0x7f46380ffa90 msgr2=0x7f4638100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:49.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.257+0000 7f463fe00640 1 -- 172.21.15.2:0/2644995866 shutdown_connections 2024-09-10T08:16:49.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:49.257+0000 7f463fe00640 1 -- 172.21.15.2:0/2644995866 wait complete. 2024-09-10T08:16:49.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:49 smithi155 ceph-mon[35405]: pgmap v647: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 KiB/s wr, 13 op/s 2024-09-10T08:16:49.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:49 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2644995866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:16:49.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:49 smithi107 ceph-mon[35048]: pgmap v647: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 KiB/s wr, 13 op/s 2024-09-10T08:16:49.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:49 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2644995866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:16:50.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:49 smithi002 ceph-mon[30563]: pgmap v647: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 KiB/s wr, 13 op/s 2024-09-10T08:16:50.040 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:49 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2644995866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:16:50.118 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id 32b5535c --release --await' 2024-09-10T08:16:50.118 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id 32b5535c --release --await 2024-09-10T08:16:50.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.316+0000 7f8aad600640 1 -- 172.21.15.2:0/1902017224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aa8074510 msgr2=0x7f8aa80748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.316+0000 7f8aad600640 1 --2- 172.21.15.2:0/1902017224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aa8074510 0x7f8aa80748f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009900 tx=0x7f8a9802a480 comp rx=0 tx=0).stop 2024-09-10T08:16:50.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.316+0000 7f8aad600640 1 -- 172.21.15.2:0/1902017224 shutdown_connections 2024-09-10T08:16:50.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.316+0000 7f8aad600640 1 --2- 172.21.15.2:0/1902017224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aa810e440 0x7f8aa8110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.316+0000 7f8aad600640 1 --2- 172.21.15.2:0/1902017224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa8074e30 0x7f8aa810df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.316+0000 7f8aad600640 1 --2- 172.21.15.2:0/1902017224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aa8074510 0x7f8aa80748f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.316+0000 7f8aad600640 1 -- 172.21.15.2:0/1902017224 >> 172.21.15.2:0/1902017224 conn(0x7f8aa80fd8c0 msgr2=0x7f8aa80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:50.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.316+0000 7f8aad600640 1 -- 172.21.15.2:0/1902017224 shutdown_connections 2024-09-10T08:16:50.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.316+0000 7f8aad600640 1 -- 172.21.15.2:0/1902017224 wait complete. 2024-09-10T08:16:50.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.318+0000 7f8aad600640 1 Processor -- start 2024-09-10T08:16:50.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.318+0000 7f8aad600640 1 -- start start 2024-09-10T08:16:50.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.318+0000 7f8aad600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aa8074510 0x7f8aa819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aad600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aa8074e30 0x7f8aa819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aad600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa810e440 0x7f8aa819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aad600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa8114150 con 0x7f8aa8074e30 2024-09-10T08:16:50.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aad600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa8113fd0 con 0x7f8aa810e440 2024-09-10T08:16:50.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aad600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81142d0 con 0x7f8aa8074510 2024-09-10T08:16:50.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aa8074e30 0x7f8aa819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aa7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa810e440 0x7f8aa819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aa7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa810e440 0x7f8aa819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:43970/0 (socket says 172.21.15.2:43970) 2024-09-10T08:16:50.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aa8074e30 0x7f8aa819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50236/0 (socket says 172.21.15.2:50236) 2024-09-10T08:16:50.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aa7400640 1 -- 172.21.15.2:0/3876142412 learned_addr learned my addr 172.21.15.2:0/3876142412 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:50.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aa6a00640 1 --2- 172.21.15.2:0/3876142412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aa8074510 0x7f8aa819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aa7400640 1 -- 172.21.15.2:0/3876142412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aa8074510 msgr2=0x7f8aa819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aa7400640 1 --2- 172.21.15.2:0/3876142412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aa8074510 0x7f8aa819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aa7400640 1 -- 172.21.15.2:0/3876142412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aa8074e30 msgr2=0x7f8aa819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aa7400640 1 --2- 172.21.15.2:0/3876142412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aa8074e30 0x7f8aa819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.319+0000 7f8aa7400640 1 -- 172.21.15.2:0/3876142412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa819f6c0 con 0x7f8aa810e440 2024-09-10T08:16:50.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.320+0000 7f8aa7400640 1 --2- 172.21.15.2:0/3876142412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa810e440 0x7f8aa819efe0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000eb40 tx=0x7f8aa000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:50.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.320+0000 7f8a9f400640 1 -- 172.21.15.2:0/3876142412 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0004b20 con 0x7f8aa810e440 2024-09-10T08:16:50.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.320+0000 7f8a9f400640 1 -- 172.21.15.2:0/3876142412 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa0004cc0 con 0x7f8aa810e440 2024-09-10T08:16:50.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.320+0000 7f8aad600640 1 -- 172.21.15.2:0/3876142412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa819f9b0 con 0x7f8aa810e440 2024-09-10T08:16:50.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.320+0000 7f8a9f400640 1 -- 172.21.15.2:0/3876142412 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0010850 con 0x7f8aa810e440 2024-09-10T08:16:50.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.320+0000 7f8aad600640 1 -- 172.21.15.2:0/3876142412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa81a7290 con 0x7f8aa810e440 2024-09-10T08:16:50.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.323+0000 7f8ab01ee740 1 Processor -- start 2024-09-10T08:16:50.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.323+0000 7f8a9f400640 1 -- 172.21.15.2:0/3876142412 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8aa0005020 con 0x7f8aa810e440 2024-09-10T08:16:50.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.324+0000 7f8ab01ee740 1 -- start start 2024-09-10T08:16:50.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.324+0000 7f8ab01ee740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 0x563822b2d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.324+0000 7f8ab01ee740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b37910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.324+0000 7f8ab01ee740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 0x563822b3a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.324+0000 7f8ab01ee740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563822a933e0 con 0x563822b2cc70 2024-09-10T08:16:50.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.324+0000 7f8ab01ee740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563822a3ad80 con 0x563822b2d590 2024-09-10T08:16:50.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.324+0000 7f8ab01ee740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563822a8f3a0 con 0x563822b37e50 2024-09-10T08:16:50.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.324+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 0x563822b2d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.324+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 0x563822b2d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50238/0 (socket says 172.21.15.2:50238) 2024-09-10T08:16:50.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.324+0000 7f8aa6a00640 1 --2- 172.21.15.2:0/2200973336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b37910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.324+0000 7f8a9fe00640 1 -- 172.21.15.2:0/2200973336 learned_addr learned my addr 172.21.15.2:0/2200973336 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:50.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.325+0000 7f8a9fe00640 1 --2- 172.21.15.2:0/2200973336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 0x563822b3a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.325+0000 7f8aa6a00640 1 -- 172.21.15.2:0/2200973336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 msgr2=0x563822b3a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.325+0000 7f8aa6a00640 1 --2- 172.21.15.2:0/2200973336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 0x563822b3a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.325+0000 7f8aa6a00640 1 -- 172.21.15.2:0/2200973336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 msgr2=0x563822b2d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.325+0000 7f8aa6a00640 1 --2- 172.21.15.2:0/2200973336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 0x563822b2d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.325+0000 7f8aa6a00640 1 -- 172.21.15.2:0/2200973336 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563822a59ae0 con 0x563822b2d590 2024-09-10T08:16:50.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.325+0000 7f8a9fe00640 1 --2- 172.21.15.2:0/2200973336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 0x563822b2d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:50.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.325+0000 7f8aa6a00640 1 --2- 172.21.15.2:0/2200973336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b37910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8a98005940 tx=0x7f8a98005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd43d4b28b763623 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:50.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.325+0000 7f8a9f400640 1 --2- 172.21.15.2:0/3876142412 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8a8c0a2b80 0x7f8a8c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.326+0000 7f8a9f400640 1 -- 172.21.15.2:0/3876142412 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8aa00cfc20 con 0x7f8aa810e440 2024-09-10T08:16:50.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.326+0000 7f8a97400640 1 -- 172.21.15.2:0/2200973336 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98040070 con 0x563822b2d590 2024-09-10T08:16:50.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.326+0000 7f8a97400640 1 -- 172.21.15.2:0/2200973336 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a98031870 con 0x563822b2d590 2024-09-10T08:16:50.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.326+0000 7f8a97400640 1 -- 172.21.15.2:0/2200973336 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98035040 con 0x563822b2d590 2024-09-10T08:16:50.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.326+0000 7f8a9fe00640 1 --2- 172.21.15.2:0/3876142412 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8a8c0a2b80 0x7f8a8c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.327+0000 7f8ab01ee740 1 -- 172.21.15.2:0/2200973336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 msgr2=0x563822b37910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.327+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/2200973336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b37910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8a98005940 tx=0x7f8a98005be0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.327+0000 7f8a9fe00640 1 --2- 172.21.15.2:0/3876142412 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8a8c0a2b80 0x7f8a8c0a5040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x563822b2a580 tx=0x7f8a90009950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:50.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.327+0000 7f8ab01ee740 1 -- 172.21.15.2:0/2200973336 shutdown_connections 2024-09-10T08:16:50.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.327+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/2200973336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 0x563822b3a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.327+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/2200973336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b37910 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.327+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/2200973336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 0x563822b2d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.327+0000 7f8ab01ee740 1 -- 172.21.15.2:0/2200973336 >> 172.21.15.2:0/2200973336 conn(0x563822b29200 msgr2=0x563822b295e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:50.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.327+0000 7f8ab01ee740 1 -- 172.21.15.2:0/2200973336 shutdown_connections 2024-09-10T08:16:50.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.327+0000 7f8ab01ee740 1 -- 172.21.15.2:0/2200973336 wait complete. 2024-09-10T08:16:50.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.327+0000 7f8ab01ee740 1 -- start start 2024-09-10T08:16:50.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.328+0000 7f8ab01ee740 1 Processor -- start 2024-09-10T08:16:50.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.328+0000 7f8ab01ee740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 0x563822b34d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.328+0000 7f8ab01ee740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b352b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.328+0000 7f8ab01ee740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 0x563822b357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.328+0000 7f8ab01ee740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563822b3aa20 con 0x563822b2cc70 2024-09-10T08:16:50.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.328+0000 7f8ab01ee740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563822a4e0f0 con 0x563822b2d590 2024-09-10T08:16:50.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.328+0000 7f8ab01ee740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563822b3aba0 con 0x563822b37e50 2024-09-10T08:16:50.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.328+0000 7f8ab01ee740 10 client.0 _ll_register_callbacks cb 0x563822a35bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:50.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.329+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.329+0000 7f8aa6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 0x563822b34d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.329+0000 7f8aa7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 0x563822b357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.329+0000 7f8aa6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 0x563822b34d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50254/0 (socket says 172.21.15.2:50254) 2024-09-10T08:16:50.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.329+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b352b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:43998/0 (socket says 172.21.15.2:43998) 2024-09-10T08:16:50.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.329+0000 7f8aa6a00640 1 -- 172.21.15.2:0/337853286 learned_addr learned my addr 172.21.15.2:0/337853286 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:50.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.329+0000 7f8aa6a00640 1 -- 172.21.15.2:0/337853286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 msgr2=0x563822b357f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.329+0000 7f8aa6a00640 1 --2- 172.21.15.2:0/337853286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 0x563822b357f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.329+0000 7f8aa6a00640 1 -- 172.21.15.2:0/337853286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 msgr2=0x563822b352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.329+0000 7f8aa6a00640 1 --2- 172.21.15.2:0/337853286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.329+0000 7f8aa6a00640 1 -- 172.21.15.2:0/337853286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638228043c0 con 0x563822b2cc70 2024-09-10T08:16:50.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.329+0000 7f8aa6a00640 1 --2- 172.21.15.2:0/337853286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 0x563822b34d70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8a9802a4c0 tx=0x7f8a98005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4445dfdb7dc15494 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:50.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.329+0000 7f8a8be00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:16:50.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.330+0000 7f8a8be00640 1 -- 172.21.15.2:0/337853286 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98052060 con 0x563822b2cc70 2024-09-10T08:16:50.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.330+0000 7f8a8be00640 1 -- 172.21.15.2:0/337853286 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8a98031670 con 0x563822b2cc70 2024-09-10T08:16:50.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.330+0000 7f8ab01ee740 1 -- 172.21.15.2:0/337853286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a98031870 con 0x563822b2cc70 2024-09-10T08:16:50.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.330+0000 7f8a8be00640 1 -- 172.21.15.2:0/337853286 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a98059050 con 0x563822b2cc70 2024-09-10T08:16:50.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.330+0000 7f8ab01ee740 10 client.17808 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:50.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.330+0000 7f8ab01ee740 1 -- 172.21.15.2:0/337853286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563822b46a70 con 0x563822b2cc70 2024-09-10T08:16:50.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.331+0000 7f8a8be00640 1 -- 172.21.15.2:0/337853286 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a98031870 con 0x563822b2cc70 2024-09-10T08:16:50.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.331+0000 7f8a8be00640 1 -- 172.21.15.2:0/337853286 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8a98004240 con 0x563822b2cc70 2024-09-10T08:16:50.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.331+0000 7f8ab01ee740 10 client.17808 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:50.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.331+0000 7f8ab01ee740 10 client.17808 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:16:50.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.331+0000 7f8ab01ee740 20 client.17808 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:50.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.331+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/337853286 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563822b47dc0 0x563822b4a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.331+0000 7f8ab01ee740 4 client.17808 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:50.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.331+0000 7f8ab01ee740 1 -- 172.21.15.2:0/337853286 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563822aa0950 con 0x563822b47dc0 2024-09-10T08:16:50.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.332+0000 7f8a9fe00640 1 --2- 172.21.15.2:0/337853286 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563822b47dc0 0x563822b4a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.332+0000 7f8a9fe00640 1 --2- 172.21.15.2:0/337853286 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563822b47dc0 0x563822b4a1b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8bc09d6076383a4a server_cookie=ebb1cf829e99871f in_seq=0 out_seq=0 2024-09-10T08:16:50.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.332+0000 7f8a8be00640 10 client.17808 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:16:50.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.333+0000 7f8a8be00640 1 -- 172.21.15.2:0/337853286 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563822aa0950 con 0x563822b47dc0 2024-09-10T08:16:50.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.333+0000 7f8a8be00640 10 client.17808 handle_command_reply: tid=0 2024-09-10T08:16:50.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.333+0000 7f8ab01ee740 1 client.17808 shutdown 2024-09-10T08:16:50.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.333+0000 7f8ab01ee740 1 -- 172.21.15.2:0/337853286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 msgr2=0x563822b34d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.333+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/337853286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 0x563822b34d70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8a9802a4c0 tx=0x7f8a98005250 comp rx=0 tx=0).stop 2024-09-10T08:16:50.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.333+0000 7f8ab01ee740 1 -- 172.21.15.2:0/337853286 shutdown_connections 2024-09-10T08:16:50.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.333+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/337853286 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563822b47dc0 0x563822b4a1b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.333+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/337853286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 0x563822b357f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.333+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/337853286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b352b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.333+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/337853286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2cc70 0x563822b34d70 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.333+0000 7f8a8be00640 0 client.17808 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:16:50.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.333+0000 7f8ab01ee740 1 -- 172.21.15.2:0/337853286 >> 172.21.15.2:0/337853286 conn(0x563822b29200 msgr2=0x563822b28380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.334+0000 7f8ab01ee740 1 -- 172.21.15.2:0/337853286 shutdown_connections 2024-09-10T08:16:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.334+0000 7f8ab01ee740 1 -- 172.21.15.2:0/337853286 wait complete. 2024-09-10T08:16:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.334+0000 7f8ab01ee740 20 client.17808 trim_cache size 0 max 16384 2024-09-10T08:16:50.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.348+0000 7f8ab01ee740 1 Processor -- start 2024-09-10T08:16:50.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.348+0000 7f8ab01ee740 1 -- start start 2024-09-10T08:16:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.348+0000 7f8ab01ee740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2cc70 0x563822b4f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.348+0000 7f8ab01ee740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2d590 0x563822b4fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.348+0000 7f8ab01ee740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 0x563822b31ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8ab01ee740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81b38f0 con 0x563822b2d590 2024-09-10T08:16:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8ab01ee740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81b11c0 con 0x563822b2cc70 2024-09-10T08:16:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8ab01ee740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81af010 con 0x563822b37e50 2024-09-10T08:16:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2d590 0x563822b4fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8aa7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 0x563822b31ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2d590 0x563822b4fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50270/0 (socket says 172.21.15.2:50270) 2024-09-10T08:16:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8a9fe00640 1 -- 172.21.15.2:0/3149558551 learned_addr learned my addr 172.21.15.2:0/3149558551 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8aa7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 0x563822b31ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35374/0 (socket says 172.21.15.2:35374) 2024-09-10T08:16:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8aa6a00640 1 --2- 172.21.15.2:0/3149558551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2cc70 0x563822b4f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8a9fe00640 1 -- 172.21.15.2:0/3149558551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 msgr2=0x563822b31ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8a9fe00640 1 --2- 172.21.15.2:0/3149558551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 0x563822b31ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8a9fe00640 1 -- 172.21.15.2:0/3149558551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2cc70 msgr2=0x563822b4f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8a9fe00640 1 --2- 172.21.15.2:0/3149558551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2cc70 0x563822b4f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.349+0000 7f8a9fe00640 1 -- 172.21.15.2:0/3149558551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563822b2c9f0 con 0x563822b2d590 2024-09-10T08:16:50.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.350+0000 7f8a9fe00640 1 --2- 172.21.15.2:0/3149558551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2d590 0x563822b4fcc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8a90005b90 tx=0x7f8a90015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b373bc6e98e3c8b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:50.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.350+0000 7f8a96a00640 1 -- 172.21.15.2:0/3149558551 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9001a030 con 0x563822b2d590 2024-09-10T08:16:50.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.350+0000 7f8a96a00640 1 -- 172.21.15.2:0/3149558551 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a90005020 con 0x563822b2d590 2024-09-10T08:16:50.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.350+0000 7f8a96a00640 1 -- 172.21.15.2:0/3149558551 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90004bc0 con 0x563822b2d590 2024-09-10T08:16:50.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.351+0000 7f8ab01ee740 1 -- 172.21.15.2:0/3149558551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2d590 msgr2=0x563822b4fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.351+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/3149558551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2d590 0x563822b4fcc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8a90005b90 tx=0x7f8a90015000 comp rx=0 tx=0).stop 2024-09-10T08:16:50.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.351+0000 7f8ab01ee740 1 -- 172.21.15.2:0/3149558551 shutdown_connections 2024-09-10T08:16:50.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.351+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/3149558551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b37e50 0x563822b31ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.351+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/3149558551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b2d590 0x563822b4fcc0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.351+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/3149558551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2cc70 0x563822b4f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.351+0000 7f8ab01ee740 1 -- 172.21.15.2:0/3149558551 >> 172.21.15.2:0/3149558551 conn(0x563822b29200 msgr2=0x563822b36df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:50.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.351+0000 7f8ab01ee740 1 -- 172.21.15.2:0/3149558551 shutdown_connections 2024-09-10T08:16:50.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.351+0000 7f8ab01ee740 1 -- 172.21.15.2:0/3149558551 wait complete. 2024-09-10T08:16:50.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.352+0000 7f8ab01ee740 1 -- start start 2024-09-10T08:16:50.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.352+0000 7f8ab01ee740 1 Processor -- start 2024-09-10T08:16:50.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.352+0000 7f8ab01ee740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b2cc70 0x563822b40410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.352+0000 7f8ab01ee740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b31a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.352+0000 7f8ab01ee740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b37e50 0x563822b31f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.352+0000 7f8ab01ee740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81af680 con 0x563822b37e50 2024-09-10T08:16:50.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.352+0000 7f8ab01ee740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81b0af0 con 0x563822b2d590 2024-09-10T08:16:50.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.352+0000 7f8ab01ee740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81afd50 con 0x563822b2cc70 2024-09-10T08:16:50.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.352+0000 7f8ab01ee740 10 client.0 _ll_register_callbacks cb 0x7f8a90009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.353+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b31a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.353+0000 7f8aa6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b2cc70 0x563822b40410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.353+0000 7f8aa7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b37e50 0x563822b31f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.353+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b31a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44008/0 (socket says 172.21.15.2:44008) 2024-09-10T08:16:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.353+0000 7f8aa6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b2cc70 0x563822b40410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35386/0 (socket says 172.21.15.2:35386) 2024-09-10T08:16:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.353+0000 7f8a9fe00640 1 -- 172.21.15.2:0/4182301107 learned_addr learned my addr 172.21.15.2:0/4182301107 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.353+0000 7f8a9fe00640 1 -- 172.21.15.2:0/4182301107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b2cc70 msgr2=0x563822b40410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.353+0000 7f8a9fe00640 1 --2- 172.21.15.2:0/4182301107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b2cc70 0x563822b40410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.353+0000 7f8a9fe00640 1 -- 172.21.15.2:0/4182301107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b37e50 msgr2=0x563822b31f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.353+0000 7f8a9fe00640 1 --2- 172.21.15.2:0/4182301107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b37e50 0x563822b31f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.353+0000 7f8a9fe00640 1 -- 172.21.15.2:0/4182301107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563822b32680 con 0x563822b2d590 2024-09-10T08:16:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.353+0000 7f8aa6a00640 1 --2- 172.21.15.2:0/4182301107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b2cc70 0x563822b40410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.354+0000 7f8a9fe00640 1 --2- 172.21.15.2:0/4182301107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b31a00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8a90010f00 tx=0x7f8a90009d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21b45dc90a219cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:50.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.354+0000 7f8a8be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:50.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.354+0000 7f8a8be00640 1 -- 172.21.15.2:0/4182301107 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9003c070 con 0x563822b2d590 2024-09-10T08:16:50.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.354+0000 7f8a8be00640 1 -- 172.21.15.2:0/4182301107 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8a900053e0 con 0x563822b2d590 2024-09-10T08:16:50.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.354+0000 7f8ab01ee740 1 -- 172.21.15.2:0/4182301107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a90005020 con 0x563822b2d590 2024-09-10T08:16:50.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.354+0000 7f8a8be00640 1 -- 172.21.15.2:0/4182301107 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a9000fab0 con 0x563822b2d590 2024-09-10T08:16:50.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.354+0000 7f8ab01ee740 10 client.27842 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:50.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.354+0000 7f8ab01ee740 1 -- 172.21.15.2:0/4182301107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563822bb26a0 con 0x563822b2d590 2024-09-10T08:16:50.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.355+0000 7f8a8be00640 1 -- 172.21.15.2:0/4182301107 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a90005020 con 0x563822b2d590 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.355+0000 7f8a8be00640 1 -- 172.21.15.2:0/4182301107 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8a9002d530 con 0x563822b2d590 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.355+0000 7f8ab01ee740 10 client.27842 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.355+0000 7f8ab01ee740 10 client.27842 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.355+0000 7f8ab01ee740 20 client.27842 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.355+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/4182301107 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563822bb2dc0 0x563822bb51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.355+0000 7f8ab01ee740 4 client.27842 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.355+0000 7f8ab01ee740 1 -- 172.21.15.2:0/4182301107 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563822a9ae60 con 0x563822bb2dc0 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.356+0000 7f8aa6a00640 1 --2- 172.21.15.2:0/4182301107 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563822bb2dc0 0x563822bb51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.356+0000 7f8aa6a00640 1 --2- 172.21.15.2:0/4182301107 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563822bb2dc0 0x563822bb51e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90469bd5d62d42e server_cookie=d77dc6c3a714aaf in_seq=0 out_seq=0 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.356+0000 7f8a8be00640 10 client.27842 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.356+0000 7f8a8be00640 1 -- 172.21.15.2:0/4182301107 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563822a9ae60 con 0x563822bb2dc0 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.356+0000 7f8a8be00640 10 client.27842 handle_command_reply: tid=0 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.356+0000 7f8ab01ee740 1 client.27842 shutdown 2024-09-10T08:16:50.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.356+0000 7f8ab01ee740 1 -- 172.21.15.2:0/4182301107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 msgr2=0x563822b31a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.356+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/4182301107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b31a00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8a90010f00 tx=0x7f8a90009d70 comp rx=0 tx=0).stop 2024-09-10T08:16:50.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.357+0000 7f8ab01ee740 1 -- 172.21.15.2:0/4182301107 shutdown_connections 2024-09-10T08:16:50.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.357+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/4182301107 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x563822bb2dc0 0x563822bb51e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.357+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/4182301107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563822b37e50 0x563822b31f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.357+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/4182301107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563822b2d590 0x563822b31a00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.357+0000 7f8ab01ee740 1 --2- 172.21.15.2:0/4182301107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563822b2cc70 0x563822b40410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.357+0000 7f8a8be00640 0 client.27842 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:16:50.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.357+0000 7f8ab01ee740 1 -- 172.21.15.2:0/4182301107 >> 172.21.15.2:0/4182301107 conn(0x563822b29200 msgr2=0x563822b27b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:50.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.357+0000 7f8ab01ee740 1 -- 172.21.15.2:0/4182301107 shutdown_connections 2024-09-10T08:16:50.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.357+0000 7f8ab01ee740 1 -- 172.21.15.2:0/4182301107 wait complete. 2024-09-10T08:16:50.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.357+0000 7f8ab01ee740 20 client.27842 trim_cache size 0 max 16384 2024-09-10T08:16:50.360 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:16:50.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.358+0000 7f8aad600640 1 -- 172.21.15.2:0/3876142412 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8a8c0a2b80 msgr2=0x7f8a8c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.358+0000 7f8aad600640 1 --2- 172.21.15.2:0/3876142412 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8a8c0a2b80 0x7f8a8c0a5040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x563822b2a580 tx=0x7f8a90009950 comp rx=0 tx=0).stop 2024-09-10T08:16:50.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.358+0000 7f8aad600640 1 -- 172.21.15.2:0/3876142412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa810e440 msgr2=0x7f8aa819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.358+0000 7f8aad600640 1 --2- 172.21.15.2:0/3876142412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa810e440 0x7f8aa819efe0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000eb40 tx=0x7f8aa000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.358+0000 7f8aad600640 1 -- 172.21.15.2:0/3876142412 shutdown_connections 2024-09-10T08:16:50.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.358+0000 7f8aad600640 1 --2- 172.21.15.2:0/3876142412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aa810e440 0x7f8aa819efe0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.358+0000 7f8aad600640 1 --2- 172.21.15.2:0/3876142412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aa8074e30 0x7f8aa819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.358+0000 7f8aad600640 1 --2- 172.21.15.2:0/3876142412 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8a8c0a2b80 0x7f8a8c0a5040 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.358+0000 7f8aad600640 1 --2- 172.21.15.2:0/3876142412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aa8074510 0x7f8aa819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.358+0000 7f8aad600640 1 -- 172.21.15.2:0/3876142412 >> 172.21.15.2:0/3876142412 conn(0x7f8aa80fd8c0 msgr2=0x7f8aa80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:50.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.358+0000 7f8aad600640 1 -- 172.21.15.2:0/3876142412 shutdown_connections 2024-09-10T08:16:50.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:50.358+0000 7f8aad600640 1 -- 172.21.15.2:0/3876142412 wait complete. 2024-09-10T08:16:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.702+0000 7fc9b0400640 1 -- 172.21.15.2:0/641910278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a8105910 msgr2=0x7fc9a810c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.702+0000 7fc9b0400640 1 --2- 172.21.15.2:0/641910278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a8105910 0x7fc9a810c1a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009ca0 tx=0x7fc9a002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.702+0000 7fc9b0400640 1 -- 172.21.15.2:0/641910278 shutdown_connections 2024-09-10T08:16:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.702+0000 7fc9b0400640 1 --2- 172.21.15.2:0/641910278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a8105910 0x7fc9a810c1a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.702+0000 7fc9b0400640 1 --2- 172.21.15.2:0/641910278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a8104f50 0x7fc9a81053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.702+0000 7fc9b0400640 1 --2- 172.21.15.2:0/641910278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8103d60 0x7fc9a8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.702+0000 7fc9b0400640 1 -- 172.21.15.2:0/641910278 >> 172.21.15.2:0/641910278 conn(0x7fc9a80ff4e0 msgr2=0x7fc9a8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.703+0000 7fc9b0400640 1 -- 172.21.15.2:0/641910278 shutdown_connections 2024-09-10T08:16:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.703+0000 7fc9b0400640 1 -- 172.21.15.2:0/641910278 wait complete. 2024-09-10T08:16:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.703+0000 7fc9b0400640 1 Processor -- start 2024-09-10T08:16:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.704+0000 7fc9b0400640 1 -- start start 2024-09-10T08:16:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.704+0000 7fc9b0400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a8103d60 0x7fc9a8199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.704+0000 7fc9b0400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a8104f50 0x7fc9a819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.704+0000 7fc9b0400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8105910 0x7fc9a81a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.704+0000 7fc9b0400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a810f9e0 con 0x7fc9a8105910 2024-09-10T08:16:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.704+0000 7fc9b0400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a810f860 con 0x7fc9a8104f50 2024-09-10T08:16:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.704+0000 7fc9b0400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a810fb60 con 0x7fc9a8103d60 2024-09-10T08:16:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.704+0000 7fc9ae400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8105910 0x7fc9a81a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.704+0000 7fc9ae400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8105910 0x7fc9a81a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50308/0 (socket says 172.21.15.2:50308) 2024-09-10T08:16:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.704+0000 7fc9ae400640 1 -- 172.21.15.2:0/1873550920 learned_addr learned my addr 172.21.15.2:0/1873550920 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.704+0000 7fc9ad000640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a8104f50 0x7fc9a819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.704+0000 7fc9ada00640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a8103d60 0x7fc9a8199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.705+0000 7fc9ad000640 1 -- 172.21.15.2:0/1873550920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a8103d60 msgr2=0x7fc9a8199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.705+0000 7fc9ad000640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a8103d60 0x7fc9a8199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.705+0000 7fc9ad000640 1 -- 172.21.15.2:0/1873550920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8105910 msgr2=0x7fc9a81a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.705+0000 7fc9ad000640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8105910 0x7fc9a81a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.705+0000 7fc9ad000640 1 -- 172.21.15.2:0/1873550920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a81a1ba0 con 0x7fc9a8104f50 2024-09-10T08:16:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.705+0000 7fc9ae400640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8105910 0x7fc9a81a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.705+0000 7fc9ad000640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a8104f50 0x7fc9a819a4c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc99400e9f0 tx=0x7fc99400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.705+0000 7fc9a6000640 1 -- 172.21.15.2:0/1873550920 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99400ccc0 con 0x7fc9a8104f50 2024-09-10T08:16:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.705+0000 7fc9a6000640 1 -- 172.21.15.2:0/1873550920 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc994005020 con 0x7fc9a8104f50 2024-09-10T08:16:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.705+0000 7fc9b0400640 1 -- 172.21.15.2:0/1873550920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a81a1e90 con 0x7fc9a8104f50 2024-09-10T08:16:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.705+0000 7fc9a6000640 1 -- 172.21.15.2:0/1873550920 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9940049e0 con 0x7fc9a8104f50 2024-09-10T08:16:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.706+0000 7fc9b0400640 1 -- 172.21.15.2:0/1873550920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a81a2350 con 0x7fc9a8104f50 2024-09-10T08:16:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.708+0000 7fc9b3254740 1 Processor -- start 2024-09-10T08:16:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.708+0000 7fc9a6000640 1 -- 172.21.15.2:0/1873550920 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc994005560 con 0x7fc9a8104f50 2024-09-10T08:16:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.708+0000 7fc9b3254740 1 -- start start 2024-09-10T08:16:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9b3254740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a090 0x55a64765a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9b3254740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a9d0 0x55a64765f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9b3254740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765fa60 0x55a647661f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9b3254740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a6475bf100 con 0x55a64765a9d0 2024-09-10T08:16:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9b3254740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a6475671b0 con 0x55a64765fa60 2024-09-10T08:16:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9b3254740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a6475bb0c0 con 0x55a64765a090 2024-09-10T08:16:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9ada00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a9d0 0x55a64765f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9ad000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765fa60 0x55a647661f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9ada00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a9d0 0x55a64765f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50316/0 (socket says 172.21.15.2:50316) 2024-09-10T08:16:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9ada00640 1 -- 172.21.15.2:0/3682130832 learned_addr learned my addr 172.21.15.2:0/3682130832 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9ae400640 1 --2- 172.21.15.2:0/3682130832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a090 0x55a64765a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9ada00640 1 -- 172.21.15.2:0/3682130832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a090 msgr2=0x55a64765a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9ada00640 1 --2- 172.21.15.2:0/3682130832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a090 0x55a64765a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9ada00640 1 -- 172.21.15.2:0/3682130832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765fa60 msgr2=0x55a647661f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9ada00640 1 --2- 172.21.15.2:0/3682130832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765fa60 0x55a647661f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.709+0000 7fc9ada00640 1 -- 172.21.15.2:0/3682130832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6472d2210 con 0x55a64765a9d0 2024-09-10T08:16:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.710+0000 7fc9ada00640 1 --2- 172.21.15.2:0/3682130832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a9d0 0x55a64765f490 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009ca0 tx=0x7fc99c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2818e1d052f4363 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.710+0000 7fc99aa00640 1 -- 172.21.15.2:0/3682130832 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c004220 con 0x55a64765a9d0 2024-09-10T08:16:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.710+0000 7fc99aa00640 1 -- 172.21.15.2:0/3682130832 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc99c0043c0 con 0x55a64765a9d0 2024-09-10T08:16:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.710+0000 7fc99aa00640 1 -- 172.21.15.2:0/3682130832 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0198d0 con 0x55a64765a9d0 2024-09-10T08:16:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.710+0000 7fc9a6000640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc9900a2cb0 0x7fc9900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.710+0000 7fc9a6000640 1 -- 172.21.15.2:0/1873550920 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc9940cf2e0 con 0x7fc9a8104f50 2024-09-10T08:16:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.711+0000 7fc9ae400640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc9900a2cb0 0x7fc9900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.711+0000 7fc9b3254740 1 -- 172.21.15.2:0/3682130832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a9d0 msgr2=0x55a64765f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.711+0000 7fc9b3254740 1 --2- 172.21.15.2:0/3682130832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a9d0 0x55a64765f490 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009ca0 tx=0x7fc99c011810 comp rx=0 tx=0).stop 2024-09-10T08:16:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.711+0000 7fc9b3254740 1 -- 172.21.15.2:0/3682130832 shutdown_connections 2024-09-10T08:16:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.711+0000 7fc9b3254740 1 --2- 172.21.15.2:0/3682130832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765fa60 0x55a647661f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.711+0000 7fc9b3254740 1 --2- 172.21.15.2:0/3682130832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a9d0 0x55a64765f490 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.711+0000 7fc9b3254740 1 --2- 172.21.15.2:0/3682130832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a090 0x55a64765a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.711+0000 7fc9b3254740 1 -- 172.21.15.2:0/3682130832 >> 172.21.15.2:0/3682130832 conn(0x55a647655700 msgr2=0x55a647655b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.711+0000 7fc9b3254740 1 -- 172.21.15.2:0/3682130832 shutdown_connections 2024-09-10T08:16:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.711+0000 7fc9b3254740 1 -- 172.21.15.2:0/3682130832 wait complete. 2024-09-10T08:16:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.711+0000 7fc9ae400640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc9900a2cb0 0x7fc9900a5170 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005e30 tx=0x7fc9a003d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.711+0000 7fc9b3254740 1 -- start start 2024-09-10T08:16:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.712+0000 7fc9b3254740 1 Processor -- start 2024-09-10T08:16:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.712+0000 7fc9b3254740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a090 0x55a647664220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.712+0000 7fc9b3254740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a9d0 0x55a647664760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.712+0000 7fc9b3254740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765fa60 0x55a64766b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.712+0000 7fc9b3254740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a647662a30 con 0x55a64765a090 2024-09-10T08:16:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.712+0000 7fc9b3254740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a647548090 con 0x55a64765fa60 2024-09-10T08:16:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.712+0000 7fc9b3254740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a647662bb0 con 0x55a64765a9d0 2024-09-10T08:16:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.712+0000 7fc9b3254740 10 client.0 _ll_register_callbacks cb 0x55a64759cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.712+0000 7fc9ada00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a090 0x55a647664220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.712+0000 7fc9ada00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a090 0x55a647664220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50318/0 (socket says 172.21.15.2:50318) 2024-09-10T08:16:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.712+0000 7fc9ada00640 1 -- 172.21.15.2:0/737759269 learned_addr learned my addr 172.21.15.2:0/737759269 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.712+0000 7fc9ad000640 1 --2- 172.21.15.2:0/737759269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a9d0 0x55a647664760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.712+0000 7fc9ae400640 1 --2- 172.21.15.2:0/737759269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765fa60 0x55a64766b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.713+0000 7fc9ada00640 1 -- 172.21.15.2:0/737759269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a9d0 msgr2=0x55a647664760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.713+0000 7fc9ada00640 1 --2- 172.21.15.2:0/737759269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a9d0 0x55a647664760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.713+0000 7fc9ada00640 1 -- 172.21.15.2:0/737759269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765fa60 msgr2=0x55a64766b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.713+0000 7fc9ada00640 1 --2- 172.21.15.2:0/737759269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765fa60 0x55a64766b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.713+0000 7fc9ada00640 1 -- 172.21.15.2:0/737759269 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a64739e620 con 0x55a64765a090 2024-09-10T08:16:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.713+0000 7fc9ada00640 1 --2- 172.21.15.2:0/737759269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a090 0x55a647664220 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc99c011d10 tx=0x7fc99c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa2c02d2d639a3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.713+0000 7fc98f400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:16:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.713+0000 7fc98f400640 1 -- 172.21.15.2:0/737759269 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c028070 con 0x55a64765a090 2024-09-10T08:16:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.713+0000 7fc98f400640 1 -- 172.21.15.2:0/737759269 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc99c005040 con 0x55a64765a090 2024-09-10T08:16:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.713+0000 7fc9b3254740 1 -- 172.21.15.2:0/737759269 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc99c0043c0 con 0x55a64765a090 2024-09-10T08:16:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.714+0000 7fc98f400640 1 -- 172.21.15.2:0/737759269 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc99c034050 con 0x55a64765a090 2024-09-10T08:16:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.714+0000 7fc9b3254740 10 client.17826 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.714+0000 7fc9b3254740 1 -- 172.21.15.2:0/737759269 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a64766e520 con 0x55a64765a090 2024-09-10T08:16:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.714+0000 7fc98f400640 1 -- 172.21.15.2:0/737759269 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc99c0043c0 con 0x55a64765a090 2024-09-10T08:16:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.714+0000 7fc98f400640 1 -- 172.21.15.2:0/737759269 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc99c004a50 con 0x55a64765a090 2024-09-10T08:16:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.714+0000 7fc9b3254740 10 client.17826 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.714+0000 7fc9b3254740 10 client.17826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.714+0000 7fc9b3254740 20 client.17826 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.714+0000 7fc9b3254740 1 --2- 172.21.15.2:0/737759269 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a64766faa0 0x55a647671eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.714+0000 7fc9b3254740 4 client.17826 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.714+0000 7fc9b3254740 1 -- 172.21.15.2:0/737759269 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6475cc830 con 0x55a64766faa0 2024-09-10T08:16:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.714+0000 7fc9ad000640 1 --2- 172.21.15.2:0/737759269 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a64766faa0 0x55a647671eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.715+0000 7fc9ad000640 1 --2- 172.21.15.2:0/737759269 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a64766faa0 0x55a647671eb0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a97777591e9f4fb server_cookie=faf51d13f8bf7756 in_seq=0 out_seq=0 2024-09-10T08:16:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.715+0000 7fc98f400640 10 client.17826 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.716+0000 7fc98f400640 1 -- 172.21.15.2:0/737759269 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6475cc830 con 0x55a64766faa0 2024-09-10T08:16:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.716+0000 7fc98f400640 10 client.17826 handle_command_reply: tid=0 2024-09-10T08:16:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.716+0000 7fc9b3254740 1 client.17826 shutdown 2024-09-10T08:16:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.716+0000 7fc9b3254740 1 -- 172.21.15.2:0/737759269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a090 msgr2=0x55a647664220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.716+0000 7fc9b3254740 1 --2- 172.21.15.2:0/737759269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a090 0x55a647664220 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc99c011d10 tx=0x7fc99c0038a0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.716+0000 7fc9b3254740 1 -- 172.21.15.2:0/737759269 shutdown_connections 2024-09-10T08:16:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.716+0000 7fc9b3254740 1 --2- 172.21.15.2:0/737759269 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a64766faa0 0x55a647671eb0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.716+0000 7fc9b3254740 1 --2- 172.21.15.2:0/737759269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765fa60 0x55a64766b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.716+0000 7fc98f400640 0 client.17826 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.716+0000 7fc9b3254740 1 --2- 172.21.15.2:0/737759269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a9d0 0x55a647664760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.716+0000 7fc9b3254740 1 --2- 172.21.15.2:0/737759269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a090 0x55a647664220 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.716+0000 7fc9b3254740 1 -- 172.21.15.2:0/737759269 >> 172.21.15.2:0/737759269 conn(0x55a647655700 msgr2=0x55a647656f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.716+0000 7fc9b3254740 1 -- 172.21.15.2:0/737759269 shutdown_connections 2024-09-10T08:16:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.717+0000 7fc9b3254740 1 -- 172.21.15.2:0/737759269 wait complete. 2024-09-10T08:16:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.717+0000 7fc9b3254740 20 client.17826 trim_cache size 0 max 16384 2024-09-10T08:16:50.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.732+0000 7fc9b3254740 1 Processor -- start 2024-09-10T08:16:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.732+0000 7fc9b3254740 1 -- start start 2024-09-10T08:16:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.732+0000 7fc9b3254740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a090 0x55a647663fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9b3254740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a9d0 0x55a64766b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9b3254740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765fa60 0x55a647664510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9b3254740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81af350 con 0x55a64765a090 2024-09-10T08:16:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9b3254740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81acc20 con 0x55a64765a9d0 2024-09-10T08:16:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9b3254740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81aaa70 con 0x55a64765fa60 2024-09-10T08:16:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9ada00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a090 0x55a647663fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9ad000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a9d0 0x55a64766b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9ae400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765fa60 0x55a647664510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9ad000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a9d0 0x55a64766b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44060/0 (socket says 172.21.15.2:44060) 2024-09-10T08:16:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9ada00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a090 0x55a647663fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50322/0 (socket says 172.21.15.2:50322) 2024-09-10T08:16:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9ad000640 1 -- 172.21.15.2:0/1234149402 learned_addr learned my addr 172.21.15.2:0/1234149402 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9ad000640 1 -- 172.21.15.2:0/1234149402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765fa60 msgr2=0x55a647664510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9ad000640 1 --2- 172.21.15.2:0/1234149402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765fa60 0x55a647664510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9ad000640 1 -- 172.21.15.2:0/1234149402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a090 msgr2=0x55a647663fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9ad000640 1 --2- 172.21.15.2:0/1234149402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a090 0x55a647663fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.733+0000 7fc9ad000640 1 -- 172.21.15.2:0/1234149402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a64765aeb0 con 0x55a64765a9d0 2024-09-10T08:16:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.734+0000 7fc9ad000640 1 --2- 172.21.15.2:0/1234149402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a9d0 0x55a64766b6f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc994097bb0 tx=0x7fc9940d17f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b925838037258fc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.734+0000 7fc99a000640 1 -- 172.21.15.2:0/1234149402 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9940b4070 con 0x55a64765a9d0 2024-09-10T08:16:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.734+0000 7fc99a000640 1 -- 172.21.15.2:0/1234149402 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9940cf120 con 0x55a64765a9d0 2024-09-10T08:16:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.734+0000 7fc99a000640 1 -- 172.21.15.2:0/1234149402 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99401c040 con 0x55a64765a9d0 2024-09-10T08:16:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.735+0000 7fc9b3254740 1 -- 172.21.15.2:0/1234149402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a9d0 msgr2=0x55a64766b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.735+0000 7fc9b3254740 1 --2- 172.21.15.2:0/1234149402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a9d0 0x55a64766b6f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc994097bb0 tx=0x7fc9940d17f0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.735+0000 7fc9b3254740 1 -- 172.21.15.2:0/1234149402 shutdown_connections 2024-09-10T08:16:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.735+0000 7fc9b3254740 1 --2- 172.21.15.2:0/1234149402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765fa60 0x55a647664510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.735+0000 7fc9b3254740 1 --2- 172.21.15.2:0/1234149402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a9d0 0x55a64766b6f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.735+0000 7fc9b3254740 1 --2- 172.21.15.2:0/1234149402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765a090 0x55a647663fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.735+0000 7fc9b3254740 1 -- 172.21.15.2:0/1234149402 >> 172.21.15.2:0/1234149402 conn(0x55a647655700 msgr2=0x55a64765bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.735+0000 7fc9b3254740 1 -- 172.21.15.2:0/1234149402 shutdown_connections 2024-09-10T08:16:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.735+0000 7fc9b3254740 1 -- 172.21.15.2:0/1234149402 wait complete. 2024-09-10T08:16:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.735+0000 7fc9b3254740 1 -- start start 2024-09-10T08:16:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.735+0000 7fc9b3254740 1 Processor -- start 2024-09-10T08:16:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.736+0000 7fc9b3254740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a090 0x55a6476689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.736+0000 7fc9b3254740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a9d0 0x55a647668f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.736+0000 7fc9b3254740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765fa60 0x55a647669f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.736+0000 7fc9b3254740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81ab0e0 con 0x55a64765fa60 2024-09-10T08:16:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.736+0000 7fc9b3254740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81ac550 con 0x55a64765a090 2024-09-10T08:16:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.736+0000 7fc9b3254740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81ab7b0 con 0x55a64765a9d0 2024-09-10T08:16:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.736+0000 7fc9b3254740 10 client.0 _ll_register_callbacks cb 0x7fc9940d2c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.736+0000 7fc9ada00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a090 0x55a6476689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.736+0000 7fc9ae400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765fa60 0x55a647669f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.736+0000 7fc9ada00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a090 0x55a6476689c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44070/0 (socket says 172.21.15.2:44070) 2024-09-10T08:16:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.737+0000 7fc9ada00640 1 -- 172.21.15.2:0/1673775098 learned_addr learned my addr 172.21.15.2:0/1673775098 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.736+0000 7fc9ae400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765fa60 0x55a647669f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50330/0 (socket says 172.21.15.2:50330) 2024-09-10T08:16:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.737+0000 7fc9ada00640 1 -- 172.21.15.2:0/1673775098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a9d0 msgr2=0x55a647668f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:16:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.737+0000 7fc9ada00640 1 --2- 172.21.15.2:0/1673775098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a9d0 0x55a647668f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.737+0000 7fc9ada00640 1 -- 172.21.15.2:0/1673775098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765fa60 msgr2=0x55a647669f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.737+0000 7fc9ada00640 1 --2- 172.21.15.2:0/1673775098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765fa60 0x55a647669f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.737+0000 7fc9ada00640 1 -- 172.21.15.2:0/1673775098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a64766a520 con 0x55a64765a090 2024-09-10T08:16:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.737+0000 7fc9ada00640 1 --2- 172.21.15.2:0/1673775098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a090 0x55a6476689c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc99c011d10 tx=0x7fc99c019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f3e953f4e786d9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.737+0000 7fc98f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.737+0000 7fc98f400640 1 -- 172.21.15.2:0/1673775098 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c028040 con 0x55a64765a090 2024-09-10T08:16:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.737+0000 7fc9b3254740 1 -- 172.21.15.2:0/1673775098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a6475c6b80 con 0x55a64765a090 2024-09-10T08:16:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.737+0000 7fc98f400640 1 -- 172.21.15.2:0/1673775098 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc99c019c40 con 0x55a64765a090 2024-09-10T08:16:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.738+0000 7fc98f400640 1 -- 172.21.15.2:0/1673775098 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc99c019e60 con 0x55a64765a090 2024-09-10T08:16:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.738+0000 7fc9b3254740 10 client.27860 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.738+0000 7fc9b3254740 1 -- 172.21.15.2:0/1673775098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a64766b860 con 0x55a64765a090 2024-09-10T08:16:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.738+0000 7fc98f400640 1 -- 172.21.15.2:0/1673775098 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a6475c6b80 con 0x55a64765a090 2024-09-10T08:16:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.738+0000 7fc98f400640 1 -- 172.21.15.2:0/1673775098 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc99c004250 con 0x55a64765a090 2024-09-10T08:16:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.738+0000 7fc9b3254740 10 client.27860 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.738+0000 7fc9b3254740 10 client.27860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.738+0000 7fc9b3254740 20 client.27860 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.738+0000 7fc9b3254740 1 --2- 172.21.15.2:0/1673775098 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a6476dae30 0x55a6476dd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.738+0000 7fc9b3254740 4 client.27860 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:16:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.738+0000 7fc9b3254740 1 -- 172.21.15.2:0/1673775098 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6475437c0 con 0x55a6476dae30 2024-09-10T08:16:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.739+0000 7fc9ad000640 1 --2- 172.21.15.2:0/1673775098 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a6476dae30 0x55a6476dd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.739+0000 7fc9ad000640 1 --2- 172.21.15.2:0/1673775098 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a6476dae30 0x55a6476dd250 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c6d1965ebb0d91 server_cookie=442d3fc25cb6009b in_seq=0 out_seq=0 2024-09-10T08:16:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.739+0000 7fc98f400640 10 client.27860 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.753+0000 7fc98f400640 1 -- 172.21.15.2:0/1673775098 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6475437c0 con 0x55a6476dae30 2024-09-10T08:16:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.753+0000 7fc98f400640 10 client.27860 handle_command_reply: tid=0 2024-09-10T08:16:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.753+0000 7fc9b3254740 1 client.27860 shutdown 2024-09-10T08:16:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.754+0000 7fc9b3254740 1 -- 172.21.15.2:0/1673775098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a090 msgr2=0x55a6476689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.754+0000 7fc9b3254740 1 --2- 172.21.15.2:0/1673775098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a090 0x55a6476689c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc99c011d10 tx=0x7fc99c019640 comp rx=0 tx=0).stop 2024-09-10T08:16:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.754+0000 7fc9b3254740 1 -- 172.21.15.2:0/1673775098 shutdown_connections 2024-09-10T08:16:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.754+0000 7fc9b3254740 1 --2- 172.21.15.2:0/1673775098 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a6476dae30 0x55a6476dd250 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.754+0000 7fc9b3254740 1 --2- 172.21.15.2:0/1673775098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a64765fa60 0x55a647669f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.754+0000 7fc98f400640 0 client.27860 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.754+0000 7fc9b3254740 1 --2- 172.21.15.2:0/1673775098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a64765a9d0 0x55a647668f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.754+0000 7fc9b3254740 1 --2- 172.21.15.2:0/1673775098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a64765a090 0x55a6476689c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.754+0000 7fc9b3254740 1 -- 172.21.15.2:0/1673775098 >> 172.21.15.2:0/1673775098 conn(0x55a647655700 msgr2=0x55a647663fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.754+0000 7fc9b3254740 1 -- 172.21.15.2:0/1673775098 shutdown_connections 2024-09-10T08:16:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.754+0000 7fc9b3254740 1 -- 172.21.15.2:0/1673775098 wait complete. 2024-09-10T08:16:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.754+0000 7fc9b3254740 20 client.27860 trim_cache size 0 max 16384 2024-09-10T08:16:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:16:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:16:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:16:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:16:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.755+0000 7fc9b0400640 1 -- 172.21.15.2:0/1873550920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc9900a2cb0 msgr2=0x7fc9900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.755+0000 7fc9b0400640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc9900a2cb0 0x7fc9900a5170 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005e30 tx=0x7fc9a003d040 comp rx=0 tx=0).stop 2024-09-10T08:16:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.755+0000 7fc9b0400640 1 -- 172.21.15.2:0/1873550920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a8104f50 msgr2=0x7fc9a819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.755+0000 7fc9b0400640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a8104f50 0x7fc9a819a4c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc99400e9f0 tx=0x7fc99400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.755+0000 7fc9b0400640 1 -- 172.21.15.2:0/1873550920 shutdown_connections 2024-09-10T08:16:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.755+0000 7fc9b0400640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8105910 0x7fc9a81a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.755+0000 7fc9b0400640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a8104f50 0x7fc9a819a4c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.755+0000 7fc9b0400640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc9900a2cb0 0x7fc9900a5170 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.755+0000 7fc9b0400640 1 --2- 172.21.15.2:0/1873550920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a8103d60 0x7fc9a8199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.755+0000 7fc9b0400640 1 -- 172.21.15.2:0/1873550920 >> 172.21.15.2:0/1873550920 conn(0x7fc9a80ff4e0 msgr2=0x7fc9a81081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.755+0000 7fc9b0400640 1 -- 172.21.15.2:0/1873550920 shutdown_connections 2024-09-10T08:16:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:16:50.755+0000 7fc9b0400640 1 -- 172.21.15.2:0/1873550920 wait complete. 2024-09-10T08:16:51.112 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:16:51.142 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:51.591 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:51 smithi002 ceph-mon[30563]: pgmap v648: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-09-10T08:16:51.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.711+0000 7ff10cc00640 1 -- 172.21.15.2:0/2148563017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1081049a0 msgr2=0x7ff1081089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:51.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.711+0000 7ff10cc00640 1 --2- 172.21.15.2:0/2148563017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1081049a0 0x7ff1081089f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009860 tx=0x7ff0f002a340 comp rx=0 tx=0).stop 2024-09-10T08:16:51.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.711+0000 7ff10cc00640 1 -- 172.21.15.2:0/2148563017 shutdown_connections 2024-09-10T08:16:51.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.711+0000 7ff10cc00640 1 --2- 172.21.15.2:0/2148563017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff108109120 0x7ff108110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.711+0000 7ff10cc00640 1 --2- 172.21.15.2:0/2148563017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1081049a0 0x7ff1081089f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.711+0000 7ff10cc00640 1 --2- 172.21.15.2:0/2148563017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff108103ff0 0x7ff1081043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.711+0000 7ff10cc00640 1 -- 172.21.15.2:0/2148563017 >> 172.21.15.2:0/2148563017 conn(0x7ff1080ffa90 msgr2=0x7ff108101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:51.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.712+0000 7ff10cc00640 1 -- 172.21.15.2:0/2148563017 shutdown_connections 2024-09-10T08:16:51.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.712+0000 7ff10cc00640 1 -- 172.21.15.2:0/2148563017 wait complete. 2024-09-10T08:16:51.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.713+0000 7ff10cc00640 1 Processor -- start 2024-09-10T08:16:51.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.713+0000 7ff10cc00640 1 -- start start 2024-09-10T08:16:51.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.713+0000 7ff10cc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff108103ff0 0x7ff10819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.713+0000 7ff10cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1081049a0 0x7ff10819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.713+0000 7ff10cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff108109120 0x7ff10819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.713+0000 7ff10cc00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff108114140 con 0x7ff1081049a0 2024-09-10T08:16:51.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.713+0000 7ff10cc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff108113fc0 con 0x7ff108109120 2024-09-10T08:16:51.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.713+0000 7ff10cc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081142c0 con 0x7ff108103ff0 2024-09-10T08:16:51.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.714+0000 7ff0fd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1081049a0 0x7ff10819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.714+0000 7ff105e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff108103ff0 0x7ff10819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.714+0000 7ff105e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff108103ff0 0x7ff10819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35466/0 (socket says 172.21.15.2:35466) 2024-09-10T08:16:51.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.714+0000 7ff105e00640 1 -- 172.21.15.2:0/2312748127 learned_addr learned my addr 172.21.15.2:0/2312748127 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:51.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.714+0000 7ff106800640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff108109120 0x7ff10819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.714+0000 7ff0fd400640 1 -- 172.21.15.2:0/2312748127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff108103ff0 msgr2=0x7ff10819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:51.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.714+0000 7ff0fd400640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff108103ff0 0x7ff10819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.714+0000 7ff0fd400640 1 -- 172.21.15.2:0/2312748127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff108109120 msgr2=0x7ff10819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:51.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.714+0000 7ff0fd400640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff108109120 0x7ff10819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.714+0000 7ff0fd400640 1 -- 172.21.15.2:0/2312748127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10819f670 con 0x7ff1081049a0 2024-09-10T08:16:51.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.714+0000 7ff105e00640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff108103ff0 0x7ff10819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:51.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.715+0000 7ff0fd400640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1081049a0 0x7ff10819ac00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff0f002a860 tx=0x7ff0f0033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:51.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.715+0000 7ff0ff400640 1 -- 172.21.15.2:0/2312748127 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0037040 con 0x7ff1081049a0 2024-09-10T08:16:51.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.715+0000 7ff0ff400640 1 -- 172.21.15.2:0/2312748127 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f0033d50 con 0x7ff1081049a0 2024-09-10T08:16:51.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.715+0000 7ff0ff400640 1 -- 172.21.15.2:0/2312748127 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0003960 con 0x7ff1081049a0 2024-09-10T08:16:51.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.715+0000 7ff10cc00640 1 -- 172.21.15.2:0/2312748127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10819f900 con 0x7ff1081049a0 2024-09-10T08:16:51.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.715+0000 7ff10cc00640 1 -- 172.21.15.2:0/2312748127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff10819fd90 con 0x7ff1081049a0 2024-09-10T08:16:51.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.718+0000 7ff0ff400640 1 -- 172.21.15.2:0/2312748127 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff0f0042020 con 0x7ff1081049a0 2024-09-10T08:16:51.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.718+0000 7ff10f6ce740 1 Processor -- start 2024-09-10T08:16:51.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.718+0000 7ff10f6ce740 1 -- start start 2024-09-10T08:16:51.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.719+0000 7ff10f6ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269310 0x55601f2696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.719+0000 7ff10f6ce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269c30 0x55601f273fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.719+0000 7ff10f6ce740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f2744f0 0x55601f2768e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.719+0000 7ff10f6ce740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55601f1cf650 con 0x55601f2744f0 2024-09-10T08:16:51.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.719+0000 7ff10f6ce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55601f176f10 con 0x55601f269c30 2024-09-10T08:16:51.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.719+0000 7ff10f6ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55601f1cb610 con 0x55601f269310 2024-09-10T08:16:51.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.719+0000 7ff106800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269310 0x55601f2696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.719+0000 7ff106800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269310 0x55601f2696f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35474/0 (socket says 172.21.15.2:35474) 2024-09-10T08:16:51.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.719+0000 7ff106800640 1 -- 172.21.15.2:0/2121833477 learned_addr learned my addr 172.21.15.2:0/2121833477 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:51.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.719+0000 7ff105e00640 1 --2- 172.21.15.2:0/2121833477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269c30 0x55601f273fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.720+0000 7ff0fd400640 1 --2- 172.21.15.2:0/2121833477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f2744f0 0x55601f2768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.720+0000 7ff105e00640 1 -- 172.21.15.2:0/2121833477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269310 msgr2=0x55601f2696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:51.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.720+0000 7ff105e00640 1 --2- 172.21.15.2:0/2121833477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269310 0x55601f2696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.720+0000 7ff105e00640 1 -- 172.21.15.2:0/2121833477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f2744f0 msgr2=0x55601f2768e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:51.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.720+0000 7ff105e00640 1 --2- 172.21.15.2:0/2121833477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f2744f0 0x55601f2768e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.720+0000 7ff105e00640 1 -- 172.21.15.2:0/2121833477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601eede580 con 0x55601f269c30 2024-09-10T08:16:51.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.720+0000 7ff0ff400640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff0ec0a2c00 0x7ff0ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.720+0000 7ff105e00640 1 --2- 172.21.15.2:0/2121833477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269c30 0x55601f273fb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8011510 tx=0x7ff0f80119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5f816c54d61118f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:51.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.721+0000 7ff0ff400640 1 -- 172.21.15.2:0/2312748127 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff0f00ee980 con 0x7ff1081049a0 2024-09-10T08:16:51.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.721+0000 7ff0f7000640 1 -- 172.21.15.2:0/2121833477 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f801e070 con 0x55601f269c30 2024-09-10T08:16:51.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.721+0000 7ff0f7000640 1 -- 172.21.15.2:0/2121833477 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f800faa0 con 0x55601f269c30 2024-09-10T08:16:51.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.721+0000 7ff0f7000640 1 -- 172.21.15.2:0/2121833477 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8019bc0 con 0x55601f269c30 2024-09-10T08:16:51.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.721+0000 7ff106800640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff0ec0a2c00 0x7ff0ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.722+0000 7ff106800640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff0ec0a2c00 0x7ff0ec0a50c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff100012560 tx=0x7ff100009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:51.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.722+0000 7ff10f6ce740 1 -- 172.21.15.2:0/2121833477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269c30 msgr2=0x55601f273fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:51.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.722+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/2121833477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269c30 0x55601f273fb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8011510 tx=0x7ff0f80119d0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.722+0000 7ff10f6ce740 1 -- 172.21.15.2:0/2121833477 shutdown_connections 2024-09-10T08:16:51.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.722+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/2121833477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f2744f0 0x55601f2768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.722+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/2121833477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269c30 0x55601f273fb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.722+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/2121833477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269310 0x55601f2696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.722+0000 7ff10f6ce740 1 -- 172.21.15.2:0/2121833477 >> 172.21.15.2:0/2121833477 conn(0x55601f2658a0 msgr2=0x55601f265c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:51.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.722+0000 7ff10f6ce740 1 -- 172.21.15.2:0/2121833477 shutdown_connections 2024-09-10T08:16:51.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.722+0000 7ff10f6ce740 1 -- 172.21.15.2:0/2121833477 wait complete. 2024-09-10T08:16:51.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.723+0000 7ff10f6ce740 1 -- start start 2024-09-10T08:16:51.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.723+0000 7ff10f6ce740 1 Processor -- start 2024-09-10T08:16:51.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff10f6ce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269310 0x55601f2713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff10f6ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269c30 0x55601f2718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff10f6ce740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f2744f0 0x55601f271e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff10f6ce740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55601f2770c0 con 0x55601f2744f0 2024-09-10T08:16:51.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff10f6ce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55601f18a350 con 0x55601f269310 2024-09-10T08:16:51.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff10f6ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55601f277240 con 0x55601f269c30 2024-09-10T08:16:51.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff10f6ce740 10 client.0 _ll_register_callbacks cb 0x55601f1ad290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:51.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff105e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269310 0x55601f2713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff106800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f2744f0 0x55601f271e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff105e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269310 0x55601f2713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44104/0 (socket says 172.21.15.2:44104) 2024-09-10T08:16:51.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff105e00640 1 -- 172.21.15.2:0/2317184034 learned_addr learned my addr 172.21.15.2:0/2317184034 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:51.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff0fd400640 1 --2- 172.21.15.2:0/2317184034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269c30 0x55601f2718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff105e00640 1 -- 172.21.15.2:0/2317184034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269c30 msgr2=0x55601f2718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:51.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff105e00640 1 --2- 172.21.15.2:0/2317184034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269c30 0x55601f2718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff105e00640 1 -- 172.21.15.2:0/2317184034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f2744f0 msgr2=0x55601f271e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:51.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff105e00640 1 --2- 172.21.15.2:0/2317184034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f2744f0 0x55601f271e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.724+0000 7ff105e00640 1 -- 172.21.15.2:0/2317184034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601efa8960 con 0x55601f269310 2024-09-10T08:16:51.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.725+0000 7ff105e00640 1 --2- 172.21.15.2:0/2317184034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269310 0x55601f2713b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009880 tx=0x7ff0f8019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8059c9448bf48a80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:51.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.725+0000 7ff0eb400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:51.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.725+0000 7ff0eb400640 1 -- 172.21.15.2:0/2317184034 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f801e040 con 0x55601f269310 2024-09-10T08:16:51.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.725+0000 7ff10f6ce740 1 -- 172.21.15.2:0/2317184034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f800faa0 con 0x55601f269310 2024-09-10T08:16:51.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.725+0000 7ff0eb400640 1 -- 172.21.15.2:0/2317184034 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff0f800f700 con 0x55601f269310 2024-09-10T08:16:51.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.726+0000 7ff0eb400640 1 -- 172.21.15.2:0/2317184034 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0f802f050 con 0x55601f269310 2024-09-10T08:16:51.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.726+0000 7ff10f6ce740 10 client.27872 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:51.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.726+0000 7ff10f6ce740 1 -- 172.21.15.2:0/2317184034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55601f282d30 con 0x55601f269310 2024-09-10T08:16:51.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.726+0000 7ff0eb400640 1 -- 172.21.15.2:0/2317184034 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f800faa0 con 0x55601f269310 2024-09-10T08:16:51.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.726+0000 7ff0eb400640 1 -- 172.21.15.2:0/2317184034 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff0f802a030 con 0x55601f269310 2024-09-10T08:16:51.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.726+0000 7ff10f6ce740 10 client.27872 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:51.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.726+0000 7ff10f6ce740 10 client.27872 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:16:51.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.726+0000 7ff10f6ce740 20 client.27872 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:51.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.726+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/2317184034 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55601f284140 0x55601f286530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.726+0000 7ff10f6ce740 4 client.27872 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:51.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.726+0000 7ff10f6ce740 1 -- 172.21.15.2:0/2317184034 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55601f1dcbc0 con 0x55601f284140 2024-09-10T08:16:51.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.727+0000 7ff0fd400640 1 --2- 172.21.15.2:0/2317184034 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55601f284140 0x55601f286530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.727+0000 7ff0eb400640 10 client.27872 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:51.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.727+0000 7ff0fd400640 1 --2- 172.21.15.2:0/2317184034 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55601f284140 0x55601f286530 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bae62a97ce7089b server_cookie=39d281b76f3b2427 in_seq=0 out_seq=0 2024-09-10T08:16:51.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.728+0000 7ff0eb400640 1 -- 172.21.15.2:0/2317184034 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55601f1dcbc0 con 0x55601f284140 2024-09-10T08:16:51.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.728+0000 7ff0eb400640 10 client.27872 handle_command_reply: tid=0 2024-09-10T08:16:51.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.728+0000 7ff10f6ce740 1 client.27872 shutdown 2024-09-10T08:16:51.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.728+0000 7ff10f6ce740 1 -- 172.21.15.2:0/2317184034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269310 msgr2=0x55601f2713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:51.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.728+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/2317184034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269310 0x55601f2713b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009880 tx=0x7ff0f8019640 comp rx=0 tx=0).stop 2024-09-10T08:16:51.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.728+0000 7ff10f6ce740 1 -- 172.21.15.2:0/2317184034 shutdown_connections 2024-09-10T08:16:51.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.728+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/2317184034 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55601f284140 0x55601f286530 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.728+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/2317184034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f2744f0 0x55601f271e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.728+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/2317184034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269c30 0x55601f2718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.728+0000 7ff0eb400640 0 client.27872 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:51.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.728+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/2317184034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269310 0x55601f2713b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.728+0000 7ff10f6ce740 1 -- 172.21.15.2:0/2317184034 >> 172.21.15.2:0/2317184034 conn(0x55601f2658a0 msgr2=0x55601f264aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:51.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.729+0000 7ff10f6ce740 1 -- 172.21.15.2:0/2317184034 shutdown_connections 2024-09-10T08:16:51.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.729+0000 7ff10f6ce740 1 -- 172.21.15.2:0/2317184034 wait complete. 2024-09-10T08:16:51.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.729+0000 7ff10f6ce740 20 client.27872 trim_cache size 0 max 16384 2024-09-10T08:16:51.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.747+0000 7ff10f6ce740 1 Processor -- start 2024-09-10T08:16:51.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.747+0000 7ff10f6ce740 1 -- start start 2024-09-10T08:16:51.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.747+0000 7ff10f6ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269310 0x55601f26b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.747+0000 7ff10f6ce740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f269c30 0x55601f28bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.747+0000 7ff10f6ce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f2744f0 0x55601f28c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.747+0000 7ff10f6ce740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081b39b0 con 0x55601f269c30 2024-09-10T08:16:51.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.747+0000 7ff10f6ce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081b1280 con 0x55601f2744f0 2024-09-10T08:16:51.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.747+0000 7ff10f6ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081af0d0 con 0x55601f269310 2024-09-10T08:16:51.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.748+0000 7ff0fd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f269c30 0x55601f28bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.748+0000 7ff106800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f2744f0 0x55601f28c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.748+0000 7ff105e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269310 0x55601f26b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.748+0000 7ff105e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269310 0x55601f26b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35496/0 (socket says 172.21.15.2:35496) 2024-09-10T08:16:51.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.748+0000 7ff0fd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f269c30 0x55601f28bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50374/0 (socket says 172.21.15.2:50374) 2024-09-10T08:16:51.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.748+0000 7ff105e00640 1 -- 172.21.15.2:0/85262859 learned_addr learned my addr 172.21.15.2:0/85262859 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:51.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.748+0000 7ff106800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f2744f0 0x55601f28c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44118/0 (socket says 172.21.15.2:44118) 2024-09-10T08:16:51.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.748+0000 7ff105e00640 1 -- 172.21.15.2:0/85262859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f2744f0 msgr2=0x55601f28c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:51.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.748+0000 7ff105e00640 1 --2- 172.21.15.2:0/85262859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f2744f0 0x55601f28c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.748+0000 7ff105e00640 1 -- 172.21.15.2:0/85262859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f269c30 msgr2=0x55601f28bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:51.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.748+0000 7ff105e00640 1 --2- 172.21.15.2:0/85262859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f269c30 0x55601f28bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.748+0000 7ff105e00640 1 -- 172.21.15.2:0/85262859 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601f266a30 con 0x55601f269310 2024-09-10T08:16:51.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.748+0000 7ff105e00640 1 --2- 172.21.15.2:0/85262859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269310 0x55601f26b470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8010010 tx=0x7ff0f8005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e43e9b1c837c27ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:51.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.749+0000 7ff0f6600640 1 -- 172.21.15.2:0/85262859 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f801e040 con 0x55601f269310 2024-09-10T08:16:51.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.749+0000 7ff0f6600640 1 -- 172.21.15.2:0/85262859 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f8009cd0 con 0x55601f269310 2024-09-10T08:16:51.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.749+0000 7ff0f6600640 1 -- 172.21.15.2:0/85262859 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f80035f0 con 0x55601f269310 2024-09-10T08:16:51.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.749+0000 7ff10f6ce740 1 -- 172.21.15.2:0/85262859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269310 msgr2=0x55601f26b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:51.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.749+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/85262859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269310 0x55601f26b470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8010010 tx=0x7ff0f8005df0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.750+0000 7ff10f6ce740 1 -- 172.21.15.2:0/85262859 shutdown_connections 2024-09-10T08:16:51.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.750+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/85262859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f2744f0 0x55601f28c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.750+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/85262859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f269c30 0x55601f28bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.750+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/85262859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f269310 0x55601f26b470 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.750+0000 7ff10f6ce740 1 -- 172.21.15.2:0/85262859 >> 172.21.15.2:0/85262859 conn(0x55601f2658a0 msgr2=0x55601f26ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:51.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.750+0000 7ff10f6ce740 1 -- 172.21.15.2:0/85262859 shutdown_connections 2024-09-10T08:16:51.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.750+0000 7ff10f6ce740 1 -- 172.21.15.2:0/85262859 wait complete. 2024-09-10T08:16:51.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.750+0000 7ff10f6ce740 1 -- start start 2024-09-10T08:16:51.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.751+0000 7ff10f6ce740 1 Processor -- start 2024-09-10T08:16:51.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.751+0000 7ff10f6ce740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f269310 0x55601f26e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.751+0000 7ff10f6ce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269c30 0x55601f26e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.751+0000 7ff10f6ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f2744f0 0x55601f26ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.751+0000 7ff10f6ce740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081af740 con 0x55601f269310 2024-09-10T08:16:51.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.751+0000 7ff10f6ce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081b0bb0 con 0x55601f269c30 2024-09-10T08:16:51.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.751+0000 7ff10f6ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081afe10 con 0x55601f2744f0 2024-09-10T08:16:51.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.751+0000 7ff10f6ce740 10 client.0 _ll_register_callbacks cb 0x7ff0f801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:51.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.751+0000 7ff105e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f269310 0x55601f26e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.751+0000 7ff106800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f2744f0 0x55601f26ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.751+0000 7ff105e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f269310 0x55601f26e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50388/0 (socket says 172.21.15.2:50388) 2024-09-10T08:16:51.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.751+0000 7ff105e00640 1 -- 172.21.15.2:0/3151411166 learned_addr learned my addr 172.21.15.2:0/3151411166 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:51.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.751+0000 7ff0fd400640 1 --2- 172.21.15.2:0/3151411166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269c30 0x55601f26e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.752+0000 7ff106800640 1 -- 172.21.15.2:0/3151411166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269c30 msgr2=0x55601f26e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:51.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.752+0000 7ff106800640 1 --2- 172.21.15.2:0/3151411166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269c30 0x55601f26e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.752+0000 7ff106800640 1 -- 172.21.15.2:0/3151411166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f269310 msgr2=0x55601f26e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:51.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.752+0000 7ff106800640 1 --2- 172.21.15.2:0/3151411166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f269310 0x55601f26e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.752+0000 7ff106800640 1 -- 172.21.15.2:0/3151411166 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601f2edc90 con 0x55601f2744f0 2024-09-10T08:16:51.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.752+0000 7ff106800640 1 --2- 172.21.15.2:0/3151411166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f2744f0 0x55601f26ede0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55601f2810d0 tx=0x7ff100004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d722805057a5b5ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:51.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.752+0000 7ff0eb400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:16:51.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.753+0000 7ff0eb400640 1 -- 172.21.15.2:0/3151411166 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10002f060 con 0x55601f2744f0 2024-09-10T08:16:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.753+0000 7ff10f6ce740 1 -- 172.21.15.2:0/3151411166 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f8009cd0 con 0x55601f2744f0 2024-09-10T08:16:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.753+0000 7ff0eb400640 1 -- 172.21.15.2:0/3151411166 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff10000f990 con 0x55601f2744f0 2024-09-10T08:16:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.753+0000 7ff0eb400640 1 -- 172.21.15.2:0/3151411166 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff100036050 con 0x55601f2744f0 2024-09-10T08:16:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.753+0000 7ff10f6ce740 10 client.27814 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.753+0000 7ff10f6ce740 1 -- 172.21.15.2:0/3151411166 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55601f2ef010 con 0x55601f2744f0 2024-09-10T08:16:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.754+0000 7ff0eb400640 1 -- 172.21.15.2:0/3151411166 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f8009cd0 con 0x55601f2744f0 2024-09-10T08:16:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.754+0000 7ff0eb400640 1 -- 172.21.15.2:0/3151411166 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff10001e420 con 0x55601f2744f0 2024-09-10T08:16:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.754+0000 7ff10f6ce740 10 client.27814 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.754+0000 7ff10f6ce740 10 client.27814 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:16:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.754+0000 7ff10f6ce740 20 client.27814 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.754+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/3151411166 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55601f2ef710 0x55601f2f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.754+0000 7ff10f6ce740 4 client.27814 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "32b5535c", "release": true, "await": true}] 2024-09-10T08:16:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.754+0000 7ff10f6ce740 1 -- 172.21.15.2:0/3151411166 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "32b5535c", "release": true, "await": true}) -- 0x55601f1d70d0 con 0x55601f2ef710 2024-09-10T08:16:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.754+0000 7ff105e00640 1 --2- 172.21.15.2:0/3151411166 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55601f2ef710 0x55601f2f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.755+0000 7ff105e00640 1 --2- 172.21.15.2:0/3151411166 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55601f2ef710 0x55601f2f1b30 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ee97c32ea4b76a server_cookie=892c45b54e7a1d91 in_seq=0 out_seq=0 2024-09-10T08:16:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.755+0000 7ff0eb400640 10 client.27814 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff0eb400640 1 -- 172.21.15.2:0/3151411166 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55601f1d70d0 con 0x55601f2ef710 2024-09-10T08:16:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff0eb400640 10 client.27814 handle_command_reply: tid=0 2024-09-10T08:16:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff10f6ce740 1 client.27814 shutdown 2024-09-10T08:16:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff10f6ce740 1 -- 172.21.15.2:0/3151411166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f2744f0 msgr2=0x55601f26ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/3151411166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f2744f0 0x55601f26ede0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55601f2810d0 tx=0x7ff100004a40 comp rx=0 tx=0).stop 2024-09-10T08:16:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff10f6ce740 1 -- 172.21.15.2:0/3151411166 shutdown_connections 2024-09-10T08:16:51.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/3151411166 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55601f2ef710 0x55601f2f1b30 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/3151411166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55601f2744f0 0x55601f26ede0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/3151411166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55601f269c30 0x55601f26e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff10f6ce740 1 --2- 172.21.15.2:0/3151411166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55601f269310 0x55601f26e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff0eb400640 0 client.27814 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:51.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff10f6ce740 1 -- 172.21.15.2:0/3151411166 >> 172.21.15.2:0/3151411166 conn(0x55601f2658a0 msgr2=0x55601f273240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:51.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff10f6ce740 1 -- 172.21.15.2:0/3151411166 shutdown_connections 2024-09-10T08:16:51.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.757+0000 7ff10f6ce740 1 -- 172.21.15.2:0/3151411166 wait complete. 2024-09-10T08:16:51.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.758+0000 7ff10f6ce740 20 client.27814 trim_cache size 0 max 16384 2024-09-10T08:16:51.759 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:16:51.760 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:16:51.760 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:16:51.760 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 28, 2024-09-10T08:16:51.760 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:16:51.760 INFO:teuthology.orchestra.run.smithi002.stdout: "32b5535c": { 2024-09-10T08:16:51.760 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 28, 2024-09-10T08:16:51.760 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:16:51.760 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:16:51.760 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:16:51.760 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:16:51.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:16:51.761 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:16:51.761 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 133.4, 2024-09-10T08:16:51.761 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:16:51.761 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:16:51.761 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:16:51.761 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:16:51.761 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:16:51.761 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:16:51.761 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:16:51.761 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:16:51.761 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:16:51.761 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:16:51.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:16:51.762 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:16:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.758+0000 7ff10cc00640 1 -- 172.21.15.2:0/2312748127 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff0ec0a2c00 msgr2=0x7ff0ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.758+0000 7ff10cc00640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff0ec0a2c00 0x7ff0ec0a50c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff100012560 tx=0x7ff100009500 comp rx=0 tx=0).stop 2024-09-10T08:16:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.758+0000 7ff10cc00640 1 -- 172.21.15.2:0/2312748127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1081049a0 msgr2=0x7ff10819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.758+0000 7ff10cc00640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1081049a0 0x7ff10819ac00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff0f002a860 tx=0x7ff0f0033b60 comp rx=0 tx=0).stop 2024-09-10T08:16:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.759+0000 7ff10cc00640 1 -- 172.21.15.2:0/2312748127 shutdown_connections 2024-09-10T08:16:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.759+0000 7ff10cc00640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff108109120 0x7ff10819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.759+0000 7ff10cc00640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1081049a0 0x7ff10819ac00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.759+0000 7ff10cc00640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff0ec0a2c00 0x7ff0ec0a50c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.759+0000 7ff10cc00640 1 --2- 172.21.15.2:0/2312748127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff108103ff0 0x7ff10819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.759+0000 7ff10cc00640 1 -- 172.21.15.2:0/2312748127 >> 172.21.15.2:0/2312748127 conn(0x7ff1080ffa90 msgr2=0x7ff108100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.759+0000 7ff10cc00640 1 -- 172.21.15.2:0/2312748127 shutdown_connections 2024-09-10T08:16:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:51.759+0000 7ff10cc00640 1 -- 172.21.15.2:0/2312748127 wait complete. 2024-09-10T08:16:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:51 smithi155 ceph-mon[35405]: pgmap v648: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-09-10T08:16:51.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:51 smithi107 ceph-mon[35048]: pgmap v648: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-09-10T08:16:52.015 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:52.536 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '32b5535c', total seconds elapsed: 18.3 2024-09-10T08:16:53.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.047+0000 7fa35e000640 1 -- 172.21.15.2:0/518433091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3581049a0 msgr2=0x7fa3581089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.047+0000 7fa35e000640 1 --2- 172.21.15.2:0/518433091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3581049a0 0x7fa3581089f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa340009860 tx=0x7fa34002a340 comp rx=0 tx=0).stop 2024-09-10T08:16:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.047+0000 7fa35e000640 1 -- 172.21.15.2:0/518433091 shutdown_connections 2024-09-10T08:16:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.047+0000 7fa35e000640 1 --2- 172.21.15.2:0/518433091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa358109120 0x7fa358110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.047+0000 7fa35e000640 1 --2- 172.21.15.2:0/518433091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3581049a0 0x7fa3581089f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.047+0000 7fa35e000640 1 --2- 172.21.15.2:0/518433091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa358103ff0 0x7fa3581043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.047+0000 7fa35e000640 1 -- 172.21.15.2:0/518433091 >> 172.21.15.2:0/518433091 conn(0x7fa3580ffa90 msgr2=0x7fa358101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.048+0000 7fa35e000640 1 -- 172.21.15.2:0/518433091 shutdown_connections 2024-09-10T08:16:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.048+0000 7fa35e000640 1 -- 172.21.15.2:0/518433091 wait complete. 2024-09-10T08:16:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.048+0000 7fa35e000640 1 Processor -- start 2024-09-10T08:16:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.049+0000 7fa35e000640 1 -- start start 2024-09-10T08:16:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.049+0000 7fa35e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa358103ff0 0x7fa35819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.049+0000 7fa35e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3581049a0 0x7fa35819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.049+0000 7fa35e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa358109120 0x7fa35819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.049+0000 7fa35e000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa358114120 con 0x7fa3581049a0 2024-09-10T08:16:53.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.049+0000 7fa35e000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa358113fa0 con 0x7fa358103ff0 2024-09-10T08:16:53.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.049+0000 7fa35e000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581142a0 con 0x7fa358109120 2024-09-10T08:16:53.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.049+0000 7fa357400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa358103ff0 0x7fa35819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.049+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3581049a0 0x7fa35819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.049+0000 7fa357400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa358103ff0 0x7fa35819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44138/0 (socket says 172.21.15.2:44138) 2024-09-10T08:16:53.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.049+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3581049a0 0x7fa35819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50412/0 (socket says 172.21.15.2:50412) 2024-09-10T08:16:53.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.049+0000 7fa357e00640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa358109120 0x7fa35819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.049+0000 7fa357400640 1 -- 172.21.15.2:0/1027610237 learned_addr learned my addr 172.21.15.2:0/1027610237 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:53.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.050+0000 7fa357400640 1 -- 172.21.15.2:0/1027610237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa358109120 msgr2=0x7fa35819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:53.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.050+0000 7fa357400640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa358109120 0x7fa35819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.050+0000 7fa357400640 1 -- 172.21.15.2:0/1027610237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3581049a0 msgr2=0x7fa35819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.050+0000 7fa357400640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3581049a0 0x7fa35819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.050+0000 7fa357400640 1 -- 172.21.15.2:0/1027610237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35819f5d0 con 0x7fa358103ff0 2024-09-10T08:16:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.050+0000 7fa34fe00640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3581049a0 0x7fa35819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.050+0000 7fa357400640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa358103ff0 0x7fa35819a620 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa34800e9f0 tx=0x7fa34800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.050+0000 7fa354c00640 1 -- 172.21.15.2:0/1027610237 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34800cd80 con 0x7fa358103ff0 2024-09-10T08:16:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.050+0000 7fa35e000640 1 -- 172.21.15.2:0/1027610237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35819f8c0 con 0x7fa358103ff0 2024-09-10T08:16:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.050+0000 7fa354c00640 1 -- 172.21.15.2:0/1027610237 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa348010910 con 0x7fa358103ff0 2024-09-10T08:16:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.050+0000 7fa354c00640 1 -- 172.21.15.2:0/1027610237 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3480188f0 con 0x7fa358103ff0 2024-09-10T08:16:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.051+0000 7fa35e000640 1 -- 172.21.15.2:0/1027610237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3581a71a0 con 0x7fa358103ff0 2024-09-10T08:16:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.053+0000 7fa360c09740 1 Processor -- start 2024-09-10T08:16:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.054+0000 7fa360c09740 1 -- start start 2024-09-10T08:16:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.054+0000 7fa354c00640 1 -- 172.21.15.2:0/1027610237 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa348010ac0 con 0x7fa358103ff0 2024-09-10T08:16:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.054+0000 7fa360c09740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dbc70 0x5557906dc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.054+0000 7fa360c09740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dc590 0x5557906e6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.054+0000 7fa360c09740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906e9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.054+0000 7fa360c09740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5557906423e0 con 0x5557906e6e50 2024-09-10T08:16:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.054+0000 7fa360c09740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5557905e9d80 con 0x5557906dc590 2024-09-10T08:16:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.054+0000 7fa360c09740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55579063e3a0 con 0x5557906dbc70 2024-09-10T08:16:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.054+0000 7fa357e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dbc70 0x5557906dc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.054+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.054+0000 7fa357e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dbc70 0x5557906dc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35524/0 (socket says 172.21.15.2:35524) 2024-09-10T08:16:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.054+0000 7fa357e00640 1 -- 172.21.15.2:0/3223017782 learned_addr learned my addr 172.21.15.2:0/3223017782 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.054+0000 7fa357400640 1 --2- 172.21.15.2:0/3223017782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dc590 0x5557906e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.055+0000 7fa357e00640 1 -- 172.21.15.2:0/3223017782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dc590 msgr2=0x5557906e6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.055+0000 7fa357e00640 1 --2- 172.21.15.2:0/3223017782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dc590 0x5557906e6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.055+0000 7fa357e00640 1 -- 172.21.15.2:0/3223017782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 msgr2=0x5557906e9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.055+0000 7fa357e00640 1 --2- 172.21.15.2:0/3223017782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906e9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.055+0000 7fa357e00640 1 -- 172.21.15.2:0/3223017782 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555790608ae0 con 0x5557906dbc70 2024-09-10T08:16:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.055+0000 7fa34fe00640 1 --2- 172.21.15.2:0/3223017782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906e9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.055+0000 7fa357e00640 1 --2- 172.21.15.2:0/3223017782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dbc70 0x5557906dc050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa3500099f0 tx=0x7fa3500095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6b83f8319c63413 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.055+0000 7fa347e00640 1 -- 172.21.15.2:0/3223017782 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35001d070 con 0x5557906dbc70 2024-09-10T08:16:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.055+0000 7fa347e00640 1 -- 172.21.15.2:0/3223017782 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa350003e40 con 0x5557906dbc70 2024-09-10T08:16:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.055+0000 7fa347e00640 1 -- 172.21.15.2:0/3223017782 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35000dba0 con 0x5557906dbc70 2024-09-10T08:16:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.056+0000 7fa354c00640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa33c0a2c00 0x7fa33c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.056+0000 7fa354c00640 1 -- 172.21.15.2:0/1027610237 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa3480cf7d0 con 0x7fa358103ff0 2024-09-10T08:16:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.056+0000 7fa360c09740 1 -- 172.21.15.2:0/3223017782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dbc70 msgr2=0x5557906dc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:53.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.056+0000 7fa360c09740 1 --2- 172.21.15.2:0/3223017782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dbc70 0x5557906dc050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa3500099f0 tx=0x7fa3500095c0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.056+0000 7fa357400640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa33c0a2c00 0x7fa33c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.056+0000 7fa360c09740 1 -- 172.21.15.2:0/3223017782 shutdown_connections 2024-09-10T08:16:53.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.056+0000 7fa360c09740 1 --2- 172.21.15.2:0/3223017782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906e9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.056+0000 7fa360c09740 1 --2- 172.21.15.2:0/3223017782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dc590 0x5557906e6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.056+0000 7fa360c09740 1 --2- 172.21.15.2:0/3223017782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dbc70 0x5557906dc050 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.056+0000 7fa360c09740 1 -- 172.21.15.2:0/3223017782 >> 172.21.15.2:0/3223017782 conn(0x5557906d8200 msgr2=0x5557906d85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:53.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.056+0000 7fa360c09740 1 -- 172.21.15.2:0/3223017782 shutdown_connections 2024-09-10T08:16:53.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.056+0000 7fa360c09740 1 -- 172.21.15.2:0/3223017782 wait complete. 2024-09-10T08:16:53.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.056+0000 7fa357400640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa33c0a2c00 0x7fa33c0a50c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa34809b990 tx=0x7fa3480d5260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:53.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.057+0000 7fa360c09740 1 -- start start 2024-09-10T08:16:53.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.057+0000 7fa360c09740 1 Processor -- start 2024-09-10T08:16:53.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.057+0000 7fa360c09740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dbc70 0x5557906e3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.057+0000 7fa360c09740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dc590 0x5557906e42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.057+0000 7fa360c09740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906e47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.057+0000 7fa360c09740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5557906e9a20 con 0x5557906e6e50 2024-09-10T08:16:53.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.057+0000 7fa360c09740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5557905fd0f0 con 0x5557906dc590 2024-09-10T08:16:53.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.057+0000 7fa360c09740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5557906e9ba0 con 0x5557906dbc70 2024-09-10T08:16:53.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.057+0000 7fa360c09740 10 client.0 _ll_register_callbacks cb 0x5557905e4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:53.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.058+0000 7fa357e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906e47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.058+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dc590 0x5557906e42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.058+0000 7fa357e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906e47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50424/0 (socket says 172.21.15.2:50424) 2024-09-10T08:16:53.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.058+0000 7fa357e00640 1 -- 172.21.15.2:0/1258447062 learned_addr learned my addr 172.21.15.2:0/1258447062 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:53.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.058+0000 7fa357e00640 1 --2- 172.21.15.2:0/1258447062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dbc70 0x5557906e3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.058+0000 7fa34fe00640 1 -- 172.21.15.2:0/1258447062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dbc70 msgr2=0x5557906e3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.058+0000 7fa34fe00640 1 --2- 172.21.15.2:0/1258447062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dbc70 0x5557906e3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.058+0000 7fa34fe00640 1 -- 172.21.15.2:0/1258447062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 msgr2=0x5557906e47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.058+0000 7fa34fe00640 1 --2- 172.21.15.2:0/1258447062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906e47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.058+0000 7fa34fe00640 1 -- 172.21.15.2:0/1258447062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557903b33c0 con 0x5557906dc590 2024-09-10T08:16:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.058+0000 7fa357e00640 1 --2- 172.21.15.2:0/1258447062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906e47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.058+0000 7fa34fe00640 1 --2- 172.21.15.2:0/1258447062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dc590 0x5557906e42a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa340035040 tx=0x7fa3400045d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=815c7fe73a327e02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.058+0000 7fa344c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.059+0000 7fa344c00640 1 -- 172.21.15.2:0/1258447062 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340042070 con 0x5557906dc590 2024-09-10T08:16:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.059+0000 7fa344c00640 1 -- 172.21.15.2:0/1258447062 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa340030960 con 0x5557906dc590 2024-09-10T08:16:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.059+0000 7fa360c09740 1 -- 172.21.15.2:0/1258447062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55579064f950 con 0x5557906dc590 2024-09-10T08:16:53.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.059+0000 7fa344c00640 1 -- 172.21.15.2:0/1258447062 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa34004b050 con 0x5557906dc590 2024-09-10T08:16:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.059+0000 7fa360c09740 10 client.27890 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.059+0000 7fa360c09740 1 -- 172.21.15.2:0/1258447062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5557906f5a90 con 0x5557906dc590 2024-09-10T08:16:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.059+0000 7fa344c00640 1 -- 172.21.15.2:0/1258447062 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55579064f950 con 0x5557906dc590 2024-09-10T08:16:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.060+0000 7fa344c00640 1 -- 172.21.15.2:0/1258447062 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa340045020 con 0x5557906dc590 2024-09-10T08:16:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.060+0000 7fa360c09740 10 client.27890 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.060+0000 7fa360c09740 10 client.27890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.060+0000 7fa360c09740 20 client.27890 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.060+0000 7fa360c09740 1 --2- 172.21.15.2:0/1258447062 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5557906f6fa0 0x5557906f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.060+0000 7fa360c09740 4 client.27890 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.060+0000 7fa360c09740 1 -- 172.21.15.2:0/1258447062 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555790649e60 con 0x5557906f6fa0 2024-09-10T08:16:53.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.060+0000 7fa357e00640 1 --2- 172.21.15.2:0/1258447062 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5557906f6fa0 0x5557906f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.061+0000 7fa357e00640 1 --2- 172.21.15.2:0/1258447062 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5557906f6fa0 0x5557906f93c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b70cb7c66bba7bd5 server_cookie=33e89a360826ee27 in_seq=0 out_seq=0 2024-09-10T08:16:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.061+0000 7fa344c00640 10 client.27890 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa344c00640 1 -- 172.21.15.2:0/1258447062 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555790649e60 con 0x5557906f6fa0 2024-09-10T08:16:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa344c00640 10 client.27890 handle_command_reply: tid=0 2024-09-10T08:16:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa360c09740 1 client.27890 shutdown 2024-09-10T08:16:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa360c09740 1 -- 172.21.15.2:0/1258447062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dc590 msgr2=0x5557906e42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa360c09740 1 --2- 172.21.15.2:0/1258447062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dc590 0x5557906e42a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa340035040 tx=0x7fa3400045d0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa360c09740 1 -- 172.21.15.2:0/1258447062 shutdown_connections 2024-09-10T08:16:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa360c09740 1 --2- 172.21.15.2:0/1258447062 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5557906f6fa0 0x5557906f93c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa360c09740 1 --2- 172.21.15.2:0/1258447062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906e47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa360c09740 1 --2- 172.21.15.2:0/1258447062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dc590 0x5557906e42a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa344c00640 0 client.27890 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:53.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa360c09740 1 --2- 172.21.15.2:0/1258447062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dbc70 0x5557906e3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa360c09740 1 -- 172.21.15.2:0/1258447062 >> 172.21.15.2:0/1258447062 conn(0x5557906d8200 msgr2=0x5557906d7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa360c09740 1 -- 172.21.15.2:0/1258447062 shutdown_connections 2024-09-10T08:16:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa360c09740 1 -- 172.21.15.2:0/1258447062 wait complete. 2024-09-10T08:16:53.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.062+0000 7fa360c09740 20 client.27890 trim_cache size 0 max 16384 2024-09-10T08:16:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.076+0000 7fa360c09740 1 Processor -- start 2024-09-10T08:16:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.076+0000 7fa360c09740 1 -- start start 2024-09-10T08:16:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.077+0000 7fa360c09740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dbc70 0x5557906e4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.077+0000 7fa360c09740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dc590 0x5557906fec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.077+0000 7fa360c09740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906ff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.077+0000 7fa360c09740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581b3940 con 0x5557906e6e50 2024-09-10T08:16:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.077+0000 7fa360c09740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581b1210 con 0x5557906dbc70 2024-09-10T08:16:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.077+0000 7fa360c09740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581af060 con 0x5557906dc590 2024-09-10T08:16:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.077+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dc590 0x5557906fec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.077+0000 7fa357e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906ff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.077+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dc590 0x5557906fec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35544/0 (socket says 172.21.15.2:35544) 2024-09-10T08:16:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.077+0000 7fa34fe00640 1 -- 172.21.15.2:0/411994176 learned_addr learned my addr 172.21.15.2:0/411994176 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.077+0000 7fa357e00640 1 --2- 172.21.15.2:0/411994176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dbc70 0x5557906e4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.078+0000 7fa34fe00640 1 -- 172.21.15.2:0/411994176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dbc70 msgr2=0x5557906e4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.078+0000 7fa34fe00640 1 --2- 172.21.15.2:0/411994176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dbc70 0x5557906e4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.078+0000 7fa34fe00640 1 -- 172.21.15.2:0/411994176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 msgr2=0x5557906ff1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.078+0000 7fa34fe00640 1 --2- 172.21.15.2:0/411994176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906ff1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.078+0000 7fa34fe00640 1 -- 172.21.15.2:0/411994176 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557906d3770 con 0x5557906dc590 2024-09-10T08:16:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.078+0000 7fa357e00640 1 --2- 172.21.15.2:0/411994176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:16:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.078+0000 7fa34fe00640 1 --2- 172.21.15.2:0/411994176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dc590 0x5557906fec90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa34003d960 tx=0x7fa34002a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9703c99c9745c32f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.078+0000 7fa347400640 1 -- 172.21.15.2:0/411994176 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340000e60 con 0x5557906dc590 2024-09-10T08:16:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.078+0000 7fa347400640 1 -- 172.21.15.2:0/411994176 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa340009450 con 0x5557906dc590 2024-09-10T08:16:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.078+0000 7fa347400640 1 -- 172.21.15.2:0/411994176 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340042040 con 0x5557906dc590 2024-09-10T08:16:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.079+0000 7fa360c09740 1 -- 172.21.15.2:0/411994176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dc590 msgr2=0x5557906fec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.079+0000 7fa360c09740 1 --2- 172.21.15.2:0/411994176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dc590 0x5557906fec90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa34003d960 tx=0x7fa34002a340 comp rx=0 tx=0).stop 2024-09-10T08:16:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.079+0000 7fa360c09740 1 -- 172.21.15.2:0/411994176 shutdown_connections 2024-09-10T08:16:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.079+0000 7fa360c09740 1 --2- 172.21.15.2:0/411994176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906e6e50 0x5557906ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.079+0000 7fa360c09740 1 --2- 172.21.15.2:0/411994176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906dc590 0x5557906fec90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.079+0000 7fa360c09740 1 --2- 172.21.15.2:0/411994176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dbc70 0x5557906e4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.079+0000 7fa360c09740 1 -- 172.21.15.2:0/411994176 >> 172.21.15.2:0/411994176 conn(0x5557906d8200 msgr2=0x5557906e5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:53.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.079+0000 7fa360c09740 1 -- 172.21.15.2:0/411994176 shutdown_connections 2024-09-10T08:16:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.079+0000 7fa360c09740 1 -- 172.21.15.2:0/411994176 wait complete. 2024-09-10T08:16:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.079+0000 7fa360c09740 1 -- start start 2024-09-10T08:16:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.080+0000 7fa360c09740 1 Processor -- start 2024-09-10T08:16:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.080+0000 7fa360c09740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dbc70 0x5557906ef4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.080+0000 7fa360c09740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906dc590 0x5557906ea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.080+0000 7fa360c09740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906e6e50 0x5557906ead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.080+0000 7fa360c09740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581af6d0 con 0x5557906dc590 2024-09-10T08:16:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.080+0000 7fa360c09740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581b0b40 con 0x5557906dbc70 2024-09-10T08:16:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.080+0000 7fa360c09740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581afda0 con 0x5557906e6e50 2024-09-10T08:16:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.080+0000 7fa360c09740 10 client.0 _ll_register_callbacks cb 0x7fa350009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.080+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906dc590 0x5557906ea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.080+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906dc590 0x5557906ea7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50448/0 (socket says 172.21.15.2:50448) 2024-09-10T08:16:53.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.080+0000 7fa34fe00640 1 -- 172.21.15.2:0/3313361120 learned_addr learned my addr 172.21.15.2:0/3313361120 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.080+0000 7fa357e00640 1 --2- 172.21.15.2:0/3313361120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906e6e50 0x5557906ead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.080+0000 7fa357e00640 1 --2- 172.21.15.2:0/3313361120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dbc70 0x5557906ef4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.081+0000 7fa34fe00640 1 -- 172.21.15.2:0/3313361120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906e6e50 msgr2=0x5557906ead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.081+0000 7fa34fe00640 1 --2- 172.21.15.2:0/3313361120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906e6e50 0x5557906ead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.081+0000 7fa34fe00640 1 -- 172.21.15.2:0/3313361120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dbc70 msgr2=0x5557906ef4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.081+0000 7fa34fe00640 1 --2- 172.21.15.2:0/3313361120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dbc70 0x5557906ef4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.081+0000 7fa34fe00640 1 -- 172.21.15.2:0/3313361120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557906d8e90 con 0x5557906dc590 2024-09-10T08:16:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.081+0000 7fa34fe00640 1 --2- 172.21.15.2:0/3313361120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906dc590 0x5557906ea7d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa340006010 tx=0x7fa340005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f78af64baa478aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.081+0000 7fa344c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:16:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.081+0000 7fa344c00640 1 -- 172.21.15.2:0/3313361120 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34005c060 con 0x5557906dc590 2024-09-10T08:16:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.081+0000 7fa360c09740 1 -- 172.21.15.2:0/3313361120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa340009450 con 0x5557906dc590 2024-09-10T08:16:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.081+0000 7fa344c00640 1 -- 172.21.15.2:0/3313361120 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa340030960 con 0x5557906dc590 2024-09-10T08:16:53.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.082+0000 7fa344c00640 1 -- 172.21.15.2:0/3313361120 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa340063050 con 0x5557906dc590 2024-09-10T08:16:53.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.082+0000 7fa360c09740 10 client.17868 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:53.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.082+0000 7fa360c09740 1 -- 172.21.15.2:0/3313361120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555790761a00 con 0x5557906dc590 2024-09-10T08:16:53.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.082+0000 7fa344c00640 1 -- 172.21.15.2:0/3313361120 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa340009450 con 0x5557906dc590 2024-09-10T08:16:53.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.082+0000 7fa344c00640 1 -- 172.21.15.2:0/3313361120 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa340045050 con 0x5557906dc590 2024-09-10T08:16:53.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.082+0000 7fa360c09740 10 client.17868 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:53.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.082+0000 7fa360c09740 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.082+0000 7fa360c09740 20 client.17868 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.082+0000 7fa360c09740 1 --2- 172.21.15.2:0/3313361120 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5557907620a0 0x5557907644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.082+0000 7fa360c09740 4 client.17868 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:16:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.082+0000 7fa360c09740 1 -- 172.21.15.2:0/3313361120 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557905c6010 con 0x5557907620a0 2024-09-10T08:16:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.083+0000 7fa357e00640 1 --2- 172.21.15.2:0/3313361120 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5557907620a0 0x5557907644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.083+0000 7fa357e00640 1 --2- 172.21.15.2:0/3313361120 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5557907620a0 0x5557907644c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cda21c42e555d7c9 server_cookie=cc519ce6d2e4f1d8 in_seq=0 out_seq=0 2024-09-10T08:16:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.083+0000 7fa344c00640 10 client.17868 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.083+0000 7fa344c00640 1 -- 172.21.15.2:0/3313361120 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557905c6010 con 0x5557907620a0 2024-09-10T08:16:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.083+0000 7fa344c00640 10 client.17868 handle_command_reply: tid=0 2024-09-10T08:16:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.083+0000 7fa360c09740 1 client.17868 shutdown 2024-09-10T08:16:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.083+0000 7fa360c09740 1 -- 172.21.15.2:0/3313361120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906dc590 msgr2=0x5557906ea7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.083+0000 7fa360c09740 1 --2- 172.21.15.2:0/3313361120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906dc590 0x5557906ea7d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa340006010 tx=0x7fa340005850 comp rx=0 tx=0).stop 2024-09-10T08:16:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.084+0000 7fa360c09740 1 -- 172.21.15.2:0/3313361120 shutdown_connections 2024-09-10T08:16:53.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.084+0000 7fa360c09740 1 --2- 172.21.15.2:0/3313361120 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5557907620a0 0x5557907644c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.084+0000 7fa360c09740 1 --2- 172.21.15.2:0/3313361120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557906e6e50 0x5557906ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.084+0000 7fa360c09740 1 --2- 172.21.15.2:0/3313361120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5557906dc590 0x5557906ea7d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.084+0000 7fa360c09740 1 --2- 172.21.15.2:0/3313361120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557906dbc70 0x5557906ef4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.084+0000 7fa360c09740 1 -- 172.21.15.2:0/3313361120 >> 172.21.15.2:0/3313361120 conn(0x5557906d8200 msgr2=0x5557906d6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:53.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.084+0000 7fa344c00640 0 client.17868 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:53.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.084+0000 7fa360c09740 1 -- 172.21.15.2:0/3313361120 shutdown_connections 2024-09-10T08:16:53.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.084+0000 7fa360c09740 1 -- 172.21.15.2:0/3313361120 wait complete. 2024-09-10T08:16:53.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.084+0000 7fa360c09740 20 client.17868 trim_cache size 0 max 16384 2024-09-10T08:16:53.098 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:16:53.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.085+0000 7fa35e000640 1 -- 172.21.15.2:0/1027610237 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa33c0a2c00 msgr2=0x7fa33c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:53.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.085+0000 7fa35e000640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa33c0a2c00 0x7fa33c0a50c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa34809b990 tx=0x7fa3480d5260 comp rx=0 tx=0).stop 2024-09-10T08:16:53.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.085+0000 7fa35e000640 1 -- 172.21.15.2:0/1027610237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa358103ff0 msgr2=0x7fa35819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:53.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.085+0000 7fa35e000640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa358103ff0 0x7fa35819a620 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa34800e9f0 tx=0x7fa34800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.085+0000 7fa35e000640 1 -- 172.21.15.2:0/1027610237 shutdown_connections 2024-09-10T08:16:53.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.085+0000 7fa35e000640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa33c0a2c00 0x7fa33c0a50c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.085+0000 7fa35e000640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa358109120 0x7fa35819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.085+0000 7fa35e000640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3581049a0 0x7fa35819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.085+0000 7fa35e000640 1 --2- 172.21.15.2:0/1027610237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa358103ff0 0x7fa35819a620 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:53.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.085+0000 7fa35e000640 1 -- 172.21.15.2:0/1027610237 >> 172.21.15.2:0/1027610237 conn(0x7fa3580ffa90 msgr2=0x7fa358100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:53.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.085+0000 7fa35e000640 1 -- 172.21.15.2:0/1027610237 shutdown_connections 2024-09-10T08:16:53.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:53.085+0000 7fa35e000640 1 -- 172.21.15.2:0/1027610237 wait complete. 2024-09-10T08:16:53.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:53 smithi002 ceph-mon[30563]: pgmap v649: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-09-10T08:16:53.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:53 smithi155 ceph-mon[35405]: pgmap v649: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-09-10T08:16:53.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:53 smithi107 ceph-mon[35048]: pgmap v649: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-09-10T08:16:54.878 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:16:54.879 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:16:55.268 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:55.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:55 smithi002 ceph-mon[30563]: pgmap v650: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-10T08:16:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.904+0000 7f3d95800640 1 -- 172.21.15.2:0/956551709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d90109140 msgr2=0x7f3d90110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.904+0000 7f3d95800640 1 --2- 172.21.15.2:0/956551709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d90109140 0x7f3d90110cc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009ca0 tx=0x7f3d8802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.904+0000 7f3d95800640 1 -- 172.21.15.2:0/956551709 shutdown_connections 2024-09-10T08:16:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.904+0000 7f3d95800640 1 --2- 172.21.15.2:0/956551709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d90109140 0x7f3d90110cc0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.904+0000 7f3d95800640 1 --2- 172.21.15.2:0/956551709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d901049c0 0x7f3d90108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.904+0000 7f3d95800640 1 --2- 172.21.15.2:0/956551709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d90104010 0x7f3d901043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.904+0000 7f3d95800640 1 -- 172.21.15.2:0/956551709 >> 172.21.15.2:0/956551709 conn(0x7f3d900ffa50 msgr2=0x7f3d90101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:55.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.904+0000 7f3d95800640 1 -- 172.21.15.2:0/956551709 shutdown_connections 2024-09-10T08:16:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.904+0000 7f3d95800640 1 -- 172.21.15.2:0/956551709 wait complete. 2024-09-10T08:16:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.905+0000 7f3d95800640 1 Processor -- start 2024-09-10T08:16:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.905+0000 7f3d95800640 1 -- start start 2024-09-10T08:16:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.905+0000 7f3d95800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d90104010 0x7f3d9019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.905+0000 7f3d95800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d901049c0 0x7f3d9019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.905+0000 7f3d95800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d90109140 0x7f3d9019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.905+0000 7f3d95800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901141c0 con 0x7f3d90109140 2024-09-10T08:16:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.905+0000 7f3d95800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90114040 con 0x7f3d90104010 2024-09-10T08:16:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.905+0000 7f3d95800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90114340 con 0x7f3d901049c0 2024-09-10T08:16:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.906+0000 7f3d8ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d90104010 0x7f3d9019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.906+0000 7f3d8ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d90104010 0x7f3d9019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44192/0 (socket says 172.21.15.2:44192) 2024-09-10T08:16:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.906+0000 7f3d8ea00640 1 -- 172.21.15.2:0/2117241524 learned_addr learned my addr 172.21.15.2:0/2117241524 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.906+0000 7f3d87e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d901049c0 0x7f3d9019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.906+0000 7f3d8f400640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d90109140 0x7f3d9019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:55 smithi155 ceph-mon[35405]: pgmap v650: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-10T08:16:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.906+0000 7f3d87e00640 1 -- 172.21.15.2:0/2117241524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d90104010 msgr2=0x7f3d9019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.906+0000 7f3d87e00640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d90104010 0x7f3d9019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.906+0000 7f3d87e00640 1 -- 172.21.15.2:0/2117241524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d90109140 msgr2=0x7f3d9019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.906+0000 7f3d87e00640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d90109140 0x7f3d9019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.906+0000 7f3d87e00640 1 -- 172.21.15.2:0/2117241524 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9019f670 con 0x7f3d901049c0 2024-09-10T08:16:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.906+0000 7f3d8ea00640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d90104010 0x7f3d9019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.907+0000 7f3d87e00640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d901049c0 0x7f3d9019ac00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3d78005fd0 tx=0x7f3d7800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.907+0000 7f3d87400640 1 -- 172.21.15.2:0/2117241524 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78009c30 con 0x7f3d901049c0 2024-09-10T08:16:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.907+0000 7f3d95800640 1 -- 172.21.15.2:0/2117241524 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9019f960 con 0x7f3d901049c0 2024-09-10T08:16:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.907+0000 7f3d87400640 1 -- 172.21.15.2:0/2117241524 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d78009dd0 con 0x7f3d901049c0 2024-09-10T08:16:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.907+0000 7f3d87400640 1 -- 172.21.15.2:0/2117241524 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78010460 con 0x7f3d901049c0 2024-09-10T08:16:55.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.907+0000 7f3d95800640 1 -- 172.21.15.2:0/2117241524 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d901a7240 con 0x7f3d901049c0 2024-09-10T08:16:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.910+0000 7f3d982da740 1 Processor -- start 2024-09-10T08:16:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.910+0000 7f3d982da740 1 -- start start 2024-09-10T08:16:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.910+0000 7f3d87400640 1 -- 172.21.15.2:0/2117241524 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3d78003ef0 con 0x7f3d901049c0 2024-09-10T08:16:55.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.910+0000 7f3d982da740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57aac0 0x55766f57aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.910+0000 7f3d982da740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57b3e0 0x55766f585760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d982da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f585ca0 0x55766f588090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d982da740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55766f4e1290 con 0x55766f57b3e0 2024-09-10T08:16:55.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d982da740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55766f488b40 con 0x55766f57aac0 2024-09-10T08:16:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d982da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55766f4dd250 con 0x55766f585ca0 2024-09-10T08:16:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d8ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57b3e0 0x55766f585760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d8f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57aac0 0x55766f57aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d8ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57b3e0 0x55766f585760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50478/0 (socket says 172.21.15.2:50478) 2024-09-10T08:16:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d8ea00640 1 -- 172.21.15.2:0/320658691 learned_addr learned my addr 172.21.15.2:0/320658691 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d87e00640 1 --2- 172.21.15.2:0/320658691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f585ca0 0x55766f588090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d8f400640 1 -- 172.21.15.2:0/320658691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f585ca0 msgr2=0x55766f588090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d8f400640 1 --2- 172.21.15.2:0/320658691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f585ca0 0x55766f588090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d8f400640 1 -- 172.21.15.2:0/320658691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57b3e0 msgr2=0x55766f585760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d8f400640 1 --2- 172.21.15.2:0/320658691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57b3e0 0x55766f585760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d8f400640 1 -- 172.21.15.2:0/320658691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55766f2526a0 con 0x55766f57aac0 2024-09-10T08:16:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.911+0000 7f3d8ea00640 1 --2- 172.21.15.2:0/320658691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57b3e0 0x55766f585760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.912+0000 7f3d8f400640 1 --2- 172.21.15.2:0/320658691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57aac0 0x55766f57aea0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005e30 tx=0x7f3d8802f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b29acef7d8ab8015 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.912+0000 7f3d87400640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3d740a2b80 0x7f3d740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.912+0000 7f3d87400640 1 -- 172.21.15.2:0/2117241524 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3d780cef20 con 0x7f3d901049c0 2024-09-10T08:16:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.912+0000 7f3d7fe00640 1 -- 172.21.15.2:0/320658691 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88003970 con 0x55766f57aac0 2024-09-10T08:16:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.912+0000 7f3d8ea00640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3d740a2b80 0x7f3d740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.912+0000 7f3d7fe00640 1 -- 172.21.15.2:0/320658691 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d88036030 con 0x55766f57aac0 2024-09-10T08:16:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.912+0000 7f3d7fe00640 1 -- 172.21.15.2:0/320658691 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88004330 con 0x55766f57aac0 2024-09-10T08:16:55.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.913+0000 7f3d982da740 1 -- 172.21.15.2:0/320658691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57aac0 msgr2=0x55766f57aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:55.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.913+0000 7f3d982da740 1 --2- 172.21.15.2:0/320658691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57aac0 0x55766f57aea0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005e30 tx=0x7f3d8802f3c0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.913+0000 7f3d8ea00640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3d740a2b80 0x7f3d740a5040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009fd0 tx=0x7f3d80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:55.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.913+0000 7f3d982da740 1 -- 172.21.15.2:0/320658691 shutdown_connections 2024-09-10T08:16:55.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.913+0000 7f3d982da740 1 --2- 172.21.15.2:0/320658691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f585ca0 0x55766f588090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.913+0000 7f3d982da740 1 --2- 172.21.15.2:0/320658691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57b3e0 0x55766f585760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.913+0000 7f3d982da740 1 --2- 172.21.15.2:0/320658691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57aac0 0x55766f57aea0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.913+0000 7f3d982da740 1 -- 172.21.15.2:0/320658691 >> 172.21.15.2:0/320658691 conn(0x55766f577050 msgr2=0x55766f577430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:55.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.913+0000 7f3d982da740 1 -- 172.21.15.2:0/320658691 shutdown_connections 2024-09-10T08:16:55.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.913+0000 7f3d982da740 1 -- 172.21.15.2:0/320658691 wait complete. 2024-09-10T08:16:55.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.914+0000 7f3d982da740 1 -- start start 2024-09-10T08:16:55.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.914+0000 7f3d982da740 1 Processor -- start 2024-09-10T08:16:55.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.914+0000 7f3d982da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57aac0 0x55766f585690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.914+0000 7f3d982da740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57b3e0 0x55766f582940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.914+0000 7f3d982da740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f585ca0 0x55766f582f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.914+0000 7f3d982da740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55766f588790 con 0x55766f585ca0 2024-09-10T08:16:55.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.914+0000 7f3d982da740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55766f49bfa0 con 0x55766f57b3e0 2024-09-10T08:16:55.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.914+0000 7f3d982da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55766f588910 con 0x55766f57aac0 2024-09-10T08:16:55.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.914+0000 7f3d982da740 10 client.0 _ll_register_callbacks cb 0x55766f1efba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:55.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.915+0000 7f3d8ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57aac0 0x55766f585690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.915+0000 7f3d87e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57b3e0 0x55766f582940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.915+0000 7f3d8ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57aac0 0x55766f585690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35582/0 (socket says 172.21.15.2:35582) 2024-09-10T08:16:55.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.915+0000 7f3d8ea00640 1 -- 172.21.15.2:0/35394747 learned_addr learned my addr 172.21.15.2:0/35394747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:55.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.915+0000 7f3d8f400640 1 --2- 172.21.15.2:0/35394747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f585ca0 0x55766f582f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.915+0000 7f3d87e00640 1 -- 172.21.15.2:0/35394747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57aac0 msgr2=0x55766f585690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:55.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.915+0000 7f3d87e00640 1 --2- 172.21.15.2:0/35394747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57aac0 0x55766f585690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.915+0000 7f3d87e00640 1 -- 172.21.15.2:0/35394747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f585ca0 msgr2=0x55766f582f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:55.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.915+0000 7f3d87e00640 1 --2- 172.21.15.2:0/35394747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f585ca0 0x55766f582f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.915+0000 7f3d87e00640 1 -- 172.21.15.2:0/35394747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55766f583760 con 0x55766f57b3e0 2024-09-10T08:16:55.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.915+0000 7f3d87e00640 1 --2- 172.21.15.2:0/35394747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57b3e0 0x55766f582940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55766f5783d0 tx=0x7f3d78003950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e36c8a7bc840f59d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:55.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.915+0000 7f3d7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:55.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.916+0000 7f3d7cc00640 1 -- 172.21.15.2:0/35394747 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78020070 con 0x55766f57b3e0 2024-09-10T08:16:55.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.916+0000 7f3d7cc00640 1 -- 172.21.15.2:0/35394747 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3d78097a50 con 0x55766f57b3e0 2024-09-10T08:16:55.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.916+0000 7f3d982da740 1 -- 172.21.15.2:0/35394747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d88036030 con 0x55766f57b3e0 2024-09-10T08:16:55.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.916+0000 7f3d7cc00640 1 -- 172.21.15.2:0/35394747 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3d7809ad40 con 0x55766f57b3e0 2024-09-10T08:16:55.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.916+0000 7f3d982da740 10 client.27908 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:55.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.916+0000 7f3d982da740 1 -- 172.21.15.2:0/35394747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55766f594800 con 0x55766f57b3e0 2024-09-10T08:16:55.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.916+0000 7f3d7cc00640 1 -- 172.21.15.2:0/35394747 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d88036030 con 0x55766f57b3e0 2024-09-10T08:16:55.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.916+0000 7f3d7cc00640 1 -- 172.21.15.2:0/35394747 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3d7809a830 con 0x55766f57b3e0 2024-09-10T08:16:55.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.916+0000 7f3d982da740 10 client.27908 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:55.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.917+0000 7f3d982da740 10 client.27908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:55.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.917+0000 7f3d982da740 20 client.27908 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:55.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.917+0000 7f3d982da740 1 --2- 172.21.15.2:0/35394747 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55766f595cc0 0x55766f598100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.917+0000 7f3d982da740 4 client.27908 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:55.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.917+0000 7f3d982da740 1 -- 172.21.15.2:0/35394747 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55766f4ee800 con 0x55766f595cc0 2024-09-10T08:16:55.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.917+0000 7f3d8ea00640 1 --2- 172.21.15.2:0/35394747 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55766f595cc0 0x55766f598100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.921 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:55 smithi107 ceph-mon[35048]: pgmap v650: 129 pgs: 129 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-10T08:16:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.917+0000 7f3d8ea00640 1 --2- 172.21.15.2:0/35394747 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55766f595cc0 0x55766f598100 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3619baecce6e35 server_cookie=8db2fa500bd5a804 in_seq=0 out_seq=0 2024-09-10T08:16:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.917+0000 7f3d7cc00640 10 client.27908 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.918+0000 7f3d7cc00640 1 -- 172.21.15.2:0/35394747 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55766f4ee800 con 0x55766f595cc0 2024-09-10T08:16:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.918+0000 7f3d7cc00640 10 client.27908 handle_command_reply: tid=0 2024-09-10T08:16:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.918+0000 7f3d982da740 1 client.27908 shutdown 2024-09-10T08:16:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.918+0000 7f3d982da740 1 -- 172.21.15.2:0/35394747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57b3e0 msgr2=0x55766f582940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.918+0000 7f3d982da740 1 --2- 172.21.15.2:0/35394747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57b3e0 0x55766f582940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55766f5783d0 tx=0x7f3d78003950 comp rx=0 tx=0).stop 2024-09-10T08:16:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.918+0000 7f3d982da740 1 -- 172.21.15.2:0/35394747 shutdown_connections 2024-09-10T08:16:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.918+0000 7f3d982da740 1 --2- 172.21.15.2:0/35394747 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55766f595cc0 0x55766f598100 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.918+0000 7f3d982da740 1 --2- 172.21.15.2:0/35394747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f585ca0 0x55766f582f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.918+0000 7f3d982da740 1 --2- 172.21.15.2:0/35394747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57b3e0 0x55766f582940 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.918+0000 7f3d7cc00640 0 client.27908 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:55.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.919+0000 7f3d982da740 1 --2- 172.21.15.2:0/35394747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57aac0 0x55766f585690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.919+0000 7f3d982da740 1 -- 172.21.15.2:0/35394747 >> 172.21.15.2:0/35394747 conn(0x55766f577050 msgr2=0x55766f57cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:55.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.919+0000 7f3d982da740 1 -- 172.21.15.2:0/35394747 shutdown_connections 2024-09-10T08:16:55.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.919+0000 7f3d982da740 1 -- 172.21.15.2:0/35394747 wait complete. 2024-09-10T08:16:55.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.919+0000 7f3d982da740 20 client.27908 trim_cache size 0 max 16384 2024-09-10T08:16:55.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.937+0000 7f3d982da740 1 Processor -- start 2024-09-10T08:16:55.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.937+0000 7f3d982da740 1 -- start start 2024-09-10T08:16:55.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.937+0000 7f3d982da740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57aac0 0x55766f5848b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.937+0000 7f3d982da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57b3e0 0x55766f59d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.937+0000 7f3d982da740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f585ca0 0x55766f59df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.937+0000 7f3d982da740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901b38a0 con 0x55766f57aac0 2024-09-10T08:16:55.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.937+0000 7f3d982da740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901b1170 con 0x55766f585ca0 2024-09-10T08:16:55.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.937+0000 7f3d982da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901aefc0 con 0x55766f57b3e0 2024-09-10T08:16:55.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.937+0000 7f3d8ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57aac0 0x55766f5848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.937+0000 7f3d8ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57aac0 0x55766f5848b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50502/0 (socket says 172.21.15.2:50502) 2024-09-10T08:16:55.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.937+0000 7f3d8ea00640 1 -- 172.21.15.2:0/4106947651 learned_addr learned my addr 172.21.15.2:0/4106947651 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:55.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.938+0000 7f3d87e00640 1 --2- 172.21.15.2:0/4106947651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57b3e0 0x55766f59d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.938+0000 7f3d8f400640 1 --2- 172.21.15.2:0/4106947651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f585ca0 0x55766f59df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.938+0000 7f3d8ea00640 1 -- 172.21.15.2:0/4106947651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57b3e0 msgr2=0x55766f59d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:55.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.938+0000 7f3d8ea00640 1 --2- 172.21.15.2:0/4106947651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57b3e0 0x55766f59d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.938+0000 7f3d8ea00640 1 -- 172.21.15.2:0/4106947651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f585ca0 msgr2=0x55766f59df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:55.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.938+0000 7f3d8ea00640 1 --2- 172.21.15.2:0/4106947651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f585ca0 0x55766f59df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.938+0000 7f3d8ea00640 1 -- 172.21.15.2:0/4106947651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55766f57a840 con 0x55766f57aac0 2024-09-10T08:16:55.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.938+0000 7f3d8ea00640 1 --2- 172.21.15.2:0/4106947651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57aac0 0x55766f5848b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55766f598970 tx=0x7f3d8001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30734ca701ba6dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:55.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.939+0000 7f3d7f400640 1 -- 172.21.15.2:0/4106947651 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80005020 con 0x55766f57aac0 2024-09-10T08:16:55.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.939+0000 7f3d7f400640 1 -- 172.21.15.2:0/4106947651 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d800047a0 con 0x55766f57aac0 2024-09-10T08:16:55.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.939+0000 7f3d7f400640 1 -- 172.21.15.2:0/4106947651 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8002b710 con 0x55766f57aac0 2024-09-10T08:16:55.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.939+0000 7f3d982da740 1 -- 172.21.15.2:0/4106947651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57aac0 msgr2=0x55766f5848b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:55.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.939+0000 7f3d982da740 1 --2- 172.21.15.2:0/4106947651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57aac0 0x55766f5848b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55766f598970 tx=0x7f3d8001a000 comp rx=0 tx=0).stop 2024-09-10T08:16:55.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.939+0000 7f3d982da740 1 -- 172.21.15.2:0/4106947651 shutdown_connections 2024-09-10T08:16:55.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.939+0000 7f3d982da740 1 --2- 172.21.15.2:0/4106947651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f585ca0 0x55766f59df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.939+0000 7f3d982da740 1 --2- 172.21.15.2:0/4106947651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57b3e0 0x55766f59d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.939+0000 7f3d982da740 1 --2- 172.21.15.2:0/4106947651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f57aac0 0x55766f5848b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.939+0000 7f3d982da740 1 -- 172.21.15.2:0/4106947651 >> 172.21.15.2:0/4106947651 conn(0x55766f577050 msgr2=0x55766f5841a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:55.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.939+0000 7f3d982da740 1 -- 172.21.15.2:0/4106947651 shutdown_connections 2024-09-10T08:16:55.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.939+0000 7f3d982da740 1 -- 172.21.15.2:0/4106947651 wait complete. 2024-09-10T08:16:55.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.940+0000 7f3d982da740 1 -- start start 2024-09-10T08:16:55.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.940+0000 7f3d982da740 1 Processor -- start 2024-09-10T08:16:55.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.940+0000 7f3d982da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57aac0 0x55766f58a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.940+0000 7f3d982da740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57b3e0 0x55766f58a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.940+0000 7f3d982da740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f585ca0 0x55766f5ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.940+0000 7f3d982da740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901af630 con 0x55766f585ca0 2024-09-10T08:16:55.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.940+0000 7f3d982da740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901b0aa0 con 0x55766f57b3e0 2024-09-10T08:16:55.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.940+0000 7f3d982da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901afd00 con 0x55766f57aac0 2024-09-10T08:16:55.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.941+0000 7f3d982da740 10 client.0 _ll_register_callbacks cb 0x7f3d78095f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:55.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.941+0000 7f3d8ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57aac0 0x55766f58a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.941+0000 7f3d8f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f585ca0 0x55766f5ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.941+0000 7f3d8ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57aac0 0x55766f58a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35610/0 (socket says 172.21.15.2:35610) 2024-09-10T08:16:55.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.941+0000 7f3d8ea00640 1 -- 172.21.15.2:0/1729097814 learned_addr learned my addr 172.21.15.2:0/1729097814 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:55.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.941+0000 7f3d87e00640 1 --2- 172.21.15.2:0/1729097814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57b3e0 0x55766f58a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.941+0000 7f3d8ea00640 1 -- 172.21.15.2:0/1729097814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57b3e0 msgr2=0x55766f58a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:55.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.941+0000 7f3d8ea00640 1 --2- 172.21.15.2:0/1729097814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57b3e0 0x55766f58a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.941+0000 7f3d8ea00640 1 -- 172.21.15.2:0/1729097814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f585ca0 msgr2=0x55766f5ff9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:55.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.941+0000 7f3d8ea00640 1 --2- 172.21.15.2:0/1729097814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f585ca0 0x55766f5ff9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.941+0000 7f3d8ea00640 1 -- 172.21.15.2:0/1729097814 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55766f5fffe0 con 0x55766f57aac0 2024-09-10T08:16:55.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.941+0000 7f3d87e00640 1 --2- 172.21.15.2:0/1729097814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57b3e0 0x55766f58a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:55.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.942+0000 7f3d8ea00640 1 --2- 172.21.15.2:0/1729097814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57aac0 0x55766f58a0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3d80004700 tx=0x7f3d80004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baa344327f1a5abb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:55.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.942+0000 7f3d7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:16:55.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.942+0000 7f3d7cc00640 1 -- 172.21.15.2:0/1729097814 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8002b750 con 0x55766f57aac0 2024-09-10T08:16:55.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.942+0000 7f3d982da740 1 -- 172.21.15.2:0/1729097814 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d800047a0 con 0x55766f57aac0 2024-09-10T08:16:55.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.942+0000 7f3d7cc00640 1 -- 172.21.15.2:0/1729097814 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3d8002b8f0 con 0x55766f57aac0 2024-09-10T08:16:55.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.942+0000 7f3d7cc00640 1 -- 172.21.15.2:0/1729097814 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3d8002baf0 con 0x55766f57aac0 2024-09-10T08:16:55.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.942+0000 7f3d982da740 10 client.27850 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:55.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.942+0000 7f3d982da740 1 -- 172.21.15.2:0/1729097814 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55766f601300 con 0x55766f57aac0 2024-09-10T08:16:55.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.943+0000 7f3d7cc00640 1 -- 172.21.15.2:0/1729097814 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d800047a0 con 0x55766f57aac0 2024-09-10T08:16:55.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.943+0000 7f3d7cc00640 1 -- 172.21.15.2:0/1729097814 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3d8001f5e0 con 0x55766f57aac0 2024-09-10T08:16:55.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.943+0000 7f3d982da740 10 client.27850 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:55.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.943+0000 7f3d982da740 10 client.27850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:55.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.943+0000 7f3d982da740 20 client.27850 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:55.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.943+0000 7f3d982da740 1 --2- 172.21.15.2:0/1729097814 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55766f6019d0 0x55766f603df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:55.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.943+0000 7f3d982da740 4 client.27850 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:16:55.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.943+0000 7f3d982da740 1 -- 172.21.15.2:0/1729097814 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55766f4e8d10 con 0x55766f6019d0 2024-09-10T08:16:55.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.944+0000 7f3d87e00640 1 --2- 172.21.15.2:0/1729097814 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55766f6019d0 0x55766f603df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:55.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.944+0000 7f3d87e00640 1 --2- 172.21.15.2:0/1729097814 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55766f6019d0 0x55766f603df0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bc5efd9d0f44128 server_cookie=94e6d8d52bb1cf4 in_seq=0 out_seq=0 2024-09-10T08:16:55.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.944+0000 7f3d7cc00640 10 client.27850 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:55.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.945+0000 7f3d7cc00640 1 -- 172.21.15.2:0/1729097814 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55766f4e8d10 con 0x55766f6019d0 2024-09-10T08:16:55.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.945+0000 7f3d7cc00640 10 client.27850 handle_command_reply: tid=0 2024-09-10T08:16:55.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.945+0000 7f3d982da740 1 client.27850 shutdown 2024-09-10T08:16:55.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.945+0000 7f3d982da740 1 -- 172.21.15.2:0/1729097814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57aac0 msgr2=0x55766f58a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:55.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.945+0000 7f3d982da740 1 --2- 172.21.15.2:0/1729097814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57aac0 0x55766f58a0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3d80004700 tx=0x7f3d80004730 comp rx=0 tx=0).stop 2024-09-10T08:16:55.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d982da740 1 -- 172.21.15.2:0/1729097814 shutdown_connections 2024-09-10T08:16:55.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d982da740 1 --2- 172.21.15.2:0/1729097814 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55766f6019d0 0x55766f603df0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d982da740 1 --2- 172.21.15.2:0/1729097814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55766f585ca0 0x55766f5ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d982da740 1 --2- 172.21.15.2:0/1729097814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55766f57b3e0 0x55766f58a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d982da740 1 --2- 172.21.15.2:0/1729097814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55766f57aac0 0x55766f58a0d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d7cc00640 0 client.27850 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:55.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d982da740 1 -- 172.21.15.2:0/1729097814 >> 172.21.15.2:0/1729097814 conn(0x55766f577050 msgr2=0x55766f5849c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:55.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d982da740 1 -- 172.21.15.2:0/1729097814 shutdown_connections 2024-09-10T08:16:55.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d982da740 1 -- 172.21.15.2:0/1729097814 wait complete. 2024-09-10T08:16:55.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d982da740 20 client.27850 trim_cache size 0 max 16384 2024-09-10T08:16:55.948 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:16:55.949 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:16:55.949 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "3807ae70-77d8-49bd-98f3-a6c0432f8ef3", 2024-09-10T08:16:55.949 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:16:55.949 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:16:55.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d95800640 1 -- 172.21.15.2:0/2117241524 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3d740a2b80 msgr2=0x7f3d740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:55.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d95800640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3d740a2b80 0x7f3d740a5040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009fd0 tx=0x7f3d80005f90 comp rx=0 tx=0).stop 2024-09-10T08:16:55.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d95800640 1 -- 172.21.15.2:0/2117241524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d901049c0 msgr2=0x7f3d9019ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:55.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.946+0000 7f3d95800640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d901049c0 0x7f3d9019ac00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3d78005fd0 tx=0x7f3d7800dde0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.947+0000 7f3d95800640 1 -- 172.21.15.2:0/2117241524 shutdown_connections 2024-09-10T08:16:55.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.947+0000 7f3d95800640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d90109140 0x7f3d9019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.947+0000 7f3d95800640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3d740a2b80 0x7f3d740a5040 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.947+0000 7f3d95800640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d901049c0 0x7f3d9019ac00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.947+0000 7f3d95800640 1 --2- 172.21.15.2:0/2117241524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d90104010 0x7f3d9019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:55.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.947+0000 7f3d95800640 1 -- 172.21.15.2:0/2117241524 >> 172.21.15.2:0/2117241524 conn(0x7f3d900ffa50 msgr2=0x7f3d90100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:55.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.947+0000 7f3d95800640 1 -- 172.21.15.2:0/2117241524 shutdown_connections 2024-09-10T08:16:55.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:55.947+0000 7f3d95800640 1 -- 172.21.15.2:0/2117241524 wait complete. 2024-09-10T08:16:56.713 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:16:57.092 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:16:57.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:57 smithi002 ceph-mon[30563]: pgmap v651: 129 pgs: 129 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-09-10T08:16:57.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:57 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:16:57.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:57 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:16:57.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:57 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:16:57.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:57 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:16:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.749+0000 7fca43600640 1 -- 172.21.15.2:0/2261761419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca3c076720 msgr2=0x7fca3c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.749+0000 7fca43600640 1 --2- 172.21.15.2:0/2261761419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca3c076720 0x7fca3c076b00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fca30009900 tx=0x7fca3002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.749+0000 7fca43600640 1 -- 172.21.15.2:0/2261761419 shutdown_connections 2024-09-10T08:16:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.749+0000 7fca43600640 1 --2- 172.21.15.2:0/2261761419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca3c112a30 0x7fca3c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.749+0000 7fca43600640 1 --2- 172.21.15.2:0/2261761419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca3c077040 0x7fca3c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.749+0000 7fca43600640 1 --2- 172.21.15.2:0/2261761419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca3c076720 0x7fca3c076b00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.749+0000 7fca43600640 1 -- 172.21.15.2:0/2261761419 >> 172.21.15.2:0/2261761419 conn(0x7fca3c0ffa90 msgr2=0x7fca3c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.750+0000 7fca43600640 1 -- 172.21.15.2:0/2261761419 shutdown_connections 2024-09-10T08:16:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.750+0000 7fca43600640 1 -- 172.21.15.2:0/2261761419 wait complete. 2024-09-10T08:16:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.751+0000 7fca43600640 1 Processor -- start 2024-09-10T08:16:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.751+0000 7fca43600640 1 -- start start 2024-09-10T08:16:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.751+0000 7fca43600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca3c076720 0x7fca3c1a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.751+0000 7fca43600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca3c077040 0x7fca3c1a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.751+0000 7fca43600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca3c112a30 0x7fca3c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.751+0000 7fca43600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1185c0 con 0x7fca3c077040 2024-09-10T08:16:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.751+0000 7fca43600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c118440 con 0x7fca3c112a30 2024-09-10T08:16:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.751+0000 7fca43600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c118740 con 0x7fca3c076720 2024-09-10T08:16:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.751+0000 7fca3be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca3c077040 0x7fca3c1a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.752+0000 7fca41600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca3c112a30 0x7fca3c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.752+0000 7fca3be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca3c077040 0x7fca3c1a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58098/0 (socket says 172.21.15.2:58098) 2024-09-10T08:16:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.752+0000 7fca41600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca3c112a30 0x7fca3c19e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44174/0 (socket says 172.21.15.2:44174) 2024-09-10T08:16:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.752+0000 7fca3be00640 1 -- 172.21.15.2:0/917655039 learned_addr learned my addr 172.21.15.2:0/917655039 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.752+0000 7fca40c00640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca3c076720 0x7fca3c1a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.752+0000 7fca41600640 1 -- 172.21.15.2:0/917655039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca3c076720 msgr2=0x7fca3c1a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.752+0000 7fca41600640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca3c076720 0x7fca3c1a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.752+0000 7fca41600640 1 -- 172.21.15.2:0/917655039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca3c077040 msgr2=0x7fca3c1a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.752+0000 7fca41600640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca3c077040 0x7fca3c1a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.752+0000 7fca41600640 1 -- 172.21.15.2:0/917655039 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c19efe0 con 0x7fca3c112a30 2024-09-10T08:16:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.752+0000 7fca3be00640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca3c077040 0x7fca3c1a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.752+0000 7fca41600640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca3c112a30 0x7fca3c19e8b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fca3c07aa10 tx=0x7fca3400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.752+0000 7fca39600640 1 -- 172.21.15.2:0/917655039 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3400cdf0 con 0x7fca3c112a30 2024-09-10T08:16:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.753+0000 7fca39600640 1 -- 172.21.15.2:0/917655039 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca34005020 con 0x7fca3c112a30 2024-09-10T08:16:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.753+0000 7fca43600640 1 -- 172.21.15.2:0/917655039 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3c19f2d0 con 0x7fca3c112a30 2024-09-10T08:16:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.753+0000 7fca39600640 1 -- 172.21.15.2:0/917655039 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34010670 con 0x7fca3c112a30 2024-09-10T08:16:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.753+0000 7fca43600640 1 -- 172.21.15.2:0/917655039 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3c1ab5c0 con 0x7fca3c112a30 2024-09-10T08:16:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.756+0000 7fca4607e740 1 Processor -- start 2024-09-10T08:16:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.756+0000 7fca39600640 1 -- 172.21.15.2:0/917655039 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fca340054d0 con 0x7fca3c112a30 2024-09-10T08:16:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.756+0000 7fca4607e740 1 -- start start 2024-09-10T08:16:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.756+0000 7fca4607e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1028b90 0x5631c1028f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.756+0000 7fca4607e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c10294b0 0x5631c1033830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.756+0000 7fca4607e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c1033d70 0x5631c1036160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.756+0000 7fca4607e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5631c0f8f3e0 con 0x5631c10294b0 2024-09-10T08:16:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.756+0000 7fca4607e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5631c0f36d80 con 0x5631c1033d70 2024-09-10T08:16:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.756+0000 7fca4607e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5631c0f8b3a0 con 0x5631c1028b90 2024-09-10T08:16:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.756+0000 7fca40c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c10294b0 0x5631c1033830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.756+0000 7fca3be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1028b90 0x5631c1028f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.757+0000 7fca3be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1028b90 0x5631c1028f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56908/0 (socket says 172.21.15.2:56908) 2024-09-10T08:16:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.757+0000 7fca3be00640 1 -- 172.21.15.2:0/4085753486 learned_addr learned my addr 172.21.15.2:0/4085753486 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.757+0000 7fca40c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c10294b0 0x5631c1033830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58112/0 (socket says 172.21.15.2:58112) 2024-09-10T08:16:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.757+0000 7fca3be00640 1 --2- 172.21.15.2:0/4085753486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c1033d70 0x5631c1036160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.757+0000 7fca40c00640 1 -- 172.21.15.2:0/4085753486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1028b90 msgr2=0x5631c1028f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.757+0000 7fca40c00640 1 --2- 172.21.15.2:0/4085753486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1028b90 0x5631c1028f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.757+0000 7fca40c00640 1 -- 172.21.15.2:0/4085753486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c1033d70 msgr2=0x5631c1036160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.757+0000 7fca40c00640 1 --2- 172.21.15.2:0/4085753486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c1033d70 0x5631c1036160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.757+0000 7fca40c00640 1 -- 172.21.15.2:0/4085753486 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631c0f55ae0 con 0x5631c10294b0 2024-09-10T08:16:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.757+0000 7fca3be00640 1 --2- 172.21.15.2:0/4085753486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1028b90 0x5631c1028f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.757+0000 7fca39600640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca240a2c00 0x7fca240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.757+0000 7fca39600640 1 -- 172.21.15.2:0/917655039 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fca340cfa40 con 0x7fca3c112a30 2024-09-10T08:16:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.758+0000 7fca40c00640 1 --2- 172.21.15.2:0/4085753486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c10294b0 0x5631c1033830 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fca30005940 tx=0x7fca30005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc47c8f9a8bcefe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.758+0000 7fca3be00640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca240a2c00 0x7fca240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.758+0000 7fca2dc00640 1 -- 172.21.15.2:0/4085753486 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30040070 con 0x5631c10294b0 2024-09-10T08:16:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.758+0000 7fca2dc00640 1 -- 172.21.15.2:0/4085753486 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca30005500 con 0x5631c10294b0 2024-09-10T08:16:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.758+0000 7fca2dc00640 1 -- 172.21.15.2:0/4085753486 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30035040 con 0x5631c10294b0 2024-09-10T08:16:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.759+0000 7fca4607e740 1 -- 172.21.15.2:0/4085753486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c10294b0 msgr2=0x5631c1033830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.759+0000 7fca3be00640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca240a2c00 0x7fca240a50c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x5631c10264a0 tx=0x7fca280046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.759+0000 7fca4607e740 1 --2- 172.21.15.2:0/4085753486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c10294b0 0x5631c1033830 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fca30005940 tx=0x7fca30005be0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.759+0000 7fca4607e740 1 -- 172.21.15.2:0/4085753486 shutdown_connections 2024-09-10T08:16:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.759+0000 7fca4607e740 1 --2- 172.21.15.2:0/4085753486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c1033d70 0x5631c1036160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.759+0000 7fca4607e740 1 --2- 172.21.15.2:0/4085753486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c10294b0 0x5631c1033830 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.759+0000 7fca4607e740 1 --2- 172.21.15.2:0/4085753486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1028b90 0x5631c1028f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.759+0000 7fca4607e740 1 -- 172.21.15.2:0/4085753486 >> 172.21.15.2:0/4085753486 conn(0x5631c1025120 msgr2=0x5631c1025500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:57.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.759+0000 7fca4607e740 1 -- 172.21.15.2:0/4085753486 shutdown_connections 2024-09-10T08:16:57.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.759+0000 7fca4607e740 1 -- 172.21.15.2:0/4085753486 wait complete. 2024-09-10T08:16:57.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.760+0000 7fca4607e740 1 -- start start 2024-09-10T08:16:57.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.760+0000 7fca4607e740 1 Processor -- start 2024-09-10T08:16:57.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.760+0000 7fca4607e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c1028b90 0x5631c1030c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.760+0000 7fca4607e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c10294b0 0x5631c1031190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.760+0000 7fca4607e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1033d70 0x5631c10316d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.760+0000 7fca4607e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5631c10369e0 con 0x5631c10294b0 2024-09-10T08:16:57.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.760+0000 7fca4607e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5631c0f4a0f0 con 0x5631c1028b90 2024-09-10T08:16:57.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.760+0000 7fca4607e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5631c1036b60 con 0x5631c1033d70 2024-09-10T08:16:57.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.760+0000 7fca4607e740 10 client.0 _ll_register_callbacks cb 0x5631c0f3b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:57.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.761+0000 7fca40c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c1028b90 0x5631c1030c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.761+0000 7fca40c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c1028b90 0x5631c1030c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44198/0 (socket says 172.21.15.2:44198) 2024-09-10T08:16:57.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.761+0000 7fca40c00640 1 -- 172.21.15.2:0/281849313 learned_addr learned my addr 172.21.15.2:0/281849313 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:57.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.761+0000 7fca3be00640 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c10294b0 0x5631c1031190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.761+0000 7fca41600640 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1033d70 0x5631c10316d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.761+0000 7fca40c00640 1 -- 172.21.15.2:0/281849313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1033d70 msgr2=0x5631c10316d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:57.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.761+0000 7fca40c00640 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1033d70 0x5631c10316d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.761+0000 7fca40c00640 1 -- 172.21.15.2:0/281849313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c10294b0 msgr2=0x5631c1031190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:57.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.761+0000 7fca40c00640 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c10294b0 0x5631c1031190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.761+0000 7fca40c00640 1 -- 172.21.15.2:0/281849313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631c0d003c0 con 0x5631c1028b90 2024-09-10T08:16:57.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.761+0000 7fca41600640 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1033d70 0x5631c10316d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:16:57.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.761+0000 7fca40c00640 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c1028b90 0x5631c1030c50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fca3002a910 tx=0x7fca3002aaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2832f035afd77e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:57.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.761+0000 7fca22000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:16:57.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.762+0000 7fca22000640 1 -- 172.21.15.2:0/281849313 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30040070 con 0x5631c1028b90 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.762+0000 7fca22000640 1 -- 172.21.15.2:0/281849313 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fca30031880 con 0x5631c1028b90 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.762+0000 7fca4607e740 1 -- 172.21.15.2:0/281849313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca30005500 con 0x5631c1028b90 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.762+0000 7fca22000640 1 -- 172.21.15.2:0/281849313 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca30035040 con 0x5631c1028b90 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.762+0000 7fca4607e740 10 client.27926 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.762+0000 7fca4607e740 1 -- 172.21.15.2:0/281849313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5631c1042690 con 0x5631c1028b90 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.762+0000 7fca22000640 1 -- 172.21.15.2:0/281849313 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca30005500 con 0x5631c1028b90 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.763+0000 7fca22000640 1 -- 172.21.15.2:0/281849313 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fca300035f0 con 0x5631c1028b90 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.763+0000 7fca4607e740 10 client.27926 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.763+0000 7fca4607e740 10 client.27926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.763+0000 7fca4607e740 20 client.27926 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.763+0000 7fca4607e740 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5631c10439e0 0x5631c1045dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.763+0000 7fca4607e740 4 client.27926 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.763+0000 7fca4607e740 1 -- 172.21.15.2:0/281849313 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631c0f9c950 con 0x5631c10439e0 2024-09-10T08:16:57.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.763+0000 7fca3be00640 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5631c10439e0 0x5631c1045dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.764+0000 7fca3be00640 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5631c10439e0 0x5631c1045dd0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7aef1b9de477b7e server_cookie=902480f4d88d0538 in_seq=0 out_seq=0 2024-09-10T08:16:57.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.764+0000 7fca22000640 10 client.27926 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:57.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.764+0000 7fca22000640 1 -- 172.21.15.2:0/281849313 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631c0f9c950 con 0x5631c10439e0 2024-09-10T08:16:57.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.764+0000 7fca22000640 10 client.27926 handle_command_reply: tid=0 2024-09-10T08:16:57.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.764+0000 7fca4607e740 1 client.27926 shutdown 2024-09-10T08:16:57.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.765+0000 7fca4607e740 1 -- 172.21.15.2:0/281849313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c1028b90 msgr2=0x5631c1030c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:57.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.765+0000 7fca4607e740 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c1028b90 0x5631c1030c50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fca3002a910 tx=0x7fca3002aaf0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.765+0000 7fca4607e740 1 -- 172.21.15.2:0/281849313 shutdown_connections 2024-09-10T08:16:57.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.765+0000 7fca4607e740 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5631c10439e0 0x5631c1045dd0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.765+0000 7fca4607e740 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1033d70 0x5631c10316d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.765+0000 7fca4607e740 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c10294b0 0x5631c1031190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.765+0000 7fca4607e740 1 --2- 172.21.15.2:0/281849313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c1028b90 0x5631c1030c50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.765+0000 7fca22000640 0 client.27926 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:57.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.765+0000 7fca4607e740 1 -- 172.21.15.2:0/281849313 >> 172.21.15.2:0/281849313 conn(0x5631c1025120 msgr2=0x5631c102ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:57.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.765+0000 7fca4607e740 1 -- 172.21.15.2:0/281849313 shutdown_connections 2024-09-10T08:16:57.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.765+0000 7fca4607e740 1 -- 172.21.15.2:0/281849313 wait complete. 2024-09-10T08:16:57.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.765+0000 7fca4607e740 20 client.27926 trim_cache size 0 max 16384 2024-09-10T08:16:57.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.783+0000 7fca4607e740 1 Processor -- start 2024-09-10T08:16:57.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.783+0000 7fca4607e740 1 -- start start 2024-09-10T08:16:57.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.783+0000 7fca4607e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1028b90 0x5631c104b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.783+0000 7fca4607e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c10294b0 0x5631c104b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.783+0000 7fca4607e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1033d70 0x5631c102ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.783+0000 7fca4607e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b7cf0 con 0x5631c1033d70 2024-09-10T08:16:57.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.783+0000 7fca4607e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b55c0 con 0x5631c10294b0 2024-09-10T08:16:57.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.783+0000 7fca4607e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b3410 con 0x5631c1028b90 2024-09-10T08:16:57.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.783+0000 7fca41600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1033d70 0x5631c102ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca41600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1033d70 0x5631c102ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58132/0 (socket says 172.21.15.2:58132) 2024-09-10T08:16:57.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca41600640 1 -- 172.21.15.2:0/2586017957 learned_addr learned my addr 172.21.15.2:0/2586017957 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:57.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca40c00640 1 --2- 172.21.15.2:0/2586017957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1028b90 0x5631c104b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca3be00640 1 --2- 172.21.15.2:0/2586017957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c10294b0 0x5631c104b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca41600640 1 -- 172.21.15.2:0/2586017957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1028b90 msgr2=0x5631c104b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:57.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca41600640 1 --2- 172.21.15.2:0/2586017957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1028b90 0x5631c104b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca41600640 1 -- 172.21.15.2:0/2586017957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c10294b0 msgr2=0x5631c104b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:57.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca41600640 1 --2- 172.21.15.2:0/2586017957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c10294b0 0x5631c104b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca41600640 1 -- 172.21.15.2:0/2586017957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631c10292e0 con 0x5631c1033d70 2024-09-10T08:16:57.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca3be00640 1 --2- 172.21.15.2:0/2586017957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c10294b0 0x5631c104b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:16:57.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca41600640 1 --2- 172.21.15.2:0/2586017957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1033d70 0x5631c102ded0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fca340987d0 tx=0x7fca340ccfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de557bea832245d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:57.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca2d200640 1 -- 172.21.15.2:0/2586017957 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca340a4ac0 con 0x5631c1033d70 2024-09-10T08:16:57.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca2d200640 1 -- 172.21.15.2:0/2586017957 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca3409e070 con 0x5631c1033d70 2024-09-10T08:16:57.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.784+0000 7fca2d200640 1 -- 172.21.15.2:0/2586017957 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3409b500 con 0x5631c1033d70 2024-09-10T08:16:57.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.785+0000 7fca4607e740 1 -- 172.21.15.2:0/2586017957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1033d70 msgr2=0x5631c102ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:57.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.785+0000 7fca4607e740 1 --2- 172.21.15.2:0/2586017957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1033d70 0x5631c102ded0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fca340987d0 tx=0x7fca340ccfc0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.785+0000 7fca4607e740 1 -- 172.21.15.2:0/2586017957 shutdown_connections 2024-09-10T08:16:57.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.785+0000 7fca4607e740 1 --2- 172.21.15.2:0/2586017957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1033d70 0x5631c102ded0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.785+0000 7fca4607e740 1 --2- 172.21.15.2:0/2586017957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c10294b0 0x5631c104b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.785+0000 7fca4607e740 1 --2- 172.21.15.2:0/2586017957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1028b90 0x5631c104b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.785+0000 7fca4607e740 1 -- 172.21.15.2:0/2586017957 >> 172.21.15.2:0/2586017957 conn(0x5631c1025120 msgr2=0x5631c102ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:57.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.785+0000 7fca4607e740 1 -- 172.21.15.2:0/2586017957 shutdown_connections 2024-09-10T08:16:57.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.785+0000 7fca4607e740 1 -- 172.21.15.2:0/2586017957 wait complete. 2024-09-10T08:16:57.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.785+0000 7fca4607e740 1 -- start start 2024-09-10T08:16:57.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.786+0000 7fca4607e740 1 Processor -- start 2024-09-10T08:16:57.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.786+0000 7fca4607e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1028b90 0x5631c103c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.786+0000 7fca4607e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c10294b0 0x5631c102da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.786+0000 7fca4607e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1033d70 0x5631c102e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.786+0000 7fca4607e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b3a80 con 0x5631c1028b90 2024-09-10T08:16:57.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.786+0000 7fca4607e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b4ef0 con 0x5631c10294b0 2024-09-10T08:16:57.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.786+0000 7fca4607e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b4150 con 0x5631c1033d70 2024-09-10T08:16:57.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.786+0000 7fca4607e740 10 client.0 _ll_register_callbacks cb 0x7fca30009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:16:57.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.786+0000 7fca40c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1028b90 0x5631c103c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.786+0000 7fca40c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1028b90 0x5631c103c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58138/0 (socket says 172.21.15.2:58138) 2024-09-10T08:16:57.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.786+0000 7fca40c00640 1 -- 172.21.15.2:0/48644966 learned_addr learned my addr 172.21.15.2:0/48644966 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:16:57.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.786+0000 7fca3be00640 1 --2- 172.21.15.2:0/48644966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c10294b0 0x5631c102da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.786+0000 7fca41600640 1 --2- 172.21.15.2:0/48644966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1033d70 0x5631c102e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.787+0000 7fca40c00640 1 -- 172.21.15.2:0/48644966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1033d70 msgr2=0x5631c102e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:57.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.787+0000 7fca40c00640 1 --2- 172.21.15.2:0/48644966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1033d70 0x5631c102e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.787+0000 7fca40c00640 1 -- 172.21.15.2:0/48644966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c10294b0 msgr2=0x5631c102da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:16:57.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.787+0000 7fca40c00640 1 --2- 172.21.15.2:0/48644966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c10294b0 0x5631c102da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.787+0000 7fca40c00640 1 -- 172.21.15.2:0/48644966 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631c102e870 con 0x5631c1028b90 2024-09-10T08:16:57.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.787+0000 7fca40c00640 1 --2- 172.21.15.2:0/48644966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1028b90 0x5631c103c5e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fca30001eb0 tx=0x7fca30031ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b1778adf7b154c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:16:57.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.787+0000 7fca22000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:16:57.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.787+0000 7fca22000640 1 -- 172.21.15.2:0/48644966 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30040050 con 0x5631c1028b90 2024-09-10T08:16:57.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.787+0000 7fca22000640 1 -- 172.21.15.2:0/48644966 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fca30003e90 con 0x5631c1028b90 2024-09-10T08:16:57.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.787+0000 7fca4607e740 1 -- 172.21.15.2:0/48644966 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca3409e070 con 0x5631c1028b90 2024-09-10T08:16:57.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.788+0000 7fca22000640 1 -- 172.21.15.2:0/48644966 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca30005850 con 0x5631c1028b90 2024-09-10T08:16:57.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.788+0000 7fca4607e740 10 client.17904 fetch_fsmap learned FSMap version 26 2024-09-10T08:16:57.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.788+0000 7fca4607e740 1 -- 172.21.15.2:0/48644966 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5631c10ae460 con 0x5631c1028b90 2024-09-10T08:16:57.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.788+0000 7fca22000640 1 -- 172.21.15.2:0/48644966 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca3409e070 con 0x5631c1028b90 2024-09-10T08:16:57.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.788+0000 7fca22000640 1 -- 172.21.15.2:0/48644966 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fca30059be0 con 0x5631c1028b90 2024-09-10T08:16:57.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.788+0000 7fca4607e740 10 client.17904 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:16:57.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.788+0000 7fca4607e740 10 client.17904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:16:57.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.788+0000 7fca4607e740 20 client.17904 populate_metadata read hostname 'smithi002' 2024-09-10T08:16:57.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.788+0000 7fca4607e740 1 --2- 172.21.15.2:0/48644966 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5631c10ae9e0 0x5631c10b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:16:57.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.788+0000 7fca4607e740 4 client.17904 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:16:57.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.788+0000 7fca4607e740 1 -- 172.21.15.2:0/48644966 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5631c0f96e60 con 0x5631c10ae9e0 2024-09-10T08:16:57.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.788+0000 7fca3be00640 1 --2- 172.21.15.2:0/48644966 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5631c10ae9e0 0x5631c10b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:16:57.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.789+0000 7fca3be00640 1 --2- 172.21.15.2:0/48644966 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5631c10ae9e0 0x5631c10b0dd0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383d54f2a433a43b server_cookie=8426003125177442 in_seq=0 out_seq=0 2024-09-10T08:16:57.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.789+0000 7fca22000640 10 client.17904 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:57.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.789+0000 7fca22000640 1 -- 172.21.15.2:0/48644966 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5631c0f96e60 con 0x5631c10ae9e0 2024-09-10T08:16:57.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.789+0000 7fca22000640 10 client.17904 handle_command_reply: tid=0 2024-09-10T08:16:57.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.789+0000 7fca4607e740 1 client.17904 shutdown 2024-09-10T08:16:57.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.789+0000 7fca4607e740 1 -- 172.21.15.2:0/48644966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1028b90 msgr2=0x5631c103c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:57.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.789+0000 7fca4607e740 1 --2- 172.21.15.2:0/48644966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1028b90 0x5631c103c5e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fca30001eb0 tx=0x7fca30031ab0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.789+0000 7fca4607e740 1 -- 172.21.15.2:0/48644966 shutdown_connections 2024-09-10T08:16:57.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.789+0000 7fca4607e740 1 --2- 172.21.15.2:0/48644966 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5631c10ae9e0 0x5631c10b0dd0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.789+0000 7fca22000640 0 client.17904 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:16:57.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.789+0000 7fca4607e740 1 --2- 172.21.15.2:0/48644966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631c1033d70 0x5631c102e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.789+0000 7fca4607e740 1 --2- 172.21.15.2:0/48644966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631c10294b0 0x5631c102da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.789+0000 7fca4607e740 1 --2- 172.21.15.2:0/48644966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5631c1028b90 0x5631c103c5e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.790+0000 7fca4607e740 1 -- 172.21.15.2:0/48644966 >> 172.21.15.2:0/48644966 conn(0x5631c1025120 msgr2=0x5631c10239a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:57.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.790+0000 7fca4607e740 1 -- 172.21.15.2:0/48644966 shutdown_connections 2024-09-10T08:16:57.812 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:16:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:16:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:16:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "3807ae70-77d8-49bd-98f3-a6c0432f8ef3": { 2024-09-10T08:16:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:16:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "3807ae70-77d8-49bd-98f3-a6c0432f8ef3", 2024-09-10T08:16:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:16:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:16:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:16:57.813 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:16:57.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.790+0000 7fca4607e740 1 -- 172.21.15.2:0/48644966 wait complete. 2024-09-10T08:16:57.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.790+0000 7fca4607e740 20 client.17904 trim_cache size 0 max 16384 2024-09-10T08:16:57.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.790+0000 7fca43600640 1 -- 172.21.15.2:0/917655039 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca240a2c00 msgr2=0x7fca240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:57.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.790+0000 7fca43600640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca240a2c00 0x7fca240a50c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x5631c10264a0 tx=0x7fca280046c0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.790+0000 7fca43600640 1 -- 172.21.15.2:0/917655039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca3c112a30 msgr2=0x7fca3c19e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:16:57.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.790+0000 7fca43600640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca3c112a30 0x7fca3c19e8b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fca3c07aa10 tx=0x7fca3400ef10 comp rx=0 tx=0).stop 2024-09-10T08:16:57.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.791+0000 7fca43600640 1 -- 172.21.15.2:0/917655039 shutdown_connections 2024-09-10T08:16:57.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.791+0000 7fca43600640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca3c112a30 0x7fca3c19e8b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.791+0000 7fca43600640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca3c077040 0x7fca3c1a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.791+0000 7fca43600640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fca240a2c00 0x7fca240a50c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.791+0000 7fca43600640 1 --2- 172.21.15.2:0/917655039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca3c076720 0x7fca3c1a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:16:57.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.791+0000 7fca43600640 1 -- 172.21.15.2:0/917655039 >> 172.21.15.2:0/917655039 conn(0x7fca3c0ffa90 msgr2=0x7fca3c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:16:57.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.791+0000 7fca43600640 1 -- 172.21.15.2:0/917655039 shutdown_connections 2024-09-10T08:16:57.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:16:57.791+0000 7fca43600640 1 -- 172.21.15.2:0/917655039 wait complete. 2024-09-10T08:16:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:57 smithi155 ceph-mon[35405]: pgmap v651: 129 pgs: 129 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-09-10T08:16:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:57 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:16:57.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:57 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:16:57.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:57 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:16:57.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:57 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:16:57.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:57 smithi107 ceph-mon[35048]: pgmap v651: 129 pgs: 129 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-09-10T08:16:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:57 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:16:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:57 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:16:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:57 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:16:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:57 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:16:58.598 INFO:tasks.cephfs.filesystem:scrub status for tag:3807ae70-77d8-49bd-98f3-a6c0432f8ef3 - {'path': '/', 'tag': '3807ae70-77d8-49bd-98f3-a6c0432f8ef3', 'options': 'recursive,force'} 2024-09-10T08:16:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:59 smithi155 ceph-mon[35405]: pgmap v652: 129 pgs: 129 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-09-10T08:16:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:16:59 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:16:59.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:59 smithi107 ceph-mon[35048]: pgmap v652: 129 pgs: 129 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-09-10T08:16:59.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:16:59 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:17:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:59 smithi002 ceph-mon[30563]: pgmap v652: 129 pgs: 129 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-09-10T08:17:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:16:59 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:17:01.835 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.835 INFO:tasks.workunit.client.0.smithi002.stdout:251/600: open fd = 3 2024-09-10T08:17:01.835 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.835 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.835 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.836 INFO:tasks.workunit.client.0.smithi002.stdout:252/600: open fd = 3 2024-09-10T08:17:01.836 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.836 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.836 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.836 INFO:tasks.workunit.client.0.smithi002.stdout:253/600: open fd = 3 2024-09-10T08:17:01.836 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.836 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.836 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.836 INFO:tasks.workunit.client.0.smithi002.stdout:254/600: open fd = 3 2024-09-10T08:17:01.836 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.836 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.836 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.837 INFO:tasks.workunit.client.0.smithi002.stdout:255/600: open fd = 3 2024-09-10T08:17:01.837 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.837 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.837 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.837 INFO:tasks.workunit.client.0.smithi002.stdout:256/600: open fd = 3 2024-09-10T08:17:01.837 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.837 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.837 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.837 INFO:tasks.workunit.client.0.smithi002.stdout:257/600: open fd = 3 2024-09-10T08:17:01.837 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.837 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.837 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.838 INFO:tasks.workunit.client.0.smithi002.stdout:258/600: open fd = 3 2024-09-10T08:17:01.838 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.838 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.838 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.838 INFO:tasks.workunit.client.0.smithi002.stdout:259/600: open fd = 3 2024-09-10T08:17:01.838 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.838 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.838 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.838 INFO:tasks.workunit.client.0.smithi002.stdout:260/600: open fd = 3 2024-09-10T08:17:01.838 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.838 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.838 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.838 INFO:tasks.workunit.client.0.smithi002.stdout:261/600: open fd = 3 2024-09-10T08:17:01.839 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.839 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.839 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.839 INFO:tasks.workunit.client.0.smithi002.stdout:262/600: open fd = 3 2024-09-10T08:17:01.839 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.839 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.839 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.839 INFO:tasks.workunit.client.0.smithi002.stdout:263/600: open fd = 3 2024-09-10T08:17:01.841 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.841 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.841 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.842 INFO:tasks.workunit.client.0.smithi002.stdout:264/600: open fd = 3 2024-09-10T08:17:01.842 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.842 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.842 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.842 INFO:tasks.workunit.client.0.smithi002.stdout:265/600: open fd = 3 2024-09-10T08:17:01.842 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.842 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.842 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.842 INFO:tasks.workunit.client.0.smithi002.stdout:266/600: open fd = 3 2024-09-10T08:17:01.842 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.842 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.842 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.842 INFO:tasks.workunit.client.0.smithi002.stdout:267/600: open fd = 3 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:268/600: open fd = 3 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:269/600: open fd = 3 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:270/600: open fd = 3 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.844 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.844 INFO:tasks.workunit.client.0.smithi002.stdout:271/600: open fd = 3 2024-09-10T08:17:01.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.844 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.844 INFO:tasks.workunit.client.0.smithi002.stdout:272/600: open fd = 3 2024-09-10T08:17:01.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.844 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.844 INFO:tasks.workunit.client.0.smithi002.stdout:273/600: open fd = 3 2024-09-10T08:17:01.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.845 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.845 INFO:tasks.workunit.client.0.smithi002.stdout:274/600: open fd = 3 2024-09-10T08:17:01.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.845 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.845 INFO:tasks.workunit.client.0.smithi002.stdout:275/600: open fd = 3 2024-09-10T08:17:01.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.845 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.845 INFO:tasks.workunit.client.0.smithi002.stdout:276/600: open fd = 3 2024-09-10T08:17:01.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.845 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.846 INFO:tasks.workunit.client.0.smithi002.stdout:277/600: open fd = 3 2024-09-10T08:17:01.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.846 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.846 INFO:tasks.workunit.client.0.smithi002.stdout:278/600: open fd = 3 2024-09-10T08:17:01.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.846 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.846 INFO:tasks.workunit.client.0.smithi002.stdout:279/600: open fd = 3 2024-09-10T08:17:01.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.846 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.846 INFO:tasks.workunit.client.0.smithi002.stdout:280/600: open fd = 3 2024-09-10T08:17:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:281/600: open fd = 3 2024-09-10T08:17:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:282/600: open fd = 3 2024-09-10T08:17:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:283/600: open fd = 3 2024-09-10T08:17:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.848 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.848 INFO:tasks.workunit.client.0.smithi002.stdout:284/600: open fd = 3 2024-09-10T08:17:01.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.848 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.848 INFO:tasks.workunit.client.0.smithi002.stdout:285/600: open fd = 3 2024-09-10T08:17:01.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.848 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.848 INFO:tasks.workunit.client.0.smithi002.stdout:286/600: open fd = 3 2024-09-10T08:17:01.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:287/600: open fd = 3 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:288/600: open fd = 3 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:289/600: open fd = 3 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.849 INFO:tasks.workunit.client.0.smithi002.stdout:290/600: open fd = 3 2024-09-10T08:17:01.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.850 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.850 INFO:tasks.workunit.client.0.smithi002.stdout:291/600: open fd = 3 2024-09-10T08:17:01.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.850 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.850 INFO:tasks.workunit.client.0.smithi002.stdout:292/600: open fd = 3 2024-09-10T08:17:01.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.850 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.850 INFO:tasks.workunit.client.0.smithi002.stdout:293/600: open fd = 3 2024-09-10T08:17:01.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.851 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.851 INFO:tasks.workunit.client.0.smithi002.stdout:294/600: open fd = 3 2024-09-10T08:17:01.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.851 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.851 INFO:tasks.workunit.client.0.smithi002.stdout:295/600: open fd = 3 2024-09-10T08:17:01.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.851 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.851 INFO:tasks.workunit.client.0.smithi002.stdout:296/600: open fd = 3 2024-09-10T08:17:01.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.852 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.852 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.852 INFO:tasks.workunit.client.0.smithi002.stdout:297/600: open fd = 3 2024-09-10T08:17:01.852 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.852 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.852 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.852 INFO:tasks.workunit.client.0.smithi002.stdout:298/600: open fd = 3 2024-09-10T08:17:01.852 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.853 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.853 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.853 INFO:tasks.workunit.client.0.smithi002.stdout:299/600: open fd = 3 2024-09-10T08:17:01.853 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.853 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.853 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.853 INFO:tasks.workunit.client.0.smithi002.stdout:300/600: open fd = 3 2024-09-10T08:17:01.853 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.853 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.853 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.853 INFO:tasks.workunit.client.0.smithi002.stdout:301/600: open fd = 3 2024-09-10T08:17:01.853 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.853 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.854 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.854 INFO:tasks.workunit.client.0.smithi002.stdout:302/600: open fd = 3 2024-09-10T08:17:01.854 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.854 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.854 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.854 INFO:tasks.workunit.client.0.smithi002.stdout:303/600: open fd = 3 2024-09-10T08:17:01.854 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.854 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.854 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.854 INFO:tasks.workunit.client.0.smithi002.stdout:304/600: open fd = 3 2024-09-10T08:17:01.854 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.854 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.854 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.855 INFO:tasks.workunit.client.0.smithi002.stdout:305/600: open fd = 3 2024-09-10T08:17:01.855 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.855 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.855 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.855 INFO:tasks.workunit.client.0.smithi002.stdout:306/600: open fd = 3 2024-09-10T08:17:01.855 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.855 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.855 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.855 INFO:tasks.workunit.client.0.smithi002.stdout:307/600: open fd = 3 2024-09-10T08:17:01.855 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.855 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.855 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.855 INFO:tasks.workunit.client.0.smithi002.stdout:308/600: open fd = 3 2024-09-10T08:17:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:309/600: open fd = 3 2024-09-10T08:17:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:310/600: open fd = 3 2024-09-10T08:17:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:311/600: open fd = 3 2024-09-10T08:17:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.857 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.857 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:17:01.857 INFO:tasks.workunit.client.0.smithi002.stdout:312/600: open fd = 3 2024-09-10T08:17:01.857 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.857 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:17:01.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:01 smithi155 ceph-mon[35405]: pgmap v653: 129 pgs: 129 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-10T08:17:01.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:01 smithi107 ceph-mon[35048]: pgmap v653: 129 pgs: 129 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-10T08:17:02.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:01 smithi002 ceph-mon[30563]: pgmap v653: 129 pgs: 129 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-10T08:17:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.884+0000 7f5834200640 1 -- 172.21.15.2:0/3125166537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c076bb0 msgr2=0x7f582c076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.884+0000 7f5834200640 1 --2- 172.21.15.2:0/3125166537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c076bb0 0x7f582c076fb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5820009900 tx=0x7f582002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.885+0000 7f5834200640 1 -- 172.21.15.2:0/3125166537 shutdown_connections 2024-09-10T08:17:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.885+0000 7f5834200640 1 --2- 172.21.15.2:0/3125166537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f582c078790 0x7f582c079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.885+0000 7f5834200640 1 --2- 172.21.15.2:0/3125166537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f582c077df0 0x7f582c078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.885+0000 7f5834200640 1 --2- 172.21.15.2:0/3125166537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c076bb0 0x7f582c076fb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.885+0000 7f5834200640 1 -- 172.21.15.2:0/3125166537 >> 172.21.15.2:0/3125166537 conn(0x7f582c0ff4b0 msgr2=0x7f582c1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.885+0000 7f5834200640 1 -- 172.21.15.2:0/3125166537 shutdown_connections 2024-09-10T08:17:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.885+0000 7f5834200640 1 -- 172.21.15.2:0/3125166537 wait complete. 2024-09-10T08:17:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.886+0000 7f5834200640 1 Processor -- start 2024-09-10T08:17:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.886+0000 7f5834200640 1 -- start start 2024-09-10T08:17:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.886+0000 7f5834200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c076bb0 0x7f582c19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.886+0000 7f5834200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f582c077df0 0x7f582c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.886+0000 7f5834200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f582c078790 0x7f582c1a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.886+0000 7f5834200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f582c113e30 con 0x7f582c078790 2024-09-10T08:17:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.886+0000 7f5834200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f582c113cb0 con 0x7f582c076bb0 2024-09-10T08:17:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.886+0000 7f5834200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f582c113fb0 con 0x7f582c077df0 2024-09-10T08:17:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.887+0000 7f5831800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c076bb0 0x7f582c19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.887+0000 7f5831800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c076bb0 0x7f582c19e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44230/0 (socket says 172.21.15.2:44230) 2024-09-10T08:17:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.887+0000 7f5831800640 1 -- 172.21.15.2:0/3370237857 learned_addr learned my addr 172.21.15.2:0/3370237857 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.887+0000 7f5832200640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f582c078790 0x7f582c1a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.887+0000 7f5830e00640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f582c077df0 0x7f582c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.887+0000 7f5830e00640 1 -- 172.21.15.2:0/3370237857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c076bb0 msgr2=0x7f582c19e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.887+0000 7f5830e00640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c076bb0 0x7f582c19e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.887+0000 7f5830e00640 1 -- 172.21.15.2:0/3370237857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f582c078790 msgr2=0x7f582c1a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.887+0000 7f5830e00640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f582c078790 0x7f582c1a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.887+0000 7f5830e00640 1 -- 172.21.15.2:0/3370237857 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582c1a5ff0 con 0x7f582c077df0 2024-09-10T08:17:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.887+0000 7f5831800640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c076bb0 0x7f582c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.887+0000 7f5830e00640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f582c077df0 0x7f582c19e910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f581800e9f0 tx=0x7f581800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.888+0000 7f582a000640 1 -- 172.21.15.2:0/3370237857 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581800ccc0 con 0x7f582c077df0 2024-09-10T08:17:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.888+0000 7f582a000640 1 -- 172.21.15.2:0/3370237857 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5818005020 con 0x7f582c077df0 2024-09-10T08:17:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.888+0000 7f5834200640 1 -- 172.21.15.2:0/3370237857 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582c1a62e0 con 0x7f582c077df0 2024-09-10T08:17:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.888+0000 7f582a000640 1 -- 172.21.15.2:0/3370237857 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5818004a00 con 0x7f582c077df0 2024-09-10T08:17:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.888+0000 7f5834200640 1 -- 172.21.15.2:0/3370237857 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f582c1a6830 con 0x7f582c077df0 2024-09-10T08:17:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.891+0000 7f58371c7740 1 Processor -- start 2024-09-10T08:17:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.891+0000 7f58371c7740 1 -- start start 2024-09-10T08:17:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.891+0000 7f58371c7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da03090 0x55d25da03490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.891+0000 7f58371c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da08490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.891+0000 7f58371c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da08a60 0x55d25da0af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.891+0000 7f58371c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d25d968100 con 0x55d25da08a60 2024-09-10T08:17:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.891+0000 7f58371c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d25d9101b0 con 0x55d25da039d0 2024-09-10T08:17:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.891+0000 7f58371c7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d25d9640c0 con 0x55d25da03090 2024-09-10T08:17:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.892+0000 7f582a000640 1 -- 172.21.15.2:0/3370237857 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f58180051d0 con 0x7f582c077df0 2024-09-10T08:17:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.892+0000 7f5832200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da03090 0x55d25da03490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.892+0000 7f5832200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da03090 0x55d25da03490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56960/0 (socket says 172.21.15.2:56960) 2024-09-10T08:17:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.892+0000 7f5831800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da08490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.892+0000 7f5832200640 1 -- 172.21.15.2:0/2413463417 learned_addr learned my addr 172.21.15.2:0/2413463417 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.892+0000 7f5830e00640 1 --2- 172.21.15.2:0/2413463417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da08a60 0x55d25da0af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.892+0000 7f5832200640 1 -- 172.21.15.2:0/2413463417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 msgr2=0x55d25da08490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.892+0000 7f5832200640 1 --2- 172.21.15.2:0/2413463417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da08490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.892+0000 7f5832200640 1 -- 172.21.15.2:0/2413463417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da08a60 msgr2=0x55d25da0af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.892+0000 7f5832200640 1 --2- 172.21.15.2:0/2413463417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da08a60 0x55d25da0af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.892+0000 7f5832200640 1 -- 172.21.15.2:0/2413463417 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25d67b210 con 0x55d25da03090 2024-09-10T08:17:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.893+0000 7f5832200640 1 --2- 172.21.15.2:0/2413463417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da03090 0x55d25da03490 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5824011440 tx=0x7f5824011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b20205437d49d3cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.893+0000 7f581ea00640 1 -- 172.21.15.2:0/2413463417 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824003c30 con 0x55d25da03090 2024-09-10T08:17:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.893+0000 7f581ea00640 1 -- 172.21.15.2:0/2413463417 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5824003dd0 con 0x55d25da03090 2024-09-10T08:17:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.893+0000 7f581ea00640 1 -- 172.21.15.2:0/2413463417 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582400fac0 con 0x55d25da03090 2024-09-10T08:17:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.893+0000 7f582a000640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f58140a2cb0 0x7f58140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.893+0000 7f582a000640 1 -- 172.21.15.2:0/3370237857 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f58180cf2f0 con 0x7f582c077df0 2024-09-10T08:17:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.893+0000 7f5831800640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f58140a2cb0 0x7f58140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.894+0000 7f58371c7740 1 -- 172.21.15.2:0/2413463417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da03090 msgr2=0x55d25da03490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.894+0000 7f58371c7740 1 --2- 172.21.15.2:0/2413463417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da03090 0x55d25da03490 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5824011440 tx=0x7f5824011900 comp rx=0 tx=0).stop 2024-09-10T08:17:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.894+0000 7f58371c7740 1 -- 172.21.15.2:0/2413463417 shutdown_connections 2024-09-10T08:17:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.894+0000 7f58371c7740 1 --2- 172.21.15.2:0/2413463417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da08a60 0x55d25da0af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.894+0000 7f58371c7740 1 --2- 172.21.15.2:0/2413463417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da08490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.894+0000 7f58371c7740 1 --2- 172.21.15.2:0/2413463417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da03090 0x55d25da03490 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.894+0000 7f58371c7740 1 -- 172.21.15.2:0/2413463417 >> 172.21.15.2:0/2413463417 conn(0x55d25d9fe700 msgr2=0x55d25d9feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.894+0000 7f58371c7740 1 -- 172.21.15.2:0/2413463417 shutdown_connections 2024-09-10T08:17:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.894+0000 7f58371c7740 1 -- 172.21.15.2:0/2413463417 wait complete. 2024-09-10T08:17:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.894+0000 7f5831800640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f58140a2cb0 0x7f58140a5170 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55d25da04400 tx=0x7f58200315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.895+0000 7f58371c7740 1 -- start start 2024-09-10T08:17:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.895+0000 7f58371c7740 1 Processor -- start 2024-09-10T08:17:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.895+0000 7f58371c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da03090 0x55d25da0d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.895+0000 7f58371c7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da039d0 0x55d25da0d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.895+0000 7f58371c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da08a60 0x55d25da14770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.895+0000 7f58371c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d25da0ba30 con 0x55d25da08a60 2024-09-10T08:17:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.895+0000 7f58371c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d25d8f1090 con 0x55d25da03090 2024-09-10T08:17:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.895+0000 7f58371c7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d25da0bbb0 con 0x55d25da039d0 2024-09-10T08:17:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.895+0000 7f58371c7740 10 client.0 _ll_register_callbacks cb 0x55d25d945f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.896+0000 7f5830e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da039d0 0x55d25da0d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.896+0000 7f5830e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da039d0 0x55d25da0d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56964/0 (socket says 172.21.15.2:56964) 2024-09-10T08:17:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.896+0000 7f5830e00640 1 -- 172.21.15.2:0/918384173 learned_addr learned my addr 172.21.15.2:0/918384173 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.896+0000 7f5832200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da08a60 0x55d25da14770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.896+0000 7f5831800640 1 --2- 172.21.15.2:0/918384173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da03090 0x55d25da0d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.896+0000 7f5830e00640 1 -- 172.21.15.2:0/918384173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da03090 msgr2=0x55d25da0d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.896+0000 7f5830e00640 1 --2- 172.21.15.2:0/918384173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da03090 0x55d25da0d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.896+0000 7f5830e00640 1 -- 172.21.15.2:0/918384173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da08a60 msgr2=0x55d25da14770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.896+0000 7f5830e00640 1 --2- 172.21.15.2:0/918384173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da08a60 0x55d25da14770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.896+0000 7f5830e00640 1 -- 172.21.15.2:0/918384173 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25d747620 con 0x55d25da039d0 2024-09-10T08:17:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.896+0000 7f5830e00640 1 --2- 172.21.15.2:0/918384173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da039d0 0x55d25da0d730 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f58180a3ea0 tx=0x7f581809b5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aff594e88832c663 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.896+0000 7f5813400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:17:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f5813400640 1 -- 172.21.15.2:0/918384173 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581801c070 con 0x55d25da039d0 2024-09-10T08:17:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f5813400640 1 -- 172.21.15.2:0/918384173 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f58180860c0 con 0x55d25da039d0 2024-09-10T08:17:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f58371c7740 1 -- 172.21.15.2:0/918384173 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5824003dd0 con 0x55d25da039d0 2024-09-10T08:17:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f5813400640 1 -- 172.21.15.2:0/918384173 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f581809b760 con 0x55d25da039d0 2024-09-10T08:17:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f58371c7740 10 client.27880 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f58371c7740 1 -- 172.21.15.2:0/918384173 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d25da17630 con 0x55d25da039d0 2024-09-10T08:17:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f5813400640 1 -- 172.21.15.2:0/918384173 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5824003dd0 con 0x55d25da039d0 2024-09-10T08:17:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f5813400640 1 -- 172.21.15.2:0/918384173 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f58180a3f40 con 0x55d25da039d0 2024-09-10T08:17:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f58371c7740 10 client.27880 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f58371c7740 10 client.27880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f58371c7740 20 client.27880 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f58371c7740 1 --2- 172.21.15.2:0/918384173 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d25da18d00 0x55d25da1b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f58371c7740 4 client.27880 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:17:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.897+0000 7f58371c7740 1 -- 172.21.15.2:0/918384173 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d25d975830 con 0x55d25da18d00 2024-09-10T08:17:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.898+0000 7f5831800640 1 --2- 172.21.15.2:0/918384173 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d25da18d00 0x55d25da1b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.898+0000 7f5831800640 1 --2- 172.21.15.2:0/918384173 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d25da18d00 0x55d25da1b0f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bed7c38d1372c05a server_cookie=5bb136fe305f442 in_seq=0 out_seq=0 2024-09-10T08:17:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.898+0000 7f5813400640 10 client.27880 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.899+0000 7f5813400640 1 -- 172.21.15.2:0/918384173 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d25d975830 con 0x55d25da18d00 2024-09-10T08:17:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.899+0000 7f5813400640 10 client.27880 handle_command_reply: tid=0 2024-09-10T08:17:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.899+0000 7f58371c7740 1 client.27880 shutdown 2024-09-10T08:17:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.900+0000 7f58371c7740 1 -- 172.21.15.2:0/918384173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da039d0 msgr2=0x55d25da0d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.900+0000 7f58371c7740 1 --2- 172.21.15.2:0/918384173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da039d0 0x55d25da0d730 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f58180a3ea0 tx=0x7f581809b5f0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.900+0000 7f58371c7740 1 -- 172.21.15.2:0/918384173 shutdown_connections 2024-09-10T08:17:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.900+0000 7f58371c7740 1 --2- 172.21.15.2:0/918384173 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d25da18d00 0x55d25da1b0f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.900+0000 7f58371c7740 1 --2- 172.21.15.2:0/918384173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da08a60 0x55d25da14770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.900+0000 7f58371c7740 1 --2- 172.21.15.2:0/918384173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da039d0 0x55d25da0d730 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.900+0000 7f58371c7740 1 --2- 172.21.15.2:0/918384173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da03090 0x55d25da0d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.900+0000 7f5813400640 0 client.27880 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.900+0000 7f58371c7740 1 -- 172.21.15.2:0/918384173 >> 172.21.15.2:0/918384173 conn(0x55d25d9fe700 msgr2=0x55d25d9fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.900+0000 7f58371c7740 1 -- 172.21.15.2:0/918384173 shutdown_connections 2024-09-10T08:17:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.900+0000 7f58371c7740 1 -- 172.21.15.2:0/918384173 wait complete. 2024-09-10T08:17:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.900+0000 7f58371c7740 20 client.27880 trim_cache size 0 max 16384 2024-09-10T08:17:03.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:03 smithi155 ceph-mon[35405]: pgmap v654: 129 pgs: 129 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-10T08:17:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.915+0000 7f58371c7740 1 Processor -- start 2024-09-10T08:17:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.915+0000 7f58371c7740 1 -- start start 2024-09-10T08:17:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f58371c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 0x55d25d9fd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f58371c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da14730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f58371c7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da08a60 0x55d25d9fd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f58371c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b3680 con 0x55d25da03090 2024-09-10T08:17:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f58371c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b0f50 con 0x55d25da039d0 2024-09-10T08:17:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f58371c7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1aeda0 con 0x55d25da08a60 2024-09-10T08:17:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f5831800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 0x55d25d9fd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f5831800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 0x55d25d9fd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58198/0 (socket says 172.21.15.2:58198) 2024-09-10T08:17:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f5831800640 1 -- 172.21.15.2:0/296430571 learned_addr learned my addr 172.21.15.2:0/296430571 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f5832200640 1 --2- 172.21.15.2:0/296430571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da08a60 0x55d25d9fd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f5830e00640 1 --2- 172.21.15.2:0/296430571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da14730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f5831800640 1 -- 172.21.15.2:0/296430571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da08a60 msgr2=0x55d25d9fd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f5831800640 1 --2- 172.21.15.2:0/296430571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da08a60 0x55d25d9fd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f5831800640 1 -- 172.21.15.2:0/296430571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 msgr2=0x55d25da14730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:03.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f5831800640 1 --2- 172.21.15.2:0/296430571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da14730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f5831800640 1 -- 172.21.15.2:0/296430571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25da03eb0 con 0x55d25da03090 2024-09-10T08:17:03.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f5832200640 1 --2- 172.21.15.2:0/296430571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da08a60 0x55d25d9fd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:03.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.916+0000 7f5830e00640 1 --2- 172.21.15.2:0/296430571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da14730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:03.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.917+0000 7f5831800640 1 --2- 172.21.15.2:0/296430571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 0x55d25d9fd060 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5820031440 tx=0x7f5820040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640b3b6f2aaaa9f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:03.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:03 smithi107 ceph-mon[35048]: pgmap v654: 129 pgs: 129 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-10T08:17:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.917+0000 7f581e000640 1 -- 172.21.15.2:0/296430571 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820035040 con 0x55d25da03090 2024-09-10T08:17:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.917+0000 7f581e000640 1 -- 172.21.15.2:0/296430571 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582003d450 con 0x55d25da03090 2024-09-10T08:17:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.917+0000 7f581e000640 1 -- 172.21.15.2:0/296430571 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820031e60 con 0x55d25da03090 2024-09-10T08:17:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.917+0000 7f58371c7740 1 -- 172.21.15.2:0/296430571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 msgr2=0x55d25d9fd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.917+0000 7f58371c7740 1 --2- 172.21.15.2:0/296430571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 0x55d25d9fd060 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5820031440 tx=0x7f5820040000 comp rx=0 tx=0).stop 2024-09-10T08:17:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.918+0000 7f58371c7740 1 -- 172.21.15.2:0/296430571 shutdown_connections 2024-09-10T08:17:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.918+0000 7f58371c7740 1 --2- 172.21.15.2:0/296430571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da08a60 0x55d25d9fd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.918+0000 7f58371c7740 1 --2- 172.21.15.2:0/296430571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da14730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.918+0000 7f58371c7740 1 --2- 172.21.15.2:0/296430571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 0x55d25d9fd060 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.918+0000 7f58371c7740 1 -- 172.21.15.2:0/296430571 >> 172.21.15.2:0/296430571 conn(0x55d25d9fe700 msgr2=0x55d25da0d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.918+0000 7f58371c7740 1 -- 172.21.15.2:0/296430571 shutdown_connections 2024-09-10T08:17:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.918+0000 7f58371c7740 1 -- 172.21.15.2:0/296430571 wait complete. 2024-09-10T08:17:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.918+0000 7f58371c7740 1 -- start start 2024-09-10T08:17:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.918+0000 7f58371c7740 1 Processor -- start 2024-09-10T08:17:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f58371c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 0x55d25da11f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f58371c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da12460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f58371c7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da08a60 0x55d25da133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f58371c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1af410 con 0x55d25da03090 2024-09-10T08:17:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f58371c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b0880 con 0x55d25da039d0 2024-09-10T08:17:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f58371c7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1afae0 con 0x55d25da08a60 2024-09-10T08:17:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f58371c7740 10 client.0 _ll_register_callbacks cb 0x7f581800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f5831800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 0x55d25da11f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f5831800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 0x55d25da11f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58212/0 (socket says 172.21.15.2:58212) 2024-09-10T08:17:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f5831800640 1 -- 172.21.15.2:0/3157247208 learned_addr learned my addr 172.21.15.2:0/3157247208 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f5830e00640 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da12460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f5832200640 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da08a60 0x55d25da133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f5831800640 1 -- 172.21.15.2:0/3157247208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da08a60 msgr2=0x55d25da133b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f5831800640 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da08a60 0x55d25da133b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f5831800640 1 -- 172.21.15.2:0/3157247208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 msgr2=0x55d25da12460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f5831800640 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da12460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f5831800640 1 -- 172.21.15.2:0/3157247208 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25da139c0 con 0x55d25da03090 2024-09-10T08:17:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f5832200640 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da08a60 0x55d25da133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:17:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.919+0000 7f5830e00640 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da12460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.920+0000 7f5831800640 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 0x55d25da11f20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5820009850 tx=0x7f582002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df5e8307ee7567f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.920+0000 7f5813400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:17:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.920+0000 7f5813400640 1 -- 172.21.15.2:0/3157247208 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820063070 con 0x55d25da03090 2024-09-10T08:17:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.920+0000 7f5813400640 1 -- 172.21.15.2:0/3157247208 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5820003bd0 con 0x55d25da03090 2024-09-10T08:17:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.920+0000 7f58371c7740 1 -- 172.21.15.2:0/3157247208 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582003d450 con 0x55d25da03090 2024-09-10T08:17:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.920+0000 7f5813400640 1 -- 172.21.15.2:0/3157247208 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5820003ea0 con 0x55d25da03090 2024-09-10T08:17:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.920+0000 7f58371c7740 10 client.17922 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.920+0000 7f58371c7740 1 -- 172.21.15.2:0/3157247208 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d25da83a30 con 0x55d25da03090 2024-09-10T08:17:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.920+0000 7f5813400640 1 -- 172.21.15.2:0/3157247208 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582003d450 con 0x55d25da03090 2024-09-10T08:17:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.921+0000 7f5813400640 1 -- 172.21.15.2:0/3157247208 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5820044070 con 0x55d25da03090 2024-09-10T08:17:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.921+0000 7f58371c7740 10 client.17922 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.921+0000 7f58371c7740 10 client.17922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.921+0000 7f58371c7740 20 client.17922 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.921+0000 7f58371c7740 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d25da843c0 0x55d25da867e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.921+0000 7f58371c7740 4 client.17922 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:17:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.921+0000 7f58371c7740 1 -- 172.21.15.2:0/3157247208 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d25d96fb80 con 0x55d25da843c0 2024-09-10T08:17:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.921+0000 7f5830e00640 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d25da843c0 0x55d25da867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.922+0000 7f5830e00640 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d25da843c0 0x55d25da867e0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5abfe2996c6d708e server_cookie=360741b554b999af in_seq=0 out_seq=0 2024-09-10T08:17:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.922+0000 7f5813400640 10 client.17922 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.940+0000 7f5813400640 1 -- 172.21.15.2:0/3157247208 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d25d96fb80 con 0x55d25da843c0 2024-09-10T08:17:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.940+0000 7f5813400640 10 client.17922 handle_command_reply: tid=0 2024-09-10T08:17:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.940+0000 7f58371c7740 1 client.17922 shutdown 2024-09-10T08:17:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.940+0000 7f58371c7740 1 -- 172.21.15.2:0/3157247208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 msgr2=0x55d25da11f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.940+0000 7f58371c7740 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 0x55d25da11f20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5820009850 tx=0x7f582002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.940+0000 7f58371c7740 1 -- 172.21.15.2:0/3157247208 shutdown_connections 2024-09-10T08:17:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.940+0000 7f58371c7740 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d25da843c0 0x55d25da867e0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.940+0000 7f58371c7740 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25da08a60 0x55d25da133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.940+0000 7f58371c7740 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d25da039d0 0x55d25da12460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.940+0000 7f58371c7740 1 --2- 172.21.15.2:0/3157247208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d25da03090 0x55d25da11f20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.940+0000 7f5813400640 0 client.17922 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.940+0000 7f58371c7740 1 -- 172.21.15.2:0/3157247208 >> 172.21.15.2:0/3157247208 conn(0x55d25d9fe700 msgr2=0x55d25d9ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.941+0000 7f58371c7740 1 -- 172.21.15.2:0/3157247208 shutdown_connections 2024-09-10T08:17:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.941+0000 7f58371c7740 1 -- 172.21.15.2:0/3157247208 wait complete. 2024-09-10T08:17:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.941+0000 7f58371c7740 20 client.17922 trim_cache size 0 max 16384 2024-09-10T08:17:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:17:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:17:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:17:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:17:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.941+0000 7f5834200640 1 -- 172.21.15.2:0/3370237857 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f58140a2cb0 msgr2=0x7f58140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.941+0000 7f5834200640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f58140a2cb0 0x7f58140a5170 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55d25da04400 tx=0x7f58200315b0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.941+0000 7f5834200640 1 -- 172.21.15.2:0/3370237857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f582c077df0 msgr2=0x7f582c19e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.941+0000 7f5834200640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f582c077df0 0x7f582c19e910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f581800e9f0 tx=0x7f581800edf0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.942+0000 7f5834200640 1 -- 172.21.15.2:0/3370237857 shutdown_connections 2024-09-10T08:17:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.942+0000 7f5834200640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f582c078790 0x7f582c1a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.942+0000 7f5834200640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f58140a2cb0 0x7f58140a5170 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.942+0000 7f5834200640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f582c077df0 0x7f582c19e910 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.942+0000 7f5834200640 1 --2- 172.21.15.2:0/3370237857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c076bb0 0x7f582c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.942+0000 7f5834200640 1 -- 172.21.15.2:0/3370237857 >> 172.21.15.2:0/3370237857 conn(0x7f582c0ff4b0 msgr2=0x7f582c100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.942+0000 7f5834200640 1 -- 172.21.15.2:0/3370237857 shutdown_connections 2024-09-10T08:17:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:03.942+0000 7f5834200640 1 -- 172.21.15.2:0/3370237857 wait complete. 2024-09-10T08:17:04.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:03 smithi002 ceph-mon[30563]: pgmap v654: 129 pgs: 129 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-10T08:17:05.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:05 smithi155 ceph-mon[35405]: pgmap v655: 129 pgs: 129 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:17:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:05 smithi107 ceph-mon[35048]: pgmap v655: 129 pgs: 129 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:17:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:05 smithi002 ceph-mon[30563]: pgmap v655: 129 pgs: 129 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:17:07.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:07 smithi155 ceph-mon[35405]: pgmap v656: 129 pgs: 129 active+clean; 594 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-10T08:17:07.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:07 smithi107 ceph-mon[35048]: pgmap v656: 129 pgs: 129 active+clean; 594 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-10T08:17:08.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:07 smithi002 ceph-mon[30563]: pgmap v656: 129 pgs: 129 active+clean; 594 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-10T08:17:09.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:09 smithi155 ceph-mon[35405]: pgmap v657: 129 pgs: 129 active+clean; 661 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-10T08:17:09.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:09 smithi107 ceph-mon[35048]: pgmap v657: 129 pgs: 129 active+clean; 661 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-10T08:17:10.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:09 smithi002 ceph-mon[30563]: pgmap v657: 129 pgs: 129 active+clean; 661 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-10T08:17:11.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:11 smithi155 ceph-mon[35405]: pgmap v658: 129 pgs: 129 active+clean; 661 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-10T08:17:11.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:11 smithi107 ceph-mon[35048]: pgmap v658: 129 pgs: 129 active+clean; 661 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-10T08:17:12.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:11 smithi002 ceph-mon[30563]: pgmap v658: 129 pgs: 129 active+clean; 661 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-10T08:17:13.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:13 smithi155 ceph-mon[35405]: pgmap v659: 129 pgs: 129 active+clean; 661 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-10T08:17:13.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:13 smithi107 ceph-mon[35048]: pgmap v659: 129 pgs: 129 active+clean; 661 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-10T08:17:14.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:13 smithi002 ceph-mon[30563]: pgmap v659: 129 pgs: 129 active+clean; 661 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-10T08:17:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:15 smithi155 ceph-mon[35405]: pgmap v660: 129 pgs: 129 active+clean; 718 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-10T08:17:15.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:15 smithi107 ceph-mon[35048]: pgmap v660: 129 pgs: 129 active+clean; 718 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-10T08:17:16.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:15 smithi002 ceph-mon[30563]: pgmap v660: 129 pgs: 129 active+clean; 718 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-10T08:17:17.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.070+0000 7f7435e00640 1 -- 172.21.15.2:0/3462849437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f743010e130 msgr2=0x7f7430110520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.070+0000 7f7435e00640 1 --2- 172.21.15.2:0/3462849437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f743010e130 0x7f7430110520 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7428009ca0 tx=0x7f742802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.070+0000 7f7435e00640 1 -- 172.21.15.2:0/3462849437 shutdown_connections 2024-09-10T08:17:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.070+0000 7f7435e00640 1 --2- 172.21.15.2:0/3462849437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f743010e130 0x7f7430110520 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.070+0000 7f7435e00640 1 --2- 172.21.15.2:0/3462849437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7430074c50 0x7f74300750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.070+0000 7f7435e00640 1 --2- 172.21.15.2:0/3462849437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7430076230 0x7f7430074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.070+0000 7f7435e00640 1 -- 172.21.15.2:0/3462849437 >> 172.21.15.2:0/3462849437 conn(0x7f74300fd2e0 msgr2=0x7f74300ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.070+0000 7f7435e00640 1 -- 172.21.15.2:0/3462849437 shutdown_connections 2024-09-10T08:17:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.070+0000 7f7435e00640 1 -- 172.21.15.2:0/3462849437 wait complete. 2024-09-10T08:17:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.071+0000 7f7435e00640 1 Processor -- start 2024-09-10T08:17:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.071+0000 7f7435e00640 1 -- start start 2024-09-10T08:17:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.072+0000 7f7435e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7430074c50 0x7f743019e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.072+0000 7f7435e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7430076230 0x7f743019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.072+0000 7f7435e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f743010e130 0x7f74301a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.072+0000 7f7435e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7430113df0 con 0x7f743010e130 2024-09-10T08:17:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.072+0000 7f7435e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7430113c70 con 0x7f7430076230 2024-09-10T08:17:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.072+0000 7f7435e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7430113f70 con 0x7f7430074c50 2024-09-10T08:17:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.072+0000 7f742f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f743010e130 0x7f74301a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.072+0000 7f742f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f743010e130 0x7f74301a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48776/0 (socket says 172.21.15.2:48776) 2024-09-10T08:17:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.072+0000 7f742f400640 1 -- 172.21.15.2:0/2198255492 learned_addr learned my addr 172.21.15.2:0/2198255492 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.072+0000 7f742ea00640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7430074c50 0x7f743019e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.072+0000 7f742e000640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7430076230 0x7f743019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.072+0000 7f742f400640 1 -- 172.21.15.2:0/2198255492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7430074c50 msgr2=0x7f743019e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.073+0000 7f742f400640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7430074c50 0x7f743019e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.073+0000 7f742f400640 1 -- 172.21.15.2:0/2198255492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7430076230 msgr2=0x7f743019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.073+0000 7f742f400640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7430076230 0x7f743019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.073+0000 7f742f400640 1 -- 172.21.15.2:0/2198255492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74301a5f90 con 0x7f743010e130 2024-09-10T08:17:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.073+0000 7f742ea00640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7430074c50 0x7f743019e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.073+0000 7f742f400640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f743010e130 0x7f74301a5920 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7428031ae0 tx=0x7f7428031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.073+0000 7f7434a00640 1 -- 172.21.15.2:0/2198255492 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428003b70 con 0x7f743010e130 2024-09-10T08:17:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.073+0000 7f7434a00640 1 -- 172.21.15.2:0/2198255492 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7428003d10 con 0x7f743010e130 2024-09-10T08:17:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.073+0000 7f7435e00640 1 -- 172.21.15.2:0/2198255492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74301a6220 con 0x7f743010e130 2024-09-10T08:17:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.073+0000 7f7434a00640 1 -- 172.21.15.2:0/2198255492 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742802fc40 con 0x7f743010e130 2024-09-10T08:17:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.073+0000 7f7435e00640 1 -- 172.21.15.2:0/2198255492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74301a67c0 con 0x7f743010e130 2024-09-10T08:17:17.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.076+0000 7f7438c1f740 1 Processor -- start 2024-09-10T08:17:17.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.076+0000 7f7434a00640 1 -- 172.21.15.2:0/2198255492 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7428004050 con 0x7f743010e130 2024-09-10T08:17:17.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.077+0000 7f7438c1f740 1 -- start start 2024-09-10T08:17:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.077+0000 7f7438c1f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 0x55ba3a564490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.077+0000 7f7438c1f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a5649d0 0x55ba3a569490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.077+0000 7f7438c1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a569a60 0x55ba3a56bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.077+0000 7f7438c1f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ba3a4c9100 con 0x55ba3a5649d0 2024-09-10T08:17:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.077+0000 7f7438c1f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ba3a4711b0 con 0x55ba3a564090 2024-09-10T08:17:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.077+0000 7f7438c1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ba3a4c50c0 con 0x55ba3a569a60 2024-09-10T08:17:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.077+0000 7f742ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a5649d0 0x55ba3a569490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.077+0000 7f742e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 0x55ba3a564490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.077+0000 7f742ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a5649d0 0x55ba3a569490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48780/0 (socket says 172.21.15.2:48780) 2024-09-10T08:17:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.077+0000 7f742ea00640 1 -- 172.21.15.2:0/280460626 learned_addr learned my addr 172.21.15.2:0/280460626 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.077+0000 7f742e000640 1 --2- 172.21.15.2:0/280460626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a569a60 0x55ba3a56bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.078+0000 7f7434a00640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f74140a2cb0 0x7f74140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.078+0000 7f742ea00640 1 -- 172.21.15.2:0/280460626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a569a60 msgr2=0x55ba3a56bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.078+0000 7f742ea00640 1 --2- 172.21.15.2:0/280460626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a569a60 0x55ba3a56bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.078+0000 7f742ea00640 1 -- 172.21.15.2:0/280460626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 msgr2=0x55ba3a564490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.078+0000 7f742ea00640 1 --2- 172.21.15.2:0/280460626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 0x55ba3a564490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.078+0000 7f742ea00640 1 -- 172.21.15.2:0/280460626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba3a1dc210 con 0x55ba3a5649d0 2024-09-10T08:17:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.078+0000 7f742e000640 1 --2- 172.21.15.2:0/280460626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a569a60 0x55ba3a56bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.078+0000 7f7434a00640 1 -- 172.21.15.2:0/2198255492 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f74280f0490 con 0x7f743010e130 2024-09-10T08:17:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.078+0000 7f742f400640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f74140a2cb0 0x7f74140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.078+0000 7f742ea00640 1 --2- 172.21.15.2:0/280460626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a5649d0 0x55ba3a569490 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f74200115c0 tx=0x7f7420011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e92a61fa1682f54 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.078+0000 7f741be00640 1 -- 172.21.15.2:0/280460626 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742000f640 con 0x55ba3a5649d0 2024-09-10T08:17:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.078+0000 7f741be00640 1 -- 172.21.15.2:0/280460626 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f742000fcc0 con 0x55ba3a5649d0 2024-09-10T08:17:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.078+0000 7f741be00640 1 -- 172.21.15.2:0/280460626 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420019d60 con 0x55ba3a5649d0 2024-09-10T08:17:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.079+0000 7f7438c1f740 1 -- 172.21.15.2:0/280460626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a5649d0 msgr2=0x55ba3a569490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.079+0000 7f7438c1f740 1 --2- 172.21.15.2:0/280460626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a5649d0 0x55ba3a569490 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f74200115c0 tx=0x7f7420011a80 comp rx=0 tx=0).stop 2024-09-10T08:17:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.079+0000 7f742f400640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f74140a2cb0 0x7f74140a5170 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f74280bc6c0 tx=0x7f742802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.079+0000 7f7438c1f740 1 -- 172.21.15.2:0/280460626 shutdown_connections 2024-09-10T08:17:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.079+0000 7f7438c1f740 1 --2- 172.21.15.2:0/280460626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a569a60 0x55ba3a56bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.079+0000 7f7438c1f740 1 --2- 172.21.15.2:0/280460626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a5649d0 0x55ba3a569490 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.079+0000 7f7438c1f740 1 --2- 172.21.15.2:0/280460626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 0x55ba3a564490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.079+0000 7f7438c1f740 1 -- 172.21.15.2:0/280460626 >> 172.21.15.2:0/280460626 conn(0x55ba3a55f700 msgr2=0x55ba3a55fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.079+0000 7f7438c1f740 1 -- 172.21.15.2:0/280460626 shutdown_connections 2024-09-10T08:17:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.079+0000 7f7438c1f740 1 -- 172.21.15.2:0/280460626 wait complete. 2024-09-10T08:17:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.080+0000 7f7438c1f740 1 -- start start 2024-09-10T08:17:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.080+0000 7f7438c1f740 1 Processor -- start 2024-09-10T08:17:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.080+0000 7f7438c1f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 0x55ba3a56e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.080+0000 7f7438c1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 0x55ba3a56e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.080+0000 7f7438c1f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 0x55ba3a5757d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.080+0000 7f7438c1f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ba3a56ca30 con 0x55ba3a569a60 2024-09-10T08:17:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.080+0000 7f7438c1f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ba3a452090 con 0x55ba3a564090 2024-09-10T08:17:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.080+0000 7f7438c1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ba3a56cbb0 con 0x55ba3a5649d0 2024-09-10T08:17:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.081+0000 7f7438c1f740 10 client.0 _ll_register_callbacks cb 0x55ba3a4a6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.081+0000 7f742e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 0x55ba3a56e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.081+0000 7f742ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 0x55ba3a56e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.081+0000 7f742ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 0x55ba3a56e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35674/0 (socket says 172.21.15.2:35674) 2024-09-10T08:17:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.081+0000 7f742ea00640 1 -- 172.21.15.2:0/2239821939 learned_addr learned my addr 172.21.15.2:0/2239821939 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.081+0000 7f742e000640 1 --2- 172.21.15.2:0/2239821939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 0x55ba3a5757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.081+0000 7f742ea00640 1 -- 172.21.15.2:0/2239821939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 msgr2=0x55ba3a56e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.081+0000 7f742ea00640 1 --2- 172.21.15.2:0/2239821939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 0x55ba3a56e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.081+0000 7f742ea00640 1 -- 172.21.15.2:0/2239821939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 msgr2=0x55ba3a5757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.081+0000 7f742ea00640 1 --2- 172.21.15.2:0/2239821939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 0x55ba3a5757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.081+0000 7f742ea00640 1 -- 172.21.15.2:0/2239821939 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba3a2a8620 con 0x55ba3a5649d0 2024-09-10T08:17:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.081+0000 7f742ea00640 1 --2- 172.21.15.2:0/2239821939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 0x55ba3a56e790 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f74200057c0 tx=0x7f742000fe70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca9d08f46cfd147f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.081+0000 7f7418c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:17:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.082+0000 7f7418c00640 1 -- 172.21.15.2:0/2239821939 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420004040 con 0x55ba3a5649d0 2024-09-10T08:17:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.082+0000 7f7438c1f740 1 -- 172.21.15.2:0/2239821939 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f742000fcc0 con 0x55ba3a5649d0 2024-09-10T08:17:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.082+0000 7f7418c00640 1 -- 172.21.15.2:0/2239821939 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f74200041e0 con 0x55ba3a5649d0 2024-09-10T08:17:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.082+0000 7f7418c00640 1 -- 172.21.15.2:0/2239821939 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7420030050 con 0x55ba3a5649d0 2024-09-10T08:17:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.082+0000 7f7438c1f740 10 client.27895 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.082+0000 7f7438c1f740 1 -- 172.21.15.2:0/2239821939 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba3a578610 con 0x55ba3a5649d0 2024-09-10T08:17:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.082+0000 7f7418c00640 1 -- 172.21.15.2:0/2239821939 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f742000fcc0 con 0x55ba3a5649d0 2024-09-10T08:17:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.083+0000 7f7418c00640 1 -- 172.21.15.2:0/2239821939 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f74200046c0 con 0x55ba3a5649d0 2024-09-10T08:17:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.083+0000 7f7438c1f740 10 client.27895 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.083+0000 7f7438c1f740 10 client.27895 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.083+0000 7f7438c1f740 20 client.27895 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.083+0000 7f7438c1f740 1 --2- 172.21.15.2:0/2239821939 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ba3a579ae0 0x55ba3a57bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.083+0000 7f7438c1f740 4 client.27895 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:17:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.083+0000 7f7438c1f740 1 -- 172.21.15.2:0/2239821939 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba3a4d6830 con 0x55ba3a579ae0 2024-09-10T08:17:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.083+0000 7f742e000640 1 --2- 172.21.15.2:0/2239821939 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ba3a579ae0 0x55ba3a57bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.084+0000 7f742e000640 1 --2- 172.21.15.2:0/2239821939 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ba3a579ae0 0x55ba3a57bf20 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77214801f7799905 server_cookie=be95ca4b3dd3ed02 in_seq=0 out_seq=0 2024-09-10T08:17:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.084+0000 7f7418c00640 10 client.27895 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.084+0000 7f7418c00640 1 -- 172.21.15.2:0/2239821939 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba3a4d6830 con 0x55ba3a579ae0 2024-09-10T08:17:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.084+0000 7f7418c00640 10 client.27895 handle_command_reply: tid=0 2024-09-10T08:17:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.085+0000 7f7438c1f740 1 client.27895 shutdown 2024-09-10T08:17:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.085+0000 7f7438c1f740 1 -- 172.21.15.2:0/2239821939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 msgr2=0x55ba3a56e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.085+0000 7f7438c1f740 1 --2- 172.21.15.2:0/2239821939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 0x55ba3a56e790 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f74200057c0 tx=0x7f742000fe70 comp rx=0 tx=0).stop 2024-09-10T08:17:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.085+0000 7f7438c1f740 1 -- 172.21.15.2:0/2239821939 shutdown_connections 2024-09-10T08:17:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.085+0000 7f7438c1f740 1 --2- 172.21.15.2:0/2239821939 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ba3a579ae0 0x55ba3a57bf20 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.085+0000 7f7438c1f740 1 --2- 172.21.15.2:0/2239821939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 0x55ba3a5757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.085+0000 7f7438c1f740 1 --2- 172.21.15.2:0/2239821939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 0x55ba3a56e790 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.085+0000 7f7418c00640 0 client.27895 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.085+0000 7f7438c1f740 1 --2- 172.21.15.2:0/2239821939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 0x55ba3a56e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.085+0000 7f7438c1f740 1 -- 172.21.15.2:0/2239821939 >> 172.21.15.2:0/2239821939 conn(0x55ba3a55f700 msgr2=0x55ba3a560f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.085+0000 7f7438c1f740 1 -- 172.21.15.2:0/2239821939 shutdown_connections 2024-09-10T08:17:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.085+0000 7f7438c1f740 1 -- 172.21.15.2:0/2239821939 wait complete. 2024-09-10T08:17:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.085+0000 7f7438c1f740 20 client.27895 trim_cache size 0 max 16384 2024-09-10T08:17:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.100+0000 7f7438c1f740 1 Processor -- start 2024-09-10T08:17:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.100+0000 7f7438c1f740 1 -- start start 2024-09-10T08:17:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f7438c1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a564090 0x55ba3a55a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f7438c1f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a5649d0 0x55ba3a575790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f7438c1f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 0x55ba3a55fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f7438c1f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f74301b36a0 con 0x55ba3a569a60 2024-09-10T08:17:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f7438c1f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74301b0f70 con 0x55ba3a5649d0 2024-09-10T08:17:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f7438c1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f74301aedc0 con 0x55ba3a564090 2024-09-10T08:17:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f742e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a564090 0x55ba3a55a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f742e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a564090 0x55ba3a55a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35690/0 (socket says 172.21.15.2:35690) 2024-09-10T08:17:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f742e000640 1 -- 172.21.15.2:0/786813134 learned_addr learned my addr 172.21.15.2:0/786813134 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f742ea00640 1 --2- 172.21.15.2:0/786813134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a5649d0 0x55ba3a575790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f742e000640 1 --2- 172.21.15.2:0/786813134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 0x55ba3a55fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f742e000640 1 -- 172.21.15.2:0/786813134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a5649d0 msgr2=0x55ba3a575790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f742e000640 1 --2- 172.21.15.2:0/786813134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a5649d0 0x55ba3a575790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f742e000640 1 -- 172.21.15.2:0/786813134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 msgr2=0x55ba3a55fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f742e000640 1 --2- 172.21.15.2:0/786813134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 0x55ba3a55fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f742e000640 1 -- 172.21.15.2:0/786813134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba3a564eb0 con 0x55ba3a564090 2024-09-10T08:17:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.101+0000 7f742ea00640 1 --2- 172.21.15.2:0/786813134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a5649d0 0x55ba3a575790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.102+0000 7f742e000640 1 --2- 172.21.15.2:0/786813134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a564090 0x55ba3a55a8d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f743019f880 tx=0x7f741c009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391833f9e398b221 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.102+0000 7f741b400640 1 -- 172.21.15.2:0/786813134 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c032070 con 0x55ba3a564090 2024-09-10T08:17:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.102+0000 7f741b400640 1 -- 172.21.15.2:0/786813134 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f741c003ed0 con 0x55ba3a564090 2024-09-10T08:17:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.102+0000 7f741b400640 1 -- 172.21.15.2:0/786813134 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c023de0 con 0x55ba3a564090 2024-09-10T08:17:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.102+0000 7f7438c1f740 1 -- 172.21.15.2:0/786813134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a564090 msgr2=0x55ba3a55a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:17.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.102+0000 7f7438c1f740 1 --2- 172.21.15.2:0/786813134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a564090 0x55ba3a55a8d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f743019f880 tx=0x7f741c009770 comp rx=0 tx=0).stop 2024-09-10T08:17:17.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.103+0000 7f7438c1f740 1 -- 172.21.15.2:0/786813134 shutdown_connections 2024-09-10T08:17:17.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.103+0000 7f7438c1f740 1 --2- 172.21.15.2:0/786813134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 0x55ba3a55fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.103+0000 7f7438c1f740 1 --2- 172.21.15.2:0/786813134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a5649d0 0x55ba3a575790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.103+0000 7f7438c1f740 1 --2- 172.21.15.2:0/786813134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a564090 0x55ba3a55a8d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.103+0000 7f7438c1f740 1 -- 172.21.15.2:0/786813134 >> 172.21.15.2:0/786813134 conn(0x55ba3a55f700 msgr2=0x55ba3a56d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:17.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.103+0000 7f7438c1f740 1 -- 172.21.15.2:0/786813134 shutdown_connections 2024-09-10T08:17:17.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.103+0000 7f7438c1f740 1 -- 172.21.15.2:0/786813134 wait complete. 2024-09-10T08:17:17.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.103+0000 7f7438c1f740 1 -- start start 2024-09-10T08:17:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.103+0000 7f7438c1f740 1 Processor -- start 2024-09-10T08:17:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f7438c1f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 0x55ba3a575570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f7438c1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 0x55ba3a5721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f7438c1f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 0x55ba3a5726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f7438c1f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f74301af430 con 0x55ba3a569a60 2024-09-10T08:17:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f7438c1f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74301b08a0 con 0x55ba3a564090 2024-09-10T08:17:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f7438c1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f74301afb00 con 0x55ba3a5649d0 2024-09-10T08:17:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f7438c1f740 10 client.0 _ll_register_callbacks cb 0x7f741c009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f742e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 0x55ba3a5726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f742ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 0x55ba3a5721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f742e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 0x55ba3a575570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f742ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 0x55ba3a5721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35692/0 (socket says 172.21.15.2:35692) 2024-09-10T08:17:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f742e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 0x55ba3a575570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:45456/0 (socket says 172.21.15.2:45456) 2024-09-10T08:17:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f742ea00640 1 -- 172.21.15.2:0/1815326434 learned_addr learned my addr 172.21.15.2:0/1815326434 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f742ea00640 1 -- 172.21.15.2:0/1815326434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 msgr2=0x55ba3a575570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f742ea00640 1 --2- 172.21.15.2:0/1815326434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 0x55ba3a575570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f742ea00640 1 -- 172.21.15.2:0/1815326434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 msgr2=0x55ba3a5726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f742ea00640 1 --2- 172.21.15.2:0/1815326434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 0x55ba3a5726e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.104+0000 7f742ea00640 1 -- 172.21.15.2:0/1815326434 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba3a5e27b0 con 0x55ba3a5649d0 2024-09-10T08:17:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.105+0000 7f742ea00640 1 --2- 172.21.15.2:0/1815326434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 0x55ba3a5721a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55ba3a56f730 tx=0x7f742000fee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb361abc4d5603ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.105+0000 7f7418c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:17:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.105+0000 7f7418c00640 1 -- 172.21.15.2:0/1815326434 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742000f640 con 0x55ba3a5649d0 2024-09-10T08:17:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.105+0000 7f7418c00640 1 -- 172.21.15.2:0/1815326434 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f742000f7e0 con 0x55ba3a5649d0 2024-09-10T08:17:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.105+0000 7f7438c1f740 1 -- 172.21.15.2:0/1815326434 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f741c003ed0 con 0x55ba3a5649d0 2024-09-10T08:17:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.105+0000 7f7418c00640 1 -- 172.21.15.2:0/1815326434 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7420043050 con 0x55ba3a5649d0 2024-09-10T08:17:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.105+0000 7f7438c1f740 10 client.27901 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.105+0000 7f7438c1f740 1 -- 172.21.15.2:0/1815326434 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba3a5e39d0 con 0x55ba3a5649d0 2024-09-10T08:17:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.105+0000 7f7418c00640 1 -- 172.21.15.2:0/1815326434 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f741c003ed0 con 0x55ba3a5649d0 2024-09-10T08:17:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.106+0000 7f7418c00640 1 -- 172.21.15.2:0/1815326434 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7420004040 con 0x55ba3a5649d0 2024-09-10T08:17:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.106+0000 7f7438c1f740 10 client.27901 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.106+0000 7f7438c1f740 10 client.27901 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.106+0000 7f7438c1f740 20 client.27901 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.106+0000 7f7438c1f740 1 --2- 172.21.15.2:0/1815326434 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ba3a5e43e0 0x55ba3a5e6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.106+0000 7f7438c1f740 4 client.27901 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:17:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.106+0000 7f7438c1f740 1 -- 172.21.15.2:0/1815326434 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba3a4d0b80 con 0x55ba3a5e43e0 2024-09-10T08:17:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.106+0000 7f742e000640 1 --2- 172.21.15.2:0/1815326434 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ba3a5e43e0 0x55ba3a5e6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.107+0000 7f742e000640 1 --2- 172.21.15.2:0/1815326434 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ba3a5e43e0 0x55ba3a5e6800 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac84a226bd5350c7 server_cookie=2fbc3aa29e6acebc in_seq=0 out_seq=0 2024-09-10T08:17:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.107+0000 7f7418c00640 10 client.27901 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7418c00640 1 -- 172.21.15.2:0/1815326434 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba3a4d0b80 con 0x55ba3a5e43e0 2024-09-10T08:17:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7418c00640 10 client.27901 handle_command_reply: tid=0 2024-09-10T08:17:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7438c1f740 1 client.27901 shutdown 2024-09-10T08:17:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7438c1f740 1 -- 172.21.15.2:0/1815326434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 msgr2=0x55ba3a5721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7438c1f740 1 --2- 172.21.15.2:0/1815326434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 0x55ba3a5721a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55ba3a56f730 tx=0x7f742000fee0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7438c1f740 1 -- 172.21.15.2:0/1815326434 shutdown_connections 2024-09-10T08:17:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7438c1f740 1 --2- 172.21.15.2:0/1815326434 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ba3a5e43e0 0x55ba3a5e6800 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7438c1f740 1 --2- 172.21.15.2:0/1815326434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba3a569a60 0x55ba3a5726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7438c1f740 1 --2- 172.21.15.2:0/1815326434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3a5649d0 0x55ba3a5721a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7418c00640 0 client.27901 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7438c1f740 1 --2- 172.21.15.2:0/1815326434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ba3a564090 0x55ba3a575570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7438c1f740 1 -- 172.21.15.2:0/1815326434 >> 172.21.15.2:0/1815326434 conn(0x55ba3a55f700 msgr2=0x55ba3a56cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7438c1f740 1 -- 172.21.15.2:0/1815326434 shutdown_connections 2024-09-10T08:17:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7438c1f740 1 -- 172.21.15.2:0/1815326434 wait complete. 2024-09-10T08:17:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.120+0000 7f7438c1f740 20 client.27901 trim_cache size 0 max 16384 2024-09-10T08:17:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:17:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:17:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:17:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:17:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.121+0000 7f7435e00640 1 -- 172.21.15.2:0/2198255492 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f74140a2cb0 msgr2=0x7f74140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.121+0000 7f7435e00640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f74140a2cb0 0x7f74140a5170 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f74280bc6c0 tx=0x7f742802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.121+0000 7f7435e00640 1 -- 172.21.15.2:0/2198255492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f743010e130 msgr2=0x7f74301a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.121+0000 7f7435e00640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f743010e130 0x7f74301a5920 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7428031ae0 tx=0x7f7428031ee0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.121+0000 7f7435e00640 1 -- 172.21.15.2:0/2198255492 shutdown_connections 2024-09-10T08:17:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.121+0000 7f7435e00640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f743010e130 0x7f74301a5920 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.121+0000 7f7435e00640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7430076230 0x7f743019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.121+0000 7f7435e00640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f74140a2cb0 0x7f74140a5170 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.121+0000 7f7435e00640 1 --2- 172.21.15.2:0/2198255492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7430074c50 0x7f743019e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.121+0000 7f7435e00640 1 -- 172.21.15.2:0/2198255492 >> 172.21.15.2:0/2198255492 conn(0x7f74300fd2e0 msgr2=0x7f74301013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.121+0000 7f7435e00640 1 -- 172.21.15.2:0/2198255492 shutdown_connections 2024-09-10T08:17:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:17.121+0000 7f7435e00640 1 -- 172.21.15.2:0/2198255492 wait complete. 2024-09-10T08:17:17.423 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.6 2024-09-10T08:17:17.423 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:17:17.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:17 smithi002 ceph-mon[30563]: pgmap v661: 129 pgs: 129 active+clean; 718 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-10T08:17:17.807 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:17:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:17 smithi155 ceph-mon[35405]: pgmap v661: 129 pgs: 129 active+clean; 718 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-10T08:17:17.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:17 smithi107 ceph-mon[35048]: pgmap v661: 129 pgs: 129 active+clean; 718 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-10T08:17:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.373+0000 7f164de00640 1 -- 172.21.15.2:0/4248896928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648101de0 msgr2=0x7f16481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.373+0000 7f164de00640 1 --2- 172.21.15.2:0/4248896928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648101de0 0x7f16481021c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1638009900 tx=0x7f163802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:17:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.374+0000 7f164de00640 1 -- 172.21.15.2:0/4248896928 shutdown_connections 2024-09-10T08:17:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.374+0000 7f164de00640 1 --2- 172.21.15.2:0/4248896928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1648106f10 0x7f164810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.374+0000 7f164de00640 1 --2- 172.21.15.2:0/4248896928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1648102790 0x7f16481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.374+0000 7f164de00640 1 --2- 172.21.15.2:0/4248896928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648101de0 0x7f16481021c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.374+0000 7f164de00640 1 -- 172.21.15.2:0/4248896928 >> 172.21.15.2:0/4248896928 conn(0x7f16480fd8c0 msgr2=0x7f16480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.374+0000 7f164de00640 1 -- 172.21.15.2:0/4248896928 shutdown_connections 2024-09-10T08:17:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.374+0000 7f164de00640 1 -- 172.21.15.2:0/4248896928 wait complete. 2024-09-10T08:17:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.375+0000 7f164de00640 1 Processor -- start 2024-09-10T08:17:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.375+0000 7f164de00640 1 -- start start 2024-09-10T08:17:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.376+0000 7f164de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1648101de0 0x7f1648198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.376+0000 7f164de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1648102790 0x7f1648198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.376+0000 7f164de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648106f10 0x7f164819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.376+0000 7f164de00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1648111f70 con 0x7f1648106f10 2024-09-10T08:17:18.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.376+0000 7f164de00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1648111df0 con 0x7f1648101de0 2024-09-10T08:17:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.376+0000 7f164de00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16481120f0 con 0x7f1648102790 2024-09-10T08:17:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.376+0000 7f163e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1648102790 0x7f1648198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.376+0000 7f1647400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648106f10 0x7f164819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.376+0000 7f163e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1648102790 0x7f1648198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35702/0 (socket says 172.21.15.2:35702) 2024-09-10T08:17:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.376+0000 7f163e000640 1 -- 172.21.15.2:0/2550486945 learned_addr learned my addr 172.21.15.2:0/2550486945 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.376+0000 7f1646a00640 1 --2- 172.21.15.2:0/2550486945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1648101de0 0x7f1648198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.377+0000 7f1647400640 1 -- 172.21.15.2:0/2550486945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1648102790 msgr2=0x7f1648198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.377+0000 7f1647400640 1 --2- 172.21.15.2:0/2550486945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1648102790 0x7f1648198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.377+0000 7f1647400640 1 -- 172.21.15.2:0/2550486945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1648101de0 msgr2=0x7f1648198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.377+0000 7f1647400640 1 --2- 172.21.15.2:0/2550486945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1648101de0 0x7f1648198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.377+0000 7f1647400640 1 -- 172.21.15.2:0/2550486945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164819d400 con 0x7f1648106f10 2024-09-10T08:17:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.377+0000 7f1647400640 1 --2- 172.21.15.2:0/2550486945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648106f10 0x7f164819cd20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f164000da10 tx=0x7f164000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.377+0000 7f164ca00640 1 -- 172.21.15.2:0/2550486945 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164000bd70 con 0x7f1648106f10 2024-09-10T08:17:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.377+0000 7f164ca00640 1 -- 172.21.15.2:0/2550486945 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1640003e50 con 0x7f1648106f10 2024-09-10T08:17:18.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.377+0000 7f164ca00640 1 -- 172.21.15.2:0/2550486945 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16400106a0 con 0x7f1648106f10 2024-09-10T08:17:18.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.377+0000 7f164de00640 1 -- 172.21.15.2:0/2550486945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164819d6f0 con 0x7f1648106f10 2024-09-10T08:17:18.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.377+0000 7f164de00640 1 -- 172.21.15.2:0/2550486945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16481a4fd0 con 0x7f1648106f10 2024-09-10T08:17:18.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.379+0000 7f164de00640 1 -- 172.21.15.2:0/2550486945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16481021c0 con 0x7f1648106f10 2024-09-10T08:17:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.381+0000 7f164ca00640 1 -- 172.21.15.2:0/2550486945 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1640003970 con 0x7f1648106f10 2024-09-10T08:17:18.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.382+0000 7f164ca00640 1 --2- 172.21.15.2:0/2550486945 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f162c0a2c00 0x7f162c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:18.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.382+0000 7f164ca00640 1 -- 172.21.15.2:0/2550486945 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f16400cf710 con 0x7f1648106f10 2024-09-10T08:17:18.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.382+0000 7f1646a00640 1 --2- 172.21.15.2:0/2550486945 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f162c0a2c00 0x7f162c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:18.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.383+0000 7f1646a00640 1 --2- 172.21.15.2:0/2550486945 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f162c0a2c00 0x7f162c0a50c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1638009230 tx=0x7f16380058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.386+0000 7f164ca00640 1 -- 172.21.15.2:0/2550486945 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f164009c220 con 0x7f1648106f10 2024-09-10T08:17:18.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.610+0000 7f164de00640 1 -- 172.21.15.2:0/2550486945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f164802d7c0 con 0x7f1648106f10 2024-09-10T08:17:18.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.610+0000 7f164ca00640 1 -- 172.21.15.2:0/2550486945 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f164009bb20 con 0x7f1648106f10 2024-09-10T08:17:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:17:18.611 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:17:18.612 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:17:18.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.613+0000 7f164de00640 1 -- 172.21.15.2:0/2550486945 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f162c0a2c00 msgr2=0x7f162c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:18.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.613+0000 7f164de00640 1 --2- 172.21.15.2:0/2550486945 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f162c0a2c00 0x7f162c0a50c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1638009230 tx=0x7f16380058d0 comp rx=0 tx=0).stop 2024-09-10T08:17:18.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.613+0000 7f164de00640 1 -- 172.21.15.2:0/2550486945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648106f10 msgr2=0x7f164819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:18.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.613+0000 7f164de00640 1 --2- 172.21.15.2:0/2550486945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648106f10 0x7f164819cd20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f164000da10 tx=0x7f164000ded0 comp rx=0 tx=0).stop 2024-09-10T08:17:18.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.613+0000 7f164de00640 1 -- 172.21.15.2:0/2550486945 shutdown_connections 2024-09-10T08:17:18.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.613+0000 7f164de00640 1 --2- 172.21.15.2:0/2550486945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648106f10 0x7f164819cd20 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.613+0000 7f164de00640 1 --2- 172.21.15.2:0/2550486945 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f162c0a2c00 0x7f162c0a50c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.613+0000 7f164de00640 1 --2- 172.21.15.2:0/2550486945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1648102790 0x7f1648198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.613+0000 7f164de00640 1 --2- 172.21.15.2:0/2550486945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1648101de0 0x7f1648198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.613+0000 7f164de00640 1 -- 172.21.15.2:0/2550486945 >> 172.21.15.2:0/2550486945 conn(0x7f16480fd8c0 msgr2=0x7f16480fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.613+0000 7f164de00640 1 -- 172.21.15.2:0/2550486945 shutdown_connections 2024-09-10T08:17:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:18.613+0000 7f164de00640 1 -- 172.21.15.2:0/2550486945 wait complete. 2024-09-10T08:17:19.806 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 144.6 --await-for 45.0' 2024-09-10T08:17:19.806 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 144.6 --await-for 45.0 2024-09-10T08:17:19.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:19 smithi002 ceph-mon[30563]: pgmap v662: 129 pgs: 129 active+clean; 794 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:17:19.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:19 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2550486945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:17:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:19 smithi155 ceph-mon[35405]: pgmap v662: 129 pgs: 129 active+clean; 794 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:17:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:19 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2550486945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:17:19.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:19 smithi107 ceph-mon[35048]: pgmap v662: 129 pgs: 129 active+clean; 794 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:17:19.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:19 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2550486945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:17:20.188 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:17:20.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.761+0000 7f75a8400640 1 -- 172.21.15.2:0/1341844525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0101de0 msgr2=0x7f75a01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:20.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.761+0000 7f75a8400640 1 --2- 172.21.15.2:0/1341844525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0101de0 0x7f75a01021c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7594009900 tx=0x7f759402a410 comp rx=0 tx=0).stop 2024-09-10T08:17:20.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.761+0000 7f75a8400640 1 -- 172.21.15.2:0/1341844525 shutdown_connections 2024-09-10T08:17:20.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.761+0000 7f75a8400640 1 --2- 172.21.15.2:0/1341844525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a0106f10 0x7f75a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.761+0000 7f75a8400640 1 --2- 172.21.15.2:0/1341844525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75a0102790 0x7f75a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.761+0000 7f75a8400640 1 --2- 172.21.15.2:0/1341844525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0101de0 0x7f75a01021c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.761+0000 7f75a8400640 1 -- 172.21.15.2:0/1341844525 >> 172.21.15.2:0/1341844525 conn(0x7f75a00fd8c0 msgr2=0x7f75a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.761+0000 7f75a8400640 1 -- 172.21.15.2:0/1341844525 shutdown_connections 2024-09-10T08:17:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.761+0000 7f75a8400640 1 -- 172.21.15.2:0/1341844525 wait complete. 2024-09-10T08:17:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.762+0000 7f75a8400640 1 Processor -- start 2024-09-10T08:17:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.762+0000 7f75a8400640 1 -- start start 2024-09-10T08:17:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.762+0000 7f75a8400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0101de0 0x7f75a019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.762+0000 7f75a8400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75a0102790 0x7f75a019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a8400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a0106f10 0x7f75a019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a8400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0111f80 con 0x7f75a0101de0 2024-09-10T08:17:20.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a8400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0111e00 con 0x7f75a0106f10 2024-09-10T08:17:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a8400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0112100 con 0x7f75a0102790 2024-09-10T08:17:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a5a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0101de0 0x7f75a019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a5000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75a0102790 0x7f75a019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a5a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0101de0 0x7f75a019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48838/0 (socket says 172.21.15.2:48838) 2024-09-10T08:17:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a5a00640 1 -- 172.21.15.2:0/149876778 learned_addr learned my addr 172.21.15.2:0/149876778 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a5000640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75a0102790 0x7f75a019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35724/0 (socket says 172.21.15.2:35724) 2024-09-10T08:17:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a6400640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a0106f10 0x7f75a019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a5a00640 1 -- 172.21.15.2:0/149876778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75a0102790 msgr2=0x7f75a019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a5a00640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75a0102790 0x7f75a019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a5a00640 1 -- 172.21.15.2:0/149876778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a0106f10 msgr2=0x7f75a019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a5a00640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a0106f10 0x7f75a019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.763+0000 7f75a5a00640 1 -- 172.21.15.2:0/149876778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a019f620 con 0x7f75a0101de0 2024-09-10T08:17:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.764+0000 7f75a5a00640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0101de0 0x7f75a019a670 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7594004770 tx=0x7f7594003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.764+0000 7f759e000640 1 -- 172.21.15.2:0/149876778 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759403f070 con 0x7f75a0101de0 2024-09-10T08:17:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.764+0000 7f759e000640 1 -- 172.21.15.2:0/149876778 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7594031ca0 con 0x7f75a0101de0 2024-09-10T08:17:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.764+0000 7f75a8400640 1 -- 172.21.15.2:0/149876778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a019f8b0 con 0x7f75a0101de0 2024-09-10T08:17:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.764+0000 7f759e000640 1 -- 172.21.15.2:0/149876778 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594035030 con 0x7f75a0101de0 2024-09-10T08:17:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.764+0000 7f75a8400640 1 -- 172.21.15.2:0/149876778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a01a71f0 con 0x7f75a0101de0 2024-09-10T08:17:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.767+0000 7f759e000640 1 -- 172.21.15.2:0/149876778 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7594042070 con 0x7f75a0101de0 2024-09-10T08:17:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.767+0000 7f75aae31740 1 Processor -- start 2024-09-10T08:17:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.767+0000 7f75aae31740 1 -- start start 2024-09-10T08:17:20.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.768+0000 7f75aae31740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf263b0 0x55b9ddf26790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.768+0000 7f75aae31740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf31050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.768+0000 7f75aae31740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf31590 0x55b9ddf33980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.768+0000 7f75aae31740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b9dde8c530 con 0x55b9ddf31590 2024-09-10T08:17:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.768+0000 7f75aae31740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b9dde33860 con 0x55b9ddf263b0 2024-09-10T08:17:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.768+0000 7f75aae31740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b9dde884f0 con 0x55b9ddf26cd0 2024-09-10T08:17:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.768+0000 7f75a5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf31590 0x55b9ddf33980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.768+0000 7f75a5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf31050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.768+0000 7f75a5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf31590 0x55b9ddf33980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48844/0 (socket says 172.21.15.2:48844) 2024-09-10T08:17:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.768+0000 7f75a5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf31050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35732/0 (socket says 172.21.15.2:35732) 2024-09-10T08:17:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.768+0000 7f75a5000640 1 -- 172.21.15.2:0/863109848 learned_addr learned my addr 172.21.15.2:0/863109848 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.768+0000 7f75a6400640 1 --2- 172.21.15.2:0/863109848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf263b0 0x55b9ddf26790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.769+0000 7f75a5000640 1 -- 172.21.15.2:0/863109848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 msgr2=0x55b9ddf31050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.769+0000 7f75a5000640 1 --2- 172.21.15.2:0/863109848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf31050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.769+0000 7f75a5000640 1 -- 172.21.15.2:0/863109848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf263b0 msgr2=0x55b9ddf26790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.769+0000 7f75a5000640 1 --2- 172.21.15.2:0/863109848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf263b0 0x55b9ddf26790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.769+0000 7f75a5000640 1 -- 172.21.15.2:0/863109848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9ddbfdb90 con 0x55b9ddf31590 2024-09-10T08:17:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.769+0000 7f75a6400640 1 --2- 172.21.15.2:0/863109848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf263b0 0x55b9ddf26790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.769+0000 7f759e000640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75880a2b80 0x7f75880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.769+0000 7f75a5000640 1 --2- 172.21.15.2:0/863109848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf31590 0x55b9ddf33980 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f758c00bc90 tx=0x7f758c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7972bbd6d1766c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.769+0000 7f759e000640 1 -- 172.21.15.2:0/149876778 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f75940efb30 con 0x7f75a0101de0 2024-09-10T08:17:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.769+0000 7f75a6400640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75880a2b80 0x7f75880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.769+0000 7f7592a00640 1 -- 172.21.15.2:0/863109848 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c005220 con 0x55b9ddf31590 2024-09-10T08:17:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.770+0000 7f7592a00640 1 -- 172.21.15.2:0/863109848 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758c00be40 con 0x55b9ddf31590 2024-09-10T08:17:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.770+0000 7f7592a00640 1 -- 172.21.15.2:0/863109848 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c011770 con 0x55b9ddf31590 2024-09-10T08:17:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.770+0000 7f75a6400640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75880a2b80 0x7f75880a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7598009fd0 tx=0x7f7598009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.770+0000 7f75aae31740 1 -- 172.21.15.2:0/863109848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf31590 msgr2=0x55b9ddf33980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.770+0000 7f75aae31740 1 --2- 172.21.15.2:0/863109848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf31590 0x55b9ddf33980 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f758c00bc90 tx=0x7f758c0107a0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.771+0000 7f75aae31740 1 -- 172.21.15.2:0/863109848 shutdown_connections 2024-09-10T08:17:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.771+0000 7f75aae31740 1 --2- 172.21.15.2:0/863109848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf31590 0x55b9ddf33980 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.771+0000 7f75aae31740 1 --2- 172.21.15.2:0/863109848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf31050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.771+0000 7f75aae31740 1 --2- 172.21.15.2:0/863109848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf263b0 0x55b9ddf26790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.771+0000 7f75aae31740 1 -- 172.21.15.2:0/863109848 >> 172.21.15.2:0/863109848 conn(0x55b9ddf22940 msgr2=0x55b9ddf22d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.771+0000 7f75aae31740 1 -- 172.21.15.2:0/863109848 shutdown_connections 2024-09-10T08:17:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.771+0000 7f75aae31740 1 -- 172.21.15.2:0/863109848 wait complete. 2024-09-10T08:17:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.771+0000 7f75aae31740 1 -- start start 2024-09-10T08:17:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.771+0000 7f75aae31740 1 Processor -- start 2024-09-10T08:17:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.772+0000 7f75aae31740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf263b0 0x55b9ddf2e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.772+0000 7f75aae31740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf2ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.772+0000 7f75aae31740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf31590 0x55b9ddf2f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.772+0000 7f75aae31740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b9ddf33fb0 con 0x55b9ddf263b0 2024-09-10T08:17:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.772+0000 7f75aae31740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b9dde46e60 con 0x55b9ddf31590 2024-09-10T08:17:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.772+0000 7f75aae31740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b9ddf34130 con 0x55b9ddf26cd0 2024-09-10T08:17:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.772+0000 7f75aae31740 10 client.0 _ll_register_callbacks cb 0x55b9dde4bcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.772+0000 7f75a5a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf263b0 0x55b9ddf2e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.772+0000 7f75a6400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf31590 0x55b9ddf2f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.772+0000 7f75a6400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf31590 0x55b9ddf2f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:45536/0 (socket says 172.21.15.2:45536) 2024-09-10T08:17:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.772+0000 7f75a6400640 1 -- 172.21.15.2:0/1070539712 learned_addr learned my addr 172.21.15.2:0/1070539712 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.772+0000 7f75a5000640 1 --2- 172.21.15.2:0/1070539712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf2ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.773+0000 7f75a6400640 1 -- 172.21.15.2:0/1070539712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 msgr2=0x55b9ddf2ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.773+0000 7f75a6400640 1 --2- 172.21.15.2:0/1070539712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf2ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.773+0000 7f75a6400640 1 -- 172.21.15.2:0/1070539712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf263b0 msgr2=0x55b9ddf2e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.773+0000 7f75a6400640 1 --2- 172.21.15.2:0/1070539712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf263b0 0x55b9ddf2e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.773+0000 7f75a6400640 1 -- 172.21.15.2:0/1070539712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9dde52990 con 0x55b9ddf31590 2024-09-10T08:17:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.773+0000 7f75a6400640 1 --2- 172.21.15.2:0/1070539712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf31590 0x55b9ddf2f110 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f759800f6a0 tx=0x7f7598004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b043a9949358c46b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.773+0000 7f7587400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:17:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.773+0000 7f7587400640 1 -- 172.21.15.2:0/1070539712 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759802c060 con 0x55b9ddf31590 2024-09-10T08:17:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.773+0000 7f75aae31740 1 -- 172.21.15.2:0/1070539712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b9dde2f310 con 0x55b9ddf31590 2024-09-10T08:17:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.773+0000 7f7587400640 1 -- 172.21.15.2:0/1070539712 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7598017030 con 0x55b9ddf31590 2024-09-10T08:17:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.774+0000 7f7587400640 1 -- 172.21.15.2:0/1070539712 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7598033050 con 0x55b9ddf31590 2024-09-10T08:17:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.774+0000 7f75aae31740 10 client.27986 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.774+0000 7f75aae31740 1 -- 172.21.15.2:0/1070539712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b9ddf3ff10 con 0x55b9ddf31590 2024-09-10T08:17:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.774+0000 7f7587400640 1 -- 172.21.15.2:0/1070539712 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b9dde2f310 con 0x55b9ddf31590 2024-09-10T08:17:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.774+0000 7f7587400640 1 -- 172.21.15.2:0/1070539712 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7598027ab0 con 0x55b9ddf31590 2024-09-10T08:17:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.774+0000 7f75aae31740 10 client.27986 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.774+0000 7f75aae31740 10 client.27986 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:17:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.774+0000 7f75aae31740 20 client.27986 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.774+0000 7f75aae31740 1 --2- 172.21.15.2:0/1070539712 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9ddf41320 0x55b9ddf43770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.774+0000 7f75aae31740 4 client.27986 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:17:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.774+0000 7f75aae31740 1 -- 172.21.15.2:0/1070539712 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9dde99c60 con 0x55b9ddf41320 2024-09-10T08:17:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.775+0000 7f75a5a00640 1 --2- 172.21.15.2:0/1070539712 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9ddf41320 0x55b9ddf43770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.775+0000 7f75a5a00640 1 --2- 172.21.15.2:0/1070539712 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9ddf41320 0x55b9ddf43770 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39c310cf83fdc907 server_cookie=4ce5ba21c8a847a8 in_seq=0 out_seq=0 2024-09-10T08:17:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.775+0000 7f7587400640 10 client.27986 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f7587400640 1 -- 172.21.15.2:0/1070539712 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9dde99c60 con 0x55b9ddf41320 2024-09-10T08:17:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f7587400640 10 client.27986 handle_command_reply: tid=0 2024-09-10T08:17:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f75aae31740 1 client.27986 shutdown 2024-09-10T08:17:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f75aae31740 1 -- 172.21.15.2:0/1070539712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf31590 msgr2=0x55b9ddf2f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f75aae31740 1 --2- 172.21.15.2:0/1070539712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf31590 0x55b9ddf2f110 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f759800f6a0 tx=0x7f7598004ab0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f75aae31740 1 -- 172.21.15.2:0/1070539712 shutdown_connections 2024-09-10T08:17:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f75aae31740 1 --2- 172.21.15.2:0/1070539712 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9ddf41320 0x55b9ddf43770 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f75aae31740 1 --2- 172.21.15.2:0/1070539712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf31590 0x55b9ddf2f110 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f75aae31740 1 --2- 172.21.15.2:0/1070539712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf2ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f7587400640 0 client.27986 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f75aae31740 1 --2- 172.21.15.2:0/1070539712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf263b0 0x55b9ddf2e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f75aae31740 1 -- 172.21.15.2:0/1070539712 >> 172.21.15.2:0/1070539712 conn(0x55b9ddf22940 msgr2=0x55b9ddf21c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f75aae31740 1 -- 172.21.15.2:0/1070539712 shutdown_connections 2024-09-10T08:17:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f75aae31740 1 -- 172.21.15.2:0/1070539712 wait complete. 2024-09-10T08:17:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.776+0000 7f75aae31740 20 client.27986 trim_cache size 0 max 16384 2024-09-10T08:17:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.794+0000 7f75aae31740 1 Processor -- start 2024-09-10T08:17:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.794+0000 7f75aae31740 1 -- start start 2024-09-10T08:17:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.794+0000 7f75aae31740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf22940 0x55b9ddf2cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.794+0000 7f75aae31740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf263b0 0x55b9ddf49040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.794+0000 7f75aae31740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf49580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.794+0000 7f75aae31740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01b3970 con 0x55b9ddf26cd0 2024-09-10T08:17:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.794+0000 7f75aae31740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01b1240 con 0x55b9ddf22940 2024-09-10T08:17:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.794+0000 7f75aae31740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01af090 con 0x55b9ddf263b0 2024-09-10T08:17:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.795+0000 7f75a6400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf49580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.795+0000 7f75a5000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf263b0 0x55b9ddf49040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.795+0000 7f75a5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf22940 0x55b9ddf2cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.795+0000 7f75a5000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf263b0 0x55b9ddf49040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35742/0 (socket says 172.21.15.2:35742) 2024-09-10T08:17:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.795+0000 7f75a5000640 1 -- 172.21.15.2:0/3944336449 learned_addr learned my addr 172.21.15.2:0/3944336449 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.795+0000 7f75a5a00640 1 -- 172.21.15.2:0/3944336449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf263b0 msgr2=0x55b9ddf49040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.795+0000 7f75a5a00640 1 --2- 172.21.15.2:0/3944336449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf263b0 0x55b9ddf49040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.795+0000 7f75a5a00640 1 -- 172.21.15.2:0/3944336449 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf26cd0 msgr2=0x55b9ddf49580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.795+0000 7f75a5a00640 1 --2- 172.21.15.2:0/3944336449 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf49580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.795+0000 7f75a5a00640 1 -- 172.21.15.2:0/3944336449 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9ddf235d0 con 0x55b9ddf22940 2024-09-10T08:17:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.795+0000 7f75a6400640 1 --2- 172.21.15.2:0/3944336449 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf49580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.795+0000 7f75a5a00640 1 --2- 172.21.15.2:0/3944336449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf22940 0x55b9ddf2cf60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55b9ddf237a0 tx=0x7f75940f3570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7daee228d283d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.796+0000 7f7592000640 1 -- 172.21.15.2:0/3944336449 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594049040 con 0x55b9ddf22940 2024-09-10T08:17:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.796+0000 7f7592000640 1 -- 172.21.15.2:0/3944336449 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75940bbff0 con 0x55b9ddf22940 2024-09-10T08:17:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.796+0000 7f7592000640 1 -- 172.21.15.2:0/3944336449 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75940ba2f0 con 0x55b9ddf22940 2024-09-10T08:17:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.796+0000 7f75aae31740 1 -- 172.21.15.2:0/3944336449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf22940 msgr2=0x55b9ddf2cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.796+0000 7f75aae31740 1 --2- 172.21.15.2:0/3944336449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf22940 0x55b9ddf2cf60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55b9ddf237a0 tx=0x7f75940f3570 comp rx=0 tx=0).stop 2024-09-10T08:17:20.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.796+0000 7f75aae31740 1 -- 172.21.15.2:0/3944336449 shutdown_connections 2024-09-10T08:17:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.796+0000 7f75aae31740 1 --2- 172.21.15.2:0/3944336449 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf49580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.796+0000 7f75aae31740 1 --2- 172.21.15.2:0/3944336449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf263b0 0x55b9ddf49040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.796+0000 7f75aae31740 1 --2- 172.21.15.2:0/3944336449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf22940 0x55b9ddf2cf60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.796+0000 7f75aae31740 1 -- 172.21.15.2:0/3944336449 >> 172.21.15.2:0/3944336449 conn(0x55b9ddf29290 msgr2=0x55b9ddf21c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.796+0000 7f75aae31740 1 -- 172.21.15.2:0/3944336449 shutdown_connections 2024-09-10T08:17:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.796+0000 7f75aae31740 1 -- 172.21.15.2:0/3944336449 wait complete. 2024-09-10T08:17:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.797+0000 7f75aae31740 1 -- start start 2024-09-10T08:17:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.797+0000 7f75aae31740 1 Processor -- start 2024-09-10T08:17:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.797+0000 7f75aae31740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf22940 0x55b9ddf39460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.797+0000 7f75aae31740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf263b0 0x55b9ddf399a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.797+0000 7f75aae31740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf36430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.797+0000 7f75aae31740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01af700 con 0x55b9ddf263b0 2024-09-10T08:17:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.797+0000 7f75aae31740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01b0b70 con 0x55b9ddf22940 2024-09-10T08:17:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.797+0000 7f75aae31740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01afdd0 con 0x55b9ddf26cd0 2024-09-10T08:17:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.797+0000 7f75aae31740 10 client.0 _ll_register_callbacks cb 0x7f7598009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.798+0000 7f75a5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf22940 0x55b9ddf39460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.798+0000 7f75a5000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf263b0 0x55b9ddf399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.798+0000 7f75a5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf22940 0x55b9ddf39460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:45552/0 (socket says 172.21.15.2:45552) 2024-09-10T08:17:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.798+0000 7f75a5a00640 1 -- 172.21.15.2:0/2416868890 learned_addr learned my addr 172.21.15.2:0/2416868890 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.798+0000 7f75a6400640 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf36430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.798+0000 7f75a5000640 1 -- 172.21.15.2:0/2416868890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 msgr2=0x55b9ddf36430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.798+0000 7f75a5000640 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf36430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.798+0000 7f75a5000640 1 -- 172.21.15.2:0/2416868890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf22940 msgr2=0x55b9ddf39460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:20.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.798+0000 7f75a5000640 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf22940 0x55b9ddf39460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.798+0000 7f75a5000640 1 -- 172.21.15.2:0/2416868890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9ddf3d2f0 con 0x55b9ddf263b0 2024-09-10T08:17:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.798+0000 7f75a6400640 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf36430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.798+0000 7f75a5a00640 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf22940 0x55b9ddf39460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.799+0000 7f75a5000640 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf263b0 0x55b9ddf399a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f758c005750 tx=0x7f758c005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578904c86b0f5bd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.799+0000 7f7587400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:17:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.799+0000 7f7587400640 1 -- 172.21.15.2:0/2416868890 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c02d060 con 0x55b9ddf263b0 2024-09-10T08:17:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.799+0000 7f75aae31740 1 -- 172.21.15.2:0/2416868890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75940bbff0 con 0x55b9ddf263b0 2024-09-10T08:17:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.799+0000 7f7587400640 1 -- 172.21.15.2:0/2416868890 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f758c004660 con 0x55b9ddf263b0 2024-09-10T08:17:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.799+0000 7f7587400640 1 -- 172.21.15.2:0/2416868890 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f758c034050 con 0x55b9ddf263b0 2024-09-10T08:17:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.799+0000 7f75aae31740 10 client.17964 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:20.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.799+0000 7f75aae31740 1 -- 172.21.15.2:0/2416868890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b9ddfac5c0 con 0x55b9ddf263b0 2024-09-10T08:17:20.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.799+0000 7f7587400640 1 -- 172.21.15.2:0/2416868890 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75940bbff0 con 0x55b9ddf263b0 2024-09-10T08:17:20.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.800+0000 7f7587400640 1 -- 172.21.15.2:0/2416868890 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f758c0288b0 con 0x55b9ddf263b0 2024-09-10T08:17:20.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.800+0000 7f75aae31740 10 client.17964 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:20.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.800+0000 7f75aae31740 10 client.17964 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:17:20.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.800+0000 7f75aae31740 20 client.17964 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:20.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.800+0000 7f75aae31740 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9ddf31590 0x55b9ddfaeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:20.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.800+0000 7f75aae31740 4 client.17964 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.6, "await_for": 45.0}] 2024-09-10T08:17:20.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.800+0000 7f75aae31740 1 -- 172.21.15.2:0/2416868890 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.6, "await_for": 45.0}) -- 0x55b9dde93fb0 con 0x55b9ddf31590 2024-09-10T08:17:20.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.800+0000 7f75a5a00640 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9ddf31590 0x55b9ddfaeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:20.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.801+0000 7f75a5a00640 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9ddf31590 0x55b9ddfaeb80 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f98c6ddffed534dc server_cookie=2145c5f8d27a08c2 in_seq=0 out_seq=0 2024-09-10T08:17:20.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.801+0000 7f7587400640 10 client.17964 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:20.936 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:17:20.936 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:17:20.936 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:17:20.937 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 30, 2024-09-10T08:17:20.937 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:17:20.937 INFO:teuthology.orchestra.run.smithi002.stdout: "2bebef3": { 2024-09-10T08:17:20.937 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 30, 2024-09-10T08:17:20.937 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:17:20.937 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:17:20.937 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:17:20.937 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:17:20.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:17:20.938 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:17:20.938 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 144.6, 2024-09-10T08:17:20.938 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:17:20.938 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:17:20.938 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:17:20.938 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:17:20.938 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:17:20.938 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:17:20.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:17:20.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:17:20.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:17:20.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:17:20.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:17:20.939 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:17:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.811+0000 7f7587400640 1 -- 172.21.15.2:0/2416868890 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55b9dde93fb0 con 0x55b9ddf31590 2024-09-10T08:17:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.811+0000 7f7587400640 10 client.17964 handle_command_reply: tid=0 2024-09-10T08:17:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.811+0000 7f75aae31740 1 client.17964 shutdown 2024-09-10T08:17:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.811+0000 7f75aae31740 1 -- 172.21.15.2:0/2416868890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf263b0 msgr2=0x55b9ddf399a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.811+0000 7f75aae31740 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf263b0 0x55b9ddf399a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f758c005750 tx=0x7f758c005b40 comp rx=0 tx=0).stop 2024-09-10T08:17:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.811+0000 7f75aae31740 1 -- 172.21.15.2:0/2416868890 shutdown_connections 2024-09-10T08:17:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.811+0000 7f75aae31740 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9ddf31590 0x55b9ddfaeb80 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.811+0000 7f75aae31740 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ddf26cd0 0x55b9ddf36430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.811+0000 7f75aae31740 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9ddf263b0 0x55b9ddf399a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.811+0000 7f75aae31740 1 --2- 172.21.15.2:0/2416868890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9ddf22940 0x55b9ddf39460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.811+0000 7f75aae31740 1 -- 172.21.15.2:0/2416868890 >> 172.21.15.2:0/2416868890 conn(0x55b9ddf29290 msgr2=0x55b9ddf27e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:20.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.811+0000 7f7587400640 0 client.17964 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:20.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.812+0000 7f75aae31740 1 -- 172.21.15.2:0/2416868890 shutdown_connections 2024-09-10T08:17:20.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.812+0000 7f75aae31740 1 -- 172.21.15.2:0/2416868890 wait complete. 2024-09-10T08:17:20.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.812+0000 7f75aae31740 20 client.17964 trim_cache size 0 max 16384 2024-09-10T08:17:20.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.812+0000 7f75a8400640 1 -- 172.21.15.2:0/149876778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75880a2b80 msgr2=0x7f75880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:20.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.812+0000 7f75a8400640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75880a2b80 0x7f75880a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7598009fd0 tx=0x7f7598009500 comp rx=0 tx=0).stop 2024-09-10T08:17:20.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.812+0000 7f75a8400640 1 -- 172.21.15.2:0/149876778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0101de0 msgr2=0x7f75a019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:20.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.812+0000 7f75a8400640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0101de0 0x7f75a019a670 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7594004770 tx=0x7f7594003c10 comp rx=0 tx=0).stop 2024-09-10T08:17:20.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.812+0000 7f75a8400640 1 -- 172.21.15.2:0/149876778 shutdown_connections 2024-09-10T08:17:20.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.812+0000 7f75a8400640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a0106f10 0x7f75a019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.812+0000 7f75a8400640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75880a2b80 0x7f75880a5040 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.812+0000 7f75a8400640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75a0102790 0x7f75a019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.812+0000 7f75a8400640 1 --2- 172.21.15.2:0/149876778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75a0101de0 0x7f75a019a670 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:20.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.812+0000 7f75a8400640 1 -- 172.21.15.2:0/149876778 >> 172.21.15.2:0/149876778 conn(0x7f75a00fd8c0 msgr2=0x7f75a00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:20.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.813+0000 7f75a8400640 1 -- 172.21.15.2:0/149876778 shutdown_connections 2024-09-10T08:17:20.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:20.813+0000 7f75a8400640 1 -- 172.21.15.2:0/149876778 wait complete. 2024-09-10T08:17:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:21 smithi002 ceph-mon[30563]: pgmap v663: 129 pgs: 129 active+clean; 794 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-10T08:17:21.821 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2bebef3', quiesce took 4.4 seconds. Will release after: 20.200000000000003 2024-09-10T08:17:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:21 smithi155 ceph-mon[35405]: pgmap v663: 129 pgs: 129 active+clean; 794 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-10T08:17:21.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:21 smithi107 ceph-mon[35048]: pgmap v663: 129 pgs: 129 active+clean; 794 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-10T08:17:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:23 smithi155 ceph-mon[35405]: pgmap v664: 129 pgs: 129 active+clean; 794 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:17:23.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:23 smithi107 ceph-mon[35048]: pgmap v664: 129 pgs: 129 active+clean; 794 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:17:24.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:23 smithi002 ceph-mon[30563]: pgmap v664: 129 pgs: 129 active+clean; 794 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:17:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:25 smithi155 ceph-mon[35405]: pgmap v665: 129 pgs: 129 active+clean; 829 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:17:25.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:25 smithi107 ceph-mon[35048]: pgmap v665: 129 pgs: 129 active+clean; 829 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:17:26.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:25 smithi002 ceph-mon[30563]: pgmap v665: 129 pgs: 129 active+clean; 829 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:17:27.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:27 smithi155 ceph-mon[35405]: pgmap v666: 129 pgs: 129 active+clean; 829 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:17:27.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:27 smithi107 ceph-mon[35048]: pgmap v666: 129 pgs: 129 active+clean; 829 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:17:28.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:27 smithi002 ceph-mon[30563]: pgmap v666: 129 pgs: 129 active+clean; 829 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:17:28.599 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:17:28.989 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:17:29.662 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:29 smithi002 ceph-mon[30563]: pgmap v667: 129 pgs: 129 active+clean; 836 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:17:29.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.678+0000 7f22e7200640 1 -- 172.21.15.2:0/584256660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e80745c0 msgr2=0x7f22e80749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:29.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.678+0000 7f22e7200640 1 --2- 172.21.15.2:0/584256660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e80745c0 0x7f22e80749a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009ca0 tx=0x7f22d802a950 comp rx=0 tx=0).stop 2024-09-10T08:17:29.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.678+0000 7f22e7200640 1 -- 172.21.15.2:0/584256660 shutdown_connections 2024-09-10T08:17:29.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.678+0000 7f22e7200640 1 --2- 172.21.15.2:0/584256660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e810e260 0x7f22e8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.678+0000 7f22e7200640 1 --2- 172.21.15.2:0/584256660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22e8074ee0 0x7f22e810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.678+0000 7f22e7200640 1 --2- 172.21.15.2:0/584256660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e80745c0 0x7f22e80749a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.678+0000 7f22e7200640 1 -- 172.21.15.2:0/584256660 >> 172.21.15.2:0/584256660 conn(0x7f22e80fd740 msgr2=0x7f22e80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:29.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.678+0000 7f22e7200640 1 -- 172.21.15.2:0/584256660 shutdown_connections 2024-09-10T08:17:29.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.678+0000 7f22e7200640 1 -- 172.21.15.2:0/584256660 wait complete. 2024-09-10T08:17:29.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.679+0000 7f22e7200640 1 Processor -- start 2024-09-10T08:17:29.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.679+0000 7f22e7200640 1 -- start start 2024-09-10T08:17:29.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.680+0000 7f22e7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80745c0 0x7f22e819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.680+0000 7f22e7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22e8074ee0 0x7f22e819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.680+0000 7f22e7200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e810e260 0x7f22e819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.680+0000 7f22e7200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f22e8114000 con 0x7f22e810e260 2024-09-10T08:17:29.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.680+0000 7f22e7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f22e8113e80 con 0x7f22e80745c0 2024-09-10T08:17:29.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.680+0000 7f22e7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22e8114180 con 0x7f22e8074ee0 2024-09-10T08:17:29.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.680+0000 7f22e5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80745c0 0x7f22e819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.680+0000 7f22e6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e810e260 0x7f22e819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.680+0000 7f22e5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80745c0 0x7f22e819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:57276/0 (socket says 172.21.15.2:57276) 2024-09-10T08:17:29.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.680+0000 7f22dd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22e8074ee0 0x7f22e819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.680+0000 7f22e5e00640 1 -- 172.21.15.2:0/2674517952 learned_addr learned my addr 172.21.15.2:0/2674517952 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:29.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.680+0000 7f22e6800640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e810e260 0x7f22e819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49914/0 (socket says 172.21.15.2:49914) 2024-09-10T08:17:29.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.681+0000 7f22e5e00640 1 -- 172.21.15.2:0/2674517952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22e8074ee0 msgr2=0x7f22e819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:29.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.681+0000 7f22e5e00640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22e8074ee0 0x7f22e819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.681+0000 7f22e5e00640 1 -- 172.21.15.2:0/2674517952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e810e260 msgr2=0x7f22e819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:29.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.681+0000 7f22e5e00640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e810e260 0x7f22e819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.681+0000 7f22e5e00640 1 -- 172.21.15.2:0/2674517952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e819f450 con 0x7f22e80745c0 2024-09-10T08:17:29.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.681+0000 7f22e6800640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e810e260 0x7f22e819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:29.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.681+0000 7f22e5e00640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80745c0 0x7f22e819a4a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f22d8005e30 tx=0x7f22d8003c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:29.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.681+0000 7f22df400640 1 -- 172.21.15.2:0/2674517952 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8034040 con 0x7f22e80745c0 2024-09-10T08:17:29.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.681+0000 7f22df400640 1 -- 172.21.15.2:0/2674517952 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22d802fe20 con 0x7f22e80745c0 2024-09-10T08:17:29.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.681+0000 7f22e7200640 1 -- 172.21.15.2:0/2674517952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e819f6e0 con 0x7f22e80745c0 2024-09-10T08:17:29.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.681+0000 7f22df400640 1 -- 172.21.15.2:0/2674517952 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d80047c0 con 0x7f22e80745c0 2024-09-10T08:17:29.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.681+0000 7f22e7200640 1 -- 172.21.15.2:0/2674517952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e81a7020 con 0x7f22e80745c0 2024-09-10T08:17:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.686+0000 7f22ef4d1740 1 Processor -- start 2024-09-10T08:17:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.686+0000 7f22df400640 1 -- 172.21.15.2:0/2674517952 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f22d8037020 con 0x7f22e80745c0 2024-09-10T08:17:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.686+0000 7f22ef4d1740 1 -- start start 2024-09-10T08:17:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.686+0000 7f22ef4d1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9ead50 0x5594bd9eb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.686+0000 7f22ef4d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9eb670 0x5594bd9f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.686+0000 7f22ef4d1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9f5f30 0x5594bd9f8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.686+0000 7f22ef4d1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5594bd9513e0 con 0x5594bd9ead50 2024-09-10T08:17:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.686+0000 7f22ef4d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5594bd8f8d80 con 0x5594bd9eb670 2024-09-10T08:17:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.686+0000 7f22ef4d1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5594bd94d3a0 con 0x5594bd9f5f30 2024-09-10T08:17:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.686+0000 7f22e6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9ead50 0x5594bd9eb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.687+0000 7f22e6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9ead50 0x5594bd9eb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49926/0 (socket says 172.21.15.2:49926) 2024-09-10T08:17:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.687+0000 7f22e6800640 1 -- 172.21.15.2:0/3888798771 learned_addr learned my addr 172.21.15.2:0/3888798771 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.687+0000 7f22dd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9f5f30 0x5594bd9f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.687+0000 7f22e5e00640 1 --2- 172.21.15.2:0/3888798771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9eb670 0x5594bd9f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.687+0000 7f22e6800640 1 -- 172.21.15.2:0/3888798771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9f5f30 msgr2=0x5594bd9f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.687+0000 7f22e6800640 1 --2- 172.21.15.2:0/3888798771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9f5f30 0x5594bd9f8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.687+0000 7f22e6800640 1 -- 172.21.15.2:0/3888798771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9eb670 msgr2=0x5594bd9f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.687+0000 7f22e6800640 1 --2- 172.21.15.2:0/3888798771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9eb670 0x5594bd9f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.687+0000 7f22e6800640 1 -- 172.21.15.2:0/3888798771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594bd917ae0 con 0x5594bd9ead50 2024-09-10T08:17:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.687+0000 7f22e6800640 1 --2- 172.21.15.2:0/3888798771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9ead50 0x5594bd9eb130 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009f90 tx=0x7f22e00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbff3c6c73b6d17c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.688+0000 7f22d7000640 1 -- 172.21.15.2:0/3888798771 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e001e070 con 0x5594bd9ead50 2024-09-10T08:17:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.688+0000 7f22df400640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22cc0a2b80 0x7f22cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.688+0000 7f22d7000640 1 -- 172.21.15.2:0/3888798771 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e0004310 con 0x5594bd9ead50 2024-09-10T08:17:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.688+0000 7f22d7000640 1 -- 172.21.15.2:0/3888798771 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e000f9a0 con 0x5594bd9ead50 2024-09-10T08:17:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.688+0000 7f22df400640 1 -- 172.21.15.2:0/2674517952 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f22d80f0260 con 0x7f22e80745c0 2024-09-10T08:17:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.688+0000 7f22e5e00640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22cc0a2b80 0x7f22cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.688+0000 7f22ef4d1740 1 -- 172.21.15.2:0/3888798771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9ead50 msgr2=0x5594bd9eb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.688+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/3888798771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9ead50 0x5594bd9eb130 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009f90 tx=0x7f22e00118d0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.689+0000 7f22ef4d1740 1 -- 172.21.15.2:0/3888798771 shutdown_connections 2024-09-10T08:17:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.689+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/3888798771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9f5f30 0x5594bd9f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.689+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/3888798771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9eb670 0x5594bd9f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.689+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/3888798771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9ead50 0x5594bd9eb130 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.689+0000 7f22ef4d1740 1 -- 172.21.15.2:0/3888798771 >> 172.21.15.2:0/3888798771 conn(0x5594bd9e72e0 msgr2=0x5594bd9e76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.689+0000 7f22ef4d1740 1 -- 172.21.15.2:0/3888798771 shutdown_connections 2024-09-10T08:17:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.689+0000 7f22e5e00640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22cc0a2b80 0x7f22cc0a5040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f22d80edce0 tx=0x7f22d80f3d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.689+0000 7f22ef4d1740 1 -- 172.21.15.2:0/3888798771 wait complete. 2024-09-10T08:17:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.689+0000 7f22ef4d1740 1 -- start start 2024-09-10T08:17:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.690+0000 7f22ef4d1740 1 Processor -- start 2024-09-10T08:17:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.690+0000 7f22ef4d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 0x5594bd9f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.690+0000 7f22ef4d1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9eb670 0x5594bd9f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.690+0000 7f22ef4d1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 0x5594bd9f3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.690+0000 7f22ef4d1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5594bd9f8ba0 con 0x5594bd9f5f30 2024-09-10T08:17:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.690+0000 7f22ef4d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5594bd90c0f0 con 0x5594bd9ead50 2024-09-10T08:17:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.690+0000 7f22ef4d1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5594bd9f8d20 con 0x5594bd9eb670 2024-09-10T08:17:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.690+0000 7f22ef4d1740 10 client.0 _ll_register_callbacks cb 0x5594bd8fd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.690+0000 7f22e6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 0x5594bd9f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.690+0000 7f22e6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 0x5594bd9f3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49936/0 (socket says 172.21.15.2:49936) 2024-09-10T08:17:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.690+0000 7f22e6800640 1 -- 172.21.15.2:0/51253964 learned_addr learned my addr 172.21.15.2:0/51253964 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.690+0000 7f22e6800640 1 --2- 172.21.15.2:0/51253964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 0x5594bd9f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.690+0000 7f22dd400640 1 --2- 172.21.15.2:0/51253964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9eb670 0x5594bd9f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.691+0000 7f22e6800640 1 -- 172.21.15.2:0/51253964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9eb670 msgr2=0x5594bd9f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.691+0000 7f22e6800640 1 --2- 172.21.15.2:0/51253964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9eb670 0x5594bd9f33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.691+0000 7f22e6800640 1 -- 172.21.15.2:0/51253964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 msgr2=0x5594bd9f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.691+0000 7f22e6800640 1 --2- 172.21.15.2:0/51253964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 0x5594bd9f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.691+0000 7f22e6800640 1 -- 172.21.15.2:0/51253964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594bd6c23c0 con 0x5594bd9f5f30 2024-09-10T08:17:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.691+0000 7f22e6800640 1 --2- 172.21.15.2:0/51253964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 0x5594bd9f3930 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f22e000f680 tx=0x7f22e000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274a000b396a4fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.691+0000 7f22cb400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:17:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.691+0000 7f22cb400640 1 -- 172.21.15.2:0/51253964 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e002d060 con 0x5594bd9f5f30 2024-09-10T08:17:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.691+0000 7f22ef4d1740 1 -- 172.21.15.2:0/51253964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e0004310 con 0x5594bd9f5f30 2024-09-10T08:17:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.691+0000 7f22cb400640 1 -- 172.21.15.2:0/51253964 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f22e0005650 con 0x5594bd9f5f30 2024-09-10T08:17:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.692+0000 7f22cb400640 1 -- 172.21.15.2:0/51253964 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f22e001e030 con 0x5594bd9f5f30 2024-09-10T08:17:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.692+0000 7f22ef4d1740 10 client.17976 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.692+0000 7f22ef4d1740 1 -- 172.21.15.2:0/51253964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594bda04900 con 0x5594bd9f5f30 2024-09-10T08:17:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.692+0000 7f22cb400640 1 -- 172.21.15.2:0/51253964 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e0004310 con 0x5594bd9f5f30 2024-09-10T08:17:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.692+0000 7f22cb400640 1 -- 172.21.15.2:0/51253964 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f22e00046c0 con 0x5594bd9f5f30 2024-09-10T08:17:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.692+0000 7f22ef4d1740 10 client.17976 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.693+0000 7f22ef4d1740 10 client.17976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.693+0000 7f22ef4d1740 20 client.17976 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.693+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/51253964 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594bda05d10 0x5594bda08100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.693+0000 7f22ef4d1740 4 client.17976 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:17:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.693+0000 7f22ef4d1740 1 -- 172.21.15.2:0/51253964 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594bd95e950 con 0x5594bda05d10 2024-09-10T08:17:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.693+0000 7f22dd400640 1 --2- 172.21.15.2:0/51253964 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594bda05d10 0x5594bda08100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.693+0000 7f22dd400640 1 --2- 172.21.15.2:0/51253964 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594bda05d10 0x5594bda08100 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4ee6094c857f772 server_cookie=26f60c8b2b8d79b in_seq=0 out_seq=0 2024-09-10T08:17:29.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.693+0000 7f22cb400640 10 client.17976 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:29.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.694+0000 7f22cb400640 1 -- 172.21.15.2:0/51253964 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594bd95e950 con 0x5594bda05d10 2024-09-10T08:17:29.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.694+0000 7f22cb400640 10 client.17976 handle_command_reply: tid=0 2024-09-10T08:17:29.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.694+0000 7f22ef4d1740 1 client.17976 shutdown 2024-09-10T08:17:29.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.694+0000 7f22ef4d1740 1 -- 172.21.15.2:0/51253964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 msgr2=0x5594bd9f3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:29.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.694+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/51253964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 0x5594bd9f3930 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f22e000f680 tx=0x7f22e000f580 comp rx=0 tx=0).stop 2024-09-10T08:17:29.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.695+0000 7f22ef4d1740 1 -- 172.21.15.2:0/51253964 shutdown_connections 2024-09-10T08:17:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.695+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/51253964 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594bda05d10 0x5594bda08100 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.695+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/51253964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 0x5594bd9f3930 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.695+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/51253964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9eb670 0x5594bd9f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.695+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/51253964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 0x5594bd9f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.695+0000 7f22ef4d1740 1 -- 172.21.15.2:0/51253964 >> 172.21.15.2:0/51253964 conn(0x5594bd9e72e0 msgr2=0x5594bd9ed010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.695+0000 7f22cb400640 0 client.17976 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.695+0000 7f22ef4d1740 1 -- 172.21.15.2:0/51253964 shutdown_connections 2024-09-10T08:17:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.695+0000 7f22ef4d1740 1 -- 172.21.15.2:0/51253964 wait complete. 2024-09-10T08:17:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.695+0000 7f22ef4d1740 20 client.17976 trim_cache size 0 max 16384 2024-09-10T08:17:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.713+0000 7f22ef4d1740 1 Processor -- start 2024-09-10T08:17:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.713+0000 7f22ef4d1740 1 -- start start 2024-09-10T08:17:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.713+0000 7f22ef4d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 0x5594bd9ed7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.713+0000 7f22ef4d1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9eb670 0x5594bd9e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.713+0000 7f22ef4d1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 0x5594bd9f3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.713+0000 7f22ef4d1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f22e81b3780 con 0x5594bd9f5f30 2024-09-10T08:17:29.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.713+0000 7f22ef4d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f22e81b1050 con 0x5594bd9ead50 2024-09-10T08:17:29.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.713+0000 7f22ef4d1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22e81aeea0 con 0x5594bd9eb670 2024-09-10T08:17:29.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.714+0000 7f22e6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 0x5594bd9f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.714+0000 7f22e6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 0x5594bd9f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49944/0 (socket says 172.21.15.2:49944) 2024-09-10T08:17:29.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.714+0000 7f22e6800640 1 -- 172.21.15.2:0/341715006 learned_addr learned my addr 172.21.15.2:0/341715006 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:29.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.714+0000 7f22e6800640 1 --2- 172.21.15.2:0/341715006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 0x5594bd9ed7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.714+0000 7f22dd400640 1 --2- 172.21.15.2:0/341715006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9eb670 0x5594bd9e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.714+0000 7f22e6800640 1 -- 172.21.15.2:0/341715006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9eb670 msgr2=0x5594bd9e76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:29.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.714+0000 7f22e6800640 1 --2- 172.21.15.2:0/341715006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9eb670 0x5594bd9e76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.714+0000 7f22e6800640 1 -- 172.21.15.2:0/341715006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 msgr2=0x5594bd9ed7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:29.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.714+0000 7f22e6800640 1 --2- 172.21.15.2:0/341715006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 0x5594bd9ed7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.714+0000 7f22e6800640 1 -- 172.21.15.2:0/341715006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594bd9f1e60 con 0x5594bd9f5f30 2024-09-10T08:17:29.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.714+0000 7f22e6800640 1 --2- 172.21.15.2:0/341715006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 0x5594bd9f3580 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f22e0006010 tx=0x7f22e00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c749429c1845eec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:29.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.714+0000 7f22d6600640 1 -- 172.21.15.2:0/341715006 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0005630 con 0x5594bd9f5f30 2024-09-10T08:17:29.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.715+0000 7f22d6600640 1 -- 172.21.15.2:0/341715006 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e0015040 con 0x5594bd9f5f30 2024-09-10T08:17:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.715+0000 7f22d6600640 1 -- 172.21.15.2:0/341715006 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0004340 con 0x5594bd9f5f30 2024-09-10T08:17:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.715+0000 7f22ef4d1740 1 -- 172.21.15.2:0/341715006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 msgr2=0x5594bd9f3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.715+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/341715006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 0x5594bd9f3580 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f22e0006010 tx=0x7f22e00118d0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.715+0000 7f22ef4d1740 1 -- 172.21.15.2:0/341715006 shutdown_connections 2024-09-10T08:17:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.715+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/341715006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9f5f30 0x5594bd9f3580 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.715+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/341715006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9eb670 0x5594bd9e76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.715+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/341715006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 0x5594bd9ed7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.715+0000 7f22ef4d1740 1 -- 172.21.15.2:0/341715006 >> 172.21.15.2:0/341715006 conn(0x5594bd9e72e0 msgr2=0x5594bd9f2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.715+0000 7f22ef4d1740 1 -- 172.21.15.2:0/341715006 shutdown_connections 2024-09-10T08:17:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.715+0000 7f22ef4d1740 1 -- 172.21.15.2:0/341715006 wait complete. 2024-09-10T08:17:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.716+0000 7f22ef4d1740 1 -- start start 2024-09-10T08:17:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.716+0000 7f22ef4d1740 1 Processor -- start 2024-09-10T08:17:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.716+0000 7f22ef4d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 0x5594bd9f9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.716+0000 7f22ef4d1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9eb670 0x5594bd9fa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.716+0000 7f22ef4d1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9f5f30 0x5594bd9fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.716+0000 7f22ef4d1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f22e81af510 con 0x5594bd9eb670 2024-09-10T08:17:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.716+0000 7f22ef4d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f22e81b0980 con 0x5594bd9ead50 2024-09-10T08:17:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.716+0000 7f22ef4d1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22e81afbe0 con 0x5594bd9f5f30 2024-09-10T08:17:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.716+0000 7f22ef4d1740 10 client.0 _ll_register_callbacks cb 0x7f22e0009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.717+0000 7f22dd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9eb670 0x5594bd9fa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.717+0000 7f22dd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9eb670 0x5594bd9fa290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49946/0 (socket says 172.21.15.2:49946) 2024-09-10T08:17:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.717+0000 7f22dd400640 1 -- 172.21.15.2:0/874154297 learned_addr learned my addr 172.21.15.2:0/874154297 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.717+0000 7f22e6800640 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 0x5594bd9f9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.717+0000 7f22e6800640 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9f5f30 0x5594bd9fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.717+0000 7f22dd400640 1 -- 172.21.15.2:0/874154297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9f5f30 msgr2=0x5594bd9fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.717+0000 7f22dd400640 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9f5f30 0x5594bd9fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.717+0000 7f22dd400640 1 -- 172.21.15.2:0/874154297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 msgr2=0x5594bd9f9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.717+0000 7f22dd400640 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 0x5594bd9f9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.717+0000 7f22dd400640 1 -- 172.21.15.2:0/874154297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594bda6f840 con 0x5594bd9eb670 2024-09-10T08:17:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.717+0000 7f22e6800640 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9f5f30 0x5594bd9fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.717+0000 7f22dd400640 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9eb670 0x5594bd9fa290 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f22e810f5f0 tx=0x7f22d001f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e28a5cf049b489c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.717+0000 7f22cb400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:17:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.718+0000 7f22cb400640 1 -- 172.21.15.2:0/874154297 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d002b060 con 0x5594bd9eb670 2024-09-10T08:17:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.718+0000 7f22cb400640 1 -- 172.21.15.2:0/874154297 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f22d0010ab0 con 0x5594bd9eb670 2024-09-10T08:17:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.718+0000 7f22ef4d1740 1 -- 172.21.15.2:0/874154297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e0015040 con 0x5594bd9eb670 2024-09-10T08:17:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.718+0000 7f22cb400640 1 -- 172.21.15.2:0/874154297 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f22d0032050 con 0x5594bd9eb670 2024-09-10T08:17:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.718+0000 7f22ef4d1740 10 client.17982 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.718+0000 7f22ef4d1740 1 -- 172.21.15.2:0/874154297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594bda70bc0 con 0x5594bd9eb670 2024-09-10T08:17:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.718+0000 7f22cb400640 1 -- 172.21.15.2:0/874154297 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e0015040 con 0x5594bd9eb670 2024-09-10T08:17:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.718+0000 7f22cb400640 1 -- 172.21.15.2:0/874154297 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f22d000f3c0 con 0x5594bd9eb670 2024-09-10T08:17:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.719+0000 7f22ef4d1740 10 client.17982 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.719+0000 7f22ef4d1740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.719+0000 7f22ef4d1740 20 client.17982 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.719+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594bda71340 0x5594bda73760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.719+0000 7f22ef4d1740 4 client.17982 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:17:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.719+0000 7f22ef4d1740 1 -- 172.21.15.2:0/874154297 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594bd958e60 con 0x5594bda71340 2024-09-10T08:17:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.719+0000 7f22e6800640 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594bda71340 0x5594bda73760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22e6800640 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594bda71340 0x5594bda73760 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95cd1e3243c0f74d server_cookie=4683f29ae74ac0bc in_seq=0 out_seq=0 2024-09-10T08:17:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22cb400640 10 client.17982 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22cb400640 1 -- 172.21.15.2:0/874154297 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5594bd958e60 con 0x5594bda71340 2024-09-10T08:17:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22cb400640 10 client.17982 handle_command_reply: tid=0 2024-09-10T08:17:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22ef4d1740 1 client.17982 shutdown 2024-09-10T08:17:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22ef4d1740 1 -- 172.21.15.2:0/874154297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9eb670 msgr2=0x5594bd9fa290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9eb670 0x5594bd9fa290 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f22e810f5f0 tx=0x7f22d001f990 comp rx=0 tx=0).stop 2024-09-10T08:17:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22ef4d1740 1 -- 172.21.15.2:0/874154297 shutdown_connections 2024-09-10T08:17:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5594bda71340 0x5594bda73760 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594bd9f5f30 0x5594bd9fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5594bd9eb670 0x5594bd9fa290 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22ef4d1740 1 --2- 172.21.15.2:0/874154297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5594bd9ead50 0x5594bd9f9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22cb400640 0 client.17982 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22ef4d1740 1 -- 172.21.15.2:0/874154297 >> 172.21.15.2:0/874154297 conn(0x5594bd9e72e0 msgr2=0x5594bd9e6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22ef4d1740 1 -- 172.21.15.2:0/874154297 shutdown_connections 2024-09-10T08:17:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22ef4d1740 1 -- 172.21.15.2:0/874154297 wait complete. 2024-09-10T08:17:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.720+0000 7f22ef4d1740 20 client.17982 trim_cache size 0 max 16384 2024-09-10T08:17:29.727 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:17:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:17:29.727 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:17:29.727 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:17:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.721+0000 7f22e7200640 1 -- 172.21.15.2:0/2674517952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22cc0a2b80 msgr2=0x7f22cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.721+0000 7f22e7200640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22cc0a2b80 0x7f22cc0a5040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f22d80edce0 tx=0x7f22d80f3d40 comp rx=0 tx=0).stop 2024-09-10T08:17:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.721+0000 7f22e7200640 1 -- 172.21.15.2:0/2674517952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80745c0 msgr2=0x7f22e819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.721+0000 7f22e7200640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80745c0 0x7f22e819a4a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f22d8005e30 tx=0x7f22d8003c90 comp rx=0 tx=0).stop 2024-09-10T08:17:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.721+0000 7f22e7200640 1 -- 172.21.15.2:0/2674517952 shutdown_connections 2024-09-10T08:17:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.721+0000 7f22e7200640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e810e260 0x7f22e819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.721+0000 7f22e7200640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22cc0a2b80 0x7f22cc0a5040 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.721+0000 7f22e7200640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22e8074ee0 0x7f22e819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.721+0000 7f22e7200640 1 --2- 172.21.15.2:0/2674517952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80745c0 0x7f22e819a4a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.721+0000 7f22e7200640 1 -- 172.21.15.2:0/2674517952 >> 172.21.15.2:0/2674517952 conn(0x7f22e80fd740 msgr2=0x7f22e80fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.721+0000 7f22e7200640 1 -- 172.21.15.2:0/2674517952 shutdown_connections 2024-09-10T08:17:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:29.721+0000 7f22e7200640 1 -- 172.21.15.2:0/2674517952 wait complete. 2024-09-10T08:17:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:29 smithi155 ceph-mon[35405]: pgmap v667: 129 pgs: 129 active+clean; 836 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:17:29.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:29 smithi107 ceph-mon[35048]: pgmap v667: 129 pgs: 129 active+clean; 836 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:17:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.240+0000 7fc795200640 1 -- 172.21.15.2:0/103511941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc790077df0 msgr2=0x7fc790078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.240+0000 7fc795200640 1 --2- 172.21.15.2:0/103511941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc790077df0 0x7fc790078250 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009900 tx=0x7fc77c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.240+0000 7fc795200640 1 -- 172.21.15.2:0/103511941 shutdown_connections 2024-09-10T08:17:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.240+0000 7fc795200640 1 --2- 172.21.15.2:0/103511941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc790078790 0x7fc790079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.240+0000 7fc795200640 1 --2- 172.21.15.2:0/103511941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc790077df0 0x7fc790078250 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.240+0000 7fc795200640 1 --2- 172.21.15.2:0/103511941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc790076bb0 0x7fc790076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.240+0000 7fc795200640 1 -- 172.21.15.2:0/103511941 >> 172.21.15.2:0/103511941 conn(0x7fc7900ff310 msgr2=0x7fc790101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.240+0000 7fc795200640 1 -- 172.21.15.2:0/103511941 shutdown_connections 2024-09-10T08:17:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.241+0000 7fc795200640 1 -- 172.21.15.2:0/103511941 wait complete. 2024-09-10T08:17:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.241+0000 7fc795200640 1 Processor -- start 2024-09-10T08:17:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.241+0000 7fc795200640 1 -- start start 2024-09-10T08:17:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc795200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc790076bb0 0x7fc79019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc795200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc790077df0 0x7fc79019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc795200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc790078790 0x7fc7901a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc795200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc790113c50 con 0x7fc790078790 2024-09-10T08:17:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc795200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc790113ad0 con 0x7fc790077df0 2024-09-10T08:17:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc795200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc790113dd0 con 0x7fc790076bb0 2024-09-10T08:17:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc78fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc790078790 0x7fc7901a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc78f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc790076bb0 0x7fc79019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc78fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc790078790 0x7fc7901a5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49952/0 (socket says 172.21.15.2:49952) 2024-09-10T08:17:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc78fe00640 1 -- 172.21.15.2:0/3666761942 learned_addr learned my addr 172.21.15.2:0/3666761942 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc78ea00640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc790077df0 0x7fc79019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc78fe00640 1 -- 172.21.15.2:0/3666761942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc790076bb0 msgr2=0x7fc79019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc78fe00640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc790076bb0 0x7fc79019e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc78fe00640 1 -- 172.21.15.2:0/3666761942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc790077df0 msgr2=0x7fc79019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc78fe00640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc790077df0 0x7fc79019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc78fe00640 1 -- 172.21.15.2:0/3666761942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7901a5ea0 con 0x7fc790078790 2024-09-10T08:17:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.242+0000 7fc78ea00640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc790077df0 0x7fc79019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.243+0000 7fc78fe00640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc790078790 0x7fc7901a5830 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc78800eb10 tx=0x7fc78800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.243+0000 7fc787e00640 1 -- 172.21.15.2:0/3666761942 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788004b20 con 0x7fc790078790 2024-09-10T08:17:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.243+0000 7fc787e00640 1 -- 172.21.15.2:0/3666761942 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc788004cc0 con 0x7fc790078790 2024-09-10T08:17:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.243+0000 7fc787e00640 1 -- 172.21.15.2:0/3666761942 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788010760 con 0x7fc790078790 2024-09-10T08:17:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.243+0000 7fc795200640 1 -- 172.21.15.2:0/3666761942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7901a6190 con 0x7fc790078790 2024-09-10T08:17:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.243+0000 7fc795200640 1 -- 172.21.15.2:0/3666761942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7901a6660 con 0x7fc790078790 2024-09-10T08:17:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.246+0000 7fc7997dc740 1 Processor -- start 2024-09-10T08:17:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.246+0000 7fc787e00640 1 -- 172.21.15.2:0/3666761942 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc788005020 con 0x7fc790078790 2024-09-10T08:17:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.247+0000 7fc7997dc740 1 -- start start 2024-09-10T08:17:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.247+0000 7fc7997dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951260 0x56553e951660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.247+0000 7fc7997dc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951ba0 0x56553e956660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.247+0000 7fc7997dc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 0x56553e9590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.247+0000 7fc7997dc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56553e8b6100 con 0x56553e951ba0 2024-09-10T08:17:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.247+0000 7fc7997dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56553e85e1b0 con 0x56553e951260 2024-09-10T08:17:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.247+0000 7fc7997dc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56553e8b20c0 con 0x56553e956c30 2024-09-10T08:17:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.247+0000 7fc78f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951ba0 0x56553e956660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.247+0000 7fc78f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951ba0 0x56553e956660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49962/0 (socket says 172.21.15.2:49962) 2024-09-10T08:17:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.247+0000 7fc78f400640 1 -- 172.21.15.2:0/1863689373 learned_addr learned my addr 172.21.15.2:0/1863689373 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.247+0000 7fc78ea00640 1 --2- 172.21.15.2:0/1863689373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951260 0x56553e951660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.247+0000 7fc78ea00640 1 --2- 172.21.15.2:0/1863689373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 0x56553e9590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.248+0000 7fc78f400640 1 -- 172.21.15.2:0/1863689373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 msgr2=0x56553e9590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.248+0000 7fc78f400640 1 --2- 172.21.15.2:0/1863689373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 0x56553e9590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.248+0000 7fc78f400640 1 -- 172.21.15.2:0/1863689373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951260 msgr2=0x56553e951660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.248+0000 7fc78f400640 1 --2- 172.21.15.2:0/1863689373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951260 0x56553e951660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.248+0000 7fc78f400640 1 -- 172.21.15.2:0/1863689373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56553e5c9210 con 0x56553e951ba0 2024-09-10T08:17:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.248+0000 7fc78ea00640 1 --2- 172.21.15.2:0/1863689373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951260 0x56553e951660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.248+0000 7fc787e00640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7740a2c00 0x7fc7740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.248+0000 7fc787e00640 1 -- 172.21.15.2:0/3666761942 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc7880cfa10 con 0x7fc790078790 2024-09-10T08:17:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.248+0000 7fc78f400640 1 --2- 172.21.15.2:0/1863689373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951ba0 0x56553e956660 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc780009d20 tx=0x7fc780011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebaf67c2e2f745 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.248+0000 7fc78ea00640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7740a2c00 0x7fc7740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.248+0000 7fc784c00640 1 -- 172.21.15.2:0/1863689373 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78001e070 con 0x56553e951ba0 2024-09-10T08:17:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.248+0000 7fc784c00640 1 -- 172.21.15.2:0/1863689373 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc780019b10 con 0x56553e951ba0 2024-09-10T08:17:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.248+0000 7fc784c00640 1 -- 172.21.15.2:0/1863689373 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78000eb90 con 0x56553e951ba0 2024-09-10T08:17:30.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.249+0000 7fc7997dc740 1 -- 172.21.15.2:0/1863689373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951ba0 msgr2=0x56553e956660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:30.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.249+0000 7fc78ea00640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7740a2c00 0x7fc7740a50c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc77c006010 tx=0x7fc77c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:30.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.249+0000 7fc7997dc740 1 --2- 172.21.15.2:0/1863689373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951ba0 0x56553e956660 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc780009d20 tx=0x7fc780011810 comp rx=0 tx=0).stop 2024-09-10T08:17:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.249+0000 7fc7997dc740 1 -- 172.21.15.2:0/1863689373 shutdown_connections 2024-09-10T08:17:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.249+0000 7fc7997dc740 1 --2- 172.21.15.2:0/1863689373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 0x56553e9590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.249+0000 7fc7997dc740 1 --2- 172.21.15.2:0/1863689373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951ba0 0x56553e956660 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.249+0000 7fc7997dc740 1 --2- 172.21.15.2:0/1863689373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951260 0x56553e951660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.249+0000 7fc7997dc740 1 -- 172.21.15.2:0/1863689373 >> 172.21.15.2:0/1863689373 conn(0x56553e94c8d0 msgr2=0x56553e94ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.249+0000 7fc7997dc740 1 -- 172.21.15.2:0/1863689373 shutdown_connections 2024-09-10T08:17:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.249+0000 7fc7997dc740 1 -- 172.21.15.2:0/1863689373 wait complete. 2024-09-10T08:17:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.250+0000 7fc7997dc740 1 -- start start 2024-09-10T08:17:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.250+0000 7fc7997dc740 1 Processor -- start 2024-09-10T08:17:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.250+0000 7fc7997dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951260 0x56553e95b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.250+0000 7fc7997dc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951ba0 0x56553e95b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.250+0000 7fc7997dc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 0x56553e962970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.250+0000 7fc7997dc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56553e959c00 con 0x56553e951ba0 2024-09-10T08:17:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.250+0000 7fc7997dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56553e83f090 con 0x56553e951260 2024-09-10T08:17:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.250+0000 7fc7997dc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56553e959d80 con 0x56553e956c30 2024-09-10T08:17:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.250+0000 7fc7997dc740 10 client.0 _ll_register_callbacks cb 0x56553e893f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc78f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951260 0x56553e95b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc78fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 0x56553e962970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc78f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951260 0x56553e95b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:57340/0 (socket says 172.21.15.2:57340) 2024-09-10T08:17:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc78f400640 1 -- 172.21.15.2:0/3534573005 learned_addr learned my addr 172.21.15.2:0/3534573005 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc78ea00640 1 --2- 172.21.15.2:0/3534573005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951ba0 0x56553e95b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc78fe00640 1 -- 172.21.15.2:0/3534573005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951260 msgr2=0x56553e95b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc78fe00640 1 --2- 172.21.15.2:0/3534573005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951260 0x56553e95b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc78fe00640 1 -- 172.21.15.2:0/3534573005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951ba0 msgr2=0x56553e95b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc78fe00640 1 --2- 172.21.15.2:0/3534573005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951ba0 0x56553e95b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc78fe00640 1 -- 172.21.15.2:0/3534573005 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56553e695620 con 0x56553e956c30 2024-09-10T08:17:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc78ea00640 1 --2- 172.21.15.2:0/3534573005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951ba0 0x56553e95b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc78fe00640 1 --2- 172.21.15.2:0/3534573005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 0x56553e962970 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc7880cd170 tx=0x7fc78809b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2bf343362feea76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc779600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:17:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc779600640 1 -- 172.21.15.2:0/3534573005 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7880a4860 con 0x56553e956c30 2024-09-10T08:17:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc7997dc740 1 -- 172.21.15.2:0/3534573005 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc780019b10 con 0x56553e956c30 2024-09-10T08:17:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.251+0000 7fc779600640 1 -- 172.21.15.2:0/3534573005 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc788005310 con 0x56553e956c30 2024-09-10T08:17:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.252+0000 7fc779600640 1 -- 172.21.15.2:0/3534573005 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc7880a0230 con 0x56553e956c30 2024-09-10T08:17:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.252+0000 7fc7997dc740 10 client.27955 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.252+0000 7fc7997dc740 1 -- 172.21.15.2:0/3534573005 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56553e9657d0 con 0x56553e956c30 2024-09-10T08:17:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.252+0000 7fc779600640 1 -- 172.21.15.2:0/3534573005 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc780019b10 con 0x56553e956c30 2024-09-10T08:17:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.252+0000 7fc779600640 1 -- 172.21.15.2:0/3534573005 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc7880baac0 con 0x56553e956c30 2024-09-10T08:17:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.252+0000 7fc7997dc740 10 client.27955 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.252+0000 7fc7997dc740 10 client.27955 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.252+0000 7fc7997dc740 20 client.27955 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.252+0000 7fc7997dc740 1 --2- 172.21.15.2:0/3534573005 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56553e966f10 0x56553e969300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.252+0000 7fc7997dc740 4 client.27955 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:17:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.252+0000 7fc7997dc740 1 -- 172.21.15.2:0/3534573005 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56553e8c3830 con 0x56553e966f10 2024-09-10T08:17:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.253+0000 7fc78f400640 1 --2- 172.21.15.2:0/3534573005 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56553e966f10 0x56553e969300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.253+0000 7fc78f400640 1 --2- 172.21.15.2:0/3534573005 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56553e966f10 0x56553e969300 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7495883639dcea server_cookie=f05d48e1b7463e5f in_seq=0 out_seq=0 2024-09-10T08:17:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.253+0000 7fc779600640 10 client.27955 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.254+0000 7fc779600640 1 -- 172.21.15.2:0/3534573005 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56553e8c3830 con 0x56553e966f10 2024-09-10T08:17:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.254+0000 7fc779600640 10 client.27955 handle_command_reply: tid=0 2024-09-10T08:17:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.254+0000 7fc7997dc740 1 client.27955 shutdown 2024-09-10T08:17:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.254+0000 7fc7997dc740 1 -- 172.21.15.2:0/3534573005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 msgr2=0x56553e962970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.254+0000 7fc7997dc740 1 --2- 172.21.15.2:0/3534573005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 0x56553e962970 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc7880cd170 tx=0x7fc78809b7e0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.254+0000 7fc7997dc740 1 -- 172.21.15.2:0/3534573005 shutdown_connections 2024-09-10T08:17:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.254+0000 7fc7997dc740 1 --2- 172.21.15.2:0/3534573005 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56553e966f10 0x56553e969300 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.254+0000 7fc7997dc740 1 --2- 172.21.15.2:0/3534573005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 0x56553e962970 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.254+0000 7fc7997dc740 1 --2- 172.21.15.2:0/3534573005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951ba0 0x56553e95b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.254+0000 7fc7997dc740 1 --2- 172.21.15.2:0/3534573005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951260 0x56553e95b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.254+0000 7fc7997dc740 1 -- 172.21.15.2:0/3534573005 >> 172.21.15.2:0/3534573005 conn(0x56553e94c8d0 msgr2=0x56553e94e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.254+0000 7fc779600640 0 client.27955 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.255+0000 7fc7997dc740 1 -- 172.21.15.2:0/3534573005 shutdown_connections 2024-09-10T08:17:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.255+0000 7fc7997dc740 1 -- 172.21.15.2:0/3534573005 wait complete. 2024-09-10T08:17:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.255+0000 7fc7997dc740 20 client.27955 trim_cache size 0 max 16384 2024-09-10T08:17:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc7997dc740 1 Processor -- start 2024-09-10T08:17:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc7997dc740 1 -- start start 2024-09-10T08:17:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc7997dc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 0x56553e951640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc7997dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951ba0 0x56553e962930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc7997dc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 0x56553e94ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc7997dc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901b3500 con 0x56553e951260 2024-09-10T08:17:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc7997dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901b0dd0 con 0x56553e951ba0 2024-09-10T08:17:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc7997dc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901aec20 con 0x56553e956c30 2024-09-10T08:17:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc78f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 0x56553e951640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc78f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 0x56553e951640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49976/0 (socket says 172.21.15.2:49976) 2024-09-10T08:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc78f400640 1 -- 172.21.15.2:0/2668185086 learned_addr learned my addr 172.21.15.2:0/2668185086 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc78fe00640 1 --2- 172.21.15.2:0/2668185086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 0x56553e94ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc78ea00640 1 --2- 172.21.15.2:0/2668185086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951ba0 0x56553e962930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc78f400640 1 -- 172.21.15.2:0/2668185086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 msgr2=0x56553e94ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.270+0000 7fc78f400640 1 --2- 172.21.15.2:0/2668185086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 0x56553e94ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.271+0000 7fc78f400640 1 -- 172.21.15.2:0/2668185086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951ba0 msgr2=0x56553e962930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.271+0000 7fc78f400640 1 --2- 172.21.15.2:0/2668185086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951ba0 0x56553e962930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.271+0000 7fc78f400640 1 -- 172.21.15.2:0/2668185086 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56553e969670 con 0x56553e951260 2024-09-10T08:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.271+0000 7fc78f400640 1 --2- 172.21.15.2:0/2668185086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 0x56553e951640 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc780009c40 tx=0x7fc78001bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1dddd3fd27026b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.271+0000 7fc77be00640 1 -- 172.21.15.2:0/2668185086 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780033070 con 0x56553e951260 2024-09-10T08:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.271+0000 7fc77be00640 1 -- 172.21.15.2:0/2668185086 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78001f040 con 0x56553e951260 2024-09-10T08:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.271+0000 7fc77be00640 1 -- 172.21.15.2:0/2668185086 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78001e040 con 0x56553e951260 2024-09-10T08:17:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.272+0000 7fc7997dc740 1 -- 172.21.15.2:0/2668185086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 msgr2=0x56553e951640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.272+0000 7fc7997dc740 1 --2- 172.21.15.2:0/2668185086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 0x56553e951640 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc780009c40 tx=0x7fc78001bf90 comp rx=0 tx=0).stop 2024-09-10T08:17:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.272+0000 7fc7997dc740 1 -- 172.21.15.2:0/2668185086 shutdown_connections 2024-09-10T08:17:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.272+0000 7fc7997dc740 1 --2- 172.21.15.2:0/2668185086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e956c30 0x56553e94ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.272+0000 7fc7997dc740 1 --2- 172.21.15.2:0/2668185086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e951ba0 0x56553e962930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.272+0000 7fc7997dc740 1 --2- 172.21.15.2:0/2668185086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 0x56553e951640 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.272+0000 7fc7997dc740 1 -- 172.21.15.2:0/2668185086 >> 172.21.15.2:0/2668185086 conn(0x56553e94c8d0 msgr2=0x56553e9476a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.272+0000 7fc7997dc740 1 -- 172.21.15.2:0/2668185086 shutdown_connections 2024-09-10T08:17:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.272+0000 7fc7997dc740 1 -- 172.21.15.2:0/2668185086 wait complete. 2024-09-10T08:17:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.272+0000 7fc7997dc740 1 -- start start 2024-09-10T08:17:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.272+0000 7fc7997dc740 1 Processor -- start 2024-09-10T08:17:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc7997dc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 0x56553e95f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc7997dc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e951ba0 0x56553e95fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc7997dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e956c30 0x56553e9cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc7997dc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901af290 con 0x56553e951260 2024-09-10T08:17:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc7997dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901b0700 con 0x56553e956c30 2024-09-10T08:17:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc7997dc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901af960 con 0x56553e951ba0 2024-09-10T08:17:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc7997dc740 10 client.0 _ll_register_callbacks cb 0x7fc78809bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc78f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 0x56553e95f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc78f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 0x56553e95f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49988/0 (socket says 172.21.15.2:49988) 2024-09-10T08:17:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc78f400640 1 -- 172.21.15.2:0/679026766 learned_addr learned my addr 172.21.15.2:0/679026766 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc78ea00640 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e951ba0 0x56553e95fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc78fe00640 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e956c30 0x56553e9cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc78f400640 1 -- 172.21.15.2:0/679026766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e951ba0 msgr2=0x56553e95fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc78f400640 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e951ba0 0x56553e95fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc78f400640 1 -- 172.21.15.2:0/679026766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e956c30 msgr2=0x56553e9cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc78f400640 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e956c30 0x56553e9cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc78f400640 1 -- 172.21.15.2:0/679026766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56553e9d0100 con 0x56553e951260 2024-09-10T08:17:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.273+0000 7fc78fe00640 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e956c30 0x56553e9cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.274+0000 7fc78f400640 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 0x56553e95f890 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc780009380 tx=0x7fc780005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c11a259c340a69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.274+0000 7fc779600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:17:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.274+0000 7fc779600640 1 -- 172.21.15.2:0/679026766 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78003c060 con 0x56553e951260 2024-09-10T08:17:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.274+0000 7fc779600640 1 -- 172.21.15.2:0/679026766 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc780005040 con 0x56553e951260 2024-09-10T08:17:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.274+0000 7fc7997dc740 1 -- 172.21.15.2:0/679026766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc78001f040 con 0x56553e951260 2024-09-10T08:17:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.274+0000 7fc779600640 1 -- 172.21.15.2:0/679026766 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc780043050 con 0x56553e951260 2024-09-10T08:17:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.274+0000 7fc7997dc740 10 client.18000 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.274+0000 7fc7997dc740 1 -- 172.21.15.2:0/679026766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56553e9d1410 con 0x56553e951260 2024-09-10T08:17:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.274+0000 7fc779600640 1 -- 172.21.15.2:0/679026766 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78001f040 con 0x56553e951260 2024-09-10T08:17:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.275+0000 7fc779600640 1 -- 172.21.15.2:0/679026766 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc78000e590 con 0x56553e951260 2024-09-10T08:17:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.275+0000 7fc7997dc740 10 client.18000 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.275+0000 7fc7997dc740 10 client.18000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.275+0000 7fc7997dc740 20 client.18000 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.275+0000 7fc7997dc740 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56553e9d1be0 0x56553e9d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.275+0000 7fc7997dc740 4 client.18000 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:17:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.275+0000 7fc7997dc740 1 -- 172.21.15.2:0/679026766 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56553e8bdb80 con 0x56553e9d1be0 2024-09-10T08:17:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.275+0000 7fc78ea00640 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56553e9d1be0 0x56553e9d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.276+0000 7fc78ea00640 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56553e9d1be0 0x56553e9d4000 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126cf4179466e9d8 server_cookie=decde069c15c91ac in_seq=0 out_seq=0 2024-09-10T08:17:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.276+0000 7fc779600640 10 client.18000 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.288+0000 7fc779600640 1 -- 172.21.15.2:0/679026766 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56553e8bdb80 con 0x56553e9d1be0 2024-09-10T08:17:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.288+0000 7fc779600640 10 client.18000 handle_command_reply: tid=0 2024-09-10T08:17:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.288+0000 7fc7997dc740 1 client.18000 shutdown 2024-09-10T08:17:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.289+0000 7fc7997dc740 1 -- 172.21.15.2:0/679026766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 msgr2=0x56553e95f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.289+0000 7fc7997dc740 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 0x56553e95f890 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc780009380 tx=0x7fc780005af0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.289+0000 7fc7997dc740 1 -- 172.21.15.2:0/679026766 shutdown_connections 2024-09-10T08:17:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.289+0000 7fc7997dc740 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56553e9d1be0 0x56553e9d4000 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.289+0000 7fc7997dc740 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56553e956c30 0x56553e9cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.289+0000 7fc7997dc740 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56553e951ba0 0x56553e95fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.289+0000 7fc7997dc740 1 --2- 172.21.15.2:0/679026766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56553e951260 0x56553e95f890 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.289+0000 7fc779600640 0 client.18000 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.289+0000 7fc7997dc740 1 -- 172.21.15.2:0/679026766 >> 172.21.15.2:0/679026766 conn(0x56553e94c8d0 msgr2=0x56553e94b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.289+0000 7fc7997dc740 1 -- 172.21.15.2:0/679026766 shutdown_connections 2024-09-10T08:17:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.289+0000 7fc7997dc740 1 -- 172.21.15.2:0/679026766 wait complete. 2024-09-10T08:17:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.289+0000 7fc7997dc740 20 client.18000 trim_cache size 0 max 16384 2024-09-10T08:17:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:17:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:17:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:17:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:17:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.290+0000 7fc795200640 1 -- 172.21.15.2:0/3666761942 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7740a2c00 msgr2=0x7fc7740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.290+0000 7fc795200640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7740a2c00 0x7fc7740a50c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc77c006010 tx=0x7fc77c000bc0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.290+0000 7fc795200640 1 -- 172.21.15.2:0/3666761942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc790078790 msgr2=0x7fc7901a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.290+0000 7fc795200640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc790078790 0x7fc7901a5830 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc78800eb10 tx=0x7fc78800c2d0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.290+0000 7fc795200640 1 -- 172.21.15.2:0/3666761942 shutdown_connections 2024-09-10T08:17:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.290+0000 7fc795200640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc790078790 0x7fc7901a5830 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.290+0000 7fc795200640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc790077df0 0x7fc79019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.290+0000 7fc795200640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc7740a2c00 0x7fc7740a50c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.290+0000 7fc795200640 1 --2- 172.21.15.2:0/3666761942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc790076bb0 0x7fc79019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.290+0000 7fc795200640 1 -- 172.21.15.2:0/3666761942 >> 172.21.15.2:0/3666761942 conn(0x7fc7900ff310 msgr2=0x7fc790100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.290+0000 7fc795200640 1 -- 172.21.15.2:0/3666761942 shutdown_connections 2024-09-10T08:17:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:30.290+0000 7fc795200640 1 -- 172.21.15.2:0/3666761942 wait complete. 2024-09-10T08:17:30.689 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:17:30.690 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:17:31.067 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:17:31.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:31 smithi002 ceph-mon[30563]: pgmap v668: 129 pgs: 129 active+clean; 836 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-09-10T08:17:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.895+0000 7f0bf8a00640 1 -- 172.21.15.2:0/3518589242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bf4109120 msgr2=0x7f0bf4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.895+0000 7f0bf8a00640 1 --2- 172.21.15.2:0/3518589242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bf4109120 0x7f0bf4110ca0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009ca0 tx=0x7f0bec02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:17:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.895+0000 7f0bf8a00640 1 -- 172.21.15.2:0/3518589242 shutdown_connections 2024-09-10T08:17:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.895+0000 7f0bf8a00640 1 --2- 172.21.15.2:0/3518589242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bf4109120 0x7f0bf4110ca0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.895+0000 7f0bf8a00640 1 --2- 172.21.15.2:0/3518589242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bf41049a0 0x7f0bf41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.895+0000 7f0bf8a00640 1 --2- 172.21.15.2:0/3518589242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bf4103ff0 0x7f0bf41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:31.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.895+0000 7f0bf8a00640 1 -- 172.21.15.2:0/3518589242 >> 172.21.15.2:0/3518589242 conn(0x7f0bf40ffa90 msgr2=0x7f0bf4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:31.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.895+0000 7f0bf8a00640 1 -- 172.21.15.2:0/3518589242 shutdown_connections 2024-09-10T08:17:31.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.895+0000 7f0bf8a00640 1 -- 172.21.15.2:0/3518589242 wait complete. 2024-09-10T08:17:31.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.896+0000 7f0bf8a00640 1 Processor -- start 2024-09-10T08:17:31.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.896+0000 7f0bf8a00640 1 -- start start 2024-09-10T08:17:31.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.896+0000 7f0bf8a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bf4103ff0 0x7f0bf419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:31.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.896+0000 7f0bf8a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bf41049a0 0x7f0bf419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:31.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.896+0000 7f0bf8a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bf4109120 0x7f0bf419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:31.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.896+0000 7f0bf8a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41141a0 con 0x7f0bf4103ff0 2024-09-10T08:17:31.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.896+0000 7f0bf8a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4114020 con 0x7f0bf41049a0 2024-09-10T08:17:31.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.896+0000 7f0bf8a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4114320 con 0x7f0bf4109120 2024-09-10T08:17:31.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.897+0000 7f0bf1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bf4103ff0 0x7f0bf419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:31.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.897+0000 7f0be9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bf41049a0 0x7f0bf419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:31.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.897+0000 7f0bf1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bf4103ff0 0x7f0bf419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50010/0 (socket says 172.21.15.2:50010) 2024-09-10T08:17:31.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.897+0000 7f0bf1e00640 1 -- 172.21.15.2:0/1779287195 learned_addr learned my addr 172.21.15.2:0/1779287195 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:31.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.897+0000 7f0bf2800640 1 --2- 172.21.15.2:0/1779287195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bf4109120 0x7f0bf419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:31.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.897+0000 7f0bf1e00640 1 -- 172.21.15.2:0/1779287195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bf4109120 msgr2=0x7f0bf419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:31.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.897+0000 7f0bf1e00640 1 --2- 172.21.15.2:0/1779287195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bf4109120 0x7f0bf419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:31.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.897+0000 7f0bf1e00640 1 -- 172.21.15.2:0/1779287195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bf41049a0 msgr2=0x7f0bf419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:31.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.897+0000 7f0bf1e00640 1 --2- 172.21.15.2:0/1779287195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bf41049a0 0x7f0bf419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:31.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.897+0000 7f0bf1e00640 1 -- 172.21.15.2:0/1779287195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf419f570 con 0x7f0bf4103ff0 2024-09-10T08:17:31.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.897+0000 7f0bf1e00640 1 --2- 172.21.15.2:0/1779287195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bf4103ff0 0x7f0bf419a5c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0be400e9f0 tx=0x7f0be400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:31.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.898+0000 7f0beb400640 1 -- 172.21.15.2:0/1779287195 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be400cd80 con 0x7f0bf4103ff0 2024-09-10T08:17:31.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.898+0000 7f0beb400640 1 -- 172.21.15.2:0/1779287195 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be4005500 con 0x7f0bf4103ff0 2024-09-10T08:17:31.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.898+0000 7f0beb400640 1 -- 172.21.15.2:0/1779287195 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4010690 con 0x7f0bf4103ff0 2024-09-10T08:17:31.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.898+0000 7f0bf8a00640 1 -- 172.21.15.2:0/1779287195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf419f860 con 0x7f0bf4103ff0 2024-09-10T08:17:31.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.898+0000 7f0bf8a00640 1 -- 172.21.15.2:0/1779287195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf41a7140 con 0x7f0bf4103ff0 2024-09-10T08:17:31.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.899+0000 7f0bf8a00640 1 -- 172.21.15.2:0/1779287195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf41043d0 con 0x7f0bf4103ff0 2024-09-10T08:17:31.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.901+0000 7f0beb400640 1 -- 172.21.15.2:0/1779287195 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0be40046c0 con 0x7f0bf4103ff0 2024-09-10T08:17:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.902+0000 7f0beb400640 1 --2- 172.21.15.2:0/1779287195 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0bd80a2c00 0x7f0bd80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.902+0000 7f0beb400640 1 -- 172.21.15.2:0/1779287195 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0be40cf8c0 con 0x7f0bf4103ff0 2024-09-10T08:17:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.903+0000 7f0be9400640 1 --2- 172.21.15.2:0/1779287195 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0bd80a2c00 0x7f0bd80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:31.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.903+0000 7f0be9400640 1 --2- 172.21.15.2:0/1779287195 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0bd80a2c00 0x7f0bd80a50c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0095f0 tx=0x7f0bdc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:31.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:31 smithi155 ceph-mon[35405]: pgmap v668: 129 pgs: 129 active+clean; 836 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-09-10T08:17:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:31.907+0000 7f0beb400640 1 -- 172.21.15.2:0/1779287195 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0be409c3d0 con 0x7f0bf4103ff0 2024-09-10T08:17:31.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:31 smithi107 ceph-mon[35048]: pgmap v668: 129 pgs: 129 active+clean; 836 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-09-10T08:17:32.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.128+0000 7f0bf8a00640 1 -- 172.21.15.2:0/1779287195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bf419b920 con 0x7f0bf4103ff0 2024-09-10T08:17:32.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.129+0000 7f0beb400640 1 -- 172.21.15.2:0/1779287195 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f0be409bcd0 con 0x7f0bf4103ff0 2024-09-10T08:17:32.130 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:17:32.130 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:17:32.130 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:17:32.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.132+0000 7f0bf8a00640 1 -- 172.21.15.2:0/1779287195 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0bd80a2c00 msgr2=0x7f0bd80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:32.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.132+0000 7f0bf8a00640 1 --2- 172.21.15.2:0/1779287195 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0bd80a2c00 0x7f0bd80a50c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0095f0 tx=0x7f0bdc005f90 comp rx=0 tx=0).stop 2024-09-10T08:17:32.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.132+0000 7f0bf8a00640 1 -- 172.21.15.2:0/1779287195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bf4103ff0 msgr2=0x7f0bf419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:32.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.132+0000 7f0bf8a00640 1 --2- 172.21.15.2:0/1779287195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bf4103ff0 0x7f0bf419a5c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0be400e9f0 tx=0x7f0be400edf0 comp rx=0 tx=0).stop 2024-09-10T08:17:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.132+0000 7f0bf8a00640 1 -- 172.21.15.2:0/1779287195 shutdown_connections 2024-09-10T08:17:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.132+0000 7f0bf8a00640 1 --2- 172.21.15.2:0/1779287195 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0bd80a2c00 0x7f0bd80a50c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.132+0000 7f0bf8a00640 1 --2- 172.21.15.2:0/1779287195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bf4109120 0x7f0bf419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.132+0000 7f0bf8a00640 1 --2- 172.21.15.2:0/1779287195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bf41049a0 0x7f0bf419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.132+0000 7f0bf8a00640 1 --2- 172.21.15.2:0/1779287195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bf4103ff0 0x7f0bf419a5c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.132+0000 7f0bf8a00640 1 -- 172.21.15.2:0/1779287195 >> 172.21.15.2:0/1779287195 conn(0x7f0bf40ffa90 msgr2=0x7f0bf41005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.132+0000 7f0bf8a00640 1 -- 172.21.15.2:0/1779287195 shutdown_connections 2024-09-10T08:17:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:32.132+0000 7f0bf8a00640 1 -- 172.21.15.2:0/1779287195 wait complete. 2024-09-10T08:17:32.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:32 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1779287195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:17:32.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:32 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1779287195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:17:33.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:32 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1779287195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:17:33.142 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:17:33.523 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:17:33.898 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:33 smithi002 ceph-mon[30563]: pgmap v669: 129 pgs: 129 active+clean; 836 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-09-10T08:17:33.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:33 smithi155 ceph-mon[35405]: pgmap v669: 129 pgs: 129 active+clean; 836 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-09-10T08:17:33.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:33 smithi107 ceph-mon[35048]: pgmap v669: 129 pgs: 129 active+clean; 836 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-09-10T08:17:34.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.114+0000 7f11d1600640 1 -- 172.21.15.2:0/2514746067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11cc106f10 msgr2=0x7f11cc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:34.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.114+0000 7f11d1600640 1 --2- 172.21.15.2:0/2514746067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11cc106f10 0x7f11cc10ea90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009ca0 tx=0x7f11c402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.114+0000 7f11d1600640 1 -- 172.21.15.2:0/2514746067 shutdown_connections 2024-09-10T08:17:34.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.114+0000 7f11d1600640 1 --2- 172.21.15.2:0/2514746067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11cc106f10 0x7f11cc10ea90 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.114+0000 7f11d1600640 1 --2- 172.21.15.2:0/2514746067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11cc102790 0x7f11cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.115+0000 7f11d1600640 1 --2- 172.21.15.2:0/2514746067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11cc101de0 0x7f11cc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.115+0000 7f11d1600640 1 -- 172.21.15.2:0/2514746067 >> 172.21.15.2:0/2514746067 conn(0x7f11cc0fd8c0 msgr2=0x7f11cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.115+0000 7f11d1600640 1 -- 172.21.15.2:0/2514746067 shutdown_connections 2024-09-10T08:17:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.115+0000 7f11d1600640 1 -- 172.21.15.2:0/2514746067 wait complete. 2024-09-10T08:17:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.116+0000 7f11d1600640 1 Processor -- start 2024-09-10T08:17:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.116+0000 7f11d1600640 1 -- start start 2024-09-10T08:17:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.116+0000 7f11d1600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11cc101de0 0x7f11cc19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.116+0000 7f11d1600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11cc102790 0x7f11cc19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.116+0000 7f11d1600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11cc106f10 0x7f11cc19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.116+0000 7f11d1600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc111f70 con 0x7f11cc101de0 2024-09-10T08:17:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.116+0000 7f11d1600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc111df0 con 0x7f11cc106f10 2024-09-10T08:17:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.116+0000 7f11d1600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1120f0 con 0x7f11cc102790 2024-09-10T08:17:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.116+0000 7f11caa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11cc101de0 0x7f11cc19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.116+0000 7f11caa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11cc101de0 0x7f11cc19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50026/0 (socket says 172.21.15.2:50026) 2024-09-10T08:17:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.116+0000 7f11caa00640 1 -- 172.21.15.2:0/103655166 learned_addr learned my addr 172.21.15.2:0/103655166 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.116+0000 7f11cb400640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11cc106f10 0x7f11cc19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.116+0000 7f11c2000640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11cc102790 0x7f11cc19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.117+0000 7f11caa00640 1 -- 172.21.15.2:0/103655166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11cc102790 msgr2=0x7f11cc19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.117+0000 7f11caa00640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11cc102790 0x7f11cc19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.117+0000 7f11caa00640 1 -- 172.21.15.2:0/103655166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11cc106f10 msgr2=0x7f11cc19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.117+0000 7f11caa00640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11cc106f10 0x7f11cc19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.117+0000 7f11caa00640 1 -- 172.21.15.2:0/103655166 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc19f580 con 0x7f11cc101de0 2024-09-10T08:17:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.117+0000 7f11cb400640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11cc106f10 0x7f11cc19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.117+0000 7f11caa00640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11cc101de0 0x7f11cc19a5d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00e9f0 tx=0x7f11bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.117+0000 7f11c3e00640 1 -- 172.21.15.2:0/103655166 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc00cd80 con 0x7f11cc101de0 2024-09-10T08:17:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.117+0000 7f11c3e00640 1 -- 172.21.15.2:0/103655166 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11bc010910 con 0x7f11cc101de0 2024-09-10T08:17:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.117+0000 7f11d1600640 1 -- 172.21.15.2:0/103655166 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc19f870 con 0x7f11cc101de0 2024-09-10T08:17:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.117+0000 7f11c3e00640 1 -- 172.21.15.2:0/103655166 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc018990 con 0x7f11cc101de0 2024-09-10T08:17:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.118+0000 7f11d1600640 1 -- 172.21.15.2:0/103655166 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc1a7150 con 0x7f11cc101de0 2024-09-10T08:17:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.121+0000 7f11c3e00640 1 -- 172.21.15.2:0/103655166 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f11bc010ac0 con 0x7f11cc101de0 2024-09-10T08:17:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.121+0000 7f11d3fe5740 1 Processor -- start 2024-09-10T08:17:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.121+0000 7f11d3fe5740 1 -- start start 2024-09-10T08:17:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.121+0000 7f11d3fe5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.121+0000 7f11d3fe5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.121+0000 7f11d3fe5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 0x5586514c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.121+0000 7f11d3fe5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55865141d3e0 con 0x5586514b7590 2024-09-10T08:17:34.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.121+0000 7f11d3fe5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5586513c4d80 con 0x5586514c1e50 2024-09-10T08:17:34.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.121+0000 7f11d3fe5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5586514193a0 con 0x5586514b6c70 2024-09-10T08:17:34.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.122+0000 7f11cb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.122+0000 7f11caa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.122+0000 7f11cb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514b7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56734/0 (socket says 172.21.15.2:56734) 2024-09-10T08:17:34.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.122+0000 7f11cb400640 1 -- 172.21.15.2:0/1056111957 learned_addr learned my addr 172.21.15.2:0/1056111957 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:34.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.122+0000 7f11c2000640 1 --2- 172.21.15.2:0/1056111957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 0x5586514c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.122+0000 7f11cb400640 1 -- 172.21.15.2:0/1056111957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 msgr2=0x5586514c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:34.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.122+0000 7f11cb400640 1 --2- 172.21.15.2:0/1056111957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 0x5586514c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.122+0000 7f11cb400640 1 -- 172.21.15.2:0/1056111957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 msgr2=0x5586514c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.122+0000 7f11cb400640 1 --2- 172.21.15.2:0/1056111957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514c1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.122+0000 7f11cb400640 1 -- 172.21.15.2:0/1056111957 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586513e3ae0 con 0x5586514b6c70 2024-09-10T08:17:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.122+0000 7f11c2000640 1 --2- 172.21.15.2:0/1056111957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 0x5586514c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.122+0000 7f11c3e00640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f11b00a2b80 0x7f11b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.122+0000 7f11cb400640 1 --2- 172.21.15.2:0/1056111957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514b7050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f11c4031f80 tx=0x7f11c4005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d166fe3dbbe29f35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.123+0000 7f11c3e00640 1 -- 172.21.15.2:0/103655166 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f11bc0cf7a0 con 0x7f11cc101de0 2024-09-10T08:17:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.123+0000 7f11caa00640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f11b00a2b80 0x7f11b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.123+0000 7f11bb400640 1 -- 172.21.15.2:0/1056111957 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4003b70 con 0x5586514b6c70 2024-09-10T08:17:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.123+0000 7f11bb400640 1 -- 172.21.15.2:0/1056111957 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c4036040 con 0x5586514b6c70 2024-09-10T08:17:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.123+0000 7f11bb400640 1 -- 172.21.15.2:0/1056111957 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c40045f0 con 0x5586514b6c70 2024-09-10T08:17:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.123+0000 7f11d3fe5740 1 -- 172.21.15.2:0/1056111957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 msgr2=0x5586514b7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.123+0000 7f11caa00640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f11b00a2b80 0x7f11b00a5040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0980c0 tx=0x7f11bc0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.123+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/1056111957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514b7050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f11c4031f80 tx=0x7f11c4005ef0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.124+0000 7f11d3fe5740 1 -- 172.21.15.2:0/1056111957 shutdown_connections 2024-09-10T08:17:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.124+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/1056111957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 0x5586514c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.124+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/1056111957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.124+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/1056111957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514b7050 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.124+0000 7f11d3fe5740 1 -- 172.21.15.2:0/1056111957 >> 172.21.15.2:0/1056111957 conn(0x5586514b3200 msgr2=0x5586514b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.124+0000 7f11d3fe5740 1 -- 172.21.15.2:0/1056111957 shutdown_connections 2024-09-10T08:17:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.124+0000 7f11d3fe5740 1 -- 172.21.15.2:0/1056111957 wait complete. 2024-09-10T08:17:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.124+0000 7f11d3fe5740 1 -- start start 2024-09-10T08:17:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.124+0000 7f11d3fe5740 1 Processor -- start 2024-09-10T08:17:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.125+0000 7f11d3fe5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514bed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.125+0000 7f11d3fe5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514bf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.125+0000 7f11d3fe5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 0x5586514bf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.125+0000 7f11d3fe5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5586514c48a0 con 0x5586514b7590 2024-09-10T08:17:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.125+0000 7f11d3fe5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5586513d80f0 con 0x5586514c1e50 2024-09-10T08:17:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.125+0000 7f11d3fe5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5586514c4a20 con 0x5586514b6c70 2024-09-10T08:17:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.125+0000 7f11d3fe5740 10 client.0 _ll_register_callbacks cb 0x5586513bfbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.125+0000 7f11c2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514bf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.125+0000 7f11c2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514bf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50044/0 (socket says 172.21.15.2:50044) 2024-09-10T08:17:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.125+0000 7f11c2000640 1 -- 172.21.15.2:0/4218812310 learned_addr learned my addr 172.21.15.2:0/4218812310 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.125+0000 7f11cb400640 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514bed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.125+0000 7f11cb400640 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 0x5586514bf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.126+0000 7f11cb400640 1 -- 172.21.15.2:0/4218812310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 msgr2=0x5586514bf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.126+0000 7f11cb400640 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 0x5586514bf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.126+0000 7f11cb400640 1 -- 172.21.15.2:0/4218812310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 msgr2=0x5586514bf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.126+0000 7f11cb400640 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514bf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.126+0000 7f11cb400640 1 -- 172.21.15.2:0/4218812310 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55865118e3c0 con 0x5586514b6c70 2024-09-10T08:17:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.126+0000 7f11c2000640 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514bf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.126+0000 7f11cb400640 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514bed40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f11c4004af0 tx=0x7f11c4004290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a07ae2324547aa41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.126+0000 7f11afe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:17:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.126+0000 7f11afe00640 1 -- 172.21.15.2:0/4218812310 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4058060 con 0x5586514b6c70 2024-09-10T08:17:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.126+0000 7f11d3fe5740 1 -- 172.21.15.2:0/4218812310 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11c4036040 con 0x5586514b6c70 2024-09-10T08:17:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.127+0000 7f11afe00640 1 -- 172.21.15.2:0/4218812310 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f11c4003c50 con 0x5586514b6c70 2024-09-10T08:17:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.127+0000 7f11afe00640 1 -- 172.21.15.2:0/4218812310 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11c4003e50 con 0x5586514b6c70 2024-09-10T08:17:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.127+0000 7f11d3fe5740 10 client.27979 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.127+0000 7f11d3fe5740 1 -- 172.21.15.2:0/4218812310 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5586514c48a0 con 0x5586514b6c70 2024-09-10T08:17:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.127+0000 7f11afe00640 1 -- 172.21.15.2:0/4218812310 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c4036040 con 0x5586514b6c70 2024-09-10T08:17:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.127+0000 7f11afe00640 1 -- 172.21.15.2:0/4218812310 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f11c404d030 con 0x5586514b6c70 2024-09-10T08:17:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.127+0000 7f11d3fe5740 10 client.27979 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.127+0000 7f11d3fe5740 10 client.27979 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:17:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.127+0000 7f11d3fe5740 20 client.27979 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.127+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5586514d1a60 0x5586514d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.127+0000 7f11d3fe5740 4 client.27979 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:17:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.127+0000 7f11d3fe5740 1 -- 172.21.15.2:0/4218812310 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55865142a950 con 0x5586514d1a60 2024-09-10T08:17:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.128+0000 7f11c2000640 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5586514d1a60 0x5586514d3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.128+0000 7f11c2000640 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5586514d1a60 0x5586514d3e50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20644f6b31a35a74 server_cookie=d2ee196c4fd947b7 in_seq=0 out_seq=0 2024-09-10T08:17:34.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.128+0000 7f11afe00640 10 client.27979 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:17:34.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.129+0000 7f11afe00640 1 -- 172.21.15.2:0/4218812310 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55865142a950 con 0x5586514d1a60 2024-09-10T08:17:34.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.129+0000 7f11afe00640 10 client.27979 handle_command_reply: tid=0 2024-09-10T08:17:34.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.129+0000 7f11d3fe5740 1 client.27979 shutdown 2024-09-10T08:17:34.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.129+0000 7f11d3fe5740 1 -- 172.21.15.2:0/4218812310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 msgr2=0x5586514bed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:34.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.129+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514bed40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f11c4004af0 tx=0x7f11c4004290 comp rx=0 tx=0).stop 2024-09-10T08:17:34.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.130+0000 7f11d3fe5740 1 -- 172.21.15.2:0/4218812310 shutdown_connections 2024-09-10T08:17:34.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.130+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 0x5586514bf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.130+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5586514d1a60 0x5586514d3e50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.130+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514bf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.130+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/4218812310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514bed40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.130+0000 7f11afe00640 0 client.27979 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:17:34.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.130+0000 7f11d3fe5740 1 -- 172.21.15.2:0/4218812310 >> 172.21.15.2:0/4218812310 conn(0x5586514b3200 msgr2=0x5586514b8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:34.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.130+0000 7f11d3fe5740 1 -- 172.21.15.2:0/4218812310 shutdown_connections 2024-09-10T08:17:34.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.130+0000 7f11d3fe5740 1 -- 172.21.15.2:0/4218812310 wait complete. 2024-09-10T08:17:34.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.130+0000 7f11d3fe5740 20 client.27979 trim_cache size 0 max 16384 2024-09-10T08:17:34.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11d3fe5740 1 Processor -- start 2024-09-10T08:17:34.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11d3fe5740 1 -- start start 2024-09-10T08:17:34.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11d3fe5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11d3fe5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514d9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11d3fe5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 0x5586514d9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11d3fe5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1b3860 con 0x5586514b7590 2024-09-10T08:17:34.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11d3fe5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1b1130 con 0x5586514c1e50 2024-09-10T08:17:34.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11d3fe5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1aef80 con 0x5586514b6c70 2024-09-10T08:17:34.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11c2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514d9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11cb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11cb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56744/0 (socket says 172.21.15.2:56744) 2024-09-10T08:17:34.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11cb400640 1 -- 172.21.15.2:0/3673611237 learned_addr learned my addr 172.21.15.2:0/3673611237 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:34.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11c2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514d9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50046/0 (socket says 172.21.15.2:50046) 2024-09-10T08:17:34.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.144+0000 7f11cb400640 1 --2- 172.21.15.2:0/3673611237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 0x5586514d9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.145+0000 7f11c2000640 1 -- 172.21.15.2:0/3673611237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 msgr2=0x5586514b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:34.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.145+0000 7f11c2000640 1 --2- 172.21.15.2:0/3673611237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.145+0000 7f11c2000640 1 -- 172.21.15.2:0/3673611237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 msgr2=0x5586514d9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.145+0000 7f11c2000640 1 --2- 172.21.15.2:0/3673611237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 0x5586514d9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.145+0000 7f11c2000640 1 -- 172.21.15.2:0/3673611237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586514d41c0 con 0x5586514b7590 2024-09-10T08:17:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.145+0000 7f11cb400640 1 --2- 172.21.15.2:0/3673611237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.145+0000 7f11c2000640 1 --2- 172.21.15.2:0/3673611237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514d9720 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x5586514d4640 tx=0x7f11b4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9fb1cb6827469c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.145+0000 7f11baa00640 1 -- 172.21.15.2:0/3673611237 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4004250 con 0x5586514b7590 2024-09-10T08:17:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.145+0000 7f11baa00640 1 -- 172.21.15.2:0/3673611237 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b40043f0 con 0x5586514b7590 2024-09-10T08:17:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.145+0000 7f11baa00640 1 -- 172.21.15.2:0/3673611237 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b40109f0 con 0x5586514b7590 2024-09-10T08:17:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.146+0000 7f11d3fe5740 1 -- 172.21.15.2:0/3673611237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 msgr2=0x5586514d9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.146+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/3673611237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514d9720 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x5586514d4640 tx=0x7f11b4005f90 comp rx=0 tx=0).stop 2024-09-10T08:17:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.146+0000 7f11d3fe5740 1 -- 172.21.15.2:0/3673611237 shutdown_connections 2024-09-10T08:17:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.146+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/3673611237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514c1e50 0x5586514d9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.146+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/3673611237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514b7590 0x5586514d9720 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.146+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/3673611237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b6c70 0x5586514b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.146+0000 7f11d3fe5740 1 -- 172.21.15.2:0/3673611237 >> 172.21.15.2:0/3673611237 conn(0x5586514b3200 msgr2=0x5586514c0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:34.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.146+0000 7f11d3fe5740 1 -- 172.21.15.2:0/3673611237 shutdown_connections 2024-09-10T08:17:34.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.146+0000 7f11d3fe5740 1 -- 172.21.15.2:0/3673611237 wait complete. 2024-09-10T08:17:34.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.147+0000 7f11d3fe5740 1 -- start start 2024-09-10T08:17:34.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.147+0000 7f11d3fe5740 1 Processor -- start 2024-09-10T08:17:34.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.147+0000 7f11d3fe5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514b6c70 0x5586514ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.147+0000 7f11d3fe5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b7590 0x5586514bbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.147+0000 7f11d3fe5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514c1e50 0x5586514bc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.147+0000 7f11d3fe5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1af5f0 con 0x5586514c1e50 2024-09-10T08:17:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.147+0000 7f11d3fe5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1b0a60 con 0x5586514b6c70 2024-09-10T08:17:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.147+0000 7f11d3fe5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1afcc0 con 0x5586514b7590 2024-09-10T08:17:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.147+0000 7f11d3fe5740 10 client.0 _ll_register_callbacks cb 0x7f11bc0d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.148+0000 7f11c2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b7590 0x5586514bbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.148+0000 7f11c2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b7590 0x5586514bbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56756/0 (socket says 172.21.15.2:56756) 2024-09-10T08:17:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.148+0000 7f11c2000640 1 -- 172.21.15.2:0/88091981 learned_addr learned my addr 172.21.15.2:0/88091981 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.148+0000 7f11cb400640 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514c1e50 0x5586514bc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.148+0000 7f11cb400640 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514b6c70 0x5586514ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.148+0000 7f11c2000640 1 -- 172.21.15.2:0/88091981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514b6c70 msgr2=0x5586514ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.148+0000 7f11c2000640 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514b6c70 0x5586514ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.148+0000 7f11c2000640 1 -- 172.21.15.2:0/88091981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514c1e50 msgr2=0x5586514bc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.148+0000 7f11c2000640 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514c1e50 0x5586514bc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.148+0000 7f11c2000640 1 -- 172.21.15.2:0/88091981 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586514bc890 con 0x5586514b7590 2024-09-10T08:17:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.148+0000 7f11cb400640 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514c1e50 0x5586514bc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.148+0000 7f11c2000640 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b7590 0x5586514bbb20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f11b40106d0 tx=0x7f11b40056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=650ed83e4ef3ebd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.148+0000 7f11afe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:17:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.149+0000 7f11afe00640 1 -- 172.21.15.2:0/88091981 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4010770 con 0x5586514b7590 2024-09-10T08:17:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.149+0000 7f11d3fe5740 1 -- 172.21.15.2:0/88091981 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b40043f0 con 0x5586514b7590 2024-09-10T08:17:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.149+0000 7f11afe00640 1 -- 172.21.15.2:0/88091981 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f11b4010910 con 0x5586514b7590 2024-09-10T08:17:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.149+0000 7f11afe00640 1 -- 172.21.15.2:0/88091981 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11b4038050 con 0x5586514b7590 2024-09-10T08:17:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.149+0000 7f11d3fe5740 10 client.27985 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.149+0000 7f11d3fe5740 1 -- 172.21.15.2:0/88091981 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55865153c750 con 0x5586514b7590 2024-09-10T08:17:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.149+0000 7f11afe00640 1 -- 172.21.15.2:0/88091981 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b40043f0 con 0x5586514b7590 2024-09-10T08:17:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.150+0000 7f11afe00640 1 -- 172.21.15.2:0/88091981 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f11b4003bd0 con 0x5586514b7590 2024-09-10T08:17:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.150+0000 7f11d3fe5740 10 client.27985 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.150+0000 7f11d3fe5740 10 client.27985 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:17:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.150+0000 7f11d3fe5740 20 client.27985 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.150+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55865153cdc0 0x55865153f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.150+0000 7f11d3fe5740 4 client.27985 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:17:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.150+0000 7f11d3fe5740 1 -- 172.21.15.2:0/88091981 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558651424e60 con 0x55865153cdc0 2024-09-10T08:17:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.150+0000 7f11cb400640 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55865153cdc0 0x55865153f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.150+0000 7f11cb400640 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55865153cdc0 0x55865153f1e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf44204bc653bb server_cookie=4632f9326041ed0e in_seq=0 out_seq=0 2024-09-10T08:17:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.150+0000 7f11afe00640 10 client.27985 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:17:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11afe00640 1 -- 172.21.15.2:0/88091981 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558651424e60 con 0x55865153cdc0 2024-09-10T08:17:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11afe00640 10 client.27985 handle_command_reply: tid=0 2024-09-10T08:17:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11d3fe5740 1 client.27985 shutdown 2024-09-10T08:17:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11d3fe5740 1 -- 172.21.15.2:0/88091981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b7590 msgr2=0x5586514bbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:34.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b7590 0x5586514bbb20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f11b40106d0 tx=0x7f11b40056d0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11d3fe5740 1 -- 172.21.15.2:0/88091981 shutdown_connections 2024-09-10T08:17:34.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55865153cdc0 0x55865153f1e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586514c1e50 0x5586514bc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586514b7590 0x5586514bbb20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11d3fe5740 1 --2- 172.21.15.2:0/88091981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586514b6c70 0x5586514ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11afe00640 0 client.27985 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:17:34.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11d3fe5740 1 -- 172.21.15.2:0/88091981 >> 172.21.15.2:0/88091981 conn(0x5586514b3200 msgr2=0x5586514b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:34.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11d3fe5740 1 -- 172.21.15.2:0/88091981 shutdown_connections 2024-09-10T08:17:34.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11d3fe5740 1 -- 172.21.15.2:0/88091981 wait complete. 2024-09-10T08:17:34.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.151+0000 7f11d3fe5740 20 client.27985 trim_cache size 0 max 16384 2024-09-10T08:17:34.166 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:17:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.152+0000 7f11d1600640 1 -- 172.21.15.2:0/103655166 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f11b00a2b80 msgr2=0x7f11b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.152+0000 7f11d1600640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f11b00a2b80 0x7f11b00a5040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0980c0 tx=0x7f11bc0d5530 comp rx=0 tx=0).stop 2024-09-10T08:17:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.152+0000 7f11d1600640 1 -- 172.21.15.2:0/103655166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11cc101de0 msgr2=0x7f11cc19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.152+0000 7f11d1600640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11cc101de0 0x7f11cc19a5d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00e9f0 tx=0x7f11bc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.152+0000 7f11d1600640 1 -- 172.21.15.2:0/103655166 shutdown_connections 2024-09-10T08:17:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.152+0000 7f11d1600640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11cc106f10 0x7f11cc19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.152+0000 7f11d1600640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f11b00a2b80 0x7f11b00a5040 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.152+0000 7f11d1600640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11cc102790 0x7f11cc19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.152+0000 7f11d1600640 1 --2- 172.21.15.2:0/103655166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11cc101de0 0x7f11cc19a5d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.152+0000 7f11d1600640 1 -- 172.21.15.2:0/103655166 >> 172.21.15.2:0/103655166 conn(0x7f11cc0fd8c0 msgr2=0x7f11cc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.152+0000 7f11d1600640 1 -- 172.21.15.2:0/103655166 shutdown_connections 2024-09-10T08:17:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:34.152+0000 7f11d1600640 1 -- 172.21.15.2:0/103655166 wait complete. 2024-09-10T08:17:34.968 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:17:35.343 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:17:35.749 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:35 smithi002 ceph-mon[30563]: pgmap v670: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-10T08:17:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:35 smithi155 ceph-mon[35405]: pgmap v670: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-10T08:17:35.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:35 smithi107 ceph-mon[35048]: pgmap v670: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-10T08:17:35.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.974+0000 7fd848800640 1 -- 172.21.15.2:0/3087702226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8401049c0 msgr2=0x7fd840108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:35.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.974+0000 7fd848800640 1 --2- 172.21.15.2:0/3087702226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8401049c0 0x7fd840108a10 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd830009900 tx=0x7fd83002a410 comp rx=0 tx=0).stop 2024-09-10T08:17:35.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.974+0000 7fd848800640 1 -- 172.21.15.2:0/3087702226 shutdown_connections 2024-09-10T08:17:35.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.974+0000 7fd848800640 1 --2- 172.21.15.2:0/3087702226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd840109140 0x7fd840110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.974+0000 7fd848800640 1 --2- 172.21.15.2:0/3087702226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8401049c0 0x7fd840108a10 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.974+0000 7fd848800640 1 --2- 172.21.15.2:0/3087702226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd840104010 0x7fd8401043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.974+0000 7fd848800640 1 -- 172.21.15.2:0/3087702226 >> 172.21.15.2:0/3087702226 conn(0x7fd8400ffa50 msgr2=0x7fd840101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:35.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.974+0000 7fd848800640 1 -- 172.21.15.2:0/3087702226 shutdown_connections 2024-09-10T08:17:35.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.974+0000 7fd848800640 1 -- 172.21.15.2:0/3087702226 wait complete. 2024-09-10T08:17:35.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.975+0000 7fd848800640 1 Processor -- start 2024-09-10T08:17:35.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.975+0000 7fd848800640 1 -- start start 2024-09-10T08:17:35.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.975+0000 7fd848800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd840104010 0x7fd84019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:35.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.975+0000 7fd848800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8401049c0 0x7fd84019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:35.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd848800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd840109140 0x7fd84019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:35.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd848800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd840114140 con 0x7fd840109140 2024-09-10T08:17:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd848800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd840113fc0 con 0x7fd840104010 2024-09-10T08:17:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd848800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401142c0 con 0x7fd8401049c0 2024-09-10T08:17:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd846800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd840109140 0x7fd84019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd845e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd840104010 0x7fd84019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd845e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd840104010 0x7fd84019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:57484/0 (socket says 172.21.15.2:57484) 2024-09-10T08:17:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd846800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd840109140 0x7fd84019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50078/0 (socket says 172.21.15.2:50078) 2024-09-10T08:17:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd845e00640 1 -- 172.21.15.2:0/15875828 learned_addr learned my addr 172.21.15.2:0/15875828 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd845400640 1 --2- 172.21.15.2:0/15875828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8401049c0 0x7fd84019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd846800640 1 -- 172.21.15.2:0/15875828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8401049c0 msgr2=0x7fd84019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd846800640 1 --2- 172.21.15.2:0/15875828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8401049c0 0x7fd84019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd846800640 1 -- 172.21.15.2:0/15875828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd840104010 msgr2=0x7fd84019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:35.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd846800640 1 --2- 172.21.15.2:0/15875828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd840104010 0x7fd84019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.976+0000 7fd846800640 1 -- 172.21.15.2:0/15875828 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84019f660 con 0x7fd840109140 2024-09-10T08:17:35.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.977+0000 7fd846800640 1 --2- 172.21.15.2:0/15875828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd840109140 0x7fd84019ef80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00e940 tx=0x7fd83c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:35.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.977+0000 7fd83aa00640 1 -- 172.21.15.2:0/15875828 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c004d20 con 0x7fd840109140 2024-09-10T08:17:35.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.977+0000 7fd83aa00640 1 -- 172.21.15.2:0/15875828 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd83c005020 con 0x7fd840109140 2024-09-10T08:17:35.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.977+0000 7fd83aa00640 1 -- 172.21.15.2:0/15875828 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c0116b0 con 0x7fd840109140 2024-09-10T08:17:35.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.977+0000 7fd848800640 1 -- 172.21.15.2:0/15875828 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84019f950 con 0x7fd840109140 2024-09-10T08:17:35.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.977+0000 7fd848800640 1 -- 172.21.15.2:0/15875828 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8401a7230 con 0x7fd840109140 2024-09-10T08:17:35.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.980+0000 7fd84b42f740 1 Processor -- start 2024-09-10T08:17:35.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.981+0000 7fd83aa00640 1 -- 172.21.15.2:0/15875828 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd83c0054d0 con 0x7fd840109140 2024-09-10T08:17:35.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.981+0000 7fd84b42f740 1 -- start start 2024-09-10T08:17:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.981+0000 7fd84b42f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b04c70 0x55ca64b05050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.981+0000 7fd84b42f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 0x55ca64b0f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.981+0000 7fd84b42f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b0fe50 0x55ca64b12240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.981+0000 7fd84b42f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ca64a6b3e0 con 0x55ca64b0fe50 2024-09-10T08:17:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.981+0000 7fd84b42f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ca64a12d80 con 0x55ca64b04c70 2024-09-10T08:17:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.981+0000 7fd84b42f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ca64a673a0 con 0x55ca64b05590 2024-09-10T08:17:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.981+0000 7fd845400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b0fe50 0x55ca64b12240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.981+0000 7fd845e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 0x55ca64b0f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.982+0000 7fd845e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 0x55ca64b0f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56796/0 (socket says 172.21.15.2:56796) 2024-09-10T08:17:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.982+0000 7fd845e00640 1 -- 172.21.15.2:0/1984501034 learned_addr learned my addr 172.21.15.2:0/1984501034 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.982+0000 7fd845400640 1 --2- 172.21.15.2:0/1984501034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b04c70 0x55ca64b05050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:35.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.982+0000 7fd845e00640 1 -- 172.21.15.2:0/1984501034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b04c70 msgr2=0x55ca64b05050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:35.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.982+0000 7fd845e00640 1 --2- 172.21.15.2:0/1984501034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b04c70 0x55ca64b05050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.982+0000 7fd845e00640 1 -- 172.21.15.2:0/1984501034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b0fe50 msgr2=0x55ca64b12240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:35.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.982+0000 7fd845e00640 1 --2- 172.21.15.2:0/1984501034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b0fe50 0x55ca64b12240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.982+0000 7fd845e00640 1 -- 172.21.15.2:0/1984501034 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca64a31ae0 con 0x55ca64b05590 2024-09-10T08:17:35.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.982+0000 7fd845e00640 1 --2- 172.21.15.2:0/1984501034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 0x55ca64b0f910 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd834009dc0 tx=0x7fd834011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab3163b7c279bfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:35.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.982+0000 7fd83aa00640 1 --2- 172.21.15.2:0/15875828 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8280a2cd0 0x7fd8280a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:35.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.983+0000 7fd82f000640 1 -- 172.21.15.2:0/1984501034 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834005220 con 0x55ca64b05590 2024-09-10T08:17:35.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.983+0000 7fd82f000640 1 -- 172.21.15.2:0/1984501034 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd834004c80 con 0x55ca64b05590 2024-09-10T08:17:35.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.983+0000 7fd82f000640 1 -- 172.21.15.2:0/1984501034 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340198a0 con 0x55ca64b05590 2024-09-10T08:17:35.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.983+0000 7fd83aa00640 1 -- 172.21.15.2:0/15875828 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd83c0cf3a0 con 0x7fd840109140 2024-09-10T08:17:35.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.983+0000 7fd845400640 1 --2- 172.21.15.2:0/15875828 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8280a2cd0 0x7fd8280a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:35.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.983+0000 7fd84b42f740 1 -- 172.21.15.2:0/1984501034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 msgr2=0x55ca64b0f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:35.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.983+0000 7fd84b42f740 1 --2- 172.21.15.2:0/1984501034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 0x55ca64b0f910 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd834009dc0 tx=0x7fd834011810 comp rx=0 tx=0).stop 2024-09-10T08:17:35.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.983+0000 7fd845400640 1 --2- 172.21.15.2:0/15875828 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8280a2cd0 0x7fd8280a5190 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd830009230 tx=0x7fd8300314e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:35.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.984+0000 7fd84b42f740 1 -- 172.21.15.2:0/1984501034 shutdown_connections 2024-09-10T08:17:35.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.984+0000 7fd84b42f740 1 --2- 172.21.15.2:0/1984501034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b0fe50 0x55ca64b12240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.984+0000 7fd84b42f740 1 --2- 172.21.15.2:0/1984501034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 0x55ca64b0f910 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.984+0000 7fd84b42f740 1 --2- 172.21.15.2:0/1984501034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b04c70 0x55ca64b05050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.984+0000 7fd84b42f740 1 -- 172.21.15.2:0/1984501034 >> 172.21.15.2:0/1984501034 conn(0x55ca64b01200 msgr2=0x55ca64b015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:35.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.984+0000 7fd84b42f740 1 -- 172.21.15.2:0/1984501034 shutdown_connections 2024-09-10T08:17:35.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.984+0000 7fd84b42f740 1 -- 172.21.15.2:0/1984501034 wait complete. 2024-09-10T08:17:35.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.984+0000 7fd84b42f740 1 -- start start 2024-09-10T08:17:35.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.984+0000 7fd84b42f740 1 Processor -- start 2024-09-10T08:17:35.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd84b42f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b04c70 0x55ca64b0cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:35.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd84b42f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b05590 0x55ca64b0d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:35.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd84b42f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b0fe50 0x55ca64b0d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:35.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd84b42f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ca64b12a20 con 0x55ca64b05590 2024-09-10T08:17:35.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd84b42f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ca64a260f0 con 0x55ca64b04c70 2024-09-10T08:17:35.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd84b42f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ca64b12ba0 con 0x55ca64b0fe50 2024-09-10T08:17:35.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd84b42f740 10 client.0 _ll_register_callbacks cb 0x55ca64a0dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:35.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd845e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b04c70 0x55ca64b0cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:35.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd845e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b04c70 0x55ca64b0cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:57492/0 (socket says 172.21.15.2:57492) 2024-09-10T08:17:35.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd845400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b05590 0x55ca64b0d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:35.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd845e00640 1 -- 172.21.15.2:0/4086037186 learned_addr learned my addr 172.21.15.2:0/4086037186 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:35.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd846800640 1 --2- 172.21.15.2:0/4086037186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b0fe50 0x55ca64b0d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:35.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd845e00640 1 -- 172.21.15.2:0/4086037186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b0fe50 msgr2=0x55ca64b0d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:35.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd845e00640 1 --2- 172.21.15.2:0/4086037186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b0fe50 0x55ca64b0d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd845e00640 1 -- 172.21.15.2:0/4086037186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b05590 msgr2=0x55ca64b0d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:35.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd845e00640 1 --2- 172.21.15.2:0/4086037186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b05590 0x55ca64b0d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.985+0000 7fd845e00640 1 -- 172.21.15.2:0/4086037186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca647dc3c0 con 0x55ca64b04c70 2024-09-10T08:17:35.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.986+0000 7fd845e00640 1 --2- 172.21.15.2:0/4086037186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b04c70 0x55ca64b0cca0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd8340048f0 tx=0x7fd834005430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b8c59b3c8b83bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:35.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.986+0000 7fd823400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:17:35.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.986+0000 7fd823400640 1 -- 172.21.15.2:0/4086037186 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834027020 con 0x55ca64b04c70 2024-09-10T08:17:35.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.986+0000 7fd84b42f740 1 -- 172.21.15.2:0/4086037186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd834004c80 con 0x55ca64b04c70 2024-09-10T08:17:35.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.986+0000 7fd823400640 1 -- 172.21.15.2:0/4086037186 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd834004460 con 0x55ca64b04c70 2024-09-10T08:17:35.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.987+0000 7fd823400640 1 -- 172.21.15.2:0/4086037186 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd834032050 con 0x55ca64b04c70 2024-09-10T08:17:35.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.987+0000 7fd84b42f740 10 client.28064 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:35.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.987+0000 7fd84b42f740 1 -- 172.21.15.2:0/4086037186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca64b1e9c0 con 0x55ca64b04c70 2024-09-10T08:17:35.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.987+0000 7fd823400640 1 -- 172.21.15.2:0/4086037186 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd834004c80 con 0x55ca64b04c70 2024-09-10T08:17:35.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.987+0000 7fd823400640 1 -- 172.21.15.2:0/4086037186 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd834003c00 con 0x55ca64b04c70 2024-09-10T08:17:35.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.987+0000 7fd84b42f740 10 client.28064 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:35.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.987+0000 7fd84b42f740 10 client.28064 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:17:35.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.988+0000 7fd84b42f740 20 client.28064 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:35.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.988+0000 7fd84b42f740 1 --2- 172.21.15.2:0/4086037186 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55ca64b1fdd0 0x55ca64b221c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:35.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.988+0000 7fd84b42f740 4 client.28064 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:17:35.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.988+0000 7fd84b42f740 1 -- 172.21.15.2:0/4086037186 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca64a78950 con 0x55ca64b1fdd0 2024-09-10T08:17:35.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.988+0000 7fd845400640 1 --2- 172.21.15.2:0/4086037186 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55ca64b1fdd0 0x55ca64b221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:35.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.988+0000 7fd845400640 1 --2- 172.21.15.2:0/4086037186 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55ca64b1fdd0 0x55ca64b221c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=42fb9ecb1eff12fc server_cookie=be6f76d116085d97 in_seq=0 out_seq=0 2024-09-10T08:17:35.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.988+0000 7fd823400640 10 client.28064 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:17:35.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.989+0000 7fd823400640 1 -- 172.21.15.2:0/4086037186 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca64a78950 con 0x55ca64b1fdd0 2024-09-10T08:17:35.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.989+0000 7fd823400640 10 client.28064 handle_command_reply: tid=0 2024-09-10T08:17:35.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.989+0000 7fd84b42f740 1 client.28064 shutdown 2024-09-10T08:17:35.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.989+0000 7fd84b42f740 1 -- 172.21.15.2:0/4086037186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b04c70 msgr2=0x55ca64b0cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:35.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.989+0000 7fd84b42f740 1 --2- 172.21.15.2:0/4086037186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b04c70 0x55ca64b0cca0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd8340048f0 tx=0x7fd834005430 comp rx=0 tx=0).stop 2024-09-10T08:17:35.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.989+0000 7fd84b42f740 1 -- 172.21.15.2:0/4086037186 shutdown_connections 2024-09-10T08:17:35.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.990+0000 7fd84b42f740 1 --2- 172.21.15.2:0/4086037186 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55ca64b1fdd0 0x55ca64b221c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.990+0000 7fd84b42f740 1 --2- 172.21.15.2:0/4086037186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b0fe50 0x55ca64b0d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.990+0000 7fd84b42f740 1 --2- 172.21.15.2:0/4086037186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b05590 0x55ca64b0d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.990+0000 7fd84b42f740 1 --2- 172.21.15.2:0/4086037186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b04c70 0x55ca64b0cca0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:35.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.990+0000 7fd823400640 0 client.28064 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:17:35.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.990+0000 7fd84b42f740 1 -- 172.21.15.2:0/4086037186 >> 172.21.15.2:0/4086037186 conn(0x55ca64b01200 msgr2=0x55ca64b00380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:35.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.990+0000 7fd84b42f740 1 -- 172.21.15.2:0/4086037186 shutdown_connections 2024-09-10T08:17:35.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.990+0000 7fd84b42f740 1 -- 172.21.15.2:0/4086037186 wait complete. 2024-09-10T08:17:35.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:35.990+0000 7fd84b42f740 20 client.28064 trim_cache size 0 max 16384 2024-09-10T08:17:36.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.003+0000 7fd84b42f740 1 Processor -- start 2024-09-10T08:17:36.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.004+0000 7fd84b42f740 1 -- start start 2024-09-10T08:17:36.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.004+0000 7fd84b42f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 0x55ca64b0efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:36.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.004+0000 7fd84b42f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b05590 0x55ca64b279d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:36.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.004+0000 7fd84b42f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b0fe50 0x55ca64b27f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:36.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.004+0000 7fd84b42f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401b38f0 con 0x55ca64b04c70 2024-09-10T08:17:36.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.004+0000 7fd84b42f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401b11c0 con 0x55ca64b05590 2024-09-10T08:17:36.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.004+0000 7fd84b42f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401af010 con 0x55ca64b0fe50 2024-09-10T08:17:36.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.004+0000 7fd845e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 0x55ca64b0efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:36.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.004+0000 7fd845e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 0x55ca64b0efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50108/0 (socket says 172.21.15.2:50108) 2024-09-10T08:17:36.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.004+0000 7fd845e00640 1 -- 172.21.15.2:0/298886044 learned_addr learned my addr 172.21.15.2:0/298886044 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:36.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.004+0000 7fd845400640 1 --2- 172.21.15.2:0/298886044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b05590 0x55ca64b279d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:36.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.005+0000 7fd846800640 1 --2- 172.21.15.2:0/298886044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b0fe50 0x55ca64b27f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:36.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.005+0000 7fd845400640 1 -- 172.21.15.2:0/298886044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b0fe50 msgr2=0x55ca64b27f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:36.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.005+0000 7fd845400640 1 --2- 172.21.15.2:0/298886044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b0fe50 0x55ca64b27f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.005+0000 7fd845400640 1 -- 172.21.15.2:0/298886044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 msgr2=0x55ca64b0efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:36.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.005+0000 7fd845400640 1 --2- 172.21.15.2:0/298886044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 0x55ca64b0efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.005+0000 7fd845400640 1 -- 172.21.15.2:0/298886044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca64afc770 con 0x55ca64b05590 2024-09-10T08:17:36.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.005+0000 7fd845e00640 1 --2- 172.21.15.2:0/298886044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 0x55ca64b0efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:36.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.005+0000 7fd845400640 1 --2- 172.21.15.2:0/298886044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b05590 0x55ca64b279d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd830005940 tx=0x7fd830009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d474328cdf9c150 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:36.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.006+0000 7fd82e600640 1 -- 172.21.15.2:0/298886044 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830035040 con 0x55ca64b05590 2024-09-10T08:17:36.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.006+0000 7fd82e600640 1 -- 172.21.15.2:0/298886044 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd830003cd0 con 0x55ca64b05590 2024-09-10T08:17:36.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.006+0000 7fd82e600640 1 -- 172.21.15.2:0/298886044 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830052b90 con 0x55ca64b05590 2024-09-10T08:17:36.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.006+0000 7fd84b42f740 1 -- 172.21.15.2:0/298886044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b05590 msgr2=0x55ca64b279d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:36.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.006+0000 7fd84b42f740 1 --2- 172.21.15.2:0/298886044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b05590 0x55ca64b279d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd830005940 tx=0x7fd830009260 comp rx=0 tx=0).stop 2024-09-10T08:17:36.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.006+0000 7fd84b42f740 1 -- 172.21.15.2:0/298886044 shutdown_connections 2024-09-10T08:17:36.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.006+0000 7fd84b42f740 1 --2- 172.21.15.2:0/298886044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b0fe50 0x55ca64b27f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.006+0000 7fd84b42f740 1 --2- 172.21.15.2:0/298886044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b05590 0x55ca64b279d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.006+0000 7fd84b42f740 1 --2- 172.21.15.2:0/298886044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 0x55ca64b0efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.006+0000 7fd84b42f740 1 -- 172.21.15.2:0/298886044 >> 172.21.15.2:0/298886044 conn(0x55ca64b01200 msgr2=0x55ca64b0bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.006+0000 7fd84b42f740 1 -- 172.21.15.2:0/298886044 shutdown_connections 2024-09-10T08:17:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.006+0000 7fd84b42f740 1 -- 172.21.15.2:0/298886044 wait complete. 2024-09-10T08:17:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.007+0000 7fd84b42f740 1 -- start start 2024-09-10T08:17:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.007+0000 7fd84b42f740 1 Processor -- start 2024-09-10T08:17:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.007+0000 7fd84b42f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 0x55ca64b185e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.007+0000 7fd84b42f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 0x55ca64b09a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.007+0000 7fd84b42f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b0fe50 0x55ca64b09fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd84b42f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401af680 con 0x55ca64b04c70 2024-09-10T08:17:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd84b42f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401b0af0 con 0x55ca64b0fe50 2024-09-10T08:17:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd84b42f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401afd50 con 0x55ca64b05590 2024-09-10T08:17:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd84b42f740 10 client.0 _ll_register_callbacks cb 0x7fd830009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd845e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 0x55ca64b185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd845400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 0x55ca64b09a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd845e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 0x55ca64b185e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50118/0 (socket says 172.21.15.2:50118) 2024-09-10T08:17:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd845400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 0x55ca64b09a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56828/0 (socket says 172.21.15.2:56828) 2024-09-10T08:17:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd845e00640 1 -- 172.21.15.2:0/43305429 learned_addr learned my addr 172.21.15.2:0/43305429 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd846800640 1 --2- 172.21.15.2:0/43305429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b0fe50 0x55ca64b09fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd845400640 1 -- 172.21.15.2:0/43305429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b0fe50 msgr2=0x55ca64b09fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd845400640 1 --2- 172.21.15.2:0/43305429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b0fe50 0x55ca64b09fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd845400640 1 -- 172.21.15.2:0/43305429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 msgr2=0x55ca64b185e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd845400640 1 --2- 172.21.15.2:0/43305429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 0x55ca64b185e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd845400640 1 -- 172.21.15.2:0/43305429 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca64b0a810 con 0x55ca64b05590 2024-09-10T08:17:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.008+0000 7fd845e00640 1 --2- 172.21.15.2:0/43305429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 0x55ca64b185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.009+0000 7fd845400640 1 --2- 172.21.15.2:0/43305429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 0x55ca64b09a60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd830031a70 tx=0x7fd830004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deda87403e648f3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.009+0000 7fd823400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:17:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.009+0000 7fd823400640 1 -- 172.21.15.2:0/43305429 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830035070 con 0x55ca64b05590 2024-09-10T08:17:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.009+0000 7fd823400640 1 -- 172.21.15.2:0/43305429 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd8300522f0 con 0x55ca64b05590 2024-09-10T08:17:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.009+0000 7fd84b42f740 1 -- 172.21.15.2:0/43305429 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd830003cd0 con 0x55ca64b05590 2024-09-10T08:17:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.009+0000 7fd823400640 1 -- 172.21.15.2:0/43305429 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd830057070 con 0x55ca64b05590 2024-09-10T08:17:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.009+0000 7fd84b42f740 10 client.28003 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.009+0000 7fd84b42f740 1 -- 172.21.15.2:0/43305429 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca64b8aa40 con 0x55ca64b05590 2024-09-10T08:17:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.010+0000 7fd823400640 1 -- 172.21.15.2:0/43305429 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd830003cd0 con 0x55ca64b05590 2024-09-10T08:17:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.010+0000 7fd823400640 1 -- 172.21.15.2:0/43305429 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd83003d450 con 0x55ca64b05590 2024-09-10T08:17:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.010+0000 7fd84b42f740 10 client.28003 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.010+0000 7fd84b42f740 10 client.28003 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:17:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.010+0000 7fd84b42f740 20 client.28003 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.010+0000 7fd84b42f740 1 --2- 172.21.15.2:0/43305429 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55ca64b8b0e0 0x55ca64b8d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.010+0000 7fd84b42f740 4 client.28003 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:17:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.010+0000 7fd84b42f740 1 -- 172.21.15.2:0/43305429 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca64a72e60 con 0x55ca64b8b0e0 2024-09-10T08:17:36.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.011+0000 7fd845e00640 1 --2- 172.21.15.2:0/43305429 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55ca64b8b0e0 0x55ca64b8d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:36.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.011+0000 7fd845e00640 1 --2- 172.21.15.2:0/43305429 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55ca64b8b0e0 0x55ca64b8d500 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f19dde53a07fcfd server_cookie=62d68a9c5af392b8 in_seq=0 out_seq=0 2024-09-10T08:17:36.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.011+0000 7fd823400640 10 client.28003 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:17:36.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.011+0000 7fd823400640 1 -- 172.21.15.2:0/43305429 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca64a72e60 con 0x55ca64b8b0e0 2024-09-10T08:17:36.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.011+0000 7fd823400640 10 client.28003 handle_command_reply: tid=0 2024-09-10T08:17:36.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.011+0000 7fd84b42f740 1 client.28003 shutdown 2024-09-10T08:17:36.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.012+0000 7fd84b42f740 1 -- 172.21.15.2:0/43305429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 msgr2=0x55ca64b09a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:36.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.012+0000 7fd84b42f740 1 --2- 172.21.15.2:0/43305429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 0x55ca64b09a60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd830031a70 tx=0x7fd830004610 comp rx=0 tx=0).stop 2024-09-10T08:17:36.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.012+0000 7fd84b42f740 1 -- 172.21.15.2:0/43305429 shutdown_connections 2024-09-10T08:17:36.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.012+0000 7fd84b42f740 1 --2- 172.21.15.2:0/43305429 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55ca64b8b0e0 0x55ca64b8d500 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.012+0000 7fd84b42f740 1 --2- 172.21.15.2:0/43305429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca64b0fe50 0x55ca64b09fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.012+0000 7fd84b42f740 1 --2- 172.21.15.2:0/43305429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca64b05590 0x55ca64b09a60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.012+0000 7fd84b42f740 1 --2- 172.21.15.2:0/43305429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca64b04c70 0x55ca64b185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.012+0000 7fd84b42f740 1 -- 172.21.15.2:0/43305429 >> 172.21.15.2:0/43305429 conn(0x55ca64b01200 msgr2=0x55ca64b00380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:36.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.012+0000 7fd823400640 0 client.28003 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:17:36.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.012+0000 7fd84b42f740 1 -- 172.21.15.2:0/43305429 shutdown_connections 2024-09-10T08:17:36.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.012+0000 7fd84b42f740 1 -- 172.21.15.2:0/43305429 wait complete. 2024-09-10T08:17:36.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.012+0000 7fd84b42f740 20 client.28003 trim_cache size 0 max 16384 2024-09-10T08:17:36.018 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:17:36.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.013+0000 7fd848800640 1 -- 172.21.15.2:0/15875828 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8280a2cd0 msgr2=0x7fd8280a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:36.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.013+0000 7fd848800640 1 --2- 172.21.15.2:0/15875828 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8280a2cd0 0x7fd8280a5190 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd830009230 tx=0x7fd8300314e0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.013+0000 7fd848800640 1 -- 172.21.15.2:0/15875828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd840109140 msgr2=0x7fd84019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:36.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.013+0000 7fd848800640 1 --2- 172.21.15.2:0/15875828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd840109140 0x7fd84019ef80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00e940 tx=0x7fd83c00ee00 comp rx=0 tx=0).stop 2024-09-10T08:17:36.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.013+0000 7fd848800640 1 -- 172.21.15.2:0/15875828 shutdown_connections 2024-09-10T08:17:36.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.013+0000 7fd848800640 1 --2- 172.21.15.2:0/15875828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd840109140 0x7fd84019ef80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.013+0000 7fd848800640 1 --2- 172.21.15.2:0/15875828 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd8280a2cd0 0x7fd8280a5190 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.013+0000 7fd848800640 1 --2- 172.21.15.2:0/15875828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8401049c0 0x7fd84019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.013+0000 7fd848800640 1 --2- 172.21.15.2:0/15875828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd840104010 0x7fd84019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:36.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.013+0000 7fd848800640 1 -- 172.21.15.2:0/15875828 >> 172.21.15.2:0/15875828 conn(0x7fd8400ffa50 msgr2=0x7fd840100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:36.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.013+0000 7fd848800640 1 -- 172.21.15.2:0/15875828 shutdown_connections 2024-09-10T08:17:36.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:36.013+0000 7fd848800640 1 -- 172.21.15.2:0/15875828 wait complete. 2024-09-10T08:17:36.881 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:17:37.256 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:17:37.656 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:37 smithi002 ceph-mon[30563]: pgmap v671: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-10T08:17:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:37 smithi155 ceph-mon[35405]: pgmap v671: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-10T08:17:37.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:37 smithi107 ceph-mon[35048]: pgmap v671: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-10T08:17:37.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.926+0000 7f70bf800640 1 -- 172.21.15.2:0/2190131578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70b8104010 msgr2=0x7f70b81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:37.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.926+0000 7f70bf800640 1 --2- 172.21.15.2:0/2190131578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70b8104010 0x7f70b81043f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009900 tx=0x7f70a802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.926+0000 7f70bf800640 1 -- 172.21.15.2:0/2190131578 shutdown_connections 2024-09-10T08:17:37.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.926+0000 7f70bf800640 1 --2- 172.21.15.2:0/2190131578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70b8109140 0x7f70b8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.926+0000 7f70bf800640 1 --2- 172.21.15.2:0/2190131578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b81049c0 0x7f70b8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.926+0000 7f70bf800640 1 --2- 172.21.15.2:0/2190131578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70b8104010 0x7f70b81043f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.926+0000 7f70bf800640 1 -- 172.21.15.2:0/2190131578 >> 172.21.15.2:0/2190131578 conn(0x7f70b80ffa50 msgr2=0x7f70b8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:37.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.926+0000 7f70bf800640 1 -- 172.21.15.2:0/2190131578 shutdown_connections 2024-09-10T08:17:37.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.926+0000 7f70bf800640 1 -- 172.21.15.2:0/2190131578 wait complete. 2024-09-10T08:17:37.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.927+0000 7f70bf800640 1 Processor -- start 2024-09-10T08:17:37.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.927+0000 7f70bf800640 1 -- start start 2024-09-10T08:17:37.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.928+0000 7f70bf800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70b8104010 0x7f70b819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.928+0000 7f70bf800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b81049c0 0x7f70b819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.928+0000 7f70bf800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70b8109140 0x7f70b819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.928+0000 7f70bf800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81141c0 con 0x7f70b81049c0 2024-09-10T08:17:37.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.928+0000 7f70bf800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8114040 con 0x7f70b8109140 2024-09-10T08:17:37.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.928+0000 7f70bf800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8114340 con 0x7f70b8104010 2024-09-10T08:17:37.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.928+0000 7f70afe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b81049c0 0x7f70b819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.928+0000 7f70afe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b81049c0 0x7f70b819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33384/0 (socket says 172.21.15.2:33384) 2024-09-10T08:17:37.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.928+0000 7f70afe00640 1 -- 172.21.15.2:0/762290759 learned_addr learned my addr 172.21.15.2:0/762290759 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:37.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.929+0000 7f70bd800640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70b8109140 0x7f70b819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.929+0000 7f70bce00640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70b8104010 0x7f70b819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.929+0000 7f70afe00640 1 -- 172.21.15.2:0/762290759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70b8104010 msgr2=0x7f70b819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:37.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.929+0000 7f70afe00640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70b8104010 0x7f70b819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.929+0000 7f70afe00640 1 -- 172.21.15.2:0/762290759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70b8109140 msgr2=0x7f70b819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:37.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.929+0000 7f70afe00640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70b8109140 0x7f70b819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.929+0000 7f70afe00640 1 -- 172.21.15.2:0/762290759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b819f600 con 0x7f70b81049c0 2024-09-10T08:17:37.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.930+0000 7f70afe00640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b81049c0 0x7f70b819ab70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f70a000ea60 tx=0x7f70a000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:37.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.930+0000 7f70b6000640 1 -- 172.21.15.2:0/762290759 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a000cde0 con 0x7f70b81049c0 2024-09-10T08:17:37.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.930+0000 7f70b6000640 1 -- 172.21.15.2:0/762290759 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a00035f0 con 0x7f70b81049c0 2024-09-10T08:17:37.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.930+0000 7f70b6000640 1 -- 172.21.15.2:0/762290759 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0010670 con 0x7f70b81049c0 2024-09-10T08:17:37.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.930+0000 7f70bf800640 1 -- 172.21.15.2:0/762290759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b819f8f0 con 0x7f70b81049c0 2024-09-10T08:17:37.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.930+0000 7f70bf800640 1 -- 172.21.15.2:0/762290759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b806bc10 con 0x7f70b81049c0 2024-09-10T08:17:37.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.933+0000 7f70b6000640 1 -- 172.21.15.2:0/762290759 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f70a00037a0 con 0x7f70b81049c0 2024-09-10T08:17:37.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.935+0000 7f70c2243740 1 Processor -- start 2024-09-10T08:17:37.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.935+0000 7f70c2243740 1 -- start start 2024-09-10T08:17:37.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.935+0000 7f70b6000640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f709c0a2c00 0x7f709c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.935+0000 7f70b6000640 1 -- 172.21.15.2:0/762290759 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f70a00cf6e0 con 0x7f70b81049c0 2024-09-10T08:17:37.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.935+0000 7f70c2243740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a798c70 0x55595a799050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.935+0000 7f70c2243740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a799590 0x55595a7a3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.935+0000 7f70c2243740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a7a3e50 0x55595a7a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.935+0000 7f70c2243740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55595a6ff3e0 con 0x55595a798c70 2024-09-10T08:17:37.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.935+0000 7f70c2243740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55595a6a6d80 con 0x55595a799590 2024-09-10T08:17:37.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.935+0000 7f70c2243740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55595a6fb3a0 con 0x55595a7a3e50 2024-09-10T08:17:37.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.935+0000 7f70bd800640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f709c0a2c00 0x7f709c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.936+0000 7f70afe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a799590 0x55595a7a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.936+0000 7f70afe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a799590 0x55595a7a3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42324/0 (socket says 172.21.15.2:42324) 2024-09-10T08:17:37.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.936+0000 7f70afe00640 1 -- 172.21.15.2:0/514931594 learned_addr learned my addr 172.21.15.2:0/514931594 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:37.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.936+0000 7f70bce00640 1 --2- 172.21.15.2:0/514931594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a798c70 0x55595a799050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.936+0000 7f70bd800640 1 --2- 172.21.15.2:0/514931594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a7a3e50 0x55595a7a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.936+0000 7f70afe00640 1 -- 172.21.15.2:0/514931594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a7a3e50 msgr2=0x55595a7a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:37.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.936+0000 7f70afe00640 1 --2- 172.21.15.2:0/514931594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a7a3e50 0x55595a7a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.936+0000 7f70afe00640 1 -- 172.21.15.2:0/514931594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a798c70 msgr2=0x55595a799050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:37.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.936+0000 7f70afe00640 1 --2- 172.21.15.2:0/514931594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a798c70 0x55595a799050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.936+0000 7f70afe00640 1 -- 172.21.15.2:0/514931594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55595a6c5ae0 con 0x55595a799590 2024-09-10T08:17:37.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.936+0000 7f70bce00640 1 --2- 172.21.15.2:0/514931594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a798c70 0x55595a799050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:17:37.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.936+0000 7f70afe00640 1 --2- 172.21.15.2:0/514931594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a799590 0x55595a7a3910 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f70a00a4230 tx=0x7f70a009b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a82f2f67c2e8641f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:37.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.936+0000 7f70bd800640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f709c0a2c00 0x7f709c0a50c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f70b0005fd0 tx=0x7f70b0013ae0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:37.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.937+0000 7f70ae000640 1 -- 172.21.15.2:0/514931594 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a009baf0 con 0x55595a799590 2024-09-10T08:17:37.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.937+0000 7f70ae000640 1 -- 172.21.15.2:0/514931594 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a009fea0 con 0x55595a799590 2024-09-10T08:17:37.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.937+0000 7f70ae000640 1 -- 172.21.15.2:0/514931594 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0098160 con 0x55595a799590 2024-09-10T08:17:37.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.937+0000 7f70c2243740 1 -- 172.21.15.2:0/514931594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a799590 msgr2=0x55595a7a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:37.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.937+0000 7f70c2243740 1 --2- 172.21.15.2:0/514931594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a799590 0x55595a7a3910 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f70a00a4230 tx=0x7f70a009b630 comp rx=0 tx=0).stop 2024-09-10T08:17:37.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.938+0000 7f70c2243740 1 -- 172.21.15.2:0/514931594 shutdown_connections 2024-09-10T08:17:37.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.938+0000 7f70c2243740 1 --2- 172.21.15.2:0/514931594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a7a3e50 0x55595a7a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.938+0000 7f70c2243740 1 --2- 172.21.15.2:0/514931594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a799590 0x55595a7a3910 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.938+0000 7f70c2243740 1 --2- 172.21.15.2:0/514931594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a798c70 0x55595a799050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.938+0000 7f70c2243740 1 -- 172.21.15.2:0/514931594 >> 172.21.15.2:0/514931594 conn(0x55595a795200 msgr2=0x55595a7955e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:37.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.938+0000 7f70c2243740 1 -- 172.21.15.2:0/514931594 shutdown_connections 2024-09-10T08:17:37.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.938+0000 7f70c2243740 1 -- 172.21.15.2:0/514931594 wait complete. 2024-09-10T08:17:37.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.938+0000 7f70c2243740 1 -- start start 2024-09-10T08:17:37.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.938+0000 7f70c2243740 1 Processor -- start 2024-09-10T08:17:37.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.939+0000 7f70c2243740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a798c70 0x55595a7a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.939+0000 7f70c2243740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a799590 0x55595a7a1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.939+0000 7f70c2243740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a7a3e50 0x55595a7a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.939+0000 7f70c2243740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55595a7a6a20 con 0x55595a7a3e50 2024-09-10T08:17:37.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.939+0000 7f70c2243740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55595a6ba0f0 con 0x55595a798c70 2024-09-10T08:17:37.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.939+0000 7f70c2243740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55595a7a6ba0 con 0x55595a799590 2024-09-10T08:17:37.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.939+0000 7f70c2243740 10 client.0 _ll_register_callbacks cb 0x55595a6a1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:37.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.939+0000 7f70bce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a798c70 0x55595a7a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.939+0000 7f70bce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a798c70 0x55595a7a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42336/0 (socket says 172.21.15.2:42336) 2024-09-10T08:17:37.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.939+0000 7f70bce00640 1 -- 172.21.15.2:0/937846527 learned_addr learned my addr 172.21.15.2:0/937846527 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:37.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.939+0000 7f70bd800640 1 --2- 172.21.15.2:0/937846527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a7a3e50 0x55595a7a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.939+0000 7f70afe00640 1 --2- 172.21.15.2:0/937846527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a799590 0x55595a7a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.940+0000 7f70bce00640 1 -- 172.21.15.2:0/937846527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a799590 msgr2=0x55595a7a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.940+0000 7f70bce00640 1 --2- 172.21.15.2:0/937846527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a799590 0x55595a7a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.940+0000 7f70bce00640 1 -- 172.21.15.2:0/937846527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a7a3e50 msgr2=0x55595a7a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.940+0000 7f70bce00640 1 --2- 172.21.15.2:0/937846527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a7a3e50 0x55595a7a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.940+0000 7f70bce00640 1 -- 172.21.15.2:0/937846527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55595a4703c0 con 0x55595a798c70 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.940+0000 7f70bce00640 1 --2- 172.21.15.2:0/937846527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a798c70 0x55595a7a0d20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009230 tx=0x7f70a80042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92b138f49761f630 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.940+0000 7f70a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.940+0000 7f70a6a00640 1 -- 172.21.15.2:0/937846527 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8035040 con 0x55595a798c70 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.940+0000 7f70a6a00640 1 -- 172.21.15.2:0/937846527 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f70a8005450 con 0x55595a798c70 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.940+0000 7f70c2243740 1 -- 172.21.15.2:0/937846527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a009fea0 con 0x55595a798c70 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.941+0000 7f70a6a00640 1 -- 172.21.15.2:0/937846527 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f70a804d050 con 0x55595a798c70 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.941+0000 7f70c2243740 10 client.28082 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.941+0000 7f70c2243740 1 -- 172.21.15.2:0/937846527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55595a7b29e0 con 0x55595a798c70 2024-09-10T08:17:37.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.941+0000 7f70a6a00640 1 -- 172.21.15.2:0/937846527 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a009fea0 con 0x55595a798c70 2024-09-10T08:17:37.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.941+0000 7f70a6a00640 1 -- 172.21.15.2:0/937846527 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f70a8003850 con 0x55595a798c70 2024-09-10T08:17:37.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.941+0000 7f70c2243740 10 client.28082 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:37.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.941+0000 7f70c2243740 10 client.28082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:37.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.941+0000 7f70c2243740 20 client.28082 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:37.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.942+0000 7f70c2243740 1 --2- 172.21.15.2:0/937846527 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55595a7b3ea0 0x55595a7b6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.942+0000 7f70c2243740 4 client.28082 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:17:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.942+0000 7f70c2243740 1 -- 172.21.15.2:0/937846527 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55595a70c950 con 0x55595a7b3ea0 2024-09-10T08:17:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.942+0000 7f70afe00640 1 --2- 172.21.15.2:0/937846527 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55595a7b3ea0 0x55595a7b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.943+0000 7f70afe00640 1 --2- 172.21.15.2:0/937846527 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55595a7b3ea0 0x55595a7b6290 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=980518768d03311 server_cookie=6700f19dc1eb21ed in_seq=0 out_seq=0 2024-09-10T08:17:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.943+0000 7f70a6a00640 10 client.28082 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.943+0000 7f70a6a00640 1 -- 172.21.15.2:0/937846527 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55595a70c950 con 0x55595a7b3ea0 2024-09-10T08:17:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.943+0000 7f70a6a00640 10 client.28082 handle_command_reply: tid=0 2024-09-10T08:17:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.943+0000 7f70c2243740 1 client.28082 shutdown 2024-09-10T08:17:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.944+0000 7f70c2243740 1 -- 172.21.15.2:0/937846527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a798c70 msgr2=0x55595a7a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.944+0000 7f70c2243740 1 --2- 172.21.15.2:0/937846527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a798c70 0x55595a7a0d20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009230 tx=0x7f70a80042e0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.944+0000 7f70c2243740 1 -- 172.21.15.2:0/937846527 shutdown_connections 2024-09-10T08:17:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.944+0000 7f70c2243740 1 --2- 172.21.15.2:0/937846527 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55595a7b3ea0 0x55595a7b6290 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.944+0000 7f70c2243740 1 --2- 172.21.15.2:0/937846527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a7a3e50 0x55595a7a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.944+0000 7f70c2243740 1 --2- 172.21.15.2:0/937846527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a799590 0x55595a7a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.944+0000 7f70c2243740 1 --2- 172.21.15.2:0/937846527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a798c70 0x55595a7a0d20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.944+0000 7f70a6a00640 0 client.28082 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:37.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.944+0000 7f70c2243740 1 -- 172.21.15.2:0/937846527 >> 172.21.15.2:0/937846527 conn(0x55595a795200 msgr2=0x55595a79af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:37.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.944+0000 7f70c2243740 1 -- 172.21.15.2:0/937846527 shutdown_connections 2024-09-10T08:17:37.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.944+0000 7f70c2243740 1 -- 172.21.15.2:0/937846527 wait complete. 2024-09-10T08:17:37.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.944+0000 7f70c2243740 20 client.28082 trim_cache size 0 max 16384 2024-09-10T08:17:37.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.958+0000 7f70c2243740 1 Processor -- start 2024-09-10T08:17:37.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.958+0000 7f70c2243740 1 -- start start 2024-09-10T08:17:37.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.959+0000 7f70c2243740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a795200 0x55595a794270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.959+0000 7f70c2243740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a798c70 0x55595a7bbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.959+0000 7f70c2243740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a799590 0x55595a7bc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.959+0000 7f70c2243740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81bc280 con 0x55595a799590 2024-09-10T08:17:37.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.959+0000 7f70c2243740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81b9b50 con 0x55595a795200 2024-09-10T08:17:37.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.959+0000 7f70c2243740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8073bf0 con 0x55595a798c70 2024-09-10T08:17:37.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.959+0000 7f70bce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a795200 0x55595a794270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.959+0000 7f70bce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a795200 0x55595a794270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42346/0 (socket says 172.21.15.2:42346) 2024-09-10T08:17:37.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.959+0000 7f70bce00640 1 -- 172.21.15.2:0/1390011858 learned_addr learned my addr 172.21.15.2:0/1390011858 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:37.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.959+0000 7f70bd800640 1 --2- 172.21.15.2:0/1390011858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a799590 0x55595a7bc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.959+0000 7f70afe00640 1 --2- 172.21.15.2:0/1390011858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a798c70 0x55595a7bbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.960+0000 7f70bce00640 1 -- 172.21.15.2:0/1390011858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a798c70 msgr2=0x55595a7bbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:37.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.960+0000 7f70bce00640 1 --2- 172.21.15.2:0/1390011858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a798c70 0x55595a7bbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.960+0000 7f70bce00640 1 -- 172.21.15.2:0/1390011858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a799590 msgr2=0x55595a7bc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:37.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.960+0000 7f70bce00640 1 --2- 172.21.15.2:0/1390011858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a799590 0x55595a7bc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.960+0000 7f70bce00640 1 -- 172.21.15.2:0/1390011858 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55595a7b68f0 con 0x55595a795200 2024-09-10T08:17:37.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.960+0000 7f70afe00640 1 --2- 172.21.15.2:0/1390011858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a798c70 0x55595a7bbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:37.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.960+0000 7f70bce00640 1 --2- 172.21.15.2:0/1390011858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a795200 0x55595a794270 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f70a8004080 tx=0x7f70a8009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ae6513f2227581a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:37.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.960+0000 7f70ad600640 1 -- 172.21.15.2:0/1390011858 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8056070 con 0x55595a795200 2024-09-10T08:17:37.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.960+0000 7f70ad600640 1 -- 172.21.15.2:0/1390011858 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a8003e10 con 0x55595a795200 2024-09-10T08:17:37.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.960+0000 7f70ad600640 1 -- 172.21.15.2:0/1390011858 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8035070 con 0x55595a795200 2024-09-10T08:17:37.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.961+0000 7f70c2243740 1 -- 172.21.15.2:0/1390011858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a795200 msgr2=0x55595a794270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:37.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.961+0000 7f70c2243740 1 --2- 172.21.15.2:0/1390011858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a795200 0x55595a794270 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f70a8004080 tx=0x7f70a8009680 comp rx=0 tx=0).stop 2024-09-10T08:17:37.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.961+0000 7f70c2243740 1 -- 172.21.15.2:0/1390011858 shutdown_connections 2024-09-10T08:17:37.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.961+0000 7f70c2243740 1 --2- 172.21.15.2:0/1390011858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a799590 0x55595a7bc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.961+0000 7f70c2243740 1 --2- 172.21.15.2:0/1390011858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a798c70 0x55595a7bbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.961+0000 7f70c2243740 1 --2- 172.21.15.2:0/1390011858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a795200 0x55595a794270 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.961+0000 7f70c2243740 1 -- 172.21.15.2:0/1390011858 >> 172.21.15.2:0/1390011858 conn(0x55595a79c710 msgr2=0x55595a793b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:37.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.961+0000 7f70c2243740 1 -- 172.21.15.2:0/1390011858 shutdown_connections 2024-09-10T08:17:37.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.961+0000 7f70c2243740 1 -- 172.21.15.2:0/1390011858 wait complete. 2024-09-10T08:17:37.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.962+0000 7f70c2243740 1 -- start start 2024-09-10T08:17:37.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.962+0000 7f70c2243740 1 Processor -- start 2024-09-10T08:17:37.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.962+0000 7f70c2243740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a795200 0x55595a7ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.962+0000 7f70c2243740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a798c70 0x55595a7a7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.962+0000 7f70c2243740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a799590 0x55595a7a7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.962+0000 7f70c2243740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8074260 con 0x55595a795200 2024-09-10T08:17:37.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.962+0000 7f70c2243740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81b9480 con 0x55595a799590 2024-09-10T08:17:37.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.962+0000 7f70c2243740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81b86e0 con 0x55595a798c70 2024-09-10T08:17:37.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.963+0000 7f70c2243740 10 client.0 _ll_register_callbacks cb 0x7f70b00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:37.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.963+0000 7f70bce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a795200 0x55595a7ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.963+0000 7f70bce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a795200 0x55595a7ac630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33434/0 (socket says 172.21.15.2:33434) 2024-09-10T08:17:37.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.963+0000 7f70bce00640 1 -- 172.21.15.2:0/564688467 learned_addr learned my addr 172.21.15.2:0/564688467 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:37.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.963+0000 7f70afe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a798c70 0x55595a7a7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.963+0000 7f70bd800640 1 --2- 172.21.15.2:0/564688467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a799590 0x55595a7a7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.963+0000 7f70bce00640 1 -- 172.21.15.2:0/564688467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a798c70 msgr2=0x55595a7a7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:37.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.963+0000 7f70bce00640 1 --2- 172.21.15.2:0/564688467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a798c70 0x55595a7a7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.963+0000 7f70bce00640 1 -- 172.21.15.2:0/564688467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a799590 msgr2=0x55595a7a7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:37.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.963+0000 7f70bce00640 1 --2- 172.21.15.2:0/564688467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a799590 0x55595a7a7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.963+0000 7f70bce00640 1 -- 172.21.15.2:0/564688467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55595a790770 con 0x55595a795200 2024-09-10T08:17:37.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.964+0000 7f70bce00640 1 --2- 172.21.15.2:0/564688467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a795200 0x55595a7ac630 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f70a803e040 tx=0x7f70a80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d283cc854bc4d784 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:37.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.964+0000 7f70a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:17:37.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.964+0000 7f70a6a00640 1 -- 172.21.15.2:0/564688467 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8056070 con 0x55595a795200 2024-09-10T08:17:37.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.964+0000 7f70a6a00640 1 -- 172.21.15.2:0/564688467 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f70a8005460 con 0x55595a795200 2024-09-10T08:17:37.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.964+0000 7f70c2243740 1 -- 172.21.15.2:0/564688467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a8003e10 con 0x55595a795200 2024-09-10T08:17:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.964+0000 7f70a6a00640 1 -- 172.21.15.2:0/564688467 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f70a8064050 con 0x55595a795200 2024-09-10T08:17:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.964+0000 7f70c2243740 10 client.18057 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.964+0000 7f70c2243740 1 -- 172.21.15.2:0/564688467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55595a81e9e0 con 0x55595a795200 2024-09-10T08:17:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.965+0000 7f70a6a00640 1 -- 172.21.15.2:0/564688467 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a8003e10 con 0x55595a795200 2024-09-10T08:17:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.965+0000 7f70a6a00640 1 -- 172.21.15.2:0/564688467 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f70a80315b0 con 0x55595a795200 2024-09-10T08:17:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.965+0000 7f70c2243740 10 client.18057 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.965+0000 7f70c2243740 10 client.18057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.965+0000 7f70c2243740 20 client.18057 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.965+0000 7f70c2243740 1 --2- 172.21.15.2:0/564688467 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55595a7a3e50 0x55595a820ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.965+0000 7f70c2243740 4 client.18057 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:17:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.965+0000 7f70c2243740 1 -- 172.21.15.2:0/564688467 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55595a706e60 con 0x55595a7a3e50 2024-09-10T08:17:37.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.965+0000 7f70afe00640 1 --2- 172.21.15.2:0/564688467 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55595a7a3e50 0x55595a820ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:37.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.966+0000 7f70afe00640 1 --2- 172.21.15.2:0/564688467 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55595a7a3e50 0x55595a820ff0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf216537f68f70b server_cookie=2bcb234e9bca1069 in_seq=0 out_seq=0 2024-09-10T08:17:37.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.966+0000 7f70a6a00640 10 client.18057 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:37.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.966+0000 7f70a6a00640 1 -- 172.21.15.2:0/564688467 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55595a706e60 con 0x55595a7a3e50 2024-09-10T08:17:37.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.966+0000 7f70a6a00640 10 client.18057 handle_command_reply: tid=0 2024-09-10T08:17:37.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.966+0000 7f70c2243740 1 client.18057 shutdown 2024-09-10T08:17:37.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.966+0000 7f70c2243740 1 -- 172.21.15.2:0/564688467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a795200 msgr2=0x55595a7ac630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:37.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.966+0000 7f70c2243740 1 --2- 172.21.15.2:0/564688467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a795200 0x55595a7ac630 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f70a803e040 tx=0x7f70a80035f0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.966+0000 7f70c2243740 1 -- 172.21.15.2:0/564688467 shutdown_connections 2024-09-10T08:17:37.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.966+0000 7f70c2243740 1 --2- 172.21.15.2:0/564688467 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55595a7a3e50 0x55595a820ff0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.967+0000 7f70c2243740 1 --2- 172.21.15.2:0/564688467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a799590 0x55595a7a7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.967+0000 7f70c2243740 1 --2- 172.21.15.2:0/564688467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55595a798c70 0x55595a7a7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.967+0000 7f70c2243740 1 --2- 172.21.15.2:0/564688467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55595a795200 0x55595a7ac630 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.967+0000 7f70c2243740 1 -- 172.21.15.2:0/564688467 >> 172.21.15.2:0/564688467 conn(0x55595a79c710 msgr2=0x55595a794380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:37.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.967+0000 7f70a6a00640 0 client.18057 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:37.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.967+0000 7f70c2243740 1 -- 172.21.15.2:0/564688467 shutdown_connections 2024-09-10T08:17:37.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.967+0000 7f70c2243740 1 -- 172.21.15.2:0/564688467 wait complete. 2024-09-10T08:17:37.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.967+0000 7f70c2243740 20 client.18057 trim_cache size 0 max 16384 2024-09-10T08:17:37.971 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:17:37.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.967+0000 7f70bf800640 1 -- 172.21.15.2:0/762290759 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f709c0a2c00 msgr2=0x7f709c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:37.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.967+0000 7f70bf800640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f709c0a2c00 0x7f709c0a50c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f70b0005fd0 tx=0x7f70b0013ae0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.967+0000 7f70bf800640 1 -- 172.21.15.2:0/762290759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b81049c0 msgr2=0x7f70b819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:37.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.967+0000 7f70bf800640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b81049c0 0x7f70b819ab70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f70a000ea60 tx=0x7f70a000ef20 comp rx=0 tx=0).stop 2024-09-10T08:17:37.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.968+0000 7f70bf800640 1 -- 172.21.15.2:0/762290759 shutdown_connections 2024-09-10T08:17:37.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.968+0000 7f70bf800640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70b8109140 0x7f70b819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.968+0000 7f70bf800640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b81049c0 0x7f70b819ab70 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.968+0000 7f70bf800640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f709c0a2c00 0x7f709c0a50c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.968+0000 7f70bf800640 1 --2- 172.21.15.2:0/762290759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70b8104010 0x7f70b819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:37.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.968+0000 7f70bf800640 1 -- 172.21.15.2:0/762290759 >> 172.21.15.2:0/762290759 conn(0x7f70b80ffa50 msgr2=0x7f70b8100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:37.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.968+0000 7f70bf800640 1 -- 172.21.15.2:0/762290759 shutdown_connections 2024-09-10T08:17:37.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:37.968+0000 7f70bf800640 1 -- 172.21.15.2:0/762290759 wait complete. 2024-09-10T08:17:39.772 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:17:39.773 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:17:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:39 smithi155 ceph-mon[35405]: pgmap v672: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-10T08:17:39.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:39 smithi107 ceph-mon[35048]: pgmap v672: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-10T08:17:40.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:39 smithi002 ceph-mon[30563]: pgmap v672: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-10T08:17:40.162 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:17:40.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.786+0000 7fcbc9c00640 1 -- 172.21.15.2:0/428480539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc41019c0 msgr2=0x7fcbc4101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:40.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.786+0000 7fcbc9c00640 1 --2- 172.21.15.2:0/428480539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc41019c0 0x7fcbc4101e20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009890 tx=0x7fcbb002a370 comp rx=0 tx=0).stop 2024-09-10T08:17:40.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.786+0000 7fcbc9c00640 1 -- 172.21.15.2:0/428480539 shutdown_connections 2024-09-10T08:17:40.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.786+0000 7fcbc9c00640 1 --2- 172.21.15.2:0/428480539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbc4102360 0x7fcbc410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.786+0000 7fcbc9c00640 1 --2- 172.21.15.2:0/428480539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc41019c0 0x7fcbc4101e20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.786+0000 7fcbc9c00640 1 --2- 172.21.15.2:0/428480539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc41079c0 0x7fcbc4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.786+0000 7fcbc9c00640 1 -- 172.21.15.2:0/428480539 >> 172.21.15.2:0/428480539 conn(0x7fcbc40fd740 msgr2=0x7fcbc40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.786+0000 7fcbc9c00640 1 -- 172.21.15.2:0/428480539 shutdown_connections 2024-09-10T08:17:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.786+0000 7fcbc9c00640 1 -- 172.21.15.2:0/428480539 wait complete. 2024-09-10T08:17:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.787+0000 7fcbc9c00640 1 Processor -- start 2024-09-10T08:17:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.788+0000 7fcbc9c00640 1 -- start start 2024-09-10T08:17:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.788+0000 7fcbc9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc41019c0 0x7fcbc41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.788+0000 7fcbc9c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc4102360 0x7fcbc4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.788+0000 7fcbc9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbc41079c0 0x7fcbc419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.788+0000 7fcbc9c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4111dc0 con 0x7fcbc4102360 2024-09-10T08:17:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.788+0000 7fcbc9c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4111c40 con 0x7fcbc41079c0 2024-09-10T08:17:40.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.788+0000 7fcbc9c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4111f40 con 0x7fcbc41019c0 2024-09-10T08:17:40.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.788+0000 7fcbc3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc4102360 0x7fcbc4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.788+0000 7fcbc3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc4102360 0x7fcbc4198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33454/0 (socket says 172.21.15.2:33454) 2024-09-10T08:17:40.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.788+0000 7fcbc3400640 1 -- 172.21.15.2:0/2462996233 learned_addr learned my addr 172.21.15.2:0/2462996233 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:40.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.788+0000 7fcbc3e00640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc41019c0 0x7fcbc41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.788+0000 7fcbc9200640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbc41079c0 0x7fcbc419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.789+0000 7fcbc3400640 1 -- 172.21.15.2:0/2462996233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc41019c0 msgr2=0x7fcbc41982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:40.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.789+0000 7fcbc3400640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc41019c0 0x7fcbc41982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.789+0000 7fcbc3400640 1 -- 172.21.15.2:0/2462996233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbc41079c0 msgr2=0x7fcbc419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:40.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.789+0000 7fcbc3400640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbc41079c0 0x7fcbc419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.789+0000 7fcbc3400640 1 -- 172.21.15.2:0/2462996233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc419d270 con 0x7fcbc4102360 2024-09-10T08:17:40.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.789+0000 7fcbc3400640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc4102360 0x7fcbc4198800 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0000c00 tx=0x7fcbb0033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.789+0000 7fcbc0c00640 1 -- 172.21.15.2:0/2462996233 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb003f020 con 0x7fcbc4102360 2024-09-10T08:17:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.789+0000 7fcbc0c00640 1 -- 172.21.15.2:0/2462996233 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb0032770 con 0x7fcbc4102360 2024-09-10T08:17:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.789+0000 7fcbc9c00640 1 -- 172.21.15.2:0/2462996233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc419d500 con 0x7fcbc4102360 2024-09-10T08:17:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.789+0000 7fcbc0c00640 1 -- 172.21.15.2:0/2462996233 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0037040 con 0x7fcbc4102360 2024-09-10T08:17:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.789+0000 7fcbc9c00640 1 -- 172.21.15.2:0/2462996233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc41a4e40 con 0x7fcbc4102360 2024-09-10T08:17:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.793+0000 7fcbc0c00640 1 -- 172.21.15.2:0/2462996233 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcbb0042020 con 0x7fcbc4102360 2024-09-10T08:17:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.793+0000 7fcbcde5e740 1 Processor -- start 2024-09-10T08:17:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.793+0000 7fcbcde5e740 1 -- start start 2024-09-10T08:17:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.793+0000 7fcbcde5e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f018cb0 0x55fb6f019090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.793+0000 7fcbcde5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f023950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.793+0000 7fcbcde5e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f023e90 0x55fb6f026280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.793+0000 7fcbcde5e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fb6ef7f290 con 0x55fb6f023e90 2024-09-10T08:17:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.793+0000 7fcbcde5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fb6ef26b40 con 0x55fb6f0195d0 2024-09-10T08:17:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.793+0000 7fcbcde5e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fb6ef7b250 con 0x55fb6f018cb0 2024-09-10T08:17:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.793+0000 7fcbc3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f023e90 0x55fb6f026280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.793+0000 7fcbc9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f018cb0 0x55fb6f019090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.794+0000 7fcbc9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f018cb0 0x55fb6f019090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36340/0 (socket says 172.21.15.2:36340) 2024-09-10T08:17:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.794+0000 7fcbc9200640 1 -- 172.21.15.2:0/1457929136 learned_addr learned my addr 172.21.15.2:0/1457929136 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.794+0000 7fcbc3e00640 1 --2- 172.21.15.2:0/1457929136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f023950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.794+0000 7fcbc9200640 1 -- 172.21.15.2:0/1457929136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 msgr2=0x55fb6f023950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.794+0000 7fcbc9200640 1 --2- 172.21.15.2:0/1457929136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f023950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.794+0000 7fcbc9200640 1 -- 172.21.15.2:0/1457929136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f023e90 msgr2=0x55fb6f026280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.794+0000 7fcbc9200640 1 --2- 172.21.15.2:0/1457929136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f023e90 0x55fb6f026280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.794+0000 7fcbc9200640 1 -- 172.21.15.2:0/1457929136 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb6ecf06a0 con 0x55fb6f018cb0 2024-09-10T08:17:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.794+0000 7fcbc3400640 1 --2- 172.21.15.2:0/1457929136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f023e90 0x55fb6f026280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:40.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.794+0000 7fcbc0c00640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcba80a2b80 0x7fcba80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.794+0000 7fcbc9200640 1 --2- 172.21.15.2:0/1457929136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f018cb0 0x55fb6f019090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009fd0 tx=0x7fcbbc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd7fe9ed3bb0f6b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:40.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.795+0000 7fcbc0c00640 1 -- 172.21.15.2:0/2462996233 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcbb00ef530 con 0x7fcbc4102360 2024-09-10T08:17:40.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.795+0000 7fcbc3e00640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcba80a2b80 0x7fcba80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.795+0000 7fcbb9600640 1 -- 172.21.15.2:0/1457929136 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc004d20 con 0x55fb6f018cb0 2024-09-10T08:17:40.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.795+0000 7fcbb9600640 1 -- 172.21.15.2:0/1457929136 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbbc004050 con 0x55fb6f018cb0 2024-09-10T08:17:40.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.795+0000 7fcbb9600640 1 -- 172.21.15.2:0/1457929136 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc005240 con 0x55fb6f018cb0 2024-09-10T08:17:40.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.796+0000 7fcbcde5e740 1 -- 172.21.15.2:0/1457929136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f018cb0 msgr2=0x55fb6f019090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:40.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.796+0000 7fcbc3e00640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcba80a2b80 0x7fcba80a5040 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55fb6f0160a0 tx=0x7fcbb4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:40.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.796+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/1457929136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f018cb0 0x55fb6f019090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009fd0 tx=0x7fcbbc0118d0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.796+0000 7fcbcde5e740 1 -- 172.21.15.2:0/1457929136 shutdown_connections 2024-09-10T08:17:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.796+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/1457929136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f023e90 0x55fb6f026280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.796+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/1457929136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f023950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.796+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/1457929136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f018cb0 0x55fb6f019090 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.796+0000 7fcbcde5e740 1 -- 172.21.15.2:0/1457929136 >> 172.21.15.2:0/1457929136 conn(0x55fb6f015240 msgr2=0x55fb6f015620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.796+0000 7fcbcde5e740 1 -- 172.21.15.2:0/1457929136 shutdown_connections 2024-09-10T08:17:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.796+0000 7fcbcde5e740 1 -- 172.21.15.2:0/1457929136 wait complete. 2024-09-10T08:17:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.796+0000 7fcbcde5e740 1 -- start start 2024-09-10T08:17:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.797+0000 7fcbcde5e740 1 Processor -- start 2024-09-10T08:17:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.797+0000 7fcbcde5e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 0x55fb6f028040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.797+0000 7fcbcde5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f028580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.797+0000 7fcbcde5e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f023e90 0x55fb6f02c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.797+0000 7fcbcde5e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fb6f026b00 con 0x55fb6f018cb0 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.797+0000 7fcbcde5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fb6ef39fa0 con 0x55fb6f0195d0 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.797+0000 7fcbcde5e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fb6f026c80 con 0x55fb6f023e90 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.797+0000 7fcbcde5e740 10 client.0 _ll_register_callbacks cb 0x55fb6ec8dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.797+0000 7fcbc3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 0x55fb6f028040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.797+0000 7fcbc3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f028580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.797+0000 7fcbc3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 0x55fb6f028040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33470/0 (socket says 172.21.15.2:33470) 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.797+0000 7fcbc9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f023e90 0x55fb6f02c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.797+0000 7fcbc3e00640 1 -- 172.21.15.2:0/2648164900 learned_addr learned my addr 172.21.15.2:0/2648164900 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.798+0000 7fcbc3e00640 1 -- 172.21.15.2:0/2648164900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f023e90 msgr2=0x55fb6f02c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.798+0000 7fcbc3e00640 1 --2- 172.21.15.2:0/2648164900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f023e90 0x55fb6f02c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.798+0000 7fcbc3e00640 1 -- 172.21.15.2:0/2648164900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 msgr2=0x55fb6f028580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:40.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.798+0000 7fcbc3e00640 1 --2- 172.21.15.2:0/2648164900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f028580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.798+0000 7fcbc3e00640 1 -- 172.21.15.2:0/2648164900 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb6f02cd90 con 0x55fb6f018cb0 2024-09-10T08:17:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.798+0000 7fcbc3400640 1 --2- 172.21.15.2:0/2648164900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f028580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.798+0000 7fcbc3e00640 1 --2- 172.21.15.2:0/2648164900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 0x55fb6f028040 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcbb401d9c0 tx=0x7fcbb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148c0bfb1a9d3e48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.798+0000 7fcbae000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:17:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.798+0000 7fcbae000640 1 -- 172.21.15.2:0/2648164900 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb402d060 con 0x55fb6f018cb0 2024-09-10T08:17:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.799+0000 7fcbcde5e740 1 -- 172.21.15.2:0/2648164900 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbbc004050 con 0x55fb6f018cb0 2024-09-10T08:17:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.799+0000 7fcbae000640 1 -- 172.21.15.2:0/2648164900 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcbb4018030 con 0x55fb6f018cb0 2024-09-10T08:17:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.799+0000 7fcbae000640 1 -- 172.21.15.2:0/2648164900 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcbb4034050 con 0x55fb6f018cb0 2024-09-10T08:17:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.799+0000 7fcbcde5e740 10 client.18069 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.799+0000 7fcbcde5e740 1 -- 172.21.15.2:0/2648164900 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fb6f036e10 con 0x55fb6f018cb0 2024-09-10T08:17:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.799+0000 7fcbae000640 1 -- 172.21.15.2:0/2648164900 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbbc004050 con 0x55fb6f018cb0 2024-09-10T08:17:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.799+0000 7fcbae000640 1 -- 172.21.15.2:0/2648164900 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcbb401c430 con 0x55fb6f018cb0 2024-09-10T08:17:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.799+0000 7fcbcde5e740 10 client.18069 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.799+0000 7fcbcde5e740 10 client.18069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.799+0000 7fcbcde5e740 20 client.18069 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.800+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/2648164900 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb6f0382d0 0x55fb6f03a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.800+0000 7fcbcde5e740 4 client.18069 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:17:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.800+0000 7fcbcde5e740 1 -- 172.21.15.2:0/2648164900 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb6ef8c800 con 0x55fb6f0382d0 2024-09-10T08:17:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.800+0000 7fcbc3400640 1 --2- 172.21.15.2:0/2648164900 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb6f0382d0 0x55fb6f03a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.800+0000 7fcbc3400640 1 --2- 172.21.15.2:0/2648164900 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb6f0382d0 0x55fb6f03a720 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2159669f635bac77 server_cookie=9d60a6e5f35db2f in_seq=0 out_seq=0 2024-09-10T08:17:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.800+0000 7fcbae000640 10 client.18069 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.801+0000 7fcbae000640 1 -- 172.21.15.2:0/2648164900 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb6ef8c800 con 0x55fb6f0382d0 2024-09-10T08:17:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.801+0000 7fcbae000640 10 client.18069 handle_command_reply: tid=0 2024-09-10T08:17:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.801+0000 7fcbcde5e740 1 client.18069 shutdown 2024-09-10T08:17:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.801+0000 7fcbcde5e740 1 -- 172.21.15.2:0/2648164900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 msgr2=0x55fb6f028040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.801+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/2648164900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 0x55fb6f028040 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcbb401d9c0 tx=0x7fcbb4004730 comp rx=0 tx=0).stop 2024-09-10T08:17:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.801+0000 7fcbcde5e740 1 -- 172.21.15.2:0/2648164900 shutdown_connections 2024-09-10T08:17:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.801+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/2648164900 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb6f0382d0 0x55fb6f03a720 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.802+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/2648164900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f023e90 0x55fb6f02c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.802+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/2648164900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f028580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.802+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/2648164900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 0x55fb6f028040 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.802+0000 7fcbcde5e740 1 -- 172.21.15.2:0/2648164900 >> 172.21.15.2:0/2648164900 conn(0x55fb6f015240 msgr2=0x55fb6f01af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.802+0000 7fcbae000640 0 client.18069 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.802+0000 7fcbcde5e740 1 -- 172.21.15.2:0/2648164900 shutdown_connections 2024-09-10T08:17:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.802+0000 7fcbcde5e740 1 -- 172.21.15.2:0/2648164900 wait complete. 2024-09-10T08:17:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.802+0000 7fcbcde5e740 20 client.18069 trim_cache size 0 max 16384 2024-09-10T08:17:40.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.820+0000 7fcbcde5e740 1 Processor -- start 2024-09-10T08:17:40.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.820+0000 7fcbcde5e740 1 -- start start 2024-09-10T08:17:40.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.820+0000 7fcbcde5e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 0x55fb6f02c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.820+0000 7fcbcde5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f01ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbcde5e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f023e90 0x55fb6f03fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbcde5e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc41af2c0 con 0x55fb6f018cb0 2024-09-10T08:17:40.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbcde5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc41acb90 con 0x55fb6f0195d0 2024-09-10T08:17:40.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbcde5e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc41aa9e0 con 0x55fb6f023e90 2024-09-10T08:17:40.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbc3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 0x55fb6f02c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbc3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 0x55fb6f02c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33486/0 (socket says 172.21.15.2:33486) 2024-09-10T08:17:40.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbc3e00640 1 -- 172.21.15.2:0/2005802837 learned_addr learned my addr 172.21.15.2:0/2005802837 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:40.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbc3400640 1 --2- 172.21.15.2:0/2005802837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f01ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbc9200640 1 --2- 172.21.15.2:0/2005802837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f023e90 0x55fb6f03fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbc3e00640 1 -- 172.21.15.2:0/2005802837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f023e90 msgr2=0x55fb6f03fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:40.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbc3e00640 1 --2- 172.21.15.2:0/2005802837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f023e90 0x55fb6f03fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbc3e00640 1 -- 172.21.15.2:0/2005802837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 msgr2=0x55fb6f01ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:40.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbc3e00640 1 --2- 172.21.15.2:0/2005802837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f01ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbc3e00640 1 -- 172.21.15.2:0/2005802837 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb6f0193c0 con 0x55fb6f018cb0 2024-09-10T08:17:40.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.821+0000 7fcbc3400640 1 --2- 172.21.15.2:0/2005802837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f01ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:40.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.822+0000 7fcbc3e00640 1 --2- 172.21.15.2:0/2005802837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 0x55fb6f02c6e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4019040 tx=0x7fcbb4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0243c35558153a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:40.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.822+0000 7fcbb8c00640 1 -- 172.21.15.2:0/2005802837 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb403c070 con 0x55fb6f018cb0 2024-09-10T08:17:40.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.822+0000 7fcbb8c00640 1 -- 172.21.15.2:0/2005802837 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb4009d80 con 0x55fb6f018cb0 2024-09-10T08:17:40.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.822+0000 7fcbb8c00640 1 -- 172.21.15.2:0/2005802837 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4028bf0 con 0x55fb6f018cb0 2024-09-10T08:17:40.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.822+0000 7fcbcde5e740 1 -- 172.21.15.2:0/2005802837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 msgr2=0x55fb6f02c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:40.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.822+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/2005802837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 0x55fb6f02c6e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4019040 tx=0x7fcbb4000bc0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.823+0000 7fcbcde5e740 1 -- 172.21.15.2:0/2005802837 shutdown_connections 2024-09-10T08:17:40.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.823+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/2005802837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f023e90 0x55fb6f03fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.823+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/2005802837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f01ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.823+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/2005802837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f018cb0 0x55fb6f02c6e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.823+0000 7fcbcde5e740 1 -- 172.21.15.2:0/2005802837 >> 172.21.15.2:0/2005802837 conn(0x55fb6f015240 msgr2=0x55fb6f01a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:40.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.823+0000 7fcbcde5e740 1 -- 172.21.15.2:0/2005802837 shutdown_connections 2024-09-10T08:17:40.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.823+0000 7fcbcde5e740 1 -- 172.21.15.2:0/2005802837 wait complete. 2024-09-10T08:17:40.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.823+0000 7fcbcde5e740 1 -- start start 2024-09-10T08:17:40.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.823+0000 7fcbcde5e740 1 Processor -- start 2024-09-10T08:17:40.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.824+0000 7fcbcde5e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f018cb0 0x55fb6f02ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.824+0000 7fcbcde5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f0304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.824+0000 7fcbcde5e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f023e90 0x55fb6f030a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.824+0000 7fcbcde5e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc41ab050 con 0x55fb6f023e90 2024-09-10T08:17:40.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.824+0000 7fcbcde5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc41ac4c0 con 0x55fb6f0195d0 2024-09-10T08:17:40.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.824+0000 7fcbcde5e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc41ab720 con 0x55fb6f018cb0 2024-09-10T08:17:40.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.824+0000 7fcbcde5e740 10 client.0 _ll_register_callbacks cb 0x7fcbb4009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:40.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.824+0000 7fcbc9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f023e90 0x55fb6f030a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.824+0000 7fcbc3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f018cb0 0x55fb6f02ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.824+0000 7fcbc9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f023e90 0x55fb6f030a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33488/0 (socket says 172.21.15.2:33488) 2024-09-10T08:17:40.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.824+0000 7fcbc9200640 1 -- 172.21.15.2:0/3386617975 learned_addr learned my addr 172.21.15.2:0/3386617975 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:40.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.824+0000 7fcbc3400640 1 --2- 172.21.15.2:0/3386617975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f0304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.825+0000 7fcbc3e00640 1 -- 172.21.15.2:0/3386617975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 msgr2=0x55fb6f0304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:40.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.825+0000 7fcbc3e00640 1 --2- 172.21.15.2:0/3386617975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f0304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.825+0000 7fcbc3e00640 1 -- 172.21.15.2:0/3386617975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f023e90 msgr2=0x55fb6f030a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:40.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.825+0000 7fcbc3e00640 1 --2- 172.21.15.2:0/3386617975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f023e90 0x55fb6f030a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.825+0000 7fcbc3e00640 1 -- 172.21.15.2:0/3386617975 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb6f0a22b0 con 0x55fb6f018cb0 2024-09-10T08:17:40.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.825+0000 7fcbc9200640 1 --2- 172.21.15.2:0/3386617975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f023e90 0x55fb6f030a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:40.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.825+0000 7fcbc3e00640 1 --2- 172.21.15.2:0/3386617975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f018cb0 0x55fb6f02ff90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4012e20 tx=0x7fcbb4005d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54cd14077488fe40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:40.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.825+0000 7fcbae000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:17:40.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.825+0000 7fcbae000640 1 -- 172.21.15.2:0/3386617975 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb402d070 con 0x55fb6f018cb0 2024-09-10T08:17:40.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.825+0000 7fcbcde5e740 1 -- 172.21.15.2:0/3386617975 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb4009d80 con 0x55fb6f018cb0 2024-09-10T08:17:40.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.825+0000 7fcbae000640 1 -- 172.21.15.2:0/3386617975 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcbb401cc00 con 0x55fb6f018cb0 2024-09-10T08:17:40.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.826+0000 7fcbae000640 1 -- 172.21.15.2:0/3386617975 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcbb4052050 con 0x55fb6f018cb0 2024-09-10T08:17:40.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.826+0000 7fcbcde5e740 10 client.28039 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:40.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.826+0000 7fcbcde5e740 1 -- 172.21.15.2:0/3386617975 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fb6f0a35f0 con 0x55fb6f018cb0 2024-09-10T08:17:40.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.826+0000 7fcbae000640 1 -- 172.21.15.2:0/3386617975 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb4009d80 con 0x55fb6f018cb0 2024-09-10T08:17:40.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.826+0000 7fcbae000640 1 -- 172.21.15.2:0/3386617975 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcbb4005310 con 0x55fb6f018cb0 2024-09-10T08:17:40.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.826+0000 7fcbcde5e740 10 client.28039 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:40.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.826+0000 7fcbcde5e740 10 client.28039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:40.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.826+0000 7fcbcde5e740 20 client.28039 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:40.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.826+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/3386617975 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb6f0a3c00 0x55fb6f0a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:40.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.826+0000 7fcbcde5e740 4 client.28039 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:17:40.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.826+0000 7fcbcde5e740 1 -- 172.21.15.2:0/3386617975 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fb6ef86d10 con 0x55fb6f0a3c00 2024-09-10T08:17:40.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.827+0000 7fcbc3400640 1 --2- 172.21.15.2:0/3386617975 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb6f0a3c00 0x55fb6f0a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:40.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.827+0000 7fcbc3400640 1 --2- 172.21.15.2:0/3386617975 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb6f0a3c00 0x55fb6f0a5ff0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab2a372e14733a0f server_cookie=c239112f584d5c4c in_seq=0 out_seq=0 2024-09-10T08:17:40.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.827+0000 7fcbae000640 10 client.28039 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:40.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.828+0000 7fcbae000640 1 -- 172.21.15.2:0/3386617975 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fb6ef86d10 con 0x55fb6f0a3c00 2024-09-10T08:17:40.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.828+0000 7fcbae000640 10 client.28039 handle_command_reply: tid=0 2024-09-10T08:17:40.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.828+0000 7fcbcde5e740 1 client.28039 shutdown 2024-09-10T08:17:40.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.828+0000 7fcbcde5e740 1 -- 172.21.15.2:0/3386617975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f018cb0 msgr2=0x55fb6f02ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:40.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.828+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/3386617975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f018cb0 0x55fb6f02ff90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4012e20 tx=0x7fcbb4005d10 comp rx=0 tx=0).stop 2024-09-10T08:17:40.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.828+0000 7fcbcde5e740 1 -- 172.21.15.2:0/3386617975 shutdown_connections 2024-09-10T08:17:40.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.828+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/3386617975 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fb6f0a3c00 0x55fb6f0a5ff0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.828+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/3386617975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fb6f023e90 0x55fb6f030a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.828+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/3386617975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb6f0195d0 0x55fb6f0304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.828+0000 7fcbcde5e740 1 --2- 172.21.15.2:0/3386617975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb6f018cb0 0x55fb6f02ff90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.829+0000 7fcbae000640 0 client.28039 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:40.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.829+0000 7fcbcde5e740 1 -- 172.21.15.2:0/3386617975 >> 172.21.15.2:0/3386617975 conn(0x55fb6f015240 msgr2=0x55fb6f01af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:40.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.829+0000 7fcbcde5e740 1 -- 172.21.15.2:0/3386617975 shutdown_connections 2024-09-10T08:17:40.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.829+0000 7fcbcde5e740 1 -- 172.21.15.2:0/3386617975 wait complete. 2024-09-10T08:17:40.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.829+0000 7fcbcde5e740 20 client.28039 trim_cache size 0 max 16384 2024-09-10T08:17:40.833 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:17:40.833 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:17:40.833 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "a51b41bf-e7a4-4d7a-86ad-4a60755a35f6", 2024-09-10T08:17:40.833 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:17:40.833 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:17:40.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.829+0000 7fcbc9c00640 1 -- 172.21.15.2:0/2462996233 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcba80a2b80 msgr2=0x7fcba80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:40.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.829+0000 7fcbc9c00640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcba80a2b80 0x7fcba80a5040 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55fb6f0160a0 tx=0x7fcbb4009260 comp rx=0 tx=0).stop 2024-09-10T08:17:40.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.830+0000 7fcbc9c00640 1 -- 172.21.15.2:0/2462996233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc4102360 msgr2=0x7fcbc4198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:40.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.830+0000 7fcbc9c00640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc4102360 0x7fcbc4198800 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0000c00 tx=0x7fcbb0033d10 comp rx=0 tx=0).stop 2024-09-10T08:17:40.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.830+0000 7fcbc9c00640 1 -- 172.21.15.2:0/2462996233 shutdown_connections 2024-09-10T08:17:40.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.830+0000 7fcbc9c00640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbc41079c0 0x7fcbc419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.830+0000 7fcbc9c00640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc4102360 0x7fcbc4198800 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.830+0000 7fcbc9c00640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcba80a2b80 0x7fcba80a5040 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.830+0000 7fcbc9c00640 1 --2- 172.21.15.2:0/2462996233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbc41019c0 0x7fcbc41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:40.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.830+0000 7fcbc9c00640 1 -- 172.21.15.2:0/2462996233 >> 172.21.15.2:0/2462996233 conn(0x7fcbc40fd740 msgr2=0x7fcbc40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:40.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.830+0000 7fcbc9c00640 1 -- 172.21.15.2:0/2462996233 shutdown_connections 2024-09-10T08:17:40.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:40.830+0000 7fcbc9c00640 1 -- 172.21.15.2:0/2462996233 wait complete. 2024-09-10T08:17:41.674 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:17:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:41 smithi155 ceph-mon[35405]: pgmap v673: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-10T08:17:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:41 smithi107 ceph-mon[35048]: pgmap v673: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-10T08:17:42.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:41 smithi002 ceph-mon[30563]: pgmap v673: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-10T08:17:42.022 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:17:42.054 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:17:42.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:42 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:17:42.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:42 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:17:42.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:42 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:17:42.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:42 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:17:42.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:42 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:17:42.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.833+0000 7ff19b200640 1 -- 172.21.15.2:0/4149272596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c1129e0 msgr2=0x7ff19c114dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:42.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.833+0000 7ff19b200640 1 --2- 172.21.15.2:0/4149272596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c1129e0 0x7ff19c114dd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff194009be0 tx=0x7ff19402a750 comp rx=0 tx=0).stop 2024-09-10T08:17:42.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.833+0000 7ff19b200640 1 -- 172.21.15.2:0/4149272596 shutdown_connections 2024-09-10T08:17:42.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.833+0000 7ff19b200640 1 --2- 172.21.15.2:0/4149272596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c1129e0 0x7ff19c114dd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.833+0000 7ff19b200640 1 --2- 172.21.15.2:0/4149272596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff19c074eb0 0x7ff19c1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.833+0000 7ff19b200640 1 --2- 172.21.15.2:0/4149272596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff19c074590 0x7ff19c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.833+0000 7ff19b200640 1 -- 172.21.15.2:0/4149272596 >> 172.21.15.2:0/4149272596 conn(0x7ff19c0fd710 msgr2=0x7ff19c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:42.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.833+0000 7ff19b200640 1 -- 172.21.15.2:0/4149272596 shutdown_connections 2024-09-10T08:17:42.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.833+0000 7ff19b200640 1 -- 172.21.15.2:0/4149272596 wait complete. 2024-09-10T08:17:42.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.834+0000 7ff19b200640 1 Processor -- start 2024-09-10T08:17:42.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.835+0000 7ff19b200640 1 -- start start 2024-09-10T08:17:42.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.835+0000 7ff19b200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff19c074590 0x7ff19c1a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.835+0000 7ff19b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c074eb0 0x7ff19c1a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.835+0000 7ff19b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff19c1129e0 0x7ff19c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.835+0000 7ff19b200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c118350 con 0x7ff19c074590 2024-09-10T08:17:42.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.835+0000 7ff19b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1181d0 con 0x7ff19c074eb0 2024-09-10T08:17:42.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.835+0000 7ff19b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1184d0 con 0x7ff19c1129e0 2024-09-10T08:17:42.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.835+0000 7ff199e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff19c074590 0x7ff19c1a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.835+0000 7ff199e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff19c074590 0x7ff19c1a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33504/0 (socket says 172.21.15.2:33504) 2024-09-10T08:17:42.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.835+0000 7ff199400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c074eb0 0x7ff19c1a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.835+0000 7ff199e00640 1 -- 172.21.15.2:0/1521144425 learned_addr learned my addr 172.21.15.2:0/1521144425 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:42.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.836+0000 7ff19a800640 1 --2- 172.21.15.2:0/1521144425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff19c1129e0 0x7ff19c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.836+0000 7ff199400640 1 -- 172.21.15.2:0/1521144425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff19c1129e0 msgr2=0x7ff19c19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:42.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.836+0000 7ff199400640 1 --2- 172.21.15.2:0/1521144425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff19c1129e0 0x7ff19c19e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.836+0000 7ff199400640 1 -- 172.21.15.2:0/1521144425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff19c074590 msgr2=0x7ff19c1a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:42.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.836+0000 7ff199400640 1 --2- 172.21.15.2:0/1521144425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff19c074590 0x7ff19c1a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.836+0000 7ff199400640 1 -- 172.21.15.2:0/1521144425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c19ef30 con 0x7ff19c074eb0 2024-09-10T08:17:42.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.836+0000 7ff199400640 1 --2- 172.21.15.2:0/1521144425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c074eb0 0x7ff19c1a4b30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff18800e9f0 tx=0x7ff18800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:42.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.837+0000 7ff192a00640 1 -- 172.21.15.2:0/1521144425 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18800ccc0 con 0x7ff19c074eb0 2024-09-10T08:17:42.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.837+0000 7ff19b200640 1 -- 172.21.15.2:0/1521144425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19c19f220 con 0x7ff19c074eb0 2024-09-10T08:17:42.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.837+0000 7ff192a00640 1 -- 172.21.15.2:0/1521144425 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff188010910 con 0x7ff19c074eb0 2024-09-10T08:17:42.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.837+0000 7ff192a00640 1 -- 172.21.15.2:0/1521144425 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188018960 con 0x7ff19c074eb0 2024-09-10T08:17:42.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.838+0000 7ff19b200640 1 -- 172.21.15.2:0/1521144425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19c1ab3a0 con 0x7ff19c074eb0 2024-09-10T08:17:42.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.840+0000 7ff192a00640 1 -- 172.21.15.2:0/1521144425 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff188010ac0 con 0x7ff19c074eb0 2024-09-10T08:17:42.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.842+0000 7ff1a3672740 1 Processor -- start 2024-09-10T08:17:42.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.842+0000 7ff1a3672740 1 -- start start 2024-09-10T08:17:42.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff192a00640 1 --2- 172.21.15.2:0/1521144425 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff1800a2b80 0x7ff1800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff1a3672740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a9988f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff1a3672740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988f670 0x560a998999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff1a3672740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 0x560a9989c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff1a3672740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560a997f53e0 con 0x560a9988ed50 2024-09-10T08:17:42.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff1a3672740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560a9979cd80 con 0x560a99899f30 2024-09-10T08:17:42.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff1a3672740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560a997f13a0 con 0x560a9988f670 2024-09-10T08:17:42.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff192a00640 1 -- 172.21.15.2:0/1521144425 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff1880cf7b0 con 0x7ff19c074eb0 2024-09-10T08:17:42.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff199e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a9988f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff199e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a9988f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33506/0 (socket says 172.21.15.2:33506) 2024-09-10T08:17:42.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff199e00640 1 -- 172.21.15.2:0/321460772 learned_addr learned my addr 172.21.15.2:0/321460772 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:42.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff199400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988f670 0x560a998999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff19a800640 1 --2- 172.21.15.2:0/1521144425 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff1800a2b80 0x7ff1800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.843+0000 7ff19a800640 1 --2- 172.21.15.2:0/321460772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 0x560a9989c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.844+0000 7ff199400640 1 -- 172.21.15.2:0/321460772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 msgr2=0x560a9989c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:42.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.844+0000 7ff199400640 1 --2- 172.21.15.2:0/321460772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 0x560a9989c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.844+0000 7ff199400640 1 -- 172.21.15.2:0/321460772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 msgr2=0x560a9988f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:42.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.844+0000 7ff199400640 1 --2- 172.21.15.2:0/321460772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a9988f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.844+0000 7ff199400640 1 -- 172.21.15.2:0/321460772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a997bbae0 con 0x560a9988f670 2024-09-10T08:17:42.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.844+0000 7ff199400640 1 --2- 172.21.15.2:0/321460772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988f670 0x560a998999f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff18809b970 tx=0x7ff18809b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79405d49fc599a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:42.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.845+0000 7ff19a800640 1 --2- 172.21.15.2:0/1521144425 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff1800a2b80 0x7ff1800a5040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x560a9988c660 tx=0x7ff194005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:42.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.844+0000 7ff187000640 1 -- 172.21.15.2:0/321460772 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188099d10 con 0x560a9988f670 2024-09-10T08:17:42.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.845+0000 7ff187000640 1 -- 172.21.15.2:0/321460772 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1880b2020 con 0x560a9988f670 2024-09-10T08:17:42.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.845+0000 7ff187000640 1 -- 172.21.15.2:0/321460772 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18809b0c0 con 0x560a9988f670 2024-09-10T08:17:42.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.845+0000 7ff1a3672740 1 -- 172.21.15.2:0/321460772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988f670 msgr2=0x560a998999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:42.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.845+0000 7ff1a3672740 1 --2- 172.21.15.2:0/321460772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988f670 0x560a998999f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff18809b970 tx=0x7ff18809b690 comp rx=0 tx=0).stop 2024-09-10T08:17:42.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.846+0000 7ff1a3672740 1 -- 172.21.15.2:0/321460772 shutdown_connections 2024-09-10T08:17:42.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.846+0000 7ff1a3672740 1 --2- 172.21.15.2:0/321460772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 0x560a9989c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.846+0000 7ff1a3672740 1 --2- 172.21.15.2:0/321460772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988f670 0x560a998999f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.846+0000 7ff1a3672740 1 --2- 172.21.15.2:0/321460772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a9988f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.846+0000 7ff1a3672740 1 -- 172.21.15.2:0/321460772 >> 172.21.15.2:0/321460772 conn(0x560a9988b2e0 msgr2=0x560a9988b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:42.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.846+0000 7ff1a3672740 1 -- 172.21.15.2:0/321460772 shutdown_connections 2024-09-10T08:17:42.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.846+0000 7ff1a3672740 1 -- 172.21.15.2:0/321460772 wait complete. 2024-09-10T08:17:42.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.846+0000 7ff1a3672740 1 -- start start 2024-09-10T08:17:42.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.847+0000 7ff1a3672740 1 Processor -- start 2024-09-10T08:17:42.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.847+0000 7ff1a3672740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a99896e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.847+0000 7ff1a3672740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988f670 0x560a99897380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff1a3672740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 0x560a998978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff1a3672740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560a9989cba0 con 0x560a9988ed50 2024-09-10T08:17:42.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff1a3672740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560a997b00f0 con 0x560a99899f30 2024-09-10T08:17:42.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff1a3672740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560a9989cd20 con 0x560a9988f670 2024-09-10T08:17:42.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff1a3672740 10 client.0 _ll_register_callbacks cb 0x560a997a19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:42.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff199400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988f670 0x560a99897380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff199400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988f670 0x560a99897380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36408/0 (socket says 172.21.15.2:36408) 2024-09-10T08:17:42.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff199400640 1 -- 172.21.15.2:0/3530615365 learned_addr learned my addr 172.21.15.2:0/3530615365 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:42.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff199e00640 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a99896e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff19a800640 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 0x560a998978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff199400640 1 -- 172.21.15.2:0/3530615365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 msgr2=0x560a998978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:42.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff199400640 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 0x560a998978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff199400640 1 -- 172.21.15.2:0/3530615365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 msgr2=0x560a99896e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:42.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff199400640 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a99896e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff199400640 1 -- 172.21.15.2:0/3530615365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a995663c0 con 0x560a9988f670 2024-09-10T08:17:42.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff19a800640 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 0x560a998978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:42.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.848+0000 7ff199e00640 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a99896e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:42.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.849+0000 7ff199400640 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988f670 0x560a99897380 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff18800c6a0 tx=0x7ff18809b520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32139b6f023571c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:42.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.849+0000 7ff17b400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:17:42.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.849+0000 7ff17b400640 1 -- 172.21.15.2:0/3530615365 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1880982e0 con 0x560a9988f670 2024-09-10T08:17:42.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.849+0000 7ff17b400640 1 -- 172.21.15.2:0/3530615365 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff188010430 con 0x560a9988f670 2024-09-10T08:17:42.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.849+0000 7ff1a3672740 1 -- 172.21.15.2:0/3530615365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1880b2020 con 0x560a9988f670 2024-09-10T08:17:42.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.849+0000 7ff17b400640 1 -- 172.21.15.2:0/3530615365 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff188082050 con 0x560a9988f670 2024-09-10T08:17:42.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.849+0000 7ff1a3672740 10 client.28051 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:42.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.849+0000 7ff1a3672740 1 -- 172.21.15.2:0/3530615365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a9989cba0 con 0x560a9988f670 2024-09-10T08:17:42.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.850+0000 7ff17b400640 1 -- 172.21.15.2:0/3530615365 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1880b2020 con 0x560a9988f670 2024-09-10T08:17:42.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.850+0000 7ff17b400640 1 -- 172.21.15.2:0/3530615365 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff18807cba0 con 0x560a9988f670 2024-09-10T08:17:42.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.850+0000 7ff1a3672740 10 client.28051 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:42.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.850+0000 7ff1a3672740 10 client.28051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:42.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.850+0000 7ff1a3672740 20 client.28051 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:42.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.850+0000 7ff1a3672740 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560a998a9a00 0x560a998abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.850+0000 7ff1a3672740 4 client.28051 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:17:42.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.850+0000 7ff1a3672740 1 -- 172.21.15.2:0/3530615365 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a99802950 con 0x560a998a9a00 2024-09-10T08:17:42.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.851+0000 7ff199e00640 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560a998a9a00 0x560a998abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.851+0000 7ff199e00640 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560a998a9a00 0x560a998abdf0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53635a8af20ea950 server_cookie=18dc05d7c6979397 in_seq=0 out_seq=0 2024-09-10T08:17:42.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.851+0000 7ff17b400640 10 client.28051 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:42.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.852+0000 7ff17b400640 1 -- 172.21.15.2:0/3530615365 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a99802950 con 0x560a998a9a00 2024-09-10T08:17:42.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.852+0000 7ff17b400640 10 client.28051 handle_command_reply: tid=0 2024-09-10T08:17:42.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.852+0000 7ff1a3672740 1 client.28051 shutdown 2024-09-10T08:17:42.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.852+0000 7ff1a3672740 1 -- 172.21.15.2:0/3530615365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988f670 msgr2=0x560a99897380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:42.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.852+0000 7ff1a3672740 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988f670 0x560a99897380 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff18800c6a0 tx=0x7ff18809b520 comp rx=0 tx=0).stop 2024-09-10T08:17:42.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.852+0000 7ff1a3672740 1 -- 172.21.15.2:0/3530615365 shutdown_connections 2024-09-10T08:17:42.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.853+0000 7ff1a3672740 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560a998a9a00 0x560a998abdf0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.853+0000 7ff1a3672740 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 0x560a998978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.853+0000 7ff1a3672740 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988f670 0x560a99897380 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.853+0000 7ff1a3672740 1 --2- 172.21.15.2:0/3530615365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a99896e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.853+0000 7ff17b400640 0 client.28051 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:42.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.853+0000 7ff1a3672740 1 -- 172.21.15.2:0/3530615365 >> 172.21.15.2:0/3530615365 conn(0x560a9988b2e0 msgr2=0x560a99891010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:42.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.853+0000 7ff1a3672740 1 -- 172.21.15.2:0/3530615365 shutdown_connections 2024-09-10T08:17:42.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.853+0000 7ff1a3672740 1 -- 172.21.15.2:0/3530615365 wait complete. 2024-09-10T08:17:42.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.853+0000 7ff1a3672740 20 client.28051 trim_cache size 0 max 16384 2024-09-10T08:17:42.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.888+0000 7ff1a3672740 1 Processor -- start 2024-09-10T08:17:42.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.888+0000 7ff1a3672740 1 -- start start 2024-09-10T08:17:42.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.889+0000 7ff1a3672740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a99897710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.889+0000 7ff1a3672740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a9988f670 0x560a998b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.889+0000 7ff1a3672740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a99899f30 0x560a998b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.889+0000 7ff1a3672740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1b7c20 con 0x560a9988ed50 2024-09-10T08:17:42.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.889+0000 7ff1a3672740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1b54f0 con 0x560a9988f670 2024-09-10T08:17:42.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.889+0000 7ff1a3672740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1b3340 con 0x560a99899f30 2024-09-10T08:17:42.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.889+0000 7ff199400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a9988f670 0x560a998b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.889+0000 7ff199400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a9988f670 0x560a998b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42458/0 (socket says 172.21.15.2:42458) 2024-09-10T08:17:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.889+0000 7ff199400640 1 -- 172.21.15.2:0/4045704377 learned_addr learned my addr 172.21.15.2:0/4045704377 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.889+0000 7ff19a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a99899f30 0x560a998b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.890+0000 7ff199400640 1 -- 172.21.15.2:0/4045704377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a99899f30 msgr2=0x560a998b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.890+0000 7ff199400640 1 --2- 172.21.15.2:0/4045704377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a99899f30 0x560a998b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.890+0000 7ff199400640 1 -- 172.21.15.2:0/4045704377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 msgr2=0x560a99897710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:17:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.890+0000 7ff199e00640 1 --2- 172.21.15.2:0/4045704377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a99897710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.890+0000 7ff199400640 1 --2- 172.21.15.2:0/4045704377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a99897710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.890+0000 7ff199400640 1 -- 172.21.15.2:0/4045704377 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a998997b0 con 0x560a9988f670 2024-09-10T08:17:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.890+0000 7ff199e00640 1 --2- 172.21.15.2:0/4045704377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a99897710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:17:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.890+0000 7ff199400640 1 --2- 172.21.15.2:0/4045704377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a9988f670 0x560a998b1510 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff18807cf40 tx=0x7ff18800cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4145c80113859cd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.891+0000 7ff186600640 1 -- 172.21.15.2:0/4045704377 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18809b7b0 con 0x560a9988f670 2024-09-10T08:17:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.891+0000 7ff186600640 1 -- 172.21.15.2:0/4045704377 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1880b9070 con 0x560a9988f670 2024-09-10T08:17:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.891+0000 7ff186600640 1 -- 172.21.15.2:0/4045704377 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188018b00 con 0x560a9988f670 2024-09-10T08:17:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.891+0000 7ff1a3672740 1 -- 172.21.15.2:0/4045704377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a9988f670 msgr2=0x560a998b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.891+0000 7ff1a3672740 1 --2- 172.21.15.2:0/4045704377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a9988f670 0x560a998b1510 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff18807cf40 tx=0x7ff18800cf60 comp rx=0 tx=0).stop 2024-09-10T08:17:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.892+0000 7ff1a3672740 1 -- 172.21.15.2:0/4045704377 shutdown_connections 2024-09-10T08:17:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.892+0000 7ff1a3672740 1 --2- 172.21.15.2:0/4045704377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a99899f30 0x560a998b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.892+0000 7ff1a3672740 1 --2- 172.21.15.2:0/4045704377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a9988f670 0x560a998b1510 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.892+0000 7ff1a3672740 1 --2- 172.21.15.2:0/4045704377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988ed50 0x560a99897710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.892+0000 7ff1a3672740 1 -- 172.21.15.2:0/4045704377 >> 172.21.15.2:0/4045704377 conn(0x560a9988b2e0 msgr2=0x560a99890fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.892+0000 7ff1a3672740 1 -- 172.21.15.2:0/4045704377 shutdown_connections 2024-09-10T08:17:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.892+0000 7ff1a3672740 1 -- 172.21.15.2:0/4045704377 wait complete. 2024-09-10T08:17:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.892+0000 7ff1a3672740 1 -- start start 2024-09-10T08:17:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.892+0000 7ff1a3672740 1 Processor -- start 2024-09-10T08:17:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.893+0000 7ff1a3672740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988ed50 0x560a998a2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.893+0000 7ff1a3672740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988f670 0x560a9989db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.893+0000 7ff1a3672740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 0x560a9989e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.893+0000 7ff1a3672740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1b39b0 con 0x560a9988f670 2024-09-10T08:17:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.893+0000 7ff1a3672740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1b4e20 con 0x560a99899f30 2024-09-10T08:17:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.893+0000 7ff1a3672740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1b4080 con 0x560a9988ed50 2024-09-10T08:17:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.893+0000 7ff1a3672740 10 client.0 _ll_register_callbacks cb 0x7ff18809b430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.893+0000 7ff199e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988ed50 0x560a998a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.893+0000 7ff199e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988ed50 0x560a998a2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36422/0 (socket says 172.21.15.2:36422) 2024-09-10T08:17:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.893+0000 7ff199e00640 1 -- 172.21.15.2:0/2671681400 learned_addr learned my addr 172.21.15.2:0/2671681400 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.893+0000 7ff199400640 1 --2- 172.21.15.2:0/2671681400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988f670 0x560a9989db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.893+0000 7ff19a800640 1 --2- 172.21.15.2:0/2671681400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 0x560a9989e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:17:42.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.894+0000 7ff199e00640 1 -- 172.21.15.2:0/2671681400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 msgr2=0x560a9989e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:42.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.894+0000 7ff199e00640 1 --2- 172.21.15.2:0/2671681400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 0x560a9989e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.894+0000 7ff199e00640 1 -- 172.21.15.2:0/2671681400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988f670 msgr2=0x560a9989db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:42.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.894+0000 7ff199e00640 1 --2- 172.21.15.2:0/2671681400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988f670 0x560a9989db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.894+0000 7ff199e00640 1 -- 172.21.15.2:0/2671681400 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a9989e970 con 0x560a9988ed50 2024-09-10T08:17:42.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.894+0000 7ff199e00640 1 --2- 172.21.15.2:0/2671681400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988ed50 0x560a998a2800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x560a998ac4b0 tx=0x7ff18c004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e4b0323a44bbbba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:42.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.894+0000 7ff17b400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:17:42.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.894+0000 7ff17b400640 1 -- 172.21.15.2:0/2671681400 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c026640 con 0x560a9988ed50 2024-09-10T08:17:42.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.894+0000 7ff1a3672740 1 -- 172.21.15.2:0/2671681400 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1880b9070 con 0x560a9988ed50 2024-09-10T08:17:42.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.894+0000 7ff17b400640 1 -- 172.21.15.2:0/2671681400 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff18c010430 con 0x560a9988ed50 2024-09-10T08:17:42.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.895+0000 7ff17b400640 1 -- 172.21.15.2:0/2671681400 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff18c0267e0 con 0x560a9988ed50 2024-09-10T08:17:42.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.895+0000 7ff1a3672740 10 client.28057 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:42.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.895+0000 7ff1a3672740 1 -- 172.21.15.2:0/2671681400 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a99914640 con 0x560a9988ed50 2024-09-10T08:17:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.895+0000 7ff17b400640 1 -- 172.21.15.2:0/2671681400 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1880b9070 con 0x560a9988ed50 2024-09-10T08:17:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.895+0000 7ff17b400640 1 -- 172.21.15.2:0/2671681400 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff18c024070 con 0x560a9988ed50 2024-09-10T08:17:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.895+0000 7ff1a3672740 10 client.28057 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.895+0000 7ff1a3672740 10 client.28057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.895+0000 7ff1a3672740 20 client.28057 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.895+0000 7ff1a3672740 1 --2- 172.21.15.2:0/2671681400 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560a99914d90 0x560a999171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.895+0000 7ff1a3672740 4 client.28057 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:17:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.895+0000 7ff1a3672740 1 -- 172.21.15.2:0/2671681400 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a997fce60 con 0x560a99914d90 2024-09-10T08:17:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.896+0000 7ff199400640 1 --2- 172.21.15.2:0/2671681400 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560a99914d90 0x560a999171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.896+0000 7ff17b400640 10 client.28057 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.896+0000 7ff199400640 1 --2- 172.21.15.2:0/2671681400 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560a99914d90 0x560a999171b0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10433c4c1b4aa44d server_cookie=d6fa9b0261c6833a in_seq=0 out_seq=0 2024-09-10T08:17:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.897+0000 7ff17b400640 1 -- 172.21.15.2:0/2671681400 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560a997fce60 con 0x560a99914d90 2024-09-10T08:17:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.897+0000 7ff17b400640 10 client.28057 handle_command_reply: tid=0 2024-09-10T08:17:42.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.897+0000 7ff1a3672740 1 client.28057 shutdown 2024-09-10T08:17:42.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.897+0000 7ff1a3672740 1 -- 172.21.15.2:0/2671681400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988ed50 msgr2=0x560a998a2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:42.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.897+0000 7ff1a3672740 1 --2- 172.21.15.2:0/2671681400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988ed50 0x560a998a2800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x560a998ac4b0 tx=0x7ff18c004570 comp rx=0 tx=0).stop 2024-09-10T08:17:42.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.897+0000 7ff1a3672740 1 -- 172.21.15.2:0/2671681400 shutdown_connections 2024-09-10T08:17:42.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.897+0000 7ff1a3672740 1 --2- 172.21.15.2:0/2671681400 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560a99914d90 0x560a999171b0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.897+0000 7ff1a3672740 1 --2- 172.21.15.2:0/2671681400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a99899f30 0x560a9989e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.897+0000 7ff1a3672740 1 --2- 172.21.15.2:0/2671681400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560a9988f670 0x560a9989db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.897+0000 7ff1a3672740 1 --2- 172.21.15.2:0/2671681400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9988ed50 0x560a998a2800 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.897+0000 7ff17b400640 0 client.28057 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:42.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.897+0000 7ff1a3672740 1 -- 172.21.15.2:0/2671681400 >> 172.21.15.2:0/2671681400 conn(0x560a9988b2e0 msgr2=0x560a99889b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:42.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.897+0000 7ff1a3672740 1 -- 172.21.15.2:0/2671681400 shutdown_connections 2024-09-10T08:17:42.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.898+0000 7ff1a3672740 1 -- 172.21.15.2:0/2671681400 wait complete. 2024-09-10T08:17:42.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.898+0000 7ff1a3672740 20 client.28057 trim_cache size 0 max 16384 2024-09-10T08:17:42.900 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:17:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:17:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:17:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "a51b41bf-e7a4-4d7a-86ad-4a60755a35f6": { 2024-09-10T08:17:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:17:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "a51b41bf-e7a4-4d7a-86ad-4a60755a35f6", 2024-09-10T08:17:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:17:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:17:42.901 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:17:42.901 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:17:42.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.898+0000 7ff19b200640 1 -- 172.21.15.2:0/1521144425 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff1800a2b80 msgr2=0x7ff1800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:42.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.898+0000 7ff19b200640 1 --2- 172.21.15.2:0/1521144425 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff1800a2b80 0x7ff1800a5040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x560a9988c660 tx=0x7ff194005e30 comp rx=0 tx=0).stop 2024-09-10T08:17:42.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.898+0000 7ff19b200640 1 -- 172.21.15.2:0/1521144425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c074eb0 msgr2=0x7ff19c1a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:42.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.898+0000 7ff19b200640 1 --2- 172.21.15.2:0/1521144425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c074eb0 0x7ff19c1a4b30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff18800e9f0 tx=0x7ff18800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.899+0000 7ff19b200640 1 -- 172.21.15.2:0/1521144425 shutdown_connections 2024-09-10T08:17:42.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.899+0000 7ff19b200640 1 --2- 172.21.15.2:0/1521144425 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff1800a2b80 0x7ff1800a5040 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.899+0000 7ff19b200640 1 --2- 172.21.15.2:0/1521144425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff19c1129e0 0x7ff19c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.899+0000 7ff19b200640 1 --2- 172.21.15.2:0/1521144425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c074eb0 0x7ff19c1a4b30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.899+0000 7ff19b200640 1 --2- 172.21.15.2:0/1521144425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff19c074590 0x7ff19c1a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:42.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.899+0000 7ff19b200640 1 -- 172.21.15.2:0/1521144425 >> 172.21.15.2:0/1521144425 conn(0x7ff19c0fd710 msgr2=0x7ff19c0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:42.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.899+0000 7ff19b200640 1 -- 172.21.15.2:0/1521144425 shutdown_connections 2024-09-10T08:17:42.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:42.899+0000 7ff19b200640 1 -- 172.21.15.2:0/1521144425 wait complete. 2024-09-10T08:17:42.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:42 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:17:42.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:42 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:17:42.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:42 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:17:42.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:42 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:17:42.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:42 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:17:42.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:42 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:17:42.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:42 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:17:42.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:42 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:17:42.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:42 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:17:42.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:42 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:17:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.408+0000 7fe8d9400640 1 -- 172.21.15.2:0/3623971314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4076bb0 msgr2=0x7fe8d4076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.408+0000 7fe8d9400640 1 --2- 172.21.15.2:0/3623971314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4076bb0 0x7fe8d4076fb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009900 tx=0x7fe8c402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.408+0000 7fe8d9400640 1 -- 172.21.15.2:0/3623971314 shutdown_connections 2024-09-10T08:17:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.408+0000 7fe8d9400640 1 --2- 172.21.15.2:0/3623971314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8d4078790 0x7fe8d4079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.408+0000 7fe8d9400640 1 --2- 172.21.15.2:0/3623971314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d4077df0 0x7fe8d4078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.408+0000 7fe8d9400640 1 --2- 172.21.15.2:0/3623971314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4076bb0 0x7fe8d4076fb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.408+0000 7fe8d9400640 1 -- 172.21.15.2:0/3623971314 >> 172.21.15.2:0/3623971314 conn(0x7fe8d40ff4e0 msgr2=0x7fe8d4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.408+0000 7fe8d9400640 1 -- 172.21.15.2:0/3623971314 shutdown_connections 2024-09-10T08:17:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.408+0000 7fe8d9400640 1 -- 172.21.15.2:0/3623971314 wait complete. 2024-09-10T08:17:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.409+0000 7fe8d9400640 1 Processor -- start 2024-09-10T08:17:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.409+0000 7fe8d9400640 1 -- start start 2024-09-10T08:17:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.409+0000 7fe8d9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4076bb0 0x7fe8d419e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.409+0000 7fe8d9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8d4077df0 0x7fe8d419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.409+0000 7fe8d9400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d4078790 0x7fe8d41a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.409+0000 7fe8d9400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4113e40 con 0x7fe8d4078790 2024-09-10T08:17:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.409+0000 7fe8d9400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4113cc0 con 0x7fe8d4076bb0 2024-09-10T08:17:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.409+0000 7fe8d9400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4113fc0 con 0x7fe8d4077df0 2024-09-10T08:17:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.410+0000 7fe8d2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4076bb0 0x7fe8d419e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.410+0000 7fe8d2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4076bb0 0x7fe8d419e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42480/0 (socket says 172.21.15.2:42480) 2024-09-10T08:17:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.410+0000 7fe8d2800640 1 -- 172.21.15.2:0/13847193 learned_addr learned my addr 172.21.15.2:0/13847193 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.410+0000 7fe8d3200640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d4078790 0x7fe8d41a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.410+0000 7fe8d1e00640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8d4077df0 0x7fe8d419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.410+0000 7fe8d2800640 1 -- 172.21.15.2:0/13847193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8d4077df0 msgr2=0x7fe8d419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.410+0000 7fe8d2800640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8d4077df0 0x7fe8d419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.410+0000 7fe8d2800640 1 -- 172.21.15.2:0/13847193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d4078790 msgr2=0x7fe8d41a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.410+0000 7fe8d2800640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d4078790 0x7fe8d41a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.410+0000 7fe8d2800640 1 -- 172.21.15.2:0/13847193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d41a5ff0 con 0x7fe8d4076bb0 2024-09-10T08:17:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.410+0000 7fe8d3200640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d4078790 0x7fe8d41a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.410+0000 7fe8d2800640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4076bb0 0x7fe8d419e3d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4031440 tx=0x7fe8c4032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.411+0000 7fe8cb400640 1 -- 172.21.15.2:0/13847193 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c403f030 con 0x7fe8d4076bb0 2024-09-10T08:17:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.411+0000 7fe8cb400640 1 -- 172.21.15.2:0/13847193 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8c4003ce0 con 0x7fe8d4076bb0 2024-09-10T08:17:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.411+0000 7fe8cb400640 1 -- 172.21.15.2:0/13847193 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4035070 con 0x7fe8d4076bb0 2024-09-10T08:17:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.411+0000 7fe8d9400640 1 -- 172.21.15.2:0/13847193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d41a6280 con 0x7fe8d4076bb0 2024-09-10T08:17:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.411+0000 7fe8d9400640 1 -- 172.21.15.2:0/13847193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d41a67c0 con 0x7fe8d4076bb0 2024-09-10T08:17:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.414+0000 7fe8dc43a740 1 Processor -- start 2024-09-10T08:17:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.414+0000 7fe8cb400640 1 -- 172.21.15.2:0/13847193 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe8c4042030 con 0x7fe8d4076bb0 2024-09-10T08:17:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.414+0000 7fe8dc43a740 1 -- start start 2024-09-10T08:17:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.414+0000 7fe8dc43a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a529090 0x55d92a529490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.414+0000 7fe8dc43a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a5299d0 0x55d92a52e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.414+0000 7fe8dc43a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 0x55d92a530f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.414+0000 7fe8dc43a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d92a48e100 con 0x55d92a52ea60 2024-09-10T08:17:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.414+0000 7fe8dc43a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d92a4361b0 con 0x55d92a529090 2024-09-10T08:17:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.414+0000 7fe8dc43a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d92a48a0c0 con 0x55d92a5299d0 2024-09-10T08:17:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.414+0000 7fe8d3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a529090 0x55d92a529490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.414+0000 7fe8d3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a529090 0x55d92a529490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42490/0 (socket says 172.21.15.2:42490) 2024-09-10T08:17:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.414+0000 7fe8d3200640 1 -- 172.21.15.2:0/863611345 learned_addr learned my addr 172.21.15.2:0/863611345 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.415+0000 7fe8d2800640 1 --2- 172.21.15.2:0/863611345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a5299d0 0x55d92a52e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.415+0000 7fe8d1e00640 1 --2- 172.21.15.2:0/863611345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 0x55d92a530f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.415+0000 7fe8d3200640 1 -- 172.21.15.2:0/863611345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a5299d0 msgr2=0x55d92a52e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.415+0000 7fe8d3200640 1 --2- 172.21.15.2:0/863611345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a5299d0 0x55d92a52e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.415+0000 7fe8d3200640 1 -- 172.21.15.2:0/863611345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 msgr2=0x55d92a530f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.415+0000 7fe8d3200640 1 --2- 172.21.15.2:0/863611345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 0x55d92a530f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.415+0000 7fe8d3200640 1 -- 172.21.15.2:0/863611345 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d92a1a1210 con 0x55d92a529090 2024-09-10T08:17:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.415+0000 7fe8d2800640 1 --2- 172.21.15.2:0/863611345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a5299d0 0x55d92a52e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.415+0000 7fe8d3200640 1 --2- 172.21.15.2:0/863611345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a529090 0x55d92a529490 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0117e0 tx=0x7fe8cc011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a471607f8d29d939 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.415+0000 7fe8cb400640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe8b80a2cb0 0x7fe8b80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.415+0000 7fe8bf400640 1 -- 172.21.15.2:0/863611345 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc011310 con 0x55d92a529090 2024-09-10T08:17:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.415+0000 7fe8bf400640 1 -- 172.21.15.2:0/863611345 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8cc003e50 con 0x55d92a529090 2024-09-10T08:17:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.415+0000 7fe8bf400640 1 -- 172.21.15.2:0/863611345 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0045e0 con 0x55d92a529090 2024-09-10T08:17:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.416+0000 7fe8cb400640 1 -- 172.21.15.2:0/13847193 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe8c40ef860 con 0x7fe8d4076bb0 2024-09-10T08:17:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.416+0000 7fe8d2800640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe8b80a2cb0 0x7fe8b80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.416+0000 7fe8dc43a740 1 -- 172.21.15.2:0/863611345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a529090 msgr2=0x55d92a529490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.416+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/863611345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a529090 0x55d92a529490 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0117e0 tx=0x7fe8cc011ca0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.416+0000 7fe8dc43a740 1 -- 172.21.15.2:0/863611345 shutdown_connections 2024-09-10T08:17:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.416+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/863611345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 0x55d92a530f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.416+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/863611345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a5299d0 0x55d92a52e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.416+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/863611345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a529090 0x55d92a529490 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.416+0000 7fe8dc43a740 1 -- 172.21.15.2:0/863611345 >> 172.21.15.2:0/863611345 conn(0x55d92a524700 msgr2=0x55d92a524b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.416+0000 7fe8d2800640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe8b80a2cb0 0x7fe8b80a5170 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4006010 tx=0x7fe8c402a3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.416+0000 7fe8dc43a740 1 -- 172.21.15.2:0/863611345 shutdown_connections 2024-09-10T08:17:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.416+0000 7fe8dc43a740 1 -- 172.21.15.2:0/863611345 wait complete. 2024-09-10T08:17:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.417+0000 7fe8dc43a740 1 -- start start 2024-09-10T08:17:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.417+0000 7fe8dc43a740 1 Processor -- start 2024-09-10T08:17:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.417+0000 7fe8dc43a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a529090 0x55d92a533240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.417+0000 7fe8dc43a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a5299d0 0x55d92a533780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.417+0000 7fe8dc43a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a52ea60 0x55d92a53a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.417+0000 7fe8dc43a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d92a531a30 con 0x55d92a529090 2024-09-10T08:17:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.417+0000 7fe8dc43a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d92a417090 con 0x55d92a52ea60 2024-09-10T08:17:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.417+0000 7fe8dc43a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d92a531bb0 con 0x55d92a5299d0 2024-09-10T08:17:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.417+0000 7fe8dc43a740 10 client.0 _ll_register_callbacks cb 0x55d92a46bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.418+0000 7fe8d1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a5299d0 0x55d92a533780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.418+0000 7fe8d3200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a529090 0x55d92a533240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.418+0000 7fe8d1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a5299d0 0x55d92a533780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36456/0 (socket says 172.21.15.2:36456) 2024-09-10T08:17:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.418+0000 7fe8d1e00640 1 -- 172.21.15.2:0/169222475 learned_addr learned my addr 172.21.15.2:0/169222475 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.418+0000 7fe8d3200640 1 --2- 172.21.15.2:0/169222475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a52ea60 0x55d92a53a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.418+0000 7fe8d1e00640 1 -- 172.21.15.2:0/169222475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a52ea60 msgr2=0x55d92a53a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.418+0000 7fe8d1e00640 1 --2- 172.21.15.2:0/169222475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a52ea60 0x55d92a53a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.418+0000 7fe8d1e00640 1 -- 172.21.15.2:0/169222475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a529090 msgr2=0x55d92a533240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.418+0000 7fe8d1e00640 1 --2- 172.21.15.2:0/169222475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a529090 0x55d92a533240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.418+0000 7fe8d1e00640 1 -- 172.21.15.2:0/169222475 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d92a26d620 con 0x55d92a5299d0 2024-09-10T08:17:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.418+0000 7fe8d3200640 1 --2- 172.21.15.2:0/169222475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a52ea60 0x55d92a53a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.418+0000 7fe8d1e00640 1 --2- 172.21.15.2:0/169222475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a5299d0 0x55d92a533780 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0012700 tx=0x7fe8c0012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=930655243d68ac68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.418+0000 7fe8b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:17:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.419+0000 7fe8b3e00640 1 -- 172.21.15.2:0/169222475 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0004050 con 0x55d92a5299d0 2024-09-10T08:17:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.419+0000 7fe8dc43a740 1 -- 172.21.15.2:0/169222475 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8cc003e50 con 0x55d92a5299d0 2024-09-10T08:17:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.419+0000 7fe8b3e00640 1 -- 172.21.15.2:0/169222475 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe8c00041f0 con 0x55d92a5299d0 2024-09-10T08:17:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.419+0000 7fe8b3e00640 1 -- 172.21.15.2:0/169222475 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe8c0025050 con 0x55d92a5299d0 2024-09-10T08:17:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.419+0000 7fe8dc43a740 10 client.28069 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.419+0000 7fe8dc43a740 1 -- 172.21.15.2:0/169222475 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d92a53d6d0 con 0x55d92a5299d0 2024-09-10T08:17:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.419+0000 7fe8b3e00640 1 -- 172.21.15.2:0/169222475 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8cc003e50 con 0x55d92a5299d0 2024-09-10T08:17:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.419+0000 7fe8b3e00640 1 -- 172.21.15.2:0/169222475 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe8c0003a40 con 0x55d92a5299d0 2024-09-10T08:17:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.420+0000 7fe8dc43a740 10 client.28069 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.420+0000 7fe8dc43a740 10 client.28069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.420+0000 7fe8dc43a740 20 client.28069 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.420+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/169222475 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d92a53ed50 0x55d92a541160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.420+0000 7fe8dc43a740 4 client.28069 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:17:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.420+0000 7fe8dc43a740 1 -- 172.21.15.2:0/169222475 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d92a49b830 con 0x55d92a53ed50 2024-09-10T08:17:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.420+0000 7fe8d3200640 1 --2- 172.21.15.2:0/169222475 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d92a53ed50 0x55d92a541160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.420+0000 7fe8d3200640 1 --2- 172.21.15.2:0/169222475 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d92a53ed50 0x55d92a541160 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97264e7a4b3dc57f server_cookie=5f6364366f0443b2 in_seq=0 out_seq=0 2024-09-10T08:17:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.420+0000 7fe8b3e00640 10 client.28069 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8b3e00640 1 -- 172.21.15.2:0/169222475 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d92a49b830 con 0x55d92a53ed50 2024-09-10T08:17:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8b3e00640 10 client.28069 handle_command_reply: tid=0 2024-09-10T08:17:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8dc43a740 1 client.28069 shutdown 2024-09-10T08:17:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8dc43a740 1 -- 172.21.15.2:0/169222475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a5299d0 msgr2=0x55d92a533780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/169222475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a5299d0 0x55d92a533780 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0012700 tx=0x7fe8c0012bc0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8dc43a740 1 -- 172.21.15.2:0/169222475 shutdown_connections 2024-09-10T08:17:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/169222475 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d92a53ed50 0x55d92a541160 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/169222475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a52ea60 0x55d92a53a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/169222475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a5299d0 0x55d92a533780 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8b3e00640 0 client.28069 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/169222475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a529090 0x55d92a533240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8dc43a740 1 -- 172.21.15.2:0/169222475 >> 172.21.15.2:0/169222475 conn(0x55d92a524700 msgr2=0x55d92a525f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8dc43a740 1 -- 172.21.15.2:0/169222475 shutdown_connections 2024-09-10T08:17:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.421+0000 7fe8dc43a740 1 -- 172.21.15.2:0/169222475 wait complete. 2024-09-10T08:17:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.422+0000 7fe8dc43a740 20 client.28069 trim_cache size 0 max 16384 2024-09-10T08:17:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.437+0000 7fe8dc43a740 1 Processor -- start 2024-09-10T08:17:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.437+0000 7fe8dc43a740 1 -- start start 2024-09-10T08:17:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.437+0000 7fe8dc43a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a529090 0x55d92a524ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.437+0000 7fe8dc43a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 0x55d92a53a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.437+0000 7fe8dc43a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 0x55d92a525020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.437+0000 7fe8dc43a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41b3680 con 0x55d92a52ea60 2024-09-10T08:17:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.437+0000 7fe8dc43a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41b0f50 con 0x55d92a5299d0 2024-09-10T08:17:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.437+0000 7fe8dc43a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41aeda0 con 0x55d92a529090 2024-09-10T08:17:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.437+0000 7fe8d1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 0x55d92a53a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.437+0000 7fe8d3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a529090 0x55d92a524ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.437+0000 7fe8d3200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 0x55d92a525020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.437+0000 7fe8d1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 0x55d92a53a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42514/0 (socket says 172.21.15.2:42514) 2024-09-10T08:17:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.437+0000 7fe8d1e00640 1 -- 172.21.15.2:0/2076593810 learned_addr learned my addr 172.21.15.2:0/2076593810 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.438+0000 7fe8d1e00640 1 -- 172.21.15.2:0/2076593810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a529090 msgr2=0x55d92a524ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.438+0000 7fe8d1e00640 1 --2- 172.21.15.2:0/2076593810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a529090 0x55d92a524ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.438+0000 7fe8d1e00640 1 -- 172.21.15.2:0/2076593810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 msgr2=0x55d92a525020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.438+0000 7fe8d1e00640 1 --2- 172.21.15.2:0/2076593810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 0x55d92a525020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.438+0000 7fe8d1e00640 1 -- 172.21.15.2:0/2076593810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d92a524520 con 0x55d92a5299d0 2024-09-10T08:17:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.438+0000 7fe8d3200640 1 --2- 172.21.15.2:0/2076593810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a529090 0x55d92a524ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.438+0000 7fe8d1e00640 1 --2- 172.21.15.2:0/2076593810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 0x55d92a53a780 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0003e90 tx=0x7fe8c0009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c097d973602e0a48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.438+0000 7fe8bea00640 1 -- 172.21.15.2:0/2076593810 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c002d020 con 0x55d92a5299d0 2024-09-10T08:17:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.438+0000 7fe8bea00640 1 -- 172.21.15.2:0/2076593810 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8c00050c0 con 0x55d92a5299d0 2024-09-10T08:17:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.438+0000 7fe8bea00640 1 -- 172.21.15.2:0/2076593810 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c00037e0 con 0x55d92a5299d0 2024-09-10T08:17:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.439+0000 7fe8dc43a740 1 -- 172.21.15.2:0/2076593810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 msgr2=0x55d92a53a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.439+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/2076593810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 0x55d92a53a780 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0003e90 tx=0x7fe8c0009fd0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.439+0000 7fe8dc43a740 1 -- 172.21.15.2:0/2076593810 shutdown_connections 2024-09-10T08:17:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.439+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/2076593810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 0x55d92a525020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.439+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/2076593810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 0x55d92a53a780 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.439+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/2076593810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a529090 0x55d92a524ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.439+0000 7fe8dc43a740 1 -- 172.21.15.2:0/2076593810 >> 172.21.15.2:0/2076593810 conn(0x55d92a524700 msgr2=0x55d92a52abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.439+0000 7fe8dc43a740 1 -- 172.21.15.2:0/2076593810 shutdown_connections 2024-09-10T08:17:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.439+0000 7fe8dc43a740 1 -- 172.21.15.2:0/2076593810 wait complete. 2024-09-10T08:17:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.439+0000 7fe8dc43a740 1 -- start start 2024-09-10T08:17:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.439+0000 7fe8dc43a740 1 Processor -- start 2024-09-10T08:17:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8dc43a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a529090 0x55d92a5379b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8dc43a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 0x55d92a537ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8dc43a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 0x55d92a538eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8dc43a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41af410 con 0x55d92a52ea60 2024-09-10T08:17:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8dc43a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41b0880 con 0x55d92a5299d0 2024-09-10T08:17:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8dc43a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41afae0 con 0x55d92a529090 2024-09-10T08:17:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8dc43a740 10 client.0 _ll_register_callbacks cb 0x7fe8c0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8d1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 0x55d92a537ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8d1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 0x55d92a537ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42526/0 (socket says 172.21.15.2:42526) 2024-09-10T08:17:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8d1e00640 1 -- 172.21.15.2:0/2543281214 learned_addr learned my addr 172.21.15.2:0/2543281214 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8d3200640 1 --2- 172.21.15.2:0/2543281214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a529090 0x55d92a5379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8d3200640 1 --2- 172.21.15.2:0/2543281214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 0x55d92a538eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8d1e00640 1 -- 172.21.15.2:0/2543281214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a529090 msgr2=0x55d92a5379b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8d1e00640 1 --2- 172.21.15.2:0/2543281214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a529090 0x55d92a5379b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8d1e00640 1 -- 172.21.15.2:0/2543281214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 msgr2=0x55d92a538eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8d1e00640 1 --2- 172.21.15.2:0/2543281214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 0x55d92a538eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.440+0000 7fe8d1e00640 1 -- 172.21.15.2:0/2543281214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d92a5394c0 con 0x55d92a5299d0 2024-09-10T08:17:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.441+0000 7fe8d1e00640 1 --2- 172.21.15.2:0/2543281214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 0x55d92a537ef0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0000c00 tx=0x7fe8c000fc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28cd9d9f03189a1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.441+0000 7fe8b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:17:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.441+0000 7fe8b3e00640 1 -- 172.21.15.2:0/2543281214 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0036020 con 0x55d92a5299d0 2024-09-10T08:17:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.441+0000 7fe8dc43a740 1 -- 172.21.15.2:0/2543281214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c00050c0 con 0x55d92a5299d0 2024-09-10T08:17:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.441+0000 7fe8b3e00640 1 -- 172.21.15.2:0/2543281214 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe8c0022040 con 0x55d92a5299d0 2024-09-10T08:17:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.441+0000 7fe8b3e00640 1 -- 172.21.15.2:0/2543281214 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe8c0041050 con 0x55d92a5299d0 2024-09-10T08:17:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.441+0000 7fe8dc43a740 10 client.28142 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.441+0000 7fe8dc43a740 1 -- 172.21.15.2:0/2543281214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d92a53a7d0 con 0x55d92a5299d0 2024-09-10T08:17:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.441+0000 7fe8b3e00640 1 -- 172.21.15.2:0/2543281214 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8c00050c0 con 0x55d92a5299d0 2024-09-10T08:17:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.441+0000 7fe8b3e00640 1 -- 172.21.15.2:0/2543281214 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe8c0010760 con 0x55d92a5299d0 2024-09-10T08:17:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.441+0000 7fe8dc43a740 10 client.28142 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.441+0000 7fe8dc43a740 10 client.28142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.441+0000 7fe8dc43a740 20 client.28142 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.442+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/2543281214 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d92a5a9e30 0x55d92a5ac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.442+0000 7fe8dc43a740 4 client.28142 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:17:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.442+0000 7fe8dc43a740 1 -- 172.21.15.2:0/2543281214 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d92a51fd80 con 0x55d92a5a9e30 2024-09-10T08:17:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.442+0000 7fe8d3200640 1 --2- 172.21.15.2:0/2543281214 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d92a5a9e30 0x55d92a5ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.442+0000 7fe8d3200640 1 --2- 172.21.15.2:0/2543281214 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d92a5a9e30 0x55d92a5ac250 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d117b943989561 server_cookie=ff0cd5f04b33c0c6 in_seq=0 out_seq=0 2024-09-10T08:17:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.442+0000 7fe8b3e00640 10 client.28142 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8b3e00640 1 -- 172.21.15.2:0/2543281214 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d92a51fd80 con 0x55d92a5a9e30 2024-09-10T08:17:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8b3e00640 10 client.28142 handle_command_reply: tid=0 2024-09-10T08:17:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8dc43a740 1 client.28142 shutdown 2024-09-10T08:17:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8dc43a740 1 -- 172.21.15.2:0/2543281214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 msgr2=0x55d92a537ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/2543281214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 0x55d92a537ef0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0000c00 tx=0x7fe8c000fc10 comp rx=0 tx=0).stop 2024-09-10T08:17:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8dc43a740 1 -- 172.21.15.2:0/2543281214 shutdown_connections 2024-09-10T08:17:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/2543281214 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d92a5a9e30 0x55d92a5ac250 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/2543281214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d92a52ea60 0x55d92a538eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/2543281214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d92a5299d0 0x55d92a537ef0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8dc43a740 1 --2- 172.21.15.2:0/2543281214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d92a529090 0x55d92a5379b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8b3e00640 0 client.28142 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8dc43a740 1 -- 172.21.15.2:0/2543281214 >> 172.21.15.2:0/2543281214 conn(0x55d92a524700 msgr2=0x55d92a523060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8dc43a740 1 -- 172.21.15.2:0/2543281214 shutdown_connections 2024-09-10T08:17:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8dc43a740 1 -- 172.21.15.2:0/2543281214 wait complete. 2024-09-10T08:17:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.452+0000 7fe8dc43a740 20 client.28142 trim_cache size 0 max 16384 2024-09-10T08:17:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:17:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:17:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:17:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:17:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.453+0000 7fe8d9400640 1 -- 172.21.15.2:0/13847193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe8b80a2cb0 msgr2=0x7fe8b80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.453+0000 7fe8d9400640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe8b80a2cb0 0x7fe8b80a5170 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4006010 tx=0x7fe8c402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.453+0000 7fe8d9400640 1 -- 172.21.15.2:0/13847193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4076bb0 msgr2=0x7fe8d419e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.453+0000 7fe8d9400640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4076bb0 0x7fe8d419e3d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4031440 tx=0x7fe8c4032ce0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.453+0000 7fe8d9400640 1 -- 172.21.15.2:0/13847193 shutdown_connections 2024-09-10T08:17:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.453+0000 7fe8d9400640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8d4078790 0x7fe8d41a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.453+0000 7fe8d9400640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe8b80a2cb0 0x7fe8b80a5170 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.453+0000 7fe8d9400640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8d4077df0 0x7fe8d419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.453+0000 7fe8d9400640 1 --2- 172.21.15.2:0/13847193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4076bb0 0x7fe8d419e3d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.453+0000 7fe8d9400640 1 -- 172.21.15.2:0/13847193 >> 172.21.15.2:0/13847193 conn(0x7fe8d40ff4e0 msgr2=0x7fe8d4100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.453+0000 7fe8d9400640 1 -- 172.21.15.2:0/13847193 shutdown_connections 2024-09-10T08:17:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:43.453+0000 7fe8d9400640 1 -- 172.21.15.2:0/13847193 wait complete. 2024-09-10T08:17:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:43 smithi155 ceph-mon[35405]: pgmap v674: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-10T08:17:43.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:43 smithi107 ceph-mon[35048]: pgmap v674: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-10T08:17:44.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:43 smithi002 ceph-mon[30563]: pgmap v674: 129 pgs: 129 active+clean; 848 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-10T08:17:44.279 INFO:tasks.cephfs.filesystem:scrub status for tag:a51b41bf-e7a4-4d7a-86ad-4a60755a35f6 - {'path': '/', 'tag': 'a51b41bf-e7a4-4d7a-86ad-4a60755a35f6', 'options': 'recursive,force'} 2024-09-10T08:17:44.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.870+0000 7f31ed200640 1 -- 172.21.15.2:0/1611138886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e8103e30 msgr2=0x7f31e8104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:44.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.870+0000 7f31ed200640 1 --2- 172.21.15.2:0/1611138886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e8103e30 0x7f31e8104210 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009860 tx=0x7f31d802a340 comp rx=0 tx=0).stop 2024-09-10T08:17:44.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.871+0000 7f31ed200640 1 -- 172.21.15.2:0/1611138886 shutdown_connections 2024-09-10T08:17:44.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.871+0000 7f31ed200640 1 --2- 172.21.15.2:0/1611138886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e8108f60 0x7f31e8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:44.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.871+0000 7f31ed200640 1 --2- 172.21.15.2:0/1611138886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e81047e0 0x7f31e8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:44.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.871+0000 7f31ed200640 1 --2- 172.21.15.2:0/1611138886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e8103e30 0x7f31e8104210 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:44.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.871+0000 7f31ed200640 1 -- 172.21.15.2:0/1611138886 >> 172.21.15.2:0/1611138886 conn(0x7f31e80ff910 msgr2=0x7f31e8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:44.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.871+0000 7f31ed200640 1 -- 172.21.15.2:0/1611138886 shutdown_connections 2024-09-10T08:17:44.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.871+0000 7f31ed200640 1 -- 172.21.15.2:0/1611138886 wait complete. 2024-09-10T08:17:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.872+0000 7f31ed200640 1 Processor -- start 2024-09-10T08:17:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.872+0000 7f31ed200640 1 -- start start 2024-09-10T08:17:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.872+0000 7f31ed200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e8103e30 0x7f31e819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.872+0000 7f31ed200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e81047e0 0x7f31e819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.872+0000 7f31ed200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8108f60 0x7f31e819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.872+0000 7f31ed200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f31e8113f80 con 0x7f31e8108f60 2024-09-10T08:17:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.872+0000 7f31ed200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f31e8113e00 con 0x7f31e81047e0 2024-09-10T08:17:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.872+0000 7f31ed200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f31e8114100 con 0x7f31e8103e30 2024-09-10T08:17:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.873+0000 7f31e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e81047e0 0x7f31e819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.873+0000 7f31e7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e8103e30 0x7f31e819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.873+0000 7f31e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e81047e0 0x7f31e819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42550/0 (socket says 172.21.15.2:42550) 2024-09-10T08:17:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.873+0000 7f31e6a00640 1 -- 172.21.15.2:0/115679513 learned_addr learned my addr 172.21.15.2:0/115679513 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.873+0000 7f31e7e00640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8108f60 0x7f31e819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.873+0000 7f31e7400640 1 -- 172.21.15.2:0/115679513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e81047e0 msgr2=0x7f31e819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.873+0000 7f31e7400640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e81047e0 0x7f31e819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.873+0000 7f31e7400640 1 -- 172.21.15.2:0/115679513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8108f60 msgr2=0x7f31e819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.873+0000 7f31e7400640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8108f60 0x7f31e819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.873+0000 7f31e7400640 1 -- 172.21.15.2:0/115679513 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e819f410 con 0x7f31e8103e30 2024-09-10T08:17:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.873+0000 7f31e6a00640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e81047e0 0x7f31e819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.873+0000 7f31e7e00640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8108f60 0x7f31e819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.873+0000 7f31e7400640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e8103e30 0x7f31e819a460 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f31d8035040 tx=0x7f31d8033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.874+0000 7f31dfe00640 1 -- 172.21.15.2:0/115679513 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d803f020 con 0x7f31e8103e30 2024-09-10T08:17:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.874+0000 7f31ed200640 1 -- 172.21.15.2:0/115679513 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e819f6a0 con 0x7f31e8103e30 2024-09-10T08:17:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.874+0000 7f31dfe00640 1 -- 172.21.15.2:0/115679513 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d8030550 con 0x7f31e8103e30 2024-09-10T08:17:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.874+0000 7f31dfe00640 1 -- 172.21.15.2:0/115679513 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8037040 con 0x7f31e8103e30 2024-09-10T08:17:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.874+0000 7f31ed200640 1 -- 172.21.15.2:0/115679513 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e81a6fe0 con 0x7f31e8103e30 2024-09-10T08:17:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.875+0000 7f31ed200640 1 -- 172.21.15.2:0/115679513 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e8105310 con 0x7f31e8103e30 2024-09-10T08:17:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.877+0000 7f31dfe00640 1 -- 172.21.15.2:0/115679513 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f31d8042020 con 0x7f31e8103e30 2024-09-10T08:17:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.878+0000 7f31dfe00640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f31cc0a2b80 0x7f31cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.878+0000 7f31dfe00640 1 -- 172.21.15.2:0/115679513 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f31d80ef540 con 0x7f31e8103e30 2024-09-10T08:17:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.878+0000 7f31e6a00640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f31cc0a2b80 0x7f31cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.879+0000 7f31e6a00640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f31cc0a2b80 0x7f31cc0a5040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f31e80ffd30 tx=0x7f31d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:43.881+0000 7f31dfe00640 1 -- 172.21.15.2:0/115679513 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f31d8049020 con 0x7f31e8103e30 2024-09-10T08:17:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.105+0000 7f31ed200640 1 -- 172.21.15.2:0/115679513 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31e8104210 con 0x7f31e8103e30 2024-09-10T08:17:44.386 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:17:44.387 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:17:44.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.106+0000 7f31dfe00640 1 -- 172.21.15.2:0/115679513 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f31d80bc050 con 0x7f31e8103e30 2024-09-10T08:17:44.387 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:17:44.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.109+0000 7f31ed200640 1 -- 172.21.15.2:0/115679513 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f31cc0a2b80 msgr2=0x7f31cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:44.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.109+0000 7f31ed200640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f31cc0a2b80 0x7f31cc0a5040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f31e80ffd30 tx=0x7f31d4009260 comp rx=0 tx=0).stop 2024-09-10T08:17:44.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.109+0000 7f31ed200640 1 -- 172.21.15.2:0/115679513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e8103e30 msgr2=0x7f31e819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:44.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.109+0000 7f31ed200640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e8103e30 0x7f31e819a460 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f31d8035040 tx=0x7f31d8033b60 comp rx=0 tx=0).stop 2024-09-10T08:17:44.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.110+0000 7f31ed200640 1 -- 172.21.15.2:0/115679513 shutdown_connections 2024-09-10T08:17:44.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.110+0000 7f31ed200640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8108f60 0x7f31e819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:44.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.110+0000 7f31ed200640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e81047e0 0x7f31e819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:44.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.110+0000 7f31ed200640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f31cc0a2b80 0x7f31cc0a5040 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:44.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.110+0000 7f31ed200640 1 --2- 172.21.15.2:0/115679513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e8103e30 0x7f31e819a460 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:44.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.110+0000 7f31ed200640 1 -- 172.21.15.2:0/115679513 >> 172.21.15.2:0/115679513 conn(0x7f31e80ff910 msgr2=0x7f31e8100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:44.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.110+0000 7f31ed200640 1 -- 172.21.15.2:0/115679513 shutdown_connections 2024-09-10T08:17:44.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:44.110+0000 7f31ed200640 1 -- 172.21.15.2:0/115679513 wait complete. 2024-09-10T08:17:44.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:44 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:17:44.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:44 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:44.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:44 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:44.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:44 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:44.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:44 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:44.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:44 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/115679513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:17:45.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:44 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:17:45.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:44 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:45.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:44 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:45.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:44 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:45.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:44 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:45.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:44 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/115679513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:17:45.144 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id 2bebef3 --release --await' 2024-09-10T08:17:45.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id 2bebef3 --release --await 2024-09-10T08:17:45.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:44 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:17:45.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:44 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:45.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:44 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:45.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:44 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:45.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:44 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:45.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:44 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/115679513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:17:45.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:45 smithi107 ceph-mon[35048]: pgmap v675: 129 pgs: 129 active+clean; 853 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T08:17:45.972 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:17:46.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:45 smithi002 ceph-mon[30563]: pgmap v675: 129 pgs: 129 active+clean; 853 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T08:17:46.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:45 smithi155 ceph-mon[35405]: pgmap v675: 129 pgs: 129 active+clean; 853 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T08:17:46.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.650+0000 7f6de4c00640 1 -- 172.21.15.2:0/3079180414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de0103e30 msgr2=0x7f6de0104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:46.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.650+0000 7f6de4c00640 1 --2- 172.21.15.2:0/3079180414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de0103e30 0x7f6de0104210 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009ca0 tx=0x7f6dd002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.650+0000 7f6de4c00640 1 -- 172.21.15.2:0/3079180414 shutdown_connections 2024-09-10T08:17:46.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.650+0000 7f6de4c00640 1 --2- 172.21.15.2:0/3079180414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6de0108f60 0x7f6de0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.650+0000 7f6de4c00640 1 --2- 172.21.15.2:0/3079180414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de01047e0 0x7f6de0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.650+0000 7f6de4c00640 1 --2- 172.21.15.2:0/3079180414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de0103e30 0x7f6de0104210 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.650+0000 7f6de4c00640 1 -- 172.21.15.2:0/3079180414 >> 172.21.15.2:0/3079180414 conn(0x7f6de00ff8d0 msgr2=0x7f6de0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:46.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.650+0000 7f6de4c00640 1 -- 172.21.15.2:0/3079180414 shutdown_connections 2024-09-10T08:17:46.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.650+0000 7f6de4c00640 1 -- 172.21.15.2:0/3079180414 wait complete. 2024-09-10T08:17:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.651+0000 7f6de4c00640 1 Processor -- start 2024-09-10T08:17:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.651+0000 7f6de4c00640 1 -- start start 2024-09-10T08:17:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.652+0000 7f6de4c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de0103e30 0x7f6de019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.652+0000 7f6de4c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de01047e0 0x7f6de019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.652+0000 7f6de4c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6de0108f60 0x7f6de019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.652+0000 7f6de4c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0114010 con 0x7f6de0103e30 2024-09-10T08:17:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.652+0000 7f6de4c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0113e90 con 0x7f6de01047e0 2024-09-10T08:17:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.652+0000 7f6de4c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0114190 con 0x7f6de0108f60 2024-09-10T08:17:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.652+0000 7f6ddf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de0103e30 0x7f6de019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.652+0000 7f6ddf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de0103e30 0x7f6de019a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33614/0 (socket says 172.21.15.2:33614) 2024-09-10T08:17:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.652+0000 7f6ddf400640 1 -- 172.21.15.2:0/3184774023 learned_addr learned my addr 172.21.15.2:0/3184774023 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.652+0000 7f6dd6a00640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de01047e0 0x7f6de019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.653+0000 7f6ddfe00640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6de0108f60 0x7f6de019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.653+0000 7f6dd6a00640 1 -- 172.21.15.2:0/3184774023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6de0108f60 msgr2=0x7f6de019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.653+0000 7f6dd6a00640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6de0108f60 0x7f6de019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.653+0000 7f6dd6a00640 1 -- 172.21.15.2:0/3184774023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de0103e30 msgr2=0x7f6de019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.653+0000 7f6dd6a00640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de0103e30 0x7f6de019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.653+0000 7f6dd6a00640 1 -- 172.21.15.2:0/3184774023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de019f3c0 con 0x7f6de01047e0 2024-09-10T08:17:46.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.653+0000 7f6dd6a00640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de01047e0 0x7f6de019a950 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800e9f0 tx=0x7f6dc800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.653+0000 7f6ddcc00640 1 -- 172.21.15.2:0/3184774023 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc800ccc0 con 0x7f6de01047e0 2024-09-10T08:17:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.654+0000 7f6de4c00640 1 -- 172.21.15.2:0/3184774023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de019f6b0 con 0x7f6de01047e0 2024-09-10T08:17:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.654+0000 7f6ddcc00640 1 -- 172.21.15.2:0/3184774023 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc8010910 con 0x7f6de01047e0 2024-09-10T08:17:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.654+0000 7f6ddcc00640 1 -- 172.21.15.2:0/3184774023 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8018960 con 0x7f6de01047e0 2024-09-10T08:17:46.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.654+0000 7f6de4c00640 1 -- 172.21.15.2:0/3184774023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de01a6f90 con 0x7f6de01047e0 2024-09-10T08:17:46.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.658+0000 7f6de8e60740 1 Processor -- start 2024-09-10T08:17:46.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.658+0000 7f6de8e60740 1 -- start start 2024-09-10T08:17:46.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.658+0000 7f6de8e60740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b03c520 0x55de5b03c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.659+0000 7f6de8e60740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b0471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.659+0000 7f6de8e60740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b047700 0x55de5b049af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.659+0000 7f6de8e60740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55de5afa2650 con 0x55de5b03ce40 2024-09-10T08:17:46.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.659+0000 7f6de8e60740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55de5af49f10 con 0x55de5b047700 2024-09-10T08:17:46.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.659+0000 7f6de8e60740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55de5af9e610 con 0x55de5b03c520 2024-09-10T08:17:46.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.659+0000 7f6ddcc00640 1 -- 172.21.15.2:0/3184774023 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6dc8010ac0 con 0x7f6de01047e0 2024-09-10T08:17:46.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.659+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b03c520 0x55de5b03c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.659+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b03c520 0x55de5b03c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36538/0 (socket says 172.21.15.2:36538) 2024-09-10T08:17:46.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.659+0000 7f6ddfe00640 1 -- 172.21.15.2:0/3343772701 learned_addr learned my addr 172.21.15.2:0/3343772701 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:46.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.659+0000 7f6dd6a00640 1 --2- 172.21.15.2:0/3343772701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b047700 0x55de5b049af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.659+0000 7f6ddf400640 1 --2- 172.21.15.2:0/3343772701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b0471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.660+0000 7f6ddfe00640 1 -- 172.21.15.2:0/3343772701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b047700 msgr2=0x55de5b049af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:46.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.660+0000 7f6ddfe00640 1 --2- 172.21.15.2:0/3343772701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b047700 0x55de5b049af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.660+0000 7f6ddfe00640 1 -- 172.21.15.2:0/3343772701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 msgr2=0x55de5b0471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:46.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.660+0000 7f6ddfe00640 1 --2- 172.21.15.2:0/3343772701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b0471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.660+0000 7f6ddfe00640 1 -- 172.21.15.2:0/3343772701 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de5acb1580 con 0x55de5b03c520 2024-09-10T08:17:46.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.660+0000 7f6dd6a00640 1 --2- 172.21.15.2:0/3343772701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b047700 0x55de5b049af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:46.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.660+0000 7f6ddf400640 1 --2- 172.21.15.2:0/3343772701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b0471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:17:46.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.660+0000 7f6ddfe00640 1 --2- 172.21.15.2:0/3343772701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b03c520 0x55de5b03c900 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009c20 tx=0x7f6dd8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10c42e233c5191f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:46.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.661+0000 7f6ddcc00640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6dc40a2b80 0x7f6dc40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.661+0000 7f6ddcc00640 1 -- 172.21.15.2:0/3184774023 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6dc80cf820 con 0x7f6de01047e0 2024-09-10T08:17:46.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.661+0000 7f6ddf400640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6dc40a2b80 0x7f6dc40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.662+0000 7f6ddf400640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6dc40a2b80 0x7f6dc40a5040 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55de5b039910 tx=0x7f6dd0005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:46.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.663+0000 7f6dcfe00640 1 -- 172.21.15.2:0/3343772701 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd801e070 con 0x55de5b03c520 2024-09-10T08:17:46.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.664+0000 7f6dcfe00640 1 -- 172.21.15.2:0/3343772701 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd8019b10 con 0x55de5b03c520 2024-09-10T08:17:46.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.664+0000 7f6dcfe00640 1 -- 172.21.15.2:0/3343772701 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800eae0 con 0x55de5b03c520 2024-09-10T08:17:46.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.664+0000 7f6de8e60740 1 -- 172.21.15.2:0/3343772701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b03c520 msgr2=0x55de5b03c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:46.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.664+0000 7f6de8e60740 1 --2- 172.21.15.2:0/3343772701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b03c520 0x55de5b03c900 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009c20 tx=0x7f6dd8011810 comp rx=0 tx=0).stop 2024-09-10T08:17:46.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.664+0000 7f6de8e60740 1 -- 172.21.15.2:0/3343772701 shutdown_connections 2024-09-10T08:17:46.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.664+0000 7f6de8e60740 1 --2- 172.21.15.2:0/3343772701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b047700 0x55de5b049af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.664+0000 7f6de8e60740 1 --2- 172.21.15.2:0/3343772701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b0471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.664+0000 7f6de8e60740 1 --2- 172.21.15.2:0/3343772701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b03c520 0x55de5b03c900 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.664+0000 7f6de8e60740 1 -- 172.21.15.2:0/3343772701 >> 172.21.15.2:0/3343772701 conn(0x55de5b038ab0 msgr2=0x55de5b038e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:46.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.665+0000 7f6de8e60740 1 -- 172.21.15.2:0/3343772701 shutdown_connections 2024-09-10T08:17:46.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.665+0000 7f6de8e60740 1 -- 172.21.15.2:0/3343772701 wait complete. 2024-09-10T08:17:46.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.665+0000 7f6de8e60740 1 -- start start 2024-09-10T08:17:46.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.665+0000 7f6de8e60740 1 Processor -- start 2024-09-10T08:17:46.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.666+0000 7f6de8e60740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 0x55de5b044630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.666+0000 7f6de8e60740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b03ce40 0x55de5b044b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.666+0000 7f6de8e60740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b047700 0x55de5b0450b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.666+0000 7f6de8e60740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55de5b04a2d0 con 0x55de5b047700 2024-09-10T08:17:46.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.666+0000 7f6de8e60740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55de5af5d350 con 0x55de5b03c520 2024-09-10T08:17:46.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.666+0000 7f6de8e60740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55de5b04a450 con 0x55de5b03ce40 2024-09-10T08:17:46.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.666+0000 7f6de8e60740 10 client.0 _ll_register_callbacks cb 0x55de5af80290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:46.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.666+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b047700 0x55de5b0450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.666+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b047700 0x55de5b0450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33620/0 (socket says 172.21.15.2:33620) 2024-09-10T08:17:46.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.666+0000 7f6ddfe00640 1 -- 172.21.15.2:0/1538944054 learned_addr learned my addr 172.21.15.2:0/1538944054 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:46.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.666+0000 7f6dd6a00640 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b03ce40 0x55de5b044b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.666+0000 7f6ddf400640 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 0x55de5b044630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.667+0000 7f6ddfe00640 1 -- 172.21.15.2:0/1538944054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b03ce40 msgr2=0x55de5b044b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:46.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.667+0000 7f6ddfe00640 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b03ce40 0x55de5b044b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.667+0000 7f6ddfe00640 1 -- 172.21.15.2:0/1538944054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 msgr2=0x55de5b044630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:46.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.667+0000 7f6ddfe00640 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 0x55de5b044630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.667+0000 7f6ddfe00640 1 -- 172.21.15.2:0/1538944054 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de5ad7b960 con 0x55de5b047700 2024-09-10T08:17:46.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.667+0000 7f6dd6a00640 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b03ce40 0x55de5b044b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:46.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.667+0000 7f6ddfe00640 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b047700 0x55de5b0450b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009bf0 tx=0x7f6dd8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230f16ad5af7bc44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:46.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.667+0000 7f6dccc00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:17:46.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.668+0000 7f6dccc00640 1 -- 172.21.15.2:0/1538944054 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd801e050 con 0x55de5b047700 2024-09-10T08:17:46.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.668+0000 7f6dccc00640 1 -- 172.21.15.2:0/1538944054 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6dd8005040 con 0x55de5b047700 2024-09-10T08:17:46.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.668+0000 7f6de8e60740 1 -- 172.21.15.2:0/1538944054 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd8019b10 con 0x55de5b047700 2024-09-10T08:17:46.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.668+0000 7f6dccc00640 1 -- 172.21.15.2:0/1538944054 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6dd8032050 con 0x55de5b047700 2024-09-10T08:17:46.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.668+0000 7f6de8e60740 10 client.18126 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:46.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.668+0000 7f6de8e60740 1 -- 172.21.15.2:0/1538944054 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55de5b055f10 con 0x55de5b047700 2024-09-10T08:17:46.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.668+0000 7f6dccc00640 1 -- 172.21.15.2:0/1538944054 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd8019b10 con 0x55de5b047700 2024-09-10T08:17:46.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.669+0000 7f6dccc00640 1 -- 172.21.15.2:0/1538944054 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6dd800eca0 con 0x55de5b047700 2024-09-10T08:17:46.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.669+0000 7f6de8e60740 10 client.18126 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:46.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.669+0000 7f6de8e60740 10 client.18126 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:17:46.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.669+0000 7f6de8e60740 20 client.18126 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:46.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.669+0000 7f6de8e60740 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55de5b057370 0x55de5b059760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.669+0000 7f6de8e60740 4 client.18126 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:17:46.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.669+0000 7f6de8e60740 1 -- 172.21.15.2:0/1538944054 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de5afafbc0 con 0x55de5b057370 2024-09-10T08:17:46.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.669+0000 7f6ddf400640 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55de5b057370 0x55de5b059760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.670+0000 7f6dccc00640 10 client.18126 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:46.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.670+0000 7f6ddf400640 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55de5b057370 0x55de5b059760 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e3ffde3fd892fd server_cookie=853442bf3f455c63 in_seq=0 out_seq=0 2024-09-10T08:17:46.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.671+0000 7f6dccc00640 1 -- 172.21.15.2:0/1538944054 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de5afafbc0 con 0x55de5b057370 2024-09-10T08:17:46.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.671+0000 7f6dccc00640 10 client.18126 handle_command_reply: tid=0 2024-09-10T08:17:46.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.671+0000 7f6de8e60740 1 client.18126 shutdown 2024-09-10T08:17:46.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.671+0000 7f6de8e60740 1 -- 172.21.15.2:0/1538944054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b047700 msgr2=0x55de5b0450b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:46.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.671+0000 7f6de8e60740 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b047700 0x55de5b0450b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009bf0 tx=0x7f6dd8009f00 comp rx=0 tx=0).stop 2024-09-10T08:17:46.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.671+0000 7f6de8e60740 1 -- 172.21.15.2:0/1538944054 shutdown_connections 2024-09-10T08:17:46.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.671+0000 7f6de8e60740 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55de5b057370 0x55de5b059760 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.671+0000 7f6de8e60740 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b047700 0x55de5b0450b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.672+0000 7f6de8e60740 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b03ce40 0x55de5b044b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.672+0000 7f6de8e60740 1 --2- 172.21.15.2:0/1538944054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 0x55de5b044630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.672+0000 7f6de8e60740 1 -- 172.21.15.2:0/1538944054 >> 172.21.15.2:0/1538944054 conn(0x55de5b038ab0 msgr2=0x55de5b037cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:46.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.672+0000 7f6dccc00640 0 client.18126 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:46.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.672+0000 7f6de8e60740 1 -- 172.21.15.2:0/1538944054 shutdown_connections 2024-09-10T08:17:46.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.672+0000 7f6de8e60740 1 -- 172.21.15.2:0/1538944054 wait complete. 2024-09-10T08:17:46.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.672+0000 7f6de8e60740 20 client.18126 trim_cache size 0 max 16384 2024-09-10T08:17:46.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.697+0000 7f6de8e60740 1 Processor -- start 2024-09-10T08:17:46.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.697+0000 7f6de8e60740 1 -- start start 2024-09-10T08:17:46.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.697+0000 7f6de8e60740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b038ab0 0x55de5b046490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.697+0000 7f6de8e60740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 0x55de5b05ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.697+0000 7f6de8e60740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b05f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.697+0000 7f6de8e60740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01b37f0 con 0x55de5b03ce40 2024-09-10T08:17:46.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.697+0000 7f6de8e60740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01b10c0 con 0x55de5b03c520 2024-09-10T08:17:46.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.697+0000 7f6de8e60740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01aef10 con 0x55de5b038ab0 2024-09-10T08:17:46.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.698+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b05f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.698+0000 7f6ddf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b038ab0 0x55de5b046490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.698+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b05f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33632/0 (socket says 172.21.15.2:33632) 2024-09-10T08:17:46.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.698+0000 7f6ddfe00640 1 -- 172.21.15.2:0/2543320945 learned_addr learned my addr 172.21.15.2:0/2543320945 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:46.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.698+0000 7f6dd6a00640 1 --2- 172.21.15.2:0/2543320945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 0x55de5b05ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.698+0000 7f6ddfe00640 1 -- 172.21.15.2:0/2543320945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b038ab0 msgr2=0x55de5b046490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:46.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.698+0000 7f6ddfe00640 1 --2- 172.21.15.2:0/2543320945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b038ab0 0x55de5b046490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.698+0000 7f6ddfe00640 1 -- 172.21.15.2:0/2543320945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 msgr2=0x55de5b05ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:46.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.698+0000 7f6ddfe00640 1 --2- 172.21.15.2:0/2543320945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 0x55de5b05ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.698+0000 7f6ddfe00640 1 -- 172.21.15.2:0/2543320945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de5b039740 con 0x55de5b03ce40 2024-09-10T08:17:46.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.698+0000 7f6dd6a00640 1 --2- 172.21.15.2:0/2543320945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 0x55de5b05ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:46.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.698+0000 7f6ddf400640 1 --2- 172.21.15.2:0/2543320945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b038ab0 0x55de5b046490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:46.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.699+0000 7f6ddfe00640 1 --2- 172.21.15.2:0/2543320945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b05f3c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8004b40 tx=0x7f6dd8011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12217dede11f850c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:46.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.699+0000 7f6dcf400640 1 -- 172.21.15.2:0/2543320945 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8015070 con 0x55de5b03ce40 2024-09-10T08:17:46.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.699+0000 7f6dcf400640 1 -- 172.21.15.2:0/2543320945 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd801f040 con 0x55de5b03ce40 2024-09-10T08:17:46.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.699+0000 7f6dcf400640 1 -- 172.21.15.2:0/2543320945 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd801e070 con 0x55de5b03ce40 2024-09-10T08:17:46.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.700+0000 7f6de8e60740 1 -- 172.21.15.2:0/2543320945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 msgr2=0x55de5b05f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:46.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.700+0000 7f6de8e60740 1 --2- 172.21.15.2:0/2543320945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b05f3c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8004b40 tx=0x7f6dd8011d50 comp rx=0 tx=0).stop 2024-09-10T08:17:46.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.700+0000 7f6de8e60740 1 -- 172.21.15.2:0/2543320945 shutdown_connections 2024-09-10T08:17:46.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.700+0000 7f6de8e60740 1 --2- 172.21.15.2:0/2543320945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b05f3c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.700+0000 7f6de8e60740 1 --2- 172.21.15.2:0/2543320945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 0x55de5b05ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.700+0000 7f6de8e60740 1 --2- 172.21.15.2:0/2543320945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b038ab0 0x55de5b046490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.700+0000 7f6de8e60740 1 -- 172.21.15.2:0/2543320945 >> 172.21.15.2:0/2543320945 conn(0x55de5b040020 msgr2=0x55de5b045d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:46.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.700+0000 7f6de8e60740 1 -- 172.21.15.2:0/2543320945 shutdown_connections 2024-09-10T08:17:46.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.700+0000 7f6de8e60740 1 -- 172.21.15.2:0/2543320945 wait complete. 2024-09-10T08:17:46.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.700+0000 7f6de8e60740 1 -- start start 2024-09-10T08:17:46.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.701+0000 7f6de8e60740 1 Processor -- start 2024-09-10T08:17:46.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.701+0000 7f6de8e60740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b038ab0 0x55de5b04b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.701+0000 7f6de8e60740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 0x55de5b04bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.701+0000 7f6de8e60740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b0c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.701+0000 7f6de8e60740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01af580 con 0x55de5b03ce40 2024-09-10T08:17:46.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.701+0000 7f6de8e60740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01b09f0 con 0x55de5b03c520 2024-09-10T08:17:46.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.701+0000 7f6de8e60740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01afc50 con 0x55de5b038ab0 2024-09-10T08:17:46.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.701+0000 7f6de8e60740 10 client.0 _ll_register_callbacks cb 0x7f6dd801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:46.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.702+0000 7f6ddf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b038ab0 0x55de5b04b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.702+0000 7f6dd6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 0x55de5b04bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.702+0000 7f6ddf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b038ab0 0x55de5b04b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:36566/0 (socket says 172.21.15.2:36566) 2024-09-10T08:17:46.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.702+0000 7f6ddf400640 1 -- 172.21.15.2:0/4275182359 learned_addr learned my addr 172.21.15.2:0/4275182359 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:46.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.702+0000 7f6ddfe00640 1 --2- 172.21.15.2:0/4275182359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b0c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.702+0000 7f6ddf400640 1 -- 172.21.15.2:0/4275182359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 msgr2=0x55de5b04bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:46.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.702+0000 7f6ddf400640 1 --2- 172.21.15.2:0/4275182359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 0x55de5b04bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.702+0000 7f6ddf400640 1 -- 172.21.15.2:0/4275182359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 msgr2=0x55de5b0c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:46.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.702+0000 7f6ddf400640 1 --2- 172.21.15.2:0/4275182359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b0c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.702+0000 7f6ddf400640 1 -- 172.21.15.2:0/4275182359 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de5b045600 con 0x55de5b038ab0 2024-09-10T08:17:46.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.703+0000 7f6ddf400640 1 --2- 172.21.15.2:0/4275182359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b038ab0 0x55de5b04b810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0005e30 tx=0x7f6dd002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f4e712c70974099 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:46.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.703+0000 7f6dccc00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:17:46.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.703+0000 7f6dccc00640 1 -- 172.21.15.2:0/4275182359 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0031450 con 0x55de5b038ab0 2024-09-10T08:17:46.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.703+0000 7f6de8e60740 1 -- 172.21.15.2:0/4275182359 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd801f040 con 0x55de5b038ab0 2024-09-10T08:17:46.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.703+0000 7f6dccc00640 1 -- 172.21.15.2:0/4275182359 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6dd00315f0 con 0x55de5b038ab0 2024-09-10T08:17:46.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.703+0000 7f6dccc00640 1 -- 172.21.15.2:0/4275182359 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6dd0066050 con 0x55de5b038ab0 2024-09-10T08:17:46.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.703+0000 7f6de8e60740 10 client.28099 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:46.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.703+0000 7f6de8e60740 1 -- 172.21.15.2:0/4275182359 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55de5b0c2420 con 0x55de5b038ab0 2024-09-10T08:17:46.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.704+0000 7f6dccc00640 1 -- 172.21.15.2:0/4275182359 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd801f040 con 0x55de5b038ab0 2024-09-10T08:17:46.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.704+0000 7f6dccc00640 1 -- 172.21.15.2:0/4275182359 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6dd002f610 con 0x55de5b038ab0 2024-09-10T08:17:46.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.704+0000 7f6de8e60740 10 client.28099 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:46.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.704+0000 7f6de8e60740 10 client.28099 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:17:46.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.704+0000 7f6de8e60740 20 client.28099 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:46.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.704+0000 7f6de8e60740 1 --2- 172.21.15.2:0/4275182359 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55de5b047700 0x55de5b0c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:46.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.704+0000 7f6de8e60740 4 client.28099 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2bebef3", "release": true, "await": true}] 2024-09-10T08:17:46.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.704+0000 7f6de8e60740 1 -- 172.21.15.2:0/4275182359 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2bebef3", "release": true, "await": true}) -- 0x55de5afaa0d0 con 0x55de5b047700 2024-09-10T08:17:46.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.704+0000 7f6dd6a00640 1 --2- 172.21.15.2:0/4275182359 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55de5b047700 0x55de5b0c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:46.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.705+0000 7f6dd6a00640 1 --2- 172.21.15.2:0/4275182359 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55de5b047700 0x55de5b0c4b60 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a29fdf2bd16c7913 server_cookie=13d71b87cf44e351 in_seq=0 out_seq=0 2024-09-10T08:17:46.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.705+0000 7f6dccc00640 10 client.28099 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:46.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.707+0000 7f6dccc00640 1 -- 172.21.15.2:0/4275182359 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55de5afaa0d0 con 0x55de5b047700 2024-09-10T08:17:46.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.707+0000 7f6dccc00640 10 client.28099 handle_command_reply: tid=0 2024-09-10T08:17:46.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.707+0000 7f6de8e60740 1 client.28099 shutdown 2024-09-10T08:17:46.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.707+0000 7f6de8e60740 1 -- 172.21.15.2:0/4275182359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b038ab0 msgr2=0x55de5b04b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:46.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.707+0000 7f6de8e60740 1 --2- 172.21.15.2:0/4275182359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b038ab0 0x55de5b04b810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0005e30 tx=0x7f6dd002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.708+0000 7f6de8e60740 1 -- 172.21.15.2:0/4275182359 shutdown_connections 2024-09-10T08:17:46.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.708+0000 7f6de8e60740 1 --2- 172.21.15.2:0/4275182359 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55de5b047700 0x55de5b0c4b60 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.708+0000 7f6de8e60740 1 --2- 172.21.15.2:0/4275182359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5b03ce40 0x55de5b0c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.708+0000 7f6de8e60740 1 --2- 172.21.15.2:0/4275182359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55de5b03c520 0x55de5b04bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.708+0000 7f6dccc00640 0 client.28099 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:46.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.708+0000 7f6de8e60740 1 --2- 172.21.15.2:0/4275182359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de5b038ab0 0x55de5b04b810 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.708+0000 7f6de8e60740 1 -- 172.21.15.2:0/4275182359 >> 172.21.15.2:0/4275182359 conn(0x55de5b040020 msgr2=0x55de5b044df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:46.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.708+0000 7f6de8e60740 1 -- 172.21.15.2:0/4275182359 shutdown_connections 2024-09-10T08:17:46.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.708+0000 7f6de8e60740 1 -- 172.21.15.2:0/4275182359 wait complete. 2024-09-10T08:17:46.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.708+0000 7f6de8e60740 20 client.28099 trim_cache size 0 max 16384 2024-09-10T08:17:46.709 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:17:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:17:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:17:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 32, 2024-09-10T08:17:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:17:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "2bebef3": { 2024-09-10T08:17:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 32, 2024-09-10T08:17:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:17:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:17:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:17:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:17:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:17:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:17:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 144.6, 2024-09-10T08:17:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:17:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:17:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:17:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:17:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:17:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:17:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:17:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:17:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:17:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:17:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:17:46.711 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:17:46.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.709+0000 7f6de4c00640 1 -- 172.21.15.2:0/3184774023 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6dc40a2b80 msgr2=0x7f6dc40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:46.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.709+0000 7f6de4c00640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6dc40a2b80 0x7f6dc40a5040 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55de5b039910 tx=0x7f6dd0005d80 comp rx=0 tx=0).stop 2024-09-10T08:17:46.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.709+0000 7f6de4c00640 1 -- 172.21.15.2:0/3184774023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de01047e0 msgr2=0x7f6de019a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:46.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.709+0000 7f6de4c00640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de01047e0 0x7f6de019a950 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800e9f0 tx=0x7f6dc800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.709+0000 7f6de4c00640 1 -- 172.21.15.2:0/3184774023 shutdown_connections 2024-09-10T08:17:46.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.709+0000 7f6de4c00640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6dc40a2b80 0x7f6dc40a5040 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.709+0000 7f6de4c00640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6de0108f60 0x7f6de019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.709+0000 7f6de4c00640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de01047e0 0x7f6de019a950 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.709+0000 7f6de4c00640 1 --2- 172.21.15.2:0/3184774023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de0103e30 0x7f6de019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.709+0000 7f6de4c00640 1 -- 172.21.15.2:0/3184774023 >> 172.21.15.2:0/3184774023 conn(0x7f6de00ff8d0 msgr2=0x7f6de0100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.709+0000 7f6de4c00640 1 -- 172.21.15.2:0/3184774023 shutdown_connections 2024-09-10T08:17:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:17:46.710+0000 7f6de4c00640 1 -- 172.21.15.2:0/3184774023 wait complete. 2024-09-10T08:17:47.737 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2bebef3', total seconds elapsed: 30.3 2024-09-10T08:17:47.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:47 smithi107 ceph-mon[35048]: pgmap v676: 129 pgs: 129 active+clean; 853 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-10T08:17:48.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:47 smithi002 ceph-mon[30563]: pgmap v676: 129 pgs: 129 active+clean; 853 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-10T08:17:48.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:47 smithi155 ceph-mon[35405]: pgmap v676: 129 pgs: 129 active+clean; 853 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-10T08:17:49.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:49 smithi002 ceph-mon[30563]: pgmap v677: 129 pgs: 129 active+clean; 871 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-10T08:17:49.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:49 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:49.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:49 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:49.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:49 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:17:49.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:49 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:17:49.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:49 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:49.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:49 smithi155 ceph-mon[35405]: pgmap v677: 129 pgs: 129 active+clean; 871 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-10T08:17:49.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:49 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:49.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:49 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:49.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:49 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:17:49.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:49 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:17:49.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:49 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:49 smithi107 ceph-mon[35048]: pgmap v677: 129 pgs: 129 active+clean; 871 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-10T08:17:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:17:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:17:49.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:17:51.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:51 smithi002 ceph-mon[30563]: pgmap v678: 129 pgs: 129 active+clean; 871 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-10T08:17:51.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:51 smithi155 ceph-mon[35405]: pgmap v678: 129 pgs: 129 active+clean; 871 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-10T08:17:51.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:51 smithi107 ceph-mon[35048]: pgmap v678: 129 pgs: 129 active+clean; 871 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-10T08:17:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:53 smithi002 ceph-mon[30563]: pgmap v679: 129 pgs: 129 active+clean; 870 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-10T08:17:53.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:53 smithi155 ceph-mon[35405]: pgmap v679: 129 pgs: 129 active+clean; 870 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-10T08:17:53.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:53 smithi107 ceph-mon[35048]: pgmap v679: 129 pgs: 129 active+clean; 870 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-10T08:17:55.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:55 smithi155 ceph-mon[35405]: pgmap v680: 129 pgs: 129 active+clean; 928 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2024-09-10T08:17:55.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:55 smithi107 ceph-mon[35048]: pgmap v680: 129 pgs: 129 active+clean; 928 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2024-09-10T08:17:55.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:55 smithi002 ceph-mon[30563]: pgmap v680: 129 pgs: 129 active+clean; 928 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2024-09-10T08:17:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.580+0000 7f90d6000640 1 -- 172.21.15.2:0/2208822331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90d0104f50 msgr2=0x7f90d01053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.580+0000 7f90d6000640 1 --2- 172.21.15.2:0/2208822331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90d0104f50 0x7f90d01053d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009900 tx=0x7f90bc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.580+0000 7f90d6000640 1 -- 172.21.15.2:0/2208822331 shutdown_connections 2024-09-10T08:17:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.580+0000 7f90d6000640 1 --2- 172.21.15.2:0/2208822331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90d0105910 0x7f90d010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.580+0000 7f90d6000640 1 --2- 172.21.15.2:0/2208822331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90d0104f50 0x7f90d01053d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.580+0000 7f90d6000640 1 --2- 172.21.15.2:0/2208822331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90d0103d60 0x7f90d0104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.580+0000 7f90d6000640 1 -- 172.21.15.2:0/2208822331 >> 172.21.15.2:0/2208822331 conn(0x7f90d00ff4e0 msgr2=0x7f90d0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.580+0000 7f90d6000640 1 -- 172.21.15.2:0/2208822331 shutdown_connections 2024-09-10T08:17:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.580+0000 7f90d6000640 1 -- 172.21.15.2:0/2208822331 wait complete. 2024-09-10T08:17:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.581+0000 7f90d6000640 1 Processor -- start 2024-09-10T08:17:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.581+0000 7f90d6000640 1 -- start start 2024-09-10T08:17:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90d6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90d0103d60 0x7f90d0199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90d6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90d0104f50 0x7f90d019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90d6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90d0105910 0x7f90d01a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90d6000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f90d010fa30 con 0x7f90d0103d60 2024-09-10T08:17:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90d6000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90d010f8b0 con 0x7f90d0104f50 2024-09-10T08:17:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90d6000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f90d010fbb0 con 0x7f90d0105910 2024-09-10T08:17:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90cf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90d0103d60 0x7f90d0199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90cf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90d0103d60 0x7f90d0199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58034/0 (socket says 172.21.15.2:58034) 2024-09-10T08:17:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90cf400640 1 -- 172.21.15.2:0/1188003846 learned_addr learned my addr 172.21.15.2:0/1188003846 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90cea00640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90d0104f50 0x7f90d019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90cfe00640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90d0105910 0x7f90d01a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90cf400640 1 -- 172.21.15.2:0/1188003846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90d0105910 msgr2=0x7f90d01a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90cf400640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90d0105910 0x7f90d01a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90cf400640 1 -- 172.21.15.2:0/1188003846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90d0104f50 msgr2=0x7f90d019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90cf400640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90d0104f50 0x7f90d019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.582+0000 7f90cf400640 1 -- 172.21.15.2:0/1188003846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d01a1bc0 con 0x7f90d0103d60 2024-09-10T08:17:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.583+0000 7f90cf400640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90d0103d60 0x7f90d0199fa0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f90c000e9c0 tx=0x7f90c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.583+0000 7f90c7e00640 1 -- 172.21.15.2:0/1188003846 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c000cd80 con 0x7f90d0103d60 2024-09-10T08:17:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.583+0000 7f90c7e00640 1 -- 172.21.15.2:0/1188003846 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c00035f0 con 0x7f90d0103d60 2024-09-10T08:17:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.583+0000 7f90d6000640 1 -- 172.21.15.2:0/1188003846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d01a1eb0 con 0x7f90d0103d60 2024-09-10T08:17:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.583+0000 7f90c7e00640 1 -- 172.21.15.2:0/1188003846 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0010690 con 0x7f90d0103d60 2024-09-10T08:17:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.583+0000 7f90d6000640 1 -- 172.21.15.2:0/1188003846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d01a2310 con 0x7f90d0103d60 2024-09-10T08:17:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.586+0000 7f90d8dbc740 1 Processor -- start 2024-09-10T08:17:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.586+0000 7f90c7e00640 1 -- 172.21.15.2:0/1188003846 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f90c00037a0 con 0x7f90d0103d60 2024-09-10T08:17:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.587+0000 7f90d8dbc740 1 -- start start 2024-09-10T08:17:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.587+0000 7f90d8dbc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126221090 0x55d126221490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.587+0000 7f90d8dbc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d126226490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.587+0000 7f90d8dbc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126226a60 0x55d126228f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.587+0000 7f90d8dbc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d126186100 con 0x55d126226a60 2024-09-10T08:17:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.587+0000 7f90d8dbc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d12612e1b0 con 0x55d126221090 2024-09-10T08:17:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.587+0000 7f90d8dbc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1261820c0 con 0x55d1262219d0 2024-09-10T08:17:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.587+0000 7f90cea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126226a60 0x55d126228f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.587+0000 7f90cf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d126226490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.587+0000 7f90cea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126226a60 0x55d126228f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58048/0 (socket says 172.21.15.2:58048) 2024-09-10T08:17:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.587+0000 7f90cfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126221090 0x55d126221490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.587+0000 7f90cea00640 1 -- 172.21.15.2:0/4164626191 learned_addr learned my addr 172.21.15.2:0/4164626191 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.588+0000 7f90cea00640 1 -- 172.21.15.2:0/4164626191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 msgr2=0x55d126226490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.588+0000 7f90cea00640 1 --2- 172.21.15.2:0/4164626191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d126226490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.588+0000 7f90cea00640 1 -- 172.21.15.2:0/4164626191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126221090 msgr2=0x55d126221490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.588+0000 7f90cea00640 1 --2- 172.21.15.2:0/4164626191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126221090 0x55d126221490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.588+0000 7f90cea00640 1 -- 172.21.15.2:0/4164626191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d125e99210 con 0x55d126226a60 2024-09-10T08:17:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.588+0000 7f90cfe00640 1 --2- 172.21.15.2:0/4164626191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126221090 0x55d126221490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.588+0000 7f90cf400640 1 --2- 172.21.15.2:0/4164626191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d126226490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.588+0000 7f90cea00640 1 --2- 172.21.15.2:0/4164626191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126226a60 0x55d126228f20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f90bc000c00 tx=0x7f90bc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dad915ac58fb087 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.588+0000 7f90c7e00640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f90b40a2c30 0x7f90b40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.589+0000 7f90c7e00640 1 -- 172.21.15.2:0/1188003846 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f90c00cf850 con 0x7f90d0103d60 2024-09-10T08:17:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.589+0000 7f90cfe00640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f90b40a2c30 0x7f90b40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.589+0000 7f90bbe00640 1 -- 172.21.15.2:0/4164626191 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc035040 con 0x55d126226a60 2024-09-10T08:17:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.589+0000 7f90bbe00640 1 -- 172.21.15.2:0/4164626191 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90bc005520 con 0x55d126226a60 2024-09-10T08:17:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.589+0000 7f90bbe00640 1 -- 172.21.15.2:0/4164626191 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc0316b0 con 0x55d126226a60 2024-09-10T08:17:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.589+0000 7f90d8dbc740 1 -- 172.21.15.2:0/4164626191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126226a60 msgr2=0x55d126228f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.589+0000 7f90cfe00640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f90b40a2c30 0x7f90b40a50f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009fd0 tx=0x7f90c8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.589+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/4164626191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126226a60 0x55d126228f20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f90bc000c00 tx=0x7f90bc004040 comp rx=0 tx=0).stop 2024-09-10T08:17:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.590+0000 7f90d8dbc740 1 -- 172.21.15.2:0/4164626191 shutdown_connections 2024-09-10T08:17:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.590+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/4164626191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126226a60 0x55d126228f20 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.590+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/4164626191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d126226490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.590+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/4164626191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126221090 0x55d126221490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.590+0000 7f90d8dbc740 1 -- 172.21.15.2:0/4164626191 >> 172.21.15.2:0/4164626191 conn(0x55d12621c700 msgr2=0x55d12621cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.590+0000 7f90d8dbc740 1 -- 172.21.15.2:0/4164626191 shutdown_connections 2024-09-10T08:17:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.590+0000 7f90d8dbc740 1 -- 172.21.15.2:0/4164626191 wait complete. 2024-09-10T08:17:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.590+0000 7f90d8dbc740 1 -- start start 2024-09-10T08:17:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.590+0000 7f90d8dbc740 1 Processor -- start 2024-09-10T08:17:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.591+0000 7f90d8dbc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d126221090 0x55d12622b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.591+0000 7f90d8dbc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1262219d0 0x55d12622b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.591+0000 7f90d8dbc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126226a60 0x55d126232760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.591+0000 7f90d8dbc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d1262299d0 con 0x55d1262219d0 2024-09-10T08:17:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.591+0000 7f90d8dbc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d12610f090 con 0x55d126226a60 2024-09-10T08:17:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.591+0000 7f90d8dbc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d126229b50 con 0x55d126221090 2024-09-10T08:17:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.591+0000 7f90d8dbc740 10 client.0 _ll_register_callbacks cb 0x55d126163f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.591+0000 7f90cea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1262219d0 0x55d12622b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.591+0000 7f90cea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1262219d0 0x55d12622b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58058/0 (socket says 172.21.15.2:58058) 2024-09-10T08:17:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.591+0000 7f90cea00640 1 -- 172.21.15.2:0/3828413239 learned_addr learned my addr 172.21.15.2:0/3828413239 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.591+0000 7f90cf400640 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d126221090 0x55d12622b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.591+0000 7f90cfe00640 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126226a60 0x55d126232760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90cf400640 1 -- 172.21.15.2:0/3828413239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126226a60 msgr2=0x55d126232760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90cf400640 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126226a60 0x55d126232760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90cf400640 1 -- 172.21.15.2:0/3828413239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1262219d0 msgr2=0x55d12622b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90cf400640 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1262219d0 0x55d12622b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90cf400640 1 -- 172.21.15.2:0/3828413239 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d125f65620 con 0x55d126221090 2024-09-10T08:17:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90cea00640 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1262219d0 0x55d12622b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90cf400640 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d126221090 0x55d12622b1e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55d126222400 tx=0x7f90c00978a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43077b78348db46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:17:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90b8c00640 1 -- 172.21.15.2:0/3828413239 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c009b5c0 con 0x55d126221090 2024-09-10T08:17:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90d8dbc740 1 -- 172.21.15.2:0/3828413239 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90bc005520 con 0x55d126221090 2024-09-10T08:17:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90b8c00640 1 -- 172.21.15.2:0/3828413239 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f90c0003b30 con 0x55d126221090 2024-09-10T08:17:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90b8c00640 1 -- 172.21.15.2:0/3828413239 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f90c00b9050 con 0x55d126221090 2024-09-10T08:17:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90d8dbc740 10 client.28111 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.592+0000 7f90d8dbc740 1 -- 172.21.15.2:0/3828413239 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d126235610 con 0x55d126221090 2024-09-10T08:17:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.593+0000 7f90b8c00640 1 -- 172.21.15.2:0/3828413239 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90bc005520 con 0x55d126221090 2024-09-10T08:17:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.593+0000 7f90b8c00640 1 -- 172.21.15.2:0/3828413239 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f90c009b140 con 0x55d126221090 2024-09-10T08:17:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.593+0000 7f90d8dbc740 10 client.28111 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.593+0000 7f90d8dbc740 10 client.28111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.593+0000 7f90d8dbc740 20 client.28111 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.593+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d126236c40 0x55d126239030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.593+0000 7f90d8dbc740 4 client.28111 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:17:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.593+0000 7f90d8dbc740 1 -- 172.21.15.2:0/3828413239 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d126193830 con 0x55d126236c40 2024-09-10T08:17:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.593+0000 7f90cea00640 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d126236c40 0x55d126239030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.594+0000 7f90cea00640 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d126236c40 0x55d126239030 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c9d2b8412b57bef server_cookie=299d0a3a63a19265 in_seq=0 out_seq=0 2024-09-10T08:17:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.594+0000 7f90b8c00640 10 client.28111 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90b8c00640 1 -- 172.21.15.2:0/3828413239 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d126193830 con 0x55d126236c40 2024-09-10T08:17:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90b8c00640 10 client.28111 handle_command_reply: tid=0 2024-09-10T08:17:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90d8dbc740 1 client.28111 shutdown 2024-09-10T08:17:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90d8dbc740 1 -- 172.21.15.2:0/3828413239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d126221090 msgr2=0x55d12622b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d126221090 0x55d12622b1e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55d126222400 tx=0x7f90c00978a0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90d8dbc740 1 -- 172.21.15.2:0/3828413239 shutdown_connections 2024-09-10T08:17:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d126236c40 0x55d126239030 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126226a60 0x55d126232760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1262219d0 0x55d12622b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/3828413239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d126221090 0x55d12622b1e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90b8c00640 0 client.28111 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90d8dbc740 1 -- 172.21.15.2:0/3828413239 >> 172.21.15.2:0/3828413239 conn(0x55d12621c700 msgr2=0x55d12621df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90d8dbc740 1 -- 172.21.15.2:0/3828413239 shutdown_connections 2024-09-10T08:17:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90d8dbc740 1 -- 172.21.15.2:0/3828413239 wait complete. 2024-09-10T08:17:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.595+0000 7f90d8dbc740 20 client.28111 trim_cache size 0 max 16384 2024-09-10T08:17:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.611+0000 7f90d8dbc740 1 Processor -- start 2024-09-10T08:17:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.611+0000 7f90d8dbc740 1 -- start start 2024-09-10T08:17:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.611+0000 7f90d8dbc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126221090 0x55d12622a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.611+0000 7f90d8dbc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d126232720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.611+0000 7f90d8dbc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126226a60 0x55d12621cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.611+0000 7f90d8dbc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01af2d0 con 0x55d126226a60 2024-09-10T08:17:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.611+0000 7f90d8dbc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01acba0 con 0x55d126221090 2024-09-10T08:17:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.611+0000 7f90d8dbc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01aa9f0 con 0x55d1262219d0 2024-09-10T08:17:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.611+0000 7f90cfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126226a60 0x55d12621cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.611+0000 7f90cf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126221090 0x55d12622a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.611+0000 7f90cf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126221090 0x55d12622a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36646/0 (socket says 172.21.15.2:36646) 2024-09-10T08:17:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.611+0000 7f90cf400640 1 -- 172.21.15.2:0/1371527376 learned_addr learned my addr 172.21.15.2:0/1371527376 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.611+0000 7f90cea00640 1 --2- 172.21.15.2:0/1371527376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d126232720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.612+0000 7f90cf400640 1 -- 172.21.15.2:0/1371527376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 msgr2=0x55d126232720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:56.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.612+0000 7f90cf400640 1 --2- 172.21.15.2:0/1371527376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d126232720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.612+0000 7f90cf400640 1 -- 172.21.15.2:0/1371527376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126226a60 msgr2=0x55d12621cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:56.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.612+0000 7f90cf400640 1 --2- 172.21.15.2:0/1371527376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126226a60 0x55d12621cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.612+0000 7f90cf400640 1 -- 172.21.15.2:0/1371527376 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d12622a250 con 0x55d126221090 2024-09-10T08:17:56.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.612+0000 7f90cf400640 1 --2- 172.21.15.2:0/1371527376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126221090 0x55d12622a7e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f90c009bbf0 tx=0x7f90c007abc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3342c095fa110ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:56.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.612+0000 7f90bb400640 1 -- 172.21.15.2:0/1371527376 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00b4e60 con 0x55d126221090 2024-09-10T08:17:56.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.612+0000 7f90bb400640 1 -- 172.21.15.2:0/1371527376 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c00b1020 con 0x55d126221090 2024-09-10T08:17:56.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.612+0000 7f90bb400640 1 -- 172.21.15.2:0/1371527376 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00a48d0 con 0x55d126221090 2024-09-10T08:17:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.613+0000 7f90d8dbc740 1 -- 172.21.15.2:0/1371527376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126221090 msgr2=0x55d12622a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.613+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/1371527376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126221090 0x55d12622a7e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f90c009bbf0 tx=0x7f90c007abc0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.613+0000 7f90d8dbc740 1 -- 172.21.15.2:0/1371527376 shutdown_connections 2024-09-10T08:17:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.613+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/1371527376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126226a60 0x55d12621cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.613+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/1371527376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d126232720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.613+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/1371527376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126221090 0x55d12622a7e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.613+0000 7f90d8dbc740 1 -- 172.21.15.2:0/1371527376 >> 172.21.15.2:0/1371527376 conn(0x55d12621c700 msgr2=0x55d12622b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.613+0000 7f90d8dbc740 1 -- 172.21.15.2:0/1371527376 shutdown_connections 2024-09-10T08:17:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.613+0000 7f90d8dbc740 1 -- 172.21.15.2:0/1371527376 wait complete. 2024-09-10T08:17:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.613+0000 7f90d8dbc740 1 -- start start 2024-09-10T08:17:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.613+0000 7f90d8dbc740 1 Processor -- start 2024-09-10T08:17:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90d8dbc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126221090 0x55d12622f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90d8dbc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d12622fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90d8dbc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126226a60 0x55d12629f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90d8dbc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01ab060 con 0x55d126221090 2024-09-10T08:17:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90d8dbc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01ac4d0 con 0x55d126226a60 2024-09-10T08:17:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90d8dbc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01ab730 con 0x55d1262219d0 2024-09-10T08:17:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90d8dbc740 10 client.0 _ll_register_callbacks cb 0x7f90c00d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:17:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90cf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126221090 0x55d12622f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90cf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126221090 0x55d12622f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58068/0 (socket says 172.21.15.2:58068) 2024-09-10T08:17:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90cf400640 1 -- 172.21.15.2:0/2857394151 learned_addr learned my addr 172.21.15.2:0/2857394151 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:17:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90cea00640 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d12622fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90cfe00640 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126226a60 0x55d12629f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90cf400640 1 -- 172.21.15.2:0/2857394151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 msgr2=0x55d12622fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90cf400640 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d12622fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90cf400640 1 -- 172.21.15.2:0/2857394151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126226a60 msgr2=0x55d12629f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:17:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90cf400640 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126226a60 0x55d12629f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90cf400640 1 -- 172.21.15.2:0/2857394151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d12629fe60 con 0x55d126221090 2024-09-10T08:17:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.614+0000 7f90cea00640 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d12622fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:17:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.615+0000 7f90cf400640 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126221090 0x55d12622f5f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f90c00cfc10 tx=0x7f90c0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5d58b9cb66aad1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:17:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.615+0000 7f90b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:17:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.615+0000 7f90b8c00640 1 -- 172.21.15.2:0/2857394151 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00b4e60 con 0x55d126221090 2024-09-10T08:17:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.615+0000 7f90b8c00640 1 -- 172.21.15.2:0/2857394151 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f90c0097bc0 con 0x55d126221090 2024-09-10T08:17:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.615+0000 7f90d8dbc740 1 -- 172.21.15.2:0/2857394151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c00b1020 con 0x55d126221090 2024-09-10T08:17:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.615+0000 7f90b8c00640 1 -- 172.21.15.2:0/2857394151 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f90c009b5c0 con 0x55d126221090 2024-09-10T08:17:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.615+0000 7f90d8dbc740 10 client.18150 fetch_fsmap learned FSMap version 26 2024-09-10T08:17:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.615+0000 7f90d8dbc740 1 -- 172.21.15.2:0/2857394151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d1262a1030 con 0x55d126221090 2024-09-10T08:17:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.615+0000 7f90b8c00640 1 -- 172.21.15.2:0/2857394151 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c00b1020 con 0x55d126221090 2024-09-10T08:17:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.616+0000 7f90b8c00640 1 -- 172.21.15.2:0/2857394151 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f90c00ba240 con 0x55d126221090 2024-09-10T08:17:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.616+0000 7f90d8dbc740 10 client.18150 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:17:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.616+0000 7f90d8dbc740 10 client.18150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:17:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.616+0000 7f90d8dbc740 20 client.18150 populate_metadata read hostname 'smithi002' 2024-09-10T08:17:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.616+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d1262a1770 0x55d1262a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:17:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.616+0000 7f90d8dbc740 4 client.18150 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:17:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.616+0000 7f90d8dbc740 1 -- 172.21.15.2:0/2857394151 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d12618db80 con 0x55d1262a1770 2024-09-10T08:17:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.616+0000 7f90cea00640 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d1262a1770 0x55d1262a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:17:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.616+0000 7f90cea00640 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d1262a1770 0x55d1262a3b90 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e43847d4c52f2a9 server_cookie=8f7a4d250ed658d in_seq=0 out_seq=0 2024-09-10T08:17:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.616+0000 7f90b8c00640 10 client.18150 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.628+0000 7f90b8c00640 1 -- 172.21.15.2:0/2857394151 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d12618db80 con 0x55d1262a1770 2024-09-10T08:17:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.628+0000 7f90b8c00640 10 client.18150 handle_command_reply: tid=0 2024-09-10T08:17:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.628+0000 7f90d8dbc740 1 client.18150 shutdown 2024-09-10T08:17:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.628+0000 7f90d8dbc740 1 -- 172.21.15.2:0/2857394151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126221090 msgr2=0x55d12622f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.628+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126221090 0x55d12622f5f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f90c00cfc10 tx=0x7f90c0003d40 comp rx=0 tx=0).stop 2024-09-10T08:17:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.628+0000 7f90d8dbc740 1 -- 172.21.15.2:0/2857394151 shutdown_connections 2024-09-10T08:17:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.628+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d1262a1770 0x55d1262a3b90 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.629+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d126226a60 0x55d12629f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.629+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1262219d0 0x55d12622fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.629+0000 7f90b8c00640 0 client.18150 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:17:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.629+0000 7f90d8dbc740 1 --2- 172.21.15.2:0/2857394151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d126221090 0x55d12622f5f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.629+0000 7f90d8dbc740 1 -- 172.21.15.2:0/2857394151 >> 172.21.15.2:0/2857394151 conn(0x55d12621c700 msgr2=0x55d12621b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.629+0000 7f90d8dbc740 1 -- 172.21.15.2:0/2857394151 shutdown_connections 2024-09-10T08:17:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.629+0000 7f90d8dbc740 1 -- 172.21.15.2:0/2857394151 wait complete. 2024-09-10T08:17:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.629+0000 7f90d8dbc740 20 client.18150 trim_cache size 0 max 16384 2024-09-10T08:17:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:17:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:17:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:17:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:17:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.629+0000 7f90d6000640 1 -- 172.21.15.2:0/1188003846 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f90b40a2c30 msgr2=0x7f90b40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.629+0000 7f90d6000640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f90b40a2c30 0x7f90b40a50f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009fd0 tx=0x7f90c8009500 comp rx=0 tx=0).stop 2024-09-10T08:17:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.629+0000 7f90d6000640 1 -- 172.21.15.2:0/1188003846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90d0103d60 msgr2=0x7f90d0199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:17:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.629+0000 7f90d6000640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90d0103d60 0x7f90d0199fa0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f90c000e9c0 tx=0x7f90c000ee80 comp rx=0 tx=0).stop 2024-09-10T08:17:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.630+0000 7f90d6000640 1 -- 172.21.15.2:0/1188003846 shutdown_connections 2024-09-10T08:17:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.630+0000 7f90d6000640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f90b40a2c30 0x7f90b40a50f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.630+0000 7f90d6000640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90d0105910 0x7f90d01a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.630+0000 7f90d6000640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90d0104f50 0x7f90d019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.630+0000 7f90d6000640 1 --2- 172.21.15.2:0/1188003846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90d0103d60 0x7f90d0199fa0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:17:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.630+0000 7f90d6000640 1 -- 172.21.15.2:0/1188003846 >> 172.21.15.2:0/1188003846 conn(0x7f90d00ff4e0 msgr2=0x7f90d0108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:17:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.630+0000 7f90d6000640 1 -- 172.21.15.2:0/1188003846 shutdown_connections 2024-09-10T08:17:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:17:56.630+0000 7f90d6000640 1 -- 172.21.15.2:0/1188003846 wait complete. 2024-09-10T08:17:57.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:57 smithi155 ceph-mon[35405]: pgmap v681: 129 pgs: 129 active+clean; 928 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-09-10T08:17:57.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:57 smithi107 ceph-mon[35048]: pgmap v681: 129 pgs: 129 active+clean; 928 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-09-10T08:17:57.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:57 smithi002 ceph-mon[30563]: pgmap v681: 129 pgs: 129 active+clean; 928 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-09-10T08:17:59.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:17:59 smithi155 ceph-mon[35405]: pgmap v682: 129 pgs: 129 active+clean; 991 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:17:59.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:17:59 smithi107 ceph-mon[35048]: pgmap v682: 129 pgs: 129 active+clean; 991 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:17:59.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:17:59 smithi002 ceph-mon[30563]: pgmap v682: 129 pgs: 129 active+clean; 991 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:18:01.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:01 smithi155 ceph-mon[35405]: pgmap v683: 129 pgs: 129 active+clean; 991 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-10T08:18:01.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:01 smithi107 ceph-mon[35048]: pgmap v683: 129 pgs: 129 active+clean; 991 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-10T08:18:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:01 smithi002 ceph-mon[30563]: pgmap v683: 129 pgs: 129 active+clean; 991 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-10T08:18:03.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:03 smithi155 ceph-mon[35405]: pgmap v684: 129 pgs: 129 active+clean; 991 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:18:03.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:03 smithi107 ceph-mon[35048]: pgmap v684: 129 pgs: 129 active+clean; 991 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:18:03.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:03 smithi002 ceph-mon[30563]: pgmap v684: 129 pgs: 129 active+clean; 991 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:18:05.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:05 smithi155 ceph-mon[35405]: pgmap v685: 129 pgs: 129 active+clean; 1.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:18:05.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:05 smithi107 ceph-mon[35048]: pgmap v685: 129 pgs: 129 active+clean; 1.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:18:05.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:05 smithi002 ceph-mon[30563]: pgmap v685: 129 pgs: 129 active+clean; 1.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:18:07.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:07 smithi155 ceph-mon[35405]: pgmap v686: 129 pgs: 129 active+clean; 1.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:18:07.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:07 smithi107 ceph-mon[35048]: pgmap v686: 129 pgs: 129 active+clean; 1.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:18:07.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:07 smithi002 ceph-mon[30563]: pgmap v686: 129 pgs: 129 active+clean; 1.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:18:09.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:09 smithi155 ceph-mon[35405]: pgmap v687: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:18:09.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:09 smithi107 ceph-mon[35048]: pgmap v687: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:18:09.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:09 smithi002 ceph-mon[30563]: pgmap v687: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:18:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.757+0000 7faf7dc00640 1 -- 172.21.15.2:0/3113516313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf78078790 msgr2=0x7faf78079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.757+0000 7faf7dc00640 1 --2- 172.21.15.2:0/3113516313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf78078790 0x7faf78079030 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7faf70009ca0 tx=0x7faf7002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.758+0000 7faf7dc00640 1 -- 172.21.15.2:0/3113516313 shutdown_connections 2024-09-10T08:18:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.758+0000 7faf7dc00640 1 --2- 172.21.15.2:0/3113516313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf78078790 0x7faf78079030 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.758+0000 7faf7dc00640 1 --2- 172.21.15.2:0/3113516313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf78077df0 0x7faf78078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.758+0000 7faf7dc00640 1 --2- 172.21.15.2:0/3113516313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf78076bb0 0x7faf78076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.758+0000 7faf7dc00640 1 -- 172.21.15.2:0/3113516313 >> 172.21.15.2:0/3113516313 conn(0x7faf780ff4e0 msgr2=0x7faf78101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.758+0000 7faf7dc00640 1 -- 172.21.15.2:0/3113516313 shutdown_connections 2024-09-10T08:18:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.758+0000 7faf7dc00640 1 -- 172.21.15.2:0/3113516313 wait complete. 2024-09-10T08:18:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.759+0000 7faf7dc00640 1 Processor -- start 2024-09-10T08:18:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.759+0000 7faf7dc00640 1 -- start start 2024-09-10T08:18:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.759+0000 7faf7dc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf78076bb0 0x7faf7819e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.759+0000 7faf7dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf78077df0 0x7faf7819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.759+0000 7faf7dc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf78078790 0x7faf781a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.759+0000 7faf7dc00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf78113e30 con 0x7faf78076bb0 2024-09-10T08:18:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.759+0000 7faf7dc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf78113cb0 con 0x7faf78077df0 2024-09-10T08:18:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.759+0000 7faf7dc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf78113fb0 con 0x7faf78078790 2024-09-10T08:18:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.759+0000 7faf76a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf78076bb0 0x7faf7819e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.759+0000 7faf76a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf78076bb0 0x7faf7819e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35518/0 (socket says 172.21.15.2:35518) 2024-09-10T08:18:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.759+0000 7faf76a00640 1 -- 172.21.15.2:0/2116049035 learned_addr learned my addr 172.21.15.2:0/2116049035 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.759+0000 7faf6fe00640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf78077df0 0x7faf7819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.760+0000 7faf77400640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf78078790 0x7faf781a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.760+0000 7faf76a00640 1 -- 172.21.15.2:0/2116049035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf78078790 msgr2=0x7faf781a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.760+0000 7faf76a00640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf78078790 0x7faf781a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.760+0000 7faf76a00640 1 -- 172.21.15.2:0/2116049035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf78077df0 msgr2=0x7faf7819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.760+0000 7faf76a00640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf78077df0 0x7faf7819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.760+0000 7faf76a00640 1 -- 172.21.15.2:0/2116049035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf781a5f80 con 0x7faf78076bb0 2024-09-10T08:18:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.760+0000 7faf76a00640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf78076bb0 0x7faf7819e360 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7faf6800e9f0 tx=0x7faf6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.761+0000 7faf6f400640 1 -- 172.21.15.2:0/2116049035 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6800cd80 con 0x7faf78076bb0 2024-09-10T08:18:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.761+0000 7faf6f400640 1 -- 172.21.15.2:0/2116049035 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf68005500 con 0x7faf78076bb0 2024-09-10T08:18:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.761+0000 7faf7dc00640 1 -- 172.21.15.2:0/2116049035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf781a6270 con 0x7faf78076bb0 2024-09-10T08:18:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.761+0000 7faf6f400640 1 -- 172.21.15.2:0/2116049035 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf68010690 con 0x7faf78076bb0 2024-09-10T08:18:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.761+0000 7faf7dc00640 1 -- 172.21.15.2:0/2116049035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf781a6770 con 0x7faf78076bb0 2024-09-10T08:18:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.764+0000 7faf6f400640 1 -- 172.21.15.2:0/2116049035 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7faf680046c0 con 0x7faf78076bb0 2024-09-10T08:18:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.764+0000 7faf80be6740 1 Processor -- start 2024-09-10T08:18:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.764+0000 7faf80be6740 1 -- start start 2024-09-10T08:18:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.764+0000 7faf80be6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1a2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.764+0000 7faf80be6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a29d0 0x56093e1a7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.764+0000 7faf80be6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e1a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.764+0000 7faf80be6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56093e107100 con 0x56093e1a2090 2024-09-10T08:18:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.764+0000 7faf80be6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56093e0af1b0 con 0x56093e1a7a60 2024-09-10T08:18:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.764+0000 7faf80be6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56093e1030c0 con 0x56093e1a29d0 2024-09-10T08:18:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.765+0000 7faf77400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.765+0000 7faf6fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e1a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.765+0000 7faf6fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e1a9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:53546/0 (socket says 172.21.15.2:53546) 2024-09-10T08:18:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.765+0000 7faf6fe00640 1 -- 172.21.15.2:0/2341482642 learned_addr learned my addr 172.21.15.2:0/2341482642 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.765+0000 7faf76a00640 1 --2- 172.21.15.2:0/2341482642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a29d0 0x56093e1a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.765+0000 7faf6fe00640 1 -- 172.21.15.2:0/2341482642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a29d0 msgr2=0x56093e1a7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.765+0000 7faf6fe00640 1 --2- 172.21.15.2:0/2341482642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a29d0 0x56093e1a7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.765+0000 7faf6fe00640 1 -- 172.21.15.2:0/2341482642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 msgr2=0x56093e1a2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.765+0000 7faf6fe00640 1 --2- 172.21.15.2:0/2341482642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1a2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.765+0000 7faf6fe00640 1 -- 172.21.15.2:0/2341482642 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56093de1a210 con 0x56093e1a7a60 2024-09-10T08:18:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.765+0000 7faf77400640 1 --2- 172.21.15.2:0/2341482642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1a2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.765+0000 7faf6f400640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf5c0a2c30 0x7faf5c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.765+0000 7faf6fe00640 1 --2- 172.21.15.2:0/2341482642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e1a9f20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faf6000bc90 tx=0x7faf600107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68e1a17bc6dc9406 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.766+0000 7faf6f400640 1 -- 172.21.15.2:0/2116049035 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faf680cf870 con 0x7faf78076bb0 2024-09-10T08:18:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.766+0000 7faf67e00640 1 -- 172.21.15.2:0/2341482642 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60005220 con 0x56093e1a7a60 2024-09-10T08:18:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.766+0000 7faf67e00640 1 -- 172.21.15.2:0/2341482642 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf6000be40 con 0x56093e1a7a60 2024-09-10T08:18:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.766+0000 7faf67e00640 1 -- 172.21.15.2:0/2341482642 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60011730 con 0x56093e1a7a60 2024-09-10T08:18:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.766+0000 7faf77400640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf5c0a2c30 0x7faf5c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.766+0000 7faf80be6740 1 -- 172.21.15.2:0/2341482642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 msgr2=0x56093e1a9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.766+0000 7faf80be6740 1 --2- 172.21.15.2:0/2341482642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e1a9f20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faf6000bc90 tx=0x7faf600107a0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.766+0000 7faf77400640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf5c0a2c30 0x7faf5c0a50f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7faf70005e30 tx=0x7faf7003d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.767+0000 7faf80be6740 1 -- 172.21.15.2:0/2341482642 shutdown_connections 2024-09-10T08:18:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.767+0000 7faf80be6740 1 --2- 172.21.15.2:0/2341482642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e1a9f20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.767+0000 7faf80be6740 1 --2- 172.21.15.2:0/2341482642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a29d0 0x56093e1a7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.767+0000 7faf80be6740 1 --2- 172.21.15.2:0/2341482642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1a2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.767+0000 7faf80be6740 1 -- 172.21.15.2:0/2341482642 >> 172.21.15.2:0/2341482642 conn(0x56093e19d700 msgr2=0x56093e19db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.767+0000 7faf80be6740 1 -- 172.21.15.2:0/2341482642 shutdown_connections 2024-09-10T08:18:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.767+0000 7faf80be6740 1 -- 172.21.15.2:0/2341482642 wait complete. 2024-09-10T08:18:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.767+0000 7faf80be6740 1 -- start start 2024-09-10T08:18:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.767+0000 7faf80be6740 1 Processor -- start 2024-09-10T08:18:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.768+0000 7faf80be6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1ac210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.768+0000 7faf80be6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a29d0 0x56093e1ac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.768+0000 7faf80be6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a7a60 0x56093e1b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.768+0000 7faf80be6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56093e1aaa30 con 0x56093e1a2090 2024-09-10T08:18:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.768+0000 7faf80be6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56093e090090 con 0x56093e1a29d0 2024-09-10T08:18:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.768+0000 7faf80be6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56093e1aabb0 con 0x56093e1a7a60 2024-09-10T08:18:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.768+0000 7faf80be6740 10 client.0 _ll_register_callbacks cb 0x56093e0e4f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.768+0000 7faf76a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.768+0000 7faf76a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1ac210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35526/0 (socket says 172.21.15.2:35526) 2024-09-10T08:18:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.768+0000 7faf76a00640 1 -- 172.21.15.2:0/3315742948 learned_addr learned my addr 172.21.15.2:0/3315742948 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.768+0000 7faf6fe00640 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a29d0 0x56093e1ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.768+0000 7faf77400640 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a7a60 0x56093e1b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf76a00640 1 -- 172.21.15.2:0/3315742948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a7a60 msgr2=0x56093e1b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf76a00640 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a7a60 0x56093e1b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf76a00640 1 -- 172.21.15.2:0/3315742948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a29d0 msgr2=0x56093e1ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf76a00640 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a29d0 0x56093e1ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf76a00640 1 -- 172.21.15.2:0/3315742948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56093dee6620 con 0x56093e1a2090 2024-09-10T08:18:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf6fe00640 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a29d0 0x56093e1ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf76a00640 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1ac210 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7faf6809bc10 tx=0x7faf68005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c512ec81a9ecb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf64c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf64c00640 1 -- 172.21.15.2:0/3315742948 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6809b5e0 con 0x56093e1a2090 2024-09-10T08:18:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf64c00640 1 -- 172.21.15.2:0/3315742948 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faf680a4660 con 0x56093e1a2090 2024-09-10T08:18:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf80be6740 1 -- 172.21.15.2:0/3315742948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56093e114830 con 0x56093e1a2090 2024-09-10T08:18:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf64c00640 1 -- 172.21.15.2:0/3315742948 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faf6809c380 con 0x56093e1a2090 2024-09-10T08:18:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf80be6740 10 client.18162 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.769+0000 7faf80be6740 1 -- 172.21.15.2:0/3315742948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56093e1b66a0 con 0x56093e1a2090 2024-09-10T08:18:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.770+0000 7faf64c00640 1 -- 172.21.15.2:0/3315742948 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56093e114830 con 0x56093e1a2090 2024-09-10T08:18:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.770+0000 7faf64c00640 1 -- 172.21.15.2:0/3315742948 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faf6809ab40 con 0x56093e1a2090 2024-09-10T08:18:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.770+0000 7faf80be6740 10 client.18162 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.770+0000 7faf80be6740 10 client.18162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.770+0000 7faf80be6740 20 client.18162 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.770+0000 7faf80be6740 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56093e1b7cd0 0x56093e1ba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.770+0000 7faf80be6740 4 client.18162 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:18:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.770+0000 7faf80be6740 1 -- 172.21.15.2:0/3315742948 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56093e10eb80 con 0x56093e1b7cd0 2024-09-10T08:18:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.771+0000 7faf6fe00640 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56093e1b7cd0 0x56093e1ba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.771+0000 7faf6fe00640 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56093e1b7cd0 0x56093e1ba0c0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb66f959dd8d119 server_cookie=4fa5b79a52dc7a75 in_seq=0 out_seq=0 2024-09-10T08:18:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.771+0000 7faf64c00640 10 client.18162 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.772+0000 7faf64c00640 1 -- 172.21.15.2:0/3315742948 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56093e10eb80 con 0x56093e1b7cd0 2024-09-10T08:18:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.772+0000 7faf64c00640 10 client.18162 handle_command_reply: tid=0 2024-09-10T08:18:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.772+0000 7faf80be6740 1 client.18162 shutdown 2024-09-10T08:18:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.773+0000 7faf80be6740 1 -- 172.21.15.2:0/3315742948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 msgr2=0x56093e1ac210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.773+0000 7faf80be6740 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1ac210 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7faf6809bc10 tx=0x7faf68005730 comp rx=0 tx=0).stop 2024-09-10T08:18:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.773+0000 7faf80be6740 1 -- 172.21.15.2:0/3315742948 shutdown_connections 2024-09-10T08:18:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.773+0000 7faf80be6740 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56093e1b7cd0 0x56093e1ba0c0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.773+0000 7faf80be6740 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a7a60 0x56093e1b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.773+0000 7faf80be6740 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a29d0 0x56093e1ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.773+0000 7faf80be6740 1 --2- 172.21.15.2:0/3315742948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1ac210 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.773+0000 7faf80be6740 1 -- 172.21.15.2:0/3315742948 >> 172.21.15.2:0/3315742948 conn(0x56093e19d700 msgr2=0x56093e19ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.773+0000 7faf64c00640 0 client.18162 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.773+0000 7faf80be6740 1 -- 172.21.15.2:0/3315742948 shutdown_connections 2024-09-10T08:18:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.773+0000 7faf80be6740 1 -- 172.21.15.2:0/3315742948 wait complete. 2024-09-10T08:18:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.773+0000 7faf80be6740 20 client.18162 trim_cache size 0 max 16384 2024-09-10T08:18:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.788+0000 7faf80be6740 1 Processor -- start 2024-09-10T08:18:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.788+0000 7faf80be6740 1 -- start start 2024-09-10T08:18:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf80be6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf80be6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a29d0 0x56093e1b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf80be6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e1ac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf80be6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf781b36e0 con 0x56093e1a2090 2024-09-10T08:18:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf80be6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf781b0fb0 con 0x56093e1a7a60 2024-09-10T08:18:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf80be6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf781aee00 con 0x56093e1a29d0 2024-09-10T08:18:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf76a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf76a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1abfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35538/0 (socket says 172.21.15.2:35538) 2024-09-10T08:18:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf76a00640 1 -- 172.21.15.2:0/2208125713 learned_addr learned my addr 172.21.15.2:0/2208125713 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf6fe00640 1 --2- 172.21.15.2:0/2208125713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a29d0 0x56093e1b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf77400640 1 --2- 172.21.15.2:0/2208125713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e1ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf76a00640 1 -- 172.21.15.2:0/2208125713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a29d0 msgr2=0x56093e1b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf76a00640 1 --2- 172.21.15.2:0/2208125713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a29d0 0x56093e1b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf76a00640 1 -- 172.21.15.2:0/2208125713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 msgr2=0x56093e1ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf76a00640 1 --2- 172.21.15.2:0/2208125713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e1ac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.789+0000 7faf76a00640 1 -- 172.21.15.2:0/2208125713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56093e19d520 con 0x56093e1a2090 2024-09-10T08:18:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.790+0000 7faf76a00640 1 --2- 172.21.15.2:0/2208125713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1abfc0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faf6809b980 tx=0x7faf68009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7382b88b4878053 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.790+0000 7faf67400640 1 -- 172.21.15.2:0/2208125713 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6809b5e0 con 0x56093e1a2090 2024-09-10T08:18:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.790+0000 7faf67400640 1 -- 172.21.15.2:0/2208125713 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf680d5050 con 0x56093e1a2090 2024-09-10T08:18:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.790+0000 7faf67400640 1 -- 172.21.15.2:0/2208125713 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf680a4460 con 0x56093e1a2090 2024-09-10T08:18:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.790+0000 7faf80be6740 1 -- 172.21.15.2:0/2208125713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 msgr2=0x56093e1abfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.790+0000 7faf80be6740 1 --2- 172.21.15.2:0/2208125713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1abfc0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faf6809b980 tx=0x7faf68009760 comp rx=0 tx=0).stop 2024-09-10T08:18:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.790+0000 7faf80be6740 1 -- 172.21.15.2:0/2208125713 shutdown_connections 2024-09-10T08:18:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.790+0000 7faf80be6740 1 --2- 172.21.15.2:0/2208125713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e1ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.790+0000 7faf80be6740 1 --2- 172.21.15.2:0/2208125713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a29d0 0x56093e1b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.790+0000 7faf80be6740 1 --2- 172.21.15.2:0/2208125713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a2090 0x56093e1abfc0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.790+0000 7faf80be6740 1 -- 172.21.15.2:0/2208125713 >> 172.21.15.2:0/2208125713 conn(0x56093e19d700 msgr2=0x56093e1a3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.791+0000 7faf80be6740 1 -- 172.21.15.2:0/2208125713 shutdown_connections 2024-09-10T08:18:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.791+0000 7faf80be6740 1 -- 172.21.15.2:0/2208125713 wait complete. 2024-09-10T08:18:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.791+0000 7faf80be6740 1 -- start start 2024-09-10T08:18:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.791+0000 7faf80be6740 1 Processor -- start 2024-09-10T08:18:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf80be6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a2090 0x56093e1b0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf80be6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a29d0 0x56093e1b0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf80be6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e2209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf80be6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf781af470 con 0x56093e1a29d0 2024-09-10T08:18:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf80be6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf781b08e0 con 0x56093e1a7a60 2024-09-10T08:18:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf80be6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf781afb40 con 0x56093e1a2090 2024-09-10T08:18:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf80be6740 10 client.0 _ll_register_callbacks cb 0x7faf6800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf6fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a29d0 0x56093e1b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf6fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a29d0 0x56093e1b0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35552/0 (socket says 172.21.15.2:35552) 2024-09-10T08:18:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf77400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e2209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf6fe00640 1 -- 172.21.15.2:0/1136588400 learned_addr learned my addr 172.21.15.2:0/1136588400 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf76a00640 1 --2- 172.21.15.2:0/1136588400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a2090 0x56093e1b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf6fe00640 1 -- 172.21.15.2:0/1136588400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a2090 msgr2=0x56093e1b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf6fe00640 1 --2- 172.21.15.2:0/1136588400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a2090 0x56093e1b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf6fe00640 1 -- 172.21.15.2:0/1136588400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 msgr2=0x56093e2209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf6fe00640 1 --2- 172.21.15.2:0/1136588400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e2209e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.792+0000 7faf6fe00640 1 -- 172.21.15.2:0/1136588400 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56093e1b38f0 con 0x56093e1a29d0 2024-09-10T08:18:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.793+0000 7faf6fe00640 1 --2- 172.21.15.2:0/1136588400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a29d0 0x56093e1b0850 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x56093e19e580 tx=0x7faf6000b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcead49746c1c9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.793+0000 7faf64c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.793+0000 7faf64c00640 1 -- 172.21.15.2:0/1136588400 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6002c030 con 0x56093e1a29d0 2024-09-10T08:18:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.793+0000 7faf64c00640 1 -- 172.21.15.2:0/1136588400 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faf600055f0 con 0x56093e1a29d0 2024-09-10T08:18:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.793+0000 7faf80be6740 1 -- 172.21.15.2:0/1136588400 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf680d5050 con 0x56093e1a29d0 2024-09-10T08:18:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.793+0000 7faf64c00640 1 -- 172.21.15.2:0/1136588400 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faf60039050 con 0x56093e1a29d0 2024-09-10T08:18:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.793+0000 7faf80be6740 10 client.18168 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.793+0000 7faf80be6740 1 -- 172.21.15.2:0/1136588400 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56093e221e50 con 0x56093e1a29d0 2024-09-10T08:18:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.793+0000 7faf64c00640 1 -- 172.21.15.2:0/1136588400 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf680d5050 con 0x56093e1a29d0 2024-09-10T08:18:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.794+0000 7faf64c00640 1 -- 172.21.15.2:0/1136588400 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faf60011600 con 0x56093e1a29d0 2024-09-10T08:18:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.794+0000 7faf80be6740 10 client.18168 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.794+0000 7faf80be6740 10 client.18168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.794+0000 7faf80be6740 20 client.18168 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.794+0000 7faf80be6740 1 --2- 172.21.15.2:0/1136588400 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56093e2225a0 0x56093e2249c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.794+0000 7faf80be6740 4 client.18168 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:18:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.794+0000 7faf80be6740 1 -- 172.21.15.2:0/1136588400 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56093e08b7c0 con 0x56093e2225a0 2024-09-10T08:18:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.794+0000 7faf76a00640 1 --2- 172.21.15.2:0/1136588400 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56093e2225a0 0x56093e2249c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.795+0000 7faf76a00640 1 --2- 172.21.15.2:0/1136588400 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56093e2225a0 0x56093e2249c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a71e7074e27b693 server_cookie=47e3ddd5e263f233 in_seq=0 out_seq=0 2024-09-10T08:18:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.795+0000 7faf64c00640 10 client.18168 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.807+0000 7faf64c00640 1 -- 172.21.15.2:0/1136588400 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56093e08b7c0 con 0x56093e2225a0 2024-09-10T08:18:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.807+0000 7faf64c00640 10 client.18168 handle_command_reply: tid=0 2024-09-10T08:18:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.807+0000 7faf80be6740 1 client.18168 shutdown 2024-09-10T08:18:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.808+0000 7faf80be6740 1 -- 172.21.15.2:0/1136588400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a29d0 msgr2=0x56093e1b0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.808+0000 7faf80be6740 1 --2- 172.21.15.2:0/1136588400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a29d0 0x56093e1b0850 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x56093e19e580 tx=0x7faf6000b970 comp rx=0 tx=0).stop 2024-09-10T08:18:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.808+0000 7faf80be6740 1 -- 172.21.15.2:0/1136588400 shutdown_connections 2024-09-10T08:18:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.808+0000 7faf80be6740 1 --2- 172.21.15.2:0/1136588400 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56093e2225a0 0x56093e2249c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.808+0000 7faf80be6740 1 --2- 172.21.15.2:0/1136588400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56093e1a7a60 0x56093e2209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.808+0000 7faf80be6740 1 --2- 172.21.15.2:0/1136588400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56093e1a29d0 0x56093e1b0850 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.808+0000 7faf80be6740 1 --2- 172.21.15.2:0/1136588400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56093e1a2090 0x56093e1b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.808+0000 7faf80be6740 1 -- 172.21.15.2:0/1136588400 >> 172.21.15.2:0/1136588400 conn(0x56093e19d700 msgr2=0x56093e1abf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.808+0000 7faf64c00640 0 client.18168 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.808+0000 7faf80be6740 1 -- 172.21.15.2:0/1136588400 shutdown_connections 2024-09-10T08:18:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.808+0000 7faf80be6740 1 -- 172.21.15.2:0/1136588400 wait complete. 2024-09-10T08:18:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.808+0000 7faf80be6740 20 client.18168 trim_cache size 0 max 16384 2024-09-10T08:18:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:18:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:18:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:18:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:18:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.809+0000 7faf7dc00640 1 -- 172.21.15.2:0/2116049035 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf5c0a2c30 msgr2=0x7faf5c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.809+0000 7faf7dc00640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf5c0a2c30 0x7faf5c0a50f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7faf70005e30 tx=0x7faf7003d040 comp rx=0 tx=0).stop 2024-09-10T08:18:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.809+0000 7faf7dc00640 1 -- 172.21.15.2:0/2116049035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf78076bb0 msgr2=0x7faf7819e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.809+0000 7faf7dc00640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf78076bb0 0x7faf7819e360 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7faf6800e9f0 tx=0x7faf6800edf0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.809+0000 7faf7dc00640 1 -- 172.21.15.2:0/2116049035 shutdown_connections 2024-09-10T08:18:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.809+0000 7faf7dc00640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf5c0a2c30 0x7faf5c0a50f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.809+0000 7faf7dc00640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf78078790 0x7faf781a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.809+0000 7faf7dc00640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf78077df0 0x7faf7819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.809+0000 7faf7dc00640 1 --2- 172.21.15.2:0/2116049035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf78076bb0 0x7faf7819e360 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.809+0000 7faf7dc00640 1 -- 172.21.15.2:0/2116049035 >> 172.21.15.2:0/2116049035 conn(0x7faf780ff4e0 msgr2=0x7faf78100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.809+0000 7faf7dc00640 1 -- 172.21.15.2:0/2116049035 shutdown_connections 2024-09-10T08:18:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:09.809+0000 7faf7dc00640 1 -- 172.21.15.2:0/2116049035 wait complete. 2024-09-10T08:18:11.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:11 smithi155 ceph-mon[35405]: pgmap v688: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:18:11.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:11 smithi107 ceph-mon[35048]: pgmap v688: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:18:11.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:11 smithi002 ceph-mon[30563]: pgmap v688: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:18:13.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:13 smithi155 ceph-mon[35405]: pgmap v689: 129 pgs: 129 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:18:13.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:13 smithi107 ceph-mon[35048]: pgmap v689: 129 pgs: 129 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:18:13.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:13 smithi002 ceph-mon[30563]: pgmap v689: 129 pgs: 129 active+clean; 1.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:18:14.281 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:18:14.675 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:18:15.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:15 smithi002 ceph-mon[30563]: pgmap v690: 129 pgs: 129 active+clean; 1.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:18:15.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.261+0000 7f4860800640 1 -- 172.21.15.2:0/661450224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f485810e3c0 msgr2=0x7f4858110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:15.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.261+0000 7f4860800640 1 --2- 172.21.15.2:0/661450224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f485810e3c0 0x7f4858110810 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4854009be0 tx=0x7f485402a750 comp rx=0 tx=0).stop 2024-09-10T08:18:15.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.261+0000 7f4860800640 1 -- 172.21.15.2:0/661450224 shutdown_connections 2024-09-10T08:18:15.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.261+0000 7f4860800640 1 --2- 172.21.15.2:0/661450224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f485810e3c0 0x7f4858110810 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.261+0000 7f4860800640 1 --2- 172.21.15.2:0/661450224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4858074f40 0x7f485810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.261+0000 7f4860800640 1 --2- 172.21.15.2:0/661450224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4858074620 0x7f4858074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.261+0000 7f4860800640 1 -- 172.21.15.2:0/661450224 >> 172.21.15.2:0/661450224 conn(0x7f48580fd8c0 msgr2=0x7f48580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:15.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.261+0000 7f4860800640 1 -- 172.21.15.2:0/661450224 shutdown_connections 2024-09-10T08:18:15.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.261+0000 7f4860800640 1 -- 172.21.15.2:0/661450224 wait complete. 2024-09-10T08:18:15.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.263+0000 7f4860800640 1 Processor -- start 2024-09-10T08:18:15.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.263+0000 7f4860800640 1 -- start start 2024-09-10T08:18:15.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.263+0000 7f4860800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4858074620 0x7f485819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.263+0000 7f4860800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4858074f40 0x7f485819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.263+0000 7f4860800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f485810e3c0 0x7f485819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.263+0000 7f4860800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4858114150 con 0x7f4858074f40 2024-09-10T08:18:15.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.263+0000 7f4860800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4858113fd0 con 0x7f485810e3c0 2024-09-10T08:18:15.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.263+0000 7f4860800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f48581142d0 con 0x7f4858074620 2024-09-10T08:18:15.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.264+0000 7f485de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4858074620 0x7f485819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.264+0000 7f485d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4858074f40 0x7f485819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.264+0000 7f485de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4858074620 0x7f485819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54544/0 (socket says 172.21.15.2:54544) 2024-09-10T08:18:15.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.264+0000 7f485de00640 1 -- 172.21.15.2:0/92389891 learned_addr learned my addr 172.21.15.2:0/92389891 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:15.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.264+0000 7f485e800640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f485810e3c0 0x7f485819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.264+0000 7f485e800640 1 -- 172.21.15.2:0/92389891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4858074620 msgr2=0x7f485819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.264+0000 7f485e800640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4858074620 0x7f485819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.264+0000 7f485e800640 1 -- 172.21.15.2:0/92389891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4858074f40 msgr2=0x7f485819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.264+0000 7f485e800640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4858074f40 0x7f485819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.264+0000 7f485e800640 1 -- 172.21.15.2:0/92389891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485819f5d0 con 0x7f485810e3c0 2024-09-10T08:18:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.264+0000 7f485d400640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4858074f40 0x7f485819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.264+0000 7f485de00640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4858074620 0x7f485819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.265+0000 7f485e800640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f485810e3c0 0x7f485819eef0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4854009b30 tx=0x7f48540305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.265+0000 7f4852a00640 1 -- 172.21.15.2:0/92389891 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854041020 con 0x7f485810e3c0 2024-09-10T08:18:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.265+0000 7f4852a00640 1 -- 172.21.15.2:0/92389891 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4854003e50 con 0x7f485810e3c0 2024-09-10T08:18:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.265+0000 7f4860800640 1 -- 172.21.15.2:0/92389891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485819f860 con 0x7f485810e3c0 2024-09-10T08:18:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.265+0000 7f4852a00640 1 -- 172.21.15.2:0/92389891 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854036040 con 0x7f485810e3c0 2024-09-10T08:18:15.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.265+0000 7f4860800640 1 -- 172.21.15.2:0/92389891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48581a71a0 con 0x7f485810e3c0 2024-09-10T08:18:15.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.269+0000 7f4863226740 1 Processor -- start 2024-09-10T08:18:15.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.269+0000 7f4852a00640 1 -- 172.21.15.2:0/92389891 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4854044020 con 0x7f485810e3c0 2024-09-10T08:18:15.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.270+0000 7f4863226740 1 -- start start 2024-09-10T08:18:15.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.270+0000 7f4863226740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c390f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.270+0000 7f4863226740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 0x560d2c39b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.270+0000 7f4863226740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c39e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.270+0000 7f4863226740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560d2c2f73e0 con 0x560d2c390b90 2024-09-10T08:18:15.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.270+0000 7f4863226740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560d2c29ed80 con 0x560d2c39bd70 2024-09-10T08:18:15.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.270+0000 7f4863226740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d2c2f33a0 con 0x560d2c3914b0 2024-09-10T08:18:15.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.270+0000 7f485d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c390f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.270+0000 7f485d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c390f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35586/0 (socket says 172.21.15.2:35586) 2024-09-10T08:18:15.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.270+0000 7f485d400640 1 -- 172.21.15.2:0/3502459944 learned_addr learned my addr 172.21.15.2:0/3502459944 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:15.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.270+0000 7f485de00640 1 --2- 172.21.15.2:0/3502459944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 0x560d2c39b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.271+0000 7f485d400640 1 --2- 172.21.15.2:0/3502459944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c39e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.271+0000 7f485d400640 1 -- 172.21.15.2:0/3502459944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 msgr2=0x560d2c39b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:15.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.271+0000 7f485d400640 1 --2- 172.21.15.2:0/3502459944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 0x560d2c39b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.271+0000 7f485d400640 1 -- 172.21.15.2:0/3502459944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 msgr2=0x560d2c39e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:15.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.271+0000 7f485d400640 1 --2- 172.21.15.2:0/3502459944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c39e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.271+0000 7f485d400640 1 -- 172.21.15.2:0/3502459944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d2c2bdae0 con 0x560d2c390b90 2024-09-10T08:18:15.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.271+0000 7f485de00640 1 --2- 172.21.15.2:0/3502459944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 0x560d2c39b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:15.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.271+0000 7f485d400640 1 --2- 172.21.15.2:0/3502459944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c390f70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4848013970 tx=0x7f4848009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1683e7fa0495bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:15.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.271+0000 7f4852a00640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f48400a2b80 0x7f48400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.272+0000 7f4852a00640 1 -- 172.21.15.2:0/92389891 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f48540eff50 con 0x7f485810e3c0 2024-09-10T08:18:15.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.272+0000 7f4847000640 1 -- 172.21.15.2:0/3502459944 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848004030 con 0x560d2c390b90 2024-09-10T08:18:15.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.272+0000 7f4847000640 1 -- 172.21.15.2:0/3502459944 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f484800fda0 con 0x560d2c390b90 2024-09-10T08:18:15.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.272+0000 7f485de00640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f48400a2b80 0x7f48400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.272+0000 7f4847000640 1 -- 172.21.15.2:0/3502459944 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848003ab0 con 0x560d2c390b90 2024-09-10T08:18:15.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.272+0000 7f4863226740 1 -- 172.21.15.2:0/3502459944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 msgr2=0x560d2c390f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:15.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.272+0000 7f485de00640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f48400a2b80 0x7f48400a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x560d2c38df80 tx=0x7f484c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:15.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.272+0000 7f4863226740 1 --2- 172.21.15.2:0/3502459944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c390f70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4848013970 tx=0x7f4848009d80 comp rx=0 tx=0).stop 2024-09-10T08:18:15.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.273+0000 7f4863226740 1 -- 172.21.15.2:0/3502459944 shutdown_connections 2024-09-10T08:18:15.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.273+0000 7f4863226740 1 --2- 172.21.15.2:0/3502459944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c39e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.273+0000 7f4863226740 1 --2- 172.21.15.2:0/3502459944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 0x560d2c39b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.273+0000 7f4863226740 1 --2- 172.21.15.2:0/3502459944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c390f70 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.273+0000 7f4863226740 1 -- 172.21.15.2:0/3502459944 >> 172.21.15.2:0/3502459944 conn(0x560d2c38d120 msgr2=0x560d2c38d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:15.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.273+0000 7f4863226740 1 -- 172.21.15.2:0/3502459944 shutdown_connections 2024-09-10T08:18:15.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.273+0000 7f4863226740 1 -- 172.21.15.2:0/3502459944 wait complete. 2024-09-10T08:18:15.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.273+0000 7f4863226740 1 -- start start 2024-09-10T08:18:15.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.273+0000 7f4863226740 1 Processor -- start 2024-09-10T08:18:15.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.274+0000 7f4863226740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c398cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.274+0000 7f4863226740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 0x560d2c399210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.274+0000 7f4863226740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c399750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.274+0000 7f4863226740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560d2c39e9e0 con 0x560d2c390b90 2024-09-10T08:18:15.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.274+0000 7f4863226740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560d2c2b20f0 con 0x560d2c39bd70 2024-09-10T08:18:15.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.274+0000 7f4863226740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d2c39eb60 con 0x560d2c3914b0 2024-09-10T08:18:15.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.274+0000 7f4863226740 10 client.0 _ll_register_callbacks cb 0x560d2c2a39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:15.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.274+0000 7f485de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c398cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.274+0000 7f485de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c398cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35598/0 (socket says 172.21.15.2:35598) 2024-09-10T08:18:15.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.274+0000 7f485de00640 1 -- 172.21.15.2:0/2420725613 learned_addr learned my addr 172.21.15.2:0/2420725613 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:15.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.274+0000 7f485e800640 1 --2- 172.21.15.2:0/2420725613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c399750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.274+0000 7f485d400640 1 --2- 172.21.15.2:0/2420725613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 0x560d2c399210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.275+0000 7f485de00640 1 -- 172.21.15.2:0/2420725613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 msgr2=0x560d2c399210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:15.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.275+0000 7f485de00640 1 --2- 172.21.15.2:0/2420725613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 0x560d2c399210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.275+0000 7f485de00640 1 -- 172.21.15.2:0/2420725613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 msgr2=0x560d2c399750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:15.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.275+0000 7f485de00640 1 --2- 172.21.15.2:0/2420725613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c399750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.275+0000 7f485de00640 1 -- 172.21.15.2:0/2420725613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d2c0683c0 con 0x560d2c390b90 2024-09-10T08:18:15.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.275+0000 7f485de00640 1 --2- 172.21.15.2:0/2420725613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c398cd0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f484c009d90 tx=0x7f484c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a20cd07b6b356 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:15.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.275+0000 7f483b400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:15.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.275+0000 7f483b400640 1 -- 172.21.15.2:0/2420725613 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c003af0 con 0x560d2c390b90 2024-09-10T08:18:15.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.275+0000 7f4863226740 1 -- 172.21.15.2:0/2420725613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f484800fda0 con 0x560d2c390b90 2024-09-10T08:18:15.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.275+0000 7f483b400640 1 -- 172.21.15.2:0/2420725613 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f484c003c90 con 0x560d2c390b90 2024-09-10T08:18:15.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.276+0000 7f483b400640 1 -- 172.21.15.2:0/2420725613 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f484c029440 con 0x560d2c390b90 2024-09-10T08:18:15.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.276+0000 7f4863226740 10 client.18180 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:15.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.276+0000 7f4863226740 1 -- 172.21.15.2:0/2420725613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d2c3aa7b0 con 0x560d2c390b90 2024-09-10T08:18:15.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.276+0000 7f483b400640 1 -- 172.21.15.2:0/2420725613 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f484800fda0 con 0x560d2c390b90 2024-09-10T08:18:15.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.276+0000 7f483b400640 1 -- 172.21.15.2:0/2420725613 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f484c027020 con 0x560d2c390b90 2024-09-10T08:18:15.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.276+0000 7f4863226740 10 client.18180 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:15.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.276+0000 7f4863226740 10 client.18180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:15.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.276+0000 7f4863226740 20 client.18180 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:15.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.276+0000 7f4863226740 1 --2- 172.21.15.2:0/2420725613 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d2c3abbc0 0x560d2c3ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.276+0000 7f4863226740 4 client.18180 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:18:15.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.276+0000 7f4863226740 1 -- 172.21.15.2:0/2420725613 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d2c304950 con 0x560d2c3abbc0 2024-09-10T08:18:15.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.277+0000 7f485d400640 1 --2- 172.21.15.2:0/2420725613 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d2c3abbc0 0x560d2c3ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.277+0000 7f485d400640 1 --2- 172.21.15.2:0/2420725613 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d2c3abbc0 0x560d2c3ae040 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22ec1eb6ebf8eb5f server_cookie=14ee3b54f539fcb1 in_seq=0 out_seq=0 2024-09-10T08:18:15.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.277+0000 7f483b400640 10 client.18180 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:15.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.278+0000 7f483b400640 1 -- 172.21.15.2:0/2420725613 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d2c304950 con 0x560d2c3abbc0 2024-09-10T08:18:15.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.278+0000 7f483b400640 10 client.18180 handle_command_reply: tid=0 2024-09-10T08:18:15.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.278+0000 7f4863226740 1 client.18180 shutdown 2024-09-10T08:18:15.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.278+0000 7f4863226740 1 -- 172.21.15.2:0/2420725613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 msgr2=0x560d2c398cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:15.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.278+0000 7f4863226740 1 --2- 172.21.15.2:0/2420725613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c398cd0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f484c009d90 tx=0x7f484c004730 comp rx=0 tx=0).stop 2024-09-10T08:18:15.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.279+0000 7f4863226740 1 -- 172.21.15.2:0/2420725613 shutdown_connections 2024-09-10T08:18:15.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.279+0000 7f4863226740 1 --2- 172.21.15.2:0/2420725613 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d2c3abbc0 0x560d2c3ae040 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.279+0000 7f4863226740 1 --2- 172.21.15.2:0/2420725613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c399750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.279+0000 7f4863226740 1 --2- 172.21.15.2:0/2420725613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 0x560d2c399210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.279+0000 7f4863226740 1 --2- 172.21.15.2:0/2420725613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c398cd0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.279+0000 7f4863226740 1 -- 172.21.15.2:0/2420725613 >> 172.21.15.2:0/2420725613 conn(0x560d2c38d120 msgr2=0x560d2c392e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:15.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.279+0000 7f483b400640 0 client.18180 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:15.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.279+0000 7f4863226740 1 -- 172.21.15.2:0/2420725613 shutdown_connections 2024-09-10T08:18:15.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.279+0000 7f4863226740 1 -- 172.21.15.2:0/2420725613 wait complete. 2024-09-10T08:18:15.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.279+0000 7f4863226740 20 client.18180 trim_cache size 0 max 16384 2024-09-10T08:18:15.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.297+0000 7f4863226740 1 Processor -- start 2024-09-10T08:18:15.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.297+0000 7f4863226740 1 -- start start 2024-09-10T08:18:15.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f4863226740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c390b90 0x560d2c3935f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f4863226740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c3914b0 0x560d2c3b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f4863226740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c3b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f4863226740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f48581b3950 con 0x560d2c3914b0 2024-09-10T08:18:15.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f4863226740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48581b1220 con 0x560d2c39bd70 2024-09-10T08:18:15.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f4863226740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f48581af070 con 0x560d2c390b90 2024-09-10T08:18:15.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f485d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c3914b0 0x560d2c3b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f485d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c3914b0 0x560d2c3b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35610/0 (socket says 172.21.15.2:35610) 2024-09-10T08:18:15.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f485d400640 1 -- 172.21.15.2:0/92808880 learned_addr learned my addr 172.21.15.2:0/92808880 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:15.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f485de00640 1 --2- 172.21.15.2:0/92808880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c390b90 0x560d2c3935f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f485e800640 1 --2- 172.21.15.2:0/92808880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c3b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f485d400640 1 -- 172.21.15.2:0/92808880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c390b90 msgr2=0x560d2c3935f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:15.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f485d400640 1 --2- 172.21.15.2:0/92808880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c390b90 0x560d2c3935f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f485d400640 1 -- 172.21.15.2:0/92808880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 msgr2=0x560d2c3b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:15.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f485d400640 1 --2- 172.21.15.2:0/92808880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c3b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.298+0000 7f485d400640 1 -- 172.21.15.2:0/92808880 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d2c397220 con 0x560d2c3914b0 2024-09-10T08:18:15.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.299+0000 7f485d400640 1 --2- 172.21.15.2:0/92808880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c3914b0 0x560d2c3b3910 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4848013fd0 tx=0x7f4848021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a358f5a1bd83334 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:15.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.299+0000 7f4846600640 1 -- 172.21.15.2:0/92808880 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848005110 con 0x560d2c3914b0 2024-09-10T08:18:15.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.299+0000 7f4846600640 1 -- 172.21.15.2:0/92808880 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f484800f8c0 con 0x560d2c3914b0 2024-09-10T08:18:15.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.299+0000 7f4846600640 1 -- 172.21.15.2:0/92808880 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848003b40 con 0x560d2c3914b0 2024-09-10T08:18:15.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.299+0000 7f4863226740 1 -- 172.21.15.2:0/92808880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c3914b0 msgr2=0x560d2c3b3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:15.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.299+0000 7f4863226740 1 --2- 172.21.15.2:0/92808880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c3914b0 0x560d2c3b3910 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4848013fd0 tx=0x7f4848021040 comp rx=0 tx=0).stop 2024-09-10T08:18:15.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.300+0000 7f4863226740 1 -- 172.21.15.2:0/92808880 shutdown_connections 2024-09-10T08:18:15.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.300+0000 7f4863226740 1 --2- 172.21.15.2:0/92808880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c3b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.300+0000 7f4863226740 1 --2- 172.21.15.2:0/92808880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c3914b0 0x560d2c3b3910 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.300+0000 7f4863226740 1 --2- 172.21.15.2:0/92808880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c390b90 0x560d2c3935f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.300+0000 7f4863226740 1 -- 172.21.15.2:0/92808880 >> 172.21.15.2:0/92808880 conn(0x560d2c38d120 msgr2=0x560d2c397dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.300+0000 7f4863226740 1 -- 172.21.15.2:0/92808880 shutdown_connections 2024-09-10T08:18:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.300+0000 7f4863226740 1 -- 172.21.15.2:0/92808880 wait complete. 2024-09-10T08:18:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.300+0000 7f4863226740 1 -- start start 2024-09-10T08:18:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.300+0000 7f4863226740 1 Processor -- start 2024-09-10T08:18:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.301+0000 7f4863226740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c39f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.301+0000 7f4863226740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 0x560d2c39f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.301+0000 7f4863226740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c4148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.301+0000 7f4863226740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f48581af6e0 con 0x560d2c390b90 2024-09-10T08:18:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.301+0000 7f4863226740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48581b0b50 con 0x560d2c39bd70 2024-09-10T08:18:15.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.301+0000 7f4863226740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f48581afdb0 con 0x560d2c3914b0 2024-09-10T08:18:15.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.301+0000 7f4863226740 10 client.0 _ll_register_callbacks cb 0x7f484c0044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:15.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.301+0000 7f485de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c39f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.301+0000 7f485de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c39f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35618/0 (socket says 172.21.15.2:35618) 2024-09-10T08:18:15.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.301+0000 7f485de00640 1 -- 172.21.15.2:0/313911004 learned_addr learned my addr 172.21.15.2:0/313911004 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:15.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.301+0000 7f485d400640 1 --2- 172.21.15.2:0/313911004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 0x560d2c39f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.301+0000 7f485e800640 1 --2- 172.21.15.2:0/313911004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c4148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.301+0000 7f485de00640 1 -- 172.21.15.2:0/313911004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 msgr2=0x560d2c39f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:15.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.302+0000 7f485de00640 1 --2- 172.21.15.2:0/313911004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 0x560d2c39f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.302+0000 7f485de00640 1 -- 172.21.15.2:0/313911004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 msgr2=0x560d2c4148d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.302+0000 7f485de00640 1 --2- 172.21.15.2:0/313911004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c4148d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.302+0000 7f485de00640 1 -- 172.21.15.2:0/313911004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d2c414e90 con 0x560d2c390b90 2024-09-10T08:18:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.302+0000 7f485de00640 1 --2- 172.21.15.2:0/313911004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c39f310 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f484c01f7a0 tx=0x7f484c003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dff36148472e251 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.302+0000 7f483b400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.302+0000 7f483b400640 1 -- 172.21.15.2:0/313911004 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c01f400 con 0x560d2c390b90 2024-09-10T08:18:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.302+0000 7f483b400640 1 -- 172.21.15.2:0/313911004 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f484c0295e0 con 0x560d2c390b90 2024-09-10T08:18:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.302+0000 7f4863226740 1 -- 172.21.15.2:0/313911004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f484800f8c0 con 0x560d2c390b90 2024-09-10T08:18:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.302+0000 7f483b400640 1 -- 172.21.15.2:0/313911004 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f484c044050 con 0x560d2c390b90 2024-09-10T08:18:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.302+0000 7f4863226740 10 client.18186 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:15.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.303+0000 7f4863226740 1 -- 172.21.15.2:0/313911004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d2c4161b0 con 0x560d2c390b90 2024-09-10T08:18:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.303+0000 7f483b400640 1 -- 172.21.15.2:0/313911004 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f484800f8c0 con 0x560d2c390b90 2024-09-10T08:18:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.303+0000 7f483b400640 1 -- 172.21.15.2:0/313911004 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f484c027040 con 0x560d2c390b90 2024-09-10T08:18:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.303+0000 7f4863226740 10 client.18186 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.303+0000 7f4863226740 10 client.18186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.303+0000 7f4863226740 20 client.18186 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.303+0000 7f4863226740 1 --2- 172.21.15.2:0/313911004 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d2c4169e0 0x560d2c418e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.303+0000 7f4863226740 4 client.18186 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:18:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.303+0000 7f4863226740 1 -- 172.21.15.2:0/313911004 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d2c2fee60 con 0x560d2c4169e0 2024-09-10T08:18:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.303+0000 7f485d400640 1 --2- 172.21.15.2:0/313911004 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d2c4169e0 0x560d2c418e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.304+0000 7f485d400640 1 --2- 172.21.15.2:0/313911004 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d2c4169e0 0x560d2c418e00 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0706b3b2777107 server_cookie=97bcf2d18ce1cc3f in_seq=0 out_seq=0 2024-09-10T08:18:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.304+0000 7f483b400640 10 client.18186 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:15.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.304+0000 7f483b400640 1 -- 172.21.15.2:0/313911004 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560d2c2fee60 con 0x560d2c4169e0 2024-09-10T08:18:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.304+0000 7f483b400640 10 client.18186 handle_command_reply: tid=0 2024-09-10T08:18:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.304+0000 7f4863226740 1 client.18186 shutdown 2024-09-10T08:18:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.304+0000 7f4863226740 1 -- 172.21.15.2:0/313911004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 msgr2=0x560d2c39f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.304+0000 7f4863226740 1 --2- 172.21.15.2:0/313911004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c39f310 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f484c01f7a0 tx=0x7f484c003bf0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.304+0000 7f4863226740 1 -- 172.21.15.2:0/313911004 shutdown_connections 2024-09-10T08:18:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.304+0000 7f4863226740 1 --2- 172.21.15.2:0/313911004 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d2c4169e0 0x560d2c418e00 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.304+0000 7f4863226740 1 --2- 172.21.15.2:0/313911004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d2c39bd70 0x560d2c4148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.305+0000 7f4863226740 1 --2- 172.21.15.2:0/313911004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2c3914b0 0x560d2c39f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.305+0000 7f4863226740 1 --2- 172.21.15.2:0/313911004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d2c390b90 0x560d2c39f310 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.305+0000 7f483b400640 0 client.18186 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.305+0000 7f4863226740 1 -- 172.21.15.2:0/313911004 >> 172.21.15.2:0/313911004 conn(0x560d2c38d120 msgr2=0x560d2c38b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.305+0000 7f4863226740 1 -- 172.21.15.2:0/313911004 shutdown_connections 2024-09-10T08:18:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.305+0000 7f4863226740 1 -- 172.21.15.2:0/313911004 wait complete. 2024-09-10T08:18:15.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.305+0000 7f4863226740 20 client.18186 trim_cache size 0 max 16384 2024-09-10T08:18:15.311 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:18:15.311 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:18:15.311 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:18:15.311 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:18:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.305+0000 7f4860800640 1 -- 172.21.15.2:0/92389891 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f48400a2b80 msgr2=0x7f48400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.305+0000 7f4860800640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f48400a2b80 0x7f48400a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x560d2c38df80 tx=0x7f484c009260 comp rx=0 tx=0).stop 2024-09-10T08:18:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.305+0000 7f4860800640 1 -- 172.21.15.2:0/92389891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f485810e3c0 msgr2=0x7f485819eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.305+0000 7f4860800640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f485810e3c0 0x7f485819eef0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4854009b30 tx=0x7f48540305d0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.306+0000 7f4860800640 1 -- 172.21.15.2:0/92389891 shutdown_connections 2024-09-10T08:18:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.306+0000 7f4860800640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f485810e3c0 0x7f485819eef0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.306+0000 7f4860800640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4858074f40 0x7f485819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.306+0000 7f4860800640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f48400a2b80 0x7f48400a5040 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.306+0000 7f4860800640 1 --2- 172.21.15.2:0/92389891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4858074620 0x7f485819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.306+0000 7f4860800640 1 -- 172.21.15.2:0/92389891 >> 172.21.15.2:0/92389891 conn(0x7f48580fd8c0 msgr2=0x7f48580fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.306+0000 7f4860800640 1 -- 172.21.15.2:0/92389891 shutdown_connections 2024-09-10T08:18:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:15.306+0000 7f4860800640 1 -- 172.21.15.2:0/92389891 wait complete. 2024-09-10T08:18:15.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:15 smithi155 ceph-mon[35405]: pgmap v690: 129 pgs: 129 active+clean; 1.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:18:15.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:15 smithi107 ceph-mon[35048]: pgmap v690: 129 pgs: 129 active+clean; 1.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:18:16.244 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:18:16.244 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:18:16.623 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:18:17.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.194+0000 7f0192000640 1 -- 172.21.15.2:0/3780383208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f018c1049a0 msgr2=0x7f018c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:17.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.194+0000 7f0192000640 1 --2- 172.21.15.2:0/3780383208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f018c1049a0 0x7f018c1089f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0178009860 tx=0x7f017802a340 comp rx=0 tx=0).stop 2024-09-10T08:18:17.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.194+0000 7f0192000640 1 -- 172.21.15.2:0/3780383208 shutdown_connections 2024-09-10T08:18:17.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.194+0000 7f0192000640 1 --2- 172.21.15.2:0/3780383208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f018c109120 0x7f018c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:17.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.194+0000 7f0192000640 1 --2- 172.21.15.2:0/3780383208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f018c1049a0 0x7f018c1089f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:17.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.194+0000 7f0192000640 1 --2- 172.21.15.2:0/3780383208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f018c103ff0 0x7f018c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:17.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.194+0000 7f0192000640 1 -- 172.21.15.2:0/3780383208 >> 172.21.15.2:0/3780383208 conn(0x7f018c0ffa90 msgr2=0x7f018c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:17.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.194+0000 7f0192000640 1 -- 172.21.15.2:0/3780383208 shutdown_connections 2024-09-10T08:18:17.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.194+0000 7f0192000640 1 -- 172.21.15.2:0/3780383208 wait complete. 2024-09-10T08:18:17.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.195+0000 7f0192000640 1 Processor -- start 2024-09-10T08:18:17.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.195+0000 7f0192000640 1 -- start start 2024-09-10T08:18:17.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f0192000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f018c103ff0 0x7f018c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:17.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f0192000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f018c1049a0 0x7f018c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:17.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f0192000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f018c109120 0x7f018c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:17.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f0192000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f018c114140 con 0x7f018c109120 2024-09-10T08:18:17.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f0192000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f018c113fc0 con 0x7f018c1049a0 2024-09-10T08:18:17.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f0192000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f018c1142c0 con 0x7f018c103ff0 2024-09-10T08:18:17.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f018be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f018c109120 0x7f018c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:17.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f018be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f018c109120 0x7f018c19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40366/0 (socket says 172.21.15.2:40366) 2024-09-10T08:18:17.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f018be00640 1 -- 172.21.15.2:0/3915670139 learned_addr learned my addr 172.21.15.2:0/3915670139 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:17.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f018b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f018c103ff0 0x7f018c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:17.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f018aa00640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f018c1049a0 0x7f018c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:17.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f018be00640 1 -- 172.21.15.2:0/3915670139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f018c103ff0 msgr2=0x7f018c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:17.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f018be00640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f018c103ff0 0x7f018c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:17.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f018be00640 1 -- 172.21.15.2:0/3915670139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f018c1049a0 msgr2=0x7f018c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:17.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f018be00640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f018c1049a0 0x7f018c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:17.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f018be00640 1 -- 172.21.15.2:0/3915670139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018c19f5d0 con 0x7f018c109120 2024-09-10T08:18:17.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.196+0000 7f018aa00640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f018c1049a0 0x7f018c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:17.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.197+0000 7f018be00640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f018c109120 0x7f018c19eef0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0184009fd0 tx=0x7f018400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:17.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.197+0000 7f0183e00640 1 -- 172.21.15.2:0/3915670139 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0184004b20 con 0x7f018c109120 2024-09-10T08:18:17.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.197+0000 7f0183e00640 1 -- 172.21.15.2:0/3915670139 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0184003e50 con 0x7f018c109120 2024-09-10T08:18:17.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.197+0000 7f0183e00640 1 -- 172.21.15.2:0/3915670139 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018400b630 con 0x7f018c109120 2024-09-10T08:18:17.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.197+0000 7f0192000640 1 -- 172.21.15.2:0/3915670139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018c19f8c0 con 0x7f018c109120 2024-09-10T08:18:17.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.198+0000 7f0192000640 1 -- 172.21.15.2:0/3915670139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f018c1a71a0 con 0x7f018c109120 2024-09-10T08:18:17.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.199+0000 7f0192000640 1 -- 172.21.15.2:0/3915670139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f018c1043d0 con 0x7f018c109120 2024-09-10T08:18:17.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.201+0000 7f0183e00640 1 -- 172.21.15.2:0/3915670139 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f018401d020 con 0x7f018c109120 2024-09-10T08:18:17.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.202+0000 7f0183e00640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f01700a2b80 0x7f01700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:17.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.202+0000 7f0183e00640 1 -- 172.21.15.2:0/3915670139 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f01840cf3e0 con 0x7f018c109120 2024-09-10T08:18:17.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.202+0000 7f018b400640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f01700a2b80 0x7f01700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:17.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.203+0000 7f018b400640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f01700a2b80 0x7f01700a5040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f017c009bd0 tx=0x7f017c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:17.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.207+0000 7f0183e00640 1 -- 172.21.15.2:0/3915670139 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f018409bef0 con 0x7f018c109120 2024-09-10T08:18:17.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:17 smithi155 ceph-mon[35405]: pgmap v691: 129 pgs: 129 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:18:17.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:17 smithi107 ceph-mon[35048]: pgmap v691: 129 pgs: 129 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:18:17.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.429+0000 7f0192000640 1 -- 172.21.15.2:0/3915670139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f018c10c360 con 0x7f018c109120 2024-09-10T08:18:17.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:17 smithi002 ceph-mon[30563]: pgmap v691: 129 pgs: 129 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:18:17.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.430+0000 7f0183e00640 1 -- 172.21.15.2:0/3915670139 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f018409b7f0 con 0x7f018c109120 2024-09-10T08:18:17.430 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:18:17.430 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:18:17.431 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:18:17.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.432+0000 7f0192000640 1 -- 172.21.15.2:0/3915670139 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f01700a2b80 msgr2=0x7f01700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:17.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.432+0000 7f0192000640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f01700a2b80 0x7f01700a5040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f017c009bd0 tx=0x7f017c005f90 comp rx=0 tx=0).stop 2024-09-10T08:18:17.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.432+0000 7f0192000640 1 -- 172.21.15.2:0/3915670139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f018c109120 msgr2=0x7f018c19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:17.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.432+0000 7f0192000640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f018c109120 0x7f018c19eef0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0184009fd0 tx=0x7f018400c800 comp rx=0 tx=0).stop 2024-09-10T08:18:17.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.432+0000 7f0192000640 1 -- 172.21.15.2:0/3915670139 shutdown_connections 2024-09-10T08:18:17.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.432+0000 7f0192000640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f018c109120 0x7f018c19eef0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:17.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.432+0000 7f0192000640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f018c1049a0 0x7f018c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:17.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.432+0000 7f0192000640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f01700a2b80 0x7f01700a5040 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:17.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.432+0000 7f0192000640 1 --2- 172.21.15.2:0/3915670139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f018c103ff0 0x7f018c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:17.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.432+0000 7f0192000640 1 -- 172.21.15.2:0/3915670139 >> 172.21.15.2:0/3915670139 conn(0x7f018c0ffa90 msgr2=0x7f018c100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:17.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.433+0000 7f0192000640 1 -- 172.21.15.2:0/3915670139 shutdown_connections 2024-09-10T08:18:17.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:17.433+0000 7f0192000640 1 -- 172.21.15.2:0/3915670139 wait complete. 2024-09-10T08:18:18.211 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:18 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3915670139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:18:18.249 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:18:18.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:18 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3915670139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:18:18.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:18 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3915670139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:18:18.628 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:18:19.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.214+0000 7fc69b800640 1 -- 172.21.15.2:0/3405539304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc694103ff0 msgr2=0x7fc6941043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:19.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.214+0000 7fc69b800640 1 --2- 172.21.15.2:0/3405539304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc694103ff0 0x7fc6941043d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc684009900 tx=0x7fc68402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.214+0000 7fc69b800640 1 -- 172.21.15.2:0/3405539304 shutdown_connections 2024-09-10T08:18:19.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.214+0000 7fc69b800640 1 --2- 172.21.15.2:0/3405539304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc694109120 0x7fc694110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.214+0000 7fc69b800640 1 --2- 172.21.15.2:0/3405539304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6941049a0 0x7fc6941089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.214+0000 7fc69b800640 1 --2- 172.21.15.2:0/3405539304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc694103ff0 0x7fc6941043d0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.214+0000 7fc69b800640 1 -- 172.21.15.2:0/3405539304 >> 172.21.15.2:0/3405539304 conn(0x7fc6940ffa90 msgr2=0x7fc694101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:19.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.214+0000 7fc69b800640 1 -- 172.21.15.2:0/3405539304 shutdown_connections 2024-09-10T08:18:19.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.214+0000 7fc69b800640 1 -- 172.21.15.2:0/3405539304 wait complete. 2024-09-10T08:18:19.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.215+0000 7fc69b800640 1 Processor -- start 2024-09-10T08:18:19.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.215+0000 7fc69b800640 1 -- start start 2024-09-10T08:18:19.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.215+0000 7fc69b800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc694103ff0 0x7fc69419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.215+0000 7fc69b800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6941049a0 0x7fc69419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.215+0000 7fc69b800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc694109120 0x7fc69419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.215+0000 7fc69b800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc694114150 con 0x7fc6941049a0 2024-09-10T08:18:19.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.215+0000 7fc69b800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc694113fd0 con 0x7fc694103ff0 2024-09-10T08:18:19.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.215+0000 7fc69b800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941142d0 con 0x7fc694109120 2024-09-10T08:18:19.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.216+0000 7fc698e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc694103ff0 0x7fc69419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.216+0000 7fc68be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6941049a0 0x7fc69419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.216+0000 7fc698e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc694103ff0 0x7fc69419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44952/0 (socket says 172.21.15.2:44952) 2024-09-10T08:18:19.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.216+0000 7fc698e00640 1 -- 172.21.15.2:0/1930873346 learned_addr learned my addr 172.21.15.2:0/1930873346 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:19.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.216+0000 7fc699800640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc694109120 0x7fc69419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.216+0000 7fc68be00640 1 -- 172.21.15.2:0/1930873346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc694109120 msgr2=0x7fc69419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:19.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.216+0000 7fc68be00640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc694109120 0x7fc69419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.216+0000 7fc68be00640 1 -- 172.21.15.2:0/1930873346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc694103ff0 msgr2=0x7fc69419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:19.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.216+0000 7fc68be00640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc694103ff0 0x7fc69419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.216+0000 7fc68be00640 1 -- 172.21.15.2:0/1930873346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69419f5a0 con 0x7fc6941049a0 2024-09-10T08:18:19.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.216+0000 7fc698e00640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc694103ff0 0x7fc69419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:19.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.217+0000 7fc68be00640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6941049a0 0x7fc69419ab30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00e9f0 tx=0x7fc67c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:19.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.217+0000 7fc692000640 1 -- 172.21.15.2:0/1930873346 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c00ccc0 con 0x7fc6941049a0 2024-09-10T08:18:19.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.217+0000 7fc692000640 1 -- 172.21.15.2:0/1930873346 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c010910 con 0x7fc6941049a0 2024-09-10T08:18:19.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.217+0000 7fc69b800640 1 -- 172.21.15.2:0/1930873346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69419f890 con 0x7fc6941049a0 2024-09-10T08:18:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.217+0000 7fc692000640 1 -- 172.21.15.2:0/1930873346 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c018960 con 0x7fc6941049a0 2024-09-10T08:18:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.217+0000 7fc69b800640 1 -- 172.21.15.2:0/1930873346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6941a7170 con 0x7fc6941049a0 2024-09-10T08:18:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.220+0000 7fc69e261740 1 Processor -- start 2024-09-10T08:18:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.220+0000 7fc692000640 1 -- 172.21.15.2:0/1930873346 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc67c010ac0 con 0x7fc6941049a0 2024-09-10T08:18:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.221+0000 7fc69e261740 1 -- start start 2024-09-10T08:18:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.221+0000 7fc69e261740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fcc70 0x55eea30fd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.221+0000 7fc69e261740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fd590 0x55eea3107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.221+0000 7fc69e261740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 0x55eea310a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.221+0000 7fc69e261740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55eea30633e0 con 0x55eea30fcc70 2024-09-10T08:18:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.221+0000 7fc69e261740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eea300ad80 con 0x55eea30fd590 2024-09-10T08:18:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.221+0000 7fc69e261740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eea305f3a0 con 0x55eea3107e50 2024-09-10T08:18:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.221+0000 7fc699800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fcc70 0x55eea30fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.221+0000 7fc699800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fcc70 0x55eea30fd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40398/0 (socket says 172.21.15.2:40398) 2024-09-10T08:18:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.221+0000 7fc699800640 1 -- 172.21.15.2:0/4158283573 learned_addr learned my addr 172.21.15.2:0/4158283573 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.221+0000 7fc698e00640 1 --2- 172.21.15.2:0/4158283573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fd590 0x55eea3107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.221+0000 7fc68be00640 1 --2- 172.21.15.2:0/4158283573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 0x55eea310a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.222+0000 7fc698e00640 1 -- 172.21.15.2:0/4158283573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 msgr2=0x55eea310a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.222+0000 7fc698e00640 1 --2- 172.21.15.2:0/4158283573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 0x55eea310a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.222+0000 7fc698e00640 1 -- 172.21.15.2:0/4158283573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fcc70 msgr2=0x55eea30fd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.222+0000 7fc698e00640 1 --2- 172.21.15.2:0/4158283573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fcc70 0x55eea30fd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.222+0000 7fc698e00640 1 -- 172.21.15.2:0/4158283573 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eea3029ae0 con 0x55eea30fd590 2024-09-10T08:18:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.222+0000 7fc698e00640 1 --2- 172.21.15.2:0/4158283573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fd590 0x55eea3107910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc684009230 tx=0x7fc684005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f975454c1e5fffa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.222+0000 7fc692000640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6780a2b80 0x7fc6780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.222+0000 7fc68a000640 1 -- 172.21.15.2:0/4158283573 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684040070 con 0x55eea30fd590 2024-09-10T08:18:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.223+0000 7fc68a000640 1 -- 172.21.15.2:0/4158283573 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684031970 con 0x55eea30fd590 2024-09-10T08:18:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.223+0000 7fc68a000640 1 -- 172.21.15.2:0/4158283573 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684035040 con 0x55eea30fd590 2024-09-10T08:18:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.223+0000 7fc692000640 1 -- 172.21.15.2:0/1930873346 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc67c0cf820 con 0x7fc6941049a0 2024-09-10T08:18:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.223+0000 7fc699800640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6780a2b80 0x7fc6780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.223+0000 7fc69e261740 1 -- 172.21.15.2:0/4158283573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fd590 msgr2=0x55eea3107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.223+0000 7fc69e261740 1 --2- 172.21.15.2:0/4158283573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fd590 0x55eea3107910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc684009230 tx=0x7fc684005dd0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.223+0000 7fc699800640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6780a2b80 0x7fc6780a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc68c011630 tx=0x7fc68c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.224+0000 7fc69e261740 1 -- 172.21.15.2:0/4158283573 shutdown_connections 2024-09-10T08:18:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.224+0000 7fc69e261740 1 --2- 172.21.15.2:0/4158283573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 0x55eea310a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.224+0000 7fc69e261740 1 --2- 172.21.15.2:0/4158283573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fd590 0x55eea3107910 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.224+0000 7fc69e261740 1 --2- 172.21.15.2:0/4158283573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fcc70 0x55eea30fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.224+0000 7fc69e261740 1 -- 172.21.15.2:0/4158283573 >> 172.21.15.2:0/4158283573 conn(0x55eea30f9200 msgr2=0x55eea30f95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.224+0000 7fc69e261740 1 -- 172.21.15.2:0/4158283573 shutdown_connections 2024-09-10T08:18:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.224+0000 7fc69e261740 1 -- 172.21.15.2:0/4158283573 wait complete. 2024-09-10T08:18:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.224+0000 7fc69e261740 1 -- start start 2024-09-10T08:18:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.224+0000 7fc69e261740 1 Processor -- start 2024-09-10T08:18:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.225+0000 7fc69e261740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fcc70 0x55eea3104cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.225+0000 7fc69e261740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea31051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.225+0000 7fc69e261740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 0x55eea3105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.225+0000 7fc69e261740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55eea310aa20 con 0x55eea30fd590 2024-09-10T08:18:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.225+0000 7fc69e261740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eea301e0f0 con 0x55eea30fcc70 2024-09-10T08:18:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.225+0000 7fc69e261740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eea310aba0 con 0x55eea3107e50 2024-09-10T08:18:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.225+0000 7fc69e261740 10 client.0 _ll_register_callbacks cb 0x55eea3005bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.225+0000 7fc68be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea31051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.225+0000 7fc698e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fcc70 0x55eea3104cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.225+0000 7fc698e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fcc70 0x55eea3104cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44974/0 (socket says 172.21.15.2:44974) 2024-09-10T08:18:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.225+0000 7fc68be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea31051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40408/0 (socket says 172.21.15.2:40408) 2024-09-10T08:18:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.225+0000 7fc698e00640 1 -- 172.21.15.2:0/1906353677 learned_addr learned my addr 172.21.15.2:0/1906353677 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.226+0000 7fc699800640 1 --2- 172.21.15.2:0/1906353677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 0x55eea3105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.226+0000 7fc698e00640 1 -- 172.21.15.2:0/1906353677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 msgr2=0x55eea3105730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.226+0000 7fc698e00640 1 --2- 172.21.15.2:0/1906353677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 0x55eea3105730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.226+0000 7fc698e00640 1 -- 172.21.15.2:0/1906353677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 msgr2=0x55eea31051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.226+0000 7fc698e00640 1 --2- 172.21.15.2:0/1906353677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea31051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.226+0000 7fc698e00640 1 -- 172.21.15.2:0/1906353677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eea2dd43c0 con 0x55eea30fcc70 2024-09-10T08:18:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.226+0000 7fc699800640 1 --2- 172.21.15.2:0/1906353677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 0x55eea3105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:18:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.226+0000 7fc698e00640 1 --2- 172.21.15.2:0/1906353677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fcc70 0x55eea3104cb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc68402a430 tx=0x7fc684031770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f30ff1876b49e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.226+0000 7fc682a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:18:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.226+0000 7fc682a00640 1 -- 172.21.15.2:0/1906353677 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68404b020 con 0x55eea30fcc70 2024-09-10T08:18:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.226+0000 7fc69e261740 1 -- 172.21.15.2:0/1906353677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc684031970 con 0x55eea30fcc70 2024-09-10T08:18:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.226+0000 7fc682a00640 1 -- 172.21.15.2:0/1906353677 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc6840035f0 con 0x55eea30fcc70 2024-09-10T08:18:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.226+0000 7fc682a00640 1 -- 172.21.15.2:0/1906353677 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc684056050 con 0x55eea30fcc70 2024-09-10T08:18:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.227+0000 7fc69e261740 10 client.28238 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.227+0000 7fc69e261740 1 -- 172.21.15.2:0/1906353677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eea3116a10 con 0x55eea30fcc70 2024-09-10T08:18:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.227+0000 7fc682a00640 1 -- 172.21.15.2:0/1906353677 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684031970 con 0x55eea30fcc70 2024-09-10T08:18:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.227+0000 7fc682a00640 1 -- 172.21.15.2:0/1906353677 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc68403c9d0 con 0x55eea30fcc70 2024-09-10T08:18:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.227+0000 7fc69e261740 10 client.28238 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.227+0000 7fc69e261740 10 client.28238 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:18:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.227+0000 7fc69e261740 20 client.28238 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:19.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.227+0000 7fc69e261740 1 --2- 172.21.15.2:0/1906353677 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55eea3117d60 0x55eea311a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.227+0000 7fc69e261740 4 client.28238 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:18:19.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.227+0000 7fc69e261740 1 -- 172.21.15.2:0/1906353677 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eea3070950 con 0x55eea3117d60 2024-09-10T08:18:19.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.228+0000 7fc68be00640 1 --2- 172.21.15.2:0/1906353677 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55eea3117d60 0x55eea311a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.228+0000 7fc68be00640 1 --2- 172.21.15.2:0/1906353677 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55eea3117d60 0x55eea311a150 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40c44bcd2b89194a server_cookie=3e6110f1b3bb7f3d in_seq=0 out_seq=0 2024-09-10T08:18:19.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.228+0000 7fc682a00640 10 client.28238 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:18:19.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc682a00640 1 -- 172.21.15.2:0/1906353677 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eea3070950 con 0x55eea3117d60 2024-09-10T08:18:19.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc682a00640 10 client.28238 handle_command_reply: tid=0 2024-09-10T08:18:19.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc69e261740 1 client.28238 shutdown 2024-09-10T08:18:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc69e261740 1 -- 172.21.15.2:0/1906353677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fcc70 msgr2=0x55eea3104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc69e261740 1 --2- 172.21.15.2:0/1906353677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fcc70 0x55eea3104cb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc68402a430 tx=0x7fc684031770 comp rx=0 tx=0).stop 2024-09-10T08:18:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc69e261740 1 -- 172.21.15.2:0/1906353677 shutdown_connections 2024-09-10T08:18:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc69e261740 1 --2- 172.21.15.2:0/1906353677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 0x55eea3105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc69e261740 1 --2- 172.21.15.2:0/1906353677 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55eea3117d60 0x55eea311a150 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc69e261740 1 --2- 172.21.15.2:0/1906353677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea31051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc69e261740 1 --2- 172.21.15.2:0/1906353677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fcc70 0x55eea3104cb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc69e261740 1 -- 172.21.15.2:0/1906353677 >> 172.21.15.2:0/1906353677 conn(0x55eea30f9200 msgr2=0x55eea30f8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc682a00640 0 client.28238 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:18:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc69e261740 1 -- 172.21.15.2:0/1906353677 shutdown_connections 2024-09-10T08:18:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.229+0000 7fc69e261740 1 -- 172.21.15.2:0/1906353677 wait complete. 2024-09-10T08:18:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.230+0000 7fc69e261740 20 client.28238 trim_cache size 0 max 16384 2024-09-10T08:18:19.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.243+0000 7fc69e261740 1 Processor -- start 2024-09-10T08:18:19.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.243+0000 7fc69e261740 1 -- start start 2024-09-10T08:18:19.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc69e261740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fcc70 0x55eea3103f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc69e261740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea311f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc69e261740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 0x55eea311fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc69e261740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941b3890 con 0x55eea30fd590 2024-09-10T08:18:19.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc69e261740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941b1160 con 0x55eea30fcc70 2024-09-10T08:18:19.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc69e261740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941aefb0 con 0x55eea3107e50 2024-09-10T08:18:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc68be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea311f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc68be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea311f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40420/0 (socket says 172.21.15.2:40420) 2024-09-10T08:18:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc68be00640 1 -- 172.21.15.2:0/81677498 learned_addr learned my addr 172.21.15.2:0/81677498 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc699800640 1 --2- 172.21.15.2:0/81677498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 0x55eea311fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc698e00640 1 --2- 172.21.15.2:0/81677498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fcc70 0x55eea3103f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc68be00640 1 -- 172.21.15.2:0/81677498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 msgr2=0x55eea311fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc68be00640 1 --2- 172.21.15.2:0/81677498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 0x55eea311fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc68be00640 1 -- 172.21.15.2:0/81677498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fcc70 msgr2=0x55eea3103f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc68be00640 1 --2- 172.21.15.2:0/81677498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fcc70 0x55eea3103f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.244+0000 7fc68be00640 1 -- 172.21.15.2:0/81677498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eea30f9030 con 0x55eea30fd590 2024-09-10T08:18:19.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.245+0000 7fc68be00640 1 --2- 172.21.15.2:0/81677498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea311f780 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc67c09bc70 tx=0x7fc67c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af1e830d3bff785 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:19.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.245+0000 7fc689600640 1 -- 172.21.15.2:0/81677498 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0a48a0 con 0x55eea30fd590 2024-09-10T08:18:19.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.245+0000 7fc689600640 1 -- 172.21.15.2:0/81677498 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c099fe0 con 0x55eea30fd590 2024-09-10T08:18:19.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.245+0000 7fc689600640 1 -- 172.21.15.2:0/81677498 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c018b00 con 0x55eea30fd590 2024-09-10T08:18:19.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.245+0000 7fc69e261740 1 -- 172.21.15.2:0/81677498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 msgr2=0x55eea311f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:19.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.245+0000 7fc69e261740 1 --2- 172.21.15.2:0/81677498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea311f780 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc67c09bc70 tx=0x7fc67c009700 comp rx=0 tx=0).stop 2024-09-10T08:18:19.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.246+0000 7fc69e261740 1 -- 172.21.15.2:0/81677498 shutdown_connections 2024-09-10T08:18:19.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.246+0000 7fc69e261740 1 --2- 172.21.15.2:0/81677498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea3107e50 0x55eea311fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.246+0000 7fc69e261740 1 --2- 172.21.15.2:0/81677498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea311f780 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.246+0000 7fc69e261740 1 --2- 172.21.15.2:0/81677498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea30fcc70 0x55eea3103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.246+0000 7fc69e261740 1 -- 172.21.15.2:0/81677498 >> 172.21.15.2:0/81677498 conn(0x55eea30f9200 msgr2=0x55eea3106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:19.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.246+0000 7fc69e261740 1 -- 172.21.15.2:0/81677498 shutdown_connections 2024-09-10T08:18:19.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.246+0000 7fc69e261740 1 -- 172.21.15.2:0/81677498 wait complete. 2024-09-10T08:18:19.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.246+0000 7fc69e261740 1 -- start start 2024-09-10T08:18:19.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.247+0000 7fc69e261740 1 Processor -- start 2024-09-10T08:18:19.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.247+0000 7fc69e261740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea30fcc70 0x55eea3110000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.247+0000 7fc69e261740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea3110540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.247+0000 7fc69e261740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea3107e50 0x55eea31012c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.247+0000 7fc69e261740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941af620 con 0x55eea30fd590 2024-09-10T08:18:19.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.247+0000 7fc69e261740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941b0a90 con 0x55eea3107e50 2024-09-10T08:18:19.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.247+0000 7fc69e261740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941afcf0 con 0x55eea30fcc70 2024-09-10T08:18:19.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.247+0000 7fc69e261740 10 client.0 _ll_register_callbacks cb 0x7fc68c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:19.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.247+0000 7fc68be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea3110540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.247+0000 7fc699800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea3107e50 0x55eea31012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.247+0000 7fc68be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea3110540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40428/0 (socket says 172.21.15.2:40428) 2024-09-10T08:18:19.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.247+0000 7fc68be00640 1 -- 172.21.15.2:0/3085437646 learned_addr learned my addr 172.21.15.2:0/3085437646 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:19.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.247+0000 7fc698e00640 1 --2- 172.21.15.2:0/3085437646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea30fcc70 0x55eea3110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.248+0000 7fc68be00640 1 -- 172.21.15.2:0/3085437646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea30fcc70 msgr2=0x55eea3110000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:19.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.248+0000 7fc68be00640 1 --2- 172.21.15.2:0/3085437646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea30fcc70 0x55eea3110000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.248+0000 7fc68be00640 1 -- 172.21.15.2:0/3085437646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea3107e50 msgr2=0x55eea31012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:19.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.248+0000 7fc68be00640 1 --2- 172.21.15.2:0/3085437646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea3107e50 0x55eea31012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.248+0000 7fc68be00640 1 -- 172.21.15.2:0/3085437646 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eea30f4770 con 0x55eea30fd590 2024-09-10T08:18:19.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.248+0000 7fc68be00640 1 --2- 172.21.15.2:0/3085437646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea3110540 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0a4370 tx=0x7fc67c0a0160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b638f7f1d6c957 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:19.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.248+0000 7fc682a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:19.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.248+0000 7fc682a00640 1 -- 172.21.15.2:0/3085437646 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c09b440 con 0x55eea30fd590 2024-09-10T08:18:19.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.248+0000 7fc682a00640 1 -- 172.21.15.2:0/3085437646 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc67c010430 con 0x55eea30fd590 2024-09-10T08:18:19.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.248+0000 7fc69e261740 1 -- 172.21.15.2:0/3085437646 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc67c099fe0 con 0x55eea30fd590 2024-09-10T08:18:19.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.249+0000 7fc682a00640 1 -- 172.21.15.2:0/3085437646 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc67c0a48a0 con 0x55eea30fd590 2024-09-10T08:18:19.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.249+0000 7fc69e261740 10 client.18210 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:19.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.249+0000 7fc69e261740 1 -- 172.21.15.2:0/3085437646 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eea3181f10 con 0x55eea30fd590 2024-09-10T08:18:19.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.249+0000 7fc682a00640 1 -- 172.21.15.2:0/3085437646 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c099fe0 con 0x55eea30fd590 2024-09-10T08:18:19.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.249+0000 7fc682a00640 1 -- 172.21.15.2:0/3085437646 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc67c0d1ce0 con 0x55eea30fd590 2024-09-10T08:18:19.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.249+0000 7fc69e261740 10 client.18210 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:19.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.249+0000 7fc69e261740 10 client.18210 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:18:19.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.249+0000 7fc69e261740 20 client.18210 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:19.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.249+0000 7fc69e261740 1 --2- 172.21.15.2:0/3085437646 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55eea3182630 0x55eea3184a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:19.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.249+0000 7fc69e261740 4 client.18210 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:18:19.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.249+0000 7fc69e261740 1 -- 172.21.15.2:0/3085437646 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eea306ae60 con 0x55eea3182630 2024-09-10T08:18:19.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.250+0000 7fc698e00640 1 --2- 172.21.15.2:0/3085437646 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55eea3182630 0x55eea3184a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:19.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.250+0000 7fc698e00640 1 --2- 172.21.15.2:0/3085437646 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55eea3182630 0x55eea3184a50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb9b8111ce0252d5 server_cookie=8481f1afcc3efeb1 in_seq=0 out_seq=0 2024-09-10T08:18:19.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.250+0000 7fc682a00640 10 client.18210 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:18:19.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.250+0000 7fc682a00640 1 -- 172.21.15.2:0/3085437646 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eea306ae60 con 0x55eea3182630 2024-09-10T08:18:19.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.250+0000 7fc682a00640 10 client.18210 handle_command_reply: tid=0 2024-09-10T08:18:19.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.250+0000 7fc69e261740 1 client.18210 shutdown 2024-09-10T08:18:19.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.250+0000 7fc69e261740 1 -- 172.21.15.2:0/3085437646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 msgr2=0x55eea3110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:19.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.250+0000 7fc69e261740 1 --2- 172.21.15.2:0/3085437646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea3110540 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0a4370 tx=0x7fc67c0a0160 comp rx=0 tx=0).stop 2024-09-10T08:18:19.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc69e261740 1 -- 172.21.15.2:0/3085437646 shutdown_connections 2024-09-10T08:18:19.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc69e261740 1 --2- 172.21.15.2:0/3085437646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eea3107e50 0x55eea31012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc69e261740 1 --2- 172.21.15.2:0/3085437646 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55eea3182630 0x55eea3184a50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc69e261740 1 --2- 172.21.15.2:0/3085437646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eea30fd590 0x55eea3110540 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc69e261740 1 --2- 172.21.15.2:0/3085437646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eea30fcc70 0x55eea3110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc69e261740 1 -- 172.21.15.2:0/3085437646 >> 172.21.15.2:0/3085437646 conn(0x55eea30f9200 msgr2=0x55eea30f7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:19.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc682a00640 0 client.18210 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:18:19.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc69e261740 1 -- 172.21.15.2:0/3085437646 shutdown_connections 2024-09-10T08:18:19.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc69e261740 1 -- 172.21.15.2:0/3085437646 wait complete. 2024-09-10T08:18:19.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc69e261740 20 client.18210 trim_cache size 0 max 16384 2024-09-10T08:18:19.256 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:18:19.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc69b800640 1 -- 172.21.15.2:0/1930873346 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6780a2b80 msgr2=0x7fc6780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:19.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc69b800640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6780a2b80 0x7fc6780a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc68c011630 tx=0x7fc68c009500 comp rx=0 tx=0).stop 2024-09-10T08:18:19.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc69b800640 1 -- 172.21.15.2:0/1930873346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6941049a0 msgr2=0x7fc69419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:19.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.251+0000 7fc69b800640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6941049a0 0x7fc69419ab30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00e9f0 tx=0x7fc67c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.252+0000 7fc69b800640 1 -- 172.21.15.2:0/1930873346 shutdown_connections 2024-09-10T08:18:19.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.252+0000 7fc69b800640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6780a2b80 0x7fc6780a5040 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.252+0000 7fc69b800640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc694109120 0x7fc69419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.252+0000 7fc69b800640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6941049a0 0x7fc69419ab30 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.252+0000 7fc69b800640 1 --2- 172.21.15.2:0/1930873346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc694103ff0 0x7fc69419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:19.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.252+0000 7fc69b800640 1 -- 172.21.15.2:0/1930873346 >> 172.21.15.2:0/1930873346 conn(0x7fc6940ffa90 msgr2=0x7fc694100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:19.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.252+0000 7fc69b800640 1 -- 172.21.15.2:0/1930873346 shutdown_connections 2024-09-10T08:18:19.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:19.252+0000 7fc69b800640 1 -- 172.21.15.2:0/1930873346 wait complete. 2024-09-10T08:18:19.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:19 smithi155 ceph-mon[35405]: pgmap v692: 129 pgs: 129 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:18:19.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:19 smithi107 ceph-mon[35048]: pgmap v692: 129 pgs: 129 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:18:19.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:19 smithi002 ceph-mon[30563]: pgmap v692: 129 pgs: 129 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:18:20.077 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:18:20.454 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:18:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.048+0000 7f6bf3e00640 1 -- 172.21.15.2:0/2845439302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bec077040 msgr2=0x7f6bec1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.048+0000 7f6bf3e00640 1 --2- 172.21.15.2:0/2845439302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bec077040 0x7f6bec1124f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009900 tx=0x7f6bd802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.048+0000 7f6bf3e00640 1 -- 172.21.15.2:0/2845439302 shutdown_connections 2024-09-10T08:18:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.048+0000 7f6bf3e00640 1 --2- 172.21.15.2:0/2845439302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bec112a30 0x7f6bec114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.048+0000 7f6bf3e00640 1 --2- 172.21.15.2:0/2845439302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bec077040 0x7f6bec1124f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.048+0000 7f6bf3e00640 1 --2- 172.21.15.2:0/2845439302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bec076720 0x7f6bec076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.048+0000 7f6bf3e00640 1 -- 172.21.15.2:0/2845439302 >> 172.21.15.2:0/2845439302 conn(0x7f6bec0ffa90 msgr2=0x7f6bec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.048+0000 7f6bf3e00640 1 -- 172.21.15.2:0/2845439302 shutdown_connections 2024-09-10T08:18:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.048+0000 7f6bf3e00640 1 -- 172.21.15.2:0/2845439302 wait complete. 2024-09-10T08:18:21.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.050+0000 7f6bf3e00640 1 Processor -- start 2024-09-10T08:18:21.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.050+0000 7f6bf3e00640 1 -- start start 2024-09-10T08:18:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.050+0000 7f6bf3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bec076720 0x7f6bec1a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.050+0000 7f6bf3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bec077040 0x7f6bec1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.050+0000 7f6bf3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bec112a30 0x7f6bec19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.050+0000 7f6bf3e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1185b0 con 0x7f6bec076720 2024-09-10T08:18:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.050+0000 7f6bf3e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec118430 con 0x7f6bec112a30 2024-09-10T08:18:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.050+0000 7f6bf3e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec118730 con 0x7f6bec077040 2024-09-10T08:18:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.051+0000 7f6bf1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bec076720 0x7f6bec1a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.051+0000 7f6bf1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bec076720 0x7f6bec1a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40454/0 (socket says 172.21.15.2:40454) 2024-09-10T08:18:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.051+0000 7f6bf1400640 1 -- 172.21.15.2:0/3332248585 learned_addr learned my addr 172.21.15.2:0/3332248585 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.051+0000 7f6bf0a00640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bec077040 0x7f6bec1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.051+0000 7f6bf1e00640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bec112a30 0x7f6bec19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.051+0000 7f6bf1400640 1 -- 172.21.15.2:0/3332248585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bec077040 msgr2=0x7f6bec1a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.051+0000 7f6bf1400640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bec077040 0x7f6bec1a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.051+0000 7f6bf1400640 1 -- 172.21.15.2:0/3332248585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bec112a30 msgr2=0x7f6bec19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.051+0000 7f6bf1400640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bec112a30 0x7f6bec19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.051+0000 7f6bf1400640 1 -- 172.21.15.2:0/3332248585 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bec19eff0 con 0x7f6bec076720 2024-09-10T08:18:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.051+0000 7f6bf1400640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bec076720 0x7f6bec1a4700 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6be000e9f0 tx=0x7f6be000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.052+0000 7f6bea000640 1 -- 172.21.15.2:0/3332248585 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be000cd80 con 0x7f6bec076720 2024-09-10T08:18:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.052+0000 7f6bea000640 1 -- 172.21.15.2:0/3332248585 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be0010910 con 0x7f6bec076720 2024-09-10T08:18:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.052+0000 7f6bf3e00640 1 -- 172.21.15.2:0/3332248585 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bec19f2e0 con 0x7f6bec076720 2024-09-10T08:18:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.052+0000 7f6bea000640 1 -- 172.21.15.2:0/3332248585 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0018990 con 0x7f6bec076720 2024-09-10T08:18:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.052+0000 7f6bf3e00640 1 -- 172.21.15.2:0/3332248585 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bec1ab5d0 con 0x7f6bec076720 2024-09-10T08:18:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.055+0000 7f6bf6818740 1 Processor -- start 2024-09-10T08:18:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.055+0000 7f6bea000640 1 -- 172.21.15.2:0/3332248585 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6be0010ac0 con 0x7f6bec076720 2024-09-10T08:18:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.055+0000 7f6bf6818740 1 -- start start 2024-09-10T08:18:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.055+0000 7f6bf6818740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee60c70 0x55f17ee61050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.055+0000 7f6bf6818740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee61590 0x55f17ee6b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.055+0000 7f6bf6818740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee6be50 0x55f17ee6e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.056+0000 7f6bf6818740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f17edc73e0 con 0x55f17ee6be50 2024-09-10T08:18:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.056+0000 7f6bf6818740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f17ed6ed80 con 0x55f17ee60c70 2024-09-10T08:18:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.056+0000 7f6bf6818740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f17edc33a0 con 0x55f17ee61590 2024-09-10T08:18:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.056+0000 7f6bf1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee60c70 0x55f17ee61050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.056+0000 7f6bf1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee60c70 0x55f17ee61050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:45030/0 (socket says 172.21.15.2:45030) 2024-09-10T08:18:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.056+0000 7f6bf1e00640 1 -- 172.21.15.2:0/1958749678 learned_addr learned my addr 172.21.15.2:0/1958749678 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.056+0000 7f6bf1400640 1 --2- 172.21.15.2:0/1958749678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee61590 0x55f17ee6b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.056+0000 7f6bf0a00640 1 --2- 172.21.15.2:0/1958749678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee6be50 0x55f17ee6e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.056+0000 7f6bf1400640 1 -- 172.21.15.2:0/1958749678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee60c70 msgr2=0x55f17ee61050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.056+0000 7f6bf1400640 1 --2- 172.21.15.2:0/1958749678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee60c70 0x55f17ee61050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.056+0000 7f6bf1400640 1 -- 172.21.15.2:0/1958749678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee6be50 msgr2=0x55f17ee6e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.056+0000 7f6bf1400640 1 --2- 172.21.15.2:0/1958749678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee6be50 0x55f17ee6e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.056+0000 7f6bf1400640 1 -- 172.21.15.2:0/1958749678 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17ed8dae0 con 0x55f17ee61590 2024-09-10T08:18:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.057+0000 7f6bf1400640 1 --2- 172.21.15.2:0/1958749678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee61590 0x55f17ee6b910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6be00ccdf0 tx=0x7f6be00d7670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b40395a1ded44b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.057+0000 7f6bea000640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6bd40a2b80 0x7f6bd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.057+0000 7f6bde600640 1 -- 172.21.15.2:0/1958749678 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be00de070 con 0x55f17ee61590 2024-09-10T08:18:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.057+0000 7f6bde600640 1 -- 172.21.15.2:0/1958749678 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be009bc60 con 0x55f17ee61590 2024-09-10T08:18:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.057+0000 7f6bde600640 1 -- 172.21.15.2:0/1958749678 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be009b5c0 con 0x55f17ee61590 2024-09-10T08:18:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.057+0000 7f6bea000640 1 -- 172.21.15.2:0/3332248585 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6be00cf7a0 con 0x7f6bec076720 2024-09-10T08:18:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.057+0000 7f6bf1e00640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6bd40a2b80 0x7f6bd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.058+0000 7f6bf6818740 1 -- 172.21.15.2:0/1958749678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee61590 msgr2=0x55f17ee6b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.058+0000 7f6bf1e00640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6bd40a2b80 0x7f6bd40a5040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6be40049d0 tx=0x7f6be4009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.058+0000 7f6bf6818740 1 --2- 172.21.15.2:0/1958749678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee61590 0x55f17ee6b910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6be00ccdf0 tx=0x7f6be00d7670 comp rx=0 tx=0).stop 2024-09-10T08:18:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.058+0000 7f6bf6818740 1 -- 172.21.15.2:0/1958749678 shutdown_connections 2024-09-10T08:18:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.058+0000 7f6bf6818740 1 --2- 172.21.15.2:0/1958749678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee6be50 0x55f17ee6e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.058+0000 7f6bf6818740 1 --2- 172.21.15.2:0/1958749678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee61590 0x55f17ee6b910 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.058+0000 7f6bf6818740 1 --2- 172.21.15.2:0/1958749678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee60c70 0x55f17ee61050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.058+0000 7f6bf6818740 1 -- 172.21.15.2:0/1958749678 >> 172.21.15.2:0/1958749678 conn(0x55f17ee5d200 msgr2=0x55f17ee5d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.058+0000 7f6bf6818740 1 -- 172.21.15.2:0/1958749678 shutdown_connections 2024-09-10T08:18:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.058+0000 7f6bf6818740 1 -- 172.21.15.2:0/1958749678 wait complete. 2024-09-10T08:18:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.059+0000 7f6bf6818740 1 -- start start 2024-09-10T08:18:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.059+0000 7f6bf6818740 1 Processor -- start 2024-09-10T08:18:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf6818740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 0x55f17ee68c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf6818740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee61590 0x55f17ee69190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf6818740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee6be50 0x55f17ee696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf6818740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f17ee6ea20 con 0x55f17ee60c70 2024-09-10T08:18:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf6818740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f17ed820f0 con 0x55f17ee61590 2024-09-10T08:18:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf6818740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f17ee6eba0 con 0x55f17ee6be50 2024-09-10T08:18:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf6818740 10 client.0 _ll_register_callbacks cb 0x55f17ed69bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 0x55f17ee68c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 0x55f17ee68c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40476/0 (socket says 172.21.15.2:40476) 2024-09-10T08:18:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf1400640 1 -- 172.21.15.2:0/2599588563 learned_addr learned my addr 172.21.15.2:0/2599588563 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee6be50 0x55f17ee696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf0a00640 1 --2- 172.21.15.2:0/2599588563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee61590 0x55f17ee69190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf1400640 1 -- 172.21.15.2:0/2599588563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee6be50 msgr2=0x55f17ee696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf1400640 1 --2- 172.21.15.2:0/2599588563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee6be50 0x55f17ee696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf1400640 1 -- 172.21.15.2:0/2599588563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee61590 msgr2=0x55f17ee69190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.060+0000 7f6bf1400640 1 --2- 172.21.15.2:0/2599588563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee61590 0x55f17ee69190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.061+0000 7f6bf1400640 1 -- 172.21.15.2:0/2599588563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17eb383c0 con 0x55f17ee60c70 2024-09-10T08:18:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.061+0000 7f6bf1e00640 1 --2- 172.21.15.2:0/2599588563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee6be50 0x55f17ee696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.061+0000 7f6bf1400640 1 --2- 172.21.15.2:0/2599588563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 0x55f17ee68c50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6be000e9c0 tx=0x7f6be00cfa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971c614433fc91d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.061+0000 7f6bd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.061+0000 7f6bd2a00640 1 -- 172.21.15.2:0/2599588563 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0024db0 con 0x55f17ee60c70 2024-09-10T08:18:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.061+0000 7f6bd2a00640 1 -- 172.21.15.2:0/2599588563 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6be0026000 con 0x55f17ee60c70 2024-09-10T08:18:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.061+0000 7f6bf6818740 1 -- 172.21.15.2:0/2599588563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6be009bc60 con 0x55f17ee60c70 2024-09-10T08:18:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.061+0000 7f6bd2a00640 1 -- 172.21.15.2:0/2599588563 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6be001a2d0 con 0x55f17ee60c70 2024-09-10T08:18:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.061+0000 7f6bf6818740 10 client.18222 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.061+0000 7f6bf6818740 1 -- 172.21.15.2:0/2599588563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f17ee7a980 con 0x55f17ee60c70 2024-09-10T08:18:21.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.062+0000 7f6bd2a00640 1 -- 172.21.15.2:0/2599588563 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be009bc60 con 0x55f17ee60c70 2024-09-10T08:18:21.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.062+0000 7f6bd2a00640 1 -- 172.21.15.2:0/2599588563 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6be00982d0 con 0x55f17ee60c70 2024-09-10T08:18:21.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.062+0000 7f6bf6818740 10 client.18222 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:21.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.062+0000 7f6bf6818740 10 client.18222 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:18:21.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.062+0000 7f6bf6818740 20 client.18222 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:21.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.062+0000 7f6bf6818740 1 --2- 172.21.15.2:0/2599588563 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f17ee7bcd0 0x55f17ee7e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.062+0000 7f6bf6818740 4 client.18222 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:18:21.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.062+0000 7f6bf6818740 1 -- 172.21.15.2:0/2599588563 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f17edd4950 con 0x55f17ee7bcd0 2024-09-10T08:18:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.063+0000 7f6bf0a00640 1 --2- 172.21.15.2:0/2599588563 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f17ee7bcd0 0x55f17ee7e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.063+0000 7f6bf0a00640 1 --2- 172.21.15.2:0/2599588563 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f17ee7bcd0 0x55f17ee7e0c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=889797330e332565 server_cookie=a5088f7095e2be7e in_seq=0 out_seq=0 2024-09-10T08:18:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.063+0000 7f6bd2a00640 10 client.18222 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:18:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bd2a00640 1 -- 172.21.15.2:0/2599588563 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f17edd4950 con 0x55f17ee7bcd0 2024-09-10T08:18:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bd2a00640 10 client.18222 handle_command_reply: tid=0 2024-09-10T08:18:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bf6818740 1 client.18222 shutdown 2024-09-10T08:18:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bf6818740 1 -- 172.21.15.2:0/2599588563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 msgr2=0x55f17ee68c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bf6818740 1 --2- 172.21.15.2:0/2599588563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 0x55f17ee68c50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6be000e9c0 tx=0x7f6be00cfa20 comp rx=0 tx=0).stop 2024-09-10T08:18:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bf6818740 1 -- 172.21.15.2:0/2599588563 shutdown_connections 2024-09-10T08:18:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bf6818740 1 --2- 172.21.15.2:0/2599588563 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f17ee7bcd0 0x55f17ee7e0c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bf6818740 1 --2- 172.21.15.2:0/2599588563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee6be50 0x55f17ee696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bf6818740 1 --2- 172.21.15.2:0/2599588563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee61590 0x55f17ee69190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bf6818740 1 --2- 172.21.15.2:0/2599588563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 0x55f17ee68c50 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bf6818740 1 -- 172.21.15.2:0/2599588563 >> 172.21.15.2:0/2599588563 conn(0x55f17ee5d200 msgr2=0x55f17ee62ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bd2a00640 0 client.18222 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:18:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bf6818740 1 -- 172.21.15.2:0/2599588563 shutdown_connections 2024-09-10T08:18:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.064+0000 7f6bf6818740 1 -- 172.21.15.2:0/2599588563 wait complete. 2024-09-10T08:18:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.065+0000 7f6bf6818740 20 client.18222 trim_cache size 0 max 16384 2024-09-10T08:18:21.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.079+0000 7f6bf6818740 1 Processor -- start 2024-09-10T08:18:21.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.079+0000 7f6bf6818740 1 -- start start 2024-09-10T08:18:21.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.079+0000 7f6bf6818740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 0x55f17ee65ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.079+0000 7f6bf6818740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee61590 0x55f17ee66430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.079+0000 7f6bf6818740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee6be50 0x55f17ee6a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.079+0000 7f6bf6818740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1b7e20 con 0x55f17ee60c70 2024-09-10T08:18:21.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.079+0000 7f6bf6818740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1b56f0 con 0x55f17ee61590 2024-09-10T08:18:21.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.079+0000 7f6bf6818740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1b3540 con 0x55f17ee6be50 2024-09-10T08:18:21.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.080+0000 7f6bf1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 0x55f17ee65ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.080+0000 7f6bf0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee61590 0x55f17ee66430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.080+0000 7f6bf1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 0x55f17ee65ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40484/0 (socket says 172.21.15.2:40484) 2024-09-10T08:18:21.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.080+0000 7f6bf1400640 1 -- 172.21.15.2:0/3072836886 learned_addr learned my addr 172.21.15.2:0/3072836886 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:21.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.080+0000 7f6bf1e00640 1 --2- 172.21.15.2:0/3072836886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee6be50 0x55f17ee6a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.080+0000 7f6bf1400640 1 -- 172.21.15.2:0/3072836886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee6be50 msgr2=0x55f17ee6a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:21.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.080+0000 7f6bf1400640 1 --2- 172.21.15.2:0/3072836886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee6be50 0x55f17ee6a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.080+0000 7f6bf1400640 1 -- 172.21.15.2:0/3072836886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee61590 msgr2=0x55f17ee66430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:21.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.080+0000 7f6bf1400640 1 --2- 172.21.15.2:0/3072836886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee61590 0x55f17ee66430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.080+0000 7f6bf1400640 1 -- 172.21.15.2:0/3072836886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17ee613c0 con 0x55f17ee60c70 2024-09-10T08:18:21.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.080+0000 7f6bf1400640 1 --2- 172.21.15.2:0/3072836886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 0x55f17ee65ef0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6be00cfa90 tx=0x7f6be004ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c99d22afb15d8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:21.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.081+0000 7f6bddc00640 1 -- 172.21.15.2:0/3072836886 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be00de020 con 0x55f17ee60c70 2024-09-10T08:18:21.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.081+0000 7f6bddc00640 1 -- 172.21.15.2:0/3072836886 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be001a110 con 0x55f17ee60c70 2024-09-10T08:18:21.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.081+0000 7f6bddc00640 1 -- 172.21.15.2:0/3072836886 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be001a4c0 con 0x55f17ee60c70 2024-09-10T08:18:21.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.081+0000 7f6bf6818740 1 -- 172.21.15.2:0/3072836886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 msgr2=0x55f17ee65ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.081+0000 7f6bf6818740 1 --2- 172.21.15.2:0/3072836886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 0x55f17ee65ef0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6be00cfa90 tx=0x7f6be004ad20 comp rx=0 tx=0).stop 2024-09-10T08:18:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.082+0000 7f6bf6818740 1 -- 172.21.15.2:0/3072836886 shutdown_connections 2024-09-10T08:18:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.082+0000 7f6bf6818740 1 --2- 172.21.15.2:0/3072836886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee6be50 0x55f17ee6a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.082+0000 7f6bf6818740 1 --2- 172.21.15.2:0/3072836886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee61590 0x55f17ee66430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.082+0000 7f6bf6818740 1 --2- 172.21.15.2:0/3072836886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee60c70 0x55f17ee65ef0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.082+0000 7f6bf6818740 1 -- 172.21.15.2:0/3072836886 >> 172.21.15.2:0/3072836886 conn(0x55f17ee5d200 msgr2=0x55f17ee5c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.082+0000 7f6bf6818740 1 -- 172.21.15.2:0/3072836886 shutdown_connections 2024-09-10T08:18:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.082+0000 7f6bf6818740 1 -- 172.21.15.2:0/3072836886 wait complete. 2024-09-10T08:18:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.082+0000 7f6bf6818740 1 -- start start 2024-09-10T08:18:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.082+0000 7f6bf6818740 1 Processor -- start 2024-09-10T08:18:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.083+0000 7f6bf6818740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee60c70 0x55f17ee7c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.083+0000 7f6bf6818740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee61590 0x55f17ee7cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.083+0000 7f6bf6818740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee6be50 0x55f17eede5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.083+0000 7f6bf6818740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1b3bb0 con 0x55f17ee6be50 2024-09-10T08:18:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.083+0000 7f6bf6818740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1b5020 con 0x55f17ee60c70 2024-09-10T08:18:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.083+0000 7f6bf6818740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1b4280 con 0x55f17ee61590 2024-09-10T08:18:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.083+0000 7f6bf6818740 10 client.0 _ll_register_callbacks cb 0x7f6bd8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.083+0000 7f6bf0a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee61590 0x55f17ee7cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.083+0000 7f6bf1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee60c70 0x55f17ee7c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.083+0000 7f6bf0a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee61590 0x55f17ee7cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:45580/0 (socket says 172.21.15.2:45580) 2024-09-10T08:18:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.083+0000 7f6bf0a00640 1 -- 172.21.15.2:0/2168046520 learned_addr learned my addr 172.21.15.2:0/2168046520 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.083+0000 7f6bf1e00640 1 --2- 172.21.15.2:0/2168046520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee6be50 0x55f17eede5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.084+0000 7f6bf0a00640 1 -- 172.21.15.2:0/2168046520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee60c70 msgr2=0x55f17ee7c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:21.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.084+0000 7f6bf0a00640 1 --2- 172.21.15.2:0/2168046520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee60c70 0x55f17ee7c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.084+0000 7f6bf0a00640 1 -- 172.21.15.2:0/2168046520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee6be50 msgr2=0x55f17eede5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:21.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.084+0000 7f6bf0a00640 1 --2- 172.21.15.2:0/2168046520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee6be50 0x55f17eede5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.084+0000 7f6bf0a00640 1 -- 172.21.15.2:0/2168046520 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17eedec00 con 0x55f17ee61590 2024-09-10T08:18:21.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.084+0000 7f6bf0a00640 1 --2- 172.21.15.2:0/2168046520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee61590 0x55f17ee7cd10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009900 tx=0x7f6bd8031aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f47094ac44d343 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:21.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.084+0000 7f6bd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:18:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.084+0000 7f6bd2a00640 1 -- 172.21.15.2:0/2168046520 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8035040 con 0x55f17ee61590 2024-09-10T08:18:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.084+0000 7f6bd2a00640 1 -- 172.21.15.2:0/2168046520 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6bd8004450 con 0x55f17ee61590 2024-09-10T08:18:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.084+0000 7f6bf6818740 1 -- 172.21.15.2:0/2168046520 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f17edcee60 con 0x55f17ee61590 2024-09-10T08:18:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.084+0000 7f6bd2a00640 1 -- 172.21.15.2:0/2168046520 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6bd8058050 con 0x55f17ee61590 2024-09-10T08:18:21.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.084+0000 7f6bf6818740 10 client.28192 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.084+0000 7f6bf6818740 1 -- 172.21.15.2:0/2168046520 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f17eee5fc0 con 0x55f17ee61590 2024-09-10T08:18:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.085+0000 7f6bd2a00640 1 -- 172.21.15.2:0/2168046520 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f17edcee60 con 0x55f17ee61590 2024-09-10T08:18:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.085+0000 7f6bd2a00640 1 -- 172.21.15.2:0/2168046520 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6bd8004c90 con 0x55f17ee61590 2024-09-10T08:18:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.085+0000 7f6bf6818740 10 client.28192 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.085+0000 7f6bf6818740 10 client.28192 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:18:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.085+0000 7f6bf6818740 20 client.28192 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.085+0000 7f6bf6818740 1 --2- 172.21.15.2:0/2168046520 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f17eee67c0 0x55f17eee8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.085+0000 7f6bf6818740 4 client.28192 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:18:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.085+0000 7f6bf6818740 1 -- 172.21.15.2:0/2168046520 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f17ed4b010 con 0x55f17eee67c0 2024-09-10T08:18:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.085+0000 7f6bf1400640 1 --2- 172.21.15.2:0/2168046520 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f17eee67c0 0x55f17eee8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bf1400640 1 --2- 172.21.15.2:0/2168046520 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f17eee67c0 0x55f17eee8be0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d09679c2cbd161d1 server_cookie=5d9e9c6e39e72a12 in_seq=0 out_seq=0 2024-09-10T08:18:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bd2a00640 10 client.28192 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bd2a00640 1 -- 172.21.15.2:0/2168046520 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f17ed4b010 con 0x55f17eee67c0 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bd2a00640 10 client.28192 handle_command_reply: tid=0 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bf6818740 1 client.28192 shutdown 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bf6818740 1 -- 172.21.15.2:0/2168046520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee61590 msgr2=0x55f17ee7cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bf6818740 1 --2- 172.21.15.2:0/2168046520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee61590 0x55f17ee7cd10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009900 tx=0x7f6bd8031aa0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bf6818740 1 -- 172.21.15.2:0/2168046520 shutdown_connections 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bf6818740 1 --2- 172.21.15.2:0/2168046520 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55f17eee67c0 0x55f17eee8be0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bf6818740 1 --2- 172.21.15.2:0/2168046520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f17ee6be50 0x55f17eede5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bf6818740 1 --2- 172.21.15.2:0/2168046520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17ee61590 0x55f17ee7cd10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bf6818740 1 --2- 172.21.15.2:0/2168046520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f17ee60c70 0x55f17ee7c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bd2a00640 0 client.28192 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.086+0000 7f6bf6818740 1 -- 172.21.15.2:0/2168046520 >> 172.21.15.2:0/2168046520 conn(0x55f17ee5d200 msgr2=0x55f17ee76df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.087+0000 7f6bf6818740 1 -- 172.21.15.2:0/2168046520 shutdown_connections 2024-09-10T08:18:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.087+0000 7f6bf6818740 1 -- 172.21.15.2:0/2168046520 wait complete. 2024-09-10T08:18:21.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.087+0000 7f6bf6818740 20 client.28192 trim_cache size 0 max 16384 2024-09-10T08:18:21.091 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:18:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.087+0000 7f6bf3e00640 1 -- 172.21.15.2:0/3332248585 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6bd40a2b80 msgr2=0x7f6bd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.087+0000 7f6bf3e00640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6bd40a2b80 0x7f6bd40a5040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6be40049d0 tx=0x7f6be4009600 comp rx=0 tx=0).stop 2024-09-10T08:18:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.087+0000 7f6bf3e00640 1 -- 172.21.15.2:0/3332248585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bec076720 msgr2=0x7f6bec1a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.087+0000 7f6bf3e00640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bec076720 0x7f6bec1a4700 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6be000e9f0 tx=0x7f6be000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.088+0000 7f6bf3e00640 1 -- 172.21.15.2:0/3332248585 shutdown_connections 2024-09-10T08:18:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.088+0000 7f6bf3e00640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bec112a30 0x7f6bec19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.088+0000 7f6bf3e00640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6bd40a2b80 0x7f6bd40a5040 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.088+0000 7f6bf3e00640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bec077040 0x7f6bec1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.088+0000 7f6bf3e00640 1 --2- 172.21.15.2:0/3332248585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bec076720 0x7f6bec1a4700 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.088+0000 7f6bf3e00640 1 -- 172.21.15.2:0/3332248585 >> 172.21.15.2:0/3332248585 conn(0x7f6bec0ffa90 msgr2=0x7f6bec101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.088+0000 7f6bf3e00640 1 -- 172.21.15.2:0/3332248585 shutdown_connections 2024-09-10T08:18:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:21.088+0000 7f6bf3e00640 1 -- 172.21.15.2:0/3332248585 wait complete. 2024-09-10T08:18:21.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:21 smithi002 ceph-mon[30563]: pgmap v693: 129 pgs: 129 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:18:21.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:21 smithi155 ceph-mon[35405]: pgmap v693: 129 pgs: 129 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:18:21.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:21 smithi107 ceph-mon[35048]: pgmap v693: 129 pgs: 129 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:18:21.961 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:18:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:18:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.923+0000 7fe3c9000640 1 -- 172.21.15.2:0/4175642929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c41079c0 msgr2=0x7fe3c4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.923+0000 7fe3c9000640 1 --2- 172.21.15.2:0/4175642929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c41079c0 0x7fe3c4107da0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009930 tx=0x7fe3b402a420 comp rx=0 tx=0).stop 2024-09-10T08:18:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.923+0000 7fe3c9000640 1 -- 172.21.15.2:0/4175642929 shutdown_connections 2024-09-10T08:18:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.923+0000 7fe3c9000640 1 --2- 172.21.15.2:0/4175642929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3c4102360 0x7fe3c410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.923+0000 7fe3c9000640 1 --2- 172.21.15.2:0/4175642929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c41019c0 0x7fe3c4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.923+0000 7fe3c9000640 1 --2- 172.21.15.2:0/4175642929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c41079c0 0x7fe3c4107da0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.923+0000 7fe3c9000640 1 -- 172.21.15.2:0/4175642929 >> 172.21.15.2:0/4175642929 conn(0x7fe3c40fd740 msgr2=0x7fe3c40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.923+0000 7fe3c9000640 1 -- 172.21.15.2:0/4175642929 shutdown_connections 2024-09-10T08:18:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.923+0000 7fe3c9000640 1 -- 172.21.15.2:0/4175642929 wait complete. 2024-09-10T08:18:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.924+0000 7fe3c9000640 1 Processor -- start 2024-09-10T08:18:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.925+0000 7fe3c9000640 1 -- start start 2024-09-10T08:18:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.925+0000 7fe3c9000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c41019c0 0x7fe3c419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.925+0000 7fe3c9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3c4102360 0x7fe3c419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.925+0000 7fe3c9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c41079c0 0x7fe3c419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.925+0000 7fe3c9000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4111dc0 con 0x7fe3c41019c0 2024-09-10T08:18:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.925+0000 7fe3c9000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4111c40 con 0x7fe3c4102360 2024-09-10T08:18:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.925+0000 7fe3c9000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4111f40 con 0x7fe3c41079c0 2024-09-10T08:18:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.925+0000 7fe3c3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c41019c0 0x7fe3c419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.926+0000 7fe3c3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c41019c0 0x7fe3c419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40500/0 (socket says 172.21.15.2:40500) 2024-09-10T08:18:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.926+0000 7fe3c3400640 1 -- 172.21.15.2:0/964692610 learned_addr learned my addr 172.21.15.2:0/964692610 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.926+0000 7fe3c3e00640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c41079c0 0x7fe3c419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.926+0000 7fe3bbe00640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3c4102360 0x7fe3c419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.926+0000 7fe3c3400640 1 -- 172.21.15.2:0/964692610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c41079c0 msgr2=0x7fe3c419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.926+0000 7fe3c3400640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c41079c0 0x7fe3c419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.926+0000 7fe3c3400640 1 -- 172.21.15.2:0/964692610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3c4102360 msgr2=0x7fe3c419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.926+0000 7fe3c3400640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3c4102360 0x7fe3c419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.926+0000 7fe3c3400640 1 -- 172.21.15.2:0/964692610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c419f4d0 con 0x7fe3c41019c0 2024-09-10T08:18:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.926+0000 7fe3c3e00640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c41079c0 0x7fe3c419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.927+0000 7fe3c3400640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c41019c0 0x7fe3c419a520 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009230 tx=0x7fe3b4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.927+0000 7fb1b9000640 1 -- 172.21.15.2:0/4068605818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1b4107830 msgr2=0x7fb1b4109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.927+0000 7fb1b9000640 1 --2- 172.21.15.2:0/4068605818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1b4107830 0x7fb1b4109ca0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009be0 tx=0x7fb1ac02a6c0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.927+0000 7fe3c0c00640 1 -- 172.21.15.2:0/964692610 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b403f030 con 0x7fe3c41019c0 2024-09-10T08:18:22.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.927+0000 7fe3c9000640 1 -- 172.21.15.2:0/964692610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c419f760 con 0x7fe3c41019c0 2024-09-10T08:18:22.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.927+0000 7fe3c0c00640 1 -- 172.21.15.2:0/964692610 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3b4004440 con 0x7fe3c41019c0 2024-09-10T08:18:22.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.927+0000 7fe3c0c00640 1 -- 172.21.15.2:0/964692610 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4035070 con 0x7fe3c41019c0 2024-09-10T08:18:22.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.927+0000 7fe3c9000640 1 -- 172.21.15.2:0/964692610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c41a70a0 con 0x7fe3c41019c0 2024-09-10T08:18:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.928+0000 7fb1b9000640 1 -- 172.21.15.2:0/4068605818 shutdown_connections 2024-09-10T08:18:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.928+0000 7fb1b9000640 1 --2- 172.21.15.2:0/4068605818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1b4107830 0x7fb1b4109ca0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.928+0000 7fb1b9000640 1 --2- 172.21.15.2:0/4068605818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1b4104f00 0x7fb1b41072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.928+0000 7fb1b9000640 1 --2- 172.21.15.2:0/4068605818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1b41025d0 0x7fb1b41049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.928+0000 7fb1b9000640 1 -- 172.21.15.2:0/4068605818 >> 172.21.15.2:0/4068605818 conn(0x7fb1b40fc1b0 msgr2=0x7fb1b40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.928+0000 7fb1b9000640 1 -- 172.21.15.2:0/4068605818 shutdown_connections 2024-09-10T08:18:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.928+0000 7fb1b9000640 1 -- 172.21.15.2:0/4068605818 wait complete. 2024-09-10T08:18:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.929+0000 7fb1b9000640 1 Processor -- start 2024-09-10T08:18:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.929+0000 7fb1b9000640 1 -- start start 2024-09-10T08:18:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.929+0000 7fb1b9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1b41025d0 0x7fb1b4101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.929+0000 7fb1b9000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1b4104f00 0x7fb1b4100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.929+0000 7fb1b9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1b4107830 0x7fb1b4100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.929+0000 7fb1b9000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b410d7f0 con 0x7fb1b4104f00 2024-09-10T08:18:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.929+0000 7fb1b9000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b410d670 con 0x7fb1b41025d0 2024-09-10T08:18:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.929+0000 7fb1b9000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b410d970 con 0x7fb1b4107830 2024-09-10T08:18:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.930+0000 7fb1b1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1b4104f00 0x7fb1b4100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.930+0000 7fb1b1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1b41025d0 0x7fb1b4101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.930+0000 7fb1b1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1b4104f00 0x7fb1b4100580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40508/0 (socket says 172.21.15.2:40508) 2024-09-10T08:18:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.930+0000 7fb1b1400640 1 -- 172.21.15.2:0/2039323796 learned_addr learned my addr 172.21.15.2:0/2039323796 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.930+0000 7fb1b1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1b41025d0 0x7fb1b4101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:45098/0 (socket says 172.21.15.2:45098) 2024-09-10T08:18:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.930+0000 7fb1b2800640 1 --2- 172.21.15.2:0/2039323796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1b4107830 0x7fb1b4100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.930+0000 7fb1b1400640 1 -- 172.21.15.2:0/2039323796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1b4107830 msgr2=0x7fb1b4100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.930+0000 7fb1b1400640 1 --2- 172.21.15.2:0/2039323796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1b4107830 0x7fb1b4100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.930+0000 7fb1b1400640 1 -- 172.21.15.2:0/2039323796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1b41025d0 msgr2=0x7fb1b4101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.930+0000 7fb1b1400640 1 --2- 172.21.15.2:0/2039323796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1b41025d0 0x7fb1b4101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.930+0000 7fb1b1400640 1 -- 172.21.15.2:0/2039323796 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b4101350 con 0x7fb1b4104f00 2024-09-10T08:18:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.931+0000 7fb1b1400640 1 --2- 172.21.15.2:0/2039323796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1b4104f00 0x7fb1b4100580 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000e9c0 tx=0x7fb1a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.931+0000 7fb1aaa00640 1 -- 172.21.15.2:0/2039323796 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a000cd80 con 0x7fb1b4104f00 2024-09-10T08:18:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.931+0000 7fb1b9000640 1 -- 172.21.15.2:0/2039323796 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b41a1ef0 con 0x7fb1b4104f00 2024-09-10T08:18:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.931+0000 7fb1aaa00640 1 -- 172.21.15.2:0/2039323796 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a0005500 con 0x7fb1b4104f00 2024-09-10T08:18:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.931+0000 7fb1aaa00640 1 -- 172.21.15.2:0/2039323796 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0010690 con 0x7fb1b4104f00 2024-09-10T08:18:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.931+0000 7fb1b9000640 1 -- 172.21.15.2:0/2039323796 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b41a2360 con 0x7fb1b4104f00 2024-09-10T08:18:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.931+0000 7fe3cd279740 1 Processor -- start 2024-09-10T08:18:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.931+0000 7fe3c0c00640 1 -- 172.21.15.2:0/964692610 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe3b4042030 con 0x7fe3c41019c0 2024-09-10T08:18:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.931+0000 7fe3cd279740 1 -- start start 2024-09-10T08:18:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.932+0000 7fe3cd279740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e6d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.932+0000 7fe3cd279740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6d680 0x55e5a0e802f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.932+0000 7fe3cd279740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e82c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.932+0000 7fe3cd279740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e5a0dd33e0 con 0x55e5a0e6cd60 2024-09-10T08:18:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.932+0000 7fe3cd279740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e5a0d7ad80 con 0x55e5a0e6d680 2024-09-10T08:18:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.932+0000 7fe3cd279740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e5a0dcf3a0 con 0x55e5a0e80830 2024-09-10T08:18:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.932+0000 7fe3bbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e82c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.932+0000 7fe3bbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e82c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:45626/0 (socket says 172.21.15.2:45626) 2024-09-10T08:18:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.932+0000 7fe3bbe00640 1 -- 172.21.15.2:0/944173532 learned_addr learned my addr 172.21.15.2:0/944173532 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.932+0000 7fe3c3400640 1 --2- 172.21.15.2:0/944173532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6d680 0x55e5a0e802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.932+0000 7fe3c3e00640 1 --2- 172.21.15.2:0/944173532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e6d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.933+0000 7fe3bbe00640 1 -- 172.21.15.2:0/944173532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6d680 msgr2=0x55e5a0e802f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.933+0000 7fe3bbe00640 1 --2- 172.21.15.2:0/944173532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6d680 0x55e5a0e802f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.933+0000 7fe3bbe00640 1 -- 172.21.15.2:0/944173532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6cd60 msgr2=0x55e5a0e6d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.933+0000 7fe3bbe00640 1 --2- 172.21.15.2:0/944173532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e6d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.933+0000 7fe3bbe00640 1 -- 172.21.15.2:0/944173532 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a0d99ae0 con 0x55e5a0e80830 2024-09-10T08:18:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.933+0000 7fe3c3400640 1 --2- 172.21.15.2:0/944173532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6d680 0x55e5a0e802f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:18:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.933+0000 7fe3c0c00640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3a80a2c00 0x7fe3a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.933+0000 7fe3c0c00640 1 -- 172.21.15.2:0/964692610 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe3b40ef6d0 con 0x7fe3c41019c0 2024-09-10T08:18:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.933+0000 7fe3bbe00640 1 --2- 172.21.15.2:0/944173532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e82c20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009ca0 tx=0x7fe3ac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6f016865cf301d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.933+0000 7fe3c3e00640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3a80a2c00 0x7fe3a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.934+0000 7fe3b8c00640 1 -- 172.21.15.2:0/944173532 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac01e070 con 0x55e5a0e80830 2024-09-10T08:18:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.934+0000 7fe3b8c00640 1 -- 172.21.15.2:0/944173532 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3ac019b10 con 0x55e5a0e80830 2024-09-10T08:18:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.934+0000 7fe3b8c00640 1 -- 172.21.15.2:0/944173532 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00ea30 con 0x55e5a0e80830 2024-09-10T08:18:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.934+0000 7fe3c3e00640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3a80a2c00 0x7fe3a80a50c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc005fd0 tx=0x7fe3bc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.935+0000 7fe3cd279740 1 -- 172.21.15.2:0/944173532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 msgr2=0x55e5a0e82c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.935+0000 7fe3cd279740 1 --2- 172.21.15.2:0/944173532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e82c20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009ca0 tx=0x7fe3ac011810 comp rx=0 tx=0).stop 2024-09-10T08:18:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.935+0000 7fe3cd279740 1 -- 172.21.15.2:0/944173532 shutdown_connections 2024-09-10T08:18:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.935+0000 7fe3cd279740 1 --2- 172.21.15.2:0/944173532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e82c20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.935+0000 7fe3cd279740 1 --2- 172.21.15.2:0/944173532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6d680 0x55e5a0e802f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.935+0000 7fe3cd279740 1 --2- 172.21.15.2:0/944173532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e6d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.935+0000 7fe3cd279740 1 -- 172.21.15.2:0/944173532 >> 172.21.15.2:0/944173532 conn(0x55e5a0e693d0 msgr2=0x55e5a0e697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.935+0000 7fe3cd279740 1 -- 172.21.15.2:0/944173532 shutdown_connections 2024-09-10T08:18:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.935+0000 7fe3cd279740 1 -- 172.21.15.2:0/944173532 wait complete. 2024-09-10T08:18:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.936+0000 7fb1bbfc2740 1 Processor -- start 2024-09-10T08:18:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.936+0000 7fb1bbfc2740 1 -- start start 2024-09-10T08:18:22.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.936+0000 7fe3cd279740 1 -- start start 2024-09-10T08:18:22.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.936+0000 7fe3cd279740 1 Processor -- start 2024-09-10T08:18:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.936+0000 7fb1bbfc2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c090 0x55f51600c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.936+0000 7fb1bbfc2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f516011490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.936+0000 7fb1bbfc2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 0x55f516013f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.936+0000 7fb1bbfc2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f515f71100 con 0x55f51600c9d0 2024-09-10T08:18:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.936+0000 7fb1bbfc2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f515f191b0 con 0x55f516011a60 2024-09-10T08:18:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.936+0000 7fb1bbfc2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f515f6d0c0 con 0x55f51600c090 2024-09-10T08:18:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c090 0x55f51600c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c090 0x55f51600c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:45630/0 (socket says 172.21.15.2:45630) 2024-09-10T08:18:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fb1b2800640 1 -- 172.21.15.2:0/715481630 learned_addr learned my addr 172.21.15.2:0/715481630 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3cd279740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e7d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3cd279740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6d680 0x55e5a0e7dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3cd279740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e80830 0x55e5a0e7e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3cd279740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e5a0e833a0 con 0x55e5a0e6d680 2024-09-10T08:18:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3cd279740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e5a0d8e0f0 con 0x55e5a0e80830 2024-09-10T08:18:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3cd279740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e5a0e83520 con 0x55e5a0e6cd60 2024-09-10T08:18:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3cd279740 10 client.0 _ll_register_callbacks cb 0x55e5a0d75bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fb1b1e00640 1 --2- 172.21.15.2:0/715481630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f516011490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3c3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e7d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3c3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e7d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:45642/0 (socket says 172.21.15.2:45642) 2024-09-10T08:18:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3c3400640 1 -- 172.21.15.2:0/903855993 learned_addr learned my addr 172.21.15.2:0/903855993 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3bbe00640 1 --2- 172.21.15.2:0/903855993 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6d680 0x55e5a0e7dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3c3e00640 1 --2- 172.21.15.2:0/903855993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e80830 0x55e5a0e7e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fb1b2800640 1 -- 172.21.15.2:0/715481630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 msgr2=0x55f516013f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:18:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fb1b2800640 1 --2- 172.21.15.2:0/715481630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 0x55f516013f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fb1b2800640 1 -- 172.21.15.2:0/715481630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 msgr2=0x55f516011490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fb1b2800640 1 --2- 172.21.15.2:0/715481630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f516011490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fb1b2800640 1 -- 172.21.15.2:0/715481630 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f515c84210 con 0x55f51600c090 2024-09-10T08:18:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3c3400640 1 -- 172.21.15.2:0/903855993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e80830 msgr2=0x55e5a0e7e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fb1aaa00640 1 -- 172.21.15.2:0/2039323796 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb1a00046c0 con 0x7fb1b4104f00 2024-09-10T08:18:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.937+0000 7fe3c3400640 1 --2- 172.21.15.2:0/903855993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e80830 0x55e5a0e7e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fe3c3400640 1 -- 172.21.15.2:0/903855993 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6d680 msgr2=0x55e5a0e7dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fe3c3400640 1 --2- 172.21.15.2:0/903855993 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6d680 0x55e5a0e7dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fe3c3400640 1 -- 172.21.15.2:0/903855993 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a0b443c0 con 0x55e5a0e6cd60 2024-09-10T08:18:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fe3c3400640 1 --2- 172.21.15.2:0/903855993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e7d7d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40bb890 tx=0x7fe3b40bb790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=421a22a1fb48333b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fe3b1600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:18:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fb1b2800640 1 --2- 172.21.15.2:0/715481630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c090 0x55f51600c490 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb1b406b3e0 tx=0x7fb1ac005eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4379b578c947a6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fe3b1600640 1 -- 172.21.15.2:0/903855993 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4049070 con 0x55e5a0e6cd60 2024-09-10T08:18:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fe3cd279740 1 -- 172.21.15.2:0/903855993 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3ac019b10 con 0x55e5a0e6cd60 2024-09-10T08:18:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fe3b1600640 1 -- 172.21.15.2:0/903855993 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe3b40b8200 con 0x55e5a0e6cd60 2024-09-10T08:18:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fb19f400640 1 -- 172.21.15.2:0/715481630 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac041030 con 0x55f51600c090 2024-09-10T08:18:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fb19f400640 1 -- 172.21.15.2:0/715481630 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1ac030c70 con 0x55f51600c090 2024-09-10T08:18:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fb19f400640 1 -- 172.21.15.2:0/715481630 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac036040 con 0x55f51600c090 2024-09-10T08:18:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fe3b1600640 1 -- 172.21.15.2:0/903855993 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3b40c4750 con 0x55e5a0e6cd60 2024-09-10T08:18:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.938+0000 7fe3cd279740 10 client.28213 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.939+0000 7fe3cd279740 1 -- 172.21.15.2:0/903855993 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5a0e833a0 con 0x55e5a0e6cd60 2024-09-10T08:18:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.939+0000 7fe3b1600640 1 -- 172.21.15.2:0/903855993 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3ac019b10 con 0x55e5a0e6cd60 2024-09-10T08:18:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.939+0000 7fe3b1600640 1 -- 172.21.15.2:0/903855993 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe3b40c42c0 con 0x55e5a0e6cd60 2024-09-10T08:18:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.939+0000 7fe3cd279740 10 client.28213 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.939+0000 7fe3cd279740 10 client.28213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.939+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/715481630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c090 msgr2=0x55f51600c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.939+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/715481630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c090 0x55f51600c490 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb1b406b3e0 tx=0x7fb1ac005eb0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.939+0000 7fe3cd279740 20 client.28213 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:22.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.939+0000 7fe3cd279740 1 --2- 172.21.15.2:0/903855993 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e5a0e908e0 0x55e5a0e92d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.939+0000 7fe3cd279740 4 client.28213 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:18:22.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.939+0000 7fe3cd279740 1 -- 172.21.15.2:0/903855993 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5a0de0950 con 0x55e5a0e908e0 2024-09-10T08:18:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.940+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/715481630 shutdown_connections 2024-09-10T08:18:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.940+0000 7fb1aaa00640 1 --2- 172.21.15.2:0/2039323796 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb1980a2cb0 0x7fb1980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.940+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/715481630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 0x55f516013f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.940+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/715481630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f516011490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.940+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/715481630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c090 0x55f51600c490 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.940+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/715481630 >> 172.21.15.2:0/715481630 conn(0x55f516007700 msgr2=0x55f516007b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.940+0000 7fb1aaa00640 1 -- 172.21.15.2:0/2039323796 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb1a00cf770 con 0x7fb1b4104f00 2024-09-10T08:18:22.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.940+0000 7fe3bbe00640 1 --2- 172.21.15.2:0/903855993 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e5a0e908e0 0x55e5a0e92d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.940+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/715481630 shutdown_connections 2024-09-10T08:18:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.940+0000 7fb1b2800640 1 --2- 172.21.15.2:0/2039323796 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb1980a2cb0 0x7fb1980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.940+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/715481630 wait complete. 2024-09-10T08:18:22.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fe3b1600640 10 client.28213 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:22.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fe3bbe00640 1 --2- 172.21.15.2:0/903855993 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e5a0e908e0 0x55e5a0e92d10 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=851b76c707ba7e9a server_cookie=3b3b65b684388a4f in_seq=0 out_seq=0 2024-09-10T08:18:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fb1bbfc2740 1 -- start start 2024-09-10T08:18:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fb1b2800640 1 --2- 172.21.15.2:0/2039323796 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb1980a2cb0 0x7fb1980a5170 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac030960 tx=0x7fb1ac005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fb1bbfc2740 1 Processor -- start 2024-09-10T08:18:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fb1bbfc2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f51600c090 0x55f516016210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fb1bbfc2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f516016750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fb1bbfc2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f516011a60 0x55f51601d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fe3b1600640 1 -- 172.21.15.2:0/903855993 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5a0de0950 con 0x55e5a0e908e0 2024-09-10T08:18:22.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fe3b1600640 10 client.28213 handle_command_reply: tid=0 2024-09-10T08:18:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fb1bbfc2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f5160149d0 con 0x55f51600c9d0 2024-09-10T08:18:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fb1bbfc2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f515efa090 con 0x55f51600c090 2024-09-10T08:18:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fb1bbfc2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f516014b50 con 0x55f516011a60 2024-09-10T08:18:22.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.941+0000 7fe3cd279740 1 client.28213 shutdown 2024-09-10T08:18:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fb1bbfc2740 10 client.0 _ll_register_callbacks cb 0x55f515f4ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f516011a60 0x55f51601d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f516011a60 0x55f51601d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:45656/0 (socket says 172.21.15.2:45656) 2024-09-10T08:18:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fb1b2800640 1 -- 172.21.15.2:0/228886212 learned_addr learned my addr 172.21.15.2:0/228886212 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fb1b1400640 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f516016750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fe3cd279740 1 -- 172.21.15.2:0/903855993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e6cd60 msgr2=0x55e5a0e7d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fe3cd279740 1 --2- 172.21.15.2:0/903855993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e7d7d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40bb890 tx=0x7fe3b40bb790 comp rx=0 tx=0).stop 2024-09-10T08:18:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fb1b1e00640 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f51600c090 0x55f516016210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fe3cd279740 1 -- 172.21.15.2:0/903855993 shutdown_connections 2024-09-10T08:18:22.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fe3cd279740 1 --2- 172.21.15.2:0/903855993 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e5a0e908e0 0x55e5a0e92d10 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fe3cd279740 1 --2- 172.21.15.2:0/903855993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e80830 0x55e5a0e7e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fe3cd279740 1 --2- 172.21.15.2:0/903855993 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6d680 0x55e5a0e7dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fe3cd279740 1 --2- 172.21.15.2:0/903855993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e7d7d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fe3cd279740 1 -- 172.21.15.2:0/903855993 >> 172.21.15.2:0/903855993 conn(0x55e5a0e693d0 msgr2=0x55e5a0e6f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:22.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fe3b1600640 0 client.28213 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fb1b2800640 1 -- 172.21.15.2:0/228886212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f51600c090 msgr2=0x55f516016210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fb1b2800640 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f51600c090 0x55f516016210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fb1b2800640 1 -- 172.21.15.2:0/228886212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 msgr2=0x55f516016750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fb1b2800640 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f516016750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fb1b2800640 1 -- 172.21.15.2:0/228886212 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f515d50620 con 0x55f516011a60 2024-09-10T08:18:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fb1b1400640 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f516016750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:22.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fe3cd279740 1 -- 172.21.15.2:0/903855993 shutdown_connections 2024-09-10T08:18:22.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.942+0000 7fe3cd279740 1 -- 172.21.15.2:0/903855993 wait complete. 2024-09-10T08:18:22.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.943+0000 7fe3cd279740 20 client.28213 trim_cache size 0 max 16384 2024-09-10T08:18:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.943+0000 7fb1b2800640 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f516011a60 0x55f51601d790 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009b30 tx=0x7fb1ac005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e19dc62f74091025 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.943+0000 7fb193e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:18:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.943+0000 7fb193e00640 1 -- 172.21.15.2:0/228886212 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac056030 con 0x55f516011a60 2024-09-10T08:18:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.943+0000 7fb193e00640 1 -- 172.21.15.2:0/228886212 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb1ac003f80 con 0x55f516011a60 2024-09-10T08:18:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.943+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/228886212 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1ac030c70 con 0x55f516011a60 2024-09-10T08:18:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.943+0000 7fb193e00640 1 -- 172.21.15.2:0/228886212 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb1ac063050 con 0x55f516011a60 2024-09-10T08:18:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.943+0000 7fb1bbfc2740 10 client.28216 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.943+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/228886212 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f516014a90 con 0x55f516011a60 2024-09-10T08:18:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.944+0000 7fb193e00640 1 -- 172.21.15.2:0/228886212 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1ac030c70 con 0x55f516011a60 2024-09-10T08:18:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.944+0000 7fb193e00640 1 -- 172.21.15.2:0/228886212 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb1ac03d860 con 0x55f516011a60 2024-09-10T08:18:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.944+0000 7fb1bbfc2740 10 client.28216 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.944+0000 7fb1bbfc2740 10 client.28216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.944+0000 7fb1bbfc2740 20 client.28216 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.944+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f5160218d0 0x55f516023cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.944+0000 7fb1bbfc2740 4 client.28216 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:18:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.944+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/228886212 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f515f7e830 con 0x55f5160218d0 2024-09-10T08:18:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.944+0000 7fb1b1e00640 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f5160218d0 0x55f516023cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.945+0000 7fb1b1e00640 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f5160218d0 0x55f516023cc0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7f8269da995932 server_cookie=5f8de35dba5e3bc9 in_seq=0 out_seq=0 2024-09-10T08:18:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.945+0000 7fb193e00640 10 client.28216 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb193e00640 1 -- 172.21.15.2:0/228886212 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f515f7e830 con 0x55f5160218d0 2024-09-10T08:18:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb193e00640 10 client.28216 handle_command_reply: tid=0 2024-09-10T08:18:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb1bbfc2740 1 client.28216 shutdown 2024-09-10T08:18:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/228886212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f516011a60 msgr2=0x55f51601d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f516011a60 0x55f51601d790 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009b30 tx=0x7fb1ac005fd0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/228886212 shutdown_connections 2024-09-10T08:18:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f5160218d0 0x55f516023cc0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f516011a60 0x55f51601d790 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f516016750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/228886212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f51600c090 0x55f516016210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/228886212 >> 172.21.15.2:0/228886212 conn(0x55f516007700 msgr2=0x55f516008f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb193e00640 0 client.28216 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/228886212 shutdown_connections 2024-09-10T08:18:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/228886212 wait complete. 2024-09-10T08:18:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.946+0000 7fb1bbfc2740 20 client.28216 trim_cache size 0 max 16384 2024-09-10T08:18:22.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.957+0000 7fe3cd279740 1 Processor -- start 2024-09-10T08:18:22.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.957+0000 7fe3cd279740 1 -- start start 2024-09-10T08:18:22.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3cd279740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e6ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3cd279740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6d680 0x55e5a0e98580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3cd279740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e98ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3cd279740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41af410 con 0x55e5a0e6cd60 2024-09-10T08:18:22.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3cd279740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41acce0 con 0x55e5a0e6d680 2024-09-10T08:18:22.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3cd279740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41aab30 con 0x55e5a0e80830 2024-09-10T08:18:22.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3c3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e6ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3c3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e6ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40554/0 (socket says 172.21.15.2:40554) 2024-09-10T08:18:22.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3c3400640 1 -- 172.21.15.2:0/2165154862 learned_addr learned my addr 172.21.15.2:0/2165154862 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:22.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3bbe00640 1 --2- 172.21.15.2:0/2165154862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6d680 0x55e5a0e98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3c3e00640 1 --2- 172.21.15.2:0/2165154862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e98ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3c3400640 1 -- 172.21.15.2:0/2165154862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 msgr2=0x55e5a0e98ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3c3400640 1 --2- 172.21.15.2:0/2165154862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e98ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3c3400640 1 -- 172.21.15.2:0/2165154862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6d680 msgr2=0x55e5a0e98580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.958+0000 7fe3c3400640 1 --2- 172.21.15.2:0/2165154862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6d680 0x55e5a0e98580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.959+0000 7fe3c3400640 1 -- 172.21.15.2:0/2165154862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a0e7c7a0 con 0x55e5a0e6cd60 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.959+0000 7fe3c3400640 1 --2- 172.21.15.2:0/2165154862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e6ef10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40bbb20 tx=0x7fe3b403b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743bf9e5ca38adf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.959+0000 7fe3b3e00640 1 -- 172.21.15.2:0/2165154862 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4049070 con 0x55e5a0e6cd60 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.959+0000 7fe3b3e00640 1 -- 172.21.15.2:0/2165154862 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3b40c42c0 con 0x55e5a0e6cd60 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.959+0000 7fe3b3e00640 1 -- 172.21.15.2:0/2165154862 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4031bd0 con 0x55e5a0e6cd60 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.960+0000 7fe3cd279740 1 -- 172.21.15.2:0/2165154862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6cd60 msgr2=0x55e5a0e6ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.960+0000 7fe3cd279740 1 --2- 172.21.15.2:0/2165154862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e6ef10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40bbb20 tx=0x7fe3b403b480 comp rx=0 tx=0).stop 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.960+0000 7fe3cd279740 1 -- 172.21.15.2:0/2165154862 shutdown_connections 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.960+0000 7fe3cd279740 1 --2- 172.21.15.2:0/2165154862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e98ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.960+0000 7fe3cd279740 1 --2- 172.21.15.2:0/2165154862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6d680 0x55e5a0e98580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.960+0000 7fe3cd279740 1 --2- 172.21.15.2:0/2165154862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e6ef10 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.960+0000 7fe3cd279740 1 -- 172.21.15.2:0/2165154862 >> 172.21.15.2:0/2165154862 conn(0x55e5a0e693d0 msgr2=0x55e5a0e6e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:22.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.960+0000 7fe3cd279740 1 -- 172.21.15.2:0/2165154862 shutdown_connections 2024-09-10T08:18:22.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.960+0000 7fe3cd279740 1 -- 172.21.15.2:0/2165154862 wait complete. 2024-09-10T08:18:22.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.961+0000 7fe3cd279740 1 -- start start 2024-09-10T08:18:22.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.961+0000 7fe3cd279740 1 Processor -- start 2024-09-10T08:18:22.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.961+0000 7fe3cd279740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e88ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.961+0000 7fe3cd279740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6d680 0x55e5a0e7a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.961+0000 7fe3cd279740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e7aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.961+0000 7fe3cd279740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ab1a0 con 0x55e5a0e6d680 2024-09-10T08:18:22.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.961+0000 7fe3cd279740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ac610 con 0x55e5a0e6cd60 2024-09-10T08:18:22.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.961+0000 7fe3cd279740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ab870 con 0x55e5a0e80830 2024-09-10T08:18:22.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.961+0000 7fe3cd279740 10 client.0 _ll_register_callbacks cb 0x7fe3b4009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:22.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.962+0000 7fe3bbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6d680 0x55e5a0e7a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.962+0000 7fe3c3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e88ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.962+0000 7fe3c3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e88ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:45140/0 (socket says 172.21.15.2:45140) 2024-09-10T08:18:22.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.962+0000 7fe3c3400640 1 -- 172.21.15.2:0/1748943864 learned_addr learned my addr 172.21.15.2:0/1748943864 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:22.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.962+0000 7fe3c3e00640 1 --2- 172.21.15.2:0/1748943864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e7aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.962+0000 7fe3c3400640 1 -- 172.21.15.2:0/1748943864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 msgr2=0x55e5a0e7aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.962+0000 7fe3c3400640 1 --2- 172.21.15.2:0/1748943864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e7aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.962+0000 7fe3c3400640 1 -- 172.21.15.2:0/1748943864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6d680 msgr2=0x55e5a0e7a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.962+0000 7fe3c3400640 1 --2- 172.21.15.2:0/1748943864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6d680 0x55e5a0e7a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.962+0000 7fe3c3400640 1 -- 172.21.15.2:0/1748943864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a0e7b230 con 0x55e5a0e6cd60 2024-09-10T08:18:22.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.962+0000 7fe3c3400640 1 --2- 172.21.15.2:0/1748943864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e88ef0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4000c00 tx=0x7fe3b40bafe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e62de97087bdca43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:22.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.962+0000 7fe3b1600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:18:22.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3b1600640 1 -- 172.21.15.2:0/1748943864 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b409f060 con 0x55e5a0e6cd60 2024-09-10T08:18:22.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3b1600640 1 -- 172.21.15.2:0/1748943864 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe3b40da300 con 0x55e5a0e6cd60 2024-09-10T08:18:22.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3cd279740 1 -- 172.21.15.2:0/1748943864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3b40c42c0 con 0x55e5a0e6cd60 2024-09-10T08:18:22.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3b1600640 1 -- 172.21.15.2:0/1748943864 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3b4049070 con 0x55e5a0e6cd60 2024-09-10T08:18:22.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3cd279740 10 client.28289 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:22.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3cd279740 1 -- 172.21.15.2:0/1748943864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5a0e7b230 con 0x55e5a0e6cd60 2024-09-10T08:18:22.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3b1600640 1 -- 172.21.15.2:0/1748943864 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3b40c42c0 con 0x55e5a0e6cd60 2024-09-10T08:18:22.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3b1600640 1 -- 172.21.15.2:0/1748943864 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe3b40d4ce0 con 0x55e5a0e6cd60 2024-09-10T08:18:22.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3cd279740 10 client.28289 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:22.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3cd279740 10 client.28289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:22.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3cd279740 20 client.28289 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:22.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3cd279740 1 --2- 172.21.15.2:0/1748943864 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e5a0efb750 0x55e5a0efdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3cd279740 4 client.28289 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:18:22.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.963+0000 7fe3cd279740 1 -- 172.21.15.2:0/1748943864 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5a0ddae60 con 0x55e5a0efb750 2024-09-10T08:18:22.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.964+0000 7fe3bbe00640 1 --2- 172.21.15.2:0/1748943864 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e5a0efb750 0x55e5a0efdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.964+0000 7fe3bbe00640 1 --2- 172.21.15.2:0/1748943864 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e5a0efb750 0x55e5a0efdb70 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a21b934afb8bad server_cookie=2bdf94d917ccc207 in_seq=0 out_seq=0 2024-09-10T08:18:22.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.964+0000 7fe3b1600640 10 client.28289 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:22.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.964+0000 7fe3b1600640 1 -- 172.21.15.2:0/1748943864 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5a0ddae60 con 0x55e5a0efb750 2024-09-10T08:18:22.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.964+0000 7fe3b1600640 10 client.28289 handle_command_reply: tid=0 2024-09-10T08:18:22.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fe3cd279740 1 client.28289 shutdown 2024-09-10T08:18:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fb1bbfc2740 1 Processor -- start 2024-09-10T08:18:22.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fe3cd279740 1 -- 172.21.15.2:0/1748943864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6cd60 msgr2=0x55e5a0e88ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fe3cd279740 1 --2- 172.21.15.2:0/1748943864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e88ef0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4000c00 tx=0x7fe3b40bafe0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fb1bbfc2740 1 -- start start 2024-09-10T08:18:22.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fe3cd279740 1 -- 172.21.15.2:0/1748943864 shutdown_connections 2024-09-10T08:18:22.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fe3cd279740 1 --2- 172.21.15.2:0/1748943864 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e5a0efb750 0x55e5a0efdb70 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fe3cd279740 1 --2- 172.21.15.2:0/1748943864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a0e80830 0x55e5a0e7aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fe3cd279740 1 --2- 172.21.15.2:0/1748943864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e5a0e6d680 0x55e5a0e7a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fe3cd279740 1 --2- 172.21.15.2:0/1748943864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5a0e6cd60 0x55e5a0e88ef0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fe3cd279740 1 -- 172.21.15.2:0/1748943864 >> 172.21.15.2:0/1748943864 conn(0x55e5a0e693d0 msgr2=0x55e5a0e6f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:22.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fe3b1600640 0 client.28289 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fb1bbfc2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c090 0x55f516028580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fe3cd279740 1 -- 172.21.15.2:0/1748943864 shutdown_connections 2024-09-10T08:18:22.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fe3cd279740 1 -- 172.21.15.2:0/1748943864 wait complete. 2024-09-10T08:18:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fb1bbfc2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c9d0 0x55f51601d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fb1bbfc2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 0x55f516028ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fb1bbfc2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41af280 con 0x55f51600c090 2024-09-10T08:18:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fb1bbfc2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41acb50 con 0x55f516011a60 2024-09-10T08:18:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fb1bbfc2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41aa9a0 con 0x55f51600c9d0 2024-09-10T08:18:22.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.965+0000 7fe3cd279740 20 client.28289 trim_cache size 0 max 16384 2024-09-10T08:18:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fb1b1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c090 0x55f516028580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fb1b1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c090 0x55f516028580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40574/0 (socket says 172.21.15.2:40574) 2024-09-10T08:18:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 0x55f516028ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fb1b1e00640 1 -- 172.21.15.2:0/828469493 learned_addr learned my addr 172.21.15.2:0/828469493 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fb1b1400640 1 --2- 172.21.15.2:0/828469493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c9d0 0x55f51601d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.975 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:18:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fb1b1e00640 1 -- 172.21.15.2:0/828469493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c9d0 msgr2=0x55f51601d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fb1b1e00640 1 --2- 172.21.15.2:0/828469493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c9d0 0x55f51601d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fb1b1e00640 1 -- 172.21.15.2:0/828469493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 msgr2=0x55f516028ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fb1b1e00640 1 --2- 172.21.15.2:0/828469493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 0x55f516028ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fb1b1e00640 1 -- 172.21.15.2:0/828469493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f516002d60 con 0x55f51600c090 2024-09-10T08:18:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fb1b2800640 1 --2- 172.21.15.2:0/828469493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 0x55f516028ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:22.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fe3c9000640 1 -- 172.21.15.2:0/964692610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3a80a2c00 msgr2=0x7fe3a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fe3c9000640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3a80a2c00 0x7fe3a80a50c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc005fd0 tx=0x7fe3bc009500 comp rx=0 tx=0).stop 2024-09-10T08:18:22.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fe3c9000640 1 -- 172.21.15.2:0/964692610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c41019c0 msgr2=0x7fe3c419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fe3c9000640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c41019c0 0x7fe3c419a520 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009230 tx=0x7fe3b4004040 comp rx=0 tx=0).stop 2024-09-10T08:18:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.966+0000 7fb1b1e00640 1 --2- 172.21.15.2:0/828469493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c090 0x55f516028580 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x55f51600d400 tx=0x7fb1a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef89c055bb41210e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:22.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.967+0000 7fe3c9000640 1 -- 172.21.15.2:0/964692610 shutdown_connections 2024-09-10T08:18:22.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.967+0000 7fe3c9000640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3a80a2c00 0x7fe3a80a50c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.967+0000 7fe3c9000640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c41079c0 0x7fe3c419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.967+0000 7fe3c9000640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3c4102360 0x7fe3c419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.967+0000 7fe3c9000640 1 --2- 172.21.15.2:0/964692610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c41019c0 0x7fe3c419a520 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.967+0000 7fe3c9000640 1 -- 172.21.15.2:0/964692610 >> 172.21.15.2:0/964692610 conn(0x7fe3c40fd740 msgr2=0x7fe3c40fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.967+0000 7fb19ea00640 1 -- 172.21.15.2:0/828469493 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4005220 con 0x55f51600c090 2024-09-10T08:18:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.967+0000 7fb19ea00640 1 -- 172.21.15.2:0/828469493 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a4004c80 con 0x55f51600c090 2024-09-10T08:18:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.967+0000 7fb19ea00640 1 -- 172.21.15.2:0/828469493 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a401e8d0 con 0x55f51600c090 2024-09-10T08:18:22.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.967+0000 7fe3c9000640 1 -- 172.21.15.2:0/964692610 shutdown_connections 2024-09-10T08:18:22.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:22.967+0000 7fe3c9000640 1 -- 172.21.15.2:0/964692610 wait complete. 2024-09-10T08:18:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.968+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/828469493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c090 msgr2=0x55f516028580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.968+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/828469493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c090 0x55f516028580 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x55f51600d400 tx=0x7fb1a40046c0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.968+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/828469493 shutdown_connections 2024-09-10T08:18:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.968+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/828469493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 0x55f516028ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.968+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/828469493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c9d0 0x55f51601d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.968+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/828469493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c090 0x55f516028580 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.968+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/828469493 >> 172.21.15.2:0/828469493 conn(0x55f516007700 msgr2=0x55f516006880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.968+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/828469493 shutdown_connections 2024-09-10T08:18:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.968+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/828469493 wait complete. 2024-09-10T08:18:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.968+0000 7fb1bbfc2740 1 -- start start 2024-09-10T08:18:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.968+0000 7fb1bbfc2740 1 Processor -- start 2024-09-10T08:18:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.969+0000 7fb1bbfc2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c090 0x55f51601af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.969+0000 7fb1bbfc2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f51601b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.969+0000 7fb1bbfc2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 0x55f51601c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.969+0000 7fb1bbfc2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41ab010 con 0x55f51600c9d0 2024-09-10T08:18:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.969+0000 7fb1bbfc2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41ac480 con 0x55f516011a60 2024-09-10T08:18:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.969+0000 7fb1bbfc2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41ab6e0 con 0x55f51600c090 2024-09-10T08:18:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.969+0000 7fb1bbfc2740 10 client.0 _ll_register_callbacks cb 0x7fb1ac04a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.969+0000 7fb1b1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f51601b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.969+0000 7fb1b1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f51601b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40582/0 (socket says 172.21.15.2:40582) 2024-09-10T08:18:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.969+0000 7fb1b1400640 1 -- 172.21.15.2:0/2148205186 learned_addr learned my addr 172.21.15.2:0/2148205186 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.969+0000 7fb1b2800640 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 0x55f51601c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.969+0000 7fb1b1e00640 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c090 0x55f51601af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb1b1400640 1 -- 172.21.15.2:0/2148205186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c090 msgr2=0x55f51601af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb1b1400640 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c090 0x55f51601af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb1b1400640 1 -- 172.21.15.2:0/2148205186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 msgr2=0x55f51601c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb1b1400640 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 0x55f51601c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb1b1400640 1 -- 172.21.15.2:0/2148205186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f51601c950 con 0x55f51600c9d0 2024-09-10T08:18:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb1b2800640 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 0x55f51601c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb1b1400640 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f51601b480 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000f010 tx=0x7fb1a009b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90d3fcfa57f61ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb193e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb193e00640 1 -- 172.21.15.2:0/2148205186 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a009ff30 con 0x55f51600c9d0 2024-09-10T08:18:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/2148205186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1a4004c80 con 0x55f51600c9d0 2024-09-10T08:18:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb193e00640 1 -- 172.21.15.2:0/2148205186 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb1a00ba330 con 0x55f51600c9d0 2024-09-10T08:18:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb193e00640 1 -- 172.21.15.2:0/2148205186 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb1a00a47f0 con 0x55f51600c9d0 2024-09-10T08:18:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb1bbfc2740 10 client.18264 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.970+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/2148205186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f51608c560 con 0x55f51600c9d0 2024-09-10T08:18:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.971+0000 7fb193e00640 1 -- 172.21.15.2:0/2148205186 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a4004c80 con 0x55f51600c9d0 2024-09-10T08:18:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.971+0000 7fb193e00640 1 -- 172.21.15.2:0/2148205186 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb1a009b0f0 con 0x55f51600c9d0 2024-09-10T08:18:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.971+0000 7fb1bbfc2740 10 client.18264 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.971+0000 7fb1bbfc2740 10 client.18264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.971+0000 7fb1bbfc2740 20 client.18264 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.971+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f51608ce10 0x55f51608f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.971+0000 7fb1bbfc2740 4 client.18264 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:18:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.971+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/2148205186 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f515f78b80 con 0x55f51608ce10 2024-09-10T08:18:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.971+0000 7fb1b1e00640 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f51608ce10 0x55f51608f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.972+0000 7fb1b1e00640 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f51608ce10 0x55f51608f200 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c00bb0b77ee58cb server_cookie=3184495201c2ddcb in_seq=0 out_seq=0 2024-09-10T08:18:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.972+0000 7fb193e00640 10 client.18264 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb193e00640 1 -- 172.21.15.2:0/2148205186 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f515f78b80 con 0x55f51608ce10 2024-09-10T08:18:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb193e00640 10 client.18264 handle_command_reply: tid=0 2024-09-10T08:18:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb1bbfc2740 1 client.18264 shutdown 2024-09-10T08:18:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/2148205186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 msgr2=0x55f51601b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f51601b480 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000f010 tx=0x7fb1a009b4e0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/2148205186 shutdown_connections 2024-09-10T08:18:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f51608ce10 0x55f51608f200 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb193e00640 0 client.18264 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f516011a60 0x55f51601c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f51600c9d0 0x55f51601b480 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb1bbfc2740 1 --2- 172.21.15.2:0/2148205186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f51600c090 0x55f51601af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/2148205186 >> 172.21.15.2:0/2148205186 conn(0x55f516007700 msgr2=0x55f516008500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/2148205186 shutdown_connections 2024-09-10T08:18:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb1bbfc2740 1 -- 172.21.15.2:0/2148205186 wait complete. 2024-09-10T08:18:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.989+0000 7fb1bbfc2740 20 client.18264 trim_cache size 0 max 16384 2024-09-10T08:18:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:18:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:18:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:18:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:18:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.990+0000 7fb1b9000640 1 -- 172.21.15.2:0/2039323796 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb1980a2cb0 msgr2=0x7fb1980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.990+0000 7fb1b9000640 1 --2- 172.21.15.2:0/2039323796 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb1980a2cb0 0x7fb1980a5170 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac030960 tx=0x7fb1ac005bd0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.990+0000 7fb1b9000640 1 -- 172.21.15.2:0/2039323796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1b4104f00 msgr2=0x7fb1b4100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.990+0000 7fb1b9000640 1 --2- 172.21.15.2:0/2039323796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1b4104f00 0x7fb1b4100580 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000e9c0 tx=0x7fb1a000ee80 comp rx=0 tx=0).stop 2024-09-10T08:18:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.990+0000 7fb1b9000640 1 -- 172.21.15.2:0/2039323796 shutdown_connections 2024-09-10T08:18:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.990+0000 7fb1b9000640 1 --2- 172.21.15.2:0/2039323796 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb1980a2cb0 0x7fb1980a5170 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.990+0000 7fb1b9000640 1 --2- 172.21.15.2:0/2039323796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1b4107830 0x7fb1b4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.990+0000 7fb1b9000640 1 --2- 172.21.15.2:0/2039323796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1b4104f00 0x7fb1b4100580 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.990+0000 7fb1b9000640 1 --2- 172.21.15.2:0/2039323796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1b41025d0 0x7fb1b4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.990+0000 7fb1b9000640 1 -- 172.21.15.2:0/2039323796 >> 172.21.15.2:0/2039323796 conn(0x7fb1b40fc1b0 msgr2=0x7fb1b4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.990+0000 7fb1b9000640 1 -- 172.21.15.2:0/2039323796 shutdown_connections 2024-09-10T08:18:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:22.990+0000 7fb1b9000640 1 -- 172.21.15.2:0/2039323796 wait complete. 2024-09-10T08:18:23.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:23 smithi155 ceph-mon[35405]: pgmap v694: 129 pgs: 129 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:18:23.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:23 smithi107 ceph-mon[35048]: pgmap v694: 129 pgs: 129 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:18:23.436 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:23 smithi002 ceph-mon[30563]: pgmap v694: 129 pgs: 129 active+clean; 1.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:18:24.879 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:18:24.880 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:18:25.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:25 smithi002 ceph-mon[30563]: pgmap v695: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T08:18:25.270 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:18:25.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:25 smithi155 ceph-mon[35405]: pgmap v695: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T08:18:25.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:25 smithi107 ceph-mon[35048]: pgmap v695: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T08:18:25.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.860+0000 7f1f1e800640 1 -- 172.21.15.2:0/1414796948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f18109120 msgr2=0x7f1f18110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:25.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.860+0000 7f1f1e800640 1 --2- 172.21.15.2:0/1414796948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f18109120 0x7f1f18110ca0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009ca0 tx=0x7f1f1002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.860+0000 7f1f1e800640 1 -- 172.21.15.2:0/1414796948 shutdown_connections 2024-09-10T08:18:25.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.860+0000 7f1f1e800640 1 --2- 172.21.15.2:0/1414796948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f18109120 0x7f1f18110ca0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.860+0000 7f1f1e800640 1 --2- 172.21.15.2:0/1414796948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f181049a0 0x7f1f181089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.860+0000 7f1f1e800640 1 --2- 172.21.15.2:0/1414796948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f18103ff0 0x7f1f181043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.860+0000 7f1f1e800640 1 -- 172.21.15.2:0/1414796948 >> 172.21.15.2:0/1414796948 conn(0x7f1f180ffa90 msgr2=0x7f1f18101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:25.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.860+0000 7f1f1e800640 1 -- 172.21.15.2:0/1414796948 shutdown_connections 2024-09-10T08:18:25.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.860+0000 7f1f1e800640 1 -- 172.21.15.2:0/1414796948 wait complete. 2024-09-10T08:18:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.861+0000 7f1f1e800640 1 Processor -- start 2024-09-10T08:18:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.861+0000 7f1f1e800640 1 -- start start 2024-09-10T08:18:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f1e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f18103ff0 0x7f1f1819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f1e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f181049a0 0x7f1f1819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f1e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f18109120 0x7f1f1819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f1e800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18114180 con 0x7f1f18109120 2024-09-10T08:18:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f1e800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18114000 con 0x7f1f18103ff0 2024-09-10T08:18:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f1e800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18114300 con 0x7f1f181049a0 2024-09-10T08:18:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f18109120 0x7f1f1819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f17400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f18103ff0 0x7f1f1819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f18109120 0x7f1f1819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40610/0 (socket says 172.21.15.2:40610) 2024-09-10T08:18:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f17e00640 1 -- 172.21.15.2:0/2626112718 learned_addr learned my addr 172.21.15.2:0/2626112718 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f17400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f18103ff0 0x7f1f1819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:45178/0 (socket says 172.21.15.2:45178) 2024-09-10T08:18:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f0fe00640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f181049a0 0x7f1f1819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f17400640 1 -- 172.21.15.2:0/2626112718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f181049a0 msgr2=0x7f1f1819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f17400640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f181049a0 0x7f1f1819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f17400640 1 -- 172.21.15.2:0/2626112718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f18109120 msgr2=0x7f1f1819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f17400640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f18109120 0x7f1f1819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.862+0000 7f1f17400640 1 -- 172.21.15.2:0/2626112718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1819f5f0 con 0x7f1f18103ff0 2024-09-10T08:18:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.863+0000 7f1f17e00640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f18109120 0x7f1f1819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.863+0000 7f1f17400640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f18103ff0 0x7f1f1819a640 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800d6d0 tx=0x7f1f0800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.863+0000 7f1f14c00640 1 -- 172.21.15.2:0/2626112718 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08017070 con 0x7f1f18103ff0 2024-09-10T08:18:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.863+0000 7f1f1e800640 1 -- 172.21.15.2:0/2626112718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1819f8e0 con 0x7f1f18103ff0 2024-09-10T08:18:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.863+0000 7f1f14c00640 1 -- 172.21.15.2:0/2626112718 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f08009820 con 0x7f1f18103ff0 2024-09-10T08:18:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.863+0000 7f1f14c00640 1 -- 172.21.15.2:0/2626112718 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08004cd0 con 0x7f1f18103ff0 2024-09-10T08:18:25.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.863+0000 7f1f1e800640 1 -- 172.21.15.2:0/2626112718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f181a71c0 con 0x7f1f18103ff0 2024-09-10T08:18:25.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.866+0000 7f1f21275740 1 Processor -- start 2024-09-10T08:18:25.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.866+0000 7f1f21275740 1 -- start start 2024-09-10T08:18:25.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.866+0000 7f1f14c00640 1 -- 172.21.15.2:0/2626112718 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1f080099d0 con 0x7f1f18103ff0 2024-09-10T08:18:25.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.867+0000 7f1f21275740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 0x560917e66ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.867+0000 7f1f21275740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e673e0 0x560917e71760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.867+0000 7f1f21275740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e71ca0 0x560917e74090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.867+0000 7f1f21275740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560917dcd290 con 0x560917e66ac0 2024-09-10T08:18:25.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.867+0000 7f1f21275740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560917d74b40 con 0x560917e71ca0 2024-09-10T08:18:25.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.867+0000 7f1f21275740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560917dc9250 con 0x560917e673e0 2024-09-10T08:18:25.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.867+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 0x560917e66ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.867+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 0x560917e66ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40624/0 (socket says 172.21.15.2:40624) 2024-09-10T08:18:25.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.867+0000 7f1f17e00640 1 -- 172.21.15.2:0/1192939096 learned_addr learned my addr 172.21.15.2:0/1192939096 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:25.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.867+0000 7f1f0fe00640 1 --2- 172.21.15.2:0/1192939096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e71ca0 0x560917e74090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.867+0000 7f1f17400640 1 --2- 172.21.15.2:0/1192939096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e673e0 0x560917e71760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.868+0000 7f1f17e00640 1 -- 172.21.15.2:0/1192939096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e673e0 msgr2=0x560917e71760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:25.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.868+0000 7f1f17e00640 1 --2- 172.21.15.2:0/1192939096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e673e0 0x560917e71760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.868+0000 7f1f17e00640 1 -- 172.21.15.2:0/1192939096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e71ca0 msgr2=0x560917e74090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:25.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.868+0000 7f1f17e00640 1 --2- 172.21.15.2:0/1192939096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e71ca0 0x560917e74090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.868+0000 7f1f17e00640 1 -- 172.21.15.2:0/1192939096 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560917b3e6a0 con 0x560917e66ac0 2024-09-10T08:18:25.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.868+0000 7f1f17400640 1 --2- 172.21.15.2:0/1192939096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e673e0 0x560917e71760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:18:25.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.868+0000 7f1f17e00640 1 --2- 172.21.15.2:0/1192939096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 0x560917e66ea0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1f10005fa0 tx=0x7f1f10009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99de683f49d3b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:25.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.868+0000 7f1f14c00640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1efc0a2b80 0x7f1efc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.868+0000 7f1f0cc00640 1 -- 172.21.15.2:0/1192939096 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10003b70 con 0x560917e66ac0 2024-09-10T08:18:25.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.868+0000 7f1f0cc00640 1 -- 172.21.15.2:0/1192939096 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f10036040 con 0x560917e66ac0 2024-09-10T08:18:25.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.868+0000 7f1f14c00640 1 -- 172.21.15.2:0/2626112718 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1f080cef80 con 0x7f1f18103ff0 2024-09-10T08:18:25.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.868+0000 7f1f0cc00640 1 -- 172.21.15.2:0/1192939096 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10004630 con 0x560917e66ac0 2024-09-10T08:18:25.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.869+0000 7f1f17400640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1efc0a2b80 0x7f1efc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.869+0000 7f1f21275740 1 -- 172.21.15.2:0/1192939096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 msgr2=0x560917e66ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:25.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.869+0000 7f1f21275740 1 --2- 172.21.15.2:0/1192939096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 0x560917e66ea0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1f10005fa0 tx=0x7f1f10009a10 comp rx=0 tx=0).stop 2024-09-10T08:18:25.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.869+0000 7f1f17400640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1efc0a2b80 0x7f1efc0a5040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1f08097e10 tx=0x7f1f08097040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:25.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.869+0000 7f1f21275740 1 -- 172.21.15.2:0/1192939096 shutdown_connections 2024-09-10T08:18:25.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.869+0000 7f1f21275740 1 --2- 172.21.15.2:0/1192939096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e71ca0 0x560917e74090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.869+0000 7f1f21275740 1 --2- 172.21.15.2:0/1192939096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e673e0 0x560917e71760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.869+0000 7f1f21275740 1 --2- 172.21.15.2:0/1192939096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 0x560917e66ea0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.869+0000 7f1f21275740 1 -- 172.21.15.2:0/1192939096 >> 172.21.15.2:0/1192939096 conn(0x560917e63050 msgr2=0x560917e63430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:25.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.869+0000 7f1f21275740 1 -- 172.21.15.2:0/1192939096 shutdown_connections 2024-09-10T08:18:25.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.870+0000 7f1f21275740 1 -- 172.21.15.2:0/1192939096 wait complete. 2024-09-10T08:18:25.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.870+0000 7f1f21275740 1 -- start start 2024-09-10T08:18:25.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.870+0000 7f1f21275740 1 Processor -- start 2024-09-10T08:18:25.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.871+0000 7f1f21275740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e66ac0 0x560917e715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.871+0000 7f1f21275740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e673e0 0x560917e6e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.871+0000 7f1f21275740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 0x560917e6ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.871+0000 7f1f21275740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560917e74790 con 0x560917e71ca0 2024-09-10T08:18:25.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.871+0000 7f1f21275740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560917d87fa0 con 0x560917e673e0 2024-09-10T08:18:25.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.871+0000 7f1f21275740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560917e74910 con 0x560917e66ac0 2024-09-10T08:18:25.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.871+0000 7f1f21275740 10 client.0 _ll_register_callbacks cb 0x560917adbba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:25.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.871+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e66ac0 0x560917e715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.871+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 0x560917e6ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.871+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 0x560917e6ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40628/0 (socket says 172.21.15.2:40628) 2024-09-10T08:18:25.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.871+0000 7f1f17e00640 1 -- 172.21.15.2:0/984620705 learned_addr learned my addr 172.21.15.2:0/984620705 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:25.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.871+0000 7f1f0fe00640 1 --2- 172.21.15.2:0/984620705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e673e0 0x560917e6e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.872+0000 7f1f17e00640 1 -- 172.21.15.2:0/984620705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e66ac0 msgr2=0x560917e715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:25.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.872+0000 7f1f17e00640 1 --2- 172.21.15.2:0/984620705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e66ac0 0x560917e715e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.872+0000 7f1f17e00640 1 -- 172.21.15.2:0/984620705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e673e0 msgr2=0x560917e6e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:25.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.872+0000 7f1f17e00640 1 --2- 172.21.15.2:0/984620705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e673e0 0x560917e6e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.872+0000 7f1f17e00640 1 -- 172.21.15.2:0/984620705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560917e6f690 con 0x560917e71ca0 2024-09-10T08:18:25.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.872+0000 7f1f17e00640 1 --2- 172.21.15.2:0/984620705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 0x560917e6ee60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1f100099e0 tx=0x7f1f100042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b63e5b5c9aa92d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:25.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.872+0000 7f1f05600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:25.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.872+0000 7f1f05600640 1 -- 172.21.15.2:0/984620705 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10059060 con 0x560917e71ca0 2024-09-10T08:18:25.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.872+0000 7f1f05600640 1 -- 172.21.15.2:0/984620705 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1f10031650 con 0x560917e71ca0 2024-09-10T08:18:25.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.872+0000 7f1f21275740 1 -- 172.21.15.2:0/984620705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f10036040 con 0x560917e71ca0 2024-09-10T08:18:25.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.872+0000 7f1f05600640 1 -- 172.21.15.2:0/984620705 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1f10031850 con 0x560917e71ca0 2024-09-10T08:18:25.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.872+0000 7f1f21275740 10 client.18276 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:25.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.872+0000 7f1f21275740 1 -- 172.21.15.2:0/984620705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560917e74790 con 0x560917e71ca0 2024-09-10T08:18:25.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.873+0000 7f1f05600640 1 -- 172.21.15.2:0/984620705 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f10036040 con 0x560917e71ca0 2024-09-10T08:18:25.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.873+0000 7f1f05600640 1 -- 172.21.15.2:0/984620705 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1f10044070 con 0x560917e71ca0 2024-09-10T08:18:25.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.873+0000 7f1f21275740 10 client.18276 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:25.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.873+0000 7f1f21275740 10 client.18276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:25.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.873+0000 7f1f21275740 20 client.18276 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:25.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.873+0000 7f1f21275740 1 --2- 172.21.15.2:0/984620705 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560917e81860 0x560917e83c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.873+0000 7f1f21275740 4 client.18276 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:18:25.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.873+0000 7f1f21275740 1 -- 172.21.15.2:0/984620705 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560917dda800 con 0x560917e81860 2024-09-10T08:18:25.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.873+0000 7f1f0fe00640 1 --2- 172.21.15.2:0/984620705 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560917e81860 0x560917e83c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.874+0000 7f1f0fe00640 1 --2- 172.21.15.2:0/984620705 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560917e81860 0x560917e83c80 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91422e26dac1acd1 server_cookie=4a253055767d0bde in_seq=0 out_seq=0 2024-09-10T08:18:25.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.874+0000 7f1f05600640 10 client.18276 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:25.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f05600640 1 -- 172.21.15.2:0/984620705 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560917dda800 con 0x560917e81860 2024-09-10T08:18:25.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f05600640 10 client.18276 handle_command_reply: tid=0 2024-09-10T08:18:25.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f21275740 1 client.18276 shutdown 2024-09-10T08:18:25.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f21275740 1 -- 172.21.15.2:0/984620705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 msgr2=0x560917e6ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:25.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f21275740 1 --2- 172.21.15.2:0/984620705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 0x560917e6ee60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1f100099e0 tx=0x7f1f100042d0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f21275740 1 -- 172.21.15.2:0/984620705 shutdown_connections 2024-09-10T08:18:25.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f21275740 1 --2- 172.21.15.2:0/984620705 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560917e81860 0x560917e83c80 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f21275740 1 --2- 172.21.15.2:0/984620705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 0x560917e6ee60 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f21275740 1 --2- 172.21.15.2:0/984620705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e673e0 0x560917e6e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f21275740 1 --2- 172.21.15.2:0/984620705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e66ac0 0x560917e715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f21275740 1 -- 172.21.15.2:0/984620705 >> 172.21.15.2:0/984620705 conn(0x560917e63050 msgr2=0x560917e621c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:25.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f05600640 0 client.18276 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:25.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f21275740 1 -- 172.21.15.2:0/984620705 shutdown_connections 2024-09-10T08:18:25.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f21275740 1 -- 172.21.15.2:0/984620705 wait complete. 2024-09-10T08:18:25.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.875+0000 7f1f21275740 20 client.18276 trim_cache size 0 max 16384 2024-09-10T08:18:25.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.894+0000 7f1f21275740 1 Processor -- start 2024-09-10T08:18:25.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.894+0000 7f1f21275740 1 -- start start 2024-09-10T08:18:25.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.894+0000 7f1f21275740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 0x560917e70850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.894+0000 7f1f21275740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e673e0 0x560917e89530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.894+0000 7f1f21275740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e71ca0 0x560917e89a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.894+0000 7f1f21275740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181b3830 con 0x560917e66ac0 2024-09-10T08:18:25.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.894+0000 7f1f21275740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181b1100 con 0x560917e71ca0 2024-09-10T08:18:25.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.894+0000 7f1f21275740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181aef50 con 0x560917e673e0 2024-09-10T08:18:25.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.894+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 0x560917e70850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.895+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 0x560917e70850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40630/0 (socket says 172.21.15.2:40630) 2024-09-10T08:18:25.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.895+0000 7f1f17e00640 1 -- 172.21.15.2:0/3189813253 learned_addr learned my addr 172.21.15.2:0/3189813253 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:25.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.895+0000 7f1f0fe00640 1 --2- 172.21.15.2:0/3189813253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e673e0 0x560917e89530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.895+0000 7f1f17e00640 1 --2- 172.21.15.2:0/3189813253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e71ca0 0x560917e89a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.895+0000 7f1f17e00640 1 -- 172.21.15.2:0/3189813253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e673e0 msgr2=0x560917e89530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:25.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.895+0000 7f1f17e00640 1 --2- 172.21.15.2:0/3189813253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e673e0 0x560917e89530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.895+0000 7f1f17e00640 1 -- 172.21.15.2:0/3189813253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e71ca0 msgr2=0x560917e89a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:25.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.895+0000 7f1f17e00640 1 --2- 172.21.15.2:0/3189813253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e71ca0 0x560917e89a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.895+0000 7f1f17e00640 1 -- 172.21.15.2:0/3189813253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560917e63ce0 con 0x560917e66ac0 2024-09-10T08:18:25.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.895+0000 7f1f17e00640 1 --2- 172.21.15.2:0/3189813253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 0x560917e70850 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1f1002ae80 tx=0x7f1f10009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2775dc3cb9b30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:25.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.895+0000 7f1f07e00640 1 -- 172.21.15.2:0/3189813253 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10059070 con 0x560917e66ac0 2024-09-10T08:18:25.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.895+0000 7f1f07e00640 1 -- 172.21.15.2:0/3189813253 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f10036040 con 0x560917e66ac0 2024-09-10T08:18:25.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.895+0000 7f1f07e00640 1 -- 172.21.15.2:0/3189813253 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10003e20 con 0x560917e66ac0 2024-09-10T08:18:25.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.896+0000 7f1f21275740 1 -- 172.21.15.2:0/3189813253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 msgr2=0x560917e70850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:25.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.896+0000 7f1f21275740 1 --2- 172.21.15.2:0/3189813253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 0x560917e70850 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1f1002ae80 tx=0x7f1f10009900 comp rx=0 tx=0).stop 2024-09-10T08:18:25.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.896+0000 7f1f21275740 1 -- 172.21.15.2:0/3189813253 shutdown_connections 2024-09-10T08:18:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.896+0000 7f1f21275740 1 --2- 172.21.15.2:0/3189813253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e71ca0 0x560917e89a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.896+0000 7f1f21275740 1 --2- 172.21.15.2:0/3189813253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e673e0 0x560917e89530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.896+0000 7f1f21275740 1 --2- 172.21.15.2:0/3189813253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e66ac0 0x560917e70850 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.896+0000 7f1f21275740 1 -- 172.21.15.2:0/3189813253 >> 172.21.15.2:0/3189813253 conn(0x560917e63050 msgr2=0x560917e70140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.896+0000 7f1f21275740 1 -- 172.21.15.2:0/3189813253 shutdown_connections 2024-09-10T08:18:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.896+0000 7f1f21275740 1 -- 172.21.15.2:0/3189813253 wait complete. 2024-09-10T08:18:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.897+0000 7f1f21275740 1 -- start start 2024-09-10T08:18:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.897+0000 7f1f21275740 1 Processor -- start 2024-09-10T08:18:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.897+0000 7f1f21275740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e66ac0 0x560917e79df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.897+0000 7f1f21275740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e673e0 0x560917e7a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.897+0000 7f1f21275740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 0x560917e76b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.897+0000 7f1f21275740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181af5c0 con 0x560917e71ca0 2024-09-10T08:18:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.897+0000 7f1f21275740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181b0a30 con 0x560917e673e0 2024-09-10T08:18:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.897+0000 7f1f21275740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181afc90 con 0x560917e66ac0 2024-09-10T08:18:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.897+0000 7f1f21275740 10 client.0 _ll_register_callbacks cb 0x7f1f000097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.898+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e66ac0 0x560917e79df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.898+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 0x560917e76b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.898+0000 7f1f0fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e673e0 0x560917e7a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.898+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 0x560917e76b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40638/0 (socket says 172.21.15.2:40638) 2024-09-10T08:18:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.898+0000 7f1f17e00640 1 -- 172.21.15.2:0/470296392 learned_addr learned my addr 172.21.15.2:0/470296392 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.898+0000 7f1f17e00640 1 -- 172.21.15.2:0/470296392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e66ac0 msgr2=0x560917e79df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.898+0000 7f1f17e00640 1 --2- 172.21.15.2:0/470296392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e66ac0 0x560917e79df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.898+0000 7f1f17e00640 1 -- 172.21.15.2:0/470296392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e673e0 msgr2=0x560917e7a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.898+0000 7f1f17e00640 1 --2- 172.21.15.2:0/470296392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e673e0 0x560917e7a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.898+0000 7f1f17e00640 1 -- 172.21.15.2:0/470296392 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560917e773c0 con 0x560917e71ca0 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.898+0000 7f1f17e00640 1 --2- 172.21.15.2:0/470296392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 0x560917e76b90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1f100044c0 tx=0x7f1f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e5f007884e0177 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.898+0000 7f1f05600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.899+0000 7f1f05600640 1 -- 172.21.15.2:0/470296392 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10059040 con 0x560917e71ca0 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.899+0000 7f1f05600640 1 -- 172.21.15.2:0/470296392 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1f10030b60 con 0x560917e71ca0 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.899+0000 7f1f21275740 1 -- 172.21.15.2:0/470296392 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f10036040 con 0x560917e71ca0 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.899+0000 7f1f05600640 1 -- 172.21.15.2:0/470296392 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1f10030d60 con 0x560917e71ca0 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.899+0000 7f1f21275740 10 client.18282 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.899+0000 7f1f21275740 1 -- 172.21.15.2:0/470296392 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560917eed0d0 con 0x560917e71ca0 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.899+0000 7f1f05600640 1 -- 172.21.15.2:0/470296392 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f10036040 con 0x560917e71ca0 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.900+0000 7f1f05600640 1 -- 172.21.15.2:0/470296392 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1f10044050 con 0x560917e71ca0 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.900+0000 7f1f21275740 10 client.18282 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.900+0000 7f1f21275740 10 client.18282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:25.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.900+0000 7f1f21275740 20 client.18282 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:25.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.900+0000 7f1f21275740 1 --2- 172.21.15.2:0/470296392 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560917eed5a0 0x560917eef990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:25.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.900+0000 7f1f21275740 4 client.18282 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:18:25.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.900+0000 7f1f21275740 1 -- 172.21.15.2:0/470296392 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560917dd4d10 con 0x560917eed5a0 2024-09-10T08:18:25.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.900+0000 7f1f0fe00640 1 --2- 172.21.15.2:0/470296392 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560917eed5a0 0x560917eef990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:25.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.901+0000 7f1f0fe00640 1 --2- 172.21.15.2:0/470296392 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560917eed5a0 0x560917eef990 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6cfd2e391011f27 server_cookie=ac938db2e42c4cf4 in_seq=0 out_seq=0 2024-09-10T08:18:25.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.901+0000 7f1f05600640 10 client.18282 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:25.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.902+0000 7f1f05600640 1 -- 172.21.15.2:0/470296392 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560917dd4d10 con 0x560917eed5a0 2024-09-10T08:18:25.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.902+0000 7f1f05600640 10 client.18282 handle_command_reply: tid=0 2024-09-10T08:18:25.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.902+0000 7f1f21275740 1 client.18282 shutdown 2024-09-10T08:18:25.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.902+0000 7f1f21275740 1 -- 172.21.15.2:0/470296392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 msgr2=0x560917e76b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:25.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.902+0000 7f1f21275740 1 --2- 172.21.15.2:0/470296392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 0x560917e76b90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1f100044c0 tx=0x7f1f10009bb0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.902+0000 7f1f21275740 1 -- 172.21.15.2:0/470296392 shutdown_connections 2024-09-10T08:18:25.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.902+0000 7f1f21275740 1 --2- 172.21.15.2:0/470296392 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560917eed5a0 0x560917eef990 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.902+0000 7f1f21275740 1 --2- 172.21.15.2:0/470296392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560917e71ca0 0x560917e76b90 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.902+0000 7f1f21275740 1 --2- 172.21.15.2:0/470296392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560917e673e0 0x560917e7a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.902+0000 7f1f21275740 1 --2- 172.21.15.2:0/470296392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560917e66ac0 0x560917e79df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.902+0000 7f1f05600640 0 client.18282 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:25.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.902+0000 7f1f21275740 1 -- 172.21.15.2:0/470296392 >> 172.21.15.2:0/470296392 conn(0x560917e63050 msgr2=0x560917e619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:25.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.903+0000 7f1f21275740 1 -- 172.21.15.2:0/470296392 shutdown_connections 2024-09-10T08:18:25.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.903+0000 7f1f21275740 1 -- 172.21.15.2:0/470296392 wait complete. 2024-09-10T08:18:25.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.903+0000 7f1f21275740 20 client.18282 trim_cache size 0 max 16384 2024-09-10T08:18:25.906 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:18:25.906 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:18:25.906 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "623700a9-1437-40bd-a458-a28b71a08583", 2024-09-10T08:18:25.906 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:18:25.906 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:18:25.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.903+0000 7f1f1e800640 1 -- 172.21.15.2:0/2626112718 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1efc0a2b80 msgr2=0x7f1efc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:25.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.903+0000 7f1f1e800640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1efc0a2b80 0x7f1efc0a5040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1f08097e10 tx=0x7f1f08097040 comp rx=0 tx=0).stop 2024-09-10T08:18:25.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.903+0000 7f1f1e800640 1 -- 172.21.15.2:0/2626112718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f18103ff0 msgr2=0x7f1f1819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:25.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.903+0000 7f1f1e800640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f18103ff0 0x7f1f1819a640 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800d6d0 tx=0x7f1f0800db90 comp rx=0 tx=0).stop 2024-09-10T08:18:25.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.903+0000 7f1f1e800640 1 -- 172.21.15.2:0/2626112718 shutdown_connections 2024-09-10T08:18:25.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.903+0000 7f1f1e800640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f18109120 0x7f1f1819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.903+0000 7f1f1e800640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1efc0a2b80 0x7f1efc0a5040 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.904+0000 7f1f1e800640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f181049a0 0x7f1f1819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.904+0000 7f1f1e800640 1 --2- 172.21.15.2:0/2626112718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f18103ff0 0x7f1f1819a640 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:25.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.904+0000 7f1f1e800640 1 -- 172.21.15.2:0/2626112718 >> 172.21.15.2:0/2626112718 conn(0x7f1f180ffa90 msgr2=0x7f1f18100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:25.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.904+0000 7f1f1e800640 1 -- 172.21.15.2:0/2626112718 shutdown_connections 2024-09-10T08:18:25.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:25.904+0000 7f1f1e800640 1 -- 172.21.15.2:0/2626112718 wait complete. 2024-09-10T08:18:26.799 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:18:27.179 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:18:27.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:27 smithi002 ceph-mon[30563]: pgmap v696: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:18:27.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:27 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:18:27.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:27 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:18:27.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:27 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:18:27.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:27 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:18:27.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:27 smithi155 ceph-mon[35405]: pgmap v696: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:18:27.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:27 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:18:27.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:27 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:18:27.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:27 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:18:27.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:27 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:18:27.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:27 smithi107 ceph-mon[35048]: pgmap v696: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:18:27.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:27 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:18:27.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:27 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:18:27.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:27 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:18:27.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:27 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:18:27.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.727+0000 7f18c1600640 1 -- 172.21.15.2:0/4082676250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc109140 msgr2=0x7f18bc110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:27.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.727+0000 7f18c1600640 1 --2- 172.21.15.2:0/4082676250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc109140 0x7f18bc110cc0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009ca0 tx=0x7f18b402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.727+0000 7f18c1600640 1 -- 172.21.15.2:0/4082676250 shutdown_connections 2024-09-10T08:18:27.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.727+0000 7f18c1600640 1 --2- 172.21.15.2:0/4082676250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc109140 0x7f18bc110cc0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.727+0000 7f18c1600640 1 --2- 172.21.15.2:0/4082676250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18bc1049c0 0x7f18bc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.727+0000 7f18c1600640 1 --2- 172.21.15.2:0/4082676250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18bc104010 0x7f18bc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.727+0000 7f18c1600640 1 -- 172.21.15.2:0/4082676250 >> 172.21.15.2:0/4082676250 conn(0x7f18bc0ffa50 msgr2=0x7f18bc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.727+0000 7f18c1600640 1 -- 172.21.15.2:0/4082676250 shutdown_connections 2024-09-10T08:18:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.727+0000 7f18c1600640 1 -- 172.21.15.2:0/4082676250 wait complete. 2024-09-10T08:18:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.728+0000 7f18c1600640 1 Processor -- start 2024-09-10T08:18:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.728+0000 7f18c1600640 1 -- start start 2024-09-10T08:18:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18c1600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18bc104010 0x7f18bc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18c1600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18bc1049c0 0x7f18bc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18c1600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc109140 0x7f18bc19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18c1600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc114150 con 0x7f18bc109140 2024-09-10T08:18:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18c1600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc113fd0 con 0x7f18bc104010 2024-09-10T08:18:27.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18c1600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1142d0 con 0x7f18bc1049c0 2024-09-10T08:18:27.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18bb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc109140 0x7f18bc19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18bb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc109140 0x7f18bc19eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56314/0 (socket says 172.21.15.2:56314) 2024-09-10T08:18:27.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18bb400640 1 -- 172.21.15.2:0/2693840965 learned_addr learned my addr 172.21.15.2:0/2693840965 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:27.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18baa00640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18bc104010 0x7f18bc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18b3e00640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18bc1049c0 0x7f18bc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18baa00640 1 -- 172.21.15.2:0/2693840965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18bc1049c0 msgr2=0x7f18bc19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:27.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18baa00640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18bc1049c0 0x7f18bc19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.729+0000 7f18baa00640 1 -- 172.21.15.2:0/2693840965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc109140 msgr2=0x7f18bc19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:27.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.730+0000 7f18baa00640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc109140 0x7f18bc19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.730+0000 7f18baa00640 1 -- 172.21.15.2:0/2693840965 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc19f5a0 con 0x7f18bc104010 2024-09-10T08:18:27.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.730+0000 7f18bb400640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc109140 0x7f18bc19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:27.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.730+0000 7f18baa00640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18bc104010 0x7f18bc19a5f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f18ac005fd0 tx=0x7f18ac00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:27.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.730+0000 7f18b3400640 1 -- 172.21.15.2:0/2693840965 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac009c30 con 0x7f18bc104010 2024-09-10T08:18:27.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.730+0000 7f18b3400640 1 -- 172.21.15.2:0/2693840965 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18ac009dd0 con 0x7f18bc104010 2024-09-10T08:18:27.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.730+0000 7f18c1600640 1 -- 172.21.15.2:0/2693840965 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18bc19f890 con 0x7f18bc104010 2024-09-10T08:18:27.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.730+0000 7f18b3400640 1 -- 172.21.15.2:0/2693840965 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac010460 con 0x7f18bc104010 2024-09-10T08:18:27.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.730+0000 7f18c1600640 1 -- 172.21.15.2:0/2693840965 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18bc1a7170 con 0x7f18bc104010 2024-09-10T08:18:27.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.733+0000 7f18c400c740 1 Processor -- start 2024-09-10T08:18:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.733+0000 7f18c400c740 1 -- start start 2024-09-10T08:18:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.733+0000 7f18b3400640 1 -- 172.21.15.2:0/2693840965 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f18ac003ef0 con 0x7f18bc104010 2024-09-10T08:18:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.734+0000 7f18c400c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33f9b90 0x559ce33f9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.734+0000 7f18c400c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33fa4b0 0x559ce3404830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.734+0000 7f18c400c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 0x559ce3407160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.734+0000 7f18c400c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559ce33603e0 con 0x559ce33f9b90 2024-09-10T08:18:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.734+0000 7f18c400c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559ce3307d80 con 0x559ce33fa4b0 2024-09-10T08:18:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.734+0000 7f18c400c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559ce335c3a0 con 0x559ce3404d70 2024-09-10T08:18:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.734+0000 7f18bb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33f9b90 0x559ce33f9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.734+0000 7f18baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33fa4b0 0x559ce3404830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.734+0000 7f18bb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33f9b90 0x559ce33f9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56328/0 (socket says 172.21.15.2:56328) 2024-09-10T08:18:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.734+0000 7f18bb400640 1 -- 172.21.15.2:0/706411586 learned_addr learned my addr 172.21.15.2:0/706411586 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.734+0000 7f18b3e00640 1 --2- 172.21.15.2:0/706411586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 0x559ce3407160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.735+0000 7f18bb400640 1 -- 172.21.15.2:0/706411586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 msgr2=0x559ce3407160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.735+0000 7f18bb400640 1 --2- 172.21.15.2:0/706411586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 0x559ce3407160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.735+0000 7f18bb400640 1 -- 172.21.15.2:0/706411586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33fa4b0 msgr2=0x559ce3404830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.735+0000 7f18bb400640 1 --2- 172.21.15.2:0/706411586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33fa4b0 0x559ce3404830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.735+0000 7f18bb400640 1 -- 172.21.15.2:0/706411586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce3326ae0 con 0x559ce33f9b90 2024-09-10T08:18:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.735+0000 7f18baa00640 1 --2- 172.21.15.2:0/706411586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33fa4b0 0x559ce3404830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.735+0000 7f18b3400640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f18a00a2b80 0x7f18a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.735+0000 7f18bb400640 1 --2- 172.21.15.2:0/706411586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33f9b90 0x559ce33f9f70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f18b4031f80 tx=0x7f18b40098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741017ee48e7b5bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.735+0000 7f18b3400640 1 -- 172.21.15.2:0/2693840965 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f18ac0cefd0 con 0x7f18bc104010 2024-09-10T08:18:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.735+0000 7f18baa00640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f18a00a2b80 0x7f18a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.736+0000 7f18ab400640 1 -- 172.21.15.2:0/706411586 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4003b70 con 0x559ce33f9b90 2024-09-10T08:18:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.736+0000 7f18ab400640 1 -- 172.21.15.2:0/706411586 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b4036040 con 0x559ce33f9b90 2024-09-10T08:18:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.736+0000 7f18ab400640 1 -- 172.21.15.2:0/706411586 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4004700 con 0x559ce33f9b90 2024-09-10T08:18:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.736+0000 7f18c400c740 1 -- 172.21.15.2:0/706411586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33f9b90 msgr2=0x559ce33f9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.736+0000 7f18c400c740 1 --2- 172.21.15.2:0/706411586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33f9b90 0x559ce33f9f70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f18b4031f80 tx=0x7f18b40098d0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.736+0000 7f18baa00640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f18a00a2b80 0x7f18a00a5040 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f18ac005d00 tx=0x7f18ac097040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.737+0000 7f18c400c740 1 -- 172.21.15.2:0/706411586 shutdown_connections 2024-09-10T08:18:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.737+0000 7f18c400c740 1 --2- 172.21.15.2:0/706411586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 0x559ce3407160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.737+0000 7f18c400c740 1 --2- 172.21.15.2:0/706411586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33fa4b0 0x559ce3404830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.737+0000 7f18c400c740 1 --2- 172.21.15.2:0/706411586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33f9b90 0x559ce33f9f70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.737+0000 7f18c400c740 1 -- 172.21.15.2:0/706411586 >> 172.21.15.2:0/706411586 conn(0x559ce33f6120 msgr2=0x559ce33f6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.737+0000 7f18c400c740 1 -- 172.21.15.2:0/706411586 shutdown_connections 2024-09-10T08:18:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.737+0000 7f18c400c740 1 -- 172.21.15.2:0/706411586 wait complete. 2024-09-10T08:18:27.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.737+0000 7f18c400c740 1 -- start start 2024-09-10T08:18:27.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.737+0000 7f18c400c740 1 Processor -- start 2024-09-10T08:18:27.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.738+0000 7f18c400c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33f9b90 0x559ce3401c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.738+0000 7f18c400c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 0x559ce34021d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.738+0000 7f18c400c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 0x559ce3402710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.738+0000 7f18c400c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559ce3407860 con 0x559ce33fa4b0 2024-09-10T08:18:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.738+0000 7f18c400c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559ce331b0f0 con 0x559ce33f9b90 2024-09-10T08:18:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.738+0000 7f18c400c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559ce34079e0 con 0x559ce3404d70 2024-09-10T08:18:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.738+0000 7f18c400c740 10 client.0 _ll_register_callbacks cb 0x559ce330c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.738+0000 7f18b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 0x559ce34021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.738+0000 7f18b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 0x559ce34021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56334/0 (socket says 172.21.15.2:56334) 2024-09-10T08:18:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.738+0000 7f18b3e00640 1 -- 172.21.15.2:0/1043207192 learned_addr learned my addr 172.21.15.2:0/1043207192 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.738+0000 7f18bb400640 1 --2- 172.21.15.2:0/1043207192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33f9b90 0x559ce3401c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.738+0000 7f18bb400640 1 --2- 172.21.15.2:0/1043207192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 0x559ce3402710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.739+0000 7f18b3e00640 1 -- 172.21.15.2:0/1043207192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 msgr2=0x559ce3402710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.739+0000 7f18b3e00640 1 --2- 172.21.15.2:0/1043207192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 0x559ce3402710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.739+0000 7f18b3e00640 1 -- 172.21.15.2:0/1043207192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33f9b90 msgr2=0x559ce3401c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.739+0000 7f18b3e00640 1 --2- 172.21.15.2:0/1043207192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33f9b90 0x559ce3401c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.739+0000 7f18b3e00640 1 -- 172.21.15.2:0/1043207192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce30d13c0 con 0x559ce33fa4b0 2024-09-10T08:18:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.739+0000 7f18b3e00640 1 --2- 172.21.15.2:0/1043207192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 0x559ce34021d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f18a4012700 tx=0x7f18a4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad4ef42b7ffe353 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.739+0000 7f189fe00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.739+0000 7f189fe00640 1 -- 172.21.15.2:0/1043207192 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a400f8c0 con 0x559ce33fa4b0 2024-09-10T08:18:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.739+0000 7f189fe00640 1 -- 172.21.15.2:0/1043207192 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f18a400fa60 con 0x559ce33fa4b0 2024-09-10T08:18:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.739+0000 7f18c400c740 1 -- 172.21.15.2:0/1043207192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18b4036040 con 0x559ce33fa4b0 2024-09-10T08:18:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.740+0000 7f189fe00640 1 -- 172.21.15.2:0/1043207192 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f18a400fc60 con 0x559ce33fa4b0 2024-09-10T08:18:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.740+0000 7f18c400c740 10 client.18294 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.740+0000 7f18c400c740 1 -- 172.21.15.2:0/1043207192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ce34135f0 con 0x559ce33fa4b0 2024-09-10T08:18:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.740+0000 7f189fe00640 1 -- 172.21.15.2:0/1043207192 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b4036040 con 0x559ce33fa4b0 2024-09-10T08:18:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.740+0000 7f189fe00640 1 -- 172.21.15.2:0/1043207192 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f18a4010530 con 0x559ce33fa4b0 2024-09-10T08:18:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.740+0000 7f18c400c740 10 client.18294 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.740+0000 7f18c400c740 10 client.18294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.740+0000 7f18c400c740 20 client.18294 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.740+0000 7f18c400c740 1 --2- 172.21.15.2:0/1043207192 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559ce3414ab0 0x559ce3416f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.740+0000 7f18c400c740 4 client.18294 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:18:27.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.740+0000 7f18c400c740 1 -- 172.21.15.2:0/1043207192 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ce336d950 con 0x559ce3414ab0 2024-09-10T08:18:27.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.741+0000 7f18bb400640 1 --2- 172.21.15.2:0/1043207192 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559ce3414ab0 0x559ce3416f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.741+0000 7f18bb400640 1 --2- 172.21.15.2:0/1043207192 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559ce3414ab0 0x559ce3416f00 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2605aaa5e3f6aea3 server_cookie=9c816263f2c208fc in_seq=0 out_seq=0 2024-09-10T08:18:27.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.741+0000 7f189fe00640 10 client.18294 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:27.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.742+0000 7f189fe00640 1 -- 172.21.15.2:0/1043207192 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ce336d950 con 0x559ce3414ab0 2024-09-10T08:18:27.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.742+0000 7f189fe00640 10 client.18294 handle_command_reply: tid=0 2024-09-10T08:18:27.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.742+0000 7f18c400c740 1 client.18294 shutdown 2024-09-10T08:18:27.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.742+0000 7f18c400c740 1 -- 172.21.15.2:0/1043207192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 msgr2=0x559ce34021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:27.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.742+0000 7f18c400c740 1 --2- 172.21.15.2:0/1043207192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 0x559ce34021d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f18a4012700 tx=0x7f18a4012bc0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.742+0000 7f18c400c740 1 -- 172.21.15.2:0/1043207192 shutdown_connections 2024-09-10T08:18:27.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.742+0000 7f18c400c740 1 --2- 172.21.15.2:0/1043207192 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559ce3414ab0 0x559ce3416f00 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.742+0000 7f18c400c740 1 --2- 172.21.15.2:0/1043207192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 0x559ce3402710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.742+0000 7f18c400c740 1 --2- 172.21.15.2:0/1043207192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 0x559ce34021d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.742+0000 7f18c400c740 1 --2- 172.21.15.2:0/1043207192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33f9b90 0x559ce3401c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.742+0000 7f189fe00640 0 client.18294 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:27.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.742+0000 7f18c400c740 1 -- 172.21.15.2:0/1043207192 >> 172.21.15.2:0/1043207192 conn(0x559ce33f6120 msgr2=0x559ce33fbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:27.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.743+0000 7f18c400c740 1 -- 172.21.15.2:0/1043207192 shutdown_connections 2024-09-10T08:18:27.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.743+0000 7f18c400c740 1 -- 172.21.15.2:0/1043207192 wait complete. 2024-09-10T08:18:27.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.743+0000 7f18c400c740 20 client.18294 trim_cache size 0 max 16384 2024-09-10T08:18:27.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.760+0000 7f18c400c740 1 Processor -- start 2024-09-10T08:18:27.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.761+0000 7f18c400c740 1 -- start start 2024-09-10T08:18:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.761+0000 7f18c400c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33f9b90 0x559ce33fc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.761+0000 7f18c400c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 0x559ce3403c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.761+0000 7f18c400c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 0x559ce341c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.761+0000 7f18c400c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1b38b0 con 0x559ce33fa4b0 2024-09-10T08:18:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.761+0000 7f18c400c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1b1180 con 0x559ce33f9b90 2024-09-10T08:18:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.761+0000 7f18c400c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1aefd0 con 0x559ce3404d70 2024-09-10T08:18:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.761+0000 7f18b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 0x559ce3403c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.761+0000 7f18b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 0x559ce3403c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56340/0 (socket says 172.21.15.2:56340) 2024-09-10T08:18:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.761+0000 7f18b3e00640 1 -- 172.21.15.2:0/1919214634 learned_addr learned my addr 172.21.15.2:0/1919214634 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.761+0000 7f18bb400640 1 --2- 172.21.15.2:0/1919214634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33f9b90 0x559ce33fc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.761+0000 7f18bb400640 1 --2- 172.21.15.2:0/1919214634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 0x559ce341c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.762+0000 7f18b3e00640 1 -- 172.21.15.2:0/1919214634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 msgr2=0x559ce341c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.762+0000 7f18b3e00640 1 --2- 172.21.15.2:0/1919214634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 0x559ce341c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.762+0000 7f18b3e00640 1 -- 172.21.15.2:0/1919214634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33f9b90 msgr2=0x559ce33fc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.762+0000 7f18b3e00640 1 --2- 172.21.15.2:0/1919214634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33f9b90 0x559ce33fc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.762+0000 7f18b3e00640 1 -- 172.21.15.2:0/1919214634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce34074c0 con 0x559ce33fa4b0 2024-09-10T08:18:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.762+0000 7f18b3e00640 1 --2- 172.21.15.2:0/1919214634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 0x559ce3403c00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f18a4010950 tx=0x7f18a400ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba5943758b90265 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.762+0000 7f18aaa00640 1 -- 172.21.15.2:0/1919214634 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4010bc0 con 0x559ce33fa4b0 2024-09-10T08:18:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.762+0000 7f18aaa00640 1 -- 172.21.15.2:0/1919214634 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a4010d60 con 0x559ce33fa4b0 2024-09-10T08:18:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.762+0000 7f18aaa00640 1 -- 172.21.15.2:0/1919214634 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a40037f0 con 0x559ce33fa4b0 2024-09-10T08:18:27.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.763+0000 7f18c400c740 1 -- 172.21.15.2:0/1919214634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 msgr2=0x559ce3403c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:27.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.763+0000 7f18c400c740 1 --2- 172.21.15.2:0/1919214634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 0x559ce3403c00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f18a4010950 tx=0x7f18a400ff90 comp rx=0 tx=0).stop 2024-09-10T08:18:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.763+0000 7f18c400c740 1 -- 172.21.15.2:0/1919214634 shutdown_connections 2024-09-10T08:18:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.763+0000 7f18c400c740 1 --2- 172.21.15.2:0/1919214634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce3404d70 0x559ce341c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.763+0000 7f18c400c740 1 --2- 172.21.15.2:0/1919214634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce33fa4b0 0x559ce3403c00 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.763+0000 7f18c400c740 1 --2- 172.21.15.2:0/1919214634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33f9b90 0x559ce33fc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.763+0000 7f18c400c740 1 -- 172.21.15.2:0/1919214634 >> 172.21.15.2:0/1919214634 conn(0x559ce33f6120 msgr2=0x559ce34034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.763+0000 7f18c400c740 1 -- 172.21.15.2:0/1919214634 shutdown_connections 2024-09-10T08:18:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.763+0000 7f18c400c740 1 -- 172.21.15.2:0/1919214634 wait complete. 2024-09-10T08:18:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.763+0000 7f18c400c740 1 -- start start 2024-09-10T08:18:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.763+0000 7f18c400c740 1 Processor -- start 2024-09-10T08:18:27.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.764+0000 7f18c400c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce33f9b90 0x559ce3408ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.764+0000 7f18c400c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33fa4b0 0x559ce3408ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.764+0000 7f18c400c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce3404d70 0x559ce3409530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.764+0000 7f18c400c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1af640 con 0x559ce3404d70 2024-09-10T08:18:27.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.764+0000 7f18c400c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1b0ab0 con 0x559ce33fa4b0 2024-09-10T08:18:27.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.764+0000 7f18c400c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1afd10 con 0x559ce33f9b90 2024-09-10T08:18:27.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.764+0000 7f18c400c740 10 client.0 _ll_register_callbacks cb 0x7f18a4009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:27.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.764+0000 7f18b3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33fa4b0 0x559ce3408ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.764+0000 7f18bb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce3404d70 0x559ce3409530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.764+0000 7f18b3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33fa4b0 0x559ce3408ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39034/0 (socket says 172.21.15.2:39034) 2024-09-10T08:18:27.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.764+0000 7f18b3e00640 1 -- 172.21.15.2:0/3475008231 learned_addr learned my addr 172.21.15.2:0/3475008231 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:27.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.764+0000 7f18bb400640 1 --2- 172.21.15.2:0/3475008231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce33f9b90 0x559ce3408ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.765+0000 7f18b3e00640 1 -- 172.21.15.2:0/3475008231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce33f9b90 msgr2=0x559ce3408ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:27.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.765+0000 7f18b3e00640 1 --2- 172.21.15.2:0/3475008231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce33f9b90 0x559ce3408ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.765+0000 7f18b3e00640 1 -- 172.21.15.2:0/3475008231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce3404d70 msgr2=0x559ce3409530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:27.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.765+0000 7f18b3e00640 1 --2- 172.21.15.2:0/3475008231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce3404d70 0x559ce3409530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.765+0000 7f18b3e00640 1 -- 172.21.15.2:0/3475008231 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce347e940 con 0x559ce33fa4b0 2024-09-10T08:18:27.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.765+0000 7f18bb400640 1 --2- 172.21.15.2:0/3475008231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce33f9b90 0x559ce3408ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:18:27.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.765+0000 7f18b3e00640 1 --2- 172.21.15.2:0/3475008231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33fa4b0 0x559ce3408ff0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f18a4015000 tx=0x7f18a4010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5746e356810b7d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:27.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.765+0000 7f189fe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:18:27.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.765+0000 7f189fe00640 1 -- 172.21.15.2:0/3475008231 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a40050c0 con 0x559ce33fa4b0 2024-09-10T08:18:27.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.765+0000 7f18c400c740 1 -- 172.21.15.2:0/3475008231 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a4010d60 con 0x559ce33fa4b0 2024-09-10T08:18:27.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.765+0000 7f189fe00640 1 -- 172.21.15.2:0/3475008231 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f18a4005260 con 0x559ce33fa4b0 2024-09-10T08:18:27.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.765+0000 7f189fe00640 1 -- 172.21.15.2:0/3475008231 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f18a40054a0 con 0x559ce33fa4b0 2024-09-10T08:18:27.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.766+0000 7f18c400c740 10 client.28328 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:27.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.766+0000 7f18c400c740 1 -- 172.21.15.2:0/3475008231 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ce347fbf0 con 0x559ce33fa4b0 2024-09-10T08:18:27.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.766+0000 7f189fe00640 1 -- 172.21.15.2:0/3475008231 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a4010d60 con 0x559ce33fa4b0 2024-09-10T08:18:27.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.766+0000 7f189fe00640 1 -- 172.21.15.2:0/3475008231 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f18a401c770 con 0x559ce33fa4b0 2024-09-10T08:18:27.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.766+0000 7f18c400c740 10 client.28328 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:27.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.766+0000 7f18c400c740 10 client.28328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:27.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.766+0000 7f18c400c740 20 client.28328 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:27.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.766+0000 7f18c400c740 1 --2- 172.21.15.2:0/3475008231 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559ce34802a0 0x559ce3482690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:27.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.766+0000 7f18c400c740 4 client.28328 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:18:27.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.766+0000 7f18c400c740 1 -- 172.21.15.2:0/3475008231 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ce3367e60 con 0x559ce34802a0 2024-09-10T08:18:27.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f18bb400640 1 --2- 172.21.15.2:0/3475008231 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559ce34802a0 0x559ce3482690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:27.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f18bb400640 1 --2- 172.21.15.2:0/3475008231 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559ce34802a0 0x559ce3482690 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4682ae8b41b345 server_cookie=1e3cfb3d723178f2 in_seq=0 out_seq=0 2024-09-10T08:18:27.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f189fe00640 10 client.28328 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:27.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f189fe00640 1 -- 172.21.15.2:0/3475008231 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559ce3367e60 con 0x559ce34802a0 2024-09-10T08:18:27.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f189fe00640 10 client.28328 handle_command_reply: tid=0 2024-09-10T08:18:27.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f18c400c740 1 client.28328 shutdown 2024-09-10T08:18:27.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f18c400c740 1 -- 172.21.15.2:0/3475008231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33fa4b0 msgr2=0x559ce3408ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:27.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f18c400c740 1 --2- 172.21.15.2:0/3475008231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33fa4b0 0x559ce3408ff0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f18a4015000 tx=0x7f18a4010430 comp rx=0 tx=0).stop 2024-09-10T08:18:27.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f18c400c740 1 -- 172.21.15.2:0/3475008231 shutdown_connections 2024-09-10T08:18:27.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f18c400c740 1 --2- 172.21.15.2:0/3475008231 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559ce34802a0 0x559ce3482690 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f18c400c740 1 --2- 172.21.15.2:0/3475008231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ce3404d70 0x559ce3409530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f18c400c740 1 --2- 172.21.15.2:0/3475008231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ce33fa4b0 0x559ce3408ff0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f18c400c740 1 --2- 172.21.15.2:0/3475008231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ce33f9b90 0x559ce3408ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f18c400c740 1 -- 172.21.15.2:0/3475008231 >> 172.21.15.2:0/3475008231 conn(0x559ce33f6120 msgr2=0x559ce3403d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:27.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.767+0000 7f189fe00640 0 client.28328 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:27.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.768+0000 7f18c400c740 1 -- 172.21.15.2:0/3475008231 shutdown_connections 2024-09-10T08:18:27.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.768+0000 7f18c400c740 1 -- 172.21.15.2:0/3475008231 wait complete. 2024-09-10T08:18:27.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.768+0000 7f18c400c740 20 client.28328 trim_cache size 0 max 16384 2024-09-10T08:18:27.774 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:18:27.774 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:18:27.774 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:18:27.774 INFO:teuthology.orchestra.run.smithi002.stdout: "623700a9-1437-40bd-a458-a28b71a08583": { 2024-09-10T08:18:27.774 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:18:27.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "623700a9-1437-40bd-a458-a28b71a08583", 2024-09-10T08:18:27.775 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:18:27.775 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:18:27.775 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:18:27.775 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:18:27.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.768+0000 7f18c1600640 1 -- 172.21.15.2:0/2693840965 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f18a00a2b80 msgr2=0x7f18a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:27.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.768+0000 7f18c1600640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f18a00a2b80 0x7f18a00a5040 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f18ac005d00 tx=0x7f18ac097040 comp rx=0 tx=0).stop 2024-09-10T08:18:27.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.768+0000 7f18c1600640 1 -- 172.21.15.2:0/2693840965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18bc104010 msgr2=0x7f18bc19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:27.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.768+0000 7f18c1600640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18bc104010 0x7f18bc19a5f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f18ac005fd0 tx=0x7f18ac00dde0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.768+0000 7f18c1600640 1 -- 172.21.15.2:0/2693840965 shutdown_connections 2024-09-10T08:18:27.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.768+0000 7f18c1600640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc109140 0x7f18bc19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.768+0000 7f18c1600640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f18a00a2b80 0x7f18a00a5040 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.768+0000 7f18c1600640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18bc1049c0 0x7f18bc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.768+0000 7f18c1600640 1 --2- 172.21.15.2:0/2693840965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18bc104010 0x7f18bc19a5f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:27.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.768+0000 7f18c1600640 1 -- 172.21.15.2:0/2693840965 >> 172.21.15.2:0/2693840965 conn(0x7f18bc0ffa50 msgr2=0x7f18bc100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:27.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.769+0000 7f18c1600640 1 -- 172.21.15.2:0/2693840965 shutdown_connections 2024-09-10T08:18:27.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:27.769+0000 7f18c1600640 1 -- 172.21.15.2:0/2693840965 wait complete. 2024-09-10T08:18:28.783 INFO:tasks.cephfs.filesystem:scrub status for tag:623700a9-1437-40bd-a458-a28b71a08583 - {'path': '/', 'tag': '623700a9-1437-40bd-a458-a28b71a08583', 'options': 'recursive,force'} 2024-09-10T08:18:29.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:29 smithi155 ceph-mon[35405]: pgmap v697: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:18:29.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:29 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:18:29.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:29 smithi107 ceph-mon[35048]: pgmap v697: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:18:29.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:29 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:18:29.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:29 smithi002 ceph-mon[30563]: pgmap v697: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:18:29.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:29 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:18:30.028 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.028 INFO:tasks.workunit.client.0.smithi002.stdout:313/600: open fd = 3 2024-09-10T08:18:30.028 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.029 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.029 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.029 INFO:tasks.workunit.client.0.smithi002.stdout:314/600: open fd = 3 2024-09-10T08:18:30.029 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.029 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.029 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.029 INFO:tasks.workunit.client.0.smithi002.stdout:315/600: open fd = 3 2024-09-10T08:18:30.029 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.029 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.029 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.029 INFO:tasks.workunit.client.0.smithi002.stdout:316/600: open fd = 3 2024-09-10T08:18:30.030 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.030 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.030 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.030 INFO:tasks.workunit.client.0.smithi002.stdout:317/600: open fd = 3 2024-09-10T08:18:30.030 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.030 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.030 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.030 INFO:tasks.workunit.client.0.smithi002.stdout:318/600: open fd = 3 2024-09-10T08:18:30.030 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.030 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.030 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.030 INFO:tasks.workunit.client.0.smithi002.stdout:319/600: open fd = 3 2024-09-10T08:18:30.030 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:320/600: open fd = 3 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:321/600: open fd = 3 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:322/600: open fd = 3 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:323/600: open fd = 3 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:324/600: open fd = 3 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:325/600: open fd = 3 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:326/600: open fd = 3 2024-09-10T08:18:30.032 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.033 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.033 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.033 INFO:tasks.workunit.client.0.smithi002.stdout:327/600: open fd = 3 2024-09-10T08:18:30.033 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.033 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.033 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.033 INFO:tasks.workunit.client.0.smithi002.stdout:328/600: open fd = 3 2024-09-10T08:18:30.033 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.033 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.033 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.033 INFO:tasks.workunit.client.0.smithi002.stdout:329/600: open fd = 3 2024-09-10T08:18:30.033 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.033 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.034 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.034 INFO:tasks.workunit.client.0.smithi002.stdout:330/600: open fd = 3 2024-09-10T08:18:30.034 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.034 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.034 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.034 INFO:tasks.workunit.client.0.smithi002.stdout:331/600: open fd = 3 2024-09-10T08:18:30.034 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.034 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.035 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.035 INFO:tasks.workunit.client.0.smithi002.stdout:332/600: open fd = 3 2024-09-10T08:18:30.035 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.035 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.035 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.035 INFO:tasks.workunit.client.0.smithi002.stdout:333/600: open fd = 3 2024-09-10T08:18:30.035 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.035 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.035 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.035 INFO:tasks.workunit.client.0.smithi002.stdout:334/600: open fd = 3 2024-09-10T08:18:30.035 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.035 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.035 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.036 INFO:tasks.workunit.client.0.smithi002.stdout:335/600: open fd = 3 2024-09-10T08:18:30.036 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.036 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.036 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.036 INFO:tasks.workunit.client.0.smithi002.stdout:336/600: open fd = 3 2024-09-10T08:18:30.036 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.036 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.036 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.036 INFO:tasks.workunit.client.0.smithi002.stdout:337/600: open fd = 3 2024-09-10T08:18:30.036 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.037 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.037 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.037 INFO:tasks.workunit.client.0.smithi002.stdout:338/600: open fd = 3 2024-09-10T08:18:30.037 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.037 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.037 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.037 INFO:tasks.workunit.client.0.smithi002.stdout:339/600: open fd = 3 2024-09-10T08:18:30.037 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.037 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.038 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.038 INFO:tasks.workunit.client.0.smithi002.stdout:340/600: open fd = 3 2024-09-10T08:18:30.038 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.038 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.038 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.038 INFO:tasks.workunit.client.0.smithi002.stdout:341/600: open fd = 3 2024-09-10T08:18:30.038 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.038 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.038 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.038 INFO:tasks.workunit.client.0.smithi002.stdout:342/600: open fd = 3 2024-09-10T08:18:30.038 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.039 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.039 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.039 INFO:tasks.workunit.client.0.smithi002.stdout:343/600: open fd = 3 2024-09-10T08:18:30.039 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.039 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.039 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.039 INFO:tasks.workunit.client.0.smithi002.stdout:344/600: open fd = 3 2024-09-10T08:18:30.040 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.040 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.040 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.040 INFO:tasks.workunit.client.0.smithi002.stdout:345/600: open fd = 3 2024-09-10T08:18:30.040 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.040 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.040 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.040 INFO:tasks.workunit.client.0.smithi002.stdout:346/600: open fd = 3 2024-09-10T08:18:30.041 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.041 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.041 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.041 INFO:tasks.workunit.client.0.smithi002.stdout:347/600: open fd = 3 2024-09-10T08:18:30.041 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.041 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.041 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.041 INFO:tasks.workunit.client.0.smithi002.stdout:348/600: open fd = 3 2024-09-10T08:18:30.041 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.041 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.041 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:349/600: open fd = 3 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:350/600: open fd = 3 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:351/600: open fd = 3 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:352/600: open fd = 3 2024-09-10T08:18:30.042 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:353/600: open fd = 3 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:354/600: open fd = 3 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:355/600: open fd = 3 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:356/600: open fd = 3 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:357/600: open fd = 3 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:358/600: open fd = 3 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:359/600: open fd = 3 2024-09-10T08:18:30.044 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.045 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.045 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.045 INFO:tasks.workunit.client.0.smithi002.stdout:360/600: open fd = 3 2024-09-10T08:18:30.045 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.045 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.045 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.045 INFO:tasks.workunit.client.0.smithi002.stdout:361/600: open fd = 3 2024-09-10T08:18:30.045 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.045 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.045 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.045 INFO:tasks.workunit.client.0.smithi002.stdout:362/600: open fd = 3 2024-09-10T08:18:30.045 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.045 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:363/600: open fd = 3 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:364/600: open fd = 3 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:365/600: open fd = 3 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:366/600: open fd = 3 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:367/600: open fd = 3 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:368/600: open fd = 3 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:369/600: open fd = 3 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.047 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:370/600: open fd = 3 2024-09-10T08:18:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:371/600: open fd = 3 2024-09-10T08:18:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:372/600: open fd = 3 2024-09-10T08:18:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:373/600: open fd = 3 2024-09-10T08:18:30.049 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.049 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.049 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:30.049 INFO:tasks.workunit.client.0.smithi002.stdout:374/600: open fd = 3 2024-09-10T08:18:30.049 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.049 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:18:30.049 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:18:31.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:31 smithi155 ceph-mon[35405]: pgmap v698: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:18:31.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:31 smithi107 ceph-mon[35048]: pgmap v698: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:18:31.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:31 smithi002 ceph-mon[30563]: pgmap v698: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:18:33.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:33 smithi155 ceph-mon[35405]: pgmap v699: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:18:33.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:33 smithi107 ceph-mon[35048]: pgmap v699: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:18:33.424 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-10T08:18:33.425 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:18:33.480 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:33 smithi002 ceph-mon[30563]: pgmap v699: 129 pgs: 129 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:18:33.817 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:18:34.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.368+0000 7ff5d0600640 1 -- 172.21.15.2:0/735517408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5c8112a30 msgr2=0x7ff5c8114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:34.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.368+0000 7ff5d0600640 1 --2- 172.21.15.2:0/735517408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5c8112a30 0x7ff5c8114e20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009ca0 tx=0x7ff5c002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:18:34.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.368+0000 7ff5d0600640 1 -- 172.21.15.2:0/735517408 shutdown_connections 2024-09-10T08:18:34.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.368+0000 7ff5d0600640 1 --2- 172.21.15.2:0/735517408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5c8112a30 0x7ff5c8114e20 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:34.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.368+0000 7ff5d0600640 1 --2- 172.21.15.2:0/735517408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5c8077040 0x7ff5c81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:34.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.368+0000 7ff5d0600640 1 --2- 172.21.15.2:0/735517408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5c8076720 0x7ff5c8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:34.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.368+0000 7ff5d0600640 1 -- 172.21.15.2:0/735517408 >> 172.21.15.2:0/735517408 conn(0x7ff5c80ffa90 msgr2=0x7ff5c8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:34.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.368+0000 7ff5d0600640 1 -- 172.21.15.2:0/735517408 shutdown_connections 2024-09-10T08:18:34.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.368+0000 7ff5d0600640 1 -- 172.21.15.2:0/735517408 wait complete. 2024-09-10T08:18:34.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.369+0000 7ff5d0600640 1 Processor -- start 2024-09-10T08:18:34.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.369+0000 7ff5d0600640 1 -- start start 2024-09-10T08:18:34.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.370+0000 7ff5d0600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5c8076720 0x7ff5c81a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:34.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.370+0000 7ff5d0600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5c8077040 0x7ff5c81a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:34.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.370+0000 7ff5d0600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5c8112a30 0x7ff5c819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:34.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.370+0000 7ff5d0600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c81185d0 con 0x7ff5c8077040 2024-09-10T08:18:34.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.370+0000 7ff5d0600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8118450 con 0x7ff5c8112a30 2024-09-10T08:18:34.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.370+0000 7ff5d0600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8118750 con 0x7ff5c8076720 2024-09-10T08:18:34.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.370+0000 7ff5cd200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5c8077040 0x7ff5c81a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:34.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.370+0000 7ff5cd200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5c8077040 0x7ff5c81a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56386/0 (socket says 172.21.15.2:56386) 2024-09-10T08:18:34.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.370+0000 7ff5cdc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5c8076720 0x7ff5c81a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:34.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.370+0000 7ff5cd200640 1 -- 172.21.15.2:0/491830718 learned_addr learned my addr 172.21.15.2:0/491830718 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:34.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.370+0000 7ff5ce600640 1 --2- 172.21.15.2:0/491830718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5c8112a30 0x7ff5c819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:34.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.371+0000 7ff5cdc00640 1 -- 172.21.15.2:0/491830718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5c8112a30 msgr2=0x7ff5c819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:34.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.371+0000 7ff5cdc00640 1 --2- 172.21.15.2:0/491830718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5c8112a30 0x7ff5c819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:34.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.371+0000 7ff5cdc00640 1 -- 172.21.15.2:0/491830718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5c8077040 msgr2=0x7ff5c81a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:34.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.371+0000 7ff5cdc00640 1 --2- 172.21.15.2:0/491830718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5c8077040 0x7ff5c81a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:34.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.371+0000 7ff5cdc00640 1 -- 172.21.15.2:0/491830718 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c819efd0 con 0x7ff5c8076720 2024-09-10T08:18:34.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.371+0000 7ff5cdc00640 1 --2- 172.21.15.2:0/491830718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5c8076720 0x7ff5c81a46e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00e9f0 tx=0x7ff5bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:34.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.371+0000 7ff5c6000640 1 -- 172.21.15.2:0/491830718 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc00ccc0 con 0x7ff5c8076720 2024-09-10T08:18:34.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.371+0000 7ff5d0600640 1 -- 172.21.15.2:0/491830718 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c819f2c0 con 0x7ff5c8076720 2024-09-10T08:18:34.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.372+0000 7ff5c6000640 1 -- 172.21.15.2:0/491830718 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5bc005020 con 0x7ff5c8076720 2024-09-10T08:18:34.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.372+0000 7ff5c6000640 1 -- 172.21.15.2:0/491830718 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc0049e0 con 0x7ff5c8076720 2024-09-10T08:18:34.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.372+0000 7ff5d0600640 1 -- 172.21.15.2:0/491830718 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c81ab5b0 con 0x7ff5c8076720 2024-09-10T08:18:34.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.373+0000 7ff5d0600640 1 -- 172.21.15.2:0/491830718 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c8076b00 con 0x7ff5c8076720 2024-09-10T08:18:34.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.375+0000 7ff5c6000640 1 -- 172.21.15.2:0/491830718 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff5bc005560 con 0x7ff5c8076720 2024-09-10T08:18:34.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.376+0000 7ff5c6000640 1 --2- 172.21.15.2:0/491830718 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5b00a2b80 0x7ff5b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:34.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.376+0000 7ff5c6000640 1 -- 172.21.15.2:0/491830718 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff5bc0cf440 con 0x7ff5c8076720 2024-09-10T08:18:34.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.376+0000 7ff5cd200640 1 --2- 172.21.15.2:0/491830718 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5b00a2b80 0x7ff5b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:34.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.377+0000 7ff5cd200640 1 --2- 172.21.15.2:0/491830718 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5b00a2b80 0x7ff5b00a5040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8079290 tx=0x7ff5b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:34.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.378+0000 7ff5c6000640 1 -- 172.21.15.2:0/491830718 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5bc0051d0 con 0x7ff5c8076720 2024-09-10T08:18:34.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.605+0000 7ff5d0600640 1 -- 172.21.15.2:0/491830718 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5c810dc20 con 0x7ff5c8076720 2024-09-10T08:18:34.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.606+0000 7ff5c6000640 1 -- 172.21.15.2:0/491830718 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7ff5bc09c040 con 0x7ff5c8076720 2024-09-10T08:18:34.606 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:18:34.606 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:18:34.606 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:18:34.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.608+0000 7ff5d0600640 1 -- 172.21.15.2:0/491830718 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5b00a2b80 msgr2=0x7ff5b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.608+0000 7ff5d0600640 1 --2- 172.21.15.2:0/491830718 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5b00a2b80 0x7ff5b00a5040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8079290 tx=0x7ff5b4005f90 comp rx=0 tx=0).stop 2024-09-10T08:18:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.608+0000 7ff5d0600640 1 -- 172.21.15.2:0/491830718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5c8076720 msgr2=0x7ff5c81a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.608+0000 7ff5d0600640 1 --2- 172.21.15.2:0/491830718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5c8076720 0x7ff5c81a46e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00e9f0 tx=0x7ff5bc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:18:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.608+0000 7ff5d0600640 1 -- 172.21.15.2:0/491830718 shutdown_connections 2024-09-10T08:18:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.608+0000 7ff5d0600640 1 --2- 172.21.15.2:0/491830718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5c8112a30 0x7ff5c819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.608+0000 7ff5d0600640 1 --2- 172.21.15.2:0/491830718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5c8077040 0x7ff5c81a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.608+0000 7ff5d0600640 1 --2- 172.21.15.2:0/491830718 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5b00a2b80 0x7ff5b00a5040 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.608+0000 7ff5d0600640 1 --2- 172.21.15.2:0/491830718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5c8076720 0x7ff5c81a46e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.608+0000 7ff5d0600640 1 -- 172.21.15.2:0/491830718 >> 172.21.15.2:0/491830718 conn(0x7ff5c80ffa90 msgr2=0x7ff5c8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.608+0000 7ff5d0600640 1 -- 172.21.15.2:0/491830718 shutdown_connections 2024-09-10T08:18:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:34.608+0000 7ff5d0600640 1 -- 172.21.15.2:0/491830718 wait complete. 2024-09-10T08:18:35.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:35 smithi002 ceph-mon[30563]: pgmap v700: 129 pgs: 129 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:18:35.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:35 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/491830718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:18:35.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:35 smithi155 ceph-mon[35405]: pgmap v700: 129 pgs: 129 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:18:35.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:35 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/491830718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:18:35.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:35 smithi107 ceph-mon[35048]: pgmap v700: 129 pgs: 129 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:18:35.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:35 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/491830718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:18:35.526 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-10T08:18:35.527 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-10T08:18:35.909 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:18:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.109+0000 7faf90a00640 1 -- 172.21.15.2:0/3037788016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf8c1036f0 msgr2=0x7faf8c109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.109+0000 7faf90a00640 1 --2- 172.21.15.2:0/3037788016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf8c1036f0 0x7faf8c109f80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7faf84009ca0 tx=0x7faf8402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.109+0000 7faf90a00640 1 -- 172.21.15.2:0/3037788016 shutdown_connections 2024-09-10T08:18:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.109+0000 7faf90a00640 1 --2- 172.21.15.2:0/3037788016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf8c1036f0 0x7faf8c109f80 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.109+0000 7faf90a00640 1 --2- 172.21.15.2:0/3037788016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c102d30 0x7faf8c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.109+0000 7faf90a00640 1 --2- 172.21.15.2:0/3037788016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf8c101b40 0x7faf8c101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.109+0000 7faf90a00640 1 -- 172.21.15.2:0/3037788016 >> 172.21.15.2:0/3037788016 conn(0x7faf8c0fd2e0 msgr2=0x7faf8c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.109+0000 7faf90a00640 1 -- 172.21.15.2:0/3037788016 shutdown_connections 2024-09-10T08:18:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.109+0000 7faf90a00640 1 -- 172.21.15.2:0/3037788016 wait complete. 2024-09-10T08:18:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.110+0000 7faf90a00640 1 Processor -- start 2024-09-10T08:18:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.110+0000 7faf90a00640 1 -- start start 2024-09-10T08:18:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.110+0000 7faf90a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf8c101b40 0x7faf8c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf90a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c102d30 0x7faf8c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf90a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf8c1036f0 0x7faf8c1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf90a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c10d7f0 con 0x7faf8c1036f0 2024-09-10T08:18:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf90a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c10d670 con 0x7faf8c101b40 2024-09-10T08:18:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf90a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c10d970 con 0x7faf8c102d30 2024-09-10T08:18:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf89400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c102d30 0x7faf8c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf8a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf8c1036f0 0x7faf8c1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf89400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c102d30 0x7faf8c19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:47074/0 (socket says 172.21.15.2:47074) 2024-09-10T08:18:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf89400640 1 -- 172.21.15.2:0/1137764322 learned_addr learned my addr 172.21.15.2:0/1137764322 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf89e00640 1 --2- 172.21.15.2:0/1137764322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf8c101b40 0x7faf8c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf89e00640 1 -- 172.21.15.2:0/1137764322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c102d30 msgr2=0x7faf8c19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf89e00640 1 --2- 172.21.15.2:0/1137764322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c102d30 0x7faf8c19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf89e00640 1 -- 172.21.15.2:0/1137764322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf8c1036f0 msgr2=0x7faf8c1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf89e00640 1 --2- 172.21.15.2:0/1137764322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf8c1036f0 0x7faf8c1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.111+0000 7faf89e00640 1 -- 172.21.15.2:0/1137764322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf8c1a1bf0 con 0x7faf8c101b40 2024-09-10T08:18:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.112+0000 7faf89e00640 1 --2- 172.21.15.2:0/1137764322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf8c101b40 0x7faf8c199fd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00e9c0 tx=0x7faf7c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.112+0000 7faf82a00640 1 -- 172.21.15.2:0/1137764322 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00cd80 con 0x7faf8c101b40 2024-09-10T08:18:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.112+0000 7faf82a00640 1 -- 172.21.15.2:0/1137764322 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7c010910 con 0x7faf8c101b40 2024-09-10T08:18:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.112+0000 7faf90a00640 1 -- 172.21.15.2:0/1137764322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf8c1a1ee0 con 0x7faf8c101b40 2024-09-10T08:18:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.112+0000 7faf82a00640 1 -- 172.21.15.2:0/1137764322 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c018980 con 0x7faf8c101b40 2024-09-10T08:18:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.112+0000 7faf90a00640 1 -- 172.21.15.2:0/1137764322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf8c1a2340 con 0x7faf8c101b40 2024-09-10T08:18:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.116+0000 7faf93a54740 1 Processor -- start 2024-09-10T08:18:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.117+0000 7faf93a54740 1 -- start start 2024-09-10T08:18:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.117+0000 7faf82a00640 1 -- 172.21.15.2:0/1137764322 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7faf7c01f020 con 0x7faf8c101b40 2024-09-10T08:18:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.117+0000 7faf93a54740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4a090 0x56286ba4a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.117+0000 7faf93a54740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a9d0 0x56286ba4f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.117+0000 7faf93a54740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4fa60 0x56286ba51f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.117+0000 7faf93a54740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56286b9af100 con 0x56286ba4a9d0 2024-09-10T08:18:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.117+0000 7faf93a54740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56286b9571b0 con 0x56286ba4a090 2024-09-10T08:18:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.117+0000 7faf93a54740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56286b9ab0c0 con 0x56286ba4fa60 2024-09-10T08:18:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.117+0000 7faf89400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4fa60 0x56286ba51f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.117+0000 7faf89400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4fa60 0x56286ba51f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:47076/0 (socket says 172.21.15.2:47076) 2024-09-10T08:18:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.117+0000 7faf89400640 1 -- 172.21.15.2:0/4154440432 learned_addr learned my addr 172.21.15.2:0/4154440432 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.117+0000 7faf8a800640 1 --2- 172.21.15.2:0/4154440432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4a090 0x56286ba4a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.118+0000 7faf89e00640 1 --2- 172.21.15.2:0/4154440432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a9d0 0x56286ba4f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.118+0000 7faf89400640 1 -- 172.21.15.2:0/4154440432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4a090 msgr2=0x56286ba4a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.118+0000 7faf89400640 1 --2- 172.21.15.2:0/4154440432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4a090 0x56286ba4a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.118+0000 7faf89400640 1 -- 172.21.15.2:0/4154440432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a9d0 msgr2=0x56286ba4f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.118+0000 7faf89400640 1 --2- 172.21.15.2:0/4154440432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a9d0 0x56286ba4f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.118+0000 7faf89400640 1 -- 172.21.15.2:0/4154440432 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56286b6c2210 con 0x56286ba4fa60 2024-09-10T08:18:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.118+0000 7faf89e00640 1 --2- 172.21.15.2:0/4154440432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a9d0 0x56286ba4f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:18:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.118+0000 7faf89400640 1 --2- 172.21.15.2:0/4154440432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4fa60 0x56286ba51f20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faf78004630 tx=0x7faf7800bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b91aa6e7251f807b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.118+0000 7faf77000640 1 -- 172.21.15.2:0/4154440432 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78003dd0 con 0x56286ba4fa60 2024-09-10T08:18:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.119+0000 7faf77000640 1 -- 172.21.15.2:0/4154440432 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf78015070 con 0x56286ba4fa60 2024-09-10T08:18:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.119+0000 7faf77000640 1 -- 172.21.15.2:0/4154440432 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78011750 con 0x56286ba4fa60 2024-09-10T08:18:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.119+0000 7faf82a00640 1 --2- 172.21.15.2:0/1137764322 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf700a2c30 0x7faf700a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.119+0000 7faf82a00640 1 -- 172.21.15.2:0/1137764322 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faf7c0cf400 con 0x7faf8c101b40 2024-09-10T08:18:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.119+0000 7faf8a800640 1 --2- 172.21.15.2:0/1137764322 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf700a2c30 0x7faf700a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.119+0000 7faf93a54740 1 -- 172.21.15.2:0/4154440432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4fa60 msgr2=0x56286ba51f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.119+0000 7faf93a54740 1 --2- 172.21.15.2:0/4154440432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4fa60 0x56286ba51f20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faf78004630 tx=0x7faf7800bc70 comp rx=0 tx=0).stop 2024-09-10T08:18:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.119+0000 7faf93a54740 1 -- 172.21.15.2:0/4154440432 shutdown_connections 2024-09-10T08:18:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.119+0000 7faf93a54740 1 --2- 172.21.15.2:0/4154440432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4fa60 0x56286ba51f20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.119+0000 7faf93a54740 1 --2- 172.21.15.2:0/4154440432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a9d0 0x56286ba4f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.119+0000 7faf93a54740 1 --2- 172.21.15.2:0/4154440432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4a090 0x56286ba4a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.119+0000 7faf93a54740 1 -- 172.21.15.2:0/4154440432 >> 172.21.15.2:0/4154440432 conn(0x56286ba45700 msgr2=0x56286ba45b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.120+0000 7faf93a54740 1 -- 172.21.15.2:0/4154440432 shutdown_connections 2024-09-10T08:18:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.120+0000 7faf93a54740 1 -- 172.21.15.2:0/4154440432 wait complete. 2024-09-10T08:18:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.120+0000 7faf8a800640 1 --2- 172.21.15.2:0/1137764322 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf700a2c30 0x7faf700a50f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faf84005e30 tx=0x7faf84031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.120+0000 7faf93a54740 1 -- start start 2024-09-10T08:18:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.120+0000 7faf93a54740 1 Processor -- start 2024-09-10T08:18:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf93a54740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 0x56286ba54240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf93a54740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4a9d0 0x56286ba54780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf93a54740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4fa60 0x56286ba5b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf93a54740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56286ba52a30 con 0x56286ba4a090 2024-09-10T08:18:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf93a54740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56286b938090 con 0x56286ba4fa60 2024-09-10T08:18:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf93a54740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56286ba52bb0 con 0x56286ba4a9d0 2024-09-10T08:18:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf93a54740 10 client.0 _ll_register_callbacks cb 0x56286b98cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf89e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 0x56286ba54240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf89e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 0x56286ba54240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56420/0 (socket says 172.21.15.2:56420) 2024-09-10T08:18:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf89e00640 1 -- 172.21.15.2:0/202226742 learned_addr learned my addr 172.21.15.2:0/202226742 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf8a800640 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4fa60 0x56286ba5b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf89400640 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4a9d0 0x56286ba54780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf89e00640 1 -- 172.21.15.2:0/202226742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4a9d0 msgr2=0x56286ba54780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf89e00640 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4a9d0 0x56286ba54780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf89e00640 1 -- 172.21.15.2:0/202226742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4fa60 msgr2=0x56286ba5b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf89e00640 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4fa60 0x56286ba5b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf89e00640 1 -- 172.21.15.2:0/202226742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56286b78e620 con 0x56286ba4a090 2024-09-10T08:18:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.121+0000 7faf8a800640 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4fa60 0x56286ba5b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:18:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.122+0000 7faf89e00640 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 0x56286ba54240 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00c6a0 tx=0x7faf7c09b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5a53eb770f79e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.122+0000 7faf6b400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.122+0000 7faf6b400640 1 -- 172.21.15.2:0/202226742 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c09b870 con 0x56286ba4a090 2024-09-10T08:18:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.122+0000 7faf6b400640 1 -- 172.21.15.2:0/202226742 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faf7c0cf130 con 0x56286ba4a090 2024-09-10T08:18:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.122+0000 7faf93a54740 1 -- 172.21.15.2:0/202226742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf78015070 con 0x56286ba4a090 2024-09-10T08:18:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.122+0000 7faf6b400640 1 -- 172.21.15.2:0/202226742 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faf7c09fc20 con 0x56286ba4a090 2024-09-10T08:18:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.122+0000 7faf93a54740 10 client.18315 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.122+0000 7faf93a54740 1 -- 172.21.15.2:0/202226742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56286ba5e640 con 0x56286ba4a090 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.122+0000 7faf6b400640 1 -- 172.21.15.2:0/202226742 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf78015070 con 0x56286ba4a090 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.123+0000 7faf6b400640 1 -- 172.21.15.2:0/202226742 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faf7c010ac0 con 0x56286ba4a090 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.123+0000 7faf93a54740 10 client.18315 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.123+0000 7faf93a54740 10 client.18315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.123+0000 7faf93a54740 20 client.18315 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.123+0000 7faf93a54740 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56286ba5fcc0 0x56286ba620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.123+0000 7faf93a54740 4 client.18315 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.123+0000 7faf93a54740 1 -- 172.21.15.2:0/202226742 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56286b9bc830 con 0x56286ba5fcc0 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.123+0000 7faf89400640 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56286ba5fcc0 0x56286ba620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.124+0000 7faf89400640 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56286ba5fcc0 0x56286ba620b0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e303b11b02b5bdcd server_cookie=caa9bffa72520e48 in_seq=0 out_seq=0 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.124+0000 7faf6b400640 10 client.18315 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf6b400640 1 -- 172.21.15.2:0/202226742 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56286b9bc830 con 0x56286ba5fcc0 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf6b400640 10 client.18315 handle_command_reply: tid=0 2024-09-10T08:18:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf93a54740 1 client.18315 shutdown 2024-09-10T08:18:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf93a54740 1 -- 172.21.15.2:0/202226742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 msgr2=0x56286ba54240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf93a54740 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 0x56286ba54240 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00c6a0 tx=0x7faf7c09b700 comp rx=0 tx=0).stop 2024-09-10T08:18:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf93a54740 1 -- 172.21.15.2:0/202226742 shutdown_connections 2024-09-10T08:18:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf93a54740 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56286ba5fcc0 0x56286ba620b0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf93a54740 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4fa60 0x56286ba5b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf93a54740 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4a9d0 0x56286ba54780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf93a54740 1 --2- 172.21.15.2:0/202226742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 0x56286ba54240 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf6b400640 0 client.18315 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf93a54740 1 -- 172.21.15.2:0/202226742 >> 172.21.15.2:0/202226742 conn(0x56286ba45700 msgr2=0x56286ba46f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf93a54740 1 -- 172.21.15.2:0/202226742 shutdown_connections 2024-09-10T08:18:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf93a54740 1 -- 172.21.15.2:0/202226742 wait complete. 2024-09-10T08:18:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.125+0000 7faf93a54740 20 client.18315 trim_cache size 0 max 16384 2024-09-10T08:18:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.141+0000 7faf93a54740 1 Processor -- start 2024-09-10T08:18:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.141+0000 7faf93a54740 1 -- start start 2024-09-10T08:18:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.141+0000 7faf93a54740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 0x56286ba465b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.141+0000 7faf93a54740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4a9d0 0x56286ba5b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.141+0000 7faf93a54740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4fa60 0x56286ba46af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.141+0000 7faf93a54740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c1af2c0 con 0x56286ba4a090 2024-09-10T08:18:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.141+0000 7faf93a54740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c1acb90 con 0x56286ba4a9d0 2024-09-10T08:18:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.141+0000 7faf93a54740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c1aa9e0 con 0x56286ba4fa60 2024-09-10T08:18:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.141+0000 7faf89e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 0x56286ba465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.141+0000 7faf89e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 0x56286ba465b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56430/0 (socket says 172.21.15.2:56430) 2024-09-10T08:18:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.141+0000 7faf89e00640 1 -- 172.21.15.2:0/1008458477 learned_addr learned my addr 172.21.15.2:0/1008458477 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.141+0000 7faf89400640 1 --2- 172.21.15.2:0/1008458477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4a9d0 0x56286ba5b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.142+0000 7faf8a800640 1 --2- 172.21.15.2:0/1008458477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4fa60 0x56286ba46af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.142+0000 7faf89e00640 1 -- 172.21.15.2:0/1008458477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4fa60 msgr2=0x56286ba46af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.142+0000 7faf89e00640 1 --2- 172.21.15.2:0/1008458477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4fa60 0x56286ba46af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.142+0000 7faf89e00640 1 -- 172.21.15.2:0/1008458477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4a9d0 msgr2=0x56286ba5b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.142+0000 7faf89e00640 1 --2- 172.21.15.2:0/1008458477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4a9d0 0x56286ba5b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.142+0000 7faf89e00640 1 -- 172.21.15.2:0/1008458477 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56286ba4aeb0 con 0x56286ba4a090 2024-09-10T08:18:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.142+0000 7faf8a800640 1 --2- 172.21.15.2:0/1008458477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4fa60 0x56286ba46af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:18:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.142+0000 7faf89e00640 1 --2- 172.21.15.2:0/1008458477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 0x56286ba465b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7faf7c097cd0 tx=0x7faf7c00efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e7b24d433894b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.142+0000 7faf76600640 1 -- 172.21.15.2:0/1008458477 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c09b870 con 0x56286ba4a090 2024-09-10T08:18:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.142+0000 7faf76600640 1 -- 172.21.15.2:0/1008458477 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7c097ee0 con 0x56286ba4a090 2024-09-10T08:18:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.142+0000 7faf76600640 1 -- 172.21.15.2:0/1008458477 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c018b20 con 0x56286ba4a090 2024-09-10T08:18:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.143+0000 7faf93a54740 1 -- 172.21.15.2:0/1008458477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 msgr2=0x56286ba465b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.143+0000 7faf93a54740 1 --2- 172.21.15.2:0/1008458477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 0x56286ba465b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7faf7c097cd0 tx=0x7faf7c00efb0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.143+0000 7faf93a54740 1 -- 172.21.15.2:0/1008458477 shutdown_connections 2024-09-10T08:18:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.143+0000 7faf93a54740 1 --2- 172.21.15.2:0/1008458477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4fa60 0x56286ba46af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.143+0000 7faf93a54740 1 --2- 172.21.15.2:0/1008458477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4a9d0 0x56286ba5b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.143+0000 7faf93a54740 1 --2- 172.21.15.2:0/1008458477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba4a090 0x56286ba465b0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.143+0000 7faf93a54740 1 -- 172.21.15.2:0/1008458477 >> 172.21.15.2:0/1008458477 conn(0x56286ba45700 msgr2=0x56286ba53ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.143+0000 7faf93a54740 1 -- 172.21.15.2:0/1008458477 shutdown_connections 2024-09-10T08:18:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.143+0000 7faf93a54740 1 -- 172.21.15.2:0/1008458477 wait complete. 2024-09-10T08:18:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.143+0000 7faf93a54740 1 -- start start 2024-09-10T08:18:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.144+0000 7faf93a54740 1 Processor -- start 2024-09-10T08:18:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.144+0000 7faf93a54740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4a9d0 0x56286ba58a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.144+0000 7faf93a54740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4fa60 0x56286ba58fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.144+0000 7faf93a54740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba59fe0 0x56286ba5a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.144+0000 7faf93a54740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c1ab050 con 0x56286ba59fe0 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.144+0000 7faf93a54740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c1ac4c0 con 0x56286ba4fa60 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.144+0000 7faf93a54740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c1ab720 con 0x56286ba4a9d0 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.144+0000 7faf93a54740 10 client.0 _ll_register_callbacks cb 0x7faf7c009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.144+0000 7faf8a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba59fe0 0x56286ba5a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.144+0000 7faf8a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba59fe0 0x56286ba5a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56432/0 (socket says 172.21.15.2:56432) 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.144+0000 7faf8a800640 1 -- 172.21.15.2:0/1885156257 learned_addr learned my addr 172.21.15.2:0/1885156257 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.144+0000 7faf89400640 1 --2- 172.21.15.2:0/1885156257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4fa60 0x56286ba58fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.144+0000 7faf89e00640 1 --2- 172.21.15.2:0/1885156257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4a9d0 0x56286ba58a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.145+0000 7faf8a800640 1 -- 172.21.15.2:0/1885156257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4a9d0 msgr2=0x56286ba58a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.145+0000 7faf8a800640 1 --2- 172.21.15.2:0/1885156257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4a9d0 0x56286ba58a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.145+0000 7faf8a800640 1 -- 172.21.15.2:0/1885156257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4fa60 msgr2=0x56286ba58fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.145+0000 7faf8a800640 1 --2- 172.21.15.2:0/1885156257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4fa60 0x56286ba58fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.145+0000 7faf8a800640 1 -- 172.21.15.2:0/1885156257 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56286ba5a9d0 con 0x56286ba59fe0 2024-09-10T08:18:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.145+0000 7faf8a800640 1 --2- 172.21.15.2:0/1885156257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba59fe0 0x56286ba5a440 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7faf84035000 tx=0x7faf840095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2801383c31b95f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.145+0000 7faf6b400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.145+0000 7faf6b400640 1 -- 172.21.15.2:0/1885156257 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84059060 con 0x56286ba59fe0 2024-09-10T08:18:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.145+0000 7faf93a54740 1 -- 172.21.15.2:0/1885156257 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7c097ee0 con 0x56286ba59fe0 2024-09-10T08:18:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.145+0000 7faf6b400640 1 -- 172.21.15.2:0/1885156257 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faf84004250 con 0x56286ba59fe0 2024-09-10T08:18:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.146+0000 7faf6b400640 1 -- 172.21.15.2:0/1885156257 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faf84060050 con 0x56286ba59fe0 2024-09-10T08:18:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.146+0000 7faf93a54740 10 client.18321 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.146+0000 7faf93a54740 1 -- 172.21.15.2:0/1885156257 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56286bacaa50 con 0x56286ba59fe0 2024-09-10T08:18:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.146+0000 7faf6b400640 1 -- 172.21.15.2:0/1885156257 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7c097ee0 con 0x56286ba59fe0 2024-09-10T08:18:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.146+0000 7faf6b400640 1 -- 172.21.15.2:0/1885156257 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faf840305c0 con 0x56286ba59fe0 2024-09-10T08:18:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.146+0000 7faf93a54740 10 client.18321 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.146+0000 7faf93a54740 10 client.18321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.146+0000 7faf93a54740 20 client.18321 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.146+0000 7faf93a54740 1 --2- 172.21.15.2:0/1885156257 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56286bacb3b0 0x56286bacd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.146+0000 7faf93a54740 4 client.18321 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:18:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.146+0000 7faf93a54740 1 -- 172.21.15.2:0/1885156257 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56286b9b6b80 con 0x56286bacb3b0 2024-09-10T08:18:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.146+0000 7faf89e00640 1 --2- 172.21.15.2:0/1885156257 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56286bacb3b0 0x56286bacd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.147+0000 7faf89e00640 1 --2- 172.21.15.2:0/1885156257 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56286bacb3b0 0x56286bacd7d0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d9ff79fcbf6d111 server_cookie=7faa09ecea3cf0b4 in_seq=0 out_seq=0 2024-09-10T08:18:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.147+0000 7faf6b400640 10 client.18321 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf6b400640 1 -- 172.21.15.2:0/1885156257 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56286b9b6b80 con 0x56286bacb3b0 2024-09-10T08:18:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf6b400640 10 client.18321 handle_command_reply: tid=0 2024-09-10T08:18:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf93a54740 1 client.18321 shutdown 2024-09-10T08:18:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf93a54740 1 -- 172.21.15.2:0/1885156257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba59fe0 msgr2=0x56286ba5a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf93a54740 1 --2- 172.21.15.2:0/1885156257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba59fe0 0x56286ba5a440 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7faf84035000 tx=0x7faf840095c0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf93a54740 1 -- 172.21.15.2:0/1885156257 shutdown_connections 2024-09-10T08:18:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf93a54740 1 --2- 172.21.15.2:0/1885156257 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56286bacb3b0 0x56286bacd7d0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf93a54740 1 --2- 172.21.15.2:0/1885156257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56286ba59fe0 0x56286ba5a440 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf93a54740 1 --2- 172.21.15.2:0/1885156257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56286ba4fa60 0x56286ba58fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf6b400640 0 client.18321 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf93a54740 1 --2- 172.21.15.2:0/1885156257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56286ba4a9d0 0x56286ba58a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf93a54740 1 -- 172.21.15.2:0/1885156257 >> 172.21.15.2:0/1885156257 conn(0x56286ba45700 msgr2=0x56286ba53f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf93a54740 1 -- 172.21.15.2:0/1885156257 shutdown_connections 2024-09-10T08:18:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf93a54740 1 -- 172.21.15.2:0/1885156257 wait complete. 2024-09-10T08:18:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.160+0000 7faf93a54740 20 client.18321 trim_cache size 0 max 16384 2024-09-10T08:18:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:18:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:18:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:18:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:18:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.161+0000 7faf90a00640 1 -- 172.21.15.2:0/1137764322 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf700a2c30 msgr2=0x7faf700a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.161+0000 7faf90a00640 1 --2- 172.21.15.2:0/1137764322 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf700a2c30 0x7faf700a50f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faf84005e30 tx=0x7faf84031bb0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.161+0000 7faf90a00640 1 -- 172.21.15.2:0/1137764322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf8c101b40 msgr2=0x7faf8c199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.161+0000 7faf90a00640 1 --2- 172.21.15.2:0/1137764322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf8c101b40 0x7faf8c199fd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00e9c0 tx=0x7faf7c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:18:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.161+0000 7faf90a00640 1 -- 172.21.15.2:0/1137764322 shutdown_connections 2024-09-10T08:18:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.161+0000 7faf90a00640 1 --2- 172.21.15.2:0/1137764322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf8c1036f0 0x7faf8c1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.161+0000 7faf90a00640 1 --2- 172.21.15.2:0/1137764322 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf700a2c30 0x7faf700a50f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.161+0000 7faf90a00640 1 --2- 172.21.15.2:0/1137764322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c102d30 0x7faf8c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.161+0000 7faf90a00640 1 --2- 172.21.15.2:0/1137764322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf8c101b40 0x7faf8c199fd0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.161+0000 7faf90a00640 1 -- 172.21.15.2:0/1137764322 >> 172.21.15.2:0/1137764322 conn(0x7faf8c0fd2e0 msgr2=0x7faf8c074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.161+0000 7faf90a00640 1 -- 172.21.15.2:0/1137764322 shutdown_connections 2024-09-10T08:18:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:36.161+0000 7faf90a00640 1 -- 172.21.15.2:0/1137764322 wait complete. 2024-09-10T08:18:36.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.544+0000 7ff7b5200640 1 -- 172.21.15.2:0/940456706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7b01049c0 msgr2=0x7ff7b0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.544+0000 7ff7b5200640 1 --2- 172.21.15.2:0/940456706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7b01049c0 0x7ff7b0108a10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff798009900 tx=0x7ff79802a410 comp rx=0 tx=0).stop 2024-09-10T08:18:36.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.545+0000 7ff7b5200640 1 -- 172.21.15.2:0/940456706 shutdown_connections 2024-09-10T08:18:36.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.545+0000 7ff7b5200640 1 --2- 172.21.15.2:0/940456706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b0109140 0x7ff7b0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.545+0000 7ff7b5200640 1 --2- 172.21.15.2:0/940456706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7b01049c0 0x7ff7b0108a10 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.545+0000 7ff7b5200640 1 --2- 172.21.15.2:0/940456706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7b0104010 0x7ff7b01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.545+0000 7ff7b5200640 1 -- 172.21.15.2:0/940456706 >> 172.21.15.2:0/940456706 conn(0x7ff7b00ffa50 msgr2=0x7ff7b0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:36.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.545+0000 7ff7b5200640 1 -- 172.21.15.2:0/940456706 shutdown_connections 2024-09-10T08:18:36.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.545+0000 7ff7b5200640 1 -- 172.21.15.2:0/940456706 wait complete. 2024-09-10T08:18:36.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.546+0000 7ff7b5200640 1 Processor -- start 2024-09-10T08:18:36.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.546+0000 7ff7b5200640 1 -- start start 2024-09-10T08:18:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.547+0000 7ff7b5200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7b0104010 0x7ff7b019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.547+0000 7ff7b5200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01049c0 0x7ff7b019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.547+0000 7ff7b5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7b0109140 0x7ff7b019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.547+0000 7ff7b5200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0114190 con 0x7ff7b0104010 2024-09-10T08:18:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.547+0000 7ff7b5200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0114010 con 0x7ff7b01049c0 2024-09-10T08:18:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.547+0000 7ff7b5200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0114310 con 0x7ff7b0109140 2024-09-10T08:18:36.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.547+0000 7ff7ade00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7b0104010 0x7ff7b019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.547+0000 7ff7a5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01049c0 0x7ff7b019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.547+0000 7ff7ade00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7b0104010 0x7ff7b019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56458/0 (socket says 172.21.15.2:56458) 2024-09-10T08:18:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.547+0000 7ff7a5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01049c0 0x7ff7b019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39126/0 (socket says 172.21.15.2:39126) 2024-09-10T08:18:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.547+0000 7ff7ade00640 1 -- 172.21.15.2:0/1126871537 learned_addr learned my addr 172.21.15.2:0/1126871537 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:36.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.547+0000 7ff7ae800640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7b0109140 0x7ff7b019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.548+0000 7ff7ade00640 1 -- 172.21.15.2:0/1126871537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7b0109140 msgr2=0x7ff7b019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.548+0000 7ff7ade00640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7b0109140 0x7ff7b019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.548+0000 7ff7ade00640 1 -- 172.21.15.2:0/1126871537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01049c0 msgr2=0x7ff7b019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.548+0000 7ff7ade00640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01049c0 0x7ff7b019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.548+0000 7ff7ade00640 1 -- 172.21.15.2:0/1126871537 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b019f600 con 0x7ff7b0104010 2024-09-10T08:18:36.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.548+0000 7ff7a5400640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01049c0 0x7ff7b019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:36.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.548+0000 7ff7ade00640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7b0104010 0x7ff7b019a650 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000d950 tx=0x7ff7a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:36.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.548+0000 7ff7a7400640 1 -- 172.21.15.2:0/1126871537 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0017030 con 0x7ff7b0104010 2024-09-10T08:18:36.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.548+0000 7ff7a7400640 1 -- 172.21.15.2:0/1126871537 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7a0004020 con 0x7ff7b0104010 2024-09-10T08:18:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.548+0000 7ff7b5200640 1 -- 172.21.15.2:0/1126871537 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b019f8f0 con 0x7ff7b0104010 2024-09-10T08:18:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.548+0000 7ff7a7400640 1 -- 172.21.15.2:0/1126871537 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00047a0 con 0x7ff7b0104010 2024-09-10T08:18:36.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.548+0000 7ff7b5200640 1 -- 172.21.15.2:0/1126871537 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b01a71d0 con 0x7ff7b0104010 2024-09-10T08:18:36.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.551+0000 7ff7a7400640 1 -- 172.21.15.2:0/1126871537 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff7a00041d0 con 0x7ff7b0104010 2024-09-10T08:18:36.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.551+0000 7ff7b7e5d740 1 Processor -- start 2024-09-10T08:18:36.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.552+0000 7ff7b7e5d740 1 -- start start 2024-09-10T08:18:36.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.552+0000 7ff7b7e5d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61a3b0 0x56429d61a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.552+0000 7ff7b7e5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61acd0 0x56429d625050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.552+0000 7ff7b7e5d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d625590 0x56429d627980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.552+0000 7ff7b7e5d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56429d580530 con 0x56429d625590 2024-09-10T08:18:36.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.552+0000 7ff7b7e5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56429d527860 con 0x56429d61acd0 2024-09-10T08:18:36.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.552+0000 7ff7b7e5d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56429d57c4f0 con 0x56429d61a3b0 2024-09-10T08:18:36.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.552+0000 7ff7ade00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61acd0 0x56429d625050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.552+0000 7ff7ae800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61a3b0 0x56429d61a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.553+0000 7ff7ae800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61a3b0 0x56429d61a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:47132/0 (socket says 172.21.15.2:47132) 2024-09-10T08:18:36.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.553+0000 7ff7ae800640 1 -- 172.21.15.2:0/2737737871 learned_addr learned my addr 172.21.15.2:0/2737737871 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:36.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.553+0000 7ff7a5400640 1 --2- 172.21.15.2:0/2737737871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d625590 0x56429d627980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.553+0000 7ff7ade00640 1 --2- 172.21.15.2:0/2737737871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61acd0 0x56429d625050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39136/0 (socket says 172.21.15.2:39136) 2024-09-10T08:18:36.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.553+0000 7ff7a5400640 1 -- 172.21.15.2:0/2737737871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61a3b0 msgr2=0x56429d61a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.553+0000 7ff7a5400640 1 --2- 172.21.15.2:0/2737737871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61a3b0 0x56429d61a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.553+0000 7ff7a5400640 1 -- 172.21.15.2:0/2737737871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61acd0 msgr2=0x56429d625050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.553+0000 7ff7a5400640 1 --2- 172.21.15.2:0/2737737871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61acd0 0x56429d625050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.553+0000 7ff7a5400640 1 -- 172.21.15.2:0/2737737871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56429d2f1b90 con 0x56429d625590 2024-09-10T08:18:36.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.553+0000 7ff7ade00640 1 --2- 172.21.15.2:0/2737737871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61acd0 0x56429d625050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:36.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.553+0000 7ff7a7400640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff7940a2c00 0x7ff7940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.553+0000 7ff7a5400640 1 --2- 172.21.15.2:0/2737737871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d625590 0x56429d627980 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff79802aa00 tx=0x7ff7980043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd5d4dd01bd6095 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:36.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.553+0000 7ff7a7400640 1 -- 172.21.15.2:0/1126871537 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff7a00ced70 con 0x7ff7b0104010 2024-09-10T08:18:36.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.554+0000 7ff7ae800640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff7940a2c00 0x7ff7940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.554+0000 7ff79f400640 1 -- 172.21.15.2:0/2737737871 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798035040 con 0x56429d625590 2024-09-10T08:18:36.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.554+0000 7ff79f400640 1 -- 172.21.15.2:0/2737737871 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff798004760 con 0x56429d625590 2024-09-10T08:18:36.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.554+0000 7ff79f400640 1 -- 172.21.15.2:0/2737737871 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798003980 con 0x56429d625590 2024-09-10T08:18:36.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.555+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/2737737871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d625590 msgr2=0x56429d627980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.555+0000 7ff7ae800640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff7940a2c00 0x7ff7940a50c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8011630 tx=0x7ff7a8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:36.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.555+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/2737737871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d625590 0x56429d627980 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff79802aa00 tx=0x7ff7980043f0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.555+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/2737737871 shutdown_connections 2024-09-10T08:18:36.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.555+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/2737737871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d625590 0x56429d627980 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.555+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/2737737871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61acd0 0x56429d625050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.555+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/2737737871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61a3b0 0x56429d61a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.555+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/2737737871 >> 172.21.15.2:0/2737737871 conn(0x56429d616940 msgr2=0x56429d616d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:36.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.555+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/2737737871 shutdown_connections 2024-09-10T08:18:36.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.555+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/2737737871 wait complete. 2024-09-10T08:18:36.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.555+0000 7ff7b7e5d740 1 -- start start 2024-09-10T08:18:36.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.556+0000 7ff7b7e5d740 1 Processor -- start 2024-09-10T08:18:36.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.556+0000 7ff7b7e5d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d61a3b0 0x56429d6224d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.556+0000 7ff7b7e5d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61acd0 0x56429d622a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.556+0000 7ff7b7e5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d625590 0x56429d622f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.556+0000 7ff7b7e5d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56429d628070 con 0x56429d61a3b0 2024-09-10T08:18:36.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.556+0000 7ff7b7e5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56429d53ae60 con 0x56429d625590 2024-09-10T08:18:36.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.556+0000 7ff7b7e5d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56429d6281f0 con 0x56429d61acd0 2024-09-10T08:18:36.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.556+0000 7ff7b7e5d740 10 client.0 _ll_register_callbacks cb 0x56429d53fcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:36.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.557+0000 7ff7ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d625590 0x56429d622f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.557+0000 7ff7ade00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d61a3b0 0x56429d6224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.557+0000 7ff7a5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61acd0 0x56429d622a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.557+0000 7ff7ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d625590 0x56429d622f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39142/0 (socket says 172.21.15.2:39142) 2024-09-10T08:18:36.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.557+0000 7ff7ae800640 1 -- 172.21.15.2:0/3797651932 learned_addr learned my addr 172.21.15.2:0/3797651932 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:36.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.557+0000 7ff7ade00640 1 -- 172.21.15.2:0/3797651932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61acd0 msgr2=0x56429d622a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.557+0000 7ff7ade00640 1 --2- 172.21.15.2:0/3797651932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61acd0 0x56429d622a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.557+0000 7ff7ade00640 1 -- 172.21.15.2:0/3797651932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d625590 msgr2=0x56429d622f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.557+0000 7ff7ade00640 1 --2- 172.21.15.2:0/3797651932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d625590 0x56429d622f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.557+0000 7ff7ade00640 1 -- 172.21.15.2:0/3797651932 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56429d546990 con 0x56429d61a3b0 2024-09-10T08:18:36.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.557+0000 7ff7ae800640 1 --2- 172.21.15.2:0/3797651932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d625590 0x56429d622f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:36.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.557+0000 7ff7ade00640 1 --2- 172.21.15.2:0/3797651932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d61a3b0 0x56429d6224d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x56429d6177a0 tx=0x7ff7a00051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fb36c251a5d28c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:36.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.557+0000 7ff793e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:36.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.558+0000 7ff793e00640 1 -- 172.21.15.2:0/3797651932 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a009aae0 con 0x56429d61a3b0 2024-09-10T08:18:36.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.558+0000 7ff793e00640 1 -- 172.21.15.2:0/3797651932 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff7a00a3960 con 0x56429d61a3b0 2024-09-10T08:18:36.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.558+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3797651932 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff798004760 con 0x56429d61a3b0 2024-09-10T08:18:36.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.558+0000 7ff793e00640 1 -- 172.21.15.2:0/3797651932 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff7a009b880 con 0x56429d61a3b0 2024-09-10T08:18:36.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.558+0000 7ff7b7e5d740 10 client.18333 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:36.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.558+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3797651932 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56429d633d90 con 0x56429d61a3b0 2024-09-10T08:18:36.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.558+0000 7ff793e00640 1 -- 172.21.15.2:0/3797651932 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff798004760 con 0x56429d61a3b0 2024-09-10T08:18:36.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.558+0000 7ff793e00640 1 -- 172.21.15.2:0/3797651932 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff7a009f0d0 con 0x56429d61a3b0 2024-09-10T08:18:36.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.558+0000 7ff7b7e5d740 10 client.18333 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:36.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.558+0000 7ff7b7e5d740 10 client.18333 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:18:36.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.559+0000 7ff7b7e5d740 20 client.18333 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:36.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.559+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/3797651932 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56429d6351f0 0x56429d637660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.559+0000 7ff7b7e5d740 4 client.18333 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:18:36.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.559+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3797651932 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56429d523310 con 0x56429d6351f0 2024-09-10T08:18:36.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.559+0000 7ff7a5400640 1 --2- 172.21.15.2:0/3797651932 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56429d6351f0 0x56429d637660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.560+0000 7ff7a5400640 1 --2- 172.21.15.2:0/3797651932 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56429d6351f0 0x56429d637660 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70d961fe5bfd4b11 server_cookie=7a8ef6da95b8e5 in_seq=0 out_seq=0 2024-09-10T08:18:36.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.560+0000 7ff793e00640 10 client.18333 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:36.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.560+0000 7ff793e00640 1 -- 172.21.15.2:0/3797651932 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56429d523310 con 0x56429d6351f0 2024-09-10T08:18:36.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.560+0000 7ff793e00640 10 client.18333 handle_command_reply: tid=0 2024-09-10T08:18:36.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.560+0000 7ff7b7e5d740 1 client.18333 shutdown 2024-09-10T08:18:36.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.561+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3797651932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d61a3b0 msgr2=0x56429d6224d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.561+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/3797651932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d61a3b0 0x56429d6224d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x56429d6177a0 tx=0x7ff7a00051f0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.561+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3797651932 shutdown_connections 2024-09-10T08:18:36.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.561+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/3797651932 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56429d6351f0 0x56429d637660 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.561+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/3797651932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d625590 0x56429d622f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.561+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/3797651932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61acd0 0x56429d622a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.561+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/3797651932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d61a3b0 0x56429d6224d0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.561+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3797651932 >> 172.21.15.2:0/3797651932 conn(0x56429d616940 msgr2=0x56429d615c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:36.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.561+0000 7ff793e00640 0 client.18333 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:36.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.561+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3797651932 shutdown_connections 2024-09-10T08:18:36.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.561+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3797651932 wait complete. 2024-09-10T08:18:36.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.561+0000 7ff7b7e5d740 20 client.18333 trim_cache size 0 max 16384 2024-09-10T08:18:36.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.579+0000 7ff7b7e5d740 1 Processor -- start 2024-09-10T08:18:36.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.579+0000 7ff7b7e5d740 1 -- start start 2024-09-10T08:18:36.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.579+0000 7ff7b7e5d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61a3b0 0x56429d61ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.579+0000 7ff7b7e5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61acd0 0x56429d6242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.579+0000 7ff7b7e5d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d625590 0x56429d63cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.579+0000 7ff7b7e5d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01b3960 con 0x56429d625590 2024-09-10T08:18:36.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.579+0000 7ff7b7e5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01b1230 con 0x56429d61acd0 2024-09-10T08:18:36.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.579+0000 7ff7b7e5d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01af080 con 0x56429d61a3b0 2024-09-10T08:18:36.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.580+0000 7ff7ae800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d625590 0x56429d63cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.580+0000 7ff7ae800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d625590 0x56429d63cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56500/0 (socket says 172.21.15.2:56500) 2024-09-10T08:18:36.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.580+0000 7ff7ae800640 1 -- 172.21.15.2:0/707317943 learned_addr learned my addr 172.21.15.2:0/707317943 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:36.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.580+0000 7ff7a5400640 1 --2- 172.21.15.2:0/707317943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61acd0 0x56429d6242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.580+0000 7ff7ade00640 1 --2- 172.21.15.2:0/707317943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61a3b0 0x56429d61ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.580+0000 7ff7a5400640 1 -- 172.21.15.2:0/707317943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61a3b0 msgr2=0x56429d61ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.580+0000 7ff7a5400640 1 --2- 172.21.15.2:0/707317943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61a3b0 0x56429d61ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.580+0000 7ff7a5400640 1 -- 172.21.15.2:0/707317943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d625590 msgr2=0x56429d63cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.580+0000 7ff7a5400640 1 --2- 172.21.15.2:0/707317943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d625590 0x56429d63cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.580+0000 7ff7a5400640 1 -- 172.21.15.2:0/707317943 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56429d616770 con 0x56429d61acd0 2024-09-10T08:18:36.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.580+0000 7ff7ae800640 1 --2- 172.21.15.2:0/707317943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d625590 0x56429d63cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:36.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.580+0000 7ff7a5400640 1 --2- 172.21.15.2:0/707317943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61acd0 0x56429d6242c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x56429d621d60 tx=0x7ff798031510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29a2418e90738cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:36.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.581+0000 7ff79ea00640 1 -- 172.21.15.2:0/707317943 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798035040 con 0x56429d61acd0 2024-09-10T08:18:36.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.581+0000 7ff79ea00640 1 -- 172.21.15.2:0/707317943 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff798003860 con 0x56429d61acd0 2024-09-10T08:18:36.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.581+0000 7ff79ea00640 1 -- 172.21.15.2:0/707317943 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798004ac0 con 0x56429d61acd0 2024-09-10T08:18:36.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.581+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/707317943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61acd0 msgr2=0x56429d6242c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.581+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/707317943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61acd0 0x56429d6242c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x56429d621d60 tx=0x7ff798031510 comp rx=0 tx=0).stop 2024-09-10T08:18:36.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.581+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/707317943 shutdown_connections 2024-09-10T08:18:36.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.581+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/707317943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d625590 0x56429d63cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.581+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/707317943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61acd0 0x56429d6242c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.581+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/707317943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d61a3b0 0x56429d61ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.581+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/707317943 >> 172.21.15.2:0/707317943 conn(0x56429d616940 msgr2=0x56429d623bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:36.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.581+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/707317943 shutdown_connections 2024-09-10T08:18:36.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.581+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/707317943 wait complete. 2024-09-10T08:18:36.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.582+0000 7ff7b7e5d740 1 -- start start 2024-09-10T08:18:36.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.582+0000 7ff7b7e5d740 1 Processor -- start 2024-09-10T08:18:36.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.582+0000 7ff7b7e5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61a3b0 0x56429d629680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.582+0000 7ff7b7e5d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d61acd0 0x56429d629bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.582+0000 7ff7b7e5d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d625590 0x56429d69ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.582+0000 7ff7b7e5d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01af6f0 con 0x56429d61acd0 2024-09-10T08:18:36.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7b7e5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01b0b60 con 0x56429d61a3b0 2024-09-10T08:18:36.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7b7e5d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01afdc0 con 0x56429d625590 2024-09-10T08:18:36.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7b7e5d740 10 client.0 _ll_register_callbacks cb 0x7ff7a00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:36.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7ade00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61a3b0 0x56429d629680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7a5400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d61acd0 0x56429d629bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7ade00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61a3b0 0x56429d629680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39156/0 (socket says 172.21.15.2:39156) 2024-09-10T08:18:36.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7ade00640 1 -- 172.21.15.2:0/3003543006 learned_addr learned my addr 172.21.15.2:0/3003543006 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:36.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7ae800640 1 --2- 172.21.15.2:0/3003543006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d625590 0x56429d69ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7a5400640 1 -- 172.21.15.2:0/3003543006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d625590 msgr2=0x56429d69ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7a5400640 1 --2- 172.21.15.2:0/3003543006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d625590 0x56429d69ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7a5400640 1 -- 172.21.15.2:0/3003543006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61a3b0 msgr2=0x56429d629680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:36.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7a5400640 1 --2- 172.21.15.2:0/3003543006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61a3b0 0x56429d629680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7a5400640 1 -- 172.21.15.2:0/3003543006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56429d69f440 con 0x56429d61acd0 2024-09-10T08:18:36.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.583+0000 7ff7ade00640 1 --2- 172.21.15.2:0/3003543006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61a3b0 0x56429d629680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:36.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.584+0000 7ff7a5400640 1 --2- 172.21.15.2:0/3003543006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d61acd0 0x56429d629bc0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x56429d621cb0 tx=0x7ff79802ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b712243233c71713 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:36.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.584+0000 7ff793e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:36.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.584+0000 7ff793e00640 1 -- 172.21.15.2:0/3003543006 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798035030 con 0x56429d61acd0 2024-09-10T08:18:36.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.584+0000 7ff793e00640 1 -- 172.21.15.2:0/3003543006 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff798005040 con 0x56429d61acd0 2024-09-10T08:18:36.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.584+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3003543006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff798003860 con 0x56429d61acd0 2024-09-10T08:18:36.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.584+0000 7ff793e00640 1 -- 172.21.15.2:0/3003543006 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff798005280 con 0x56429d61acd0 2024-09-10T08:18:36.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.584+0000 7ff7b7e5d740 10 client.18339 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:36.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.584+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3003543006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56429d6a05e0 con 0x56429d61acd0 2024-09-10T08:18:36.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.584+0000 7ff793e00640 1 -- 172.21.15.2:0/3003543006 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff798003860 con 0x56429d61acd0 2024-09-10T08:18:36.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.585+0000 7ff793e00640 1 -- 172.21.15.2:0/3003543006 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff798031740 con 0x56429d61acd0 2024-09-10T08:18:36.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.585+0000 7ff7b7e5d740 10 client.18339 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:36.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.585+0000 7ff7b7e5d740 10 client.18339 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:18:36.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.585+0000 7ff7b7e5d740 20 client.18339 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:36.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.585+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/3003543006 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56429d6a08f0 0x56429d6a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:36.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.585+0000 7ff7b7e5d740 4 client.18339 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-10T08:18:36.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.585+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3003543006 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x56429d58dc60 con 0x56429d6a08f0 2024-09-10T08:18:36.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.585+0000 7ff7ade00640 1 --2- 172.21.15.2:0/3003543006 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56429d6a08f0 0x56429d6a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:36.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.585+0000 7ff7ade00640 1 --2- 172.21.15.2:0/3003543006 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56429d6a08f0 0x56429d6a2ce0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8bbbac156d7790e server_cookie=d2debe6f6fd98385 in_seq=0 out_seq=0 2024-09-10T08:18:36.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.585+0000 7ff793e00640 10 client.18339 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.596+0000 7ff793e00640 1 -- 172.21.15.2:0/3003543006 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56429d58dc60 con 0x56429d6a08f0 2024-09-10T08:18:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.596+0000 7ff793e00640 10 client.18339 handle_command_reply: tid=0 2024-09-10T08:18:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.597+0000 7ff7b7e5d740 1 client.18339 shutdown 2024-09-10T08:18:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.597+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3003543006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d61acd0 msgr2=0x56429d629bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.597+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/3003543006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d61acd0 0x56429d629bc0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x56429d621cb0 tx=0x7ff79802ab90 comp rx=0 tx=0).stop 2024-09-10T08:18:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.597+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3003543006 shutdown_connections 2024-09-10T08:18:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.597+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/3003543006 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56429d6a08f0 0x56429d6a2ce0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.597+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/3003543006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56429d625590 0x56429d69ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.597+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/3003543006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56429d61acd0 0x56429d629bc0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.597+0000 7ff7b7e5d740 1 --2- 172.21.15.2:0/3003543006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56429d61a3b0 0x56429d629680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.597+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3003543006 >> 172.21.15.2:0/3003543006 conn(0x56429d616940 msgr2=0x56429d622cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:36.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.597+0000 7ff793e00640 0 client.18339 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:36.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.597+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3003543006 shutdown_connections 2024-09-10T08:18:36.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.597+0000 7ff7b7e5d740 1 -- 172.21.15.2:0/3003543006 wait complete. 2024-09-10T08:18:36.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.597+0000 7ff7b7e5d740 20 client.18339 trim_cache size 0 max 16384 2024-09-10T08:18:36.601 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:18:36.601 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:18:36.601 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:18:36.601 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 34, 2024-09-10T08:18:36.601 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:18:36.601 INFO:teuthology.orchestra.run.smithi002.stdout: "ba7e0923": { 2024-09-10T08:18:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 34, 2024-09-10T08:18:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:18:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:18:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:18:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:18:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:18:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:18:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 130.0, 2024-09-10T08:18:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:18:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:18:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:18:36.602 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:18:36.603 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:18:36.603 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:18:36.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:18:36.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:18:36.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:18:36.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:18:36.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:18:36.603 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:18:36.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.598+0000 7ff7b5200640 1 -- 172.21.15.2:0/1126871537 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff7940a2c00 msgr2=0x7ff7940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.598+0000 7ff7b5200640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff7940a2c00 0x7ff7940a50c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8011630 tx=0x7ff7a8009500 comp rx=0 tx=0).stop 2024-09-10T08:18:36.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.598+0000 7ff7b5200640 1 -- 172.21.15.2:0/1126871537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7b0104010 msgr2=0x7ff7b019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:36.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.598+0000 7ff7b5200640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7b0104010 0x7ff7b019a650 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000d950 tx=0x7ff7a000de10 comp rx=0 tx=0).stop 2024-09-10T08:18:36.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.598+0000 7ff7b5200640 1 -- 172.21.15.2:0/1126871537 shutdown_connections 2024-09-10T08:18:36.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.598+0000 7ff7b5200640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff7940a2c00 0x7ff7940a50c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.598+0000 7ff7b5200640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7b0109140 0x7ff7b019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.598+0000 7ff7b5200640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7b01049c0 0x7ff7b019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.598+0000 7ff7b5200640 1 --2- 172.21.15.2:0/1126871537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7b0104010 0x7ff7b019a650 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.598+0000 7ff7b5200640 1 -- 172.21.15.2:0/1126871537 >> 172.21.15.2:0/1126871537 conn(0x7ff7b00ffa50 msgr2=0x7ff7b0100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.598+0000 7ff7b5200640 1 -- 172.21.15.2:0/1126871537 shutdown_connections 2024-09-10T08:18:36.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:36.598+0000 7ff7b5200640 1 -- 172.21.15.2:0/1126871537 wait complete. 2024-09-10T08:18:37.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:37 smithi002 ceph-mon[30563]: pgmap v701: 129 pgs: 129 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:18:37.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:37 smithi155 ceph-mon[35405]: pgmap v701: 129 pgs: 129 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:18:37.411 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ba7e0923', quiesce took 4.0 seconds. Will release after: 6.0 2024-09-10T08:18:37.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:37 smithi107 ceph-mon[35048]: pgmap v701: 129 pgs: 129 active+clean; 1.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:18:39.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:39 smithi155 ceph-mon[35405]: pgmap v702: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:18:39.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:39 smithi107 ceph-mon[35048]: pgmap v702: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:18:39.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:39 smithi002 ceph-mon[30563]: pgmap v702: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:18:41.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:41 smithi155 ceph-mon[35405]: pgmap v703: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:18:41.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:41 smithi107 ceph-mon[35048]: pgmap v703: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:18:41.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:41 smithi002 ceph-mon[30563]: pgmap v703: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:18:43.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:43 smithi155 ceph-mon[35405]: pgmap v704: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:18:43.412 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:18:43.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:43 smithi107 ceph-mon[35048]: pgmap v704: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:18:43.467 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:43 smithi002 ceph-mon[30563]: pgmap v704: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:18:43.801 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:18:44.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.523+0000 7f2bf3200640 1 -- 172.21.15.2:0/2866021511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bf4102b90 msgr2=0x7f2bf4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:44.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.523+0000 7f2bf3200640 1 --2- 172.21.15.2:0/2866021511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bf4102b90 0x7f2bf4102f70 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009900 tx=0x7f2be402a480 comp rx=0 tx=0).stop 2024-09-10T08:18:44.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.523+0000 7f2bf3200640 1 -- 172.21.15.2:0/2866021511 shutdown_connections 2024-09-10T08:18:44.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.523+0000 7f2bf3200640 1 --2- 172.21.15.2:0/2866021511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bf410bfc0 0x7f2bf410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:44.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.523+0000 7f2bf3200640 1 --2- 172.21.15.2:0/2866021511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf41034b0 0x7f2bf410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:44.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.523+0000 7f2bf3200640 1 --2- 172.21.15.2:0/2866021511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bf4102b90 0x7f2bf4102f70 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.523+0000 7f2bf3200640 1 -- 172.21.15.2:0/2866021511 >> 172.21.15.2:0/2866021511 conn(0x7f2bf40fc610 msgr2=0x7f2bf40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.523+0000 7f2bf3200640 1 -- 172.21.15.2:0/2866021511 shutdown_connections 2024-09-10T08:18:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.523+0000 7f2bf3200640 1 -- 172.21.15.2:0/2866021511 wait complete. 2024-09-10T08:18:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.524+0000 7f2bf3200640 1 Processor -- start 2024-09-10T08:18:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.525+0000 7f2bf3200640 1 -- start start 2024-09-10T08:18:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.525+0000 7f2bf3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bf4102b90 0x7f2bf41007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.525+0000 7f2bf3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf41034b0 0x7f2bf4100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.525+0000 7f2bf3200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bf410bfc0 0x7f2bf4101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.525+0000 7f2bf3200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf4111dc0 con 0x7f2bf410bfc0 2024-09-10T08:18:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.525+0000 7f2bf3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf4111c40 con 0x7f2bf41034b0 2024-09-10T08:18:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.525+0000 7f2bf3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf4111f40 con 0x7f2bf4102b90 2024-09-10T08:18:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.525+0000 7f2bf2800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bf410bfc0 0x7f2bf4101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:44.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.525+0000 7f2bf2800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bf410bfc0 0x7f2bf4101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47168/0 (socket says 172.21.15.2:47168) 2024-09-10T08:18:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.525+0000 7f2bf2800640 1 -- 172.21.15.2:0/4231047455 learned_addr learned my addr 172.21.15.2:0/4231047455 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.525+0000 7f2be9400640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf41034b0 0x7f2bf4100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.525+0000 7f2bf1e00640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bf4102b90 0x7f2bf41007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.526+0000 7f2bf2800640 1 -- 172.21.15.2:0/4231047455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bf4102b90 msgr2=0x7f2bf41007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.526+0000 7f2bf2800640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bf4102b90 0x7f2bf41007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.526+0000 7f2bf2800640 1 -- 172.21.15.2:0/4231047455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf41034b0 msgr2=0x7f2bf4100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.526+0000 7f2bf2800640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf41034b0 0x7f2bf4100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.526+0000 7f2bf2800640 1 -- 172.21.15.2:0/4231047455 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf41a6760 con 0x7f2bf410bfc0 2024-09-10T08:18:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.526+0000 7f2be9400640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf41034b0 0x7f2bf4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.526+0000 7f2bf2800640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bf410bfc0 0x7f2bf4101270 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00eb10 tx=0x7f2bec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.526+0000 7f2beb400640 1 -- 172.21.15.2:0/4231047455 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec004b20 con 0x7f2bf410bfc0 2024-09-10T08:18:44.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.526+0000 7f2beb400640 1 -- 172.21.15.2:0/4231047455 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bec004cc0 con 0x7f2bf410bfc0 2024-09-10T08:18:44.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.526+0000 7f2bf3200640 1 -- 172.21.15.2:0/4231047455 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf41a6a50 con 0x7f2bf410bfc0 2024-09-10T08:18:44.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.526+0000 7f2beb400640 1 -- 172.21.15.2:0/4231047455 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec010760 con 0x7f2bf410bfc0 2024-09-10T08:18:44.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.527+0000 7f2bf3200640 1 -- 172.21.15.2:0/4231047455 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf41a6f90 con 0x7f2bf410bfc0 2024-09-10T08:18:44.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.528+0000 7f2bf3200640 1 -- 172.21.15.2:0/4231047455 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf4102f70 con 0x7f2bf410bfc0 2024-09-10T08:18:44.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.530+0000 7f2beb400640 1 -- 172.21.15.2:0/4231047455 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2bec005000 con 0x7f2bf410bfc0 2024-09-10T08:18:44.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.531+0000 7f2beb400640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2bd80a2b80 0x7f2bd80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:44.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.531+0000 7f2beb400640 1 -- 172.21.15.2:0/4231047455 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2bec0cfe60 con 0x7f2bf410bfc0 2024-09-10T08:18:44.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.531+0000 7f2bf1e00640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2bd80a2b80 0x7f2bd80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:44.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.532+0000 7f2bf1e00640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2bd80a2b80 0x7f2bd80a5040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009230 tx=0x7f2be40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:44.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.533+0000 7f2beb400640 1 -- 172.21.15.2:0/4231047455 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2bec09c970 con 0x7f2bf410bfc0 2024-09-10T08:18:44.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.755+0000 7f2bf3200640 1 -- 172.21.15.2:0/4231047455 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bf402d7c0 con 0x7f2bf410bfc0 2024-09-10T08:18:44.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.756+0000 7f2beb400640 1 -- 172.21.15.2:0/4231047455 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f2bec09c270 con 0x7f2bf410bfc0 2024-09-10T08:18:44.756 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:18:44.756 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:18:44.756 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:18:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.758+0000 7f2bf3200640 1 -- 172.21.15.2:0/4231047455 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2bd80a2b80 msgr2=0x7f2bd80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.758+0000 7f2bf3200640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2bd80a2b80 0x7f2bd80a5040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009230 tx=0x7f2be40058d0 comp rx=0 tx=0).stop 2024-09-10T08:18:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.758+0000 7f2bf3200640 1 -- 172.21.15.2:0/4231047455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bf410bfc0 msgr2=0x7f2bf4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.758+0000 7f2bf3200640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bf410bfc0 0x7f2bf4101270 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00eb10 tx=0x7f2bec00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:18:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.758+0000 7f2bf3200640 1 -- 172.21.15.2:0/4231047455 shutdown_connections 2024-09-10T08:18:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.758+0000 7f2bf3200640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bf410bfc0 0x7f2bf4101270 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.758+0000 7f2bf3200640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf41034b0 0x7f2bf4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.758+0000 7f2bf3200640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2bd80a2b80 0x7f2bd80a5040 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.758+0000 7f2bf3200640 1 --2- 172.21.15.2:0/4231047455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bf4102b90 0x7f2bf41007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.758+0000 7f2bf3200640 1 -- 172.21.15.2:0/4231047455 >> 172.21.15.2:0/4231047455 conn(0x7f2bf40fc610 msgr2=0x7f2bf40fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.759+0000 7f2bf3200640 1 -- 172.21.15.2:0/4231047455 shutdown_connections 2024-09-10T08:18:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:44.759+0000 7f2bf3200640 1 -- 172.21.15.2:0/4231047455 wait complete. 2024-09-10T08:18:45.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:45 smithi155 ceph-mon[35405]: pgmap v705: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:18:45.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:45 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/4231047455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:18:45.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:45 smithi107 ceph-mon[35048]: pgmap v705: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:18:45.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:45 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/4231047455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:18:45.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:45 smithi002 ceph-mon[30563]: pgmap v705: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:18:45.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:45 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/4231047455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:18:45.644 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id ba7e0923 --release --await' 2024-09-10T08:18:45.644 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id ba7e0923 --release --await 2024-09-10T08:18:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:18:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.541+0000 7f7051800640 1 -- 172.21.15.2:0/3876465802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c104800 msgr2=0x7f704c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.541+0000 7f7051800640 1 --2- 172.21.15.2:0/3876465802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c104800 0x7f704c108960 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7034009900 tx=0x7f703402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.541+0000 7f7051800640 1 -- 172.21.15.2:0/3876465802 shutdown_connections 2024-09-10T08:18:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.541+0000 7f7051800640 1 --2- 172.21.15.2:0/3876465802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c109090 0x7f704c110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.541+0000 7f7051800640 1 --2- 172.21.15.2:0/3876465802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c104800 0x7f704c108960 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.541+0000 7f7051800640 1 --2- 172.21.15.2:0/3876465802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c103e50 0x7f704c104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.541+0000 7f7051800640 1 -- 172.21.15.2:0/3876465802 >> 172.21.15.2:0/3876465802 conn(0x7f704c0ff8b0 msgr2=0x7f704c101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.541+0000 7f7051800640 1 -- 172.21.15.2:0/3876465802 shutdown_connections 2024-09-10T08:18:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.541+0000 7f7051800640 1 -- 172.21.15.2:0/3876465802 wait complete. 2024-09-10T08:18:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.542+0000 7f7051800640 1 Processor -- start 2024-09-10T08:18:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.542+0000 7f7051800640 1 -- start start 2024-09-10T08:18:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.542+0000 7f7051800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e50 0x7f704c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.542+0000 7f7051800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c104800 0x7f704c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.542+0000 7f7051800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c109090 0x7f704c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.542+0000 7f7051800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f704c1140e0 con 0x7f704c104800 2024-09-10T08:18:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.542+0000 7f7051800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f704c113f60 con 0x7f704c103e50 2024-09-10T08:18:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.542+0000 7f7051800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f704c114260 con 0x7f704c109090 2024-09-10T08:18:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.543+0000 7f704be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e50 0x7f704c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.543+0000 7f7043400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c104800 0x7f704c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.543+0000 7f704be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e50 0x7f704c19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40184/0 (socket says 172.21.15.2:40184) 2024-09-10T08:18:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.543+0000 7f704be00640 1 -- 172.21.15.2:0/2118663003 learned_addr learned my addr 172.21.15.2:0/2118663003 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.543+0000 7f7050e00640 1 --2- 172.21.15.2:0/2118663003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c109090 0x7f704c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.543+0000 7f704be00640 1 -- 172.21.15.2:0/2118663003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c109090 msgr2=0x7f704c19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.543+0000 7f704be00640 1 --2- 172.21.15.2:0/2118663003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c109090 0x7f704c19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.543+0000 7f704be00640 1 -- 172.21.15.2:0/2118663003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c104800 msgr2=0x7f704c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.543+0000 7f704be00640 1 --2- 172.21.15.2:0/2118663003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c104800 0x7f704c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.543+0000 7f704be00640 1 -- 172.21.15.2:0/2118663003 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c19f550 con 0x7f704c103e50 2024-09-10T08:18:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.543+0000 7f704be00640 1 --2- 172.21.15.2:0/2118663003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e50 0x7f704c19a5a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f703c00ebc0 tx=0x7f703c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.544+0000 7f7049600640 1 -- 172.21.15.2:0/2118663003 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c010430 con 0x7f704c103e50 2024-09-10T08:18:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.544+0000 7f7051800640 1 -- 172.21.15.2:0/2118663003 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c19f840 con 0x7f704c103e50 2024-09-10T08:18:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.544+0000 7f7049600640 1 -- 172.21.15.2:0/2118663003 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c010ab0 con 0x7f704c103e50 2024-09-10T08:18:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.544+0000 7f7049600640 1 -- 172.21.15.2:0/2118663003 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c018b30 con 0x7f704c103e50 2024-09-10T08:18:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.544+0000 7f7051800640 1 -- 172.21.15.2:0/2118663003 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c1a7120 con 0x7f704c103e50 2024-09-10T08:18:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.547+0000 7f70559d6740 1 Processor -- start 2024-09-10T08:18:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.547+0000 7f7049600640 1 -- 172.21.15.2:0/2118663003 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f703c018cd0 con 0x7f704c103e50 2024-09-10T08:18:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.547+0000 7f70559d6740 1 -- start start 2024-09-10T08:18:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.547+0000 7f70559d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8520 0x55a2855b8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f70559d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8e40 0x55a2855c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f70559d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855c3700 0x55a2855c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f70559d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a28551e650 con 0x55a2855b8e40 2024-09-10T08:18:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f70559d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a2854c5f10 con 0x55a2855c3700 2024-09-10T08:18:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f70559d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a28551a610 con 0x55a2855b8520 2024-09-10T08:18:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f704be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8e40 0x55a2855c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f7050e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8520 0x55a2855b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f704be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8e40 0x55a2855c31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47182/0 (socket says 172.21.15.2:47182) 2024-09-10T08:18:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f704be00640 1 -- 172.21.15.2:0/3712575922 learned_addr learned my addr 172.21.15.2:0/3712575922 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f7043400640 1 --2- 172.21.15.2:0/3712575922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855c3700 0x55a2855c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f704be00640 1 -- 172.21.15.2:0/3712575922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8520 msgr2=0x55a2855b8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:46.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f704be00640 1 --2- 172.21.15.2:0/3712575922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8520 0x55a2855b8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f704be00640 1 -- 172.21.15.2:0/3712575922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855c3700 msgr2=0x55a2855c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:46.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f704be00640 1 --2- 172.21.15.2:0/3712575922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855c3700 0x55a2855c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.548+0000 7f704be00640 1 -- 172.21.15.2:0/3712575922 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a28522d580 con 0x55a2855b8e40 2024-09-10T08:18:46.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.549+0000 7f704be00640 1 --2- 172.21.15.2:0/3712575922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8e40 0x55a2855c31c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f703c0d3b00 tx=0x7f703c0d81e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3a923fd5c9920 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:46.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.549+0000 7f7049600640 1 --2- 172.21.15.2:0/2118663003 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70300a2b80 0x7f70300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.549+0000 7f7049600640 1 -- 172.21.15.2:0/2118663003 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f703c0cfb60 con 0x7f704c103e50 2024-09-10T08:18:46.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.549+0000 7f7041600640 1 -- 172.21.15.2:0/3712575922 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c098210 con 0x55a2855b8e40 2024-09-10T08:18:46.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.549+0000 7f7050e00640 1 --2- 172.21.15.2:0/2118663003 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70300a2b80 0x7f70300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.549+0000 7f7041600640 1 -- 172.21.15.2:0/3712575922 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c0e1070 con 0x55a2855b8e40 2024-09-10T08:18:46.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.549+0000 7f7041600640 1 -- 172.21.15.2:0/3712575922 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c09b8d0 con 0x55a2855b8e40 2024-09-10T08:18:46.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.550+0000 7f70559d6740 1 -- 172.21.15.2:0/3712575922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8e40 msgr2=0x55a2855c31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:46.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.550+0000 7f70559d6740 1 --2- 172.21.15.2:0/3712575922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8e40 0x55a2855c31c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f703c0d3b00 tx=0x7f703c0d81e0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.550+0000 7f7050e00640 1 --2- 172.21.15.2:0/2118663003 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70300a2b80 0x7f70300a5040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7044009d30 tx=0x7f7044005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:46.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.550+0000 7f70559d6740 1 -- 172.21.15.2:0/3712575922 shutdown_connections 2024-09-10T08:18:46.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.550+0000 7f70559d6740 1 --2- 172.21.15.2:0/3712575922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855c3700 0x55a2855c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.550+0000 7f70559d6740 1 --2- 172.21.15.2:0/3712575922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8e40 0x55a2855c31c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.550+0000 7f70559d6740 1 --2- 172.21.15.2:0/3712575922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8520 0x55a2855b8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.550+0000 7f70559d6740 1 -- 172.21.15.2:0/3712575922 >> 172.21.15.2:0/3712575922 conn(0x55a2855b4ab0 msgr2=0x55a2855b4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:46.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.550+0000 7f70559d6740 1 -- 172.21.15.2:0/3712575922 shutdown_connections 2024-09-10T08:18:46.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.550+0000 7f70559d6740 1 -- 172.21.15.2:0/3712575922 wait complete. 2024-09-10T08:18:46.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.551+0000 7f70559d6740 1 -- start start 2024-09-10T08:18:46.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.551+0000 7f70559d6740 1 Processor -- start 2024-09-10T08:18:46.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.551+0000 7f70559d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8520 0x55a2855c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.551+0000 7f70559d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8e40 0x55a2855c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.551+0000 7f70559d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855c3700 0x55a2855c1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.551+0000 7f70559d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a2855c62d0 con 0x55a2855b8520 2024-09-10T08:18:46.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.551+0000 7f70559d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a2854d9350 con 0x55a2855c3700 2024-09-10T08:18:46.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.551+0000 7f70559d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a2855c6450 con 0x55a2855b8e40 2024-09-10T08:18:46.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.551+0000 7f70559d6740 10 client.0 _ll_register_callbacks cb 0x55a2854fc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:46.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.552+0000 7f704be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8520 0x55a2855c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.552+0000 7f704be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8520 0x55a2855c05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47194/0 (socket says 172.21.15.2:47194) 2024-09-10T08:18:46.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.552+0000 7f704be00640 1 -- 172.21.15.2:0/3064663805 learned_addr learned my addr 172.21.15.2:0/3064663805 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:46.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.552+0000 7f7050e00640 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855c3700 0x55a2855c1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.552+0000 7f7043400640 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8e40 0x55a2855c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.552+0000 7f704be00640 1 -- 172.21.15.2:0/3064663805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8e40 msgr2=0x55a2855c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:46.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.552+0000 7f704be00640 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8e40 0x55a2855c0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.552+0000 7f704be00640 1 -- 172.21.15.2:0/3064663805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855c3700 msgr2=0x55a2855c1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:46.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.552+0000 7f704be00640 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855c3700 0x55a2855c1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.552+0000 7f704be00640 1 -- 172.21.15.2:0/3064663805 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2852f7960 con 0x55a2855b8520 2024-09-10T08:18:46.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.552+0000 7f7043400640 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8e40 0x55a2855c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:18:46.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.552+0000 7f704be00640 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8520 0x55a2855c05b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f703c0a46b0 tx=0x7f703c018590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c27e430ad4d781c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:46.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.552+0000 7f703a000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:18:46.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.553+0000 7f703a000640 1 -- 172.21.15.2:0/3064663805 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c09bab0 con 0x55a2855b8520 2024-09-10T08:18:46.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.553+0000 7f703a000640 1 -- 172.21.15.2:0/3064663805 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f703c010c60 con 0x55a2855b8520 2024-09-10T08:18:46.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.553+0000 7f70559d6740 1 -- 172.21.15.2:0/3064663805 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f703c0e1070 con 0x55a2855b8520 2024-09-10T08:18:46.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.553+0000 7f703a000640 1 -- 172.21.15.2:0/3064663805 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f703c09bd40 con 0x55a2855b8520 2024-09-10T08:18:46.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.553+0000 7f70559d6740 10 client.18357 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:46.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.553+0000 7f70559d6740 1 -- 172.21.15.2:0/3064663805 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2855d1ea0 con 0x55a2855b8520 2024-09-10T08:18:46.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.553+0000 7f703a000640 1 -- 172.21.15.2:0/3064663805 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c0e1070 con 0x55a2855b8520 2024-09-10T08:18:46.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.553+0000 7f703a000640 1 -- 172.21.15.2:0/3064663805 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f703c09c1d0 con 0x55a2855b8520 2024-09-10T08:18:46.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.553+0000 7f70559d6740 10 client.18357 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:46.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.553+0000 7f70559d6740 10 client.18357 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:18:46.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.554+0000 7f70559d6740 20 client.18357 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:46.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.554+0000 7f70559d6740 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a2855d31f0 0x55a2855d55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.554+0000 7f70559d6740 4 client.18357 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:18:46.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.554+0000 7f70559d6740 1 -- 172.21.15.2:0/3064663805 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a28552bbc0 con 0x55a2855d31f0 2024-09-10T08:18:46.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.554+0000 7f7043400640 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a2855d31f0 0x55a2855d55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.554+0000 7f7043400640 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a2855d31f0 0x55a2855d55e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f06adfae2f82a38 server_cookie=5b62dccb819159a9 in_seq=0 out_seq=0 2024-09-10T08:18:46.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.554+0000 7f703a000640 10 client.18357 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:46.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.555+0000 7f703a000640 1 -- 172.21.15.2:0/3064663805 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a28552bbc0 con 0x55a2855d31f0 2024-09-10T08:18:46.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.555+0000 7f703a000640 10 client.18357 handle_command_reply: tid=0 2024-09-10T08:18:46.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.555+0000 7f70559d6740 1 client.18357 shutdown 2024-09-10T08:18:46.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.555+0000 7f70559d6740 1 -- 172.21.15.2:0/3064663805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8520 msgr2=0x55a2855c05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:46.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.555+0000 7f70559d6740 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8520 0x55a2855c05b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f703c0a46b0 tx=0x7f703c018590 comp rx=0 tx=0).stop 2024-09-10T08:18:46.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.555+0000 7f70559d6740 1 -- 172.21.15.2:0/3064663805 shutdown_connections 2024-09-10T08:18:46.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.555+0000 7f70559d6740 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a2855d31f0 0x55a2855d55e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.555+0000 7f70559d6740 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855c3700 0x55a2855c1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.555+0000 7f70559d6740 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8e40 0x55a2855c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.555+0000 7f70559d6740 1 --2- 172.21.15.2:0/3064663805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855b8520 0x55a2855c05b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.555+0000 7f703a000640 0 client.18357 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:46.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.555+0000 7f70559d6740 1 -- 172.21.15.2:0/3064663805 >> 172.21.15.2:0/3064663805 conn(0x55a2855b4ab0 msgr2=0x55a2855b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:46.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.556+0000 7f70559d6740 1 -- 172.21.15.2:0/3064663805 shutdown_connections 2024-09-10T08:18:46.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.556+0000 7f70559d6740 1 -- 172.21.15.2:0/3064663805 wait complete. 2024-09-10T08:18:46.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.556+0000 7f70559d6740 20 client.18357 trim_cache size 0 max 16384 2024-09-10T08:18:46.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.574+0000 7f70559d6740 1 Processor -- start 2024-09-10T08:18:46.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.574+0000 7f70559d6740 1 -- start start 2024-09-10T08:18:46.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.574+0000 7f70559d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855b8520 0x55a2855b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.574+0000 7f70559d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8e40 0x55a2855d8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.574+0000 7f70559d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855c3700 0x55a2855d8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.574+0000 7f70559d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f704c1b37e0 con 0x55a2855c3700 2024-09-10T08:18:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.574+0000 7f70559d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f704c1b10b0 con 0x55a2855b8520 2024-09-10T08:18:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.574+0000 7f70559d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f704c1aef00 con 0x55a2855b8e40 2024-09-10T08:18:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.574+0000 7f7050e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855c3700 0x55a2855d8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.574+0000 7f7043400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8e40 0x55a2855d8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.574+0000 7f7050e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855c3700 0x55a2855d8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47196/0 (socket says 172.21.15.2:47196) 2024-09-10T08:18:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.574+0000 7f7050e00640 1 -- 172.21.15.2:0/1137448042 learned_addr learned my addr 172.21.15.2:0/1137448042 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.574+0000 7f704be00640 1 --2- 172.21.15.2:0/1137448042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855b8520 0x55a2855b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.575+0000 7f7043400640 1 -- 172.21.15.2:0/1137448042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855b8520 msgr2=0x55a2855b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:46.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.575+0000 7f7043400640 1 --2- 172.21.15.2:0/1137448042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855b8520 0x55a2855b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.575+0000 7f7043400640 1 -- 172.21.15.2:0/1137448042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855c3700 msgr2=0x55a2855d8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.575+0000 7f7043400640 1 --2- 172.21.15.2:0/1137448042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855c3700 0x55a2855d8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.575+0000 7f7043400640 1 -- 172.21.15.2:0/1137448042 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2855b48e0 con 0x55a2855b8e40 2024-09-10T08:18:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.575+0000 7f7043400640 1 --2- 172.21.15.2:0/1137448042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8e40 0x55a2855d8600 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f703402a9e0 tx=0x7f7034005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a2ba2022484db31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.575+0000 7f7040c00640 1 -- 172.21.15.2:0/1137448042 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034035040 con 0x55a2855b8e40 2024-09-10T08:18:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.576+0000 7f7040c00640 1 -- 172.21.15.2:0/1137448042 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703404a070 con 0x55a2855b8e40 2024-09-10T08:18:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.576+0000 7f7040c00640 1 -- 172.21.15.2:0/1137448042 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034031810 con 0x55a2855b8e40 2024-09-10T08:18:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.576+0000 7f70559d6740 1 -- 172.21.15.2:0/1137448042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8e40 msgr2=0x55a2855d8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.576+0000 7f70559d6740 1 --2- 172.21.15.2:0/1137448042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8e40 0x55a2855d8600 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f703402a9e0 tx=0x7f7034005b50 comp rx=0 tx=0).stop 2024-09-10T08:18:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.576+0000 7f70559d6740 1 -- 172.21.15.2:0/1137448042 shutdown_connections 2024-09-10T08:18:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.576+0000 7f70559d6740 1 --2- 172.21.15.2:0/1137448042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855c3700 0x55a2855d8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.576+0000 7f70559d6740 1 --2- 172.21.15.2:0/1137448042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8e40 0x55a2855d8600 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.576+0000 7f70559d6740 1 --2- 172.21.15.2:0/1137448042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855b8520 0x55a2855b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.576+0000 7f70559d6740 1 -- 172.21.15.2:0/1137448042 >> 172.21.15.2:0/1137448042 conn(0x55a2855b4ab0 msgr2=0x55a2855bf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.576+0000 7f70559d6740 1 -- 172.21.15.2:0/1137448042 shutdown_connections 2024-09-10T08:18:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.576+0000 7f70559d6740 1 -- 172.21.15.2:0/1137448042 wait complete. 2024-09-10T08:18:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.577+0000 7f70559d6740 1 -- start start 2024-09-10T08:18:46.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.577+0000 7f70559d6740 1 Processor -- start 2024-09-10T08:18:46.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.577+0000 7f70559d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8520 0x55a2855d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.577+0000 7f70559d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855b8e40 0x55a2855d7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.577+0000 7f70559d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855c3700 0x55a28563cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.577+0000 7f70559d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f704c1af570 con 0x55a2855c3700 2024-09-10T08:18:46.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.578+0000 7f70559d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f704c1b09e0 con 0x55a2855b8e40 2024-09-10T08:18:46.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.578+0000 7f70559d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f704c1afc40 con 0x55a2855b8520 2024-09-10T08:18:46.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.578+0000 7f70559d6740 10 client.0 _ll_register_callbacks cb 0x7f703c0cff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:46.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.578+0000 7f704be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8520 0x55a2855d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.578+0000 7f7050e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855c3700 0x55a28563cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.578+0000 7f704be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8520 0x55a2855d6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:52334/0 (socket says 172.21.15.2:52334) 2024-09-10T08:18:46.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.578+0000 7f704be00640 1 -- 172.21.15.2:0/977259227 learned_addr learned my addr 172.21.15.2:0/977259227 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:46.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.578+0000 7f7043400640 1 --2- 172.21.15.2:0/977259227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855b8e40 0x55a2855d7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.578+0000 7f704be00640 1 -- 172.21.15.2:0/977259227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855b8e40 msgr2=0x55a2855d7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:46.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.578+0000 7f704be00640 1 --2- 172.21.15.2:0/977259227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855b8e40 0x55a2855d7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.578+0000 7f704be00640 1 -- 172.21.15.2:0/977259227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855c3700 msgr2=0x55a28563cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:46.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.578+0000 7f704be00640 1 --2- 172.21.15.2:0/977259227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855c3700 0x55a28563cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.578+0000 7f704be00640 1 -- 172.21.15.2:0/977259227 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a28563d0f0 con 0x55a2855b8520 2024-09-10T08:18:46.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.579+0000 7f704be00640 1 --2- 172.21.15.2:0/977259227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8520 0x55a2855d6b00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f703c09bf00 tx=0x7f703c00cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9cc5fa84a0ab293 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:46.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.579+0000 7f703a000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:18:46.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.579+0000 7f703a000640 1 -- 172.21.15.2:0/977259227 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c0d3040 con 0x55a2855b8520 2024-09-10T08:18:46.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.579+0000 7f703a000640 1 -- 172.21.15.2:0/977259227 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f703c010c60 con 0x55a2855b8520 2024-09-10T08:18:46.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.579+0000 7f70559d6740 1 -- 172.21.15.2:0/977259227 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f703404a070 con 0x55a2855b8520 2024-09-10T08:18:46.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.579+0000 7f703a000640 1 -- 172.21.15.2:0/977259227 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f703c0b6030 con 0x55a2855b8520 2024-09-10T08:18:46.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.579+0000 7f70559d6740 10 client.28330 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:46.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.579+0000 7f70559d6740 1 -- 172.21.15.2:0/977259227 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a28563e3f0 con 0x55a2855b8520 2024-09-10T08:18:46.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.579+0000 7f703a000640 1 -- 172.21.15.2:0/977259227 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703404a070 con 0x55a2855b8520 2024-09-10T08:18:46.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.580+0000 7f703a000640 1 -- 172.21.15.2:0/977259227 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f703c0a4750 con 0x55a2855b8520 2024-09-10T08:18:46.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.580+0000 7f70559d6740 10 client.28330 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:46.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.580+0000 7f70559d6740 10 client.28330 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:18:46.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.580+0000 7f70559d6740 20 client.28330 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:46.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.580+0000 7f70559d6740 1 --2- 172.21.15.2:0/977259227 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a28563e920 0x55a285640d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:46.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.580+0000 7f70559d6740 4 client.28330 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ba7e0923", "release": true, "await": true}] 2024-09-10T08:18:46.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.580+0000 7f70559d6740 1 -- 172.21.15.2:0/977259227 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ba7e0923", "release": true, "await": true}) -- 0x55a2855260d0 con 0x55a28563e920 2024-09-10T08:18:46.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.580+0000 7f7043400640 1 --2- 172.21.15.2:0/977259227 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a28563e920 0x55a285640d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:46.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.581+0000 7f7043400640 1 --2- 172.21.15.2:0/977259227 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a28563e920 0x55a285640d10 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14df36a4c17c91c2 server_cookie=8f1b04873122c25b in_seq=0 out_seq=0 2024-09-10T08:18:46.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.581+0000 7f703a000640 10 client.28330 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:46.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.583+0000 7f703a000640 1 -- 172.21.15.2:0/977259227 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a2855260d0 con 0x55a28563e920 2024-09-10T08:18:46.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.583+0000 7f703a000640 10 client.28330 handle_command_reply: tid=0 2024-09-10T08:18:46.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.583+0000 7f70559d6740 1 client.28330 shutdown 2024-09-10T08:18:46.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.583+0000 7f70559d6740 1 -- 172.21.15.2:0/977259227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8520 msgr2=0x55a2855d6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:46.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.583+0000 7f70559d6740 1 --2- 172.21.15.2:0/977259227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8520 0x55a2855d6b00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f703c09bf00 tx=0x7f703c00cf10 comp rx=0 tx=0).stop 2024-09-10T08:18:46.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.583+0000 7f70559d6740 1 -- 172.21.15.2:0/977259227 shutdown_connections 2024-09-10T08:18:46.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.583+0000 7f70559d6740 1 --2- 172.21.15.2:0/977259227 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a28563e920 0x55a285640d10 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.584+0000 7f70559d6740 1 --2- 172.21.15.2:0/977259227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2855c3700 0x55a28563cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.584+0000 7f703a000640 0 client.28330 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:46.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.584+0000 7f70559d6740 1 --2- 172.21.15.2:0/977259227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2855b8e40 0x55a2855d7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.584+0000 7f70559d6740 1 --2- 172.21.15.2:0/977259227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2855b8520 0x55a2855d6b00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.584+0000 7f70559d6740 1 -- 172.21.15.2:0/977259227 >> 172.21.15.2:0/977259227 conn(0x55a2855b4ab0 msgr2=0x55a2855c0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:46.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.584+0000 7f70559d6740 1 -- 172.21.15.2:0/977259227 shutdown_connections 2024-09-10T08:18:46.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.584+0000 7f70559d6740 1 -- 172.21.15.2:0/977259227 wait complete. 2024-09-10T08:18:46.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.584+0000 7f70559d6740 20 client.28330 trim_cache size 0 max 16384 2024-09-10T08:18:46.585 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:18:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:18:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:18:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 36, 2024-09-10T08:18:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:18:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "ba7e0923": { 2024-09-10T08:18:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 36, 2024-09-10T08:18:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:18:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:18:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:18:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:18:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:18:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:18:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 130.0, 2024-09-10T08:18:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:18:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:18:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:18:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:18:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:18:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:18:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:18:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:18:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:18:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:18:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:18:46.588 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:18:46.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.584+0000 7f7051800640 1 -- 172.21.15.2:0/2118663003 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70300a2b80 msgr2=0x7f70300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:46.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.584+0000 7f7051800640 1 --2- 172.21.15.2:0/2118663003 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70300a2b80 0x7f70300a5040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7044009d30 tx=0x7f7044005f90 comp rx=0 tx=0).stop 2024-09-10T08:18:46.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.584+0000 7f7051800640 1 -- 172.21.15.2:0/2118663003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e50 msgr2=0x7f704c19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:46.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.584+0000 7f7051800640 1 --2- 172.21.15.2:0/2118663003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e50 0x7f704c19a5a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f703c00ebc0 tx=0x7f703c00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.585+0000 7f7051800640 1 -- 172.21.15.2:0/2118663003 shutdown_connections 2024-09-10T08:18:46.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.585+0000 7f7051800640 1 --2- 172.21.15.2:0/2118663003 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70300a2b80 0x7f70300a5040 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.585+0000 7f7051800640 1 --2- 172.21.15.2:0/2118663003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c109090 0x7f704c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.585+0000 7f7051800640 1 --2- 172.21.15.2:0/2118663003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f704c104800 0x7f704c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.585+0000 7f7051800640 1 --2- 172.21.15.2:0/2118663003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e50 0x7f704c19a5a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:46.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.585+0000 7f7051800640 1 -- 172.21.15.2:0/2118663003 >> 172.21.15.2:0/2118663003 conn(0x7f704c0ff8b0 msgr2=0x7f704c100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:46.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.585+0000 7f7051800640 1 -- 172.21.15.2:0/2118663003 shutdown_connections 2024-09-10T08:18:46.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:46.585+0000 7f7051800640 1 -- 172.21.15.2:0/2118663003 wait complete. 2024-09-10T08:18:47.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:47 smithi002 ceph-mon[30563]: pgmap v706: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-09-10T08:18:47.379 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ba7e0923', total seconds elapsed: 14.0 2024-09-10T08:18:47.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:47 smithi155 ceph-mon[35405]: pgmap v706: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-09-10T08:18:47.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:47 smithi107 ceph-mon[35048]: pgmap v706: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-09-10T08:18:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.282+0000 7fa1eca00640 1 -- 172.21.15.2:0/1596749806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1e8104cf0 msgr2=0x7fa1e81070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.282+0000 7fa1eca00640 1 --2- 172.21.15.2:0/1596749806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1e8104cf0 0x7fa1e81070e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009900 tx=0x7fa1d002a410 comp rx=0 tx=0).stop 2024-09-10T08:18:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.282+0000 7fa1eca00640 1 -- 172.21.15.2:0/1596749806 shutdown_connections 2024-09-10T08:18:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.282+0000 7fa1eca00640 1 --2- 172.21.15.2:0/1596749806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1e8107620 0x7fa1e8109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.282+0000 7fa1eca00640 1 --2- 172.21.15.2:0/1596749806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1e8104cf0 0x7fa1e81070e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.282+0000 7fa1eca00640 1 --2- 172.21.15.2:0/1596749806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e81023c0 0x7fa1e81047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.282+0000 7fa1eca00640 1 -- 172.21.15.2:0/1596749806 >> 172.21.15.2:0/1596749806 conn(0x7fa1e80fbfa0 msgr2=0x7fa1e80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.282+0000 7fa1eca00640 1 -- 172.21.15.2:0/1596749806 shutdown_connections 2024-09-10T08:18:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.282+0000 7fa1eca00640 1 -- 172.21.15.2:0/1596749806 wait complete. 2024-09-10T08:18:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.283+0000 7fa1eca00640 1 Processor -- start 2024-09-10T08:18:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.283+0000 7fa1eca00640 1 -- start start 2024-09-10T08:18:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.283+0000 7fa1eca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1e81023c0 0x7fa1e8101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.283+0000 7fa1eca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e8104cf0 0x7fa1e8100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.283+0000 7fa1eca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1e8107620 0x7fa1e81008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.283+0000 7fa1eca00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e810d620 con 0x7fa1e8107620 2024-09-10T08:18:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.283+0000 7fa1eca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e810d4a0 con 0x7fa1e8104cf0 2024-09-10T08:18:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.283+0000 7fa1eca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e810d7a0 con 0x7fa1e81023c0 2024-09-10T08:18:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.283+0000 7fa1dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e8104cf0 0x7fa1e8100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.283+0000 7fa1dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e8104cf0 0x7fa1e8100370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35946/0 (socket says 172.21.15.2:35946) 2024-09-10T08:18:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.283+0000 7fa1dfe00640 1 -- 172.21.15.2:0/2777570324 learned_addr learned my addr 172.21.15.2:0/2777570324 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.283+0000 7fa1e7400640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1e81023c0 0x7fa1e8101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.283+0000 7fa1e7e00640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1e8107620 0x7fa1e81008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.284+0000 7fa1dfe00640 1 -- 172.21.15.2:0/2777570324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1e81023c0 msgr2=0x7fa1e8101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.284+0000 7fa1dfe00640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1e81023c0 0x7fa1e8101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.284+0000 7fa1dfe00640 1 -- 172.21.15.2:0/2777570324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1e8107620 msgr2=0x7fa1e81008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.284+0000 7fa1dfe00640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1e8107620 0x7fa1e81008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.284+0000 7fa1dfe00640 1 -- 172.21.15.2:0/2777570324 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e8101140 con 0x7fa1e8104cf0 2024-09-10T08:18:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.284+0000 7fa1e7400640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1e81023c0 0x7fa1e8101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.284+0000 7fa1dfe00640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e8104cf0 0x7fa1e8100370 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0031440 tx=0x7fa1d0032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.284+0000 7fa1e4c00640 1 -- 172.21.15.2:0/2777570324 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d003f030 con 0x7fa1e8104cf0 2024-09-10T08:18:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.284+0000 7fa1e4c00640 1 -- 172.21.15.2:0/2777570324 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d0003ce0 con 0x7fa1e8104cf0 2024-09-10T08:18:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.285+0000 7fa1eca00640 1 -- 172.21.15.2:0/2777570324 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e81a1dd0 con 0x7fa1e8104cf0 2024-09-10T08:18:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.285+0000 7fa1e4c00640 1 -- 172.21.15.2:0/2777570324 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0035070 con 0x7fa1e8104cf0 2024-09-10T08:18:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.285+0000 7fa1eca00640 1 -- 172.21.15.2:0/2777570324 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e81a2230 con 0x7fa1e8104cf0 2024-09-10T08:18:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.288+0000 7fa1f0e9c740 1 Processor -- start 2024-09-10T08:18:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.288+0000 7fa1f0e9c740 1 -- start start 2024-09-10T08:18:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.288+0000 7fa1f0e9c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6260 0x563a6e2c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.288+0000 7fa1e4c00640 1 -- 172.21.15.2:0/2777570324 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa1d0042030 con 0x7fa1e8104cf0 2024-09-10T08:18:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.288+0000 7fa1f0e9c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.288+0000 7fa1f0e9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2cbc30 0x563a6e2ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.288+0000 7fa1f0e9c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563a6e22b100 con 0x563a6e2c6260 2024-09-10T08:18:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.288+0000 7fa1f0e9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563a6e1d31b0 con 0x563a6e2cbc30 2024-09-10T08:18:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.288+0000 7fa1f0e9c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563a6e2270c0 con 0x563a6e2c6ba0 2024-09-10T08:18:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.289+0000 7fa1e7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.289+0000 7fa1e7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:43328/0 (socket says 172.21.15.2:43328) 2024-09-10T08:18:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.289+0000 7fa1e7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6260 0x563a6e2c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.289+0000 7fa1e7400640 1 -- 172.21.15.2:0/4038415905 learned_addr learned my addr 172.21.15.2:0/4038415905 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.289+0000 7fa1dfe00640 1 --2- 172.21.15.2:0/4038415905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2cbc30 0x563a6e2ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.289+0000 7fa1e7400640 1 -- 172.21.15.2:0/4038415905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2cbc30 msgr2=0x563a6e2ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.289+0000 7fa1e7400640 1 --2- 172.21.15.2:0/4038415905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2cbc30 0x563a6e2ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.289+0000 7fa1e7400640 1 -- 172.21.15.2:0/4038415905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6260 msgr2=0x563a6e2c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.289+0000 7fa1e7400640 1 --2- 172.21.15.2:0/4038415905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6260 0x563a6e2c6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.289+0000 7fa1e7400640 1 -- 172.21.15.2:0/4038415905 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6df3e210 con 0x563a6e2c6ba0 2024-09-10T08:18:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.290+0000 7fa1e7400640 1 --2- 172.21.15.2:0/4038415905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2cb660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80115f0 tx=0x7fa1d8011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76b39166b3517ec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.290+0000 7fa1d7e00640 1 -- 172.21.15.2:0/4038415905 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d801e070 con 0x563a6e2c6ba0 2024-09-10T08:18:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.290+0000 7fa1d7e00640 1 -- 172.21.15.2:0/4038415905 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d800fb20 con 0x563a6e2c6ba0 2024-09-10T08:18:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.290+0000 7fa1d7e00640 1 -- 172.21.15.2:0/4038415905 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8019c80 con 0x563a6e2c6ba0 2024-09-10T08:18:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.290+0000 7fa1e4c00640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1cc0a2b80 0x7fa1cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.290+0000 7fa1e4c00640 1 -- 172.21.15.2:0/2777570324 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa1d00ef820 con 0x7fa1e8104cf0 2024-09-10T08:18:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.290+0000 7fa1e7e00640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1cc0a2b80 0x7fa1cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.291+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/4038415905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 msgr2=0x563a6e2cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.291+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/4038415905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2cb660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80115f0 tx=0x7fa1d8011ab0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.291+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/4038415905 shutdown_connections 2024-09-10T08:18:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.291+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/4038415905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2cbc30 0x563a6e2ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.291+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/4038415905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2cb660 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.291+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/4038415905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6260 0x563a6e2c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.291+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/4038415905 >> 172.21.15.2:0/4038415905 conn(0x563a6e2c18d0 msgr2=0x563a6e2c1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.291+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/4038415905 shutdown_connections 2024-09-10T08:18:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.291+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/4038415905 wait complete. 2024-09-10T08:18:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.291+0000 7fa1e7e00640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1cc0a2b80 0x7fa1cc0a5040 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa1e806a980 tx=0x7fa1e0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.292+0000 7fa1f0e9c740 1 -- start start 2024-09-10T08:18:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.292+0000 7fa1f0e9c740 1 Processor -- start 2024-09-10T08:18:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.292+0000 7fa1f0e9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2c6260 0x563a6e2d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.292+0000 7fa1f0e9c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.292+0000 7fa1f0e9c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2cbc30 0x563a6e2d79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.292+0000 7fa1f0e9c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563a6e2cec00 con 0x563a6e2cbc30 2024-09-10T08:18:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.292+0000 7fa1f0e9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563a6e1b4090 con 0x563a6e2c6260 2024-09-10T08:18:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.292+0000 7fa1f0e9c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563a6e2ced80 con 0x563a6e2c6ba0 2024-09-10T08:18:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.292+0000 7fa1f0e9c740 10 client.0 _ll_register_callbacks cb 0x563a6e208f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.293+0000 7fa1e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2c6260 0x563a6e2d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.293+0000 7fa1e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2c6260 0x563a6e2d0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35970/0 (socket says 172.21.15.2:35970) 2024-09-10T08:18:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.293+0000 7fa1e7400640 1 -- 172.21.15.2:0/2544950006 learned_addr learned my addr 172.21.15.2:0/2544950006 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.293+0000 7fa1e7e00640 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2cbc30 0x563a6e2d79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.293+0000 7fa1dfe00640 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.293+0000 7fa1e7400640 1 -- 172.21.15.2:0/2544950006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 msgr2=0x563a6e2d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.293+0000 7fa1e7400640 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.293+0000 7fa1e7400640 1 -- 172.21.15.2:0/2544950006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2cbc30 msgr2=0x563a6e2d79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.293+0000 7fa1e7400640 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2cbc30 0x563a6e2d79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.293+0000 7fa1e7400640 1 -- 172.21.15.2:0/2544950006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a6e00a620 con 0x563a6e2c6260 2024-09-10T08:18:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.293+0000 7fa1e7e00640 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2cbc30 0x563a6e2d79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:18:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.293+0000 7fa1e7400640 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2c6260 0x563a6e2d0450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80115c0 tx=0x7fa1d800f870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b3b886376020d80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.293+0000 7fa1d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:18:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.294+0000 7fa1d4c00640 1 -- 172.21.15.2:0/2544950006 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d801e040 con 0x563a6e2c6260 2024-09-10T08:18:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.294+0000 7fa1d4c00640 1 -- 172.21.15.2:0/2544950006 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa1d80199c0 con 0x563a6e2c6260 2024-09-10T08:18:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.294+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/2544950006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d800fb20 con 0x563a6e2c6260 2024-09-10T08:18:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.294+0000 7fa1d4c00640 1 -- 172.21.15.2:0/2544950006 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa1d802f050 con 0x563a6e2c6260 2024-09-10T08:18:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.294+0000 7fa1f0e9c740 10 client.28406 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.294+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/2544950006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a6e2da7c0 con 0x563a6e2c6260 2024-09-10T08:18:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.294+0000 7fa1d4c00640 1 -- 172.21.15.2:0/2544950006 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d800fb20 con 0x563a6e2c6260 2024-09-10T08:18:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.295+0000 7fa1d4c00640 1 -- 172.21.15.2:0/2544950006 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa1d802a030 con 0x563a6e2c6260 2024-09-10T08:18:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.295+0000 7fa1f0e9c740 10 client.28406 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.295+0000 7fa1f0e9c740 10 client.28406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.295+0000 7fa1f0e9c740 20 client.28406 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.295+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563a6e2dbcc0 0x563a6e2de130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.295+0000 7fa1f0e9c740 4 client.28406 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:18:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.295+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/2544950006 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a6e238830 con 0x563a6e2dbcc0 2024-09-10T08:18:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.295+0000 7fa1dfe00640 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563a6e2dbcc0 0x563a6e2de130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.295+0000 7fa1dfe00640 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563a6e2dbcc0 0x563a6e2de130 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b679aa31015d288 server_cookie=901704a9ee0cb67c in_seq=0 out_seq=0 2024-09-10T08:18:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.295+0000 7fa1d4c00640 10 client.28406 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.296+0000 7fa1d4c00640 1 -- 172.21.15.2:0/2544950006 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a6e238830 con 0x563a6e2dbcc0 2024-09-10T08:18:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.296+0000 7fa1d4c00640 10 client.28406 handle_command_reply: tid=0 2024-09-10T08:18:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.296+0000 7fa1f0e9c740 1 client.28406 shutdown 2024-09-10T08:18:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.296+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/2544950006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2c6260 msgr2=0x563a6e2d0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.296+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2c6260 0x563a6e2d0450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80115c0 tx=0x7fa1d800f870 comp rx=0 tx=0).stop 2024-09-10T08:18:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.296+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/2544950006 shutdown_connections 2024-09-10T08:18:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.296+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563a6e2dbcc0 0x563a6e2de130 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.297+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2cbc30 0x563a6e2d79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.297+0000 7fa1d4c00640 0 client.28406 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.297+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.297+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/2544950006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2c6260 0x563a6e2d0450 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.297+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/2544950006 >> 172.21.15.2:0/2544950006 conn(0x563a6e2c18d0 msgr2=0x563a6e2c30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.297+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/2544950006 shutdown_connections 2024-09-10T08:18:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.297+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/2544950006 wait complete. 2024-09-10T08:18:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.297+0000 7fa1f0e9c740 20 client.28406 trim_cache size 0 max 16384 2024-09-10T08:18:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.317+0000 7fa1f0e9c740 1 Processor -- start 2024-09-10T08:18:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.317+0000 7fa1f0e9c740 1 -- start start 2024-09-10T08:18:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.317+0000 7fa1f0e9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2c6260 0x563a6e2c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.317+0000 7fa1f0e9c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.317+0000 7fa1f0e9c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2cbc30 0x563a6e2d7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.317+0000 7fa1f0e9c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81af260 con 0x563a6e2c6ba0 2024-09-10T08:18:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.317+0000 7fa1f0e9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81acb30 con 0x563a6e2c6260 2024-09-10T08:18:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.317+0000 7fa1f0e9c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81aa980 con 0x563a6e2cbc30 2024-09-10T08:18:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.317+0000 7fa1e7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2cbc30 0x563a6e2d7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.317+0000 7fa1dfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.318+0000 7fa1e7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2cbc30 0x563a6e2d7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:43348/0 (socket says 172.21.15.2:43348) 2024-09-10T08:18:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.318+0000 7fa1e7e00640 1 -- 172.21.15.2:0/2270308812 learned_addr learned my addr 172.21.15.2:0/2270308812 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.318+0000 7fa1e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2c6260 0x563a6e2c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.318+0000 7fa1e7e00640 1 -- 172.21.15.2:0/2270308812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2c6260 msgr2=0x563a6e2c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.318+0000 7fa1e7e00640 1 --2- 172.21.15.2:0/2270308812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2c6260 0x563a6e2c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.318+0000 7fa1e7e00640 1 -- 172.21.15.2:0/2270308812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6ba0 msgr2=0x563a6e2d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.318+0000 7fa1e7e00640 1 --2- 172.21.15.2:0/2270308812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.318+0000 7fa1e7e00640 1 -- 172.21.15.2:0/2270308812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6e2c16f0 con 0x563a6e2cbc30 2024-09-10T08:18:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.318+0000 7fa1e7400640 1 --2- 172.21.15.2:0/2270308812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2c6260 0x563a6e2c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.318+0000 7fa1dfe00640 1 --2- 172.21.15.2:0/2270308812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.318+0000 7fa1e7e00640 1 --2- 172.21.15.2:0/2270308812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2cbc30 0x563a6e2d7940 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0003e20 tx=0x7fa1e0016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a19d4af634d64013 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.318+0000 7fa1d7400640 1 -- 172.21.15.2:0/2270308812 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0004b20 con 0x563a6e2cbc30 2024-09-10T08:18:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.318+0000 7fa1d7400640 1 -- 172.21.15.2:0/2270308812 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e0004cc0 con 0x563a6e2cbc30 2024-09-10T08:18:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.319+0000 7fa1d7400640 1 -- 172.21.15.2:0/2270308812 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0005150 con 0x563a6e2cbc30 2024-09-10T08:18:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.319+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/2270308812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2cbc30 msgr2=0x563a6e2d7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.319+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/2270308812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2cbc30 0x563a6e2d7940 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0003e20 tx=0x7fa1e0016000 comp rx=0 tx=0).stop 2024-09-10T08:18:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.319+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/2270308812 shutdown_connections 2024-09-10T08:18:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.319+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/2270308812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2cbc30 0x563a6e2d7940 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.319+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/2270308812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.319+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/2270308812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2c6260 0x563a6e2c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.319+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/2270308812 >> 172.21.15.2:0/2270308812 conn(0x563a6e2c18d0 msgr2=0x563a6e2c0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.319+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/2270308812 shutdown_connections 2024-09-10T08:18:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.319+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/2270308812 wait complete. 2024-09-10T08:18:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.320+0000 7fa1f0e9c740 1 -- start start 2024-09-10T08:18:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.320+0000 7fa1f0e9c740 1 Processor -- start 2024-09-10T08:18:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1f0e9c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6260 0x563a6e2d5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1f0e9c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1f0e9c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2cbc30 0x563a6e2d6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1f0e9c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81aaff0 con 0x563a6e2c6260 2024-09-10T08:18:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1f0e9c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81ac460 con 0x563a6e2cbc30 2024-09-10T08:18:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1f0e9c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81ab6c0 con 0x563a6e2c6ba0 2024-09-10T08:18:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1f0e9c740 10 client.0 _ll_register_callbacks cb 0x7fa1d801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1e7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6260 0x563a6e2d5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1dfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1e7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2cbc30 0x563a6e2d6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1dfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:43358/0 (socket says 172.21.15.2:43358) 2024-09-10T08:18:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1dfe00640 1 -- 172.21.15.2:0/180916623 learned_addr learned my addr 172.21.15.2:0/180916623 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1dfe00640 1 -- 172.21.15.2:0/180916623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2cbc30 msgr2=0x563a6e2d6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1dfe00640 1 --2- 172.21.15.2:0/180916623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2cbc30 0x563a6e2d6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1dfe00640 1 -- 172.21.15.2:0/180916623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6260 msgr2=0x563a6e2d5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1dfe00640 1 --2- 172.21.15.2:0/180916623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6260 0x563a6e2d5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.321+0000 7fa1dfe00640 1 -- 172.21.15.2:0/180916623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a6e2d6d90 con 0x563a6e2c6ba0 2024-09-10T08:18:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1dfe00640 1 --2- 172.21.15.2:0/180916623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d5770 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00b81a0 tx=0x7fa1d00ece00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce90060968bbff9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:18:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1d4c00640 1 -- 172.21.15.2:0/180916623 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0049040 con 0x563a6e2c6ba0 2024-09-10T08:18:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/180916623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1e0004cc0 con 0x563a6e2c6ba0 2024-09-10T08:18:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1d4c00640 1 -- 172.21.15.2:0/180916623 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa1d00b83b0 con 0x563a6e2c6ba0 2024-09-10T08:18:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1d4c00640 1 -- 172.21.15.2:0/180916623 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa1d00bbd40 con 0x563a6e2c6ba0 2024-09-10T08:18:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1f0e9c740 10 client.28348 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/180916623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a6e346ba0 con 0x563a6e2c6ba0 2024-09-10T08:18:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1d4c00640 1 -- 172.21.15.2:0/180916623 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e0004cc0 con 0x563a6e2c6ba0 2024-09-10T08:18:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1d4c00640 1 -- 172.21.15.2:0/180916623 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa1d00d3030 con 0x563a6e2c6ba0 2024-09-10T08:18:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1f0e9c740 10 client.28348 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1f0e9c740 10 client.28348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1f0e9c740 20 client.28348 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/180916623 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563a6e3474d0 0x563a6e3498f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1f0e9c740 4 client.28348 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:18:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.322+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/180916623 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a6e232b80 con 0x563a6e3474d0 2024-09-10T08:18:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.323+0000 7fa1e7400640 1 --2- 172.21.15.2:0/180916623 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563a6e3474d0 0x563a6e3498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.323+0000 7fa1d4c00640 10 client.28348 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.323+0000 7fa1e7400640 1 --2- 172.21.15.2:0/180916623 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563a6e3474d0 0x563a6e3498f0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=444926f1ac1422f3 server_cookie=ef8e1008f1490576 in_seq=0 out_seq=0 2024-09-10T08:18:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.335+0000 7fa1d4c00640 1 -- 172.21.15.2:0/180916623 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563a6e232b80 con 0x563a6e3474d0 2024-09-10T08:18:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.335+0000 7fa1d4c00640 10 client.28348 handle_command_reply: tid=0 2024-09-10T08:18:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.335+0000 7fa1f0e9c740 1 client.28348 shutdown 2024-09-10T08:18:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.335+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/180916623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 msgr2=0x563a6e2d5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.335+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/180916623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d5770 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00b81a0 tx=0x7fa1d00ece00 comp rx=0 tx=0).stop 2024-09-10T08:18:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.335+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/180916623 shutdown_connections 2024-09-10T08:18:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.335+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/180916623 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563a6e3474d0 0x563a6e3498f0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.335+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/180916623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6e2cbc30 0x563a6e2d6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.335+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/180916623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6e2c6ba0 0x563a6e2d5770 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.335+0000 7fa1f0e9c740 1 --2- 172.21.15.2:0/180916623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563a6e2c6260 0x563a6e2d5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.335+0000 7fa1d4c00640 0 client.28348 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.336+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/180916623 >> 172.21.15.2:0/180916623 conn(0x563a6e2c18d0 msgr2=0x563a6e2c1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.336+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/180916623 shutdown_connections 2024-09-10T08:18:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.336+0000 7fa1f0e9c740 1 -- 172.21.15.2:0/180916623 wait complete. 2024-09-10T08:18:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.336+0000 7fa1f0e9c740 20 client.28348 trim_cache size 0 max 16384 2024-09-10T08:18:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:18:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:18:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:18:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:18:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.336+0000 7fa1eca00640 1 -- 172.21.15.2:0/2777570324 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1cc0a2b80 msgr2=0x7fa1cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.336+0000 7fa1eca00640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1cc0a2b80 0x7fa1cc0a5040 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa1e806a980 tx=0x7fa1e0009500 comp rx=0 tx=0).stop 2024-09-10T08:18:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.336+0000 7fa1eca00640 1 -- 172.21.15.2:0/2777570324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e8104cf0 msgr2=0x7fa1e8100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.336+0000 7fa1eca00640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e8104cf0 0x7fa1e8100370 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0031440 tx=0x7fa1d0032ce0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.337+0000 7fa1eca00640 1 -- 172.21.15.2:0/2777570324 shutdown_connections 2024-09-10T08:18:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.337+0000 7fa1eca00640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1e8107620 0x7fa1e81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.337+0000 7fa1eca00640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e8104cf0 0x7fa1e8100370 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.337+0000 7fa1eca00640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1cc0a2b80 0x7fa1cc0a5040 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.337+0000 7fa1eca00640 1 --2- 172.21.15.2:0/2777570324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1e81023c0 0x7fa1e8101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.337+0000 7fa1eca00640 1 -- 172.21.15.2:0/2777570324 >> 172.21.15.2:0/2777570324 conn(0x7fa1e80fbfa0 msgr2=0x7fa1e8102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.337+0000 7fa1eca00640 1 -- 172.21.15.2:0/2777570324 shutdown_connections 2024-09-10T08:18:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:18:49.337+0000 7fa1eca00640 1 -- 172.21.15.2:0/2777570324 wait complete. 2024-09-10T08:18:49.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:49 smithi155 ceph-mon[35405]: pgmap v707: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 12 op/s 2024-09-10T08:18:49.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:49 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:18:49.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:49 smithi107 ceph-mon[35048]: pgmap v707: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 12 op/s 2024-09-10T08:18:49.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:49 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:18:49.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:49 smithi002 ceph-mon[30563]: pgmap v707: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 12 op/s 2024-09-10T08:18:49.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:49 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:18:50.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:18:50.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:18:50.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:18:50.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:18:50.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:50.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:18:50.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:18:50.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:18:51.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:51 smithi107 ceph-mon[35048]: pgmap v708: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-10T08:18:51.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:51 smithi002 ceph-mon[30563]: pgmap v708: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-10T08:18:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:51 smithi155 ceph-mon[35405]: pgmap v708: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-10T08:18:53.669 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:53 smithi107 ceph-mon[35048]: pgmap v709: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-10T08:18:53.709 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:53 smithi155 ceph-mon[35405]: pgmap v709: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-10T08:18:53.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:53 smithi002 ceph-mon[30563]: pgmap v709: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-10T08:18:55.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:55 smithi002 ceph-mon[30563]: pgmap v710: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-09-10T08:18:55.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:55 smithi155 ceph-mon[35405]: pgmap v710: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-09-10T08:18:55.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:55 smithi107 ceph-mon[35048]: pgmap v710: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-09-10T08:18:57.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:57 smithi002 ceph-mon[30563]: pgmap v711: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-09-10T08:18:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:57 smithi155 ceph-mon[35405]: pgmap v711: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-09-10T08:18:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:57 smithi107 ceph-mon[35048]: pgmap v711: 129 pgs: 129 active+clean; 1.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-09-10T08:18:58.785 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:18:59.176 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:18:59.662 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:18:59 smithi002 ceph-mon[30563]: pgmap v712: 129 pgs: 129 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-10T08:18:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.779+0000 7f53e6a00640 1 -- 172.21.15.2:0/4002880092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53e0102790 msgr2=0x7f53e01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.779+0000 7f53e6a00640 1 --2- 172.21.15.2:0/4002880092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53e0102790 0x7f53e01067e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009900 tx=0x7f53c802a410 comp rx=0 tx=0).stop 2024-09-10T08:18:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.779+0000 7f53e6a00640 1 -- 172.21.15.2:0/4002880092 shutdown_connections 2024-09-10T08:18:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.779+0000 7f53e6a00640 1 --2- 172.21.15.2:0/4002880092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0106f10 0x7f53e010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.779+0000 7f53e6a00640 1 --2- 172.21.15.2:0/4002880092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53e0102790 0x7f53e01067e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.779+0000 7f53e6a00640 1 --2- 172.21.15.2:0/4002880092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e0101de0 0x7f53e01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.779+0000 7f53e6a00640 1 -- 172.21.15.2:0/4002880092 >> 172.21.15.2:0/4002880092 conn(0x7f53e00fd8c0 msgr2=0x7f53e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:59.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.779+0000 7f53e6a00640 1 -- 172.21.15.2:0/4002880092 shutdown_connections 2024-09-10T08:18:59.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.779+0000 7f53e6a00640 1 -- 172.21.15.2:0/4002880092 wait complete. 2024-09-10T08:18:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.780+0000 7f53e6a00640 1 Processor -- start 2024-09-10T08:18:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.780+0000 7f53e6a00640 1 -- start start 2024-09-10T08:18:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.780+0000 7f53e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0101de0 0x7f53e019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.780+0000 7f53e6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e0102790 0x7f53e019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.780+0000 7f53e6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53e0106f10 0x7f53e019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.780+0000 7f53e6a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f53e0111fc0 con 0x7f53e0106f10 2024-09-10T08:18:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.780+0000 7f53e6a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53e0111e40 con 0x7f53e0101de0 2024-09-10T08:18:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.780+0000 7f53e6a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53e0112140 con 0x7f53e0102790 2024-09-10T08:18:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.781+0000 7f53d7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e0102790 0x7f53e019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.781+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0101de0 0x7f53e019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.781+0000 7f53d7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e0102790 0x7f53e019ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55324/0 (socket says 172.21.15.2:55324) 2024-09-10T08:18:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.781+0000 7f53e4a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53e0106f10 0x7f53e019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.781+0000 7f53d7400640 1 -- 172.21.15.2:0/518564258 learned_addr learned my addr 172.21.15.2:0/518564258 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.781+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0101de0 0x7f53e019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52750/0 (socket says 172.21.15.2:52750) 2024-09-10T08:18:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.781+0000 7f53d7400640 1 -- 172.21.15.2:0/518564258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0101de0 msgr2=0x7f53e019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.781+0000 7f53d7400640 1 --2- 172.21.15.2:0/518564258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0101de0 0x7f53e019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.781+0000 7f53d7400640 1 -- 172.21.15.2:0/518564258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53e0106f10 msgr2=0x7f53e019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:59.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.781+0000 7f53d7400640 1 --2- 172.21.15.2:0/518564258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53e0106f10 0x7f53e019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.781+0000 7f53d7400640 1 -- 172.21.15.2:0/518564258 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e019f690 con 0x7f53e0102790 2024-09-10T08:18:59.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.782+0000 7f53d7400640 1 --2- 172.21.15.2:0/518564258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e0102790 0x7f53e019ac20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f53c8004770 tx=0x7f53c8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:59.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.782+0000 7f53dd600640 1 -- 172.21.15.2:0/518564258 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c803f070 con 0x7f53e0102790 2024-09-10T08:18:59.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.782+0000 7f53e6a00640 1 -- 172.21.15.2:0/518564258 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e019f920 con 0x7f53e0102790 2024-09-10T08:18:59.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.782+0000 7f53dd600640 1 -- 172.21.15.2:0/518564258 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53c8031ca0 con 0x7f53e0102790 2024-09-10T08:18:59.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.782+0000 7f53dd600640 1 -- 172.21.15.2:0/518564258 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8035030 con 0x7f53e0102790 2024-09-10T08:18:59.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.782+0000 7f53e6a00640 1 -- 172.21.15.2:0/518564258 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e01a7260 con 0x7f53e0102790 2024-09-10T08:18:59.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.785+0000 7f53e945b740 1 Processor -- start 2024-09-10T08:18:59.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.785+0000 7f53e945b740 1 -- start start 2024-09-10T08:18:59.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.786+0000 7f53dd600640 1 -- 172.21.15.2:0/518564258 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f53c8042070 con 0x7f53e0102790 2024-09-10T08:18:59.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.786+0000 7f53e945b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c557ab0 0x558d0c557e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.786+0000 7f53e945b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c56b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.786+0000 7f53e945b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c56b580 0x558d0c56d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.786+0000 7f53e945b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558d0c4be3e0 con 0x558d0c56b580 2024-09-10T08:18:59.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.786+0000 7f53e945b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558d0c465d80 con 0x558d0c5583d0 2024-09-10T08:18:59.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.786+0000 7f53e945b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558d0c4ba3a0 con 0x558d0c557ab0 2024-09-10T08:18:59.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.786+0000 7f53d7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c56b580 0x558d0c56d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.786+0000 7f53e4a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c557ab0 0x558d0c557e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.786+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c56b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.786+0000 7f53e4a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c557ab0 0x558d0c557e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55340/0 (socket says 172.21.15.2:55340) 2024-09-10T08:18:59.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.786+0000 7f53e4a00640 1 -- 172.21.15.2:0/1003906766 learned_addr learned my addr 172.21.15.2:0/1003906766 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:59.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.787+0000 7f53e4a00640 1 -- 172.21.15.2:0/1003906766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 msgr2=0x558d0c56b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:59.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.787+0000 7f53e4a00640 1 --2- 172.21.15.2:0/1003906766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c56b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.787+0000 7f53e4a00640 1 -- 172.21.15.2:0/1003906766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c56b580 msgr2=0x558d0c56d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:59.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.787+0000 7f53e4a00640 1 --2- 172.21.15.2:0/1003906766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c56b580 0x558d0c56d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.787+0000 7f53e4a00640 1 -- 172.21.15.2:0/1003906766 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d0c484ae0 con 0x558d0c557ab0 2024-09-10T08:18:59.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.787+0000 7f53dfe00640 1 --2- 172.21.15.2:0/1003906766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c56b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:59.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.787+0000 7f53e4a00640 1 --2- 172.21.15.2:0/1003906766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c557ab0 0x558d0c557e90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009f50 tx=0x7f53d80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=500054d7cb846719 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:59.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.787+0000 7f53dd600640 1 --2- 172.21.15.2:0/518564258 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f53c40a2b80 0x7f53c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.787+0000 7f53d5200640 1 -- 172.21.15.2:0/1003906766 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8004b20 con 0x558d0c557ab0 2024-09-10T08:18:59.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.787+0000 7f53dd600640 1 -- 172.21.15.2:0/518564258 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f53c80efa10 con 0x7f53e0102790 2024-09-10T08:18:59.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.788+0000 7f53dfe00640 1 --2- 172.21.15.2:0/518564258 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f53c40a2b80 0x7f53c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.788+0000 7f53d5200640 1 -- 172.21.15.2:0/1003906766 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d8004cc0 con 0x558d0c557ab0 2024-09-10T08:18:59.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.788+0000 7f53d5200640 1 -- 172.21.15.2:0/1003906766 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8005020 con 0x558d0c557ab0 2024-09-10T08:18:59.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.788+0000 7f53e945b740 1 -- 172.21.15.2:0/1003906766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c557ab0 msgr2=0x558d0c557e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:59.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.788+0000 7f53e945b740 1 --2- 172.21.15.2:0/1003906766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c557ab0 0x558d0c557e90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009f50 tx=0x7f53d80118d0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.788+0000 7f53dfe00640 1 --2- 172.21.15.2:0/518564258 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f53c40a2b80 0x7f53c40a5040 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009f70 tx=0x7f53d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:59.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.788+0000 7f53e945b740 1 -- 172.21.15.2:0/1003906766 shutdown_connections 2024-09-10T08:18:59.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.788+0000 7f53e945b740 1 --2- 172.21.15.2:0/1003906766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c56b580 0x558d0c56d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.788+0000 7f53e945b740 1 --2- 172.21.15.2:0/1003906766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c56b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.788+0000 7f53e945b740 1 --2- 172.21.15.2:0/1003906766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c557ab0 0x558d0c557e90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.788+0000 7f53e945b740 1 -- 172.21.15.2:0/1003906766 >> 172.21.15.2:0/1003906766 conn(0x558d0c554120 msgr2=0x558d0c554500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:59.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.789+0000 7f53e945b740 1 -- 172.21.15.2:0/1003906766 shutdown_connections 2024-09-10T08:18:59.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.789+0000 7f53e945b740 1 -- 172.21.15.2:0/1003906766 wait complete. 2024-09-10T08:18:59.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.789+0000 7f53e945b740 1 -- start start 2024-09-10T08:18:59.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.789+0000 7f53e945b740 1 Processor -- start 2024-09-10T08:18:59.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.790+0000 7f53e945b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c557ab0 0x558d0c568520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.790+0000 7f53e945b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c5583d0 0x558d0c568a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.790+0000 7f53e945b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c56b580 0x558d0c568fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.790+0000 7f53e945b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558d0c56e190 con 0x558d0c5583d0 2024-09-10T08:18:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.790+0000 7f53e945b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558d0c4790f0 con 0x558d0c557ab0 2024-09-10T08:18:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.790+0000 7f53e945b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558d0c56e310 con 0x558d0c56b580 2024-09-10T08:18:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.790+0000 7f53e945b740 10 client.0 _ll_register_callbacks cb 0x558d0c46a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.790+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c557ab0 0x558d0c568520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.790+0000 7f53d7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c5583d0 0x558d0c568a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.790+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c557ab0 0x558d0c568520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52774/0 (socket says 172.21.15.2:52774) 2024-09-10T08:18:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.790+0000 7f53dfe00640 1 -- 172.21.15.2:0/2458013192 learned_addr learned my addr 172.21.15.2:0/2458013192 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.790+0000 7f53e4a00640 1 --2- 172.21.15.2:0/2458013192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c56b580 0x558d0c568fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.791+0000 7f53dfe00640 1 -- 172.21.15.2:0/2458013192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c56b580 msgr2=0x558d0c568fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.791+0000 7f53dfe00640 1 --2- 172.21.15.2:0/2458013192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c56b580 0x558d0c568fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.791+0000 7f53dfe00640 1 -- 172.21.15.2:0/2458013192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c5583d0 msgr2=0x558d0c568a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.791+0000 7f53dfe00640 1 --2- 172.21.15.2:0/2458013192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c5583d0 0x558d0c568a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.791+0000 7f53dfe00640 1 -- 172.21.15.2:0/2458013192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d0c22f3c0 con 0x558d0c557ab0 2024-09-10T08:18:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.791+0000 7f53d7400640 1 --2- 172.21.15.2:0/2458013192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c5583d0 0x558d0c568a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.791+0000 7f53dfe00640 1 --2- 172.21.15.2:0/2458013192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c557ab0 0x558d0c568520 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f53d001ea00 tx=0x7f53d0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1c36dc4e3b0264 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.791+0000 7f53cd600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:18:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.791+0000 7f53cd600640 1 -- 172.21.15.2:0/2458013192 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d002e060 con 0x558d0c557ab0 2024-09-10T08:18:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.791+0000 7f53e945b740 1 -- 172.21.15.2:0/2458013192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d8004cc0 con 0x558d0c557ab0 2024-09-10T08:18:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.791+0000 7f53cd600640 1 -- 172.21.15.2:0/2458013192 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f53d0018030 con 0x558d0c557ab0 2024-09-10T08:18:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.792+0000 7f53cd600640 1 -- 172.21.15.2:0/2458013192 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53d0035050 con 0x558d0c557ab0 2024-09-10T08:18:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.792+0000 7f53e945b740 10 client.28424 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.792+0000 7f53e945b740 1 -- 172.21.15.2:0/2458013192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558d0c579fd0 con 0x558d0c557ab0 2024-09-10T08:18:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.792+0000 7f53cd600640 1 -- 172.21.15.2:0/2458013192 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d8004cc0 con 0x558d0c557ab0 2024-09-10T08:18:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.792+0000 7f53cd600640 1 -- 172.21.15.2:0/2458013192 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f53d001c510 con 0x558d0c557ab0 2024-09-10T08:18:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.792+0000 7f53e945b740 10 client.28424 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.792+0000 7f53e945b740 10 client.28424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.792+0000 7f53e945b740 20 client.28424 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.792+0000 7f53e945b740 1 --2- 172.21.15.2:0/2458013192 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558d0c57b3e0 0x558d0c57d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.792+0000 7f53e945b740 4 client.28424 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:18:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.792+0000 7f53e945b740 1 -- 172.21.15.2:0/2458013192 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d0c4cb950 con 0x558d0c57b3e0 2024-09-10T08:18:59.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.793+0000 7f53d7400640 1 --2- 172.21.15.2:0/2458013192 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558d0c57b3e0 0x558d0c57d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.793+0000 7f53d7400640 1 --2- 172.21.15.2:0/2458013192 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558d0c57b3e0 0x558d0c57d830 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e79344f94cfd91a server_cookie=e19526e788096ef2 in_seq=0 out_seq=0 2024-09-10T08:18:59.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.793+0000 7f53cd600640 10 client.28424 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:59.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53cd600640 1 -- 172.21.15.2:0/2458013192 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d0c4cb950 con 0x558d0c57b3e0 2024-09-10T08:18:59.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53cd600640 10 client.28424 handle_command_reply: tid=0 2024-09-10T08:18:59.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53e945b740 1 client.28424 shutdown 2024-09-10T08:18:59.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53e945b740 1 -- 172.21.15.2:0/2458013192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c557ab0 msgr2=0x558d0c568520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:59.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53e945b740 1 --2- 172.21.15.2:0/2458013192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c557ab0 0x558d0c568520 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f53d001ea00 tx=0x7f53d0004730 comp rx=0 tx=0).stop 2024-09-10T08:18:59.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53e945b740 1 -- 172.21.15.2:0/2458013192 shutdown_connections 2024-09-10T08:18:59.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53e945b740 1 --2- 172.21.15.2:0/2458013192 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558d0c57b3e0 0x558d0c57d830 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53e945b740 1 --2- 172.21.15.2:0/2458013192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c56b580 0x558d0c568fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53e945b740 1 --2- 172.21.15.2:0/2458013192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c5583d0 0x558d0c568a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53cd600640 0 client.28424 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:59.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53e945b740 1 --2- 172.21.15.2:0/2458013192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c557ab0 0x558d0c568520 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53e945b740 1 -- 172.21.15.2:0/2458013192 >> 172.21.15.2:0/2458013192 conn(0x558d0c554120 msgr2=0x558d0c559db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:59.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53e945b740 1 -- 172.21.15.2:0/2458013192 shutdown_connections 2024-09-10T08:18:59.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.794+0000 7f53e945b740 1 -- 172.21.15.2:0/2458013192 wait complete. 2024-09-10T08:18:59.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.795+0000 7f53e945b740 20 client.28424 trim_cache size 0 max 16384 2024-09-10T08:18:59.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.812+0000 7f53e945b740 1 Processor -- start 2024-09-10T08:18:59.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.813+0000 7f53e945b740 1 -- start start 2024-09-10T08:18:59.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.813+0000 7f53e945b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c557ab0 0x558d0c583100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.813+0000 7f53e945b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c583640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.813+0000 7f53e945b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c56b580 0x558d0c565780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.813+0000 7f53e945b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f53e01af540 con 0x558d0c56b580 2024-09-10T08:18:59.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.813+0000 7f53e945b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53e01ace10 con 0x558d0c5583d0 2024-09-10T08:18:59.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.813+0000 7f53e945b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53e01aacc0 con 0x558d0c557ab0 2024-09-10T08:18:59.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.813+0000 7f53e4a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c56b580 0x558d0c565780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.813+0000 7f53e4a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c56b580 0x558d0c565780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52748/0 (socket says 172.21.15.2:52748) 2024-09-10T08:18:59.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.813+0000 7f53e4a00640 1 -- 172.21.15.2:0/2984124869 learned_addr learned my addr 172.21.15.2:0/2984124869 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:59.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.813+0000 7f53d7400640 1 --2- 172.21.15.2:0/2984124869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c583640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.813+0000 7f53dfe00640 1 --2- 172.21.15.2:0/2984124869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c557ab0 0x558d0c583100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.814+0000 7f53dfe00640 1 -- 172.21.15.2:0/2984124869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 msgr2=0x558d0c583640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:59.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.814+0000 7f53dfe00640 1 --2- 172.21.15.2:0/2984124869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c583640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.814+0000 7f53dfe00640 1 -- 172.21.15.2:0/2984124869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c56b580 msgr2=0x558d0c565780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.814+0000 7f53dfe00640 1 --2- 172.21.15.2:0/2984124869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c56b580 0x558d0c565780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.814+0000 7f53dfe00640 1 -- 172.21.15.2:0/2984124869 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d0c557380 con 0x558d0c557ab0 2024-09-10T08:18:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.814+0000 7f53dfe00640 1 --2- 172.21.15.2:0/2984124869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c557ab0 0x558d0c583100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f53d001e9d0 tx=0x7f53d0012d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d4009cc19878ece server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.814+0000 7f53cfe00640 1 -- 172.21.15.2:0/2984124869 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d003e070 con 0x558d0c557ab0 2024-09-10T08:18:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.814+0000 7f53cfe00640 1 -- 172.21.15.2:0/2984124869 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d001cc50 con 0x558d0c557ab0 2024-09-10T08:18:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.814+0000 7f53cfe00640 1 -- 172.21.15.2:0/2984124869 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d00039b0 con 0x558d0c557ab0 2024-09-10T08:18:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.815+0000 7f53e945b740 1 -- 172.21.15.2:0/2984124869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c557ab0 msgr2=0x558d0c583100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.815+0000 7f53e945b740 1 --2- 172.21.15.2:0/2984124869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c557ab0 0x558d0c583100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f53d001e9d0 tx=0x7f53d0012d30 comp rx=0 tx=0).stop 2024-09-10T08:18:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.815+0000 7f53e945b740 1 -- 172.21.15.2:0/2984124869 shutdown_connections 2024-09-10T08:18:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.815+0000 7f53e945b740 1 --2- 172.21.15.2:0/2984124869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c56b580 0x558d0c565780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.815+0000 7f53e945b740 1 --2- 172.21.15.2:0/2984124869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c583640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.815+0000 7f53e945b740 1 --2- 172.21.15.2:0/2984124869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c557ab0 0x558d0c583100 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.815+0000 7f53e945b740 1 -- 172.21.15.2:0/2984124869 >> 172.21.15.2:0/2984124869 conn(0x558d0c554120 msgr2=0x558d0c5531e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:59.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.815+0000 7f53e945b740 1 -- 172.21.15.2:0/2984124869 shutdown_connections 2024-09-10T08:18:59.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.815+0000 7f53e945b740 1 -- 172.21.15.2:0/2984124869 wait complete. 2024-09-10T08:18:59.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.816+0000 7f53e945b740 1 -- start start 2024-09-10T08:18:59.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.816+0000 7f53e945b740 1 Processor -- start 2024-09-10T08:18:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.816+0000 7f53e945b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c557ab0 0x558d0c56f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.816+0000 7f53e945b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c56f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.816+0000 7f53e945b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c56b580 0x558d0c56fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.816+0000 7f53e945b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f53e01ab310 con 0x558d0c557ab0 2024-09-10T08:18:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.816+0000 7f53e945b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53e01ac740 con 0x558d0c5583d0 2024-09-10T08:18:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.816+0000 7f53e945b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53e01ab9a0 con 0x558d0c56b580 2024-09-10T08:18:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.816+0000 7f53e945b740 10 client.0 _ll_register_callbacks cb 0x7f53d001e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:18:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53d7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c56f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c557ab0 0x558d0c56f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53d7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c56f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52790/0 (socket says 172.21.15.2:52790) 2024-09-10T08:18:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53d7400640 1 -- 172.21.15.2:0/2289461842 learned_addr learned my addr 172.21.15.2:0/2289461842 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:18:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53e4a00640 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c56b580 0x558d0c56fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53d7400640 1 -- 172.21.15.2:0/2289461842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c56b580 msgr2=0x558d0c56fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53d7400640 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c56b580 0x558d0c56fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53d7400640 1 -- 172.21.15.2:0/2289461842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c557ab0 msgr2=0x558d0c56f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:18:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53d7400640 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c557ab0 0x558d0c56f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53d7400640 1 -- 172.21.15.2:0/2289461842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d0c5e51a0 con 0x558d0c5583d0 2024-09-10T08:18:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53e4a00640 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c56b580 0x558d0c56fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:18:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53dfe00640 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c557ab0 0x558d0c56f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:18:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53d7400640 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c56f900 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f53c80bbcf0 tx=0x7f53c802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91fc69e65d2d054 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:18:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53cd600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:18:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.817+0000 7f53cd600640 1 -- 172.21.15.2:0/2289461842 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8049040 con 0x558d0c5583d0 2024-09-10T08:18:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.818+0000 7f53cd600640 1 -- 172.21.15.2:0/2289461842 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f53c80ba2f0 con 0x558d0c5583d0 2024-09-10T08:18:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.818+0000 7f53e945b740 1 -- 172.21.15.2:0/2289461842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d001cc50 con 0x558d0c5583d0 2024-09-10T08:18:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.818+0000 7f53cd600640 1 -- 172.21.15.2:0/2289461842 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53c80bbff0 con 0x558d0c5583d0 2024-09-10T08:18:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.818+0000 7f53e945b740 10 client.28430 fetch_fsmap learned FSMap version 26 2024-09-10T08:18:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.818+0000 7f53e945b740 1 -- 172.21.15.2:0/2289461842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558d0c5e64f0 con 0x558d0c5583d0 2024-09-10T08:18:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.818+0000 7f53cd600640 1 -- 172.21.15.2:0/2289461842 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d001cc50 con 0x558d0c5583d0 2024-09-10T08:18:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.818+0000 7f53cd600640 1 -- 172.21.15.2:0/2289461842 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f53c803b4f0 con 0x558d0c5583d0 2024-09-10T08:18:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.818+0000 7f53e945b740 10 client.28430 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:18:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.818+0000 7f53e945b740 10 client.28430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:18:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.818+0000 7f53e945b740 20 client.28430 populate_metadata read hostname 'smithi002' 2024-09-10T08:18:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.818+0000 7f53e945b740 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558d0c5e6c50 0x558d0c5e9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:18:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.818+0000 7f53e945b740 4 client.28430 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:18:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.818+0000 7f53e945b740 1 -- 172.21.15.2:0/2289461842 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d0c4c5e60 con 0x558d0c5e6c50 2024-09-10T08:18:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.819+0000 7f53dfe00640 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558d0c5e6c50 0x558d0c5e9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:18:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.819+0000 7f53dfe00640 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558d0c5e6c50 0x558d0c5e9070 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361f3a1e1fc4e17c server_cookie=8b73cbf010d71df6 in_seq=0 out_seq=0 2024-09-10T08:18:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.819+0000 7f53cd600640 10 client.28430 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.819+0000 7f53cd600640 1 -- 172.21.15.2:0/2289461842 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558d0c4c5e60 con 0x558d0c5e6c50 2024-09-10T08:18:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.819+0000 7f53cd600640 10 client.28430 handle_command_reply: tid=0 2024-09-10T08:18:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.819+0000 7f53e945b740 1 client.28430 shutdown 2024-09-10T08:18:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.820+0000 7f53e945b740 1 -- 172.21.15.2:0/2289461842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 msgr2=0x558d0c56f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.820+0000 7f53e945b740 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c56f900 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f53c80bbcf0 tx=0x7f53c802a410 comp rx=0 tx=0).stop 2024-09-10T08:18:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.820+0000 7f53e945b740 1 -- 172.21.15.2:0/2289461842 shutdown_connections 2024-09-10T08:18:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.820+0000 7f53e945b740 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558d0c5e6c50 0x558d0c5e9070 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.820+0000 7f53e945b740 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d0c56b580 0x558d0c56fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.820+0000 7f53e945b740 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d0c5583d0 0x558d0c56f900 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.820+0000 7f53e945b740 1 --2- 172.21.15.2:0/2289461842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558d0c557ab0 0x558d0c56f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.820+0000 7f53e945b740 1 -- 172.21.15.2:0/2289461842 >> 172.21.15.2:0/2289461842 conn(0x558d0c554120 msgr2=0x558d0c569d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.820+0000 7f53cd600640 0 client.28430 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:18:59.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.820+0000 7f53e945b740 1 -- 172.21.15.2:0/2289461842 shutdown_connections 2024-09-10T08:18:59.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.820+0000 7f53e945b740 1 -- 172.21.15.2:0/2289461842 wait complete. 2024-09-10T08:18:59.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.820+0000 7f53e945b740 20 client.28430 trim_cache size 0 max 16384 2024-09-10T08:18:59.824 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:18:59.824 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:18:59.824 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:18:59.824 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:18:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.821+0000 7f53e6a00640 1 -- 172.21.15.2:0/518564258 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f53c40a2b80 msgr2=0x7f53c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.821+0000 7f53e6a00640 1 --2- 172.21.15.2:0/518564258 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f53c40a2b80 0x7f53c40a5040 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009f70 tx=0x7f53d0005f90 comp rx=0 tx=0).stop 2024-09-10T08:18:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.821+0000 7f53e6a00640 1 -- 172.21.15.2:0/518564258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e0102790 msgr2=0x7f53e019ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:18:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.821+0000 7f53e6a00640 1 --2- 172.21.15.2:0/518564258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e0102790 0x7f53e019ac20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f53c8004770 tx=0x7f53c8003c10 comp rx=0 tx=0).stop 2024-09-10T08:18:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.821+0000 7f53e6a00640 1 -- 172.21.15.2:0/518564258 shutdown_connections 2024-09-10T08:18:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.821+0000 7f53e6a00640 1 --2- 172.21.15.2:0/518564258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53e0106f10 0x7f53e019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.821+0000 7f53e6a00640 1 --2- 172.21.15.2:0/518564258 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f53c40a2b80 0x7f53c40a5040 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.821+0000 7f53e6a00640 1 --2- 172.21.15.2:0/518564258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e0102790 0x7f53e019ac20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.821+0000 7f53e6a00640 1 --2- 172.21.15.2:0/518564258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0101de0 0x7f53e019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:18:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.821+0000 7f53e6a00640 1 -- 172.21.15.2:0/518564258 >> 172.21.15.2:0/518564258 conn(0x7f53e00fd8c0 msgr2=0x7f53e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:18:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.821+0000 7f53e6a00640 1 -- 172.21.15.2:0/518564258 shutdown_connections 2024-09-10T08:18:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:18:59.821+0000 7f53e6a00640 1 -- 172.21.15.2:0/518564258 wait complete. 2024-09-10T08:18:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:18:59 smithi155 ceph-mon[35405]: pgmap v712: 129 pgs: 129 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-10T08:18:59.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:18:59 smithi107 ceph-mon[35048]: pgmap v712: 129 pgs: 129 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-10T08:19:00.621 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:19:00.621 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:19:01.001 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:01.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.586+0000 7fb5c7c00640 1 -- 172.21.15.2:0/3109303641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0104010 msgr2=0x7fb5c01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:01.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.586+0000 7fb5c7c00640 1 --2- 172.21.15.2:0/3109303641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0104010 0x7fb5c01043f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009900 tx=0x7fb5b002a410 comp rx=0 tx=0).stop 2024-09-10T08:19:01.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.586+0000 7fb5c7c00640 1 -- 172.21.15.2:0/3109303641 shutdown_connections 2024-09-10T08:19:01.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.586+0000 7fb5c7c00640 1 --2- 172.21.15.2:0/3109303641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5c0109140 0x7fb5c0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:01.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.586+0000 7fb5c7c00640 1 --2- 172.21.15.2:0/3109303641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c01049c0 0x7fb5c0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:01.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.586+0000 7fb5c7c00640 1 --2- 172.21.15.2:0/3109303641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0104010 0x7fb5c01043f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:01.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.586+0000 7fb5c7c00640 1 -- 172.21.15.2:0/3109303641 >> 172.21.15.2:0/3109303641 conn(0x7fb5c00ffa50 msgr2=0x7fb5c0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:01.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.586+0000 7fb5c7c00640 1 -- 172.21.15.2:0/3109303641 shutdown_connections 2024-09-10T08:19:01.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.586+0000 7fb5c7c00640 1 -- 172.21.15.2:0/3109303641 wait complete. 2024-09-10T08:19:01.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.587+0000 7fb5c7c00640 1 Processor -- start 2024-09-10T08:19:01.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.587+0000 7fb5c7c00640 1 -- start start 2024-09-10T08:19:01.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.587+0000 7fb5c7c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c0104010 0x7fb5c019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:01.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.587+0000 7fb5c7c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5c01049c0 0x7fb5c019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:01.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.587+0000 7fb5c7c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0109140 0x7fb5c019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:01.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.587+0000 7fb5c7c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c0114140 con 0x7fb5c0109140 2024-09-10T08:19:01.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.587+0000 7fb5c7c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c0113fc0 con 0x7fb5c0104010 2024-09-10T08:19:01.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.587+0000 7fb5c7c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c01142c0 con 0x7fb5c01049c0 2024-09-10T08:19:01.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.588+0000 7fb5c5200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c0104010 0x7fb5c019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:01.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.588+0000 7fb5c5200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c0104010 0x7fb5c019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52806/0 (socket says 172.21.15.2:52806) 2024-09-10T08:19:01.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.588+0000 7fb5c5200640 1 -- 172.21.15.2:0/2942717834 learned_addr learned my addr 172.21.15.2:0/2942717834 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:01.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.588+0000 7fb5c5c00640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0109140 0x7fb5c019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:01.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.588+0000 7fb5b7e00640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5c01049c0 0x7fb5c019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:01.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.588+0000 7fb5c5200640 1 -- 172.21.15.2:0/2942717834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5c01049c0 msgr2=0x7fb5c019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:01.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.588+0000 7fb5c5200640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5c01049c0 0x7fb5c019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:01.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.588+0000 7fb5c5200640 1 -- 172.21.15.2:0/2942717834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0109140 msgr2=0x7fb5c019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:01.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.588+0000 7fb5c5200640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0109140 0x7fb5c019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:01.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.588+0000 7fb5c5200640 1 -- 172.21.15.2:0/2942717834 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c019f5d0 con 0x7fb5c0104010 2024-09-10T08:19:01.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.588+0000 7fb5c5200640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c0104010 0x7fb5c019a620 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0006010 tx=0x7fb5b00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:01.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.589+0000 7fb5be000640 1 -- 172.21.15.2:0/2942717834 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b003f030 con 0x7fb5c0104010 2024-09-10T08:19:01.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.589+0000 7fb5c7c00640 1 -- 172.21.15.2:0/2942717834 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c019f860 con 0x7fb5c0104010 2024-09-10T08:19:01.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.589+0000 7fb5c7c00640 1 -- 172.21.15.2:0/2942717834 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c01a71a0 con 0x7fb5c0104010 2024-09-10T08:19:01.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.589+0000 7fb5be000640 1 -- 172.21.15.2:0/2942717834 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b0003ce0 con 0x7fb5c0104010 2024-09-10T08:19:01.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.589+0000 7fb5be000640 1 -- 172.21.15.2:0/2942717834 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0049060 con 0x7fb5c0104010 2024-09-10T08:19:01.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.590+0000 7fb5c7c00640 1 -- 172.21.15.2:0/2942717834 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb59c005290 con 0x7fb5c0104010 2024-09-10T08:19:01.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.592+0000 7fb5be000640 1 -- 172.21.15.2:0/2942717834 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb5b0042030 con 0x7fb5c0104010 2024-09-10T08:19:01.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.593+0000 7fb5be000640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb5a40a2c00 0x7fb5a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:01.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.594+0000 7fb5be000640 1 -- 172.21.15.2:0/2942717834 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb5b00ef9f0 con 0x7fb5c0104010 2024-09-10T08:19:01.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.594+0000 7fb5b7e00640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb5a40a2c00 0x7fb5a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:01.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.594+0000 7fb5b7e00640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb5a40a2c00 0x7fb5a40a50c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00fff70 tx=0x7fb5a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:01.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.596+0000 7fb5be000640 1 -- 172.21.15.2:0/2942717834 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5b003b460 con 0x7fb5c0104010 2024-09-10T08:19:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:01 smithi002 ceph-mon[30563]: pgmap v713: 129 pgs: 129 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:19:01.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.819+0000 7fb5c7c00640 1 -- 172.21.15.2:0/2942717834 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb59c005580 con 0x7fb5c0104010 2024-09-10T08:19:01.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.820+0000 7fb5be000640 1 -- 172.21.15.2:0/2942717834 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fb5b00bbf70 con 0x7fb5c0104010 2024-09-10T08:19:01.821 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:19:01.821 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:19:01.821 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:19:01.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.822+0000 7fb5c7c00640 1 -- 172.21.15.2:0/2942717834 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb5a40a2c00 msgr2=0x7fb5a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:01.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.822+0000 7fb5c7c00640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb5a40a2c00 0x7fb5a40a50c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00fff70 tx=0x7fb5a8005f90 comp rx=0 tx=0).stop 2024-09-10T08:19:01.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.822+0000 7fb5c7c00640 1 -- 172.21.15.2:0/2942717834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c0104010 msgr2=0x7fb5c019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:01.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.822+0000 7fb5c7c00640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c0104010 0x7fb5c019a620 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0006010 tx=0x7fb5b00042f0 comp rx=0 tx=0).stop 2024-09-10T08:19:01.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.823+0000 7fb5c7c00640 1 -- 172.21.15.2:0/2942717834 shutdown_connections 2024-09-10T08:19:01.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.823+0000 7fb5c7c00640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0109140 0x7fb5c019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:01.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.823+0000 7fb5c7c00640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb5a40a2c00 0x7fb5a40a50c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:01.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.823+0000 7fb5c7c00640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5c01049c0 0x7fb5c019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:01.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.823+0000 7fb5c7c00640 1 --2- 172.21.15.2:0/2942717834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c0104010 0x7fb5c019a620 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:01.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.823+0000 7fb5c7c00640 1 -- 172.21.15.2:0/2942717834 >> 172.21.15.2:0/2942717834 conn(0x7fb5c00ffa50 msgr2=0x7fb5c0100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:01.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.823+0000 7fb5c7c00640 1 -- 172.21.15.2:0/2942717834 shutdown_connections 2024-09-10T08:19:01.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:01.823+0000 7fb5c7c00640 1 -- 172.21.15.2:0/2942717834 wait complete. 2024-09-10T08:19:01.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:01 smithi155 ceph-mon[35405]: pgmap v713: 129 pgs: 129 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:19:01.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:01 smithi107 ceph-mon[35048]: pgmap v713: 129 pgs: 129 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:19:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.458+0000 7f1da4a00640 1 -- 172.21.15.2:0/1309380519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da0074bf0 msgr2=0x7f1da0075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.458+0000 7f1da4a00640 1 --2- 172.21.15.2:0/1309380519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da0074bf0 0x7f1da0075050 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009900 tx=0x7f1d8802a480 comp rx=0 tx=0).stop 2024-09-10T08:19:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.458+0000 7f1da4a00640 1 -- 172.21.15.2:0/1309380519 shutdown_connections 2024-09-10T08:19:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.458+0000 7f1da4a00640 1 --2- 172.21.15.2:0/1309380519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1da01099f0 0x7f1da010beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.458+0000 7f1da4a00640 1 --2- 172.21.15.2:0/1309380519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da0074bf0 0x7f1da0075050 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.458+0000 7f1da4a00640 1 --2- 172.21.15.2:0/1309380519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da00761d0 0x7f1da0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.458+0000 7f1da4a00640 1 -- 172.21.15.2:0/1309380519 >> 172.21.15.2:0/1309380519 conn(0x7f1da00fd2e0 msgr2=0x7f1da00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.459+0000 7f1da4a00640 1 -- 172.21.15.2:0/1309380519 shutdown_connections 2024-09-10T08:19:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.459+0000 7f1da4a00640 1 -- 172.21.15.2:0/1309380519 wait complete. 2024-09-10T08:19:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.459+0000 7f1da4a00640 1 Processor -- start 2024-09-10T08:19:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.460+0000 7f1da4a00640 1 -- start start 2024-09-10T08:19:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.460+0000 7f1da4a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1da0074bf0 0x7f1da019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.460+0000 7f1da4a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da00761d0 0x7f1da019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.460+0000 7f1da4a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da01099f0 0x7f1da01a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.460+0000 7f1da4a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010fa40 con 0x7f1da01099f0 2024-09-10T08:19:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.460+0000 7f1da4a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010f8c0 con 0x7f1da0074bf0 2024-09-10T08:19:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.460+0000 7f1da4a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010fbc0 con 0x7f1da00761d0 2024-09-10T08:19:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.460+0000 7f1d9e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da01099f0 0x7f1da01a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.460+0000 7f1d9e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da01099f0 0x7f1da01a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52792/0 (socket says 172.21.15.2:52792) 2024-09-10T08:19:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.460+0000 7f1d9e800640 1 -- 172.21.15.2:0/2079843809 learned_addr learned my addr 172.21.15.2:0/2079843809 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.460+0000 7f1d97e00640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da00761d0 0x7f1da019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.460+0000 7f1d9de00640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1da0074bf0 0x7f1da019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.461+0000 7f1d9e800640 1 -- 172.21.15.2:0/2079843809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da00761d0 msgr2=0x7f1da019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.461+0000 7f1d9e800640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da00761d0 0x7f1da019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.461+0000 7f1d9e800640 1 -- 172.21.15.2:0/2079843809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1da0074bf0 msgr2=0x7f1da019a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.461+0000 7f1d9e800640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1da0074bf0 0x7f1da019a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.461+0000 7f1d9e800640 1 -- 172.21.15.2:0/2079843809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da01a1c50 con 0x7f1da01099f0 2024-09-10T08:19:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.461+0000 7f1d97e00640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da00761d0 0x7f1da019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.461+0000 7f1d9e800640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da01099f0 0x7f1da01a15e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009fd0 tx=0x7f1d9800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.461+0000 7f1d96a00640 1 -- 172.21.15.2:0/2079843809 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98017070 con 0x7f1da01099f0 2024-09-10T08:19:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.461+0000 7f1d96a00640 1 -- 172.21.15.2:0/2079843809 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d98003e50 con 0x7f1da01099f0 2024-09-10T08:19:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.461+0000 7f1da4a00640 1 -- 172.21.15.2:0/2079843809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da01a1f40 con 0x7f1da01099f0 2024-09-10T08:19:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.461+0000 7f1d96a00640 1 -- 172.21.15.2:0/2079843809 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d980106c0 con 0x7f1da01099f0 2024-09-10T08:19:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.462+0000 7f1da4a00640 1 -- 172.21.15.2:0/2079843809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da01a2400 con 0x7f1da01099f0 2024-09-10T08:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.464+0000 7f1da7823740 1 Processor -- start 2024-09-10T08:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.465+0000 7f1d96a00640 1 -- 172.21.15.2:0/2079843809 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1d98003970 con 0x7f1da01099f0 2024-09-10T08:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.465+0000 7f1da7823740 1 -- start start 2024-09-10T08:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.465+0000 7f1da7823740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f8537090 0x55d1f8537490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.465+0000 7f1da7823740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f85379d0 0x55d1f853c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.465+0000 7f1da7823740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f853ca60 0x55d1f853ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.465+0000 7f1da7823740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d1f849c100 con 0x55d1f853ca60 2024-09-10T08:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.465+0000 7f1da7823740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d1f84441b0 con 0x55d1f85379d0 2024-09-10T08:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.465+0000 7f1da7823740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1f84980c0 con 0x55d1f8537090 2024-09-10T08:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.465+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f85379d0 0x55d1f853c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f8537090 0x55d1f8537490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f85379d0 0x55d1f853c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52840/0 (socket says 172.21.15.2:52840) 2024-09-10T08:19:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d9de00640 1 -- 172.21.15.2:0/3432870940 learned_addr learned my addr 172.21.15.2:0/3432870940 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d97e00640 1 --2- 172.21.15.2:0/3432870940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f853ca60 0x55d1f853ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d9de00640 1 -- 172.21.15.2:0/3432870940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f8537090 msgr2=0x55d1f8537490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d9de00640 1 --2- 172.21.15.2:0/3432870940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f8537090 0x55d1f8537490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d9de00640 1 -- 172.21.15.2:0/3432870940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f853ca60 msgr2=0x55d1f853ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d9de00640 1 --2- 172.21.15.2:0/3432870940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f853ca60 0x55d1f853ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d9de00640 1 -- 172.21.15.2:0/3432870940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f81af210 con 0x55d1f85379d0 2024-09-10T08:19:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d97e00640 1 --2- 172.21.15.2:0/3432870940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f8537090 0x55d1f8537490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:19:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d9de00640 1 --2- 172.21.15.2:0/3432870940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f85379d0 0x55d1f853c490 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009d20 tx=0x7f1d90011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=644649aef40788b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d96a00640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d840a2cb0 0x7f1d840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d96a00640 1 -- 172.21.15.2:0/2079843809 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1d980cf0f0 con 0x7f1da01099f0 2024-09-10T08:19:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.466+0000 7f1d8f000640 1 -- 172.21.15.2:0/3432870940 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9001e070 con 0x55d1f85379d0 2024-09-10T08:19:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.467+0000 7f1d8f000640 1 -- 172.21.15.2:0/3432870940 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d90019b10 con 0x55d1f85379d0 2024-09-10T08:19:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.467+0000 7f1d8f000640 1 -- 172.21.15.2:0/3432870940 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9000ebb0 con 0x55d1f85379d0 2024-09-10T08:19:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.467+0000 7f1d97e00640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d840a2cb0 0x7f1d840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.467+0000 7f1da7823740 1 -- 172.21.15.2:0/3432870940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f85379d0 msgr2=0x55d1f853c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.467+0000 7f1da7823740 1 --2- 172.21.15.2:0/3432870940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f85379d0 0x55d1f853c490 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009d20 tx=0x7f1d90011810 comp rx=0 tx=0).stop 2024-09-10T08:19:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.467+0000 7f1d97e00640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d840a2cb0 0x7f1d840a5170 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1d88031440 tx=0x7f1d880315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.467+0000 7f1da7823740 1 -- 172.21.15.2:0/3432870940 shutdown_connections 2024-09-10T08:19:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.467+0000 7f1da7823740 1 --2- 172.21.15.2:0/3432870940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f853ca60 0x55d1f853ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.467+0000 7f1da7823740 1 --2- 172.21.15.2:0/3432870940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f85379d0 0x55d1f853c490 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.467+0000 7f1da7823740 1 --2- 172.21.15.2:0/3432870940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f8537090 0x55d1f8537490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.467+0000 7f1da7823740 1 -- 172.21.15.2:0/3432870940 >> 172.21.15.2:0/3432870940 conn(0x55d1f8532700 msgr2=0x55d1f8532b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.467+0000 7f1da7823740 1 -- 172.21.15.2:0/3432870940 shutdown_connections 2024-09-10T08:19:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.468+0000 7f1da7823740 1 -- 172.21.15.2:0/3432870940 wait complete. 2024-09-10T08:19:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.468+0000 7f1da7823740 1 -- start start 2024-09-10T08:19:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.468+0000 7f1da7823740 1 Processor -- start 2024-09-10T08:19:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.468+0000 7f1da7823740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f8537090 0x55d1f8541280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.468+0000 7f1da7823740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f85379d0 0x55d1f85417c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1da7823740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f853ca60 0x55d1f8548790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1da7823740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d1f853fa30 con 0x55d1f85379d0 2024-09-10T08:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1da7823740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d1f8425090 con 0x55d1f8537090 2024-09-10T08:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1da7823740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1f853fbb0 con 0x55d1f853ca60 2024-09-10T08:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1da7823740 10 client.0 _ll_register_callbacks cb 0x55d1f8479f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f85379d0 0x55d1f85417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f8537090 0x55d1f8541280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f8537090 0x55d1f8541280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52854/0 (socket says 172.21.15.2:52854) 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1d9de00640 1 -- 172.21.15.2:0/2874593148 learned_addr learned my addr 172.21.15.2:0/2874593148 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1d9e800640 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f853ca60 0x55d1f8548790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1d97e00640 1 -- 172.21.15.2:0/2874593148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f853ca60 msgr2=0x55d1f8548790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1d97e00640 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f853ca60 0x55d1f8548790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1d97e00640 1 -- 172.21.15.2:0/2874593148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f8537090 msgr2=0x55d1f8541280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1d97e00640 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f8537090 0x55d1f8541280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1d97e00640 1 -- 172.21.15.2:0/2874593148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1f827b620 con 0x55d1f85379d0 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1d9e800640 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f853ca60 0x55d1f8548790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.469+0000 7f1d9de00640 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f8537090 0x55d1f8541280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.470+0000 7f1d97e00640 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f85379d0 0x55d1f85417c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1d8802a9a0 tx=0x7f1d88005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db94b1d810006229 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.470+0000 7f1d83400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.470+0000 7f1d83400640 1 -- 172.21.15.2:0/2874593148 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d880328c0 con 0x55d1f85379d0 2024-09-10T08:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.470+0000 7f1d83400640 1 -- 172.21.15.2:0/2874593148 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1d88032a60 con 0x55d1f85379d0 2024-09-10T08:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.470+0000 7f1da7823740 1 -- 172.21.15.2:0/2874593148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d90019b10 con 0x55d1f85379d0 2024-09-10T08:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.470+0000 7f1d83400640 1 -- 172.21.15.2:0/2874593148 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d88032c60 con 0x55d1f85379d0 2024-09-10T08:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.470+0000 7f1da7823740 10 client.18417 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.470+0000 7f1da7823740 1 -- 172.21.15.2:0/2874593148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d1f854b580 con 0x55d1f85379d0 2024-09-10T08:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.470+0000 7f1d83400640 1 -- 172.21.15.2:0/2874593148 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d90019b10 con 0x55d1f85379d0 2024-09-10T08:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.471+0000 7f1d83400640 1 -- 172.21.15.2:0/2874593148 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1d8804f070 con 0x55d1f85379d0 2024-09-10T08:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.471+0000 7f1da7823740 10 client.18417 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.471+0000 7f1da7823740 10 client.18417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.471+0000 7f1da7823740 20 client.18417 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.471+0000 7f1da7823740 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d1f854ccc0 0x55d1f854f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.471+0000 7f1da7823740 4 client.18417 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.471+0000 7f1da7823740 1 -- 172.21.15.2:0/2874593148 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1f84a9830 con 0x55d1f854ccc0 2024-09-10T08:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.471+0000 7f1d9de00640 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d1f854ccc0 0x55d1f854f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.472+0000 7f1d9de00640 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d1f854ccc0 0x55d1f854f130 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abc87875bd99afdc server_cookie=69ef03c3ee06dc9 in_seq=0 out_seq=0 2024-09-10T08:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.472+0000 7f1d83400640 10 client.18417 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1d83400640 1 -- 172.21.15.2:0/2874593148 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1f84a9830 con 0x55d1f854ccc0 2024-09-10T08:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1d83400640 10 client.18417 handle_command_reply: tid=0 2024-09-10T08:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1da7823740 1 client.18417 shutdown 2024-09-10T08:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1da7823740 1 -- 172.21.15.2:0/2874593148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f85379d0 msgr2=0x55d1f85417c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1da7823740 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f85379d0 0x55d1f85417c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1d8802a9a0 tx=0x7f1d88005e50 comp rx=0 tx=0).stop 2024-09-10T08:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1da7823740 1 -- 172.21.15.2:0/2874593148 shutdown_connections 2024-09-10T08:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1da7823740 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d1f854ccc0 0x55d1f854f130 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1da7823740 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f853ca60 0x55d1f8548790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1da7823740 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f85379d0 0x55d1f85417c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1da7823740 1 --2- 172.21.15.2:0/2874593148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f8537090 0x55d1f8541280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1da7823740 1 -- 172.21.15.2:0/2874593148 >> 172.21.15.2:0/2874593148 conn(0x55d1f8532700 msgr2=0x55d1f8533f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1d83400640 0 client.18417 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1da7823740 1 -- 172.21.15.2:0/2874593148 shutdown_connections 2024-09-10T08:19:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1da7823740 1 -- 172.21.15.2:0/2874593148 wait complete. 2024-09-10T08:19:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.473+0000 7f1da7823740 20 client.18417 trim_cache size 0 max 16384 2024-09-10T08:19:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.488+0000 7f1da7823740 1 Processor -- start 2024-09-10T08:19:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.488+0000 7f1da7823740 1 -- start start 2024-09-10T08:19:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1da7823740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f8537090 0x55d1f8533af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1da7823740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f85379d0 0x55d1f8548750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1da7823740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f853ca60 0x55d1f8534030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1da7823740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01af3d0 con 0x55d1f85379d0 2024-09-10T08:19:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1da7823740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01acca0 con 0x55d1f8537090 2024-09-10T08:19:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1da7823740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01aaaf0 con 0x55d1f853ca60 2024-09-10T08:19:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f85379d0 0x55d1f8548750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f85379d0 0x55d1f8548750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52808/0 (socket says 172.21.15.2:52808) 2024-09-10T08:19:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1d97e00640 1 -- 172.21.15.2:0/1089544882 learned_addr learned my addr 172.21.15.2:0/1089544882 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1d9de00640 1 --2- 172.21.15.2:0/1089544882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f8537090 0x55d1f8533af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1d9e800640 1 --2- 172.21.15.2:0/1089544882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f853ca60 0x55d1f8534030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1d97e00640 1 -- 172.21.15.2:0/1089544882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f853ca60 msgr2=0x55d1f8534030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1d97e00640 1 --2- 172.21.15.2:0/1089544882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f853ca60 0x55d1f8534030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1d97e00640 1 -- 172.21.15.2:0/1089544882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f8537090 msgr2=0x55d1f8533af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1d97e00640 1 --2- 172.21.15.2:0/1089544882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f8537090 0x55d1f8533af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1d97e00640 1 -- 172.21.15.2:0/1089544882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f8532520 con 0x55d1f85379d0 2024-09-10T08:19:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.489+0000 7f1d97e00640 1 --2- 172.21.15.2:0/1089544882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f85379d0 0x55d1f8548750 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1d88046720 tx=0x7f1d88032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e36fd817fa93ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.490+0000 7f1d8e600640 1 -- 172.21.15.2:0/1089544882 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d880326c0 con 0x55d1f85379d0 2024-09-10T08:19:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.490+0000 7f1d8e600640 1 -- 172.21.15.2:0/1089544882 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d88046c80 con 0x55d1f85379d0 2024-09-10T08:19:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.490+0000 7f1d8e600640 1 -- 172.21.15.2:0/1089544882 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d880039e0 con 0x55d1f85379d0 2024-09-10T08:19:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.490+0000 7f1da7823740 1 -- 172.21.15.2:0/1089544882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f85379d0 msgr2=0x55d1f8548750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.490+0000 7f1da7823740 1 --2- 172.21.15.2:0/1089544882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f85379d0 0x55d1f8548750 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1d88046720 tx=0x7f1d88032f90 comp rx=0 tx=0).stop 2024-09-10T08:19:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.490+0000 7f1da7823740 1 -- 172.21.15.2:0/1089544882 shutdown_connections 2024-09-10T08:19:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.490+0000 7f1da7823740 1 --2- 172.21.15.2:0/1089544882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f853ca60 0x55d1f8534030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.490+0000 7f1da7823740 1 --2- 172.21.15.2:0/1089544882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f85379d0 0x55d1f8548750 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.490+0000 7f1da7823740 1 --2- 172.21.15.2:0/1089544882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f8537090 0x55d1f8533af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.490+0000 7f1da7823740 1 -- 172.21.15.2:0/1089544882 >> 172.21.15.2:0/1089544882 conn(0x55d1f8532700 msgr2=0x55d1f8531880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.490+0000 7f1da7823740 1 -- 172.21.15.2:0/1089544882 shutdown_connections 2024-09-10T08:19:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.490+0000 7f1da7823740 1 -- 172.21.15.2:0/1089544882 wait complete. 2024-09-10T08:19:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.491+0000 7f1da7823740 1 -- start start 2024-09-10T08:19:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.491+0000 7f1da7823740 1 Processor -- start 2024-09-10T08:19:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.491+0000 7f1da7823740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f8537090 0x55d1f8545d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.491+0000 7f1da7823740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f85379d0 0x55d1f85462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.491+0000 7f1da7823740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f853ca60 0x55d1f85472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.491+0000 7f1da7823740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01ab160 con 0x55d1f853ca60 2024-09-10T08:19:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.491+0000 7f1da7823740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01ac5d0 con 0x55d1f85379d0 2024-09-10T08:19:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.491+0000 7f1da7823740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01ab830 con 0x55d1f8537090 2024-09-10T08:19:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.491+0000 7f1da7823740 10 client.0 _ll_register_callbacks cb 0x7f1d88009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f85379d0 0x55d1f85462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d9e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f853ca60 0x55d1f85472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f85379d0 0x55d1f85462a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52872/0 (socket says 172.21.15.2:52872) 2024-09-10T08:19:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d97e00640 1 -- 172.21.15.2:0/114903773 learned_addr learned my addr 172.21.15.2:0/114903773 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d9de00640 1 --2- 172.21.15.2:0/114903773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f8537090 0x55d1f8545d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d9e800640 1 -- 172.21.15.2:0/114903773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f8537090 msgr2=0x55d1f8545d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d9e800640 1 --2- 172.21.15.2:0/114903773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f8537090 0x55d1f8545d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d9e800640 1 -- 172.21.15.2:0/114903773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f85379d0 msgr2=0x55d1f85462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d9e800640 1 --2- 172.21.15.2:0/114903773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f85379d0 0x55d1f85462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d9e800640 1 -- 172.21.15.2:0/114903773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1f85478c0 con 0x55d1f853ca60 2024-09-10T08:19:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d9e800640 1 --2- 172.21.15.2:0/114903773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f853ca60 0x55d1f85472b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1d980cc7a0 tx=0x7f1d9809af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a650434f25a7999f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d83400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d83400640 1 -- 172.21.15.2:0/114903773 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d980b3070 con 0x55d1f853ca60 2024-09-10T08:19:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1d83400640 1 -- 172.21.15.2:0/114903773 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1d980b4260 con 0x55d1f853ca60 2024-09-10T08:19:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.492+0000 7f1da7823740 1 -- 172.21.15.2:0/114903773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d88046c80 con 0x55d1f853ca60 2024-09-10T08:19:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.493+0000 7f1d83400640 1 -- 172.21.15.2:0/114903773 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d98020040 con 0x55d1f853ca60 2024-09-10T08:19:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.493+0000 7f1da7823740 10 client.18423 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.493+0000 7f1da7823740 1 -- 172.21.15.2:0/114903773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d1f85b7a10 con 0x55d1f853ca60 2024-09-10T08:19:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.493+0000 7f1d83400640 1 -- 172.21.15.2:0/114903773 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d88046c80 con 0x55d1f853ca60 2024-09-10T08:19:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.493+0000 7f1d83400640 1 -- 172.21.15.2:0/114903773 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1d98084c60 con 0x55d1f853ca60 2024-09-10T08:19:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.493+0000 7f1da7823740 10 client.18423 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.493+0000 7f1da7823740 10 client.18423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.493+0000 7f1da7823740 20 client.18423 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.493+0000 7f1da7823740 1 --2- 172.21.15.2:0/114903773 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d1f85b8290 0x55d1f85ba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.493+0000 7f1da7823740 4 client.18423 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:19:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.493+0000 7f1da7823740 1 -- 172.21.15.2:0/114903773 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1f84a3b80 con 0x55d1f85b8290 2024-09-10T08:19:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.493+0000 7f1d9de00640 1 --2- 172.21.15.2:0/114903773 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d1f85b8290 0x55d1f85ba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.494+0000 7f1d9de00640 1 --2- 172.21.15.2:0/114903773 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d1f85b8290 0x55d1f85ba680 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6461bd7e883a3147 server_cookie=56627d9f5ec24dd0 in_seq=0 out_seq=0 2024-09-10T08:19:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.494+0000 7f1d83400640 10 client.18423 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.506+0000 7f1d83400640 1 -- 172.21.15.2:0/114903773 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1f84a3b80 con 0x55d1f85b8290 2024-09-10T08:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.506+0000 7f1d83400640 10 client.18423 handle_command_reply: tid=0 2024-09-10T08:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.507+0000 7f1da7823740 1 client.18423 shutdown 2024-09-10T08:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.507+0000 7f1da7823740 1 -- 172.21.15.2:0/114903773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f853ca60 msgr2=0x55d1f85472b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.507+0000 7f1da7823740 1 --2- 172.21.15.2:0/114903773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f853ca60 0x55d1f85472b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1d980cc7a0 tx=0x7f1d9809af10 comp rx=0 tx=0).stop 2024-09-10T08:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.507+0000 7f1da7823740 1 -- 172.21.15.2:0/114903773 shutdown_connections 2024-09-10T08:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.507+0000 7f1da7823740 1 --2- 172.21.15.2:0/114903773 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d1f85b8290 0x55d1f85ba680 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.507+0000 7f1da7823740 1 --2- 172.21.15.2:0/114903773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1f853ca60 0x55d1f85472b0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.507+0000 7f1da7823740 1 --2- 172.21.15.2:0/114903773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1f85379d0 0x55d1f85462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.507+0000 7f1d83400640 0 client.18423 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.507+0000 7f1da7823740 1 --2- 172.21.15.2:0/114903773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1f8537090 0x55d1f8545d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.507+0000 7f1da7823740 1 -- 172.21.15.2:0/114903773 >> 172.21.15.2:0/114903773 conn(0x55d1f8532700 msgr2=0x55d1f8532ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.507+0000 7f1da7823740 1 -- 172.21.15.2:0/114903773 shutdown_connections 2024-09-10T08:19:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.507+0000 7f1da7823740 1 -- 172.21.15.2:0/114903773 wait complete. 2024-09-10T08:19:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.507+0000 7f1da7823740 20 client.18423 trim_cache size 0 max 16384 2024-09-10T08:19:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:19:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:19:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:19:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:19:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.508+0000 7f1da4a00640 1 -- 172.21.15.2:0/2079843809 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d840a2cb0 msgr2=0x7f1d840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.508+0000 7f1da4a00640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d840a2cb0 0x7f1d840a5170 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1d88031440 tx=0x7f1d880315b0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.508+0000 7f1da4a00640 1 -- 172.21.15.2:0/2079843809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da01099f0 msgr2=0x7f1da01a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.508+0000 7f1da4a00640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da01099f0 0x7f1da01a15e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009fd0 tx=0x7f1d9800b260 comp rx=0 tx=0).stop 2024-09-10T08:19:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.508+0000 7f1da4a00640 1 -- 172.21.15.2:0/2079843809 shutdown_connections 2024-09-10T08:19:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.508+0000 7f1da4a00640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da01099f0 0x7f1da01a15e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.508+0000 7f1da4a00640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d840a2cb0 0x7f1d840a5170 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.508+0000 7f1da4a00640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da00761d0 0x7f1da019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.508+0000 7f1da4a00640 1 --2- 172.21.15.2:0/2079843809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1da0074bf0 0x7f1da019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.508+0000 7f1da4a00640 1 -- 172.21.15.2:0/2079843809 >> 172.21.15.2:0/2079843809 conn(0x7f1da00fd2e0 msgr2=0x7f1da0105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.508+0000 7f1da4a00640 1 -- 172.21.15.2:0/2079843809 shutdown_connections 2024-09-10T08:19:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:02.508+0000 7f1da4a00640 1 -- 172.21.15.2:0/2079843809 wait complete. 2024-09-10T08:19:02.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:02 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2942717834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:19:02.749 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:19:02.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:02 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2942717834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:19:02.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:02 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2942717834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:19:03.169 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:03.655 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:03 smithi002 ceph-mon[30563]: pgmap v714: 129 pgs: 129 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:19:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.771+0000 7ff146200640 1 -- 172.21.15.2:0/209556426 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff140076720 msgr2=0x7ff140076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.771+0000 7ff146200640 1 --2- 172.21.15.2:0/209556426 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff140076720 0x7ff140076b00 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff134009900 tx=0x7ff13402a410 comp rx=0 tx=0).stop 2024-09-10T08:19:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.771+0000 7ff146200640 1 -- 172.21.15.2:0/209556426 shutdown_connections 2024-09-10T08:19:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.771+0000 7ff146200640 1 --2- 172.21.15.2:0/209556426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff140112870 0x7ff140114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.771+0000 7ff146200640 1 --2- 172.21.15.2:0/209556426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff140077040 0x7ff140112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.771+0000 7ff146200640 1 --2- 172.21.15.2:0/209556426 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff140076720 0x7ff140076b00 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.771+0000 7ff146200640 1 -- 172.21.15.2:0/209556426 >> 172.21.15.2:0/209556426 conn(0x7ff1400ff8d0 msgr2=0x7ff140101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.772+0000 7ff146200640 1 -- 172.21.15.2:0/209556426 shutdown_connections 2024-09-10T08:19:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.772+0000 7ff146200640 1 -- 172.21.15.2:0/209556426 wait complete. 2024-09-10T08:19:03.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.772+0000 7ff146200640 1 Processor -- start 2024-09-10T08:19:03.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.773+0000 7ff146200640 1 -- start start 2024-09-10T08:19:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.773+0000 7ff146200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff140076720 0x7ff1401a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.773+0000 7ff146200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff140077040 0x7ff1401a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.773+0000 7ff146200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff140112870 0x7ff14019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.773+0000 7ff146200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff140118430 con 0x7ff140077040 2024-09-10T08:19:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.773+0000 7ff146200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff1401182b0 con 0x7ff140076720 2024-09-10T08:19:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.773+0000 7ff146200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1401185b0 con 0x7ff140112870 2024-09-10T08:19:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.773+0000 7ff13fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff140077040 0x7ff1401a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.773+0000 7ff145800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff140112870 0x7ff14019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.773+0000 7ff144e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff140076720 0x7ff1401a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.773+0000 7ff144e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff140076720 0x7ff1401a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52886/0 (socket says 172.21.15.2:52886) 2024-09-10T08:19:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.774+0000 7ff144e00640 1 -- 172.21.15.2:0/506835328 learned_addr learned my addr 172.21.15.2:0/506835328 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.774+0000 7ff145800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff140112870 0x7ff14019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55466/0 (socket says 172.21.15.2:55466) 2024-09-10T08:19:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.774+0000 7ff13fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff140077040 0x7ff1401a4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52844/0 (socket says 172.21.15.2:52844) 2024-09-10T08:19:03.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.774+0000 7ff144e00640 1 -- 172.21.15.2:0/506835328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff140112870 msgr2=0x7ff14019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:03.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.774+0000 7ff144e00640 1 --2- 172.21.15.2:0/506835328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff140112870 0x7ff14019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.774+0000 7ff144e00640 1 -- 172.21.15.2:0/506835328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff140077040 msgr2=0x7ff1401a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.774+0000 7ff144e00640 1 --2- 172.21.15.2:0/506835328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff140077040 0x7ff1401a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.774+0000 7ff144e00640 1 -- 172.21.15.2:0/506835328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14019ef10 con 0x7ff140076720 2024-09-10T08:19:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.774+0000 7ff144e00640 1 --2- 172.21.15.2:0/506835328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff140076720 0x7ff1401a45d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff134004770 tx=0x7ff134003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.775+0000 7ff13d600640 1 -- 172.21.15.2:0/506835328 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13403f070 con 0x7ff140076720 2024-09-10T08:19:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.775+0000 7ff146200640 1 -- 172.21.15.2:0/506835328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14019f1a0 con 0x7ff140076720 2024-09-10T08:19:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.775+0000 7ff146200640 1 -- 172.21.15.2:0/506835328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1401ab380 con 0x7ff140076720 2024-09-10T08:19:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.775+0000 7ff13d600640 1 -- 172.21.15.2:0/506835328 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff134031ca0 con 0x7ff140076720 2024-09-10T08:19:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.775+0000 7ff13d600640 1 -- 172.21.15.2:0/506835328 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134049060 con 0x7ff140076720 2024-09-10T08:19:03.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.779+0000 7ff14a1e1740 1 Processor -- start 2024-09-10T08:19:03.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.779+0000 7ff14a1e1740 1 -- start start 2024-09-10T08:19:03.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.779+0000 7ff14a1e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba2ce40 0x55accba2d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.779+0000 7ff14a1e1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2d760 0x55accba37ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.779+0000 7ff14a1e1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba38020 0x55accba3a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.779+0000 7ff14a1e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55accb9933e0 con 0x55accba2ce40 2024-09-10T08:19:03.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.779+0000 7ff14a1e1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55accb93ad80 con 0x55accba38020 2024-09-10T08:19:03.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.779+0000 7ff14a1e1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55accb98f3a0 con 0x55accba2d760 2024-09-10T08:19:03.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.780+0000 7ff145800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba2ce40 0x55accba2d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.780+0000 7ff13d600640 1 -- 172.21.15.2:0/506835328 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff134042070 con 0x7ff140076720 2024-09-10T08:19:03.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.780+0000 7ff145800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba2ce40 0x55accba2d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52850/0 (socket says 172.21.15.2:52850) 2024-09-10T08:19:03.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.780+0000 7ff145800640 1 -- 172.21.15.2:0/2490907346 learned_addr learned my addr 172.21.15.2:0/2490907346 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:03.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.780+0000 7ff13fe00640 1 --2- 172.21.15.2:0/2490907346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba38020 0x55accba3a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.780+0000 7ff145800640 1 -- 172.21.15.2:0/2490907346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2d760 msgr2=0x55accba37ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:03.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.780+0000 7ff144e00640 1 --2- 172.21.15.2:0/2490907346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2d760 0x55accba37ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.780+0000 7ff145800640 1 --2- 172.21.15.2:0/2490907346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2d760 0x55accba37ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.780+0000 7ff145800640 1 -- 172.21.15.2:0/2490907346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba38020 msgr2=0x55accba3a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:03.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.780+0000 7ff145800640 1 --2- 172.21.15.2:0/2490907346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba38020 0x55accba3a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.780+0000 7ff145800640 1 -- 172.21.15.2:0/2490907346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55accb959ae0 con 0x55accba2ce40 2024-09-10T08:19:03.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.780+0000 7ff144e00640 1 --2- 172.21.15.2:0/2490907346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2d760 0x55accba37ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:19:03.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.781+0000 7ff145800640 1 --2- 172.21.15.2:0/2490907346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba2ce40 0x55accba2d220 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff138007e30 tx=0x7ff13800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e498aa33b33d4a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:03.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.781+0000 7ff132000640 1 -- 172.21.15.2:0/2490907346 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138010840 con 0x55accba2ce40 2024-09-10T08:19:03.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.781+0000 7ff132000640 1 -- 172.21.15.2:0/2490907346 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1380183f0 con 0x55accba2ce40 2024-09-10T08:19:03.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.781+0000 7ff132000640 1 -- 172.21.15.2:0/2490907346 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138017600 con 0x55accba2ce40 2024-09-10T08:19:03.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.781+0000 7ff13d600640 1 --2- 172.21.15.2:0/506835328 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff1280a2c00 0x7ff1280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.782+0000 7ff13d600640 1 -- 172.21.15.2:0/506835328 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff1340ef780 con 0x7ff140076720 2024-09-10T08:19:03.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.782+0000 7ff144e00640 1 --2- 172.21.15.2:0/506835328 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff1280a2c00 0x7ff1280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.782+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2490907346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba2ce40 msgr2=0x55accba2d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:03.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.782+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2490907346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba2ce40 0x55accba2d220 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff138007e30 tx=0x7ff13800f790 comp rx=0 tx=0).stop 2024-09-10T08:19:03.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.782+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2490907346 shutdown_connections 2024-09-10T08:19:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.782+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2490907346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba38020 0x55accba3a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.782+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2490907346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2d760 0x55accba37ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.782+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2490907346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba2ce40 0x55accba2d220 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.782+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2490907346 >> 172.21.15.2:0/2490907346 conn(0x55accba293d0 msgr2=0x55accba297b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.782+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2490907346 shutdown_connections 2024-09-10T08:19:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.782+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2490907346 wait complete. 2024-09-10T08:19:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.782+0000 7ff144e00640 1 --2- 172.21.15.2:0/506835328 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff1280a2c00 0x7ff1280a50c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff134036040 tx=0x7ff134009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.783+0000 7ff14a1e1740 1 -- start start 2024-09-10T08:19:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.783+0000 7ff14a1e1740 1 Processor -- start 2024-09-10T08:19:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.783+0000 7ff14a1e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba2ce40 0x55accba34e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.783+0000 7ff14a1e1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.783+0000 7ff14a1e1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba38020 0x55accba358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.783+0000 7ff14a1e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55accba3abf0 con 0x55accba2ce40 2024-09-10T08:19:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.783+0000 7ff14a1e1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55accb94e0f0 con 0x55accba2d760 2024-09-10T08:19:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.783+0000 7ff14a1e1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55accba3ad70 con 0x55accba38020 2024-09-10T08:19:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.784+0000 7ff14a1e1740 10 client.0 _ll_register_callbacks cb 0x55accb935bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.784+0000 7ff13fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.784+0000 7ff13fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba353b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52906/0 (socket says 172.21.15.2:52906) 2024-09-10T08:19:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.784+0000 7ff13fe00640 1 -- 172.21.15.2:0/1476314550 learned_addr learned my addr 172.21.15.2:0/1476314550 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.784+0000 7ff145800640 1 --2- 172.21.15.2:0/1476314550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba2ce40 0x55accba34e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.784+0000 7ff13fe00640 1 -- 172.21.15.2:0/1476314550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba38020 msgr2=0x55accba358f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.784+0000 7ff13fe00640 1 --2- 172.21.15.2:0/1476314550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba38020 0x55accba358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.784+0000 7ff13fe00640 1 -- 172.21.15.2:0/1476314550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba2ce40 msgr2=0x55accba34e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.784+0000 7ff13fe00640 1 --2- 172.21.15.2:0/1476314550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba2ce40 0x55accba34e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.784+0000 7ff13fe00640 1 -- 172.21.15.2:0/1476314550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55accb7043c0 con 0x55accba2d760 2024-09-10T08:19:03.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.784+0000 7ff145800640 1 --2- 172.21.15.2:0/1476314550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba2ce40 0x55accba34e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:03.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.785+0000 7ff13fe00640 1 --2- 172.21.15.2:0/1476314550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba353b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff12c012730 tx=0x7ff12c012bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=198eaa4a6461b4e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:03.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.785+0000 7ff126a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:19:03.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.785+0000 7ff126a00640 1 -- 172.21.15.2:0/1476314550 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00f8c0 con 0x55accba2d760 2024-09-10T08:19:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.785+0000 7ff14a1e1740 1 -- 172.21.15.2:0/1476314550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1380183f0 con 0x55accba2d760 2024-09-10T08:19:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.785+0000 7ff126a00640 1 -- 172.21.15.2:0/1476314550 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff12c00fa60 con 0x55accba2d760 2024-09-10T08:19:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.785+0000 7ff126a00640 1 -- 172.21.15.2:0/1476314550 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff12c025050 con 0x55accba2d760 2024-09-10T08:19:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.785+0000 7ff14a1e1740 10 client.28466 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.785+0000 7ff14a1e1740 1 -- 172.21.15.2:0/1476314550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55accba46be0 con 0x55accba2d760 2024-09-10T08:19:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.785+0000 7ff126a00640 1 -- 172.21.15.2:0/1476314550 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1380183f0 con 0x55accba2d760 2024-09-10T08:19:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.786+0000 7ff126a00640 1 -- 172.21.15.2:0/1476314550 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff12c010430 con 0x55accba2d760 2024-09-10T08:19:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.786+0000 7ff14a1e1740 10 client.28466 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.786+0000 7ff14a1e1740 10 client.28466 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:19:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.786+0000 7ff14a1e1740 20 client.28466 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.786+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/1476314550 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55accba48040 0x55accba4a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.786+0000 7ff14a1e1740 4 client.28466 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.786+0000 7ff14a1e1740 1 -- 172.21.15.2:0/1476314550 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55accb9a0950 con 0x55accba48040 2024-09-10T08:19:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.786+0000 7ff145800640 1 --2- 172.21.15.2:0/1476314550 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55accba48040 0x55accba4a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.787+0000 7ff145800640 1 --2- 172.21.15.2:0/1476314550 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55accba48040 0x55accba4a4a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c39d299f77bdffbf server_cookie=39f3204029cc686 in_seq=0 out_seq=0 2024-09-10T08:19:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.787+0000 7ff126a00640 10 client.28466 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:19:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.787+0000 7ff126a00640 1 -- 172.21.15.2:0/1476314550 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55accb9a0950 con 0x55accba48040 2024-09-10T08:19:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.787+0000 7ff126a00640 10 client.28466 handle_command_reply: tid=0 2024-09-10T08:19:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.787+0000 7ff14a1e1740 1 client.28466 shutdown 2024-09-10T08:19:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.788+0000 7ff14a1e1740 1 -- 172.21.15.2:0/1476314550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 msgr2=0x55accba353b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.788+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/1476314550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba353b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff12c012730 tx=0x7ff12c012bf0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.788+0000 7ff14a1e1740 1 -- 172.21.15.2:0/1476314550 shutdown_connections 2024-09-10T08:19:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.788+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/1476314550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba38020 0x55accba358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.788+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/1476314550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba353b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.788+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/1476314550 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55accba48040 0x55accba4a4a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.788+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/1476314550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba2ce40 0x55accba34e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.788+0000 7ff14a1e1740 1 -- 172.21.15.2:0/1476314550 >> 172.21.15.2:0/1476314550 conn(0x55accba293d0 msgr2=0x55accba284b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:03.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.788+0000 7ff126a00640 0 client.28466 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:19:03.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.788+0000 7ff14a1e1740 1 -- 172.21.15.2:0/1476314550 shutdown_connections 2024-09-10T08:19:03.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.788+0000 7ff14a1e1740 1 -- 172.21.15.2:0/1476314550 wait complete. 2024-09-10T08:19:03.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.788+0000 7ff14a1e1740 20 client.28466 trim_cache size 0 max 16384 2024-09-10T08:19:03.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.802+0000 7ff14a1e1740 1 Processor -- start 2024-09-10T08:19:03.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.802+0000 7ff14a1e1740 1 -- start start 2024-09-10T08:19:03.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.802+0000 7ff14a1e1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2ce40 0x55accba24300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.802+0000 7ff14a1e1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba4fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.802+0000 7ff14a1e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba38020 0x55accba502b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.802+0000 7ff14a1e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff1401b7ac0 con 0x55accba38020 2024-09-10T08:19:03.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.802+0000 7ff14a1e1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff1401b5390 con 0x55accba2d760 2024-09-10T08:19:03.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.802+0000 7ff14a1e1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1401b31e0 con 0x55accba2ce40 2024-09-10T08:19:03.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.803+0000 7ff145800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba38020 0x55accba502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.803+0000 7ff13fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba4fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.803+0000 7ff13fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba4fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52916/0 (socket says 172.21.15.2:52916) 2024-09-10T08:19:03.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.803+0000 7ff13fe00640 1 -- 172.21.15.2:0/2033988129 learned_addr learned my addr 172.21.15.2:0/2033988129 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:03.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.803+0000 7ff145800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba38020 0x55accba502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52860/0 (socket says 172.21.15.2:52860) 2024-09-10T08:19:03.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.803+0000 7ff145800640 1 --2- 172.21.15.2:0/2033988129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2ce40 0x55accba24300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.803+0000 7ff13fe00640 1 -- 172.21.15.2:0/2033988129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2ce40 msgr2=0x55accba24300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:03.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.803+0000 7ff13fe00640 1 --2- 172.21.15.2:0/2033988129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2ce40 0x55accba24300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.803+0000 7ff13fe00640 1 -- 172.21.15.2:0/2033988129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba38020 msgr2=0x55accba502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.803+0000 7ff13fe00640 1 --2- 172.21.15.2:0/2033988129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba38020 0x55accba502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.803+0000 7ff13fe00640 1 -- 172.21.15.2:0/2033988129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55accba2d590 con 0x55accba2d760 2024-09-10T08:19:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.803+0000 7ff13fe00640 1 --2- 172.21.15.2:0/2033988129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba4fd70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0096c0 tx=0x7ff12c0096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd37c35e3b239dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.804+0000 7ff131600640 1 -- 172.21.15.2:0/2033988129 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0104f0 con 0x55accba2d760 2024-09-10T08:19:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.804+0000 7ff131600640 1 -- 172.21.15.2:0/2033988129 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff12c0055b0 con 0x55accba2d760 2024-09-10T08:19:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.804+0000 7ff131600640 1 -- 172.21.15.2:0/2033988129 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00f8d0 con 0x55accba2d760 2024-09-10T08:19:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.804+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2033988129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 msgr2=0x55accba4fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.804+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2033988129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba4fd70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0096c0 tx=0x7ff12c0096f0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.804+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2033988129 shutdown_connections 2024-09-10T08:19:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.804+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2033988129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba38020 0x55accba502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.804+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2033988129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba4fd70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.804+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2033988129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2ce40 0x55accba24300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.804+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2033988129 >> 172.21.15.2:0/2033988129 conn(0x55accba293d0 msgr2=0x55accba284b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.805+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2033988129 shutdown_connections 2024-09-10T08:19:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.805+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2033988129 wait complete. 2024-09-10T08:19:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.805+0000 7ff14a1e1740 1 -- start start 2024-09-10T08:19:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.805+0000 7ff14a1e1740 1 Processor -- start 2024-09-10T08:19:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff14a1e1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2ce40 0x55accba40890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff14a1e1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba31ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff14a1e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba38020 0x55accba32470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff14a1e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff1401b3850 con 0x55accba38020 2024-09-10T08:19:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff14a1e1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff1401b4cc0 con 0x55accba2d760 2024-09-10T08:19:03.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff14a1e1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1401b3f20 con 0x55accba2ce40 2024-09-10T08:19:03.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff14a1e1740 10 client.0 _ll_register_callbacks cb 0x7ff12c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:03.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff145800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba38020 0x55accba32470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff145800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba38020 0x55accba32470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52868/0 (socket says 172.21.15.2:52868) 2024-09-10T08:19:03.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff145800640 1 -- 172.21.15.2:0/2094748541 learned_addr learned my addr 172.21.15.2:0/2094748541 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:03.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff145800640 1 --2- 172.21.15.2:0/2094748541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2ce40 0x55accba40890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff13fe00640 1 --2- 172.21.15.2:0/2094748541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba31ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff145800640 1 -- 172.21.15.2:0/2094748541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2ce40 msgr2=0x55accba40890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:03.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff145800640 1 --2- 172.21.15.2:0/2094748541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2ce40 0x55accba40890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff145800640 1 -- 172.21.15.2:0/2094748541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 msgr2=0x55accba31ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:03.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff145800640 1 --2- 172.21.15.2:0/2094748541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba31ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.806+0000 7ff145800640 1 -- 172.21.15.2:0/2094748541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55accbab1d40 con 0x55accba38020 2024-09-10T08:19:03.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.807+0000 7ff145800640 1 --2- 172.21.15.2:0/2094748541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba38020 0x55accba32470 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff138018db0 tx=0x7ff138012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7041a00707dd926a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:03.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.807+0000 7ff126a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:03.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.807+0000 7ff126a00640 1 -- 172.21.15.2:0/2094748541 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138010840 con 0x55accba38020 2024-09-10T08:19:03.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.807+0000 7ff126a00640 1 -- 172.21.15.2:0/2094748541 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff1380109e0 con 0x55accba38020 2024-09-10T08:19:03.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.807+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2094748541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff12c0055b0 con 0x55accba38020 2024-09-10T08:19:03.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.807+0000 7ff126a00640 1 -- 172.21.15.2:0/2094748541 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff138004e30 con 0x55accba38020 2024-09-10T08:19:03.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.807+0000 7ff14a1e1740 10 client.18441 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:03.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.807+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2094748541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55accbab2e90 con 0x55accba38020 2024-09-10T08:19:03.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.807+0000 7ff126a00640 1 -- 172.21.15.2:0/2094748541 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff12c0055b0 con 0x55accba38020 2024-09-10T08:19:03.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.808+0000 7ff126a00640 1 -- 172.21.15.2:0/2094748541 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff13802f540 con 0x55accba38020 2024-09-10T08:19:03.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.808+0000 7ff14a1e1740 10 client.18441 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:03.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.808+0000 7ff14a1e1740 10 client.18441 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:19:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.808+0000 7ff14a1e1740 20 client.18441 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.808+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2094748541 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55accbab3410 0x55accbab5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.808+0000 7ff14a1e1740 4 client.18441 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:19:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.808+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2094748541 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55accb99ae60 con 0x55accbab3410 2024-09-10T08:19:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.808+0000 7ff13fe00640 1 --2- 172.21.15.2:0/2094748541 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55accbab3410 0x55accbab5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff13fe00640 1 --2- 172.21.15.2:0/2094748541 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55accbab3410 0x55accbab5800 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bffe7610b63bc3 server_cookie=7b8083cdf69a4a79 in_seq=0 out_seq=0 2024-09-10T08:19:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff126a00640 10 client.18441 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:19:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff126a00640 1 -- 172.21.15.2:0/2094748541 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55accb99ae60 con 0x55accbab3410 2024-09-10T08:19:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff126a00640 10 client.18441 handle_command_reply: tid=0 2024-09-10T08:19:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff14a1e1740 1 client.18441 shutdown 2024-09-10T08:19:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2094748541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba38020 msgr2=0x55accba32470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2094748541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba38020 0x55accba32470 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff138018db0 tx=0x7ff138012040 comp rx=0 tx=0).stop 2024-09-10T08:19:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2094748541 shutdown_connections 2024-09-10T08:19:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2094748541 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55accbab3410 0x55accbab5800 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2094748541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55accba38020 0x55accba32470 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2094748541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55accba2d760 0x55accba31ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff14a1e1740 1 --2- 172.21.15.2:0/2094748541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55accba2ce40 0x55accba40890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2094748541 >> 172.21.15.2:0/2094748541 conn(0x55accba293d0 msgr2=0x55accba33740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:03.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff126a00640 0 client.18441 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:19:03.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2094748541 shutdown_connections 2024-09-10T08:19:03.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.809+0000 7ff14a1e1740 1 -- 172.21.15.2:0/2094748541 wait complete. 2024-09-10T08:19:03.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.810+0000 7ff14a1e1740 20 client.18441 trim_cache size 0 max 16384 2024-09-10T08:19:03.816 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:19:03.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.810+0000 7ff146200640 1 -- 172.21.15.2:0/506835328 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff1280a2c00 msgr2=0x7ff1280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:03.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.810+0000 7ff146200640 1 --2- 172.21.15.2:0/506835328 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff1280a2c00 0x7ff1280a50c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff134036040 tx=0x7ff134009260 comp rx=0 tx=0).stop 2024-09-10T08:19:03.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.810+0000 7ff146200640 1 -- 172.21.15.2:0/506835328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff140076720 msgr2=0x7ff1401a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:03.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.810+0000 7ff146200640 1 --2- 172.21.15.2:0/506835328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff140076720 0x7ff1401a45d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff134004770 tx=0x7ff134003c10 comp rx=0 tx=0).stop 2024-09-10T08:19:03.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.810+0000 7ff146200640 1 -- 172.21.15.2:0/506835328 shutdown_connections 2024-09-10T08:19:03.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.810+0000 7ff146200640 1 --2- 172.21.15.2:0/506835328 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff1280a2c00 0x7ff1280a50c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.810+0000 7ff146200640 1 --2- 172.21.15.2:0/506835328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff140112870 0x7ff14019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.810+0000 7ff146200640 1 --2- 172.21.15.2:0/506835328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff140077040 0x7ff1401a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.810+0000 7ff146200640 1 --2- 172.21.15.2:0/506835328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff140076720 0x7ff1401a45d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:03.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.810+0000 7ff146200640 1 -- 172.21.15.2:0/506835328 >> 172.21.15.2:0/506835328 conn(0x7ff1400ff8d0 msgr2=0x7ff140101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:03.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.810+0000 7ff146200640 1 -- 172.21.15.2:0/506835328 shutdown_connections 2024-09-10T08:19:03.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:03.811+0000 7ff146200640 1 -- 172.21.15.2:0/506835328 wait complete. 2024-09-10T08:19:03.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:03 smithi155 ceph-mon[35405]: pgmap v714: 129 pgs: 129 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:19:03.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:03 smithi107 ceph-mon[35048]: pgmap v714: 129 pgs: 129 active+clean; 1.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:19:04.624 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:19:05.001 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:05.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.612+0000 7f16f0a00640 1 -- 172.21.15.2:0/3514388477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16ec112a30 msgr2=0x7f16ec114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:05.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.612+0000 7f16f0a00640 1 --2- 172.21.15.2:0/3514388477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16ec112a30 0x7f16ec114e20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009ca0 tx=0x7f16e402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.612+0000 7f16f0a00640 1 -- 172.21.15.2:0/3514388477 shutdown_connections 2024-09-10T08:19:05.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.612+0000 7f16f0a00640 1 --2- 172.21.15.2:0/3514388477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16ec112a30 0x7f16ec114e20 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.612+0000 7f16f0a00640 1 --2- 172.21.15.2:0/3514388477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec077040 0x7f16ec1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.612+0000 7f16f0a00640 1 --2- 172.21.15.2:0/3514388477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16ec076720 0x7f16ec076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.612+0000 7f16f0a00640 1 -- 172.21.15.2:0/3514388477 >> 172.21.15.2:0/3514388477 conn(0x7f16ec0ffa90 msgr2=0x7f16ec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:05.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.613+0000 7f16f0a00640 1 -- 172.21.15.2:0/3514388477 shutdown_connections 2024-09-10T08:19:05.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.613+0000 7f16f0a00640 1 -- 172.21.15.2:0/3514388477 wait complete. 2024-09-10T08:19:05.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.613+0000 7f16f0a00640 1 Processor -- start 2024-09-10T08:19:05.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.614+0000 7f16f0a00640 1 -- start start 2024-09-10T08:19:05.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.614+0000 7f16f0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16ec076720 0x7f16ec1a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.614+0000 7f16f0a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec077040 0x7f16ec1a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.614+0000 7f16f0a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16ec112a30 0x7f16ec19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.614+0000 7f16f0a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec118590 con 0x7f16ec112a30 2024-09-10T08:19:05.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.614+0000 7f16f0a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec118410 con 0x7f16ec076720 2024-09-10T08:19:05.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.614+0000 7f16f0a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec118710 con 0x7f16ec077040 2024-09-10T08:19:05.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.614+0000 7f16ea800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16ec112a30 0x7f16ec19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.614+0000 7f16e9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec077040 0x7f16ec1a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.614+0000 7f16e9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec077040 0x7f16ec1a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55528/0 (socket says 172.21.15.2:55528) 2024-09-10T08:19:05.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.614+0000 7f16e9400640 1 -- 172.21.15.2:0/4184297136 learned_addr learned my addr 172.21.15.2:0/4184297136 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:05.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.614+0000 7f16e9e00640 1 --2- 172.21.15.2:0/4184297136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16ec076720 0x7f16ec1a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.615+0000 7f16e9400640 1 -- 172.21.15.2:0/4184297136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16ec076720 msgr2=0x7f16ec1a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:05.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.615+0000 7f16e9400640 1 --2- 172.21.15.2:0/4184297136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16ec076720 0x7f16ec1a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.615+0000 7f16e9400640 1 -- 172.21.15.2:0/4184297136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16ec112a30 msgr2=0x7f16ec19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:05.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.615+0000 7f16e9400640 1 --2- 172.21.15.2:0/4184297136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16ec112a30 0x7f16ec19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.615+0000 7f16e9400640 1 -- 172.21.15.2:0/4184297136 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ec19f000 con 0x7f16ec077040 2024-09-10T08:19:05.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.615+0000 7f16e9400640 1 --2- 172.21.15.2:0/4184297136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec077040 0x7f16ec1a4c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f16d800e9f0 tx=0x7f16d800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:05.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.615+0000 7f16e2a00640 1 -- 172.21.15.2:0/4184297136 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d800cd80 con 0x7f16ec077040 2024-09-10T08:19:05.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.615+0000 7f16e2a00640 1 -- 172.21.15.2:0/4184297136 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d8005500 con 0x7f16ec077040 2024-09-10T08:19:05.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.615+0000 7f16f0a00640 1 -- 172.21.15.2:0/4184297136 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ec19f2f0 con 0x7f16ec077040 2024-09-10T08:19:05.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.615+0000 7f16e2a00640 1 -- 172.21.15.2:0/4184297136 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8010690 con 0x7f16ec077040 2024-09-10T08:19:05.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.616+0000 7f16f0a00640 1 -- 172.21.15.2:0/4184297136 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ec1ab5e0 con 0x7f16ec077040 2024-09-10T08:19:05.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.618+0000 7f16f349c740 1 Processor -- start 2024-09-10T08:19:05.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.619+0000 7f16f349c740 1 -- start start 2024-09-10T08:19:05.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.619+0000 7f16e2a00640 1 -- 172.21.15.2:0/4184297136 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f16d80046c0 con 0x7f16ec077040 2024-09-10T08:19:05.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.619+0000 7f16f349c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4dc70 0x55d135a4e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.619+0000 7f16f349c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a58910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.619+0000 7f16f349c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a58e50 0x55d135a5b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.619+0000 7f16f349c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d1359b43e0 con 0x55d135a58e50 2024-09-10T08:19:05.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.619+0000 7f16f349c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d13595bd80 con 0x55d135a4dc70 2024-09-10T08:19:05.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.619+0000 7f16f349c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1359b03a0 con 0x55d135a4e590 2024-09-10T08:19:05.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.619+0000 7f16e9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a58910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.619+0000 7f16e9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a58910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55536/0 (socket says 172.21.15.2:55536) 2024-09-10T08:19:05.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.619+0000 7f16e9e00640 1 -- 172.21.15.2:0/2095384763 learned_addr learned my addr 172.21.15.2:0/2095384763 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:05.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.619+0000 7f16ea800640 1 --2- 172.21.15.2:0/2095384763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4dc70 0x55d135a4e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.619+0000 7f16e9400640 1 --2- 172.21.15.2:0/2095384763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a58e50 0x55d135a5b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.620+0000 7f16e9e00640 1 -- 172.21.15.2:0/2095384763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4dc70 msgr2=0x55d135a4e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:05.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.620+0000 7f16e9e00640 1 --2- 172.21.15.2:0/2095384763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4dc70 0x55d135a4e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.620+0000 7f16e9e00640 1 -- 172.21.15.2:0/2095384763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a58e50 msgr2=0x55d135a5b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:05.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.620+0000 7f16e9e00640 1 --2- 172.21.15.2:0/2095384763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a58e50 0x55d135a5b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.620+0000 7f16e9e00640 1 -- 172.21.15.2:0/2095384763 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d13597aae0 con 0x55d135a4e590 2024-09-10T08:19:05.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.620+0000 7f16e9e00640 1 --2- 172.21.15.2:0/2095384763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a58910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009c90 tx=0x7f16dc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6da8c77910de4cb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:05.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.620+0000 7f16e2a00640 1 --2- 172.21.15.2:0/4184297136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16d00a2b80 0x7f16d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.620+0000 7f16d7000640 1 -- 172.21.15.2:0/2095384763 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc005220 con 0x55d135a4e590 2024-09-10T08:19:05.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.621+0000 7f16d7000640 1 -- 172.21.15.2:0/2095384763 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16dc004c80 con 0x55d135a4e590 2024-09-10T08:19:05.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.621+0000 7f16d7000640 1 -- 172.21.15.2:0/2095384763 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0197d0 con 0x55d135a4e590 2024-09-10T08:19:05.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.621+0000 7f16e2a00640 1 -- 172.21.15.2:0/4184297136 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f16d80cf830 con 0x7f16ec077040 2024-09-10T08:19:05.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.621+0000 7f16ea800640 1 --2- 172.21.15.2:0/4184297136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16d00a2b80 0x7f16d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.621+0000 7f16f349c740 1 -- 172.21.15.2:0/2095384763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 msgr2=0x55d135a58910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:05.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.621+0000 7f16f349c740 1 --2- 172.21.15.2:0/2095384763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a58910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009c90 tx=0x7f16dc011810 comp rx=0 tx=0).stop 2024-09-10T08:19:05.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.621+0000 7f16f349c740 1 -- 172.21.15.2:0/2095384763 shutdown_connections 2024-09-10T08:19:05.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.621+0000 7f16f349c740 1 --2- 172.21.15.2:0/2095384763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a58e50 0x55d135a5b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.621+0000 7f16f349c740 1 --2- 172.21.15.2:0/2095384763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a58910 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.621+0000 7f16f349c740 1 --2- 172.21.15.2:0/2095384763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4dc70 0x55d135a4e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.621+0000 7f16f349c740 1 -- 172.21.15.2:0/2095384763 >> 172.21.15.2:0/2095384763 conn(0x55d135a4a200 msgr2=0x55d135a4a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:05.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.622+0000 7f16f349c740 1 -- 172.21.15.2:0/2095384763 shutdown_connections 2024-09-10T08:19:05.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.622+0000 7f16ea800640 1 --2- 172.21.15.2:0/4184297136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16d00a2b80 0x7f16d00a5040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006010 tx=0x7f16e4004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:05.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.622+0000 7f16f349c740 1 -- 172.21.15.2:0/2095384763 wait complete. 2024-09-10T08:19:05.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.622+0000 7f16f349c740 1 -- start start 2024-09-10T08:19:05.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.622+0000 7f16f349c740 1 Processor -- start 2024-09-10T08:19:05.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.623+0000 7f16f349c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 0x55d135a55c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.623+0000 7f16f349c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4e590 0x55d135a561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.623+0000 7f16f349c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a58e50 0x55d135a566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.623+0000 7f16f349c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d135a5ba20 con 0x55d135a4dc70 2024-09-10T08:19:05.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.623+0000 7f16f349c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d13596f0f0 con 0x55d135a4e590 2024-09-10T08:19:05.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.623+0000 7f16f349c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d135a5bba0 con 0x55d135a58e50 2024-09-10T08:19:05.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.623+0000 7f16f349c740 10 client.0 _ll_register_callbacks cb 0x55d135956bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:05.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.623+0000 7f16e9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 0x55d135a55c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.623+0000 7f16e9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 0x55d135a55c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52904/0 (socket says 172.21.15.2:52904) 2024-09-10T08:19:05.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.623+0000 7f16e9e00640 1 -- 172.21.15.2:0/3607646012 learned_addr learned my addr 172.21.15.2:0/3607646012 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:05.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.623+0000 7f16e9400640 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4e590 0x55d135a561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.623+0000 7f16ea800640 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a58e50 0x55d135a566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.624+0000 7f16e9e00640 1 -- 172.21.15.2:0/3607646012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a58e50 msgr2=0x55d135a566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.624+0000 7f16e9e00640 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a58e50 0x55d135a566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.624+0000 7f16e9e00640 1 -- 172.21.15.2:0/3607646012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4e590 msgr2=0x55d135a561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.624+0000 7f16e9e00640 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4e590 0x55d135a561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.624+0000 7f16e9e00640 1 -- 172.21.15.2:0/3607646012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1357253c0 con 0x55d135a4dc70 2024-09-10T08:19:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.624+0000 7f16ea800640 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a58e50 0x55d135a566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:19:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.624+0000 7f16e9400640 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4e590 0x55d135a561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.624+0000 7f16e9e00640 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 0x55d135a55c60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f16dc004970 tx=0x7f16dc0049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca2ce7436b6c135 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.624+0000 7f16cb400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.624+0000 7f16cb400640 1 -- 172.21.15.2:0/3607646012 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc029070 con 0x55d135a4dc70 2024-09-10T08:19:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.624+0000 7f16cb400640 1 -- 172.21.15.2:0/3607646012 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f16dc019920 con 0x55d135a4dc70 2024-09-10T08:19:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.624+0000 7f16f349c740 1 -- 172.21.15.2:0/3607646012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16dc004c80 con 0x55d135a4dc70 2024-09-10T08:19:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.625+0000 7f16cb400640 1 -- 172.21.15.2:0/3607646012 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16dc035050 con 0x55d135a4dc70 2024-09-10T08:19:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.625+0000 7f16f349c740 10 client.18453 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.625+0000 7f16f349c740 1 -- 172.21.15.2:0/3607646012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d135a67980 con 0x55d135a4dc70 2024-09-10T08:19:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.625+0000 7f16cb400640 1 -- 172.21.15.2:0/3607646012 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16dc004c80 con 0x55d135a4dc70 2024-09-10T08:19:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.625+0000 7f16cb400640 1 -- 172.21.15.2:0/3607646012 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f16dc0235f0 con 0x55d135a4dc70 2024-09-10T08:19:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.625+0000 7f16f349c740 10 client.18453 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.625+0000 7f16f349c740 10 client.18453 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:19:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.625+0000 7f16f349c740 20 client.18453 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.625+0000 7f16f349c740 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55d135a68cd0 0x55d135a6b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.625+0000 7f16f349c740 4 client.18453 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.625+0000 7f16f349c740 1 -- 172.21.15.2:0/3607646012 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1359c1950 con 0x55d135a68cd0 2024-09-10T08:19:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.626+0000 7f16e9400640 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55d135a68cd0 0x55d135a6b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.626+0000 7f16e9400640 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55d135a68cd0 0x55d135a6b0c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d09155eba8fdbf00 server_cookie=f7b6fdd93ecf7d9a in_seq=0 out_seq=0 2024-09-10T08:19:05.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.626+0000 7f16cb400640 10 client.18453 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:19:05.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16cb400640 1 -- 172.21.15.2:0/3607646012 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1359c1950 con 0x55d135a68cd0 2024-09-10T08:19:05.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16cb400640 10 client.18453 handle_command_reply: tid=0 2024-09-10T08:19:05.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16f349c740 1 client.18453 shutdown 2024-09-10T08:19:05.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16f349c740 1 -- 172.21.15.2:0/3607646012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 msgr2=0x55d135a55c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:05.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16f349c740 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 0x55d135a55c60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f16dc004970 tx=0x7f16dc0049a0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16f349c740 1 -- 172.21.15.2:0/3607646012 shutdown_connections 2024-09-10T08:19:05.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16f349c740 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55d135a68cd0 0x55d135a6b0c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16f349c740 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a58e50 0x55d135a566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16f349c740 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4e590 0x55d135a561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16f349c740 1 --2- 172.21.15.2:0/3607646012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 0x55d135a55c60 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16cb400640 0 client.18453 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:19:05.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16f349c740 1 -- 172.21.15.2:0/3607646012 >> 172.21.15.2:0/3607646012 conn(0x55d135a4a200 msgr2=0x55d135a49380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:05.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16f349c740 1 -- 172.21.15.2:0/3607646012 shutdown_connections 2024-09-10T08:19:05.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.627+0000 7f16f349c740 1 -- 172.21.15.2:0/3607646012 wait complete. 2024-09-10T08:19:05.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.628+0000 7f16f349c740 20 client.18453 trim_cache size 0 max 16384 2024-09-10T08:19:05.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.641+0000 7f16f349c740 1 Processor -- start 2024-09-10T08:19:05.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.641+0000 7f16f349c740 1 -- start start 2024-09-10T08:19:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16f349c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 0x55d135a708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16f349c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a70e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16f349c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a58e50 0x55d135a4f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16f349c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1b7db0 con 0x55d135a4dc70 2024-09-10T08:19:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16f349c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1b5680 con 0x55d135a58e50 2024-09-10T08:19:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16f349c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1b34d0 con 0x55d135a4e590 2024-09-10T08:19:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16e9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 0x55d135a708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16e9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 0x55d135a708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52908/0 (socket says 172.21.15.2:52908) 2024-09-10T08:19:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16e9e00640 1 -- 172.21.15.2:0/2388284408 learned_addr learned my addr 172.21.15.2:0/2388284408 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16e9400640 1 --2- 172.21.15.2:0/2388284408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a70e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16ea800640 1 --2- 172.21.15.2:0/2388284408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a58e50 0x55d135a4f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16e9e00640 1 -- 172.21.15.2:0/2388284408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 msgr2=0x55d135a70e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16e9e00640 1 --2- 172.21.15.2:0/2388284408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a70e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16e9e00640 1 -- 172.21.15.2:0/2388284408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a58e50 msgr2=0x55d135a4f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16e9e00640 1 --2- 172.21.15.2:0/2388284408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a58e50 0x55d135a4f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16e9e00640 1 -- 172.21.15.2:0/2388284408 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d135a4a030 con 0x55d135a4dc70 2024-09-10T08:19:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.642+0000 7f16ea800640 1 --2- 172.21.15.2:0/2388284408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a58e50 0x55d135a4f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:19:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.643+0000 7f16e9e00640 1 --2- 172.21.15.2:0/2388284408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 0x55d135a708c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0194e0 tx=0x7f16dc005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72683d0a698d679 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.643+0000 7f16d6600640 1 -- 172.21.15.2:0/2388284408 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc032020 con 0x55d135a4dc70 2024-09-10T08:19:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.643+0000 7f16d6600640 1 -- 172.21.15.2:0/2388284408 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16dc004c80 con 0x55d135a4dc70 2024-09-10T08:19:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.643+0000 7f16d6600640 1 -- 172.21.15.2:0/2388284408 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc029030 con 0x55d135a4dc70 2024-09-10T08:19:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.644+0000 7f16f349c740 1 -- 172.21.15.2:0/2388284408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 msgr2=0x55d135a708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.644+0000 7f16f349c740 1 --2- 172.21.15.2:0/2388284408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 0x55d135a708c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0194e0 tx=0x7f16dc005f70 comp rx=0 tx=0).stop 2024-09-10T08:19:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.644+0000 7f16f349c740 1 -- 172.21.15.2:0/2388284408 shutdown_connections 2024-09-10T08:19:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.644+0000 7f16f349c740 1 --2- 172.21.15.2:0/2388284408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a58e50 0x55d135a4f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.644+0000 7f16f349c740 1 --2- 172.21.15.2:0/2388284408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a70e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.644+0000 7f16f349c740 1 --2- 172.21.15.2:0/2388284408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a4dc70 0x55d135a708c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.644+0000 7f16f349c740 1 -- 172.21.15.2:0/2388284408 >> 172.21.15.2:0/2388284408 conn(0x55d135a4a200 msgr2=0x55d135a57ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.644+0000 7f16f349c740 1 -- 172.21.15.2:0/2388284408 shutdown_connections 2024-09-10T08:19:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.644+0000 7f16f349c740 1 -- 172.21.15.2:0/2388284408 wait complete. 2024-09-10T08:19:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.644+0000 7f16f349c740 1 -- start start 2024-09-10T08:19:05.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.644+0000 7f16f349c740 1 Processor -- start 2024-09-10T08:19:05.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.645+0000 7f16f349c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4dc70 0x55d135a61120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.645+0000 7f16f349c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a61660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.645+0000 7f16f349c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a58e50 0x55d135a5c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.645+0000 7f16f349c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1b3b40 con 0x55d135a58e50 2024-09-10T08:19:05.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.645+0000 7f16f349c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1b4fb0 con 0x55d135a4dc70 2024-09-10T08:19:05.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.645+0000 7f16f349c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1b4210 con 0x55d135a4e590 2024-09-10T08:19:05.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.645+0000 7f16f349c740 10 client.0 _ll_register_callbacks cb 0x7f16d80d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:05.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.645+0000 7f16ea800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a58e50 0x55d135a5c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.645+0000 7f16e9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4dc70 0x55d135a61120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.645+0000 7f16e9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a61660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.645+0000 7f16e9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4dc70 0x55d135a61120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52972/0 (socket says 172.21.15.2:52972) 2024-09-10T08:19:05.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.645+0000 7f16e9e00640 1 -- 172.21.15.2:0/142984321 learned_addr learned my addr 172.21.15.2:0/142984321 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:05.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.646+0000 7f16e9400640 1 -- 172.21.15.2:0/142984321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4dc70 msgr2=0x55d135a61120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:05.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.646+0000 7f16e9400640 1 --2- 172.21.15.2:0/142984321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4dc70 0x55d135a61120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.646+0000 7f16e9400640 1 -- 172.21.15.2:0/142984321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a58e50 msgr2=0x55d135a5c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:05.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.646+0000 7f16e9400640 1 --2- 172.21.15.2:0/142984321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a58e50 0x55d135a5c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.646+0000 7f16e9400640 1 -- 172.21.15.2:0/142984321 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d135a45770 con 0x55d135a4e590 2024-09-10T08:19:05.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.646+0000 7f16e9400640 1 --2- 172.21.15.2:0/142984321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a61660 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55d135a4b580 tx=0x7f16d8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30c80c1a7805c47f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:05.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.646+0000 7f16cb400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:19:05.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.646+0000 7f16cb400640 1 -- 172.21.15.2:0/142984321 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d809b5a0 con 0x55d135a4e590 2024-09-10T08:19:05.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.646+0000 7f16cb400640 1 -- 172.21.15.2:0/142984321 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f16d80ccff0 con 0x55d135a4e590 2024-09-10T08:19:05.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.646+0000 7f16f349c740 1 -- 172.21.15.2:0/142984321 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16dc004c80 con 0x55d135a4e590 2024-09-10T08:19:05.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.646+0000 7f16cb400640 1 -- 172.21.15.2:0/142984321 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16d809c340 con 0x55d135a4e590 2024-09-10T08:19:05.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.647+0000 7f16f349c740 10 client.28414 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:05.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.647+0000 7f16f349c740 1 -- 172.21.15.2:0/142984321 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d135ad32f0 con 0x55d135a4e590 2024-09-10T08:19:05.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.647+0000 7f16cb400640 1 -- 172.21.15.2:0/142984321 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16dc004c80 con 0x55d135a4e590 2024-09-10T08:19:05.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.647+0000 7f16cb400640 1 -- 172.21.15.2:0/142984321 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f16d809b090 con 0x55d135a4e590 2024-09-10T08:19:05.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.647+0000 7f16f349c740 10 client.28414 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:05.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.647+0000 7f16f349c740 10 client.28414 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:19:05.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.647+0000 7f16f349c740 20 client.28414 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:05.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.647+0000 7f16f349c740 1 --2- 172.21.15.2:0/142984321 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55d135ad3a70 0x55d135ad5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.647+0000 7f16f349c740 4 client.28414 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:19:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.647+0000 7f16f349c740 1 -- 172.21.15.2:0/142984321 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1359bbe60 con 0x55d135ad3a70 2024-09-10T08:19:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.647+0000 7f16e9e00640 1 --2- 172.21.15.2:0/142984321 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55d135ad3a70 0x55d135ad5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.648+0000 7f16e9e00640 1 --2- 172.21.15.2:0/142984321 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55d135ad3a70 0x55d135ad5e90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d349791c2c39bf4f server_cookie=c244ba8a5355e1a6 in_seq=0 out_seq=0 2024-09-10T08:19:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.648+0000 7f16cb400640 10 client.28414 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:19:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.648+0000 7f16cb400640 1 -- 172.21.15.2:0/142984321 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1359bbe60 con 0x55d135ad3a70 2024-09-10T08:19:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.648+0000 7f16cb400640 10 client.28414 handle_command_reply: tid=0 2024-09-10T08:19:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.648+0000 7f16f349c740 1 client.28414 shutdown 2024-09-10T08:19:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.648+0000 7f16f349c740 1 -- 172.21.15.2:0/142984321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 msgr2=0x55d135a61660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.648+0000 7f16f349c740 1 --2- 172.21.15.2:0/142984321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a61660 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55d135a4b580 tx=0x7f16d8004a40 comp rx=0 tx=0).stop 2024-09-10T08:19:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16f349c740 1 -- 172.21.15.2:0/142984321 shutdown_connections 2024-09-10T08:19:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16f349c740 1 --2- 172.21.15.2:0/142984321 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55d135ad3a70 0x55d135ad5e90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16f349c740 1 --2- 172.21.15.2:0/142984321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d135a58e50 0x55d135a5c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16f349c740 1 --2- 172.21.15.2:0/142984321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d135a4e590 0x55d135a61660 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16f349c740 1 --2- 172.21.15.2:0/142984321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d135a4dc70 0x55d135a61120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16f349c740 1 -- 172.21.15.2:0/142984321 >> 172.21.15.2:0/142984321 conn(0x55d135a4a200 msgr2=0x55d135a48b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:05.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16cb400640 0 client.28414 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:19:05.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16f349c740 1 -- 172.21.15.2:0/142984321 shutdown_connections 2024-09-10T08:19:05.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16f349c740 1 -- 172.21.15.2:0/142984321 wait complete. 2024-09-10T08:19:05.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16f349c740 20 client.28414 trim_cache size 0 max 16384 2024-09-10T08:19:05.655 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:19:05.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16f0a00640 1 -- 172.21.15.2:0/4184297136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16d00a2b80 msgr2=0x7f16d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:05.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16f0a00640 1 --2- 172.21.15.2:0/4184297136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16d00a2b80 0x7f16d00a5040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006010 tx=0x7f16e4004a80 comp rx=0 tx=0).stop 2024-09-10T08:19:05.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16f0a00640 1 -- 172.21.15.2:0/4184297136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec077040 msgr2=0x7f16ec1a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:05.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.649+0000 7f16f0a00640 1 --2- 172.21.15.2:0/4184297136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec077040 0x7f16ec1a4c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f16d800e9f0 tx=0x7f16d800edf0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.650+0000 7f16f0a00640 1 -- 172.21.15.2:0/4184297136 shutdown_connections 2024-09-10T08:19:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.650+0000 7f16f0a00640 1 --2- 172.21.15.2:0/4184297136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16ec112a30 0x7f16ec19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.650+0000 7f16f0a00640 1 --2- 172.21.15.2:0/4184297136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16d00a2b80 0x7f16d00a5040 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.650+0000 7f16f0a00640 1 --2- 172.21.15.2:0/4184297136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec077040 0x7f16ec1a4c50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.650+0000 7f16f0a00640 1 --2- 172.21.15.2:0/4184297136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16ec076720 0x7f16ec1a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.650+0000 7f16f0a00640 1 -- 172.21.15.2:0/4184297136 >> 172.21.15.2:0/4184297136 conn(0x7f16ec0ffa90 msgr2=0x7f16ec101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.650+0000 7f16f0a00640 1 -- 172.21.15.2:0/4184297136 shutdown_connections 2024-09-10T08:19:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:05.650+0000 7f16f0a00640 1 -- 172.21.15.2:0/4184297136 wait complete. 2024-09-10T08:19:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:05 smithi002 ceph-mon[30563]: pgmap v715: 129 pgs: 129 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T08:19:05.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:05 smithi155 ceph-mon[35405]: pgmap v715: 129 pgs: 129 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T08:19:05.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:05 smithi107 ceph-mon[35048]: pgmap v715: 129 pgs: 129 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T08:19:05.951 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.2 2024-09-10T08:19:05.952 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:19:06.696 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:19:06.784 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:07.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.554+0000 7ff78c800640 1 -- 172.21.15.2:0/1789800725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff784104010 msgr2=0x7ff7841043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:07.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.554+0000 7ff78c800640 1 --2- 172.21.15.2:0/1789800725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff784104010 0x7ff7841043f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff778009900 tx=0x7ff77802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:19:07.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.554+0000 7ff78c800640 1 -- 172.21.15.2:0/1789800725 shutdown_connections 2024-09-10T08:19:07.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.554+0000 7ff78c800640 1 --2- 172.21.15.2:0/1789800725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff784109140 0x7ff784110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:07.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.554+0000 7ff78c800640 1 --2- 172.21.15.2:0/1789800725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7841049c0 0x7ff784108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:07.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.554+0000 7ff78c800640 1 --2- 172.21.15.2:0/1789800725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff784104010 0x7ff7841043f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:07.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.554+0000 7ff78c800640 1 -- 172.21.15.2:0/1789800725 >> 172.21.15.2:0/1789800725 conn(0x7ff7840ffa50 msgr2=0x7ff784101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:07.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.554+0000 7ff78c800640 1 -- 172.21.15.2:0/1789800725 shutdown_connections 2024-09-10T08:19:07.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.554+0000 7ff78c800640 1 -- 172.21.15.2:0/1789800725 wait complete. 2024-09-10T08:19:07.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.555+0000 7ff78c800640 1 Processor -- start 2024-09-10T08:19:07.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.555+0000 7ff78c800640 1 -- start start 2024-09-10T08:19:07.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.556+0000 7ff78c800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff784104010 0x7ff78419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:07.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.556+0000 7ff78c800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7841049c0 0x7ff78419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:07.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.556+0000 7ff78c800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff784109140 0x7ff78419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:07.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.556+0000 7ff78c800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841141b0 con 0x7ff784109140 2024-09-10T08:19:07.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.556+0000 7ff78c800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff784114030 con 0x7ff7841049c0 2024-09-10T08:19:07.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.556+0000 7ff78c800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff784114330 con 0x7ff784104010 2024-09-10T08:19:07.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.556+0000 7ff78a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff784109140 0x7ff78419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:07.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.556+0000 7ff78a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff784109140 0x7ff78419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35582/0 (socket says 172.21.15.2:35582) 2024-09-10T08:19:07.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.556+0000 7ff78a800640 1 -- 172.21.15.2:0/358351721 learned_addr learned my addr 172.21.15.2:0/358351721 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:07.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.556+0000 7ff789e00640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff784104010 0x7ff78419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:07.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.556+0000 7ff789400640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7841049c0 0x7ff78419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:07.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.557+0000 7ff78a800640 1 -- 172.21.15.2:0/358351721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff784104010 msgr2=0x7ff78419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:07.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.557+0000 7ff78a800640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff784104010 0x7ff78419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:07.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.557+0000 7ff78a800640 1 -- 172.21.15.2:0/358351721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7841049c0 msgr2=0x7ff78419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:07.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.557+0000 7ff78a800640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7841049c0 0x7ff78419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:07.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.557+0000 7ff78a800640 1 -- 172.21.15.2:0/358351721 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78419f5e0 con 0x7ff784109140 2024-09-10T08:19:07.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.557+0000 7ff78a800640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff784109140 0x7ff78419ef00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff78000eb10 tx=0x7ff78000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:07.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.557+0000 7ff77ea00640 1 -- 172.21.15.2:0/358351721 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780004b20 con 0x7ff784109140 2024-09-10T08:19:07.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.557+0000 7ff77ea00640 1 -- 172.21.15.2:0/358351721 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff780004cc0 con 0x7ff784109140 2024-09-10T08:19:07.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.557+0000 7ff77ea00640 1 -- 172.21.15.2:0/358351721 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780010760 con 0x7ff784109140 2024-09-10T08:19:07.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.557+0000 7ff78c800640 1 -- 172.21.15.2:0/358351721 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78419f8d0 con 0x7ff784109140 2024-09-10T08:19:07.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.558+0000 7ff78c800640 1 -- 172.21.15.2:0/358351721 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7841a71b0 con 0x7ff784109140 2024-09-10T08:19:07.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.559+0000 7ff78c800640 1 -- 172.21.15.2:0/358351721 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7841043f0 con 0x7ff784109140 2024-09-10T08:19:07.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.560+0000 7ff77ea00640 1 -- 172.21.15.2:0/358351721 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff780005020 con 0x7ff784109140 2024-09-10T08:19:07.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.563+0000 7ff77ea00640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff76c0a2b80 0x7ff76c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:07.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.563+0000 7ff77ea00640 1 -- 172.21.15.2:0/358351721 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff7800cf980 con 0x7ff784109140 2024-09-10T08:19:07.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.563+0000 7ff789e00640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff76c0a2b80 0x7ff76c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:07.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.564+0000 7ff789e00640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff76c0a2b80 0x7ff76c0a5040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff778000c00 tx=0x7ff778005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:07.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.567+0000 7ff77ea00640 1 -- 172.21.15.2:0/358351721 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff78009c490 con 0x7ff784109140 2024-09-10T08:19:07.705 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:07.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:07 smithi002 ceph-mon[30563]: pgmap v716: 129 pgs: 129 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:19:07.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.818+0000 7ff78c800640 1 -- 172.21.15.2:0/358351721 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff78402d7c0 con 0x7ff784109140 2024-09-10T08:19:07.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.818+0000 7ff77ea00640 1 -- 172.21.15.2:0/358351721 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7ff78009bd90 con 0x7ff784109140 2024-09-10T08:19:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:19:07.820 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:19:07.820 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:19:07.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.821+0000 7ff78c800640 1 -- 172.21.15.2:0/358351721 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff76c0a2b80 msgr2=0x7ff76c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:07.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.821+0000 7ff78c800640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff76c0a2b80 0x7ff76c0a5040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff778000c00 tx=0x7ff778005b50 comp rx=0 tx=0).stop 2024-09-10T08:19:07.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.821+0000 7ff78c800640 1 -- 172.21.15.2:0/358351721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff784109140 msgr2=0x7ff78419ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:07.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.821+0000 7ff78c800640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff784109140 0x7ff78419ef00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff78000eb10 tx=0x7ff78000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:19:07.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.821+0000 7ff78c800640 1 -- 172.21.15.2:0/358351721 shutdown_connections 2024-09-10T08:19:07.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.821+0000 7ff78c800640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff784109140 0x7ff78419ef00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:07.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.821+0000 7ff78c800640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7841049c0 0x7ff78419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:07.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.821+0000 7ff78c800640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff76c0a2b80 0x7ff76c0a5040 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:07.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.821+0000 7ff78c800640 1 --2- 172.21.15.2:0/358351721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff784104010 0x7ff78419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:07.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.821+0000 7ff78c800640 1 -- 172.21.15.2:0/358351721 >> 172.21.15.2:0/358351721 conn(0x7ff7840ffa50 msgr2=0x7ff784100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:07.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.821+0000 7ff78c800640 1 -- 172.21.15.2:0/358351721 shutdown_connections 2024-09-10T08:19:07.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:07.821+0000 7ff78c800640 1 -- 172.21.15.2:0/358351721 wait complete. 2024-09-10T08:19:07.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:07 smithi155 ceph-mon[35405]: pgmap v716: 129 pgs: 129 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:19:07.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:07 smithi107 ceph-mon[35048]: pgmap v716: 129 pgs: 129 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:19:08.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.454+0000 7fbf2ea00640 1 -- 172.21.15.2:0/958407835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf281049c0 msgr2=0x7fbf28108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:08.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.454+0000 7fbf2ea00640 1 --2- 172.21.15.2:0/958407835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf281049c0 0x7fbf28108a10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009900 tx=0x7fbf1002a410 comp rx=0 tx=0).stop 2024-09-10T08:19:08.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.454+0000 7fbf2ea00640 1 -- 172.21.15.2:0/958407835 shutdown_connections 2024-09-10T08:19:08.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.454+0000 7fbf2ea00640 1 --2- 172.21.15.2:0/958407835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf28109140 0x7fbf28110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.454+0000 7fbf2ea00640 1 --2- 172.21.15.2:0/958407835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf281049c0 0x7fbf28108a10 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.454+0000 7fbf2ea00640 1 --2- 172.21.15.2:0/958407835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf28104010 0x7fbf281043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.454+0000 7fbf2ea00640 1 -- 172.21.15.2:0/958407835 >> 172.21.15.2:0/958407835 conn(0x7fbf280ffa50 msgr2=0x7fbf28101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:08.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.454+0000 7fbf2ea00640 1 -- 172.21.15.2:0/958407835 shutdown_connections 2024-09-10T08:19:08.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.454+0000 7fbf2ea00640 1 -- 172.21.15.2:0/958407835 wait complete. 2024-09-10T08:19:08.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.455+0000 7fbf2ea00640 1 Processor -- start 2024-09-10T08:19:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.455+0000 7fbf2ea00640 1 -- start start 2024-09-10T08:19:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf2ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf28104010 0x7fbf2819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf2ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf281049c0 0x7fbf2819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf2ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf28109140 0x7fbf2819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf2ea00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbf28114160 con 0x7fbf281049c0 2024-09-10T08:19:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf2ea00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf28113fe0 con 0x7fbf28109140 2024-09-10T08:19:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf2ea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281142e0 con 0x7fbf28104010 2024-09-10T08:19:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf27e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf28104010 0x7fbf2819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf281049c0 0x7fbf2819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf27e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf28104010 0x7fbf2819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:37954/0 (socket says 172.21.15.2:37954) 2024-09-10T08:19:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf27e00640 1 -- 172.21.15.2:0/3634837868 learned_addr learned my addr 172.21.15.2:0/3634837868 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:08.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf2ca00640 1 --2- 172.21.15.2:0/3634837868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf28109140 0x7fbf2819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf27e00640 1 -- 172.21.15.2:0/3634837868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf28109140 msgr2=0x7fbf2819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:08.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf27e00640 1 --2- 172.21.15.2:0/3634837868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf28109140 0x7fbf2819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf27e00640 1 -- 172.21.15.2:0/3634837868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf281049c0 msgr2=0x7fbf2819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:08.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf27e00640 1 --2- 172.21.15.2:0/3634837868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf281049c0 0x7fbf2819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.456+0000 7fbf27e00640 1 -- 172.21.15.2:0/3634837868 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2819f610 con 0x7fbf28104010 2024-09-10T08:19:08.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.457+0000 7fbf27e00640 1 --2- 172.21.15.2:0/3634837868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf28104010 0x7fbf2819a660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800e9f0 tx=0x7fbf1800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:08.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.457+0000 7fbf25600640 1 -- 172.21.15.2:0/3634837868 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800cd80 con 0x7fbf28104010 2024-09-10T08:19:08.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.457+0000 7fbf2ea00640 1 -- 172.21.15.2:0/3634837868 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2819f900 con 0x7fbf28104010 2024-09-10T08:19:08.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.457+0000 7fbf25600640 1 -- 172.21.15.2:0/3634837868 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf18005500 con 0x7fbf28104010 2024-09-10T08:19:08.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.457+0000 7fbf25600640 1 -- 172.21.15.2:0/3634837868 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18010690 con 0x7fbf28104010 2024-09-10T08:19:08.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.457+0000 7fbf2ea00640 1 -- 172.21.15.2:0/3634837868 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf281a71e0 con 0x7fbf28104010 2024-09-10T08:19:08.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.460+0000 7fbf25600640 1 -- 172.21.15.2:0/3634837868 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbf180046c0 con 0x7fbf28104010 2024-09-10T08:19:08.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.460+0000 7fbf315be740 1 Processor -- start 2024-09-10T08:19:08.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.460+0000 7fbf315be740 1 -- start start 2024-09-10T08:19:08.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.461+0000 7fbf315be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f6fc70 0x556101f70050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.461+0000 7fbf315be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f70590 0x556101f7a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.461+0000 7fbf315be740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f7ae50 0x556101f7d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.461+0000 7fbf315be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556101ed63e0 con 0x556101f6fc70 2024-09-10T08:19:08.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.461+0000 7fbf315be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556101e7dd80 con 0x556101f70590 2024-09-10T08:19:08.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.461+0000 7fbf315be740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556101ed23a0 con 0x556101f7ae50 2024-09-10T08:19:08.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.461+0000 7fbf2ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f6fc70 0x556101f70050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.461+0000 7fbf27e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f70590 0x556101f7a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.461+0000 7fbf2ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f6fc70 0x556101f70050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35604/0 (socket says 172.21.15.2:35604) 2024-09-10T08:19:08.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.461+0000 7fbf2ca00640 1 -- 172.21.15.2:0/747186067 learned_addr learned my addr 172.21.15.2:0/747186067 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:08.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.461+0000 7fbf1fe00640 1 --2- 172.21.15.2:0/747186067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f7ae50 0x556101f7d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.462+0000 7fbf27e00640 1 -- 172.21.15.2:0/747186067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f7ae50 msgr2=0x556101f7d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:08.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.462+0000 7fbf27e00640 1 --2- 172.21.15.2:0/747186067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f7ae50 0x556101f7d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.462+0000 7fbf27e00640 1 -- 172.21.15.2:0/747186067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f6fc70 msgr2=0x556101f70050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:08.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.462+0000 7fbf27e00640 1 --2- 172.21.15.2:0/747186067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f6fc70 0x556101f70050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.462+0000 7fbf27e00640 1 -- 172.21.15.2:0/747186067 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556101e9cae0 con 0x556101f70590 2024-09-10T08:19:08.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.462+0000 7fbf2ca00640 1 --2- 172.21.15.2:0/747186067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f6fc70 0x556101f70050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:08.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.462+0000 7fbf27e00640 1 --2- 172.21.15.2:0/747186067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f70590 0x556101f7a910 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbf180cce60 tx=0x7fbf180d5b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96a3030075aa85b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:08.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.462+0000 7fbf25600640 1 --2- 172.21.15.2:0/3634837868 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbf0c0a2c00 0x7fbf0c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.462+0000 7fbf25600640 1 -- 172.21.15.2:0/3634837868 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbf180cf810 con 0x7fbf28104010 2024-09-10T08:19:08.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.462+0000 7fbf2ca00640 1 --2- 172.21.15.2:0/3634837868 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbf0c0a2c00 0x7fbf0c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.463+0000 7fbf1d200640 1 -- 172.21.15.2:0/747186067 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf180de070 con 0x556101f70590 2024-09-10T08:19:08.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.463+0000 7fbf1d200640 1 -- 172.21.15.2:0/747186067 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf1809c320 con 0x556101f70590 2024-09-10T08:19:08.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.463+0000 7fbf1d200640 1 -- 172.21.15.2:0/747186067 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1809b760 con 0x556101f70590 2024-09-10T08:19:08.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.463+0000 7fbf315be740 1 -- 172.21.15.2:0/747186067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f70590 msgr2=0x556101f7a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:08.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.463+0000 7fbf2ca00640 1 --2- 172.21.15.2:0/3634837868 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbf0c0a2c00 0x7fbf0c0a50c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009fd0 tx=0x7fbf20009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:08.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.463+0000 7fbf315be740 1 --2- 172.21.15.2:0/747186067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f70590 0x556101f7a910 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbf180cce60 tx=0x7fbf180d5b20 comp rx=0 tx=0).stop 2024-09-10T08:19:08.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.464+0000 7fbf315be740 1 -- 172.21.15.2:0/747186067 shutdown_connections 2024-09-10T08:19:08.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.464+0000 7fbf315be740 1 --2- 172.21.15.2:0/747186067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f7ae50 0x556101f7d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.464+0000 7fbf315be740 1 --2- 172.21.15.2:0/747186067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f70590 0x556101f7a910 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.464+0000 7fbf315be740 1 --2- 172.21.15.2:0/747186067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f6fc70 0x556101f70050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.464+0000 7fbf315be740 1 -- 172.21.15.2:0/747186067 >> 172.21.15.2:0/747186067 conn(0x556101f6c200 msgr2=0x556101f6c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:08.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.464+0000 7fbf315be740 1 -- 172.21.15.2:0/747186067 shutdown_connections 2024-09-10T08:19:08.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.464+0000 7fbf315be740 1 -- 172.21.15.2:0/747186067 wait complete. 2024-09-10T08:19:08.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.464+0000 7fbf315be740 1 -- start start 2024-09-10T08:19:08.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.464+0000 7fbf315be740 1 Processor -- start 2024-09-10T08:19:08.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.465+0000 7fbf315be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f77c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.465+0000 7fbf315be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f70590 0x556101f781d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.465+0000 7fbf315be740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f7ae50 0x556101f78710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.465+0000 7fbf315be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556101f7da20 con 0x556101f70590 2024-09-10T08:19:08.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.465+0000 7fbf315be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556101e910f0 con 0x556101f6fc70 2024-09-10T08:19:08.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.465+0000 7fbf315be740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556101f7dba0 con 0x556101f7ae50 2024-09-10T08:19:08.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.465+0000 7fbf315be740 10 client.0 _ll_register_callbacks cb 0x556101e78bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:08.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.465+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f70590 0x556101f781d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.465+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f70590 0x556101f781d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35612/0 (socket says 172.21.15.2:35612) 2024-09-10T08:19:08.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.465+0000 7fbf1fe00640 1 -- 172.21.15.2:0/1253091382 learned_addr learned my addr 172.21.15.2:0/1253091382 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:08.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.465+0000 7fbf27e00640 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f77c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.465+0000 7fbf2ca00640 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f7ae50 0x556101f78710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.466+0000 7fbf1fe00640 1 -- 172.21.15.2:0/1253091382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f7ae50 msgr2=0x556101f78710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:08.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.466+0000 7fbf1fe00640 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f7ae50 0x556101f78710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.466+0000 7fbf1fe00640 1 -- 172.21.15.2:0/1253091382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 msgr2=0x556101f77c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:08.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.466+0000 7fbf1fe00640 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f77c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.466+0000 7fbf1fe00640 1 -- 172.21.15.2:0/1253091382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556101c473c0 con 0x556101f70590 2024-09-10T08:19:08.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.466+0000 7fbf27e00640 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f77c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:08.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.466+0000 7fbf1fe00640 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f70590 0x556101f781d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbf1002a930 tx=0x7fbf10031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82eccf9206ae080b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:08.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.466+0000 7fbf15600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:08.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.466+0000 7fbf15600640 1 -- 172.21.15.2:0/1253091382 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10035040 con 0x556101f70590 2024-09-10T08:19:08.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.466+0000 7fbf15600640 1 -- 172.21.15.2:0/1253091382 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbf10004240 con 0x556101f70590 2024-09-10T08:19:08.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.466+0000 7fbf315be740 1 -- 172.21.15.2:0/1253091382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf1809c320 con 0x556101f70590 2024-09-10T08:19:08.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.467+0000 7fbf15600640 1 -- 172.21.15.2:0/1253091382 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf10004440 con 0x556101f70590 2024-09-10T08:19:08.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.467+0000 7fbf315be740 10 client.18477 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:08.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.467+0000 7fbf315be740 1 -- 172.21.15.2:0/1253091382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556101f899b0 con 0x556101f70590 2024-09-10T08:19:08.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.467+0000 7fbf15600640 1 -- 172.21.15.2:0/1253091382 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf1809c320 con 0x556101f70590 2024-09-10T08:19:08.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.467+0000 7fbf15600640 1 -- 172.21.15.2:0/1253091382 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf1003d6f0 con 0x556101f70590 2024-09-10T08:19:08.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.467+0000 7fbf315be740 10 client.18477 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:08.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.467+0000 7fbf315be740 10 client.18477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:08.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.467+0000 7fbf315be740 20 client.18477 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:08.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.467+0000 7fbf315be740 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556101f8ae70 0x556101f8d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.467+0000 7fbf315be740 4 client.18477 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:08.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.467+0000 7fbf315be740 1 -- 172.21.15.2:0/1253091382 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556101ee3950 con 0x556101f8ae70 2024-09-10T08:19:08.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.468+0000 7fbf27e00640 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556101f8ae70 0x556101f8d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.468+0000 7fbf27e00640 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556101f8ae70 0x556101f8d2e0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6249ed4c3ef867 server_cookie=59d5395213faf7b in_seq=0 out_seq=0 2024-09-10T08:19:08.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.468+0000 7fbf15600640 10 client.18477 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:08.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf15600640 1 -- 172.21.15.2:0/1253091382 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556101ee3950 con 0x556101f8ae70 2024-09-10T08:19:08.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf15600640 10 client.18477 handle_command_reply: tid=0 2024-09-10T08:19:08.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf315be740 1 client.18477 shutdown 2024-09-10T08:19:08.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf315be740 1 -- 172.21.15.2:0/1253091382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f70590 msgr2=0x556101f781d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:08.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf315be740 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f70590 0x556101f781d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbf1002a930 tx=0x7fbf10031850 comp rx=0 tx=0).stop 2024-09-10T08:19:08.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf315be740 1 -- 172.21.15.2:0/1253091382 shutdown_connections 2024-09-10T08:19:08.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf315be740 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556101f8ae70 0x556101f8d2e0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf315be740 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f7ae50 0x556101f78710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf315be740 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f70590 0x556101f781d0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf315be740 1 --2- 172.21.15.2:0/1253091382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f77c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf315be740 1 -- 172.21.15.2:0/1253091382 >> 172.21.15.2:0/1253091382 conn(0x556101f6c200 msgr2=0x556101f71f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:08.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf15600640 0 client.18477 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:08.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf315be740 1 -- 172.21.15.2:0/1253091382 shutdown_connections 2024-09-10T08:19:08.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.469+0000 7fbf315be740 1 -- 172.21.15.2:0/1253091382 wait complete. 2024-09-10T08:19:08.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.470+0000 7fbf315be740 20 client.18477 trim_cache size 0 max 16384 2024-09-10T08:19:08.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.483+0000 7fbf315be740 1 Processor -- start 2024-09-10T08:19:08.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.483+0000 7fbf315be740 1 -- start start 2024-09-10T08:19:08.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.483+0000 7fbf315be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f92bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.483+0000 7fbf315be740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f70590 0x556101f930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf315be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f7ae50 0x556101f74ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf315be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281b38f0 con 0x556101f7ae50 2024-09-10T08:19:08.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf315be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281b11c0 con 0x556101f6fc70 2024-09-10T08:19:08.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf315be740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281af010 con 0x556101f70590 2024-09-10T08:19:08.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f70590 0x556101f930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf27e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f92bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f70590 0x556101f930f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:37978/0 (socket says 172.21.15.2:37978) 2024-09-10T08:19:08.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf1fe00640 1 -- 172.21.15.2:0/761059432 learned_addr learned my addr 172.21.15.2:0/761059432 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:08.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf2ca00640 1 --2- 172.21.15.2:0/761059432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f7ae50 0x556101f74ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf27e00640 1 -- 172.21.15.2:0/761059432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f70590 msgr2=0x556101f930f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:08.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf27e00640 1 --2- 172.21.15.2:0/761059432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f70590 0x556101f930f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf27e00640 1 -- 172.21.15.2:0/761059432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f7ae50 msgr2=0x556101f74ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:08.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf27e00640 1 --2- 172.21.15.2:0/761059432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f7ae50 0x556101f74ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf27e00640 1 -- 172.21.15.2:0/761059432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556101f6c030 con 0x556101f6fc70 2024-09-10T08:19:08.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.484+0000 7fbf1fe00640 1 --2- 172.21.15.2:0/761059432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f70590 0x556101f930f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:08.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.485+0000 7fbf27e00640 1 --2- 172.21.15.2:0/761059432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f92bb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbf1809bb60 tx=0x7fbf180d76f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fa2c49fafac816b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:08.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.485+0000 7fbf17e00640 1 -- 172.21.15.2:0/761059432 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf180de040 con 0x556101f6fc70 2024-09-10T08:19:08.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.485+0000 7fbf17e00640 1 -- 172.21.15.2:0/761059432 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf1809c320 con 0x556101f6fc70 2024-09-10T08:19:08.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.485+0000 7fbf17e00640 1 -- 172.21.15.2:0/761059432 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1809b580 con 0x556101f6fc70 2024-09-10T08:19:08.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.485+0000 7fbf315be740 1 -- 172.21.15.2:0/761059432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 msgr2=0x556101f92bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:08.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.485+0000 7fbf315be740 1 --2- 172.21.15.2:0/761059432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f92bb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbf1809bb60 tx=0x7fbf180d76f0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.486+0000 7fbf315be740 1 -- 172.21.15.2:0/761059432 shutdown_connections 2024-09-10T08:19:08.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.486+0000 7fbf315be740 1 --2- 172.21.15.2:0/761059432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f7ae50 0x556101f74ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.486+0000 7fbf315be740 1 --2- 172.21.15.2:0/761059432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f70590 0x556101f930f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.486+0000 7fbf315be740 1 --2- 172.21.15.2:0/761059432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f92bb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.486+0000 7fbf315be740 1 -- 172.21.15.2:0/761059432 >> 172.21.15.2:0/761059432 conn(0x556101f6c200 msgr2=0x556101f79d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:08.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.486+0000 7fbf315be740 1 -- 172.21.15.2:0/761059432 shutdown_connections 2024-09-10T08:19:08.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.486+0000 7fbf315be740 1 -- 172.21.15.2:0/761059432 wait complete. 2024-09-10T08:19:08.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.486+0000 7fbf315be740 1 -- start start 2024-09-10T08:19:08.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.486+0000 7fbf315be740 1 Processor -- start 2024-09-10T08:19:08.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.487+0000 7fbf315be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.487+0000 7fbf315be740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f70590 0x556101f7e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.487+0000 7fbf315be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f7ae50 0x556101f7eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.487+0000 7fbf315be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281af680 con 0x556101f7ae50 2024-09-10T08:19:08.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.487+0000 7fbf315be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281b0af0 con 0x556101f6fc70 2024-09-10T08:19:08.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.487+0000 7fbf315be740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281afd50 con 0x556101f70590 2024-09-10T08:19:08.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.487+0000 7fbf315be740 10 client.0 _ll_register_callbacks cb 0x7fbf10009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:08.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.487+0000 7fbf2ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f7ae50 0x556101f7eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.487+0000 7fbf2ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f7ae50 0x556101f7eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35642/0 (socket says 172.21.15.2:35642) 2024-09-10T08:19:08.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.487+0000 7fbf2ca00640 1 -- 172.21.15.2:0/3509790010 learned_addr learned my addr 172.21.15.2:0/3509790010 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:08.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.487+0000 7fbf27e00640 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.487+0000 7fbf1fe00640 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f70590 0x556101f7e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.488+0000 7fbf1fe00640 1 -- 172.21.15.2:0/3509790010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 msgr2=0x556101f834e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:08.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.488+0000 7fbf1fe00640 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f834e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.488+0000 7fbf1fe00640 1 -- 172.21.15.2:0/3509790010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f7ae50 msgr2=0x556101f7eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:08.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.488+0000 7fbf1fe00640 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f7ae50 0x556101f7eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.488+0000 7fbf1fe00640 1 -- 172.21.15.2:0/3509790010 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556101f67770 con 0x556101f70590 2024-09-10T08:19:08.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.488+0000 7fbf27e00640 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:08.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.488+0000 7fbf1fe00640 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f70590 0x556101f7e860 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x556101f6ced0 tx=0x7fbf100315b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e6ac9022fbb258c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:08.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.488+0000 7fbf15600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:19:08.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.488+0000 7fbf15600640 1 -- 172.21.15.2:0/3509790010 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10035030 con 0x556101f70590 2024-09-10T08:19:08.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.488+0000 7fbf315be740 1 -- 172.21.15.2:0/3509790010 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf1809c320 con 0x556101f70590 2024-09-10T08:19:08.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.488+0000 7fbf15600640 1 -- 172.21.15.2:0/3509790010 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbf1003d860 con 0x556101f70590 2024-09-10T08:19:08.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.489+0000 7fbf15600640 1 -- 172.21.15.2:0/3509790010 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf10005d50 con 0x556101f70590 2024-09-10T08:19:08.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.489+0000 7fbf315be740 10 client.28438 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:08.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.489+0000 7fbf315be740 1 -- 172.21.15.2:0/3509790010 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556101ff5970 con 0x556101f70590 2024-09-10T08:19:08.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.489+0000 7fbf15600640 1 -- 172.21.15.2:0/3509790010 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf1809c320 con 0x556101f70590 2024-09-10T08:19:08.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.489+0000 7fbf15600640 1 -- 172.21.15.2:0/3509790010 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf100047e0 con 0x556101f70590 2024-09-10T08:19:08.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.489+0000 7fbf315be740 10 client.28438 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:08.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.489+0000 7fbf315be740 10 client.28438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:08.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.489+0000 7fbf315be740 20 client.28438 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:08.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.489+0000 7fbf315be740 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556101ff5f90 0x556101ff83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:08.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.489+0000 7fbf315be740 4 client.28438 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:19:08.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.489+0000 7fbf315be740 1 -- 172.21.15.2:0/3509790010 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556101edde60 con 0x556101ff5f90 2024-09-10T08:19:08.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf27e00640 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556101ff5f90 0x556101ff83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:08.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf27e00640 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556101ff5f90 0x556101ff83b0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef6b0efbfda11090 server_cookie=8aa0d8e7617f8329 in_seq=0 out_seq=0 2024-09-10T08:19:08.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf15600640 10 client.28438 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:08.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf15600640 1 -- 172.21.15.2:0/3509790010 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556101edde60 con 0x556101ff5f90 2024-09-10T08:19:08.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf15600640 10 client.28438 handle_command_reply: tid=0 2024-09-10T08:19:08.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf315be740 1 client.28438 shutdown 2024-09-10T08:19:08.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf315be740 1 -- 172.21.15.2:0/3509790010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f70590 msgr2=0x556101f7e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:08.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf315be740 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f70590 0x556101f7e860 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x556101f6ced0 tx=0x7fbf100315b0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf315be740 1 -- 172.21.15.2:0/3509790010 shutdown_connections 2024-09-10T08:19:08.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf315be740 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556101ff5f90 0x556101ff83b0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf315be740 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556101f7ae50 0x556101f7eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf315be740 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556101f70590 0x556101f7e860 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf315be740 1 --2- 172.21.15.2:0/3509790010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556101f6fc70 0x556101f834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf15600640 0 client.28438 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:08.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.490+0000 7fbf315be740 1 -- 172.21.15.2:0/3509790010 >> 172.21.15.2:0/3509790010 conn(0x556101f6c200 msgr2=0x556101f794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:08.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.491+0000 7fbf315be740 1 -- 172.21.15.2:0/3509790010 shutdown_connections 2024-09-10T08:19:08.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.491+0000 7fbf315be740 1 -- 172.21.15.2:0/3509790010 wait complete. 2024-09-10T08:19:08.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.491+0000 7fbf315be740 20 client.28438 trim_cache size 0 max 16384 2024-09-10T08:19:08.495 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:19:08.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.491+0000 7fbf2ea00640 1 -- 172.21.15.2:0/3634837868 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbf0c0a2c00 msgr2=0x7fbf0c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:08.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.491+0000 7fbf2ea00640 1 --2- 172.21.15.2:0/3634837868 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbf0c0a2c00 0x7fbf0c0a50c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009fd0 tx=0x7fbf20009500 comp rx=0 tx=0).stop 2024-09-10T08:19:08.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.491+0000 7fbf2ea00640 1 -- 172.21.15.2:0/3634837868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf28104010 msgr2=0x7fbf2819a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:08.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.491+0000 7fbf2ea00640 1 --2- 172.21.15.2:0/3634837868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf28104010 0x7fbf2819a660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800e9f0 tx=0x7fbf1800edf0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.491+0000 7fbf2ea00640 1 -- 172.21.15.2:0/3634837868 shutdown_connections 2024-09-10T08:19:08.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.491+0000 7fbf2ea00640 1 --2- 172.21.15.2:0/3634837868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf28109140 0x7fbf2819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.491+0000 7fbf2ea00640 1 --2- 172.21.15.2:0/3634837868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf281049c0 0x7fbf2819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.491+0000 7fbf2ea00640 1 --2- 172.21.15.2:0/3634837868 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbf0c0a2c00 0x7fbf0c0a50c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.491+0000 7fbf2ea00640 1 --2- 172.21.15.2:0/3634837868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf28104010 0x7fbf2819a660 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:08.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.491+0000 7fbf2ea00640 1 -- 172.21.15.2:0/3634837868 >> 172.21.15.2:0/3634837868 conn(0x7fbf280ffa50 msgr2=0x7fbf28100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:08.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.492+0000 7fbf2ea00640 1 -- 172.21.15.2:0/3634837868 shutdown_connections 2024-09-10T08:19:08.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:08.492+0000 7fbf2ea00640 1 -- 172.21.15.2:0/3634837868 wait complete. 2024-09-10T08:19:08.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:08 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/358351721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:19:08.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:08 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/358351721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:19:08.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:08 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/358351721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:19:09.261 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0' 2024-09-10T08:19:09.262 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0 2024-09-10T08:19:09.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:09 smithi002 ceph-mon[30563]: pgmap v717: 129 pgs: 129 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:19:09.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:09 smithi155 ceph-mon[35405]: pgmap v717: 129 pgs: 129 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:19:09.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:09 smithi107 ceph-mon[35048]: pgmap v717: 129 pgs: 129 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:19:09.983 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:10.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.542+0000 7fed65c00640 1 -- 172.21.15.2:0/1823219076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed60102360 msgr2=0x7fed6010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:10.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.542+0000 7fed65c00640 1 --2- 172.21.15.2:0/1823219076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed60102360 0x7fed6010e890 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fed58009cd0 tx=0x7fed5802a8f0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.542+0000 7fed65c00640 1 -- 172.21.15.2:0/1823219076 shutdown_connections 2024-09-10T08:19:10.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.542+0000 7fed65c00640 1 --2- 172.21.15.2:0/1823219076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed60102360 0x7fed6010e890 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.542+0000 7fed65c00640 1 --2- 172.21.15.2:0/1823219076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed601019c0 0x7fed60101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.542+0000 7fed65c00640 1 --2- 172.21.15.2:0/1823219076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed601079c0 0x7fed60107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.542+0000 7fed65c00640 1 -- 172.21.15.2:0/1823219076 >> 172.21.15.2:0/1823219076 conn(0x7fed600fd740 msgr2=0x7fed600ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:10.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.542+0000 7fed65c00640 1 -- 172.21.15.2:0/1823219076 shutdown_connections 2024-09-10T08:19:10.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.542+0000 7fed65c00640 1 -- 172.21.15.2:0/1823219076 wait complete. 2024-09-10T08:19:10.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.543+0000 7fed65c00640 1 Processor -- start 2024-09-10T08:19:10.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.543+0000 7fed65c00640 1 -- start start 2024-09-10T08:19:10.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.543+0000 7fed65c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed601019c0 0x7fed6019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.543+0000 7fed65c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed60102360 0x7fed6019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.543+0000 7fed65c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed601079c0 0x7fed6019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.543+0000 7fed65c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fed60111dc0 con 0x7fed601079c0 2024-09-10T08:19:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.543+0000 7fed65c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed60111c40 con 0x7fed601019c0 2024-09-10T08:19:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.543+0000 7fed65c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed60111f40 con 0x7fed60102360 2024-09-10T08:19:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.544+0000 7fed65200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed601079c0 0x7fed6019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.544+0000 7fed65200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed601079c0 0x7fed6019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35666/0 (socket says 172.21.15.2:35666) 2024-09-10T08:19:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.544+0000 7fed65200640 1 -- 172.21.15.2:0/2284650991 learned_addr learned my addr 172.21.15.2:0/2284650991 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.544+0000 7fed5f400640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed60102360 0x7fed6019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.544+0000 7fed5fe00640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed601019c0 0x7fed6019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.544+0000 7fed65200640 1 -- 172.21.15.2:0/2284650991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed60102360 msgr2=0x7fed6019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.544+0000 7fed65200640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed60102360 0x7fed6019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.544+0000 7fed65200640 1 -- 172.21.15.2:0/2284650991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed601019c0 msgr2=0x7fed6019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.544+0000 7fed65200640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed601019c0 0x7fed6019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.544+0000 7fed65200640 1 -- 172.21.15.2:0/2284650991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6019f450 con 0x7fed601079c0 2024-09-10T08:19:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.545+0000 7fed65200640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed601079c0 0x7fed6019ed70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fed58009ca0 tx=0x7fed58030ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.545+0000 7fed5cc00640 1 -- 172.21.15.2:0/2284650991 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58003e00 con 0x7fed601079c0 2024-09-10T08:19:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.545+0000 7fed5cc00640 1 -- 172.21.15.2:0/2284650991 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed58040070 con 0x7fed601079c0 2024-09-10T08:19:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.545+0000 7fed65c00640 1 -- 172.21.15.2:0/2284650991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6019f6e0 con 0x7fed601079c0 2024-09-10T08:19:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.545+0000 7fed5cc00640 1 -- 172.21.15.2:0/2284650991 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58045c20 con 0x7fed601079c0 2024-09-10T08:19:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.545+0000 7fed65c00640 1 -- 172.21.15.2:0/2284650991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed601a7020 con 0x7fed601079c0 2024-09-10T08:19:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.548+0000 7fed5cc00640 1 -- 172.21.15.2:0/2284650991 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fed58004040 con 0x7fed601079c0 2024-09-10T08:19:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.548+0000 7fed69c63740 1 Processor -- start 2024-09-10T08:19:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.548+0000 7fed69c63740 1 -- start start 2024-09-10T08:19:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.549+0000 7fed69c63740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5f560 0x55a92bd5f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.549+0000 7fed69c63740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5fe80 0x55a92bd6a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.549+0000 7fed69c63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd6a740 0x55a92bd6cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.549+0000 7fed69c63740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a92bcc5530 con 0x55a92bd5f560 2024-09-10T08:19:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.549+0000 7fed69c63740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a92bc6c860 con 0x55a92bd5fe80 2024-09-10T08:19:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.549+0000 7fed69c63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a92bcc14f0 con 0x55a92bd6a740 2024-09-10T08:19:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.549+0000 7fed5f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5f560 0x55a92bd5f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.549+0000 7fed5f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5f560 0x55a92bd5f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35682/0 (socket says 172.21.15.2:35682) 2024-09-10T08:19:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.549+0000 7fed5f400640 1 -- 172.21.15.2:0/2404808578 learned_addr learned my addr 172.21.15.2:0/2404808578 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.549+0000 7fed5fe00640 1 --2- 172.21.15.2:0/2404808578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5fe80 0x55a92bd6a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.549+0000 7fed5f400640 1 --2- 172.21.15.2:0/2404808578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd6a740 0x55a92bd6cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.550+0000 7fed5f400640 1 -- 172.21.15.2:0/2404808578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd6a740 msgr2=0x55a92bd6cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.550+0000 7fed5cc00640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fed440a2c00 0x7fed440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.550+0000 7fed5f400640 1 --2- 172.21.15.2:0/2404808578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd6a740 0x55a92bd6cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.550+0000 7fed5f400640 1 -- 172.21.15.2:0/2404808578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5fe80 msgr2=0x55a92bd6a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.550+0000 7fed5f400640 1 --2- 172.21.15.2:0/2404808578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5fe80 0x55a92bd6a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.550+0000 7fed5f400640 1 -- 172.21.15.2:0/2404808578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a92ba36b90 con 0x55a92bd5f560 2024-09-10T08:19:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.550+0000 7fed5cc00640 1 -- 172.21.15.2:0/2284650991 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fed580f0560 con 0x7fed601079c0 2024-09-10T08:19:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.550+0000 7fed5f400640 1 --2- 172.21.15.2:0/2404808578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5f560 0x55a92bd5f940 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0058a0 tx=0x7fed4c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f03ab2f17d5681 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.550+0000 7fed65200640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fed440a2c00 0x7fed440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.550+0000 7fed55600640 1 -- 172.21.15.2:0/2404808578 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c004030 con 0x55a92bd5f560 2024-09-10T08:19:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.550+0000 7fed55600640 1 -- 172.21.15.2:0/2404808578 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed4c0041d0 con 0x55a92bd5f560 2024-09-10T08:19:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.550+0000 7fed55600640 1 -- 172.21.15.2:0/2404808578 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c010660 con 0x55a92bd5f560 2024-09-10T08:19:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.551+0000 7fed69c63740 1 -- 172.21.15.2:0/2404808578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5f560 msgr2=0x55a92bd5f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.551+0000 7fed65200640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fed440a2c00 0x7fed440a50c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fed58006010 tx=0x7fed5802a8f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.551+0000 7fed69c63740 1 --2- 172.21.15.2:0/2404808578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5f560 0x55a92bd5f940 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0058a0 tx=0x7fed4c00b840 comp rx=0 tx=0).stop 2024-09-10T08:19:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.551+0000 7fed69c63740 1 -- 172.21.15.2:0/2404808578 shutdown_connections 2024-09-10T08:19:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.551+0000 7fed69c63740 1 --2- 172.21.15.2:0/2404808578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd6a740 0x55a92bd6cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.551+0000 7fed69c63740 1 --2- 172.21.15.2:0/2404808578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5fe80 0x55a92bd6a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.551+0000 7fed69c63740 1 --2- 172.21.15.2:0/2404808578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5f560 0x55a92bd5f940 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.551+0000 7fed69c63740 1 -- 172.21.15.2:0/2404808578 >> 172.21.15.2:0/2404808578 conn(0x55a92bd5baf0 msgr2=0x55a92bd5bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.551+0000 7fed69c63740 1 -- 172.21.15.2:0/2404808578 shutdown_connections 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.551+0000 7fed69c63740 1 -- 172.21.15.2:0/2404808578 wait complete. 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.552+0000 7fed69c63740 1 -- start start 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.552+0000 7fed69c63740 1 Processor -- start 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed69c63740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 0x55a92bd675d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed69c63740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5fe80 0x55a92bd67b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed69c63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd6a740 0x55a92bd68050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed69c63740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a92bd6d310 con 0x55a92bd5fe80 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed69c63740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a92bc7fe60 con 0x55a92bd5f560 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed69c63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a92bd6d490 con 0x55a92bd6a740 2024-09-10T08:19:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed69c63740 10 client.0 _ll_register_callbacks cb 0x55a92bc84cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed5fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5fe80 0x55a92bd67b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed5fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5fe80 0x55a92bd67b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35686/0 (socket says 172.21.15.2:35686) 2024-09-10T08:19:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed5fe00640 1 -- 172.21.15.2:0/2282753358 learned_addr learned my addr 172.21.15.2:0/2282753358 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed5f400640 1 --2- 172.21.15.2:0/2282753358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd6a740 0x55a92bd68050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed5f400640 1 --2- 172.21.15.2:0/2282753358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 0x55a92bd675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed5fe00640 1 -- 172.21.15.2:0/2282753358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd6a740 msgr2=0x55a92bd68050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed5fe00640 1 --2- 172.21.15.2:0/2282753358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd6a740 0x55a92bd68050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed5fe00640 1 -- 172.21.15.2:0/2282753358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 msgr2=0x55a92bd675d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed5fe00640 1 --2- 172.21.15.2:0/2282753358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 0x55a92bd675d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.553+0000 7fed5fe00640 1 -- 172.21.15.2:0/2282753358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a92bc8b990 con 0x55a92bd5fe80 2024-09-10T08:19:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.554+0000 7fed5fe00640 1 --2- 172.21.15.2:0/2282753358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5fe80 0x55a92bd67b10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fed5000b9f0 tx=0x7fed5000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87cc4aa4cb73547 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.554+0000 7fed4a000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.554+0000 7fed4a000640 1 -- 172.21.15.2:0/2282753358 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50003e20 con 0x55a92bd5fe80 2024-09-10T08:19:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.554+0000 7fed69c63740 1 -- 172.21.15.2:0/2282753358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed4c0041d0 con 0x55a92bd5fe80 2024-09-10T08:19:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.554+0000 7fed4a000640 1 -- 172.21.15.2:0/2282753358 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fed50011440 con 0x55a92bd5fe80 2024-09-10T08:19:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.554+0000 7fed4a000640 1 -- 172.21.15.2:0/2282753358 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed50011640 con 0x55a92bd5fe80 2024-09-10T08:19:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.554+0000 7fed69c63740 10 client.18495 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.554+0000 7fed69c63740 1 -- 172.21.15.2:0/2282753358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a92bd78f70 con 0x55a92bd5fe80 2024-09-10T08:19:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.554+0000 7fed4a000640 1 -- 172.21.15.2:0/2282753358 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed4c0041d0 con 0x55a92bd5fe80 2024-09-10T08:19:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.555+0000 7fed4a000640 1 -- 172.21.15.2:0/2282753358 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fed5001baa0 con 0x55a92bd5fe80 2024-09-10T08:19:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.555+0000 7fed69c63740 10 client.18495 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.555+0000 7fed69c63740 10 client.18495 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:19:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.555+0000 7fed69c63740 20 client.18495 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.555+0000 7fed69c63740 1 --2- 172.21.15.2:0/2282753358 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a92bd7a380 0x55a92bd7c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.555+0000 7fed69c63740 4 client.18495 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.555+0000 7fed69c63740 1 -- 172.21.15.2:0/2282753358 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a92bc68310 con 0x55a92bd7a380 2024-09-10T08:19:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.555+0000 7fed5f400640 1 --2- 172.21.15.2:0/2282753358 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a92bd7a380 0x55a92bd7c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.556+0000 7fed5f400640 1 --2- 172.21.15.2:0/2282753358 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a92bd7a380 0x55a92bd7c7e0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cedc19926cb8c086 server_cookie=2cd2530c154bd6be in_seq=0 out_seq=0 2024-09-10T08:19:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.556+0000 7fed4a000640 10 client.18495 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.556+0000 7fed4a000640 1 -- 172.21.15.2:0/2282753358 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a92bc68310 con 0x55a92bd7a380 2024-09-10T08:19:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed4a000640 10 client.18495 handle_command_reply: tid=0 2024-09-10T08:19:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed69c63740 1 client.18495 shutdown 2024-09-10T08:19:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed69c63740 1 -- 172.21.15.2:0/2282753358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5fe80 msgr2=0x55a92bd67b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed69c63740 1 --2- 172.21.15.2:0/2282753358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5fe80 0x55a92bd67b10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fed5000b9f0 tx=0x7fed5000beb0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed69c63740 1 -- 172.21.15.2:0/2282753358 shutdown_connections 2024-09-10T08:19:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed69c63740 1 --2- 172.21.15.2:0/2282753358 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a92bd7a380 0x55a92bd7c7e0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed69c63740 1 --2- 172.21.15.2:0/2282753358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd6a740 0x55a92bd68050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed69c63740 1 --2- 172.21.15.2:0/2282753358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd5fe80 0x55a92bd67b10 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed4a000640 0 client.18495 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:10.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed69c63740 1 --2- 172.21.15.2:0/2282753358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 0x55a92bd675d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed69c63740 1 -- 172.21.15.2:0/2282753358 >> 172.21.15.2:0/2282753358 conn(0x55a92bd5baf0 msgr2=0x55a92bd61820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:10.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed69c63740 1 -- 172.21.15.2:0/2282753358 shutdown_connections 2024-09-10T08:19:10.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed69c63740 1 -- 172.21.15.2:0/2282753358 wait complete. 2024-09-10T08:19:10.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.557+0000 7fed69c63740 20 client.18495 trim_cache size 0 max 16384 2024-09-10T08:19:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.575+0000 7fed69c63740 1 Processor -- start 2024-09-10T08:19:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.575+0000 7fed69c63740 1 -- start start 2024-09-10T08:19:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.575+0000 7fed69c63740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 0x55a92bd62010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.575+0000 7fed69c63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd5fe80 0x55a92bd5ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.575+0000 7fed69c63740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 0x55a92bd820b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.575+0000 7fed69c63740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b3750 con 0x55a92bd6a740 2024-09-10T08:19:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.575+0000 7fed69c63740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b1020 con 0x55a92bd5f560 2024-09-10T08:19:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.575+0000 7fed69c63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed601aee70 con 0x55a92bd5fe80 2024-09-10T08:19:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.576+0000 7fed5f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 0x55a92bd820b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.576+0000 7fed5f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 0x55a92bd820b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35698/0 (socket says 172.21.15.2:35698) 2024-09-10T08:19:10.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.576+0000 7fed5f400640 1 -- 172.21.15.2:0/2639910665 learned_addr learned my addr 172.21.15.2:0/2639910665 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:10.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.576+0000 7fed5fe00640 1 --2- 172.21.15.2:0/2639910665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd5fe80 0x55a92bd5ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.576+0000 7fed5f400640 1 --2- 172.21.15.2:0/2639910665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 0x55a92bd62010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.576+0000 7fed5f400640 1 -- 172.21.15.2:0/2639910665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd5fe80 msgr2=0x55a92bd5ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:10.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.576+0000 7fed5f400640 1 --2- 172.21.15.2:0/2639910665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd5fe80 0x55a92bd5ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.576+0000 7fed5f400640 1 -- 172.21.15.2:0/2639910665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 msgr2=0x55a92bd62010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:10.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.576+0000 7fed5f400640 1 --2- 172.21.15.2:0/2639910665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 0x55a92bd62010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.576+0000 7fed5f400640 1 -- 172.21.15.2:0/2639910665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a92bd631f0 con 0x55a92bd6a740 2024-09-10T08:19:10.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.577+0000 7fed5f400640 1 --2- 172.21.15.2:0/2639910665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 0x55a92bd820b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fed4c012570 tx=0x7fed4c021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d18d4ebfe24ac7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:10.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.577+0000 7fed54c00640 1 -- 172.21.15.2:0/2639910665 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c028040 con 0x55a92bd6a740 2024-09-10T08:19:10.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.577+0000 7fed54c00640 1 -- 172.21.15.2:0/2639910665 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed4c033070 con 0x55a92bd6a740 2024-09-10T08:19:10.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.577+0000 7fed54c00640 1 -- 172.21.15.2:0/2639910665 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c02eac0 con 0x55a92bd6a740 2024-09-10T08:19:10.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.577+0000 7fed69c63740 1 -- 172.21.15.2:0/2639910665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 msgr2=0x55a92bd820b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:10.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.577+0000 7fed69c63740 1 --2- 172.21.15.2:0/2639910665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 0x55a92bd820b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fed4c012570 tx=0x7fed4c021040 comp rx=0 tx=0).stop 2024-09-10T08:19:10.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.578+0000 7fed69c63740 1 -- 172.21.15.2:0/2639910665 shutdown_connections 2024-09-10T08:19:10.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.578+0000 7fed69c63740 1 --2- 172.21.15.2:0/2639910665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 0x55a92bd820b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.578+0000 7fed69c63740 1 --2- 172.21.15.2:0/2639910665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd5fe80 0x55a92bd5ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.578+0000 7fed69c63740 1 --2- 172.21.15.2:0/2639910665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 0x55a92bd62010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.578+0000 7fed69c63740 1 -- 172.21.15.2:0/2639910665 >> 172.21.15.2:0/2639910665 conn(0x55a92bd5baf0 msgr2=0x55a92bd5a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:10.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.578+0000 7fed69c63740 1 -- 172.21.15.2:0/2639910665 shutdown_connections 2024-09-10T08:19:10.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.578+0000 7fed69c63740 1 -- 172.21.15.2:0/2639910665 wait complete. 2024-09-10T08:19:10.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.578+0000 7fed69c63740 1 -- start start 2024-09-10T08:19:10.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.578+0000 7fed69c63740 1 Processor -- start 2024-09-10T08:19:10.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.579+0000 7fed69c63740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 0x55a92bd728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.579+0000 7fed69c63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd5fe80 0x55a92bd72e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.579+0000 7fed69c63740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 0x55a92bd6f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.579+0000 7fed69c63740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fed601af4e0 con 0x55a92bd6a740 2024-09-10T08:19:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.579+0000 7fed69c63740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b0950 con 0x55a92bd5f560 2024-09-10T08:19:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.579+0000 7fed69c63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed601afbb0 con 0x55a92bd5fe80 2024-09-10T08:19:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.579+0000 7fed69c63740 10 client.0 _ll_register_callbacks cb 0x7fed50004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.579+0000 7fed5f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 0x55a92bd6f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.579+0000 7fed5f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 0x55a92bd6f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35712/0 (socket says 172.21.15.2:35712) 2024-09-10T08:19:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.579+0000 7fed5f400640 1 -- 172.21.15.2:0/4052893038 learned_addr learned my addr 172.21.15.2:0/4052893038 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.579+0000 7fed5fe00640 1 --2- 172.21.15.2:0/4052893038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd5fe80 0x55a92bd72e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.579+0000 7fed5f400640 1 --2- 172.21.15.2:0/4052893038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 0x55a92bd728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.580+0000 7fed5f400640 1 -- 172.21.15.2:0/4052893038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd5fe80 msgr2=0x55a92bd72e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.580+0000 7fed5f400640 1 --2- 172.21.15.2:0/4052893038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd5fe80 0x55a92bd72e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.580+0000 7fed5f400640 1 -- 172.21.15.2:0/4052893038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 msgr2=0x55a92bd728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.580+0000 7fed5f400640 1 --2- 172.21.15.2:0/4052893038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 0x55a92bd728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.580+0000 7fed5f400640 1 -- 172.21.15.2:0/4052893038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a92bd6feb0 con 0x55a92bd6a740 2024-09-10T08:19:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.580+0000 7fed5f400640 1 --2- 172.21.15.2:0/4052893038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 0x55a92bd6f6b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fed4c02e510 tx=0x7fed4c0123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5e0c2eb16cfb02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.580+0000 7fed4a000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.580+0000 7fed4a000640 1 -- 172.21.15.2:0/4052893038 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c02e570 con 0x55a92bd6a740 2024-09-10T08:19:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.580+0000 7fed4a000640 1 -- 172.21.15.2:0/4052893038 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fed4c0055f0 con 0x55a92bd6a740 2024-09-10T08:19:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.580+0000 7fed69c63740 1 -- 172.21.15.2:0/4052893038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed4c033070 con 0x55a92bd6a740 2024-09-10T08:19:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.581+0000 7fed4a000640 1 -- 172.21.15.2:0/4052893038 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed4c0035f0 con 0x55a92bd6a740 2024-09-10T08:19:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.581+0000 7fed69c63740 10 client.18501 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.581+0000 7fed69c63740 1 -- 172.21.15.2:0/4052893038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a92bde5af0 con 0x55a92bd6a740 2024-09-10T08:19:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.581+0000 7fed4a000640 1 -- 172.21.15.2:0/4052893038 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed4c033070 con 0x55a92bd6a740 2024-09-10T08:19:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.581+0000 7fed4a000640 1 -- 172.21.15.2:0/4052893038 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fed4c037030 con 0x55a92bd6a740 2024-09-10T08:19:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.581+0000 7fed69c63740 10 client.18501 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.581+0000 7fed69c63740 10 client.18501 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:19:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.581+0000 7fed69c63740 20 client.18501 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.581+0000 7fed69c63740 1 --2- 172.21.15.2:0/4052893038 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a92bde5ee0 0x55a92bde82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.581+0000 7fed69c63740 4 client.18501 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}] 2024-09-10T08:19:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.581+0000 7fed69c63740 1 -- 172.21.15.2:0/4052893038 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}) -- 0x55a92bcd2c60 con 0x55a92bde5ee0 2024-09-10T08:19:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.582+0000 7fed5fe00640 1 --2- 172.21.15.2:0/4052893038 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a92bde5ee0 0x55a92bde82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.582+0000 7fed5fe00640 1 --2- 172.21.15.2:0/4052893038 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a92bde5ee0 0x55a92bde82d0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51c9d7a4edb31f30 server_cookie=ae3e3183315318c9 in_seq=0 out_seq=0 2024-09-10T08:19:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.582+0000 7fed4a000640 10 client.18501 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:10.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.593+0000 7fed4a000640 1 -- 172.21.15.2:0/4052893038 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a92bcd2c60 con 0x55a92bde5ee0 2024-09-10T08:19:10.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.593+0000 7fed4a000640 10 client.18501 handle_command_reply: tid=0 2024-09-10T08:19:10.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.593+0000 7fed69c63740 1 client.18501 shutdown 2024-09-10T08:19:10.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.593+0000 7fed69c63740 1 -- 172.21.15.2:0/4052893038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 msgr2=0x55a92bd6f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:10.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.593+0000 7fed69c63740 1 --2- 172.21.15.2:0/4052893038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 0x55a92bd6f6b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fed4c02e510 tx=0x7fed4c0123f0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.593+0000 7fed69c63740 1 -- 172.21.15.2:0/4052893038 shutdown_connections 2024-09-10T08:19:10.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.593+0000 7fed69c63740 1 --2- 172.21.15.2:0/4052893038 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a92bde5ee0 0x55a92bde82d0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.593+0000 7fed69c63740 1 --2- 172.21.15.2:0/4052893038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a92bd6a740 0x55a92bd6f6b0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.593+0000 7fed69c63740 1 --2- 172.21.15.2:0/4052893038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a92bd5fe80 0x55a92bd72e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.593+0000 7fed69c63740 1 --2- 172.21.15.2:0/4052893038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a92bd5f560 0x55a92bd728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.593+0000 7fed69c63740 1 -- 172.21.15.2:0/4052893038 >> 172.21.15.2:0/4052893038 conn(0x55a92bd5baf0 msgr2=0x55a92bd67d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:10.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.593+0000 7fed4a000640 0 client.18501 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:10.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.594+0000 7fed69c63740 1 -- 172.21.15.2:0/4052893038 shutdown_connections 2024-09-10T08:19:10.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.594+0000 7fed69c63740 1 -- 172.21.15.2:0/4052893038 wait complete. 2024-09-10T08:19:10.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.594+0000 7fed69c63740 20 client.18501 trim_cache size 0 max 16384 2024-09-10T08:19:10.595 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:19:10.595 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:19:10.595 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:19:10.595 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 38, 2024-09-10T08:19:10.595 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:19:10.596 INFO:teuthology.orchestra.run.smithi002.stdout: "dd7972fa": { 2024-09-10T08:19:10.596 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 38, 2024-09-10T08:19:10.596 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:19:10.596 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:19:10.596 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:19:10.596 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:19:10.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:19:10.596 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:19:10.596 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 145.2, 2024-09-10T08:19:10.596 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:19:10.596 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:19:10.596 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:19:10.596 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:19:10.597 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:19:10.597 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:19:10.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:19:10.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:19:10.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:19:10.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:19:10.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:19:10.597 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:19:10.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.595+0000 7fed65c00640 1 -- 172.21.15.2:0/2284650991 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fed440a2c00 msgr2=0x7fed440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:10.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.595+0000 7fed65c00640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fed440a2c00 0x7fed440a50c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fed58006010 tx=0x7fed5802a8f0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.595+0000 7fed65c00640 1 -- 172.21.15.2:0/2284650991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed601079c0 msgr2=0x7fed6019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:10.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.595+0000 7fed65c00640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed601079c0 0x7fed6019ed70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fed58009ca0 tx=0x7fed58030ee0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.595+0000 7fed65c00640 1 -- 172.21.15.2:0/2284650991 shutdown_connections 2024-09-10T08:19:10.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.595+0000 7fed65c00640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed601079c0 0x7fed6019ed70 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.595+0000 7fed65c00640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fed440a2c00 0x7fed440a50c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.595+0000 7fed65c00640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed60102360 0x7fed6019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.595+0000 7fed65c00640 1 --2- 172.21.15.2:0/2284650991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed601019c0 0x7fed6019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:10.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.595+0000 7fed65c00640 1 -- 172.21.15.2:0/2284650991 >> 172.21.15.2:0/2284650991 conn(0x7fed600fd740 msgr2=0x7fed600fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:10.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.595+0000 7fed65c00640 1 -- 172.21.15.2:0/2284650991 shutdown_connections 2024-09-10T08:19:10.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:10.595+0000 7fed65c00640 1 -- 172.21.15.2:0/2284650991 wait complete. 2024-09-10T08:19:10.853 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:19:10.853 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:19:11.403 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'dd7972fa', quiesce took 5.5 seconds. Will release after: 19.7 2024-09-10T08:19:11.566 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:11.706 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:11 smithi002 ceph-mon[30563]: pgmap v718: 129 pgs: 129 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:19:11.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:11 smithi155 ceph-mon[35405]: pgmap v718: 129 pgs: 129 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:19:11.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:11 smithi107 ceph-mon[35048]: pgmap v718: 129 pgs: 129 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:19:12.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.163+0000 7fe4f7200640 1 -- 172.21.15.2:0/2497953774 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4f8108f60 msgr2=0x7fe4f8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:12.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.163+0000 7fe4f7200640 1 --2- 172.21.15.2:0/2497953774 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4f8108f60 0x7fe4f8110ae0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009ca0 tx=0x7fe4f002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.163+0000 7fe4f7200640 1 -- 172.21.15.2:0/2497953774 shutdown_connections 2024-09-10T08:19:12.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.163+0000 7fe4f7200640 1 --2- 172.21.15.2:0/2497953774 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4f8108f60 0x7fe4f8110ae0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.163+0000 7fe4f7200640 1 --2- 172.21.15.2:0/2497953774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4f81047e0 0x7fe4f8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.163+0000 7fe4f7200640 1 --2- 172.21.15.2:0/2497953774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4f8103e30 0x7fe4f8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.163+0000 7fe4f7200640 1 -- 172.21.15.2:0/2497953774 >> 172.21.15.2:0/2497953774 conn(0x7fe4f80ff8d0 msgr2=0x7fe4f8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:12.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.163+0000 7fe4f7200640 1 -- 172.21.15.2:0/2497953774 shutdown_connections 2024-09-10T08:19:12.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.163+0000 7fe4f7200640 1 -- 172.21.15.2:0/2497953774 wait complete. 2024-09-10T08:19:12.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.164+0000 7fe4f7200640 1 Processor -- start 2024-09-10T08:19:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.164+0000 7fe4f7200640 1 -- start start 2024-09-10T08:19:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.164+0000 7fe4f7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4f8103e30 0x7fe4f819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.164+0000 7fe4f7200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4f81047e0 0x7fe4f819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.164+0000 7fe4f7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4f8108f60 0x7fe4f819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.164+0000 7fe4f7200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8114010 con 0x7fe4f81047e0 2024-09-10T08:19:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.164+0000 7fe4f7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8113e90 con 0x7fe4f8108f60 2024-09-10T08:19:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.164+0000 7fe4f7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8114190 con 0x7fe4f8103e30 2024-09-10T08:19:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.165+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4f8103e30 0x7fe4f819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.165+0000 7fe4efe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4f81047e0 0x7fe4f819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.165+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4f8103e30 0x7fe4f819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:38054/0 (socket says 172.21.15.2:38054) 2024-09-10T08:19:12.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.165+0000 7fe4f5e00640 1 -- 172.21.15.2:0/1975493122 learned_addr learned my addr 172.21.15.2:0/1975493122 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.165+0000 7fe4f6800640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4f8108f60 0x7fe4f819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.165+0000 7fe4efe00640 1 -- 172.21.15.2:0/1975493122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4f8103e30 msgr2=0x7fe4f819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.165+0000 7fe4efe00640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4f8103e30 0x7fe4f819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.165+0000 7fe4efe00640 1 -- 172.21.15.2:0/1975493122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4f8108f60 msgr2=0x7fe4f819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.165+0000 7fe4efe00640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4f8108f60 0x7fe4f819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.165+0000 7fe4efe00640 1 -- 172.21.15.2:0/1975493122 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f819f3f0 con 0x7fe4f81047e0 2024-09-10T08:19:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.165+0000 7fe4f5e00640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4f8103e30 0x7fe4f819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.165+0000 7fe4f6800640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4f8108f60 0x7fe4f819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.165+0000 7fe4efe00640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4f81047e0 0x7fe4f819a980 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000e9f0 tx=0x7fe4e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.166+0000 7fe4eea00640 1 -- 172.21.15.2:0/1975493122 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e000ccc0 con 0x7fe4f81047e0 2024-09-10T08:19:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.166+0000 7fe4eea00640 1 -- 172.21.15.2:0/1975493122 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e0005020 con 0x7fe4f81047e0 2024-09-10T08:19:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.166+0000 7fe4eea00640 1 -- 172.21.15.2:0/1975493122 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00049e0 con 0x7fe4f81047e0 2024-09-10T08:19:12.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.166+0000 7fe4f7200640 1 -- 172.21.15.2:0/1975493122 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f819f6e0 con 0x7fe4f81047e0 2024-09-10T08:19:12.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.166+0000 7fe4f7200640 1 -- 172.21.15.2:0/1975493122 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f81a6fc0 con 0x7fe4f81047e0 2024-09-10T08:19:12.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.169+0000 7fe4ffa2b740 1 Processor -- start 2024-09-10T08:19:12.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.169+0000 7fe4eea00640 1 -- 172.21.15.2:0/1975493122 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe4e0005560 con 0x7fe4f81047e0 2024-09-10T08:19:12.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.169+0000 7fe4ffa2b740 1 -- start start 2024-09-10T08:19:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.170+0000 7fe4ffa2b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f46fcb0 0x557d3f470090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.170+0000 7fe4ffa2b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f4705d0 0x557d3f47a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.170+0000 7fe4ffa2b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f47ae90 0x557d3f47d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.170+0000 7fe4ffa2b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557d3f3d6290 con 0x557d3f4705d0 2024-09-10T08:19:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.170+0000 7fe4ffa2b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557d3f37db40 con 0x557d3f46fcb0 2024-09-10T08:19:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.170+0000 7fe4ffa2b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557d3f3d2250 con 0x557d3f47ae90 2024-09-10T08:19:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.170+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f4705d0 0x557d3f47a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.170+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f4705d0 0x557d3f47a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35736/0 (socket says 172.21.15.2:35736) 2024-09-10T08:19:12.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.170+0000 7fe4f6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f46fcb0 0x557d3f470090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.170+0000 7fe4f5e00640 1 -- 172.21.15.2:0/1349680300 learned_addr learned my addr 172.21.15.2:0/1349680300 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:12.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.170+0000 7fe4efe00640 1 --2- 172.21.15.2:0/1349680300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f47ae90 0x557d3f47d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.170+0000 7fe4f5e00640 1 -- 172.21.15.2:0/1349680300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f47ae90 msgr2=0x557d3f47d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:12.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.171+0000 7fe4f5e00640 1 --2- 172.21.15.2:0/1349680300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f47ae90 0x557d3f47d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.171+0000 7fe4f5e00640 1 -- 172.21.15.2:0/1349680300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f46fcb0 msgr2=0x557d3f470090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:12.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.171+0000 7fe4f5e00640 1 --2- 172.21.15.2:0/1349680300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f46fcb0 0x557d3f470090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.171+0000 7fe4f5e00640 1 -- 172.21.15.2:0/1349680300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d3f1476a0 con 0x557d3f4705d0 2024-09-10T08:19:12.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.171+0000 7fe4f6800640 1 --2- 172.21.15.2:0/1349680300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f46fcb0 0x557d3f470090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:12.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.171+0000 7fe4eea00640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4dc0a2c00 0x7fe4dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.171+0000 7fe4f5e00640 1 --2- 172.21.15.2:0/1349680300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f4705d0 0x557d3f47a950 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8011650 tx=0x7fe4e8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541da8fdfe0e304d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:12.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.171+0000 7fe4eea00640 1 -- 172.21.15.2:0/1975493122 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe4e00cf2b0 con 0x7fe4f81047e0 2024-09-10T08:19:12.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.171+0000 7fe4e7400640 1 -- 172.21.15.2:0/1349680300 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e801e070 con 0x557d3f4705d0 2024-09-10T08:19:12.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.171+0000 7fe4f6800640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4dc0a2c00 0x7fe4dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.171+0000 7fe4e7400640 1 -- 172.21.15.2:0/1349680300 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e800fba0 con 0x557d3f4705d0 2024-09-10T08:19:12.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.171+0000 7fe4e7400640 1 -- 172.21.15.2:0/1349680300 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8019cd0 con 0x557d3f4705d0 2024-09-10T08:19:12.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.172+0000 7fe4f6800640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4dc0a2c00 0x7fe4dc0a50c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0006010 tx=0x7fe4f0005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:12.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.172+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/1349680300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f4705d0 msgr2=0x557d3f47a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:12.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.172+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/1349680300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f4705d0 0x557d3f47a950 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8011650 tx=0x7fe4e8009f90 comp rx=0 tx=0).stop 2024-09-10T08:19:12.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.172+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/1349680300 shutdown_connections 2024-09-10T08:19:12.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.172+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/1349680300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f47ae90 0x557d3f47d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.172+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/1349680300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f4705d0 0x557d3f47a950 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.172+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/1349680300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f46fcb0 0x557d3f470090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.172+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/1349680300 >> 172.21.15.2:0/1349680300 conn(0x557d3f46c240 msgr2=0x557d3f46c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:12.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.172+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/1349680300 shutdown_connections 2024-09-10T08:19:12.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.172+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/1349680300 wait complete. 2024-09-10T08:19:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.173+0000 7fe4ffa2b740 1 -- start start 2024-09-10T08:19:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.173+0000 7fe4ffa2b740 1 Processor -- start 2024-09-10T08:19:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.173+0000 7fe4ffa2b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f46fcb0 0x557d3f47a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.173+0000 7fe4ffa2b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f4705d0 0x557d3f477a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4ffa2b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f47ae90 0x557d3f477fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4ffa2b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557d3f47db00 con 0x557d3f4705d0 2024-09-10T08:19:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4ffa2b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557d3f390fa0 con 0x557d3f47ae90 2024-09-10T08:19:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4ffa2b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557d3f47dc80 con 0x557d3f46fcb0 2024-09-10T08:19:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4ffa2b740 10 client.0 _ll_register_callbacks cb 0x557d3f0e4ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f46fcb0 0x557d3f47a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f46fcb0 0x557d3f47a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:38066/0 (socket says 172.21.15.2:38066) 2024-09-10T08:19:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4f5e00640 1 -- 172.21.15.2:0/780212162 learned_addr learned my addr 172.21.15.2:0/780212162 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4efe00640 1 --2- 172.21.15.2:0/780212162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f4705d0 0x557d3f477a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4f6800640 1 --2- 172.21.15.2:0/780212162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f47ae90 0x557d3f477fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4efe00640 1 -- 172.21.15.2:0/780212162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f46fcb0 msgr2=0x557d3f47a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4efe00640 1 --2- 172.21.15.2:0/780212162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f46fcb0 0x557d3f47a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4efe00640 1 -- 172.21.15.2:0/780212162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f47ae90 msgr2=0x557d3f477fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4efe00640 1 --2- 172.21.15.2:0/780212162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f47ae90 0x557d3f477fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.174+0000 7fe4efe00640 1 -- 172.21.15.2:0/780212162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d3f4787a0 con 0x557d3f4705d0 2024-09-10T08:19:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.175+0000 7fe4efe00640 1 --2- 172.21.15.2:0/780212162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f4705d0 0x557d3f477a60 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x557d3f46d5c0 tx=0x7fe4e00d2f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccbb473103bb044 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.175+0000 7fe4dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.175+0000 7fe4dbe00640 1 -- 172.21.15.2:0/780212162 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e009b020 con 0x557d3f4705d0 2024-09-10T08:19:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.175+0000 7fe4dbe00640 1 -- 172.21.15.2:0/780212162 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe4e009b920 con 0x557d3f4705d0 2024-09-10T08:19:12.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.175+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/780212162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4e800fba0 con 0x557d3f4705d0 2024-09-10T08:19:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.175+0000 7fe4dbe00640 1 -- 172.21.15.2:0/780212162 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4e00051d0 con 0x557d3f4705d0 2024-09-10T08:19:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.175+0000 7fe4ffa2b740 10 client.18513 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.175+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/780212162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557d3f489a70 con 0x557d3f4705d0 2024-09-10T08:19:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.175+0000 7fe4dbe00640 1 -- 172.21.15.2:0/780212162 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e800fba0 con 0x557d3f4705d0 2024-09-10T08:19:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.176+0000 7fe4dbe00640 1 -- 172.21.15.2:0/780212162 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe4e00afbe0 con 0x557d3f4705d0 2024-09-10T08:19:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.176+0000 7fe4ffa2b740 10 client.18513 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.176+0000 7fe4ffa2b740 10 client.18513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.176+0000 7fe4ffa2b740 20 client.18513 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.176+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/780212162 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d3f48af80 0x557d3f48d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.176+0000 7fe4ffa2b740 4 client.18513 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.176+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/780212162 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d3f3e3800 con 0x557d3f48af80 2024-09-10T08:19:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.176+0000 7fe4f5e00640 1 --2- 172.21.15.2:0/780212162 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d3f48af80 0x557d3f48d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.177+0000 7fe4f5e00640 1 --2- 172.21.15.2:0/780212162 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d3f48af80 0x557d3f48d3b0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=962f30ee0fb05464 server_cookie=47285cc8b926bb0b in_seq=0 out_seq=0 2024-09-10T08:19:12.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.177+0000 7fe4dbe00640 10 client.18513 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:12.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4dbe00640 1 -- 172.21.15.2:0/780212162 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d3f3e3800 con 0x557d3f48af80 2024-09-10T08:19:12.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4dbe00640 10 client.18513 handle_command_reply: tid=0 2024-09-10T08:19:12.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4ffa2b740 1 client.18513 shutdown 2024-09-10T08:19:12.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/780212162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f4705d0 msgr2=0x557d3f477a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:12.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/780212162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f4705d0 0x557d3f477a60 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x557d3f46d5c0 tx=0x7fe4e00d2f60 comp rx=0 tx=0).stop 2024-09-10T08:19:12.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/780212162 shutdown_connections 2024-09-10T08:19:12.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/780212162 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d3f48af80 0x557d3f48d3b0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/780212162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f47ae90 0x557d3f477fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/780212162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f4705d0 0x557d3f477a60 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/780212162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f46fcb0 0x557d3f47a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/780212162 >> 172.21.15.2:0/780212162 conn(0x557d3f46c240 msgr2=0x557d3f471f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:12.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4dbe00640 0 client.18513 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:12.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/780212162 shutdown_connections 2024-09-10T08:19:12.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/780212162 wait complete. 2024-09-10T08:19:12.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.178+0000 7fe4ffa2b740 20 client.18513 trim_cache size 0 max 16384 2024-09-10T08:19:12.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.196+0000 7fe4ffa2b740 1 Processor -- start 2024-09-10T08:19:12.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.197+0000 7fe4ffa2b740 1 -- start start 2024-09-10T08:19:12.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.197+0000 7fe4ffa2b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f46fcb0 0x557d3f492c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.197+0000 7fe4ffa2b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f4705d0 0x557d3f4931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.197+0000 7fe4ffa2b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f47ae90 0x557d3f474cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.197+0000 7fe4ffa2b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81b3790 con 0x557d3f46fcb0 2024-09-10T08:19:12.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.197+0000 7fe4ffa2b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81b1060 con 0x557d3f4705d0 2024-09-10T08:19:12.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.197+0000 7fe4ffa2b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81aeeb0 con 0x557d3f47ae90 2024-09-10T08:19:12.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.197+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f46fcb0 0x557d3f492c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.197+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f46fcb0 0x557d3f492c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35740/0 (socket says 172.21.15.2:35740) 2024-09-10T08:19:12.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.197+0000 7fe4f5e00640 1 -- 172.21.15.2:0/2632008083 learned_addr learned my addr 172.21.15.2:0/2632008083 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:12.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.197+0000 7fe4efe00640 1 --2- 172.21.15.2:0/2632008083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f4705d0 0x557d3f4931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.197+0000 7fe4f6800640 1 --2- 172.21.15.2:0/2632008083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f47ae90 0x557d3f474cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.198+0000 7fe4efe00640 1 -- 172.21.15.2:0/2632008083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f47ae90 msgr2=0x557d3f474cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:12.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.198+0000 7fe4efe00640 1 --2- 172.21.15.2:0/2632008083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f47ae90 0x557d3f474cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.198+0000 7fe4efe00640 1 -- 172.21.15.2:0/2632008083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f46fcb0 msgr2=0x557d3f492c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:12.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.198+0000 7fe4efe00640 1 --2- 172.21.15.2:0/2632008083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f46fcb0 0x557d3f492c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.198+0000 7fe4efe00640 1 -- 172.21.15.2:0/2632008083 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d3f46d3c0 con 0x557d3f4705d0 2024-09-10T08:19:12.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.198+0000 7fe4f6800640 1 --2- 172.21.15.2:0/2632008083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f47ae90 0x557d3f474cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:19:12.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.198+0000 7fe4f5e00640 1 --2- 172.21.15.2:0/2632008083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f46fcb0 0x557d3f492c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:12.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.198+0000 7fe4efe00640 1 --2- 172.21.15.2:0/2632008083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f4705d0 0x557d3f4931c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x557d3f479010 tx=0x7fe4e00d2d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e59aafc886d000c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:12.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.198+0000 7fe4e6a00640 1 -- 172.21.15.2:0/2632008083 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e009bb70 con 0x557d3f4705d0 2024-09-10T08:19:12.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.198+0000 7fe4e6a00640 1 -- 172.21.15.2:0/2632008083 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e00b4020 con 0x557d3f4705d0 2024-09-10T08:19:12.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.198+0000 7fe4e6a00640 1 -- 172.21.15.2:0/2632008083 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00afbe0 con 0x557d3f4705d0 2024-09-10T08:19:12.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.199+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/2632008083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f4705d0 msgr2=0x557d3f4931c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:12.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.199+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/2632008083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f4705d0 0x557d3f4931c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x557d3f479010 tx=0x7fe4e00d2d60 comp rx=0 tx=0).stop 2024-09-10T08:19:12.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.199+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/2632008083 shutdown_connections 2024-09-10T08:19:12.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.199+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/2632008083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f47ae90 0x557d3f474cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.199+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/2632008083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f4705d0 0x557d3f4931c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.199+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/2632008083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f46fcb0 0x557d3f492c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.199+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/2632008083 >> 172.21.15.2:0/2632008083 conn(0x557d3f46c240 msgr2=0x557d3f4730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:12.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.199+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/2632008083 shutdown_connections 2024-09-10T08:19:12.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.199+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/2632008083 wait complete. 2024-09-10T08:19:12.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.199+0000 7fe4ffa2b740 1 -- start start 2024-09-10T08:19:12.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.200+0000 7fe4ffa2b740 1 Processor -- start 2024-09-10T08:19:12.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.200+0000 7fe4ffa2b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f46fcb0 0x557d3f47ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.200+0000 7fe4ffa2b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f4705d0 0x557d3f47f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.200+0000 7fe4ffa2b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f47ae90 0x557d3f47f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.200+0000 7fe4ffa2b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81af520 con 0x557d3f47ae90 2024-09-10T08:19:12.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.200+0000 7fe4ffa2b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81b0990 con 0x557d3f4705d0 2024-09-10T08:19:12.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.200+0000 7fe4ffa2b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81afbf0 con 0x557d3f46fcb0 2024-09-10T08:19:12.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.200+0000 7fe4ffa2b740 10 client.0 _ll_register_callbacks cb 0x7fe4e00d29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:12.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.200+0000 7fe4efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f4705d0 0x557d3f47f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.200+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f46fcb0 0x557d3f47ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.200+0000 7fe4efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f4705d0 0x557d3f47f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36604/0 (socket says 172.21.15.2:36604) 2024-09-10T08:19:12.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.200+0000 7fe4efe00640 1 -- 172.21.15.2:0/662135336 learned_addr learned my addr 172.21.15.2:0/662135336 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:12.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.200+0000 7fe4f6800640 1 --2- 172.21.15.2:0/662135336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f47ae90 0x557d3f47f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.201+0000 7fe4efe00640 1 -- 172.21.15.2:0/662135336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f46fcb0 msgr2=0x557d3f47ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:12.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.201+0000 7fe4efe00640 1 --2- 172.21.15.2:0/662135336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f46fcb0 0x557d3f47ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.201+0000 7fe4efe00640 1 -- 172.21.15.2:0/662135336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f47ae90 msgr2=0x557d3f47f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:12.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.201+0000 7fe4efe00640 1 --2- 172.21.15.2:0/662135336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f47ae90 0x557d3f47f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.201+0000 7fe4efe00640 1 -- 172.21.15.2:0/662135336 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d3f4f4ef0 con 0x557d3f4705d0 2024-09-10T08:19:12.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.201+0000 7fe4f5e00640 1 --2- 172.21.15.2:0/662135336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f46fcb0 0x557d3f47ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:12.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.201+0000 7fe4efe00640 1 --2- 172.21.15.2:0/662135336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f4705d0 0x557d3f47f440 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0097b20 tx=0x7fe4e00d2b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2d6d4e4210e0122 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:12.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.201+0000 7fe4dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:19:12.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.201+0000 7fe4dbe00640 1 -- 172.21.15.2:0/662135336 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e009bb20 con 0x557d3f4705d0 2024-09-10T08:19:12.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.201+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/662135336 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4e00b4020 con 0x557d3f4705d0 2024-09-10T08:19:12.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.201+0000 7fe4dbe00640 1 -- 172.21.15.2:0/662135336 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe4e009ab10 con 0x557d3f4705d0 2024-09-10T08:19:12.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.202+0000 7fe4dbe00640 1 -- 172.21.15.2:0/662135336 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4e00afdf0 con 0x557d3f4705d0 2024-09-10T08:19:12.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.202+0000 7fe4ffa2b740 10 client.28550 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:12.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.202+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/662135336 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557d3f4f6210 con 0x557d3f4705d0 2024-09-10T08:19:12.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.202+0000 7fe4dbe00640 1 -- 172.21.15.2:0/662135336 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e00b4020 con 0x557d3f4705d0 2024-09-10T08:19:12.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.202+0000 7fe4dbe00640 1 -- 172.21.15.2:0/662135336 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe4e009a610 con 0x557d3f4705d0 2024-09-10T08:19:12.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.202+0000 7fe4ffa2b740 10 client.28550 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:12.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.202+0000 7fe4ffa2b740 10 client.28550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:12.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.202+0000 7fe4ffa2b740 20 client.28550 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:12.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.202+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/662135336 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d3f4f6690 0x557d3f4f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:12.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.202+0000 7fe4ffa2b740 4 client.28550 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:19:12.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.202+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/662135336 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557d3f3ddd10 con 0x557d3f4f6690 2024-09-10T08:19:12.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.203+0000 7fe4f5e00640 1 --2- 172.21.15.2:0/662135336 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d3f4f6690 0x557d3f4f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:12.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.203+0000 7fe4f5e00640 1 --2- 172.21.15.2:0/662135336 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d3f4f6690 0x557d3f4f8a80 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58ee07450287cac3 server_cookie=aa149705333df218 in_seq=0 out_seq=0 2024-09-10T08:19:12.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.203+0000 7fe4dbe00640 10 client.28550 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:12.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.204+0000 7fe4dbe00640 1 -- 172.21.15.2:0/662135336 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557d3f3ddd10 con 0x557d3f4f6690 2024-09-10T08:19:12.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.204+0000 7fe4dbe00640 10 client.28550 handle_command_reply: tid=0 2024-09-10T08:19:12.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.204+0000 7fe4ffa2b740 1 client.28550 shutdown 2024-09-10T08:19:12.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.204+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/662135336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f4705d0 msgr2=0x557d3f47f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:12.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.204+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/662135336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f4705d0 0x557d3f47f440 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0097b20 tx=0x7fe4e00d2b00 comp rx=0 tx=0).stop 2024-09-10T08:19:12.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.204+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/662135336 shutdown_connections 2024-09-10T08:19:12.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.204+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/662135336 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557d3f4f6690 0x557d3f4f8a80 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.204+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/662135336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d3f47ae90 0x557d3f47f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.204+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/662135336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d3f4705d0 0x557d3f47f440 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.204+0000 7fe4ffa2b740 1 --2- 172.21.15.2:0/662135336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d3f46fcb0 0x557d3f47ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.204+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/662135336 >> 172.21.15.2:0/662135336 conn(0x557d3f46c240 msgr2=0x557d3f472790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:12.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.204+0000 7fe4dbe00640 0 client.28550 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:12.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.205+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/662135336 shutdown_connections 2024-09-10T08:19:12.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.205+0000 7fe4ffa2b740 1 -- 172.21.15.2:0/662135336 wait complete. 2024-09-10T08:19:12.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.205+0000 7fe4ffa2b740 20 client.28550 trim_cache size 0 max 16384 2024-09-10T08:19:12.209 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:19:12.210 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:19:12.210 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "65259958-f205-4935-8dda-75b8cd60a643", 2024-09-10T08:19:12.210 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:19:12.210 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:19:12.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.205+0000 7fe4f7200640 1 -- 172.21.15.2:0/1975493122 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4dc0a2c00 msgr2=0x7fe4dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:12.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.205+0000 7fe4f7200640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4dc0a2c00 0x7fe4dc0a50c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0006010 tx=0x7fe4f0005d80 comp rx=0 tx=0).stop 2024-09-10T08:19:12.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.205+0000 7fe4f7200640 1 -- 172.21.15.2:0/1975493122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4f81047e0 msgr2=0x7fe4f819a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:12.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.205+0000 7fe4f7200640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4f81047e0 0x7fe4f819a980 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000e9f0 tx=0x7fe4e000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.206+0000 7fe4f7200640 1 -- 172.21.15.2:0/1975493122 shutdown_connections 2024-09-10T08:19:12.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.206+0000 7fe4f7200640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4f8108f60 0x7fe4f819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.206+0000 7fe4f7200640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4f81047e0 0x7fe4f819a980 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.206+0000 7fe4f7200640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe4dc0a2c00 0x7fe4dc0a50c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.206+0000 7fe4f7200640 1 --2- 172.21.15.2:0/1975493122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4f8103e30 0x7fe4f819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:12.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.206+0000 7fe4f7200640 1 -- 172.21.15.2:0/1975493122 >> 172.21.15.2:0/1975493122 conn(0x7fe4f80ff8d0 msgr2=0x7fe4f8100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:12.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.206+0000 7fe4f7200640 1 -- 172.21.15.2:0/1975493122 shutdown_connections 2024-09-10T08:19:12.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:12.206+0000 7fe4f7200640 1 -- 172.21.15.2:0/1975493122 wait complete. 2024-09-10T08:19:13.097 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:19:13.482 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:13.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:13 smithi002 ceph-mon[30563]: pgmap v719: 129 pgs: 129 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:19:13.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:13 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:19:13.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:13 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:19:13.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:13 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:19:13.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:13 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:19:13.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:13 smithi155 ceph-mon[35405]: pgmap v719: 129 pgs: 129 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:19:13.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:13 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:19:13.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:13 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:19:13.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:13 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:19:13.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:13 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:19:13.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:13 smithi107 ceph-mon[35048]: pgmap v719: 129 pgs: 129 active+clean; 1.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:19:13.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:13 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:19:13.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:13 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:19:13.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:13 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:19:13.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:13 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:19:14.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.086+0000 7fc3e1200640 1 -- 172.21.15.2:0/1251060110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3dc109140 msgr2=0x7fc3dc110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:14.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.086+0000 7fc3e1200640 1 --2- 172.21.15.2:0/1251060110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3dc109140 0x7fc3dc110cc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009ca0 tx=0x7fc3d402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.086+0000 7fc3e1200640 1 -- 172.21.15.2:0/1251060110 shutdown_connections 2024-09-10T08:19:14.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.086+0000 7fc3e1200640 1 --2- 172.21.15.2:0/1251060110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3dc109140 0x7fc3dc110cc0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.086+0000 7fc3e1200640 1 --2- 172.21.15.2:0/1251060110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3dc1049c0 0x7fc3dc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.086+0000 7fc3e1200640 1 --2- 172.21.15.2:0/1251060110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3dc104010 0x7fc3dc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.086+0000 7fc3e1200640 1 -- 172.21.15.2:0/1251060110 >> 172.21.15.2:0/1251060110 conn(0x7fc3dc0ffa50 msgr2=0x7fc3dc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:14.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.086+0000 7fc3e1200640 1 -- 172.21.15.2:0/1251060110 shutdown_connections 2024-09-10T08:19:14.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.086+0000 7fc3e1200640 1 -- 172.21.15.2:0/1251060110 wait complete. 2024-09-10T08:19:14.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.087+0000 7fc3e1200640 1 Processor -- start 2024-09-10T08:19:14.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.087+0000 7fc3e1200640 1 -- start start 2024-09-10T08:19:14.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.087+0000 7fc3e1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3dc104010 0x7fc3dc19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.087+0000 7fc3e1200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3dc1049c0 0x7fc3dc19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.087+0000 7fc3e1200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3dc109140 0x7fc3dc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3e1200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1141a0 con 0x7fc3dc109140 2024-09-10T08:19:14.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3e1200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc114020 con 0x7fc3dc104010 2024-09-10T08:19:14.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3e1200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc114320 con 0x7fc3dc1049c0 2024-09-10T08:19:14.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3d9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3dc104010 0x7fc3dc19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3d9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3dc104010 0x7fc3dc19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36614/0 (socket says 172.21.15.2:36614) 2024-09-10T08:19:14.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3d9e00640 1 -- 172.21.15.2:0/4226831458 learned_addr learned my addr 172.21.15.2:0/4226831458 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:14.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3d1400640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3dc1049c0 0x7fc3dc19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3da800640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3dc109140 0x7fc3dc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3d9e00640 1 -- 172.21.15.2:0/4226831458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3dc1049c0 msgr2=0x7fc3dc19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:14.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3d9e00640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3dc1049c0 0x7fc3dc19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3d9e00640 1 -- 172.21.15.2:0/4226831458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3dc109140 msgr2=0x7fc3dc19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:14.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3d9e00640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3dc109140 0x7fc3dc19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3d9e00640 1 -- 172.21.15.2:0/4226831458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc19f630 con 0x7fc3dc104010 2024-09-10T08:19:14.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.088+0000 7fc3d1400640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3dc1049c0 0x7fc3dc19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:14.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.089+0000 7fc3d9e00640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3dc104010 0x7fc3dc19a680 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00e9f0 tx=0x7fc3cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:14.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.089+0000 7fc3d3400640 1 -- 172.21.15.2:0/4226831458 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc00cd80 con 0x7fc3dc104010 2024-09-10T08:19:14.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.089+0000 7fc3e1200640 1 -- 172.21.15.2:0/4226831458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3dc19f920 con 0x7fc3dc104010 2024-09-10T08:19:14.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.089+0000 7fc3d3400640 1 -- 172.21.15.2:0/4226831458 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3cc010910 con 0x7fc3dc104010 2024-09-10T08:19:14.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.089+0000 7fc3d3400640 1 -- 172.21.15.2:0/4226831458 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc0188f0 con 0x7fc3dc104010 2024-09-10T08:19:14.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.089+0000 7fc3e1200640 1 -- 172.21.15.2:0/4226831458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3dc1a7200 con 0x7fc3dc104010 2024-09-10T08:19:14.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.093+0000 7fc3e3e0c740 1 Processor -- start 2024-09-10T08:19:14.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.093+0000 7fc3e3e0c740 1 -- start start 2024-09-10T08:19:14.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.093+0000 7fc3e3e0c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.093+0000 7fc3e3e0c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc14b0 0x561763ccb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.093+0000 7fc3e3e0c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 0x561763cce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.094+0000 7fc3e3e0c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561763c273e0 con 0x561763ccbd70 2024-09-10T08:19:14.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.094+0000 7fc3e3e0c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561763bced80 con 0x561763cc14b0 2024-09-10T08:19:14.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.094+0000 7fc3e3e0c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561763c233a0 con 0x561763cc0b90 2024-09-10T08:19:14.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.094+0000 7fc3da800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.094+0000 7fc3d3400640 1 -- 172.21.15.2:0/4226831458 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc3cc010ac0 con 0x7fc3dc104010 2024-09-10T08:19:14.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.094+0000 7fc3da800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:38106/0 (socket says 172.21.15.2:38106) 2024-09-10T08:19:14.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.094+0000 7fc3da800640 1 -- 172.21.15.2:0/1925330332 learned_addr learned my addr 172.21.15.2:0/1925330332 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:14.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.094+0000 7fc3d1400640 1 --2- 172.21.15.2:0/1925330332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 0x561763cce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.094+0000 7fc3da800640 1 -- 172.21.15.2:0/1925330332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc14b0 msgr2=0x561763ccb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:14.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.094+0000 7fc3da800640 1 --2- 172.21.15.2:0/1925330332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc14b0 0x561763ccb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.094+0000 7fc3da800640 1 -- 172.21.15.2:0/1925330332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 msgr2=0x561763cce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:14.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.094+0000 7fc3da800640 1 --2- 172.21.15.2:0/1925330332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 0x561763cce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.094+0000 7fc3da800640 1 -- 172.21.15.2:0/1925330332 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561763bedae0 con 0x561763cc0b90 2024-09-10T08:19:14.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.095+0000 7fc3da800640 1 --2- 172.21.15.2:0/1925330332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc0f70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009ca0 tx=0x7fc3d40093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec028ced06ddc7bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:14.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.095+0000 7fc3cb400640 1 -- 172.21.15.2:0/1925330332 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4031db0 con 0x561763cc0b90 2024-09-10T08:19:14.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.095+0000 7fc3cb400640 1 -- 172.21.15.2:0/1925330332 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d4036030 con 0x561763cc0b90 2024-09-10T08:19:14.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.095+0000 7fc3cb400640 1 -- 172.21.15.2:0/1925330332 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40042a0 con 0x561763cc0b90 2024-09-10T08:19:14.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.096+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/1925330332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 msgr2=0x561763cc0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:14.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.096+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/1925330332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc0f70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009ca0 tx=0x7fc3d40093e0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.096+0000 7fc3d3400640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3c00a2c00 0x7fc3c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.096+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/1925330332 shutdown_connections 2024-09-10T08:19:14.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.096+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/1925330332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 0x561763cce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.096+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/1925330332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc14b0 0x561763ccb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.096+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/1925330332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc0f70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.096+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/1925330332 >> 172.21.15.2:0/1925330332 conn(0x561763cbd120 msgr2=0x561763cbd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:14.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.096+0000 7fc3d3400640 1 -- 172.21.15.2:0/4226831458 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc3cc0cf7d0 con 0x7fc3dc104010 2024-09-10T08:19:14.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.096+0000 7fc3da800640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3c00a2c00 0x7fc3c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.096+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/1925330332 shutdown_connections 2024-09-10T08:19:14.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.096+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/1925330332 wait complete. 2024-09-10T08:19:14.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.097+0000 7fc3e3e0c740 1 -- start start 2024-09-10T08:19:14.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.097+0000 7fc3da800640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3c00a2c00 0x7fc3c00a50c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4006010 tx=0x7fc3d40319d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:14.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.097+0000 7fc3e3e0c740 1 Processor -- start 2024-09-10T08:19:14.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.097+0000 7fc3e3e0c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc0b90 0x561763cc8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.097+0000 7fc3e3e0c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc14b0 0x561763cc9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.097+0000 7fc3e3e0c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 0x561763cc96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.097+0000 7fc3e3e0c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561763cce860 con 0x561763ccbd70 2024-09-10T08:19:14.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.097+0000 7fc3e3e0c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561763be20f0 con 0x561763cc0b90 2024-09-10T08:19:14.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.097+0000 7fc3e3e0c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561763cce9e0 con 0x561763cc14b0 2024-09-10T08:19:14.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.097+0000 7fc3e3e0c740 10 client.0 _ll_register_callbacks cb 0x561763bd39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:14.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.098+0000 7fc3d9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc0b90 0x561763cc8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.098+0000 7fc3d9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc0b90 0x561763cc8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36634/0 (socket says 172.21.15.2:36634) 2024-09-10T08:19:14.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.098+0000 7fc3d9e00640 1 -- 172.21.15.2:0/1092318558 learned_addr learned my addr 172.21.15.2:0/1092318558 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:14.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.098+0000 7fc3da800640 1 --2- 172.21.15.2:0/1092318558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 0x561763cc96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.098+0000 7fc3d1400640 1 --2- 172.21.15.2:0/1092318558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc14b0 0x561763cc9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.098+0000 7fc3d9e00640 1 -- 172.21.15.2:0/1092318558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc14b0 msgr2=0x561763cc9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:14.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.098+0000 7fc3d9e00640 1 --2- 172.21.15.2:0/1092318558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc14b0 0x561763cc9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.098+0000 7fc3d9e00640 1 -- 172.21.15.2:0/1092318558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 msgr2=0x561763cc96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:14.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.098+0000 7fc3d9e00640 1 --2- 172.21.15.2:0/1092318558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 0x561763cc96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.098+0000 7fc3d9e00640 1 -- 172.21.15.2:0/1092318558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617639983c0 con 0x561763cc0b90 2024-09-10T08:19:14.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.098+0000 7fc3d9e00640 1 --2- 172.21.15.2:0/1092318558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc0b90 0x561763cc8c50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc09bc20 tx=0x7fc3cc09bc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b86529161146ecb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:14.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.098+0000 7fc3bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:19:14.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.098+0000 7fc3bfe00640 1 -- 172.21.15.2:0/1092318558 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc0a4850 con 0x561763cc0b90 2024-09-10T08:19:14.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.099+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/1092318558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d4036030 con 0x561763cc0b90 2024-09-10T08:19:14.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.099+0000 7fc3bfe00640 1 -- 172.21.15.2:0/1092318558 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc3cc0cfbc0 con 0x561763cc0b90 2024-09-10T08:19:14.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.099+0000 7fc3bfe00640 1 -- 172.21.15.2:0/1092318558 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc3cc0b9050 con 0x561763cc0b90 2024-09-10T08:19:14.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.099+0000 7fc3e3e0c740 10 client.28559 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:14.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.099+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/1092318558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561763cda540 con 0x561763cc0b90 2024-09-10T08:19:14.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.099+0000 7fc3bfe00640 1 -- 172.21.15.2:0/1092318558 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d4036030 con 0x561763cc0b90 2024-09-10T08:19:14.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.099+0000 7fc3bfe00640 1 -- 172.21.15.2:0/1092318558 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc3cc097b70 con 0x561763cc0b90 2024-09-10T08:19:14.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.099+0000 7fc3e3e0c740 10 client.28559 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:14.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.099+0000 7fc3e3e0c740 10 client.28559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:14.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.099+0000 7fc3e3e0c740 20 client.28559 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:14.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.099+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/1092318558 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561763cdb950 0x561763cddd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.099+0000 7fc3e3e0c740 4 client.28559 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:14.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.099+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/1092318558 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561763c34950 con 0x561763cdb950 2024-09-10T08:19:14.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.100+0000 7fc3d1400640 1 --2- 172.21.15.2:0/1092318558 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561763cdb950 0x561763cddd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.100+0000 7fc3d1400640 1 --2- 172.21.15.2:0/1092318558 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561763cdb950 0x561763cddd90 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ea48c1d2f45111e server_cookie=d8e092fe1c25e65f in_seq=0 out_seq=0 2024-09-10T08:19:14.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.100+0000 7fc3bfe00640 10 client.28559 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:14.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3bfe00640 1 -- 172.21.15.2:0/1092318558 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561763c34950 con 0x561763cdb950 2024-09-10T08:19:14.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3bfe00640 10 client.28559 handle_command_reply: tid=0 2024-09-10T08:19:14.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3e3e0c740 1 client.28559 shutdown 2024-09-10T08:19:14.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/1092318558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc0b90 msgr2=0x561763cc8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:14.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/1092318558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc0b90 0x561763cc8c50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc09bc20 tx=0x7fc3cc09bc90 comp rx=0 tx=0).stop 2024-09-10T08:19:14.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/1092318558 shutdown_connections 2024-09-10T08:19:14.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/1092318558 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561763cdb950 0x561763cddd90 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/1092318558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 0x561763cc96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/1092318558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc14b0 0x561763cc9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/1092318558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc0b90 0x561763cc8c50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/1092318558 >> 172.21.15.2:0/1092318558 conn(0x561763cbd120 msgr2=0x561763cc2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:14.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3bfe00640 0 client.28559 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:14.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/1092318558 shutdown_connections 2024-09-10T08:19:14.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.101+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/1092318558 wait complete. 2024-09-10T08:19:14.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.102+0000 7fc3e3e0c740 20 client.28559 trim_cache size 0 max 16384 2024-09-10T08:19:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.120+0000 7fc3e3e0c740 1 Processor -- start 2024-09-10T08:19:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.120+0000 7fc3e3e0c740 1 -- start start 2024-09-10T08:19:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.120+0000 7fc3e3e0c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.120+0000 7fc3e3e0c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc14b0 0x561763ce3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.120+0000 7fc3e3e0c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 0x561763ce3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.120+0000 7fc3e3e0c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1b38a0 con 0x561763ccbd70 2024-09-10T08:19:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.120+0000 7fc3e3e0c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1b1170 con 0x561763cc14b0 2024-09-10T08:19:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.120+0000 7fc3e3e0c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1aefc0 con 0x561763cc0b90 2024-09-10T08:19:14.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.121+0000 7fc3d9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.121+0000 7fc3da800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 0x561763ce3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.121+0000 7fc3d9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:38120/0 (socket says 172.21.15.2:38120) 2024-09-10T08:19:14.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.121+0000 7fc3d9e00640 1 -- 172.21.15.2:0/3485632806 learned_addr learned my addr 172.21.15.2:0/3485632806 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:14.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.121+0000 7fc3d1400640 1 --2- 172.21.15.2:0/3485632806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc14b0 0x561763ce3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.121+0000 7fc3d9e00640 1 -- 172.21.15.2:0/3485632806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc14b0 msgr2=0x561763ce3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:14.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.121+0000 7fc3d9e00640 1 --2- 172.21.15.2:0/3485632806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc14b0 0x561763ce3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.121+0000 7fc3d9e00640 1 -- 172.21.15.2:0/3485632806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 msgr2=0x561763ce3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:14.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.121+0000 7fc3d9e00640 1 --2- 172.21.15.2:0/3485632806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 0x561763ce3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.121+0000 7fc3d9e00640 1 -- 172.21.15.2:0/3485632806 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561763cbcf50 con 0x561763cc0b90 2024-09-10T08:19:14.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.121+0000 7fc3d9e00640 1 --2- 172.21.15.2:0/3485632806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc35f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0980f0 tx=0x7fc3cc09abc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0441331c9aae5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:14.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.121+0000 7fc3caa00640 1 -- 172.21.15.2:0/3485632806 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc09b0e0 con 0x561763cc0b90 2024-09-10T08:19:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.122+0000 7fc3caa00640 1 -- 172.21.15.2:0/3485632806 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3cc0a9070 con 0x561763cc0b90 2024-09-10T08:19:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.122+0000 7fc3caa00640 1 -- 172.21.15.2:0/3485632806 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc097c40 con 0x561763cc0b90 2024-09-10T08:19:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.122+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/3485632806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 msgr2=0x561763cc35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.122+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/3485632806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc35f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0980f0 tx=0x7fc3cc09abc0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.122+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/3485632806 shutdown_connections 2024-09-10T08:19:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.122+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/3485632806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763ccbd70 0x561763ce3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.122+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/3485632806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763cc14b0 0x561763ce3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.122+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/3485632806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc35f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.122+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/3485632806 >> 172.21.15.2:0/3485632806 conn(0x561763cbd120 msgr2=0x561763ccacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.122+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/3485632806 shutdown_connections 2024-09-10T08:19:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.122+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/3485632806 wait complete. 2024-09-10T08:19:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.123+0000 7fc3e3e0c740 1 -- start start 2024-09-10T08:19:14.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.123+0000 7fc3e3e0c740 1 Processor -- start 2024-09-10T08:19:14.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3e3e0c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3e3e0c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763cc14b0 0x561763cc6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3e3e0c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763ccbd70 0x561763cc66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3e3e0c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1af630 con 0x561763cc14b0 2024-09-10T08:19:14.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3e3e0c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1b0aa0 con 0x561763ccbd70 2024-09-10T08:19:14.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3e3e0c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1afd00 con 0x561763cc0b90 2024-09-10T08:19:14.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3e3e0c740 10 client.0 _ll_register_callbacks cb 0x7fc3cc09b7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:14.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3d1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763cc14b0 0x561763cc6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3da800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763ccbd70 0x561763cc66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3d1400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763cc14b0 0x561763cc6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35794/0 (socket says 172.21.15.2:35794) 2024-09-10T08:19:14.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3d1400640 1 -- 172.21.15.2:0/829712885 learned_addr learned my addr 172.21.15.2:0/829712885 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:14.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3d9e00640 1 --2- 172.21.15.2:0/829712885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3d1400640 1 -- 172.21.15.2:0/829712885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 msgr2=0x561763cc5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:14.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3d1400640 1 --2- 172.21.15.2:0/829712885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3d1400640 1 -- 172.21.15.2:0/829712885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763ccbd70 msgr2=0x561763cc66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:14.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3d1400640 1 --2- 172.21.15.2:0/829712885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763ccbd70 0x561763cc66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3d1400640 1 -- 172.21.15.2:0/829712885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561763d45630 con 0x561763cc14b0 2024-09-10T08:19:14.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.124+0000 7fc3da800640 1 --2- 172.21.15.2:0/829712885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763ccbd70 0x561763cc66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:14.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.125+0000 7fc3d1400640 1 --2- 172.21.15.2:0/829712885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763cc14b0 0x561763cc6160 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x561763cbe4a0 tx=0x7fc3c4012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1896c800442967b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:14.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.125+0000 7fc3bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:14.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.125+0000 7fc3bfe00640 1 -- 172.21.15.2:0/829712885 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4022070 con 0x561763cc14b0 2024-09-10T08:19:14.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.125+0000 7fc3bfe00640 1 -- 172.21.15.2:0/829712885 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc3c401f990 con 0x561763cc14b0 2024-09-10T08:19:14.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.125+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/829712885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3cc0a9070 con 0x561763cc14b0 2024-09-10T08:19:14.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.125+0000 7fc3bfe00640 1 -- 172.21.15.2:0/829712885 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc3c401fb90 con 0x561763cc14b0 2024-09-10T08:19:14.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.125+0000 7fc3e3e0c740 10 client.18537 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:14.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.125+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/829712885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561763d46970 con 0x561763cc14b0 2024-09-10T08:19:14.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.126+0000 7fc3bfe00640 1 -- 172.21.15.2:0/829712885 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3cc0a9070 con 0x561763cc14b0 2024-09-10T08:19:14.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.126+0000 7fc3bfe00640 1 -- 172.21.15.2:0/829712885 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc3c40265d0 con 0x561763cc14b0 2024-09-10T08:19:14.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.126+0000 7fc3e3e0c740 10 client.18537 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:14.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.126+0000 7fc3e3e0c740 10 client.18537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:14.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.126+0000 7fc3e3e0c740 20 client.18537 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:14.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.126+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/829712885 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561763d47140 0x561763d49560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:14.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.126+0000 7fc3e3e0c740 4 client.18537 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:19:14.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.126+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/829712885 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561763c2ee60 con 0x561763d47140 2024-09-10T08:19:14.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.126+0000 7fc3d9e00640 1 --2- 172.21.15.2:0/829712885 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561763d47140 0x561763d49560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:14.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3d9e00640 1 --2- 172.21.15.2:0/829712885 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561763d47140 0x561763d49560 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b97dfbad164d204 server_cookie=8e4057818b62ed64 in_seq=0 out_seq=0 2024-09-10T08:19:14.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3bfe00640 10 client.18537 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3bfe00640 1 -- 172.21.15.2:0/829712885 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561763c2ee60 con 0x561763d47140 2024-09-10T08:19:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3bfe00640 10 client.18537 handle_command_reply: tid=0 2024-09-10T08:19:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3e3e0c740 1 client.18537 shutdown 2024-09-10T08:19:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/829712885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763cc14b0 msgr2=0x561763cc6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/829712885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763cc14b0 0x561763cc6160 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x561763cbe4a0 tx=0x7fc3c4012b00 comp rx=0 tx=0).stop 2024-09-10T08:19:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/829712885 shutdown_connections 2024-09-10T08:19:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/829712885 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561763d47140 0x561763d49560 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/829712885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561763ccbd70 0x561763cc66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/829712885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561763cc14b0 0x561763cc6160 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3e3e0c740 1 --2- 172.21.15.2:0/829712885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561763cc0b90 0x561763cc5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3bfe00640 0 client.18537 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/829712885 >> 172.21.15.2:0/829712885 conn(0x561763cbd120 msgr2=0x561763cbb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:14.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.127+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/829712885 shutdown_connections 2024-09-10T08:19:14.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.128+0000 7fc3e3e0c740 1 -- 172.21.15.2:0/829712885 wait complete. 2024-09-10T08:19:14.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.128+0000 7fc3e3e0c740 20 client.18537 trim_cache size 0 max 16384 2024-09-10T08:19:14.135 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:19:14.135 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:19:14.135 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:19:14.135 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:19:14.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.128+0000 7fc3e1200640 1 -- 172.21.15.2:0/4226831458 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3c00a2c00 msgr2=0x7fc3c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:14.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.128+0000 7fc3e1200640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3c00a2c00 0x7fc3c00a50c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4006010 tx=0x7fc3d40319d0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.128+0000 7fc3e1200640 1 -- 172.21.15.2:0/4226831458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3dc104010 msgr2=0x7fc3dc19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:14.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.128+0000 7fc3e1200640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3dc104010 0x7fc3dc19a680 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00e9f0 tx=0x7fc3cc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.128+0000 7fc3e1200640 1 -- 172.21.15.2:0/4226831458 shutdown_connections 2024-09-10T08:19:14.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.128+0000 7fc3e1200640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3dc109140 0x7fc3dc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.128+0000 7fc3e1200640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc3c00a2c00 0x7fc3c00a50c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.128+0000 7fc3e1200640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3dc1049c0 0x7fc3dc19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.128+0000 7fc3e1200640 1 --2- 172.21.15.2:0/4226831458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3dc104010 0x7fc3dc19a680 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:14.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.128+0000 7fc3e1200640 1 -- 172.21.15.2:0/4226831458 >> 172.21.15.2:0/4226831458 conn(0x7fc3dc0ffa50 msgr2=0x7fc3dc100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:14.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.129+0000 7fc3e1200640 1 -- 172.21.15.2:0/4226831458 shutdown_connections 2024-09-10T08:19:14.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:14.129+0000 7fc3e1200640 1 -- 172.21.15.2:0/4226831458 wait complete. 2024-09-10T08:19:14.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:14 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:19:14.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:14 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:19:14.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:14 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:19:15.067 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:19:15.067 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:19:15.442 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.626+0000 7f334aa00640 1 -- 172.21.15.2:0/2526819604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3344105910 msgr2=0x7f334410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.626+0000 7f334aa00640 1 --2- 172.21.15.2:0/2526819604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3344105910 0x7f334410c1a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f333c009860 tx=0x7f333c02a340 comp rx=0 tx=0).stop 2024-09-10T08:19:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.626+0000 7f334aa00640 1 -- 172.21.15.2:0/2526819604 shutdown_connections 2024-09-10T08:19:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.626+0000 7f334aa00640 1 --2- 172.21.15.2:0/2526819604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3344105910 0x7f334410c1a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.626+0000 7f334aa00640 1 --2- 172.21.15.2:0/2526819604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3344104f50 0x7f33441053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.626+0000 7f334aa00640 1 --2- 172.21.15.2:0/2526819604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3344103d60 0x7f3344104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.626+0000 7f334aa00640 1 -- 172.21.15.2:0/2526819604 >> 172.21.15.2:0/2526819604 conn(0x7f33440ff4e0 msgr2=0x7f3344101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.626+0000 7f334aa00640 1 -- 172.21.15.2:0/2526819604 shutdown_connections 2024-09-10T08:19:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.627+0000 7f334aa00640 1 -- 172.21.15.2:0/2526819604 wait complete. 2024-09-10T08:19:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.627+0000 7f334aa00640 1 Processor -- start 2024-09-10T08:19:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f334aa00640 1 -- start start 2024-09-10T08:19:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f334aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3344103d60 0x7f3344199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f334aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3344104f50 0x7f334419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f334aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3344105910 0x7f33441a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f334aa00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f334410f9c0 con 0x7f3344105910 2024-09-10T08:19:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f334aa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f334410f840 con 0x7f3344104f50 2024-09-10T08:19:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f334aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f334410fb40 con 0x7f3344103d60 2024-09-10T08:19:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f3348a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3344105910 0x7f33441a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f3348a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3344105910 0x7f33441a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35810/0 (socket says 172.21.15.2:35810) 2024-09-10T08:19:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f3348a00640 1 -- 172.21.15.2:0/3216792263 learned_addr learned my addr 172.21.15.2:0/3216792263 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f3343e00640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3344103d60 0x7f3344199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f333b400640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3344104f50 0x7f334419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f3348a00640 1 -- 172.21.15.2:0/3216792263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3344103d60 msgr2=0x7f3344199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f3348a00640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3344103d60 0x7f3344199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f3348a00640 1 -- 172.21.15.2:0/3216792263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3344104f50 msgr2=0x7f334419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f3348a00640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3344104f50 0x7f334419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.628+0000 7f3348a00640 1 -- 172.21.15.2:0/3216792263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33441a1ba0 con 0x7f3344105910 2024-09-10T08:19:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.629+0000 7f3348a00640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3344105910 0x7f33441a1530 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f333c005940 tx=0x7f333c033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.629+0000 7f3341600640 1 -- 172.21.15.2:0/3216792263 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c037040 con 0x7f3344105910 2024-09-10T08:19:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.629+0000 7f334aa00640 1 -- 172.21.15.2:0/3216792263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33441a1e30 con 0x7f3344105910 2024-09-10T08:19:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.629+0000 7f3341600640 1 -- 172.21.15.2:0/3216792263 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f333c040070 con 0x7f3344105910 2024-09-10T08:19:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.629+0000 7f3341600640 1 -- 172.21.15.2:0/3216792263 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c0055f0 con 0x7f3344105910 2024-09-10T08:19:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.629+0000 7f334aa00640 1 -- 172.21.15.2:0/3216792263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33441a2290 con 0x7f3344105910 2024-09-10T08:19:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.632+0000 7f334da1c740 1 Processor -- start 2024-09-10T08:19:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.632+0000 7f3341600640 1 -- 172.21.15.2:0/3216792263 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f333c030850 con 0x7f3344105910 2024-09-10T08:19:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.632+0000 7f334da1c740 1 -- start start 2024-09-10T08:19:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f334da1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f090 0x563556e7f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f334da1c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e84490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f334da1c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e84a60 0x563556e86f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f334da1c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563556de4100 con 0x563556e7f9d0 2024-09-10T08:19:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f334da1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563556d8c1b0 con 0x563556e7f090 2024-09-10T08:19:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f334da1c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563556de00c0 con 0x563556e84a60 2024-09-10T08:19:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f3343e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e84490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f3343e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e84490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35822/0 (socket says 172.21.15.2:35822) 2024-09-10T08:19:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f3343e00640 1 -- 172.21.15.2:0/4032814016 learned_addr learned my addr 172.21.15.2:0/4032814016 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f333b400640 1 --2- 172.21.15.2:0/4032814016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f090 0x563556e7f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f333b400640 1 --2- 172.21.15.2:0/4032814016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e84a60 0x563556e86f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f3343e00640 1 -- 172.21.15.2:0/4032814016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e84a60 msgr2=0x563556e86f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f3343e00640 1 --2- 172.21.15.2:0/4032814016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e84a60 0x563556e86f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f3343e00640 1 -- 172.21.15.2:0/4032814016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f090 msgr2=0x563556e7f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f3343e00640 1 --2- 172.21.15.2:0/4032814016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f090 0x563556e7f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.633+0000 7f3343e00640 1 -- 172.21.15.2:0/4032814016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563556af7210 con 0x563556e7f9d0 2024-09-10T08:19:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.634+0000 7f3343e00640 1 --2- 172.21.15.2:0/4032814016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e84490 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3334011470 tx=0x7f3334011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a157dcda996c5959 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.634+0000 7f3341600640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f33280a2c30 0x7f33280a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.634+0000 7f3341600640 1 -- 172.21.15.2:0/3216792263 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f333c0ef460 con 0x7f3344105910 2024-09-10T08:19:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.634+0000 7f3339200640 1 -- 172.21.15.2:0/4032814016 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333401e070 con 0x563556e7f9d0 2024-09-10T08:19:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.634+0000 7f3339200640 1 -- 172.21.15.2:0/4032814016 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3334003c30 con 0x563556e7f9d0 2024-09-10T08:19:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.634+0000 7f3339200640 1 -- 172.21.15.2:0/4032814016 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333400f9e0 con 0x563556e7f9d0 2024-09-10T08:19:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.634+0000 7f333b400640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f33280a2c30 0x7f33280a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.635+0000 7f334da1c740 1 -- 172.21.15.2:0/4032814016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 msgr2=0x563556e84490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.635+0000 7f334da1c740 1 --2- 172.21.15.2:0/4032814016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e84490 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3334011470 tx=0x7f3334011930 comp rx=0 tx=0).stop 2024-09-10T08:19:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.635+0000 7f333b400640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f33280a2c30 0x7f33280a50f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f332c009690 tx=0x7f332c009620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.635+0000 7f334da1c740 1 -- 172.21.15.2:0/4032814016 shutdown_connections 2024-09-10T08:19:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.635+0000 7f334da1c740 1 --2- 172.21.15.2:0/4032814016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e84a60 0x563556e86f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.635+0000 7f334da1c740 1 --2- 172.21.15.2:0/4032814016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e84490 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.635+0000 7f334da1c740 1 --2- 172.21.15.2:0/4032814016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f090 0x563556e7f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.635+0000 7f334da1c740 1 -- 172.21.15.2:0/4032814016 >> 172.21.15.2:0/4032814016 conn(0x563556e7a700 msgr2=0x563556e7ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.635+0000 7f334da1c740 1 -- 172.21.15.2:0/4032814016 shutdown_connections 2024-09-10T08:19:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.635+0000 7f334da1c740 1 -- 172.21.15.2:0/4032814016 wait complete. 2024-09-10T08:19:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.635+0000 7f334da1c740 1 -- start start 2024-09-10T08:19:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.636+0000 7f334da1c740 1 Processor -- start 2024-09-10T08:19:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.636+0000 7f334da1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f090 0x563556e89210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.636+0000 7f334da1c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e89750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.636+0000 7f334da1c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e84a60 0x563556e90790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.636+0000 7f334da1c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563556e87a30 con 0x563556e7f9d0 2024-09-10T08:19:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.636+0000 7f334da1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563556d6d090 con 0x563556e7f090 2024-09-10T08:19:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.636+0000 7f334da1c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563556e87bb0 con 0x563556e84a60 2024-09-10T08:19:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.636+0000 7f334da1c740 10 client.0 _ll_register_callbacks cb 0x563556dc1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f333b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e89750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f3343e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f090 0x563556e89210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f333b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e89750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35826/0 (socket says 172.21.15.2:35826) 2024-09-10T08:19:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f333b400640 1 -- 172.21.15.2:0/1911055576 learned_addr learned my addr 172.21.15.2:0/1911055576 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f3348a00640 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e84a60 0x563556e90790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f3343e00640 1 -- 172.21.15.2:0/1911055576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e84a60 msgr2=0x563556e90790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f3343e00640 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e84a60 0x563556e90790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f3343e00640 1 -- 172.21.15.2:0/1911055576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 msgr2=0x563556e89750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f3343e00640 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e89750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f3343e00640 1 -- 172.21.15.2:0/1911055576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563556bc3620 con 0x563556e7f090 2024-09-10T08:19:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f3348a00640 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e84a60 0x563556e90790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f333b400640 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e89750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f3343e00640 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f090 0x563556e89210 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3334011e30 tx=0x7f3334004f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb81336bb0aca347 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.637+0000 7f3331600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:19:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.638+0000 7f3331600640 1 -- 172.21.15.2:0/1911055576 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333401e050 con 0x563556e7f090 2024-09-10T08:19:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.638+0000 7f3331600640 1 -- 172.21.15.2:0/1911055576 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f333400fb70 con 0x563556e7f090 2024-09-10T08:19:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.638+0000 7f334da1c740 1 -- 172.21.15.2:0/1911055576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3334003c30 con 0x563556e7f090 2024-09-10T08:19:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.638+0000 7f3331600640 1 -- 172.21.15.2:0/1911055576 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3334032050 con 0x563556e7f090 2024-09-10T08:19:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.638+0000 7f334da1c740 10 client.28577 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.638+0000 7f334da1c740 1 -- 172.21.15.2:0/1911055576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563556e935d0 con 0x563556e7f090 2024-09-10T08:19:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.638+0000 7f3331600640 1 -- 172.21.15.2:0/1911055576 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3334003c30 con 0x563556e7f090 2024-09-10T08:19:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.639+0000 7f3331600640 1 -- 172.21.15.2:0/1911055576 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f333402d030 con 0x563556e7f090 2024-09-10T08:19:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.639+0000 7f334da1c740 10 client.28577 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.639+0000 7f334da1c740 10 client.28577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.639+0000 7f334da1c740 20 client.28577 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.639+0000 7f334da1c740 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563556e94af0 0x563556e96f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.639+0000 7f334da1c740 4 client.28577 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.639+0000 7f334da1c740 1 -- 172.21.15.2:0/1911055576 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563556df1830 con 0x563556e94af0 2024-09-10T08:19:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.639+0000 7f333b400640 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563556e94af0 0x563556e96f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.640+0000 7f333b400640 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563556e94af0 0x563556e96f00 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49b13582fb9e5d0 server_cookie=2ae6ed7512443220 in_seq=0 out_seq=0 2024-09-10T08:19:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.640+0000 7f3331600640 10 client.28577 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f3331600640 1 -- 172.21.15.2:0/1911055576 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563556df1830 con 0x563556e94af0 2024-09-10T08:19:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f3331600640 10 client.28577 handle_command_reply: tid=0 2024-09-10T08:19:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f334da1c740 1 client.28577 shutdown 2024-09-10T08:19:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f334da1c740 1 -- 172.21.15.2:0/1911055576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f090 msgr2=0x563556e89210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f334da1c740 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f090 0x563556e89210 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3334011e30 tx=0x7f3334004f30 comp rx=0 tx=0).stop 2024-09-10T08:19:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f334da1c740 1 -- 172.21.15.2:0/1911055576 shutdown_connections 2024-09-10T08:19:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f334da1c740 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563556e94af0 0x563556e96f00 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f334da1c740 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e84a60 0x563556e90790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f334da1c740 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e89750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f3331600640 0 client.28577 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f334da1c740 1 --2- 172.21.15.2:0/1911055576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f090 0x563556e89210 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f334da1c740 1 -- 172.21.15.2:0/1911055576 >> 172.21.15.2:0/1911055576 conn(0x563556e7a700 msgr2=0x563556e7bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f334da1c740 1 -- 172.21.15.2:0/1911055576 shutdown_connections 2024-09-10T08:19:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f334da1c740 1 -- 172.21.15.2:0/1911055576 wait complete. 2024-09-10T08:19:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.641+0000 7f334da1c740 20 client.28577 trim_cache size 0 max 16384 2024-09-10T08:19:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.656+0000 7f334da1c740 1 Processor -- start 2024-09-10T08:19:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.656+0000 7f334da1c740 1 -- start start 2024-09-10T08:19:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f334da1c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e7f090 0x563556e7f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f334da1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f9d0 0x563556e90750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f334da1c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e84a60 0x563556e7aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f334da1c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f33441af2b0 con 0x563556e84a60 2024-09-10T08:19:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f334da1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f33441acb80 con 0x563556e7f9d0 2024-09-10T08:19:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f334da1c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f33441aa9d0 con 0x563556e7f090 2024-09-10T08:19:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f333b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f9d0 0x563556e90750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f3343e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e7f090 0x563556e7f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f3343e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e7f090 0x563556e7f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:38162/0 (socket says 172.21.15.2:38162) 2024-09-10T08:19:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f333b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f9d0 0x563556e90750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36694/0 (socket says 172.21.15.2:36694) 2024-09-10T08:19:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f3343e00640 1 -- 172.21.15.2:0/3607861845 learned_addr learned my addr 172.21.15.2:0/3607861845 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f3348a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e84a60 0x563556e7aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f333b400640 1 -- 172.21.15.2:0/3607861845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e7f090 msgr2=0x563556e7f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.657+0000 7f333b400640 1 --2- 172.21.15.2:0/3607861845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e7f090 0x563556e7f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.658+0000 7f333b400640 1 -- 172.21.15.2:0/3607861845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e84a60 msgr2=0x563556e7aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.658+0000 7f333b400640 1 --2- 172.21.15.2:0/3607861845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e84a60 0x563556e7aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.658+0000 7f333b400640 1 -- 172.21.15.2:0/3607861845 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563556e97580 con 0x563556e7f9d0 2024-09-10T08:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.658+0000 7f3343e00640 1 --2- 172.21.15.2:0/3607861845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e7f090 0x563556e7f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.658+0000 7f333b400640 1 --2- 172.21.15.2:0/3607861845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f9d0 0x563556e90750 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x563556e8a6f0 tx=0x7f332c012070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cab5a57d0c342d8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.658+0000 7f3333e00640 1 -- 172.21.15.2:0/3607861845 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c005020 con 0x563556e7f9d0 2024-09-10T08:19:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.658+0000 7f3333e00640 1 -- 172.21.15.2:0/3607861845 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f332c0047a0 con 0x563556e7f9d0 2024-09-10T08:19:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.658+0000 7f3333e00640 1 -- 172.21.15.2:0/3607861845 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c01e790 con 0x563556e7f9d0 2024-09-10T08:19:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.659+0000 7f334da1c740 1 -- 172.21.15.2:0/3607861845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f9d0 msgr2=0x563556e90750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.659+0000 7f334da1c740 1 --2- 172.21.15.2:0/3607861845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f9d0 0x563556e90750 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x563556e8a6f0 tx=0x7f332c012070 comp rx=0 tx=0).stop 2024-09-10T08:19:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.659+0000 7f334da1c740 1 -- 172.21.15.2:0/3607861845 shutdown_connections 2024-09-10T08:19:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.659+0000 7f334da1c740 1 --2- 172.21.15.2:0/3607861845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e84a60 0x563556e7aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.659+0000 7f334da1c740 1 --2- 172.21.15.2:0/3607861845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e7f9d0 0x563556e90750 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.659+0000 7f334da1c740 1 --2- 172.21.15.2:0/3607861845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e7f090 0x563556e7f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.659+0000 7f334da1c740 1 -- 172.21.15.2:0/3607861845 >> 172.21.15.2:0/3607861845 conn(0x563556e7a700 msgr2=0x563556e88fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.659+0000 7f334da1c740 1 -- 172.21.15.2:0/3607861845 shutdown_connections 2024-09-10T08:19:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.659+0000 7f334da1c740 1 -- 172.21.15.2:0/3607861845 wait complete. 2024-09-10T08:19:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.659+0000 7f334da1c740 1 -- start start 2024-09-10T08:19:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.660+0000 7f334da1c740 1 Processor -- start 2024-09-10T08:19:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.660+0000 7f334da1c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e7f090 0x563556e903f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.660+0000 7f334da1c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e8ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.660+0000 7f334da1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e84a60 0x563556e8d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.660+0000 7f334da1c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f33441ab040 con 0x563556e7f9d0 2024-09-10T08:19:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.660+0000 7f334da1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f33441ac4b0 con 0x563556e84a60 2024-09-10T08:19:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.660+0000 7f334da1c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f33441ab710 con 0x563556e7f090 2024-09-10T08:19:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.660+0000 7f334da1c740 10 client.0 _ll_register_callbacks cb 0x7f333401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.660+0000 7f333b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e8ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.660+0000 7f333b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e8ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35836/0 (socket says 172.21.15.2:35836) 2024-09-10T08:19:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.660+0000 7f333b400640 1 -- 172.21.15.2:0/3777698976 learned_addr learned my addr 172.21.15.2:0/3777698976 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.660+0000 7f3348a00640 1 --2- 172.21.15.2:0/3777698976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e84a60 0x563556e8d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.660+0000 7f3343e00640 1 --2- 172.21.15.2:0/3777698976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e7f090 0x563556e903f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.661+0000 7f333b400640 1 -- 172.21.15.2:0/3777698976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e7f090 msgr2=0x563556e903f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.661+0000 7f333b400640 1 --2- 172.21.15.2:0/3777698976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e7f090 0x563556e903f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.661+0000 7f333b400640 1 -- 172.21.15.2:0/3777698976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e84a60 msgr2=0x563556e8d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.661+0000 7f333b400640 1 --2- 172.21.15.2:0/3777698976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e84a60 0x563556e8d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.661+0000 7f333b400640 1 -- 172.21.15.2:0/3777698976 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563556e90a50 con 0x563556e7f9d0 2024-09-10T08:19:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.661+0000 7f333b400640 1 --2- 172.21.15.2:0/3777698976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e8ce90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f332c01b040 tx=0x7f332c01e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24898bbef27c288 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.661+0000 7f3331600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.661+0000 7f3331600640 1 -- 172.21.15.2:0/3777698976 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c01ecd0 con 0x563556e7f9d0 2024-09-10T08:19:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.661+0000 7f334da1c740 1 -- 172.21.15.2:0/3777698976 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f332c0047a0 con 0x563556e7f9d0 2024-09-10T08:19:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.661+0000 7f3331600640 1 -- 172.21.15.2:0/3777698976 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f332c03f480 con 0x563556e7f9d0 2024-09-10T08:19:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.661+0000 7f3331600640 1 -- 172.21.15.2:0/3777698976 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f332c048050 con 0x563556e7f9d0 2024-09-10T08:19:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.662+0000 7f334da1c740 10 client.18555 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.662+0000 7f334da1c740 1 -- 172.21.15.2:0/3777698976 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563556efe790 con 0x563556e7f9d0 2024-09-10T08:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.662+0000 7f3331600640 1 -- 172.21.15.2:0/3777698976 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f332c0047a0 con 0x563556e7f9d0 2024-09-10T08:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.662+0000 7f3331600640 1 -- 172.21.15.2:0/3777698976 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f332c010790 con 0x563556e7f9d0 2024-09-10T08:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.662+0000 7f334da1c740 10 client.18555 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.662+0000 7f334da1c740 10 client.18555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.662+0000 7f334da1c740 20 client.18555 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.662+0000 7f334da1c740 1 --2- 172.21.15.2:0/3777698976 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563556eff120 0x563556f01540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.662+0000 7f334da1c740 4 client.18555 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.662+0000 7f334da1c740 1 -- 172.21.15.2:0/3777698976 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563556debb80 con 0x563556eff120 2024-09-10T08:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.663+0000 7f3343e00640 1 --2- 172.21.15.2:0/3777698976 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563556eff120 0x563556f01540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.663+0000 7f3343e00640 1 --2- 172.21.15.2:0/3777698976 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563556eff120 0x563556f01540 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46f54a1d07d592c4 server_cookie=a50f53300e025260 in_seq=0 out_seq=0 2024-09-10T08:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.663+0000 7f3331600640 10 client.18555 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f3331600640 1 -- 172.21.15.2:0/3777698976 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563556debb80 con 0x563556eff120 2024-09-10T08:19:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f3331600640 10 client.18555 handle_command_reply: tid=0 2024-09-10T08:19:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f334da1c740 1 client.18555 shutdown 2024-09-10T08:19:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f334da1c740 1 -- 172.21.15.2:0/3777698976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 msgr2=0x563556e8ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f334da1c740 1 --2- 172.21.15.2:0/3777698976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e8ce90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f332c01b040 tx=0x7f332c01e6d0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f334da1c740 1 -- 172.21.15.2:0/3777698976 shutdown_connections 2024-09-10T08:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f334da1c740 1 --2- 172.21.15.2:0/3777698976 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563556eff120 0x563556f01540 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f334da1c740 1 --2- 172.21.15.2:0/3777698976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563556e84a60 0x563556e8d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f3331600640 0 client.18555 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f334da1c740 1 --2- 172.21.15.2:0/3777698976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563556e7f9d0 0x563556e8ce90 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f334da1c740 1 --2- 172.21.15.2:0/3777698976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563556e7f090 0x563556e903f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f334da1c740 1 -- 172.21.15.2:0/3777698976 >> 172.21.15.2:0/3777698976 conn(0x563556e7a700 msgr2=0x563556e79060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f334da1c740 1 -- 172.21.15.2:0/3777698976 shutdown_connections 2024-09-10T08:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.676+0000 7f334da1c740 1 -- 172.21.15.2:0/3777698976 wait complete. 2024-09-10T08:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.677+0000 7f334da1c740 20 client.18555 trim_cache size 0 max 16384 2024-09-10T08:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:19:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.677+0000 7f334aa00640 1 -- 172.21.15.2:0/3216792263 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f33280a2c30 msgr2=0x7f33280a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.677+0000 7f334aa00640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f33280a2c30 0x7f33280a50f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f332c009690 tx=0x7f332c009620 comp rx=0 tx=0).stop 2024-09-10T08:19:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.677+0000 7f334aa00640 1 -- 172.21.15.2:0/3216792263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3344105910 msgr2=0x7f33441a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.677+0000 7f334aa00640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3344105910 0x7f33441a1530 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f333c005940 tx=0x7f333c033d90 comp rx=0 tx=0).stop 2024-09-10T08:19:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.677+0000 7f334aa00640 1 -- 172.21.15.2:0/3216792263 shutdown_connections 2024-09-10T08:19:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.677+0000 7f334aa00640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3344105910 0x7f33441a1530 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.677+0000 7f334aa00640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3344104f50 0x7f334419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.677+0000 7f334aa00640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f33280a2c30 0x7f33280a50f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.677+0000 7f334aa00640 1 --2- 172.21.15.2:0/3216792263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3344103d60 0x7f3344199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.677+0000 7f334aa00640 1 -- 172.21.15.2:0/3216792263 >> 172.21.15.2:0/3216792263 conn(0x7f33440ff4e0 msgr2=0x7f3344108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.677+0000 7f334aa00640 1 -- 172.21.15.2:0/3216792263 shutdown_connections 2024-09-10T08:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:15.677+0000 7f334aa00640 1 -- 172.21.15.2:0/3216792263 wait complete. 2024-09-10T08:19:15.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:15 smithi002 ceph-mon[30563]: pgmap v720: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-10T08:19:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:15 smithi155 ceph-mon[35405]: pgmap v720: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-10T08:19:15.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:15 smithi107 ceph-mon[35048]: pgmap v720: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-10T08:19:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.064+0000 7f9c97000640 1 -- 172.21.15.2:0/2265514505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c90101de0 msgr2=0x7f9c901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.064+0000 7f9c97000640 1 --2- 172.21.15.2:0/2265514505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c90101de0 0x7f9c901021c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009900 tx=0x7f9c8002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:19:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.065+0000 7f9c97000640 1 -- 172.21.15.2:0/2265514505 shutdown_connections 2024-09-10T08:19:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.065+0000 7f9c97000640 1 --2- 172.21.15.2:0/2265514505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c90106f10 0x7f9c9010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.065+0000 7f9c97000640 1 --2- 172.21.15.2:0/2265514505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c90102790 0x7f9c901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.065+0000 7f9c97000640 1 --2- 172.21.15.2:0/2265514505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c90101de0 0x7f9c901021c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.065+0000 7f9c97000640 1 -- 172.21.15.2:0/2265514505 >> 172.21.15.2:0/2265514505 conn(0x7f9c900fd8c0 msgr2=0x7f9c900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.065+0000 7f9c97000640 1 -- 172.21.15.2:0/2265514505 shutdown_connections 2024-09-10T08:19:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.065+0000 7f9c97000640 1 -- 172.21.15.2:0/2265514505 wait complete. 2024-09-10T08:19:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.066+0000 7f9c97000640 1 Processor -- start 2024-09-10T08:19:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.066+0000 7f9c97000640 1 -- start start 2024-09-10T08:19:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.066+0000 7f9c97000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c90101de0 0x7f9c9019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.066+0000 7f9c97000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c90102790 0x7f9c9019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.066+0000 7f9c97000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c90106f10 0x7f9c9019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.066+0000 7f9c97000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9c90111fc0 con 0x7f9c90101de0 2024-09-10T08:19:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.066+0000 7f9c97000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9c90111e40 con 0x7f9c90102790 2024-09-10T08:19:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.066+0000 7f9c97000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9c90112140 con 0x7f9c90106f10 2024-09-10T08:19:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.066+0000 7f9c8fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c90101de0 0x7f9c9019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.067+0000 7f9c8fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c90101de0 0x7f9c9019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35856/0 (socket says 172.21.15.2:35856) 2024-09-10T08:19:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.067+0000 7f9c8fe00640 1 -- 172.21.15.2:0/378789649 learned_addr learned my addr 172.21.15.2:0/378789649 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.067+0000 7f9c95000640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c90106f10 0x7f9c9019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.067+0000 7f9c8f400640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c90102790 0x7f9c9019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.067+0000 7f9c95000640 1 -- 172.21.15.2:0/378789649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c90102790 msgr2=0x7f9c9019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.067+0000 7f9c95000640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c90102790 0x7f9c9019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.067+0000 7f9c95000640 1 -- 172.21.15.2:0/378789649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c90101de0 msgr2=0x7f9c9019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.067+0000 7f9c95000640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c90101de0 0x7f9c9019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.067+0000 7f9c95000640 1 -- 172.21.15.2:0/378789649 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9019f5e0 con 0x7f9c90106f10 2024-09-10T08:19:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.067+0000 7f9c95000640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c90106f10 0x7f9c9019ef00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009fd0 tx=0x7f9c8800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.068+0000 7f9c8cc00640 1 -- 172.21.15.2:0/378789649 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800ce20 con 0x7f9c90106f10 2024-09-10T08:19:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.068+0000 7f9c97000640 1 -- 172.21.15.2:0/378789649 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9019f8d0 con 0x7f9c90106f10 2024-09-10T08:19:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.068+0000 7f9c97000640 1 -- 172.21.15.2:0/378789649 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c901a71b0 con 0x7f9c90106f10 2024-09-10T08:19:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.068+0000 7f9c8cc00640 1 -- 172.21.15.2:0/378789649 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c88010430 con 0x7f9c90106f10 2024-09-10T08:19:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.068+0000 7f9c8cc00640 1 -- 172.21.15.2:0/378789649 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800f620 con 0x7f9c90106f10 2024-09-10T08:19:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.069+0000 7f9c97000640 1 -- 172.21.15.2:0/378789649 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c901021c0 con 0x7f9c90106f10 2024-09-10T08:19:16.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.071+0000 7f9c8cc00640 1 -- 172.21.15.2:0/378789649 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9c88010950 con 0x7f9c90106f10 2024-09-10T08:19:16.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.072+0000 7f9c8cc00640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9c740a2c40 0x7f9c740a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:16.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.073+0000 7f9c8cc00640 1 -- 172.21.15.2:0/378789649 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9c880cfa70 con 0x7f9c90106f10 2024-09-10T08:19:16.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.073+0000 7f9c8fe00640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9c740a2c40 0x7f9c740a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:16.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.073+0000 7f9c8fe00640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9c740a2c40 0x7f9c740a5100 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9c80000c00 tx=0x7f9c8002a8f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:16.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.074+0000 7f9c8cc00640 1 -- 172.21.15.2:0/378789649 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c8809c580 con 0x7f9c90106f10 2024-09-10T08:19:16.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.300+0000 7f9c97000640 1 -- 172.21.15.2:0/378789649 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c901a74a0 con 0x7f9c90106f10 2024-09-10T08:19:16.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.301+0000 7f9c8cc00640 1 -- 172.21.15.2:0/378789649 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f9c8809be80 con 0x7f9c90106f10 2024-09-10T08:19:16.302 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:19:16.302 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:19:16.302 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:19:16.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.304+0000 7f9c97000640 1 -- 172.21.15.2:0/378789649 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9c740a2c40 msgr2=0x7f9c740a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:16.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.304+0000 7f9c97000640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9c740a2c40 0x7f9c740a5100 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9c80000c00 tx=0x7f9c8002a8f0 comp rx=0 tx=0).stop 2024-09-10T08:19:16.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.304+0000 7f9c97000640 1 -- 172.21.15.2:0/378789649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c90106f10 msgr2=0x7f9c9019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:16.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.304+0000 7f9c97000640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c90106f10 0x7f9c9019ef00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009fd0 tx=0x7f9c8800ef10 comp rx=0 tx=0).stop 2024-09-10T08:19:16.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.304+0000 7f9c97000640 1 -- 172.21.15.2:0/378789649 shutdown_connections 2024-09-10T08:19:16.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.304+0000 7f9c97000640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9c740a2c40 0x7f9c740a5100 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:16.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.304+0000 7f9c97000640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c90106f10 0x7f9c9019ef00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:16.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.304+0000 7f9c97000640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c90102790 0x7f9c9019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:16.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.304+0000 7f9c97000640 1 --2- 172.21.15.2:0/378789649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c90101de0 0x7f9c9019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:16.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.304+0000 7f9c97000640 1 -- 172.21.15.2:0/378789649 >> 172.21.15.2:0/378789649 conn(0x7f9c900fd8c0 msgr2=0x7f9c900ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:16.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.304+0000 7f9c97000640 1 -- 172.21.15.2:0/378789649 shutdown_connections 2024-09-10T08:19:16.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:16.304+0000 7f9c97000640 1 -- 172.21.15.2:0/378789649 wait complete. 2024-09-10T08:19:16.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:16 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/378789649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:19:16.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:16 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/378789649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:19:16.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:16 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/378789649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:19:17.259 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:19:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:17.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:17 smithi002 ceph-mon[30563]: pgmap v721: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-10T08:19:17.820 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:17 smithi107 ceph-mon[35048]: pgmap v721: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-10T08:19:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:17 smithi155 ceph-mon[35405]: pgmap v721: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-10T08:19:18.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.230+0000 7f5211000640 1 -- 172.21.15.2:0/4087211564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f520c102360 msgr2=0x7f520c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:18.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.230+0000 7f5211000640 1 --2- 172.21.15.2:0/4087211564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f520c102360 0x7f520c10e890 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5204009cd0 tx=0x7f520402a8f0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.230+0000 7f5211000640 1 -- 172.21.15.2:0/4087211564 shutdown_connections 2024-09-10T08:19:18.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.230+0000 7f5211000640 1 --2- 172.21.15.2:0/4087211564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f520c102360 0x7f520c10e890 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.230+0000 7f5211000640 1 --2- 172.21.15.2:0/4087211564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f520c1019c0 0x7f520c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.230+0000 7f5211000640 1 --2- 172.21.15.2:0/4087211564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f520c1079c0 0x7f520c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.230+0000 7f5211000640 1 -- 172.21.15.2:0/4087211564 >> 172.21.15.2:0/4087211564 conn(0x7f520c0fd740 msgr2=0x7f520c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:18.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.230+0000 7f5211000640 1 -- 172.21.15.2:0/4087211564 shutdown_connections 2024-09-10T08:19:18.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.230+0000 7f5211000640 1 -- 172.21.15.2:0/4087211564 wait complete. 2024-09-10T08:19:18.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.231+0000 7f5211000640 1 Processor -- start 2024-09-10T08:19:18.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.231+0000 7f5211000640 1 -- start start 2024-09-10T08:19:18.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.232+0000 7f5211000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f520c1019c0 0x7f520c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.232+0000 7f5211000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f520c102360 0x7f520c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.232+0000 7f5211000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f520c1079c0 0x7f520c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.232+0000 7f5211000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f520c111dc0 con 0x7f520c1019c0 2024-09-10T08:19:18.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.232+0000 7f5211000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f520c111c40 con 0x7f520c1079c0 2024-09-10T08:19:18.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.232+0000 7f5211000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f520c111f40 con 0x7f520c102360 2024-09-10T08:19:18.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.232+0000 7f520b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f520c1019c0 0x7f520c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.232+0000 7f520b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f520c1019c0 0x7f520c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48596/0 (socket says 172.21.15.2:48596) 2024-09-10T08:19:18.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.232+0000 7f520b400640 1 -- 172.21.15.2:0/3068044840 learned_addr learned my addr 172.21.15.2:0/3068044840 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:18.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.232+0000 7f520be00640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f520c1079c0 0x7f520c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.232+0000 7f5203e00640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f520c102360 0x7f520c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.233+0000 7f520b400640 1 -- 172.21.15.2:0/3068044840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f520c102360 msgr2=0x7f520c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:18.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.233+0000 7f520b400640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f520c102360 0x7f520c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.233+0000 7f520b400640 1 -- 172.21.15.2:0/3068044840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f520c1079c0 msgr2=0x7f520c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:18.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.233+0000 7f520b400640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f520c1079c0 0x7f520c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.233+0000 7f520b400640 1 -- 172.21.15.2:0/3068044840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c19f410 con 0x7f520c1019c0 2024-09-10T08:19:18.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.233+0000 7f5203e00640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f520c102360 0x7f520c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:18.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.233+0000 7f520be00640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f520c1079c0 0x7f520c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:18.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.233+0000 7f520b400640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f520c1019c0 0x7f520c19a460 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00e9f0 tx=0x7f51fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:18.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.233+0000 7f5208c00640 1 -- 172.21.15.2:0/3068044840 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00cd80 con 0x7f520c1019c0 2024-09-10T08:19:18.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.233+0000 7f5208c00640 1 -- 172.21.15.2:0/3068044840 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51fc010910 con 0x7f520c1019c0 2024-09-10T08:19:18.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.233+0000 7f5208c00640 1 -- 172.21.15.2:0/3068044840 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc018990 con 0x7f520c1019c0 2024-09-10T08:19:18.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.233+0000 7f5211000640 1 -- 172.21.15.2:0/3068044840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520c19f700 con 0x7f520c1019c0 2024-09-10T08:19:18.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.233+0000 7f5211000640 1 -- 172.21.15.2:0/3068044840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520c1a6fe0 con 0x7f520c1019c0 2024-09-10T08:19:18.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.236+0000 7f5208c00640 1 -- 172.21.15.2:0/3068044840 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f51fc010ac0 con 0x7f520c1019c0 2024-09-10T08:19:18.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.237+0000 7f5214bfd740 1 Processor -- start 2024-09-10T08:19:18.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.237+0000 7f5214bfd740 1 -- start start 2024-09-10T08:19:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.237+0000 7f5214bfd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cee40 0x56414e1cf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.237+0000 7f5214bfd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cf760 0x56414e1d9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.237+0000 7f5214bfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1da020 0x56414e1dc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.237+0000 7f5214bfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56414e1353e0 con 0x56414e1da020 2024-09-10T08:19:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.237+0000 7f5214bfd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56414e0dcd80 con 0x56414e1cee40 2024-09-10T08:19:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.237+0000 7f5214bfd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56414e1313a0 con 0x56414e1cf760 2024-09-10T08:19:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.238+0000 7f520be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cee40 0x56414e1cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.238+0000 7f520b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cf760 0x56414e1d9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.238+0000 7f520be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cee40 0x56414e1cf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:48828/0 (socket says 172.21.15.2:48828) 2024-09-10T08:19:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.238+0000 7f520be00640 1 -- 172.21.15.2:0/3996069995 learned_addr learned my addr 172.21.15.2:0/3996069995 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.238+0000 7f5203e00640 1 --2- 172.21.15.2:0/3996069995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1da020 0x56414e1dc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.238+0000 7f520be00640 1 -- 172.21.15.2:0/3996069995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cf760 msgr2=0x56414e1d9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.238+0000 7f520be00640 1 --2- 172.21.15.2:0/3996069995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cf760 0x56414e1d9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.238+0000 7f520be00640 1 -- 172.21.15.2:0/3996069995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1da020 msgr2=0x56414e1dc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.238+0000 7f520be00640 1 --2- 172.21.15.2:0/3996069995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1da020 0x56414e1dc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.238+0000 7f520be00640 1 -- 172.21.15.2:0/3996069995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414e0fbae0 con 0x56414e1cee40 2024-09-10T08:19:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.239+0000 7f520be00640 1 --2- 172.21.15.2:0/3996069995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cee40 0x56414e1cf220 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5204004ae0 tx=0x7f5204005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4a6707103a9adde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.239+0000 7f5208c00640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f51f00a2c00 0x7f51f00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.239+0000 7f5208c00640 1 -- 172.21.15.2:0/3068044840 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f51fc0cf660 con 0x7f520c1019c0 2024-09-10T08:19:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.239+0000 7f5200c00640 1 -- 172.21.15.2:0/3996069995 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204030b30 con 0x56414e1cee40 2024-09-10T08:19:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.239+0000 7f520b400640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f51f00a2c00 0x7f51f00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.239+0000 7f5200c00640 1 -- 172.21.15.2:0/3996069995 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5204036030 con 0x56414e1cee40 2024-09-10T08:19:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.239+0000 7f5200c00640 1 -- 172.21.15.2:0/3996069995 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204004700 con 0x56414e1cee40 2024-09-10T08:19:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.240+0000 7f5214bfd740 1 -- 172.21.15.2:0/3996069995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cee40 msgr2=0x56414e1cf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.240+0000 7f520b400640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f51f00a2c00 0x7f51f00a50c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0984f0 tx=0x7f51fc0d5110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.240+0000 7f5214bfd740 1 --2- 172.21.15.2:0/3996069995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cee40 0x56414e1cf220 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5204004ae0 tx=0x7f5204005ef0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.240+0000 7f5214bfd740 1 -- 172.21.15.2:0/3996069995 shutdown_connections 2024-09-10T08:19:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.240+0000 7f5214bfd740 1 --2- 172.21.15.2:0/3996069995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1da020 0x56414e1dc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.240+0000 7f5214bfd740 1 --2- 172.21.15.2:0/3996069995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cf760 0x56414e1d9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.240+0000 7f5214bfd740 1 --2- 172.21.15.2:0/3996069995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cee40 0x56414e1cf220 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.240+0000 7f5214bfd740 1 -- 172.21.15.2:0/3996069995 >> 172.21.15.2:0/3996069995 conn(0x56414e1cb3d0 msgr2=0x56414e1cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.240+0000 7f5214bfd740 1 -- 172.21.15.2:0/3996069995 shutdown_connections 2024-09-10T08:19:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.240+0000 7f5214bfd740 1 -- 172.21.15.2:0/3996069995 wait complete. 2024-09-10T08:19:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.241+0000 7f5214bfd740 1 -- start start 2024-09-10T08:19:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.241+0000 7f5214bfd740 1 Processor -- start 2024-09-10T08:19:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.241+0000 7f5214bfd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cee40 0x56414e1d6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.241+0000 7f5214bfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1cf760 0x56414e1d7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.241+0000 7f5214bfd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1da020 0x56414e1d79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.241+0000 7f5214bfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56414e1dcbf0 con 0x56414e1cf760 2024-09-10T08:19:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.241+0000 7f5214bfd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56414e0f00f0 con 0x56414e1da020 2024-09-10T08:19:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.241+0000 7f5214bfd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56414e1dcd70 con 0x56414e1cee40 2024-09-10T08:19:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.241+0000 7f5214bfd740 10 client.0 _ll_register_callbacks cb 0x56414e0d7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.242+0000 7f5203e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1cf760 0x56414e1d7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.242+0000 7f520be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cee40 0x56414e1d6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.242+0000 7f520be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cee40 0x56414e1d6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50708/0 (socket says 172.21.15.2:50708) 2024-09-10T08:19:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.242+0000 7f5203e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1cf760 0x56414e1d7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48618/0 (socket says 172.21.15.2:48618) 2024-09-10T08:19:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.242+0000 7f520be00640 1 -- 172.21.15.2:0/3761961830 learned_addr learned my addr 172.21.15.2:0/3761961830 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.242+0000 7f520be00640 1 --2- 172.21.15.2:0/3761961830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1da020 0x56414e1d79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.242+0000 7f520be00640 1 -- 172.21.15.2:0/3761961830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1da020 msgr2=0x56414e1d79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.242+0000 7f520be00640 1 --2- 172.21.15.2:0/3761961830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1da020 0x56414e1d79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.242+0000 7f520be00640 1 -- 172.21.15.2:0/3761961830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1cf760 msgr2=0x56414e1d7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.242+0000 7f520be00640 1 --2- 172.21.15.2:0/3761961830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1cf760 0x56414e1d7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.242+0000 7f520be00640 1 -- 172.21.15.2:0/3761961830 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414dea63c0 con 0x56414e1cee40 2024-09-10T08:19:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.243+0000 7f520be00640 1 --2- 172.21.15.2:0/3761961830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cee40 0x56414e1d6f20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5204009620 tx=0x7f5204004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a12744232d3a4d9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.243+0000 7f51f9600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:19:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.243+0000 7f51f9600640 1 -- 172.21.15.2:0/3761961830 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204059060 con 0x56414e1cee40 2024-09-10T08:19:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.243+0000 7f5214bfd740 1 -- 172.21.15.2:0/3761961830 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5204036030 con 0x56414e1cee40 2024-09-10T08:19:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.243+0000 7f51f9600640 1 -- 172.21.15.2:0/3761961830 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5204003b70 con 0x56414e1cee40 2024-09-10T08:19:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.243+0000 7f51f9600640 1 -- 172.21.15.2:0/3761961830 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5204003d70 con 0x56414e1cee40 2024-09-10T08:19:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.243+0000 7f5214bfd740 10 client.28528 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.243+0000 7f5214bfd740 1 -- 172.21.15.2:0/3761961830 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56414e1dcbf0 con 0x56414e1cee40 2024-09-10T08:19:18.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.244+0000 7f51f9600640 1 -- 172.21.15.2:0/3761961830 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5204036030 con 0x56414e1cee40 2024-09-10T08:19:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.244+0000 7f51f9600640 1 -- 172.21.15.2:0/3761961830 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5204043070 con 0x56414e1cee40 2024-09-10T08:19:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.244+0000 7f5214bfd740 10 client.28528 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.244+0000 7f5214bfd740 10 client.28528 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:19:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.244+0000 7f5214bfd740 20 client.28528 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.244+0000 7f5214bfd740 1 --2- 172.21.15.2:0/3761961830 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56414e1e9dc0 0x56414e1ec1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.244+0000 7f5214bfd740 4 client.28528 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.244+0000 7f5214bfd740 1 -- 172.21.15.2:0/3761961830 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56414e142950 con 0x56414e1e9dc0 2024-09-10T08:19:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.244+0000 7f5203e00640 1 --2- 172.21.15.2:0/3761961830 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56414e1e9dc0 0x56414e1ec1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.245+0000 7f5203e00640 1 --2- 172.21.15.2:0/3761961830 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56414e1e9dc0 0x56414e1ec1b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc0c13b0a994644f server_cookie=cfdced15a331ee6b in_seq=0 out_seq=0 2024-09-10T08:19:18.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.245+0000 7f51f9600640 10 client.28528 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:19:18.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f51f9600640 1 -- 172.21.15.2:0/3761961830 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56414e142950 con 0x56414e1e9dc0 2024-09-10T08:19:18.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f51f9600640 10 client.28528 handle_command_reply: tid=0 2024-09-10T08:19:18.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f5214bfd740 1 client.28528 shutdown 2024-09-10T08:19:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f5214bfd740 1 -- 172.21.15.2:0/3761961830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cee40 msgr2=0x56414e1d6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f5214bfd740 1 --2- 172.21.15.2:0/3761961830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cee40 0x56414e1d6f20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5204009620 tx=0x7f5204004260 comp rx=0 tx=0).stop 2024-09-10T08:19:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f5214bfd740 1 -- 172.21.15.2:0/3761961830 shutdown_connections 2024-09-10T08:19:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f5214bfd740 1 --2- 172.21.15.2:0/3761961830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1da020 0x56414e1d79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f5214bfd740 1 --2- 172.21.15.2:0/3761961830 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56414e1e9dc0 0x56414e1ec1b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f5214bfd740 1 --2- 172.21.15.2:0/3761961830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1cf760 0x56414e1d7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f5214bfd740 1 --2- 172.21.15.2:0/3761961830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cee40 0x56414e1d6f20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f51f9600640 0 client.28528 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:19:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f5214bfd740 1 -- 172.21.15.2:0/3761961830 >> 172.21.15.2:0/3761961830 conn(0x56414e1cb3d0 msgr2=0x56414e1d10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f5214bfd740 1 -- 172.21.15.2:0/3761961830 shutdown_connections 2024-09-10T08:19:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f5214bfd740 1 -- 172.21.15.2:0/3761961830 wait complete. 2024-09-10T08:19:18.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.246+0000 7f5214bfd740 20 client.28528 trim_cache size 0 max 16384 2024-09-10T08:19:18.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.260+0000 7f5214bfd740 1 Processor -- start 2024-09-10T08:19:18.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.260+0000 7f5214bfd740 1 -- start start 2024-09-10T08:19:18.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.261+0000 7f5214bfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1cee40 0x56414e1d77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.261+0000 7f5214bfd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cf760 0x56414e1f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.261+0000 7f5214bfd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1da020 0x56414e1f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.261+0000 7f5214bfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1b36d0 con 0x56414e1cee40 2024-09-10T08:19:18.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.261+0000 7f5214bfd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1b0fa0 con 0x56414e1cf760 2024-09-10T08:19:18.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.261+0000 7f5214bfd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1aedf0 con 0x56414e1da020 2024-09-10T08:19:18.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.261+0000 7f520be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1cee40 0x56414e1d77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.261+0000 7f5203e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cf760 0x56414e1f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.261+0000 7f520be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1cee40 0x56414e1d77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48622/0 (socket says 172.21.15.2:48622) 2024-09-10T08:19:18.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.261+0000 7f520be00640 1 -- 172.21.15.2:0/710620113 learned_addr learned my addr 172.21.15.2:0/710620113 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:18.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.261+0000 7f5203e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cf760 0x56414e1f18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:48846/0 (socket says 172.21.15.2:48846) 2024-09-10T08:19:18.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.262+0000 7f520be00640 1 --2- 172.21.15.2:0/710620113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1da020 0x56414e1f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.262+0000 7f520be00640 1 -- 172.21.15.2:0/710620113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1da020 msgr2=0x56414e1f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:18.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.262+0000 7f520be00640 1 --2- 172.21.15.2:0/710620113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1da020 0x56414e1f1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.262+0000 7f520be00640 1 -- 172.21.15.2:0/710620113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cf760 msgr2=0x56414e1f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:18.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.262+0000 7f520be00640 1 --2- 172.21.15.2:0/710620113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cf760 0x56414e1f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.262+0000 7f520be00640 1 -- 172.21.15.2:0/710620113 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414e1d9890 con 0x56414e1cee40 2024-09-10T08:19:18.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.262+0000 7f5203e00640 1 --2- 172.21.15.2:0/710620113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cf760 0x56414e1f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:18.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.262+0000 7f520be00640 1 --2- 172.21.15.2:0/710620113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1cee40 0x56414e1d77f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x56414e1cf6c0 tx=0x7f52040093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e9aca5b2e74054 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:18.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.262+0000 7f51fbe00640 1 -- 172.21.15.2:0/710620113 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204059070 con 0x56414e1cee40 2024-09-10T08:19:18.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.262+0000 7f51fbe00640 1 -- 172.21.15.2:0/710620113 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5204036040 con 0x56414e1cee40 2024-09-10T08:19:18.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.262+0000 7f51fbe00640 1 -- 172.21.15.2:0/710620113 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204003970 con 0x56414e1cee40 2024-09-10T08:19:18.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.263+0000 7f5214bfd740 1 -- 172.21.15.2:0/710620113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1cee40 msgr2=0x56414e1d77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:18.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.263+0000 7f5214bfd740 1 --2- 172.21.15.2:0/710620113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1cee40 0x56414e1d77f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x56414e1cf6c0 tx=0x7f52040093f0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.263+0000 7f5214bfd740 1 -- 172.21.15.2:0/710620113 shutdown_connections 2024-09-10T08:19:18.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.263+0000 7f5214bfd740 1 --2- 172.21.15.2:0/710620113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1da020 0x56414e1f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.263+0000 7f5214bfd740 1 --2- 172.21.15.2:0/710620113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cf760 0x56414e1f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.263+0000 7f5214bfd740 1 --2- 172.21.15.2:0/710620113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1cee40 0x56414e1d77f0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.263+0000 7f5214bfd740 1 -- 172.21.15.2:0/710620113 >> 172.21.15.2:0/710620113 conn(0x56414e1cb3d0 msgr2=0x56414e1d8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:18.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.263+0000 7f5214bfd740 1 -- 172.21.15.2:0/710620113 shutdown_connections 2024-09-10T08:19:18.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.263+0000 7f5214bfd740 1 -- 172.21.15.2:0/710620113 wait complete. 2024-09-10T08:19:18.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.264+0000 7f5214bfd740 1 -- start start 2024-09-10T08:19:18.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.264+0000 7f5214bfd740 1 Processor -- start 2024-09-10T08:19:18.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.264+0000 7f5214bfd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cee40 0x56414e1e2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.264+0000 7f5214bfd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cf760 0x56414e1dd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.264+0000 7f5214bfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1da020 0x56414e1dde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.264+0000 7f5214bfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1af460 con 0x56414e1da020 2024-09-10T08:19:18.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.264+0000 7f5214bfd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1b08d0 con 0x56414e1cf760 2024-09-10T08:19:18.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.264+0000 7f5214bfd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f520c1afb30 con 0x56414e1cee40 2024-09-10T08:19:18.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.264+0000 7f5214bfd740 10 client.0 _ll_register_callbacks cb 0x7f51fc0d5060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:18.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.265+0000 7f520be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cee40 0x56414e1e2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.265+0000 7f520be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cee40 0x56414e1e2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50736/0 (socket says 172.21.15.2:50736) 2024-09-10T08:19:18.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.265+0000 7f520be00640 1 -- 172.21.15.2:0/4049258025 learned_addr learned my addr 172.21.15.2:0/4049258025 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:18.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.265+0000 7f520be00640 1 --2- 172.21.15.2:0/4049258025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1da020 0x56414e1dde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.265+0000 7f5203e00640 1 --2- 172.21.15.2:0/4049258025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cf760 0x56414e1dd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.265+0000 7f5203e00640 1 -- 172.21.15.2:0/4049258025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cee40 msgr2=0x56414e1e2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:18.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.265+0000 7f5203e00640 1 --2- 172.21.15.2:0/4049258025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cee40 0x56414e1e2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.265+0000 7f5203e00640 1 -- 172.21.15.2:0/4049258025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1da020 msgr2=0x56414e1dde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:18.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.265+0000 7f5203e00640 1 --2- 172.21.15.2:0/4049258025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1da020 0x56414e1dde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.265+0000 7f5203e00640 1 -- 172.21.15.2:0/4049258025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414e1de4a0 con 0x56414e1cf760 2024-09-10T08:19:18.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.265+0000 7f5203e00640 1 --2- 172.21.15.2:0/4049258025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cf760 0x56414e1dd8c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x56414e1ec890 tx=0x7f51f40035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17abc2833c2fde72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:18.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.265+0000 7f51f9600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:19:18.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f51f9600640 1 -- 172.21.15.2:0/4049258025 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4024070 con 0x56414e1cf760 2024-09-10T08:19:18.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f51f9600640 1 -- 172.21.15.2:0/4049258025 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f51f40139f0 con 0x56414e1cf760 2024-09-10T08:19:18.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f5214bfd740 1 -- 172.21.15.2:0/4049258025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5204036040 con 0x56414e1cf760 2024-09-10T08:19:18.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f51f9600640 1 -- 172.21.15.2:0/4049258025 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51f4003e30 con 0x56414e1cf760 2024-09-10T08:19:18.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f5214bfd740 10 client.28607 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:18.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f5214bfd740 1 -- 172.21.15.2:0/4049258025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56414e254730 con 0x56414e1cf760 2024-09-10T08:19:18.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f51f9600640 1 -- 172.21.15.2:0/4049258025 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5204036040 con 0x56414e1cf760 2024-09-10T08:19:18.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f51f9600640 1 -- 172.21.15.2:0/4049258025 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f51f4010430 con 0x56414e1cf760 2024-09-10T08:19:18.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f5214bfd740 10 client.28607 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:18.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f5214bfd740 10 client.28607 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:19:18.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f5214bfd740 20 client.28607 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:18.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f5214bfd740 1 --2- 172.21.15.2:0/4049258025 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56414e254e80 0x56414e2572a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:18.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f5214bfd740 4 client.28607 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:19:18.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.266+0000 7f5214bfd740 1 -- 172.21.15.2:0/4049258025 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56414e13ce60 con 0x56414e254e80 2024-09-10T08:19:18.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.267+0000 7f520be00640 1 --2- 172.21.15.2:0/4049258025 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56414e254e80 0x56414e2572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:18.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.267+0000 7f520be00640 1 --2- 172.21.15.2:0/4049258025 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56414e254e80 0x56414e2572a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1be91653d7cdce90 server_cookie=f838a30abbd192ee in_seq=0 out_seq=0 2024-09-10T08:19:18.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.267+0000 7f51f9600640 10 client.28607 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:19:18.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f51f9600640 1 -- 172.21.15.2:0/4049258025 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56414e13ce60 con 0x56414e254e80 2024-09-10T08:19:18.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f51f9600640 10 client.28607 handle_command_reply: tid=0 2024-09-10T08:19:18.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f5214bfd740 1 client.28607 shutdown 2024-09-10T08:19:18.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f5214bfd740 1 -- 172.21.15.2:0/4049258025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cf760 msgr2=0x56414e1dd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:18.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f5214bfd740 1 --2- 172.21.15.2:0/4049258025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cf760 0x56414e1dd8c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x56414e1ec890 tx=0x7f51f40035f0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f5214bfd740 1 -- 172.21.15.2:0/4049258025 shutdown_connections 2024-09-10T08:19:18.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f5214bfd740 1 --2- 172.21.15.2:0/4049258025 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x56414e254e80 0x56414e2572a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f5214bfd740 1 --2- 172.21.15.2:0/4049258025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56414e1da020 0x56414e1dde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f5214bfd740 1 --2- 172.21.15.2:0/4049258025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56414e1cf760 0x56414e1dd8c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f5214bfd740 1 --2- 172.21.15.2:0/4049258025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414e1cee40 0x56414e1e2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f51f9600640 0 client.28607 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:19:18.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f5214bfd740 1 -- 172.21.15.2:0/4049258025 >> 172.21.15.2:0/4049258025 conn(0x56414e1cb3d0 msgr2=0x56414e1d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:18.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f5214bfd740 1 -- 172.21.15.2:0/4049258025 shutdown_connections 2024-09-10T08:19:18.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f5214bfd740 1 -- 172.21.15.2:0/4049258025 wait complete. 2024-09-10T08:19:18.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.268+0000 7f5214bfd740 20 client.28607 trim_cache size 0 max 16384 2024-09-10T08:19:18.272 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:19:18.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.269+0000 7f5211000640 1 -- 172.21.15.2:0/3068044840 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f51f00a2c00 msgr2=0x7f51f00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:18.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.269+0000 7f5211000640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f51f00a2c00 0x7f51f00a50c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0984f0 tx=0x7f51fc0d5110 comp rx=0 tx=0).stop 2024-09-10T08:19:18.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.269+0000 7f5211000640 1 -- 172.21.15.2:0/3068044840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f520c1019c0 msgr2=0x7f520c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:18.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.269+0000 7f5211000640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f520c1019c0 0x7f520c19a460 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00e9f0 tx=0x7f51fc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.269+0000 7f5211000640 1 -- 172.21.15.2:0/3068044840 shutdown_connections 2024-09-10T08:19:18.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.269+0000 7f5211000640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f520c1079c0 0x7f520c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.269+0000 7f5211000640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f51f00a2c00 0x7f51f00a50c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.269+0000 7f5211000640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f520c102360 0x7f520c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.269+0000 7f5211000640 1 --2- 172.21.15.2:0/3068044840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f520c1019c0 0x7f520c19a460 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:18.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.269+0000 7f5211000640 1 -- 172.21.15.2:0/3068044840 >> 172.21.15.2:0/3068044840 conn(0x7f520c0fd740 msgr2=0x7f520c10d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:18.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.269+0000 7f5211000640 1 -- 172.21.15.2:0/3068044840 shutdown_connections 2024-09-10T08:19:18.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:18.269+0000 7f5211000640 1 -- 172.21.15.2:0/3068044840 wait complete. 2024-09-10T08:19:19.211 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:19:19.595 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:19.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:19 smithi002 ceph-mon[30563]: pgmap v722: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:19:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:19 smithi155 ceph-mon[35405]: pgmap v722: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:19:19.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:19 smithi107 ceph-mon[35048]: pgmap v722: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:19:20.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.182+0000 7f9e8da00640 1 -- 172.21.15.2:0/1591852401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e88109120 msgr2=0x7f9e88110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:20.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.182+0000 7f9e8da00640 1 --2- 172.21.15.2:0/1591852401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e88109120 0x7f9e88110ca0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009ca0 tx=0x7f9e8002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.182+0000 7f9e8da00640 1 -- 172.21.15.2:0/1591852401 shutdown_connections 2024-09-10T08:19:20.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.182+0000 7f9e8da00640 1 --2- 172.21.15.2:0/1591852401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e88109120 0x7f9e88110ca0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.182+0000 7f9e8da00640 1 --2- 172.21.15.2:0/1591852401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e881049a0 0x7f9e881089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.182+0000 7f9e8da00640 1 --2- 172.21.15.2:0/1591852401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e88103ff0 0x7f9e881043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.182+0000 7f9e8da00640 1 -- 172.21.15.2:0/1591852401 >> 172.21.15.2:0/1591852401 conn(0x7f9e880ffa90 msgr2=0x7f9e88101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:20.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.182+0000 7f9e8da00640 1 -- 172.21.15.2:0/1591852401 shutdown_connections 2024-09-10T08:19:20.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.182+0000 7f9e8da00640 1 -- 172.21.15.2:0/1591852401 wait complete. 2024-09-10T08:19:20.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.183+0000 7f9e8da00640 1 Processor -- start 2024-09-10T08:19:20.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.183+0000 7f9e8da00640 1 -- start start 2024-09-10T08:19:20.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.184+0000 7f9e8da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e88103ff0 0x7f9e8819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.184+0000 7f9e8da00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e881049a0 0x7f9e8819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.184+0000 7f9e8da00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e88109120 0x7f9e8819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.184+0000 7f9e8da00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88114130 con 0x7f9e88109120 2024-09-10T08:19:20.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.184+0000 7f9e8da00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88113fb0 con 0x7f9e881049a0 2024-09-10T08:19:20.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.184+0000 7f9e8da00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881142b0 con 0x7f9e88103ff0 2024-09-10T08:19:20.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.184+0000 7f9e87400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e88109120 0x7f9e8819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.184+0000 7f9e86a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e88103ff0 0x7f9e8819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.184+0000 7f9e87400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e88109120 0x7f9e8819ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48662/0 (socket says 172.21.15.2:48662) 2024-09-10T08:19:20.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.184+0000 7f9e87400640 1 -- 172.21.15.2:0/1120284338 learned_addr learned my addr 172.21.15.2:0/1120284338 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:20.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.184+0000 7f9e86000640 1 --2- 172.21.15.2:0/1120284338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e881049a0 0x7f9e8819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.185+0000 7f9e86a00640 1 -- 172.21.15.2:0/1120284338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e881049a0 msgr2=0x7f9e8819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:20.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.185+0000 7f9e86a00640 1 --2- 172.21.15.2:0/1120284338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e881049a0 0x7f9e8819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.185+0000 7f9e86a00640 1 -- 172.21.15.2:0/1120284338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e88109120 msgr2=0x7f9e8819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:20.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.185+0000 7f9e86a00640 1 --2- 172.21.15.2:0/1120284338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e88109120 0x7f9e8819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.185+0000 7f9e86a00640 1 -- 172.21.15.2:0/1120284338 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8819f540 con 0x7f9e88103ff0 2024-09-10T08:19:20.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.185+0000 7f9e86a00640 1 --2- 172.21.15.2:0/1120284338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e88103ff0 0x7f9e8819a590 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800e9f0 tx=0x7f9e7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:20.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.185+0000 7f9e7f400640 1 -- 172.21.15.2:0/1120284338 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800ccc0 con 0x7f9e88103ff0 2024-09-10T08:19:20.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.185+0000 7f9e8da00640 1 -- 172.21.15.2:0/1120284338 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8819f830 con 0x7f9e88103ff0 2024-09-10T08:19:20.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.185+0000 7f9e7f400640 1 -- 172.21.15.2:0/1120284338 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e78010910 con 0x7f9e88103ff0 2024-09-10T08:19:20.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.185+0000 7f9e7f400640 1 -- 172.21.15.2:0/1120284338 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78018960 con 0x7f9e88103ff0 2024-09-10T08:19:20.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.185+0000 7f9e8da00640 1 -- 172.21.15.2:0/1120284338 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e881a7110 con 0x7f9e88103ff0 2024-09-10T08:19:20.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.188+0000 7f9e90654740 1 Processor -- start 2024-09-10T08:19:20.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.189+0000 7f9e90654740 1 -- start start 2024-09-10T08:19:20.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.189+0000 7f9e7f400640 1 -- 172.21.15.2:0/1120284338 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9e78010ac0 con 0x7f9e88103ff0 2024-09-10T08:19:20.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.189+0000 7f9e90654740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc45298050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.189+0000 7f9e90654740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452a2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.189+0000 7f9e90654740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc452a2e50 0x55cc452a5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.189+0000 7f9e90654740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cc451fe3e0 con 0x55cc45298590 2024-09-10T08:19:20.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.189+0000 7f9e90654740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cc451a5d80 con 0x55cc45297c70 2024-09-10T08:19:20.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.189+0000 7f9e90654740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cc451fa3a0 con 0x55cc452a2e50 2024-09-10T08:19:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.189+0000 7f9e86a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.189+0000 7f9e87400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc45298050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.189+0000 7f9e87400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc45298050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:48896/0 (socket says 172.21.15.2:48896) 2024-09-10T08:19:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.189+0000 7f9e87400640 1 -- 172.21.15.2:0/3559180272 learned_addr learned my addr 172.21.15.2:0/3559180272 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.189+0000 7f9e86000640 1 --2- 172.21.15.2:0/3559180272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc452a2e50 0x55cc452a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.190+0000 7f9e87400640 1 -- 172.21.15.2:0/3559180272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc452a2e50 msgr2=0x55cc452a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.190+0000 7f9e87400640 1 --2- 172.21.15.2:0/3559180272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc452a2e50 0x55cc452a5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.190+0000 7f9e87400640 1 -- 172.21.15.2:0/3559180272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 msgr2=0x55cc452a2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.190+0000 7f9e87400640 1 --2- 172.21.15.2:0/3559180272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452a2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.190+0000 7f9e87400640 1 -- 172.21.15.2:0/3559180272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc451c4ae0 con 0x55cc45297c70 2024-09-10T08:19:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.190+0000 7f9e87400640 1 --2- 172.21.15.2:0/3559180272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc45298050 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9e800097b0 tx=0x7f9e80005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4db658acc64e602 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.190+0000 7f9e73e00640 1 -- 172.21.15.2:0/3559180272 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80031db0 con 0x55cc45297c70 2024-09-10T08:19:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.190+0000 7f9e73e00640 1 -- 172.21.15.2:0/3559180272 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e80036040 con 0x55cc45297c70 2024-09-10T08:19:20.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.190+0000 7f9e73e00640 1 -- 172.21.15.2:0/3559180272 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e800043c0 con 0x55cc45297c70 2024-09-10T08:19:20.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.191+0000 7f9e7f400640 1 --2- 172.21.15.2:0/1120284338 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9e6c0a2b80 0x7f9e6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.191+0000 7f9e7f400640 1 -- 172.21.15.2:0/1120284338 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9e780cf7c0 con 0x7f9e88103ff0 2024-09-10T08:19:20.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.191+0000 7f9e86a00640 1 --2- 172.21.15.2:0/1120284338 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9e6c0a2b80 0x7f9e6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.191+0000 7f9e90654740 1 -- 172.21.15.2:0/3559180272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 msgr2=0x55cc45298050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:20.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.191+0000 7f9e90654740 1 --2- 172.21.15.2:0/3559180272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc45298050 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9e800097b0 tx=0x7f9e80005f90 comp rx=0 tx=0).stop 2024-09-10T08:19:20.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.191+0000 7f9e90654740 1 -- 172.21.15.2:0/3559180272 shutdown_connections 2024-09-10T08:19:20.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.191+0000 7f9e90654740 1 --2- 172.21.15.2:0/3559180272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc452a2e50 0x55cc452a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.191+0000 7f9e90654740 1 --2- 172.21.15.2:0/3559180272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.191+0000 7f9e90654740 1 --2- 172.21.15.2:0/3559180272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc45298050 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.191+0000 7f9e90654740 1 -- 172.21.15.2:0/3559180272 >> 172.21.15.2:0/3559180272 conn(0x55cc45294200 msgr2=0x55cc452945e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:20.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.192+0000 7f9e90654740 1 -- 172.21.15.2:0/3559180272 shutdown_connections 2024-09-10T08:19:20.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.192+0000 7f9e86a00640 1 --2- 172.21.15.2:0/1120284338 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9e6c0a2b80 0x7f9e6c0a5040 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9e78098140 tx=0x7f9e78009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:20.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.192+0000 7f9e90654740 1 -- 172.21.15.2:0/3559180272 wait complete. 2024-09-10T08:19:20.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.192+0000 7f9e90654740 1 -- start start 2024-09-10T08:19:20.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.192+0000 7f9e90654740 1 Processor -- start 2024-09-10T08:19:20.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.193+0000 7f9e90654740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc45297c70 0x55cc4529fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.193+0000 7f9e90654740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452a0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.193+0000 7f9e90654740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc452a2e50 0x55cc452a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.193+0000 7f9e90654740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cc452a58a0 con 0x55cc45298590 2024-09-10T08:19:20.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.193+0000 7f9e90654740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cc451b90f0 con 0x55cc452a2e50 2024-09-10T08:19:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.193+0000 7f9e90654740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cc452a5a20 con 0x55cc45297c70 2024-09-10T08:19:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.193+0000 7f9e90654740 10 client.0 _ll_register_callbacks cb 0x55cc451a0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.193+0000 7f9e86000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.193+0000 7f9e86000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48688/0 (socket says 172.21.15.2:48688) 2024-09-10T08:19:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.193+0000 7f9e86000640 1 -- 172.21.15.2:0/3235381043 learned_addr learned my addr 172.21.15.2:0/3235381043 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.193+0000 7f9e87400640 1 --2- 172.21.15.2:0/3235381043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc45297c70 0x55cc4529fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.193+0000 7f9e87400640 1 --2- 172.21.15.2:0/3235381043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc452a2e50 0x55cc452a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.194+0000 7f9e86000640 1 -- 172.21.15.2:0/3235381043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc45297c70 msgr2=0x55cc4529fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.194+0000 7f9e86000640 1 --2- 172.21.15.2:0/3235381043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc45297c70 0x55cc4529fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.194+0000 7f9e86000640 1 -- 172.21.15.2:0/3235381043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc452a2e50 msgr2=0x55cc452a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.194+0000 7f9e86000640 1 --2- 172.21.15.2:0/3235381043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc452a2e50 0x55cc452a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.194+0000 7f9e86000640 1 -- 172.21.15.2:0/3235381043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc44f6f3c0 con 0x55cc45298590 2024-09-10T08:19:20.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.194+0000 7f9e86000640 1 --2- 172.21.15.2:0/3235381043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452a0260 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9e74012730 tx=0x7f9e74012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33882fdf95dded43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:20.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.194+0000 7f9e70c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:20.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.194+0000 7f9e70c00640 1 -- 172.21.15.2:0/3235381043 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7400f8c0 con 0x55cc45298590 2024-09-10T08:19:20.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.194+0000 7f9e70c00640 1 -- 172.21.15.2:0/3235381043 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9e7400fa60 con 0x55cc45298590 2024-09-10T08:19:20.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.194+0000 7f9e90654740 1 -- 172.21.15.2:0/3235381043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e80036040 con 0x55cc45298590 2024-09-10T08:19:20.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.194+0000 7f9e70c00640 1 -- 172.21.15.2:0/3235381043 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9e7400fca0 con 0x55cc45298590 2024-09-10T08:19:20.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.194+0000 7f9e90654740 10 client.18588 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:20.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.194+0000 7f9e90654740 1 -- 172.21.15.2:0/3235381043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cc452b1970 con 0x55cc45298590 2024-09-10T08:19:20.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.195+0000 7f9e70c00640 1 -- 172.21.15.2:0/3235381043 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e80036040 con 0x55cc45298590 2024-09-10T08:19:20.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.195+0000 7f9e70c00640 1 -- 172.21.15.2:0/3235381043 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9e74003b50 con 0x55cc45298590 2024-09-10T08:19:20.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.195+0000 7f9e90654740 10 client.18588 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:20.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.195+0000 7f9e90654740 10 client.18588 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:19:20.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.195+0000 7f9e90654740 20 client.18588 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:20.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.195+0000 7f9e90654740 1 --2- 172.21.15.2:0/3235381043 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55cc452b2ed0 0x55cc452b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.195+0000 7f9e90654740 4 client.18588 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:20.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.195+0000 7f9e90654740 1 -- 172.21.15.2:0/3235381043 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc4520b950 con 0x55cc452b2ed0 2024-09-10T08:19:20.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.195+0000 7f9e87400640 1 --2- 172.21.15.2:0/3235381043 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55cc452b2ed0 0x55cc452b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.196+0000 7f9e87400640 1 --2- 172.21.15.2:0/3235381043 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55cc452b2ed0 0x55cc452b52f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d558d402202db9e8 server_cookie=e89d87b4d9fe586a in_seq=0 out_seq=0 2024-09-10T08:19:20.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.196+0000 7f9e70c00640 10 client.18588 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:19:20.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e70c00640 1 -- 172.21.15.2:0/3235381043 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc4520b950 con 0x55cc452b2ed0 2024-09-10T08:19:20.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e70c00640 10 client.18588 handle_command_reply: tid=0 2024-09-10T08:19:20.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e90654740 1 client.18588 shutdown 2024-09-10T08:19:20.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e90654740 1 -- 172.21.15.2:0/3235381043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 msgr2=0x55cc452a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:20.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e90654740 1 --2- 172.21.15.2:0/3235381043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452a0260 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9e74012730 tx=0x7f9e74012bf0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e90654740 1 -- 172.21.15.2:0/3235381043 shutdown_connections 2024-09-10T08:19:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e90654740 1 --2- 172.21.15.2:0/3235381043 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55cc452b2ed0 0x55cc452b52f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e90654740 1 --2- 172.21.15.2:0/3235381043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc452a2e50 0x55cc452a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e90654740 1 --2- 172.21.15.2:0/3235381043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452a0260 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e90654740 1 --2- 172.21.15.2:0/3235381043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc45297c70 0x55cc4529fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e90654740 1 -- 172.21.15.2:0/3235381043 >> 172.21.15.2:0/3235381043 conn(0x55cc45294200 msgr2=0x55cc45293380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e70c00640 0 client.18588 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:19:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e90654740 1 -- 172.21.15.2:0/3235381043 shutdown_connections 2024-09-10T08:19:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.197+0000 7f9e90654740 1 -- 172.21.15.2:0/3235381043 wait complete. 2024-09-10T08:19:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.198+0000 7f9e90654740 20 client.18588 trim_cache size 0 max 16384 2024-09-10T08:19:20.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.211+0000 7f9e90654740 1 Processor -- start 2024-09-10T08:19:20.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.211+0000 7f9e90654740 1 -- start start 2024-09-10T08:19:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e90654740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc452a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e90654740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452baba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e90654740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc452a2e50 0x55cc452bb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e90654740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881b3820 con 0x55cc45298590 2024-09-10T08:19:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e90654740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881b10f0 con 0x55cc45297c70 2024-09-10T08:19:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e90654740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881aef40 con 0x55cc452a2e50 2024-09-10T08:19:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e86000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452baba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e86000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452baba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48704/0 (socket says 172.21.15.2:48704) 2024-09-10T08:19:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e86000640 1 -- 172.21.15.2:0/2461545665 learned_addr learned my addr 172.21.15.2:0/2461545665 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e87400640 1 --2- 172.21.15.2:0/2461545665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc452a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e87400640 1 --2- 172.21.15.2:0/2461545665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc452a2e50 0x55cc452bb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e87400640 1 -- 172.21.15.2:0/2461545665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc452a2e50 msgr2=0x55cc452bb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e87400640 1 --2- 172.21.15.2:0/2461545665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc452a2e50 0x55cc452bb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e87400640 1 -- 172.21.15.2:0/2461545665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 msgr2=0x55cc452baba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e87400640 1 --2- 172.21.15.2:0/2461545665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452baba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e87400640 1 -- 172.21.15.2:0/2461545665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc4528f770 con 0x55cc45297c70 2024-09-10T08:19:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.212+0000 7f9e86000640 1 --2- 172.21.15.2:0/2461545665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.213+0000 7f9e87400640 1 --2- 172.21.15.2:0/2461545665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc452a05f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9e80005e30 tx=0x7f9e80047040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2af0df59ac15d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.213+0000 7f9e73400640 1 -- 172.21.15.2:0/2461545665 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80031bb0 con 0x55cc45297c70 2024-09-10T08:19:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.213+0000 7f9e73400640 1 -- 172.21.15.2:0/2461545665 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e80036040 con 0x55cc45297c70 2024-09-10T08:19:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.213+0000 7f9e73400640 1 -- 172.21.15.2:0/2461545665 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80004660 con 0x55cc45297c70 2024-09-10T08:19:20.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.213+0000 7f9e90654740 1 -- 172.21.15.2:0/2461545665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 msgr2=0x55cc452a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:20.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.213+0000 7f9e90654740 1 --2- 172.21.15.2:0/2461545665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc452a05f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9e80005e30 tx=0x7f9e80047040 comp rx=0 tx=0).stop 2024-09-10T08:19:20.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.214+0000 7f9e90654740 1 -- 172.21.15.2:0/2461545665 shutdown_connections 2024-09-10T08:19:20.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.214+0000 7f9e90654740 1 --2- 172.21.15.2:0/2461545665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc452a2e50 0x55cc452bb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.214+0000 7f9e90654740 1 --2- 172.21.15.2:0/2461545665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc45298590 0x55cc452baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.214+0000 7f9e90654740 1 --2- 172.21.15.2:0/2461545665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc452a05f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.214+0000 7f9e90654740 1 -- 172.21.15.2:0/2461545665 >> 172.21.15.2:0/2461545665 conn(0x55cc45294200 msgr2=0x55cc4529ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:20.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.214+0000 7f9e90654740 1 -- 172.21.15.2:0/2461545665 shutdown_connections 2024-09-10T08:19:20.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.214+0000 7f9e90654740 1 -- 172.21.15.2:0/2461545665 wait complete. 2024-09-10T08:19:20.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.214+0000 7f9e90654740 1 -- start start 2024-09-10T08:19:20.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.214+0000 7f9e90654740 1 Processor -- start 2024-09-10T08:19:20.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.215+0000 7f9e90654740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc452aad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.215+0000 7f9e90654740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc45298590 0x55cc452ab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.215+0000 7f9e90654740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc452a2e50 0x55cc452a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.215+0000 7f9e90654740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881af5b0 con 0x55cc452a2e50 2024-09-10T08:19:20.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.215+0000 7f9e90654740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881b0a20 con 0x55cc45297c70 2024-09-10T08:19:20.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.215+0000 7f9e90654740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881afc80 con 0x55cc45298590 2024-09-10T08:19:20.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.215+0000 7f9e90654740 10 client.0 _ll_register_callbacks cb 0x7f9e74009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:20.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.215+0000 7f9e87400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc452a2e50 0x55cc452a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.215+0000 7f9e87400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc452a2e50 0x55cc452a61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48714/0 (socket says 172.21.15.2:48714) 2024-09-10T08:19:20.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.215+0000 7f9e87400640 1 -- 172.21.15.2:0/2170837691 learned_addr learned my addr 172.21.15.2:0/2170837691 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.215+0000 7f9e87400640 1 --2- 172.21.15.2:0/2170837691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc452aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.215+0000 7f9e86000640 1 --2- 172.21.15.2:0/2170837691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc45298590 0x55cc452ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.216+0000 7f9e87400640 1 -- 172.21.15.2:0/2170837691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc45298590 msgr2=0x55cc452ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.216+0000 7f9e87400640 1 --2- 172.21.15.2:0/2170837691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc45298590 0x55cc452ab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.216+0000 7f9e87400640 1 -- 172.21.15.2:0/2170837691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 msgr2=0x55cc452aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.216+0000 7f9e87400640 1 --2- 172.21.15.2:0/2170837691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc452aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.216+0000 7f9e87400640 1 -- 172.21.15.2:0/2170837691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc4529aae0 con 0x55cc452a2e50 2024-09-10T08:19:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.216+0000 7f9e87400640 1 --2- 172.21.15.2:0/2170837691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc452a2e50 0x55cc452a61b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9e80006010 tx=0x7f9e80004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab8a89377a5fb43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.216+0000 7f9e70c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.216+0000 7f9e70c00640 1 -- 172.21.15.2:0/2170837691 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80003c30 con 0x55cc452a2e50 2024-09-10T08:19:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.216+0000 7f9e70c00640 1 -- 172.21.15.2:0/2170837691 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9e80003dd0 con 0x55cc452a2e50 2024-09-10T08:19:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.216+0000 7f9e90654740 1 -- 172.21.15.2:0/2170837691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e80036040 con 0x55cc452a2e50 2024-09-10T08:19:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.216+0000 7f9e70c00640 1 -- 172.21.15.2:0/2170837691 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9e80004100 con 0x55cc452a2e50 2024-09-10T08:19:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.216+0000 7f9e90654740 10 client.18594 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.216+0000 7f9e90654740 1 -- 172.21.15.2:0/2170837691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cc4531d490 con 0x55cc452a2e50 2024-09-10T08:19:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.217+0000 7f9e70c00640 1 -- 172.21.15.2:0/2170837691 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e80036040 con 0x55cc452a2e50 2024-09-10T08:19:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.217+0000 7f9e70c00640 1 -- 172.21.15.2:0/2170837691 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9e80043020 con 0x55cc452a2e50 2024-09-10T08:19:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.217+0000 7f9e90654740 10 client.18594 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.217+0000 7f9e90654740 10 client.18594 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:19:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.217+0000 7f9e90654740 20 client.18594 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.217+0000 7f9e90654740 1 --2- 172.21.15.2:0/2170837691 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55cc4531d810 0x55cc4531fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.217+0000 7f9e90654740 4 client.18594 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:19:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.217+0000 7f9e90654740 1 -- 172.21.15.2:0/2170837691 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc45205e60 con 0x55cc4531d810 2024-09-10T08:19:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.217+0000 7f9e86000640 1 --2- 172.21.15.2:0/2170837691 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55cc4531d810 0x55cc4531fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.218+0000 7f9e86000640 1 --2- 172.21.15.2:0/2170837691 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55cc4531d810 0x55cc4531fc00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d89a6fcba3e7e8e server_cookie=ba72f5c73902ee9c in_seq=0 out_seq=0 2024-09-10T08:19:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.218+0000 7f9e70c00640 10 client.18594 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:19:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.218+0000 7f9e70c00640 1 -- 172.21.15.2:0/2170837691 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc45205e60 con 0x55cc4531d810 2024-09-10T08:19:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.218+0000 7f9e70c00640 10 client.18594 handle_command_reply: tid=0 2024-09-10T08:19:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.218+0000 7f9e90654740 1 client.18594 shutdown 2024-09-10T08:19:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.218+0000 7f9e90654740 1 -- 172.21.15.2:0/2170837691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc452a2e50 msgr2=0x55cc452a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.218+0000 7f9e90654740 1 --2- 172.21.15.2:0/2170837691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc452a2e50 0x55cc452a61b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9e80006010 tx=0x7f9e80004a80 comp rx=0 tx=0).stop 2024-09-10T08:19:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.218+0000 7f9e90654740 1 -- 172.21.15.2:0/2170837691 shutdown_connections 2024-09-10T08:19:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.218+0000 7f9e90654740 1 --2- 172.21.15.2:0/2170837691 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55cc4531d810 0x55cc4531fc00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.218+0000 7f9e90654740 1 --2- 172.21.15.2:0/2170837691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cc452a2e50 0x55cc452a61b0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.218+0000 7f9e90654740 1 --2- 172.21.15.2:0/2170837691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc45298590 0x55cc452ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.218+0000 7f9e90654740 1 --2- 172.21.15.2:0/2170837691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cc45297c70 0x55cc452aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.219+0000 7f9e90654740 1 -- 172.21.15.2:0/2170837691 >> 172.21.15.2:0/2170837691 conn(0x55cc45294200 msgr2=0x55cc452a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.219+0000 7f9e70c00640 0 client.18594 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:19:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.219+0000 7f9e90654740 1 -- 172.21.15.2:0/2170837691 shutdown_connections 2024-09-10T08:19:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.219+0000 7f9e90654740 1 -- 172.21.15.2:0/2170837691 wait complete. 2024-09-10T08:19:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.219+0000 7f9e90654740 20 client.18594 trim_cache size 0 max 16384 2024-09-10T08:19:20.223 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:19:20.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.219+0000 7f9e8da00640 1 -- 172.21.15.2:0/1120284338 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9e6c0a2b80 msgr2=0x7f9e6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:20.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.219+0000 7f9e8da00640 1 --2- 172.21.15.2:0/1120284338 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9e6c0a2b80 0x7f9e6c0a5040 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9e78098140 tx=0x7f9e78009700 comp rx=0 tx=0).stop 2024-09-10T08:19:20.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.219+0000 7f9e8da00640 1 -- 172.21.15.2:0/1120284338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e88103ff0 msgr2=0x7f9e8819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:20.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.219+0000 7f9e8da00640 1 --2- 172.21.15.2:0/1120284338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e88103ff0 0x7f9e8819a590 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800e9f0 tx=0x7f9e7800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.220+0000 7f9e8da00640 1 -- 172.21.15.2:0/1120284338 shutdown_connections 2024-09-10T08:19:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.220+0000 7f9e8da00640 1 --2- 172.21.15.2:0/1120284338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e88109120 0x7f9e8819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.220+0000 7f9e8da00640 1 --2- 172.21.15.2:0/1120284338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e881049a0 0x7f9e8819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.220+0000 7f9e8da00640 1 --2- 172.21.15.2:0/1120284338 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9e6c0a2b80 0x7f9e6c0a5040 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.220+0000 7f9e8da00640 1 --2- 172.21.15.2:0/1120284338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e88103ff0 0x7f9e8819a590 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.220+0000 7f9e8da00640 1 -- 172.21.15.2:0/1120284338 >> 172.21.15.2:0/1120284338 conn(0x7f9e880ffa90 msgr2=0x7f9e88101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.220+0000 7f9e8da00640 1 -- 172.21.15.2:0/1120284338 shutdown_connections 2024-09-10T08:19:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:20.220+0000 7f9e8da00640 1 -- 172.21.15.2:0/1120284338 wait complete. 2024-09-10T08:19:21.054 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:19:21.435 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:21.754 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:21 smithi002 ceph-mon[30563]: pgmap v723: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-10T08:19:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:21 smithi155 ceph-mon[35405]: pgmap v723: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-10T08:19:21.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:21 smithi107 ceph-mon[35048]: pgmap v723: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-10T08:19:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.034+0000 7fe332600640 1 -- 172.21.15.2:0/3770318160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe32c076720 msgr2=0x7fe32c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.034+0000 7fe332600640 1 --2- 172.21.15.2:0/3770318160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe32c076720 0x7fe32c076b00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009900 tx=0x7fe31c02a410 comp rx=0 tx=0).stop 2024-09-10T08:19:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.034+0000 7fe332600640 1 -- 172.21.15.2:0/3770318160 shutdown_connections 2024-09-10T08:19:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.034+0000 7fe332600640 1 --2- 172.21.15.2:0/3770318160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe32c112a50 0x7fe32c114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.034+0000 7fe332600640 1 --2- 172.21.15.2:0/3770318160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c077040 0x7fe32c112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.034+0000 7fe332600640 1 --2- 172.21.15.2:0/3770318160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe32c076720 0x7fe32c076b00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.034+0000 7fe332600640 1 -- 172.21.15.2:0/3770318160 >> 172.21.15.2:0/3770318160 conn(0x7fe32c0ffa50 msgr2=0x7fe32c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.034+0000 7fe332600640 1 -- 172.21.15.2:0/3770318160 shutdown_connections 2024-09-10T08:19:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.034+0000 7fe332600640 1 -- 172.21.15.2:0/3770318160 wait complete. 2024-09-10T08:19:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.036+0000 7fe332600640 1 Processor -- start 2024-09-10T08:19:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.036+0000 7fe332600640 1 -- start start 2024-09-10T08:19:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.036+0000 7fe332600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c076720 0x7fe32c1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.036+0000 7fe332600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe32c077040 0x7fe32c1a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.036+0000 7fe332600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe32c112a50 0x7fe32c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.036+0000 7fe332600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1185c0 con 0x7fe32c112a50 2024-09-10T08:19:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.036+0000 7fe332600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c118440 con 0x7fe32c076720 2024-09-10T08:19:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.036+0000 7fe332600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c118740 con 0x7fe32c077040 2024-09-10T08:19:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.037+0000 7fe32aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe32c077040 0x7fe32c1a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.037+0000 7fe32aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe32c077040 0x7fe32c1a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50814/0 (socket says 172.21.15.2:50814) 2024-09-10T08:19:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.037+0000 7fe32aa00640 1 -- 172.21.15.2:0/1568747555 learned_addr learned my addr 172.21.15.2:0/1568747555 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.037+0000 7fe32b400640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c076720 0x7fe32c1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.037+0000 7fe32be00640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe32c112a50 0x7fe32c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.037+0000 7fe32aa00640 1 -- 172.21.15.2:0/1568747555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c076720 msgr2=0x7fe32c1a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.037+0000 7fe32aa00640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c076720 0x7fe32c1a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.037+0000 7fe32aa00640 1 -- 172.21.15.2:0/1568747555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe32c112a50 msgr2=0x7fe32c19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.037+0000 7fe32aa00640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe32c112a50 0x7fe32c19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.037+0000 7fe32aa00640 1 -- 172.21.15.2:0/1568747555 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32c19f020 con 0x7fe32c077040 2024-09-10T08:19:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.037+0000 7fe32b400640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c076720 0x7fe32c1a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:19:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.038+0000 7fe32aa00640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe32c077040 0x7fe32c1a4c70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe31800e9f0 tx=0x7fe31800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.038+0000 7fe323e00640 1 -- 172.21.15.2:0/1568747555 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31800cd80 con 0x7fe32c077040 2024-09-10T08:19:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.038+0000 7fe323e00640 1 -- 172.21.15.2:0/1568747555 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe318010910 con 0x7fe32c077040 2024-09-10T08:19:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.038+0000 7fe332600640 1 -- 172.21.15.2:0/1568747555 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32c19f310 con 0x7fe32c077040 2024-09-10T08:19:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.038+0000 7fe323e00640 1 -- 172.21.15.2:0/1568747555 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3180188f0 con 0x7fe32c077040 2024-09-10T08:19:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.038+0000 7fe332600640 1 -- 172.21.15.2:0/1568747555 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe32c1ab600 con 0x7fe32c077040 2024-09-10T08:19:22.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.041+0000 7fe334fd2740 1 Processor -- start 2024-09-10T08:19:22.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.041+0000 7fe334fd2740 1 -- start start 2024-09-10T08:19:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe334fd2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efbc70 0x558962efc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe334fd2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efc590 0x558962f06910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe334fd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f09240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe334fd2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558962e623e0 con 0x558962efc590 2024-09-10T08:19:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe334fd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558962e09d80 con 0x558962f06e50 2024-09-10T08:19:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe334fd2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558962e5e3a0 con 0x558962efbc70 2024-09-10T08:19:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe323e00640 1 -- 172.21.15.2:0/1568747555 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe318010ac0 con 0x7fe32c077040 2024-09-10T08:19:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe32be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efbc70 0x558962efc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe32be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efbc70 0x558962efc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50826/0 (socket says 172.21.15.2:50826) 2024-09-10T08:19:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe32be00640 1 -- 172.21.15.2:0/2668731392 learned_addr learned my addr 172.21.15.2:0/2668731392 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe32b400640 1 --2- 172.21.15.2:0/2668731392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efc590 0x558962f06910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe32aa00640 1 --2- 172.21.15.2:0/2668731392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f09240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe32be00640 1 -- 172.21.15.2:0/2668731392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 msgr2=0x558962f09240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe32be00640 1 --2- 172.21.15.2:0/2668731392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f09240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe32be00640 1 -- 172.21.15.2:0/2668731392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efc590 msgr2=0x558962f06910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe32be00640 1 --2- 172.21.15.2:0/2668731392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efc590 0x558962f06910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.042+0000 7fe32be00640 1 -- 172.21.15.2:0/2668731392 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558962e28ae0 con 0x558962efbc70 2024-09-10T08:19:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.043+0000 7fe32aa00640 1 --2- 172.21.15.2:0/2668731392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f09240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:19:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.043+0000 7fe32be00640 1 --2- 172.21.15.2:0/2668731392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efbc70 0x558962efc050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe324009f30 tx=0x7fe3240118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c965e0ff59a2212 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.044+0000 7fe320c00640 1 -- 172.21.15.2:0/2668731392 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324004d20 con 0x558962efbc70 2024-09-10T08:19:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.044+0000 7fe320c00640 1 -- 172.21.15.2:0/2668731392 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe324004050 con 0x558962efbc70 2024-09-10T08:19:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.044+0000 7fe320c00640 1 -- 172.21.15.2:0/2668731392 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324005280 con 0x558962efbc70 2024-09-10T08:19:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.044+0000 7fe323e00640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3100a2b80 0x7fe3100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.044+0000 7fe323e00640 1 -- 172.21.15.2:0/1568747555 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe3180cf8d0 con 0x7fe32c077040 2024-09-10T08:19:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.044+0000 7fe32b400640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3100a2b80 0x7fe3100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.045+0000 7fe334fd2740 1 -- 172.21.15.2:0/2668731392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efbc70 msgr2=0x558962efc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.045+0000 7fe334fd2740 1 --2- 172.21.15.2:0/2668731392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efbc70 0x558962efc050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe324009f30 tx=0x7fe3240118d0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.045+0000 7fe334fd2740 1 -- 172.21.15.2:0/2668731392 shutdown_connections 2024-09-10T08:19:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.045+0000 7fe334fd2740 1 --2- 172.21.15.2:0/2668731392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f09240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.045+0000 7fe334fd2740 1 --2- 172.21.15.2:0/2668731392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efc590 0x558962f06910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.045+0000 7fe334fd2740 1 --2- 172.21.15.2:0/2668731392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efbc70 0x558962efc050 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.045+0000 7fe334fd2740 1 -- 172.21.15.2:0/2668731392 >> 172.21.15.2:0/2668731392 conn(0x558962ef8200 msgr2=0x558962ef85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.045+0000 7fe334fd2740 1 -- 172.21.15.2:0/2668731392 shutdown_connections 2024-09-10T08:19:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.045+0000 7fe334fd2740 1 -- 172.21.15.2:0/2668731392 wait complete. 2024-09-10T08:19:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.045+0000 7fe32b400640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3100a2b80 0x7fe3100a5040 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe31c000c00 tx=0x7fe31c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.046+0000 7fe334fd2740 1 -- start start 2024-09-10T08:19:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.046+0000 7fe334fd2740 1 Processor -- start 2024-09-10T08:19:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.046+0000 7fe334fd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962efbc70 0x558962f03cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.046+0000 7fe334fd2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efc590 0x558962f041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.046+0000 7fe334fd2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962f06e50 0x558962f04730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.046+0000 7fe334fd2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558962f09a20 con 0x558962efc590 2024-09-10T08:19:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.046+0000 7fe334fd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558962e1d0f0 con 0x558962efbc70 2024-09-10T08:19:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.046+0000 7fe334fd2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558962f09ba0 con 0x558962f06e50 2024-09-10T08:19:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.047+0000 7fe334fd2740 10 client.0 _ll_register_callbacks cb 0x558962e04bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.047+0000 7fe32b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962efbc70 0x558962f03cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.047+0000 7fe32aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efc590 0x558962f041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.047+0000 7fe32b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962efbc70 0x558962f03cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:48952/0 (socket says 172.21.15.2:48952) 2024-09-10T08:19:22.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.047+0000 7fe32b400640 1 -- 172.21.15.2:0/1654695395 learned_addr learned my addr 172.21.15.2:0/1654695395 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:22.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.047+0000 7fe32be00640 1 --2- 172.21.15.2:0/1654695395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962f06e50 0x558962f04730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.047+0000 7fe32b400640 1 -- 172.21.15.2:0/1654695395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962f06e50 msgr2=0x558962f04730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:22.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.047+0000 7fe32b400640 1 --2- 172.21.15.2:0/1654695395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962f06e50 0x558962f04730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.047+0000 7fe32b400640 1 -- 172.21.15.2:0/1654695395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efc590 msgr2=0x558962f041f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.047+0000 7fe32b400640 1 --2- 172.21.15.2:0/1654695395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efc590 0x558962f041f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.047+0000 7fe32b400640 1 -- 172.21.15.2:0/1654695395 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558962bd33c0 con 0x558962efbc70 2024-09-10T08:19:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.047+0000 7fe32aa00640 1 --2- 172.21.15.2:0/1654695395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efc590 0x558962f041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.048+0000 7fe32b400640 1 --2- 172.21.15.2:0/1654695395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962efbc70 0x558962f03cb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe31c033000 tx=0x7fe31c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7d6b34de1d6cd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.048+0000 7fe315600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:19:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.048+0000 7fe315600640 1 -- 172.21.15.2:0/1654695395 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c04c030 con 0x558962efbc70 2024-09-10T08:19:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.048+0000 7fe334fd2740 1 -- 172.21.15.2:0/1654695395 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe324004050 con 0x558962efbc70 2024-09-10T08:19:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.048+0000 7fe315600640 1 -- 172.21.15.2:0/1654695395 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe31c003800 con 0x558962efbc70 2024-09-10T08:19:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.048+0000 7fe315600640 1 -- 172.21.15.2:0/1654695395 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe31c059050 con 0x558962efbc70 2024-09-10T08:19:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.048+0000 7fe334fd2740 10 client.28634 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.048+0000 7fe334fd2740 1 -- 172.21.15.2:0/1654695395 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558962f159a0 con 0x558962efbc70 2024-09-10T08:19:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.049+0000 7fe315600640 1 -- 172.21.15.2:0/1654695395 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe324004050 con 0x558962efbc70 2024-09-10T08:19:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.049+0000 7fe315600640 1 -- 172.21.15.2:0/1654695395 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe31c048860 con 0x558962efbc70 2024-09-10T08:19:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.049+0000 7fe334fd2740 10 client.28634 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.049+0000 7fe334fd2740 10 client.28634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.049+0000 7fe334fd2740 20 client.28634 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.049+0000 7fe334fd2740 1 --2- 172.21.15.2:0/1654695395 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558962f16da0 0x558962f19260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.049+0000 7fe334fd2740 4 client.28634 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.049+0000 7fe334fd2740 1 -- 172.21.15.2:0/1654695395 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558962e6f950 con 0x558962f16da0 2024-09-10T08:19:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.049+0000 7fe32aa00640 1 --2- 172.21.15.2:0/1654695395 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558962f16da0 0x558962f19260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.050+0000 7fe32aa00640 1 --2- 172.21.15.2:0/1654695395 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558962f16da0 0x558962f19260 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=674e5a5c74e52673 server_cookie=24dcee9235f25550 in_seq=0 out_seq=0 2024-09-10T08:19:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.050+0000 7fe315600640 10 client.28634 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.050+0000 7fe315600640 1 -- 172.21.15.2:0/1654695395 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558962e6f950 con 0x558962f16da0 2024-09-10T08:19:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.050+0000 7fe315600640 10 client.28634 handle_command_reply: tid=0 2024-09-10T08:19:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.051+0000 7fe334fd2740 1 client.28634 shutdown 2024-09-10T08:19:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.051+0000 7fe334fd2740 1 -- 172.21.15.2:0/1654695395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962efbc70 msgr2=0x558962f03cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.051+0000 7fe334fd2740 1 --2- 172.21.15.2:0/1654695395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962efbc70 0x558962f03cb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe31c033000 tx=0x7fe31c009260 comp rx=0 tx=0).stop 2024-09-10T08:19:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.051+0000 7fe334fd2740 1 -- 172.21.15.2:0/1654695395 shutdown_connections 2024-09-10T08:19:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.051+0000 7fe334fd2740 1 --2- 172.21.15.2:0/1654695395 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558962f16da0 0x558962f19260 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.051+0000 7fe334fd2740 1 --2- 172.21.15.2:0/1654695395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962f06e50 0x558962f04730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.051+0000 7fe334fd2740 1 --2- 172.21.15.2:0/1654695395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efc590 0x558962f041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.051+0000 7fe334fd2740 1 --2- 172.21.15.2:0/1654695395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962efbc70 0x558962f03cb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.051+0000 7fe315600640 0 client.28634 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.051+0000 7fe334fd2740 1 -- 172.21.15.2:0/1654695395 >> 172.21.15.2:0/1654695395 conn(0x558962ef8200 msgr2=0x558962efdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.051+0000 7fe334fd2740 1 -- 172.21.15.2:0/1654695395 shutdown_connections 2024-09-10T08:19:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.051+0000 7fe334fd2740 1 -- 172.21.15.2:0/1654695395 wait complete. 2024-09-10T08:19:22.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.051+0000 7fe334fd2740 20 client.28634 trim_cache size 0 max 16384 2024-09-10T08:19:22.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.065+0000 7fe334fd2740 1 Processor -- start 2024-09-10T08:19:22.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.065+0000 7fe334fd2740 1 -- start start 2024-09-10T08:19:22.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.065+0000 7fe334fd2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 0x558962efe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.065+0000 7fe334fd2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efc590 0x558962f1eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.065+0000 7fe334fd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f1f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.065+0000 7fe334fd2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b7e10 con 0x558962efbc70 2024-09-10T08:19:22.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.065+0000 7fe334fd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b56e0 con 0x558962f06e50 2024-09-10T08:19:22.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.065+0000 7fe334fd2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b3530 con 0x558962efc590 2024-09-10T08:19:22.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.066+0000 7fe32b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 0x558962efe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.066+0000 7fe32b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 0x558962efe6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48754/0 (socket says 172.21.15.2:48754) 2024-09-10T08:19:22.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.066+0000 7fe32b400640 1 -- 172.21.15.2:0/3880066607 learned_addr learned my addr 172.21.15.2:0/3880066607 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:22.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.066+0000 7fe32aa00640 1 --2- 172.21.15.2:0/3880066607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efc590 0x558962f1eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.066+0000 7fe32be00640 1 --2- 172.21.15.2:0/3880066607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f1f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.066+0000 7fe32b400640 1 -- 172.21.15.2:0/3880066607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efc590 msgr2=0x558962f1eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:22.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.066+0000 7fe32b400640 1 --2- 172.21.15.2:0/3880066607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efc590 0x558962f1eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.066+0000 7fe32b400640 1 -- 172.21.15.2:0/3880066607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 msgr2=0x558962f1f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:22.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.066+0000 7fe32b400640 1 --2- 172.21.15.2:0/3880066607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f1f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.066+0000 7fe32b400640 1 -- 172.21.15.2:0/3880066607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558962ef92b0 con 0x558962efbc70 2024-09-10T08:19:22.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.066+0000 7fe32aa00640 1 --2- 172.21.15.2:0/3880066607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efc590 0x558962f1eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:22.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.066+0000 7fe32be00640 1 --2- 172.21.15.2:0/3880066607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f1f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:22.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.067+0000 7fe32b400640 1 --2- 172.21.15.2:0/3880066607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 0x558962efe6d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009900 tx=0x7fe31c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb486e34b362ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:22.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.067+0000 7fe317e00640 1 -- 172.21.15.2:0/3880066607 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c062070 con 0x558962efbc70 2024-09-10T08:19:22.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.067+0000 7fe317e00640 1 -- 172.21.15.2:0/3880066607 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe31c004380 con 0x558962efbc70 2024-09-10T08:19:22.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.067+0000 7fe317e00640 1 -- 172.21.15.2:0/3880066607 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c04c040 con 0x558962efbc70 2024-09-10T08:19:22.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.068+0000 7fe334fd2740 1 -- 172.21.15.2:0/3880066607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 msgr2=0x558962efe6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:22.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.068+0000 7fe334fd2740 1 --2- 172.21.15.2:0/3880066607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 0x558962efe6d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009900 tx=0x7fe31c009780 comp rx=0 tx=0).stop 2024-09-10T08:19:22.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.068+0000 7fe334fd2740 1 -- 172.21.15.2:0/3880066607 shutdown_connections 2024-09-10T08:19:22.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.068+0000 7fe334fd2740 1 --2- 172.21.15.2:0/3880066607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f1f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.068+0000 7fe334fd2740 1 --2- 172.21.15.2:0/3880066607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efc590 0x558962f1eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.068+0000 7fe334fd2740 1 --2- 172.21.15.2:0/3880066607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 0x558962efe6d0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.068+0000 7fe334fd2740 1 -- 172.21.15.2:0/3880066607 >> 172.21.15.2:0/3880066607 conn(0x558962ef8200 msgr2=0x558962f02da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:22.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.068+0000 7fe334fd2740 1 -- 172.21.15.2:0/3880066607 shutdown_connections 2024-09-10T08:19:22.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.068+0000 7fe334fd2740 1 -- 172.21.15.2:0/3880066607 wait complete. 2024-09-10T08:19:22.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.068+0000 7fe334fd2740 1 -- start start 2024-09-10T08:19:22.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.068+0000 7fe334fd2740 1 Processor -- start 2024-09-10T08:19:22.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.069+0000 7fe334fd2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 0x558962f0f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.069+0000 7fe334fd2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efc590 0x558962f0f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.069+0000 7fe334fd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f00660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.069+0000 7fe334fd2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b3ba0 con 0x558962efbc70 2024-09-10T08:19:22.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.069+0000 7fe334fd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b5010 con 0x558962f06e50 2024-09-10T08:19:22.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.069+0000 7fe334fd2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b4270 con 0x558962efc590 2024-09-10T08:19:22.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.069+0000 7fe334fd2740 10 client.0 _ll_register_callbacks cb 0x7fe318005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:22.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.069+0000 7fe32b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 0x558962f0f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.069+0000 7fe32b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 0x558962f0f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48756/0 (socket says 172.21.15.2:48756) 2024-09-10T08:19:22.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.069+0000 7fe32b400640 1 -- 172.21.15.2:0/1402651503 learned_addr learned my addr 172.21.15.2:0/1402651503 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:22.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.069+0000 7fe32aa00640 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efc590 0x558962f0f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.069+0000 7fe32be00640 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f00660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.070+0000 7fe32b400640 1 -- 172.21.15.2:0/1402651503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efc590 msgr2=0x558962f0f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:22.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.070+0000 7fe32b400640 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efc590 0x558962f0f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.070+0000 7fe32b400640 1 -- 172.21.15.2:0/1402651503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 msgr2=0x558962f00660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:22.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.070+0000 7fe32b400640 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f00660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.070+0000 7fe32b400640 1 -- 172.21.15.2:0/1402651503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558962ef3770 con 0x558962efbc70 2024-09-10T08:19:22.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.070+0000 7fe32aa00640 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efc590 0x558962f0f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:22.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.070+0000 7fe32b400640 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 0x558962f0f060 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe31c060040 tx=0x7fe31c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3535de5df25d1b0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:22.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.070+0000 7fe315600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:22.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.070+0000 7fe315600640 1 -- 172.21.15.2:0/1402651503 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c06a060 con 0x558962efbc70 2024-09-10T08:19:22.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.070+0000 7fe315600640 1 -- 172.21.15.2:0/1402651503 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe31c0489a0 con 0x558962efbc70 2024-09-10T08:19:22.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.070+0000 7fe334fd2740 1 -- 172.21.15.2:0/1402651503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe31c004380 con 0x558962efbc70 2024-09-10T08:19:22.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.071+0000 7fe315600640 1 -- 172.21.15.2:0/1402651503 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe31c071050 con 0x558962efbc70 2024-09-10T08:19:22.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.071+0000 7fe334fd2740 10 client.18612 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:22.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.071+0000 7fe334fd2740 1 -- 172.21.15.2:0/1402651503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558962f814e0 con 0x558962efbc70 2024-09-10T08:19:22.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.071+0000 7fe315600640 1 -- 172.21.15.2:0/1402651503 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe31c004380 con 0x558962efbc70 2024-09-10T08:19:22.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.071+0000 7fe315600640 1 -- 172.21.15.2:0/1402651503 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe31c03c8c0 con 0x558962efbc70 2024-09-10T08:19:22.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.071+0000 7fe334fd2740 10 client.18612 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:22.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.071+0000 7fe334fd2740 10 client.18612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:22.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.071+0000 7fe334fd2740 20 client.18612 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:22.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.071+0000 7fe334fd2740 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558962f81b40 0x558962f83f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:22.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.071+0000 7fe334fd2740 4 client.18612 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:19:22.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.071+0000 7fe334fd2740 1 -- 172.21.15.2:0/1402651503 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558962e69e60 con 0x558962f81b40 2024-09-10T08:19:22.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.072+0000 7fe32aa00640 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558962f81b40 0x558962f83f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:22.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.072+0000 7fe32aa00640 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558962f81b40 0x558962f83f60 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=818f1aacae064bcd server_cookie=b68097288e883399 in_seq=0 out_seq=0 2024-09-10T08:19:22.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.072+0000 7fe315600640 10 client.18612 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:22.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.072+0000 7fe315600640 1 -- 172.21.15.2:0/1402651503 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558962e69e60 con 0x558962f81b40 2024-09-10T08:19:22.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.072+0000 7fe315600640 10 client.18612 handle_command_reply: tid=0 2024-09-10T08:19:22.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.072+0000 7fe334fd2740 1 client.18612 shutdown 2024-09-10T08:19:22.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.072+0000 7fe334fd2740 1 -- 172.21.15.2:0/1402651503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 msgr2=0x558962f0f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:22.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.072+0000 7fe334fd2740 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 0x558962f0f060 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe31c060040 tx=0x7fe31c005c90 comp rx=0 tx=0).stop 2024-09-10T08:19:22.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.073+0000 7fe334fd2740 1 -- 172.21.15.2:0/1402651503 shutdown_connections 2024-09-10T08:19:22.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.073+0000 7fe334fd2740 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558962f81b40 0x558962f83f60 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.073+0000 7fe334fd2740 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558962f06e50 0x558962f00660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.073+0000 7fe334fd2740 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558962efc590 0x558962f0f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.073+0000 7fe334fd2740 1 --2- 172.21.15.2:0/1402651503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558962efbc70 0x558962f0f060 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.073+0000 7fe334fd2740 1 -- 172.21.15.2:0/1402651503 >> 172.21.15.2:0/1402651503 conn(0x558962ef8200 msgr2=0x558962ef6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:22.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.073+0000 7fe315600640 0 client.18612 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:22.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.073+0000 7fe334fd2740 1 -- 172.21.15.2:0/1402651503 shutdown_connections 2024-09-10T08:19:22.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.073+0000 7fe334fd2740 1 -- 172.21.15.2:0/1402651503 wait complete. 2024-09-10T08:19:22.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.073+0000 7fe334fd2740 20 client.18612 trim_cache size 0 max 16384 2024-09-10T08:19:22.078 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:19:22.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.074+0000 7fe332600640 1 -- 172.21.15.2:0/1568747555 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3100a2b80 msgr2=0x7fe3100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:22.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.074+0000 7fe332600640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3100a2b80 0x7fe3100a5040 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe31c000c00 tx=0x7fe31c005b50 comp rx=0 tx=0).stop 2024-09-10T08:19:22.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.074+0000 7fe332600640 1 -- 172.21.15.2:0/1568747555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe32c077040 msgr2=0x7fe32c1a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:22.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.074+0000 7fe332600640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe32c077040 0x7fe32c1a4c70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe31800e9f0 tx=0x7fe31800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.074+0000 7fe332600640 1 -- 172.21.15.2:0/1568747555 shutdown_connections 2024-09-10T08:19:22.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.074+0000 7fe332600640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe32c112a50 0x7fe32c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.074+0000 7fe332600640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3100a2b80 0x7fe3100a5040 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.074+0000 7fe332600640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe32c077040 0x7fe32c1a4c70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.074+0000 7fe332600640 1 --2- 172.21.15.2:0/1568747555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c076720 0x7fe32c1a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:22.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.074+0000 7fe332600640 1 -- 172.21.15.2:0/1568747555 >> 172.21.15.2:0/1568747555 conn(0x7fe32c0ffa50 msgr2=0x7fe32c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:22.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.074+0000 7fe332600640 1 -- 172.21.15.2:0/1568747555 shutdown_connections 2024-09-10T08:19:22.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:22.074+0000 7fe332600640 1 -- 172.21.15.2:0/1568747555 wait complete. 2024-09-10T08:19:23.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:23 smithi002 ceph-mon[30563]: pgmap v724: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-09-10T08:19:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:23 smithi155 ceph-mon[35405]: pgmap v724: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-09-10T08:19:23.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:23 smithi107 ceph-mon[35048]: pgmap v724: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-09-10T08:19:24.008 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:19:24.009 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:19:24.391 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:25.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.033+0000 7ff4b8200640 1 -- 172.21.15.2:0/2232519653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4b01049a0 msgr2=0x7ff4b01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:25.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.033+0000 7ff4b8200640 1 --2- 172.21.15.2:0/2232519653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4b01049a0 0x7ff4b01089f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009900 tx=0x7ff49c02a410 comp rx=0 tx=0).stop 2024-09-10T08:19:25.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.033+0000 7ff4b8200640 1 -- 172.21.15.2:0/2232519653 shutdown_connections 2024-09-10T08:19:25.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.033+0000 7ff4b8200640 1 --2- 172.21.15.2:0/2232519653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4b0109120 0x7ff4b0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.033+0000 7ff4b8200640 1 --2- 172.21.15.2:0/2232519653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4b01049a0 0x7ff4b01089f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.033+0000 7ff4b8200640 1 --2- 172.21.15.2:0/2232519653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b0103ff0 0x7ff4b01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.033+0000 7ff4b8200640 1 -- 172.21.15.2:0/2232519653 >> 172.21.15.2:0/2232519653 conn(0x7ff4b00ffa90 msgr2=0x7ff4b0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:25.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.033+0000 7ff4b8200640 1 -- 172.21.15.2:0/2232519653 shutdown_connections 2024-09-10T08:19:25.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.033+0000 7ff4b8200640 1 -- 172.21.15.2:0/2232519653 wait complete. 2024-09-10T08:19:25.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.034+0000 7ff4b8200640 1 Processor -- start 2024-09-10T08:19:25.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.034+0000 7ff4b8200640 1 -- start start 2024-09-10T08:19:25.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.034+0000 7ff4b8200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b0103ff0 0x7ff4b019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.034+0000 7ff4b8200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4b01049a0 0x7ff4b019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.034+0000 7ff4b8200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4b0109120 0x7ff4b019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.034+0000 7ff4b8200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b0114120 con 0x7ff4b0109120 2024-09-10T08:19:25.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.034+0000 7ff4b8200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b0113fa0 con 0x7ff4b01049a0 2024-09-10T08:19:25.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.034+0000 7ff4b8200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01142a0 con 0x7ff4b0103ff0 2024-09-10T08:19:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.035+0000 7ff4b5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b0103ff0 0x7ff4b019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.035+0000 7ff4b6200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4b0109120 0x7ff4b019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.035+0000 7ff4b5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b0103ff0 0x7ff4b019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50870/0 (socket says 172.21.15.2:50870) 2024-09-10T08:19:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.035+0000 7ff4b5800640 1 -- 172.21.15.2:0/859316098 learned_addr learned my addr 172.21.15.2:0/859316098 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.035+0000 7ff4b4e00640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4b01049a0 0x7ff4b019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.035+0000 7ff4b5800640 1 -- 172.21.15.2:0/859316098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4b01049a0 msgr2=0x7ff4b019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.035+0000 7ff4b5800640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4b01049a0 0x7ff4b019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.035+0000 7ff4b5800640 1 -- 172.21.15.2:0/859316098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4b0109120 msgr2=0x7ff4b019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.035+0000 7ff4b5800640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4b0109120 0x7ff4b019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.035+0000 7ff4b5800640 1 -- 172.21.15.2:0/859316098 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b019f650 con 0x7ff4b0103ff0 2024-09-10T08:19:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.035+0000 7ff4b6200640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4b0109120 0x7ff4b019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.036+0000 7ff4b5800640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b0103ff0 0x7ff4b019a6a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400e9f0 tx=0x7ff4a400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:25.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.036+0000 7ff4ae000640 1 -- 172.21.15.2:0/859316098 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400cd80 con 0x7ff4b0103ff0 2024-09-10T08:19:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.036+0000 7ff4b8200640 1 -- 172.21.15.2:0/859316098 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b019f940 con 0x7ff4b0103ff0 2024-09-10T08:19:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.036+0000 7ff4ae000640 1 -- 172.21.15.2:0/859316098 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a4010910 con 0x7ff4b0103ff0 2024-09-10T08:19:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.036+0000 7ff4ae000640 1 -- 172.21.15.2:0/859316098 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40188f0 con 0x7ff4b0103ff0 2024-09-10T08:19:25.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.036+0000 7ff4b8200640 1 -- 172.21.15.2:0/859316098 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b01a7220 con 0x7ff4b0103ff0 2024-09-10T08:19:25.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.039+0000 7ff4bac42740 1 Processor -- start 2024-09-10T08:19:25.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.039+0000 7ff4bac42740 1 -- start start 2024-09-10T08:19:25.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.039+0000 7ff4bac42740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aedac0 0x558ce1aedea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.039+0000 7ff4bac42740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aee3e0 0x558ce1af8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.039+0000 7ff4bac42740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 0x558ce1afb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.039+0000 7ff4bac42740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558ce1a54290 con 0x558ce1aedac0 2024-09-10T08:19:25.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.039+0000 7ff4bac42740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558ce19fbb40 con 0x558ce1aee3e0 2024-09-10T08:19:25.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.039+0000 7ff4bac42740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558ce1a50250 con 0x558ce1af8ca0 2024-09-10T08:19:25.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.040+0000 7ff4ae000640 1 -- 172.21.15.2:0/859316098 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff4a4010ac0 con 0x7ff4b0103ff0 2024-09-10T08:19:25.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.040+0000 7ff4b6200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aedac0 0x558ce1aedea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.040+0000 7ff4b6200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aedac0 0x558ce1aedea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48772/0 (socket says 172.21.15.2:48772) 2024-09-10T08:19:25.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.040+0000 7ff4b6200640 1 -- 172.21.15.2:0/4054435379 learned_addr learned my addr 172.21.15.2:0/4054435379 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:25.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.040+0000 7ff4b4e00640 1 --2- 172.21.15.2:0/4054435379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 0x558ce1afb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.040+0000 7ff4b5800640 1 --2- 172.21.15.2:0/4054435379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aee3e0 0x558ce1af8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.040+0000 7ff4b6200640 1 -- 172.21.15.2:0/4054435379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 msgr2=0x558ce1afb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:25.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.040+0000 7ff4b6200640 1 --2- 172.21.15.2:0/4054435379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 0x558ce1afb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.040+0000 7ff4b6200640 1 -- 172.21.15.2:0/4054435379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aee3e0 msgr2=0x558ce1af8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:25.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.040+0000 7ff4b6200640 1 --2- 172.21.15.2:0/4054435379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aee3e0 0x558ce1af8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.040+0000 7ff4b6200640 1 -- 172.21.15.2:0/4054435379 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce17c56a0 con 0x558ce1aedac0 2024-09-10T08:19:25.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.040+0000 7ff4b5800640 1 --2- 172.21.15.2:0/4054435379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aee3e0 0x558ce1af8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:19:25.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.041+0000 7ff4b6200640 1 --2- 172.21.15.2:0/4054435379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aedac0 0x558ce1aedea0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8011790 tx=0x7ff4a8011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4922ccf9f551b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:25.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.041+0000 7ff4a2a00640 1 -- 172.21.15.2:0/4054435379 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8003c30 con 0x558ce1aedac0 2024-09-10T08:19:25.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.041+0000 7ff4a2a00640 1 -- 172.21.15.2:0/4054435379 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a800fe80 con 0x558ce1aedac0 2024-09-10T08:19:25.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.041+0000 7ff4a2a00640 1 -- 172.21.15.2:0/4054435379 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80193e0 con 0x558ce1aedac0 2024-09-10T08:19:25.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.041+0000 7ff4ae000640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4980a2b80 0x7ff4980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.042+0000 7ff4ae000640 1 -- 172.21.15.2:0/859316098 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff4a40cf7e0 con 0x7ff4b0103ff0 2024-09-10T08:19:25.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.042+0000 7ff4b5800640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4980a2b80 0x7ff4980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.042+0000 7ff4bac42740 1 -- 172.21.15.2:0/4054435379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aedac0 msgr2=0x558ce1aedea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:25.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.042+0000 7ff4bac42740 1 --2- 172.21.15.2:0/4054435379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aedac0 0x558ce1aedea0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8011790 tx=0x7ff4a8011c50 comp rx=0 tx=0).stop 2024-09-10T08:19:25.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.042+0000 7ff4bac42740 1 -- 172.21.15.2:0/4054435379 shutdown_connections 2024-09-10T08:19:25.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.042+0000 7ff4bac42740 1 --2- 172.21.15.2:0/4054435379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 0x558ce1afb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.042+0000 7ff4bac42740 1 --2- 172.21.15.2:0/4054435379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aee3e0 0x558ce1af8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.042+0000 7ff4bac42740 1 --2- 172.21.15.2:0/4054435379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aedac0 0x558ce1aedea0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.042+0000 7ff4bac42740 1 -- 172.21.15.2:0/4054435379 >> 172.21.15.2:0/4054435379 conn(0x558ce1aea050 msgr2=0x558ce1aea430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:25.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.042+0000 7ff4bac42740 1 -- 172.21.15.2:0/4054435379 shutdown_connections 2024-09-10T08:19:25.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.042+0000 7ff4bac42740 1 -- 172.21.15.2:0/4054435379 wait complete. 2024-09-10T08:19:25.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.042+0000 7ff4b5800640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4980a2b80 0x7ff4980a5040 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40cd260 tx=0x7ff4a401f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:25.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.043+0000 7ff4bac42740 1 -- start start 2024-09-10T08:19:25.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.043+0000 7ff4bac42740 1 Processor -- start 2024-09-10T08:19:25.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.043+0000 7ff4bac42740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aedac0 0x558ce1af8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.043+0000 7ff4bac42740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 0x558ce1af58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.043+0000 7ff4bac42740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 0x558ce1af5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.043+0000 7ff4bac42740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558ce1afb910 con 0x558ce1aee3e0 2024-09-10T08:19:25.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.043+0000 7ff4bac42740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558ce1a0efa0 con 0x558ce1aedac0 2024-09-10T08:19:25.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.043+0000 7ff4bac42740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558ce1afba90 con 0x558ce1af8ca0 2024-09-10T08:19:25.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.044+0000 7ff4bac42740 10 client.0 _ll_register_callbacks cb 0x558ce1762ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:25.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.044+0000 7ff4b4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 0x558ce1af58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.044+0000 7ff4b6200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aedac0 0x558ce1af8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.044+0000 7ff4b4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 0x558ce1af58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48780/0 (socket says 172.21.15.2:48780) 2024-09-10T08:19:25.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.044+0000 7ff4b4e00640 1 -- 172.21.15.2:0/4284514801 learned_addr learned my addr 172.21.15.2:0/4284514801 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:25.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.044+0000 7ff4b6200640 1 --2- 172.21.15.2:0/4284514801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 0x558ce1af5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.044+0000 7ff4b4e00640 1 -- 172.21.15.2:0/4284514801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 msgr2=0x558ce1af5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:25.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.044+0000 7ff4b4e00640 1 --2- 172.21.15.2:0/4284514801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 0x558ce1af5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.044+0000 7ff4b4e00640 1 -- 172.21.15.2:0/4284514801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aedac0 msgr2=0x558ce1af8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:25.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.044+0000 7ff4b4e00640 1 --2- 172.21.15.2:0/4284514801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aedac0 0x558ce1af8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.044+0000 7ff4b4e00640 1 -- 172.21.15.2:0/4284514801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce1af6690 con 0x558ce1aee3e0 2024-09-10T08:19:25.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.044+0000 7ff4b6200640 1 --2- 172.21.15.2:0/4284514801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 0x558ce1af5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:19:25.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.045+0000 7ff4b4e00640 1 --2- 172.21.15.2:0/4284514801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 0x558ce1af58e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff49c02a930 tx=0x7ff49c0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f18e65d1b168c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:25.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.045+0000 7ff497400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:25.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.045+0000 7ff497400640 1 -- 172.21.15.2:0/4284514801 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c035040 con 0x558ce1aee3e0 2024-09-10T08:19:25.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.045+0000 7ff4bac42740 1 -- 172.21.15.2:0/4284514801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558ce1a61800 con 0x558ce1aee3e0 2024-09-10T08:19:25.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.045+0000 7ff497400640 1 -- 172.21.15.2:0/4284514801 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff49c004240 con 0x558ce1aee3e0 2024-09-10T08:19:25.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.045+0000 7ff497400640 1 -- 172.21.15.2:0/4284514801 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff49c004440 con 0x558ce1aee3e0 2024-09-10T08:19:25.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.045+0000 7ff4bac42740 10 client.18624 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:25.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.045+0000 7ff4bac42740 1 -- 172.21.15.2:0/4284514801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558ce1b07860 con 0x558ce1aee3e0 2024-09-10T08:19:25.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.045+0000 7ff497400640 1 -- 172.21.15.2:0/4284514801 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558ce1a61800 con 0x558ce1aee3e0 2024-09-10T08:19:25.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.046+0000 7ff497400640 1 -- 172.21.15.2:0/4284514801 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff49c03d5c0 con 0x558ce1aee3e0 2024-09-10T08:19:25.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.046+0000 7ff4bac42740 10 client.18624 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:25.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.046+0000 7ff4bac42740 10 client.18624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:25.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.046+0000 7ff4bac42740 20 client.18624 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:25.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.046+0000 7ff4bac42740 1 --2- 172.21.15.2:0/4284514801 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558ce1b08d70 0x558ce1b0b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.046+0000 7ff4bac42740 4 client.18624 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:25.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.046+0000 7ff4bac42740 1 -- 172.21.15.2:0/4284514801 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ce1a5bd10 con 0x558ce1b08d70 2024-09-10T08:19:25.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.046+0000 7ff4b6200640 1 --2- 172.21.15.2:0/4284514801 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558ce1b08d70 0x558ce1b0b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.047+0000 7ff4b6200640 1 --2- 172.21.15.2:0/4284514801 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558ce1b08d70 0x558ce1b0b190 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e6342b213d50d1c server_cookie=e347de5df9941c71 in_seq=0 out_seq=0 2024-09-10T08:19:25.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.047+0000 7ff497400640 10 client.18624 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:25.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.047+0000 7ff497400640 1 -- 172.21.15.2:0/4284514801 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ce1a5bd10 con 0x558ce1b08d70 2024-09-10T08:19:25.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.047+0000 7ff497400640 10 client.18624 handle_command_reply: tid=0 2024-09-10T08:19:25.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.048+0000 7ff4bac42740 1 client.18624 shutdown 2024-09-10T08:19:25.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.048+0000 7ff4bac42740 1 -- 172.21.15.2:0/4284514801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 msgr2=0x558ce1af58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:25.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.048+0000 7ff4bac42740 1 --2- 172.21.15.2:0/4284514801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 0x558ce1af58e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff49c02a930 tx=0x7ff49c0317e0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.048+0000 7ff4bac42740 1 -- 172.21.15.2:0/4284514801 shutdown_connections 2024-09-10T08:19:25.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.048+0000 7ff4bac42740 1 --2- 172.21.15.2:0/4284514801 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558ce1b08d70 0x558ce1b0b190 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.048+0000 7ff4bac42740 1 --2- 172.21.15.2:0/4284514801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 0x558ce1af5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.048+0000 7ff4bac42740 1 --2- 172.21.15.2:0/4284514801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 0x558ce1af58e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.048+0000 7ff4bac42740 1 --2- 172.21.15.2:0/4284514801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aedac0 0x558ce1af8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.048+0000 7ff4bac42740 1 -- 172.21.15.2:0/4284514801 >> 172.21.15.2:0/4284514801 conn(0x558ce1aea050 msgr2=0x558ce1aefd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:25.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.048+0000 7ff497400640 0 client.18624 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:25.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.048+0000 7ff4bac42740 1 -- 172.21.15.2:0/4284514801 shutdown_connections 2024-09-10T08:19:25.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.048+0000 7ff4bac42740 1 -- 172.21.15.2:0/4284514801 wait complete. 2024-09-10T08:19:25.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.048+0000 7ff4bac42740 20 client.18624 trim_cache size 0 max 16384 2024-09-10T08:19:25.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.066+0000 7ff4bac42740 1 Processor -- start 2024-09-10T08:19:25.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.066+0000 7ff4bac42740 1 -- start start 2024-09-10T08:19:25.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.067+0000 7ff4bac42740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1aedac0 0x558ce1b10a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.067+0000 7ff4bac42740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aee3e0 0x558ce1b10fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.067+0000 7ff4bac42740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1af8ca0 0x558ce1ae89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.067+0000 7ff4bac42740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01b3990 con 0x558ce1af8ca0 2024-09-10T08:19:25.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.067+0000 7ff4bac42740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01b1260 con 0x558ce1aee3e0 2024-09-10T08:19:25.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.067+0000 7ff4bac42740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01af0b0 con 0x558ce1aedac0 2024-09-10T08:19:25.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.067+0000 7ff4b6200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1aedac0 0x558ce1b10a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.067+0000 7ff4b6200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1aedac0 0x558ce1b10a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50896/0 (socket says 172.21.15.2:50896) 2024-09-10T08:19:25.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.067+0000 7ff4b6200640 1 -- 172.21.15.2:0/697747162 learned_addr learned my addr 172.21.15.2:0/697747162 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:25.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.067+0000 7ff4b6200640 1 --2- 172.21.15.2:0/697747162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1af8ca0 0x558ce1ae89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.067+0000 7ff4b4e00640 1 --2- 172.21.15.2:0/697747162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aee3e0 0x558ce1b10fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.068+0000 7ff4b6200640 1 -- 172.21.15.2:0/697747162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aee3e0 msgr2=0x558ce1b10fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:25.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.068+0000 7ff4b6200640 1 --2- 172.21.15.2:0/697747162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aee3e0 0x558ce1b10fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.068+0000 7ff4b6200640 1 -- 172.21.15.2:0/697747162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1af8ca0 msgr2=0x558ce1ae89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:25.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.068+0000 7ff4b6200640 1 --2- 172.21.15.2:0/697747162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1af8ca0 0x558ce1ae89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.068+0000 7ff4b6200640 1 -- 172.21.15.2:0/697747162 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce1af3e70 con 0x558ce1aedac0 2024-09-10T08:19:25.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.068+0000 7ff4b6200640 1 --2- 172.21.15.2:0/697747162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1aedac0 0x558ce1b10a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005b40 tx=0x7ff4a800f7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f937b5c78c45d0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:25.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.068+0000 7ff4a2000640 1 -- 172.21.15.2:0/697747162 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800fc80 con 0x558ce1aedac0 2024-09-10T08:19:25.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.068+0000 7ff4a2000640 1 -- 172.21.15.2:0/697747162 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a8011020 con 0x558ce1aedac0 2024-09-10T08:19:25.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.068+0000 7ff4a2000640 1 -- 172.21.15.2:0/697747162 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8010430 con 0x558ce1aedac0 2024-09-10T08:19:25.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.069+0000 7ff4bac42740 1 -- 172.21.15.2:0/697747162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1aedac0 msgr2=0x558ce1b10a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:25.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.069+0000 7ff4bac42740 1 --2- 172.21.15.2:0/697747162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1aedac0 0x558ce1b10a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005b40 tx=0x7ff4a800f7d0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.069+0000 7ff4bac42740 1 -- 172.21.15.2:0/697747162 shutdown_connections 2024-09-10T08:19:25.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.069+0000 7ff4bac42740 1 --2- 172.21.15.2:0/697747162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1af8ca0 0x558ce1ae89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.069+0000 7ff4bac42740 1 --2- 172.21.15.2:0/697747162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aee3e0 0x558ce1b10fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.069+0000 7ff4bac42740 1 --2- 172.21.15.2:0/697747162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1aedac0 0x558ce1b10a60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.069+0000 7ff4bac42740 1 -- 172.21.15.2:0/697747162 >> 172.21.15.2:0/697747162 conn(0x558ce1aea050 msgr2=0x558ce1af0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:25.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.069+0000 7ff4bac42740 1 -- 172.21.15.2:0/697747162 shutdown_connections 2024-09-10T08:19:25.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.069+0000 7ff4bac42740 1 -- 172.21.15.2:0/697747162 wait complete. 2024-09-10T08:19:25.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.069+0000 7ff4bac42740 1 -- start start 2024-09-10T08:19:25.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.070+0000 7ff4bac42740 1 Processor -- start 2024-09-10T08:19:25.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.070+0000 7ff4bac42740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aedac0 0x558ce1af2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.070+0000 7ff4bac42740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 0x558ce1af2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.070+0000 7ff4bac42740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 0x558ce1af33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.070+0000 7ff4bac42740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01af720 con 0x558ce1aee3e0 2024-09-10T08:19:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.070+0000 7ff4bac42740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01b0b90 con 0x558ce1aedac0 2024-09-10T08:19:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.070+0000 7ff4bac42740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01afdf0 con 0x558ce1af8ca0 2024-09-10T08:19:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.070+0000 7ff4bac42740 10 client.0 _ll_register_callbacks cb 0x7ff49c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.070+0000 7ff4b4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 0x558ce1af2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.071+0000 7ff4b6200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aedac0 0x558ce1af2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.071+0000 7ff4b4e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 0x558ce1af2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48796/0 (socket says 172.21.15.2:48796) 2024-09-10T08:19:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.071+0000 7ff4b4e00640 1 -- 172.21.15.2:0/2396626462 learned_addr learned my addr 172.21.15.2:0/2396626462 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.071+0000 7ff4b6200640 1 --2- 172.21.15.2:0/2396626462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 0x558ce1af33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.071+0000 7ff4b4e00640 1 -- 172.21.15.2:0/2396626462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 msgr2=0x558ce1af33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.071+0000 7ff4b4e00640 1 --2- 172.21.15.2:0/2396626462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 0x558ce1af33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.071+0000 7ff4b4e00640 1 -- 172.21.15.2:0/2396626462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aedac0 msgr2=0x558ce1af2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.071+0000 7ff4b4e00640 1 --2- 172.21.15.2:0/2396626462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aedac0 0x558ce1af2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.071+0000 7ff4b4e00640 1 -- 172.21.15.2:0/2396626462 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce1b72a60 con 0x558ce1aee3e0 2024-09-10T08:19:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.071+0000 7ff4b4e00640 1 --2- 172.21.15.2:0/2396626462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 0x558ce1af2eb0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff49c03da10 tx=0x7ff49c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3de056c9b22b67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.071+0000 7ff497400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.072+0000 7ff497400640 1 -- 172.21.15.2:0/2396626462 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c035030 con 0x558ce1aee3e0 2024-09-10T08:19:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.072+0000 7ff4bac42740 1 -- 172.21.15.2:0/2396626462 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a8011020 con 0x558ce1aee3e0 2024-09-10T08:19:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.072+0000 7ff497400640 1 -- 172.21.15.2:0/2396626462 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff49c005030 con 0x558ce1aee3e0 2024-09-10T08:19:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.072+0000 7ff497400640 1 -- 172.21.15.2:0/2396626462 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff49c062050 con 0x558ce1aee3e0 2024-09-10T08:19:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.072+0000 7ff4bac42740 10 client.18630 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.072+0000 7ff4bac42740 1 -- 172.21.15.2:0/2396626462 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558ce1b73d60 con 0x558ce1aee3e0 2024-09-10T08:19:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.072+0000 7ff497400640 1 -- 172.21.15.2:0/2396626462 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a8011020 con 0x558ce1aee3e0 2024-09-10T08:19:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.072+0000 7ff497400640 1 -- 172.21.15.2:0/2396626462 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff49c004970 con 0x558ce1aee3e0 2024-09-10T08:19:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.072+0000 7ff4bac42740 10 client.18630 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.072+0000 7ff4bac42740 10 client.18630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.073+0000 7ff4bac42740 20 client.18630 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.073+0000 7ff4bac42740 1 --2- 172.21.15.2:0/2396626462 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558ce1b74470 0x558ce1b76860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.073+0000 7ff4bac42740 4 client.18630 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:19:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.073+0000 7ff4bac42740 1 -- 172.21.15.2:0/2396626462 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558ce19d7fb0 con 0x558ce1b74470 2024-09-10T08:19:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.073+0000 7ff4b6200640 1 --2- 172.21.15.2:0/2396626462 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558ce1b74470 0x558ce1b76860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.073+0000 7ff4b6200640 1 --2- 172.21.15.2:0/2396626462 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558ce1b74470 0x558ce1b76860 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2220eb78b637f2a server_cookie=429a048510d6a85c in_seq=0 out_seq=0 2024-09-10T08:19:25.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.073+0000 7ff497400640 10 client.18630 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:25.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.074+0000 7ff497400640 1 -- 172.21.15.2:0/2396626462 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558ce19d7fb0 con 0x558ce1b74470 2024-09-10T08:19:25.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.074+0000 7ff497400640 10 client.18630 handle_command_reply: tid=0 2024-09-10T08:19:25.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.074+0000 7ff4bac42740 1 client.18630 shutdown 2024-09-10T08:19:25.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.075+0000 7ff4bac42740 1 -- 172.21.15.2:0/2396626462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 msgr2=0x558ce1af2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:25.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.075+0000 7ff4bac42740 1 --2- 172.21.15.2:0/2396626462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 0x558ce1af2eb0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff49c03da10 tx=0x7ff49c02a410 comp rx=0 tx=0).stop 2024-09-10T08:19:25.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.075+0000 7ff4bac42740 1 -- 172.21.15.2:0/2396626462 shutdown_connections 2024-09-10T08:19:25.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.075+0000 7ff4bac42740 1 --2- 172.21.15.2:0/2396626462 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558ce1b74470 0x558ce1b76860 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.075+0000 7ff4bac42740 1 --2- 172.21.15.2:0/2396626462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ce1af8ca0 0x558ce1af33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.075+0000 7ff4bac42740 1 --2- 172.21.15.2:0/2396626462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558ce1aee3e0 0x558ce1af2eb0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.075+0000 7ff4bac42740 1 --2- 172.21.15.2:0/2396626462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ce1aedac0 0x558ce1af2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.075+0000 7ff4bac42740 1 -- 172.21.15.2:0/2396626462 >> 172.21.15.2:0/2396626462 conn(0x558ce1aea050 msgr2=0x558ce1af7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:25.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.075+0000 7ff497400640 0 client.18630 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:25.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.075+0000 7ff4bac42740 1 -- 172.21.15.2:0/2396626462 shutdown_connections 2024-09-10T08:19:25.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.075+0000 7ff4bac42740 1 -- 172.21.15.2:0/2396626462 wait complete. 2024-09-10T08:19:25.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.075+0000 7ff4bac42740 20 client.18630 trim_cache size 0 max 16384 2024-09-10T08:19:25.079 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:19:25.079 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:19:25.079 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "81d30e72-0854-4b9a-a94b-4dc3efa635d0", 2024-09-10T08:19:25.079 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:19:25.079 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:19:25.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.076+0000 7ff4b8200640 1 -- 172.21.15.2:0/859316098 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4980a2b80 msgr2=0x7ff4980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:25.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.076+0000 7ff4b8200640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4980a2b80 0x7ff4980a5040 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40cd260 tx=0x7ff4a401f000 comp rx=0 tx=0).stop 2024-09-10T08:19:25.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.076+0000 7ff4b8200640 1 -- 172.21.15.2:0/859316098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b0103ff0 msgr2=0x7ff4b019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:25.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.076+0000 7ff4b8200640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b0103ff0 0x7ff4b019a6a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400e9f0 tx=0x7ff4a400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.076+0000 7ff4b8200640 1 -- 172.21.15.2:0/859316098 shutdown_connections 2024-09-10T08:19:25.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.076+0000 7ff4b8200640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4b0109120 0x7ff4b019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.076+0000 7ff4b8200640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4b01049a0 0x7ff4b019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.076+0000 7ff4b8200640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4980a2b80 0x7ff4980a5040 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.076+0000 7ff4b8200640 1 --2- 172.21.15.2:0/859316098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b0103ff0 0x7ff4b019a6a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:25.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.076+0000 7ff4b8200640 1 -- 172.21.15.2:0/859316098 >> 172.21.15.2:0/859316098 conn(0x7ff4b00ffa90 msgr2=0x7ff4b0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:25.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.076+0000 7ff4b8200640 1 -- 172.21.15.2:0/859316098 shutdown_connections 2024-09-10T08:19:25.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:25.076+0000 7ff4b8200640 1 -- 172.21.15.2:0/859316098 wait complete. 2024-09-10T08:19:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:25 smithi002 ceph-mon[30563]: pgmap v725: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-09-10T08:19:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:25 smithi155 ceph-mon[35405]: pgmap v725: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-09-10T08:19:25.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:25 smithi107 ceph-mon[35048]: pgmap v725: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-09-10T08:19:26.074 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:19:26.457 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:26.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:26 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:19:26.763 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:26 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:19:26.763 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:26 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:19:26.763 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:26 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:19:26.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:26 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:19:26.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:26 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:19:26.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:26 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:19:26.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:26 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:19:26.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:26 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:19:26.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:26 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:19:26.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:26 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:19:26.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:26 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:19:27.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.104+0000 7f14ece00640 1 -- 172.21.15.2:0/4001059637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e8106f10 msgr2=0x7f14e810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:27.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.104+0000 7f14ece00640 1 --2- 172.21.15.2:0/4001059637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e8106f10 0x7f14e810ea90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009ca0 tx=0x7f14e002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.105+0000 7f14ece00640 1 -- 172.21.15.2:0/4001059637 shutdown_connections 2024-09-10T08:19:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.105+0000 7f14ece00640 1 --2- 172.21.15.2:0/4001059637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e8106f10 0x7f14e810ea90 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.105+0000 7f14ece00640 1 --2- 172.21.15.2:0/4001059637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14e8102790 0x7f14e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.105+0000 7f14ece00640 1 --2- 172.21.15.2:0/4001059637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14e8101de0 0x7f14e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.105+0000 7f14ece00640 1 -- 172.21.15.2:0/4001059637 >> 172.21.15.2:0/4001059637 conn(0x7f14e80fd8c0 msgr2=0x7f14e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.105+0000 7f14ece00640 1 -- 172.21.15.2:0/4001059637 shutdown_connections 2024-09-10T08:19:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.105+0000 7f14ece00640 1 -- 172.21.15.2:0/4001059637 wait complete. 2024-09-10T08:19:27.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.106+0000 7f14ece00640 1 Processor -- start 2024-09-10T08:19:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.106+0000 7f14ece00640 1 -- start start 2024-09-10T08:19:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.107+0000 7f14ece00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14e8101de0 0x7f14e819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.107+0000 7f14ece00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14e8102790 0x7f14e819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.107+0000 7f14ece00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e8106f10 0x7f14e819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.107+0000 7f14ece00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8111fc0 con 0x7f14e8106f10 2024-09-10T08:19:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.107+0000 7f14ece00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8111e40 con 0x7f14e8101de0 2024-09-10T08:19:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.107+0000 7f14ece00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8112140 con 0x7f14e8102790 2024-09-10T08:19:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.107+0000 7f14e6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e8106f10 0x7f14e819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.107+0000 7f14e6800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e8106f10 0x7f14e819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40480/0 (socket says 172.21.15.2:40480) 2024-09-10T08:19:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.107+0000 7f14e6800640 1 -- 172.21.15.2:0/149601921 learned_addr learned my addr 172.21.15.2:0/149601921 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.107+0000 7f14e5e00640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14e8101de0 0x7f14e819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.107+0000 7f14dd400640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14e8102790 0x7f14e819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.108+0000 7f14e6800640 1 -- 172.21.15.2:0/149601921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14e8102790 msgr2=0x7f14e819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.108+0000 7f14e6800640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14e8102790 0x7f14e819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.108+0000 7f14e6800640 1 -- 172.21.15.2:0/149601921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14e8101de0 msgr2=0x7f14e819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.108+0000 7f14e6800640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14e8101de0 0x7f14e819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.108+0000 7f14e6800640 1 -- 172.21.15.2:0/149601921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e819f580 con 0x7f14e8106f10 2024-09-10T08:19:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.108+0000 7f14e5e00640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14e8101de0 0x7f14e819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.108+0000 7f14e6800640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e8106f10 0x7f14e819eea0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f14e0033040 tx=0x7f14e0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.108+0000 7f14df400640 1 -- 172.21.15.2:0/149601921 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0031650 con 0x7f14e8106f10 2024-09-10T08:19:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.108+0000 7f14df400640 1 -- 172.21.15.2:0/149601921 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e00317f0 con 0x7f14e8106f10 2024-09-10T08:19:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.108+0000 7f14df400640 1 -- 172.21.15.2:0/149601921 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e002fa40 con 0x7f14e8106f10 2024-09-10T08:19:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.108+0000 7f14ece00640 1 -- 172.21.15.2:0/149601921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e819f810 con 0x7f14e8106f10 2024-09-10T08:19:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.108+0000 7f14ece00640 1 -- 172.21.15.2:0/149601921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e81a7150 con 0x7f14e8106f10 2024-09-10T08:19:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.111+0000 7f14df400640 1 -- 172.21.15.2:0/149601921 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f14e002fc60 con 0x7f14e8106f10 2024-09-10T08:19:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.112+0000 7f14ef7e4740 1 Processor -- start 2024-09-10T08:19:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.112+0000 7f14ef7e4740 1 -- start start 2024-09-10T08:19:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.112+0000 7f14ef7e4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd1b90 0x564717cd1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.112+0000 7f14ef7e4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd24b0 0x564717cdc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.112+0000 7f14ef7e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 0x564717cdf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.112+0000 7f14ef7e4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564717c383e0 con 0x564717cd1b90 2024-09-10T08:19:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.112+0000 7f14ef7e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564717bdfd80 con 0x564717cdcd70 2024-09-10T08:19:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.112+0000 7f14ef7e4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564717c343a0 con 0x564717cd24b0 2024-09-10T08:19:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.112+0000 7f14dd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd1b90 0x564717cd1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.113+0000 7f14e5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd24b0 0x564717cdc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.113+0000 7f14dd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd1b90 0x564717cd1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40496/0 (socket says 172.21.15.2:40496) 2024-09-10T08:19:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.113+0000 7f14dd400640 1 -- 172.21.15.2:0/307829196 learned_addr learned my addr 172.21.15.2:0/307829196 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.113+0000 7f14e5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd24b0 0x564717cdc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54876/0 (socket says 172.21.15.2:54876) 2024-09-10T08:19:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.113+0000 7f14dd400640 1 --2- 172.21.15.2:0/307829196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 0x564717cdf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.113+0000 7f14e5e00640 1 -- 172.21.15.2:0/307829196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 msgr2=0x564717cdf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:27.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.113+0000 7f14e5e00640 1 --2- 172.21.15.2:0/307829196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 0x564717cdf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.113+0000 7f14e5e00640 1 -- 172.21.15.2:0/307829196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd1b90 msgr2=0x564717cd1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:27.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.113+0000 7f14e5e00640 1 --2- 172.21.15.2:0/307829196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd1b90 0x564717cd1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.113+0000 7f14e5e00640 1 -- 172.21.15.2:0/307829196 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564717bfeae0 con 0x564717cd24b0 2024-09-10T08:19:27.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.113+0000 7f14df400640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f14cc0a2b80 0x7f14cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.113+0000 7f14df400640 1 -- 172.21.15.2:0/149601921 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f14e00f0a40 con 0x7f14e8106f10 2024-09-10T08:19:27.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.113+0000 7f14e6800640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f14cc0a2b80 0x7f14cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.114+0000 7f14e5e00640 1 --2- 172.21.15.2:0/307829196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd24b0 0x564717cdc830 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f14d80103d0 tx=0x7f14d8010890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2009de794fb2de23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:27.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.114+0000 7f14d7400640 1 -- 172.21.15.2:0/307829196 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8004220 con 0x564717cd24b0 2024-09-10T08:19:27.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.114+0000 7f14d7400640 1 -- 172.21.15.2:0/307829196 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d80043c0 con 0x564717cd24b0 2024-09-10T08:19:27.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.114+0000 7f14d7400640 1 -- 172.21.15.2:0/307829196 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8011a20 con 0x564717cd24b0 2024-09-10T08:19:27.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.115+0000 7f14ef7e4740 1 -- 172.21.15.2:0/307829196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd24b0 msgr2=0x564717cdc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:27.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.115+0000 7f14e6800640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f14cc0a2b80 0x7f14cc0a5040 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f14e00bcb50 tx=0x7f14e00095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:27.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.115+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/307829196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd24b0 0x564717cdc830 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f14d80103d0 tx=0x7f14d8010890 comp rx=0 tx=0).stop 2024-09-10T08:19:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.115+0000 7f14ef7e4740 1 -- 172.21.15.2:0/307829196 shutdown_connections 2024-09-10T08:19:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.115+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/307829196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 0x564717cdf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.115+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/307829196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd24b0 0x564717cdc830 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.115+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/307829196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd1b90 0x564717cd1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.115+0000 7f14ef7e4740 1 -- 172.21.15.2:0/307829196 >> 172.21.15.2:0/307829196 conn(0x564717cce120 msgr2=0x564717cce500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.115+0000 7f14ef7e4740 1 -- 172.21.15.2:0/307829196 shutdown_connections 2024-09-10T08:19:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.115+0000 7f14ef7e4740 1 -- 172.21.15.2:0/307829196 wait complete. 2024-09-10T08:19:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.115+0000 7f14ef7e4740 1 -- start start 2024-09-10T08:19:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.116+0000 7f14ef7e4740 1 Processor -- start 2024-09-10T08:19:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.116+0000 7f14ef7e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cd1b90 0x564717cd9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.116+0000 7f14ef7e4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd24b0 0x564717cda210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.116+0000 7f14ef7e4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cdcd70 0x564717cda750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.116+0000 7f14ef7e4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564717cdf9e0 con 0x564717cdcd70 2024-09-10T08:19:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.116+0000 7f14ef7e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564717bf30f0 con 0x564717cd1b90 2024-09-10T08:19:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.116+0000 7f14ef7e4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564717cdfb60 con 0x564717cd24b0 2024-09-10T08:19:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.116+0000 7f14ef7e4740 10 client.0 _ll_register_callbacks cb 0x564717be49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.116+0000 7f14dd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cdcd70 0x564717cda750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.116+0000 7f14e5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd24b0 0x564717cda210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.117+0000 7f14dd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cdcd70 0x564717cda750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40500/0 (socket says 172.21.15.2:40500) 2024-09-10T08:19:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.117+0000 7f14dd400640 1 -- 172.21.15.2:0/2676072840 learned_addr learned my addr 172.21.15.2:0/2676072840 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.117+0000 7f14dd400640 1 --2- 172.21.15.2:0/2676072840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cd1b90 0x564717cd9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.117+0000 7f14dd400640 1 -- 172.21.15.2:0/2676072840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd24b0 msgr2=0x564717cda210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.117+0000 7f14dd400640 1 --2- 172.21.15.2:0/2676072840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd24b0 0x564717cda210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.117+0000 7f14dd400640 1 -- 172.21.15.2:0/2676072840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cd1b90 msgr2=0x564717cd9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:27.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.117+0000 7f14dd400640 1 --2- 172.21.15.2:0/2676072840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cd1b90 0x564717cd9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.117+0000 7f14dd400640 1 -- 172.21.15.2:0/2676072840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647179a93c0 con 0x564717cdcd70 2024-09-10T08:19:27.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.117+0000 7f14dd400640 1 --2- 172.21.15.2:0/2676072840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cdcd70 0x564717cda750 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f14d0013420 tx=0x7f14d0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c207a93c81d43881 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:27.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.117+0000 7f14cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:27.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.117+0000 7f14cbe00640 1 -- 172.21.15.2:0/2676072840 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0004260 con 0x564717cdcd70 2024-09-10T08:19:27.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.117+0000 7f14cbe00640 1 -- 172.21.15.2:0/2676072840 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f14d0004400 con 0x564717cdcd70 2024-09-10T08:19:27.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.117+0000 7f14ef7e4740 1 -- 172.21.15.2:0/2676072840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d80043c0 con 0x564717cdcd70 2024-09-10T08:19:27.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.118+0000 7f14cbe00640 1 -- 172.21.15.2:0/2676072840 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14d0005020 con 0x564717cdcd70 2024-09-10T08:19:27.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.118+0000 7f14ef7e4740 10 client.18642 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:27.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.118+0000 7f14ef7e4740 1 -- 172.21.15.2:0/2676072840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564717ceb710 con 0x564717cdcd70 2024-09-10T08:19:27.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.118+0000 7f14cbe00640 1 -- 172.21.15.2:0/2676072840 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d80043c0 con 0x564717cdcd70 2024-09-10T08:19:27.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.118+0000 7f14cbe00640 1 -- 172.21.15.2:0/2676072840 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f14d000f7c0 con 0x564717cdcd70 2024-09-10T08:19:27.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.118+0000 7f14ef7e4740 10 client.18642 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:27.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.118+0000 7f14ef7e4740 10 client.18642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:27.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.118+0000 7f14ef7e4740 20 client.18642 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:27.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.118+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/2676072840 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564717cecb20 0x564717ceef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.118+0000 7f14ef7e4740 4 client.18642 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:27.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.118+0000 7f14ef7e4740 1 -- 172.21.15.2:0/2676072840 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564717c45950 con 0x564717cecb20 2024-09-10T08:19:27.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.119+0000 7f14e5e00640 1 --2- 172.21.15.2:0/2676072840 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564717cecb20 0x564717ceef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.119+0000 7f14e5e00640 1 --2- 172.21.15.2:0/2676072840 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564717cecb20 0x564717ceef30 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8509932fdb182e3c server_cookie=be6c9ff12141a0d4 in_seq=0 out_seq=0 2024-09-10T08:19:27.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.119+0000 7f14cbe00640 10 client.18642 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:27.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.120+0000 7f14cbe00640 1 -- 172.21.15.2:0/2676072840 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564717c45950 con 0x564717cecb20 2024-09-10T08:19:27.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.120+0000 7f14cbe00640 10 client.18642 handle_command_reply: tid=0 2024-09-10T08:19:27.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.120+0000 7f14ef7e4740 1 client.18642 shutdown 2024-09-10T08:19:27.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.120+0000 7f14ef7e4740 1 -- 172.21.15.2:0/2676072840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cdcd70 msgr2=0x564717cda750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:27.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.120+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/2676072840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cdcd70 0x564717cda750 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f14d0013420 tx=0x7f14d0005f90 comp rx=0 tx=0).stop 2024-09-10T08:19:27.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.120+0000 7f14ef7e4740 1 -- 172.21.15.2:0/2676072840 shutdown_connections 2024-09-10T08:19:27.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.120+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/2676072840 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564717cecb20 0x564717ceef30 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.120+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/2676072840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cdcd70 0x564717cda750 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.120+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/2676072840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd24b0 0x564717cda210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.120+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/2676072840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cd1b90 0x564717cd9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.120+0000 7f14cbe00640 0 client.18642 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:27.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.120+0000 7f14ef7e4740 1 -- 172.21.15.2:0/2676072840 >> 172.21.15.2:0/2676072840 conn(0x564717cce120 msgr2=0x564717cd3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:27.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.121+0000 7f14ef7e4740 1 -- 172.21.15.2:0/2676072840 shutdown_connections 2024-09-10T08:19:27.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.121+0000 7f14ef7e4740 1 -- 172.21.15.2:0/2676072840 wait complete. 2024-09-10T08:19:27.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.121+0000 7f14ef7e4740 20 client.18642 trim_cache size 0 max 16384 2024-09-10T08:19:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.139+0000 7f14ef7e4740 1 Processor -- start 2024-09-10T08:19:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.139+0000 7f14ef7e4740 1 -- start start 2024-09-10T08:19:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.139+0000 7f14ef7e4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd1b90 0x564717cd3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.139+0000 7f14ef7e4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd24b0 0x564717cf4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.139+0000 7f14ef7e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 0x564717cf4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.139+0000 7f14ef7e4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81b38f0 con 0x564717cd24b0 2024-09-10T08:19:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.139+0000 7f14ef7e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81b11c0 con 0x564717cdcd70 2024-09-10T08:19:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.139+0000 7f14ef7e4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81af010 con 0x564717cd1b90 2024-09-10T08:19:27.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.139+0000 7f14e5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd24b0 0x564717cf4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.139+0000 7f14e5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd24b0 0x564717cf4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40510/0 (socket says 172.21.15.2:40510) 2024-09-10T08:19:27.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.139+0000 7f14e5e00640 1 -- 172.21.15.2:0/2617270693 learned_addr learned my addr 172.21.15.2:0/2617270693 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:27.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.139+0000 7f14dd400640 1 --2- 172.21.15.2:0/2617270693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd1b90 0x564717cd3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.140+0000 7f14dd400640 1 --2- 172.21.15.2:0/2617270693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 0x564717cf4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.140+0000 7f14e5e00640 1 -- 172.21.15.2:0/2617270693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd1b90 msgr2=0x564717cd3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:27.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.140+0000 7f14e5e00640 1 --2- 172.21.15.2:0/2617270693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd1b90 0x564717cd3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.140+0000 7f14e5e00640 1 -- 172.21.15.2:0/2617270693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 msgr2=0x564717cf4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:27.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.140+0000 7f14e5e00640 1 --2- 172.21.15.2:0/2617270693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 0x564717cf4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.140+0000 7f14e5e00640 1 -- 172.21.15.2:0/2617270693 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564717ccedb0 con 0x564717cd24b0 2024-09-10T08:19:27.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.140+0000 7f14dd400640 1 --2- 172.21.15.2:0/2617270693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 0x564717cf4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:19:27.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.140+0000 7f14e5e00640 1 --2- 172.21.15.2:0/2617270693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd24b0 0x564717cf4800 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f14d80103d0 tx=0x7f14d800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9c53fd182ed715 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:27.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.140+0000 7f14d6a00640 1 -- 172.21.15.2:0/2617270693 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8004220 con 0x564717cd24b0 2024-09-10T08:19:27.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.140+0000 7f14d6a00640 1 -- 172.21.15.2:0/2617270693 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d80043c0 con 0x564717cd24b0 2024-09-10T08:19:27.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.140+0000 7f14d6a00640 1 -- 172.21.15.2:0/2617270693 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8003c90 con 0x564717cd24b0 2024-09-10T08:19:27.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.141+0000 7f14ef7e4740 1 -- 172.21.15.2:0/2617270693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd24b0 msgr2=0x564717cf4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:27.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.141+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/2617270693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd24b0 0x564717cf4800 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f14d80103d0 tx=0x7f14d800b390 comp rx=0 tx=0).stop 2024-09-10T08:19:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.141+0000 7f14ef7e4740 1 -- 172.21.15.2:0/2617270693 shutdown_connections 2024-09-10T08:19:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.141+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/2617270693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 0x564717cf4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.141+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/2617270693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd24b0 0x564717cf4800 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.141+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/2617270693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd1b90 0x564717cd3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.141+0000 7f14ef7e4740 1 -- 172.21.15.2:0/2617270693 >> 172.21.15.2:0/2617270693 conn(0x564717cce120 msgr2=0x564717cd35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.141+0000 7f14ef7e4740 1 -- 172.21.15.2:0/2617270693 shutdown_connections 2024-09-10T08:19:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.141+0000 7f14ef7e4740 1 -- 172.21.15.2:0/2617270693 wait complete. 2024-09-10T08:19:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.142+0000 7f14ef7e4740 1 -- start start 2024-09-10T08:19:27.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.142+0000 7f14ef7e4740 1 Processor -- start 2024-09-10T08:19:27.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.142+0000 7f14ef7e4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd1b90 0x564717ce0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.142+0000 7f14ef7e4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd24b0 0x564717ce1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.142+0000 7f14ef7e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 0x564717ce15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.142+0000 7f14ef7e4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81af680 con 0x564717cd24b0 2024-09-10T08:19:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.142+0000 7f14ef7e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81b0af0 con 0x564717cdcd70 2024-09-10T08:19:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.142+0000 7f14ef7e4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81afd50 con 0x564717cd1b90 2024-09-10T08:19:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.142+0000 7f14ef7e4740 10 client.0 _ll_register_callbacks cb 0x7f14d0013730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.143+0000 7f14e5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd24b0 0x564717ce1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.143+0000 7f14dd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd1b90 0x564717ce0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.143+0000 7f14dd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd1b90 0x564717ce0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54894/0 (socket says 172.21.15.2:54894) 2024-09-10T08:19:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.143+0000 7f14dd400640 1 -- 172.21.15.2:0/4175190005 learned_addr learned my addr 172.21.15.2:0/4175190005 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.143+0000 7f14dd400640 1 --2- 172.21.15.2:0/4175190005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 0x564717ce15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.143+0000 7f14e5e00640 1 -- 172.21.15.2:0/4175190005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd1b90 msgr2=0x564717ce0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.143+0000 7f14e5e00640 1 --2- 172.21.15.2:0/4175190005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd1b90 0x564717ce0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.143+0000 7f14e5e00640 1 -- 172.21.15.2:0/4175190005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 msgr2=0x564717ce15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.143+0000 7f14e5e00640 1 --2- 172.21.15.2:0/4175190005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 0x564717ce15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.143+0000 7f14e5e00640 1 -- 172.21.15.2:0/4175190005 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564717d56800 con 0x564717cd24b0 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.144+0000 7f14e5e00640 1 --2- 172.21.15.2:0/4175190005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd24b0 0x564717ce1060 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x564717be4a60 tx=0x7f14d800bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339715fa39ec4221 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.144+0000 7f14cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.144+0000 7f14cbe00640 1 -- 172.21.15.2:0/4175190005 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8004020 con 0x564717cd24b0 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.144+0000 7f14ef7e4740 1 -- 172.21.15.2:0/4175190005 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d80043c0 con 0x564717cd24b0 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.144+0000 7f14cbe00640 1 -- 172.21.15.2:0/4175190005 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f14d80041c0 con 0x564717cd24b0 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.144+0000 7f14cbe00640 1 -- 172.21.15.2:0/4175190005 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14d8041050 con 0x564717cd24b0 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.144+0000 7f14ef7e4740 10 client.18648 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.144+0000 7f14ef7e4740 1 -- 172.21.15.2:0/4175190005 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564717d57a40 con 0x564717cd24b0 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.144+0000 7f14cbe00640 1 -- 172.21.15.2:0/4175190005 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d80043c0 con 0x564717cd24b0 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.145+0000 7f14cbe00640 1 -- 172.21.15.2:0/4175190005 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f14d8004780 con 0x564717cd24b0 2024-09-10T08:19:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.145+0000 7f14ef7e4740 10 client.18648 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.145+0000 7f14ef7e4740 10 client.18648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.145+0000 7f14ef7e4740 20 client.18648 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.145+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/4175190005 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564717d57f50 0x564717d5a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.145+0000 7f14ef7e4740 4 client.18648 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:19:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.145+0000 7f14ef7e4740 1 -- 172.21.15.2:0/4175190005 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564717c3fe60 con 0x564717d57f50 2024-09-10T08:19:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.145+0000 7f14dd400640 1 --2- 172.21.15.2:0/4175190005 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564717d57f50 0x564717d5a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14dd400640 1 --2- 172.21.15.2:0/4175190005 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564717d57f50 0x564717d5a340 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce5b55f46386687 server_cookie=1dedc18350c09add in_seq=0 out_seq=0 2024-09-10T08:19:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14cbe00640 10 client.18648 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:27.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14cbe00640 1 -- 172.21.15.2:0/4175190005 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564717c3fe60 con 0x564717d57f50 2024-09-10T08:19:27.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14cbe00640 10 client.18648 handle_command_reply: tid=0 2024-09-10T08:19:27.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14ef7e4740 1 client.18648 shutdown 2024-09-10T08:19:27.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14ef7e4740 1 -- 172.21.15.2:0/4175190005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd24b0 msgr2=0x564717ce1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:27.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/4175190005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd24b0 0x564717ce1060 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x564717be4a60 tx=0x7f14d800bb20 comp rx=0 tx=0).stop 2024-09-10T08:19:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14ef7e4740 1 -- 172.21.15.2:0/4175190005 shutdown_connections 2024-09-10T08:19:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/4175190005 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564717d57f50 0x564717d5a340 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/4175190005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564717cdcd70 0x564717ce15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/4175190005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564717cd24b0 0x564717ce1060 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14ef7e4740 1 --2- 172.21.15.2:0/4175190005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564717cd1b90 0x564717ce0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14ef7e4740 1 -- 172.21.15.2:0/4175190005 >> 172.21.15.2:0/4175190005 conn(0x564717cce120 msgr2=0x564717cd3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14cbe00640 0 client.18648 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14ef7e4740 1 -- 172.21.15.2:0/4175190005 shutdown_connections 2024-09-10T08:19:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.146+0000 7f14ef7e4740 1 -- 172.21.15.2:0/4175190005 wait complete. 2024-09-10T08:19:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.147+0000 7f14ef7e4740 20 client.18648 trim_cache size 0 max 16384 2024-09-10T08:19:27.151 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:19:27.151 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:19:27.151 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:19:27.151 INFO:teuthology.orchestra.run.smithi002.stdout: "81d30e72-0854-4b9a-a94b-4dc3efa635d0": { 2024-09-10T08:19:27.151 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:19:27.151 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "81d30e72-0854-4b9a-a94b-4dc3efa635d0", 2024-09-10T08:19:27.151 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:19:27.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:19:27.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:19:27.151 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:19:27.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.147+0000 7f14ece00640 1 -- 172.21.15.2:0/149601921 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f14cc0a2b80 msgr2=0x7f14cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:27.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.147+0000 7f14ece00640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f14cc0a2b80 0x7f14cc0a5040 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f14e00bcb50 tx=0x7f14e00095c0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.147+0000 7f14ece00640 1 -- 172.21.15.2:0/149601921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e8106f10 msgr2=0x7f14e819eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:27.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.147+0000 7f14ece00640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e8106f10 0x7f14e819eea0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f14e0033040 tx=0x7f14e0003f10 comp rx=0 tx=0).stop 2024-09-10T08:19:27.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.147+0000 7f14ece00640 1 -- 172.21.15.2:0/149601921 shutdown_connections 2024-09-10T08:19:27.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.147+0000 7f14ece00640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e8106f10 0x7f14e819eea0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.147+0000 7f14ece00640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f14cc0a2b80 0x7f14cc0a5040 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.147+0000 7f14ece00640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14e8102790 0x7f14e819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.147+0000 7f14ece00640 1 --2- 172.21.15.2:0/149601921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14e8101de0 0x7f14e819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:27.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.147+0000 7f14ece00640 1 -- 172.21.15.2:0/149601921 >> 172.21.15.2:0/149601921 conn(0x7f14e80fd8c0 msgr2=0x7f14e80ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:27.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.148+0000 7f14ece00640 1 -- 172.21.15.2:0/149601921 shutdown_connections 2024-09-10T08:19:27.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:27.148+0000 7f14ece00640 1 -- 172.21.15.2:0/149601921 wait complete. 2024-09-10T08:19:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:27 smithi002 ceph-mon[30563]: pgmap v726: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T08:19:27.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:27 smithi155 ceph-mon[35405]: pgmap v726: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T08:19:27.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:27 smithi107 ceph-mon[35048]: pgmap v726: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T08:19:28.153 INFO:tasks.cephfs.filesystem:scrub status for tag:81d30e72-0854-4b9a-a94b-4dc3efa635d0 - {'path': '/', 'tag': '81d30e72-0854-4b9a-a94b-4dc3efa635d0', 'options': 'recursive,force'} 2024-09-10T08:19:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.806+0000 7f6e1c400640 1 -- 172.21.15.2:0/3521317724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e14077df0 msgr2=0x7f6e14078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.806+0000 7f6e1c400640 1 --2- 172.21.15.2:0/3521317724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e14077df0 0x7f6e14078250 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009900 tx=0x7f6e0002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.806+0000 7f6e1c400640 1 -- 172.21.15.2:0/3521317724 shutdown_connections 2024-09-10T08:19:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.806+0000 7f6e1c400640 1 --2- 172.21.15.2:0/3521317724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e14078790 0x7f6e14079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.806+0000 7f6e1c400640 1 --2- 172.21.15.2:0/3521317724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e14077df0 0x7f6e14078250 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.806+0000 7f6e1c400640 1 --2- 172.21.15.2:0/3521317724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e14076bb0 0x7f6e14076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.806+0000 7f6e1c400640 1 -- 172.21.15.2:0/3521317724 >> 172.21.15.2:0/3521317724 conn(0x7f6e140ff4e0 msgr2=0x7f6e14101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.806+0000 7f6e1c400640 1 -- 172.21.15.2:0/3521317724 shutdown_connections 2024-09-10T08:19:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.806+0000 7f6e1c400640 1 -- 172.21.15.2:0/3521317724 wait complete. 2024-09-10T08:19:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.807+0000 7f6e1c400640 1 Processor -- start 2024-09-10T08:19:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.808+0000 7f6e1c400640 1 -- start start 2024-09-10T08:19:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.808+0000 7f6e1c400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e14076bb0 0x7f6e1419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.808+0000 7f6e1c400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e14077df0 0x7f6e1419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.808+0000 7f6e1c400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e14078790 0x7f6e141a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.808+0000 7f6e1c400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14113e30 con 0x7f6e14077df0 2024-09-10T08:19:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.808+0000 7f6e1c400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14113cb0 con 0x7f6e14078790 2024-09-10T08:19:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.808+0000 7f6e1c400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14113fb0 con 0x7f6e14076bb0 2024-09-10T08:19:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.808+0000 7f6e19000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e14077df0 0x7f6e1419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.808+0000 7f6e19000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e14077df0 0x7f6e1419e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40518/0 (socket says 172.21.15.2:40518) 2024-09-10T08:19:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.808+0000 7f6e19000640 1 -- 172.21.15.2:0/701877602 learned_addr learned my addr 172.21.15.2:0/701877602 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.808+0000 7f6e19a00640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e14076bb0 0x7f6e1419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.808+0000 7f6e1a400640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e14078790 0x7f6e141a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.809+0000 7f6e19000640 1 -- 172.21.15.2:0/701877602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e14076bb0 msgr2=0x7f6e1419e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.809+0000 7f6e19000640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e14076bb0 0x7f6e1419e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.809+0000 7f6e19000640 1 -- 172.21.15.2:0/701877602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e14078790 msgr2=0x7f6e141a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.809+0000 7f6e19000640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e14078790 0x7f6e141a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.809+0000 7f6e19000640 1 -- 172.21.15.2:0/701877602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e141a6040 con 0x7f6e14077df0 2024-09-10T08:19:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.809+0000 7f6e19a00640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e14076bb0 0x7f6e1419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.809+0000 7f6e19000640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e14077df0 0x7f6e1419e960 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009230 tx=0x7f6e000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.809+0000 7f6e12000640 1 -- 172.21.15.2:0/701877602 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0003f030 con 0x7f6e14077df0 2024-09-10T08:19:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.809+0000 7f6e1c400640 1 -- 172.21.15.2:0/701877602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e141a62d0 con 0x7f6e14077df0 2024-09-10T08:19:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.809+0000 7f6e12000640 1 -- 172.21.15.2:0/701877602 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e00003ce0 con 0x7f6e14077df0 2024-09-10T08:19:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.810+0000 7f6e12000640 1 -- 172.21.15.2:0/701877602 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00049060 con 0x7f6e14077df0 2024-09-10T08:19:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.810+0000 7f6e1c400640 1 -- 172.21.15.2:0/701877602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e141a6810 con 0x7f6e14077df0 2024-09-10T08:19:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.813+0000 7f6e1f239740 1 Processor -- start 2024-09-10T08:19:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.813+0000 7f6e12000640 1 -- 172.21.15.2:0/701877602 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6e00042030 con 0x7f6e14077df0 2024-09-10T08:19:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.813+0000 7f6e1f239740 1 -- start start 2024-09-10T08:19:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.813+0000 7f6e1f239740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee4c090 0x55cd2ee4c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.813+0000 7f6e1f239740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee51490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.813+0000 7f6e1f239740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee51a60 0x55cd2ee53f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.813+0000 7f6e1f239740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cd2edb1100 con 0x55cd2ee51a60 2024-09-10T08:19:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.813+0000 7f6e1f239740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cd2ed591b0 con 0x55cd2ee4c090 2024-09-10T08:19:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.813+0000 7f6e1f239740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cd2edad0c0 con 0x55cd2ee4c9d0 2024-09-10T08:19:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.814+0000 7f6e19a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee51490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.814+0000 7f6e1a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee4c090 0x55cd2ee4c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.814+0000 7f6e19a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee51490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54918/0 (socket says 172.21.15.2:54918) 2024-09-10T08:19:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.814+0000 7f6e19000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee51a60 0x55cd2ee53f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.814+0000 7f6e19a00640 1 -- 172.21.15.2:0/2700559671 learned_addr learned my addr 172.21.15.2:0/2700559671 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.814+0000 7f6e1a400640 1 -- 172.21.15.2:0/2700559671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c9d0 msgr2=0x55cd2ee51490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.814+0000 7f6e1a400640 1 --2- 172.21.15.2:0/2700559671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee51490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.814+0000 7f6e1a400640 1 -- 172.21.15.2:0/2700559671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee51a60 msgr2=0x55cd2ee53f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.814+0000 7f6e1a400640 1 --2- 172.21.15.2:0/2700559671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee51a60 0x55cd2ee53f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.814+0000 7f6e1a400640 1 -- 172.21.15.2:0/2700559671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd2eac4210 con 0x55cd2ee4c090 2024-09-10T08:19:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.814+0000 7f6e1a400640 1 --2- 172.21.15.2:0/2700559671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee4c090 0x55cd2ee4c490 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009f00 tx=0x7f6e0c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3afa15d785785696 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.814+0000 7f6e12000640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6dfc0a2c30 0x7f6dfc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.815+0000 7f6e12000640 1 -- 172.21.15.2:0/701877602 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6e000efbe0 con 0x7f6e14077df0 2024-09-10T08:19:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.815+0000 7f6e06a00640 1 -- 172.21.15.2:0/2700559671 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c004d20 con 0x55cd2ee4c090 2024-09-10T08:19:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.815+0000 7f6e06a00640 1 -- 172.21.15.2:0/2700559671 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0c004030 con 0x55cd2ee4c090 2024-09-10T08:19:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.815+0000 7f6e06a00640 1 -- 172.21.15.2:0/2700559671 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c005220 con 0x55cd2ee4c090 2024-09-10T08:19:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.815+0000 7f6e19a00640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6dfc0a2c30 0x7f6dfc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.815+0000 7f6e19a00640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6dfc0a2c30 0x7f6dfc0a50f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55cd2ee4d400 tx=0x7f6e08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.815+0000 7f6e1f239740 1 -- 172.21.15.2:0/2700559671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee4c090 msgr2=0x55cd2ee4c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.815+0000 7f6e1f239740 1 --2- 172.21.15.2:0/2700559671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee4c090 0x55cd2ee4c490 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009f00 tx=0x7f6e0c011810 comp rx=0 tx=0).stop 2024-09-10T08:19:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.815+0000 7f6e1f239740 1 -- 172.21.15.2:0/2700559671 shutdown_connections 2024-09-10T08:19:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.815+0000 7f6e1f239740 1 --2- 172.21.15.2:0/2700559671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee51a60 0x55cd2ee53f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.815+0000 7f6e1f239740 1 --2- 172.21.15.2:0/2700559671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee51490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.815+0000 7f6e1f239740 1 --2- 172.21.15.2:0/2700559671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee4c090 0x55cd2ee4c490 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.815+0000 7f6e1f239740 1 -- 172.21.15.2:0/2700559671 >> 172.21.15.2:0/2700559671 conn(0x55cd2ee47700 msgr2=0x55cd2ee47b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.816+0000 7f6e1f239740 1 -- 172.21.15.2:0/2700559671 shutdown_connections 2024-09-10T08:19:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.816+0000 7f6e1f239740 1 -- 172.21.15.2:0/2700559671 wait complete. 2024-09-10T08:19:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.816+0000 7f6e1f239740 1 -- start start 2024-09-10T08:19:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.816+0000 7f6e1f239740 1 Processor -- start 2024-09-10T08:19:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e1f239740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c090 0x55cd2ee56220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e1f239740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee56760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e1f239740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee51a60 0x55cd2ee5d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e1f239740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cd2ee54a30 con 0x55cd2ee4c9d0 2024-09-10T08:19:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e1f239740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cd2ed3a090 con 0x55cd2ee51a60 2024-09-10T08:19:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e1f239740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cd2ee54bb0 con 0x55cd2ee4c090 2024-09-10T08:19:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e1f239740 10 client.0 _ll_register_callbacks cb 0x55cd2ed8ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e19000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee56760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e19000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee56760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40522/0 (socket says 172.21.15.2:40522) 2024-09-10T08:19:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e19000640 1 -- 172.21.15.2:0/4249415094 learned_addr learned my addr 172.21.15.2:0/4249415094 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e19a00640 1 --2- 172.21.15.2:0/4249415094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c090 0x55cd2ee56220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e1a400640 1 --2- 172.21.15.2:0/4249415094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee51a60 0x55cd2ee5d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e19000640 1 -- 172.21.15.2:0/4249415094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c090 msgr2=0x55cd2ee56220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e19000640 1 --2- 172.21.15.2:0/4249415094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c090 0x55cd2ee56220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e19000640 1 -- 172.21.15.2:0/4249415094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee51a60 msgr2=0x55cd2ee5d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e19000640 1 --2- 172.21.15.2:0/4249415094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee51a60 0x55cd2ee5d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.817+0000 7f6e19000640 1 -- 172.21.15.2:0/4249415094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd2eb90620 con 0x55cd2ee4c9d0 2024-09-10T08:19:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.818+0000 7f6e19000640 1 --2- 172.21.15.2:0/4249415094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee56760 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009a20 tx=0x7f6e000f32a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a26743e3cd04f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.818+0000 7f6dfb400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.818+0000 7f6dfb400640 1 -- 172.21.15.2:0/4249415094 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00035070 con 0x55cd2ee4c9d0 2024-09-10T08:19:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.818+0000 7f6e1f239740 1 -- 172.21.15.2:0/4249415094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e0c004030 con 0x55cd2ee4c9d0 2024-09-10T08:19:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.818+0000 7f6dfb400640 1 -- 172.21.15.2:0/4249415094 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6e0003b460 con 0x55cd2ee4c9d0 2024-09-10T08:19:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.818+0000 7f6dfb400640 1 -- 172.21.15.2:0/4249415094 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6e000b83a0 con 0x55cd2ee4c9d0 2024-09-10T08:19:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.818+0000 7f6e1f239740 10 client.18660 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.818+0000 7f6e1f239740 1 -- 172.21.15.2:0/4249415094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cd2ee60590 con 0x55cd2ee4c9d0 2024-09-10T08:19:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.819+0000 7f6dfb400640 1 -- 172.21.15.2:0/4249415094 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0c004030 con 0x55cd2ee4c9d0 2024-09-10T08:19:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.819+0000 7f6dfb400640 1 -- 172.21.15.2:0/4249415094 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6e000b8770 con 0x55cd2ee4c9d0 2024-09-10T08:19:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.819+0000 7f6e1f239740 10 client.18660 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.819+0000 7f6e1f239740 10 client.18660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.819+0000 7f6e1f239740 20 client.18660 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.819+0000 7f6e1f239740 1 --2- 172.21.15.2:0/4249415094 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd2ee61d40 0x55cd2ee64180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.819+0000 7f6e1f239740 4 client.18660 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.819+0000 7f6e1f239740 1 -- 172.21.15.2:0/4249415094 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd2edbe830 con 0x55cd2ee61d40 2024-09-10T08:19:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.819+0000 7f6e19a00640 1 --2- 172.21.15.2:0/4249415094 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd2ee61d40 0x55cd2ee64180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.820+0000 7f6e19a00640 1 --2- 172.21.15.2:0/4249415094 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd2ee61d40 0x55cd2ee64180 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f187f75265ab99 server_cookie=4886fc598a280f5d in_seq=0 out_seq=0 2024-09-10T08:19:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.820+0000 7f6dfb400640 10 client.18660 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6dfb400640 1 -- 172.21.15.2:0/4249415094 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd2edbe830 con 0x55cd2ee61d40 2024-09-10T08:19:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6dfb400640 10 client.18660 handle_command_reply: tid=0 2024-09-10T08:19:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6e1f239740 1 client.18660 shutdown 2024-09-10T08:19:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6e1f239740 1 -- 172.21.15.2:0/4249415094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c9d0 msgr2=0x55cd2ee56760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6e1f239740 1 --2- 172.21.15.2:0/4249415094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee56760 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009a20 tx=0x7f6e000f32a0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6e1f239740 1 -- 172.21.15.2:0/4249415094 shutdown_connections 2024-09-10T08:19:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6e1f239740 1 --2- 172.21.15.2:0/4249415094 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd2ee61d40 0x55cd2ee64180 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6e1f239740 1 --2- 172.21.15.2:0/4249415094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee51a60 0x55cd2ee5d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6e1f239740 1 --2- 172.21.15.2:0/4249415094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee56760 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6e1f239740 1 --2- 172.21.15.2:0/4249415094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c090 0x55cd2ee56220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6dfb400640 0 client.18660 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6e1f239740 1 -- 172.21.15.2:0/4249415094 >> 172.21.15.2:0/4249415094 conn(0x55cd2ee47700 msgr2=0x55cd2ee48f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6e1f239740 1 -- 172.21.15.2:0/4249415094 shutdown_connections 2024-09-10T08:19:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.821+0000 7f6e1f239740 1 -- 172.21.15.2:0/4249415094 wait complete. 2024-09-10T08:19:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.822+0000 7f6e1f239740 20 client.18660 trim_cache size 0 max 16384 2024-09-10T08:19:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.837+0000 7f6e1f239740 1 Processor -- start 2024-09-10T08:19:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.837+0000 7f6e1f239740 1 -- start start 2024-09-10T08:19:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.837+0000 7f6e1f239740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 0x55cd2ee46770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.837+0000 7f6e1f239740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee5d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.837+0000 7f6e1f239740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee51a60 0x55cd2ee68a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.837+0000 7f6e1f239740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141b36f0 con 0x55cd2ee4c090 2024-09-10T08:19:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.837+0000 7f6e1f239740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141b0fc0 con 0x55cd2ee4c9d0 2024-09-10T08:19:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.837+0000 7f6e1f239740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141aee10 con 0x55cd2ee51a60 2024-09-10T08:19:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.838+0000 7f6e19a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 0x55cd2ee46770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.838+0000 7f6e19a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 0x55cd2ee46770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40538/0 (socket says 172.21.15.2:40538) 2024-09-10T08:19:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.838+0000 7f6e19a00640 1 -- 172.21.15.2:0/4097887527 learned_addr learned my addr 172.21.15.2:0/4097887527 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.838+0000 7f6e19000640 1 --2- 172.21.15.2:0/4097887527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee5d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.838+0000 7f6e1a400640 1 --2- 172.21.15.2:0/4097887527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee51a60 0x55cd2ee68a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.838+0000 7f6e19a00640 1 -- 172.21.15.2:0/4097887527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee51a60 msgr2=0x55cd2ee68a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.838+0000 7f6e19a00640 1 --2- 172.21.15.2:0/4097887527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee51a60 0x55cd2ee68a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.838+0000 7f6e19a00640 1 -- 172.21.15.2:0/4097887527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee4c9d0 msgr2=0x55cd2ee5d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.838+0000 7f6e19a00640 1 --2- 172.21.15.2:0/4097887527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee5d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.838+0000 7f6e19a00640 1 -- 172.21.15.2:0/4097887527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd2ee64800 con 0x55cd2ee4c090 2024-09-10T08:19:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.838+0000 7f6e19000640 1 --2- 172.21.15.2:0/4097887527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee5d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.838+0000 7f6e19a00640 1 --2- 172.21.15.2:0/4097887527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 0x55cd2ee46770 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6e08012520 tx=0x7f6e0801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3766740d7139599 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.839+0000 7f6e06000640 1 -- 172.21.15.2:0/4097887527 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800fe60 con 0x55cd2ee4c090 2024-09-10T08:19:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.839+0000 7f6e06000640 1 -- 172.21.15.2:0/4097887527 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e08005500 con 0x55cd2ee4c090 2024-09-10T08:19:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.839+0000 7f6e06000640 1 -- 172.21.15.2:0/4097887527 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0802b680 con 0x55cd2ee4c090 2024-09-10T08:19:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.839+0000 7f6e1f239740 1 -- 172.21.15.2:0/4097887527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 msgr2=0x55cd2ee46770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.839+0000 7f6e1f239740 1 --2- 172.21.15.2:0/4097887527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 0x55cd2ee46770 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6e08012520 tx=0x7f6e0801a000 comp rx=0 tx=0).stop 2024-09-10T08:19:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.839+0000 7f6e1f239740 1 -- 172.21.15.2:0/4097887527 shutdown_connections 2024-09-10T08:19:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.839+0000 7f6e1f239740 1 --2- 172.21.15.2:0/4097887527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee51a60 0x55cd2ee68a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.839+0000 7f6e1f239740 1 --2- 172.21.15.2:0/4097887527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee5d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.839+0000 7f6e1f239740 1 --2- 172.21.15.2:0/4097887527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 0x55cd2ee46770 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.839+0000 7f6e1f239740 1 -- 172.21.15.2:0/4097887527 >> 172.21.15.2:0/4097887527 conn(0x55cd2ee47700 msgr2=0x55cd2ee46060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.839+0000 7f6e1f239740 1 -- 172.21.15.2:0/4097887527 shutdown_connections 2024-09-10T08:19:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.839+0000 7f6e1f239740 1 -- 172.21.15.2:0/4097887527 wait complete. 2024-09-10T08:19:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.840+0000 7f6e1f239740 1 -- start start 2024-09-10T08:19:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.840+0000 7f6e1f239740 1 Processor -- start 2024-09-10T08:19:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.840+0000 7f6e1f239740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 0x55cd2ee5ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.840+0000 7f6e1f239740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee5b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.840+0000 7f6e1f239740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee51a60 0x55cd2ee5c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.840+0000 7f6e1f239740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141af480 con 0x55cd2ee4c090 2024-09-10T08:19:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.840+0000 7f6e1f239740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141b08f0 con 0x55cd2ee51a60 2024-09-10T08:19:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.840+0000 7f6e1f239740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141afb50 con 0x55cd2ee4c9d0 2024-09-10T08:19:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.840+0000 7f6e1f239740 10 client.0 _ll_register_callbacks cb 0x7f6e000f33a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6e19a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 0x55cd2ee5ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6e19a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 0x55cd2ee5ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40552/0 (socket says 172.21.15.2:40552) 2024-09-10T08:19:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6e19a00640 1 -- 172.21.15.2:0/2293851354 learned_addr learned my addr 172.21.15.2:0/2293851354 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6e1a400640 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee51a60 0x55cd2ee5c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6e19000640 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee5b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6e19a00640 1 -- 172.21.15.2:0/2293851354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c9d0 msgr2=0x55cd2ee5b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6e19a00640 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee5b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6e19a00640 1 -- 172.21.15.2:0/2293851354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee51a60 msgr2=0x55cd2ee5c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6e19a00640 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee51a60 0x55cd2ee5c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6e19a00640 1 -- 172.21.15.2:0/2293851354 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd2ee5c820 con 0x55cd2ee4c090 2024-09-10T08:19:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6e19000640 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee5b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6e1a400640 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee51a60 0x55cd2ee5c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6e19a00640 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 0x55cd2ee5ad80 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800f790 tx=0x7f6e0800fde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19247e6df13f313d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.841+0000 7f6dfb400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6dfb400640 1 -- 172.21.15.2:0/2293851354 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08004980 con 0x55cd2ee4c090 2024-09-10T08:19:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6dfb400640 1 -- 172.21.15.2:0/2293851354 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6e08004e70 con 0x55cd2ee4c090 2024-09-10T08:19:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6e1f239740 1 -- 172.21.15.2:0/2293851354 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e08005500 con 0x55cd2ee4c090 2024-09-10T08:19:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6dfb400640 1 -- 172.21.15.2:0/2293851354 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6e08049050 con 0x55cd2ee4c090 2024-09-10T08:19:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6e1f239740 10 client.18666 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6e1f239740 1 -- 172.21.15.2:0/2293851354 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cd2ee5db30 con 0x55cd2ee4c090 2024-09-10T08:19:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6dfb400640 1 -- 172.21.15.2:0/2293851354 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e08005500 con 0x55cd2ee4c090 2024-09-10T08:19:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6dfb400640 1 -- 172.21.15.2:0/2293851354 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6e08040470 con 0x55cd2ee4c090 2024-09-10T08:19:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6e1f239740 10 client.18666 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6e1f239740 10 client.18666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6e1f239740 20 client.18666 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6e1f239740 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd2eecd160 0x55cd2eecf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6e1f239740 4 client.18666 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:19:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.842+0000 7f6e1f239740 1 -- 172.21.15.2:0/2293851354 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd2edb8b80 con 0x55cd2eecd160 2024-09-10T08:19:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.843+0000 7f6e19000640 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd2eecd160 0x55cd2eecf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.843+0000 7f6e19000640 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd2eecd160 0x55cd2eecf580 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5a6f2bd907f50b server_cookie=582f33bea5211344 in_seq=0 out_seq=0 2024-09-10T08:19:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.843+0000 7f6dfb400640 10 client.18666 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6dfb400640 1 -- 172.21.15.2:0/2293851354 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd2edb8b80 con 0x55cd2eecd160 2024-09-10T08:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6dfb400640 10 client.18666 handle_command_reply: tid=0 2024-09-10T08:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6e1f239740 1 client.18666 shutdown 2024-09-10T08:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6e1f239740 1 -- 172.21.15.2:0/2293851354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 msgr2=0x55cd2ee5ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6e1f239740 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 0x55cd2ee5ad80 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800f790 tx=0x7f6e0800fde0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6e1f239740 1 -- 172.21.15.2:0/2293851354 shutdown_connections 2024-09-10T08:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6e1f239740 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cd2eecd160 0x55cd2eecf580 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6e1f239740 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd2ee51a60 0x55cd2ee5c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6e1f239740 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd2ee4c9d0 0x55cd2ee5b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6dfb400640 0 client.18666 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6e1f239740 1 --2- 172.21.15.2:0/2293851354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2ee4c090 0x55cd2ee5ad80 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6e1f239740 1 -- 172.21.15.2:0/2293851354 >> 172.21.15.2:0/2293851354 conn(0x55cd2ee47700 msgr2=0x55cd2ee46880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6e1f239740 1 -- 172.21.15.2:0/2293851354 shutdown_connections 2024-09-10T08:19:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6e1f239740 1 -- 172.21.15.2:0/2293851354 wait complete. 2024-09-10T08:19:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.855+0000 7f6e1f239740 20 client.18666 trim_cache size 0 max 16384 2024-09-10T08:19:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:19:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:19:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:19:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:19:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.856+0000 7f6e1c400640 1 -- 172.21.15.2:0/701877602 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6dfc0a2c30 msgr2=0x7f6dfc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.856+0000 7f6e1c400640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6dfc0a2c30 0x7f6dfc0a50f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55cd2ee4d400 tx=0x7f6e08005f90 comp rx=0 tx=0).stop 2024-09-10T08:19:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.856+0000 7f6e1c400640 1 -- 172.21.15.2:0/701877602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e14077df0 msgr2=0x7f6e1419e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.856+0000 7f6e1c400640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e14077df0 0x7f6e1419e960 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009230 tx=0x7f6e000042f0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.856+0000 7f6e1c400640 1 -- 172.21.15.2:0/701877602 shutdown_connections 2024-09-10T08:19:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.856+0000 7f6e1c400640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e14078790 0x7f6e141a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.856+0000 7f6e1c400640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e14077df0 0x7f6e1419e960 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.856+0000 7f6e1c400640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6dfc0a2c30 0x7f6dfc0a50f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.856+0000 7f6e1c400640 1 --2- 172.21.15.2:0/701877602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e14076bb0 0x7f6e1419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.856+0000 7f6e1c400640 1 -- 172.21.15.2:0/701877602 >> 172.21.15.2:0/701877602 conn(0x7f6e140ff4e0 msgr2=0x7f6e14100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.856+0000 7f6e1c400640 1 -- 172.21.15.2:0/701877602 shutdown_connections 2024-09-10T08:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:28.856+0000 7f6e1c400640 1 -- 172.21.15.2:0/701877602 wait complete. 2024-09-10T08:19:29.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:29 smithi002 ceph-mon[30563]: pgmap v727: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-10T08:19:29.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:29 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:19:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:29 smithi155 ceph-mon[35405]: pgmap v727: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-10T08:19:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:29 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:19:29.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:29 smithi107 ceph-mon[35048]: pgmap v727: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-10T08:19:29.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:29 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:19:31.105 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:19:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:31.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:31 smithi002 ceph-mon[30563]: pgmap v728: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-10T08:19:31.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:31 smithi155 ceph-mon[35405]: pgmap v728: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-10T08:19:31.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:31 smithi107 ceph-mon[35048]: pgmap v728: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-10T08:19:32.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.110+0000 7f3f05400640 1 -- 172.21.15.2:0/1463557215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f00109140 msgr2=0x7f3f00110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:32.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.110+0000 7f3f05400640 1 --2- 172.21.15.2:0/1463557215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f00109140 0x7f3f00110cc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009ca0 tx=0x7f3ef802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:19:32.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.110+0000 7f3f05400640 1 -- 172.21.15.2:0/1463557215 shutdown_connections 2024-09-10T08:19:32.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.110+0000 7f3f05400640 1 --2- 172.21.15.2:0/1463557215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f00109140 0x7f3f00110cc0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:32.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.110+0000 7f3f05400640 1 --2- 172.21.15.2:0/1463557215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f001049c0 0x7f3f00108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:32.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.110+0000 7f3f05400640 1 --2- 172.21.15.2:0/1463557215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f00104010 0x7f3f001043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:32.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.110+0000 7f3f05400640 1 -- 172.21.15.2:0/1463557215 >> 172.21.15.2:0/1463557215 conn(0x7f3f000ffa50 msgr2=0x7f3f00101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:32.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.110+0000 7f3f05400640 1 -- 172.21.15.2:0/1463557215 shutdown_connections 2024-09-10T08:19:32.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.110+0000 7f3f05400640 1 -- 172.21.15.2:0/1463557215 wait complete. 2024-09-10T08:19:32.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.111+0000 7f3f05400640 1 Processor -- start 2024-09-10T08:19:32.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.111+0000 7f3f05400640 1 -- start start 2024-09-10T08:19:32.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3f05400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f00104010 0x7f3f0019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:32.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3f05400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f001049c0 0x7f3f0019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3f05400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f00109140 0x7f3f0019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3f05400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3f001141a0 con 0x7f3f00109140 2024-09-10T08:19:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3f05400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00114020 con 0x7f3f00104010 2024-09-10T08:19:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3f05400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00114320 con 0x7f3f001049c0 2024-09-10T08:19:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3efe800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f00104010 0x7f3f0019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3eff200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f00109140 0x7f3f0019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3efe800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f00104010 0x7f3f0019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52260/0 (socket says 172.21.15.2:52260) 2024-09-10T08:19:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3efe800640 1 -- 172.21.15.2:0/3129546362 learned_addr learned my addr 172.21.15.2:0/3129546362 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3efde00640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f001049c0 0x7f3f0019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3eff200640 1 -- 172.21.15.2:0/3129546362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f001049c0 msgr2=0x7f3f0019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3eff200640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f001049c0 0x7f3f0019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.112+0000 7f3eff200640 1 -- 172.21.15.2:0/3129546362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f00104010 msgr2=0x7f3f0019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.113+0000 7f3eff200640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f00104010 0x7f3f0019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.113+0000 7f3eff200640 1 -- 172.21.15.2:0/3129546362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0019f650 con 0x7f3f00109140 2024-09-10T08:19:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.113+0000 7f3efe800640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f00104010 0x7f3f0019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.113+0000 7f3eff200640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f00109140 0x7f3f0019ef70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8005e30 tx=0x7f3ef8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:32.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.113+0000 7f3ef7400640 1 -- 172.21.15.2:0/3129546362 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8031650 con 0x7f3f00109140 2024-09-10T08:19:32.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.113+0000 7f3ef7400640 1 -- 172.21.15.2:0/3129546362 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef80317f0 con 0x7f3f00109140 2024-09-10T08:19:32.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.113+0000 7f3ef7400640 1 -- 172.21.15.2:0/3129546362 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef802fb70 con 0x7f3f00109140 2024-09-10T08:19:32.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.113+0000 7f3f05400640 1 -- 172.21.15.2:0/3129546362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0019f8e0 con 0x7f3f00109140 2024-09-10T08:19:32.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.113+0000 7f3f05400640 1 -- 172.21.15.2:0/3129546362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f001a7220 con 0x7f3f00109140 2024-09-10T08:19:32.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.115+0000 7f3f05400640 1 -- 172.21.15.2:0/3129546362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f001054f0 con 0x7f3f00109140 2024-09-10T08:19:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.116+0000 7f3ef7400640 1 -- 172.21.15.2:0/3129546362 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3ef802fd10 con 0x7f3f00109140 2024-09-10T08:19:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.118+0000 7f3ef7400640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3ee40a2c00 0x7f3ee40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.118+0000 7f3ef7400640 1 -- 172.21.15.2:0/3129546362 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3ef80f0780 con 0x7f3f00109140 2024-09-10T08:19:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.118+0000 7f3efe800640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3ee40a2c00 0x7f3ee40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.119+0000 7f3efe800640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3ee40a2c00 0x7f3ee40a50c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0005fd0 tx=0x7f3ef000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:32.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.122+0000 7f3ef7400640 1 -- 172.21.15.2:0/3129546362 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ef80bd3b0 con 0x7f3f00109140 2024-09-10T08:19:32.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.344+0000 7f3f05400640 1 -- 172.21.15.2:0/3129546362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f0010c380 con 0x7f3f00109140 2024-09-10T08:19:32.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.344+0000 7f3ef7400640 1 -- 172.21.15.2:0/3129546362 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f3ef80bccb0 con 0x7f3f00109140 2024-09-10T08:19:32.345 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:19:32.345 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:19:32.345 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:19:32.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.347+0000 7f3f05400640 1 -- 172.21.15.2:0/3129546362 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3ee40a2c00 msgr2=0x7f3ee40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:32.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.347+0000 7f3f05400640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3ee40a2c00 0x7f3ee40a50c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0005fd0 tx=0x7f3ef000b4a0 comp rx=0 tx=0).stop 2024-09-10T08:19:32.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.347+0000 7f3f05400640 1 -- 172.21.15.2:0/3129546362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f00109140 msgr2=0x7f3f0019ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:32.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.347+0000 7f3f05400640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f00109140 0x7f3f0019ef70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8005e30 tx=0x7f3ef8003f10 comp rx=0 tx=0).stop 2024-09-10T08:19:32.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.347+0000 7f3f05400640 1 -- 172.21.15.2:0/3129546362 shutdown_connections 2024-09-10T08:19:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.347+0000 7f3f05400640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f00109140 0x7f3f0019ef70 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.347+0000 7f3f05400640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3ee40a2c00 0x7f3ee40a50c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.347+0000 7f3f05400640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f001049c0 0x7f3f0019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.347+0000 7f3f05400640 1 --2- 172.21.15.2:0/3129546362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f00104010 0x7f3f0019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.347+0000 7f3f05400640 1 -- 172.21.15.2:0/3129546362 >> 172.21.15.2:0/3129546362 conn(0x7f3f000ffa50 msgr2=0x7f3f00100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.347+0000 7f3f05400640 1 -- 172.21.15.2:0/3129546362 shutdown_connections 2024-09-10T08:19:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:32.347+0000 7f3f05400640 1 -- 172.21.15.2:0/3129546362 wait complete. 2024-09-10T08:19:32.720 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:32 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3129546362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:19:32.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:32 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3129546362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:19:32.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:32 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3129546362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:19:33.286 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id dd7972fa --release --await' 2024-09-10T08:19:33.287 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id dd7972fa --release --await 2024-09-10T08:19:33.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:33 smithi002 ceph-mon[30563]: pgmap v729: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-10T08:19:33.676 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:33.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:33 smithi155 ceph-mon[35405]: pgmap v729: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-10T08:19:33.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:33 smithi107 ceph-mon[35048]: pgmap v729: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-10T08:19:34.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 -- 172.21.15.2:0/3651171572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ab81079c0 msgr2=0x7f2ab8107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 --2- 172.21.15.2:0/3651171572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ab81079c0 0x7f2ab8107da0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009930 tx=0x7f2aac02a440 comp rx=0 tx=0).stop 2024-09-10T08:19:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 -- 172.21.15.2:0/3651171572 shutdown_connections 2024-09-10T08:19:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 --2- 172.21.15.2:0/3651171572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ab8102360 0x7f2ab810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 --2- 172.21.15.2:0/3651171572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ab81019c0 0x7f2ab8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 --2- 172.21.15.2:0/3651171572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ab81079c0 0x7f2ab8107da0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 -- 172.21.15.2:0/3651171572 >> 172.21.15.2:0/3651171572 conn(0x7f2ab80fd740 msgr2=0x7f2ab80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 -- 172.21.15.2:0/3651171572 shutdown_connections 2024-09-10T08:19:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 -- 172.21.15.2:0/3651171572 wait complete. 2024-09-10T08:19:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 Processor -- start 2024-09-10T08:19:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 -- start start 2024-09-10T08:19:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ab81019c0 0x7f2ab819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ab8102360 0x7f2ab819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ab81079c0 0x7f2ab819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8111d90 con 0x7f2ab8102360 2024-09-10T08:19:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8111c10 con 0x7f2ab81079c0 2024-09-10T08:19:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.289+0000 7f2abec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8111f10 con 0x7f2ab81019c0 2024-09-10T08:19:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.290+0000 7f2abce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ab8102360 0x7f2ab819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.290+0000 7f2abce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ab8102360 0x7f2ab819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40588/0 (socket says 172.21.15.2:40588) 2024-09-10T08:19:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.290+0000 7f2abce00640 1 -- 172.21.15.2:0/2159430923 learned_addr learned my addr 172.21.15.2:0/2159430923 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.290+0000 7f2abe200640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ab81079c0 0x7f2ab819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.290+0000 7f2abd800640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ab81019c0 0x7f2ab819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.290+0000 7f2abce00640 1 -- 172.21.15.2:0/2159430923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ab81019c0 msgr2=0x7f2ab819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.290+0000 7f2abce00640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ab81019c0 0x7f2ab819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.290+0000 7f2abce00640 1 -- 172.21.15.2:0/2159430923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ab81079c0 msgr2=0x7f2ab819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.290+0000 7f2abce00640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ab81079c0 0x7f2ab819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.290+0000 7f2abce00640 1 -- 172.21.15.2:0/2159430923 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab819f460 con 0x7f2ab8102360 2024-09-10T08:19:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.290+0000 7f2abe200640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ab81079c0 0x7f2ab819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.290+0000 7f2abce00640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ab8102360 0x7f2ab819a9f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400d950 tx=0x7f2aa400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.291+0000 7f2ab6000640 1 -- 172.21.15.2:0/2159430923 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4017030 con 0x7f2ab8102360 2024-09-10T08:19:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.291+0000 7f2ab6000640 1 -- 172.21.15.2:0/2159430923 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aa4004020 con 0x7f2ab8102360 2024-09-10T08:19:34.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.291+0000 7f2ab6000640 1 -- 172.21.15.2:0/2159430923 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa40047a0 con 0x7f2ab8102360 2024-09-10T08:19:34.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.291+0000 7f2abec00640 1 -- 172.21.15.2:0/2159430923 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab819f750 con 0x7f2ab8102360 2024-09-10T08:19:34.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.291+0000 7f2abec00640 1 -- 172.21.15.2:0/2159430923 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab81a7030 con 0x7f2ab8102360 2024-09-10T08:19:34.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.296+0000 7f2ac2bd2740 1 Processor -- start 2024-09-10T08:19:34.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.296+0000 7f2ac2bd2740 1 -- start start 2024-09-10T08:19:34.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.296+0000 7f2ac2bd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72520 0x55b1b6c72900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.296+0000 7f2ac2bd2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72e40 0x55b1b6c7d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.296+0000 7f2ac2bd2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 0x55b1b6c7faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.296+0000 7f2ac2bd2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b1b6bd8650 con 0x55b1b6c72e40 2024-09-10T08:19:34.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.296+0000 7f2ac2bd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b1b6b7ff10 con 0x55b1b6c72520 2024-09-10T08:19:34.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.296+0000 7f2ac2bd2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b1b6bd4610 con 0x55b1b6c7d700 2024-09-10T08:19:34.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.296+0000 7f2ab6000640 1 -- 172.21.15.2:0/2159430923 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2aa40041d0 con 0x7f2ab8102360 2024-09-10T08:19:34.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.296+0000 7f2abd800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72e40 0x55b1b6c7d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.296+0000 7f2abe200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72520 0x55b1b6c72900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.296+0000 7f2abd800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72e40 0x55b1b6c7d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40590/0 (socket says 172.21.15.2:40590) 2024-09-10T08:19:34.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.296+0000 7f2abd800640 1 -- 172.21.15.2:0/1058075070 learned_addr learned my addr 172.21.15.2:0/1058075070 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:34.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.297+0000 7f2abce00640 1 --2- 172.21.15.2:0/1058075070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 0x55b1b6c7faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.297+0000 7f2abe200640 1 -- 172.21.15.2:0/1058075070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 msgr2=0x55b1b6c7faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:34.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.297+0000 7f2abe200640 1 --2- 172.21.15.2:0/1058075070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 0x55b1b6c7faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.297+0000 7f2abe200640 1 -- 172.21.15.2:0/1058075070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72e40 msgr2=0x55b1b6c7d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:34.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.297+0000 7f2abe200640 1 --2- 172.21.15.2:0/1058075070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72e40 0x55b1b6c7d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.297+0000 7f2abe200640 1 -- 172.21.15.2:0/1058075070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1b68e7580 con 0x55b1b6c72520 2024-09-10T08:19:34.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.297+0000 7f2abe200640 1 --2- 172.21.15.2:0/1058075070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72520 0x55b1b6c72900 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0010710 tx=0x7f2ab0010b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca69355bd8bae0b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:34.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.297+0000 7f2aaaa00640 1 -- 172.21.15.2:0/1058075070 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0004b20 con 0x55b1b6c72520 2024-09-10T08:19:34.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.297+0000 7f2aaaa00640 1 -- 172.21.15.2:0/1058075070 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ab000edb0 con 0x55b1b6c72520 2024-09-10T08:19:34.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.298+0000 7f2aaaa00640 1 -- 172.21.15.2:0/1058075070 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0005620 con 0x55b1b6c72520 2024-09-10T08:19:34.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.298+0000 7f2ab6000640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2aa00a2b80 0x7f2aa00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.298+0000 7f2ab6000640 1 -- 172.21.15.2:0/2159430923 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2aa40ced70 con 0x7f2ab8102360 2024-09-10T08:19:34.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.298+0000 7f2abd800640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2aa00a2b80 0x7f2aa00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.298+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/1058075070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72520 msgr2=0x55b1b6c72900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:34.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.298+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/1058075070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72520 0x55b1b6c72900 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0010710 tx=0x7f2ab0010b10 comp rx=0 tx=0).stop 2024-09-10T08:19:34.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.299+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/1058075070 shutdown_connections 2024-09-10T08:19:34.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.299+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/1058075070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 0x55b1b6c7faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.299+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/1058075070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72e40 0x55b1b6c7d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.299+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/1058075070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72520 0x55b1b6c72900 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.299+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/1058075070 >> 172.21.15.2:0/1058075070 conn(0x55b1b6c6eab0 msgr2=0x55b1b6c6ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:34.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.299+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/1058075070 shutdown_connections 2024-09-10T08:19:34.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.299+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/1058075070 wait complete. 2024-09-10T08:19:34.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.299+0000 7f2ac2bd2740 1 -- start start 2024-09-10T08:19:34.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.299+0000 7f2abd800640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2aa00a2b80 0x7f2aa00a5040 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2aac006010 tx=0x7f2aac0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:34.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.299+0000 7f2ac2bd2740 1 Processor -- start 2024-09-10T08:19:34.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.300+0000 7f2ac2bd2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c7a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.300+0000 7f2ac2bd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72e40 0x55b1b6c7aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.300+0000 7f2ac2bd2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 0x55b1b6c7afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.300+0000 7f2ac2bd2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b1b6c802d0 con 0x55b1b6c72520 2024-09-10T08:19:34.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.300+0000 7f2ac2bd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b1b6b93350 con 0x55b1b6c72e40 2024-09-10T08:19:34.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.300+0000 7f2ac2bd2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b1b6c80450 con 0x55b1b6c7d700 2024-09-10T08:19:34.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.300+0000 7f2ac2bd2740 10 client.0 _ll_register_callbacks cb 0x55b1b6bb6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:34.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.300+0000 7f2abd800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c7a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.300+0000 7f2abd800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c7a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40592/0 (socket says 172.21.15.2:40592) 2024-09-10T08:19:34.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.300+0000 7f2abd800640 1 -- 172.21.15.2:0/1326777932 learned_addr learned my addr 172.21.15.2:0/1326777932 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:34.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.300+0000 7f2abce00640 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72e40 0x55b1b6c7aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.300+0000 7f2abe200640 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 0x55b1b6c7afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.301+0000 7f2abd800640 1 -- 172.21.15.2:0/1326777932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 msgr2=0x55b1b6c7afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:34.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.301+0000 7f2abd800640 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 0x55b1b6c7afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.301+0000 7f2abd800640 1 -- 172.21.15.2:0/1326777932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72e40 msgr2=0x55b1b6c7aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:34.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.301+0000 7f2abd800640 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72e40 0x55b1b6c7aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.301+0000 7f2abd800640 1 -- 172.21.15.2:0/1326777932 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1b69b1960 con 0x55b1b6c72520 2024-09-10T08:19:34.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.301+0000 7f2abe200640 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 0x55b1b6c7afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:19:34.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.301+0000 7f2abd800640 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c7a550 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009900 tx=0x7f2aac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf6077d70f62fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:34.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.301+0000 7f2a9f400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:34.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.301+0000 7f2a9f400640 1 -- 172.21.15.2:0/1326777932 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac04e070 con 0x55b1b6c72520 2024-09-10T08:19:34.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.301+0000 7f2a9f400640 1 -- 172.21.15.2:0/1326777932 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2aac003cc0 con 0x55b1b6c72520 2024-09-10T08:19:34.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.301+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/1326777932 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ab000edb0 con 0x55b1b6c72520 2024-09-10T08:19:34.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.302+0000 7f2a9f400640 1 -- 172.21.15.2:0/1326777932 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2aac05a050 con 0x55b1b6c72520 2024-09-10T08:19:34.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.302+0000 7f2ac2bd2740 10 client.18684 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:34.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.302+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/1326777932 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b1b6c8bec0 con 0x55b1b6c72520 2024-09-10T08:19:34.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.302+0000 7f2a9f400640 1 -- 172.21.15.2:0/1326777932 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ab000edb0 con 0x55b1b6c72520 2024-09-10T08:19:34.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.302+0000 7f2a9f400640 1 -- 172.21.15.2:0/1326777932 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2aac03d4e0 con 0x55b1b6c72520 2024-09-10T08:19:34.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.302+0000 7f2ac2bd2740 10 client.18684 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:34.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.302+0000 7f2ac2bd2740 10 client.18684 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:19:34.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.302+0000 7f2ac2bd2740 20 client.18684 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:34.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.302+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b1b6c8d2d0 0x55b1b6c8f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.302+0000 7f2ac2bd2740 4 client.18684 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:34.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.302+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/1326777932 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1b6be5bc0 con 0x55b1b6c8d2d0 2024-09-10T08:19:34.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.303+0000 7f2abce00640 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b1b6c8d2d0 0x55b1b6c8f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.303+0000 7f2abce00640 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b1b6c8d2d0 0x55b1b6c8f760 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4051d862799e72c server_cookie=58617bf86e9f6096 in_seq=0 out_seq=0 2024-09-10T08:19:34.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.303+0000 7f2a9f400640 10 client.18684 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:34.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.304+0000 7f2a9f400640 1 -- 172.21.15.2:0/1326777932 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1b6be5bc0 con 0x55b1b6c8d2d0 2024-09-10T08:19:34.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.304+0000 7f2a9f400640 10 client.18684 handle_command_reply: tid=0 2024-09-10T08:19:34.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.304+0000 7f2ac2bd2740 1 client.18684 shutdown 2024-09-10T08:19:34.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.304+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/1326777932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 msgr2=0x55b1b6c7a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:34.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.304+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c7a550 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009900 tx=0x7f2aac009260 comp rx=0 tx=0).stop 2024-09-10T08:19:34.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.304+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/1326777932 shutdown_connections 2024-09-10T08:19:34.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.304+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b1b6c8d2d0 0x55b1b6c8f760 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.304+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 0x55b1b6c7afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.304+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72e40 0x55b1b6c7aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.304+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/1326777932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c7a550 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.304+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/1326777932 >> 172.21.15.2:0/1326777932 conn(0x55b1b6c6eab0 msgr2=0x55b1b6c6dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:34.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.304+0000 7f2a9f400640 0 client.18684 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:34.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.305+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/1326777932 shutdown_connections 2024-09-10T08:19:34.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.305+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/1326777932 wait complete. 2024-09-10T08:19:34.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.305+0000 7f2ac2bd2740 20 client.18684 trim_cache size 0 max 16384 2024-09-10T08:19:34.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.323+0000 7f2ac2bd2740 1 Processor -- start 2024-09-10T08:19:34.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.323+0000 7f2ac2bd2740 1 -- start start 2024-09-10T08:19:34.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.323+0000 7f2ac2bd2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c95080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.323+0000 7f2ac2bd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72e40 0x55b1b6c955c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.323+0000 7f2ac2bd2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 0x55b1b6c777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2ac2bd2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81b37c0 con 0x55b1b6c72520 2024-09-10T08:19:34.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2ac2bd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81b1090 con 0x55b1b6c72e40 2024-09-10T08:19:34.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2ac2bd2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81aeee0 con 0x55b1b6c7d700 2024-09-10T08:19:34.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2abd800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c95080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2abce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72e40 0x55b1b6c955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2abce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72e40 0x55b1b6c955c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52298/0 (socket says 172.21.15.2:52298) 2024-09-10T08:19:34.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2abce00640 1 -- 172.21.15.2:0/2601988643 learned_addr learned my addr 172.21.15.2:0/2601988643 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:34.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2abe200640 1 --2- 172.21.15.2:0/2601988643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 0x55b1b6c777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2abce00640 1 -- 172.21.15.2:0/2601988643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 msgr2=0x55b1b6c777b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:34.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2abce00640 1 --2- 172.21.15.2:0/2601988643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 0x55b1b6c777b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2abce00640 1 -- 172.21.15.2:0/2601988643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 msgr2=0x55b1b6c95080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:34.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2abce00640 1 --2- 172.21.15.2:0/2601988643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c95080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2abce00640 1 -- 172.21.15.2:0/2601988643 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1b6c6fb60 con 0x55b1b6c72e40 2024-09-10T08:19:34.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2abce00640 1 --2- 172.21.15.2:0/2601988643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72e40 0x55b1b6c955c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4097bb0 tx=0x7f2aa40cc350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27db0fc51e650a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2aaa000640 1 -- 172.21.15.2:0/2601988643 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa409ad20 con 0x55b1b6c72e40 2024-09-10T08:19:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2aaa000640 1 -- 172.21.15.2:0/2601988643 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aa409b8e0 con 0x55b1b6c72e40 2024-09-10T08:19:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.325+0000 7f2aaa000640 1 -- 172.21.15.2:0/2601988643 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa409ab40 con 0x55b1b6c72e40 2024-09-10T08:19:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.326+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/2601988643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72e40 msgr2=0x55b1b6c955c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.326+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/2601988643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72e40 0x55b1b6c955c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4097bb0 tx=0x7f2aa40cc350 comp rx=0 tx=0).stop 2024-09-10T08:19:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.326+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/2601988643 shutdown_connections 2024-09-10T08:19:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.326+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/2601988643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c7d700 0x55b1b6c777b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.326+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/2601988643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c72e40 0x55b1b6c955c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.326+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/2601988643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c95080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.326+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/2601988643 >> 172.21.15.2:0/2601988643 conn(0x55b1b6c6eab0 msgr2=0x55b1b6c6dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.326+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/2601988643 shutdown_connections 2024-09-10T08:19:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.326+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/2601988643 wait complete. 2024-09-10T08:19:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.326+0000 7f2ac2bd2740 1 -- start start 2024-09-10T08:19:34.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.327+0000 7f2ac2bd2740 1 Processor -- start 2024-09-10T08:19:34.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.327+0000 7f2ac2bd2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c81d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.327+0000 7f2ac2bd2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c72e40 0x55b1b6c82270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.327+0000 7f2ac2bd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c7d700 0x55b1b6cf1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.327+0000 7f2ac2bd2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81af550 con 0x55b1b6c72520 2024-09-10T08:19:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.327+0000 7f2ac2bd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81b09c0 con 0x55b1b6c7d700 2024-09-10T08:19:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.327+0000 7f2ac2bd2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81afc20 con 0x55b1b6c72e40 2024-09-10T08:19:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.327+0000 7f2ac2bd2740 10 client.0 _ll_register_callbacks cb 0x7f2aac009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.327+0000 7f2abd800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c81d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.327+0000 7f2abce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c72e40 0x55b1b6c82270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.327+0000 7f2abd800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c81d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40614/0 (socket says 172.21.15.2:40614) 2024-09-10T08:19:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.327+0000 7f2abd800640 1 -- 172.21.15.2:0/780304891 learned_addr learned my addr 172.21.15.2:0/780304891 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.327+0000 7f2abe200640 1 --2- 172.21.15.2:0/780304891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c7d700 0x55b1b6cf1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.328+0000 7f2abce00640 1 -- 172.21.15.2:0/780304891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c7d700 msgr2=0x55b1b6cf1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.328+0000 7f2abce00640 1 --2- 172.21.15.2:0/780304891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c7d700 0x55b1b6cf1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.328+0000 7f2abce00640 1 -- 172.21.15.2:0/780304891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 msgr2=0x55b1b6c81d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.328+0000 7f2abce00640 1 --2- 172.21.15.2:0/780304891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c81d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.328+0000 7f2abce00640 1 -- 172.21.15.2:0/780304891 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1b6cf1660 con 0x55b1b6c72e40 2024-09-10T08:19:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.328+0000 7f2abd800640 1 --2- 172.21.15.2:0/780304891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c81d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.328+0000 7f2abce00640 1 --2- 172.21.15.2:0/780304891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c72e40 0x55b1b6c82270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55b1b6c7d2f0 tx=0x7f2aa40d2490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a141c10942e53bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.328+0000 7f2a9f400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:19:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.328+0000 7f2a9f400640 1 -- 172.21.15.2:0/780304891 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa40b3070 con 0x55b1b6c72e40 2024-09-10T08:19:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.328+0000 7f2a9f400640 1 -- 172.21.15.2:0/780304891 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2aa409f120 con 0x55b1b6c72e40 2024-09-10T08:19:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.328+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/780304891 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aa409b8e0 con 0x55b1b6c72e40 2024-09-10T08:19:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.329+0000 7f2a9f400640 1 -- 172.21.15.2:0/780304891 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2aa409ab40 con 0x55b1b6c72e40 2024-09-10T08:19:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.329+0000 7f2ac2bd2740 10 client.28648 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.329+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/780304891 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b1b6cf8a10 con 0x55b1b6c72e40 2024-09-10T08:19:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.329+0000 7f2a9f400640 1 -- 172.21.15.2:0/780304891 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aa409b8e0 con 0x55b1b6c72e40 2024-09-10T08:19:34.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.329+0000 7f2a9f400640 1 -- 172.21.15.2:0/780304891 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2aa40a39c0 con 0x55b1b6c72e40 2024-09-10T08:19:34.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.329+0000 7f2ac2bd2740 10 client.28648 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:34.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.329+0000 7f2ac2bd2740 10 client.28648 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:19:34.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.329+0000 7f2ac2bd2740 20 client.28648 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:34.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.329+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/780304891 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b1b6cf9160 0x55b1b6cfb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:34.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.329+0000 7f2ac2bd2740 4 client.28648 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "dd7972fa", "release": true, "await": true}] 2024-09-10T08:19:34.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.329+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/780304891 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "dd7972fa", "release": true, "await": true}) -- 0x55b1b6be00d0 con 0x55b1b6cf9160 2024-09-10T08:19:34.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.330+0000 7f2abd800640 1 --2- 172.21.15.2:0/780304891 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b1b6cf9160 0x55b1b6cfb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:34.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.330+0000 7f2abd800640 1 --2- 172.21.15.2:0/780304891 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b1b6cf9160 0x55b1b6cfb580 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdd88ef151c30ef server_cookie=bcc7d50553c832ec in_seq=0 out_seq=0 2024-09-10T08:19:34.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.330+0000 7f2a9f400640 10 client.28648 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:34.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2a9f400640 1 -- 172.21.15.2:0/780304891 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b1b6be00d0 con 0x55b1b6cf9160 2024-09-10T08:19:34.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2a9f400640 10 client.28648 handle_command_reply: tid=0 2024-09-10T08:19:34.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2ac2bd2740 1 client.28648 shutdown 2024-09-10T08:19:34.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/780304891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c72e40 msgr2=0x55b1b6c82270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:34.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/780304891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c72e40 0x55b1b6c82270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55b1b6c7d2f0 tx=0x7f2aa40d2490 comp rx=0 tx=0).stop 2024-09-10T08:19:34.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/780304891 shutdown_connections 2024-09-10T08:19:34.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/780304891 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b1b6cf9160 0x55b1b6cfb580 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/780304891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b1b6c7d700 0x55b1b6cf1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/780304891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1b6c72e40 0x55b1b6c82270 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2ac2bd2740 1 --2- 172.21.15.2:0/780304891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1b6c72520 0x55b1b6c81d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2a9f400640 0 client.28648 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:34.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/780304891 >> 172.21.15.2:0/780304891 conn(0x55b1b6c6eab0 msgr2=0x55b1b6c78cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:34.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/780304891 shutdown_connections 2024-09-10T08:19:34.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2ac2bd2740 1 -- 172.21.15.2:0/780304891 wait complete. 2024-09-10T08:19:34.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.333+0000 7f2ac2bd2740 20 client.28648 trim_cache size 0 max 16384 2024-09-10T08:19:34.335 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:19:34.335 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:19:34.336 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:19:34.336 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 40, 2024-09-10T08:19:34.336 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:19:34.336 INFO:teuthology.orchestra.run.smithi002.stdout: "dd7972fa": { 2024-09-10T08:19:34.336 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 40, 2024-09-10T08:19:34.336 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:19:34.336 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:19:34.336 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:19:34.336 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:19:34.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:19:34.337 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:19:34.337 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 145.2, 2024-09-10T08:19:34.337 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:19:34.337 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:19:34.337 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:19:34.337 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:19:34.337 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:19:34.337 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:19:34.337 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:19:34.337 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:19:34.337 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:19:34.338 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:19:34.338 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:19:34.338 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:19:34.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.334+0000 7f2abec00640 1 -- 172.21.15.2:0/2159430923 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2aa00a2b80 msgr2=0x7f2aa00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:34.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.334+0000 7f2abec00640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2aa00a2b80 0x7f2aa00a5040 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2aac006010 tx=0x7f2aac0058d0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.334+0000 7f2abec00640 1 -- 172.21.15.2:0/2159430923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ab8102360 msgr2=0x7f2ab819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:34.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.334+0000 7f2abec00640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ab8102360 0x7f2ab819a9f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400d950 tx=0x7f2aa400de10 comp rx=0 tx=0).stop 2024-09-10T08:19:34.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.334+0000 7f2abec00640 1 -- 172.21.15.2:0/2159430923 shutdown_connections 2024-09-10T08:19:34.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.334+0000 7f2abec00640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ab81079c0 0x7f2ab819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.334+0000 7f2abec00640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ab8102360 0x7f2ab819a9f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.334+0000 7f2abec00640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2aa00a2b80 0x7f2aa00a5040 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.334+0000 7f2abec00640 1 --2- 172.21.15.2:0/2159430923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ab81019c0 0x7f2ab819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:34.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.334+0000 7f2abec00640 1 -- 172.21.15.2:0/2159430923 >> 172.21.15.2:0/2159430923 conn(0x7f2ab80fd740 msgr2=0x7f2ab80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:34.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.335+0000 7f2abec00640 1 -- 172.21.15.2:0/2159430923 shutdown_connections 2024-09-10T08:19:34.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:34.335+0000 7f2abec00640 1 -- 172.21.15.2:0/2159430923 wait complete. 2024-09-10T08:19:35.218 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'dd7972fa', total seconds elapsed: 29.3 2024-09-10T08:19:35.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:35 smithi002 ceph-mon[30563]: pgmap v730: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-10T08:19:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:35 smithi155 ceph-mon[35405]: pgmap v730: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-10T08:19:35.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:35 smithi107 ceph-mon[35048]: pgmap v730: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-10T08:19:37.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:37 smithi002 ceph-mon[30563]: pgmap v731: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-10T08:19:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:37 smithi155 ceph-mon[35405]: pgmap v731: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-10T08:19:37.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:37 smithi107 ceph-mon[35048]: pgmap v731: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-10T08:19:39.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:39 smithi002 ceph-mon[30563]: pgmap v732: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-09-10T08:19:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:39 smithi155 ceph-mon[35405]: pgmap v732: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-09-10T08:19:39.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:39 smithi107 ceph-mon[35048]: pgmap v732: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-09-10T08:19:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:41 smithi002 ceph-mon[30563]: pgmap v733: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-09-10T08:19:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:41 smithi155 ceph-mon[35405]: pgmap v733: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-09-10T08:19:41.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:41 smithi107 ceph-mon[35048]: pgmap v733: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-09-10T08:19:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.985+0000 7fccf7400640 1 -- 172.21.15.2:0/2250125791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf0104f50 msgr2=0x7fccf01053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.985+0000 7fccf7400640 1 --2- 172.21.15.2:0/2250125791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf0104f50 0x7fccf01053d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009900 tx=0x7fccdc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:19:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.985+0000 7fccf7400640 1 -- 172.21.15.2:0/2250125791 shutdown_connections 2024-09-10T08:19:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.985+0000 7fccf7400640 1 --2- 172.21.15.2:0/2250125791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf0105910 0x7fccf010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.985+0000 7fccf7400640 1 --2- 172.21.15.2:0/2250125791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf0104f50 0x7fccf01053d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.985+0000 7fccf7400640 1 --2- 172.21.15.2:0/2250125791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0103d60 0x7fccf0104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.985+0000 7fccf7400640 1 -- 172.21.15.2:0/2250125791 >> 172.21.15.2:0/2250125791 conn(0x7fccf00ff4e0 msgr2=0x7fccf0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.985+0000 7fccf7400640 1 -- 172.21.15.2:0/2250125791 shutdown_connections 2024-09-10T08:19:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.985+0000 7fccf7400640 1 -- 172.21.15.2:0/2250125791 wait complete. 2024-09-10T08:19:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.986+0000 7fccf7400640 1 Processor -- start 2024-09-10T08:19:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.986+0000 7fccf7400640 1 -- start start 2024-09-10T08:19:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0103d60 0x7fccf0199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf0104f50 0x7fccf019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf0105910 0x7fccf01a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf7400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fccf010f9e0 con 0x7fccf0103d60 2024-09-10T08:19:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf7400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fccf010f860 con 0x7fccf0104f50 2024-09-10T08:19:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf7400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fccf010fb60 con 0x7fccf0105910 2024-09-10T08:19:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf4a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0103d60 0x7fccf0199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf4a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0103d60 0x7fccf0199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44706/0 (socket says 172.21.15.2:44706) 2024-09-10T08:19:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf4a00640 1 -- 172.21.15.2:0/3767281308 learned_addr learned my addr 172.21.15.2:0/3767281308 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf5400640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf0105910 0x7fccf01a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccefe00640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf0104f50 0x7fccf019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf4a00640 1 -- 172.21.15.2:0/3767281308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf0105910 msgr2=0x7fccf01a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf4a00640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf0105910 0x7fccf01a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf4a00640 1 -- 172.21.15.2:0/3767281308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf0104f50 msgr2=0x7fccf019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf4a00640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf0104f50 0x7fccf019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccf4a00640 1 -- 172.21.15.2:0/3767281308 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf01a1b90 con 0x7fccf0103d60 2024-09-10T08:19:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.987+0000 7fccefe00640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf0104f50 0x7fccf019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:19:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.988+0000 7fccf4a00640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0103d60 0x7fccf0199f70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcce400e9c0 tx=0x7fcce400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.988+0000 7fcced600640 1 -- 172.21.15.2:0/3767281308 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce400cde0 con 0x7fccf0103d60 2024-09-10T08:19:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.988+0000 7fcced600640 1 -- 172.21.15.2:0/3767281308 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce4010910 con 0x7fccf0103d60 2024-09-10T08:19:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.988+0000 7fcced600640 1 -- 172.21.15.2:0/3767281308 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4018a00 con 0x7fccf0103d60 2024-09-10T08:19:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.988+0000 7fccf7400640 1 -- 172.21.15.2:0/3767281308 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf01a1e80 con 0x7fccf0103d60 2024-09-10T08:19:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.988+0000 7fccf7400640 1 -- 172.21.15.2:0/3767281308 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf01a2290 con 0x7fccf0103d60 2024-09-10T08:19:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.991+0000 7fccfa208740 1 Processor -- start 2024-09-10T08:19:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.991+0000 7fcced600640 1 -- 172.21.15.2:0/3767281308 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcce401f020 con 0x7fccf0103d60 2024-09-10T08:19:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.991+0000 7fccfa208740 1 -- start start 2024-09-10T08:19:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.992+0000 7fccfa208740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b956564090 0x55b956564490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.992+0000 7fccfa208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9565649d0 0x55b956569490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.992+0000 7fccfa208740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956569a60 0x55b95656bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.992+0000 7fccfa208740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b9564c9100 con 0x55b956569a60 2024-09-10T08:19:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.992+0000 7fccfa208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b9564711b0 con 0x55b9565649d0 2024-09-10T08:19:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.992+0000 7fccfa208740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b9564c50c0 con 0x55b956564090 2024-09-10T08:19:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.992+0000 7fccf5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b956564090 0x55b956564490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.992+0000 7fccefe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956569a60 0x55b95656bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.992+0000 7fccf5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b956564090 0x55b956564490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54290/0 (socket says 172.21.15.2:54290) 2024-09-10T08:19:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.992+0000 7fccf5400640 1 -- 172.21.15.2:0/3087884908 learned_addr learned my addr 172.21.15.2:0/3087884908 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.992+0000 7fccf4a00640 1 --2- 172.21.15.2:0/3087884908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9565649d0 0x55b956569490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.993+0000 7fccf5400640 1 -- 172.21.15.2:0/3087884908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9565649d0 msgr2=0x55b956569490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.993+0000 7fccf5400640 1 --2- 172.21.15.2:0/3087884908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9565649d0 0x55b956569490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.993+0000 7fccf5400640 1 -- 172.21.15.2:0/3087884908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956569a60 msgr2=0x55b95656bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.993+0000 7fccf5400640 1 --2- 172.21.15.2:0/3087884908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956569a60 0x55b95656bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.993+0000 7fccf5400640 1 -- 172.21.15.2:0/3087884908 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9561dc210 con 0x55b956564090 2024-09-10T08:19:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.993+0000 7fccefe00640 1 --2- 172.21.15.2:0/3087884908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956569a60 0x55b95656bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.993+0000 7fccf5400640 1 --2- 172.21.15.2:0/3087884908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b956564090 0x55b956564490 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009fd0 tx=0x7fcce80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0a24aa25b1e20f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.993+0000 7fcce1c00640 1 -- 172.21.15.2:0/3087884908 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8004d20 con 0x55b956564090 2024-09-10T08:19:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.993+0000 7fcce1c00640 1 -- 172.21.15.2:0/3087884908 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce8004050 con 0x55b956564090 2024-09-10T08:19:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.993+0000 7fcce1c00640 1 -- 172.21.15.2:0/3087884908 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8005260 con 0x55b956564090 2024-09-10T08:19:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.994+0000 7fcced600640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fccd80a2cb0 0x7fccd80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.994+0000 7fcced600640 1 -- 172.21.15.2:0/3767281308 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcce40cf620 con 0x7fccf0103d60 2024-09-10T08:19:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.994+0000 7fccfa208740 1 -- 172.21.15.2:0/3087884908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b956564090 msgr2=0x55b956564490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.994+0000 7fccfa208740 1 --2- 172.21.15.2:0/3087884908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b956564090 0x55b956564490 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009fd0 tx=0x7fcce80118d0 comp rx=0 tx=0).stop 2024-09-10T08:19:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.994+0000 7fccf4a00640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fccd80a2cb0 0x7fccd80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.994+0000 7fccfa208740 1 -- 172.21.15.2:0/3087884908 shutdown_connections 2024-09-10T08:19:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.994+0000 7fccfa208740 1 --2- 172.21.15.2:0/3087884908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956569a60 0x55b95656bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.994+0000 7fccfa208740 1 --2- 172.21.15.2:0/3087884908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9565649d0 0x55b956569490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.994+0000 7fccfa208740 1 --2- 172.21.15.2:0/3087884908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b956564090 0x55b956564490 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.994+0000 7fccfa208740 1 -- 172.21.15.2:0/3087884908 >> 172.21.15.2:0/3087884908 conn(0x55b95655f700 msgr2=0x55b95655fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.994+0000 7fccfa208740 1 -- 172.21.15.2:0/3087884908 shutdown_connections 2024-09-10T08:19:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.994+0000 7fccfa208740 1 -- 172.21.15.2:0/3087884908 wait complete. 2024-09-10T08:19:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.995+0000 7fccf4a00640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fccd80a2cb0 0x7fccd80a5170 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcce40cd0a0 tx=0x7fcce400efb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.995+0000 7fccfa208740 1 -- start start 2024-09-10T08:19:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.995+0000 7fccfa208740 1 Processor -- start 2024-09-10T08:19:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.995+0000 7fccfa208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956564090 0x55b95656e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccfa208740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9565649d0 0x55b95656e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccfa208740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b956569a60 0x55b956575760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccfa208740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b95656ca30 con 0x55b9565649d0 2024-09-10T08:19:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccfa208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b956452090 con 0x55b956564090 2024-09-10T08:19:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccfa208740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b95656cbb0 con 0x55b956569a60 2024-09-10T08:19:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccfa208740 10 client.0 _ll_register_callbacks cb 0x55b9564a6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccefe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9565649d0 0x55b95656e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccf5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956564090 0x55b95656e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccefe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9565649d0 0x55b95656e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44724/0 (socket says 172.21.15.2:44724) 2024-09-10T08:19:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccefe00640 1 -- 172.21.15.2:0/3218781625 learned_addr learned my addr 172.21.15.2:0/3218781625 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccf5400640 1 --2- 172.21.15.2:0/3218781625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b956569a60 0x55b956575760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccefe00640 1 -- 172.21.15.2:0/3218781625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b956569a60 msgr2=0x55b956575760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccefe00640 1 --2- 172.21.15.2:0/3218781625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b956569a60 0x55b956575760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccefe00640 1 -- 172.21.15.2:0/3218781625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956564090 msgr2=0x55b95656e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccefe00640 1 --2- 172.21.15.2:0/3218781625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956564090 0x55b95656e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.996+0000 7fccefe00640 1 -- 172.21.15.2:0/3218781625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9562a8620 con 0x55b9565649d0 2024-09-10T08:19:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.997+0000 7fccefe00640 1 --2- 172.21.15.2:0/3218781625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9565649d0 0x55b95656e720 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009900 tx=0x7fccdc031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e202ab3d32697099 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.997+0000 7fccd6000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:19:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.997+0000 7fccd6000640 1 -- 172.21.15.2:0/3218781625 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc035040 con 0x55b9565649d0 2024-09-10T08:19:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.997+0000 7fccfa208740 1 -- 172.21.15.2:0/3218781625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce8004050 con 0x55b9565649d0 2024-09-10T08:19:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.997+0000 7fccd6000640 1 -- 172.21.15.2:0/3218781625 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fccdc0042c0 con 0x55b9565649d0 2024-09-10T08:19:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.997+0000 7fccd6000640 1 -- 172.21.15.2:0/3218781625 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fccdc04e050 con 0x55b9565649d0 2024-09-10T08:19:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.997+0000 7fccfa208740 10 client.18702 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.997+0000 7fccfa208740 1 -- 172.21.15.2:0/3218781625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b956578610 con 0x55b9565649d0 2024-09-10T08:19:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.997+0000 7fccd6000640 1 -- 172.21.15.2:0/3218781625 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce8004050 con 0x55b9565649d0 2024-09-10T08:19:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.998+0000 7fccd6000640 1 -- 172.21.15.2:0/3218781625 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fccdc004bf0 con 0x55b9565649d0 2024-09-10T08:19:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.998+0000 7fccfa208740 10 client.18702 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.998+0000 7fccfa208740 10 client.18702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.998+0000 7fccfa208740 20 client.18702 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.998+0000 7fccfa208740 1 --2- 172.21.15.2:0/3218781625 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b956579c40 0x55b95657c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.998+0000 7fccfa208740 4 client.18702 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.998+0000 7fccfa208740 1 -- 172.21.15.2:0/3218781625 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9564d6830 con 0x55b956579c40 2024-09-10T08:19:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.998+0000 7fccf5400640 1 --2- 172.21.15.2:0/3218781625 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b956579c40 0x55b95657c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.999+0000 7fccf5400640 1 --2- 172.21.15.2:0/3218781625 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b956579c40 0x55b95657c050 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e635a021fdbef5 server_cookie=30f53b73a44fff67 in_seq=0 out_seq=0 2024-09-10T08:19:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:41.999+0000 7fccd6000640 10 client.18702 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccd6000640 1 -- 172.21.15.2:0/3218781625 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9564d6830 con 0x55b956579c40 2024-09-10T08:19:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccd6000640 10 client.18702 handle_command_reply: tid=0 2024-09-10T08:19:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccfa208740 1 client.18702 shutdown 2024-09-10T08:19:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccfa208740 1 -- 172.21.15.2:0/3218781625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9565649d0 msgr2=0x55b95656e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccfa208740 1 --2- 172.21.15.2:0/3218781625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9565649d0 0x55b95656e720 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009900 tx=0x7fccdc031a20 comp rx=0 tx=0).stop 2024-09-10T08:19:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccfa208740 1 -- 172.21.15.2:0/3218781625 shutdown_connections 2024-09-10T08:19:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccfa208740 1 --2- 172.21.15.2:0/3218781625 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b956579c40 0x55b95657c050 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccfa208740 1 --2- 172.21.15.2:0/3218781625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b956569a60 0x55b956575760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccfa208740 1 --2- 172.21.15.2:0/3218781625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b9565649d0 0x55b95656e720 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccd6000640 0 client.18702 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccfa208740 1 --2- 172.21.15.2:0/3218781625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956564090 0x55b95656e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccfa208740 1 -- 172.21.15.2:0/3218781625 >> 172.21.15.2:0/3218781625 conn(0x55b95655f700 msgr2=0x55b956560f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccfa208740 1 -- 172.21.15.2:0/3218781625 shutdown_connections 2024-09-10T08:19:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccfa208740 1 -- 172.21.15.2:0/3218781625 wait complete. 2024-09-10T08:19:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.000+0000 7fccfa208740 20 client.18702 trim_cache size 0 max 16384 2024-09-10T08:19:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.016+0000 7fccfa208740 1 Processor -- start 2024-09-10T08:19:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.016+0000 7fccfa208740 1 -- start start 2024-09-10T08:19:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.016+0000 7fccfa208740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956564090 0x55b956560d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.016+0000 7fccfa208740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9565649d0 0x55b956575720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.016+0000 7fccfa208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956569a60 0x55b9565809a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.016+0000 7fccfa208740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01af310 con 0x55b956564090 2024-09-10T08:19:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.016+0000 7fccfa208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01acbe0 con 0x55b956569a60 2024-09-10T08:19:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.016+0000 7fccfa208740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01aaa30 con 0x55b9565649d0 2024-09-10T08:19:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.016+0000 7fccf5400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956564090 0x55b956560d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.016+0000 7fccf5400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956564090 0x55b956560d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44734/0 (socket says 172.21.15.2:44734) 2024-09-10T08:19:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.016+0000 7fccf5400640 1 -- 172.21.15.2:0/1120812682 learned_addr learned my addr 172.21.15.2:0/1120812682 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.016+0000 7fccefe00640 1 --2- 172.21.15.2:0/1120812682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9565649d0 0x55b956575720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.017+0000 7fccf5400640 1 -- 172.21.15.2:0/1120812682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9565649d0 msgr2=0x55b956575720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.017+0000 7fccf5400640 1 --2- 172.21.15.2:0/1120812682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9565649d0 0x55b956575720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.017+0000 7fccf5400640 1 -- 172.21.15.2:0/1120812682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956569a60 msgr2=0x55b9565809a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.017+0000 7fccf5400640 1 --2- 172.21.15.2:0/1120812682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956569a60 0x55b9565809a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.017+0000 7fccf5400640 1 -- 172.21.15.2:0/1120812682 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b95656cd30 con 0x55b956564090 2024-09-10T08:19:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.017+0000 7fccf5400640 1 --2- 172.21.15.2:0/1120812682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956564090 0x55b956560d20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009fd0 tx=0x7fcce8009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f220cfe0c7e0048a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.017+0000 7fcce1200640 1 -- 172.21.15.2:0/1120812682 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8032020 con 0x55b956564090 2024-09-10T08:19:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.017+0000 7fcce1200640 1 -- 172.21.15.2:0/1120812682 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce8004730 con 0x55b956564090 2024-09-10T08:19:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.017+0000 7fcce1200640 1 -- 172.21.15.2:0/1120812682 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce800fc70 con 0x55b956564090 2024-09-10T08:19:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.018+0000 7fccfa208740 1 -- 172.21.15.2:0/1120812682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956564090 msgr2=0x55b956560d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.018+0000 7fccfa208740 1 --2- 172.21.15.2:0/1120812682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956564090 0x55b956560d20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009fd0 tx=0x7fcce8009320 comp rx=0 tx=0).stop 2024-09-10T08:19:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.018+0000 7fccfa208740 1 -- 172.21.15.2:0/1120812682 shutdown_connections 2024-09-10T08:19:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.018+0000 7fccfa208740 1 --2- 172.21.15.2:0/1120812682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956569a60 0x55b9565809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.018+0000 7fccfa208740 1 --2- 172.21.15.2:0/1120812682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9565649d0 0x55b956575720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.018+0000 7fccfa208740 1 --2- 172.21.15.2:0/1120812682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956564090 0x55b956560d20 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.018+0000 7fccfa208740 1 -- 172.21.15.2:0/1120812682 >> 172.21.15.2:0/1120812682 conn(0x55b95655f700 msgr2=0x55b956560610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.018+0000 7fccfa208740 1 -- 172.21.15.2:0/1120812682 shutdown_connections 2024-09-10T08:19:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.018+0000 7fccfa208740 1 -- 172.21.15.2:0/1120812682 wait complete. 2024-09-10T08:19:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.018+0000 7fccfa208740 1 -- start start 2024-09-10T08:19:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.019+0000 7fccfa208740 1 Processor -- start 2024-09-10T08:19:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.019+0000 7fccfa208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956564090 0x55b9565728a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.019+0000 7fccfa208740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9565649d0 0x55b956572de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.019+0000 7fccfa208740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956569a60 0x55b956573df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.019+0000 7fccfa208740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01ab0a0 con 0x55b956569a60 2024-09-10T08:19:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.019+0000 7fccfa208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01ac510 con 0x55b956564090 2024-09-10T08:19:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.019+0000 7fccfa208740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01ab770 con 0x55b9565649d0 2024-09-10T08:19:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.019+0000 7fccfa208740 10 client.0 _ll_register_callbacks cb 0x7fcce80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.019+0000 7fccf5400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956569a60 0x55b956573df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.019+0000 7fccf5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956564090 0x55b9565728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.019+0000 7fccf5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956564090 0x55b9565728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39184/0 (socket says 172.21.15.2:39184) 2024-09-10T08:19:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.019+0000 7fccf5400640 1 -- 172.21.15.2:0/363477476 learned_addr learned my addr 172.21.15.2:0/363477476 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.019+0000 7fccefe00640 1 --2- 172.21.15.2:0/363477476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9565649d0 0x55b956572de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.020+0000 7fccf5400640 1 -- 172.21.15.2:0/363477476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9565649d0 msgr2=0x55b956572de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.020+0000 7fccf5400640 1 --2- 172.21.15.2:0/363477476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9565649d0 0x55b956572de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.020+0000 7fccf5400640 1 -- 172.21.15.2:0/363477476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956569a60 msgr2=0x55b956573df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.020+0000 7fccf5400640 1 --2- 172.21.15.2:0/363477476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956569a60 0x55b956573df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.020+0000 7fccf5400640 1 -- 172.21.15.2:0/363477476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b956574400 con 0x55b956564090 2024-09-10T08:19:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.020+0000 7fccf5400640 1 --2- 172.21.15.2:0/363477476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956564090 0x55b9565728a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcce800f8c0 tx=0x7fcce80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0074a2e5d14a2d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.020+0000 7fccd6000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:19:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.020+0000 7fccd6000640 1 -- 172.21.15.2:0/363477476 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce803c020 con 0x55b956564090 2024-09-10T08:19:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.020+0000 7fccd6000640 1 -- 172.21.15.2:0/363477476 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcce800fb50 con 0x55b956564090 2024-09-10T08:19:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.020+0000 7fccfa208740 1 -- 172.21.15.2:0/363477476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce8004730 con 0x55b956564090 2024-09-10T08:19:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.020+0000 7fccd6000640 1 -- 172.21.15.2:0/363477476 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcce804a050 con 0x55b956564090 2024-09-10T08:19:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.020+0000 7fccfa208740 10 client.28733 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.020+0000 7fccfa208740 1 -- 172.21.15.2:0/363477476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b956575710 con 0x55b956564090 2024-09-10T08:19:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.021+0000 7fccd6000640 1 -- 172.21.15.2:0/363477476 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce8004730 con 0x55b956564090 2024-09-10T08:19:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.021+0000 7fccd6000640 1 -- 172.21.15.2:0/363477476 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcce8025040 con 0x55b956564090 2024-09-10T08:19:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.021+0000 7fccfa208740 10 client.28733 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.021+0000 7fccfa208740 10 client.28733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.021+0000 7fccfa208740 20 client.28733 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.021+0000 7fccfa208740 1 --2- 172.21.15.2:0/363477476 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9565e4cc0 0x55b9565e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.021+0000 7fccfa208740 4 client.28733 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:19:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.021+0000 7fccfa208740 1 -- 172.21.15.2:0/363477476 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9564d0b80 con 0x55b9565e4cc0 2024-09-10T08:19:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.021+0000 7fccefe00640 1 --2- 172.21.15.2:0/363477476 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9565e4cc0 0x55b9565e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.022+0000 7fccefe00640 1 --2- 172.21.15.2:0/363477476 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9565e4cc0 0x55b9565e70b0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f81cd5f27b3390 server_cookie=d9175908b352c6c6 in_seq=0 out_seq=0 2024-09-10T08:19:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.022+0000 7fccd6000640 10 client.28733 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.033+0000 7fccd6000640 1 -- 172.21.15.2:0/363477476 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9564d0b80 con 0x55b9565e4cc0 2024-09-10T08:19:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.033+0000 7fccd6000640 10 client.28733 handle_command_reply: tid=0 2024-09-10T08:19:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.033+0000 7fccfa208740 1 client.28733 shutdown 2024-09-10T08:19:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.034+0000 7fccfa208740 1 -- 172.21.15.2:0/363477476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956564090 msgr2=0x55b9565728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.034+0000 7fccfa208740 1 --2- 172.21.15.2:0/363477476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956564090 0x55b9565728a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcce800f8c0 tx=0x7fcce80118d0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.034+0000 7fccfa208740 1 -- 172.21.15.2:0/363477476 shutdown_connections 2024-09-10T08:19:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.034+0000 7fccfa208740 1 --2- 172.21.15.2:0/363477476 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b9565e4cc0 0x55b9565e70b0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.034+0000 7fccfa208740 1 --2- 172.21.15.2:0/363477476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b956569a60 0x55b956573df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.034+0000 7fccfa208740 1 --2- 172.21.15.2:0/363477476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9565649d0 0x55b956572de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.034+0000 7fccfa208740 1 --2- 172.21.15.2:0/363477476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b956564090 0x55b9565728a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.034+0000 7fccd6000640 0 client.28733 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.034+0000 7fccfa208740 1 -- 172.21.15.2:0/363477476 >> 172.21.15.2:0/363477476 conn(0x55b95655f700 msgr2=0x55b95655e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.034+0000 7fccfa208740 1 -- 172.21.15.2:0/363477476 shutdown_connections 2024-09-10T08:19:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.034+0000 7fccfa208740 1 -- 172.21.15.2:0/363477476 wait complete. 2024-09-10T08:19:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.034+0000 7fccfa208740 20 client.28733 trim_cache size 0 max 16384 2024-09-10T08:19:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:19:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:19:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:19:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:19:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.035+0000 7fccf7400640 1 -- 172.21.15.2:0/3767281308 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fccd80a2cb0 msgr2=0x7fccd80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.035+0000 7fccf7400640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fccd80a2cb0 0x7fccd80a5170 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcce40cd0a0 tx=0x7fcce400efb0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.035+0000 7fccf7400640 1 -- 172.21.15.2:0/3767281308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0103d60 msgr2=0x7fccf0199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.035+0000 7fccf7400640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0103d60 0x7fccf0199f70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcce400e9c0 tx=0x7fcce400ee80 comp rx=0 tx=0).stop 2024-09-10T08:19:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.035+0000 7fccf7400640 1 -- 172.21.15.2:0/3767281308 shutdown_connections 2024-09-10T08:19:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.035+0000 7fccf7400640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fccd80a2cb0 0x7fccd80a5170 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.035+0000 7fccf7400640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf0105910 0x7fccf01a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.035+0000 7fccf7400640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf0104f50 0x7fccf019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.035+0000 7fccf7400640 1 --2- 172.21.15.2:0/3767281308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fccf0103d60 0x7fccf0199f70 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.035+0000 7fccf7400640 1 -- 172.21.15.2:0/3767281308 >> 172.21.15.2:0/3767281308 conn(0x7fccf00ff4e0 msgr2=0x7fccf0108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.035+0000 7fccf7400640 1 -- 172.21.15.2:0/3767281308 shutdown_connections 2024-09-10T08:19:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:42.035+0000 7fccf7400640 1 -- 172.21.15.2:0/3767281308 wait complete. 2024-09-10T08:19:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:43 smithi155 ceph-mon[35405]: pgmap v734: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-10T08:19:43.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:43 smithi107 ceph-mon[35048]: pgmap v734: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-10T08:19:44.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:43 smithi002 ceph-mon[30563]: pgmap v734: 129 pgs: 129 active+clean; 1.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-10T08:19:45.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:45 smithi155 ceph-mon[35405]: pgmap v735: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:19:45.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:45 smithi107 ceph-mon[35048]: pgmap v735: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:19:46.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:45 smithi002 ceph-mon[30563]: pgmap v735: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:19:47.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:47 smithi155 ceph-mon[35405]: pgmap v736: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T08:19:47.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:47 smithi107 ceph-mon[35048]: pgmap v736: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T08:19:48.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:47 smithi002 ceph-mon[30563]: pgmap v736: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T08:19:49.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:49 smithi155 ceph-mon[35405]: pgmap v737: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-10T08:19:49.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:49 smithi107 ceph-mon[35048]: pgmap v737: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-10T08:19:50.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:49 smithi002 ceph-mon[30563]: pgmap v737: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-10T08:19:50.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:50 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:19:50.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:50 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:19:51.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:50 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:19:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:51 smithi155 ceph-mon[35405]: pgmap v738: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:19:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:51 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:19:51.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:51 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:19:51.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:51 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:19:51.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:51 smithi107 ceph-mon[35048]: pgmap v738: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:19:51.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:51 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:19:51.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:51 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:19:51.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:51 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:19:52.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:51 smithi002 ceph-mon[30563]: pgmap v738: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:19:52.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:51 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:19:52.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:51 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:19:52.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:51 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:19:53.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:53 smithi002 ceph-mon[30563]: pgmap v739: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:19:53.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:53 smithi155 ceph-mon[35405]: pgmap v739: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:19:53.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:53 smithi107 ceph-mon[35048]: pgmap v739: 129 pgs: 129 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:19:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.153+0000 7fac4e000640 1 -- 172.21.15.2:0/3274660358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac48104f50 msgr2=0x7fac481053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.153+0000 7fac4e000640 1 --2- 172.21.15.2:0/3274660358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac48104f50 0x7fac481053d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fac34009900 tx=0x7fac3402a410 comp rx=0 tx=0).stop 2024-09-10T08:19:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.153+0000 7fac4e000640 1 -- 172.21.15.2:0/3274660358 shutdown_connections 2024-09-10T08:19:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.153+0000 7fac4e000640 1 --2- 172.21.15.2:0/3274660358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac48105910 0x7fac4810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.153+0000 7fac4e000640 1 --2- 172.21.15.2:0/3274660358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac48104f50 0x7fac481053d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.153+0000 7fac4e000640 1 --2- 172.21.15.2:0/3274660358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac48103d60 0x7fac48104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.153+0000 7fac4e000640 1 -- 172.21.15.2:0/3274660358 >> 172.21.15.2:0/3274660358 conn(0x7fac480ff4e0 msgr2=0x7fac48101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.153+0000 7fac4e000640 1 -- 172.21.15.2:0/3274660358 shutdown_connections 2024-09-10T08:19:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.153+0000 7fac4e000640 1 -- 172.21.15.2:0/3274660358 wait complete. 2024-09-10T08:19:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.154+0000 7fac4e000640 1 Processor -- start 2024-09-10T08:19:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.154+0000 7fac4e000640 1 -- start start 2024-09-10T08:19:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.154+0000 7fac4e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac48103d60 0x7fac48199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.154+0000 7fac4e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac48104f50 0x7fac4819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.154+0000 7fac4e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac48105910 0x7fac481a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.154+0000 7fac4e000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac4810f9e0 con 0x7fac48104f50 2024-09-10T08:19:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.154+0000 7fac4e000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac4810f860 con 0x7fac48105910 2024-09-10T08:19:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.154+0000 7fac4e000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac4810fb60 con 0x7fac48103d60 2024-09-10T08:19:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.155+0000 7fac47e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac48105910 0x7fac481a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.155+0000 7fac47400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac48103d60 0x7fac48199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.155+0000 7fac47e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac48105910 0x7fac481a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:57648/0 (socket says 172.21.15.2:57648) 2024-09-10T08:19:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.155+0000 7fac47e00640 1 -- 172.21.15.2:0/1329457505 learned_addr learned my addr 172.21.15.2:0/1329457505 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.155+0000 7fac46a00640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac48104f50 0x7fac4819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.155+0000 7fac47e00640 1 -- 172.21.15.2:0/1329457505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac48103d60 msgr2=0x7fac48199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.155+0000 7fac47e00640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac48103d60 0x7fac48199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.155+0000 7fac47e00640 1 -- 172.21.15.2:0/1329457505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac48104f50 msgr2=0x7fac4819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.155+0000 7fac47e00640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac48104f50 0x7fac4819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.155+0000 7fac47e00640 1 -- 172.21.15.2:0/1329457505 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac481a1bd0 con 0x7fac48105910 2024-09-10T08:19:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.155+0000 7fac47400640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac48103d60 0x7fac48199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.155+0000 7fac47e00640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac48105910 0x7fac481a1560 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fac4000eb10 tx=0x7fac4000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.156+0000 7fac3fe00640 1 -- 172.21.15.2:0/1329457505 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40004b20 con 0x7fac48105910 2024-09-10T08:19:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.156+0000 7fac3fe00640 1 -- 172.21.15.2:0/1329457505 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac40004cc0 con 0x7fac48105910 2024-09-10T08:19:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.156+0000 7fac4e000640 1 -- 172.21.15.2:0/1329457505 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac481a1ec0 con 0x7fac48105910 2024-09-10T08:19:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.156+0000 7fac3fe00640 1 -- 172.21.15.2:0/1329457505 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40010760 con 0x7fac48105910 2024-09-10T08:19:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.156+0000 7fac4e000640 1 -- 172.21.15.2:0/1329457505 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac481a2330 con 0x7fac48105910 2024-09-10T08:19:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.159+0000 7fac50dba740 1 Processor -- start 2024-09-10T08:19:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.159+0000 7fac3fe00640 1 -- 172.21.15.2:0/1329457505 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fac40005020 con 0x7fac48105910 2024-09-10T08:19:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.159+0000 7fac50dba740 1 -- start start 2024-09-10T08:19:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.160+0000 7fac50dba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343be490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.160+0000 7fac50dba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343c3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.160+0000 7fac50dba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343c3a60 0x5576343c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.160+0000 7fac50dba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557634323100 con 0x5576343be090 2024-09-10T08:19:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.160+0000 7fac50dba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5576342cb1b0 con 0x5576343c3a60 2024-09-10T08:19:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.160+0000 7fac50dba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55763431f0c0 con 0x5576343be9d0 2024-09-10T08:19:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.160+0000 7fac46a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343be490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.160+0000 7fac47400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.160+0000 7fac47400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343c3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:33136/0 (socket says 172.21.15.2:33136) 2024-09-10T08:19:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.160+0000 7fac47400640 1 -- 172.21.15.2:0/1883092718 learned_addr learned my addr 172.21.15.2:0/1883092718 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.160+0000 7fac46a00640 1 --2- 172.21.15.2:0/1883092718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343c3a60 0x5576343c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.160+0000 7fac47400640 1 -- 172.21.15.2:0/1883092718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343c3a60 msgr2=0x5576343c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.160+0000 7fac47400640 1 --2- 172.21.15.2:0/1883092718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343c3a60 0x5576343c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.161+0000 7fac47400640 1 -- 172.21.15.2:0/1883092718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 msgr2=0x5576343be490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.161+0000 7fac47400640 1 --2- 172.21.15.2:0/1883092718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343be490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.161+0000 7fac47400640 1 -- 172.21.15.2:0/1883092718 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557634036210 con 0x5576343be9d0 2024-09-10T08:19:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.161+0000 7fac46a00640 1 --2- 172.21.15.2:0/1883092718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343be490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:19:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.161+0000 7fac3fe00640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fac2c0a2c30 0x7fac2c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.161+0000 7fac3fe00640 1 -- 172.21.15.2:0/1329457505 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fac400cfa40 con 0x7fac48105910 2024-09-10T08:19:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.161+0000 7fac47400640 1 --2- 172.21.15.2:0/1883092718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343c3490 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fac380114f0 tx=0x7fac380119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b60cab091b31d387 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.161+0000 7fac46a00640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fac2c0a2c30 0x7fac2c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.161+0000 7fac33e00640 1 -- 172.21.15.2:0/1883092718 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3801e070 con 0x5576343be9d0 2024-09-10T08:19:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.161+0000 7fac33e00640 1 -- 172.21.15.2:0/1883092718 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac380035f0 con 0x5576343be9d0 2024-09-10T08:19:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.161+0000 7fac33e00640 1 -- 172.21.15.2:0/1883092718 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3800f7c0 con 0x5576343be9d0 2024-09-10T08:19:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.162+0000 7fac50dba740 1 -- 172.21.15.2:0/1883092718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 msgr2=0x5576343c3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.162+0000 7fac50dba740 1 --2- 172.21.15.2:0/1883092718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343c3490 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fac380114f0 tx=0x7fac380119b0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.162+0000 7fac46a00640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fac2c0a2c30 0x7fac2c0a50f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fac340314e0 tx=0x7fac34005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.162+0000 7fac50dba740 1 -- 172.21.15.2:0/1883092718 shutdown_connections 2024-09-10T08:19:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.162+0000 7fac50dba740 1 --2- 172.21.15.2:0/1883092718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343c3a60 0x5576343c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.162+0000 7fac50dba740 1 --2- 172.21.15.2:0/1883092718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343c3490 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.162+0000 7fac50dba740 1 --2- 172.21.15.2:0/1883092718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343be490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.162+0000 7fac50dba740 1 -- 172.21.15.2:0/1883092718 >> 172.21.15.2:0/1883092718 conn(0x5576343b9700 msgr2=0x5576343b9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.162+0000 7fac50dba740 1 -- 172.21.15.2:0/1883092718 shutdown_connections 2024-09-10T08:19:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.162+0000 7fac50dba740 1 -- 172.21.15.2:0/1883092718 wait complete. 2024-09-10T08:19:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.163+0000 7fac50dba740 1 -- start start 2024-09-10T08:19:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.163+0000 7fac50dba740 1 Processor -- start 2024-09-10T08:19:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.163+0000 7fac50dba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343be090 0x5576343c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.163+0000 7fac50dba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.163+0000 7fac50dba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343c3a60 0x5576343cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.163+0000 7fac50dba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5576343c6a80 con 0x5576343c3a60 2024-09-10T08:19:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.163+0000 7fac50dba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5576342ac090 con 0x5576343be090 2024-09-10T08:19:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.163+0000 7fac50dba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5576343c6c00 con 0x5576343be9d0 2024-09-10T08:19:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.164+0000 7fac50dba740 10 client.0 _ll_register_callbacks cb 0x557634300f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.164+0000 7fac47400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343be090 0x5576343c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.164+0000 7fac47400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343be090 0x5576343c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:57666/0 (socket says 172.21.15.2:57666) 2024-09-10T08:19:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.164+0000 7fac47400640 1 -- 172.21.15.2:0/2349988250 learned_addr learned my addr 172.21.15.2:0/2349988250 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.164+0000 7fac47e00640 1 --2- 172.21.15.2:0/2349988250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343c3a60 0x5576343cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.164+0000 7fac46a00640 1 --2- 172.21.15.2:0/2349988250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.164+0000 7fac47400640 1 -- 172.21.15.2:0/2349988250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 msgr2=0x5576343c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.164+0000 7fac47400640 1 --2- 172.21.15.2:0/2349988250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.164+0000 7fac47400640 1 -- 172.21.15.2:0/2349988250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343c3a60 msgr2=0x5576343cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.164+0000 7fac47400640 1 --2- 172.21.15.2:0/2349988250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343c3a60 0x5576343cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.164+0000 7fac47400640 1 -- 172.21.15.2:0/2349988250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557634102620 con 0x5576343be090 2024-09-10T08:19:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.164+0000 7fac47400640 1 --2- 172.21.15.2:0/2349988250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343be090 0x5576343c82f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fac38011eb0 tx=0x7fac380042b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59b9395237e12bfb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.164+0000 7fac30c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:19:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.165+0000 7fac30c00640 1 -- 172.21.15.2:0/2349988250 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38026020 con 0x5576343be090 2024-09-10T08:19:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.165+0000 7fac30c00640 1 -- 172.21.15.2:0/2349988250 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fac38005040 con 0x5576343be090 2024-09-10T08:19:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.165+0000 7fac50dba740 1 -- 172.21.15.2:0/2349988250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac380035f0 con 0x5576343be090 2024-09-10T08:19:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.165+0000 7fac30c00640 1 -- 172.21.15.2:0/2349988250 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fac38031050 con 0x5576343be090 2024-09-10T08:19:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.165+0000 7fac50dba740 10 client.28745 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.165+0000 7fac50dba740 1 -- 172.21.15.2:0/2349988250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576343d2660 con 0x5576343be090 2024-09-10T08:19:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.165+0000 7fac30c00640 1 -- 172.21.15.2:0/2349988250 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac380035f0 con 0x5576343be090 2024-09-10T08:19:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.166+0000 7fac30c00640 1 -- 172.21.15.2:0/2349988250 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fac38021030 con 0x5576343be090 2024-09-10T08:19:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.166+0000 7fac50dba740 10 client.28745 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.166+0000 7fac50dba740 10 client.28745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.166+0000 7fac50dba740 20 client.28745 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.166+0000 7fac50dba740 1 --2- 172.21.15.2:0/2349988250 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5576343d3bb0 0x5576343d5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.166+0000 7fac50dba740 4 client.28745 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.166+0000 7fac50dba740 1 -- 172.21.15.2:0/2349988250 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557634330830 con 0x5576343d3bb0 2024-09-10T08:19:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.166+0000 7fac46a00640 1 --2- 172.21.15.2:0/2349988250 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5576343d3bb0 0x5576343d5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.167+0000 7fac46a00640 1 --2- 172.21.15.2:0/2349988250 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5576343d3bb0 0x5576343d5fc0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc4100adae3a8e8 server_cookie=41aaec3072461fd4 in_seq=0 out_seq=0 2024-09-10T08:19:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.167+0000 7fac30c00640 10 client.28745 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac30c00640 1 -- 172.21.15.2:0/2349988250 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557634330830 con 0x5576343d3bb0 2024-09-10T08:19:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac30c00640 10 client.28745 handle_command_reply: tid=0 2024-09-10T08:19:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac50dba740 1 client.28745 shutdown 2024-09-10T08:19:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac50dba740 1 -- 172.21.15.2:0/2349988250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343be090 msgr2=0x5576343c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac50dba740 1 --2- 172.21.15.2:0/2349988250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343be090 0x5576343c82f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fac38011eb0 tx=0x7fac380042b0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac50dba740 1 -- 172.21.15.2:0/2349988250 shutdown_connections 2024-09-10T08:19:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac50dba740 1 --2- 172.21.15.2:0/2349988250 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5576343d3bb0 0x5576343d5fc0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac50dba740 1 --2- 172.21.15.2:0/2349988250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343c3a60 0x5576343cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac50dba740 1 --2- 172.21.15.2:0/2349988250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac50dba740 1 --2- 172.21.15.2:0/2349988250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343be090 0x5576343c82f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac30c00640 0 client.28745 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac50dba740 1 -- 172.21.15.2:0/2349988250 >> 172.21.15.2:0/2349988250 conn(0x5576343b9700 msgr2=0x5576343baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac50dba740 1 -- 172.21.15.2:0/2349988250 shutdown_connections 2024-09-10T08:19:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac50dba740 1 -- 172.21.15.2:0/2349988250 wait complete. 2024-09-10T08:19:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.168+0000 7fac50dba740 20 client.28745 trim_cache size 0 max 16384 2024-09-10T08:19:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.183+0000 7fac50dba740 1 Processor -- start 2024-09-10T08:19:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.183+0000 7fac50dba740 1 -- start start 2024-09-10T08:19:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac50dba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac50dba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac50dba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343c3a60 0x5576343c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac50dba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac481af2e0 con 0x5576343be090 2024-09-10T08:19:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac50dba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac481acbb0 con 0x5576343c3a60 2024-09-10T08:19:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac50dba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac481aaa00 con 0x5576343be9d0 2024-09-10T08:19:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac47400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac47400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53984/0 (socket says 172.21.15.2:53984) 2024-09-10T08:19:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac47400640 1 -- 172.21.15.2:0/2240079849 learned_addr learned my addr 172.21.15.2:0/2240079849 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac46a00640 1 --2- 172.21.15.2:0/2240079849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac47e00640 1 --2- 172.21.15.2:0/2240079849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343c3a60 0x5576343c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac47400640 1 -- 172.21.15.2:0/2240079849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 msgr2=0x5576343cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac47400640 1 --2- 172.21.15.2:0/2240079849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac47400640 1 -- 172.21.15.2:0/2240079849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343c3a60 msgr2=0x5576343c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac47400640 1 --2- 172.21.15.2:0/2240079849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343c3a60 0x5576343c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.184+0000 7fac47400640 1 -- 172.21.15.2:0/2240079849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576343b9520 con 0x5576343be090 2024-09-10T08:19:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.185+0000 7fac47400640 1 --2- 172.21.15.2:0/2240079849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343c80a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fac38009bb0 tx=0x7fac38009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2711952b4c5e56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.185+0000 7fac33400640 1 -- 172.21.15.2:0/2240079849 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38026030 con 0x5576343be090 2024-09-10T08:19:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.185+0000 7fac33400640 1 -- 172.21.15.2:0/2240079849 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac38009ca0 con 0x5576343be090 2024-09-10T08:19:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.185+0000 7fac33400640 1 -- 172.21.15.2:0/2240079849 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3801e040 con 0x5576343be090 2024-09-10T08:19:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.186+0000 7fac50dba740 1 -- 172.21.15.2:0/2240079849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 msgr2=0x5576343c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.186+0000 7fac50dba740 1 --2- 172.21.15.2:0/2240079849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343c80a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fac38009bb0 tx=0x7fac38009f20 comp rx=0 tx=0).stop 2024-09-10T08:19:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.186+0000 7fac50dba740 1 -- 172.21.15.2:0/2240079849 shutdown_connections 2024-09-10T08:19:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.186+0000 7fac50dba740 1 --2- 172.21.15.2:0/2240079849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343c3a60 0x5576343c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.186+0000 7fac50dba740 1 --2- 172.21.15.2:0/2240079849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343be9d0 0x5576343cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.186+0000 7fac50dba740 1 --2- 172.21.15.2:0/2240079849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343c80a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.186+0000 7fac50dba740 1 -- 172.21.15.2:0/2240079849 >> 172.21.15.2:0/2240079849 conn(0x5576343b9700 msgr2=0x5576343b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.186+0000 7fac50dba740 1 -- 172.21.15.2:0/2240079849 shutdown_connections 2024-09-10T08:19:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.186+0000 7fac50dba740 1 -- 172.21.15.2:0/2240079849 wait complete. 2024-09-10T08:19:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.186+0000 7fac50dba740 1 -- start start 2024-09-10T08:19:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.187+0000 7fac50dba740 1 Processor -- start 2024-09-10T08:19:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.187+0000 7fac50dba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.187+0000 7fac50dba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343be9d0 0x5576343cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.187+0000 7fac50dba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343c3a60 0x55763443c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.187+0000 7fac50dba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac481ab070 con 0x5576343be090 2024-09-10T08:19:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.187+0000 7fac50dba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac481ac4e0 con 0x5576343be9d0 2024-09-10T08:19:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.187+0000 7fac50dba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac481ab740 con 0x5576343c3a60 2024-09-10T08:19:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.187+0000 7fac50dba740 10 client.0 _ll_register_callbacks cb 0x7fac3801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.187+0000 7fac47400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.187+0000 7fac47400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53994/0 (socket says 172.21.15.2:53994) 2024-09-10T08:19:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.187+0000 7fac47e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343c3a60 0x55763443c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.187+0000 7fac47400640 1 -- 172.21.15.2:0/3248314796 learned_addr learned my addr 172.21.15.2:0/3248314796 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.188+0000 7fac46a00640 1 --2- 172.21.15.2:0/3248314796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343be9d0 0x5576343cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.188+0000 7fac47e00640 1 -- 172.21.15.2:0/3248314796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343be9d0 msgr2=0x5576343cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.188+0000 7fac47e00640 1 --2- 172.21.15.2:0/3248314796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343be9d0 0x5576343cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.188+0000 7fac47e00640 1 -- 172.21.15.2:0/3248314796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 msgr2=0x5576343cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.188+0000 7fac47e00640 1 --2- 172.21.15.2:0/3248314796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.188+0000 7fac47e00640 1 -- 172.21.15.2:0/3248314796 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55763443cdd0 con 0x5576343c3a60 2024-09-10T08:19:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.188+0000 7fac47e00640 1 --2- 172.21.15.2:0/3248314796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343c3a60 0x55763443c890 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fac400cd0f0 tx=0x7fac4009bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34c7ea9c8ed8a02d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.188+0000 7fac30c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:19:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.188+0000 7fac30c00640 1 -- 172.21.15.2:0/3248314796 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac400b9070 con 0x5576343c3a60 2024-09-10T08:19:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.188+0000 7fac50dba740 1 -- 172.21.15.2:0/3248314796 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac38009ca0 con 0x5576343c3a60 2024-09-10T08:19:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.188+0000 7fac30c00640 1 -- 172.21.15.2:0/3248314796 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fac400a4830 con 0x5576343c3a60 2024-09-10T08:19:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.189+0000 7fac30c00640 1 -- 172.21.15.2:0/3248314796 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fac40082050 con 0x5576343c3a60 2024-09-10T08:19:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.189+0000 7fac50dba740 10 client.28684 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.189+0000 7fac50dba740 1 -- 172.21.15.2:0/3248314796 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55763443df90 con 0x5576343c3a60 2024-09-10T08:19:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.189+0000 7fac30c00640 1 -- 172.21.15.2:0/3248314796 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac38009ca0 con 0x5576343c3a60 2024-09-10T08:19:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.189+0000 7fac30c00640 1 -- 172.21.15.2:0/3248314796 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fac4009b2a0 con 0x5576343c3a60 2024-09-10T08:19:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.189+0000 7fac50dba740 10 client.28684 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.189+0000 7fac50dba740 10 client.28684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.189+0000 7fac50dba740 20 client.28684 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.189+0000 7fac50dba740 1 --2- 172.21.15.2:0/3248314796 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55763443e960 0x557634440d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.189+0000 7fac50dba740 4 client.28684 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:19:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.189+0000 7fac50dba740 1 -- 172.21.15.2:0/3248314796 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55763432ab80 con 0x55763443e960 2024-09-10T08:19:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.190+0000 7fac47400640 1 --2- 172.21.15.2:0/3248314796 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55763443e960 0x557634440d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.190+0000 7fac47400640 1 --2- 172.21.15.2:0/3248314796 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55763443e960 0x557634440d80 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee4ffe45d24118d2 server_cookie=89f03a8e4d69f014 in_seq=0 out_seq=0 2024-09-10T08:19:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.190+0000 7fac30c00640 10 client.28684 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.203+0000 7fac30c00640 1 -- 172.21.15.2:0/3248314796 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55763432ab80 con 0x55763443e960 2024-09-10T08:19:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.203+0000 7fac30c00640 10 client.28684 handle_command_reply: tid=0 2024-09-10T08:19:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.203+0000 7fac50dba740 1 client.28684 shutdown 2024-09-10T08:19:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.203+0000 7fac50dba740 1 -- 172.21.15.2:0/3248314796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343c3a60 msgr2=0x55763443c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.203+0000 7fac50dba740 1 --2- 172.21.15.2:0/3248314796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343c3a60 0x55763443c890 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fac400cd0f0 tx=0x7fac4009bd20 comp rx=0 tx=0).stop 2024-09-10T08:19:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.203+0000 7fac50dba740 1 -- 172.21.15.2:0/3248314796 shutdown_connections 2024-09-10T08:19:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.203+0000 7fac50dba740 1 --2- 172.21.15.2:0/3248314796 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55763443e960 0x557634440d80 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.203+0000 7fac50dba740 1 --2- 172.21.15.2:0/3248314796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576343c3a60 0x55763443c890 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.203+0000 7fac50dba740 1 --2- 172.21.15.2:0/3248314796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576343be9d0 0x5576343cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.203+0000 7fac30c00640 0 client.28684 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.203+0000 7fac50dba740 1 --2- 172.21.15.2:0/3248314796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5576343be090 0x5576343cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.203+0000 7fac50dba740 1 -- 172.21.15.2:0/3248314796 >> 172.21.15.2:0/3248314796 conn(0x5576343b9700 msgr2=0x5576343c8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.204+0000 7fac50dba740 1 -- 172.21.15.2:0/3248314796 shutdown_connections 2024-09-10T08:19:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.204+0000 7fac50dba740 1 -- 172.21.15.2:0/3248314796 wait complete. 2024-09-10T08:19:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.204+0000 7fac50dba740 20 client.28684 trim_cache size 0 max 16384 2024-09-10T08:19:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:19:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:19:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:19:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:19:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.204+0000 7fac4e000640 1 -- 172.21.15.2:0/1329457505 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fac2c0a2c30 msgr2=0x7fac2c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.204+0000 7fac4e000640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fac2c0a2c30 0x7fac2c0a50f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fac340314e0 tx=0x7fac34005fd0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.204+0000 7fac4e000640 1 -- 172.21.15.2:0/1329457505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac48105910 msgr2=0x7fac481a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.204+0000 7fac4e000640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac48105910 0x7fac481a1560 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fac4000eb10 tx=0x7fac4000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.204+0000 7fac4e000640 1 -- 172.21.15.2:0/1329457505 shutdown_connections 2024-09-10T08:19:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.204+0000 7fac4e000640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac48105910 0x7fac481a1560 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.204+0000 7fac4e000640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac48104f50 0x7fac4819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.204+0000 7fac4e000640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fac2c0a2c30 0x7fac2c0a50f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.204+0000 7fac4e000640 1 --2- 172.21.15.2:0/1329457505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac48103d60 0x7fac48199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.204+0000 7fac4e000640 1 -- 172.21.15.2:0/1329457505 >> 172.21.15.2:0/1329457505 conn(0x7fac480ff4e0 msgr2=0x7fac48108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.205+0000 7fac4e000640 1 -- 172.21.15.2:0/1329457505 shutdown_connections 2024-09-10T08:19:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:19:55.205+0000 7fac4e000640 1 -- 172.21.15.2:0/1329457505 wait complete. 2024-09-10T08:19:55.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:55 smithi002 ceph-mon[30563]: pgmap v740: 129 pgs: 129 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:19:55.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:55 smithi155 ceph-mon[35405]: pgmap v740: 129 pgs: 129 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:19:55.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:55 smithi107 ceph-mon[35048]: pgmap v740: 129 pgs: 129 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:19:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:57 smithi155 ceph-mon[35405]: pgmap v741: 129 pgs: 129 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:19:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:57 smithi107 ceph-mon[35048]: pgmap v741: 129 pgs: 129 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:19:58.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:57 smithi002 ceph-mon[30563]: pgmap v741: 129 pgs: 129 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:19:58.154 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:19:58.547 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:19:59.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.142+0000 7f62aae00640 1 -- 172.21.15.2:0/3002517625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62a41049a0 msgr2=0x7f62a41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:59.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.142+0000 7f62aae00640 1 --2- 172.21.15.2:0/3002517625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62a41049a0 0x7f62a41089f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f628c009900 tx=0x7f628c02a400 comp rx=0 tx=0).stop 2024-09-10T08:19:59.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.142+0000 7f62aae00640 1 -- 172.21.15.2:0/3002517625 shutdown_connections 2024-09-10T08:19:59.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.142+0000 7f62aae00640 1 --2- 172.21.15.2:0/3002517625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62a4109120 0x7f62a4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.142+0000 7f62aae00640 1 --2- 172.21.15.2:0/3002517625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62a41049a0 0x7f62a41089f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.142+0000 7f62aae00640 1 --2- 172.21.15.2:0/3002517625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62a4103ff0 0x7f62a41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.142+0000 7f62aae00640 1 -- 172.21.15.2:0/3002517625 >> 172.21.15.2:0/3002517625 conn(0x7f62a40ffa90 msgr2=0x7f62a4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:59.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.142+0000 7f62aae00640 1 -- 172.21.15.2:0/3002517625 shutdown_connections 2024-09-10T08:19:59.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.142+0000 7f62aae00640 1 -- 172.21.15.2:0/3002517625 wait complete. 2024-09-10T08:19:59.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.143+0000 7f62aae00640 1 Processor -- start 2024-09-10T08:19:59.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.143+0000 7f62aae00640 1 -- start start 2024-09-10T08:19:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.144+0000 7f62aae00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62a4103ff0 0x7f62a419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.144+0000 7f62aae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62a41049a0 0x7f62a419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.144+0000 7f62aae00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62a4109120 0x7f62a419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.144+0000 7f62aae00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f62a4114120 con 0x7f62a4109120 2024-09-10T08:19:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.144+0000 7f62aae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f62a4113fa0 con 0x7f62a41049a0 2024-09-10T08:19:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.144+0000 7f62aae00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62a41142a0 con 0x7f62a4103ff0 2024-09-10T08:19:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.144+0000 7f62a8e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62a4109120 0x7f62a419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.144+0000 7f629be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62a41049a0 0x7f62a419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.144+0000 7f62a8e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62a4109120 0x7f62a419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34018/0 (socket says 172.21.15.2:34018) 2024-09-10T08:19:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.144+0000 7f62a8e00640 1 -- 172.21.15.2:0/836548771 learned_addr learned my addr 172.21.15.2:0/836548771 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.144+0000 7f62a3e00640 1 --2- 172.21.15.2:0/836548771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62a4103ff0 0x7f62a419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.145+0000 7f62a8e00640 1 -- 172.21.15.2:0/836548771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62a4103ff0 msgr2=0x7f62a419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.145+0000 7f62a8e00640 1 --2- 172.21.15.2:0/836548771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62a4103ff0 0x7f62a419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.145+0000 7f62a8e00640 1 -- 172.21.15.2:0/836548771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62a41049a0 msgr2=0x7f62a419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.145+0000 7f62a8e00640 1 --2- 172.21.15.2:0/836548771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62a41049a0 0x7f62a419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.145+0000 7f62a8e00640 1 -- 172.21.15.2:0/836548771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a419f670 con 0x7f62a4109120 2024-09-10T08:19:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.145+0000 7f62a8e00640 1 --2- 172.21.15.2:0/836548771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62a4109120 0x7f62a419ef90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f629c00eb10 tx=0x7f629c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.145+0000 7f62a1600640 1 -- 172.21.15.2:0/836548771 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c004b20 con 0x7f62a4109120 2024-09-10T08:19:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.145+0000 7f62a1600640 1 -- 172.21.15.2:0/836548771 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f629c004cc0 con 0x7f62a4109120 2024-09-10T08:19:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.145+0000 7f62a1600640 1 -- 172.21.15.2:0/836548771 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c010760 con 0x7f62a4109120 2024-09-10T08:19:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.145+0000 7f62aae00640 1 -- 172.21.15.2:0/836548771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a419f960 con 0x7f62a4109120 2024-09-10T08:19:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.145+0000 7f62aae00640 1 -- 172.21.15.2:0/836548771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a41a7240 con 0x7f62a4109120 2024-09-10T08:19:59.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.149+0000 7f62a1600640 1 -- 172.21.15.2:0/836548771 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f629c005020 con 0x7f62a4109120 2024-09-10T08:19:59.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.149+0000 7f62ad89b740 1 Processor -- start 2024-09-10T08:19:59.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.149+0000 7f62ad89b740 1 -- start start 2024-09-10T08:19:59.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.149+0000 7f62ad89b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f1fb90 0x5606b9f1ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.149+0000 7f62ad89b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f204b0 0x5606b9f2a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.149+0000 7f62ad89b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f2ad70 0x5606b9f2d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.149+0000 7f62ad89b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5606b9e863e0 con 0x5606b9f2ad70 2024-09-10T08:19:59.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.149+0000 7f62ad89b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5606b9e2dd80 con 0x5606b9f204b0 2024-09-10T08:19:59.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.149+0000 7f62ad89b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5606b9e823a0 con 0x5606b9f1fb90 2024-09-10T08:19:59.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.150+0000 7f62a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f204b0 0x5606b9f2a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.150+0000 7f629be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f1fb90 0x5606b9f1ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.150+0000 7f62a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f204b0 0x5606b9f2a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40680/0 (socket says 172.21.15.2:40680) 2024-09-10T08:19:59.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.150+0000 7f62a3e00640 1 -- 172.21.15.2:0/2805539170 learned_addr learned my addr 172.21.15.2:0/2805539170 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:59.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.150+0000 7f629be00640 1 --2- 172.21.15.2:0/2805539170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f2ad70 0x5606b9f2d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.150+0000 7f62a3e00640 1 -- 172.21.15.2:0/2805539170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f1fb90 msgr2=0x5606b9f1ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:59.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.150+0000 7f62a3e00640 1 --2- 172.21.15.2:0/2805539170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f1fb90 0x5606b9f1ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.150+0000 7f62a3e00640 1 -- 172.21.15.2:0/2805539170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f2ad70 msgr2=0x5606b9f2d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:59.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.150+0000 7f62a3e00640 1 --2- 172.21.15.2:0/2805539170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f2ad70 0x5606b9f2d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.150+0000 7f62a3e00640 1 -- 172.21.15.2:0/2805539170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606b9e4cae0 con 0x5606b9f204b0 2024-09-10T08:19:59.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.150+0000 7f629be00640 1 --2- 172.21.15.2:0/2805539170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f2ad70 0x5606b9f2d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:19:59.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.150+0000 7f62a1600640 1 --2- 172.21.15.2:0/836548771 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62880a2b80 0x7f62880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.150+0000 7f62a3e00640 1 --2- 172.21.15.2:0/2805539170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f204b0 0x5606b9f2a830 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6294009cf0 tx=0x7f6294011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45c2099ef504731d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:59.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.151+0000 7f62a1600640 1 -- 172.21.15.2:0/836548771 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f629c0cf980 con 0x7f62a4109120 2024-09-10T08:19:59.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.151+0000 7f629be00640 1 --2- 172.21.15.2:0/836548771 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62880a2b80 0x7f62880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.151+0000 7f6299600640 1 -- 172.21.15.2:0/2805539170 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629401e070 con 0x5606b9f204b0 2024-09-10T08:19:59.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.151+0000 7f6299600640 1 -- 172.21.15.2:0/2805539170 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6294019b10 con 0x5606b9f204b0 2024-09-10T08:19:59.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.151+0000 7f6299600640 1 -- 172.21.15.2:0/2805539170 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629400ea60 con 0x5606b9f204b0 2024-09-10T08:19:59.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.152+0000 7f62ad89b740 1 -- 172.21.15.2:0/2805539170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f204b0 msgr2=0x5606b9f2a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:59.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.152+0000 7f62ad89b740 1 --2- 172.21.15.2:0/2805539170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f204b0 0x5606b9f2a830 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6294009cf0 tx=0x7f6294011810 comp rx=0 tx=0).stop 2024-09-10T08:19:59.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.152+0000 7f629be00640 1 --2- 172.21.15.2:0/836548771 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62880a2b80 0x7f62880a5040 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f628c009230 tx=0x7f628c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:59.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.152+0000 7f62ad89b740 1 -- 172.21.15.2:0/2805539170 shutdown_connections 2024-09-10T08:19:59.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.152+0000 7f62ad89b740 1 --2- 172.21.15.2:0/2805539170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f2ad70 0x5606b9f2d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.152+0000 7f62ad89b740 1 --2- 172.21.15.2:0/2805539170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f204b0 0x5606b9f2a830 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.152+0000 7f62ad89b740 1 --2- 172.21.15.2:0/2805539170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f1fb90 0x5606b9f1ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.152+0000 7f62ad89b740 1 -- 172.21.15.2:0/2805539170 >> 172.21.15.2:0/2805539170 conn(0x5606b9f1c120 msgr2=0x5606b9f1c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:59.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.152+0000 7f62ad89b740 1 -- 172.21.15.2:0/2805539170 shutdown_connections 2024-09-10T08:19:59.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.152+0000 7f62ad89b740 1 -- 172.21.15.2:0/2805539170 wait complete. 2024-09-10T08:19:59.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.152+0000 7f62ad89b740 1 -- start start 2024-09-10T08:19:59.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.152+0000 7f62ad89b740 1 Processor -- start 2024-09-10T08:19:59.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.153+0000 7f62ad89b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f1fb90 0x5606b9f27c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.153+0000 7f62ad89b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 0x5606b9f28170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.153+0000 7f62ad89b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f2ad70 0x5606b9f286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.153+0000 7f62ad89b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5606b9f2d9e0 con 0x5606b9f204b0 2024-09-10T08:19:59.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.153+0000 7f62ad89b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5606b9e410f0 con 0x5606b9f2ad70 2024-09-10T08:19:59.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.153+0000 7f62ad89b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5606b9f2db60 con 0x5606b9f1fb90 2024-09-10T08:19:59.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.153+0000 7f62ad89b740 10 client.0 _ll_register_callbacks cb 0x5606b9e329e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:59.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.153+0000 7f629be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 0x5606b9f28170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.153+0000 7f629be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 0x5606b9f28170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34048/0 (socket says 172.21.15.2:34048) 2024-09-10T08:19:59.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.153+0000 7f629be00640 1 -- 172.21.15.2:0/3985572376 learned_addr learned my addr 172.21.15.2:0/3985572376 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:59.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.153+0000 7f62a3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f1fb90 0x5606b9f27c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.153+0000 7f62a8e00640 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f2ad70 0x5606b9f286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.154+0000 7f62a3e00640 1 -- 172.21.15.2:0/3985572376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f2ad70 msgr2=0x5606b9f286b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:59.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.154+0000 7f62a3e00640 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f2ad70 0x5606b9f286b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.154+0000 7f62a3e00640 1 -- 172.21.15.2:0/3985572376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 msgr2=0x5606b9f28170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:59.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.154+0000 7f62a3e00640 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 0x5606b9f28170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.154+0000 7f62a3e00640 1 -- 172.21.15.2:0/3985572376 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606b9bf73c0 con 0x5606b9f1fb90 2024-09-10T08:19:59.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.154+0000 7f629be00640 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 0x5606b9f28170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:59.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.154+0000 7f62a8e00640 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f2ad70 0x5606b9f286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:59.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.154+0000 7f62a3e00640 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f1fb90 0x5606b9f27c30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f629401c040 tx=0x7f6294019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d6ce32699de29f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:59.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.154+0000 7f6292000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:19:59.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.154+0000 7f6292000640 1 -- 172.21.15.2:0/3985572376 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629401e050 con 0x5606b9f1fb90 2024-09-10T08:19:59.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.154+0000 7f62ad89b740 1 -- 172.21.15.2:0/3985572376 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6294019b10 con 0x5606b9f1fb90 2024-09-10T08:19:59.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.154+0000 7f6292000640 1 -- 172.21.15.2:0/3985572376 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6294005240 con 0x5606b9f1fb90 2024-09-10T08:19:59.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.155+0000 7f6292000640 1 -- 172.21.15.2:0/3985572376 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6294033050 con 0x5606b9f1fb90 2024-09-10T08:19:59.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.155+0000 7f62ad89b740 10 client.28696 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:59.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.155+0000 7f62ad89b740 1 -- 172.21.15.2:0/3985572376 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5606b9f39660 con 0x5606b9f1fb90 2024-09-10T08:19:59.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.155+0000 7f6292000640 1 -- 172.21.15.2:0/3985572376 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6294019b10 con 0x5606b9f1fb90 2024-09-10T08:19:59.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.155+0000 7f6292000640 1 -- 172.21.15.2:0/3985572376 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f62940046c0 con 0x5606b9f1fb90 2024-09-10T08:19:59.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.155+0000 7f62ad89b740 10 client.28696 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:59.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.155+0000 7f62ad89b740 10 client.28696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:59.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.155+0000 7f62ad89b740 20 client.28696 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:59.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.155+0000 7f62ad89b740 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5606b9f3aac0 0x5606b9f3ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.155+0000 7f62ad89b740 4 client.28696 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:19:59.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.155+0000 7f62ad89b740 1 -- 172.21.15.2:0/3985572376 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606b9e93950 con 0x5606b9f3aac0 2024-09-10T08:19:59.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.156+0000 7f629be00640 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5606b9f3aac0 0x5606b9f3ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.156+0000 7f629be00640 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5606b9f3aac0 0x5606b9f3ceb0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990c4d2b090db7f6 server_cookie=ea3802f45db7ba90 in_seq=0 out_seq=0 2024-09-10T08:19:59.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.156+0000 7f6292000640 10 client.28696 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:59.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.157+0000 7f6292000640 1 -- 172.21.15.2:0/3985572376 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606b9e93950 con 0x5606b9f3aac0 2024-09-10T08:19:59.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.157+0000 7f6292000640 10 client.28696 handle_command_reply: tid=0 2024-09-10T08:19:59.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.157+0000 7f62ad89b740 1 client.28696 shutdown 2024-09-10T08:19:59.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.157+0000 7f62ad89b740 1 -- 172.21.15.2:0/3985572376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f1fb90 msgr2=0x5606b9f27c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:59.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.157+0000 7f62ad89b740 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f1fb90 0x5606b9f27c30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f629401c040 tx=0x7f6294019660 comp rx=0 tx=0).stop 2024-09-10T08:19:59.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.157+0000 7f62ad89b740 1 -- 172.21.15.2:0/3985572376 shutdown_connections 2024-09-10T08:19:59.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.157+0000 7f62ad89b740 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5606b9f3aac0 0x5606b9f3ceb0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.157+0000 7f62ad89b740 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f2ad70 0x5606b9f286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.157+0000 7f62ad89b740 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 0x5606b9f28170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.157+0000 7f6292000640 0 client.28696 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:59.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.157+0000 7f62ad89b740 1 --2- 172.21.15.2:0/3985572376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f1fb90 0x5606b9f27c30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.157+0000 7f62ad89b740 1 -- 172.21.15.2:0/3985572376 >> 172.21.15.2:0/3985572376 conn(0x5606b9f1c120 msgr2=0x5606b9f21e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:59.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.157+0000 7f62ad89b740 1 -- 172.21.15.2:0/3985572376 shutdown_connections 2024-09-10T08:19:59.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.158+0000 7f62ad89b740 1 -- 172.21.15.2:0/3985572376 wait complete. 2024-09-10T08:19:59.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.158+0000 7f62ad89b740 20 client.28696 trim_cache size 0 max 16384 2024-09-10T08:19:59.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.176+0000 7f62ad89b740 1 Processor -- start 2024-09-10T08:19:59.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.176+0000 7f62ad89b740 1 -- start start 2024-09-10T08:19:59.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.176+0000 7f62ad89b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f1fb90 0x5606b9f426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.176+0000 7f62ad89b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 0x5606b9f42c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.176+0000 7f62ad89b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f2ad70 0x5606b9f1a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.176+0000 7f62ad89b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f62a41b3940 con 0x5606b9f204b0 2024-09-10T08:19:59.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.176+0000 7f62ad89b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f62a41b1210 con 0x5606b9f1fb90 2024-09-10T08:19:59.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.176+0000 7f62ad89b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62a41af060 con 0x5606b9f2ad70 2024-09-10T08:19:59.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.176+0000 7f629be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 0x5606b9f42c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.176+0000 7f62a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f1fb90 0x5606b9f426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.176+0000 7f629be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 0x5606b9f42c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34060/0 (socket says 172.21.15.2:34060) 2024-09-10T08:19:59.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.176+0000 7f629be00640 1 -- 172.21.15.2:0/497873156 learned_addr learned my addr 172.21.15.2:0/497873156 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:59.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.177+0000 7f62a8e00640 1 --2- 172.21.15.2:0/497873156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f2ad70 0x5606b9f1a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.177+0000 7f62a8e00640 1 -- 172.21.15.2:0/497873156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f1fb90 msgr2=0x5606b9f426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:59.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.177+0000 7f62a8e00640 1 --2- 172.21.15.2:0/497873156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f1fb90 0x5606b9f426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.177+0000 7f62a8e00640 1 -- 172.21.15.2:0/497873156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 msgr2=0x5606b9f42c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:59.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.177+0000 7f62a8e00640 1 --2- 172.21.15.2:0/497873156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 0x5606b9f42c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.177+0000 7f62a8e00640 1 -- 172.21.15.2:0/497873156 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606b9f22a00 con 0x5606b9f2ad70 2024-09-10T08:19:59.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.177+0000 7f629be00640 1 --2- 172.21.15.2:0/497873156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 0x5606b9f42c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:59.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.177+0000 7f62a3e00640 1 --2- 172.21.15.2:0/497873156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f1fb90 0x5606b9f426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:59.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.177+0000 7f62a8e00640 1 --2- 172.21.15.2:0/497873156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f2ad70 0x5606b9f1a9a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f629c0cd030 tx=0x7f629c09b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84aac00bb9a3b973 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:59.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.178+0000 7f6298c00640 1 -- 172.21.15.2:0/497873156 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c09bd90 con 0x5606b9f2ad70 2024-09-10T08:19:59.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.178+0000 7f6298c00640 1 -- 172.21.15.2:0/497873156 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f629c0a0140 con 0x5606b9f2ad70 2024-09-10T08:19:59.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.178+0000 7f6298c00640 1 -- 172.21.15.2:0/497873156 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c005570 con 0x5606b9f2ad70 2024-09-10T08:19:59.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.178+0000 7f62ad89b740 1 -- 172.21.15.2:0/497873156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f2ad70 msgr2=0x5606b9f1a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:59.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.178+0000 7f62ad89b740 1 --2- 172.21.15.2:0/497873156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f2ad70 0x5606b9f1a9a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f629c0cd030 tx=0x7f629c09b8d0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.178+0000 7f62ad89b740 1 -- 172.21.15.2:0/497873156 shutdown_connections 2024-09-10T08:19:59.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.178+0000 7f62ad89b740 1 --2- 172.21.15.2:0/497873156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f2ad70 0x5606b9f1a9a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.178+0000 7f62ad89b740 1 --2- 172.21.15.2:0/497873156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f204b0 0x5606b9f42c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.178+0000 7f62ad89b740 1 --2- 172.21.15.2:0/497873156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f1fb90 0x5606b9f426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.178+0000 7f62ad89b740 1 -- 172.21.15.2:0/497873156 >> 172.21.15.2:0/497873156 conn(0x5606b9f1c120 msgr2=0x5606b9f29cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:59.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.178+0000 7f62ad89b740 1 -- 172.21.15.2:0/497873156 shutdown_connections 2024-09-10T08:19:59.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.178+0000 7f62ad89b740 1 -- 172.21.15.2:0/497873156 wait complete. 2024-09-10T08:19:59.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.179+0000 7f62ad89b740 1 -- start start 2024-09-10T08:19:59.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.179+0000 7f62ad89b740 1 Processor -- start 2024-09-10T08:19:59.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.179+0000 7f62ad89b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f1fb90 0x5606b9f2e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.179+0000 7f62ad89b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f204b0 0x5606b9f2e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.179+0000 7f62ad89b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f2ad70 0x5606b9fa3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.179+0000 7f62ad89b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f62a41af6d0 con 0x5606b9f1fb90 2024-09-10T08:19:59.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.179+0000 7f62ad89b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f62a41b0b40 con 0x5606b9f204b0 2024-09-10T08:19:59.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.179+0000 7f62ad89b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62a41afda0 con 0x5606b9f2ad70 2024-09-10T08:19:59.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.179+0000 7f62ad89b740 10 client.0 _ll_register_callbacks cb 0x7f6294005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:19:59.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f62a3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f1fb90 0x5606b9f2e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f62a8e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f2ad70 0x5606b9fa3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f629be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f204b0 0x5606b9f2e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f62a3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f1fb90 0x5606b9f2e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34074/0 (socket says 172.21.15.2:34074) 2024-09-10T08:19:59.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f62a3e00640 1 -- 172.21.15.2:0/4022588060 learned_addr learned my addr 172.21.15.2:0/4022588060 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:19:59.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f62a8e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f2ad70 0x5606b9fa3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49856/0 (socket says 172.21.15.2:49856) 2024-09-10T08:19:59.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f629be00640 1 -- 172.21.15.2:0/4022588060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f2ad70 msgr2=0x5606b9fa3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:59.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f629be00640 1 --2- 172.21.15.2:0/4022588060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f2ad70 0x5606b9fa3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f629be00640 1 -- 172.21.15.2:0/4022588060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f1fb90 msgr2=0x5606b9f2e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:19:59.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f629be00640 1 --2- 172.21.15.2:0/4022588060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f1fb90 0x5606b9f2e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f629be00640 1 -- 172.21.15.2:0/4022588060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606b9fa3bc0 con 0x5606b9f204b0 2024-09-10T08:19:59.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f62a3e00640 1 --2- 172.21.15.2:0/4022588060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f1fb90 0x5606b9f2e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:59.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f62a8e00640 1 --2- 172.21.15.2:0/4022588060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f2ad70 0x5606b9fa3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:19:59.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f629be00640 1 --2- 172.21.15.2:0/4022588060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f204b0 0x5606b9f2e670 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x5606b9f37aa0 tx=0x7f628c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=705229b6831a2db6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:19:59.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.180+0000 7f6292000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:19:59.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f6292000640 1 -- 172.21.15.2:0/4022588060 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c05b060 con 0x5606b9f204b0 2024-09-10T08:19:59.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f62ad89b740 1 -- 172.21.15.2:0/4022588060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f629c0a0140 con 0x5606b9f204b0 2024-09-10T08:19:59.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f6292000640 1 -- 172.21.15.2:0/4022588060 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f628c003c40 con 0x5606b9f204b0 2024-09-10T08:19:59.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f6292000640 1 -- 172.21.15.2:0/4022588060 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f628c062050 con 0x5606b9f204b0 2024-09-10T08:19:59.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f62ad89b740 10 client.28769 fetch_fsmap learned FSMap version 26 2024-09-10T08:19:59.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f62ad89b740 1 -- 172.21.15.2:0/4022588060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5606b9fa4da0 con 0x5606b9f204b0 2024-09-10T08:19:59.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f6292000640 1 -- 172.21.15.2:0/4022588060 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f629c0a0140 con 0x5606b9f204b0 2024-09-10T08:19:59.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f6292000640 1 -- 172.21.15.2:0/4022588060 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f628c04c420 con 0x5606b9f204b0 2024-09-10T08:19:59.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f62ad89b740 10 client.28769 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:19:59.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f62ad89b740 10 client.28769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:19:59.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f62ad89b740 20 client.28769 populate_metadata read hostname 'smithi002' 2024-09-10T08:19:59.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f62ad89b740 1 --2- 172.21.15.2:0/4022588060 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5606b9fa5500 0x5606b9fa7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:19:59.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f62ad89b740 4 client.28769 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:19:59.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.181+0000 7f62ad89b740 1 -- 172.21.15.2:0/4022588060 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5606b9e8de60 con 0x5606b9fa5500 2024-09-10T08:19:59.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.182+0000 7f62a3e00640 1 --2- 172.21.15.2:0/4022588060 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5606b9fa5500 0x5606b9fa7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:19:59.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.182+0000 7f62a3e00640 1 --2- 172.21.15.2:0/4022588060 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5606b9fa5500 0x5606b9fa7920 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d697c7d6b32635c2 server_cookie=fd2a987bfa5b5daa in_seq=0 out_seq=0 2024-09-10T08:19:59.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.182+0000 7f6292000640 10 client.28769 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:59.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.182+0000 7f6292000640 1 -- 172.21.15.2:0/4022588060 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5606b9e8de60 con 0x5606b9fa5500 2024-09-10T08:19:59.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.182+0000 7f6292000640 10 client.28769 handle_command_reply: tid=0 2024-09-10T08:19:59.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.182+0000 7f62ad89b740 1 client.28769 shutdown 2024-09-10T08:19:59.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.183+0000 7f62ad89b740 1 -- 172.21.15.2:0/4022588060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f204b0 msgr2=0x5606b9f2e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:59.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.183+0000 7f62ad89b740 1 --2- 172.21.15.2:0/4022588060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f204b0 0x5606b9f2e670 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x5606b9f37aa0 tx=0x7f628c000bc0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.183+0000 7f62ad89b740 1 -- 172.21.15.2:0/4022588060 shutdown_connections 2024-09-10T08:19:59.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.183+0000 7f62ad89b740 1 --2- 172.21.15.2:0/4022588060 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5606b9fa5500 0x5606b9fa7920 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.183+0000 7f62ad89b740 1 --2- 172.21.15.2:0/4022588060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5606b9f2ad70 0x5606b9fa3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.183+0000 7f62ad89b740 1 --2- 172.21.15.2:0/4022588060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606b9f204b0 0x5606b9f2e670 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.183+0000 7f6292000640 0 client.28769 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:19:59.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.183+0000 7f62ad89b740 1 --2- 172.21.15.2:0/4022588060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606b9f1fb90 0x5606b9f2e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.183+0000 7f62ad89b740 1 -- 172.21.15.2:0/4022588060 >> 172.21.15.2:0/4022588060 conn(0x5606b9f1c120 msgr2=0x5606b9f27b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:59.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.183+0000 7f62ad89b740 1 -- 172.21.15.2:0/4022588060 shutdown_connections 2024-09-10T08:19:59.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.183+0000 7f62ad89b740 1 -- 172.21.15.2:0/4022588060 wait complete. 2024-09-10T08:19:59.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.183+0000 7f62ad89b740 20 client.28769 trim_cache size 0 max 16384 2024-09-10T08:19:59.188 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:19:59.188 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:19:59.188 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:19:59.188 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:19:59.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.184+0000 7f62aae00640 1 -- 172.21.15.2:0/836548771 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62880a2b80 msgr2=0x7f62880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:59.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.184+0000 7f62aae00640 1 --2- 172.21.15.2:0/836548771 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62880a2b80 0x7f62880a5040 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f628c009230 tx=0x7f628c0058d0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.184+0000 7f62aae00640 1 -- 172.21.15.2:0/836548771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62a4109120 msgr2=0x7f62a419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:19:59.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.184+0000 7f62aae00640 1 --2- 172.21.15.2:0/836548771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62a4109120 0x7f62a419ef90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f629c00eb10 tx=0x7f629c00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.184+0000 7f62aae00640 1 -- 172.21.15.2:0/836548771 shutdown_connections 2024-09-10T08:19:59.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.184+0000 7f62aae00640 1 --2- 172.21.15.2:0/836548771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62a4109120 0x7f62a419ef90 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.184+0000 7f62aae00640 1 --2- 172.21.15.2:0/836548771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62a41049a0 0x7f62a419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.184+0000 7f62aae00640 1 --2- 172.21.15.2:0/836548771 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62880a2b80 0x7f62880a5040 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.184+0000 7f62aae00640 1 --2- 172.21.15.2:0/836548771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62a4103ff0 0x7f62a419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:19:59.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.184+0000 7f62aae00640 1 -- 172.21.15.2:0/836548771 >> 172.21.15.2:0/836548771 conn(0x7f62a40ffa90 msgr2=0x7f62a4100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:19:59.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.184+0000 7f62aae00640 1 -- 172.21.15.2:0/836548771 shutdown_connections 2024-09-10T08:19:59.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:19:59.184+0000 7f62aae00640 1 -- 172.21.15.2:0/836548771 wait complete. 2024-09-10T08:19:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:19:59 smithi155 ceph-mon[35405]: pgmap v742: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-10T08:19:59.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:19:59 smithi107 ceph-mon[35048]: pgmap v742: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-10T08:20:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:19:59 smithi002 ceph-mon[30563]: pgmap v742: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-10T08:20:00.125 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:20:00.126 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:20:00.511 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:00.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:00 smithi002 ceph-mon[30563]: overall HEALTH_OK 2024-09-10T08:20:00.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:00 smithi155 ceph-mon[35405]: overall HEALTH_OK 2024-09-10T08:20:00.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:00 smithi107 ceph-mon[35048]: overall HEALTH_OK 2024-09-10T08:20:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.106+0000 7fecf0c00640 1 -- 172.21.15.2:0/2034006453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecec10c590 msgr2=0x7fecec10c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.106+0000 7fecf0c00640 1 --2- 172.21.15.2:0/2034006453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecec10c590 0x7fecec10c9f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009900 tx=0x7fecd402a410 comp rx=0 tx=0).stop 2024-09-10T08:20:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.106+0000 7fecf0c00640 1 -- 172.21.15.2:0/2034006453 shutdown_connections 2024-09-10T08:20:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.106+0000 7fecf0c00640 1 --2- 172.21.15.2:0/2034006453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecec10cf30 0x7fecec119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.106+0000 7fecf0c00640 1 --2- 172.21.15.2:0/2034006453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecec10c590 0x7fecec10c9f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.106+0000 7fecf0c00640 1 --2- 172.21.15.2:0/2034006453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecec112640 0x7fecec112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.106+0000 7fecf0c00640 1 -- 172.21.15.2:0/2034006453 >> 172.21.15.2:0/2034006453 conn(0x7fecec06cdc0 msgr2=0x7fecec06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.106+0000 7fecf0c00640 1 -- 172.21.15.2:0/2034006453 shutdown_connections 2024-09-10T08:20:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.106+0000 7fecf0c00640 1 -- 172.21.15.2:0/2034006453 wait complete. 2024-09-10T08:20:01.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.107+0000 7fecf0c00640 1 Processor -- start 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.108+0000 7fecf0c00640 1 -- start start 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.108+0000 7fecf0c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecec10c590 0x7fecec1a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.108+0000 7fecf0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecec10cf30 0x7fecec1a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.108+0000 7fecf0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecec112640 0x7fecec1a7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.108+0000 7fecf0c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fecec11c9b0 con 0x7fecec10c590 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.108+0000 7fecf0c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fecec11c830 con 0x7fecec10cf30 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.108+0000 7fecf0c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fecec11cb30 con 0x7fecec112640 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.108+0000 7fece9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecec10c590 0x7fecec1a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.108+0000 7fece9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecec10c590 0x7fecec1a2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34106/0 (socket says 172.21.15.2:34106) 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.108+0000 7fece9e00640 1 -- 172.21.15.2:0/3500099635 learned_addr learned my addr 172.21.15.2:0/3500099635 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.108+0000 7fece3e00640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecec10cf30 0x7fecec1a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.108+0000 7fecea800640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecec112640 0x7fecec1a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.109+0000 7fece9e00640 1 -- 172.21.15.2:0/3500099635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecec112640 msgr2=0x7fecec1a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.109+0000 7fece9e00640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecec112640 0x7fecec1a7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.109+0000 7fece9e00640 1 -- 172.21.15.2:0/3500099635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecec10cf30 msgr2=0x7fecec1a32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.109+0000 7fece9e00640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecec10cf30 0x7fecec1a32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.109+0000 7fece9e00640 1 -- 172.21.15.2:0/3500099635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec1a7d40 con 0x7fecec10c590 2024-09-10T08:20:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.109+0000 7fece9e00640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecec10c590 0x7fecec1a2d90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00e9f0 tx=0x7fecdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.109+0000 7fece2a00640 1 -- 172.21.15.2:0/3500099635 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc00cd80 con 0x7fecec10c590 2024-09-10T08:20:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.109+0000 7fece2a00640 1 -- 172.21.15.2:0/3500099635 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecdc010910 con 0x7fecec10c590 2024-09-10T08:20:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.109+0000 7fece2a00640 1 -- 172.21.15.2:0/3500099635 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc018990 con 0x7fecec10c590 2024-09-10T08:20:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.110+0000 7fecf0c00640 1 -- 172.21.15.2:0/3500099635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecec1a8030 con 0x7fecec10c590 2024-09-10T08:20:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.110+0000 7fecf0c00640 1 -- 172.21.15.2:0/3500099635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecec1af910 con 0x7fecec10c590 2024-09-10T08:20:01.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.111+0000 7fecf0c00640 1 -- 172.21.15.2:0/3500099635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecec10dcb0 con 0x7fecec10c590 2024-09-10T08:20:01.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.113+0000 7fece2a00640 1 -- 172.21.15.2:0/3500099635 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fecdc010ac0 con 0x7fecec10c590 2024-09-10T08:20:01.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.114+0000 7fece2a00640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fecd00a2c00 0x7fecd00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:01.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.114+0000 7fece2a00640 1 -- 172.21.15.2:0/3500099635 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fecdc0cfb60 con 0x7fecec10c590 2024-09-10T08:20:01.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.115+0000 7fece3e00640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fecd00a2c00 0x7fecd00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:01.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.115+0000 7fece3e00640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fecd00a2c00 0x7fecd00a50c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fecd4000c00 tx=0x7fecd4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:01.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.117+0000 7fece2a00640 1 -- 172.21.15.2:0/3500099635 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fecdc09c020 con 0x7fecec10c590 2024-09-10T08:20:01.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.337+0000 7fecf0c00640 1 -- 172.21.15.2:0/3500099635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecec10c9f0 con 0x7fecec10c590 2024-09-10T08:20:01.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.337+0000 7fece2a00640 1 -- 172.21.15.2:0/3500099635 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fecdc0a0320 con 0x7fecec10c590 2024-09-10T08:20:01.338 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:20:01.338 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:20:01.338 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:20:01.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.340+0000 7fecf0c00640 1 -- 172.21.15.2:0/3500099635 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fecd00a2c00 msgr2=0x7fecd00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:01.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.340+0000 7fecf0c00640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fecd00a2c00 0x7fecd00a50c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fecd4000c00 tx=0x7fecd4005b50 comp rx=0 tx=0).stop 2024-09-10T08:20:01.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.340+0000 7fecf0c00640 1 -- 172.21.15.2:0/3500099635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecec10c590 msgr2=0x7fecec1a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:01.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.340+0000 7fecf0c00640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecec10c590 0x7fecec1a2d90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00e9f0 tx=0x7fecdc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:20:01.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.340+0000 7fecf0c00640 1 -- 172.21.15.2:0/3500099635 shutdown_connections 2024-09-10T08:20:01.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.340+0000 7fecf0c00640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fecd00a2c00 0x7fecd00a50c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:01.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.340+0000 7fecf0c00640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecec112640 0x7fecec1a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:01.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.340+0000 7fecf0c00640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecec10cf30 0x7fecec1a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:01.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.340+0000 7fecf0c00640 1 --2- 172.21.15.2:0/3500099635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecec10c590 0x7fecec1a2d90 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:01.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.340+0000 7fecf0c00640 1 -- 172.21.15.2:0/3500099635 >> 172.21.15.2:0/3500099635 conn(0x7fecec06cdc0 msgr2=0x7fecec10ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:01.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.340+0000 7fecf0c00640 1 -- 172.21.15.2:0/3500099635 shutdown_connections 2024-09-10T08:20:01.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:01.340+0000 7fecf0c00640 1 -- 172.21.15.2:0/3500099635 wait complete. 2024-09-10T08:20:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:01 smithi002 ceph-mon[30563]: pgmap v743: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:20:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:01 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3500099635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:20:01.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:01 smithi155 ceph-mon[35405]: pgmap v743: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:20:01.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:01 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3500099635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:20:01.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:01 smithi107 ceph-mon[35048]: pgmap v743: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:20:01.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:01 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3500099635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:20:02.362 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:20:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.326+0000 7f0eb2600640 1 -- 172.21.15.2:0/2010913193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0eac103e50 msgr2=0x7f0eac104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.326+0000 7f0eb2600640 1 --2- 172.21.15.2:0/2010913193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0eac103e50 0x7f0eac104230 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009860 tx=0x7f0ea002a340 comp rx=0 tx=0).stop 2024-09-10T08:20:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.327+0000 7f0eb2600640 1 -- 172.21.15.2:0/2010913193 shutdown_connections 2024-09-10T08:20:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.327+0000 7f0eb2600640 1 --2- 172.21.15.2:0/2010913193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0eac109090 0x7f0eac110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.327+0000 7f0eb2600640 1 --2- 172.21.15.2:0/2010913193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eac104800 0x7f0eac108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.327+0000 7f0eb2600640 1 --2- 172.21.15.2:0/2010913193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0eac103e50 0x7f0eac104230 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.327+0000 7f0eb2600640 1 -- 172.21.15.2:0/2010913193 >> 172.21.15.2:0/2010913193 conn(0x7f0eac0ff8b0 msgr2=0x7f0eac101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.327+0000 7f0eb2600640 1 -- 172.21.15.2:0/2010913193 shutdown_connections 2024-09-10T08:20:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.327+0000 7f0eb2600640 1 -- 172.21.15.2:0/2010913193 wait complete. 2024-09-10T08:20:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.328+0000 7f0eb2600640 1 Processor -- start 2024-09-10T08:20:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.328+0000 7f0eb2600640 1 -- start start 2024-09-10T08:20:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.329+0000 7f0eb2600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eac103e50 0x7f0eac19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.329+0000 7f0eb2600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0eac104800 0x7f0eac19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.329+0000 7f0eb2600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0eac109090 0x7f0eac19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.329+0000 7f0eb2600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1140b0 con 0x7f0eac109090 2024-09-10T08:20:03.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.329+0000 7f0eb2600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac113f30 con 0x7f0eac103e50 2024-09-10T08:20:03.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.329+0000 7f0eb2600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac114230 con 0x7f0eac104800 2024-09-10T08:20:03.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.329+0000 7f0eb1c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0eac109090 0x7f0eac19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.329+0000 7f0eb1c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0eac109090 0x7f0eac19ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34126/0 (socket says 172.21.15.2:34126) 2024-09-10T08:20:03.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.329+0000 7f0eb1c00640 1 -- 172.21.15.2:0/3846893747 learned_addr learned my addr 172.21.15.2:0/3846893747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:03.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.329+0000 7f0eb1200640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eac103e50 0x7f0eac19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.329+0000 7f0eabe00640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0eac104800 0x7f0eac19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.330+0000 7f0eb1200640 1 -- 172.21.15.2:0/3846893747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0eac104800 msgr2=0x7f0eac19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:03.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.330+0000 7f0eb1200640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0eac104800 0x7f0eac19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.330+0000 7f0eb1200640 1 -- 172.21.15.2:0/3846893747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0eac109090 msgr2=0x7f0eac19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.330+0000 7f0eb1200640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0eac109090 0x7f0eac19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.330+0000 7f0eb1200640 1 -- 172.21.15.2:0/3846893747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac19f520 con 0x7f0eac103e50 2024-09-10T08:20:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.330+0000 7f0eb1c00640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0eac109090 0x7f0eac19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.330+0000 7f0eb1200640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eac103e50 0x7f0eac19a570 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0ea002a860 tx=0x7f0ea0033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.330+0000 7f0ea9600640 1 -- 172.21.15.2:0/3846893747 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0037040 con 0x7f0eac103e50 2024-09-10T08:20:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.330+0000 7f0eb2600640 1 -- 172.21.15.2:0/3846893747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eac19f7b0 con 0x7f0eac103e50 2024-09-10T08:20:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.330+0000 7f0ea9600640 1 -- 172.21.15.2:0/3846893747 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea0033d50 con 0x7f0eac103e50 2024-09-10T08:20:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.330+0000 7f0ea9600640 1 -- 172.21.15.2:0/3846893747 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0003960 con 0x7f0eac103e50 2024-09-10T08:20:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.330+0000 7f0eb2600640 1 -- 172.21.15.2:0/3846893747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eac1a70f0 con 0x7f0eac103e50 2024-09-10T08:20:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.333+0000 7f0eb65de740 1 Processor -- start 2024-09-10T08:20:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.333+0000 7f0eb65de740 1 -- start start 2024-09-10T08:20:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.334+0000 7f0ea9600640 1 -- 172.21.15.2:0/3846893747 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0ea0042020 con 0x7f0eac103e50 2024-09-10T08:20:03.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.334+0000 7f0eb65de740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699ce40 0x5562d699d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.334+0000 7f0eb65de740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699d760 0x5562d69a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.334+0000 7f0eb65de740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d69a8020 0x5562d69aa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.334+0000 7f0eb65de740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5562d69033e0 con 0x5562d699d760 2024-09-10T08:20:03.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.334+0000 7f0eb65de740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5562d68aad80 con 0x5562d69a8020 2024-09-10T08:20:03.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.334+0000 7f0eb65de740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5562d68ff3a0 con 0x5562d699ce40 2024-09-10T08:20:03.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.334+0000 7f0eb1200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699d760 0x5562d69a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.334+0000 7f0eb1200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699d760 0x5562d69a7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34138/0 (socket says 172.21.15.2:34138) 2024-09-10T08:20:03.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.334+0000 7f0eb1200640 1 -- 172.21.15.2:0/881181307 learned_addr learned my addr 172.21.15.2:0/881181307 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:03.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.334+0000 7f0eabe00640 1 --2- 172.21.15.2:0/881181307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d69a8020 0x5562d69aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.334+0000 7f0eb1c00640 1 --2- 172.21.15.2:0/881181307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699ce40 0x5562d699d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.334+0000 7f0eb1200640 1 -- 172.21.15.2:0/881181307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699ce40 msgr2=0x5562d699d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.335+0000 7f0eb1200640 1 --2- 172.21.15.2:0/881181307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699ce40 0x5562d699d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.335+0000 7f0eb1200640 1 -- 172.21.15.2:0/881181307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d69a8020 msgr2=0x5562d69aa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.335+0000 7f0eb1200640 1 --2- 172.21.15.2:0/881181307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d69a8020 0x5562d69aa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.335+0000 7f0eb1200640 1 -- 172.21.15.2:0/881181307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562d68c9ae0 con 0x5562d699d760 2024-09-10T08:20:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.335+0000 7f0eb1200640 1 --2- 172.21.15.2:0/881181307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699d760 0x5562d69a7ae0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00f49e0 tx=0x7f0ea00f4ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860d43f2099ccf00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.335+0000 7f0e9e000640 1 -- 172.21.15.2:0/881181307 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00fd070 con 0x5562d699d760 2024-09-10T08:20:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.335+0000 7f0e9e000640 1 -- 172.21.15.2:0/881181307 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea00f9420 con 0x5562d699d760 2024-09-10T08:20:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.335+0000 7f0e9e000640 1 -- 172.21.15.2:0/881181307 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00f85e0 con 0x5562d699d760 2024-09-10T08:20:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.336+0000 7f0ea9600640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0e940a2b80 0x7f0e940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.336+0000 7f0eb65de740 1 -- 172.21.15.2:0/881181307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699d760 msgr2=0x5562d69a7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.336+0000 7f0eb65de740 1 --2- 172.21.15.2:0/881181307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699d760 0x5562d69a7ae0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00f49e0 tx=0x7f0ea00f4ad0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.336+0000 7f0eb1c00640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0e940a2b80 0x7f0e940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.336+0000 7f0ea9600640 1 -- 172.21.15.2:0/3846893747 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0ea00ee9e0 con 0x7f0eac103e50 2024-09-10T08:20:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.336+0000 7f0eb65de740 1 -- 172.21.15.2:0/881181307 shutdown_connections 2024-09-10T08:20:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.336+0000 7f0eb65de740 1 --2- 172.21.15.2:0/881181307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d69a8020 0x5562d69aa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.336+0000 7f0eb65de740 1 --2- 172.21.15.2:0/881181307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699d760 0x5562d69a7ae0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.336+0000 7f0eb65de740 1 --2- 172.21.15.2:0/881181307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699ce40 0x5562d699d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.336+0000 7f0eb65de740 1 -- 172.21.15.2:0/881181307 >> 172.21.15.2:0/881181307 conn(0x5562d69993d0 msgr2=0x5562d69997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.336+0000 7f0eb65de740 1 -- 172.21.15.2:0/881181307 shutdown_connections 2024-09-10T08:20:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.336+0000 7f0eb65de740 1 -- 172.21.15.2:0/881181307 wait complete. 2024-09-10T08:20:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.337+0000 7f0eb1c00640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0e940a2b80 0x7f0e940a5040 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009fd0 tx=0x7f0ea40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.337+0000 7f0eb65de740 1 -- start start 2024-09-10T08:20:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.337+0000 7f0eb65de740 1 Processor -- start 2024-09-10T08:20:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb65de740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699ce40 0x5562d69a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb65de740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d699d760 0x5562d69a5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb65de740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d69a8020 0x5562d69a5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb65de740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5562d69aa9b0 con 0x5562d69a8020 2024-09-10T08:20:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb65de740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5562d68be0f0 con 0x5562d699d760 2024-09-10T08:20:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb65de740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5562d69aab30 con 0x5562d699ce40 2024-09-10T08:20:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb65de740 10 client.0 _ll_register_callbacks cb 0x5562d68a5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb1c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d69a8020 0x5562d69a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb1c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d69a8020 0x5562d69a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34150/0 (socket says 172.21.15.2:34150) 2024-09-10T08:20:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb1c00640 1 -- 172.21.15.2:0/260188094 learned_addr learned my addr 172.21.15.2:0/260188094 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb1200640 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699ce40 0x5562d69a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eabe00640 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d699d760 0x5562d69a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb1c00640 1 -- 172.21.15.2:0/260188094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699ce40 msgr2=0x5562d69a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb1c00640 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699ce40 0x5562d69a4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb1c00640 1 -- 172.21.15.2:0/260188094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d699d760 msgr2=0x5562d69a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb1c00640 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d699d760 0x5562d69a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb1c00640 1 -- 172.21.15.2:0/260188094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562d66743c0 con 0x5562d69a8020 2024-09-10T08:20:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.338+0000 7f0eb1200640 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699ce40 0x5562d69a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.339+0000 7f0eb1c00640 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d69a8020 0x5562d69a5970 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400f3f0 tx=0x7f0ea4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2678a2f7da446ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.339+0000 7f0e92a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.339+0000 7f0e92a00640 1 -- 172.21.15.2:0/260188094 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea402a060 con 0x5562d69a8020 2024-09-10T08:20:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.339+0000 7f0e92a00640 1 -- 172.21.15.2:0/260188094 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0ea401a880 con 0x5562d69a8020 2024-09-10T08:20:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.339+0000 7f0eb65de740 1 -- 172.21.15.2:0/260188094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea00f9420 con 0x5562d69a8020 2024-09-10T08:20:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.339+0000 7f0e92a00640 1 -- 172.21.15.2:0/260188094 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0ea4031050 con 0x5562d69a8020 2024-09-10T08:20:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.339+0000 7f0eb65de740 10 client.18762 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.339+0000 7f0eb65de740 1 -- 172.21.15.2:0/260188094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5562d69b69f0 con 0x5562d69a8020 2024-09-10T08:20:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.340+0000 7f0e92a00640 1 -- 172.21.15.2:0/260188094 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea00f9420 con 0x5562d69a8020 2024-09-10T08:20:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.340+0000 7f0e92a00640 1 -- 172.21.15.2:0/260188094 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0ea400b040 con 0x5562d69a8020 2024-09-10T08:20:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.340+0000 7f0eb65de740 10 client.18762 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.340+0000 7f0eb65de740 10 client.18762 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:20:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.340+0000 7f0eb65de740 20 client.18762 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.340+0000 7f0eb65de740 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5562d69b7eb0 0x5562d69ba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.340+0000 7f0eb65de740 4 client.18762 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.340+0000 7f0eb65de740 1 -- 172.21.15.2:0/260188094 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562d6910950 con 0x5562d69b7eb0 2024-09-10T08:20:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.340+0000 7f0eb1200640 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5562d69b7eb0 0x5562d69ba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.341+0000 7f0eb1200640 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5562d69b7eb0 0x5562d69ba2e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55573b664e9dfd44 server_cookie=a5ba9a0964c5d8c9 in_seq=0 out_seq=0 2024-09-10T08:20:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.341+0000 7f0e92a00640 10 client.18762 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:20:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0e92a00640 1 -- 172.21.15.2:0/260188094 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562d6910950 con 0x5562d69b7eb0 2024-09-10T08:20:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0e92a00640 10 client.18762 handle_command_reply: tid=0 2024-09-10T08:20:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0eb65de740 1 client.18762 shutdown 2024-09-10T08:20:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0eb65de740 1 -- 172.21.15.2:0/260188094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d69a8020 msgr2=0x5562d69a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0eb65de740 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d69a8020 0x5562d69a5970 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400f3f0 tx=0x7f0ea4004ab0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0eb65de740 1 -- 172.21.15.2:0/260188094 shutdown_connections 2024-09-10T08:20:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0eb65de740 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5562d69b7eb0 0x5562d69ba2e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0eb65de740 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d69a8020 0x5562d69a5970 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0eb65de740 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d699d760 0x5562d69a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0e92a00640 0 client.18762 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:20:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0eb65de740 1 --2- 172.21.15.2:0/260188094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699ce40 0x5562d69a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0eb65de740 1 -- 172.21.15.2:0/260188094 >> 172.21.15.2:0/260188094 conn(0x5562d69993d0 msgr2=0x5562d699f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0eb65de740 1 -- 172.21.15.2:0/260188094 shutdown_connections 2024-09-10T08:20:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.342+0000 7f0eb65de740 1 -- 172.21.15.2:0/260188094 wait complete. 2024-09-10T08:20:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.343+0000 7f0eb65de740 20 client.18762 trim_cache size 0 max 16384 2024-09-10T08:20:03.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.356+0000 7f0eb65de740 1 Processor -- start 2024-09-10T08:20:03.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.356+0000 7f0eb65de740 1 -- start start 2024-09-10T08:20:03.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb65de740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 0x5562d699efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb65de740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699d760 0x5562d69bfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb65de740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d69a8020 0x5562d69c00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb65de740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1b3720 con 0x5562d699ce40 2024-09-10T08:20:03.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb65de740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1b0ff0 con 0x5562d69a8020 2024-09-10T08:20:03.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb65de740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1aee40 con 0x5562d699d760 2024-09-10T08:20:03.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb1200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 0x5562d699efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb1200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 0x5562d699efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34164/0 (socket says 172.21.15.2:34164) 2024-09-10T08:20:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb1200640 1 -- 172.21.15.2:0/711602272 learned_addr learned my addr 172.21.15.2:0/711602272 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eabe00640 1 --2- 172.21.15.2:0/711602272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699d760 0x5562d69bfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb1c00640 1 --2- 172.21.15.2:0/711602272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d69a8020 0x5562d69c00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb1200640 1 -- 172.21.15.2:0/711602272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699d760 msgr2=0x5562d69bfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb1200640 1 --2- 172.21.15.2:0/711602272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699d760 0x5562d69bfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb1200640 1 -- 172.21.15.2:0/711602272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d69a8020 msgr2=0x5562d69c00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.357+0000 7f0eb1200640 1 --2- 172.21.15.2:0/711602272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d69a8020 0x5562d69c00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.358+0000 7f0eb1200640 1 -- 172.21.15.2:0/711602272 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562d699a560 con 0x5562d699ce40 2024-09-10T08:20:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.358+0000 7f0eb1200640 1 --2- 172.21.15.2:0/711602272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 0x5562d699efa0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00f4770 tx=0x7f0ea00f9ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e00d96a1b88ed8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.358+0000 7f0e9d600640 1 -- 172.21.15.2:0/711602272 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00fd040 con 0x5562d699ce40 2024-09-10T08:20:03.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.358+0000 7f0e9d600640 1 -- 172.21.15.2:0/711602272 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea00bb4f0 con 0x5562d699ce40 2024-09-10T08:20:03.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.358+0000 7f0e9d600640 1 -- 172.21.15.2:0/711602272 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00ba750 con 0x5562d699ce40 2024-09-10T08:20:03.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.359+0000 7f0eb65de740 1 -- 172.21.15.2:0/711602272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 msgr2=0x5562d699efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:03.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.359+0000 7f0eb65de740 1 --2- 172.21.15.2:0/711602272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 0x5562d699efa0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00f4770 tx=0x7f0ea00f9ec0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.359+0000 7f0eb65de740 1 -- 172.21.15.2:0/711602272 shutdown_connections 2024-09-10T08:20:03.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.359+0000 7f0eb65de740 1 --2- 172.21.15.2:0/711602272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d69a8020 0x5562d69c00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.359+0000 7f0eb65de740 1 --2- 172.21.15.2:0/711602272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d699d760 0x5562d69bfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.359+0000 7f0eb65de740 1 --2- 172.21.15.2:0/711602272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 0x5562d699efa0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.359+0000 7f0eb65de740 1 -- 172.21.15.2:0/711602272 >> 172.21.15.2:0/711602272 conn(0x5562d69993d0 msgr2=0x5562d699e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:03.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.359+0000 7f0eb65de740 1 -- 172.21.15.2:0/711602272 shutdown_connections 2024-09-10T08:20:03.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.359+0000 7f0eb65de740 1 -- 172.21.15.2:0/711602272 wait complete. 2024-09-10T08:20:03.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.359+0000 7f0eb65de740 1 -- start start 2024-09-10T08:20:03.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.360+0000 7f0eb65de740 1 Processor -- start 2024-09-10T08:20:03.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.360+0000 7f0eb65de740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 0x5562d69b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.360+0000 7f0eb65de740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d699d760 0x5562d69ab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.360+0000 7f0eb65de740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d69a8020 0x5562d69abd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.360+0000 7f0eb65de740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1af4b0 con 0x5562d699ce40 2024-09-10T08:20:03.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.360+0000 7f0eb65de740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1b0920 con 0x5562d699d760 2024-09-10T08:20:03.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.360+0000 7f0eb65de740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1afb80 con 0x5562d69a8020 2024-09-10T08:20:03.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.360+0000 7f0eb65de740 10 client.0 _ll_register_callbacks cb 0x7f0ea400ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:03.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.360+0000 7f0eb1200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 0x5562d69b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.360+0000 7f0eb1200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 0x5562d69b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34166/0 (socket says 172.21.15.2:34166) 2024-09-10T08:20:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.360+0000 7f0eb1200640 1 -- 172.21.15.2:0/3645609326 learned_addr learned my addr 172.21.15.2:0/3645609326 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.360+0000 7f0eb1c00640 1 --2- 172.21.15.2:0/3645609326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d69a8020 0x5562d69abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.360+0000 7f0eabe00640 1 --2- 172.21.15.2:0/3645609326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d699d760 0x5562d69ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.361+0000 7f0eb1200640 1 -- 172.21.15.2:0/3645609326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d69a8020 msgr2=0x5562d69abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.361+0000 7f0eb1200640 1 --2- 172.21.15.2:0/3645609326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d69a8020 0x5562d69abd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.361+0000 7f0eb1200640 1 -- 172.21.15.2:0/3645609326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d699d760 msgr2=0x5562d69ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.361+0000 7f0eb1200640 1 --2- 172.21.15.2:0/3645609326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d699d760 0x5562d69ab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.361+0000 7f0eb1200640 1 -- 172.21.15.2:0/3645609326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562d69ac570 con 0x5562d699ce40 2024-09-10T08:20:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.361+0000 7f0eb1200640 1 --2- 172.21.15.2:0/3645609326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 0x5562d69b04d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00b7250 tx=0x7f0ea0030960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257e92429b71476 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.361+0000 7f0e92a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.361+0000 7f0e92a00640 1 -- 172.21.15.2:0/3645609326 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00fd070 con 0x5562d699ce40 2024-09-10T08:20:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.361+0000 7f0e92a00640 1 -- 172.21.15.2:0/3645609326 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0ea00ec1a0 con 0x5562d699ce40 2024-09-10T08:20:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.361+0000 7f0eb65de740 1 -- 172.21.15.2:0/3645609326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea00bb4f0 con 0x5562d699ce40 2024-09-10T08:20:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.362+0000 7f0e92a00640 1 -- 172.21.15.2:0/3645609326 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0ea00ba750 con 0x5562d699ce40 2024-09-10T08:20:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.362+0000 7f0eb65de740 10 client.18768 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.362+0000 7f0eb65de740 1 -- 172.21.15.2:0/3645609326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5562d69ac570 con 0x5562d699ce40 2024-09-10T08:20:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.362+0000 7f0e92a00640 1 -- 172.21.15.2:0/3645609326 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea00bb4f0 con 0x5562d699ce40 2024-09-10T08:20:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.362+0000 7f0e92a00640 1 -- 172.21.15.2:0/3645609326 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0ea003a070 con 0x5562d699ce40 2024-09-10T08:20:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.362+0000 7f0eb65de740 10 client.18768 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.362+0000 7f0eb65de740 10 client.18768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:20:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.362+0000 7f0eb65de740 20 client.18768 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.362+0000 7f0eb65de740 1 --2- 172.21.15.2:0/3645609326 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5562d6a22ca0 0x5562d6a25090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.362+0000 7f0eb65de740 4 client.18768 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:20:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.362+0000 7f0eb65de740 1 -- 172.21.15.2:0/3645609326 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5562d690ae60 con 0x5562d6a22ca0 2024-09-10T08:20:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.363+0000 7f0eabe00640 1 --2- 172.21.15.2:0/3645609326 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5562d6a22ca0 0x5562d6a25090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.363+0000 7f0eabe00640 1 --2- 172.21.15.2:0/3645609326 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5562d6a22ca0 0x5562d6a25090 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=789ed412a470ca77 server_cookie=55aa0676c47621de in_seq=0 out_seq=0 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.363+0000 7f0e92a00640 10 client.18768 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0e92a00640 1 -- 172.21.15.2:0/3645609326 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5562d690ae60 con 0x5562d6a22ca0 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0e92a00640 10 client.18768 handle_command_reply: tid=0 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0eb65de740 1 client.18768 shutdown 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0eb65de740 1 -- 172.21.15.2:0/3645609326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 msgr2=0x5562d69b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0eb65de740 1 --2- 172.21.15.2:0/3645609326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 0x5562d69b04d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00b7250 tx=0x7f0ea0030960 comp rx=0 tx=0).stop 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0eb65de740 1 -- 172.21.15.2:0/3645609326 shutdown_connections 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0eb65de740 1 --2- 172.21.15.2:0/3645609326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5562d69a8020 0x5562d69abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0eb65de740 1 --2- 172.21.15.2:0/3645609326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562d699d760 0x5562d69ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0eb65de740 1 --2- 172.21.15.2:0/3645609326 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5562d6a22ca0 0x5562d6a25090 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0eb65de740 1 --2- 172.21.15.2:0/3645609326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562d699ce40 0x5562d69b04d0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0eb65de740 1 -- 172.21.15.2:0/3645609326 >> 172.21.15.2:0/3645609326 conn(0x5562d69993d0 msgr2=0x5562d699f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0e92a00640 0 client.18768 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:20:03.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0eb65de740 1 -- 172.21.15.2:0/3645609326 shutdown_connections 2024-09-10T08:20:03.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0eb65de740 1 -- 172.21.15.2:0/3645609326 wait complete. 2024-09-10T08:20:03.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.364+0000 7f0eb65de740 20 client.18768 trim_cache size 0 max 16384 2024-09-10T08:20:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:20:03.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.365+0000 7f0eb2600640 1 -- 172.21.15.2:0/3846893747 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0e940a2b80 msgr2=0x7f0e940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.365+0000 7f0eb2600640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0e940a2b80 0x7f0e940a5040 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009fd0 tx=0x7f0ea40095c0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.365+0000 7f0eb2600640 1 -- 172.21.15.2:0/3846893747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eac103e50 msgr2=0x7f0eac19a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.365+0000 7f0eb2600640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eac103e50 0x7f0eac19a570 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0ea002a860 tx=0x7f0ea0033b60 comp rx=0 tx=0).stop 2024-09-10T08:20:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.365+0000 7f0eb2600640 1 -- 172.21.15.2:0/3846893747 shutdown_connections 2024-09-10T08:20:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.365+0000 7f0eb2600640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0eac109090 0x7f0eac19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.365+0000 7f0eb2600640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f0e940a2b80 0x7f0e940a5040 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.365+0000 7f0eb2600640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0eac104800 0x7f0eac19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.365+0000 7f0eb2600640 1 --2- 172.21.15.2:0/3846893747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eac103e50 0x7f0eac19a570 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.365+0000 7f0eb2600640 1 -- 172.21.15.2:0/3846893747 >> 172.21.15.2:0/3846893747 conn(0x7f0eac0ff8b0 msgr2=0x7f0eac100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.365+0000 7f0eb2600640 1 -- 172.21.15.2:0/3846893747 shutdown_connections 2024-09-10T08:20:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:03.365+0000 7f0eb2600640 1 -- 172.21.15.2:0/3846893747 wait complete. 2024-09-10T08:20:03.839 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:03 smithi002 ceph-mon[30563]: pgmap v744: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:20:03.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:03 smithi155 ceph-mon[35405]: pgmap v744: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:20:03.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:03 smithi107 ceph-mon[35048]: pgmap v744: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:20:04.326 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:20:04.707 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:05.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.349+0000 7f839a400640 1 -- 172.21.15.2:0/1114526742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8394103e30 msgr2=0x7f8394104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.349+0000 7f839a400640 1 --2- 172.21.15.2:0/1114526742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8394103e30 0x7f8394104210 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8384009860 tx=0x7f838402a340 comp rx=0 tx=0).stop 2024-09-10T08:20:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.349+0000 7f839a400640 1 -- 172.21.15.2:0/1114526742 shutdown_connections 2024-09-10T08:20:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.349+0000 7f839a400640 1 --2- 172.21.15.2:0/1114526742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8394108f60 0x7f8394110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.349+0000 7f839a400640 1 --2- 172.21.15.2:0/1114526742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83941047e0 0x7f8394108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.349+0000 7f839a400640 1 --2- 172.21.15.2:0/1114526742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8394103e30 0x7f8394104210 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.349+0000 7f839a400640 1 -- 172.21.15.2:0/1114526742 >> 172.21.15.2:0/1114526742 conn(0x7f83940ff8d0 msgr2=0x7f8394101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.349+0000 7f839a400640 1 -- 172.21.15.2:0/1114526742 shutdown_connections 2024-09-10T08:20:05.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.349+0000 7f839a400640 1 -- 172.21.15.2:0/1114526742 wait complete. 2024-09-10T08:20:05.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.350+0000 7f839a400640 1 Processor -- start 2024-09-10T08:20:05.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.350+0000 7f839a400640 1 -- start start 2024-09-10T08:20:05.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.351+0000 7f839a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8394103e30 0x7f839419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.351+0000 7f839a400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83941047e0 0x7f839419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.351+0000 7f839a400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8394108f60 0x7f839419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.351+0000 7f839a400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8394113fc0 con 0x7f8394108f60 2024-09-10T08:20:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.351+0000 7f839a400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8394113e40 con 0x7f8394103e30 2024-09-10T08:20:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.351+0000 7f839a400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8394114140 con 0x7f83941047e0 2024-09-10T08:20:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.351+0000 7f8399000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8394103e30 0x7f839419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.351+0000 7f838be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83941047e0 0x7f839419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.351+0000 7f8399a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8394108f60 0x7f839419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.351+0000 7f8399000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8394103e30 0x7f839419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40818/0 (socket says 172.21.15.2:40818) 2024-09-10T08:20:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.351+0000 7f8399000640 1 -- 172.21.15.2:0/2589573361 learned_addr learned my addr 172.21.15.2:0/2589573361 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.351+0000 7f838be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83941047e0 0x7f839419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:49966/0 (socket says 172.21.15.2:49966) 2024-09-10T08:20:05.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.352+0000 7f8399000640 1 -- 172.21.15.2:0/2589573361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83941047e0 msgr2=0x7f839419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:05.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.352+0000 7f8399000640 1 --2- 172.21.15.2:0/2589573361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83941047e0 0x7f839419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.352+0000 7f8399000640 1 -- 172.21.15.2:0/2589573361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8394108f60 msgr2=0x7f839419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:05.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.352+0000 7f8399000640 1 --2- 172.21.15.2:0/2589573361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8394108f60 0x7f839419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.352+0000 7f8399000640 1 -- 172.21.15.2:0/2589573361 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839419f430 con 0x7f8394103e30 2024-09-10T08:20:05.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.352+0000 7f8399000640 1 --2- 172.21.15.2:0/2589573361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8394103e30 0x7f839419a480 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8384035040 tx=0x7f8384033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:05.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.352+0000 7f8392000640 1 -- 172.21.15.2:0/2589573361 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838403f020 con 0x7f8394103e30 2024-09-10T08:20:05.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.352+0000 7f839a400640 1 -- 172.21.15.2:0/2589573361 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839419f6c0 con 0x7f8394103e30 2024-09-10T08:20:05.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.352+0000 7f8392000640 1 -- 172.21.15.2:0/2589573361 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8384032770 con 0x7f8394103e30 2024-09-10T08:20:05.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.352+0000 7f8392000640 1 -- 172.21.15.2:0/2589573361 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384037040 con 0x7f8394103e30 2024-09-10T08:20:05.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.352+0000 7f839a400640 1 -- 172.21.15.2:0/2589573361 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83941a7000 con 0x7f8394103e30 2024-09-10T08:20:05.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.355+0000 7f839e439740 1 Processor -- start 2024-09-10T08:20:05.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.356+0000 7f839e439740 1 -- start start 2024-09-10T08:20:05.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.356+0000 7f8392000640 1 -- 172.21.15.2:0/2589573361 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8384042020 con 0x7f8394103e30 2024-09-10T08:20:05.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.356+0000 7f839e439740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c24220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.356+0000 7f839e439740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c24760 0x565293c2eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.356+0000 7f839e439740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c2f020 0x565293c31410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.356+0000 7f839e439740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x565293b8a3e0 con 0x565293c24760 2024-09-10T08:20:05.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.356+0000 7f839e439740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x565293b31d80 con 0x565293c23e40 2024-09-10T08:20:05.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.356+0000 7f839e439740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565293b863a0 con 0x565293c2f020 2024-09-10T08:20:05.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.356+0000 7f8399a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c24220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.356+0000 7f8399000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c24760 0x565293c2eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.356+0000 7f8399a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c24220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40824/0 (socket says 172.21.15.2:40824) 2024-09-10T08:20:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.356+0000 7f8399a00640 1 -- 172.21.15.2:0/1937124610 learned_addr learned my addr 172.21.15.2:0/1937124610 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.356+0000 7f838be00640 1 --2- 172.21.15.2:0/1937124610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c2f020 0x565293c31410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.357+0000 7f8399a00640 1 -- 172.21.15.2:0/1937124610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c2f020 msgr2=0x565293c31410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.357+0000 7f8399a00640 1 --2- 172.21.15.2:0/1937124610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c2f020 0x565293c31410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.357+0000 7f8399a00640 1 -- 172.21.15.2:0/1937124610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c24760 msgr2=0x565293c2eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.357+0000 7f8399a00640 1 --2- 172.21.15.2:0/1937124610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c24760 0x565293c2eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.357+0000 7f8399a00640 1 -- 172.21.15.2:0/1937124610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565293b50ae0 con 0x565293c23e40 2024-09-10T08:20:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.357+0000 7f838be00640 1 --2- 172.21.15.2:0/1937124610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c2f020 0x565293c31410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:05.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.357+0000 7f8399a00640 1 --2- 172.21.15.2:0/1937124610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c24220 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f838c00e480 tx=0x7f838c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc813a5d3e76500 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.357+0000 7f838a000640 1 -- 172.21.15.2:0/1937124610 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c003970 con 0x565293c23e40 2024-09-10T08:20:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.357+0000 7f838a000640 1 -- 172.21.15.2:0/1937124610 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f838c00d7d0 con 0x565293c23e40 2024-09-10T08:20:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.357+0000 7f838a000640 1 -- 172.21.15.2:0/1937124610 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c004560 con 0x565293c23e40 2024-09-10T08:20:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.358+0000 7f8392000640 1 --2- 172.21.15.2:0/2589573361 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83780a2b80 0x7f83780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.358+0000 7f8392000640 1 -- 172.21.15.2:0/2589573361 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f83840ef5f0 con 0x7f8394103e30 2024-09-10T08:20:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.358+0000 7f8399000640 1 --2- 172.21.15.2:0/2589573361 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83780a2b80 0x7f83780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.358+0000 7f839e439740 1 -- 172.21.15.2:0/1937124610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 msgr2=0x565293c24220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.358+0000 7f839e439740 1 --2- 172.21.15.2:0/1937124610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c24220 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f838c00e480 tx=0x7f838c0095c0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.358+0000 7f839e439740 1 -- 172.21.15.2:0/1937124610 shutdown_connections 2024-09-10T08:20:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.358+0000 7f839e439740 1 --2- 172.21.15.2:0/1937124610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c2f020 0x565293c31410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.358+0000 7f839e439740 1 --2- 172.21.15.2:0/1937124610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c24760 0x565293c2eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.358+0000 7f839e439740 1 --2- 172.21.15.2:0/1937124610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c24220 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.358+0000 7f839e439740 1 -- 172.21.15.2:0/1937124610 >> 172.21.15.2:0/1937124610 conn(0x565293c203d0 msgr2=0x565293c207b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.359+0000 7f839e439740 1 -- 172.21.15.2:0/1937124610 shutdown_connections 2024-09-10T08:20:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.359+0000 7f839e439740 1 -- 172.21.15.2:0/1937124610 wait complete. 2024-09-10T08:20:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.359+0000 7f8399000640 1 --2- 172.21.15.2:0/2589573361 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83780a2b80 0x7f83780a5040 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8384000c00 tx=0x7f83840f2f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.359+0000 7f839e439740 1 -- start start 2024-09-10T08:20:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.359+0000 7f839e439740 1 Processor -- start 2024-09-10T08:20:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.360+0000 7f839e439740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c23e40 0x565293c2bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.360+0000 7f839e439740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 0x565293c2c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.360+0000 7f839e439740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c2f020 0x565293c2c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.360+0000 7f839e439740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x565293c31bf0 con 0x565293c23e40 2024-09-10T08:20:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.360+0000 7f839e439740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x565293b450f0 con 0x565293c2f020 2024-09-10T08:20:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.360+0000 7f839e439740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565293c31d70 con 0x565293c24760 2024-09-10T08:20:05.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.360+0000 7f839e439740 10 client.0 _ll_register_callbacks cb 0x565293b2cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.360+0000 7f8399a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c23e40 0x565293c2bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.360+0000 7f8399a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c23e40 0x565293c2bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34190/0 (socket says 172.21.15.2:34190) 2024-09-10T08:20:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.360+0000 7f8399a00640 1 -- 172.21.15.2:0/1153942085 learned_addr learned my addr 172.21.15.2:0/1153942085 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.360+0000 7f838be00640 1 --2- 172.21.15.2:0/1153942085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 0x565293c2c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.360+0000 7f8399a00640 1 --2- 172.21.15.2:0/1153942085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c2f020 0x565293c2c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.361+0000 7f8399a00640 1 -- 172.21.15.2:0/1153942085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 msgr2=0x565293c2c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.361+0000 7f8399a00640 1 --2- 172.21.15.2:0/1153942085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 0x565293c2c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.361+0000 7f8399a00640 1 -- 172.21.15.2:0/1153942085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c2f020 msgr2=0x565293c2c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.361+0000 7f8399a00640 1 --2- 172.21.15.2:0/1153942085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c2f020 0x565293c2c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.361+0000 7f8399a00640 1 -- 172.21.15.2:0/1153942085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652938fb3c0 con 0x565293c23e40 2024-09-10T08:20:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.361+0000 7f8399a00640 1 --2- 172.21.15.2:0/1153942085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c23e40 0x565293c2bf00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f838c00de50 tx=0x7f838c009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9c6ada99e65aed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.361+0000 7f8382a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.361+0000 7f8382a00640 1 -- 172.21.15.2:0/1153942085 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c003db0 con 0x565293c23e40 2024-09-10T08:20:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.361+0000 7f8382a00640 1 -- 172.21.15.2:0/1153942085 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f838c004a40 con 0x565293c23e40 2024-09-10T08:20:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.361+0000 7f839e439740 1 -- 172.21.15.2:0/1153942085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f838c00d7d0 con 0x565293c23e40 2024-09-10T08:20:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.361+0000 7f8382a00640 1 -- 172.21.15.2:0/1153942085 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f838c004c40 con 0x565293c23e40 2024-09-10T08:20:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.361+0000 7f839e439740 10 client.18780 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.361+0000 7f839e439740 1 -- 172.21.15.2:0/1153942085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565293c3dc30 con 0x565293c23e40 2024-09-10T08:20:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.362+0000 7f8382a00640 1 -- 172.21.15.2:0/1153942085 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f838c00d7d0 con 0x565293c23e40 2024-09-10T08:20:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.362+0000 7f8382a00640 1 -- 172.21.15.2:0/1153942085 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f838c00b040 con 0x565293c23e40 2024-09-10T08:20:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.362+0000 7f839e439740 10 client.18780 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.362+0000 7f839e439740 10 client.18780 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:20:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.362+0000 7f839e439740 20 client.18780 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.362+0000 7f839e439740 1 --2- 172.21.15.2:0/1153942085 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x565293c3efe0 0x565293c413d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.362+0000 7f839e439740 4 client.18780 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.362+0000 7f839e439740 1 -- 172.21.15.2:0/1153942085 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565293b97950 con 0x565293c3efe0 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.362+0000 7f838be00640 1 --2- 172.21.15.2:0/1153942085 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x565293c3efe0 0x565293c413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.363+0000 7f838be00640 1 --2- 172.21.15.2:0/1153942085 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x565293c3efe0 0x565293c413d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2ddb087256ce0af server_cookie=99f157f67fc8fa34 in_seq=0 out_seq=0 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.363+0000 7f8382a00640 10 client.18780 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.364+0000 7f8382a00640 1 -- 172.21.15.2:0/1153942085 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565293b97950 con 0x565293c3efe0 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.364+0000 7f8382a00640 10 client.18780 handle_command_reply: tid=0 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.364+0000 7f839e439740 1 client.18780 shutdown 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.364+0000 7f839e439740 1 -- 172.21.15.2:0/1153942085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c23e40 msgr2=0x565293c2bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.364+0000 7f839e439740 1 --2- 172.21.15.2:0/1153942085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c23e40 0x565293c2bf00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f838c00de50 tx=0x7f838c009b10 comp rx=0 tx=0).stop 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.364+0000 7f839e439740 1 -- 172.21.15.2:0/1153942085 shutdown_connections 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.364+0000 7f839e439740 1 --2- 172.21.15.2:0/1153942085 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x565293c3efe0 0x565293c413d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.364+0000 7f839e439740 1 --2- 172.21.15.2:0/1153942085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c2f020 0x565293c2c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.364+0000 7f839e439740 1 --2- 172.21.15.2:0/1153942085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 0x565293c2c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.364+0000 7f839e439740 1 --2- 172.21.15.2:0/1153942085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c23e40 0x565293c2bf00 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.364+0000 7f839e439740 1 -- 172.21.15.2:0/1153942085 >> 172.21.15.2:0/1153942085 conn(0x565293c203d0 msgr2=0x565293c26100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.364+0000 7f8382a00640 0 client.18780 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:20:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.365+0000 7f839e439740 1 -- 172.21.15.2:0/1153942085 shutdown_connections 2024-09-10T08:20:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.365+0000 7f839e439740 1 -- 172.21.15.2:0/1153942085 wait complete. 2024-09-10T08:20:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.365+0000 7f839e439740 20 client.18780 trim_cache size 0 max 16384 2024-09-10T08:20:05.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.378+0000 7f839e439740 1 Processor -- start 2024-09-10T08:20:05.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.379+0000 7f839e439740 1 -- start start 2024-09-10T08:20:05.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.379+0000 7f839e439740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c26360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.379+0000 7f839e439740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 0x565293c46a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.379+0000 7f839e439740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c2f020 0x565293c46f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.379+0000 7f839e439740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83941b37c0 con 0x565293c2f020 2024-09-10T08:20:05.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.379+0000 7f839e439740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f83941b1090 con 0x565293c23e40 2024-09-10T08:20:05.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.379+0000 7f839e439740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f83941aeee0 con 0x565293c24760 2024-09-10T08:20:05.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.379+0000 7f8399a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c2f020 0x565293c46f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.379+0000 7f8399a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c26360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.379+0000 7f8399a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c26360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40848/0 (socket says 172.21.15.2:40848) 2024-09-10T08:20:05.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.379+0000 7f8399a00640 1 -- 172.21.15.2:0/231851432 learned_addr learned my addr 172.21.15.2:0/231851432 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:05.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.379+0000 7f838be00640 1 --2- 172.21.15.2:0/231851432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 0x565293c46a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.380+0000 7f8399a00640 1 -- 172.21.15.2:0/231851432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 msgr2=0x565293c46a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:05.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.380+0000 7f8399a00640 1 --2- 172.21.15.2:0/231851432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 0x565293c46a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.380+0000 7f8399a00640 1 -- 172.21.15.2:0/231851432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c2f020 msgr2=0x565293c46f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:05.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.380+0000 7f8399a00640 1 --2- 172.21.15.2:0/231851432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c2f020 0x565293c46f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.380+0000 7f8399a00640 1 -- 172.21.15.2:0/231851432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565293c20200 con 0x565293c23e40 2024-09-10T08:20:05.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.380+0000 7f8399a00640 1 --2- 172.21.15.2:0/231851432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c26360 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f838c031800 tx=0x7f838c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94371297867149de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:05.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.380+0000 7f8389600640 1 -- 172.21.15.2:0/231851432 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c00edd0 con 0x565293c23e40 2024-09-10T08:20:05.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.380+0000 7f8389600640 1 -- 172.21.15.2:0/231851432 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f838c009c10 con 0x565293c23e40 2024-09-10T08:20:05.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.380+0000 7f8389600640 1 -- 172.21.15.2:0/231851432 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c0045e0 con 0x565293c23e40 2024-09-10T08:20:05.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.381+0000 7f839e439740 1 -- 172.21.15.2:0/231851432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 msgr2=0x565293c26360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:05.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.381+0000 7f839e439740 1 --2- 172.21.15.2:0/231851432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c26360 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f838c031800 tx=0x7f838c005fd0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.381+0000 7f839e439740 1 -- 172.21.15.2:0/231851432 shutdown_connections 2024-09-10T08:20:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.381+0000 7f839e439740 1 --2- 172.21.15.2:0/231851432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c2f020 0x565293c46f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.381+0000 7f839e439740 1 --2- 172.21.15.2:0/231851432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 0x565293c46a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.381+0000 7f839e439740 1 --2- 172.21.15.2:0/231851432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c26360 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.381+0000 7f839e439740 1 -- 172.21.15.2:0/231851432 >> 172.21.15.2:0/231851432 conn(0x565293c203d0 msgr2=0x565293c2aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.381+0000 7f839e439740 1 -- 172.21.15.2:0/231851432 shutdown_connections 2024-09-10T08:20:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.381+0000 7f839e439740 1 -- 172.21.15.2:0/231851432 wait complete. 2024-09-10T08:20:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.381+0000 7f839e439740 1 -- start start 2024-09-10T08:20:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.382+0000 7f839e439740 1 Processor -- start 2024-09-10T08:20:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.382+0000 7f839e439740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c37830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.382+0000 7f839e439740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 0x565293c32b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.382+0000 7f839e439740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c2f020 0x565293c33150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.382+0000 7f839e439740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83941af550 con 0x565293c2f020 2024-09-10T08:20:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.382+0000 7f839e439740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f83941b09c0 con 0x565293c23e40 2024-09-10T08:20:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.382+0000 7f839e439740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f83941afc20 con 0x565293c24760 2024-09-10T08:20:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.382+0000 7f839e439740 10 client.0 _ll_register_callbacks cb 0x7f83840f5060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.382+0000 7f8399a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c2f020 0x565293c33150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.382+0000 7f8399a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c37830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.382+0000 7f8399a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c37830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40864/0 (socket says 172.21.15.2:40864) 2024-09-10T08:20:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.382+0000 7f8399a00640 1 -- 172.21.15.2:0/332582191 learned_addr learned my addr 172.21.15.2:0/332582191 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.383+0000 7f838be00640 1 --2- 172.21.15.2:0/332582191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 0x565293c32b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.383+0000 7f8399a00640 1 -- 172.21.15.2:0/332582191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 msgr2=0x565293c32b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.383+0000 7f8399a00640 1 --2- 172.21.15.2:0/332582191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 0x565293c32b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.383+0000 7f8399a00640 1 -- 172.21.15.2:0/332582191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 msgr2=0x565293c37830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:05.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.383+0000 7f8399a00640 1 --2- 172.21.15.2:0/332582191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c37830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.383+0000 7f8399a00640 1 -- 172.21.15.2:0/332582191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565293c33980 con 0x565293c2f020 2024-09-10T08:20:05.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.383+0000 7f838be00640 1 --2- 172.21.15.2:0/332582191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 0x565293c32b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:05.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.383+0000 7f8399a00640 1 --2- 172.21.15.2:0/332582191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c2f020 0x565293c33150 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f838c009e40 tx=0x7f838c003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a3eb56ccd0998f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:05.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.383+0000 7f8382a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:05.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.383+0000 7f8382a00640 1 -- 172.21.15.2:0/332582191 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c00edd0 con 0x565293c2f020 2024-09-10T08:20:05.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.383+0000 7f8382a00640 1 -- 172.21.15.2:0/332582191 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f838c01c040 con 0x565293c2f020 2024-09-10T08:20:05.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.383+0000 7f839e439740 1 -- 172.21.15.2:0/332582191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f838c009c10 con 0x565293c2f020 2024-09-10T08:20:05.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.384+0000 7f8382a00640 1 -- 172.21.15.2:0/332582191 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f838c031b70 con 0x565293c2f020 2024-09-10T08:20:05.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.384+0000 7f839e439740 10 client.18786 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:05.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.384+0000 7f839e439740 1 -- 172.21.15.2:0/332582191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565293ca9b10 con 0x565293c2f020 2024-09-10T08:20:05.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.384+0000 7f8382a00640 1 -- 172.21.15.2:0/332582191 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f838c009c10 con 0x565293c2f020 2024-09-10T08:20:05.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.384+0000 7f8382a00640 1 -- 172.21.15.2:0/332582191 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f838c039070 con 0x565293c2f020 2024-09-10T08:20:05.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.384+0000 7f839e439740 10 client.18786 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:05.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.384+0000 7f839e439740 10 client.18786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:20:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.384+0000 7f839e439740 20 client.18786 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.384+0000 7f839e439740 1 --2- 172.21.15.2:0/332582191 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x565293ca9fe0 0x565293cac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.384+0000 7f839e439740 4 client.18786 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:20:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.384+0000 7f839e439740 1 -- 172.21.15.2:0/332582191 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565293b91e60 con 0x565293ca9fe0 2024-09-10T08:20:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.385+0000 7f838be00640 1 --2- 172.21.15.2:0/332582191 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x565293ca9fe0 0x565293cac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.385+0000 7f838be00640 1 --2- 172.21.15.2:0/332582191 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x565293ca9fe0 0x565293cac3d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ffdcb6e9e790abe server_cookie=7cb773cfed4cb24c in_seq=0 out_seq=0 2024-09-10T08:20:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.385+0000 7f8382a00640 10 client.18786 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:20:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.385+0000 7f8382a00640 1 -- 172.21.15.2:0/332582191 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565293b91e60 con 0x565293ca9fe0 2024-09-10T08:20:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.385+0000 7f8382a00640 10 client.18786 handle_command_reply: tid=0 2024-09-10T08:20:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.385+0000 7f839e439740 1 client.18786 shutdown 2024-09-10T08:20:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.386+0000 7f839e439740 1 -- 172.21.15.2:0/332582191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c2f020 msgr2=0x565293c33150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.386+0000 7f839e439740 1 --2- 172.21.15.2:0/332582191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c2f020 0x565293c33150 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f838c009e40 tx=0x7f838c003e50 comp rx=0 tx=0).stop 2024-09-10T08:20:05.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.386+0000 7f839e439740 1 -- 172.21.15.2:0/332582191 shutdown_connections 2024-09-10T08:20:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.386+0000 7f839e439740 1 --2- 172.21.15.2:0/332582191 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x565293ca9fe0 0x565293cac3d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.386+0000 7f839e439740 1 --2- 172.21.15.2:0/332582191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565293c2f020 0x565293c33150 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.386+0000 7f839e439740 1 --2- 172.21.15.2:0/332582191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565293c24760 0x565293c32b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.386+0000 7f839e439740 1 --2- 172.21.15.2:0/332582191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565293c23e40 0x565293c37830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.386+0000 7f8382a00640 0 client.18786 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:20:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.386+0000 7f839e439740 1 -- 172.21.15.2:0/332582191 >> 172.21.15.2:0/332582191 conn(0x565293c203d0 msgr2=0x565293c2bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.386+0000 7f839e439740 1 -- 172.21.15.2:0/332582191 shutdown_connections 2024-09-10T08:20:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.386+0000 7f839e439740 1 -- 172.21.15.2:0/332582191 wait complete. 2024-09-10T08:20:05.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.386+0000 7f839e439740 20 client.18786 trim_cache size 0 max 16384 2024-09-10T08:20:05.391 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:20:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.387+0000 7f839a400640 1 -- 172.21.15.2:0/2589573361 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83780a2b80 msgr2=0x7f83780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.387+0000 7f839a400640 1 --2- 172.21.15.2:0/2589573361 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83780a2b80 0x7f83780a5040 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8384000c00 tx=0x7f83840f2f90 comp rx=0 tx=0).stop 2024-09-10T08:20:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.387+0000 7f839a400640 1 -- 172.21.15.2:0/2589573361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8394103e30 msgr2=0x7f839419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:05.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.387+0000 7f839a400640 1 --2- 172.21.15.2:0/2589573361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8394103e30 0x7f839419a480 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8384035040 tx=0x7f8384033d90 comp rx=0 tx=0).stop 2024-09-10T08:20:05.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.387+0000 7f839a400640 1 -- 172.21.15.2:0/2589573361 shutdown_connections 2024-09-10T08:20:05.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.387+0000 7f839a400640 1 --2- 172.21.15.2:0/2589573361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8394108f60 0x7f839419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.387+0000 7f839a400640 1 --2- 172.21.15.2:0/2589573361 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83780a2b80 0x7f83780a5040 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.387+0000 7f839a400640 1 --2- 172.21.15.2:0/2589573361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83941047e0 0x7f839419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.387+0000 7f839a400640 1 --2- 172.21.15.2:0/2589573361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8394103e30 0x7f839419a480 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:05.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.387+0000 7f839a400640 1 -- 172.21.15.2:0/2589573361 >> 172.21.15.2:0/2589573361 conn(0x7f83940ff8d0 msgr2=0x7f8394100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:05.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.387+0000 7f839a400640 1 -- 172.21.15.2:0/2589573361 shutdown_connections 2024-09-10T08:20:05.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:05.387+0000 7f839a400640 1 -- 172.21.15.2:0/2589573361 wait complete. 2024-09-10T08:20:05.578 INFO:tasks.workunit.client.0.smithi002.stdout:375/600: open fd = 3 2024-09-10T08:20:05.578 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.578 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.578 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.578 INFO:tasks.workunit.client.0.smithi002.stdout:376/600: open fd = 3 2024-09-10T08:20:05.578 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.578 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.578 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.578 INFO:tasks.workunit.client.0.smithi002.stdout:377/600: open fd = 3 2024-09-10T08:20:05.579 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.579 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.579 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.579 INFO:tasks.workunit.client.0.smithi002.stdout:378/600: open fd = 3 2024-09-10T08:20:05.579 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.579 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.579 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.579 INFO:tasks.workunit.client.0.smithi002.stdout:379/600: open fd = 3 2024-09-10T08:20:05.579 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.579 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.579 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.579 INFO:tasks.workunit.client.0.smithi002.stdout:380/600: open fd = 3 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:381/600: open fd = 3 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:382/600: open fd = 3 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:383/600: open fd = 3 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.580 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.581 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.581 INFO:tasks.workunit.client.0.smithi002.stdout:384/600: open fd = 3 2024-09-10T08:20:05.581 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.581 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.581 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.581 INFO:tasks.workunit.client.0.smithi002.stdout:385/600: open fd = 3 2024-09-10T08:20:05.581 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.581 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.581 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.581 INFO:tasks.workunit.client.0.smithi002.stdout:386/600: open fd = 3 2024-09-10T08:20:05.581 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.581 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:387/600: open fd = 3 2024-09-10T08:20:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:388/600: open fd = 3 2024-09-10T08:20:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:389/600: open fd = 3 2024-09-10T08:20:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.583 INFO:tasks.workunit.client.0.smithi002.stdout:390/600: open fd = 3 2024-09-10T08:20:05.583 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.583 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.583 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.583 INFO:tasks.workunit.client.0.smithi002.stdout:391/600: open fd = 3 2024-09-10T08:20:05.583 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.583 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.583 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.583 INFO:tasks.workunit.client.0.smithi002.stdout:392/600: open fd = 3 2024-09-10T08:20:05.583 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.583 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.583 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.584 INFO:tasks.workunit.client.0.smithi002.stdout:393/600: open fd = 3 2024-09-10T08:20:05.584 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.584 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.584 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.584 INFO:tasks.workunit.client.0.smithi002.stdout:394/600: open fd = 3 2024-09-10T08:20:05.584 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.584 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.584 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.584 INFO:tasks.workunit.client.0.smithi002.stdout:395/600: open fd = 3 2024-09-10T08:20:05.584 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.584 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.584 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.585 INFO:tasks.workunit.client.0.smithi002.stdout:396/600: open fd = 3 2024-09-10T08:20:05.585 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.585 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.585 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.585 INFO:tasks.workunit.client.0.smithi002.stdout:397/600: open fd = 3 2024-09-10T08:20:05.585 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.585 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.585 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.585 INFO:tasks.workunit.client.0.smithi002.stdout:398/600: open fd = 3 2024-09-10T08:20:05.585 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.585 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.585 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:399/600: open fd = 3 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:400/600: open fd = 3 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:401/600: open fd = 3 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:402/600: open fd = 3 2024-09-10T08:20:05.586 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.587 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.587 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.587 INFO:tasks.workunit.client.0.smithi002.stdout:403/600: open fd = 3 2024-09-10T08:20:05.587 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.587 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.587 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.587 INFO:tasks.workunit.client.0.smithi002.stdout:404/600: open fd = 3 2024-09-10T08:20:05.587 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.587 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.587 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.587 INFO:tasks.workunit.client.0.smithi002.stdout:405/600: open fd = 3 2024-09-10T08:20:05.587 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.587 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.588 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.588 INFO:tasks.workunit.client.0.smithi002.stdout:406/600: open fd = 3 2024-09-10T08:20:05.588 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.588 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.588 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.588 INFO:tasks.workunit.client.0.smithi002.stdout:407/600: open fd = 3 2024-09-10T08:20:05.588 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.588 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.588 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.588 INFO:tasks.workunit.client.0.smithi002.stdout:408/600: open fd = 3 2024-09-10T08:20:05.588 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.588 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.588 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:409/600: open fd = 3 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:410/600: open fd = 3 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:411/600: open fd = 3 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:412/600: open fd = 3 2024-09-10T08:20:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:413/600: open fd = 3 2024-09-10T08:20:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:414/600: open fd = 3 2024-09-10T08:20:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:415/600: open fd = 3 2024-09-10T08:20:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:416/600: open fd = 3 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:417/600: open fd = 3 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:418/600: open fd = 3 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:419/600: open fd = 3 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:420/600: open fd = 3 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:421/600: open fd = 3 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:422/600: open fd = 3 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.592 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:423/600: open fd = 3 2024-09-10T08:20:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:424/600: open fd = 3 2024-09-10T08:20:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:425/600: open fd = 3 2024-09-10T08:20:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:426/600: open fd = 3 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:427/600: open fd = 3 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:428/600: open fd = 3 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:429/600: open fd = 3 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.594 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.595 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.595 INFO:tasks.workunit.client.0.smithi002.stdout:430/600: open fd = 3 2024-09-10T08:20:05.595 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.595 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.595 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.595 INFO:tasks.workunit.client.0.smithi002.stdout:431/600: open fd = 3 2024-09-10T08:20:05.595 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.595 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.595 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.595 INFO:tasks.workunit.client.0.smithi002.stdout:432/600: open fd = 3 2024-09-10T08:20:05.595 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.595 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.595 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:433/600: open fd = 3 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:434/600: open fd = 3 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:435/600: open fd = 3 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:436/600: open fd = 3 2024-09-10T08:20:05.596 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.597 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:20:05.597 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:20:05.869 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:05 smithi002 ceph-mon[30563]: pgmap v745: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:20:05.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:05 smithi155 ceph-mon[35405]: pgmap v745: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:20:05.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:05 smithi107 ceph-mon[35048]: pgmap v745: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:20:06.448 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:20:06.825 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:07.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.406+0000 7f23c7000640 1 -- 172.21.15.2:0/1773915524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23c0103e30 msgr2=0x7f23c0104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:07.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.406+0000 7f23c7000640 1 --2- 172.21.15.2:0/1773915524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23c0103e30 0x7f23c0104210 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009900 tx=0x7f23b402a410 comp rx=0 tx=0).stop 2024-09-10T08:20:07.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.406+0000 7f23c7000640 1 -- 172.21.15.2:0/1773915524 shutdown_connections 2024-09-10T08:20:07.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.406+0000 7f23c7000640 1 --2- 172.21.15.2:0/1773915524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c0108f60 0x7f23c0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.406+0000 7f23c7000640 1 --2- 172.21.15.2:0/1773915524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c01047e0 0x7f23c0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.406+0000 7f23c7000640 1 --2- 172.21.15.2:0/1773915524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23c0103e30 0x7f23c0104210 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.406+0000 7f23c7000640 1 -- 172.21.15.2:0/1773915524 >> 172.21.15.2:0/1773915524 conn(0x7f23c00ff8d0 msgr2=0x7f23c0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:07.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.406+0000 7f23c7000640 1 -- 172.21.15.2:0/1773915524 shutdown_connections 2024-09-10T08:20:07.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.406+0000 7f23c7000640 1 -- 172.21.15.2:0/1773915524 wait complete. 2024-09-10T08:20:07.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.408+0000 7f23c7000640 1 Processor -- start 2024-09-10T08:20:07.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.408+0000 7f23c7000640 1 -- start start 2024-09-10T08:20:07.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.408+0000 7f23c7000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23c0103e30 0x7f23c019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.408+0000 7f23c7000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c01047e0 0x7f23c019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.408+0000 7f23c7000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c0108f60 0x7f23c019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.408+0000 7f23c7000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0114010 con 0x7f23c0103e30 2024-09-10T08:20:07.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.408+0000 7f23c7000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0113e90 con 0x7f23c01047e0 2024-09-10T08:20:07.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.408+0000 7f23c7000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0114190 con 0x7f23c0108f60 2024-09-10T08:20:07.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.409+0000 7f23c5c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23c0103e30 0x7f23c019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.409+0000 7f23c5c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23c0103e30 0x7f23c019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45684/0 (socket says 172.21.15.2:45684) 2024-09-10T08:20:07.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.409+0000 7f23c5c00640 1 -- 172.21.15.2:0/847547170 learned_addr learned my addr 172.21.15.2:0/847547170 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:07.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.409+0000 7f23c5200640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c01047e0 0x7f23c019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.409+0000 7f23c6600640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c0108f60 0x7f23c019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.409+0000 7f23c5c00640 1 -- 172.21.15.2:0/847547170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c0108f60 msgr2=0x7f23c019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:07.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.409+0000 7f23c5c00640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c0108f60 0x7f23c019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.409+0000 7f23c5c00640 1 -- 172.21.15.2:0/847547170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c01047e0 msgr2=0x7f23c019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:07.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.409+0000 7f23c5c00640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c01047e0 0x7f23c019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.409+0000 7f23c5c00640 1 -- 172.21.15.2:0/847547170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c019f400 con 0x7f23c0103e30 2024-09-10T08:20:07.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.409+0000 7f23c5200640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c01047e0 0x7f23c019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:07.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.409+0000 7f23c5c00640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23c0103e30 0x7f23c019a450 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f23b4004770 tx=0x7f23b4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:07.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.410+0000 7f23be000640 1 -- 172.21.15.2:0/847547170 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b403f070 con 0x7f23c0103e30 2024-09-10T08:20:07.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.410+0000 7f23be000640 1 -- 172.21.15.2:0/847547170 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b4031ca0 con 0x7f23c0103e30 2024-09-10T08:20:07.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.410+0000 7f23c7000640 1 -- 172.21.15.2:0/847547170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c019f690 con 0x7f23c0103e30 2024-09-10T08:20:07.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.410+0000 7f23be000640 1 -- 172.21.15.2:0/847547170 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4035030 con 0x7f23c0103e30 2024-09-10T08:20:07.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.410+0000 7f23c7000640 1 -- 172.21.15.2:0/847547170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c01a6fd0 con 0x7f23c0103e30 2024-09-10T08:20:07.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.413+0000 7f23be000640 1 -- 172.21.15.2:0/847547170 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f23b4042070 con 0x7f23c0103e30 2024-09-10T08:20:07.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.413+0000 7f23cb1e7740 1 Processor -- start 2024-09-10T08:20:07.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.413+0000 7f23cb1e7740 1 -- start start 2024-09-10T08:20:07.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.413+0000 7f23cb1e7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6ae40 0x565508d6b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.413+0000 7f23cb1e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 0x565508d75ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.413+0000 7f23cb1e7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d78410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.413+0000 7f23cb1e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x565508cd13e0 con 0x565508d6b760 2024-09-10T08:20:07.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.413+0000 7f23cb1e7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x565508c78d80 con 0x565508d6ae40 2024-09-10T08:20:07.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.414+0000 7f23cb1e7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565508ccd3a0 con 0x565508d76020 2024-09-10T08:20:07.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.414+0000 7f23c5c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 0x565508d75ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.414+0000 7f23c6600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6ae40 0x565508d6b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.414+0000 7f23c5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d78410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.414+0000 7f23c6600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6ae40 0x565508d6b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50070/0 (socket says 172.21.15.2:50070) 2024-09-10T08:20:07.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.414+0000 7f23c5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d78410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59984/0 (socket says 172.21.15.2:59984) 2024-09-10T08:20:07.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.414+0000 7f23c6600640 1 -- 172.21.15.2:0/2350653600 learned_addr learned my addr 172.21.15.2:0/2350653600 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:07.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.414+0000 7f23c5200640 1 -- 172.21.15.2:0/2350653600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6ae40 msgr2=0x565508d6b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:07.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.414+0000 7f23c5200640 1 --2- 172.21.15.2:0/2350653600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6ae40 0x565508d6b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.414+0000 7f23c5200640 1 -- 172.21.15.2:0/2350653600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 msgr2=0x565508d75ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:07.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.414+0000 7f23c5200640 1 --2- 172.21.15.2:0/2350653600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 0x565508d75ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.414+0000 7f23c5200640 1 -- 172.21.15.2:0/2350653600 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565508c97ae0 con 0x565508d76020 2024-09-10T08:20:07.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.415+0000 7f23c5200640 1 --2- 172.21.15.2:0/2350653600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d78410 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009fa0 tx=0x7f23ac0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a33678600d5aa05b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:07.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.415+0000 7f23be000640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23a80a2b80 0x7f23a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.415+0000 7f23b2a00640 1 -- 172.21.15.2:0/2350653600 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac004310 con 0x565508d76020 2024-09-10T08:20:07.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.415+0000 7f23b2a00640 1 -- 172.21.15.2:0/2350653600 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23ac0044b0 con 0x565508d76020 2024-09-10T08:20:07.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.415+0000 7f23b2a00640 1 -- 172.21.15.2:0/2350653600 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0035f0 con 0x565508d76020 2024-09-10T08:20:07.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.415+0000 7f23be000640 1 -- 172.21.15.2:0/847547170 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f23b40efa70 con 0x7f23c0103e30 2024-09-10T08:20:07.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.415+0000 7f23c6600640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23a80a2b80 0x7f23a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.416+0000 7f23cb1e7740 1 -- 172.21.15.2:0/2350653600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 msgr2=0x565508d78410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:07.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.416+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/2350653600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d78410 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009fa0 tx=0x7f23ac0118d0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.416+0000 7f23cb1e7740 1 -- 172.21.15.2:0/2350653600 shutdown_connections 2024-09-10T08:20:07.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.416+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/2350653600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d78410 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.416+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/2350653600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 0x565508d75ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.416+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/2350653600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6ae40 0x565508d6b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.416+0000 7f23cb1e7740 1 -- 172.21.15.2:0/2350653600 >> 172.21.15.2:0/2350653600 conn(0x565508d673d0 msgr2=0x565508d677b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:07.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.416+0000 7f23c6600640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23a80a2b80 0x7f23a80a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f23b8011630 tx=0x7f23b8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:07.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.416+0000 7f23cb1e7740 1 -- 172.21.15.2:0/2350653600 shutdown_connections 2024-09-10T08:20:07.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.416+0000 7f23cb1e7740 1 -- 172.21.15.2:0/2350653600 wait complete. 2024-09-10T08:20:07.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.417+0000 7f23cb1e7740 1 -- start start 2024-09-10T08:20:07.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.417+0000 7f23cb1e7740 1 Processor -- start 2024-09-10T08:20:07.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.417+0000 7f23cb1e7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6ae40 0x565508d72e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.417+0000 7f23cb1e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 0x565508d733c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.417+0000 7f23cb1e7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d73900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.417+0000 7f23cb1e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x565508d78bf0 con 0x565508d6b760 2024-09-10T08:20:07.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.417+0000 7f23cb1e7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x565508c8c0f0 con 0x565508d6ae40 2024-09-10T08:20:07.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.417+0000 7f23cb1e7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565508d78d70 con 0x565508d76020 2024-09-10T08:20:07.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.417+0000 7f23cb1e7740 10 client.0 _ll_register_callbacks cb 0x565508c73bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:07.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.418+0000 7f23c5200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 0x565508d733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.418+0000 7f23c5c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6ae40 0x565508d72e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.418+0000 7f23c5200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 0x565508d733c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45694/0 (socket says 172.21.15.2:45694) 2024-09-10T08:20:07.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.418+0000 7f23c5200640 1 -- 172.21.15.2:0/1138378498 learned_addr learned my addr 172.21.15.2:0/1138378498 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:07.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.418+0000 7f23c6600640 1 --2- 172.21.15.2:0/1138378498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d73900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.418+0000 7f23c5c00640 1 -- 172.21.15.2:0/1138378498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 msgr2=0x565508d73900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:07.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.418+0000 7f23c5c00640 1 --2- 172.21.15.2:0/1138378498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d73900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.418+0000 7f23c5c00640 1 -- 172.21.15.2:0/1138378498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 msgr2=0x565508d733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:07.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.418+0000 7f23c5c00640 1 --2- 172.21.15.2:0/1138378498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 0x565508d733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.418+0000 7f23c5c00640 1 -- 172.21.15.2:0/1138378498 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565508a423c0 con 0x565508d6ae40 2024-09-10T08:20:07.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.418+0000 7f23c5c00640 1 --2- 172.21.15.2:0/1138378498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6ae40 0x565508d72e80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x565508d68230 tx=0x7f23b403b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f429e7e005c9bba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:07.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.418+0000 7f23a7400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:20:07.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.419+0000 7f23a7400640 1 -- 172.21.15.2:0/1138378498 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b40efe60 con 0x565508d6ae40 2024-09-10T08:20:07.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.419+0000 7f23a7400640 1 -- 172.21.15.2:0/1138378498 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f23b40b8660 con 0x565508d6ae40 2024-09-10T08:20:07.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.419+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1138378498 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23ac0044b0 con 0x565508d6ae40 2024-09-10T08:20:07.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.419+0000 7f23a7400640 1 -- 172.21.15.2:0/1138378498 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f23b4049070 con 0x565508d6ae40 2024-09-10T08:20:07.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.419+0000 7f23cb1e7740 10 client.28823 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:07.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.419+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1138378498 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565508d84bb0 con 0x565508d6ae40 2024-09-10T08:20:07.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.419+0000 7f23a7400640 1 -- 172.21.15.2:0/1138378498 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23ac0044b0 con 0x565508d6ae40 2024-09-10T08:20:07.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.419+0000 7f23a7400640 1 -- 172.21.15.2:0/1138378498 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f23b403b4f0 con 0x565508d6ae40 2024-09-10T08:20:07.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.419+0000 7f23cb1e7740 10 client.28823 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:07.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.419+0000 7f23cb1e7740 10 client.28823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:07.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.419+0000 7f23cb1e7740 20 client.28823 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:07.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.420+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/1138378498 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x565508d85f70 0x565508d883e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.420+0000 7f23cb1e7740 4 client.28823 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:07.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.420+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1138378498 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565508cde950 con 0x565508d85f70 2024-09-10T08:20:07.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.420+0000 7f23c5200640 1 --2- 172.21.15.2:0/1138378498 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x565508d85f70 0x565508d883e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.421+0000 7f23c5200640 1 --2- 172.21.15.2:0/1138378498 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x565508d85f70 0x565508d883e0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92879c48402ea84e server_cookie=cb7002e61fdfe9a6 in_seq=0 out_seq=0 2024-09-10T08:20:07.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.421+0000 7f23a7400640 10 client.28823 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:07.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.421+0000 7f23a7400640 1 -- 172.21.15.2:0/1138378498 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565508cde950 con 0x565508d85f70 2024-09-10T08:20:07.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.421+0000 7f23a7400640 10 client.28823 handle_command_reply: tid=0 2024-09-10T08:20:07.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.421+0000 7f23cb1e7740 1 client.28823 shutdown 2024-09-10T08:20:07.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.422+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1138378498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6ae40 msgr2=0x565508d72e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:07.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.422+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/1138378498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6ae40 0x565508d72e80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x565508d68230 tx=0x7f23b403b460 comp rx=0 tx=0).stop 2024-09-10T08:20:07.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.422+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1138378498 shutdown_connections 2024-09-10T08:20:07.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.422+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/1138378498 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x565508d85f70 0x565508d883e0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.422+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/1138378498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d73900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.422+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/1138378498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 0x565508d733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.422+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/1138378498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6ae40 0x565508d72e80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.422+0000 7f23a7400640 0 client.28823 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:07.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.422+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1138378498 >> 172.21.15.2:0/1138378498 conn(0x565508d673d0 msgr2=0x565508d6d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:07.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.422+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1138378498 shutdown_connections 2024-09-10T08:20:07.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.422+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1138378498 wait complete. 2024-09-10T08:20:07.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.422+0000 7f23cb1e7740 20 client.28823 trim_cache size 0 max 16384 2024-09-10T08:20:07.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.436+0000 7f23cb1e7740 1 Processor -- start 2024-09-10T08:20:07.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.436+0000 7f23cb1e7740 1 -- start start 2024-09-10T08:20:07.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.436+0000 7f23cb1e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6ae40 0x565508d6d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.436+0000 7f23cb1e7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6b760 0x565508d8dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.436+0000 7f23cb1e7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d8e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.436+0000 7f23cb1e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01b3720 con 0x565508d6ae40 2024-09-10T08:20:07.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.436+0000 7f23cb1e7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01b0ff0 con 0x565508d6b760 2024-09-10T08:20:07.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.436+0000 7f23cb1e7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01aee40 con 0x565508d76020 2024-09-10T08:20:07.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.437+0000 7f23c5c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6ae40 0x565508d6d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.437+0000 7f23c5200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6b760 0x565508d8dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.437+0000 7f23c5200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6b760 0x565508d8dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50082/0 (socket says 172.21.15.2:50082) 2024-09-10T08:20:07.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.437+0000 7f23c5200640 1 -- 172.21.15.2:0/3399252985 learned_addr learned my addr 172.21.15.2:0/3399252985 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:07.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.437+0000 7f23c6600640 1 --2- 172.21.15.2:0/3399252985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d8e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.437+0000 7f23c5200640 1 -- 172.21.15.2:0/3399252985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 msgr2=0x565508d8e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:07.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.437+0000 7f23c5200640 1 --2- 172.21.15.2:0/3399252985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d8e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.437+0000 7f23c5200640 1 -- 172.21.15.2:0/3399252985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6ae40 msgr2=0x565508d6d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:07.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.437+0000 7f23c5200640 1 --2- 172.21.15.2:0/3399252985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6ae40 0x565508d6d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.437+0000 7f23c5200640 1 -- 172.21.15.2:0/3399252985 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565508d6b590 con 0x565508d6b760 2024-09-10T08:20:07.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.437+0000 7f23c5200640 1 --2- 172.21.15.2:0/3399252985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6b760 0x565508d8dcb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f23ac004be0 tx=0x7f23ac004c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee8565fb35c270c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:07.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.438+0000 7f23b2000640 1 -- 172.21.15.2:0/3399252985 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac01d040 con 0x565508d6b760 2024-09-10T08:20:07.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.438+0000 7f23b2000640 1 -- 172.21.15.2:0/3399252985 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23ac0054b0 con 0x565508d6b760 2024-09-10T08:20:07.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.438+0000 7f23b2000640 1 -- 172.21.15.2:0/3399252985 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0035f0 con 0x565508d6b760 2024-09-10T08:20:07.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.438+0000 7f23cb1e7740 1 -- 172.21.15.2:0/3399252985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6b760 msgr2=0x565508d8dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:07.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.438+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/3399252985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6b760 0x565508d8dcb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f23ac004be0 tx=0x7f23ac004c10 comp rx=0 tx=0).stop 2024-09-10T08:20:07.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.438+0000 7f23cb1e7740 1 -- 172.21.15.2:0/3399252985 shutdown_connections 2024-09-10T08:20:07.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.438+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/3399252985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d76020 0x565508d8e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.438+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/3399252985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d6b760 0x565508d8dcb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.438+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/3399252985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6ae40 0x565508d6d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.438+0000 7f23cb1e7740 1 -- 172.21.15.2:0/3399252985 >> 172.21.15.2:0/3399252985 conn(0x565508d673d0 msgr2=0x565508d74f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:07.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.439+0000 7f23cb1e7740 1 -- 172.21.15.2:0/3399252985 shutdown_connections 2024-09-10T08:20:07.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.439+0000 7f23cb1e7740 1 -- 172.21.15.2:0/3399252985 wait complete. 2024-09-10T08:20:07.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.439+0000 7f23cb1e7740 1 -- start start 2024-09-10T08:20:07.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.439+0000 7f23cb1e7740 1 Processor -- start 2024-09-10T08:20:07.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.440+0000 7f23cb1e7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d6ae40 0x565508d7e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.440+0000 7f23cb1e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 0x565508d79c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.440+0000 7f23cb1e7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d76020 0x565508d7a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.440+0000 7f23cb1e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01af4b0 con 0x565508d6b760 2024-09-10T08:20:07.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.440+0000 7f23cb1e7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01b0920 con 0x565508d76020 2024-09-10T08:20:07.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.440+0000 7f23cb1e7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01afb80 con 0x565508d6ae40 2024-09-10T08:20:07.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.440+0000 7f23cb1e7740 10 client.0 _ll_register_callbacks cb 0x7f23b40f5200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:07.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.440+0000 7f23c5c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d6ae40 0x565508d7e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.440+0000 7f23c5c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d6ae40 0x565508d7e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60012/0 (socket says 172.21.15.2:60012) 2024-09-10T08:20:07.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.440+0000 7f23c5c00640 1 -- 172.21.15.2:0/1951012255 learned_addr learned my addr 172.21.15.2:0/1951012255 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:07.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.440+0000 7f23c6600640 1 --2- 172.21.15.2:0/1951012255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d76020 0x565508d7a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.440+0000 7f23c5200640 1 --2- 172.21.15.2:0/1951012255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 0x565508d79c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.441+0000 7f23c5c00640 1 -- 172.21.15.2:0/1951012255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d76020 msgr2=0x565508d7a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:07.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.441+0000 7f23c5c00640 1 --2- 172.21.15.2:0/1951012255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d76020 0x565508d7a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.441+0000 7f23c5c00640 1 -- 172.21.15.2:0/1951012255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 msgr2=0x565508d79c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:07.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.441+0000 7f23c5c00640 1 --2- 172.21.15.2:0/1951012255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 0x565508d79c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.441+0000 7f23c5c00640 1 -- 172.21.15.2:0/1951012255 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565508defc80 con 0x565508d6ae40 2024-09-10T08:20:07.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.441+0000 7f23c5c00640 1 --2- 172.21.15.2:0/1951012255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d6ae40 0x565508d7e8e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f23b40ba540 tx=0x7f23b40ed3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f9f53238ba71bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.441+0000 7f23a7400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:20:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.441+0000 7f23a7400640 1 -- 172.21.15.2:0/1951012255 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b40aa020 con 0x565508d6ae40 2024-09-10T08:20:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.441+0000 7f23a7400640 1 -- 172.21.15.2:0/1951012255 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f23b40c4920 con 0x565508d6ae40 2024-09-10T08:20:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.441+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1951012255 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23ac0054b0 con 0x565508d6ae40 2024-09-10T08:20:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.441+0000 7f23a7400640 1 -- 172.21.15.2:0/1951012255 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f23b40b2050 con 0x565508d6ae40 2024-09-10T08:20:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.442+0000 7f23cb1e7740 10 client.28762 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.442+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1951012255 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565508df0e60 con 0x565508d6ae40 2024-09-10T08:20:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.442+0000 7f23a7400640 1 -- 172.21.15.2:0/1951012255 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23ac0054b0 con 0x565508d6ae40 2024-09-10T08:20:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.442+0000 7f23a7400640 1 -- 172.21.15.2:0/1951012255 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f23b40bc250 con 0x565508d6ae40 2024-09-10T08:20:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.442+0000 7f23cb1e7740 10 client.28762 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.442+0000 7f23cb1e7740 10 client.28762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.442+0000 7f23cb1e7740 20 client.28762 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.442+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/1951012255 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x565508df13f0 0x565508df37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.442+0000 7f23cb1e7740 4 client.28762 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:20:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.442+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1951012255 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565508cd8e60 con 0x565508df13f0 2024-09-10T08:20:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.442+0000 7f23c5200640 1 --2- 172.21.15.2:0/1951012255 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x565508df13f0 0x565508df37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.443+0000 7f23c5200640 1 --2- 172.21.15.2:0/1951012255 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x565508df13f0 0x565508df37e0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86dd5c70c24c9fe6 server_cookie=839d4c3b897cce3f in_seq=0 out_seq=0 2024-09-10T08:20:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.443+0000 7f23a7400640 10 client.28762 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.443+0000 7f23a7400640 1 -- 172.21.15.2:0/1951012255 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565508cd8e60 con 0x565508df13f0 2024-09-10T08:20:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.443+0000 7f23a7400640 10 client.28762 handle_command_reply: tid=0 2024-09-10T08:20:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.443+0000 7f23cb1e7740 1 client.28762 shutdown 2024-09-10T08:20:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.443+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1951012255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d6ae40 msgr2=0x565508d7e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.443+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/1951012255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d6ae40 0x565508d7e8e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f23b40ba540 tx=0x7f23b40ed3d0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1951012255 shutdown_connections 2024-09-10T08:20:07.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/1951012255 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x565508df13f0 0x565508df37e0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/1951012255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565508d76020 0x565508d7a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/1951012255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565508d6b760 0x565508d79c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23cb1e7740 1 --2- 172.21.15.2:0/1951012255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565508d6ae40 0x565508d7e8e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1951012255 >> 172.21.15.2:0/1951012255 conn(0x565508d673d0 msgr2=0x565508d65c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:07.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23a7400640 0 client.28762 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:07.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1951012255 shutdown_connections 2024-09-10T08:20:07.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23cb1e7740 1 -- 172.21.15.2:0/1951012255 wait complete. 2024-09-10T08:20:07.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23cb1e7740 20 client.28762 trim_cache size 0 max 16384 2024-09-10T08:20:07.449 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:20:07.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23c7000640 1 -- 172.21.15.2:0/847547170 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23a80a2b80 msgr2=0x7f23a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23c7000640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23a80a2b80 0x7f23a80a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f23b8011630 tx=0x7f23b8009500 comp rx=0 tx=0).stop 2024-09-10T08:20:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23c7000640 1 -- 172.21.15.2:0/847547170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23c0103e30 msgr2=0x7f23c019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.444+0000 7f23c7000640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23c0103e30 0x7f23c019a450 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f23b4004770 tx=0x7f23b4003c10 comp rx=0 tx=0).stop 2024-09-10T08:20:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.445+0000 7f23c7000640 1 -- 172.21.15.2:0/847547170 shutdown_connections 2024-09-10T08:20:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.445+0000 7f23c7000640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f23a80a2b80 0x7f23a80a5040 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.445+0000 7f23c7000640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c0108f60 0x7f23c019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.445+0000 7f23c7000640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c01047e0 0x7f23c019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.445+0000 7f23c7000640 1 --2- 172.21.15.2:0/847547170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23c0103e30 0x7f23c019a450 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.445+0000 7f23c7000640 1 -- 172.21.15.2:0/847547170 >> 172.21.15.2:0/847547170 conn(0x7f23c00ff8d0 msgr2=0x7f23c0100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.445+0000 7f23c7000640 1 -- 172.21.15.2:0/847547170 shutdown_connections 2024-09-10T08:20:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:07.445+0000 7f23c7000640 1 -- 172.21.15.2:0/847547170 wait complete. 2024-09-10T08:20:07.832 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:07 smithi002 ceph-mon[30563]: pgmap v746: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:20:07.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:07 smithi155 ceph-mon[35405]: pgmap v746: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:20:07.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:07 smithi107 ceph-mon[35048]: pgmap v746: 129 pgs: 129 active+clean; 2.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:20:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.323+0000 7fbf29400640 1 -- 172.21.15.2:0/3276635209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf24105910 msgr2=0x7fbf2410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.323+0000 7fbf29400640 1 --2- 172.21.15.2:0/3276635209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf24105910 0x7fbf2410c1a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009ca0 tx=0x7fbf1c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.323+0000 7fbf29400640 1 -- 172.21.15.2:0/3276635209 shutdown_connections 2024-09-10T08:20:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.323+0000 7fbf29400640 1 --2- 172.21.15.2:0/3276635209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf24105910 0x7fbf2410c1a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.323+0000 7fbf29400640 1 --2- 172.21.15.2:0/3276635209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf24104f50 0x7fbf241053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.323+0000 7fbf29400640 1 --2- 172.21.15.2:0/3276635209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf24103d60 0x7fbf24104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.323+0000 7fbf29400640 1 -- 172.21.15.2:0/3276635209 >> 172.21.15.2:0/3276635209 conn(0x7fbf240ff4e0 msgr2=0x7fbf24101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.323+0000 7fbf29400640 1 -- 172.21.15.2:0/3276635209 shutdown_connections 2024-09-10T08:20:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.323+0000 7fbf29400640 1 -- 172.21.15.2:0/3276635209 wait complete. 2024-09-10T08:20:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.324+0000 7fbf29400640 1 Processor -- start 2024-09-10T08:20:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.324+0000 7fbf29400640 1 -- start start 2024-09-10T08:20:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.324+0000 7fbf29400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf24103d60 0x7fbf24199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.324+0000 7fbf29400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf24104f50 0x7fbf2419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.324+0000 7fbf29400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf24105910 0x7fbf241a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.324+0000 7fbf29400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2410f9e0 con 0x7fbf24103d60 2024-09-10T08:20:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.324+0000 7fbf29400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2410f860 con 0x7fbf24104f50 2024-09-10T08:20:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.324+0000 7fbf29400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2410fb60 con 0x7fbf24105910 2024-09-10T08:20:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.325+0000 7fbf22800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf24103d60 0x7fbf24199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.325+0000 7fbf19e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf24104f50 0x7fbf2419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.325+0000 7fbf22800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf24103d60 0x7fbf24199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45728/0 (socket says 172.21.15.2:45728) 2024-09-10T08:20:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.325+0000 7fbf22800640 1 -- 172.21.15.2:0/1122699698 learned_addr learned my addr 172.21.15.2:0/1122699698 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.325+0000 7fbf23200640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf24105910 0x7fbf241a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.325+0000 7fbf19e00640 1 -- 172.21.15.2:0/1122699698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf24105910 msgr2=0x7fbf241a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.325+0000 7fbf19e00640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf24105910 0x7fbf241a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.325+0000 7fbf19e00640 1 -- 172.21.15.2:0/1122699698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf24103d60 msgr2=0x7fbf24199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.325+0000 7fbf19e00640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf24103d60 0x7fbf24199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.325+0000 7fbf19e00640 1 -- 172.21.15.2:0/1122699698 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf241a1b70 con 0x7fbf24104f50 2024-09-10T08:20:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.325+0000 7fbf22800640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf24103d60 0x7fbf24199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.325+0000 7fbf19e00640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf24104f50 0x7fbf2419a490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00e9f0 tx=0x7fbf0c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.326+0000 7fbf1be00640 1 -- 172.21.15.2:0/1122699698 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c00ccc0 con 0x7fbf24104f50 2024-09-10T08:20:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.326+0000 7fbf1be00640 1 -- 172.21.15.2:0/1122699698 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf0c010910 con 0x7fbf24104f50 2024-09-10T08:20:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.326+0000 7fbf29400640 1 -- 172.21.15.2:0/1122699698 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf241a1e60 con 0x7fbf24104f50 2024-09-10T08:20:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.326+0000 7fbf1be00640 1 -- 172.21.15.2:0/1122699698 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c018960 con 0x7fbf24104f50 2024-09-10T08:20:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.326+0000 7fbf29400640 1 -- 172.21.15.2:0/1122699698 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf241a2320 con 0x7fbf24104f50 2024-09-10T08:20:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf2c276740 1 Processor -- start 2024-09-10T08:20:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf2c276740 1 -- start start 2024-09-10T08:20:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf2c276740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d6615090 0x5560d6615490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf2c276740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 0x5560d661a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf2c276740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d661aa60 0x5560d661cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf2c276740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5560d657a100 con 0x5560d66159d0 2024-09-10T08:20:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf2c276740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5560d65221b0 con 0x5560d6615090 2024-09-10T08:20:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf2c276740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5560d65760c0 con 0x5560d661aa60 2024-09-10T08:20:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf1be00640 1 -- 172.21.15.2:0/1122699698 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbf0c010ac0 con 0x7fbf24104f50 2024-09-10T08:20:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf22800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 0x5560d661a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf22800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 0x5560d661a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45732/0 (socket says 172.21.15.2:45732) 2024-09-10T08:20:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf22800640 1 -- 172.21.15.2:0/2617485591 learned_addr learned my addr 172.21.15.2:0/2617485591 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf23200640 1 --2- 172.21.15.2:0/2617485591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d6615090 0x5560d6615490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.329+0000 7fbf19e00640 1 --2- 172.21.15.2:0/2617485591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d661aa60 0x5560d661cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.330+0000 7fbf22800640 1 -- 172.21.15.2:0/2617485591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d661aa60 msgr2=0x5560d661cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.330+0000 7fbf22800640 1 --2- 172.21.15.2:0/2617485591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d661aa60 0x5560d661cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.330+0000 7fbf22800640 1 -- 172.21.15.2:0/2617485591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d6615090 msgr2=0x5560d6615490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.330+0000 7fbf22800640 1 --2- 172.21.15.2:0/2617485591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d6615090 0x5560d6615490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.330+0000 7fbf22800640 1 -- 172.21.15.2:0/2617485591 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d628d210 con 0x5560d66159d0 2024-09-10T08:20:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.330+0000 7fbf19e00640 1 --2- 172.21.15.2:0/2617485591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d661aa60 0x5560d661cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:20:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.330+0000 7fbf22800640 1 --2- 172.21.15.2:0/2617485591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 0x5560d661a490 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fbf140115f0 tx=0x7fbf14011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b98a30775a14b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.330+0000 7fbf13400640 1 -- 172.21.15.2:0/2617485591 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14004020 con 0x5560d66159d0 2024-09-10T08:20:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.330+0000 7fbf13400640 1 -- 172.21.15.2:0/2617485591 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf140041c0 con 0x5560d66159d0 2024-09-10T08:20:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.330+0000 7fbf13400640 1 -- 172.21.15.2:0/2617485591 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14003980 con 0x5560d66159d0 2024-09-10T08:20:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.331+0000 7fbf1be00640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbf080a2c30 0x7fbf080a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.331+0000 7fbf1be00640 1 -- 172.21.15.2:0/1122699698 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbf0c0cf7e0 con 0x7fbf24104f50 2024-09-10T08:20:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.331+0000 7fbf23200640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbf080a2c30 0x7fbf080a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.331+0000 7fbf2c276740 1 -- 172.21.15.2:0/2617485591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 msgr2=0x5560d661a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.331+0000 7fbf2c276740 1 --2- 172.21.15.2:0/2617485591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 0x5560d661a490 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fbf140115f0 tx=0x7fbf14011ab0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.331+0000 7fbf2c276740 1 -- 172.21.15.2:0/2617485591 shutdown_connections 2024-09-10T08:20:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.331+0000 7fbf2c276740 1 --2- 172.21.15.2:0/2617485591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d661aa60 0x5560d661cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.331+0000 7fbf2c276740 1 --2- 172.21.15.2:0/2617485591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 0x5560d661a490 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.331+0000 7fbf2c276740 1 --2- 172.21.15.2:0/2617485591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d6615090 0x5560d6615490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.331+0000 7fbf2c276740 1 -- 172.21.15.2:0/2617485591 >> 172.21.15.2:0/2617485591 conn(0x5560d6610700 msgr2=0x5560d6610b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.332+0000 7fbf2c276740 1 -- 172.21.15.2:0/2617485591 shutdown_connections 2024-09-10T08:20:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.332+0000 7fbf2c276740 1 -- 172.21.15.2:0/2617485591 wait complete. 2024-09-10T08:20:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.332+0000 7fbf23200640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbf080a2c30 0x7fbf080a50f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c005e30 tx=0x7fbf1c03d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.332+0000 7fbf2c276740 1 -- start start 2024-09-10T08:20:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.332+0000 7fbf2c276740 1 Processor -- start 2024-09-10T08:20:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf2c276740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d6615090 0x5560d661f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf2c276740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 0x5560d661f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf2c276740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d661aa60 0x5560d6626730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf2c276740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5560d661da30 con 0x5560d66159d0 2024-09-10T08:20:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf2c276740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5560d6503090 con 0x5560d661aa60 2024-09-10T08:20:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf2c276740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5560d661dbb0 con 0x5560d6615090 2024-09-10T08:20:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf2c276740 10 client.0 _ll_register_callbacks cb 0x5560d6557f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf19e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 0x5560d661f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf19e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 0x5560d661f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45748/0 (socket says 172.21.15.2:45748) 2024-09-10T08:20:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf19e00640 1 -- 172.21.15.2:0/3757962072 learned_addr learned my addr 172.21.15.2:0/3757962072 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf22800640 1 --2- 172.21.15.2:0/3757962072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d6615090 0x5560d661f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf23200640 1 --2- 172.21.15.2:0/3757962072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d661aa60 0x5560d6626730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf19e00640 1 -- 172.21.15.2:0/3757962072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d6615090 msgr2=0x5560d661f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf19e00640 1 --2- 172.21.15.2:0/3757962072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d6615090 0x5560d661f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf19e00640 1 -- 172.21.15.2:0/3757962072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d661aa60 msgr2=0x5560d6626730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf19e00640 1 --2- 172.21.15.2:0/3757962072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d661aa60 0x5560d6626730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.333+0000 7fbf19e00640 1 -- 172.21.15.2:0/3757962072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d6359620 con 0x5560d66159d0 2024-09-10T08:20:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.334+0000 7fbf19e00640 1 --2- 172.21.15.2:0/3757962072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 0x5560d661f6f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c098100 tx=0x7fbf0c09bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf5c450cc83962b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.334+0000 7fbf07e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.334+0000 7fbf07e00640 1 -- 172.21.15.2:0/3757962072 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0cfdd0 con 0x5560d66159d0 2024-09-10T08:20:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.334+0000 7fbf07e00640 1 -- 172.21.15.2:0/3757962072 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbf0c09b0f0 con 0x5560d66159d0 2024-09-10T08:20:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.334+0000 7fbf2c276740 1 -- 172.21.15.2:0/3757962072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf140041c0 con 0x5560d66159d0 2024-09-10T08:20:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.334+0000 7fbf07e00640 1 -- 172.21.15.2:0/3757962072 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf0c09ffa0 con 0x5560d66159d0 2024-09-10T08:20:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.334+0000 7fbf2c276740 10 client.18816 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.334+0000 7fbf2c276740 1 -- 172.21.15.2:0/3757962072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5560d66295e0 con 0x5560d66159d0 2024-09-10T08:20:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.334+0000 7fbf07e00640 1 -- 172.21.15.2:0/3757962072 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf140041c0 con 0x5560d66159d0 2024-09-10T08:20:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.335+0000 7fbf07e00640 1 -- 172.21.15.2:0/3757962072 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf0c0d1ce0 con 0x5560d66159d0 2024-09-10T08:20:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.335+0000 7fbf2c276740 10 client.18816 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.335+0000 7fbf2c276740 10 client.18816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.335+0000 7fbf2c276740 20 client.18816 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.335+0000 7fbf2c276740 1 --2- 172.21.15.2:0/3757962072 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5560d662ac60 0x5560d662d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.335+0000 7fbf2c276740 4 client.18816 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.335+0000 7fbf2c276740 1 -- 172.21.15.2:0/3757962072 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560d6587830 con 0x5560d662ac60 2024-09-10T08:20:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.335+0000 7fbf22800640 1 --2- 172.21.15.2:0/3757962072 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5560d662ac60 0x5560d662d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.336+0000 7fbf22800640 1 --2- 172.21.15.2:0/3757962072 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5560d662ac60 0x5560d662d050 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f57e2f0c8890cd10 server_cookie=40bcb3434649ed5 in_seq=0 out_seq=0 2024-09-10T08:20:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.336+0000 7fbf07e00640 10 client.18816 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.336+0000 7fbf07e00640 1 -- 172.21.15.2:0/3757962072 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560d6587830 con 0x5560d662ac60 2024-09-10T08:20:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.336+0000 7fbf07e00640 10 client.18816 handle_command_reply: tid=0 2024-09-10T08:20:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.336+0000 7fbf2c276740 1 client.18816 shutdown 2024-09-10T08:20:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.337+0000 7fbf2c276740 1 -- 172.21.15.2:0/3757962072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 msgr2=0x5560d661f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.337+0000 7fbf2c276740 1 --2- 172.21.15.2:0/3757962072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 0x5560d661f6f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c098100 tx=0x7fbf0c09bb60 comp rx=0 tx=0).stop 2024-09-10T08:20:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.337+0000 7fbf2c276740 1 -- 172.21.15.2:0/3757962072 shutdown_connections 2024-09-10T08:20:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.337+0000 7fbf2c276740 1 --2- 172.21.15.2:0/3757962072 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5560d662ac60 0x5560d662d050 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.337+0000 7fbf2c276740 1 --2- 172.21.15.2:0/3757962072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d661aa60 0x5560d6626730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.337+0000 7fbf2c276740 1 --2- 172.21.15.2:0/3757962072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d66159d0 0x5560d661f6f0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.337+0000 7fbf2c276740 1 --2- 172.21.15.2:0/3757962072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d6615090 0x5560d661f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.337+0000 7fbf07e00640 0 client.18816 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.337+0000 7fbf2c276740 1 -- 172.21.15.2:0/3757962072 >> 172.21.15.2:0/3757962072 conn(0x5560d6610700 msgr2=0x5560d6611f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.337+0000 7fbf2c276740 1 -- 172.21.15.2:0/3757962072 shutdown_connections 2024-09-10T08:20:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.337+0000 7fbf2c276740 1 -- 172.21.15.2:0/3757962072 wait complete. 2024-09-10T08:20:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.337+0000 7fbf2c276740 20 client.18816 trim_cache size 0 max 16384 2024-09-10T08:20:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.352+0000 7fbf2c276740 1 Processor -- start 2024-09-10T08:20:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.352+0000 7fbf2c276740 1 -- start start 2024-09-10T08:20:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.352+0000 7fbf2c276740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d6615090 0x5560d66115b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.352+0000 7fbf2c276740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d66159d0 0x5560d66266f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.352+0000 7fbf2c276740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d661aa60 0x5560d6611af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.352+0000 7fbf2c276740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241af2d0 con 0x5560d661aa60 2024-09-10T08:20:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.352+0000 7fbf2c276740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241acba0 con 0x5560d66159d0 2024-09-10T08:20:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.352+0000 7fbf2c276740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241aa9f0 con 0x5560d6615090 2024-09-10T08:20:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.352+0000 7fbf23200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d661aa60 0x5560d6611af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.353+0000 7fbf22800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d6615090 0x5560d66115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.353+0000 7fbf23200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d661aa60 0x5560d6611af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45758/0 (socket says 172.21.15.2:45758) 2024-09-10T08:20:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.353+0000 7fbf23200640 1 -- 172.21.15.2:0/294234176 learned_addr learned my addr 172.21.15.2:0/294234176 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.353+0000 7fbf19e00640 1 --2- 172.21.15.2:0/294234176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d66159d0 0x5560d66266f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.353+0000 7fbf22800640 1 -- 172.21.15.2:0/294234176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d66159d0 msgr2=0x5560d66266f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.353+0000 7fbf22800640 1 --2- 172.21.15.2:0/294234176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d66159d0 0x5560d66266f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.353+0000 7fbf22800640 1 -- 172.21.15.2:0/294234176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d661aa60 msgr2=0x5560d6611af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.353+0000 7fbf22800640 1 --2- 172.21.15.2:0/294234176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d661aa60 0x5560d6611af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.353+0000 7fbf22800640 1 -- 172.21.15.2:0/294234176 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d6616660 con 0x5560d6615090 2024-09-10T08:20:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.353+0000 7fbf22800640 1 --2- 172.21.15.2:0/294234176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d6615090 0x5560d66115b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf140115c0 tx=0x7fbf14009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6d6d9da6b99251c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.353+0000 7fbf12a00640 1 -- 172.21.15.2:0/294234176 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400f830 con 0x5560d6615090 2024-09-10T08:20:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.354+0000 7fbf12a00640 1 -- 172.21.15.2:0/294234176 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf14005040 con 0x5560d6615090 2024-09-10T08:20:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.354+0000 7fbf12a00640 1 -- 172.21.15.2:0/294234176 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf140047a0 con 0x5560d6615090 2024-09-10T08:20:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.354+0000 7fbf2c276740 1 -- 172.21.15.2:0/294234176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d6615090 msgr2=0x5560d66115b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.354+0000 7fbf2c276740 1 --2- 172.21.15.2:0/294234176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d6615090 0x5560d66115b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf140115c0 tx=0x7fbf14009330 comp rx=0 tx=0).stop 2024-09-10T08:20:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.354+0000 7fbf2c276740 1 -- 172.21.15.2:0/294234176 shutdown_connections 2024-09-10T08:20:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.354+0000 7fbf2c276740 1 --2- 172.21.15.2:0/294234176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d661aa60 0x5560d6611af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.354+0000 7fbf2c276740 1 --2- 172.21.15.2:0/294234176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d66159d0 0x5560d66266f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.354+0000 7fbf2c276740 1 --2- 172.21.15.2:0/294234176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d6615090 0x5560d66115b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.354+0000 7fbf2c276740 1 -- 172.21.15.2:0/294234176 >> 172.21.15.2:0/294234176 conn(0x5560d6610700 msgr2=0x5560d661f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.354+0000 7fbf2c276740 1 -- 172.21.15.2:0/294234176 shutdown_connections 2024-09-10T08:20:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.354+0000 7fbf2c276740 1 -- 172.21.15.2:0/294234176 wait complete. 2024-09-10T08:20:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf2c276740 1 -- start start 2024-09-10T08:20:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf2c276740 1 Processor -- start 2024-09-10T08:20:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf2c276740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d6615090 0x5560d6624090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf2c276740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d66159d0 0x5560d66245d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf2c276740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d661aa60 0x5560d6625570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf2c276740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241ab060 con 0x5560d6615090 2024-09-10T08:20:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf2c276740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241ac4d0 con 0x5560d66159d0 2024-09-10T08:20:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf2c276740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241ab730 con 0x5560d661aa60 2024-09-10T08:20:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf2c276740 10 client.0 _ll_register_callbacks cb 0x7fbf0c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf22800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d6615090 0x5560d6624090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf23200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d661aa60 0x5560d6625570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf22800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d6615090 0x5560d6624090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45766/0 (socket says 172.21.15.2:45766) 2024-09-10T08:20:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf22800640 1 -- 172.21.15.2:0/1243915962 learned_addr learned my addr 172.21.15.2:0/1243915962 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.355+0000 7fbf19e00640 1 --2- 172.21.15.2:0/1243915962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d66159d0 0x5560d66245d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.356+0000 7fbf22800640 1 -- 172.21.15.2:0/1243915962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d661aa60 msgr2=0x5560d6625570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.356+0000 7fbf22800640 1 --2- 172.21.15.2:0/1243915962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d661aa60 0x5560d6625570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.356+0000 7fbf22800640 1 -- 172.21.15.2:0/1243915962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d66159d0 msgr2=0x5560d66245d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.356+0000 7fbf22800640 1 --2- 172.21.15.2:0/1243915962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d66159d0 0x5560d66245d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.356+0000 7fbf22800640 1 -- 172.21.15.2:0/1243915962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d6625b80 con 0x5560d6615090 2024-09-10T08:20:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.356+0000 7fbf22800640 1 --2- 172.21.15.2:0/1243915962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d6615090 0x5560d6624090 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009250 tx=0x7fbf14004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6292f33db0f475b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.356+0000 7fbf07e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.356+0000 7fbf07e00640 1 -- 172.21.15.2:0/1243915962 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14033020 con 0x5560d6615090 2024-09-10T08:20:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.356+0000 7fbf07e00640 1 -- 172.21.15.2:0/1243915962 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbf14005480 con 0x5560d6615090 2024-09-10T08:20:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.356+0000 7fbf2c276740 1 -- 172.21.15.2:0/1243915962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf14005040 con 0x5560d6615090 2024-09-10T08:20:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.357+0000 7fbf07e00640 1 -- 172.21.15.2:0/1243915962 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf1403e050 con 0x5560d6615090 2024-09-10T08:20:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.357+0000 7fbf2c276740 10 client.18822 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.357+0000 7fbf2c276740 1 -- 172.21.15.2:0/1243915962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5560d6695b90 con 0x5560d6615090 2024-09-10T08:20:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.357+0000 7fbf07e00640 1 -- 172.21.15.2:0/1243915962 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf14005040 con 0x5560d6615090 2024-09-10T08:20:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.357+0000 7fbf07e00640 1 -- 172.21.15.2:0/1243915962 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf14026420 con 0x5560d6615090 2024-09-10T08:20:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.357+0000 7fbf2c276740 10 client.18822 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.357+0000 7fbf2c276740 10 client.18822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.357+0000 7fbf2c276740 20 client.18822 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.357+0000 7fbf2c276740 1 --2- 172.21.15.2:0/1243915962 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5560d66962f0 0x5560d6698710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.357+0000 7fbf2c276740 4 client.18822 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:20:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.357+0000 7fbf2c276740 1 -- 172.21.15.2:0/1243915962 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560d660bd80 con 0x5560d66962f0 2024-09-10T08:20:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.358+0000 7fbf19e00640 1 --2- 172.21.15.2:0/1243915962 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5560d66962f0 0x5560d6698710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.358+0000 7fbf19e00640 1 --2- 172.21.15.2:0/1243915962 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5560d66962f0 0x5560d6698710 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53e89908c78d3445 server_cookie=901cac36546c0a9f in_seq=0 out_seq=0 2024-09-10T08:20:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.358+0000 7fbf07e00640 10 client.18822 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf07e00640 1 -- 172.21.15.2:0/1243915962 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5560d660bd80 con 0x5560d66962f0 2024-09-10T08:20:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf07e00640 10 client.18822 handle_command_reply: tid=0 2024-09-10T08:20:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf2c276740 1 client.18822 shutdown 2024-09-10T08:20:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf2c276740 1 -- 172.21.15.2:0/1243915962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d6615090 msgr2=0x5560d6624090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf2c276740 1 --2- 172.21.15.2:0/1243915962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d6615090 0x5560d6624090 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009250 tx=0x7fbf14004490 comp rx=0 tx=0).stop 2024-09-10T08:20:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf2c276740 1 -- 172.21.15.2:0/1243915962 shutdown_connections 2024-09-10T08:20:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf2c276740 1 --2- 172.21.15.2:0/1243915962 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5560d66962f0 0x5560d6698710 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf2c276740 1 --2- 172.21.15.2:0/1243915962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560d661aa60 0x5560d6625570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf2c276740 1 --2- 172.21.15.2:0/1243915962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560d66159d0 0x5560d66245d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf2c276740 1 --2- 172.21.15.2:0/1243915962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5560d6615090 0x5560d6624090 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf2c276740 1 -- 172.21.15.2:0/1243915962 >> 172.21.15.2:0/1243915962 conn(0x5560d6610700 msgr2=0x5560d660f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf07e00640 0 client.18822 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf2c276740 1 -- 172.21.15.2:0/1243915962 shutdown_connections 2024-09-10T08:20:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf2c276740 1 -- 172.21.15.2:0/1243915962 wait complete. 2024-09-10T08:20:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.371+0000 7fbf2c276740 20 client.18822 trim_cache size 0 max 16384 2024-09-10T08:20:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:20:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:20:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:20:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:20:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.372+0000 7fbf29400640 1 -- 172.21.15.2:0/1122699698 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbf080a2c30 msgr2=0x7fbf080a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.372+0000 7fbf29400640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbf080a2c30 0x7fbf080a50f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c005e30 tx=0x7fbf1c03d040 comp rx=0 tx=0).stop 2024-09-10T08:20:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.372+0000 7fbf29400640 1 -- 172.21.15.2:0/1122699698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf24104f50 msgr2=0x7fbf2419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.372+0000 7fbf29400640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf24104f50 0x7fbf2419a490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00e9f0 tx=0x7fbf0c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.372+0000 7fbf29400640 1 -- 172.21.15.2:0/1122699698 shutdown_connections 2024-09-10T08:20:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.372+0000 7fbf29400640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbf080a2c30 0x7fbf080a50f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.372+0000 7fbf29400640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf24105910 0x7fbf241a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.372+0000 7fbf29400640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf24104f50 0x7fbf2419a490 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.372+0000 7fbf29400640 1 --2- 172.21.15.2:0/1122699698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf24103d60 0x7fbf24199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.372+0000 7fbf29400640 1 -- 172.21.15.2:0/1122699698 >> 172.21.15.2:0/1122699698 conn(0x7fbf240ff4e0 msgr2=0x7fbf24108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.372+0000 7fbf29400640 1 -- 172.21.15.2:0/1122699698 shutdown_connections 2024-09-10T08:20:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:08.372+0000 7fbf29400640 1 -- 172.21.15.2:0/1122699698 wait complete. 2024-09-10T08:20:09.406 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:20:09.407 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:20:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:09 smithi002 ceph-mon[30563]: pgmap v747: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:20:09.800 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:09.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:09 smithi155 ceph-mon[35405]: pgmap v747: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:20:09.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:09 smithi107 ceph-mon[35048]: pgmap v747: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:20:10.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.420+0000 7f3dbd600640 1 -- 172.21.15.2:0/2693034527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3db81049a0 msgr2=0x7f3db81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.420+0000 7f3dbd600640 1 --2- 172.21.15.2:0/2693034527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3db81049a0 0x7f3db81089f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009900 tx=0x7f3da402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.420+0000 7f3dbd600640 1 -- 172.21.15.2:0/2693034527 shutdown_connections 2024-09-10T08:20:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.420+0000 7f3dbd600640 1 --2- 172.21.15.2:0/2693034527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db8109120 0x7f3db8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.420+0000 7f3dbd600640 1 --2- 172.21.15.2:0/2693034527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3db81049a0 0x7f3db81089f0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.420+0000 7f3dbd600640 1 --2- 172.21.15.2:0/2693034527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db8103ff0 0x7f3db81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.420+0000 7f3dbd600640 1 -- 172.21.15.2:0/2693034527 >> 172.21.15.2:0/2693034527 conn(0x7f3db80ffa90 msgr2=0x7f3db8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.420+0000 7f3dbd600640 1 -- 172.21.15.2:0/2693034527 shutdown_connections 2024-09-10T08:20:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.420+0000 7f3dbd600640 1 -- 172.21.15.2:0/2693034527 wait complete. 2024-09-10T08:20:10.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.421+0000 7f3dbd600640 1 Processor -- start 2024-09-10T08:20:10.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.421+0000 7f3dbd600640 1 -- start start 2024-09-10T08:20:10.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.422+0000 7f3dbd600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3db8103ff0 0x7f3db8076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.422+0000 7f3dbd600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db81049a0 0x7f3db8076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.422+0000 7f3dbd600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db8109120 0x7f3db8077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.422+0000 7f3dbd600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8114190 con 0x7f3db8103ff0 2024-09-10T08:20:10.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.422+0000 7f3dbd600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8114010 con 0x7f3db81049a0 2024-09-10T08:20:10.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.422+0000 7f3dbd600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8114310 con 0x7f3db8109120 2024-09-10T08:20:10.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.422+0000 7f3db6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3db8103ff0 0x7f3db8076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.422+0000 7f3db6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3db8103ff0 0x7f3db8076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45788/0 (socket says 172.21.15.2:45788) 2024-09-10T08:20:10.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.422+0000 7f3db6a00640 1 -- 172.21.15.2:0/406664564 learned_addr learned my addr 172.21.15.2:0/406664564 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.422+0000 7f3db6000640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db81049a0 0x7f3db8076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.422+0000 7f3db7400640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db8109120 0x7f3db8077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.423+0000 7f3db6a00640 1 -- 172.21.15.2:0/406664564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db8109120 msgr2=0x7f3db8077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.423+0000 7f3db6a00640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db8109120 0x7f3db8077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.423+0000 7f3db6a00640 1 -- 172.21.15.2:0/406664564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db81049a0 msgr2=0x7f3db8076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.423+0000 7f3db6a00640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db81049a0 0x7f3db8076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.423+0000 7f3db6a00640 1 -- 172.21.15.2:0/406664564 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db81aad10 con 0x7f3db8103ff0 2024-09-10T08:20:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.423+0000 7f3db6a00640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3db8103ff0 0x7f3db8076700 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3da800d980 tx=0x7f3da800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.423+0000 7f3daf400640 1 -- 172.21.15.2:0/406664564 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8017030 con 0x7f3db8103ff0 2024-09-10T08:20:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.423+0000 7f3daf400640 1 -- 172.21.15.2:0/406664564 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3da8004020 con 0x7f3db8103ff0 2024-09-10T08:20:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.423+0000 7f3daf400640 1 -- 172.21.15.2:0/406664564 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da80047a0 con 0x7f3db8103ff0 2024-09-10T08:20:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.423+0000 7f3dbd600640 1 -- 172.21.15.2:0/406664564 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db81aafa0 con 0x7f3db8103ff0 2024-09-10T08:20:10.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.423+0000 7f3dbd600640 1 -- 172.21.15.2:0/406664564 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db81ab400 con 0x7f3db8103ff0 2024-09-10T08:20:10.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.427+0000 7f3daf400640 1 -- 172.21.15.2:0/406664564 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3da8009a30 con 0x7f3db8103ff0 2024-09-10T08:20:10.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.427+0000 7f3dc01d0740 1 Processor -- start 2024-09-10T08:20:10.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.427+0000 7f3dc01d0740 1 -- start start 2024-09-10T08:20:10.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.427+0000 7f3dc01d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b21ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.427+0000 7f3dc01d0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b223e0 0x555a55b2c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.427+0000 7f3dc01d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 0x555a55b2f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.427+0000 7f3dc01d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555a55a88290 con 0x555a55b2cca0 2024-09-10T08:20:10.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.427+0000 7f3dc01d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555a55a2fb40 con 0x555a55b21ac0 2024-09-10T08:20:10.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.427+0000 7f3dc01d0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555a55a84250 con 0x555a55b223e0 2024-09-10T08:20:10.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.428+0000 7f3db6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b223e0 0x555a55b2c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.428+0000 7f3db7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b21ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.428+0000 7f3db6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b223e0 0x555a55b2c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60118/0 (socket says 172.21.15.2:60118) 2024-09-10T08:20:10.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.428+0000 7f3db7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b21ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50176/0 (socket says 172.21.15.2:50176) 2024-09-10T08:20:10.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.428+0000 7f3db6a00640 1 -- 172.21.15.2:0/1764485978 learned_addr learned my addr 172.21.15.2:0/1764485978 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:10.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.428+0000 7f3db6000640 1 --2- 172.21.15.2:0/1764485978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 0x555a55b2f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.428+0000 7f3db7400640 1 -- 172.21.15.2:0/1764485978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b223e0 msgr2=0x555a55b2c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:10.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.428+0000 7f3db7400640 1 --2- 172.21.15.2:0/1764485978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b223e0 0x555a55b2c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.428+0000 7f3db7400640 1 -- 172.21.15.2:0/1764485978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 msgr2=0x555a55b2f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:10.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.428+0000 7f3db7400640 1 --2- 172.21.15.2:0/1764485978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 0x555a55b2f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.428+0000 7f3db7400640 1 -- 172.21.15.2:0/1764485978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a557f96a0 con 0x555a55b21ac0 2024-09-10T08:20:10.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.428+0000 7f3daf400640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3d9c0a2b80 0x7f3d9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.429+0000 7f3daf400640 1 -- 172.21.15.2:0/406664564 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3da80cee50 con 0x7f3db8103ff0 2024-09-10T08:20:10.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.429+0000 7f3db6a00640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3d9c0a2b80 0x7f3d9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.429+0000 7f3db7400640 1 --2- 172.21.15.2:0/1764485978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b21ea0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009f00 tx=0x7f3db0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2936a09f5dfcba2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:10.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.429+0000 7f3da3400640 1 -- 172.21.15.2:0/1764485978 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0004d20 con 0x555a55b21ac0 2024-09-10T08:20:10.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.429+0000 7f3da3400640 1 -- 172.21.15.2:0/1764485978 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db0004050 con 0x555a55b21ac0 2024-09-10T08:20:10.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.429+0000 7f3da3400640 1 -- 172.21.15.2:0/1764485978 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0005280 con 0x555a55b21ac0 2024-09-10T08:20:10.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.430+0000 7f3dc01d0740 1 -- 172.21.15.2:0/1764485978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 msgr2=0x555a55b21ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:10.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.430+0000 7f3db6a00640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3d9c0a2b80 0x7f3d9c0a5040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3da80a39a0 tx=0x7f3da80d2890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:10.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.430+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/1764485978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b21ea0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009f00 tx=0x7f3db0011810 comp rx=0 tx=0).stop 2024-09-10T08:20:10.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.430+0000 7f3dc01d0740 1 -- 172.21.15.2:0/1764485978 shutdown_connections 2024-09-10T08:20:10.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.430+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/1764485978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 0x555a55b2f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.430+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/1764485978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b223e0 0x555a55b2c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.430+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/1764485978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b21ea0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.430+0000 7f3dc01d0740 1 -- 172.21.15.2:0/1764485978 >> 172.21.15.2:0/1764485978 conn(0x555a55b1e050 msgr2=0x555a55b1e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:10.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.430+0000 7f3dc01d0740 1 -- 172.21.15.2:0/1764485978 shutdown_connections 2024-09-10T08:20:10.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.430+0000 7f3dc01d0740 1 -- 172.21.15.2:0/1764485978 wait complete. 2024-09-10T08:20:10.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.431+0000 7f3dc01d0740 1 -- start start 2024-09-10T08:20:10.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.431+0000 7f3dc01d0740 1 Processor -- start 2024-09-10T08:20:10.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.431+0000 7f3dc01d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b2c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.431+0000 7f3dc01d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b223e0 0x555a55b298c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.431+0000 7f3dc01d0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b2cca0 0x555a55b29e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.431+0000 7f3dc01d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555a55b2f910 con 0x555a55b223e0 2024-09-10T08:20:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.431+0000 7f3dc01d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555a55a42fa0 con 0x555a55b21ac0 2024-09-10T08:20:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.431+0000 7f3dc01d0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555a55b2fa90 con 0x555a55b2cca0 2024-09-10T08:20:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.431+0000 7f3dc01d0740 10 client.0 _ll_register_callbacks cb 0x555a55796ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.432+0000 7f3db6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b223e0 0x555a55b298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.432+0000 7f3db7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b2c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.432+0000 7f3db6000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b223e0 0x555a55b298c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45810/0 (socket says 172.21.15.2:45810) 2024-09-10T08:20:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.432+0000 7f3db6000640 1 -- 172.21.15.2:0/1339809084 learned_addr learned my addr 172.21.15.2:0/1339809084 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.432+0000 7f3db7400640 1 --2- 172.21.15.2:0/1339809084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b2cca0 0x555a55b29e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.432+0000 7f3db6000640 1 -- 172.21.15.2:0/1339809084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b2cca0 msgr2=0x555a55b29e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:10.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.432+0000 7f3db6000640 1 --2- 172.21.15.2:0/1339809084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b2cca0 0x555a55b29e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.432+0000 7f3db6000640 1 -- 172.21.15.2:0/1339809084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 msgr2=0x555a55b2c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.432+0000 7f3db6000640 1 --2- 172.21.15.2:0/1339809084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b2c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.432+0000 7f3db6000640 1 -- 172.21.15.2:0/1339809084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a55b2a650 con 0x555a55b223e0 2024-09-10T08:20:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.432+0000 7f3db6000640 1 --2- 172.21.15.2:0/1339809084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b223e0 0x555a55b298c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009900 tx=0x7f3da4004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f56908f7affebd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.432+0000 7f3d97e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.433+0000 7f3d97e00640 1 -- 172.21.15.2:0/1339809084 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4035040 con 0x555a55b223e0 2024-09-10T08:20:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.433+0000 7f3dc01d0740 1 -- 172.21.15.2:0/1339809084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db0004050 con 0x555a55b223e0 2024-09-10T08:20:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.433+0000 7f3d97e00640 1 -- 172.21.15.2:0/1339809084 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3da4005450 con 0x555a55b223e0 2024-09-10T08:20:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.433+0000 7f3d97e00640 1 -- 172.21.15.2:0/1339809084 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3da404f050 con 0x555a55b223e0 2024-09-10T08:20:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.433+0000 7f3dc01d0740 10 client.18834 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.433+0000 7f3dc01d0740 1 -- 172.21.15.2:0/1339809084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a55b3b850 con 0x555a55b223e0 2024-09-10T08:20:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.433+0000 7f3d97e00640 1 -- 172.21.15.2:0/1339809084 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db0004050 con 0x555a55b223e0 2024-09-10T08:20:10.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.434+0000 7f3d97e00640 1 -- 172.21.15.2:0/1339809084 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3da4003a30 con 0x555a55b223e0 2024-09-10T08:20:10.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.434+0000 7f3dc01d0740 10 client.18834 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:10.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.434+0000 7f3dc01d0740 10 client.18834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:10.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.434+0000 7f3dc01d0740 20 client.18834 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:10.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.434+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/1339809084 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555a55b3cd10 0x555a55b3f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.434+0000 7f3dc01d0740 4 client.18834 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:10.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.434+0000 7f3dc01d0740 1 -- 172.21.15.2:0/1339809084 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a55a95800 con 0x555a55b3cd10 2024-09-10T08:20:10.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.434+0000 7f3db7400640 1 --2- 172.21.15.2:0/1339809084 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555a55b3cd10 0x555a55b3f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.435+0000 7f3db7400640 1 --2- 172.21.15.2:0/1339809084 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555a55b3cd10 0x555a55b3f130 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=102202599108cc92 server_cookie=765f41ebf64693e2 in_seq=0 out_seq=0 2024-09-10T08:20:10.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.435+0000 7f3d97e00640 10 client.18834 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:10.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.435+0000 7f3d97e00640 1 -- 172.21.15.2:0/1339809084 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a55a95800 con 0x555a55b3cd10 2024-09-10T08:20:10.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.435+0000 7f3d97e00640 10 client.18834 handle_command_reply: tid=0 2024-09-10T08:20:10.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.435+0000 7f3dc01d0740 1 client.18834 shutdown 2024-09-10T08:20:10.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.436+0000 7f3dc01d0740 1 -- 172.21.15.2:0/1339809084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b223e0 msgr2=0x555a55b298c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:10.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.436+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/1339809084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b223e0 0x555a55b298c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009900 tx=0x7f3da4004420 comp rx=0 tx=0).stop 2024-09-10T08:20:10.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.436+0000 7f3dc01d0740 1 -- 172.21.15.2:0/1339809084 shutdown_connections 2024-09-10T08:20:10.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.436+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/1339809084 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555a55b3cd10 0x555a55b3f130 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.436+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/1339809084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b2cca0 0x555a55b29e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.436+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/1339809084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b223e0 0x555a55b298c0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.436+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/1339809084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b2c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.436+0000 7f3dc01d0740 1 -- 172.21.15.2:0/1339809084 >> 172.21.15.2:0/1339809084 conn(0x555a55b1e050 msgr2=0x555a55b23d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:10.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.436+0000 7f3d97e00640 0 client.18834 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:10.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.436+0000 7f3dc01d0740 1 -- 172.21.15.2:0/1339809084 shutdown_connections 2024-09-10T08:20:10.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.436+0000 7f3dc01d0740 1 -- 172.21.15.2:0/1339809084 wait complete. 2024-09-10T08:20:10.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.436+0000 7f3dc01d0740 20 client.18834 trim_cache size 0 max 16384 2024-09-10T08:20:10.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.454+0000 7f3dc01d0740 1 Processor -- start 2024-09-10T08:20:10.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.454+0000 7f3dc01d0740 1 -- start start 2024-09-10T08:20:10.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.454+0000 7f3dc01d0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b21ac0 0x555a55b44a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.454+0000 7f3dc01d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b223e0 0x555a55b44f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.454+0000 7f3dc01d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 0x555a55b2b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.454+0000 7f3dc01d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81b7cd0 con 0x555a55b2cca0 2024-09-10T08:20:10.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.454+0000 7f3dc01d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81b55a0 con 0x555a55b223e0 2024-09-10T08:20:10.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.454+0000 7f3dc01d0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81b33f0 con 0x555a55b21ac0 2024-09-10T08:20:10.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.455+0000 7f3db6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b223e0 0x555a55b44f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.455+0000 7f3db7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 0x555a55b2b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.455+0000 7f3db6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b223e0 0x555a55b44f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50192/0 (socket says 172.21.15.2:50192) 2024-09-10T08:20:10.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.455+0000 7f3db6000640 1 -- 172.21.15.2:0/2328820809 learned_addr learned my addr 172.21.15.2:0/2328820809 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:10.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.455+0000 7f3db7400640 1 --2- 172.21.15.2:0/2328820809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b21ac0 0x555a55b44a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.455+0000 7f3db6000640 1 -- 172.21.15.2:0/2328820809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b21ac0 msgr2=0x555a55b44a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:10.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.455+0000 7f3db6000640 1 --2- 172.21.15.2:0/2328820809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b21ac0 0x555a55b44a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.455+0000 7f3db6000640 1 -- 172.21.15.2:0/2328820809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 msgr2=0x555a55b2b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:10.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.455+0000 7f3db6000640 1 --2- 172.21.15.2:0/2328820809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 0x555a55b2b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.455+0000 7f3db6000640 1 -- 172.21.15.2:0/2328820809 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a55b2f400 con 0x555a55b223e0 2024-09-10T08:20:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.455+0000 7f3db6000640 1 --2- 172.21.15.2:0/2328820809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b223e0 0x555a55b44f40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3da4000c00 tx=0x7f3da4005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e7c7678a09ba1e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.456+0000 7f3da2a00640 1 -- 172.21.15.2:0/2328820809 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4058070 con 0x555a55b223e0 2024-09-10T08:20:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.456+0000 7f3da2a00640 1 -- 172.21.15.2:0/2328820809 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3da4041070 con 0x555a55b223e0 2024-09-10T08:20:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.456+0000 7f3da2a00640 1 -- 172.21.15.2:0/2328820809 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4035030 con 0x555a55b223e0 2024-09-10T08:20:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.456+0000 7f3dc01d0740 1 -- 172.21.15.2:0/2328820809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b223e0 msgr2=0x555a55b44f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.456+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/2328820809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b223e0 0x555a55b44f40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3da4000c00 tx=0x7f3da4005bc0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.456+0000 7f3dc01d0740 1 -- 172.21.15.2:0/2328820809 shutdown_connections 2024-09-10T08:20:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.456+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/2328820809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 0x555a55b2b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.456+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/2328820809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b223e0 0x555a55b44f40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.456+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/2328820809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b21ac0 0x555a55b44a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.456+0000 7f3dc01d0740 1 -- 172.21.15.2:0/2328820809 >> 172.21.15.2:0/2328820809 conn(0x555a55b1e050 msgr2=0x555a55b24f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.456+0000 7f3dc01d0740 1 -- 172.21.15.2:0/2328820809 shutdown_connections 2024-09-10T08:20:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.457+0000 7f3dc01d0740 1 -- 172.21.15.2:0/2328820809 wait complete. 2024-09-10T08:20:10.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.457+0000 7f3dc01d0740 1 -- start start 2024-09-10T08:20:10.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.457+0000 7f3dc01d0740 1 Processor -- start 2024-09-10T08:20:10.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3dc01d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b30950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3dc01d0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b223e0 0x555a55b30e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3dc01d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 0x555a55b313d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3dc01d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81b3a60 con 0x555a55b2cca0 2024-09-10T08:20:10.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3dc01d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81b4ed0 con 0x555a55b21ac0 2024-09-10T08:20:10.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3dc01d0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81b4130 con 0x555a55b223e0 2024-09-10T08:20:10.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3dc01d0740 10 client.0 _ll_register_callbacks cb 0x555a55b2a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:10.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3db7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b30950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3db6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b223e0 0x555a55b30e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3db6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b223e0 0x555a55b30e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:60146/0 (socket says 172.21.15.2:60146) 2024-09-10T08:20:10.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3db6000640 1 -- 172.21.15.2:0/3070269585 learned_addr learned my addr 172.21.15.2:0/3070269585 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:10.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3db7400640 1 --2- 172.21.15.2:0/3070269585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 0x555a55b313d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3db6000640 1 -- 172.21.15.2:0/3070269585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 msgr2=0x555a55b30950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:10.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3db6000640 1 --2- 172.21.15.2:0/3070269585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b30950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3db6000640 1 -- 172.21.15.2:0/3070269585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 msgr2=0x555a55b313d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:10.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3db6000640 1 --2- 172.21.15.2:0/3070269585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 0x555a55b313d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.458+0000 7f3db6000640 1 -- 172.21.15.2:0/3070269585 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a55ba69d0 con 0x555a55b223e0 2024-09-10T08:20:10.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.459+0000 7f3db6000640 1 --2- 172.21.15.2:0/3070269585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b223e0 0x555a55b30e90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3da402a910 tx=0x7f3da4031760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=685ad85126f5adb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:10.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.459+0000 7f3d97e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:20:10.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.459+0000 7f3d97e00640 1 -- 172.21.15.2:0/3070269585 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4035070 con 0x555a55b223e0 2024-09-10T08:20:10.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.459+0000 7f3dc01d0740 1 -- 172.21.15.2:0/3070269585 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3da4041070 con 0x555a55b223e0 2024-09-10T08:20:10.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.459+0000 7f3d97e00640 1 -- 172.21.15.2:0/3070269585 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3da4031a50 con 0x555a55b223e0 2024-09-10T08:20:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.459+0000 7f3d97e00640 1 -- 172.21.15.2:0/3070269585 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3da4058040 con 0x555a55b223e0 2024-09-10T08:20:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.459+0000 7f3dc01d0740 10 client.28798 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.459+0000 7f3dc01d0740 1 -- 172.21.15.2:0/3070269585 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a55ba7c10 con 0x555a55b223e0 2024-09-10T08:20:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.460+0000 7f3d97e00640 1 -- 172.21.15.2:0/3070269585 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3da4041070 con 0x555a55b223e0 2024-09-10T08:20:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.460+0000 7f3d97e00640 1 -- 172.21.15.2:0/3070269585 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3da405b070 con 0x555a55b223e0 2024-09-10T08:20:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.460+0000 7f3dc01d0740 10 client.28798 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.460+0000 7f3dc01d0740 10 client.28798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.460+0000 7f3dc01d0740 20 client.28798 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.460+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/3070269585 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555a55ba81c0 0x555a55baa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.460+0000 7f3dc01d0740 4 client.28798 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:20:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.460+0000 7f3dc01d0740 1 -- 172.21.15.2:0/3070269585 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555a55a8fd10 con 0x555a55ba81c0 2024-09-10T08:20:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.460+0000 7f3db7400640 1 --2- 172.21.15.2:0/3070269585 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555a55ba81c0 0x555a55baa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.461+0000 7f3db7400640 1 --2- 172.21.15.2:0/3070269585 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555a55ba81c0 0x555a55baa5e0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc8fd9e99e1a8a40 server_cookie=41f9eea4f75e7352 in_seq=0 out_seq=0 2024-09-10T08:20:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.461+0000 7f3d97e00640 10 client.28798 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.462+0000 7f3d97e00640 1 -- 172.21.15.2:0/3070269585 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555a55a8fd10 con 0x555a55ba81c0 2024-09-10T08:20:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.462+0000 7f3d97e00640 10 client.28798 handle_command_reply: tid=0 2024-09-10T08:20:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.462+0000 7f3dc01d0740 1 client.28798 shutdown 2024-09-10T08:20:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.462+0000 7f3dc01d0740 1 -- 172.21.15.2:0/3070269585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b223e0 msgr2=0x555a55b30e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.462+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/3070269585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b223e0 0x555a55b30e90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3da402a910 tx=0x7f3da4031760 comp rx=0 tx=0).stop 2024-09-10T08:20:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.462+0000 7f3dc01d0740 1 -- 172.21.15.2:0/3070269585 shutdown_connections 2024-09-10T08:20:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.462+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/3070269585 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x555a55ba81c0 0x555a55baa5e0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.462+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/3070269585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555a55b2cca0 0x555a55b313d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.462+0000 7f3d97e00640 0 client.28798 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.462+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/3070269585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a55b223e0 0x555a55b30e90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.462+0000 7f3dc01d0740 1 --2- 172.21.15.2:0/3070269585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a55b21ac0 0x555a55b30950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.462+0000 7f3dc01d0740 1 -- 172.21.15.2:0/3070269585 >> 172.21.15.2:0/3070269585 conn(0x555a55b1e050 msgr2=0x555a55b2b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.463+0000 7f3dc01d0740 1 -- 172.21.15.2:0/3070269585 shutdown_connections 2024-09-10T08:20:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.463+0000 7f3dc01d0740 1 -- 172.21.15.2:0/3070269585 wait complete. 2024-09-10T08:20:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.463+0000 7f3dc01d0740 20 client.28798 trim_cache size 0 max 16384 2024-09-10T08:20:10.466 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:20:10.466 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:20:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "093d0b0e-4cd2-4a0b-b027-bb71ad341f0c", 2024-09-10T08:20:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:20:10.467 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:20:10.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.464+0000 7f3dbd600640 1 -- 172.21.15.2:0/406664564 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3d9c0a2b80 msgr2=0x7f3d9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:10.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.464+0000 7f3dbd600640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3d9c0a2b80 0x7f3d9c0a5040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3da80a39a0 tx=0x7f3da80d2890 comp rx=0 tx=0).stop 2024-09-10T08:20:10.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.464+0000 7f3dbd600640 1 -- 172.21.15.2:0/406664564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3db8103ff0 msgr2=0x7f3db8076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.464+0000 7f3dbd600640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3db8103ff0 0x7f3db8076700 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3da800d980 tx=0x7f3da800de40 comp rx=0 tx=0).stop 2024-09-10T08:20:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.464+0000 7f3dbd600640 1 -- 172.21.15.2:0/406664564 shutdown_connections 2024-09-10T08:20:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.464+0000 7f3dbd600640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3d9c0a2b80 0x7f3d9c0a5040 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.464+0000 7f3dbd600640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db8109120 0x7f3db8077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.464+0000 7f3dbd600640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db81049a0 0x7f3db8076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.464+0000 7f3dbd600640 1 --2- 172.21.15.2:0/406664564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3db8103ff0 0x7f3db8076700 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.464+0000 7f3dbd600640 1 -- 172.21.15.2:0/406664564 >> 172.21.15.2:0/406664564 conn(0x7f3db80ffa90 msgr2=0x7f3db8100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.465+0000 7f3dbd600640 1 -- 172.21.15.2:0/406664564 shutdown_connections 2024-09-10T08:20:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:10.465+0000 7f3dbd600640 1 -- 172.21.15.2:0/406664564 wait complete. 2024-09-10T08:20:11.378 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:20:11.762 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:11 smithi002 ceph-mon[30563]: pgmap v748: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:20:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:11 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:20:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:11 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:20:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:11 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:20:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:11 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:20:11.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:11 smithi155 ceph-mon[35405]: pgmap v748: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:20:11.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:11 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:20:11.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:11 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:20:11.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:11 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:20:11.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:11 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:20:11.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:11 smithi107 ceph-mon[35048]: pgmap v748: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:20:11.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:11 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:20:11.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:11 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:20:11.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:11 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:20:11.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:11 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:20:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.403+0000 7f182d000640 1 -- 172.21.15.2:0/3462472400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1828074510 msgr2=0x7f18280748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.403+0000 7f182d000640 1 --2- 172.21.15.2:0/3462472400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1828074510 0x7f18280748f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f1818009900 tx=0x7f181802a480 comp rx=0 tx=0).stop 2024-09-10T08:20:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.403+0000 7f182d000640 1 -- 172.21.15.2:0/3462472400 shutdown_connections 2024-09-10T08:20:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.403+0000 7f182d000640 1 --2- 172.21.15.2:0/3462472400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f182810e280 0x7f1828110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.403+0000 7f182d000640 1 --2- 172.21.15.2:0/3462472400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1828074e30 0x7f182810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.403+0000 7f182d000640 1 --2- 172.21.15.2:0/3462472400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1828074510 0x7f18280748f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.403+0000 7f182d000640 1 -- 172.21.15.2:0/3462472400 >> 172.21.15.2:0/3462472400 conn(0x7f18280fd740 msgr2=0x7f18280ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.403+0000 7f182d000640 1 -- 172.21.15.2:0/3462472400 shutdown_connections 2024-09-10T08:20:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.403+0000 7f182d000640 1 -- 172.21.15.2:0/3462472400 wait complete. 2024-09-10T08:20:12.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.404+0000 7f182d000640 1 Processor -- start 2024-09-10T08:20:12.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.404+0000 7f182d000640 1 -- start start 2024-09-10T08:20:12.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.405+0000 7f182d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1828074510 0x7f182819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.405+0000 7f182d000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1828074e30 0x7f182819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.405+0000 7f182d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182810e280 0x7f182819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.405+0000 7f182d000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1828114000 con 0x7f1828074e30 2024-09-10T08:20:12.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.405+0000 7f182d000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1828113e80 con 0x7f1828074510 2024-09-10T08:20:12.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.405+0000 7f182d000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1828114180 con 0x7f182810e280 2024-09-10T08:20:12.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.405+0000 7f1826a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1828074e30 0x7f182819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.405+0000 7f1827400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1828074510 0x7f182819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.405+0000 7f1827400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1828074510 0x7f182819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:50234/0 (socket says 172.21.15.2:50234) 2024-09-10T08:20:12.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.405+0000 7f1827400640 1 -- 172.21.15.2:0/4083967260 learned_addr learned my addr 172.21.15.2:0/4083967260 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:12.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.405+0000 7f1827e00640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182810e280 0x7f182819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.406+0000 7f1826a00640 1 -- 172.21.15.2:0/4083967260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182810e280 msgr2=0x7f182819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:12.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.406+0000 7f1826a00640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182810e280 0x7f182819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.406+0000 7f1826a00640 1 -- 172.21.15.2:0/4083967260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1828074510 msgr2=0x7f182819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:12.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.406+0000 7f1826a00640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1828074510 0x7f182819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.406+0000 7f1826a00640 1 -- 172.21.15.2:0/4083967260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f182819f450 con 0x7f1828074e30 2024-09-10T08:20:12.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.406+0000 7f1827400640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1828074510 0x7f182819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:12.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.406+0000 7f1826a00640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1828074e30 0x7f182819a9e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f181400e9f0 tx=0x7f181400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:12.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.406+0000 7f181fe00640 1 -- 172.21.15.2:0/4083967260 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181400ccc0 con 0x7f1828074e30 2024-09-10T08:20:12.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.406+0000 7f181fe00640 1 -- 172.21.15.2:0/4083967260 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1814010910 con 0x7f1828074e30 2024-09-10T08:20:12.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.406+0000 7f182d000640 1 -- 172.21.15.2:0/4083967260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f182819f740 con 0x7f1828074e30 2024-09-10T08:20:12.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.406+0000 7f181fe00640 1 -- 172.21.15.2:0/4083967260 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814018960 con 0x7f1828074e30 2024-09-10T08:20:12.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.406+0000 7f182d000640 1 -- 172.21.15.2:0/4083967260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18281a7020 con 0x7f1828074e30 2024-09-10T08:20:12.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.410+0000 7f1831261740 1 Processor -- start 2024-09-10T08:20:12.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.410+0000 7f1831261740 1 -- start start 2024-09-10T08:20:12.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.410+0000 7f1831261740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc20130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.410+0000 7f1831261740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc2a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.410+0000 7f1831261740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 0x556c5dc2d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.410+0000 7f1831261740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556c5db863e0 con 0x556c5dc1fd50 2024-09-10T08:20:12.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.410+0000 7f1831261740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556c5db2dd80 con 0x556c5dc2af30 2024-09-10T08:20:12.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.410+0000 7f1831261740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556c5db823a0 con 0x556c5dc20670 2024-09-10T08:20:12.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.410+0000 7f181fe00640 1 -- 172.21.15.2:0/4083967260 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1814010ac0 con 0x7f1828074e30 2024-09-10T08:20:12.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.410+0000 7f1827e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc20130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.411+0000 7f1827e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc20130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45864/0 (socket says 172.21.15.2:45864) 2024-09-10T08:20:12.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.411+0000 7f1827400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc2a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.411+0000 7f1827e00640 1 -- 172.21.15.2:0/1383253531 learned_addr learned my addr 172.21.15.2:0/1383253531 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:12.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.411+0000 7f1826a00640 1 --2- 172.21.15.2:0/1383253531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 0x556c5dc2d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.411+0000 7f1827400640 1 -- 172.21.15.2:0/1383253531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 msgr2=0x556c5dc2d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:12.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.411+0000 7f1827400640 1 --2- 172.21.15.2:0/1383253531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 0x556c5dc2d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.411+0000 7f1827400640 1 -- 172.21.15.2:0/1383253531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 msgr2=0x556c5dc20130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:12.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.411+0000 7f1827400640 1 --2- 172.21.15.2:0/1383253531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc20130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.411+0000 7f1827400640 1 -- 172.21.15.2:0/1383253531 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5db4cae0 con 0x556c5dc20670 2024-09-10T08:20:12.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.411+0000 7f1826a00640 1 --2- 172.21.15.2:0/1383253531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 0x556c5dc2d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:20:12.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.411+0000 7f1827400640 1 --2- 172.21.15.2:0/1383253531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc2a9f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1818031550 tx=0x7f1818005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99c4e4de340a6d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:12.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.412+0000 7f181cc00640 1 -- 172.21.15.2:0/1383253531 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818040070 con 0x556c5dc20670 2024-09-10T08:20:12.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.412+0000 7f181cc00640 1 -- 172.21.15.2:0/1383253531 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18180047a0 con 0x556c5dc20670 2024-09-10T08:20:12.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.412+0000 7f181cc00640 1 -- 172.21.15.2:0/1383253531 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818035040 con 0x556c5dc20670 2024-09-10T08:20:12.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.412+0000 7f181fe00640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f180c0a2b80 0x7f180c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.412+0000 7f181fe00640 1 -- 172.21.15.2:0/4083967260 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f18140cf820 con 0x7f1828074e30 2024-09-10T08:20:12.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.412+0000 7f1827e00640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f180c0a2b80 0x7f180c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.412+0000 7f1831261740 1 -- 172.21.15.2:0/1383253531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 msgr2=0x556c5dc2a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:12.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.413+0000 7f1831261740 1 --2- 172.21.15.2:0/1383253531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc2a9f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1818031550 tx=0x7f1818005dd0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.413+0000 7f1831261740 1 -- 172.21.15.2:0/1383253531 shutdown_connections 2024-09-10T08:20:12.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.413+0000 7f1831261740 1 --2- 172.21.15.2:0/1383253531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 0x556c5dc2d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.413+0000 7f1831261740 1 --2- 172.21.15.2:0/1383253531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc2a9f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.413+0000 7f1831261740 1 --2- 172.21.15.2:0/1383253531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc20130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.413+0000 7f1831261740 1 -- 172.21.15.2:0/1383253531 >> 172.21.15.2:0/1383253531 conn(0x556c5dc1c2e0 msgr2=0x556c5dc1c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:12.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.413+0000 7f1831261740 1 -- 172.21.15.2:0/1383253531 shutdown_connections 2024-09-10T08:20:12.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.413+0000 7f1831261740 1 -- 172.21.15.2:0/1383253531 wait complete. 2024-09-10T08:20:12.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.413+0000 7f1827e00640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f180c0a2b80 0x7f180c0a5040 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1820011630 tx=0x7f1820009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:12.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.414+0000 7f1831261740 1 -- start start 2024-09-10T08:20:12.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.414+0000 7f1831261740 1 Processor -- start 2024-09-10T08:20:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.414+0000 7f1831261740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc27e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.414+0000 7f1831261740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc28340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.414+0000 7f1831261740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 0x556c5dc28880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.414+0000 7f1831261740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556c5dc2dba0 con 0x556c5dc1fd50 2024-09-10T08:20:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.414+0000 7f1831261740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556c5db410f0 con 0x556c5dc2af30 2024-09-10T08:20:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.414+0000 7f1831261740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556c5dc2dd20 con 0x556c5dc20670 2024-09-10T08:20:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.415+0000 7f1831261740 10 client.0 _ll_register_callbacks cb 0x556c5db329e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.415+0000 7f1827400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc27e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.415+0000 7f1827400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc27e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45878/0 (socket says 172.21.15.2:45878) 2024-09-10T08:20:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.415+0000 7f1827400640 1 -- 172.21.15.2:0/2508742983 learned_addr learned my addr 172.21.15.2:0/2508742983 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.415+0000 7f1826a00640 1 --2- 172.21.15.2:0/2508742983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc28340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.415+0000 7f1827e00640 1 --2- 172.21.15.2:0/2508742983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 0x556c5dc28880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.415+0000 7f1826a00640 1 -- 172.21.15.2:0/2508742983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 msgr2=0x556c5dc28880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.415+0000 7f1826a00640 1 --2- 172.21.15.2:0/2508742983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 0x556c5dc28880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.415+0000 7f1826a00640 1 -- 172.21.15.2:0/2508742983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 msgr2=0x556c5dc27e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.415+0000 7f1826a00640 1 --2- 172.21.15.2:0/2508742983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc27e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.415+0000 7f1826a00640 1 -- 172.21.15.2:0/2508742983 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c5d8f73c0 con 0x556c5dc20670 2024-09-10T08:20:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.416+0000 7f1826a00640 1 --2- 172.21.15.2:0/2508742983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc28340 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x556c5dc1d660 tx=0x7f181409bba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be536e3eb58aeba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.416+0000 7f1811600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:20:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.416+0000 7f1811600640 1 -- 172.21.15.2:0/2508742983 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181409b330 con 0x556c5dc20670 2024-09-10T08:20:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.416+0000 7f1811600640 1 -- 172.21.15.2:0/2508742983 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1814010430 con 0x556c5dc20670 2024-09-10T08:20:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.416+0000 7f1831261740 1 -- 172.21.15.2:0/2508742983 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18180047a0 con 0x556c5dc20670 2024-09-10T08:20:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.416+0000 7f1811600640 1 -- 172.21.15.2:0/2508742983 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f181409ffe0 con 0x556c5dc20670 2024-09-10T08:20:12.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.416+0000 7f1831261740 10 client.28810 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:12.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.416+0000 7f1831261740 1 -- 172.21.15.2:0/2508742983 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556c5dc39830 con 0x556c5dc20670 2024-09-10T08:20:12.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.416+0000 7f1811600640 1 -- 172.21.15.2:0/2508742983 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18180047a0 con 0x556c5dc20670 2024-09-10T08:20:12.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.417+0000 7f1811600640 1 -- 172.21.15.2:0/2508742983 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f181409aba0 con 0x556c5dc20670 2024-09-10T08:20:12.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.417+0000 7f1831261740 10 client.28810 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:12.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.417+0000 7f1831261740 10 client.28810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:12.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.417+0000 7f1831261740 20 client.28810 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:12.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.417+0000 7f1831261740 1 --2- 172.21.15.2:0/2508742983 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556c5dc3ac40 0x556c5dc3d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.417+0000 7f1831261740 4 client.28810 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:12.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.417+0000 7f1831261740 1 -- 172.21.15.2:0/2508742983 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c5db93950 con 0x556c5dc3ac40 2024-09-10T08:20:12.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.417+0000 7f1827400640 1 --2- 172.21.15.2:0/2508742983 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556c5dc3ac40 0x556c5dc3d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.418+0000 7f1827400640 1 --2- 172.21.15.2:0/2508742983 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556c5dc3ac40 0x556c5dc3d050 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff47a46b73dbde5 server_cookie=d412c883ea0cec2e in_seq=0 out_seq=0 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.418+0000 7f1811600640 10 client.28810 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.418+0000 7f1811600640 1 -- 172.21.15.2:0/2508742983 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c5db93950 con 0x556c5dc3ac40 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.418+0000 7f1811600640 10 client.28810 handle_command_reply: tid=0 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.419+0000 7f1831261740 1 client.28810 shutdown 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.419+0000 7f1831261740 1 -- 172.21.15.2:0/2508742983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 msgr2=0x556c5dc28340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.419+0000 7f1831261740 1 --2- 172.21.15.2:0/2508742983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc28340 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x556c5dc1d660 tx=0x7f181409bba0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.419+0000 7f1831261740 1 -- 172.21.15.2:0/2508742983 shutdown_connections 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.419+0000 7f1831261740 1 --2- 172.21.15.2:0/2508742983 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556c5dc3ac40 0x556c5dc3d050 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.419+0000 7f1831261740 1 --2- 172.21.15.2:0/2508742983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 0x556c5dc28880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.419+0000 7f1831261740 1 --2- 172.21.15.2:0/2508742983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc28340 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.419+0000 7f1831261740 1 --2- 172.21.15.2:0/2508742983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc27e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.419+0000 7f1811600640 0 client.28810 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:12.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.419+0000 7f1831261740 1 -- 172.21.15.2:0/2508742983 >> 172.21.15.2:0/2508742983 conn(0x556c5dc1c2e0 msgr2=0x556c5dc22010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:12.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.419+0000 7f1831261740 1 -- 172.21.15.2:0/2508742983 shutdown_connections 2024-09-10T08:20:12.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.419+0000 7f1831261740 1 -- 172.21.15.2:0/2508742983 wait complete. 2024-09-10T08:20:12.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.419+0000 7f1831261740 20 client.28810 trim_cache size 0 max 16384 2024-09-10T08:20:12.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.437+0000 7f1831261740 1 Processor -- start 2024-09-10T08:20:12.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.437+0000 7f1831261740 1 -- start start 2024-09-10T08:20:12.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1831261740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc1fd50 0x556c5dc17060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1831261740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc20670 0x556c5dc21eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1831261740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc2af30 0x556c5dc42920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1831261740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f18281b37a0 con 0x556c5dc20670 2024-09-10T08:20:12.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1831261740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18281b1070 con 0x556c5dc1fd50 2024-09-10T08:20:12.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1831261740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f18281aeec0 con 0x556c5dc2af30 2024-09-10T08:20:12.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1826a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc20670 0x556c5dc21eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1826a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc20670 0x556c5dc21eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45888/0 (socket says 172.21.15.2:45888) 2024-09-10T08:20:12.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1826a00640 1 -- 172.21.15.2:0/3370765503 learned_addr learned my addr 172.21.15.2:0/3370765503 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:12.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1827400640 1 --2- 172.21.15.2:0/3370765503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc1fd50 0x556c5dc17060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1827e00640 1 --2- 172.21.15.2:0/3370765503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc2af30 0x556c5dc42920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1826a00640 1 -- 172.21.15.2:0/3370765503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc2af30 msgr2=0x556c5dc42920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:12.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1826a00640 1 --2- 172.21.15.2:0/3370765503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc2af30 0x556c5dc42920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1826a00640 1 -- 172.21.15.2:0/3370765503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc1fd50 msgr2=0x556c5dc17060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:12.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1826a00640 1 --2- 172.21.15.2:0/3370765503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc1fd50 0x556c5dc17060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1826a00640 1 -- 172.21.15.2:0/3370765503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5dc26db0 con 0x556c5dc20670 2024-09-10T08:20:12.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.438+0000 7f1827400640 1 --2- 172.21.15.2:0/3370765503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc1fd50 0x556c5dc17060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:20:12.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.439+0000 7f1826a00640 1 --2- 172.21.15.2:0/3370765503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc20670 0x556c5dc21eb0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f181409bc70 tx=0x7f1814005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5051a3acfe5c2ffe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:12.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.439+0000 7f1813e00640 1 -- 172.21.15.2:0/3370765503 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18140a48a0 con 0x556c5dc20670 2024-09-10T08:20:12.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.439+0000 7f1813e00640 1 -- 172.21.15.2:0/3370765503 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f181409aba0 con 0x556c5dc20670 2024-09-10T08:20:12.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.439+0000 7f1813e00640 1 -- 172.21.15.2:0/3370765503 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814018d60 con 0x556c5dc20670 2024-09-10T08:20:12.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.439+0000 7f1831261740 1 -- 172.21.15.2:0/3370765503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc20670 msgr2=0x556c5dc21eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:12.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.439+0000 7f1831261740 1 --2- 172.21.15.2:0/3370765503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc20670 0x556c5dc21eb0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f181409bc70 tx=0x7f1814005e30 comp rx=0 tx=0).stop 2024-09-10T08:20:12.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.440+0000 7f1831261740 1 -- 172.21.15.2:0/3370765503 shutdown_connections 2024-09-10T08:20:12.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.440+0000 7f1831261740 1 --2- 172.21.15.2:0/3370765503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc2af30 0x556c5dc42920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.440+0000 7f1831261740 1 --2- 172.21.15.2:0/3370765503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc20670 0x556c5dc21eb0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.440+0000 7f1831261740 1 --2- 172.21.15.2:0/3370765503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc1fd50 0x556c5dc17060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.440+0000 7f1831261740 1 -- 172.21.15.2:0/3370765503 >> 172.21.15.2:0/3370765503 conn(0x556c5dc1c2e0 msgr2=0x556c5dc217a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:12.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.440+0000 7f1831261740 1 -- 172.21.15.2:0/3370765503 shutdown_connections 2024-09-10T08:20:12.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.440+0000 7f1831261740 1 -- 172.21.15.2:0/3370765503 wait complete. 2024-09-10T08:20:12.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.440+0000 7f1831261740 1 -- start start 2024-09-10T08:20:12.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.440+0000 7f1831261740 1 Processor -- start 2024-09-10T08:20:12.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1831261740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc2eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1831261740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc2f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1831261740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 0x556c5dc2f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1831261740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f18281af530 con 0x556c5dc1fd50 2024-09-10T08:20:12.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1831261740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18281b09a0 con 0x556c5dc2af30 2024-09-10T08:20:12.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1831261740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f18281afc00 con 0x556c5dc20670 2024-09-10T08:20:12.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1831261740 10 client.0 _ll_register_callbacks cb 0x7f1814098050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:12.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1827400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc2eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1827400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc2eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45902/0 (socket says 172.21.15.2:45902) 2024-09-10T08:20:12.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1827400640 1 -- 172.21.15.2:0/820693129 learned_addr learned my addr 172.21.15.2:0/820693129 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:12.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1826a00640 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc2f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1827e00640 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 0x556c5dc2f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1826a00640 1 -- 172.21.15.2:0/820693129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 msgr2=0x556c5dc2f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:12.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1826a00640 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 0x556c5dc2f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1826a00640 1 -- 172.21.15.2:0/820693129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 msgr2=0x556c5dc2eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:12.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1826a00640 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc2eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.441+0000 7f1826a00640 1 -- 172.21.15.2:0/820693129 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c5dca4af0 con 0x556c5dc20670 2024-09-10T08:20:12.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.442+0000 7f1827400640 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc2eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:12.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.442+0000 7f1826a00640 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc2f430 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x556c5dc17d30 tx=0x7f181409b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90db0f9df852bdcd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:12.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.442+0000 7f1811600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:20:12.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.442+0000 7f1811600640 1 -- 172.21.15.2:0/820693129 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814010760 con 0x556c5dc20670 2024-09-10T08:20:12.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.442+0000 7f1811600640 1 -- 172.21.15.2:0/820693129 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f18140a1170 con 0x556c5dc20670 2024-09-10T08:20:12.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.442+0000 7f1831261740 1 -- 172.21.15.2:0/820693129 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f181409aba0 con 0x556c5dc20670 2024-09-10T08:20:12.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.442+0000 7f1811600640 1 -- 172.21.15.2:0/820693129 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f18140a48a0 con 0x556c5dc20670 2024-09-10T08:20:12.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.442+0000 7f1831261740 10 client.28816 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:12.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.442+0000 7f1831261740 1 -- 172.21.15.2:0/820693129 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556c5dca4af0 con 0x556c5dc20670 2024-09-10T08:20:12.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.442+0000 7f1811600640 1 -- 172.21.15.2:0/820693129 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f181409aba0 con 0x556c5dc20670 2024-09-10T08:20:12.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.443+0000 7f1811600640 1 -- 172.21.15.2:0/820693129 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f18140ba120 con 0x556c5dc20670 2024-09-10T08:20:12.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.443+0000 7f1831261740 10 client.28816 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:12.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.443+0000 7f1831261740 10 client.28816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:12.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.443+0000 7f1831261740 20 client.28816 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:12.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.443+0000 7f1831261740 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556c5dca6040 0x556c5dca8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:12.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.443+0000 7f1831261740 4 client.28816 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:20:12.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.443+0000 7f1831261740 1 -- 172.21.15.2:0/820693129 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c5db8de60 con 0x556c5dca6040 2024-09-10T08:20:12.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.443+0000 7f1827400640 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556c5dca6040 0x556c5dca8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:12.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1827400640 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556c5dca6040 0x556c5dca8430 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc4852ec0f5a1e40 server_cookie=96191e51bdc91f36 in_seq=0 out_seq=0 2024-09-10T08:20:12.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1811600640 10 client.28816 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:12.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1811600640 1 -- 172.21.15.2:0/820693129 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556c5db8de60 con 0x556c5dca6040 2024-09-10T08:20:12.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1811600640 10 client.28816 handle_command_reply: tid=0 2024-09-10T08:20:12.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1831261740 1 client.28816 shutdown 2024-09-10T08:20:12.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1831261740 1 -- 172.21.15.2:0/820693129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 msgr2=0x556c5dc2f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:12.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1831261740 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc2f430 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x556c5dc17d30 tx=0x7f181409b260 comp rx=0 tx=0).stop 2024-09-10T08:20:12.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1831261740 1 -- 172.21.15.2:0/820693129 shutdown_connections 2024-09-10T08:20:12.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1831261740 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556c5dca6040 0x556c5dca8430 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1831261740 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c5dc2af30 0x556c5dc2f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1831261740 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5dc20670 0x556c5dc2f430 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1831261740 1 --2- 172.21.15.2:0/820693129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556c5dc1fd50 0x556c5dc2eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1831261740 1 -- 172.21.15.2:0/820693129 >> 172.21.15.2:0/820693129 conn(0x556c5dc1c2e0 msgr2=0x556c5dc21fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:12.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1811600640 0 client.28816 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:12.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1831261740 1 -- 172.21.15.2:0/820693129 shutdown_connections 2024-09-10T08:20:12.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1831261740 1 -- 172.21.15.2:0/820693129 wait complete. 2024-09-10T08:20:12.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.444+0000 7f1831261740 20 client.28816 trim_cache size 0 max 16384 2024-09-10T08:20:12.449 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:20:12.449 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:20:12.449 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:20:12.450 INFO:teuthology.orchestra.run.smithi002.stdout: "093d0b0e-4cd2-4a0b-b027-bb71ad341f0c": { 2024-09-10T08:20:12.450 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:20:12.450 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "093d0b0e-4cd2-4a0b-b027-bb71ad341f0c", 2024-09-10T08:20:12.450 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:20:12.450 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:12.450 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:12.450 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:20:12.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.445+0000 7f182d000640 1 -- 172.21.15.2:0/4083967260 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f180c0a2b80 msgr2=0x7f180c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:12.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.445+0000 7f182d000640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f180c0a2b80 0x7f180c0a5040 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1820011630 tx=0x7f1820009500 comp rx=0 tx=0).stop 2024-09-10T08:20:12.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.445+0000 7f182d000640 1 -- 172.21.15.2:0/4083967260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1828074e30 msgr2=0x7f182819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:12.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.445+0000 7f182d000640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1828074e30 0x7f182819a9e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f181400e9f0 tx=0x7f181400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.445+0000 7f182d000640 1 -- 172.21.15.2:0/4083967260 shutdown_connections 2024-09-10T08:20:12.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.445+0000 7f182d000640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f180c0a2b80 0x7f180c0a5040 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.445+0000 7f182d000640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182810e280 0x7f182819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.445+0000 7f182d000640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1828074e30 0x7f182819a9e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.445+0000 7f182d000640 1 --2- 172.21.15.2:0/4083967260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1828074510 0x7f182819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:12.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.445+0000 7f182d000640 1 -- 172.21.15.2:0/4083967260 >> 172.21.15.2:0/4083967260 conn(0x7f18280fd740 msgr2=0x7f18280fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:12.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.445+0000 7f182d000640 1 -- 172.21.15.2:0/4083967260 shutdown_connections 2024-09-10T08:20:12.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:12.445+0000 7f182d000640 1 -- 172.21.15.2:0/4083967260 wait complete. 2024-09-10T08:20:13.452 INFO:tasks.cephfs.filesystem:scrub status for tag:093d0b0e-4cd2-4a0b-b027-bb71ad341f0c - {'path': '/', 'tag': '093d0b0e-4cd2-4a0b-b027-bb71ad341f0c', 'options': 'recursive,force'} 2024-09-10T08:20:13.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:13 smithi002 ceph-mon[30563]: pgmap v749: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:20:13.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:13 smithi155 ceph-mon[35405]: pgmap v749: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:20:13.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:13 smithi107 ceph-mon[35048]: pgmap v749: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:20:14.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:14 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:20:14.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:14 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:20:15.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:14 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:20:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:15 smithi155 ceph-mon[35405]: pgmap v750: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:20:15.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:15 smithi107 ceph-mon[35048]: pgmap v750: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:20:16.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:15 smithi002 ceph-mon[30563]: pgmap v750: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:20:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:17 smithi155 ceph-mon[35405]: pgmap v751: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:20:17.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:17 smithi107 ceph-mon[35048]: pgmap v751: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:20:18.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:17 smithi002 ceph-mon[30563]: pgmap v751: 129 pgs: 129 active+clean; 2.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:20:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:19 smithi155 ceph-mon[35405]: pgmap v752: 129 pgs: 129 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:20:19.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:19 smithi107 ceph-mon[35048]: pgmap v752: 129 pgs: 129 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:20:20.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:19 smithi002 ceph-mon[30563]: pgmap v752: 129 pgs: 129 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:20:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.502+0000 7f0685e00640 1 -- 172.21.15.2:0/477650467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0680105910 msgr2=0x7f068010c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.502+0000 7f0685e00640 1 --2- 172.21.15.2:0/477650467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0680105910 0x7f068010c1a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0678009ca0 tx=0x7f067802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.502+0000 7f0685e00640 1 -- 172.21.15.2:0/477650467 shutdown_connections 2024-09-10T08:20:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.502+0000 7f0685e00640 1 --2- 172.21.15.2:0/477650467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0680105910 0x7f068010c1a0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.502+0000 7f0685e00640 1 --2- 172.21.15.2:0/477650467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0680104f50 0x7f06801053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.502+0000 7f0685e00640 1 --2- 172.21.15.2:0/477650467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0680103d60 0x7f0680104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.502+0000 7f0685e00640 1 -- 172.21.15.2:0/477650467 >> 172.21.15.2:0/477650467 conn(0x7f06800ff4e0 msgr2=0x7f0680101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.502+0000 7f0685e00640 1 -- 172.21.15.2:0/477650467 shutdown_connections 2024-09-10T08:20:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.502+0000 7f0685e00640 1 -- 172.21.15.2:0/477650467 wait complete. 2024-09-10T08:20:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.503+0000 7f0685e00640 1 Processor -- start 2024-09-10T08:20:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.503+0000 7f0685e00640 1 -- start start 2024-09-10T08:20:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.503+0000 7f0685e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0680103d60 0x7f0680199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.503+0000 7f0685e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0680104f50 0x7f068019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.503+0000 7f0685e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0680105910 0x7f06801a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.503+0000 7f0685e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f068010fa30 con 0x7f0680104f50 2024-09-10T08:20:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.503+0000 7f0685e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f068010f8b0 con 0x7f0680103d60 2024-09-10T08:20:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.503+0000 7f0685e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f068010fbb0 con 0x7f0680105910 2024-09-10T08:20:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.504+0000 7f0677e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0680104f50 0x7f068019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.504+0000 7f0677e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0680104f50 0x7f068019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48164/0 (socket says 172.21.15.2:48164) 2024-09-10T08:20:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.504+0000 7f0677e00640 1 -- 172.21.15.2:0/2654539926 learned_addr learned my addr 172.21.15.2:0/2654539926 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.504+0000 7f067f400640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0680105910 0x7f06801a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.504+0000 7f067ea00640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0680103d60 0x7f0680199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.504+0000 7f0677e00640 1 -- 172.21.15.2:0/2654539926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0680105910 msgr2=0x7f06801a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.504+0000 7f0677e00640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0680105910 0x7f06801a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.504+0000 7f0677e00640 1 -- 172.21.15.2:0/2654539926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0680103d60 msgr2=0x7f0680199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.504+0000 7f0677e00640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0680103d60 0x7f0680199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.504+0000 7f0677e00640 1 -- 172.21.15.2:0/2654539926 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06801a1b70 con 0x7f0680104f50 2024-09-10T08:20:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.505+0000 7f0677e00640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0680104f50 0x7f068019a490 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f066800e9f0 tx=0x7f066800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.505+0000 7f0684a00640 1 -- 172.21.15.2:0/2654539926 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066800cd80 con 0x7f0680104f50 2024-09-10T08:20:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.505+0000 7f0684a00640 1 -- 172.21.15.2:0/2654539926 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0668010910 con 0x7f0680104f50 2024-09-10T08:20:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.505+0000 7f0685e00640 1 -- 172.21.15.2:0/2654539926 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06801a1e60 con 0x7f0680104f50 2024-09-10T08:20:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.505+0000 7f0684a00640 1 -- 172.21.15.2:0/2654539926 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668018990 con 0x7f0680104f50 2024-09-10T08:20:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.505+0000 7f0685e00640 1 -- 172.21.15.2:0/2654539926 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06801a2320 con 0x7f0680104f50 2024-09-10T08:20:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.508+0000 7f0688e5e740 1 Processor -- start 2024-09-10T08:20:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.508+0000 7f0684a00640 1 -- 172.21.15.2:0/2654539926 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0668010ac0 con 0x7f0680104f50 2024-09-10T08:20:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.508+0000 7f0688e5e740 1 -- start start 2024-09-10T08:20:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f0688e5e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 0x55faf4418490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f0688e5e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf44189d0 0x55faf441d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f0688e5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf441da60 0x55faf441ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f0688e5e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55faf437d100 con 0x55faf4418090 2024-09-10T08:20:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f0688e5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55faf43251b0 con 0x55faf441da60 2024-09-10T08:20:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f0688e5e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55faf43790c0 con 0x55faf44189d0 2024-09-10T08:20:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f067f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 0x55faf4418490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f067f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 0x55faf4418490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48176/0 (socket says 172.21.15.2:48176) 2024-09-10T08:20:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f067f400640 1 -- 172.21.15.2:0/3585810740 learned_addr learned my addr 172.21.15.2:0/3585810740 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f0677e00640 1 --2- 172.21.15.2:0/3585810740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf441da60 0x55faf441ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f067ea00640 1 --2- 172.21.15.2:0/3585810740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf44189d0 0x55faf441d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f067f400640 1 -- 172.21.15.2:0/3585810740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf44189d0 msgr2=0x55faf441d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f067f400640 1 --2- 172.21.15.2:0/3585810740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf44189d0 0x55faf441d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f067f400640 1 -- 172.21.15.2:0/3585810740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf441da60 msgr2=0x55faf441ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f067f400640 1 --2- 172.21.15.2:0/3585810740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf441da60 0x55faf441ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.509+0000 7f067f400640 1 -- 172.21.15.2:0/3585810740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faf4090210 con 0x55faf4418090 2024-09-10T08:20:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.510+0000 7f067f400640 1 --2- 172.21.15.2:0/3585810740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 0x55faf4418490 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0678009ca0 tx=0x7f06780093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6377434047e4dec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.510+0000 7f0684a00640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06640a2c30 0x7f06640a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.510+0000 7f0684a00640 1 -- 172.21.15.2:0/2654539926 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f06680cf7b0 con 0x7f0680104f50 2024-09-10T08:20:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.510+0000 7f067ea00640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06640a2c30 0x7f06640a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.510+0000 7f066fe00640 1 -- 172.21.15.2:0/3585810740 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678031650 con 0x55faf4418090 2024-09-10T08:20:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.511+0000 7f066fe00640 1 -- 172.21.15.2:0/3585810740 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0678036030 con 0x55faf4418090 2024-09-10T08:20:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.511+0000 7f066fe00640 1 -- 172.21.15.2:0/3585810740 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678004740 con 0x55faf4418090 2024-09-10T08:20:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.512+0000 7f0688e5e740 1 -- 172.21.15.2:0/3585810740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 msgr2=0x55faf4418490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.512+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3585810740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 0x55faf4418490 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0678009ca0 tx=0x7f06780093c0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.512+0000 7f067ea00640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06640a2c30 0x7f06640a50f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55faf4419400 tx=0x7f0670005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.512+0000 7f0688e5e740 1 -- 172.21.15.2:0/3585810740 shutdown_connections 2024-09-10T08:20:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.512+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3585810740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf441da60 0x55faf441ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.512+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3585810740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf44189d0 0x55faf441d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.512+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3585810740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 0x55faf4418490 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.512+0000 7f0688e5e740 1 -- 172.21.15.2:0/3585810740 >> 172.21.15.2:0/3585810740 conn(0x55faf4413700 msgr2=0x55faf4413b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.512+0000 7f0688e5e740 1 -- 172.21.15.2:0/3585810740 shutdown_connections 2024-09-10T08:20:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.512+0000 7f0688e5e740 1 -- 172.21.15.2:0/3585810740 wait complete. 2024-09-10T08:20:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.513+0000 7f0688e5e740 1 -- start start 2024-09-10T08:20:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.513+0000 7f0688e5e740 1 Processor -- start 2024-09-10T08:20:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.514+0000 7f0688e5e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf4418090 0x55faf442a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.514+0000 7f0688e5e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf44189d0 0x55faf442ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.514+0000 7f0688e5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf441da60 0x55faf4431eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.514+0000 7f0688e5e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55faf4420850 con 0x55faf44189d0 2024-09-10T08:20:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.514+0000 7f0688e5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55faf4306090 con 0x55faf441da60 2024-09-10T08:20:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.514+0000 7f0688e5e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55faf44209d0 con 0x55faf4418090 2024-09-10T08:20:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.514+0000 7f0688e5e740 10 client.0 _ll_register_callbacks cb 0x55faf435af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.514+0000 7f067ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf4418090 0x55faf442a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.514+0000 7f0677e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf44189d0 0x55faf442ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.514+0000 7f067f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf441da60 0x55faf4431eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.514+0000 7f067ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf4418090 0x55faf442a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:43994/0 (socket says 172.21.15.2:43994) 2024-09-10T08:20:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.514+0000 7f067ea00640 1 -- 172.21.15.2:0/416677734 learned_addr learned my addr 172.21.15.2:0/416677734 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.515+0000 7f067f400640 1 -- 172.21.15.2:0/416677734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf4418090 msgr2=0x55faf442a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.515+0000 7f067f400640 1 --2- 172.21.15.2:0/416677734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf4418090 0x55faf442a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.515+0000 7f067f400640 1 -- 172.21.15.2:0/416677734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf44189d0 msgr2=0x55faf442ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.515+0000 7f067f400640 1 --2- 172.21.15.2:0/416677734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf44189d0 0x55faf442ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.515+0000 7f067f400640 1 -- 172.21.15.2:0/416677734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faf415c620 con 0x55faf441da60 2024-09-10T08:20:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.515+0000 7f0677e00640 1 --2- 172.21.15.2:0/416677734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf44189d0 0x55faf442ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.515+0000 7f067f400640 1 --2- 172.21.15.2:0/416677734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf441da60 0x55faf4431eb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f067802ade0 tx=0x7f0678031680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c9974320adbac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.515+0000 7f066cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:20:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.515+0000 7f066cc00640 1 -- 172.21.15.2:0/416677734 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067804a040 con 0x55faf441da60 2024-09-10T08:20:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.515+0000 7f0688e5e740 1 -- 172.21.15.2:0/416677734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0678036030 con 0x55faf441da60 2024-09-10T08:20:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.515+0000 7f066cc00640 1 -- 172.21.15.2:0/416677734 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f06780317f0 con 0x55faf441da60 2024-09-10T08:20:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.516+0000 7f066cc00640 1 -- 172.21.15.2:0/416677734 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f067805d050 con 0x55faf441da60 2024-09-10T08:20:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.516+0000 7f0688e5e740 10 client.28895 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.516+0000 7f0688e5e740 1 -- 172.21.15.2:0/416677734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55faf4420850 con 0x55faf441da60 2024-09-10T08:20:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.516+0000 7f066cc00640 1 -- 172.21.15.2:0/416677734 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0678036030 con 0x55faf441da60 2024-09-10T08:20:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.516+0000 7f066cc00640 1 -- 172.21.15.2:0/416677734 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f067804d030 con 0x55faf441da60 2024-09-10T08:20:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.516+0000 7f0688e5e740 10 client.28895 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.516+0000 7f0688e5e740 10 client.28895 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.516+0000 7f0688e5e740 20 client.28895 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.516+0000 7f0688e5e740 1 --2- 172.21.15.2:0/416677734 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55faf44360f0 0x55faf44384e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.516+0000 7f0688e5e740 4 client.28895 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.516+0000 7f0688e5e740 1 -- 172.21.15.2:0/416677734 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faf438a830 con 0x55faf44360f0 2024-09-10T08:20:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.517+0000 7f067ea00640 1 --2- 172.21.15.2:0/416677734 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55faf44360f0 0x55faf44384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.517+0000 7f067ea00640 1 --2- 172.21.15.2:0/416677734 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55faf44360f0 0x55faf44384e0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13b2b86926075f35 server_cookie=c34e1d396d8cfcf in_seq=0 out_seq=0 2024-09-10T08:20:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.517+0000 7f066cc00640 10 client.28895 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f066cc00640 1 -- 172.21.15.2:0/416677734 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55faf438a830 con 0x55faf44360f0 2024-09-10T08:20:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f066cc00640 10 client.28895 handle_command_reply: tid=0 2024-09-10T08:20:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f0688e5e740 1 client.28895 shutdown 2024-09-10T08:20:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f0688e5e740 1 -- 172.21.15.2:0/416677734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf441da60 msgr2=0x55faf4431eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f0688e5e740 1 --2- 172.21.15.2:0/416677734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf441da60 0x55faf4431eb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f067802ade0 tx=0x7f0678031680 comp rx=0 tx=0).stop 2024-09-10T08:20:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f0688e5e740 1 -- 172.21.15.2:0/416677734 shutdown_connections 2024-09-10T08:20:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f0688e5e740 1 --2- 172.21.15.2:0/416677734 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55faf44360f0 0x55faf44384e0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f0688e5e740 1 --2- 172.21.15.2:0/416677734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf441da60 0x55faf4431eb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f0688e5e740 1 --2- 172.21.15.2:0/416677734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf44189d0 0x55faf442ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f0688e5e740 1 --2- 172.21.15.2:0/416677734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf4418090 0x55faf442a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f066cc00640 0 client.28895 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f0688e5e740 1 -- 172.21.15.2:0/416677734 >> 172.21.15.2:0/416677734 conn(0x55faf4413700 msgr2=0x55faf4414f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f0688e5e740 1 -- 172.21.15.2:0/416677734 shutdown_connections 2024-09-10T08:20:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.518+0000 7f0688e5e740 1 -- 172.21.15.2:0/416677734 wait complete. 2024-09-10T08:20:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.519+0000 7f0688e5e740 20 client.28895 trim_cache size 0 max 16384 2024-09-10T08:20:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.535+0000 7f0688e5e740 1 Processor -- start 2024-09-10T08:20:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.535+0000 7f0688e5e740 1 -- start start 2024-09-10T08:20:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.535+0000 7f0688e5e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 0x55faf4418470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.535+0000 7f0688e5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf44189d0 0x55faf4431e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.535+0000 7f0688e5e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf441da60 0x55faf4414580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.535+0000 7f0688e5e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f06801aad90 con 0x55faf4418090 2024-09-10T08:20:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.535+0000 7f0688e5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06801a8660 con 0x55faf44189d0 2024-09-10T08:20:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.535+0000 7f0688e5e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f06801a64b0 con 0x55faf441da60 2024-09-10T08:20:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.536+0000 7f067ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 0x55faf4418470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.536+0000 7f067f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf441da60 0x55faf4414580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.536+0000 7f067ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 0x55faf4418470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48204/0 (socket says 172.21.15.2:48204) 2024-09-10T08:20:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.536+0000 7f067ea00640 1 -- 172.21.15.2:0/3591751756 learned_addr learned my addr 172.21.15.2:0/3591751756 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.536+0000 7f0677e00640 1 --2- 172.21.15.2:0/3591751756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf44189d0 0x55faf4431e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.536+0000 7f067ea00640 1 -- 172.21.15.2:0/3591751756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf441da60 msgr2=0x55faf4414580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.536+0000 7f067ea00640 1 --2- 172.21.15.2:0/3591751756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf441da60 0x55faf4414580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.536+0000 7f067ea00640 1 -- 172.21.15.2:0/3591751756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf44189d0 msgr2=0x55faf4431e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.536+0000 7f067ea00640 1 --2- 172.21.15.2:0/3591751756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf44189d0 0x55faf4431e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.536+0000 7f067ea00640 1 -- 172.21.15.2:0/3591751756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faf4438850 con 0x55faf4418090 2024-09-10T08:20:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.536+0000 7f067f400640 1 --2- 172.21.15.2:0/3591751756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf441da60 0x55faf4414580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.536+0000 7f067ea00640 1 --2- 172.21.15.2:0/3591751756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 0x55faf4418470 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0670009f20 tx=0x7f0670019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5730fc95459740f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.537+0000 7f066f400640 1 -- 172.21.15.2:0/3591751756 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067001dc70 con 0x55faf4418090 2024-09-10T08:20:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.537+0000 7f066f400640 1 -- 172.21.15.2:0/3591751756 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f067001de10 con 0x55faf4418090 2024-09-10T08:20:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.537+0000 7f066f400640 1 -- 172.21.15.2:0/3591751756 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067000f840 con 0x55faf4418090 2024-09-10T08:20:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.537+0000 7f0688e5e740 1 -- 172.21.15.2:0/3591751756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 msgr2=0x55faf4418470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.537+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3591751756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 0x55faf4418470 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0670009f20 tx=0x7f0670019000 comp rx=0 tx=0).stop 2024-09-10T08:20:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.537+0000 7f0688e5e740 1 -- 172.21.15.2:0/3591751756 shutdown_connections 2024-09-10T08:20:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.537+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3591751756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf441da60 0x55faf4414580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.537+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3591751756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf44189d0 0x55faf4431e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.537+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3591751756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf4418090 0x55faf4418470 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.537+0000 7f0688e5e740 1 -- 172.21.15.2:0/3591751756 >> 172.21.15.2:0/3591751756 conn(0x55faf4413700 msgr2=0x55faf442a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.537+0000 7f0688e5e740 1 -- 172.21.15.2:0/3591751756 shutdown_connections 2024-09-10T08:20:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.537+0000 7f0688e5e740 1 -- 172.21.15.2:0/3591751756 wait complete. 2024-09-10T08:20:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.538+0000 7f0688e5e740 1 -- start start 2024-09-10T08:20:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.538+0000 7f0688e5e740 1 Processor -- start 2024-09-10T08:20:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.538+0000 7f0688e5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf4418090 0x55faf442f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.538+0000 7f0688e5e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf44189d0 0x55faf442f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.538+0000 7f0688e5e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf441da60 0x55faf44307a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.538+0000 7f0688e5e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f06801a6b20 con 0x55faf441da60 2024-09-10T08:20:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.538+0000 7f0688e5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06801a7f90 con 0x55faf4418090 2024-09-10T08:20:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.538+0000 7f0688e5e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f06801a71f0 con 0x55faf44189d0 2024-09-10T08:20:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.538+0000 7f0688e5e740 10 client.0 _ll_register_callbacks cb 0x7f06680d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.539+0000 7f067f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf441da60 0x55faf44307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.539+0000 7f067ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf4418090 0x55faf442f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.539+0000 7f067f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf441da60 0x55faf44307a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48206/0 (socket says 172.21.15.2:48206) 2024-09-10T08:20:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.539+0000 7f067f400640 1 -- 172.21.15.2:0/3227402787 learned_addr learned my addr 172.21.15.2:0/3227402787 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.539+0000 7f0677e00640 1 --2- 172.21.15.2:0/3227402787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf44189d0 0x55faf442f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.539+0000 7f067f400640 1 -- 172.21.15.2:0/3227402787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf44189d0 msgr2=0x55faf442f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.539+0000 7f067f400640 1 --2- 172.21.15.2:0/3227402787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf44189d0 0x55faf442f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.539+0000 7f067f400640 1 -- 172.21.15.2:0/3227402787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf4418090 msgr2=0x55faf442f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.539+0000 7f067f400640 1 --2- 172.21.15.2:0/3227402787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf4418090 0x55faf442f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.539+0000 7f067f400640 1 -- 172.21.15.2:0/3227402787 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faf4430db0 con 0x55faf441da60 2024-09-10T08:20:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.539+0000 7f067ea00640 1 --2- 172.21.15.2:0/3227402787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf4418090 0x55faf442f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.539+0000 7f067f400640 1 --2- 172.21.15.2:0/3227402787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf441da60 0x55faf44307a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0678009620 tx=0x7f0678009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2141e84156e70b33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.539+0000 7f066cc00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f066cc00640 1 -- 172.21.15.2:0/3227402787 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067802f610 con 0x55faf441da60 2024-09-10T08:20:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f0688e5e740 1 -- 172.21.15.2:0/3227402787 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f067001de10 con 0x55faf441da60 2024-09-10T08:20:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f066cc00640 1 -- 172.21.15.2:0/3227402787 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f06780316f0 con 0x55faf441da60 2024-09-10T08:20:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f066cc00640 1 -- 172.21.15.2:0/3227402787 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f067806a050 con 0x55faf441da60 2024-09-10T08:20:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f0688e5e740 10 client.18876 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f0688e5e740 1 -- 172.21.15.2:0/3227402787 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55faf44320c0 con 0x55faf441da60 2024-09-10T08:20:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f066cc00640 1 -- 172.21.15.2:0/3227402787 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f067001de10 con 0x55faf441da60 2024-09-10T08:20:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f066cc00640 1 -- 172.21.15.2:0/3227402787 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f067804d040 con 0x55faf441da60 2024-09-10T08:20:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f0688e5e740 10 client.18876 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f0688e5e740 10 client.18876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f0688e5e740 20 client.18876 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3227402787 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55faf44a1380 0x55faf44a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f0688e5e740 4 client.18876 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:20:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.540+0000 7f0688e5e740 1 -- 172.21.15.2:0/3227402787 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55faf440ed80 con 0x55faf44a1380 2024-09-10T08:20:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.541+0000 7f067ea00640 1 --2- 172.21.15.2:0/3227402787 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55faf44a1380 0x55faf44a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.541+0000 7f067ea00640 1 --2- 172.21.15.2:0/3227402787 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55faf44a1380 0x55faf44a37a0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde53a83b1374622 server_cookie=18530a2906344574 in_seq=0 out_seq=0 2024-09-10T08:20:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.541+0000 7f066cc00640 10 client.18876 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.554+0000 7f066cc00640 1 -- 172.21.15.2:0/3227402787 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55faf440ed80 con 0x55faf44a1380 2024-09-10T08:20:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.554+0000 7f066cc00640 10 client.18876 handle_command_reply: tid=0 2024-09-10T08:20:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.554+0000 7f0688e5e740 1 client.18876 shutdown 2024-09-10T08:20:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.555+0000 7f0688e5e740 1 -- 172.21.15.2:0/3227402787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf441da60 msgr2=0x55faf44307a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.555+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3227402787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf441da60 0x55faf44307a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0678009620 tx=0x7f0678009bb0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.555+0000 7f0688e5e740 1 -- 172.21.15.2:0/3227402787 shutdown_connections 2024-09-10T08:20:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.555+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3227402787 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55faf44a1380 0x55faf44a37a0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.555+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3227402787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55faf441da60 0x55faf44307a0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.555+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3227402787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf44189d0 0x55faf442f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.555+0000 7f066cc00640 0 client.18876 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.555+0000 7f0688e5e740 1 --2- 172.21.15.2:0/3227402787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55faf4418090 0x55faf442f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.555+0000 7f0688e5e740 1 -- 172.21.15.2:0/3227402787 >> 172.21.15.2:0/3227402787 conn(0x55faf4413700 msgr2=0x55faf4412060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.555+0000 7f0688e5e740 1 -- 172.21.15.2:0/3227402787 shutdown_connections 2024-09-10T08:20:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.555+0000 7f0688e5e740 1 -- 172.21.15.2:0/3227402787 wait complete. 2024-09-10T08:20:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.555+0000 7f0688e5e740 20 client.18876 trim_cache size 0 max 16384 2024-09-10T08:20:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:20:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:20:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:20:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:20:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.556+0000 7f0685e00640 1 -- 172.21.15.2:0/2654539926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06640a2c30 msgr2=0x7f06640a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.556+0000 7f0685e00640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06640a2c30 0x7f06640a50f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55faf4419400 tx=0x7f0670005f90 comp rx=0 tx=0).stop 2024-09-10T08:20:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.556+0000 7f0685e00640 1 -- 172.21.15.2:0/2654539926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0680104f50 msgr2=0x7f068019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.556+0000 7f0685e00640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0680104f50 0x7f068019a490 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f066800e9f0 tx=0x7f066800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.556+0000 7f0685e00640 1 -- 172.21.15.2:0/2654539926 shutdown_connections 2024-09-10T08:20:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.556+0000 7f0685e00640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f06640a2c30 0x7f06640a50f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.556+0000 7f0685e00640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0680105910 0x7f06801a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.556+0000 7f0685e00640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0680104f50 0x7f068019a490 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.556+0000 7f0685e00640 1 --2- 172.21.15.2:0/2654539926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0680103d60 0x7f0680199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.556+0000 7f0685e00640 1 -- 172.21.15.2:0/2654539926 >> 172.21.15.2:0/2654539926 conn(0x7f06800ff4e0 msgr2=0x7f06801081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.556+0000 7f0685e00640 1 -- 172.21.15.2:0/2654539926 shutdown_connections 2024-09-10T08:20:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:21.556+0000 7f0685e00640 1 -- 172.21.15.2:0/2654539926 wait complete. 2024-09-10T08:20:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:21 smithi155 ceph-mon[35405]: pgmap v753: 129 pgs: 129 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:20:21.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:21 smithi107 ceph-mon[35048]: pgmap v753: 129 pgs: 129 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:20:22.019 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.1 2024-09-10T08:20:22.019 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:20:22.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:21 smithi002 ceph-mon[30563]: pgmap v753: 129 pgs: 129 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:20:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.120+0000 7f1894c00640 1 -- 172.21.15.2:0/1114693701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1890074f40 msgr2=0x7f189010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.120+0000 7f1894c00640 1 --2- 172.21.15.2:0/1114693701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1890074f40 0x7f189010de80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f187c009900 tx=0x7f187c02a480 comp rx=0 tx=0).stop 2024-09-10T08:20:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.120+0000 7f1894c00640 1 -- 172.21.15.2:0/1114693701 shutdown_connections 2024-09-10T08:20:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.120+0000 7f1894c00640 1 --2- 172.21.15.2:0/1114693701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f189010e3c0 0x7f1890110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.120+0000 7f1894c00640 1 --2- 172.21.15.2:0/1114693701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1890074f40 0x7f189010de80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.120+0000 7f1894c00640 1 --2- 172.21.15.2:0/1114693701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1890074620 0x7f1890074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.120+0000 7f1894c00640 1 -- 172.21.15.2:0/1114693701 >> 172.21.15.2:0/1114693701 conn(0x7f18900fd8c0 msgr2=0x7f18900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.120+0000 7f1894c00640 1 -- 172.21.15.2:0/1114693701 shutdown_connections 2024-09-10T08:20:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.120+0000 7f1894c00640 1 -- 172.21.15.2:0/1114693701 wait complete. 2024-09-10T08:20:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.121+0000 7f1894c00640 1 Processor -- start 2024-09-10T08:20:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.121+0000 7f1894c00640 1 -- start start 2024-09-10T08:20:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.122+0000 7f1894c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1890074620 0x7f189019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.122+0000 7f1894c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1890074f40 0x7f189019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.122+0000 7f1894c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f189010e3c0 0x7f189019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.122+0000 7f1894c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1890114140 con 0x7f189010e3c0 2024-09-10T08:20:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.122+0000 7f1894c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1890113fc0 con 0x7f1890074f40 2024-09-10T08:20:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.122+0000 7f1894c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f18901142c0 con 0x7f1890074620 2024-09-10T08:20:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.122+0000 7f188e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f189010e3c0 0x7f189019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.122+0000 7f188e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f189010e3c0 0x7f189019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48210/0 (socket says 172.21.15.2:48210) 2024-09-10T08:20:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.122+0000 7f188e800640 1 -- 172.21.15.2:0/1152824658 learned_addr learned my addr 172.21.15.2:0/1152824658 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.122+0000 7f188d400640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1890074f40 0x7f189019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.122+0000 7f188de00640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1890074620 0x7f189019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.123+0000 7f188e800640 1 -- 172.21.15.2:0/1152824658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1890074620 msgr2=0x7f189019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:23.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.123+0000 7f188e800640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1890074620 0x7f189019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:23.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.123+0000 7f188e800640 1 -- 172.21.15.2:0/1152824658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1890074f40 msgr2=0x7f189019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:23.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.123+0000 7f188e800640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1890074f40 0x7f189019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:23.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.123+0000 7f188e800640 1 -- 172.21.15.2:0/1152824658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189019f630 con 0x7f189010e3c0 2024-09-10T08:20:23.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.123+0000 7f188d400640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1890074f40 0x7f189019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:23.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.123+0000 7f188e800640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f189010e3c0 0x7f189019ef50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f188800eb10 tx=0x7f188800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:23.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.123+0000 7f1886a00640 1 -- 172.21.15.2:0/1152824658 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888004b20 con 0x7f189010e3c0 2024-09-10T08:20:23.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.123+0000 7f1894c00640 1 -- 172.21.15.2:0/1152824658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189019f920 con 0x7f189010e3c0 2024-09-10T08:20:23.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.123+0000 7f1886a00640 1 -- 172.21.15.2:0/1152824658 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1888004cc0 con 0x7f189010e3c0 2024-09-10T08:20:23.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.123+0000 7f1886a00640 1 -- 172.21.15.2:0/1152824658 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888010760 con 0x7f189010e3c0 2024-09-10T08:20:23.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.124+0000 7f1894c00640 1 -- 172.21.15.2:0/1152824658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18901a7200 con 0x7f189010e3c0 2024-09-10T08:20:23.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.125+0000 7f1894c00640 1 -- 172.21.15.2:0/1152824658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1890074a00 con 0x7f189010e3c0 2024-09-10T08:20:23.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.127+0000 7f1886a00640 1 -- 172.21.15.2:0/1152824658 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1888005000 con 0x7f189010e3c0 2024-09-10T08:20:23.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.128+0000 7f1886a00640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f18740a2c00 0x7f18740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:23.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.128+0000 7f1886a00640 1 -- 172.21.15.2:0/1152824658 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f18880cfb30 con 0x7f189010e3c0 2024-09-10T08:20:23.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.128+0000 7f188de00640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f18740a2c00 0x7f18740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:23.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.129+0000 7f188de00640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f18740a2c00 0x7f18740a50c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1880009bd0 tx=0x7f1880005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:23.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.130+0000 7f1886a00640 1 -- 172.21.15.2:0/1152824658 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18880541b0 con 0x7f189010e3c0 2024-09-10T08:20:23.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.354+0000 7f1894c00640 1 -- 172.21.15.2:0/1152824658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f189002d7c0 con 0x7f189010e3c0 2024-09-10T08:20:23.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.354+0000 7f1886a00640 1 -- 172.21.15.2:0/1152824658 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f188809c640 con 0x7f189010e3c0 2024-09-10T08:20:23.354 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:20:23.354 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:20:23.354 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:20:23.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.357+0000 7f1894c00640 1 -- 172.21.15.2:0/1152824658 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f18740a2c00 msgr2=0x7f18740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:23.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.357+0000 7f1894c00640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f18740a2c00 0x7f18740a50c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1880009bd0 tx=0x7f1880005f90 comp rx=0 tx=0).stop 2024-09-10T08:20:23.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.357+0000 7f1894c00640 1 -- 172.21.15.2:0/1152824658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f189010e3c0 msgr2=0x7f189019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:23.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.357+0000 7f1894c00640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f189010e3c0 0x7f189019ef50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f188800eb10 tx=0x7f188800c2d0 comp rx=0 tx=0).stop 2024-09-10T08:20:23.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.357+0000 7f1894c00640 1 -- 172.21.15.2:0/1152824658 shutdown_connections 2024-09-10T08:20:23.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.357+0000 7f1894c00640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f189010e3c0 0x7f189019ef50 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:23.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.357+0000 7f1894c00640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1890074f40 0x7f189019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:23.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.357+0000 7f1894c00640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f18740a2c00 0x7f18740a50c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:23.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.357+0000 7f1894c00640 1 --2- 172.21.15.2:0/1152824658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1890074620 0x7f189019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:23.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.357+0000 7f1894c00640 1 -- 172.21.15.2:0/1152824658 >> 172.21.15.2:0/1152824658 conn(0x7f18900fd8c0 msgr2=0x7f18900fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:23.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.357+0000 7f1894c00640 1 -- 172.21.15.2:0/1152824658 shutdown_connections 2024-09-10T08:20:23.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:23.357+0000 7f1894c00640 1 -- 172.21.15.2:0/1152824658 wait complete. 2024-09-10T08:20:23.722 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:23 smithi002 ceph-mon[30563]: pgmap v754: 129 pgs: 129 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:20:23.722 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:23 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1152824658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:20:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:23 smithi155 ceph-mon[35405]: pgmap v754: 129 pgs: 129 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:20:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:23 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1152824658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:20:23.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:23 smithi107 ceph-mon[35048]: pgmap v754: 129 pgs: 129 active+clean; 2.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:20:23.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:23 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1152824658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:20:24.410 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 138.1 --await-for 45.0' 2024-09-10T08:20:24.410 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 138.1 --await-for 45.0 2024-09-10T08:20:24.797 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.408+0000 7f82d8a00640 1 -- 172.21.15.2:0/3555445850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82d4108f60 msgr2=0x7f82d4110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.408+0000 7f82d8a00640 1 --2- 172.21.15.2:0/3555445850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82d4108f60 0x7f82d4110ae0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009ca0 tx=0x7f82cc02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.408+0000 7f82d8a00640 1 -- 172.21.15.2:0/3555445850 shutdown_connections 2024-09-10T08:20:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.408+0000 7f82d8a00640 1 --2- 172.21.15.2:0/3555445850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82d4108f60 0x7f82d4110ae0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.408+0000 7f82d8a00640 1 --2- 172.21.15.2:0/3555445850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d41047e0 0x7f82d4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.408+0000 7f82d8a00640 1 --2- 172.21.15.2:0/3555445850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d4103e30 0x7f82d4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.408+0000 7f82d8a00640 1 -- 172.21.15.2:0/3555445850 >> 172.21.15.2:0/3555445850 conn(0x7f82d40ff8d0 msgr2=0x7f82d4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.408+0000 7f82d8a00640 1 -- 172.21.15.2:0/3555445850 shutdown_connections 2024-09-10T08:20:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.408+0000 7f82d8a00640 1 -- 172.21.15.2:0/3555445850 wait complete. 2024-09-10T08:20:25.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.409+0000 7f82d8a00640 1 Processor -- start 2024-09-10T08:20:25.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.410+0000 7f82d8a00640 1 -- start start 2024-09-10T08:20:25.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.410+0000 7f82d8a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d4103e30 0x7f82d419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.410+0000 7f82d8a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d41047e0 0x7f82d419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.410+0000 7f82d8a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82d4108f60 0x7f82d419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.410+0000 7f82d8a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4113fc0 con 0x7f82d4108f60 2024-09-10T08:20:25.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.410+0000 7f82d8a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4113e40 con 0x7f82d4103e30 2024-09-10T08:20:25.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.410+0000 7f82d8a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4114140 con 0x7f82d41047e0 2024-09-10T08:20:25.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.410+0000 7f82d3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d4103e30 0x7f82d419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.410+0000 7f82d3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d4103e30 0x7f82d419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44376/0 (socket says 172.21.15.2:44376) 2024-09-10T08:20:25.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.410+0000 7f82cbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d41047e0 0x7f82d419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.410+0000 7f82d3400640 1 -- 172.21.15.2:0/3751667942 learned_addr learned my addr 172.21.15.2:0/3751667942 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:25.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.410+0000 7f82d3e00640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82d4108f60 0x7f82d419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.411+0000 7f82cbe00640 1 -- 172.21.15.2:0/3751667942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d4103e30 msgr2=0x7f82d419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:25.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.411+0000 7f82cbe00640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d4103e30 0x7f82d419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.411+0000 7f82cbe00640 1 -- 172.21.15.2:0/3751667942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82d4108f60 msgr2=0x7f82d419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:25.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.411+0000 7f82cbe00640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82d4108f60 0x7f82d419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.411+0000 7f82cbe00640 1 -- 172.21.15.2:0/3751667942 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d419f390 con 0x7f82d41047e0 2024-09-10T08:20:25.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.411+0000 7f82d3400640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d4103e30 0x7f82d419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:25.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.411+0000 7f82cbe00640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d41047e0 0x7f82d419a920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00e9f0 tx=0x7f82bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:25.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.411+0000 7f82d0c00640 1 -- 172.21.15.2:0/3751667942 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00cd80 con 0x7f82d41047e0 2024-09-10T08:20:25.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.411+0000 7f82d8a00640 1 -- 172.21.15.2:0/3751667942 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d419f680 con 0x7f82d41047e0 2024-09-10T08:20:25.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.412+0000 7f82d0c00640 1 -- 172.21.15.2:0/3751667942 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82bc0035f0 con 0x7f82d41047e0 2024-09-10T08:20:25.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.412+0000 7f82d0c00640 1 -- 172.21.15.2:0/3751667942 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc018690 con 0x7f82d41047e0 2024-09-10T08:20:25.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.412+0000 7f82d8a00640 1 -- 172.21.15.2:0/3751667942 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d41a6f60 con 0x7f82d41047e0 2024-09-10T08:20:25.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.415+0000 7f82dc9c5740 1 Processor -- start 2024-09-10T08:20:25.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.415+0000 7f82dc9c5740 1 -- start start 2024-09-10T08:20:25.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.415+0000 7f82d0c00640 1 -- 172.21.15.2:0/3751667942 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f82bc004020 con 0x7f82d41047e0 2024-09-10T08:20:25.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.415+0000 7f82dc9c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65560 0x55dab3b65940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.415+0000 7f82dc9c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65e80 0x55dab3b70200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.415+0000 7f82dc9c5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 0x55dab3b72b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.415+0000 7f82dc9c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55dab3acb530 con 0x55dab3b65560 2024-09-10T08:20:25.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.415+0000 7f82dc9c5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dab3a72860 con 0x55dab3b70740 2024-09-10T08:20:25.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.415+0000 7f82dc9c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dab3ac74f0 con 0x55dab3b65e80 2024-09-10T08:20:25.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.416+0000 7f82d3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65560 0x55dab3b65940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.416+0000 7f82d3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65560 0x55dab3b65940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48234/0 (socket says 172.21.15.2:48234) 2024-09-10T08:20:25.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.416+0000 7f82d3e00640 1 -- 172.21.15.2:0/1089831649 learned_addr learned my addr 172.21.15.2:0/1089831649 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:25.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.416+0000 7f82d3400640 1 --2- 172.21.15.2:0/1089831649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65e80 0x55dab3b70200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.416+0000 7f82cbe00640 1 --2- 172.21.15.2:0/1089831649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 0x55dab3b72b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.416+0000 7f82d3e00640 1 -- 172.21.15.2:0/1089831649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65e80 msgr2=0x55dab3b70200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:25.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.416+0000 7f82d3e00640 1 --2- 172.21.15.2:0/1089831649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65e80 0x55dab3b70200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.416+0000 7f82d3e00640 1 -- 172.21.15.2:0/1089831649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 msgr2=0x55dab3b72b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:25.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.416+0000 7f82d3e00640 1 --2- 172.21.15.2:0/1089831649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 0x55dab3b72b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.416+0000 7f82d3e00640 1 -- 172.21.15.2:0/1089831649 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab383cb90 con 0x55dab3b65560 2024-09-10T08:20:25.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.416+0000 7f82d3400640 1 --2- 172.21.15.2:0/1089831649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65e80 0x55dab3b70200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:25.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.416+0000 7f82d3e00640 1 --2- 172.21.15.2:0/1089831649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65560 0x55dab3b65940 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009ca0 tx=0x7f82cc005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f6a8f771c4cc68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:25.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.417+0000 7f82c3e00640 1 -- 172.21.15.2:0/1089831649 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc031db0 con 0x55dab3b65560 2024-09-10T08:20:25.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.417+0000 7f82c3e00640 1 -- 172.21.15.2:0/1089831649 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82cc036040 con 0x55dab3b65560 2024-09-10T08:20:25.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.417+0000 7f82c3e00640 1 -- 172.21.15.2:0/1089831649 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0042d0 con 0x55dab3b65560 2024-09-10T08:20:25.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.417+0000 7f82d0c00640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82b80a2b80 0x7f82b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.417+0000 7f82d0c00640 1 -- 172.21.15.2:0/3751667942 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f82bc0cf640 con 0x7f82d41047e0 2024-09-10T08:20:25.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.417+0000 7f82d3400640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82b80a2b80 0x7f82b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.418+0000 7f82dc9c5740 1 -- 172.21.15.2:0/1089831649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65560 msgr2=0x55dab3b65940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:25.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.418+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/1089831649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65560 0x55dab3b65940 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009ca0 tx=0x7f82cc005f10 comp rx=0 tx=0).stop 2024-09-10T08:20:25.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.418+0000 7f82dc9c5740 1 -- 172.21.15.2:0/1089831649 shutdown_connections 2024-09-10T08:20:25.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.418+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/1089831649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 0x55dab3b72b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.418+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/1089831649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65e80 0x55dab3b70200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.418+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/1089831649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65560 0x55dab3b65940 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.418+0000 7f82dc9c5740 1 -- 172.21.15.2:0/1089831649 >> 172.21.15.2:0/1089831649 conn(0x55dab3b61af0 msgr2=0x55dab3b61ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:25.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.418+0000 7f82dc9c5740 1 -- 172.21.15.2:0/1089831649 shutdown_connections 2024-09-10T08:20:25.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.418+0000 7f82d3400640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82b80a2b80 0x7f82b80a5040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009620 tx=0x7f82c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:25.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.418+0000 7f82dc9c5740 1 -- 172.21.15.2:0/1089831649 wait complete. 2024-09-10T08:20:25.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.419+0000 7f82dc9c5740 1 -- start start 2024-09-10T08:20:25.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.419+0000 7f82dc9c5740 1 Processor -- start 2024-09-10T08:20:25.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.419+0000 7f82dc9c5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b65560 0x55dab3b6d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.419+0000 7f82dc9c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65e80 0x55dab3b6dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.419+0000 7f82dc9c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b70740 0x55dab3b6e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.419+0000 7f82dc9c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55dab3b73190 con 0x55dab3b65e80 2024-09-10T08:20:25.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.419+0000 7f82dc9c5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dab3a85e60 con 0x55dab3b65560 2024-09-10T08:20:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.419+0000 7f82dc9c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dab3b73310 con 0x55dab3b70740 2024-09-10T08:20:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.420+0000 7f82dc9c5740 10 client.0 _ll_register_callbacks cb 0x55dab3a8acc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.420+0000 7f82d3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b65560 0x55dab3b6d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.420+0000 7f82cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65e80 0x55dab3b6dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.420+0000 7f82d3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b65560 0x55dab3b6d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:44394/0 (socket says 172.21.15.2:44394) 2024-09-10T08:20:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.420+0000 7f82d3400640 1 -- 172.21.15.2:0/3752421510 learned_addr learned my addr 172.21.15.2:0/3752421510 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.420+0000 7f82d3e00640 1 --2- 172.21.15.2:0/3752421510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b70740 0x55dab3b6e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.420+0000 7f82cbe00640 1 -- 172.21.15.2:0/3752421510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b70740 msgr2=0x55dab3b6e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.420+0000 7f82cbe00640 1 --2- 172.21.15.2:0/3752421510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b70740 0x55dab3b6e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.420+0000 7f82cbe00640 1 -- 172.21.15.2:0/3752421510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b65560 msgr2=0x55dab3b6d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.420+0000 7f82cbe00640 1 --2- 172.21.15.2:0/3752421510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b65560 0x55dab3b6d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.420+0000 7f82cbe00640 1 -- 172.21.15.2:0/3752421510 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab3a91990 con 0x55dab3b65e80 2024-09-10T08:20:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.421+0000 7f82cbe00640 1 --2- 172.21.15.2:0/3752421510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65e80 0x55dab3b6dae0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0ccda0 tx=0x7f82bc09b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2edb20abaeae0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:25.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.421+0000 7f82c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:25.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.421+0000 7f82c0c00640 1 -- 172.21.15.2:0/3752421510 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc09c1b0 con 0x55dab3b65e80 2024-09-10T08:20:25.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.421+0000 7f82dc9c5740 1 -- 172.21.15.2:0/3752421510 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82cc036040 con 0x55dab3b65e80 2024-09-10T08:20:25.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.421+0000 7f82c0c00640 1 -- 172.21.15.2:0/3752421510 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f82bc099db0 con 0x55dab3b65e80 2024-09-10T08:20:25.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.421+0000 7f82c0c00640 1 -- 172.21.15.2:0/3752421510 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f82bc09bab0 con 0x55dab3b65e80 2024-09-10T08:20:25.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.421+0000 7f82dc9c5740 10 client.18894 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:25.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.421+0000 7f82dc9c5740 1 -- 172.21.15.2:0/3752421510 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dab3b7ee00 con 0x55dab3b65e80 2024-09-10T08:20:25.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.421+0000 7f82c0c00640 1 -- 172.21.15.2:0/3752421510 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82cc036040 con 0x55dab3b65e80 2024-09-10T08:20:25.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.422+0000 7f82c0c00640 1 -- 172.21.15.2:0/3752421510 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f82bc0cf310 con 0x55dab3b65e80 2024-09-10T08:20:25.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.422+0000 7f82dc9c5740 10 client.18894 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:25.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.422+0000 7f82dc9c5740 10 client.18894 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:20:25.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.422+0000 7f82dc9c5740 20 client.18894 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:25.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.422+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/3752421510 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55dab3b80210 0x55dab3b82600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.422+0000 7f82dc9c5740 4 client.18894 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:25.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.422+0000 7f82dc9c5740 1 -- 172.21.15.2:0/3752421510 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dab3a6e310 con 0x55dab3b80210 2024-09-10T08:20:25.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.422+0000 7f82d3400640 1 --2- 172.21.15.2:0/3752421510 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55dab3b80210 0x55dab3b82600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.423+0000 7f82d3400640 1 --2- 172.21.15.2:0/3752421510 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55dab3b80210 0x55dab3b82600 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73154921a8950679 server_cookie=4f432e5b4a2590da in_seq=0 out_seq=0 2024-09-10T08:20:25.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.423+0000 7f82c0c00640 10 client.18894 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:25.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82c0c00640 1 -- 172.21.15.2:0/3752421510 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dab3a6e310 con 0x55dab3b80210 2024-09-10T08:20:25.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82c0c00640 10 client.18894 handle_command_reply: tid=0 2024-09-10T08:20:25.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82dc9c5740 1 client.18894 shutdown 2024-09-10T08:20:25.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82dc9c5740 1 -- 172.21.15.2:0/3752421510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65e80 msgr2=0x55dab3b6dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:25.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/3752421510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65e80 0x55dab3b6dae0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0ccda0 tx=0x7f82bc09b880 comp rx=0 tx=0).stop 2024-09-10T08:20:25.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82dc9c5740 1 -- 172.21.15.2:0/3752421510 shutdown_connections 2024-09-10T08:20:25.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/3752421510 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55dab3b80210 0x55dab3b82600 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/3752421510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b70740 0x55dab3b6e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/3752421510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65e80 0x55dab3b6dae0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/3752421510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b65560 0x55dab3b6d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82c0c00640 0 client.18894 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:25.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82dc9c5740 1 -- 172.21.15.2:0/3752421510 >> 172.21.15.2:0/3752421510 conn(0x55dab3b61af0 msgr2=0x55dab3b60c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:25.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82dc9c5740 1 -- 172.21.15.2:0/3752421510 shutdown_connections 2024-09-10T08:20:25.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82dc9c5740 1 -- 172.21.15.2:0/3752421510 wait complete. 2024-09-10T08:20:25.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.424+0000 7f82dc9c5740 20 client.18894 trim_cache size 0 max 16384 2024-09-10T08:20:25.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.442+0000 7f82dc9c5740 1 Processor -- start 2024-09-10T08:20:25.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.442+0000 7f82dc9c5740 1 -- start start 2024-09-10T08:20:25.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.442+0000 7f82dc9c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65560 0x55dab3b87ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82dc9c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65e80 0x55dab3b883e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82dc9c5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 0x55dab3b6a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82dc9c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41b3690 con 0x55dab3b65e80 2024-09-10T08:20:25.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82dc9c5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41b0f60 con 0x55dab3b70740 2024-09-10T08:20:25.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82dc9c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41aedb0 con 0x55dab3b65560 2024-09-10T08:20:25.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65e80 0x55dab3b883e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82d3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65560 0x55dab3b87ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82cbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65e80 0x55dab3b883e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48256/0 (socket says 172.21.15.2:48256) 2024-09-10T08:20:25.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82cbe00640 1 -- 172.21.15.2:0/3835160652 learned_addr learned my addr 172.21.15.2:0/3835160652 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:25.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82d3e00640 1 --2- 172.21.15.2:0/3835160652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 0x55dab3b6a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82d3e00640 1 -- 172.21.15.2:0/3835160652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65560 msgr2=0x55dab3b87ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:25.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82d3e00640 1 --2- 172.21.15.2:0/3835160652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65560 0x55dab3b87ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82d3e00640 1 -- 172.21.15.2:0/3835160652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65e80 msgr2=0x55dab3b883e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:25.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82d3e00640 1 --2- 172.21.15.2:0/3835160652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65e80 0x55dab3b883e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.443+0000 7f82d3e00640 1 -- 172.21.15.2:0/3835160652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab3b62ba0 con 0x55dab3b70740 2024-09-10T08:20:25.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.444+0000 7f82d3e00640 1 --2- 172.21.15.2:0/3835160652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 0x55dab3b6a800 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f82cc004af0 tx=0x7f82cc02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a08e3e9bb32a5d54 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:25.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.444+0000 7f82c3400640 1 -- 172.21.15.2:0/3835160652 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc052070 con 0x55dab3b70740 2024-09-10T08:20:25.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.444+0000 7f82c3400640 1 -- 172.21.15.2:0/3835160652 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82cc036040 con 0x55dab3b70740 2024-09-10T08:20:25.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.444+0000 7f82c3400640 1 -- 172.21.15.2:0/3835160652 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc003e60 con 0x55dab3b70740 2024-09-10T08:20:25.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.445+0000 7f82dc9c5740 1 -- 172.21.15.2:0/3835160652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 msgr2=0x55dab3b6a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:25.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.445+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/3835160652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 0x55dab3b6a800 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f82cc004af0 tx=0x7f82cc02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.445+0000 7f82dc9c5740 1 -- 172.21.15.2:0/3835160652 shutdown_connections 2024-09-10T08:20:25.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.445+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/3835160652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 0x55dab3b6a800 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.445+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/3835160652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65e80 0x55dab3b883e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.445+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/3835160652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65560 0x55dab3b87ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.445+0000 7f82dc9c5740 1 -- 172.21.15.2:0/3835160652 >> 172.21.15.2:0/3835160652 conn(0x55dab3b61af0 msgr2=0x55dab3b677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:25.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.445+0000 7f82dc9c5740 1 -- 172.21.15.2:0/3835160652 shutdown_connections 2024-09-10T08:20:25.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.445+0000 7f82dc9c5740 1 -- 172.21.15.2:0/3835160652 wait complete. 2024-09-10T08:20:25.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.446+0000 7f82dc9c5740 1 -- start start 2024-09-10T08:20:25.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.446+0000 7f82dc9c5740 1 Processor -- start 2024-09-10T08:20:25.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.446+0000 7f82dc9c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65560 0x55dab3b74e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.446+0000 7f82dc9c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65e80 0x55dab3b753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.446+0000 7f82dc9c5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 0x55dab3be2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.446+0000 7f82dc9c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41af420 con 0x55dab3b65560 2024-09-10T08:20:25.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.446+0000 7f82dc9c5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41b0890 con 0x55dab3b70740 2024-09-10T08:20:25.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.446+0000 7f82dc9c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41afaf0 con 0x55dab3b65e80 2024-09-10T08:20:25.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.446+0000 7f82dc9c5740 10 client.0 _ll_register_callbacks cb 0x7f82bc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:25.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.447+0000 7f82d3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65560 0x55dab3b74e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.447+0000 7f82cbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65e80 0x55dab3b753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.447+0000 7f82d3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 0x55dab3be2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.447+0000 7f82cbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65e80 0x55dab3b753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:44096/0 (socket says 172.21.15.2:44096) 2024-09-10T08:20:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.447+0000 7f82cbe00640 1 -- 172.21.15.2:0/122212956 learned_addr learned my addr 172.21.15.2:0/122212956 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.447+0000 7f82cbe00640 1 -- 172.21.15.2:0/122212956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 msgr2=0x55dab3be2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.447+0000 7f82cbe00640 1 --2- 172.21.15.2:0/122212956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 0x55dab3be2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.447+0000 7f82cbe00640 1 -- 172.21.15.2:0/122212956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65560 msgr2=0x55dab3b74e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.447+0000 7f82cbe00640 1 --2- 172.21.15.2:0/122212956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65560 0x55dab3b74e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.447+0000 7f82cbe00640 1 -- 172.21.15.2:0/122212956 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab3be34f0 con 0x55dab3b65e80 2024-09-10T08:20:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.447+0000 7f82d3400640 1 --2- 172.21.15.2:0/122212956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65560 0x55dab3b74e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.447+0000 7f82cbe00640 1 --2- 172.21.15.2:0/122212956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65e80 0x55dab3b753c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0d51b0 tx=0x7f82bc09b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70d8fe8aa8fc2f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.447+0000 7f82c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:20:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.448+0000 7f82c0c00640 1 -- 172.21.15.2:0/122212956 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc080060 con 0x55dab3b65e80 2024-09-10T08:20:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.448+0000 7f82c0c00640 1 -- 172.21.15.2:0/122212956 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f82bc0037a0 con 0x55dab3b65e80 2024-09-10T08:20:25.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.448+0000 7f82dc9c5740 1 -- 172.21.15.2:0/122212956 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82cc036040 con 0x55dab3b65e80 2024-09-10T08:20:25.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.448+0000 7f82c0c00640 1 -- 172.21.15.2:0/122212956 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f82bc09b5f0 con 0x55dab3b65e80 2024-09-10T08:20:25.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.448+0000 7f82dc9c5740 10 client.28858 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:25.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.448+0000 7f82dc9c5740 1 -- 172.21.15.2:0/122212956 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dab3beb990 con 0x55dab3b65e80 2024-09-10T08:20:25.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.448+0000 7f82c0c00640 1 -- 172.21.15.2:0/122212956 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82cc036040 con 0x55dab3b65e80 2024-09-10T08:20:25.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.448+0000 7f82c0c00640 1 -- 172.21.15.2:0/122212956 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f82bc0ba2b0 con 0x55dab3b65e80 2024-09-10T08:20:25.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.448+0000 7f82dc9c5740 10 client.28858 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:25.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.448+0000 7f82dc9c5740 10 client.28858 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:20:25.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.448+0000 7f82dc9c5740 20 client.28858 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:25.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.448+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/122212956 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55dab3bec130 0x55dab3bee550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:25.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.449+0000 7f82dc9c5740 4 client.28858 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.1, "await_for": 45.0}] 2024-09-10T08:20:25.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.449+0000 7f82dc9c5740 1 -- 172.21.15.2:0/122212956 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.1, "await_for": 45.0}) -- 0x55dab3ad8c60 con 0x55dab3bec130 2024-09-10T08:20:25.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.449+0000 7f82d3400640 1 --2- 172.21.15.2:0/122212956 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55dab3bec130 0x55dab3bee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:25.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.449+0000 7f82d3400640 1 --2- 172.21.15.2:0/122212956 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55dab3bec130 0x55dab3bee550 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=887dd662591675e0 server_cookie=26625bf5ed7e21c8 in_seq=0 out_seq=0 2024-09-10T08:20:25.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.449+0000 7f82c0c00640 10 client.28858 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:25.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82c0c00640 1 -- 172.21.15.2:0/122212956 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55dab3ad8c60 con 0x55dab3bec130 2024-09-10T08:20:25.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82c0c00640 10 client.28858 handle_command_reply: tid=0 2024-09-10T08:20:25.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82dc9c5740 1 client.28858 shutdown 2024-09-10T08:20:25.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82dc9c5740 1 -- 172.21.15.2:0/122212956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65e80 msgr2=0x55dab3b753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:25.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/122212956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65e80 0x55dab3b753c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0d51b0 tx=0x7f82bc09b9b0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82dc9c5740 1 -- 172.21.15.2:0/122212956 shutdown_connections 2024-09-10T08:20:25.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/122212956 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55dab3bec130 0x55dab3bee550 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/122212956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dab3b70740 0x55dab3be2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/122212956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dab3b65e80 0x55dab3b753c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82dc9c5740 1 --2- 172.21.15.2:0/122212956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dab3b65560 0x55dab3b74e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82c0c00640 0 client.28858 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:25.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82dc9c5740 1 -- 172.21.15.2:0/122212956 >> 172.21.15.2:0/122212956 conn(0x55dab3b61af0 msgr2=0x55dab3b60450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:25.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82dc9c5740 1 -- 172.21.15.2:0/122212956 shutdown_connections 2024-09-10T08:20:25.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82dc9c5740 1 -- 172.21.15.2:0/122212956 wait complete. 2024-09-10T08:20:25.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.461+0000 7f82dc9c5740 20 client.28858 trim_cache size 0 max 16384 2024-09-10T08:20:25.463 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:20:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:20:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:20:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 42, 2024-09-10T08:20:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:20:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: "41453dd0": { 2024-09-10T08:20:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 42, 2024-09-10T08:20:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:20:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:20:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:20:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:20:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:20:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:20:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 138.1, 2024-09-10T08:20:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:20:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:20:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:20:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:20:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:20:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:20:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:25.465 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:20:25.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.462+0000 7f82d8a00640 1 -- 172.21.15.2:0/3751667942 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82b80a2b80 msgr2=0x7f82b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:25.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.462+0000 7f82d8a00640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82b80a2b80 0x7f82b80a5040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009620 tx=0x7f82c4005f90 comp rx=0 tx=0).stop 2024-09-10T08:20:25.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.462+0000 7f82d8a00640 1 -- 172.21.15.2:0/3751667942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d41047e0 msgr2=0x7f82d419a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:25.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.462+0000 7f82d8a00640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d41047e0 0x7f82d419a920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00e9f0 tx=0x7f82bc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.462+0000 7f82d8a00640 1 -- 172.21.15.2:0/3751667942 shutdown_connections 2024-09-10T08:20:25.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.462+0000 7f82d8a00640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82d4108f60 0x7f82d419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.462+0000 7f82d8a00640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82b80a2b80 0x7f82b80a5040 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.462+0000 7f82d8a00640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d41047e0 0x7f82d419a920 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.462+0000 7f82d8a00640 1 --2- 172.21.15.2:0/3751667942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d4103e30 0x7f82d419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:25.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.462+0000 7f82d8a00640 1 -- 172.21.15.2:0/3751667942 >> 172.21.15.2:0/3751667942 conn(0x7f82d40ff8d0 msgr2=0x7f82d4100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:25.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.463+0000 7f82d8a00640 1 -- 172.21.15.2:0/3751667942 shutdown_connections 2024-09-10T08:20:25.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:25.463+0000 7f82d8a00640 1 -- 172.21.15.2:0/3751667942 wait complete. 2024-09-10T08:20:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:25 smithi155 ceph-mon[35405]: pgmap v755: 129 pgs: 129 active+clean; 2.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:20:25.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:25 smithi107 ceph-mon[35048]: pgmap v755: 129 pgs: 129 active+clean; 2.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:20:25.977 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:25 smithi002 ceph-mon[30563]: pgmap v755: 129 pgs: 129 active+clean; 2.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:20:26.265 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '41453dd0', quiesce took 4.2 seconds. Will release after: 13.900000000000002 2024-09-10T08:20:27.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:27 smithi155 ceph-mon[35405]: pgmap v756: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:20:27.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:27 smithi107 ceph-mon[35048]: pgmap v756: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:20:28.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:27 smithi002 ceph-mon[30563]: pgmap v756: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:20:29.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:29 smithi155 ceph-mon[35405]: pgmap v757: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:20:29.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:29 smithi107 ceph-mon[35048]: pgmap v757: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:20:30.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:29 smithi002 ceph-mon[30563]: pgmap v757: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:20:31.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:31 smithi155 ceph-mon[35405]: pgmap v758: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T08:20:31.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:31 smithi107 ceph-mon[35048]: pgmap v758: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T08:20:32.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:31 smithi002 ceph-mon[30563]: pgmap v758: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T08:20:33.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:33 smithi155 ceph-mon[35405]: pgmap v759: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-10T08:20:33.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:33 smithi107 ceph-mon[35048]: pgmap v759: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-10T08:20:34.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:33 smithi002 ceph-mon[30563]: pgmap v759: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-10T08:20:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.684+0000 7f91cd000640 1 -- 172.21.15.2:0/1900113434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91c8103d60 msgr2=0x7f91c8104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.684+0000 7f91cd000640 1 --2- 172.21.15.2:0/1900113434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91c8103d60 0x7f91c8104160 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009900 tx=0x7f91b802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.684+0000 7f91cd000640 1 -- 172.21.15.2:0/1900113434 shutdown_connections 2024-09-10T08:20:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.684+0000 7f91cd000640 1 --2- 172.21.15.2:0/1900113434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c8105910 0x7f91c810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.684+0000 7f91cd000640 1 --2- 172.21.15.2:0/1900113434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91c8104f50 0x7f91c81053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.685+0000 7f91cd000640 1 --2- 172.21.15.2:0/1900113434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91c8103d60 0x7f91c8104160 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.685+0000 7f91cd000640 1 -- 172.21.15.2:0/1900113434 >> 172.21.15.2:0/1900113434 conn(0x7f91c80ff4e0 msgr2=0x7f91c8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.685+0000 7f91cd000640 1 -- 172.21.15.2:0/1900113434 shutdown_connections 2024-09-10T08:20:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.685+0000 7f91cd000640 1 -- 172.21.15.2:0/1900113434 wait complete. 2024-09-10T08:20:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.686+0000 7f91cd000640 1 Processor -- start 2024-09-10T08:20:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.686+0000 7f91cd000640 1 -- start start 2024-09-10T08:20:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.686+0000 7f91cd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c8103d60 0x7f91c8199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.686+0000 7f91cd000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91c8104f50 0x7f91c819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.686+0000 7f91cd000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91c8105910 0x7f91c81a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.686+0000 7f91cd000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f91c810fa10 con 0x7f91c8104f50 2024-09-10T08:20:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.686+0000 7f91cd000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91c810f890 con 0x7f91c8103d60 2024-09-10T08:20:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.686+0000 7f91cd000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f91c810fb90 con 0x7f91c8105910 2024-09-10T08:20:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.686+0000 7f91c5400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91c8104f50 0x7f91c819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.686+0000 7f91c5400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91c8104f50 0x7f91c819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37138/0 (socket says 172.21.15.2:37138) 2024-09-10T08:20:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.686+0000 7f91c5400640 1 -- 172.21.15.2:0/3595441161 learned_addr learned my addr 172.21.15.2:0/3595441161 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.686+0000 7f91c5e00640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c8103d60 0x7f91c8199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.687+0000 7f91c6800640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91c8105910 0x7f91c81a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.687+0000 7f91c5400640 1 -- 172.21.15.2:0/3595441161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91c8105910 msgr2=0x7f91c81a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.687+0000 7f91c5400640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91c8105910 0x7f91c81a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.687+0000 7f91c5400640 1 -- 172.21.15.2:0/3595441161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c8103d60 msgr2=0x7f91c8199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.687+0000 7f91c5400640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c8103d60 0x7f91c8199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.687+0000 7f91c5400640 1 -- 172.21.15.2:0/3595441161 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c81a1bf0 con 0x7f91c8104f50 2024-09-10T08:20:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.687+0000 7f91c5e00640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c8103d60 0x7f91c8199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.687+0000 7f91c5400640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91c8104f50 0x7f91c819a510 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f91b400e9f0 tx=0x7f91b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.687+0000 7f91bea00640 1 -- 172.21.15.2:0/3595441161 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b400cd80 con 0x7f91c8104f50 2024-09-10T08:20:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.687+0000 7f91bea00640 1 -- 172.21.15.2:0/3595441161 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b4010910 con 0x7f91c8104f50 2024-09-10T08:20:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.688+0000 7f91cd000640 1 -- 172.21.15.2:0/3595441161 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c81a1ee0 con 0x7f91c8104f50 2024-09-10T08:20:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.688+0000 7f91bea00640 1 -- 172.21.15.2:0/3595441161 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4018990 con 0x7f91c8104f50 2024-09-10T08:20:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.688+0000 7f91cd000640 1 -- 172.21.15.2:0/3595441161 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c81a23f0 con 0x7f91c8104f50 2024-09-10T08:20:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.691+0000 7f91cfe94740 1 Processor -- start 2024-09-10T08:20:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.691+0000 7f91bea00640 1 -- 172.21.15.2:0/3595441161 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f91b4010ac0 con 0x7f91c8104f50 2024-09-10T08:20:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.691+0000 7f91cfe94740 1 -- start start 2024-09-10T08:20:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.691+0000 7f91cfe94740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab08090 0x55a43ab08490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.691+0000 7f91cfe94740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab089d0 0x55a43ab0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.691+0000 7f91cfe94740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab0da60 0x55a43ab0ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.691+0000 7f91cfe94740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a43aa6d100 con 0x55a43ab089d0 2024-09-10T08:20:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.691+0000 7f91cfe94740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a43aa151b0 con 0x55a43ab0da60 2024-09-10T08:20:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.691+0000 7f91cfe94740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a43aa690c0 con 0x55a43ab08090 2024-09-10T08:20:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.691+0000 7f91c5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab089d0 0x55a43ab0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.692+0000 7f91c5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab089d0 0x55a43ab0d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37146/0 (socket says 172.21.15.2:37146) 2024-09-10T08:20:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.692+0000 7f91c5e00640 1 -- 172.21.15.2:0/2874327049 learned_addr learned my addr 172.21.15.2:0/2874327049 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.692+0000 7f91c5400640 1 --2- 172.21.15.2:0/2874327049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab0da60 0x55a43ab0ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.692+0000 7f91c6800640 1 --2- 172.21.15.2:0/2874327049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab08090 0x55a43ab08490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.692+0000 7f91c5e00640 1 -- 172.21.15.2:0/2874327049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab08090 msgr2=0x55a43ab08490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.692+0000 7f91c5e00640 1 --2- 172.21.15.2:0/2874327049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab08090 0x55a43ab08490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.692+0000 7f91c5e00640 1 -- 172.21.15.2:0/2874327049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab0da60 msgr2=0x55a43ab0ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.692+0000 7f91c5e00640 1 --2- 172.21.15.2:0/2874327049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab0da60 0x55a43ab0ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.692+0000 7f91c5e00640 1 -- 172.21.15.2:0/2874327049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43a780210 con 0x55a43ab089d0 2024-09-10T08:20:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.692+0000 7f91c6800640 1 --2- 172.21.15.2:0/2874327049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab08090 0x55a43ab08490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.692+0000 7f91c5e00640 1 --2- 172.21.15.2:0/2874327049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab089d0 0x55a43ab0d490 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009900 tx=0x7f91b80043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1976785afc0f6a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.692+0000 7f91bea00640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f91ac0a2cb0 0x7f91ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.693+0000 7f91b3400640 1 -- 172.21.15.2:0/2874327049 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8035040 con 0x55a43ab089d0 2024-09-10T08:20:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.693+0000 7f91b3400640 1 -- 172.21.15.2:0/2874327049 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b8004750 con 0x55a43ab089d0 2024-09-10T08:20:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.693+0000 7f91b3400640 1 -- 172.21.15.2:0/2874327049 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8031a10 con 0x55a43ab089d0 2024-09-10T08:20:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.693+0000 7f91bea00640 1 -- 172.21.15.2:0/3595441161 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f91b40cf710 con 0x7f91c8104f50 2024-09-10T08:20:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.693+0000 7f91c6800640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f91ac0a2cb0 0x7f91ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.693+0000 7f91cfe94740 1 -- 172.21.15.2:0/2874327049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab089d0 msgr2=0x55a43ab0d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.693+0000 7f91cfe94740 1 --2- 172.21.15.2:0/2874327049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab089d0 0x55a43ab0d490 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009900 tx=0x7f91b80043f0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.693+0000 7f91cfe94740 1 -- 172.21.15.2:0/2874327049 shutdown_connections 2024-09-10T08:20:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.693+0000 7f91cfe94740 1 --2- 172.21.15.2:0/2874327049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab0da60 0x55a43ab0ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.693+0000 7f91cfe94740 1 --2- 172.21.15.2:0/2874327049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab089d0 0x55a43ab0d490 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.693+0000 7f91cfe94740 1 --2- 172.21.15.2:0/2874327049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab08090 0x55a43ab08490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.693+0000 7f91cfe94740 1 -- 172.21.15.2:0/2874327049 >> 172.21.15.2:0/2874327049 conn(0x55a43ab03700 msgr2=0x55a43ab03b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.694+0000 7f91cfe94740 1 -- 172.21.15.2:0/2874327049 shutdown_connections 2024-09-10T08:20:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.694+0000 7f91c6800640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f91ac0a2cb0 0x7f91ac0a5170 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009fd0 tx=0x7f91c0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.694+0000 7f91cfe94740 1 -- 172.21.15.2:0/2874327049 wait complete. 2024-09-10T08:20:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.694+0000 7f91cfe94740 1 -- start start 2024-09-10T08:20:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.694+0000 7f91cfe94740 1 Processor -- start 2024-09-10T08:20:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.695+0000 7f91cfe94740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab08090 0x55a43ab121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.695+0000 7f91cfe94740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab089d0 0x55a43ab12720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.695+0000 7f91cfe94740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab0da60 0x55a43ab19760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.695+0000 7f91cfe94740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a43ab109d0 con 0x55a43ab089d0 2024-09-10T08:20:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.695+0000 7f91cfe94740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a43a9f6090 con 0x55a43ab0da60 2024-09-10T08:20:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.695+0000 7f91cfe94740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a43ab10b50 con 0x55a43ab08090 2024-09-10T08:20:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.695+0000 7f91cfe94740 10 client.0 _ll_register_callbacks cb 0x55a43aa4af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.695+0000 7f91c5400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab089d0 0x55a43ab12720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.695+0000 7f91c5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab08090 0x55a43ab121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.695+0000 7f91c5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab08090 0x55a43ab121e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:58730/0 (socket says 172.21.15.2:58730) 2024-09-10T08:20:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.695+0000 7f91c5e00640 1 -- 172.21.15.2:0/3622221009 learned_addr learned my addr 172.21.15.2:0/3622221009 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.695+0000 7f91c6800640 1 --2- 172.21.15.2:0/3622221009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab0da60 0x55a43ab19760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.696+0000 7f91c5e00640 1 -- 172.21.15.2:0/3622221009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab0da60 msgr2=0x55a43ab19760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.696+0000 7f91c5e00640 1 --2- 172.21.15.2:0/3622221009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab0da60 0x55a43ab19760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.696+0000 7f91c5e00640 1 -- 172.21.15.2:0/3622221009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab089d0 msgr2=0x55a43ab12720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.696+0000 7f91c5e00640 1 --2- 172.21.15.2:0/3622221009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab089d0 0x55a43ab12720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.696+0000 7f91c5e00640 1 -- 172.21.15.2:0/3622221009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a43a84c620 con 0x55a43ab08090 2024-09-10T08:20:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.696+0000 7f91c5400640 1 --2- 172.21.15.2:0/3622221009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab089d0 0x55a43ab12720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.696+0000 7f91c5e00640 1 --2- 172.21.15.2:0/3622221009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab08090 0x55a43ab121e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f91b8031440 tx=0x7f91b8004b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=915902f3903fd585 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.696+0000 7f91a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:20:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.696+0000 7f91a7e00640 1 -- 172.21.15.2:0/3622221009 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8052060 con 0x55a43ab08090 2024-09-10T08:20:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.696+0000 7f91a7e00640 1 -- 172.21.15.2:0/3622221009 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f91b8005000 con 0x55a43ab08090 2024-09-10T08:20:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.696+0000 7f91cfe94740 1 -- 172.21.15.2:0/3622221009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91b8004750 con 0x55a43ab08090 2024-09-10T08:20:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.696+0000 7f91a7e00640 1 -- 172.21.15.2:0/3622221009 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f91b8035070 con 0x55a43ab08090 2024-09-10T08:20:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.697+0000 7f91cfe94740 10 client.28870 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.697+0000 7f91cfe94740 1 -- 172.21.15.2:0/3622221009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a43ab109d0 con 0x55a43ab08090 2024-09-10T08:20:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.697+0000 7f91a7e00640 1 -- 172.21.15.2:0/3622221009 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b8004750 con 0x55a43ab08090 2024-09-10T08:20:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.697+0000 7f91a7e00640 1 -- 172.21.15.2:0/3622221009 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f91b8003680 con 0x55a43ab08090 2024-09-10T08:20:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.697+0000 7f91cfe94740 10 client.28870 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.697+0000 7f91cfe94740 10 client.28870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.697+0000 7f91cfe94740 20 client.28870 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.697+0000 7f91cfe94740 1 --2- 172.21.15.2:0/3622221009 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a43ab1d7c0 0x55a43ab1fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.697+0000 7f91cfe94740 4 client.28870 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.697+0000 7f91cfe94740 1 -- 172.21.15.2:0/3622221009 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a43aa7a830 con 0x55a43ab1d7c0 2024-09-10T08:20:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.698+0000 7f91c5400640 1 --2- 172.21.15.2:0/3622221009 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a43ab1d7c0 0x55a43ab1fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.698+0000 7f91c5400640 1 --2- 172.21.15.2:0/3622221009 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a43ab1d7c0 0x55a43ab1fbb0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a77f0a1a6cee20b server_cookie=19bbc1cc847206a9 in_seq=0 out_seq=0 2024-09-10T08:20:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.698+0000 7f91a7e00640 10 client.28870 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.699+0000 7f91a7e00640 1 -- 172.21.15.2:0/3622221009 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a43aa7a830 con 0x55a43ab1d7c0 2024-09-10T08:20:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.699+0000 7f91a7e00640 10 client.28870 handle_command_reply: tid=0 2024-09-10T08:20:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.699+0000 7f91cfe94740 1 client.28870 shutdown 2024-09-10T08:20:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.699+0000 7f91cfe94740 1 -- 172.21.15.2:0/3622221009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab08090 msgr2=0x55a43ab121e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.699+0000 7f91cfe94740 1 --2- 172.21.15.2:0/3622221009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab08090 0x55a43ab121e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f91b8031440 tx=0x7f91b8004b30 comp rx=0 tx=0).stop 2024-09-10T08:20:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.699+0000 7f91cfe94740 1 -- 172.21.15.2:0/3622221009 shutdown_connections 2024-09-10T08:20:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.699+0000 7f91cfe94740 1 --2- 172.21.15.2:0/3622221009 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a43ab1d7c0 0x55a43ab1fbb0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.699+0000 7f91cfe94740 1 --2- 172.21.15.2:0/3622221009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab0da60 0x55a43ab19760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.699+0000 7f91cfe94740 1 --2- 172.21.15.2:0/3622221009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab089d0 0x55a43ab12720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.699+0000 7f91cfe94740 1 --2- 172.21.15.2:0/3622221009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab08090 0x55a43ab121e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.699+0000 7f91a7e00640 0 client.28870 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.699+0000 7f91cfe94740 1 -- 172.21.15.2:0/3622221009 >> 172.21.15.2:0/3622221009 conn(0x55a43ab03700 msgr2=0x55a43ab04f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.699+0000 7f91cfe94740 1 -- 172.21.15.2:0/3622221009 shutdown_connections 2024-09-10T08:20:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.700+0000 7f91cfe94740 1 -- 172.21.15.2:0/3622221009 wait complete. 2024-09-10T08:20:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.700+0000 7f91cfe94740 20 client.28870 trim_cache size 0 max 16384 2024-09-10T08:20:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.715+0000 7f91cfe94740 1 Processor -- start 2024-09-10T08:20:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.715+0000 7f91cfe94740 1 -- start start 2024-09-10T08:20:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.715+0000 7f91cfe94740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 0x55a43ab04af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.715+0000 7f91cfe94740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab089d0 0x55a43ab19720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.715+0000 7f91cfe94740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab0da60 0x55a43ab05030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.715+0000 7f91cfe94740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81af2f0 con 0x55a43ab08090 2024-09-10T08:20:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.715+0000 7f91cfe94740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81acbc0 con 0x55a43ab089d0 2024-09-10T08:20:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.715+0000 7f91cfe94740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81aaa10 con 0x55a43ab0da60 2024-09-10T08:20:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.715+0000 7f91c5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 0x55a43ab04af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.716+0000 7f91c5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 0x55a43ab04af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37170/0 (socket says 172.21.15.2:37170) 2024-09-10T08:20:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.716+0000 7f91c5e00640 1 -- 172.21.15.2:0/500328308 learned_addr learned my addr 172.21.15.2:0/500328308 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.716+0000 7f91c5400640 1 --2- 172.21.15.2:0/500328308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab089d0 0x55a43ab19720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.716+0000 7f91c6800640 1 --2- 172.21.15.2:0/500328308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab0da60 0x55a43ab05030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.716+0000 7f91c5e00640 1 -- 172.21.15.2:0/500328308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab0da60 msgr2=0x55a43ab05030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.716+0000 7f91c5e00640 1 --2- 172.21.15.2:0/500328308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab0da60 0x55a43ab05030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.716+0000 7f91c5e00640 1 -- 172.21.15.2:0/500328308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab089d0 msgr2=0x55a43ab19720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.716+0000 7f91c5e00640 1 --2- 172.21.15.2:0/500328308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab089d0 0x55a43ab19720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.716+0000 7f91c5e00640 1 -- 172.21.15.2:0/500328308 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43ab03520 con 0x55a43ab08090 2024-09-10T08:20:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.716+0000 7f91c5e00640 1 --2- 172.21.15.2:0/500328308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 0x55a43ab04af0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f91b80092a0 tx=0x7f91b8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0a82a3b1e619fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.716+0000 7f91b2a00640 1 -- 172.21.15.2:0/500328308 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8052030 con 0x55a43ab08090 2024-09-10T08:20:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.716+0000 7f91b2a00640 1 -- 172.21.15.2:0/500328308 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b8003c90 con 0x55a43ab08090 2024-09-10T08:20:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.716+0000 7f91b2a00640 1 -- 172.21.15.2:0/500328308 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b80316b0 con 0x55a43ab08090 2024-09-10T08:20:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.717+0000 7f91cfe94740 1 -- 172.21.15.2:0/500328308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 msgr2=0x55a43ab04af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.717+0000 7f91cfe94740 1 --2- 172.21.15.2:0/500328308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 0x55a43ab04af0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f91b80092a0 tx=0x7f91b8000bc0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.717+0000 7f91cfe94740 1 -- 172.21.15.2:0/500328308 shutdown_connections 2024-09-10T08:20:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.717+0000 7f91cfe94740 1 --2- 172.21.15.2:0/500328308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab0da60 0x55a43ab05030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.717+0000 7f91cfe94740 1 --2- 172.21.15.2:0/500328308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab089d0 0x55a43ab19720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.717+0000 7f91cfe94740 1 --2- 172.21.15.2:0/500328308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 0x55a43ab04af0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.717+0000 7f91cfe94740 1 -- 172.21.15.2:0/500328308 >> 172.21.15.2:0/500328308 conn(0x55a43ab03700 msgr2=0x55a43ab02880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.717+0000 7f91cfe94740 1 -- 172.21.15.2:0/500328308 shutdown_connections 2024-09-10T08:20:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.717+0000 7f91cfe94740 1 -- 172.21.15.2:0/500328308 wait complete. 2024-09-10T08:20:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.718+0000 7f91cfe94740 1 -- start start 2024-09-10T08:20:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.718+0000 7f91cfe94740 1 Processor -- start 2024-09-10T08:20:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.718+0000 7f91cfe94740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 0x55a43ab16a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.718+0000 7f91cfe94740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab089d0 0x55a43ab16fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.718+0000 7f91cfe94740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab0da60 0x55a43ab17fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.718+0000 7f91cfe94740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81ab080 con 0x55a43ab08090 2024-09-10T08:20:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.718+0000 7f91cfe94740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81ac4f0 con 0x55a43ab089d0 2024-09-10T08:20:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.718+0000 7f91cfe94740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f91c81ab750 con 0x55a43ab0da60 2024-09-10T08:20:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.718+0000 7f91cfe94740 10 client.0 _ll_register_callbacks cb 0x7f91b40d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.718+0000 7f91c5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 0x55a43ab16a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.718+0000 7f91c5e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 0x55a43ab16a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37176/0 (socket says 172.21.15.2:37176) 2024-09-10T08:20:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.718+0000 7f91c5e00640 1 -- 172.21.15.2:0/3321656059 learned_addr learned my addr 172.21.15.2:0/3321656059 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.719+0000 7f91c5400640 1 --2- 172.21.15.2:0/3321656059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab089d0 0x55a43ab16fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.719+0000 7f91c6800640 1 --2- 172.21.15.2:0/3321656059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab0da60 0x55a43ab17fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.719+0000 7f91c5e00640 1 -- 172.21.15.2:0/3321656059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab0da60 msgr2=0x55a43ab17fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.719+0000 7f91c5e00640 1 --2- 172.21.15.2:0/3321656059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab0da60 0x55a43ab17fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.719+0000 7f91c5e00640 1 -- 172.21.15.2:0/3321656059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab089d0 msgr2=0x55a43ab16fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.719+0000 7f91c5e00640 1 --2- 172.21.15.2:0/3321656059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab089d0 0x55a43ab16fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.719+0000 7f91c5e00640 1 -- 172.21.15.2:0/3321656059 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a43ab185d0 con 0x55a43ab08090 2024-09-10T08:20:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.719+0000 7f91c5e00640 1 --2- 172.21.15.2:0/3321656059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 0x55a43ab16a70 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f91b8003a20 tx=0x7f91b8003940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57584fec706e800e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.719+0000 7f91a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.719+0000 7f91a7e00640 1 -- 172.21.15.2:0/3321656059 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b806b060 con 0x55a43ab08090 2024-09-10T08:20:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.719+0000 7f91a7e00640 1 -- 172.21.15.2:0/3321656059 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f91b8003e40 con 0x55a43ab08090 2024-09-10T08:20:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.719+0000 7f91cfe94740 1 -- 172.21.15.2:0/3321656059 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91b8003c90 con 0x55a43ab08090 2024-09-10T08:20:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.720+0000 7f91a7e00640 1 -- 172.21.15.2:0/3321656059 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f91b8052030 con 0x55a43ab08090 2024-09-10T08:20:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.720+0000 7f91cfe94740 10 client.18918 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.720+0000 7f91cfe94740 1 -- 172.21.15.2:0/3321656059 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a43ab198e0 con 0x55a43ab08090 2024-09-10T08:20:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.720+0000 7f91a7e00640 1 -- 172.21.15.2:0/3321656059 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b8003c90 con 0x55a43ab08090 2024-09-10T08:20:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.720+0000 7f91a7e00640 1 -- 172.21.15.2:0/3321656059 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f91b8066510 con 0x55a43ab08090 2024-09-10T08:20:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.720+0000 7f91cfe94740 10 client.18918 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.720+0000 7f91cfe94740 10 client.18918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.720+0000 7f91cfe94740 20 client.18918 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.720+0000 7f91cfe94740 1 --2- 172.21.15.2:0/3321656059 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a43ab887e0 0x55a43ab8abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.720+0000 7f91cfe94740 4 client.18918 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:20:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.720+0000 7f91cfe94740 1 -- 172.21.15.2:0/3321656059 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a43aa74b80 con 0x55a43ab887e0 2024-09-10T08:20:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.721+0000 7f91c5400640 1 --2- 172.21.15.2:0/3321656059 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a43ab887e0 0x55a43ab8abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.721+0000 7f91c5400640 1 --2- 172.21.15.2:0/3321656059 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a43ab887e0 0x55a43ab8abd0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee28ee7fca1ce3c server_cookie=7359c14a45dcbd9a in_seq=0 out_seq=0 2024-09-10T08:20:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.721+0000 7f91a7e00640 10 client.18918 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.734+0000 7f91a7e00640 1 -- 172.21.15.2:0/3321656059 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a43aa74b80 con 0x55a43ab887e0 2024-09-10T08:20:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.734+0000 7f91a7e00640 10 client.18918 handle_command_reply: tid=0 2024-09-10T08:20:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.734+0000 7f91cfe94740 1 client.18918 shutdown 2024-09-10T08:20:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.735+0000 7f91cfe94740 1 -- 172.21.15.2:0/3321656059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 msgr2=0x55a43ab16a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.735+0000 7f91cfe94740 1 --2- 172.21.15.2:0/3321656059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 0x55a43ab16a70 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f91b8003a20 tx=0x7f91b8003940 comp rx=0 tx=0).stop 2024-09-10T08:20:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.735+0000 7f91cfe94740 1 -- 172.21.15.2:0/3321656059 shutdown_connections 2024-09-10T08:20:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.735+0000 7f91cfe94740 1 --2- 172.21.15.2:0/3321656059 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a43ab887e0 0x55a43ab8abd0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.735+0000 7f91cfe94740 1 --2- 172.21.15.2:0/3321656059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a43ab0da60 0x55a43ab17fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.735+0000 7f91cfe94740 1 --2- 172.21.15.2:0/3321656059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a43ab089d0 0x55a43ab16fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.735+0000 7f91cfe94740 1 --2- 172.21.15.2:0/3321656059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a43ab08090 0x55a43ab16a70 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.735+0000 7f91a7e00640 0 client.18918 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.735+0000 7f91cfe94740 1 -- 172.21.15.2:0/3321656059 >> 172.21.15.2:0/3321656059 conn(0x55a43ab03700 msgr2=0x55a43ab03ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.735+0000 7f91cfe94740 1 -- 172.21.15.2:0/3321656059 shutdown_connections 2024-09-10T08:20:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.735+0000 7f91cfe94740 1 -- 172.21.15.2:0/3321656059 wait complete. 2024-09-10T08:20:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.735+0000 7f91cfe94740 20 client.18918 trim_cache size 0 max 16384 2024-09-10T08:20:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:20:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:20:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:20:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:20:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.736+0000 7f91cd000640 1 -- 172.21.15.2:0/3595441161 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f91ac0a2cb0 msgr2=0x7f91ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.736+0000 7f91cd000640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f91ac0a2cb0 0x7f91ac0a5170 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009fd0 tx=0x7f91c0009500 comp rx=0 tx=0).stop 2024-09-10T08:20:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.736+0000 7f91cd000640 1 -- 172.21.15.2:0/3595441161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91c8104f50 msgr2=0x7f91c819a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.736+0000 7f91cd000640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91c8104f50 0x7f91c819a510 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f91b400e9f0 tx=0x7f91b400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.736+0000 7f91cd000640 1 -- 172.21.15.2:0/3595441161 shutdown_connections 2024-09-10T08:20:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.736+0000 7f91cd000640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f91ac0a2cb0 0x7f91ac0a5170 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.736+0000 7f91cd000640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91c8105910 0x7f91c81a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.736+0000 7f91cd000640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91c8104f50 0x7f91c819a510 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.736+0000 7f91cd000640 1 --2- 172.21.15.2:0/3595441161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c8103d60 0x7f91c8199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.736+0000 7f91cd000640 1 -- 172.21.15.2:0/3595441161 >> 172.21.15.2:0/3595441161 conn(0x7f91c80ff4e0 msgr2=0x7f91c81082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.736+0000 7f91cd000640 1 -- 172.21.15.2:0/3595441161 shutdown_connections 2024-09-10T08:20:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:34.736+0000 7f91cd000640 1 -- 172.21.15.2:0/3595441161 wait complete. 2024-09-10T08:20:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:35 smithi155 ceph-mon[35405]: pgmap v760: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:20:35.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:35 smithi107 ceph-mon[35048]: pgmap v760: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:20:36.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:35 smithi002 ceph-mon[30563]: pgmap v760: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:20:37.908 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:37 smithi155 ceph-mon[35405]: pgmap v761: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-09-10T08:20:37.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:37 smithi107 ceph-mon[35048]: pgmap v761: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-09-10T08:20:38.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:37 smithi002 ceph-mon[30563]: pgmap v761: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-09-10T08:20:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:39 smithi155 ceph-mon[35405]: pgmap v762: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.4 KiB/s wr, 11 op/s 2024-09-10T08:20:39.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:39 smithi107 ceph-mon[35048]: pgmap v762: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.4 KiB/s wr, 11 op/s 2024-09-10T08:20:40.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:39 smithi002 ceph-mon[30563]: pgmap v762: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.4 KiB/s wr, 11 op/s 2024-09-10T08:20:40.167 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:20:40.558 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:41.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.137+0000 7f4b2de00640 1 -- 172.21.15.2:0/2398000364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b28112a50 msgr2=0x7f4b28114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:41.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.137+0000 7f4b2de00640 1 --2- 172.21.15.2:0/2398000364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b28112a50 0x7f4b28114e40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009ca0 tx=0x7f4b2002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:20:41.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.138+0000 7f4b2de00640 1 -- 172.21.15.2:0/2398000364 shutdown_connections 2024-09-10T08:20:41.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.138+0000 7f4b2de00640 1 --2- 172.21.15.2:0/2398000364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b28112a50 0x7f4b28114e40 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:41.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.138+0000 7f4b2de00640 1 --2- 172.21.15.2:0/2398000364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b28077040 0x7f4b28112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:41.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.138+0000 7f4b2de00640 1 --2- 172.21.15.2:0/2398000364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b28076720 0x7f4b28076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:41.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.138+0000 7f4b2de00640 1 -- 172.21.15.2:0/2398000364 >> 172.21.15.2:0/2398000364 conn(0x7f4b280ffa50 msgr2=0x7f4b28101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:41.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.138+0000 7f4b2de00640 1 -- 172.21.15.2:0/2398000364 shutdown_connections 2024-09-10T08:20:41.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.138+0000 7f4b2de00640 1 -- 172.21.15.2:0/2398000364 wait complete. 2024-09-10T08:20:41.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.139+0000 7f4b2de00640 1 Processor -- start 2024-09-10T08:20:41.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.139+0000 7f4b2de00640 1 -- start start 2024-09-10T08:20:41.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.139+0000 7f4b2de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b28076720 0x7f4b2819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:41.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.139+0000 7f4b2de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b28077040 0x7f4b2819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:41.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.139+0000 7f4b2de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b28112a50 0x7f4b281a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:41.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.139+0000 7f4b2de00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4b281185e0 con 0x7f4b28112a50 2024-09-10T08:20:41.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.139+0000 7f4b2de00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28118460 con 0x7f4b28076720 2024-09-10T08:20:41.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.139+0000 7f4b2de00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28118760 con 0x7f4b28077040 2024-09-10T08:20:41.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.140+0000 7f4b27400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b28112a50 0x7f4b281a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:41.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.140+0000 7f4b26a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b28076720 0x7f4b2819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:41.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.140+0000 7f4b26a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b28076720 0x7f4b2819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40560/0 (socket says 172.21.15.2:40560) 2024-09-10T08:20:41.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.140+0000 7f4b26a00640 1 -- 172.21.15.2:0/3000487766 learned_addr learned my addr 172.21.15.2:0/3000487766 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:41.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.140+0000 7f4b1fe00640 1 --2- 172.21.15.2:0/3000487766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b28077040 0x7f4b2819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:41.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.140+0000 7f4b26a00640 1 -- 172.21.15.2:0/3000487766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b28077040 msgr2=0x7f4b2819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:41.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.140+0000 7f4b26a00640 1 --2- 172.21.15.2:0/3000487766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b28077040 0x7f4b2819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:41.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.140+0000 7f4b26a00640 1 -- 172.21.15.2:0/3000487766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b28112a50 msgr2=0x7f4b281a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:41.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.140+0000 7f4b26a00640 1 --2- 172.21.15.2:0/3000487766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b28112a50 0x7f4b281a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:41.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.140+0000 7f4b26a00640 1 -- 172.21.15.2:0/3000487766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b281a3bb0 con 0x7f4b28076720 2024-09-10T08:20:41.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.140+0000 7f4b26a00640 1 --2- 172.21.15.2:0/3000487766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b28076720 0x7f4b2819eb80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800e9f0 tx=0x7f4b1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:41.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.141+0000 7f4b2ca00640 1 -- 172.21.15.2:0/3000487766 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1800cd80 con 0x7f4b28076720 2024-09-10T08:20:41.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.141+0000 7f4b2de00640 1 -- 172.21.15.2:0/3000487766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b281a3ea0 con 0x7f4b28076720 2024-09-10T08:20:41.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.141+0000 7f4b2ca00640 1 -- 172.21.15.2:0/3000487766 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b18010910 con 0x7f4b28076720 2024-09-10T08:20:41.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.141+0000 7f4b2ca00640 1 -- 172.21.15.2:0/3000487766 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b180188f0 con 0x7f4b28076720 2024-09-10T08:20:41.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.141+0000 7f4b2de00640 1 -- 172.21.15.2:0/3000487766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b281ab730 con 0x7f4b28076720 2024-09-10T08:20:41.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.142+0000 7f4b2de00640 1 -- 172.21.15.2:0/3000487766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b28076b00 con 0x7f4b28076720 2024-09-10T08:20:41.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.144+0000 7f4b2ca00640 1 -- 172.21.15.2:0/3000487766 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4b18010ac0 con 0x7f4b28076720 2024-09-10T08:20:41.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.145+0000 7f4b2ca00640 1 --2- 172.21.15.2:0/3000487766 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b0c0a2c00 0x7f4b0c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:41.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.145+0000 7f4b2ca00640 1 -- 172.21.15.2:0/3000487766 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4b180cfd00 con 0x7f4b28076720 2024-09-10T08:20:41.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.146+0000 7f4b1fe00640 1 --2- 172.21.15.2:0/3000487766 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b0c0a2c00 0x7f4b0c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:41.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.146+0000 7f4b1fe00640 1 --2- 172.21.15.2:0/3000487766 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b0c0a2c00 0x7f4b0c0a50c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4b28079360 tx=0x7f4b10005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:41.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.147+0000 7f4b2ca00640 1 -- 172.21.15.2:0/3000487766 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b1809c1c0 con 0x7f4b28076720 2024-09-10T08:20:41.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.373+0000 7f4b2de00640 1 -- 172.21.15.2:0/3000487766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b2810dc40 con 0x7f4b28076720 2024-09-10T08:20:41.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.374+0000 7f4b2ca00640 1 -- 172.21.15.2:0/3000487766 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f4b180a04c0 con 0x7f4b28076720 2024-09-10T08:20:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:20:41.374 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:20:41.375 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:20:41.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.377+0000 7f4b2de00640 1 -- 172.21.15.2:0/3000487766 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b0c0a2c00 msgr2=0x7f4b0c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:41.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.377+0000 7f4b2de00640 1 --2- 172.21.15.2:0/3000487766 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b0c0a2c00 0x7f4b0c0a50c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4b28079360 tx=0x7f4b10005f90 comp rx=0 tx=0).stop 2024-09-10T08:20:41.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.377+0000 7f4b2de00640 1 -- 172.21.15.2:0/3000487766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b28076720 msgr2=0x7f4b2819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:41.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.377+0000 7f4b2de00640 1 --2- 172.21.15.2:0/3000487766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b28076720 0x7f4b2819eb80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800e9f0 tx=0x7f4b1800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:20:41.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.378+0000 7f4b2de00640 1 -- 172.21.15.2:0/3000487766 shutdown_connections 2024-09-10T08:20:41.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.378+0000 7f4b2de00640 1 --2- 172.21.15.2:0/3000487766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b28112a50 0x7f4b281a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:41.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.378+0000 7f4b2de00640 1 --2- 172.21.15.2:0/3000487766 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4b0c0a2c00 0x7f4b0c0a50c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:41.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.378+0000 7f4b2de00640 1 --2- 172.21.15.2:0/3000487766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b28077040 0x7f4b2819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:41.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.378+0000 7f4b2de00640 1 --2- 172.21.15.2:0/3000487766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b28076720 0x7f4b2819eb80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:41.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.378+0000 7f4b2de00640 1 -- 172.21.15.2:0/3000487766 >> 172.21.15.2:0/3000487766 conn(0x7f4b280ffa50 msgr2=0x7f4b28101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:41.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.378+0000 7f4b2de00640 1 -- 172.21.15.2:0/3000487766 shutdown_connections 2024-09-10T08:20:41.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:41.378+0000 7f4b2de00640 1 -- 172.21.15.2:0/3000487766 wait complete. 2024-09-10T08:20:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:41 smithi155 ceph-mon[35405]: pgmap v763: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-10T08:20:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:41 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3000487766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:20:41.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:41 smithi107 ceph-mon[35048]: pgmap v763: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-10T08:20:41.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:41 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3000487766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:20:41.947 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:41 smithi002 ceph-mon[30563]: pgmap v763: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-10T08:20:41.947 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:41 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3000487766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:20:42.377 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id 41453dd0 --release --await' 2024-09-10T08:20:42.377 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id 41453dd0 --release --await 2024-09-10T08:20:42.767 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:43.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.398+0000 7fb2af400640 1 -- 172.21.15.2:0/3164739198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a8103ff0 msgr2=0x7fb2a81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:43.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.398+0000 7fb2af400640 1 --2- 172.21.15.2:0/3164739198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a8103ff0 0x7fb2a81043d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009860 tx=0x7fb29c02a340 comp rx=0 tx=0).stop 2024-09-10T08:20:43.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.398+0000 7fb2af400640 1 -- 172.21.15.2:0/3164739198 shutdown_connections 2024-09-10T08:20:43.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.398+0000 7fb2af400640 1 --2- 172.21.15.2:0/3164739198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a8109120 0x7fb2a8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.398+0000 7fb2af400640 1 --2- 172.21.15.2:0/3164739198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a81049a0 0x7fb2a81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.398+0000 7fb2af400640 1 --2- 172.21.15.2:0/3164739198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a8103ff0 0x7fb2a81043d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.398+0000 7fb2af400640 1 -- 172.21.15.2:0/3164739198 >> 172.21.15.2:0/3164739198 conn(0x7fb2a80ffa90 msgr2=0x7fb2a8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:43.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.399+0000 7fb2af400640 1 -- 172.21.15.2:0/3164739198 shutdown_connections 2024-09-10T08:20:43.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.399+0000 7fb2af400640 1 -- 172.21.15.2:0/3164739198 wait complete. 2024-09-10T08:20:43.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.400+0000 7fb2af400640 1 Processor -- start 2024-09-10T08:20:43.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.400+0000 7fb2af400640 1 -- start start 2024-09-10T08:20:43.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.400+0000 7fb2af400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a8103ff0 0x7fb2a819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.400+0000 7fb2af400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a81049a0 0x7fb2a819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.400+0000 7fb2af400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a8109120 0x7fb2a819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.400+0000 7fb2af400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a8114150 con 0x7fb2a81049a0 2024-09-10T08:20:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.400+0000 7fb2af400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a8113fd0 con 0x7fb2a8103ff0 2024-09-10T08:20:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.400+0000 7fb2af400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a81142d0 con 0x7fb2a8109120 2024-09-10T08:20:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.401+0000 7fb2aca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a8103ff0 0x7fb2a819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.401+0000 7fb2aca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a8103ff0 0x7fb2a819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40568/0 (socket says 172.21.15.2:40568) 2024-09-10T08:20:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.401+0000 7fb2aca00640 1 -- 172.21.15.2:0/1743730607 learned_addr learned my addr 172.21.15.2:0/1743730607 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.401+0000 7fb2a7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a81049a0 0x7fb2a819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.401+0000 7fb2ad400640 1 --2- 172.21.15.2:0/1743730607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a8109120 0x7fb2a819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.401+0000 7fb2aca00640 1 -- 172.21.15.2:0/1743730607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a8109120 msgr2=0x7fb2a819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.401+0000 7fb2aca00640 1 --2- 172.21.15.2:0/1743730607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a8109120 0x7fb2a819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.401+0000 7fb2aca00640 1 -- 172.21.15.2:0/1743730607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a81049a0 msgr2=0x7fb2a819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:43.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.401+0000 7fb2aca00640 1 --2- 172.21.15.2:0/1743730607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a81049a0 0x7fb2a819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.401+0000 7fb2aca00640 1 -- 172.21.15.2:0/1743730607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a819f5e0 con 0x7fb2a8103ff0 2024-09-10T08:20:43.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.402+0000 7fb2aca00640 1 --2- 172.21.15.2:0/1743730607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a8103ff0 0x7fb2a819a630 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb29c035040 tx=0x7fb29c033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:43.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.402+0000 7fb2a5600640 1 -- 172.21.15.2:0/1743730607 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c037040 con 0x7fb2a8103ff0 2024-09-10T08:20:43.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.402+0000 7fb2a5600640 1 -- 172.21.15.2:0/1743730607 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29c040070 con 0x7fb2a8103ff0 2024-09-10T08:20:43.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.402+0000 7fb2af400640 1 -- 172.21.15.2:0/1743730607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a819f870 con 0x7fb2a8103ff0 2024-09-10T08:20:43.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.402+0000 7fb2a5600640 1 -- 172.21.15.2:0/1743730607 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c0039b0 con 0x7fb2a8103ff0 2024-09-10T08:20:43.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.402+0000 7fb2af400640 1 -- 172.21.15.2:0/1743730607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a81a71b0 con 0x7fb2a8103ff0 2024-09-10T08:20:43.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.405+0000 7fb2a5600640 1 -- 172.21.15.2:0/1743730607 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb29c0305e0 con 0x7fb2a8103ff0 2024-09-10T08:20:43.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.405+0000 7fb2b1e21740 1 Processor -- start 2024-09-10T08:20:43.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.406+0000 7fb2b1e21740 1 -- start start 2024-09-10T08:20:43.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.406+0000 7fb2b1e21740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556829990310 0x5568299906f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.406+0000 7fb2b1e21740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x55682999afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.406+0000 7fb2b1e21740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55682999b4f0 0x55682999d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.406+0000 7fb2b1e21740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5568298f6650 con 0x556829990310 2024-09-10T08:20:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.406+0000 7fb2b1e21740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55682989df10 con 0x556829990c30 2024-09-10T08:20:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.406+0000 7fb2b1e21740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5568298f2610 con 0x55682999b4f0 2024-09-10T08:20:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.407+0000 7fb2aca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x55682999afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.407+0000 7fb2ad400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556829990310 0x5568299906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.407+0000 7fb2aca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x55682999afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40574/0 (socket says 172.21.15.2:40574) 2024-09-10T08:20:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.407+0000 7fb2aca00640 1 -- 172.21.15.2:0/2456712810 learned_addr learned my addr 172.21.15.2:0/2456712810 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.407+0000 7fb2a7e00640 1 --2- 172.21.15.2:0/2456712810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55682999b4f0 0x55682999d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.407+0000 7fb2a7e00640 1 -- 172.21.15.2:0/2456712810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 msgr2=0x55682999afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:43.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.407+0000 7fb2a7e00640 1 --2- 172.21.15.2:0/2456712810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x55682999afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.407+0000 7fb2a7e00640 1 -- 172.21.15.2:0/2456712810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556829990310 msgr2=0x5568299906f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:43.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.407+0000 7fb2a7e00640 1 --2- 172.21.15.2:0/2456712810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556829990310 0x5568299906f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.407+0000 7fb2a7e00640 1 -- 172.21.15.2:0/2456712810 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556829605580 con 0x55682999b4f0 2024-09-10T08:20:43.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.408+0000 7fb2a5600640 1 --2- 172.21.15.2:0/1743730607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb2900a2b80 0x7fb2900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.408+0000 7fb2a7e00640 1 --2- 172.21.15.2:0/2456712810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55682999b4f0 0x55682999d8e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb294005fd0 tx=0x7fb294011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b7f156c7ab4e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:43.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.408+0000 7fb2a5600640 1 -- 172.21.15.2:0/1743730607 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb29c0ef4a0 con 0x7fb2a8103ff0 2024-09-10T08:20:43.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.408+0000 7fb299c00640 1 -- 172.21.15.2:0/2456712810 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29401e070 con 0x55682999b4f0 2024-09-10T08:20:43.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.408+0000 7fb2ad400640 1 --2- 172.21.15.2:0/1743730607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb2900a2b80 0x7fb2900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.408+0000 7fb299c00640 1 -- 172.21.15.2:0/2456712810 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2940035f0 con 0x55682999b4f0 2024-09-10T08:20:43.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.408+0000 7fb299c00640 1 -- 172.21.15.2:0/2456712810 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294019670 con 0x55682999b4f0 2024-09-10T08:20:43.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.409+0000 7fb2ad400640 1 --2- 172.21.15.2:0/1743730607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb2900a2b80 0x7fb2900a5040 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0005fd0 tx=0x7fb2a0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:43.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.409+0000 7fb2b1e21740 1 -- 172.21.15.2:0/2456712810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55682999b4f0 msgr2=0x55682999d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:43.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.409+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/2456712810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55682999b4f0 0x55682999d8e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb294005fd0 tx=0x7fb294011810 comp rx=0 tx=0).stop 2024-09-10T08:20:43.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.409+0000 7fb2b1e21740 1 -- 172.21.15.2:0/2456712810 shutdown_connections 2024-09-10T08:20:43.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.409+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/2456712810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55682999b4f0 0x55682999d8e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.409+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/2456712810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x55682999afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.409+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/2456712810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556829990310 0x5568299906f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.409+0000 7fb2b1e21740 1 -- 172.21.15.2:0/2456712810 >> 172.21.15.2:0/2456712810 conn(0x55682998c8a0 msgr2=0x55682998cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:43.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.410+0000 7fb2b1e21740 1 -- 172.21.15.2:0/2456712810 shutdown_connections 2024-09-10T08:20:43.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.410+0000 7fb2b1e21740 1 -- 172.21.15.2:0/2456712810 wait complete. 2024-09-10T08:20:43.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.410+0000 7fb2b1e21740 1 -- start start 2024-09-10T08:20:43.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.410+0000 7fb2b1e21740 1 Processor -- start 2024-09-10T08:20:43.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.411+0000 7fb2b1e21740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990310 0x556829998390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.411+0000 7fb2b1e21740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x5568299988d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.411+0000 7fb2b1e21740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55682999b4f0 0x556829998e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.411+0000 7fb2b1e21740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55682999e0c0 con 0x55682999b4f0 2024-09-10T08:20:43.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.411+0000 7fb2b1e21740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5568298b1350 con 0x556829990c30 2024-09-10T08:20:43.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.411+0000 7fb2b1e21740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55682999e240 con 0x556829990310 2024-09-10T08:20:43.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.411+0000 7fb2b1e21740 10 client.0 _ll_register_callbacks cb 0x5568298d4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:43.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.411+0000 7fb2aca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990310 0x556829998390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.411+0000 7fb2ad400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55682999b4f0 0x556829998e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.411+0000 7fb2aca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990310 0x556829998390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:47958/0 (socket says 172.21.15.2:47958) 2024-09-10T08:20:43.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.411+0000 7fb2aca00640 1 -- 172.21.15.2:0/62399227 learned_addr learned my addr 172.21.15.2:0/62399227 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:43.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.411+0000 7fb2a7e00640 1 --2- 172.21.15.2:0/62399227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x5568299988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.412+0000 7fb2aca00640 1 -- 172.21.15.2:0/62399227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 msgr2=0x5568299988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:43.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.412+0000 7fb2aca00640 1 --2- 172.21.15.2:0/62399227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x5568299988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.412+0000 7fb2aca00640 1 -- 172.21.15.2:0/62399227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55682999b4f0 msgr2=0x556829998e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:43.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.412+0000 7fb2aca00640 1 --2- 172.21.15.2:0/62399227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55682999b4f0 0x556829998e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.412+0000 7fb2aca00640 1 -- 172.21.15.2:0/62399227 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568296cf960 con 0x556829990310 2024-09-10T08:20:43.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.412+0000 7fb2aca00640 1 --2- 172.21.15.2:0/62399227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990310 0x556829998390 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb29c004650 tx=0x7fb29c030980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b22ad4e46b553b7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:43.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.412+0000 7fb28e000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:20:43.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.412+0000 7fb28e000640 1 -- 172.21.15.2:0/62399227 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c0bb3f0 con 0x556829990310 2024-09-10T08:20:43.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.412+0000 7fb28e000640 1 -- 172.21.15.2:0/62399227 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb29c0a60e0 con 0x556829990310 2024-09-10T08:20:43.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.412+0000 7fb2b1e21740 1 -- 172.21.15.2:0/62399227 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2940035f0 con 0x556829990310 2024-09-10T08:20:43.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.413+0000 7fb28e000640 1 -- 172.21.15.2:0/62399227 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb29c0bb210 con 0x556829990310 2024-09-10T08:20:43.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.413+0000 7fb2b1e21740 10 client.28894 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:43.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.413+0000 7fb2b1e21740 1 -- 172.21.15.2:0/62399227 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55682999e120 con 0x556829990310 2024-09-10T08:20:43.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.413+0000 7fb28e000640 1 -- 172.21.15.2:0/62399227 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2940035f0 con 0x556829990310 2024-09-10T08:20:43.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.413+0000 7fb28e000640 1 -- 172.21.15.2:0/62399227 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb29c03a070 con 0x556829990310 2024-09-10T08:20:43.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.413+0000 7fb2b1e21740 10 client.28894 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:43.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.413+0000 7fb2b1e21740 10 client.28894 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:20:43.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.413+0000 7fb2b1e21740 20 client.28894 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:43.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.413+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/62399227 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5568299ab0f0 0x5568299ad540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.413+0000 7fb2b1e21740 4 client.28894 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:43.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.413+0000 7fb2b1e21740 1 -- 172.21.15.2:0/62399227 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556829903bc0 con 0x5568299ab0f0 2024-09-10T08:20:43.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.414+0000 7fb2a7e00640 1 --2- 172.21.15.2:0/62399227 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5568299ab0f0 0x5568299ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.414+0000 7fb2a7e00640 1 --2- 172.21.15.2:0/62399227 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5568299ab0f0 0x5568299ad540 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84b254fc1d0bd880 server_cookie=ba286d507fa296e9 in_seq=0 out_seq=0 2024-09-10T08:20:43.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.414+0000 7fb28e000640 10 client.28894 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:43.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.415+0000 7fb28e000640 1 -- 172.21.15.2:0/62399227 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556829903bc0 con 0x5568299ab0f0 2024-09-10T08:20:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.415+0000 7fb28e000640 10 client.28894 handle_command_reply: tid=0 2024-09-10T08:20:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.415+0000 7fb2b1e21740 1 client.28894 shutdown 2024-09-10T08:20:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.415+0000 7fb2b1e21740 1 -- 172.21.15.2:0/62399227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990310 msgr2=0x556829998390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.415+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/62399227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990310 0x556829998390 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb29c004650 tx=0x7fb29c030980 comp rx=0 tx=0).stop 2024-09-10T08:20:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.416+0000 7fb2b1e21740 1 -- 172.21.15.2:0/62399227 shutdown_connections 2024-09-10T08:20:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.416+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/62399227 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5568299ab0f0 0x5568299ad540 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.416+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/62399227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55682999b4f0 0x556829998e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.416+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/62399227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x5568299988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.416+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/62399227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990310 0x556829998390 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.416+0000 7fb28e000640 0 client.28894 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.416+0000 7fb2b1e21740 1 -- 172.21.15.2:0/62399227 >> 172.21.15.2:0/62399227 conn(0x55682998c8a0 msgr2=0x5568299925d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.416+0000 7fb2b1e21740 1 -- 172.21.15.2:0/62399227 shutdown_connections 2024-09-10T08:20:43.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.416+0000 7fb2b1e21740 1 -- 172.21.15.2:0/62399227 wait complete. 2024-09-10T08:20:43.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.416+0000 7fb2b1e21740 20 client.28894 trim_cache size 0 max 16384 2024-09-10T08:20:43.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.434+0000 7fb2b1e21740 1 Processor -- start 2024-09-10T08:20:43.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.434+0000 7fb2b1e21740 1 -- start start 2024-09-10T08:20:43.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.435+0000 7fb2b1e21740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990310 0x5568299972d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.435+0000 7fb2b1e21740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x5568299b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.435+0000 7fb2b1e21740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55682999b4f0 0x5568299b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.435+0000 7fb2b1e21740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a81b3950 con 0x55682999b4f0 2024-09-10T08:20:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.435+0000 7fb2b1e21740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a81b1220 con 0x556829990c30 2024-09-10T08:20:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.435+0000 7fb2b1e21740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a81af070 con 0x556829990310 2024-09-10T08:20:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.435+0000 7fb2ad400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55682999b4f0 0x5568299b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.435+0000 7fb2a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x5568299b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.435+0000 7fb2ad400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55682999b4f0 0x5568299b3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39574/0 (socket says 172.21.15.2:39574) 2024-09-10T08:20:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.435+0000 7fb2aca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990310 0x5568299972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.435+0000 7fb2ad400640 1 -- 172.21.15.2:0/3244515740 learned_addr learned my addr 172.21.15.2:0/3244515740 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.436+0000 7fb2aca00640 1 -- 172.21.15.2:0/3244515740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 msgr2=0x5568299b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.436+0000 7fb2aca00640 1 --2- 172.21.15.2:0/3244515740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x5568299b2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.436+0000 7fb2aca00640 1 -- 172.21.15.2:0/3244515740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55682999b4f0 msgr2=0x5568299b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.436+0000 7fb2aca00640 1 --2- 172.21.15.2:0/3244515740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55682999b4f0 0x5568299b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.436+0000 7fb2aca00640 1 -- 172.21.15.2:0/3244515740 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682998c6d0 con 0x556829990310 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.436+0000 7fb2a7e00640 1 --2- 172.21.15.2:0/3244515740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x5568299b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.436+0000 7fb2aca00640 1 --2- 172.21.15.2:0/3244515740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990310 0x5568299972d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0b8230 tx=0x7fb29c033f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e12fcab60c8f73bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.436+0000 7fb299200640 1 -- 172.21.15.2:0/3244515740 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c0bb3f0 con 0x556829990310 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.436+0000 7fb299200640 1 -- 172.21.15.2:0/3244515740 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29c0bbfb0 con 0x556829990310 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.436+0000 7fb299200640 1 -- 172.21.15.2:0/3244515740 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c0c4090 con 0x556829990310 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.437+0000 7fb2b1e21740 1 -- 172.21.15.2:0/3244515740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990310 msgr2=0x5568299972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.437+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/3244515740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990310 0x5568299972d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0b8230 tx=0x7fb29c033f50 comp rx=0 tx=0).stop 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.437+0000 7fb2b1e21740 1 -- 172.21.15.2:0/3244515740 shutdown_connections 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.437+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/3244515740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55682999b4f0 0x5568299b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.437+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/3244515740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556829990c30 0x5568299b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.437+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/3244515740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990310 0x5568299972d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.437+0000 7fb2b1e21740 1 -- 172.21.15.2:0/3244515740 >> 172.21.15.2:0/3244515740 conn(0x55682998c8a0 msgr2=0x5568299925d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:43.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.437+0000 7fb2b1e21740 1 -- 172.21.15.2:0/3244515740 shutdown_connections 2024-09-10T08:20:43.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.438+0000 7fb2b1e21740 1 -- 172.21.15.2:0/3244515740 wait complete. 2024-09-10T08:20:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.438+0000 7fb2b1e21740 1 -- start start 2024-09-10T08:20:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.438+0000 7fb2b1e21740 1 Processor -- start 2024-09-10T08:20:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.439+0000 7fb2b1e21740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556829990310 0x55682999f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.439+0000 7fb2b1e21740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990c30 0x55682999faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.439+0000 7fb2b1e21740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682999b4f0 0x556829a14db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.439+0000 7fb2b1e21740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a81af6e0 con 0x556829990310 2024-09-10T08:20:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.439+0000 7fb2b1e21740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a81b0b50 con 0x55682999b4f0 2024-09-10T08:20:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.439+0000 7fb2b1e21740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a81afdb0 con 0x556829990c30 2024-09-10T08:20:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.439+0000 7fb2b1e21740 10 client.0 _ll_register_callbacks cb 0x7fb29c009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.439+0000 7fb2aca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556829990310 0x55682999f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.439+0000 7fb2aca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556829990310 0x55682999f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39586/0 (socket says 172.21.15.2:39586) 2024-09-10T08:20:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.439+0000 7fb2aca00640 1 -- 172.21.15.2:0/1145689904 learned_addr learned my addr 172.21.15.2:0/1145689904 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.439+0000 7fb2a7e00640 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990c30 0x55682999faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.439+0000 7fb2ad400640 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682999b4f0 0x556829a14db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.439+0000 7fb2a7e00640 1 -- 172.21.15.2:0/1145689904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682999b4f0 msgr2=0x556829a14db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.440+0000 7fb2a7e00640 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682999b4f0 0x556829a14db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.440+0000 7fb2a7e00640 1 -- 172.21.15.2:0/1145689904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556829990310 msgr2=0x55682999f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.440+0000 7fb2a7e00640 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556829990310 0x55682999f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.440+0000 7fb2a7e00640 1 -- 172.21.15.2:0/1145689904 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568299a03f0 con 0x556829990c30 2024-09-10T08:20:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.440+0000 7fb2aca00640 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556829990310 0x55682999f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.440+0000 7fb2a7e00640 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990c30 0x55682999faf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb29401c040 tx=0x7fb294009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f1940276947592 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.440+0000 7fb28e000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:20:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.440+0000 7fb28e000640 1 -- 172.21.15.2:0/1145689904 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29401e040 con 0x556829990c30 2024-09-10T08:20:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.440+0000 7fb28e000640 1 -- 172.21.15.2:0/1145689904 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb294004250 con 0x556829990c30 2024-09-10T08:20:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.440+0000 7fb2b1e21740 1 -- 172.21.15.2:0/1145689904 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb29c0bbfb0 con 0x556829990c30 2024-09-10T08:20:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.441+0000 7fb28e000640 1 -- 172.21.15.2:0/1145689904 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb294005030 con 0x556829990c30 2024-09-10T08:20:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.441+0000 7fb2b1e21740 10 client.28900 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.441+0000 7fb2b1e21740 1 -- 172.21.15.2:0/1145689904 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556829a164c0 con 0x556829990c30 2024-09-10T08:20:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.441+0000 7fb28e000640 1 -- 172.21.15.2:0/1145689904 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29c0bbfb0 con 0x556829990c30 2024-09-10T08:20:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.441+0000 7fb28e000640 1 -- 172.21.15.2:0/1145689904 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb294003a70 con 0x556829990c30 2024-09-10T08:20:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.441+0000 7fb2b1e21740 10 client.28900 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.441+0000 7fb2b1e21740 10 client.28900 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:20:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.441+0000 7fb2b1e21740 20 client.28900 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.441+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556829a16a90 0x556829a18eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:43.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.441+0000 7fb2b1e21740 4 client.28900 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "41453dd0", "release": true, "await": true}] 2024-09-10T08:20:43.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.441+0000 7fb2b1e21740 1 -- 172.21.15.2:0/1145689904 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "41453dd0", "release": true, "await": true}) -- 0x5568298fe0d0 con 0x556829a16a90 2024-09-10T08:20:43.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.442+0000 7fb2aca00640 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556829a16a90 0x556829a18eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:43.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.442+0000 7fb2aca00640 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556829a16a90 0x556829a18eb0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ad90f5815ae3e5 server_cookie=d9ca65c8869a7558 in_seq=0 out_seq=0 2024-09-10T08:20:43.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.442+0000 7fb28e000640 10 client.28900 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:43.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb28e000640 1 -- 172.21.15.2:0/1145689904 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5568298fe0d0 con 0x556829a16a90 2024-09-10T08:20:43.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb28e000640 10 client.28900 handle_command_reply: tid=0 2024-09-10T08:20:43.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb2b1e21740 1 client.28900 shutdown 2024-09-10T08:20:43.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb2b1e21740 1 -- 172.21.15.2:0/1145689904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990c30 msgr2=0x55682999faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:43.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990c30 0x55682999faf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb29401c040 tx=0x7fb294009790 comp rx=0 tx=0).stop 2024-09-10T08:20:43.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb2b1e21740 1 -- 172.21.15.2:0/1145689904 shutdown_connections 2024-09-10T08:20:43.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x556829a16a90 0x556829a18eb0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55682999b4f0 0x556829a14db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556829990c30 0x55682999faf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb2b1e21740 1 --2- 172.21.15.2:0/1145689904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556829990310 0x55682999f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb2b1e21740 1 -- 172.21.15.2:0/1145689904 >> 172.21.15.2:0/1145689904 conn(0x55682998c8a0 msgr2=0x55682998b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:43.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb28e000640 0 client.28900 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:43.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb2b1e21740 1 -- 172.21.15.2:0/1145689904 shutdown_connections 2024-09-10T08:20:43.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.444+0000 7fb2b1e21740 1 -- 172.21.15.2:0/1145689904 wait complete. 2024-09-10T08:20:43.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.445+0000 7fb2b1e21740 20 client.28900 trim_cache size 0 max 16384 2024-09-10T08:20:43.447 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:20:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:20:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:20:43.447 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 44, 2024-09-10T08:20:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:20:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "41453dd0": { 2024-09-10T08:20:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 44, 2024-09-10T08:20:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:20:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:20:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:20:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:20:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:20:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:20:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 138.1, 2024-09-10T08:20:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:20:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:20:43.448 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:20:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:20:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:20:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:20:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:43.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:43.449 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:20:43.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.445+0000 7fb2af400640 1 -- 172.21.15.2:0/1743730607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb2900a2b80 msgr2=0x7fb2900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:43.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.445+0000 7fb2af400640 1 --2- 172.21.15.2:0/1743730607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb2900a2b80 0x7fb2900a5040 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0005fd0 tx=0x7fb2a0009500 comp rx=0 tx=0).stop 2024-09-10T08:20:43.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.445+0000 7fb2af400640 1 -- 172.21.15.2:0/1743730607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a8103ff0 msgr2=0x7fb2a819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:43.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.445+0000 7fb2af400640 1 --2- 172.21.15.2:0/1743730607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a8103ff0 0x7fb2a819a630 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb29c035040 tx=0x7fb29c033b60 comp rx=0 tx=0).stop 2024-09-10T08:20:43.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.445+0000 7fb2af400640 1 -- 172.21.15.2:0/1743730607 shutdown_connections 2024-09-10T08:20:43.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.445+0000 7fb2af400640 1 --2- 172.21.15.2:0/1743730607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb2900a2b80 0x7fb2900a5040 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.445+0000 7fb2af400640 1 --2- 172.21.15.2:0/1743730607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a8109120 0x7fb2a819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.445+0000 7fb2af400640 1 --2- 172.21.15.2:0/1743730607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a81049a0 0x7fb2a819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.445+0000 7fb2af400640 1 --2- 172.21.15.2:0/1743730607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a8103ff0 0x7fb2a819a630 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:43.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.445+0000 7fb2af400640 1 -- 172.21.15.2:0/1743730607 >> 172.21.15.2:0/1743730607 conn(0x7fb2a80ffa90 msgr2=0x7fb2a8100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:43.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.446+0000 7fb2af400640 1 -- 172.21.15.2:0/1743730607 shutdown_connections 2024-09-10T08:20:43.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:43.446+0000 7fb2af400640 1 -- 172.21.15.2:0/1743730607 wait complete. 2024-09-10T08:20:43.453 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:20:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:43 smithi155 ceph-mon[35405]: pgmap v764: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-10T08:20:43.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:43 smithi107 ceph-mon[35048]: pgmap v764: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-10T08:20:43.986 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:43 smithi002 ceph-mon[30563]: pgmap v764: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-10T08:20:44.473 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '41453dd0', total seconds elapsed: 22.5 2024-09-10T08:20:44.493 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:45.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.144+0000 7f96ba600640 1 -- 172.21.15.2:0/3972308110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96b4101de0 msgr2=0x7f96b41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:45.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.144+0000 7f96ba600640 1 --2- 172.21.15.2:0/3972308110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96b4101de0 0x7f96b41021c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009900 tx=0x7f96a402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.144+0000 7f96ba600640 1 -- 172.21.15.2:0/3972308110 shutdown_connections 2024-09-10T08:20:45.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.144+0000 7f96ba600640 1 --2- 172.21.15.2:0/3972308110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96b4106f10 0x7f96b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.144+0000 7f96ba600640 1 --2- 172.21.15.2:0/3972308110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b4102790 0x7f96b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.144+0000 7f96ba600640 1 --2- 172.21.15.2:0/3972308110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96b4101de0 0x7f96b41021c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.144+0000 7f96ba600640 1 -- 172.21.15.2:0/3972308110 >> 172.21.15.2:0/3972308110 conn(0x7f96b40fd8c0 msgr2=0x7f96b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:45.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.144+0000 7f96ba600640 1 -- 172.21.15.2:0/3972308110 shutdown_connections 2024-09-10T08:20:45.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.144+0000 7f96ba600640 1 -- 172.21.15.2:0/3972308110 wait complete. 2024-09-10T08:20:45.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.145+0000 7f96ba600640 1 Processor -- start 2024-09-10T08:20:45.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.145+0000 7f96ba600640 1 -- start start 2024-09-10T08:20:45.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.145+0000 7f96ba600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b4101de0 0x7f96b419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.145+0000 7f96ba600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96b4102790 0x7f96b419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.145+0000 7f96ba600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96b4106f10 0x7f96b419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96ba600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4111fc0 con 0x7f96b4106f10 2024-09-10T08:20:45.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96ba600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4111e40 con 0x7f96b4102790 2024-09-10T08:20:45.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96ba600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4112140 con 0x7f96b4101de0 2024-09-10T08:20:45.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96b4106f10 0x7f96b419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96b4106f10 0x7f96b419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39604/0 (socket says 172.21.15.2:39604) 2024-09-10T08:20:45.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96b3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b4101de0 0x7f96b419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96b3e00640 1 -- 172.21.15.2:0/482772225 learned_addr learned my addr 172.21.15.2:0/482772225 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96abe00640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96b4102790 0x7f96b419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96b3400640 1 -- 172.21.15.2:0/482772225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96b4102790 msgr2=0x7f96b419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96b3400640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96b4102790 0x7f96b419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96b3400640 1 -- 172.21.15.2:0/482772225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96b4106f10 msgr2=0x7f96b419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96b3400640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96b4106f10 0x7f96b419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96b3400640 1 -- 172.21.15.2:0/482772225 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b419f5f0 con 0x7f96b4101de0 2024-09-10T08:20:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.146+0000 7f96b3e00640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96b4106f10 0x7f96b419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.147+0000 7f96b3400640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b4101de0 0x7f96b419a6e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f96a402a910 tx=0x7f96a4003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.147+0000 7f96b0c00640 1 -- 172.21.15.2:0/482772225 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a403f070 con 0x7f96b4101de0 2024-09-10T08:20:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.147+0000 7f96ba600640 1 -- 172.21.15.2:0/482772225 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b419f880 con 0x7f96b4101de0 2024-09-10T08:20:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.147+0000 7f96b0c00640 1 -- 172.21.15.2:0/482772225 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a4031c90 con 0x7f96b4101de0 2024-09-10T08:20:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.147+0000 7f96b0c00640 1 -- 172.21.15.2:0/482772225 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4035030 con 0x7f96b4101de0 2024-09-10T08:20:45.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.147+0000 7f96ba600640 1 -- 172.21.15.2:0/482772225 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b41a71c0 con 0x7f96b4101de0 2024-09-10T08:20:45.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.151+0000 7f96bd244740 1 Processor -- start 2024-09-10T08:20:45.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.151+0000 7f96bd244740 1 -- start start 2024-09-10T08:20:45.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96bd244740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9eb90 0x55f4c1f9ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96bd244740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fa9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96bd244740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fac160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96bd244740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f4c1f053e0 con 0x55f4c1fa9d70 2024-09-10T08:20:45.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96bd244740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f4c1eacd80 con 0x55f4c1f9eb90 2024-09-10T08:20:45.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96bd244740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f4c1f013a0 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96b0c00640 1 -- 172.21.15.2:0/482772225 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f96a4042070 con 0x7f96b4101de0 2024-09-10T08:20:45.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96abe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96abe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39620/0 (socket says 172.21.15.2:39620) 2024-09-10T08:20:45.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96abe00640 1 -- 172.21.15.2:0/357293994 learned_addr learned my addr 172.21.15.2:0/357293994 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:45.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9eb90 0x55f4c1f9ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96b3400640 1 --2- 172.21.15.2:0/357293994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fa9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96abe00640 1 -- 172.21.15.2:0/357293994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1f9f4b0 msgr2=0x55f4c1fa9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:45.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96abe00640 1 --2- 172.21.15.2:0/357293994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fa9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.152+0000 7f96abe00640 1 -- 172.21.15.2:0/357293994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9eb90 msgr2=0x55f4c1f9ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:45.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.153+0000 7f96abe00640 1 --2- 172.21.15.2:0/357293994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9eb90 0x55f4c1f9ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.153+0000 7f96abe00640 1 -- 172.21.15.2:0/357293994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c1ecbae0 con 0x55f4c1fa9d70 2024-09-10T08:20:45.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.153+0000 7f96b3e00640 1 --2- 172.21.15.2:0/357293994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9eb90 0x55f4c1f9ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:45.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.153+0000 7f96abe00640 1 --2- 172.21.15.2:0/357293994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fac160 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f969c009c90 tx=0x7f969c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a4727f1622c7ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:45.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.153+0000 7f96a8c00640 1 -- 172.21.15.2:0/357293994 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c01e070 con 0x55f4c1fa9d70 2024-09-10T08:20:45.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.153+0000 7f96a8c00640 1 -- 172.21.15.2:0/357293994 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f969c019b10 con 0x55f4c1fa9d70 2024-09-10T08:20:45.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.153+0000 7f96a8c00640 1 -- 172.21.15.2:0/357293994 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c00ea60 con 0x55f4c1fa9d70 2024-09-10T08:20:45.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.154+0000 7f96b0c00640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f96980a2c80 0x7f96980a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.154+0000 7f96b0c00640 1 -- 172.21.15.2:0/482772225 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f96a40efd30 con 0x7f96b4101de0 2024-09-10T08:20:45.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.154+0000 7f96b3e00640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f96980a2c80 0x7f96980a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.154+0000 7f96bd244740 1 -- 172.21.15.2:0/357293994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1fa9d70 msgr2=0x55f4c1fac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:45.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.154+0000 7f96bd244740 1 --2- 172.21.15.2:0/357293994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fac160 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f969c009c90 tx=0x7f969c011810 comp rx=0 tx=0).stop 2024-09-10T08:20:45.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.154+0000 7f96bd244740 1 -- 172.21.15.2:0/357293994 shutdown_connections 2024-09-10T08:20:45.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.154+0000 7f96bd244740 1 --2- 172.21.15.2:0/357293994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fac160 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.154+0000 7f96bd244740 1 --2- 172.21.15.2:0/357293994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fa9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.154+0000 7f96bd244740 1 --2- 172.21.15.2:0/357293994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9eb90 0x55f4c1f9ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.154+0000 7f96bd244740 1 -- 172.21.15.2:0/357293994 >> 172.21.15.2:0/357293994 conn(0x55f4c1f9b120 msgr2=0x55f4c1f9b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:45.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.154+0000 7f96bd244740 1 -- 172.21.15.2:0/357293994 shutdown_connections 2024-09-10T08:20:45.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.155+0000 7f96bd244740 1 -- 172.21.15.2:0/357293994 wait complete. 2024-09-10T08:20:45.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.155+0000 7f96b3e00640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f96980a2c80 0x7f96980a5140 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009fd0 tx=0x7f96ac009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:45.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.155+0000 7f96bd244740 1 -- start start 2024-09-10T08:20:45.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.155+0000 7f96bd244740 1 Processor -- start 2024-09-10T08:20:45.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96bd244740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fa6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96bd244740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fa71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96bd244740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fa7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96bd244740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f4c1fac9e0 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96bd244740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f4c1ec00f0 con 0x55f4c1f9eb90 2024-09-10T08:20:45.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96bd244740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f4c1facb60 con 0x55f4c1fa9d70 2024-09-10T08:20:45.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96bd244740 10 client.0 _ll_register_callbacks cb 0x55f4c1eb19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:45.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96abe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fa71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96abe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fa71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39632/0 (socket says 172.21.15.2:39632) 2024-09-10T08:20:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96abe00640 1 -- 172.21.15.2:0/403521425 learned_addr learned my addr 172.21.15.2:0/403521425 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96b3400640 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fa6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96b3e00640 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fa7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96abe00640 1 -- 172.21.15.2:0/403521425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1fa9d70 msgr2=0x55f4c1fa7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96abe00640 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fa7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96abe00640 1 -- 172.21.15.2:0/403521425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9eb90 msgr2=0x55f4c1fa6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96abe00640 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fa6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96abe00640 1 -- 172.21.15.2:0/403521425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c1c763c0 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.156+0000 7f96b3400640 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fa6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.157+0000 7f96abe00640 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fa71c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f969c005bc0 tx=0x7f969c00e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5337f0f1b14afa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.157+0000 7f96a1600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.157+0000 7f96a1600640 1 -- 172.21.15.2:0/403521425 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c029020 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.157+0000 7f96a1600640 1 -- 172.21.15.2:0/403521425 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f969c005040 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.157+0000 7f96bd244740 1 -- 172.21.15.2:0/403521425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f969c019b10 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.157+0000 7f96a1600640 1 -- 172.21.15.2:0/403521425 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f969c034050 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.157+0000 7f96bd244740 10 client.18954 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.157+0000 7f96bd244740 1 -- 172.21.15.2:0/403521425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4c1fb86c0 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.158+0000 7f96a1600640 1 -- 172.21.15.2:0/403521425 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f969c019b10 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.158+0000 7f96a1600640 1 -- 172.21.15.2:0/403521425 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f969c00ec70 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.158+0000 7f96bd244740 10 client.18954 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.158+0000 7f96bd244740 10 client.18954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.158+0000 7f96bd244740 20 client.18954 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.158+0000 7f96bd244740 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f4c1fb9b80 0x55f4c1fbbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.158+0000 7f96bd244740 4 client.18954 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.158+0000 7f96bd244740 1 -- 172.21.15.2:0/403521425 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4c1f12950 con 0x55f4c1fb9b80 2024-09-10T08:20:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.158+0000 7f96b3400640 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f4c1fb9b80 0x55f4c1fbbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.159+0000 7f96b3400640 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f4c1fb9b80 0x55f4c1fbbf70 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2348c0a06d9dcce server_cookie=d36d2efcfc9e757e in_seq=0 out_seq=0 2024-09-10T08:20:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.159+0000 7f96a1600640 10 client.18954 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96a1600640 1 -- 172.21.15.2:0/403521425 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4c1f12950 con 0x55f4c1fb9b80 2024-09-10T08:20:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96a1600640 10 client.18954 handle_command_reply: tid=0 2024-09-10T08:20:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96bd244740 1 client.18954 shutdown 2024-09-10T08:20:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96bd244740 1 -- 172.21.15.2:0/403521425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9f4b0 msgr2=0x55f4c1fa71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96bd244740 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fa71c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f969c005bc0 tx=0x7f969c00e5a0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96bd244740 1 -- 172.21.15.2:0/403521425 shutdown_connections 2024-09-10T08:20:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96bd244740 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f4c1fb9b80 0x55f4c1fbbf70 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96bd244740 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fa7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96bd244740 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fa71c0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96bd244740 1 --2- 172.21.15.2:0/403521425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fa6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96a1600640 0 client.18954 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96bd244740 1 -- 172.21.15.2:0/403521425 >> 172.21.15.2:0/403521425 conn(0x55f4c1f9b120 msgr2=0x55f4c1fa0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96bd244740 1 -- 172.21.15.2:0/403521425 shutdown_connections 2024-09-10T08:20:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96bd244740 1 -- 172.21.15.2:0/403521425 wait complete. 2024-09-10T08:20:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.160+0000 7f96bd244740 20 client.18954 trim_cache size 0 max 16384 2024-09-10T08:20:45.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.178+0000 7f96bd244740 1 Processor -- start 2024-09-10T08:20:45.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.178+0000 7f96bd244740 1 -- start start 2024-09-10T08:20:45.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.179+0000 7f96bd244740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fc1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.179+0000 7f96bd244740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fc1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.179+0000 7f96bd244740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fa3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.179+0000 7f96bd244740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41b3910 con 0x55f4c1f9eb90 2024-09-10T08:20:45.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.179+0000 7f96bd244740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41b11e0 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.179+0000 7f96bd244740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41af030 con 0x55f4c1fa9d70 2024-09-10T08:20:45.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.179+0000 7f96b3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fc1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.179+0000 7f96b3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fc1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39648/0 (socket says 172.21.15.2:39648) 2024-09-10T08:20:45.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.179+0000 7f96b3400640 1 -- 172.21.15.2:0/2231164008 learned_addr learned my addr 172.21.15.2:0/2231164008 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:45.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.179+0000 7f96abe00640 1 --2- 172.21.15.2:0/2231164008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fc1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.179+0000 7f96b3e00640 1 --2- 172.21.15.2:0/2231164008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fa3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.179+0000 7f96b3400640 1 -- 172.21.15.2:0/2231164008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1fa9d70 msgr2=0x55f4c1fa3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:45.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.180+0000 7f96b3400640 1 --2- 172.21.15.2:0/2231164008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fa3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.180+0000 7f96b3400640 1 -- 172.21.15.2:0/2231164008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9f4b0 msgr2=0x55f4c1fc1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:45.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.180+0000 7f96b3400640 1 --2- 172.21.15.2:0/2231164008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fc1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.180+0000 7f96b3400640 1 -- 172.21.15.2:0/2231164008 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c1f9bdb0 con 0x55f4c1f9eb90 2024-09-10T08:20:45.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.180+0000 7f96b3400640 1 --2- 172.21.15.2:0/2231164008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fc1700 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009900 tx=0x7f96a4031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d647f6b9c411ec0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:45.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.180+0000 7f96a3e00640 1 -- 172.21.15.2:0/2231164008 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4049040 con 0x55f4c1f9eb90 2024-09-10T08:20:45.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.180+0000 7f96a3e00640 1 -- 172.21.15.2:0/2231164008 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a40bc1f0 con 0x55f4c1f9eb90 2024-09-10T08:20:45.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.180+0000 7f96a3e00640 1 -- 172.21.15.2:0/2231164008 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a403b460 con 0x55f4c1f9eb90 2024-09-10T08:20:45.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.181+0000 7f96bd244740 1 -- 172.21.15.2:0/2231164008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9eb90 msgr2=0x55f4c1fc1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:45.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.181+0000 7f96bd244740 1 --2- 172.21.15.2:0/2231164008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fc1700 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009900 tx=0x7f96a4031490 comp rx=0 tx=0).stop 2024-09-10T08:20:45.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.181+0000 7f96bd244740 1 -- 172.21.15.2:0/2231164008 shutdown_connections 2024-09-10T08:20:45.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.181+0000 7f96bd244740 1 --2- 172.21.15.2:0/2231164008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fa3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.181+0000 7f96bd244740 1 --2- 172.21.15.2:0/2231164008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fc1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.181+0000 7f96bd244740 1 --2- 172.21.15.2:0/2231164008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fc1700 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.181+0000 7f96bd244740 1 -- 172.21.15.2:0/2231164008 >> 172.21.15.2:0/2231164008 conn(0x55f4c1f9b120 msgr2=0x55f4c1fa5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:45.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.181+0000 7f96bd244740 1 -- 172.21.15.2:0/2231164008 shutdown_connections 2024-09-10T08:20:45.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.181+0000 7f96bd244740 1 -- 172.21.15.2:0/2231164008 wait complete. 2024-09-10T08:20:45.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.181+0000 7f96bd244740 1 -- start start 2024-09-10T08:20:45.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96bd244740 1 Processor -- start 2024-09-10T08:20:45.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96bd244740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fadd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96bd244740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96bd244740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96bd244740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41af6a0 con 0x55f4c1fa9d70 2024-09-10T08:20:45.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96bd244740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41b0b10 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96bd244740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41afd70 con 0x55f4c1f9eb90 2024-09-10T08:20:45.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96bd244740 10 client.0 _ll_register_callbacks cb 0x7f969c009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:45.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96b3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fadd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96b3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fadd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:48042/0 (socket says 172.21.15.2:48042) 2024-09-10T08:20:45.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96abe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39654/0 (socket says 172.21.15.2:39654) 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.182+0000 7f96b3400640 1 -- 172.21.15.2:0/1767686453 learned_addr learned my addr 172.21.15.2:0/1767686453 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.183+0000 7f96abe00640 1 -- 172.21.15.2:0/1767686453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1f9eb90 msgr2=0x55f4c1fadd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.183+0000 7f96abe00640 1 --2- 172.21.15.2:0/1767686453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fadd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.183+0000 7f96abe00640 1 -- 172.21.15.2:0/1767686453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1fa9d70 msgr2=0x55f4c1fae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.183+0000 7f96abe00640 1 --2- 172.21.15.2:0/1767686453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.183+0000 7f96abe00640 1 -- 172.21.15.2:0/1767686453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c20239c0 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.183+0000 7f96b3400640 1 --2- 172.21.15.2:0/1767686453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fadd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.183+0000 7f96abe00640 1 --2- 172.21.15.2:0/1767686453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fae2d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f969c005b90 tx=0x7f969c009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b52e887b2f4d6d3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.183+0000 7f96a1600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.183+0000 7f96a1600640 1 -- 172.21.15.2:0/1767686453 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c03d060 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.184+0000 7f96bd244740 1 -- 172.21.15.2:0/1767686453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a40bc1f0 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.184+0000 7f96a1600640 1 -- 172.21.15.2:0/1767686453 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f969c019da0 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.184+0000 7f96a1600640 1 -- 172.21.15.2:0/1767686453 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f969c044050 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.184+0000 7f96bd244740 10 client.28985 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:45.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.184+0000 7f96bd244740 1 -- 172.21.15.2:0/1767686453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4c2024c50 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.184+0000 7f96a1600640 1 -- 172.21.15.2:0/1767686453 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a40bc1f0 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.185+0000 7f96a1600640 1 -- 172.21.15.2:0/1767686453 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f969c003ab0 con 0x55f4c1f9f4b0 2024-09-10T08:20:45.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.185+0000 7f96bd244740 10 client.28985 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:45.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.185+0000 7f96bd244740 10 client.28985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:45.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.185+0000 7f96bd244740 20 client.28985 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:45.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.185+0000 7f96bd244740 1 --2- 172.21.15.2:0/1767686453 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f4c20252b0 0x55f4c20276d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:45.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.185+0000 7f96bd244740 4 client.28985 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:20:45.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.185+0000 7f96bd244740 1 -- 172.21.15.2:0/1767686453 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f4c1f0ce60 con 0x55f4c20252b0 2024-09-10T08:20:45.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.185+0000 7f96b3400640 1 --2- 172.21.15.2:0/1767686453 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f4c20252b0 0x55f4c20276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:45.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.185+0000 7f96b3400640 1 --2- 172.21.15.2:0/1767686453 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f4c20252b0 0x55f4c20276d0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbbca2cc6bd91616 server_cookie=4d51778c89baca05 in_seq=0 out_seq=0 2024-09-10T08:20:45.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.185+0000 7f96a1600640 10 client.28985 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:45.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96a1600640 1 -- 172.21.15.2:0/1767686453 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f4c1f0ce60 con 0x55f4c20252b0 2024-09-10T08:20:45.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96a1600640 10 client.28985 handle_command_reply: tid=0 2024-09-10T08:20:45.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96bd244740 1 client.28985 shutdown 2024-09-10T08:20:45.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96bd244740 1 -- 172.21.15.2:0/1767686453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9f4b0 msgr2=0x55f4c1fae2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:45.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96bd244740 1 --2- 172.21.15.2:0/1767686453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fae2d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f969c005b90 tx=0x7f969c009f00 comp rx=0 tx=0).stop 2024-09-10T08:20:45.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96bd244740 1 -- 172.21.15.2:0/1767686453 shutdown_connections 2024-09-10T08:20:45.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96bd244740 1 --2- 172.21.15.2:0/1767686453 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f4c20252b0 0x55f4c20276d0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96bd244740 1 --2- 172.21.15.2:0/1767686453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f4c1fa9d70 0x55f4c1fae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96bd244740 1 --2- 172.21.15.2:0/1767686453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4c1f9f4b0 0x55f4c1fae2d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96bd244740 1 --2- 172.21.15.2:0/1767686453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4c1f9eb90 0x55f4c1fadd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96a1600640 0 client.28985 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:45.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96bd244740 1 -- 172.21.15.2:0/1767686453 >> 172.21.15.2:0/1767686453 conn(0x55f4c1f9b120 msgr2=0x55f4c1f9a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:45.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96bd244740 1 -- 172.21.15.2:0/1767686453 shutdown_connections 2024-09-10T08:20:45.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96bd244740 1 -- 172.21.15.2:0/1767686453 wait complete. 2024-09-10T08:20:45.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.186+0000 7f96bd244740 20 client.28985 trim_cache size 0 max 16384 2024-09-10T08:20:45.191 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:20:45.191 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:20:45.191 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:20:45.191 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:20:45.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.187+0000 7f96ba600640 1 -- 172.21.15.2:0/482772225 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f96980a2c80 msgr2=0x7f96980a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:45.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.187+0000 7f96ba600640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f96980a2c80 0x7f96980a5140 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009fd0 tx=0x7f96ac009500 comp rx=0 tx=0).stop 2024-09-10T08:20:45.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.187+0000 7f96ba600640 1 -- 172.21.15.2:0/482772225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b4101de0 msgr2=0x7f96b419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:45.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.187+0000 7f96ba600640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b4101de0 0x7f96b419a6e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f96a402a910 tx=0x7f96a4003b20 comp rx=0 tx=0).stop 2024-09-10T08:20:45.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.187+0000 7f96ba600640 1 -- 172.21.15.2:0/482772225 shutdown_connections 2024-09-10T08:20:45.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.187+0000 7f96ba600640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96b4106f10 0x7f96b419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.187+0000 7f96ba600640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96b4102790 0x7f96b419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.187+0000 7f96ba600640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f96980a2c80 0x7f96980a5140 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.187+0000 7f96ba600640 1 --2- 172.21.15.2:0/482772225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b4101de0 0x7f96b419a6e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:45.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.187+0000 7f96ba600640 1 -- 172.21.15.2:0/482772225 >> 172.21.15.2:0/482772225 conn(0x7f96b40fd8c0 msgr2=0x7f96b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:45.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.187+0000 7f96ba600640 1 -- 172.21.15.2:0/482772225 shutdown_connections 2024-09-10T08:20:45.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:45.187+0000 7f96ba600640 1 -- 172.21.15.2:0/482772225 wait complete. 2024-09-10T08:20:45.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:45 smithi155 ceph-mon[35405]: pgmap v765: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-09-10T08:20:45.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:45 smithi107 ceph-mon[35048]: pgmap v765: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-09-10T08:20:46.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:45 smithi002 ceph-mon[30563]: pgmap v765: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-09-10T08:20:46.039 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:20:46.040 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:20:46.460 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:47.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.068+0000 7faadce00640 1 -- 172.21.15.2:0/3875545611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad8108f60 msgr2=0x7faad8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.068+0000 7faadce00640 1 --2- 172.21.15.2:0/3875545611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad8108f60 0x7faad8110ae0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7faad0009900 tx=0x7faad002a410 comp rx=0 tx=0).stop 2024-09-10T08:20:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.068+0000 7faadce00640 1 -- 172.21.15.2:0/3875545611 shutdown_connections 2024-09-10T08:20:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.068+0000 7faadce00640 1 --2- 172.21.15.2:0/3875545611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad8108f60 0x7faad8110ae0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.068+0000 7faadce00640 1 --2- 172.21.15.2:0/3875545611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad81047e0 0x7faad8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.068+0000 7faadce00640 1 --2- 172.21.15.2:0/3875545611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faad8103e30 0x7faad8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.068+0000 7faadce00640 1 -- 172.21.15.2:0/3875545611 >> 172.21.15.2:0/3875545611 conn(0x7faad80ff910 msgr2=0x7faad8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.068+0000 7faadce00640 1 -- 172.21.15.2:0/3875545611 shutdown_connections 2024-09-10T08:20:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.068+0000 7faadce00640 1 -- 172.21.15.2:0/3875545611 wait complete. 2024-09-10T08:20:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.069+0000 7faadce00640 1 Processor -- start 2024-09-10T08:20:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.069+0000 7faadce00640 1 -- start start 2024-09-10T08:20:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faadce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad8103e30 0x7faad819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faadce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad81047e0 0x7faad819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faadce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faad8108f60 0x7faad819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faadce00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faad8113fc0 con 0x7faad81047e0 2024-09-10T08:20:47.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faadce00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faad8113e40 con 0x7faad8108f60 2024-09-10T08:20:47.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faadce00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faad8114140 con 0x7faad8103e30 2024-09-10T08:20:47.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faad6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad81047e0 0x7faad819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faad6a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad81047e0 0x7faad819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44182/0 (socket says 172.21.15.2:44182) 2024-09-10T08:20:47.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faad6a00640 1 -- 172.21.15.2:0/29931725 learned_addr learned my addr 172.21.15.2:0/29931725 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:47.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faad7400640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad8103e30 0x7faad819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faad7e00640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faad8108f60 0x7faad819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faad6a00640 1 -- 172.21.15.2:0/29931725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad8103e30 msgr2=0x7faad819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faad6a00640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad8103e30 0x7faad819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faad6a00640 1 -- 172.21.15.2:0/29931725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faad8108f60 msgr2=0x7faad819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faad6a00640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faad8108f60 0x7faad819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.070+0000 7faad6a00640 1 -- 172.21.15.2:0/29931725 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad819f3d0 con 0x7faad81047e0 2024-09-10T08:20:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.071+0000 7faad6a00640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad81047e0 0x7faad819a960 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7faac400e9f0 tx=0x7faac400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.071+0000 7faacfe00640 1 -- 172.21.15.2:0/29931725 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac400cd80 con 0x7faad81047e0 2024-09-10T08:20:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.071+0000 7faadce00640 1 -- 172.21.15.2:0/29931725 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad819f6c0 con 0x7faad81047e0 2024-09-10T08:20:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.071+0000 7faacfe00640 1 -- 172.21.15.2:0/29931725 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac4005500 con 0x7faad81047e0 2024-09-10T08:20:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.071+0000 7faadce00640 1 -- 172.21.15.2:0/29931725 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad81a6fa0 con 0x7faad81047e0 2024-09-10T08:20:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.071+0000 7faacfe00640 1 -- 172.21.15.2:0/29931725 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4018630 con 0x7faad81047e0 2024-09-10T08:20:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.072+0000 7faadce00640 1 -- 172.21.15.2:0/29931725 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad8105310 con 0x7faad81047e0 2024-09-10T08:20:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.075+0000 7faacfe00640 1 -- 172.21.15.2:0/29931725 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7faac40046c0 con 0x7faad81047e0 2024-09-10T08:20:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.076+0000 7faacfe00640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faabc0a2b80 0x7faabc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.076+0000 7faacfe00640 1 -- 172.21.15.2:0/29931725 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faac40cf950 con 0x7faad81047e0 2024-09-10T08:20:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.076+0000 7faad7400640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faabc0a2b80 0x7faabc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.077+0000 7faad7400640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faabc0a2b80 0x7faabc0a5040 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faac80099a0 tx=0x7faac8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:47.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.080+0000 7faacfe00640 1 -- 172.21.15.2:0/29931725 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faac409c460 con 0x7faad81047e0 2024-09-10T08:20:47.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.304+0000 7faadce00640 1 -- 172.21.15.2:0/29931725 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faad8104210 con 0x7faad81047e0 2024-09-10T08:20:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.305+0000 7faacfe00640 1 -- 172.21.15.2:0/29931725 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7faad8104210 con 0x7faad81047e0 2024-09-10T08:20:47.305 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:20:47.305 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:20:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:20:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.307+0000 7faadce00640 1 -- 172.21.15.2:0/29931725 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faabc0a2b80 msgr2=0x7faabc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.307+0000 7faadce00640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faabc0a2b80 0x7faabc0a5040 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faac80099a0 tx=0x7faac8009500 comp rx=0 tx=0).stop 2024-09-10T08:20:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.307+0000 7faadce00640 1 -- 172.21.15.2:0/29931725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad81047e0 msgr2=0x7faad819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.307+0000 7faadce00640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad81047e0 0x7faad819a960 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7faac400e9f0 tx=0x7faac400edf0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.307+0000 7faadce00640 1 -- 172.21.15.2:0/29931725 shutdown_connections 2024-09-10T08:20:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.308+0000 7faadce00640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faad8108f60 0x7faad819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.308+0000 7faadce00640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faad81047e0 0x7faad819a960 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.308+0000 7faadce00640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faabc0a2b80 0x7faabc0a5040 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.308+0000 7faadce00640 1 --2- 172.21.15.2:0/29931725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faad8103e30 0x7faad819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.308+0000 7faadce00640 1 -- 172.21.15.2:0/29931725 >> 172.21.15.2:0/29931725 conn(0x7faad80ff910 msgr2=0x7faad8100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.308+0000 7faadce00640 1 -- 172.21.15.2:0/29931725 shutdown_connections 2024-09-10T08:20:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:47.308+0000 7faadce00640 1 -- 172.21.15.2:0/29931725 wait complete. 2024-09-10T08:20:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.853+0000 7f116de00640 1 -- 172.21.15.2:0/788765352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1168074bf0 msgr2=0x7f1168075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.853+0000 7f116de00640 1 --2- 172.21.15.2:0/788765352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1168074bf0 0x7f1168075050 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1154009900 tx=0x7f115402a480 comp rx=0 tx=0).stop 2024-09-10T08:20:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.853+0000 7f116de00640 1 -- 172.21.15.2:0/788765352 shutdown_connections 2024-09-10T08:20:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.853+0000 7f116de00640 1 --2- 172.21.15.2:0/788765352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11681099f0 0x7f116810beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.853+0000 7f116de00640 1 --2- 172.21.15.2:0/788765352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1168074bf0 0x7f1168075050 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.853+0000 7f116de00640 1 --2- 172.21.15.2:0/788765352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11680761d0 0x7f1168074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.853+0000 7f116de00640 1 -- 172.21.15.2:0/788765352 >> 172.21.15.2:0/788765352 conn(0x7f11680fd2e0 msgr2=0x7f11680ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.853+0000 7f116de00640 1 -- 172.21.15.2:0/788765352 shutdown_connections 2024-09-10T08:20:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.854+0000 7f116de00640 1 -- 172.21.15.2:0/788765352 wait complete. 2024-09-10T08:20:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.854+0000 7f116de00640 1 Processor -- start 2024-09-10T08:20:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.855+0000 7f116de00640 1 -- start start 2024-09-10T08:20:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.855+0000 7f116de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1168074bf0 0x7f116819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.855+0000 7f116de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11680761d0 0x7f116819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.855+0000 7f116de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11681099f0 0x7f11681a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.855+0000 7f116de00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f116810f9f0 con 0x7f11680761d0 2024-09-10T08:20:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.855+0000 7f116de00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f116810f870 con 0x7f1168074bf0 2024-09-10T08:20:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.855+0000 7f116de00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f116810fb70 con 0x7f11681099f0 2024-09-10T08:20:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.855+0000 7f1166000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11680761d0 0x7f116819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.855+0000 7f1166000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11680761d0 0x7f116819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44194/0 (socket says 172.21.15.2:44194) 2024-09-10T08:20:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.855+0000 7f1166000640 1 -- 172.21.15.2:0/3007780341 learned_addr learned my addr 172.21.15.2:0/3007780341 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.855+0000 7f1166a00640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1168074bf0 0x7f116819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.855+0000 7f1167400640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11681099f0 0x7f11681a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.856+0000 7f1166000640 1 -- 172.21.15.2:0/3007780341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11681099f0 msgr2=0x7f11681a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.856+0000 7f1166000640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11681099f0 0x7f11681a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.856+0000 7f1166000640 1 -- 172.21.15.2:0/3007780341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1168074bf0 msgr2=0x7f116819a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.856+0000 7f1166000640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1168074bf0 0x7f116819a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.856+0000 7f1166000640 1 -- 172.21.15.2:0/3007780341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11681a1c40 con 0x7f11680761d0 2024-09-10T08:20:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.856+0000 7f1166000640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11680761d0 0x7f116819a560 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1154009230 tx=0x7f11540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.856+0000 7f116ca00640 1 -- 172.21.15.2:0/3007780341 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115403f030 con 0x7f11680761d0 2024-09-10T08:20:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.856+0000 7f116ca00640 1 -- 172.21.15.2:0/3007780341 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1154003ce0 con 0x7f11680761d0 2024-09-10T08:20:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.856+0000 7f116ca00640 1 -- 172.21.15.2:0/3007780341 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154035070 con 0x7f11680761d0 2024-09-10T08:20:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.856+0000 7f116de00640 1 -- 172.21.15.2:0/3007780341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11681a1ed0 con 0x7f11680761d0 2024-09-10T08:20:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.856+0000 7f116de00640 1 -- 172.21.15.2:0/3007780341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11681a2340 con 0x7f11680761d0 2024-09-10T08:20:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.859+0000 7f1170cbd740 1 Processor -- start 2024-09-10T08:20:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.860+0000 7f116ca00640 1 -- 172.21.15.2:0/3007780341 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1154042030 con 0x7f11680761d0 2024-09-10T08:20:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.860+0000 7f1170cbd740 1 -- start start 2024-09-10T08:20:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.860+0000 7f1170cbd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a090 0x562cbf69a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.860+0000 7f1170cbd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a9d0 0x562cbf69f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.860+0000 7f1170cbd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 0x562cbf6a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.860+0000 7f1170cbd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562cbf5ff100 con 0x562cbf69a090 2024-09-10T08:20:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.860+0000 7f1170cbd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562cbf5a71b0 con 0x562cbf69fa60 2024-09-10T08:20:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.860+0000 7f1170cbd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562cbf5fb0c0 con 0x562cbf69a9d0 2024-09-10T08:20:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.860+0000 7f1167400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a090 0x562cbf69a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.860+0000 7f1166a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a9d0 0x562cbf69f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.860+0000 7f1167400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a090 0x562cbf69a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44202/0 (socket says 172.21.15.2:44202) 2024-09-10T08:20:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.860+0000 7f1167400640 1 -- 172.21.15.2:0/2550339257 learned_addr learned my addr 172.21.15.2:0/2550339257 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.860+0000 7f1166000640 1 --2- 172.21.15.2:0/2550339257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 0x562cbf6a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.861+0000 7f1166a00640 1 -- 172.21.15.2:0/2550339257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 msgr2=0x562cbf6a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.861+0000 7f1166a00640 1 --2- 172.21.15.2:0/2550339257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 0x562cbf6a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.861+0000 7f1166a00640 1 -- 172.21.15.2:0/2550339257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a090 msgr2=0x562cbf69a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.861+0000 7f1166a00640 1 --2- 172.21.15.2:0/2550339257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a090 0x562cbf69a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.861+0000 7f1166a00640 1 -- 172.21.15.2:0/2550339257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cbf312210 con 0x562cbf69a9d0 2024-09-10T08:20:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.861+0000 7f1167400640 1 --2- 172.21.15.2:0/2550339257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a090 0x562cbf69a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.861+0000 7f1166a00640 1 --2- 172.21.15.2:0/2550339257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a9d0 0x562cbf69f490 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1158009c00 tx=0x7f1158011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d28cf620fed229a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.861+0000 7f116ca00640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f114c0a2cb0 0x7f114c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.861+0000 7f1153e00640 1 -- 172.21.15.2:0/2550339257 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158004230 con 0x562cbf69a9d0 2024-09-10T08:20:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.861+0000 7f116ca00640 1 -- 172.21.15.2:0/3007780341 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f11540ef540 con 0x7f11680761d0 2024-09-10T08:20:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.861+0000 7f1153e00640 1 -- 172.21.15.2:0/2550339257 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11580043d0 con 0x562cbf69a9d0 2024-09-10T08:20:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.861+0000 7f1153e00640 1 -- 172.21.15.2:0/2550339257 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158004d90 con 0x562cbf69a9d0 2024-09-10T08:20:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.861+0000 7f1167400640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f114c0a2cb0 0x7f114c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.862+0000 7f1170cbd740 1 -- 172.21.15.2:0/2550339257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a9d0 msgr2=0x562cbf69f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.862+0000 7f1167400640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f114c0a2cb0 0x7f114c0a5170 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1160011890 tx=0x7f1160009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.862+0000 7f1170cbd740 1 --2- 172.21.15.2:0/2550339257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a9d0 0x562cbf69f490 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1158009c00 tx=0x7f1158011810 comp rx=0 tx=0).stop 2024-09-10T08:20:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.862+0000 7f1170cbd740 1 -- 172.21.15.2:0/2550339257 shutdown_connections 2024-09-10T08:20:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.862+0000 7f1170cbd740 1 --2- 172.21.15.2:0/2550339257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 0x562cbf6a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.862+0000 7f1170cbd740 1 --2- 172.21.15.2:0/2550339257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a9d0 0x562cbf69f490 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.862+0000 7f1170cbd740 1 --2- 172.21.15.2:0/2550339257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a090 0x562cbf69a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.862+0000 7f1170cbd740 1 -- 172.21.15.2:0/2550339257 >> 172.21.15.2:0/2550339257 conn(0x562cbf695700 msgr2=0x562cbf695b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.862+0000 7f1170cbd740 1 -- 172.21.15.2:0/2550339257 shutdown_connections 2024-09-10T08:20:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.862+0000 7f1170cbd740 1 -- 172.21.15.2:0/2550339257 wait complete. 2024-09-10T08:20:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.863+0000 7f1170cbd740 1 -- start start 2024-09-10T08:20:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.863+0000 7f1170cbd740 1 Processor -- start 2024-09-10T08:20:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.863+0000 7f1170cbd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.863+0000 7f1170cbd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a9d0 0x562cbf6a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.863+0000 7f1170cbd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 0x562cbf6ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.863+0000 7f1170cbd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562cbf6a2a30 con 0x562cbf69a9d0 2024-09-10T08:20:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.863+0000 7f1170cbd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562cbf588090 con 0x562cbf69fa60 2024-09-10T08:20:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.863+0000 7f1170cbd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562cbf6a2bb0 con 0x562cbf69a090 2024-09-10T08:20:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.863+0000 7f1170cbd740 10 client.0 _ll_register_callbacks cb 0x562cbf5dcf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.864+0000 7f1166000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a9d0 0x562cbf6a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.864+0000 7f1166a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.864+0000 7f1166000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a9d0 0x562cbf6a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44218/0 (socket says 172.21.15.2:44218) 2024-09-10T08:20:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.864+0000 7f1166000640 1 -- 172.21.15.2:0/3083944213 learned_addr learned my addr 172.21.15.2:0/3083944213 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.864+0000 7f1167400640 1 --2- 172.21.15.2:0/3083944213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 0x562cbf6ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.864+0000 7f1166000640 1 -- 172.21.15.2:0/3083944213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 msgr2=0x562cbf6a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.864+0000 7f1166000640 1 --2- 172.21.15.2:0/3083944213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.864+0000 7f1166000640 1 -- 172.21.15.2:0/3083944213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 msgr2=0x562cbf6ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.864+0000 7f1166000640 1 --2- 172.21.15.2:0/3083944213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 0x562cbf6ab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.864+0000 7f1166000640 1 -- 172.21.15.2:0/3083944213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cbf3de620 con 0x562cbf69a9d0 2024-09-10T08:20:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.864+0000 7f1166000640 1 --2- 172.21.15.2:0/3083944213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a9d0 0x562cbf6a4750 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f11540c4090 tx=0x7f11540045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb83a1570fc3a17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.864+0000 7f1150c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.865+0000 7f1150c00640 1 -- 172.21.15.2:0/3083944213 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154049070 con 0x562cbf69a9d0 2024-09-10T08:20:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.865+0000 7f1150c00640 1 -- 172.21.15.2:0/3083944213 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f11540b8070 con 0x562cbf69a9d0 2024-09-10T08:20:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.865+0000 7f1170cbd740 1 -- 172.21.15.2:0/3083944213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11580043d0 con 0x562cbf69a9d0 2024-09-10T08:20:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.865+0000 7f1150c00640 1 -- 172.21.15.2:0/3083944213 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11540bb540 con 0x562cbf69a9d0 2024-09-10T08:20:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.865+0000 7f1170cbd740 10 client.18978 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.865+0000 7f1170cbd740 1 -- 172.21.15.2:0/3083944213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562cbf6ae580 con 0x562cbf69a9d0 2024-09-10T08:20:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.865+0000 7f1150c00640 1 -- 172.21.15.2:0/3083944213 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11580043d0 con 0x562cbf69a9d0 2024-09-10T08:20:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.865+0000 7f1150c00640 1 -- 172.21.15.2:0/3083944213 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f115404c060 con 0x562cbf69a9d0 2024-09-10T08:20:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.865+0000 7f1170cbd740 10 client.18978 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.865+0000 7f1170cbd740 10 client.18978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.865+0000 7f1170cbd740 20 client.18978 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.865+0000 7f1170cbd740 1 --2- 172.21.15.2:0/3083944213 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562cbf6afc90 0x562cbf6b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.866+0000 7f1170cbd740 4 client.18978 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.866+0000 7f1170cbd740 1 -- 172.21.15.2:0/3083944213 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cbf60c830 con 0x562cbf6afc90 2024-09-10T08:20:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.866+0000 7f1166a00640 1 --2- 172.21.15.2:0/3083944213 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562cbf6afc90 0x562cbf6b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.866+0000 7f1166a00640 1 --2- 172.21.15.2:0/3083944213 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562cbf6afc90 0x562cbf6b20c0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c43aa1cab9c0ff8 server_cookie=a7aba2524baf9ae1 in_seq=0 out_seq=0 2024-09-10T08:20:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.866+0000 7f1150c00640 10 client.18978 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.867+0000 7f1150c00640 1 -- 172.21.15.2:0/3083944213 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cbf60c830 con 0x562cbf6afc90 2024-09-10T08:20:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.867+0000 7f1150c00640 10 client.18978 handle_command_reply: tid=0 2024-09-10T08:20:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.867+0000 7f1170cbd740 1 client.18978 shutdown 2024-09-10T08:20:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.867+0000 7f1170cbd740 1 -- 172.21.15.2:0/3083944213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a9d0 msgr2=0x562cbf6a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.867+0000 7f1170cbd740 1 --2- 172.21.15.2:0/3083944213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a9d0 0x562cbf6a4750 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f11540c4090 tx=0x7f11540045a0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.867+0000 7f1170cbd740 1 -- 172.21.15.2:0/3083944213 shutdown_connections 2024-09-10T08:20:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.867+0000 7f1170cbd740 1 --2- 172.21.15.2:0/3083944213 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562cbf6afc90 0x562cbf6b20c0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.867+0000 7f1170cbd740 1 --2- 172.21.15.2:0/3083944213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 0x562cbf6ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.867+0000 7f1170cbd740 1 --2- 172.21.15.2:0/3083944213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a9d0 0x562cbf6a4750 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.867+0000 7f1170cbd740 1 --2- 172.21.15.2:0/3083944213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.867+0000 7f1150c00640 0 client.18978 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.867+0000 7f1170cbd740 1 -- 172.21.15.2:0/3083944213 >> 172.21.15.2:0/3083944213 conn(0x562cbf695700 msgr2=0x562cbf696f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.868+0000 7f1170cbd740 1 -- 172.21.15.2:0/3083944213 shutdown_connections 2024-09-10T08:20:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.868+0000 7f1170cbd740 1 -- 172.21.15.2:0/3083944213 wait complete. 2024-09-10T08:20:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.868+0000 7f1170cbd740 20 client.18978 trim_cache size 0 max 16384 2024-09-10T08:20:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1170cbd740 1 Processor -- start 2024-09-10T08:20:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1170cbd740 1 -- start start 2024-09-10T08:20:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1170cbd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1170cbd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69a9d0 0x562cbf6ab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1170cbd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69fa60 0x562cbf6965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1170cbd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f11681af2b0 con 0x562cbf69fa60 2024-09-10T08:20:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1170cbd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11681acb80 con 0x562cbf69a9d0 2024-09-10T08:20:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1170cbd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11681aa9d0 con 0x562cbf69a090 2024-09-10T08:20:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1167400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69fa60 0x562cbf6965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1166000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69a9d0 0x562cbf6ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1166a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1166000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69a9d0 0x562cbf6ab750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:51868/0 (socket says 172.21.15.2:51868) 2024-09-10T08:20:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1166000640 1 -- 172.21.15.2:0/2971504563 learned_addr learned my addr 172.21.15.2:0/2971504563 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.883+0000 7f1167400640 1 --2- 172.21.15.2:0/2971504563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69fa60 0x562cbf6965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44234/0 (socket says 172.21.15.2:44234) 2024-09-10T08:20:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.884+0000 7f1166000640 1 -- 172.21.15.2:0/2971504563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 msgr2=0x562cbf6a2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.884+0000 7f1166000640 1 --2- 172.21.15.2:0/2971504563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.884+0000 7f1166000640 1 -- 172.21.15.2:0/2971504563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69fa60 msgr2=0x562cbf6965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.884+0000 7f1166000640 1 --2- 172.21.15.2:0/2971504563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69fa60 0x562cbf6965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.884+0000 7f1166000640 1 -- 172.21.15.2:0/2971504563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cbf695520 con 0x562cbf69a9d0 2024-09-10T08:20:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.884+0000 7f1166a00640 1 --2- 172.21.15.2:0/2971504563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.884+0000 7f1166000640 1 --2- 172.21.15.2:0/2971504563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69a9d0 0x562cbf6ab750 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f11540314e0 tx=0x7f11540bb890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4f587dad380cb0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.884+0000 7f1153400640 1 -- 172.21.15.2:0/2971504563 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11540d3070 con 0x562cbf69a9d0 2024-09-10T08:20:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.885+0000 7f1153400640 1 -- 172.21.15.2:0/2971504563 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11540ef320 con 0x562cbf69a9d0 2024-09-10T08:20:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.885+0000 7f1153400640 1 -- 172.21.15.2:0/2971504563 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154049070 con 0x562cbf69a9d0 2024-09-10T08:20:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.885+0000 7f1170cbd740 1 -- 172.21.15.2:0/2971504563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69a9d0 msgr2=0x562cbf6ab750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.885+0000 7f1170cbd740 1 --2- 172.21.15.2:0/2971504563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69a9d0 0x562cbf6ab750 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f11540314e0 tx=0x7f11540bb890 comp rx=0 tx=0).stop 2024-09-10T08:20:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.885+0000 7f1170cbd740 1 -- 172.21.15.2:0/2971504563 shutdown_connections 2024-09-10T08:20:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.885+0000 7f1170cbd740 1 --2- 172.21.15.2:0/2971504563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69fa60 0x562cbf6965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.885+0000 7f1170cbd740 1 --2- 172.21.15.2:0/2971504563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69a9d0 0x562cbf6ab750 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.885+0000 7f1170cbd740 1 --2- 172.21.15.2:0/2971504563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.885+0000 7f1170cbd740 1 -- 172.21.15.2:0/2971504563 >> 172.21.15.2:0/2971504563 conn(0x562cbf695700 msgr2=0x562cbf6a4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.885+0000 7f1170cbd740 1 -- 172.21.15.2:0/2971504563 shutdown_connections 2024-09-10T08:20:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.885+0000 7f1170cbd740 1 -- 172.21.15.2:0/2971504563 wait complete. 2024-09-10T08:20:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.886+0000 7f1170cbd740 1 -- start start 2024-09-10T08:20:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.886+0000 7f1170cbd740 1 Processor -- start 2024-09-10T08:20:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.886+0000 7f1170cbd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.886+0000 7f1170cbd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a9d0 0x562cbf6a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.886+0000 7f1170cbd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 0x562cbf6aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.886+0000 7f1170cbd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f11681ab040 con 0x562cbf69a9d0 2024-09-10T08:20:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.886+0000 7f1170cbd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11681ac4b0 con 0x562cbf69fa60 2024-09-10T08:20:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.886+0000 7f1170cbd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11681ab710 con 0x562cbf69a090 2024-09-10T08:20:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.886+0000 7f1170cbd740 10 client.0 _ll_register_callbacks cb 0x7f11540f2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.886+0000 7f1166a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.886+0000 7f1167400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 0x562cbf6aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.887+0000 7f1166a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:46998/0 (socket says 172.21.15.2:46998) 2024-09-10T08:20:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.887+0000 7f1166a00640 1 -- 172.21.15.2:0/324596235 learned_addr learned my addr 172.21.15.2:0/324596235 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.887+0000 7f1166000640 1 --2- 172.21.15.2:0/324596235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a9d0 0x562cbf6a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.887+0000 7f1166a00640 1 -- 172.21.15.2:0/324596235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 msgr2=0x562cbf6aa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.887+0000 7f1166a00640 1 --2- 172.21.15.2:0/324596235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 0x562cbf6aa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.887+0000 7f1166a00640 1 -- 172.21.15.2:0/324596235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a9d0 msgr2=0x562cbf6a92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.887+0000 7f1166a00640 1 --2- 172.21.15.2:0/324596235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a9d0 0x562cbf6a92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.887+0000 7f1166a00640 1 -- 172.21.15.2:0/324596235 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cbf6aa8c0 con 0x562cbf69a090 2024-09-10T08:20:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.887+0000 7f1166a00640 1 --2- 172.21.15.2:0/324596235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a8d60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1158009b20 tx=0x7f11580045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25604a961400faff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.887+0000 7f1150c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:20:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.887+0000 7f1150c00640 1 -- 172.21.15.2:0/324596235 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115801e040 con 0x562cbf69a090 2024-09-10T08:20:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.888+0000 7f1170cbd740 1 -- 172.21.15.2:0/324596235 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562cbf690d80 con 0x562cbf69a090 2024-09-10T08:20:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.888+0000 7f1150c00640 1 -- 172.21.15.2:0/324596235 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1158026650 con 0x562cbf69a090 2024-09-10T08:20:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.888+0000 7f1150c00640 1 -- 172.21.15.2:0/324596235 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f115803a050 con 0x562cbf69a090 2024-09-10T08:20:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.888+0000 7f1170cbd740 10 client.28942 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.888+0000 7f1170cbd740 1 -- 172.21.15.2:0/324596235 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562cbf71a9a0 con 0x562cbf69a090 2024-09-10T08:20:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.888+0000 7f1150c00640 1 -- 172.21.15.2:0/324596235 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562cbf690d80 con 0x562cbf69a090 2024-09-10T08:20:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.888+0000 7f1150c00640 1 -- 172.21.15.2:0/324596235 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1158024040 con 0x562cbf69a090 2024-09-10T08:20:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.888+0000 7f1170cbd740 10 client.28942 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.888+0000 7f1170cbd740 10 client.28942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.888+0000 7f1170cbd740 20 client.28942 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.888+0000 7f1170cbd740 1 --2- 172.21.15.2:0/324596235 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562cbf71b120 0x562cbf71d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.888+0000 7f1170cbd740 4 client.28942 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:20:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.888+0000 7f1170cbd740 1 -- 172.21.15.2:0/324596235 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562cbf606b80 con 0x562cbf71b120 2024-09-10T08:20:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.889+0000 7f1166000640 1 --2- 172.21.15.2:0/324596235 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562cbf71b120 0x562cbf71d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.889+0000 7f1166000640 1 --2- 172.21.15.2:0/324596235 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562cbf71b120 0x562cbf71d540 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66844b2b54b7b52 server_cookie=bbb9b66ecc0c56d6 in_seq=0 out_seq=0 2024-09-10T08:20:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.889+0000 7f1150c00640 10 client.28942 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1150c00640 1 -- 172.21.15.2:0/324596235 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562cbf606b80 con 0x562cbf71b120 2024-09-10T08:20:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1150c00640 10 client.28942 handle_command_reply: tid=0 2024-09-10T08:20:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1170cbd740 1 client.28942 shutdown 2024-09-10T08:20:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1170cbd740 1 -- 172.21.15.2:0/324596235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 msgr2=0x562cbf6a8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1170cbd740 1 --2- 172.21.15.2:0/324596235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a8d60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1158009b20 tx=0x7f11580045f0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1170cbd740 1 -- 172.21.15.2:0/324596235 shutdown_connections 2024-09-10T08:20:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1170cbd740 1 --2- 172.21.15.2:0/324596235 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562cbf71b120 0x562cbf71d540 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1170cbd740 1 --2- 172.21.15.2:0/324596235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cbf69fa60 0x562cbf6aa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1170cbd740 1 --2- 172.21.15.2:0/324596235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562cbf69a9d0 0x562cbf6a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1170cbd740 1 --2- 172.21.15.2:0/324596235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cbf69a090 0x562cbf6a8d60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1150c00640 0 client.28942 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1170cbd740 1 -- 172.21.15.2:0/324596235 >> 172.21.15.2:0/324596235 conn(0x562cbf695700 msgr2=0x562cbf694060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1170cbd740 1 -- 172.21.15.2:0/324596235 shutdown_connections 2024-09-10T08:20:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1170cbd740 1 -- 172.21.15.2:0/324596235 wait complete. 2024-09-10T08:20:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.902+0000 7f1170cbd740 20 client.28942 trim_cache size 0 max 16384 2024-09-10T08:20:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:20:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:20:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:20:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:20:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.903+0000 7f116de00640 1 -- 172.21.15.2:0/3007780341 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f114c0a2cb0 msgr2=0x7f114c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.903+0000 7f116de00640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f114c0a2cb0 0x7f114c0a5170 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1160011890 tx=0x7f1160009500 comp rx=0 tx=0).stop 2024-09-10T08:20:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.903+0000 7f116de00640 1 -- 172.21.15.2:0/3007780341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11680761d0 msgr2=0x7f116819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.903+0000 7f116de00640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11680761d0 0x7f116819a560 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1154009230 tx=0x7f11540042f0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.903+0000 7f116de00640 1 -- 172.21.15.2:0/3007780341 shutdown_connections 2024-09-10T08:20:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.903+0000 7f116de00640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f114c0a2cb0 0x7f114c0a5170 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.903+0000 7f116de00640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11681099f0 0x7f11681a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.903+0000 7f116de00640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11680761d0 0x7f116819a560 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.903+0000 7f116de00640 1 --2- 172.21.15.2:0/3007780341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1168074bf0 0x7f116819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.903+0000 7f116de00640 1 -- 172.21.15.2:0/3007780341 >> 172.21.15.2:0/3007780341 conn(0x7f11680fd2e0 msgr2=0x7f1168105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.903+0000 7f116de00640 1 -- 172.21.15.2:0/3007780341 shutdown_connections 2024-09-10T08:20:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:20:47.903+0000 7f116de00640 1 -- 172.21.15.2:0/3007780341 wait complete. 2024-09-10T08:20:47.909 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:47 smithi155 ceph-mon[35405]: pgmap v766: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-10T08:20:47.909 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:47 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/29931725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:20:47.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:47 smithi107 ceph-mon[35048]: pgmap v766: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-10T08:20:47.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:47 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/29931725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:20:48.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:47 smithi002 ceph-mon[30563]: pgmap v766: 129 pgs: 129 active+clean; 2.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-10T08:20:48.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:47 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/29931725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:20:48.216 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:20:48.595 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:49.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.230+0000 7f2d40200640 1 -- 172.21.15.2:0/3996204652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38104010 msgr2=0x7f2d381043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:49.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.230+0000 7f2d40200640 1 --2- 172.21.15.2:0/3996204652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38104010 0x7f2d381043f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009900 tx=0x7f2d2c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.231+0000 7f2d40200640 1 -- 172.21.15.2:0/3996204652 shutdown_connections 2024-09-10T08:20:49.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.231+0000 7f2d40200640 1 --2- 172.21.15.2:0/3996204652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d38109140 0x7f2d38110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.231+0000 7f2d40200640 1 --2- 172.21.15.2:0/3996204652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d381049c0 0x7f2d38108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.231+0000 7f2d40200640 1 --2- 172.21.15.2:0/3996204652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38104010 0x7f2d381043f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.231+0000 7f2d40200640 1 -- 172.21.15.2:0/3996204652 >> 172.21.15.2:0/3996204652 conn(0x7f2d380ffa50 msgr2=0x7f2d38101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:49.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.231+0000 7f2d40200640 1 -- 172.21.15.2:0/3996204652 shutdown_connections 2024-09-10T08:20:49.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.231+0000 7f2d40200640 1 -- 172.21.15.2:0/3996204652 wait complete. 2024-09-10T08:20:49.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.232+0000 7f2d40200640 1 Processor -- start 2024-09-10T08:20:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.232+0000 7f2d40200640 1 -- start start 2024-09-10T08:20:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.232+0000 7f2d40200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38104010 0x7f2d3819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.232+0000 7f2d40200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d381049c0 0x7f2d3819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.232+0000 7f2d40200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d38109140 0x7f2d3819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.232+0000 7f2d40200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2d38114160 con 0x7f2d38109140 2024-09-10T08:20:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.232+0000 7f2d40200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d38113fe0 con 0x7f2d38104010 2024-09-10T08:20:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.232+0000 7f2d40200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381142e0 con 0x7f2d381049c0 2024-09-10T08:20:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.233+0000 7f2d3d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38104010 0x7f2d3819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.233+0000 7f2d3d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38104010 0x7f2d3819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:51892/0 (socket says 172.21.15.2:51892) 2024-09-10T08:20:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.233+0000 7f2d3d800640 1 -- 172.21.15.2:0/162558968 learned_addr learned my addr 172.21.15.2:0/162558968 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:49.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.233+0000 7f2d3e200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d38109140 0x7f2d3819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.233+0000 7f2d3ce00640 1 --2- 172.21.15.2:0/162558968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d381049c0 0x7f2d3819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.233+0000 7f2d3d800640 1 -- 172.21.15.2:0/162558968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d381049c0 msgr2=0x7f2d3819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.233+0000 7f2d3d800640 1 --2- 172.21.15.2:0/162558968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d381049c0 0x7f2d3819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.233+0000 7f2d3d800640 1 -- 172.21.15.2:0/162558968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d38109140 msgr2=0x7f2d3819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.233+0000 7f2d3d800640 1 --2- 172.21.15.2:0/162558968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d38109140 0x7f2d3819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.233+0000 7f2d3d800640 1 -- 172.21.15.2:0/162558968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3819f5d0 con 0x7f2d38104010 2024-09-10T08:20:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.233+0000 7f2d3d800640 1 --2- 172.21.15.2:0/162558968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38104010 0x7f2d3819a620 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c006010 tx=0x7f2d2c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.234+0000 7f2d36000640 1 -- 172.21.15.2:0/162558968 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c03f030 con 0x7f2d38104010 2024-09-10T08:20:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.234+0000 7f2d40200640 1 -- 172.21.15.2:0/162558968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3819f860 con 0x7f2d38104010 2024-09-10T08:20:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.234+0000 7f2d36000640 1 -- 172.21.15.2:0/162558968 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2c004440 con 0x7f2d38104010 2024-09-10T08:20:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.234+0000 7f2d36000640 1 -- 172.21.15.2:0/162558968 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c035070 con 0x7f2d38104010 2024-09-10T08:20:49.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.234+0000 7f2d40200640 1 -- 172.21.15.2:0/162558968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d381a71a0 con 0x7f2d38104010 2024-09-10T08:20:49.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.237+0000 7f2d42e66740 1 Processor -- start 2024-09-10T08:20:49.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.237+0000 7f2d42e66740 1 -- start start 2024-09-10T08:20:49.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.237+0000 7f2d36000640 1 -- 172.21.15.2:0/162558968 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2d2c042030 con 0x7f2d38104010 2024-09-10T08:20:49.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.237+0000 7f2d42e66740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777cdc70 0x55b3777ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.237+0000 7f2d42e66740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777ce590 0x55b3777d8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.237+0000 7f2d42e66740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777d8e50 0x55b3777db240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d42e66740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b3777343e0 con 0x55b3777cdc70 2024-09-10T08:20:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d42e66740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b3776dbd80 con 0x55b3777d8e50 2024-09-10T08:20:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d42e66740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b3777303a0 con 0x55b3777ce590 2024-09-10T08:20:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d3e200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777cdc70 0x55b3777ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d3e200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777cdc70 0x55b3777ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44272/0 (socket says 172.21.15.2:44272) 2024-09-10T08:20:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d3e200640 1 -- 172.21.15.2:0/320705002 learned_addr learned my addr 172.21.15.2:0/320705002 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d3d800640 1 --2- 172.21.15.2:0/320705002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777ce590 0x55b3777d8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d3ce00640 1 --2- 172.21.15.2:0/320705002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777d8e50 0x55b3777db240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d3e200640 1 -- 172.21.15.2:0/320705002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777ce590 msgr2=0x55b3777d8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d3e200640 1 --2- 172.21.15.2:0/320705002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777ce590 0x55b3777d8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d3e200640 1 -- 172.21.15.2:0/320705002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777d8e50 msgr2=0x55b3777db240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d3e200640 1 --2- 172.21.15.2:0/320705002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777d8e50 0x55b3777db240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d3e200640 1 -- 172.21.15.2:0/320705002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3776faae0 con 0x55b3777cdc70 2024-09-10T08:20:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.238+0000 7f2d3d800640 1 --2- 172.21.15.2:0/320705002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777ce590 0x55b3777d8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.239+0000 7f2d3e200640 1 --2- 172.21.15.2:0/320705002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777cdc70 0x55b3777ce050 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009f50 tx=0x7f2d300118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f845412d34875a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:49.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.239+0000 7f2d36000640 1 --2- 172.21.15.2:0/162558968 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d200a2b80 0x7f2d200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.239+0000 7f2d2aa00640 1 -- 172.21.15.2:0/320705002 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30004d20 con 0x55b3777cdc70 2024-09-10T08:20:49.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.239+0000 7f2d2aa00640 1 -- 172.21.15.2:0/320705002 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d30004050 con 0x55b3777cdc70 2024-09-10T08:20:49.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.239+0000 7f2d2aa00640 1 -- 172.21.15.2:0/320705002 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30005280 con 0x55b3777cdc70 2024-09-10T08:20:49.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.239+0000 7f2d36000640 1 -- 172.21.15.2:0/162558968 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2d2c0ef7e0 con 0x7f2d38104010 2024-09-10T08:20:49.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.239+0000 7f2d3d800640 1 --2- 172.21.15.2:0/162558968 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d200a2b80 0x7f2d200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.240+0000 7f2d42e66740 1 -- 172.21.15.2:0/320705002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777cdc70 msgr2=0x55b3777ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:49.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.240+0000 7f2d42e66740 1 --2- 172.21.15.2:0/320705002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777cdc70 0x55b3777ce050 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009f50 tx=0x7f2d300118d0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.240+0000 7f2d3d800640 1 --2- 172.21.15.2:0/162558968 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d200a2b80 0x7f2d200a5040 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0c4330 tx=0x7f2d2c0f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.240+0000 7f2d42e66740 1 -- 172.21.15.2:0/320705002 shutdown_connections 2024-09-10T08:20:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.240+0000 7f2d42e66740 1 --2- 172.21.15.2:0/320705002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777d8e50 0x55b3777db240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.240+0000 7f2d42e66740 1 --2- 172.21.15.2:0/320705002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777ce590 0x55b3777d8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.240+0000 7f2d42e66740 1 --2- 172.21.15.2:0/320705002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777cdc70 0x55b3777ce050 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.240+0000 7f2d42e66740 1 -- 172.21.15.2:0/320705002 >> 172.21.15.2:0/320705002 conn(0x55b3777ca200 msgr2=0x55b3777ca5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.240+0000 7f2d42e66740 1 -- 172.21.15.2:0/320705002 shutdown_connections 2024-09-10T08:20:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.240+0000 7f2d42e66740 1 -- 172.21.15.2:0/320705002 wait complete. 2024-09-10T08:20:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.241+0000 7f2d42e66740 1 -- start start 2024-09-10T08:20:49.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.241+0000 7f2d42e66740 1 Processor -- start 2024-09-10T08:20:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.241+0000 7f2d42e66740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777cdc70 0x55b3777d5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.241+0000 7f2d42e66740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777ce590 0x55b3777d6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d42e66740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777d8e50 0x55b3777d67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d42e66740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b3777dba20 con 0x55b3777d8e50 2024-09-10T08:20:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d42e66740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b3776ef0f0 con 0x55b3777ce590 2024-09-10T08:20:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d42e66740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b3777dbba0 con 0x55b3777cdc70 2024-09-10T08:20:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d42e66740 10 client.0 _ll_register_callbacks cb 0x55b3776d6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d3ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777ce590 0x55b3777d6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d3ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777ce590 0x55b3777d6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:51908/0 (socket says 172.21.15.2:51908) 2024-09-10T08:20:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d3ce00640 1 -- 172.21.15.2:0/1738850917 learned_addr learned my addr 172.21.15.2:0/1738850917 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d3e200640 1 --2- 172.21.15.2:0/1738850917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777d8e50 0x55b3777d67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d3e200640 1 --2- 172.21.15.2:0/1738850917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777cdc70 0x55b3777d5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d3ce00640 1 -- 172.21.15.2:0/1738850917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777cdc70 msgr2=0x55b3777d5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d3ce00640 1 --2- 172.21.15.2:0/1738850917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777cdc70 0x55b3777d5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d3ce00640 1 -- 172.21.15.2:0/1738850917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777d8e50 msgr2=0x55b3777d67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d3ce00640 1 --2- 172.21.15.2:0/1738850917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777d8e50 0x55b3777d67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.242+0000 7f2d3ce00640 1 -- 172.21.15.2:0/1738850917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3774a53c0 con 0x55b3777ce590 2024-09-10T08:20:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.243+0000 7f2d3ce00640 1 --2- 172.21.15.2:0/1738850917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777ce590 0x55b3777d6270 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2d240126e0 tx=0x7f2d24012ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f832f068cfc30a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.243+0000 7f2d1f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:20:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.243+0000 7f2d1f400640 1 -- 172.21.15.2:0/1738850917 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2400f920 con 0x55b3777ce590 2024-09-10T08:20:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.243+0000 7f2d42e66740 1 -- 172.21.15.2:0/1738850917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d30004050 con 0x55b3777ce590 2024-09-10T08:20:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.243+0000 7f2d1f400640 1 -- 172.21.15.2:0/1738850917 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2d2400fac0 con 0x55b3777ce590 2024-09-10T08:20:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.243+0000 7f2d1f400640 1 -- 172.21.15.2:0/1738850917 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d2400fcc0 con 0x55b3777ce590 2024-09-10T08:20:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.243+0000 7f2d42e66740 10 client.29021 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.243+0000 7f2d42e66740 1 -- 172.21.15.2:0/1738850917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b3777e7b10 con 0x55b3777ce590 2024-09-10T08:20:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.243+0000 7f2d1f400640 1 -- 172.21.15.2:0/1738850917 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d30004050 con 0x55b3777ce590 2024-09-10T08:20:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.244+0000 7f2d1f400640 1 -- 172.21.15.2:0/1738850917 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2d2401c420 con 0x55b3777ce590 2024-09-10T08:20:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.244+0000 7f2d42e66740 10 client.29021 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.244+0000 7f2d42e66740 10 client.29021 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:20:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.244+0000 7f2d42e66740 20 client.29021 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.244+0000 7f2d42e66740 1 --2- 172.21.15.2:0/1738850917 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b3777e8fe0 0x55b3777eb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.244+0000 7f2d42e66740 4 client.29021 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.244+0000 7f2d42e66740 1 -- 172.21.15.2:0/1738850917 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b377741950 con 0x55b3777e8fe0 2024-09-10T08:20:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.244+0000 7f2d3e200640 1 --2- 172.21.15.2:0/1738850917 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b3777e8fe0 0x55b3777eb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.245+0000 7f2d3e200640 1 --2- 172.21.15.2:0/1738850917 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b3777e8fe0 0x55b3777eb400 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4480ab5da51b542 server_cookie=dab23a286b4afa06 in_seq=0 out_seq=0 2024-09-10T08:20:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.245+0000 7f2d1f400640 10 client.29021 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:20:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d1f400640 1 -- 172.21.15.2:0/1738850917 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b377741950 con 0x55b3777e8fe0 2024-09-10T08:20:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d1f400640 10 client.29021 handle_command_reply: tid=0 2024-09-10T08:20:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d42e66740 1 client.29021 shutdown 2024-09-10T08:20:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d42e66740 1 -- 172.21.15.2:0/1738850917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777ce590 msgr2=0x55b3777d6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d42e66740 1 --2- 172.21.15.2:0/1738850917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777ce590 0x55b3777d6270 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2d240126e0 tx=0x7f2d24012ba0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d42e66740 1 -- 172.21.15.2:0/1738850917 shutdown_connections 2024-09-10T08:20:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d42e66740 1 --2- 172.21.15.2:0/1738850917 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b3777e8fe0 0x55b3777eb400 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d42e66740 1 --2- 172.21.15.2:0/1738850917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777d8e50 0x55b3777d67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d42e66740 1 --2- 172.21.15.2:0/1738850917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777ce590 0x55b3777d6270 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d42e66740 1 --2- 172.21.15.2:0/1738850917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777cdc70 0x55b3777d5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d42e66740 1 -- 172.21.15.2:0/1738850917 >> 172.21.15.2:0/1738850917 conn(0x55b3777ca200 msgr2=0x55b3777c9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d1f400640 0 client.29021 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:20:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d42e66740 1 -- 172.21.15.2:0/1738850917 shutdown_connections 2024-09-10T08:20:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d42e66740 1 -- 172.21.15.2:0/1738850917 wait complete. 2024-09-10T08:20:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.246+0000 7f2d42e66740 20 client.29021 trim_cache size 0 max 16384 2024-09-10T08:20:49.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.260+0000 7f2d42e66740 1 Processor -- start 2024-09-10T08:20:49.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.260+0000 7f2d42e66740 1 -- start start 2024-09-10T08:20:49.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.260+0000 7f2d42e66740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777cdc70 0x55b3777f0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.260+0000 7f2d42e66740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777ce590 0x55b3777f1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.260+0000 7f2d42e66740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777d8e50 0x55b3777d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.260+0000 7f2d42e66740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381b3960 con 0x55b3777d8e50 2024-09-10T08:20:49.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.260+0000 7f2d42e66740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381b1230 con 0x55b3777ce590 2024-09-10T08:20:49.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.260+0000 7f2d42e66740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381af080 con 0x55b3777cdc70 2024-09-10T08:20:49.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.261+0000 7f2d3e200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777d8e50 0x55b3777d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.261+0000 7f2d3e200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777d8e50 0x55b3777d2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44302/0 (socket says 172.21.15.2:44302) 2024-09-10T08:20:49.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.261+0000 7f2d3ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777ce590 0x55b3777f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.261+0000 7f2d3e200640 1 -- 172.21.15.2:0/2361537797 learned_addr learned my addr 172.21.15.2:0/2361537797 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:49.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.261+0000 7f2d3e200640 1 --2- 172.21.15.2:0/2361537797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777cdc70 0x55b3777f0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.261+0000 7f2d3ce00640 1 -- 172.21.15.2:0/2361537797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777cdc70 msgr2=0x55b3777f0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:49.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.261+0000 7f2d3ce00640 1 --2- 172.21.15.2:0/2361537797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777cdc70 0x55b3777f0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.261+0000 7f2d3ce00640 1 -- 172.21.15.2:0/2361537797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777d8e50 msgr2=0x55b3777d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:49.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.261+0000 7f2d3ce00640 1 --2- 172.21.15.2:0/2361537797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777d8e50 0x55b3777d2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.261+0000 7f2d3ce00640 1 -- 172.21.15.2:0/2361537797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3777cae90 con 0x55b3777ce590 2024-09-10T08:20:49.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.261+0000 7f2d3ce00640 1 --2- 172.21.15.2:0/2361537797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777ce590 0x55b3777f1210 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2d2401ca80 tx=0x7f2d24009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2755482f8297d7e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.262+0000 7f2d2a000640 1 -- 172.21.15.2:0/2361537797 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d240056d0 con 0x55b3777ce590 2024-09-10T08:20:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.262+0000 7f2d2a000640 1 -- 172.21.15.2:0/2361537797 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2400f920 con 0x55b3777ce590 2024-09-10T08:20:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.262+0000 7f2d2a000640 1 -- 172.21.15.2:0/2361537797 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d240035f0 con 0x55b3777ce590 2024-09-10T08:20:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.262+0000 7f2d42e66740 1 -- 172.21.15.2:0/2361537797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777ce590 msgr2=0x55b3777f1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.262+0000 7f2d42e66740 1 --2- 172.21.15.2:0/2361537797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777ce590 0x55b3777f1210 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2d2401ca80 tx=0x7f2d24009e80 comp rx=0 tx=0).stop 2024-09-10T08:20:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.262+0000 7f2d42e66740 1 -- 172.21.15.2:0/2361537797 shutdown_connections 2024-09-10T08:20:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.262+0000 7f2d42e66740 1 --2- 172.21.15.2:0/2361537797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777d8e50 0x55b3777d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.262+0000 7f2d42e66740 1 --2- 172.21.15.2:0/2361537797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777ce590 0x55b3777f1210 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.262+0000 7f2d42e66740 1 --2- 172.21.15.2:0/2361537797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777cdc70 0x55b3777f0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.262+0000 7f2d42e66740 1 -- 172.21.15.2:0/2361537797 >> 172.21.15.2:0/2361537797 conn(0x55b3777ca200 msgr2=0x55b3777d7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.262+0000 7f2d42e66740 1 -- 172.21.15.2:0/2361537797 shutdown_connections 2024-09-10T08:20:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.263+0000 7f2d42e66740 1 -- 172.21.15.2:0/2361537797 wait complete. 2024-09-10T08:20:49.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.263+0000 7f2d42e66740 1 -- start start 2024-09-10T08:20:49.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.263+0000 7f2d42e66740 1 Processor -- start 2024-09-10T08:20:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.263+0000 7f2d42e66740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777cdc70 0x55b3777dcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.263+0000 7f2d42e66740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777ce590 0x55b3777dd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.263+0000 7f2d42e66740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777d8e50 0x55b3777dd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.263+0000 7f2d42e66740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381af6f0 con 0x55b3777ce590 2024-09-10T08:20:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.263+0000 7f2d42e66740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381b0b60 con 0x55b3777cdc70 2024-09-10T08:20:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.263+0000 7f2d42e66740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381afdc0 con 0x55b3777d8e50 2024-09-10T08:20:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.264+0000 7f2d42e66740 10 client.0 _ll_register_callbacks cb 0x7f2d24009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.264+0000 7f2d3ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777ce590 0x55b3777dd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.264+0000 7f2d3ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777ce590 0x55b3777dd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44318/0 (socket says 172.21.15.2:44318) 2024-09-10T08:20:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.264+0000 7f2d3e200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777cdc70 0x55b3777dcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.264+0000 7f2d3ce00640 1 -- 172.21.15.2:0/3975629867 learned_addr learned my addr 172.21.15.2:0/3975629867 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.264+0000 7f2d3e200640 1 --2- 172.21.15.2:0/3975629867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777d8e50 0x55b3777dd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.264+0000 7f2d3ce00640 1 -- 172.21.15.2:0/3975629867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777d8e50 msgr2=0x55b3777dd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.264+0000 7f2d3ce00640 1 --2- 172.21.15.2:0/3975629867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777d8e50 0x55b3777dd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.264+0000 7f2d3ce00640 1 -- 172.21.15.2:0/3975629867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777cdc70 msgr2=0x55b3777dcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.264+0000 7f2d3ce00640 1 --2- 172.21.15.2:0/3975629867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777cdc70 0x55b3777dcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.264+0000 7f2d3ce00640 1 -- 172.21.15.2:0/3975629867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b377852cd0 con 0x55b3777ce590 2024-09-10T08:20:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.264+0000 7f2d3e200640 1 --2- 172.21.15.2:0/3975629867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777cdc70 0x55b3777dcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.265+0000 7f2d3ce00640 1 --2- 172.21.15.2:0/3975629867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777ce590 0x55b3777dd0a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2d24000c00 tx=0x7f2d2400fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3d82c341b29b2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.265+0000 7f2d1f400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.265+0000 7f2d1f400640 1 -- 172.21.15.2:0/3975629867 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d240050c0 con 0x55b3777ce590 2024-09-10T08:20:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.265+0000 7f2d1f400640 1 -- 172.21.15.2:0/3975629867 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2d24005260 con 0x55b3777ce590 2024-09-10T08:20:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.265+0000 7f2d42e66740 1 -- 172.21.15.2:0/3975629867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d2400f920 con 0x55b3777ce590 2024-09-10T08:20:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.265+0000 7f2d1f400640 1 -- 172.21.15.2:0/3975629867 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d2403c050 con 0x55b3777ce590 2024-09-10T08:20:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.265+0000 7f2d42e66740 10 client.19002 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.265+0000 7f2d42e66740 1 -- 172.21.15.2:0/3975629867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b377853f90 con 0x55b3777ce590 2024-09-10T08:20:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.265+0000 7f2d1f400640 1 -- 172.21.15.2:0/3975629867 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2400f920 con 0x55b3777ce590 2024-09-10T08:20:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.266+0000 7f2d1f400640 1 -- 172.21.15.2:0/3975629867 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2d2402a5a0 con 0x55b3777ce590 2024-09-10T08:20:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.266+0000 7f2d42e66740 10 client.19002 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.266+0000 7f2d42e66740 10 client.19002 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:20:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.266+0000 7f2d42e66740 20 client.19002 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.266+0000 7f2d42e66740 1 --2- 172.21.15.2:0/3975629867 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b3778545f0 0x55b377856a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.266+0000 7f2d42e66740 4 client.19002 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:20:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.266+0000 7f2d42e66740 1 -- 172.21.15.2:0/3975629867 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b37773be60 con 0x55b3778545f0 2024-09-10T08:20:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.266+0000 7f2d3e200640 1 --2- 172.21.15.2:0/3975629867 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b3778545f0 0x55b377856a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d3e200640 1 --2- 172.21.15.2:0/3975629867 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b3778545f0 0x55b377856a10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b883a994b5d90e87 server_cookie=abefe050f32307f8 in_seq=0 out_seq=0 2024-09-10T08:20:49.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d1f400640 10 client.19002 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:20:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d1f400640 1 -- 172.21.15.2:0/3975629867 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b37773be60 con 0x55b3778545f0 2024-09-10T08:20:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d1f400640 10 client.19002 handle_command_reply: tid=0 2024-09-10T08:20:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d42e66740 1 client.19002 shutdown 2024-09-10T08:20:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d42e66740 1 -- 172.21.15.2:0/3975629867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777ce590 msgr2=0x55b3777dd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d42e66740 1 --2- 172.21.15.2:0/3975629867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777ce590 0x55b3777dd0a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2d24000c00 tx=0x7f2d2400fad0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d42e66740 1 -- 172.21.15.2:0/3975629867 shutdown_connections 2024-09-10T08:20:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d42e66740 1 --2- 172.21.15.2:0/3975629867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3777d8e50 0x55b3777dd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d42e66740 1 --2- 172.21.15.2:0/3975629867 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55b3778545f0 0x55b377856a10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d42e66740 1 --2- 172.21.15.2:0/3975629867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b3777ce590 0x55b3777dd0a0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d42e66740 1 --2- 172.21.15.2:0/3975629867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3777cdc70 0x55b3777dcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d42e66740 1 -- 172.21.15.2:0/3975629867 >> 172.21.15.2:0/3975629867 conn(0x55b3777ca200 msgr2=0x55b3777c8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d1f400640 0 client.19002 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:20:49.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d42e66740 1 -- 172.21.15.2:0/3975629867 shutdown_connections 2024-09-10T08:20:49.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.267+0000 7f2d42e66740 1 -- 172.21.15.2:0/3975629867 wait complete. 2024-09-10T08:20:49.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.268+0000 7f2d42e66740 20 client.19002 trim_cache size 0 max 16384 2024-09-10T08:20:49.272 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:20:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.268+0000 7f2d40200640 1 -- 172.21.15.2:0/162558968 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d200a2b80 msgr2=0x7f2d200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.268+0000 7f2d40200640 1 --2- 172.21.15.2:0/162558968 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d200a2b80 0x7f2d200a5040 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0c4330 tx=0x7f2d2c0f5530 comp rx=0 tx=0).stop 2024-09-10T08:20:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.268+0000 7f2d40200640 1 -- 172.21.15.2:0/162558968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38104010 msgr2=0x7f2d3819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.268+0000 7f2d40200640 1 --2- 172.21.15.2:0/162558968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38104010 0x7f2d3819a620 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c006010 tx=0x7f2d2c004040 comp rx=0 tx=0).stop 2024-09-10T08:20:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.268+0000 7f2d40200640 1 -- 172.21.15.2:0/162558968 shutdown_connections 2024-09-10T08:20:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.268+0000 7f2d40200640 1 --2- 172.21.15.2:0/162558968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d38109140 0x7f2d3819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.268+0000 7f2d40200640 1 --2- 172.21.15.2:0/162558968 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2d200a2b80 0x7f2d200a5040 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.268+0000 7f2d40200640 1 --2- 172.21.15.2:0/162558968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d381049c0 0x7f2d3819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.268+0000 7f2d40200640 1 --2- 172.21.15.2:0/162558968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d38104010 0x7f2d3819a620 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.268+0000 7f2d40200640 1 -- 172.21.15.2:0/162558968 >> 172.21.15.2:0/162558968 conn(0x7f2d380ffa50 msgr2=0x7f2d38100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:49.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.268+0000 7f2d40200640 1 -- 172.21.15.2:0/162558968 shutdown_connections 2024-09-10T08:20:49.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:49.269+0000 7f2d40200640 1 -- 172.21.15.2:0/162558968 wait complete. 2024-09-10T08:20:49.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:49 smithi155 ceph-mon[35405]: pgmap v767: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-09-10T08:20:49.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:49 smithi107 ceph-mon[35048]: pgmap v767: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-09-10T08:20:50.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:49 smithi002 ceph-mon[30563]: pgmap v767: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-09-10T08:20:50.191 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:20:50.571 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:51.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.164+0000 7f5258000640 1 -- 172.21.15.2:0/3233567885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f525010e3c0 msgr2=0x7f5250110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:51.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.164+0000 7f5258000640 1 --2- 172.21.15.2:0/3233567885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f525010e3c0 0x7f5250110810 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5248009be0 tx=0x7f524802a750 comp rx=0 tx=0).stop 2024-09-10T08:20:51.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.164+0000 7f5258000640 1 -- 172.21.15.2:0/3233567885 shutdown_connections 2024-09-10T08:20:51.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.164+0000 7f5258000640 1 --2- 172.21.15.2:0/3233567885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f525010e3c0 0x7f5250110810 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.164+0000 7f5258000640 1 --2- 172.21.15.2:0/3233567885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5250074f40 0x7f525010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.164+0000 7f5258000640 1 --2- 172.21.15.2:0/3233567885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5250074620 0x7f5250074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.164+0000 7f5258000640 1 -- 172.21.15.2:0/3233567885 >> 172.21.15.2:0/3233567885 conn(0x7f52500fd8c0 msgr2=0x7f52500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:51.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.164+0000 7f5258000640 1 -- 172.21.15.2:0/3233567885 shutdown_connections 2024-09-10T08:20:51.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.164+0000 7f5258000640 1 -- 172.21.15.2:0/3233567885 wait complete. 2024-09-10T08:20:51.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.165+0000 7f5258000640 1 Processor -- start 2024-09-10T08:20:51.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.165+0000 7f5258000640 1 -- start start 2024-09-10T08:20:51.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.165+0000 7f5258000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5250074620 0x7f525019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.165+0000 7f5258000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5250074f40 0x7f525019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.165+0000 7f5258000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f525010e3c0 0x7f525019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.165+0000 7f5258000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5250114160 con 0x7f5250074f40 2024-09-10T08:20:51.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.165+0000 7f5258000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5250113fe0 con 0x7f525010e3c0 2024-09-10T08:20:51.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.165+0000 7f5258000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52501142e0 con 0x7f5250074620 2024-09-10T08:20:51.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.166+0000 7f5255600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5250074620 0x7f525019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.166+0000 7f5254c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5250074f40 0x7f525019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.166+0000 7f5255600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5250074620 0x7f525019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:47080/0 (socket says 172.21.15.2:47080) 2024-09-10T08:20:51.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.166+0000 7f5255600640 1 -- 172.21.15.2:0/158326471 learned_addr learned my addr 172.21.15.2:0/158326471 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:51.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.166+0000 7f5256000640 1 --2- 172.21.15.2:0/158326471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f525010e3c0 0x7f525019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.166+0000 7f5255600640 1 -- 172.21.15.2:0/158326471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f525010e3c0 msgr2=0x7f525019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:51.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.166+0000 7f5255600640 1 --2- 172.21.15.2:0/158326471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f525010e3c0 0x7f525019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.166+0000 7f5255600640 1 -- 172.21.15.2:0/158326471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5250074f40 msgr2=0x7f525019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:51.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.166+0000 7f5255600640 1 --2- 172.21.15.2:0/158326471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5250074f40 0x7f525019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.166+0000 7f5255600640 1 -- 172.21.15.2:0/158326471 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525019f5d0 con 0x7f5250074620 2024-09-10T08:20:51.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.167+0000 7f5255600640 1 --2- 172.21.15.2:0/158326471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5250074620 0x7f525019a620 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f524400e9f0 tx=0x7f524400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:51.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.167+0000 7f524e000640 1 -- 172.21.15.2:0/158326471 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524400cd80 con 0x7f5250074620 2024-09-10T08:20:51.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.167+0000 7f5258000640 1 -- 172.21.15.2:0/158326471 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525019f8c0 con 0x7f5250074620 2024-09-10T08:20:51.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.167+0000 7f524e000640 1 -- 172.21.15.2:0/158326471 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5244005500 con 0x7f5250074620 2024-09-10T08:20:51.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.167+0000 7f524e000640 1 -- 172.21.15.2:0/158326471 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244010690 con 0x7f5250074620 2024-09-10T08:20:51.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.167+0000 7f5258000640 1 -- 172.21.15.2:0/158326471 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52501a71a0 con 0x7f5250074620 2024-09-10T08:20:51.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.170+0000 7f525aac1740 1 Processor -- start 2024-09-10T08:20:51.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.170+0000 7f525aac1740 1 -- start start 2024-09-10T08:20:51.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.170+0000 7f525aac1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d31331050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.170+0000 7f525aac1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d31331590 0x561d3133b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.170+0000 7f524e000640 1 -- 172.21.15.2:0/158326471 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f52440046c0 con 0x7f5250074620 2024-09-10T08:20:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.170+0000 7f525aac1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d3133be50 0x561d3133e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f525aac1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561d312973e0 con 0x561d31330c70 2024-09-10T08:20:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f525aac1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561d3123ed80 con 0x561d3133be50 2024-09-10T08:20:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f525aac1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561d312933a0 con 0x561d31331590 2024-09-10T08:20:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f5256000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d31331050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f5256000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d31331050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44330/0 (socket says 172.21.15.2:44330) 2024-09-10T08:20:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f5256000640 1 -- 172.21.15.2:0/3759319340 learned_addr learned my addr 172.21.15.2:0/3759319340 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f5255600640 1 --2- 172.21.15.2:0/3759319340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d31331590 0x561d3133b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f5254c00640 1 --2- 172.21.15.2:0/3759319340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d3133be50 0x561d3133e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f5256000640 1 -- 172.21.15.2:0/3759319340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d31331590 msgr2=0x561d3133b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f5256000640 1 --2- 172.21.15.2:0/3759319340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d31331590 0x561d3133b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f5256000640 1 -- 172.21.15.2:0/3759319340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d3133be50 msgr2=0x561d3133e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f5256000640 1 --2- 172.21.15.2:0/3759319340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d3133be50 0x561d3133e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f5256000640 1 -- 172.21.15.2:0/3759319340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d3125dae0 con 0x561d31330c70 2024-09-10T08:20:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.171+0000 7f5254c00640 1 --2- 172.21.15.2:0/3759319340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d3133be50 0x561d3133e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:20:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.172+0000 7f5256000640 1 --2- 172.21.15.2:0/3759319340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d31331050 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5248009be0 tx=0x7f5248030aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9ea8aa0c2d134e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.172+0000 7f5242600640 1 -- 172.21.15.2:0/3759319340 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5248041030 con 0x561d31330c70 2024-09-10T08:20:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.172+0000 7f524e000640 1 --2- 172.21.15.2:0/158326471 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52380a2b80 0x7f52380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.172+0000 7f5242600640 1 -- 172.21.15.2:0/3759319340 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5248030c50 con 0x561d31330c70 2024-09-10T08:20:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.172+0000 7f5242600640 1 -- 172.21.15.2:0/3759319340 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5248036040 con 0x561d31330c70 2024-09-10T08:20:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.172+0000 7f524e000640 1 -- 172.21.15.2:0/158326471 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f52440cf870 con 0x7f5250074620 2024-09-10T08:20:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.172+0000 7f5255600640 1 --2- 172.21.15.2:0/158326471 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52380a2b80 0x7f52380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.173+0000 7f525aac1740 1 -- 172.21.15.2:0/3759319340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 msgr2=0x561d31331050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.173+0000 7f525aac1740 1 --2- 172.21.15.2:0/3759319340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d31331050 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5248009be0 tx=0x7f5248030aa0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.173+0000 7f525aac1740 1 -- 172.21.15.2:0/3759319340 shutdown_connections 2024-09-10T08:20:51.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.173+0000 7f525aac1740 1 --2- 172.21.15.2:0/3759319340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d3133be50 0x561d3133e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.173+0000 7f525aac1740 1 --2- 172.21.15.2:0/3759319340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d31331590 0x561d3133b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.173+0000 7f525aac1740 1 --2- 172.21.15.2:0/3759319340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d31331050 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.173+0000 7f525aac1740 1 -- 172.21.15.2:0/3759319340 >> 172.21.15.2:0/3759319340 conn(0x561d3132d200 msgr2=0x561d3132d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:51.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.173+0000 7f5255600640 1 --2- 172.21.15.2:0/158326471 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52380a2b80 0x7f52380a5040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f52440980e0 tx=0x7f52440d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:51.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.173+0000 7f525aac1740 1 -- 172.21.15.2:0/3759319340 shutdown_connections 2024-09-10T08:20:51.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.173+0000 7f525aac1740 1 -- 172.21.15.2:0/3759319340 wait complete. 2024-09-10T08:20:51.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.174+0000 7f525aac1740 1 -- start start 2024-09-10T08:20:51.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.174+0000 7f525aac1740 1 Processor -- start 2024-09-10T08:20:51.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f525aac1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d31338d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f525aac1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d31331590 0x561d31339260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f525aac1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d3133be50 0x561d313397a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f525aac1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561d3133ea20 con 0x561d31330c70 2024-09-10T08:20:51.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f525aac1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561d312520f0 con 0x561d3133be50 2024-09-10T08:20:51.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f525aac1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561d3133eba0 con 0x561d31331590 2024-09-10T08:20:51.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f525aac1740 10 client.0 _ll_register_callbacks cb 0x561d31239bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:51.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f5254c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d31331590 0x561d31339260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f5256000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d31338d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f5254c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d31331590 0x561d31339260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:47104/0 (socket says 172.21.15.2:47104) 2024-09-10T08:20:51.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f5256000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d31338d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44340/0 (socket says 172.21.15.2:44340) 2024-09-10T08:20:51.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f5254c00640 1 -- 172.21.15.2:0/3866844604 learned_addr learned my addr 172.21.15.2:0/3866844604 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:51.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f5256000640 1 --2- 172.21.15.2:0/3866844604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d3133be50 0x561d313397a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f5254c00640 1 -- 172.21.15.2:0/3866844604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d3133be50 msgr2=0x561d313397a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:51.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f5254c00640 1 --2- 172.21.15.2:0/3866844604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d3133be50 0x561d313397a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f5254c00640 1 -- 172.21.15.2:0/3866844604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 msgr2=0x561d31338d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:51.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f5254c00640 1 --2- 172.21.15.2:0/3866844604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d31338d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.175+0000 7f5254c00640 1 -- 172.21.15.2:0/3866844604 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d310083c0 con 0x561d31331590 2024-09-10T08:20:51.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.176+0000 7f5254c00640 1 --2- 172.21.15.2:0/3866844604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d31331590 0x561d31339260 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f523c011720 tx=0x7f523c011be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c54ae8e25cd65230 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:51.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.176+0000 7f5236a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:20:51.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.176+0000 7f5236a00640 1 -- 172.21.15.2:0/3866844604 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c00fbb0 con 0x561d31331590 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.176+0000 7f5236a00640 1 -- 172.21.15.2:0/3866844604 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f523c00fd50 con 0x561d31331590 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.176+0000 7f525aac1740 1 -- 172.21.15.2:0/3866844604 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5248030c50 con 0x561d31331590 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.176+0000 7f5236a00640 1 -- 172.21.15.2:0/3866844604 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f523c027050 con 0x561d31331590 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.176+0000 7f525aac1740 10 client.28972 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.176+0000 7f525aac1740 1 -- 172.21.15.2:0/3866844604 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561d3134aaa0 con 0x561d31331590 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.177+0000 7f5236a00640 1 -- 172.21.15.2:0/3866844604 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5248030c50 con 0x561d31331590 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.177+0000 7f5236a00640 1 -- 172.21.15.2:0/3866844604 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f523c022030 con 0x561d31331590 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.177+0000 7f525aac1740 10 client.28972 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.177+0000 7f525aac1740 10 client.28972 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.177+0000 7f525aac1740 20 client.28972 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.177+0000 7f525aac1740 1 --2- 172.21.15.2:0/3866844604 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x561d3134bf60 0x561d3134e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.177+0000 7f525aac1740 4 client.28972 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.177+0000 7f525aac1740 1 -- 172.21.15.2:0/3866844604 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d312a4950 con 0x561d3134bf60 2024-09-10T08:20:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.177+0000 7f5256000640 1 --2- 172.21.15.2:0/3866844604 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x561d3134bf60 0x561d3134e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.178+0000 7f5256000640 1 --2- 172.21.15.2:0/3866844604 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x561d3134bf60 0x561d3134e3e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22225c1c73119a67 server_cookie=1308a8fdeefb2899 in_seq=0 out_seq=0 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.178+0000 7f5236a00640 10 client.28972 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.179+0000 7f5236a00640 1 -- 172.21.15.2:0/3866844604 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d312a4950 con 0x561d3134bf60 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.179+0000 7f5236a00640 10 client.28972 handle_command_reply: tid=0 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.179+0000 7f525aac1740 1 client.28972 shutdown 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.179+0000 7f525aac1740 1 -- 172.21.15.2:0/3866844604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d31331590 msgr2=0x561d31339260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.179+0000 7f525aac1740 1 --2- 172.21.15.2:0/3866844604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d31331590 0x561d31339260 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f523c011720 tx=0x7f523c011be0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.179+0000 7f525aac1740 1 -- 172.21.15.2:0/3866844604 shutdown_connections 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.179+0000 7f525aac1740 1 --2- 172.21.15.2:0/3866844604 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x561d3134bf60 0x561d3134e3e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.179+0000 7f525aac1740 1 --2- 172.21.15.2:0/3866844604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d3133be50 0x561d313397a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.179+0000 7f525aac1740 1 --2- 172.21.15.2:0/3866844604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d31331590 0x561d31339260 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.179+0000 7f5236a00640 0 client.28972 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.179+0000 7f525aac1740 1 --2- 172.21.15.2:0/3866844604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d31338d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.179+0000 7f525aac1740 1 -- 172.21.15.2:0/3866844604 >> 172.21.15.2:0/3866844604 conn(0x561d3132d200 msgr2=0x561d3132c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:51.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.179+0000 7f525aac1740 1 -- 172.21.15.2:0/3866844604 shutdown_connections 2024-09-10T08:20:51.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.180+0000 7f525aac1740 1 -- 172.21.15.2:0/3866844604 wait complete. 2024-09-10T08:20:51.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.180+0000 7f525aac1740 20 client.28972 trim_cache size 0 max 16384 2024-09-10T08:20:51.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.198+0000 7f525aac1740 1 Processor -- start 2024-09-10T08:20:51.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.198+0000 7f525aac1740 1 -- start start 2024-09-10T08:20:51.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.199+0000 7f525aac1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d3133ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.199+0000 7f525aac1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 0x561d31353cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.199+0000 7f525aac1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d3133be50 0x561d313541f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.199+0000 7f525aac1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52501b3890 con 0x561d31330c70 2024-09-10T08:20:51.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.199+0000 7f525aac1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f52501b1160 con 0x561d31331590 2024-09-10T08:20:51.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.199+0000 7f525aac1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52501aefb0 con 0x561d3133be50 2024-09-10T08:20:51.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.199+0000 7f5254c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 0x561d31353cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.199+0000 7f5254c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 0x561d31353cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:51974/0 (socket says 172.21.15.2:51974) 2024-09-10T08:20:51.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.199+0000 7f5254c00640 1 -- 172.21.15.2:0/1179968914 learned_addr learned my addr 172.21.15.2:0/1179968914 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:51.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.199+0000 7f5256000640 1 --2- 172.21.15.2:0/1179968914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d3133ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.200+0000 7f5256000640 1 --2- 172.21.15.2:0/1179968914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d3133be50 0x561d313541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.200+0000 7f5254c00640 1 -- 172.21.15.2:0/1179968914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d3133be50 msgr2=0x561d313541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:51.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.200+0000 7f5254c00640 1 --2- 172.21.15.2:0/1179968914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d3133be50 0x561d313541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.200+0000 7f5254c00640 1 -- 172.21.15.2:0/1179968914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 msgr2=0x561d3133ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:51.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.200+0000 7f5254c00640 1 --2- 172.21.15.2:0/1179968914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d3133ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.200+0000 7f5254c00640 1 -- 172.21.15.2:0/1179968914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d31333ae0 con 0x561d31331590 2024-09-10T08:20:51.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.200+0000 7f5256000640 1 --2- 172.21.15.2:0/1179968914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d3133ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:20:51.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.200+0000 7f5254c00640 1 --2- 172.21.15.2:0/1179968914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 0x561d31353cb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f523c009530 tx=0x7f523c019390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51779e22c280f948 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:51.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.200+0000 7f5241c00640 1 -- 172.21.15.2:0/1179968914 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c00f790 con 0x561d31331590 2024-09-10T08:20:51.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.200+0000 7f5241c00640 1 -- 172.21.15.2:0/1179968914 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f523c030070 con 0x561d31331590 2024-09-10T08:20:51.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.201+0000 7f5241c00640 1 -- 172.21.15.2:0/1179968914 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c019a70 con 0x561d31331590 2024-09-10T08:20:51.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.201+0000 7f525aac1740 1 -- 172.21.15.2:0/1179968914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 msgr2=0x561d31353cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:51.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.201+0000 7f525aac1740 1 --2- 172.21.15.2:0/1179968914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 0x561d31353cb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f523c009530 tx=0x7f523c019390 comp rx=0 tx=0).stop 2024-09-10T08:20:51.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.201+0000 7f525aac1740 1 -- 172.21.15.2:0/1179968914 shutdown_connections 2024-09-10T08:20:51.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.201+0000 7f525aac1740 1 --2- 172.21.15.2:0/1179968914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d3133be50 0x561d313541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.201+0000 7f525aac1740 1 --2- 172.21.15.2:0/1179968914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 0x561d31353cb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.201+0000 7f525aac1740 1 --2- 172.21.15.2:0/1179968914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d3133ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.201+0000 7f525aac1740 1 -- 172.21.15.2:0/1179968914 >> 172.21.15.2:0/1179968914 conn(0x561d3132d200 msgr2=0x561d3133a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:51.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.202+0000 7f525aac1740 1 -- 172.21.15.2:0/1179968914 shutdown_connections 2024-09-10T08:20:51.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.202+0000 7f525aac1740 1 -- 172.21.15.2:0/1179968914 wait complete. 2024-09-10T08:20:51.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.202+0000 7f525aac1740 1 -- start start 2024-09-10T08:20:51.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.202+0000 7f525aac1740 1 Processor -- start 2024-09-10T08:20:51.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.203+0000 7f525aac1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d3133fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.203+0000 7f525aac1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 0x561d313400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.203+0000 7f525aac1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d3133be50 0x561d313405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.203+0000 7f525aac1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52501af620 con 0x561d31330c70 2024-09-10T08:20:51.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.203+0000 7f525aac1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f52501b0a90 con 0x561d31331590 2024-09-10T08:20:51.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.203+0000 7f525aac1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52501afcf0 con 0x561d3133be50 2024-09-10T08:20:51.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.203+0000 7f525aac1740 10 client.0 _ll_register_callbacks cb 0x7f523c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:51.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.203+0000 7f5254c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 0x561d313400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.203+0000 7f5256000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d3133fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.203+0000 7f5254c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 0x561d313400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:51990/0 (socket says 172.21.15.2:51990) 2024-09-10T08:20:51.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.203+0000 7f5254c00640 1 -- 172.21.15.2:0/4004238697 learned_addr learned my addr 172.21.15.2:0/4004238697 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:51.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.203+0000 7f5256000640 1 --2- 172.21.15.2:0/4004238697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d3133be50 0x561d313405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.204+0000 7f5254c00640 1 -- 172.21.15.2:0/4004238697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d3133be50 msgr2=0x561d313405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:51.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.204+0000 7f5254c00640 1 --2- 172.21.15.2:0/4004238697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d3133be50 0x561d313405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.204+0000 7f5254c00640 1 -- 172.21.15.2:0/4004238697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 msgr2=0x561d3133fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:51.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.204+0000 7f5254c00640 1 --2- 172.21.15.2:0/4004238697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d3133fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.204+0000 7f5254c00640 1 -- 172.21.15.2:0/4004238697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d313be4c0 con 0x561d31331590 2024-09-10T08:20:51.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.204+0000 7f5256000640 1 --2- 172.21.15.2:0/4004238697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d3133be50 0x561d313405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:20:51.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.204+0000 7f5254c00640 1 --2- 172.21.15.2:0/4004238697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 0x561d313400a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f523c011720 tx=0x7f523c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94d39655cb19db10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:51.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.204+0000 7f5236a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:20:51.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.204+0000 7f5236a00640 1 -- 172.21.15.2:0/4004238697 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c03a070 con 0x561d31331590 2024-09-10T08:20:51.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.204+0000 7f525aac1740 1 -- 172.21.15.2:0/4004238697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f523c030070 con 0x561d31331590 2024-09-10T08:20:51.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.204+0000 7f5236a00640 1 -- 172.21.15.2:0/4004238697 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f523c019be0 con 0x561d31331590 2024-09-10T08:20:51.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.205+0000 7f5236a00640 1 -- 172.21.15.2:0/4004238697 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f523c043050 con 0x561d31331590 2024-09-10T08:20:51.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.205+0000 7f525aac1740 10 client.29045 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:51.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.205+0000 7f525aac1740 1 -- 172.21.15.2:0/4004238697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561d313bf780 con 0x561d31331590 2024-09-10T08:20:51.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.205+0000 7f5236a00640 1 -- 172.21.15.2:0/4004238697 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f523c030070 con 0x561d31331590 2024-09-10T08:20:51.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.205+0000 7f5236a00640 1 -- 172.21.15.2:0/4004238697 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f523c03d020 con 0x561d31331590 2024-09-10T08:20:51.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.205+0000 7f525aac1740 10 client.29045 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:51.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.205+0000 7f525aac1740 10 client.29045 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:20:51.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.205+0000 7f525aac1740 20 client.29045 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:51.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.205+0000 7f525aac1740 1 --2- 172.21.15.2:0/4004238697 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x561d313bfeb0 0x561d313c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:51.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.205+0000 7f525aac1740 4 client.29045 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:20:51.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.205+0000 7f525aac1740 1 -- 172.21.15.2:0/4004238697 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d3129ee60 con 0x561d313bfeb0 2024-09-10T08:20:51.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.206+0000 7f5256000640 1 --2- 172.21.15.2:0/4004238697 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x561d313bfeb0 0x561d313c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:51.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.206+0000 7f5256000640 1 --2- 172.21.15.2:0/4004238697 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x561d313bfeb0 0x561d313c22d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b41d786b252d00c server_cookie=40318c8ce93fce8e in_seq=0 out_seq=0 2024-09-10T08:20:51.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.206+0000 7f5236a00640 10 client.29045 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:20:51.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f5236a00640 1 -- 172.21.15.2:0/4004238697 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d3129ee60 con 0x561d313bfeb0 2024-09-10T08:20:51.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f5236a00640 10 client.29045 handle_command_reply: tid=0 2024-09-10T08:20:51.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f525aac1740 1 client.29045 shutdown 2024-09-10T08:20:51.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f525aac1740 1 -- 172.21.15.2:0/4004238697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 msgr2=0x561d313400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:51.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f525aac1740 1 --2- 172.21.15.2:0/4004238697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 0x561d313400a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f523c011720 tx=0x7f523c009560 comp rx=0 tx=0).stop 2024-09-10T08:20:51.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f525aac1740 1 -- 172.21.15.2:0/4004238697 shutdown_connections 2024-09-10T08:20:51.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f525aac1740 1 --2- 172.21.15.2:0/4004238697 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x561d313bfeb0 0x561d313c22d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f525aac1740 1 --2- 172.21.15.2:0/4004238697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d3133be50 0x561d313405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f525aac1740 1 --2- 172.21.15.2:0/4004238697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d31331590 0x561d313400a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f525aac1740 1 --2- 172.21.15.2:0/4004238697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561d31330c70 0x561d3133fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f5236a00640 0 client.29045 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:20:51.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f525aac1740 1 -- 172.21.15.2:0/4004238697 >> 172.21.15.2:0/4004238697 conn(0x561d3132d200 msgr2=0x561d313326c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:51.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f525aac1740 1 -- 172.21.15.2:0/4004238697 shutdown_connections 2024-09-10T08:20:51.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f525aac1740 1 -- 172.21.15.2:0/4004238697 wait complete. 2024-09-10T08:20:51.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.207+0000 7f525aac1740 20 client.29045 trim_cache size 0 max 16384 2024-09-10T08:20:51.213 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:20:51.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.208+0000 7f5258000640 1 -- 172.21.15.2:0/158326471 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52380a2b80 msgr2=0x7f52380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:51.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.208+0000 7f5258000640 1 --2- 172.21.15.2:0/158326471 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52380a2b80 0x7f52380a5040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f52440980e0 tx=0x7f52440d5530 comp rx=0 tx=0).stop 2024-09-10T08:20:51.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.208+0000 7f5258000640 1 -- 172.21.15.2:0/158326471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5250074620 msgr2=0x7f525019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:51.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.208+0000 7f5258000640 1 --2- 172.21.15.2:0/158326471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5250074620 0x7f525019a620 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f524400e9f0 tx=0x7f524400edf0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.208+0000 7f5258000640 1 -- 172.21.15.2:0/158326471 shutdown_connections 2024-09-10T08:20:51.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.208+0000 7f5258000640 1 --2- 172.21.15.2:0/158326471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f525010e3c0 0x7f525019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.208+0000 7f5258000640 1 --2- 172.21.15.2:0/158326471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5250074f40 0x7f525019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.208+0000 7f5258000640 1 --2- 172.21.15.2:0/158326471 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f52380a2b80 0x7f52380a5040 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.208+0000 7f5258000640 1 --2- 172.21.15.2:0/158326471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5250074620 0x7f525019a620 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:51.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.208+0000 7f5258000640 1 -- 172.21.15.2:0/158326471 >> 172.21.15.2:0/158326471 conn(0x7f52500fd8c0 msgr2=0x7f52500fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:51.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.208+0000 7f5258000640 1 -- 172.21.15.2:0/158326471 shutdown_connections 2024-09-10T08:20:51.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:51.209+0000 7f5258000640 1 -- 172.21.15.2:0/158326471 wait complete. 2024-09-10T08:20:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:51 smithi155 ceph-mon[35405]: pgmap v768: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-10T08:20:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:51 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:20:51.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:51 smithi107 ceph-mon[35048]: pgmap v768: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-10T08:20:51.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:51 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:20:52.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:51 smithi002 ceph-mon[30563]: pgmap v768: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-10T08:20:52.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:51 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:20:52.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:20:52.586 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:52.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:52 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:20:52.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:52 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:20:52.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:52 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:20:53.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:52 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:20:53.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:52 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:20:53.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:52 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:20:53.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:52 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:20:53.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:52 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:20:53.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:52 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:20:53.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.168+0000 7fc700c00640 1 -- 172.21.15.2:0/46603896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6fc1034b0 msgr2=0x7fc6fc10b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.168+0000 7fc700c00640 1 --2- 172.21.15.2:0/46603896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6fc1034b0 0x7fc6fc10b9a0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009900 tx=0x7fc6e402a480 comp rx=0 tx=0).stop 2024-09-10T08:20:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.168+0000 7fc700c00640 1 -- 172.21.15.2:0/46603896 shutdown_connections 2024-09-10T08:20:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.168+0000 7fc700c00640 1 --2- 172.21.15.2:0/46603896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6fc10bfc0 0x7fc6fc10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.168+0000 7fc700c00640 1 --2- 172.21.15.2:0/46603896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6fc1034b0 0x7fc6fc10b9a0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.168+0000 7fc700c00640 1 --2- 172.21.15.2:0/46603896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6fc102b90 0x7fc6fc102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.168+0000 7fc700c00640 1 -- 172.21.15.2:0/46603896 >> 172.21.15.2:0/46603896 conn(0x7fc6fc0fc610 msgr2=0x7fc6fc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.168+0000 7fc700c00640 1 -- 172.21.15.2:0/46603896 shutdown_connections 2024-09-10T08:20:53.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.168+0000 7fc700c00640 1 -- 172.21.15.2:0/46603896 wait complete. 2024-09-10T08:20:53.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.169+0000 7fc700c00640 1 Processor -- start 2024-09-10T08:20:53.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.169+0000 7fc700c00640 1 -- start start 2024-09-10T08:20:53.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.170+0000 7fc700c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6fc102b90 0x7fc6fc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.170+0000 7fc700c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6fc1034b0 0x7fc6fc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.170+0000 7fc700c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6fc10bfc0 0x7fc6fc19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.170+0000 7fc700c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc111dc0 con 0x7fc6fc1034b0 2024-09-10T08:20:53.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.170+0000 7fc700c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc111c40 con 0x7fc6fc10bfc0 2024-09-10T08:20:53.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.170+0000 7fc700c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc111f40 con 0x7fc6fc102b90 2024-09-10T08:20:53.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.170+0000 7fc6f2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6fc1034b0 0x7fc6fc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.170+0000 7fc6f2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6fc1034b0 0x7fc6fc19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44382/0 (socket says 172.21.15.2:44382) 2024-09-10T08:20:53.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.170+0000 7fc6f2a00640 1 -- 172.21.15.2:0/2307679937 learned_addr learned my addr 172.21.15.2:0/2307679937 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:53.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.170+0000 7fc6fb400640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6fc102b90 0x7fc6fc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.170+0000 7fc6fbe00640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6fc10bfc0 0x7fc6fc19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.171+0000 7fc6fb400640 1 -- 172.21.15.2:0/2307679937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6fc10bfc0 msgr2=0x7fc6fc19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:53.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.171+0000 7fc6fb400640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6fc10bfc0 0x7fc6fc19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.171+0000 7fc6fb400640 1 -- 172.21.15.2:0/2307679937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6fc1034b0 msgr2=0x7fc6fc19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:53.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.171+0000 7fc6fb400640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6fc1034b0 0x7fc6fc19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.171+0000 7fc6fb400640 1 -- 172.21.15.2:0/2307679937 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6fc19f4d0 con 0x7fc6fc102b90 2024-09-10T08:20:53.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.171+0000 7fc6fb400640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6fc102b90 0x7fc6fc19a520 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00e9f0 tx=0x7fc6ec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:53.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.171+0000 7fc6f8c00640 1 -- 172.21.15.2:0/2307679937 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec00ccc0 con 0x7fc6fc102b90 2024-09-10T08:20:53.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.171+0000 7fc700c00640 1 -- 172.21.15.2:0/2307679937 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6fc19f7c0 con 0x7fc6fc102b90 2024-09-10T08:20:53.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.172+0000 7fc700c00640 1 -- 172.21.15.2:0/2307679937 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6fc1a70a0 con 0x7fc6fc102b90 2024-09-10T08:20:53.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.172+0000 7fc6f8c00640 1 -- 172.21.15.2:0/2307679937 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ec010910 con 0x7fc6fc102b90 2024-09-10T08:20:53.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.172+0000 7fc6f8c00640 1 -- 172.21.15.2:0/2307679937 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec018880 con 0x7fc6fc102b90 2024-09-10T08:20:53.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.174+0000 7fc704c83740 1 Processor -- start 2024-09-10T08:20:53.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.175+0000 7fc704c83740 1 -- start start 2024-09-10T08:20:53.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.175+0000 7fc704c83740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c1e40 0x55d8269c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.175+0000 7fc704c83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c2760 0x55d8269ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.175+0000 7fc704c83740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269cd020 0x55d8269cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.175+0000 7fc704c83740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d8269283e0 con 0x55d8269cd020 2024-09-10T08:20:53.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.175+0000 7fc704c83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d8268cfd80 con 0x55d8269c2760 2024-09-10T08:20:53.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.175+0000 7fc704c83740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d8269243a0 con 0x55d8269c1e40 2024-09-10T08:20:53.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.175+0000 7fc6f8c00640 1 -- 172.21.15.2:0/2307679937 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc6ec010430 con 0x7fc6fc102b90 2024-09-10T08:20:53.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.175+0000 7fc6f2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269cd020 0x55d8269cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.175+0000 7fc6fb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c2760 0x55d8269ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.175+0000 7fc6fbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c1e40 0x55d8269c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.175+0000 7fc6fbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c1e40 0x55d8269c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:47136/0 (socket says 172.21.15.2:47136) 2024-09-10T08:20:53.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.175+0000 7fc6fbe00640 1 -- 172.21.15.2:0/1622610876 learned_addr learned my addr 172.21.15.2:0/1622610876 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:53.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.176+0000 7fc6f2a00640 1 -- 172.21.15.2:0/1622610876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c1e40 msgr2=0x55d8269c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:53.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.176+0000 7fc6f2a00640 1 --2- 172.21.15.2:0/1622610876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c1e40 0x55d8269c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.176+0000 7fc6f2a00640 1 -- 172.21.15.2:0/1622610876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c2760 msgr2=0x55d8269ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:53.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.176+0000 7fc6f2a00640 1 --2- 172.21.15.2:0/1622610876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c2760 0x55d8269ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.176+0000 7fc6f2a00640 1 -- 172.21.15.2:0/1622610876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8268eeae0 con 0x55d8269cd020 2024-09-10T08:20:53.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.176+0000 7fc6f2a00640 1 --2- 172.21.15.2:0/1622610876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269cd020 0x55d8269cf410 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009230 tx=0x7fc6e4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98f9c5680873637 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:53.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.176+0000 7fc6ebe00640 1 -- 172.21.15.2:0/1622610876 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4040070 con 0x55d8269cd020 2024-09-10T08:20:53.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.176+0000 7fc6ebe00640 1 -- 172.21.15.2:0/1622610876 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e4004030 con 0x55d8269cd020 2024-09-10T08:20:53.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.176+0000 7fc6ebe00640 1 -- 172.21.15.2:0/1622610876 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4035040 con 0x55d8269cd020 2024-09-10T08:20:53.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.177+0000 7fc6f8c00640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6e00a2c00 0x7fc6e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.177+0000 7fc6f8c00640 1 -- 172.21.15.2:0/2307679937 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc6ec0cf670 con 0x7fc6fc102b90 2024-09-10T08:20:53.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.177+0000 7fc6fbe00640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6e00a2c00 0x7fc6e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.177+0000 7fc704c83740 1 -- 172.21.15.2:0/1622610876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269cd020 msgr2=0x55d8269cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:53.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.177+0000 7fc704c83740 1 --2- 172.21.15.2:0/1622610876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269cd020 0x55d8269cf410 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009230 tx=0x7fc6e4005dd0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.177+0000 7fc704c83740 1 -- 172.21.15.2:0/1622610876 shutdown_connections 2024-09-10T08:20:53.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.177+0000 7fc704c83740 1 --2- 172.21.15.2:0/1622610876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269cd020 0x55d8269cf410 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.177+0000 7fc704c83740 1 --2- 172.21.15.2:0/1622610876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c2760 0x55d8269ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.177+0000 7fc704c83740 1 --2- 172.21.15.2:0/1622610876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c1e40 0x55d8269c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.177+0000 7fc704c83740 1 -- 172.21.15.2:0/1622610876 >> 172.21.15.2:0/1622610876 conn(0x55d8269be3d0 msgr2=0x55d8269be7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.178+0000 7fc704c83740 1 -- 172.21.15.2:0/1622610876 shutdown_connections 2024-09-10T08:20:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.178+0000 7fc704c83740 1 -- 172.21.15.2:0/1622610876 wait complete. 2024-09-10T08:20:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.178+0000 7fc6fbe00640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6e00a2c00 0x7fc6e00a50c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4011630 tx=0x7fc6f4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.178+0000 7fc704c83740 1 -- start start 2024-09-10T08:20:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.178+0000 7fc704c83740 1 Processor -- start 2024-09-10T08:20:53.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.179+0000 7fc704c83740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c1e40 0x55d8269c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.179+0000 7fc704c83740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 0x55d8269ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.179+0000 7fc704c83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269cd020 0x55d8269ca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.179+0000 7fc704c83740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d8269cfbf0 con 0x55d8269c2760 2024-09-10T08:20:53.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.179+0000 7fc704c83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d8268e30f0 con 0x55d8269cd020 2024-09-10T08:20:53.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.179+0000 7fc704c83740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d8269cfd70 con 0x55d8269c1e40 2024-09-10T08:20:53.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.179+0000 7fc704c83740 10 client.0 _ll_register_callbacks cb 0x55d8268cabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:53.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.179+0000 7fc6f2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 0x55d8269ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.179+0000 7fc6f2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 0x55d8269ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44392/0 (socket says 172.21.15.2:44392) 2024-09-10T08:20:53.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.179+0000 7fc6f2a00640 1 -- 172.21.15.2:0/2675623892 learned_addr learned my addr 172.21.15.2:0/2675623892 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:53.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.179+0000 7fc6fb400640 1 --2- 172.21.15.2:0/2675623892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c1e40 0x55d8269c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.179+0000 7fc6fbe00640 1 --2- 172.21.15.2:0/2675623892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269cd020 0x55d8269ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.180+0000 7fc6f2a00640 1 -- 172.21.15.2:0/2675623892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c1e40 msgr2=0x55d8269c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:53.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.180+0000 7fc6f2a00640 1 --2- 172.21.15.2:0/2675623892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c1e40 0x55d8269c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.180+0000 7fc6f2a00640 1 -- 172.21.15.2:0/2675623892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269cd020 msgr2=0x55d8269ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.180+0000 7fc6f2a00640 1 --2- 172.21.15.2:0/2675623892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269cd020 0x55d8269ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.180+0000 7fc6f2a00640 1 -- 172.21.15.2:0/2675623892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8266993c0 con 0x55d8269c2760 2024-09-10T08:20:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.180+0000 7fc6f2a00640 1 --2- 172.21.15.2:0/2675623892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 0x55d8269ca3e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4031550 tx=0x7fc6e4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e242de0fa24bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.180+0000 7fc6e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.180+0000 7fc6e8c00640 1 -- 172.21.15.2:0/2675623892 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4052060 con 0x55d8269c2760 2024-09-10T08:20:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.180+0000 7fc6e8c00640 1 -- 172.21.15.2:0/2675623892 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc6e4005550 con 0x55d8269c2760 2024-09-10T08:20:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.180+0000 7fc704c83740 1 -- 172.21.15.2:0/2675623892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e4004030 con 0x55d8269c2760 2024-09-10T08:20:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.181+0000 7fc6e8c00640 1 -- 172.21.15.2:0/2675623892 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc6e4059050 con 0x55d8269c2760 2024-09-10T08:20:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.181+0000 7fc704c83740 10 client.19032 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.181+0000 7fc704c83740 1 -- 172.21.15.2:0/2675623892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d8269dbc20 con 0x55d8269c2760 2024-09-10T08:20:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.181+0000 7fc6e8c00640 1 -- 172.21.15.2:0/2675623892 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e4004030 con 0x55d8269c2760 2024-09-10T08:20:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.181+0000 7fc6e8c00640 1 -- 172.21.15.2:0/2675623892 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc6e404c070 con 0x55d8269c2760 2024-09-10T08:20:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.181+0000 7fc704c83740 10 client.19032 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.181+0000 7fc704c83740 10 client.19032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.181+0000 7fc704c83740 20 client.19032 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.181+0000 7fc704c83740 1 --2- 172.21.15.2:0/2675623892 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d8269dcf70 0x55d8269df360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.181+0000 7fc704c83740 4 client.19032 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.181+0000 7fc704c83740 1 -- 172.21.15.2:0/2675623892 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d826935950 con 0x55d8269dcf70 2024-09-10T08:20:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.182+0000 7fc6fb400640 1 --2- 172.21.15.2:0/2675623892 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d8269dcf70 0x55d8269df360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.182+0000 7fc6fb400640 1 --2- 172.21.15.2:0/2675623892 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d8269dcf70 0x55d8269df360 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642acf605c7fc0bd server_cookie=810e9550c4a1aab1 in_seq=0 out_seq=0 2024-09-10T08:20:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.182+0000 7fc6e8c00640 10 client.19032 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:53.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.183+0000 7fc6e8c00640 1 -- 172.21.15.2:0/2675623892 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d826935950 con 0x55d8269dcf70 2024-09-10T08:20:53.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.183+0000 7fc6e8c00640 10 client.19032 handle_command_reply: tid=0 2024-09-10T08:20:53.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.183+0000 7fc704c83740 1 client.19032 shutdown 2024-09-10T08:20:53.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.183+0000 7fc704c83740 1 -- 172.21.15.2:0/2675623892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 msgr2=0x55d8269ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:53.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.183+0000 7fc704c83740 1 --2- 172.21.15.2:0/2675623892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 0x55d8269ca3e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4031550 tx=0x7fc6e4009680 comp rx=0 tx=0).stop 2024-09-10T08:20:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.183+0000 7fc704c83740 1 -- 172.21.15.2:0/2675623892 shutdown_connections 2024-09-10T08:20:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.183+0000 7fc704c83740 1 --2- 172.21.15.2:0/2675623892 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d8269dcf70 0x55d8269df360 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.183+0000 7fc704c83740 1 --2- 172.21.15.2:0/2675623892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269cd020 0x55d8269ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.183+0000 7fc704c83740 1 --2- 172.21.15.2:0/2675623892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 0x55d8269ca3e0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.183+0000 7fc6e8c00640 0 client.19032 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.183+0000 7fc704c83740 1 --2- 172.21.15.2:0/2675623892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c1e40 0x55d8269c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.183+0000 7fc704c83740 1 -- 172.21.15.2:0/2675623892 >> 172.21.15.2:0/2675623892 conn(0x55d8269be3d0 msgr2=0x55d8269c40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.183+0000 7fc704c83740 1 -- 172.21.15.2:0/2675623892 shutdown_connections 2024-09-10T08:20:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.184+0000 7fc704c83740 1 -- 172.21.15.2:0/2675623892 wait complete. 2024-09-10T08:20:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.184+0000 7fc704c83740 20 client.19032 trim_cache size 0 max 16384 2024-09-10T08:20:53.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.197+0000 7fc704c83740 1 Processor -- start 2024-09-10T08:20:53.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.197+0000 7fc704c83740 1 -- start start 2024-09-10T08:20:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc704c83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c1e40 0x55d8269c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc704c83740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 0x55d8269cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc704c83740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269cd020 0x55d8269e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc704c83740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d80125e0 con 0x55d8269c2760 2024-09-10T08:20:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc704c83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d800feb0 con 0x55d8269c1e40 2024-09-10T08:20:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc704c83740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d800dd00 con 0x55d8269cd020 2024-09-10T08:20:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc6f2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 0x55d8269cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc6f2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 0x55d8269cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44398/0 (socket says 172.21.15.2:44398) 2024-09-10T08:20:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc6f2a00640 1 -- 172.21.15.2:0/3646752650 learned_addr learned my addr 172.21.15.2:0/3646752650 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc6fb400640 1 --2- 172.21.15.2:0/3646752650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c1e40 0x55d8269c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc6fbe00640 1 --2- 172.21.15.2:0/3646752650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269cd020 0x55d8269e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc6f2a00640 1 -- 172.21.15.2:0/3646752650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269cd020 msgr2=0x55d8269e49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc6f2a00640 1 --2- 172.21.15.2:0/3646752650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269cd020 0x55d8269e49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.198+0000 7fc6f2a00640 1 -- 172.21.15.2:0/3646752650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c1e40 msgr2=0x55d8269c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.199+0000 7fc6f2a00640 1 --2- 172.21.15.2:0/3646752650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c1e40 0x55d8269c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.199+0000 7fc6f2a00640 1 -- 172.21.15.2:0/3646752650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8269be200 con 0x55d8269c2760 2024-09-10T08:20:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.199+0000 7fc6f2a00640 1 --2- 172.21.15.2:0/3646752650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 0x55d8269cbe10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc6e402a4c0 tx=0x7fc6e4004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cfc23988723c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:53.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.199+0000 7fc6eb400640 1 -- 172.21.15.2:0/3646752650 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4052070 con 0x55d8269c2760 2024-09-10T08:20:53.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.199+0000 7fc6eb400640 1 -- 172.21.15.2:0/3646752650 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e40035f0 con 0x55d8269c2760 2024-09-10T08:20:53.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.199+0000 7fc6eb400640 1 -- 172.21.15.2:0/3646752650 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4040050 con 0x55d8269c2760 2024-09-10T08:20:53.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.200+0000 7fc704c83740 1 -- 172.21.15.2:0/3646752650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 msgr2=0x55d8269cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:53.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.200+0000 7fc704c83740 1 --2- 172.21.15.2:0/3646752650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 0x55d8269cbe10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc6e402a4c0 tx=0x7fc6e4004950 comp rx=0 tx=0).stop 2024-09-10T08:20:53.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.200+0000 7fc704c83740 1 -- 172.21.15.2:0/3646752650 shutdown_connections 2024-09-10T08:20:53.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.200+0000 7fc704c83740 1 --2- 172.21.15.2:0/3646752650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269cd020 0x55d8269e49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.200+0000 7fc704c83740 1 --2- 172.21.15.2:0/3646752650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269c2760 0x55d8269cbe10 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.200+0000 7fc704c83740 1 --2- 172.21.15.2:0/3646752650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c1e40 0x55d8269c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.200+0000 7fc704c83740 1 -- 172.21.15.2:0/3646752650 >> 172.21.15.2:0/3646752650 conn(0x55d8269be3d0 msgr2=0x55d8269cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:53.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.200+0000 7fc704c83740 1 -- 172.21.15.2:0/3646752650 shutdown_connections 2024-09-10T08:20:53.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.200+0000 7fc704c83740 1 -- 172.21.15.2:0/3646752650 wait complete. 2024-09-10T08:20:53.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.200+0000 7fc704c83740 1 -- start start 2024-09-10T08:20:53.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.201+0000 7fc704c83740 1 Processor -- start 2024-09-10T08:20:53.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.201+0000 7fc704c83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c1e40 0x55d8269d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.201+0000 7fc704c83740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c2760 0x55d8269d57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.201+0000 7fc704c83740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269cd020 0x55d8269d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.201+0000 7fc704c83740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d800e370 con 0x55d8269cd020 2024-09-10T08:20:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.201+0000 7fc704c83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d800f7e0 con 0x55d8269c1e40 2024-09-10T08:20:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.201+0000 7fc704c83740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d800ea40 con 0x55d8269c2760 2024-09-10T08:20:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.201+0000 7fc704c83740 10 client.0 _ll_register_callbacks cb 0x7fc6ec0d5060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.201+0000 7fc6fbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269cd020 0x55d8269d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.201+0000 7fc6fbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269cd020 0x55d8269d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44406/0 (socket says 172.21.15.2:44406) 2024-09-10T08:20:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.201+0000 7fc6fbe00640 1 -- 172.21.15.2:0/1674186810 learned_addr learned my addr 172.21.15.2:0/1674186810 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.201+0000 7fc6fb400640 1 --2- 172.21.15.2:0/1674186810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c1e40 0x55d8269d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.202+0000 7fc6f2a00640 1 --2- 172.21.15.2:0/1674186810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c2760 0x55d8269d57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.202+0000 7fc6fbe00640 1 -- 172.21.15.2:0/1674186810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c2760 msgr2=0x55d8269d57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.202+0000 7fc6fbe00640 1 --2- 172.21.15.2:0/1674186810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c2760 0x55d8269d57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.202+0000 7fc6fbe00640 1 -- 172.21.15.2:0/1674186810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c1e40 msgr2=0x55d8269d52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:53.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.202+0000 7fc6fbe00640 1 --2- 172.21.15.2:0/1674186810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c1e40 0x55d8269d52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.202+0000 7fc6fbe00640 1 -- 172.21.15.2:0/1674186810 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8269bdc80 con 0x55d8269cd020 2024-09-10T08:20:53.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.202+0000 7fc6fbe00640 1 --2- 172.21.15.2:0/1674186810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269cd020 0x55d8269d0040 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4003fa0 tx=0x7fc6f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48afc961f6dead7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:53.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.202+0000 7fc6e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:53.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc6e8c00640 1 -- 172.21.15.2:0/1674186810 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f402d060 con 0x55d8269cd020 2024-09-10T08:20:53.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc6e8c00640 1 -- 172.21.15.2:0/1674186810 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc6f400f8f0 con 0x55d8269cd020 2024-09-10T08:20:53.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc704c83740 1 -- 172.21.15.2:0/1674186810 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e40035f0 con 0x55d8269cd020 2024-09-10T08:20:53.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc6e8c00640 1 -- 172.21.15.2:0/1674186810 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc6f4034050 con 0x55d8269cd020 2024-09-10T08:20:53.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc704c83740 10 client.19038 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:53.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc704c83740 1 -- 172.21.15.2:0/1674186810 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d826a483b0 con 0x55d8269cd020 2024-09-10T08:20:53.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc6e8c00640 1 -- 172.21.15.2:0/1674186810 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e40035f0 con 0x55d8269cd020 2024-09-10T08:20:53.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc6e8c00640 1 -- 172.21.15.2:0/1674186810 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc6f4028420 con 0x55d8269cd020 2024-09-10T08:20:53.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc704c83740 10 client.19038 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:53.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc704c83740 10 client.19038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:53.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc704c83740 20 client.19038 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc704c83740 1 --2- 172.21.15.2:0/1674186810 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d826a48b60 0x55d826a4af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc704c83740 4 client.19038 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:20:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.203+0000 7fc704c83740 1 -- 172.21.15.2:0/1674186810 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d82692fe60 con 0x55d826a48b60 2024-09-10T08:20:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.204+0000 7fc6fb400640 1 --2- 172.21.15.2:0/1674186810 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d826a48b60 0x55d826a4af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.204+0000 7fc6fb400640 1 --2- 172.21.15.2:0/1674186810 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d826a48b60 0x55d826a4af80 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d1eddf79686bce server_cookie=d241254d1cee976a in_seq=0 out_seq=0 2024-09-10T08:20:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.204+0000 7fc6e8c00640 10 client.19038 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.204+0000 7fc6e8c00640 1 -- 172.21.15.2:0/1674186810 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d82692fe60 con 0x55d826a48b60 2024-09-10T08:20:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.204+0000 7fc6e8c00640 10 client.19038 handle_command_reply: tid=0 2024-09-10T08:20:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.204+0000 7fc704c83740 1 client.19038 shutdown 2024-09-10T08:20:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.205+0000 7fc704c83740 1 -- 172.21.15.2:0/1674186810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269cd020 msgr2=0x55d8269d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.205+0000 7fc704c83740 1 --2- 172.21.15.2:0/1674186810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269cd020 0x55d8269d0040 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4003fa0 tx=0x7fc6f4004a40 comp rx=0 tx=0).stop 2024-09-10T08:20:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.205+0000 7fc704c83740 1 -- 172.21.15.2:0/1674186810 shutdown_connections 2024-09-10T08:20:53.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.205+0000 7fc704c83740 1 --2- 172.21.15.2:0/1674186810 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d826a48b60 0x55d826a4af80 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.205+0000 7fc704c83740 1 --2- 172.21.15.2:0/1674186810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d8269cd020 0x55d8269d0040 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.205+0000 7fc704c83740 1 --2- 172.21.15.2:0/1674186810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8269c2760 0x55d8269d57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.205+0000 7fc704c83740 1 --2- 172.21.15.2:0/1674186810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8269c1e40 0x55d8269d52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.205+0000 7fc6e8c00640 0 client.19038 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:53.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.205+0000 7fc704c83740 1 -- 172.21.15.2:0/1674186810 >> 172.21.15.2:0/1674186810 conn(0x55d8269be3d0 msgr2=0x55d8269d86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:53.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.205+0000 7fc704c83740 1 -- 172.21.15.2:0/1674186810 shutdown_connections 2024-09-10T08:20:53.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.205+0000 7fc704c83740 1 -- 172.21.15.2:0/1674186810 wait complete. 2024-09-10T08:20:53.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.205+0000 7fc704c83740 20 client.19038 trim_cache size 0 max 16384 2024-09-10T08:20:53.210 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:20:53.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.206+0000 7fc700c00640 1 -- 172.21.15.2:0/2307679937 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6e00a2c00 msgr2=0x7fc6e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:53.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.206+0000 7fc700c00640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6e00a2c00 0x7fc6e00a50c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4011630 tx=0x7fc6f4009500 comp rx=0 tx=0).stop 2024-09-10T08:20:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.206+0000 7fc700c00640 1 -- 172.21.15.2:0/2307679937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6fc102b90 msgr2=0x7fc6fc19a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.206+0000 7fc700c00640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6fc102b90 0x7fc6fc19a520 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00e9f0 tx=0x7fc6ec00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.206+0000 7fc700c00640 1 -- 172.21.15.2:0/2307679937 shutdown_connections 2024-09-10T08:20:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.206+0000 7fc700c00640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6fc10bfc0 0x7fc6fc19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.206+0000 7fc700c00640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6fc1034b0 0x7fc6fc19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.206+0000 7fc700c00640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc6e00a2c00 0x7fc6e00a50c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.206+0000 7fc700c00640 1 --2- 172.21.15.2:0/2307679937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6fc102b90 0x7fc6fc19a520 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.206+0000 7fc700c00640 1 -- 172.21.15.2:0/2307679937 >> 172.21.15.2:0/2307679937 conn(0x7fc6fc0fc610 msgr2=0x7fc6fc0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.206+0000 7fc700c00640 1 -- 172.21.15.2:0/2307679937 shutdown_connections 2024-09-10T08:20:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:53.206+0000 7fc700c00640 1 -- 172.21.15.2:0/2307679937 wait complete. 2024-09-10T08:20:53.920 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:53 smithi107 ceph-mon[35048]: pgmap v769: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-10T08:20:54.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:53 smithi002 ceph-mon[30563]: pgmap v769: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-10T08:20:54.058 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:53 smithi155 ceph-mon[35405]: pgmap v769: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-10T08:20:55.101 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:20:55.101 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:20:55.494 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:55.888 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:55 smithi002 ceph-mon[30563]: pgmap v770: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:20:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:55 smithi107 ceph-mon[35048]: pgmap v770: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:20:56.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.146+0000 7f00be400640 1 -- 172.21.15.2:0/909542760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00b8106f10 msgr2=0x7f00b810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:56.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.146+0000 7f00be400640 1 --2- 172.21.15.2:0/909542760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00b8106f10 0x7f00b810ea90 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009ca0 tx=0x7f00b002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.146+0000 7f00be400640 1 -- 172.21.15.2:0/909542760 shutdown_connections 2024-09-10T08:20:56.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.146+0000 7f00be400640 1 --2- 172.21.15.2:0/909542760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00b8106f10 0x7f00b810ea90 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.146+0000 7f00be400640 1 --2- 172.21.15.2:0/909542760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8102790 0x7f00b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.146+0000 7f00be400640 1 --2- 172.21.15.2:0/909542760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b8101de0 0x7f00b81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.146+0000 7f00be400640 1 -- 172.21.15.2:0/909542760 >> 172.21.15.2:0/909542760 conn(0x7f00b80fd8c0 msgr2=0x7f00b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:56.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.146+0000 7f00be400640 1 -- 172.21.15.2:0/909542760 shutdown_connections 2024-09-10T08:20:56.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.146+0000 7f00be400640 1 -- 172.21.15.2:0/909542760 wait complete. 2024-09-10T08:20:56.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.147+0000 7f00be400640 1 Processor -- start 2024-09-10T08:20:56.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.147+0000 7f00be400640 1 -- start start 2024-09-10T08:20:56.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.148+0000 7f00be400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00b8101de0 0x7f00b8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.148+0000 7f00be400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b8102790 0x7f00b8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.148+0000 7f00be400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8106f10 0x7f00b819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.148+0000 7f00be400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8111fc0 con 0x7f00b8101de0 2024-09-10T08:20:56.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.148+0000 7f00be400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8111e40 con 0x7f00b8106f10 2024-09-10T08:20:56.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.148+0000 7f00be400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8112140 con 0x7f00b8102790 2024-09-10T08:20:56.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.148+0000 7f00b7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00b8101de0 0x7f00b8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.148+0000 7f00b7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00b8101de0 0x7f00b8198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44436/0 (socket says 172.21.15.2:44436) 2024-09-10T08:20:56.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.148+0000 7f00b7400640 1 -- 172.21.15.2:0/3621361489 learned_addr learned my addr 172.21.15.2:0/3621361489 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:56.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.148+0000 7f00b7e00640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8106f10 0x7f00b819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.148+0000 7f00afe00640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b8102790 0x7f00b8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.149+0000 7f00b7400640 1 -- 172.21.15.2:0/3621361489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b8102790 msgr2=0x7f00b8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:56.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.149+0000 7f00b7400640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b8102790 0x7f00b8198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.149+0000 7f00b7400640 1 -- 172.21.15.2:0/3621361489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8106f10 msgr2=0x7f00b819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:56.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.149+0000 7f00b7400640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8106f10 0x7f00b819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.149+0000 7f00b7400640 1 -- 172.21.15.2:0/3621361489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b819d3d0 con 0x7f00b8101de0 2024-09-10T08:20:56.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.149+0000 7f00b7400640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00b8101de0 0x7f00b8198420 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f00a800e9f0 tx=0x7f00a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:56.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.149+0000 7f00b4c00640 1 -- 172.21.15.2:0/3621361489 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a800cd80 con 0x7f00b8101de0 2024-09-10T08:20:56.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.149+0000 7f00b4c00640 1 -- 172.21.15.2:0/3621361489 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a8005500 con 0x7f00b8101de0 2024-09-10T08:20:56.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.149+0000 7f00be400640 1 -- 172.21.15.2:0/3621361489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b819d6c0 con 0x7f00b8101de0 2024-09-10T08:20:56.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.149+0000 7f00be400640 1 -- 172.21.15.2:0/3621361489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b81a4fa0 con 0x7f00b8101de0 2024-09-10T08:20:56.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.150+0000 7f00b4c00640 1 -- 172.21.15.2:0/3621361489 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8010690 con 0x7f00b8101de0 2024-09-10T08:20:56.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.154+0000 7f00c0e23740 1 Processor -- start 2024-09-10T08:20:56.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.154+0000 7f00b4c00640 1 -- 172.21.15.2:0/3621361489 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f00a80046c0 con 0x7f00b8101de0 2024-09-10T08:20:56.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.154+0000 7f00c0e23740 1 -- start start 2024-09-10T08:20:56.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.154+0000 7f00c0e23740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b44ac0 0x5654e4b44ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.154+0000 7f00c0e23740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b4f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.154+0000 7f00c0e23740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b4fca0 0x5654e4b52090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.154+0000 7f00c0e23740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5654e4aab290 con 0x5654e4b4fca0 2024-09-10T08:20:56.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.154+0000 7f00c0e23740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5654e4a52b40 con 0x5654e4b44ac0 2024-09-10T08:20:56.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.154+0000 7f00c0e23740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5654e4aa7250 con 0x5654e4b453e0 2024-09-10T08:20:56.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.155+0000 7f00b7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b4f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.155+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b44ac0 0x5654e4b44ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.155+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b44ac0 0x5654e4b44ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52102/0 (socket says 172.21.15.2:52102) 2024-09-10T08:20:56.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.155+0000 7f00b7e00640 1 -- 172.21.15.2:0/861920909 learned_addr learned my addr 172.21.15.2:0/861920909 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:56.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.155+0000 7f00afe00640 1 --2- 172.21.15.2:0/861920909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b4fca0 0x5654e4b52090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.155+0000 7f00b7e00640 1 -- 172.21.15.2:0/861920909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 msgr2=0x5654e4b4f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.155+0000 7f00b7e00640 1 --2- 172.21.15.2:0/861920909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b4f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.155+0000 7f00b7e00640 1 -- 172.21.15.2:0/861920909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b4fca0 msgr2=0x5654e4b52090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.155+0000 7f00b7e00640 1 --2- 172.21.15.2:0/861920909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b4fca0 0x5654e4b52090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.155+0000 7f00b7e00640 1 -- 172.21.15.2:0/861920909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654e481c6a0 con 0x5654e4b44ac0 2024-09-10T08:20:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.156+0000 7f00b7e00640 1 --2- 172.21.15.2:0/861920909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b44ac0 0x5654e4b44ea0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009ca0 tx=0x7f00b00093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92adccc75609aa2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.156+0000 7f00acc00640 1 -- 172.21.15.2:0/861920909 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0031db0 con 0x5654e4b44ac0 2024-09-10T08:20:56.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.156+0000 7f00acc00640 1 -- 172.21.15.2:0/861920909 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b0036030 con 0x5654e4b44ac0 2024-09-10T08:20:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.156+0000 7f00acc00640 1 -- 172.21.15.2:0/861920909 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b00042d0 con 0x5654e4b44ac0 2024-09-10T08:20:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.156+0000 7f00b4c00640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f009c0a2b80 0x7f009c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.156+0000 7f00b4c00640 1 -- 172.21.15.2:0/3621361489 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f00a80cf860 con 0x7f00b8101de0 2024-09-10T08:20:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.156+0000 7f00b7400640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f009c0a2b80 0x7f009c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.157+0000 7f00c0e23740 1 -- 172.21.15.2:0/861920909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b44ac0 msgr2=0x5654e4b44ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.157+0000 7f00c0e23740 1 --2- 172.21.15.2:0/861920909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b44ac0 0x5654e4b44ea0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009ca0 tx=0x7f00b00093e0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.157+0000 7f00c0e23740 1 -- 172.21.15.2:0/861920909 shutdown_connections 2024-09-10T08:20:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.157+0000 7f00c0e23740 1 --2- 172.21.15.2:0/861920909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b4fca0 0x5654e4b52090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.157+0000 7f00c0e23740 1 --2- 172.21.15.2:0/861920909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b4f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.157+0000 7f00c0e23740 1 --2- 172.21.15.2:0/861920909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b44ac0 0x5654e4b44ea0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.157+0000 7f00c0e23740 1 -- 172.21.15.2:0/861920909 >> 172.21.15.2:0/861920909 conn(0x5654e4b41050 msgr2=0x5654e4b41430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:56.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.157+0000 7f00c0e23740 1 -- 172.21.15.2:0/861920909 shutdown_connections 2024-09-10T08:20:56.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.157+0000 7f00c0e23740 1 -- 172.21.15.2:0/861920909 wait complete. 2024-09-10T08:20:56.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.157+0000 7f00b7400640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f009c0a2b80 0x7f009c0a5040 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f00a80980d0 tx=0x7f00a80d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:56.159 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:55 smithi155 ceph-mon[35405]: pgmap v770: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-10T08:20:56.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.158+0000 7f00c0e23740 1 -- start start 2024-09-10T08:20:56.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.158+0000 7f00c0e23740 1 Processor -- start 2024-09-10T08:20:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.158+0000 7f00c0e23740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 0x5654e4b53d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.158+0000 7f00c0e23740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.158+0000 7f00c0e23740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b4fca0 0x5654e4b58430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.158+0000 7f00c0e23740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5654e4b52790 con 0x5654e4b44ac0 2024-09-10T08:20:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.158+0000 7f00c0e23740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5654e4a65fa0 con 0x5654e4b4fca0 2024-09-10T08:20:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.158+0000 7f00c0e23740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5654e4b52910 con 0x5654e4b453e0 2024-09-10T08:20:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.158+0000 7f00c0e23740 10 client.0 _ll_register_callbacks cb 0x5654e47b9ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.159+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 0x5654e4b53d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.159+0000 7f00afe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.159+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 0x5654e4b53d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44442/0 (socket says 172.21.15.2:44442) 2024-09-10T08:20:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.159+0000 7f00b7e00640 1 -- 172.21.15.2:0/1327035334 learned_addr learned my addr 172.21.15.2:0/1327035334 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:56.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.159+0000 7f00b7e00640 1 --2- 172.21.15.2:0/1327035334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b4fca0 0x5654e4b58430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.159+0000 7f00b7e00640 1 -- 172.21.15.2:0/1327035334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 msgr2=0x5654e4b542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.159+0000 7f00b7e00640 1 --2- 172.21.15.2:0/1327035334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.159+0000 7f00b7e00640 1 -- 172.21.15.2:0/1327035334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b4fca0 msgr2=0x5654e4b58430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.159+0000 7f00b7e00640 1 --2- 172.21.15.2:0/1327035334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b4fca0 0x5654e4b58430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.159+0000 7f00b7e00640 1 -- 172.21.15.2:0/1327035334 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654e4b58ad0 con 0x5654e4b44ac0 2024-09-10T08:20:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.159+0000 7f00afe00640 1 --2- 172.21.15.2:0/1327035334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.160+0000 7f00b7e00640 1 --2- 172.21.15.2:0/1327035334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 0x5654e4b53d80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f00b0004af0 tx=0x7f00b0031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90cbc380202e1c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.160+0000 7f00a5600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.160+0000 7f00a5600640 1 -- 172.21.15.2:0/1327035334 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0058060 con 0x5654e4b44ac0 2024-09-10T08:20:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.160+0000 7f00c0e23740 1 -- 172.21.15.2:0/1327035334 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00b0036030 con 0x5654e4b44ac0 2024-09-10T08:20:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.160+0000 7f00a5600640 1 -- 172.21.15.2:0/1327035334 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f00b0003d80 con 0x5654e4b44ac0 2024-09-10T08:20:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.160+0000 7f00a5600640 1 -- 172.21.15.2:0/1327035334 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00b005f050 con 0x5654e4b44ac0 2024-09-10T08:20:56.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.160+0000 7f00c0e23740 10 client.19050 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.160+0000 7f00c0e23740 1 -- 172.21.15.2:0/1327035334 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5654e4b52790 con 0x5654e4b44ac0 2024-09-10T08:20:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.160+0000 7f00a5600640 1 -- 172.21.15.2:0/1327035334 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b0036030 con 0x5654e4b44ac0 2024-09-10T08:20:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.161+0000 7f00a5600640 1 -- 172.21.15.2:0/1327035334 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f00b0043070 con 0x5654e4b44ac0 2024-09-10T08:20:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.161+0000 7f00c0e23740 10 client.19050 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.161+0000 7f00c0e23740 10 client.19050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.161+0000 7f00c0e23740 20 client.19050 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.161+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1327035334 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654e4b63c70 0x5654e4b66060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.161+0000 7f00c0e23740 4 client.19050 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.161+0000 7f00c0e23740 1 -- 172.21.15.2:0/1327035334 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654e4ab8800 con 0x5654e4b63c70 2024-09-10T08:20:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.161+0000 7f00afe00640 1 --2- 172.21.15.2:0/1327035334 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654e4b63c70 0x5654e4b66060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.162+0000 7f00afe00640 1 --2- 172.21.15.2:0/1327035334 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654e4b63c70 0x5654e4b66060 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95bfa67c5b9a3ab server_cookie=8ac84b8b9c9092b9 in_seq=0 out_seq=0 2024-09-10T08:20:56.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.162+0000 7f00a5600640 10 client.19050 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:56.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00a5600640 1 -- 172.21.15.2:0/1327035334 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654e4ab8800 con 0x5654e4b63c70 2024-09-10T08:20:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00a5600640 10 client.19050 handle_command_reply: tid=0 2024-09-10T08:20:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00c0e23740 1 client.19050 shutdown 2024-09-10T08:20:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00c0e23740 1 -- 172.21.15.2:0/1327035334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 msgr2=0x5654e4b53d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1327035334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 0x5654e4b53d80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f00b0004af0 tx=0x7f00b0031980 comp rx=0 tx=0).stop 2024-09-10T08:20:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00c0e23740 1 -- 172.21.15.2:0/1327035334 shutdown_connections 2024-09-10T08:20:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1327035334 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654e4b63c70 0x5654e4b66060 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1327035334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b4fca0 0x5654e4b58430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1327035334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00a5600640 0 client.19050 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1327035334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 0x5654e4b53d80 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00c0e23740 1 -- 172.21.15.2:0/1327035334 >> 172.21.15.2:0/1327035334 conn(0x5654e4b41050 msgr2=0x5654e4b401c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00c0e23740 1 -- 172.21.15.2:0/1327035334 shutdown_connections 2024-09-10T08:20:56.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00c0e23740 1 -- 172.21.15.2:0/1327035334 wait complete. 2024-09-10T08:20:56.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.163+0000 7f00c0e23740 20 client.19050 trim_cache size 0 max 16384 2024-09-10T08:20:56.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.182+0000 7f00c0e23740 1 Processor -- start 2024-09-10T08:20:56.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.182+0000 7f00c0e23740 1 -- start start 2024-09-10T08:20:56.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.182+0000 7f00c0e23740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 0x5654e4b58420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.182+0000 7f00c0e23740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b453e0 0x5654e4b53b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.182+0000 7f00c0e23740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b4fca0 0x5654e4b540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.182+0000 7f00c0e23740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81af440 con 0x5654e4b44ac0 2024-09-10T08:20:56.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.182+0000 7f00c0e23740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81acd10 con 0x5654e4b453e0 2024-09-10T08:20:56.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.182+0000 7f00c0e23740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81aab60 con 0x5654e4b4fca0 2024-09-10T08:20:56.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.182+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 0x5654e4b58420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.183+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 0x5654e4b58420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44446/0 (socket says 172.21.15.2:44446) 2024-09-10T08:20:56.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.183+0000 7f00b7e00640 1 -- 172.21.15.2:0/1288603778 learned_addr learned my addr 172.21.15.2:0/1288603778 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:56.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.183+0000 7f00afe00640 1 --2- 172.21.15.2:0/1288603778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b453e0 0x5654e4b53b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.183+0000 7f00b7e00640 1 --2- 172.21.15.2:0/1288603778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b4fca0 0x5654e4b540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.183+0000 7f00afe00640 1 -- 172.21.15.2:0/1288603778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b4fca0 msgr2=0x5654e4b540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:56.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.183+0000 7f00afe00640 1 --2- 172.21.15.2:0/1288603778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b4fca0 0x5654e4b540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.183+0000 7f00afe00640 1 -- 172.21.15.2:0/1288603778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 msgr2=0x5654e4b58420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:56.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.183+0000 7f00afe00640 1 --2- 172.21.15.2:0/1288603778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 0x5654e4b58420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.183+0000 7f00afe00640 1 -- 172.21.15.2:0/1288603778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654e4b448d0 con 0x5654e4b453e0 2024-09-10T08:20:56.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.183+0000 7f00b7e00640 1 --2- 172.21.15.2:0/1288603778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 0x5654e4b58420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:56.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.183+0000 7f00afe00640 1 --2- 172.21.15.2:0/1288603778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b453e0 0x5654e4b53b70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x5654e4b668d0 tx=0x7f00a0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=897707252512583e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:56.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.184+0000 7f00a7e00640 1 -- 172.21.15.2:0/1288603778 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a00122d0 con 0x5654e4b453e0 2024-09-10T08:20:56.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.184+0000 7f00a7e00640 1 -- 172.21.15.2:0/1288603778 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a000f7f0 con 0x5654e4b453e0 2024-09-10T08:20:56.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.184+0000 7f00a7e00640 1 -- 172.21.15.2:0/1288603778 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0003950 con 0x5654e4b453e0 2024-09-10T08:20:56.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.184+0000 7f00c0e23740 1 -- 172.21.15.2:0/1288603778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b453e0 msgr2=0x5654e4b53b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.184+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1288603778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b453e0 0x5654e4b53b70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x5654e4b668d0 tx=0x7f00a0005f90 comp rx=0 tx=0).stop 2024-09-10T08:20:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.184+0000 7f00c0e23740 1 -- 172.21.15.2:0/1288603778 shutdown_connections 2024-09-10T08:20:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.184+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1288603778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b4fca0 0x5654e4b540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.184+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1288603778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b453e0 0x5654e4b53b70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.184+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1288603778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b44ac0 0x5654e4b58420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.184+0000 7f00c0e23740 1 -- 172.21.15.2:0/1288603778 >> 172.21.15.2:0/1288603778 conn(0x5654e4b41050 msgr2=0x5654e4b53460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.184+0000 7f00c0e23740 1 -- 172.21.15.2:0/1288603778 shutdown_connections 2024-09-10T08:20:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.184+0000 7f00c0e23740 1 -- 172.21.15.2:0/1288603778 wait complete. 2024-09-10T08:20:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.185+0000 7f00c0e23740 1 -- start start 2024-09-10T08:20:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.185+0000 7f00c0e23740 1 Processor -- start 2024-09-10T08:20:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.185+0000 7f00c0e23740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b44ac0 0x5654e4b4a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.185+0000 7f00c0e23740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b4a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.185+0000 7f00c0e23740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b4fca0 0x5654e4b5c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.185+0000 7f00c0e23740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81ab1d0 con 0x5654e4b4fca0 2024-09-10T08:20:56.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.185+0000 7f00c0e23740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81ac640 con 0x5654e4b44ac0 2024-09-10T08:20:56.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.185+0000 7f00c0e23740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81ab8a0 con 0x5654e4b453e0 2024-09-10T08:20:56.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.186+0000 7f00c0e23740 10 client.0 _ll_register_callbacks cb 0x7f00a00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:56.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.186+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b4fca0 0x5654e4b5c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.186+0000 7f00afe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b4a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.186+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b44ac0 0x5654e4b4a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.186+0000 7f00afe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b4a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:47234/0 (socket says 172.21.15.2:47234) 2024-09-10T08:20:56.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.186+0000 7f00afe00640 1 -- 172.21.15.2:0/1123879638 learned_addr learned my addr 172.21.15.2:0/1123879638 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:56.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.186+0000 7f00b7e00640 1 -- 172.21.15.2:0/1123879638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 msgr2=0x5654e4b4a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.186+0000 7f00b7e00640 1 --2- 172.21.15.2:0/1123879638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b4a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.186+0000 7f00b7e00640 1 -- 172.21.15.2:0/1123879638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b4fca0 msgr2=0x5654e4b5c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.186+0000 7f00b7e00640 1 --2- 172.21.15.2:0/1123879638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b4fca0 0x5654e4b5c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.186+0000 7f00b7e00640 1 -- 172.21.15.2:0/1123879638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654e4b5ca10 con 0x5654e4b44ac0 2024-09-10T08:20:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.186+0000 7f00afe00640 1 --2- 172.21.15.2:0/1123879638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b4a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.187+0000 7f00b7e00640 1 --2- 172.21.15.2:0/1123879638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b44ac0 0x5654e4b4a3c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f00b002fd60 tx=0x7f00b0000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e070e62b266e41c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.187+0000 7f00a5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:20:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.187+0000 7f00a5600640 1 -- 172.21.15.2:0/1123879638 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0058040 con 0x5654e4b44ac0 2024-09-10T08:20:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.187+0000 7f00c0e23740 1 -- 172.21.15.2:0/1123879638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a000f7f0 con 0x5654e4b44ac0 2024-09-10T08:20:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.187+0000 7f00a5600640 1 -- 172.21.15.2:0/1123879638 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f00b00307d0 con 0x5654e4b44ac0 2024-09-10T08:20:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.187+0000 7f00a5600640 1 -- 172.21.15.2:0/1123879638 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00b006f050 con 0x5654e4b44ac0 2024-09-10T08:20:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.187+0000 7f00c0e23740 10 client.29081 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.187+0000 7f00c0e23740 1 -- 172.21.15.2:0/1123879638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5654e4bd3830 con 0x5654e4b44ac0 2024-09-10T08:20:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.188+0000 7f00a5600640 1 -- 172.21.15.2:0/1123879638 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a000f7f0 con 0x5654e4b44ac0 2024-09-10T08:20:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.188+0000 7f00a5600640 1 -- 172.21.15.2:0/1123879638 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f00b0043040 con 0x5654e4b44ac0 2024-09-10T08:20:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.188+0000 7f00c0e23740 10 client.29081 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.188+0000 7f00c0e23740 10 client.29081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.188+0000 7f00c0e23740 20 client.29081 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.188+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1123879638 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654e4bd3e30 0x5654e4bd6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.188+0000 7f00c0e23740 4 client.29081 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:20:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.188+0000 7f00c0e23740 1 -- 172.21.15.2:0/1123879638 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5654e4ab2d10 con 0x5654e4bd3e30 2024-09-10T08:20:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.188+0000 7f00afe00640 1 --2- 172.21.15.2:0/1123879638 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654e4bd3e30 0x5654e4bd6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.189+0000 7f00afe00640 1 --2- 172.21.15.2:0/1123879638 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654e4bd3e30 0x5654e4bd6220 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e5f92764fc009d server_cookie=d3b7ce5c41bda10 in_seq=0 out_seq=0 2024-09-10T08:20:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.189+0000 7f00a5600640 10 client.29081 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00a5600640 1 -- 172.21.15.2:0/1123879638 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5654e4ab2d10 con 0x5654e4bd3e30 2024-09-10T08:20:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00a5600640 10 client.29081 handle_command_reply: tid=0 2024-09-10T08:20:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00c0e23740 1 client.29081 shutdown 2024-09-10T08:20:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00c0e23740 1 -- 172.21.15.2:0/1123879638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b44ac0 msgr2=0x5654e4b4a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1123879638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b44ac0 0x5654e4b4a3c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f00b002fd60 tx=0x7f00b0000f40 comp rx=0 tx=0).stop 2024-09-10T08:20:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00c0e23740 1 -- 172.21.15.2:0/1123879638 shutdown_connections 2024-09-10T08:20:56.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1123879638 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5654e4bd3e30 0x5654e4bd6220 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1123879638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654e4b4fca0 0x5654e4b5c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1123879638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654e4b453e0 0x5654e4b4a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00c0e23740 1 --2- 172.21.15.2:0/1123879638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e4b44ac0 0x5654e4b4a3c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00a5600640 0 client.29081 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:56.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00c0e23740 1 -- 172.21.15.2:0/1123879638 >> 172.21.15.2:0/1123879638 conn(0x5654e4b41050 msgr2=0x5654e4b53bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:56.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00c0e23740 1 -- 172.21.15.2:0/1123879638 shutdown_connections 2024-09-10T08:20:56.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.190+0000 7f00c0e23740 1 -- 172.21.15.2:0/1123879638 wait complete. 2024-09-10T08:20:56.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.191+0000 7f00c0e23740 20 client.29081 trim_cache size 0 max 16384 2024-09-10T08:20:56.193 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:20:56.194 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:20:56.194 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "4aca35fd-0e1a-4792-8e8f-9ce86666a2f6", 2024-09-10T08:20:56.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:20:56.194 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:20:56.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.191+0000 7f00be400640 1 -- 172.21.15.2:0/3621361489 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f009c0a2b80 msgr2=0x7f009c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:56.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.191+0000 7f00be400640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f009c0a2b80 0x7f009c0a5040 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f00a80980d0 tx=0x7f00a80d5530 comp rx=0 tx=0).stop 2024-09-10T08:20:56.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.191+0000 7f00be400640 1 -- 172.21.15.2:0/3621361489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00b8101de0 msgr2=0x7f00b8198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:56.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.191+0000 7f00be400640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00b8101de0 0x7f00b8198420 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f00a800e9f0 tx=0x7f00a800edf0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.191+0000 7f00be400640 1 -- 172.21.15.2:0/3621361489 shutdown_connections 2024-09-10T08:20:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.191+0000 7f00be400640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8106f10 0x7f00b819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.191+0000 7f00be400640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f009c0a2b80 0x7f009c0a5040 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.191+0000 7f00be400640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b8102790 0x7f00b8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.191+0000 7f00be400640 1 --2- 172.21.15.2:0/3621361489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00b8101de0 0x7f00b8198420 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.191+0000 7f00be400640 1 -- 172.21.15.2:0/3621361489 >> 172.21.15.2:0/3621361489 conn(0x7f00b80fd8c0 msgr2=0x7f00b80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.192+0000 7f00be400640 1 -- 172.21.15.2:0/3621361489 shutdown_connections 2024-09-10T08:20:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:56.192+0000 7f00be400640 1 -- 172.21.15.2:0/3621361489 wait complete. 2024-09-10T08:20:57.140 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:20:57.519 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:20:58.013 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:57 smithi002 ceph-mon[30563]: pgmap v771: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:20:58.013 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:57 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:20:58.013 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:57 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:20:58.013 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:57 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:20:58.014 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:57 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:20:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.128+0000 7f626f200640 1 -- 172.21.15.2:0/663864378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6268109090 msgr2=0x7f6268110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.128+0000 7f626f200640 1 --2- 172.21.15.2:0/663864378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6268109090 0x7f6268110c10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f6264009ca0 tx=0x7f626402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.128+0000 7f626f200640 1 -- 172.21.15.2:0/663864378 shutdown_connections 2024-09-10T08:20:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.128+0000 7f626f200640 1 --2- 172.21.15.2:0/663864378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6268109090 0x7f6268110c10 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.128+0000 7f626f200640 1 --2- 172.21.15.2:0/663864378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6268104800 0x7f6268108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.128+0000 7f626f200640 1 --2- 172.21.15.2:0/663864378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6268103e50 0x7f6268104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.128+0000 7f626f200640 1 -- 172.21.15.2:0/663864378 >> 172.21.15.2:0/663864378 conn(0x7f62680ff8b0 msgr2=0x7f6268101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.128+0000 7f626f200640 1 -- 172.21.15.2:0/663864378 shutdown_connections 2024-09-10T08:20:58.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.128+0000 7f626f200640 1 -- 172.21.15.2:0/663864378 wait complete. 2024-09-10T08:20:58.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.129+0000 7f626f200640 1 Processor -- start 2024-09-10T08:20:58.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.129+0000 7f626f200640 1 -- start start 2024-09-10T08:20:58.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6268103e50 0x7f626819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6268104800 0x7f626819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626f200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6268109090 0x7f626819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626f200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6268114110 con 0x7f6268109090 2024-09-10T08:20:58.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6268113f90 con 0x7f6268104800 2024-09-10T08:20:58.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6268114290 con 0x7f6268103e50 2024-09-10T08:20:58.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6268103e50 0x7f626819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6268109090 0x7f626819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6268104800 0x7f626819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6268103e50 0x7f626819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50430/0 (socket says 172.21.15.2:50430) 2024-09-10T08:20:58.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626de00640 1 -- 172.21.15.2:0/927814800 learned_addr learned my addr 172.21.15.2:0/927814800 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:58.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626de00640 1 -- 172.21.15.2:0/927814800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6268104800 msgr2=0x7f626819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:58.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626de00640 1 --2- 172.21.15.2:0/927814800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6268104800 0x7f626819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626de00640 1 -- 172.21.15.2:0/927814800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6268109090 msgr2=0x7f626819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:58.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.130+0000 7f626de00640 1 --2- 172.21.15.2:0/927814800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6268109090 0x7f626819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.131+0000 7f626de00640 1 -- 172.21.15.2:0/927814800 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626819f500 con 0x7f6268103e50 2024-09-10T08:20:58.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.131+0000 7f626de00640 1 --2- 172.21.15.2:0/927814800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6268103e50 0x7f626819a550 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f625c00e9f0 tx=0x7f625c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:58.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.131+0000 7f6262a00640 1 -- 172.21.15.2:0/927814800 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c00cd80 con 0x7f6268103e50 2024-09-10T08:20:58.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.131+0000 7f626f200640 1 -- 172.21.15.2:0/927814800 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626819f7f0 con 0x7f6268103e50 2024-09-10T08:20:58.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.131+0000 7f6262a00640 1 -- 172.21.15.2:0/927814800 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c005500 con 0x7f6268103e50 2024-09-10T08:20:58.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.131+0000 7f6262a00640 1 -- 172.21.15.2:0/927814800 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c010690 con 0x7f6268103e50 2024-09-10T08:20:58.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.131+0000 7f626f200640 1 -- 172.21.15.2:0/927814800 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62681a70d0 con 0x7f6268103e50 2024-09-10T08:20:58.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.134+0000 7f6273470740 1 Processor -- start 2024-09-10T08:20:58.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.134+0000 7f6273470740 1 -- start start 2024-09-10T08:20:58.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.134+0000 7f6262a00640 1 -- 172.21.15.2:0/927814800 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f625c0046c0 con 0x7f6268103e50 2024-09-10T08:20:58.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.135+0000 7f6273470740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12ccd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.135+0000 7f6273470740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cd79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.135+0000 7f6273470740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cd7f30 0x55ef12cda320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.135+0000 7f6273470740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ef12c333e0 con 0x55ef12ccd670 2024-09-10T08:20:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.135+0000 7f6273470740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ef12bdad80 con 0x55ef12cd7f30 2024-09-10T08:20:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.135+0000 7f6273470740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ef12c2f3a0 con 0x55ef12cccd50 2024-09-10T08:20:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.135+0000 7f626de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cd79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.135+0000 7f626de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cd79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57198/0 (socket says 172.21.15.2:57198) 2024-09-10T08:20:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.135+0000 7f626de00640 1 -- 172.21.15.2:0/2103983073 learned_addr learned my addr 172.21.15.2:0/2103983073 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.135+0000 7f626e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12ccd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.135+0000 7f626d400640 1 --2- 172.21.15.2:0/2103983073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cd7f30 0x55ef12cda320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.135+0000 7f626e800640 1 -- 172.21.15.2:0/2103983073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cd7f30 msgr2=0x55ef12cda320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.136+0000 7f626e800640 1 --2- 172.21.15.2:0/2103983073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cd7f30 0x55ef12cda320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.136+0000 7f626e800640 1 -- 172.21.15.2:0/2103983073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 msgr2=0x55ef12cd79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:58.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.136+0000 7f626e800640 1 --2- 172.21.15.2:0/2103983073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cd79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.136+0000 7f626e800640 1 -- 172.21.15.2:0/2103983073 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef12bf9ae0 con 0x55ef12cccd50 2024-09-10T08:20:58.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.136+0000 7f626d400640 1 --2- 172.21.15.2:0/2103983073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cd7f30 0x55ef12cda320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:20:58.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.136+0000 7f626e800640 1 --2- 172.21.15.2:0/2103983073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12ccd130 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6264005e30 tx=0x7f62640093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51f71d27a55afe97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:58.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.136+0000 7f6257000640 1 -- 172.21.15.2:0/2103983073 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264031db0 con 0x55ef12cccd50 2024-09-10T08:20:58.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.136+0000 7f6257000640 1 -- 172.21.15.2:0/2103983073 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6264036030 con 0x55ef12cccd50 2024-09-10T08:20:58.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.136+0000 7f6257000640 1 -- 172.21.15.2:0/2103983073 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62640042a0 con 0x55ef12cccd50 2024-09-10T08:20:58.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.136+0000 7f6262a00640 1 --2- 172.21.15.2:0/927814800 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62500a2b80 0x7f62500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.137+0000 7f6262a00640 1 -- 172.21.15.2:0/927814800 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f625c0cf870 con 0x7f6268103e50 2024-09-10T08:20:58.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.137+0000 7f626de00640 1 --2- 172.21.15.2:0/927814800 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62500a2b80 0x7f62500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.137+0000 7f6273470740 1 -- 172.21.15.2:0/2103983073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 msgr2=0x55ef12ccd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.137+0000 7f6273470740 1 --2- 172.21.15.2:0/2103983073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12ccd130 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6264005e30 tx=0x7f62640093e0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.137+0000 7f6273470740 1 -- 172.21.15.2:0/2103983073 shutdown_connections 2024-09-10T08:20:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.137+0000 7f6273470740 1 --2- 172.21.15.2:0/2103983073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cd7f30 0x55ef12cda320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.137+0000 7f6273470740 1 --2- 172.21.15.2:0/2103983073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cd79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.137+0000 7f6273470740 1 --2- 172.21.15.2:0/2103983073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12ccd130 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.137+0000 7f6273470740 1 -- 172.21.15.2:0/2103983073 >> 172.21.15.2:0/2103983073 conn(0x55ef12cc92e0 msgr2=0x55ef12cc96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.137+0000 7f6273470740 1 -- 172.21.15.2:0/2103983073 shutdown_connections 2024-09-10T08:20:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.137+0000 7f626de00640 1 --2- 172.21.15.2:0/927814800 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62500a2b80 0x7f62500a5040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f625c0cfb10 tx=0x7f625c0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.138+0000 7f6273470740 1 -- 172.21.15.2:0/2103983073 wait complete. 2024-09-10T08:20:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.138+0000 7f6273470740 1 -- start start 2024-09-10T08:20:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.138+0000 7f6273470740 1 Processor -- start 2024-09-10T08:20:58.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f6273470740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12cd4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f6273470740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cd5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f6273470740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cd7f30 0x55ef12cd58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f6273470740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ef12cdaa20 con 0x55ef12ccd670 2024-09-10T08:20:58.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f6273470740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ef12bee0f0 con 0x55ef12cd7f30 2024-09-10T08:20:58.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f6273470740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ef12cdaba0 con 0x55ef12cccd50 2024-09-10T08:20:58.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f6273470740 10 client.0 _ll_register_callbacks cb 0x55ef12bdf9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:58.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f626e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12cd4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f626d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cd5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f626e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12cd4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50434/0 (socket says 172.21.15.2:50434) 2024-09-10T08:20:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f626e800640 1 -- 172.21.15.2:0/929820225 learned_addr learned my addr 172.21.15.2:0/929820225 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f626e800640 1 --2- 172.21.15.2:0/929820225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cd7f30 0x55ef12cd58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f626e800640 1 -- 172.21.15.2:0/929820225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cd7f30 msgr2=0x55ef12cd58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f626e800640 1 --2- 172.21.15.2:0/929820225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cd7f30 0x55ef12cd58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f626e800640 1 -- 172.21.15.2:0/929820225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 msgr2=0x55ef12cd5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f626e800640 1 --2- 172.21.15.2:0/929820225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cd5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f626e800640 1 -- 172.21.15.2:0/929820225 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef129a43c0 con 0x55ef12cccd50 2024-09-10T08:20:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.139+0000 7f626d400640 1 --2- 172.21.15.2:0/929820225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cd5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:20:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.140+0000 7f626e800640 1 --2- 172.21.15.2:0/929820225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12cd4e50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6264031eb0 tx=0x7f626402f4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cf917b07724317 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.140+0000 7f624b400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:20:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.140+0000 7f624b400640 1 -- 172.21.15.2:0/929820225 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264058060 con 0x55ef12cccd50 2024-09-10T08:20:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.140+0000 7f6273470740 1 -- 172.21.15.2:0/929820225 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6264036030 con 0x55ef12cccd50 2024-09-10T08:20:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.140+0000 7f624b400640 1 -- 172.21.15.2:0/929820225 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6264004440 con 0x55ef12cccd50 2024-09-10T08:20:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.140+0000 7f624b400640 1 -- 172.21.15.2:0/929820225 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6264031760 con 0x55ef12cccd50 2024-09-10T08:20:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.140+0000 7f6273470740 10 client.29023 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.140+0000 7f6273470740 1 -- 172.21.15.2:0/929820225 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef12cdaa20 con 0x55ef12cccd50 2024-09-10T08:20:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.141+0000 7f624b400640 1 -- 172.21.15.2:0/929820225 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6264036030 con 0x55ef12cccd50 2024-09-10T08:20:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.141+0000 7f624b400640 1 -- 172.21.15.2:0/929820225 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6264050070 con 0x55ef12cccd50 2024-09-10T08:20:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.141+0000 7f6273470740 10 client.29023 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.141+0000 7f6273470740 10 client.29023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.141+0000 7f6273470740 20 client.29023 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.141+0000 7f6273470740 1 --2- 172.21.15.2:0/929820225 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ef12ce7880 0x55ef12ce9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.141+0000 7f6273470740 4 client.29023 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:20:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.141+0000 7f6273470740 1 -- 172.21.15.2:0/929820225 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef12c40950 con 0x55ef12ce7880 2024-09-10T08:20:58.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.142+0000 7f626d400640 1 --2- 172.21.15.2:0/929820225 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ef12ce7880 0x55ef12ce9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.142+0000 7f626d400640 1 --2- 172.21.15.2:0/929820225 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ef12ce7880 0x55ef12ce9ca0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb4805729d9e8b9a server_cookie=859094e017fbfd4c in_seq=0 out_seq=0 2024-09-10T08:20:58.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.142+0000 7f624b400640 10 client.29023 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.143+0000 7f624b400640 1 -- 172.21.15.2:0/929820225 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef12c40950 con 0x55ef12ce7880 2024-09-10T08:20:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.143+0000 7f624b400640 10 client.29023 handle_command_reply: tid=0 2024-09-10T08:20:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.143+0000 7f6273470740 1 client.29023 shutdown 2024-09-10T08:20:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.143+0000 7f6273470740 1 -- 172.21.15.2:0/929820225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 msgr2=0x55ef12cd4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.143+0000 7f6273470740 1 --2- 172.21.15.2:0/929820225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12cd4e50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6264031eb0 tx=0x7f626402f4d0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.143+0000 7f6273470740 1 -- 172.21.15.2:0/929820225 shutdown_connections 2024-09-10T08:20:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.143+0000 7f6273470740 1 --2- 172.21.15.2:0/929820225 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ef12ce7880 0x55ef12ce9ca0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.143+0000 7f6273470740 1 --2- 172.21.15.2:0/929820225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cd7f30 0x55ef12cd58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.143+0000 7f6273470740 1 --2- 172.21.15.2:0/929820225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cd5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.143+0000 7f624b400640 0 client.29023 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.143+0000 7f6273470740 1 --2- 172.21.15.2:0/929820225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12cd4e50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.143+0000 7f6273470740 1 -- 172.21.15.2:0/929820225 >> 172.21.15.2:0/929820225 conn(0x55ef12cc92e0 msgr2=0x55ef12ccf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:58.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.144+0000 7f6273470740 1 -- 172.21.15.2:0/929820225 shutdown_connections 2024-09-10T08:20:58.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.144+0000 7f6273470740 1 -- 172.21.15.2:0/929820225 wait complete. 2024-09-10T08:20:58.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.144+0000 7f6273470740 20 client.29023 trim_cache size 0 max 16384 2024-09-10T08:20:58.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:57 smithi155 ceph-mon[35405]: pgmap v771: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:20:58.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:57 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:20:58.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:57 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:20:58.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:57 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:20:58.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:57 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:20:58.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.162+0000 7f6273470740 1 Processor -- start 2024-09-10T08:20:58.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.162+0000 7f6273470740 1 -- start start 2024-09-10T08:20:58.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.162+0000 7f6273470740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12cc96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.162+0000 7f6273470740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12ccd670 0x55ef12cd5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.162+0000 7f6273470740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12cd7f30 0x55ef12cceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.162+0000 7f6273470740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f62681b3650 con 0x55ef12cd7f30 2024-09-10T08:20:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.162+0000 7f6273470740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f62681b0f20 con 0x55ef12ccd670 2024-09-10T08:20:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.162+0000 7f6273470740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62681aed70 con 0x55ef12cccd50 2024-09-10T08:20:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.163+0000 7f626e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12cc96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.163+0000 7f626d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12ccd670 0x55ef12cd5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.163+0000 7f626d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12ccd670 0x55ef12cd5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:48664/0 (socket says 172.21.15.2:48664) 2024-09-10T08:20:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.163+0000 7f626d400640 1 -- 172.21.15.2:0/918623344 learned_addr learned my addr 172.21.15.2:0/918623344 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.163+0000 7f626d400640 1 -- 172.21.15.2:0/918623344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 msgr2=0x55ef12cc96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.163+0000 7f626d400640 1 --2- 172.21.15.2:0/918623344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12cc96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.163+0000 7f626d400640 1 -- 172.21.15.2:0/918623344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12cd7f30 msgr2=0x55ef12cceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.163+0000 7f626d400640 1 --2- 172.21.15.2:0/918623344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12cd7f30 0x55ef12cceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.163+0000 7f626d400640 1 -- 172.21.15.2:0/918623344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef12cd3e00 con 0x55ef12ccd670 2024-09-10T08:20:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.163+0000 7f626d400640 1 --2- 172.21.15.2:0/918623344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12ccd670 0x55ef12cd5720 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6258012470 tx=0x7f6258005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c43bac5cc2768203 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.164+0000 7f6256600640 1 -- 172.21.15.2:0/918623344 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258004030 con 0x55ef12ccd670 2024-09-10T08:20:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.164+0000 7f6256600640 1 -- 172.21.15.2:0/918623344 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6258005500 con 0x55ef12ccd670 2024-09-10T08:20:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.164+0000 7f6256600640 1 -- 172.21.15.2:0/918623344 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625800fd50 con 0x55ef12ccd670 2024-09-10T08:20:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.164+0000 7f6273470740 1 -- 172.21.15.2:0/918623344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12ccd670 msgr2=0x55ef12cd5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.164+0000 7f6273470740 1 --2- 172.21.15.2:0/918623344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12ccd670 0x55ef12cd5720 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6258012470 tx=0x7f6258005f90 comp rx=0 tx=0).stop 2024-09-10T08:20:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.164+0000 7f6273470740 1 -- 172.21.15.2:0/918623344 shutdown_connections 2024-09-10T08:20:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.164+0000 7f6273470740 1 --2- 172.21.15.2:0/918623344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12cd7f30 0x55ef12cceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.164+0000 7f6273470740 1 --2- 172.21.15.2:0/918623344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12ccd670 0x55ef12cd5720 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.164+0000 7f6273470740 1 --2- 172.21.15.2:0/918623344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cccd50 0x55ef12cc96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.164+0000 7f6273470740 1 -- 172.21.15.2:0/918623344 >> 172.21.15.2:0/918623344 conn(0x55ef12cc92e0 msgr2=0x55ef12cce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:58.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.165+0000 7f6273470740 1 -- 172.21.15.2:0/918623344 shutdown_connections 2024-09-10T08:20:58.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.165+0000 7f6273470740 1 -- 172.21.15.2:0/918623344 wait complete. 2024-09-10T08:20:58.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.165+0000 7f6273470740 1 -- start start 2024-09-10T08:20:58.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.165+0000 7f6273470740 1 Processor -- start 2024-09-10T08:20:58.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f6273470740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cccd50 0x55ef12cdbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f6273470740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cdc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f6273470740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cd7f30 0x55ef12cdc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f6273470740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f62681af3e0 con 0x55ef12ccd670 2024-09-10T08:20:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f6273470740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f62681b0850 con 0x55ef12cccd50 2024-09-10T08:20:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f6273470740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62681afab0 con 0x55ef12cd7f30 2024-09-10T08:20:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f6273470740 10 client.0 _ll_register_callbacks cb 0x7f6264031bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:20:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f626d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cdc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f626e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cccd50 0x55ef12cdbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f626e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cccd50 0x55ef12cdbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:48674/0 (socket says 172.21.15.2:48674) 2024-09-10T08:20:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f626e800640 1 -- 172.21.15.2:0/516522085 learned_addr learned my addr 172.21.15.2:0/516522085 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:20:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f626d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cdc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57238/0 (socket says 172.21.15.2:57238) 2024-09-10T08:20:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f626e800640 1 --2- 172.21.15.2:0/516522085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cd7f30 0x55ef12cdc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f626d400640 1 -- 172.21.15.2:0/516522085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cd7f30 msgr2=0x55ef12cdc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f626d400640 1 --2- 172.21.15.2:0/516522085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cd7f30 0x55ef12cdc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f626d400640 1 -- 172.21.15.2:0/516522085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cccd50 msgr2=0x55ef12cdbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:20:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f626d400640 1 --2- 172.21.15.2:0/516522085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cccd50 0x55ef12cdbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.166+0000 7f626d400640 1 -- 172.21.15.2:0/516522085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef12d51750 con 0x55ef12ccd670 2024-09-10T08:20:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.167+0000 7f626d400640 1 --2- 172.21.15.2:0/516522085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cdc2b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f625800f830 tx=0x7f6258012710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaa43103d8ba509 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:20:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.167+0000 7f624b400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:20:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.167+0000 7f624b400640 1 -- 172.21.15.2:0/516522085 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258012290 con 0x55ef12ccd670 2024-09-10T08:20:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.167+0000 7f624b400640 1 -- 172.21.15.2:0/516522085 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f625800f9d0 con 0x55ef12ccd670 2024-09-10T08:20:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.167+0000 7f6273470740 1 -- 172.21.15.2:0/516522085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6258005500 con 0x55ef12ccd670 2024-09-10T08:20:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.167+0000 7f624b400640 1 -- 172.21.15.2:0/516522085 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f625803b050 con 0x55ef12ccd670 2024-09-10T08:20:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.167+0000 7f6273470740 10 client.19071 fetch_fsmap learned FSMap version 26 2024-09-10T08:20:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.167+0000 7f6273470740 1 -- 172.21.15.2:0/516522085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef12d52a20 con 0x55ef12ccd670 2024-09-10T08:20:58.170 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:57 smithi107 ceph-mon[35048]: pgmap v771: 129 pgs: 129 active+clean; 2.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:20:58.170 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:57 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:20:58.170 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:57 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:20:58.170 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:57 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:20:58.170 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:57 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:20:58.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.168+0000 7f624b400640 1 -- 172.21.15.2:0/516522085 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6258005500 con 0x55ef12ccd670 2024-09-10T08:20:58.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.168+0000 7f624b400640 1 -- 172.21.15.2:0/516522085 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f62580324c0 con 0x55ef12ccd670 2024-09-10T08:20:58.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.168+0000 7f6273470740 10 client.19071 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:20:58.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.168+0000 7f6273470740 10 client.19071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:20:58.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.168+0000 7f6273470740 20 client.19071 populate_metadata read hostname 'smithi002' 2024-09-10T08:20:58.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.168+0000 7f6273470740 1 --2- 172.21.15.2:0/516522085 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ef12d53080 0x55ef12d554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:20:58.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.168+0000 7f6273470740 4 client.19071 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:20:58.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.168+0000 7f6273470740 1 -- 172.21.15.2:0/516522085 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef12c3ae60 con 0x55ef12d53080 2024-09-10T08:20:58.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.168+0000 7f626e800640 1 --2- 172.21.15.2:0/516522085 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ef12d53080 0x55ef12d554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:20:58.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.169+0000 7f626e800640 1 --2- 172.21.15.2:0/516522085 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ef12d53080 0x55ef12d554a0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b5898557dcdd36 server_cookie=5bda21dcac6345b3 in_seq=0 out_seq=0 2024-09-10T08:20:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.169+0000 7f624b400640 10 client.19071 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.169+0000 7f624b400640 1 -- 172.21.15.2:0/516522085 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ef12c3ae60 con 0x55ef12d53080 2024-09-10T08:20:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.169+0000 7f624b400640 10 client.19071 handle_command_reply: tid=0 2024-09-10T08:20:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.169+0000 7f6273470740 1 client.19071 shutdown 2024-09-10T08:20:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.169+0000 7f6273470740 1 -- 172.21.15.2:0/516522085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 msgr2=0x55ef12cdc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.169+0000 7f6273470740 1 --2- 172.21.15.2:0/516522085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cdc2b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f625800f830 tx=0x7f6258012710 comp rx=0 tx=0).stop 2024-09-10T08:20:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.169+0000 7f6273470740 1 -- 172.21.15.2:0/516522085 shutdown_connections 2024-09-10T08:20:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.169+0000 7f6273470740 1 --2- 172.21.15.2:0/516522085 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ef12d53080 0x55ef12d554a0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.170+0000 7f6273470740 1 --2- 172.21.15.2:0/516522085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef12cd7f30 0x55ef12cdc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.170+0000 7f6273470740 1 --2- 172.21.15.2:0/516522085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef12ccd670 0x55ef12cdc2b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.170+0000 7f6273470740 1 --2- 172.21.15.2:0/516522085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef12cccd50 0x55ef12cdbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.170+0000 7f624b400640 0 client.19071 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:20:58.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.170+0000 7f6273470740 1 -- 172.21.15.2:0/516522085 >> 172.21.15.2:0/516522085 conn(0x55ef12cc92e0 msgr2=0x55ef12ccefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.170+0000 7f6273470740 1 -- 172.21.15.2:0/516522085 shutdown_connections 2024-09-10T08:20:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.170+0000 7f6273470740 1 -- 172.21.15.2:0/516522085 wait complete. 2024-09-10T08:20:58.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.170+0000 7f6273470740 20 client.19071 trim_cache size 0 max 16384 2024-09-10T08:20:58.174 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:20:58.174 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:20:58.174 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:20:58.174 INFO:teuthology.orchestra.run.smithi002.stdout: "4aca35fd-0e1a-4792-8e8f-9ce86666a2f6": { 2024-09-10T08:20:58.174 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:20:58.174 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "4aca35fd-0e1a-4792-8e8f-9ce86666a2f6", 2024-09-10T08:20:58.174 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:20:58.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:58.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:20:58.175 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:20:58.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.170+0000 7f626f200640 1 -- 172.21.15.2:0/927814800 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62500a2b80 msgr2=0x7f62500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:58.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.170+0000 7f626f200640 1 --2- 172.21.15.2:0/927814800 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62500a2b80 0x7f62500a5040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f625c0cfb10 tx=0x7f625c0d5530 comp rx=0 tx=0).stop 2024-09-10T08:20:58.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.170+0000 7f626f200640 1 -- 172.21.15.2:0/927814800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6268103e50 msgr2=0x7f626819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:20:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.170+0000 7f626f200640 1 --2- 172.21.15.2:0/927814800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6268103e50 0x7f626819a550 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f625c00e9f0 tx=0x7f625c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.171+0000 7f626f200640 1 -- 172.21.15.2:0/927814800 shutdown_connections 2024-09-10T08:20:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.171+0000 7f626f200640 1 --2- 172.21.15.2:0/927814800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6268109090 0x7f626819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.171+0000 7f626f200640 1 --2- 172.21.15.2:0/927814800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6268104800 0x7f626819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.171+0000 7f626f200640 1 --2- 172.21.15.2:0/927814800 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f62500a2b80 0x7f62500a5040 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.171+0000 7f626f200640 1 --2- 172.21.15.2:0/927814800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6268103e50 0x7f626819a550 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:20:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.171+0000 7f626f200640 1 -- 172.21.15.2:0/927814800 >> 172.21.15.2:0/927814800 conn(0x7f62680ff8b0 msgr2=0x7f6268101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:20:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.171+0000 7f626f200640 1 -- 172.21.15.2:0/927814800 shutdown_connections 2024-09-10T08:20:58.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:20:58.171+0000 7f626f200640 1 -- 172.21.15.2:0/927814800 wait complete. 2024-09-10T08:20:58.984 INFO:tasks.cephfs.filesystem:scrub status for tag:4aca35fd-0e1a-4792-8e8f-9ce86666a2f6 - {'path': '/', 'tag': '4aca35fd-0e1a-4792-8e8f-9ce86666a2f6', 'options': 'recursive,force'} 2024-09-10T08:21:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:59 smithi002 ceph-mon[30563]: pgmap v772: 129 pgs: 129 active+clean; 2.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-10T08:21:00.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:20:59 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:21:00.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:59 smithi155 ceph-mon[35405]: pgmap v772: 129 pgs: 129 active+clean; 2.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-10T08:21:00.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:20:59 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:21:00.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:59 smithi107 ceph-mon[35048]: pgmap v772: 129 pgs: 129 active+clean; 2.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-10T08:21:00.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:20:59 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:21:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.032+0000 7ff43b400640 1 -- 172.21.15.2:0/3108530246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff434104f50 msgr2=0x7ff4341053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.032+0000 7ff43b400640 1 --2- 172.21.15.2:0/3108530246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff434104f50 0x7ff4341053d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009900 tx=0x7ff41c02a410 comp rx=0 tx=0).stop 2024-09-10T08:21:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.033+0000 7ff43b400640 1 -- 172.21.15.2:0/3108530246 shutdown_connections 2024-09-10T08:21:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.033+0000 7ff43b400640 1 --2- 172.21.15.2:0/3108530246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff434105910 0x7ff43410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.033+0000 7ff43b400640 1 --2- 172.21.15.2:0/3108530246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff434104f50 0x7ff4341053d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.033+0000 7ff43b400640 1 --2- 172.21.15.2:0/3108530246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff434103d60 0x7ff434104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.033+0000 7ff43b400640 1 -- 172.21.15.2:0/3108530246 >> 172.21.15.2:0/3108530246 conn(0x7ff4340ff4e0 msgr2=0x7ff434101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.033+0000 7ff43b400640 1 -- 172.21.15.2:0/3108530246 shutdown_connections 2024-09-10T08:21:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.033+0000 7ff43b400640 1 -- 172.21.15.2:0/3108530246 wait complete. 2024-09-10T08:21:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.034+0000 7ff43b400640 1 Processor -- start 2024-09-10T08:21:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.034+0000 7ff43b400640 1 -- start start 2024-09-10T08:21:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.034+0000 7ff43b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff434103d60 0x7ff434199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.034+0000 7ff43b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff434104f50 0x7ff43419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.034+0000 7ff43b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff434105910 0x7ff4341a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.034+0000 7ff43b400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410f9e0 con 0x7ff434105910 2024-09-10T08:21:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.034+0000 7ff43b400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410f860 con 0x7ff434103d60 2024-09-10T08:21:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.034+0000 7ff43b400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410fb60 con 0x7ff434104f50 2024-09-10T08:21:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.035+0000 7ff42be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff434104f50 0x7ff43419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.035+0000 7ff42be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff434104f50 0x7ff43419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50486/0 (socket says 172.21.15.2:50486) 2024-09-10T08:21:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.035+0000 7ff42be00640 1 -- 172.21.15.2:0/1702092633 learned_addr learned my addr 172.21.15.2:0/1702092633 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.035+0000 7ff439400640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff434105910 0x7ff4341a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.035+0000 7ff438a00640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff434103d60 0x7ff434199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.035+0000 7ff42be00640 1 -- 172.21.15.2:0/1702092633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff434103d60 msgr2=0x7ff434199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.035+0000 7ff42be00640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff434103d60 0x7ff434199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.035+0000 7ff42be00640 1 -- 172.21.15.2:0/1702092633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff434105910 msgr2=0x7ff4341a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.035+0000 7ff42be00640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff434105910 0x7ff4341a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.035+0000 7ff42be00640 1 -- 172.21.15.2:0/1702092633 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4341a1c00 con 0x7ff434104f50 2024-09-10T08:21:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.035+0000 7ff42be00640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff434104f50 0x7ff43419a520 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009230 tx=0x7ff41c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.035+0000 7ff432000640 1 -- 172.21.15.2:0/1702092633 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c03f030 con 0x7ff434104f50 2024-09-10T08:21:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.036+0000 7ff432000640 1 -- 172.21.15.2:0/1702092633 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff41c003ce0 con 0x7ff434104f50 2024-09-10T08:21:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.036+0000 7ff43b400640 1 -- 172.21.15.2:0/1702092633 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4341a1e90 con 0x7ff434104f50 2024-09-10T08:21:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.036+0000 7ff432000640 1 -- 172.21.15.2:0/1702092633 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c035070 con 0x7ff434104f50 2024-09-10T08:21:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.036+0000 7ff43b400640 1 -- 172.21.15.2:0/1702092633 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4341a22f0 con 0x7ff434104f50 2024-09-10T08:21:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.039+0000 7ff43e42a740 1 Processor -- start 2024-09-10T08:21:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.039+0000 7ff43e42a740 1 -- start start 2024-09-10T08:21:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.039+0000 7ff432000640 1 -- 172.21.15.2:0/1702092633 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff41c042030 con 0x7ff434104f50 2024-09-10T08:21:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.039+0000 7ff43e42a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x56230095c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.039+0000 7ff43e42a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c9d0 0x562300961490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.039+0000 7ff43e42a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562300961a60 0x562300963f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.039+0000 7ff43e42a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5623008c1100 con 0x562300961a60 2024-09-10T08:21:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.039+0000 7ff43e42a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5623008691b0 con 0x56230095c9d0 2024-09-10T08:21:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.039+0000 7ff43e42a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5623008bd0c0 con 0x56230095c090 2024-09-10T08:21:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.040+0000 7ff439400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x56230095c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.040+0000 7ff439400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x56230095c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50494/0 (socket says 172.21.15.2:50494) 2024-09-10T08:21:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.040+0000 7ff439400640 1 -- 172.21.15.2:0/1929576789 learned_addr learned my addr 172.21.15.2:0/1929576789 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.040+0000 7ff42be00640 1 --2- 172.21.15.2:0/1929576789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562300961a60 0x562300963f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.040+0000 7ff438a00640 1 --2- 172.21.15.2:0/1929576789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c9d0 0x562300961490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.040+0000 7ff439400640 1 -- 172.21.15.2:0/1929576789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c9d0 msgr2=0x562300961490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.040+0000 7ff439400640 1 --2- 172.21.15.2:0/1929576789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c9d0 0x562300961490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.040+0000 7ff439400640 1 -- 172.21.15.2:0/1929576789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562300961a60 msgr2=0x562300963f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.040+0000 7ff439400640 1 --2- 172.21.15.2:0/1929576789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562300961a60 0x562300963f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.040+0000 7ff439400640 1 -- 172.21.15.2:0/1929576789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623005d4210 con 0x56230095c090 2024-09-10T08:21:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.040+0000 7ff438a00640 1 --2- 172.21.15.2:0/1929576789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c9d0 0x562300961490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:21:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.040+0000 7ff439400640 1 --2- 172.21.15.2:0/1929576789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x56230095c490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009fd0 tx=0x7ff42c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35f6d0a9661fc826 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.041+0000 7ff429c00640 1 -- 172.21.15.2:0/1929576789 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c004d20 con 0x56230095c090 2024-09-10T08:21:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.041+0000 7ff429c00640 1 -- 172.21.15.2:0/1929576789 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff42c004050 con 0x56230095c090 2024-09-10T08:21:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.041+0000 7ff429c00640 1 -- 172.21.15.2:0/1929576789 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c005220 con 0x56230095c090 2024-09-10T08:21:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.041+0000 7ff432000640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4180a2c30 0x7ff4180a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.041+0000 7ff43e42a740 1 -- 172.21.15.2:0/1929576789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 msgr2=0x56230095c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.041+0000 7ff43e42a740 1 --2- 172.21.15.2:0/1929576789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x56230095c490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009fd0 tx=0x7ff42c0118d0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.041+0000 7ff43e42a740 1 -- 172.21.15.2:0/1929576789 shutdown_connections 2024-09-10T08:21:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.041+0000 7ff43e42a740 1 --2- 172.21.15.2:0/1929576789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562300961a60 0x562300963f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.041+0000 7ff43e42a740 1 --2- 172.21.15.2:0/1929576789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c9d0 0x562300961490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.041+0000 7ff43e42a740 1 --2- 172.21.15.2:0/1929576789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x56230095c490 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.041+0000 7ff43e42a740 1 -- 172.21.15.2:0/1929576789 >> 172.21.15.2:0/1929576789 conn(0x562300957700 msgr2=0x562300957b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.041+0000 7ff432000640 1 -- 172.21.15.2:0/1702092633 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff41c0ef850 con 0x7ff434104f50 2024-09-10T08:21:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.041+0000 7ff438a00640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4180a2c30 0x7ff4180a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.042+0000 7ff43e42a740 1 -- 172.21.15.2:0/1929576789 shutdown_connections 2024-09-10T08:21:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.042+0000 7ff43e42a740 1 -- 172.21.15.2:0/1929576789 wait complete. 2024-09-10T08:21:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.042+0000 7ff43e42a740 1 -- start start 2024-09-10T08:21:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.042+0000 7ff438a00640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4180a2c30 0x7ff4180a50f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x562300958580 tx=0x7ff424005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.042+0000 7ff43e42a740 1 Processor -- start 2024-09-10T08:21:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.043+0000 7ff43e42a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x562300966220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.043+0000 7ff43e42a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 0x562300966760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.043+0000 7ff43e42a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562300961a60 0x56230096d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.043+0000 7ff43e42a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562300964a30 con 0x56230095c9d0 2024-09-10T08:21:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.043+0000 7ff43e42a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56230084a090 con 0x562300961a60 2024-09-10T08:21:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.043+0000 7ff43e42a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562300964bb0 con 0x56230095c090 2024-09-10T08:21:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.043+0000 7ff43e42a740 10 client.0 _ll_register_callbacks cb 0x56230089ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.043+0000 7ff42be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 0x562300966760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.043+0000 7ff438a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x562300966220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.043+0000 7ff438a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x562300966220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50508/0 (socket says 172.21.15.2:50508) 2024-09-10T08:21:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.043+0000 7ff42be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 0x562300966760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57280/0 (socket says 172.21.15.2:57280) 2024-09-10T08:21:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.043+0000 7ff438a00640 1 -- 172.21.15.2:0/3547369072 learned_addr learned my addr 172.21.15.2:0/3547369072 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.043+0000 7ff439400640 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562300961a60 0x56230096d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.044+0000 7ff438a00640 1 -- 172.21.15.2:0/3547369072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562300961a60 msgr2=0x56230096d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.044+0000 7ff438a00640 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562300961a60 0x56230096d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.044+0000 7ff438a00640 1 -- 172.21.15.2:0/3547369072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 msgr2=0x562300966760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.044+0000 7ff438a00640 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 0x562300966760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.044+0000 7ff438a00640 1 -- 172.21.15.2:0/3547369072 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623006a0620 con 0x56230095c090 2024-09-10T08:21:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.044+0000 7ff439400640 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562300961a60 0x56230096d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.044+0000 7ff42be00640 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 0x562300966760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.044+0000 7ff438a00640 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x562300966220 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff424004250 tx=0x7ff424004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5496ade40e8b4d45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.044+0000 7ff422000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:21:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.044+0000 7ff422000640 1 -- 172.21.15.2:0/3547369072 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424029440 con 0x56230095c090 2024-09-10T08:21:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.044+0000 7ff422000640 1 -- 172.21.15.2:0/3547369072 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff42401f430 con 0x56230095c090 2024-09-10T08:21:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.044+0000 7ff43e42a740 1 -- 172.21.15.2:0/3547369072 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff42c004050 con 0x56230095c090 2024-09-10T08:21:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.045+0000 7ff422000640 1 -- 172.21.15.2:0/3547369072 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff42401f630 con 0x56230095c090 2024-09-10T08:21:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.045+0000 7ff43e42a740 10 client.29041 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.045+0000 7ff43e42a740 1 -- 172.21.15.2:0/3547369072 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5623009706b0 con 0x56230095c090 2024-09-10T08:21:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.045+0000 7ff422000640 1 -- 172.21.15.2:0/3547369072 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff42c004050 con 0x56230095c090 2024-09-10T08:21:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.045+0000 7ff422000640 1 -- 172.21.15.2:0/3547369072 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff424027070 con 0x56230095c090 2024-09-10T08:21:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.045+0000 7ff43e42a740 10 client.29041 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.045+0000 7ff43e42a740 10 client.29041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.045+0000 7ff43e42a740 20 client.29041 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.045+0000 7ff43e42a740 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562300971df0 0x562300974240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.045+0000 7ff43e42a740 4 client.29041 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.045+0000 7ff43e42a740 1 -- 172.21.15.2:0/3547369072 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623008ce830 con 0x562300971df0 2024-09-10T08:21:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.046+0000 7ff42be00640 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562300971df0 0x562300974240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.046+0000 7ff42be00640 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562300971df0 0x562300974240 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d15fbdb303c2d5ff server_cookie=c678f14dc273c4f9 in_seq=0 out_seq=0 2024-09-10T08:21:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.046+0000 7ff422000640 10 client.29041 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.047+0000 7ff422000640 1 -- 172.21.15.2:0/3547369072 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623008ce830 con 0x562300971df0 2024-09-10T08:21:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.047+0000 7ff422000640 10 client.29041 handle_command_reply: tid=0 2024-09-10T08:21:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.047+0000 7ff43e42a740 1 client.29041 shutdown 2024-09-10T08:21:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.047+0000 7ff43e42a740 1 -- 172.21.15.2:0/3547369072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 msgr2=0x562300966220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.047+0000 7ff43e42a740 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x562300966220 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff424004250 tx=0x7ff424004730 comp rx=0 tx=0).stop 2024-09-10T08:21:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.047+0000 7ff43e42a740 1 -- 172.21.15.2:0/3547369072 shutdown_connections 2024-09-10T08:21:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.047+0000 7ff43e42a740 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562300971df0 0x562300974240 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.047+0000 7ff43e42a740 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562300961a60 0x56230096d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.047+0000 7ff43e42a740 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 0x562300966760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.047+0000 7ff422000640 0 client.29041 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.047+0000 7ff43e42a740 1 --2- 172.21.15.2:0/3547369072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x562300966220 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.047+0000 7ff43e42a740 1 -- 172.21.15.2:0/3547369072 >> 172.21.15.2:0/3547369072 conn(0x562300957700 msgr2=0x562300958f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.047+0000 7ff43e42a740 1 -- 172.21.15.2:0/3547369072 shutdown_connections 2024-09-10T08:21:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.048+0000 7ff43e42a740 1 -- 172.21.15.2:0/3547369072 wait complete. 2024-09-10T08:21:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.048+0000 7ff43e42a740 20 client.29041 trim_cache size 0 max 16384 2024-09-10T08:21:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.063+0000 7ff43e42a740 1 Processor -- start 2024-09-10T08:21:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.063+0000 7ff43e42a740 1 -- start start 2024-09-10T08:21:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.063+0000 7ff43e42a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562300957700 0x5623009666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.063+0000 7ff43e42a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x56230096d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.063+0000 7ff43e42a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c9d0 0x5623009526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.063+0000 7ff43e42a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341af2e0 con 0x562300957700 2024-09-10T08:21:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.063+0000 7ff43e42a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341acbb0 con 0x56230095c9d0 2024-09-10T08:21:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.063+0000 7ff43e42a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341aaa00 con 0x56230095c090 2024-09-10T08:21:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.063+0000 7ff438a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562300957700 0x5623009666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.063+0000 7ff438a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562300957700 0x5623009666e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57292/0 (socket says 172.21.15.2:57292) 2024-09-10T08:21:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.063+0000 7ff438a00640 1 -- 172.21.15.2:0/2247415966 learned_addr learned my addr 172.21.15.2:0/2247415966 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.064+0000 7ff42be00640 1 --2- 172.21.15.2:0/2247415966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x56230096d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.064+0000 7ff439400640 1 --2- 172.21.15.2:0/2247415966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c9d0 0x5623009526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.064+0000 7ff438a00640 1 -- 172.21.15.2:0/2247415966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 msgr2=0x56230096d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.064+0000 7ff438a00640 1 --2- 172.21.15.2:0/2247415966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x56230096d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.064+0000 7ff438a00640 1 -- 172.21.15.2:0/2247415966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c9d0 msgr2=0x5623009526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.064+0000 7ff438a00640 1 --2- 172.21.15.2:0/2247415966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c9d0 0x5623009526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.064+0000 7ff438a00640 1 -- 172.21.15.2:0/2247415966 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623009583b0 con 0x562300957700 2024-09-10T08:21:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.064+0000 7ff439400640 1 --2- 172.21.15.2:0/2247415966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c9d0 0x5623009526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:21:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.064+0000 7ff438a00640 1 --2- 172.21.15.2:0/2247415966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562300957700 0x5623009666e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff424019040 tx=0x7ff424009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c83938212879c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.064+0000 7ff429200640 1 -- 172.21.15.2:0/2247415966 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42403c070 con 0x562300957700 2024-09-10T08:21:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.064+0000 7ff429200640 1 -- 172.21.15.2:0/2247415966 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff424029920 con 0x562300957700 2024-09-10T08:21:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.064+0000 7ff429200640 1 -- 172.21.15.2:0/2247415966 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42401d6a0 con 0x562300957700 2024-09-10T08:21:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.065+0000 7ff43e42a740 1 -- 172.21.15.2:0/2247415966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562300957700 msgr2=0x5623009666e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.065+0000 7ff43e42a740 1 --2- 172.21.15.2:0/2247415966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562300957700 0x5623009666e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff424019040 tx=0x7ff424009920 comp rx=0 tx=0).stop 2024-09-10T08:21:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.065+0000 7ff43e42a740 1 -- 172.21.15.2:0/2247415966 shutdown_connections 2024-09-10T08:21:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.065+0000 7ff43e42a740 1 --2- 172.21.15.2:0/2247415966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c9d0 0x5623009526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.065+0000 7ff43e42a740 1 --2- 172.21.15.2:0/2247415966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56230095c090 0x56230096d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.065+0000 7ff43e42a740 1 --2- 172.21.15.2:0/2247415966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562300957700 0x5623009666e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.065+0000 7ff43e42a740 1 -- 172.21.15.2:0/2247415966 >> 172.21.15.2:0/2247415966 conn(0x562300961580 msgr2=0x562300965fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.065+0000 7ff43e42a740 1 -- 172.21.15.2:0/2247415966 shutdown_connections 2024-09-10T08:21:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.065+0000 7ff43e42a740 1 -- 172.21.15.2:0/2247415966 wait complete. 2024-09-10T08:21:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.065+0000 7ff43e42a740 1 -- start start 2024-09-10T08:21:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.066+0000 7ff43e42a740 1 Processor -- start 2024-09-10T08:21:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.066+0000 7ff43e42a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562300957700 0x56230096a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.066+0000 7ff43e42a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c090 0x56230096aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.066+0000 7ff43e42a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 0x56230096be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.066+0000 7ff43e42a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341ab070 con 0x56230095c9d0 2024-09-10T08:21:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.066+0000 7ff43e42a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341ac4e0 con 0x56230095c090 2024-09-10T08:21:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.066+0000 7ff43e42a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341ab740 con 0x562300957700 2024-09-10T08:21:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.066+0000 7ff43e42a740 10 client.0 _ll_register_callbacks cb 0x7ff424012a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.066+0000 7ff439400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 0x56230096be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.066+0000 7ff439400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 0x56230096be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57294/0 (socket says 172.21.15.2:57294) 2024-09-10T08:21:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.066+0000 7ff439400640 1 -- 172.21.15.2:0/3927481044 learned_addr learned my addr 172.21.15.2:0/3927481044 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.066+0000 7ff438a00640 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562300957700 0x56230096a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.066+0000 7ff42be00640 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c090 0x56230096aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.067+0000 7ff439400640 1 -- 172.21.15.2:0/3927481044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562300957700 msgr2=0x56230096a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.067+0000 7ff439400640 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562300957700 0x56230096a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.067+0000 7ff439400640 1 -- 172.21.15.2:0/3927481044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c090 msgr2=0x56230096aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.067+0000 7ff439400640 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c090 0x56230096aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.067+0000 7ff439400640 1 -- 172.21.15.2:0/3927481044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56230096c430 con 0x56230095c9d0 2024-09-10T08:21:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.067+0000 7ff438a00640 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562300957700 0x56230096a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.067+0000 7ff439400640 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 0x56230096be20 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009f70 tx=0x7ff42c003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d67c4ce39b4ff6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.067+0000 7ff422000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.067+0000 7ff422000640 1 -- 172.21.15.2:0/3927481044 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c004260 con 0x56230095c9d0 2024-09-10T08:21:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.067+0000 7ff422000640 1 -- 172.21.15.2:0/3927481044 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff42c004400 con 0x56230095c9d0 2024-09-10T08:21:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.067+0000 7ff43e42a740 1 -- 172.21.15.2:0/3927481044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff424029920 con 0x56230095c9d0 2024-09-10T08:21:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.068+0000 7ff422000640 1 -- 172.21.15.2:0/3927481044 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff42c035050 con 0x56230095c9d0 2024-09-10T08:21:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.068+0000 7ff43e42a740 10 client.19089 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.068+0000 7ff43e42a740 1 -- 172.21.15.2:0/3927481044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56230096d7b0 con 0x56230095c9d0 2024-09-10T08:21:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.068+0000 7ff422000640 1 -- 172.21.15.2:0/3927481044 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff424029920 con 0x56230095c9d0 2024-09-10T08:21:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.068+0000 7ff422000640 1 -- 172.21.15.2:0/3927481044 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff42c00f4e0 con 0x56230095c9d0 2024-09-10T08:21:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.068+0000 7ff43e42a740 10 client.19089 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.068+0000 7ff43e42a740 10 client.19089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.068+0000 7ff43e42a740 20 client.19089 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.068+0000 7ff43e42a740 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562300961a60 0x5623009deea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.068+0000 7ff43e42a740 4 client.19089 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:21:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.068+0000 7ff43e42a740 1 -- 172.21.15.2:0/3927481044 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562300952d80 con 0x562300961a60 2024-09-10T08:21:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.068+0000 7ff438a00640 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562300961a60 0x5623009deea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.069+0000 7ff438a00640 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562300961a60 0x5623009deea0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbdecec9523cf8f server_cookie=c2745c6a40f04c97 in_seq=0 out_seq=0 2024-09-10T08:21:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.069+0000 7ff422000640 10 client.19089 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff422000640 1 -- 172.21.15.2:0/3927481044 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562300952d80 con 0x562300961a60 2024-09-10T08:21:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff422000640 10 client.19089 handle_command_reply: tid=0 2024-09-10T08:21:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff43e42a740 1 client.19089 shutdown 2024-09-10T08:21:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff43e42a740 1 -- 172.21.15.2:0/3927481044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 msgr2=0x56230096be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff43e42a740 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 0x56230096be20 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009f70 tx=0x7ff42c003f70 comp rx=0 tx=0).stop 2024-09-10T08:21:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff43e42a740 1 -- 172.21.15.2:0/3927481044 shutdown_connections 2024-09-10T08:21:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff43e42a740 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562300961a60 0x5623009deea0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff43e42a740 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230095c9d0 0x56230096be20 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff43e42a740 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56230095c090 0x56230096aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff43e42a740 1 --2- 172.21.15.2:0/3927481044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562300957700 0x56230096a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff422000640 0 client.19089 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff43e42a740 1 -- 172.21.15.2:0/3927481044 >> 172.21.15.2:0/3927481044 conn(0x562300961580 msgr2=0x562300956060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff43e42a740 1 -- 172.21.15.2:0/3927481044 shutdown_connections 2024-09-10T08:21:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff43e42a740 1 -- 172.21.15.2:0/3927481044 wait complete. 2024-09-10T08:21:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.082+0000 7ff43e42a740 20 client.19089 trim_cache size 0 max 16384 2024-09-10T08:21:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:21:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:21:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:21:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:21:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.083+0000 7ff43b400640 1 -- 172.21.15.2:0/1702092633 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4180a2c30 msgr2=0x7ff4180a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.083+0000 7ff43b400640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4180a2c30 0x7ff4180a50f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x562300958580 tx=0x7ff424005f90 comp rx=0 tx=0).stop 2024-09-10T08:21:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.083+0000 7ff43b400640 1 -- 172.21.15.2:0/1702092633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff434104f50 msgr2=0x7ff43419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.083+0000 7ff43b400640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff434104f50 0x7ff43419a520 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009230 tx=0x7ff41c0042f0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.083+0000 7ff43b400640 1 -- 172.21.15.2:0/1702092633 shutdown_connections 2024-09-10T08:21:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.083+0000 7ff43b400640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff434105910 0x7ff4341a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.083+0000 7ff43b400640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff4180a2c30 0x7ff4180a50f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.083+0000 7ff43b400640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff434104f50 0x7ff43419a520 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.083+0000 7ff43b400640 1 --2- 172.21.15.2:0/1702092633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff434103d60 0x7ff434199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.083+0000 7ff43b400640 1 -- 172.21.15.2:0/1702092633 >> 172.21.15.2:0/1702092633 conn(0x7ff4340ff4e0 msgr2=0x7ff434108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.083+0000 7ff43b400640 1 -- 172.21.15.2:0/1702092633 shutdown_connections 2024-09-10T08:21:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:01.083+0000 7ff43b400640 1 -- 172.21.15.2:0/1702092633 wait complete. 2024-09-10T08:21:02.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:01 smithi002 ceph-mon[30563]: pgmap v773: 129 pgs: 129 active+clean; 2.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:21:02.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:01 smithi155 ceph-mon[35405]: pgmap v773: 129 pgs: 129 active+clean; 2.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:21:02.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:01 smithi107 ceph-mon[35048]: pgmap v773: 129 pgs: 129 active+clean; 2.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:21:04.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:03 smithi002 ceph-mon[30563]: pgmap v774: 129 pgs: 129 active+clean; 2.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:21:04.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:03 smithi155 ceph-mon[35405]: pgmap v774: 129 pgs: 129 active+clean; 2.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:21:04.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:03 smithi107 ceph-mon[35048]: pgmap v774: 129 pgs: 129 active+clean; 2.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:21:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:05 smithi002 ceph-mon[30563]: pgmap v775: 129 pgs: 129 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:21:06.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:05 smithi155 ceph-mon[35405]: pgmap v775: 129 pgs: 129 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:21:06.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:05 smithi107 ceph-mon[35048]: pgmap v775: 129 pgs: 129 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:21:08.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:07 smithi002 ceph-mon[30563]: pgmap v776: 129 pgs: 129 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:21:08.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:07 smithi155 ceph-mon[35405]: pgmap v776: 129 pgs: 129 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:21:08.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:07 smithi107 ceph-mon[35048]: pgmap v776: 129 pgs: 129 active+clean; 2.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:21:10.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:09 smithi002 ceph-mon[30563]: pgmap v777: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:21:10.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:09 smithi155 ceph-mon[35405]: pgmap v777: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:21:10.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:09 smithi107 ceph-mon[35048]: pgmap v777: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:21:12.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:11 smithi002 ceph-mon[30563]: pgmap v778: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:21:12.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:11 smithi155 ceph-mon[35405]: pgmap v778: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:21:12.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:11 smithi107 ceph-mon[35048]: pgmap v778: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:21:14.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:13 smithi002 ceph-mon[30563]: pgmap v779: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:21:14.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:13 smithi155 ceph-mon[35405]: pgmap v779: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:21:14.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:13 smithi107 ceph-mon[35048]: pgmap v779: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:21:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.209+0000 7f9597600640 1 -- 172.21.15.2:0/2340567613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9590074bf0 msgr2=0x7f9590075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.209+0000 7f9597600640 1 --2- 172.21.15.2:0/2340567613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9590074bf0 0x7f9590075050 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f957c009900 tx=0x7f957c02a480 comp rx=0 tx=0).stop 2024-09-10T08:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.210+0000 7f9597600640 1 -- 172.21.15.2:0/2340567613 shutdown_connections 2024-09-10T08:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.210+0000 7f9597600640 1 --2- 172.21.15.2:0/2340567613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95901099d0 0x7f959010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.210+0000 7f9597600640 1 --2- 172.21.15.2:0/2340567613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9590074bf0 0x7f9590075050 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.210+0000 7f9597600640 1 --2- 172.21.15.2:0/2340567613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95900761d0 0x7f9590074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.210+0000 7f9597600640 1 -- 172.21.15.2:0/2340567613 >> 172.21.15.2:0/2340567613 conn(0x7f95900fd2e0 msgr2=0x7f95900ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.210+0000 7f9597600640 1 -- 172.21.15.2:0/2340567613 shutdown_connections 2024-09-10T08:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.210+0000 7f9597600640 1 -- 172.21.15.2:0/2340567613 wait complete. 2024-09-10T08:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.211+0000 7f9597600640 1 Processor -- start 2024-09-10T08:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.211+0000 7f9597600640 1 -- start start 2024-09-10T08:21:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.211+0000 7f9597600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9590074bf0 0x7f9590199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.211+0000 7f9597600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95900761d0 0x7f959019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.211+0000 7f9597600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95901099d0 0x7f95901a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.211+0000 7f9597600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f959010f9b0 con 0x7f95900761d0 2024-09-10T08:21:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.211+0000 7f9597600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f959010f830 con 0x7f95901099d0 2024-09-10T08:21:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.211+0000 7f9597600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f959010fb30 con 0x7f9590074bf0 2024-09-10T08:21:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.211+0000 7f958fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95900761d0 0x7f959019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.211+0000 7f9594c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9590074bf0 0x7f9590199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.211+0000 7f958fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95900761d0 0x7f959019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38040/0 (socket says 172.21.15.2:38040) 2024-09-10T08:21:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.211+0000 7f958fe00640 1 -- 172.21.15.2:0/2208614560 learned_addr learned my addr 172.21.15.2:0/2208614560 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.212+0000 7f9595600640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95901099d0 0x7f95901a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.212+0000 7f9594c00640 1 -- 172.21.15.2:0/2208614560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95901099d0 msgr2=0x7f95901a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.212+0000 7f9594c00640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95901099d0 0x7f95901a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.212+0000 7f9594c00640 1 -- 172.21.15.2:0/2208614560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95900761d0 msgr2=0x7f959019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.212+0000 7f9594c00640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95900761d0 0x7f959019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.212+0000 7f9594c00640 1 -- 172.21.15.2:0/2208614560 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95901a1bd0 con 0x7f9590074bf0 2024-09-10T08:21:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.212+0000 7f958fe00640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95900761d0 0x7f959019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.212+0000 7f9594c00640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9590074bf0 0x7f9590199fb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f958400e9f0 tx=0x7f958400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.212+0000 7f958d600640 1 -- 172.21.15.2:0/2208614560 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958400cd80 con 0x7f9590074bf0 2024-09-10T08:21:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.212+0000 7f958d600640 1 -- 172.21.15.2:0/2208614560 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95840035f0 con 0x7f9590074bf0 2024-09-10T08:21:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.212+0000 7f958d600640 1 -- 172.21.15.2:0/2208614560 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584010690 con 0x7f9590074bf0 2024-09-10T08:21:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.213+0000 7f9597600640 1 -- 172.21.15.2:0/2208614560 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95901a1ec0 con 0x7f9590074bf0 2024-09-10T08:21:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.213+0000 7f9597600640 1 -- 172.21.15.2:0/2208614560 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95901a2320 con 0x7f9590074bf0 2024-09-10T08:21:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.215+0000 7f959a464740 1 Processor -- start 2024-09-10T08:21:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.216+0000 7f959a464740 1 -- start start 2024-09-10T08:21:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.216+0000 7f959a464740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2d9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.216+0000 7f959a464740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2d99d0 0x561cfd2de490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.216+0000 7f959a464740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2dea60 0x561cfd2e0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.216+0000 7f959a464740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561cfd23e100 con 0x561cfd2d99d0 2024-09-10T08:21:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.216+0000 7f959a464740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561cfd1e61b0 con 0x561cfd2d9090 2024-09-10T08:21:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.216+0000 7f959a464740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561cfd23a0c0 con 0x561cfd2dea60 2024-09-10T08:21:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.216+0000 7f958d600640 1 -- 172.21.15.2:0/2208614560 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9584003b30 con 0x7f9590074bf0 2024-09-10T08:21:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.216+0000 7f958fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2dea60 0x561cfd2e0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.216+0000 7f9595600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2d9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.216+0000 7f958fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2dea60 0x561cfd2e0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:53504/0 (socket says 172.21.15.2:53504) 2024-09-10T08:21:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.216+0000 7f958fe00640 1 -- 172.21.15.2:0/3696660259 learned_addr learned my addr 172.21.15.2:0/3696660259 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.216+0000 7f9594c00640 1 --2- 172.21.15.2:0/3696660259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2d99d0 0x561cfd2de490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.217+0000 7f9595600640 1 -- 172.21.15.2:0/3696660259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2dea60 msgr2=0x561cfd2e0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.217+0000 7f9595600640 1 --2- 172.21.15.2:0/3696660259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2dea60 0x561cfd2e0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.217+0000 7f9595600640 1 -- 172.21.15.2:0/3696660259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2d99d0 msgr2=0x561cfd2de490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.217+0000 7f9595600640 1 --2- 172.21.15.2:0/3696660259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2d99d0 0x561cfd2de490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.217+0000 7f9595600640 1 -- 172.21.15.2:0/3696660259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cfcf51210 con 0x561cfd2d9090 2024-09-10T08:21:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.217+0000 7f958fe00640 1 --2- 172.21.15.2:0/3696660259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2dea60 0x561cfd2e0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.217+0000 7f9595600640 1 --2- 172.21.15.2:0/3696660259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2d9490 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9588009f30 tx=0x7f9588010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f898f4452139fdba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.217+0000 7f9581c00640 1 -- 172.21.15.2:0/3696660259 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588016070 con 0x561cfd2d9090 2024-09-10T08:21:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.217+0000 7f9581c00640 1 -- 172.21.15.2:0/3696660259 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9588004d20 con 0x561cfd2d9090 2024-09-10T08:21:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.217+0000 7f9581c00640 1 -- 172.21.15.2:0/3696660259 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588012870 con 0x561cfd2d9090 2024-09-10T08:21:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.218+0000 7f959a464740 1 -- 172.21.15.2:0/3696660259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 msgr2=0x561cfd2d9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.218+0000 7f959a464740 1 --2- 172.21.15.2:0/3696660259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2d9490 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9588009f30 tx=0x7f9588010800 comp rx=0 tx=0).stop 2024-09-10T08:21:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.218+0000 7f959a464740 1 -- 172.21.15.2:0/3696660259 shutdown_connections 2024-09-10T08:21:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.218+0000 7f959a464740 1 --2- 172.21.15.2:0/3696660259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2dea60 0x561cfd2e0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.218+0000 7f959a464740 1 --2- 172.21.15.2:0/3696660259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2d99d0 0x561cfd2de490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.218+0000 7f959a464740 1 --2- 172.21.15.2:0/3696660259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2d9490 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.218+0000 7f959a464740 1 -- 172.21.15.2:0/3696660259 >> 172.21.15.2:0/3696660259 conn(0x561cfd2d4700 msgr2=0x561cfd2d4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.218+0000 7f958d600640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f95780a2c30 0x7f95780a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.218+0000 7f959a464740 1 -- 172.21.15.2:0/3696660259 shutdown_connections 2024-09-10T08:21:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.218+0000 7f959a464740 1 -- 172.21.15.2:0/3696660259 wait complete. 2024-09-10T08:21:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.218+0000 7f958d600640 1 -- 172.21.15.2:0/2208614560 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f95840cf710 con 0x7f9590074bf0 2024-09-10T08:21:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.219+0000 7f958fe00640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f95780a2c30 0x7f95780a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.219+0000 7f959a464740 1 -- start start 2024-09-10T08:21:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.219+0000 7f959a464740 1 Processor -- start 2024-09-10T08:21:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.219+0000 7f958fe00640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f95780a2c30 0x7f95780a50f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f957c000c00 tx=0x7f957c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.220+0000 7f959a464740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2e3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.220+0000 7f959a464740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2d99d0 0x561cfd2e37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.220+0000 7f959a464740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2dea60 0x561cfd2ea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.220+0000 7f959a464740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561cfd2e1a30 con 0x561cfd2d99d0 2024-09-10T08:21:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.220+0000 7f959a464740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561cfd1c7090 con 0x561cfd2d9090 2024-09-10T08:21:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.220+0000 7f959a464740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561cfd2e1bb0 con 0x561cfd2dea60 2024-09-10T08:21:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.220+0000 7f959a464740 10 client.0 _ll_register_callbacks cb 0x561cfd21bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.220+0000 7f958fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2d99d0 0x561cfd2e37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.220+0000 7f958fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2d99d0 0x561cfd2e37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38046/0 (socket says 172.21.15.2:38046) 2024-09-10T08:21:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.220+0000 7f958fe00640 1 -- 172.21.15.2:0/1627862890 learned_addr learned my addr 172.21.15.2:0/1627862890 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.220+0000 7f9594c00640 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2e3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.220+0000 7f9595600640 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2dea60 0x561cfd2ea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f9594c00640 1 -- 172.21.15.2:0/1627862890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2dea60 msgr2=0x561cfd2ea7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f9594c00640 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2dea60 0x561cfd2ea7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f9594c00640 1 -- 172.21.15.2:0/1627862890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2d99d0 msgr2=0x561cfd2e37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f9594c00640 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2d99d0 0x561cfd2e37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f9594c00640 1 -- 172.21.15.2:0/1627862890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cfd01d620 con 0x561cfd2d9090 2024-09-10T08:21:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f958fe00640 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2d99d0 0x561cfd2e37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f9595600640 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2dea60 0x561cfd2ea7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f9594c00640 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2e3270 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f95840cce70 tx=0x7f9584097e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d4b4dd260988c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f9576000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:21:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f9576000640 1 -- 172.21.15.2:0/1627862890 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958409b4e0 con 0x561cfd2d9090 2024-09-10T08:21:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f9576000640 1 -- 172.21.15.2:0/1627862890 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f95840981f0 con 0x561cfd2d9090 2024-09-10T08:21:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f959a464740 1 -- 172.21.15.2:0/1627862890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9588004d20 con 0x561cfd2d9090 2024-09-10T08:21:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f9576000640 1 -- 172.21.15.2:0/1627862890 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f958409c280 con 0x561cfd2d9090 2024-09-10T08:21:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f959a464740 10 client.29129 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.221+0000 7f959a464740 1 -- 172.21.15.2:0/1627862890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561cfd2ed700 con 0x561cfd2d9090 2024-09-10T08:21:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.222+0000 7f9576000640 1 -- 172.21.15.2:0/1627862890 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9588004d20 con 0x561cfd2d9090 2024-09-10T08:21:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.222+0000 7f9576000640 1 -- 172.21.15.2:0/1627862890 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f95840cf3d0 con 0x561cfd2d9090 2024-09-10T08:21:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.222+0000 7f959a464740 10 client.29129 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.222+0000 7f959a464740 10 client.29129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.222+0000 7f959a464740 20 client.29129 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.222+0000 7f959a464740 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561cfd2eee40 0x561cfd2f1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.222+0000 7f959a464740 4 client.29129 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.222+0000 7f959a464740 1 -- 172.21.15.2:0/1627862890 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cfd24b830 con 0x561cfd2eee40 2024-09-10T08:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.222+0000 7f958fe00640 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561cfd2eee40 0x561cfd2f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.223+0000 7f958fe00640 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561cfd2eee40 0x561cfd2f1230 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad2dcc05fea6ca90 server_cookie=ce130dbf5190275 in_seq=0 out_seq=0 2024-09-10T08:21:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.223+0000 7f9576000640 10 client.29129 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f9576000640 1 -- 172.21.15.2:0/1627862890 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cfd24b830 con 0x561cfd2eee40 2024-09-10T08:21:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f9576000640 10 client.29129 handle_command_reply: tid=0 2024-09-10T08:21:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f959a464740 1 client.29129 shutdown 2024-09-10T08:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f959a464740 1 -- 172.21.15.2:0/1627862890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 msgr2=0x561cfd2e3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f959a464740 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2e3270 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f95840cce70 tx=0x7f9584097e20 comp rx=0 tx=0).stop 2024-09-10T08:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f959a464740 1 -- 172.21.15.2:0/1627862890 shutdown_connections 2024-09-10T08:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f959a464740 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561cfd2eee40 0x561cfd2f1230 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f959a464740 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2dea60 0x561cfd2ea7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f9576000640 0 client.29129 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f959a464740 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2d99d0 0x561cfd2e37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f959a464740 1 --2- 172.21.15.2:0/1627862890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2e3270 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f959a464740 1 -- 172.21.15.2:0/1627862890 >> 172.21.15.2:0/1627862890 conn(0x561cfd2d4700 msgr2=0x561cfd2d5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f959a464740 1 -- 172.21.15.2:0/1627862890 shutdown_connections 2024-09-10T08:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f959a464740 1 -- 172.21.15.2:0/1627862890 wait complete. 2024-09-10T08:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.224+0000 7f959a464740 20 client.29129 trim_cache size 0 max 16384 2024-09-10T08:21:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.240+0000 7f959a464740 1 Processor -- start 2024-09-10T08:21:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.240+0000 7f959a464740 1 -- start start 2024-09-10T08:21:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.240+0000 7f959a464740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2d9090 0x561cfd2e3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.240+0000 7f959a464740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d99d0 0x561cfd2eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.240+0000 7f959a464740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 0x561cfd2ea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.240+0000 7f959a464740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f95901af320 con 0x561cfd2dea60 2024-09-10T08:21:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.240+0000 7f959a464740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95901acbf0 con 0x561cfd2d99d0 2024-09-10T08:21:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.240+0000 7f959a464740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f95901aaa40 con 0x561cfd2d9090 2024-09-10T08:21:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.240+0000 7f9595600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 0x561cfd2ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.240+0000 7f958fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d99d0 0x561cfd2eaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.240+0000 7f9594c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2d9090 0x561cfd2e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.240+0000 7f9595600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 0x561cfd2ea780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38050/0 (socket says 172.21.15.2:38050) 2024-09-10T08:21:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.240+0000 7f9595600640 1 -- 172.21.15.2:0/4100219362 learned_addr learned my addr 172.21.15.2:0/4100219362 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.241+0000 7f9594c00640 1 -- 172.21.15.2:0/4100219362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d99d0 msgr2=0x561cfd2eaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.241+0000 7f9594c00640 1 --2- 172.21.15.2:0/4100219362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d99d0 0x561cfd2eaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.241+0000 7f9594c00640 1 -- 172.21.15.2:0/4100219362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 msgr2=0x561cfd2ea780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.241+0000 7f9594c00640 1 --2- 172.21.15.2:0/4100219362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 0x561cfd2ea780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.241+0000 7f9594c00640 1 -- 172.21.15.2:0/4100219362 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cfd2d9eb0 con 0x561cfd2d9090 2024-09-10T08:21:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.241+0000 7f9595600640 1 --2- 172.21.15.2:0/4100219362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 0x561cfd2ea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.241+0000 7f9594c00640 1 --2- 172.21.15.2:0/4100219362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2d9090 0x561cfd2e3840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f958409bb10 tx=0x7f9584003aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d03186358604643 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.241+0000 7f9581200640 1 -- 172.21.15.2:0/4100219362 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958409b4e0 con 0x561cfd2d9090 2024-09-10T08:21:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.241+0000 7f9581200640 1 -- 172.21.15.2:0/4100219362 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95840b9070 con 0x561cfd2d9090 2024-09-10T08:21:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.241+0000 7f9581200640 1 -- 172.21.15.2:0/4100219362 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958409bb80 con 0x561cfd2d9090 2024-09-10T08:21:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.242+0000 7f959a464740 1 -- 172.21.15.2:0/4100219362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2d9090 msgr2=0x561cfd2e3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.242+0000 7f959a464740 1 --2- 172.21.15.2:0/4100219362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2d9090 0x561cfd2e3840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f958409bb10 tx=0x7f9584003aa0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.242+0000 7f959a464740 1 -- 172.21.15.2:0/4100219362 shutdown_connections 2024-09-10T08:21:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.242+0000 7f959a464740 1 --2- 172.21.15.2:0/4100219362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 0x561cfd2ea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.242+0000 7f959a464740 1 --2- 172.21.15.2:0/4100219362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d99d0 0x561cfd2eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.242+0000 7f959a464740 1 --2- 172.21.15.2:0/4100219362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2d9090 0x561cfd2e3840 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.242+0000 7f959a464740 1 -- 172.21.15.2:0/4100219362 >> 172.21.15.2:0/4100219362 conn(0x561cfd2d4700 msgr2=0x561cfd2d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.242+0000 7f959a464740 1 -- 172.21.15.2:0/4100219362 shutdown_connections 2024-09-10T08:21:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.242+0000 7f959a464740 1 -- 172.21.15.2:0/4100219362 wait complete. 2024-09-10T08:21:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.242+0000 7f959a464740 1 -- start start 2024-09-10T08:21:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.242+0000 7f959a464740 1 Processor -- start 2024-09-10T08:21:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.243+0000 7f959a464740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2e7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.243+0000 7f959a464740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2d99d0 0x561cfd2e79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.243+0000 7f959a464740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 0x561cfd357a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.243+0000 7f959a464740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f95901ab0b0 con 0x561cfd2dea60 2024-09-10T08:21:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.243+0000 7f959a464740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95901ac520 con 0x561cfd2d9090 2024-09-10T08:21:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.243+0000 7f959a464740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f95901ab780 con 0x561cfd2d99d0 2024-09-10T08:21:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.243+0000 7f959a464740 10 client.0 _ll_register_callbacks cb 0x7f9584097fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.243+0000 7f9595600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 0x561cfd357a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.243+0000 7f9595600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 0x561cfd357a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38058/0 (socket says 172.21.15.2:38058) 2024-09-10T08:21:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.243+0000 7f9595600640 1 -- 172.21.15.2:0/1680760365 learned_addr learned my addr 172.21.15.2:0/1680760365 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.243+0000 7f958fe00640 1 --2- 172.21.15.2:0/1680760365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2d99d0 0x561cfd2e79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.243+0000 7f9594c00640 1 --2- 172.21.15.2:0/1680760365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2e7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.244+0000 7f9595600640 1 -- 172.21.15.2:0/1680760365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2d99d0 msgr2=0x561cfd2e79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.244+0000 7f9595600640 1 --2- 172.21.15.2:0/1680760365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2d99d0 0x561cfd2e79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.244+0000 7f9595600640 1 -- 172.21.15.2:0/1680760365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 msgr2=0x561cfd2e7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.244+0000 7f9595600640 1 --2- 172.21.15.2:0/1680760365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2e7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.244+0000 7f9595600640 1 -- 172.21.15.2:0/1680760365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cfd2eaa40 con 0x561cfd2dea60 2024-09-10T08:21:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.244+0000 7f9595600640 1 --2- 172.21.15.2:0/1680760365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 0x561cfd357a90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9588000f80 tx=0x7f9588005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544006f09e107009 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.244+0000 7f9576000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.244+0000 7f9576000640 1 -- 172.21.15.2:0/1680760365 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958802a060 con 0x561cfd2dea60 2024-09-10T08:21:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.244+0000 7f9576000640 1 -- 172.21.15.2:0/1680760365 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f95880057f0 con 0x561cfd2dea60 2024-09-10T08:21:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.244+0000 7f959a464740 1 -- 172.21.15.2:0/1680760365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95840b9070 con 0x561cfd2dea60 2024-09-10T08:21:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.244+0000 7f9576000640 1 -- 172.21.15.2:0/1680760365 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9588031050 con 0x561cfd2dea60 2024-09-10T08:21:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.244+0000 7f959a464740 10 client.19107 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.244+0000 7f959a464740 1 -- 172.21.15.2:0/1680760365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561cfd359070 con 0x561cfd2dea60 2024-09-10T08:21:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.245+0000 7f9576000640 1 -- 172.21.15.2:0/1680760365 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95840b9070 con 0x561cfd2dea60 2024-09-10T08:21:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.245+0000 7f9576000640 1 -- 172.21.15.2:0/1680760365 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f958800f040 con 0x561cfd2dea60 2024-09-10T08:21:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.245+0000 7f959a464740 10 client.19107 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.245+0000 7f959a464740 10 client.19107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.245+0000 7f959a464740 20 client.19107 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.245+0000 7f959a464740 1 --2- 172.21.15.2:0/1680760365 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561cfd3596d0 0x561cfd35bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.245+0000 7f959a464740 4 client.19107 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:21:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.245+0000 7f959a464740 1 -- 172.21.15.2:0/1680760365 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561cfd245b80 con 0x561cfd3596d0 2024-09-10T08:21:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.245+0000 7f9594c00640 1 --2- 172.21.15.2:0/1680760365 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561cfd3596d0 0x561cfd35bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.246+0000 7f9594c00640 1 --2- 172.21.15.2:0/1680760365 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561cfd3596d0 0x561cfd35bac0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=526195d5bfe32cf6 server_cookie=1f1170e567d0745c in_seq=0 out_seq=0 2024-09-10T08:21:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.246+0000 7f9576000640 10 client.19107 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f9576000640 1 -- 172.21.15.2:0/1680760365 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561cfd245b80 con 0x561cfd3596d0 2024-09-10T08:21:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f9576000640 10 client.19107 handle_command_reply: tid=0 2024-09-10T08:21:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f959a464740 1 client.19107 shutdown 2024-09-10T08:21:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f959a464740 1 -- 172.21.15.2:0/1680760365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 msgr2=0x561cfd357a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f959a464740 1 --2- 172.21.15.2:0/1680760365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 0x561cfd357a90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9588000f80 tx=0x7f9588005270 comp rx=0 tx=0).stop 2024-09-10T08:21:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f959a464740 1 -- 172.21.15.2:0/1680760365 shutdown_connections 2024-09-10T08:21:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f959a464740 1 --2- 172.21.15.2:0/1680760365 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561cfd3596d0 0x561cfd35bac0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f959a464740 1 --2- 172.21.15.2:0/1680760365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd2dea60 0x561cfd357a90 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f959a464740 1 --2- 172.21.15.2:0/1680760365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cfd2d99d0 0x561cfd2e79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f959a464740 1 --2- 172.21.15.2:0/1680760365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cfd2d9090 0x561cfd2e7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f9576000640 0 client.19107 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f959a464740 1 -- 172.21.15.2:0/1680760365 >> 172.21.15.2:0/1680760365 conn(0x561cfd2d4700 msgr2=0x561cfd2d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f959a464740 1 -- 172.21.15.2:0/1680760365 shutdown_connections 2024-09-10T08:21:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f959a464740 1 -- 172.21.15.2:0/1680760365 wait complete. 2024-09-10T08:21:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.259+0000 7f959a464740 20 client.19107 trim_cache size 0 max 16384 2024-09-10T08:21:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:21:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:21:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:21:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:21:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.260+0000 7f9597600640 1 -- 172.21.15.2:0/2208614560 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f95780a2c30 msgr2=0x7f95780a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.260+0000 7f9597600640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f95780a2c30 0x7f95780a50f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f957c000c00 tx=0x7f957c0315b0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.260+0000 7f9597600640 1 -- 172.21.15.2:0/2208614560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9590074bf0 msgr2=0x7f9590199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.260+0000 7f9597600640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9590074bf0 0x7f9590199fb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f958400e9f0 tx=0x7f958400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.260+0000 7f9597600640 1 -- 172.21.15.2:0/2208614560 shutdown_connections 2024-09-10T08:21:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.260+0000 7f9597600640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95901099d0 0x7f95901a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.260+0000 7f9597600640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95900761d0 0x7f959019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.260+0000 7f9597600640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f95780a2c30 0x7f95780a50f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.260+0000 7f9597600640 1 --2- 172.21.15.2:0/2208614560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9590074bf0 0x7f9590199fb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.260+0000 7f9597600640 1 -- 172.21.15.2:0/2208614560 >> 172.21.15.2:0/2208614560 conn(0x7f95900fd2e0 msgr2=0x7f9590105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.260+0000 7f9597600640 1 -- 172.21.15.2:0/2208614560 shutdown_connections 2024-09-10T08:21:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:14.260+0000 7f9597600640 1 -- 172.21.15.2:0/2208614560 wait complete. 2024-09-10T08:21:16.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:15 smithi002 ceph-mon[30563]: pgmap v780: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:21:16.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:15 smithi155 ceph-mon[35405]: pgmap v780: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:21:16.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:15 smithi107 ceph-mon[35048]: pgmap v780: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:21:17.969 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:17 smithi107 ceph-mon[35048]: pgmap v781: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:21:18.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:17 smithi002 ceph-mon[30563]: pgmap v781: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:21:18.089 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T08:21:18.089 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.5 2024-09-10T08:21:18.090 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:21:18.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:17 smithi155 ceph-mon[35405]: pgmap v781: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:21:18.481 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:19.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.027+0000 7f5c22200640 1 -- 172.21.15.2:0/183530634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c1c109120 msgr2=0x7f5c1c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:19.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.027+0000 7f5c22200640 1 --2- 172.21.15.2:0/183530634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c1c109120 0x7f5c1c110ca0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009ca0 tx=0x7f5c1402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:21:19.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.027+0000 7f5c22200640 1 -- 172.21.15.2:0/183530634 shutdown_connections 2024-09-10T08:21:19.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.027+0000 7f5c22200640 1 --2- 172.21.15.2:0/183530634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c1c109120 0x7f5c1c110ca0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:19.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.027+0000 7f5c22200640 1 --2- 172.21.15.2:0/183530634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c1c1049a0 0x7f5c1c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:19.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.027+0000 7f5c22200640 1 --2- 172.21.15.2:0/183530634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c1c103ff0 0x7f5c1c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:19.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.027+0000 7f5c22200640 1 -- 172.21.15.2:0/183530634 >> 172.21.15.2:0/183530634 conn(0x7f5c1c0ffa90 msgr2=0x7f5c1c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:19.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.027+0000 7f5c22200640 1 -- 172.21.15.2:0/183530634 shutdown_connections 2024-09-10T08:21:19.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.027+0000 7f5c22200640 1 -- 172.21.15.2:0/183530634 wait complete. 2024-09-10T08:21:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.028+0000 7f5c22200640 1 Processor -- start 2024-09-10T08:21:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.028+0000 7f5c22200640 1 -- start start 2024-09-10T08:21:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c22200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c1c103ff0 0x7f5c1c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c22200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c1c1049a0 0x7f5c1c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c22200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c1c109120 0x7f5c1c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c22200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c1141a0 con 0x7f5c1c103ff0 2024-09-10T08:21:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c22200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c114020 con 0x7f5c1c1049a0 2024-09-10T08:21:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c22200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c114320 con 0x7f5c1c109120 2024-09-10T08:21:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c1b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c1c103ff0 0x7f5c1c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c1b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c1c103ff0 0x7f5c1c19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46606/0 (socket says 172.21.15.2:46606) 2024-09-10T08:21:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c1b400640 1 -- 172.21.15.2:0/76838739 learned_addr learned my addr 172.21.15.2:0/76838739 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c1aa00640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c1c1049a0 0x7f5c1c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c1be00640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c1c109120 0x7f5c1c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c1b400640 1 -- 172.21.15.2:0/76838739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c1c109120 msgr2=0x7f5c1c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c1b400640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c1c109120 0x7f5c1c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c1b400640 1 -- 172.21.15.2:0/76838739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c1c1049a0 msgr2=0x7f5c1c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c1b400640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c1c1049a0 0x7f5c1c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.029+0000 7f5c1b400640 1 -- 172.21.15.2:0/76838739 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1c19f610 con 0x7f5c1c103ff0 2024-09-10T08:21:19.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.030+0000 7f5c1b400640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c1c103ff0 0x7f5c1c19a660 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00e9f0 tx=0x7f5c0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:19.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.030+0000 7f5c13e00640 1 -- 172.21.15.2:0/76838739 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c00cd80 con 0x7f5c1c103ff0 2024-09-10T08:21:19.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.030+0000 7f5c22200640 1 -- 172.21.15.2:0/76838739 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1c19f900 con 0x7f5c1c103ff0 2024-09-10T08:21:19.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.030+0000 7f5c13e00640 1 -- 172.21.15.2:0/76838739 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c0c005500 con 0x7f5c1c103ff0 2024-09-10T08:21:19.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.030+0000 7f5c13e00640 1 -- 172.21.15.2:0/76838739 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c010690 con 0x7f5c1c103ff0 2024-09-10T08:21:19.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.030+0000 7f5c22200640 1 -- 172.21.15.2:0/76838739 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c1c1a71e0 con 0x7f5c1c103ff0 2024-09-10T08:21:19.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.031+0000 7f5c22200640 1 -- 172.21.15.2:0/76838739 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c1c1043d0 con 0x7f5c1c103ff0 2024-09-10T08:21:19.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.033+0000 7f5c13e00640 1 -- 172.21.15.2:0/76838739 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5c0c0046c0 con 0x7f5c1c103ff0 2024-09-10T08:21:19.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.034+0000 7f5c13e00640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5c000a2c00 0x7f5c000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:19.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.035+0000 7f5c13e00640 1 -- 172.21.15.2:0/76838739 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5c0c0cf8c0 con 0x7f5c1c103ff0 2024-09-10T08:21:19.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.035+0000 7f5c1aa00640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5c000a2c00 0x7f5c000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:19.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.036+0000 7f5c1aa00640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5c000a2c00 0x7f5c000a50c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c10aa90 tx=0x7f5c08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:19.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.039+0000 7f5c13e00640 1 -- 172.21.15.2:0/76838739 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c0c09c3d0 con 0x7f5c1c103ff0 2024-09-10T08:21:19.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.261+0000 7f5c22200640 1 -- 172.21.15.2:0/76838739 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c1c19b9c0 con 0x7f5c1c103ff0 2024-09-10T08:21:19.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.262+0000 7f5c13e00640 1 -- 172.21.15.2:0/76838739 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f5c0c09bcd0 con 0x7f5c1c103ff0 2024-09-10T08:21:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:21:19.263 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:21:19.263 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:21:19.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.266+0000 7f5c22200640 1 -- 172.21.15.2:0/76838739 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5c000a2c00 msgr2=0x7f5c000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:19.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.266+0000 7f5c22200640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5c000a2c00 0x7f5c000a50c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c10aa90 tx=0x7f5c08005f90 comp rx=0 tx=0).stop 2024-09-10T08:21:19.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.266+0000 7f5c22200640 1 -- 172.21.15.2:0/76838739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c1c103ff0 msgr2=0x7f5c1c19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:19.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.266+0000 7f5c22200640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c1c103ff0 0x7f5c1c19a660 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00e9f0 tx=0x7f5c0c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:21:19.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.266+0000 7f5c22200640 1 -- 172.21.15.2:0/76838739 shutdown_connections 2024-09-10T08:21:19.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.266+0000 7f5c22200640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f5c000a2c00 0x7f5c000a50c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:19.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.266+0000 7f5c22200640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c1c109120 0x7f5c1c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:19.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.266+0000 7f5c22200640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c1c1049a0 0x7f5c1c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:19.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.266+0000 7f5c22200640 1 --2- 172.21.15.2:0/76838739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c1c103ff0 0x7f5c1c19a660 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:19.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.266+0000 7f5c22200640 1 -- 172.21.15.2:0/76838739 >> 172.21.15.2:0/76838739 conn(0x7f5c1c0ffa90 msgr2=0x7f5c1c100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:19.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.266+0000 7f5c22200640 1 -- 172.21.15.2:0/76838739 shutdown_connections 2024-09-10T08:21:19.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:19.266+0000 7f5c22200640 1 -- 172.21.15.2:0/76838739 wait complete. 2024-09-10T08:21:19.983 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:19 smithi002 ceph-mon[30563]: pgmap v782: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:21:19.983 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:19 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/76838739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:20.019 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0' 2024-09-10T08:21:20.020 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0 2024-09-10T08:21:20.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:19 smithi155 ceph-mon[35405]: pgmap v782: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:21:20.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:19 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/76838739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:20.168 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:19 smithi107 ceph-mon[35048]: pgmap v782: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:21:20.168 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:19 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/76838739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:20.400 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:20.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.963+0000 7f94fe800640 1 -- 172.21.15.2:0/1259261538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f81049c0 msgr2=0x7f94f8108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:20.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.963+0000 7f94fe800640 1 --2- 172.21.15.2:0/1259261538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f81049c0 0x7f94f8108a10 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009900 tx=0x7f94e002a410 comp rx=0 tx=0).stop 2024-09-10T08:21:20.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.963+0000 7f94fe800640 1 -- 172.21.15.2:0/1259261538 shutdown_connections 2024-09-10T08:21:20.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.963+0000 7f94fe800640 1 --2- 172.21.15.2:0/1259261538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94f8109140 0x7f94f8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.963+0000 7f94fe800640 1 --2- 172.21.15.2:0/1259261538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f81049c0 0x7f94f8108a10 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.963+0000 7f94fe800640 1 --2- 172.21.15.2:0/1259261538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f8104010 0x7f94f81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.963+0000 7f94fe800640 1 -- 172.21.15.2:0/1259261538 >> 172.21.15.2:0/1259261538 conn(0x7f94f80ffa50 msgr2=0x7f94f8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:20.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.963+0000 7f94fe800640 1 -- 172.21.15.2:0/1259261538 shutdown_connections 2024-09-10T08:21:20.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.963+0000 7f94fe800640 1 -- 172.21.15.2:0/1259261538 wait complete. 2024-09-10T08:21:20.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.964+0000 7f94fe800640 1 Processor -- start 2024-09-10T08:21:20.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.964+0000 7f94fe800640 1 -- start start 2024-09-10T08:21:20.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.964+0000 7f94fe800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f8104010 0x7f94f819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.964+0000 7f94fe800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f81049c0 0x7f94f819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.964+0000 7f94fe800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94f8109140 0x7f94f819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.964+0000 7f94fe800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f94f8114190 con 0x7f94f8104010 2024-09-10T08:21:20.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.964+0000 7f94fe800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f94f8114010 con 0x7f94f81049c0 2024-09-10T08:21:20.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.964+0000 7f94fe800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94f8114310 con 0x7f94f8109140 2024-09-10T08:21:20.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.965+0000 7f94f7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f8104010 0x7f94f819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:20.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.965+0000 7f94f7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f8104010 0x7f94f819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46614/0 (socket says 172.21.15.2:46614) 2024-09-10T08:21:20.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.965+0000 7f94f7400640 1 -- 172.21.15.2:0/3419330542 learned_addr learned my addr 172.21.15.2:0/3419330542 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:20.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.965+0000 7f94f7e00640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94f8109140 0x7f94f819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:20.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.965+0000 7f94efe00640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f81049c0 0x7f94f819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:20.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.965+0000 7f94f7400640 1 -- 172.21.15.2:0/3419330542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94f8109140 msgr2=0x7f94f819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:20.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.965+0000 7f94f7400640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94f8109140 0x7f94f819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.965+0000 7f94f7400640 1 -- 172.21.15.2:0/3419330542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f81049c0 msgr2=0x7f94f819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:20.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.965+0000 7f94f7400640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f81049c0 0x7f94f819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.965+0000 7f94f7400640 1 -- 172.21.15.2:0/3419330542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f819f6a0 con 0x7f94f8104010 2024-09-10T08:21:20.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.965+0000 7f94f7400640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f8104010 0x7f94f819a6f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f94e800e9f0 tx=0x7f94e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:20.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.966+0000 7f94f4c00640 1 -- 172.21.15.2:0/3419330542 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e800cd80 con 0x7f94f8104010 2024-09-10T08:21:20.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.966+0000 7f94f4c00640 1 -- 172.21.15.2:0/3419330542 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e8010910 con 0x7f94f8104010 2024-09-10T08:21:20.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.966+0000 7f94f4c00640 1 -- 172.21.15.2:0/3419330542 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e8018990 con 0x7f94f8104010 2024-09-10T08:21:20.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.966+0000 7f94fe800640 1 -- 172.21.15.2:0/3419330542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f819f990 con 0x7f94f8104010 2024-09-10T08:21:20.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.966+0000 7f94fe800640 1 -- 172.21.15.2:0/3419330542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f81a7270 con 0x7f94f8104010 2024-09-10T08:21:20.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.970+0000 7f95012b3740 1 Processor -- start 2024-09-10T08:21:20.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.970+0000 7f95012b3740 1 -- start start 2024-09-10T08:21:20.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.970+0000 7f94f4c00640 1 -- 172.21.15.2:0/3419330542 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f94e8010ac0 con 0x7f94f8104010 2024-09-10T08:21:20.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.971+0000 7f95012b3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45dfef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.971+0000 7f95012b3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfefcd0 0x55c45dffa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.971+0000 7f95012b3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dffa590 0x55c45dffc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.971+0000 7f95012b3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c45df55530 con 0x55c45dfef3b0 2024-09-10T08:21:20.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.971+0000 7f95012b3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c45defc860 con 0x55c45dffa590 2024-09-10T08:21:20.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.971+0000 7f95012b3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c45df514f0 con 0x55c45dfefcd0 2024-09-10T08:21:20.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.971+0000 7f94f7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfefcd0 0x55c45dffa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:20.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.971+0000 7f94f7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45dfef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:20.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.971+0000 7f94f7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45dfef790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46626/0 (socket says 172.21.15.2:46626) 2024-09-10T08:21:20.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.971+0000 7f94f7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfefcd0 0x55c45dffa050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:56886/0 (socket says 172.21.15.2:56886) 2024-09-10T08:21:20.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.971+0000 7f94f7e00640 1 -- 172.21.15.2:0/2187449509 learned_addr learned my addr 172.21.15.2:0/2187449509 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:20.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.971+0000 7f94efe00640 1 --2- 172.21.15.2:0/2187449509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dffa590 0x55c45dffc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:20.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.972+0000 7f94f7e00640 1 -- 172.21.15.2:0/2187449509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfefcd0 msgr2=0x55c45dffa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:20.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.972+0000 7f94f7e00640 1 --2- 172.21.15.2:0/2187449509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfefcd0 0x55c45dffa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.972+0000 7f94f7e00640 1 -- 172.21.15.2:0/2187449509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dffa590 msgr2=0x55c45dffc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:20.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.972+0000 7f94f7e00640 1 --2- 172.21.15.2:0/2187449509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dffa590 0x55c45dffc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.972+0000 7f94f7e00640 1 -- 172.21.15.2:0/2187449509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c45dcc6b90 con 0x55c45dfef3b0 2024-09-10T08:21:20.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.972+0000 7f94efe00640 1 --2- 172.21.15.2:0/2187449509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dffa590 0x55c45dffc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:21:20.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.972+0000 7f94f7e00640 1 --2- 172.21.15.2:0/2187449509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45dfef790 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009f50 tx=0x7f94f00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24cef6bc8c4f1d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:20.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.972+0000 7f94f4c00640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94dc0a2c00 0x7f94dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.972+0000 7f94ecc00640 1 -- 172.21.15.2:0/2187449509 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0004d20 con 0x55c45dfef3b0 2024-09-10T08:21:20.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.972+0000 7f94ecc00640 1 -- 172.21.15.2:0/2187449509 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94f0004050 con 0x55c45dfef3b0 2024-09-10T08:21:20.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.972+0000 7f94ecc00640 1 -- 172.21.15.2:0/2187449509 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0005260 con 0x55c45dfef3b0 2024-09-10T08:21:20.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.972+0000 7f94f4c00640 1 -- 172.21.15.2:0/3419330542 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f94e80cf700 con 0x7f94f8104010 2024-09-10T08:21:20.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.973+0000 7f94f7400640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94dc0a2c00 0x7f94dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:20.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.973+0000 7f95012b3740 1 -- 172.21.15.2:0/2187449509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 msgr2=0x55c45dfef790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:20.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.973+0000 7f95012b3740 1 --2- 172.21.15.2:0/2187449509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45dfef790 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009f50 tx=0x7f94f00118d0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.973+0000 7f95012b3740 1 -- 172.21.15.2:0/2187449509 shutdown_connections 2024-09-10T08:21:20.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.973+0000 7f95012b3740 1 --2- 172.21.15.2:0/2187449509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dffa590 0x55c45dffc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.973+0000 7f95012b3740 1 --2- 172.21.15.2:0/2187449509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfefcd0 0x55c45dffa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.973+0000 7f95012b3740 1 --2- 172.21.15.2:0/2187449509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45dfef790 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.973+0000 7f95012b3740 1 -- 172.21.15.2:0/2187449509 >> 172.21.15.2:0/2187449509 conn(0x55c45dfeb940 msgr2=0x55c45dfebd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:20.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.973+0000 7f94f7400640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94dc0a2c00 0x7f94dc0a50c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f94e8098020 tx=0x7f94e80d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:20.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.973+0000 7f95012b3740 1 -- 172.21.15.2:0/2187449509 shutdown_connections 2024-09-10T08:21:20.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.973+0000 7f95012b3740 1 -- 172.21.15.2:0/2187449509 wait complete. 2024-09-10T08:21:20.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.974+0000 7f95012b3740 1 -- start start 2024-09-10T08:21:20.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.974+0000 7f95012b3740 1 Processor -- start 2024-09-10T08:21:20.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.974+0000 7f95012b3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45dff74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.974+0000 7f95012b3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfefcd0 0x55c45dff7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f95012b3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dffa590 0x55c45dff7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f95012b3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c45dffd010 con 0x55c45dfef3b0 2024-09-10T08:21:20.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f95012b3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c45df0fe60 con 0x55c45dffa590 2024-09-10T08:21:20.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f95012b3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c45dffd190 con 0x55c45dfefcd0 2024-09-10T08:21:20.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f95012b3740 10 client.0 _ll_register_callbacks cb 0x55c45df14cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:20.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f94f7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45dff74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:20.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f94efe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfefcd0 0x55c45dff7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:20.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f94f7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45dff74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46634/0 (socket says 172.21.15.2:46634) 2024-09-10T08:21:20.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f94f7e00640 1 -- 172.21.15.2:0/3745372501 learned_addr learned my addr 172.21.15.2:0/3745372501 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:20.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f94f7e00640 1 --2- 172.21.15.2:0/3745372501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dffa590 0x55c45dff7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:20.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f94efe00640 1 -- 172.21.15.2:0/3745372501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dffa590 msgr2=0x55c45dff7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:20.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f94efe00640 1 --2- 172.21.15.2:0/3745372501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dffa590 0x55c45dff7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f94efe00640 1 -- 172.21.15.2:0/3745372501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 msgr2=0x55c45dff74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:20.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f94efe00640 1 --2- 172.21.15.2:0/3745372501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45dff74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f94efe00640 1 -- 172.21.15.2:0/3745372501 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c45df1b990 con 0x55c45dfefcd0 2024-09-10T08:21:20.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.975+0000 7f94f7e00640 1 --2- 172.21.15.2:0/3745372501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dffa590 0x55c45dff7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:21:20.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.976+0000 7f94efe00640 1 --2- 172.21.15.2:0/3745372501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfefcd0 0x55c45dff7a00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f94e002aa00 tx=0x7f94e0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb5ca9ea13beeaaa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:20.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.976+0000 7f94e5600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:21:20.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.976+0000 7f94e5600640 1 -- 172.21.15.2:0/3745372501 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0035040 con 0x55c45dfefcd0 2024-09-10T08:21:20.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.976+0000 7f95012b3740 1 -- 172.21.15.2:0/3745372501 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94f0004050 con 0x55c45dfefcd0 2024-09-10T08:21:20.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.976+0000 7f94e5600640 1 -- 172.21.15.2:0/3745372501 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f94e0005230 con 0x55c45dfefcd0 2024-09-10T08:21:20.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.976+0000 7f94e5600640 1 -- 172.21.15.2:0/3745372501 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f94e0005430 con 0x55c45dfefcd0 2024-09-10T08:21:20.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.976+0000 7f95012b3740 10 client.29083 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:20.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.976+0000 7f95012b3740 1 -- 172.21.15.2:0/3745372501 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c45e008d80 con 0x55c45dfefcd0 2024-09-10T08:21:20.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.977+0000 7f94e5600640 1 -- 172.21.15.2:0/3745372501 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94f0004050 con 0x55c45dfefcd0 2024-09-10T08:21:20.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.977+0000 7f94e5600640 1 -- 172.21.15.2:0/3745372501 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f94e0003b60 con 0x55c45dfefcd0 2024-09-10T08:21:20.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.977+0000 7f95012b3740 10 client.29083 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:20.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.977+0000 7f95012b3740 10 client.29083 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:21:20.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.977+0000 7f95012b3740 20 client.29083 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:20.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.977+0000 7f95012b3740 1 --2- 172.21.15.2:0/3745372501 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c45e00a190 0x55c45e00c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.977+0000 7f95012b3740 4 client.29083 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:20.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.977+0000 7f95012b3740 1 -- 172.21.15.2:0/3745372501 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c45def8310 con 0x55c45e00a190 2024-09-10T08:21:20.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.977+0000 7f94f7e00640 1 --2- 172.21.15.2:0/3745372501 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c45e00a190 0x55c45e00c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.978+0000 7f94f7e00640 1 --2- 172.21.15.2:0/3745372501 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c45e00a190 0x55c45e00c580 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3cb49c09f842c78 server_cookie=c32313f7db5ec4b1 in_seq=0 out_seq=0 2024-09-10T08:21:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.978+0000 7f94e5600640 10 client.29083 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f94e5600640 1 -- 172.21.15.2:0/3745372501 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c45def8310 con 0x55c45e00a190 2024-09-10T08:21:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f94e5600640 10 client.29083 handle_command_reply: tid=0 2024-09-10T08:21:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f95012b3740 1 client.29083 shutdown 2024-09-10T08:21:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f95012b3740 1 -- 172.21.15.2:0/3745372501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfefcd0 msgr2=0x55c45dff7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f95012b3740 1 --2- 172.21.15.2:0/3745372501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfefcd0 0x55c45dff7a00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f94e002aa00 tx=0x7f94e0004270 comp rx=0 tx=0).stop 2024-09-10T08:21:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f95012b3740 1 -- 172.21.15.2:0/3745372501 shutdown_connections 2024-09-10T08:21:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f95012b3740 1 --2- 172.21.15.2:0/3745372501 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c45e00a190 0x55c45e00c580 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f95012b3740 1 --2- 172.21.15.2:0/3745372501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dffa590 0x55c45dff7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f95012b3740 1 --2- 172.21.15.2:0/3745372501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfefcd0 0x55c45dff7a00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f95012b3740 1 --2- 172.21.15.2:0/3745372501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45dff74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:20.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f94e5600640 0 client.29083 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:20.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f95012b3740 1 -- 172.21.15.2:0/3745372501 >> 172.21.15.2:0/3745372501 conn(0x55c45dfeb940 msgr2=0x55c45dfeac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:20.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f95012b3740 1 -- 172.21.15.2:0/3745372501 shutdown_connections 2024-09-10T08:21:20.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f95012b3740 1 -- 172.21.15.2:0/3745372501 wait complete. 2024-09-10T08:21:20.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.979+0000 7f95012b3740 20 client.29083 trim_cache size 0 max 16384 2024-09-10T08:21:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.997+0000 7f95012b3740 1 Processor -- start 2024-09-10T08:21:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.997+0000 7f95012b3740 1 -- start start 2024-09-10T08:21:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.997+0000 7f95012b3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45e011e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.997+0000 7f95012b3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dfefcd0 0x55c45e012350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.997+0000 7f95012b3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dffa590 0x55c45dff4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.997+0000 7f95012b3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f94f81b39e0 con 0x55c45dfef3b0 2024-09-10T08:21:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.997+0000 7f95012b3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f94f81b12b0 con 0x55c45dfefcd0 2024-09-10T08:21:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.997+0000 7f95012b3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94f81af100 con 0x55c45dffa590 2024-09-10T08:21:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.998+0000 7f94f7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45e011e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.998+0000 7f94f7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45e011e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46638/0 (socket says 172.21.15.2:46638) 2024-09-10T08:21:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.998+0000 7f94f7e00640 1 -- 172.21.15.2:0/196938085 learned_addr learned my addr 172.21.15.2:0/196938085 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.998+0000 7f94efe00640 1 --2- 172.21.15.2:0/196938085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dfefcd0 0x55c45e012350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.998+0000 7f94f7e00640 1 --2- 172.21.15.2:0/196938085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dffa590 0x55c45dff4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.998+0000 7f94efe00640 1 -- 172.21.15.2:0/196938085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dffa590 msgr2=0x55c45dff4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.998+0000 7f94efe00640 1 --2- 172.21.15.2:0/196938085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dffa590 0x55c45dff4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.998+0000 7f94efe00640 1 -- 172.21.15.2:0/196938085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 msgr2=0x55c45e011e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.998+0000 7f94efe00640 1 --2- 172.21.15.2:0/196938085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45e011e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.998+0000 7f94efe00640 1 -- 172.21.15.2:0/196938085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c45e00cbe0 con 0x55c45dfefcd0 2024-09-10T08:21:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.998+0000 7f94f7e00640 1 --2- 172.21.15.2:0/196938085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45e011e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.998+0000 7f94efe00640 1 --2- 172.21.15.2:0/196938085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dfefcd0 0x55c45e012350 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f94e0005c40 tx=0x7f94e0005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3b72910ccf8a983 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.999+0000 7f94e7e00640 1 -- 172.21.15.2:0/196938085 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0035030 con 0x55c45dfefcd0 2024-09-10T08:21:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.999+0000 7f94e7e00640 1 -- 172.21.15.2:0/196938085 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e0040070 con 0x55c45dfefcd0 2024-09-10T08:21:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.999+0000 7f94e7e00640 1 -- 172.21.15.2:0/196938085 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0003c20 con 0x55c45dfefcd0 2024-09-10T08:21:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.999+0000 7f95012b3740 1 -- 172.21.15.2:0/196938085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dfefcd0 msgr2=0x55c45e012350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.999+0000 7f95012b3740 1 --2- 172.21.15.2:0/196938085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dfefcd0 0x55c45e012350 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f94e0005c40 tx=0x7f94e0005e80 comp rx=0 tx=0).stop 2024-09-10T08:21:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.999+0000 7f95012b3740 1 -- 172.21.15.2:0/196938085 shutdown_connections 2024-09-10T08:21:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.999+0000 7f95012b3740 1 --2- 172.21.15.2:0/196938085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dffa590 0x55c45dff4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.999+0000 7f95012b3740 1 --2- 172.21.15.2:0/196938085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dfefcd0 0x55c45e012350 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.999+0000 7f95012b3740 1 --2- 172.21.15.2:0/196938085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dfef3b0 0x55c45e011e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:20.999+0000 7f95012b3740 1 -- 172.21.15.2:0/196938085 >> 172.21.15.2:0/196938085 conn(0x55c45dfeb940 msgr2=0x55c45dfeac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:21.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.000+0000 7f95012b3740 1 -- 172.21.15.2:0/196938085 shutdown_connections 2024-09-10T08:21:21.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.000+0000 7f95012b3740 1 -- 172.21.15.2:0/196938085 wait complete. 2024-09-10T08:21:21.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.000+0000 7f95012b3740 1 -- start start 2024-09-10T08:21:21.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.000+0000 7f95012b3740 1 Processor -- start 2024-09-10T08:21:21.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.000+0000 7f95012b3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfef3b0 0x55c45e0027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:21.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f95012b3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dfefcd0 0x55c45e002d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:21.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f95012b3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dffa590 0x55c45dfff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:21.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f95012b3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f94f81af770 con 0x55c45dffa590 2024-09-10T08:21:21.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f95012b3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f94f81b0be0 con 0x55c45dfefcd0 2024-09-10T08:21:21.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f95012b3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94f81afe40 con 0x55c45dfef3b0 2024-09-10T08:21:21.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f95012b3740 10 client.0 _ll_register_callbacks cb 0x7f94f00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:21.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f94efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dfefcd0 0x55c45e002d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:21.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f94f7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dffa590 0x55c45dfff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:21.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f94efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dfefcd0 0x55c45e002d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:56326/0 (socket says 172.21.15.2:56326) 2024-09-10T08:21:21.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f94efe00640 1 -- 172.21.15.2:0/1412372512 learned_addr learned my addr 172.21.15.2:0/1412372512 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:21.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f94f7e00640 1 --2- 172.21.15.2:0/1412372512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfef3b0 0x55c45e0027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:21.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f94efe00640 1 -- 172.21.15.2:0/1412372512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfef3b0 msgr2=0x55c45e0027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:21.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f94efe00640 1 --2- 172.21.15.2:0/1412372512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfef3b0 0x55c45e0027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f94efe00640 1 -- 172.21.15.2:0/1412372512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dffa590 msgr2=0x55c45dfff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:21.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f94efe00640 1 --2- 172.21.15.2:0/1412372512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dffa590 0x55c45dfff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.001+0000 7f94efe00640 1 -- 172.21.15.2:0/1412372512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c45dfeb770 con 0x55c45dfefcd0 2024-09-10T08:21:21.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.002+0000 7f94efe00640 1 --2- 172.21.15.2:0/1412372512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dfefcd0 0x55c45e002d00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f94e0031550 tx=0x7f94e0003fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b6ff3e3b82a89b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:21.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.002+0000 7f94e5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:21:21.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.002+0000 7f94e5600640 1 -- 172.21.15.2:0/1412372512 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e005e060 con 0x55c45dfefcd0 2024-09-10T08:21:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.002+0000 7f95012b3740 1 -- 172.21.15.2:0/1412372512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94e0040070 con 0x55c45dfefcd0 2024-09-10T08:21:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.002+0000 7f94e5600640 1 -- 172.21.15.2:0/1412372512 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f94e0003a00 con 0x55c45dfefcd0 2024-09-10T08:21:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.002+0000 7f94e5600640 1 -- 172.21.15.2:0/1412372512 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f94e0065050 con 0x55c45dfefcd0 2024-09-10T08:21:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.002+0000 7f95012b3740 10 client.29156 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.002+0000 7f95012b3740 1 -- 172.21.15.2:0/1412372512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c45e075810 con 0x55c45dfefcd0 2024-09-10T08:21:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.003+0000 7f94e5600640 1 -- 172.21.15.2:0/1412372512 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e0040070 con 0x55c45dfefcd0 2024-09-10T08:21:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.003+0000 7f94e5600640 1 -- 172.21.15.2:0/1412372512 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f94e00050e0 con 0x55c45dfefcd0 2024-09-10T08:21:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.003+0000 7f95012b3740 10 client.29156 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.003+0000 7f95012b3740 10 client.29156 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:21:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.003+0000 7f95012b3740 20 client.29156 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.003+0000 7f95012b3740 1 --2- 172.21.15.2:0/1412372512 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c45e075fa0 0x55c45e0783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.003+0000 7f95012b3740 4 client.29156 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}] 2024-09-10T08:21:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.003+0000 7f95012b3740 1 -- 172.21.15.2:0/1412372512 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}) -- 0x55c45df62c60 con 0x55c45e075fa0 2024-09-10T08:21:21.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.003+0000 7f94f7e00640 1 --2- 172.21.15.2:0/1412372512 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c45e075fa0 0x55c45e0783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:21.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.004+0000 7f94f7e00640 1 --2- 172.21.15.2:0/1412372512 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c45e075fa0 0x55c45e0783c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641073d9d80b7841 server_cookie=786493844491358a in_seq=0 out_seq=0 2024-09-10T08:21:21.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.004+0000 7f94e5600640 10 client.29156 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:21.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f94e5600640 1 -- 172.21.15.2:0/1412372512 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c45df62c60 con 0x55c45e075fa0 2024-09-10T08:21:21.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f94e5600640 10 client.29156 handle_command_reply: tid=0 2024-09-10T08:21:21.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f95012b3740 1 client.29156 shutdown 2024-09-10T08:21:21.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f95012b3740 1 -- 172.21.15.2:0/1412372512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dfefcd0 msgr2=0x55c45e002d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:21.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f95012b3740 1 --2- 172.21.15.2:0/1412372512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dfefcd0 0x55c45e002d00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f94e0031550 tx=0x7f94e0003fd0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f95012b3740 1 -- 172.21.15.2:0/1412372512 shutdown_connections 2024-09-10T08:21:21.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f95012b3740 1 --2- 172.21.15.2:0/1412372512 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c45e075fa0 0x55c45e0783c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f95012b3740 1 --2- 172.21.15.2:0/1412372512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c45dffa590 0x55c45dfff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f95012b3740 1 --2- 172.21.15.2:0/1412372512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c45dfefcd0 0x55c45e002d00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f95012b3740 1 --2- 172.21.15.2:0/1412372512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c45dfef3b0 0x55c45e0027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f94e5600640 0 client.29156 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:21.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f95012b3740 1 -- 172.21.15.2:0/1412372512 >> 172.21.15.2:0/1412372512 conn(0x55c45dfeb940 msgr2=0x55c45dff0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:21.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f95012b3740 1 -- 172.21.15.2:0/1412372512 shutdown_connections 2024-09-10T08:21:21.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f95012b3740 1 -- 172.21.15.2:0/1412372512 wait complete. 2024-09-10T08:21:21.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.015+0000 7f95012b3740 20 client.29156 trim_cache size 0 max 16384 2024-09-10T08:21:21.017 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:21:21.017 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:21:21.017 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:21:21.017 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 46, 2024-09-10T08:21:21.017 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:21:21.017 INFO:teuthology.orchestra.run.smithi002.stdout: "4958be9": { 2024-09-10T08:21:21.017 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 46, 2024-09-10T08:21:21.017 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:21:21.017 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:21:21.017 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:21:21.017 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:21:21.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:21:21.018 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:21:21.018 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 141.5, 2024-09-10T08:21:21.018 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:21:21.018 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:21:21.018 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:21:21.018 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:21:21.018 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:21:21.018 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:21:21.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:21.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:21.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:21.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:21.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:21.019 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:21:21.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.016+0000 7f94fe800640 1 -- 172.21.15.2:0/3419330542 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94dc0a2c00 msgr2=0x7f94dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:21.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.016+0000 7f94fe800640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94dc0a2c00 0x7f94dc0a50c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f94e8098020 tx=0x7f94e80d5530 comp rx=0 tx=0).stop 2024-09-10T08:21:21.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.016+0000 7f94fe800640 1 -- 172.21.15.2:0/3419330542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f8104010 msgr2=0x7f94f819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:21.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.016+0000 7f94fe800640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f8104010 0x7f94f819a6f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f94e800e9f0 tx=0x7f94e800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.016+0000 7f94fe800640 1 -- 172.21.15.2:0/3419330542 shutdown_connections 2024-09-10T08:21:21.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.016+0000 7f94fe800640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f94dc0a2c00 0x7f94dc0a50c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.016+0000 7f94fe800640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94f8109140 0x7f94f819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.016+0000 7f94fe800640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94f81049c0 0x7f94f819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.016+0000 7f94fe800640 1 --2- 172.21.15.2:0/3419330542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94f8104010 0x7f94f819a6f0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:21.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.016+0000 7f94fe800640 1 -- 172.21.15.2:0/3419330542 >> 172.21.15.2:0/3419330542 conn(0x7f94f80ffa50 msgr2=0x7f94f8100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:21.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.016+0000 7f94fe800640 1 -- 172.21.15.2:0/3419330542 shutdown_connections 2024-09-10T08:21:21.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:21.016+0000 7f94fe800640 1 -- 172.21.15.2:0/3419330542 wait complete. 2024-09-10T08:21:21.850 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4958be9', quiesce took 3.8 seconds. Will release after: 17.7 2024-09-10T08:21:22.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:21 smithi002 ceph-mon[30563]: pgmap v783: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:21:22.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:21 smithi155 ceph-mon[35405]: pgmap v783: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:21:22.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:21 smithi107 ceph-mon[35048]: pgmap v783: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:21:24.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:23 smithi002 ceph-mon[30563]: pgmap v784: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:21:24.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:23 smithi155 ceph-mon[35405]: pgmap v784: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:21:24.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:23 smithi107 ceph-mon[35048]: pgmap v784: 129 pgs: 129 active+clean; 2.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:21:26.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:25 smithi002 ceph-mon[30563]: pgmap v785: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:21:26.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:25 smithi155 ceph-mon[35405]: pgmap v785: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:21:26.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:25 smithi107 ceph-mon[35048]: pgmap v785: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:21:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.389+0000 7f84c7200640 1 -- 172.21.15.2:0/2296692010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84c0078790 msgr2=0x7f84c0079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.389+0000 7f84c7200640 1 --2- 172.21.15.2:0/2296692010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84c0078790 0x7f84c0079030 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009ca0 tx=0x7f84b802a8b0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.389+0000 7f84c7200640 1 -- 172.21.15.2:0/2296692010 shutdown_connections 2024-09-10T08:21:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.389+0000 7f84c7200640 1 --2- 172.21.15.2:0/2296692010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84c0078790 0x7f84c0079030 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.389+0000 7f84c7200640 1 --2- 172.21.15.2:0/2296692010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84c0077df0 0x7f84c0078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.389+0000 7f84c7200640 1 --2- 172.21.15.2:0/2296692010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c0076bb0 0x7f84c0076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.389+0000 7f84c7200640 1 -- 172.21.15.2:0/2296692010 >> 172.21.15.2:0/2296692010 conn(0x7f84c00ff4e0 msgr2=0x7f84c0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.389+0000 7f84c7200640 1 -- 172.21.15.2:0/2296692010 shutdown_connections 2024-09-10T08:21:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.389+0000 7f84c7200640 1 -- 172.21.15.2:0/2296692010 wait complete. 2024-09-10T08:21:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.390+0000 7f84c7200640 1 Processor -- start 2024-09-10T08:21:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.390+0000 7f84c7200640 1 -- start start 2024-09-10T08:21:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84c7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c0076bb0 0x7f84c019e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84c7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84c0077df0 0x7f84c019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84c7200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84c0078790 0x7f84c01a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84c7200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0113e30 con 0x7f84c0078790 2024-09-10T08:21:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84c7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0113cb0 con 0x7f84c0076bb0 2024-09-10T08:21:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84c7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0113fb0 con 0x7f84c0077df0 2024-09-10T08:21:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84c5200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84c0078790 0x7f84c01a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84bfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c0076bb0 0x7f84c019e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84c5200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84c0078790 0x7f84c01a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49754/0 (socket says 172.21.15.2:49754) 2024-09-10T08:21:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84c5200640 1 -- 172.21.15.2:0/4052623619 learned_addr learned my addr 172.21.15.2:0/4052623619 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84b7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84c0077df0 0x7f84c019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84bfe00640 1 -- 172.21.15.2:0/4052623619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84c0077df0 msgr2=0x7f84c019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84bfe00640 1 --2- 172.21.15.2:0/4052623619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84c0077df0 0x7f84c019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84bfe00640 1 -- 172.21.15.2:0/4052623619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84c0078790 msgr2=0x7f84c01a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84bfe00640 1 --2- 172.21.15.2:0/4052623619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84c0078790 0x7f84c01a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.391+0000 7f84bfe00640 1 -- 172.21.15.2:0/4052623619 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c01a5f80 con 0x7f84c0076bb0 2024-09-10T08:21:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.392+0000 7f84bfe00640 1 --2- 172.21.15.2:0/4052623619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c0076bb0 0x7f84c019e360 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f84b000e9f0 tx=0x7f84b000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.392+0000 7f84bd600640 1 -- 172.21.15.2:0/4052623619 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b000ccc0 con 0x7f84c0076bb0 2024-09-10T08:21:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.392+0000 7f84c7200640 1 -- 172.21.15.2:0/4052623619 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c01a6270 con 0x7f84c0076bb0 2024-09-10T08:21:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.392+0000 7f84bd600640 1 -- 172.21.15.2:0/4052623619 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b0010910 con 0x7f84c0076bb0 2024-09-10T08:21:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.392+0000 7f84bd600640 1 -- 172.21.15.2:0/4052623619 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0018960 con 0x7f84c0076bb0 2024-09-10T08:21:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.392+0000 7f84c7200640 1 -- 172.21.15.2:0/4052623619 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c01a6810 con 0x7f84c0076bb0 2024-09-10T08:21:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.395+0000 7f84ca077740 1 Processor -- start 2024-09-10T08:21:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.395+0000 7f84ca077740 1 -- start start 2024-09-10T08:21:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84ca077740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff34f6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84ca077740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34f69d0 0x561ff34fb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84ca077740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34fba60 0x561ff34fdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84ca077740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561ff345b100 con 0x561ff34f6090 2024-09-10T08:21:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84ca077740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561ff34031b0 con 0x561ff34f69d0 2024-09-10T08:21:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84ca077740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561ff34570c0 con 0x561ff34fba60 2024-09-10T08:21:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84bd600640 1 -- 172.21.15.2:0/4052623619 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f84b0010ac0 con 0x7f84c0076bb0 2024-09-10T08:21:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84c5200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff34f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84c5200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff34f6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49760/0 (socket says 172.21.15.2:49760) 2024-09-10T08:21:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84c5200640 1 -- 172.21.15.2:0/1701451929 learned_addr learned my addr 172.21.15.2:0/1701451929 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84b7e00640 1 --2- 172.21.15.2:0/1701451929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34fba60 0x561ff34fdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84bfe00640 1 --2- 172.21.15.2:0/1701451929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34f69d0 0x561ff34fb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84c5200640 1 -- 172.21.15.2:0/1701451929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34fba60 msgr2=0x561ff34fdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84c5200640 1 --2- 172.21.15.2:0/1701451929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34fba60 0x561ff34fdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84c5200640 1 -- 172.21.15.2:0/1701451929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34f69d0 msgr2=0x561ff34fb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84c5200640 1 --2- 172.21.15.2:0/1701451929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34f69d0 0x561ff34fb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84c5200640 1 -- 172.21.15.2:0/1701451929 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff316e210 con 0x561ff34f6090 2024-09-10T08:21:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.396+0000 7f84b7e00640 1 --2- 172.21.15.2:0/1701451929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34fba60 0x561ff34fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:21:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.397+0000 7f84c5200640 1 --2- 172.21.15.2:0/1701451929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff34f6490 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009410 tx=0x7f84b8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03dd4da55cebcc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.397+0000 7f84b5600640 1 -- 172.21.15.2:0/1701451929 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8030a30 con 0x561ff34f6090 2024-09-10T08:21:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.397+0000 7f84b5600640 1 -- 172.21.15.2:0/1701451929 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b8036030 con 0x561ff34f6090 2024-09-10T08:21:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.397+0000 7f84b5600640 1 -- 172.21.15.2:0/1701451929 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8004560 con 0x561ff34f6090 2024-09-10T08:21:27.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.397+0000 7f84bd600640 1 --2- 172.21.15.2:0/4052623619 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f84a40a2cb0 0x7f84a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.398+0000 7f84ca077740 1 -- 172.21.15.2:0/1701451929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 msgr2=0x561ff34f6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:27.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.398+0000 7f84ca077740 1 --2- 172.21.15.2:0/1701451929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff34f6490 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009410 tx=0x7f84b8009a20 comp rx=0 tx=0).stop 2024-09-10T08:21:27.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.398+0000 7f84bd600640 1 -- 172.21.15.2:0/4052623619 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f84b00cf720 con 0x7f84c0076bb0 2024-09-10T08:21:27.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.398+0000 7f84bfe00640 1 --2- 172.21.15.2:0/4052623619 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f84a40a2cb0 0x7f84a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.398+0000 7f84ca077740 1 -- 172.21.15.2:0/1701451929 shutdown_connections 2024-09-10T08:21:27.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.398+0000 7f84ca077740 1 --2- 172.21.15.2:0/1701451929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34fba60 0x561ff34fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.398+0000 7f84ca077740 1 --2- 172.21.15.2:0/1701451929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34f69d0 0x561ff34fb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.398+0000 7f84ca077740 1 --2- 172.21.15.2:0/1701451929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff34f6490 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.398+0000 7f84ca077740 1 -- 172.21.15.2:0/1701451929 >> 172.21.15.2:0/1701451929 conn(0x561ff34f1700 msgr2=0x561ff34f1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.398+0000 7f84ca077740 1 -- 172.21.15.2:0/1701451929 shutdown_connections 2024-09-10T08:21:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.398+0000 7f84ca077740 1 -- 172.21.15.2:0/1701451929 wait complete. 2024-09-10T08:21:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.399+0000 7f84bfe00640 1 --2- 172.21.15.2:0/4052623619 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f84a40a2cb0 0x7f84a40a5170 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f84b00cd1a0 tx=0x7f84b0009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.399+0000 7f84ca077740 1 -- start start 2024-09-10T08:21:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.399+0000 7f84ca077740 1 Processor -- start 2024-09-10T08:21:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.399+0000 7f84ca077740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34f6090 0x561ff35001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.399+0000 7f84ca077740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f69d0 0x561ff3500700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.399+0000 7f84ca077740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34fba60 0x561ff3507740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.399+0000 7f84ca077740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561ff34fe9d0 con 0x561ff34f69d0 2024-09-10T08:21:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.399+0000 7f84ca077740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561ff33e4090 con 0x561ff34fba60 2024-09-10T08:21:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.399+0000 7f84ca077740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561ff34feb50 con 0x561ff34f6090 2024-09-10T08:21:27.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.399+0000 7f84ca077740 10 client.0 _ll_register_callbacks cb 0x561ff3438f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.400+0000 7f84b7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f69d0 0x561ff3500700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.400+0000 7f84b7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f69d0 0x561ff3500700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49764/0 (socket says 172.21.15.2:49764) 2024-09-10T08:21:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.400+0000 7f84b7e00640 1 -- 172.21.15.2:0/501177991 learned_addr learned my addr 172.21.15.2:0/501177991 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.400+0000 7f84c5200640 1 --2- 172.21.15.2:0/501177991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34f6090 0x561ff35001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.400+0000 7f84c5200640 1 --2- 172.21.15.2:0/501177991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34fba60 0x561ff3507740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.400+0000 7f84b7e00640 1 -- 172.21.15.2:0/501177991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34f6090 msgr2=0x561ff35001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.400+0000 7f84b7e00640 1 --2- 172.21.15.2:0/501177991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34f6090 0x561ff35001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.400+0000 7f84b7e00640 1 -- 172.21.15.2:0/501177991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34fba60 msgr2=0x561ff3507740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.400+0000 7f84b7e00640 1 --2- 172.21.15.2:0/501177991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34fba60 0x561ff3507740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.400+0000 7f84b7e00640 1 -- 172.21.15.2:0/501177991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ff323a620 con 0x561ff34f69d0 2024-09-10T08:21:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.400+0000 7f84b7e00640 1 --2- 172.21.15.2:0/501177991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f69d0 0x561ff3500700 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f84a8012700 tx=0x7f84a8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2327d9438ddbc58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.400+0000 7f84ae000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.401+0000 7f84ae000640 1 -- 172.21.15.2:0/501177991 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8004050 con 0x561ff34f69d0 2024-09-10T08:21:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.401+0000 7f84ae000640 1 -- 172.21.15.2:0/501177991 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f84a80041f0 con 0x561ff34f69d0 2024-09-10T08:21:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.401+0000 7f84ca077740 1 -- 172.21.15.2:0/501177991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b8036030 con 0x561ff34f69d0 2024-09-10T08:21:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.401+0000 7f84ae000640 1 -- 172.21.15.2:0/501177991 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84a8025050 con 0x561ff34f69d0 2024-09-10T08:21:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.401+0000 7f84ca077740 10 client.19143 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.401+0000 7f84ca077740 1 -- 172.21.15.2:0/501177991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561ff350a600 con 0x561ff34f69d0 2024-09-10T08:21:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.401+0000 7f84ae000640 1 -- 172.21.15.2:0/501177991 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b8036030 con 0x561ff34f69d0 2024-09-10T08:21:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.401+0000 7f84ae000640 1 -- 172.21.15.2:0/501177991 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f84a8003b00 con 0x561ff34f69d0 2024-09-10T08:21:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.401+0000 7f84ca077740 10 client.19143 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.401+0000 7f84ca077740 10 client.19143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.402+0000 7f84ca077740 20 client.19143 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.402+0000 7f84ca077740 1 --2- 172.21.15.2:0/501177991 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561ff350bd50 0x561ff350e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.402+0000 7f84ca077740 4 client.19143 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.402+0000 7f84ca077740 1 -- 172.21.15.2:0/501177991 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ff3468830 con 0x561ff350bd50 2024-09-10T08:21:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.402+0000 7f84c5200640 1 --2- 172.21.15.2:0/501177991 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561ff350bd50 0x561ff350e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.403+0000 7f84c5200640 1 --2- 172.21.15.2:0/501177991 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561ff350bd50 0x561ff350e190 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688d4cb38719b9e0 server_cookie=d0d8f26c62c6ff94 in_seq=0 out_seq=0 2024-09-10T08:21:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.403+0000 7f84ae000640 10 client.19143 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.403+0000 7f84ae000640 1 -- 172.21.15.2:0/501177991 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ff3468830 con 0x561ff350bd50 2024-09-10T08:21:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.403+0000 7f84ae000640 10 client.19143 handle_command_reply: tid=0 2024-09-10T08:21:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.404+0000 7f84ca077740 1 client.19143 shutdown 2024-09-10T08:21:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.404+0000 7f84ca077740 1 -- 172.21.15.2:0/501177991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f69d0 msgr2=0x561ff3500700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.404+0000 7f84ca077740 1 --2- 172.21.15.2:0/501177991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f69d0 0x561ff3500700 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f84a8012700 tx=0x7f84a8012bc0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.404+0000 7f84ca077740 1 -- 172.21.15.2:0/501177991 shutdown_connections 2024-09-10T08:21:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.404+0000 7f84ca077740 1 --2- 172.21.15.2:0/501177991 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561ff350bd50 0x561ff350e190 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.404+0000 7f84ca077740 1 --2- 172.21.15.2:0/501177991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34fba60 0x561ff3507740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.404+0000 7f84ca077740 1 --2- 172.21.15.2:0/501177991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f69d0 0x561ff3500700 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.404+0000 7f84ca077740 1 --2- 172.21.15.2:0/501177991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34f6090 0x561ff35001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.404+0000 7f84ca077740 1 -- 172.21.15.2:0/501177991 >> 172.21.15.2:0/501177991 conn(0x561ff34f1700 msgr2=0x561ff34f2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.404+0000 7f84ae000640 0 client.19143 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.404+0000 7f84ca077740 1 -- 172.21.15.2:0/501177991 shutdown_connections 2024-09-10T08:21:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.404+0000 7f84ca077740 1 -- 172.21.15.2:0/501177991 wait complete. 2024-09-10T08:21:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.404+0000 7f84ca077740 20 client.19143 trim_cache size 0 max 16384 2024-09-10T08:21:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.419+0000 7f84ca077740 1 Processor -- start 2024-09-10T08:21:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.419+0000 7f84ca077740 1 -- start start 2024-09-10T08:21:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84ca077740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff34f2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84ca077740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34f69d0 0x561ff3507700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84ca077740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34fba60 0x561ff34f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84ca077740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b3670 con 0x561ff34f6090 2024-09-10T08:21:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84ca077740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b0f40 con 0x561ff34fba60 2024-09-10T08:21:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84ca077740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01aed90 con 0x561ff34f69d0 2024-09-10T08:21:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84c5200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff34f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84c5200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff34f2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49768/0 (socket says 172.21.15.2:49768) 2024-09-10T08:21:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84c5200640 1 -- 172.21.15.2:0/3984449362 learned_addr learned my addr 172.21.15.2:0/3984449362 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84b7e00640 1 --2- 172.21.15.2:0/3984449362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34f69d0 0x561ff3507700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84c5200640 1 --2- 172.21.15.2:0/3984449362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34fba60 0x561ff34f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84c5200640 1 -- 172.21.15.2:0/3984449362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34f69d0 msgr2=0x561ff3507700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84c5200640 1 --2- 172.21.15.2:0/3984449362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34f69d0 0x561ff3507700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84c5200640 1 -- 172.21.15.2:0/3984449362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34fba60 msgr2=0x561ff34f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84c5200640 1 --2- 172.21.15.2:0/3984449362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34fba60 0x561ff34f2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.420+0000 7f84c5200640 1 -- 172.21.15.2:0/3984449362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff34f2390 con 0x561ff34f6090 2024-09-10T08:21:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.421+0000 7f84c5200640 1 --2- 172.21.15.2:0/3984449362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff34f2580 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f84b80307c0 tx=0x7f84b8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdac870d9cfe03cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.421+0000 7f84b4c00640 1 -- 172.21.15.2:0/3984449362 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8030b30 con 0x561ff34f6090 2024-09-10T08:21:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.421+0000 7f84b4c00640 1 -- 172.21.15.2:0/3984449362 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b8058030 con 0x561ff34f6090 2024-09-10T08:21:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.421+0000 7f84b4c00640 1 -- 172.21.15.2:0/3984449362 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8003e80 con 0x561ff34f6090 2024-09-10T08:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.422+0000 7f84ca077740 1 -- 172.21.15.2:0/3984449362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 msgr2=0x561ff34f2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.422+0000 7f84ca077740 1 --2- 172.21.15.2:0/3984449362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff34f2580 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f84b80307c0 tx=0x7f84b8009a20 comp rx=0 tx=0).stop 2024-09-10T08:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.422+0000 7f84ca077740 1 -- 172.21.15.2:0/3984449362 shutdown_connections 2024-09-10T08:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.422+0000 7f84ca077740 1 --2- 172.21.15.2:0/3984449362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34fba60 0x561ff34f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.422+0000 7f84ca077740 1 --2- 172.21.15.2:0/3984449362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34f69d0 0x561ff3507700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.422+0000 7f84ca077740 1 --2- 172.21.15.2:0/3984449362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff34f2580 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.422+0000 7f84ca077740 1 -- 172.21.15.2:0/3984449362 >> 172.21.15.2:0/3984449362 conn(0x561ff34f1700 msgr2=0x561ff34f7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.422+0000 7f84ca077740 1 -- 172.21.15.2:0/3984449362 shutdown_connections 2024-09-10T08:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.422+0000 7f84ca077740 1 -- 172.21.15.2:0/3984449362 wait complete. 2024-09-10T08:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.422+0000 7f84ca077740 1 -- start start 2024-09-10T08:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.422+0000 7f84ca077740 1 Processor -- start 2024-09-10T08:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84ca077740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff3504de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84ca077740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34f69d0 0x561ff3505320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84ca077740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34fba60 0x561ff3506330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84ca077740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01af400 con 0x561ff34f6090 2024-09-10T08:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84ca077740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b0870 con 0x561ff34f69d0 2024-09-10T08:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84ca077740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01afad0 con 0x561ff34fba60 2024-09-10T08:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84ca077740 10 client.0 _ll_register_callbacks cb 0x7f84a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84c5200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff3504de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84c5200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff3504de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49784/0 (socket says 172.21.15.2:49784) 2024-09-10T08:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84c5200640 1 -- 172.21.15.2:0/3601305638 learned_addr learned my addr 172.21.15.2:0/3601305638 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84b7e00640 1 --2- 172.21.15.2:0/3601305638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34f69d0 0x561ff3505320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84c5200640 1 --2- 172.21.15.2:0/3601305638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34fba60 0x561ff3506330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84c5200640 1 -- 172.21.15.2:0/3601305638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34fba60 msgr2=0x561ff3506330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84c5200640 1 --2- 172.21.15.2:0/3601305638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34fba60 0x561ff3506330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84c5200640 1 -- 172.21.15.2:0/3601305638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34f69d0 msgr2=0x561ff3505320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84c5200640 1 --2- 172.21.15.2:0/3601305638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34f69d0 0x561ff3505320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.423+0000 7f84c5200640 1 -- 172.21.15.2:0/3601305638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ff3506940 con 0x561ff34f6090 2024-09-10T08:21:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.424+0000 7f84c5200640 1 --2- 172.21.15.2:0/3601305638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff3504de0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f84b8030990 tx=0x7f84b8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c2bfdd927d8c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.424+0000 7f84ae000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.424+0000 7f84ae000640 1 -- 172.21.15.2:0/3601305638 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b804a040 con 0x561ff34f6090 2024-09-10T08:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.424+0000 7f84ca077740 1 -- 172.21.15.2:0/3601305638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b8058030 con 0x561ff34f6090 2024-09-10T08:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.424+0000 7f84ae000640 1 -- 172.21.15.2:0/3601305638 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f84b8065470 con 0x561ff34f6090 2024-09-10T08:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.424+0000 7f84ae000640 1 -- 172.21.15.2:0/3601305638 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84b806e050 con 0x561ff34f6090 2024-09-10T08:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.424+0000 7f84ca077740 10 client.19149 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.424+0000 7f84ca077740 1 -- 172.21.15.2:0/3601305638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561ff3576a30 con 0x561ff34f6090 2024-09-10T08:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.424+0000 7f84ae000640 1 -- 172.21.15.2:0/3601305638 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b8058030 con 0x561ff34f6090 2024-09-10T08:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.425+0000 7f84ae000640 1 -- 172.21.15.2:0/3601305638 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f84b8043070 con 0x561ff34f6090 2024-09-10T08:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.425+0000 7f84ca077740 10 client.19149 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.425+0000 7f84ca077740 10 client.19149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.425+0000 7f84ca077740 20 client.19149 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.425+0000 7f84ca077740 1 --2- 172.21.15.2:0/3601305638 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561ff3576ec0 0x561ff35792e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.425+0000 7f84ca077740 4 client.19149 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:21:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.425+0000 7f84ca077740 1 -- 172.21.15.2:0/3601305638 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ff3462b80 con 0x561ff3576ec0 2024-09-10T08:21:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.425+0000 7f84b7e00640 1 --2- 172.21.15.2:0/3601305638 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561ff3576ec0 0x561ff35792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.426+0000 7f84b7e00640 1 --2- 172.21.15.2:0/3601305638 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561ff3576ec0 0x561ff35792e0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d0c85d6ce271b8 server_cookie=87baaa37c111f14f in_seq=0 out_seq=0 2024-09-10T08:21:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.426+0000 7f84ae000640 10 client.19149 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ae000640 1 -- 172.21.15.2:0/3601305638 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ff3462b80 con 0x561ff3576ec0 2024-09-10T08:21:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ae000640 10 client.19149 handle_command_reply: tid=0 2024-09-10T08:21:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ca077740 1 client.19149 shutdown 2024-09-10T08:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ca077740 1 -- 172.21.15.2:0/3601305638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 msgr2=0x561ff3504de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ca077740 1 --2- 172.21.15.2:0/3601305638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff3504de0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f84b8030990 tx=0x7f84b8003970 comp rx=0 tx=0).stop 2024-09-10T08:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ca077740 1 -- 172.21.15.2:0/3601305638 shutdown_connections 2024-09-10T08:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ca077740 1 --2- 172.21.15.2:0/3601305638 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561ff3576ec0 0x561ff35792e0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ca077740 1 --2- 172.21.15.2:0/3601305638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ff34fba60 0x561ff3506330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ca077740 1 --2- 172.21.15.2:0/3601305638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ff34f69d0 0x561ff3505320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ca077740 1 --2- 172.21.15.2:0/3601305638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561ff34f6090 0x561ff3504de0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ae000640 0 client.19149 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ca077740 1 -- 172.21.15.2:0/3601305638 >> 172.21.15.2:0/3601305638 conn(0x561ff34f1700 msgr2=0x561ff34f0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ca077740 1 -- 172.21.15.2:0/3601305638 shutdown_connections 2024-09-10T08:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ca077740 1 -- 172.21.15.2:0/3601305638 wait complete. 2024-09-10T08:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.440+0000 7f84ca077740 20 client.19149 trim_cache size 0 max 16384 2024-09-10T08:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:21:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.441+0000 7f84c7200640 1 -- 172.21.15.2:0/4052623619 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f84a40a2cb0 msgr2=0x7f84a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.441+0000 7f84c7200640 1 --2- 172.21.15.2:0/4052623619 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f84a40a2cb0 0x7f84a40a5170 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f84b00cd1a0 tx=0x7f84b0009700 comp rx=0 tx=0).stop 2024-09-10T08:21:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.441+0000 7f84c7200640 1 -- 172.21.15.2:0/4052623619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c0076bb0 msgr2=0x7f84c019e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.441+0000 7f84c7200640 1 --2- 172.21.15.2:0/4052623619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c0076bb0 0x7f84c019e360 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f84b000e9f0 tx=0x7f84b000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.441+0000 7f84c7200640 1 -- 172.21.15.2:0/4052623619 shutdown_connections 2024-09-10T08:21:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.441+0000 7f84c7200640 1 --2- 172.21.15.2:0/4052623619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84c0078790 0x7f84c01a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.441+0000 7f84c7200640 1 --2- 172.21.15.2:0/4052623619 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f84a40a2cb0 0x7f84a40a5170 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.441+0000 7f84c7200640 1 --2- 172.21.15.2:0/4052623619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84c0077df0 0x7f84c019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.441+0000 7f84c7200640 1 --2- 172.21.15.2:0/4052623619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c0076bb0 0x7f84c019e360 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.441+0000 7f84c7200640 1 -- 172.21.15.2:0/4052623619 >> 172.21.15.2:0/4052623619 conn(0x7f84c00ff4e0 msgr2=0x7f84c0100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.441+0000 7f84c7200640 1 -- 172.21.15.2:0/4052623619 shutdown_connections 2024-09-10T08:21:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:27.441+0000 7f84c7200640 1 -- 172.21.15.2:0/4052623619 wait complete. 2024-09-10T08:21:28.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:27 smithi002 ceph-mon[30563]: pgmap v786: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:21:28.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:27 smithi155 ceph-mon[35405]: pgmap v786: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:21:28.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:27 smithi107 ceph-mon[35048]: pgmap v786: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:21:28.985 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:21:29.378 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:29.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:29 smithi002 ceph-mon[30563]: pgmap v787: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:21:29.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.980+0000 7f7b83800640 1 -- 172.21.15.2:0/234023310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b7c112a30 msgr2=0x7f7b7c114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:29.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.980+0000 7f7b83800640 1 --2- 172.21.15.2:0/234023310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b7c112a30 0x7f7b7c114e20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009ca0 tx=0x7f7b7402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.980+0000 7f7b83800640 1 -- 172.21.15.2:0/234023310 shutdown_connections 2024-09-10T08:21:30.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.980+0000 7f7b83800640 1 --2- 172.21.15.2:0/234023310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b7c112a30 0x7f7b7c114e20 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.980+0000 7f7b83800640 1 --2- 172.21.15.2:0/234023310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b7c077040 0x7f7b7c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.980+0000 7f7b83800640 1 --2- 172.21.15.2:0/234023310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b7c076720 0x7f7b7c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.980+0000 7f7b83800640 1 -- 172.21.15.2:0/234023310 >> 172.21.15.2:0/234023310 conn(0x7f7b7c0ffa90 msgr2=0x7f7b7c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:30.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.980+0000 7f7b83800640 1 -- 172.21.15.2:0/234023310 shutdown_connections 2024-09-10T08:21:30.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.980+0000 7f7b83800640 1 -- 172.21.15.2:0/234023310 wait complete. 2024-09-10T08:21:30.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.982+0000 7f7b83800640 1 Processor -- start 2024-09-10T08:21:30.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.982+0000 7f7b83800640 1 -- start start 2024-09-10T08:21:30.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.982+0000 7f7b83800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b7c076720 0x7f7b7c1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.982+0000 7f7b83800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b7c077040 0x7f7b7c1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.982+0000 7f7b83800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b7c112a30 0x7f7b7c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.982+0000 7f7b83800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c118570 con 0x7f7b7c077040 2024-09-10T08:21:30.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.982+0000 7f7b83800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1183f0 con 0x7f7b7c112a30 2024-09-10T08:21:30.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.982+0000 7f7b83800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1186f0 con 0x7f7b7c076720 2024-09-10T08:21:30.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.983+0000 7f7b80e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b7c076720 0x7f7b7c1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.983+0000 7f7b80e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b7c076720 0x7f7b7c1a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55438/0 (socket says 172.21.15.2:55438) 2024-09-10T08:21:30.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.983+0000 7f7b7be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b7c077040 0x7f7b7c1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.983+0000 7f7b80e00640 1 -- 172.21.15.2:0/3861374941 learned_addr learned my addr 172.21.15.2:0/3861374941 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:30.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.983+0000 7f7b81800640 1 --2- 172.21.15.2:0/3861374941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b7c112a30 0x7f7b7c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.983+0000 7f7b80e00640 1 -- 172.21.15.2:0/3861374941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b7c112a30 msgr2=0x7f7b7c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:30.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.983+0000 7f7b80e00640 1 --2- 172.21.15.2:0/3861374941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b7c112a30 0x7f7b7c19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.983+0000 7f7b80e00640 1 -- 172.21.15.2:0/3861374941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b7c077040 msgr2=0x7f7b7c1a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:30.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.983+0000 7f7b80e00640 1 --2- 172.21.15.2:0/3861374941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b7c077040 0x7f7b7c1a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.983+0000 7f7b80e00640 1 -- 172.21.15.2:0/3861374941 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7c19efd0 con 0x7f7b7c076720 2024-09-10T08:21:30.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.983+0000 7f7b80e00640 1 --2- 172.21.15.2:0/3861374941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b7c076720 0x7f7b7c1a46b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000d950 tx=0x7f7b7000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:30.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.984+0000 7f7b79600640 1 -- 172.21.15.2:0/3861374941 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70017030 con 0x7f7b7c076720 2024-09-10T08:21:30.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.984+0000 7f7b79600640 1 -- 172.21.15.2:0/3861374941 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b70004020 con 0x7f7b7c076720 2024-09-10T08:21:30.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.984+0000 7f7b79600640 1 -- 172.21.15.2:0/3861374941 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b700047a0 con 0x7f7b7c076720 2024-09-10T08:21:30.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.984+0000 7f7b83800640 1 -- 172.21.15.2:0/3861374941 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7c19f2c0 con 0x7f7b7c076720 2024-09-10T08:21:30.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.984+0000 7f7b83800640 1 -- 172.21.15.2:0/3861374941 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7c1ab510 con 0x7f7b7c076720 2024-09-10T08:21:30.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.987+0000 7f7b862c3740 1 Processor -- start 2024-09-10T08:21:30.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.987+0000 7f7b862c3740 1 -- start start 2024-09-10T08:21:30.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.987+0000 7f7b79600640 1 -- 172.21.15.2:0/3861374941 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7b700041d0 con 0x7f7b7c076720 2024-09-10T08:21:30.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.987+0000 7f7b862c3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0e9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.987+0000 7f7b862c3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0ea4b0 0x55c97e0f4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.987+0000 7f7b862c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0f4d70 0x55c97e0f7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.987+0000 7f7b862c3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c97e0503e0 con 0x55c97e0ea4b0 2024-09-10T08:21:30.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.987+0000 7f7b862c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c97dff7d80 con 0x55c97e0f4d70 2024-09-10T08:21:30.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.987+0000 7f7b862c3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c97e04c3a0 con 0x55c97e0e9b90 2024-09-10T08:21:30.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.988+0000 7f7b81800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0e9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.988+0000 7f7b80e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0ea4b0 0x55c97e0f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.988+0000 7f7b81800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0e9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55452/0 (socket says 172.21.15.2:55452) 2024-09-10T08:21:30.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.988+0000 7f7b81800640 1 -- 172.21.15.2:0/1375785828 learned_addr learned my addr 172.21.15.2:0/1375785828 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:30.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.988+0000 7f7b7be00640 1 --2- 172.21.15.2:0/1375785828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0f4d70 0x55c97e0f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.988+0000 7f7b81800640 1 -- 172.21.15.2:0/1375785828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0f4d70 msgr2=0x55c97e0f7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:30.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.988+0000 7f7b81800640 1 --2- 172.21.15.2:0/1375785828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0f4d70 0x55c97e0f7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.988+0000 7f7b81800640 1 -- 172.21.15.2:0/1375785828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0ea4b0 msgr2=0x55c97e0f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:30.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.988+0000 7f7b81800640 1 --2- 172.21.15.2:0/1375785828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0ea4b0 0x55c97e0f4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.988+0000 7f7b81800640 1 -- 172.21.15.2:0/1375785828 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97e016ae0 con 0x55c97e0e9b90 2024-09-10T08:21:30.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.989+0000 7f7b81800640 1 --2- 172.21.15.2:0/1375785828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0e9f70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7b74004af0 tx=0x7f7b740098b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78b0d2cd5efddc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:30.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.989+0000 7f7b6e000640 1 -- 172.21.15.2:0/1375785828 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74003970 con 0x55c97e0e9b90 2024-09-10T08:21:30.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.989+0000 7f7b6e000640 1 -- 172.21.15.2:0/1375785828 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b74036040 con 0x55c97e0e9b90 2024-09-10T08:21:30.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.989+0000 7f7b6e000640 1 -- 172.21.15.2:0/1375785828 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74004310 con 0x55c97e0e9b90 2024-09-10T08:21:30.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.989+0000 7f7b79600640 1 --2- 172.21.15.2:0/3861374941 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7b640a2c00 0x7f7b640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.989+0000 7f7b79600640 1 -- 172.21.15.2:0/3861374941 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7b700ced50 con 0x7f7b7c076720 2024-09-10T08:21:30.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.989+0000 7f7b80e00640 1 --2- 172.21.15.2:0/3861374941 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7b640a2c00 0x7f7b640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.990+0000 7f7b862c3740 1 -- 172.21.15.2:0/1375785828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 msgr2=0x55c97e0e9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:30.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.990+0000 7f7b862c3740 1 --2- 172.21.15.2:0/1375785828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0e9f70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7b74004af0 tx=0x7f7b740098b0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.990+0000 7f7b862c3740 1 -- 172.21.15.2:0/1375785828 shutdown_connections 2024-09-10T08:21:30.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.990+0000 7f7b862c3740 1 --2- 172.21.15.2:0/1375785828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0f4d70 0x55c97e0f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.990+0000 7f7b862c3740 1 --2- 172.21.15.2:0/1375785828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0ea4b0 0x55c97e0f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.990+0000 7f7b862c3740 1 --2- 172.21.15.2:0/1375785828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0e9f70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.990+0000 7f7b862c3740 1 -- 172.21.15.2:0/1375785828 >> 172.21.15.2:0/1375785828 conn(0x55c97e0e6120 msgr2=0x55c97e0e6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:30.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.990+0000 7f7b862c3740 1 -- 172.21.15.2:0/1375785828 shutdown_connections 2024-09-10T08:21:30.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.990+0000 7f7b80e00640 1 --2- 172.21.15.2:0/3861374941 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7b640a2c00 0x7f7b640a50c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7b7009b0f0 tx=0x7f7b700d2790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:30.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.990+0000 7f7b862c3740 1 -- 172.21.15.2:0/1375785828 wait complete. 2024-09-10T08:21:30.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.990+0000 7f7b862c3740 1 -- start start 2024-09-10T08:21:30.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.991+0000 7f7b862c3740 1 Processor -- start 2024-09-10T08:21:30.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.991+0000 7f7b862c3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.991+0000 7f7b862c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0ea4b0 0x55c97e0f2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.991+0000 7f7b862c3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0f4d70 0x55c97e0f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.991+0000 7f7b862c3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c97e0f7860 con 0x55c97e0f4d70 2024-09-10T08:21:30.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.991+0000 7f7b862c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c97e00b0f0 con 0x55c97e0ea4b0 2024-09-10T08:21:30.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.991+0000 7f7b862c3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c97e0f79e0 con 0x55c97e0e9b90 2024-09-10T08:21:30.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.991+0000 7f7b862c3740 10 client.0 _ll_register_callbacks cb 0x55c97dffc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:30.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.991+0000 7f7b81800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0f4d70 0x55c97e0f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.991+0000 7f7b7be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0ea4b0 0x55c97e0f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.991+0000 7f7b81800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.991+0000 7f7b7be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0ea4b0 0x55c97e0f2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58618/0 (socket says 172.21.15.2:58618) 2024-09-10T08:21:30.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.991+0000 7f7b7be00640 1 -- 172.21.15.2:0/652123432 learned_addr learned my addr 172.21.15.2:0/652123432 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:30.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.992+0000 7f7b7be00640 1 -- 172.21.15.2:0/652123432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 msgr2=0x55c97e0f1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:30.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.992+0000 7f7b7be00640 1 --2- 172.21.15.2:0/652123432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0f1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.992+0000 7f7b7be00640 1 -- 172.21.15.2:0/652123432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0f4d70 msgr2=0x55c97e0f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:30.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.992+0000 7f7b7be00640 1 --2- 172.21.15.2:0/652123432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0f4d70 0x55c97e0f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.992+0000 7f7b7be00640 1 -- 172.21.15.2:0/652123432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c97ddc13c0 con 0x55c97e0ea4b0 2024-09-10T08:21:30.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.992+0000 7f7b81800640 1 --2- 172.21.15.2:0/652123432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:30.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.992+0000 7f7b7be00640 1 --2- 172.21.15.2:0/652123432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0ea4b0 0x55c97e0f2190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800bed0 tx=0x7f7b68011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7380c91c4a57365 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:30.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.992+0000 7f7b62a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:21:30.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.992+0000 7f7b62a00640 1 -- 172.21.15.2:0/652123432 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68004020 con 0x55c97e0ea4b0 2024-09-10T08:21:30.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.993+0000 7f7b862c3740 1 -- 172.21.15.2:0/652123432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b74036040 con 0x55c97e0ea4b0 2024-09-10T08:21:30.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.993+0000 7f7b62a00640 1 -- 172.21.15.2:0/652123432 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7b680041c0 con 0x55c97e0ea4b0 2024-09-10T08:21:30.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.993+0000 7f7b62a00640 1 -- 172.21.15.2:0/652123432 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b68024050 con 0x55c97e0ea4b0 2024-09-10T08:21:30.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.993+0000 7f7b862c3740 10 client.29183 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:30.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.993+0000 7f7b862c3740 1 -- 172.21.15.2:0/652123432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c97e103560 con 0x55c97e0ea4b0 2024-09-10T08:21:30.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.993+0000 7f7b62a00640 1 -- 172.21.15.2:0/652123432 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b74036040 con 0x55c97e0ea4b0 2024-09-10T08:21:30.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.993+0000 7f7b62a00640 1 -- 172.21.15.2:0/652123432 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7b68003850 con 0x55c97e0ea4b0 2024-09-10T08:21:30.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.993+0000 7f7b862c3740 10 client.29183 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:30.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.993+0000 7f7b862c3740 10 client.29183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:30.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.993+0000 7f7b862c3740 20 client.29183 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:30.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.994+0000 7f7b862c3740 1 --2- 172.21.15.2:0/652123432 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c97e104a30 0x55c97e106e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.994+0000 7f7b862c3740 4 client.29183 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:30.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.994+0000 7f7b862c3740 1 -- 172.21.15.2:0/652123432 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c97e05d950 con 0x55c97e104a30 2024-09-10T08:21:30.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.994+0000 7f7b81800640 1 --2- 172.21.15.2:0/652123432 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c97e104a30 0x55c97e106e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.994+0000 7f7b81800640 1 --2- 172.21.15.2:0/652123432 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c97e104a30 0x55c97e106e70 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c251aa4ff5d7766 server_cookie=ae55b3d33fcdb401 in_seq=0 out_seq=0 2024-09-10T08:21:30.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.994+0000 7f7b62a00640 10 client.29183 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:30.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.995+0000 7f7b62a00640 1 -- 172.21.15.2:0/652123432 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c97e05d950 con 0x55c97e104a30 2024-09-10T08:21:30.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.995+0000 7f7b62a00640 10 client.29183 handle_command_reply: tid=0 2024-09-10T08:21:30.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.995+0000 7f7b862c3740 1 client.29183 shutdown 2024-09-10T08:21:30.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.995+0000 7f7b862c3740 1 -- 172.21.15.2:0/652123432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0ea4b0 msgr2=0x55c97e0f2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:30.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.995+0000 7f7b862c3740 1 --2- 172.21.15.2:0/652123432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0ea4b0 0x55c97e0f2190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800bed0 tx=0x7f7b68011810 comp rx=0 tx=0).stop 2024-09-10T08:21:30.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.996+0000 7f7b862c3740 1 -- 172.21.15.2:0/652123432 shutdown_connections 2024-09-10T08:21:30.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.996+0000 7f7b862c3740 1 --2- 172.21.15.2:0/652123432 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c97e104a30 0x55c97e106e70 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.996+0000 7f7b862c3740 1 --2- 172.21.15.2:0/652123432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0f4d70 0x55c97e0f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.996+0000 7f7b862c3740 1 --2- 172.21.15.2:0/652123432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0ea4b0 0x55c97e0f2190 secure :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800bed0 tx=0x7f7b68011810 comp rx=0 tx=0).stop 2024-09-10T08:21:30.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.996+0000 7f7b862c3740 1 --2- 172.21.15.2:0/652123432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.996+0000 7f7b862c3740 1 -- 172.21.15.2:0/652123432 >> 172.21.15.2:0/652123432 conn(0x55c97e0e6120 msgr2=0x55c97e0ebe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:30.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.996+0000 7f7b62a00640 0 client.29183 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:30.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.996+0000 7f7b862c3740 1 -- 172.21.15.2:0/652123432 shutdown_connections 2024-09-10T08:21:30.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.996+0000 7f7b862c3740 1 -- 172.21.15.2:0/652123432 wait complete. 2024-09-10T08:21:30.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:29.996+0000 7f7b862c3740 20 client.29183 trim_cache size 0 max 16384 2024-09-10T08:21:30.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.014+0000 7f7b862c3740 1 Processor -- start 2024-09-10T08:21:30.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.014+0000 7f7b862c3740 1 -- start start 2024-09-10T08:21:30.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.014+0000 7f7b862c3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.014+0000 7f7b862c3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0f4d70 0x55c97e10c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.014+0000 7f7b862c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e10cc80 0x55c97e0f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.014+0000 7f7b862c3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1b7ca0 con 0x55c97e0f4d70 2024-09-10T08:21:30.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.014+0000 7f7b862c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1b5570 con 0x55c97e10cc80 2024-09-10T08:21:30.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.014+0000 7f7b862c3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1b33c0 con 0x55c97e0e9b90 2024-09-10T08:21:30.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.015+0000 7f7b7be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0f4d70 0x55c97e10c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.015+0000 7f7b81800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e10cc80 0x55c97e0f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.015+0000 7f7b81800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e10cc80 0x55c97e0f1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58622/0 (socket says 172.21.15.2:58622) 2024-09-10T08:21:30.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.015+0000 7f7b81800640 1 -- 172.21.15.2:0/315342829 learned_addr learned my addr 172.21.15.2:0/315342829 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:30.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.015+0000 7f7b81800640 1 --2- 172.21.15.2:0/315342829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.015+0000 7f7b7be00640 1 -- 172.21.15.2:0/315342829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 msgr2=0x55c97e0ec5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:30.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.015+0000 7f7b7be00640 1 --2- 172.21.15.2:0/315342829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0ec5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.015+0000 7f7b7be00640 1 -- 172.21.15.2:0/315342829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e10cc80 msgr2=0x55c97e0f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:30.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.015+0000 7f7b7be00640 1 --2- 172.21.15.2:0/315342829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e10cc80 0x55c97e0f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.015+0000 7f7b7be00640 1 -- 172.21.15.2:0/315342829 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97e0e5f50 con 0x55c97e0f4d70 2024-09-10T08:21:30.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.015+0000 7f7b81800640 1 --2- 172.21.15.2:0/315342829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e10cc80 0x55c97e0f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:30.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.015+0000 7f7b7be00640 1 --2- 172.21.15.2:0/315342829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0f4d70 0x55c97e10c740 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x55c97e0f2d40 tx=0x7f7b6800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7ec784c8c104b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:30.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.016+0000 7f7b6d600640 1 -- 172.21.15.2:0/315342829 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6802c070 con 0x55c97e0f4d70 2024-09-10T08:21:30.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.016+0000 7f7b6d600640 1 -- 172.21.15.2:0/315342829 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b68003ee0 con 0x55c97e0f4d70 2024-09-10T08:21:30.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.016+0000 7f7b6d600640 1 -- 172.21.15.2:0/315342829 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68005280 con 0x55c97e0f4d70 2024-09-10T08:21:30.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.016+0000 7f7b862c3740 1 -- 172.21.15.2:0/315342829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0f4d70 msgr2=0x55c97e10c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:30.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.016+0000 7f7b862c3740 1 --2- 172.21.15.2:0/315342829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0f4d70 0x55c97e10c740 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x55c97e0f2d40 tx=0x7f7b6800b480 comp rx=0 tx=0).stop 2024-09-10T08:21:30.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.016+0000 7f7b862c3740 1 -- 172.21.15.2:0/315342829 shutdown_connections 2024-09-10T08:21:30.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.016+0000 7f7b862c3740 1 --2- 172.21.15.2:0/315342829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e10cc80 0x55c97e0f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.016+0000 7f7b862c3740 1 --2- 172.21.15.2:0/315342829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0f4d70 0x55c97e10c740 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.016+0000 7f7b862c3740 1 --2- 172.21.15.2:0/315342829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e0e9b90 0x55c97e0ec5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.016+0000 7f7b862c3740 1 -- 172.21.15.2:0/315342829 >> 172.21.15.2:0/315342829 conn(0x55c97e0e6120 msgr2=0x55c97e0f0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:30.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.017+0000 7f7b862c3740 1 -- 172.21.15.2:0/315342829 shutdown_connections 2024-09-10T08:21:30.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.017+0000 7f7b862c3740 1 -- 172.21.15.2:0/315342829 wait complete. 2024-09-10T08:21:30.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.017+0000 7f7b862c3740 1 -- start start 2024-09-10T08:21:30.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.017+0000 7f7b862c3740 1 Processor -- start 2024-09-10T08:21:30.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b862c3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0e9b90 0x55c97e0fd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b862c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0f4d70 0x55c97e0f8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b862c3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e10cc80 0x55c97e0f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b862c3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1b3a30 con 0x55c97e0e9b90 2024-09-10T08:21:30.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b862c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1b4ea0 con 0x55c97e0f4d70 2024-09-10T08:21:30.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b862c3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1b4100 con 0x55c97e10cc80 2024-09-10T08:21:30.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b862c3740 10 client.0 _ll_register_callbacks cb 0x7f7b6800b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:30.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b81800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0e9b90 0x55c97e0fd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b7be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0f4d70 0x55c97e0f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b81800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0e9b90 0x55c97e0fd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49830/0 (socket says 172.21.15.2:49830) 2024-09-10T08:21:30.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b81800640 1 -- 172.21.15.2:0/2331241856 learned_addr learned my addr 172.21.15.2:0/2331241856 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:30.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b81800640 1 --2- 172.21.15.2:0/2331241856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e10cc80 0x55c97e0f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b7be00640 1 -- 172.21.15.2:0/2331241856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e10cc80 msgr2=0x55c97e0f8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:30.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b7be00640 1 --2- 172.21.15.2:0/2331241856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e10cc80 0x55c97e0f8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b7be00640 1 -- 172.21.15.2:0/2331241856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0e9b90 msgr2=0x55c97e0fd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:30.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.018+0000 7f7b7be00640 1 --2- 172.21.15.2:0/2331241856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0e9b90 0x55c97e0fd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.019+0000 7f7b7be00640 1 -- 172.21.15.2:0/2331241856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c97e0f9380 con 0x55c97e0f4d70 2024-09-10T08:21:30.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.019+0000 7f7b81800640 1 --2- 172.21.15.2:0/2331241856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e10cc80 0x55c97e0f8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:21:30.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.019+0000 7f7b7be00640 1 --2- 172.21.15.2:0/2331241856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0f4d70 0x55c97e0f8610 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x55c97e0f79d0 tx=0x7f7b6800b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2a10a3faaceccfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:30.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.019+0000 7f7b62a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:21:30.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.019+0000 7f7b62a00640 1 -- 172.21.15.2:0/2331241856 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68037070 con 0x55c97e0f4d70 2024-09-10T08:21:30.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.019+0000 7f7b862c3740 1 -- 172.21.15.2:0/2331241856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b68003ee0 con 0x55c97e0f4d70 2024-09-10T08:21:30.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.019+0000 7f7b62a00640 1 -- 172.21.15.2:0/2331241856 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7b68010700 con 0x55c97e0f4d70 2024-09-10T08:21:30.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.020+0000 7f7b62a00640 1 -- 172.21.15.2:0/2331241856 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b6802c040 con 0x55c97e0f4d70 2024-09-10T08:21:30.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.020+0000 7f7b862c3740 10 client.29189 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:30.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.020+0000 7f7b862c3740 1 -- 172.21.15.2:0/2331241856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c97e16f630 con 0x55c97e0f4d70 2024-09-10T08:21:30.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.020+0000 7f7b62a00640 1 -- 172.21.15.2:0/2331241856 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b68003ee0 con 0x55c97e0f4d70 2024-09-10T08:21:30.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.020+0000 7f7b62a00640 1 -- 172.21.15.2:0/2331241856 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7b680315e0 con 0x55c97e0f4d70 2024-09-10T08:21:30.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.020+0000 7f7b862c3740 10 client.29189 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:30.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.020+0000 7f7b862c3740 10 client.29189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:30.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.020+0000 7f7b862c3740 20 client.29189 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:30.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.020+0000 7f7b862c3740 1 --2- 172.21.15.2:0/2331241856 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c97e16fd20 0x55c97e172140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:30.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.020+0000 7f7b862c3740 4 client.29189 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:21:30.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.020+0000 7f7b862c3740 1 -- 172.21.15.2:0/2331241856 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c97e057e60 con 0x55c97e16fd20 2024-09-10T08:21:30.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.020+0000 7f7b81800640 1 --2- 172.21.15.2:0/2331241856 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c97e16fd20 0x55c97e172140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:30.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b81800640 1 --2- 172.21.15.2:0/2331241856 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c97e16fd20 0x55c97e172140 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c14c5931f1ba389 server_cookie=8de5560e5f56a8c9 in_seq=0 out_seq=0 2024-09-10T08:21:30.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b62a00640 10 client.29189 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b62a00640 1 -- 172.21.15.2:0/2331241856 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c97e057e60 con 0x55c97e16fd20 2024-09-10T08:21:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b62a00640 10 client.29189 handle_command_reply: tid=0 2024-09-10T08:21:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b862c3740 1 client.29189 shutdown 2024-09-10T08:21:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b862c3740 1 -- 172.21.15.2:0/2331241856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0f4d70 msgr2=0x55c97e0f8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b862c3740 1 --2- 172.21.15.2:0/2331241856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0f4d70 0x55c97e0f8610 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x55c97e0f79d0 tx=0x7f7b6800b410 comp rx=0 tx=0).stop 2024-09-10T08:21:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b862c3740 1 -- 172.21.15.2:0/2331241856 shutdown_connections 2024-09-10T08:21:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b862c3740 1 --2- 172.21.15.2:0/2331241856 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c97e16fd20 0x55c97e172140 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b862c3740 1 --2- 172.21.15.2:0/2331241856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97e10cc80 0x55c97e0f8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b862c3740 1 --2- 172.21.15.2:0/2331241856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c97e0f4d70 0x55c97e0f8610 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b862c3740 1 --2- 172.21.15.2:0/2331241856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97e0e9b90 0x55c97e0fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b862c3740 1 -- 172.21.15.2:0/2331241856 >> 172.21.15.2:0/2331241856 conn(0x55c97e0e6120 msgr2=0x55c97e0f34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.021+0000 7f7b62a00640 0 client.29189 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:30.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.022+0000 7f7b862c3740 1 -- 172.21.15.2:0/2331241856 shutdown_connections 2024-09-10T08:21:30.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.022+0000 7f7b862c3740 1 -- 172.21.15.2:0/2331241856 wait complete. 2024-09-10T08:21:30.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.022+0000 7f7b862c3740 20 client.29189 trim_cache size 0 max 16384 2024-09-10T08:21:30.026 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:21:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:21:30.026 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:21:30.026 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:21:30.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.022+0000 7f7b83800640 1 -- 172.21.15.2:0/3861374941 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7b640a2c00 msgr2=0x7f7b640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:30.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.022+0000 7f7b83800640 1 --2- 172.21.15.2:0/3861374941 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7b640a2c00 0x7f7b640a50c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7b7009b0f0 tx=0x7f7b700d2790 comp rx=0 tx=0).stop 2024-09-10T08:21:30.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.022+0000 7f7b83800640 1 -- 172.21.15.2:0/3861374941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b7c076720 msgr2=0x7f7b7c1a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:30.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.022+0000 7f7b83800640 1 --2- 172.21.15.2:0/3861374941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b7c076720 0x7f7b7c1a46b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000d950 tx=0x7f7b7000de10 comp rx=0 tx=0).stop 2024-09-10T08:21:30.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.022+0000 7f7b83800640 1 -- 172.21.15.2:0/3861374941 shutdown_connections 2024-09-10T08:21:30.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.022+0000 7f7b83800640 1 --2- 172.21.15.2:0/3861374941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b7c112a30 0x7f7b7c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.022+0000 7f7b83800640 1 --2- 172.21.15.2:0/3861374941 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b7c077040 0x7f7b7c1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.022+0000 7f7b83800640 1 --2- 172.21.15.2:0/3861374941 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7b640a2c00 0x7f7b640a50c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.022+0000 7f7b83800640 1 --2- 172.21.15.2:0/3861374941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b7c076720 0x7f7b7c1a46b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:30.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.022+0000 7f7b83800640 1 -- 172.21.15.2:0/3861374941 >> 172.21.15.2:0/3861374941 conn(0x7f7b7c0ffa90 msgr2=0x7f7b7c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:30.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.023+0000 7f7b83800640 1 -- 172.21.15.2:0/3861374941 shutdown_connections 2024-09-10T08:21:30.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:30.023+0000 7f7b83800640 1 -- 172.21.15.2:0/3861374941 wait complete. 2024-09-10T08:21:30.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:29 smithi155 ceph-mon[35405]: pgmap v787: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:21:30.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:29 smithi107 ceph-mon[35048]: pgmap v787: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:21:30.916 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:21:30.917 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:21:31.296 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:31.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.853+0000 7fc4d9a00640 1 -- 172.21.15.2:0/284206071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4d4102790 msgr2=0x7fc4d41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:31.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.853+0000 7fc4d9a00640 1 --2- 172.21.15.2:0/284206071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4d4102790 0x7fc4d41067e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009900 tx=0x7fc4c002a410 comp rx=0 tx=0).stop 2024-09-10T08:21:31.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.853+0000 7fc4d9a00640 1 -- 172.21.15.2:0/284206071 shutdown_connections 2024-09-10T08:21:31.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.853+0000 7fc4d9a00640 1 --2- 172.21.15.2:0/284206071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4d4106f10 0x7fc4d410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:31.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.853+0000 7fc4d9a00640 1 --2- 172.21.15.2:0/284206071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4d4102790 0x7fc4d41067e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:31.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.853+0000 7fc4d9a00640 1 --2- 172.21.15.2:0/284206071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4d4101de0 0x7fc4d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:31.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.853+0000 7fc4d9a00640 1 -- 172.21.15.2:0/284206071 >> 172.21.15.2:0/284206071 conn(0x7fc4d40fd8c0 msgr2=0x7fc4d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.853+0000 7fc4d9a00640 1 -- 172.21.15.2:0/284206071 shutdown_connections 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.853+0000 7fc4d9a00640 1 -- 172.21.15.2:0/284206071 wait complete. 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.854+0000 7fc4d9a00640 1 Processor -- start 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.854+0000 7fc4d9a00640 1 -- start start 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.855+0000 7fc4d9a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4d4101de0 0x7fc4d40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.855+0000 7fc4d9a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4d4102790 0x7fc4d4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.855+0000 7fc4d9a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4d4106f10 0x7fc4d4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.855+0000 7fc4d9a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4111fc0 con 0x7fc4d4106f10 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.855+0000 7fc4d9a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4111e40 con 0x7fc4d4101de0 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.855+0000 7fc4d9a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4112140 con 0x7fc4d4102790 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.855+0000 7fc4d2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4d4101de0 0x7fc4d40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.855+0000 7fc4d2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4d4101de0 0x7fc4d40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58644/0 (socket says 172.21.15.2:58644) 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.855+0000 7fc4d2a00640 1 -- 172.21.15.2:0/149707280 learned_addr learned my addr 172.21.15.2:0/149707280 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:31.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.855+0000 7fc4d3400640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4d4106f10 0x7fc4d4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:31.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.855+0000 7fc4d2000640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4d4102790 0x7fc4d4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:31.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.856+0000 7fc4d2a00640 1 -- 172.21.15.2:0/149707280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4d4102790 msgr2=0x7fc4d4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:31.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.856+0000 7fc4d2a00640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4d4102790 0x7fc4d4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:31.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.856+0000 7fc4d2a00640 1 -- 172.21.15.2:0/149707280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4d4106f10 msgr2=0x7fc4d4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:31.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.856+0000 7fc4d2a00640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4d4106f10 0x7fc4d4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:31.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.856+0000 7fc4d2a00640 1 -- 172.21.15.2:0/149707280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d41a68c0 con 0x7fc4d4101de0 2024-09-10T08:21:31.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.856+0000 7fc4d2a00640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4d4101de0 0x7fc4d40744f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400ea60 tx=0x7fc4c400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:31.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.856+0000 7fc4cb400640 1 -- 172.21.15.2:0/149707280 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400cde0 con 0x7fc4d4101de0 2024-09-10T08:21:31.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.856+0000 7fc4d9a00640 1 -- 172.21.15.2:0/149707280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d41a6bb0 con 0x7fc4d4101de0 2024-09-10T08:21:31.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.856+0000 7fc4cb400640 1 -- 172.21.15.2:0/149707280 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c40035f0 con 0x7fc4d4101de0 2024-09-10T08:21:31.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.856+0000 7fc4cb400640 1 -- 172.21.15.2:0/149707280 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4010670 con 0x7fc4d4101de0 2024-09-10T08:21:31.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.857+0000 7fc4d9a00640 1 -- 172.21.15.2:0/149707280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d41a7110 con 0x7fc4d4101de0 2024-09-10T08:21:31.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.858+0000 7fc4d9a00640 1 -- 172.21.15.2:0/149707280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d41021c0 con 0x7fc4d4101de0 2024-09-10T08:21:31.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.859+0000 7fc4cb400640 1 -- 172.21.15.2:0/149707280 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc4c40037a0 con 0x7fc4d4101de0 2024-09-10T08:21:31.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.861+0000 7fc4cb400640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc4b80a2c00 0x7fc4b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:31.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.861+0000 7fc4cb400640 1 -- 172.21.15.2:0/149707280 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fc4c40cf680 con 0x7fc4d4101de0 2024-09-10T08:21:31.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.861+0000 7fc4d2000640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc4b80a2c00 0x7fc4b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:31.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.862+0000 7fc4d2000640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc4b80a2c00 0x7fc4b80a50c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0000c00 tx=0x7fc4c0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:31.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:31.865+0000 7fc4cb400640 1 -- 172.21.15.2:0/149707280 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4c409c190 con 0x7fc4d4101de0 2024-09-10T08:21:32.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:31 smithi002 ceph-mon[30563]: pgmap v788: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-09-10T08:21:32.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.091+0000 7fc4d9a00640 1 -- 172.21.15.2:0/149707280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4d4109b60 con 0x7fc4d4101de0 2024-09-10T08:21:32.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.093+0000 7fc4cb400640 1 -- 172.21.15.2:0/149707280 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fc4c409ba90 con 0x7fc4d4101de0 2024-09-10T08:21:32.093 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:21:32.093 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:21:32.093 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:21:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.095+0000 7fc4d9a00640 1 -- 172.21.15.2:0/149707280 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc4b80a2c00 msgr2=0x7fc4b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.095+0000 7fc4d9a00640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc4b80a2c00 0x7fc4b80a50c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0000c00 tx=0x7fc4c0005b50 comp rx=0 tx=0).stop 2024-09-10T08:21:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.095+0000 7fc4d9a00640 1 -- 172.21.15.2:0/149707280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4d4101de0 msgr2=0x7fc4d40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.095+0000 7fc4d9a00640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4d4101de0 0x7fc4d40744f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400ea60 tx=0x7fc4c400ef20 comp rx=0 tx=0).stop 2024-09-10T08:21:32.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.095+0000 7fc4d9a00640 1 -- 172.21.15.2:0/149707280 shutdown_connections 2024-09-10T08:21:32.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.095+0000 7fc4d9a00640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4d4106f10 0x7fc4d4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:32.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.095+0000 7fc4d9a00640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fc4b80a2c00 0x7fc4b80a50c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:32.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.095+0000 7fc4d9a00640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4d4102790 0x7fc4d4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:32.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.095+0000 7fc4d9a00640 1 --2- 172.21.15.2:0/149707280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4d4101de0 0x7fc4d40744f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:32.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.095+0000 7fc4d9a00640 1 -- 172.21.15.2:0/149707280 >> 172.21.15.2:0/149707280 conn(0x7fc4d40fd8c0 msgr2=0x7fc4d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:32.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.095+0000 7fc4d9a00640 1 -- 172.21.15.2:0/149707280 shutdown_connections 2024-09-10T08:21:32.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:32.095+0000 7fc4d9a00640 1 -- 172.21.15.2:0/149707280 wait complete. 2024-09-10T08:21:32.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:31 smithi155 ceph-mon[35405]: pgmap v788: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-09-10T08:21:32.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:31 smithi107 ceph-mon[35048]: pgmap v788: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-09-10T08:21:33.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:32 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/149707280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:33.145 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:21:33.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:32 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/149707280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:33.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:32 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/149707280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:33.525 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:34.018 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:33 smithi002 ceph-mon[30563]: pgmap v789: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-09-10T08:21:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.132+0000 7fcfdc600640 1 -- 172.21.15.2:0/804925513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd41049a0 msgr2=0x7fcfd41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.132+0000 7fcfdc600640 1 --2- 172.21.15.2:0/804925513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd41049a0 0x7fcfd41089f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009900 tx=0x7fcfc002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.132+0000 7fcfdc600640 1 -- 172.21.15.2:0/804925513 shutdown_connections 2024-09-10T08:21:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.132+0000 7fcfdc600640 1 --2- 172.21.15.2:0/804925513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfd4109120 0x7fcfd4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.132+0000 7fcfdc600640 1 --2- 172.21.15.2:0/804925513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd41049a0 0x7fcfd41089f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.132+0000 7fcfdc600640 1 --2- 172.21.15.2:0/804925513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd4103ff0 0x7fcfd41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.132+0000 7fcfdc600640 1 -- 172.21.15.2:0/804925513 >> 172.21.15.2:0/804925513 conn(0x7fcfd40ffa90 msgr2=0x7fcfd4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.132+0000 7fcfdc600640 1 -- 172.21.15.2:0/804925513 shutdown_connections 2024-09-10T08:21:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.133+0000 7fcfdc600640 1 -- 172.21.15.2:0/804925513 wait complete. 2024-09-10T08:21:34.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.133+0000 7fcfdc600640 1 Processor -- start 2024-09-10T08:21:34.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.134+0000 7fcfdc600640 1 -- start start 2024-09-10T08:21:34.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.134+0000 7fcfdc600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd4103ff0 0x7fcfd419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.134+0000 7fcfdc600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfd41049a0 0x7fcfd419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.134+0000 7fcfdc600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd4109120 0x7fcfd419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.134+0000 7fcfdc600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd4114120 con 0x7fcfd4103ff0 2024-09-10T08:21:34.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.134+0000 7fcfdc600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd4113fa0 con 0x7fcfd41049a0 2024-09-10T08:21:34.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.134+0000 7fcfdc600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41142a0 con 0x7fcfd4109120 2024-09-10T08:21:34.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.134+0000 7fcfd9c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd4103ff0 0x7fcfd419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.134+0000 7fcfd9c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd4103ff0 0x7fcfd419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49852/0 (socket says 172.21.15.2:49852) 2024-09-10T08:21:34.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.134+0000 7fcfd9c00640 1 -- 172.21.15.2:0/31982252 learned_addr learned my addr 172.21.15.2:0/31982252 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:34.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.134+0000 7fcfd9200640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfd41049a0 0x7fcfd419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.134+0000 7fcfda600640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd4109120 0x7fcfd419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.135+0000 7fcfd9c00640 1 -- 172.21.15.2:0/31982252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd4109120 msgr2=0x7fcfd419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:34.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.135+0000 7fcfd9c00640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd4109120 0x7fcfd419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.135+0000 7fcfd9c00640 1 -- 172.21.15.2:0/31982252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfd41049a0 msgr2=0x7fcfd419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:34.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.135+0000 7fcfd9c00640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfd41049a0 0x7fcfd419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.135+0000 7fcfd9c00640 1 -- 172.21.15.2:0/31982252 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd419f650 con 0x7fcfd4103ff0 2024-09-10T08:21:34.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.135+0000 7fcfd9c00640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd4103ff0 0x7fcfd419a6a0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800e9f0 tx=0x7fcfc800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:34.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.135+0000 7fcfd2000640 1 -- 172.21.15.2:0/31982252 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc800cd80 con 0x7fcfd4103ff0 2024-09-10T08:21:34.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.135+0000 7fcfd2000640 1 -- 172.21.15.2:0/31982252 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc8010910 con 0x7fcfd4103ff0 2024-09-10T08:21:34.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.135+0000 7fcfdc600640 1 -- 172.21.15.2:0/31982252 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd419f940 con 0x7fcfd4103ff0 2024-09-10T08:21:34.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.135+0000 7fcfd2000640 1 -- 172.21.15.2:0/31982252 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8018990 con 0x7fcfd4103ff0 2024-09-10T08:21:34.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.135+0000 7fcfdc600640 1 -- 172.21.15.2:0/31982252 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd41a7220 con 0x7fcfd4103ff0 2024-09-10T08:21:34.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.139+0000 7fcfdf208740 1 Processor -- start 2024-09-10T08:21:34.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.139+0000 7fcfdf208740 1 -- start start 2024-09-10T08:21:34.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.139+0000 7fcfdf208740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c93050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.139+0000 7fcfdf208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c93590 0x55d1a5c9d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.139+0000 7fcfdf208740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 0x55d1a5ca0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.139+0000 7fcfdf208740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d1a5bf93e0 con 0x55d1a5c9de50 2024-09-10T08:21:34.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.139+0000 7fcfdf208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d1a5ba0d80 con 0x55d1a5c93590 2024-09-10T08:21:34.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.139+0000 7fcfdf208740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1a5bf53a0 con 0x55d1a5c92c70 2024-09-10T08:21:34.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.139+0000 7fcfd2000640 1 -- 172.21.15.2:0/31982252 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcfc8010ac0 con 0x7fcfd4103ff0 2024-09-10T08:21:34.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.139+0000 7fcfda600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c93050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.139+0000 7fcfd9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 0x55d1a5ca0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.140+0000 7fcfda600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c93050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55536/0 (socket says 172.21.15.2:55536) 2024-09-10T08:21:34.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.140+0000 7fcfda600640 1 -- 172.21.15.2:0/3102374967 learned_addr learned my addr 172.21.15.2:0/3102374967 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:34.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.140+0000 7fcfd9c00640 1 --2- 172.21.15.2:0/3102374967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c93590 0x55d1a5c9d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.140+0000 7fcfda600640 1 -- 172.21.15.2:0/3102374967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c93590 msgr2=0x55d1a5c9d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:34.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.140+0000 7fcfda600640 1 --2- 172.21.15.2:0/3102374967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c93590 0x55d1a5c9d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.140+0000 7fcfda600640 1 -- 172.21.15.2:0/3102374967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 msgr2=0x55d1a5ca0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:34.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.140+0000 7fcfda600640 1 --2- 172.21.15.2:0/3102374967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 0x55d1a5ca0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.140+0000 7fcfda600640 1 -- 172.21.15.2:0/3102374967 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a5bbfae0 con 0x55d1a5c92c70 2024-09-10T08:21:34.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.140+0000 7fcfda600640 1 --2- 172.21.15.2:0/3102374967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c93050 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009f50 tx=0x7fcfcc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c89540405b0b414 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:34.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.141+0000 7fcfc6a00640 1 -- 172.21.15.2:0/3102374967 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc004d20 con 0x55d1a5c92c70 2024-09-10T08:21:34.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.141+0000 7fcfc6a00640 1 -- 172.21.15.2:0/3102374967 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfcc004050 con 0x55d1a5c92c70 2024-09-10T08:21:34.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.141+0000 7fcfc6a00640 1 -- 172.21.15.2:0/3102374967 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc005220 con 0x55d1a5c92c70 2024-09-10T08:21:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.141+0000 7fcfdf208740 1 -- 172.21.15.2:0/3102374967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 msgr2=0x55d1a5c93050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.141+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3102374967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c93050 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009f50 tx=0x7fcfcc0118d0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.141+0000 7fcfd2000640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcfbc0a2c00 0x7fcfbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.141+0000 7fcfdf208740 1 -- 172.21.15.2:0/3102374967 shutdown_connections 2024-09-10T08:21:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.141+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3102374967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 0x55d1a5ca0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.141+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3102374967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c93590 0x55d1a5c9d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.141+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3102374967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c93050 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.141+0000 7fcfdf208740 1 -- 172.21.15.2:0/3102374967 >> 172.21.15.2:0/3102374967 conn(0x55d1a5c8f200 msgr2=0x55d1a5c8f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.142+0000 7fcfd2000640 1 -- 172.21.15.2:0/31982252 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcfc80cf790 con 0x7fcfd4103ff0 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.142+0000 7fcfd9c00640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcfbc0a2c00 0x7fcfbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.142+0000 7fcfdf208740 1 -- 172.21.15.2:0/3102374967 shutdown_connections 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.142+0000 7fcfdf208740 1 -- 172.21.15.2:0/3102374967 wait complete. 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.142+0000 7fcfd9c00640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcfbc0a2c00 0x7fcfbc0a50c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8098620 tx=0x7fcfc8018420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.142+0000 7fcfdf208740 1 -- start start 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.143+0000 7fcfdf208740 1 Processor -- start 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.143+0000 7fcfdf208740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c9aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.143+0000 7fcfdf208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c93590 0x55d1a5c9b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.143+0000 7fcfdf208740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 0x55d1a5c9b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.143+0000 7fcfdf208740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d1a5ca0a20 con 0x55d1a5c9de50 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.143+0000 7fcfdf208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d1a5bb40f0 con 0x55d1a5c93590 2024-09-10T08:21:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.143+0000 7fcfdf208740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1a5ca0ba0 con 0x55d1a5c92c70 2024-09-10T08:21:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.143+0000 7fcfdf208740 10 client.0 _ll_register_callbacks cb 0x55d1a5b9bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.143+0000 7fcfd9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c93590 0x55d1a5c9b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.143+0000 7fcfd9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c93590 0x55d1a5c9b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58684/0 (socket says 172.21.15.2:58684) 2024-09-10T08:21:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.143+0000 7fcfd9200640 1 -- 172.21.15.2:0/3146059254 learned_addr learned my addr 172.21.15.2:0/3146059254 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.144+0000 7fcfda600640 1 --2- 172.21.15.2:0/3146059254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c9aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.144+0000 7fcfda600640 1 --2- 172.21.15.2:0/3146059254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 0x55d1a5c9b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.144+0000 7fcfd9200640 1 -- 172.21.15.2:0/3146059254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 msgr2=0x55d1a5c9aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.144+0000 7fcfd9200640 1 --2- 172.21.15.2:0/3146059254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c9aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.144+0000 7fcfd9200640 1 -- 172.21.15.2:0/3146059254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 msgr2=0x55d1a5c9b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.144+0000 7fcfd9200640 1 --2- 172.21.15.2:0/3146059254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 0x55d1a5c9b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.144+0000 7fcfd9200640 1 -- 172.21.15.2:0/3146059254 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a596a3c0 con 0x55d1a5c93590 2024-09-10T08:21:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.144+0000 7fcfd9200640 1 --2- 172.21.15.2:0/3146059254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c93590 0x55d1a5c9b1e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009900 tx=0x7fcfc00317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23e0d682ecc37152 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:34.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.144+0000 7fcfbb400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:21:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.145+0000 7fcfbb400640 1 -- 172.21.15.2:0/3146059254 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0035040 con 0x55d1a5c93590 2024-09-10T08:21:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.145+0000 7fcfbb400640 1 -- 172.21.15.2:0/3146059254 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcfc0004460 con 0x55d1a5c93590 2024-09-10T08:21:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.145+0000 7fcfdf208740 1 -- 172.21.15.2:0/3146059254 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfcc004050 con 0x55d1a5c93590 2024-09-10T08:21:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.145+0000 7fcfbb400640 1 -- 172.21.15.2:0/3146059254 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcfc00035f0 con 0x55d1a5c93590 2024-09-10T08:21:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.145+0000 7fcfdf208740 10 client.29207 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.145+0000 7fcfdf208740 1 -- 172.21.15.2:0/3146059254 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d1a5cac9c0 con 0x55d1a5c93590 2024-09-10T08:21:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.145+0000 7fcfbb400640 1 -- 172.21.15.2:0/3146059254 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfcc004050 con 0x55d1a5c93590 2024-09-10T08:21:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.145+0000 7fcfbb400640 1 -- 172.21.15.2:0/3146059254 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcfc0046880 con 0x55d1a5c93590 2024-09-10T08:21:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.146+0000 7fcfdf208740 10 client.29207 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.146+0000 7fcfdf208740 10 client.29207 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:21:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.146+0000 7fcfdf208740 20 client.29207 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.146+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3146059254 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d1a5caded0 0x55d1a5cb02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.146+0000 7fcfdf208740 4 client.29207 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.146+0000 7fcfdf208740 1 -- 172.21.15.2:0/3146059254 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1a5c06950 con 0x55d1a5caded0 2024-09-10T08:21:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.146+0000 7fcfda600640 1 --2- 172.21.15.2:0/3146059254 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d1a5caded0 0x55d1a5cb02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.147+0000 7fcfda600640 1 --2- 172.21.15.2:0/3146059254 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d1a5caded0 0x55d1a5cb02f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a2afb288ef6ca2 server_cookie=b269d24db93778bf in_seq=0 out_seq=0 2024-09-10T08:21:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.147+0000 7fcfbb400640 10 client.29207 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:21:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.147+0000 7fcfbb400640 1 -- 172.21.15.2:0/3146059254 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1a5c06950 con 0x55d1a5caded0 2024-09-10T08:21:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.147+0000 7fcfbb400640 10 client.29207 handle_command_reply: tid=0 2024-09-10T08:21:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.147+0000 7fcfdf208740 1 client.29207 shutdown 2024-09-10T08:21:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.148+0000 7fcfdf208740 1 -- 172.21.15.2:0/3146059254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c93590 msgr2=0x55d1a5c9b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.148+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3146059254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c93590 0x55d1a5c9b1e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009900 tx=0x7fcfc00317e0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.148+0000 7fcfdf208740 1 -- 172.21.15.2:0/3146059254 shutdown_connections 2024-09-10T08:21:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.148+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3146059254 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d1a5caded0 0x55d1a5cb02f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.148+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3146059254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 0x55d1a5c9b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.148+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3146059254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c93590 0x55d1a5c9b1e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.148+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3146059254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c9aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.148+0000 7fcfbb400640 0 client.29207 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:21:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.148+0000 7fcfdf208740 1 -- 172.21.15.2:0/3146059254 >> 172.21.15.2:0/3146059254 conn(0x55d1a5c8f200 msgr2=0x55d1a5c8e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.148+0000 7fcfdf208740 1 -- 172.21.15.2:0/3146059254 shutdown_connections 2024-09-10T08:21:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.148+0000 7fcfdf208740 1 -- 172.21.15.2:0/3146059254 wait complete. 2024-09-10T08:21:34.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.148+0000 7fcfdf208740 20 client.29207 trim_cache size 0 max 16384 2024-09-10T08:21:34.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:33 smithi155 ceph-mon[35405]: pgmap v789: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-09-10T08:21:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.162+0000 7fcfdf208740 1 Processor -- start 2024-09-10T08:21:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.162+0000 7fcfdf208740 1 -- start start 2024-09-10T08:21:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.162+0000 7fcfdf208740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c99570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.162+0000 7fcfdf208740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c93590 0x55d1a5c9b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.162+0000 7fcfdf208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c9de50 0x55d1a5cb5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.162+0000 7fcfdf208740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41b39e0 con 0x55d1a5c93590 2024-09-10T08:21:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.162+0000 7fcfdf208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41b12b0 con 0x55d1a5c9de50 2024-09-10T08:21:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.162+0000 7fcfdf208740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41af100 con 0x55d1a5c92c70 2024-09-10T08:21:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.162+0000 7fcfd9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c93590 0x55d1a5c9b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.162+0000 7fcfd9200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c93590 0x55d1a5c9b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49870/0 (socket says 172.21.15.2:49870) 2024-09-10T08:21:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.162+0000 7fcfd9200640 1 -- 172.21.15.2:0/3666361587 learned_addr learned my addr 172.21.15.2:0/3666361587 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:34.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.162+0000 7fcfda600640 1 --2- 172.21.15.2:0/3666361587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c99570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.163+0000 7fcfda600640 1 --2- 172.21.15.2:0/3666361587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c9de50 0x55d1a5cb5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.163+0000 7fcfd9200640 1 -- 172.21.15.2:0/3666361587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 msgr2=0x55d1a5c99570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.163+0000 7fcfd9200640 1 --2- 172.21.15.2:0/3666361587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c99570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.163+0000 7fcfd9200640 1 -- 172.21.15.2:0/3666361587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c9de50 msgr2=0x55d1a5cb5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.163+0000 7fcfd9200640 1 --2- 172.21.15.2:0/3666361587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c9de50 0x55d1a5cb5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.163+0000 7fcfd9200640 1 -- 172.21.15.2:0/3666361587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a5c902b0 con 0x55d1a5c93590 2024-09-10T08:21:34.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.163+0000 7fcfd9200640 1 --2- 172.21.15.2:0/3666361587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c93590 0x55d1a5c9b570 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0006010 tx=0x7fcfc002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f43ac844ad85f95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.163+0000 7fcfc6000640 1 -- 172.21.15.2:0/3666361587 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0035030 con 0x55d1a5c93590 2024-09-10T08:21:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.164+0000 7fcfc6000640 1 -- 172.21.15.2:0/3666361587 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc0041070 con 0x55d1a5c93590 2024-09-10T08:21:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.164+0000 7fcfc6000640 1 -- 172.21.15.2:0/3666361587 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0003790 con 0x55d1a5c93590 2024-09-10T08:21:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.164+0000 7fcfdf208740 1 -- 172.21.15.2:0/3666361587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c93590 msgr2=0x55d1a5c9b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.164+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3666361587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c93590 0x55d1a5c9b570 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0006010 tx=0x7fcfc002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.164+0000 7fcfdf208740 1 -- 172.21.15.2:0/3666361587 shutdown_connections 2024-09-10T08:21:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.164+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3666361587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c9de50 0x55d1a5cb5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.164+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3666361587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c93590 0x55d1a5c9b570 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.164+0000 7fcfdf208740 1 --2- 172.21.15.2:0/3666361587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c92c70 0x55d1a5c99570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.164+0000 7fcfdf208740 1 -- 172.21.15.2:0/3666361587 >> 172.21.15.2:0/3666361587 conn(0x55d1a5c8f200 msgr2=0x55d1a5c9cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:34.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.164+0000 7fcfdf208740 1 -- 172.21.15.2:0/3666361587 shutdown_connections 2024-09-10T08:21:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.164+0000 7fcfdf208740 1 -- 172.21.15.2:0/3666361587 wait complete. 2024-09-10T08:21:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.165+0000 7fcfdf208740 1 -- start start 2024-09-10T08:21:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.165+0000 7fcfdf208740 1 Processor -- start 2024-09-10T08:21:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.165+0000 7fcfdf208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c92c70 0x55d1a5c97be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.165+0000 7fcfdf208740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c93590 0x55d1a5c98120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.165+0000 7fcfdf208740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 0x55d1a5c98660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.165+0000 7fcfdf208740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41af770 con 0x55d1a5c9de50 2024-09-10T08:21:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.165+0000 7fcfdf208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41b0be0 con 0x55d1a5c92c70 2024-09-10T08:21:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.165+0000 7fcfdf208740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41afe40 con 0x55d1a5c93590 2024-09-10T08:21:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.166+0000 7fcfdf208740 10 client.0 _ll_register_callbacks cb 0x7fcfc0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.166+0000 7fcfda600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c92c70 0x55d1a5c97be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.166+0000 7fcfd9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c93590 0x55d1a5c98120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.166+0000 7fcfd9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c93590 0x55d1a5c98120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55572/0 (socket says 172.21.15.2:55572) 2024-09-10T08:21:34.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.166+0000 7fcfd9200640 1 -- 172.21.15.2:0/442826293 learned_addr learned my addr 172.21.15.2:0/442826293 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:34.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.166+0000 7fcfda600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 0x55d1a5c98660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.168 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:33 smithi107 ceph-mon[35048]: pgmap v789: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-09-10T08:21:34.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.166+0000 7fcfd9200640 1 -- 172.21.15.2:0/442826293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c92c70 msgr2=0x55d1a5c97be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:34.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.166+0000 7fcfd9200640 1 --2- 172.21.15.2:0/442826293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c92c70 0x55d1a5c97be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.166+0000 7fcfd9200640 1 -- 172.21.15.2:0/442826293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 msgr2=0x55d1a5c98660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:34.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.166+0000 7fcfd9200640 1 --2- 172.21.15.2:0/442826293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 0x55d1a5c98660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.166+0000 7fcfd9200640 1 -- 172.21.15.2:0/442826293 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a5c8fe90 con 0x55d1a5c93590 2024-09-10T08:21:34.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.167+0000 7fcfd9200640 1 --2- 172.21.15.2:0/442826293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c93590 0x55d1a5c98120 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0031bf0 tx=0x7fcfc0005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2b7689b438f3815 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:34.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.167+0000 7fcfbb400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:21:34.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.167+0000 7fcfbb400640 1 -- 172.21.15.2:0/442826293 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0061020 con 0x55d1a5c93590 2024-09-10T08:21:34.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.167+0000 7fcfbb400640 1 -- 172.21.15.2:0/442826293 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcfc0003790 con 0x55d1a5c93590 2024-09-10T08:21:34.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.167+0000 7fcfdf208740 1 -- 172.21.15.2:0/442826293 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc0041070 con 0x55d1a5c93590 2024-09-10T08:21:34.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.167+0000 7fcfbb400640 1 -- 172.21.15.2:0/442826293 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcfc00039b0 con 0x55d1a5c93590 2024-09-10T08:21:34.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.167+0000 7fcfdf208740 10 client.29146 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:34.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.167+0000 7fcfdf208740 1 -- 172.21.15.2:0/442826293 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d1a5d18ba0 con 0x55d1a5c93590 2024-09-10T08:21:34.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.168+0000 7fcfbb400640 1 -- 172.21.15.2:0/442826293 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc0041070 con 0x55d1a5c93590 2024-09-10T08:21:34.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.168+0000 7fcfbb400640 1 -- 172.21.15.2:0/442826293 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcfc005ca00 con 0x55d1a5c93590 2024-09-10T08:21:34.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.168+0000 7fcfdf208740 10 client.29146 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:34.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.168+0000 7fcfdf208740 10 client.29146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:21:34.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.168+0000 7fcfdf208740 20 client.29146 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:34.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.168+0000 7fcfdf208740 1 --2- 172.21.15.2:0/442826293 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d1a5d190b0 0x55d1a5d1b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:34.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.168+0000 7fcfdf208740 4 client.29146 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:21:34.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.168+0000 7fcfdf208740 1 -- 172.21.15.2:0/442826293 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1a5c00e60 con 0x55d1a5d190b0 2024-09-10T08:21:34.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.168+0000 7fcfda600640 1 --2- 172.21.15.2:0/442826293 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d1a5d190b0 0x55d1a5d1b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:34.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfda600640 1 --2- 172.21.15.2:0/442826293 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d1a5d190b0 0x55d1a5d1b4a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1c63a6b5334a91e server_cookie=67d4efa7fa31f150 in_seq=0 out_seq=0 2024-09-10T08:21:34.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfbb400640 10 client.29146 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:21:34.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfbb400640 1 -- 172.21.15.2:0/442826293 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1a5c00e60 con 0x55d1a5d190b0 2024-09-10T08:21:34.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfbb400640 10 client.29146 handle_command_reply: tid=0 2024-09-10T08:21:34.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfdf208740 1 client.29146 shutdown 2024-09-10T08:21:34.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfdf208740 1 -- 172.21.15.2:0/442826293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c93590 msgr2=0x55d1a5c98120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:34.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfdf208740 1 --2- 172.21.15.2:0/442826293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c93590 0x55d1a5c98120 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0031bf0 tx=0x7fcfc0005a20 comp rx=0 tx=0).stop 2024-09-10T08:21:34.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfdf208740 1 -- 172.21.15.2:0/442826293 shutdown_connections 2024-09-10T08:21:34.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfdf208740 1 --2- 172.21.15.2:0/442826293 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55d1a5d190b0 0x55d1a5d1b4a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfdf208740 1 --2- 172.21.15.2:0/442826293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1a5c9de50 0x55d1a5c98660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfdf208740 1 --2- 172.21.15.2:0/442826293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1a5c93590 0x55d1a5c98120 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfdf208740 1 --2- 172.21.15.2:0/442826293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d1a5c92c70 0x55d1a5c97be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfbb400640 0 client.29146 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:21:34.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.169+0000 7fcfdf208740 1 -- 172.21.15.2:0/442826293 >> 172.21.15.2:0/442826293 conn(0x55d1a5c8f200 msgr2=0x55d1a5c8db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:34.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.170+0000 7fcfdf208740 1 -- 172.21.15.2:0/442826293 shutdown_connections 2024-09-10T08:21:34.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.170+0000 7fcfdf208740 1 -- 172.21.15.2:0/442826293 wait complete. 2024-09-10T08:21:34.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.170+0000 7fcfdf208740 20 client.29146 trim_cache size 0 max 16384 2024-09-10T08:21:34.173 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:21:34.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.170+0000 7fcfdc600640 1 -- 172.21.15.2:0/31982252 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcfbc0a2c00 msgr2=0x7fcfbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:34.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.170+0000 7fcfdc600640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcfbc0a2c00 0x7fcfbc0a50c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8098620 tx=0x7fcfc8018420 comp rx=0 tx=0).stop 2024-09-10T08:21:34.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.170+0000 7fcfdc600640 1 -- 172.21.15.2:0/31982252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd4103ff0 msgr2=0x7fcfd419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:34.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.170+0000 7fcfdc600640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd4103ff0 0x7fcfd419a6a0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800e9f0 tx=0x7fcfc800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.170+0000 7fcfdc600640 1 -- 172.21.15.2:0/31982252 shutdown_connections 2024-09-10T08:21:34.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.170+0000 7fcfdc600640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcfbc0a2c00 0x7fcfbc0a50c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.170+0000 7fcfdc600640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd4109120 0x7fcfd419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.170+0000 7fcfdc600640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfd41049a0 0x7fcfd419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.170+0000 7fcfdc600640 1 --2- 172.21.15.2:0/31982252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd4103ff0 0x7fcfd419a6a0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:34.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.170+0000 7fcfdc600640 1 -- 172.21.15.2:0/31982252 >> 172.21.15.2:0/31982252 conn(0x7fcfd40ffa90 msgr2=0x7fcfd4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:34.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.171+0000 7fcfdc600640 1 -- 172.21.15.2:0/31982252 shutdown_connections 2024-09-10T08:21:34.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:34.171+0000 7fcfdc600640 1 -- 172.21.15.2:0/31982252 wait complete. 2024-09-10T08:21:35.335 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:21:35.715 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:36.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:35 smithi002 ceph-mon[30563]: pgmap v790: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-09-10T08:21:36.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:35 smithi155 ceph-mon[35405]: pgmap v790: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-09-10T08:21:36.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:35 smithi107 ceph-mon[35048]: pgmap v790: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-09-10T08:21:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.495+0000 7f8be4a00640 1 -- 172.21.15.2:0/4205540698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be0112870 msgr2=0x7f8be0114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.495+0000 7f8be4a00640 1 --2- 172.21.15.2:0/4205540698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be0112870 0x7f8be0114c60 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009ca0 tx=0x7f8bd802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.495+0000 7f8be4a00640 1 -- 172.21.15.2:0/4205540698 shutdown_connections 2024-09-10T08:21:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.495+0000 7f8be4a00640 1 --2- 172.21.15.2:0/4205540698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be0112870 0x7f8be0114c60 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.495+0000 7f8be4a00640 1 --2- 172.21.15.2:0/4205540698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be0077040 0x7f8be0112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.495+0000 7f8be4a00640 1 --2- 172.21.15.2:0/4205540698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8be0076720 0x7f8be0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.495+0000 7f8be4a00640 1 -- 172.21.15.2:0/4205540698 >> 172.21.15.2:0/4205540698 conn(0x7f8be00ff8d0 msgr2=0x7f8be0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.495+0000 7f8be4a00640 1 -- 172.21.15.2:0/4205540698 shutdown_connections 2024-09-10T08:21:36.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.495+0000 7f8be4a00640 1 -- 172.21.15.2:0/4205540698 wait complete. 2024-09-10T08:21:36.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.496+0000 7f8be4a00640 1 Processor -- start 2024-09-10T08:21:36.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.497+0000 7f8be4a00640 1 -- start start 2024-09-10T08:21:36.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.497+0000 7f8be4a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be0076720 0x7f8be01a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.497+0000 7f8be4a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be0077040 0x7f8be01a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.497+0000 7f8be4a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8be0112870 0x7f8be019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.497+0000 7f8be4a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0118420 con 0x7f8be0076720 2024-09-10T08:21:36.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.497+0000 7f8be4a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01182a0 con 0x7f8be0112870 2024-09-10T08:21:36.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.497+0000 7f8be4a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01185a0 con 0x7f8be0077040 2024-09-10T08:21:36.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.497+0000 7f8bdf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be0076720 0x7f8be01a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.497+0000 7f8bdf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be0076720 0x7f8be01a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49882/0 (socket says 172.21.15.2:49882) 2024-09-10T08:21:36.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.497+0000 7f8bdf400640 1 -- 172.21.15.2:0/154129963 learned_addr learned my addr 172.21.15.2:0/154129963 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:36.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.497+0000 7f8bdea00640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be0077040 0x7f8be01a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.497+0000 7f8bdfe00640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8be0112870 0x7f8be019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.498+0000 7f8bdf400640 1 -- 172.21.15.2:0/154129963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be0077040 msgr2=0x7f8be01a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:36.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.498+0000 7f8bdf400640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be0077040 0x7f8be01a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.498+0000 7f8bdf400640 1 -- 172.21.15.2:0/154129963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8be0112870 msgr2=0x7f8be019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:36.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.498+0000 7f8bdf400640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8be0112870 0x7f8be019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.498+0000 7f8bdf400640 1 -- 172.21.15.2:0/154129963 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be019ef40 con 0x7f8be0076720 2024-09-10T08:21:36.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.498+0000 7f8bdea00640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be0077040 0x7f8be01a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:36.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.498+0000 7f8bdf400640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be0076720 0x7f8be01a4600 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000e9f0 tx=0x7f8bd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:36.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.498+0000 7f8bd7e00640 1 -- 172.21.15.2:0/154129963 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd000cd80 con 0x7f8be0076720 2024-09-10T08:21:36.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.498+0000 7f8bd7e00640 1 -- 172.21.15.2:0/154129963 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd0010910 con 0x7f8be0076720 2024-09-10T08:21:36.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.498+0000 7f8bd7e00640 1 -- 172.21.15.2:0/154129963 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0018990 con 0x7f8be0076720 2024-09-10T08:21:36.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.498+0000 7f8be4a00640 1 -- 172.21.15.2:0/154129963 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be019f230 con 0x7f8be0076720 2024-09-10T08:21:36.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.499+0000 7f8be4a00640 1 -- 172.21.15.2:0/154129963 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be01ab580 con 0x7f8be0076720 2024-09-10T08:21:36.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.502+0000 7f8bd7e00640 1 -- 172.21.15.2:0/154129963 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8bd0010ac0 con 0x7f8be0076720 2024-09-10T08:21:36.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.502+0000 7f8be8a7d740 1 Processor -- start 2024-09-10T08:21:36.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.502+0000 7f8be8a7d740 1 -- start start 2024-09-10T08:21:36.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.502+0000 7f8be8a7d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b03220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.502+0000 7f8be8a7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b03760 0x555da3b0dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.502+0000 7f8be8a7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b0e020 0x555da3b10410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.502+0000 7f8be8a7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555da3a693e0 con 0x555da3b03760 2024-09-10T08:21:36.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.502+0000 7f8be8a7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555da3a10d80 con 0x555da3b0e020 2024-09-10T08:21:36.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.502+0000 7f8be8a7d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555da3a653a0 con 0x555da3b02e40 2024-09-10T08:21:36.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.503+0000 7f8bdf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b03760 0x555da3b0dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.503+0000 7f8bdea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b0e020 0x555da3b10410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.503+0000 7f8bdfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b03220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.503+0000 7f8bdea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b0e020 0x555da3b10410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58722/0 (socket says 172.21.15.2:58722) 2024-09-10T08:21:36.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.503+0000 7f8bdea00640 1 -- 172.21.15.2:0/3067331638 learned_addr learned my addr 172.21.15.2:0/3067331638 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:36.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.503+0000 7f8bdea00640 1 -- 172.21.15.2:0/3067331638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 msgr2=0x555da3b03220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:36.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.503+0000 7f8bdea00640 1 --2- 172.21.15.2:0/3067331638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b03220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.503+0000 7f8bdea00640 1 -- 172.21.15.2:0/3067331638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b03760 msgr2=0x555da3b0dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:36.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.503+0000 7f8bdea00640 1 --2- 172.21.15.2:0/3067331638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b03760 0x555da3b0dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.503+0000 7f8bdea00640 1 -- 172.21.15.2:0/3067331638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555da3a2fae0 con 0x555da3b0e020 2024-09-10T08:21:36.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.503+0000 7f8bd7e00640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8bc40a2c00 0x7f8bc40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.504+0000 7f8bdea00640 1 --2- 172.21.15.2:0/3067331638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b0e020 0x555da3b10410 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc011470 tx=0x7f8bcc011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e037d253e0c73ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:36.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.504+0000 7f8bd7e00640 1 -- 172.21.15.2:0/154129963 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8bd00cf700 con 0x7f8be0076720 2024-09-10T08:21:36.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.504+0000 7f8bdfe00640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8bc40a2c00 0x7f8bc40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.504+0000 7f8bcbe00640 1 -- 172.21.15.2:0/3067331638 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc01e070 con 0x555da3b0e020 2024-09-10T08:21:36.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.504+0000 7f8bcbe00640 1 -- 172.21.15.2:0/3067331638 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bcc004310 con 0x555da3b0e020 2024-09-10T08:21:36.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.504+0000 7f8bcbe00640 1 -- 172.21.15.2:0/3067331638 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00faa0 con 0x555da3b0e020 2024-09-10T08:21:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.505+0000 7f8be8a7d740 1 -- 172.21.15.2:0/3067331638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b0e020 msgr2=0x555da3b10410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.505+0000 7f8bdfe00640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8bc40a2c00 0x7f8bc40a50c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8005e30 tx=0x7f8bd8005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.505+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/3067331638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b0e020 0x555da3b10410 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc011470 tx=0x7f8bcc011930 comp rx=0 tx=0).stop 2024-09-10T08:21:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.505+0000 7f8be8a7d740 1 -- 172.21.15.2:0/3067331638 shutdown_connections 2024-09-10T08:21:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.505+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/3067331638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b0e020 0x555da3b10410 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.505+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/3067331638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b03760 0x555da3b0dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.505+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/3067331638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b03220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.505+0000 7f8be8a7d740 1 -- 172.21.15.2:0/3067331638 >> 172.21.15.2:0/3067331638 conn(0x555da3aff3d0 msgr2=0x555da3aff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.505+0000 7f8be8a7d740 1 -- 172.21.15.2:0/3067331638 shutdown_connections 2024-09-10T08:21:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.505+0000 7f8be8a7d740 1 -- 172.21.15.2:0/3067331638 wait complete. 2024-09-10T08:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.506+0000 7f8be8a7d740 1 -- start start 2024-09-10T08:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.506+0000 7f8be8a7d740 1 Processor -- start 2024-09-10T08:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.506+0000 7f8be8a7d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b0aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.506+0000 7f8be8a7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 0x555da3b0b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.506+0000 7f8be8a7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b0b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.506+0000 7f8be8a7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555da3b10bf0 con 0x555da3b0e020 2024-09-10T08:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.506+0000 7f8be8a7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555da3a240f0 con 0x555da3b03760 2024-09-10T08:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.506+0000 7f8be8a7d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555da3b10d70 con 0x555da3b02e40 2024-09-10T08:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.506+0000 7f8be8a7d740 10 client.0 _ll_register_callbacks cb 0x555da3a0bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.507+0000 7f8bdf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b0aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.507+0000 7f8bdfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b0b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.507+0000 7f8bdf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b0aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:55618/0 (socket says 172.21.15.2:55618) 2024-09-10T08:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.507+0000 7f8bdf400640 1 -- 172.21.15.2:0/2836335785 learned_addr learned my addr 172.21.15.2:0/2836335785 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:36.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.507+0000 7f8bdea00640 1 --2- 172.21.15.2:0/2836335785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 0x555da3b0b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.507+0000 7f8bdf400640 1 -- 172.21.15.2:0/2836335785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 msgr2=0x555da3b0b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:36.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.507+0000 7f8bdf400640 1 --2- 172.21.15.2:0/2836335785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 0x555da3b0b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.507+0000 7f8bdf400640 1 -- 172.21.15.2:0/2836335785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 msgr2=0x555da3b0b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:36.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.507+0000 7f8bdf400640 1 --2- 172.21.15.2:0/2836335785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b0b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.507+0000 7f8bdf400640 1 -- 172.21.15.2:0/2836335785 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555da37da3c0 con 0x555da3b02e40 2024-09-10T08:21:36.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.507+0000 7f8bdf400640 1 --2- 172.21.15.2:0/2836335785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b0aec0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8bd009b8c0 tx=0x7f8bd009b700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86a82139759c5cca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:36.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.507+0000 7f8bc8c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:21:36.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.508+0000 7f8bc8c00640 1 -- 172.21.15.2:0/2836335785 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd009bbc0 con 0x555da3b02e40 2024-09-10T08:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.508+0000 7f8be8a7d740 1 -- 172.21.15.2:0/2836335785 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bcc004310 con 0x555da3b02e40 2024-09-10T08:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.508+0000 7f8bc8c00640 1 -- 172.21.15.2:0/2836335785 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8bd0018b30 con 0x555da3b02e40 2024-09-10T08:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.508+0000 7f8bc8c00640 1 -- 172.21.15.2:0/2836335785 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bd009fec0 con 0x555da3b02e40 2024-09-10T08:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.508+0000 7f8be8a7d740 10 client.29158 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.508+0000 7f8be8a7d740 1 -- 172.21.15.2:0/2836335785 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555da3b1cc10 con 0x555da3b02e40 2024-09-10T08:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.508+0000 7f8bc8c00640 1 -- 172.21.15.2:0/2836335785 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bcc004310 con 0x555da3b02e40 2024-09-10T08:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.508+0000 7f8bc8c00640 1 -- 172.21.15.2:0/2836335785 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8bd00cfaf0 con 0x555da3b02e40 2024-09-10T08:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.508+0000 7f8be8a7d740 10 client.29158 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.509+0000 7f8be8a7d740 10 client.29158 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.509+0000 7f8be8a7d740 20 client.29158 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.509+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/2836335785 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555da3b1e0d0 0x555da3b204c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.509+0000 7f8be8a7d740 4 client.29158 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.509+0000 7f8be8a7d740 1 -- 172.21.15.2:0/2836335785 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555da3a76950 con 0x555da3b1e0d0 2024-09-10T08:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.509+0000 7f8bdea00640 1 --2- 172.21.15.2:0/2836335785 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555da3b1e0d0 0x555da3b204c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.509+0000 7f8bdea00640 1 --2- 172.21.15.2:0/2836335785 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555da3b1e0d0 0x555da3b204c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=959b8bfd579b94c8 server_cookie=4ecea9b98898538e in_seq=0 out_seq=0 2024-09-10T08:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.509+0000 7f8bc8c00640 10 client.29158 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.510+0000 7f8bc8c00640 1 -- 172.21.15.2:0/2836335785 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555da3a76950 con 0x555da3b1e0d0 2024-09-10T08:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.510+0000 7f8bc8c00640 10 client.29158 handle_command_reply: tid=0 2024-09-10T08:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.510+0000 7f8be8a7d740 1 client.29158 shutdown 2024-09-10T08:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.510+0000 7f8be8a7d740 1 -- 172.21.15.2:0/2836335785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 msgr2=0x555da3b0aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.510+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/2836335785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b0aec0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8bd009b8c0 tx=0x7f8bd009b700 comp rx=0 tx=0).stop 2024-09-10T08:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.511+0000 7f8be8a7d740 1 -- 172.21.15.2:0/2836335785 shutdown_connections 2024-09-10T08:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.511+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/2836335785 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555da3b1e0d0 0x555da3b204c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.511+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/2836335785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b0b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.511+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/2836335785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 0x555da3b0b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.511+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/2836335785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b0aec0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.511+0000 7f8bc8c00640 0 client.29158 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:21:36.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.511+0000 7f8be8a7d740 1 -- 172.21.15.2:0/2836335785 >> 172.21.15.2:0/2836335785 conn(0x555da3aff3d0 msgr2=0x555da3b05100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:36.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.511+0000 7f8be8a7d740 1 -- 172.21.15.2:0/2836335785 shutdown_connections 2024-09-10T08:21:36.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.511+0000 7f8be8a7d740 1 -- 172.21.15.2:0/2836335785 wait complete. 2024-09-10T08:21:36.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.511+0000 7f8be8a7d740 20 client.29158 trim_cache size 0 max 16384 2024-09-10T08:21:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.524+0000 7f8be8a7d740 1 Processor -- start 2024-09-10T08:21:36.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.524+0000 7f8be8a7d740 1 -- start start 2024-09-10T08:21:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.525+0000 7f8be8a7d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b04fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.525+0000 7f8be8a7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 0x555da3b25c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.525+0000 7f8be8a7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b261b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.525+0000 7f8be8a7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01b7c00 con 0x555da3b0e020 2024-09-10T08:21:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.525+0000 7f8be8a7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01b54d0 con 0x555da3b03760 2024-09-10T08:21:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.525+0000 7f8be8a7d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01b3320 con 0x555da3b02e40 2024-09-10T08:21:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.525+0000 7f8bdfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.525+0000 7f8bdf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b04fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.525+0000 7f8bdfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b261b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49898/0 (socket says 172.21.15.2:49898) 2024-09-10T08:21:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.525+0000 7f8bdfe00640 1 -- 172.21.15.2:0/2409850896 learned_addr learned my addr 172.21.15.2:0/2409850896 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.525+0000 7f8bdea00640 1 --2- 172.21.15.2:0/2409850896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 0x555da3b25c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.526+0000 7f8bdf400640 1 -- 172.21.15.2:0/2409850896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 msgr2=0x555da3b25c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.526+0000 7f8bdf400640 1 --2- 172.21.15.2:0/2409850896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 0x555da3b25c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.526+0000 7f8bdf400640 1 -- 172.21.15.2:0/2409850896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 msgr2=0x555da3b261b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.526+0000 7f8bdf400640 1 --2- 172.21.15.2:0/2409850896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b261b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.526+0000 7f8bdf400640 1 -- 172.21.15.2:0/2409850896 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555da3aff200 con 0x555da3b02e40 2024-09-10T08:21:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.526+0000 7f8bdea00640 1 --2- 172.21.15.2:0/2409850896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 0x555da3b25c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.526+0000 7f8bdfe00640 1 --2- 172.21.15.2:0/2409850896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b261b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.526+0000 7f8bdf400640 1 --2- 172.21.15.2:0/2409850896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b04fa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000e9c0 tx=0x7f8bd0097e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f091e6f0b35d9beb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:36.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.526+0000 7f8bcb400640 1 -- 172.21.15.2:0/2409850896 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00a4780 con 0x555da3b02e40 2024-09-10T08:21:36.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.526+0000 7f8bcb400640 1 -- 172.21.15.2:0/2409850896 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd00d5040 con 0x555da3b02e40 2024-09-10T08:21:36.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.526+0000 7f8bcb400640 1 -- 172.21.15.2:0/2409850896 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0018db0 con 0x555da3b02e40 2024-09-10T08:21:36.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.527+0000 7f8be8a7d740 1 -- 172.21.15.2:0/2409850896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 msgr2=0x555da3b04fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:36.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.527+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/2409850896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b04fa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000e9c0 tx=0x7f8bd0097e60 comp rx=0 tx=0).stop 2024-09-10T08:21:36.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.527+0000 7f8be8a7d740 1 -- 172.21.15.2:0/2409850896 shutdown_connections 2024-09-10T08:21:36.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.527+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/2409850896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b261b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.527+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/2409850896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 0x555da3b25c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.527+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/2409850896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b04fa0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.527+0000 7f8be8a7d740 1 -- 172.21.15.2:0/2409850896 >> 172.21.15.2:0/2409850896 conn(0x555da3aff3d0 msgr2=0x555da3b04890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.527+0000 7f8be8a7d740 1 -- 172.21.15.2:0/2409850896 shutdown_connections 2024-09-10T08:21:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.527+0000 7f8be8a7d740 1 -- 172.21.15.2:0/2409850896 wait complete. 2024-09-10T08:21:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.527+0000 7f8be8a7d740 1 -- start start 2024-09-10T08:21:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.528+0000 7f8be8a7d740 1 Processor -- start 2024-09-10T08:21:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.528+0000 7f8be8a7d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b16880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.528+0000 7f8be8a7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 0x555da3b11bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.528+0000 7f8be8a7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.528+0000 7f8be8a7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01b3990 con 0x555da3b0e020 2024-09-10T08:21:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.528+0000 7f8be8a7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01b4e00 con 0x555da3b03760 2024-09-10T08:21:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.528+0000 7f8be8a7d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01b4060 con 0x555da3b02e40 2024-09-10T08:21:36.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.528+0000 7f8be8a7d740 10 client.0 _ll_register_callbacks cb 0x7f8bd00d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:36.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.528+0000 7f8bdfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.528+0000 7f8bdf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b16880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.528+0000 7f8bdfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b121c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49902/0 (socket says 172.21.15.2:49902) 2024-09-10T08:21:36.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.529+0000 7f8bdfe00640 1 -- 172.21.15.2:0/3107953326 learned_addr learned my addr 172.21.15.2:0/3107953326 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:36.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.529+0000 7f8bdea00640 1 --2- 172.21.15.2:0/3107953326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 0x555da3b11bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.529+0000 7f8bdf400640 1 -- 172.21.15.2:0/3107953326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 msgr2=0x555da3b11bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:36.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.529+0000 7f8bdf400640 1 --2- 172.21.15.2:0/3107953326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 0x555da3b11bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.529+0000 7f8bdf400640 1 -- 172.21.15.2:0/3107953326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 msgr2=0x555da3b121c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:36.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.529+0000 7f8bdf400640 1 --2- 172.21.15.2:0/3107953326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b121c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.529+0000 7f8bdf400640 1 -- 172.21.15.2:0/3107953326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555da3b129f0 con 0x555da3b02e40 2024-09-10T08:21:36.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.529+0000 7f8bdf400640 1 --2- 172.21.15.2:0/3107953326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b16880 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0099e70 tx=0x7f8bd0009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=371ab8bc1a42562a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:36.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.529+0000 7f8bc8c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:21:36.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.529+0000 7f8bc8c00640 1 -- 172.21.15.2:0/3107953326 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00a4780 con 0x555da3b02e40 2024-09-10T08:21:36.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.529+0000 7f8bc8c00640 1 -- 172.21.15.2:0/3107953326 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8bd00ba2f0 con 0x555da3b02e40 2024-09-10T08:21:36.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.529+0000 7f8be8a7d740 1 -- 172.21.15.2:0/3107953326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd00d5040 con 0x555da3b02e40 2024-09-10T08:21:36.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.530+0000 7f8bc8c00640 1 -- 172.21.15.2:0/3107953326 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bd00ad050 con 0x555da3b02e40 2024-09-10T08:21:36.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.530+0000 7f8be8a7d740 10 client.29164 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:36.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.530+0000 7f8be8a7d740 1 -- 172.21.15.2:0/3107953326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555da3b88d70 con 0x555da3b02e40 2024-09-10T08:21:36.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.530+0000 7f8bc8c00640 1 -- 172.21.15.2:0/3107953326 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd00d5040 con 0x555da3b02e40 2024-09-10T08:21:36.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.530+0000 7f8bc8c00640 1 -- 172.21.15.2:0/3107953326 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8bd009aa80 con 0x555da3b02e40 2024-09-10T08:21:36.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.530+0000 7f8be8a7d740 10 client.29164 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:36.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.530+0000 7f8be8a7d740 10 client.29164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:21:36.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.530+0000 7f8be8a7d740 20 client.29164 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:36.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.530+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/3107953326 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555da3b89240 0x555da3b8b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:36.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.530+0000 7f8be8a7d740 4 client.29164 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:21:36.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.530+0000 7f8be8a7d740 1 -- 172.21.15.2:0/3107953326 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555da3a70e60 con 0x555da3b89240 2024-09-10T08:21:36.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.530+0000 7f8bdea00640 1 --2- 172.21.15.2:0/3107953326 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555da3b89240 0x555da3b8b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:36.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.531+0000 7f8bdea00640 1 --2- 172.21.15.2:0/3107953326 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555da3b89240 0x555da3b8b630 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96fd91d8e6f09a26 server_cookie=c7cc6f185df4c25e in_seq=0 out_seq=0 2024-09-10T08:21:36.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.531+0000 7f8bc8c00640 10 client.29164 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:21:36.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8bc8c00640 1 -- 172.21.15.2:0/3107953326 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555da3a70e60 con 0x555da3b89240 2024-09-10T08:21:36.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8bc8c00640 10 client.29164 handle_command_reply: tid=0 2024-09-10T08:21:36.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8be8a7d740 1 client.29164 shutdown 2024-09-10T08:21:36.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8be8a7d740 1 -- 172.21.15.2:0/3107953326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 msgr2=0x555da3b16880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:36.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/3107953326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b16880 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0099e70 tx=0x7f8bd0009770 comp rx=0 tx=0).stop 2024-09-10T08:21:36.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8be8a7d740 1 -- 172.21.15.2:0/3107953326 shutdown_connections 2024-09-10T08:21:36.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/3107953326 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x555da3b89240 0x555da3b8b630 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/3107953326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555da3b0e020 0x555da3b121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/3107953326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555da3b03760 0x555da3b11bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8be8a7d740 1 --2- 172.21.15.2:0/3107953326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555da3b02e40 0x555da3b16880 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8be8a7d740 1 -- 172.21.15.2:0/3107953326 >> 172.21.15.2:0/3107953326 conn(0x555da3aff3d0 msgr2=0x555da3afe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:36.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8bc8c00640 0 client.29164 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:21:36.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8be8a7d740 1 -- 172.21.15.2:0/3107953326 shutdown_connections 2024-09-10T08:21:36.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8be8a7d740 1 -- 172.21.15.2:0/3107953326 wait complete. 2024-09-10T08:21:36.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.532+0000 7f8be8a7d740 20 client.29164 trim_cache size 0 max 16384 2024-09-10T08:21:36.536 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:21:36.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.533+0000 7f8be4a00640 1 -- 172.21.15.2:0/154129963 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8bc40a2c00 msgr2=0x7f8bc40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:36.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.533+0000 7f8be4a00640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8bc40a2c00 0x7f8bc40a50c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8005e30 tx=0x7f8bd8005d80 comp rx=0 tx=0).stop 2024-09-10T08:21:36.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.533+0000 7f8be4a00640 1 -- 172.21.15.2:0/154129963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be0076720 msgr2=0x7f8be01a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:36.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.533+0000 7f8be4a00640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be0076720 0x7f8be01a4600 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000e9f0 tx=0x7f8bd000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.533+0000 7f8be4a00640 1 -- 172.21.15.2:0/154129963 shutdown_connections 2024-09-10T08:21:36.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.533+0000 7f8be4a00640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8be0112870 0x7f8be019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.533+0000 7f8be4a00640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f8bc40a2c00 0x7f8bc40a50c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.533+0000 7f8be4a00640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be0077040 0x7f8be01a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.533+0000 7f8be4a00640 1 --2- 172.21.15.2:0/154129963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be0076720 0x7f8be01a4600 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.533+0000 7f8be4a00640 1 -- 172.21.15.2:0/154129963 >> 172.21.15.2:0/154129963 conn(0x7f8be00ff8d0 msgr2=0x7f8be0101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.533+0000 7f8be4a00640 1 -- 172.21.15.2:0/154129963 shutdown_connections 2024-09-10T08:21:36.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:36.533+0000 7f8be4a00640 1 -- 172.21.15.2:0/154129963 wait complete. 2024-09-10T08:21:37.449 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:21:37.829 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:38.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:37 smithi002 ceph-mon[30563]: pgmap v791: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-10T08:21:38.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:37 smithi155 ceph-mon[35405]: pgmap v791: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-10T08:21:38.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:37 smithi107 ceph-mon[35048]: pgmap v791: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-10T08:21:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.381+0000 7fd111c00640 1 -- 172.21.15.2:0/1333624862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd10c1019c0 msgr2=0x7fd10c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.381+0000 7fd111c00640 1 --2- 172.21.15.2:0/1333624862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd10c1019c0 0x7fd10c101e20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009930 tx=0x7fd0f402a440 comp rx=0 tx=0).stop 2024-09-10T08:21:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.381+0000 7fd111c00640 1 -- 172.21.15.2:0/1333624862 shutdown_connections 2024-09-10T08:21:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.381+0000 7fd111c00640 1 --2- 172.21.15.2:0/1333624862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd10c102360 0x7fd10c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.381+0000 7fd111c00640 1 --2- 172.21.15.2:0/1333624862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd10c1019c0 0x7fd10c101e20 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.381+0000 7fd111c00640 1 --2- 172.21.15.2:0/1333624862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10c1079c0 0x7fd10c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.381+0000 7fd111c00640 1 -- 172.21.15.2:0/1333624862 >> 172.21.15.2:0/1333624862 conn(0x7fd10c0fd740 msgr2=0x7fd10c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.381+0000 7fd111c00640 1 -- 172.21.15.2:0/1333624862 shutdown_connections 2024-09-10T08:21:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.381+0000 7fd111c00640 1 -- 172.21.15.2:0/1333624862 wait complete. 2024-09-10T08:21:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.383+0000 7fd111c00640 1 Processor -- start 2024-09-10T08:21:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.383+0000 7fd111c00640 1 -- start start 2024-09-10T08:21:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.383+0000 7fd111c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10c1019c0 0x7fd10c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.383+0000 7fd111c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd10c102360 0x7fd10c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.383+0000 7fd111c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd10c1079c0 0x7fd10c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.383+0000 7fd111c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c111d90 con 0x7fd10c102360 2024-09-10T08:21:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.383+0000 7fd111c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c111c10 con 0x7fd10c1019c0 2024-09-10T08:21:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.383+0000 7fd111c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c111f10 con 0x7fd10c1079c0 2024-09-10T08:21:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.383+0000 7fd103400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd10c102360 0x7fd10c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.384+0000 7fd10be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10c1019c0 0x7fd10c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.384+0000 7fd103400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd10c102360 0x7fd10c198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50654/0 (socket says 172.21.15.2:50654) 2024-09-10T08:21:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.384+0000 7fd103400640 1 -- 172.21.15.2:0/59283608 learned_addr learned my addr 172.21.15.2:0/59283608 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:38.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.384+0000 7fd111200640 1 --2- 172.21.15.2:0/59283608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd10c1079c0 0x7fd10c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.384+0000 7fd10be00640 1 -- 172.21.15.2:0/59283608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd10c1079c0 msgr2=0x7fd10c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.384+0000 7fd10be00640 1 --2- 172.21.15.2:0/59283608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd10c1079c0 0x7fd10c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.384+0000 7fd10be00640 1 -- 172.21.15.2:0/59283608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd10c102360 msgr2=0x7fd10c198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.384+0000 7fd10be00640 1 --2- 172.21.15.2:0/59283608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd10c102360 0x7fd10c198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.384+0000 7fd10be00640 1 -- 172.21.15.2:0/59283608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10c19d2a0 con 0x7fd10c1019c0 2024-09-10T08:21:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.384+0000 7fd10be00640 1 --2- 172.21.15.2:0/59283608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10c1019c0 0x7fd10c1982f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009bc0 tx=0x7fd0fc00b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.385+0000 7fd109600640 1 -- 172.21.15.2:0/59283608 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc01e070 con 0x7fd10c1019c0 2024-09-10T08:21:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.385+0000 7fd111c00640 1 -- 172.21.15.2:0/59283608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10c19d590 con 0x7fd10c1019c0 2024-09-10T08:21:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.385+0000 7fd109600640 1 -- 172.21.15.2:0/59283608 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0fc0198f0 con 0x7fd10c1019c0 2024-09-10T08:21:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.385+0000 7fd109600640 1 -- 172.21.15.2:0/59283608 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0187f0 con 0x7fd10c1019c0 2024-09-10T08:21:38.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.385+0000 7fd111c00640 1 -- 172.21.15.2:0/59283608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd10c074420 con 0x7fd10c1019c0 2024-09-10T08:21:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.388+0000 7fd115c69740 1 Processor -- start 2024-09-10T08:21:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.388+0000 7fd115c69740 1 -- start start 2024-09-10T08:21:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.388+0000 7fd109600640 1 -- 172.21.15.2:0/59283608 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd0fc021070 con 0x7fd10c1019c0 2024-09-10T08:21:38.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.388+0000 7fd115c69740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201c3e40 0x5610201c4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.388+0000 7fd115c69740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c4760 0x5610201ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.388+0000 7fd115c69740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201cf020 0x5610201d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.388+0000 7fd115c69740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56102012a3e0 con 0x5610201c4760 2024-09-10T08:21:38.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.388+0000 7fd115c69740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5610200d1d80 con 0x5610201c3e40 2024-09-10T08:21:38.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.388+0000 7fd115c69740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5610201263a0 con 0x5610201cf020 2024-09-10T08:21:38.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.389+0000 7fd111200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201c3e40 0x5610201c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.389+0000 7fd10be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c4760 0x5610201ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.389+0000 7fd111200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201c3e40 0x5610201c4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52100/0 (socket says 172.21.15.2:52100) 2024-09-10T08:21:38.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.389+0000 7fd111200640 1 -- 172.21.15.2:0/4072042595 learned_addr learned my addr 172.21.15.2:0/4072042595 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.389+0000 7fd103400640 1 --2- 172.21.15.2:0/4072042595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201cf020 0x5610201d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.389+0000 7fd111200640 1 -- 172.21.15.2:0/4072042595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201cf020 msgr2=0x5610201d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.389+0000 7fd111200640 1 --2- 172.21.15.2:0/4072042595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201cf020 0x5610201d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.389+0000 7fd111200640 1 -- 172.21.15.2:0/4072042595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c4760 msgr2=0x5610201ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.389+0000 7fd111200640 1 --2- 172.21.15.2:0/4072042595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c4760 0x5610201ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.389+0000 7fd111200640 1 -- 172.21.15.2:0/4072042595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610200f0ae0 con 0x5610201c3e40 2024-09-10T08:21:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.389+0000 7fd103400640 1 --2- 172.21.15.2:0/4072042595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201cf020 0x5610201d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:21:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.390+0000 7fd111200640 1 --2- 172.21.15.2:0/4072042595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201c3e40 0x5610201c4220 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd104009fd0 tx=0x7fd1040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=797e9392641c64b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.390+0000 7fd109600640 1 --2- 172.21.15.2:0/59283608 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0f00a2c00 0x7fd0f00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.390+0000 7fd101600640 1 -- 172.21.15.2:0/4072042595 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104004d20 con 0x5610201c3e40 2024-09-10T08:21:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.390+0000 7fd101600640 1 -- 172.21.15.2:0/4072042595 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd104004050 con 0x5610201c3e40 2024-09-10T08:21:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.390+0000 7fd101600640 1 -- 172.21.15.2:0/4072042595 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104005240 con 0x5610201c3e40 2024-09-10T08:21:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.390+0000 7fd109600640 1 -- 172.21.15.2:0/59283608 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd0fc0d2b60 con 0x7fd10c1019c0 2024-09-10T08:21:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.390+0000 7fd10be00640 1 --2- 172.21.15.2:0/59283608 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0f00a2c00 0x7fd0f00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.391+0000 7fd115c69740 1 -- 172.21.15.2:0/4072042595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201c3e40 msgr2=0x5610201c4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.391+0000 7fd115c69740 1 --2- 172.21.15.2:0/4072042595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201c3e40 0x5610201c4220 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd104009fd0 tx=0x7fd1040118d0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.391+0000 7fd10be00640 1 --2- 172.21.15.2:0/59283608 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0f00a2c00 0x7fd0f00a50c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc09b9f0 tx=0x7fd0fc0d64f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.391+0000 7fd115c69740 1 -- 172.21.15.2:0/4072042595 shutdown_connections 2024-09-10T08:21:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.391+0000 7fd115c69740 1 --2- 172.21.15.2:0/4072042595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201cf020 0x5610201d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.391+0000 7fd115c69740 1 --2- 172.21.15.2:0/4072042595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c4760 0x5610201ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.391+0000 7fd115c69740 1 --2- 172.21.15.2:0/4072042595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201c3e40 0x5610201c4220 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.391+0000 7fd115c69740 1 -- 172.21.15.2:0/4072042595 >> 172.21.15.2:0/4072042595 conn(0x5610201c03d0 msgr2=0x5610201c07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.391+0000 7fd115c69740 1 -- 172.21.15.2:0/4072042595 shutdown_connections 2024-09-10T08:21:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.391+0000 7fd115c69740 1 -- 172.21.15.2:0/4072042595 wait complete. 2024-09-10T08:21:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.391+0000 7fd115c69740 1 -- start start 2024-09-10T08:21:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.392+0000 7fd115c69740 1 Processor -- start 2024-09-10T08:21:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.392+0000 7fd115c69740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c3e40 0x5610201cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.392+0000 7fd115c69740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c4760 0x5610201cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.392+0000 7fd115c69740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 0x5610201cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.392+0000 7fd115c69740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5610201d1bf0 con 0x5610201c4760 2024-09-10T08:21:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.392+0000 7fd115c69740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5610200e50f0 con 0x5610201cf020 2024-09-10T08:21:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.392+0000 7fd115c69740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5610201d1d70 con 0x5610201c3e40 2024-09-10T08:21:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.392+0000 7fd115c69740 10 client.0 _ll_register_callbacks cb 0x5610200ccbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.392+0000 7fd103400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c4760 0x5610201cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.392+0000 7fd103400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c4760 0x5610201cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50680/0 (socket says 172.21.15.2:50680) 2024-09-10T08:21:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.392+0000 7fd103400640 1 -- 172.21.15.2:0/3597498227 learned_addr learned my addr 172.21.15.2:0/3597498227 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.393+0000 7fd111200640 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c3e40 0x5610201cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.393+0000 7fd111200640 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 0x5610201cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.393+0000 7fd103400640 1 -- 172.21.15.2:0/3597498227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c3e40 msgr2=0x5610201cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.393+0000 7fd103400640 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c3e40 0x5610201cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.393+0000 7fd103400640 1 -- 172.21.15.2:0/3597498227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 msgr2=0x5610201cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.393+0000 7fd103400640 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 0x5610201cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.393+0000 7fd103400640 1 -- 172.21.15.2:0/3597498227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56101fe9b3c0 con 0x5610201c4760 2024-09-10T08:21:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.393+0000 7fd111200640 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 0x5610201cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.393+0000 7fd103400640 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c4760 0x5610201cc400 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4006010 tx=0x7fd0f40317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3795859214930e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.393+0000 7fd0fa000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.394+0000 7fd0fa000640 1 -- 172.21.15.2:0/3597498227 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4035040 con 0x5610201c4760 2024-09-10T08:21:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.394+0000 7fd0fa000640 1 -- 172.21.15.2:0/3597498227 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd0f4004250 con 0x5610201c4760 2024-09-10T08:21:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.394+0000 7fd115c69740 1 -- 172.21.15.2:0/3597498227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd104004050 con 0x5610201c4760 2024-09-10T08:21:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.394+0000 7fd0fa000640 1 -- 172.21.15.2:0/3597498227 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd0f404d050 con 0x5610201c4760 2024-09-10T08:21:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.394+0000 7fd115c69740 10 client.19221 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.394+0000 7fd115c69740 1 -- 172.21.15.2:0/3597498227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5610201ddbb0 con 0x5610201c4760 2024-09-10T08:21:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.394+0000 7fd0fa000640 1 -- 172.21.15.2:0/3597498227 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd104004050 con 0x5610201c4760 2024-09-10T08:21:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.394+0000 7fd0fa000640 1 -- 172.21.15.2:0/3597498227 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd0f403c670 con 0x5610201c4760 2024-09-10T08:21:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.394+0000 7fd115c69740 10 client.19221 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.394+0000 7fd115c69740 10 client.19221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.395+0000 7fd115c69740 20 client.19221 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.395+0000 7fd115c69740 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5610201df0c0 0x5610201e14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.395+0000 7fd115c69740 4 client.19221 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.395+0000 7fd115c69740 1 -- 172.21.15.2:0/3597498227 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561020137950 con 0x5610201df0c0 2024-09-10T08:21:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.395+0000 7fd111200640 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5610201df0c0 0x5610201e14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.395+0000 7fd111200640 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5610201df0c0 0x5610201e14e0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802fd94b8a08c15b server_cookie=cf4fab47b1b07e4e in_seq=0 out_seq=0 2024-09-10T08:21:38.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.395+0000 7fd0fa000640 10 client.19221 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:38.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.396+0000 7fd0fa000640 1 -- 172.21.15.2:0/3597498227 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561020137950 con 0x5610201df0c0 2024-09-10T08:21:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.396+0000 7fd0fa000640 10 client.19221 handle_command_reply: tid=0 2024-09-10T08:21:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.397+0000 7fd115c69740 1 client.19221 shutdown 2024-09-10T08:21:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.397+0000 7fd115c69740 1 -- 172.21.15.2:0/3597498227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c4760 msgr2=0x5610201cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.397+0000 7fd115c69740 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c4760 0x5610201cc400 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4006010 tx=0x7fd0f40317e0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.397+0000 7fd115c69740 1 -- 172.21.15.2:0/3597498227 shutdown_connections 2024-09-10T08:21:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.397+0000 7fd115c69740 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5610201df0c0 0x5610201e14e0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.397+0000 7fd115c69740 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 0x5610201cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.397+0000 7fd115c69740 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c4760 0x5610201cc400 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.397+0000 7fd115c69740 1 --2- 172.21.15.2:0/3597498227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c3e40 0x5610201cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.397+0000 7fd115c69740 1 -- 172.21.15.2:0/3597498227 >> 172.21.15.2:0/3597498227 conn(0x5610201c03d0 msgr2=0x5610201c6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.397+0000 7fd0fa000640 0 client.19221 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:38.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.397+0000 7fd115c69740 1 -- 172.21.15.2:0/3597498227 shutdown_connections 2024-09-10T08:21:38.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.397+0000 7fd115c69740 1 -- 172.21.15.2:0/3597498227 wait complete. 2024-09-10T08:21:38.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.397+0000 7fd115c69740 20 client.19221 trim_cache size 0 max 16384 2024-09-10T08:21:38.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.411+0000 7fd115c69740 1 Processor -- start 2024-09-10T08:21:38.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.411+0000 7fd115c69740 1 -- start start 2024-09-10T08:21:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd115c69740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 0x5610201cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd115c69740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c4760 0x5610201e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd115c69740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 0x5610201e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd115c69740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c1af4b0 con 0x5610201c3e40 2024-09-10T08:21:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd115c69740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c1acd80 con 0x5610201cf020 2024-09-10T08:21:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd115c69740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c1aabd0 con 0x5610201c4760 2024-09-10T08:21:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd111200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 0x5610201cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd111200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 0x5610201cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50688/0 (socket says 172.21.15.2:50688) 2024-09-10T08:21:38.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd111200640 1 -- 172.21.15.2:0/4050099393 learned_addr learned my addr 172.21.15.2:0/4050099393 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd103400640 1 --2- 172.21.15.2:0/4050099393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c4760 0x5610201e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd111200640 1 --2- 172.21.15.2:0/4050099393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 0x5610201e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd111200640 1 -- 172.21.15.2:0/4050099393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c4760 msgr2=0x5610201e6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd111200640 1 --2- 172.21.15.2:0/4050099393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c4760 0x5610201e6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd111200640 1 -- 172.21.15.2:0/4050099393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 msgr2=0x5610201e72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd111200640 1 --2- 172.21.15.2:0/4050099393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 0x5610201e72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd111200640 1 -- 172.21.15.2:0/4050099393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610201c4590 con 0x5610201c3e40 2024-09-10T08:21:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.412+0000 7fd103400640 1 --2- 172.21.15.2:0/4050099393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c4760 0x5610201e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.413+0000 7fd111200640 1 --2- 172.21.15.2:0/4050099393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 0x5610201cc790 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd104004930 tx=0x7fd1040118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5a6093b18e752d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:38.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.413+0000 7fd100c00640 1 -- 172.21.15.2:0/4050099393 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104004390 con 0x5610201c3e40 2024-09-10T08:21:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.413+0000 7fd100c00640 1 -- 172.21.15.2:0/4050099393 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd104004530 con 0x5610201c3e40 2024-09-10T08:21:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.413+0000 7fd100c00640 1 -- 172.21.15.2:0/4050099393 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1040057a0 con 0x5610201c3e40 2024-09-10T08:21:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.414+0000 7fd115c69740 1 -- 172.21.15.2:0/4050099393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 msgr2=0x5610201cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.414+0000 7fd115c69740 1 --2- 172.21.15.2:0/4050099393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 0x5610201cc790 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd104004930 tx=0x7fd1040118d0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.414+0000 7fd115c69740 1 -- 172.21.15.2:0/4050099393 shutdown_connections 2024-09-10T08:21:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.414+0000 7fd115c69740 1 --2- 172.21.15.2:0/4050099393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 0x5610201e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.414+0000 7fd115c69740 1 --2- 172.21.15.2:0/4050099393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c4760 0x5610201e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.414+0000 7fd115c69740 1 --2- 172.21.15.2:0/4050099393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 0x5610201cc790 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.414+0000 7fd115c69740 1 -- 172.21.15.2:0/4050099393 >> 172.21.15.2:0/4050099393 conn(0x5610201c03d0 msgr2=0x5610201cafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.414+0000 7fd115c69740 1 -- 172.21.15.2:0/4050099393 shutdown_connections 2024-09-10T08:21:38.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.414+0000 7fd115c69740 1 -- 172.21.15.2:0/4050099393 wait complete. 2024-09-10T08:21:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.414+0000 7fd115c69740 1 -- start start 2024-09-10T08:21:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.414+0000 7fd115c69740 1 Processor -- start 2024-09-10T08:21:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.415+0000 7fd115c69740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 0x5610201d7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.415+0000 7fd115c69740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c4760 0x5610201d7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.415+0000 7fd115c69740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 0x5610201d2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.415+0000 7fd115c69740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c1ab240 con 0x5610201c3e40 2024-09-10T08:21:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.415+0000 7fd115c69740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c1ac6b0 con 0x5610201cf020 2024-09-10T08:21:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.415+0000 7fd115c69740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c1ab910 con 0x5610201c4760 2024-09-10T08:21:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.415+0000 7fd115c69740 10 client.0 _ll_register_callbacks cb 0x7fd1040093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.415+0000 7fd111200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 0x5610201d7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.415+0000 7fd111200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 0x5610201d7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50700/0 (socket says 172.21.15.2:50700) 2024-09-10T08:21:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.415+0000 7fd111200640 1 -- 172.21.15.2:0/3704046217 learned_addr learned my addr 172.21.15.2:0/3704046217 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.415+0000 7fd103400640 1 --2- 172.21.15.2:0/3704046217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c4760 0x5610201d7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.415+0000 7fd111200640 1 --2- 172.21.15.2:0/3704046217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 0x5610201d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.416+0000 7fd111200640 1 -- 172.21.15.2:0/3704046217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c4760 msgr2=0x5610201d7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.416+0000 7fd111200640 1 --2- 172.21.15.2:0/3704046217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c4760 0x5610201d7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.416+0000 7fd111200640 1 -- 172.21.15.2:0/3704046217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 msgr2=0x5610201d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.416+0000 7fd111200640 1 --2- 172.21.15.2:0/3704046217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 0x5610201d2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.416+0000 7fd111200640 1 -- 172.21.15.2:0/3704046217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610201d2fc0 con 0x5610201c3e40 2024-09-10T08:21:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.416+0000 7fd111200640 1 --2- 172.21.15.2:0/3704046217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 0x5610201d7340 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd104009540 tx=0x7fd104005380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29aac20ceca96ccc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.416+0000 7fd0fa000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.416+0000 7fd0fa000640 1 -- 172.21.15.2:0/3704046217 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104004310 con 0x5610201c3e40 2024-09-10T08:21:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.416+0000 7fd115c69740 1 -- 172.21.15.2:0/3704046217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd104004530 con 0x5610201c3e40 2024-09-10T08:21:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.416+0000 7fd0fa000640 1 -- 172.21.15.2:0/3704046217 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd104004960 con 0x5610201c3e40 2024-09-10T08:21:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.416+0000 7fd0fa000640 1 -- 172.21.15.2:0/3704046217 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd104004b80 con 0x5610201c3e40 2024-09-10T08:21:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.417+0000 7fd115c69740 10 client.19227 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:38.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.417+0000 7fd115c69740 1 -- 172.21.15.2:0/3704046217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561020249960 con 0x5610201c3e40 2024-09-10T08:21:38.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.417+0000 7fd0fa000640 1 -- 172.21.15.2:0/3704046217 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd104004530 con 0x5610201c3e40 2024-09-10T08:21:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.417+0000 7fd0fa000640 1 -- 172.21.15.2:0/3704046217 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd10400f6a0 con 0x5610201c3e40 2024-09-10T08:21:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.417+0000 7fd115c69740 10 client.19227 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.417+0000 7fd115c69740 10 client.19227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.417+0000 7fd115c69740 20 client.19227 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.417+0000 7fd115c69740 1 --2- 172.21.15.2:0/3704046217 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561020249e90 0x56102024c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.417+0000 7fd115c69740 4 client.19227 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:21:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.417+0000 7fd115c69740 1 -- 172.21.15.2:0/3704046217 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561020131e60 con 0x561020249e90 2024-09-10T08:21:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.418+0000 7fd103400640 1 --2- 172.21.15.2:0/3704046217 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561020249e90 0x56102024c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.418+0000 7fd103400640 1 --2- 172.21.15.2:0/3704046217 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561020249e90 0x56102024c280 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f302930a935f56 server_cookie=1f4562c18b9b060 in_seq=0 out_seq=0 2024-09-10T08:21:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.418+0000 7fd0fa000640 10 client.19227 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.418+0000 7fd0fa000640 1 -- 172.21.15.2:0/3704046217 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561020131e60 con 0x561020249e90 2024-09-10T08:21:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.418+0000 7fd0fa000640 10 client.19227 handle_command_reply: tid=0 2024-09-10T08:21:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.418+0000 7fd115c69740 1 client.19227 shutdown 2024-09-10T08:21:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.419+0000 7fd115c69740 1 -- 172.21.15.2:0/3704046217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 msgr2=0x5610201d7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.419+0000 7fd115c69740 1 --2- 172.21.15.2:0/3704046217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 0x5610201d7340 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd104009540 tx=0x7fd104005380 comp rx=0 tx=0).stop 2024-09-10T08:21:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.419+0000 7fd115c69740 1 -- 172.21.15.2:0/3704046217 shutdown_connections 2024-09-10T08:21:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.419+0000 7fd115c69740 1 --2- 172.21.15.2:0/3704046217 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561020249e90 0x56102024c280 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.419+0000 7fd115c69740 1 --2- 172.21.15.2:0/3704046217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610201cf020 0x5610201d2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.419+0000 7fd115c69740 1 --2- 172.21.15.2:0/3704046217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610201c4760 0x5610201d7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.419+0000 7fd115c69740 1 --2- 172.21.15.2:0/3704046217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5610201c3e40 0x5610201d7340 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.419+0000 7fd115c69740 1 -- 172.21.15.2:0/3704046217 >> 172.21.15.2:0/3704046217 conn(0x5610201c03d0 msgr2=0x5610201cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.419+0000 7fd0fa000640 0 client.19227 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.419+0000 7fd115c69740 1 -- 172.21.15.2:0/3704046217 shutdown_connections 2024-09-10T08:21:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.419+0000 7fd115c69740 1 -- 172.21.15.2:0/3704046217 wait complete. 2024-09-10T08:21:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.419+0000 7fd115c69740 20 client.19227 trim_cache size 0 max 16384 2024-09-10T08:21:38.423 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:21:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.420+0000 7fd111c00640 1 -- 172.21.15.2:0/59283608 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0f00a2c00 msgr2=0x7fd0f00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.420+0000 7fd111c00640 1 --2- 172.21.15.2:0/59283608 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0f00a2c00 0x7fd0f00a50c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc09b9f0 tx=0x7fd0fc0d64f0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.420+0000 7fd111c00640 1 -- 172.21.15.2:0/59283608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10c1019c0 msgr2=0x7fd10c1982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.420+0000 7fd111c00640 1 --2- 172.21.15.2:0/59283608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10c1019c0 0x7fd10c1982f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009bc0 tx=0x7fd0fc00b690 comp rx=0 tx=0).stop 2024-09-10T08:21:38.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.420+0000 7fd111c00640 1 -- 172.21.15.2:0/59283608 shutdown_connections 2024-09-10T08:21:38.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.420+0000 7fd111c00640 1 --2- 172.21.15.2:0/59283608 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0f00a2c00 0x7fd0f00a50c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.420+0000 7fd111c00640 1 --2- 172.21.15.2:0/59283608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd10c1079c0 0x7fd10c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.420+0000 7fd111c00640 1 --2- 172.21.15.2:0/59283608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd10c102360 0x7fd10c198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.420+0000 7fd111c00640 1 --2- 172.21.15.2:0/59283608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd10c1019c0 0x7fd10c1982f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:38.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.420+0000 7fd111c00640 1 -- 172.21.15.2:0/59283608 >> 172.21.15.2:0/59283608 conn(0x7fd10c0fd740 msgr2=0x7fd10c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:38.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.420+0000 7fd111c00640 1 -- 172.21.15.2:0/59283608 shutdown_connections 2024-09-10T08:21:38.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:38.420+0000 7fd111c00640 1 -- 172.21.15.2:0/59283608 wait complete. 2024-09-10T08:21:39.551 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:21:39.936 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:40.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:39 smithi002 ceph-mon[30563]: pgmap v792: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-10T08:21:40.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:39 smithi155 ceph-mon[35405]: pgmap v792: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-10T08:21:40.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:39 smithi107 ceph-mon[35048]: pgmap v792: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-10T08:21:40.426 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:21:40.427 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:21:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.568+0000 7f3ccba00640 1 -- 172.21.15.2:0/3041366456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cc4103d60 msgr2=0x7f3cc4104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.568+0000 7f3ccba00640 1 --2- 172.21.15.2:0/3041366456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cc4103d60 0x7f3cc4104160 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009900 tx=0x7f3cb802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.569+0000 7f3ccba00640 1 -- 172.21.15.2:0/3041366456 shutdown_connections 2024-09-10T08:21:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.569+0000 7f3ccba00640 1 --2- 172.21.15.2:0/3041366456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cc4105910 0x7f3cc410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.569+0000 7f3ccba00640 1 --2- 172.21.15.2:0/3041366456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc4104f50 0x7f3cc41053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.569+0000 7f3ccba00640 1 --2- 172.21.15.2:0/3041366456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cc4103d60 0x7f3cc4104160 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.569+0000 7f3ccba00640 1 -- 172.21.15.2:0/3041366456 >> 172.21.15.2:0/3041366456 conn(0x7f3cc40ff4e0 msgr2=0x7f3cc4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.569+0000 7f3ccba00640 1 -- 172.21.15.2:0/3041366456 shutdown_connections 2024-09-10T08:21:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.569+0000 7f3ccba00640 1 -- 172.21.15.2:0/3041366456 wait complete. 2024-09-10T08:21:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.570+0000 7f3ccba00640 1 Processor -- start 2024-09-10T08:21:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.570+0000 7f3ccba00640 1 -- start start 2024-09-10T08:21:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3ccba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc4103d60 0x7f3cc419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3ccba00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cc4104f50 0x7f3cc419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3ccba00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cc4105910 0x7f3cc41a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3ccba00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc410f9e0 con 0x7f3cc4104f50 2024-09-10T08:21:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3ccba00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc410f860 con 0x7f3cc4105910 2024-09-10T08:21:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3ccba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc410fb60 con 0x7f3cc4103d60 2024-09-10T08:21:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3cc9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc4103d60 0x7f3cc419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3cc3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cc4104f50 0x7f3cc419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3cc9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc4103d60 0x7f3cc419a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51782/0 (socket says 172.21.15.2:51782) 2024-09-10T08:21:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3cc9000640 1 -- 172.21.15.2:0/1178964139 learned_addr learned my addr 172.21.15.2:0/1178964139 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3cc9a00640 1 --2- 172.21.15.2:0/1178964139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cc4105910 0x7f3cc41a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3cc9000640 1 -- 172.21.15.2:0/1178964139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cc4105910 msgr2=0x7f3cc41a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3cc9000640 1 --2- 172.21.15.2:0/1178964139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cc4105910 0x7f3cc41a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3cc9000640 1 -- 172.21.15.2:0/1178964139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cc4104f50 msgr2=0x7f3cc419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.571+0000 7f3cc9000640 1 --2- 172.21.15.2:0/1178964139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cc4104f50 0x7f3cc419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.572+0000 7f3cc9000640 1 -- 172.21.15.2:0/1178964139 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc41a1c20 con 0x7f3cc4103d60 2024-09-10T08:21:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.572+0000 7f3cc9000640 1 --2- 172.21.15.2:0/1178964139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc4103d60 0x7f3cc419a000 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8031610 tx=0x7f3cb8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.572+0000 7f3cc1600640 1 -- 172.21.15.2:0/1178964139 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb803f030 con 0x7f3cc4103d60 2024-09-10T08:21:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.572+0000 7f3cc1600640 1 -- 172.21.15.2:0/1178964139 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb8003ad0 con 0x7f3cc4103d60 2024-09-10T08:21:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.572+0000 7f3ccba00640 1 -- 172.21.15.2:0/1178964139 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc41a1eb0 con 0x7f3cc4103d60 2024-09-10T08:21:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.572+0000 7f3cc1600640 1 -- 172.21.15.2:0/1178964139 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8035070 con 0x7f3cc4103d60 2024-09-10T08:21:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.573+0000 7f3ccba00640 1 -- 172.21.15.2:0/1178964139 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc41a22c0 con 0x7f3cc4103d60 2024-09-10T08:21:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.576+0000 7f3cc1600640 1 -- 172.21.15.2:0/1178964139 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3cb8042030 con 0x7f3cc4103d60 2024-09-10T08:21:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.577+0000 7f3cce82e740 1 Processor -- start 2024-09-10T08:21:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.577+0000 7f3cce82e740 1 -- start start 2024-09-10T08:21:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.577+0000 7f3cc1600640 1 --2- 172.21.15.2:0/1178964139 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3cac0a2c30 0x7f3cac0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.577+0000 7f3cce82e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d455090 0x55a49d455490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.577+0000 7f3cce82e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d45a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.577+0000 7f3cce82e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d45aa60 0x55a49d45cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.577+0000 7f3cce82e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a49d3ba100 con 0x55a49d4559d0 2024-09-10T08:21:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.577+0000 7f3cce82e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a49d3621b0 con 0x55a49d45aa60 2024-09-10T08:21:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.577+0000 7f3cce82e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a49d3b60c0 con 0x55a49d455090 2024-09-10T08:21:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc1600640 1 -- 172.21.15.2:0/1178964139 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3cb80efa20 con 0x7f3cc4103d60 2024-09-10T08:21:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d455090 0x55a49d455490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc9a00640 1 --2- 172.21.15.2:0/1178964139 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3cac0a2c30 0x7f3cac0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d455090 0x55a49d455490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51798/0 (socket says 172.21.15.2:51798) 2024-09-10T08:21:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc9000640 1 -- 172.21.15.2:0/2471839024 learned_addr learned my addr 172.21.15.2:0/2471839024 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc3e00640 1 --2- 172.21.15.2:0/2471839024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d45a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc9a00640 1 --2- 172.21.15.2:0/2471839024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d45aa60 0x55a49d45cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc9000640 1 -- 172.21.15.2:0/2471839024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d45aa60 msgr2=0x55a49d45cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc9000640 1 --2- 172.21.15.2:0/2471839024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d45aa60 0x55a49d45cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc9000640 1 -- 172.21.15.2:0/2471839024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 msgr2=0x55a49d45a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc9000640 1 --2- 172.21.15.2:0/2471839024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d45a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc9000640 1 -- 172.21.15.2:0/2471839024 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a49d0cd210 con 0x55a49d455090 2024-09-10T08:21:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc3e00640 1 --2- 172.21.15.2:0/2471839024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d45a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:21:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.578+0000 7f3cc9a00640 1 --2- 172.21.15.2:0/2471839024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d45aa60 0x55a49d45cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:21:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.579+0000 7f3cc9000640 1 --2- 172.21.15.2:0/2471839024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d455090 0x55a49d455490 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80bbe70 tx=0x7f3cb80bb840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe1c794c0650cac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.579+0000 7f3cb6000640 1 -- 172.21.15.2:0/2471839024 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8049040 con 0x55a49d455090 2024-09-10T08:21:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.579+0000 7f3cc9a00640 1 --2- 172.21.15.2:0/1178964139 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3cac0a2c30 0x7f3cac0a50f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc005fd0 tx=0x7f3cbc009bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.579+0000 7f3cb6000640 1 -- 172.21.15.2:0/2471839024 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb803be40 con 0x55a49d455090 2024-09-10T08:21:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.579+0000 7f3cb6000640 1 -- 172.21.15.2:0/2471839024 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80ed1e0 con 0x55a49d455090 2024-09-10T08:21:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.580+0000 7f3cce82e740 1 -- 172.21.15.2:0/2471839024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d455090 msgr2=0x55a49d455490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.580+0000 7f3cce82e740 1 --2- 172.21.15.2:0/2471839024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d455090 0x55a49d455490 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80bbe70 tx=0x7f3cb80bb840 comp rx=0 tx=0).stop 2024-09-10T08:21:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.580+0000 7f3cce82e740 1 -- 172.21.15.2:0/2471839024 shutdown_connections 2024-09-10T08:21:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.580+0000 7f3cce82e740 1 --2- 172.21.15.2:0/2471839024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d45aa60 0x55a49d45cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.580+0000 7f3cce82e740 1 --2- 172.21.15.2:0/2471839024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d45a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.580+0000 7f3cce82e740 1 --2- 172.21.15.2:0/2471839024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d455090 0x55a49d455490 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.580+0000 7f3cce82e740 1 -- 172.21.15.2:0/2471839024 >> 172.21.15.2:0/2471839024 conn(0x55a49d450700 msgr2=0x55a49d450b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.580+0000 7f3cce82e740 1 -- 172.21.15.2:0/2471839024 shutdown_connections 2024-09-10T08:21:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.580+0000 7f3cce82e740 1 -- 172.21.15.2:0/2471839024 wait complete. 2024-09-10T08:21:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.580+0000 7f3cce82e740 1 -- start start 2024-09-10T08:21:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.580+0000 7f3cce82e740 1 Processor -- start 2024-09-10T08:21:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.581+0000 7f3cce82e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d455090 0x55a49d45efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.581+0000 7f3cce82e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d45f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.581+0000 7f3cce82e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d45aa60 0x55a49d466540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.581+0000 7f3cce82e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a49d45d7f0 con 0x55a49d4559d0 2024-09-10T08:21:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.581+0000 7f3cce82e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a49d343090 con 0x55a49d455090 2024-09-10T08:21:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.581+0000 7f3cce82e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a49d45d970 con 0x55a49d45aa60 2024-09-10T08:21:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.581+0000 7f3cce82e740 10 client.0 _ll_register_callbacks cb 0x55a49d397f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.581+0000 7f3cc3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d45f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.581+0000 7f3cc3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d45f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50730/0 (socket says 172.21.15.2:50730) 2024-09-10T08:21:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.581+0000 7f3cc9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d455090 0x55a49d45efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.581+0000 7f3cc3e00640 1 -- 172.21.15.2:0/4030741994 learned_addr learned my addr 172.21.15.2:0/4030741994 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.581+0000 7f3cc9a00640 1 --2- 172.21.15.2:0/4030741994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d45aa60 0x55a49d466540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.582+0000 7f3cc9000640 1 -- 172.21.15.2:0/4030741994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d45aa60 msgr2=0x55a49d466540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.582+0000 7f3cc9000640 1 --2- 172.21.15.2:0/4030741994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d45aa60 0x55a49d466540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.582+0000 7f3cc9000640 1 -- 172.21.15.2:0/4030741994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 msgr2=0x55a49d45f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.582+0000 7f3cc9000640 1 --2- 172.21.15.2:0/4030741994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d45f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.582+0000 7f3cc9000640 1 -- 172.21.15.2:0/4030741994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49d199620 con 0x55a49d455090 2024-09-10T08:21:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.582+0000 7f3cc9000640 1 --2- 172.21.15.2:0/4030741994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d455090 0x55a49d45efc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8003fd0 tx=0x7f3cb80b7ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f1b04df914eccc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.582+0000 7f3caaa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:21:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.582+0000 7f3caaa00640 1 -- 172.21.15.2:0/4030741994 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8049050 con 0x55a49d455090 2024-09-10T08:21:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.582+0000 7f3cce82e740 1 -- 172.21.15.2:0/4030741994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a49d3c7830 con 0x55a49d455090 2024-09-10T08:21:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.582+0000 7f3caaa00640 1 -- 172.21.15.2:0/4030741994 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3cb803b460 con 0x55a49d455090 2024-09-10T08:21:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.582+0000 7f3caaa00640 1 -- 172.21.15.2:0/4030741994 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3cb80035f0 con 0x55a49d455090 2024-09-10T08:21:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.582+0000 7f3cce82e740 10 client.29261 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.582+0000 7f3cce82e740 1 -- 172.21.15.2:0/4030741994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a49d45d7f0 con 0x55a49d455090 2024-09-10T08:21:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.583+0000 7f3caaa00640 1 -- 172.21.15.2:0/4030741994 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a49d3c7830 con 0x55a49d455090 2024-09-10T08:21:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.583+0000 7f3caaa00640 1 -- 172.21.15.2:0/4030741994 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3cb80c4610 con 0x55a49d455090 2024-09-10T08:21:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.583+0000 7f3cce82e740 10 client.29261 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.583+0000 7f3cce82e740 10 client.29261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.583+0000 7f3cce82e740 20 client.29261 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.583+0000 7f3cce82e740 1 --2- 172.21.15.2:0/4030741994 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a49d46a730 0x55a49d46cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.583+0000 7f3cce82e740 4 client.29261 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.583+0000 7f3cce82e740 1 -- 172.21.15.2:0/4030741994 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a49d3c1b80 con 0x55a49d46a730 2024-09-10T08:21:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.583+0000 7f3cc3e00640 1 --2- 172.21.15.2:0/4030741994 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a49d46a730 0x55a49d46cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.584+0000 7f3cc3e00640 1 --2- 172.21.15.2:0/4030741994 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a49d46a730 0x55a49d46cb20 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57ce22dddd6f0ed6 server_cookie=1d4c804f9dc666d5 in_seq=0 out_seq=0 2024-09-10T08:21:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.584+0000 7f3caaa00640 10 client.29261 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3caaa00640 1 -- 172.21.15.2:0/4030741994 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a49d3c1b80 con 0x55a49d46a730 2024-09-10T08:21:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3caaa00640 10 client.29261 handle_command_reply: tid=0 2024-09-10T08:21:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3cce82e740 1 client.29261 shutdown 2024-09-10T08:21:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3cce82e740 1 -- 172.21.15.2:0/4030741994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d455090 msgr2=0x55a49d45efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3cce82e740 1 --2- 172.21.15.2:0/4030741994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d455090 0x55a49d45efc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8003fd0 tx=0x7f3cb80b7ad0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3cce82e740 1 -- 172.21.15.2:0/4030741994 shutdown_connections 2024-09-10T08:21:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3cce82e740 1 --2- 172.21.15.2:0/4030741994 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a49d46a730 0x55a49d46cb20 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3cce82e740 1 --2- 172.21.15.2:0/4030741994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d45aa60 0x55a49d466540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3cce82e740 1 --2- 172.21.15.2:0/4030741994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d45f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3cce82e740 1 --2- 172.21.15.2:0/4030741994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d455090 0x55a49d45efc0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3cce82e740 1 -- 172.21.15.2:0/4030741994 >> 172.21.15.2:0/4030741994 conn(0x55a49d450700 msgr2=0x55a49d451f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3caaa00640 0 client.29261 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3cce82e740 1 -- 172.21.15.2:0/4030741994 shutdown_connections 2024-09-10T08:21:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3cce82e740 1 -- 172.21.15.2:0/4030741994 wait complete. 2024-09-10T08:21:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.585+0000 7f3cce82e740 20 client.29261 trim_cache size 0 max 16384 2024-09-10T08:21:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.601+0000 7f3cce82e740 1 Processor -- start 2024-09-10T08:21:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.601+0000 7f3cce82e740 1 -- start start 2024-09-10T08:21:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.601+0000 7f3cce82e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d455090 0x55a49d45ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.601+0000 7f3cce82e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d466500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.601+0000 7f3cce82e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d45aa60 0x55a49d45f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.601+0000 7f3cce82e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41af2c0 con 0x55a49d4559d0 2024-09-10T08:21:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.601+0000 7f3cce82e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41acb90 con 0x55a49d455090 2024-09-10T08:21:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.601+0000 7f3cce82e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41aa9e0 con 0x55a49d45aa60 2024-09-10T08:21:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.602+0000 7f3cc3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d466500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.602+0000 7f3cc3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d466500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50740/0 (socket says 172.21.15.2:50740) 2024-09-10T08:21:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.602+0000 7f3cc9a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d45aa60 0x55a49d45f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.602+0000 7f3cc3e00640 1 -- 172.21.15.2:0/1658251307 learned_addr learned my addr 172.21.15.2:0/1658251307 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.602+0000 7f3cc9000640 1 --2- 172.21.15.2:0/1658251307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d455090 0x55a49d45ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.602+0000 7f3cc9000640 1 -- 172.21.15.2:0/1658251307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d45aa60 msgr2=0x55a49d45f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.602+0000 7f3cc9000640 1 --2- 172.21.15.2:0/1658251307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d45aa60 0x55a49d45f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.602+0000 7f3cc9000640 1 -- 172.21.15.2:0/1658251307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 msgr2=0x55a49d466500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.602+0000 7f3cc9000640 1 --2- 172.21.15.2:0/1658251307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d466500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.602+0000 7f3cc9000640 1 -- 172.21.15.2:0/1658251307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a49d450520 con 0x55a49d455090 2024-09-10T08:21:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.602+0000 7f3cc3e00640 1 --2- 172.21.15.2:0/1658251307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d466500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.602+0000 7f3cc9000640 1 --2- 172.21.15.2:0/1658251307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d455090 0x55a49d45ed70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80bc0a0 tx=0x7f3cb80bb840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae16d2c40075f86b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.602+0000 7f3cb5600640 1 -- 172.21.15.2:0/1658251307 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8049070 con 0x55a49d455090 2024-09-10T08:21:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.603+0000 7f3cb5600640 1 -- 172.21.15.2:0/1658251307 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb80c01e0 con 0x55a49d455090 2024-09-10T08:21:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.603+0000 7f3cb5600640 1 -- 172.21.15.2:0/1658251307 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80c4610 con 0x55a49d455090 2024-09-10T08:21:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.603+0000 7f3cce82e740 1 -- 172.21.15.2:0/1658251307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d455090 msgr2=0x55a49d45ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.603+0000 7f3cce82e740 1 --2- 172.21.15.2:0/1658251307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d455090 0x55a49d45ed70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80bc0a0 tx=0x7f3cb80bb840 comp rx=0 tx=0).stop 2024-09-10T08:21:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.603+0000 7f3cce82e740 1 -- 172.21.15.2:0/1658251307 shutdown_connections 2024-09-10T08:21:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.603+0000 7f3cce82e740 1 --2- 172.21.15.2:0/1658251307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d45aa60 0x55a49d45f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.603+0000 7f3cce82e740 1 --2- 172.21.15.2:0/1658251307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d4559d0 0x55a49d466500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.603+0000 7f3cce82e740 1 --2- 172.21.15.2:0/1658251307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d455090 0x55a49d45ed70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.603+0000 7f3cce82e740 1 -- 172.21.15.2:0/1658251307 >> 172.21.15.2:0/1658251307 conn(0x55a49d450700 msgr2=0x55a49d456c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.603+0000 7f3cce82e740 1 -- 172.21.15.2:0/1658251307 shutdown_connections 2024-09-10T08:21:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.603+0000 7f3cce82e740 1 -- 172.21.15.2:0/1658251307 wait complete. 2024-09-10T08:21:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.604+0000 7f3cce82e740 1 -- start start 2024-09-10T08:21:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.604+0000 7f3cce82e740 1 Processor -- start 2024-09-10T08:21:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.604+0000 7f3cce82e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d455090 0x55a49d4663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.604+0000 7f3cce82e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d4559d0 0x55a49d462e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.604+0000 7f3cce82e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d45aa60 0x55a49d4633d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.604+0000 7f3cce82e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41ab050 con 0x55a49d455090 2024-09-10T08:21:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.604+0000 7f3cce82e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41ac4c0 con 0x55a49d45aa60 2024-09-10T08:21:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.604+0000 7f3cce82e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc41ab720 con 0x55a49d4559d0 2024-09-10T08:21:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.604+0000 7f3cce82e740 10 client.0 _ll_register_callbacks cb 0x7f3cb80b8270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.605+0000 7f3cc9000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d455090 0x55a49d4663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.605+0000 7f3cc9a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d45aa60 0x55a49d4633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.605+0000 7f3cc9000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d455090 0x55a49d4663f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50742/0 (socket says 172.21.15.2:50742) 2024-09-10T08:21:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.605+0000 7f3cc9000640 1 -- 172.21.15.2:0/3805319832 learned_addr learned my addr 172.21.15.2:0/3805319832 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.605+0000 7f3cc3e00640 1 --2- 172.21.15.2:0/3805319832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d4559d0 0x55a49d462e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.605+0000 7f3cc9000640 1 -- 172.21.15.2:0/3805319832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d4559d0 msgr2=0x55a49d462e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.605+0000 7f3cc9000640 1 --2- 172.21.15.2:0/3805319832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d4559d0 0x55a49d462e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.605+0000 7f3cc9000640 1 -- 172.21.15.2:0/3805319832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d45aa60 msgr2=0x55a49d4633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.605+0000 7f3cc9000640 1 --2- 172.21.15.2:0/3805319832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d45aa60 0x55a49d4633d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.605+0000 7f3cc9000640 1 -- 172.21.15.2:0/3805319832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49d4d3360 con 0x55a49d455090 2024-09-10T08:21:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.605+0000 7f3cc9a00640 1 --2- 172.21.15.2:0/3805319832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d45aa60 0x55a49d4633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.606+0000 7f3cc9000640 1 --2- 172.21.15.2:0/3805319832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d455090 0x55a49d4663f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80b8340 tx=0x7f3cb80ed150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68214cf9a4059fad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.606+0000 7f3caaa00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.606+0000 7f3caaa00640 1 -- 172.21.15.2:0/3805319832 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb809f060 con 0x55a49d455090 2024-09-10T08:21:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.606+0000 7f3caaa00640 1 -- 172.21.15.2:0/3805319832 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3cb8031bd0 con 0x55a49d455090 2024-09-10T08:21:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.606+0000 7f3cce82e740 1 -- 172.21.15.2:0/3805319832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cb80c01e0 con 0x55a49d455090 2024-09-10T08:21:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.606+0000 7f3caaa00640 1 -- 172.21.15.2:0/3805319832 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3cb8049070 con 0x55a49d455090 2024-09-10T08:21:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.606+0000 7f3cce82e740 10 client.19245 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.606+0000 7f3cce82e740 1 -- 172.21.15.2:0/3805319832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a49d4d3360 con 0x55a49d455090 2024-09-10T08:21:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.606+0000 7f3caaa00640 1 -- 172.21.15.2:0/3805319832 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb80c01e0 con 0x55a49d455090 2024-09-10T08:21:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.607+0000 7f3caaa00640 1 -- 172.21.15.2:0/3805319832 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3cb8084cd0 con 0x55a49d455090 2024-09-10T08:21:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.607+0000 7f3cce82e740 10 client.19245 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.607+0000 7f3cce82e740 10 client.19245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.607+0000 7f3cce82e740 20 client.19245 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.607+0000 7f3cce82e740 1 --2- 172.21.15.2:0/3805319832 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a49d4d4ab0 0x55a49d4d6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.607+0000 7f3cce82e740 4 client.19245 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:21:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.607+0000 7f3cce82e740 1 -- 172.21.15.2:0/3805319832 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a49d33e7c0 con 0x55a49d4d4ab0 2024-09-10T08:21:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.607+0000 7f3cc3e00640 1 --2- 172.21.15.2:0/3805319832 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a49d4d4ab0 0x55a49d4d6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.608+0000 7f3caaa00640 10 client.19245 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.608+0000 7f3cc3e00640 1 --2- 172.21.15.2:0/3805319832 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a49d4d4ab0 0x55a49d4d6ed0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=299f91cfafbe0279 server_cookie=d19739cf784f4bf5 in_seq=0 out_seq=0 2024-09-10T08:21:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.619+0000 7f3caaa00640 1 -- 172.21.15.2:0/3805319832 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a49d33e7c0 con 0x55a49d4d4ab0 2024-09-10T08:21:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.619+0000 7f3caaa00640 10 client.19245 handle_command_reply: tid=0 2024-09-10T08:21:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.619+0000 7f3cce82e740 1 client.19245 shutdown 2024-09-10T08:21:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.619+0000 7f3cce82e740 1 -- 172.21.15.2:0/3805319832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d455090 msgr2=0x55a49d4663f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.619+0000 7f3cce82e740 1 --2- 172.21.15.2:0/3805319832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d455090 0x55a49d4663f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80b8340 tx=0x7f3cb80ed150 comp rx=0 tx=0).stop 2024-09-10T08:21:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.620+0000 7f3cce82e740 1 -- 172.21.15.2:0/3805319832 shutdown_connections 2024-09-10T08:21:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.620+0000 7f3cce82e740 1 --2- 172.21.15.2:0/3805319832 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a49d4d4ab0 0x55a49d4d6ed0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.620+0000 7f3cce82e740 1 --2- 172.21.15.2:0/3805319832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a49d45aa60 0x55a49d4633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.620+0000 7f3cce82e740 1 --2- 172.21.15.2:0/3805319832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a49d4559d0 0x55a49d462e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.620+0000 7f3cce82e740 1 --2- 172.21.15.2:0/3805319832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a49d455090 0x55a49d4663f0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.620+0000 7f3cce82e740 1 -- 172.21.15.2:0/3805319832 >> 172.21.15.2:0/3805319832 conn(0x55a49d450700 msgr2=0x55a49d44f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.620+0000 7f3caaa00640 0 client.19245 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.620+0000 7f3cce82e740 1 -- 172.21.15.2:0/3805319832 shutdown_connections 2024-09-10T08:21:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.620+0000 7f3cce82e740 1 -- 172.21.15.2:0/3805319832 wait complete. 2024-09-10T08:21:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.620+0000 7f3cce82e740 20 client.19245 trim_cache size 0 max 16384 2024-09-10T08:21:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:21:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:21:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:21:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:21:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.621+0000 7f3ccba00640 1 -- 172.21.15.2:0/1178964139 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3cac0a2c30 msgr2=0x7f3cac0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.621+0000 7f3ccba00640 1 --2- 172.21.15.2:0/1178964139 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3cac0a2c30 0x7f3cac0a50f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc005fd0 tx=0x7f3cbc009bf0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.621+0000 7f3ccba00640 1 -- 172.21.15.2:0/1178964139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc4103d60 msgr2=0x7f3cc419a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.621+0000 7f3ccba00640 1 --2- 172.21.15.2:0/1178964139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc4103d60 0x7f3cc419a000 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8031610 tx=0x7f3cb8004270 comp rx=0 tx=0).stop 2024-09-10T08:21:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.621+0000 7f3ccba00640 1 -- 172.21.15.2:0/1178964139 shutdown_connections 2024-09-10T08:21:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.621+0000 7f3ccba00640 1 --2- 172.21.15.2:0/1178964139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cc4105910 0x7f3cc41a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.621+0000 7f3ccba00640 1 --2- 172.21.15.2:0/1178964139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cc4104f50 0x7f3cc419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.621+0000 7f3ccba00640 1 --2- 172.21.15.2:0/1178964139 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3cac0a2c30 0x7f3cac0a50f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.621+0000 7f3ccba00640 1 --2- 172.21.15.2:0/1178964139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc4103d60 0x7f3cc419a000 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.621+0000 7f3ccba00640 1 -- 172.21.15.2:0/1178964139 >> 172.21.15.2:0/1178964139 conn(0x7f3cc40ff4e0 msgr2=0x7f3cc41082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.621+0000 7f3ccba00640 1 -- 172.21.15.2:0/1178964139 shutdown_connections 2024-09-10T08:21:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:40.621+0000 7f3ccba00640 1 -- 172.21.15.2:0/1178964139 wait complete. 2024-09-10T08:21:40.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.663+0000 7fa18e000640 1 -- 172.21.15.2:0/600388874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa188102360 msgr2=0x7fa18810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.663+0000 7fa18e000640 1 --2- 172.21.15.2:0/600388874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa188102360 0x7fa18810e890 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fa180009cd0 tx=0x7fa18002a8f0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.663+0000 7fa18e000640 1 -- 172.21.15.2:0/600388874 shutdown_connections 2024-09-10T08:21:40.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.663+0000 7fa18e000640 1 --2- 172.21.15.2:0/600388874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa188102360 0x7fa18810e890 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.663+0000 7fa18e000640 1 --2- 172.21.15.2:0/600388874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1881019c0 0x7fa188101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.663+0000 7fa18e000640 1 --2- 172.21.15.2:0/600388874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1881079c0 0x7fa188107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.663+0000 7fa18e000640 1 -- 172.21.15.2:0/600388874 >> 172.21.15.2:0/600388874 conn(0x7fa1880fd740 msgr2=0x7fa1880ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:40.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.663+0000 7fa18e000640 1 -- 172.21.15.2:0/600388874 shutdown_connections 2024-09-10T08:21:40.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.663+0000 7fa18e000640 1 -- 172.21.15.2:0/600388874 wait complete. 2024-09-10T08:21:40.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.664+0000 7fa18e000640 1 Processor -- start 2024-09-10T08:21:40.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.664+0000 7fa18e000640 1 -- start start 2024-09-10T08:21:40.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.665+0000 7fa18e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1881019c0 0x7fa1881088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.665+0000 7fa18e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa188102360 0x7fa188108e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.665+0000 7fa18e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1881079c0 0x7fa188109370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.665+0000 7fa18e000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa188111e10 con 0x7fa188102360 2024-09-10T08:21:40.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.665+0000 7fa18e000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa188111c90 con 0x7fa1881019c0 2024-09-10T08:21:40.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.665+0000 7fa18e000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa188111f90 con 0x7fa1881079c0 2024-09-10T08:21:40.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.665+0000 7fa187e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa188102360 0x7fa188108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.665+0000 7fa187e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa188102360 0x7fa188108e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50754/0 (socket says 172.21.15.2:50754) 2024-09-10T08:21:40.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.665+0000 7fa187e00640 1 -- 172.21.15.2:0/150050193 learned_addr learned my addr 172.21.15.2:0/150050193 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:40.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.665+0000 7fa18cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1881019c0 0x7fa1881088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.665+0000 7fa18d600640 1 --2- 172.21.15.2:0/150050193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1881079c0 0x7fa188109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.666+0000 7fa18cc00640 1 -- 172.21.15.2:0/150050193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1881079c0 msgr2=0x7fa188109370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.666+0000 7fa18cc00640 1 --2- 172.21.15.2:0/150050193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1881079c0 0x7fa188109370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.666+0000 7fa18cc00640 1 -- 172.21.15.2:0/150050193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa188102360 msgr2=0x7fa188108e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.666+0000 7fa18cc00640 1 --2- 172.21.15.2:0/150050193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa188102360 0x7fa188108e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.666+0000 7fa18cc00640 1 -- 172.21.15.2:0/150050193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1881aab70 con 0x7fa1881019c0 2024-09-10T08:21:40.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.666+0000 7fa18cc00640 1 --2- 172.21.15.2:0/150050193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1881019c0 0x7fa1881088f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00e9f0 tx=0x7fa17c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:40.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.666+0000 7fa185600640 1 -- 172.21.15.2:0/150050193 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00cd80 con 0x7fa1881019c0 2024-09-10T08:21:40.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.666+0000 7fa185600640 1 -- 172.21.15.2:0/150050193 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa17c0035f0 con 0x7fa1881019c0 2024-09-10T08:21:40.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.666+0000 7fa18e000640 1 -- 172.21.15.2:0/150050193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1881aae60 con 0x7fa1881019c0 2024-09-10T08:21:40.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.666+0000 7fa185600640 1 -- 172.21.15.2:0/150050193 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c010670 con 0x7fa1881019c0 2024-09-10T08:21:40.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.667+0000 7fa18e000640 1 -- 172.21.15.2:0/150050193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1881ab310 con 0x7fa1881019c0 2024-09-10T08:21:40.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.668+0000 7fa18e000640 1 -- 172.21.15.2:0/150050193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa188101e20 con 0x7fa1881019c0 2024-09-10T08:21:40.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.670+0000 7fa185600640 1 -- 172.21.15.2:0/150050193 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa17c0037a0 con 0x7fa1881019c0 2024-09-10T08:21:40.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.672+0000 7fa185600640 1 --2- 172.21.15.2:0/150050193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1700a2b80 0x7fa1700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:40.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.672+0000 7fa185600640 1 -- 172.21.15.2:0/150050193 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa17c0cf690 con 0x7fa1881019c0 2024-09-10T08:21:40.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.672+0000 7fa187e00640 1 --2- 172.21.15.2:0/150050193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1700a2b80 0x7fa1700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:40.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.673+0000 7fa187e00640 1 --2- 172.21.15.2:0/150050193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1700a2b80 0x7fa1700a5040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa1880fffe0 tx=0x7fa174005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:40.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.676+0000 7fa185600640 1 -- 172.21.15.2:0/150050193 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa17c09c1a0 con 0x7fa1881019c0 2024-09-10T08:21:40.831 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:40.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.923+0000 7fa18e000640 1 -- 172.21.15.2:0/150050193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1881ab700 con 0x7fa1881019c0 2024-09-10T08:21:40.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.924+0000 7fa185600640 1 -- 172.21.15.2:0/150050193 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fa17c09baa0 con 0x7fa1881019c0 2024-09-10T08:21:40.924 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:21:40.924 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:21:40.925 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:21:40.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.926+0000 7fa18e000640 1 -- 172.21.15.2:0/150050193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1700a2b80 msgr2=0x7fa1700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.926+0000 7fa18e000640 1 --2- 172.21.15.2:0/150050193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1700a2b80 0x7fa1700a5040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa1880fffe0 tx=0x7fa174005f90 comp rx=0 tx=0).stop 2024-09-10T08:21:40.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.926+0000 7fa18e000640 1 -- 172.21.15.2:0/150050193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1881019c0 msgr2=0x7fa1881088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:40.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.926+0000 7fa18e000640 1 --2- 172.21.15.2:0/150050193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1881019c0 0x7fa1881088f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00e9f0 tx=0x7fa17c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.926+0000 7fa18e000640 1 -- 172.21.15.2:0/150050193 shutdown_connections 2024-09-10T08:21:40.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.926+0000 7fa18e000640 1 --2- 172.21.15.2:0/150050193 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa1700a2b80 0x7fa1700a5040 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.926+0000 7fa18e000640 1 --2- 172.21.15.2:0/150050193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1881079c0 0x7fa188109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.926+0000 7fa18e000640 1 --2- 172.21.15.2:0/150050193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa188102360 0x7fa188108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.926+0000 7fa18e000640 1 --2- 172.21.15.2:0/150050193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1881019c0 0x7fa1881088f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:40.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.926+0000 7fa18e000640 1 -- 172.21.15.2:0/150050193 >> 172.21.15.2:0/150050193 conn(0x7fa1880fd740 msgr2=0x7fa18810cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:40.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.926+0000 7fa18e000640 1 -- 172.21.15.2:0/150050193 shutdown_connections 2024-09-10T08:21:40.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:40.926+0000 7fa18e000640 1 -- 172.21.15.2:0/150050193 wait complete. 2024-09-10T08:21:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:41 smithi002 ceph-mon[30563]: pgmap v793: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-10T08:21:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:41 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/150050193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:41.867 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id 4958be9 --release --await' 2024-09-10T08:21:41.867 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id 4958be9 --release --await 2024-09-10T08:21:42.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.144+0000 7f8692e00640 1 -- 172.21.15.2:0/3645806775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f868c074f00 msgr2=0x7f868c10deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:42.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.144+0000 7f8692e00640 1 --2- 172.21.15.2:0/3645806775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f868c074f00 0x7f868c10deb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8674009860 tx=0x7f867402a3d0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.144+0000 7f8692e00640 1 -- 172.21.15.2:0/3645806775 shutdown_connections 2024-09-10T08:21:42.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.144+0000 7f8692e00640 1 --2- 172.21.15.2:0/3645806775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f868c10e3f0 0x7f868c110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.144+0000 7f8692e00640 1 --2- 172.21.15.2:0/3645806775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f868c074f00 0x7f868c10deb0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.144+0000 7f8692e00640 1 --2- 172.21.15.2:0/3645806775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868c0745e0 0x7f868c0749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.144+0000 7f8692e00640 1 -- 172.21.15.2:0/3645806775 >> 172.21.15.2:0/3645806775 conn(0x7f868c0fd8b0 msgr2=0x7f868c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:42.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.144+0000 7f8692e00640 1 -- 172.21.15.2:0/3645806775 shutdown_connections 2024-09-10T08:21:42.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.144+0000 7f8692e00640 1 -- 172.21.15.2:0/3645806775 wait complete. 2024-09-10T08:21:42.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.145+0000 7f8692e00640 1 Processor -- start 2024-09-10T08:21:42.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.145+0000 7f8692e00640 1 -- start start 2024-09-10T08:21:42.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.146+0000 7f8692e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f868c0745e0 0x7f868c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.146+0000 7f8692e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f868c074f00 0x7f868c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.146+0000 7f8692e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868c10e3f0 0x7f868c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.146+0000 7f8692e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f868c114160 con 0x7f868c0745e0 2024-09-10T08:21:42.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.146+0000 7f8692e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f868c113fe0 con 0x7f868c074f00 2024-09-10T08:21:42.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.146+0000 7f8692e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1142e0 con 0x7f868c10e3f0 2024-09-10T08:21:42.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.146+0000 7f868be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f868c0745e0 0x7f868c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.146+0000 7f868be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f868c0745e0 0x7f868c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50762/0 (socket says 172.21.15.2:50762) 2024-09-10T08:21:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.146+0000 7f868be00640 1 -- 172.21.15.2:0/3902935886 learned_addr learned my addr 172.21.15.2:0/3902935886 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.146+0000 7f8683400640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f868c074f00 0x7f868c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.146+0000 7f8690e00640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868c10e3f0 0x7f868c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.147+0000 7f8683400640 1 -- 172.21.15.2:0/3902935886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868c10e3f0 msgr2=0x7f868c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.147+0000 7f8683400640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868c10e3f0 0x7f868c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.147+0000 7f8683400640 1 -- 172.21.15.2:0/3902935886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f868c0745e0 msgr2=0x7f868c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.147+0000 7f8683400640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f868c0745e0 0x7f868c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.147+0000 7f8683400640 1 -- 172.21.15.2:0/3902935886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c19f670 con 0x7f868c074f00 2024-09-10T08:21:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.147+0000 7f8683400640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f868c074f00 0x7f868c19ac00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f867402a8f0 tx=0x7f8674030640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.147+0000 7f8689600640 1 -- 172.21.15.2:0/3902935886 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674005250 con 0x7f868c074f00 2024-09-10T08:21:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.147+0000 7f8689600640 1 -- 172.21.15.2:0/3902935886 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86740318e0 con 0x7f868c074f00 2024-09-10T08:21:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.147+0000 7f8692e00640 1 -- 172.21.15.2:0/3902935886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c19f900 con 0x7f868c074f00 2024-09-10T08:21:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.147+0000 7f8689600640 1 -- 172.21.15.2:0/3902935886 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674004c20 con 0x7f868c074f00 2024-09-10T08:21:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.148+0000 7f8692e00640 1 -- 172.21.15.2:0/3902935886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c1a7240 con 0x7f868c074f00 2024-09-10T08:21:42.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.151+0000 7f8695a44740 1 Processor -- start 2024-09-10T08:21:42.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.152+0000 7f8695a44740 1 -- start start 2024-09-10T08:21:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.152+0000 7f8695a44740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64eac0 0x55f34f64eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.152+0000 7f8695a44740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64f3e0 0x55f34f659760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.152+0000 7f8695a44740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f659ca0 0x55f34f65c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.152+0000 7f8695a44740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f34f5b5290 con 0x55f34f659ca0 2024-09-10T08:21:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.152+0000 7f8695a44740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f34f55cb40 con 0x55f34f64eac0 2024-09-10T08:21:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.152+0000 7f8695a44740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f34f5b1250 con 0x55f34f64f3e0 2024-09-10T08:21:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.152+0000 7f8689600640 1 -- 172.21.15.2:0/3902935886 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8674039040 con 0x7f868c074f00 2024-09-10T08:21:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.153+0000 7f8690e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64eac0 0x55f34f64eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.153+0000 7f8690e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64eac0 0x55f34f64eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52220/0 (socket says 172.21.15.2:52220) 2024-09-10T08:21:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.153+0000 7f8690e00640 1 -- 172.21.15.2:0/2557909015 learned_addr learned my addr 172.21.15.2:0/2557909015 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.153+0000 7f868be00640 1 --2- 172.21.15.2:0/2557909015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64f3e0 0x55f34f659760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.153+0000 7f8683400640 1 --2- 172.21.15.2:0/2557909015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f659ca0 0x55f34f65c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.153+0000 7f868be00640 1 -- 172.21.15.2:0/2557909015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64eac0 msgr2=0x55f34f64eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.153+0000 7f868be00640 1 --2- 172.21.15.2:0/2557909015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64eac0 0x55f34f64eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.153+0000 7f868be00640 1 -- 172.21.15.2:0/2557909015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f659ca0 msgr2=0x55f34f65c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.153+0000 7f868be00640 1 --2- 172.21.15.2:0/2557909015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f659ca0 0x55f34f65c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.153+0000 7f868be00640 1 -- 172.21.15.2:0/2557909015 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34f3266a0 con 0x55f34f64f3e0 2024-09-10T08:21:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.153+0000 7f8683400640 1 --2- 172.21.15.2:0/2557909015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f659ca0 0x55f34f65c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:21:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.153+0000 7f868be00640 1 --2- 172.21.15.2:0/2557909015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64f3e0 0x55f34f659760 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f867c009cf0 tx=0x7f867c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60711cee4281d1f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.154+0000 7f8681600640 1 -- 172.21.15.2:0/2557909015 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c005220 con 0x55f34f64f3e0 2024-09-10T08:21:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.154+0000 7f8681600640 1 -- 172.21.15.2:0/2557909015 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f867c004c80 con 0x55f34f64f3e0 2024-09-10T08:21:42.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.154+0000 7f8681600640 1 -- 172.21.15.2:0/2557909015 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c0198e0 con 0x55f34f64f3e0 2024-09-10T08:21:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.154+0000 7f8689600640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f86700a2c00 0x7f86700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.154+0000 7f8689600640 1 -- 172.21.15.2:0/3902935886 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f86740f0220 con 0x7f868c074f00 2024-09-10T08:21:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.155+0000 7f8695a44740 1 -- 172.21.15.2:0/2557909015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64f3e0 msgr2=0x55f34f659760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.155+0000 7f8695a44740 1 --2- 172.21.15.2:0/2557909015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64f3e0 0x55f34f659760 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f867c009cf0 tx=0x7f867c011810 comp rx=0 tx=0).stop 2024-09-10T08:21:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.155+0000 7f8690e00640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f86700a2c00 0x7f86700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.155+0000 7f8695a44740 1 -- 172.21.15.2:0/2557909015 shutdown_connections 2024-09-10T08:21:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.155+0000 7f8695a44740 1 --2- 172.21.15.2:0/2557909015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f659ca0 0x55f34f65c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.155+0000 7f8695a44740 1 --2- 172.21.15.2:0/2557909015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64f3e0 0x55f34f659760 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.155+0000 7f8695a44740 1 --2- 172.21.15.2:0/2557909015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64eac0 0x55f34f64eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.155+0000 7f8695a44740 1 -- 172.21.15.2:0/2557909015 >> 172.21.15.2:0/2557909015 conn(0x55f34f64b050 msgr2=0x55f34f64b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.155+0000 7f8695a44740 1 -- 172.21.15.2:0/2557909015 shutdown_connections 2024-09-10T08:21:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.155+0000 7f8695a44740 1 -- 172.21.15.2:0/2557909015 wait complete. 2024-09-10T08:21:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.156+0000 7f8690e00640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f86700a2c00 0x7f86700a50c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f868400e560 tx=0x7f8684010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.156+0000 7f8695a44740 1 -- start start 2024-09-10T08:21:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.156+0000 7f8695a44740 1 Processor -- start 2024-09-10T08:21:42.158 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:41 smithi155 ceph-mon[35405]: pgmap v793: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-10T08:21:42.158 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:41 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/150050193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:42.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.156+0000 7f8695a44740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f64eac0 0x55f34f659590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f8695a44740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64f3e0 0x55f34f656840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f8695a44740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f659ca0 0x55f34f656d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f8695a44740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f34f65c910 con 0x55f34f64eac0 2024-09-10T08:21:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f8695a44740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f34f56ffa0 con 0x55f34f659ca0 2024-09-10T08:21:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f8695a44740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f34f65ca90 con 0x55f34f64f3e0 2024-09-10T08:21:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f8695a44740 10 client.0 _ll_register_callbacks cb 0x55f34f2c3ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f868be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f64eac0 0x55f34f659590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f868be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f64eac0 0x55f34f659590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50770/0 (socket says 172.21.15.2:50770) 2024-09-10T08:21:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f868be00640 1 -- 172.21.15.2:0/2390141871 learned_addr learned my addr 172.21.15.2:0/2390141871 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f8683400640 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64f3e0 0x55f34f656840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f8690e00640 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f659ca0 0x55f34f656d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f868be00640 1 -- 172.21.15.2:0/2390141871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64f3e0 msgr2=0x55f34f656840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:42.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f868be00640 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64f3e0 0x55f34f656840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f868be00640 1 -- 172.21.15.2:0/2390141871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f659ca0 msgr2=0x55f34f656d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:42.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f868be00640 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f659ca0 0x55f34f656d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f868be00640 1 -- 172.21.15.2:0/2390141871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f34f657560 con 0x55f34f64eac0 2024-09-10T08:21:42.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.157+0000 7f8690e00640 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f659ca0 0x55f34f656d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:21:42.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.158+0000 7f868be00640 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f64eac0 0x55f34f659590 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f867c01b040 tx=0x7f867c004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59da3c732605f6a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:42.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.158+0000 7f867a000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:42.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.158+0000 7f867a000640 1 -- 172.21.15.2:0/2390141871 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c029070 con 0x55f34f64eac0 2024-09-10T08:21:42.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.158+0000 7f8695a44740 1 -- 172.21.15.2:0/2390141871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f867c004c80 con 0x55f34f64eac0 2024-09-10T08:21:42.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.158+0000 7f867a000640 1 -- 172.21.15.2:0/2390141871 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f867c019640 con 0x55f34f64eac0 2024-09-10T08:21:42.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.158+0000 7f867a000640 1 -- 172.21.15.2:0/2390141871 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f867c035050 con 0x55f34f64eac0 2024-09-10T08:21:42.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.158+0000 7f8695a44740 10 client.19263 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:42.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.158+0000 7f8695a44740 1 -- 172.21.15.2:0/2390141871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f34f668800 con 0x55f34f64eac0 2024-09-10T08:21:42.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.159+0000 7f867a000640 1 -- 172.21.15.2:0/2390141871 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f867c004c80 con 0x55f34f64eac0 2024-09-10T08:21:42.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.159+0000 7f867a000640 1 -- 172.21.15.2:0/2390141871 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f867c02e740 con 0x55f34f64eac0 2024-09-10T08:21:42.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.159+0000 7f8695a44740 10 client.19263 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:42.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.159+0000 7f8695a44740 10 client.19263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:42.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.159+0000 7f8695a44740 20 client.19263 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:42.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.159+0000 7f8695a44740 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f34f669b50 0x55f34f66bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.159+0000 7f8695a44740 4 client.19263 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:42.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.159+0000 7f8695a44740 1 -- 172.21.15.2:0/2390141871 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f34f5c2800 con 0x55f34f669b50 2024-09-10T08:21:42.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.160+0000 7f8683400640 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f34f669b50 0x55f34f66bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.160+0000 7f867a000640 10 client.19263 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:42.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.160+0000 7f8683400640 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f34f669b50 0x55f34f66bf40 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62924f90dd039e9f server_cookie=695077b91e87fabf in_seq=0 out_seq=0 2024-09-10T08:21:42.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.161+0000 7f867a000640 1 -- 172.21.15.2:0/2390141871 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f34f5c2800 con 0x55f34f669b50 2024-09-10T08:21:42.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.161+0000 7f867a000640 10 client.19263 handle_command_reply: tid=0 2024-09-10T08:21:42.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.161+0000 7f8695a44740 1 client.19263 shutdown 2024-09-10T08:21:42.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.161+0000 7f8695a44740 1 -- 172.21.15.2:0/2390141871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f64eac0 msgr2=0x55f34f659590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:42.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.161+0000 7f8695a44740 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f64eac0 0x55f34f659590 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f867c01b040 tx=0x7f867c004ba0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.162+0000 7f8695a44740 1 -- 172.21.15.2:0/2390141871 shutdown_connections 2024-09-10T08:21:42.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.162+0000 7f8695a44740 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f34f669b50 0x55f34f66bf40 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.162+0000 7f8695a44740 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f659ca0 0x55f34f656d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.162+0000 7f8695a44740 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64f3e0 0x55f34f656840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.162+0000 7f8695a44740 1 --2- 172.21.15.2:0/2390141871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f64eac0 0x55f34f659590 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.162+0000 7f8695a44740 1 -- 172.21.15.2:0/2390141871 >> 172.21.15.2:0/2390141871 conn(0x55f34f64b050 msgr2=0x55f34f650d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:42.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.162+0000 7f867a000640 0 client.19263 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:42.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.162+0000 7f8695a44740 1 -- 172.21.15.2:0/2390141871 shutdown_connections 2024-09-10T08:21:42.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.162+0000 7f8695a44740 1 -- 172.21.15.2:0/2390141871 wait complete. 2024-09-10T08:21:42.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.162+0000 7f8695a44740 20 client.19263 trim_cache size 0 max 16384 2024-09-10T08:21:42.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:41 smithi107 ceph-mon[35048]: pgmap v793: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-10T08:21:42.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:41 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/150050193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:42.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.181+0000 7f8695a44740 1 Processor -- start 2024-09-10T08:21:42.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.181+0000 7f8695a44740 1 -- start start 2024-09-10T08:21:42.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.182+0000 7f8695a44740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64eac0 0x55f34f656c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.182+0000 7f8695a44740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f64f3e0 0x55f34f6587b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.182+0000 7f8695a44740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f659ca0 0x55f34f671720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.182+0000 7f8695a44740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1b3890 con 0x55f34f64f3e0 2024-09-10T08:21:42.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.182+0000 7f8695a44740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1b1160 con 0x55f34f64eac0 2024-09-10T08:21:42.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.182+0000 7f8695a44740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1aefb0 con 0x55f34f659ca0 2024-09-10T08:21:42.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.182+0000 7f868be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64eac0 0x55f34f656c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.182+0000 7f8683400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f64f3e0 0x55f34f6587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.182+0000 7f8690e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f659ca0 0x55f34f671720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.182+0000 7f868be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64eac0 0x55f34f656c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52238/0 (socket says 172.21.15.2:52238) 2024-09-10T08:21:42.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.182+0000 7f868be00640 1 -- 172.21.15.2:0/466296016 learned_addr learned my addr 172.21.15.2:0/466296016 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:42.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.183+0000 7f868be00640 1 -- 172.21.15.2:0/466296016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f659ca0 msgr2=0x55f34f671720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:42.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.183+0000 7f868be00640 1 --2- 172.21.15.2:0/466296016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f659ca0 0x55f34f671720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.183+0000 7f868be00640 1 -- 172.21.15.2:0/466296016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f64f3e0 msgr2=0x55f34f6587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:42.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.183+0000 7f868be00640 1 --2- 172.21.15.2:0/466296016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f64f3e0 0x55f34f6587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.183+0000 7f868be00640 1 -- 172.21.15.2:0/466296016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34f64bd40 con 0x55f34f64eac0 2024-09-10T08:21:42.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.183+0000 7f868be00640 1 --2- 172.21.15.2:0/466296016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64eac0 0x55f34f656c60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f867c0059f0 tx=0x7f867c005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd580c2909aff262 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:42.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.183+0000 7f8680c00640 1 -- 172.21.15.2:0/466296016 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c029070 con 0x55f34f64eac0 2024-09-10T08:21:42.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.183+0000 7f8680c00640 1 -- 172.21.15.2:0/466296016 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f867c02e920 con 0x55f34f64eac0 2024-09-10T08:21:42.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.183+0000 7f8680c00640 1 -- 172.21.15.2:0/466296016 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c000e60 con 0x55f34f64eac0 2024-09-10T08:21:42.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.184+0000 7f8695a44740 1 -- 172.21.15.2:0/466296016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64eac0 msgr2=0x55f34f656c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:42.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.184+0000 7f8695a44740 1 --2- 172.21.15.2:0/466296016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64eac0 0x55f34f656c60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f867c0059f0 tx=0x7f867c005b90 comp rx=0 tx=0).stop 2024-09-10T08:21:42.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.184+0000 7f8695a44740 1 -- 172.21.15.2:0/466296016 shutdown_connections 2024-09-10T08:21:42.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.184+0000 7f8695a44740 1 --2- 172.21.15.2:0/466296016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f659ca0 0x55f34f671720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.184+0000 7f8695a44740 1 --2- 172.21.15.2:0/466296016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f64f3e0 0x55f34f6587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.184+0000 7f8695a44740 1 --2- 172.21.15.2:0/466296016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64eac0 0x55f34f656c60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.184+0000 7f8695a44740 1 -- 172.21.15.2:0/466296016 >> 172.21.15.2:0/466296016 conn(0x55f34f64b050 msgr2=0x55f34f6580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:42.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.185+0000 7f8695a44740 1 -- 172.21.15.2:0/466296016 shutdown_connections 2024-09-10T08:21:42.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.185+0000 7f8695a44740 1 -- 172.21.15.2:0/466296016 wait complete. 2024-09-10T08:21:42.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.185+0000 7f8695a44740 1 -- start start 2024-09-10T08:21:42.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.185+0000 7f8695a44740 1 Processor -- start 2024-09-10T08:21:42.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f8695a44740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64eac0 0x55f34f661fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f8695a44740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64f3e0 0x55f34f662520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f8695a44740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f659ca0 0x55f34f65d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f8695a44740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1af620 con 0x55f34f659ca0 2024-09-10T08:21:42.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f8695a44740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1b0a90 con 0x55f34f64f3e0 2024-09-10T08:21:42.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f8695a44740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1afcf0 con 0x55f34f64eac0 2024-09-10T08:21:42.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f8695a44740 10 client.0 _ll_register_callbacks cb 0x7f8684009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:42.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f8690e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f659ca0 0x55f34f65d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f8683400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64f3e0 0x55f34f662520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f868be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64eac0 0x55f34f661fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f8683400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64f3e0 0x55f34f662520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52244/0 (socket says 172.21.15.2:52244) 2024-09-10T08:21:42.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f8690e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f659ca0 0x55f34f65d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50796/0 (socket says 172.21.15.2:50796) 2024-09-10T08:21:42.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f8683400640 1 -- 172.21.15.2:0/4025046727 learned_addr learned my addr 172.21.15.2:0/4025046727 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:42.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.186+0000 7f868be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64eac0 0x55f34f661fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51920/0 (socket says 172.21.15.2:51920) 2024-09-10T08:21:42.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.187+0000 7f868be00640 1 -- 172.21.15.2:0/4025046727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64f3e0 msgr2=0x55f34f662520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:42.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.187+0000 7f868be00640 1 --2- 172.21.15.2:0/4025046727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64f3e0 0x55f34f662520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.187+0000 7f868be00640 1 -- 172.21.15.2:0/4025046727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f659ca0 msgr2=0x55f34f65d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:42.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.187+0000 7f868be00640 1 --2- 172.21.15.2:0/4025046727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f659ca0 0x55f34f65d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.187+0000 7f868be00640 1 -- 172.21.15.2:0/4025046727 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f34f65dc40 con 0x55f34f64eac0 2024-09-10T08:21:42.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.187+0000 7f868be00640 1 --2- 172.21.15.2:0/4025046727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64eac0 0x55f34f661fe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f867c02eb40 tx=0x7f867c005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f207dd627c93305f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:42.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.187+0000 7f867a000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:21:42.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.188+0000 7f867a000640 1 -- 172.21.15.2:0/4025046727 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c03e070 con 0x55f34f64eac0 2024-09-10T08:21:42.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.188+0000 7f8695a44740 1 -- 172.21.15.2:0/4025046727 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f867c02e920 con 0x55f34f64eac0 2024-09-10T08:21:42.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.188+0000 7f867a000640 1 -- 172.21.15.2:0/4025046727 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f867c02e640 con 0x55f34f64eac0 2024-09-10T08:21:42.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.188+0000 7f867a000640 1 -- 172.21.15.2:0/4025046727 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f867c04a050 con 0x55f34f64eac0 2024-09-10T08:21:42.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.188+0000 7f8695a44740 10 client.29221 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:42.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.188+0000 7f8695a44740 1 -- 172.21.15.2:0/4025046727 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f34f6d42b0 con 0x55f34f64eac0 2024-09-10T08:21:42.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.188+0000 7f867a000640 1 -- 172.21.15.2:0/4025046727 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f867c02e920 con 0x55f34f64eac0 2024-09-10T08:21:42.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.188+0000 7f867a000640 1 -- 172.21.15.2:0/4025046727 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f867c019430 con 0x55f34f64eac0 2024-09-10T08:21:42.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.188+0000 7f8695a44740 10 client.29221 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:42.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.189+0000 7f8695a44740 10 client.29221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:42.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.189+0000 7f8695a44740 20 client.29221 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:42.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.189+0000 7f8695a44740 1 --2- 172.21.15.2:0/4025046727 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f34f6d48b0 0x55f34f6d6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:42.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.189+0000 7f8695a44740 4 client.29221 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:21:42.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.189+0000 7f8695a44740 1 -- 172.21.15.2:0/4025046727 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f34f5bcd10 con 0x55f34f6d48b0 2024-09-10T08:21:42.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.189+0000 7f8683400640 1 --2- 172.21.15.2:0/4025046727 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f34f6d48b0 0x55f34f6d6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:42.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.189+0000 7f8683400640 1 --2- 172.21.15.2:0/4025046727 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f34f6d48b0 0x55f34f6d6ca0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=858df062bd0c3882 server_cookie=b172fd17a2c2f280 in_seq=0 out_seq=0 2024-09-10T08:21:42.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.189+0000 7f867a000640 10 client.29221 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:42.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.190+0000 7f867a000640 1 -- 172.21.15.2:0/4025046727 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f34f5bcd10 con 0x55f34f6d48b0 2024-09-10T08:21:42.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.190+0000 7f867a000640 10 client.29221 handle_command_reply: tid=0 2024-09-10T08:21:42.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.190+0000 7f8695a44740 1 client.29221 shutdown 2024-09-10T08:21:42.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.191+0000 7f8695a44740 1 -- 172.21.15.2:0/4025046727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64eac0 msgr2=0x55f34f661fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:42.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.191+0000 7f8695a44740 1 --2- 172.21.15.2:0/4025046727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64eac0 0x55f34f661fe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f867c02eb40 tx=0x7f867c005420 comp rx=0 tx=0).stop 2024-09-10T08:21:42.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.191+0000 7f8695a44740 1 -- 172.21.15.2:0/4025046727 shutdown_connections 2024-09-10T08:21:42.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.191+0000 7f8695a44740 1 --2- 172.21.15.2:0/4025046727 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f34f6d48b0 0x55f34f6d6ca0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.191+0000 7f867a000640 0 client.29221 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:42.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.191+0000 7f8695a44740 1 --2- 172.21.15.2:0/4025046727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f34f659ca0 0x55f34f65d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.191+0000 7f8695a44740 1 --2- 172.21.15.2:0/4025046727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f34f64f3e0 0x55f34f662520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.191+0000 7f8695a44740 1 --2- 172.21.15.2:0/4025046727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f64eac0 0x55f34f661fe0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.191+0000 7f8695a44740 1 -- 172.21.15.2:0/4025046727 >> 172.21.15.2:0/4025046727 conn(0x55f34f64b050 msgr2=0x55f34f6588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:42.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.191+0000 7f8695a44740 1 -- 172.21.15.2:0/4025046727 shutdown_connections 2024-09-10T08:21:42.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.191+0000 7f8695a44740 1 -- 172.21.15.2:0/4025046727 wait complete. 2024-09-10T08:21:42.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.192+0000 7f8695a44740 20 client.29221 trim_cache size 0 max 16384 2024-09-10T08:21:42.194 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:21:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:21:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "c902e2e6-2388-43f2-819e-2a863b12da9c", 2024-09-10T08:21:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:21:42.194 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:21:42.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.192+0000 7f8692e00640 1 -- 172.21.15.2:0/3902935886 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f86700a2c00 msgr2=0x7f86700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:42.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.192+0000 7f8692e00640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f86700a2c00 0x7f86700a50c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f868400e560 tx=0x7f8684010040 comp rx=0 tx=0).stop 2024-09-10T08:21:42.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.192+0000 7f8692e00640 1 -- 172.21.15.2:0/3902935886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f868c074f00 msgr2=0x7f868c19ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:42.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.192+0000 7f8692e00640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f868c074f00 0x7f868c19ac00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f867402a8f0 tx=0x7f8674030640 comp rx=0 tx=0).stop 2024-09-10T08:21:42.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.193+0000 7f8692e00640 1 -- 172.21.15.2:0/3902935886 shutdown_connections 2024-09-10T08:21:42.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.193+0000 7f8692e00640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f86700a2c00 0x7f86700a50c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.193+0000 7f8692e00640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868c10e3f0 0x7f868c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.193+0000 7f8692e00640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f868c074f00 0x7f868c19ac00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.193+0000 7f8692e00640 1 --2- 172.21.15.2:0/3902935886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f868c0745e0 0x7f868c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:42.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.193+0000 7f8692e00640 1 -- 172.21.15.2:0/3902935886 >> 172.21.15.2:0/3902935886 conn(0x7f868c0fd8b0 msgr2=0x7f868c0ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:42.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.193+0000 7f8692e00640 1 -- 172.21.15.2:0/3902935886 shutdown_connections 2024-09-10T08:21:42.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:42.193+0000 7f8692e00640 1 -- 172.21.15.2:0/3902935886 wait complete. 2024-09-10T08:21:42.498 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:43.125 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:21:43.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.742+0000 7f969f000640 1 -- 172.21.15.2:0/2598234908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698074510 msgr2=0x7f96980748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:43.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.742+0000 7f969f000640 1 --2- 172.21.15.2:0/2598234908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698074510 0x7f96980748f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9688009900 tx=0x7f968802a490 comp rx=0 tx=0).stop 2024-09-10T08:21:43.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.742+0000 7f969f000640 1 -- 172.21.15.2:0/2598234908 shutdown_connections 2024-09-10T08:21:43.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.742+0000 7f969f000640 1 --2- 172.21.15.2:0/2598234908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f969810e430 0x7f9698110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.742+0000 7f969f000640 1 --2- 172.21.15.2:0/2598234908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9698074e30 0x7f969810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.742+0000 7f969f000640 1 --2- 172.21.15.2:0/2598234908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698074510 0x7f96980748f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.742+0000 7f969f000640 1 -- 172.21.15.2:0/2598234908 >> 172.21.15.2:0/2598234908 conn(0x7f96980fd890 msgr2=0x7f96980ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:43.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.742+0000 7f969f000640 1 -- 172.21.15.2:0/2598234908 shutdown_connections 2024-09-10T08:21:43.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.742+0000 7f969f000640 1 -- 172.21.15.2:0/2598234908 wait complete. 2024-09-10T08:21:43.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.743+0000 7f969f000640 1 Processor -- start 2024-09-10T08:21:43.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.743+0000 7f969f000640 1 -- start start 2024-09-10T08:21:43.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.743+0000 7f969f000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698074510 0x7f969819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.743+0000 7f969f000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698074e30 0x7f969819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.743+0000 7f969f000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f969810e430 0x7f969819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.743+0000 7f969f000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9698114160 con 0x7f9698074e30 2024-09-10T08:21:43.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.743+0000 7f969f000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9698113fe0 con 0x7f9698074510 2024-09-10T08:21:43.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.743+0000 7f969f000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96981142e0 con 0x7f969810e430 2024-09-10T08:21:43.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.744+0000 7f9697e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698074510 0x7f969819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.744+0000 7f9697400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698074e30 0x7f969819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.744+0000 7f9697e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698074510 0x7f969819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52274/0 (socket says 172.21.15.2:52274) 2024-09-10T08:21:43.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.744+0000 7f9697e00640 1 -- 172.21.15.2:0/2344006530 learned_addr learned my addr 172.21.15.2:0/2344006530 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:43.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.744+0000 7f9697400640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698074e30 0x7f969819ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50806/0 (socket says 172.21.15.2:50806) 2024-09-10T08:21:43.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.744+0000 7f969d000640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f969810e430 0x7f969819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.744+0000 7f9697e00640 1 -- 172.21.15.2:0/2344006530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f969810e430 msgr2=0x7f969819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:43.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.744+0000 7f9697e00640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f969810e430 0x7f969819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.744+0000 7f9697e00640 1 -- 172.21.15.2:0/2344006530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698074e30 msgr2=0x7f969819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:43.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.744+0000 7f9697e00640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698074e30 0x7f969819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.744+0000 7f9697e00640 1 -- 172.21.15.2:0/2344006530 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969819f670 con 0x7f9698074510 2024-09-10T08:21:43.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.745+0000 7f9697e00640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698074510 0x7f969819a6c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9688006010 tx=0x7f96880042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:43.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.745+0000 7f9694c00640 1 -- 172.21.15.2:0/2344006530 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968803f030 con 0x7f9698074510 2024-09-10T08:21:43.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.745+0000 7f9694c00640 1 -- 172.21.15.2:0/2344006530 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9688003ce0 con 0x7f9698074510 2024-09-10T08:21:43.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.745+0000 7f9694c00640 1 -- 172.21.15.2:0/2344006530 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688035070 con 0x7f9698074510 2024-09-10T08:21:43.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.745+0000 7f969f000640 1 -- 172.21.15.2:0/2344006530 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969819f900 con 0x7f9698074510 2024-09-10T08:21:43.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.745+0000 7f969f000640 1 -- 172.21.15.2:0/2344006530 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f969819fd90 con 0x7f9698074510 2024-09-10T08:21:43.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.749+0000 7f96a1c5a740 1 Processor -- start 2024-09-10T08:21:43.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.749+0000 7f96a1c5a740 1 -- start start 2024-09-10T08:21:43.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.749+0000 7f9694c00640 1 -- 172.21.15.2:0/2344006530 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9688042030 con 0x7f9698074510 2024-09-10T08:21:43.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.749+0000 7f96a1c5a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853310 0x55fe468536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.749+0000 7f96a1c5a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853c30 0x55fe4685dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.749+0000 7f96a1c5a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 0x55fe468608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.749+0000 7f96a1c5a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fe467b9650 con 0x55fe4685e4f0 2024-09-10T08:21:43.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.749+0000 7f96a1c5a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fe46760f10 con 0x55fe46853c30 2024-09-10T08:21:43.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.749+0000 7f96a1c5a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fe467b5610 con 0x55fe46853310 2024-09-10T08:21:43.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.750+0000 7f9697e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853c30 0x55fe4685dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.750+0000 7f9697e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853c30 0x55fe4685dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52280/0 (socket says 172.21.15.2:52280) 2024-09-10T08:21:43.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.750+0000 7f9697e00640 1 -- 172.21.15.2:0/1750194596 learned_addr learned my addr 172.21.15.2:0/1750194596 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:43.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.750+0000 7f969d000640 1 --2- 172.21.15.2:0/1750194596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853310 0x55fe468536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.750+0000 7f9697400640 1 --2- 172.21.15.2:0/1750194596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 0x55fe468608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.751+0000 7f9697e00640 1 -- 172.21.15.2:0/1750194596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853310 msgr2=0x55fe468536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.751+0000 7f9697e00640 1 --2- 172.21.15.2:0/1750194596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853310 0x55fe468536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.751+0000 7f9697e00640 1 -- 172.21.15.2:0/1750194596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 msgr2=0x55fe468608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.751+0000 7f9697e00640 1 --2- 172.21.15.2:0/1750194596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 0x55fe468608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.751+0000 7f9697e00640 1 -- 172.21.15.2:0/1750194596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe464c8580 con 0x55fe46853c30 2024-09-10T08:21:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.751+0000 7f9697400640 1 --2- 172.21.15.2:0/1750194596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 0x55fe468608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:21:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.751+0000 7f9697e00640 1 --2- 172.21.15.2:0/1750194596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853c30 0x55fe4685dfb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f96880f7570 tx=0x7f96880f7660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec34cf52cd5ee9c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.752+0000 7f9694c00640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f967c0a2c80 0x7f967c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.752+0000 7f9694c00640 1 -- 172.21.15.2:0/2344006530 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f96880ef470 con 0x7f9698074510 2024-09-10T08:21:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.752+0000 7f969d000640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f967c0a2c80 0x7f967c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.753+0000 7f969d000640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f967c0a2c80 0x7f967c0a5140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f96980754f0 tx=0x7f9690009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:43.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.755+0000 7f968d600640 1 -- 172.21.15.2:0/1750194596 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96880b7bd0 con 0x55fe46853c30 2024-09-10T08:21:43.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.755+0000 7f968d600640 1 -- 172.21.15.2:0/1750194596 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96880bb930 con 0x55fe46853c30 2024-09-10T08:21:43.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.755+0000 7f968d600640 1 -- 172.21.15.2:0/1750194596 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96880044e0 con 0x55fe46853c30 2024-09-10T08:21:43.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.756+0000 7f96a1c5a740 1 -- 172.21.15.2:0/1750194596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853c30 msgr2=0x55fe4685dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:43.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.756+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/1750194596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853c30 0x55fe4685dfb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f96880f7570 tx=0x7f96880f7660 comp rx=0 tx=0).stop 2024-09-10T08:21:43.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.756+0000 7f96a1c5a740 1 -- 172.21.15.2:0/1750194596 shutdown_connections 2024-09-10T08:21:43.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.756+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/1750194596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 0x55fe468608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.756+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/1750194596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853c30 0x55fe4685dfb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.756+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/1750194596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853310 0x55fe468536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.756+0000 7f96a1c5a740 1 -- 172.21.15.2:0/1750194596 >> 172.21.15.2:0/1750194596 conn(0x55fe4684f8a0 msgr2=0x55fe4684fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:43.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.756+0000 7f96a1c5a740 1 -- 172.21.15.2:0/1750194596 shutdown_connections 2024-09-10T08:21:43.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.756+0000 7f96a1c5a740 1 -- 172.21.15.2:0/1750194596 wait complete. 2024-09-10T08:21:43.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.757+0000 7f96a1c5a740 1 -- start start 2024-09-10T08:21:43.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.757+0000 7f96a1c5a740 1 Processor -- start 2024-09-10T08:21:43.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.758+0000 7f96a1c5a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853310 0x55fe4685b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.758+0000 7f96a1c5a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853c30 0x55fe4685b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.758+0000 7f96a1c5a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 0x55fe4685be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.758+0000 7f96a1c5a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fe46860e80 con 0x55fe4685e4f0 2024-09-10T08:21:43.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.758+0000 7f96a1c5a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fe46774350 con 0x55fe46853310 2024-09-10T08:21:43.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.758+0000 7f96a1c5a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fe46861000 con 0x55fe46853c30 2024-09-10T08:21:43.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.758+0000 7f96a1c5a740 10 client.0 _ll_register_callbacks cb 0x55fe46797290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.758+0000 7f9697400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853c30 0x55fe4685b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.758+0000 7f969d000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 0x55fe4685be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.758+0000 7f9697400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853c30 0x55fe4685b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51944/0 (socket says 172.21.15.2:51944) 2024-09-10T08:21:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.758+0000 7f9697400640 1 -- 172.21.15.2:0/3820573384 learned_addr learned my addr 172.21.15.2:0/3820573384 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.759+0000 7f9697e00640 1 --2- 172.21.15.2:0/3820573384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853310 0x55fe4685b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.759+0000 7f9697400640 1 -- 172.21.15.2:0/3820573384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853310 msgr2=0x55fe4685b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.759+0000 7f9697400640 1 --2- 172.21.15.2:0/3820573384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853310 0x55fe4685b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.759+0000 7f9697400640 1 -- 172.21.15.2:0/3820573384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 msgr2=0x55fe4685be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:43.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.759+0000 7f9697400640 1 --2- 172.21.15.2:0/3820573384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 0x55fe4685be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.759+0000 7f9697400640 1 -- 172.21.15.2:0/3820573384 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe46592960 con 0x55fe46853c30 2024-09-10T08:21:43.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.759+0000 7f9697400640 1 --2- 172.21.15.2:0/3820573384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853c30 0x55fe4685b900 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9684013710 tx=0x7f9684013b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cf95e5f9b60d789 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:43.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.759+0000 7f9682000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:21:43.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.760+0000 7f9682000640 1 -- 172.21.15.2:0/3820573384 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684004460 con 0x55fe46853c30 2024-09-10T08:21:43.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.760+0000 7f96a1c5a740 1 -- 172.21.15.2:0/3820573384 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96880bb930 con 0x55fe46853c30 2024-09-10T08:21:43.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.760+0000 7f9682000640 1 -- 172.21.15.2:0/3820573384 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f96840119c0 con 0x55fe46853c30 2024-09-10T08:21:43.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.760+0000 7f9682000640 1 -- 172.21.15.2:0/3820573384 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9684011bc0 con 0x55fe46853c30 2024-09-10T08:21:43.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.760+0000 7f96a1c5a740 10 client.29233 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:43.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.760+0000 7f96a1c5a740 1 -- 172.21.15.2:0/3820573384 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fe4686ca10 con 0x55fe46853c30 2024-09-10T08:21:43.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.761+0000 7f9682000640 1 -- 172.21.15.2:0/3820573384 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96880bb930 con 0x55fe46853c30 2024-09-10T08:21:43.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.761+0000 7f9682000640 1 -- 172.21.15.2:0/3820573384 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f968401a070 con 0x55fe46853c30 2024-09-10T08:21:43.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.761+0000 7f96a1c5a740 10 client.29233 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:43.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.761+0000 7f96a1c5a740 10 client.29233 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:21:43.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.761+0000 7f96a1c5a740 20 client.29233 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:43.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.761+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/3820573384 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fe4686dec0 0x55fe468702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.761+0000 7f96a1c5a740 4 client.29233 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:43.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.761+0000 7f96a1c5a740 1 -- 172.21.15.2:0/3820573384 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe467c6bc0 con 0x55fe4686dec0 2024-09-10T08:21:43.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.762+0000 7f9697e00640 1 --2- 172.21.15.2:0/3820573384 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fe4686dec0 0x55fe468702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.762+0000 7f9697e00640 1 --2- 172.21.15.2:0/3820573384 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fe4686dec0 0x55fe468702e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e756f92660a98be server_cookie=2e20b358913ad3b2 in_seq=0 out_seq=0 2024-09-10T08:21:43.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.762+0000 7f9682000640 10 client.29233 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:43.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.763+0000 7f9682000640 1 -- 172.21.15.2:0/3820573384 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe467c6bc0 con 0x55fe4686dec0 2024-09-10T08:21:43.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.763+0000 7f9682000640 10 client.29233 handle_command_reply: tid=0 2024-09-10T08:21:43.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.763+0000 7f96a1c5a740 1 client.29233 shutdown 2024-09-10T08:21:43.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.763+0000 7f96a1c5a740 1 -- 172.21.15.2:0/3820573384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853c30 msgr2=0x55fe4685b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:43.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.763+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/3820573384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853c30 0x55fe4685b900 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9684013710 tx=0x7f9684013b10 comp rx=0 tx=0).stop 2024-09-10T08:21:43.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.763+0000 7f96a1c5a740 1 -- 172.21.15.2:0/3820573384 shutdown_connections 2024-09-10T08:21:43.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.763+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/3820573384 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fe4686dec0 0x55fe468702e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.763+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/3820573384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 0x55fe4685be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.763+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/3820573384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853c30 0x55fe4685b900 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.763+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/3820573384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853310 0x55fe4685b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.763+0000 7f9682000640 0 client.29233 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:43.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.763+0000 7f96a1c5a740 1 -- 172.21.15.2:0/3820573384 >> 172.21.15.2:0/3820573384 conn(0x55fe4684f8a0 msgr2=0x55fe4684eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:43.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.764+0000 7f96a1c5a740 1 -- 172.21.15.2:0/3820573384 shutdown_connections 2024-09-10T08:21:43.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.764+0000 7f96a1c5a740 1 -- 172.21.15.2:0/3820573384 wait complete. 2024-09-10T08:21:43.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.764+0000 7f96a1c5a740 20 client.29233 trim_cache size 0 max 16384 2024-09-10T08:21:43.783 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:43.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.785+0000 7f96a1c5a740 1 Processor -- start 2024-09-10T08:21:43.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.785+0000 7f96a1c5a740 1 -- start start 2024-09-10T08:21:43.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.785+0000 7f96a1c5a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853310 0x55fe4685a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.785+0000 7f96a1c5a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853c30 0x55fe46875b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.785+0000 7f96a1c5a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 0x55fe468760b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.785+0000 7f96a1c5a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96981b3980 con 0x55fe4685e4f0 2024-09-10T08:21:43.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.785+0000 7f96a1c5a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96981b1250 con 0x55fe46853310 2024-09-10T08:21:43.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.785+0000 7f96a1c5a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96981af0a0 con 0x55fe46853c30 2024-09-10T08:21:43.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.786+0000 7f9697e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853310 0x55fe4685a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.786+0000 7f9697400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853c30 0x55fe46875b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.786+0000 7f969d000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 0x55fe468760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.786+0000 7f9697e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853310 0x55fe4685a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52296/0 (socket says 172.21.15.2:52296) 2024-09-10T08:21:43.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.786+0000 7f9697e00640 1 -- 172.21.15.2:0/3276718712 learned_addr learned my addr 172.21.15.2:0/3276718712 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:43.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.786+0000 7f9697e00640 1 -- 172.21.15.2:0/3276718712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853c30 msgr2=0x55fe46875b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:43.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.786+0000 7f9697e00640 1 --2- 172.21.15.2:0/3276718712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853c30 0x55fe46875b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.786+0000 7f9697e00640 1 -- 172.21.15.2:0/3276718712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 msgr2=0x55fe468760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:43.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.786+0000 7f9697e00640 1 --2- 172.21.15.2:0/3276718712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 0x55fe468760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.786+0000 7f9697e00640 1 -- 172.21.15.2:0/3276718712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe46870940 con 0x55fe46853310 2024-09-10T08:21:43.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.786+0000 7f9697e00640 1 --2- 172.21.15.2:0/3276718712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853310 0x55fe4685a300 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f96880b8300 tx=0x7f968802a490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84dfaaccdd105873 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:43.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.787+0000 7f968cc00640 1 -- 172.21.15.2:0/3276718712 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688049040 con 0x55fe46853310 2024-09-10T08:21:43.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.787+0000 7f968cc00640 1 -- 172.21.15.2:0/3276718712 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96880f7e90 con 0x55fe46853310 2024-09-10T08:21:43.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.787+0000 7f968cc00640 1 -- 172.21.15.2:0/3276718712 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96880bb290 con 0x55fe46853310 2024-09-10T08:21:43.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.787+0000 7f96a1c5a740 1 -- 172.21.15.2:0/3276718712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853310 msgr2=0x55fe4685a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:43.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.787+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/3276718712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853310 0x55fe4685a300 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f96880b8300 tx=0x7f968802a490 comp rx=0 tx=0).stop 2024-09-10T08:21:43.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.787+0000 7f96a1c5a740 1 -- 172.21.15.2:0/3276718712 shutdown_connections 2024-09-10T08:21:43.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.787+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/3276718712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe4685e4f0 0x55fe468760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.787+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/3276718712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe46853c30 0x55fe46875b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.787+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/3276718712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853310 0x55fe4685a300 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.787+0000 7f96a1c5a740 1 -- 172.21.15.2:0/3276718712 >> 172.21.15.2:0/3276718712 conn(0x55fe4684f8a0 msgr2=0x55fe4685d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:43.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.788+0000 7f96a1c5a740 1 -- 172.21.15.2:0/3276718712 shutdown_connections 2024-09-10T08:21:43.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.788+0000 7f96a1c5a740 1 -- 172.21.15.2:0/3276718712 wait complete. 2024-09-10T08:21:43.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.788+0000 7f96a1c5a740 1 -- start start 2024-09-10T08:21:43.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.788+0000 7f96a1c5a740 1 Processor -- start 2024-09-10T08:21:43.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.789+0000 7f96a1c5a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe46853310 0x55fe46861fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.789+0000 7f96a1c5a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853c30 0x55fe46862500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.789+0000 7f96a1c5a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe4685e4f0 0x55fe46862a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.789+0000 7f96a1c5a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96981af710 con 0x55fe46853310 2024-09-10T08:21:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.789+0000 7f96a1c5a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96981b0b80 con 0x55fe46853c30 2024-09-10T08:21:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.789+0000 7f96a1c5a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96981afde0 con 0x55fe4685e4f0 2024-09-10T08:21:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.789+0000 7f96a1c5a740 10 client.0 _ll_register_callbacks cb 0x7f9684009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.789+0000 7f9697e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe46853310 0x55fe46861fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.789+0000 7f969d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe4685e4f0 0x55fe46862a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.789+0000 7f9697400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853c30 0x55fe46862500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.789+0000 7f969d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe4685e4f0 0x55fe46862a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51966/0 (socket says 172.21.15.2:51966) 2024-09-10T08:21:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.789+0000 7f9697e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe46853310 0x55fe46861fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50834/0 (socket says 172.21.15.2:50834) 2024-09-10T08:21:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.789+0000 7f969d000640 1 -- 172.21.15.2:0/1218055018 learned_addr learned my addr 172.21.15.2:0/1218055018 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.790+0000 7f9697400640 1 -- 172.21.15.2:0/1218055018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe4685e4f0 msgr2=0x55fe46862a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.790+0000 7f9697400640 1 --2- 172.21.15.2:0/1218055018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe4685e4f0 0x55fe46862a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.790+0000 7f9697400640 1 -- 172.21.15.2:0/1218055018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe46853310 msgr2=0x55fe46861fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.790+0000 7f9697400640 1 --2- 172.21.15.2:0/1218055018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe46853310 0x55fe46861fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.790+0000 7f9697400640 1 -- 172.21.15.2:0/1218055018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe468d7ce0 con 0x55fe46853c30 2024-09-10T08:21:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.790+0000 7f9697400640 1 --2- 172.21.15.2:0/1218055018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853c30 0x55fe46862500 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55fe4685c970 tx=0x7f96840118f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cbde82b9cb516dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.790+0000 7f9682000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:21:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.790+0000 7f9682000640 1 -- 172.21.15.2:0/1218055018 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968402f020 con 0x55fe46853c30 2024-09-10T08:21:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.790+0000 7f96a1c5a740 1 -- 172.21.15.2:0/1218055018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96880f7e90 con 0x55fe46853c30 2024-09-10T08:21:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.791+0000 7f9682000640 1 -- 172.21.15.2:0/1218055018 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f968401c9c0 con 0x55fe46853c30 2024-09-10T08:21:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.791+0000 7f9682000640 1 -- 172.21.15.2:0/1218055018 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9684005c50 con 0x55fe46853c30 2024-09-10T08:21:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.791+0000 7f96a1c5a740 10 client.29309 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.791+0000 7f96a1c5a740 1 -- 172.21.15.2:0/1218055018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fe468d8fe0 con 0x55fe46853c30 2024-09-10T08:21:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.791+0000 7f9682000640 1 -- 172.21.15.2:0/1218055018 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96880f7e90 con 0x55fe46853c30 2024-09-10T08:21:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.791+0000 7f9682000640 1 -- 172.21.15.2:0/1218055018 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f968401a040 con 0x55fe46853c30 2024-09-10T08:21:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.791+0000 7f96a1c5a740 10 client.29309 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.791+0000 7f96a1c5a740 10 client.29309 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:21:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.791+0000 7f96a1c5a740 20 client.29309 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.791+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/1218055018 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fe468d97b0 0x55fe468dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.791+0000 7f96a1c5a740 4 client.29309 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4958be9", "release": true, "await": true}] 2024-09-10T08:21:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.791+0000 7f96a1c5a740 1 -- 172.21.15.2:0/1218055018 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4958be9", "release": true, "await": true}) -- 0x55fe467c10d0 con 0x55fe468d97b0 2024-09-10T08:21:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.792+0000 7f9697e00640 1 --2- 172.21.15.2:0/1218055018 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fe468d97b0 0x55fe468dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.792+0000 7f9697e00640 1 --2- 172.21.15.2:0/1218055018 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fe468d97b0 0x55fe468dbbd0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642a9d7342e50b3b server_cookie=ba0d6065a034db85 in_seq=0 out_seq=0 2024-09-10T08:21:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.792+0000 7f9682000640 10 client.29309 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.794+0000 7f9682000640 1 -- 172.21.15.2:0/1218055018 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55fe467c10d0 con 0x55fe468d97b0 2024-09-10T08:21:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.794+0000 7f9682000640 10 client.29309 handle_command_reply: tid=0 2024-09-10T08:21:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.795+0000 7f96a1c5a740 1 client.29309 shutdown 2024-09-10T08:21:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.795+0000 7f96a1c5a740 1 -- 172.21.15.2:0/1218055018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853c30 msgr2=0x55fe46862500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.795+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/1218055018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853c30 0x55fe46862500 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55fe4685c970 tx=0x7f96840118f0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.795+0000 7f96a1c5a740 1 -- 172.21.15.2:0/1218055018 shutdown_connections 2024-09-10T08:21:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.795+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/1218055018 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55fe468d97b0 0x55fe468dbbd0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.795+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/1218055018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe4685e4f0 0x55fe46862a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.795+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/1218055018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe46853c30 0x55fe46862500 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.795+0000 7f9682000640 0 client.29309 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.795+0000 7f96a1c5a740 1 --2- 172.21.15.2:0/1218055018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe46853310 0x55fe46861fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.795+0000 7f96a1c5a740 1 -- 172.21.15.2:0/1218055018 >> 172.21.15.2:0/1218055018 conn(0x55fe4684f8a0 msgr2=0x55fe4684e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.795+0000 7f96a1c5a740 1 -- 172.21.15.2:0/1218055018 shutdown_connections 2024-09-10T08:21:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.795+0000 7f96a1c5a740 1 -- 172.21.15.2:0/1218055018 wait complete. 2024-09-10T08:21:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.795+0000 7f96a1c5a740 20 client.29309 trim_cache size 0 max 16384 2024-09-10T08:21:43.796 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:21:43.796 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:21:43.796 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:21:43.796 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 48, 2024-09-10T08:21:43.796 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:21:43.796 INFO:teuthology.orchestra.run.smithi002.stdout: "4958be9": { 2024-09-10T08:21:43.796 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 48, 2024-09-10T08:21:43.797 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:21:43.797 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:21:43.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:21:43.797 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:21:43.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:21:43.797 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:21:43.797 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 141.5, 2024-09-10T08:21:43.797 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:21:43.797 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:21:43.797 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:21:43.797 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:21:43.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:21:43.797 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:21:43.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:43.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:43.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:43.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:43.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:43.798 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:21:43.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.796+0000 7f969f000640 1 -- 172.21.15.2:0/2344006530 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f967c0a2c80 msgr2=0x7f967c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:43.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.796+0000 7f969f000640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f967c0a2c80 0x7f967c0a5140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f96980754f0 tx=0x7f9690009600 comp rx=0 tx=0).stop 2024-09-10T08:21:43.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.796+0000 7f969f000640 1 -- 172.21.15.2:0/2344006530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698074510 msgr2=0x7f969819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:43.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.796+0000 7f969f000640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698074510 0x7f969819a6c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9688006010 tx=0x7f96880042f0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.796+0000 7f969f000640 1 -- 172.21.15.2:0/2344006530 shutdown_connections 2024-09-10T08:21:43.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.796+0000 7f969f000640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f967c0a2c80 0x7f967c0a5140 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.796+0000 7f969f000640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f969810e430 0x7f969819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.796+0000 7f969f000640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698074e30 0x7f969819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.796+0000 7f969f000640 1 --2- 172.21.15.2:0/2344006530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9698074510 0x7f969819a6c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:43.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.796+0000 7f969f000640 1 -- 172.21.15.2:0/2344006530 >> 172.21.15.2:0/2344006530 conn(0x7f96980fd890 msgr2=0x7f96980fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:43.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.796+0000 7f969f000640 1 -- 172.21.15.2:0/2344006530 shutdown_connections 2024-09-10T08:21:43.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:43.796+0000 7f969f000640 1 -- 172.21.15.2:0/2344006530 wait complete. 2024-09-10T08:21:44.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:43 smithi002 ceph-mon[30563]: pgmap v794: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-10T08:21:44.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:43 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:21:44.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:43 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:21:44.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:43 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:21:44.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:43 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:21:44.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:43 smithi155 ceph-mon[35405]: pgmap v794: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-10T08:21:44.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:43 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:21:44.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:43 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:21:44.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:43 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:21:44.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:43 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:21:44.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:43 smithi107 ceph-mon[35048]: pgmap v794: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-10T08:21:44.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:43 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:21:44.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:43 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:21:44.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:43 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:21:44.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:43 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:21:44.723 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4958be9', total seconds elapsed: 26.6 2024-09-10T08:21:44.723 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.5 2024-09-10T08:21:44.723 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:21:45.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:44 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:21:45.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:44 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:21:45.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:44 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:21:45.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.323+0000 7fb312e00640 1 -- 172.21.15.2:0/597630684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb30c1079d0 msgr2=0x7fb30c107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:45.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.323+0000 7fb312e00640 1 --2- 172.21.15.2:0/597630684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb30c1079d0 0x7fb30c107db0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb300009890 tx=0x7fb30002a370 comp rx=0 tx=0).stop 2024-09-10T08:21:45.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.323+0000 7fb312e00640 1 -- 172.21.15.2:0/597630684 shutdown_connections 2024-09-10T08:21:45.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.323+0000 7fb312e00640 1 --2- 172.21.15.2:0/597630684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb30c102370 0x7fb30c10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.323+0000 7fb312e00640 1 --2- 172.21.15.2:0/597630684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb30c1019d0 0x7fb30c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.323+0000 7fb312e00640 1 --2- 172.21.15.2:0/597630684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb30c1079d0 0x7fb30c107db0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.323+0000 7fb312e00640 1 -- 172.21.15.2:0/597630684 >> 172.21.15.2:0/597630684 conn(0x7fb30c0fd710 msgr2=0x7fb30c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:45.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.323+0000 7fb312e00640 1 -- 172.21.15.2:0/597630684 shutdown_connections 2024-09-10T08:21:45.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.323+0000 7fb312e00640 1 -- 172.21.15.2:0/597630684 wait complete. 2024-09-10T08:21:45.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.325+0000 7fb312e00640 1 Processor -- start 2024-09-10T08:21:45.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.325+0000 7fb312e00640 1 -- start start 2024-09-10T08:21:45.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.325+0000 7fb312e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb30c1019d0 0x7fb30c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.325+0000 7fb312e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb30c102370 0x7fb30c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.325+0000 7fb312e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb30c1079d0 0x7fb30c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.325+0000 7fb312e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c111dd0 con 0x7fb30c1079d0 2024-09-10T08:21:45.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.325+0000 7fb312e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c111c50 con 0x7fb30c102370 2024-09-10T08:21:45.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.325+0000 7fb312e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c111f50 con 0x7fb30c1019d0 2024-09-10T08:21:45.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.326+0000 7fb311000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb30c102370 0x7fb30c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.326+0000 7fb311000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb30c102370 0x7fb30c198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52304/0 (socket says 172.21.15.2:52304) 2024-09-10T08:21:45.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.326+0000 7fb312400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb30c1079d0 0x7fb30c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.326+0000 7fb311a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb30c1019d0 0x7fb30c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.326+0000 7fb311000640 1 -- 172.21.15.2:0/3677608588 learned_addr learned my addr 172.21.15.2:0/3677608588 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:45.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.326+0000 7fb311a00640 1 -- 172.21.15.2:0/3677608588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb30c102370 msgr2=0x7fb30c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:45.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.326+0000 7fb311a00640 1 --2- 172.21.15.2:0/3677608588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb30c102370 0x7fb30c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.326+0000 7fb311a00640 1 -- 172.21.15.2:0/3677608588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb30c1079d0 msgr2=0x7fb30c19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:45.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.326+0000 7fb311a00640 1 --2- 172.21.15.2:0/3677608588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb30c1079d0 0x7fb30c19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.326+0000 7fb311a00640 1 -- 172.21.15.2:0/3677608588 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c19d1e0 con 0x7fb30c1019d0 2024-09-10T08:21:45.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.326+0000 7fb311000640 1 --2- 172.21.15.2:0/3677608588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb30c102370 0x7fb30c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:45.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.327+0000 7fb311a00640 1 --2- 172.21.15.2:0/3677608588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb30c1019d0 0x7fb30c198230 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb300035040 tx=0x7fb3000304e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:45.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.327+0000 7fb30a000640 1 -- 172.21.15.2:0/3677608588 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300037040 con 0x7fb30c1019d0 2024-09-10T08:21:45.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.327+0000 7fb30a000640 1 -- 172.21.15.2:0/3677608588 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb300033d20 con 0x7fb30c1019d0 2024-09-10T08:21:45.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.327+0000 7fb312e00640 1 -- 172.21.15.2:0/3677608588 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c19d470 con 0x7fb30c1019d0 2024-09-10T08:21:45.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.327+0000 7fb30a000640 1 -- 172.21.15.2:0/3677608588 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300003850 con 0x7fb30c1019d0 2024-09-10T08:21:45.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.327+0000 7fb312e00640 1 -- 172.21.15.2:0/3677608588 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c1a4db0 con 0x7fb30c1019d0 2024-09-10T08:21:45.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.330+0000 7fb316deb740 1 Processor -- start 2024-09-10T08:21:45.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.330+0000 7fb316deb740 1 -- start start 2024-09-10T08:21:45.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.331+0000 7fb30a000640 1 -- 172.21.15.2:0/3677608588 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb300042020 con 0x7fb30c1019d0 2024-09-10T08:21:45.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.331+0000 7fb316deb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 0x5589ff6e8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.331+0000 7fb316deb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 0x5589ff6f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.331+0000 7fb316deb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6f7340 0x5589ff6f9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.331+0000 7fb316deb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5589ff64e3e0 con 0x5589ff6e7c50 2024-09-10T08:21:45.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.331+0000 7fb316deb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5589ff5f5d80 con 0x5589ff6f7340 2024-09-10T08:21:45.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.331+0000 7fb316deb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5589ff64a3a0 con 0x5589ff6e8570 2024-09-10T08:21:45.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.331+0000 7fb311a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 0x5589ff6f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.331+0000 7fb311000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6f7340 0x5589ff6f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.331+0000 7fb311000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6f7340 0x5589ff6f9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52312/0 (socket says 172.21.15.2:52312) 2024-09-10T08:21:45.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.331+0000 7fb311000640 1 -- 172.21.15.2:0/1448810179 learned_addr learned my addr 172.21.15.2:0/1448810179 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:45.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.331+0000 7fb311a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 0x5589ff6f6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51982/0 (socket says 172.21.15.2:51982) 2024-09-10T08:21:45.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.332+0000 7fb312400640 1 --2- 172.21.15.2:0/1448810179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 0x5589ff6e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.332+0000 7fb311000640 1 -- 172.21.15.2:0/1448810179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 msgr2=0x5589ff6f6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:45.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.332+0000 7fb311000640 1 --2- 172.21.15.2:0/1448810179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 0x5589ff6f6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.332+0000 7fb311000640 1 -- 172.21.15.2:0/1448810179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 msgr2=0x5589ff6e8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:45.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.332+0000 7fb311000640 1 --2- 172.21.15.2:0/1448810179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 0x5589ff6e8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.332+0000 7fb311000640 1 -- 172.21.15.2:0/1448810179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ff614ae0 con 0x5589ff6f7340 2024-09-10T08:21:45.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.332+0000 7fb311a00640 1 --2- 172.21.15.2:0/1448810179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 0x5589ff6f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:21:45.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.333+0000 7fb30a000640 1 --2- 172.21.15.2:0/3677608588 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb2f40a2b80 0x7fb2f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.333+0000 7fb311000640 1 --2- 172.21.15.2:0/1448810179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6f7340 0x5589ff6f9730 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80115f0 tx=0x7fb2f8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d9e2923fec7bda0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:45.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.333+0000 7fb30a000640 1 -- 172.21.15.2:0/3677608588 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb3000eea80 con 0x7fb30c1019d0 2024-09-10T08:21:45.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.333+0000 7fb2fea00640 1 -- 172.21.15.2:0/1448810179 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f801e070 con 0x5589ff6f7340 2024-09-10T08:21:45.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.333+0000 7fb312400640 1 --2- 172.21.15.2:0/3677608588 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb2f40a2b80 0x7fb2f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.333+0000 7fb2fea00640 1 -- 172.21.15.2:0/1448810179 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2f800f640 con 0x5589ff6f7340 2024-09-10T08:21:45.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.333+0000 7fb2fea00640 1 -- 172.21.15.2:0/1448810179 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8003850 con 0x5589ff6f7340 2024-09-10T08:21:45.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.334+0000 7fb312400640 1 --2- 172.21.15.2:0/3677608588 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb2f40a2b80 0x7fb2f40a5040 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb304005fd0 tx=0x7fb304009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:45.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.334+0000 7fb316deb740 1 -- 172.21.15.2:0/1448810179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6f7340 msgr2=0x5589ff6f9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:45.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.334+0000 7fb316deb740 1 --2- 172.21.15.2:0/1448810179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6f7340 0x5589ff6f9730 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80115f0 tx=0x7fb2f8009f90 comp rx=0 tx=0).stop 2024-09-10T08:21:45.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.335+0000 7fb316deb740 1 -- 172.21.15.2:0/1448810179 shutdown_connections 2024-09-10T08:21:45.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.335+0000 7fb316deb740 1 --2- 172.21.15.2:0/1448810179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6f7340 0x5589ff6f9730 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.335+0000 7fb316deb740 1 --2- 172.21.15.2:0/1448810179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 0x5589ff6f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.335+0000 7fb316deb740 1 --2- 172.21.15.2:0/1448810179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 0x5589ff6e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.335+0000 7fb316deb740 1 -- 172.21.15.2:0/1448810179 >> 172.21.15.2:0/1448810179 conn(0x5589ff6e42e0 msgr2=0x5589ff6e46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:45.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.335+0000 7fb316deb740 1 -- 172.21.15.2:0/1448810179 shutdown_connections 2024-09-10T08:21:45.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.335+0000 7fb316deb740 1 -- 172.21.15.2:0/1448810179 wait complete. 2024-09-10T08:21:45.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.336+0000 7fb316deb740 1 -- start start 2024-09-10T08:21:45.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.336+0000 7fb316deb740 1 Processor -- start 2024-09-10T08:21:45.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.337+0000 7fb316deb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 0x5589ff6f4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.337+0000 7fb316deb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 0x5589ff6f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.337+0000 7fb316deb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6f7340 0x5589ff6f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.337+0000 7fb316deb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5589ff6fa010 con 0x5589ff6e7c50 2024-09-10T08:21:45.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.337+0000 7fb316deb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5589ff6090f0 con 0x5589ff6e8570 2024-09-10T08:21:45.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.337+0000 7fb316deb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5589ff6fa190 con 0x5589ff6f7340 2024-09-10T08:21:45.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.337+0000 7fb316deb740 10 client.0 _ll_register_callbacks cb 0x5589ff5fa9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:45.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.337+0000 7fb311000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 0x5589ff6f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.337+0000 7fb311a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 0x5589ff6f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.337+0000 7fb312400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6f7340 0x5589ff6f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.337+0000 7fb311000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 0x5589ff6f47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52324/0 (socket says 172.21.15.2:52324) 2024-09-10T08:21:45.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.337+0000 7fb311000640 1 -- 172.21.15.2:0/3942614197 learned_addr learned my addr 172.21.15.2:0/3942614197 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:45.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.338+0000 7fb311000640 1 -- 172.21.15.2:0/3942614197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6f7340 msgr2=0x5589ff6f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:45.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.338+0000 7fb311000640 1 --2- 172.21.15.2:0/3942614197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6f7340 0x5589ff6f4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.338+0000 7fb311000640 1 -- 172.21.15.2:0/3942614197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 msgr2=0x5589ff6f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:45.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.338+0000 7fb311000640 1 --2- 172.21.15.2:0/3942614197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 0x5589ff6f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.338+0000 7fb311000640 1 -- 172.21.15.2:0/3942614197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ff3bf3c0 con 0x5589ff6e8570 2024-09-10T08:21:45.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.338+0000 7fb311000640 1 --2- 172.21.15.2:0/3942614197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 0x5589ff6f47d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80058c0 tx=0x7fb2f8005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e0bd3acfcbd79f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:45.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.338+0000 7fb2f3400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:21:45.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.339+0000 7fb2f3400640 1 -- 172.21.15.2:0/3942614197 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f801e030 con 0x5589ff6e8570 2024-09-10T08:21:45.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.339+0000 7fb316deb740 1 -- 172.21.15.2:0/3942614197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2f800f640 con 0x5589ff6e8570 2024-09-10T08:21:45.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.339+0000 7fb2f3400640 1 -- 172.21.15.2:0/3942614197 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb2f8005500 con 0x5589ff6e8570 2024-09-10T08:21:45.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.339+0000 7fb2f3400640 1 -- 172.21.15.2:0/3942614197 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb2f8031050 con 0x5589ff6e8570 2024-09-10T08:21:45.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.339+0000 7fb316deb740 10 client.29321 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:45.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.339+0000 7fb316deb740 1 -- 172.21.15.2:0/3942614197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589ff6ec250 con 0x5589ff6e8570 2024-09-10T08:21:45.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.339+0000 7fb2f3400640 1 -- 172.21.15.2:0/3942614197 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2f800f640 con 0x5589ff6e8570 2024-09-10T08:21:45.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.340+0000 7fb2f3400640 1 -- 172.21.15.2:0/3942614197 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb2f8019730 con 0x5589ff6e8570 2024-09-10T08:21:45.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.340+0000 7fb316deb740 10 client.29321 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:45.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.340+0000 7fb316deb740 10 client.29321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:45.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.340+0000 7fb316deb740 20 client.29321 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:45.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.340+0000 7fb316deb740 1 --2- 172.21.15.2:0/3942614197 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5589ff6ed6b0 0x5589ff6edb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.340+0000 7fb316deb740 4 client.29321 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:45.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.340+0000 7fb316deb740 1 -- 172.21.15.2:0/3942614197 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589ff65b950 con 0x5589ff6ed6b0 2024-09-10T08:21:45.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.340+0000 7fb311a00640 1 --2- 172.21.15.2:0/3942614197 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5589ff6ed6b0 0x5589ff6edb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.341+0000 7fb311a00640 1 --2- 172.21.15.2:0/3942614197 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5589ff6ed6b0 0x5589ff6edb30 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2488a12479f6332 server_cookie=ca201ae15b0c8981 in_seq=0 out_seq=0 2024-09-10T08:21:45.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.341+0000 7fb2f3400640 10 client.29321 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:45.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.342+0000 7fb2f3400640 1 -- 172.21.15.2:0/3942614197 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589ff65b950 con 0x5589ff6ed6b0 2024-09-10T08:21:45.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.342+0000 7fb2f3400640 10 client.29321 handle_command_reply: tid=0 2024-09-10T08:21:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.342+0000 7fb316deb740 1 client.29321 shutdown 2024-09-10T08:21:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.342+0000 7fb316deb740 1 -- 172.21.15.2:0/3942614197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 msgr2=0x5589ff6f47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.342+0000 7fb316deb740 1 --2- 172.21.15.2:0/3942614197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 0x5589ff6f47d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80058c0 tx=0x7fb2f8005040 comp rx=0 tx=0).stop 2024-09-10T08:21:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.342+0000 7fb316deb740 1 -- 172.21.15.2:0/3942614197 shutdown_connections 2024-09-10T08:21:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.342+0000 7fb316deb740 1 --2- 172.21.15.2:0/3942614197 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5589ff6ed6b0 0x5589ff6edb30 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.342+0000 7fb316deb740 1 --2- 172.21.15.2:0/3942614197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6f7340 0x5589ff6f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.342+0000 7fb316deb740 1 --2- 172.21.15.2:0/3942614197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 0x5589ff6f47d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.342+0000 7fb2f3400640 0 client.29321 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.342+0000 7fb316deb740 1 --2- 172.21.15.2:0/3942614197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 0x5589ff6f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.342+0000 7fb316deb740 1 -- 172.21.15.2:0/3942614197 >> 172.21.15.2:0/3942614197 conn(0x5589ff6e42e0 msgr2=0x5589ff6ea040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.343+0000 7fb316deb740 1 -- 172.21.15.2:0/3942614197 shutdown_connections 2024-09-10T08:21:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.343+0000 7fb316deb740 1 -- 172.21.15.2:0/3942614197 wait complete. 2024-09-10T08:21:45.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.343+0000 7fb316deb740 20 client.29321 trim_cache size 0 max 16384 2024-09-10T08:21:45.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.370+0000 7fb316deb740 1 Processor -- start 2024-09-10T08:21:45.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.370+0000 7fb316deb740 1 -- start start 2024-09-10T08:21:45.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb316deb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e7c50 0x5589ff6f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb316deb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 0x5589ff713d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb316deb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6f7340 0x5589ff714280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb316deb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1af280 con 0x5589ff6f7340 2024-09-10T08:21:45.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb316deb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1acb50 con 0x5589ff6e8570 2024-09-10T08:21:45.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb316deb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1aa9a0 con 0x5589ff6e7c50 2024-09-10T08:21:45.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb311000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 0x5589ff713d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb311000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 0x5589ff713d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52332/0 (socket says 172.21.15.2:52332) 2024-09-10T08:21:45.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb311000640 1 -- 172.21.15.2:0/2750534367 learned_addr learned my addr 172.21.15.2:0/2750534367 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:45.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb312400640 1 --2- 172.21.15.2:0/2750534367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6f7340 0x5589ff714280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb311a00640 1 --2- 172.21.15.2:0/2750534367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e7c50 0x5589ff6f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb311000640 1 -- 172.21.15.2:0/2750534367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e7c50 msgr2=0x5589ff6f65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:45.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb311000640 1 --2- 172.21.15.2:0/2750534367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e7c50 0x5589ff6f65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb311000640 1 -- 172.21.15.2:0/2750534367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6f7340 msgr2=0x5589ff714280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:45.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.371+0000 7fb311000640 1 --2- 172.21.15.2:0/2750534367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6f7340 0x5589ff714280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.372+0000 7fb311000640 1 -- 172.21.15.2:0/2750534367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ff6e5470 con 0x5589ff6e8570 2024-09-10T08:21:45.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.372+0000 7fb311000640 1 --2- 172.21.15.2:0/2750534367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 0x5589ff713d40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80092a0 tx=0x7fb2f8012f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=572974d668c6e982 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:45.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.372+0000 7fb2fe000640 1 -- 172.21.15.2:0/2750534367 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f801e050 con 0x5589ff6e8570 2024-09-10T08:21:45.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.372+0000 7fb2fe000640 1 -- 172.21.15.2:0/2750534367 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2f803a070 con 0x5589ff6e8570 2024-09-10T08:21:45.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.372+0000 7fb2fe000640 1 -- 172.21.15.2:0/2750534367 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80235e0 con 0x5589ff6e8570 2024-09-10T08:21:45.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.373+0000 7fb316deb740 1 -- 172.21.15.2:0/2750534367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 msgr2=0x5589ff713d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:45.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.373+0000 7fb316deb740 1 --2- 172.21.15.2:0/2750534367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 0x5589ff713d40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80092a0 tx=0x7fb2f8012f50 comp rx=0 tx=0).stop 2024-09-10T08:21:45.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.373+0000 7fb316deb740 1 -- 172.21.15.2:0/2750534367 shutdown_connections 2024-09-10T08:21:45.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.373+0000 7fb316deb740 1 --2- 172.21.15.2:0/2750534367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6f7340 0x5589ff714280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.373+0000 7fb316deb740 1 --2- 172.21.15.2:0/2750534367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6e8570 0x5589ff713d40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.373+0000 7fb316deb740 1 --2- 172.21.15.2:0/2750534367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e7c50 0x5589ff6f65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.373+0000 7fb316deb740 1 -- 172.21.15.2:0/2750534367 >> 172.21.15.2:0/2750534367 conn(0x5589ff6e42e0 msgr2=0x5589ff6eea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:45.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.373+0000 7fb316deb740 1 -- 172.21.15.2:0/2750534367 shutdown_connections 2024-09-10T08:21:45.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.373+0000 7fb316deb740 1 -- 172.21.15.2:0/2750534367 wait complete. 2024-09-10T08:21:45.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.374+0000 7fb316deb740 1 -- start start 2024-09-10T08:21:45.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.374+0000 7fb316deb740 1 Processor -- start 2024-09-10T08:21:45.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb316deb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 0x5589ff70ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb316deb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 0x5589ff70f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb316deb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6f7340 0x5589ff70f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb316deb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1ab010 con 0x5589ff6e7c50 2024-09-10T08:21:45.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb316deb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1ac480 con 0x5589ff6f7340 2024-09-10T08:21:45.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb316deb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1ab6e0 con 0x5589ff6e8570 2024-09-10T08:21:45.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb316deb740 10 client.0 _ll_register_callbacks cb 0x7fb2f800fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:45.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb311a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 0x5589ff70ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb311000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 0x5589ff70f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb311000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 0x5589ff70f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:52006/0 (socket says 172.21.15.2:52006) 2024-09-10T08:21:45.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb311000640 1 -- 172.21.15.2:0/2783803482 learned_addr learned my addr 172.21.15.2:0/2783803482 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:45.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb312400640 1 --2- 172.21.15.2:0/2783803482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6f7340 0x5589ff70f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb311000640 1 -- 172.21.15.2:0/2783803482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6f7340 msgr2=0x5589ff70f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:45.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb311000640 1 --2- 172.21.15.2:0/2783803482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6f7340 0x5589ff70f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb311000640 1 -- 172.21.15.2:0/2783803482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 msgr2=0x5589ff70ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:45.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb311000640 1 --2- 172.21.15.2:0/2783803482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 0x5589ff70ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.375+0000 7fb311000640 1 -- 172.21.15.2:0/2783803482 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ff774fa0 con 0x5589ff6e8570 2024-09-10T08:21:45.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.376+0000 7fb311000640 1 --2- 172.21.15.2:0/2783803482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 0x5589ff70f2d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009be0 tx=0x7fb2f8005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec13dcc267b91c04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:45.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.376+0000 7fb2f3400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:21:45.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.376+0000 7fb2f3400640 1 -- 172.21.15.2:0/2783803482 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f801e040 con 0x5589ff6e8570 2024-09-10T08:21:45.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.376+0000 7fb2f3400640 1 -- 172.21.15.2:0/2783803482 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb2f8019450 con 0x5589ff6e8570 2024-09-10T08:21:45.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.376+0000 7fb316deb740 1 -- 172.21.15.2:0/2783803482 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2f803a070 con 0x5589ff6e8570 2024-09-10T08:21:45.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.376+0000 7fb2f3400640 1 -- 172.21.15.2:0/2783803482 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb2f804b050 con 0x5589ff6e8570 2024-09-10T08:21:45.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.377+0000 7fb316deb740 10 client.29257 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:45.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.377+0000 7fb316deb740 1 -- 172.21.15.2:0/2783803482 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589ff7762b0 con 0x5589ff6e8570 2024-09-10T08:21:45.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.377+0000 7fb2f3400640 1 -- 172.21.15.2:0/2783803482 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2f803a070 con 0x5589ff6e8570 2024-09-10T08:21:45.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.377+0000 7fb2f3400640 1 -- 172.21.15.2:0/2783803482 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb2f8003b60 con 0x5589ff6e8570 2024-09-10T08:21:45.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.377+0000 7fb316deb740 10 client.29257 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:45.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.377+0000 7fb316deb740 10 client.29257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:45.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.377+0000 7fb316deb740 20 client.29257 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:45.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.377+0000 7fb316deb740 1 --2- 172.21.15.2:0/2783803482 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5589ff776960 0x5589ff778d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:45.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.377+0000 7fb316deb740 4 client.29257 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:21:45.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.377+0000 7fb316deb740 1 -- 172.21.15.2:0/2783803482 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5589ff655e60 con 0x5589ff776960 2024-09-10T08:21:45.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.378+0000 7fb311a00640 1 --2- 172.21.15.2:0/2783803482 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5589ff776960 0x5589ff778d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:45.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.378+0000 7fb311a00640 1 --2- 172.21.15.2:0/2783803482 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5589ff776960 0x5589ff778d80 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32721ba87923ef40 server_cookie=6ac94a655f43279f in_seq=0 out_seq=0 2024-09-10T08:21:45.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.378+0000 7fb2f3400640 10 client.29257 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:45.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.378+0000 7fb2f3400640 1 -- 172.21.15.2:0/2783803482 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5589ff655e60 con 0x5589ff776960 2024-09-10T08:21:45.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb2f3400640 10 client.29257 handle_command_reply: tid=0 2024-09-10T08:21:45.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb316deb740 1 client.29257 shutdown 2024-09-10T08:21:45.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb316deb740 1 -- 172.21.15.2:0/2783803482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 msgr2=0x5589ff70f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:45.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb316deb740 1 --2- 172.21.15.2:0/2783803482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 0x5589ff70f2d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009be0 tx=0x7fb2f8005750 comp rx=0 tx=0).stop 2024-09-10T08:21:45.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb316deb740 1 -- 172.21.15.2:0/2783803482 shutdown_connections 2024-09-10T08:21:45.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb316deb740 1 --2- 172.21.15.2:0/2783803482 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5589ff776960 0x5589ff778d80 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb316deb740 1 --2- 172.21.15.2:0/2783803482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589ff6f7340 0x5589ff70f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb316deb740 1 --2- 172.21.15.2:0/2783803482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ff6e8570 0x5589ff70f2d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb316deb740 1 --2- 172.21.15.2:0/2783803482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5589ff6e7c50 0x5589ff70ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb316deb740 1 -- 172.21.15.2:0/2783803482 >> 172.21.15.2:0/2783803482 conn(0x5589ff6e42e0 msgr2=0x5589ff6eea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:45.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb2f3400640 0 client.29257 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:45.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb316deb740 1 -- 172.21.15.2:0/2783803482 shutdown_connections 2024-09-10T08:21:45.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb316deb740 1 -- 172.21.15.2:0/2783803482 wait complete. 2024-09-10T08:21:45.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.379+0000 7fb316deb740 20 client.29257 trim_cache size 0 max 16384 2024-09-10T08:21:45.381 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:21:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:21:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:21:45.381 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:21:45.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.380+0000 7fb312e00640 1 -- 172.21.15.2:0/3677608588 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb2f40a2b80 msgr2=0x7fb2f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:45.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.380+0000 7fb312e00640 1 --2- 172.21.15.2:0/3677608588 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb2f40a2b80 0x7fb2f40a5040 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb304005fd0 tx=0x7fb304009500 comp rx=0 tx=0).stop 2024-09-10T08:21:45.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.380+0000 7fb312e00640 1 -- 172.21.15.2:0/3677608588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb30c1019d0 msgr2=0x7fb30c198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:45.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.380+0000 7fb312e00640 1 --2- 172.21.15.2:0/3677608588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb30c1019d0 0x7fb30c198230 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb300035040 tx=0x7fb3000304e0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.380+0000 7fb312e00640 1 -- 172.21.15.2:0/3677608588 shutdown_connections 2024-09-10T08:21:45.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.380+0000 7fb312e00640 1 --2- 172.21.15.2:0/3677608588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb30c1079d0 0x7fb30c19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.380+0000 7fb312e00640 1 --2- 172.21.15.2:0/3677608588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb30c102370 0x7fb30c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.380+0000 7fb312e00640 1 --2- 172.21.15.2:0/3677608588 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb2f40a2b80 0x7fb2f40a5040 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.380+0000 7fb312e00640 1 --2- 172.21.15.2:0/3677608588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb30c1019d0 0x7fb30c198230 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:45.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.380+0000 7fb312e00640 1 -- 172.21.15.2:0/3677608588 >> 172.21.15.2:0/3677608588 conn(0x7fb30c0fd710 msgr2=0x7fb30c0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:45.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.381+0000 7fb312e00640 1 -- 172.21.15.2:0/3677608588 shutdown_connections 2024-09-10T08:21:45.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:45.381+0000 7fb312e00640 1 -- 172.21.15.2:0/3677608588 wait complete. 2024-09-10T08:21:45.385 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:46.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:45 smithi002 ceph-mon[30563]: pgmap v795: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-10T08:21:46.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:45 smithi155 ceph-mon[35405]: pgmap v795: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-10T08:21:46.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:45 smithi107 ceph-mon[35048]: pgmap v795: 129 pgs: 129 active+clean; 2.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-10T08:21:46.273 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:21:46.274 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:21:46.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.751+0000 7fa377e00640 1 -- 172.21.15.2:0/2893741693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa378102360 msgr2=0x7fa37810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:46.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.751+0000 7fa377e00640 1 --2- 172.21.15.2:0/2893741693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa378102360 0x7fa37810e890 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa370009cd0 tx=0x7fa37002a8f0 comp rx=0 tx=0).stop 2024-09-10T08:21:46.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.751+0000 7fa377e00640 1 -- 172.21.15.2:0/2893741693 shutdown_connections 2024-09-10T08:21:46.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.751+0000 7fa377e00640 1 --2- 172.21.15.2:0/2893741693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa378102360 0x7fa37810e890 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:46.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.751+0000 7fa377e00640 1 --2- 172.21.15.2:0/2893741693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3781019c0 0x7fa378101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:46.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.751+0000 7fa377e00640 1 --2- 172.21.15.2:0/2893741693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3781079c0 0x7fa378107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:46.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.751+0000 7fa377e00640 1 -- 172.21.15.2:0/2893741693 >> 172.21.15.2:0/2893741693 conn(0x7fa3780fd740 msgr2=0x7fa3780ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:46.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.751+0000 7fa377e00640 1 -- 172.21.15.2:0/2893741693 shutdown_connections 2024-09-10T08:21:46.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.751+0000 7fa377e00640 1 -- 172.21.15.2:0/2893741693 wait complete. 2024-09-10T08:21:46.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.753+0000 7fa377e00640 1 Processor -- start 2024-09-10T08:21:46.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.753+0000 7fa377e00640 1 -- start start 2024-09-10T08:21:46.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.753+0000 7fa377e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3781019c0 0x7fa37819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:46.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.753+0000 7fa377e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa378102360 0x7fa37819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:46.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.753+0000 7fa377e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3781079c0 0x7fa37819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:46.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.753+0000 7fa377e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa378111e10 con 0x7fa3781019c0 2024-09-10T08:21:46.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.753+0000 7fa377e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa378111c90 con 0x7fa3781079c0 2024-09-10T08:21:46.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.753+0000 7fa377e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa378111f90 con 0x7fa378102360 2024-09-10T08:21:46.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.753+0000 7fa376a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3781019c0 0x7fa37819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:46.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.753+0000 7fa376a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3781019c0 0x7fa37819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50914/0 (socket says 172.21.15.2:50914) 2024-09-10T08:21:46.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.754+0000 7fa376a00640 1 -- 172.21.15.2:0/475027822 learned_addr learned my addr 172.21.15.2:0/475027822 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:46.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.754+0000 7fa377400640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3781079c0 0x7fa37819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:46.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.754+0000 7fa376000640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa378102360 0x7fa37819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:46.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.754+0000 7fa376a00640 1 -- 172.21.15.2:0/475027822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa378102360 msgr2=0x7fa37819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:46.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.754+0000 7fa376a00640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa378102360 0x7fa37819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:46.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.754+0000 7fa376a00640 1 -- 172.21.15.2:0/475027822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3781079c0 msgr2=0x7fa37819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:46.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.754+0000 7fa376a00640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3781079c0 0x7fa37819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:46.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.754+0000 7fa376a00640 1 -- 172.21.15.2:0/475027822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37819f490 con 0x7fa3781019c0 2024-09-10T08:21:46.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.754+0000 7fa377400640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3781079c0 0x7fa37819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:46.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.754+0000 7fa376a00640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3781019c0 0x7fa37819a4e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa36800e9f0 tx=0x7fa36800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:46.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.754+0000 7fa36f400640 1 -- 172.21.15.2:0/475027822 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36800cd80 con 0x7fa3781019c0 2024-09-10T08:21:46.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.755+0000 7fa36f400640 1 -- 172.21.15.2:0/475027822 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa368005500 con 0x7fa3781019c0 2024-09-10T08:21:46.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.755+0000 7fa377e00640 1 -- 172.21.15.2:0/475027822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37819f780 con 0x7fa3781019c0 2024-09-10T08:21:46.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.755+0000 7fa36f400640 1 -- 172.21.15.2:0/475027822 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368010690 con 0x7fa3781019c0 2024-09-10T08:21:46.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.755+0000 7fa377e00640 1 -- 172.21.15.2:0/475027822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3781a7060 con 0x7fa3781019c0 2024-09-10T08:21:46.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.756+0000 7fa377e00640 1 -- 172.21.15.2:0/475027822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa378101e20 con 0x7fa3781019c0 2024-09-10T08:21:46.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.758+0000 7fa36f400640 1 -- 172.21.15.2:0/475027822 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa3680046c0 con 0x7fa3781019c0 2024-09-10T08:21:46.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.759+0000 7fa36f400640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa35c0a2c00 0x7fa35c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:46.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.759+0000 7fa36f400640 1 -- 172.21.15.2:0/475027822 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa3680cf7b0 con 0x7fa3781019c0 2024-09-10T08:21:46.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.759+0000 7fa376000640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa35c0a2c00 0x7fa35c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:46.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.760+0000 7fa376000640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa35c0a2c00 0x7fa35c0a50c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa37810e550 tx=0x7fa364005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:46.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:46.762+0000 7fa36f400640 1 -- 172.21.15.2:0/475027822 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa36809c2c0 con 0x7fa3781019c0 2024-09-10T08:21:46.908 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:47.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:46 smithi002 ceph-mon[30563]: pgmap v796: 129 pgs: 129 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-10T08:21:47.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.028+0000 7fa377e00640 1 -- 172.21.15.2:0/475027822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa37819b840 con 0x7fa3781019c0 2024-09-10T08:21:47.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.028+0000 7fa36f400640 1 -- 172.21.15.2:0/475027822 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fa36809bbc0 con 0x7fa3781019c0 2024-09-10T08:21:47.028 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:21:47.028 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:21:47.028 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:21:47.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.031+0000 7fa377e00640 1 -- 172.21.15.2:0/475027822 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa35c0a2c00 msgr2=0x7fa35c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:47.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.031+0000 7fa377e00640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa35c0a2c00 0x7fa35c0a50c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa37810e550 tx=0x7fa364005f90 comp rx=0 tx=0).stop 2024-09-10T08:21:47.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.031+0000 7fa377e00640 1 -- 172.21.15.2:0/475027822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3781019c0 msgr2=0x7fa37819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:47.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.031+0000 7fa377e00640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3781019c0 0x7fa37819a4e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa36800e9f0 tx=0x7fa36800edf0 comp rx=0 tx=0).stop 2024-09-10T08:21:47.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.031+0000 7fa377e00640 1 -- 172.21.15.2:0/475027822 shutdown_connections 2024-09-10T08:21:47.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.031+0000 7fa377e00640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3781079c0 0x7fa37819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:47.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.031+0000 7fa377e00640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa35c0a2c00 0x7fa35c0a50c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:47.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.031+0000 7fa377e00640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa378102360 0x7fa37819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:47.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.031+0000 7fa377e00640 1 --2- 172.21.15.2:0/475027822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3781019c0 0x7fa37819a4e0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:47.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.031+0000 7fa377e00640 1 -- 172.21.15.2:0/475027822 >> 172.21.15.2:0/475027822 conn(0x7fa3780fd740 msgr2=0x7fa3780fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:47.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.031+0000 7fa377e00640 1 -- 172.21.15.2:0/475027822 shutdown_connections 2024-09-10T08:21:47.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.031+0000 7fa377e00640 1 -- 172.21.15.2:0/475027822 wait complete. 2024-09-10T08:21:47.068 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:46 smithi107 ceph-mon[35048]: pgmap v796: 129 pgs: 129 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-10T08:21:47.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:46 smithi155 ceph-mon[35405]: pgmap v796: 129 pgs: 129 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-10T08:21:47.832 INFO:tasks.workunit.client.0.smithi002.stdout:437/600: open fd = 3 2024-09-10T08:21:47.832 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.832 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.832 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.832 INFO:tasks.workunit.client.0.smithi002.stdout:438/600: open fd = 3 2024-09-10T08:21:47.832 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:439/600: open fd = 3 2024-09-10T08:21:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:440/600: open fd = 3 2024-09-10T08:21:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:441/600: open fd = 3 2024-09-10T08:21:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.834 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.834 INFO:tasks.workunit.client.0.smithi002.stdout:442/600: open fd = 3 2024-09-10T08:21:47.834 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.834 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.834 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.834 INFO:tasks.workunit.client.0.smithi002.stdout:443/600: open fd = 3 2024-09-10T08:21:47.834 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.834 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.834 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.834 INFO:tasks.workunit.client.0.smithi002.stdout:444/600: open fd = 3 2024-09-10T08:21:47.834 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.834 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.834 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.835 INFO:tasks.workunit.client.0.smithi002.stdout:445/600: open fd = 3 2024-09-10T08:21:47.835 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.835 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.835 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.835 INFO:tasks.workunit.client.0.smithi002.stdout:446/600: open fd = 3 2024-09-10T08:21:47.835 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.835 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.835 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.835 INFO:tasks.workunit.client.0.smithi002.stdout:447/600: open fd = 3 2024-09-10T08:21:47.835 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.835 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.835 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.835 INFO:tasks.workunit.client.0.smithi002.stdout:448/600: open fd = 3 2024-09-10T08:21:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:449/600: open fd = 3 2024-09-10T08:21:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:450/600: open fd = 3 2024-09-10T08:21:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:451/600: open fd = 3 2024-09-10T08:21:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.837 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.837 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.837 INFO:tasks.workunit.client.0.smithi002.stdout:452/600: open fd = 3 2024-09-10T08:21:47.837 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.837 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.837 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.837 INFO:tasks.workunit.client.0.smithi002.stdout:453/600: open fd = 3 2024-09-10T08:21:47.837 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.837 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.837 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.837 INFO:tasks.workunit.client.0.smithi002.stdout:454/600: open fd = 3 2024-09-10T08:21:47.837 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.837 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.838 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.838 INFO:tasks.workunit.client.0.smithi002.stdout:455/600: open fd = 3 2024-09-10T08:21:47.838 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.838 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.838 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.838 INFO:tasks.workunit.client.0.smithi002.stdout:456/600: open fd = 3 2024-09-10T08:21:47.838 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.838 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.838 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.838 INFO:tasks.workunit.client.0.smithi002.stdout:457/600: open fd = 3 2024-09-10T08:21:47.838 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.838 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.838 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:458/600: open fd = 3 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:459/600: open fd = 3 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:460/600: open fd = 3 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:461/600: open fd = 3 2024-09-10T08:21:47.839 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:462/600: open fd = 3 2024-09-10T08:21:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:463/600: open fd = 3 2024-09-10T08:21:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:464/600: open fd = 3 2024-09-10T08:21:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.841 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.841 INFO:tasks.workunit.client.0.smithi002.stdout:465/600: open fd = 3 2024-09-10T08:21:47.841 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.841 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.841 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.841 INFO:tasks.workunit.client.0.smithi002.stdout:466/600: open fd = 3 2024-09-10T08:21:47.841 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.841 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.841 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.841 INFO:tasks.workunit.client.0.smithi002.stdout:467/600: open fd = 3 2024-09-10T08:21:47.841 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.841 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.842 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.842 INFO:tasks.workunit.client.0.smithi002.stdout:468/600: open fd = 3 2024-09-10T08:21:47.842 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.842 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.842 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.842 INFO:tasks.workunit.client.0.smithi002.stdout:469/600: open fd = 3 2024-09-10T08:21:47.842 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.842 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.842 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.842 INFO:tasks.workunit.client.0.smithi002.stdout:470/600: open fd = 3 2024-09-10T08:21:47.842 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.842 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.842 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:471/600: open fd = 3 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:472/600: open fd = 3 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:473/600: open fd = 3 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:474/600: open fd = 3 2024-09-10T08:21:47.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:475/600: open fd = 3 2024-09-10T08:21:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:476/600: open fd = 3 2024-09-10T08:21:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:477/600: open fd = 3 2024-09-10T08:21:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:478/600: open fd = 3 2024-09-10T08:21:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:479/600: open fd = 3 2024-09-10T08:21:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:480/600: open fd = 3 2024-09-10T08:21:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:481/600: open fd = 3 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:482/600: open fd = 3 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:483/600: open fd = 3 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.846 INFO:tasks.workunit.client.0.smithi002.stdout:484/600: open fd = 3 2024-09-10T08:21:47.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.847 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.847 INFO:tasks.workunit.client.0.smithi002.stdout:485/600: open fd = 3 2024-09-10T08:21:47.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.847 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.847 INFO:tasks.workunit.client.0.smithi002.stdout:486/600: open fd = 3 2024-09-10T08:21:47.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.848 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.848 INFO:tasks.workunit.client.0.smithi002.stdout:487/600: open fd = 3 2024-09-10T08:21:47.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.848 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.848 INFO:tasks.workunit.client.0.smithi002.stdout:488/600: open fd = 3 2024-09-10T08:21:47.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:489/600: open fd = 3 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:490/600: open fd = 3 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:491/600: open fd = 3 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.849 INFO:tasks.workunit.client.0.smithi002.stdout:492/600: open fd = 3 2024-09-10T08:21:47.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.850 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.850 INFO:tasks.workunit.client.0.smithi002.stdout:493/600: open fd = 3 2024-09-10T08:21:47.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.850 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.850 INFO:tasks.workunit.client.0.smithi002.stdout:494/600: open fd = 3 2024-09-10T08:21:47.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.850 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.850 INFO:tasks.workunit.client.0.smithi002.stdout:495/600: open fd = 3 2024-09-10T08:21:47.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:496/600: open fd = 3 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:497/600: open fd = 3 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:498/600: open fd = 3 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:21:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:21:47.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.850+0000 7f10a3200640 1 -- 172.21.15.2:0/3224107179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10a4076720 msgr2=0x7f10a4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:47.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.850+0000 7f10a3200640 1 --2- 172.21.15.2:0/3224107179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10a4076720 0x7f10a4076b00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1094009900 tx=0x7f109402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:21:47.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.851+0000 7f10a3200640 1 -- 172.21.15.2:0/3224107179 shutdown_connections 2024-09-10T08:21:47.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.851+0000 7f10a3200640 1 --2- 172.21.15.2:0/3224107179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a4112870 0x7f10a4114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:47.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.851+0000 7f10a3200640 1 --2- 172.21.15.2:0/3224107179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10a4077040 0x7f10a4112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.851+0000 7f10a3200640 1 --2- 172.21.15.2:0/3224107179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10a4076720 0x7f10a4076b00 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.851+0000 7f10a3200640 1 -- 172.21.15.2:0/3224107179 >> 172.21.15.2:0/3224107179 conn(0x7f10a40ff8d0 msgr2=0x7f10a4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.851+0000 7f10a3200640 1 -- 172.21.15.2:0/3224107179 shutdown_connections 2024-09-10T08:21:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.851+0000 7f10a3200640 1 -- 172.21.15.2:0/3224107179 wait complete. 2024-09-10T08:21:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.852+0000 7f10a3200640 1 Processor -- start 2024-09-10T08:21:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.852+0000 7f10a3200640 1 -- start start 2024-09-10T08:21:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.852+0000 7f10a3200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10a4076720 0x7f10a41a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.852+0000 7f10a3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a4077040 0x7f10a41a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.852+0000 7f10a3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10a4112870 0x7f10a419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.852+0000 7f10a3200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f10a4118400 con 0x7f10a4076720 2024-09-10T08:21:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.852+0000 7f10a3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10a4118280 con 0x7f10a4112870 2024-09-10T08:21:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.852+0000 7f10a3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f10a4118580 con 0x7f10a4077040 2024-09-10T08:21:47.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.852+0000 7f10a1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10a4076720 0x7f10a41a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.852+0000 7f10a1e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10a4076720 0x7f10a41a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58740/0 (socket says 172.21.15.2:58740) 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.852+0000 7f10a1e00640 1 -- 172.21.15.2:0/1529761695 learned_addr learned my addr 172.21.15.2:0/1529761695 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.852+0000 7f10a1400640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a4077040 0x7f10a41a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.852+0000 7f10a2800640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10a4112870 0x7f10a419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.853+0000 7f10a1e00640 1 -- 172.21.15.2:0/1529761695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a4077040 msgr2=0x7f10a41a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.853+0000 7f10a1e00640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a4077040 0x7f10a41a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.853+0000 7f10a1e00640 1 -- 172.21.15.2:0/1529761695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10a4112870 msgr2=0x7f10a419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.853+0000 7f10a1e00640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10a4112870 0x7f10a419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.853+0000 7f10a1e00640 1 -- 172.21.15.2:0/1529761695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a419efe0 con 0x7f10a4076720 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.853+0000 7f10a1400640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a4077040 0x7f10a41a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.853+0000 7f10a2800640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10a4112870 0x7f10a419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.854+0000 7f10a1e00640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10a4076720 0x7f10a41a46a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1094006010 tx=0x7f10940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.854+0000 7f109aa00640 1 -- 172.21.15.2:0/1529761695 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109403f030 con 0x7f10a4076720 2024-09-10T08:21:47.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.854+0000 7f109aa00640 1 -- 172.21.15.2:0/1529761695 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1094003ce0 con 0x7f10a4076720 2024-09-10T08:21:47.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.854+0000 7f10a3200640 1 -- 172.21.15.2:0/1529761695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a419f270 con 0x7f10a4076720 2024-09-10T08:21:47.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.854+0000 7f109aa00640 1 -- 172.21.15.2:0/1529761695 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094035070 con 0x7f10a4076720 2024-09-10T08:21:47.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.854+0000 7f10a3200640 1 -- 172.21.15.2:0/1529761695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a41ab450 con 0x7f10a4076720 2024-09-10T08:21:47.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.855+0000 7f10a3200640 1 -- 172.21.15.2:0/1529761695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a4077e60 con 0x7f10a4076720 2024-09-10T08:21:47.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.857+0000 7f109aa00640 1 -- 172.21.15.2:0/1529761695 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1094042030 con 0x7f10a4076720 2024-09-10T08:21:47.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.858+0000 7f109aa00640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f10880a2c00 0x7f10880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:47.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.859+0000 7f109aa00640 1 -- 172.21.15.2:0/1529761695 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f10940ef550 con 0x7f10a4076720 2024-09-10T08:21:47.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.859+0000 7f10a1400640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f10880a2c00 0x7f10880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:47.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.859+0000 7f10a1400640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f10880a2c00 0x7f10880a50c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f10a419ffc0 tx=0x7f109000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:47.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:47.862+0000 7f109aa00640 1 -- 172.21.15.2:0/1529761695 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f10940bbad0 con 0x7f10a4076720 2024-09-10T08:21:48.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:47 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/475027822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:48.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.093+0000 7f10a3200640 1 -- 172.21.15.2:0/1529761695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10a410da00 con 0x7f10a4076720 2024-09-10T08:21:48.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.093+0000 7f109aa00640 1 -- 172.21.15.2:0/1529761695 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f10940bfdd0 con 0x7f10a4076720 2024-09-10T08:21:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:21:48.093 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:21:48.093 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:21:48.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.096+0000 7f10a3200640 1 -- 172.21.15.2:0/1529761695 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f10880a2c00 msgr2=0x7f10880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:48.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.096+0000 7f10a3200640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f10880a2c00 0x7f10880a50c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f10a419ffc0 tx=0x7f109000b410 comp rx=0 tx=0).stop 2024-09-10T08:21:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.096+0000 7f10a3200640 1 -- 172.21.15.2:0/1529761695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10a4076720 msgr2=0x7f10a41a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.096+0000 7f10a3200640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10a4076720 0x7f10a41a46a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1094006010 tx=0x7f10940042f0 comp rx=0 tx=0).stop 2024-09-10T08:21:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.096+0000 7f10a3200640 1 -- 172.21.15.2:0/1529761695 shutdown_connections 2024-09-10T08:21:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.096+0000 7f10a3200640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10a4112870 0x7f10a419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.096+0000 7f10a3200640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f10880a2c00 0x7f10880a50c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.096+0000 7f10a3200640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a4077040 0x7f10a41a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.096+0000 7f10a3200640 1 --2- 172.21.15.2:0/1529761695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10a4076720 0x7f10a41a46a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.096+0000 7f10a3200640 1 -- 172.21.15.2:0/1529761695 >> 172.21.15.2:0/1529761695 conn(0x7f10a40ff8d0 msgr2=0x7f10a4101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.096+0000 7f10a3200640 1 -- 172.21.15.2:0/1529761695 shutdown_connections 2024-09-10T08:21:48.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:48.096+0000 7f10a3200640 1 -- 172.21.15.2:0/1529761695 wait complete. 2024-09-10T08:21:48.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:47 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/475027822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:48.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:47 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/475027822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:48.452 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0' 2024-09-10T08:21:48.452 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0 2024-09-10T08:21:49.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:48 smithi155 ceph-mon[35405]: pgmap v797: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-09-10T08:21:49.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:48 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1529761695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:49.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:48 smithi107 ceph-mon[35048]: pgmap v797: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-09-10T08:21:49.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:48 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1529761695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:49.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:48 smithi002 ceph-mon[30563]: pgmap v797: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-09-10T08:21:49.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:48 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1529761695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:21:49.448 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:21:49.477 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:50.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.076+0000 7f4571400640 1 -- 172.21.15.2:0/1951338228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1047e0 msgr2=0x7f456c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:50.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.076+0000 7f4571400640 1 --2- 172.21.15.2:0/1951338228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1047e0 0x7f456c108830 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4554009900 tx=0x7f455402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.076+0000 7f4571400640 1 -- 172.21.15.2:0/1951338228 shutdown_connections 2024-09-10T08:21:50.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.076+0000 7f4571400640 1 --2- 172.21.15.2:0/1951338228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f456c108f60 0x7f456c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.076+0000 7f4571400640 1 --2- 172.21.15.2:0/1951338228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1047e0 0x7f456c108830 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.076+0000 7f4571400640 1 --2- 172.21.15.2:0/1951338228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f456c103e30 0x7f456c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.076+0000 7f4571400640 1 -- 172.21.15.2:0/1951338228 >> 172.21.15.2:0/1951338228 conn(0x7f456c0ff8d0 msgr2=0x7f456c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:50.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.076+0000 7f4571400640 1 -- 172.21.15.2:0/1951338228 shutdown_connections 2024-09-10T08:21:50.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.076+0000 7f4571400640 1 -- 172.21.15.2:0/1951338228 wait complete. 2024-09-10T08:21:50.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.077+0000 7f4571400640 1 Processor -- start 2024-09-10T08:21:50.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.077+0000 7f4571400640 1 -- start start 2024-09-10T08:21:50.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.078+0000 7f4571400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f456c103e30 0x7f456c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.078+0000 7f4571400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1047e0 0x7f456c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.078+0000 7f4571400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f456c108f60 0x7f456c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.078+0000 7f4571400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f456c113fc0 con 0x7f456c1047e0 2024-09-10T08:21:50.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.078+0000 7f4571400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f456c113e40 con 0x7f456c103e30 2024-09-10T08:21:50.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.078+0000 7f4571400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f456c114140 con 0x7f456c108f60 2024-09-10T08:21:50.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.078+0000 7f4563400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1047e0 0x7f456c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.078+0000 7f4570a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f456c108f60 0x7f456c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.078+0000 7f456be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f456c103e30 0x7f456c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.078+0000 7f4570a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f456c108f60 0x7f456c19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:40506/0 (socket says 172.21.15.2:40506) 2024-09-10T08:21:50.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.078+0000 7f4570a00640 1 -- 172.21.15.2:0/2604058530 learned_addr learned my addr 172.21.15.2:0/2604058530 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:50.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.079+0000 7f4570a00640 1 -- 172.21.15.2:0/2604058530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f456c103e30 msgr2=0x7f456c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:50.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.079+0000 7f4570a00640 1 --2- 172.21.15.2:0/2604058530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f456c103e30 0x7f456c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.079+0000 7f4570a00640 1 -- 172.21.15.2:0/2604058530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1047e0 msgr2=0x7f456c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:50.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.079+0000 7f4570a00640 1 --2- 172.21.15.2:0/2604058530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1047e0 0x7f456c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.079+0000 7f4570a00640 1 -- 172.21.15.2:0/2604058530 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c19f430 con 0x7f456c108f60 2024-09-10T08:21:50.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.079+0000 7f456be00640 1 --2- 172.21.15.2:0/2604058530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f456c103e30 0x7f456c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:50.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.079+0000 7f4570a00640 1 --2- 172.21.15.2:0/2604058530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f456c108f60 0x7f456c19ed50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f456400eaa0 tx=0x7f456400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:50.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.079+0000 7f4569600640 1 -- 172.21.15.2:0/2604058530 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456400cdb0 con 0x7f456c108f60 2024-09-10T08:21:50.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.079+0000 7f4571400640 1 -- 172.21.15.2:0/2604058530 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456c19f720 con 0x7f456c108f60 2024-09-10T08:21:50.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.079+0000 7f4569600640 1 -- 172.21.15.2:0/2604058530 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4564005020 con 0x7f456c108f60 2024-09-10T08:21:50.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.079+0000 7f4569600640 1 -- 172.21.15.2:0/2604058530 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45640116b0 con 0x7f456c108f60 2024-09-10T08:21:50.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.080+0000 7f4571400640 1 -- 172.21.15.2:0/2604058530 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456c1a7000 con 0x7f456c108f60 2024-09-10T08:21:50.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.083+0000 7f4569600640 1 -- 172.21.15.2:0/2604058530 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4564005560 con 0x7f456c108f60 2024-09-10T08:21:50.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.083+0000 7f4575430740 1 Processor -- start 2024-09-10T08:21:50.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.083+0000 7f4575430740 1 -- start start 2024-09-10T08:21:50.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.083+0000 7f4575430740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580560 0x55b81a580940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.083+0000 7f4575430740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a580e80 0x55b81a58b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.083+0000 7f4575430740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a58b740 0x55b81a58db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.083+0000 7f4575430740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b81a4e6530 con 0x55b81a580560 2024-09-10T08:21:50.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.083+0000 7f4575430740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b81a48d860 con 0x55b81a580e80 2024-09-10T08:21:50.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.083+0000 7f4575430740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b81a4e24f0 con 0x55b81a58b740 2024-09-10T08:21:50.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.084+0000 7f4563400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580560 0x55b81a580940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.084+0000 7f456be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a580e80 0x55b81a58b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.084+0000 7f456be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a580e80 0x55b81a58b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40242/0 (socket says 172.21.15.2:40242) 2024-09-10T08:21:50.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.084+0000 7f456be00640 1 -- 172.21.15.2:0/4101280814 learned_addr learned my addr 172.21.15.2:0/4101280814 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:50.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.084+0000 7f4563400640 1 --2- 172.21.15.2:0/4101280814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a58b740 0x55b81a58db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.084+0000 7f456be00640 1 -- 172.21.15.2:0/4101280814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a58b740 msgr2=0x55b81a58db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:50.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.084+0000 7f456be00640 1 --2- 172.21.15.2:0/4101280814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a58b740 0x55b81a58db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.084+0000 7f456be00640 1 -- 172.21.15.2:0/4101280814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580560 msgr2=0x55b81a580940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:50.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.084+0000 7f456be00640 1 --2- 172.21.15.2:0/4101280814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580560 0x55b81a580940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.084+0000 7f456be00640 1 -- 172.21.15.2:0/4101280814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b81a257b90 con 0x55b81a580e80 2024-09-10T08:21:50.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.084+0000 7f4563400640 1 --2- 172.21.15.2:0/4101280814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a58b740 0x55b81a58db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:21:50.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.084+0000 7f4569600640 1 --2- 172.21.15.2:0/2604058530 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f45500a2b80 0x7f45500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.084+0000 7f456be00640 1 --2- 172.21.15.2:0/4101280814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a580e80 0x55b81a58b200 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f455c0115c0 tx=0x7f455c011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=228fc1c70db77f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:50.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.085+0000 7f4569600640 1 -- 172.21.15.2:0/2604058530 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f45640cf5d0 con 0x7f456c108f60 2024-09-10T08:21:50.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.085+0000 7f4561200640 1 -- 172.21.15.2:0/4101280814 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c00f640 con 0x55b81a580e80 2024-09-10T08:21:50.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.085+0000 7f4561200640 1 -- 172.21.15.2:0/4101280814 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f455c00fcc0 con 0x55b81a580e80 2024-09-10T08:21:50.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.085+0000 7f4561200640 1 -- 172.21.15.2:0/4101280814 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c019e10 con 0x55b81a580e80 2024-09-10T08:21:50.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.085+0000 7f4563400640 1 --2- 172.21.15.2:0/2604058530 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f45500a2b80 0x7f45500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.085+0000 7f4575430740 1 -- 172.21.15.2:0/4101280814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a580e80 msgr2=0x55b81a58b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:50.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.085+0000 7f4563400640 1 --2- 172.21.15.2:0/2604058530 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f45500a2b80 0x7f45500a5040 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4554009230 tx=0x7f4554000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:50.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.085+0000 7f4575430740 1 --2- 172.21.15.2:0/4101280814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a580e80 0x55b81a58b200 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f455c0115c0 tx=0x7f455c011a80 comp rx=0 tx=0).stop 2024-09-10T08:21:50.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.086+0000 7f4575430740 1 -- 172.21.15.2:0/4101280814 shutdown_connections 2024-09-10T08:21:50.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.086+0000 7f4575430740 1 --2- 172.21.15.2:0/4101280814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a58b740 0x55b81a58db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.086+0000 7f4575430740 1 --2- 172.21.15.2:0/4101280814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a580e80 0x55b81a58b200 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.086+0000 7f4575430740 1 --2- 172.21.15.2:0/4101280814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580560 0x55b81a580940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.086+0000 7f4575430740 1 -- 172.21.15.2:0/4101280814 >> 172.21.15.2:0/4101280814 conn(0x55b81a57caf0 msgr2=0x55b81a57ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:50.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.086+0000 7f4575430740 1 -- 172.21.15.2:0/4101280814 shutdown_connections 2024-09-10T08:21:50.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.086+0000 7f4575430740 1 -- 172.21.15.2:0/4101280814 wait complete. 2024-09-10T08:21:50.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.086+0000 7f4575430740 1 -- start start 2024-09-10T08:21:50.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.086+0000 7f4575430740 1 Processor -- start 2024-09-10T08:21:50.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.087+0000 7f4575430740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a580560 0x55b81a588640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.087+0000 7f4575430740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580e80 0x55b81a588b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.087+0000 7f4575430740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a58b740 0x55b81a5890c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.087+0000 7f4575430740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b81a58e310 con 0x55b81a580e80 2024-09-10T08:21:50.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.087+0000 7f4575430740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b81a4a0e60 con 0x55b81a58b740 2024-09-10T08:21:50.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.087+0000 7f4575430740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b81a58e490 con 0x55b81a580560 2024-09-10T08:21:50.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.087+0000 7f4575430740 10 client.0 _ll_register_callbacks cb 0x55b81a4a5cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:50.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.087+0000 7f4563400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580e80 0x55b81a588b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.087+0000 7f4563400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580e80 0x55b81a588b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58768/0 (socket says 172.21.15.2:58768) 2024-09-10T08:21:50.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.087+0000 7f4563400640 1 -- 172.21.15.2:0/2436973193 learned_addr learned my addr 172.21.15.2:0/2436973193 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:50.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.087+0000 7f456be00640 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a580560 0x55b81a588640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.087+0000 7f4570a00640 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a58b740 0x55b81a5890c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.088+0000 7f4563400640 1 -- 172.21.15.2:0/2436973193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a580560 msgr2=0x55b81a588640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:50.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.088+0000 7f4563400640 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a580560 0x55b81a588640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.088+0000 7f4563400640 1 -- 172.21.15.2:0/2436973193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a58b740 msgr2=0x55b81a5890c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:50.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.088+0000 7f4563400640 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a58b740 0x55b81a5890c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.088+0000 7f4563400640 1 -- 172.21.15.2:0/2436973193 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b81a4ac990 con 0x55b81a580e80 2024-09-10T08:21:50.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.088+0000 7f456be00640 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a580560 0x55b81a588640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:50.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.088+0000 7f4570a00640 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a58b740 0x55b81a5890c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:50.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.088+0000 7f4563400640 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580e80 0x55b81a588b80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f45540042c0 tx=0x7f455402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3446019f6f11cf86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:50.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.088+0000 7f4559600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:50.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.088+0000 7f4559600640 1 -- 172.21.15.2:0/2436973193 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554035040 con 0x55b81a580e80 2024-09-10T08:21:50.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.088+0000 7f4575430740 1 -- 172.21.15.2:0/2436973193 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f455c00fcc0 con 0x55b81a580e80 2024-09-10T08:21:50.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.088+0000 7f4559600640 1 -- 172.21.15.2:0/2436973193 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4554003cf0 con 0x55b81a580e80 2024-09-10T08:21:50.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.089+0000 7f4559600640 1 -- 172.21.15.2:0/2436973193 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f455404b450 con 0x55b81a580e80 2024-09-10T08:21:50.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.089+0000 7f4575430740 10 client.19326 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:50.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.089+0000 7f4575430740 1 -- 172.21.15.2:0/2436973193 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b81a599eb0 con 0x55b81a580e80 2024-09-10T08:21:50.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.089+0000 7f4559600640 1 -- 172.21.15.2:0/2436973193 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f455c00fcc0 con 0x55b81a580e80 2024-09-10T08:21:50.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.089+0000 7f4559600640 1 -- 172.21.15.2:0/2436973193 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f455403c740 con 0x55b81a580e80 2024-09-10T08:21:50.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.089+0000 7f4575430740 10 client.19326 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:50.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.089+0000 7f4575430740 10 client.19326 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:21:50.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.089+0000 7f4575430740 20 client.19326 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:50.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.089+0000 7f4575430740 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b81a59b460 0x55b81a59d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.089+0000 7f4575430740 4 client.19326 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:50.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.089+0000 7f4575430740 1 -- 172.21.15.2:0/2436973193 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b81a489310 con 0x55b81a59b460 2024-09-10T08:21:50.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.090+0000 7f456be00640 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b81a59b460 0x55b81a59d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.090+0000 7f456be00640 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b81a59b460 0x55b81a59d8a0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d95006ecc4d5a45 server_cookie=83a625ee13721361 in_seq=0 out_seq=0 2024-09-10T08:21:50.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.090+0000 7f4559600640 10 client.19326 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:50.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.091+0000 7f4559600640 1 -- 172.21.15.2:0/2436973193 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b81a489310 con 0x55b81a59b460 2024-09-10T08:21:50.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.091+0000 7f4559600640 10 client.19326 handle_command_reply: tid=0 2024-09-10T08:21:50.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.091+0000 7f4575430740 1 client.19326 shutdown 2024-09-10T08:21:50.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.091+0000 7f4575430740 1 -- 172.21.15.2:0/2436973193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580e80 msgr2=0x55b81a588b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:50.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.091+0000 7f4575430740 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580e80 0x55b81a588b80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f45540042c0 tx=0x7f455402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.091+0000 7f4575430740 1 -- 172.21.15.2:0/2436973193 shutdown_connections 2024-09-10T08:21:50.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.091+0000 7f4575430740 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b81a59b460 0x55b81a59d8a0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.091+0000 7f4575430740 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a58b740 0x55b81a5890c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.091+0000 7f4575430740 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580e80 0x55b81a588b80 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.091+0000 7f4575430740 1 --2- 172.21.15.2:0/2436973193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a580560 0x55b81a588640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.091+0000 7f4575430740 1 -- 172.21.15.2:0/2436973193 >> 172.21.15.2:0/2436973193 conn(0x55b81a57caf0 msgr2=0x55b81a582820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:50.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.091+0000 7f4559600640 0 client.19326 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:50.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.091+0000 7f4575430740 1 -- 172.21.15.2:0/2436973193 shutdown_connections 2024-09-10T08:21:50.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.092+0000 7f4575430740 1 -- 172.21.15.2:0/2436973193 wait complete. 2024-09-10T08:21:50.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.092+0000 7f4575430740 20 client.19326 trim_cache size 0 max 16384 2024-09-10T08:21:50.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.110+0000 7f4575430740 1 Processor -- start 2024-09-10T08:21:50.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.111+0000 7f4575430740 1 -- start start 2024-09-10T08:21:50.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.111+0000 7f4575430740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580560 0x55b81a5a3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.111+0000 7f4575430740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a580e80 0x55b81a5a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.111+0000 7f4575430740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a58b740 0x55b81a57b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.111+0000 7f4575430740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1b3660 con 0x55b81a580560 2024-09-10T08:21:50.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.111+0000 7f4575430740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1b0f30 con 0x55b81a580e80 2024-09-10T08:21:50.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.111+0000 7f4575430740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1aed80 con 0x55b81a58b740 2024-09-10T08:21:50.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.111+0000 7f456be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580560 0x55b81a5a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.111+0000 7f4563400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a580e80 0x55b81a5a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.111+0000 7f456be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580560 0x55b81a5a3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58776/0 (socket says 172.21.15.2:58776) 2024-09-10T08:21:50.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.111+0000 7f456be00640 1 -- 172.21.15.2:0/3029257463 learned_addr learned my addr 172.21.15.2:0/3029257463 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:50.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.111+0000 7f4570a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a58b740 0x55b81a57b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.112+0000 7f4563400640 1 -- 172.21.15.2:0/3029257463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a58b740 msgr2=0x55b81a57b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:50.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.112+0000 7f4563400640 1 --2- 172.21.15.2:0/3029257463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a58b740 0x55b81a57b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.112+0000 7f4563400640 1 -- 172.21.15.2:0/3029257463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580560 msgr2=0x55b81a5a3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:50.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.112+0000 7f4563400640 1 --2- 172.21.15.2:0/3029257463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580560 0x55b81a5a3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.112+0000 7f4563400640 1 -- 172.21.15.2:0/3029257463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b81a578530 con 0x55b81a580e80 2024-09-10T08:21:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.112+0000 7f4563400640 1 --2- 172.21.15.2:0/3029257463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a580e80 0x55b81a5a36b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f455403cdf0 tx=0x7f4554009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a843103874b2539e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.112+0000 7f455be00640 1 -- 172.21.15.2:0/3029257463 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554035070 con 0x55b81a580e80 2024-09-10T08:21:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.112+0000 7f455be00640 1 -- 172.21.15.2:0/3029257463 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45540317c0 con 0x55b81a580e80 2024-09-10T08:21:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.112+0000 7f455be00640 1 -- 172.21.15.2:0/3029257463 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455404b970 con 0x55b81a580e80 2024-09-10T08:21:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.113+0000 7f4575430740 1 -- 172.21.15.2:0/3029257463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a580e80 msgr2=0x55b81a5a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.113+0000 7f4575430740 1 --2- 172.21.15.2:0/3029257463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a580e80 0x55b81a5a36b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f455403cdf0 tx=0x7f4554009260 comp rx=0 tx=0).stop 2024-09-10T08:21:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.113+0000 7f4575430740 1 -- 172.21.15.2:0/3029257463 shutdown_connections 2024-09-10T08:21:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.113+0000 7f4575430740 1 --2- 172.21.15.2:0/3029257463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a58b740 0x55b81a57b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.113+0000 7f4575430740 1 --2- 172.21.15.2:0/3029257463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a580e80 0x55b81a5a36b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.113+0000 7f4575430740 1 --2- 172.21.15.2:0/3029257463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580560 0x55b81a5a3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.113+0000 7f4575430740 1 -- 172.21.15.2:0/3029257463 >> 172.21.15.2:0/3029257463 conn(0x55b81a57caf0 msgr2=0x55b81a587730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:50.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.113+0000 7f4575430740 1 -- 172.21.15.2:0/3029257463 shutdown_connections 2024-09-10T08:21:50.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.114+0000 7f4575430740 1 -- 172.21.15.2:0/3029257463 wait complete. 2024-09-10T08:21:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.114+0000 7f4575430740 1 -- start start 2024-09-10T08:21:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.114+0000 7f4575430740 1 Processor -- start 2024-09-10T08:21:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.115+0000 7f4575430740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a580560 0x55b81a593c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.115+0000 7f4575430740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580e80 0x55b81a585150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.115+0000 7f4575430740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a58b740 0x55b81a585690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.115+0000 7f4575430740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1af3f0 con 0x55b81a580e80 2024-09-10T08:21:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.115+0000 7f4575430740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1b0860 con 0x55b81a58b740 2024-09-10T08:21:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.115+0000 7f4575430740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1afac0 con 0x55b81a580560 2024-09-10T08:21:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.115+0000 7f4575430740 10 client.0 _ll_register_callbacks cb 0x7f4554009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:50.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.115+0000 7f4563400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580e80 0x55b81a585150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.115+0000 7f456be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a580560 0x55b81a593c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.115+0000 7f456be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a580560 0x55b81a593c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:40540/0 (socket says 172.21.15.2:40540) 2024-09-10T08:21:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.115+0000 7f456be00640 1 -- 172.21.15.2:0/1385200077 learned_addr learned my addr 172.21.15.2:0/1385200077 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.115+0000 7f4570a00640 1 --2- 172.21.15.2:0/1385200077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a58b740 0x55b81a585690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.116+0000 7f4563400640 1 -- 172.21.15.2:0/1385200077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a580560 msgr2=0x55b81a593c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.116+0000 7f4563400640 1 --2- 172.21.15.2:0/1385200077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a580560 0x55b81a593c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.116+0000 7f4563400640 1 -- 172.21.15.2:0/1385200077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a58b740 msgr2=0x55b81a585690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.116+0000 7f4563400640 1 --2- 172.21.15.2:0/1385200077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a58b740 0x55b81a585690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.116+0000 7f4563400640 1 -- 172.21.15.2:0/1385200077 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b81a585c80 con 0x55b81a580e80 2024-09-10T08:21:50.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.116+0000 7f4563400640 1 --2- 172.21.15.2:0/1385200077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580e80 0x55b81a585150 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f455403cc00 tx=0x7f4554003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47ae2880d74e039 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:50.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.116+0000 7f4559600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:50.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.116+0000 7f4559600640 1 -- 172.21.15.2:0/1385200077 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554035040 con 0x55b81a580e80 2024-09-10T08:21:50.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.117+0000 7f4559600640 1 -- 172.21.15.2:0/1385200077 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f455403c850 con 0x55b81a580e80 2024-09-10T08:21:50.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.117+0000 7f4575430740 1 -- 172.21.15.2:0/1385200077 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45540317c0 con 0x55b81a580e80 2024-09-10T08:21:50.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.117+0000 7f4559600640 1 -- 172.21.15.2:0/1385200077 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f455404b6f0 con 0x55b81a580e80 2024-09-10T08:21:50.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.117+0000 7f4575430740 10 client.19332 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:50.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.117+0000 7f4575430740 1 -- 172.21.15.2:0/1385200077 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b81a605ed0 con 0x55b81a580e80 2024-09-10T08:21:50.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.117+0000 7f4559600640 1 -- 172.21.15.2:0/1385200077 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45540317c0 con 0x55b81a580e80 2024-09-10T08:21:50.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.118+0000 7f4559600640 1 -- 172.21.15.2:0/1385200077 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f455405e8a0 con 0x55b81a580e80 2024-09-10T08:21:50.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.118+0000 7f4575430740 10 client.19332 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:50.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.118+0000 7f4575430740 10 client.19332 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:21:50.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.118+0000 7f4575430740 20 client.19332 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:50.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.118+0000 7f4575430740 1 --2- 172.21.15.2:0/1385200077 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b81a606440 0x55b81a608860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:50.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.118+0000 7f4575430740 4 client.19332 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}] 2024-09-10T08:21:50.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.118+0000 7f4575430740 1 -- 172.21.15.2:0/1385200077 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}) -- 0x55b81a4f3c60 con 0x55b81a606440 2024-09-10T08:21:50.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.118+0000 7f456be00640 1 --2- 172.21.15.2:0/1385200077 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b81a606440 0x55b81a608860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:50.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.119+0000 7f456be00640 1 --2- 172.21.15.2:0/1385200077 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b81a606440 0x55b81a608860 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79821b9cb3bf1744 server_cookie=cedbcd9cc5c42b0f in_seq=0 out_seq=0 2024-09-10T08:21:50.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.119+0000 7f4559600640 10 client.19332 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:50.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.133+0000 7f4559600640 1 -- 172.21.15.2:0/1385200077 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b81a4f3c60 con 0x55b81a606440 2024-09-10T08:21:50.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.133+0000 7f4559600640 10 client.19332 handle_command_reply: tid=0 2024-09-10T08:21:50.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.133+0000 7f4575430740 1 client.19332 shutdown 2024-09-10T08:21:50.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.134+0000 7f4575430740 1 -- 172.21.15.2:0/1385200077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580e80 msgr2=0x55b81a585150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:50.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.134+0000 7f4575430740 1 --2- 172.21.15.2:0/1385200077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580e80 0x55b81a585150 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f455403cc00 tx=0x7f4554003f90 comp rx=0 tx=0).stop 2024-09-10T08:21:50.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.134+0000 7f4575430740 1 -- 172.21.15.2:0/1385200077 shutdown_connections 2024-09-10T08:21:50.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.134+0000 7f4575430740 1 --2- 172.21.15.2:0/1385200077 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b81a606440 0x55b81a608860 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.134+0000 7f4575430740 1 --2- 172.21.15.2:0/1385200077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b81a58b740 0x55b81a585690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.134+0000 7f4575430740 1 --2- 172.21.15.2:0/1385200077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b81a580e80 0x55b81a585150 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.134+0000 7f4575430740 1 --2- 172.21.15.2:0/1385200077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b81a580560 0x55b81a593c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.134+0000 7f4575430740 1 -- 172.21.15.2:0/1385200077 >> 172.21.15.2:0/1385200077 conn(0x55b81a57caf0 msgr2=0x55b81a57b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:50.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.134+0000 7f4559600640 0 client.19332 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:50.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.134+0000 7f4575430740 1 -- 172.21.15.2:0/1385200077 shutdown_connections 2024-09-10T08:21:50.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.134+0000 7f4575430740 1 -- 172.21.15.2:0/1385200077 wait complete. 2024-09-10T08:21:50.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.134+0000 7f4575430740 20 client.19332 trim_cache size 0 max 16384 2024-09-10T08:21:50.135 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:21:50.136 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:21:50.136 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:21:50.136 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 50, 2024-09-10T08:21:50.136 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:21:50.136 INFO:teuthology.orchestra.run.smithi002.stdout: "73163cc9": { 2024-09-10T08:21:50.136 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 50, 2024-09-10T08:21:50.136 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:21:50.136 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:21:50.136 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:21:50.136 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:21:50.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:21:50.136 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:21:50.137 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 141.5, 2024-09-10T08:21:50.137 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:21:50.137 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:21:50.137 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:21:50.137 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:21:50.137 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:21:50.137 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:21:50.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:50.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:50.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:50.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:50.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:21:50.138 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:21:50.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.135+0000 7f4571400640 1 -- 172.21.15.2:0/2604058530 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f45500a2b80 msgr2=0x7f45500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:50.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.135+0000 7f4571400640 1 --2- 172.21.15.2:0/2604058530 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f45500a2b80 0x7f45500a5040 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4554009230 tx=0x7f4554000bc0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.135+0000 7f4571400640 1 -- 172.21.15.2:0/2604058530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f456c108f60 msgr2=0x7f456c19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:50.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.135+0000 7f4571400640 1 --2- 172.21.15.2:0/2604058530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f456c108f60 0x7f456c19ed50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f456400eaa0 tx=0x7f456400ef60 comp rx=0 tx=0).stop 2024-09-10T08:21:50.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.135+0000 7f4571400640 1 -- 172.21.15.2:0/2604058530 shutdown_connections 2024-09-10T08:21:50.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.135+0000 7f4571400640 1 --2- 172.21.15.2:0/2604058530 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f45500a2b80 0x7f45500a5040 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.135+0000 7f4571400640 1 --2- 172.21.15.2:0/2604058530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f456c108f60 0x7f456c19ed50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.135+0000 7f4571400640 1 --2- 172.21.15.2:0/2604058530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f456c1047e0 0x7f456c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.135+0000 7f4571400640 1 --2- 172.21.15.2:0/2604058530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f456c103e30 0x7f456c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:50.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.135+0000 7f4571400640 1 -- 172.21.15.2:0/2604058530 >> 172.21.15.2:0/2604058530 conn(0x7f456c0ff8d0 msgr2=0x7f456c100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:50.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.135+0000 7f4571400640 1 -- 172.21.15.2:0/2604058530 shutdown_connections 2024-09-10T08:21:50.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:50.135+0000 7f4571400640 1 -- 172.21.15.2:0/2604058530 wait complete. 2024-09-10T08:21:50.492 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:51.176 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:51 smithi002 ceph-mon[30563]: pgmap v798: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2024-09-10T08:21:51.215 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '73163cc9', quiesce took 6.5 seconds. Will release after: 15.0 2024-09-10T08:21:51.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:51 smithi155 ceph-mon[35405]: pgmap v798: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2024-09-10T08:21:51.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:51 smithi107 ceph-mon[35048]: pgmap v798: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2024-09-10T08:21:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.755+0000 7f4810200640 1 -- 172.21.15.2:0/2372624119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4808112a30 msgr2=0x7f4808114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.755+0000 7f4810200640 1 --2- 172.21.15.2:0/2372624119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4808112a30 0x7f4808114e20 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4800009ca0 tx=0x7f480002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.755+0000 7f4810200640 1 -- 172.21.15.2:0/2372624119 shutdown_connections 2024-09-10T08:21:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.755+0000 7f4810200640 1 --2- 172.21.15.2:0/2372624119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4808112a30 0x7f4808114e20 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.755+0000 7f4810200640 1 --2- 172.21.15.2:0/2372624119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4808077040 0x7f48081124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.755+0000 7f4810200640 1 --2- 172.21.15.2:0/2372624119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4808076720 0x7f4808076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.755+0000 7f4810200640 1 -- 172.21.15.2:0/2372624119 >> 172.21.15.2:0/2372624119 conn(0x7f48080ffa90 msgr2=0x7f4808101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.755+0000 7f4810200640 1 -- 172.21.15.2:0/2372624119 shutdown_connections 2024-09-10T08:21:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.755+0000 7f4810200640 1 -- 172.21.15.2:0/2372624119 wait complete. 2024-09-10T08:21:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.756+0000 7f4810200640 1 Processor -- start 2024-09-10T08:21:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.756+0000 7f4810200640 1 -- start start 2024-09-10T08:21:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.756+0000 7f4810200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4808076720 0x7f48081a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.756+0000 7f4810200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4808077040 0x7f48081a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.756+0000 7f4810200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4808112a30 0x7f480819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.756+0000 7f4810200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4808118570 con 0x7f4808076720 2024-09-10T08:21:51.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.756+0000 7f4810200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48081183f0 con 0x7f4808077040 2024-09-10T08:21:51.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.756+0000 7f4810200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f48081186f0 con 0x7f4808112a30 2024-09-10T08:21:51.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.757+0000 7f480d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4808076720 0x7f48081a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.757+0000 7f480d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4808076720 0x7f48081a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58812/0 (socket says 172.21.15.2:58812) 2024-09-10T08:21:51.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.757+0000 7f480d800640 1 -- 172.21.15.2:0/1326899850 learned_addr learned my addr 172.21.15.2:0/1326899850 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:51.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.757+0000 7f480ce00640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4808077040 0x7f48081a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.757+0000 7f480e200640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4808112a30 0x7f480819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.757+0000 7f480ce00640 1 -- 172.21.15.2:0/1326899850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4808112a30 msgr2=0x7f480819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.757+0000 7f480ce00640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4808112a30 0x7f480819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.757+0000 7f480ce00640 1 -- 172.21.15.2:0/1326899850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4808076720 msgr2=0x7f48081a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.757+0000 7f480ce00640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4808076720 0x7f48081a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.757+0000 7f480ce00640 1 -- 172.21.15.2:0/1326899850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480819efd0 con 0x7f4808077040 2024-09-10T08:21:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.757+0000 7f480d800640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4808076720 0x7f48081a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.757+0000 7f480ce00640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4808077040 0x7f48081a4bf0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f47f400d6d0 tx=0x7f47f400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.758+0000 7f4806000640 1 -- 172.21.15.2:0/1326899850 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4017070 con 0x7f4808077040 2024-09-10T08:21:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.758+0000 7f4810200640 1 -- 172.21.15.2:0/1326899850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480819f2c0 con 0x7f4808077040 2024-09-10T08:21:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.758+0000 7f4806000640 1 -- 172.21.15.2:0/1326899850 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f4009820 con 0x7f4808077040 2024-09-10T08:21:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.758+0000 7f4806000640 1 -- 172.21.15.2:0/1326899850 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4004d00 con 0x7f4808077040 2024-09-10T08:21:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.758+0000 7f4810200640 1 -- 172.21.15.2:0/1326899850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48081ab510 con 0x7f4808077040 2024-09-10T08:21:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.762+0000 7f4812e4b740 1 Processor -- start 2024-09-10T08:21:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.762+0000 7f4812e4b740 1 -- start start 2024-09-10T08:21:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.762+0000 7f4812e4b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1c2c70 0x562a3a1c3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.762+0000 7f4812e4b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c3590 0x562a3a1cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.762+0000 7f4812e4b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1cde50 0x562a3a1d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.762+0000 7f4812e4b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562a3a1293e0 con 0x562a3a1cde50 2024-09-10T08:21:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.762+0000 7f4812e4b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562a3a0d0d80 con 0x562a3a1c3590 2024-09-10T08:21:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.762+0000 7f4812e4b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562a3a1253a0 con 0x562a3a1c2c70 2024-09-10T08:21:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.763+0000 7f480e200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1c2c70 0x562a3a1c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.763+0000 7f480e200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1c2c70 0x562a3a1c3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:40572/0 (socket says 172.21.15.2:40572) 2024-09-10T08:21:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.763+0000 7f480e200640 1 -- 172.21.15.2:0/1679915760 learned_addr learned my addr 172.21.15.2:0/1679915760 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.763+0000 7f480d800640 1 --2- 172.21.15.2:0/1679915760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c3590 0x562a3a1cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.763+0000 7f4806000640 1 -- 172.21.15.2:0/1326899850 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f47f40099d0 con 0x7f4808077040 2024-09-10T08:21:51.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.763+0000 7f480e200640 1 -- 172.21.15.2:0/1679915760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c3590 msgr2=0x562a3a1cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:51.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.763+0000 7f480e200640 1 --2- 172.21.15.2:0/1679915760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c3590 0x562a3a1cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.763+0000 7f480e200640 1 -- 172.21.15.2:0/1679915760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1cde50 msgr2=0x562a3a1d0240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:21:51.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.763+0000 7f480e200640 1 --2- 172.21.15.2:0/1679915760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1cde50 0x562a3a1d0240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.763+0000 7f480e200640 1 -- 172.21.15.2:0/1679915760 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a3a0efae0 con 0x562a3a1c2c70 2024-09-10T08:21:51.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.763+0000 7f480e200640 1 --2- 172.21.15.2:0/1679915760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1c2c70 0x562a3a1c3050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4800005e30 tx=0x7f4800009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cb66917a4830ea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:51.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.764+0000 7f47faa00640 1 -- 172.21.15.2:0/1679915760 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800031db0 con 0x562a3a1c2c70 2024-09-10T08:21:51.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.764+0000 7f47faa00640 1 -- 172.21.15.2:0/1679915760 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4800036040 con 0x562a3a1c2c70 2024-09-10T08:21:51.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.764+0000 7f47faa00640 1 -- 172.21.15.2:0/1679915760 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48000042a0 con 0x562a3a1c2c70 2024-09-10T08:21:51.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.764+0000 7f4806000640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47f00a2b80 0x7f47f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.764+0000 7f4812e4b740 1 -- 172.21.15.2:0/1679915760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1c2c70 msgr2=0x562a3a1c3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:51.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.764+0000 7f4812e4b740 1 --2- 172.21.15.2:0/1679915760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1c2c70 0x562a3a1c3050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4800005e30 tx=0x7f4800009800 comp rx=0 tx=0).stop 2024-09-10T08:21:51.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.765+0000 7f4806000640 1 -- 172.21.15.2:0/1326899850 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f47f40ceed0 con 0x7f4808077040 2024-09-10T08:21:51.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.765+0000 7f4812e4b740 1 -- 172.21.15.2:0/1679915760 shutdown_connections 2024-09-10T08:21:51.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.765+0000 7f4812e4b740 1 --2- 172.21.15.2:0/1679915760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1cde50 0x562a3a1d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.765+0000 7f4812e4b740 1 --2- 172.21.15.2:0/1679915760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c3590 0x562a3a1cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.765+0000 7f4812e4b740 1 --2- 172.21.15.2:0/1679915760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1c2c70 0x562a3a1c3050 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.765+0000 7f4812e4b740 1 -- 172.21.15.2:0/1679915760 >> 172.21.15.2:0/1679915760 conn(0x562a3a1bf200 msgr2=0x562a3a1bf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:51.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.765+0000 7f480d800640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47f00a2b80 0x7f47f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.765+0000 7f4812e4b740 1 -- 172.21.15.2:0/1679915760 shutdown_connections 2024-09-10T08:21:51.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.765+0000 7f4812e4b740 1 -- 172.21.15.2:0/1679915760 wait complete. 2024-09-10T08:21:51.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.766+0000 7f4812e4b740 1 -- start start 2024-09-10T08:21:51.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.766+0000 7f480d800640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47f00a2b80 0x7f47f00a5040 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009620 tx=0x7f47fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:51.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.766+0000 7f4812e4b740 1 Processor -- start 2024-09-10T08:21:51.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.766+0000 7f4812e4b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c2c70 0x562a3a1cada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.766+0000 7f4812e4b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c3590 0x562a3a1cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.766+0000 7f4812e4b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1cde50 0x562a3a1cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.766+0000 7f4812e4b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562a3a1d08a0 con 0x562a3a1c2c70 2024-09-10T08:21:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.766+0000 7f4812e4b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562a3a0e40f0 con 0x562a3a1c3590 2024-09-10T08:21:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.766+0000 7f4812e4b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562a3a1d0a20 con 0x562a3a1cde50 2024-09-10T08:21:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.766+0000 7f4812e4b740 10 client.0 _ll_register_callbacks cb 0x562a3a0cbbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.767+0000 7f480d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c2c70 0x562a3a1cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.767+0000 7f480d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c2c70 0x562a3a1cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58814/0 (socket says 172.21.15.2:58814) 2024-09-10T08:21:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.767+0000 7f480d800640 1 -- 172.21.15.2:0/2732554680 learned_addr learned my addr 172.21.15.2:0/2732554680 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.767+0000 7f480e200640 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1cde50 0x562a3a1cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.767+0000 7f480ce00640 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c3590 0x562a3a1cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.767+0000 7f480e200640 1 -- 172.21.15.2:0/2732554680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c3590 msgr2=0x562a3a1cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.767+0000 7f480e200640 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c3590 0x562a3a1cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.767+0000 7f480e200640 1 -- 172.21.15.2:0/2732554680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c2c70 msgr2=0x562a3a1cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.767+0000 7f480e200640 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c2c70 0x562a3a1cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.767+0000 7f480e200640 1 -- 172.21.15.2:0/2732554680 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a39e9a3c0 con 0x562a3a1cde50 2024-09-10T08:21:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.767+0000 7f480d800640 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c2c70 0x562a3a1cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.767+0000 7f480e200640 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1cde50 0x562a3a1cb820 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f480002ade0 tx=0x7f4800033000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6eb1e4b107ae1ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.767+0000 7f47ef400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:21:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.768+0000 7f47ef400640 1 -- 172.21.15.2:0/2732554680 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800003b80 con 0x562a3a1cde50 2024-09-10T08:21:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.768+0000 7f4812e4b740 1 -- 172.21.15.2:0/2732554680 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4800036040 con 0x562a3a1cde50 2024-09-10T08:21:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.768+0000 7f47ef400640 1 -- 172.21.15.2:0/2732554680 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4800003d20 con 0x562a3a1cde50 2024-09-10T08:21:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.768+0000 7f47ef400640 1 -- 172.21.15.2:0/2732554680 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f480005a050 con 0x562a3a1cde50 2024-09-10T08:21:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.768+0000 7f4812e4b740 10 client.29296 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.768+0000 7f4812e4b740 1 -- 172.21.15.2:0/2732554680 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a3a1d08a0 con 0x562a3a1cde50 2024-09-10T08:21:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.769+0000 7f47ef400640 1 -- 172.21.15.2:0/2732554680 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4800036040 con 0x562a3a1cde50 2024-09-10T08:21:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.769+0000 7f47ef400640 1 -- 172.21.15.2:0/2732554680 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4800055030 con 0x562a3a1cde50 2024-09-10T08:21:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.769+0000 7f4812e4b740 10 client.29296 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.769+0000 7f4812e4b740 10 client.29296 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:21:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.769+0000 7f4812e4b740 20 client.29296 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.769+0000 7f4812e4b740 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562a3a1ddac0 0x562a3a1dfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.769+0000 7f4812e4b740 4 client.29296 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.769+0000 7f4812e4b740 1 -- 172.21.15.2:0/2732554680 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a3a136950 con 0x562a3a1ddac0 2024-09-10T08:21:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.769+0000 7f480d800640 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562a3a1ddac0 0x562a3a1dfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.770+0000 7f480d800640 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562a3a1ddac0 0x562a3a1dfeb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d61e9ef090326f24 server_cookie=25dbe7ea793eb36b in_seq=0 out_seq=0 2024-09-10T08:21:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.770+0000 7f47ef400640 10 client.29296 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:21:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f47ef400640 1 -- 172.21.15.2:0/2732554680 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a3a136950 con 0x562a3a1ddac0 2024-09-10T08:21:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f47ef400640 10 client.29296 handle_command_reply: tid=0 2024-09-10T08:21:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f4812e4b740 1 client.29296 shutdown 2024-09-10T08:21:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f4812e4b740 1 -- 172.21.15.2:0/2732554680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1cde50 msgr2=0x562a3a1cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f4812e4b740 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1cde50 0x562a3a1cb820 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f480002ade0 tx=0x7f4800033000 comp rx=0 tx=0).stop 2024-09-10T08:21:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f4812e4b740 1 -- 172.21.15.2:0/2732554680 shutdown_connections 2024-09-10T08:21:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f4812e4b740 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1cde50 0x562a3a1cb820 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f4812e4b740 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c3590 0x562a3a1cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f4812e4b740 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562a3a1ddac0 0x562a3a1dfeb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f4812e4b740 1 --2- 172.21.15.2:0/2732554680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c2c70 0x562a3a1cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f4812e4b740 1 -- 172.21.15.2:0/2732554680 >> 172.21.15.2:0/2732554680 conn(0x562a3a1bf200 msgr2=0x562a3a1c4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f47ef400640 0 client.29296 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:21:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f4812e4b740 1 -- 172.21.15.2:0/2732554680 shutdown_connections 2024-09-10T08:21:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f4812e4b740 1 -- 172.21.15.2:0/2732554680 wait complete. 2024-09-10T08:21:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.771+0000 7f4812e4b740 20 client.29296 trim_cache size 0 max 16384 2024-09-10T08:21:51.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.785+0000 7f4812e4b740 1 Processor -- start 2024-09-10T08:21:51.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.785+0000 7f4812e4b740 1 -- start start 2024-09-10T08:21:51.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.785+0000 7f4812e4b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1c2c70 0x562a3a1cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.785+0000 7f4812e4b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c3590 0x562a3a1ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.785+0000 7f4812e4b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1cde50 0x562a3a1e5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.785+0000 7f4812e4b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b7c90 con 0x562a3a1c3590 2024-09-10T08:21:51.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.785+0000 7f4812e4b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b5560 con 0x562a3a1cde50 2024-09-10T08:21:51.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.785+0000 7f4812e4b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b33b0 con 0x562a3a1c2c70 2024-09-10T08:21:51.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.786+0000 7f480ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c3590 0x562a3a1ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.786+0000 7f480d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1c2c70 0x562a3a1cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.786+0000 7f480ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c3590 0x562a3a1ccd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58816/0 (socket says 172.21.15.2:58816) 2024-09-10T08:21:51.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.786+0000 7f480ce00640 1 -- 172.21.15.2:0/514203878 learned_addr learned my addr 172.21.15.2:0/514203878 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:51.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.786+0000 7f480e200640 1 --2- 172.21.15.2:0/514203878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1cde50 0x562a3a1e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.786+0000 7f480d800640 1 -- 172.21.15.2:0/514203878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1cde50 msgr2=0x562a3a1e5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:51.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.786+0000 7f480d800640 1 --2- 172.21.15.2:0/514203878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1cde50 0x562a3a1e5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.786+0000 7f480d800640 1 -- 172.21.15.2:0/514203878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c3590 msgr2=0x562a3a1ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:51.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.786+0000 7f480d800640 1 --2- 172.21.15.2:0/514203878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c3590 0x562a3a1ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.786+0000 7f480d800640 1 -- 172.21.15.2:0/514203878 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a3a1c02b0 con 0x562a3a1c2c70 2024-09-10T08:21:51.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.786+0000 7f480d800640 1 --2- 172.21.15.2:0/514203878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1c2c70 0x562a3a1cb670 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x562a3a1e0490 tx=0x7f47fc01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ef8f62259edbf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:51.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.787+0000 7f47fa000640 1 -- 172.21.15.2:0/514203878 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00fe00 con 0x562a3a1c2c70 2024-09-10T08:21:51.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.787+0000 7f47fa000640 1 -- 172.21.15.2:0/514203878 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47fc005500 con 0x562a3a1c2c70 2024-09-10T08:21:51.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.787+0000 7f47fa000640 1 -- 172.21.15.2:0/514203878 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc004e50 con 0x562a3a1c2c70 2024-09-10T08:21:51.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.787+0000 7f4812e4b740 1 -- 172.21.15.2:0/514203878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1c2c70 msgr2=0x562a3a1cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:51.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.787+0000 7f4812e4b740 1 --2- 172.21.15.2:0/514203878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1c2c70 0x562a3a1cb670 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x562a3a1e0490 tx=0x7f47fc01b000 comp rx=0 tx=0).stop 2024-09-10T08:21:51.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.788+0000 7f4812e4b740 1 -- 172.21.15.2:0/514203878 shutdown_connections 2024-09-10T08:21:51.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.788+0000 7f4812e4b740 1 --2- 172.21.15.2:0/514203878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1cde50 0x562a3a1e5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.788+0000 7f4812e4b740 1 --2- 172.21.15.2:0/514203878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c3590 0x562a3a1ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.788+0000 7f4812e4b740 1 --2- 172.21.15.2:0/514203878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1c2c70 0x562a3a1cb670 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.788+0000 7f4812e4b740 1 -- 172.21.15.2:0/514203878 >> 172.21.15.2:0/514203878 conn(0x562a3a1bf200 msgr2=0x562a3a1cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:51.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.788+0000 7f4812e4b740 1 -- 172.21.15.2:0/514203878 shutdown_connections 2024-09-10T08:21:51.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.788+0000 7f4812e4b740 1 -- 172.21.15.2:0/514203878 wait complete. 2024-09-10T08:21:51.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.788+0000 7f4812e4b740 1 -- start start 2024-09-10T08:21:51.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.788+0000 7f4812e4b740 1 Processor -- start 2024-09-10T08:21:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.789+0000 7f4812e4b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c2c70 0x562a3a1d60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.789+0000 7f4812e4b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c3590 0x562a3a1c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.789+0000 7f4812e4b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1cde50 0x562a3a1c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.789+0000 7f4812e4b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b3a20 con 0x562a3a1c3590 2024-09-10T08:21:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.789+0000 7f4812e4b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b4e90 con 0x562a3a1c2c70 2024-09-10T08:21:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.789+0000 7f4812e4b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b40f0 con 0x562a3a1cde50 2024-09-10T08:21:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.789+0000 7f4812e4b740 10 client.0 _ll_register_callbacks cb 0x7f47fc009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.789+0000 7f480ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c3590 0x562a3a1c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.789+0000 7f480ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c3590 0x562a3a1c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58830/0 (socket says 172.21.15.2:58830) 2024-09-10T08:21:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.789+0000 7f480ce00640 1 -- 172.21.15.2:0/1381496837 learned_addr learned my addr 172.21.15.2:0/1381496837 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.789+0000 7f480d800640 1 --2- 172.21.15.2:0/1381496837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c2c70 0x562a3a1d60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.789+0000 7f480e200640 1 --2- 172.21.15.2:0/1381496837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1cde50 0x562a3a1c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.790+0000 7f480ce00640 1 -- 172.21.15.2:0/1381496837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1cde50 msgr2=0x562a3a1c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:51.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.790+0000 7f480ce00640 1 --2- 172.21.15.2:0/1381496837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1cde50 0x562a3a1c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.790+0000 7f480ce00640 1 -- 172.21.15.2:0/1381496837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c2c70 msgr2=0x562a3a1d60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:51.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.790+0000 7f480ce00640 1 --2- 172.21.15.2:0/1381496837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c2c70 0x562a3a1d60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.790+0000 7f480ce00640 1 -- 172.21.15.2:0/1381496837 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a3a1ba770 con 0x562a3a1c3590 2024-09-10T08:21:51.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.790+0000 7f480ce00640 1 --2- 172.21.15.2:0/1381496837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c3590 0x562a3a1c7880 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f47f400d6a0 tx=0x7f47f409ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfca584b3bcb0ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:51.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.790+0000 7f47ef400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:51.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.790+0000 7f47ef400640 1 -- 172.21.15.2:0/1381496837 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f40b4060 con 0x562a3a1c3590 2024-09-10T08:21:51.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.790+0000 7f47ef400640 1 -- 172.21.15.2:0/1381496837 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f47f40b4200 con 0x562a3a1c3590 2024-09-10T08:21:51.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.790+0000 7f4812e4b740 1 -- 172.21.15.2:0/1381496837 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47fc005500 con 0x562a3a1c3590 2024-09-10T08:21:51.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.790+0000 7f47ef400640 1 -- 172.21.15.2:0/1381496837 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47f4020040 con 0x562a3a1c3590 2024-09-10T08:21:51.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.791+0000 7f4812e4b740 10 client.19347 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:51.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.791+0000 7f4812e4b740 1 -- 172.21.15.2:0/1381496837 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a3a248320 con 0x562a3a1c3590 2024-09-10T08:21:51.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.791+0000 7f47ef400640 1 -- 172.21.15.2:0/1381496837 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47fc005500 con 0x562a3a1c3590 2024-09-10T08:21:51.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.791+0000 7f47ef400640 1 -- 172.21.15.2:0/1381496837 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f47f4097140 con 0x562a3a1c3590 2024-09-10T08:21:51.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.791+0000 7f4812e4b740 10 client.19347 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:51.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.791+0000 7f4812e4b740 10 client.19347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:21:51.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.791+0000 7f4812e4b740 20 client.19347 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:51.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.791+0000 7f4812e4b740 1 --2- 172.21.15.2:0/1381496837 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562a3a248a80 0x562a3a24aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:51.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.791+0000 7f4812e4b740 4 client.19347 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:21:51.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.791+0000 7f4812e4b740 1 -- 172.21.15.2:0/1381496837 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a3a130e60 con 0x562a3a248a80 2024-09-10T08:21:51.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.792+0000 7f480d800640 1 --2- 172.21.15.2:0/1381496837 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562a3a248a80 0x562a3a24aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:51.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.792+0000 7f480d800640 1 --2- 172.21.15.2:0/1381496837 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562a3a248a80 0x562a3a24aea0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65fd67d827653d4c server_cookie=4d2d921320ed4841 in_seq=0 out_seq=0 2024-09-10T08:21:51.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.792+0000 7f47ef400640 10 client.19347 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:21:51.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.792+0000 7f47ef400640 1 -- 172.21.15.2:0/1381496837 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a3a130e60 con 0x562a3a248a80 2024-09-10T08:21:51.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.792+0000 7f47ef400640 10 client.19347 handle_command_reply: tid=0 2024-09-10T08:21:51.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.792+0000 7f4812e4b740 1 client.19347 shutdown 2024-09-10T08:21:51.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.792+0000 7f4812e4b740 1 -- 172.21.15.2:0/1381496837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c3590 msgr2=0x562a3a1c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:51.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.792+0000 7f4812e4b740 1 --2- 172.21.15.2:0/1381496837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c3590 0x562a3a1c7880 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f47f400d6a0 tx=0x7f47f409ae10 comp rx=0 tx=0).stop 2024-09-10T08:21:51.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.793+0000 7f4812e4b740 1 -- 172.21.15.2:0/1381496837 shutdown_connections 2024-09-10T08:21:51.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.793+0000 7f4812e4b740 1 --2- 172.21.15.2:0/1381496837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a3a1cde50 0x562a3a1c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.793+0000 7f4812e4b740 1 --2- 172.21.15.2:0/1381496837 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562a3a248a80 0x562a3a24aea0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.793+0000 7f4812e4b740 1 --2- 172.21.15.2:0/1381496837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562a3a1c3590 0x562a3a1c7880 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.793+0000 7f4812e4b740 1 --2- 172.21.15.2:0/1381496837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a3a1c2c70 0x562a3a1d60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.793+0000 7f47ef400640 0 client.19347 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:21:51.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.793+0000 7f4812e4b740 1 -- 172.21.15.2:0/1381496837 >> 172.21.15.2:0/1381496837 conn(0x562a3a1bf200 msgr2=0x562a3a1be380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:51.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.793+0000 7f4812e4b740 1 -- 172.21.15.2:0/1381496837 shutdown_connections 2024-09-10T08:21:51.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.793+0000 7f4812e4b740 1 -- 172.21.15.2:0/1381496837 wait complete. 2024-09-10T08:21:51.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.793+0000 7f4812e4b740 20 client.19347 trim_cache size 0 max 16384 2024-09-10T08:21:51.797 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:21:51.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.793+0000 7f4810200640 1 -- 172.21.15.2:0/1326899850 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47f00a2b80 msgr2=0x7f47f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:51.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.793+0000 7f4810200640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47f00a2b80 0x7f47f00a5040 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009620 tx=0x7f47fc005f90 comp rx=0 tx=0).stop 2024-09-10T08:21:51.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.794+0000 7f4810200640 1 -- 172.21.15.2:0/1326899850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4808077040 msgr2=0x7f48081a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:51.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.794+0000 7f4810200640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4808077040 0x7f48081a4bf0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f47f400d6d0 tx=0x7f47f400db90 comp rx=0 tx=0).stop 2024-09-10T08:21:51.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.794+0000 7f4810200640 1 -- 172.21.15.2:0/1326899850 shutdown_connections 2024-09-10T08:21:51.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.794+0000 7f4810200640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f47f00a2b80 0x7f47f00a5040 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.794+0000 7f4810200640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4808112a30 0x7f480819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.794+0000 7f4810200640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4808077040 0x7f48081a4bf0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.794+0000 7f4810200640 1 --2- 172.21.15.2:0/1326899850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4808076720 0x7f48081a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.794+0000 7f4810200640 1 -- 172.21.15.2:0/1326899850 >> 172.21.15.2:0/1326899850 conn(0x7f48080ffa90 msgr2=0x7f4808101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.794+0000 7f4810200640 1 -- 172.21.15.2:0/1326899850 shutdown_connections 2024-09-10T08:21:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:51.794+0000 7f4810200640 1 -- 172.21.15.2:0/1326899850 wait complete. 2024-09-10T08:21:52.814 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:21:53.219 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:53 smithi002 ceph-mon[30563]: pgmap v799: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-09-10T08:21:53.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:53 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:21:53.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:53 smithi155 ceph-mon[35405]: pgmap v799: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-09-10T08:21:53.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:53 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:21:53.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:53 smithi107 ceph-mon[35048]: pgmap v799: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-09-10T08:21:53.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:53 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:21:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.738+0000 7f1d2b400640 1 -- 172.21.15.2:0/2561651798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d24105910 msgr2=0x7f1d2410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.738+0000 7f1d2b400640 1 --2- 172.21.15.2:0/2561651798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d24105910 0x7f1d2410c1a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009ca0 tx=0x7f1d1c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.738+0000 7f1d2b400640 1 -- 172.21.15.2:0/2561651798 shutdown_connections 2024-09-10T08:21:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.738+0000 7f1d2b400640 1 --2- 172.21.15.2:0/2561651798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d24105910 0x7f1d2410c1a0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.738+0000 7f1d2b400640 1 --2- 172.21.15.2:0/2561651798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d24104f50 0x7f1d241053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.738+0000 7f1d2b400640 1 --2- 172.21.15.2:0/2561651798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d24103d60 0x7f1d24104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.738+0000 7f1d2b400640 1 -- 172.21.15.2:0/2561651798 >> 172.21.15.2:0/2561651798 conn(0x7f1d240ff4e0 msgr2=0x7f1d24101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.738+0000 7f1d2b400640 1 -- 172.21.15.2:0/2561651798 shutdown_connections 2024-09-10T08:21:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.738+0000 7f1d2b400640 1 -- 172.21.15.2:0/2561651798 wait complete. 2024-09-10T08:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.739+0000 7f1d2b400640 1 Processor -- start 2024-09-10T08:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.739+0000 7f1d2b400640 1 -- start start 2024-09-10T08:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.739+0000 7f1d2b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d24103d60 0x7f1d24199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d2b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d24104f50 0x7f1d2419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d2b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d24105910 0x7f1d241a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d2b400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2410fa30 con 0x7f1d24105910 2024-09-10T08:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d2b400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2410f8b0 con 0x7f1d24103d60 2024-09-10T08:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d2b400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2410fbb0 con 0x7f1d24104f50 2024-09-10T08:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d29400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d24105910 0x7f1d241a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d23e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d24104f50 0x7f1d2419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d23e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d24104f50 0x7f1d2419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:40606/0 (socket says 172.21.15.2:40606) 2024-09-10T08:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d29400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d24105910 0x7f1d241a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58850/0 (socket says 172.21.15.2:58850) 2024-09-10T08:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d23e00640 1 -- 172.21.15.2:0/1376252671 learned_addr learned my addr 172.21.15.2:0/1376252671 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d28a00640 1 --2- 172.21.15.2:0/1376252671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d24103d60 0x7f1d24199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d23e00640 1 -- 172.21.15.2:0/1376252671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d24103d60 msgr2=0x7f1d24199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d23e00640 1 --2- 172.21.15.2:0/1376252671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d24103d60 0x7f1d24199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d23e00640 1 -- 172.21.15.2:0/1376252671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d24105910 msgr2=0x7f1d241a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d23e00640 1 --2- 172.21.15.2:0/1376252671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d24105910 0x7f1d241a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.740+0000 7f1d23e00640 1 -- 172.21.15.2:0/1376252671 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d241a1bb0 con 0x7f1d24104f50 2024-09-10T08:21:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.741+0000 7f1d23e00640 1 --2- 172.21.15.2:0/1376252671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d24104f50 0x7f1d2419a4d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000e9f0 tx=0x7f1d1000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.741+0000 7f1d21600640 1 -- 172.21.15.2:0/1376252671 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1000cd80 con 0x7f1d24104f50 2024-09-10T08:21:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.741+0000 7f1d2b400640 1 -- 172.21.15.2:0/1376252671 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d241a1ea0 con 0x7f1d24104f50 2024-09-10T08:21:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.741+0000 7f1d21600640 1 -- 172.21.15.2:0/1376252671 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d10010910 con 0x7f1d24104f50 2024-09-10T08:21:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.741+0000 7f1d21600640 1 -- 172.21.15.2:0/1376252671 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d100188f0 con 0x7f1d24104f50 2024-09-10T08:21:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.741+0000 7f1d2b400640 1 -- 172.21.15.2:0/1376252671 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d241a23b0 con 0x7f1d24104f50 2024-09-10T08:21:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.744+0000 7f1d2e2a1740 1 Processor -- start 2024-09-10T08:21:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.745+0000 7f1d2e2a1740 1 -- start start 2024-09-10T08:21:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.745+0000 7f1d2e2a1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f43490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.745+0000 7f1d2e2a1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f439d0 0x558287f48490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.745+0000 7f1d2e2a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 0x558287f4af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.745+0000 7f1d2e2a1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558287ea8100 con 0x558287f43090 2024-09-10T08:21:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.745+0000 7f1d2e2a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558287e501b0 con 0x558287f48a60 2024-09-10T08:21:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.745+0000 7f1d2e2a1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558287ea40c0 con 0x558287f439d0 2024-09-10T08:21:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.745+0000 7f1d29400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f43490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.745+0000 7f1d29400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f43490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58862/0 (socket says 172.21.15.2:58862) 2024-09-10T08:21:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.745+0000 7f1d29400640 1 -- 172.21.15.2:0/1684645969 learned_addr learned my addr 172.21.15.2:0/1684645969 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.746+0000 7f1d29400640 1 -- 172.21.15.2:0/1684645969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f439d0 msgr2=0x558287f48490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:21:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.746+0000 7f1d29400640 1 --2- 172.21.15.2:0/1684645969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f439d0 0x558287f48490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.746+0000 7f1d29400640 1 -- 172.21.15.2:0/1684645969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 msgr2=0x558287f4af20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:21:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.746+0000 7f1d29400640 1 --2- 172.21.15.2:0/1684645969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 0x558287f4af20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.746+0000 7f1d29400640 1 -- 172.21.15.2:0/1684645969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558287bbb210 con 0x558287f43090 2024-09-10T08:21:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.746+0000 7f1d21600640 1 -- 172.21.15.2:0/1376252671 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1d10010ac0 con 0x7f1d24104f50 2024-09-10T08:21:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.746+0000 7f1d29400640 1 --2- 172.21.15.2:0/1684645969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f43490 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009ca0 tx=0x7f1d1c0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3b4d348da7e3e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.747+0000 7f1d15c00640 1 -- 172.21.15.2:0/1684645969 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c031650 con 0x558287f43090 2024-09-10T08:21:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.747+0000 7f1d15c00640 1 -- 172.21.15.2:0/1684645969 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d1c036030 con 0x558287f43090 2024-09-10T08:21:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.747+0000 7f1d15c00640 1 -- 172.21.15.2:0/1684645969 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0046e0 con 0x558287f43090 2024-09-10T08:21:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.748+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/1684645969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 msgr2=0x558287f43490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.748+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/1684645969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f43490 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009ca0 tx=0x7f1d1c0093c0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.748+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/1684645969 shutdown_connections 2024-09-10T08:21:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.748+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/1684645969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 0x558287f4af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.748+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/1684645969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f439d0 0x558287f48490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.748+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/1684645969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f43490 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.748+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/1684645969 >> 172.21.15.2:0/1684645969 conn(0x558287f3e700 msgr2=0x558287f3eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.748+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/1684645969 shutdown_connections 2024-09-10T08:21:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.748+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/1684645969 wait complete. 2024-09-10T08:21:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.748+0000 7f1d21600640 1 --2- 172.21.15.2:0/1376252671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d0c0a2b40 0x7f1d0c0a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.749+0000 7f1d21600640 1 -- 172.21.15.2:0/1376252671 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1d100cf7c0 con 0x7f1d24104f50 2024-09-10T08:21:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.749+0000 7f1d2e2a1740 1 -- start start 2024-09-10T08:21:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.749+0000 7f1d29400640 1 --2- 172.21.15.2:0/1376252671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d0c0a2b40 0x7f1d0c0a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.749+0000 7f1d2e2a1740 1 Processor -- start 2024-09-10T08:21:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.749+0000 7f1d2e2a1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f43090 0x558287f4d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.749+0000 7f1d2e2a1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f439d0 0x558287f4d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.749+0000 7f1d2e2a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 0x558287f54650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.749+0000 7f1d2e2a1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558287f4b850 con 0x558287f439d0 2024-09-10T08:21:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.749+0000 7f1d2e2a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558287e31090 con 0x558287f48a60 2024-09-10T08:21:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.749+0000 7f1d2e2a1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558287f4b9d0 con 0x558287f43090 2024-09-10T08:21:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.750+0000 7f1d2e2a1740 10 client.0 _ll_register_callbacks cb 0x558287e85f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.750+0000 7f1d23e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f439d0 0x558287f4d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.750+0000 7f1d29400640 1 --2- 172.21.15.2:0/1376252671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d0c0a2b40 0x7f1d0c0a5000 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c02ade0 tx=0x7f1d1c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.750+0000 7f1d23e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f439d0 0x558287f4d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58876/0 (socket says 172.21.15.2:58876) 2024-09-10T08:21:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.750+0000 7f1d23e00640 1 -- 172.21.15.2:0/3606909915 learned_addr learned my addr 172.21.15.2:0/3606909915 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.750+0000 7f1d29400640 1 --2- 172.21.15.2:0/3606909915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 0x558287f54650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.750+0000 7f1d28a00640 1 --2- 172.21.15.2:0/3606909915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f43090 0x558287f4d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.750+0000 7f1d23e00640 1 -- 172.21.15.2:0/3606909915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f43090 msgr2=0x558287f4d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.750+0000 7f1d23e00640 1 --2- 172.21.15.2:0/3606909915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f43090 0x558287f4d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.750+0000 7f1d23e00640 1 -- 172.21.15.2:0/3606909915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 msgr2=0x558287f54650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.750+0000 7f1d23e00640 1 --2- 172.21.15.2:0/3606909915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 0x558287f54650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.750+0000 7f1d23e00640 1 -- 172.21.15.2:0/3606909915 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558287c87620 con 0x558287f439d0 2024-09-10T08:21:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.751+0000 7f1d23e00640 1 --2- 172.21.15.2:0/3606909915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f439d0 0x558287f4d610 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f1d100097b0 tx=0x7f1d1009b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85b733c366a6159 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.751+0000 7f1d0a000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.751+0000 7f1d0a000640 1 -- 172.21.15.2:0/3606909915 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1009b330 con 0x558287f439d0 2024-09-10T08:21:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.751+0000 7f1d0a000640 1 -- 172.21.15.2:0/3606909915 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1d10010430 con 0x558287f439d0 2024-09-10T08:21:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.751+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3606909915 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d1c036030 con 0x558287f439d0 2024-09-10T08:21:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.751+0000 7f1d0a000640 1 -- 172.21.15.2:0/3606909915 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d100a48a0 con 0x558287f439d0 2024-09-10T08:21:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.751+0000 7f1d2e2a1740 10 client.19359 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.751+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3606909915 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558287f574e0 con 0x558287f439d0 2024-09-10T08:21:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.751+0000 7f1d0a000640 1 -- 172.21.15.2:0/3606909915 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d1c036030 con 0x558287f439d0 2024-09-10T08:21:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.752+0000 7f1d0a000640 1 -- 172.21.15.2:0/3606909915 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1d10097b50 con 0x558287f439d0 2024-09-10T08:21:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.752+0000 7f1d2e2a1740 10 client.19359 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.752+0000 7f1d2e2a1740 10 client.19359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.752+0000 7f1d2e2a1740 20 client.19359 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.752+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/3606909915 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558287f58b60 0x558287f5af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.752+0000 7f1d2e2a1740 4 client.19359 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.752+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3606909915 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558287eb5830 con 0x558287f58b60 2024-09-10T08:21:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.752+0000 7f1d28a00640 1 --2- 172.21.15.2:0/3606909915 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558287f58b60 0x558287f5af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.753+0000 7f1d28a00640 1 --2- 172.21.15.2:0/3606909915 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558287f58b60 0x558287f5af50 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff671eb2988850dc server_cookie=e195d61be1e3affa in_seq=0 out_seq=0 2024-09-10T08:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.753+0000 7f1d0a000640 10 client.19359 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.753+0000 7f1d0a000640 1 -- 172.21.15.2:0/3606909915 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558287eb5830 con 0x558287f58b60 2024-09-10T08:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.753+0000 7f1d0a000640 10 client.19359 handle_command_reply: tid=0 2024-09-10T08:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.753+0000 7f1d2e2a1740 1 client.19359 shutdown 2024-09-10T08:21:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.754+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3606909915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f439d0 msgr2=0x558287f4d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.754+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/3606909915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f439d0 0x558287f4d610 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f1d100097b0 tx=0x7f1d1009b6c0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.754+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3606909915 shutdown_connections 2024-09-10T08:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.754+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/3606909915 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558287f58b60 0x558287f5af50 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.754+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/3606909915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 0x558287f54650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.754+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/3606909915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f439d0 0x558287f4d610 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.754+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/3606909915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f43090 0x558287f4d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.754+0000 7f1d0a000640 0 client.19359 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.754+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3606909915 >> 172.21.15.2:0/3606909915 conn(0x558287f3e700 msgr2=0x558287f3ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.754+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3606909915 shutdown_connections 2024-09-10T08:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.754+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3606909915 wait complete. 2024-09-10T08:21:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.754+0000 7f1d2e2a1740 20 client.19359 trim_cache size 0 max 16384 2024-09-10T08:21:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.769+0000 7f1d2e2a1740 1 Processor -- start 2024-09-10T08:21:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.769+0000 7f1d2e2a1740 1 -- start start 2024-09-10T08:21:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.769+0000 7f1d2e2a1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f3f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.769+0000 7f1d2e2a1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f439d0 0x558287f54610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.769+0000 7f1d2e2a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 0x558287f3faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.769+0000 7f1d2e2a1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241af280 con 0x558287f43090 2024-09-10T08:21:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.769+0000 7f1d2e2a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241acb50 con 0x558287f48a60 2024-09-10T08:21:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.769+0000 7f1d2e2a1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241aa9a0 con 0x558287f439d0 2024-09-10T08:21:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.770+0000 7f1d28a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f3f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.770+0000 7f1d28a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f3f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58880/0 (socket says 172.21.15.2:58880) 2024-09-10T08:21:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.770+0000 7f1d23e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f439d0 0x558287f54610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.770+0000 7f1d28a00640 1 -- 172.21.15.2:0/397754667 learned_addr learned my addr 172.21.15.2:0/397754667 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.770+0000 7f1d29400640 1 --2- 172.21.15.2:0/397754667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 0x558287f3faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.770+0000 7f1d28a00640 1 -- 172.21.15.2:0/397754667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f439d0 msgr2=0x558287f54610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.770+0000 7f1d28a00640 1 --2- 172.21.15.2:0/397754667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f439d0 0x558287f54610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.770+0000 7f1d28a00640 1 -- 172.21.15.2:0/397754667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 msgr2=0x558287f3faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.770+0000 7f1d28a00640 1 --2- 172.21.15.2:0/397754667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 0x558287f3faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.770+0000 7f1d28a00640 1 -- 172.21.15.2:0/397754667 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558287f3e520 con 0x558287f43090 2024-09-10T08:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.771+0000 7f1d28a00640 1 --2- 172.21.15.2:0/397754667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f3f5b0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x558287f44400 tx=0x7f1d18010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f859397973f966 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.771+0000 7f1d15200640 1 -- 172.21.15.2:0/397754667 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18022070 con 0x558287f43090 2024-09-10T08:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.771+0000 7f1d15200640 1 -- 172.21.15.2:0/397754667 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d180035f0 con 0x558287f43090 2024-09-10T08:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.771+0000 7f1d15200640 1 -- 172.21.15.2:0/397754667 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1801c680 con 0x558287f43090 2024-09-10T08:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.772+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/397754667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 msgr2=0x558287f3f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.772+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/397754667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f3f5b0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x558287f44400 tx=0x7f1d18010800 comp rx=0 tx=0).stop 2024-09-10T08:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.772+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/397754667 shutdown_connections 2024-09-10T08:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.772+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/397754667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f48a60 0x558287f3faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.772+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/397754667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f439d0 0x558287f54610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.772+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/397754667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f3f5b0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.772+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/397754667 >> 172.21.15.2:0/397754667 conn(0x558287f3e700 msgr2=0x558287f4d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.772+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/397754667 shutdown_connections 2024-09-10T08:21:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.772+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/397754667 wait complete. 2024-09-10T08:21:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.772+0000 7f1d2e2a1740 1 -- start start 2024-09-10T08:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.772+0000 7f1d2e2a1740 1 Processor -- start 2024-09-10T08:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d2e2a1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f51df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d2e2a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f439d0 0x558287f52330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d2e2a1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f48a60 0x558287f53340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d2e2a1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241ab010 con 0x558287f43090 2024-09-10T08:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d2e2a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241ac480 con 0x558287f439d0 2024-09-10T08:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d2e2a1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241ab6e0 con 0x558287f48a60 2024-09-10T08:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d2e2a1740 10 client.0 _ll_register_callbacks cb 0x7f1d10009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d28a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f51df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d28a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f51df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58892/0 (socket says 172.21.15.2:58892) 2024-09-10T08:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d28a00640 1 -- 172.21.15.2:0/3200274544 learned_addr learned my addr 172.21.15.2:0/3200274544 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d23e00640 1 --2- 172.21.15.2:0/3200274544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f439d0 0x558287f52330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d29400640 1 --2- 172.21.15.2:0/3200274544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f48a60 0x558287f53340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d28a00640 1 -- 172.21.15.2:0/3200274544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f48a60 msgr2=0x558287f53340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d28a00640 1 --2- 172.21.15.2:0/3200274544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f48a60 0x558287f53340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d28a00640 1 -- 172.21.15.2:0/3200274544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f439d0 msgr2=0x558287f52330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d28a00640 1 --2- 172.21.15.2:0/3200274544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f439d0 0x558287f52330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.773+0000 7f1d28a00640 1 -- 172.21.15.2:0/3200274544 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558287f53950 con 0x558287f43090 2024-09-10T08:21:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.774+0000 7f1d28a00640 1 --2- 172.21.15.2:0/3200274544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f51df0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f1d18003e40 tx=0x7f1d18009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fb63138c5ef7cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.774+0000 7f1d0a000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.774+0000 7f1d0a000640 1 -- 172.21.15.2:0/3200274544 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1802b070 con 0x558287f43090 2024-09-10T08:21:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.774+0000 7f1d0a000640 1 -- 172.21.15.2:0/3200274544 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1d180037a0 con 0x558287f43090 2024-09-10T08:21:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.774+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3200274544 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d180035f0 con 0x558287f43090 2024-09-10T08:21:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.774+0000 7f1d0a000640 1 -- 172.21.15.2:0/3200274544 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d18036050 con 0x558287f43090 2024-09-10T08:21:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.774+0000 7f1d2e2a1740 10 client.19365 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.774+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3200274544 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558287fc3880 con 0x558287f43090 2024-09-10T08:21:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.775+0000 7f1d0a000640 1 -- 172.21.15.2:0/3200274544 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d180035f0 con 0x558287f43090 2024-09-10T08:21:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.775+0000 7f1d0a000640 1 -- 172.21.15.2:0/3200274544 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1d18004a30 con 0x558287f43090 2024-09-10T08:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.775+0000 7f1d2e2a1740 10 client.19365 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.775+0000 7f1d2e2a1740 10 client.19365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.775+0000 7f1d2e2a1740 20 client.19365 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.775+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/3200274544 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558287fc41b0 0x558287fc65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.775+0000 7f1d2e2a1740 4 client.19365 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.775+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3200274544 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558287eafb80 con 0x558287fc41b0 2024-09-10T08:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.775+0000 7f1d23e00640 1 --2- 172.21.15.2:0/3200274544 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558287fc41b0 0x558287fc65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.776+0000 7f1d23e00640 1 --2- 172.21.15.2:0/3200274544 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558287fc41b0 0x558287fc65d0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef380471507b24d3 server_cookie=62a88c46c4824d52 in_seq=0 out_seq=0 2024-09-10T08:21:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.776+0000 7f1d0a000640 10 client.19365 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d0a000640 1 -- 172.21.15.2:0/3200274544 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558287eafb80 con 0x558287fc41b0 2024-09-10T08:21:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d0a000640 10 client.19365 handle_command_reply: tid=0 2024-09-10T08:21:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d2e2a1740 1 client.19365 shutdown 2024-09-10T08:21:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3200274544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 msgr2=0x558287f51df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/3200274544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f51df0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f1d18003e40 tx=0x7f1d18009da0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3200274544 shutdown_connections 2024-09-10T08:21:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/3200274544 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558287fc41b0 0x558287fc65d0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/3200274544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558287f48a60 0x558287f53340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/3200274544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558287f439d0 0x558287f52330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d2e2a1740 1 --2- 172.21.15.2:0/3200274544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558287f43090 0x558287f51df0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d0a000640 0 client.19365 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3200274544 >> 172.21.15.2:0/3200274544 conn(0x558287f3e700 msgr2=0x558287f3d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3200274544 shutdown_connections 2024-09-10T08:21:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.785+0000 7f1d2e2a1740 1 -- 172.21.15.2:0/3200274544 wait complete. 2024-09-10T08:21:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.786+0000 7f1d2e2a1740 20 client.19365 trim_cache size 0 max 16384 2024-09-10T08:21:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:21:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:21:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:21:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:21:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.786+0000 7f1d2b400640 1 -- 172.21.15.2:0/1376252671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d0c0a2b40 msgr2=0x7f1d0c0a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.786+0000 7f1d2b400640 1 --2- 172.21.15.2:0/1376252671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d0c0a2b40 0x7f1d0c0a5000 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c02ade0 tx=0x7f1d1c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.786+0000 7f1d2b400640 1 -- 172.21.15.2:0/1376252671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d24104f50 msgr2=0x7f1d2419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.786+0000 7f1d2b400640 1 --2- 172.21.15.2:0/1376252671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d24104f50 0x7f1d2419a4d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000e9f0 tx=0x7f1d1000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.786+0000 7f1d2b400640 1 -- 172.21.15.2:0/1376252671 shutdown_connections 2024-09-10T08:21:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.786+0000 7f1d2b400640 1 --2- 172.21.15.2:0/1376252671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d24105910 0x7f1d241a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.786+0000 7f1d2b400640 1 --2- 172.21.15.2:0/1376252671 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1d0c0a2b40 0x7f1d0c0a5000 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.786+0000 7f1d2b400640 1 --2- 172.21.15.2:0/1376252671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d24104f50 0x7f1d2419a4d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.786+0000 7f1d2b400640 1 --2- 172.21.15.2:0/1376252671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d24103d60 0x7f1d24199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.787+0000 7f1d2b400640 1 -- 172.21.15.2:0/1376252671 >> 172.21.15.2:0/1376252671 conn(0x7f1d240ff4e0 msgr2=0x7f1d241081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.787+0000 7f1d2b400640 1 -- 172.21.15.2:0/1376252671 shutdown_connections 2024-09-10T08:21:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:21:53.787+0000 7f1d2b400640 1 -- 172.21.15.2:0/1376252671 wait complete. 2024-09-10T08:21:53.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.863+0000 7f511e200640 1 -- 172.21.15.2:0/2545694414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118103e30 msgr2=0x7f5118104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.863+0000 7f511e200640 1 --2- 172.21.15.2:0/2545694414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118103e30 0x7f5118104210 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f510c009900 tx=0x7f510c02a410 comp rx=0 tx=0).stop 2024-09-10T08:21:53.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.863+0000 7f511e200640 1 -- 172.21.15.2:0/2545694414 shutdown_connections 2024-09-10T08:21:53.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.863+0000 7f511e200640 1 --2- 172.21.15.2:0/2545694414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5118108f60 0x7f5118110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.863+0000 7f511e200640 1 --2- 172.21.15.2:0/2545694414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51181047e0 0x7f5118108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.863+0000 7f511e200640 1 --2- 172.21.15.2:0/2545694414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118103e30 0x7f5118104210 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.863+0000 7f511e200640 1 -- 172.21.15.2:0/2545694414 >> 172.21.15.2:0/2545694414 conn(0x7f51180ff910 msgr2=0x7f5118101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:53.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.863+0000 7f511e200640 1 -- 172.21.15.2:0/2545694414 shutdown_connections 2024-09-10T08:21:53.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.863+0000 7f511e200640 1 -- 172.21.15.2:0/2545694414 wait complete. 2024-09-10T08:21:53.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.864+0000 7f511e200640 1 Processor -- start 2024-09-10T08:21:53.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.865+0000 7f511e200640 1 -- start start 2024-09-10T08:21:53.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.865+0000 7f511e200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118103e30 0x7f511819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.865+0000 7f511e200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51181047e0 0x7f511819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.865+0000 7f511e200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5118108f60 0x7f511819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.865+0000 7f511e200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5118113f80 con 0x7f5118103e30 2024-09-10T08:21:53.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.865+0000 7f511e200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5118113e00 con 0x7f51181047e0 2024-09-10T08:21:53.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.865+0000 7f511e200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5118114100 con 0x7f5118108f60 2024-09-10T08:21:53.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.865+0000 7f511ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118103e30 0x7f511819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.865+0000 7f511d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5118108f60 0x7f511819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.866+0000 7f5117e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51181047e0 0x7f511819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.866+0000 7f511ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118103e30 0x7f511819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58918/0 (socket says 172.21.15.2:58918) 2024-09-10T08:21:53.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.866+0000 7f511ce00640 1 -- 172.21.15.2:0/1216520438 learned_addr learned my addr 172.21.15.2:0/1216520438 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:53.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.866+0000 7f5117e00640 1 -- 172.21.15.2:0/1216520438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5118108f60 msgr2=0x7f511819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.866+0000 7f5117e00640 1 --2- 172.21.15.2:0/1216520438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5118108f60 0x7f511819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.866+0000 7f5117e00640 1 -- 172.21.15.2:0/1216520438 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118103e30 msgr2=0x7f511819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.866+0000 7f5117e00640 1 --2- 172.21.15.2:0/1216520438 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118103e30 0x7f511819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.866+0000 7f5117e00640 1 -- 172.21.15.2:0/1216520438 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511819f440 con 0x7f51181047e0 2024-09-10T08:21:53.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.866+0000 7f5117e00640 1 --2- 172.21.15.2:0/1216520438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51181047e0 0x7f511819a9d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f510400e9f0 tx=0x7f510400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:53.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.867+0000 7f5115600640 1 -- 172.21.15.2:0/1216520438 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510400ccc0 con 0x7f51181047e0 2024-09-10T08:21:53.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.867+0000 7f5115600640 1 -- 172.21.15.2:0/1216520438 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5104010910 con 0x7f51181047e0 2024-09-10T08:21:53.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.867+0000 7f511e200640 1 -- 172.21.15.2:0/1216520438 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511819f730 con 0x7f51181047e0 2024-09-10T08:21:53.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.867+0000 7f5115600640 1 -- 172.21.15.2:0/1216520438 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104018960 con 0x7f51181047e0 2024-09-10T08:21:53.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.867+0000 7f511e200640 1 -- 172.21.15.2:0/1216520438 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51181a7010 con 0x7f51181047e0 2024-09-10T08:21:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.870+0000 7f5122257740 1 Processor -- start 2024-09-10T08:21:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.870+0000 7f5122257740 1 -- start start 2024-09-10T08:21:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.870+0000 7f5122257740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b51220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.870+0000 7f5122257740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b51760 0x557ac5b5bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.870+0000 7f5115600640 1 -- 172.21.15.2:0/1216520438 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5104010ac0 con 0x7f51181047e0 2024-09-10T08:21:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.870+0000 7f5122257740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b5c020 0x557ac5b5e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f5122257740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557ac5ab73e0 con 0x557ac5b51760 2024-09-10T08:21:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f5122257740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557ac5a5ed80 con 0x557ac5b5c020 2024-09-10T08:21:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f5122257740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557ac5ab33a0 con 0x557ac5b50e40 2024-09-10T08:21:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f511ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b51760 0x557ac5b5bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f5117e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b5c020 0x557ac5b5e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f511ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b51760 0x557ac5b5bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58932/0 (socket says 172.21.15.2:58932) 2024-09-10T08:21:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f511d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b51220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f511ce00640 1 -- 172.21.15.2:0/2974444159 learned_addr learned my addr 172.21.15.2:0/2974444159 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f511ce00640 1 -- 172.21.15.2:0/2974444159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 msgr2=0x557ac5b51220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f511ce00640 1 --2- 172.21.15.2:0/2974444159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b51220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f511ce00640 1 -- 172.21.15.2:0/2974444159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b5c020 msgr2=0x557ac5b5e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f511ce00640 1 --2- 172.21.15.2:0/2974444159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b5c020 0x557ac5b5e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f511ce00640 1 -- 172.21.15.2:0/2974444159 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ac5a7dae0 con 0x557ac5b51760 2024-09-10T08:21:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.871+0000 7f511d800640 1 --2- 172.21.15.2:0/2974444159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b51220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.872+0000 7f511ce00640 1 --2- 172.21.15.2:0/2974444159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b51760 0x557ac5b5bae0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f510c009a20 tx=0x7f510c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c1f9249a9318ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.872+0000 7f510a000640 1 -- 172.21.15.2:0/2974444159 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c040070 con 0x557ac5b51760 2024-09-10T08:21:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.872+0000 7f510a000640 1 -- 172.21.15.2:0/2974444159 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f510c004030 con 0x557ac5b51760 2024-09-10T08:21:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.872+0000 7f510a000640 1 -- 172.21.15.2:0/2974444159 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c035040 con 0x557ac5b51760 2024-09-10T08:21:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.872+0000 7f5115600640 1 --2- 172.21.15.2:0/1216520438 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f51000a2c00 0x7f51000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.872+0000 7f5115600640 1 -- 172.21.15.2:0/1216520438 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f51040cf780 con 0x7f51181047e0 2024-09-10T08:21:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.873+0000 7f511d800640 1 --2- 172.21.15.2:0/1216520438 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f51000a2c00 0x7f51000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.873+0000 7f5122257740 1 -- 172.21.15.2:0/2974444159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b51760 msgr2=0x557ac5b5bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.873+0000 7f5122257740 1 --2- 172.21.15.2:0/2974444159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b51760 0x557ac5b5bae0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f510c009a20 tx=0x7f510c005dd0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.873+0000 7f5122257740 1 -- 172.21.15.2:0/2974444159 shutdown_connections 2024-09-10T08:21:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.873+0000 7f5122257740 1 --2- 172.21.15.2:0/2974444159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b5c020 0x557ac5b5e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.873+0000 7f5122257740 1 --2- 172.21.15.2:0/2974444159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b51760 0x557ac5b5bae0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.873+0000 7f5122257740 1 --2- 172.21.15.2:0/2974444159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b51220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.873+0000 7f5122257740 1 -- 172.21.15.2:0/2974444159 >> 172.21.15.2:0/2974444159 conn(0x557ac5b4d3d0 msgr2=0x557ac5b4d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.873+0000 7f5122257740 1 -- 172.21.15.2:0/2974444159 shutdown_connections 2024-09-10T08:21:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.873+0000 7f5122257740 1 -- 172.21.15.2:0/2974444159 wait complete. 2024-09-10T08:21:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.873+0000 7f511d800640 1 --2- 172.21.15.2:0/1216520438 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f51000a2c00 0x7f51000a50c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5110009fd0 tx=0x7f5110009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.874+0000 7f5122257740 1 -- start start 2024-09-10T08:21:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.874+0000 7f5122257740 1 Processor -- start 2024-09-10T08:21:53.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.874+0000 7f5122257740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b58f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.874+0000 7f5122257740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b51760 0x557ac5b59470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f5122257740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b5c020 0x557ac5b599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f5122257740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557ac5b5ebf0 con 0x557ac5b51760 2024-09-10T08:21:53.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f5122257740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557ac5a720f0 con 0x557ac5b5c020 2024-09-10T08:21:53.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f5122257740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557ac5b5ed70 con 0x557ac5b50e40 2024-09-10T08:21:53.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f5122257740 10 client.0 _ll_register_callbacks cb 0x557ac5a59bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:53.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f511ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b58f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f511ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b58f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:40672/0 (socket says 172.21.15.2:40672) 2024-09-10T08:21:53.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f511ce00640 1 -- 172.21.15.2:0/3342078164 learned_addr learned my addr 172.21.15.2:0/3342078164 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:53.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f5117e00640 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b51760 0x557ac5b59470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f511d800640 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b5c020 0x557ac5b599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f511ce00640 1 -- 172.21.15.2:0/3342078164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b5c020 msgr2=0x557ac5b599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f511ce00640 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b5c020 0x557ac5b599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f511ce00640 1 -- 172.21.15.2:0/3342078164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b51760 msgr2=0x557ac5b59470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f511ce00640 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b51760 0x557ac5b59470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f511ce00640 1 -- 172.21.15.2:0/3342078164 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ac58283c0 con 0x557ac5b50e40 2024-09-10T08:21:53.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f511d800640 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b5c020 0x557ac5b599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:21:53.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.875+0000 7f5117e00640 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b51760 0x557ac5b59470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:53.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.876+0000 7f511ce00640 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b58f30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f510c000c00 tx=0x7f510c009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=930f0b64b5305c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:53.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.876+0000 7f50fea00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:21:53.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.876+0000 7f50fea00640 1 -- 172.21.15.2:0/3342078164 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c040070 con 0x557ac5b50e40 2024-09-10T08:21:53.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.876+0000 7f50fea00640 1 -- 172.21.15.2:0/3342078164 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f510c005520 con 0x557ac5b50e40 2024-09-10T08:21:53.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.876+0000 7f5122257740 1 -- 172.21.15.2:0/3342078164 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f510c004030 con 0x557ac5b50e40 2024-09-10T08:21:53.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.876+0000 7f50fea00640 1 -- 172.21.15.2:0/3342078164 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f510c035040 con 0x557ac5b50e40 2024-09-10T08:21:53.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.876+0000 7f5122257740 10 client.29329 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:53.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.876+0000 7f5122257740 1 -- 172.21.15.2:0/3342078164 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ac5b5ebf0 con 0x557ac5b50e40 2024-09-10T08:21:53.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.877+0000 7f50fea00640 1 -- 172.21.15.2:0/3342078164 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f510c004030 con 0x557ac5b50e40 2024-09-10T08:21:53.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.877+0000 7f50fea00640 1 -- 172.21.15.2:0/3342078164 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f510c04c020 con 0x557ac5b50e40 2024-09-10T08:21:53.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.877+0000 7f5122257740 10 client.29329 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:53.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.877+0000 7f5122257740 10 client.29329 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:21:53.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.877+0000 7f5122257740 20 client.29329 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:53.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.877+0000 7f5122257740 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557ac5b6be10 0x557ac5b6e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.877+0000 7f5122257740 4 client.29329 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:53.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.877+0000 7f5122257740 1 -- 172.21.15.2:0/3342078164 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ac5ac4950 con 0x557ac5b6be10 2024-09-10T08:21:53.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.878+0000 7f5117e00640 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557ac5b6be10 0x557ac5b6e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.878+0000 7f5117e00640 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557ac5b6be10 0x557ac5b6e200 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4b8e48a3a67640b server_cookie=bc3d5d705090624c in_seq=0 out_seq=0 2024-09-10T08:21:53.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.878+0000 7f50fea00640 10 client.29329 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:21:53.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.879+0000 7f50fea00640 1 -- 172.21.15.2:0/3342078164 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ac5ac4950 con 0x557ac5b6be10 2024-09-10T08:21:53.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.879+0000 7f50fea00640 10 client.29329 handle_command_reply: tid=0 2024-09-10T08:21:53.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.879+0000 7f5122257740 1 client.29329 shutdown 2024-09-10T08:21:53.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.879+0000 7f5122257740 1 -- 172.21.15.2:0/3342078164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 msgr2=0x557ac5b58f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.879+0000 7f5122257740 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b58f30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f510c000c00 tx=0x7f510c009520 comp rx=0 tx=0).stop 2024-09-10T08:21:53.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.880+0000 7f5122257740 1 -- 172.21.15.2:0/3342078164 shutdown_connections 2024-09-10T08:21:53.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.880+0000 7f5122257740 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557ac5b6be10 0x557ac5b6e200 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.880+0000 7f5122257740 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b5c020 0x557ac5b599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.880+0000 7f5122257740 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b51760 0x557ac5b59470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.880+0000 7f5122257740 1 --2- 172.21.15.2:0/3342078164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b58f30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.880+0000 7f50fea00640 0 client.29329 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:21:53.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.880+0000 7f5122257740 1 -- 172.21.15.2:0/3342078164 >> 172.21.15.2:0/3342078164 conn(0x557ac5b4d3d0 msgr2=0x557ac5b530b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:53.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.880+0000 7f5122257740 1 -- 172.21.15.2:0/3342078164 shutdown_connections 2024-09-10T08:21:53.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.880+0000 7f5122257740 1 -- 172.21.15.2:0/3342078164 wait complete. 2024-09-10T08:21:53.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.880+0000 7f5122257740 20 client.29329 trim_cache size 0 max 16384 2024-09-10T08:21:53.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.893+0000 7f5122257740 1 Processor -- start 2024-09-10T08:21:53.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.894+0000 7f5122257740 1 -- start start 2024-09-10T08:21:53.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.894+0000 7f5122257740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b5a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.894+0000 7f5122257740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b51760 0x557ac5b73970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.894+0000 7f5122257740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b5c020 0x557ac5b73eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.894+0000 7f5122257740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f51181b37a0 con 0x557ac5b5c020 2024-09-10T08:21:53.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.894+0000 7f5122257740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f51181b1070 con 0x557ac5b51760 2024-09-10T08:21:53.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.894+0000 7f5122257740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f51181aeec0 con 0x557ac5b50e40 2024-09-10T08:21:53.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.894+0000 7f511d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b5c020 0x557ac5b73eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.894+0000 7f511d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b5c020 0x557ac5b73eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58944/0 (socket says 172.21.15.2:58944) 2024-09-10T08:21:53.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.894+0000 7f511d800640 1 -- 172.21.15.2:0/3529151296 learned_addr learned my addr 172.21.15.2:0/3529151296 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:53.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.894+0000 7f511ce00640 1 --2- 172.21.15.2:0/3529151296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b5a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.894+0000 7f5117e00640 1 --2- 172.21.15.2:0/3529151296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b51760 0x557ac5b73970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.895+0000 7f511d800640 1 -- 172.21.15.2:0/3529151296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 msgr2=0x557ac5b5a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.895+0000 7f511d800640 1 --2- 172.21.15.2:0/3529151296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b5a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.895+0000 7f511d800640 1 -- 172.21.15.2:0/3529151296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b51760 msgr2=0x557ac5b73970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.895+0000 7f511d800640 1 --2- 172.21.15.2:0/3529151296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b51760 0x557ac5b73970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.895+0000 7f511d800640 1 -- 172.21.15.2:0/3529151296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ac5b5b8a0 con 0x557ac5b5c020 2024-09-10T08:21:53.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.895+0000 7f511ce00640 1 --2- 172.21.15.2:0/3529151296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b5a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:53.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.895+0000 7f511d800640 1 --2- 172.21.15.2:0/3529151296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b5c020 0x557ac5b73eb0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f5110011480 tx=0x7f5110016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbce3fa6d45e3ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:53.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.895+0000 7f5109600640 1 -- 172.21.15.2:0/3529151296 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51100040f0 con 0x557ac5b5c020 2024-09-10T08:21:53.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.895+0000 7f5109600640 1 -- 172.21.15.2:0/3529151296 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5110004770 con 0x557ac5b5c020 2024-09-10T08:21:53.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.895+0000 7f5109600640 1 -- 172.21.15.2:0/3529151296 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110010690 con 0x557ac5b5c020 2024-09-10T08:21:53.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.896+0000 7f5122257740 1 -- 172.21.15.2:0/3529151296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b5c020 msgr2=0x557ac5b73eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.896+0000 7f5122257740 1 --2- 172.21.15.2:0/3529151296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b5c020 0x557ac5b73eb0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f5110011480 tx=0x7f5110016000 comp rx=0 tx=0).stop 2024-09-10T08:21:53.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.896+0000 7f5122257740 1 -- 172.21.15.2:0/3529151296 shutdown_connections 2024-09-10T08:21:53.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.896+0000 7f5122257740 1 --2- 172.21.15.2:0/3529151296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b5c020 0x557ac5b73eb0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.896+0000 7f5122257740 1 --2- 172.21.15.2:0/3529151296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b51760 0x557ac5b73970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.896+0000 7f5122257740 1 --2- 172.21.15.2:0/3529151296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b50e40 0x557ac5b5a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.896+0000 7f5122257740 1 -- 172.21.15.2:0/3529151296 >> 172.21.15.2:0/3529151296 conn(0x557ac5b4d3d0 msgr2=0x557ac5b57ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:53.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.896+0000 7f5122257740 1 -- 172.21.15.2:0/3529151296 shutdown_connections 2024-09-10T08:21:53.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.896+0000 7f5122257740 1 -- 172.21.15.2:0/3529151296 wait complete. 2024-09-10T08:21:53.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.897+0000 7f5122257740 1 -- start start 2024-09-10T08:21:53.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.897+0000 7f5122257740 1 Processor -- start 2024-09-10T08:21:53.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.897+0000 7f5122257740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b50e40 0x557ac5b600e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.897+0000 7f5122257740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b51760 0x557ac5b60620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.897+0000 7f5122257740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b5c020 0x557ac5bd5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.897+0000 7f5122257740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f51181af530 con 0x557ac5b50e40 2024-09-10T08:21:53.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.897+0000 7f5122257740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f51181b09a0 con 0x557ac5b51760 2024-09-10T08:21:53.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.897+0000 7f5122257740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f51181afc00 con 0x557ac5b5c020 2024-09-10T08:21:53.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.898+0000 7f5122257740 10 client.0 _ll_register_callbacks cb 0x7f51040d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:53.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.898+0000 7f511ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b50e40 0x557ac5b600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.898+0000 7f5117e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b51760 0x557ac5b60620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.898+0000 7f511ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b50e40 0x557ac5b600e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58956/0 (socket says 172.21.15.2:58956) 2024-09-10T08:21:53.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.898+0000 7f511ce00640 1 -- 172.21.15.2:0/1669540848 learned_addr learned my addr 172.21.15.2:0/1669540848 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:53.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.898+0000 7f511d800640 1 --2- 172.21.15.2:0/1669540848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b5c020 0x557ac5bd5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.898+0000 7f5117e00640 1 -- 172.21.15.2:0/1669540848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b5c020 msgr2=0x557ac5bd5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.898+0000 7f5117e00640 1 --2- 172.21.15.2:0/1669540848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b5c020 0x557ac5bd5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.898+0000 7f5117e00640 1 -- 172.21.15.2:0/1669540848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b50e40 msgr2=0x557ac5b600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:53.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.898+0000 7f5117e00640 1 --2- 172.21.15.2:0/1669540848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b50e40 0x557ac5b600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.898+0000 7f5117e00640 1 -- 172.21.15.2:0/1669540848 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ac5b60f20 con 0x557ac5b51760 2024-09-10T08:21:53.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.898+0000 7f511d800640 1 --2- 172.21.15.2:0/1669540848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b5c020 0x557ac5bd5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:53.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.899+0000 7f5117e00640 1 --2- 172.21.15.2:0/1669540848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b51760 0x557ac5b60620 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x557ac5b4e750 tx=0x7f5104005b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1a5dd1b3a1ac675 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:53.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.899+0000 7f50fea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:21:53.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.899+0000 7f50fea00640 1 -- 172.21.15.2:0/1669540848 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510409bc40 con 0x557ac5b51760 2024-09-10T08:21:53.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.899+0000 7f50fea00640 1 -- 172.21.15.2:0/1669540848 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f51040ccf40 con 0x557ac5b51760 2024-09-10T08:21:53.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.899+0000 7f5122257740 1 -- 172.21.15.2:0/1669540848 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5110004770 con 0x557ac5b51760 2024-09-10T08:21:53.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.899+0000 7f50fea00640 1 -- 172.21.15.2:0/1669540848 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f510409ff40 con 0x557ac5b51760 2024-09-10T08:21:53.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.899+0000 7f5122257740 10 client.29408 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:53.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.899+0000 7f5122257740 1 -- 172.21.15.2:0/1669540848 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ac5bd7010 con 0x557ac5b51760 2024-09-10T08:21:53.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.899+0000 7f50fea00640 1 -- 172.21.15.2:0/1669540848 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5110004770 con 0x557ac5b51760 2024-09-10T08:21:53.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.900+0000 7f50fea00640 1 -- 172.21.15.2:0/1669540848 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f51040ce040 con 0x557ac5b51760 2024-09-10T08:21:53.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.900+0000 7f5122257740 10 client.29408 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:53.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.900+0000 7f5122257740 10 client.29408 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:21:53.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.900+0000 7f5122257740 20 client.29408 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:53.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.900+0000 7f5122257740 1 --2- 172.21.15.2:0/1669540848 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557ac5bd7730 0x557ac5bd9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:53.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.900+0000 7f5122257740 4 client.29408 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:21:53.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.900+0000 7f5122257740 1 -- 172.21.15.2:0/1669540848 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ac5abee60 con 0x557ac5bd7730 2024-09-10T08:21:53.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.900+0000 7f511ce00640 1 --2- 172.21.15.2:0/1669540848 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557ac5bd7730 0x557ac5bd9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:53.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f511ce00640 1 --2- 172.21.15.2:0/1669540848 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557ac5bd7730 0x557ac5bd9b20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3958a4b6f0161b9 server_cookie=975221c145062d32 in_seq=0 out_seq=0 2024-09-10T08:21:53.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f50fea00640 10 client.29408 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:21:53.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f50fea00640 1 -- 172.21.15.2:0/1669540848 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ac5abee60 con 0x557ac5bd7730 2024-09-10T08:21:53.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f50fea00640 10 client.29408 handle_command_reply: tid=0 2024-09-10T08:21:53.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f5122257740 1 client.29408 shutdown 2024-09-10T08:21:53.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f5122257740 1 -- 172.21.15.2:0/1669540848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b51760 msgr2=0x557ac5b60620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f5122257740 1 --2- 172.21.15.2:0/1669540848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b51760 0x557ac5b60620 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x557ac5b4e750 tx=0x7f5104005b60 comp rx=0 tx=0).stop 2024-09-10T08:21:53.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f5122257740 1 -- 172.21.15.2:0/1669540848 shutdown_connections 2024-09-10T08:21:53.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f5122257740 1 --2- 172.21.15.2:0/1669540848 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x557ac5bd7730 0x557ac5bd9b20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f5122257740 1 --2- 172.21.15.2:0/1669540848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac5b5c020 0x557ac5bd5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f5122257740 1 --2- 172.21.15.2:0/1669540848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ac5b51760 0x557ac5b60620 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f50fea00640 0 client.29408 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:21:53.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f5122257740 1 --2- 172.21.15.2:0/1669540848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ac5b50e40 0x557ac5b600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f5122257740 1 -- 172.21.15.2:0/1669540848 >> 172.21.15.2:0/1669540848 conn(0x557ac5b4d3d0 msgr2=0x557ac5b4bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:53.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f5122257740 1 -- 172.21.15.2:0/1669540848 shutdown_connections 2024-09-10T08:21:53.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.901+0000 7f5122257740 1 -- 172.21.15.2:0/1669540848 wait complete. 2024-09-10T08:21:53.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.902+0000 7f5122257740 20 client.29408 trim_cache size 0 max 16384 2024-09-10T08:21:53.905 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:21:53.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.902+0000 7f511e200640 1 -- 172.21.15.2:0/1216520438 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f51000a2c00 msgr2=0x7f51000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.902+0000 7f511e200640 1 --2- 172.21.15.2:0/1216520438 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f51000a2c00 0x7f51000a50c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5110009fd0 tx=0x7f5110009500 comp rx=0 tx=0).stop 2024-09-10T08:21:53.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.902+0000 7f511e200640 1 -- 172.21.15.2:0/1216520438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51181047e0 msgr2=0x7f511819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:53.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.902+0000 7f511e200640 1 --2- 172.21.15.2:0/1216520438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51181047e0 0x7f511819a9d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f510400e9f0 tx=0x7f510400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.902+0000 7f511e200640 1 -- 172.21.15.2:0/1216520438 shutdown_connections 2024-09-10T08:21:53.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.902+0000 7f511e200640 1 --2- 172.21.15.2:0/1216520438 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f51000a2c00 0x7f51000a50c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.902+0000 7f511e200640 1 --2- 172.21.15.2:0/1216520438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5118108f60 0x7f511819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.902+0000 7f511e200640 1 --2- 172.21.15.2:0/1216520438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51181047e0 0x7f511819a9d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.902+0000 7f511e200640 1 --2- 172.21.15.2:0/1216520438 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118103e30 0x7f511819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:53.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.902+0000 7f511e200640 1 -- 172.21.15.2:0/1216520438 >> 172.21.15.2:0/1216520438 conn(0x7f51180ff910 msgr2=0x7f5118100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:53.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.902+0000 7f511e200640 1 -- 172.21.15.2:0/1216520438 shutdown_connections 2024-09-10T08:21:53.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:53.902+0000 7f511e200640 1 -- 172.21.15.2:0/1216520438 wait complete. 2024-09-10T08:21:54.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:21:54.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:21:54.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:21:54.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:21:54.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:21:54.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:21:54.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:21:54.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:21:54.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:21:54.882 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:21:55.261 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:55.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:55 smithi002 ceph-mon[30563]: pgmap v800: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 12 op/s 2024-09-10T08:21:55.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:55 smithi155 ceph-mon[35405]: pgmap v800: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 12 op/s 2024-09-10T08:21:55.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:55 smithi107 ceph-mon[35048]: pgmap v800: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 12 op/s 2024-09-10T08:21:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.031+0000 7f3d00a00640 1 -- 172.21.15.2:0/1525698572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cfc101de0 msgr2=0x7f3cfc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.031+0000 7f3d00a00640 1 --2- 172.21.15.2:0/1525698572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cfc101de0 0x7f3cfc1021c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009900 tx=0x7f3cec02a410 comp rx=0 tx=0).stop 2024-09-10T08:21:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.031+0000 7f3d00a00640 1 -- 172.21.15.2:0/1525698572 shutdown_connections 2024-09-10T08:21:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.031+0000 7f3d00a00640 1 --2- 172.21.15.2:0/1525698572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cfc106f10 0x7f3cfc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.031+0000 7f3d00a00640 1 --2- 172.21.15.2:0/1525698572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cfc102790 0x7f3cfc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.031+0000 7f3d00a00640 1 --2- 172.21.15.2:0/1525698572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cfc101de0 0x7f3cfc1021c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.031+0000 7f3d00a00640 1 -- 172.21.15.2:0/1525698572 >> 172.21.15.2:0/1525698572 conn(0x7f3cfc0fd8c0 msgr2=0x7f3cfc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.031+0000 7f3d00a00640 1 -- 172.21.15.2:0/1525698572 shutdown_connections 2024-09-10T08:21:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.031+0000 7f3d00a00640 1 -- 172.21.15.2:0/1525698572 wait complete. 2024-09-10T08:21:56.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.032+0000 7f3d00a00640 1 Processor -- start 2024-09-10T08:21:56.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.032+0000 7f3d00a00640 1 -- start start 2024-09-10T08:21:56.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.033+0000 7f3d00a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cfc101de0 0x7f3cfc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.033+0000 7f3d00a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cfc102790 0x7f3cfc19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.033+0000 7f3d00a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cfc106f10 0x7f3cfc19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.033+0000 7f3d00a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc111f70 con 0x7f3cfc102790 2024-09-10T08:21:56.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.033+0000 7f3d00a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc111df0 con 0x7f3cfc106f10 2024-09-10T08:21:56.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.033+0000 7f3d00a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc1120f0 con 0x7f3cfc101de0 2024-09-10T08:21:56.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.033+0000 7f3cf9400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cfc102790 0x7f3cfc19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.033+0000 7f3cf9400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cfc102790 0x7f3cfc19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58966/0 (socket says 172.21.15.2:58966) 2024-09-10T08:21:56.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.033+0000 7f3cf9400640 1 -- 172.21.15.2:0/3296237440 learned_addr learned my addr 172.21.15.2:0/3296237440 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:56.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.033+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cfc101de0 0x7f3cfc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.033+0000 7f3cfa800640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cfc106f10 0x7f3cfc19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.034+0000 7f3cf9400640 1 -- 172.21.15.2:0/3296237440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cfc101de0 msgr2=0x7f3cfc19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:56.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.034+0000 7f3cf9400640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cfc101de0 0x7f3cfc19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.034+0000 7f3cf9400640 1 -- 172.21.15.2:0/3296237440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cfc106f10 msgr2=0x7f3cfc19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:56.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.034+0000 7f3cf9400640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cfc106f10 0x7f3cfc19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.034+0000 7f3cf9400640 1 -- 172.21.15.2:0/3296237440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cfc19f5c0 con 0x7f3cfc102790 2024-09-10T08:21:56.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.034+0000 7f3cf9400640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cfc102790 0x7f3cfc19ab50 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800e9f0 tx=0x7f3ce800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:56.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.034+0000 7f3cf2a00640 1 -- 172.21.15.2:0/3296237440 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800cd80 con 0x7f3cfc102790 2024-09-10T08:21:56.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.034+0000 7f3cf2a00640 1 -- 172.21.15.2:0/3296237440 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce8010910 con 0x7f3cfc102790 2024-09-10T08:21:56.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.034+0000 7f3d00a00640 1 -- 172.21.15.2:0/3296237440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cfc19f8b0 con 0x7f3cfc102790 2024-09-10T08:21:56.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.034+0000 7f3cf2a00640 1 -- 172.21.15.2:0/3296237440 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8018990 con 0x7f3cfc102790 2024-09-10T08:21:56.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.034+0000 7f3d00a00640 1 -- 172.21.15.2:0/3296237440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cfc1a7190 con 0x7f3cfc102790 2024-09-10T08:21:56.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.038+0000 7f3d03671740 1 Processor -- start 2024-09-10T08:21:56.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.039+0000 7f3d03671740 1 -- start start 2024-09-10T08:21:56.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.039+0000 7f3d03671740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e3bc70 0x55edd3e3c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.039+0000 7f3cf2a00640 1 -- 172.21.15.2:0/3296237440 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3ce8010ac0 con 0x7f3cfc102790 2024-09-10T08:21:56.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.039+0000 7f3d03671740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e46910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.039+0000 7f3d03671740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e46e50 0x55edd3e49240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.039+0000 7f3d03671740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55edd3da23e0 con 0x55edd3e46e50 2024-09-10T08:21:56.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.039+0000 7f3d03671740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55edd3d49d80 con 0x55edd3e3c590 2024-09-10T08:21:56.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.039+0000 7f3d03671740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55edd3d9e3a0 con 0x55edd3e3bc70 2024-09-10T08:21:56.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.040+0000 7f3cf9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e46910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.040+0000 7f3cf9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e46910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40430/0 (socket says 172.21.15.2:40430) 2024-09-10T08:21:56.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.040+0000 7f3cf9400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e46e50 0x55edd3e49240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.040+0000 7f3cfa800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e3bc70 0x55edd3e3c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.040+0000 7f3cf9e00640 1 -- 172.21.15.2:0/4294813617 learned_addr learned my addr 172.21.15.2:0/4294813617 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.040+0000 7f3cf9e00640 1 -- 172.21.15.2:0/4294813617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e3bc70 msgr2=0x55edd3e3c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.040+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/4294813617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e3bc70 0x55edd3e3c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.040+0000 7f3cf9e00640 1 -- 172.21.15.2:0/4294813617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e46e50 msgr2=0x55edd3e49240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.040+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/4294813617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e46e50 0x55edd3e49240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.040+0000 7f3cf9e00640 1 -- 172.21.15.2:0/4294813617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edd3d68ae0 con 0x55edd3e3c590 2024-09-10T08:21:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.040+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/4294813617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e46910 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009900 tx=0x7f3cec005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f455104ea7f2aef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.041+0000 7f3cf2a00640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3ce00a2b80 0x7f3ce00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.041+0000 7f3ce7000640 1 -- 172.21.15.2:0/4294813617 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec040070 con 0x55edd3e3c590 2024-09-10T08:21:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.041+0000 7f3ce7000640 1 -- 172.21.15.2:0/4294813617 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cec031870 con 0x55edd3e3c590 2024-09-10T08:21:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.041+0000 7f3ce7000640 1 -- 172.21.15.2:0/4294813617 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec035040 con 0x55edd3e3c590 2024-09-10T08:21:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.041+0000 7f3cf2a00640 1 -- 172.21.15.2:0/3296237440 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3ce80cf7a0 con 0x7f3cfc102790 2024-09-10T08:21:56.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.041+0000 7f3cfa800640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3ce00a2b80 0x7f3ce00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.042+0000 7f3d03671740 1 -- 172.21.15.2:0/4294813617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 msgr2=0x55edd3e46910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:56.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.042+0000 7f3d03671740 1 --2- 172.21.15.2:0/4294813617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e46910 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009900 tx=0x7f3cec005be0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.042+0000 7f3d03671740 1 -- 172.21.15.2:0/4294813617 shutdown_connections 2024-09-10T08:21:56.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.042+0000 7f3d03671740 1 --2- 172.21.15.2:0/4294813617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e46e50 0x55edd3e49240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.042+0000 7f3d03671740 1 --2- 172.21.15.2:0/4294813617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e46910 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.042+0000 7f3d03671740 1 --2- 172.21.15.2:0/4294813617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e3bc70 0x55edd3e3c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.042+0000 7f3d03671740 1 -- 172.21.15.2:0/4294813617 >> 172.21.15.2:0/4294813617 conn(0x55edd3e38200 msgr2=0x55edd3e385e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:56.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.042+0000 7f3cfa800640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3ce00a2b80 0x7f3ce00a5040 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4005fd0 tx=0x7f3cf4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:56.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.042+0000 7f3d03671740 1 -- 172.21.15.2:0/4294813617 shutdown_connections 2024-09-10T08:21:56.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.042+0000 7f3d03671740 1 -- 172.21.15.2:0/4294813617 wait complete. 2024-09-10T08:21:56.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.043+0000 7f3d03671740 1 -- start start 2024-09-10T08:21:56.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.043+0000 7f3d03671740 1 Processor -- start 2024-09-10T08:21:56.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.043+0000 7f3d03671740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e3bc70 0x55edd3e43ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.043+0000 7f3d03671740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e44220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.043+0000 7f3d03671740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e46e50 0x55edd3e44760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.043+0000 7f3d03671740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55edd3e49a20 con 0x55edd3e46e50 2024-09-10T08:21:56.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.043+0000 7f3d03671740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55edd3d5d0f0 con 0x55edd3e3c590 2024-09-10T08:21:56.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.043+0000 7f3d03671740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55edd3e49ba0 con 0x55edd3e3bc70 2024-09-10T08:21:56.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.043+0000 7f3d03671740 10 client.0 _ll_register_callbacks cb 0x55edd3d44bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:56.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cfa800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e46e50 0x55edd3e44760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cf9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e44220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cf9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e44220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40444/0 (socket says 172.21.15.2:40444) 2024-09-10T08:21:56.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cf9400640 1 -- 172.21.15.2:0/3380321944 learned_addr learned my addr 172.21.15.2:0/3380321944 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:56.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e3bc70 0x55edd3e43ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cf9e00640 1 -- 172.21.15.2:0/3380321944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 msgr2=0x55edd3e44220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:56.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e44220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cf9e00640 1 -- 172.21.15.2:0/3380321944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e46e50 msgr2=0x55edd3e44760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:56.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e46e50 0x55edd3e44760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cf9e00640 1 -- 172.21.15.2:0/3380321944 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edd3b133c0 con 0x55edd3e3bc70 2024-09-10T08:21:56.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cfa800640 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e46e50 0x55edd3e44760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:56.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cf9400640 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e44220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:56.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e3bc70 0x55edd3e43ce0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3cec005940 tx=0x7f3cec02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d3facb709e0410 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:56.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.044+0000 7f3cdb400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:21:56.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.045+0000 7f3cdb400640 1 -- 172.21.15.2:0/3380321944 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec040070 con 0x55edd3e3bc70 2024-09-10T08:21:56.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.045+0000 7f3d03671740 1 -- 172.21.15.2:0/3380321944 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cec031870 con 0x55edd3e3bc70 2024-09-10T08:21:56.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.045+0000 7f3cdb400640 1 -- 172.21.15.2:0/3380321944 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3cec004240 con 0x55edd3e3bc70 2024-09-10T08:21:56.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.045+0000 7f3cdb400640 1 -- 172.21.15.2:0/3380321944 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3cec056050 con 0x55edd3e3bc70 2024-09-10T08:21:56.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.045+0000 7f3d03671740 10 client.29347 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:56.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.045+0000 7f3d03671740 1 -- 172.21.15.2:0/3380321944 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55edd3e49a20 con 0x55edd3e3bc70 2024-09-10T08:21:56.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.046+0000 7f3cdb400640 1 -- 172.21.15.2:0/3380321944 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cec031870 con 0x55edd3e3bc70 2024-09-10T08:21:56.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.046+0000 7f3cdb400640 1 -- 172.21.15.2:0/3380321944 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3cec0035f0 con 0x55edd3e3bc70 2024-09-10T08:21:56.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.046+0000 7f3d03671740 10 client.29347 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:56.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.046+0000 7f3d03671740 10 client.29347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:56.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.046+0000 7f3d03671740 20 client.29347 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:56.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.046+0000 7f3d03671740 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55edd3e56b70 0x55edd3e58f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.046+0000 7f3d03671740 4 client.29347 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.046+0000 7f3d03671740 1 -- 172.21.15.2:0/3380321944 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edd3daf950 con 0x55edd3e56b70 2024-09-10T08:21:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.046+0000 7f3cf9400640 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55edd3e56b70 0x55edd3e58f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.047+0000 7f3cf9400640 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55edd3e56b70 0x55edd3e58f60 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4877e14c4bcdd8 server_cookie=d1daf14432c2f167 in_seq=0 out_seq=0 2024-09-10T08:21:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.047+0000 7f3cdb400640 10 client.29347 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3cdb400640 1 -- 172.21.15.2:0/3380321944 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edd3daf950 con 0x55edd3e56b70 2024-09-10T08:21:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3cdb400640 10 client.29347 handle_command_reply: tid=0 2024-09-10T08:21:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3d03671740 1 client.29347 shutdown 2024-09-10T08:21:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3d03671740 1 -- 172.21.15.2:0/3380321944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e3bc70 msgr2=0x55edd3e43ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3d03671740 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e3bc70 0x55edd3e43ce0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3cec005940 tx=0x7f3cec02a410 comp rx=0 tx=0).stop 2024-09-10T08:21:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3d03671740 1 -- 172.21.15.2:0/3380321944 shutdown_connections 2024-09-10T08:21:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3d03671740 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55edd3e56b70 0x55edd3e58f60 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3d03671740 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e46e50 0x55edd3e44760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3d03671740 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e44220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3d03671740 1 --2- 172.21.15.2:0/3380321944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e3bc70 0x55edd3e43ce0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3d03671740 1 -- 172.21.15.2:0/3380321944 >> 172.21.15.2:0/3380321944 conn(0x55edd3e38200 msgr2=0x55edd3e3dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3cdb400640 0 client.29347 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3d03671740 1 -- 172.21.15.2:0/3380321944 shutdown_connections 2024-09-10T08:21:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3d03671740 1 -- 172.21.15.2:0/3380321944 wait complete. 2024-09-10T08:21:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.048+0000 7f3d03671740 20 client.29347 trim_cache size 0 max 16384 2024-09-10T08:21:56.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.062+0000 7f3d03671740 1 Processor -- start 2024-09-10T08:21:56.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.062+0000 7f3d03671740 1 -- start start 2024-09-10T08:21:56.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.062+0000 7f3d03671740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 0x55edd3e46010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.062+0000 7f3d03671740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e3e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.062+0000 7f3d03671740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e46e50 0x55edd3e5e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.062+0000 7f3d03671740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc1b3910 con 0x55edd3e3bc70 2024-09-10T08:21:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.062+0000 7f3d03671740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc1b11e0 con 0x55edd3e3c590 2024-09-10T08:21:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.062+0000 7f3d03671740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc1af030 con 0x55edd3e46e50 2024-09-10T08:21:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.062+0000 7f3cf9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 0x55edd3e46010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.062+0000 7f3cf9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 0x55edd3e46010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58986/0 (socket says 172.21.15.2:58986) 2024-09-10T08:21:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.063+0000 7f3cf9e00640 1 -- 172.21.15.2:0/849870561 learned_addr learned my addr 172.21.15.2:0/849870561 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.063+0000 7f3cfa800640 1 --2- 172.21.15.2:0/849870561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e46e50 0x55edd3e5e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.063+0000 7f3cf9400640 1 --2- 172.21.15.2:0/849870561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e3e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.063+0000 7f3cf9e00640 1 -- 172.21.15.2:0/849870561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e46e50 msgr2=0x55edd3e5e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.063+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/849870561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e46e50 0x55edd3e5e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.063+0000 7f3cf9e00640 1 -- 172.21.15.2:0/849870561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 msgr2=0x55edd3e3e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:56.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.063+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/849870561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e3e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.063+0000 7f3cf9e00640 1 -- 172.21.15.2:0/849870561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edd3e33770 con 0x55edd3e3bc70 2024-09-10T08:21:56.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.063+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/849870561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 0x55edd3e46010 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3cec004000 tx=0x7f3cec005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfab1ddb37c169de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:56.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.063+0000 7f3ce6600640 1 -- 172.21.15.2:0/849870561 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec05f070 con 0x55edd3e3bc70 2024-09-10T08:21:56.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.064+0000 7f3ce6600640 1 -- 172.21.15.2:0/849870561 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cec04d070 con 0x55edd3e3bc70 2024-09-10T08:21:56.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.064+0000 7f3ce6600640 1 -- 172.21.15.2:0/849870561 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec040070 con 0x55edd3e3bc70 2024-09-10T08:21:56.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.064+0000 7f3d03671740 1 -- 172.21.15.2:0/849870561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 msgr2=0x55edd3e46010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:56.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.064+0000 7f3d03671740 1 --2- 172.21.15.2:0/849870561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 0x55edd3e46010 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3cec004000 tx=0x7f3cec005780 comp rx=0 tx=0).stop 2024-09-10T08:21:56.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.064+0000 7f3d03671740 1 -- 172.21.15.2:0/849870561 shutdown_connections 2024-09-10T08:21:56.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.064+0000 7f3d03671740 1 --2- 172.21.15.2:0/849870561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e46e50 0x55edd3e5e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.064+0000 7f3d03671740 1 --2- 172.21.15.2:0/849870561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e3e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.064+0000 7f3d03671740 1 --2- 172.21.15.2:0/849870561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 0x55edd3e46010 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.064+0000 7f3d03671740 1 -- 172.21.15.2:0/849870561 >> 172.21.15.2:0/849870561 conn(0x55edd3e38200 msgr2=0x55edd3e42ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.064+0000 7f3d03671740 1 -- 172.21.15.2:0/849870561 shutdown_connections 2024-09-10T08:21:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.064+0000 7f3d03671740 1 -- 172.21.15.2:0/849870561 wait complete. 2024-09-10T08:21:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.065+0000 7f3d03671740 1 -- start start 2024-09-10T08:21:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.065+0000 7f3d03671740 1 Processor -- start 2024-09-10T08:21:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.065+0000 7f3d03671740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 0x55edd3e4f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3d03671740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e4a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3d03671740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e46e50 0x55edd3e4ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3d03671740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc1af6a0 con 0x55edd3e3bc70 2024-09-10T08:21:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3d03671740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc1b0b10 con 0x55edd3e3c590 2024-09-10T08:21:56.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3d03671740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc1afd70 con 0x55edd3e46e50 2024-09-10T08:21:56.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3d03671740 10 client.0 _ll_register_callbacks cb 0x7f3ce80d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:56.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3cf9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 0x55edd3e4f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3cf9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 0x55edd3e4f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58998/0 (socket says 172.21.15.2:58998) 2024-09-10T08:21:56.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3cf9e00640 1 -- 172.21.15.2:0/527442290 learned_addr learned my addr 172.21.15.2:0/527442290 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:56.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3cf9400640 1 --2- 172.21.15.2:0/527442290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e4a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3cfa800640 1 --2- 172.21.15.2:0/527442290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e46e50 0x55edd3e4ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3cf9e00640 1 -- 172.21.15.2:0/527442290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e46e50 msgr2=0x55edd3e4ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:56.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/527442290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e46e50 0x55edd3e4ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3cf9e00640 1 -- 172.21.15.2:0/527442290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 msgr2=0x55edd3e4a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:56.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/527442290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e4a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.066+0000 7f3cf9e00640 1 -- 172.21.15.2:0/527442290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edd3e4b180 con 0x55edd3e3bc70 2024-09-10T08:21:56.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.067+0000 7f3cf9e00640 1 --2- 172.21.15.2:0/527442290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 0x55edd3e4f320 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009900 tx=0x7f3cec0055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7e3aeaf4a3f9de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:56.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.067+0000 7f3cdb400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:21:56.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.067+0000 7f3cdb400640 1 -- 172.21.15.2:0/527442290 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec067060 con 0x55edd3e3bc70 2024-09-10T08:21:56.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.067+0000 7f3cdb400640 1 -- 172.21.15.2:0/527442290 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f3cec031670 con 0x55edd3e3bc70 2024-09-10T08:21:56.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.067+0000 7f3d03671740 1 -- 172.21.15.2:0/527442290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cec04d070 con 0x55edd3e3bc70 2024-09-10T08:21:56.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.067+0000 7f3cdb400640 1 -- 172.21.15.2:0/527442290 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3cec06e050 con 0x55edd3e3bc70 2024-09-10T08:21:56.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.067+0000 7f3d03671740 10 client.19401 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:56.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.067+0000 7f3d03671740 1 -- 172.21.15.2:0/527442290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55edd3ec1430 con 0x55edd3e3bc70 2024-09-10T08:21:56.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.068+0000 7f3cdb400640 1 -- 172.21.15.2:0/527442290 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cec04d070 con 0x55edd3e3bc70 2024-09-10T08:21:56.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.068+0000 7f3cdb400640 1 -- 172.21.15.2:0/527442290 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3cec052450 con 0x55edd3e3bc70 2024-09-10T08:21:56.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.068+0000 7f3d03671740 10 client.19401 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:56.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.068+0000 7f3d03671740 10 client.19401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:56.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.068+0000 7f3d03671740 20 client.19401 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:56.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.068+0000 7f3d03671740 1 --2- 172.21.15.2:0/527442290 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55edd3ec1af0 0x55edd3ec3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:56.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.068+0000 7f3d03671740 4 client.19401 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:21:56.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.068+0000 7f3d03671740 1 -- 172.21.15.2:0/527442290 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55edd3da9e60 con 0x55edd3ec1af0 2024-09-10T08:21:56.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.068+0000 7f3cf9400640 1 --2- 172.21.15.2:0/527442290 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55edd3ec1af0 0x55edd3ec3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:56.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3cf9400640 1 --2- 172.21.15.2:0/527442290 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55edd3ec1af0 0x55edd3ec3ee0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=649a94852eb0ace2 server_cookie=3673ea3e8e606cb8 in_seq=0 out_seq=0 2024-09-10T08:21:56.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3cdb400640 10 client.19401 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:56.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3cdb400640 1 -- 172.21.15.2:0/527442290 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55edd3da9e60 con 0x55edd3ec1af0 2024-09-10T08:21:56.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3cdb400640 10 client.19401 handle_command_reply: tid=0 2024-09-10T08:21:56.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3d03671740 1 client.19401 shutdown 2024-09-10T08:21:56.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3d03671740 1 -- 172.21.15.2:0/527442290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 msgr2=0x55edd3e4f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:56.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3d03671740 1 --2- 172.21.15.2:0/527442290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 0x55edd3e4f320 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009900 tx=0x7f3cec0055d0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3d03671740 1 -- 172.21.15.2:0/527442290 shutdown_connections 2024-09-10T08:21:56.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3d03671740 1 --2- 172.21.15.2:0/527442290 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55edd3ec1af0 0x55edd3ec3ee0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3d03671740 1 --2- 172.21.15.2:0/527442290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edd3e46e50 0x55edd3e4ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3d03671740 1 --2- 172.21.15.2:0/527442290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edd3e3c590 0x55edd3e4a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3d03671740 1 --2- 172.21.15.2:0/527442290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55edd3e3bc70 0x55edd3e4f320 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3d03671740 1 -- 172.21.15.2:0/527442290 >> 172.21.15.2:0/527442290 conn(0x55edd3e38200 msgr2=0x55edd3e42ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:56.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3cdb400640 0 client.19401 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:56.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.069+0000 7f3d03671740 1 -- 172.21.15.2:0/527442290 shutdown_connections 2024-09-10T08:21:56.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.070+0000 7f3d03671740 1 -- 172.21.15.2:0/527442290 wait complete. 2024-09-10T08:21:56.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.070+0000 7f3d03671740 20 client.19401 trim_cache size 0 max 16384 2024-09-10T08:21:56.077 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:21:56.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.070+0000 7f3d00a00640 1 -- 172.21.15.2:0/3296237440 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3ce00a2b80 msgr2=0x7f3ce00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:56.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.070+0000 7f3d00a00640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3ce00a2b80 0x7f3ce00a5040 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4005fd0 tx=0x7f3cf4009500 comp rx=0 tx=0).stop 2024-09-10T08:21:56.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.070+0000 7f3d00a00640 1 -- 172.21.15.2:0/3296237440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cfc102790 msgr2=0x7f3cfc19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:56.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.070+0000 7f3d00a00640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cfc102790 0x7f3cfc19ab50 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800e9f0 tx=0x7f3ce800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.070+0000 7f3d00a00640 1 -- 172.21.15.2:0/3296237440 shutdown_connections 2024-09-10T08:21:56.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.070+0000 7f3d00a00640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cfc106f10 0x7f3cfc19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.070+0000 7f3d00a00640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cfc102790 0x7f3cfc19ab50 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.070+0000 7f3d00a00640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f3ce00a2b80 0x7f3ce00a5040 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.070+0000 7f3d00a00640 1 --2- 172.21.15.2:0/3296237440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cfc101de0 0x7f3cfc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:56.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.070+0000 7f3d00a00640 1 -- 172.21.15.2:0/3296237440 >> 172.21.15.2:0/3296237440 conn(0x7f3cfc0fd8c0 msgr2=0x7f3cfc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:56.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.071+0000 7f3d00a00640 1 -- 172.21.15.2:0/3296237440 shutdown_connections 2024-09-10T08:21:56.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:56.071+0000 7f3d00a00640 1 -- 172.21.15.2:0/3296237440 wait complete. 2024-09-10T08:21:57.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:57 smithi002 ceph-mon[30563]: pgmap v801: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-09-10T08:21:57.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:57 smithi155 ceph-mon[35405]: pgmap v801: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-09-10T08:21:57.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:57 smithi107 ceph-mon[35048]: pgmap v801: 129 pgs: 129 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-09-10T08:21:57.976 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:21:57.977 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:21:58.367 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:21:58.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.985+0000 7f831b200640 1 -- 172.21.15.2:0/103678205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f831c102360 msgr2=0x7f831c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:58.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.985+0000 7f831b200640 1 --2- 172.21.15.2:0/103678205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f831c102360 0x7f831c10e890 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8314009ca0 tx=0x7f831402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:21:58.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.985+0000 7f831b200640 1 -- 172.21.15.2:0/103678205 shutdown_connections 2024-09-10T08:21:58.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.985+0000 7f831b200640 1 --2- 172.21.15.2:0/103678205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f831c102360 0x7f831c10e890 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:58.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.985+0000 7f831b200640 1 --2- 172.21.15.2:0/103678205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f831c1019c0 0x7f831c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:58.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.985+0000 7f831b200640 1 --2- 172.21.15.2:0/103678205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f831c1079c0 0x7f831c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:58.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.985+0000 7f831b200640 1 -- 172.21.15.2:0/103678205 >> 172.21.15.2:0/103678205 conn(0x7f831c0fd740 msgr2=0x7f831c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:58.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.985+0000 7f831b200640 1 -- 172.21.15.2:0/103678205 shutdown_connections 2024-09-10T08:21:58.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.985+0000 7f831b200640 1 -- 172.21.15.2:0/103678205 wait complete. 2024-09-10T08:21:58.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.986+0000 7f831b200640 1 Processor -- start 2024-09-10T08:21:58.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.987+0000 7f831b200640 1 -- start start 2024-09-10T08:21:58.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.987+0000 7f831b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f831c1019c0 0x7f831c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:58.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.987+0000 7f831b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f831c102360 0x7f831c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:58.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.987+0000 7f831b200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f831c1079c0 0x7f831c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:58.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.987+0000 7f831b200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f831c111e10 con 0x7f831c1079c0 2024-09-10T08:21:58.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.987+0000 7f831b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f831c111c90 con 0x7f831c102360 2024-09-10T08:21:58.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.987+0000 7f831b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f831c111f90 con 0x7f831c1019c0 2024-09-10T08:21:58.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.987+0000 7f831a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f831c1079c0 0x7f831c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.987+0000 7f8311400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f831c102360 0x7f831c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.987+0000 7f8319e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f831c1019c0 0x7f831c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.987+0000 7f831a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f831c1079c0 0x7f831c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57024/0 (socket says 172.21.15.2:57024) 2024-09-10T08:21:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.987+0000 7f8311400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f831c102360 0x7f831c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52124/0 (socket says 172.21.15.2:52124) 2024-09-10T08:21:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.987+0000 7f831a800640 1 -- 172.21.15.2:0/1138231148 learned_addr learned my addr 172.21.15.2:0/1138231148 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.988+0000 7f831a800640 1 -- 172.21.15.2:0/1138231148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f831c1019c0 msgr2=0x7f831c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.988+0000 7f831a800640 1 --2- 172.21.15.2:0/1138231148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f831c1019c0 0x7f831c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.988+0000 7f831a800640 1 -- 172.21.15.2:0/1138231148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f831c102360 msgr2=0x7f831c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.988+0000 7f831a800640 1 --2- 172.21.15.2:0/1138231148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f831c102360 0x7f831c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.988+0000 7f831a800640 1 -- 172.21.15.2:0/1138231148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c1a66b0 con 0x7f831c1079c0 2024-09-10T08:21:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.988+0000 7f8311400640 1 --2- 172.21.15.2:0/1138231148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f831c102360 0x7f831c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.988+0000 7f831a800640 1 --2- 172.21.15.2:0/1138231148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f831c1079c0 0x7f831c074f70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8314009430 tx=0x7f8314003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:58.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.988+0000 7f8313400640 1 -- 172.21.15.2:0/1138231148 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314031710 con 0x7f831c1079c0 2024-09-10T08:21:58.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.989+0000 7f831b200640 1 -- 172.21.15.2:0/1138231148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c1a6940 con 0x7f831c1079c0 2024-09-10T08:21:58.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.989+0000 7f8313400640 1 -- 172.21.15.2:0/1138231148 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83140318b0 con 0x7f831c1079c0 2024-09-10T08:21:58.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.989+0000 7f8313400640 1 -- 172.21.15.2:0/1138231148 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831402fad0 con 0x7f831c1079c0 2024-09-10T08:21:58.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.989+0000 7f831b200640 1 -- 172.21.15.2:0/1138231148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c1a6ed0 con 0x7f831c1079c0 2024-09-10T08:21:58.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.992+0000 7f8313400640 1 -- 172.21.15.2:0/1138231148 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f831402fcb0 con 0x7f831c1079c0 2024-09-10T08:21:58.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.992+0000 7f832364d740 1 Processor -- start 2024-09-10T08:21:58.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.992+0000 7f832364d740 1 -- start start 2024-09-10T08:21:58.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.992+0000 7f832364d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x55976702f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:58.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.992+0000 7f832364d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x559767039950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:58.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.992+0000 7f832364d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559767039e90 0x55976703c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:58.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.992+0000 7f832364d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559766f95290 con 0x559767039e90 2024-09-10T08:21:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.992+0000 7f832364d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559766f3cb40 con 0x55976702ecb0 2024-09-10T08:21:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.992+0000 7f832364d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559766f91250 con 0x55976702f5d0 2024-09-10T08:21:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.993+0000 7f8311400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x55976702f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.993+0000 7f8319e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x559767039950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.993+0000 7f8311400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x55976702f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52140/0 (socket says 172.21.15.2:52140) 2024-09-10T08:21:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.993+0000 7f8311400640 1 -- 172.21.15.2:0/1690523327 learned_addr learned my addr 172.21.15.2:0/1690523327 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.993+0000 7f8311400640 1 --2- 172.21.15.2:0/1690523327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559767039e90 0x55976703c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.993+0000 7f8319e00640 1 -- 172.21.15.2:0/1690523327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 msgr2=0x55976702f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.993+0000 7f8319e00640 1 --2- 172.21.15.2:0/1690523327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x55976702f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.993+0000 7f8319e00640 1 -- 172.21.15.2:0/1690523327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559767039e90 msgr2=0x55976703c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.993+0000 7f8319e00640 1 --2- 172.21.15.2:0/1690523327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559767039e90 0x55976703c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.993+0000 7f8319e00640 1 -- 172.21.15.2:0/1690523327 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559766d066a0 con 0x55976702f5d0 2024-09-10T08:21:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.993+0000 7f8311400640 1 --2- 172.21.15.2:0/1690523327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x55976702f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:21:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.994+0000 7f8319e00640 1 --2- 172.21.15.2:0/1690523327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x559767039950 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f830c009c20 tx=0x7f830c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6afa8cfb2292e788 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.994+0000 7f8313400640 1 --2- 172.21.15.2:0/1138231148 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83000a2b80 0x7f83000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.994+0000 7f830b000640 1 -- 172.21.15.2:0/1690523327 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c004050 con 0x55976702f5d0 2024-09-10T08:21:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.994+0000 7f8313400640 1 -- 172.21.15.2:0/1138231148 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f83140f08b0 con 0x7f831c1079c0 2024-09-10T08:21:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.994+0000 7f830b000640 1 -- 172.21.15.2:0/1690523327 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830c005500 con 0x55976702f5d0 2024-09-10T08:21:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.994+0000 7f830b000640 1 -- 172.21.15.2:0/1690523327 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c004d90 con 0x55976702f5d0 2024-09-10T08:21:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.994+0000 7f8311400640 1 --2- 172.21.15.2:0/1138231148 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83000a2b80 0x7f83000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.995+0000 7f832364d740 1 -- 172.21.15.2:0/1690523327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 msgr2=0x559767039950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.995+0000 7f832364d740 1 --2- 172.21.15.2:0/1690523327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x559767039950 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f830c009c20 tx=0x7f830c011810 comp rx=0 tx=0).stop 2024-09-10T08:21:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.995+0000 7f8311400640 1 --2- 172.21.15.2:0/1138231148 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83000a2b80 0x7f83000a5040 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55976702c5c0 tx=0x7f830400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.995+0000 7f832364d740 1 -- 172.21.15.2:0/1690523327 shutdown_connections 2024-09-10T08:21:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.995+0000 7f832364d740 1 --2- 172.21.15.2:0/1690523327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559767039e90 0x55976703c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.995+0000 7f832364d740 1 --2- 172.21.15.2:0/1690523327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x559767039950 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.995+0000 7f832364d740 1 --2- 172.21.15.2:0/1690523327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x55976702f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.995+0000 7f832364d740 1 -- 172.21.15.2:0/1690523327 >> 172.21.15.2:0/1690523327 conn(0x55976702b240 msgr2=0x55976702b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.995+0000 7f832364d740 1 -- 172.21.15.2:0/1690523327 shutdown_connections 2024-09-10T08:21:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.995+0000 7f832364d740 1 -- 172.21.15.2:0/1690523327 wait complete. 2024-09-10T08:21:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.996+0000 7f832364d740 1 -- start start 2024-09-10T08:21:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.996+0000 7f832364d740 1 Processor -- start 2024-09-10T08:21:58.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.996+0000 7f832364d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x559767039750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:58.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.996+0000 7f832364d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x559767036a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.996+0000 7f832364d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559767039e90 0x559767036f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.996+0000 7f832364d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55976703cb00 con 0x559767039e90 2024-09-10T08:21:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.996+0000 7f832364d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559766f4ffa0 con 0x55976702ecb0 2024-09-10T08:21:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.996+0000 7f832364d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55976703cc80 con 0x55976702f5d0 2024-09-10T08:21:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.996+0000 7f832364d740 10 client.0 _ll_register_callbacks cb 0x559766ca3ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.997+0000 7f8319e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x559767039750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.997+0000 7f8319e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x559767039750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52144/0 (socket says 172.21.15.2:52144) 2024-09-10T08:21:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.997+0000 7f8319e00640 1 -- 172.21.15.2:0/1427741340 learned_addr learned my addr 172.21.15.2:0/1427741340 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.997+0000 7f831a800640 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559767039e90 0x559767036f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.997+0000 7f8311400640 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x559767036a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.997+0000 7f8319e00640 1 -- 172.21.15.2:0/1427741340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 msgr2=0x559767036a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.997+0000 7f8319e00640 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x559767036a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.997+0000 7f8319e00640 1 -- 172.21.15.2:0/1427741340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559767039e90 msgr2=0x559767036f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.997+0000 7f8319e00640 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559767039e90 0x559767036f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.997+0000 7f8319e00640 1 -- 172.21.15.2:0/1427741340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559767037770 con 0x55976702ecb0 2024-09-10T08:21:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.997+0000 7f8311400640 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x559767036a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:21:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.997+0000 7f8319e00640 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x559767039750 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f830c000c00 tx=0x7f830c004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=247461ef97c2866c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.997+0000 7f82ff400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:21:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.998+0000 7f82ff400640 1 -- 172.21.15.2:0/1427741340 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c029070 con 0x55976702ecb0 2024-09-10T08:21:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.998+0000 7f832364d740 1 -- 172.21.15.2:0/1427741340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f830c005500 con 0x55976702ecb0 2024-09-10T08:21:59.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.998+0000 7f82ff400640 1 -- 172.21.15.2:0/1427741340 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f830c004f20 con 0x55976702ecb0 2024-09-10T08:21:59.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.998+0000 7f82ff400640 1 -- 172.21.15.2:0/1427741340 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f830c035050 con 0x55976702ecb0 2024-09-10T08:21:59.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.998+0000 7f832364d740 10 client.29438 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:59.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.998+0000 7f832364d740 1 -- 172.21.15.2:0/1427741340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5597670489a0 con 0x55976702ecb0 2024-09-10T08:21:59.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.998+0000 7f82ff400640 1 -- 172.21.15.2:0/1427741340 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830c005500 con 0x55976702ecb0 2024-09-10T08:21:59.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.999+0000 7f82ff400640 1 -- 172.21.15.2:0/1427741340 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f830c02e440 con 0x55976702ecb0 2024-09-10T08:21:59.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.999+0000 7f832364d740 10 client.29438 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:59.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.999+0000 7f832364d740 10 client.29438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:59.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.999+0000 7f832364d740 20 client.29438 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:59.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.999+0000 7f832364d740 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559767049d50 0x55976704c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:59.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.999+0000 7f832364d740 4 client.29438 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:21:59.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.999+0000 7f832364d740 1 -- 172.21.15.2:0/1427741340 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559766fa2800 con 0x559767049d50 2024-09-10T08:21:59.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:58.999+0000 7f8311400640 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559767049d50 0x55976704c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:59.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.000+0000 7f8311400640 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559767049d50 0x55976704c140 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fba8058bb981dee server_cookie=762dccc11905b441 in_seq=0 out_seq=0 2024-09-10T08:21:59.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.000+0000 7f82ff400640 10 client.29438 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:59.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.000+0000 7f82ff400640 1 -- 172.21.15.2:0/1427741340 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559766fa2800 con 0x559767049d50 2024-09-10T08:21:59.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.000+0000 7f82ff400640 10 client.29438 handle_command_reply: tid=0 2024-09-10T08:21:59.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.000+0000 7f832364d740 1 client.29438 shutdown 2024-09-10T08:21:59.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.001+0000 7f832364d740 1 -- 172.21.15.2:0/1427741340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 msgr2=0x559767039750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:59.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.001+0000 7f832364d740 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x559767039750 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f830c000c00 tx=0x7f830c004970 comp rx=0 tx=0).stop 2024-09-10T08:21:59.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.001+0000 7f832364d740 1 -- 172.21.15.2:0/1427741340 shutdown_connections 2024-09-10T08:21:59.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.001+0000 7f832364d740 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559767049d50 0x55976704c140 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.001+0000 7f832364d740 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559767039e90 0x559767036f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.001+0000 7f832364d740 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x559767036a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.001+0000 7f832364d740 1 --2- 172.21.15.2:0/1427741340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x559767039750 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.001+0000 7f82ff400640 0 client.29438 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:59.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.001+0000 7f832364d740 1 -- 172.21.15.2:0/1427741340 >> 172.21.15.2:0/1427741340 conn(0x55976702b240 msgr2=0x559767030f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:59.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.001+0000 7f832364d740 1 -- 172.21.15.2:0/1427741340 shutdown_connections 2024-09-10T08:21:59.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.001+0000 7f832364d740 1 -- 172.21.15.2:0/1427741340 wait complete. 2024-09-10T08:21:59.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.001+0000 7f832364d740 20 client.29438 trim_cache size 0 max 16384 2024-09-10T08:21:59.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.019+0000 7f832364d740 1 Processor -- start 2024-09-10T08:21:59.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.019+0000 7f832364d740 1 -- start start 2024-09-10T08:21:59.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.019+0000 7f832364d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x559767032110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:59.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.019+0000 7f832364d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55976702f5d0 0x559767030e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:59.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.019+0000 7f832364d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559767039e90 0x5597670518d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:59.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.019+0000 7f832364d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1b36b0 con 0x55976702f5d0 2024-09-10T08:21:59.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.019+0000 7f832364d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1b0f80 con 0x55976702ecb0 2024-09-10T08:21:59.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.019+0000 7f832364d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1aedd0 con 0x559767039e90 2024-09-10T08:21:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.020+0000 7f831a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559767039e90 0x5597670518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.020+0000 7f8311400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55976702f5d0 0x559767030e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.020+0000 7f831a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559767039e90 0x5597670518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:46922/0 (socket says 172.21.15.2:46922) 2024-09-10T08:21:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.020+0000 7f831a800640 1 -- 172.21.15.2:0/1526126089 learned_addr learned my addr 172.21.15.2:0/1526126089 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.020+0000 7f8319e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x559767032110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.020+0000 7f831a800640 1 -- 172.21.15.2:0/1526126089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 msgr2=0x559767032110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.020+0000 7f831a800640 1 --2- 172.21.15.2:0/1526126089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x559767032110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.020+0000 7f831a800640 1 -- 172.21.15.2:0/1526126089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55976702f5d0 msgr2=0x559767030e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.020+0000 7f831a800640 1 --2- 172.21.15.2:0/1526126089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55976702f5d0 0x559767030e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.020+0000 7f831a800640 1 -- 172.21.15.2:0/1526126089 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559767037540 con 0x559767039e90 2024-09-10T08:21:59.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.021+0000 7f831a800640 1 --2- 172.21.15.2:0/1526126089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559767039e90 0x5597670518d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8314005e30 tx=0x7f831402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a0f658cc96c9945 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:59.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.021+0000 7f830a600640 1 -- 172.21.15.2:0/1526126089 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140da070 con 0x559767039e90 2024-09-10T08:21:59.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.021+0000 7f830a600640 1 -- 172.21.15.2:0/1526126089 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8314036040 con 0x559767039e90 2024-09-10T08:21:59.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.021+0000 7f830a600640 1 -- 172.21.15.2:0/1526126089 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140bcf20 con 0x559767039e90 2024-09-10T08:21:59.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.021+0000 7f832364d740 1 -- 172.21.15.2:0/1526126089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559767039e90 msgr2=0x5597670518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:59.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.021+0000 7f832364d740 1 --2- 172.21.15.2:0/1526126089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559767039e90 0x5597670518d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8314005e30 tx=0x7f831402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.021+0000 7f832364d740 1 -- 172.21.15.2:0/1526126089 shutdown_connections 2024-09-10T08:21:59.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.021+0000 7f832364d740 1 --2- 172.21.15.2:0/1526126089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559767039e90 0x5597670518d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.021+0000 7f832364d740 1 --2- 172.21.15.2:0/1526126089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55976702f5d0 0x559767030e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.021+0000 7f832364d740 1 --2- 172.21.15.2:0/1526126089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55976702ecb0 0x559767032110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.021+0000 7f832364d740 1 -- 172.21.15.2:0/1526126089 >> 172.21.15.2:0/1526126089 conn(0x55976702b240 msgr2=0x559767030700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:59.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.021+0000 7f832364d740 1 -- 172.21.15.2:0/1526126089 shutdown_connections 2024-09-10T08:21:59.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.022+0000 7f832364d740 1 -- 172.21.15.2:0/1526126089 wait complete. 2024-09-10T08:21:59.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.022+0000 7f832364d740 1 -- start start 2024-09-10T08:21:59.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.022+0000 7f832364d740 1 Processor -- start 2024-09-10T08:21:59.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f832364d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55976702ecb0 0x55976703dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f832364d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x55976703e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f832364d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559767039e90 0x55976703e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f832364d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1af440 con 0x55976702ecb0 2024-09-10T08:21:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f832364d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1b08b0 con 0x559767039e90 2024-09-10T08:21:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f832364d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1afb10 con 0x55976702f5d0 2024-09-10T08:21:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f832364d740 10 client.0 _ll_register_callbacks cb 0x7f8304012660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:21:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f8319e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55976702ecb0 0x55976703dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f8311400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x55976703e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f831a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559767039e90 0x55976703e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f8311400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x55976703e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:46930/0 (socket says 172.21.15.2:46930) 2024-09-10T08:21:59.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f8311400640 1 -- 172.21.15.2:0/3555071184 learned_addr learned my addr 172.21.15.2:0/3555071184 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:21:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f831a800640 1 -- 172.21.15.2:0/3555071184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 msgr2=0x55976703e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f831a800640 1 --2- 172.21.15.2:0/3555071184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x55976703e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f831a800640 1 -- 172.21.15.2:0/3555071184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55976702ecb0 msgr2=0x55976703dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:21:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f831a800640 1 --2- 172.21.15.2:0/3555071184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55976702ecb0 0x55976703dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.023+0000 7f831a800640 1 -- 172.21.15.2:0/3555071184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597670bc180 con 0x559767039e90 2024-09-10T08:21:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.024+0000 7f831a800640 1 --2- 172.21.15.2:0/3555071184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559767039e90 0x55976703e7c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8314006010 tx=0x7f831402f4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=781ebb66f2c9a43b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:21:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.024+0000 7f82ff400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:21:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.024+0000 7f82ff400640 1 -- 172.21.15.2:0/3555071184 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140da070 con 0x559767039e90 2024-09-10T08:21:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.024+0000 7f832364d740 1 -- 172.21.15.2:0/3555071184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8314036040 con 0x559767039e90 2024-09-10T08:21:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.024+0000 7f82ff400640 1 -- 172.21.15.2:0/3555071184 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f83140bcf20 con 0x559767039e90 2024-09-10T08:21:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.024+0000 7f82ff400640 1 -- 172.21.15.2:0/3555071184 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f83140c5930 con 0x559767039e90 2024-09-10T08:21:59.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.024+0000 7f832364d740 10 client.29444 fetch_fsmap learned FSMap version 26 2024-09-10T08:21:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.024+0000 7f832364d740 1 -- 172.21.15.2:0/3555071184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5597670bd4f0 con 0x559767039e90 2024-09-10T08:21:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.024+0000 7f82ff400640 1 -- 172.21.15.2:0/3555071184 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8314036040 con 0x559767039e90 2024-09-10T08:21:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.025+0000 7f82ff400640 1 -- 172.21.15.2:0/3555071184 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f83140c54a0 con 0x559767039e90 2024-09-10T08:21:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.025+0000 7f832364d740 10 client.29444 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:21:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.025+0000 7f832364d740 10 client.29444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:21:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.025+0000 7f832364d740 20 client.29444 populate_metadata read hostname 'smithi002' 2024-09-10T08:21:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.025+0000 7f832364d740 1 --2- 172.21.15.2:0/3555071184 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5597670bdbc0 0x5597670bffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:21:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.025+0000 7f832364d740 4 client.29444 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:21:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.025+0000 7f832364d740 1 -- 172.21.15.2:0/3555071184 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559766f9cd10 con 0x5597670bdbc0 2024-09-10T08:21:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.025+0000 7f8319e00640 1 --2- 172.21.15.2:0/3555071184 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5597670bdbc0 0x5597670bffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:21:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.026+0000 7f8319e00640 1 --2- 172.21.15.2:0/3555071184 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5597670bdbc0 0x5597670bffe0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37dbda6da654adc server_cookie=7b725a57b34aec4f in_seq=0 out_seq=0 2024-09-10T08:21:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.026+0000 7f82ff400640 10 client.29444 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:59.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.026+0000 7f82ff400640 1 -- 172.21.15.2:0/3555071184 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559766f9cd10 con 0x5597670bdbc0 2024-09-10T08:21:59.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.026+0000 7f82ff400640 10 client.29444 handle_command_reply: tid=0 2024-09-10T08:21:59.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.026+0000 7f832364d740 1 client.29444 shutdown 2024-09-10T08:21:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.027+0000 7f832364d740 1 -- 172.21.15.2:0/3555071184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559767039e90 msgr2=0x55976703e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.027+0000 7f832364d740 1 --2- 172.21.15.2:0/3555071184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559767039e90 0x55976703e7c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8314006010 tx=0x7f831402f4f0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.027+0000 7f832364d740 1 -- 172.21.15.2:0/3555071184 shutdown_connections 2024-09-10T08:21:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.027+0000 7f832364d740 1 --2- 172.21.15.2:0/3555071184 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5597670bdbc0 0x5597670bffe0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.027+0000 7f832364d740 1 --2- 172.21.15.2:0/3555071184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559767039e90 0x55976703e7c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.027+0000 7f832364d740 1 --2- 172.21.15.2:0/3555071184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55976702f5d0 0x55976703e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.027+0000 7f832364d740 1 --2- 172.21.15.2:0/3555071184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55976702ecb0 0x55976703dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.027+0000 7f82ff400640 0 client.29444 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:21:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.027+0000 7f832364d740 1 -- 172.21.15.2:0/3555071184 >> 172.21.15.2:0/3555071184 conn(0x55976702b240 msgr2=0x5597670351a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.027+0000 7f832364d740 1 -- 172.21.15.2:0/3555071184 shutdown_connections 2024-09-10T08:21:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.027+0000 7f832364d740 1 -- 172.21.15.2:0/3555071184 wait complete. 2024-09-10T08:21:59.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.027+0000 7f832364d740 20 client.29444 trim_cache size 0 max 16384 2024-09-10T08:21:59.031 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:21:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:21:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "847dc39c-e1b8-4aff-8f91-99de3dd3bffb", 2024-09-10T08:21:59.031 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:21:59.031 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:21:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.028+0000 7f831b200640 1 -- 172.21.15.2:0/1138231148 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83000a2b80 msgr2=0x7f83000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.028+0000 7f831b200640 1 --2- 172.21.15.2:0/1138231148 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83000a2b80 0x7f83000a5040 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55976702c5c0 tx=0x7f830400b410 comp rx=0 tx=0).stop 2024-09-10T08:21:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.028+0000 7f831b200640 1 -- 172.21.15.2:0/1138231148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f831c1079c0 msgr2=0x7f831c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:21:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.028+0000 7f831b200640 1 --2- 172.21.15.2:0/1138231148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f831c1079c0 0x7f831c074f70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8314009430 tx=0x7f8314003f10 comp rx=0 tx=0).stop 2024-09-10T08:21:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.028+0000 7f831b200640 1 -- 172.21.15.2:0/1138231148 shutdown_connections 2024-09-10T08:21:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.028+0000 7f831b200640 1 --2- 172.21.15.2:0/1138231148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f831c1079c0 0x7f831c074f70 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.028+0000 7f831b200640 1 --2- 172.21.15.2:0/1138231148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f831c102360 0x7f831c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.028+0000 7f831b200640 1 --2- 172.21.15.2:0/1138231148 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f83000a2b80 0x7f83000a5040 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.028+0000 7f831b200640 1 --2- 172.21.15.2:0/1138231148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f831c1019c0 0x7f831c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:21:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.028+0000 7f831b200640 1 -- 172.21.15.2:0/1138231148 >> 172.21.15.2:0/1138231148 conn(0x7f831c0fd740 msgr2=0x7f831c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:21:59.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.028+0000 7f831b200640 1 -- 172.21.15.2:0/1138231148 shutdown_connections 2024-09-10T08:21:59.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:21:59.028+0000 7f831b200640 1 -- 172.21.15.2:0/1138231148 wait complete. 2024-09-10T08:21:59.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:21:59 smithi002 ceph-mon[30563]: pgmap v802: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-10T08:21:59.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:21:59 smithi155 ceph-mon[35405]: pgmap v802: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-10T08:21:59.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:21:59 smithi107 ceph-mon[35048]: pgmap v802: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-10T08:22:00.005 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:22:00.387 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:22:00.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:00 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:22:00.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:00 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:22:00.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:00 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:22:00.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:00 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:22:00.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:00 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:22:00.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:00 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:22:00.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:00 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:22:00.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:00 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:22:00.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:00 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:22:00.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:00 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:22:00.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:00 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:22:00.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:00 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:22:00.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.959+0000 7f7106a00640 1 -- 172.21.15.2:0/1826961473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100074eb0 msgr2=0x7f7100112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:00.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.959+0000 7f7106a00640 1 --2- 172.21.15.2:0/1826961473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100074eb0 0x7f7100112650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009860 tx=0x7f70ec02a3d0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.960+0000 7f7106a00640 1 -- 172.21.15.2:0/1826961473 shutdown_connections 2024-09-10T08:22:00.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.960+0000 7f7106a00640 1 --2- 172.21.15.2:0/1826961473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100112b90 0x7f7100114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.960+0000 7f7106a00640 1 --2- 172.21.15.2:0/1826961473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100074eb0 0x7f7100112650 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.960+0000 7f7106a00640 1 --2- 172.21.15.2:0/1826961473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100074590 0x7f7100074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.960+0000 7f7106a00640 1 -- 172.21.15.2:0/1826961473 >> 172.21.15.2:0/1826961473 conn(0x7f71000fd8c0 msgr2=0x7f71000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:00.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.960+0000 7f7106a00640 1 -- 172.21.15.2:0/1826961473 shutdown_connections 2024-09-10T08:22:00.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.960+0000 7f7106a00640 1 -- 172.21.15.2:0/1826961473 wait complete. 2024-09-10T08:22:00.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.961+0000 7f7106a00640 1 Processor -- start 2024-09-10T08:22:00.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.961+0000 7f7106a00640 1 -- start start 2024-09-10T08:22:00.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.961+0000 7f7106a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100074590 0x7f71001a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.961+0000 7f7106a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100074eb0 0x7f71001a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.961+0000 7f7106a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100112b90 0x7f710019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.961+0000 7f7106a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7100118590 con 0x7f7100074590 2024-09-10T08:22:00.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.961+0000 7f7106a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7100118410 con 0x7f7100074eb0 2024-09-10T08:22:00.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.961+0000 7f7106a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7100118710 con 0x7f7100112b90 2024-09-10T08:22:00.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.961+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100074590 0x7f71001a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.961+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100074590 0x7f71001a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57066/0 (socket says 172.21.15.2:57066) 2024-09-10T08:22:00.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.961+0000 7f70ffe00640 1 -- 172.21.15.2:0/3625966389 learned_addr learned my addr 172.21.15.2:0/3625966389 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:00.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.961+0000 7f70ff400640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100074eb0 0x7f71001a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.962+0000 7f7104a00640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100112b90 0x7f710019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.962+0000 7f70ffe00640 1 -- 172.21.15.2:0/3625966389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100112b90 msgr2=0x7f710019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:00.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.962+0000 7f70ffe00640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100112b90 0x7f710019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.962+0000 7f70ffe00640 1 -- 172.21.15.2:0/3625966389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100074eb0 msgr2=0x7f71001a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:00.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.962+0000 7f70ffe00640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100074eb0 0x7f71001a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.962+0000 7f70ffe00640 1 -- 172.21.15.2:0/3625966389 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710019f020 con 0x7f7100074590 2024-09-10T08:22:00.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.962+0000 7f70ffe00640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100074590 0x7f71001a4730 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f70f000e9c0 tx=0x7f70f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:00.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.962+0000 7f70fcc00640 1 -- 172.21.15.2:0/3625966389 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f000ccc0 con 0x7f7100074590 2024-09-10T08:22:00.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.963+0000 7f70fcc00640 1 -- 172.21.15.2:0/3625966389 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f0005500 con 0x7f7100074590 2024-09-10T08:22:00.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.963+0000 7f7106a00640 1 -- 172.21.15.2:0/3625966389 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710019f310 con 0x7f7100074590 2024-09-10T08:22:00.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.963+0000 7f70fcc00640 1 -- 172.21.15.2:0/3625966389 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0010430 con 0x7f7100074590 2024-09-10T08:22:00.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.963+0000 7f7106a00640 1 -- 172.21.15.2:0/3625966389 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71001ab600 con 0x7f7100074590 2024-09-10T08:22:00.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.967+0000 7f710947f740 1 Processor -- start 2024-09-10T08:22:00.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.967+0000 7f710947f740 1 -- start start 2024-09-10T08:22:00.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.967+0000 7f70fcc00640 1 -- 172.21.15.2:0/3625966389 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f70f00046c0 con 0x7f7100074590 2024-09-10T08:22:00.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.967+0000 7f710947f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bbff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f710947f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bc04b0 0x55c920bca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f710947f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bcad70 0x55c920bcd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f710947f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c920b263e0 con 0x55c920bc04b0 2024-09-10T08:22:00.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f710947f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c920acdd80 con 0x55c920bcad70 2024-09-10T08:22:00.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f710947f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c920b223a0 con 0x55c920bbfb90 2024-09-10T08:22:00.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bc04b0 0x55c920bca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f7104a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bbff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bc04b0 0x55c920bca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57070/0 (socket says 172.21.15.2:57070) 2024-09-10T08:22:00.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f70ffe00640 1 -- 172.21.15.2:0/2650121898 learned_addr learned my addr 172.21.15.2:0/2650121898 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:00.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f70ff400640 1 --2- 172.21.15.2:0/2650121898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bcad70 0x55c920bcd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f7104a00640 1 -- 172.21.15.2:0/2650121898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bcad70 msgr2=0x55c920bcd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:00.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f7104a00640 1 --2- 172.21.15.2:0/2650121898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bcad70 0x55c920bcd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f7104a00640 1 -- 172.21.15.2:0/2650121898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bc04b0 msgr2=0x55c920bca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:00.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f7104a00640 1 --2- 172.21.15.2:0/2650121898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bc04b0 0x55c920bca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.968+0000 7f7104a00640 1 -- 172.21.15.2:0/2650121898 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c920aecae0 con 0x55c920bbfb90 2024-09-10T08:22:00.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.969+0000 7f7104a00640 1 --2- 172.21.15.2:0/2650121898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bbff70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f70f8005fd0 tx=0x7f70f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6edb8a5a6345919 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:00.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.969+0000 7f70fcc00640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70e40a2b80 0x7f70e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.969+0000 7f70f5200640 1 -- 172.21.15.2:0/2650121898 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8004d20 con 0x55c920bbfb90 2024-09-10T08:22:00.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.969+0000 7f70fcc00640 1 -- 172.21.15.2:0/3625966389 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f70f00cf760 con 0x7f7100074590 2024-09-10T08:22:00.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.969+0000 7f70f5200640 1 -- 172.21.15.2:0/2650121898 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f8004050 con 0x55c920bbfb90 2024-09-10T08:22:00.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.969+0000 7f70f5200640 1 -- 172.21.15.2:0/2650121898 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8005280 con 0x55c920bbfb90 2024-09-10T08:22:00.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.969+0000 7f70ffe00640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70e40a2b80 0x7f70e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.970+0000 7f710947f740 1 -- 172.21.15.2:0/2650121898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 msgr2=0x55c920bbff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:00.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.970+0000 7f710947f740 1 --2- 172.21.15.2:0/2650121898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bbff70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f70f8005fd0 tx=0x7f70f8011810 comp rx=0 tx=0).stop 2024-09-10T08:22:00.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.970+0000 7f70ffe00640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70e40a2b80 0x7f70e40a5040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f70f000c6a0 tx=0x7f70f00d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:00.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.970+0000 7f710947f740 1 -- 172.21.15.2:0/2650121898 shutdown_connections 2024-09-10T08:22:00.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.970+0000 7f710947f740 1 --2- 172.21.15.2:0/2650121898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bcad70 0x55c920bcd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.970+0000 7f710947f740 1 --2- 172.21.15.2:0/2650121898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bc04b0 0x55c920bca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.970+0000 7f710947f740 1 --2- 172.21.15.2:0/2650121898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bbff70 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.970+0000 7f710947f740 1 -- 172.21.15.2:0/2650121898 >> 172.21.15.2:0/2650121898 conn(0x55c920bbc120 msgr2=0x55c920bbc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:00.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.970+0000 7f710947f740 1 -- 172.21.15.2:0/2650121898 shutdown_connections 2024-09-10T08:22:00.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.970+0000 7f710947f740 1 -- 172.21.15.2:0/2650121898 wait complete. 2024-09-10T08:22:00.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.971+0000 7f710947f740 1 -- start start 2024-09-10T08:22:00.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.971+0000 7f710947f740 1 Processor -- start 2024-09-10T08:22:00.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.971+0000 7f710947f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bc7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.971+0000 7f710947f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bc04b0 0x55c920bc8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.971+0000 7f710947f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bcad70 0x55c920bc86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.971+0000 7f710947f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c920bcd9e0 con 0x55c920bcad70 2024-09-10T08:22:00.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.971+0000 7f710947f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c920ae10f0 con 0x55c920bc04b0 2024-09-10T08:22:00.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.971+0000 7f710947f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c920bcdb60 con 0x55c920bbfb90 2024-09-10T08:22:00.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.972+0000 7f710947f740 10 client.0 _ll_register_callbacks cb 0x55c920ad29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:00.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.972+0000 7f7104a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bc7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.972+0000 7f7104a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bc7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:46980/0 (socket says 172.21.15.2:46980) 2024-09-10T08:22:00.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.972+0000 7f7104a00640 1 -- 172.21.15.2:0/703487769 learned_addr learned my addr 172.21.15.2:0/703487769 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:00.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.972+0000 7f70ff400640 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bc04b0 0x55c920bc8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.972+0000 7f7104a00640 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bcad70 0x55c920bc86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.972+0000 7f70ff400640 1 -- 172.21.15.2:0/703487769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 msgr2=0x55c920bc7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:00.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.972+0000 7f70ff400640 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bc7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.972+0000 7f70ff400640 1 -- 172.21.15.2:0/703487769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bcad70 msgr2=0x55c920bc86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:00.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.972+0000 7f70ff400640 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bcad70 0x55c920bc86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.972+0000 7f70ff400640 1 -- 172.21.15.2:0/703487769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9208973c0 con 0x55c920bc04b0 2024-09-10T08:22:00.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.972+0000 7f7104a00640 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bc7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:00.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.973+0000 7f70ff400640 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bc04b0 0x55c920bc8190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009830 tx=0x7f70ec004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6c155492289ba57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:00.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.973+0000 7f70e9600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:22:00.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.973+0000 7f70e9600640 1 -- 172.21.15.2:0/703487769 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec0305a0 con 0x55c920bc04b0 2024-09-10T08:22:00.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.973+0000 7f710947f740 1 -- 172.21.15.2:0/703487769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70f8004050 con 0x55c920bc04b0 2024-09-10T08:22:00.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.973+0000 7f70e9600640 1 -- 172.21.15.2:0/703487769 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f70ec030740 con 0x55c920bc04b0 2024-09-10T08:22:00.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.973+0000 7f70e9600640 1 -- 172.21.15.2:0/703487769 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f70ec04f050 con 0x55c920bc04b0 2024-09-10T08:22:00.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.973+0000 7f710947f740 10 client.29456 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:00.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.973+0000 7f710947f740 1 -- 172.21.15.2:0/703487769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c920bd96e0 con 0x55c920bc04b0 2024-09-10T08:22:00.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.974+0000 7f70e9600640 1 -- 172.21.15.2:0/703487769 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f8004050 con 0x55c920bc04b0 2024-09-10T08:22:00.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.974+0000 7f70e9600640 1 -- 172.21.15.2:0/703487769 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f70ec0035f0 con 0x55c920bc04b0 2024-09-10T08:22:00.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.974+0000 7f710947f740 10 client.29456 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:00.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.974+0000 7f710947f740 10 client.29456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:00.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.974+0000 7f710947f740 20 client.29456 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:00.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.974+0000 7f710947f740 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c920bdab40 0x55c920bdcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.974+0000 7f710947f740 4 client.29456 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:22:00.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.974+0000 7f710947f740 1 -- 172.21.15.2:0/703487769 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c920b33950 con 0x55c920bdab40 2024-09-10T08:22:00.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.974+0000 7f7104a00640 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c920bdab40 0x55c920bdcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.975+0000 7f7104a00640 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c920bdab40 0x55c920bdcf60 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c364af5680d74b6b server_cookie=9c9bcca67ec4c7d0 in_seq=0 out_seq=0 2024-09-10T08:22:00.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.975+0000 7f70e9600640 10 client.29456 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:00.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.976+0000 7f70e9600640 1 -- 172.21.15.2:0/703487769 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c920b33950 con 0x55c920bdab40 2024-09-10T08:22:00.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.976+0000 7f70e9600640 10 client.29456 handle_command_reply: tid=0 2024-09-10T08:22:00.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.976+0000 7f710947f740 1 client.29456 shutdown 2024-09-10T08:22:00.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.976+0000 7f710947f740 1 -- 172.21.15.2:0/703487769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bc04b0 msgr2=0x55c920bc8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:00.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.976+0000 7f710947f740 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bc04b0 0x55c920bc8190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009830 tx=0x7f70ec004270 comp rx=0 tx=0).stop 2024-09-10T08:22:00.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.976+0000 7f710947f740 1 -- 172.21.15.2:0/703487769 shutdown_connections 2024-09-10T08:22:00.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.976+0000 7f710947f740 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c920bdab40 0x55c920bdcf60 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.976+0000 7f710947f740 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bcad70 0x55c920bc86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.976+0000 7f710947f740 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bc04b0 0x55c920bc8190 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.976+0000 7f70e9600640 0 client.29456 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:00.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.976+0000 7f710947f740 1 --2- 172.21.15.2:0/703487769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bc7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.976+0000 7f710947f740 1 -- 172.21.15.2:0/703487769 >> 172.21.15.2:0/703487769 conn(0x55c920bbc120 msgr2=0x55c920bc1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:00.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.976+0000 7f710947f740 1 -- 172.21.15.2:0/703487769 shutdown_connections 2024-09-10T08:22:00.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.977+0000 7f710947f740 1 -- 172.21.15.2:0/703487769 wait complete. 2024-09-10T08:22:00.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.977+0000 7f710947f740 20 client.29456 trim_cache size 0 max 16384 2024-09-10T08:22:00.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.995+0000 7f710947f740 1 Processor -- start 2024-09-10T08:22:00.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.995+0000 7f710947f740 1 -- start start 2024-09-10T08:22:00.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.995+0000 7f710947f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bc9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.995+0000 7f710947f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bc04b0 0x55c920be2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.995+0000 7f710947f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bcad70 0x55c920be2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:00.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.995+0000 7f710947f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f71001b7dc0 con 0x55c920bc04b0 2024-09-10T08:22:00.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.995+0000 7f710947f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f71001b5690 con 0x55c920bcad70 2024-09-10T08:22:00.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.995+0000 7f710947f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f71001b34e0 con 0x55c920bbfb90 2024-09-10T08:22:00.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.995+0000 7f70ff400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bc04b0 0x55c920be2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.995+0000 7f7104a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bcad70 0x55c920be2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.995+0000 7f7104a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bcad70 0x55c920be2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:52206/0 (socket says 172.21.15.2:52206) 2024-09-10T08:22:00.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.995+0000 7f7104a00640 1 -- 172.21.15.2:0/3864304240 learned_addr learned my addr 172.21.15.2:0/3864304240 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:00.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.995+0000 7f7104a00640 1 --2- 172.21.15.2:0/3864304240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bc9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.996+0000 7f7104a00640 1 -- 172.21.15.2:0/3864304240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 msgr2=0x55c920bc9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.996+0000 7f7104a00640 1 --2- 172.21.15.2:0/3864304240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bc9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.996+0000 7f7104a00640 1 -- 172.21.15.2:0/3864304240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bc04b0 msgr2=0x55c920be2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.996+0000 7f7104a00640 1 --2- 172.21.15.2:0/3864304240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bc04b0 0x55c920be2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.996+0000 7f7104a00640 1 -- 172.21.15.2:0/3864304240 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c920bc02e0 con 0x55c920bcad70 2024-09-10T08:22:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.996+0000 7f70ff400640 1 --2- 172.21.15.2:0/3864304240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bc04b0 0x55c920be2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.996+0000 7f7104a00640 1 --2- 172.21.15.2:0/3864304240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bcad70 0x55c920be2d70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f70f8004910 tx=0x7f70f8009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec327bac1be48822 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.996+0000 7f70ebe00640 1 -- 172.21.15.2:0/3864304240 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8004480 con 0x55c920bcad70 2024-09-10T08:22:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.996+0000 7f70ebe00640 1 -- 172.21.15.2:0/3864304240 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f8004620 con 0x55c920bcad70 2024-09-10T08:22:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.996+0000 7f70ebe00640 1 -- 172.21.15.2:0/3864304240 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f80276a0 con 0x55c920bcad70 2024-09-10T08:22:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.997+0000 7f710947f740 1 -- 172.21.15.2:0/3864304240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bcad70 msgr2=0x55c920be2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.997+0000 7f710947f740 1 --2- 172.21.15.2:0/3864304240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bcad70 0x55c920be2d70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f70f8004910 tx=0x7f70f8009500 comp rx=0 tx=0).stop 2024-09-10T08:22:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.997+0000 7f710947f740 1 -- 172.21.15.2:0/3864304240 shutdown_connections 2024-09-10T08:22:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.997+0000 7f710947f740 1 --2- 172.21.15.2:0/3864304240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bcad70 0x55c920be2d70 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.997+0000 7f710947f740 1 --2- 172.21.15.2:0/3864304240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bc04b0 0x55c920be2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.997+0000 7f710947f740 1 --2- 172.21.15.2:0/3864304240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bbfb90 0x55c920bc9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.997+0000 7f710947f740 1 -- 172.21.15.2:0/3864304240 >> 172.21.15.2:0/3864304240 conn(0x55c920bbc120 msgr2=0x55c920bc94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.997+0000 7f710947f740 1 -- 172.21.15.2:0/3864304240 shutdown_connections 2024-09-10T08:22:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.997+0000 7f710947f740 1 -- 172.21.15.2:0/3864304240 wait complete. 2024-09-10T08:22:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.997+0000 7f710947f740 1 -- start start 2024-09-10T08:22:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.998+0000 7f710947f740 1 Processor -- start 2024-09-10T08:22:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.998+0000 7f710947f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bbfb90 0x55c920bd3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.998+0000 7f710947f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bc04b0 0x55c920bce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.998+0000 7f710947f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bcad70 0x55c920bcec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.998+0000 7f710947f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f71001b3b50 con 0x55c920bcad70 2024-09-10T08:22:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.998+0000 7f710947f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f71001b4fc0 con 0x55c920bbfb90 2024-09-10T08:22:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.998+0000 7f710947f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f71001b4220 con 0x55c920bc04b0 2024-09-10T08:22:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.998+0000 7f710947f740 10 client.0 _ll_register_callbacks cb 0x7f70ec009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.998+0000 7f7104a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bbfb90 0x55c920bd3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:01.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.998+0000 7f7104a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bcad70 0x55c920bcec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:01.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.998+0000 7f7104a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bcad70 0x55c920bcec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57088/0 (socket says 172.21.15.2:57088) 2024-09-10T08:22:01.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.999+0000 7f7104a00640 1 -- 172.21.15.2:0/649424229 learned_addr learned my addr 172.21.15.2:0/649424229 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:01.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.999+0000 7f70ff400640 1 --2- 172.21.15.2:0/649424229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bc04b0 0x55c920bce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:01.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.999+0000 7f7104a00640 1 -- 172.21.15.2:0/649424229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bc04b0 msgr2=0x55c920bce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:01.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.999+0000 7f7104a00640 1 --2- 172.21.15.2:0/649424229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bc04b0 0x55c920bce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:01.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.999+0000 7f7104a00640 1 -- 172.21.15.2:0/649424229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bbfb90 msgr2=0x55c920bd3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:01.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.999+0000 7f7104a00640 1 --2- 172.21.15.2:0/649424229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bbfb90 0x55c920bd3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:01.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.999+0000 7f7104a00640 1 -- 172.21.15.2:0/649424229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c920bcf4a0 con 0x55c920bcad70 2024-09-10T08:22:01.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.999+0000 7f70ff400640 1 --2- 172.21.15.2:0/649424229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bc04b0 0x55c920bce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.999+0000 7f7104a00640 1 --2- 172.21.15.2:0/649424229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bcad70 0x55c920bcec70 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f70f8005fd0 tx=0x7f70f8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db201de0c6a38e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.999+0000 7f70e9600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:22:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.999+0000 7f70e9600640 1 -- 172.21.15.2:0/649424229 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8045060 con 0x55c920bcad70 2024-09-10T08:22:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.999+0000 7f70e9600640 1 -- 172.21.15.2:0/649424229 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f70f8004300 con 0x55c920bcad70 2024-09-10T08:22:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:00.999+0000 7f710947f740 1 -- 172.21.15.2:0/649424229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70f8004620 con 0x55c920bcad70 2024-09-10T08:22:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.000+0000 7f70e9600640 1 -- 172.21.15.2:0/649424229 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f70f8019430 con 0x55c920bcad70 2024-09-10T08:22:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.000+0000 7f710947f740 10 client.19437 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.000+0000 7f710947f740 1 -- 172.21.15.2:0/649424229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c920c45710 con 0x55c920bcad70 2024-09-10T08:22:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.000+0000 7f70e9600640 1 -- 172.21.15.2:0/649424229 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f8004620 con 0x55c920bcad70 2024-09-10T08:22:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.000+0000 7f70e9600640 1 -- 172.21.15.2:0/649424229 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f70f80289c0 con 0x55c920bcad70 2024-09-10T08:22:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.000+0000 7f710947f740 10 client.19437 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.000+0000 7f710947f740 10 client.19437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.000+0000 7f710947f740 20 client.19437 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.000+0000 7f710947f740 1 --2- 172.21.15.2:0/649424229 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c920c45b90 0x55c920c47f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.000+0000 7f710947f740 4 client.19437 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:22:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.000+0000 7f710947f740 1 -- 172.21.15.2:0/649424229 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c920b2de60 con 0x55c920c45b90 2024-09-10T08:22:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.001+0000 7f70ff400640 1 --2- 172.21.15.2:0/649424229 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c920c45b90 0x55c920c47f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.001+0000 7f70ff400640 1 --2- 172.21.15.2:0/649424229 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c920c45b90 0x55c920c47f80 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e67a6a9f440e4a2 server_cookie=316e47d2b8a23059 in_seq=0 out_seq=0 2024-09-10T08:22:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.001+0000 7f70e9600640 10 client.19437 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.001+0000 7f70e9600640 1 -- 172.21.15.2:0/649424229 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c920b2de60 con 0x55c920c45b90 2024-09-10T08:22:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.001+0000 7f70e9600640 10 client.19437 handle_command_reply: tid=0 2024-09-10T08:22:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.001+0000 7f710947f740 1 client.19437 shutdown 2024-09-10T08:22:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.002+0000 7f710947f740 1 -- 172.21.15.2:0/649424229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bcad70 msgr2=0x55c920bcec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.002+0000 7f710947f740 1 --2- 172.21.15.2:0/649424229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bcad70 0x55c920bcec70 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f70f8005fd0 tx=0x7f70f8000f40 comp rx=0 tx=0).stop 2024-09-10T08:22:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.002+0000 7f710947f740 1 -- 172.21.15.2:0/649424229 shutdown_connections 2024-09-10T08:22:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.002+0000 7f710947f740 1 --2- 172.21.15.2:0/649424229 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55c920c45b90 0x55c920c47f80 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.002+0000 7f710947f740 1 --2- 172.21.15.2:0/649424229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c920bcad70 0x55c920bcec70 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.002+0000 7f710947f740 1 --2- 172.21.15.2:0/649424229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c920bc04b0 0x55c920bce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.002+0000 7f710947f740 1 --2- 172.21.15.2:0/649424229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c920bbfb90 0x55c920bd3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.002+0000 7f710947f740 1 -- 172.21.15.2:0/649424229 >> 172.21.15.2:0/649424229 conn(0x55c920bbc120 msgr2=0x55c920bc8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.002+0000 7f70e9600640 0 client.19437 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.002+0000 7f710947f740 1 -- 172.21.15.2:0/649424229 shutdown_connections 2024-09-10T08:22:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.002+0000 7f710947f740 1 -- 172.21.15.2:0/649424229 wait complete. 2024-09-10T08:22:01.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.002+0000 7f710947f740 20 client.19437 trim_cache size 0 max 16384 2024-09-10T08:22:01.007 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:22:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:22:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:22:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: "847dc39c-e1b8-4aff-8f91-99de3dd3bffb": { 2024-09-10T08:22:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:22:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "847dc39c-e1b8-4aff-8f91-99de3dd3bffb", 2024-09-10T08:22:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:22:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:22:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:22:01.008 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:22:01.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.003+0000 7f7106a00640 1 -- 172.21.15.2:0/3625966389 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70e40a2b80 msgr2=0x7f70e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.003+0000 7f7106a00640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70e40a2b80 0x7f70e40a5040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f70f000c6a0 tx=0x7f70f00d5530 comp rx=0 tx=0).stop 2024-09-10T08:22:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.003+0000 7f7106a00640 1 -- 172.21.15.2:0/3625966389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100074590 msgr2=0x7f71001a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.003+0000 7f7106a00640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100074590 0x7f71001a4730 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f70f000e9c0 tx=0x7f70f000ee80 comp rx=0 tx=0).stop 2024-09-10T08:22:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.003+0000 7f7106a00640 1 -- 172.21.15.2:0/3625966389 shutdown_connections 2024-09-10T08:22:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.003+0000 7f7106a00640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70e40a2b80 0x7f70e40a5040 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.003+0000 7f7106a00640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100112b90 0x7f710019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.003+0000 7f7106a00640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7100074eb0 0x7f71001a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.003+0000 7f7106a00640 1 --2- 172.21.15.2:0/3625966389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7100074590 0x7f71001a4730 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.003+0000 7f7106a00640 1 -- 172.21.15.2:0/3625966389 >> 172.21.15.2:0/3625966389 conn(0x7f71000fd8c0 msgr2=0x7f71000ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.003+0000 7f7106a00640 1 -- 172.21.15.2:0/3625966389 shutdown_connections 2024-09-10T08:22:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:01.003+0000 7f7106a00640 1 -- 172.21.15.2:0/3625966389 wait complete. 2024-09-10T08:22:01.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:01 smithi002 ceph-mon[30563]: pgmap v803: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-10T08:22:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:01 smithi155 ceph-mon[35405]: pgmap v803: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-10T08:22:01.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:01 smithi107 ceph-mon[35048]: pgmap v803: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-10T08:22:01.821 INFO:tasks.cephfs.filesystem:scrub status for tag:847dc39c-e1b8-4aff-8f91-99de3dd3bffb - {'path': '/', 'tag': '847dc39c-e1b8-4aff-8f91-99de3dd3bffb', 'options': 'recursive,force'} 2024-09-10T08:22:03.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:03 smithi002 ceph-mon[30563]: pgmap v804: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-09-10T08:22:03.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:03 smithi155 ceph-mon[35405]: pgmap v804: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-09-10T08:22:03.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:03 smithi107 ceph-mon[35048]: pgmap v804: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-09-10T08:22:04.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:04 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:22:04.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:04 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:22:04.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:04 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:22:05.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:05 smithi155 ceph-mon[35405]: pgmap v805: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-09-10T08:22:05.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:05 smithi107 ceph-mon[35048]: pgmap v805: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-09-10T08:22:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:05 smithi002 ceph-mon[30563]: pgmap v805: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-09-10T08:22:06.216 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:22:06.613 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.905+0000 7fbc6ac00640 1 -- 172.21.15.2:0/2790040975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc64104f50 msgr2=0x7fbc641053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.905+0000 7fbc6ac00640 1 --2- 172.21.15.2:0/2790040975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc64104f50 0x7fbc641053d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009900 tx=0x7fbc4c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.905+0000 7fbc6ac00640 1 -- 172.21.15.2:0/2790040975 shutdown_connections 2024-09-10T08:22:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.905+0000 7fbc6ac00640 1 --2- 172.21.15.2:0/2790040975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc64105910 0x7fbc6410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.905+0000 7fbc6ac00640 1 --2- 172.21.15.2:0/2790040975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc64104f50 0x7fbc641053d0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.905+0000 7fbc6ac00640 1 --2- 172.21.15.2:0/2790040975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc64103d60 0x7fbc64104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.905+0000 7fbc6ac00640 1 -- 172.21.15.2:0/2790040975 >> 172.21.15.2:0/2790040975 conn(0x7fbc640ff4e0 msgr2=0x7fbc64101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.905+0000 7fbc6ac00640 1 -- 172.21.15.2:0/2790040975 shutdown_connections 2024-09-10T08:22:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.905+0000 7fbc6ac00640 1 -- 172.21.15.2:0/2790040975 wait complete. 2024-09-10T08:22:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.906+0000 7fbc6ac00640 1 Processor -- start 2024-09-10T08:22:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.906+0000 7fbc6ac00640 1 -- start start 2024-09-10T08:22:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.906+0000 7fbc6ac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc64103d60 0x7fbc64199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.906+0000 7fbc6ac00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc64104f50 0x7fbc6419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.906+0000 7fbc6ac00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc64105910 0x7fbc641a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.906+0000 7fbc6ac00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6410f9e0 con 0x7fbc64104f50 2024-09-10T08:22:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.906+0000 7fbc6ac00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6410f860 con 0x7fbc64105910 2024-09-10T08:22:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.906+0000 7fbc6ac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6410fb60 con 0x7fbc64103d60 2024-09-10T08:22:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.907+0000 7fbc5be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc64104f50 0x7fbc6419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.907+0000 7fbc68c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc64105910 0x7fbc641a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.907+0000 7fbc63e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc64103d60 0x7fbc64199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.907+0000 7fbc68c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc64105910 0x7fbc641a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42810/0 (socket says 172.21.15.2:42810) 2024-09-10T08:22:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.907+0000 7fbc68c00640 1 -- 172.21.15.2:0/1112696785 learned_addr learned my addr 172.21.15.2:0/1112696785 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.907+0000 7fbc68c00640 1 -- 172.21.15.2:0/1112696785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc64103d60 msgr2=0x7fbc64199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.907+0000 7fbc68c00640 1 --2- 172.21.15.2:0/1112696785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc64103d60 0x7fbc64199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.907+0000 7fbc68c00640 1 -- 172.21.15.2:0/1112696785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc64104f50 msgr2=0x7fbc6419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.907+0000 7fbc68c00640 1 --2- 172.21.15.2:0/1112696785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc64104f50 0x7fbc6419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.907+0000 7fbc68c00640 1 -- 172.21.15.2:0/1112696785 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc641a1b90 con 0x7fbc64105910 2024-09-10T08:22:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.907+0000 7fbc63e00640 1 --2- 172.21.15.2:0/1112696785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc64103d60 0x7fbc64199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.907+0000 7fbc68c00640 1 --2- 172.21.15.2:0/1112696785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc64105910 0x7fbc641a1520 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00eb10 tx=0x7fbc5c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.908+0000 7fbc61600640 1 -- 172.21.15.2:0/1112696785 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c004b20 con 0x7fbc64105910 2024-09-10T08:22:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.908+0000 7fbc61600640 1 -- 172.21.15.2:0/1112696785 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc5c004cc0 con 0x7fbc64105910 2024-09-10T08:22:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.908+0000 7fbc6ac00640 1 -- 172.21.15.2:0/1112696785 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc641a1e80 con 0x7fbc64105910 2024-09-10T08:22:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.908+0000 7fbc61600640 1 -- 172.21.15.2:0/1112696785 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c010760 con 0x7fbc64105910 2024-09-10T08:22:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.908+0000 7fbc6ac00640 1 -- 172.21.15.2:0/1112696785 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc641a2340 con 0x7fbc64105910 2024-09-10T08:22:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.912+0000 7fbc61600640 1 -- 172.21.15.2:0/1112696785 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbc5c005020 con 0x7fbc64105910 2024-09-10T08:22:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.912+0000 7fbc6dbc8740 1 Processor -- start 2024-09-10T08:22:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.912+0000 7fbc6dbc8740 1 -- start start 2024-09-10T08:22:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.912+0000 7fbc6dbc8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 0x55a46e9d5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.912+0000 7fbc6dbc8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d59d0 0x55a46e9da490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.912+0000 7fbc6dbc8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9daa60 0x55a46e9dcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.912+0000 7fbc6dbc8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a46e93a100 con 0x55a46e9d59d0 2024-09-10T08:22:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.912+0000 7fbc6dbc8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a46e8e21b0 con 0x55a46e9d5090 2024-09-10T08:22:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.912+0000 7fbc6dbc8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a46e9360c0 con 0x55a46e9daa60 2024-09-10T08:22:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.913+0000 7fbc5be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 0x55a46e9d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.913+0000 7fbc5be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 0x55a46e9d5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42824/0 (socket says 172.21.15.2:42824) 2024-09-10T08:22:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.913+0000 7fbc5be00640 1 -- 172.21.15.2:0/2590339663 learned_addr learned my addr 172.21.15.2:0/2590339663 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.913+0000 7fbc63e00640 1 --2- 172.21.15.2:0/2590339663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d59d0 0x55a46e9da490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.913+0000 7fbc5be00640 1 --2- 172.21.15.2:0/2590339663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9daa60 0x55a46e9dcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.913+0000 7fbc5be00640 1 -- 172.21.15.2:0/2590339663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9daa60 msgr2=0x55a46e9dcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.913+0000 7fbc5be00640 1 --2- 172.21.15.2:0/2590339663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9daa60 0x55a46e9dcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.913+0000 7fbc5be00640 1 -- 172.21.15.2:0/2590339663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d59d0 msgr2=0x55a46e9da490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.913+0000 7fbc5be00640 1 --2- 172.21.15.2:0/2590339663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d59d0 0x55a46e9da490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.913+0000 7fbc5be00640 1 -- 172.21.15.2:0/2590339663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a46e64d210 con 0x55a46e9d5090 2024-09-10T08:22:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.914+0000 7fbc61600640 1 --2- 172.21.15.2:0/1112696785 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbc480a2c30 0x7fbc480a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.914+0000 7fbc5be00640 1 --2- 172.21.15.2:0/2590339663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 0x55a46e9d5490 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009900 tx=0x7fbc4c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ddc10543df85635 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.914+0000 7fbc61600640 1 -- 172.21.15.2:0/1112696785 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbc5c0cfad0 con 0x7fbc64105910 2024-09-10T08:22:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.914+0000 7fbc63e00640 1 --2- 172.21.15.2:0/1112696785 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbc480a2c30 0x7fbc480a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.914+0000 7fbc59200640 1 -- 172.21.15.2:0/2590339663 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c035070 con 0x55a46e9d5090 2024-09-10T08:22:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.914+0000 7fbc59200640 1 -- 172.21.15.2:0/2590339663 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc4c005030 con 0x55a46e9d5090 2024-09-10T08:22:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.914+0000 7fbc59200640 1 -- 172.21.15.2:0/2590339663 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0049c0 con 0x55a46e9d5090 2024-09-10T08:22:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.915+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2590339663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 msgr2=0x55a46e9d5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.915+0000 7fbc63e00640 1 --2- 172.21.15.2:0/1112696785 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbc480a2c30 0x7fbc480a50f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55a46e9d6400 tx=0x7fbc54005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.915+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2590339663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 0x55a46e9d5490 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009900 tx=0x7fbc4c005fd0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.915+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2590339663 shutdown_connections 2024-09-10T08:22:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.915+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2590339663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9daa60 0x55a46e9dcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.915+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2590339663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d59d0 0x55a46e9da490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.915+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2590339663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 0x55a46e9d5490 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.915+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2590339663 >> 172.21.15.2:0/2590339663 conn(0x55a46e9d0700 msgr2=0x55a46e9d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.915+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2590339663 shutdown_connections 2024-09-10T08:22:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.915+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2590339663 wait complete. 2024-09-10T08:22:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.916+0000 7fbc6dbc8740 1 -- start start 2024-09-10T08:22:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.916+0000 7fbc6dbc8740 1 Processor -- start 2024-09-10T08:22:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.916+0000 7fbc6dbc8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d5090 0x55a46e9df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.916+0000 7fbc6dbc8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9df7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.916+0000 7fbc6dbc8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9daa60 0x55a46e9e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.916+0000 7fbc6dbc8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a46e9dd9d0 con 0x55a46e9d5090 2024-09-10T08:22:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.916+0000 7fbc6dbc8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a46e8c3090 con 0x55a46e9daa60 2024-09-10T08:22:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.916+0000 7fbc6dbc8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a46e9ddb50 con 0x55a46e9d59d0 2024-09-10T08:22:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.916+0000 7fbc6dbc8740 10 client.0 _ll_register_callbacks cb 0x55a46e917f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.917+0000 7fbc5be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9df7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.917+0000 7fbc63e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d5090 0x55a46e9df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.917+0000 7fbc5be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9df7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:47674/0 (socket says 172.21.15.2:47674) 2024-09-10T08:22:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.917+0000 7fbc5be00640 1 -- 172.21.15.2:0/2361449410 learned_addr learned my addr 172.21.15.2:0/2361449410 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.917+0000 7fbc68c00640 1 --2- 172.21.15.2:0/2361449410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9daa60 0x55a46e9e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.917+0000 7fbc5be00640 1 -- 172.21.15.2:0/2361449410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9daa60 msgr2=0x55a46e9e67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.917+0000 7fbc5be00640 1 --2- 172.21.15.2:0/2361449410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9daa60 0x55a46e9e67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.917+0000 7fbc5be00640 1 -- 172.21.15.2:0/2361449410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d5090 msgr2=0x55a46e9df270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.917+0000 7fbc5be00640 1 --2- 172.21.15.2:0/2361449410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d5090 0x55a46e9df270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.917+0000 7fbc5be00640 1 -- 172.21.15.2:0/2361449410 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46e719620 con 0x55a46e9d59d0 2024-09-10T08:22:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.918+0000 7fbc5be00640 1 --2- 172.21.15.2:0/2361449410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9df7b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c031440 tx=0x7fbc4c009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b397a56483904ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.918+0000 7fbc51600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:22:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.918+0000 7fbc51600640 1 -- 172.21.15.2:0/2361449410 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c035050 con 0x55a46e9d59d0 2024-09-10T08:22:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.918+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2361449410 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc4c005030 con 0x55a46e9d59d0 2024-09-10T08:22:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.918+0000 7fbc51600640 1 -- 172.21.15.2:0/2361449410 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbc4c0053e0 con 0x55a46e9d59d0 2024-09-10T08:22:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.918+0000 7fbc51600640 1 -- 172.21.15.2:0/2361449410 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc4c059050 con 0x55a46e9d59d0 2024-09-10T08:22:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.918+0000 7fbc6dbc8740 10 client.29401 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.918+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2361449410 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a46e9dd9d0 con 0x55a46e9d59d0 2024-09-10T08:22:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.919+0000 7fbc51600640 1 -- 172.21.15.2:0/2361449410 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc4c005030 con 0x55a46e9d59d0 2024-09-10T08:22:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.919+0000 7fbc51600640 1 -- 172.21.15.2:0/2361449410 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbc4c03d930 con 0x55a46e9d59d0 2024-09-10T08:22:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.919+0000 7fbc6dbc8740 10 client.29401 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.919+0000 7fbc6dbc8740 10 client.29401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.919+0000 7fbc6dbc8740 20 client.29401 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.919+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2361449410 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a46e9ea7b0 0x55a46e9ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.919+0000 7fbc6dbc8740 4 client.29401 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:22:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.919+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2361449410 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a46e947830 con 0x55a46e9ea7b0 2024-09-10T08:22:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.919+0000 7fbc63e00640 1 --2- 172.21.15.2:0/2361449410 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a46e9ea7b0 0x55a46e9ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.920+0000 7fbc63e00640 1 --2- 172.21.15.2:0/2361449410 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a46e9ea7b0 0x55a46e9ecba0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f06e5977f96f7d server_cookie=4c47208e7239a988 in_seq=0 out_seq=0 2024-09-10T08:22:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.920+0000 7fbc51600640 10 client.29401 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.921+0000 7fbc51600640 1 -- 172.21.15.2:0/2361449410 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a46e947830 con 0x55a46e9ea7b0 2024-09-10T08:22:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.921+0000 7fbc51600640 10 client.29401 handle_command_reply: tid=0 2024-09-10T08:22:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.921+0000 7fbc6dbc8740 1 client.29401 shutdown 2024-09-10T08:22:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.921+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2361449410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 msgr2=0x55a46e9df7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.921+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2361449410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9df7b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c031440 tx=0x7fbc4c009520 comp rx=0 tx=0).stop 2024-09-10T08:22:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.921+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2361449410 shutdown_connections 2024-09-10T08:22:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.921+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2361449410 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a46e9ea7b0 0x55a46e9ecba0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.921+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2361449410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9daa60 0x55a46e9e67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.921+0000 7fbc51600640 0 client.29401 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.921+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2361449410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9df7b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.921+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2361449410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d5090 0x55a46e9df270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.921+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2361449410 >> 172.21.15.2:0/2361449410 conn(0x55a46e9d0700 msgr2=0x55a46e9d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.922+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2361449410 shutdown_connections 2024-09-10T08:22:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.922+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2361449410 wait complete. 2024-09-10T08:22:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.922+0000 7fbc6dbc8740 20 client.29401 trim_cache size 0 max 16384 2024-09-10T08:22:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.938+0000 7fbc6dbc8740 1 Processor -- start 2024-09-10T08:22:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.938+0000 7fbc6dbc8740 1 -- start start 2024-09-10T08:22:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.939+0000 7fbc6dbc8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 0x55a46e9d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.939+0000 7fbc6dbc8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9e67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.939+0000 7fbc6dbc8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9daa60 0x55a46e9d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.939+0000 7fbc6dbc8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641af2d0 con 0x55a46e9daa60 2024-09-10T08:22:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.939+0000 7fbc6dbc8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641acba0 con 0x55a46e9d5090 2024-09-10T08:22:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.939+0000 7fbc6dbc8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641aa9f0 con 0x55a46e9d59d0 2024-09-10T08:22:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.939+0000 7fbc68c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9daa60 0x55a46e9d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.939+0000 7fbc63e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 0x55a46e9d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.939+0000 7fbc5be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9e67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.939+0000 7fbc68c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9daa60 0x55a46e9d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57734/0 (socket says 172.21.15.2:57734) 2024-09-10T08:22:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.939+0000 7fbc63e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 0x55a46e9d15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42844/0 (socket says 172.21.15.2:42844) 2024-09-10T08:22:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.939+0000 7fbc68c00640 1 -- 172.21.15.2:0/1576828313 learned_addr learned my addr 172.21.15.2:0/1576828313 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.940+0000 7fbc5be00640 1 -- 172.21.15.2:0/1576828313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 msgr2=0x55a46e9d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.940+0000 7fbc5be00640 1 --2- 172.21.15.2:0/1576828313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 0x55a46e9d15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.940+0000 7fbc5be00640 1 -- 172.21.15.2:0/1576828313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9daa60 msgr2=0x55a46e9d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.940+0000 7fbc5be00640 1 --2- 172.21.15.2:0/1576828313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9daa60 0x55a46e9d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.940+0000 7fbc5be00640 1 -- 172.21.15.2:0/1576828313 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a46e9d0520 con 0x55a46e9d59d0 2024-09-10T08:22:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.940+0000 7fbc63e00640 1 --2- 172.21.15.2:0/1576828313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 0x55a46e9d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.940+0000 7fbc68c00640 1 --2- 172.21.15.2:0/1576828313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9daa60 0x55a46e9d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.940+0000 7fbc5be00640 1 --2- 172.21.15.2:0/1576828313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9e67b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c000c00 tx=0x7fbc4c031910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17236784ce3554d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.940+0000 7fbc53e00640 1 -- 172.21.15.2:0/1576828313 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c035070 con 0x55a46e9d59d0 2024-09-10T08:22:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.940+0000 7fbc53e00640 1 -- 172.21.15.2:0/1576828313 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc4c009780 con 0x55a46e9d59d0 2024-09-10T08:22:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.940+0000 7fbc53e00640 1 -- 172.21.15.2:0/1576828313 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c03d450 con 0x55a46e9d59d0 2024-09-10T08:22:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.941+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/1576828313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 msgr2=0x55a46e9e67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.941+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/1576828313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9e67b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c000c00 tx=0x7fbc4c031910 comp rx=0 tx=0).stop 2024-09-10T08:22:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.941+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/1576828313 shutdown_connections 2024-09-10T08:22:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.941+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/1576828313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9daa60 0x55a46e9d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.941+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/1576828313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9e67b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.941+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/1576828313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9d5090 0x55a46e9d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.941+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/1576828313 >> 172.21.15.2:0/1576828313 conn(0x55a46e9d0700 msgr2=0x55a46e9d6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.941+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/1576828313 shutdown_connections 2024-09-10T08:22:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.941+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/1576828313 wait complete. 2024-09-10T08:22:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.942+0000 7fbc6dbc8740 1 -- start start 2024-09-10T08:22:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.942+0000 7fbc6dbc8740 1 Processor -- start 2024-09-10T08:22:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.943+0000 7fbc6dbc8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d5090 0x55a46e9e3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.943+0000 7fbc6dbc8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.943+0000 7fbc6dbc8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9daa60 0x55a46e9e4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.943+0000 7fbc6dbc8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641ab060 con 0x55a46e9d5090 2024-09-10T08:22:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.943+0000 7fbc6dbc8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641ac4d0 con 0x55a46e9daa60 2024-09-10T08:22:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.943+0000 7fbc6dbc8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641ab730 con 0x55a46e9d59d0 2024-09-10T08:22:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.943+0000 7fbc6dbc8740 10 client.0 _ll_register_callbacks cb 0x7fbc54009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.943+0000 7fbc63e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d5090 0x55a46e9e3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.943+0000 7fbc63e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d5090 0x55a46e9e3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57738/0 (socket says 172.21.15.2:57738) 2024-09-10T08:22:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.943+0000 7fbc63e00640 1 -- 172.21.15.2:0/2531791221 learned_addr learned my addr 172.21.15.2:0/2531791221 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.943+0000 7fbc5be00640 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.943+0000 7fbc68c00640 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9daa60 0x55a46e9e4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.943+0000 7fbc5be00640 1 -- 172.21.15.2:0/2531791221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9daa60 msgr2=0x55a46e9e4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.944+0000 7fbc5be00640 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9daa60 0x55a46e9e4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.944+0000 7fbc5be00640 1 -- 172.21.15.2:0/2531791221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d5090 msgr2=0x55a46e9e3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.944+0000 7fbc5be00640 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d5090 0x55a46e9e3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.944+0000 7fbc5be00640 1 -- 172.21.15.2:0/2531791221 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46e9e53d0 con 0x55a46e9d59d0 2024-09-10T08:22:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.944+0000 7fbc63e00640 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d5090 0x55a46e9e3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.944+0000 7fbc5be00640 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9e3ec0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55a46e9da7f0 tx=0x7fbc4c005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86028b2209f0f2d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.944+0000 7fbc51600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:22:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.944+0000 7fbc51600640 1 -- 172.21.15.2:0/2531791221 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c035040 con 0x55a46e9d59d0 2024-09-10T08:22:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.944+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2531791221 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc4c009780 con 0x55a46e9d59d0 2024-09-10T08:22:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.944+0000 7fbc51600640 1 -- 172.21.15.2:0/2531791221 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbc4c004990 con 0x55a46e9d59d0 2024-09-10T08:22:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.945+0000 7fbc51600640 1 -- 172.21.15.2:0/2531791221 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc4c06e050 con 0x55a46e9d59d0 2024-09-10T08:22:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.945+0000 7fbc6dbc8740 10 client.29407 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.945+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2531791221 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a46e9e66e0 con 0x55a46e9d59d0 2024-09-10T08:22:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.945+0000 7fbc51600640 1 -- 172.21.15.2:0/2531791221 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc4c009780 con 0x55a46e9d59d0 2024-09-10T08:22:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.945+0000 7fbc51600640 1 -- 172.21.15.2:0/2531791221 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbc4c064030 con 0x55a46e9d59d0 2024-09-10T08:22:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.945+0000 7fbc6dbc8740 10 client.29407 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.945+0000 7fbc6dbc8740 10 client.29407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.945+0000 7fbc6dbc8740 20 client.29407 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.945+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a46ea5dea0 0x55a46ea602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.945+0000 7fbc6dbc8740 4 client.29407 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:22:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.945+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2531791221 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a46e941b80 con 0x55a46ea5dea0 2024-09-10T08:22:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.946+0000 7fbc63e00640 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a46ea5dea0 0x55a46ea602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.946+0000 7fbc63e00640 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a46ea5dea0 0x55a46ea602c0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece9c8b69d4d2d67 server_cookie=197cc658b7913c86 in_seq=0 out_seq=0 2024-09-10T08:22:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.946+0000 7fbc51600640 10 client.29407 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc51600640 1 -- 172.21.15.2:0/2531791221 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a46e941b80 con 0x55a46ea5dea0 2024-09-10T08:22:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc51600640 10 client.29407 handle_command_reply: tid=0 2024-09-10T08:22:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc6dbc8740 1 client.29407 shutdown 2024-09-10T08:22:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2531791221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 msgr2=0x55a46e9e3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9e3ec0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55a46e9da7f0 tx=0x7fbc4c005dd0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2531791221 shutdown_connections 2024-09-10T08:22:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55a46ea5dea0 0x55a46ea602c0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a46e9daa60 0x55a46e9e4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc51600640 0 client.29407 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a46e9d59d0 0x55a46e9e3ec0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc6dbc8740 1 --2- 172.21.15.2:0/2531791221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a46e9d5090 0x55a46e9e3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2531791221 >> 172.21.15.2:0/2531791221 conn(0x55a46e9d0700 msgr2=0x55a46e9cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2531791221 shutdown_connections 2024-09-10T08:22:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.954+0000 7fbc6dbc8740 1 -- 172.21.15.2:0/2531791221 wait complete. 2024-09-10T08:22:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.955+0000 7fbc6dbc8740 20 client.29407 trim_cache size 0 max 16384 2024-09-10T08:22:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:22:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:22:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:22:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:22:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.955+0000 7fbc6ac00640 1 -- 172.21.15.2:0/1112696785 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbc480a2c30 msgr2=0x7fbc480a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.955+0000 7fbc6ac00640 1 --2- 172.21.15.2:0/1112696785 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbc480a2c30 0x7fbc480a50f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55a46e9d6400 tx=0x7fbc54005f90 comp rx=0 tx=0).stop 2024-09-10T08:22:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.955+0000 7fbc6ac00640 1 -- 172.21.15.2:0/1112696785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc64105910 msgr2=0x7fbc641a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.955+0000 7fbc6ac00640 1 --2- 172.21.15.2:0/1112696785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc64105910 0x7fbc641a1520 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00eb10 tx=0x7fbc5c00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.955+0000 7fbc6ac00640 1 -- 172.21.15.2:0/1112696785 shutdown_connections 2024-09-10T08:22:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.955+0000 7fbc6ac00640 1 --2- 172.21.15.2:0/1112696785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc64105910 0x7fbc641a1520 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.955+0000 7fbc6ac00640 1 --2- 172.21.15.2:0/1112696785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc64104f50 0x7fbc6419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.955+0000 7fbc6ac00640 1 --2- 172.21.15.2:0/1112696785 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbc480a2c30 0x7fbc480a50f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.955+0000 7fbc6ac00640 1 --2- 172.21.15.2:0/1112696785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc64103d60 0x7fbc64199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.955+0000 7fbc6ac00640 1 -- 172.21.15.2:0/1112696785 >> 172.21.15.2:0/1112696785 conn(0x7fbc640ff4e0 msgr2=0x7fbc64108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.956+0000 7fbc6ac00640 1 -- 172.21.15.2:0/1112696785 shutdown_connections 2024-09-10T08:22:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:06.956+0000 7fbc6ac00640 1 -- 172.21.15.2:0/1112696785 wait complete. 2024-09-10T08:22:07.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.199+0000 7f7f3b000640 1 -- 172.21.15.2:0/2558101934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f34108f60 msgr2=0x7f7f34110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:07.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.199+0000 7f7f3b000640 1 --2- 172.21.15.2:0/2558101934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f34108f60 0x7f7f34110ae0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009ca0 tx=0x7f7f2c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:22:07.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.199+0000 7f7f3b000640 1 -- 172.21.15.2:0/2558101934 shutdown_connections 2024-09-10T08:22:07.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.199+0000 7f7f3b000640 1 --2- 172.21.15.2:0/2558101934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f34108f60 0x7f7f34110ae0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.199+0000 7f7f3b000640 1 --2- 172.21.15.2:0/2558101934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f341047e0 0x7f7f34108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.199+0000 7f7f3b000640 1 --2- 172.21.15.2:0/2558101934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f34103e30 0x7f7f34104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.199+0000 7f7f3b000640 1 -- 172.21.15.2:0/2558101934 >> 172.21.15.2:0/2558101934 conn(0x7f7f340ff8d0 msgr2=0x7f7f34101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.199+0000 7f7f3b000640 1 -- 172.21.15.2:0/2558101934 shutdown_connections 2024-09-10T08:22:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.199+0000 7f7f3b000640 1 -- 172.21.15.2:0/2558101934 wait complete. 2024-09-10T08:22:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.200+0000 7f7f3b000640 1 Processor -- start 2024-09-10T08:22:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.201+0000 7f7f3b000640 1 -- start start 2024-09-10T08:22:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.201+0000 7f7f3b000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f34103e30 0x7f7f3419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.201+0000 7f7f3b000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f341047e0 0x7f7f3419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.201+0000 7f7f3b000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f34108f60 0x7f7f3419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.201+0000 7f7f3b000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7f34114010 con 0x7f7f34103e30 2024-09-10T08:22:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.201+0000 7f7f3b000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7f34113e90 con 0x7f7f341047e0 2024-09-10T08:22:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.201+0000 7f7f3b000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f34114190 con 0x7f7f34108f60 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.201+0000 7f7f39c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f34103e30 0x7f7f3419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.201+0000 7f7f39c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f34103e30 0x7f7f3419a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57748/0 (socket says 172.21.15.2:57748) 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.201+0000 7f7f39c00640 1 -- 172.21.15.2:0/3680202416 learned_addr learned my addr 172.21.15.2:0/3680202416 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.201+0000 7f7f39200640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f341047e0 0x7f7f3419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.201+0000 7f7f3a600640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f34108f60 0x7f7f3419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.202+0000 7f7f39c00640 1 -- 172.21.15.2:0/3680202416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f34108f60 msgr2=0x7f7f3419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.202+0000 7f7f39c00640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f34108f60 0x7f7f3419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.202+0000 7f7f39c00640 1 -- 172.21.15.2:0/3680202416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f341047e0 msgr2=0x7f7f3419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.202+0000 7f7f39c00640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f341047e0 0x7f7f3419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.202+0000 7f7f39c00640 1 -- 172.21.15.2:0/3680202416 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3419f380 con 0x7f7f34103e30 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.202+0000 7f7f39c00640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f34103e30 0x7f7f3419a3d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e9f0 tx=0x7f7f2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.202+0000 7f7f32000640 1 -- 172.21.15.2:0/3680202416 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800cd80 con 0x7f7f34103e30 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.202+0000 7f7f32000640 1 -- 172.21.15.2:0/3680202416 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f28010910 con 0x7f7f34103e30 2024-09-10T08:22:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.202+0000 7f7f3b000640 1 -- 172.21.15.2:0/3680202416 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3419f670 con 0x7f7f34103e30 2024-09-10T08:22:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.202+0000 7f7f32000640 1 -- 172.21.15.2:0/3680202416 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28018990 con 0x7f7f34103e30 2024-09-10T08:22:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.202+0000 7f7f3b000640 1 -- 172.21.15.2:0/3680202416 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f341a6f50 con 0x7f7f34103e30 2024-09-10T08:22:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.203+0000 7f7f3b000640 1 -- 172.21.15.2:0/3680202416 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f34104210 con 0x7f7f34103e30 2024-09-10T08:22:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.205+0000 7f7f32000640 1 -- 172.21.15.2:0/3680202416 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7f28010ac0 con 0x7f7f34103e30 2024-09-10T08:22:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.207+0000 7f7f32000640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f1c0a2b80 0x7f7f1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.207+0000 7f7f32000640 1 -- 172.21.15.2:0/3680202416 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7f280cf830 con 0x7f7f34103e30 2024-09-10T08:22:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.207+0000 7f7f39200640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f1c0a2b80 0x7f7f1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.207+0000 7f7f39200640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f1c0a2b80 0x7f7f1c0a5040 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7f3410a930 tx=0x7f7f20005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.211+0000 7f7f32000640 1 -- 172.21.15.2:0/3680202416 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f2809c340 con 0x7f7f34103e30 2024-09-10T08:22:07.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.432+0000 7f7f3b000640 1 -- 172.21.15.2:0/3680202416 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f3402d7c0 con 0x7f7f34103e30 2024-09-10T08:22:07.484 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:07 smithi002 ceph-mon[30563]: pgmap v806: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T08:22:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.432+0000 7f7f32000640 1 -- 172.21.15.2:0/3680202416 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f7f2809bc40 con 0x7f7f34103e30 2024-09-10T08:22:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:22:07.486 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:22:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:22:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.435+0000 7f7f3b000640 1 -- 172.21.15.2:0/3680202416 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f1c0a2b80 msgr2=0x7f7f1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.435+0000 7f7f3b000640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f1c0a2b80 0x7f7f1c0a5040 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7f3410a930 tx=0x7f7f20005f90 comp rx=0 tx=0).stop 2024-09-10T08:22:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.435+0000 7f7f3b000640 1 -- 172.21.15.2:0/3680202416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f34103e30 msgr2=0x7f7f3419a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.435+0000 7f7f3b000640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f34103e30 0x7f7f3419a3d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e9f0 tx=0x7f7f2800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:22:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.435+0000 7f7f3b000640 1 -- 172.21.15.2:0/3680202416 shutdown_connections 2024-09-10T08:22:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.435+0000 7f7f3b000640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f1c0a2b80 0x7f7f1c0a5040 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.435+0000 7f7f3b000640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f34108f60 0x7f7f3419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.435+0000 7f7f3b000640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f341047e0 0x7f7f3419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.435+0000 7f7f3b000640 1 --2- 172.21.15.2:0/3680202416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f34103e30 0x7f7f3419a3d0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.435+0000 7f7f3b000640 1 -- 172.21.15.2:0/3680202416 >> 172.21.15.2:0/3680202416 conn(0x7f7f340ff8d0 msgr2=0x7f7f34100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.435+0000 7f7f3b000640 1 -- 172.21.15.2:0/3680202416 shutdown_connections 2024-09-10T08:22:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:07.435+0000 7f7f3b000640 1 -- 172.21.15.2:0/3680202416 wait complete. 2024-09-10T08:22:07.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:07 smithi155 ceph-mon[35405]: pgmap v806: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T08:22:07.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:07 smithi107 ceph-mon[35048]: pgmap v806: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-10T08:22:08.257 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id 73163cc9 --release --await' 2024-09-10T08:22:08.257 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id 73163cc9 --release --await 2024-09-10T08:22:08.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:08 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3680202416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:22:08.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:08 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3680202416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:22:08.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:08 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3680202416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:22:08.673 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:22:09.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.327+0000 7f7e9b200640 1 -- 172.21.15.2:0/918063072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e94101de0 msgr2=0x7f7e941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:09.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.327+0000 7f7e9b200640 1 --2- 172.21.15.2:0/918063072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e94101de0 0x7f7e941021c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009900 tx=0x7f7e8402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.327+0000 7f7e9b200640 1 -- 172.21.15.2:0/918063072 shutdown_connections 2024-09-10T08:22:09.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.327+0000 7f7e9b200640 1 --2- 172.21.15.2:0/918063072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e94106f10 0x7f7e9410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.327+0000 7f7e9b200640 1 --2- 172.21.15.2:0/918063072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e94102790 0x7f7e941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.327+0000 7f7e9b200640 1 --2- 172.21.15.2:0/918063072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e94101de0 0x7f7e941021c0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.327+0000 7f7e9b200640 1 -- 172.21.15.2:0/918063072 >> 172.21.15.2:0/918063072 conn(0x7f7e940fd8c0 msgr2=0x7f7e940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.327+0000 7f7e9b200640 1 -- 172.21.15.2:0/918063072 shutdown_connections 2024-09-10T08:22:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.327+0000 7f7e9b200640 1 -- 172.21.15.2:0/918063072 wait complete. 2024-09-10T08:22:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.328+0000 7f7e9b200640 1 Processor -- start 2024-09-10T08:22:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.328+0000 7f7e9b200640 1 -- start start 2024-09-10T08:22:09.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.329+0000 7f7e9b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e94101de0 0x7f7e9419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.329+0000 7f7e9b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e94102790 0x7f7e9419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.329+0000 7f7e9b200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e94106f10 0x7f7e9419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.329+0000 7f7e9b200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94111fc0 con 0x7f7e94106f10 2024-09-10T08:22:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.329+0000 7f7e9b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94111e40 con 0x7f7e94101de0 2024-09-10T08:22:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.329+0000 7f7e9b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94112140 con 0x7f7e94102790 2024-09-10T08:22:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.329+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e94101de0 0x7f7e9419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.329+0000 7f7e99200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e94106f10 0x7f7e9419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.329+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e94101de0 0x7f7e9419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42878/0 (socket says 172.21.15.2:42878) 2024-09-10T08:22:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.329+0000 7f7e93e00640 1 -- 172.21.15.2:0/2095371136 learned_addr learned my addr 172.21.15.2:0/2095371136 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.329+0000 7f7e93400640 1 --2- 172.21.15.2:0/2095371136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e94102790 0x7f7e9419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.330+0000 7f7e93e00640 1 -- 172.21.15.2:0/2095371136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e94102790 msgr2=0x7f7e9419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.330+0000 7f7e93e00640 1 --2- 172.21.15.2:0/2095371136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e94102790 0x7f7e9419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.330+0000 7f7e93e00640 1 -- 172.21.15.2:0/2095371136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e94106f10 msgr2=0x7f7e9419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.330+0000 7f7e93e00640 1 --2- 172.21.15.2:0/2095371136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e94106f10 0x7f7e9419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.330+0000 7f7e93e00640 1 -- 172.21.15.2:0/2095371136 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9419f5e0 con 0x7f7e94101de0 2024-09-10T08:22:09.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.330+0000 7f7e93e00640 1 --2- 172.21.15.2:0/2095371136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e94101de0 0x7f7e9419a630 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7e84004770 tx=0x7f7e84003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:09.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.330+0000 7f7e90c00640 1 -- 172.21.15.2:0/2095371136 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8403f070 con 0x7f7e94101de0 2024-09-10T08:22:09.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.330+0000 7f7e90c00640 1 -- 172.21.15.2:0/2095371136 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e84031ca0 con 0x7f7e94101de0 2024-09-10T08:22:09.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.330+0000 7f7e9b200640 1 -- 172.21.15.2:0/2095371136 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9419f870 con 0x7f7e94101de0 2024-09-10T08:22:09.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.330+0000 7f7e90c00640 1 -- 172.21.15.2:0/2095371136 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84035030 con 0x7f7e94101de0 2024-09-10T08:22:09.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.330+0000 7f7e9b200640 1 -- 172.21.15.2:0/2095371136 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e941a71b0 con 0x7f7e94101de0 2024-09-10T08:22:09.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.333+0000 7f7e9dc85740 1 Processor -- start 2024-09-10T08:22:09.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.333+0000 7f7e90c00640 1 -- 172.21.15.2:0/2095371136 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7e84042070 con 0x7f7e94101de0 2024-09-10T08:22:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.334+0000 7f7e9dc85740 1 -- start start 2024-09-10T08:22:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.334+0000 7f7e9dc85740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8e310 0x560d60d8e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.334+0000 7f7e9dc85740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8ec30 0x560d60d98fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.334+0000 7f7e9dc85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d994f0 0x560d60d9b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.334+0000 7f7e9dc85740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560d60cf4650 con 0x560d60d8e310 2024-09-10T08:22:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.334+0000 7f7e9dc85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560d60c9bf10 con 0x560d60d994f0 2024-09-10T08:22:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.334+0000 7f7e9dc85740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d60cf0610 con 0x560d60d8ec30 2024-09-10T08:22:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.334+0000 7f7e99200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8e310 0x560d60d8e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.334+0000 7f7e99200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8e310 0x560d60d8e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57760/0 (socket says 172.21.15.2:57760) 2024-09-10T08:22:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.334+0000 7f7e99200640 1 -- 172.21.15.2:0/449667342 learned_addr learned my addr 172.21.15.2:0/449667342 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.334+0000 7f7e93e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8ec30 0x560d60d98fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.335+0000 7f7e93400640 1 --2- 172.21.15.2:0/449667342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d994f0 0x560d60d9b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.335+0000 7f7e99200640 1 -- 172.21.15.2:0/449667342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8ec30 msgr2=0x560d60d98fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.335+0000 7f7e99200640 1 --2- 172.21.15.2:0/449667342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8ec30 0x560d60d98fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.335+0000 7f7e99200640 1 -- 172.21.15.2:0/449667342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d994f0 msgr2=0x560d60d9b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.335+0000 7f7e99200640 1 --2- 172.21.15.2:0/449667342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d994f0 0x560d60d9b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.335+0000 7f7e99200640 1 -- 172.21.15.2:0/449667342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d60a03580 con 0x560d60d8e310 2024-09-10T08:22:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.335+0000 7f7e90c00640 1 --2- 172.21.15.2:0/2095371136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7e780a2b80 0x7f7e780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.335+0000 7f7e99200640 1 --2- 172.21.15.2:0/449667342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8e310 0x560d60d8e6f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009f50 tx=0x7f7e8c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb389f7ff8005e6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.335+0000 7f7e90c00640 1 -- 172.21.15.2:0/2095371136 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7e840efa70 con 0x7f7e94101de0 2024-09-10T08:22:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.336+0000 7f7e93e00640 1 --2- 172.21.15.2:0/2095371136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7e780a2b80 0x7f7e780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.336+0000 7f7e89600640 1 -- 172.21.15.2:0/449667342 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c004d20 con 0x560d60d8e310 2024-09-10T08:22:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.336+0000 7f7e89600640 1 -- 172.21.15.2:0/449667342 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8c004050 con 0x560d60d8e310 2024-09-10T08:22:09.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.336+0000 7f7e89600640 1 -- 172.21.15.2:0/449667342 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c005260 con 0x560d60d8e310 2024-09-10T08:22:09.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.336+0000 7f7e9dc85740 1 -- 172.21.15.2:0/449667342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8e310 msgr2=0x560d60d8e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:09.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.336+0000 7f7e93e00640 1 --2- 172.21.15.2:0/2095371136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7e780a2b80 0x7f7e780a5040 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7e840bbf80 tx=0x7f7e840f5510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:09.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.336+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/449667342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8e310 0x560d60d8e6f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009f50 tx=0x7f7e8c0118d0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.337+0000 7f7e9dc85740 1 -- 172.21.15.2:0/449667342 shutdown_connections 2024-09-10T08:22:09.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.337+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/449667342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d994f0 0x560d60d9b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.337+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/449667342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8ec30 0x560d60d98fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.337+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/449667342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8e310 0x560d60d8e6f0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.337+0000 7f7e9dc85740 1 -- 172.21.15.2:0/449667342 >> 172.21.15.2:0/449667342 conn(0x560d60d8a8a0 msgr2=0x560d60d8ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:09.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.337+0000 7f7e9dc85740 1 -- 172.21.15.2:0/449667342 shutdown_connections 2024-09-10T08:22:09.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.337+0000 7f7e9dc85740 1 -- 172.21.15.2:0/449667342 wait complete. 2024-09-10T08:22:09.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.337+0000 7f7e9dc85740 1 -- start start 2024-09-10T08:22:09.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.338+0000 7f7e9dc85740 1 Processor -- start 2024-09-10T08:22:09.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.338+0000 7f7e9dc85740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8e310 0x560d60d96420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.338+0000 7f7e9dc85740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 0x560d60d96960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.338+0000 7f7e9dc85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d994f0 0x560d60d96ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.338+0000 7f7e9dc85740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560d60d9c0c0 con 0x560d60d8ec30 2024-09-10T08:22:09.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.338+0000 7f7e9dc85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560d60caf350 con 0x560d60d994f0 2024-09-10T08:22:09.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.338+0000 7f7e9dc85740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d60d9c240 con 0x560d60d8e310 2024-09-10T08:22:09.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.338+0000 7f7e9dc85740 10 client.0 _ll_register_callbacks cb 0x560d60cd2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:09.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.338+0000 7f7e93400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 0x560d60d96960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.338+0000 7f7e93400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 0x560d60d96960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57766/0 (socket says 172.21.15.2:57766) 2024-09-10T08:22:09.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.338+0000 7f7e93400640 1 -- 172.21.15.2:0/593652482 learned_addr learned my addr 172.21.15.2:0/593652482 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:09.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.338+0000 7f7e99200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8e310 0x560d60d96420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.339+0000 7f7e99200640 1 --2- 172.21.15.2:0/593652482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d994f0 0x560d60d96ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.339+0000 7f7e93400640 1 -- 172.21.15.2:0/593652482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8e310 msgr2=0x560d60d96420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:09.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.339+0000 7f7e93400640 1 --2- 172.21.15.2:0/593652482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8e310 0x560d60d96420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.339+0000 7f7e93400640 1 -- 172.21.15.2:0/593652482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d994f0 msgr2=0x560d60d96ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:09.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.339+0000 7f7e93400640 1 --2- 172.21.15.2:0/593652482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d994f0 0x560d60d96ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.339+0000 7f7e93400640 1 -- 172.21.15.2:0/593652482 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d60acd960 con 0x560d60d8ec30 2024-09-10T08:22:09.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.339+0000 7f7e99200640 1 --2- 172.21.15.2:0/593652482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d994f0 0x560d60d96ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:22:09.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.339+0000 7f7e93400640 1 --2- 172.21.15.2:0/593652482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 0x560d60d96960 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f7e80012700 tx=0x7f7e80012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f325167a4499bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:09.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.339+0000 7f7e7e000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:22:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.339+0000 7f7e7e000640 1 -- 172.21.15.2:0/593652482 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80004050 con 0x560d60d8ec30 2024-09-10T08:22:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.339+0000 7f7e9dc85740 1 -- 172.21.15.2:0/593652482 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e8c004050 con 0x560d60d8ec30 2024-09-10T08:22:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.339+0000 7f7e7e000640 1 -- 172.21.15.2:0/593652482 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7e800041f0 con 0x560d60d8ec30 2024-09-10T08:22:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.340+0000 7f7e7e000640 1 -- 172.21.15.2:0/593652482 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e80025050 con 0x560d60d8ec30 2024-09-10T08:22:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.340+0000 7f7e9dc85740 10 client.19473 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.340+0000 7f7e9dc85740 1 -- 172.21.15.2:0/593652482 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d60da7de0 con 0x560d60d8ec30 2024-09-10T08:22:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.340+0000 7f7e7e000640 1 -- 172.21.15.2:0/593652482 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8c004050 con 0x560d60d8ec30 2024-09-10T08:22:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.340+0000 7f7e7e000640 1 -- 172.21.15.2:0/593652482 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7e800039c0 con 0x560d60d8ec30 2024-09-10T08:22:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.340+0000 7f7e9dc85740 10 client.19473 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.340+0000 7f7e9dc85740 10 client.19473 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:22:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.340+0000 7f7e9dc85740 20 client.19473 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.340+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/593652482 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d60da9240 0x560d60dab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.340+0000 7f7e9dc85740 4 client.19473 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:22:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.340+0000 7f7e9dc85740 1 -- 172.21.15.2:0/593652482 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d60d01bc0 con 0x560d60da9240 2024-09-10T08:22:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.341+0000 7f7e99200640 1 --2- 172.21.15.2:0/593652482 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d60da9240 0x560d60dab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.341+0000 7f7e99200640 1 --2- 172.21.15.2:0/593652482 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d60da9240 0x560d60dab660 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b64adeb904dc22 server_cookie=dd44eb683a65f92a in_seq=0 out_seq=0 2024-09-10T08:22:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.341+0000 7f7e7e000640 10 client.19473 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.342+0000 7f7e7e000640 1 -- 172.21.15.2:0/593652482 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d60d01bc0 con 0x560d60da9240 2024-09-10T08:22:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.342+0000 7f7e7e000640 10 client.19473 handle_command_reply: tid=0 2024-09-10T08:22:09.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.342+0000 7f7e9dc85740 1 client.19473 shutdown 2024-09-10T08:22:09.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.342+0000 7f7e9dc85740 1 -- 172.21.15.2:0/593652482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 msgr2=0x560d60d96960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:09.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.342+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/593652482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 0x560d60d96960 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f7e80012700 tx=0x7f7e80012bc0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.343+0000 7f7e9dc85740 1 -- 172.21.15.2:0/593652482 shutdown_connections 2024-09-10T08:22:09.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.343+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/593652482 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d60da9240 0x560d60dab660 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.343+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/593652482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d994f0 0x560d60d96ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.343+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/593652482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 0x560d60d96960 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.343+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/593652482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8e310 0x560d60d96420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.343+0000 7f7e9dc85740 1 -- 172.21.15.2:0/593652482 >> 172.21.15.2:0/593652482 conn(0x560d60d8a8a0 msgr2=0x560d60d89aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:09.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.343+0000 7f7e7e000640 0 client.19473 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:09.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.343+0000 7f7e9dc85740 1 -- 172.21.15.2:0/593652482 shutdown_connections 2024-09-10T08:22:09.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.343+0000 7f7e9dc85740 1 -- 172.21.15.2:0/593652482 wait complete. 2024-09-10T08:22:09.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.343+0000 7f7e9dc85740 20 client.19473 trim_cache size 0 max 16384 2024-09-10T08:22:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.361+0000 7f7e9dc85740 1 Processor -- start 2024-09-10T08:22:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.361+0000 7f7e9dc85740 1 -- start start 2024-09-10T08:22:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.361+0000 7f7e9dc85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d8e310 0x560d60d95360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.361+0000 7f7e9dc85740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 0x560d60d91960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.361+0000 7f7e9dc85740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d994f0 0x560d60d96cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.361+0000 7f7e9dc85740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941b3930 con 0x560d60d8ec30 2024-09-10T08:22:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.361+0000 7f7e9dc85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941b1200 con 0x560d60d8e310 2024-09-10T08:22:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.361+0000 7f7e9dc85740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941af050 con 0x560d60d994f0 2024-09-10T08:22:09.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.361+0000 7f7e93400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 0x560d60d91960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.362+0000 7f7e93400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 0x560d60d91960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57774/0 (socket says 172.21.15.2:57774) 2024-09-10T08:22:09.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.362+0000 7f7e93400640 1 -- 172.21.15.2:0/285708239 learned_addr learned my addr 172.21.15.2:0/285708239 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:09.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.362+0000 7f7e99200640 1 --2- 172.21.15.2:0/285708239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d8e310 0x560d60d95360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.362+0000 7f7e99200640 1 --2- 172.21.15.2:0/285708239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d994f0 0x560d60d96cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.362+0000 7f7e93400640 1 -- 172.21.15.2:0/285708239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d994f0 msgr2=0x560d60d96cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:09.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.362+0000 7f7e93400640 1 --2- 172.21.15.2:0/285708239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d994f0 0x560d60d96cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.362+0000 7f7e93400640 1 -- 172.21.15.2:0/285708239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d8e310 msgr2=0x560d60d95360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:09.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.362+0000 7f7e93400640 1 --2- 172.21.15.2:0/285708239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d8e310 0x560d60d95360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.362+0000 7f7e93400640 1 -- 172.21.15.2:0/285708239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d60d98cf0 con 0x560d60d8ec30 2024-09-10T08:22:09.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.362+0000 7f7e99200640 1 --2- 172.21.15.2:0/285708239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d994f0 0x560d60d96cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:22:09.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.362+0000 7f7e93400640 1 --2- 172.21.15.2:0/285708239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 0x560d60d91960 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f7e80003f50 tx=0x7f7e80009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92289583046c82d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:09.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.362+0000 7f7e88c00640 1 -- 172.21.15.2:0/285708239 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e800050e0 con 0x560d60d8ec30 2024-09-10T08:22:09.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.363+0000 7f7e88c00640 1 -- 172.21.15.2:0/285708239 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8000f920 con 0x560d60d8ec30 2024-09-10T08:22:09.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.363+0000 7f7e88c00640 1 -- 172.21.15.2:0/285708239 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e800036e0 con 0x560d60d8ec30 2024-09-10T08:22:09.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.363+0000 7f7e9dc85740 1 -- 172.21.15.2:0/285708239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 msgr2=0x560d60d91960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:09.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.363+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/285708239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 0x560d60d91960 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f7e80003f50 tx=0x7f7e80009fd0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.363+0000 7f7e9dc85740 1 -- 172.21.15.2:0/285708239 shutdown_connections 2024-09-10T08:22:09.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.363+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/285708239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d994f0 0x560d60d96cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.363+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/285708239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d8ec30 0x560d60d91960 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.363+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/285708239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d8e310 0x560d60d95360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.363+0000 7f7e9dc85740 1 -- 172.21.15.2:0/285708239 >> 172.21.15.2:0/285708239 conn(0x560d60d8a8a0 msgr2=0x560d60d89aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:09.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.363+0000 7f7e9dc85740 1 -- 172.21.15.2:0/285708239 shutdown_connections 2024-09-10T08:22:09.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.363+0000 7f7e9dc85740 1 -- 172.21.15.2:0/285708239 wait complete. 2024-09-10T08:22:09.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.364+0000 7f7e9dc85740 1 -- start start 2024-09-10T08:22:09.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.364+0000 7f7e9dc85740 1 Processor -- start 2024-09-10T08:22:09.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.364+0000 7f7e9dc85740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8e310 0x560d60e0c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.364+0000 7f7e9dc85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d8ec30 0x560d60e0c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.364+0000 7f7e9dc85740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d994f0 0x560d60d9ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.364+0000 7f7e9dc85740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941af6c0 con 0x560d60d994f0 2024-09-10T08:22:09.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.364+0000 7f7e9dc85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941b0b30 con 0x560d60d8ec30 2024-09-10T08:22:09.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.364+0000 7f7e9dc85740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941afd90 con 0x560d60d8e310 2024-09-10T08:22:09.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.364+0000 7f7e9dc85740 10 client.0 _ll_register_callbacks cb 0x7f7e800097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:09.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.365+0000 7f7e93400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d8ec30 0x560d60e0c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.365+0000 7f7e93400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d8ec30 0x560d60e0c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42922/0 (socket says 172.21.15.2:42922) 2024-09-10T08:22:09.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.365+0000 7f7e93400640 1 -- 172.21.15.2:0/2514282416 learned_addr learned my addr 172.21.15.2:0/2514282416 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:09.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.365+0000 7f7e99200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8e310 0x560d60e0c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.365+0000 7f7e99200640 1 --2- 172.21.15.2:0/2514282416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d994f0 0x560d60d9ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.365+0000 7f7e93400640 1 -- 172.21.15.2:0/2514282416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8e310 msgr2=0x560d60e0c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:09.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.365+0000 7f7e93400640 1 --2- 172.21.15.2:0/2514282416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8e310 0x560d60e0c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.365+0000 7f7e93400640 1 -- 172.21.15.2:0/2514282416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d994f0 msgr2=0x560d60d9ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:09.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.365+0000 7f7e93400640 1 --2- 172.21.15.2:0/2514282416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d994f0 0x560d60d9ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.365+0000 7f7e93400640 1 -- 172.21.15.2:0/2514282416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d60d9d130 con 0x560d60d8ec30 2024-09-10T08:22:09.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.365+0000 7f7e99200640 1 --2- 172.21.15.2:0/2514282416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d994f0 0x560d60d9ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:22:09.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.365+0000 7f7e93400640 1 --2- 172.21.15.2:0/2514282416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d8ec30 0x560d60e0c730 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7e80005e00 tx=0x7f7e8000fd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa88d58d5108572 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:09.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.365+0000 7f7e7e000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:22:09.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.366+0000 7f7e7e000640 1 -- 172.21.15.2:0/2514282416 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e800035f0 con 0x560d60d8ec30 2024-09-10T08:22:09.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.366+0000 7f7e7e000640 1 -- 172.21.15.2:0/2514282416 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7e8002e040 con 0x560d60d8ec30 2024-09-10T08:22:09.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.366+0000 7f7e9dc85740 1 -- 172.21.15.2:0/2514282416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e8000f920 con 0x560d60d8ec30 2024-09-10T08:22:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.366+0000 7f7e7e000640 1 -- 172.21.15.2:0/2514282416 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e8003d050 con 0x560d60d8ec30 2024-09-10T08:22:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.366+0000 7f7e9dc85740 10 client.29504 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.366+0000 7f7e9dc85740 1 -- 172.21.15.2:0/2514282416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d60e13a00 con 0x560d60d8ec30 2024-09-10T08:22:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.367+0000 7f7e7e000640 1 -- 172.21.15.2:0/2514282416 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8000f920 con 0x560d60d8ec30 2024-09-10T08:22:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.367+0000 7f7e7e000640 1 -- 172.21.15.2:0/2514282416 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7e80010aa0 con 0x560d60d8ec30 2024-09-10T08:22:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.367+0000 7f7e9dc85740 10 client.29504 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.367+0000 7f7e9dc85740 10 client.29504 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:22:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.367+0000 7f7e9dc85740 20 client.29504 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.367+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/2514282416 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d60e14030 0x560d60e16450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.367+0000 7f7e9dc85740 4 client.29504 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "73163cc9", "release": true, "await": true}] 2024-09-10T08:22:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.367+0000 7f7e9dc85740 1 -- 172.21.15.2:0/2514282416 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "73163cc9", "release": true, "await": true}) -- 0x560d60cfc0d0 con 0x560d60e14030 2024-09-10T08:22:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.367+0000 7f7e99200640 1 --2- 172.21.15.2:0/2514282416 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d60e14030 0x560d60e16450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.368+0000 7f7e99200640 1 --2- 172.21.15.2:0/2514282416 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d60e14030 0x560d60e16450 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363dfcd1664dd63 server_cookie=e010d5b2623b936c in_seq=0 out_seq=0 2024-09-10T08:22:09.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.368+0000 7f7e7e000640 10 client.29504 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.370+0000 7f7e7e000640 1 -- 172.21.15.2:0/2514282416 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560d60cfc0d0 con 0x560d60e14030 2024-09-10T08:22:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.370+0000 7f7e7e000640 10 client.29504 handle_command_reply: tid=0 2024-09-10T08:22:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.370+0000 7f7e9dc85740 1 client.29504 shutdown 2024-09-10T08:22:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.370+0000 7f7e9dc85740 1 -- 172.21.15.2:0/2514282416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d8ec30 msgr2=0x560d60e0c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.370+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/2514282416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d8ec30 0x560d60e0c730 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7e80005e00 tx=0x7f7e8000fd00 comp rx=0 tx=0).stop 2024-09-10T08:22:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.370+0000 7f7e9dc85740 1 -- 172.21.15.2:0/2514282416 shutdown_connections 2024-09-10T08:22:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.370+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/2514282416 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d60e14030 0x560d60e16450 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.370+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/2514282416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d60d994f0 0x560d60d9ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.370+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/2514282416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d60d8ec30 0x560d60e0c730 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.370+0000 7f7e9dc85740 1 --2- 172.21.15.2:0/2514282416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d60d8e310 0x560d60e0c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.371+0000 7f7e7e000640 0 client.29504 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:09.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.371+0000 7f7e9dc85740 1 -- 172.21.15.2:0/2514282416 >> 172.21.15.2:0/2514282416 conn(0x560d60d8a8a0 msgr2=0x560d60d8fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:09.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.371+0000 7f7e9dc85740 1 -- 172.21.15.2:0/2514282416 shutdown_connections 2024-09-10T08:22:09.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.371+0000 7f7e9dc85740 1 -- 172.21.15.2:0/2514282416 wait complete. 2024-09-10T08:22:09.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.371+0000 7f7e9dc85740 20 client.29504 trim_cache size 0 max 16384 2024-09-10T08:22:09.373 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:22:09.373 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:22:09.373 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:22:09.373 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 52, 2024-09-10T08:22:09.373 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:22:09.373 INFO:teuthology.orchestra.run.smithi002.stdout: "73163cc9": { 2024-09-10T08:22:09.374 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 52, 2024-09-10T08:22:09.374 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:22:09.374 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:22:09.374 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:22:09.374 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:22:09.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:22:09.374 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:22:09.374 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 141.5, 2024-09-10T08:22:09.374 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:22:09.374 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:22:09.374 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:22:09.374 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:22:09.375 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:22:09.375 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:22:09.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:22:09.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:22:09.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:22:09.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:22:09.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:22:09.375 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:22:09.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.372+0000 7f7e9b200640 1 -- 172.21.15.2:0/2095371136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7e780a2b80 msgr2=0x7f7e780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:09.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.372+0000 7f7e9b200640 1 --2- 172.21.15.2:0/2095371136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7e780a2b80 0x7f7e780a5040 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7e840bbf80 tx=0x7f7e840f5510 comp rx=0 tx=0).stop 2024-09-10T08:22:09.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.372+0000 7f7e9b200640 1 -- 172.21.15.2:0/2095371136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e94101de0 msgr2=0x7f7e9419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:09.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.372+0000 7f7e9b200640 1 --2- 172.21.15.2:0/2095371136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e94101de0 0x7f7e9419a630 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7e84004770 tx=0x7f7e84003c10 comp rx=0 tx=0).stop 2024-09-10T08:22:09.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.372+0000 7f7e9b200640 1 -- 172.21.15.2:0/2095371136 shutdown_connections 2024-09-10T08:22:09.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.372+0000 7f7e9b200640 1 --2- 172.21.15.2:0/2095371136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e94106f10 0x7f7e9419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.372+0000 7f7e9b200640 1 --2- 172.21.15.2:0/2095371136 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7e780a2b80 0x7f7e780a5040 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.372+0000 7f7e9b200640 1 --2- 172.21.15.2:0/2095371136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e94102790 0x7f7e9419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.372+0000 7f7e9b200640 1 --2- 172.21.15.2:0/2095371136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e94101de0 0x7f7e9419a630 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:09.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.372+0000 7f7e9b200640 1 -- 172.21.15.2:0/2095371136 >> 172.21.15.2:0/2095371136 conn(0x7f7e940fd8c0 msgr2=0x7f7e940feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:09.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.372+0000 7f7e9b200640 1 -- 172.21.15.2:0/2095371136 shutdown_connections 2024-09-10T08:22:09.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:09.372+0000 7f7e9b200640 1 -- 172.21.15.2:0/2095371136 wait complete. 2024-09-10T08:22:09.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:09 smithi155 ceph-mon[35405]: pgmap v807: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-10T08:22:09.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:09 smithi107 ceph-mon[35048]: pgmap v807: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-10T08:22:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:09 smithi002 ceph-mon[30563]: pgmap v807: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-10T08:22:10.150 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '73163cc9', total seconds elapsed: 25.4 2024-09-10T08:22:11.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:11 smithi155 ceph-mon[35405]: pgmap v808: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-10T08:22:11.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:11 smithi107 ceph-mon[35048]: pgmap v808: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-10T08:22:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:11 smithi002 ceph-mon[30563]: pgmap v808: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-10T08:22:13.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:13 smithi155 ceph-mon[35405]: pgmap v809: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-10T08:22:13.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:13 smithi107 ceph-mon[35048]: pgmap v809: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-10T08:22:13.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:13 smithi002 ceph-mon[30563]: pgmap v809: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-10T08:22:15.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:15 smithi155 ceph-mon[35405]: pgmap v810: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-09-10T08:22:15.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:15 smithi107 ceph-mon[35048]: pgmap v810: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-09-10T08:22:15.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:15 smithi002 ceph-mon[30563]: pgmap v810: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-09-10T08:22:17.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:17 smithi155 ceph-mon[35405]: pgmap v811: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-10T08:22:17.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:17 smithi107 ceph-mon[35048]: pgmap v811: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-10T08:22:17.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:17 smithi002 ceph-mon[30563]: pgmap v811: 129 pgs: 129 active+clean; 3.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-10T08:22:19.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:19 smithi155 ceph-mon[35405]: pgmap v812: 129 pgs: 129 active+clean; 3.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-09-10T08:22:19.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:19 smithi107 ceph-mon[35048]: pgmap v812: 129 pgs: 129 active+clean; 3.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-09-10T08:22:19.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:19 smithi002 ceph-mon[30563]: pgmap v812: 129 pgs: 129 active+clean; 3.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-09-10T08:22:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.083+0000 7fb01be00640 1 -- 172.21.15.2:0/3711079360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb01c105720 msgr2=0x7fb01c10bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.083+0000 7fb01be00640 1 --2- 172.21.15.2:0/3711079360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb01c105720 0x7fb01c10bfb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb014009be0 tx=0x7fb01402a6c0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.083+0000 7fb01be00640 1 -- 172.21.15.2:0/3711079360 shutdown_connections 2024-09-10T08:22:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.083+0000 7fb01be00640 1 --2- 172.21.15.2:0/3711079360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb01c105720 0x7fb01c10bfb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.083+0000 7fb01be00640 1 --2- 172.21.15.2:0/3711079360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb01c104d60 0x7fb01c1051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.083+0000 7fb01be00640 1 --2- 172.21.15.2:0/3711079360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb01c103b70 0x7fb01c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.083+0000 7fb01be00640 1 -- 172.21.15.2:0/3711079360 >> 172.21.15.2:0/3711079360 conn(0x7fb01c0ff2f0 msgr2=0x7fb01c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.084+0000 7fb01be00640 1 -- 172.21.15.2:0/3711079360 shutdown_connections 2024-09-10T08:22:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.084+0000 7fb01be00640 1 -- 172.21.15.2:0/3711079360 wait complete. 2024-09-10T08:22:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.085+0000 7fb01be00640 1 Processor -- start 2024-09-10T08:22:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.085+0000 7fb01be00640 1 -- start start 2024-09-10T08:22:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.085+0000 7fb01be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb01c103b70 0x7fb01c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.085+0000 7fb01be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb01c104d60 0x7fb01c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.085+0000 7fb01be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb01c105720 0x7fb01c1a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.085+0000 7fb01be00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c10f7a0 con 0x7fb01c103b70 2024-09-10T08:22:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.085+0000 7fb01be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c10f620 con 0x7fb01c105720 2024-09-10T08:22:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.085+0000 7fb01be00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c10f920 con 0x7fb01c104d60 2024-09-10T08:22:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.085+0000 7fb01aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb01c103b70 0x7fb01c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.085+0000 7fb01aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb01c103b70 0x7fb01c199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56610/0 (socket says 172.21.15.2:56610) 2024-09-10T08:22:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.085+0000 7fb01aa00640 1 -- 172.21.15.2:0/4183479571 learned_addr learned my addr 172.21.15.2:0/4183479571 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.085+0000 7fb01b400640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb01c105720 0x7fb01c1a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.085+0000 7fb01a000640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb01c104d60 0x7fb01c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.086+0000 7fb01aa00640 1 -- 172.21.15.2:0/4183479571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb01c104d60 msgr2=0x7fb01c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.086+0000 7fb01aa00640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb01c104d60 0x7fb01c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.086+0000 7fb01aa00640 1 -- 172.21.15.2:0/4183479571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb01c105720 msgr2=0x7fb01c1a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.086+0000 7fb01aa00640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb01c105720 0x7fb01c1a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.086+0000 7fb01aa00640 1 -- 172.21.15.2:0/4183479571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c1a19f0 con 0x7fb01c103b70 2024-09-10T08:22:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.086+0000 7fb01aa00640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb01c103b70 0x7fb01c199dd0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00e9c0 tx=0x7fb00c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.086+0000 7fb013400640 1 -- 172.21.15.2:0/4183479571 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00cd80 con 0x7fb01c103b70 2024-09-10T08:22:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.086+0000 7fb01be00640 1 -- 172.21.15.2:0/4183479571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01c1a1ce0 con 0x7fb01c103b70 2024-09-10T08:22:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.086+0000 7fb013400640 1 -- 172.21.15.2:0/4183479571 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb00c0035f0 con 0x7fb01c103b70 2024-09-10T08:22:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.086+0000 7fb013400640 1 -- 172.21.15.2:0/4183479571 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c010630 con 0x7fb01c103b70 2024-09-10T08:22:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.087+0000 7fb01be00640 1 -- 172.21.15.2:0/4183479571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01c1a2180 con 0x7fb01c103b70 2024-09-10T08:22:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.090+0000 7fb02487d740 1 Processor -- start 2024-09-10T08:22:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.090+0000 7fb013400640 1 -- 172.21.15.2:0/4183479571 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb00c0037a0 con 0x7fb01c103b70 2024-09-10T08:22:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.090+0000 7fb02487d740 1 -- start start 2024-09-10T08:22:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.090+0000 7fb02487d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c0260 0x5575ff7c0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.090+0000 7fb02487d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0ba0 0x5575ff7c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.090+0000 7fb02487d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c5c30 0x5575ff7c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.090+0000 7fb02487d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5575ff725100 con 0x5575ff7c0ba0 2024-09-10T08:22:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.090+0000 7fb02487d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5575ff6cd1b0 con 0x5575ff7c5c30 2024-09-10T08:22:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.090+0000 7fb02487d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5575ff7210c0 con 0x5575ff7c0260 2024-09-10T08:22:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.091+0000 7fb01b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c0260 0x5575ff7c0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.091+0000 7fb01aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0ba0 0x5575ff7c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.091+0000 7fb01b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c0260 0x5575ff7c0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:33406/0 (socket says 172.21.15.2:33406) 2024-09-10T08:22:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.091+0000 7fb01b400640 1 -- 172.21.15.2:0/2809010651 learned_addr learned my addr 172.21.15.2:0/2809010651 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.091+0000 7fb01a000640 1 --2- 172.21.15.2:0/2809010651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c5c30 0x5575ff7c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.091+0000 7fb01b400640 1 -- 172.21.15.2:0/2809010651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c5c30 msgr2=0x5575ff7c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.091+0000 7fb01b400640 1 --2- 172.21.15.2:0/2809010651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c5c30 0x5575ff7c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.091+0000 7fb01b400640 1 -- 172.21.15.2:0/2809010651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0ba0 msgr2=0x5575ff7c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.091+0000 7fb01b400640 1 --2- 172.21.15.2:0/2809010651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0ba0 0x5575ff7c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.091+0000 7fb01b400640 1 -- 172.21.15.2:0/2809010651 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575ff438210 con 0x5575ff7c0260 2024-09-10T08:22:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.091+0000 7fb01b400640 1 --2- 172.21.15.2:0/2809010651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c0260 0x5575ff7c0660 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb014009be0 tx=0x7fb014005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=478408cc3cac3eec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.092+0000 7fb013400640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb0000a2b80 0x7fb0000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.092+0000 7fb007e00640 1 -- 172.21.15.2:0/2809010651 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014042030 con 0x5575ff7c0260 2024-09-10T08:22:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.092+0000 7fb007e00640 1 -- 172.21.15.2:0/2809010651 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb014037040 con 0x5575ff7c0260 2024-09-10T08:22:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.092+0000 7fb007e00640 1 -- 172.21.15.2:0/2809010651 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014036070 con 0x5575ff7c0260 2024-09-10T08:22:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.092+0000 7fb013400640 1 -- 172.21.15.2:0/4183479571 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb00c0cf910 con 0x7fb01c103b70 2024-09-10T08:22:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.092+0000 7fb01aa00640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb0000a2b80 0x7fb0000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.093+0000 7fb02487d740 1 -- 172.21.15.2:0/2809010651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c0260 msgr2=0x5575ff7c0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.093+0000 7fb02487d740 1 --2- 172.21.15.2:0/2809010651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c0260 0x5575ff7c0660 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb014009be0 tx=0x7fb014005ec0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.093+0000 7fb01aa00640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb0000a2b80 0x7fb0000a5040 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0a4460 tx=0x7fb00c0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.093+0000 7fb02487d740 1 -- 172.21.15.2:0/2809010651 shutdown_connections 2024-09-10T08:22:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.093+0000 7fb02487d740 1 --2- 172.21.15.2:0/2809010651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c5c30 0x5575ff7c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.093+0000 7fb02487d740 1 --2- 172.21.15.2:0/2809010651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0ba0 0x5575ff7c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.093+0000 7fb02487d740 1 --2- 172.21.15.2:0/2809010651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c0260 0x5575ff7c0660 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.093+0000 7fb02487d740 1 -- 172.21.15.2:0/2809010651 >> 172.21.15.2:0/2809010651 conn(0x5575ff7bb8d0 msgr2=0x5575ff7bbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.093+0000 7fb02487d740 1 -- 172.21.15.2:0/2809010651 shutdown_connections 2024-09-10T08:22:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.093+0000 7fb02487d740 1 -- 172.21.15.2:0/2809010651 wait complete. 2024-09-10T08:22:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.093+0000 7fb02487d740 1 -- start start 2024-09-10T08:22:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.093+0000 7fb02487d740 1 Processor -- start 2024-09-10T08:22:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.094+0000 7fb02487d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0260 0x5575ff7ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.094+0000 7fb02487d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c0ba0 0x5575ff7ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.094+0000 7fb02487d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c5c30 0x5575ff7d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.094+0000 7fb02487d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5575ff7c8ba0 con 0x5575ff7c5c30 2024-09-10T08:22:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.094+0000 7fb02487d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5575ff6ae090 con 0x5575ff7c0260 2024-09-10T08:22:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.094+0000 7fb02487d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5575ff7c8d20 con 0x5575ff7c0ba0 2024-09-10T08:22:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.094+0000 7fb02487d740 10 client.0 _ll_register_callbacks cb 0x5575ff702f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.094+0000 7fb01b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0260 0x5575ff7ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.094+0000 7fb01b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0260 0x5575ff7ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:34402/0 (socket says 172.21.15.2:34402) 2024-09-10T08:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.094+0000 7fb01b400640 1 -- 172.21.15.2:0/2432006169 learned_addr learned my addr 172.21.15.2:0/2432006169 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.094+0000 7fb01b400640 1 --2- 172.21.15.2:0/2432006169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c5c30 0x5575ff7d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.094+0000 7fb01a000640 1 --2- 172.21.15.2:0/2432006169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c0ba0 0x5575ff7ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.095+0000 7fb01b400640 1 -- 172.21.15.2:0/2432006169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c0ba0 msgr2=0x5575ff7ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.095+0000 7fb01b400640 1 --2- 172.21.15.2:0/2432006169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c0ba0 0x5575ff7ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.095+0000 7fb01b400640 1 -- 172.21.15.2:0/2432006169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c5c30 msgr2=0x5575ff7d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.095+0000 7fb01b400640 1 --2- 172.21.15.2:0/2432006169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c5c30 0x5575ff7d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.095+0000 7fb01b400640 1 -- 172.21.15.2:0/2432006169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575ff504620 con 0x5575ff7c0260 2024-09-10T08:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.095+0000 7fb01b400640 1 --2- 172.21.15.2:0/2432006169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0260 0x5575ff7ca380 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb01402abe0 tx=0x7fb014030fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b54ab4e02229afe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.095+0000 7fb004c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.095+0000 7fb004c00640 1 -- 172.21.15.2:0/2432006169 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01404b020 con 0x5575ff7c0260 2024-09-10T08:22:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.095+0000 7fb02487d740 1 -- 172.21.15.2:0/2432006169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb014037040 con 0x5575ff7c0260 2024-09-10T08:22:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.095+0000 7fb004c00640 1 -- 172.21.15.2:0/2432006169 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb0140305a0 con 0x5575ff7c0260 2024-09-10T08:22:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.095+0000 7fb004c00640 1 -- 172.21.15.2:0/2432006169 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb014059050 con 0x5575ff7c0260 2024-09-10T08:22:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.095+0000 7fb02487d740 10 client.29516 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.095+0000 7fb02487d740 1 -- 172.21.15.2:0/2432006169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5575ff7d46f0 con 0x5575ff7c0260 2024-09-10T08:22:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.096+0000 7fb004c00640 1 -- 172.21.15.2:0/2432006169 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb014037040 con 0x5575ff7c0260 2024-09-10T08:22:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.096+0000 7fb004c00640 1 -- 172.21.15.2:0/2432006169 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb01403c040 con 0x5575ff7c0260 2024-09-10T08:22:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.096+0000 7fb02487d740 10 client.29516 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.096+0000 7fb02487d740 10 client.29516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.096+0000 7fb02487d740 20 client.29516 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.096+0000 7fb02487d740 1 --2- 172.21.15.2:0/2432006169 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575ff7d5a40 0x5575ff7d7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.096+0000 7fb02487d740 4 client.29516 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:22:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.096+0000 7fb02487d740 1 -- 172.21.15.2:0/2432006169 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575ff732830 con 0x5575ff7d5a40 2024-09-10T08:22:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.097+0000 7fb01a000640 1 --2- 172.21.15.2:0/2432006169 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575ff7d5a40 0x5575ff7d7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.097+0000 7fb01a000640 1 --2- 172.21.15.2:0/2432006169 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575ff7d5a40 0x5575ff7d7e30 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c2f8a2279a8f0 server_cookie=aed22ce195bb1e32 in_seq=0 out_seq=0 2024-09-10T08:22:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.097+0000 7fb004c00640 10 client.29516 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.098+0000 7fb004c00640 1 -- 172.21.15.2:0/2432006169 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575ff732830 con 0x5575ff7d5a40 2024-09-10T08:22:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.098+0000 7fb004c00640 10 client.29516 handle_command_reply: tid=0 2024-09-10T08:22:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.098+0000 7fb02487d740 1 client.29516 shutdown 2024-09-10T08:22:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.098+0000 7fb02487d740 1 -- 172.21.15.2:0/2432006169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0260 msgr2=0x5575ff7ca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.098+0000 7fb02487d740 1 --2- 172.21.15.2:0/2432006169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0260 0x5575ff7ca380 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb01402abe0 tx=0x7fb014030fd0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.099+0000 7fb02487d740 1 -- 172.21.15.2:0/2432006169 shutdown_connections 2024-09-10T08:22:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.099+0000 7fb02487d740 1 --2- 172.21.15.2:0/2432006169 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575ff7d5a40 0x5575ff7d7e30 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.099+0000 7fb02487d740 1 --2- 172.21.15.2:0/2432006169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c5c30 0x5575ff7d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.099+0000 7fb02487d740 1 --2- 172.21.15.2:0/2432006169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c0ba0 0x5575ff7ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.099+0000 7fb004c00640 0 client.29516 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.099+0000 7fb02487d740 1 --2- 172.21.15.2:0/2432006169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0260 0x5575ff7ca380 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.099+0000 7fb02487d740 1 -- 172.21.15.2:0/2432006169 >> 172.21.15.2:0/2432006169 conn(0x5575ff7bb8d0 msgr2=0x5575ff7bd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.099+0000 7fb02487d740 1 -- 172.21.15.2:0/2432006169 shutdown_connections 2024-09-10T08:22:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.099+0000 7fb02487d740 1 -- 172.21.15.2:0/2432006169 wait complete. 2024-09-10T08:22:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.099+0000 7fb02487d740 20 client.29516 trim_cache size 0 max 16384 2024-09-10T08:22:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.114+0000 7fb02487d740 1 Processor -- start 2024-09-10T08:22:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.114+0000 7fb02487d740 1 -- start start 2024-09-10T08:22:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.114+0000 7fb02487d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0260 0x5575ff7ca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.114+0000 7fb02487d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0ba0 0x5575ff7c0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.114+0000 7fb02487d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c5c30 0x5575ff7d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.114+0000 7fb02487d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1af0e0 con 0x5575ff7c0260 2024-09-10T08:22:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.114+0000 7fb02487d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1ac9b0 con 0x5575ff7c0ba0 2024-09-10T08:22:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.114+0000 7fb02487d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1aa800 con 0x5575ff7c5c30 2024-09-10T08:22:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.115+0000 7fb01b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0260 0x5575ff7ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.115+0000 7fb01b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0260 0x5575ff7ca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56642/0 (socket says 172.21.15.2:56642) 2024-09-10T08:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.115+0000 7fb01b400640 1 -- 172.21.15.2:0/3509581624 learned_addr learned my addr 172.21.15.2:0/3509581624 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.115+0000 7fb01a000640 1 --2- 172.21.15.2:0/3509581624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0ba0 0x5575ff7c0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.115+0000 7fb01b400640 1 --2- 172.21.15.2:0/3509581624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c5c30 0x5575ff7d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.115+0000 7fb01b400640 1 -- 172.21.15.2:0/3509581624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c5c30 msgr2=0x5575ff7d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.115+0000 7fb01b400640 1 --2- 172.21.15.2:0/3509581624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c5c30 0x5575ff7d1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.115+0000 7fb01b400640 1 -- 172.21.15.2:0/3509581624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0ba0 msgr2=0x5575ff7c0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.115+0000 7fb01b400640 1 --2- 172.21.15.2:0/3509581624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0ba0 0x5575ff7c0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.115+0000 7fb01b400640 1 -- 172.21.15.2:0/3509581624 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575ff7d84b0 con 0x5575ff7c0260 2024-09-10T08:22:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.115+0000 7fb01b400640 1 --2- 172.21.15.2:0/3509581624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0260 0x5575ff7ca950 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb014031440 tx=0x7fb014004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157187104aeed3eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.116+0000 7fb007400640 1 -- 172.21.15.2:0/3509581624 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01404b070 con 0x5575ff7c0260 2024-09-10T08:22:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.116+0000 7fb007400640 1 -- 172.21.15.2:0/3509581624 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb014004d40 con 0x5575ff7c0260 2024-09-10T08:22:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.116+0000 7fb007400640 1 -- 172.21.15.2:0/3509581624 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014036050 con 0x5575ff7c0260 2024-09-10T08:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.116+0000 7fb02487d740 1 -- 172.21.15.2:0/3509581624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0260 msgr2=0x5575ff7ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.116+0000 7fb02487d740 1 --2- 172.21.15.2:0/3509581624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0260 0x5575ff7ca950 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb014031440 tx=0x7fb014004a60 comp rx=0 tx=0).stop 2024-09-10T08:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.116+0000 7fb02487d740 1 -- 172.21.15.2:0/3509581624 shutdown_connections 2024-09-10T08:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.116+0000 7fb02487d740 1 --2- 172.21.15.2:0/3509581624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c5c30 0x5575ff7d1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.116+0000 7fb02487d740 1 --2- 172.21.15.2:0/3509581624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0ba0 0x5575ff7c0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.116+0000 7fb02487d740 1 --2- 172.21.15.2:0/3509581624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0260 0x5575ff7ca950 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.116+0000 7fb02487d740 1 -- 172.21.15.2:0/3509581624 >> 172.21.15.2:0/3509581624 conn(0x5575ff7bb8d0 msgr2=0x5575ff7ba1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.116+0000 7fb02487d740 1 -- 172.21.15.2:0/3509581624 shutdown_connections 2024-09-10T08:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.116+0000 7fb02487d740 1 -- 172.21.15.2:0/3509581624 wait complete. 2024-09-10T08:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.117+0000 7fb02487d740 1 -- start start 2024-09-10T08:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.117+0000 7fb02487d740 1 Processor -- start 2024-09-10T08:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.117+0000 7fb02487d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0260 0x5575ff7ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.117+0000 7fb02487d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0ba0 0x5575ff7ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.117+0000 7fb02487d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c5c30 0x5575ff83e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.117+0000 7fb02487d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1aae70 con 0x5575ff7c0ba0 2024-09-10T08:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.117+0000 7fb02487d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1ac2e0 con 0x5575ff7c0260 2024-09-10T08:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.117+0000 7fb02487d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1ab540 con 0x5575ff7c5c30 2024-09-10T08:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.117+0000 7fb02487d740 10 client.0 _ll_register_callbacks cb 0x7fb00c0d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.118+0000 7fb01a000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0ba0 0x5575ff7ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.118+0000 7fb01a000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0ba0 0x5575ff7ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56646/0 (socket says 172.21.15.2:56646) 2024-09-10T08:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.118+0000 7fb01a000640 1 -- 172.21.15.2:0/3934733552 learned_addr learned my addr 172.21.15.2:0/3934733552 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.118+0000 7fb01b400640 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0260 0x5575ff7ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.118+0000 7fb01b400640 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c5c30 0x5575ff83e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.118+0000 7fb01a000640 1 -- 172.21.15.2:0/3934733552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c5c30 msgr2=0x5575ff83e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.118+0000 7fb01a000640 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c5c30 0x5575ff83e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.118+0000 7fb01a000640 1 -- 172.21.15.2:0/3934733552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0260 msgr2=0x5575ff7ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.118+0000 7fb01a000640 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0260 0x5575ff7ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.118+0000 7fb01a000640 1 -- 172.21.15.2:0/3934733552 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575ff83ec40 con 0x5575ff7c0ba0 2024-09-10T08:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.118+0000 7fb01b400640 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c5c30 0x5575ff83e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.118+0000 7fb01a000640 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0ba0 0x5575ff7ceba0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb01c19b2e0 tx=0x7fb008010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a0367e5436460d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.118+0000 7fb004c00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb004c00640 1 -- 172.21.15.2:0/3934733552 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008005220 con 0x5575ff7c0ba0 2024-09-10T08:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb004c00640 1 -- 172.21.15.2:0/3934733552 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb0080053c0 con 0x5575ff7c0ba0 2024-09-10T08:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb02487d740 1 -- 172.21.15.2:0/3934733552 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb014004d40 con 0x5575ff7c0ba0 2024-09-10T08:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb004c00640 1 -- 172.21.15.2:0/3934733552 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb0080055c0 con 0x5575ff7c0ba0 2024-09-10T08:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb02487d740 10 client.19497 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb02487d740 1 -- 172.21.15.2:0/3934733552 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5575ff83fe90 con 0x5575ff7c0ba0 2024-09-10T08:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb004c00640 1 -- 172.21.15.2:0/3934733552 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb014004d40 con 0x5575ff7c0ba0 2024-09-10T08:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb004c00640 1 -- 172.21.15.2:0/3934733552 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb008025070 con 0x5575ff7c0ba0 2024-09-10T08:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb02487d740 10 client.19497 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb02487d740 10 client.19497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb02487d740 20 client.19497 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb02487d740 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575ff840870 0x5575ff842c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb02487d740 4 client.19497 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.119+0000 7fb02487d740 1 -- 172.21.15.2:0/3934733552 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575ff72cb80 con 0x5575ff840870 2024-09-10T08:22:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.120+0000 7fb01b400640 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575ff840870 0x5575ff842c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.120+0000 7fb01b400640 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575ff840870 0x5575ff842c90 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b274160efcff3bb server_cookie=d2793ebee22b4898 in_seq=0 out_seq=0 2024-09-10T08:22:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.120+0000 7fb004c00640 10 client.19497 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb004c00640 1 -- 172.21.15.2:0/3934733552 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5575ff72cb80 con 0x5575ff840870 2024-09-10T08:22:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb004c00640 10 client.19497 handle_command_reply: tid=0 2024-09-10T08:22:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb02487d740 1 client.19497 shutdown 2024-09-10T08:22:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb02487d740 1 -- 172.21.15.2:0/3934733552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0ba0 msgr2=0x5575ff7ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb02487d740 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0ba0 0x5575ff7ceba0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb01c19b2e0 tx=0x7fb008010800 comp rx=0 tx=0).stop 2024-09-10T08:22:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb02487d740 1 -- 172.21.15.2:0/3934733552 shutdown_connections 2024-09-10T08:22:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb02487d740 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575ff840870 0x5575ff842c90 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb02487d740 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575ff7c5c30 0x5575ff83e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb02487d740 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575ff7c0ba0 0x5575ff7ceba0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb004c00640 0 client.19497 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb02487d740 1 --2- 172.21.15.2:0/3934733552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575ff7c0260 0x5575ff7ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb02487d740 1 -- 172.21.15.2:0/3934733552 >> 172.21.15.2:0/3934733552 conn(0x5575ff7bb8d0 msgr2=0x5575ff7bc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb02487d740 1 -- 172.21.15.2:0/3934733552 shutdown_connections 2024-09-10T08:22:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.132+0000 7fb02487d740 1 -- 172.21.15.2:0/3934733552 wait complete. 2024-09-10T08:22:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.133+0000 7fb02487d740 20 client.19497 trim_cache size 0 max 16384 2024-09-10T08:22:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:22:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:22:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:22:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:22:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.133+0000 7fb01be00640 1 -- 172.21.15.2:0/4183479571 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb0000a2b80 msgr2=0x7fb0000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.133+0000 7fb01be00640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb0000a2b80 0x7fb0000a5040 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0a4460 tx=0x7fb00c0d5530 comp rx=0 tx=0).stop 2024-09-10T08:22:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.133+0000 7fb01be00640 1 -- 172.21.15.2:0/4183479571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb01c103b70 msgr2=0x7fb01c199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.133+0000 7fb01be00640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb01c103b70 0x7fb01c199dd0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00e9c0 tx=0x7fb00c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:22:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.133+0000 7fb01be00640 1 -- 172.21.15.2:0/4183479571 shutdown_connections 2024-09-10T08:22:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.133+0000 7fb01be00640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb01c105720 0x7fb01c1a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.133+0000 7fb01be00640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb0000a2b80 0x7fb0000a5040 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.133+0000 7fb01be00640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb01c104d60 0x7fb01c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.133+0000 7fb01be00640 1 --2- 172.21.15.2:0/4183479571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb01c103b70 0x7fb01c199dd0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.133+0000 7fb01be00640 1 -- 172.21.15.2:0/4183479571 >> 172.21.15.2:0/4183479571 conn(0x7fb01c0ff2f0 msgr2=0x7fb01c1080e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.134+0000 7fb01be00640 1 -- 172.21.15.2:0/4183479571 shutdown_connections 2024-09-10T08:22:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:20.134+0000 7fb01be00640 1 -- 172.21.15.2:0/4183479571 wait complete. 2024-09-10T08:22:21.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:21 smithi155 ceph-mon[35405]: pgmap v813: 129 pgs: 129 active+clean; 3.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-09-10T08:22:21.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:21 smithi107 ceph-mon[35048]: pgmap v813: 129 pgs: 129 active+clean; 3.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-09-10T08:22:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:21 smithi002 ceph-mon[30563]: pgmap v813: 129 pgs: 129 active+clean; 3.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-09-10T08:22:23.573 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:23 smithi002 ceph-mon[30563]: pgmap v814: 129 pgs: 129 active+clean; 3.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T08:22:23.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:23 smithi155 ceph-mon[35405]: pgmap v814: 129 pgs: 129 active+clean; 3.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T08:22:23.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:23 smithi107 ceph-mon[35048]: pgmap v814: 129 pgs: 129 active+clean; 3.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T08:22:25.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:25 smithi002 ceph-mon[30563]: pgmap v815: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-10T08:22:25.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:25 smithi155 ceph-mon[35405]: pgmap v815: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-10T08:22:25.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:25 smithi107 ceph-mon[35048]: pgmap v815: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-10T08:22:27.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:27 smithi155 ceph-mon[35405]: pgmap v816: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:22:27.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:27 smithi107 ceph-mon[35048]: pgmap v816: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:22:27.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:27 smithi002 ceph-mon[30563]: pgmap v816: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:22:29.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:29 smithi155 ceph-mon[35405]: pgmap v817: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:22:29.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:29 smithi107 ceph-mon[35048]: pgmap v817: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:22:29.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:29 smithi002 ceph-mon[30563]: pgmap v817: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-10T08:22:31.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:31 smithi155 ceph-mon[35405]: pgmap v818: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:22:31.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:31 smithi107 ceph-mon[35048]: pgmap v818: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:22:31.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:31 smithi002 ceph-mon[30563]: pgmap v818: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-10T08:22:31.822 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:22:32.217 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:22:32.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.806+0000 7f0437200640 1 -- 172.21.15.2:0/3502506189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04381019c0 msgr2=0x7f0438101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:32.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.806+0000 7f0437200640 1 --2- 172.21.15.2:0/3502506189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04381019c0 0x7f0438101e20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0420009930 tx=0x7f042002a420 comp rx=0 tx=0).stop 2024-09-10T08:22:32.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.806+0000 7f0437200640 1 -- 172.21.15.2:0/3502506189 shutdown_connections 2024-09-10T08:22:32.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.806+0000 7f0437200640 1 --2- 172.21.15.2:0/3502506189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0438102360 0x7f043810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.806+0000 7f0437200640 1 --2- 172.21.15.2:0/3502506189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04381019c0 0x7f0438101e20 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.806+0000 7f0437200640 1 --2- 172.21.15.2:0/3502506189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04381079c0 0x7f0438107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.806+0000 7f0437200640 1 -- 172.21.15.2:0/3502506189 >> 172.21.15.2:0/3502506189 conn(0x7f04380fd6e0 msgr2=0x7f04380ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:32.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.806+0000 7f0437200640 1 -- 172.21.15.2:0/3502506189 shutdown_connections 2024-09-10T08:22:32.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.807+0000 7f0437200640 1 -- 172.21.15.2:0/3502506189 wait complete. 2024-09-10T08:22:32.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.807+0000 7f0437200640 1 Processor -- start 2024-09-10T08:22:32.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.808+0000 7f0437200640 1 -- start start 2024-09-10T08:22:32.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.808+0000 7f0437200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04381019c0 0x7f04380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.808+0000 7f0437200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0438102360 0x7f0438074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.808+0000 7f0437200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04381079c0 0x7f0438074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.808+0000 7f0437200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0438111dc0 con 0x7f04381019c0 2024-09-10T08:22:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.808+0000 7f0437200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0438111c40 con 0x7f04381079c0 2024-09-10T08:22:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.808+0000 7f0437200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0438111f40 con 0x7f0438102360 2024-09-10T08:22:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.808+0000 7f0435e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04381019c0 0x7f04380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.808+0000 7f0435e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04381019c0 0x7f04380744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46560/0 (socket says 172.21.15.2:46560) 2024-09-10T08:22:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.808+0000 7f0435e00640 1 -- 172.21.15.2:0/4023073798 learned_addr learned my addr 172.21.15.2:0/4023073798 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.808+0000 7f042d400640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0438102360 0x7f0438074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.808+0000 7f0436800640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04381079c0 0x7f0438074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.809+0000 7f0435e00640 1 -- 172.21.15.2:0/4023073798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0438102360 msgr2=0x7f0438074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.809+0000 7f0435e00640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0438102360 0x7f0438074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.809+0000 7f0435e00640 1 -- 172.21.15.2:0/4023073798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04381079c0 msgr2=0x7f0438074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.809+0000 7f0435e00640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04381079c0 0x7f0438074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.809+0000 7f0435e00640 1 -- 172.21.15.2:0/4023073798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04381a6760 con 0x7f04381019c0 2024-09-10T08:22:32.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.809+0000 7f0435e00640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04381019c0 0x7f04380744f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f042800e9f0 tx=0x7f042800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:32.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.809+0000 7f042f400640 1 -- 172.21.15.2:0/4023073798 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042800cd80 con 0x7f04381019c0 2024-09-10T08:22:32.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.810+0000 7f042f400640 1 -- 172.21.15.2:0/4023073798 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0428005500 con 0x7f04381019c0 2024-09-10T08:22:32.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.810+0000 7f0437200640 1 -- 172.21.15.2:0/4023073798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04381a6a50 con 0x7f04381019c0 2024-09-10T08:22:32.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.810+0000 7f042f400640 1 -- 172.21.15.2:0/4023073798 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428010690 con 0x7f04381019c0 2024-09-10T08:22:32.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.810+0000 7f0437200640 1 -- 172.21.15.2:0/4023073798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04381a6f60 con 0x7f04381019c0 2024-09-10T08:22:32.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.813+0000 7f043fc4e740 1 Processor -- start 2024-09-10T08:22:32.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.813+0000 7f042f400640 1 -- 172.21.15.2:0/4023073798 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f04280046c0 con 0x7f04381019c0 2024-09-10T08:22:32.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.813+0000 7f043fc4e740 1 -- start start 2024-09-10T08:22:32.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.813+0000 7f043fc4e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 0x55873f1ea130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.813+0000 7f043fc4e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1ea670 0x55873f1f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.813+0000 7f043fc4e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1f4f30 0x55873f1f7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.813+0000 7f043fc4e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55873f1503e0 con 0x55873f1e9d50 2024-09-10T08:22:32.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.813+0000 7f043fc4e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55873f0f7d80 con 0x55873f1f4f30 2024-09-10T08:22:32.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.813+0000 7f043fc4e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55873f14c3a0 con 0x55873f1ea670 2024-09-10T08:22:32.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.814+0000 7f0435e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1ea670 0x55873f1f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.814+0000 7f0436800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 0x55873f1ea130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.814+0000 7f0436800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 0x55873f1ea130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46572/0 (socket says 172.21.15.2:46572) 2024-09-10T08:22:32.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.814+0000 7f0435e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1ea670 0x55873f1f49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:39220/0 (socket says 172.21.15.2:39220) 2024-09-10T08:22:32.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.814+0000 7f0436800640 1 -- 172.21.15.2:0/3732903459 learned_addr learned my addr 172.21.15.2:0/3732903459 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:32.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.814+0000 7f042d400640 1 --2- 172.21.15.2:0/3732903459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1f4f30 0x55873f1f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.814+0000 7f0435e00640 1 -- 172.21.15.2:0/3732903459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1f4f30 msgr2=0x55873f1f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:32.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.814+0000 7f0435e00640 1 --2- 172.21.15.2:0/3732903459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1f4f30 0x55873f1f7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.814+0000 7f0435e00640 1 -- 172.21.15.2:0/3732903459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 msgr2=0x55873f1ea130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:32.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.814+0000 7f0435e00640 1 --2- 172.21.15.2:0/3732903459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 0x55873f1ea130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.814+0000 7f0435e00640 1 -- 172.21.15.2:0/3732903459 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873f116ae0 con 0x55873f1ea670 2024-09-10T08:22:32.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.814+0000 7f0436800640 1 --2- 172.21.15.2:0/3732903459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 0x55873f1ea130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:32.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.815+0000 7f042f400640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f041c0a2c00 0x7f041c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.815+0000 7f0435e00640 1 --2- 172.21.15.2:0/3732903459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1ea670 0x55873f1f49f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f04280ccdd0 tx=0x7f04280d5b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdba9d37d668f096 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:32.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.815+0000 7f042f400640 1 -- 172.21.15.2:0/4023073798 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f04280cf780 con 0x7f04381019c0 2024-09-10T08:22:32.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.815+0000 7f0436800640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f041c0a2c00 0x7f041c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.815+0000 7f0427400640 1 -- 172.21.15.2:0/3732903459 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428097e30 con 0x55873f1ea670 2024-09-10T08:22:32.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.815+0000 7f0427400640 1 -- 172.21.15.2:0/3732903459 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f042809c290 con 0x55873f1ea670 2024-09-10T08:22:32.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.815+0000 7f0427400640 1 -- 172.21.15.2:0/3732903459 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042809b4f0 con 0x55873f1ea670 2024-09-10T08:22:32.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.816+0000 7f043fc4e740 1 -- 172.21.15.2:0/3732903459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1ea670 msgr2=0x55873f1f49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:32.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.816+0000 7f0436800640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f041c0a2c00 0x7f041c0a50c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0430011890 tx=0x7f0430009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:32.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.816+0000 7f043fc4e740 1 --2- 172.21.15.2:0/3732903459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1ea670 0x55873f1f49f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f04280ccdd0 tx=0x7f04280d5b20 comp rx=0 tx=0).stop 2024-09-10T08:22:32.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.816+0000 7f043fc4e740 1 -- 172.21.15.2:0/3732903459 shutdown_connections 2024-09-10T08:22:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.816+0000 7f043fc4e740 1 --2- 172.21.15.2:0/3732903459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1f4f30 0x55873f1f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.816+0000 7f043fc4e740 1 --2- 172.21.15.2:0/3732903459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1ea670 0x55873f1f49f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.816+0000 7f043fc4e740 1 --2- 172.21.15.2:0/3732903459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 0x55873f1ea130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.816+0000 7f043fc4e740 1 -- 172.21.15.2:0/3732903459 >> 172.21.15.2:0/3732903459 conn(0x55873f1e62e0 msgr2=0x55873f1e66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.816+0000 7f043fc4e740 1 -- 172.21.15.2:0/3732903459 shutdown_connections 2024-09-10T08:22:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.816+0000 7f043fc4e740 1 -- 172.21.15.2:0/3732903459 wait complete. 2024-09-10T08:22:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.817+0000 7f043fc4e740 1 -- start start 2024-09-10T08:22:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.817+0000 7f043fc4e740 1 Processor -- start 2024-09-10T08:22:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.817+0000 7f043fc4e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 0x55873f1f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.817+0000 7f043fc4e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f1f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.817+0000 7f043fc4e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1f4f30 0x55873f1f2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f043fc4e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55873f1f7ba0 con 0x55873f1e9d50 2024-09-10T08:22:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f043fc4e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55873f10b0f0 con 0x55873f1ea670 2024-09-10T08:22:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f043fc4e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55873f1f7d20 con 0x55873f1f4f30 2024-09-10T08:22:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f043fc4e740 10 client.0 _ll_register_callbacks cb 0x55873f0fc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f0435e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 0x55873f1f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f0435e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 0x55873f1f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46576/0 (socket says 172.21.15.2:46576) 2024-09-10T08:22:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f0435e00640 1 -- 172.21.15.2:0/3539890641 learned_addr learned my addr 172.21.15.2:0/3539890641 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f042d400640 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f1f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f0436800640 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1f4f30 0x55873f1f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f0435e00640 1 -- 172.21.15.2:0/3539890641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1f4f30 msgr2=0x55873f1f2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f0435e00640 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1f4f30 0x55873f1f2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f0435e00640 1 -- 172.21.15.2:0/3539890641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 msgr2=0x55873f1f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f0435e00640 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f1f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f0435e00640 1 -- 172.21.15.2:0/3539890641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873eec13c0 con 0x55873f1e9d50 2024-09-10T08:22:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.818+0000 7f042d400640 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f1f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:32.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.819+0000 7f0435e00640 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 0x55873f1f1e80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f042809b890 tx=0x7f042800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf211d0d31a7b1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:32.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.819+0000 7f041be00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:22:32.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.819+0000 7f041be00640 1 -- 172.21.15.2:0/3539890641 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042809b6d0 con 0x55873f1e9d50 2024-09-10T08:22:32.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.819+0000 7f041be00640 1 -- 172.21.15.2:0/3539890641 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0428098200 con 0x55873f1e9d50 2024-09-10T08:22:32.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.819+0000 7f043fc4e740 1 -- 172.21.15.2:0/3539890641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f042809c290 con 0x55873f1e9d50 2024-09-10T08:22:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.819+0000 7f041be00640 1 -- 172.21.15.2:0/3539890641 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f042809b960 con 0x55873f1e9d50 2024-09-10T08:22:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.819+0000 7f043fc4e740 10 client.19509 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.819+0000 7f043fc4e740 1 -- 172.21.15.2:0/3539890641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55873f2038b0 con 0x55873f1e9d50 2024-09-10T08:22:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.820+0000 7f041be00640 1 -- 172.21.15.2:0/3539890641 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f042809c290 con 0x55873f1e9d50 2024-09-10T08:22:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.820+0000 7f041be00640 1 -- 172.21.15.2:0/3539890641 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f04280ce070 con 0x55873f1e9d50 2024-09-10T08:22:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.820+0000 7f043fc4e740 10 client.19509 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.820+0000 7f043fc4e740 10 client.19509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.820+0000 7f043fc4e740 20 client.19509 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.820+0000 7f043fc4e740 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55873f204c00 0x55873f206ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.820+0000 7f043fc4e740 4 client.19509 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:22:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.820+0000 7f043fc4e740 1 -- 172.21.15.2:0/3539890641 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55873f15d950 con 0x55873f204c00 2024-09-10T08:22:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.820+0000 7f042d400640 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55873f204c00 0x55873f206ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.821+0000 7f042d400640 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55873f204c00 0x55873f206ff0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57d487f4a5030c83 server_cookie=dc16d8c0cf88dd09 in_seq=0 out_seq=0 2024-09-10T08:22:32.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.821+0000 7f041be00640 10 client.19509 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:32.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f041be00640 1 -- 172.21.15.2:0/3539890641 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55873f15d950 con 0x55873f204c00 2024-09-10T08:22:32.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f041be00640 10 client.19509 handle_command_reply: tid=0 2024-09-10T08:22:32.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f043fc4e740 1 client.19509 shutdown 2024-09-10T08:22:32.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f043fc4e740 1 -- 172.21.15.2:0/3539890641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 msgr2=0x55873f1f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:32.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f043fc4e740 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 0x55873f1f1e80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f042809b890 tx=0x7f042800cf40 comp rx=0 tx=0).stop 2024-09-10T08:22:32.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f043fc4e740 1 -- 172.21.15.2:0/3539890641 shutdown_connections 2024-09-10T08:22:32.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f043fc4e740 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55873f204c00 0x55873f206ff0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f043fc4e740 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1f4f30 0x55873f1f2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f043fc4e740 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f1f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f043fc4e740 1 --2- 172.21.15.2:0/3539890641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1e9d50 0x55873f1f1e80 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f043fc4e740 1 -- 172.21.15.2:0/3539890641 >> 172.21.15.2:0/3539890641 conn(0x55873f1e62e0 msgr2=0x55873f1ec010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:32.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f041be00640 0 client.19509 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:32.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f043fc4e740 1 -- 172.21.15.2:0/3539890641 shutdown_connections 2024-09-10T08:22:32.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.822+0000 7f043fc4e740 1 -- 172.21.15.2:0/3539890641 wait complete. 2024-09-10T08:22:32.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.823+0000 7f043fc4e740 20 client.19509 trim_cache size 0 max 16384 2024-09-10T08:22:32.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.841+0000 7f043fc4e740 1 Processor -- start 2024-09-10T08:22:32.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.841+0000 7f043fc4e740 1 -- start start 2024-09-10T08:22:32.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.841+0000 7f043fc4e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1e9d50 0x55873f1ec7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.841+0000 7f043fc4e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f209c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.841+0000 7f043fc4e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1f4f30 0x55873f20a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.841+0000 7f043fc4e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f04381b36c0 con 0x55873f1f4f30 2024-09-10T08:22:32.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.841+0000 7f043fc4e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04381b0f90 con 0x55873f1ea670 2024-09-10T08:22:32.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.841+0000 7f043fc4e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04381aede0 con 0x55873f1e9d50 2024-09-10T08:22:32.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.841+0000 7f0436800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1f4f30 0x55873f20a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.841+0000 7f0436800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1f4f30 0x55873f20a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46592/0 (socket says 172.21.15.2:46592) 2024-09-10T08:22:32.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.841+0000 7f0436800640 1 -- 172.21.15.2:0/83215946 learned_addr learned my addr 172.21.15.2:0/83215946 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:32.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.842+0000 7f0435e00640 1 --2- 172.21.15.2:0/83215946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1e9d50 0x55873f1ec7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.842+0000 7f042d400640 1 --2- 172.21.15.2:0/83215946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f209c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.842+0000 7f0436800640 1 -- 172.21.15.2:0/83215946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1e9d50 msgr2=0x55873f1ec7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:32.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.842+0000 7f0436800640 1 --2- 172.21.15.2:0/83215946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1e9d50 0x55873f1ec7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.842+0000 7f0436800640 1 -- 172.21.15.2:0/83215946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 msgr2=0x55873f209c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:32.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.842+0000 7f0436800640 1 --2- 172.21.15.2:0/83215946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f209c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.842+0000 7f0436800640 1 -- 172.21.15.2:0/83215946 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873f1f0e30 con 0x55873f1f4f30 2024-09-10T08:22:32.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.842+0000 7f0436800640 1 --2- 172.21.15.2:0/83215946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1f4f30 0x55873f20a170 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f043000fa70 tx=0x7f0430016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c06f2f6a856fecd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:32.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.842+0000 7f0426a00640 1 -- 172.21.15.2:0/83215946 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430004b20 con 0x55873f1f4f30 2024-09-10T08:22:32.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.842+0000 7f0426a00640 1 -- 172.21.15.2:0/83215946 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0430004cc0 con 0x55873f1f4f30 2024-09-10T08:22:32.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.843+0000 7f0426a00640 1 -- 172.21.15.2:0/83215946 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430005040 con 0x55873f1f4f30 2024-09-10T08:22:32.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.843+0000 7f043fc4e740 1 -- 172.21.15.2:0/83215946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1f4f30 msgr2=0x55873f20a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:32.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.843+0000 7f043fc4e740 1 --2- 172.21.15.2:0/83215946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1f4f30 0x55873f20a170 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f043000fa70 tx=0x7f0430016000 comp rx=0 tx=0).stop 2024-09-10T08:22:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.843+0000 7f043fc4e740 1 -- 172.21.15.2:0/83215946 shutdown_connections 2024-09-10T08:22:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.843+0000 7f043fc4e740 1 --2- 172.21.15.2:0/83215946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1f4f30 0x55873f20a170 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.843+0000 7f043fc4e740 1 --2- 172.21.15.2:0/83215946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f209c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.843+0000 7f043fc4e740 1 --2- 172.21.15.2:0/83215946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1e9d50 0x55873f1ec7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.843+0000 7f043fc4e740 1 -- 172.21.15.2:0/83215946 >> 172.21.15.2:0/83215946 conn(0x55873f1e62e0 msgr2=0x55873f1f36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.843+0000 7f043fc4e740 1 -- 172.21.15.2:0/83215946 shutdown_connections 2024-09-10T08:22:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.843+0000 7f043fc4e740 1 -- 172.21.15.2:0/83215946 wait complete. 2024-09-10T08:22:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.844+0000 7f043fc4e740 1 -- start start 2024-09-10T08:22:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.844+0000 7f043fc4e740 1 Processor -- start 2024-09-10T08:22:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.844+0000 7f043fc4e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1e9d50 0x55873f207a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.844+0000 7f043fc4e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f207fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.844+0000 7f043fc4e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1f4f30 0x55873f208510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.844+0000 7f043fc4e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f04381af450 con 0x55873f1f4f30 2024-09-10T08:22:32.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.844+0000 7f043fc4e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04381b08c0 con 0x55873f1ea670 2024-09-10T08:22:32.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.844+0000 7f043fc4e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04381afb20 con 0x55873f1e9d50 2024-09-10T08:22:32.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.844+0000 7f043fc4e740 10 client.0 _ll_register_callbacks cb 0x7f0428005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:32.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.845+0000 7f0436800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1f4f30 0x55873f208510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.845+0000 7f0436800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1f4f30 0x55873f208510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46596/0 (socket says 172.21.15.2:46596) 2024-09-10T08:22:32.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.845+0000 7f0436800640 1 -- 172.21.15.2:0/4103987023 learned_addr learned my addr 172.21.15.2:0/4103987023 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:32.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.845+0000 7f0435e00640 1 --2- 172.21.15.2:0/4103987023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1e9d50 0x55873f207a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.845+0000 7f042d400640 1 --2- 172.21.15.2:0/4103987023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f207fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.845+0000 7f042d400640 1 -- 172.21.15.2:0/4103987023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1e9d50 msgr2=0x55873f207a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:32.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.845+0000 7f042d400640 1 --2- 172.21.15.2:0/4103987023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1e9d50 0x55873f207a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.845+0000 7f042d400640 1 -- 172.21.15.2:0/4103987023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1f4f30 msgr2=0x55873f208510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:32.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.845+0000 7f042d400640 1 --2- 172.21.15.2:0/4103987023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1f4f30 0x55873f208510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.845+0000 7f042d400640 1 -- 172.21.15.2:0/4103987023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873f26e610 con 0x55873f1ea670 2024-09-10T08:22:32.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.845+0000 7f042d400640 1 --2- 172.21.15.2:0/4103987023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f207fd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0420005940 tx=0x7f0420031b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed8096ca7458fbca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:32.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.845+0000 7f041be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:22:32.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.846+0000 7f041be00640 1 -- 172.21.15.2:0/4103987023 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420035040 con 0x55873f1ea670 2024-09-10T08:22:32.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.846+0000 7f043fc4e740 1 -- 172.21.15.2:0/4103987023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0430004cc0 con 0x55873f1ea670 2024-09-10T08:22:32.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.846+0000 7f041be00640 1 -- 172.21.15.2:0/4103987023 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0420005020 con 0x55873f1ea670 2024-09-10T08:22:32.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.846+0000 7f041be00640 1 -- 172.21.15.2:0/4103987023 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0420005220 con 0x55873f1ea670 2024-09-10T08:22:32.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.846+0000 7f043fc4e740 10 client.29540 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:32.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.846+0000 7f043fc4e740 1 -- 172.21.15.2:0/4103987023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55873f26f960 con 0x55873f1ea670 2024-09-10T08:22:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.846+0000 7f041be00640 1 -- 172.21.15.2:0/4103987023 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0430004cc0 con 0x55873f1ea670 2024-09-10T08:22:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.846+0000 7f041be00640 1 -- 172.21.15.2:0/4103987023 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f042004f070 con 0x55873f1ea670 2024-09-10T08:22:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.846+0000 7f043fc4e740 10 client.29540 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.846+0000 7f043fc4e740 10 client.29540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.847+0000 7f043fc4e740 20 client.29540 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.847+0000 7f043fc4e740 1 --2- 172.21.15.2:0/4103987023 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55873f270110 0x55873f272530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.847+0000 7f043fc4e740 4 client.29540 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:22:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.847+0000 7f043fc4e740 1 -- 172.21.15.2:0/4103987023 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55873f157e60 con 0x55873f270110 2024-09-10T08:22:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.847+0000 7f0435e00640 1 --2- 172.21.15.2:0/4103987023 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55873f270110 0x55873f272530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.847+0000 7f0435e00640 1 --2- 172.21.15.2:0/4103987023 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55873f270110 0x55873f272530 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eec52a05e986544 server_cookie=44d9bc47524a0ebe in_seq=0 out_seq=0 2024-09-10T08:22:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.847+0000 7f041be00640 10 client.29540 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f041be00640 1 -- 172.21.15.2:0/4103987023 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55873f157e60 con 0x55873f270110 2024-09-10T08:22:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f041be00640 10 client.29540 handle_command_reply: tid=0 2024-09-10T08:22:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f043fc4e740 1 client.29540 shutdown 2024-09-10T08:22:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f043fc4e740 1 -- 172.21.15.2:0/4103987023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 msgr2=0x55873f207fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f043fc4e740 1 --2- 172.21.15.2:0/4103987023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f207fd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0420005940 tx=0x7f0420031b20 comp rx=0 tx=0).stop 2024-09-10T08:22:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f043fc4e740 1 -- 172.21.15.2:0/4103987023 shutdown_connections 2024-09-10T08:22:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f043fc4e740 1 --2- 172.21.15.2:0/4103987023 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55873f270110 0x55873f272530 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f043fc4e740 1 --2- 172.21.15.2:0/4103987023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55873f1f4f30 0x55873f208510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f043fc4e740 1 --2- 172.21.15.2:0/4103987023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55873f1ea670 0x55873f207fd0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f043fc4e740 1 --2- 172.21.15.2:0/4103987023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55873f1e9d50 0x55873f207a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f043fc4e740 1 -- 172.21.15.2:0/4103987023 >> 172.21.15.2:0/4103987023 conn(0x55873f1e62e0 msgr2=0x55873f1fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f041be00640 0 client.29540 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f043fc4e740 1 -- 172.21.15.2:0/4103987023 shutdown_connections 2024-09-10T08:22:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f043fc4e740 1 -- 172.21.15.2:0/4103987023 wait complete. 2024-09-10T08:22:32.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.848+0000 7f043fc4e740 20 client.29540 trim_cache size 0 max 16384 2024-09-10T08:22:32.853 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:22:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:22:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:22:32.853 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:22:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.849+0000 7f0437200640 1 -- 172.21.15.2:0/4023073798 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f041c0a2c00 msgr2=0x7f041c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.849+0000 7f0437200640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f041c0a2c00 0x7f041c0a50c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0430011890 tx=0x7f0430009500 comp rx=0 tx=0).stop 2024-09-10T08:22:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.849+0000 7f0437200640 1 -- 172.21.15.2:0/4023073798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04381019c0 msgr2=0x7f04380744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.849+0000 7f0437200640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04381019c0 0x7f04380744f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f042800e9f0 tx=0x7f042800edf0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.849+0000 7f0437200640 1 -- 172.21.15.2:0/4023073798 shutdown_connections 2024-09-10T08:22:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.849+0000 7f0437200640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04381079c0 0x7f0438074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.849+0000 7f0437200640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f041c0a2c00 0x7f041c0a50c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.849+0000 7f0437200640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0438102360 0x7f0438074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.849+0000 7f0437200640 1 --2- 172.21.15.2:0/4023073798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04381019c0 0x7f04380744f0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.849+0000 7f0437200640 1 -- 172.21.15.2:0/4023073798 >> 172.21.15.2:0/4023073798 conn(0x7f04380fd6e0 msgr2=0x7f04380fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.849+0000 7f0437200640 1 -- 172.21.15.2:0/4023073798 shutdown_connections 2024-09-10T08:22:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:32.849+0000 7f0437200640 1 -- 172.21.15.2:0/4023073798 wait complete. 2024-09-10T08:22:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.255+0000 7f2247e00640 1 -- 172.21.15.2:0/2228084484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2240078790 msgr2=0x7f2240079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.255+0000 7f2247e00640 1 --2- 172.21.15.2:0/2228084484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2240078790 0x7f2240079030 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2238009ca0 tx=0x7f223802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.255+0000 7f2247e00640 1 -- 172.21.15.2:0/2228084484 shutdown_connections 2024-09-10T08:22:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.255+0000 7f2247e00640 1 --2- 172.21.15.2:0/2228084484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2240078790 0x7f2240079030 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.255+0000 7f2247e00640 1 --2- 172.21.15.2:0/2228084484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2240077df0 0x7f2240078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.255+0000 7f2247e00640 1 --2- 172.21.15.2:0/2228084484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2240076bb0 0x7f2240076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.255+0000 7f2247e00640 1 -- 172.21.15.2:0/2228084484 >> 172.21.15.2:0/2228084484 conn(0x7f22400ff4e0 msgr2=0x7f2240101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.255+0000 7f2247e00640 1 -- 172.21.15.2:0/2228084484 shutdown_connections 2024-09-10T08:22:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.255+0000 7f2247e00640 1 -- 172.21.15.2:0/2228084484 wait complete. 2024-09-10T08:22:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.256+0000 7f2247e00640 1 Processor -- start 2024-09-10T08:22:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.256+0000 7f2247e00640 1 -- start start 2024-09-10T08:22:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.256+0000 7f2247e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2240076bb0 0x7f224019e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.256+0000 7f2247e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2240077df0 0x7f224019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.256+0000 7f2247e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2240078790 0x7f22401a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.256+0000 7f2247e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2240113e30 con 0x7f2240077df0 2024-09-10T08:22:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.256+0000 7f2247e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2240113cb0 con 0x7f2240078790 2024-09-10T08:22:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.256+0000 7f2247e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2240113fb0 con 0x7f2240076bb0 2024-09-10T08:22:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.256+0000 7f2244a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2240077df0 0x7f224019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.256+0000 7f2244a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2240077df0 0x7f224019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46622/0 (socket says 172.21.15.2:46622) 2024-09-10T08:22:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.256+0000 7f2244a00640 1 -- 172.21.15.2:0/1835070173 learned_addr learned my addr 172.21.15.2:0/1835070173 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.256+0000 7f2245400640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2240076bb0 0x7f224019e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.256+0000 7f2245e00640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2240078790 0x7f22401a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.257+0000 7f2244a00640 1 -- 172.21.15.2:0/1835070173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2240076bb0 msgr2=0x7f224019e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.257+0000 7f2244a00640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2240076bb0 0x7f224019e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.257+0000 7f2244a00640 1 -- 172.21.15.2:0/1835070173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2240078790 msgr2=0x7f22401a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.257+0000 7f2244a00640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2240078790 0x7f22401a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.257+0000 7f2244a00640 1 -- 172.21.15.2:0/1835070173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22401a5f80 con 0x7f2240077df0 2024-09-10T08:22:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.257+0000 7f2245e00640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2240078790 0x7f22401a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:22:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.257+0000 7f2244a00640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2240077df0 0x7f224019e8a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f222c00e9f0 tx=0x7f222c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.257+0000 7f223e000640 1 -- 172.21.15.2:0/1835070173 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c00cd80 con 0x7f2240077df0 2024-09-10T08:22:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.257+0000 7f223e000640 1 -- 172.21.15.2:0/1835070173 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222c005500 con 0x7f2240077df0 2024-09-10T08:22:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.257+0000 7f2247e00640 1 -- 172.21.15.2:0/1835070173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22401a6270 con 0x7f2240077df0 2024-09-10T08:22:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.257+0000 7f223e000640 1 -- 172.21.15.2:0/1835070173 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c010690 con 0x7f2240077df0 2024-09-10T08:22:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.257+0000 7f2247e00640 1 -- 172.21.15.2:0/1835070173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22401a6770 con 0x7f2240077df0 2024-09-10T08:22:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.260+0000 7f224ac18740 1 Processor -- start 2024-09-10T08:22:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.260+0000 7f223e000640 1 -- 172.21.15.2:0/1835070173 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f222c0046c0 con 0x7f2240077df0 2024-09-10T08:22:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.260+0000 7f224ac18740 1 -- start start 2024-09-10T08:22:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f224ac18740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d6090 0x55b0410d6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f224ac18740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d69d0 0x55b0410db490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f224ac18740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410dba60 0x55b0410ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f224ac18740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b04103b100 con 0x55b0410d6090 2024-09-10T08:22:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f224ac18740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b040fe31b0 con 0x55b0410d69d0 2024-09-10T08:22:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f224ac18740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b0410370c0 con 0x55b0410dba60 2024-09-10T08:22:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f2245e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d6090 0x55b0410d6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f2245e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d6090 0x55b0410d6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46634/0 (socket says 172.21.15.2:46634) 2024-09-10T08:22:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f2245e00640 1 -- 172.21.15.2:0/3068725785 learned_addr learned my addr 172.21.15.2:0/3068725785 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f2245400640 1 --2- 172.21.15.2:0/3068725785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d69d0 0x55b0410db490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f2244a00640 1 --2- 172.21.15.2:0/3068725785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410dba60 0x55b0410ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f2245e00640 1 -- 172.21.15.2:0/3068725785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410dba60 msgr2=0x55b0410ddf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f2245e00640 1 --2- 172.21.15.2:0/3068725785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410dba60 0x55b0410ddf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f2245e00640 1 -- 172.21.15.2:0/3068725785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d69d0 msgr2=0x55b0410db490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f2245e00640 1 --2- 172.21.15.2:0/3068725785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d69d0 0x55b0410db490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.261+0000 7f2245e00640 1 -- 172.21.15.2:0/3068725785 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b040d4e210 con 0x55b0410d6090 2024-09-10T08:22:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.262+0000 7f2245e00640 1 --2- 172.21.15.2:0/3068725785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d6090 0x55b0410d6490 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2238035000 tx=0x7f22380093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3ce7f5324f15d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.262+0000 7f2232600640 1 -- 172.21.15.2:0/3068725785 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238031cb0 con 0x55b0410d6090 2024-09-10T08:22:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.262+0000 7f223e000640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22280a2c30 0x7f22280a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.262+0000 7f2232600640 1 -- 172.21.15.2:0/3068725785 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2238036070 con 0x55b0410d6090 2024-09-10T08:22:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.262+0000 7f2232600640 1 -- 172.21.15.2:0/3068725785 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238004390 con 0x55b0410d6090 2024-09-10T08:22:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.262+0000 7f223e000640 1 -- 172.21.15.2:0/1835070173 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f222c0cf920 con 0x7f2240077df0 2024-09-10T08:22:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.262+0000 7f2245400640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22280a2c30 0x7f22280a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.263+0000 7f224ac18740 1 -- 172.21.15.2:0/3068725785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d6090 msgr2=0x55b0410d6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.263+0000 7f224ac18740 1 --2- 172.21.15.2:0/3068725785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d6090 0x55b0410d6490 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2238035000 tx=0x7f22380093e0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.263+0000 7f2245400640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22280a2c30 0x7f22280a50f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x55b0410d7400 tx=0x7f2234005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.263+0000 7f224ac18740 1 -- 172.21.15.2:0/3068725785 shutdown_connections 2024-09-10T08:22:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.263+0000 7f224ac18740 1 --2- 172.21.15.2:0/3068725785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410dba60 0x55b0410ddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.263+0000 7f224ac18740 1 --2- 172.21.15.2:0/3068725785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d69d0 0x55b0410db490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.263+0000 7f224ac18740 1 --2- 172.21.15.2:0/3068725785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d6090 0x55b0410d6490 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.263+0000 7f224ac18740 1 -- 172.21.15.2:0/3068725785 >> 172.21.15.2:0/3068725785 conn(0x55b0410d1700 msgr2=0x55b0410d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.263+0000 7f224ac18740 1 -- 172.21.15.2:0/3068725785 shutdown_connections 2024-09-10T08:22:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.263+0000 7f224ac18740 1 -- 172.21.15.2:0/3068725785 wait complete. 2024-09-10T08:22:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.264+0000 7f224ac18740 1 -- start start 2024-09-10T08:22:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.264+0000 7f224ac18740 1 Processor -- start 2024-09-10T08:22:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.264+0000 7f224ac18740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d6090 0x55b0410e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.264+0000 7f224ac18740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d69d0 0x55b0410e05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.264+0000 7f224ac18740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410dba60 0x55b0410e7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.264+0000 7f224ac18740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b0410de850 con 0x55b0410dba60 2024-09-10T08:22:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.264+0000 7f224ac18740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b040fc4090 con 0x55b0410d6090 2024-09-10T08:22:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.264+0000 7f224ac18740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b0410de9d0 con 0x55b0410d69d0 2024-09-10T08:22:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.264+0000 7f224ac18740 10 client.0 _ll_register_callbacks cb 0x55b041018f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.264+0000 7f2245400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d6090 0x55b0410e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.265+0000 7f2245e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410dba60 0x55b0410e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.265+0000 7f2245400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d6090 0x55b0410e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:58416/0 (socket says 172.21.15.2:58416) 2024-09-10T08:22:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.265+0000 7f2245400640 1 -- 172.21.15.2:0/2606817541 learned_addr learned my addr 172.21.15.2:0/2606817541 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.265+0000 7f2244a00640 1 --2- 172.21.15.2:0/2606817541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d69d0 0x55b0410e05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.265+0000 7f2245e00640 1 -- 172.21.15.2:0/2606817541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d69d0 msgr2=0x55b0410e05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.265+0000 7f2245e00640 1 --2- 172.21.15.2:0/2606817541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d69d0 0x55b0410e05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.265+0000 7f2245e00640 1 -- 172.21.15.2:0/2606817541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d6090 msgr2=0x55b0410e0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.265+0000 7f2245e00640 1 --2- 172.21.15.2:0/2606817541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d6090 0x55b0410e0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.265+0000 7f2245e00640 1 -- 172.21.15.2:0/2606817541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b040e1a620 con 0x55b0410dba60 2024-09-10T08:22:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.265+0000 7f2245e00640 1 --2- 172.21.15.2:0/2606817541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410dba60 0x55b0410e7610 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f223802ade0 tx=0x7f2238031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbc568498ba95cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.265+0000 7f2226a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:22:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.265+0000 7f2226a00640 1 -- 172.21.15.2:0/2606817541 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238004330 con 0x55b0410dba60 2024-09-10T08:22:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.265+0000 7f2226a00640 1 -- 172.21.15.2:0/2606817541 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f22380044d0 con 0x55b0410dba60 2024-09-10T08:22:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.266+0000 7f224ac18740 1 -- 172.21.15.2:0/2606817541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2238036070 con 0x55b0410dba60 2024-09-10T08:22:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.266+0000 7f2226a00640 1 -- 172.21.15.2:0/2606817541 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f22380046d0 con 0x55b0410dba60 2024-09-10T08:22:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.266+0000 7f224ac18740 10 client.19527 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.266+0000 7f224ac18740 1 -- 172.21.15.2:0/2606817541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0410de850 con 0x55b0410dba60 2024-09-10T08:22:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.266+0000 7f2226a00640 1 -- 172.21.15.2:0/2606817541 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2238036070 con 0x55b0410dba60 2024-09-10T08:22:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.266+0000 7f2226a00640 1 -- 172.21.15.2:0/2606817541 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2238042020 con 0x55b0410dba60 2024-09-10T08:22:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.266+0000 7f224ac18740 10 client.19527 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.266+0000 7f224ac18740 10 client.19527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.266+0000 7f224ac18740 20 client.19527 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.266+0000 7f224ac18740 1 --2- 172.21.15.2:0/2606817541 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b0410eb770 0x55b0410edb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.266+0000 7f224ac18740 4 client.19527 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:22:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.266+0000 7f224ac18740 1 -- 172.21.15.2:0/2606817541 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b041048830 con 0x55b0410eb770 2024-09-10T08:22:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.267+0000 7f2245400640 1 --2- 172.21.15.2:0/2606817541 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b0410eb770 0x55b0410edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.267+0000 7f2245400640 1 --2- 172.21.15.2:0/2606817541 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b0410eb770 0x55b0410edb60 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=137e8711e1b196da server_cookie=9346bf57c65fba0d in_seq=0 out_seq=0 2024-09-10T08:22:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.267+0000 7f2226a00640 10 client.19527 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.268+0000 7f2226a00640 1 -- 172.21.15.2:0/2606817541 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b041048830 con 0x55b0410eb770 2024-09-10T08:22:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.268+0000 7f2226a00640 10 client.19527 handle_command_reply: tid=0 2024-09-10T08:22:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.268+0000 7f224ac18740 1 client.19527 shutdown 2024-09-10T08:22:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.268+0000 7f224ac18740 1 -- 172.21.15.2:0/2606817541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410dba60 msgr2=0x55b0410e7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.268+0000 7f224ac18740 1 --2- 172.21.15.2:0/2606817541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410dba60 0x55b0410e7610 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f223802ade0 tx=0x7f2238031920 comp rx=0 tx=0).stop 2024-09-10T08:22:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.269+0000 7f224ac18740 1 -- 172.21.15.2:0/2606817541 shutdown_connections 2024-09-10T08:22:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.269+0000 7f224ac18740 1 --2- 172.21.15.2:0/2606817541 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b0410eb770 0x55b0410edb60 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.269+0000 7f224ac18740 1 --2- 172.21.15.2:0/2606817541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410dba60 0x55b0410e7610 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.269+0000 7f224ac18740 1 --2- 172.21.15.2:0/2606817541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d69d0 0x55b0410e05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.269+0000 7f224ac18740 1 --2- 172.21.15.2:0/2606817541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d6090 0x55b0410e0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.269+0000 7f2226a00640 0 client.19527 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.269+0000 7f224ac18740 1 -- 172.21.15.2:0/2606817541 >> 172.21.15.2:0/2606817541 conn(0x55b0410d1700 msgr2=0x55b0410d2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.269+0000 7f224ac18740 1 -- 172.21.15.2:0/2606817541 shutdown_connections 2024-09-10T08:22:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.269+0000 7f224ac18740 1 -- 172.21.15.2:0/2606817541 wait complete. 2024-09-10T08:22:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.269+0000 7f224ac18740 20 client.19527 trim_cache size 0 max 16384 2024-09-10T08:22:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.284+0000 7f224ac18740 1 Processor -- start 2024-09-10T08:22:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.284+0000 7f224ac18740 1 -- start start 2024-09-10T08:22:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.284+0000 7f224ac18740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 0x55b0410d2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.284+0000 7f224ac18740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d69d0 0x55b0410e75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.284+0000 7f224ac18740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410dba60 0x55b0410d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.284+0000 7f224ac18740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f22401b36c0 con 0x55b0410d69d0 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.284+0000 7f224ac18740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f22401b0f90 con 0x55b0410dba60 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.284+0000 7f224ac18740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22401aede0 con 0x55b0410d6090 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.284+0000 7f2244a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d69d0 0x55b0410e75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.284+0000 7f2245400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 0x55b0410d2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2244a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d69d0 0x55b0410e75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46656/0 (socket says 172.21.15.2:46656) 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2245400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 0x55b0410d2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:39292/0 (socket says 172.21.15.2:39292) 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2244a00640 1 -- 172.21.15.2:0/891025377 learned_addr learned my addr 172.21.15.2:0/891025377 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2245e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410dba60 0x55b0410d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2245400640 1 -- 172.21.15.2:0/891025377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410dba60 msgr2=0x55b0410d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2245400640 1 --2- 172.21.15.2:0/891025377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410dba60 0x55b0410d3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2245400640 1 -- 172.21.15.2:0/891025377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d69d0 msgr2=0x55b0410e75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2245400640 1 --2- 172.21.15.2:0/891025377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d69d0 0x55b0410e75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2245400640 1 -- 172.21.15.2:0/891025377 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0410ee1c0 con 0x55b0410d6090 2024-09-10T08:22:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2245e00640 1 --2- 172.21.15.2:0/891025377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410dba60 0x55b0410d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2244a00640 1 --2- 172.21.15.2:0/891025377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d69d0 0x55b0410e75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2245400640 1 --2- 172.21.15.2:0/891025377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 0x55b0410d2af0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55b0410ee140 tx=0x7f223401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee9300392f9f0254 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2231c00640 1 -- 172.21.15.2:0/891025377 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234004450 con 0x55b0410d6090 2024-09-10T08:22:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2231c00640 1 -- 172.21.15.2:0/891025377 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2234005500 con 0x55b0410d6090 2024-09-10T08:22:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.285+0000 7f2231c00640 1 -- 172.21.15.2:0/891025377 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223402b680 con 0x55b0410d6090 2024-09-10T08:22:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.286+0000 7f224ac18740 1 -- 172.21.15.2:0/891025377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 msgr2=0x55b0410d2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.286+0000 7f224ac18740 1 --2- 172.21.15.2:0/891025377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 0x55b0410d2af0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55b0410ee140 tx=0x7f223401b000 comp rx=0 tx=0).stop 2024-09-10T08:22:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.286+0000 7f224ac18740 1 -- 172.21.15.2:0/891025377 shutdown_connections 2024-09-10T08:22:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.286+0000 7f224ac18740 1 --2- 172.21.15.2:0/891025377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410dba60 0x55b0410d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.286+0000 7f224ac18740 1 --2- 172.21.15.2:0/891025377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410d69d0 0x55b0410e75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.286+0000 7f224ac18740 1 --2- 172.21.15.2:0/891025377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 0x55b0410d2af0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.286+0000 7f224ac18740 1 -- 172.21.15.2:0/891025377 >> 172.21.15.2:0/891025377 conn(0x55b0410d1700 msgr2=0x55b0410d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.286+0000 7f224ac18740 1 -- 172.21.15.2:0/891025377 shutdown_connections 2024-09-10T08:22:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.286+0000 7f224ac18740 1 -- 172.21.15.2:0/891025377 wait complete. 2024-09-10T08:22:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f224ac18740 1 -- start start 2024-09-10T08:22:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f224ac18740 1 Processor -- start 2024-09-10T08:22:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f224ac18740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 0x55b0410e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f224ac18740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d69d0 0x55b0410e4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f224ac18740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410dba60 0x55b0410e5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f224ac18740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f22401af450 con 0x55b0410dba60 2024-09-10T08:22:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f224ac18740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f22401b08c0 con 0x55b0410d69d0 2024-09-10T08:22:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f224ac18740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22401afb20 con 0x55b0410d6090 2024-09-10T08:22:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f224ac18740 10 client.0 _ll_register_callbacks cb 0x7f22340097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f2245400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 0x55b0410e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f2245400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 0x55b0410e4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:39298/0 (socket says 172.21.15.2:39298) 2024-09-10T08:22:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f2245400640 1 -- 172.21.15.2:0/1059915119 learned_addr learned my addr 172.21.15.2:0/1059915119 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f2244a00640 1 --2- 172.21.15.2:0/1059915119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d69d0 0x55b0410e4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.287+0000 7f2245e00640 1 --2- 172.21.15.2:0/1059915119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410dba60 0x55b0410e5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.288+0000 7f2245400640 1 -- 172.21.15.2:0/1059915119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d69d0 msgr2=0x55b0410e4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.288+0000 7f2245400640 1 --2- 172.21.15.2:0/1059915119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d69d0 0x55b0410e4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.288+0000 7f2245400640 1 -- 172.21.15.2:0/1059915119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410dba60 msgr2=0x55b0410e5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.288+0000 7f2245400640 1 --2- 172.21.15.2:0/1059915119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410dba60 0x55b0410e5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.288+0000 7f2245400640 1 -- 172.21.15.2:0/1059915119 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0410e6570 con 0x55b0410d6090 2024-09-10T08:22:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.288+0000 7f2245400640 1 --2- 172.21.15.2:0/1059915119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 0x55b0410e4a10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2234004230 tx=0x7f2234009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cc158884033dd4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.288+0000 7f2226a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:22:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.288+0000 7f2226a00640 1 -- 172.21.15.2:0/1059915119 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22340050c0 con 0x55b0410d6090 2024-09-10T08:22:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.288+0000 7f224ac18740 1 -- 172.21.15.2:0/1059915119 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2234005500 con 0x55b0410d6090 2024-09-10T08:22:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.288+0000 7f2226a00640 1 -- 172.21.15.2:0/1059915119 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2234005260 con 0x55b0410d6090 2024-09-10T08:22:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.289+0000 7f2226a00640 1 -- 172.21.15.2:0/1059915119 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2234047050 con 0x55b0410d6090 2024-09-10T08:22:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.289+0000 7f224ac18740 10 client.29485 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.289+0000 7f224ac18740 1 -- 172.21.15.2:0/1059915119 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0410e78d0 con 0x55b0410d6090 2024-09-10T08:22:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.289+0000 7f2226a00640 1 -- 172.21.15.2:0/1059915119 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2234005500 con 0x55b0410d6090 2024-09-10T08:22:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.289+0000 7f2226a00640 1 -- 172.21.15.2:0/1059915119 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f223402bb20 con 0x55b0410d6090 2024-09-10T08:22:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.289+0000 7f224ac18740 10 client.29485 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.289+0000 7f224ac18740 10 client.29485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.289+0000 7f224ac18740 20 client.29485 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.289+0000 7f224ac18740 1 --2- 172.21.15.2:0/1059915119 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b041156b10 0x55b041158f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.289+0000 7f224ac18740 4 client.29485 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:22:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.289+0000 7f224ac18740 1 -- 172.21.15.2:0/1059915119 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b041042b80 con 0x55b041156b10 2024-09-10T08:22:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.290+0000 7f2244a00640 1 --2- 172.21.15.2:0/1059915119 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b041156b10 0x55b041158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.290+0000 7f2244a00640 1 --2- 172.21.15.2:0/1059915119 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b041156b10 0x55b041158f30 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b189ea72e4e52d6 server_cookie=e93a9413e4b8cb27 in_seq=0 out_seq=0 2024-09-10T08:22:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.290+0000 7f2226a00640 10 client.29485 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f2226a00640 1 -- 172.21.15.2:0/1059915119 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b041042b80 con 0x55b041156b10 2024-09-10T08:22:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f2226a00640 10 client.29485 handle_command_reply: tid=0 2024-09-10T08:22:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f224ac18740 1 client.29485 shutdown 2024-09-10T08:22:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f224ac18740 1 -- 172.21.15.2:0/1059915119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 msgr2=0x55b0410e4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f224ac18740 1 --2- 172.21.15.2:0/1059915119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 0x55b0410e4a10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2234004230 tx=0x7f2234009e50 comp rx=0 tx=0).stop 2024-09-10T08:22:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f224ac18740 1 -- 172.21.15.2:0/1059915119 shutdown_connections 2024-09-10T08:22:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f224ac18740 1 --2- 172.21.15.2:0/1059915119 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b041156b10 0x55b041158f30 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f224ac18740 1 --2- 172.21.15.2:0/1059915119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0410dba60 0x55b0410e5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f224ac18740 1 --2- 172.21.15.2:0/1059915119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0410d69d0 0x55b0410e4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f224ac18740 1 --2- 172.21.15.2:0/1059915119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b0410d6090 0x55b0410e4a10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f2226a00640 0 client.29485 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f224ac18740 1 -- 172.21.15.2:0/1059915119 >> 172.21.15.2:0/1059915119 conn(0x55b0410d1700 msgr2=0x55b0410d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f224ac18740 1 -- 172.21.15.2:0/1059915119 shutdown_connections 2024-09-10T08:22:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.303+0000 7f224ac18740 1 -- 172.21.15.2:0/1059915119 wait complete. 2024-09-10T08:22:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.304+0000 7f224ac18740 20 client.29485 trim_cache size 0 max 16384 2024-09-10T08:22:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:22:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:22:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:22:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:22:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.304+0000 7f2247e00640 1 -- 172.21.15.2:0/1835070173 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22280a2c30 msgr2=0x7f22280a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.304+0000 7f2247e00640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22280a2c30 0x7f22280a50f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x55b0410d7400 tx=0x7f2234005f90 comp rx=0 tx=0).stop 2024-09-10T08:22:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.304+0000 7f2247e00640 1 -- 172.21.15.2:0/1835070173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2240077df0 msgr2=0x7f224019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.304+0000 7f2247e00640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2240077df0 0x7f224019e8a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f222c00e9f0 tx=0x7f222c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.304+0000 7f2247e00640 1 -- 172.21.15.2:0/1835070173 shutdown_connections 2024-09-10T08:22:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.304+0000 7f2247e00640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2240078790 0x7f22401a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.304+0000 7f2247e00640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2240077df0 0x7f224019e8a0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.304+0000 7f2247e00640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22280a2c30 0x7f22280a50f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.304+0000 7f2247e00640 1 --2- 172.21.15.2:0/1835070173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2240076bb0 0x7f224019e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.304+0000 7f2247e00640 1 -- 172.21.15.2:0/1835070173 >> 172.21.15.2:0/1835070173 conn(0x7f22400ff4e0 msgr2=0x7f2240100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.304+0000 7f2247e00640 1 -- 172.21.15.2:0/1835070173 shutdown_connections 2024-09-10T08:22:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:33.305+0000 7f2247e00640 1 -- 172.21.15.2:0/1835070173 wait complete. 2024-09-10T08:22:33.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:33 smithi155 ceph-mon[35405]: pgmap v819: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:22:33.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:33 smithi107 ceph-mon[35048]: pgmap v819: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:22:33.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:33 smithi002 ceph-mon[30563]: pgmap v819: 129 pgs: 129 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:22:33.890 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:22:33.891 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:22:34.268 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:22:34.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.879+0000 7f6c7c200640 1 -- 172.21.15.2:0/3269098241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c74101de0 msgr2=0x7f6c741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:34.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.879+0000 7f6c7c200640 1 --2- 172.21.15.2:0/3269098241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c74101de0 0x7f6c741021c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009900 tx=0x7f6c6802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:22:34.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.879+0000 7f6c7c200640 1 -- 172.21.15.2:0/3269098241 shutdown_connections 2024-09-10T08:22:34.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.879+0000 7f6c7c200640 1 --2- 172.21.15.2:0/3269098241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c74106f10 0x7f6c7410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:34.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.879+0000 7f6c7c200640 1 --2- 172.21.15.2:0/3269098241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c74102790 0x7f6c741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:34.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.879+0000 7f6c7c200640 1 --2- 172.21.15.2:0/3269098241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c74101de0 0x7f6c741021c0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:34.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.879+0000 7f6c7c200640 1 -- 172.21.15.2:0/3269098241 >> 172.21.15.2:0/3269098241 conn(0x7f6c740fd8c0 msgr2=0x7f6c740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:34.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.879+0000 7f6c7c200640 1 -- 172.21.15.2:0/3269098241 shutdown_connections 2024-09-10T08:22:34.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.879+0000 7f6c7c200640 1 -- 172.21.15.2:0/3269098241 wait complete. 2024-09-10T08:22:34.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.880+0000 7f6c7c200640 1 Processor -- start 2024-09-10T08:22:34.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.880+0000 7f6c7c200640 1 -- start start 2024-09-10T08:22:34.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c7c200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c74101de0 0x7f6c74198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:34.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c7c200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c74102790 0x7f6c74198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:34.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c7c200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c74106f10 0x7f6c7419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:34.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c7c200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74111f70 con 0x7f6c74101de0 2024-09-10T08:22:34.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c7c200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74111df0 con 0x7f6c74106f10 2024-09-10T08:22:34.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c7c200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741120f0 con 0x7f6c74102790 2024-09-10T08:22:34.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c79800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c74101de0 0x7f6c74198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:34.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c79800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c74101de0 0x7f6c74198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46680/0 (socket says 172.21.15.2:46680) 2024-09-10T08:22:34.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c79800640 1 -- 172.21.15.2:0/3410926084 learned_addr learned my addr 172.21.15.2:0/3410926084 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:34.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c7a200640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c74106f10 0x7f6c7419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:34.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c78e00640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c74102790 0x7f6c74198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:34.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c79800640 1 -- 172.21.15.2:0/3410926084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c74102790 msgr2=0x7f6c74198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:34.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c79800640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c74102790 0x7f6c74198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:34.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c79800640 1 -- 172.21.15.2:0/3410926084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c74106f10 msgr2=0x7f6c7419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:34.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c79800640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c74106f10 0x7f6c7419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:34.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.881+0000 7f6c79800640 1 -- 172.21.15.2:0/3410926084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7419d3f0 con 0x7f6c74101de0 2024-09-10T08:22:34.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.882+0000 7f6c79800640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c74101de0 0x7f6c74198440 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6c68006010 tx=0x7f6c680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:34.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.882+0000 7f6c72000640 1 -- 172.21.15.2:0/3410926084 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6803f030 con 0x7f6c74101de0 2024-09-10T08:22:34.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.882+0000 7f6c72000640 1 -- 172.21.15.2:0/3410926084 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c68003ce0 con 0x7f6c74101de0 2024-09-10T08:22:34.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.882+0000 7f6c7c200640 1 -- 172.21.15.2:0/3410926084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7419d680 con 0x7f6c74101de0 2024-09-10T08:22:34.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.882+0000 7f6c72000640 1 -- 172.21.15.2:0/3410926084 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68035070 con 0x7f6c74101de0 2024-09-10T08:22:34.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.882+0000 7f6c7c200640 1 -- 172.21.15.2:0/3410926084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c741a4fc0 con 0x7f6c74101de0 2024-09-10T08:22:34.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.883+0000 7f6c7c200640 1 -- 172.21.15.2:0/3410926084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c741021c0 con 0x7f6c74101de0 2024-09-10T08:22:34.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.885+0000 7f6c72000640 1 -- 172.21.15.2:0/3410926084 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6c68042030 con 0x7f6c74101de0 2024-09-10T08:22:34.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.887+0000 7f6c72000640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6c5c0a2c00 0x7f6c5c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:34.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.887+0000 7f6c72000640 1 -- 172.21.15.2:0/3410926084 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6c680efa80 con 0x7f6c74101de0 2024-09-10T08:22:34.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.887+0000 7f6c78e00640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6c5c0a2c00 0x7f6c5c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:34.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.888+0000 7f6c78e00640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6c5c0a2c00 0x7f6c5c0a50c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6c740ffe70 tx=0x7f6c60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:34.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:34.889+0000 7f6c72000640 1 -- 172.21.15.2:0/3410926084 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c680bc000 con 0x7f6c74101de0 2024-09-10T08:22:35.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.111+0000 7f6c7c200640 1 -- 172.21.15.2:0/3410926084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c741997a0 con 0x7f6c74101de0 2024-09-10T08:22:35.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.112+0000 7f6c72000640 1 -- 172.21.15.2:0/3410926084 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f6c741997a0 con 0x7f6c74101de0 2024-09-10T08:22:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:22:35.112 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:22:35.113 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:22:35.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.114+0000 7f6c7c200640 1 -- 172.21.15.2:0/3410926084 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6c5c0a2c00 msgr2=0x7f6c5c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:35.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.114+0000 7f6c7c200640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6c5c0a2c00 0x7f6c5c0a50c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6c740ffe70 tx=0x7f6c60005f90 comp rx=0 tx=0).stop 2024-09-10T08:22:35.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.114+0000 7f6c7c200640 1 -- 172.21.15.2:0/3410926084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c74101de0 msgr2=0x7f6c74198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:35.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.114+0000 7f6c7c200640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c74101de0 0x7f6c74198440 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6c68006010 tx=0x7f6c680042f0 comp rx=0 tx=0).stop 2024-09-10T08:22:35.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.114+0000 7f6c7c200640 1 -- 172.21.15.2:0/3410926084 shutdown_connections 2024-09-10T08:22:35.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.114+0000 7f6c7c200640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c74106f10 0x7f6c7419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:35.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.114+0000 7f6c7c200640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6c5c0a2c00 0x7f6c5c0a50c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:35.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.114+0000 7f6c7c200640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c74102790 0x7f6c74198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:35.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.114+0000 7f6c7c200640 1 --2- 172.21.15.2:0/3410926084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c74101de0 0x7f6c74198440 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:35.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.115+0000 7f6c7c200640 1 -- 172.21.15.2:0/3410926084 >> 172.21.15.2:0/3410926084 conn(0x7f6c740fd8c0 msgr2=0x7f6c740fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:35.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.115+0000 7f6c7c200640 1 -- 172.21.15.2:0/3410926084 shutdown_connections 2024-09-10T08:22:35.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:35.115+0000 7f6c7c200640 1 -- 172.21.15.2:0/3410926084 wait complete. 2024-09-10T08:22:35.504 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:35 smithi002 ceph-mon[30563]: pgmap v820: 129 pgs: 129 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-10T08:22:35.504 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:35 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3410926084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:22:35.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:35 smithi155 ceph-mon[35405]: pgmap v820: 129 pgs: 129 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-10T08:22:35.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:35 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3410926084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:22:35.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:35 smithi107 ceph-mon[35048]: pgmap v820: 129 pgs: 129 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-10T08:22:35.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:35 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3410926084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:22:36.012 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:22:36.396 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:22:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.023+0000 7ff588800640 1 -- 172.21.15.2:0/3246214490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff580109120 msgr2=0x7ff580110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.023+0000 7ff588800640 1 --2- 172.21.15.2:0/3246214490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff580109120 0x7ff580110ca0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009ca0 tx=0x7ff57c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.023+0000 7ff588800640 1 -- 172.21.15.2:0/3246214490 shutdown_connections 2024-09-10T08:22:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.023+0000 7ff588800640 1 --2- 172.21.15.2:0/3246214490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff580109120 0x7ff580110ca0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.023+0000 7ff588800640 1 --2- 172.21.15.2:0/3246214490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5801049a0 0x7ff5801089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.023+0000 7ff588800640 1 --2- 172.21.15.2:0/3246214490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff580103ff0 0x7ff5801043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.023+0000 7ff588800640 1 -- 172.21.15.2:0/3246214490 >> 172.21.15.2:0/3246214490 conn(0x7ff5800ffa90 msgr2=0x7ff580101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.023+0000 7ff588800640 1 -- 172.21.15.2:0/3246214490 shutdown_connections 2024-09-10T08:22:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.023+0000 7ff588800640 1 -- 172.21.15.2:0/3246214490 wait complete. 2024-09-10T08:22:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.024+0000 7ff588800640 1 Processor -- start 2024-09-10T08:22:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.024+0000 7ff588800640 1 -- start start 2024-09-10T08:22:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.025+0000 7ff588800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff580103ff0 0x7ff58019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.025+0000 7ff588800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5801049a0 0x7ff58019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.025+0000 7ff588800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff580109120 0x7ff58019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.025+0000 7ff588800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801141a0 con 0x7ff5801049a0 2024-09-10T08:22:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.025+0000 7ff588800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff580114020 con 0x7ff580109120 2024-09-10T08:22:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.025+0000 7ff588800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff580114320 con 0x7ff580103ff0 2024-09-10T08:22:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.025+0000 7ff585400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5801049a0 0x7ff58019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.025+0000 7ff585400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5801049a0 0x7ff58019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46004/0 (socket says 172.21.15.2:46004) 2024-09-10T08:22:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.025+0000 7ff585400640 1 -- 172.21.15.2:0/2928780088 learned_addr learned my addr 172.21.15.2:0/2928780088 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.025+0000 7ff586800640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff580109120 0x7ff58019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.025+0000 7ff585e00640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff580103ff0 0x7ff58019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.026+0000 7ff585400640 1 -- 172.21.15.2:0/2928780088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff580103ff0 msgr2=0x7ff58019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.026+0000 7ff585400640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff580103ff0 0x7ff58019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.026+0000 7ff585400640 1 -- 172.21.15.2:0/2928780088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff580109120 msgr2=0x7ff58019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.026+0000 7ff585400640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff580109120 0x7ff58019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.026+0000 7ff585400640 1 -- 172.21.15.2:0/2928780088 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58019f590 con 0x7ff5801049a0 2024-09-10T08:22:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.026+0000 7ff585e00640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff580103ff0 0x7ff58019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.026+0000 7ff586800640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff580109120 0x7ff58019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:37.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.026+0000 7ff585400640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5801049a0 0x7ff58019ab20 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff57000e9f0 tx=0x7ff57000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:37.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.026+0000 7ff57aa00640 1 -- 172.21.15.2:0/2928780088 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57000cd80 con 0x7ff5801049a0 2024-09-10T08:22:37.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.026+0000 7ff57aa00640 1 -- 172.21.15.2:0/2928780088 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff570005500 con 0x7ff5801049a0 2024-09-10T08:22:37.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.026+0000 7ff57aa00640 1 -- 172.21.15.2:0/2928780088 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570010690 con 0x7ff5801049a0 2024-09-10T08:22:37.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.026+0000 7ff588800640 1 -- 172.21.15.2:0/2928780088 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58019f880 con 0x7ff5801049a0 2024-09-10T08:22:37.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.026+0000 7ff588800640 1 -- 172.21.15.2:0/2928780088 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5801a7160 con 0x7ff5801049a0 2024-09-10T08:22:37.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.029+0000 7ff58b1f9740 1 Processor -- start 2024-09-10T08:22:37.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.030+0000 7ff57aa00640 1 -- 172.21.15.2:0/2928780088 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff5700046c0 con 0x7ff5801049a0 2024-09-10T08:22:37.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.030+0000 7ff58b1f9740 1 -- start start 2024-09-10T08:22:37.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.030+0000 7ff58b1f9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb82c70 0x5629cfb83050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.030+0000 7ff58b1f9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb83590 0x5629cfb8d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.030+0000 7ff58b1f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb8de50 0x5629cfb90240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.030+0000 7ff58b1f9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5629cfae93e0 con 0x5629cfb82c70 2024-09-10T08:22:37.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.030+0000 7ff58b1f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5629cfa90d80 con 0x5629cfb8de50 2024-09-10T08:22:37.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.030+0000 7ff58b1f9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5629cfae53a0 con 0x5629cfb83590 2024-09-10T08:22:37.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.030+0000 7ff586800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb82c70 0x5629cfb83050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.030+0000 7ff586800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb82c70 0x5629cfb83050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46016/0 (socket says 172.21.15.2:46016) 2024-09-10T08:22:37.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.030+0000 7ff586800640 1 -- 172.21.15.2:0/126133678 learned_addr learned my addr 172.21.15.2:0/126133678 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:37.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.030+0000 7ff585e00640 1 --2- 172.21.15.2:0/126133678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb83590 0x5629cfb8d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.030+0000 7ff585400640 1 --2- 172.21.15.2:0/126133678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb8de50 0x5629cfb90240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.031+0000 7ff586800640 1 -- 172.21.15.2:0/126133678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb83590 msgr2=0x5629cfb8d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:37.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.031+0000 7ff586800640 1 --2- 172.21.15.2:0/126133678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb83590 0x5629cfb8d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.031+0000 7ff586800640 1 -- 172.21.15.2:0/126133678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb8de50 msgr2=0x5629cfb90240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:37.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.031+0000 7ff586800640 1 --2- 172.21.15.2:0/126133678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb8de50 0x5629cfb90240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.031+0000 7ff586800640 1 -- 172.21.15.2:0/126133678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629cfaafae0 con 0x5629cfb82c70 2024-09-10T08:22:37.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.031+0000 7ff585e00640 1 --2- 172.21.15.2:0/126133678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb83590 0x5629cfb8d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:37.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.031+0000 7ff585400640 1 --2- 172.21.15.2:0/126133678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb8de50 0x5629cfb90240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:37.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.031+0000 7ff586800640 1 --2- 172.21.15.2:0/126133678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb82c70 0x5629cfb83050 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009ca0 tx=0x7ff57c033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5939af1b4f491292 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:37.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.032+0000 7ff56f000640 1 -- 172.21.15.2:0/126133678 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c003b70 con 0x5629cfb82c70 2024-09-10T08:22:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.032+0000 7ff56f000640 1 -- 172.21.15.2:0/126133678 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff57c036040 con 0x5629cfb82c70 2024-09-10T08:22:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.032+0000 7ff56f000640 1 -- 172.21.15.2:0/126133678 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c004500 con 0x5629cfb82c70 2024-09-10T08:22:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.032+0000 7ff57aa00640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5680a2b80 0x7ff5680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.032+0000 7ff57aa00640 1 -- 172.21.15.2:0/2928780088 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff5700cf830 con 0x7ff5801049a0 2024-09-10T08:22:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.032+0000 7ff585e00640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5680a2b80 0x7ff5680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.032+0000 7ff58b1f9740 1 -- 172.21.15.2:0/126133678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb82c70 msgr2=0x5629cfb83050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.032+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/126133678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb82c70 0x5629cfb83050 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009ca0 tx=0x7ff57c033000 comp rx=0 tx=0).stop 2024-09-10T08:22:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.032+0000 7ff58b1f9740 1 -- 172.21.15.2:0/126133678 shutdown_connections 2024-09-10T08:22:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.032+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/126133678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb8de50 0x5629cfb90240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.032+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/126133678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb83590 0x5629cfb8d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.032+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/126133678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb82c70 0x5629cfb83050 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.032+0000 7ff58b1f9740 1 -- 172.21.15.2:0/126133678 >> 172.21.15.2:0/126133678 conn(0x5629cfb7f200 msgr2=0x5629cfb7f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:37.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.033+0000 7ff58b1f9740 1 -- 172.21.15.2:0/126133678 shutdown_connections 2024-09-10T08:22:37.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.033+0000 7ff58b1f9740 1 -- 172.21.15.2:0/126133678 wait complete. 2024-09-10T08:22:37.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.033+0000 7ff585e00640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5680a2b80 0x7ff5680a5040 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff574009fd0 tx=0x7ff574005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:37.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.033+0000 7ff58b1f9740 1 -- start start 2024-09-10T08:22:37.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.033+0000 7ff58b1f9740 1 Processor -- start 2024-09-10T08:22:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.034+0000 7ff58b1f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb82c70 0x5629cfb8ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.034+0000 7ff58b1f9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb83590 0x5629cfb8b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.034+0000 7ff58b1f9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb8de50 0x5629cfb8b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.034+0000 7ff58b1f9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5629cfb908a0 con 0x5629cfb83590 2024-09-10T08:22:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.034+0000 7ff58b1f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5629cfaa40f0 con 0x5629cfb82c70 2024-09-10T08:22:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.034+0000 7ff58b1f9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5629cfb90a20 con 0x5629cfb8de50 2024-09-10T08:22:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.034+0000 7ff58b1f9740 10 client.0 _ll_register_callbacks cb 0x5629cfa8bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.034+0000 7ff586800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb8de50 0x5629cfb8b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.034+0000 7ff585e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb82c70 0x5629cfb8ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.034+0000 7ff585400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb83590 0x5629cfb8b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.034+0000 7ff586800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb8de50 0x5629cfb8b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34658/0 (socket says 172.21.15.2:34658) 2024-09-10T08:22:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.034+0000 7ff586800640 1 -- 172.21.15.2:0/3445394161 learned_addr learned my addr 172.21.15.2:0/3445394161 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.035+0000 7ff586800640 1 -- 172.21.15.2:0/3445394161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb82c70 msgr2=0x5629cfb8ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.035+0000 7ff586800640 1 --2- 172.21.15.2:0/3445394161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb82c70 0x5629cfb8ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.035+0000 7ff586800640 1 -- 172.21.15.2:0/3445394161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb83590 msgr2=0x5629cfb8b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.035+0000 7ff586800640 1 --2- 172.21.15.2:0/3445394161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb83590 0x5629cfb8b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.035+0000 7ff586800640 1 -- 172.21.15.2:0/3445394161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629cf85a3c0 con 0x5629cfb8de50 2024-09-10T08:22:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.035+0000 7ff586800640 1 --2- 172.21.15.2:0/3445394161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb8de50 0x5629cfb8b760 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff57c02aeb0 tx=0x7ff57c031700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c6a538cdf7f159f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.035+0000 7ff563400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:22:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.035+0000 7ff563400640 1 -- 172.21.15.2:0/3445394161 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c02f390 con 0x5629cfb8de50 2024-09-10T08:22:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.035+0000 7ff58b1f9740 1 -- 172.21.15.2:0/3445394161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff57c036040 con 0x5629cfb8de50 2024-09-10T08:22:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.035+0000 7ff563400640 1 -- 172.21.15.2:0/3445394161 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff57c004280 con 0x5629cfb8de50 2024-09-10T08:22:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.036+0000 7ff563400640 1 -- 172.21.15.2:0/3445394161 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff57c05e050 con 0x5629cfb8de50 2024-09-10T08:22:37.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.036+0000 7ff58b1f9740 10 client.29503 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:37.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.036+0000 7ff58b1f9740 1 -- 172.21.15.2:0/3445394161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629cfb908a0 con 0x5629cfb8de50 2024-09-10T08:22:37.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.036+0000 7ff563400640 1 -- 172.21.15.2:0/3445394161 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff57c036040 con 0x5629cfb8de50 2024-09-10T08:22:37.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.036+0000 7ff563400640 1 -- 172.21.15.2:0/3445394161 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff57c04f070 con 0x5629cfb8de50 2024-09-10T08:22:37.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.036+0000 7ff58b1f9740 10 client.29503 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:37.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.036+0000 7ff58b1f9740 10 client.29503 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:22:37.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.036+0000 7ff58b1f9740 20 client.29503 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:37.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.036+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/3445394161 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5629cfb9da40 0x5629cfb9fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.037+0000 7ff58b1f9740 4 client.29503 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:22:37.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.037+0000 7ff58b1f9740 1 -- 172.21.15.2:0/3445394161 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629cfaf6950 con 0x5629cfb9da40 2024-09-10T08:22:37.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.037+0000 7ff585e00640 1 --2- 172.21.15.2:0/3445394161 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5629cfb9da40 0x5629cfb9fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.038+0000 7ff585e00640 1 --2- 172.21.15.2:0/3445394161 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5629cfb9da40 0x5629cfb9fe30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e8382e0ddfec20e server_cookie=d11adcb5b0b813f1 in_seq=0 out_seq=0 2024-09-10T08:22:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.038+0000 7ff563400640 10 client.29503 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:22:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.038+0000 7ff563400640 1 -- 172.21.15.2:0/3445394161 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629cfaf6950 con 0x5629cfb9da40 2024-09-10T08:22:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.038+0000 7ff563400640 10 client.29503 handle_command_reply: tid=0 2024-09-10T08:22:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.038+0000 7ff58b1f9740 1 client.29503 shutdown 2024-09-10T08:22:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.039+0000 7ff58b1f9740 1 -- 172.21.15.2:0/3445394161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb8de50 msgr2=0x5629cfb8b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.039+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/3445394161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb8de50 0x5629cfb8b760 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff57c02aeb0 tx=0x7ff57c031700 comp rx=0 tx=0).stop 2024-09-10T08:22:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.039+0000 7ff58b1f9740 1 -- 172.21.15.2:0/3445394161 shutdown_connections 2024-09-10T08:22:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.039+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/3445394161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb8de50 0x5629cfb8b760 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.039+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/3445394161 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5629cfb9da40 0x5629cfb9fe30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.039+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/3445394161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb83590 0x5629cfb8b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.039+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/3445394161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb82c70 0x5629cfb8ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.039+0000 7ff563400640 0 client.29503 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:22:37.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.039+0000 7ff58b1f9740 1 -- 172.21.15.2:0/3445394161 >> 172.21.15.2:0/3445394161 conn(0x5629cfb7f200 msgr2=0x5629cfb7e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:37.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.039+0000 7ff58b1f9740 1 -- 172.21.15.2:0/3445394161 shutdown_connections 2024-09-10T08:22:37.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.039+0000 7ff58b1f9740 1 -- 172.21.15.2:0/3445394161 wait complete. 2024-09-10T08:22:37.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.039+0000 7ff58b1f9740 20 client.29503 trim_cache size 0 max 16384 2024-09-10T08:22:37.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.053+0000 7ff58b1f9740 1 Processor -- start 2024-09-10T08:22:37.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.053+0000 7ff58b1f9740 1 -- start start 2024-09-10T08:22:37.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.053+0000 7ff58b1f9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb82c70 0x5629cfb8d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.053+0000 7ff58b1f9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb83590 0x5629cfb856d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.053+0000 7ff58b1f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb8de50 0x5629cfba5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.053+0000 7ff58b1f9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801b37c0 con 0x5629cfb83590 2024-09-10T08:22:37.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.053+0000 7ff58b1f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801b1090 con 0x5629cfb8de50 2024-09-10T08:22:37.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.053+0000 7ff58b1f9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801aeee0 con 0x5629cfb82c70 2024-09-10T08:22:37.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.054+0000 7ff585400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb83590 0x5629cfb856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.054+0000 7ff585400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb83590 0x5629cfb856d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46042/0 (socket says 172.21.15.2:46042) 2024-09-10T08:22:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.054+0000 7ff585400640 1 -- 172.21.15.2:0/3206724085 learned_addr learned my addr 172.21.15.2:0/3206724085 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.054+0000 7ff586800640 1 --2- 172.21.15.2:0/3206724085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb8de50 0x5629cfba5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.054+0000 7ff585e00640 1 --2- 172.21.15.2:0/3206724085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb82c70 0x5629cfb8d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.054+0000 7ff585400640 1 -- 172.21.15.2:0/3206724085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb82c70 msgr2=0x5629cfb8d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.054+0000 7ff585400640 1 --2- 172.21.15.2:0/3206724085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb82c70 0x5629cfb8d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.054+0000 7ff585400640 1 -- 172.21.15.2:0/3206724085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb8de50 msgr2=0x5629cfba5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.054+0000 7ff585400640 1 --2- 172.21.15.2:0/3206724085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb8de50 0x5629cfba5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.054+0000 7ff585400640 1 -- 172.21.15.2:0/3206724085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629cfba01a0 con 0x5629cfb83590 2024-09-10T08:22:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.054+0000 7ff586800640 1 --2- 172.21.15.2:0/3206724085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb8de50 0x5629cfba5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:37.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.054+0000 7ff585400640 1 --2- 172.21.15.2:0/3206724085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb83590 0x5629cfb856d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff5700049d0 tx=0x7ff5700d54b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d927cb3e22852ace server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.054+0000 7ff56e600640 1 -- 172.21.15.2:0/3206724085 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57009b780 con 0x5629cfb83590 2024-09-10T08:22:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.055+0000 7ff56e600640 1 -- 172.21.15.2:0/3206724085 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff57009c340 con 0x5629cfb83590 2024-09-10T08:22:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.055+0000 7ff56e600640 1 -- 172.21.15.2:0/3206724085 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57009b5a0 con 0x5629cfb83590 2024-09-10T08:22:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.055+0000 7ff58b1f9740 1 -- 172.21.15.2:0/3206724085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb83590 msgr2=0x5629cfb856d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.055+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/3206724085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb83590 0x5629cfb856d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff5700049d0 tx=0x7ff5700d54b0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.055+0000 7ff58b1f9740 1 -- 172.21.15.2:0/3206724085 shutdown_connections 2024-09-10T08:22:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.055+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/3206724085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb8de50 0x5629cfba5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.055+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/3206724085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb83590 0x5629cfb856d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.055+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/3206724085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb82c70 0x5629cfb8d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.055+0000 7ff58b1f9740 1 -- 172.21.15.2:0/3206724085 >> 172.21.15.2:0/3206724085 conn(0x5629cfb7f200 msgr2=0x5629cfb89ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.055+0000 7ff58b1f9740 1 -- 172.21.15.2:0/3206724085 shutdown_connections 2024-09-10T08:22:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.055+0000 7ff58b1f9740 1 -- 172.21.15.2:0/3206724085 wait complete. 2024-09-10T08:22:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.056+0000 7ff58b1f9740 1 -- start start 2024-09-10T08:22:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.056+0000 7ff58b1f9740 1 Processor -- start 2024-09-10T08:22:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.056+0000 7ff58b1f9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb82c70 0x5629cfb96230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.056+0000 7ff58b1f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb83590 0x5629cfb91560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.056+0000 7ff58b1f9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb8de50 0x5629cfb91aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.056+0000 7ff58b1f9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801af550 con 0x5629cfb82c70 2024-09-10T08:22:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.056+0000 7ff58b1f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801b09c0 con 0x5629cfb83590 2024-09-10T08:22:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.056+0000 7ff58b1f9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801afc20 con 0x5629cfb8de50 2024-09-10T08:22:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.056+0000 7ff58b1f9740 10 client.0 _ll_register_callbacks cb 0x7ff5700d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.057+0000 7ff585400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb83590 0x5629cfb91560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.057+0000 7ff585400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb83590 0x5629cfb91560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40480/0 (socket says 172.21.15.2:40480) 2024-09-10T08:22:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.057+0000 7ff585e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb82c70 0x5629cfb96230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.057+0000 7ff585400640 1 -- 172.21.15.2:0/2368498989 learned_addr learned my addr 172.21.15.2:0/2368498989 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.057+0000 7ff586800640 1 --2- 172.21.15.2:0/2368498989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb8de50 0x5629cfb91aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.057+0000 7ff585400640 1 -- 172.21.15.2:0/2368498989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb8de50 msgr2=0x5629cfb91aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.057+0000 7ff585400640 1 --2- 172.21.15.2:0/2368498989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb8de50 0x5629cfb91aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.057+0000 7ff585400640 1 -- 172.21.15.2:0/2368498989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb82c70 msgr2=0x5629cfb96230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.057+0000 7ff585400640 1 --2- 172.21.15.2:0/2368498989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb82c70 0x5629cfb96230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.057+0000 7ff585400640 1 -- 172.21.15.2:0/2368498989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629cfb92110 con 0x5629cfb83590 2024-09-10T08:22:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.057+0000 7ff585e00640 1 --2- 172.21.15.2:0/2368498989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb82c70 0x5629cfb96230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.057+0000 7ff585400640 1 --2- 172.21.15.2:0/2368498989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb83590 0x5629cfb91560 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff5700980a0 tx=0x7ff5700d5340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f4f9d3aca7905d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:37.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.057+0000 7ff563400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:22:37.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.058+0000 7ff563400640 1 -- 172.21.15.2:0/2368498989 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5700b3070 con 0x5629cfb83590 2024-09-10T08:22:37.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.058+0000 7ff58b1f9740 1 -- 172.21.15.2:0/2368498989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff57009c340 con 0x5629cfb83590 2024-09-10T08:22:37.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.058+0000 7ff563400640 1 -- 172.21.15.2:0/2368498989 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff5700ba5e0 con 0x5629cfb83590 2024-09-10T08:22:37.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.058+0000 7ff563400640 1 -- 172.21.15.2:0/2368498989 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff57009b5a0 con 0x5629cfb83590 2024-09-10T08:22:37.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.058+0000 7ff58b1f9740 10 client.29582 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:37.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.058+0000 7ff58b1f9740 1 -- 172.21.15.2:0/2368498989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629cfc08590 con 0x5629cfb83590 2024-09-10T08:22:37.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.058+0000 7ff563400640 1 -- 172.21.15.2:0/2368498989 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff57009c340 con 0x5629cfb83590 2024-09-10T08:22:37.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.058+0000 7ff563400640 1 -- 172.21.15.2:0/2368498989 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff5700a4420 con 0x5629cfb83590 2024-09-10T08:22:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.058+0000 7ff58b1f9740 10 client.29582 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.058+0000 7ff58b1f9740 10 client.29582 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:22:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.058+0000 7ff58b1f9740 20 client.29582 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.058+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/2368498989 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5629cfc08c10 0x5629cfc0b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.059+0000 7ff58b1f9740 4 client.29582 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:22:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.059+0000 7ff58b1f9740 1 -- 172.21.15.2:0/2368498989 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629cfaf0e60 con 0x5629cfc08c10 2024-09-10T08:22:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.059+0000 7ff585e00640 1 --2- 172.21.15.2:0/2368498989 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5629cfc08c10 0x5629cfc0b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.059+0000 7ff585e00640 1 --2- 172.21.15.2:0/2368498989 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5629cfc08c10 0x5629cfc0b030 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef5e78950906e1bd server_cookie=1c87658988191529 in_seq=0 out_seq=0 2024-09-10T08:22:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.059+0000 7ff563400640 10 client.29582 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:22:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff563400640 1 -- 172.21.15.2:0/2368498989 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629cfaf0e60 con 0x5629cfc08c10 2024-09-10T08:22:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff563400640 10 client.29582 handle_command_reply: tid=0 2024-09-10T08:22:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff58b1f9740 1 client.29582 shutdown 2024-09-10T08:22:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff58b1f9740 1 -- 172.21.15.2:0/2368498989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb83590 msgr2=0x5629cfb91560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/2368498989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb83590 0x5629cfb91560 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff5700980a0 tx=0x7ff5700d5340 comp rx=0 tx=0).stop 2024-09-10T08:22:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff58b1f9740 1 -- 172.21.15.2:0/2368498989 shutdown_connections 2024-09-10T08:22:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/2368498989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629cfb8de50 0x5629cfb91aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/2368498989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5629cfb83590 0x5629cfb91560 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/2368498989 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x5629cfc08c10 0x5629cfc0b030 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff58b1f9740 1 --2- 172.21.15.2:0/2368498989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629cfb82c70 0x5629cfb96230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff563400640 0 client.29582 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:22:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff58b1f9740 1 -- 172.21.15.2:0/2368498989 >> 172.21.15.2:0/2368498989 conn(0x5629cfb7f200 msgr2=0x5629cfb7db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff58b1f9740 1 -- 172.21.15.2:0/2368498989 shutdown_connections 2024-09-10T08:22:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff58b1f9740 1 -- 172.21.15.2:0/2368498989 wait complete. 2024-09-10T08:22:37.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.060+0000 7ff58b1f9740 20 client.29582 trim_cache size 0 max 16384 2024-09-10T08:22:37.065 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:22:37.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.061+0000 7ff588800640 1 -- 172.21.15.2:0/2928780088 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5680a2b80 msgr2=0x7ff5680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:37.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.061+0000 7ff588800640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5680a2b80 0x7ff5680a5040 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff574009fd0 tx=0x7ff574005f90 comp rx=0 tx=0).stop 2024-09-10T08:22:37.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.061+0000 7ff588800640 1 -- 172.21.15.2:0/2928780088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5801049a0 msgr2=0x7ff58019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.061+0000 7ff588800640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5801049a0 0x7ff58019ab20 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff57000e9f0 tx=0x7ff57000edf0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.061+0000 7ff588800640 1 -- 172.21.15.2:0/2928780088 shutdown_connections 2024-09-10T08:22:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.061+0000 7ff588800640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff580109120 0x7ff58019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.061+0000 7ff588800640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5801049a0 0x7ff58019ab20 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.061+0000 7ff588800640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff5680a2b80 0x7ff5680a5040 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.061+0000 7ff588800640 1 --2- 172.21.15.2:0/2928780088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff580103ff0 0x7ff58019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.061+0000 7ff588800640 1 -- 172.21.15.2:0/2928780088 >> 172.21.15.2:0/2928780088 conn(0x7ff5800ffa90 msgr2=0x7ff5801005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.061+0000 7ff588800640 1 -- 172.21.15.2:0/2928780088 shutdown_connections 2024-09-10T08:22:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:37.061+0000 7ff588800640 1 -- 172.21.15.2:0/2928780088 wait complete. 2024-09-10T08:22:37.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:37 smithi155 ceph-mon[35405]: pgmap v821: 129 pgs: 129 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:22:37.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:37 smithi107 ceph-mon[35048]: pgmap v821: 129 pgs: 129 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:22:37.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:37 smithi002 ceph-mon[30563]: pgmap v821: 129 pgs: 129 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:22:38.137 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:22:38.517 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:22:39.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.135+0000 7faed2600640 1 -- 172.21.15.2:0/1031095192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faecc109120 msgr2=0x7faecc110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:39.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.135+0000 7faed2600640 1 --2- 172.21.15.2:0/1031095192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faecc109120 0x7faecc110ca0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7faec4009ca0 tx=0x7faec402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.136+0000 7faed2600640 1 -- 172.21.15.2:0/1031095192 shutdown_connections 2024-09-10T08:22:39.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.136+0000 7faed2600640 1 --2- 172.21.15.2:0/1031095192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faecc109120 0x7faecc110ca0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.136+0000 7faed2600640 1 --2- 172.21.15.2:0/1031095192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faecc1049a0 0x7faecc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.136+0000 7faed2600640 1 --2- 172.21.15.2:0/1031095192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc103ff0 0x7faecc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.136+0000 7faed2600640 1 -- 172.21.15.2:0/1031095192 >> 172.21.15.2:0/1031095192 conn(0x7faecc0ffa90 msgr2=0x7faecc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:39.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.136+0000 7faed2600640 1 -- 172.21.15.2:0/1031095192 shutdown_connections 2024-09-10T08:22:39.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.136+0000 7faed2600640 1 -- 172.21.15.2:0/1031095192 wait complete. 2024-09-10T08:22:39.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.137+0000 7faed2600640 1 Processor -- start 2024-09-10T08:22:39.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.137+0000 7faed2600640 1 -- start start 2024-09-10T08:22:39.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.137+0000 7faed2600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faecc103ff0 0x7faecc19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.137+0000 7faed2600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faecc1049a0 0x7faecc19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.137+0000 7faed2600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc109120 0x7faecc19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.137+0000 7faed2600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faecc114150 con 0x7faecc103ff0 2024-09-10T08:22:39.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.137+0000 7faed2600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faecc113fd0 con 0x7faecc109120 2024-09-10T08:22:39.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.137+0000 7faed2600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1142d0 con 0x7faecc1049a0 2024-09-10T08:22:39.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.137+0000 7faecb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faecc103ff0 0x7faecc19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.137+0000 7faecb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faecc103ff0 0x7faecc19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46058/0 (socket says 172.21.15.2:46058) 2024-09-10T08:22:39.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.137+0000 7faecb400640 1 -- 172.21.15.2:0/4266798844 learned_addr learned my addr 172.21.15.2:0/4266798844 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:39.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.138+0000 7faecbe00640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc109120 0x7faecc19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.138+0000 7faec2a00640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faecc1049a0 0x7faecc19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.138+0000 7faecb400640 1 -- 172.21.15.2:0/4266798844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faecc1049a0 msgr2=0x7faecc19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:39.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.138+0000 7faecb400640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faecc1049a0 0x7faecc19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.138+0000 7faecb400640 1 -- 172.21.15.2:0/4266798844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc109120 msgr2=0x7faecc19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:39.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.138+0000 7faecb400640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc109120 0x7faecc19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.138+0000 7faecb400640 1 -- 172.21.15.2:0/4266798844 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc19f560 con 0x7faecc103ff0 2024-09-10T08:22:39.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.138+0000 7faecb400640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faecc103ff0 0x7faecc19a5b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7faebc00e9f0 tx=0x7faebc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:39.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.139+0000 7faec8c00640 1 -- 172.21.15.2:0/4266798844 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc00cd80 con 0x7faecc103ff0 2024-09-10T08:22:39.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.139+0000 7faec8c00640 1 -- 172.21.15.2:0/4266798844 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faebc010910 con 0x7faecc103ff0 2024-09-10T08:22:39.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.139+0000 7faec8c00640 1 -- 172.21.15.2:0/4266798844 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc018990 con 0x7faecc103ff0 2024-09-10T08:22:39.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.139+0000 7faed2600640 1 -- 172.21.15.2:0/4266798844 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faecc19f850 con 0x7faecc103ff0 2024-09-10T08:22:39.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.139+0000 7faed2600640 1 -- 172.21.15.2:0/4266798844 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faecc1a7130 con 0x7faecc103ff0 2024-09-10T08:22:39.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.142+0000 7faed5208740 1 Processor -- start 2024-09-10T08:22:39.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.142+0000 7faec8c00640 1 -- 172.21.15.2:0/4266798844 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7faebc010ac0 con 0x7faecc103ff0 2024-09-10T08:22:39.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.142+0000 7faed5208740 1 -- start start 2024-09-10T08:22:39.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.142+0000 7faed5208740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cd9c70 0x5622e2cda050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.142+0000 7faed5208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cda590 0x5622e2ce4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.142+0000 7faed5208740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.142+0000 7faed5208740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5622e2c403e0 con 0x5622e2ce4e50 2024-09-10T08:22:39.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.142+0000 7faed5208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5622e2be7d80 con 0x5622e2cda590 2024-09-10T08:22:39.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.142+0000 7faed5208740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5622e2c3c3a0 con 0x5622e2cd9c70 2024-09-10T08:22:39.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.143+0000 7faecb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cda590 0x5622e2ce4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.143+0000 7faecbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cd9c70 0x5622e2cda050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.143+0000 7faec2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.143+0000 7faecbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cd9c70 0x5622e2cda050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34692/0 (socket says 172.21.15.2:34692) 2024-09-10T08:22:39.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.143+0000 7faecbe00640 1 -- 172.21.15.2:0/1317715989 learned_addr learned my addr 172.21.15.2:0/1317715989 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:39.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.143+0000 7faecbe00640 1 -- 172.21.15.2:0/1317715989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cda590 msgr2=0x5622e2ce4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:39.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.143+0000 7faecbe00640 1 --2- 172.21.15.2:0/1317715989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cda590 0x5622e2ce4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.143+0000 7faecbe00640 1 -- 172.21.15.2:0/1317715989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 msgr2=0x5622e2ce7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:39.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.143+0000 7faecbe00640 1 --2- 172.21.15.2:0/1317715989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.143+0000 7faecbe00640 1 -- 172.21.15.2:0/1317715989 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e2c06ae0 con 0x5622e2cd9c70 2024-09-10T08:22:39.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.143+0000 7faecb400640 1 --2- 172.21.15.2:0/1317715989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cda590 0x5622e2ce4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:39.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.143+0000 7faec2a00640 1 --2- 172.21.15.2:0/1317715989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:39.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.144+0000 7faecbe00640 1 --2- 172.21.15.2:0/1317715989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cd9c70 0x5622e2cda050 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faec4009ca0 tx=0x7faec4005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=907a69f0e79b5268 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:39.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.144+0000 7faec8c00640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faeb00a2b80 0x7faeb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.144+0000 7faec8c00640 1 -- 172.21.15.2:0/4266798844 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faebc0cf7a0 con 0x7faecc103ff0 2024-09-10T08:22:39.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.144+0000 7faecb400640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faeb00a2b80 0x7faeb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.144+0000 7faec0c00640 1 -- 172.21.15.2:0/1317715989 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4031db0 con 0x5622e2cd9c70 2024-09-10T08:22:39.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.145+0000 7faec0c00640 1 -- 172.21.15.2:0/1317715989 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec4036040 con 0x5622e2cd9c70 2024-09-10T08:22:39.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.145+0000 7faec0c00640 1 -- 172.21.15.2:0/1317715989 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec40042a0 con 0x5622e2cd9c70 2024-09-10T08:22:39.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.145+0000 7faed5208740 1 -- 172.21.15.2:0/1317715989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cd9c70 msgr2=0x5622e2cda050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:39.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.145+0000 7faecb400640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faeb00a2b80 0x7faeb00a5040 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faebc00c6a0 tx=0x7faebc0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:39.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.145+0000 7faed5208740 1 --2- 172.21.15.2:0/1317715989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cd9c70 0x5622e2cda050 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faec4009ca0 tx=0x7faec4005f10 comp rx=0 tx=0).stop 2024-09-10T08:22:39.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.145+0000 7faed5208740 1 -- 172.21.15.2:0/1317715989 shutdown_connections 2024-09-10T08:22:39.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.145+0000 7faed5208740 1 --2- 172.21.15.2:0/1317715989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.145+0000 7faed5208740 1 --2- 172.21.15.2:0/1317715989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cda590 0x5622e2ce4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.145+0000 7faed5208740 1 --2- 172.21.15.2:0/1317715989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cd9c70 0x5622e2cda050 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.145+0000 7faed5208740 1 -- 172.21.15.2:0/1317715989 >> 172.21.15.2:0/1317715989 conn(0x5622e2cd6200 msgr2=0x5622e2cd65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:39.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.146+0000 7faed5208740 1 -- 172.21.15.2:0/1317715989 shutdown_connections 2024-09-10T08:22:39.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.146+0000 7faed5208740 1 -- 172.21.15.2:0/1317715989 wait complete. 2024-09-10T08:22:39.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.146+0000 7faed5208740 1 -- start start 2024-09-10T08:22:39.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.146+0000 7faed5208740 1 Processor -- start 2024-09-10T08:22:39.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.147+0000 7faed5208740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cd9c70 0x5622e2ce1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.147+0000 7faed5208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cda590 0x5622e2ce2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.147+0000 7faed5208740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.147+0000 7faed5208740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5622e2ce78a0 con 0x5622e2ce4e50 2024-09-10T08:22:39.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.147+0000 7faed5208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5622e2bfb0f0 con 0x5622e2cda590 2024-09-10T08:22:39.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.147+0000 7faed5208740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5622e2ce7a20 con 0x5622e2cd9c70 2024-09-10T08:22:39.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.147+0000 7faed5208740 10 client.0 _ll_register_callbacks cb 0x5622e2be2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:39.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.147+0000 7faecbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cd9c70 0x5622e2ce1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.147+0000 7faecbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cd9c70 0x5622e2ce1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34696/0 (socket says 172.21.15.2:34696) 2024-09-10T08:22:39.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.147+0000 7faecbe00640 1 -- 172.21.15.2:0/488338382 learned_addr learned my addr 172.21.15.2:0/488338382 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:39.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.147+0000 7faecbe00640 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.147+0000 7faec2a00640 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cda590 0x5622e2ce2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.148+0000 7faecbe00640 1 -- 172.21.15.2:0/488338382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cd9c70 msgr2=0x5622e2ce1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:39.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.148+0000 7faecbe00640 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cd9c70 0x5622e2ce1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.148+0000 7faecbe00640 1 -- 172.21.15.2:0/488338382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cda590 msgr2=0x5622e2ce2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:39.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.148+0000 7faecbe00640 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cda590 0x5622e2ce2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.148+0000 7faecbe00640 1 -- 172.21.15.2:0/488338382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e29b13c0 con 0x5622e2ce4e50 2024-09-10T08:22:39.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.148+0000 7faec2a00640 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cda590 0x5622e2ce2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:39.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.148+0000 7faecbe00640 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce27b0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7faec4006010 tx=0x7faec402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68372fc45d9df04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:39.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.148+0000 7faeb9600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:22:39.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.148+0000 7faeb9600640 1 -- 172.21.15.2:0/488338382 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec402f610 con 0x5622e2ce4e50 2024-09-10T08:22:39.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.148+0000 7faed5208740 1 -- 172.21.15.2:0/488338382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec4036040 con 0x5622e2ce4e50 2024-09-10T08:22:39.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.148+0000 7faeb9600640 1 -- 172.21.15.2:0/488338382 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faec402f7b0 con 0x5622e2ce4e50 2024-09-10T08:22:39.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.149+0000 7faeb9600640 1 -- 172.21.15.2:0/488338382 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faec402f9b0 con 0x5622e2ce4e50 2024-09-10T08:22:39.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.149+0000 7faed5208740 10 client.19569 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:39.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.149+0000 7faed5208740 1 -- 172.21.15.2:0/488338382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5622e2ce78a0 con 0x5622e2ce4e50 2024-09-10T08:22:39.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.149+0000 7faeb9600640 1 -- 172.21.15.2:0/488338382 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec4036040 con 0x5622e2ce4e50 2024-09-10T08:22:39.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.149+0000 7faeb9600640 1 -- 172.21.15.2:0/488338382 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faec404d070 con 0x5622e2ce4e50 2024-09-10T08:22:39.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.149+0000 7faed5208740 10 client.19569 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:39.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.149+0000 7faed5208740 10 client.19569 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:22:39.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.149+0000 7faed5208740 20 client.19569 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:39.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.149+0000 7faed5208740 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5622e2cf4a60 0x5622e2cf6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.149+0000 7faed5208740 4 client.19569 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:22:39.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.149+0000 7faed5208740 1 -- 172.21.15.2:0/488338382 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622e2c4d950 con 0x5622e2cf4a60 2024-09-10T08:22:39.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.150+0000 7faec2a00640 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5622e2cf4a60 0x5622e2cf6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.151+0000 7faec2a00640 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5622e2cf4a60 0x5622e2cf6e50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72d192a1ee9f06d9 server_cookie=e978b90dc39b2805 in_seq=0 out_seq=0 2024-09-10T08:22:39.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.151+0000 7faeb9600640 10 client.19569 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:22:39.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.151+0000 7faeb9600640 1 -- 172.21.15.2:0/488338382 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622e2c4d950 con 0x5622e2cf4a60 2024-09-10T08:22:39.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.151+0000 7faeb9600640 10 client.19569 handle_command_reply: tid=0 2024-09-10T08:22:39.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.151+0000 7faed5208740 1 client.19569 shutdown 2024-09-10T08:22:39.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.152+0000 7faed5208740 1 -- 172.21.15.2:0/488338382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 msgr2=0x5622e2ce27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:39.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.152+0000 7faed5208740 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce27b0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7faec4006010 tx=0x7faec402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.152+0000 7faed5208740 1 -- 172.21.15.2:0/488338382 shutdown_connections 2024-09-10T08:22:39.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.152+0000 7faed5208740 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5622e2cf4a60 0x5622e2cf6e50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.152+0000 7faed5208740 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce27b0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.152+0000 7faed5208740 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cda590 0x5622e2ce2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.152+0000 7faeb9600640 0 client.19569 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:22:39.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.152+0000 7faed5208740 1 --2- 172.21.15.2:0/488338382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cd9c70 0x5622e2ce1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.152+0000 7faed5208740 1 -- 172.21.15.2:0/488338382 >> 172.21.15.2:0/488338382 conn(0x5622e2cd6200 msgr2=0x5622e2cd5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:39.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.152+0000 7faed5208740 1 -- 172.21.15.2:0/488338382 shutdown_connections 2024-09-10T08:22:39.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.152+0000 7faed5208740 1 -- 172.21.15.2:0/488338382 wait complete. 2024-09-10T08:22:39.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.152+0000 7faed5208740 20 client.19569 trim_cache size 0 max 16384 2024-09-10T08:22:39.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.166+0000 7faed5208740 1 Processor -- start 2024-09-10T08:22:39.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.166+0000 7faed5208740 1 -- start start 2024-09-10T08:22:39.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.166+0000 7faed5208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cd9c70 0x5622e2ce4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.166+0000 7faed5208740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cda590 0x5622e2cfc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.166+0000 7faed5208740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2cfcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.166+0000 7faed5208740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b3820 con 0x5622e2ce4e50 2024-09-10T08:22:39.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.166+0000 7faed5208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b10f0 con 0x5622e2cd9c70 2024-09-10T08:22:39.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.166+0000 7faed5208740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1aef40 con 0x5622e2cda590 2024-09-10T08:22:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.167+0000 7faecbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cd9c70 0x5622e2ce4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.167+0000 7faecbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2cfcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.167+0000 7faec2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cda590 0x5622e2cfc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.167+0000 7faecbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2cfcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46084/0 (socket says 172.21.15.2:46084) 2024-09-10T08:22:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.167+0000 7faecbe00640 1 -- 172.21.15.2:0/2061800276 learned_addr learned my addr 172.21.15.2:0/2061800276 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.167+0000 7faec2a00640 1 -- 172.21.15.2:0/2061800276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cd9c70 msgr2=0x5622e2ce4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.167+0000 7faec2a00640 1 --2- 172.21.15.2:0/2061800276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cd9c70 0x5622e2ce4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.167+0000 7faec2a00640 1 -- 172.21.15.2:0/2061800276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 msgr2=0x5622e2cfcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.167+0000 7faec2a00640 1 --2- 172.21.15.2:0/2061800276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2cfcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.167+0000 7faec2a00640 1 -- 172.21.15.2:0/2061800276 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e2cd1770 con 0x5622e2cda590 2024-09-10T08:22:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.167+0000 7faecbe00640 1 --2- 172.21.15.2:0/2061800276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2cfcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.168+0000 7faec2a00640 1 --2- 172.21.15.2:0/2061800276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cda590 0x5622e2cfc6b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x5622e2ce32e0 tx=0x7faeb4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88300a207313e622 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.168+0000 7faebbe00640 1 -- 172.21.15.2:0/2061800276 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4005020 con 0x5622e2cda590 2024-09-10T08:22:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.168+0000 7faebbe00640 1 -- 172.21.15.2:0/2061800276 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeb400f7f0 con 0x5622e2cda590 2024-09-10T08:22:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.168+0000 7faebbe00640 1 -- 172.21.15.2:0/2061800276 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4004a90 con 0x5622e2cda590 2024-09-10T08:22:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.168+0000 7faed5208740 1 -- 172.21.15.2:0/2061800276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cda590 msgr2=0x5622e2cfc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.168+0000 7faed5208740 1 --2- 172.21.15.2:0/2061800276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cda590 0x5622e2cfc6b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x5622e2ce32e0 tx=0x7faeb4005f90 comp rx=0 tx=0).stop 2024-09-10T08:22:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.169+0000 7faed5208740 1 -- 172.21.15.2:0/2061800276 shutdown_connections 2024-09-10T08:22:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.169+0000 7faed5208740 1 --2- 172.21.15.2:0/2061800276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2cfcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.169+0000 7faed5208740 1 --2- 172.21.15.2:0/2061800276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cda590 0x5622e2cfc6b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.169+0000 7faed5208740 1 --2- 172.21.15.2:0/2061800276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cd9c70 0x5622e2ce4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.169+0000 7faed5208740 1 -- 172.21.15.2:0/2061800276 >> 172.21.15.2:0/2061800276 conn(0x5622e2cd6200 msgr2=0x5622e2cd5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.169+0000 7faed5208740 1 -- 172.21.15.2:0/2061800276 shutdown_connections 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.169+0000 7faed5208740 1 -- 172.21.15.2:0/2061800276 wait complete. 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.169+0000 7faed5208740 1 -- start start 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.169+0000 7faed5208740 1 Processor -- start 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.170+0000 7faed5208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cd9c70 0x5622e2cecff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.170+0000 7faed5208740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cda590 0x5622e2ced530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.170+0000 7faed5208740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.170+0000 7faed5208740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1af5b0 con 0x5622e2ce4e50 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.170+0000 7faed5208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b0a20 con 0x5622e2cd9c70 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.170+0000 7faed5208740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1afc80 con 0x5622e2cda590 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.170+0000 7faed5208740 10 client.0 _ll_register_callbacks cb 0x7faebc0d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.170+0000 7faecbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.170+0000 7faec2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cda590 0x5622e2ced530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.170+0000 7faec2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cda590 0x5622e2ced530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34724/0 (socket says 172.21.15.2:34724) 2024-09-10T08:22:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.170+0000 7faec2a00640 1 -- 172.21.15.2:0/724942785 learned_addr learned my addr 172.21.15.2:0/724942785 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.170+0000 7faecbe00640 1 --2- 172.21.15.2:0/724942785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cd9c70 0x5622e2cecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.171+0000 7faecbe00640 1 -- 172.21.15.2:0/724942785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cda590 msgr2=0x5622e2ced530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.171+0000 7faecbe00640 1 --2- 172.21.15.2:0/724942785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cda590 0x5622e2ced530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.171+0000 7faecbe00640 1 -- 172.21.15.2:0/724942785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cd9c70 msgr2=0x5622e2cecff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.171+0000 7faecbe00640 1 --2- 172.21.15.2:0/724942785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cd9c70 0x5622e2cecff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.171+0000 7faecbe00640 1 -- 172.21.15.2:0/724942785 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e2ce8da0 con 0x5622e2ce4e50 2024-09-10T08:22:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.171+0000 7faecbe00640 1 --2- 172.21.15.2:0/724942785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce8570 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7faec4030e30 tx=0x7faec4031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e106ad4578c7d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.171+0000 7faeb9600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:22:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.171+0000 7faeb9600640 1 -- 172.21.15.2:0/724942785 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec402f610 con 0x5622e2ce4e50 2024-09-10T08:22:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.171+0000 7faeb9600640 1 -- 172.21.15.2:0/724942785 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faec4002030 con 0x5622e2ce4e50 2024-09-10T08:22:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.171+0000 7faed5208740 1 -- 172.21.15.2:0/724942785 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeb400f7f0 con 0x5622e2ce4e50 2024-09-10T08:22:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.171+0000 7faeb9600640 1 -- 172.21.15.2:0/724942785 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faec402f7b0 con 0x5622e2ce4e50 2024-09-10T08:22:39.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.172+0000 7faed5208740 10 client.19575 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:39.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.172+0000 7faed5208740 1 -- 172.21.15.2:0/724942785 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5622e2d5f2e0 con 0x5622e2ce4e50 2024-09-10T08:22:39.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.172+0000 7faeb9600640 1 -- 172.21.15.2:0/724942785 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeb400f7f0 con 0x5622e2ce4e50 2024-09-10T08:22:39.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.172+0000 7faeb9600640 1 -- 172.21.15.2:0/724942785 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faec404d040 con 0x5622e2ce4e50 2024-09-10T08:22:39.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.172+0000 7faed5208740 10 client.19575 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:39.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.172+0000 7faed5208740 10 client.19575 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:22:39.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.172+0000 7faed5208740 20 client.19575 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:39.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.172+0000 7faed5208740 1 --2- 172.21.15.2:0/724942785 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5622e2d5f9a0 0x5622e2d61dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:39.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.172+0000 7faed5208740 4 client.19575 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:22:39.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.172+0000 7faed5208740 1 -- 172.21.15.2:0/724942785 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622e2c47e60 con 0x5622e2d5f9a0 2024-09-10T08:22:39.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.173+0000 7faec2a00640 1 --2- 172.21.15.2:0/724942785 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5622e2d5f9a0 0x5622e2d61dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:39.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.173+0000 7faec2a00640 1 --2- 172.21.15.2:0/724942785 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5622e2d5f9a0 0x5622e2d61dc0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18d4615e5b37dbf0 server_cookie=e0ac35c2114d88ac in_seq=0 out_seq=0 2024-09-10T08:22:39.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.173+0000 7faeb9600640 10 client.19575 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:22:39.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.173+0000 7faeb9600640 1 -- 172.21.15.2:0/724942785 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622e2c47e60 con 0x5622e2d5f9a0 2024-09-10T08:22:39.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.173+0000 7faeb9600640 10 client.19575 handle_command_reply: tid=0 2024-09-10T08:22:39.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.173+0000 7faed5208740 1 client.19575 shutdown 2024-09-10T08:22:39.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.174+0000 7faed5208740 1 -- 172.21.15.2:0/724942785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 msgr2=0x5622e2ce8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:39.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.174+0000 7faed5208740 1 --2- 172.21.15.2:0/724942785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce8570 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7faec4030e30 tx=0x7faec4031970 comp rx=0 tx=0).stop 2024-09-10T08:22:39.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.174+0000 7faed5208740 1 -- 172.21.15.2:0/724942785 shutdown_connections 2024-09-10T08:22:39.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.174+0000 7faed5208740 1 --2- 172.21.15.2:0/724942785 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5622e2d5f9a0 0x5622e2d61dc0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.174+0000 7faed5208740 1 --2- 172.21.15.2:0/724942785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622e2ce4e50 0x5622e2ce8570 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.174+0000 7faed5208740 1 --2- 172.21.15.2:0/724942785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622e2cda590 0x5622e2ced530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.174+0000 7faed5208740 1 --2- 172.21.15.2:0/724942785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e2cd9c70 0x5622e2cecff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.174+0000 7faeb9600640 0 client.19575 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:22:39.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.174+0000 7faed5208740 1 -- 172.21.15.2:0/724942785 >> 172.21.15.2:0/724942785 conn(0x5622e2cd6200 msgr2=0x5622e2ce1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:39.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.174+0000 7faed5208740 1 -- 172.21.15.2:0/724942785 shutdown_connections 2024-09-10T08:22:39.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.174+0000 7faed5208740 1 -- 172.21.15.2:0/724942785 wait complete. 2024-09-10T08:22:39.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.174+0000 7faed5208740 20 client.19575 trim_cache size 0 max 16384 2024-09-10T08:22:39.179 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:22:39.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.175+0000 7faed2600640 1 -- 172.21.15.2:0/4266798844 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faeb00a2b80 msgr2=0x7faeb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:39.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.175+0000 7faed2600640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faeb00a2b80 0x7faeb00a5040 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faebc00c6a0 tx=0x7faebc0d5530 comp rx=0 tx=0).stop 2024-09-10T08:22:39.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.175+0000 7faed2600640 1 -- 172.21.15.2:0/4266798844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faecc103ff0 msgr2=0x7faecc19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:39.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.175+0000 7faed2600640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faecc103ff0 0x7faecc19a5b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7faebc00e9f0 tx=0x7faebc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.175+0000 7faed2600640 1 -- 172.21.15.2:0/4266798844 shutdown_connections 2024-09-10T08:22:39.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.175+0000 7faed2600640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc109120 0x7faecc19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.175+0000 7faed2600640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faeb00a2b80 0x7faeb00a5040 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.175+0000 7faed2600640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faecc1049a0 0x7faecc19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.175+0000 7faed2600640 1 --2- 172.21.15.2:0/4266798844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faecc103ff0 0x7faecc19a5b0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:39.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.175+0000 7faed2600640 1 -- 172.21.15.2:0/4266798844 >> 172.21.15.2:0/4266798844 conn(0x7faecc0ffa90 msgr2=0x7faecc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:39.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.175+0000 7faed2600640 1 -- 172.21.15.2:0/4266798844 shutdown_connections 2024-09-10T08:22:39.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:39.175+0000 7faed2600640 1 -- 172.21.15.2:0/4266798844 wait complete. 2024-09-10T08:22:39.621 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:39 smithi002 ceph-mon[30563]: pgmap v822: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:22:39.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:39 smithi155 ceph-mon[35405]: pgmap v822: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:22:39.668 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:39 smithi107 ceph-mon[35048]: pgmap v822: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-10T08:22:40.029 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:22:40.407 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:22:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.960+0000 7f4f0f200640 1 -- 172.21.15.2:0/475930625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f10077040 msgr2=0x7f4f10112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.960+0000 7f4f0f200640 1 --2- 172.21.15.2:0/475930625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f10077040 0x7f4f10112330 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009900 tx=0x7f4ef802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.960+0000 7f4f0f200640 1 -- 172.21.15.2:0/475930625 shutdown_connections 2024-09-10T08:22:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.960+0000 7f4f0f200640 1 --2- 172.21.15.2:0/475930625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f10112870 0x7f4f10114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.960+0000 7f4f0f200640 1 --2- 172.21.15.2:0/475930625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f10077040 0x7f4f10112330 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.960+0000 7f4f0f200640 1 --2- 172.21.15.2:0/475930625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f10076720 0x7f4f10076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.960+0000 7f4f0f200640 1 -- 172.21.15.2:0/475930625 >> 172.21.15.2:0/475930625 conn(0x7f4f100ff8d0 msgr2=0x7f4f10101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:40.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.960+0000 7f4f0f200640 1 -- 172.21.15.2:0/475930625 shutdown_connections 2024-09-10T08:22:40.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.960+0000 7f4f0f200640 1 -- 172.21.15.2:0/475930625 wait complete. 2024-09-10T08:22:40.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.961+0000 7f4f0f200640 1 Processor -- start 2024-09-10T08:22:40.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.961+0000 7f4f0f200640 1 -- start start 2024-09-10T08:22:40.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.962+0000 7f4f0f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f10076720 0x7f4f101a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.962+0000 7f4f0f200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f10077040 0x7f4f101a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.962+0000 7f4f0f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f10112870 0x7f4f1019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.962+0000 7f4f0f200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4f10118450 con 0x7f4f10077040 2024-09-10T08:22:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.962+0000 7f4f0f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101182d0 con 0x7f4f10112870 2024-09-10T08:22:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.962+0000 7f4f0f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101185d0 con 0x7f4f10076720 2024-09-10T08:22:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.962+0000 7f4f05400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f10077040 0x7f4f101a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.962+0000 7f4f05400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f10077040 0x7f4f101a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46096/0 (socket says 172.21.15.2:46096) 2024-09-10T08:22:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.962+0000 7f4f05400640 1 -- 172.21.15.2:0/907825841 learned_addr learned my addr 172.21.15.2:0/907825841 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.962+0000 7f4f0e800640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f10112870 0x7f4f1019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.962+0000 7f4f0de00640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f10076720 0x7f4f101a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.963+0000 7f4f05400640 1 -- 172.21.15.2:0/907825841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f10076720 msgr2=0x7f4f101a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.963+0000 7f4f05400640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f10076720 0x7f4f101a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.963+0000 7f4f05400640 1 -- 172.21.15.2:0/907825841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f10112870 msgr2=0x7f4f1019e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.963+0000 7f4f05400640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f10112870 0x7f4f1019e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.963+0000 7f4f05400640 1 -- 172.21.15.2:0/907825841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1019efa0 con 0x7f4f10077040 2024-09-10T08:22:40.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.963+0000 7f4f0e800640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f10112870 0x7f4f1019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:40.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.963+0000 7f4f05400640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f10077040 0x7f4f101a4ba0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8004770 tx=0x7f4ef8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:40.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.963+0000 7f4f07400640 1 -- 172.21.15.2:0/907825841 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef803f070 con 0x7f4f10077040 2024-09-10T08:22:40.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.964+0000 7f4f07400640 1 -- 172.21.15.2:0/907825841 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef8031ca0 con 0x7f4f10077040 2024-09-10T08:22:40.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.964+0000 7f4f0f200640 1 -- 172.21.15.2:0/907825841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1019f230 con 0x7f4f10077040 2024-09-10T08:22:40.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.964+0000 7f4f07400640 1 -- 172.21.15.2:0/907825841 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8035030 con 0x7f4f10077040 2024-09-10T08:22:40.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.964+0000 7f4f0f200640 1 -- 172.21.15.2:0/907825841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f101ab410 con 0x7f4f10077040 2024-09-10T08:22:40.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.967+0000 7f4f07400640 1 -- 172.21.15.2:0/907825841 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4ef8042070 con 0x7f4f10077040 2024-09-10T08:22:40.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.967+0000 7f4f17a1f740 1 Processor -- start 2024-09-10T08:22:40.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.967+0000 7f4f17a1f740 1 -- start start 2024-09-10T08:22:40.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.967+0000 7f4f17a1f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861ae40 0x55cf0861b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.967+0000 7f4f17a1f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf0861b760 0x55cf08625ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.967+0000 7f4f17a1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf08626020 0x55cf08628410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.967+0000 7f4f17a1f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cf085813e0 con 0x55cf0861b760 2024-09-10T08:22:40.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.967+0000 7f4f17a1f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cf08528d80 con 0x55cf0861ae40 2024-09-10T08:22:40.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.967+0000 7f4f17a1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf0857d3a0 con 0x55cf08626020 2024-09-10T08:22:40.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.968+0000 7f4f0de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf0861b760 0x55cf08625ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.968+0000 7f4f0de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf0861b760 0x55cf08625ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46098/0 (socket says 172.21.15.2:46098) 2024-09-10T08:22:40.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.968+0000 7f4f0e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861ae40 0x55cf0861b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.968+0000 7f4f0de00640 1 -- 172.21.15.2:0/793129372 learned_addr learned my addr 172.21.15.2:0/793129372 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:40.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.968+0000 7f4f05400640 1 --2- 172.21.15.2:0/793129372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf08626020 0x55cf08628410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.968+0000 7f4f0e800640 1 -- 172.21.15.2:0/793129372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf08626020 msgr2=0x55cf08628410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:40.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.968+0000 7f4f0e800640 1 --2- 172.21.15.2:0/793129372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf08626020 0x55cf08628410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.968+0000 7f4f0e800640 1 -- 172.21.15.2:0/793129372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf0861b760 msgr2=0x55cf08625ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:40.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.968+0000 7f4f0e800640 1 --2- 172.21.15.2:0/793129372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf0861b760 0x55cf08625ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.968+0000 7f4f0e800640 1 -- 172.21.15.2:0/793129372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf08547ae0 con 0x55cf0861ae40 2024-09-10T08:22:40.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.969+0000 7f4f0e800640 1 --2- 172.21.15.2:0/793129372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861ae40 0x55cf0861b220 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4f1007aa10 tx=0x7f4f08011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0ee792aecf36e3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:40.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.969+0000 7f4f07400640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4ef40a2b80 0x7f4ef40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.969+0000 7f4eff400640 1 -- 172.21.15.2:0/793129372 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08004310 con 0x55cf0861ae40 2024-09-10T08:22:40.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.969+0000 7f4f07400640 1 -- 172.21.15.2:0/907825841 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4ef80ef6f0 con 0x7f4f10077040 2024-09-10T08:22:40.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.969+0000 7f4eff400640 1 -- 172.21.15.2:0/793129372 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f080044b0 con 0x55cf0861ae40 2024-09-10T08:22:40.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.969+0000 7f4eff400640 1 -- 172.21.15.2:0/793129372 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f080035f0 con 0x55cf0861ae40 2024-09-10T08:22:40.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.969+0000 7f4f0de00640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4ef40a2b80 0x7f4ef40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.970+0000 7f4f17a1f740 1 -- 172.21.15.2:0/793129372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861ae40 msgr2=0x55cf0861b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:40.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.970+0000 7f4f0de00640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4ef40a2b80 0x7f4ef40a5040 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x55cf08618230 tx=0x7f4f00009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:40.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.970+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/793129372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861ae40 0x55cf0861b220 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4f1007aa10 tx=0x7f4f08011810 comp rx=0 tx=0).stop 2024-09-10T08:22:40.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.970+0000 7f4f17a1f740 1 -- 172.21.15.2:0/793129372 shutdown_connections 2024-09-10T08:22:40.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.970+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/793129372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf08626020 0x55cf08628410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.970+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/793129372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf0861b760 0x55cf08625ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.970+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/793129372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861ae40 0x55cf0861b220 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.970+0000 7f4f17a1f740 1 -- 172.21.15.2:0/793129372 >> 172.21.15.2:0/793129372 conn(0x55cf086173d0 msgr2=0x55cf086177b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:40.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.970+0000 7f4f17a1f740 1 -- 172.21.15.2:0/793129372 shutdown_connections 2024-09-10T08:22:40.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.970+0000 7f4f17a1f740 1 -- 172.21.15.2:0/793129372 wait complete. 2024-09-10T08:22:40.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.970+0000 7f4f17a1f740 1 -- start start 2024-09-10T08:22:40.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.971+0000 7f4f17a1f740 1 Processor -- start 2024-09-10T08:22:40.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.971+0000 7f4f17a1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861ae40 0x55cf08622e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.971+0000 7f4f17a1f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861b760 0x55cf086233c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.971+0000 7f4f17a1f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf08626020 0x55cf08623900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.971+0000 7f4f17a1f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55cf08628bf0 con 0x55cf08626020 2024-09-10T08:22:40.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.971+0000 7f4f17a1f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cf0853c0f0 con 0x55cf0861b760 2024-09-10T08:22:40.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.971+0000 7f4f17a1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf08628d70 con 0x55cf0861ae40 2024-09-10T08:22:40.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.971+0000 7f4f17a1f740 10 client.0 _ll_register_callbacks cb 0x55cf08523bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:40.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.971+0000 7f4f0e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf08626020 0x55cf08623900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.972+0000 7f4f05400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861b760 0x55cf086233c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.972+0000 7f4f0de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861ae40 0x55cf08622e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.972+0000 7f4f0e800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf08626020 0x55cf08623900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46112/0 (socket says 172.21.15.2:46112) 2024-09-10T08:22:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.972+0000 7f4f0e800640 1 -- 172.21.15.2:0/774695765 learned_addr learned my addr 172.21.15.2:0/774695765 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.972+0000 7f4f05400640 1 -- 172.21.15.2:0/774695765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861ae40 msgr2=0x55cf08622e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.972+0000 7f4f05400640 1 --2- 172.21.15.2:0/774695765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861ae40 0x55cf08622e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.972+0000 7f4f05400640 1 -- 172.21.15.2:0/774695765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf08626020 msgr2=0x55cf08623900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.972+0000 7f4f05400640 1 --2- 172.21.15.2:0/774695765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf08626020 0x55cf08623900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.972+0000 7f4f05400640 1 -- 172.21.15.2:0/774695765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf082f23c0 con 0x55cf0861b760 2024-09-10T08:22:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.972+0000 7f4f0e800640 1 --2- 172.21.15.2:0/774695765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf08626020 0x55cf08623900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.972+0000 7f4f05400640 1 --2- 172.21.15.2:0/774695765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861b760 0x55cf086233c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55cf08618750 tx=0x7f4ef80b7fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aec2b310260e935 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.972+0000 7f4ef3e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:22:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.972+0000 7f4ef3e00640 1 -- 172.21.15.2:0/774695765 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8049070 con 0x55cf0861b760 2024-09-10T08:22:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.973+0000 7f4f17a1f740 1 -- 172.21.15.2:0/774695765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f080044b0 con 0x55cf0861b760 2024-09-10T08:22:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.973+0000 7f4ef3e00640 1 -- 172.21.15.2:0/774695765 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4ef80ef360 con 0x55cf0861b760 2024-09-10T08:22:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.973+0000 7f4ef3e00640 1 -- 172.21.15.2:0/774695765 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ef8005d50 con 0x55cf0861b760 2024-09-10T08:22:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.973+0000 7f4f17a1f740 10 client.29612 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.973+0000 7f4f17a1f740 1 -- 172.21.15.2:0/774695765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cf08634be0 con 0x55cf0861b760 2024-09-10T08:22:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.973+0000 7f4ef3e00640 1 -- 172.21.15.2:0/774695765 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f080044b0 con 0x55cf0861b760 2024-09-10T08:22:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.973+0000 7f4ef3e00640 1 -- 172.21.15.2:0/774695765 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4ef804c060 con 0x55cf0861b760 2024-09-10T08:22:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.973+0000 7f4f17a1f740 10 client.29612 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.973+0000 7f4f17a1f740 10 client.29612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.973+0000 7f4f17a1f740 20 client.29612 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.973+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/774695765 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf08636090 0x55cf086384a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.973+0000 7f4f17a1f740 4 client.29612 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:22:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.973+0000 7f4f17a1f740 1 -- 172.21.15.2:0/774695765 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf0858e950 con 0x55cf08636090 2024-09-10T08:22:40.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.974+0000 7f4f0de00640 1 --2- 172.21.15.2:0/774695765 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf08636090 0x55cf086384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.974+0000 7f4f0de00640 1 --2- 172.21.15.2:0/774695765 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf08636090 0x55cf086384a0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=466c47eec41d2546 server_cookie=ac9dc75ea2a3bb11 in_seq=0 out_seq=0 2024-09-10T08:22:40.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.974+0000 7f4ef3e00640 10 client.29612 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:40.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.975+0000 7f4ef3e00640 1 -- 172.21.15.2:0/774695765 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf0858e950 con 0x55cf08636090 2024-09-10T08:22:40.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.975+0000 7f4ef3e00640 10 client.29612 handle_command_reply: tid=0 2024-09-10T08:22:40.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.976+0000 7f4f17a1f740 1 client.29612 shutdown 2024-09-10T08:22:40.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.976+0000 7f4f17a1f740 1 -- 172.21.15.2:0/774695765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861b760 msgr2=0x55cf086233c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:40.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.976+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/774695765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861b760 0x55cf086233c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55cf08618750 tx=0x7f4ef80b7fd0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.976+0000 7f4f17a1f740 1 -- 172.21.15.2:0/774695765 shutdown_connections 2024-09-10T08:22:40.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.976+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/774695765 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf08636090 0x55cf086384a0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.976+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/774695765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf08626020 0x55cf08623900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.976+0000 7f4ef3e00640 0 client.29612 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:40.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.976+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/774695765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861b760 0x55cf086233c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.976+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/774695765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861ae40 0x55cf08622e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.976+0000 7f4f17a1f740 1 -- 172.21.15.2:0/774695765 >> 172.21.15.2:0/774695765 conn(0x55cf086173d0 msgr2=0x55cf0861d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:40.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.976+0000 7f4f17a1f740 1 -- 172.21.15.2:0/774695765 shutdown_connections 2024-09-10T08:22:40.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.976+0000 7f4f17a1f740 1 -- 172.21.15.2:0/774695765 wait complete. 2024-09-10T08:22:40.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.976+0000 7f4f17a1f740 20 client.29612 trim_cache size 0 max 16384 2024-09-10T08:22:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.990+0000 7f4f17a1f740 1 Processor -- start 2024-09-10T08:22:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.990+0000 7f4f17a1f740 1 -- start start 2024-09-10T08:22:40.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.990+0000 7f4f17a1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861ae40 0x55cf0863dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.990+0000 7f4f17a1f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf0861b760 0x55cf0863e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.990+0000 7f4f17a1f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf08626020 0x55cf08615c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.990+0000 7f4f17a1f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101b7bb0 con 0x55cf0861b760 2024-09-10T08:22:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.990+0000 7f4f17a1f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101b5480 con 0x55cf08626020 2024-09-10T08:22:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.990+0000 7f4f17a1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101b32d0 con 0x55cf0861ae40 2024-09-10T08:22:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.991+0000 7f4f05400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf0861b760 0x55cf0863e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.991+0000 7f4f05400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf0861b760 0x55cf0863e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46114/0 (socket says 172.21.15.2:46114) 2024-09-10T08:22:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.991+0000 7f4f05400640 1 -- 172.21.15.2:0/3301709370 learned_addr learned my addr 172.21.15.2:0/3301709370 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.991+0000 7f4f0de00640 1 --2- 172.21.15.2:0/3301709370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861ae40 0x55cf0863dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.991+0000 7f4f0e800640 1 --2- 172.21.15.2:0/3301709370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf08626020 0x55cf08615c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.991+0000 7f4f05400640 1 -- 172.21.15.2:0/3301709370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861ae40 msgr2=0x55cf0863dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.991+0000 7f4f05400640 1 --2- 172.21.15.2:0/3301709370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861ae40 0x55cf0863dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.991+0000 7f4f05400640 1 -- 172.21.15.2:0/3301709370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf08626020 msgr2=0x55cf08615c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.991+0000 7f4f05400640 1 --2- 172.21.15.2:0/3301709370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf08626020 0x55cf08615c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.991+0000 7f4f05400640 1 -- 172.21.15.2:0/3301709370 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf08617200 con 0x55cf0861b760 2024-09-10T08:22:40.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.991+0000 7f4f0de00640 1 --2- 172.21.15.2:0/3301709370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861ae40 0x55cf0863dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:40.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.991+0000 7f4f05400640 1 --2- 172.21.15.2:0/3301709370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf0861b760 0x55cf0863e2b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80b8380 tx=0x7f4ef803b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f28ff88dcc1b3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:40.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.992+0000 7f4efea00640 1 -- 172.21.15.2:0/3301709370 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8049070 con 0x55cf0861b760 2024-09-10T08:22:40.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.992+0000 7f4efea00640 1 -- 172.21.15.2:0/3301709370 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef80d9030 con 0x55cf0861b760 2024-09-10T08:22:40.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.992+0000 7f4efea00640 1 -- 172.21.15.2:0/3301709370 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80c43f0 con 0x55cf0861b760 2024-09-10T08:22:40.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.992+0000 7f4f17a1f740 1 -- 172.21.15.2:0/3301709370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf0861b760 msgr2=0x55cf0863e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:40.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.992+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/3301709370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf0861b760 0x55cf0863e2b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80b8380 tx=0x7f4ef803b460 comp rx=0 tx=0).stop 2024-09-10T08:22:40.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.992+0000 7f4f17a1f740 1 -- 172.21.15.2:0/3301709370 shutdown_connections 2024-09-10T08:22:40.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.992+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/3301709370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf08626020 0x55cf08615c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.992+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/3301709370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf0861b760 0x55cf0863e2b0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.992+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/3301709370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861ae40 0x55cf0863dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:40.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.992+0000 7f4f17a1f740 1 -- 172.21.15.2:0/3301709370 >> 172.21.15.2:0/3301709370 conn(0x55cf086173d0 msgr2=0x55cf08621f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:40.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.993+0000 7f4f17a1f740 1 -- 172.21.15.2:0/3301709370 shutdown_connections 2024-09-10T08:22:40.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.993+0000 7f4f17a1f740 1 -- 172.21.15.2:0/3301709370 wait complete. 2024-09-10T08:22:40.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.993+0000 7f4f17a1f740 1 -- start start 2024-09-10T08:22:40.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.993+0000 7f4f17a1f740 1 Processor -- start 2024-09-10T08:22:40.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f17a1f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861ae40 0x55cf0862e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f17a1f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf08626020 0x55cf0862e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f17a1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861f720 0x55cf0861fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f17a1f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101b3940 con 0x55cf08626020 2024-09-10T08:22:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f17a1f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101b4db0 con 0x55cf0861ae40 2024-09-10T08:22:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f17a1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101b4010 con 0x55cf0861f720 2024-09-10T08:22:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f17a1f740 10 client.0 _ll_register_callbacks cb 0x7f4ef8009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f0de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861ae40 0x55cf0862e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f05400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf08626020 0x55cf0862e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f0de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861ae40 0x55cf0862e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40608/0 (socket says 172.21.15.2:40608) 2024-09-10T08:22:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f0de00640 1 -- 172.21.15.2:0/1421000063 learned_addr learned my addr 172.21.15.2:0/1421000063 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f0e800640 1 --2- 172.21.15.2:0/1421000063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861f720 0x55cf0861fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f0de00640 1 -- 172.21.15.2:0/1421000063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861f720 msgr2=0x55cf0861fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f0de00640 1 --2- 172.21.15.2:0/1421000063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861f720 0x55cf0861fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f0de00640 1 -- 172.21.15.2:0/1421000063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf08626020 msgr2=0x55cf0862e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f0de00640 1 --2- 172.21.15.2:0/1421000063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf08626020 0x55cf0862e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.994+0000 7f4f0de00640 1 -- 172.21.15.2:0/1421000063 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf08620370 con 0x55cf0861ae40 2024-09-10T08:22:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.995+0000 7f4f05400640 1 --2- 172.21.15.2:0/1421000063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf08626020 0x55cf0862e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.995+0000 7f4f0de00640 1 --2- 172.21.15.2:0/1421000063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861ae40 0x55cf0862e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x55cf08638ce0 tx=0x7f4f00004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d2e64e54d0b4c7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.995+0000 7f4ef3e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:22:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.995+0000 7f4ef3e00640 1 -- 172.21.15.2:0/1421000063 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00003970 con 0x55cf0861ae40 2024-09-10T08:22:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.995+0000 7f4ef3e00640 1 -- 172.21.15.2:0/1421000063 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4f00003b10 con 0x55cf0861ae40 2024-09-10T08:22:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.995+0000 7f4f17a1f740 1 -- 172.21.15.2:0/1421000063 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef80d9030 con 0x55cf0861ae40 2024-09-10T08:22:41.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.995+0000 7f4ef3e00640 1 -- 172.21.15.2:0/1421000063 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4f0003c050 con 0x55cf0861ae40 2024-09-10T08:22:41.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.996+0000 7f4f17a1f740 10 client.29618 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:41.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.996+0000 7f4f17a1f740 1 -- 172.21.15.2:0/1421000063 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cf086a0d30 con 0x55cf0861ae40 2024-09-10T08:22:41.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.996+0000 7f4ef3e00640 1 -- 172.21.15.2:0/1421000063 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef80d9030 con 0x55cf0861ae40 2024-09-10T08:22:41.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.996+0000 7f4ef3e00640 1 -- 172.21.15.2:0/1421000063 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4f0002a510 con 0x55cf0861ae40 2024-09-10T08:22:41.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.996+0000 7f4f17a1f740 10 client.29618 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:41.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.996+0000 7f4f17a1f740 10 client.29618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:41.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.996+0000 7f4f17a1f740 20 client.29618 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:41.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.996+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/1421000063 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf086a1480 0x55cf086a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:41.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.996+0000 7f4f17a1f740 4 client.29618 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:22:41.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.996+0000 7f4f17a1f740 1 -- 172.21.15.2:0/1421000063 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf08588e60 con 0x55cf086a1480 2024-09-10T08:22:41.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.996+0000 7f4f05400640 1 --2- 172.21.15.2:0/1421000063 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf086a1480 0x55cf086a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:41.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4f05400640 1 --2- 172.21.15.2:0/1421000063 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf086a1480 0x55cf086a38a0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f92142f388e3c691 server_cookie=cd8bb8ae69e81145 in_seq=0 out_seq=0 2024-09-10T08:22:41.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4ef3e00640 10 client.29618 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:41.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4ef3e00640 1 -- 172.21.15.2:0/1421000063 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf08588e60 con 0x55cf086a1480 2024-09-10T08:22:41.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4ef3e00640 10 client.29618 handle_command_reply: tid=0 2024-09-10T08:22:41.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4f17a1f740 1 client.29618 shutdown 2024-09-10T08:22:41.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4f17a1f740 1 -- 172.21.15.2:0/1421000063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861ae40 msgr2=0x55cf0862e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:41.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/1421000063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861ae40 0x55cf0862e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x55cf08638ce0 tx=0x7f4f00004a40 comp rx=0 tx=0).stop 2024-09-10T08:22:41.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4f17a1f740 1 -- 172.21.15.2:0/1421000063 shutdown_connections 2024-09-10T08:22:41.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/1421000063 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55cf086a1480 0x55cf086a38a0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:41.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/1421000063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf0861f720 0x55cf0861fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:41.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/1421000063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf08626020 0x55cf0862e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:41.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4f17a1f740 1 --2- 172.21.15.2:0/1421000063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf0861ae40 0x55cf0862e2f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:41.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4ef3e00640 0 client.29618 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:41.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.997+0000 7f4f17a1f740 1 -- 172.21.15.2:0/1421000063 >> 172.21.15.2:0/1421000063 conn(0x55cf086173d0 msgr2=0x55cf08615c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:41.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.998+0000 7f4f17a1f740 1 -- 172.21.15.2:0/1421000063 shutdown_connections 2024-09-10T08:22:41.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.998+0000 7f4f17a1f740 1 -- 172.21.15.2:0/1421000063 wait complete. 2024-09-10T08:22:41.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.998+0000 7f4f17a1f740 20 client.29618 trim_cache size 0 max 16384 2024-09-10T08:22:41.003 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:22:41.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.998+0000 7f4f0f200640 1 -- 172.21.15.2:0/907825841 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4ef40a2b80 msgr2=0x7f4ef40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:41.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.998+0000 7f4f0f200640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4ef40a2b80 0x7f4ef40a5040 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x55cf08618230 tx=0x7f4f00009500 comp rx=0 tx=0).stop 2024-09-10T08:22:41.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.998+0000 7f4f0f200640 1 -- 172.21.15.2:0/907825841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f10077040 msgr2=0x7f4f101a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:41.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.998+0000 7f4f0f200640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f10077040 0x7f4f101a4ba0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8004770 tx=0x7f4ef8003c10 comp rx=0 tx=0).stop 2024-09-10T08:22:41.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.998+0000 7f4f0f200640 1 -- 172.21.15.2:0/907825841 shutdown_connections 2024-09-10T08:22:41.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.998+0000 7f4f0f200640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f10112870 0x7f4f1019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:41.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.998+0000 7f4f0f200640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f10077040 0x7f4f101a4ba0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:41.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.998+0000 7f4f0f200640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4ef40a2b80 0x7f4ef40a5040 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:41.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.998+0000 7f4f0f200640 1 --2- 172.21.15.2:0/907825841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f10076720 0x7f4f101a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:41.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.998+0000 7f4f0f200640 1 -- 172.21.15.2:0/907825841 >> 172.21.15.2:0/907825841 conn(0x7f4f100ff8d0 msgr2=0x7f4f10101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:41.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.999+0000 7f4f0f200640 1 -- 172.21.15.2:0/907825841 shutdown_connections 2024-09-10T08:22:41.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:40.999+0000 7f4f0f200640 1 -- 172.21.15.2:0/907825841 wait complete. 2024-09-10T08:22:41.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:41 smithi155 ceph-mon[35405]: pgmap v823: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:22:41.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:41 smithi107 ceph-mon[35048]: pgmap v823: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:22:41.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:41 smithi002 ceph-mon[30563]: pgmap v823: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:22:42.948 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:22:42.948 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:22:43.339 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:22:43.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:43 smithi155 ceph-mon[35405]: pgmap v824: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:22:43.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:43 smithi107 ceph-mon[35048]: pgmap v824: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:22:43.681 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:43 smithi002 ceph-mon[30563]: pgmap v824: 129 pgs: 129 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:22:43.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.988+0000 7f0309e00640 1 -- 172.21.15.2:0/85476235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03041049a0 msgr2=0x7f03041089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:43.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.988+0000 7f0309e00640 1 --2- 172.21.15.2:0/85476235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03041049a0 0x7f03041089f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009900 tx=0x7f02f002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:22:43.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.988+0000 7f0309e00640 1 -- 172.21.15.2:0/85476235 shutdown_connections 2024-09-10T08:22:43.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.988+0000 7f0309e00640 1 --2- 172.21.15.2:0/85476235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0304109120 0x7f0304110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:43.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.988+0000 7f0309e00640 1 --2- 172.21.15.2:0/85476235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03041049a0 0x7f03041089f0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:43.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.988+0000 7f0309e00640 1 --2- 172.21.15.2:0/85476235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0304103ff0 0x7f03041043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:43.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.988+0000 7f0309e00640 1 -- 172.21.15.2:0/85476235 >> 172.21.15.2:0/85476235 conn(0x7f03040ffa90 msgr2=0x7f0304101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:43.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.988+0000 7f0309e00640 1 -- 172.21.15.2:0/85476235 shutdown_connections 2024-09-10T08:22:43.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.988+0000 7f0309e00640 1 -- 172.21.15.2:0/85476235 wait complete. 2024-09-10T08:22:43.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.989+0000 7f0309e00640 1 Processor -- start 2024-09-10T08:22:43.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.989+0000 7f0309e00640 1 -- start start 2024-09-10T08:22:43.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.990+0000 7f0309e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0304103ff0 0x7f030419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:43.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.990+0000 7f0309e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03041049a0 0x7f030419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:43.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.990+0000 7f0309e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0304109120 0x7f030419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:43.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.990+0000 7f0309e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0304114140 con 0x7f0304109120 2024-09-10T08:22:43.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.990+0000 7f0309e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0304113fc0 con 0x7f0304103ff0 2024-09-10T08:22:43.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.990+0000 7f0309e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03041142c0 con 0x7f03041049a0 2024-09-10T08:22:43.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.990+0000 7f0302a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0304103ff0 0x7f030419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:43.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.990+0000 7f0302000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03041049a0 0x7f030419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:43.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.990+0000 7f0303400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0304109120 0x7f030419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:43.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.990+0000 7f0302a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0304103ff0 0x7f030419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40622/0 (socket says 172.21.15.2:40622) 2024-09-10T08:22:43.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.990+0000 7f0302a00640 1 -- 172.21.15.2:0/271875829 learned_addr learned my addr 172.21.15.2:0/271875829 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:43.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.991+0000 7f0302a00640 1 -- 172.21.15.2:0/271875829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03041049a0 msgr2=0x7f030419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:43.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.991+0000 7f0302a00640 1 --2- 172.21.15.2:0/271875829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03041049a0 0x7f030419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:43.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.991+0000 7f0302a00640 1 -- 172.21.15.2:0/271875829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0304109120 msgr2=0x7f030419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:43.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.991+0000 7f0302a00640 1 --2- 172.21.15.2:0/271875829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0304109120 0x7f030419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:43.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.991+0000 7f0302a00640 1 -- 172.21.15.2:0/271875829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030419f650 con 0x7f0304103ff0 2024-09-10T08:22:43.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.991+0000 7f0302a00640 1 --2- 172.21.15.2:0/271875829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0304103ff0 0x7f030419a6a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f02f400e9f0 tx=0x7f02f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:43.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.991+0000 7f0308a00640 1 -- 172.21.15.2:0/271875829 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f400cd80 con 0x7f0304103ff0 2024-09-10T08:22:43.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.991+0000 7f0309e00640 1 -- 172.21.15.2:0/271875829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030419f940 con 0x7f0304103ff0 2024-09-10T08:22:43.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.991+0000 7f0308a00640 1 -- 172.21.15.2:0/271875829 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f4010910 con 0x7f0304103ff0 2024-09-10T08:22:43.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.991+0000 7f0308a00640 1 -- 172.21.15.2:0/271875829 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f40188f0 con 0x7f0304103ff0 2024-09-10T08:22:43.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.991+0000 7f0309e00640 1 -- 172.21.15.2:0/271875829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03041a7220 con 0x7f0304103ff0 2024-09-10T08:22:43.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.994+0000 7f030ca16740 1 Processor -- start 2024-09-10T08:22:43.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.994+0000 7f030ca16740 1 -- start start 2024-09-10T08:22:43.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.995+0000 7f0308a00640 1 -- 172.21.15.2:0/271875829 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f02f4010ac0 con 0x7f0304103ff0 2024-09-10T08:22:43.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.995+0000 7f030ca16740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c1ffac0 0x55ae3c1ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:43.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.995+0000 7f030ca16740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c20a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:43.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.995+0000 7f030ca16740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 0x55ae3c20d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:43.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.995+0000 7f030ca16740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ae3c166290 con 0x55ae3c2003e0 2024-09-10T08:22:43.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.995+0000 7f030ca16740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ae3c10db40 con 0x55ae3c1ffac0 2024-09-10T08:22:43.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.995+0000 7f030ca16740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ae3c162250 con 0x55ae3c20aca0 2024-09-10T08:22:43.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.995+0000 7f0302a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c20a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:43.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.995+0000 7f0302a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c20a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46158/0 (socket says 172.21.15.2:46158) 2024-09-10T08:22:43.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.995+0000 7f0302a00640 1 -- 172.21.15.2:0/3715103493 learned_addr learned my addr 172.21.15.2:0/3715103493 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:43.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.995+0000 7f0303400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c1ffac0 0x55ae3c1ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:43.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.995+0000 7f0302000640 1 --2- 172.21.15.2:0/3715103493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 0x55ae3c20d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:43.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.996+0000 7f0303400640 1 -- 172.21.15.2:0/3715103493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 msgr2=0x55ae3c20d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:43.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.996+0000 7f0303400640 1 --2- 172.21.15.2:0/3715103493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 0x55ae3c20d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.996+0000 7f0303400640 1 -- 172.21.15.2:0/3715103493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 msgr2=0x55ae3c20a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.996+0000 7f0303400640 1 --2- 172.21.15.2:0/3715103493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c20a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.996+0000 7f0303400640 1 -- 172.21.15.2:0/3715103493 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae3bed76a0 con 0x55ae3c1ffac0 2024-09-10T08:22:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.996+0000 7f0302000640 1 --2- 172.21.15.2:0/3715103493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 0x55ae3c20d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:22:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.996+0000 7f0303400640 1 --2- 172.21.15.2:0/3715103493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c1ffac0 0x55ae3c1ffea0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009f00 tx=0x7f02fc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51741f9a77be8c4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.996+0000 7f02efe00640 1 -- 172.21.15.2:0/3715103493 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc004d20 con 0x55ae3c1ffac0 2024-09-10T08:22:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.996+0000 7f02efe00640 1 -- 172.21.15.2:0/3715103493 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02fc004050 con 0x55ae3c1ffac0 2024-09-10T08:22:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.996+0000 7f02efe00640 1 -- 172.21.15.2:0/3715103493 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc005280 con 0x55ae3c1ffac0 2024-09-10T08:22:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.996+0000 7f0308a00640 1 --2- 172.21.15.2:0/271875829 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02e80a2b80 0x7f02e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.997+0000 7f0308a00640 1 -- 172.21.15.2:0/271875829 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f02f40cf820 con 0x7f0304103ff0 2024-09-10T08:22:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.997+0000 7f030ca16740 1 -- 172.21.15.2:0/3715103493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c1ffac0 msgr2=0x55ae3c1ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.997+0000 7f030ca16740 1 --2- 172.21.15.2:0/3715103493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c1ffac0 0x55ae3c1ffea0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009f00 tx=0x7f02fc011810 comp rx=0 tx=0).stop 2024-09-10T08:22:43.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.997+0000 7f0302a00640 1 --2- 172.21.15.2:0/271875829 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02e80a2b80 0x7f02e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:43.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.997+0000 7f030ca16740 1 -- 172.21.15.2:0/3715103493 shutdown_connections 2024-09-10T08:22:43.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.997+0000 7f030ca16740 1 --2- 172.21.15.2:0/3715103493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 0x55ae3c20d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:43.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.997+0000 7f030ca16740 1 --2- 172.21.15.2:0/3715103493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c20a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:43.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.997+0000 7f030ca16740 1 --2- 172.21.15.2:0/3715103493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c1ffac0 0x55ae3c1ffea0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:43.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.997+0000 7f030ca16740 1 -- 172.21.15.2:0/3715103493 >> 172.21.15.2:0/3715103493 conn(0x55ae3c1fc050 msgr2=0x55ae3c1fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:44.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.998+0000 7f030ca16740 1 -- 172.21.15.2:0/3715103493 shutdown_connections 2024-09-10T08:22:44.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.998+0000 7f030ca16740 1 -- 172.21.15.2:0/3715103493 wait complete. 2024-09-10T08:22:44.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.998+0000 7f0302a00640 1 --2- 172.21.15.2:0/271875829 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02e80a2b80 0x7f02e80a5040 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f02f40cfac0 tx=0x7f02f40d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:44.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.998+0000 7f030ca16740 1 -- start start 2024-09-10T08:22:44.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.999+0000 7f030ca16740 1 Processor -- start 2024-09-10T08:22:44.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.999+0000 7f030ca16740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c1ffac0 0x55ae3c20a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.999+0000 7f030ca16740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c2003e0 0x55ae3c2078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.999+0000 7f030ca16740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 0x55ae3c207e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.999+0000 7f030ca16740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ae3c20d910 con 0x55ae3c1ffac0 2024-09-10T08:22:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.999+0000 7f030ca16740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ae3c120fa0 con 0x55ae3c2003e0 2024-09-10T08:22:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.999+0000 7f030ca16740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ae3c20da90 con 0x55ae3c20aca0 2024-09-10T08:22:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.999+0000 7f030ca16740 10 client.0 _ll_register_callbacks cb 0x55ae3be74ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.999+0000 7f0303400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c1ffac0 0x55ae3c20a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.999+0000 7f0303400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c1ffac0 0x55ae3c20a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46172/0 (socket says 172.21.15.2:46172) 2024-09-10T08:22:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.999+0000 7f0303400640 1 -- 172.21.15.2:0/1608238702 learned_addr learned my addr 172.21.15.2:0/1608238702 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:43.999+0000 7f0302000640 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c2003e0 0x55ae3c2078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.000+0000 7f0303400640 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 0x55ae3c207e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.000+0000 7f0303400640 1 -- 172.21.15.2:0/1608238702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 msgr2=0x55ae3c207e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.000+0000 7f0303400640 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 0x55ae3c207e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.000+0000 7f0303400640 1 -- 172.21.15.2:0/1608238702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c2003e0 msgr2=0x55ae3c2078a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:44.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.000+0000 7f0303400640 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c2003e0 0x55ae3c2078a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.000+0000 7f0303400640 1 -- 172.21.15.2:0/1608238702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae3c208630 con 0x55ae3c1ffac0 2024-09-10T08:22:44.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.000+0000 7f0302000640 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c2003e0 0x55ae3c2078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:44.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.000+0000 7f0303400640 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c1ffac0 0x55ae3c20a5f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f02fc004970 tx=0x7f02fc009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae78611408a1fff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:44.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.000+0000 7f02ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:22:44.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.001+0000 7f02ecc00640 1 -- 172.21.15.2:0/1608238702 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc028660 con 0x55ae3c1ffac0 2024-09-10T08:22:44.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.001+0000 7f02ecc00640 1 -- 172.21.15.2:0/1608238702 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f02fc028800 con 0x55ae3c1ffac0 2024-09-10T08:22:44.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.001+0000 7f030ca16740 1 -- 172.21.15.2:0/1608238702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02fc004050 con 0x55ae3c1ffac0 2024-09-10T08:22:44.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.001+0000 7f02ecc00640 1 -- 172.21.15.2:0/1608238702 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f02fc035050 con 0x55ae3c1ffac0 2024-09-10T08:22:44.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.001+0000 7f030ca16740 10 client.19605 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:44.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.001+0000 7f030ca16740 1 -- 172.21.15.2:0/1608238702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ae3c219850 con 0x55ae3c1ffac0 2024-09-10T08:22:44.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.001+0000 7f02ecc00640 1 -- 172.21.15.2:0/1608238702 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02fc004050 con 0x55ae3c1ffac0 2024-09-10T08:22:44.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.002+0000 7f02ecc00640 1 -- 172.21.15.2:0/1608238702 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f02fc025040 con 0x55ae3c1ffac0 2024-09-10T08:22:44.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.002+0000 7f030ca16740 10 client.19605 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:44.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.002+0000 7f030ca16740 10 client.19605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:44.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.002+0000 7f030ca16740 20 client.19605 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:44.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.002+0000 7f030ca16740 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ae3c21acb0 0x55ae3c21d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.002+0000 7f030ca16740 4 client.19605 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.002+0000 7f030ca16740 1 -- 172.21.15.2:0/1608238702 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae3c173800 con 0x55ae3c21acb0 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.002+0000 7f0302000640 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ae3c21acb0 0x55ae3c21d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.003+0000 7f0302000640 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ae3c21acb0 0x55ae3c21d0a0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d088bce6403c6af server_cookie=8f2867fd0ee6e485 in_seq=0 out_seq=0 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.003+0000 7f02ecc00640 10 client.19605 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.003+0000 7f02ecc00640 1 -- 172.21.15.2:0/1608238702 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae3c173800 con 0x55ae3c21acb0 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.003+0000 7f02ecc00640 10 client.19605 handle_command_reply: tid=0 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.004+0000 7f030ca16740 1 client.19605 shutdown 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.004+0000 7f030ca16740 1 -- 172.21.15.2:0/1608238702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c1ffac0 msgr2=0x55ae3c20a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.004+0000 7f030ca16740 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c1ffac0 0x55ae3c20a5f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f02fc004970 tx=0x7f02fc009c50 comp rx=0 tx=0).stop 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.004+0000 7f030ca16740 1 -- 172.21.15.2:0/1608238702 shutdown_connections 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.004+0000 7f030ca16740 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ae3c21acb0 0x55ae3c21d0a0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.004+0000 7f030ca16740 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 0x55ae3c207e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.004+0000 7f030ca16740 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c2003e0 0x55ae3c2078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.004+0000 7f030ca16740 1 --2- 172.21.15.2:0/1608238702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c1ffac0 0x55ae3c20a5f0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.004+0000 7f030ca16740 1 -- 172.21.15.2:0/1608238702 >> 172.21.15.2:0/1608238702 conn(0x55ae3c1fc050 msgr2=0x55ae3c1fb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:44.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.004+0000 7f02ecc00640 0 client.19605 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:44.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.004+0000 7f030ca16740 1 -- 172.21.15.2:0/1608238702 shutdown_connections 2024-09-10T08:22:44.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.004+0000 7f030ca16740 1 -- 172.21.15.2:0/1608238702 wait complete. 2024-09-10T08:22:44.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.004+0000 7f030ca16740 20 client.19605 trim_cache size 0 max 16384 2024-09-10T08:22:44.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.022+0000 7f030ca16740 1 Processor -- start 2024-09-10T08:22:44.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.022+0000 7f030ca16740 1 -- start start 2024-09-10T08:22:44.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f030ca16740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c1ffac0 0x55ae3c209810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f030ca16740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c222760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f030ca16740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 0x55ae3c222ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f030ca16740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03041b39c0 con 0x55ae3c2003e0 2024-09-10T08:22:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f030ca16740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03041b1290 con 0x55ae3c1ffac0 2024-09-10T08:22:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f030ca16740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03041af0e0 con 0x55ae3c20aca0 2024-09-10T08:22:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f0302000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c222760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f0302000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c222760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46182/0 (socket says 172.21.15.2:46182) 2024-09-10T08:22:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f0302000640 1 -- 172.21.15.2:0/2844161849 learned_addr learned my addr 172.21.15.2:0/2844161849 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f0303400640 1 --2- 172.21.15.2:0/2844161849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c1ffac0 0x55ae3c209810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f0303400640 1 --2- 172.21.15.2:0/2844161849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 0x55ae3c222ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f0302000640 1 -- 172.21.15.2:0/2844161849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 msgr2=0x55ae3c222ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f0302000640 1 --2- 172.21.15.2:0/2844161849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 0x55ae3c222ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f0302000640 1 -- 172.21.15.2:0/2844161849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c1ffac0 msgr2=0x55ae3c209810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:44.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f0302000640 1 --2- 172.21.15.2:0/2844161849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c1ffac0 0x55ae3c209810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f0302000640 1 -- 172.21.15.2:0/2844161849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae3c1fd100 con 0x55ae3c2003e0 2024-09-10T08:22:44.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.023+0000 7f0303400640 1 --2- 172.21.15.2:0/2844161849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c1ffac0 0x55ae3c209810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:22:44.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.024+0000 7f0302000640 1 --2- 172.21.15.2:0/2844161849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c222760 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009900 tx=0x7f02f0031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87535094c09023d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.024+0000 7f02ef400640 1 -- 172.21.15.2:0/2844161849 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0035040 con 0x55ae3c2003e0 2024-09-10T08:22:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.024+0000 7f02ef400640 1 -- 172.21.15.2:0/2844161849 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f0005020 con 0x55ae3c2003e0 2024-09-10T08:22:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.024+0000 7f02ef400640 1 -- 172.21.15.2:0/2844161849 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0004bc0 con 0x55ae3c2003e0 2024-09-10T08:22:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.025+0000 7f030ca16740 1 -- 172.21.15.2:0/2844161849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 msgr2=0x55ae3c222760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.025+0000 7f030ca16740 1 --2- 172.21.15.2:0/2844161849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c222760 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009900 tx=0x7f02f0031850 comp rx=0 tx=0).stop 2024-09-10T08:22:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.025+0000 7f030ca16740 1 -- 172.21.15.2:0/2844161849 shutdown_connections 2024-09-10T08:22:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.025+0000 7f030ca16740 1 --2- 172.21.15.2:0/2844161849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c20aca0 0x55ae3c222ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.025+0000 7f030ca16740 1 --2- 172.21.15.2:0/2844161849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c222760 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.025+0000 7f030ca16740 1 --2- 172.21.15.2:0/2844161849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c1ffac0 0x55ae3c209810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.025+0000 7f030ca16740 1 -- 172.21.15.2:0/2844161849 >> 172.21.15.2:0/2844161849 conn(0x55ae3c1fc050 msgr2=0x55ae3c209100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.025+0000 7f030ca16740 1 -- 172.21.15.2:0/2844161849 shutdown_connections 2024-09-10T08:22:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.025+0000 7f030ca16740 1 -- 172.21.15.2:0/2844161849 wait complete. 2024-09-10T08:22:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.025+0000 7f030ca16740 1 -- start start 2024-09-10T08:22:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.025+0000 7f030ca16740 1 Processor -- start 2024-09-10T08:22:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.026+0000 7f030ca16740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c1ffac0 0x55ae3c213470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.026+0000 7f030ca16740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c2107c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.026+0000 7f030ca16740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c20aca0 0x55ae3c210d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.026+0000 7f030ca16740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03041af750 con 0x55ae3c2003e0 2024-09-10T08:22:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.026+0000 7f030ca16740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03041b0bc0 con 0x55ae3c20aca0 2024-09-10T08:22:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.026+0000 7f030ca16740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03041afe20 con 0x55ae3c1ffac0 2024-09-10T08:22:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.026+0000 7f030ca16740 10 client.0 _ll_register_callbacks cb 0x55ae3c208800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.026+0000 7f0302000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c2107c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.026+0000 7f0302000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c2107c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46192/0 (socket says 172.21.15.2:46192) 2024-09-10T08:22:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.026+0000 7f0302000640 1 -- 172.21.15.2:0/2049529650 learned_addr learned my addr 172.21.15.2:0/2049529650 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.026+0000 7f0303400640 1 --2- 172.21.15.2:0/2049529650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c1ffac0 0x55ae3c213470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.026+0000 7f0303400640 1 --2- 172.21.15.2:0/2049529650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c20aca0 0x55ae3c210d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.027+0000 7f0302000640 1 -- 172.21.15.2:0/2049529650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c1ffac0 msgr2=0x55ae3c213470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.027+0000 7f0302000640 1 --2- 172.21.15.2:0/2049529650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c1ffac0 0x55ae3c213470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.027+0000 7f0302000640 1 -- 172.21.15.2:0/2049529650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c20aca0 msgr2=0x55ae3c210d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.027+0000 7f0302000640 1 --2- 172.21.15.2:0/2049529650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c20aca0 0x55ae3c210d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.027+0000 7f0302000640 1 -- 172.21.15.2:0/2049529650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae3c211500 con 0x55ae3c2003e0 2024-09-10T08:22:44.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.027+0000 7f0302000640 1 --2- 172.21.15.2:0/2049529650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c2107c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f02f002a910 tx=0x7f02f00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83812b799c98100e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:44.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.027+0000 7f02ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:22:44.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.027+0000 7f02ecc00640 1 -- 172.21.15.2:0/2049529650 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0056070 con 0x55ae3c2003e0 2024-09-10T08:22:44.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.028+0000 7f02ecc00640 1 -- 172.21.15.2:0/2049529650 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f02f00053d0 con 0x55ae3c2003e0 2024-09-10T08:22:44.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.028+0000 7f030ca16740 1 -- 172.21.15.2:0/2049529650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f0005020 con 0x55ae3c2003e0 2024-09-10T08:22:44.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.028+0000 7f02ecc00640 1 -- 172.21.15.2:0/2049529650 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f02f0062050 con 0x55ae3c2003e0 2024-09-10T08:22:44.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.028+0000 7f030ca16740 10 client.19611 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:44.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.028+0000 7f030ca16740 1 -- 172.21.15.2:0/2049529650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ae3c2857b0 con 0x55ae3c2003e0 2024-09-10T08:22:44.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.028+0000 7f02ecc00640 1 -- 172.21.15.2:0/2049529650 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f0005020 con 0x55ae3c2003e0 2024-09-10T08:22:44.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.028+0000 7f02ecc00640 1 -- 172.21.15.2:0/2049529650 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f02f0047850 con 0x55ae3c2003e0 2024-09-10T08:22:44.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.028+0000 7f030ca16740 10 client.19611 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:44.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.028+0000 7f030ca16740 10 client.19611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:44.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.028+0000 7f030ca16740 20 client.19611 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:44.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.028+0000 7f030ca16740 1 --2- 172.21.15.2:0/2049529650 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ae3c285e20 0x55ae3c288240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:44.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.028+0000 7f030ca16740 4 client.19611 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:22:44.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.028+0000 7f030ca16740 1 -- 172.21.15.2:0/2049529650 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ae3c16dd10 con 0x55ae3c285e20 2024-09-10T08:22:44.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.029+0000 7f0303400640 1 --2- 172.21.15.2:0/2049529650 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ae3c285e20 0x55ae3c288240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:44.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.029+0000 7f0303400640 1 --2- 172.21.15.2:0/2049529650 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ae3c285e20 0x55ae3c288240 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74a0e3ae0b41dd9 server_cookie=2b3abdb58e0f669b in_seq=0 out_seq=0 2024-09-10T08:22:44.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.029+0000 7f02ecc00640 10 client.19611 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:44.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.030+0000 7f02ecc00640 1 -- 172.21.15.2:0/2049529650 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ae3c16dd10 con 0x55ae3c285e20 2024-09-10T08:22:44.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.030+0000 7f02ecc00640 10 client.19611 handle_command_reply: tid=0 2024-09-10T08:22:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.030+0000 7f030ca16740 1 client.19611 shutdown 2024-09-10T08:22:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.031+0000 7f030ca16740 1 -- 172.21.15.2:0/2049529650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 msgr2=0x55ae3c2107c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.031+0000 7f030ca16740 1 --2- 172.21.15.2:0/2049529650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c2107c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f02f002a910 tx=0x7f02f00047d0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.031+0000 7f030ca16740 1 -- 172.21.15.2:0/2049529650 shutdown_connections 2024-09-10T08:22:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.031+0000 7f030ca16740 1 --2- 172.21.15.2:0/2049529650 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55ae3c285e20 0x55ae3c288240 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.031+0000 7f030ca16740 1 --2- 172.21.15.2:0/2049529650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae3c20aca0 0x55ae3c210d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.031+0000 7f030ca16740 1 --2- 172.21.15.2:0/2049529650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae3c2003e0 0x55ae3c2107c0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.031+0000 7f030ca16740 1 --2- 172.21.15.2:0/2049529650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae3c1ffac0 0x55ae3c213470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.031+0000 7f030ca16740 1 -- 172.21.15.2:0/2049529650 >> 172.21.15.2:0/2049529650 conn(0x55ae3c1fc050 msgr2=0x55ae3c1fb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.031+0000 7f02ecc00640 0 client.19611 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.031+0000 7f030ca16740 1 -- 172.21.15.2:0/2049529650 shutdown_connections 2024-09-10T08:22:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.031+0000 7f030ca16740 1 -- 172.21.15.2:0/2049529650 wait complete. 2024-09-10T08:22:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.031+0000 7f030ca16740 20 client.19611 trim_cache size 0 max 16384 2024-09-10T08:22:44.034 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:22:44.034 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:22:44.034 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "5b2727d8-e369-423f-8fd6-d085c534eee3", 2024-09-10T08:22:44.034 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:22:44.034 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:22:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.032+0000 7f0309e00640 1 -- 172.21.15.2:0/271875829 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02e80a2b80 msgr2=0x7f02e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.032+0000 7f0309e00640 1 --2- 172.21.15.2:0/271875829 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02e80a2b80 0x7f02e80a5040 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f02f40cfac0 tx=0x7f02f40d5530 comp rx=0 tx=0).stop 2024-09-10T08:22:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.032+0000 7f0309e00640 1 -- 172.21.15.2:0/271875829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0304103ff0 msgr2=0x7f030419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.032+0000 7f0309e00640 1 --2- 172.21.15.2:0/271875829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0304103ff0 0x7f030419a6a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f02f400e9f0 tx=0x7f02f400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.032+0000 7f0309e00640 1 -- 172.21.15.2:0/271875829 shutdown_connections 2024-09-10T08:22:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.032+0000 7f0309e00640 1 --2- 172.21.15.2:0/271875829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0304109120 0x7f030419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.032+0000 7f0309e00640 1 --2- 172.21.15.2:0/271875829 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02e80a2b80 0x7f02e80a5040 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.032+0000 7f0309e00640 1 --2- 172.21.15.2:0/271875829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03041049a0 0x7f030419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.032+0000 7f0309e00640 1 --2- 172.21.15.2:0/271875829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0304103ff0 0x7f030419a6a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:44.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.032+0000 7f0309e00640 1 -- 172.21.15.2:0/271875829 >> 172.21.15.2:0/271875829 conn(0x7f03040ffa90 msgr2=0x7f0304100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:44.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.032+0000 7f0309e00640 1 -- 172.21.15.2:0/271875829 shutdown_connections 2024-09-10T08:22:44.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:44.032+0000 7f0309e00640 1 -- 172.21.15.2:0/271875829 wait complete. 2024-09-10T08:22:44.991 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:22:45.370 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:22:45.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:45 smithi155 ceph-mon[35405]: pgmap v825: 129 pgs: 129 active+clean; 3.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:22:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:45 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:22:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:45 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:22:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:45 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:22:45.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:45 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:22:45.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:45 smithi107 ceph-mon[35048]: pgmap v825: 129 pgs: 129 active+clean; 3.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:22:45.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:45 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:22:45.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:45 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:22:45.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:45 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:22:45.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:45 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:22:45.679 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:45 smithi002 ceph-mon[30563]: pgmap v825: 129 pgs: 129 active+clean; 3.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T08:22:45.679 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:45 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:22:45.679 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:45 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:22:45.679 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:45 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:22:45.680 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:45 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:22:46.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.006+0000 7f547b600640 1 -- 172.21.15.2:0/4237350465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5474074f40 msgr2=0x7f547410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.006+0000 7f547b600640 1 --2- 172.21.15.2:0/4237350465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5474074f40 0x7f547410de80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f545c009900 tx=0x7f545c02a480 comp rx=0 tx=0).stop 2024-09-10T08:22:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.006+0000 7f547b600640 1 -- 172.21.15.2:0/4237350465 shutdown_connections 2024-09-10T08:22:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.006+0000 7f547b600640 1 --2- 172.21.15.2:0/4237350465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f547410e3c0 0x7f5474110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.006+0000 7f547b600640 1 --2- 172.21.15.2:0/4237350465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5474074f40 0x7f547410de80 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.006+0000 7f547b600640 1 --2- 172.21.15.2:0/4237350465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5474074620 0x7f5474074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.006+0000 7f547b600640 1 -- 172.21.15.2:0/4237350465 >> 172.21.15.2:0/4237350465 conn(0x7f54740fd8c0 msgr2=0x7f54740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.006+0000 7f547b600640 1 -- 172.21.15.2:0/4237350465 shutdown_connections 2024-09-10T08:22:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.006+0000 7f547b600640 1 -- 172.21.15.2:0/4237350465 wait complete. 2024-09-10T08:22:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.007+0000 7f547b600640 1 Processor -- start 2024-09-10T08:22:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.008+0000 7f547b600640 1 -- start start 2024-09-10T08:22:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.008+0000 7f547b600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5474074620 0x7f547419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.008+0000 7f547b600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5474074f40 0x7f547419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.008+0000 7f547b600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f547410e3c0 0x7f547419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.008+0000 7f547b600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5474114190 con 0x7f5474074f40 2024-09-10T08:22:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.008+0000 7f547b600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5474114010 con 0x7f5474074620 2024-09-10T08:22:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.008+0000 7f547b600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5474114310 con 0x7f547410e3c0 2024-09-10T08:22:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.008+0000 7f546be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5474074f40 0x7f547419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.008+0000 7f546be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5474074f40 0x7f547419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46222/0 (socket says 172.21.15.2:46222) 2024-09-10T08:22:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.008+0000 7f546be00640 1 -- 172.21.15.2:0/4129788753 learned_addr learned my addr 172.21.15.2:0/4129788753 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.008+0000 7f5478c00640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5474074620 0x7f547419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.009+0000 7f5479600640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f547410e3c0 0x7f547419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.009+0000 7f546be00640 1 -- 172.21.15.2:0/4129788753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f547410e3c0 msgr2=0x7f547419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.009+0000 7f546be00640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f547410e3c0 0x7f547419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.009+0000 7f546be00640 1 -- 172.21.15.2:0/4129788753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5474074620 msgr2=0x7f547419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.009+0000 7f546be00640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5474074620 0x7f547419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.009+0000 7f546be00640 1 -- 172.21.15.2:0/4129788753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547419f620 con 0x7f5474074f40 2024-09-10T08:22:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.009+0000 7f546be00640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5474074f40 0x7f547419abb0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f545c004770 tx=0x7f545c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.009+0000 7f5472000640 1 -- 172.21.15.2:0/4129788753 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c03f070 con 0x7f5474074f40 2024-09-10T08:22:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.009+0000 7f5472000640 1 -- 172.21.15.2:0/4129788753 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f545c031ca0 con 0x7f5474074f40 2024-09-10T08:22:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.009+0000 7f5472000640 1 -- 172.21.15.2:0/4129788753 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c035030 con 0x7f5474074f40 2024-09-10T08:22:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.009+0000 7f547b600640 1 -- 172.21.15.2:0/4129788753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547419f8b0 con 0x7f5474074f40 2024-09-10T08:22:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.010+0000 7f547b600640 1 -- 172.21.15.2:0/4129788753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54741a71f0 con 0x7f5474074f40 2024-09-10T08:22:46.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.014+0000 7f547e059740 1 Processor -- start 2024-09-10T08:22:46.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.014+0000 7f5472000640 1 -- 172.21.15.2:0/4129788753 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f545c042070 con 0x7f5474074f40 2024-09-10T08:22:46.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.014+0000 7f547e059740 1 -- start start 2024-09-10T08:22:46.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.014+0000 7f547e059740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 0x55834f4a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.014+0000 7f547e059740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a44b0 0x55834f4ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.014+0000 7f547e059740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4aed70 0x55834f4b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.014+0000 7f547e059740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55834f40a3e0 con 0x55834f4a44b0 2024-09-10T08:22:46.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.014+0000 7f547e059740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55834f3b1d80 con 0x55834f4a3b90 2024-09-10T08:22:46.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.014+0000 7f547e059740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55834f4063a0 con 0x55834f4aed70 2024-09-10T08:22:46.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.015+0000 7f5478c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a44b0 0x55834f4ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.015+0000 7f546be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4aed70 0x55834f4b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.015+0000 7f5479600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 0x55834f4a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.015+0000 7f5478c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a44b0 0x55834f4ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46236/0 (socket says 172.21.15.2:46236) 2024-09-10T08:22:46.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.015+0000 7f546be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4aed70 0x55834f4b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34870/0 (socket says 172.21.15.2:34870) 2024-09-10T08:22:46.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.015+0000 7f5479600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 0x55834f4a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40676/0 (socket says 172.21.15.2:40676) 2024-09-10T08:22:46.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.015+0000 7f5478c00640 1 -- 172.21.15.2:0/2135471992 learned_addr learned my addr 172.21.15.2:0/2135471992 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:46.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.015+0000 7f5479600640 1 -- 172.21.15.2:0/2135471992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4aed70 msgr2=0x55834f4b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.015+0000 7f5479600640 1 --2- 172.21.15.2:0/2135471992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4aed70 0x55834f4b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.015+0000 7f5479600640 1 -- 172.21.15.2:0/2135471992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a44b0 msgr2=0x55834f4ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.015+0000 7f5479600640 1 --2- 172.21.15.2:0/2135471992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a44b0 0x55834f4ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.015+0000 7f5479600640 1 -- 172.21.15.2:0/2135471992 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55834f3d0ae0 con 0x55834f4a3b90 2024-09-10T08:22:46.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.016+0000 7f5479600640 1 --2- 172.21.15.2:0/2135471992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 0x55834f4a3f70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f546c009f00 tx=0x7f546c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=974bd7f80b72bc6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:46.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.016+0000 7f5472000640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f54580a2b80 0x7f54580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.016+0000 7f5469c00640 1 -- 172.21.15.2:0/2135471992 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c004d20 con 0x55834f4a3b90 2024-09-10T08:22:46.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.016+0000 7f5469c00640 1 -- 172.21.15.2:0/2135471992 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f546c004050 con 0x55834f4a3b90 2024-09-10T08:22:46.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.016+0000 7f5469c00640 1 -- 172.21.15.2:0/2135471992 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c005280 con 0x55834f4a3b90 2024-09-10T08:22:46.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.016+0000 7f5472000640 1 -- 172.21.15.2:0/4129788753 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f545c0efa70 con 0x7f5474074f40 2024-09-10T08:22:46.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.016+0000 7f5478c00640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f54580a2b80 0x7f54580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.017+0000 7f547e059740 1 -- 172.21.15.2:0/2135471992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 msgr2=0x55834f4a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.017+0000 7f547e059740 1 --2- 172.21.15.2:0/2135471992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 0x55834f4a3f70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f546c009f00 tx=0x7f546c011810 comp rx=0 tx=0).stop 2024-09-10T08:22:46.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.017+0000 7f5478c00640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f54580a2b80 0x7f54580a5040 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55834f4a0f80 tx=0x7f5464005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:46.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.017+0000 7f547e059740 1 -- 172.21.15.2:0/2135471992 shutdown_connections 2024-09-10T08:22:46.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.017+0000 7f547e059740 1 --2- 172.21.15.2:0/2135471992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4aed70 0x55834f4b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.017+0000 7f547e059740 1 --2- 172.21.15.2:0/2135471992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a44b0 0x55834f4ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.017+0000 7f547e059740 1 --2- 172.21.15.2:0/2135471992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 0x55834f4a3f70 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.017+0000 7f547e059740 1 -- 172.21.15.2:0/2135471992 >> 172.21.15.2:0/2135471992 conn(0x55834f4a0120 msgr2=0x55834f4a0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:46.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.017+0000 7f547e059740 1 -- 172.21.15.2:0/2135471992 shutdown_connections 2024-09-10T08:22:46.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.017+0000 7f547e059740 1 -- 172.21.15.2:0/2135471992 wait complete. 2024-09-10T08:22:46.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.018+0000 7f547e059740 1 -- start start 2024-09-10T08:22:46.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.018+0000 7f547e059740 1 Processor -- start 2024-09-10T08:22:46.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.018+0000 7f547e059740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 0x55834f4abc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.018+0000 7f547e059740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4a44b0 0x55834f4ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.018+0000 7f547e059740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4aed70 0x55834f4ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.018+0000 7f547e059740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55834f4b19e0 con 0x55834f4aed70 2024-09-10T08:22:46.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.018+0000 7f547e059740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55834f3c50f0 con 0x55834f4a3b90 2024-09-10T08:22:46.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.018+0000 7f547e059740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55834f4b1b60 con 0x55834f4a44b0 2024-09-10T08:22:46.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.019+0000 7f547e059740 10 client.0 _ll_register_callbacks cb 0x55834f3b69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:46.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.019+0000 7f5478c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 0x55834f4abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.019+0000 7f5478c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 0x55834f4abc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:40686/0 (socket says 172.21.15.2:40686) 2024-09-10T08:22:46.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.019+0000 7f5478c00640 1 -- 172.21.15.2:0/2142278578 learned_addr learned my addr 172.21.15.2:0/2142278578 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:46.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.019+0000 7f546be00640 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4a44b0 0x55834f4ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.019+0000 7f5479600640 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4aed70 0x55834f4ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.019+0000 7f546be00640 1 -- 172.21.15.2:0/2142278578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 msgr2=0x55834f4abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.019+0000 7f546be00640 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 0x55834f4abc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.019+0000 7f546be00640 1 -- 172.21.15.2:0/2142278578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4aed70 msgr2=0x55834f4ac6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.019+0000 7f546be00640 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4aed70 0x55834f4ac6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.019+0000 7f546be00640 1 -- 172.21.15.2:0/2142278578 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55834f17b3c0 con 0x55834f4a44b0 2024-09-10T08:22:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.019+0000 7f5478c00640 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 0x55834f4abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.020+0000 7f546be00640 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4a44b0 0x55834f4ac1a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55834f4a14a0 tx=0x7f545c03b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2a240a0a9fb9451 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.020+0000 7f5462000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:22:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.020+0000 7f5462000640 1 -- 172.21.15.2:0/2142278578 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c0efe60 con 0x55834f4a44b0 2024-09-10T08:22:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.020+0000 7f5462000640 1 -- 172.21.15.2:0/2142278578 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f545c0b8660 con 0x55834f4a44b0 2024-09-10T08:22:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.020+0000 7f547e059740 1 -- 172.21.15.2:0/2142278578 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f546c004050 con 0x55834f4a44b0 2024-09-10T08:22:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.020+0000 7f5462000640 1 -- 172.21.15.2:0/2142278578 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f545c049070 con 0x55834f4a44b0 2024-09-10T08:22:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.020+0000 7f547e059740 10 client.29575 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:46.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.020+0000 7f547e059740 1 -- 172.21.15.2:0/2142278578 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55834f4b1a40 con 0x55834f4a44b0 2024-09-10T08:22:46.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.020+0000 7f5462000640 1 -- 172.21.15.2:0/2142278578 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f546c004050 con 0x55834f4a44b0 2024-09-10T08:22:46.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.020+0000 7f5462000640 1 -- 172.21.15.2:0/2142278578 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f545c03b4f0 con 0x55834f4a44b0 2024-09-10T08:22:46.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.021+0000 7f547e059740 10 client.29575 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:46.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.021+0000 7f547e059740 10 client.29575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:46.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.021+0000 7f547e059740 20 client.29575 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:46.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.021+0000 7f547e059740 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55834f4bea20 0x55834f4c0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.021+0000 7f547e059740 4 client.29575 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:22:46.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.021+0000 7f547e059740 1 -- 172.21.15.2:0/2142278578 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55834f417950 con 0x55834f4bea20 2024-09-10T08:22:46.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.021+0000 7f5478c00640 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55834f4bea20 0x55834f4c0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.022+0000 7f5478c00640 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55834f4bea20 0x55834f4c0ea0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b34ce8fc2f7711f3 server_cookie=a610456a543d0cd in_seq=0 out_seq=0 2024-09-10T08:22:46.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.022+0000 7f5462000640 10 client.29575 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:46.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.022+0000 7f5462000640 1 -- 172.21.15.2:0/2142278578 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55834f417950 con 0x55834f4bea20 2024-09-10T08:22:46.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f5462000640 10 client.29575 handle_command_reply: tid=0 2024-09-10T08:22:46.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f547e059740 1 client.29575 shutdown 2024-09-10T08:22:46.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f547e059740 1 -- 172.21.15.2:0/2142278578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4a44b0 msgr2=0x55834f4ac1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f547e059740 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4a44b0 0x55834f4ac1a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55834f4a14a0 tx=0x7f545c03b460 comp rx=0 tx=0).stop 2024-09-10T08:22:46.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f547e059740 1 -- 172.21.15.2:0/2142278578 shutdown_connections 2024-09-10T08:22:46.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f547e059740 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55834f4bea20 0x55834f4c0ea0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f547e059740 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4aed70 0x55834f4ac6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f547e059740 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4a44b0 0x55834f4ac1a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f547e059740 1 --2- 172.21.15.2:0/2142278578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a3b90 0x55834f4abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f5462000640 0 client.29575 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:46.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f547e059740 1 -- 172.21.15.2:0/2142278578 >> 172.21.15.2:0/2142278578 conn(0x55834f4a0120 msgr2=0x55834f4a5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:46.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f547e059740 1 -- 172.21.15.2:0/2142278578 shutdown_connections 2024-09-10T08:22:46.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f547e059740 1 -- 172.21.15.2:0/2142278578 wait complete. 2024-09-10T08:22:46.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.023+0000 7f547e059740 20 client.29575 trim_cache size 0 max 16384 2024-09-10T08:22:46.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.041+0000 7f547e059740 1 Processor -- start 2024-09-10T08:22:46.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.041+0000 7f547e059740 1 -- start start 2024-09-10T08:22:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.042+0000 7f547e059740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 0x55834f4a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.042+0000 7f547e059740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4a44b0 0x55834f4c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.042+0000 7f547e059740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4aed70 0x55834f4c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.042+0000 7f547e059740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f54741b3970 con 0x55834f4a3b90 2024-09-10T08:22:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.042+0000 7f547e059740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54741b1240 con 0x55834f4aed70 2024-09-10T08:22:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.042+0000 7f547e059740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f54741af090 con 0x55834f4a44b0 2024-09-10T08:22:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.042+0000 7f5478c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 0x55834f4a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.042+0000 7f5478c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 0x55834f4a65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46258/0 (socket says 172.21.15.2:46258) 2024-09-10T08:22:46.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.042+0000 7f5478c00640 1 -- 172.21.15.2:0/3447042833 learned_addr learned my addr 172.21.15.2:0/3447042833 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:46.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.042+0000 7f546be00640 1 --2- 172.21.15.2:0/3447042833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4a44b0 0x55834f4c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.042+0000 7f5479600640 1 --2- 172.21.15.2:0/3447042833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4aed70 0x55834f4c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.043+0000 7f5478c00640 1 -- 172.21.15.2:0/3447042833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4a44b0 msgr2=0x55834f4c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.043+0000 7f5478c00640 1 --2- 172.21.15.2:0/3447042833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4a44b0 0x55834f4c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.043+0000 7f5478c00640 1 -- 172.21.15.2:0/3447042833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4aed70 msgr2=0x55834f4c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.043+0000 7f5478c00640 1 --2- 172.21.15.2:0/3447042833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4aed70 0x55834f4c6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.043+0000 7f5478c00640 1 -- 172.21.15.2:0/3447042833 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55834f4c1520 con 0x55834f4a3b90 2024-09-10T08:22:46.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.043+0000 7f5478c00640 1 --2- 172.21.15.2:0/3447042833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 0x55834f4a65f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x55834f4c1720 tx=0x7f546401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc27508d01a23e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:46.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.043+0000 7f5469200640 1 -- 172.21.15.2:0/3447042833 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546401ee60 con 0x55834f4a3b90 2024-09-10T08:22:46.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.043+0000 7f5469200640 1 -- 172.21.15.2:0/3447042833 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5464005500 con 0x55834f4a3b90 2024-09-10T08:22:46.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.043+0000 7f5469200640 1 -- 172.21.15.2:0/3447042833 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546400fa80 con 0x55834f4a3b90 2024-09-10T08:22:46.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.044+0000 7f547e059740 1 -- 172.21.15.2:0/3447042833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 msgr2=0x55834f4a65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.044+0000 7f547e059740 1 --2- 172.21.15.2:0/3447042833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 0x55834f4a65f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x55834f4c1720 tx=0x7f546401b000 comp rx=0 tx=0).stop 2024-09-10T08:22:46.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.044+0000 7f547e059740 1 -- 172.21.15.2:0/3447042833 shutdown_connections 2024-09-10T08:22:46.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.044+0000 7f547e059740 1 --2- 172.21.15.2:0/3447042833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4aed70 0x55834f4c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.044+0000 7f547e059740 1 --2- 172.21.15.2:0/3447042833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4a44b0 0x55834f4c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.044+0000 7f547e059740 1 --2- 172.21.15.2:0/3447042833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 0x55834f4a65f0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.044+0000 7f547e059740 1 -- 172.21.15.2:0/3447042833 >> 172.21.15.2:0/3447042833 conn(0x55834f4a0120 msgr2=0x55834f4adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:46.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.044+0000 7f547e059740 1 -- 172.21.15.2:0/3447042833 shutdown_connections 2024-09-10T08:22:46.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.044+0000 7f547e059740 1 -- 172.21.15.2:0/3447042833 wait complete. 2024-09-10T08:22:46.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.044+0000 7f547e059740 1 -- start start 2024-09-10T08:22:46.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.045+0000 7f547e059740 1 Processor -- start 2024-09-10T08:22:46.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.045+0000 7f547e059740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 0x55834f4a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.045+0000 7f547e059740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a44b0 0x55834f4a9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.045+0000 7f547e059740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4aed70 0x55834f4a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.045+0000 7f547e059740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f54741af700 con 0x55834f4a3b90 2024-09-10T08:22:46.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.045+0000 7f547e059740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54741b0b70 con 0x55834f4a44b0 2024-09-10T08:22:46.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.045+0000 7f547e059740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f54741afdd0 con 0x55834f4aed70 2024-09-10T08:22:46.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.045+0000 7f547e059740 10 client.0 _ll_register_callbacks cb 0x7f545c031e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:46.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.045+0000 7f5478c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 0x55834f4a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.046+0000 7f5478c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 0x55834f4a8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46262/0 (socket says 172.21.15.2:46262) 2024-09-10T08:22:46.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.046+0000 7f5478c00640 1 -- 172.21.15.2:0/295035414 learned_addr learned my addr 172.21.15.2:0/295035414 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:46.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.046+0000 7f546be00640 1 --2- 172.21.15.2:0/295035414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a44b0 0x55834f4a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.046+0000 7f5479600640 1 --2- 172.21.15.2:0/295035414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4aed70 0x55834f4a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.046+0000 7f5478c00640 1 -- 172.21.15.2:0/295035414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4aed70 msgr2=0x55834f4a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.046+0000 7f5478c00640 1 --2- 172.21.15.2:0/295035414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4aed70 0x55834f4a98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.046+0000 7f5478c00640 1 -- 172.21.15.2:0/295035414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a44b0 msgr2=0x55834f4a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.046+0000 7f5478c00640 1 --2- 172.21.15.2:0/295035414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a44b0 0x55834f4a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.046+0000 7f5478c00640 1 -- 172.21.15.2:0/295035414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55834f4a0db0 con 0x55834f4a3b90 2024-09-10T08:22:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.046+0000 7f5478c00640 1 --2- 172.21.15.2:0/295035414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 0x55834f4a8e40 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5464019040 tx=0x7f54640056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8470c233fc2c90a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.046+0000 7f5462000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:22:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.046+0000 7f5462000640 1 -- 172.21.15.2:0/295035414 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464044070 con 0x55834f4a3b90 2024-09-10T08:22:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.047+0000 7f5462000640 1 -- 172.21.15.2:0/295035414 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5464005110 con 0x55834f4a3b90 2024-09-10T08:22:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.047+0000 7f547e059740 1 -- 172.21.15.2:0/295035414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5464005500 con 0x55834f4a3b90 2024-09-10T08:22:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.047+0000 7f5462000640 1 -- 172.21.15.2:0/295035414 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f546401ea00 con 0x55834f4a3b90 2024-09-10T08:22:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.047+0000 7f547e059740 10 client.19629 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.047+0000 7f547e059740 1 -- 172.21.15.2:0/295035414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55834f4a11d0 con 0x55834f4a3b90 2024-09-10T08:22:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.047+0000 7f5462000640 1 -- 172.21.15.2:0/295035414 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5464050070 con 0x55834f4a3b90 2024-09-10T08:22:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.047+0000 7f5462000640 1 -- 172.21.15.2:0/295035414 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f546400fcc0 con 0x55834f4a3b90 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.047+0000 7f547e059740 10 client.19629 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.047+0000 7f547e059740 10 client.19629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.047+0000 7f547e059740 20 client.19629 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.047+0000 7f547e059740 1 --2- 172.21.15.2:0/295035414 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55834f529fa0 0x55834f52c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.047+0000 7f547e059740 4 client.19629 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.047+0000 7f547e059740 1 -- 172.21.15.2:0/295035414 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55834f411e60 con 0x55834f529fa0 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.048+0000 7f546be00640 1 --2- 172.21.15.2:0/295035414 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55834f529fa0 0x55834f52c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.048+0000 7f546be00640 1 --2- 172.21.15.2:0/295035414 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55834f529fa0 0x55834f52c390 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22d303f78c077310 server_cookie=7d2fc9b4a409b5fa in_seq=0 out_seq=0 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.048+0000 7f5462000640 10 client.19629 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f5462000640 1 -- 172.21.15.2:0/295035414 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55834f411e60 con 0x55834f529fa0 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f5462000640 10 client.19629 handle_command_reply: tid=0 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f547e059740 1 client.19629 shutdown 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f547e059740 1 -- 172.21.15.2:0/295035414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 msgr2=0x55834f4a8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f547e059740 1 --2- 172.21.15.2:0/295035414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 0x55834f4a8e40 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5464019040 tx=0x7f54640056b0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f547e059740 1 -- 172.21.15.2:0/295035414 shutdown_connections 2024-09-10T08:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f547e059740 1 --2- 172.21.15.2:0/295035414 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55834f529fa0 0x55834f52c390 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f547e059740 1 --2- 172.21.15.2:0/295035414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55834f4aed70 0x55834f4a98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f547e059740 1 --2- 172.21.15.2:0/295035414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55834f4a44b0 0x55834f4a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f547e059740 1 --2- 172.21.15.2:0/295035414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55834f4a3b90 0x55834f4a8e40 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f547e059740 1 -- 172.21.15.2:0/295035414 >> 172.21.15.2:0/295035414 conn(0x55834f4a0120 msgr2=0x55834f4abc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f5462000640 0 client.19629 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f547e059740 1 -- 172.21.15.2:0/295035414 shutdown_connections 2024-09-10T08:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f547e059740 1 -- 172.21.15.2:0/295035414 wait complete. 2024-09-10T08:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.049+0000 7f547e059740 20 client.19629 trim_cache size 0 max 16384 2024-09-10T08:22:46.052 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:22:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:22:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:22:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "5b2727d8-e369-423f-8fd6-d085c534eee3": { 2024-09-10T08:22:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:22:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "5b2727d8-e369-423f-8fd6-d085c534eee3", 2024-09-10T08:22:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:22:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:22:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:22:46.053 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.050+0000 7f547b600640 1 -- 172.21.15.2:0/4129788753 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f54580a2b80 msgr2=0x7f54580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.050+0000 7f547b600640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f54580a2b80 0x7f54580a5040 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55834f4a0f80 tx=0x7f5464005f90 comp rx=0 tx=0).stop 2024-09-10T08:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.050+0000 7f547b600640 1 -- 172.21.15.2:0/4129788753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5474074f40 msgr2=0x7f547419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.050+0000 7f547b600640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5474074f40 0x7f547419abb0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f545c004770 tx=0x7f545c003c10 comp rx=0 tx=0).stop 2024-09-10T08:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.050+0000 7f547b600640 1 -- 172.21.15.2:0/4129788753 shutdown_connections 2024-09-10T08:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.050+0000 7f547b600640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f54580a2b80 0x7f54580a5040 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.050+0000 7f547b600640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f547410e3c0 0x7f547419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.050+0000 7f547b600640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5474074f40 0x7f547419abb0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.050+0000 7f547b600640 1 --2- 172.21.15.2:0/4129788753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5474074620 0x7f547419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.050+0000 7f547b600640 1 -- 172.21.15.2:0/4129788753 >> 172.21.15.2:0/4129788753 conn(0x7f54740fd8c0 msgr2=0x7f54740fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.050+0000 7f547b600640 1 -- 172.21.15.2:0/4129788753 shutdown_connections 2024-09-10T08:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:22:46.050+0000 7f547b600640 1 -- 172.21.15.2:0/4129788753 wait complete. 2024-09-10T08:22:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.423+0000 7fd033600640 1 -- 172.21.15.2:0/2389903043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02c103d60 msgr2=0x7fd02c104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.423+0000 7fd033600640 1 --2- 172.21.15.2:0/2389903043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02c103d60 0x7fd02c104160 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd020009900 tx=0x7fd02002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.423+0000 7fd033600640 1 -- 172.21.15.2:0/2389903043 shutdown_connections 2024-09-10T08:22:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.423+0000 7fd033600640 1 --2- 172.21.15.2:0/2389903043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd02c105910 0x7fd02c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.423+0000 7fd033600640 1 --2- 172.21.15.2:0/2389903043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd02c104f50 0x7fd02c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.423+0000 7fd033600640 1 --2- 172.21.15.2:0/2389903043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02c103d60 0x7fd02c104160 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.423+0000 7fd033600640 1 -- 172.21.15.2:0/2389903043 >> 172.21.15.2:0/2389903043 conn(0x7fd02c0ff4e0 msgr2=0x7fd02c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.423+0000 7fd033600640 1 -- 172.21.15.2:0/2389903043 shutdown_connections 2024-09-10T08:22:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.423+0000 7fd033600640 1 -- 172.21.15.2:0/2389903043 wait complete. 2024-09-10T08:22:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.424+0000 7fd033600640 1 Processor -- start 2024-09-10T08:22:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.424+0000 7fd033600640 1 -- start start 2024-09-10T08:22:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd033600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd02c103d60 0x7fd02c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd033600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd02c104f50 0x7fd02c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd033600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02c105910 0x7fd02c1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd033600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c10fa10 con 0x7fd02c103d60 2024-09-10T08:22:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd033600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c10f890 con 0x7fd02c105910 2024-09-10T08:22:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd033600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c10fb90 con 0x7fd02c104f50 2024-09-10T08:22:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd030c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd02c103d60 0x7fd02c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd030c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd02c103d60 0x7fd02c19a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46268/0 (socket says 172.21.15.2:46268) 2024-09-10T08:22:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd030c00640 1 -- 172.21.15.2:0/2236834126 learned_addr learned my addr 172.21.15.2:0/2236834126 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd031600640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02c105910 0x7fd02c1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd02be00640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd02c104f50 0x7fd02c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd030c00640 1 -- 172.21.15.2:0/2236834126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd02c104f50 msgr2=0x7fd02c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd030c00640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd02c104f50 0x7fd02c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd030c00640 1 -- 172.21.15.2:0/2236834126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02c105910 msgr2=0x7fd02c1a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd030c00640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02c105910 0x7fd02c1a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.425+0000 7fd030c00640 1 -- 172.21.15.2:0/2236834126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02c1a1c20 con 0x7fd02c103d60 2024-09-10T08:22:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.426+0000 7fd030c00640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd02c103d60 0x7fd02c19a000 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd020031440 tx=0x7fd020032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.426+0000 7fd029600640 1 -- 172.21.15.2:0/2236834126 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02003f030 con 0x7fd02c103d60 2024-09-10T08:22:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.426+0000 7fd029600640 1 -- 172.21.15.2:0/2236834126 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd020003ce0 con 0x7fd02c103d60 2024-09-10T08:22:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.426+0000 7fd033600640 1 -- 172.21.15.2:0/2236834126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02c1a1eb0 con 0x7fd02c103d60 2024-09-10T08:22:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.426+0000 7fd029600640 1 -- 172.21.15.2:0/2236834126 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020035070 con 0x7fd02c103d60 2024-09-10T08:22:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.426+0000 7fd033600640 1 -- 172.21.15.2:0/2236834126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd02c1a22c0 con 0x7fd02c103d60 2024-09-10T08:22:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.429+0000 7fd029600640 1 -- 172.21.15.2:0/2236834126 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd020042030 con 0x7fd02c103d60 2024-09-10T08:22:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.430+0000 7fd0363d6740 1 Processor -- start 2024-09-10T08:22:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.430+0000 7fd0363d6740 1 -- start start 2024-09-10T08:22:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.430+0000 7fd0363d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343ced7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.430+0000 7fd0363d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cedc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.430+0000 7fd0363d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343cedef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd0363d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56343ce3c100 con 0x56343ced7090 2024-09-10T08:22:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd0363d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56343cde41b0 con 0x56343cedca60 2024-09-10T08:22:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd0363d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56343ce380c0 con 0x56343ced79d0 2024-09-10T08:22:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd031600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343ced7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd030c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cedc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd031600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343ced7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46274/0 (socket says 172.21.15.2:46274) 2024-09-10T08:22:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd02be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343cedef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd031600640 1 -- 172.21.15.2:0/3481230028 learned_addr learned my addr 172.21.15.2:0/3481230028 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd031600640 1 -- 172.21.15.2:0/3481230028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 msgr2=0x56343cedc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd031600640 1 --2- 172.21.15.2:0/3481230028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cedc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd031600640 1 -- 172.21.15.2:0/3481230028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 msgr2=0x56343cedef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd031600640 1 --2- 172.21.15.2:0/3481230028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343cedef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd031600640 1 -- 172.21.15.2:0/3481230028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56343cb4f210 con 0x56343ced7090 2024-09-10T08:22:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.431+0000 7fd02be00640 1 --2- 172.21.15.2:0/3481230028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343cedef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.432+0000 7fd029600640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0140a2c30 0x7fd0140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.432+0000 7fd031600640 1 --2- 172.21.15.2:0/3481230028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343ced7490 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fd024009fa0 tx=0x7fd02400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9decfc1ceb8624eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.432+0000 7fd029600640 1 -- 172.21.15.2:0/2236834126 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd0200efb50 con 0x7fd02c103d60 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.432+0000 7fd01dc00640 1 -- 172.21.15.2:0/3481230028 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024015070 con 0x56343ced7090 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.432+0000 7fd030c00640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0140a2c30 0x7fd0140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.432+0000 7fd01dc00640 1 -- 172.21.15.2:0/3481230028 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02401ab40 con 0x56343ced7090 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.432+0000 7fd01dc00640 1 -- 172.21.15.2:0/3481230028 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024011d80 con 0x56343ced7090 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.433+0000 7fd0363d6740 1 -- 172.21.15.2:0/3481230028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 msgr2=0x56343ced7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.433+0000 7fd030c00640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0140a2c30 0x7fd0140a50f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd02002a910 tx=0x7fd0200f3590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.433+0000 7fd0363d6740 1 --2- 172.21.15.2:0/3481230028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343ced7490 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fd024009fa0 tx=0x7fd02400f7f0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.433+0000 7fd0363d6740 1 -- 172.21.15.2:0/3481230028 shutdown_connections 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.433+0000 7fd0363d6740 1 --2- 172.21.15.2:0/3481230028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343cedef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.433+0000 7fd0363d6740 1 --2- 172.21.15.2:0/3481230028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cedc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.433+0000 7fd0363d6740 1 --2- 172.21.15.2:0/3481230028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343ced7490 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.433+0000 7fd0363d6740 1 -- 172.21.15.2:0/3481230028 >> 172.21.15.2:0/3481230028 conn(0x56343ced2700 msgr2=0x56343ced2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.433+0000 7fd0363d6740 1 -- 172.21.15.2:0/3481230028 shutdown_connections 2024-09-10T08:22:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.433+0000 7fd0363d6740 1 -- 172.21.15.2:0/3481230028 wait complete. 2024-09-10T08:22:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.434+0000 7fd0363d6740 1 -- start start 2024-09-10T08:22:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.434+0000 7fd0363d6740 1 Processor -- start 2024-09-10T08:22:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.434+0000 7fd0363d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343cee1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.434+0000 7fd0363d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cee17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.434+0000 7fd0363d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343cee87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.434+0000 7fd0363d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56343cedfa30 con 0x56343ced7090 2024-09-10T08:22:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.434+0000 7fd0363d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56343cdc5090 con 0x56343cedca60 2024-09-10T08:22:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.434+0000 7fd0363d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56343cedfbb0 con 0x56343ced79d0 2024-09-10T08:22:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.434+0000 7fd0363d6740 10 client.0 _ll_register_callbacks cb 0x56343ce19f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.435+0000 7fd031600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343cee1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.435+0000 7fd031600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343cee1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46286/0 (socket says 172.21.15.2:46286) 2024-09-10T08:22:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.435+0000 7fd031600640 1 -- 172.21.15.2:0/3072520606 learned_addr learned my addr 172.21.15.2:0/3072520606 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.435+0000 7fd02be00640 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cee17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.435+0000 7fd031600640 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343cee87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.435+0000 7fd031600640 1 -- 172.21.15.2:0/3072520606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 msgr2=0x56343cee17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.435+0000 7fd031600640 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cee17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.435+0000 7fd031600640 1 -- 172.21.15.2:0/3072520606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 msgr2=0x56343cee87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.435+0000 7fd031600640 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343cee87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.435+0000 7fd031600640 1 -- 172.21.15.2:0/3072520606 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56343cc1b620 con 0x56343ced7090 2024-09-10T08:22:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.435+0000 7fd02be00640 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cee17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.435+0000 7fd031600640 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343cee1270 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fd024005cc0 tx=0x7fd024005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5479c6d6bc2fd79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.435+0000 7fd012000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:22:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.436+0000 7fd012000640 1 -- 172.21.15.2:0/3072520606 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024015050 con 0x56343ced7090 2024-09-10T08:22:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.436+0000 7fd012000640 1 -- 172.21.15.2:0/3072520606 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd024004c40 con 0x56343ced7090 2024-09-10T08:22:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.436+0000 7fd0363d6740 1 -- 172.21.15.2:0/3072520606 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd02401ab40 con 0x56343ced7090 2024-09-10T08:22:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.436+0000 7fd012000640 1 -- 172.21.15.2:0/3072520606 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd024004e40 con 0x56343ced7090 2024-09-10T08:22:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.436+0000 7fd0363d6740 10 client.19641 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.436+0000 7fd0363d6740 1 -- 172.21.15.2:0/3072520606 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56343ceeb5e0 con 0x56343ced7090 2024-09-10T08:22:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.436+0000 7fd012000640 1 -- 172.21.15.2:0/3072520606 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02401ab40 con 0x56343ced7090 2024-09-10T08:22:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.436+0000 7fd012000640 1 -- 172.21.15.2:0/3072520606 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd024027020 con 0x56343ced7090 2024-09-10T08:22:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.437+0000 7fd0363d6740 10 client.19641 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.437+0000 7fd0363d6740 10 client.19641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.437+0000 7fd0363d6740 20 client.19641 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.437+0000 7fd0363d6740 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56343ceeca40 0x56343ceeee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.437+0000 7fd0363d6740 4 client.19641 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:22:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.437+0000 7fd0363d6740 1 -- 172.21.15.2:0/3072520606 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56343ce49830 con 0x56343ceeca40 2024-09-10T08:22:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.437+0000 7fd02be00640 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56343ceeca40 0x56343ceeee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.438+0000 7fd02be00640 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56343ceeca40 0x56343ceeee30 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a22e936ab6e8cfd5 server_cookie=59cd4131fdb2d12a in_seq=0 out_seq=0 2024-09-10T08:22:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.438+0000 7fd012000640 10 client.19641 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.438+0000 7fd012000640 1 -- 172.21.15.2:0/3072520606 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56343ce49830 con 0x56343ceeca40 2024-09-10T08:22:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.438+0000 7fd012000640 10 client.19641 handle_command_reply: tid=0 2024-09-10T08:22:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.438+0000 7fd0363d6740 1 client.19641 shutdown 2024-09-10T08:22:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.439+0000 7fd0363d6740 1 -- 172.21.15.2:0/3072520606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 msgr2=0x56343cee1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.439+0000 7fd0363d6740 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343cee1270 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fd024005cc0 tx=0x7fd024005c50 comp rx=0 tx=0).stop 2024-09-10T08:22:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.439+0000 7fd0363d6740 1 -- 172.21.15.2:0/3072520606 shutdown_connections 2024-09-10T08:22:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.439+0000 7fd0363d6740 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56343ceeca40 0x56343ceeee30 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.439+0000 7fd0363d6740 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343cee87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.439+0000 7fd0363d6740 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cee17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.439+0000 7fd012000640 0 client.19641 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.439+0000 7fd0363d6740 1 --2- 172.21.15.2:0/3072520606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343cee1270 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.439+0000 7fd0363d6740 1 -- 172.21.15.2:0/3072520606 >> 172.21.15.2:0/3072520606 conn(0x56343ced2700 msgr2=0x56343ced3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.439+0000 7fd0363d6740 1 -- 172.21.15.2:0/3072520606 shutdown_connections 2024-09-10T08:22:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.439+0000 7fd0363d6740 1 -- 172.21.15.2:0/3072520606 wait complete. 2024-09-10T08:22:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.439+0000 7fd0363d6740 20 client.19641 trim_cache size 0 max 16384 2024-09-10T08:22:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.454+0000 7fd0363d6740 1 Processor -- start 2024-09-10T08:22:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.454+0000 7fd0363d6740 1 -- start start 2024-09-10T08:22:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd0363d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343cee1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd0363d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cee87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd0363d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343ced2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd0363d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1af380 con 0x56343ced7090 2024-09-10T08:22:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd0363d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1acc50 con 0x56343cedca60 2024-09-10T08:22:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd0363d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1aaaa0 con 0x56343ced79d0 2024-09-10T08:22:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd031600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343cee1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd02be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cee87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd031600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343cee1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46302/0 (socket says 172.21.15.2:46302) 2024-09-10T08:22:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd031600640 1 -- 172.21.15.2:0/1962535507 learned_addr learned my addr 172.21.15.2:0/1962535507 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd031600640 1 --2- 172.21.15.2:0/1962535507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343ced2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd02be00640 1 -- 172.21.15.2:0/1962535507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 msgr2=0x56343ced2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd02be00640 1 --2- 172.21.15.2:0/1962535507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343ced2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd02be00640 1 -- 172.21.15.2:0/1962535507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 msgr2=0x56343cee1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd02be00640 1 --2- 172.21.15.2:0/1962535507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343cee1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.455+0000 7fd02be00640 1 -- 172.21.15.2:0/1962535507 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56343cee02e0 con 0x56343ced79d0 2024-09-10T08:22:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.456+0000 7fd031600640 1 --2- 172.21.15.2:0/1962535507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343ced2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:22:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.456+0000 7fd02be00640 1 --2- 172.21.15.2:0/1962535507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cee87b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x56343ceef570 tx=0x7fd018005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc5729278b44f0e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.456+0000 7fd01d200640 1 -- 172.21.15.2:0/1962535507 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018011840 con 0x56343ced79d0 2024-09-10T08:22:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.456+0000 7fd01d200640 1 -- 172.21.15.2:0/1962535507 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01801e3f0 con 0x56343ced79d0 2024-09-10T08:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.456+0000 7fd01d200640 1 -- 172.21.15.2:0/1962535507 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018026620 con 0x56343ced79d0 2024-09-10T08:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.457+0000 7fd0363d6740 1 -- 172.21.15.2:0/1962535507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 msgr2=0x56343cee87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.457+0000 7fd0363d6740 1 --2- 172.21.15.2:0/1962535507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cee87b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x56343ceef570 tx=0x7fd018005f90 comp rx=0 tx=0).stop 2024-09-10T08:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.457+0000 7fd0363d6740 1 -- 172.21.15.2:0/1962535507 shutdown_connections 2024-09-10T08:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.457+0000 7fd0363d6740 1 --2- 172.21.15.2:0/1962535507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343cedca60 0x56343ced2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.457+0000 7fd0363d6740 1 --2- 172.21.15.2:0/1962535507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343ced79d0 0x56343cee87b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.457+0000 7fd0363d6740 1 --2- 172.21.15.2:0/1962535507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced7090 0x56343cee1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.457+0000 7fd0363d6740 1 -- 172.21.15.2:0/1962535507 >> 172.21.15.2:0/1962535507 conn(0x56343ced2700 msgr2=0x56343ced35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.457+0000 7fd0363d6740 1 -- 172.21.15.2:0/1962535507 shutdown_connections 2024-09-10T08:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.457+0000 7fd0363d6740 1 -- 172.21.15.2:0/1962535507 wait complete. 2024-09-10T08:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.457+0000 7fd0363d6740 1 -- start start 2024-09-10T08:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.458+0000 7fd0363d6740 1 Processor -- start 2024-09-10T08:22:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.458+0000 7fd0363d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343ced7090 0x56343cee5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.458+0000 7fd0363d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced79d0 0x56343cee6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.458+0000 7fd0363d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343cedca60 0x56343cee7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.458+0000 7fd0363d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1ab110 con 0x56343ced79d0 2024-09-10T08:22:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.458+0000 7fd0363d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1ac580 con 0x56343ced7090 2024-09-10T08:22:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.458+0000 7fd0363d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1ab7e0 con 0x56343cedca60 2024-09-10T08:22:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.458+0000 7fd0363d6740 10 client.0 _ll_register_callbacks cb 0x7fd018009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.458+0000 7fd02be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced79d0 0x56343cee6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.458+0000 7fd02be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced79d0 0x56343cee6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46306/0 (socket says 172.21.15.2:46306) 2024-09-10T08:22:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.458+0000 7fd02be00640 1 -- 172.21.15.2:0/1186046532 learned_addr learned my addr 172.21.15.2:0/1186046532 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.458+0000 7fd031600640 1 --2- 172.21.15.2:0/1186046532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343ced7090 0x56343cee5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.458+0000 7fd031600640 1 --2- 172.21.15.2:0/1186046532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343cedca60 0x56343cee7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.459+0000 7fd02be00640 1 -- 172.21.15.2:0/1186046532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343cedca60 msgr2=0x56343cee7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.459+0000 7fd02be00640 1 --2- 172.21.15.2:0/1186046532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343cedca60 0x56343cee7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.459+0000 7fd02be00640 1 -- 172.21.15.2:0/1186046532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343ced7090 msgr2=0x56343cee5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.459+0000 7fd02be00640 1 --2- 172.21.15.2:0/1186046532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343ced7090 0x56343cee5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.459+0000 7fd02be00640 1 -- 172.21.15.2:0/1186046532 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56343cee7660 con 0x56343ced79d0 2024-09-10T08:22:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.459+0000 7fd02be00640 1 --2- 172.21.15.2:0/1186046532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced79d0 0x56343cee6040 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd018014010 tx=0x7fd01801e620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7777118b7667379c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.459+0000 7fd012000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:22:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.459+0000 7fd012000640 1 -- 172.21.15.2:0/1186046532 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018034030 con 0x56343ced79d0 2024-09-10T08:22:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.459+0000 7fd0363d6740 1 -- 172.21.15.2:0/1186046532 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd01801e3f0 con 0x56343ced79d0 2024-09-10T08:22:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.459+0000 7fd012000640 1 -- 172.21.15.2:0/1186046532 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fd01801eb10 con 0x56343ced79d0 2024-09-10T08:22:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.460+0000 7fd012000640 1 -- 172.21.15.2:0/1186046532 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd01803e050 con 0x56343ced79d0 2024-09-10T08:22:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.460+0000 7fd0363d6740 10 client.19647 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.460+0000 7fd0363d6740 1 -- 172.21.15.2:0/1186046532 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56343cee88e0 con 0x56343ced79d0 2024-09-10T08:22:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.460+0000 7fd012000640 1 -- 172.21.15.2:0/1186046532 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01801e3f0 con 0x56343ced79d0 2024-09-10T08:22:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.460+0000 7fd012000640 1 -- 172.21.15.2:0/1186046532 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd018011640 con 0x56343ced79d0 2024-09-10T08:22:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.460+0000 7fd0363d6740 10 client.19647 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.460+0000 7fd0363d6740 10 client.19647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.460+0000 7fd0363d6740 20 client.19647 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.460+0000 7fd0363d6740 1 --2- 172.21.15.2:0/1186046532 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56343cf57b60 0x56343cf59f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.460+0000 7fd0363d6740 4 client.19647 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:22:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.460+0000 7fd0363d6740 1 -- 172.21.15.2:0/1186046532 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56343ce43b80 con 0x56343cf57b60 2024-09-10T08:22:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.461+0000 7fd031600640 1 --2- 172.21.15.2:0/1186046532 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56343cf57b60 0x56343cf59f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.461+0000 7fd031600640 1 --2- 172.21.15.2:0/1186046532 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56343cf57b60 0x56343cf59f80 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=807fb9ae80f2474b server_cookie=66d202afb01646f1 in_seq=0 out_seq=0 2024-09-10T08:22:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.461+0000 7fd012000640 10 client.19647 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.474+0000 7fd012000640 1 -- 172.21.15.2:0/1186046532 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56343ce43b80 con 0x56343cf57b60 2024-09-10T08:22:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.474+0000 7fd012000640 10 client.19647 handle_command_reply: tid=0 2024-09-10T08:22:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.474+0000 7fd0363d6740 1 client.19647 shutdown 2024-09-10T08:22:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.475+0000 7fd0363d6740 1 -- 172.21.15.2:0/1186046532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced79d0 msgr2=0x56343cee6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.475+0000 7fd0363d6740 1 --2- 172.21.15.2:0/1186046532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced79d0 0x56343cee6040 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd018014010 tx=0x7fd01801e620 comp rx=0 tx=0).stop 2024-09-10T08:22:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.475+0000 7fd0363d6740 1 -- 172.21.15.2:0/1186046532 shutdown_connections 2024-09-10T08:22:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.475+0000 7fd0363d6740 1 --2- 172.21.15.2:0/1186046532 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x56343cf57b60 0x56343cf59f80 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.475+0000 7fd0363d6740 1 --2- 172.21.15.2:0/1186046532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343cedca60 0x56343cee7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.475+0000 7fd0363d6740 1 --2- 172.21.15.2:0/1186046532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56343ced79d0 0x56343cee6040 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.475+0000 7fd0363d6740 1 --2- 172.21.15.2:0/1186046532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56343ced7090 0x56343cee5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.475+0000 7fd012000640 0 client.19647 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.475+0000 7fd0363d6740 1 -- 172.21.15.2:0/1186046532 >> 172.21.15.2:0/1186046532 conn(0x56343ced2700 msgr2=0x56343cee15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.475+0000 7fd0363d6740 1 -- 172.21.15.2:0/1186046532 shutdown_connections 2024-09-10T08:22:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.475+0000 7fd0363d6740 1 -- 172.21.15.2:0/1186046532 wait complete. 2024-09-10T08:22:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.475+0000 7fd0363d6740 20 client.19647 trim_cache size 0 max 16384 2024-09-10T08:22:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:22:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:22:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:22:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:22:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.476+0000 7fd033600640 1 -- 172.21.15.2:0/2236834126 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0140a2c30 msgr2=0x7fd0140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.476+0000 7fd033600640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0140a2c30 0x7fd0140a50f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd02002a910 tx=0x7fd0200f3590 comp rx=0 tx=0).stop 2024-09-10T08:22:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.476+0000 7fd033600640 1 -- 172.21.15.2:0/2236834126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd02c103d60 msgr2=0x7fd02c19a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.476+0000 7fd033600640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd02c103d60 0x7fd02c19a000 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd020031440 tx=0x7fd020032ce0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.476+0000 7fd033600640 1 -- 172.21.15.2:0/2236834126 shutdown_connections 2024-09-10T08:22:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.476+0000 7fd033600640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02c105910 0x7fd02c1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.476+0000 7fd033600640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd0140a2c30 0x7fd0140a50f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.476+0000 7fd033600640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd02c104f50 0x7fd02c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.476+0000 7fd033600640 1 --2- 172.21.15.2:0/2236834126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd02c103d60 0x7fd02c19a000 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.476+0000 7fd033600640 1 -- 172.21.15.2:0/2236834126 >> 172.21.15.2:0/2236834126 conn(0x7fd02c0ff4e0 msgr2=0x7fd02c108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.476+0000 7fd033600640 1 -- 172.21.15.2:0/2236834126 shutdown_connections 2024-09-10T08:22:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:46.476+0000 7fd033600640 1 -- 172.21.15.2:0/2236834126 wait complete. 2024-09-10T08:22:47.025 INFO:tasks.cephfs.filesystem:scrub status for tag:5b2727d8-e369-423f-8fd6-d085c534eee3 - {'path': '/', 'tag': '5b2727d8-e369-423f-8fd6-d085c534eee3', 'options': 'recursive,force'} 2024-09-10T08:22:47.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:47 smithi155 ceph-mon[35405]: pgmap v826: 129 pgs: 129 active+clean; 3.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:22:47.668 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:47 smithi107 ceph-mon[35048]: pgmap v826: 129 pgs: 129 active+clean; 3.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:22:47.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:47 smithi002 ceph-mon[30563]: pgmap v826: 129 pgs: 129 active+clean; 3.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:22:49.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:49 smithi155 ceph-mon[35405]: pgmap v827: 129 pgs: 129 active+clean; 3.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:22:49.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:49 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:22:49.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:49 smithi107 ceph-mon[35048]: pgmap v827: 129 pgs: 129 active+clean; 3.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:22:49.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:49 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:22:49.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:49 smithi002 ceph-mon[30563]: pgmap v827: 129 pgs: 129 active+clean; 3.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:22:49.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:49 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:22:51.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:51 smithi155 ceph-mon[35405]: pgmap v828: 129 pgs: 129 active+clean; 3.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:22:51.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:51 smithi107 ceph-mon[35048]: pgmap v828: 129 pgs: 129 active+clean; 3.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:22:51.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:51 smithi002 ceph-mon[30563]: pgmap v828: 129 pgs: 129 active+clean; 3.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:22:53.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:53 smithi155 ceph-mon[35405]: pgmap v829: 129 pgs: 129 active+clean; 3.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:22:53.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:53 smithi107 ceph-mon[35048]: pgmap v829: 129 pgs: 129 active+clean; 3.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:22:53.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:53 smithi002 ceph-mon[30563]: pgmap v829: 129 pgs: 129 active+clean; 3.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-10T08:22:54.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:22:54.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:22:54.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:22:54.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:22:54.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:22:54.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:22:54.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:22:54.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:22:54.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:22:54.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:22:54.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:22:54.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:22:55.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:55 smithi155 ceph-mon[35405]: pgmap v830: 129 pgs: 129 active+clean; 3.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:22:55.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:55 smithi107 ceph-mon[35048]: pgmap v830: 129 pgs: 129 active+clean; 3.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:22:55.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:55 smithi002 ceph-mon[30563]: pgmap v830: 129 pgs: 129 active+clean; 3.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-10T08:22:57.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:57 smithi155 ceph-mon[35405]: pgmap v831: 129 pgs: 129 active+clean; 3.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:22:57.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:57 smithi107 ceph-mon[35048]: pgmap v831: 129 pgs: 129 active+clean; 3.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:22:57.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:57 smithi002 ceph-mon[30563]: pgmap v831: 129 pgs: 129 active+clean; 3.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:22:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.605+0000 7fbfc0800640 1 -- 172.21.15.2:0/2835861201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb8104f50 msgr2=0x7fbfb81053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.605+0000 7fbfc0800640 1 --2- 172.21.15.2:0/2835861201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb8104f50 0x7fbfb81053d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009900 tx=0x7fbfa802a400 comp rx=0 tx=0).stop 2024-09-10T08:22:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.605+0000 7fbfc0800640 1 -- 172.21.15.2:0/2835861201 shutdown_connections 2024-09-10T08:22:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.605+0000 7fbfc0800640 1 --2- 172.21.15.2:0/2835861201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb8105910 0x7fbfb810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.605+0000 7fbfc0800640 1 --2- 172.21.15.2:0/2835861201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb8104f50 0x7fbfb81053d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.605+0000 7fbfc0800640 1 --2- 172.21.15.2:0/2835861201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb8103d60 0x7fbfb8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.605+0000 7fbfc0800640 1 -- 172.21.15.2:0/2835861201 >> 172.21.15.2:0/2835861201 conn(0x7fbfb80ff4e0 msgr2=0x7fbfb8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.605+0000 7fbfc0800640 1 -- 172.21.15.2:0/2835861201 shutdown_connections 2024-09-10T08:22:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.605+0000 7fbfc0800640 1 -- 172.21.15.2:0/2835861201 wait complete. 2024-09-10T08:22:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.606+0000 7fbfc0800640 1 Processor -- start 2024-09-10T08:22:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.607+0000 7fbfc0800640 1 -- start start 2024-09-10T08:22:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.607+0000 7fbfc0800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb8103d60 0x7fbfb8199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.607+0000 7fbfc0800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb8104f50 0x7fbfb819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.607+0000 7fbfc0800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb8105910 0x7fbfb81a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.607+0000 7fbfc0800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb810fa10 con 0x7fbfb8103d60 2024-09-10T08:22:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.607+0000 7fbfc0800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb810f890 con 0x7fbfb8105910 2024-09-10T08:22:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.607+0000 7fbfc0800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb810fb90 con 0x7fbfb8104f50 2024-09-10T08:22:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.607+0000 7fbfbde00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb8103d60 0x7fbfb8199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.607+0000 7fbfbd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb8104f50 0x7fbfb819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.607+0000 7fbfbd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb8104f50 0x7fbfb819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:54560/0 (socket says 172.21.15.2:54560) 2024-09-10T08:22:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.607+0000 7fbfbd400640 1 -- 172.21.15.2:0/938786032 learned_addr learned my addr 172.21.15.2:0/938786032 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.607+0000 7fbfbe800640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb8105910 0x7fbfb81a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.608+0000 7fbfbd400640 1 -- 172.21.15.2:0/938786032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb8105910 msgr2=0x7fbfb81a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.608+0000 7fbfbd400640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb8105910 0x7fbfb81a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.608+0000 7fbfbd400640 1 -- 172.21.15.2:0/938786032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb8103d60 msgr2=0x7fbfb8199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.608+0000 7fbfbd400640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb8103d60 0x7fbfb8199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.608+0000 7fbfbd400640 1 -- 172.21.15.2:0/938786032 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb81a1bf0 con 0x7fbfb8104f50 2024-09-10T08:22:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.608+0000 7fbfbde00640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb8103d60 0x7fbfb8199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.608+0000 7fbfbd400640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb8104f50 0x7fbfb819a510 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8031440 tx=0x7fbfa8032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.608+0000 7fbfb2a00640 1 -- 172.21.15.2:0/938786032 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa803f030 con 0x7fbfb8104f50 2024-09-10T08:22:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.608+0000 7fbfb2a00640 1 -- 172.21.15.2:0/938786032 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfa8003ce0 con 0x7fbfb8104f50 2024-09-10T08:22:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.608+0000 7fbfb2a00640 1 -- 172.21.15.2:0/938786032 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8035070 con 0x7fbfb8104f50 2024-09-10T08:22:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.608+0000 7fbfc0800640 1 -- 172.21.15.2:0/938786032 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb81a1e80 con 0x7fbfb8104f50 2024-09-10T08:22:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.609+0000 7fbfc0800640 1 -- 172.21.15.2:0/938786032 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb81a22f0 con 0x7fbfb8104f50 2024-09-10T08:22:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.611+0000 7fbfc382f740 1 Processor -- start 2024-09-10T08:22:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfc382f740 1 -- start start 2024-09-10T08:22:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfb2a00640 1 -- 172.21.15.2:0/938786032 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbfa8042030 con 0x7fbfb8104f50 2024-09-10T08:22:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfc382f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a0f090 0x557703a0f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfc382f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 0x557703a14490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfc382f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a14a60 0x557703a16f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfc382f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557703974100 con 0x557703a0f090 2024-09-10T08:22:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfc382f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55770391c1b0 con 0x557703a0f9d0 2024-09-10T08:22:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfc382f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5577039700c0 con 0x557703a14a60 2024-09-10T08:22:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfbe800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a0f090 0x557703a0f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfbde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 0x557703a14490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfbe800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a0f090 0x557703a0f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50532/0 (socket says 172.21.15.2:50532) 2024-09-10T08:22:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfbde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 0x557703a14490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:57054/0 (socket says 172.21.15.2:57054) 2024-09-10T08:22:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfbe800640 1 -- 172.21.15.2:0/1591826531 learned_addr learned my addr 172.21.15.2:0/1591826531 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.612+0000 7fbfbd400640 1 --2- 172.21.15.2:0/1591826531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a14a60 0x557703a16f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.613+0000 7fbfbde00640 1 -- 172.21.15.2:0/1591826531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a14a60 msgr2=0x557703a16f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.613+0000 7fbfbde00640 1 --2- 172.21.15.2:0/1591826531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a14a60 0x557703a16f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.613+0000 7fbfbde00640 1 -- 172.21.15.2:0/1591826531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a0f090 msgr2=0x557703a0f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.613+0000 7fbfbde00640 1 --2- 172.21.15.2:0/1591826531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a0f090 0x557703a0f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.613+0000 7fbfbde00640 1 -- 172.21.15.2:0/1591826531 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557703687210 con 0x557703a0f9d0 2024-09-10T08:22:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.613+0000 7fbfbde00640 1 --2- 172.21.15.2:0/1591826531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 0x557703a14490 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009fa0 tx=0x7fbfac0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ea821ae85d7c1b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.613+0000 7fbfa7000640 1 -- 172.21.15.2:0/1591826531 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac004310 con 0x557703a0f9d0 2024-09-10T08:22:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.613+0000 7fbfa7000640 1 -- 172.21.15.2:0/1591826531 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfac0044b0 con 0x557703a0f9d0 2024-09-10T08:22:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.613+0000 7fbfa7000640 1 -- 172.21.15.2:0/1591826531 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0035f0 con 0x557703a0f9d0 2024-09-10T08:22:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.613+0000 7fbfb2a00640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbfa00a2cb0 0x7fbfa00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.614+0000 7fbfb2a00640 1 -- 172.21.15.2:0/938786032 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbfa80ef850 con 0x7fbfb8104f50 2024-09-10T08:22:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.614+0000 7fbfbe800640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbfa00a2cb0 0x7fbfa00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.614+0000 7fbfc382f740 1 -- 172.21.15.2:0/1591826531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 msgr2=0x557703a14490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.614+0000 7fbfc382f740 1 --2- 172.21.15.2:0/1591826531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 0x557703a14490 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009fa0 tx=0x7fbfac0118d0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.614+0000 7fbfc382f740 1 -- 172.21.15.2:0/1591826531 shutdown_connections 2024-09-10T08:22:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.614+0000 7fbfc382f740 1 --2- 172.21.15.2:0/1591826531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a14a60 0x557703a16f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.614+0000 7fbfc382f740 1 --2- 172.21.15.2:0/1591826531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 0x557703a14490 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.614+0000 7fbfc382f740 1 --2- 172.21.15.2:0/1591826531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a0f090 0x557703a0f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.614+0000 7fbfc382f740 1 -- 172.21.15.2:0/1591826531 >> 172.21.15.2:0/1591826531 conn(0x557703a0a700 msgr2=0x557703a0ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.614+0000 7fbfc382f740 1 -- 172.21.15.2:0/1591826531 shutdown_connections 2024-09-10T08:22:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.614+0000 7fbfbe800640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbfa00a2cb0 0x7fbfa00a5170 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40116b0 tx=0x7fbfb4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.614+0000 7fbfc382f740 1 -- 172.21.15.2:0/1591826531 wait complete. 2024-09-10T08:22:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.615+0000 7fbfc382f740 1 -- start start 2024-09-10T08:22:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.615+0000 7fbfc382f740 1 Processor -- start 2024-09-10T08:22:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.615+0000 7fbfc382f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a0f090 0x557703a19300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.615+0000 7fbfc382f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 0x557703a19840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfc382f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a14a60 0x557703a20810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfc382f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557703a17a30 con 0x557703a14a60 2024-09-10T08:22:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfc382f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5577038fd090 con 0x557703a0f9d0 2024-09-10T08:22:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfc382f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557703a17bb0 con 0x557703a0f090 2024-09-10T08:22:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfc382f740 10 client.0 _ll_register_callbacks cb 0x557703951f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfbd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 0x557703a19840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfbd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 0x557703a19840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:57064/0 (socket says 172.21.15.2:57064) 2024-09-10T08:22:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfbd400640 1 -- 172.21.15.2:0/550577311 learned_addr learned my addr 172.21.15.2:0/550577311 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfbde00640 1 --2- 172.21.15.2:0/550577311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a0f090 0x557703a19300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfbe800640 1 --2- 172.21.15.2:0/550577311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a14a60 0x557703a20810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfbd400640 1 -- 172.21.15.2:0/550577311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a0f090 msgr2=0x557703a19300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfbd400640 1 --2- 172.21.15.2:0/550577311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a0f090 0x557703a19300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfbd400640 1 -- 172.21.15.2:0/550577311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a14a60 msgr2=0x557703a20810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfbd400640 1 --2- 172.21.15.2:0/550577311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a14a60 0x557703a20810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfbd400640 1 -- 172.21.15.2:0/550577311 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557703753620 con 0x557703a0f9d0 2024-09-10T08:22:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbfbd400640 1 --2- 172.21.15.2:0/550577311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 0x557703a19840 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80bba10 tx=0x7fbfa8009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb581864dc202ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.616+0000 7fbf9b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:22:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.617+0000 7fbf9b400640 1 -- 172.21.15.2:0/550577311 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8049030 con 0x557703a0f9d0 2024-09-10T08:22:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.617+0000 7fbfc382f740 1 -- 172.21.15.2:0/550577311 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfac0044b0 con 0x557703a0f9d0 2024-09-10T08:22:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.617+0000 7fbf9b400640 1 -- 172.21.15.2:0/550577311 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbfa803bcd0 con 0x557703a0f9d0 2024-09-10T08:22:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.617+0000 7fbf9b400640 1 -- 172.21.15.2:0/550577311 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbfa80bb670 con 0x557703a0f9d0 2024-09-10T08:22:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.617+0000 7fbfc382f740 10 client.29681 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.617+0000 7fbfc382f740 1 -- 172.21.15.2:0/550577311 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557703a23600 con 0x557703a0f9d0 2024-09-10T08:22:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.617+0000 7fbf9b400640 1 -- 172.21.15.2:0/550577311 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfac0044b0 con 0x557703a0f9d0 2024-09-10T08:22:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.617+0000 7fbf9b400640 1 -- 172.21.15.2:0/550577311 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbfa803b460 con 0x557703a0f9d0 2024-09-10T08:22:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.617+0000 7fbfc382f740 10 client.29681 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.617+0000 7fbfc382f740 10 client.29681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.618+0000 7fbfc382f740 20 client.29681 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.618+0000 7fbfc382f740 1 --2- 172.21.15.2:0/550577311 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557703a24dd0 0x557703a271c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.618+0000 7fbfc382f740 4 client.29681 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:22:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.618+0000 7fbfc382f740 1 -- 172.21.15.2:0/550577311 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557703981830 con 0x557703a24dd0 2024-09-10T08:22:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.618+0000 7fbfbde00640 1 --2- 172.21.15.2:0/550577311 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557703a24dd0 0x557703a271c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.618+0000 7fbfbde00640 1 --2- 172.21.15.2:0/550577311 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557703a24dd0 0x557703a271c0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e6bc0a8eebd934d server_cookie=7cc2edfcd0230a5a in_seq=0 out_seq=0 2024-09-10T08:22:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.618+0000 7fbf9b400640 10 client.29681 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.619+0000 7fbf9b400640 1 -- 172.21.15.2:0/550577311 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557703981830 con 0x557703a24dd0 2024-09-10T08:22:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.619+0000 7fbf9b400640 10 client.29681 handle_command_reply: tid=0 2024-09-10T08:22:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.620+0000 7fbfc382f740 1 client.29681 shutdown 2024-09-10T08:22:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.620+0000 7fbfc382f740 1 -- 172.21.15.2:0/550577311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 msgr2=0x557703a19840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.620+0000 7fbfc382f740 1 --2- 172.21.15.2:0/550577311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 0x557703a19840 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80bba10 tx=0x7fbfa8009620 comp rx=0 tx=0).stop 2024-09-10T08:22:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.620+0000 7fbfc382f740 1 -- 172.21.15.2:0/550577311 shutdown_connections 2024-09-10T08:22:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.620+0000 7fbfc382f740 1 --2- 172.21.15.2:0/550577311 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557703a24dd0 0x557703a271c0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.620+0000 7fbfc382f740 1 --2- 172.21.15.2:0/550577311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a14a60 0x557703a20810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.620+0000 7fbfc382f740 1 --2- 172.21.15.2:0/550577311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f9d0 0x557703a19840 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.620+0000 7fbfc382f740 1 --2- 172.21.15.2:0/550577311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a0f090 0x557703a19300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.620+0000 7fbf9b400640 0 client.29681 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.620+0000 7fbfc382f740 1 -- 172.21.15.2:0/550577311 >> 172.21.15.2:0/550577311 conn(0x557703a0a700 msgr2=0x557703a0bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.620+0000 7fbfc382f740 1 -- 172.21.15.2:0/550577311 shutdown_connections 2024-09-10T08:22:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.620+0000 7fbfc382f740 1 -- 172.21.15.2:0/550577311 wait complete. 2024-09-10T08:22:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.620+0000 7fbfc382f740 20 client.29681 trim_cache size 0 max 16384 2024-09-10T08:22:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.635+0000 7fbfc382f740 1 Processor -- start 2024-09-10T08:22:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.635+0000 7fbfc382f740 1 -- start start 2024-09-10T08:22:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfc382f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f090 0x557703a2b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfc382f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a0f9d0 0x557703a207d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfc382f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a14a60 0x557703a2bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfc382f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81af2b0 con 0x557703a0f9d0 2024-09-10T08:22:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfc382f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81acb80 con 0x557703a0f090 2024-09-10T08:22:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfc382f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81aa9d0 con 0x557703a14a60 2024-09-10T08:22:59.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfbd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a0f9d0 0x557703a207d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfbd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a0f9d0 0x557703a207d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50556/0 (socket says 172.21.15.2:50556) 2024-09-10T08:22:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfbd400640 1 -- 172.21.15.2:0/2013906044 learned_addr learned my addr 172.21.15.2:0/2013906044 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfbde00640 1 --2- 172.21.15.2:0/2013906044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f090 0x557703a2b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfbe800640 1 --2- 172.21.15.2:0/2013906044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a14a60 0x557703a2bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfbd400640 1 -- 172.21.15.2:0/2013906044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a14a60 msgr2=0x557703a2bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfbd400640 1 --2- 172.21.15.2:0/2013906044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a14a60 0x557703a2bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfbd400640 1 -- 172.21.15.2:0/2013906044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f090 msgr2=0x557703a2b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfbd400640 1 --2- 172.21.15.2:0/2013906044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f090 0x557703a2b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.636+0000 7fbfbd400640 1 -- 172.21.15.2:0/2013906044 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557703a0a520 con 0x557703a0f9d0 2024-09-10T08:22:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.637+0000 7fbfbd400640 1 --2- 172.21.15.2:0/2013906044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a0f9d0 0x557703a207d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8006010 tx=0x7fbfa802a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d474659f79a96905 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.637+0000 7fbfa6600640 1 -- 172.21.15.2:0/2013906044 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8049070 con 0x557703a0f9d0 2024-09-10T08:22:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.637+0000 7fbfa6600640 1 -- 172.21.15.2:0/2013906044 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfa80d8020 con 0x557703a0f9d0 2024-09-10T08:22:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.637+0000 7fbfa6600640 1 -- 172.21.15.2:0/2013906044 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80d4e60 con 0x557703a0f9d0 2024-09-10T08:22:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.637+0000 7fbfc382f740 1 -- 172.21.15.2:0/2013906044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a0f9d0 msgr2=0x557703a207d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.637+0000 7fbfc382f740 1 --2- 172.21.15.2:0/2013906044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a0f9d0 0x557703a207d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8006010 tx=0x7fbfa802a400 comp rx=0 tx=0).stop 2024-09-10T08:22:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.638+0000 7fbfc382f740 1 -- 172.21.15.2:0/2013906044 shutdown_connections 2024-09-10T08:22:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.638+0000 7fbfc382f740 1 --2- 172.21.15.2:0/2013906044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a14a60 0x557703a2bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.638+0000 7fbfc382f740 1 --2- 172.21.15.2:0/2013906044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a0f9d0 0x557703a207d0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.638+0000 7fbfc382f740 1 --2- 172.21.15.2:0/2013906044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a0f090 0x557703a2b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.638+0000 7fbfc382f740 1 -- 172.21.15.2:0/2013906044 >> 172.21.15.2:0/2013906044 conn(0x557703a0a700 msgr2=0x557703a09880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.638+0000 7fbfc382f740 1 -- 172.21.15.2:0/2013906044 shutdown_connections 2024-09-10T08:22:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.638+0000 7fbfc382f740 1 -- 172.21.15.2:0/2013906044 wait complete. 2024-09-10T08:22:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.638+0000 7fbfc382f740 1 -- start start 2024-09-10T08:22:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.638+0000 7fbfc382f740 1 Processor -- start 2024-09-10T08:22:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.638+0000 7fbfc382f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a0f090 0x557703a1e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.638+0000 7fbfc382f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a14a60 0x557703a1ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfc382f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a1fbf0 0x557703a20050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfc382f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81ab040 con 0x557703a14a60 2024-09-10T08:22:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfc382f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81ac4b0 con 0x557703a1fbf0 2024-09-10T08:22:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfc382f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81ab710 con 0x557703a0f090 2024-09-10T08:22:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfc382f740 10 client.0 _ll_register_callbacks cb 0x7fbfa80f5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:22:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfbd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a14a60 0x557703a1ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfbe800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a1fbf0 0x557703a20050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfbe800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a1fbf0 0x557703a20050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:57080/0 (socket says 172.21.15.2:57080) 2024-09-10T08:22:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfbe800640 1 -- 172.21.15.2:0/1532257147 learned_addr learned my addr 172.21.15.2:0/1532257147 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:22:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfbde00640 1 --2- 172.21.15.2:0/1532257147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a0f090 0x557703a1e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfbe800640 1 -- 172.21.15.2:0/1532257147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a0f090 msgr2=0x557703a1e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfbe800640 1 --2- 172.21.15.2:0/1532257147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a0f090 0x557703a1e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfbe800640 1 -- 172.21.15.2:0/1532257147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a14a60 msgr2=0x557703a1ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:22:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfbe800640 1 --2- 172.21.15.2:0/1532257147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a14a60 0x557703a1ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfbe800640 1 -- 172.21.15.2:0/1532257147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557703a205e0 con 0x557703a1fbf0 2024-09-10T08:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.639+0000 7fbfbd400640 1 --2- 172.21.15.2:0/1532257147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a14a60 0x557703a1ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.640+0000 7fbfbe800640 1 --2- 172.21.15.2:0/1532257147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a1fbf0 0x557703a20050 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400f850 tx=0x7fbfb4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fafc52382b06816b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.640+0000 7fbf9b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.640+0000 7fbf9b400640 1 -- 172.21.15.2:0/1532257147 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4003b80 con 0x557703a1fbf0 2024-09-10T08:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.640+0000 7fbfc382f740 1 -- 172.21.15.2:0/1532257147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa80d8020 con 0x557703a1fbf0 2024-09-10T08:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.640+0000 7fbf9b400640 1 -- 172.21.15.2:0/1532257147 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbfb4003d20 con 0x557703a1fbf0 2024-09-10T08:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.640+0000 7fbf9b400640 1 -- 172.21.15.2:0/1532257147 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbfb4004050 con 0x557703a1fbf0 2024-09-10T08:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.640+0000 7fbfc382f740 10 client.29687 fetch_fsmap learned FSMap version 26 2024-09-10T08:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.640+0000 7fbfc382f740 1 -- 172.21.15.2:0/1532257147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557703a90680 con 0x557703a1fbf0 2024-09-10T08:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.640+0000 7fbf9b400640 1 -- 172.21.15.2:0/1532257147 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfa80d8020 con 0x557703a1fbf0 2024-09-10T08:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.641+0000 7fbf9b400640 1 -- 172.21.15.2:0/1532257147 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbfb40298e0 con 0x557703a1fbf0 2024-09-10T08:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.641+0000 7fbfc382f740 10 client.29687 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.641+0000 7fbfc382f740 10 client.29687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.641+0000 7fbfc382f740 20 client.29687 populate_metadata read hostname 'smithi002' 2024-09-10T08:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.641+0000 7fbfc382f740 1 --2- 172.21.15.2:0/1532257147 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557703a91060 0x557703a93480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.641+0000 7fbfc382f740 4 client.29687 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.641+0000 7fbfc382f740 1 -- 172.21.15.2:0/1532257147 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55770397bb80 con 0x557703a91060 2024-09-10T08:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.641+0000 7fbfbde00640 1 --2- 172.21.15.2:0/1532257147 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557703a91060 0x557703a93480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.641+0000 7fbfbde00640 1 --2- 172.21.15.2:0/1532257147 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557703a91060 0x557703a93480 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=178e967c46398f4e server_cookie=490d213b55ce1d95 in_seq=0 out_seq=0 2024-09-10T08:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.641+0000 7fbf9b400640 10 client.29687 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:59.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:22:59 smithi155 ceph-mon[35405]: pgmap v832: 129 pgs: 129 active+clean; 3.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:22:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbf9b400640 1 -- 172.21.15.2:0/1532257147 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55770397bb80 con 0x557703a91060 2024-09-10T08:22:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbf9b400640 10 client.29687 handle_command_reply: tid=0 2024-09-10T08:22:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbfc382f740 1 client.29687 shutdown 2024-09-10T08:22:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbfc382f740 1 -- 172.21.15.2:0/1532257147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a1fbf0 msgr2=0x557703a20050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbfc382f740 1 --2- 172.21.15.2:0/1532257147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a1fbf0 0x557703a20050 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400f850 tx=0x7fbfb4004a40 comp rx=0 tx=0).stop 2024-09-10T08:22:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbfc382f740 1 -- 172.21.15.2:0/1532257147 shutdown_connections 2024-09-10T08:22:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbfc382f740 1 --2- 172.21.15.2:0/1532257147 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x557703a91060 0x557703a93480 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbfc382f740 1 --2- 172.21.15.2:0/1532257147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557703a1fbf0 0x557703a20050 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbfc382f740 1 --2- 172.21.15.2:0/1532257147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557703a14a60 0x557703a1ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbf9b400640 0 client.29687 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:22:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbfc382f740 1 --2- 172.21.15.2:0/1532257147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557703a0f090 0x557703a1e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbfc382f740 1 -- 172.21.15.2:0/1532257147 >> 172.21.15.2:0/1532257147 conn(0x557703a0a700 msgr2=0x557703a0b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbfc382f740 1 -- 172.21.15.2:0/1532257147 shutdown_connections 2024-09-10T08:22:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbfc382f740 1 -- 172.21.15.2:0/1532257147 wait complete. 2024-09-10T08:22:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.657+0000 7fbfc382f740 20 client.29687 trim_cache size 0 max 16384 2024-09-10T08:22:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:22:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:22:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:22:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:22:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.658+0000 7fbfc0800640 1 -- 172.21.15.2:0/938786032 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbfa00a2cb0 msgr2=0x7fbfa00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.658+0000 7fbfc0800640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbfa00a2cb0 0x7fbfa00a5170 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40116b0 tx=0x7fbfb4009500 comp rx=0 tx=0).stop 2024-09-10T08:22:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.658+0000 7fbfc0800640 1 -- 172.21.15.2:0/938786032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb8104f50 msgr2=0x7fbfb819a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:22:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.658+0000 7fbfc0800640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb8104f50 0x7fbfb819a510 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8031440 tx=0x7fbfa8032ce0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.658+0000 7fbfc0800640 1 -- 172.21.15.2:0/938786032 shutdown_connections 2024-09-10T08:22:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.658+0000 7fbfc0800640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb8105910 0x7fbfb81a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.658+0000 7fbfc0800640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbfa00a2cb0 0x7fbfa00a5170 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.658+0000 7fbfc0800640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb8104f50 0x7fbfb819a510 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.658+0000 7fbfc0800640 1 --2- 172.21.15.2:0/938786032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb8103d60 0x7fbfb8199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:22:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.658+0000 7fbfc0800640 1 -- 172.21.15.2:0/938786032 >> 172.21.15.2:0/938786032 conn(0x7fbfb80ff4e0 msgr2=0x7fbfb8108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:22:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.658+0000 7fbfc0800640 1 -- 172.21.15.2:0/938786032 shutdown_connections 2024-09-10T08:22:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:22:59.658+0000 7fbfc0800640 1 -- 172.21.15.2:0/938786032 wait complete. 2024-09-10T08:22:59.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:22:59 smithi107 ceph-mon[35048]: pgmap v832: 129 pgs: 129 active+clean; 3.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:22:59.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:22:59 smithi002 ceph-mon[30563]: pgmap v832: 129 pgs: 129 active+clean; 3.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:23:01.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:01 smithi155 ceph-mon[35405]: pgmap v833: 129 pgs: 129 active+clean; 3.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:23:01.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:01 smithi107 ceph-mon[35048]: pgmap v833: 129 pgs: 129 active+clean; 3.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:23:01.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:01 smithi002 ceph-mon[30563]: pgmap v833: 129 pgs: 129 active+clean; 3.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:23:03.657 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:03 smithi155 ceph-mon[35405]: pgmap v834: 129 pgs: 129 active+clean; 3.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:23:03.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:03 smithi107 ceph-mon[35048]: pgmap v834: 129 pgs: 129 active+clean; 3.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:23:03.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:03 smithi002 ceph-mon[30563]: pgmap v834: 129 pgs: 129 active+clean; 3.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-10T08:23:05.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:05 smithi002 ceph-mon[30563]: pgmap v835: 129 pgs: 129 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:23:05.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:05 smithi155 ceph-mon[35405]: pgmap v835: 129 pgs: 129 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:23:05.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:05 smithi107 ceph-mon[35048]: pgmap v835: 129 pgs: 129 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-10T08:23:07.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:07 smithi155 ceph-mon[35405]: pgmap v836: 129 pgs: 129 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:23:07.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:07 smithi107 ceph-mon[35048]: pgmap v836: 129 pgs: 129 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:23:07.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:07 smithi002 ceph-mon[30563]: pgmap v836: 129 pgs: 129 active+clean; 3.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:23:08.843 INFO:tasks.workunit.client.0.smithi002.stdout:499/600: open fd = 3 2024-09-10T08:23:08.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.843 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.843 INFO:tasks.workunit.client.0.smithi002.stdout:500/600: open fd = 3 2024-09-10T08:23:08.843 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.844 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.844 INFO:tasks.workunit.client.0.smithi002.stdout:501/600: open fd = 3 2024-09-10T08:23:08.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.844 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.844 INFO:tasks.workunit.client.0.smithi002.stdout:502/600: open fd = 3 2024-09-10T08:23:08.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.844 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.844 INFO:tasks.workunit.client.0.smithi002.stdout:503/600: open fd = 3 2024-09-10T08:23:08.844 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.845 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.845 INFO:tasks.workunit.client.0.smithi002.stdout:504/600: open fd = 3 2024-09-10T08:23:08.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.845 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.845 INFO:tasks.workunit.client.0.smithi002.stdout:505/600: open fd = 3 2024-09-10T08:23:08.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.845 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.845 INFO:tasks.workunit.client.0.smithi002.stdout:506/600: open fd = 3 2024-09-10T08:23:08.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.845 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.846 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.846 INFO:tasks.workunit.client.0.smithi002.stdout:507/600: open fd = 3 2024-09-10T08:23:08.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.846 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.846 INFO:tasks.workunit.client.0.smithi002.stdout:508/600: open fd = 3 2024-09-10T08:23:08.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.846 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.846 INFO:tasks.workunit.client.0.smithi002.stdout:509/600: open fd = 3 2024-09-10T08:23:08.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.846 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:510/600: open fd = 3 2024-09-10T08:23:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:511/600: open fd = 3 2024-09-10T08:23:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:512/600: open fd = 3 2024-09-10T08:23:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:513/600: open fd = 3 2024-09-10T08:23:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:514/600: open fd = 3 2024-09-10T08:23:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:515/600: open fd = 3 2024-09-10T08:23:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.849 INFO:tasks.workunit.client.0.smithi002.stdout:516/600: open fd = 3 2024-09-10T08:23:08.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.849 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.849 INFO:tasks.workunit.client.0.smithi002.stdout:517/600: open fd = 3 2024-09-10T08:23:08.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.849 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.849 INFO:tasks.workunit.client.0.smithi002.stdout:518/600: open fd = 3 2024-09-10T08:23:08.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.849 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.849 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.850 INFO:tasks.workunit.client.0.smithi002.stdout:519/600: open fd = 3 2024-09-10T08:23:08.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.850 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.850 INFO:tasks.workunit.client.0.smithi002.stdout:520/600: open fd = 3 2024-09-10T08:23:08.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.850 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.850 INFO:tasks.workunit.client.0.smithi002.stdout:521/600: open fd = 3 2024-09-10T08:23:08.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.850 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.850 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.850 INFO:tasks.workunit.client.0.smithi002.stdout:522/600: open fd = 3 2024-09-10T08:23:08.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.851 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.851 INFO:tasks.workunit.client.0.smithi002.stdout:523/600: open fd = 3 2024-09-10T08:23:08.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.851 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.851 INFO:tasks.workunit.client.0.smithi002.stdout:524/600: open fd = 3 2024-09-10T08:23:08.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.851 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.851 INFO:tasks.workunit.client.0.smithi002.stdout:525/600: open fd = 3 2024-09-10T08:23:08.851 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:526/600: open fd = 3 2024-09-10T08:23:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:527/600: open fd = 3 2024-09-10T08:23:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:528/600: open fd = 3 2024-09-10T08:23:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.853 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.853 INFO:tasks.workunit.client.0.smithi002.stdout:529/600: open fd = 3 2024-09-10T08:23:08.853 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.853 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.853 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.853 INFO:tasks.workunit.client.0.smithi002.stdout:530/600: open fd = 3 2024-09-10T08:23:08.853 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.853 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.853 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.853 INFO:tasks.workunit.client.0.smithi002.stdout:531/600: open fd = 3 2024-09-10T08:23:08.853 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.853 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:532/600: open fd = 3 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:533/600: open fd = 3 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:534/600: open fd = 3 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:535/600: open fd = 3 2024-09-10T08:23:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:536/600: open fd = 3 2024-09-10T08:23:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:537/600: open fd = 3 2024-09-10T08:23:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:538/600: open fd = 3 2024-09-10T08:23:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.856 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.856 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.856 INFO:tasks.workunit.client.0.smithi002.stdout:539/600: open fd = 3 2024-09-10T08:23:08.856 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.856 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.856 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.856 INFO:tasks.workunit.client.0.smithi002.stdout:540/600: open fd = 3 2024-09-10T08:23:08.856 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.856 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.856 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.856 INFO:tasks.workunit.client.0.smithi002.stdout:541/600: open fd = 3 2024-09-10T08:23:08.856 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.857 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.857 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.857 INFO:tasks.workunit.client.0.smithi002.stdout:542/600: open fd = 3 2024-09-10T08:23:08.857 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.857 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.857 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.857 INFO:tasks.workunit.client.0.smithi002.stdout:543/600: open fd = 3 2024-09-10T08:23:08.857 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.857 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.857 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.857 INFO:tasks.workunit.client.0.smithi002.stdout:544/600: open fd = 3 2024-09-10T08:23:08.858 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.858 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.858 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.858 INFO:tasks.workunit.client.0.smithi002.stdout:545/600: open fd = 3 2024-09-10T08:23:08.858 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.858 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.858 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.858 INFO:tasks.workunit.client.0.smithi002.stdout:546/600: open fd = 3 2024-09-10T08:23:08.858 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.858 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.858 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.858 INFO:tasks.workunit.client.0.smithi002.stdout:547/600: open fd = 3 2024-09-10T08:23:08.858 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:548/600: open fd = 3 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:549/600: open fd = 3 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:550/600: open fd = 3 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:551/600: open fd = 3 2024-09-10T08:23:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:552/600: open fd = 3 2024-09-10T08:23:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:553/600: open fd = 3 2024-09-10T08:23:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:554/600: open fd = 3 2024-09-10T08:23:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:555/600: open fd = 3 2024-09-10T08:23:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:556/600: open fd = 3 2024-09-10T08:23:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:557/600: open fd = 3 2024-09-10T08:23:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:558/600: open fd = 3 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:559/600: open fd = 3 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:560/600: open fd = 3 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:23:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:23:09.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:09 smithi155 ceph-mon[35405]: pgmap v837: 129 pgs: 129 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:23:09.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:09 smithi107 ceph-mon[35048]: pgmap v837: 129 pgs: 129 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:23:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:09 smithi002 ceph-mon[30563]: pgmap v837: 129 pgs: 129 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:23:11.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:11 smithi155 ceph-mon[35405]: pgmap v838: 129 pgs: 129 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:23:11.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:11 smithi107 ceph-mon[35048]: pgmap v838: 129 pgs: 129 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:23:11.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:11 smithi002 ceph-mon[30563]: pgmap v838: 129 pgs: 129 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-10T08:23:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.786+0000 7f75b5200640 1 -- 172.21.15.2:0/386824078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b0105910 msgr2=0x7f75b010c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.786+0000 7f75b5200640 1 --2- 172.21.15.2:0/386824078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b0105910 0x7f75b010c1a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009be0 tx=0x7f75a802a6c0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.786+0000 7f75b5200640 1 -- 172.21.15.2:0/386824078 shutdown_connections 2024-09-10T08:23:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.786+0000 7f75b5200640 1 --2- 172.21.15.2:0/386824078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b0105910 0x7f75b010c1a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.786+0000 7f75b5200640 1 --2- 172.21.15.2:0/386824078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75b0104f50 0x7f75b01053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.786+0000 7f75b5200640 1 --2- 172.21.15.2:0/386824078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b0103d60 0x7f75b0104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.786+0000 7f75b5200640 1 -- 172.21.15.2:0/386824078 >> 172.21.15.2:0/386824078 conn(0x7f75b00ff4e0 msgr2=0x7f75b0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.786+0000 7f75b5200640 1 -- 172.21.15.2:0/386824078 shutdown_connections 2024-09-10T08:23:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.786+0000 7f75b5200640 1 -- 172.21.15.2:0/386824078 wait complete. 2024-09-10T08:23:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.787+0000 7f75b5200640 1 Processor -- start 2024-09-10T08:23:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.788+0000 7f75b5200640 1 -- start start 2024-09-10T08:23:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.788+0000 7f75b5200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75b0103d60 0x7f75b0199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.788+0000 7f75b5200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b0104f50 0x7f75b019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.788+0000 7f75b5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b0105910 0x7f75b01a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.788+0000 7f75b5200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f75b010f990 con 0x7f75b0103d60 2024-09-10T08:23:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.788+0000 7f75b5200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75b010f810 con 0x7f75b0104f50 2024-09-10T08:23:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.788+0000 7f75b5200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75b010fb10 con 0x7f75b0105910 2024-09-10T08:23:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.788+0000 7f75ade00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75b0103d60 0x7f75b0199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.788+0000 7f75ade00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75b0103d60 0x7f75b0199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49686/0 (socket says 172.21.15.2:49686) 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.788+0000 7f75ade00640 1 -- 172.21.15.2:0/719482172 learned_addr learned my addr 172.21.15.2:0/719482172 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.788+0000 7f75ad400640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b0104f50 0x7f75b019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.788+0000 7f75ae800640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b0105910 0x7f75b01a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.789+0000 7f75ade00640 1 -- 172.21.15.2:0/719482172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b0105910 msgr2=0x7f75b01a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.789+0000 7f75ade00640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b0105910 0x7f75b01a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.789+0000 7f75ade00640 1 -- 172.21.15.2:0/719482172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b0104f50 msgr2=0x7f75b019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.789+0000 7f75ade00640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b0104f50 0x7f75b019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.789+0000 7f75ade00640 1 -- 172.21.15.2:0/719482172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b01a1b80 con 0x7f75b0103d60 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.789+0000 7f75ade00640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75b0103d60 0x7f75b0199f60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f75a000e9c0 tx=0x7f75a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.789+0000 7f75a6a00640 1 -- 172.21.15.2:0/719482172 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a000cd80 con 0x7f75b0103d60 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.789+0000 7f75a6a00640 1 -- 172.21.15.2:0/719482172 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a00035f0 con 0x7f75b0103d60 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.789+0000 7f75b5200640 1 -- 172.21.15.2:0/719482172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b01a1e70 con 0x7f75b0103d60 2024-09-10T08:23:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.789+0000 7f75a6a00640 1 -- 172.21.15.2:0/719482172 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0010690 con 0x7f75b0103d60 2024-09-10T08:23:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.789+0000 7f75b5200640 1 -- 172.21.15.2:0/719482172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b01a22d0 con 0x7f75b0103d60 2024-09-10T08:23:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.793+0000 7f75b81be740 1 Processor -- start 2024-09-10T08:23:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.793+0000 7f75a6a00640 1 -- 172.21.15.2:0/719482172 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f75a00037a0 con 0x7f75b0103d60 2024-09-10T08:23:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.793+0000 7f75b81be740 1 -- start start 2024-09-10T08:23:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.793+0000 7f75b81be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.793+0000 7f75b81be740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 0x55b22bf12490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.793+0000 7f75b81be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf12a60 0x55b22bf14f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.793+0000 7f75b81be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b22be72100 con 0x55b22bf12a60 2024-09-10T08:23:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.793+0000 7f75b81be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b22be1a1b0 con 0x55b22bf0d090 2024-09-10T08:23:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.793+0000 7f75b81be740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b22be6e0c0 con 0x55b22bf0d9d0 2024-09-10T08:23:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.794+0000 7f75ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.794+0000 7f75ad400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf12a60 0x55b22bf14f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.794+0000 7f75ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf0d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35504/0 (socket says 172.21.15.2:35504) 2024-09-10T08:23:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.794+0000 7f75ae800640 1 -- 172.21.15.2:0/2855735896 learned_addr learned my addr 172.21.15.2:0/2855735896 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.794+0000 7f75ade00640 1 --2- 172.21.15.2:0/2855735896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 0x55b22bf12490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.794+0000 7f75ae800640 1 -- 172.21.15.2:0/2855735896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 msgr2=0x55b22bf12490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.794+0000 7f75ae800640 1 --2- 172.21.15.2:0/2855735896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 0x55b22bf12490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.794+0000 7f75ae800640 1 -- 172.21.15.2:0/2855735896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf12a60 msgr2=0x55b22bf14f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.794+0000 7f75ae800640 1 --2- 172.21.15.2:0/2855735896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf12a60 0x55b22bf14f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.794+0000 7f75ae800640 1 -- 172.21.15.2:0/2855735896 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22bb85210 con 0x55b22bf0d090 2024-09-10T08:23:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.795+0000 7f75ae800640 1 --2- 172.21.15.2:0/2855735896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf0d490 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009be0 tx=0x7f75a8005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e5113c760ba95f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.795+0000 7f759b400640 1 -- 172.21.15.2:0/2855735896 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8042030 con 0x55b22bf0d090 2024-09-10T08:23:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.795+0000 7f75a6a00640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75940a2c30 0x7f75940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.795+0000 7f759b400640 1 -- 172.21.15.2:0/2855735896 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a8037040 con 0x55b22bf0d090 2024-09-10T08:23:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.795+0000 7f759b400640 1 -- 172.21.15.2:0/2855735896 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8036070 con 0x55b22bf0d090 2024-09-10T08:23:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.795+0000 7f75a6a00640 1 -- 172.21.15.2:0/719482172 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f75a00cf7e0 con 0x7f75b0103d60 2024-09-10T08:23:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.795+0000 7f75ade00640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75940a2c30 0x7f75940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.796+0000 7f75b81be740 1 -- 172.21.15.2:0/2855735896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 msgr2=0x55b22bf0d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.796+0000 7f75b81be740 1 --2- 172.21.15.2:0/2855735896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf0d490 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009be0 tx=0x7f75a8005ec0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.796+0000 7f75b81be740 1 -- 172.21.15.2:0/2855735896 shutdown_connections 2024-09-10T08:23:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.796+0000 7f75b81be740 1 --2- 172.21.15.2:0/2855735896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf12a60 0x55b22bf14f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.796+0000 7f75b81be740 1 --2- 172.21.15.2:0/2855735896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 0x55b22bf12490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.796+0000 7f75b81be740 1 --2- 172.21.15.2:0/2855735896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf0d490 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.796+0000 7f75b81be740 1 -- 172.21.15.2:0/2855735896 >> 172.21.15.2:0/2855735896 conn(0x55b22bf08700 msgr2=0x55b22bf08b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.796+0000 7f75b81be740 1 -- 172.21.15.2:0/2855735896 shutdown_connections 2024-09-10T08:23:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.796+0000 7f75b81be740 1 -- 172.21.15.2:0/2855735896 wait complete. 2024-09-10T08:23:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.796+0000 7f75ade00640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75940a2c30 0x7f75940a50f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f75a00cfa80 tx=0x7f75a00d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.796+0000 7f75b81be740 1 -- start start 2024-09-10T08:23:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.796+0000 7f75b81be740 1 Processor -- start 2024-09-10T08:23:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75b81be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf171c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75b81be740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 0x55b22bf17700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75b81be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf12a60 0x55b22bf1e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75b81be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b22bf159d0 con 0x55b22bf12a60 2024-09-10T08:23:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75b81be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b22bdfb090 con 0x55b22bf0d090 2024-09-10T08:23:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75b81be740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b22bf15b50 con 0x55b22bf0d9d0 2024-09-10T08:23:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75b81be740 10 client.0 _ll_register_callbacks cb 0x55b22be4ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf171c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35506/0 (socket says 172.21.15.2:35506) 2024-09-10T08:23:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75ae800640 1 -- 172.21.15.2:0/3198340505 learned_addr learned my addr 172.21.15.2:0/3198340505 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75ae800640 1 --2- 172.21.15.2:0/3198340505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf12a60 0x55b22bf1e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75ad400640 1 --2- 172.21.15.2:0/3198340505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 0x55b22bf17700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75ae800640 1 -- 172.21.15.2:0/3198340505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 msgr2=0x55b22bf17700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75ae800640 1 --2- 172.21.15.2:0/3198340505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 0x55b22bf17700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75ae800640 1 -- 172.21.15.2:0/3198340505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf12a60 msgr2=0x55b22bf1e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.797+0000 7f75ae800640 1 --2- 172.21.15.2:0/3198340505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf12a60 0x55b22bf1e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.798+0000 7f75ae800640 1 -- 172.21.15.2:0/3198340505 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22bc51620 con 0x55b22bf0d090 2024-09-10T08:23:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.798+0000 7f75ae800640 1 --2- 172.21.15.2:0/3198340505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf171c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009b30 tx=0x7f75a8030fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db473294d17d635 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.798+0000 7f758fe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:23:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.798+0000 7f758fe00640 1 -- 172.21.15.2:0/3198340505 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8036050 con 0x55b22bf0d090 2024-09-10T08:23:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.798+0000 7f75b81be740 1 -- 172.21.15.2:0/3198340505 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75a8037040 con 0x55b22bf0d090 2024-09-10T08:23:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.798+0000 7f758fe00640 1 -- 172.21.15.2:0/3198340505 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f75a80305a0 con 0x55b22bf0d090 2024-09-10T08:23:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.798+0000 7f758fe00640 1 -- 172.21.15.2:0/3198340505 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f75a8058050 con 0x55b22bf0d090 2024-09-10T08:23:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.798+0000 7f75b81be740 10 client.29699 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.798+0000 7f75b81be740 1 -- 172.21.15.2:0/3198340505 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b22bf21530 con 0x55b22bf0d090 2024-09-10T08:23:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.799+0000 7f758fe00640 1 -- 172.21.15.2:0/3198340505 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a8037040 con 0x55b22bf0d090 2024-09-10T08:23:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.799+0000 7f758fe00640 1 -- 172.21.15.2:0/3198340505 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f75a803c040 con 0x55b22bf0d090 2024-09-10T08:23:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.799+0000 7f75b81be740 10 client.29699 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.799+0000 7f75b81be740 10 client.29699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.799+0000 7f75b81be740 20 client.29699 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.799+0000 7f75b81be740 1 --2- 172.21.15.2:0/3198340505 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b22bf22880 0x55b22bf24c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.799+0000 7f75b81be740 4 client.29699 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:23:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.799+0000 7f75b81be740 1 -- 172.21.15.2:0/3198340505 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b22be7f830 con 0x55b22bf22880 2024-09-10T08:23:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.799+0000 7f75ad400640 1 --2- 172.21.15.2:0/3198340505 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b22bf22880 0x55b22bf24c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.800+0000 7f75ad400640 1 --2- 172.21.15.2:0/3198340505 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b22bf22880 0x55b22bf24c70 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f509447c92e23c6 server_cookie=9843d50939a1d8fa in_seq=0 out_seq=0 2024-09-10T08:23:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.800+0000 7f758fe00640 10 client.29699 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f758fe00640 1 -- 172.21.15.2:0/3198340505 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b22be7f830 con 0x55b22bf22880 2024-09-10T08:23:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f758fe00640 10 client.29699 handle_command_reply: tid=0 2024-09-10T08:23:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f75b81be740 1 client.29699 shutdown 2024-09-10T08:23:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f75b81be740 1 -- 172.21.15.2:0/3198340505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 msgr2=0x55b22bf171c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f75b81be740 1 --2- 172.21.15.2:0/3198340505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf171c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009b30 tx=0x7f75a8030fd0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f75b81be740 1 -- 172.21.15.2:0/3198340505 shutdown_connections 2024-09-10T08:23:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f75b81be740 1 --2- 172.21.15.2:0/3198340505 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b22bf22880 0x55b22bf24c70 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f75b81be740 1 --2- 172.21.15.2:0/3198340505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf12a60 0x55b22bf1e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f75b81be740 1 --2- 172.21.15.2:0/3198340505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 0x55b22bf17700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f75b81be740 1 --2- 172.21.15.2:0/3198340505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf171c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f75b81be740 1 -- 172.21.15.2:0/3198340505 >> 172.21.15.2:0/3198340505 conn(0x55b22bf08700 msgr2=0x55b22bf09f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f758fe00640 0 client.29699 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f75b81be740 1 -- 172.21.15.2:0/3198340505 shutdown_connections 2024-09-10T08:23:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f75b81be740 1 -- 172.21.15.2:0/3198340505 wait complete. 2024-09-10T08:23:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.801+0000 7f75b81be740 20 client.29699 trim_cache size 0 max 16384 2024-09-10T08:23:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75b81be740 1 Processor -- start 2024-09-10T08:23:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75b81be740 1 -- start start 2024-09-10T08:23:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75b81be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf17680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75b81be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d9d0 0x55b22bf1e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75b81be740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf12a60 0x55b22bf095b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75b81be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01af270 con 0x55b22bf0d9d0 2024-09-10T08:23:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75b81be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01acb40 con 0x55b22bf0d090 2024-09-10T08:23:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75b81be740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01aa990 con 0x55b22bf12a60 2024-09-10T08:23:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75ad400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d9d0 0x55b22bf1e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf17680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75ad400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d9d0 0x55b22bf1e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49708/0 (socket says 172.21.15.2:49708) 2024-09-10T08:23:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75ad400640 1 -- 172.21.15.2:0/3752789940 learned_addr learned my addr 172.21.15.2:0/3752789940 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf17680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:35512/0 (socket says 172.21.15.2:35512) 2024-09-10T08:23:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.817+0000 7f75ae800640 1 --2- 172.21.15.2:0/3752789940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf12a60 0x55b22bf095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.818+0000 7f75ad400640 1 -- 172.21.15.2:0/3752789940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf12a60 msgr2=0x55b22bf095b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.818+0000 7f75ad400640 1 --2- 172.21.15.2:0/3752789940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf12a60 0x55b22bf095b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.818+0000 7f75ad400640 1 -- 172.21.15.2:0/3752789940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 msgr2=0x55b22bf17680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.818+0000 7f75ad400640 1 --2- 172.21.15.2:0/3752789940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf17680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.818+0000 7f75ad400640 1 -- 172.21.15.2:0/3752789940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22bf24fe0 con 0x55b22bf0d9d0 2024-09-10T08:23:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.818+0000 7f75ae800640 1 --2- 172.21.15.2:0/3752789940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf17680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.818+0000 7f75ad400640 1 --2- 172.21.15.2:0/3752789940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d9d0 0x55b22bf1e6d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f759c009530 tx=0x7f759c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e34e8050e12b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.818+0000 7f759aa00640 1 -- 172.21.15.2:0/3752789940 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c027640 con 0x55b22bf0d9d0 2024-09-10T08:23:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.818+0000 7f759aa00640 1 -- 172.21.15.2:0/3752789940 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759c027cc0 con 0x55b22bf0d9d0 2024-09-10T08:23:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.818+0000 7f759aa00640 1 -- 172.21.15.2:0/3752789940 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c01dd90 con 0x55b22bf0d9d0 2024-09-10T08:23:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.819+0000 7f75b81be740 1 -- 172.21.15.2:0/3752789940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d9d0 msgr2=0x55b22bf1e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.819+0000 7f75b81be740 1 --2- 172.21.15.2:0/3752789940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d9d0 0x55b22bf1e6d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f759c009530 tx=0x7f759c009260 comp rx=0 tx=0).stop 2024-09-10T08:23:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.819+0000 7f75b81be740 1 -- 172.21.15.2:0/3752789940 shutdown_connections 2024-09-10T08:23:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.819+0000 7f75b81be740 1 --2- 172.21.15.2:0/3752789940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf12a60 0x55b22bf095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.819+0000 7f75b81be740 1 --2- 172.21.15.2:0/3752789940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d9d0 0x55b22bf1e6d0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.819+0000 7f75b81be740 1 --2- 172.21.15.2:0/3752789940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf0d090 0x55b22bf17680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.819+0000 7f75b81be740 1 -- 172.21.15.2:0/3752789940 >> 172.21.15.2:0/3752789940 conn(0x55b22bf08700 msgr2=0x55b22bf16f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.819+0000 7f75b81be740 1 -- 172.21.15.2:0/3752789940 shutdown_connections 2024-09-10T08:23:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.819+0000 7f75b81be740 1 -- 172.21.15.2:0/3752789940 wait complete. 2024-09-10T08:23:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.820+0000 7f75b81be740 1 -- start start 2024-09-10T08:23:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.820+0000 7f75b81be740 1 Processor -- start 2024-09-10T08:23:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.820+0000 7f75b81be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d090 0x55b22bf1b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.820+0000 7f75b81be740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 0x55b22bf1bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.820+0000 7f75b81be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf12a60 0x55b22bf1cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.820+0000 7f75b81be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01ab000 con 0x55b22bf0d090 2024-09-10T08:23:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.820+0000 7f75b81be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01ac470 con 0x55b22bf12a60 2024-09-10T08:23:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.820+0000 7f75b81be740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01ab6d0 con 0x55b22bf0d9d0 2024-09-10T08:23:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.820+0000 7f75b81be740 10 client.0 _ll_register_callbacks cb 0x7f759c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.820+0000 7f75ae800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d090 0x55b22bf1b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.820+0000 7f75ae800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d090 0x55b22bf1b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49718/0 (socket says 172.21.15.2:49718) 2024-09-10T08:23:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.820+0000 7f75ae800640 1 -- 172.21.15.2:0/1501384922 learned_addr learned my addr 172.21.15.2:0/1501384922 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.820+0000 7f75ad400640 1 --2- 172.21.15.2:0/1501384922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 0x55b22bf1bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.821+0000 7f75ae800640 1 --2- 172.21.15.2:0/1501384922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf12a60 0x55b22bf1cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.821+0000 7f75ae800640 1 -- 172.21.15.2:0/1501384922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 msgr2=0x55b22bf1bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.821+0000 7f75ae800640 1 --2- 172.21.15.2:0/1501384922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 0x55b22bf1bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.821+0000 7f75ae800640 1 -- 172.21.15.2:0/1501384922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf12a60 msgr2=0x55b22bf1cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.821+0000 7f75ae800640 1 --2- 172.21.15.2:0/1501384922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf12a60 0x55b22bf1cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.821+0000 7f75ae800640 1 -- 172.21.15.2:0/1501384922 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22bf1d350 con 0x55b22bf0d090 2024-09-10T08:23:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.821+0000 7f75ae800640 1 --2- 172.21.15.2:0/1501384922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d090 0x55b22bf1b7f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f75a8002230 tx=0x7f75a80098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ac520385c7d972 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.821+0000 7f758fe00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:23:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.821+0000 7f758fe00640 1 -- 172.21.15.2:0/1501384922 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8036070 con 0x55b22bf0d090 2024-09-10T08:23:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.821+0000 7f758fe00640 1 -- 172.21.15.2:0/1501384922 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f75a80045d0 con 0x55b22bf0d090 2024-09-10T08:23:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.821+0000 7f75b81be740 1 -- 172.21.15.2:0/1501384922 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f759c027cc0 con 0x55b22bf0d090 2024-09-10T08:23:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.822+0000 7f758fe00640 1 -- 172.21.15.2:0/1501384922 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f75a806c050 con 0x55b22bf0d090 2024-09-10T08:23:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.822+0000 7f75b81be740 10 client.19683 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.822+0000 7f75b81be740 1 -- 172.21.15.2:0/1501384922 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b22bf1e660 con 0x55b22bf0d090 2024-09-10T08:23:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.822+0000 7f758fe00640 1 -- 172.21.15.2:0/1501384922 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759c027cc0 con 0x55b22bf0d090 2024-09-10T08:23:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.822+0000 7f758fe00640 1 -- 172.21.15.2:0/1501384922 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f75a8004050 con 0x55b22bf0d090 2024-09-10T08:23:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.822+0000 7f75b81be740 10 client.19683 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.822+0000 7f75b81be740 10 client.19683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.822+0000 7f75b81be740 20 client.19683 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.822+0000 7f75b81be740 1 --2- 172.21.15.2:0/1501384922 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b22bf8d830 0x55b22bf8fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.822+0000 7f75b81be740 4 client.19683 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:23:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.822+0000 7f75b81be740 1 -- 172.21.15.2:0/1501384922 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b22be79b80 con 0x55b22bf8d830 2024-09-10T08:23:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.823+0000 7f75ad400640 1 --2- 172.21.15.2:0/1501384922 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b22bf8d830 0x55b22bf8fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.823+0000 7f75ad400640 1 --2- 172.21.15.2:0/1501384922 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b22bf8d830 0x55b22bf8fc50 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d51cab474882bf server_cookie=c3fcb260f6568c90 in_seq=0 out_seq=0 2024-09-10T08:23:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.823+0000 7f758fe00640 10 client.19683 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.838+0000 7f758fe00640 1 -- 172.21.15.2:0/1501384922 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b22be79b80 con 0x55b22bf8d830 2024-09-10T08:23:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.838+0000 7f758fe00640 10 client.19683 handle_command_reply: tid=0 2024-09-10T08:23:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.838+0000 7f75b81be740 1 client.19683 shutdown 2024-09-10T08:23:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.838+0000 7f75b81be740 1 -- 172.21.15.2:0/1501384922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d090 msgr2=0x55b22bf1b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.838+0000 7f75b81be740 1 --2- 172.21.15.2:0/1501384922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d090 0x55b22bf1b7f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f75a8002230 tx=0x7f75a80098c0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.838+0000 7f75b81be740 1 -- 172.21.15.2:0/1501384922 shutdown_connections 2024-09-10T08:23:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.838+0000 7f75b81be740 1 --2- 172.21.15.2:0/1501384922 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55b22bf8d830 0x55b22bf8fc50 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.838+0000 7f75b81be740 1 --2- 172.21.15.2:0/1501384922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22bf12a60 0x55b22bf1cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.838+0000 7f75b81be740 1 --2- 172.21.15.2:0/1501384922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b22bf0d9d0 0x55b22bf1bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.838+0000 7f758fe00640 0 client.19683 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.838+0000 7f75b81be740 1 --2- 172.21.15.2:0/1501384922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b22bf0d090 0x55b22bf1b7f0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.838+0000 7f75b81be740 1 -- 172.21.15.2:0/1501384922 >> 172.21.15.2:0/1501384922 conn(0x55b22bf08700 msgr2=0x55b22bf16ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.838+0000 7f75b81be740 1 -- 172.21.15.2:0/1501384922 shutdown_connections 2024-09-10T08:23:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.839+0000 7f75b81be740 1 -- 172.21.15.2:0/1501384922 wait complete. 2024-09-10T08:23:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.839+0000 7f75b81be740 20 client.19683 trim_cache size 0 max 16384 2024-09-10T08:23:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:23:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:23:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:23:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:23:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.839+0000 7f75b5200640 1 -- 172.21.15.2:0/719482172 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75940a2c30 msgr2=0x7f75940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.839+0000 7f75b5200640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75940a2c30 0x7f75940a50f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f75a00cfa80 tx=0x7f75a00d5530 comp rx=0 tx=0).stop 2024-09-10T08:23:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.839+0000 7f75b5200640 1 -- 172.21.15.2:0/719482172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75b0103d60 msgr2=0x7f75b0199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.839+0000 7f75b5200640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75b0103d60 0x7f75b0199f60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f75a000e9c0 tx=0x7f75a000ee80 comp rx=0 tx=0).stop 2024-09-10T08:23:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.839+0000 7f75b5200640 1 -- 172.21.15.2:0/719482172 shutdown_connections 2024-09-10T08:23:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.839+0000 7f75b5200640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75940a2c30 0x7f75940a50f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.839+0000 7f75b5200640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b0105910 0x7f75b01a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.839+0000 7f75b5200640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b0104f50 0x7f75b019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.839+0000 7f75b5200640 1 --2- 172.21.15.2:0/719482172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75b0103d60 0x7f75b0199f60 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.839+0000 7f75b5200640 1 -- 172.21.15.2:0/719482172 >> 172.21.15.2:0/719482172 conn(0x7f75b00ff4e0 msgr2=0x7f75b0108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.840+0000 7f75b5200640 1 -- 172.21.15.2:0/719482172 shutdown_connections 2024-09-10T08:23:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:12.840+0000 7f75b5200640 1 -- 172.21.15.2:0/719482172 wait complete. 2024-09-10T08:23:13.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:13 smithi155 ceph-mon[35405]: pgmap v839: 129 pgs: 129 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:23:13.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:13 smithi107 ceph-mon[35048]: pgmap v839: 129 pgs: 129 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:23:13.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:13 smithi002 ceph-mon[30563]: pgmap v839: 129 pgs: 129 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:23:15.656 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:15 smithi155 ceph-mon[35405]: pgmap v840: 129 pgs: 129 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T08:23:15.666 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:15 smithi107 ceph-mon[35048]: pgmap v840: 129 pgs: 129 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T08:23:15.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:15 smithi002 ceph-mon[30563]: pgmap v840: 129 pgs: 129 active+clean; 3.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T08:23:17.026 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:23:17.422 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:23:17.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:17 smithi107 ceph-mon[35048]: pgmap v841: 129 pgs: 129 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:23:17.765 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:17 smithi002 ceph-mon[30563]: pgmap v841: 129 pgs: 129 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:23:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:17 smithi155 ceph-mon[35405]: pgmap v841: 129 pgs: 129 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-10T08:23:18.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.062+0000 7f1fef600640 1 -- 172.21.15.2:0/802156896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe8106f20 msgr2=0x7f1fe810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:18.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.062+0000 7f1fef600640 1 --2- 172.21.15.2:0/802156896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe8106f20 0x7f1fe810eaa0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009ca0 tx=0x7f1fe002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.062+0000 7f1fef600640 1 -- 172.21.15.2:0/802156896 shutdown_connections 2024-09-10T08:23:18.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.062+0000 7f1fef600640 1 --2- 172.21.15.2:0/802156896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe8106f20 0x7f1fe810eaa0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.062+0000 7f1fef600640 1 --2- 172.21.15.2:0/802156896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe81027a0 0x7f1fe81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.062+0000 7f1fef600640 1 --2- 172.21.15.2:0/802156896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe8101df0 0x7f1fe81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.062+0000 7f1fef600640 1 -- 172.21.15.2:0/802156896 >> 172.21.15.2:0/802156896 conn(0x7f1fe80fd8b0 msgr2=0x7f1fe80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:18.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.062+0000 7f1fef600640 1 -- 172.21.15.2:0/802156896 shutdown_connections 2024-09-10T08:23:18.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.062+0000 7f1fef600640 1 -- 172.21.15.2:0/802156896 wait complete. 2024-09-10T08:23:18.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.063+0000 7f1fef600640 1 Processor -- start 2024-09-10T08:23:18.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.063+0000 7f1fef600640 1 -- start start 2024-09-10T08:23:18.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.064+0000 7f1fef600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe8101df0 0x7f1fe80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.064+0000 7f1fef600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe81027a0 0x7f1fe8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.064+0000 7f1fef600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe8106f20 0x7f1fe8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.064+0000 7f1fef600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8111f80 con 0x7f1fe81027a0 2024-09-10T08:23:18.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.064+0000 7f1fef600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8111e00 con 0x7f1fe8101df0 2024-09-10T08:23:18.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.064+0000 7f1fef600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8112100 con 0x7f1fe8106f20 2024-09-10T08:23:18.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.064+0000 7f1fdfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe81027a0 0x7f1fe8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.064+0000 7f1fdfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe81027a0 0x7f1fe8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32826/0 (socket says 172.21.15.2:32826) 2024-09-10T08:23:18.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.064+0000 7f1fdfe00640 1 -- 172.21.15.2:0/1162624495 learned_addr learned my addr 172.21.15.2:0/1162624495 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:18.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.065+0000 7f1fecc00640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe8101df0 0x7f1fe80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.065+0000 7f1fed600640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe8106f20 0x7f1fe8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.065+0000 7f1fdfe00640 1 -- 172.21.15.2:0/1162624495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe8106f20 msgr2=0x7f1fe8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:18.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.065+0000 7f1fdfe00640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe8106f20 0x7f1fe8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.065+0000 7f1fdfe00640 1 -- 172.21.15.2:0/1162624495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe8101df0 msgr2=0x7f1fe80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:18.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.065+0000 7f1fdfe00640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe8101df0 0x7f1fe80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.065+0000 7f1fdfe00640 1 -- 172.21.15.2:0/1162624495 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe81a6840 con 0x7f1fe81027a0 2024-09-10T08:23:18.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.065+0000 7f1fed600640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe8106f20 0x7f1fe8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:18.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.065+0000 7f1fdfe00640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe81027a0 0x7f1fe8074a30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000e9f0 tx=0x7f1fd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:18.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.066+0000 7f1fe6000640 1 -- 172.21.15.2:0/1162624495 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd000cd80 con 0x7f1fe81027a0 2024-09-10T08:23:18.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.066+0000 7f1fef600640 1 -- 172.21.15.2:0/1162624495 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe81a6b30 con 0x7f1fe81027a0 2024-09-10T08:23:18.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.066+0000 7f1fe6000640 1 -- 172.21.15.2:0/1162624495 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fd0010910 con 0x7f1fe81027a0 2024-09-10T08:23:18.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.066+0000 7f1fe6000640 1 -- 172.21.15.2:0/1162624495 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00188f0 con 0x7f1fe81027a0 2024-09-10T08:23:18.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.066+0000 7f1fef600640 1 -- 172.21.15.2:0/1162624495 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe81a70c0 con 0x7f1fe81027a0 2024-09-10T08:23:18.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.069+0000 7f1ff2268740 1 Processor -- start 2024-09-10T08:23:18.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.069+0000 7f1fe6000640 1 -- 172.21.15.2:0/1162624495 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1fd0010ac0 con 0x7f1fe81027a0 2024-09-10T08:23:18.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.069+0000 7f1ff2268740 1 -- start start 2024-09-10T08:23:18.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1ff2268740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adafb90 0x55860adaff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1ff2268740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1ff2268740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adbad70 0x55860adbd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1ff2268740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55860ad163e0 con 0x55860adafb90 2024-09-10T08:23:18.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1ff2268740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55860acbdd80 con 0x55860adb04b0 2024-09-10T08:23:18.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1ff2268740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55860ad123a0 con 0x55860adbad70 2024-09-10T08:23:18.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1fed600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adafb90 0x55860adaff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1fed600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adafb90 0x55860adaff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32836/0 (socket says 172.21.15.2:32836) 2024-09-10T08:23:18.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1fecc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1fed600640 1 -- 172.21.15.2:0/44967919 learned_addr learned my addr 172.21.15.2:0/44967919 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:18.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1fdfe00640 1 --2- 172.21.15.2:0/44967919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adbad70 0x55860adbd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1fed600640 1 -- 172.21.15.2:0/44967919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adbad70 msgr2=0x55860adbd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:18.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1fed600640 1 --2- 172.21.15.2:0/44967919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adbad70 0x55860adbd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1fed600640 1 -- 172.21.15.2:0/44967919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 msgr2=0x55860adba830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:18.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.070+0000 7f1fed600640 1 --2- 172.21.15.2:0/44967919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adba830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.071+0000 7f1fed600640 1 -- 172.21.15.2:0/44967919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55860acdcae0 con 0x55860adafb90 2024-09-10T08:23:18.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.071+0000 7f1fecc00640 1 --2- 172.21.15.2:0/44967919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:18.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.071+0000 7f1fed600640 1 --2- 172.21.15.2:0/44967919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adafb90 0x55860adaff70 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0031f80 tx=0x7f1fe0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87700b117c38028 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:18.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.071+0000 7f1fddc00640 1 -- 172.21.15.2:0/44967919 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0003b70 con 0x55860adafb90 2024-09-10T08:23:18.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.071+0000 7f1fddc00640 1 -- 172.21.15.2:0/44967919 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe0036040 con 0x55860adafb90 2024-09-10T08:23:18.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.071+0000 7f1fddc00640 1 -- 172.21.15.2:0/44967919 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00045f0 con 0x55860adafb90 2024-09-10T08:23:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.071+0000 7f1fe6000640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1fcc0a2c00 0x7f1fcc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.072+0000 7f1fe6000640 1 -- 172.21.15.2:0/1162624495 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1fd00cf700 con 0x7f1fe81027a0 2024-09-10T08:23:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.072+0000 7f1fecc00640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1fcc0a2c00 0x7f1fcc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.072+0000 7f1ff2268740 1 -- 172.21.15.2:0/44967919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adafb90 msgr2=0x55860adaff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.072+0000 7f1ff2268740 1 --2- 172.21.15.2:0/44967919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adafb90 0x55860adaff70 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0031f80 tx=0x7f1fe0005ef0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.072+0000 7f1ff2268740 1 -- 172.21.15.2:0/44967919 shutdown_connections 2024-09-10T08:23:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.072+0000 7f1ff2268740 1 --2- 172.21.15.2:0/44967919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adbad70 0x55860adbd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.072+0000 7f1ff2268740 1 --2- 172.21.15.2:0/44967919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.072+0000 7f1ff2268740 1 --2- 172.21.15.2:0/44967919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adafb90 0x55860adaff70 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.072+0000 7f1ff2268740 1 -- 172.21.15.2:0/44967919 >> 172.21.15.2:0/44967919 conn(0x55860adac120 msgr2=0x55860adac500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.072+0000 7f1ff2268740 1 -- 172.21.15.2:0/44967919 shutdown_connections 2024-09-10T08:23:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.072+0000 7f1fecc00640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1fcc0a2c00 0x7f1fcc0a50c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x55860adacf80 tx=0x7f1fd8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.072+0000 7f1ff2268740 1 -- 172.21.15.2:0/44967919 wait complete. 2024-09-10T08:23:18.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.073+0000 7f1ff2268740 1 -- start start 2024-09-10T08:23:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.073+0000 7f1ff2268740 1 Processor -- start 2024-09-10T08:23:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.073+0000 7f1ff2268740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 0x55860adb7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1ff2268740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adb8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1ff2268740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adbad70 0x55860adb8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1ff2268740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55860adbd860 con 0x55860adbad70 2024-09-10T08:23:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1ff2268740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55860acd10f0 con 0x55860adb04b0 2024-09-10T08:23:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1ff2268740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55860adbd9e0 con 0x55860adafb90 2024-09-10T08:23:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1ff2268740 10 client.0 _ll_register_callbacks cb 0x55860acc29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1fdfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adb8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1fecc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 0x55860adb7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1fed600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adbad70 0x55860adb8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1fdfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adb8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:51304/0 (socket says 172.21.15.2:51304) 2024-09-10T08:23:18.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1fdfe00640 1 -- 172.21.15.2:0/3034876532 learned_addr learned my addr 172.21.15.2:0/3034876532 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:18.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1fdfe00640 1 -- 172.21.15.2:0/3034876532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 msgr2=0x55860adb7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:18.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1fdfe00640 1 --2- 172.21.15.2:0/3034876532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 0x55860adb7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1fdfe00640 1 -- 172.21.15.2:0/3034876532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adbad70 msgr2=0x55860adb8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:18.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1fdfe00640 1 --2- 172.21.15.2:0/3034876532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adbad70 0x55860adb8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1fdfe00640 1 -- 172.21.15.2:0/3034876532 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55860aa873c0 con 0x55860adb04b0 2024-09-10T08:23:18.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.074+0000 7f1fed600640 1 --2- 172.21.15.2:0/3034876532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adbad70 0x55860adb8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:18.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.075+0000 7f1fdfe00640 1 --2- 172.21.15.2:0/3034876532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adb8210 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000e9c0 tx=0x7f1fd009b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e032522b0e9b7989 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:18.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.075+0000 7f1fd6000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:23:18.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.075+0000 7f1fd6000640 1 -- 172.21.15.2:0/3034876532 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd009bbc0 con 0x55860adb04b0 2024-09-10T08:23:18.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.075+0000 7f1ff2268740 1 -- 172.21.15.2:0/3034876532 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fe0036040 con 0x55860adb04b0 2024-09-10T08:23:18.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.075+0000 7f1fd6000640 1 -- 172.21.15.2:0/3034876532 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1fd009b010 con 0x55860adb04b0 2024-09-10T08:23:18.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.075+0000 7f1fd6000640 1 -- 172.21.15.2:0/3034876532 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1fd009fec0 con 0x55860adb04b0 2024-09-10T08:23:18.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.075+0000 7f1ff2268740 10 client.29717 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:18.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.075+0000 7f1ff2268740 1 -- 172.21.15.2:0/3034876532 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55860adc9610 con 0x55860adb04b0 2024-09-10T08:23:18.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.075+0000 7f1fd6000640 1 -- 172.21.15.2:0/3034876532 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe0036040 con 0x55860adb04b0 2024-09-10T08:23:18.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.076+0000 7f1fd6000640 1 -- 172.21.15.2:0/3034876532 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1fd009aa80 con 0x55860adb04b0 2024-09-10T08:23:18.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.076+0000 7f1ff2268740 10 client.29717 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:18.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.076+0000 7f1ff2268740 10 client.29717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:18.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.076+0000 7f1ff2268740 20 client.29717 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:18.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.076+0000 7f1ff2268740 1 --2- 172.21.15.2:0/3034876532 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55860adcab20 0x55860adccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.076+0000 7f1ff2268740 4 client.29717 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:23:18.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.076+0000 7f1ff2268740 1 -- 172.21.15.2:0/3034876532 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55860ad23950 con 0x55860adcab20 2024-09-10T08:23:18.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.076+0000 7f1fecc00640 1 --2- 172.21.15.2:0/3034876532 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55860adcab20 0x55860adccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.077+0000 7f1fecc00640 1 --2- 172.21.15.2:0/3034876532 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55860adcab20 0x55860adccf60 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=236c3adf1f69c918 server_cookie=8410822e1f657e4e in_seq=0 out_seq=0 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.077+0000 7f1fd6000640 10 client.29717 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.077+0000 7f1fd6000640 1 -- 172.21.15.2:0/3034876532 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55860ad23950 con 0x55860adcab20 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.077+0000 7f1fd6000640 10 client.29717 handle_command_reply: tid=0 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.078+0000 7f1ff2268740 1 client.29717 shutdown 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.078+0000 7f1ff2268740 1 -- 172.21.15.2:0/3034876532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 msgr2=0x55860adb8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.078+0000 7f1ff2268740 1 --2- 172.21.15.2:0/3034876532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adb8210 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000e9c0 tx=0x7f1fd009b700 comp rx=0 tx=0).stop 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.078+0000 7f1ff2268740 1 -- 172.21.15.2:0/3034876532 shutdown_connections 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.078+0000 7f1ff2268740 1 --2- 172.21.15.2:0/3034876532 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55860adcab20 0x55860adccf60 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.078+0000 7f1ff2268740 1 --2- 172.21.15.2:0/3034876532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adbad70 0x55860adb8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.078+0000 7f1ff2268740 1 --2- 172.21.15.2:0/3034876532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adb8210 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.078+0000 7f1ff2268740 1 --2- 172.21.15.2:0/3034876532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 0x55860adb7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.078+0000 7f1ff2268740 1 -- 172.21.15.2:0/3034876532 >> 172.21.15.2:0/3034876532 conn(0x55860adac120 msgr2=0x55860adb1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.078+0000 7f1fd6000640 0 client.29717 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.078+0000 7f1ff2268740 1 -- 172.21.15.2:0/3034876532 shutdown_connections 2024-09-10T08:23:18.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.078+0000 7f1ff2268740 1 -- 172.21.15.2:0/3034876532 wait complete. 2024-09-10T08:23:18.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.078+0000 7f1ff2268740 20 client.29717 trim_cache size 0 max 16384 2024-09-10T08:23:18.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.096+0000 7f1ff2268740 1 Processor -- start 2024-09-10T08:23:18.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.096+0000 7f1ff2268740 1 -- start start 2024-09-10T08:23:18.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1ff2268740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 0x55860add2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1ff2268740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adb04b0 0x55860add2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1ff2268740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adbad70 0x55860adb4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1ff2268740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81b3720 con 0x55860adb04b0 2024-09-10T08:23:18.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1ff2268740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81b0ff0 con 0x55860adbad70 2024-09-10T08:23:18.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1ff2268740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81aee40 con 0x55860adafb90 2024-09-10T08:23:18.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1fdfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adb04b0 0x55860add2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1fdfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adb04b0 0x55860add2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32852/0 (socket says 172.21.15.2:32852) 2024-09-10T08:23:18.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1fdfe00640 1 -- 172.21.15.2:0/3258138792 learned_addr learned my addr 172.21.15.2:0/3258138792 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:18.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1fecc00640 1 --2- 172.21.15.2:0/3258138792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 0x55860add2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1fed600640 1 --2- 172.21.15.2:0/3258138792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adbad70 0x55860adb4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1fdfe00640 1 -- 172.21.15.2:0/3258138792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 msgr2=0x55860add2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:18.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1fdfe00640 1 --2- 172.21.15.2:0/3258138792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 0x55860add2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1fdfe00640 1 -- 172.21.15.2:0/3258138792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adbad70 msgr2=0x55860adb4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:18.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1fdfe00640 1 --2- 172.21.15.2:0/3258138792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adbad70 0x55860adb4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.097+0000 7f1fdfe00640 1 -- 172.21.15.2:0/3258138792 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55860adacdb0 con 0x55860adb04b0 2024-09-10T08:23:18.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.098+0000 7f1fdfe00640 1 --2- 172.21.15.2:0/3258138792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adb04b0 0x55860add2d70 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1fd009bb50 tx=0x7f1fd00cf980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed87866782f79633 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:18.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.098+0000 7f1fdd200640 1 -- 172.21.15.2:0/3258138792 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd009bbc0 con 0x55860adb04b0 2024-09-10T08:23:18.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.098+0000 7f1fdd200640 1 -- 172.21.15.2:0/3258138792 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fd00d5020 con 0x55860adb04b0 2024-09-10T08:23:18.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.098+0000 7f1fdd200640 1 -- 172.21.15.2:0/3258138792 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00cf500 con 0x55860adb04b0 2024-09-10T08:23:18.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.099+0000 7f1ff2268740 1 -- 172.21.15.2:0/3258138792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adb04b0 msgr2=0x55860add2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:18.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.099+0000 7f1ff2268740 1 --2- 172.21.15.2:0/3258138792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adb04b0 0x55860add2d70 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1fd009bb50 tx=0x7f1fd00cf980 comp rx=0 tx=0).stop 2024-09-10T08:23:18.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.099+0000 7f1ff2268740 1 -- 172.21.15.2:0/3258138792 shutdown_connections 2024-09-10T08:23:18.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.099+0000 7f1ff2268740 1 --2- 172.21.15.2:0/3258138792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adbad70 0x55860adb4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.099+0000 7f1ff2268740 1 --2- 172.21.15.2:0/3258138792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adb04b0 0x55860add2d70 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.099+0000 7f1ff2268740 1 --2- 172.21.15.2:0/3258138792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 0x55860add2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.099+0000 7f1ff2268740 1 -- 172.21.15.2:0/3258138792 >> 172.21.15.2:0/3258138792 conn(0x55860adac120 msgr2=0x55860adb1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.099+0000 7f1ff2268740 1 -- 172.21.15.2:0/3258138792 shutdown_connections 2024-09-10T08:23:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.099+0000 7f1ff2268740 1 -- 172.21.15.2:0/3258138792 wait complete. 2024-09-10T08:23:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.099+0000 7f1ff2268740 1 -- start start 2024-09-10T08:23:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.100+0000 7f1ff2268740 1 Processor -- start 2024-09-10T08:23:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.100+0000 7f1ff2268740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 0x55860adc2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.100+0000 7f1ff2268740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adc31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.100+0000 7f1ff2268740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adbad70 0x55860adbfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.100+0000 7f1ff2268740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81af4b0 con 0x55860adbad70 2024-09-10T08:23:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.100+0000 7f1ff2268740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81b0920 con 0x55860adb04b0 2024-09-10T08:23:18.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.100+0000 7f1ff2268740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81afb80 con 0x55860adafb90 2024-09-10T08:23:18.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.100+0000 7f1ff2268740 10 client.0 _ll_register_callbacks cb 0x7f1fd000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:18.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.100+0000 7f1fed600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adbad70 0x55860adbfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.100+0000 7f1fed600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adbad70 0x55860adbfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32854/0 (socket says 172.21.15.2:32854) 2024-09-10T08:23:18.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.100+0000 7f1fed600640 1 -- 172.21.15.2:0/850530432 learned_addr learned my addr 172.21.15.2:0/850530432 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:18.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.100+0000 7f1fecc00640 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 0x55860adc2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.100+0000 7f1fdfe00640 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adc31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.101+0000 7f1fed600640 1 -- 172.21.15.2:0/850530432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 msgr2=0x55860adc2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:18.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.101+0000 7f1fed600640 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 0x55860adc2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.101+0000 7f1fed600640 1 -- 172.21.15.2:0/850530432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 msgr2=0x55860adc31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:18.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.101+0000 7f1fed600640 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adc31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.101+0000 7f1fed600640 1 -- 172.21.15.2:0/850530432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55860adc37e0 con 0x55860adbad70 2024-09-10T08:23:18.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.101+0000 7f1fdfe00640 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adc31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:18.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.101+0000 7f1fed600640 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adbad70 0x55860adbfc30 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009ca0 tx=0x7f1fe00041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedf286ade344780 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:18.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.101+0000 7f1fd6000640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:23:18.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.101+0000 7f1fd6000640 1 -- 172.21.15.2:0/850530432 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe004a040 con 0x55860adbad70 2024-09-10T08:23:18.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.101+0000 7f1fd6000640 1 -- 172.21.15.2:0/850530432 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1fe0031880 con 0x55860adbad70 2024-09-10T08:23:18.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.101+0000 7f1ff2268740 1 -- 172.21.15.2:0/850530432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd00d5020 con 0x55860adbad70 2024-09-10T08:23:18.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.102+0000 7f1fd6000640 1 -- 172.21.15.2:0/850530432 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1fe0003970 con 0x55860adbad70 2024-09-10T08:23:18.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.102+0000 7f1ff2268740 10 client.19701 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:18.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.102+0000 7f1ff2268740 1 -- 172.21.15.2:0/850530432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55860ae35d50 con 0x55860adbad70 2024-09-10T08:23:18.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.102+0000 7f1fd6000640 1 -- 172.21.15.2:0/850530432 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fd00d5020 con 0x55860adbad70 2024-09-10T08:23:18.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.102+0000 7f1fd6000640 1 -- 172.21.15.2:0/850530432 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1fe004d030 con 0x55860adbad70 2024-09-10T08:23:18.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.102+0000 7f1ff2268740 10 client.19701 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:18.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.102+0000 7f1ff2268740 10 client.19701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:18.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.102+0000 7f1ff2268740 20 client.19701 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:18.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.102+0000 7f1ff2268740 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55860ae36190 0x55860ae38580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:18.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.102+0000 7f1ff2268740 4 client.19701 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:23:18.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.102+0000 7f1ff2268740 1 -- 172.21.15.2:0/850530432 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55860ad1de60 con 0x55860ae36190 2024-09-10T08:23:18.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.103+0000 7f1fecc00640 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55860ae36190 0x55860ae38580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:18.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.103+0000 7f1fecc00640 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55860ae36190 0x55860ae38580 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=480da45c4f0de719 server_cookie=4ea401892832e3dd in_seq=0 out_seq=0 2024-09-10T08:23:18.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.103+0000 7f1fd6000640 10 client.19701 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:18.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.103+0000 7f1fd6000640 1 -- 172.21.15.2:0/850530432 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55860ad1de60 con 0x55860ae36190 2024-09-10T08:23:18.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.103+0000 7f1fd6000640 10 client.19701 handle_command_reply: tid=0 2024-09-10T08:23:18.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.103+0000 7f1ff2268740 1 client.19701 shutdown 2024-09-10T08:23:18.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.104+0000 7f1ff2268740 1 -- 172.21.15.2:0/850530432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adbad70 msgr2=0x55860adbfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:18.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.104+0000 7f1ff2268740 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adbad70 0x55860adbfc30 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009ca0 tx=0x7f1fe00041f0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.104+0000 7f1ff2268740 1 -- 172.21.15.2:0/850530432 shutdown_connections 2024-09-10T08:23:18.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.104+0000 7f1ff2268740 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55860ae36190 0x55860ae38580 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.104+0000 7f1ff2268740 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55860adbad70 0x55860adbfc30 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.104+0000 7f1ff2268740 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55860adb04b0 0x55860adc31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.104+0000 7f1ff2268740 1 --2- 172.21.15.2:0/850530432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55860adafb90 0x55860adc2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.104+0000 7f1ff2268740 1 -- 172.21.15.2:0/850530432 >> 172.21.15.2:0/850530432 conn(0x55860adac120 msgr2=0x55860adb65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:18.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.104+0000 7f1fd6000640 0 client.19701 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:18.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.104+0000 7f1ff2268740 1 -- 172.21.15.2:0/850530432 shutdown_connections 2024-09-10T08:23:18.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.104+0000 7f1ff2268740 1 -- 172.21.15.2:0/850530432 wait complete. 2024-09-10T08:23:18.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.104+0000 7f1ff2268740 20 client.19701 trim_cache size 0 max 16384 2024-09-10T08:23:18.110 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:23:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:23:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:23:18.111 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:23:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.105+0000 7f1fef600640 1 -- 172.21.15.2:0/1162624495 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1fcc0a2c00 msgr2=0x7f1fcc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.105+0000 7f1fef600640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1fcc0a2c00 0x7f1fcc0a50c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x55860adacf80 tx=0x7f1fd8005f90 comp rx=0 tx=0).stop 2024-09-10T08:23:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.105+0000 7f1fef600640 1 -- 172.21.15.2:0/1162624495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe81027a0 msgr2=0x7f1fe8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.105+0000 7f1fef600640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe81027a0 0x7f1fe8074a30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000e9f0 tx=0x7f1fd000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.105+0000 7f1fef600640 1 -- 172.21.15.2:0/1162624495 shutdown_connections 2024-09-10T08:23:18.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.105+0000 7f1fef600640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f1fcc0a2c00 0x7f1fcc0a50c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.105+0000 7f1fef600640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe8106f20 0x7f1fe8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.105+0000 7f1fef600640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe81027a0 0x7f1fe8074a30 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.105+0000 7f1fef600640 1 --2- 172.21.15.2:0/1162624495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe8101df0 0x7f1fe80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:18.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.105+0000 7f1fef600640 1 -- 172.21.15.2:0/1162624495 >> 172.21.15.2:0/1162624495 conn(0x7f1fe80fd8b0 msgr2=0x7f1fe80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:18.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.106+0000 7f1fef600640 1 -- 172.21.15.2:0/1162624495 shutdown_connections 2024-09-10T08:23:18.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:18.106+0000 7f1fef600640 1 -- 172.21.15.2:0/1162624495 wait complete. 2024-09-10T08:23:19.035 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:23:19.036 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:23:19.412 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:23:19.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:19 smithi107 ceph-mon[35048]: pgmap v842: 129 pgs: 129 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:23:19.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:19 smithi002 ceph-mon[30563]: pgmap v842: 129 pgs: 129 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:23:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:19 smithi155 ceph-mon[35405]: pgmap v842: 129 pgs: 129 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:23:20.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.122+0000 7f70c9e00640 1 -- 172.21.15.2:0/3356560413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c4112b90 msgr2=0x7f70c4114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:20.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.122+0000 7f70c9e00640 1 --2- 172.21.15.2:0/3356560413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c4112b90 0x7f70c4114f80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f70bc009ca0 tx=0x7f70bc02a950 comp rx=0 tx=0).stop 2024-09-10T08:23:20.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.122+0000 7f70c9e00640 1 -- 172.21.15.2:0/3356560413 shutdown_connections 2024-09-10T08:23:20.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.122+0000 7f70c9e00640 1 --2- 172.21.15.2:0/3356560413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c4112b90 0x7f70c4114f80 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:20.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.122+0000 7f70c9e00640 1 --2- 172.21.15.2:0/3356560413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70c4074eb0 0x7f70c4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:20.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.122+0000 7f70c9e00640 1 --2- 172.21.15.2:0/3356560413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70c4074590 0x7f70c4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:20.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.122+0000 7f70c9e00640 1 -- 172.21.15.2:0/3356560413 >> 172.21.15.2:0/3356560413 conn(0x7f70c40fd8c0 msgr2=0x7f70c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:20.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.123+0000 7f70c9e00640 1 -- 172.21.15.2:0/3356560413 shutdown_connections 2024-09-10T08:23:20.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.123+0000 7f70c9e00640 1 -- 172.21.15.2:0/3356560413 wait complete. 2024-09-10T08:23:20.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.124+0000 7f70c9e00640 1 Processor -- start 2024-09-10T08:23:20.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.124+0000 7f70c9e00640 1 -- start start 2024-09-10T08:23:20.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.124+0000 7f70c9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70c4074590 0x7f70c419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:20.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.124+0000 7f70c9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c4074eb0 0x7f70c419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:20.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.124+0000 7f70c9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70c4112b90 0x7f70c41a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:20.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.124+0000 7f70c9e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4118550 con 0x7f70c4074eb0 2024-09-10T08:23:20.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.124+0000 7f70c9e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41183d0 con 0x7f70c4074590 2024-09-10T08:23:20.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.124+0000 7f70c9e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41186d0 con 0x7f70c4112b90 2024-09-10T08:23:20.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.124+0000 7f70c2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70c4074590 0x7f70c419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:20.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.124+0000 7f70c2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c4074eb0 0x7f70c419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.124+0000 7f70c2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70c4074590 0x7f70c419eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:51338/0 (socket says 172.21.15.2:51338) 2024-09-10T08:23:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.124+0000 7f70c2a00640 1 -- 172.21.15.2:0/43594463 learned_addr learned my addr 172.21.15.2:0/43594463 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.124+0000 7f70c3400640 1 --2- 172.21.15.2:0/43594463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70c4112b90 0x7f70c41a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.125+0000 7f70c2a00640 1 -- 172.21.15.2:0/43594463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70c4112b90 msgr2=0x7f70c41a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.125+0000 7f70c2a00640 1 --2- 172.21.15.2:0/43594463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70c4112b90 0x7f70c41a33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.125+0000 7f70c2a00640 1 -- 172.21.15.2:0/43594463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c4074eb0 msgr2=0x7f70c419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.125+0000 7f70c2a00640 1 --2- 172.21.15.2:0/43594463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c4074eb0 0x7f70c419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.125+0000 7f70c2a00640 1 -- 172.21.15.2:0/43594463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c41a3ad0 con 0x7f70c4074590 2024-09-10T08:23:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.125+0000 7f70c2a00640 1 --2- 172.21.15.2:0/43594463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70c4074590 0x7f70c419eaa0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f70b400d950 tx=0x7f70b400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.125+0000 7f70c8a00640 1 -- 172.21.15.2:0/43594463 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4017070 con 0x7f70c4074590 2024-09-10T08:23:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.125+0000 7f70c9e00640 1 -- 172.21.15.2:0/43594463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c41a3dc0 con 0x7f70c4074590 2024-09-10T08:23:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.126+0000 7f70c8a00640 1 -- 172.21.15.2:0/43594463 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b4009c30 con 0x7f70c4074590 2024-09-10T08:23:20.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.126+0000 7f70c8a00640 1 -- 172.21.15.2:0/43594463 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b400f6b0 con 0x7f70c4074590 2024-09-10T08:23:20.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.126+0000 7f70c9e00640 1 -- 172.21.15.2:0/43594463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c41ab650 con 0x7f70c4074590 2024-09-10T08:23:20.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.127+0000 7f70c9e00640 1 -- 172.21.15.2:0/43594463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c4074970 con 0x7f70c4074590 2024-09-10T08:23:20.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.129+0000 7f70c8a00640 1 -- 172.21.15.2:0/43594463 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f70b40109d0 con 0x7f70c4074590 2024-09-10T08:23:20.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.130+0000 7f70c8a00640 1 --2- 172.21.15.2:0/43594463 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70a80a2c00 0x7f70a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:20.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.130+0000 7f70c8a00640 1 -- 172.21.15.2:0/43594463 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f70b40cf350 con 0x7f70c4074590 2024-09-10T08:23:20.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.130+0000 7f70c2000640 1 --2- 172.21.15.2:0/43594463 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70a80a2c00 0x7f70a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:20.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.131+0000 7f70c2000640 1 --2- 172.21.15.2:0/43594463 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70a80a2c00 0x7f70a80a50c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f70c40fecf0 tx=0x7f70b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:20.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.132+0000 7f70c8a00640 1 -- 172.21.15.2:0/43594463 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70b4009de0 con 0x7f70c4074590 2024-09-10T08:23:20.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.353+0000 7f70c9e00640 1 -- 172.21.15.2:0/43594463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70c419fe00 con 0x7f70c4074590 2024-09-10T08:23:20.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.354+0000 7f70c8a00640 1 -- 172.21.15.2:0/43594463 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f70b409b890 con 0x7f70c4074590 2024-09-10T08:23:20.354 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:23:20.354 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:23:20.355 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:23:20.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.356+0000 7f70c9e00640 1 -- 172.21.15.2:0/43594463 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70a80a2c00 msgr2=0x7f70a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:20.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.356+0000 7f70c9e00640 1 --2- 172.21.15.2:0/43594463 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70a80a2c00 0x7f70a80a50c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f70c40fecf0 tx=0x7f70b0005f90 comp rx=0 tx=0).stop 2024-09-10T08:23:20.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.356+0000 7f70c9e00640 1 -- 172.21.15.2:0/43594463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70c4074590 msgr2=0x7f70c419eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:20.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.356+0000 7f70c9e00640 1 --2- 172.21.15.2:0/43594463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70c4074590 0x7f70c419eaa0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f70b400d950 tx=0x7f70b400de10 comp rx=0 tx=0).stop 2024-09-10T08:23:20.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.356+0000 7f70c9e00640 1 -- 172.21.15.2:0/43594463 shutdown_connections 2024-09-10T08:23:20.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.356+0000 7f70c9e00640 1 --2- 172.21.15.2:0/43594463 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f70a80a2c00 0x7f70a80a50c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:20.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.356+0000 7f70c9e00640 1 --2- 172.21.15.2:0/43594463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70c4112b90 0x7f70c41a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:20.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.356+0000 7f70c9e00640 1 --2- 172.21.15.2:0/43594463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c4074eb0 0x7f70c419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:20.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.356+0000 7f70c9e00640 1 --2- 172.21.15.2:0/43594463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70c4074590 0x7f70c419eaa0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:20.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.356+0000 7f70c9e00640 1 -- 172.21.15.2:0/43594463 >> 172.21.15.2:0/43594463 conn(0x7f70c40fd8c0 msgr2=0x7f70c40ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:20.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.357+0000 7f70c9e00640 1 -- 172.21.15.2:0/43594463 shutdown_connections 2024-09-10T08:23:20.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:20.357+0000 7f70c9e00640 1 -- 172.21.15.2:0/43594463 wait complete. 2024-09-10T08:23:20.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:20 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/43594463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:23:20.722 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:20 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/43594463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:23:20.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:20 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/43594463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:23:21.417 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:23:21.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:21 smithi002 ceph-mon[30563]: pgmap v843: 129 pgs: 129 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:23:21.808 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:23:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:21 smithi155 ceph-mon[35405]: pgmap v843: 129 pgs: 129 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:23:21.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:21 smithi107 ceph-mon[35048]: pgmap v843: 129 pgs: 129 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:23:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.403+0000 7f804b200640 1 -- 172.21.15.2:0/3675235073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f804c103e30 msgr2=0x7f804c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.403+0000 7f804b200640 1 --2- 172.21.15.2:0/3675235073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f804c103e30 0x7f804c104210 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f803c009900 tx=0x7f803c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.403+0000 7f804b200640 1 -- 172.21.15.2:0/3675235073 shutdown_connections 2024-09-10T08:23:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.403+0000 7f804b200640 1 --2- 172.21.15.2:0/3675235073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f804c108f60 0x7f804c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.403+0000 7f804b200640 1 --2- 172.21.15.2:0/3675235073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f804c1047e0 0x7f804c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.403+0000 7f804b200640 1 --2- 172.21.15.2:0/3675235073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f804c103e30 0x7f804c104210 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.403+0000 7f804b200640 1 -- 172.21.15.2:0/3675235073 >> 172.21.15.2:0/3675235073 conn(0x7f804c0ff8d0 msgr2=0x7f804c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.403+0000 7f804b200640 1 -- 172.21.15.2:0/3675235073 shutdown_connections 2024-09-10T08:23:22.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.403+0000 7f804b200640 1 -- 172.21.15.2:0/3675235073 wait complete. 2024-09-10T08:23:22.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.404+0000 7f804b200640 1 Processor -- start 2024-09-10T08:23:22.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.404+0000 7f804b200640 1 -- start start 2024-09-10T08:23:22.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.404+0000 7f804b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f804c103e30 0x7f804c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.404+0000 7f804b200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f804c1047e0 0x7f804c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.404+0000 7f804b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f804c108f60 0x7f804c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.404+0000 7f804b200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f804c114010 con 0x7f804c1047e0 2024-09-10T08:23:22.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.404+0000 7f804b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f804c113e90 con 0x7f804c103e30 2024-09-10T08:23:22.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.404+0000 7f804b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f804c114190 con 0x7f804c108f60 2024-09-10T08:23:22.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.405+0000 7f8041400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f804c1047e0 0x7f804c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.405+0000 7f8049e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f804c103e30 0x7f804c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.405+0000 7f8049e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f804c103e30 0x7f804c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:51360/0 (socket says 172.21.15.2:51360) 2024-09-10T08:23:22.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.405+0000 7f8049e00640 1 -- 172.21.15.2:0/3454418703 learned_addr learned my addr 172.21.15.2:0/3454418703 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:22.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.405+0000 7f804a800640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f804c108f60 0x7f804c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.405+0000 7f8049e00640 1 -- 172.21.15.2:0/3454418703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f804c108f60 msgr2=0x7f804c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:22.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.405+0000 7f8049e00640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f804c108f60 0x7f804c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.405+0000 7f8049e00640 1 -- 172.21.15.2:0/3454418703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f804c1047e0 msgr2=0x7f804c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:22.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.405+0000 7f8049e00640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f804c1047e0 0x7f804c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.405+0000 7f8049e00640 1 -- 172.21.15.2:0/3454418703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c19f420 con 0x7f804c103e30 2024-09-10T08:23:22.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.405+0000 7f8041400640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f804c1047e0 0x7f804c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:22.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.406+0000 7f8049e00640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f804c103e30 0x7f804c19a470 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f803c004770 tx=0x7f803c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:22.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.406+0000 7f8043400640 1 -- 172.21.15.2:0/3454418703 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c03f070 con 0x7f804c103e30 2024-09-10T08:23:22.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.406+0000 7f8043400640 1 -- 172.21.15.2:0/3454418703 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f803c031ca0 con 0x7f804c103e30 2024-09-10T08:23:22.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.406+0000 7f804b200640 1 -- 172.21.15.2:0/3454418703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804c19f6b0 con 0x7f804c103e30 2024-09-10T08:23:22.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.406+0000 7f8043400640 1 -- 172.21.15.2:0/3454418703 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c035030 con 0x7f804c103e30 2024-09-10T08:23:22.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.406+0000 7f804b200640 1 -- 172.21.15.2:0/3454418703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804c1a6ff0 con 0x7f804c103e30 2024-09-10T08:23:22.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.409+0000 7f805388b740 1 Processor -- start 2024-09-10T08:23:22.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.409+0000 7f8043400640 1 -- 172.21.15.2:0/3454418703 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f803c042070 con 0x7f804c103e30 2024-09-10T08:23:22.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.409+0000 7f805388b740 1 -- start start 2024-09-10T08:23:22.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.410+0000 7f805388b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa2e40 0x55a2e3fa3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.410+0000 7f805388b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa3760 0x55a2e3fadae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.410+0000 7f805388b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fb0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.410+0000 7f805388b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a2e3f093e0 con 0x55a2e3fae020 2024-09-10T08:23:22.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.410+0000 7f805388b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a2e3eb0d80 con 0x55a2e3fa3760 2024-09-10T08:23:22.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.410+0000 7f805388b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a2e3f053a0 con 0x55a2e3fa2e40 2024-09-10T08:23:22.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.410+0000 7f8049e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa3760 0x55a2e3fadae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.410+0000 7f8049e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa3760 0x55a2e3fadae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:51368/0 (socket says 172.21.15.2:51368) 2024-09-10T08:23:22.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.410+0000 7f8049e00640 1 -- 172.21.15.2:0/4008744462 learned_addr learned my addr 172.21.15.2:0/4008744462 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:22.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.410+0000 7f804a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa2e40 0x55a2e3fa3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.410+0000 7f8041400640 1 --2- 172.21.15.2:0/4008744462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fb0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.411+0000 7f804a800640 1 -- 172.21.15.2:0/4008744462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa3760 msgr2=0x55a2e3fadae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:22.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.411+0000 7f804a800640 1 --2- 172.21.15.2:0/4008744462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa3760 0x55a2e3fadae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.411+0000 7f804a800640 1 -- 172.21.15.2:0/4008744462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 msgr2=0x55a2e3fb0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:22.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.411+0000 7f804a800640 1 --2- 172.21.15.2:0/4008744462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fb0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.411+0000 7f804a800640 1 -- 172.21.15.2:0/4008744462 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e3ecfae0 con 0x55a2e3fa2e40 2024-09-10T08:23:22.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.411+0000 7f804a800640 1 --2- 172.21.15.2:0/4008744462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa2e40 0x55a2e3fa3220 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8044009f00 tx=0x7f8044011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2502aad9103316fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:22.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.411+0000 7f8043400640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f80300a2b80 0x7f80300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.411+0000 7f8043400640 1 -- 172.21.15.2:0/3454418703 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f803c0efa70 con 0x7f804c103e30 2024-09-10T08:23:22.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.411+0000 7f803b400640 1 -- 172.21.15.2:0/4008744462 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044004d20 con 0x55a2e3fa2e40 2024-09-10T08:23:22.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.411+0000 7f8049e00640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f80300a2b80 0x7f80300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.412+0000 7f803b400640 1 -- 172.21.15.2:0/4008744462 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8044004050 con 0x55a2e3fa2e40 2024-09-10T08:23:22.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.412+0000 7f803b400640 1 -- 172.21.15.2:0/4008744462 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044005280 con 0x55a2e3fa2e40 2024-09-10T08:23:22.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.412+0000 7f8049e00640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f80300a2b80 0x7f80300a5040 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f803c0efd10 tx=0x7f803c0f5510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:22.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.412+0000 7f805388b740 1 -- 172.21.15.2:0/4008744462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa2e40 msgr2=0x55a2e3fa3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:22.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.412+0000 7f805388b740 1 --2- 172.21.15.2:0/4008744462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa2e40 0x55a2e3fa3220 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8044009f00 tx=0x7f8044011810 comp rx=0 tx=0).stop 2024-09-10T08:23:22.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.412+0000 7f805388b740 1 -- 172.21.15.2:0/4008744462 shutdown_connections 2024-09-10T08:23:22.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.412+0000 7f805388b740 1 --2- 172.21.15.2:0/4008744462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fb0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.412+0000 7f805388b740 1 --2- 172.21.15.2:0/4008744462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa3760 0x55a2e3fadae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.412+0000 7f805388b740 1 --2- 172.21.15.2:0/4008744462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa2e40 0x55a2e3fa3220 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.412+0000 7f805388b740 1 -- 172.21.15.2:0/4008744462 >> 172.21.15.2:0/4008744462 conn(0x55a2e3f9f3d0 msgr2=0x55a2e3f9f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:22.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.412+0000 7f805388b740 1 -- 172.21.15.2:0/4008744462 shutdown_connections 2024-09-10T08:23:22.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.413+0000 7f805388b740 1 -- 172.21.15.2:0/4008744462 wait complete. 2024-09-10T08:23:22.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.413+0000 7f805388b740 1 -- start start 2024-09-10T08:23:22.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.413+0000 7f805388b740 1 Processor -- start 2024-09-10T08:23:22.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.414+0000 7f805388b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa2e40 0x55a2e3faaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.414+0000 7f805388b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fa3760 0x55a2e3fab470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.414+0000 7f805388b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fae020 0x55a2e3fab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.414+0000 7f805388b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a2e3fb0bf0 con 0x55a2e3fa3760 2024-09-10T08:23:22.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.414+0000 7f805388b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a2e3ec40f0 con 0x55a2e3fae020 2024-09-10T08:23:22.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.414+0000 7f805388b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a2e3fb0d70 con 0x55a2e3fa2e40 2024-09-10T08:23:22.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.414+0000 7f805388b740 10 client.0 _ll_register_callbacks cb 0x55a2e3eabbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:22.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.414+0000 7f8041400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fa3760 0x55a2e3fab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.414+0000 7f8041400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fa3760 0x55a2e3fab470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32900/0 (socket says 172.21.15.2:32900) 2024-09-10T08:23:22.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.414+0000 7f8041400640 1 -- 172.21.15.2:0/1588711125 learned_addr learned my addr 172.21.15.2:0/1588711125 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:22.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.414+0000 7f804a800640 1 --2- 172.21.15.2:0/1588711125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fae020 0x55a2e3fab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.414+0000 7f804a800640 1 --2- 172.21.15.2:0/1588711125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa2e40 0x55a2e3faaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.415+0000 7f8041400640 1 -- 172.21.15.2:0/1588711125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa2e40 msgr2=0x55a2e3faaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:22.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.415+0000 7f8041400640 1 --2- 172.21.15.2:0/1588711125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa2e40 0x55a2e3faaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.415+0000 7f8041400640 1 -- 172.21.15.2:0/1588711125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fae020 msgr2=0x55a2e3fab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:22.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.415+0000 7f8041400640 1 --2- 172.21.15.2:0/1588711125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fae020 0x55a2e3fab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.415+0000 7f8041400640 1 -- 172.21.15.2:0/1588711125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e3c7a3c0 con 0x55a2e3fa3760 2024-09-10T08:23:22.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.415+0000 7f8041400640 1 --2- 172.21.15.2:0/1588711125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fa3760 0x55a2e3fab470 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f8034012d00 tx=0x7f803400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a0972eb9cbd818 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:22.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.415+0000 7f802fe00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:23:22.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.415+0000 7f802fe00640 1 -- 172.21.15.2:0/1588711125 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034009d90 con 0x55a2e3fa3760 2024-09-10T08:23:22.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.415+0000 7f802fe00640 1 -- 172.21.15.2:0/1588711125 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8034012110 con 0x55a2e3fa3760 2024-09-10T08:23:22.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.415+0000 7f805388b740 1 -- 172.21.15.2:0/1588711125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8044004050 con 0x55a2e3fa3760 2024-09-10T08:23:22.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.415+0000 7f802fe00640 1 -- 172.21.15.2:0/1588711125 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8034025050 con 0x55a2e3fa3760 2024-09-10T08:23:22.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.416+0000 7f805388b740 10 client.19719 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:22.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.416+0000 7f805388b740 1 -- 172.21.15.2:0/1588711125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2e3fbccc0 con 0x55a2e3fa3760 2024-09-10T08:23:22.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.416+0000 7f802fe00640 1 -- 172.21.15.2:0/1588711125 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8044004050 con 0x55a2e3fa3760 2024-09-10T08:23:22.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.416+0000 7f802fe00640 1 -- 172.21.15.2:0/1588711125 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8034003ce0 con 0x55a2e3fa3760 2024-09-10T08:23:22.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.416+0000 7f805388b740 10 client.19719 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:22.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.416+0000 7f805388b740 10 client.19719 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:23:22.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.416+0000 7f805388b740 20 client.19719 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:22.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.416+0000 7f805388b740 1 --2- 172.21.15.2:0/1588711125 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55a2e3fbe1d0 0x55a2e3fc05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.416+0000 7f805388b740 4 client.19719 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:23:22.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.416+0000 7f805388b740 1 -- 172.21.15.2:0/1588711125 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2e3f16950 con 0x55a2e3fbe1d0 2024-09-10T08:23:22.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.417+0000 7f804a800640 1 --2- 172.21.15.2:0/1588711125 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55a2e3fbe1d0 0x55a2e3fc05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.417+0000 7f804a800640 1 --2- 172.21.15.2:0/1588711125 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55a2e3fbe1d0 0x55a2e3fc05f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90c9af0d8ee2e276 server_cookie=523246387c7890be in_seq=0 out_seq=0 2024-09-10T08:23:22.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.417+0000 7f802fe00640 10 client.19719 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:23:22.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.418+0000 7f802fe00640 1 -- 172.21.15.2:0/1588711125 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2e3f16950 con 0x55a2e3fbe1d0 2024-09-10T08:23:22.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.418+0000 7f802fe00640 10 client.19719 handle_command_reply: tid=0 2024-09-10T08:23:22.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.418+0000 7f805388b740 1 client.19719 shutdown 2024-09-10T08:23:22.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.418+0000 7f805388b740 1 -- 172.21.15.2:0/1588711125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fa3760 msgr2=0x55a2e3fab470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:22.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.418+0000 7f805388b740 1 --2- 172.21.15.2:0/1588711125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fa3760 0x55a2e3fab470 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f8034012d00 tx=0x7f803400f790 comp rx=0 tx=0).stop 2024-09-10T08:23:22.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.418+0000 7f805388b740 1 -- 172.21.15.2:0/1588711125 shutdown_connections 2024-09-10T08:23:22.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.418+0000 7f805388b740 1 --2- 172.21.15.2:0/1588711125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fae020 0x55a2e3fab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.418+0000 7f805388b740 1 --2- 172.21.15.2:0/1588711125 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55a2e3fbe1d0 0x55a2e3fc05f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.418+0000 7f805388b740 1 --2- 172.21.15.2:0/1588711125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fa3760 0x55a2e3fab470 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.418+0000 7f805388b740 1 --2- 172.21.15.2:0/1588711125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa2e40 0x55a2e3faaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.418+0000 7f802fe00640 0 client.19719 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:23:22.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.418+0000 7f805388b740 1 -- 172.21.15.2:0/1588711125 >> 172.21.15.2:0/1588711125 conn(0x55a2e3f9f3d0 msgr2=0x55a2e3fa50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:22.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.418+0000 7f805388b740 1 -- 172.21.15.2:0/1588711125 shutdown_connections 2024-09-10T08:23:22.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.419+0000 7f805388b740 1 -- 172.21.15.2:0/1588711125 wait complete. 2024-09-10T08:23:22.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.419+0000 7f805388b740 20 client.19719 trim_cache size 0 max 16384 2024-09-10T08:23:22.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.433+0000 7f805388b740 1 Processor -- start 2024-09-10T08:23:22.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.433+0000 7f805388b740 1 -- start start 2024-09-10T08:23:22.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.433+0000 7f805388b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa2e40 0x55a2e3fa9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.433+0000 7f805388b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa3760 0x55a2e3fab800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.433+0000 7f805388b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fc5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.433+0000 7f805388b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1b37b0 con 0x55a2e3fae020 2024-09-10T08:23:22.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.433+0000 7f805388b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1b1080 con 0x55a2e3fa2e40 2024-09-10T08:23:22.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.433+0000 7f805388b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1aeed0 con 0x55a2e3fa3760 2024-09-10T08:23:22.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.434+0000 7f8041400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa3760 0x55a2e3fab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.434+0000 7f804a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa2e40 0x55a2e3fa9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.434+0000 7f8041400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa3760 0x55a2e3fab800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34886/0 (socket says 172.21.15.2:34886) 2024-09-10T08:23:22.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.434+0000 7f8041400640 1 -- 172.21.15.2:0/4228928021 learned_addr learned my addr 172.21.15.2:0/4228928021 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:22.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.434+0000 7f804a800640 1 --2- 172.21.15.2:0/4228928021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fc5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.434+0000 7f8041400640 1 -- 172.21.15.2:0/4228928021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa2e40 msgr2=0x55a2e3fa9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:22.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.434+0000 7f8041400640 1 --2- 172.21.15.2:0/4228928021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa2e40 0x55a2e3fa9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.434+0000 7f8041400640 1 -- 172.21.15.2:0/4228928021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 msgr2=0x55a2e3fc5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:22.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.434+0000 7f8041400640 1 --2- 172.21.15.2:0/4228928021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fc5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.434+0000 7f8041400640 1 -- 172.21.15.2:0/4228928021 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e3fad8a0 con 0x55a2e3fa3760 2024-09-10T08:23:22.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.434+0000 7f8041400640 1 --2- 172.21.15.2:0/4228928021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa3760 0x55a2e3fab800 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8034012850 tx=0x7f8034005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee42962e721cfb79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:22.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.435+0000 7f803aa00640 1 -- 172.21.15.2:0/4228928021 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803402e070 con 0x55a2e3fa3760 2024-09-10T08:23:22.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.435+0000 7f803aa00640 1 -- 172.21.15.2:0/4228928021 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8034012110 con 0x55a2e3fa3760 2024-09-10T08:23:22.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.435+0000 7f803aa00640 1 -- 172.21.15.2:0/4228928021 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034005430 con 0x55a2e3fa3760 2024-09-10T08:23:22.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.435+0000 7f805388b740 1 -- 172.21.15.2:0/4228928021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa3760 msgr2=0x55a2e3fab800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:22.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.435+0000 7f805388b740 1 --2- 172.21.15.2:0/4228928021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa3760 0x55a2e3fab800 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8034012850 tx=0x7f8034005970 comp rx=0 tx=0).stop 2024-09-10T08:23:22.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.435+0000 7f805388b740 1 -- 172.21.15.2:0/4228928021 shutdown_connections 2024-09-10T08:23:22.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.435+0000 7f805388b740 1 --2- 172.21.15.2:0/4228928021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fc5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.435+0000 7f805388b740 1 --2- 172.21.15.2:0/4228928021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa3760 0x55a2e3fab800 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.435+0000 7f805388b740 1 --2- 172.21.15.2:0/4228928021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa2e40 0x55a2e3fa9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.435+0000 7f805388b740 1 -- 172.21.15.2:0/4228928021 >> 172.21.15.2:0/4228928021 conn(0x55a2e3f9f3d0 msgr2=0x55a2e3facfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:22.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.436+0000 7f805388b740 1 -- 172.21.15.2:0/4228928021 shutdown_connections 2024-09-10T08:23:22.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.436+0000 7f805388b740 1 -- 172.21.15.2:0/4228928021 wait complete. 2024-09-10T08:23:22.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.436+0000 7f805388b740 1 -- start start 2024-09-10T08:23:22.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.436+0000 7f805388b740 1 Processor -- start 2024-09-10T08:23:22.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.437+0000 7f805388b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa2e40 0x55a2e3fb1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.437+0000 7f805388b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa3760 0x55a2e3fb2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.437+0000 7f805388b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fb26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.437+0000 7f805388b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1af540 con 0x55a2e3fae020 2024-09-10T08:23:22.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.437+0000 7f805388b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1b09b0 con 0x55a2e3fa2e40 2024-09-10T08:23:22.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.437+0000 7f805388b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1afc10 con 0x55a2e3fa3760 2024-09-10T08:23:22.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.437+0000 7f805388b740 10 client.0 _ll_register_callbacks cb 0x7f8034009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:22.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.437+0000 7f804a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fb26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.437+0000 7f804a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fb26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32922/0 (socket says 172.21.15.2:32922) 2024-09-10T08:23:22.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.437+0000 7f804a800640 1 -- 172.21.15.2:0/1723284047 learned_addr learned my addr 172.21.15.2:0/1723284047 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:22.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.437+0000 7f804a800640 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa2e40 0x55a2e3fb1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.437+0000 7f8041400640 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa3760 0x55a2e3fb2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.438+0000 7f804a800640 1 -- 172.21.15.2:0/1723284047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa3760 msgr2=0x55a2e3fb2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:22.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.438+0000 7f804a800640 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa3760 0x55a2e3fb2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.438+0000 7f804a800640 1 -- 172.21.15.2:0/1723284047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa2e40 msgr2=0x55a2e3fb1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:22.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.438+0000 7f804a800640 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa2e40 0x55a2e3fb1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.438+0000 7f804a800640 1 -- 172.21.15.2:0/1723284047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e4030680 con 0x55a2e3fae020 2024-09-10T08:23:22.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.438+0000 7f8041400640 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa3760 0x55a2e3fb2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:22.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.438+0000 7f804a800640 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fb26d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f8044000f80 tx=0x7f8044009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5e994bf1cdf789 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:22.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.438+0000 7f802fe00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:23:22.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.438+0000 7f802fe00640 1 -- 172.21.15.2:0/1723284047 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044003cd0 con 0x55a2e3fae020 2024-09-10T08:23:22.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.438+0000 7f802fe00640 1 -- 172.21.15.2:0/1723284047 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8044028450 con 0x55a2e3fae020 2024-09-10T08:23:22.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.438+0000 7f805388b740 1 -- 172.21.15.2:0/1723284047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8034012110 con 0x55a2e3fae020 2024-09-10T08:23:22.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.439+0000 7f802fe00640 1 -- 172.21.15.2:0/1723284047 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8044028650 con 0x55a2e3fae020 2024-09-10T08:23:22.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.439+0000 7f805388b740 10 client.19725 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:22.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.439+0000 7f805388b740 1 -- 172.21.15.2:0/1723284047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2e40318a0 con 0x55a2e3fae020 2024-09-10T08:23:22.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.439+0000 7f802fe00640 1 -- 172.21.15.2:0/1723284047 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8034012110 con 0x55a2e3fae020 2024-09-10T08:23:22.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.439+0000 7f802fe00640 1 -- 172.21.15.2:0/1723284047 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8044019700 con 0x55a2e3fae020 2024-09-10T08:23:22.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.439+0000 7f805388b740 10 client.19725 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:22.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.439+0000 7f805388b740 10 client.19725 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:23:22.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.439+0000 7f805388b740 20 client.19725 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:22.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.439+0000 7f805388b740 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55a2e4031d80 0x55a2e4034170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:22.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.439+0000 7f805388b740 4 client.19725 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:23:22.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.439+0000 7f805388b740 1 -- 172.21.15.2:0/1723284047 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2e3f10e60 con 0x55a2e4031d80 2024-09-10T08:23:22.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.439+0000 7f8041400640 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55a2e4031d80 0x55a2e4034170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:22.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.440+0000 7f8041400640 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55a2e4031d80 0x55a2e4034170 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8150f3d44e2e4778 server_cookie=d5b9fd1b744ba82b in_seq=0 out_seq=0 2024-09-10T08:23:22.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.440+0000 7f802fe00640 10 client.19725 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:23:22.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.440+0000 7f802fe00640 1 -- 172.21.15.2:0/1723284047 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2e3f10e60 con 0x55a2e4031d80 2024-09-10T08:23:22.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.440+0000 7f802fe00640 10 client.19725 handle_command_reply: tid=0 2024-09-10T08:23:22.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.440+0000 7f805388b740 1 client.19725 shutdown 2024-09-10T08:23:22.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.440+0000 7f805388b740 1 -- 172.21.15.2:0/1723284047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 msgr2=0x55a2e3fb26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:22.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.440+0000 7f805388b740 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fb26d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f8044000f80 tx=0x7f8044009500 comp rx=0 tx=0).stop 2024-09-10T08:23:22.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.440+0000 7f805388b740 1 -- 172.21.15.2:0/1723284047 shutdown_connections 2024-09-10T08:23:22.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.440+0000 7f805388b740 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55a2e4031d80 0x55a2e4034170 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.441+0000 7f805388b740 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a2e3fae020 0x55a2e3fb26d0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.441+0000 7f805388b740 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2e3fa3760 0x55a2e3fb2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.441+0000 7f805388b740 1 --2- 172.21.15.2:0/1723284047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2e3fa2e40 0x55a2e3fb1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.441+0000 7f805388b740 1 -- 172.21.15.2:0/1723284047 >> 172.21.15.2:0/1723284047 conn(0x55a2e3f9f3d0 msgr2=0x55a2e3f9dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:22.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.441+0000 7f802fe00640 0 client.19725 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:23:22.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.441+0000 7f805388b740 1 -- 172.21.15.2:0/1723284047 shutdown_connections 2024-09-10T08:23:22.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.441+0000 7f805388b740 1 -- 172.21.15.2:0/1723284047 wait complete. 2024-09-10T08:23:22.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.441+0000 7f805388b740 20 client.19725 trim_cache size 0 max 16384 2024-09-10T08:23:22.446 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:23:22.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.442+0000 7f804b200640 1 -- 172.21.15.2:0/3454418703 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f80300a2b80 msgr2=0x7f80300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:22.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.442+0000 7f804b200640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f80300a2b80 0x7f80300a5040 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f803c0efd10 tx=0x7f803c0f5510 comp rx=0 tx=0).stop 2024-09-10T08:23:22.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.442+0000 7f804b200640 1 -- 172.21.15.2:0/3454418703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f804c103e30 msgr2=0x7f804c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:22.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.442+0000 7f804b200640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f804c103e30 0x7f804c19a470 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f803c004770 tx=0x7f803c003c10 comp rx=0 tx=0).stop 2024-09-10T08:23:22.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.442+0000 7f804b200640 1 -- 172.21.15.2:0/3454418703 shutdown_connections 2024-09-10T08:23:22.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.442+0000 7f804b200640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f80300a2b80 0x7f80300a5040 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.442+0000 7f804b200640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f804c108f60 0x7f804c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.442+0000 7f804b200640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f804c1047e0 0x7f804c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.442+0000 7f804b200640 1 --2- 172.21.15.2:0/3454418703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f804c103e30 0x7f804c19a470 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:22.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.442+0000 7f804b200640 1 -- 172.21.15.2:0/3454418703 >> 172.21.15.2:0/3454418703 conn(0x7f804c0ff8d0 msgr2=0x7f804c100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:22.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.442+0000 7f804b200640 1 -- 172.21.15.2:0/3454418703 shutdown_connections 2024-09-10T08:23:22.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:22.442+0000 7f804b200640 1 -- 172.21.15.2:0/3454418703 wait complete. 2024-09-10T08:23:23.489 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:23:23.540 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:23 smithi002 ceph-mon[30563]: pgmap v844: 129 pgs: 129 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:23:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:23:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:23 smithi155 ceph-mon[35405]: pgmap v844: 129 pgs: 129 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:23:23.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:23 smithi107 ceph-mon[35048]: pgmap v844: 129 pgs: 129 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:23:24.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.602+0000 7f2c9ea00640 1 -- 172.21.15.2:0/2237779980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c981047e0 msgr2=0x7f2c98108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:24.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.602+0000 7f2c9ea00640 1 --2- 172.21.15.2:0/2237779980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c981047e0 0x7f2c98108830 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009900 tx=0x7f2c8402a410 comp rx=0 tx=0).stop 2024-09-10T08:23:24.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.602+0000 7f2c9ea00640 1 -- 172.21.15.2:0/2237779980 shutdown_connections 2024-09-10T08:23:24.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.602+0000 7f2c9ea00640 1 --2- 172.21.15.2:0/2237779980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c98108f60 0x7f2c98110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.602+0000 7f2c9ea00640 1 --2- 172.21.15.2:0/2237779980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c981047e0 0x7f2c98108830 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.602+0000 7f2c9ea00640 1 --2- 172.21.15.2:0/2237779980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c98103e30 0x7f2c98104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.602+0000 7f2c9ea00640 1 -- 172.21.15.2:0/2237779980 >> 172.21.15.2:0/2237779980 conn(0x7f2c980ff8d0 msgr2=0x7f2c98101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:24.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.602+0000 7f2c9ea00640 1 -- 172.21.15.2:0/2237779980 shutdown_connections 2024-09-10T08:23:24.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.602+0000 7f2c9ea00640 1 -- 172.21.15.2:0/2237779980 wait complete. 2024-09-10T08:23:24.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.603+0000 7f2c9ea00640 1 Processor -- start 2024-09-10T08:23:24.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.603+0000 7f2c9ea00640 1 -- start start 2024-09-10T08:23:24.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.603+0000 7f2c9ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c98103e30 0x7f2c9819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.603+0000 7f2c9ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c981047e0 0x7f2c9819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.603+0000 7f2c9ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c98108f60 0x7f2c9819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.603+0000 7f2c9ea00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98113fd0 con 0x7f2c98108f60 2024-09-10T08:23:24.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.603+0000 7f2c9ea00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98113e50 con 0x7f2c981047e0 2024-09-10T08:23:24.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.603+0000 7f2c9ea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98114150 con 0x7f2c98103e30 2024-09-10T08:23:24.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.604+0000 7f2c9cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c981047e0 0x7f2c9819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.604+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c98108f60 0x7f2c9819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.604+0000 7f2c9d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c98103e30 0x7f2c9819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.604+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c98108f60 0x7f2c9819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32928/0 (socket says 172.21.15.2:32928) 2024-09-10T08:23:24.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.604+0000 7f2c9cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c981047e0 0x7f2c9819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:51420/0 (socket says 172.21.15.2:51420) 2024-09-10T08:23:24.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.604+0000 7f2c9e000640 1 -- 172.21.15.2:0/2600646731 learned_addr learned my addr 172.21.15.2:0/2600646731 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:24.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.604+0000 7f2c9d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c98103e30 0x7f2c9819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34910/0 (socket says 172.21.15.2:34910) 2024-09-10T08:23:24.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.604+0000 7f2c9d600640 1 -- 172.21.15.2:0/2600646731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c981047e0 msgr2=0x7f2c9819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:24.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.604+0000 7f2c9d600640 1 --2- 172.21.15.2:0/2600646731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c981047e0 0x7f2c9819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.604+0000 7f2c9d600640 1 -- 172.21.15.2:0/2600646731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c98108f60 msgr2=0x7f2c9819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:24.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.604+0000 7f2c9d600640 1 --2- 172.21.15.2:0/2600646731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c98108f60 0x7f2c9819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.604+0000 7f2c9d600640 1 -- 172.21.15.2:0/2600646731 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9819f440 con 0x7f2c98103e30 2024-09-10T08:23:24.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.604+0000 7f2c9d600640 1 --2- 172.21.15.2:0/2600646731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c98103e30 0x7f2c9819a490 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00d6d0 tx=0x7f2c8c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:24.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.605+0000 7f2c96000640 1 -- 172.21.15.2:0/2600646731 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c017070 con 0x7f2c98103e30 2024-09-10T08:23:24.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.605+0000 7f2c96000640 1 -- 172.21.15.2:0/2600646731 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c8c009820 con 0x7f2c98103e30 2024-09-10T08:23:24.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.605+0000 7f2c9ea00640 1 -- 172.21.15.2:0/2600646731 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9819f730 con 0x7f2c98103e30 2024-09-10T08:23:24.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.605+0000 7f2c96000640 1 -- 172.21.15.2:0/2600646731 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c004d00 con 0x7f2c98103e30 2024-09-10T08:23:24.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.605+0000 7f2c9ea00640 1 -- 172.21.15.2:0/2600646731 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c981a7010 con 0x7f2c98103e30 2024-09-10T08:23:24.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.608+0000 7f2ca29f6740 1 Processor -- start 2024-09-10T08:23:24.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.608+0000 7f2c96000640 1 -- 172.21.15.2:0/2600646731 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2c8c0099d0 con 0x7f2c98103e30 2024-09-10T08:23:24.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.608+0000 7f2ca29f6740 1 -- start start 2024-09-10T08:23:24.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.608+0000 7f2ca29f6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad23ce40 0x55efad23d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.608+0000 7f2ca29f6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23d760 0x55efad247ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.608+0000 7f2ca29f6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad248020 0x55efad24a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.608+0000 7f2ca29f6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55efad1a33e0 con 0x55efad248020 2024-09-10T08:23:24.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.608+0000 7f2ca29f6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55efad14ad80 con 0x55efad23d760 2024-09-10T08:23:24.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.608+0000 7f2ca29f6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55efad19f3a0 con 0x55efad23ce40 2024-09-10T08:23:24.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.609+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad23ce40 0x55efad23d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.609+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad23ce40 0x55efad23d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34918/0 (socket says 172.21.15.2:34918) 2024-09-10T08:23:24.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.609+0000 7f2c9e000640 1 -- 172.21.15.2:0/1702257332 learned_addr learned my addr 172.21.15.2:0/1702257332 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:24.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.609+0000 7f2c9cc00640 1 --2- 172.21.15.2:0/1702257332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad248020 0x55efad24a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.609+0000 7f2c9d600640 1 --2- 172.21.15.2:0/1702257332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23d760 0x55efad247ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.609+0000 7f2c9e000640 1 -- 172.21.15.2:0/1702257332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23d760 msgr2=0x55efad247ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:24.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.609+0000 7f2c9e000640 1 --2- 172.21.15.2:0/1702257332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23d760 0x55efad247ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.609+0000 7f2c9e000640 1 -- 172.21.15.2:0/1702257332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad248020 msgr2=0x55efad24a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:24.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.609+0000 7f2c9e000640 1 --2- 172.21.15.2:0/1702257332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad248020 0x55efad24a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.609+0000 7f2c9e000640 1 -- 172.21.15.2:0/1702257332 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efad169ae0 con 0x55efad23ce40 2024-09-10T08:23:24.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.609+0000 7f2c9cc00640 1 --2- 172.21.15.2:0/1702257332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad248020 0x55efad24a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:23:24.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.609+0000 7f2c96000640 1 --2- 172.21.15.2:0/2600646731 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c800a2b80 0x7f2c800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.609+0000 7f2c9e000640 1 --2- 172.21.15.2:0/1702257332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad23ce40 0x55efad23d220 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009fb0 tx=0x7f2c90011870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ee340a517c67636 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:24.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.610+0000 7f2c96000640 1 -- 172.21.15.2:0/2600646731 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2c8c0cef50 con 0x7f2c98103e30 2024-09-10T08:23:24.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.610+0000 7f2c9d600640 1 --2- 172.21.15.2:0/2600646731 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c800a2b80 0x7f2c800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.610+0000 7f2c8a600640 1 -- 172.21.15.2:0/1702257332 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c900127d0 con 0x55efad23ce40 2024-09-10T08:23:24.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.610+0000 7f2c8a600640 1 -- 172.21.15.2:0/1702257332 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c90012e50 con 0x55efad23ce40 2024-09-10T08:23:24.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.610+0000 7f2c8a600640 1 -- 172.21.15.2:0/1702257332 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c900193a0 con 0x55efad23ce40 2024-09-10T08:23:24.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.610+0000 7f2ca29f6740 1 -- 172.21.15.2:0/1702257332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad23ce40 msgr2=0x55efad23d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:24.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.610+0000 7f2c9d600640 1 --2- 172.21.15.2:0/2600646731 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c800a2b80 0x7f2c800a5040 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0cc9d0 tx=0x7f2c8c097040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:24.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.610+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/1702257332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad23ce40 0x55efad23d220 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009fb0 tx=0x7f2c90011870 comp rx=0 tx=0).stop 2024-09-10T08:23:24.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.611+0000 7f2ca29f6740 1 -- 172.21.15.2:0/1702257332 shutdown_connections 2024-09-10T08:23:24.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.611+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/1702257332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad248020 0x55efad24a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.611+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/1702257332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23d760 0x55efad247ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.611+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/1702257332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad23ce40 0x55efad23d220 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.611+0000 7f2ca29f6740 1 -- 172.21.15.2:0/1702257332 >> 172.21.15.2:0/1702257332 conn(0x55efad2393d0 msgr2=0x55efad2397b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:24.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.611+0000 7f2ca29f6740 1 -- 172.21.15.2:0/1702257332 shutdown_connections 2024-09-10T08:23:24.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.611+0000 7f2ca29f6740 1 -- 172.21.15.2:0/1702257332 wait complete. 2024-09-10T08:23:24.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.611+0000 7f2ca29f6740 1 -- start start 2024-09-10T08:23:24.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.611+0000 7f2ca29f6740 1 Processor -- start 2024-09-10T08:23:24.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.612+0000 7f2ca29f6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23ce40 0x55efad244ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.612+0000 7f2ca29f6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad23d760 0x55efad245410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.612+0000 7f2ca29f6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad248020 0x55efad245950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.612+0000 7f2ca29f6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55efad24abf0 con 0x55efad248020 2024-09-10T08:23:24.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.612+0000 7f2ca29f6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55efad15e0f0 con 0x55efad23ce40 2024-09-10T08:23:24.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.612+0000 7f2ca29f6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55efad24ad70 con 0x55efad23d760 2024-09-10T08:23:24.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.612+0000 7f2ca29f6740 10 client.0 _ll_register_callbacks cb 0x55efad145bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:24.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.612+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad248020 0x55efad245950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.612+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad248020 0x55efad245950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32958/0 (socket says 172.21.15.2:32958) 2024-09-10T08:23:24.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.612+0000 7f2c9e000640 1 -- 172.21.15.2:0/2633337089 learned_addr learned my addr 172.21.15.2:0/2633337089 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:24.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.612+0000 7f2c9cc00640 1 --2- 172.21.15.2:0/2633337089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad23d760 0x55efad245410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.612+0000 7f2c9e000640 1 --2- 172.21.15.2:0/2633337089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23ce40 0x55efad244ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.613+0000 7f2c9cc00640 1 -- 172.21.15.2:0/2633337089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23ce40 msgr2=0x55efad244ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:24.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.613+0000 7f2c9cc00640 1 --2- 172.21.15.2:0/2633337089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23ce40 0x55efad244ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.613+0000 7f2c9cc00640 1 -- 172.21.15.2:0/2633337089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad248020 msgr2=0x55efad245950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:24.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.613+0000 7f2c9cc00640 1 --2- 172.21.15.2:0/2633337089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad248020 0x55efad245950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.613+0000 7f2c9cc00640 1 -- 172.21.15.2:0/2633337089 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efacf143c0 con 0x55efad23d760 2024-09-10T08:23:24.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.613+0000 7f2c9cc00640 1 --2- 172.21.15.2:0/2633337089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad23d760 0x55efad245410 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402a930 tx=0x7f2c840044d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=459b6738a4bf0dd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:24.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.613+0000 7f2c7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:23:24.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.613+0000 7f2c7ea00640 1 -- 172.21.15.2:0/2633337089 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84035040 con 0x55efad23d760 2024-09-10T08:23:24.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.613+0000 7f2ca29f6740 1 -- 172.21.15.2:0/2633337089 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c90012e50 con 0x55efad23d760 2024-09-10T08:23:24.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.613+0000 7f2c7ea00640 1 -- 172.21.15.2:0/2633337089 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2c84005510 con 0x55efad23d760 2024-09-10T08:23:24.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.614+0000 7f2c7ea00640 1 -- 172.21.15.2:0/2633337089 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c8404e050 con 0x55efad23d760 2024-09-10T08:23:24.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.614+0000 7f2ca29f6740 10 client.29689 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:24.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.614+0000 7f2ca29f6740 1 -- 172.21.15.2:0/2633337089 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55efad256c50 con 0x55efad23d760 2024-09-10T08:23:24.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.614+0000 7f2c7ea00640 1 -- 172.21.15.2:0/2633337089 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c90012e50 con 0x55efad23d760 2024-09-10T08:23:24.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.614+0000 7f2c7ea00640 1 -- 172.21.15.2:0/2633337089 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2c84031980 con 0x55efad23d760 2024-09-10T08:23:24.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.614+0000 7f2ca29f6740 10 client.29689 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:24.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.614+0000 7f2ca29f6740 10 client.29689 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:23:24.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.614+0000 7f2ca29f6740 20 client.29689 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:24.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.614+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/2633337089 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55efad258110 0x55efad25a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.614+0000 7f2ca29f6740 4 client.29689 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:23:24.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.615+0000 7f2ca29f6740 1 -- 172.21.15.2:0/2633337089 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efad1b0950 con 0x55efad258110 2024-09-10T08:23:24.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.615+0000 7f2c9e000640 1 --2- 172.21.15.2:0/2633337089 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55efad258110 0x55efad25a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.615+0000 7f2c9e000640 1 --2- 172.21.15.2:0/2633337089 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55efad258110 0x55efad25a500 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc7424ac885a4192 server_cookie=517bae08d7d043c8 in_seq=0 out_seq=0 2024-09-10T08:23:24.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.615+0000 7f2c7ea00640 10 client.29689 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:23:24.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.616+0000 7f2c7ea00640 1 -- 172.21.15.2:0/2633337089 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55efad1b0950 con 0x55efad258110 2024-09-10T08:23:24.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.616+0000 7f2c7ea00640 10 client.29689 handle_command_reply: tid=0 2024-09-10T08:23:24.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.616+0000 7f2ca29f6740 1 client.29689 shutdown 2024-09-10T08:23:24.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.617+0000 7f2ca29f6740 1 -- 172.21.15.2:0/2633337089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad23d760 msgr2=0x55efad245410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:24.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.617+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/2633337089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad23d760 0x55efad245410 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402a930 tx=0x7f2c840044d0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.617+0000 7f2ca29f6740 1 -- 172.21.15.2:0/2633337089 shutdown_connections 2024-09-10T08:23:24.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.617+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/2633337089 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55efad258110 0x55efad25a500 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.617+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/2633337089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad248020 0x55efad245950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.617+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/2633337089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad23d760 0x55efad245410 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.617+0000 7f2c7ea00640 0 client.29689 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:23:24.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.617+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/2633337089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23ce40 0x55efad244ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.617+0000 7f2ca29f6740 1 -- 172.21.15.2:0/2633337089 >> 172.21.15.2:0/2633337089 conn(0x55efad2393d0 msgr2=0x55efad23f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:24.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.617+0000 7f2ca29f6740 1 -- 172.21.15.2:0/2633337089 shutdown_connections 2024-09-10T08:23:24.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.617+0000 7f2ca29f6740 1 -- 172.21.15.2:0/2633337089 wait complete. 2024-09-10T08:23:24.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.617+0000 7f2ca29f6740 20 client.29689 trim_cache size 0 max 16384 2024-09-10T08:23:24.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.631+0000 7f2ca29f6740 1 Processor -- start 2024-09-10T08:23:24.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.631+0000 7f2ca29f6740 1 -- start start 2024-09-10T08:23:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.631+0000 7f2ca29f6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23ce40 0x55efad2457a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.631+0000 7f2ca29f6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23d760 0x55efad25fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.631+0000 7f2ca29f6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad248020 0x55efad260310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.631+0000 7f2ca29f6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b37a0 con 0x55efad23ce40 2024-09-10T08:23:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.631+0000 7f2ca29f6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b1070 con 0x55efad23d760 2024-09-10T08:23:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.631+0000 7f2ca29f6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981aeec0 con 0x55efad248020 2024-09-10T08:23:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.632+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23ce40 0x55efad2457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.632+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23ce40 0x55efad2457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32972/0 (socket says 172.21.15.2:32972) 2024-09-10T08:23:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.632+0000 7f2c9cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23d760 0x55efad25fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.632+0000 7f2c9e000640 1 -- 172.21.15.2:0/474795702 learned_addr learned my addr 172.21.15.2:0/474795702 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:24.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.632+0000 7f2c9e000640 1 -- 172.21.15.2:0/474795702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad248020 msgr2=0x55efad260310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.632+0000 7f2c9e000640 1 --2- 172.21.15.2:0/474795702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad248020 0x55efad260310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.632+0000 7f2c9e000640 1 -- 172.21.15.2:0/474795702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23d760 msgr2=0x55efad25fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.632+0000 7f2c9e000640 1 --2- 172.21.15.2:0/474795702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23d760 0x55efad25fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.632+0000 7f2c9e000640 1 -- 172.21.15.2:0/474795702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efad240ab0 con 0x55efad23ce40 2024-09-10T08:23:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.632+0000 7f2c9cc00640 1 --2- 172.21.15.2:0/474795702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23d760 0x55efad25fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.632+0000 7f2c9e000640 1 --2- 172.21.15.2:0/474795702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23ce40 0x55efad2457a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f2c90011fe0 tx=0x7f2c9001aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e92c935d39c805 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.633+0000 7f2c89c00640 1 -- 172.21.15.2:0/474795702 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9001a710 con 0x55efad23ce40 2024-09-10T08:23:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.633+0000 7f2c89c00640 1 -- 172.21.15.2:0/474795702 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c9002f030 con 0x55efad23ce40 2024-09-10T08:23:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.633+0000 7f2c89c00640 1 -- 172.21.15.2:0/474795702 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90012d50 con 0x55efad23ce40 2024-09-10T08:23:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.633+0000 7f2ca29f6740 1 -- 172.21.15.2:0/474795702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23ce40 msgr2=0x55efad2457a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.633+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/474795702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23ce40 0x55efad2457a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f2c90011fe0 tx=0x7f2c9001aa60 comp rx=0 tx=0).stop 2024-09-10T08:23:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.633+0000 7f2ca29f6740 1 -- 172.21.15.2:0/474795702 shutdown_connections 2024-09-10T08:23:24.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.633+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/474795702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad248020 0x55efad260310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.634+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/474795702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23d760 0x55efad25fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.634+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/474795702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23ce40 0x55efad2457a0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.634+0000 7f2ca29f6740 1 -- 172.21.15.2:0/474795702 >> 172.21.15.2:0/474795702 conn(0x55efad2393d0 msgr2=0x55efad246f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.634+0000 7f2ca29f6740 1 -- 172.21.15.2:0/474795702 shutdown_connections 2024-09-10T08:23:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.634+0000 7f2ca29f6740 1 -- 172.21.15.2:0/474795702 wait complete. 2024-09-10T08:23:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.634+0000 7f2ca29f6740 1 -- start start 2024-09-10T08:23:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.634+0000 7f2ca29f6740 1 Processor -- start 2024-09-10T08:23:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.635+0000 7f2ca29f6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23ce40 0x55efad250540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.635+0000 7f2ca29f6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23d760 0x55efad24b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.635+0000 7f2ca29f6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad248020 0x55efad24be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.635+0000 7f2ca29f6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981af530 con 0x55efad23d760 2024-09-10T08:23:24.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.635+0000 7f2ca29f6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b09a0 con 0x55efad23ce40 2024-09-10T08:23:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.635+0000 7f2ca29f6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981afc00 con 0x55efad248020 2024-09-10T08:23:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.635+0000 7f2ca29f6740 10 client.0 _ll_register_callbacks cb 0x7f2c84009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.635+0000 7f2c9cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23d760 0x55efad24b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.635+0000 7f2c9cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23d760 0x55efad24b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32984/0 (socket says 172.21.15.2:32984) 2024-09-10T08:23:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.635+0000 7f2c9cc00640 1 -- 172.21.15.2:0/4027211176 learned_addr learned my addr 172.21.15.2:0/4027211176 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.635+0000 7f2c9e000640 1 --2- 172.21.15.2:0/4027211176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23ce40 0x55efad250540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.635+0000 7f2c9e000640 1 --2- 172.21.15.2:0/4027211176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad248020 0x55efad24be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.636+0000 7f2c9cc00640 1 -- 172.21.15.2:0/4027211176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad248020 msgr2=0x55efad24be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.636+0000 7f2c9cc00640 1 --2- 172.21.15.2:0/4027211176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad248020 0x55efad24be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.636+0000 7f2c9cc00640 1 -- 172.21.15.2:0/4027211176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23ce40 msgr2=0x55efad250540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.636+0000 7f2c9cc00640 1 --2- 172.21.15.2:0/4027211176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23ce40 0x55efad250540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.636+0000 7f2c9cc00640 1 -- 172.21.15.2:0/4027211176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efad24c440 con 0x55efad23d760 2024-09-10T08:23:24.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.636+0000 7f2c9cc00640 1 --2- 172.21.15.2:0/4027211176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23d760 0x55efad24b8c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f2c84031d50 tx=0x7f2c840055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c8cdeaed6e224c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:24.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.636+0000 7f2c7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:23:24.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.636+0000 7f2c7ea00640 1 -- 172.21.15.2:0/4027211176 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84057060 con 0x55efad23d760 2024-09-10T08:23:24.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.636+0000 7f2c7ea00640 1 -- 172.21.15.2:0/4027211176 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2c840035f0 con 0x55efad23d760 2024-09-10T08:23:24.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.636+0000 7f2ca29f6740 1 -- 172.21.15.2:0/4027211176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c9002f030 con 0x55efad23d760 2024-09-10T08:23:24.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.636+0000 7f2c7ea00640 1 -- 172.21.15.2:0/4027211176 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c8405e050 con 0x55efad23d760 2024-09-10T08:23:24.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.636+0000 7f2ca29f6740 10 client.19743 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:24.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.637+0000 7f2ca29f6740 1 -- 172.21.15.2:0/4027211176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55efad2c2bd0 con 0x55efad23d760 2024-09-10T08:23:24.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.637+0000 7f2c7ea00640 1 -- 172.21.15.2:0/4027211176 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c9002f030 con 0x55efad23d760 2024-09-10T08:23:24.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.637+0000 7f2c7ea00640 1 -- 172.21.15.2:0/4027211176 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2c84003b60 con 0x55efad23d760 2024-09-10T08:23:24.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.637+0000 7f2ca29f6740 10 client.19743 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:24.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.637+0000 7f2ca29f6740 10 client.19743 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:23:24.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.637+0000 7f2ca29f6740 20 client.19743 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:24.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.637+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/4027211176 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55efad2c32c0 0x55efad2c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:24.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.637+0000 7f2ca29f6740 4 client.19743 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:23:24.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.637+0000 7f2ca29f6740 1 -- 172.21.15.2:0/4027211176 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55efad1aae60 con 0x55efad2c32c0 2024-09-10T08:23:24.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.638+0000 7f2c9e000640 1 --2- 172.21.15.2:0/4027211176 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55efad2c32c0 0x55efad2c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:24.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.638+0000 7f2c9e000640 1 --2- 172.21.15.2:0/4027211176 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55efad2c32c0 0x55efad2c56e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e561cbd359955fd server_cookie=f2349aaf6e604fcf in_seq=0 out_seq=0 2024-09-10T08:23:24.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.638+0000 7f2c7ea00640 10 client.19743 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:23:24.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.638+0000 7f2c7ea00640 1 -- 172.21.15.2:0/4027211176 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55efad1aae60 con 0x55efad2c32c0 2024-09-10T08:23:24.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.638+0000 7f2c7ea00640 10 client.19743 handle_command_reply: tid=0 2024-09-10T08:23:24.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.638+0000 7f2ca29f6740 1 client.19743 shutdown 2024-09-10T08:23:24.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.638+0000 7f2ca29f6740 1 -- 172.21.15.2:0/4027211176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23d760 msgr2=0x55efad24b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:24.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.638+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/4027211176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23d760 0x55efad24b8c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f2c84031d50 tx=0x7f2c840055c0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.639+0000 7f2ca29f6740 1 -- 172.21.15.2:0/4027211176 shutdown_connections 2024-09-10T08:23:24.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.639+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/4027211176 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x55efad2c32c0 0x55efad2c56e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.639+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/4027211176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55efad248020 0x55efad24be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.639+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/4027211176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55efad23d760 0x55efad24b8c0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.639+0000 7f2c7ea00640 0 client.19743 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:23:24.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.639+0000 7f2ca29f6740 1 --2- 172.21.15.2:0/4027211176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55efad23ce40 0x55efad250540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.639+0000 7f2ca29f6740 1 -- 172.21.15.2:0/4027211176 >> 172.21.15.2:0/4027211176 conn(0x55efad2393d0 msgr2=0x55efad237c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:24.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.639+0000 7f2ca29f6740 1 -- 172.21.15.2:0/4027211176 shutdown_connections 2024-09-10T08:23:24.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.639+0000 7f2ca29f6740 1 -- 172.21.15.2:0/4027211176 wait complete. 2024-09-10T08:23:24.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.639+0000 7f2ca29f6740 20 client.19743 trim_cache size 0 max 16384 2024-09-10T08:23:24.653 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:23:24.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.640+0000 7f2c9ea00640 1 -- 172.21.15.2:0/2600646731 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c800a2b80 msgr2=0x7f2c800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:24.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.640+0000 7f2c9ea00640 1 --2- 172.21.15.2:0/2600646731 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c800a2b80 0x7f2c800a5040 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0cc9d0 tx=0x7f2c8c097040 comp rx=0 tx=0).stop 2024-09-10T08:23:24.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.640+0000 7f2c9ea00640 1 -- 172.21.15.2:0/2600646731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c98103e30 msgr2=0x7f2c9819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:24.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.640+0000 7f2c9ea00640 1 --2- 172.21.15.2:0/2600646731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c98103e30 0x7f2c9819a490 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00d6d0 tx=0x7f2c8c00db90 comp rx=0 tx=0).stop 2024-09-10T08:23:24.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.640+0000 7f2c9ea00640 1 -- 172.21.15.2:0/2600646731 shutdown_connections 2024-09-10T08:23:24.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.640+0000 7f2c9ea00640 1 --2- 172.21.15.2:0/2600646731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c98108f60 0x7f2c9819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.640+0000 7f2c9ea00640 1 --2- 172.21.15.2:0/2600646731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c981047e0 0x7f2c9819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.640+0000 7f2c9ea00640 1 --2- 172.21.15.2:0/2600646731 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c800a2b80 0x7f2c800a5040 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.640+0000 7f2c9ea00640 1 --2- 172.21.15.2:0/2600646731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c98103e30 0x7f2c9819a490 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:24.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.640+0000 7f2c9ea00640 1 -- 172.21.15.2:0/2600646731 >> 172.21.15.2:0/2600646731 conn(0x7f2c980ff8d0 msgr2=0x7f2c98100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:24.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.640+0000 7f2c9ea00640 1 -- 172.21.15.2:0/2600646731 shutdown_connections 2024-09-10T08:23:24.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:24.640+0000 7f2c9ea00640 1 -- 172.21.15.2:0/2600646731 wait complete. 2024-09-10T08:23:25.581 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:23:25.709 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:25 smithi155 ceph-mon[35405]: pgmap v845: 129 pgs: 129 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:23:25.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:25 smithi002 ceph-mon[30563]: pgmap v845: 129 pgs: 129 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:23:25.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:25 smithi107 ceph-mon[35048]: pgmap v845: 129 pgs: 129 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:23:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.966+0000 7fa981600640 1 -- 172.21.15.2:0/3649362769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa97c104cf0 msgr2=0x7fa97c1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.966+0000 7fa981600640 1 --2- 172.21.15.2:0/3649362769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa97c104cf0 0x7fa97c1070e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa968009900 tx=0x7fa96802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.966+0000 7fa981600640 1 -- 172.21.15.2:0/3649362769 shutdown_connections 2024-09-10T08:23:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.966+0000 7fa981600640 1 --2- 172.21.15.2:0/3649362769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c107620 0x7fa97c109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.966+0000 7fa981600640 1 --2- 172.21.15.2:0/3649362769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa97c104cf0 0x7fa97c1070e0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.966+0000 7fa981600640 1 --2- 172.21.15.2:0/3649362769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa97c1023c0 0x7fa97c1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.966+0000 7fa981600640 1 -- 172.21.15.2:0/3649362769 >> 172.21.15.2:0/3649362769 conn(0x7fa97c0fbfa0 msgr2=0x7fa97c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.966+0000 7fa981600640 1 -- 172.21.15.2:0/3649362769 shutdown_connections 2024-09-10T08:23:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.966+0000 7fa981600640 1 -- 172.21.15.2:0/3649362769 wait complete. 2024-09-10T08:23:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.967+0000 7fa981600640 1 Processor -- start 2024-09-10T08:23:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.967+0000 7fa981600640 1 -- start start 2024-09-10T08:23:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.967+0000 7fa981600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c1023c0 0x7fa97c199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.967+0000 7fa981600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa97c104cf0 0x7fa97c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.967+0000 7fa981600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa97c107620 0x7fa97c1a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.967+0000 7fa981600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c10f840 con 0x7fa97c107620 2024-09-10T08:23:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.967+0000 7fa981600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c10f6c0 con 0x7fa97c1023c0 2024-09-10T08:23:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.967+0000 7fa981600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c10f9c0 con 0x7fa97c104cf0 2024-09-10T08:23:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.968+0000 7fa980c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa97c107620 0x7fa97c1a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.968+0000 7fa980c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa97c107620 0x7fa97c1a1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33000/0 (socket says 172.21.15.2:33000) 2024-09-10T08:23:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.968+0000 7fa980c00640 1 -- 172.21.15.2:0/3312974100 learned_addr learned my addr 172.21.15.2:0/3312974100 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.968+0000 7fa97be00640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c1023c0 0x7fa97c199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.968+0000 7fa97b400640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa97c104cf0 0x7fa97c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.968+0000 7fa980c00640 1 -- 172.21.15.2:0/3312974100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa97c104cf0 msgr2=0x7fa97c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.968+0000 7fa980c00640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa97c104cf0 0x7fa97c19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.968+0000 7fa980c00640 1 -- 172.21.15.2:0/3312974100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c1023c0 msgr2=0x7fa97c199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.968+0000 7fa980c00640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c1023c0 0x7fa97c199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.968+0000 7fa980c00640 1 -- 172.21.15.2:0/3312974100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c1a1ad0 con 0x7fa97c107620 2024-09-10T08:23:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.968+0000 7fa97be00640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c1023c0 0x7fa97c199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.968+0000 7fa980c00640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa97c107620 0x7fa97c1a1460 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa97400eb40 tx=0x7fa97400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.969+0000 7fa978c00640 1 -- 172.21.15.2:0/3312974100 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974004b20 con 0x7fa97c107620 2024-09-10T08:23:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.969+0000 7fa978c00640 1 -- 172.21.15.2:0/3312974100 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa974004cc0 con 0x7fa97c107620 2024-09-10T08:23:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.969+0000 7fa981600640 1 -- 172.21.15.2:0/3312974100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97c1a1dc0 con 0x7fa97c107620 2024-09-10T08:23:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.969+0000 7fa978c00640 1 -- 172.21.15.2:0/3312974100 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974010850 con 0x7fa97c107620 2024-09-10T08:23:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.969+0000 7fa981600640 1 -- 172.21.15.2:0/3312974100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97c1a2280 con 0x7fa97c107620 2024-09-10T08:23:25.971 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:23:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.972+0000 7fa985bd6740 1 Processor -- start 2024-09-10T08:23:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.974+0000 7fa978c00640 1 -- 172.21.15.2:0/3312974100 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa974005020 con 0x7fa97c107620 2024-09-10T08:23:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.974+0000 7fa985bd6740 1 -- start start 2024-09-10T08:23:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.974+0000 7fa985bd6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8b260 0x563eb0d8b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.974+0000 7fa985bd6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d8bba0 0x563eb0d90660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.974+0000 7fa985bd6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d90c30 0x563eb0d930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.974+0000 7fa985bd6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563eb0cf0100 con 0x563eb0d8b260 2024-09-10T08:23:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.974+0000 7fa985bd6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563eb0c981b0 con 0x563eb0d90c30 2024-09-10T08:23:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.974+0000 7fa985bd6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563eb0cec0c0 con 0x563eb0d8bba0 2024-09-10T08:23:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.974+0000 7fa97be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d8bba0 0x563eb0d90660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.974+0000 7fa97be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d8bba0 0x563eb0d90660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:34974/0 (socket says 172.21.15.2:34974) 2024-09-10T08:23:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.974+0000 7fa97be00640 1 -- 172.21.15.2:0/1539847854 learned_addr learned my addr 172.21.15.2:0/1539847854 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.975+0000 7fa97b400640 1 --2- 172.21.15.2:0/1539847854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8b260 0x563eb0d8b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.975+0000 7fa97b400640 1 --2- 172.21.15.2:0/1539847854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d90c30 0x563eb0d930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.975+0000 7fa97be00640 1 -- 172.21.15.2:0/1539847854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d90c30 msgr2=0x563eb0d930f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.975+0000 7fa97be00640 1 --2- 172.21.15.2:0/1539847854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d90c30 0x563eb0d930f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.975+0000 7fa97be00640 1 -- 172.21.15.2:0/1539847854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8b260 msgr2=0x563eb0d8b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.975+0000 7fa97be00640 1 --2- 172.21.15.2:0/1539847854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8b260 0x563eb0d8b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.975+0000 7fa97be00640 1 -- 172.21.15.2:0/1539847854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eb0a03210 con 0x563eb0d8bba0 2024-09-10T08:23:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.975+0000 7fa97b400640 1 --2- 172.21.15.2:0/1539847854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d90c30 0x563eb0d930f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:23:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.975+0000 7fa978c00640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa9600a2c00 0x7fa9600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.975+0000 7fa97be00640 1 --2- 172.21.15.2:0/1539847854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d8bba0 0x563eb0d90660 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0115f0 tx=0x7fa96c011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51f09975ce1518d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.975+0000 7fa978c00640 1 -- 172.21.15.2:0/3312974100 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa9740cfc10 con 0x7fa97c107620 2024-09-10T08:23:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.975+0000 7fa97b400640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa9600a2c00 0x7fa9600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.976+0000 7fa971200640 1 -- 172.21.15.2:0/1539847854 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c01e070 con 0x563eb0d8bba0 2024-09-10T08:23:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.976+0000 7fa971200640 1 -- 172.21.15.2:0/1539847854 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96c00fb20 con 0x563eb0d8bba0 2024-09-10T08:23:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.976+0000 7fa971200640 1 -- 172.21.15.2:0/1539847854 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c019bf0 con 0x563eb0d8bba0 2024-09-10T08:23:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.976+0000 7fa985bd6740 1 -- 172.21.15.2:0/1539847854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d8bba0 msgr2=0x563eb0d90660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.976+0000 7fa97b400640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa9600a2c00 0x7fa9600a50c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa96802a9e0 tx=0x7fa968000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.976+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1539847854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d8bba0 0x563eb0d90660 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0115f0 tx=0x7fa96c011ab0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.977+0000 7fa985bd6740 1 -- 172.21.15.2:0/1539847854 shutdown_connections 2024-09-10T08:23:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.977+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1539847854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d90c30 0x563eb0d930f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.977+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1539847854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d8bba0 0x563eb0d90660 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.977+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1539847854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8b260 0x563eb0d8b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.977+0000 7fa985bd6740 1 -- 172.21.15.2:0/1539847854 >> 172.21.15.2:0/1539847854 conn(0x563eb0d868d0 msgr2=0x563eb0d86cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.977+0000 7fa985bd6740 1 -- 172.21.15.2:0/1539847854 shutdown_connections 2024-09-10T08:23:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.977+0000 7fa985bd6740 1 -- 172.21.15.2:0/1539847854 wait complete. 2024-09-10T08:23:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.977+0000 7fa985bd6740 1 -- start start 2024-09-10T08:23:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.977+0000 7fa985bd6740 1 Processor -- start 2024-09-10T08:23:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.978+0000 7fa985bd6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d95410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.978+0000 7fa985bd6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d8bba0 0x563eb0d95950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.978+0000 7fa985bd6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d90c30 0x563eb0d9c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.978+0000 7fa985bd6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563eb0d93c00 con 0x563eb0d90c30 2024-09-10T08:23:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.978+0000 7fa985bd6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563eb0c79090 con 0x563eb0d8b260 2024-09-10T08:23:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.978+0000 7fa985bd6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563eb0d93d80 con 0x563eb0d8bba0 2024-09-10T08:23:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.978+0000 7fa985bd6740 10 client.0 _ll_register_callbacks cb 0x563eb0ccdf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.978+0000 7fa97be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d95410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.978+0000 7fa97b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d8bba0 0x563eb0d95950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.978+0000 7fa97be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d95410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:51500/0 (socket says 172.21.15.2:51500) 2024-09-10T08:23:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.978+0000 7fa97be00640 1 -- 172.21.15.2:0/1210672316 learned_addr learned my addr 172.21.15.2:0/1210672316 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.978+0000 7fa980c00640 1 --2- 172.21.15.2:0/1210672316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d90c30 0x563eb0d9c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.979+0000 7fa97be00640 1 -- 172.21.15.2:0/1210672316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d8bba0 msgr2=0x563eb0d95950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.979+0000 7fa97be00640 1 --2- 172.21.15.2:0/1210672316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d8bba0 0x563eb0d95950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.979+0000 7fa97be00640 1 -- 172.21.15.2:0/1210672316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d90c30 msgr2=0x563eb0d9c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.979+0000 7fa97be00640 1 --2- 172.21.15.2:0/1210672316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d90c30 0x563eb0d9c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.979+0000 7fa97be00640 1 -- 172.21.15.2:0/1210672316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eb0acf620 con 0x563eb0d8b260 2024-09-10T08:23:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.979+0000 7fa97be00640 1 --2- 172.21.15.2:0/1210672316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d95410 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa96c014040 tx=0x7fa96c00f8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb1691a4a11055cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.979+0000 7fa965600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:23:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.979+0000 7fa965600640 1 -- 172.21.15.2:0/1210672316 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c01e040 con 0x563eb0d8b260 2024-09-10T08:23:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.979+0000 7fa965600640 1 -- 172.21.15.2:0/1210672316 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa96c004040 con 0x563eb0d8b260 2024-09-10T08:23:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.979+0000 7fa985bd6740 1 -- 172.21.15.2:0/1210672316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa96c00fb20 con 0x563eb0d8b260 2024-09-10T08:23:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.980+0000 7fa965600640 1 -- 172.21.15.2:0/1210672316 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa96c02f050 con 0x563eb0d8b260 2024-09-10T08:23:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.980+0000 7fa985bd6740 10 client.29777 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.980+0000 7fa985bd6740 1 -- 172.21.15.2:0/1210672316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563eb0d9f780 con 0x563eb0d8b260 2024-09-10T08:23:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.980+0000 7fa965600640 1 -- 172.21.15.2:0/1210672316 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96c00fb20 con 0x563eb0d8b260 2024-09-10T08:23:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.980+0000 7fa965600640 1 -- 172.21.15.2:0/1210672316 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa96c02a030 con 0x563eb0d8b260 2024-09-10T08:23:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.980+0000 7fa985bd6740 10 client.29777 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.980+0000 7fa985bd6740 10 client.29777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.980+0000 7fa985bd6740 20 client.29777 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.980+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1210672316 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563eb0da0cd0 0x563eb0da30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.980+0000 7fa985bd6740 4 client.29777 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:23:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.980+0000 7fa985bd6740 1 -- 172.21.15.2:0/1210672316 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eb0cfd830 con 0x563eb0da0cd0 2024-09-10T08:23:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.981+0000 7fa97b400640 1 --2- 172.21.15.2:0/1210672316 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563eb0da0cd0 0x563eb0da30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.981+0000 7fa97b400640 1 --2- 172.21.15.2:0/1210672316 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563eb0da0cd0 0x563eb0da30e0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff99f899acbc82d6 server_cookie=4de3d1ec98480a5b in_seq=0 out_seq=0 2024-09-10T08:23:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.981+0000 7fa965600640 10 client.29777 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa965600640 1 -- 172.21.15.2:0/1210672316 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563eb0cfd830 con 0x563eb0da0cd0 2024-09-10T08:23:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa965600640 10 client.29777 handle_command_reply: tid=0 2024-09-10T08:23:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa985bd6740 1 client.29777 shutdown 2024-09-10T08:23:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa985bd6740 1 -- 172.21.15.2:0/1210672316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 msgr2=0x563eb0d95410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1210672316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d95410 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa96c014040 tx=0x7fa96c00f8f0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa985bd6740 1 -- 172.21.15.2:0/1210672316 shutdown_connections 2024-09-10T08:23:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1210672316 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563eb0da0cd0 0x563eb0da30e0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1210672316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d90c30 0x563eb0d9c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1210672316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d8bba0 0x563eb0d95950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1210672316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d95410 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa985bd6740 1 -- 172.21.15.2:0/1210672316 >> 172.21.15.2:0/1210672316 conn(0x563eb0d868d0 msgr2=0x563eb0d880e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa965600640 0 client.29777 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa985bd6740 1 -- 172.21.15.2:0/1210672316 shutdown_connections 2024-09-10T08:23:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa985bd6740 1 -- 172.21.15.2:0/1210672316 wait complete. 2024-09-10T08:23:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.982+0000 7fa985bd6740 20 client.29777 trim_cache size 0 max 16384 2024-09-10T08:23:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.998+0000 7fa985bd6740 1 Processor -- start 2024-09-10T08:23:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.998+0000 7fa985bd6740 1 -- start start 2024-09-10T08:23:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.998+0000 7fa985bd6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d958d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.998+0000 7fa985bd6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 0x563eb0d9c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.998+0000 7fa985bd6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d90c30 0x563eb0d86cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.998+0000 7fa985bd6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c1af200 con 0x563eb0d8bba0 2024-09-10T08:23:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.998+0000 7fa985bd6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c1acad0 con 0x563eb0d8b260 2024-09-10T08:23:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.998+0000 7fa985bd6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c1aa920 con 0x563eb0d90c30 2024-09-10T08:23:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.998+0000 7fa97b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 0x563eb0d9c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.998+0000 7fa97be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.998+0000 7fa97b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 0x563eb0d9c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33030/0 (socket says 172.21.15.2:33030) 2024-09-10T08:23:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.998+0000 7fa97b400640 1 -- 172.21.15.2:0/1391959737 learned_addr learned my addr 172.21.15.2:0/1391959737 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.998+0000 7fa980c00640 1 --2- 172.21.15.2:0/1391959737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d90c30 0x563eb0d86cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.999+0000 7fa97be00640 1 -- 172.21.15.2:0/1391959737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d90c30 msgr2=0x563eb0d86cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.999+0000 7fa97be00640 1 --2- 172.21.15.2:0/1391959737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d90c30 0x563eb0d86cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.999+0000 7fa97be00640 1 -- 172.21.15.2:0/1391959737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 msgr2=0x563eb0d9c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.999+0000 7fa97be00640 1 --2- 172.21.15.2:0/1391959737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 0x563eb0d9c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.999+0000 7fa97be00640 1 -- 172.21.15.2:0/1391959737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eb0d866f0 con 0x563eb0d8b260 2024-09-10T08:23:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.999+0000 7fa97b400640 1 --2- 172.21.15.2:0/1391959737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 0x563eb0d9c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.999+0000 7fa97be00640 1 --2- 172.21.15.2:0/1391959737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d958d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa96c011fb0 tx=0x7fa96c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3098aab9ab7efb44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.999+0000 7fa967e00640 1 -- 172.21.15.2:0/1391959737 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c038070 con 0x563eb0d8b260 2024-09-10T08:23:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.999+0000 7fa967e00640 1 -- 172.21.15.2:0/1391959737 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96c004520 con 0x563eb0d8b260 2024-09-10T08:23:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:25.999+0000 7fa967e00640 1 -- 172.21.15.2:0/1391959737 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c01e070 con 0x563eb0d8b260 2024-09-10T08:23:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.000+0000 7fa985bd6740 1 -- 172.21.15.2:0/1391959737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 msgr2=0x563eb0d958d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.000+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1391959737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d958d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa96c011fb0 tx=0x7fa96c009f60 comp rx=0 tx=0).stop 2024-09-10T08:23:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.000+0000 7fa985bd6740 1 -- 172.21.15.2:0/1391959737 shutdown_connections 2024-09-10T08:23:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.000+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1391959737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d90c30 0x563eb0d86cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.000+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1391959737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 0x563eb0d9c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.000+0000 7fa985bd6740 1 --2- 172.21.15.2:0/1391959737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d958d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.000+0000 7fa985bd6740 1 -- 172.21.15.2:0/1391959737 >> 172.21.15.2:0/1391959737 conn(0x563eb0d868d0 msgr2=0x563eb0d951c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.001+0000 7fa985bd6740 1 -- 172.21.15.2:0/1391959737 shutdown_connections 2024-09-10T08:23:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.001+0000 7fa985bd6740 1 -- 172.21.15.2:0/1391959737 wait complete. 2024-09-10T08:23:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.001+0000 7fa985bd6740 1 -- start start 2024-09-10T08:23:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.001+0000 7fa985bd6740 1 Processor -- start 2024-09-10T08:23:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.001+0000 7fa985bd6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.001+0000 7fa985bd6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 0x563eb0d99c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa985bd6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d90c30 0x563eb0e099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa985bd6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c1aaf90 con 0x563eb0d8bba0 2024-09-10T08:23:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa985bd6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c1ac400 con 0x563eb0d8b260 2024-09-10T08:23:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa985bd6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c1ab660 con 0x563eb0d90c30 2024-09-10T08:23:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa985bd6740 10 client.0 _ll_register_callbacks cb 0x7fa96c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa97b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 0x563eb0d99c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa97be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa97b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 0x563eb0d99c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33046/0 (socket says 172.21.15.2:33046) 2024-09-10T08:23:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa97b400640 1 -- 172.21.15.2:0/736130039 learned_addr learned my addr 172.21.15.2:0/736130039 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa980c00640 1 --2- 172.21.15.2:0/736130039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d90c30 0x563eb0e099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa97be00640 1 -- 172.21.15.2:0/736130039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d90c30 msgr2=0x563eb0e099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa97be00640 1 --2- 172.21.15.2:0/736130039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d90c30 0x563eb0e099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa97be00640 1 -- 172.21.15.2:0/736130039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 msgr2=0x563eb0d99c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa97be00640 1 --2- 172.21.15.2:0/736130039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 0x563eb0d99c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa97be00640 1 -- 172.21.15.2:0/736130039 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eb0d9cc90 con 0x563eb0d8b260 2024-09-10T08:23:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.002+0000 7fa97b400640 1 --2- 172.21.15.2:0/736130039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 0x563eb0d99c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.003+0000 7fa97be00640 1 --2- 172.21.15.2:0/736130039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d996d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa96c006010 tx=0x7fa96c0046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d64bfb26537eb8ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.003+0000 7fa965600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:23:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.003+0000 7fa965600640 1 -- 172.21.15.2:0/736130039 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c038040 con 0x563eb0d8b260 2024-09-10T08:23:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.003+0000 7fa965600640 1 -- 172.21.15.2:0/736130039 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa96c004980 con 0x563eb0d8b260 2024-09-10T08:23:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.003+0000 7fa985bd6740 1 -- 172.21.15.2:0/736130039 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa96c004520 con 0x563eb0d8b260 2024-09-10T08:23:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.003+0000 7fa965600640 1 -- 172.21.15.2:0/736130039 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa96c049050 con 0x563eb0d8b260 2024-09-10T08:23:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.003+0000 7fa985bd6740 10 client.29783 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.003+0000 7fa985bd6740 1 -- 172.21.15.2:0/736130039 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563eb0e0af60 con 0x563eb0d8b260 2024-09-10T08:23:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.003+0000 7fa965600640 1 -- 172.21.15.2:0/736130039 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96c004520 con 0x563eb0d8b260 2024-09-10T08:23:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.004+0000 7fa965600640 1 -- 172.21.15.2:0/736130039 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa96c044030 con 0x563eb0d8b260 2024-09-10T08:23:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.004+0000 7fa985bd6740 10 client.29783 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.004+0000 7fa985bd6740 10 client.29783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.004+0000 7fa985bd6740 20 client.29783 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.004+0000 7fa985bd6740 1 --2- 172.21.15.2:0/736130039 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563eb0e0b850 0x563eb0e0dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.004+0000 7fa985bd6740 4 client.29783 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:23:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.004+0000 7fa985bd6740 1 -- 172.21.15.2:0/736130039 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563eb0cf7b80 con 0x563eb0e0b850 2024-09-10T08:23:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.004+0000 7fa97b400640 1 --2- 172.21.15.2:0/736130039 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563eb0e0b850 0x563eb0e0dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.004+0000 7fa965600640 10 client.29783 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.004+0000 7fa97b400640 1 --2- 172.21.15.2:0/736130039 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563eb0e0b850 0x563eb0e0dc70 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebf8e9f215ae3a23 server_cookie=9381537d511dbda0 in_seq=0 out_seq=0 2024-09-10T08:23:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.019+0000 7fa965600640 1 -- 172.21.15.2:0/736130039 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563eb0cf7b80 con 0x563eb0e0b850 2024-09-10T08:23:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.019+0000 7fa965600640 10 client.29783 handle_command_reply: tid=0 2024-09-10T08:23:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.019+0000 7fa985bd6740 1 client.29783 shutdown 2024-09-10T08:23:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.019+0000 7fa985bd6740 1 -- 172.21.15.2:0/736130039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 msgr2=0x563eb0d996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.019+0000 7fa985bd6740 1 --2- 172.21.15.2:0/736130039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d996d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa96c006010 tx=0x7fa96c0046d0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.019+0000 7fa985bd6740 1 -- 172.21.15.2:0/736130039 shutdown_connections 2024-09-10T08:23:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.019+0000 7fa985bd6740 1 --2- 172.21.15.2:0/736130039 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563eb0e0b850 0x563eb0e0dc70 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.019+0000 7fa985bd6740 1 --2- 172.21.15.2:0/736130039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563eb0d90c30 0x563eb0e099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.019+0000 7fa985bd6740 1 --2- 172.21.15.2:0/736130039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563eb0d8bba0 0x563eb0d99c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.019+0000 7fa965600640 0 client.29783 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.019+0000 7fa985bd6740 1 --2- 172.21.15.2:0/736130039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563eb0d8b260 0x563eb0d996d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.019+0000 7fa985bd6740 1 -- 172.21.15.2:0/736130039 >> 172.21.15.2:0/736130039 conn(0x563eb0d868d0 msgr2=0x563eb0d851f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.020+0000 7fa985bd6740 1 -- 172.21.15.2:0/736130039 shutdown_connections 2024-09-10T08:23:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.020+0000 7fa985bd6740 1 -- 172.21.15.2:0/736130039 wait complete. 2024-09-10T08:23:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.020+0000 7fa985bd6740 20 client.29783 trim_cache size 0 max 16384 2024-09-10T08:23:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:23:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:23:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:23:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:23:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.020+0000 7fa981600640 1 -- 172.21.15.2:0/3312974100 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa9600a2c00 msgr2=0x7fa9600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.020+0000 7fa981600640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa9600a2c00 0x7fa9600a50c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa96802a9e0 tx=0x7fa968000bc0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.020+0000 7fa981600640 1 -- 172.21.15.2:0/3312974100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa97c107620 msgr2=0x7fa97c1a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.020+0000 7fa981600640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa97c107620 0x7fa97c1a1460 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa97400eb40 tx=0x7fa97400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.021+0000 7fa981600640 1 -- 172.21.15.2:0/3312974100 shutdown_connections 2024-09-10T08:23:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.021+0000 7fa981600640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa97c107620 0x7fa97c1a1460 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.021+0000 7fa981600640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa9600a2c00 0x7fa9600a50c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.021+0000 7fa981600640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa97c104cf0 0x7fa97c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.021+0000 7fa981600640 1 --2- 172.21.15.2:0/3312974100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c1023c0 0x7fa97c199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.021+0000 7fa981600640 1 -- 172.21.15.2:0/3312974100 >> 172.21.15.2:0/3312974100 conn(0x7fa97c0fbfa0 msgr2=0x7fa97c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.021+0000 7fa981600640 1 -- 172.21.15.2:0/3312974100 shutdown_connections 2024-09-10T08:23:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:26.021+0000 7fa981600640 1 -- 172.21.15.2:0/3312974100 wait complete. 2024-09-10T08:23:26.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.670+0000 7f1671000640 1 -- 172.21.15.2:0/2390465977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f166c1047e0 msgr2=0x7f166c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.670+0000 7f1671000640 1 --2- 172.21.15.2:0/2390465977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f166c1047e0 0x7f166c108830 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1654009860 tx=0x7f165402a340 comp rx=0 tx=0).stop 2024-09-10T08:23:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.670+0000 7f1671000640 1 -- 172.21.15.2:0/2390465977 shutdown_connections 2024-09-10T08:23:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.670+0000 7f1671000640 1 --2- 172.21.15.2:0/2390465977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f166c108f60 0x7f166c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.670+0000 7f1671000640 1 --2- 172.21.15.2:0/2390465977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f166c1047e0 0x7f166c108830 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.670+0000 7f1671000640 1 --2- 172.21.15.2:0/2390465977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f166c103e30 0x7f166c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.670+0000 7f1671000640 1 -- 172.21.15.2:0/2390465977 >> 172.21.15.2:0/2390465977 conn(0x7f166c0ff8d0 msgr2=0x7f166c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.670+0000 7f1671000640 1 -- 172.21.15.2:0/2390465977 shutdown_connections 2024-09-10T08:23:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.670+0000 7f1671000640 1 -- 172.21.15.2:0/2390465977 wait complete. 2024-09-10T08:23:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.671+0000 7f1671000640 1 Processor -- start 2024-09-10T08:23:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.671+0000 7f1671000640 1 -- start start 2024-09-10T08:23:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f1671000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f166c103e30 0x7f166c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f1671000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f166c1047e0 0x7f166c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f1671000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f166c108f60 0x7f166c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f1671000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f166c113fc0 con 0x7f166c103e30 2024-09-10T08:23:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f1671000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f166c113e40 con 0x7f166c1047e0 2024-09-10T08:23:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f1671000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f166c114140 con 0x7f166c108f60 2024-09-10T08:23:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f166b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f166c103e30 0x7f166c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f166b400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f166c103e30 0x7f166c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33072/0 (socket says 172.21.15.2:33072) 2024-09-10T08:23:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f166b400640 1 -- 172.21.15.2:0/507105170 learned_addr learned my addr 172.21.15.2:0/507105170 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f1663e00640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f166c1047e0 0x7f166c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f166be00640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f166c108f60 0x7f166c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f166b400640 1 -- 172.21.15.2:0/507105170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f166c108f60 msgr2=0x7f166c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f166b400640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f166c108f60 0x7f166c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f166b400640 1 -- 172.21.15.2:0/507105170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f166c1047e0 msgr2=0x7f166c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f166b400640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f166c1047e0 0x7f166c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f166b400640 1 -- 172.21.15.2:0/507105170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c19f440 con 0x7f166c103e30 2024-09-10T08:23:26.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.672+0000 7f1663e00640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f166c1047e0 0x7f166c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:26.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.673+0000 7f166b400640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f166c103e30 0x7f166c19a490 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f165c00e9c0 tx=0x7f165c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:26.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.673+0000 7f1668c00640 1 -- 172.21.15.2:0/507105170 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c00cde0 con 0x7f166c103e30 2024-09-10T08:23:26.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.673+0000 7f1671000640 1 -- 172.21.15.2:0/507105170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c19f730 con 0x7f166c103e30 2024-09-10T08:23:26.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.673+0000 7f1668c00640 1 -- 172.21.15.2:0/507105170 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f165c0035f0 con 0x7f166c103e30 2024-09-10T08:23:26.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.673+0000 7f1668c00640 1 -- 172.21.15.2:0/507105170 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c018630 con 0x7f166c103e30 2024-09-10T08:23:26.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.673+0000 7f1671000640 1 -- 172.21.15.2:0/507105170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c1a7010 con 0x7f166c103e30 2024-09-10T08:23:26.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.676+0000 7f1675205740 1 Processor -- start 2024-09-10T08:23:26.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.677+0000 7f1668c00640 1 -- 172.21.15.2:0/507105170 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f165c0037a0 con 0x7f166c103e30 2024-09-10T08:23:26.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.677+0000 7f1675205740 1 -- start start 2024-09-10T08:23:26.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.677+0000 7f1675205740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bae40 0x5597276bb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.677+0000 7f1675205740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bb760 0x5597276c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.677+0000 7f1675205740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276c6020 0x5597276c8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.677+0000 7f1675205740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5597276213e0 con 0x5597276c6020 2024-09-10T08:23:26.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.677+0000 7f1675205740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5597275c8d80 con 0x5597276bb760 2024-09-10T08:23:26.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.677+0000 7f1675205740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55972761d3a0 con 0x5597276bae40 2024-09-10T08:23:26.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.677+0000 7f166be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bae40 0x5597276bb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.677+0000 7f1663e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276c6020 0x5597276c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.677+0000 7f166be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bae40 0x5597276bb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35042/0 (socket says 172.21.15.2:35042) 2024-09-10T08:23:26.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.677+0000 7f166be00640 1 -- 172.21.15.2:0/4145301607 learned_addr learned my addr 172.21.15.2:0/4145301607 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:26.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.677+0000 7f166b400640 1 --2- 172.21.15.2:0/4145301607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bb760 0x5597276c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.678+0000 7f166be00640 1 -- 172.21.15.2:0/4145301607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bb760 msgr2=0x5597276c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:26.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.678+0000 7f166be00640 1 --2- 172.21.15.2:0/4145301607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bb760 0x5597276c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.678+0000 7f166be00640 1 -- 172.21.15.2:0/4145301607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276c6020 msgr2=0x5597276c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:26.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.678+0000 7f166be00640 1 --2- 172.21.15.2:0/4145301607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276c6020 0x5597276c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.678+0000 7f166be00640 1 -- 172.21.15.2:0/4145301607 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597275e7ae0 con 0x5597276bae40 2024-09-10T08:23:26.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.678+0000 7f166be00640 1 --2- 172.21.15.2:0/4145301607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bae40 0x5597276bb220 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1664009f50 tx=0x7f16640118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74a0fa151682c8f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:26.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.678+0000 7f1660c00640 1 -- 172.21.15.2:0/4145301607 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664004d20 con 0x5597276bae40 2024-09-10T08:23:26.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.678+0000 7f1668c00640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16500a2b80 0x7f16500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.679+0000 7f1660c00640 1 -- 172.21.15.2:0/4145301607 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1664004050 con 0x5597276bae40 2024-09-10T08:23:26.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.679+0000 7f1660c00640 1 -- 172.21.15.2:0/4145301607 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664005280 con 0x5597276bae40 2024-09-10T08:23:26.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.679+0000 7f1668c00640 1 -- 172.21.15.2:0/507105170 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f165c0cf6b0 con 0x7f166c103e30 2024-09-10T08:23:26.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.679+0000 7f166b400640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16500a2b80 0x7f16500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.679+0000 7f1675205740 1 -- 172.21.15.2:0/4145301607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bae40 msgr2=0x5597276bb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:26.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.679+0000 7f1675205740 1 --2- 172.21.15.2:0/4145301607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bae40 0x5597276bb220 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1664009f50 tx=0x7f16640118d0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.679+0000 7f1675205740 1 -- 172.21.15.2:0/4145301607 shutdown_connections 2024-09-10T08:23:26.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.679+0000 7f1675205740 1 --2- 172.21.15.2:0/4145301607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276c6020 0x5597276c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.679+0000 7f1675205740 1 --2- 172.21.15.2:0/4145301607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bb760 0x5597276c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.679+0000 7f1675205740 1 --2- 172.21.15.2:0/4145301607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bae40 0x5597276bb220 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.679+0000 7f1675205740 1 -- 172.21.15.2:0/4145301607 >> 172.21.15.2:0/4145301607 conn(0x5597276b73d0 msgr2=0x5597276b77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:26.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.679+0000 7f166b400640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16500a2b80 0x7f16500a5040 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f165c0cd130 tx=0x7f165c0d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:26.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.680+0000 7f1675205740 1 -- 172.21.15.2:0/4145301607 shutdown_connections 2024-09-10T08:23:26.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.680+0000 7f1675205740 1 -- 172.21.15.2:0/4145301607 wait complete. 2024-09-10T08:23:26.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.680+0000 7f1675205740 1 -- start start 2024-09-10T08:23:26.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.680+0000 7f1675205740 1 Processor -- start 2024-09-10T08:23:26.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.681+0000 7f1675205740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bae40 0x5597276c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.681+0000 7f1675205740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bb760 0x5597276c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.681+0000 7f1675205740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276c6020 0x5597276c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.681+0000 7f1675205740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5597276c8bf0 con 0x5597276c6020 2024-09-10T08:23:26.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.681+0000 7f1675205740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5597275dc0f0 con 0x5597276bae40 2024-09-10T08:23:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.681+0000 7f1675205740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5597276c8d70 con 0x5597276bb760 2024-09-10T08:23:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.681+0000 7f1675205740 10 client.0 _ll_register_callbacks cb 0x5597275c3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.681+0000 7f166be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276c6020 0x5597276c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.681+0000 7f1663e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bb760 0x5597276c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.681+0000 7f1663e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bb760 0x5597276c34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:35054/0 (socket says 172.21.15.2:35054) 2024-09-10T08:23:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.681+0000 7f1663e00640 1 -- 172.21.15.2:0/565086385 learned_addr learned my addr 172.21.15.2:0/565086385 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.681+0000 7f166be00640 1 --2- 172.21.15.2:0/565086385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bae40 0x5597276c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.682+0000 7f1663e00640 1 -- 172.21.15.2:0/565086385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bae40 msgr2=0x5597276c2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.682+0000 7f1663e00640 1 --2- 172.21.15.2:0/565086385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bae40 0x5597276c2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.682+0000 7f1663e00640 1 -- 172.21.15.2:0/565086385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276c6020 msgr2=0x5597276c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.682+0000 7f1663e00640 1 --2- 172.21.15.2:0/565086385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276c6020 0x5597276c39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.682+0000 7f1663e00640 1 -- 172.21.15.2:0/565086385 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597273923c0 con 0x5597276bb760 2024-09-10T08:23:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.682+0000 7f1663e00640 1 --2- 172.21.15.2:0/565086385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bb760 0x5597276c34a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1654009780 tx=0x7f1654004500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=863541df8c1cbe50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:26.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.682+0000 7f1659600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:23:26.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.682+0000 7f1659600640 1 -- 172.21.15.2:0/565086385 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654041070 con 0x5597276bb760 2024-09-10T08:23:26.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.682+0000 7f1675205740 1 -- 172.21.15.2:0/565086385 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1664004050 con 0x5597276bb760 2024-09-10T08:23:26.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.682+0000 7f1659600640 1 -- 172.21.15.2:0/565086385 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f1654030960 con 0x5597276bb760 2024-09-10T08:23:26.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.682+0000 7f1659600640 1 -- 172.21.15.2:0/565086385 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f165404d050 con 0x5597276bb760 2024-09-10T08:23:26.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.682+0000 7f1675205740 10 client.29722 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:26.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.682+0000 7f1675205740 1 -- 172.21.15.2:0/565086385 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5597276d4cd0 con 0x5597276bb760 2024-09-10T08:23:26.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.683+0000 7f1659600640 1 -- 172.21.15.2:0/565086385 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1664004050 con 0x5597276bb760 2024-09-10T08:23:26.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.683+0000 7f1659600640 1 -- 172.21.15.2:0/565086385 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f165403d7b0 con 0x5597276bb760 2024-09-10T08:23:26.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.683+0000 7f1675205740 10 client.29722 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:26.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.683+0000 7f1675205740 10 client.29722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:26.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.683+0000 7f1675205740 20 client.29722 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:26.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.683+0000 7f1675205740 1 --2- 172.21.15.2:0/565086385 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5597276d60e0 0x5597276d8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.683+0000 7f1675205740 4 client.29722 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:23:26.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.683+0000 7f1675205740 1 -- 172.21.15.2:0/565086385 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55972762e950 con 0x5597276d60e0 2024-09-10T08:23:26.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.683+0000 7f166be00640 1 --2- 172.21.15.2:0/565086385 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5597276d60e0 0x5597276d8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.684+0000 7f166be00640 1 --2- 172.21.15.2:0/565086385 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5597276d60e0 0x5597276d8500 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c9401589f795e1b server_cookie=ad661d0a60638397 in_seq=0 out_seq=0 2024-09-10T08:23:26.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.684+0000 7f1659600640 10 client.29722 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:26.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1659600640 1 -- 172.21.15.2:0/565086385 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55972762e950 con 0x5597276d60e0 2024-09-10T08:23:26.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1659600640 10 client.29722 handle_command_reply: tid=0 2024-09-10T08:23:26.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1675205740 1 client.29722 shutdown 2024-09-10T08:23:26.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1675205740 1 -- 172.21.15.2:0/565086385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bb760 msgr2=0x5597276c34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:26.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1675205740 1 --2- 172.21.15.2:0/565086385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bb760 0x5597276c34a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1654009780 tx=0x7f1654004500 comp rx=0 tx=0).stop 2024-09-10T08:23:26.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1675205740 1 -- 172.21.15.2:0/565086385 shutdown_connections 2024-09-10T08:23:26.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1675205740 1 --2- 172.21.15.2:0/565086385 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5597276d60e0 0x5597276d8500 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1675205740 1 --2- 172.21.15.2:0/565086385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276c6020 0x5597276c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1675205740 1 --2- 172.21.15.2:0/565086385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bb760 0x5597276c34a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1659600640 0 client.29722 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:26.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1675205740 1 --2- 172.21.15.2:0/565086385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bae40 0x5597276c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1675205740 1 -- 172.21.15.2:0/565086385 >> 172.21.15.2:0/565086385 conn(0x5597276b73d0 msgr2=0x5597276bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:26.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1675205740 1 -- 172.21.15.2:0/565086385 shutdown_connections 2024-09-10T08:23:26.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1675205740 1 -- 172.21.15.2:0/565086385 wait complete. 2024-09-10T08:23:26.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.685+0000 7f1675205740 20 client.29722 trim_cache size 0 max 16384 2024-09-10T08:23:26.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.699+0000 7f1675205740 1 Processor -- start 2024-09-10T08:23:26.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.699+0000 7f1675205740 1 -- start start 2024-09-10T08:23:26.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.699+0000 7f1675205740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 0x5597276dddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.699+0000 7f1675205740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bb760 0x5597276de310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.699+0000 7f1675205740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276c6020 0x5597276c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.699+0000 7f1675205740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1b3740 con 0x5597276bae40 2024-09-10T08:23:26.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.699+0000 7f1675205740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1b1010 con 0x5597276c6020 2024-09-10T08:23:26.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.699+0000 7f1675205740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1aee60 con 0x5597276bb760 2024-09-10T08:23:26.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.700+0000 7f166be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 0x5597276dddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.700+0000 7f166be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 0x5597276dddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33116/0 (socket says 172.21.15.2:33116) 2024-09-10T08:23:26.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.700+0000 7f166be00640 1 -- 172.21.15.2:0/966149402 learned_addr learned my addr 172.21.15.2:0/966149402 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:26.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.700+0000 7f1663e00640 1 --2- 172.21.15.2:0/966149402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bb760 0x5597276de310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.700+0000 7f166be00640 1 --2- 172.21.15.2:0/966149402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276c6020 0x5597276c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.700+0000 7f166be00640 1 -- 172.21.15.2:0/966149402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bb760 msgr2=0x5597276de310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:26.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.700+0000 7f166be00640 1 --2- 172.21.15.2:0/966149402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bb760 0x5597276de310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.700+0000 7f166be00640 1 -- 172.21.15.2:0/966149402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276c6020 msgr2=0x5597276c01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:26.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.700+0000 7f166be00640 1 --2- 172.21.15.2:0/966149402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276c6020 0x5597276c01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.700+0000 7f166be00640 1 -- 172.21.15.2:0/966149402 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597276b8060 con 0x5597276bae40 2024-09-10T08:23:26.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.700+0000 7f1663e00640 1 --2- 172.21.15.2:0/966149402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bb760 0x5597276de310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:26.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.700+0000 7f166be00640 1 --2- 172.21.15.2:0/966149402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 0x5597276dddd0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f166401b040 tx=0x7f1664009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53383177cf9010a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:26.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.701+0000 7f165be00640 1 -- 172.21.15.2:0/966149402 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664030040 con 0x5597276bae40 2024-09-10T08:23:26.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.701+0000 7f165be00640 1 -- 172.21.15.2:0/966149402 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1664027440 con 0x5597276bae40 2024-09-10T08:23:26.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.701+0000 7f165be00640 1 -- 172.21.15.2:0/966149402 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166400f6e0 con 0x5597276bae40 2024-09-10T08:23:26.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.701+0000 7f1675205740 1 -- 172.21.15.2:0/966149402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 msgr2=0x5597276dddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:26.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.701+0000 7f1675205740 1 --2- 172.21.15.2:0/966149402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 0x5597276dddd0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f166401b040 tx=0x7f1664009500 comp rx=0 tx=0).stop 2024-09-10T08:23:26.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.701+0000 7f1675205740 1 -- 172.21.15.2:0/966149402 shutdown_connections 2024-09-10T08:23:26.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.701+0000 7f1675205740 1 --2- 172.21.15.2:0/966149402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276c6020 0x5597276c01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.701+0000 7f1675205740 1 --2- 172.21.15.2:0/966149402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276bb760 0x5597276de310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.701+0000 7f1675205740 1 --2- 172.21.15.2:0/966149402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 0x5597276dddd0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.701+0000 7f1675205740 1 -- 172.21.15.2:0/966149402 >> 172.21.15.2:0/966149402 conn(0x5597276b73d0 msgr2=0x5597276b64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:26.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.702+0000 7f1675205740 1 -- 172.21.15.2:0/966149402 shutdown_connections 2024-09-10T08:23:26.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.702+0000 7f1675205740 1 -- 172.21.15.2:0/966149402 wait complete. 2024-09-10T08:23:26.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.702+0000 7f1675205740 1 -- start start 2024-09-10T08:23:26.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.702+0000 7f1675205740 1 Processor -- start 2024-09-10T08:23:26.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f1675205740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 0x5597276c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f1675205740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bb760 0x5597276ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f1675205740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276c6020 0x5597276ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f1675205740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1af4d0 con 0x5597276bae40 2024-09-10T08:23:26.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f1675205740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1b0940 con 0x5597276bb760 2024-09-10T08:23:26.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f1675205740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1afba0 con 0x5597276c6020 2024-09-10T08:23:26.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f1675205740 10 client.0 _ll_register_callbacks cb 0x7f16640093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:26.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f166be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 0x5597276c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f166be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 0x5597276c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33118/0 (socket says 172.21.15.2:33118) 2024-09-10T08:23:26.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f166be00640 1 -- 172.21.15.2:0/1879241770 learned_addr learned my addr 172.21.15.2:0/1879241770 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:26.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f1663e00640 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bb760 0x5597276ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f166be00640 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276c6020 0x5597276ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f166be00640 1 -- 172.21.15.2:0/1879241770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276c6020 msgr2=0x5597276ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:26.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f166be00640 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276c6020 0x5597276ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f166be00640 1 -- 172.21.15.2:0/1879241770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bb760 msgr2=0x5597276ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:26.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f166be00640 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bb760 0x5597276ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f166be00640 1 -- 172.21.15.2:0/1879241770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55972773fdf0 con 0x5597276bae40 2024-09-10T08:23:26.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.703+0000 7f1663e00640 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bb760 0x5597276ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:26.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.704+0000 7f166be00640 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 0x5597276c9d60 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1664009270 tx=0x7f1664009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9daa7c07d15e8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:26.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.704+0000 7f1659600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:23:26.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.704+0000 7f1659600640 1 -- 172.21.15.2:0/1879241770 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664004250 con 0x5597276bae40 2024-09-10T08:23:26.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.704+0000 7f1675205740 1 -- 172.21.15.2:0/1879241770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1664027440 con 0x5597276bae40 2024-09-10T08:23:26.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.704+0000 7f1659600640 1 -- 172.21.15.2:0/1879241770 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f16640043f0 con 0x5597276bae40 2024-09-10T08:23:26.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.704+0000 7f1659600640 1 -- 172.21.15.2:0/1879241770 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16640045f0 con 0x5597276bae40 2024-09-10T08:23:26.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.704+0000 7f1675205740 10 client.19779 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:26.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.704+0000 7f1675205740 1 -- 172.21.15.2:0/1879241770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5597277410d0 con 0x5597276bae40 2024-09-10T08:23:26.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.704+0000 7f1659600640 1 -- 172.21.15.2:0/1879241770 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1664027440 con 0x5597276bae40 2024-09-10T08:23:26.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.704+0000 7f1659600640 1 -- 172.21.15.2:0/1879241770 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f166400f8f0 con 0x5597276bae40 2024-09-10T08:23:26.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.705+0000 7f1675205740 10 client.19779 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:26.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.705+0000 7f1675205740 10 client.19779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:26.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.705+0000 7f1675205740 20 client.19779 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:26.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.705+0000 7f1675205740 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5597277415b0 0x5597277439a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:26.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.705+0000 7f1675205740 4 client.19779 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:23:26.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.705+0000 7f1675205740 1 -- 172.21.15.2:0/1879241770 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559727628e60 con 0x5597277415b0 2024-09-10T08:23:26.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.705+0000 7f1663e00640 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5597277415b0 0x5597277439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:26.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1663e00640 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5597277415b0 0x5597277439a0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb7fb2006b3add7 server_cookie=cf77473481edd7f9 in_seq=0 out_seq=0 2024-09-10T08:23:26.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1659600640 10 client.19779 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:26.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1659600640 1 -- 172.21.15.2:0/1879241770 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559727628e60 con 0x5597277415b0 2024-09-10T08:23:26.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1659600640 10 client.19779 handle_command_reply: tid=0 2024-09-10T08:23:26.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1675205740 1 client.19779 shutdown 2024-09-10T08:23:26.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1675205740 1 -- 172.21.15.2:0/1879241770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 msgr2=0x5597276c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:26.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1675205740 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 0x5597276c9d60 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1664009270 tx=0x7f1664009b30 comp rx=0 tx=0).stop 2024-09-10T08:23:26.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1675205740 1 -- 172.21.15.2:0/1879241770 shutdown_connections 2024-09-10T08:23:26.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1675205740 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5597277415b0 0x5597277439a0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1675205740 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597276c6020 0x5597276ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1675205740 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597276bb760 0x5597276ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1675205740 1 --2- 172.21.15.2:0/1879241770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597276bae40 0x5597276c9d60 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1659600640 0 client.19779 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:26.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1675205740 1 -- 172.21.15.2:0/1879241770 >> 172.21.15.2:0/1879241770 conn(0x5597276b73d0 msgr2=0x5597276bc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:26.741 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:23:26.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1675205740 1 -- 172.21.15.2:0/1879241770 shutdown_connections 2024-09-10T08:23:26.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1675205740 1 -- 172.21.15.2:0/1879241770 wait complete. 2024-09-10T08:23:26.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.706+0000 7f1675205740 20 client.19779 trim_cache size 0 max 16384 2024-09-10T08:23:26.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.707+0000 7f1671000640 1 -- 172.21.15.2:0/507105170 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16500a2b80 msgr2=0x7f16500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:26.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.707+0000 7f1671000640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16500a2b80 0x7f16500a5040 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f165c0cd130 tx=0x7f165c0d5550 comp rx=0 tx=0).stop 2024-09-10T08:23:26.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.707+0000 7f1671000640 1 -- 172.21.15.2:0/507105170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f166c103e30 msgr2=0x7f166c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:26.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.707+0000 7f1671000640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f166c103e30 0x7f166c19a490 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f165c00e9c0 tx=0x7f165c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:23:26.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.707+0000 7f1671000640 1 -- 172.21.15.2:0/507105170 shutdown_connections 2024-09-10T08:23:26.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.707+0000 7f1671000640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f16500a2b80 0x7f16500a5040 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.707+0000 7f1671000640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f166c108f60 0x7f166c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.707+0000 7f1671000640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f166c1047e0 0x7f166c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.707+0000 7f1671000640 1 --2- 172.21.15.2:0/507105170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f166c103e30 0x7f166c19a490 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:26.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.707+0000 7f1671000640 1 -- 172.21.15.2:0/507105170 >> 172.21.15.2:0/507105170 conn(0x7f166c0ff8d0 msgr2=0x7f166c100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:26.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.707+0000 7f1671000640 1 -- 172.21.15.2:0/507105170 shutdown_connections 2024-09-10T08:23:26.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:26.707+0000 7f1671000640 1 -- 172.21.15.2:0/507105170 wait complete. 2024-09-10T08:23:27.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:27 smithi002 ceph-mon[30563]: pgmap v846: 129 pgs: 129 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:23:27.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:27 smithi155 ceph-mon[35405]: pgmap v846: 129 pgs: 129 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:23:27.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:27 smithi107 ceph-mon[35048]: pgmap v846: 129 pgs: 129 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-10T08:23:28.709 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:23:28.710 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:23:29.105 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:23:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.683+0000 7f838f400640 1 -- 172.21.15.2:0/2706307301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8388077040 msgr2=0x7f83881124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.683+0000 7f838f400640 1 --2- 172.21.15.2:0/2706307301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8388077040 0x7f83881124f0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8370009900 tx=0x7f837002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.683+0000 7f838f400640 1 -- 172.21.15.2:0/2706307301 shutdown_connections 2024-09-10T08:23:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.683+0000 7f838f400640 1 --2- 172.21.15.2:0/2706307301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8388112a30 0x7f8388114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.683+0000 7f838f400640 1 --2- 172.21.15.2:0/2706307301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8388077040 0x7f83881124f0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.683+0000 7f838f400640 1 --2- 172.21.15.2:0/2706307301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8388076720 0x7f8388076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.683+0000 7f838f400640 1 -- 172.21.15.2:0/2706307301 >> 172.21.15.2:0/2706307301 conn(0x7f83880ffa90 msgr2=0x7f8388101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.683+0000 7f838f400640 1 -- 172.21.15.2:0/2706307301 shutdown_connections 2024-09-10T08:23:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.683+0000 7f838f400640 1 -- 172.21.15.2:0/2706307301 wait complete. 2024-09-10T08:23:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.684+0000 7f838f400640 1 Processor -- start 2024-09-10T08:23:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.684+0000 7f838f400640 1 -- start start 2024-09-10T08:23:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8388076720 0x7f83881a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8388077040 0x7f83881a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8388112a30 0x7f838819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838f400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8388118610 con 0x7f8388112a30 2024-09-10T08:23:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838f400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8388118490 con 0x7f8388076720 2024-09-10T08:23:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838f400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8388118790 con 0x7f8388077040 2024-09-10T08:23:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8388112a30 0x7f838819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8388112a30 0x7f838819e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49656/0 (socket says 172.21.15.2:49656) 2024-09-10T08:23:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838d400640 1 -- 172.21.15.2:0/1759255607 learned_addr learned my addr 172.21.15.2:0/1759255607 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838ca00640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8388076720 0x7f83881a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f837fe00640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8388077040 0x7f83881a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838d400640 1 -- 172.21.15.2:0/1759255607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8388077040 msgr2=0x7f83881a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838d400640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8388077040 0x7f83881a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838d400640 1 -- 172.21.15.2:0/1759255607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8388076720 msgr2=0x7f83881a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838d400640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8388076720 0x7f83881a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f838d400640 1 -- 172.21.15.2:0/1759255607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838819eff0 con 0x7f8388112a30 2024-09-10T08:23:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.685+0000 7f837fe00640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8388077040 0x7f83881a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:23:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.686+0000 7f838d400640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8388112a30 0x7f838819e8c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f838000ec50 tx=0x7f838000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.686+0000 7f8386000640 1 -- 172.21.15.2:0/1759255607 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380003b70 con 0x7f8388112a30 2024-09-10T08:23:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.686+0000 7f8386000640 1 -- 172.21.15.2:0/1759255607 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8380003d10 con 0x7f8388112a30 2024-09-10T08:23:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.686+0000 7f838f400640 1 -- 172.21.15.2:0/1759255607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838819f2e0 con 0x7f8388112a30 2024-09-10T08:23:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.686+0000 7f8386000640 1 -- 172.21.15.2:0/1759255607 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83800109e0 con 0x7f8388112a30 2024-09-10T08:23:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.686+0000 7f838f400640 1 -- 172.21.15.2:0/1759255607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83881ab5d0 con 0x7f8388112a30 2024-09-10T08:23:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.689+0000 7f8391dcb740 1 Processor -- start 2024-09-10T08:23:29.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.690+0000 7f8386000640 1 -- 172.21.15.2:0/1759255607 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8380003ec0 con 0x7f8388112a30 2024-09-10T08:23:29.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.690+0000 7f8391dcb740 1 -- start start 2024-09-10T08:23:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.690+0000 7f8391dcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf38ac0 0x5571cdf38ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.690+0000 7f8391dcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf393e0 0x5571cdf43760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.690+0000 7f8391dcb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 0x5571cdf46090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.690+0000 7f8391dcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5571cde9f290 con 0x5571cdf38ac0 2024-09-10T08:23:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.690+0000 7f8391dcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5571cde46b40 con 0x5571cdf393e0 2024-09-10T08:23:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.690+0000 7f8391dcb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5571cde9b250 con 0x5571cdf43ca0 2024-09-10T08:23:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.690+0000 7f837fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf38ac0 0x5571cdf38ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.690+0000 7f838ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf393e0 0x5571cdf43760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.690+0000 7f837fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf38ac0 0x5571cdf38ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49660/0 (socket says 172.21.15.2:49660) 2024-09-10T08:23:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.690+0000 7f837fe00640 1 -- 172.21.15.2:0/1259999200 learned_addr learned my addr 172.21.15.2:0/1259999200 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.691+0000 7f837fe00640 1 --2- 172.21.15.2:0/1259999200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 0x5571cdf46090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.691+0000 7f838ca00640 1 -- 172.21.15.2:0/1259999200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 msgr2=0x5571cdf46090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.691+0000 7f838ca00640 1 --2- 172.21.15.2:0/1259999200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 0x5571cdf46090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.691+0000 7f838ca00640 1 -- 172.21.15.2:0/1259999200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf38ac0 msgr2=0x5571cdf38ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.691+0000 7f838ca00640 1 --2- 172.21.15.2:0/1259999200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf38ac0 0x5571cdf38ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.691+0000 7f838ca00640 1 -- 172.21.15.2:0/1259999200 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571cdc106a0 con 0x5571cdf393e0 2024-09-10T08:23:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.691+0000 7f838ca00640 1 --2- 172.21.15.2:0/1259999200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf393e0 0x5571cdf43760 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8378009bd0 tx=0x7f8378011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78f84d2987444ed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.691+0000 7f8386000640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f836c0a2b80 0x7f836c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.691+0000 7f8386000640 1 -- 172.21.15.2:0/1759255607 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f83800cfb10 con 0x7f8388112a30 2024-09-10T08:23:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.692+0000 7f837dc00640 1 -- 172.21.15.2:0/1259999200 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378004230 con 0x5571cdf393e0 2024-09-10T08:23:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.692+0000 7f837dc00640 1 -- 172.21.15.2:0/1259999200 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8378009e40 con 0x5571cdf393e0 2024-09-10T08:23:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.692+0000 7f837dc00640 1 -- 172.21.15.2:0/1259999200 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378004d70 con 0x5571cdf393e0 2024-09-10T08:23:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.692+0000 7f837fe00640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f836c0a2b80 0x7f836c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.692+0000 7f8391dcb740 1 -- 172.21.15.2:0/1259999200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf393e0 msgr2=0x5571cdf43760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.692+0000 7f8391dcb740 1 --2- 172.21.15.2:0/1259999200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf393e0 0x5571cdf43760 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8378009bd0 tx=0x7f8378011810 comp rx=0 tx=0).stop 2024-09-10T08:23:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.692+0000 7f837fe00640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f836c0a2b80 0x7f836c0a5040 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x5571cdf363d0 tx=0x7f8370005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.692+0000 7f8391dcb740 1 -- 172.21.15.2:0/1259999200 shutdown_connections 2024-09-10T08:23:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.692+0000 7f8391dcb740 1 --2- 172.21.15.2:0/1259999200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 0x5571cdf46090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.692+0000 7f8391dcb740 1 --2- 172.21.15.2:0/1259999200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf393e0 0x5571cdf43760 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.692+0000 7f8391dcb740 1 --2- 172.21.15.2:0/1259999200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf38ac0 0x5571cdf38ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.692+0000 7f8391dcb740 1 -- 172.21.15.2:0/1259999200 >> 172.21.15.2:0/1259999200 conn(0x5571cdf35050 msgr2=0x5571cdf35430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.693+0000 7f8391dcb740 1 -- 172.21.15.2:0/1259999200 shutdown_connections 2024-09-10T08:23:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.693+0000 7f8391dcb740 1 -- 172.21.15.2:0/1259999200 wait complete. 2024-09-10T08:23:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.693+0000 7f8391dcb740 1 -- start start 2024-09-10T08:23:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.693+0000 7f8391dcb740 1 Processor -- start 2024-09-10T08:23:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.694+0000 7f8391dcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf38ac0 0x5571cdf43530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.694+0000 7f8391dcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf393e0 0x5571cdf407e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.694+0000 7f8391dcb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 0x5571cdf40d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.694+0000 7f8391dcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5571cdf46910 con 0x5571cdf393e0 2024-09-10T08:23:29.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.694+0000 7f8391dcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5571cde59fa0 con 0x5571cdf38ac0 2024-09-10T08:23:29.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.694+0000 7f8391dcb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5571cdf46a90 con 0x5571cdf43ca0 2024-09-10T08:23:29.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.694+0000 7f8391dcb740 10 client.0 _ll_register_callbacks cb 0x5571cdbadba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.694+0000 7f837fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf393e0 0x5571cdf407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.694+0000 7f838ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf38ac0 0x5571cdf43530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.694+0000 7f837fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf393e0 0x5571cdf407e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49674/0 (socket says 172.21.15.2:49674) 2024-09-10T08:23:29.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.694+0000 7f838ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf38ac0 0x5571cdf43530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60594/0 (socket says 172.21.15.2:60594) 2024-09-10T08:23:29.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.694+0000 7f838ca00640 1 -- 172.21.15.2:0/412846990 learned_addr learned my addr 172.21.15.2:0/412846990 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:29.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.694+0000 7f838d400640 1 --2- 172.21.15.2:0/412846990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 0x5571cdf40d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.695+0000 7f838ca00640 1 -- 172.21.15.2:0/412846990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 msgr2=0x5571cdf40d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:29.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.695+0000 7f838ca00640 1 --2- 172.21.15.2:0/412846990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 0x5571cdf40d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.695+0000 7f838ca00640 1 -- 172.21.15.2:0/412846990 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf393e0 msgr2=0x5571cdf407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:29.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.695+0000 7f838ca00640 1 --2- 172.21.15.2:0/412846990 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf393e0 0x5571cdf407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.695+0000 7f838ca00640 1 -- 172.21.15.2:0/412846990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571cdf41500 con 0x5571cdf38ac0 2024-09-10T08:23:29.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.695+0000 7f838ca00640 1 --2- 172.21.15.2:0/412846990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf38ac0 0x5571cdf43530 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f83780048f0 tx=0x7f8378003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9925c02a9641b46b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:29.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.695+0000 7f8376000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:23:29.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.695+0000 7f8376000640 1 -- 172.21.15.2:0/412846990 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378027020 con 0x5571cdf38ac0 2024-09-10T08:23:29.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.695+0000 7f8376000640 1 -- 172.21.15.2:0/412846990 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8378003c10 con 0x5571cdf38ac0 2024-09-10T08:23:29.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.695+0000 7f8391dcb740 1 -- 172.21.15.2:0/412846990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5571cdeac800 con 0x5571cdf38ac0 2024-09-10T08:23:29.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.696+0000 7f8376000640 1 -- 172.21.15.2:0/412846990 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8378032050 con 0x5571cdf38ac0 2024-09-10T08:23:29.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.696+0000 7f8391dcb740 10 client.29813 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:29.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.696+0000 7f8391dcb740 1 -- 172.21.15.2:0/412846990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5571cdf527a0 con 0x5571cdf38ac0 2024-09-10T08:23:29.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.696+0000 7f8376000640 1 -- 172.21.15.2:0/412846990 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5571cdeac800 con 0x5571cdf38ac0 2024-09-10T08:23:29.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.696+0000 7f8376000640 1 -- 172.21.15.2:0/412846990 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8378018070 con 0x5571cdf38ac0 2024-09-10T08:23:29.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.696+0000 7f8391dcb740 10 client.29813 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:29.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.696+0000 7f8391dcb740 10 client.29813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:29.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.696+0000 7f8391dcb740 20 client.29813 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:29.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.696+0000 7f8391dcb740 1 --2- 172.21.15.2:0/412846990 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5571cdf53c60 0x5571cdf56050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.696+0000 7f8391dcb740 4 client.29813 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:23:29.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.696+0000 7f8391dcb740 1 -- 172.21.15.2:0/412846990 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571cdea6d10 con 0x5571cdf53c60 2024-09-10T08:23:29.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.697+0000 7f837fe00640 1 --2- 172.21.15.2:0/412846990 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5571cdf53c60 0x5571cdf56050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.697+0000 7f837fe00640 1 --2- 172.21.15.2:0/412846990 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5571cdf53c60 0x5571cdf56050 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c589d18420107745 server_cookie=56d3abd6ea2010cf in_seq=0 out_seq=0 2024-09-10T08:23:29.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.697+0000 7f8376000640 10 client.29813 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:29.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8376000640 1 -- 172.21.15.2:0/412846990 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571cdea6d10 con 0x5571cdf53c60 2024-09-10T08:23:29.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8376000640 10 client.29813 handle_command_reply: tid=0 2024-09-10T08:23:29.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8391dcb740 1 client.29813 shutdown 2024-09-10T08:23:29.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8391dcb740 1 -- 172.21.15.2:0/412846990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf38ac0 msgr2=0x5571cdf43530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:29.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8391dcb740 1 --2- 172.21.15.2:0/412846990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf38ac0 0x5571cdf43530 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f83780048f0 tx=0x7f8378003820 comp rx=0 tx=0).stop 2024-09-10T08:23:29.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8391dcb740 1 -- 172.21.15.2:0/412846990 shutdown_connections 2024-09-10T08:23:29.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8391dcb740 1 --2- 172.21.15.2:0/412846990 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5571cdf53c60 0x5571cdf56050 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8391dcb740 1 --2- 172.21.15.2:0/412846990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 0x5571cdf40d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8391dcb740 1 --2- 172.21.15.2:0/412846990 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf393e0 0x5571cdf407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8376000640 0 client.29813 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:29.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8391dcb740 1 --2- 172.21.15.2:0/412846990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf38ac0 0x5571cdf43530 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8391dcb740 1 -- 172.21.15.2:0/412846990 >> 172.21.15.2:0/412846990 conn(0x5571cdf35050 msgr2=0x5571cdf3ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:29.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8391dcb740 1 -- 172.21.15.2:0/412846990 shutdown_connections 2024-09-10T08:23:29.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.698+0000 7f8391dcb740 1 -- 172.21.15.2:0/412846990 wait complete. 2024-09-10T08:23:29.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.699+0000 7f8391dcb740 20 client.29813 trim_cache size 0 max 16384 2024-09-10T08:23:29.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.716+0000 7f8391dcb740 1 Processor -- start 2024-09-10T08:23:29.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.717+0000 7f8391dcb740 1 -- start start 2024-09-10T08:23:29.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.717+0000 7f8391dcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf38ac0 0x5571cdf40c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.717+0000 7f8391dcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf393e0 0x5571cdf42750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.717+0000 7f8391dcb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 0x5571cdf5b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.717+0000 7f8391dcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83881b7d50 con 0x5571cdf38ac0 2024-09-10T08:23:29.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.717+0000 7f8391dcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f83881b5620 con 0x5571cdf393e0 2024-09-10T08:23:29.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.717+0000 7f8391dcb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f83881b3470 con 0x5571cdf43ca0 2024-09-10T08:23:29.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.717+0000 7f838ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf38ac0 0x5571cdf40c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.717+0000 7f838ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf38ac0 0x5571cdf40c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49682/0 (socket says 172.21.15.2:49682) 2024-09-10T08:23:29.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.717+0000 7f838ca00640 1 -- 172.21.15.2:0/3028500344 learned_addr learned my addr 172.21.15.2:0/3028500344 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:29.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.717+0000 7f837fe00640 1 --2- 172.21.15.2:0/3028500344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf393e0 0x5571cdf42750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.717+0000 7f838d400640 1 --2- 172.21.15.2:0/3028500344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 0x5571cdf5b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.718+0000 7f838ca00640 1 -- 172.21.15.2:0/3028500344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 msgr2=0x5571cdf5b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:29.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.718+0000 7f838ca00640 1 --2- 172.21.15.2:0/3028500344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 0x5571cdf5b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.718+0000 7f838ca00640 1 -- 172.21.15.2:0/3028500344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf393e0 msgr2=0x5571cdf42750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:29.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.718+0000 7f838ca00640 1 --2- 172.21.15.2:0/3028500344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf393e0 0x5571cdf42750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.718+0000 7f838ca00640 1 -- 172.21.15.2:0/3028500344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571cdf34e80 con 0x5571cdf38ac0 2024-09-10T08:23:29.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.718+0000 7f838d400640 1 --2- 172.21.15.2:0/3028500344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 0x5571cdf5b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:23:29.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.718+0000 7f838ca00640 1 --2- 172.21.15.2:0/3028500344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf38ac0 0x5571cdf40c00 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f83780059f0 tx=0x7f8378009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c1c57f32151664 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:29.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.718+0000 7f837d200640 1 -- 172.21.15.2:0/3028500344 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378027040 con 0x5571cdf38ac0 2024-09-10T08:23:29.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.718+0000 7f837d200640 1 -- 172.21.15.2:0/3028500344 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83780238f0 con 0x5571cdf38ac0 2024-09-10T08:23:29.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.718+0000 7f837d200640 1 -- 172.21.15.2:0/3028500344 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837801e040 con 0x5571cdf38ac0 2024-09-10T08:23:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.719+0000 7f8391dcb740 1 -- 172.21.15.2:0/3028500344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf38ac0 msgr2=0x5571cdf40c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.719+0000 7f8391dcb740 1 --2- 172.21.15.2:0/3028500344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf38ac0 0x5571cdf40c00 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f83780059f0 tx=0x7f8378009a50 comp rx=0 tx=0).stop 2024-09-10T08:23:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.719+0000 7f8391dcb740 1 -- 172.21.15.2:0/3028500344 shutdown_connections 2024-09-10T08:23:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.719+0000 7f8391dcb740 1 --2- 172.21.15.2:0/3028500344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf43ca0 0x5571cdf5b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.719+0000 7f8391dcb740 1 --2- 172.21.15.2:0/3028500344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf393e0 0x5571cdf42750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.719+0000 7f8391dcb740 1 --2- 172.21.15.2:0/3028500344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf38ac0 0x5571cdf40c00 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.719+0000 7f8391dcb740 1 -- 172.21.15.2:0/3028500344 >> 172.21.15.2:0/3028500344 conn(0x5571cdf35050 msgr2=0x5571cdf42040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.719+0000 7f8391dcb740 1 -- 172.21.15.2:0/3028500344 shutdown_connections 2024-09-10T08:23:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.719+0000 7f8391dcb740 1 -- 172.21.15.2:0/3028500344 wait complete. 2024-09-10T08:23:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.719+0000 7f8391dcb740 1 -- start start 2024-09-10T08:23:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f8391dcb740 1 Processor -- start 2024-09-10T08:23:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f8391dcb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf38ac0 0x5571cdf4be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f8391dcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf393e0 0x5571cdf4c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f8391dcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf43ca0 0x5571cdf493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f8391dcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83881b3ae0 con 0x5571cdf393e0 2024-09-10T08:23:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f8391dcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f83881b4f50 con 0x5571cdf43ca0 2024-09-10T08:23:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f8391dcb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f83881b41b0 con 0x5571cdf38ac0 2024-09-10T08:23:29.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f8391dcb740 10 client.0 _ll_register_callbacks cb 0x7f83800093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f837fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf393e0 0x5571cdf4c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f838ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf38ac0 0x5571cdf4be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f837fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf393e0 0x5571cdf4c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49698/0 (socket says 172.21.15.2:49698) 2024-09-10T08:23:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f838ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf38ac0 0x5571cdf4be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:33314/0 (socket says 172.21.15.2:33314) 2024-09-10T08:23:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f837fe00640 1 -- 172.21.15.2:0/324369654 learned_addr learned my addr 172.21.15.2:0/324369654 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.720+0000 7f838d400640 1 --2- 172.21.15.2:0/324369654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf43ca0 0x5571cdf493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.721+0000 7f838ca00640 1 -- 172.21.15.2:0/324369654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf43ca0 msgr2=0x5571cdf493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.721+0000 7f838ca00640 1 --2- 172.21.15.2:0/324369654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf43ca0 0x5571cdf493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.721+0000 7f838ca00640 1 -- 172.21.15.2:0/324369654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf393e0 msgr2=0x5571cdf4c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.721+0000 7f838ca00640 1 --2- 172.21.15.2:0/324369654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf393e0 0x5571cdf4c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.721+0000 7f838ca00640 1 -- 172.21.15.2:0/324369654 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571cdf35c00 con 0x5571cdf38ac0 2024-09-10T08:23:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.721+0000 7f837fe00640 1 --2- 172.21.15.2:0/324369654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf393e0 0x5571cdf4c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.721+0000 7f838ca00640 1 --2- 172.21.15.2:0/324369654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf38ac0 0x5571cdf4be10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8378011850 tx=0x7f8378023620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b467b31a36a31b67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.721+0000 7f8376000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:23:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.721+0000 7f8376000640 1 -- 172.21.15.2:0/324369654 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837803a020 con 0x5571cdf38ac0 2024-09-10T08:23:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.722+0000 7f8391dcb740 1 -- 172.21.15.2:0/324369654 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83780238f0 con 0x5571cdf38ac0 2024-09-10T08:23:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.722+0000 7f8376000640 1 -- 172.21.15.2:0/324369654 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f8378009780 con 0x5571cdf38ac0 2024-09-10T08:23:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.722+0000 7f8376000640 1 -- 172.21.15.2:0/324369654 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8378045050 con 0x5571cdf38ac0 2024-09-10T08:23:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.722+0000 7f8391dcb740 10 client.29743 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.722+0000 7f8391dcb740 1 -- 172.21.15.2:0/324369654 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5571cdfbe0f0 con 0x5571cdf38ac0 2024-09-10T08:23:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.722+0000 7f8376000640 1 -- 172.21.15.2:0/324369654 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83780238f0 con 0x5571cdf38ac0 2024-09-10T08:23:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.723+0000 7f8376000640 1 -- 172.21.15.2:0/324369654 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8378018030 con 0x5571cdf38ac0 2024-09-10T08:23:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.723+0000 7f8391dcb740 10 client.29743 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.723+0000 7f8391dcb740 10 client.29743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.723+0000 7f8391dcb740 20 client.29743 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.723+0000 7f8391dcb740 1 --2- 172.21.15.2:0/324369654 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5571cdfbe780 0x5571cdfc0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.723+0000 7f8391dcb740 4 client.29743 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:23:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.723+0000 7f8391dcb740 1 -- 172.21.15.2:0/324369654 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5571cde22fb0 con 0x5571cdfbe780 2024-09-10T08:23:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.723+0000 7f837fe00640 1 --2- 172.21.15.2:0/324369654 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5571cdfbe780 0x5571cdfc0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.724+0000 7f8376000640 10 client.29743 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.724+0000 7f837fe00640 1 --2- 172.21.15.2:0/324369654 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5571cdfbe780 0x5571cdfc0ba0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b504fd2ee1030375 server_cookie=e13e0e495d433547 in_seq=0 out_seq=0 2024-09-10T08:23:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8376000640 1 -- 172.21.15.2:0/324369654 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5571cde22fb0 con 0x5571cdfbe780 2024-09-10T08:23:29.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8376000640 10 client.29743 handle_command_reply: tid=0 2024-09-10T08:23:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8391dcb740 1 client.29743 shutdown 2024-09-10T08:23:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8391dcb740 1 -- 172.21.15.2:0/324369654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf38ac0 msgr2=0x5571cdf4be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8391dcb740 1 --2- 172.21.15.2:0/324369654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf38ac0 0x5571cdf4be10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8378011850 tx=0x7f8378023620 comp rx=0 tx=0).stop 2024-09-10T08:23:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8391dcb740 1 -- 172.21.15.2:0/324369654 shutdown_connections 2024-09-10T08:23:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8391dcb740 1 --2- 172.21.15.2:0/324369654 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5571cdfbe780 0x5571cdfc0ba0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8391dcb740 1 --2- 172.21.15.2:0/324369654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571cdf43ca0 0x5571cdf493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8391dcb740 1 --2- 172.21.15.2:0/324369654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5571cdf393e0 0x5571cdf4c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8391dcb740 1 --2- 172.21.15.2:0/324369654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571cdf38ac0 0x5571cdf4be10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8391dcb740 1 -- 172.21.15.2:0/324369654 >> 172.21.15.2:0/324369654 conn(0x5571cdf35050 msgr2=0x5571cdf341c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8376000640 0 client.29743 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8391dcb740 1 -- 172.21.15.2:0/324369654 shutdown_connections 2024-09-10T08:23:29.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8391dcb740 1 -- 172.21.15.2:0/324369654 wait complete. 2024-09-10T08:23:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.725+0000 7f8391dcb740 20 client.29743 trim_cache size 0 max 16384 2024-09-10T08:23:29.729 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:23:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:23:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "064ae0d5-6f5e-4292-a292-5642f9fffd24", 2024-09-10T08:23:29.729 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:23:29.729 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:23:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.726+0000 7f838f400640 1 -- 172.21.15.2:0/1759255607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f836c0a2b80 msgr2=0x7f836c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.726+0000 7f838f400640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f836c0a2b80 0x7f836c0a5040 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x5571cdf363d0 tx=0x7f8370005c00 comp rx=0 tx=0).stop 2024-09-10T08:23:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.726+0000 7f838f400640 1 -- 172.21.15.2:0/1759255607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8388112a30 msgr2=0x7f838819e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.726+0000 7f838f400640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8388112a30 0x7f838819e8c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f838000ec50 tx=0x7f838000c6a0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.726+0000 7f838f400640 1 -- 172.21.15.2:0/1759255607 shutdown_connections 2024-09-10T08:23:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.726+0000 7f838f400640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8388112a30 0x7f838819e8c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.726+0000 7f838f400640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f836c0a2b80 0x7f836c0a5040 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.726+0000 7f838f400640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8388077040 0x7f83881a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.726+0000 7f838f400640 1 --2- 172.21.15.2:0/1759255607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8388076720 0x7f83881a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.726+0000 7f838f400640 1 -- 172.21.15.2:0/1759255607 >> 172.21.15.2:0/1759255607 conn(0x7f83880ffa90 msgr2=0x7f8388101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.726+0000 7f838f400640 1 -- 172.21.15.2:0/1759255607 shutdown_connections 2024-09-10T08:23:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:29.727+0000 7f838f400640 1 -- 172.21.15.2:0/1759255607 wait complete. 2024-09-10T08:23:29.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:29 smithi002 ceph-mon[30563]: pgmap v847: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-10T08:23:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:29 smithi155 ceph-mon[35405]: pgmap v847: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-10T08:23:29.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:29 smithi107 ceph-mon[35048]: pgmap v847: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-10T08:23:30.009 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T08:23:30.009 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.85 2024-09-10T08:23:30.009 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:23:30.660 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:23:30.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:30 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:23:30.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:30 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:23:30.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:30 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:23:30.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:30 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:23:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:23:30.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:30 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:23:30.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:30 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:23:30.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:30 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:23:30.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:30 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:23:30.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:30 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:23:30.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:30 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:23:30.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:30 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:23:30.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:30 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:23:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.457+0000 7f3246600640 1 -- 172.21.15.2:0/1890767414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32401049a0 msgr2=0x7f32401089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.457+0000 7f3246600640 1 --2- 172.21.15.2:0/1890767414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32401049a0 0x7f32401089f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3228009860 tx=0x7f322802a340 comp rx=0 tx=0).stop 2024-09-10T08:23:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.457+0000 7f3246600640 1 -- 172.21.15.2:0/1890767414 shutdown_connections 2024-09-10T08:23:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.457+0000 7f3246600640 1 --2- 172.21.15.2:0/1890767414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3240109120 0x7f3240110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.457+0000 7f3246600640 1 --2- 172.21.15.2:0/1890767414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32401049a0 0x7f32401089f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.457+0000 7f3246600640 1 --2- 172.21.15.2:0/1890767414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3240103ff0 0x7f32401043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.457+0000 7f3246600640 1 -- 172.21.15.2:0/1890767414 >> 172.21.15.2:0/1890767414 conn(0x7f32400ffa90 msgr2=0x7f3240101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.457+0000 7f3246600640 1 -- 172.21.15.2:0/1890767414 shutdown_connections 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.457+0000 7f3246600640 1 -- 172.21.15.2:0/1890767414 wait complete. 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.458+0000 7f3246600640 1 Processor -- start 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.458+0000 7f3246600640 1 -- start start 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.459+0000 7f3246600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3240103ff0 0x7f324019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.459+0000 7f3246600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32401049a0 0x7f324019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.459+0000 7f3246600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3240109120 0x7f324019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.459+0000 7f3246600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3240114140 con 0x7f32401049a0 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.459+0000 7f3246600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3240113fc0 con 0x7f3240103ff0 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.459+0000 7f3246600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f32401142c0 con 0x7f3240109120 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.459+0000 7f3237e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32401049a0 0x7f324019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.459+0000 7f3237e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32401049a0 0x7f324019ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49722/0 (socket says 172.21.15.2:49722) 2024-09-10T08:23:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.459+0000 7f3237e00640 1 -- 172.21.15.2:0/3832235712 learned_addr learned my addr 172.21.15.2:0/3832235712 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.459+0000 7f323f400640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3240103ff0 0x7f324019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.459+0000 7f323fe00640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3240109120 0x7f324019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.460+0000 7f3237e00640 1 -- 172.21.15.2:0/3832235712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3240109120 msgr2=0x7f324019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.460+0000 7f3237e00640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3240109120 0x7f324019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.460+0000 7f3237e00640 1 -- 172.21.15.2:0/3832235712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3240103ff0 msgr2=0x7f324019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.460+0000 7f3237e00640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3240103ff0 0x7f324019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.460+0000 7f3237e00640 1 -- 172.21.15.2:0/3832235712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324019f670 con 0x7f32401049a0 2024-09-10T08:23:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.460+0000 7f323f400640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3240103ff0 0x7f324019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:23:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.460+0000 7f3237e00640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32401049a0 0x7f324019ac00 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3228035040 tx=0x7f3228033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.460+0000 7f323cc00640 1 -- 172.21.15.2:0/3832235712 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228037040 con 0x7f32401049a0 2024-09-10T08:23:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.460+0000 7f323cc00640 1 -- 172.21.15.2:0/3832235712 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3228040070 con 0x7f32401049a0 2024-09-10T08:23:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.460+0000 7f323cc00640 1 -- 172.21.15.2:0/3832235712 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32280039b0 con 0x7f32401049a0 2024-09-10T08:23:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.460+0000 7f3246600640 1 -- 172.21.15.2:0/3832235712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324019f900 con 0x7f32401049a0 2024-09-10T08:23:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.460+0000 7f3246600640 1 -- 172.21.15.2:0/3832235712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324019fd90 con 0x7f32401049a0 2024-09-10T08:23:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.462+0000 7f3246600640 1 -- 172.21.15.2:0/3832235712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32401043d0 con 0x7f32401049a0 2024-09-10T08:23:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.463+0000 7f323cc00640 1 -- 172.21.15.2:0/3832235712 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f32280305e0 con 0x7f32401049a0 2024-09-10T08:23:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.465+0000 7f323cc00640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f32240a2cd0 0x7f32240a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.465+0000 7f323cc00640 1 -- 172.21.15.2:0/3832235712 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f32280ef440 con 0x7f32401049a0 2024-09-10T08:23:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.465+0000 7f323f400640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f32240a2cd0 0x7f32240a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:31.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.466+0000 7f323f400640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f32240a2cd0 0x7f32240a5190 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3230009c20 tx=0x7f3230005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:31.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.469+0000 7f323cc00640 1 -- 172.21.15.2:0/3832235712 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32280bbfb0 con 0x7f32401049a0 2024-09-10T08:23:31.617 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:23:31.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.701+0000 7f3246600640 1 -- 172.21.15.2:0/3832235712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f324010c360 con 0x7f32401049a0 2024-09-10T08:23:31.702 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:31 smithi002 ceph-mon[30563]: pgmap v848: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-10T08:23:31.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.703+0000 7f323cc00640 1 -- 172.21.15.2:0/3832235712 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f32280bb8b0 con 0x7f32401049a0 2024-09-10T08:23:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:23:31.703 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:23:31.703 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:23:31.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.705+0000 7f3246600640 1 -- 172.21.15.2:0/3832235712 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f32240a2cd0 msgr2=0x7f32240a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:31.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.705+0000 7f3246600640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f32240a2cd0 0x7f32240a5190 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3230009c20 tx=0x7f3230005f90 comp rx=0 tx=0).stop 2024-09-10T08:23:31.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.705+0000 7f3246600640 1 -- 172.21.15.2:0/3832235712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32401049a0 msgr2=0x7f324019ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:31.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.705+0000 7f3246600640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32401049a0 0x7f324019ac00 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3228035040 tx=0x7f3228033b60 comp rx=0 tx=0).stop 2024-09-10T08:23:31.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.705+0000 7f3246600640 1 -- 172.21.15.2:0/3832235712 shutdown_connections 2024-09-10T08:23:31.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.705+0000 7f3246600640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f32240a2cd0 0x7f32240a5190 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:31.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.705+0000 7f3246600640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3240109120 0x7f324019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:31.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.705+0000 7f3246600640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32401049a0 0x7f324019ac00 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:31.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.705+0000 7f3246600640 1 --2- 172.21.15.2:0/3832235712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3240103ff0 0x7f324019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:31.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.705+0000 7f3246600640 1 -- 172.21.15.2:0/3832235712 >> 172.21.15.2:0/3832235712 conn(0x7f32400ffa90 msgr2=0x7f3240100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:31.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.705+0000 7f3246600640 1 -- 172.21.15.2:0/3832235712 shutdown_connections 2024-09-10T08:23:31.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:31.705+0000 7f3246600640 1 -- 172.21.15.2:0/3832235712 wait complete. 2024-09-10T08:23:31.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:31 smithi155 ceph-mon[35405]: pgmap v848: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-10T08:23:31.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:31 smithi107 ceph-mon[35048]: pgmap v848: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-10T08:23:32.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.358+0000 7f4fd2200640 1 -- 172.21.15.2:0/74430132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fcc1079c0 msgr2=0x7f4fcc107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:32.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.358+0000 7f4fd2200640 1 --2- 172.21.15.2:0/74430132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fcc1079c0 0x7f4fcc107da0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009900 tx=0x7f4fbc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.359+0000 7f4fd2200640 1 -- 172.21.15.2:0/74430132 shutdown_connections 2024-09-10T08:23:32.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.359+0000 7f4fd2200640 1 --2- 172.21.15.2:0/74430132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fcc102360 0x7f4fcc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.359+0000 7f4fd2200640 1 --2- 172.21.15.2:0/74430132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fcc1019c0 0x7f4fcc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.359+0000 7f4fd2200640 1 --2- 172.21.15.2:0/74430132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fcc1079c0 0x7f4fcc107da0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.359+0000 7f4fd2200640 1 -- 172.21.15.2:0/74430132 >> 172.21.15.2:0/74430132 conn(0x7f4fcc0fd740 msgr2=0x7f4fcc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:32.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.359+0000 7f4fd2200640 1 -- 172.21.15.2:0/74430132 shutdown_connections 2024-09-10T08:23:32.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.359+0000 7f4fd2200640 1 -- 172.21.15.2:0/74430132 wait complete. 2024-09-10T08:23:32.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.360+0000 7f4fd2200640 1 Processor -- start 2024-09-10T08:23:32.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.360+0000 7f4fd2200640 1 -- start start 2024-09-10T08:23:32.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.361+0000 7f4fd2200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fcc1019c0 0x7f4fcc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.361+0000 7f4fd2200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fcc102360 0x7f4fcc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.361+0000 7f4fd2200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fcc1079c0 0x7f4fcc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.361+0000 7f4fd2200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc111e10 con 0x7f4fcc1079c0 2024-09-10T08:23:32.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.361+0000 7f4fd2200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc111c90 con 0x7f4fcc1019c0 2024-09-10T08:23:32.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.361+0000 7f4fd2200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc111f90 con 0x7f4fcc102360 2024-09-10T08:23:32.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.361+0000 7f4fd1800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fcc1079c0 0x7f4fcc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.361+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fcc102360 0x7f4fcc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.361+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fcc102360 0x7f4fcc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:33366/0 (socket says 172.21.15.2:33366) 2024-09-10T08:23:32.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.361+0000 7f4fc3e00640 1 -- 172.21.15.2:0/151249325 learned_addr learned my addr 172.21.15.2:0/151249325 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:32.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.361+0000 7f4fd0e00640 1 --2- 172.21.15.2:0/151249325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fcc1019c0 0x7f4fcc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.362+0000 7f4fc3e00640 1 -- 172.21.15.2:0/151249325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fcc1019c0 msgr2=0x7f4fcc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:32.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.362+0000 7f4fc3e00640 1 --2- 172.21.15.2:0/151249325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fcc1019c0 0x7f4fcc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.362+0000 7f4fc3e00640 1 -- 172.21.15.2:0/151249325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fcc1079c0 msgr2=0x7f4fcc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:32.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.362+0000 7f4fc3e00640 1 --2- 172.21.15.2:0/151249325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fcc1079c0 0x7f4fcc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.362+0000 7f4fc3e00640 1 -- 172.21.15.2:0/151249325 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc1a6750 con 0x7f4fcc102360 2024-09-10T08:23:32.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.362+0000 7f4fc3e00640 1 --2- 172.21.15.2:0/151249325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fcc102360 0x7f4fcc074a30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400e9f0 tx=0x7f4fb400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:32.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.362+0000 7f4fca000640 1 -- 172.21.15.2:0/151249325 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400cd80 con 0x7f4fcc102360 2024-09-10T08:23:32.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.362+0000 7f4fd2200640 1 -- 172.21.15.2:0/151249325 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fcc1a6a40 con 0x7f4fcc102360 2024-09-10T08:23:32.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.362+0000 7f4fca000640 1 -- 172.21.15.2:0/151249325 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb4010910 con 0x7f4fcc102360 2024-09-10T08:23:32.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.362+0000 7f4fca000640 1 -- 172.21.15.2:0/151249325 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40188f0 con 0x7f4fcc102360 2024-09-10T08:23:32.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.362+0000 7f4fd2200640 1 -- 172.21.15.2:0/151249325 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fcc1a6fd0 con 0x7f4fcc102360 2024-09-10T08:23:32.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.365+0000 7f4fd624e740 1 Processor -- start 2024-09-10T08:23:32.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.365+0000 7f4fd624e740 1 -- start start 2024-09-10T08:23:32.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.366+0000 7f4fca000640 1 -- 172.21.15.2:0/151249325 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4fb4010ac0 con 0x7f4fcc102360 2024-09-10T08:23:32.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.366+0000 7f4fd624e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d6f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.366+0000 7f4fd624e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d6f670 0x563482d799f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.366+0000 7f4fd624e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d79f30 0x563482d7c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.366+0000 7f4fd624e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563482cd53e0 con 0x563482d6f670 2024-09-10T08:23:32.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.366+0000 7f4fd624e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563482c7cd80 con 0x563482d6ed50 2024-09-10T08:23:32.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.366+0000 7f4fd624e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563482cd13a0 con 0x563482d79f30 2024-09-10T08:23:32.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.366+0000 7f4fd0e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d6f670 0x563482d799f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.366+0000 7f4fd0e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d6f670 0x563482d799f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49756/0 (socket says 172.21.15.2:49756) 2024-09-10T08:23:32.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.366+0000 7f4fd0e00640 1 -- 172.21.15.2:0/1208555317 learned_addr learned my addr 172.21.15.2:0/1208555317 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:32.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.366+0000 7f4fd1800640 1 --2- 172.21.15.2:0/1208555317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d6f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.366+0000 7f4fc3e00640 1 --2- 172.21.15.2:0/1208555317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d79f30 0x563482d7c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.367+0000 7f4fd0e00640 1 -- 172.21.15.2:0/1208555317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d79f30 msgr2=0x563482d7c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:32.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.367+0000 7f4fd0e00640 1 --2- 172.21.15.2:0/1208555317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d79f30 0x563482d7c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.367+0000 7f4fd0e00640 1 -- 172.21.15.2:0/1208555317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 msgr2=0x563482d6f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:32.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.367+0000 7f4fd0e00640 1 --2- 172.21.15.2:0/1208555317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d6f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.367+0000 7f4fd0e00640 1 -- 172.21.15.2:0/1208555317 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563482c9bae0 con 0x563482d6f670 2024-09-10T08:23:32.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.367+0000 7f4fd0e00640 1 --2- 172.21.15.2:0/1208555317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d6f670 0x563482d799f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009230 tx=0x7f4fbc005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da90cd4f46ee782a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:32.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.367+0000 7f4fca000640 1 --2- 172.21.15.2:0/151249325 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4fb00a2c00 0x7f4fb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.367+0000 7f4fc2000640 1 -- 172.21.15.2:0/1208555317 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc035040 con 0x563482d6f670 2024-09-10T08:23:32.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.368+0000 7f4fca000640 1 -- 172.21.15.2:0/151249325 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4fb40cf7c0 con 0x7f4fcc102360 2024-09-10T08:23:32.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.368+0000 7f4fc2000640 1 -- 172.21.15.2:0/1208555317 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fbc004400 con 0x563482d6f670 2024-09-10T08:23:32.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.368+0000 7f4fc2000640 1 -- 172.21.15.2:0/1208555317 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc0035f0 con 0x563482d6f670 2024-09-10T08:23:32.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.368+0000 7f4fd1800640 1 --2- 172.21.15.2:0/151249325 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4fb00a2c00 0x7f4fb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.368+0000 7f4fd624e740 1 -- 172.21.15.2:0/1208555317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d6f670 msgr2=0x563482d799f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:32.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.368+0000 7f4fd624e740 1 --2- 172.21.15.2:0/1208555317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d6f670 0x563482d799f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009230 tx=0x7f4fbc005e00 comp rx=0 tx=0).stop 2024-09-10T08:23:32.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.368+0000 7f4fd624e740 1 -- 172.21.15.2:0/1208555317 shutdown_connections 2024-09-10T08:23:32.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.368+0000 7f4fd624e740 1 --2- 172.21.15.2:0/1208555317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d79f30 0x563482d7c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.368+0000 7f4fd624e740 1 --2- 172.21.15.2:0/1208555317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d6f670 0x563482d799f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.368+0000 7f4fd624e740 1 --2- 172.21.15.2:0/1208555317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d6f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.368+0000 7f4fd624e740 1 -- 172.21.15.2:0/1208555317 >> 172.21.15.2:0/1208555317 conn(0x563482d6b2e0 msgr2=0x563482d6b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:32.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.368+0000 7f4fd1800640 1 --2- 172.21.15.2:0/151249325 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4fb00a2c00 0x7f4fb00a50c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005fd0 tx=0x7f4fc4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:32.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.368+0000 7f4fd624e740 1 -- 172.21.15.2:0/1208555317 shutdown_connections 2024-09-10T08:23:32.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.369+0000 7f4fd624e740 1 -- 172.21.15.2:0/1208555317 wait complete. 2024-09-10T08:23:32.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.369+0000 7f4fd624e740 1 -- start start 2024-09-10T08:23:32.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.369+0000 7f4fd624e740 1 Processor -- start 2024-09-10T08:23:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.370+0000 7f4fd624e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6ed50 0x563482d76ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.370+0000 7f4fd624e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d6f670 0x563482d77400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.370+0000 7f4fd624e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d79f30 0x563482d77940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.370+0000 7f4fd624e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563482d7cba0 con 0x563482d6f670 2024-09-10T08:23:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.370+0000 7f4fd624e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563482c900f0 con 0x563482d79f30 2024-09-10T08:23:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.370+0000 7f4fd624e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563482d7cd20 con 0x563482d6ed50 2024-09-10T08:23:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.370+0000 7f4fd624e740 10 client.0 _ll_register_callbacks cb 0x563482c819e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.370+0000 7f4fd0e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6ed50 0x563482d76ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.370+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d6f670 0x563482d77400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.370+0000 7f4fd1800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d79f30 0x563482d77940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.370+0000 7f4fd0e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6ed50 0x563482d76ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:33376/0 (socket says 172.21.15.2:33376) 2024-09-10T08:23:32.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.370+0000 7f4fd0e00640 1 -- 172.21.15.2:0/928451593 learned_addr learned my addr 172.21.15.2:0/928451593 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.371+0000 7f4fd1800640 1 -- 172.21.15.2:0/928451593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6ed50 msgr2=0x563482d76ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.371+0000 7f4fd1800640 1 --2- 172.21.15.2:0/928451593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6ed50 0x563482d76ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.371+0000 7f4fd1800640 1 -- 172.21.15.2:0/928451593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d6f670 msgr2=0x563482d77400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.371+0000 7f4fd1800640 1 --2- 172.21.15.2:0/928451593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d6f670 0x563482d77400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.371+0000 7f4fd1800640 1 -- 172.21.15.2:0/928451593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563482a463c0 con 0x563482d79f30 2024-09-10T08:23:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.371+0000 7f4fd1800640 1 --2- 172.21.15.2:0/928451593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d79f30 0x563482d77940 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4011dd0 tx=0x7f4fc4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d35c96f97a0fa67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.371+0000 7f4fbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:23:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.371+0000 7f4fbaa00640 1 -- 172.21.15.2:0/928451593 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4004b20 con 0x563482d79f30 2024-09-10T08:23:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.371+0000 7f4fd624e740 1 -- 172.21.15.2:0/928451593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fbc004400 con 0x563482d79f30 2024-09-10T08:23:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.371+0000 7f4fbaa00640 1 -- 172.21.15.2:0/928451593 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4fc4004cc0 con 0x563482d79f30 2024-09-10T08:23:32.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.371+0000 7f4fbaa00640 1 -- 172.21.15.2:0/928451593 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4fc400f810 con 0x563482d79f30 2024-09-10T08:23:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.372+0000 7f4fd624e740 10 client.29837 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.372+0000 7f4fd624e740 1 -- 172.21.15.2:0/928451593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563482d889a0 con 0x563482d79f30 2024-09-10T08:23:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.372+0000 7f4fbaa00640 1 -- 172.21.15.2:0/928451593 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fbc004400 con 0x563482d79f30 2024-09-10T08:23:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.372+0000 7f4fbaa00640 1 -- 172.21.15.2:0/928451593 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4fc4026070 con 0x563482d79f30 2024-09-10T08:23:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.372+0000 7f4fd624e740 10 client.29837 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.372+0000 7f4fd624e740 10 client.29837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.372+0000 7f4fd624e740 20 client.29837 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.372+0000 7f4fd624e740 1 --2- 172.21.15.2:0/928451593 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563482d89db0 0x563482d8c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.372+0000 7f4fd624e740 4 client.29837 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:23:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.372+0000 7f4fd624e740 1 -- 172.21.15.2:0/928451593 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563482ce2950 con 0x563482d89db0 2024-09-10T08:23:32.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.373+0000 7f4fd0e00640 1 --2- 172.21.15.2:0/928451593 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563482d89db0 0x563482d8c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.373+0000 7f4fd0e00640 1 --2- 172.21.15.2:0/928451593 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563482d89db0 0x563482d8c250 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16da946b752242ab server_cookie=738620250739f870 in_seq=0 out_seq=0 2024-09-10T08:23:32.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.373+0000 7f4fbaa00640 10 client.29837 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:32.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.374+0000 7f4fbaa00640 1 -- 172.21.15.2:0/928451593 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563482ce2950 con 0x563482d89db0 2024-09-10T08:23:32.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.374+0000 7f4fbaa00640 10 client.29837 handle_command_reply: tid=0 2024-09-10T08:23:32.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.374+0000 7f4fd624e740 1 client.29837 shutdown 2024-09-10T08:23:32.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.374+0000 7f4fd624e740 1 -- 172.21.15.2:0/928451593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d79f30 msgr2=0x563482d77940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:32.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.374+0000 7f4fd624e740 1 --2- 172.21.15.2:0/928451593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d79f30 0x563482d77940 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4011dd0 tx=0x7f4fc4004ab0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.374+0000 7f4fd624e740 1 -- 172.21.15.2:0/928451593 shutdown_connections 2024-09-10T08:23:32.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.374+0000 7f4fd624e740 1 --2- 172.21.15.2:0/928451593 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563482d89db0 0x563482d8c250 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.374+0000 7f4fd624e740 1 --2- 172.21.15.2:0/928451593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d79f30 0x563482d77940 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.374+0000 7f4fd624e740 1 --2- 172.21.15.2:0/928451593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d6f670 0x563482d77400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.374+0000 7f4fbaa00640 0 client.29837 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:32.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.374+0000 7f4fd624e740 1 --2- 172.21.15.2:0/928451593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6ed50 0x563482d76ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.374+0000 7f4fd624e740 1 -- 172.21.15.2:0/928451593 >> 172.21.15.2:0/928451593 conn(0x563482d6b2e0 msgr2=0x563482d71010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:32.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.375+0000 7f4fd624e740 1 -- 172.21.15.2:0/928451593 shutdown_connections 2024-09-10T08:23:32.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.375+0000 7f4fd624e740 1 -- 172.21.15.2:0/928451593 wait complete. 2024-09-10T08:23:32.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.375+0000 7f4fd624e740 20 client.29837 trim_cache size 0 max 16384 2024-09-10T08:23:32.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.397+0000 7f4fd624e740 1 Processor -- start 2024-09-10T08:23:32.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.397+0000 7f4fd624e740 1 -- start start 2024-09-10T08:23:32.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.397+0000 7f4fd624e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d71800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.397+0000 7f4fd624e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6f670 0x563482d91b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.397+0000 7f4fd624e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d79f30 0x563482d920b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.397+0000 7f4fd624e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1b37f0 con 0x563482d79f30 2024-09-10T08:23:32.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.397+0000 7f4fd624e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1b10c0 con 0x563482d6ed50 2024-09-10T08:23:32.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.397+0000 7f4fd624e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1aef10 con 0x563482d6f670 2024-09-10T08:23:32.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.398+0000 7f4fd1800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d79f30 0x563482d920b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.398+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6f670 0x563482d91b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.398+0000 7f4fd0e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d71800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.398+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6f670 0x563482d91b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:33382/0 (socket says 172.21.15.2:33382) 2024-09-10T08:23:32.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.398+0000 7f4fd0e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d71800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60678/0 (socket says 172.21.15.2:60678) 2024-09-10T08:23:32.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.398+0000 7f4fc3e00640 1 -- 172.21.15.2:0/2753136637 learned_addr learned my addr 172.21.15.2:0/2753136637 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:32.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.398+0000 7f4fd0e00640 1 -- 172.21.15.2:0/2753136637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6f670 msgr2=0x563482d91b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:32.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.398+0000 7f4fd0e00640 1 --2- 172.21.15.2:0/2753136637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6f670 0x563482d91b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.398+0000 7f4fd0e00640 1 -- 172.21.15.2:0/2753136637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d79f30 msgr2=0x563482d920b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:32.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.398+0000 7f4fd0e00640 1 --2- 172.21.15.2:0/2753136637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d79f30 0x563482d920b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.398+0000 7f4fd0e00640 1 -- 172.21.15.2:0/2753136637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563482d6eb80 con 0x563482d6ed50 2024-09-10T08:23:32.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.398+0000 7f4fd1800640 1 --2- 172.21.15.2:0/2753136637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d79f30 0x563482d920b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:32.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.399+0000 7f4fd0e00640 1 --2- 172.21.15.2:0/2753136637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d71800 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc004cd0 tx=0x7f4fbc004bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=689cd347fd749e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:32.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.399+0000 7f4fc1600640 1 -- 172.21.15.2:0/2753136637 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc035040 con 0x563482d6ed50 2024-09-10T08:23:32.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.399+0000 7f4fc1600640 1 -- 172.21.15.2:0/2753136637 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fbc04a040 con 0x563482d6ed50 2024-09-10T08:23:32.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.399+0000 7f4fc1600640 1 -- 172.21.15.2:0/2753136637 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc031540 con 0x563482d6ed50 2024-09-10T08:23:32.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.400+0000 7f4fd624e740 1 -- 172.21.15.2:0/2753136637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 msgr2=0x563482d71800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:32.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.400+0000 7f4fd624e740 1 --2- 172.21.15.2:0/2753136637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d71800 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc004cd0 tx=0x7f4fbc004bd0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.400+0000 7f4fd624e740 1 -- 172.21.15.2:0/2753136637 shutdown_connections 2024-09-10T08:23:32.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.400+0000 7f4fd624e740 1 --2- 172.21.15.2:0/2753136637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d79f30 0x563482d920b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.400+0000 7f4fd624e740 1 --2- 172.21.15.2:0/2753136637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6f670 0x563482d91b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.400+0000 7f4fd624e740 1 --2- 172.21.15.2:0/2753136637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d71800 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.400+0000 7f4fd624e740 1 -- 172.21.15.2:0/2753136637 >> 172.21.15.2:0/2753136637 conn(0x563482d6b2e0 msgr2=0x563482d75790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:32.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.400+0000 7f4fd624e740 1 -- 172.21.15.2:0/2753136637 shutdown_connections 2024-09-10T08:23:32.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.400+0000 7f4fd624e740 1 -- 172.21.15.2:0/2753136637 wait complete. 2024-09-10T08:23:32.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.401+0000 7f4fd624e740 1 -- start start 2024-09-10T08:23:32.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.401+0000 7f4fd624e740 1 Processor -- start 2024-09-10T08:23:32.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.402+0000 7f4fd624e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d740d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.402+0000 7f4fd624e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6f670 0x563482d74610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.402+0000 7f4fd624e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d79f30 0x563482d74b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.402+0000 7f4fd624e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1af580 con 0x563482d79f30 2024-09-10T08:23:32.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.402+0000 7f4fd624e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1b09f0 con 0x563482d6ed50 2024-09-10T08:23:32.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.402+0000 7f4fd624e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1afc50 con 0x563482d6f670 2024-09-10T08:23:32.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.402+0000 7f4fd624e740 10 client.0 _ll_register_callbacks cb 0x7f4fc4003e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:32.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.402+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6f670 0x563482d74610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.402+0000 7f4fc3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6f670 0x563482d74610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:33384/0 (socket says 172.21.15.2:33384) 2024-09-10T08:23:32.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.402+0000 7f4fc3e00640 1 -- 172.21.15.2:0/3489476140 learned_addr learned my addr 172.21.15.2:0/3489476140 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:32.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.402+0000 7f4fd0e00640 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.403+0000 7f4fd1800640 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d79f30 0x563482d74b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.403+0000 7f4fc3e00640 1 -- 172.21.15.2:0/3489476140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 msgr2=0x563482d740d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:32.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.403+0000 7f4fc3e00640 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d740d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.403+0000 7f4fc3e00640 1 -- 172.21.15.2:0/3489476140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d79f30 msgr2=0x563482d74b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:32.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.403+0000 7f4fc3e00640 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d79f30 0x563482d74b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.403+0000 7f4fc3e00640 1 -- 172.21.15.2:0/3489476140 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563482df3d40 con 0x563482d6f670 2024-09-10T08:23:32.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.403+0000 7f4fd1800640 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d79f30 0x563482d74b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:23:32.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.403+0000 7f4fc3e00640 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6f670 0x563482d74610 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400c6a0 tx=0x7f4fb409bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b906afabe7d696c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:32.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.403+0000 7f4fbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:23:32.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.404+0000 7f4fbaa00640 1 -- 172.21.15.2:0/3489476140 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40cfbb0 con 0x563482d6f670 2024-09-10T08:23:32.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.404+0000 7f4fbaa00640 1 -- 172.21.15.2:0/3489476140 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f4fb409b0d0 con 0x563482d6f670 2024-09-10T08:23:32.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.404+0000 7f4fd624e740 1 -- 172.21.15.2:0/3489476140 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fbc04a040 con 0x563482d6f670 2024-09-10T08:23:32.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.405+0000 7f4fbaa00640 1 -- 172.21.15.2:0/3489476140 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4fb409b2d0 con 0x563482d6f670 2024-09-10T08:23:32.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.405+0000 7f4fd624e740 10 client.29767 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:32.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.405+0000 7f4fd624e740 1 -- 172.21.15.2:0/3489476140 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563482df5090 con 0x563482d6f670 2024-09-10T08:23:32.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.405+0000 7f4fbaa00640 1 -- 172.21.15.2:0/3489476140 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fbc04a040 con 0x563482d6f670 2024-09-10T08:23:32.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.405+0000 7f4fbaa00640 1 -- 172.21.15.2:0/3489476140 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4fb409abd0 con 0x563482d6f670 2024-09-10T08:23:32.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.405+0000 7f4fd624e740 10 client.29767 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:32.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.405+0000 7f4fd624e740 10 client.29767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:32.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.405+0000 7f4fd624e740 20 client.29767 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:32.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.405+0000 7f4fd624e740 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563482df5790 0x563482df7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:32.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.405+0000 7f4fd624e740 4 client.29767 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:23:32.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.405+0000 7f4fd624e740 1 -- 172.21.15.2:0/3489476140 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563482cdce60 con 0x563482df5790 2024-09-10T08:23:32.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.406+0000 7f4fd0e00640 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563482df5790 0x563482df7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:32.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.406+0000 7f4fd0e00640 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563482df5790 0x563482df7bb0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=543264169fc9cef7 server_cookie=2dd2a95a8b8ef5f6 in_seq=0 out_seq=0 2024-09-10T08:23:32.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.406+0000 7f4fbaa00640 10 client.29767 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:32.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fbaa00640 1 -- 172.21.15.2:0/3489476140 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563482cdce60 con 0x563482df5790 2024-09-10T08:23:32.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fbaa00640 10 client.29767 handle_command_reply: tid=0 2024-09-10T08:23:32.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fd624e740 1 client.29767 shutdown 2024-09-10T08:23:32.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fd624e740 1 -- 172.21.15.2:0/3489476140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6f670 msgr2=0x563482d74610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:32.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fd624e740 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6f670 0x563482d74610 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400c6a0 tx=0x7f4fb409bc80 comp rx=0 tx=0).stop 2024-09-10T08:23:32.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fd624e740 1 -- 172.21.15.2:0/3489476140 shutdown_connections 2024-09-10T08:23:32.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fd624e740 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x563482df5790 0x563482df7bb0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fd624e740 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563482d79f30 0x563482d74b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fd624e740 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482d6f670 0x563482d74610 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fd624e740 1 --2- 172.21.15.2:0/3489476140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563482d6ed50 0x563482d740d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fbaa00640 0 client.29767 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:32.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fd624e740 1 -- 172.21.15.2:0/3489476140 >> 172.21.15.2:0/3489476140 conn(0x563482d6b2e0 msgr2=0x563482d76e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:32.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fd624e740 1 -- 172.21.15.2:0/3489476140 shutdown_connections 2024-09-10T08:23:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.407+0000 7f4fd624e740 1 -- 172.21.15.2:0/3489476140 wait complete. 2024-09-10T08:23:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.408+0000 7f4fd624e740 20 client.29767 trim_cache size 0 max 16384 2024-09-10T08:23:32.410 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:23:32.410 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:23:32.410 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-09-10T08:23:32.410 INFO:teuthology.orchestra.run.smithi002.stdout: "064ae0d5-6f5e-4292-a292-5642f9fffd24": { 2024-09-10T08:23:32.410 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-09-10T08:23:32.410 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "064ae0d5-6f5e-4292-a292-5642f9fffd24", 2024-09-10T08:23:32.410 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-09-10T08:23:32.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:32.411 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:32.411 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:23:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.408+0000 7f4fd2200640 1 -- 172.21.15.2:0/151249325 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4fb00a2c00 msgr2=0x7f4fb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.408+0000 7f4fd2200640 1 --2- 172.21.15.2:0/151249325 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4fb00a2c00 0x7f4fb00a50c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005fd0 tx=0x7f4fc4009500 comp rx=0 tx=0).stop 2024-09-10T08:23:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.408+0000 7f4fd2200640 1 -- 172.21.15.2:0/151249325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fcc102360 msgr2=0x7f4fcc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.408+0000 7f4fd2200640 1 --2- 172.21.15.2:0/151249325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fcc102360 0x7f4fcc074a30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400e9f0 tx=0x7f4fb400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.408+0000 7f4fd2200640 1 -- 172.21.15.2:0/151249325 shutdown_connections 2024-09-10T08:23:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.408+0000 7f4fd2200640 1 --2- 172.21.15.2:0/151249325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fcc1079c0 0x7f4fcc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.408+0000 7f4fd2200640 1 --2- 172.21.15.2:0/151249325 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f4fb00a2c00 0x7f4fb00a50c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.408+0000 7f4fd2200640 1 --2- 172.21.15.2:0/151249325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fcc102360 0x7f4fcc074a30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.409+0000 7f4fd2200640 1 --2- 172.21.15.2:0/151249325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fcc1019c0 0x7f4fcc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.409+0000 7f4fd2200640 1 -- 172.21.15.2:0/151249325 >> 172.21.15.2:0/151249325 conn(0x7f4fcc0fd740 msgr2=0x7f4fcc0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.409+0000 7f4fd2200640 1 -- 172.21.15.2:0/151249325 shutdown_connections 2024-09-10T08:23:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:32.409+0000 7f4fd2200640 1 -- 172.21.15.2:0/151249325 wait complete. 2024-09-10T08:23:32.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:32 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3832235712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:23:32.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:32 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3832235712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:23:32.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:32 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3832235712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:23:33.409 INFO:tasks.cephfs.filesystem:scrub status for tag:064ae0d5-6f5e-4292-a292-5642f9fffd24 - {'path': '/', 'tag': '064ae0d5-6f5e-4292-a292-5642f9fffd24', 'options': 'recursive,force'} 2024-09-10T08:23:33.742 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:33 smithi002 ceph-mon[30563]: pgmap v849: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:23:33.780 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0' 2024-09-10T08:23:33.780 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0 2024-09-10T08:23:33.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:33 smithi155 ceph-mon[35405]: pgmap v849: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:23:33.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:33 smithi107 ceph-mon[35048]: pgmap v849: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:23:34.160 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:23:34.765 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:34 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:23:34.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.890+0000 7f9af3e00640 1 -- 172.21.15.2:0/2526981042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9af4108f60 msgr2=0x7f9af4110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:34.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.890+0000 7f9af3e00640 1 --2- 172.21.15.2:0/2526981042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9af4108f60 0x7f9af4110ae0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009ca0 tx=0x7f9aec02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.890+0000 7f9af3e00640 1 -- 172.21.15.2:0/2526981042 shutdown_connections 2024-09-10T08:23:34.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.890+0000 7f9af3e00640 1 --2- 172.21.15.2:0/2526981042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9af4108f60 0x7f9af4110ae0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.890+0000 7f9af3e00640 1 --2- 172.21.15.2:0/2526981042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9af41047e0 0x7f9af4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.890+0000 7f9af3e00640 1 --2- 172.21.15.2:0/2526981042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9af4103e30 0x7f9af4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.890+0000 7f9af3e00640 1 -- 172.21.15.2:0/2526981042 >> 172.21.15.2:0/2526981042 conn(0x7f9af40ff910 msgr2=0x7f9af4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:34.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.891+0000 7f9af3e00640 1 -- 172.21.15.2:0/2526981042 shutdown_connections 2024-09-10T08:23:34.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.891+0000 7f9af3e00640 1 -- 172.21.15.2:0/2526981042 wait complete. 2024-09-10T08:23:34.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.891+0000 7f9af3e00640 1 Processor -- start 2024-09-10T08:23:34.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.892+0000 7f9af3e00640 1 -- start start 2024-09-10T08:23:34.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.892+0000 7f9af3e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9af4103e30 0x7f9af419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.892+0000 7f9af3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9af41047e0 0x7f9af419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.892+0000 7f9af3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9af4108f60 0x7f9af419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.892+0000 7f9af3e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4113fc0 con 0x7f9af4103e30 2024-09-10T08:23:34.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.892+0000 7f9af3e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4113e40 con 0x7f9af4108f60 2024-09-10T08:23:34.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.892+0000 7f9af3e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4114140 con 0x7f9af41047e0 2024-09-10T08:23:34.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.892+0000 7f9af2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9af4103e30 0x7f9af419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.892+0000 7f9af2a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9af4103e30 0x7f9af419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49784/0 (socket says 172.21.15.2:49784) 2024-09-10T08:23:34.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.892+0000 7f9af2a00640 1 -- 172.21.15.2:0/186983473 learned_addr learned my addr 172.21.15.2:0/186983473 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:34.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.892+0000 7f9af3400640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9af4108f60 0x7f9af419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.892+0000 7f9aea000640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9af41047e0 0x7f9af419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.893+0000 7f9af2a00640 1 -- 172.21.15.2:0/186983473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9af41047e0 msgr2=0x7f9af419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:34.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.893+0000 7f9af2a00640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9af41047e0 0x7f9af419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.893+0000 7f9af2a00640 1 -- 172.21.15.2:0/186983473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9af4108f60 msgr2=0x7f9af419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:34.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.893+0000 7f9af2a00640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9af4108f60 0x7f9af419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.893+0000 7f9af2a00640 1 -- 172.21.15.2:0/186983473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af419f410 con 0x7f9af4103e30 2024-09-10T08:23:34.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.893+0000 7f9af3400640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9af4108f60 0x7f9af419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:34.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.893+0000 7f9af2a00640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9af4103e30 0x7f9af419a460 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400e9f0 tx=0x7f9ae400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:34.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.893+0000 7f9aebe00640 1 -- 172.21.15.2:0/186983473 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae400cd80 con 0x7f9af4103e30 2024-09-10T08:23:34.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.894+0000 7f9aebe00640 1 -- 172.21.15.2:0/186983473 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae4005500 con 0x7f9af4103e30 2024-09-10T08:23:34.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.894+0000 7f9aebe00640 1 -- 172.21.15.2:0/186983473 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4010690 con 0x7f9af4103e30 2024-09-10T08:23:34.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.894+0000 7f9af3e00640 1 -- 172.21.15.2:0/186983473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af419f700 con 0x7f9af4103e30 2024-09-10T08:23:34.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.894+0000 7f9af3e00640 1 -- 172.21.15.2:0/186983473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af41a6fe0 con 0x7f9af4103e30 2024-09-10T08:23:34.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.897+0000 7f9afc2ce740 1 Processor -- start 2024-09-10T08:23:34.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.897+0000 7f9aebe00640 1 -- 172.21.15.2:0/186983473 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9ae40046c0 con 0x7f9af4103e30 2024-09-10T08:23:34.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.897+0000 7f9afc2ce740 1 -- start start 2024-09-10T08:23:34.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.897+0000 7f9afc2ce740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b4eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.897+0000 7f9afc2ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b59360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.897+0000 7f9afc2ce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b598a0 0x55d2b0b5bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.897+0000 7f9afc2ce740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d2b0ab3260 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.897+0000 7f9afc2ce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d2b0a5a500 con 0x55d2b0b598a0 2024-09-10T08:23:34.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.897+0000 7f9afc2ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d2b0aaf220 con 0x55d2b0b4efe0 2024-09-10T08:23:34.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.898+0000 7f9af2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b59360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.898+0000 7f9aea000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b598a0 0x55d2b0b5bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.898+0000 7f9aea000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b598a0 0x55d2b0b5bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60716/0 (socket says 172.21.15.2:60716) 2024-09-10T08:23:34.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.898+0000 7f9aea000640 1 -- 172.21.15.2:0/1682020576 learned_addr learned my addr 172.21.15.2:0/1682020576 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:34.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.898+0000 7f9af2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b59360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:33406/0 (socket says 172.21.15.2:33406) 2024-09-10T08:23:34.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.898+0000 7f9af3400640 1 --2- 172.21.15.2:0/1682020576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b4eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.898+0000 7f9aea000640 1 -- 172.21.15.2:0/1682020576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 msgr2=0x55d2b0b59360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:34.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.898+0000 7f9aea000640 1 --2- 172.21.15.2:0/1682020576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b59360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.898+0000 7f9aea000640 1 -- 172.21.15.2:0/1682020576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b4e6c0 msgr2=0x55d2b0b4eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:34.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.898+0000 7f9aea000640 1 --2- 172.21.15.2:0/1682020576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b4eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.898+0000 7f9aea000640 1 -- 172.21.15.2:0/1682020576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2b0824ba0 con 0x55d2b0b598a0 2024-09-10T08:23:34.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.898+0000 7f9af2a00640 1 --2- 172.21.15.2:0/1682020576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b59360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:34.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.899+0000 7f9aea000640 1 --2- 172.21.15.2:0/1682020576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b598a0 0x55d2b0b5bc90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009c10 tx=0x7f9adc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a241c074904f30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:34.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.899+0000 7f9ae3e00640 1 -- 172.21.15.2:0/1682020576 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc004230 con 0x55d2b0b598a0 2024-09-10T08:23:34.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.899+0000 7f9aebe00640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9ad80a2c00 0x7f9ad80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.899+0000 7f9ae3e00640 1 -- 172.21.15.2:0/1682020576 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc0043d0 con 0x55d2b0b598a0 2024-09-10T08:23:34.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.899+0000 7f9ae3e00640 1 -- 172.21.15.2:0/1682020576 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc004e10 con 0x55d2b0b598a0 2024-09-10T08:23:34.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.899+0000 7f9af3400640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9ad80a2c00 0x7f9ad80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.899+0000 7f9aebe00640 1 -- 172.21.15.2:0/186983473 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9ae40cf810 con 0x7f9af4103e30 2024-09-10T08:23:34.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.900+0000 7f9afc2ce740 1 -- 172.21.15.2:0/1682020576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b598a0 msgr2=0x55d2b0b5bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:34.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.900+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/1682020576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b598a0 0x55d2b0b5bc90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009c10 tx=0x7f9adc011810 comp rx=0 tx=0).stop 2024-09-10T08:23:34.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.900+0000 7f9afc2ce740 1 -- 172.21.15.2:0/1682020576 shutdown_connections 2024-09-10T08:23:34.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.900+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/1682020576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b598a0 0x55d2b0b5bc90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.900+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/1682020576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b59360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.900+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/1682020576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b4eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.900+0000 7f9afc2ce740 1 -- 172.21.15.2:0/1682020576 >> 172.21.15.2:0/1682020576 conn(0x55d2b0b4ac50 msgr2=0x55d2b0b4b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:34.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.900+0000 7f9af3400640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9ad80a2c00 0x7f9ad80a50c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9aec031f80 tx=0x7f9aec004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:34.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.900+0000 7f9afc2ce740 1 -- 172.21.15.2:0/1682020576 shutdown_connections 2024-09-10T08:23:34.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.900+0000 7f9afc2ce740 1 -- 172.21.15.2:0/1682020576 wait complete. 2024-09-10T08:23:34.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.901+0000 7f9afc2ce740 1 -- start start 2024-09-10T08:23:34.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.901+0000 7f9afc2ce740 1 Processor -- start 2024-09-10T08:23:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.901+0000 7f9afc2ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.901+0000 7f9afc2ce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b56c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.901+0000 7f9afc2ce740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b57160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.901+0000 7f9afc2ce740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d2b0b5c470 con 0x55d2b0b598a0 2024-09-10T08:23:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.901+0000 7f9afc2ce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d2b0a6dbd0 con 0x55d2b0b4efe0 2024-09-10T08:23:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.901+0000 7f9afc2ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d2b0b5c5f0 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.902+0000 7f9afc2ce740 10 client.0 _ll_register_callbacks cb 0x55d2b07c20a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.902+0000 7f9aea000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b56c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.902+0000 7f9aea000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b56c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60732/0 (socket says 172.21.15.2:60732) 2024-09-10T08:23:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.902+0000 7f9aea000640 1 -- 172.21.15.2:0/2334753465 learned_addr learned my addr 172.21.15.2:0/2334753465 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.902+0000 7f9af3400640 1 --2- 172.21.15.2:0/2334753465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b57160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.902+0000 7f9af2a00640 1 --2- 172.21.15.2:0/2334753465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.902+0000 7f9aea000640 1 -- 172.21.15.2:0/2334753465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4e6c0 msgr2=0x55d2b0b566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.902+0000 7f9aea000640 1 --2- 172.21.15.2:0/2334753465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.902+0000 7f9aea000640 1 -- 172.21.15.2:0/2334753465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 msgr2=0x55d2b0b57160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.902+0000 7f9aea000640 1 --2- 172.21.15.2:0/2334753465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b57160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.902+0000 7f9aea000640 1 -- 172.21.15.2:0/2334753465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2b0b65910 con 0x55d2b0b4efe0 2024-09-10T08:23:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.903+0000 7f9aea000640 1 --2- 172.21.15.2:0/2334753465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b56c20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0050b0 tx=0x7f9adc0050e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac289166864e05a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.903+0000 7f9ae0c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:23:34.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.903+0000 7f9ae0c00640 1 -- 172.21.15.2:0/2334753465 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc024620 con 0x55d2b0b4efe0 2024-09-10T08:23:34.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.903+0000 7f9afc2ce740 1 -- 172.21.15.2:0/2334753465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9adc0043d0 con 0x55d2b0b4efe0 2024-09-10T08:23:34.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.903+0000 7f9ae0c00640 1 -- 172.21.15.2:0/2334753465 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9adc0247c0 con 0x55d2b0b4efe0 2024-09-10T08:23:34.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.903+0000 7f9ae0c00640 1 -- 172.21.15.2:0/2334753465 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9adc032050 con 0x55d2b0b4efe0 2024-09-10T08:23:34.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.903+0000 7f9afc2ce740 10 client.29852 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:34.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.903+0000 7f9afc2ce740 1 -- 172.21.15.2:0/2334753465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2b0b681e0 con 0x55d2b0b4efe0 2024-09-10T08:23:34.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.904+0000 7f9ae0c00640 1 -- 172.21.15.2:0/2334753465 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc0043d0 con 0x55d2b0b4efe0 2024-09-10T08:23:34.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.904+0000 7f9ae0c00640 1 -- 172.21.15.2:0/2334753465 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9adc018070 con 0x55d2b0b4efe0 2024-09-10T08:23:34.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.904+0000 7f9afc2ce740 10 client.29852 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:34.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.904+0000 7f9afc2ce740 10 client.29852 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:23:34.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.904+0000 7f9afc2ce740 20 client.29852 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:34.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.904+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/2334753465 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d2b0b695f0 0x55d2b0b6b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.904+0000 7f9afc2ce740 4 client.29852 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:23:34.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.904+0000 7f9afc2ce740 1 -- 172.21.15.2:0/2334753465 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2b0ac0990 con 0x55d2b0b695f0 2024-09-10T08:23:34.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.904+0000 7f9af2a00640 1 --2- 172.21.15.2:0/2334753465 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d2b0b695f0 0x55d2b0b6b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.905+0000 7f9af2a00640 1 --2- 172.21.15.2:0/2334753465 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d2b0b695f0 0x55d2b0b6b9e0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1b8acb81fc77ccc server_cookie=e463489c52de8d7f in_seq=0 out_seq=0 2024-09-10T08:23:34.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.905+0000 7f9ae0c00640 10 client.29852 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:34.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9ae0c00640 1 -- 172.21.15.2:0/2334753465 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2b0ac0990 con 0x55d2b0b695f0 2024-09-10T08:23:34.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9ae0c00640 10 client.29852 handle_command_reply: tid=0 2024-09-10T08:23:34.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9afc2ce740 1 client.29852 shutdown 2024-09-10T08:23:34.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9afc2ce740 1 -- 172.21.15.2:0/2334753465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4efe0 msgr2=0x55d2b0b56c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:34.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/2334753465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b56c20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0050b0 tx=0x7f9adc0050e0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9afc2ce740 1 -- 172.21.15.2:0/2334753465 shutdown_connections 2024-09-10T08:23:34.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/2334753465 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d2b0b695f0 0x55d2b0b6b9e0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/2334753465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b57160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/2334753465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b56c20 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/2334753465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9afc2ce740 1 -- 172.21.15.2:0/2334753465 >> 172.21.15.2:0/2334753465 conn(0x55d2b0b4ac50 msgr2=0x55d2b0b49e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:34.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9ae0c00640 0 client.29852 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:34.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9afc2ce740 1 -- 172.21.15.2:0/2334753465 shutdown_connections 2024-09-10T08:23:34.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9afc2ce740 1 -- 172.21.15.2:0/2334753465 wait complete. 2024-09-10T08:23:34.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.906+0000 7f9afc2ce740 20 client.29852 trim_cache size 0 max 16384 2024-09-10T08:23:34.910 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:34 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:23:34.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:34 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:23:34.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.924+0000 7f9afc2ce740 1 Processor -- start 2024-09-10T08:23:34.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.924+0000 7f9afc2ce740 1 -- start start 2024-09-10T08:23:34.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.925+0000 7f9afc2ce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b55960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.925+0000 7f9afc2ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.925+0000 7f9afc2ce740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b71730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.925+0000 7f9afc2ce740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41b3780 con 0x55d2b0b598a0 2024-09-10T08:23:34.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.925+0000 7f9afc2ce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41b1050 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.925+0000 7f9afc2ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41aeea0 con 0x55d2b0b4efe0 2024-09-10T08:23:34.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.925+0000 7f9af3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b71730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.925+0000 7f9af2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b55960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.925+0000 7f9af3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b71730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49810/0 (socket says 172.21.15.2:49810) 2024-09-10T08:23:34.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.925+0000 7f9af3400640 1 -- 172.21.15.2:0/1264482878 learned_addr learned my addr 172.21.15.2:0/1264482878 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:34.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.925+0000 7f9aea000640 1 --2- 172.21.15.2:0/1264482878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.925+0000 7f9af2a00640 1 -- 172.21.15.2:0/1264482878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 msgr2=0x55d2b0b711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:34.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.925+0000 7f9af2a00640 1 --2- 172.21.15.2:0/1264482878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.926+0000 7f9af2a00640 1 -- 172.21.15.2:0/1264482878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 msgr2=0x55d2b0b71730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:34.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.926+0000 7f9af2a00640 1 --2- 172.21.15.2:0/1264482878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b71730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.926+0000 7f9af2a00640 1 -- 172.21.15.2:0/1264482878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2b0b4aa80 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.926+0000 7f9aea000640 1 --2- 172.21.15.2:0/1264482878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:23:34.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.926+0000 7f9af2a00640 1 --2- 172.21.15.2:0/1264482878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b55960 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55d2b0b4bab0 tx=0x7f9ae40ccd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f595517865fabd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:34.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.926+0000 7f9ae3400640 1 -- 172.21.15.2:0/1264482878 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae409b760 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.926+0000 7f9ae3400640 1 -- 172.21.15.2:0/1264482878 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae409c320 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.926+0000 7f9ae3400640 1 -- 172.21.15.2:0/1264482878 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae409b580 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.927+0000 7f9afc2ce740 1 -- 172.21.15.2:0/1264482878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4e6c0 msgr2=0x55d2b0b55960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:34.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.927+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/1264482878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b55960 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55d2b0b4bab0 tx=0x7f9ae40ccd90 comp rx=0 tx=0).stop 2024-09-10T08:23:34.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.927+0000 7f9afc2ce740 1 -- 172.21.15.2:0/1264482878 shutdown_connections 2024-09-10T08:23:34.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.927+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/1264482878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b71730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.927+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/1264482878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.927+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/1264482878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b55960 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.927+0000 7f9afc2ce740 1 -- 172.21.15.2:0/1264482878 >> 172.21.15.2:0/1264482878 conn(0x55d2b0b4ac50 msgr2=0x55d2b0b49e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:34.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.927+0000 7f9afc2ce740 1 -- 172.21.15.2:0/1264482878 shutdown_connections 2024-09-10T08:23:34.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.927+0000 7f9afc2ce740 1 -- 172.21.15.2:0/1264482878 wait complete. 2024-09-10T08:23:34.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.927+0000 7f9afc2ce740 1 -- start start 2024-09-10T08:23:34.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.928+0000 7f9afc2ce740 1 Processor -- start 2024-09-10T08:23:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.928+0000 7f9afc2ce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b61e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.928+0000 7f9afc2ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b53220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.928+0000 7f9afc2ce740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b53760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.928+0000 7f9afc2ce740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41af510 con 0x55d2b0b598a0 2024-09-10T08:23:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.928+0000 7f9afc2ce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41b0980 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.928+0000 7f9afc2ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41afbe0 con 0x55d2b0b4efe0 2024-09-10T08:23:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.928+0000 7f9afc2ce740 10 client.0 _ll_register_callbacks cb 0x7f9ae4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.928+0000 7f9af2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b61e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.928+0000 7f9af2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b61e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:60740/0 (socket says 172.21.15.2:60740) 2024-09-10T08:23:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.928+0000 7f9af2a00640 1 -- 172.21.15.2:0/4096670729 learned_addr learned my addr 172.21.15.2:0/4096670729 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.928+0000 7f9af3400640 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b53760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.929+0000 7f9aea000640 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b53220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.929+0000 7f9af2a00640 1 -- 172.21.15.2:0/4096670729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 msgr2=0x55d2b0b53220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.929+0000 7f9af2a00640 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b53220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.929+0000 7f9af2a00640 1 -- 172.21.15.2:0/4096670729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 msgr2=0x55d2b0b53760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.929+0000 7f9af2a00640 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b53760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.929+0000 7f9af2a00640 1 -- 172.21.15.2:0/4096670729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2b0b53da0 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.929+0000 7f9af3400640 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b53760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:23:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.929+0000 7f9af2a00640 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b61e60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40d1010 tx=0x7f9ae40d5530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b53ed4738ae7a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.929+0000 7f9ae0c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:23:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.929+0000 7f9ae0c00640 1 -- 172.21.15.2:0/4096670729 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae409b580 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.929+0000 7f9ae0c00640 1 -- 172.21.15.2:0/4096670729 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9ae4098290 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.929+0000 7f9afc2ce740 1 -- 172.21.15.2:0/4096670729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae409c320 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.930+0000 7f9ae0c00640 1 -- 172.21.15.2:0/4096670729 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ae409b760 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.930+0000 7f9afc2ce740 10 client.29858 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.930+0000 7f9afc2ce740 1 -- 172.21.15.2:0/4096670729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2b0bd3fd0 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.930+0000 7f9ae0c00640 1 -- 172.21.15.2:0/4096670729 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae409c320 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.930+0000 7f9ae0c00640 1 -- 172.21.15.2:0/4096670729 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9ae409b100 con 0x55d2b0b4e6c0 2024-09-10T08:23:34.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.930+0000 7f9afc2ce740 10 client.29858 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:34.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.930+0000 7f9afc2ce740 10 client.29858 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:23:34.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.930+0000 7f9afc2ce740 20 client.29858 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:34.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.930+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d2b0bd46b0 0x55d2b0bd6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:34.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.930+0000 7f9afc2ce740 4 client.29858 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}] 2024-09-10T08:23:34.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.930+0000 7f9afc2ce740 1 -- 172.21.15.2:0/4096670729 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}) -- 0x55d2b0abace0 con 0x55d2b0bd46b0 2024-09-10T08:23:34.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.931+0000 7f9aea000640 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d2b0bd46b0 0x55d2b0bd6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:34.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.931+0000 7f9aea000640 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d2b0bd46b0 0x55d2b0bd6ad0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2be60a7d6815139 server_cookie=f4f9c3b31f5af829 in_seq=0 out_seq=0 2024-09-10T08:23:34.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.931+0000 7f9ae0c00640 10 client.29858 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:34.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.941+0000 7f9ae0c00640 1 -- 172.21.15.2:0/4096670729 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d2b0abace0 con 0x55d2b0bd46b0 2024-09-10T08:23:34.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.941+0000 7f9ae0c00640 10 client.29858 handle_command_reply: tid=0 2024-09-10T08:23:34.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.941+0000 7f9afc2ce740 1 client.29858 shutdown 2024-09-10T08:23:34.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.941+0000 7f9afc2ce740 1 -- 172.21.15.2:0/4096670729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4e6c0 msgr2=0x55d2b0b61e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:34.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.941+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b61e60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40d1010 tx=0x7f9ae40d5530 comp rx=0 tx=0).stop 2024-09-10T08:23:34.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.942+0000 7f9afc2ce740 1 -- 172.21.15.2:0/4096670729 shutdown_connections 2024-09-10T08:23:34.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.942+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55d2b0bd46b0 0x55d2b0bd6ad0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.942+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2b0b598a0 0x55d2b0b53760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.942+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2b0b4efe0 0x55d2b0b53220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.942+0000 7f9afc2ce740 1 --2- 172.21.15.2:0/4096670729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2b0b4e6c0 0x55d2b0b61e60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.942+0000 7f9ae0c00640 0 client.29858 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:34.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.942+0000 7f9afc2ce740 1 -- 172.21.15.2:0/4096670729 >> 172.21.15.2:0/4096670729 conn(0x55d2b0b4ac50 msgr2=0x55d2b0b50110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:34.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.942+0000 7f9afc2ce740 1 -- 172.21.15.2:0/4096670729 shutdown_connections 2024-09-10T08:23:34.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.942+0000 7f9afc2ce740 1 -- 172.21.15.2:0/4096670729 wait complete. 2024-09-10T08:23:34.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.942+0000 7f9afc2ce740 20 client.29858 trim_cache size 0 max 16384 2024-09-10T08:23:34.946 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:23:34.946 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:23:34.946 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:23:34.946 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 54, 2024-09-10T08:23:34.946 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:23:34.947 INFO:teuthology.orchestra.run.smithi002.stdout: "b597e0bd": { 2024-09-10T08:23:34.947 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 54, 2024-09-10T08:23:34.947 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:23:34.947 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:23:34.947 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:23:34.947 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:23:34.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:23:34.948 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:23:34.948 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 139.8, 2024-09-10T08:23:34.948 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:23:34.948 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:23:34.948 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:23:34.949 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:23:34.949 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:23:34.949 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:23:34.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:34.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:34.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:34.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:34.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:34.950 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:23:34.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.943+0000 7f9af3e00640 1 -- 172.21.15.2:0/186983473 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9ad80a2c00 msgr2=0x7f9ad80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:34.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.943+0000 7f9af3e00640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9ad80a2c00 0x7f9ad80a50c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9aec031f80 tx=0x7f9aec004a60 comp rx=0 tx=0).stop 2024-09-10T08:23:34.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.943+0000 7f9af3e00640 1 -- 172.21.15.2:0/186983473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9af4103e30 msgr2=0x7f9af419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:34.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.943+0000 7f9af3e00640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9af4103e30 0x7f9af419a460 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400e9f0 tx=0x7f9ae400edf0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.943+0000 7f9af3e00640 1 -- 172.21.15.2:0/186983473 shutdown_connections 2024-09-10T08:23:34.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.943+0000 7f9af3e00640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9af4108f60 0x7f9af419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.943+0000 7f9af3e00640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9ad80a2c00 0x7f9ad80a50c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.943+0000 7f9af3e00640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9af41047e0 0x7f9af419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.943+0000 7f9af3e00640 1 --2- 172.21.15.2:0/186983473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9af4103e30 0x7f9af419a460 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:34.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.943+0000 7f9af3e00640 1 -- 172.21.15.2:0/186983473 >> 172.21.15.2:0/186983473 conn(0x7f9af40ff910 msgr2=0x7f9af4100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:34.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.943+0000 7f9af3e00640 1 -- 172.21.15.2:0/186983473 shutdown_connections 2024-09-10T08:23:34.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:34.943+0000 7f9af3e00640 1 -- 172.21.15.2:0/186983473 wait complete. 2024-09-10T08:23:35.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:35 smithi002 ceph-mon[30563]: pgmap v850: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:23:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:35 smithi155 ceph-mon[35405]: pgmap v850: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:23:35.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:35 smithi107 ceph-mon[35048]: pgmap v850: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-10T08:23:35.951 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b597e0bd', quiesce took 5.9 seconds. Will release after: 13.950000000000001 2024-09-10T08:23:37.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:37 smithi155 ceph-mon[35405]: pgmap v851: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:23:37.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:37 smithi107 ceph-mon[35048]: pgmap v851: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:23:38.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:37 smithi002 ceph-mon[30563]: pgmap v851: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-10T08:23:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.149+0000 7f59bfa00640 1 -- 172.21.15.2:0/123312722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59b8103d50 msgr2=0x7f59b8104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.149+0000 7f59bfa00640 1 --2- 172.21.15.2:0/123312722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59b8103d50 0x7f59b8104150 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009900 tx=0x7f59a802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.149+0000 7f59bfa00640 1 -- 172.21.15.2:0/123312722 shutdown_connections 2024-09-10T08:23:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.149+0000 7f59bfa00640 1 --2- 172.21.15.2:0/123312722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b8105900 0x7f59b810c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.149+0000 7f59bfa00640 1 --2- 172.21.15.2:0/123312722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59b8104f40 0x7f59b81053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.149+0000 7f59bfa00640 1 --2- 172.21.15.2:0/123312722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59b8103d50 0x7f59b8104150 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.149+0000 7f59bfa00640 1 -- 172.21.15.2:0/123312722 >> 172.21.15.2:0/123312722 conn(0x7f59b80ff4b0 msgr2=0x7f59b81018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.149+0000 7f59bfa00640 1 -- 172.21.15.2:0/123312722 shutdown_connections 2024-09-10T08:23:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.149+0000 7f59bfa00640 1 -- 172.21.15.2:0/123312722 wait complete. 2024-09-10T08:23:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.150+0000 7f59bfa00640 1 Processor -- start 2024-09-10T08:23:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.150+0000 7f59bfa00640 1 -- start start 2024-09-10T08:23:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.150+0000 7f59bfa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59b8103d50 0x7f59b8199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59bfa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b8104f40 0x7f59b819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59bfa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59b8105900 0x7f59b81a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59bfa00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f59b810fa20 con 0x7f59b8105900 2024-09-10T08:23:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59bfa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f59b810f8a0 con 0x7f59b8103d50 2024-09-10T08:23:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59bfa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59b810fba0 con 0x7f59b8104f40 2024-09-10T08:23:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59bda00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59b8105900 0x7f59b81a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59bd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59b8103d50 0x7f59b8199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59bda00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59b8105900 0x7f59b81a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40050/0 (socket says 172.21.15.2:40050) 2024-09-10T08:23:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59bda00640 1 -- 172.21.15.2:0/3277802906 learned_addr learned my addr 172.21.15.2:0/3277802906 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59afe00640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b8104f40 0x7f59b819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59afe00640 1 -- 172.21.15.2:0/3277802906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59b8103d50 msgr2=0x7f59b8199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59afe00640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59b8103d50 0x7f59b8199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59afe00640 1 -- 172.21.15.2:0/3277802906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59b8105900 msgr2=0x7f59b81a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59afe00640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59b8105900 0x7f59b81a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59afe00640 1 -- 172.21.15.2:0/3277802906 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b81a1bd0 con 0x7f59b8104f40 2024-09-10T08:23:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.151+0000 7f59bda00640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59b8105900 0x7f59b81a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.152+0000 7f59afe00640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b8104f40 0x7f59b819a4f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f59a000e9f0 tx=0x7f59a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.152+0000 7f59b6000640 1 -- 172.21.15.2:0/3277802906 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a000ccc0 con 0x7f59b8104f40 2024-09-10T08:23:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.152+0000 7f59bfa00640 1 -- 172.21.15.2:0/3277802906 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b81a1ec0 con 0x7f59b8104f40 2024-09-10T08:23:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.152+0000 7f59b6000640 1 -- 172.21.15.2:0/3277802906 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a0010910 con 0x7f59b8104f40 2024-09-10T08:23:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.152+0000 7f59b6000640 1 -- 172.21.15.2:0/3277802906 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0018960 con 0x7f59b8104f40 2024-09-10T08:23:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.152+0000 7f59bfa00640 1 -- 172.21.15.2:0/3277802906 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b81a23d0 con 0x7f59b8104f40 2024-09-10T08:23:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.155+0000 7f59c27d4740 1 Processor -- start 2024-09-10T08:23:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.155+0000 7f59c27d4740 1 -- start start 2024-09-10T08:23:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59c27d4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 0x5603cf0e1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59c27d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e19d0 0x5603cf0e6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59c27d4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0e8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59c27d4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5603cf046100 con 0x5603cf0e6a60 2024-09-10T08:23:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59c27d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5603cefee1b0 con 0x5603cf0e19d0 2024-09-10T08:23:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59c27d4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5603cf0420c0 con 0x5603cf0e1090 2024-09-10T08:23:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59b6000640 1 -- 172.21.15.2:0/3277802906 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f59a0010ac0 con 0x7f59b8104f40 2024-09-10T08:23:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59bd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e19d0 0x5603cf0e6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59bd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e19d0 0x5603cf0e6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36776/0 (socket says 172.21.15.2:36776) 2024-09-10T08:23:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59bd000640 1 -- 172.21.15.2:0/1429876683 learned_addr learned my addr 172.21.15.2:0/1429876683 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59bda00640 1 --2- 172.21.15.2:0/1429876683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 0x5603cf0e1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59afe00640 1 --2- 172.21.15.2:0/1429876683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0e8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59bd000640 1 -- 172.21.15.2:0/1429876683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 msgr2=0x5603cf0e1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59bd000640 1 --2- 172.21.15.2:0/1429876683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 0x5603cf0e1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59bd000640 1 -- 172.21.15.2:0/1429876683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 msgr2=0x5603cf0e8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59bd000640 1 --2- 172.21.15.2:0/1429876683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0e8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.156+0000 7f59bd000640 1 -- 172.21.15.2:0/1429876683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ced59210 con 0x5603cf0e19d0 2024-09-10T08:23:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.157+0000 7f59bd000640 1 --2- 172.21.15.2:0/1429876683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e19d0 0x5603cf0e6490 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f59a8006010 tx=0x7f59a8004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d1110c2a295fae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.157+0000 7f59ae000640 1 -- 172.21.15.2:0/1429876683 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8035040 con 0x5603cf0e19d0 2024-09-10T08:23:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.157+0000 7f59ae000640 1 -- 172.21.15.2:0/1429876683 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a8004430 con 0x5603cf0e19d0 2024-09-10T08:23:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.157+0000 7f59ae000640 1 -- 172.21.15.2:0/1429876683 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a80316b0 con 0x5603cf0e19d0 2024-09-10T08:23:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.157+0000 7f59b6000640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f599c0a2c30 0x7f599c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.158+0000 7f59b6000640 1 -- 172.21.15.2:0/3277802906 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f59a00cf7e0 con 0x7f59b8104f40 2024-09-10T08:23:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.158+0000 7f59c27d4740 1 -- 172.21.15.2:0/1429876683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e19d0 msgr2=0x5603cf0e6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.158+0000 7f59c27d4740 1 --2- 172.21.15.2:0/1429876683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e19d0 0x5603cf0e6490 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f59a8006010 tx=0x7f59a8004040 comp rx=0 tx=0).stop 2024-09-10T08:23:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.158+0000 7f59bda00640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f599c0a2c30 0x7f599c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.158+0000 7f59c27d4740 1 -- 172.21.15.2:0/1429876683 shutdown_connections 2024-09-10T08:23:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.158+0000 7f59c27d4740 1 --2- 172.21.15.2:0/1429876683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0e8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.158+0000 7f59c27d4740 1 --2- 172.21.15.2:0/1429876683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e19d0 0x5603cf0e6490 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.158+0000 7f59c27d4740 1 --2- 172.21.15.2:0/1429876683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 0x5603cf0e1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.158+0000 7f59c27d4740 1 -- 172.21.15.2:0/1429876683 >> 172.21.15.2:0/1429876683 conn(0x5603cf0dc700 msgr2=0x5603cf0dcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.158+0000 7f59c27d4740 1 -- 172.21.15.2:0/1429876683 shutdown_connections 2024-09-10T08:23:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.158+0000 7f59c27d4740 1 -- 172.21.15.2:0/1429876683 wait complete. 2024-09-10T08:23:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.159+0000 7f59c27d4740 1 -- start start 2024-09-10T08:23:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.159+0000 7f59bda00640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f599c0a2c30 0x7f599c0a50f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f59b00049d0 tx=0x7f59b0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.159+0000 7f59c27d4740 1 Processor -- start 2024-09-10T08:23:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.159+0000 7f59c27d4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 0x5603cf0eb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.159+0000 7f59c27d4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e19d0 0x5603cf0eb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.159+0000 7f59c27d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e6a60 0x5603cf0f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.159+0000 7f59c27d4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5603cf0e99d0 con 0x5603cf0e19d0 2024-09-10T08:23:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.159+0000 7f59c27d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5603cefcf090 con 0x5603cf0e6a60 2024-09-10T08:23:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.159+0000 7f59c27d4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5603cf0e9b50 con 0x5603cf0e1090 2024-09-10T08:23:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.159+0000 7f59c27d4740 10 client.0 _ll_register_callbacks cb 0x5603cf023f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.159+0000 7f59afe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e19d0 0x5603cf0eb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59afe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e19d0 0x5603cf0eb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40064/0 (socket says 172.21.15.2:40064) 2024-09-10T08:23:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59bd000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 0x5603cf0eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59afe00640 1 -- 172.21.15.2:0/2723624873 learned_addr learned my addr 172.21.15.2:0/2723624873 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59bda00640 1 --2- 172.21.15.2:0/2723624873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e6a60 0x5603cf0f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59afe00640 1 -- 172.21.15.2:0/2723624873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 msgr2=0x5603cf0eb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59afe00640 1 --2- 172.21.15.2:0/2723624873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 0x5603cf0eb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59afe00640 1 -- 172.21.15.2:0/2723624873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e6a60 msgr2=0x5603cf0f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59afe00640 1 --2- 172.21.15.2:0/2723624873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e6a60 0x5603cf0f27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59afe00640 1 -- 172.21.15.2:0/2723624873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603cee25620 con 0x5603cf0e19d0 2024-09-10T08:23:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59afe00640 1 --2- 172.21.15.2:0/2723624873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e19d0 0x5603cf0eb780 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f59a0098100 tx=0x7f59a009b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43dd70406e747eb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:23:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59a6a00640 1 -- 172.21.15.2:0/2723624873 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a00cfdd0 con 0x5603cf0e19d0 2024-09-10T08:23:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59c27d4740 1 -- 172.21.15.2:0/2723624873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59a8004430 con 0x5603cf0e19d0 2024-09-10T08:23:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.160+0000 7f59a6a00640 1 -- 172.21.15.2:0/2723624873 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f59a009b0f0 con 0x5603cf0e19d0 2024-09-10T08:23:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.161+0000 7f59a6a00640 1 -- 172.21.15.2:0/2723624873 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59a009ffa0 con 0x5603cf0e19d0 2024-09-10T08:23:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.161+0000 7f59c27d4740 10 client.19851 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.161+0000 7f59c27d4740 1 -- 172.21.15.2:0/2723624873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5603cf0f5670 con 0x5603cf0e19d0 2024-09-10T08:23:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.161+0000 7f59a6a00640 1 -- 172.21.15.2:0/2723624873 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a8004430 con 0x5603cf0e19d0 2024-09-10T08:23:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.161+0000 7f59a6a00640 1 -- 172.21.15.2:0/2723624873 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f59a0018b00 con 0x5603cf0e19d0 2024-09-10T08:23:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.161+0000 7f59c27d4740 10 client.19851 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.161+0000 7f59c27d4740 10 client.19851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.161+0000 7f59c27d4740 20 client.19851 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.161+0000 7f59c27d4740 1 --2- 172.21.15.2:0/2723624873 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5603cf0f6d40 0x5603cf0f9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.161+0000 7f59c27d4740 4 client.19851 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:23:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.161+0000 7f59c27d4740 1 -- 172.21.15.2:0/2723624873 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603cf053830 con 0x5603cf0f6d40 2024-09-10T08:23:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.162+0000 7f59bd000640 1 --2- 172.21.15.2:0/2723624873 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5603cf0f6d40 0x5603cf0f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.162+0000 7f59bd000640 1 --2- 172.21.15.2:0/2723624873 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5603cf0f6d40 0x5603cf0f9150 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1178b7cc52f31a30 server_cookie=2c929dfd693b14cc in_seq=0 out_seq=0 2024-09-10T08:23:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.162+0000 7f59a6a00640 10 client.19851 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.163+0000 7f59a6a00640 1 -- 172.21.15.2:0/2723624873 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603cf053830 con 0x5603cf0f6d40 2024-09-10T08:23:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.163+0000 7f59a6a00640 10 client.19851 handle_command_reply: tid=0 2024-09-10T08:23:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.163+0000 7f59c27d4740 1 client.19851 shutdown 2024-09-10T08:23:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.163+0000 7f59c27d4740 1 -- 172.21.15.2:0/2723624873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e19d0 msgr2=0x5603cf0eb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.163+0000 7f59c27d4740 1 --2- 172.21.15.2:0/2723624873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e19d0 0x5603cf0eb780 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f59a0098100 tx=0x7f59a009b860 comp rx=0 tx=0).stop 2024-09-10T08:23:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.163+0000 7f59c27d4740 1 -- 172.21.15.2:0/2723624873 shutdown_connections 2024-09-10T08:23:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.163+0000 7f59c27d4740 1 --2- 172.21.15.2:0/2723624873 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5603cf0f6d40 0x5603cf0f9150 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.163+0000 7f59c27d4740 1 --2- 172.21.15.2:0/2723624873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e6a60 0x5603cf0f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.163+0000 7f59c27d4740 1 --2- 172.21.15.2:0/2723624873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e19d0 0x5603cf0eb780 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.163+0000 7f59c27d4740 1 --2- 172.21.15.2:0/2723624873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 0x5603cf0eb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.163+0000 7f59c27d4740 1 -- 172.21.15.2:0/2723624873 >> 172.21.15.2:0/2723624873 conn(0x5603cf0dc700 msgr2=0x5603cf0ddf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.163+0000 7f59a6a00640 0 client.19851 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.164+0000 7f59c27d4740 1 -- 172.21.15.2:0/2723624873 shutdown_connections 2024-09-10T08:23:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.164+0000 7f59c27d4740 1 -- 172.21.15.2:0/2723624873 wait complete. 2024-09-10T08:23:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.164+0000 7f59c27d4740 20 client.19851 trim_cache size 0 max 16384 2024-09-10T08:23:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.179+0000 7f59c27d4740 1 Processor -- start 2024-09-10T08:23:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.179+0000 7f59c27d4740 1 -- start start 2024-09-10T08:23:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.179+0000 7f59c27d4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 0x5603cf0dd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.179+0000 7f59c27d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e19d0 0x5603cf0f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.179+0000 7f59c27d4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0ddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.179+0000 7f59c27d4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81af2e0 con 0x5603cf0e6a60 2024-09-10T08:23:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.179+0000 7f59c27d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81acbb0 con 0x5603cf0e19d0 2024-09-10T08:23:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.179+0000 7f59c27d4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81aaa00 con 0x5603cf0e1090 2024-09-10T08:23:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.180+0000 7f59bda00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0ddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.180+0000 7f59bda00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0ddaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40066/0 (socket says 172.21.15.2:40066) 2024-09-10T08:23:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.180+0000 7f59bda00640 1 -- 172.21.15.2:0/2984815248 learned_addr learned my addr 172.21.15.2:0/2984815248 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.180+0000 7f59afe00640 1 --2- 172.21.15.2:0/2984815248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e19d0 0x5603cf0f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.180+0000 7f59bd000640 1 --2- 172.21.15.2:0/2984815248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 0x5603cf0dd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.180+0000 7f59bda00640 1 -- 172.21.15.2:0/2984815248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 msgr2=0x5603cf0dd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.180+0000 7f59bda00640 1 --2- 172.21.15.2:0/2984815248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 0x5603cf0dd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.180+0000 7f59bda00640 1 -- 172.21.15.2:0/2984815248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e19d0 msgr2=0x5603cf0f2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.180+0000 7f59bda00640 1 --2- 172.21.15.2:0/2984815248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e19d0 0x5603cf0f2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.180+0000 7f59bda00640 1 -- 172.21.15.2:0/2984815248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603cf0dc520 con 0x5603cf0e6a60 2024-09-10T08:23:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.180+0000 7f59afe00640 1 --2- 172.21.15.2:0/2984815248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e19d0 0x5603cf0f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:23:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.180+0000 7f59bd000640 1 --2- 172.21.15.2:0/2984815248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 0x5603cf0dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:23:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.180+0000 7f59bda00640 1 --2- 172.21.15.2:0/2984815248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0ddaf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f59b000fa70 tx=0x7f59b001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c907107ab9929dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.181+0000 7f59ad600640 1 -- 172.21.15.2:0/2984815248 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0004b20 con 0x5603cf0e6a60 2024-09-10T08:23:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.181+0000 7f59ad600640 1 -- 172.21.15.2:0/2984815248 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b0004cc0 con 0x5603cf0e6a60 2024-09-10T08:23:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.181+0000 7f59ad600640 1 -- 172.21.15.2:0/2984815248 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0029420 con 0x5603cf0e6a60 2024-09-10T08:23:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.181+0000 7f59c27d4740 1 -- 172.21.15.2:0/2984815248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 msgr2=0x5603cf0ddaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.181+0000 7f59c27d4740 1 --2- 172.21.15.2:0/2984815248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0ddaf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f59b000fa70 tx=0x7f59b001a000 comp rx=0 tx=0).stop 2024-09-10T08:23:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.181+0000 7f59c27d4740 1 -- 172.21.15.2:0/2984815248 shutdown_connections 2024-09-10T08:23:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.181+0000 7f59c27d4740 1 --2- 172.21.15.2:0/2984815248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0ddaf0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.181+0000 7f59c27d4740 1 --2- 172.21.15.2:0/2984815248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e19d0 0x5603cf0f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.181+0000 7f59c27d4740 1 --2- 172.21.15.2:0/2984815248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e1090 0x5603cf0dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.181+0000 7f59c27d4740 1 -- 172.21.15.2:0/2984815248 >> 172.21.15.2:0/2984815248 conn(0x5603cf0dc700 msgr2=0x5603cf0e2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.181+0000 7f59c27d4740 1 -- 172.21.15.2:0/2984815248 shutdown_connections 2024-09-10T08:23:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.181+0000 7f59c27d4740 1 -- 172.21.15.2:0/2984815248 wait complete. 2024-09-10T08:23:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.182+0000 7f59c27d4740 1 -- start start 2024-09-10T08:23:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.182+0000 7f59c27d4740 1 Processor -- start 2024-09-10T08:23:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.182+0000 7f59c27d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e1090 0x5603cf0efe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.182+0000 7f59c27d4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e19d0 0x5603cf0f03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.182+0000 7f59c27d4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0f1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.182+0000 7f59c27d4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81ab070 con 0x5603cf0e6a60 2024-09-10T08:23:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.182+0000 7f59c27d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81ac4e0 con 0x5603cf0e1090 2024-09-10T08:23:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.182+0000 7f59c27d4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81ab740 con 0x5603cf0e19d0 2024-09-10T08:23:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.182+0000 7f59c27d4740 10 client.0 _ll_register_callbacks cb 0x7f59a000ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.183+0000 7f59bda00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.183+0000 7f59bda00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0f1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40074/0 (socket says 172.21.15.2:40074) 2024-09-10T08:23:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.183+0000 7f59bda00640 1 -- 172.21.15.2:0/1388299573 learned_addr learned my addr 172.21.15.2:0/1388299573 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.183+0000 7f59bd000640 1 --2- 172.21.15.2:0/1388299573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e1090 0x5603cf0efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.183+0000 7f59afe00640 1 --2- 172.21.15.2:0/1388299573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e19d0 0x5603cf0f03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.183+0000 7f59bda00640 1 -- 172.21.15.2:0/1388299573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e19d0 msgr2=0x5603cf0f03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.183+0000 7f59bda00640 1 --2- 172.21.15.2:0/1388299573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e19d0 0x5603cf0f03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.183+0000 7f59bda00640 1 -- 172.21.15.2:0/1388299573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e1090 msgr2=0x5603cf0efe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.183+0000 7f59bda00640 1 --2- 172.21.15.2:0/1388299573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e1090 0x5603cf0efe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.183+0000 7f59bda00640 1 -- 172.21.15.2:0/1388299573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603cf0f1980 con 0x5603cf0e6a60 2024-09-10T08:23:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.183+0000 7f59bda00640 1 --2- 172.21.15.2:0/1388299573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0f1370 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f59b0019040 tx=0x7f59b0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4122f7c3b91e9ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.183+0000 7f59a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:23:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.184+0000 7f59a6a00640 1 -- 172.21.15.2:0/1388299573 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0037060 con 0x5603cf0e6a60 2024-09-10T08:23:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.184+0000 7f59a6a00640 1 -- 172.21.15.2:0/1388299573 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f59b0011cc0 con 0x5603cf0e6a60 2024-09-10T08:23:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.184+0000 7f59c27d4740 1 -- 172.21.15.2:0/1388299573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b0004cc0 con 0x5603cf0e6a60 2024-09-10T08:23:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.184+0000 7f59a6a00640 1 -- 172.21.15.2:0/1388299573 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59b003e050 con 0x5603cf0e6a60 2024-09-10T08:23:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.184+0000 7f59c27d4740 10 client.19857 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.184+0000 7f59c27d4740 1 -- 172.21.15.2:0/1388299573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5603cf161a30 con 0x5603cf0e6a60 2024-09-10T08:23:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.184+0000 7f59a6a00640 1 -- 172.21.15.2:0/1388299573 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b0004cc0 con 0x5603cf0e6a60 2024-09-10T08:23:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.184+0000 7f59a6a00640 1 -- 172.21.15.2:0/1388299573 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f59b0031030 con 0x5603cf0e6a60 2024-09-10T08:23:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.185+0000 7f59c27d4740 10 client.19857 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.185+0000 7f59c27d4740 10 client.19857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.185+0000 7f59c27d4740 20 client.19857 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.185+0000 7f59c27d4740 1 --2- 172.21.15.2:0/1388299573 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5603cf1622a0 0x5603cf1646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.185+0000 7f59c27d4740 4 client.19857 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:23:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.185+0000 7f59c27d4740 1 -- 172.21.15.2:0/1388299573 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603cf04db80 con 0x5603cf1622a0 2024-09-10T08:23:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.185+0000 7f59bd000640 1 --2- 172.21.15.2:0/1388299573 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5603cf1622a0 0x5603cf1646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.185+0000 7f59bd000640 1 --2- 172.21.15.2:0/1388299573 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5603cf1622a0 0x5603cf1646c0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717581deed6e6416 server_cookie=cd0a57018f864dbe in_seq=0 out_seq=0 2024-09-10T08:23:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.185+0000 7f59a6a00640 10 client.19857 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59a6a00640 1 -- 172.21.15.2:0/1388299573 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603cf04db80 con 0x5603cf1622a0 2024-09-10T08:23:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59a6a00640 10 client.19857 handle_command_reply: tid=0 2024-09-10T08:23:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59c27d4740 1 client.19857 shutdown 2024-09-10T08:23:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59c27d4740 1 -- 172.21.15.2:0/1388299573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 msgr2=0x5603cf0f1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59c27d4740 1 --2- 172.21.15.2:0/1388299573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0f1370 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f59b0019040 tx=0x7f59b0004b50 comp rx=0 tx=0).stop 2024-09-10T08:23:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59c27d4740 1 -- 172.21.15.2:0/1388299573 shutdown_connections 2024-09-10T08:23:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59c27d4740 1 --2- 172.21.15.2:0/1388299573 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5603cf1622a0 0x5603cf1646c0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59c27d4740 1 --2- 172.21.15.2:0/1388299573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5603cf0e6a60 0x5603cf0f1370 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59c27d4740 1 --2- 172.21.15.2:0/1388299573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603cf0e19d0 0x5603cf0f03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59c27d4740 1 --2- 172.21.15.2:0/1388299573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603cf0e1090 0x5603cf0efe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59a6a00640 0 client.19857 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59c27d4740 1 -- 172.21.15.2:0/1388299573 >> 172.21.15.2:0/1388299573 conn(0x5603cf0dc700 msgr2=0x5603cf0db060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59c27d4740 1 -- 172.21.15.2:0/1388299573 shutdown_connections 2024-09-10T08:23:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59c27d4740 1 -- 172.21.15.2:0/1388299573 wait complete. 2024-09-10T08:23:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.201+0000 7f59c27d4740 20 client.19857 trim_cache size 0 max 16384 2024-09-10T08:23:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:23:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:23:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:23:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:23:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.202+0000 7f59bfa00640 1 -- 172.21.15.2:0/3277802906 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f599c0a2c30 msgr2=0x7f599c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.202+0000 7f59bfa00640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f599c0a2c30 0x7f599c0a50f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f59b00049d0 tx=0x7f59b0009500 comp rx=0 tx=0).stop 2024-09-10T08:23:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.202+0000 7f59bfa00640 1 -- 172.21.15.2:0/3277802906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b8104f40 msgr2=0x7f59b819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.202+0000 7f59bfa00640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b8104f40 0x7f59b819a4f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f59a000e9f0 tx=0x7f59a000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.202+0000 7f59bfa00640 1 -- 172.21.15.2:0/3277802906 shutdown_connections 2024-09-10T08:23:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.202+0000 7f59bfa00640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59b8105900 0x7f59b81a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.202+0000 7f59bfa00640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f599c0a2c30 0x7f599c0a50f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.202+0000 7f59bfa00640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b8104f40 0x7f59b819a4f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.202+0000 7f59bfa00640 1 --2- 172.21.15.2:0/3277802906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59b8103d50 0x7f59b8199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.202+0000 7f59bfa00640 1 -- 172.21.15.2:0/3277802906 >> 172.21.15.2:0/3277802906 conn(0x7f59b80ff4b0 msgr2=0x7f59b81082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.202+0000 7f59bfa00640 1 -- 172.21.15.2:0/3277802906 shutdown_connections 2024-09-10T08:23:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:39.202+0000 7f59bfa00640 1 -- 172.21.15.2:0/3277802906 wait complete. 2024-09-10T08:23:39.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:39 smithi155 ceph-mon[35405]: pgmap v852: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-10T08:23:39.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:39 smithi107 ceph-mon[35048]: pgmap v852: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-10T08:23:40.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:39 smithi002 ceph-mon[30563]: pgmap v852: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-10T08:23:41.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:41 smithi155 ceph-mon[35405]: pgmap v853: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-09-10T08:23:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:41 smithi107 ceph-mon[35048]: pgmap v853: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-09-10T08:23:42.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:41 smithi002 ceph-mon[30563]: pgmap v853: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-09-10T08:23:43.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:43 smithi155 ceph-mon[35405]: pgmap v854: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-10T08:23:43.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:43 smithi107 ceph-mon[35048]: pgmap v854: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-10T08:23:44.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:43 smithi002 ceph-mon[30563]: pgmap v854: 129 pgs: 129 active+clean; 4.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-10T08:23:45.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:45 smithi155 ceph-mon[35405]: pgmap v855: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-10T08:23:45.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:45 smithi107 ceph-mon[35048]: pgmap v855: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-10T08:23:46.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:45 smithi002 ceph-mon[30563]: pgmap v855: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-10T08:23:47.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:47 smithi155 ceph-mon[35405]: pgmap v856: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-09-10T08:23:47.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:47 smithi107 ceph-mon[35048]: pgmap v856: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-09-10T08:23:48.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:47 smithi002 ceph-mon[30563]: pgmap v856: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-09-10T08:23:49.903 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:23:49.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:49 smithi155 ceph-mon[35405]: pgmap v857: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 11 op/s 2024-09-10T08:23:49.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:49 smithi107 ceph-mon[35048]: pgmap v857: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 11 op/s 2024-09-10T08:23:49.959 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:49 smithi002 ceph-mon[30563]: pgmap v857: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 11 op/s 2024-09-10T08:23:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:23:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.878+0000 7f22a2400640 1 -- 172.21.15.2:0/3129189761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f229c101df0 msgr2=0x7f229c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.878+0000 7f22a2400640 1 --2- 172.21.15.2:0/3129189761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f229c101df0 0x7f229c1021d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f228c009900 tx=0x7f228c02a410 comp rx=0 tx=0).stop 2024-09-10T08:23:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.878+0000 7f22a2400640 1 -- 172.21.15.2:0/3129189761 shutdown_connections 2024-09-10T08:23:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.878+0000 7f22a2400640 1 --2- 172.21.15.2:0/3129189761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f229c106f20 0x7f229c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.878+0000 7f22a2400640 1 --2- 172.21.15.2:0/3129189761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f229c1027a0 0x7f229c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.878+0000 7f22a2400640 1 --2- 172.21.15.2:0/3129189761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f229c101df0 0x7f229c1021d0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.878+0000 7f22a2400640 1 -- 172.21.15.2:0/3129189761 >> 172.21.15.2:0/3129189761 conn(0x7f229c0fd8b0 msgr2=0x7f229c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.878+0000 7f22a2400640 1 -- 172.21.15.2:0/3129189761 shutdown_connections 2024-09-10T08:23:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.878+0000 7f22a2400640 1 -- 172.21.15.2:0/3129189761 wait complete. 2024-09-10T08:23:50.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.879+0000 7f22a2400640 1 Processor -- start 2024-09-10T08:23:50.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.880+0000 7f22a2400640 1 -- start start 2024-09-10T08:23:50.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.880+0000 7f22a2400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f229c101df0 0x7f229c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:50.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.880+0000 7f22a2400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f229c1027a0 0x7f229c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:50.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.880+0000 7f22a2400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f229c106f20 0x7f229c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:50.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.880+0000 7f22a2400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f229c111f80 con 0x7f229c1027a0 2024-09-10T08:23:50.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.880+0000 7f22a2400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f229c111e00 con 0x7f229c106f20 2024-09-10T08:23:50.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.880+0000 7f22a2400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f229c112100 con 0x7f229c101df0 2024-09-10T08:23:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.881+0000 7f229aa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f229c1027a0 0x7f229c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.881+0000 7f229be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f229c106f20 0x7f229c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.881+0000 7f229be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f229c106f20 0x7f229c19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:48220/0 (socket says 172.21.15.2:48220) 2024-09-10T08:23:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.881+0000 7f229b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f229c101df0 0x7f229c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.881+0000 7f229be00640 1 -- 172.21.15.2:0/1239195570 learned_addr learned my addr 172.21.15.2:0/1239195570 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.881+0000 7f229be00640 1 -- 172.21.15.2:0/1239195570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f229c101df0 msgr2=0x7f229c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.881+0000 7f229be00640 1 --2- 172.21.15.2:0/1239195570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f229c101df0 0x7f229c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.881+0000 7f229be00640 1 -- 172.21.15.2:0/1239195570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f229c1027a0 msgr2=0x7f229c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.881+0000 7f229be00640 1 --2- 172.21.15.2:0/1239195570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f229c1027a0 0x7f229c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.881+0000 7f229be00640 1 -- 172.21.15.2:0/1239195570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c19d430 con 0x7f229c106f20 2024-09-10T08:23:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.881+0000 7f229aa00640 1 --2- 172.21.15.2:0/1239195570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f229c1027a0 0x7f229c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.881+0000 7f229be00640 1 --2- 172.21.15.2:0/1239195570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f229c106f20 0x7f229c19cd50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f229400eb10 tx=0x7f229400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.882+0000 7f2293e00640 1 -- 172.21.15.2:0/1239195570 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294004b20 con 0x7f229c106f20 2024-09-10T08:23:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.882+0000 7f22a2400640 1 -- 172.21.15.2:0/1239195570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229c19d720 con 0x7f229c106f20 2024-09-10T08:23:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.882+0000 7f2293e00640 1 -- 172.21.15.2:0/1239195570 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2294004cc0 con 0x7f229c106f20 2024-09-10T08:23:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.882+0000 7f2293e00640 1 -- 172.21.15.2:0/1239195570 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294010760 con 0x7f229c106f20 2024-09-10T08:23:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.882+0000 7f22a2400640 1 -- 172.21.15.2:0/1239195570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229c1a5000 con 0x7f229c106f20 2024-09-10T08:23:50.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.883+0000 7f22a2400640 1 -- 172.21.15.2:0/1239195570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229c1021d0 con 0x7f229c106f20 2024-09-10T08:23:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.885+0000 7f2293e00640 1 -- 172.21.15.2:0/1239195570 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2294044b30 con 0x7f229c106f20 2024-09-10T08:23:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.886+0000 7f2293e00640 1 --2- 172.21.15.2:0/1239195570 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22800a2b80 0x7f22800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.886+0000 7f2293e00640 1 -- 172.21.15.2:0/1239195570 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f22940cf980 con 0x7f229c106f20 2024-09-10T08:23:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.886+0000 7f229b400640 1 --2- 172.21.15.2:0/1239195570 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22800a2b80 0x7f22800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.887+0000 7f229b400640 1 --2- 172.21.15.2:0/1239195570 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22800a2b80 0x7f22800a5040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f228c009230 tx=0x7f228c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:50.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:50.888+0000 7f2293e00640 1 -- 172.21.15.2:0/1239195570 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f229409c490 con 0x7f229c106f20 2024-09-10T08:23:51.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.109+0000 7f22a2400640 1 -- 172.21.15.2:0/1239195570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f229c1997e0 con 0x7f229c106f20 2024-09-10T08:23:51.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.110+0000 7f2293e00640 1 -- 172.21.15.2:0/1239195570 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f229c1997e0 con 0x7f229c106f20 2024-09-10T08:23:51.111 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:23:51.111 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:23:51.111 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:23:51.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.113+0000 7f22a2400640 1 -- 172.21.15.2:0/1239195570 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22800a2b80 msgr2=0x7f22800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:51.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.113+0000 7f22a2400640 1 --2- 172.21.15.2:0/1239195570 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22800a2b80 0x7f22800a5040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f228c009230 tx=0x7f228c0058d0 comp rx=0 tx=0).stop 2024-09-10T08:23:51.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.113+0000 7f22a2400640 1 -- 172.21.15.2:0/1239195570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f229c106f20 msgr2=0x7f229c19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:51.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.113+0000 7f22a2400640 1 --2- 172.21.15.2:0/1239195570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f229c106f20 0x7f229c19cd50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f229400eb10 tx=0x7f229400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:23:51.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.113+0000 7f22a2400640 1 -- 172.21.15.2:0/1239195570 shutdown_connections 2024-09-10T08:23:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.113+0000 7f22a2400640 1 --2- 172.21.15.2:0/1239195570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f229c106f20 0x7f229c19cd50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.113+0000 7f22a2400640 1 --2- 172.21.15.2:0/1239195570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f229c1027a0 0x7f229c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.113+0000 7f22a2400640 1 --2- 172.21.15.2:0/1239195570 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f22800a2b80 0x7f22800a5040 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.113+0000 7f22a2400640 1 --2- 172.21.15.2:0/1239195570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f229c101df0 0x7f229c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.113+0000 7f22a2400640 1 -- 172.21.15.2:0/1239195570 >> 172.21.15.2:0/1239195570 conn(0x7f229c0fd8b0 msgr2=0x7f229c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.113+0000 7f22a2400640 1 -- 172.21.15.2:0/1239195570 shutdown_connections 2024-09-10T08:23:51.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:51.113+0000 7f22a2400640 1 -- 172.21.15.2:0/1239195570 wait complete. 2024-09-10T08:23:51.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:51 smithi155 ceph-mon[35405]: pgmap v858: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-10T08:23:51.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:51 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1239195570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:23:51.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:51 smithi107 ceph-mon[35048]: pgmap v858: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-10T08:23:51.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:51 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1239195570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:23:52.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:51 smithi002 ceph-mon[30563]: pgmap v858: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-10T08:23:52.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:51 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1239195570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:23:52.111 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id b597e0bd --release --await' 2024-09-10T08:23:52.112 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id b597e0bd --release --await 2024-09-10T08:23:52.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.321+0000 7fb685400640 1 -- 172.21.15.2:0/3544196251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680074fc0 msgr2=0x7fb68010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:52.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.321+0000 7fb685400640 1 --2- 172.21.15.2:0/3544196251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680074fc0 0x7fb68010df40 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009900 tx=0x7fb66c02a400 comp rx=0 tx=0).stop 2024-09-10T08:23:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.321+0000 7fb685400640 1 -- 172.21.15.2:0/3544196251 shutdown_connections 2024-09-10T08:23:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.321+0000 7fb685400640 1 --2- 172.21.15.2:0/3544196251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb680105340 0x7fb680110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.321+0000 7fb685400640 1 --2- 172.21.15.2:0/3544196251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680074fc0 0x7fb68010df40 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.321+0000 7fb685400640 1 --2- 172.21.15.2:0/3544196251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb680074680 0x7fb680074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.321+0000 7fb685400640 1 -- 172.21.15.2:0/3544196251 >> 172.21.15.2:0/3544196251 conn(0x7fb6800fd0d0 msgr2=0x7fb6800ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.321+0000 7fb685400640 1 -- 172.21.15.2:0/3544196251 shutdown_connections 2024-09-10T08:23:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.321+0000 7fb685400640 1 -- 172.21.15.2:0/3544196251 wait complete. 2024-09-10T08:23:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.322+0000 7fb685400640 1 Processor -- start 2024-09-10T08:23:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.322+0000 7fb685400640 1 -- start start 2024-09-10T08:23:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.322+0000 7fb685400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680074680 0x7fb68019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.322+0000 7fb685400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb680074fc0 0x7fb68019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.322+0000 7fb685400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6801a3980 0x7fb6801a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.322+0000 7fb685400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb680113d40 con 0x7fb680074680 2024-09-10T08:23:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.322+0000 7fb685400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb680113bc0 con 0x7fb680074fc0 2024-09-10T08:23:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.322+0000 7fb685400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb680113ec0 con 0x7fb6801a3980 2024-09-10T08:23:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.323+0000 7fb67fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680074680 0x7fb68019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.323+0000 7fb67f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb680074fc0 0x7fb68019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.323+0000 7fb67fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680074680 0x7fb68019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60088/0 (socket says 172.21.15.2:60088) 2024-09-10T08:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.323+0000 7fb684a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6801a3980 0x7fb6801a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.323+0000 7fb67fe00640 1 -- 172.21.15.2:0/2345095081 learned_addr learned my addr 172.21.15.2:0/2345095081 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.323+0000 7fb67f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb680074fc0 0x7fb68019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:48230/0 (socket says 172.21.15.2:48230) 2024-09-10T08:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.323+0000 7fb67fe00640 1 -- 172.21.15.2:0/2345095081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6801a3980 msgr2=0x7fb6801a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.323+0000 7fb67fe00640 1 --2- 172.21.15.2:0/2345095081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6801a3980 0x7fb6801a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.323+0000 7fb67fe00640 1 -- 172.21.15.2:0/2345095081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb680074fc0 msgr2=0x7fb68019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.323+0000 7fb67fe00640 1 --2- 172.21.15.2:0/2345095081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb680074fc0 0x7fb68019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.323+0000 7fb67fe00640 1 -- 172.21.15.2:0/2345095081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6801a63e0 con 0x7fb680074680 2024-09-10T08:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.323+0000 7fb684a00640 1 --2- 172.21.15.2:0/2345095081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6801a3980 0x7fb6801a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.324+0000 7fb67fe00640 1 --2- 172.21.15.2:0/2345095081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680074680 0x7fb68019e3e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb67000e9f0 tx=0x7fb67000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.324+0000 7fb67cc00640 1 -- 172.21.15.2:0/2345095081 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67000cd80 con 0x7fb680074680 2024-09-10T08:23:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.324+0000 7fb67cc00640 1 -- 172.21.15.2:0/2345095081 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb670005500 con 0x7fb680074680 2024-09-10T08:23:52.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.324+0000 7fb67cc00640 1 -- 172.21.15.2:0/2345095081 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670010690 con 0x7fb680074680 2024-09-10T08:23:52.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.324+0000 7fb685400640 1 -- 172.21.15.2:0/2345095081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6801a66d0 con 0x7fb680074680 2024-09-10T08:23:52.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.324+0000 7fb685400640 1 -- 172.21.15.2:0/2345095081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6801a6ba0 con 0x7fb680074680 2024-09-10T08:23:52.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.327+0000 7fb6898af740 1 Processor -- start 2024-09-10T08:23:52.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.327+0000 7fb67cc00640 1 -- 172.21.15.2:0/2345095081 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb6700046c0 con 0x7fb680074680 2024-09-10T08:23:52.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.327+0000 7fb6898af740 1 -- start start 2024-09-10T08:23:52.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb6898af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7260 0x558dd6ac7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb6898af740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7ba0 0x558dd6acc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb6898af740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 0x558dd6acf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb6898af740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558dd6a2c100 con 0x558dd6accc30 2024-09-10T08:23:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb6898af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558dd69d41b0 con 0x558dd6ac7260 2024-09-10T08:23:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb6898af740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558dd6a280c0 con 0x558dd6ac7ba0 2024-09-10T08:23:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb67f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 0x558dd6acf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb67f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 0x558dd6acf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60102/0 (socket says 172.21.15.2:60102) 2024-09-10T08:23:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb67f400640 1 -- 172.21.15.2:0/1496169550 learned_addr learned my addr 172.21.15.2:0/1496169550 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb67fe00640 1 --2- 172.21.15.2:0/1496169550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7ba0 0x558dd6acc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb684a00640 1 --2- 172.21.15.2:0/1496169550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7260 0x558dd6ac7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb67f400640 1 -- 172.21.15.2:0/1496169550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7ba0 msgr2=0x558dd6acc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb67f400640 1 --2- 172.21.15.2:0/1496169550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7ba0 0x558dd6acc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb67f400640 1 -- 172.21.15.2:0/1496169550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7260 msgr2=0x558dd6ac7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb67f400640 1 --2- 172.21.15.2:0/1496169550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7260 0x558dd6ac7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb67f400640 1 -- 172.21.15.2:0/1496169550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dd673f210 con 0x558dd6accc30 2024-09-10T08:23:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.328+0000 7fb67fe00640 1 --2- 172.21.15.2:0/1496169550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7ba0 0x558dd6acc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:23:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.329+0000 7fb67f400640 1 --2- 172.21.15.2:0/1496169550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 0x558dd6acf0f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb66c02a920 tx=0x7fb66c005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cba444fff08bdf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.329+0000 7fb675200640 1 -- 172.21.15.2:0/1496169550 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c040070 con 0x558dd6accc30 2024-09-10T08:23:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.329+0000 7fb67cc00640 1 --2- 172.21.15.2:0/2345095081 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb6640a2c00 0x7fb6640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.329+0000 7fb675200640 1 -- 172.21.15.2:0/1496169550 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb66c004030 con 0x558dd6accc30 2024-09-10T08:23:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.329+0000 7fb675200640 1 -- 172.21.15.2:0/1496169550 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c035040 con 0x558dd6accc30 2024-09-10T08:23:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.329+0000 7fb67cc00640 1 -- 172.21.15.2:0/2345095081 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb6700cf810 con 0x7fb680074680 2024-09-10T08:23:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.329+0000 7fb684a00640 1 --2- 172.21.15.2:0/2345095081 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb6640a2c00 0x7fb6640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.330+0000 7fb6898af740 1 -- 172.21.15.2:0/1496169550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 msgr2=0x558dd6acf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.330+0000 7fb6898af740 1 --2- 172.21.15.2:0/1496169550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 0x558dd6acf0f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb66c02a920 tx=0x7fb66c005de0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.330+0000 7fb6898af740 1 -- 172.21.15.2:0/1496169550 shutdown_connections 2024-09-10T08:23:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.330+0000 7fb6898af740 1 --2- 172.21.15.2:0/1496169550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 0x558dd6acf0f0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.330+0000 7fb6898af740 1 --2- 172.21.15.2:0/1496169550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7ba0 0x558dd6acc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.330+0000 7fb6898af740 1 --2- 172.21.15.2:0/1496169550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7260 0x558dd6ac7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.330+0000 7fb6898af740 1 -- 172.21.15.2:0/1496169550 >> 172.21.15.2:0/1496169550 conn(0x558dd6ac28d0 msgr2=0x558dd6ac2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.330+0000 7fb6898af740 1 -- 172.21.15.2:0/1496169550 shutdown_connections 2024-09-10T08:23:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.330+0000 7fb684a00640 1 --2- 172.21.15.2:0/2345095081 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb6640a2c00 0x7fb6640a50c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb678009fd0 tx=0x7fb678009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.330+0000 7fb6898af740 1 -- 172.21.15.2:0/1496169550 wait complete. 2024-09-10T08:23:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.331+0000 7fb6898af740 1 -- start start 2024-09-10T08:23:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.331+0000 7fb6898af740 1 Processor -- start 2024-09-10T08:23:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.331+0000 7fb6898af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7260 0x558dd6ad13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.331+0000 7fb6898af740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.331+0000 7fb6898af740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6accc30 0x558dd6ad8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.331+0000 7fb6898af740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558dd6acfba0 con 0x558dd6ac7ba0 2024-09-10T08:23:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.331+0000 7fb6898af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558dd69b5090 con 0x558dd6ac7260 2024-09-10T08:23:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.331+0000 7fb6898af740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558dd6acfd20 con 0x558dd6accc30 2024-09-10T08:23:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.331+0000 7fb6898af740 10 client.0 _ll_register_callbacks cb 0x558dd6a09f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.332+0000 7fb67f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.332+0000 7fb67fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7260 0x558dd6ad13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.332+0000 7fb684a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6accc30 0x558dd6ad8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.332+0000 7fb67f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60112/0 (socket says 172.21.15.2:60112) 2024-09-10T08:23:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.332+0000 7fb67f400640 1 -- 172.21.15.2:0/2904505873 learned_addr learned my addr 172.21.15.2:0/2904505873 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.332+0000 7fb67f400640 1 -- 172.21.15.2:0/2904505873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6accc30 msgr2=0x558dd6ad8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.332+0000 7fb67f400640 1 --2- 172.21.15.2:0/2904505873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6accc30 0x558dd6ad8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.332+0000 7fb67f400640 1 -- 172.21.15.2:0/2904505873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7260 msgr2=0x558dd6ad13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.332+0000 7fb67f400640 1 --2- 172.21.15.2:0/2904505873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7260 0x558dd6ad13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.332+0000 7fb67f400640 1 -- 172.21.15.2:0/2904505873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dd680b620 con 0x558dd6ac7ba0 2024-09-10T08:23:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.332+0000 7fb67fe00640 1 --2- 172.21.15.2:0/2904505873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7260 0x558dd6ad13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.332+0000 7fb67f400640 1 --2- 172.21.15.2:0/2904505873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad1920 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb66c000c00 tx=0x7fb66c032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb7ce9551d8f845 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.332+0000 7fb669600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:23:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.333+0000 7fb669600640 1 -- 172.21.15.2:0/2904505873 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c04c020 con 0x558dd6ac7ba0 2024-09-10T08:23:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.333+0000 7fb669600640 1 -- 172.21.15.2:0/2904505873 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb66c0316b0 con 0x558dd6ac7ba0 2024-09-10T08:23:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.333+0000 7fb6898af740 1 -- 172.21.15.2:0/2904505873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb66c004030 con 0x558dd6ac7ba0 2024-09-10T08:23:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.333+0000 7fb669600640 1 -- 172.21.15.2:0/2904505873 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb66c057050 con 0x558dd6ac7ba0 2024-09-10T08:23:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.333+0000 7fb6898af740 10 client.19875 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.333+0000 7fb6898af740 1 -- 172.21.15.2:0/2904505873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558dd6adb750 con 0x558dd6ac7ba0 2024-09-10T08:23:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.333+0000 7fb669600640 1 -- 172.21.15.2:0/2904505873 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb66c004030 con 0x558dd6ac7ba0 2024-09-10T08:23:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.333+0000 7fb669600640 1 -- 172.21.15.2:0/2904505873 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb66c004780 con 0x558dd6ac7ba0 2024-09-10T08:23:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.334+0000 7fb6898af740 10 client.19875 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.334+0000 7fb6898af740 10 client.19875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.334+0000 7fb6898af740 20 client.19875 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.334+0000 7fb6898af740 1 --2- 172.21.15.2:0/2904505873 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558dd6adcaa0 0x558dd6adee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.334+0000 7fb6898af740 4 client.19875 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:23:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.334+0000 7fb6898af740 1 -- 172.21.15.2:0/2904505873 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dd6a39830 con 0x558dd6adcaa0 2024-09-10T08:23:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.334+0000 7fb67fe00640 1 --2- 172.21.15.2:0/2904505873 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558dd6adcaa0 0x558dd6adee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.335+0000 7fb67fe00640 1 --2- 172.21.15.2:0/2904505873 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558dd6adcaa0 0x558dd6adee90 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4069d9b1c9000a7a server_cookie=80cf9d6e79951b77 in_seq=0 out_seq=0 2024-09-10T08:23:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.335+0000 7fb669600640 10 client.19875 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.335+0000 7fb669600640 1 -- 172.21.15.2:0/2904505873 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558dd6a39830 con 0x558dd6adcaa0 2024-09-10T08:23:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.335+0000 7fb669600640 10 client.19875 handle_command_reply: tid=0 2024-09-10T08:23:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.335+0000 7fb6898af740 1 client.19875 shutdown 2024-09-10T08:23:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.336+0000 7fb6898af740 1 -- 172.21.15.2:0/2904505873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6ac7ba0 msgr2=0x558dd6ad1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.336+0000 7fb6898af740 1 --2- 172.21.15.2:0/2904505873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad1920 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb66c000c00 tx=0x7fb66c032f60 comp rx=0 tx=0).stop 2024-09-10T08:23:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.336+0000 7fb6898af740 1 -- 172.21.15.2:0/2904505873 shutdown_connections 2024-09-10T08:23:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.336+0000 7fb6898af740 1 --2- 172.21.15.2:0/2904505873 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558dd6adcaa0 0x558dd6adee90 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.336+0000 7fb6898af740 1 --2- 172.21.15.2:0/2904505873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6accc30 0x558dd6ad8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.336+0000 7fb6898af740 1 --2- 172.21.15.2:0/2904505873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad1920 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.336+0000 7fb6898af740 1 --2- 172.21.15.2:0/2904505873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7260 0x558dd6ad13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.336+0000 7fb6898af740 1 -- 172.21.15.2:0/2904505873 >> 172.21.15.2:0/2904505873 conn(0x558dd6ac28d0 msgr2=0x558dd6ac40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.336+0000 7fb669600640 0 client.19875 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.336+0000 7fb6898af740 1 -- 172.21.15.2:0/2904505873 shutdown_connections 2024-09-10T08:23:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.336+0000 7fb6898af740 1 -- 172.21.15.2:0/2904505873 wait complete. 2024-09-10T08:23:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.336+0000 7fb6898af740 20 client.19875 trim_cache size 0 max 16384 2024-09-10T08:23:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.359+0000 7fb6898af740 1 Processor -- start 2024-09-10T08:23:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.359+0000 7fb6898af740 1 -- start start 2024-09-10T08:23:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.359+0000 7fb6898af740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 0x558dd6adeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.359+0000 7fb6898af740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.359+0000 7fb6898af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6accc30 0x558dd6adeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.359+0000 7fb6898af740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801b3560 con 0x558dd6ac7ba0 2024-09-10T08:23:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.359+0000 7fb6898af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801b0e30 con 0x558dd6accc30 2024-09-10T08:23:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.359+0000 7fb6898af740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801aec80 con 0x558dd6ac7260 2024-09-10T08:23:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.360+0000 7fb67fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 0x558dd6adeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.360+0000 7fb67f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.360+0000 7fb67fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 0x558dd6adeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:37158/0 (socket says 172.21.15.2:37158) 2024-09-10T08:23:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.360+0000 7fb67fe00640 1 -- 172.21.15.2:0/4025590980 learned_addr learned my addr 172.21.15.2:0/4025590980 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.360+0000 7fb684a00640 1 --2- 172.21.15.2:0/4025590980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6accc30 0x558dd6adeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.360+0000 7fb67fe00640 1 -- 172.21.15.2:0/4025590980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6accc30 msgr2=0x558dd6adeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.360+0000 7fb67fe00640 1 --2- 172.21.15.2:0/4025590980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6accc30 0x558dd6adeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.360+0000 7fb67fe00640 1 -- 172.21.15.2:0/4025590980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6ac7ba0 msgr2=0x558dd6ad8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.360+0000 7fb67fe00640 1 --2- 172.21.15.2:0/4025590980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.360+0000 7fb67fe00640 1 -- 172.21.15.2:0/4025590980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dd6ad0450 con 0x558dd6ac7260 2024-09-10T08:23:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.360+0000 7fb67f400640 1 --2- 172.21.15.2:0/4025590980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.360+0000 7fb67fe00640 1 --2- 172.21.15.2:0/4025590980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 0x558dd6adeab0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb6700985f0 tx=0x7fb6700056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=806489e60fa2ddba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.361+0000 7fb66be00640 1 -- 172.21.15.2:0/4025590980 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67009c320 con 0x558dd6ac7260 2024-09-10T08:23:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.361+0000 7fb66be00640 1 -- 172.21.15.2:0/4025590980 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb67009bc20 con 0x558dd6ac7260 2024-09-10T08:23:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.361+0000 7fb66be00640 1 -- 172.21.15.2:0/4025590980 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6700a4640 con 0x558dd6ac7260 2024-09-10T08:23:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.361+0000 7fb6898af740 1 -- 172.21.15.2:0/4025590980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 msgr2=0x558dd6adeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.361+0000 7fb6898af740 1 --2- 172.21.15.2:0/4025590980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 0x558dd6adeab0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb6700985f0 tx=0x7fb6700056b0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.362+0000 7fb6898af740 1 -- 172.21.15.2:0/4025590980 shutdown_connections 2024-09-10T08:23:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.362+0000 7fb6898af740 1 --2- 172.21.15.2:0/4025590980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6accc30 0x558dd6adeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.362+0000 7fb6898af740 1 --2- 172.21.15.2:0/4025590980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.362+0000 7fb6898af740 1 --2- 172.21.15.2:0/4025590980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 0x558dd6adeab0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.362+0000 7fb6898af740 1 -- 172.21.15.2:0/4025590980 >> 172.21.15.2:0/4025590980 conn(0x558dd6ac28d0 msgr2=0x558dd6acc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.362+0000 7fb6898af740 1 -- 172.21.15.2:0/4025590980 shutdown_connections 2024-09-10T08:23:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.362+0000 7fb6898af740 1 -- 172.21.15.2:0/4025590980 wait complete. 2024-09-10T08:23:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.362+0000 7fb6898af740 1 -- start start 2024-09-10T08:23:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.362+0000 7fb6898af740 1 Processor -- start 2024-09-10T08:23:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb6898af740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 0x558dd6ad8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb6898af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb6898af740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 0x558dd6ad5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb6898af740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801af2f0 con 0x558dd6accc30 2024-09-10T08:23:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb6898af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801b0760 con 0x558dd6ac7ba0 2024-09-10T08:23:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb6898af740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801af9c0 con 0x558dd6ac7260 2024-09-10T08:23:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb6898af740 10 client.0 _ll_register_callbacks cb 0x7fb678009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb684a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 0x558dd6ad5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb67fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 0x558dd6ad8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb67f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb684a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 0x558dd6ad5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60134/0 (socket says 172.21.15.2:60134) 2024-09-10T08:23:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb67fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 0x558dd6ad8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:37166/0 (socket says 172.21.15.2:37166) 2024-09-10T08:23:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb684a00640 1 -- 172.21.15.2:0/861260554 learned_addr learned my addr 172.21.15.2:0/861260554 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb684a00640 1 -- 172.21.15.2:0/861260554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 msgr2=0x558dd6ad8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb684a00640 1 --2- 172.21.15.2:0/861260554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 0x558dd6ad8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb684a00640 1 -- 172.21.15.2:0/861260554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7ba0 msgr2=0x558dd6ad5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb684a00640 1 --2- 172.21.15.2:0/861260554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb684a00640 1 -- 172.21.15.2:0/861260554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dd6b45670 con 0x558dd6accc30 2024-09-10T08:23:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb67fe00640 1 --2- 172.21.15.2:0/861260554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 0x558dd6ad8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.363+0000 7fb67f400640 1 --2- 172.21.15.2:0/861260554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.364+0000 7fb684a00640 1 --2- 172.21.15.2:0/861260554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 0x558dd6ad5890 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb67800fbc0 tx=0x7fb678004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c024940a4397cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.364+0000 7fb669600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:23:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.364+0000 7fb669600640 1 -- 172.21.15.2:0/861260554 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6780047a0 con 0x558dd6accc30 2024-09-10T08:23:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.364+0000 7fb669600640 1 -- 172.21.15.2:0/861260554 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb678018030 con 0x558dd6accc30 2024-09-10T08:23:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.364+0000 7fb6898af740 1 -- 172.21.15.2:0/861260554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb67009bc20 con 0x558dd6accc30 2024-09-10T08:23:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.364+0000 7fb669600640 1 -- 172.21.15.2:0/861260554 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb678037050 con 0x558dd6accc30 2024-09-10T08:23:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.364+0000 7fb6898af740 10 client.19881 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.364+0000 7fb6898af740 1 -- 172.21.15.2:0/861260554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558dd6b469c0 con 0x558dd6accc30 2024-09-10T08:23:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.365+0000 7fb669600640 1 -- 172.21.15.2:0/861260554 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb67009bc20 con 0x558dd6accc30 2024-09-10T08:23:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.365+0000 7fb669600640 1 -- 172.21.15.2:0/861260554 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb6780057d0 con 0x558dd6accc30 2024-09-10T08:23:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.365+0000 7fb6898af740 10 client.19881 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.365+0000 7fb6898af740 10 client.19881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:23:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.365+0000 7fb6898af740 20 client.19881 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.365+0000 7fb6898af740 1 --2- 172.21.15.2:0/861260554 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558dd6b473f0 0x558dd6b49810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.365+0000 7fb6898af740 4 client.19881 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:23:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.365+0000 7fb6898af740 1 -- 172.21.15.2:0/861260554 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558dd6a33b80 con 0x558dd6b473f0 2024-09-10T08:23:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.365+0000 7fb67fe00640 1 --2- 172.21.15.2:0/861260554 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558dd6b473f0 0x558dd6b49810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.366+0000 7fb67fe00640 1 --2- 172.21.15.2:0/861260554 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558dd6b473f0 0x558dd6b49810 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9273c0dd91885d server_cookie=6b3bc40c949ac75a in_seq=0 out_seq=0 2024-09-10T08:23:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.366+0000 7fb669600640 10 client.19881 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb669600640 1 -- 172.21.15.2:0/861260554 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558dd6a33b80 con 0x558dd6b473f0 2024-09-10T08:23:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb669600640 10 client.19881 handle_command_reply: tid=0 2024-09-10T08:23:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb6898af740 1 client.19881 shutdown 2024-09-10T08:23:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb6898af740 1 -- 172.21.15.2:0/861260554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 msgr2=0x558dd6ad5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb6898af740 1 --2- 172.21.15.2:0/861260554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 0x558dd6ad5890 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb67800fbc0 tx=0x7fb678004a40 comp rx=0 tx=0).stop 2024-09-10T08:23:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb6898af740 1 -- 172.21.15.2:0/861260554 shutdown_connections 2024-09-10T08:23:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb6898af740 1 --2- 172.21.15.2:0/861260554 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x558dd6b473f0 0x558dd6b49810 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb6898af740 1 --2- 172.21.15.2:0/861260554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dd6accc30 0x558dd6ad5890 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb6898af740 1 --2- 172.21.15.2:0/861260554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558dd6ac7ba0 0x558dd6ad5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb6898af740 1 --2- 172.21.15.2:0/861260554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558dd6ac7260 0x558dd6ad8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb669600640 0 client.19881 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb6898af740 1 -- 172.21.15.2:0/861260554 >> 172.21.15.2:0/861260554 conn(0x558dd6ac28d0 msgr2=0x558dd6acbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb6898af740 1 -- 172.21.15.2:0/861260554 shutdown_connections 2024-09-10T08:23:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.378+0000 7fb6898af740 1 -- 172.21.15.2:0/861260554 wait complete. 2024-09-10T08:23:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.379+0000 7fb6898af740 20 client.19881 trim_cache size 0 max 16384 2024-09-10T08:23:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:23:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:23:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:23:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:23:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.379+0000 7fb685400640 1 -- 172.21.15.2:0/2345095081 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb6640a2c00 msgr2=0x7fb6640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.379+0000 7fb685400640 1 --2- 172.21.15.2:0/2345095081 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb6640a2c00 0x7fb6640a50c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb678009fd0 tx=0x7fb678009500 comp rx=0 tx=0).stop 2024-09-10T08:23:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.379+0000 7fb685400640 1 -- 172.21.15.2:0/2345095081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680074680 msgr2=0x7fb68019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.379+0000 7fb685400640 1 --2- 172.21.15.2:0/2345095081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680074680 0x7fb68019e3e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb67000e9f0 tx=0x7fb67000edf0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.379+0000 7fb685400640 1 -- 172.21.15.2:0/2345095081 shutdown_connections 2024-09-10T08:23:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.379+0000 7fb685400640 1 --2- 172.21.15.2:0/2345095081 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb6640a2c00 0x7fb6640a50c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.379+0000 7fb685400640 1 --2- 172.21.15.2:0/2345095081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6801a3980 0x7fb6801a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.379+0000 7fb685400640 1 --2- 172.21.15.2:0/2345095081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb680074fc0 0x7fb68019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.379+0000 7fb685400640 1 --2- 172.21.15.2:0/2345095081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb680074680 0x7fb68019e3e0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.379+0000 7fb685400640 1 -- 172.21.15.2:0/2345095081 >> 172.21.15.2:0/2345095081 conn(0x7fb6800fd0d0 msgr2=0x7fb6801101b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.380+0000 7fb685400640 1 -- 172.21.15.2:0/2345095081 shutdown_connections 2024-09-10T08:23:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:23:52.380+0000 7fb685400640 1 -- 172.21.15.2:0/2345095081 wait complete. 2024-09-10T08:23:52.506 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:23:53.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.116+0000 7fcfb9e00640 1 -- 172.21.15.2:0/3330581652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb4103ff0 msgr2=0x7fcfb41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:53.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.116+0000 7fcfb9e00640 1 --2- 172.21.15.2:0/3330581652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb4103ff0 0x7fcfb41043d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009900 tx=0x7fcfa402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.116+0000 7fcfb9e00640 1 -- 172.21.15.2:0/3330581652 shutdown_connections 2024-09-10T08:23:53.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.116+0000 7fcfb9e00640 1 --2- 172.21.15.2:0/3330581652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfb4109120 0x7fcfb4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.116+0000 7fcfb9e00640 1 --2- 172.21.15.2:0/3330581652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb41049a0 0x7fcfb41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.116+0000 7fcfb9e00640 1 --2- 172.21.15.2:0/3330581652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb4103ff0 0x7fcfb41043d0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.116+0000 7fcfb9e00640 1 -- 172.21.15.2:0/3330581652 >> 172.21.15.2:0/3330581652 conn(0x7fcfb40ffa90 msgr2=0x7fcfb4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:53.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.116+0000 7fcfb9e00640 1 -- 172.21.15.2:0/3330581652 shutdown_connections 2024-09-10T08:23:53.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.116+0000 7fcfb9e00640 1 -- 172.21.15.2:0/3330581652 wait complete. 2024-09-10T08:23:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.117+0000 7fcfb9e00640 1 Processor -- start 2024-09-10T08:23:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.117+0000 7fcfb9e00640 1 -- start start 2024-09-10T08:23:53.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.117+0000 7fcfb9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb4103ff0 0x7fcfb419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfb9e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb41049a0 0x7fcfb419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfb9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfb4109120 0x7fcfb419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfb9e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41141a0 con 0x7fcfb41049a0 2024-09-10T08:23:53.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfb9e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4114020 con 0x7fcfb4103ff0 2024-09-10T08:23:53.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfb9e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4114320 con 0x7fcfb4109120 2024-09-10T08:23:53.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfaa000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb41049a0 0x7fcfb419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfaa000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb41049a0 0x7fcfb419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60164/0 (socket says 172.21.15.2:60164) 2024-09-10T08:23:53.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfaa000640 1 -- 172.21.15.2:0/2505342208 learned_addr learned my addr 172.21.15.2:0/2505342208 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:53.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfb2a00640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb4103ff0 0x7fcfb419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfb3400640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfb4109120 0x7fcfb419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfaa000640 1 -- 172.21.15.2:0/2505342208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfb4109120 msgr2=0x7fcfb419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfaa000640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfb4109120 0x7fcfb419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfaa000640 1 -- 172.21.15.2:0/2505342208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb4103ff0 msgr2=0x7fcfb419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfaa000640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb4103ff0 0x7fcfb419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfaa000640 1 -- 172.21.15.2:0/2505342208 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb419f690 con 0x7fcfb41049a0 2024-09-10T08:23:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfb2a00640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb4103ff0 0x7fcfb419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.118+0000 7fcfb3400640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfb4109120 0x7fcfb419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.119+0000 7fcfaa000640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb41049a0 0x7fcfb419ac20 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00ea60 tx=0x7fcf9c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.119+0000 7fcfb8a00640 1 -- 172.21.15.2:0/2505342208 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c00cde0 con 0x7fcfb41049a0 2024-09-10T08:23:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.119+0000 7fcfb8a00640 1 -- 172.21.15.2:0/2505342208 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf9c0035f0 con 0x7fcfb41049a0 2024-09-10T08:23:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.119+0000 7fcfb8a00640 1 -- 172.21.15.2:0/2505342208 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c010670 con 0x7fcfb41049a0 2024-09-10T08:23:53.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.119+0000 7fcfb9e00640 1 -- 172.21.15.2:0/2505342208 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb419f980 con 0x7fcfb41049a0 2024-09-10T08:23:53.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.119+0000 7fcfb9e00640 1 -- 172.21.15.2:0/2505342208 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb41a7260 con 0x7fcfb41049a0 2024-09-10T08:23:53.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.122+0000 7fcfb8a00640 1 -- 172.21.15.2:0/2505342208 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcf9c0037a0 con 0x7fcfb41049a0 2024-09-10T08:23:53.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.122+0000 7fcfbc846740 1 Processor -- start 2024-09-10T08:23:53.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.122+0000 7fcfbc846740 1 -- start start 2024-09-10T08:23:53.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.123+0000 7fcfbc846740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5310 0x55f1b60b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.123+0000 7fcfbc846740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5c30 0x55f1b60bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.123+0000 7fcfbc846740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 0x55f1b60c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.123+0000 7fcfbc846740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f1b601b650 con 0x55f1b60b5310 2024-09-10T08:23:53.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.123+0000 7fcfbc846740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f1b5fc2f10 con 0x55f1b60b5c30 2024-09-10T08:23:53.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.123+0000 7fcfbc846740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f1b6017610 con 0x55f1b60c04f0 2024-09-10T08:23:53.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.123+0000 7fcfb3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5310 0x55f1b60b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.123+0000 7fcfaa000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 0x55f1b60c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.123+0000 7fcfb3400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5310 0x55f1b60b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60166/0 (socket says 172.21.15.2:60166) 2024-09-10T08:23:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.123+0000 7fcfb3400640 1 -- 172.21.15.2:0/3163677369 learned_addr learned my addr 172.21.15.2:0/3163677369 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.123+0000 7fcfb2a00640 1 --2- 172.21.15.2:0/3163677369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5c30 0x55f1b60bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.124+0000 7fcfb3400640 1 -- 172.21.15.2:0/3163677369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 msgr2=0x55f1b60c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.124+0000 7fcfb3400640 1 --2- 172.21.15.2:0/3163677369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 0x55f1b60c28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.124+0000 7fcfb3400640 1 -- 172.21.15.2:0/3163677369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5c30 msgr2=0x55f1b60bffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.124+0000 7fcfb3400640 1 --2- 172.21.15.2:0/3163677369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5c30 0x55f1b60bffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.124+0000 7fcfb3400640 1 -- 172.21.15.2:0/3163677369 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b5d2a580 con 0x55f1b60b5310 2024-09-10T08:23:53.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.124+0000 7fcfb8a00640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf980a2c00 0x7fcf980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.124+0000 7fcfb8a00640 1 -- 172.21.15.2:0/2505342208 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcf9c0cf700 con 0x7fcfb41049a0 2024-09-10T08:23:53.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.124+0000 7fcfb3400640 1 --2- 172.21.15.2:0/3163677369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5310 0x55f1b60b56f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fcfac011790 tx=0x7fcfac011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec0ef6dde4b4377 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:53.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.124+0000 7fcfb2a00640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf980a2c00 0x7fcf980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.125+0000 7fcfa3e00640 1 -- 172.21.15.2:0/3163677369 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac003c30 con 0x55f1b60b5310 2024-09-10T08:23:53.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.125+0000 7fcfa3e00640 1 -- 172.21.15.2:0/3163677369 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfac00fe80 con 0x55f1b60b5310 2024-09-10T08:23:53.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.125+0000 7fcfa3e00640 1 -- 172.21.15.2:0/3163677369 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac0193e0 con 0x55f1b60b5310 2024-09-10T08:23:53.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.125+0000 7fcfbc846740 1 -- 172.21.15.2:0/3163677369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5310 msgr2=0x55f1b60b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:53.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.125+0000 7fcfbc846740 1 --2- 172.21.15.2:0/3163677369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5310 0x55f1b60b56f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fcfac011790 tx=0x7fcfac011c50 comp rx=0 tx=0).stop 2024-09-10T08:23:53.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.125+0000 7fcfb2a00640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf980a2c00 0x7fcf980a50c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4000c00 tx=0x7fcfa4031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:53.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.125+0000 7fcfbc846740 1 -- 172.21.15.2:0/3163677369 shutdown_connections 2024-09-10T08:23:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.125+0000 7fcfbc846740 1 --2- 172.21.15.2:0/3163677369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 0x55f1b60c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.125+0000 7fcfbc846740 1 --2- 172.21.15.2:0/3163677369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5c30 0x55f1b60bffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.125+0000 7fcfbc846740 1 --2- 172.21.15.2:0/3163677369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5310 0x55f1b60b56f0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.125+0000 7fcfbc846740 1 -- 172.21.15.2:0/3163677369 >> 172.21.15.2:0/3163677369 conn(0x55f1b60b18a0 msgr2=0x55f1b60b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.126+0000 7fcfbc846740 1 -- 172.21.15.2:0/3163677369 shutdown_connections 2024-09-10T08:23:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.126+0000 7fcfbc846740 1 -- 172.21.15.2:0/3163677369 wait complete. 2024-09-10T08:23:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.126+0000 7fcfbc846740 1 -- start start 2024-09-10T08:23:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.126+0000 7fcfbc846740 1 Processor -- start 2024-09-10T08:23:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.127+0000 7fcfbc846740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60bd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.127+0000 7fcfbc846740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60b5c30 0x55f1b60bd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.127+0000 7fcfbc846740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60c04f0 0x55f1b60bded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.127+0000 7fcfbc846740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f1b60c30c0 con 0x55f1b60c04f0 2024-09-10T08:23:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.127+0000 7fcfbc846740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f1b5fd6350 con 0x55f1b60b5310 2024-09-10T08:23:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.127+0000 7fcfbc846740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f1b60c3240 con 0x55f1b60b5c30 2024-09-10T08:23:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.127+0000 7fcfbc846740 10 client.0 _ll_register_callbacks cb 0x55f1b5ff9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.127+0000 7fcfb2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60bd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.127+0000 7fcfaa000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60b5c30 0x55f1b60bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.127+0000 7fcfb2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60bd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:48288/0 (socket says 172.21.15.2:48288) 2024-09-10T08:23:53.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.127+0000 7fcfb2a00640 1 -- 172.21.15.2:0/182729991 learned_addr learned my addr 172.21.15.2:0/182729991 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.127+0000 7fcfb3400640 1 --2- 172.21.15.2:0/182729991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60c04f0 0x55f1b60bded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.128+0000 7fcfaa000640 1 -- 172.21.15.2:0/182729991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 msgr2=0x55f1b60bd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.128+0000 7fcfaa000640 1 --2- 172.21.15.2:0/182729991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60bd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.128+0000 7fcfaa000640 1 -- 172.21.15.2:0/182729991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60c04f0 msgr2=0x55f1b60bded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.128+0000 7fcfaa000640 1 --2- 172.21.15.2:0/182729991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60c04f0 0x55f1b60bded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.128+0000 7fcfaa000640 1 -- 172.21.15.2:0/182729991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b5df4960 con 0x55f1b60b5c30 2024-09-10T08:23:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.128+0000 7fcfaa000640 1 --2- 172.21.15.2:0/182729991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60b5c30 0x55f1b60bd990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55f1b60b2c20 tx=0x7fcf9c0d3040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a2cd969643927ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:53.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.128+0000 7fcfa0c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:23:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.128+0000 7fcfa0c00640 1 -- 172.21.15.2:0/182729991 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c09b650 con 0x55f1b60b5c30 2024-09-10T08:23:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.128+0000 7fcfbc846740 1 -- 172.21.15.2:0/182729991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f1b6028bc0 con 0x55f1b60b5c30 2024-09-10T08:23:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.128+0000 7fcfa0c00640 1 -- 172.21.15.2:0/182729991 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcf9c098180 con 0x55f1b60b5c30 2024-09-10T08:23:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.128+0000 7fcfa0c00640 1 -- 172.21.15.2:0/182729991 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf9c09b470 con 0x55f1b60b5c30 2024-09-10T08:23:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.128+0000 7fcfbc846740 10 client.29839 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.128+0000 7fcfbc846740 1 -- 172.21.15.2:0/182729991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1b60ced90 con 0x55f1b60b5c30 2024-09-10T08:23:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.129+0000 7fcfa0c00640 1 -- 172.21.15.2:0/182729991 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f1b6028bc0 con 0x55f1b60b5c30 2024-09-10T08:23:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.129+0000 7fcfa0c00640 1 -- 172.21.15.2:0/182729991 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcf9c0ba260 con 0x55f1b60b5c30 2024-09-10T08:23:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.129+0000 7fcfbc846740 10 client.29839 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.129+0000 7fcfbc846740 10 client.29839 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:23:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.129+0000 7fcfbc846740 20 client.29839 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.129+0000 7fcfbc846740 1 --2- 172.21.15.2:0/182729991 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f1b60d0130 0x55f1b60d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.129+0000 7fcfbc846740 4 client.29839 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:23:53.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.129+0000 7fcfbc846740 1 -- 172.21.15.2:0/182729991 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1b60230d0 con 0x55f1b60d0130 2024-09-10T08:23:53.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.129+0000 7fcfb2a00640 1 --2- 172.21.15.2:0/182729991 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f1b60d0130 0x55f1b60d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.130+0000 7fcfb2a00640 1 --2- 172.21.15.2:0/182729991 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f1b60d0130 0x55f1b60d2540 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1137031620b29c2 server_cookie=a522f7fe21056000 in_seq=0 out_seq=0 2024-09-10T08:23:53.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.130+0000 7fcfa0c00640 10 client.29839 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:53.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfa0c00640 1 -- 172.21.15.2:0/182729991 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1b60230d0 con 0x55f1b60d0130 2024-09-10T08:23:53.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfa0c00640 10 client.29839 handle_command_reply: tid=0 2024-09-10T08:23:53.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfbc846740 1 client.29839 shutdown 2024-09-10T08:23:53.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfbc846740 1 -- 172.21.15.2:0/182729991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60b5c30 msgr2=0x55f1b60bd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:53.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfbc846740 1 --2- 172.21.15.2:0/182729991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60b5c30 0x55f1b60bd990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55f1b60b2c20 tx=0x7fcf9c0d3040 comp rx=0 tx=0).stop 2024-09-10T08:23:53.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfbc846740 1 -- 172.21.15.2:0/182729991 shutdown_connections 2024-09-10T08:23:53.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfbc846740 1 --2- 172.21.15.2:0/182729991 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f1b60d0130 0x55f1b60d2540 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfbc846740 1 --2- 172.21.15.2:0/182729991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60c04f0 0x55f1b60bded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfbc846740 1 --2- 172.21.15.2:0/182729991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60b5c30 0x55f1b60bd990 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfbc846740 1 --2- 172.21.15.2:0/182729991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60bd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfa0c00640 0 client.29839 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:53.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfbc846740 1 -- 172.21.15.2:0/182729991 >> 172.21.15.2:0/182729991 conn(0x55f1b60b18a0 msgr2=0x55f1b60b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:53.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfbc846740 1 -- 172.21.15.2:0/182729991 shutdown_connections 2024-09-10T08:23:53.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.131+0000 7fcfbc846740 1 -- 172.21.15.2:0/182729991 wait complete. 2024-09-10T08:23:53.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.132+0000 7fcfbc846740 20 client.29839 trim_cache size 0 max 16384 2024-09-10T08:23:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.149+0000 7fcfbc846740 1 Processor -- start 2024-09-10T08:23:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.149+0000 7fcfbc846740 1 -- start start 2024-09-10T08:23:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfbc846740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60d7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfbc846740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5c30 0x55f1b60d8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfbc846740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 0x55f1b60ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfbc846740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41b3880 con 0x55f1b60b5c30 2024-09-10T08:23:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfbc846740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41b1150 con 0x55f1b60b5310 2024-09-10T08:23:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfbc846740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41aefa0 con 0x55f1b60c04f0 2024-09-10T08:23:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfb2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60d7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfb2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60d7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:48290/0 (socket says 172.21.15.2:48290) 2024-09-10T08:23:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfb2a00640 1 -- 172.21.15.2:0/3620424415 learned_addr learned my addr 172.21.15.2:0/3620424415 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfaa000640 1 --2- 172.21.15.2:0/3620424415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5c30 0x55f1b60d8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfb3400640 1 --2- 172.21.15.2:0/3620424415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 0x55f1b60ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfb2a00640 1 -- 172.21.15.2:0/3620424415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 msgr2=0x55f1b60ba6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfb2a00640 1 --2- 172.21.15.2:0/3620424415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 0x55f1b60ba6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfb2a00640 1 -- 172.21.15.2:0/3620424415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5c30 msgr2=0x55f1b60d8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfb2a00640 1 --2- 172.21.15.2:0/3620424415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5c30 0x55f1b60d8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.150+0000 7fcfb2a00640 1 -- 172.21.15.2:0/3620424415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b60bfd20 con 0x55f1b60b5310 2024-09-10T08:23:53.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.151+0000 7fcfb2a00640 1 --2- 172.21.15.2:0/3620424415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60d7e10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcfa402a9e0 tx=0x7fcfa403f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddeaed0add389a4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:53.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.151+0000 7fcfa3400640 1 -- 172.21.15.2:0/3620424415 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4035040 con 0x55f1b60b5310 2024-09-10T08:23:53.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.151+0000 7fcfa3400640 1 -- 172.21.15.2:0/3620424415 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa4058030 con 0x55f1b60b5310 2024-09-10T08:23:53.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.151+0000 7fcfa3400640 1 -- 172.21.15.2:0/3620424415 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40049c0 con 0x55f1b60b5310 2024-09-10T08:23:53.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.152+0000 7fcfbc846740 1 -- 172.21.15.2:0/3620424415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 msgr2=0x55f1b60d7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:53.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.152+0000 7fcfbc846740 1 --2- 172.21.15.2:0/3620424415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60d7e10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcfa402a9e0 tx=0x7fcfa403f040 comp rx=0 tx=0).stop 2024-09-10T08:23:53.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.152+0000 7fcfbc846740 1 -- 172.21.15.2:0/3620424415 shutdown_connections 2024-09-10T08:23:53.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.152+0000 7fcfbc846740 1 --2- 172.21.15.2:0/3620424415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 0x55f1b60ba6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.152+0000 7fcfbc846740 1 --2- 172.21.15.2:0/3620424415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5c30 0x55f1b60d8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.152+0000 7fcfbc846740 1 --2- 172.21.15.2:0/3620424415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60d7e10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.152+0000 7fcfbc846740 1 -- 172.21.15.2:0/3620424415 >> 172.21.15.2:0/3620424415 conn(0x55f1b60b18a0 msgr2=0x55f1b60bf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:53.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.152+0000 7fcfbc846740 1 -- 172.21.15.2:0/3620424415 shutdown_connections 2024-09-10T08:23:53.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.152+0000 7fcfbc846740 1 -- 172.21.15.2:0/3620424415 wait complete. 2024-09-10T08:23:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.152+0000 7fcfbc846740 1 -- start start 2024-09-10T08:23:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfbc846740 1 Processor -- start 2024-09-10T08:23:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfbc846740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfbc846740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5c30 0x55f1b60c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfbc846740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 0x55f1b60c5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfbc846740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41af610 con 0x55f1b60b5c30 2024-09-10T08:23:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfbc846740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41b0a80 con 0x55f1b60b5310 2024-09-10T08:23:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfbc846740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41afce0 con 0x55f1b60c04f0 2024-09-10T08:23:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfbc846740 10 client.0 _ll_register_callbacks cb 0x7fcf9c0d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfaa000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5c30 0x55f1b60c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfb2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfb2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:48300/0 (socket says 172.21.15.2:48300) 2024-09-10T08:23:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfb2a00640 1 -- 172.21.15.2:0/670065845 learned_addr learned my addr 172.21.15.2:0/670065845 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfaa000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5c30 0x55f1b60c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60198/0 (socket says 172.21.15.2:60198) 2024-09-10T08:23:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.153+0000 7fcfb3400640 1 --2- 172.21.15.2:0/670065845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 0x55f1b60c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.154+0000 7fcfb2a00640 1 -- 172.21.15.2:0/670065845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 msgr2=0x55f1b60c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.154+0000 7fcfb2a00640 1 --2- 172.21.15.2:0/670065845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 0x55f1b60c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.154+0000 7fcfb2a00640 1 -- 172.21.15.2:0/670065845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5c30 msgr2=0x55f1b60c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.154+0000 7fcfb2a00640 1 --2- 172.21.15.2:0/670065845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5c30 0x55f1b60c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.154+0000 7fcfb2a00640 1 -- 172.21.15.2:0/670065845 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b60c5c90 con 0x55f1b60b5310 2024-09-10T08:23:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.154+0000 7fcfaa000640 1 --2- 172.21.15.2:0/670065845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5c30 0x55f1b60c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.154+0000 7fcfb2a00640 1 --2- 172.21.15.2:0/670065845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60c8670 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4004650 tx=0x7fcfa4003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7763b4c388540db8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.154+0000 7fcfa0c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:23:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.154+0000 7fcfa0c00640 1 -- 172.21.15.2:0/670065845 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4031d80 con 0x55f1b60b5310 2024-09-10T08:23:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.154+0000 7fcfbc846740 1 -- 172.21.15.2:0/670065845 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa4058030 con 0x55f1b60b5310 2024-09-10T08:23:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.154+0000 7fcfa0c00640 1 -- 172.21.15.2:0/670065845 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcfa40035f0 con 0x55f1b60b5310 2024-09-10T08:23:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.155+0000 7fcfa0c00640 1 -- 172.21.15.2:0/670065845 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcfa40037f0 con 0x55f1b60b5310 2024-09-10T08:23:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.155+0000 7fcfbc846740 10 client.29918 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.155+0000 7fcfbc846740 1 -- 172.21.15.2:0/670065845 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1b613b8d0 con 0x55f1b60b5310 2024-09-10T08:23:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.155+0000 7fcfa0c00640 1 -- 172.21.15.2:0/670065845 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa4058030 con 0x55f1b60b5310 2024-09-10T08:23:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.155+0000 7fcfa0c00640 1 -- 172.21.15.2:0/670065845 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcfa4044070 con 0x55f1b60b5310 2024-09-10T08:23:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.155+0000 7fcfbc846740 10 client.29918 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.155+0000 7fcfbc846740 10 client.29918 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:23:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.155+0000 7fcfbc846740 20 client.29918 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.155+0000 7fcfbc846740 1 --2- 172.21.15.2:0/670065845 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f1b613bf90 0x55f1b613e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.155+0000 7fcfbc846740 4 client.29918 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b597e0bd", "release": true, "await": true}] 2024-09-10T08:23:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.155+0000 7fcfbc846740 1 -- 172.21.15.2:0/670065845 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b597e0bd", "release": true, "await": true}) -- 0x55f1b5f9f430 con 0x55f1b613bf90 2024-09-10T08:23:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.156+0000 7fcfaa000640 1 --2- 172.21.15.2:0/670065845 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f1b613bf90 0x55f1b613e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.156+0000 7fcfaa000640 1 --2- 172.21.15.2:0/670065845 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f1b613bf90 0x55f1b613e3b0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7481092934fea23f server_cookie=ff7ac2aa2b1d5cfb in_seq=0 out_seq=0 2024-09-10T08:23:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.156+0000 7fcfa0c00640 10 client.29918 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfa0c00640 1 -- 172.21.15.2:0/670065845 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f1b5f9f430 con 0x55f1b613bf90 2024-09-10T08:23:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfa0c00640 10 client.29918 handle_command_reply: tid=0 2024-09-10T08:23:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfbc846740 1 client.29918 shutdown 2024-09-10T08:23:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfbc846740 1 -- 172.21.15.2:0/670065845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 msgr2=0x55f1b60c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfbc846740 1 --2- 172.21.15.2:0/670065845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60c8670 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4004650 tx=0x7fcfa4003e90 comp rx=0 tx=0).stop 2024-09-10T08:23:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfbc846740 1 -- 172.21.15.2:0/670065845 shutdown_connections 2024-09-10T08:23:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfbc846740 1 --2- 172.21.15.2:0/670065845 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55f1b613bf90 0x55f1b613e3b0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfbc846740 1 --2- 172.21.15.2:0/670065845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1b60c04f0 0x55f1b60c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfbc846740 1 --2- 172.21.15.2:0/670065845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1b60b5c30 0x55f1b60c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfbc846740 1 --2- 172.21.15.2:0/670065845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1b60b5310 0x55f1b60c8670 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfa0c00640 0 client.29918 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfbc846740 1 -- 172.21.15.2:0/670065845 >> 172.21.15.2:0/670065845 conn(0x55f1b60b18a0 msgr2=0x55f1b60b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfbc846740 1 -- 172.21.15.2:0/670065845 shutdown_connections 2024-09-10T08:23:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.158+0000 7fcfbc846740 1 -- 172.21.15.2:0/670065845 wait complete. 2024-09-10T08:23:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.159+0000 7fcfbc846740 20 client.29918 trim_cache size 0 max 16384 2024-09-10T08:23:53.162 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:23:53.162 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:23:53.162 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:23:53.162 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 56, 2024-09-10T08:23:53.162 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:23:53.162 INFO:teuthology.orchestra.run.smithi002.stdout: "b597e0bd": { 2024-09-10T08:23:53.162 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 56, 2024-09-10T08:23:53.162 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:23:53.162 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:23:53.162 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:23:53.162 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:23:53.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:23:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:23:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 139.8, 2024-09-10T08:23:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:23:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:23:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:23:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:23:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:23:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:23:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:53.164 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:23:53.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.159+0000 7fcfb9e00640 1 -- 172.21.15.2:0/2505342208 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf980a2c00 msgr2=0x7fcf980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:53.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.159+0000 7fcfb9e00640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf980a2c00 0x7fcf980a50c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4000c00 tx=0x7fcfa4031510 comp rx=0 tx=0).stop 2024-09-10T08:23:53.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.159+0000 7fcfb9e00640 1 -- 172.21.15.2:0/2505342208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb41049a0 msgr2=0x7fcfb419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:53.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.159+0000 7fcfb9e00640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb41049a0 0x7fcfb419ac20 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00ea60 tx=0x7fcf9c00ef20 comp rx=0 tx=0).stop 2024-09-10T08:23:53.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.159+0000 7fcfb9e00640 1 -- 172.21.15.2:0/2505342208 shutdown_connections 2024-09-10T08:23:53.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.159+0000 7fcfb9e00640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf980a2c00 0x7fcf980a50c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.159+0000 7fcfb9e00640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfb4109120 0x7fcfb419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.159+0000 7fcfb9e00640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfb41049a0 0x7fcfb419ac20 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.159+0000 7fcfb9e00640 1 --2- 172.21.15.2:0/2505342208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb4103ff0 0x7fcfb419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:53.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.159+0000 7fcfb9e00640 1 -- 172.21.15.2:0/2505342208 >> 172.21.15.2:0/2505342208 conn(0x7fcfb40ffa90 msgr2=0x7fcfb4100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:53.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.160+0000 7fcfb9e00640 1 -- 172.21.15.2:0/2505342208 shutdown_connections 2024-09-10T08:23:53.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:53.160+0000 7fcfb9e00640 1 -- 172.21.15.2:0/2505342208 wait complete. 2024-09-10T08:23:53.836 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:53 smithi002 ceph-mon[30563]: pgmap v859: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-10T08:23:53.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:53 smithi155 ceph-mon[35405]: pgmap v859: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-10T08:23:53.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:53 smithi107 ceph-mon[35048]: pgmap v859: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-10T08:23:54.050 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b597e0bd', total seconds elapsed: 24.0 2024-09-10T08:23:54.050 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.85 2024-09-10T08:23:54.050 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:23:54.459 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:23:54.834 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:54 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:23:54.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:54 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:23:54.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:54 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:23:55.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.113+0000 7ff15d800640 1 -- 172.21.15.2:0/3055893474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff158103ff0 msgr2=0x7ff1581043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:55.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.113+0000 7ff15d800640 1 --2- 172.21.15.2:0/3055893474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff158103ff0 0x7ff1581043d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff148009900 tx=0x7ff14802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:23:55.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.113+0000 7ff15d800640 1 -- 172.21.15.2:0/3055893474 shutdown_connections 2024-09-10T08:23:55.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.113+0000 7ff15d800640 1 --2- 172.21.15.2:0/3055893474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff158109120 0x7ff158110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:55.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.113+0000 7ff15d800640 1 --2- 172.21.15.2:0/3055893474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1581049a0 0x7ff1581089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:55.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.113+0000 7ff15d800640 1 --2- 172.21.15.2:0/3055893474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff158103ff0 0x7ff1581043d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:55.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.113+0000 7ff15d800640 1 -- 172.21.15.2:0/3055893474 >> 172.21.15.2:0/3055893474 conn(0x7ff1580ffa90 msgr2=0x7ff158101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:55.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.113+0000 7ff15d800640 1 -- 172.21.15.2:0/3055893474 shutdown_connections 2024-09-10T08:23:55.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.113+0000 7ff15d800640 1 -- 172.21.15.2:0/3055893474 wait complete. 2024-09-10T08:23:55.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.114+0000 7ff15d800640 1 Processor -- start 2024-09-10T08:23:55.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.115+0000 7ff15d800640 1 -- start start 2024-09-10T08:23:55.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.115+0000 7ff15d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff158103ff0 0x7ff15819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:55.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.115+0000 7ff15d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1581049a0 0x7ff15819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:55.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.115+0000 7ff15d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff158109120 0x7ff15819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:55.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.115+0000 7ff15d800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff158114170 con 0x7ff1581049a0 2024-09-10T08:23:55.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.115+0000 7ff15d800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff158113ff0 con 0x7ff158109120 2024-09-10T08:23:55.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.115+0000 7ff15d800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1581142f0 con 0x7ff158103ff0 2024-09-10T08:23:55.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.115+0000 7ff156a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff158103ff0 0x7ff15819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:55.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.115+0000 7ff156a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff158103ff0 0x7ff15819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:37234/0 (socket says 172.21.15.2:37234) 2024-09-10T08:23:55.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.115+0000 7ff156a00640 1 -- 172.21.15.2:0/3946618863 learned_addr learned my addr 172.21.15.2:0/3946618863 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:55.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.115+0000 7ff157400640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff158109120 0x7ff15819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:55.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.116+0000 7ff156000640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1581049a0 0x7ff15819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:55.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.116+0000 7ff157400640 1 -- 172.21.15.2:0/3946618863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff158103ff0 msgr2=0x7ff15819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:55.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.116+0000 7ff157400640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff158103ff0 0x7ff15819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:55.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.116+0000 7ff157400640 1 -- 172.21.15.2:0/3946618863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1581049a0 msgr2=0x7ff15819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:55.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.116+0000 7ff157400640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1581049a0 0x7ff15819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:55.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.116+0000 7ff157400640 1 -- 172.21.15.2:0/3946618863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15819f5a0 con 0x7ff158109120 2024-09-10T08:23:55.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.116+0000 7ff157400640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff158109120 0x7ff15819eec0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff150009fd0 tx=0x7ff15000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:55.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.117+0000 7ff14f400640 1 -- 172.21.15.2:0/3946618863 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15000ce20 con 0x7ff158109120 2024-09-10T08:23:55.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.117+0000 7ff14f400640 1 -- 172.21.15.2:0/3946618863 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff150005020 con 0x7ff158109120 2024-09-10T08:23:55.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.117+0000 7ff14f400640 1 -- 172.21.15.2:0/3946618863 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150010720 con 0x7ff158109120 2024-09-10T08:23:55.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.117+0000 7ff15d800640 1 -- 172.21.15.2:0/3946618863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15819f890 con 0x7ff158109120 2024-09-10T08:23:55.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.117+0000 7ff15d800640 1 -- 172.21.15.2:0/3946618863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1581a7170 con 0x7ff158109120 2024-09-10T08:23:55.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.118+0000 7ff15d800640 1 -- 172.21.15.2:0/3946618863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1581043d0 con 0x7ff158109120 2024-09-10T08:23:55.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.121+0000 7ff14f400640 1 -- 172.21.15.2:0/3946618863 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff150005540 con 0x7ff158109120 2024-09-10T08:23:55.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.123+0000 7ff14f400640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff13c0a2c00 0x7ff13c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:55.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.123+0000 7ff14f400640 1 -- 172.21.15.2:0/3946618863 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff1500cfa50 con 0x7ff158109120 2024-09-10T08:23:55.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.123+0000 7ff156a00640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff13c0a2c00 0x7ff13c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:55.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.124+0000 7ff156a00640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff13c0a2c00 0x7ff13c0a50c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff1480096c0 tx=0x7ff1480046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:55.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.126+0000 7ff14f400640 1 -- 172.21.15.2:0/3946618863 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff15009c560 con 0x7ff158109120 2024-09-10T08:23:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.365+0000 7ff15d800640 1 -- 172.21.15.2:0/3946618863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff15802d7c0 con 0x7ff158109120 2024-09-10T08:23:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.366+0000 7ff14f400640 1 -- 172.21.15.2:0/3946618863 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7ff15009be60 con 0x7ff158109120 2024-09-10T08:23:55.366 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:23:55.366 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:23:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:23:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.369+0000 7ff15d800640 1 -- 172.21.15.2:0/3946618863 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff13c0a2c00 msgr2=0x7ff13c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.369+0000 7ff15d800640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff13c0a2c00 0x7ff13c0a50c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff1480096c0 tx=0x7ff1480046c0 comp rx=0 tx=0).stop 2024-09-10T08:23:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.369+0000 7ff15d800640 1 -- 172.21.15.2:0/3946618863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff158109120 msgr2=0x7ff15819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.369+0000 7ff15d800640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff158109120 0x7ff15819eec0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff150009fd0 tx=0x7ff15000ef10 comp rx=0 tx=0).stop 2024-09-10T08:23:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.369+0000 7ff15d800640 1 -- 172.21.15.2:0/3946618863 shutdown_connections 2024-09-10T08:23:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.369+0000 7ff15d800640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff158109120 0x7ff15819eec0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.369+0000 7ff15d800640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1581049a0 0x7ff15819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.369+0000 7ff15d800640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff13c0a2c00 0x7ff13c0a50c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.369+0000 7ff15d800640 1 --2- 172.21.15.2:0/3946618863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff158103ff0 0x7ff15819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.369+0000 7ff15d800640 1 -- 172.21.15.2:0/3946618863 >> 172.21.15.2:0/3946618863 conn(0x7ff1580ffa90 msgr2=0x7ff158100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.369+0000 7ff15d800640 1 -- 172.21.15.2:0/3946618863 shutdown_connections 2024-09-10T08:23:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:55.369+0000 7ff15d800640 1 -- 172.21.15.2:0/3946618863 wait complete. 2024-09-10T08:23:55.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:55 smithi002 ceph-mon[30563]: pgmap v860: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-10T08:23:55.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:23:55.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:23:55.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:23:55.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:23:55.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:23:55.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:23:55.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:23:55.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:55 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3946618863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:23:55.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:55 smithi155 ceph-mon[35405]: pgmap v860: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-10T08:23:55.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:23:55.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:23:55.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:23:55.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:23:55.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:23:55.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:23:55.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:23:55.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:55 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3946618863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:23:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:55 smithi107 ceph-mon[35048]: pgmap v860: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-10T08:23:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:23:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:23:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:23:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:23:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:23:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:23:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:23:55.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:55 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3946618863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:23:56.207 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0' 2024-09-10T08:23:56.207 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0 2024-09-10T08:23:56.588 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:23:56.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:56 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi002 to 177.5M 2024-09-10T08:23:56.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:56 smithi155 ceph-mon[35405]: Unable to set osd_memory_target on smithi002 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T08:23:56.911 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:56 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi002 to 177.5M 2024-09-10T08:23:56.911 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:56 smithi002 ceph-mon[30563]: Unable to set osd_memory_target on smithi002 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T08:23:56.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:56 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi002 to 177.5M 2024-09-10T08:23:56.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:56 smithi107 ceph-mon[35048]: Unable to set osd_memory_target on smithi002 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T08:23:57.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.179+0000 7ff0b0200640 1 -- 172.21.15.2:0/309294234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0a8112a30 msgr2=0x7ff0a8114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:57.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.179+0000 7ff0b0200640 1 --2- 172.21.15.2:0/309294234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0a8112a30 0x7ff0a8114e20 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009ca0 tx=0x7ff0a002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.179+0000 7ff0b0200640 1 -- 172.21.15.2:0/309294234 shutdown_connections 2024-09-10T08:23:57.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.179+0000 7ff0b0200640 1 --2- 172.21.15.2:0/309294234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0a8112a30 0x7ff0a8114e20 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.179+0000 7ff0b0200640 1 --2- 172.21.15.2:0/309294234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0a8077040 0x7ff0a81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.179+0000 7ff0b0200640 1 --2- 172.21.15.2:0/309294234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a8076720 0x7ff0a8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.179+0000 7ff0b0200640 1 -- 172.21.15.2:0/309294234 >> 172.21.15.2:0/309294234 conn(0x7ff0a80ffa90 msgr2=0x7ff0a8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:57.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.179+0000 7ff0b0200640 1 -- 172.21.15.2:0/309294234 shutdown_connections 2024-09-10T08:23:57.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.179+0000 7ff0b0200640 1 -- 172.21.15.2:0/309294234 wait complete. 2024-09-10T08:23:57.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.180+0000 7ff0b0200640 1 Processor -- start 2024-09-10T08:23:57.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.180+0000 7ff0b0200640 1 -- start start 2024-09-10T08:23:57.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.181+0000 7ff0b0200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0a8076720 0x7ff0a81a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.181+0000 7ff0b0200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0a8077040 0x7ff0a81a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.181+0000 7ff0b0200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a8112a30 0x7ff0a819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.181+0000 7ff0b0200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81185c0 con 0x7ff0a8076720 2024-09-10T08:23:57.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.181+0000 7ff0b0200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8118440 con 0x7ff0a8077040 2024-09-10T08:23:57.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.181+0000 7ff0b0200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8118740 con 0x7ff0a8112a30 2024-09-10T08:23:57.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.181+0000 7ff0ad800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0a8076720 0x7ff0a81a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.181+0000 7ff0ad800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0a8076720 0x7ff0a81a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33310/0 (socket says 172.21.15.2:33310) 2024-09-10T08:23:57.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.181+0000 7ff0ae200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a8112a30 0x7ff0a819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.181+0000 7ff0ad800640 1 -- 172.21.15.2:0/3246494511 learned_addr learned my addr 172.21.15.2:0/3246494511 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:57.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.181+0000 7ff0ace00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0a8077040 0x7ff0a81a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.182+0000 7ff0ace00640 1 -- 172.21.15.2:0/3246494511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a8112a30 msgr2=0x7ff0a819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:57.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.182+0000 7ff0ace00640 1 --2- 172.21.15.2:0/3246494511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a8112a30 0x7ff0a819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.182+0000 7ff0ace00640 1 -- 172.21.15.2:0/3246494511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0a8076720 msgr2=0x7ff0a81a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:57.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.182+0000 7ff0ace00640 1 --2- 172.21.15.2:0/3246494511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0a8076720 0x7ff0a81a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.182+0000 7ff0ace00640 1 -- 172.21.15.2:0/3246494511 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a819efc0 con 0x7ff0a8077040 2024-09-10T08:23:57.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.182+0000 7ff0ace00640 1 --2- 172.21.15.2:0/3246494511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0a8077040 0x7ff0a81a4c10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff09400e9f0 tx=0x7ff09400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:57.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.182+0000 7ff0a6000640 1 -- 172.21.15.2:0/3246494511 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09400ccc0 con 0x7ff0a8077040 2024-09-10T08:23:57.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.182+0000 7ff0b0200640 1 -- 172.21.15.2:0/3246494511 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a819f2b0 con 0x7ff0a8077040 2024-09-10T08:23:57.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.182+0000 7ff0a6000640 1 -- 172.21.15.2:0/3246494511 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff094010910 con 0x7ff0a8077040 2024-09-10T08:23:57.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.182+0000 7ff0a6000640 1 -- 172.21.15.2:0/3246494511 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094018960 con 0x7ff0a8077040 2024-09-10T08:23:57.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.182+0000 7ff0b0200640 1 -- 172.21.15.2:0/3246494511 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a81ab5a0 con 0x7ff0a8077040 2024-09-10T08:23:57.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.186+0000 7ff0b2e68740 1 Processor -- start 2024-09-10T08:23:57.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.186+0000 7ff0a6000640 1 -- 172.21.15.2:0/3246494511 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff094010ac0 con 0x7ff0a8077040 2024-09-10T08:23:57.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.186+0000 7ff0b2e68740 1 -- start start 2024-09-10T08:23:57.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.186+0000 7ff0b2e68740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac0c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.186+0000 7ff0b2e68740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac17180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.186+0000 7ff0b2e68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac19ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.186+0000 7ff0b2e68740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bdcab71260 con 0x55bdcac0ce00 2024-09-10T08:23:57.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.186+0000 7ff0b2e68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bdcab18500 con 0x55bdcac176c0 2024-09-10T08:23:57.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.186+0000 7ff0b2e68740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bdcab6d220 con 0x55bdcac0c4e0 2024-09-10T08:23:57.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.186+0000 7ff0ae200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac0c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.186+0000 7ff0ae200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac0c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59774/0 (socket says 172.21.15.2:59774) 2024-09-10T08:23:57.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.186+0000 7ff0ad800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac17180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.186+0000 7ff0ae200640 1 -- 172.21.15.2:0/2552138684 learned_addr learned my addr 172.21.15.2:0/2552138684 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:57.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.187+0000 7ff0ace00640 1 --2- 172.21.15.2:0/2552138684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac19ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.187+0000 7ff0ae200640 1 -- 172.21.15.2:0/2552138684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 msgr2=0x55bdcac19ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:57.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.187+0000 7ff0ae200640 1 --2- 172.21.15.2:0/2552138684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac19ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.187+0000 7ff0ae200640 1 -- 172.21.15.2:0/2552138684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 msgr2=0x55bdcac17180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:57.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.187+0000 7ff0ae200640 1 --2- 172.21.15.2:0/2552138684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac17180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.187+0000 7ff0ae200640 1 -- 172.21.15.2:0/2552138684 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdca8e2ba0 con 0x55bdcac0c4e0 2024-09-10T08:23:57.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.187+0000 7ff0ad800640 1 --2- 172.21.15.2:0/2552138684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac17180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:57.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.187+0000 7ff0a6000640 1 --2- 172.21.15.2:0/3246494511 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff0900a2c00 0x7ff0900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.187+0000 7ff0ae200640 1 --2- 172.21.15.2:0/2552138684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac0c8c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff0a807aa10 tx=0x7ff0a0005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e586e1c4de888ddf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:57.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.188+0000 7ff0a6000640 1 -- 172.21.15.2:0/3246494511 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff0940cf740 con 0x7ff0a8077040 2024-09-10T08:23:57.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.188+0000 7ff0ad800640 1 --2- 172.21.15.2:0/3246494511 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff0900a2c00 0x7ff0900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.188+0000 7ff09aa00640 1 -- 172.21.15.2:0/2552138684 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0031db0 con 0x55bdcac0c4e0 2024-09-10T08:23:57.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.188+0000 7ff09aa00640 1 -- 172.21.15.2:0/2552138684 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0036030 con 0x55bdcac0c4e0 2024-09-10T08:23:57.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.188+0000 7ff09aa00640 1 -- 172.21.15.2:0/2552138684 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0004280 con 0x55bdcac0c4e0 2024-09-10T08:23:57.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.189+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2552138684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 msgr2=0x55bdcac0c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:57.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.189+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2552138684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac0c8c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff0a807aa10 tx=0x7ff0a0005f80 comp rx=0 tx=0).stop 2024-09-10T08:23:57.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.189+0000 7ff0ad800640 1 --2- 172.21.15.2:0/3246494511 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff0900a2c00 0x7ff0900a50c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55bdcac098d0 tx=0x7ff09c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:57.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.189+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2552138684 shutdown_connections 2024-09-10T08:23:57.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.189+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2552138684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac19ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.189+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2552138684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac17180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.189+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2552138684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac0c8c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.189+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2552138684 >> 172.21.15.2:0/2552138684 conn(0x55bdcac08a70 msgr2=0x55bdcac08e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:57.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.189+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2552138684 shutdown_connections 2024-09-10T08:23:57.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.189+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2552138684 wait complete. 2024-09-10T08:23:57.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.189+0000 7ff0b2e68740 1 -- start start 2024-09-10T08:23:57.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.190+0000 7ff0b2e68740 1 Processor -- start 2024-09-10T08:23:57.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.190+0000 7ff0b2e68740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac14540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.190+0000 7ff0b2e68740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac14a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.190+0000 7ff0b2e68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac14fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.190+0000 7ff0b2e68740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bdcac1a110 con 0x55bdcac0ce00 2024-09-10T08:23:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.190+0000 7ff0b2e68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bdcab2bbd0 con 0x55bdcac176c0 2024-09-10T08:23:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.190+0000 7ff0b2e68740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bdcac1a290 con 0x55bdcac0c4e0 2024-09-10T08:23:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.190+0000 7ff0b2e68740 10 client.0 _ll_register_callbacks cb 0x55bdca8800a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ace00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac14a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ad800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac14540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ae200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac14fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ad800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac14540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59790/0 (socket says 172.21.15.2:59790) 2024-09-10T08:23:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ae200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac14fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:53486/0 (socket says 172.21.15.2:53486) 2024-09-10T08:23:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ad800640 1 -- 172.21.15.2:0/1166006801 learned_addr learned my addr 172.21.15.2:0/1166006801 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ae200640 1 -- 172.21.15.2:0/1166006801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 msgr2=0x55bdcac14540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ae200640 1 --2- 172.21.15.2:0/1166006801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac14540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ae200640 1 -- 172.21.15.2:0/1166006801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 msgr2=0x55bdcac14a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ae200640 1 --2- 172.21.15.2:0/1166006801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac14a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ae200640 1 -- 172.21.15.2:0/1166006801 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdcac235b0 con 0x55bdcac176c0 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ad800640 1 --2- 172.21.15.2:0/1166006801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac14540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ace00640 1 --2- 172.21.15.2:0/1166006801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac14a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff0ae200640 1 --2- 172.21.15.2:0/1166006801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac14fc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009bf0 tx=0x7ff0a002f4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42a435d54b04d986 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.191+0000 7ff08f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.192+0000 7ff08f400640 1 -- 172.21.15.2:0/1166006801 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0054020 con 0x55bdcac176c0 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.192+0000 7ff08f400640 1 -- 172.21.15.2:0/1166006801 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff0a0003b70 con 0x55bdcac176c0 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.192+0000 7ff0b2e68740 1 -- 172.21.15.2:0/1166006801 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a0036030 con 0x55bdcac176c0 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.192+0000 7ff08f400640 1 -- 172.21.15.2:0/1166006801 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0a004a040 con 0x55bdcac176c0 2024-09-10T08:23:57.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.192+0000 7ff0b2e68740 10 client.29936 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:57.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.192+0000 7ff0b2e68740 1 -- 172.21.15.2:0/1166006801 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bdcac1a110 con 0x55bdcac176c0 2024-09-10T08:23:57.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.192+0000 7ff08f400640 1 -- 172.21.15.2:0/1166006801 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0036030 con 0x55bdcac176c0 2024-09-10T08:23:57.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.192+0000 7ff08f400640 1 -- 172.21.15.2:0/1166006801 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff0a004d030 con 0x55bdcac176c0 2024-09-10T08:23:57.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.193+0000 7ff0b2e68740 10 client.29936 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:57.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.193+0000 7ff0b2e68740 10 client.29936 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:23:57.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.193+0000 7ff0b2e68740 20 client.29936 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:57.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.193+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/1166006801 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bdcac27090 0x55bdcac29480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.193+0000 7ff0b2e68740 4 client.29936 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:23:57.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.193+0000 7ff0b2e68740 1 -- 172.21.15.2:0/1166006801 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdcab7e990 con 0x55bdcac27090 2024-09-10T08:23:57.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.193+0000 7ff0ad800640 1 --2- 172.21.15.2:0/1166006801 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bdcac27090 0x55bdcac29480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.194+0000 7ff0ad800640 1 --2- 172.21.15.2:0/1166006801 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bdcac27090 0x55bdcac29480 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b41b62ed938ce716 server_cookie=c94f200bae4884ce in_seq=0 out_seq=0 2024-09-10T08:23:57.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.194+0000 7ff08f400640 10 client.29936 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:57.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff08f400640 1 -- 172.21.15.2:0/1166006801 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdcab7e990 con 0x55bdcac27090 2024-09-10T08:23:57.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff08f400640 10 client.29936 handle_command_reply: tid=0 2024-09-10T08:23:57.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff0b2e68740 1 client.29936 shutdown 2024-09-10T08:23:57.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff0b2e68740 1 -- 172.21.15.2:0/1166006801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 msgr2=0x55bdcac14fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:57.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/1166006801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac14fc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009bf0 tx=0x7ff0a002f4d0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff0b2e68740 1 -- 172.21.15.2:0/1166006801 shutdown_connections 2024-09-10T08:23:57.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/1166006801 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bdcac27090 0x55bdcac29480 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/1166006801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac14fc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/1166006801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac14a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/1166006801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac14540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff08f400640 0 client.29936 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:57.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff0b2e68740 1 -- 172.21.15.2:0/1166006801 >> 172.21.15.2:0/1166006801 conn(0x55bdcac08a70 msgr2=0x55bdcac07bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:57.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff0b2e68740 1 -- 172.21.15.2:0/1166006801 shutdown_connections 2024-09-10T08:23:57.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.195+0000 7ff0b2e68740 1 -- 172.21.15.2:0/1166006801 wait complete. 2024-09-10T08:23:57.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.196+0000 7ff0b2e68740 20 client.29936 trim_cache size 0 max 16384 2024-09-10T08:23:57.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.213+0000 7ff0b2e68740 1 Processor -- start 2024-09-10T08:23:57.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.213+0000 7ff0b2e68740 1 -- start start 2024-09-10T08:23:57.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.213+0000 7ff0b2e68740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac16760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0b2e68740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac14e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0b2e68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac2ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0b2e68740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81b7bb0 con 0x55bdcac0ce00 2024-09-10T08:23:57.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0b2e68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81b5480 con 0x55bdcac176c0 2024-09-10T08:23:57.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0b2e68740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81b32d0 con 0x55bdcac0c4e0 2024-09-10T08:23:57.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0ace00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac14e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0ad800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac16760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0ad800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac16760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59794/0 (socket says 172.21.15.2:59794) 2024-09-10T08:23:57.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0ad800640 1 -- 172.21.15.2:0/2185221504 learned_addr learned my addr 172.21.15.2:0/2185221504 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:57.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0ae200640 1 --2- 172.21.15.2:0/2185221504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac2ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0ad800640 1 -- 172.21.15.2:0/2185221504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 msgr2=0x55bdcac2ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:57.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0ad800640 1 --2- 172.21.15.2:0/2185221504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac2ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0ad800640 1 -- 172.21.15.2:0/2185221504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 msgr2=0x55bdcac14e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:57.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0ad800640 1 --2- 172.21.15.2:0/2185221504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac14e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0ad800640 1 -- 172.21.15.2:0/2185221504 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdcac297f0 con 0x55bdcac0c4e0 2024-09-10T08:23:57.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.214+0000 7ff0ace00640 1 --2- 172.21.15.2:0/2185221504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac14e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:57.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.215+0000 7ff0ad800640 1 --2- 172.21.15.2:0/2185221504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac16760 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55bdcac29c70 tx=0x7ff09c019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f2d6a6460946c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:57.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.215+0000 7ff09a000640 1 -- 172.21.15.2:0/2185221504 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00fd10 con 0x55bdcac0c4e0 2024-09-10T08:23:57.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.215+0000 7ff09a000640 1 -- 172.21.15.2:0/2185221504 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff09c01dd70 con 0x55bdcac0c4e0 2024-09-10T08:23:57.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.215+0000 7ff09a000640 1 -- 172.21.15.2:0/2185221504 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c02ac50 con 0x55bdcac0c4e0 2024-09-10T08:23:57.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.216+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2185221504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 msgr2=0x55bdcac16760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:57.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.216+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2185221504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac16760 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55bdcac29c70 tx=0x7ff09c019000 comp rx=0 tx=0).stop 2024-09-10T08:23:57.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.216+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2185221504 shutdown_connections 2024-09-10T08:23:57.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.216+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2185221504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac2ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.216+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2185221504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac14e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.216+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2185221504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac16760 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.216+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2185221504 >> 172.21.15.2:0/2185221504 conn(0x55bdcac08a70 msgr2=0x55bdcac07bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:57.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.216+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2185221504 shutdown_connections 2024-09-10T08:23:57.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.216+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2185221504 wait complete. 2024-09-10T08:23:57.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.216+0000 7ff0b2e68740 1 -- start start 2024-09-10T08:23:57.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.217+0000 7ff0b2e68740 1 Processor -- start 2024-09-10T08:23:57.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.217+0000 7ff0b2e68740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac1fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.217+0000 7ff0b2e68740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac1add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.217+0000 7ff0b2e68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac1b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.217+0000 7ff0b2e68740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81b3940 con 0x55bdcac0ce00 2024-09-10T08:23:57.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.217+0000 7ff0b2e68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81b4db0 con 0x55bdcac176c0 2024-09-10T08:23:57.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.217+0000 7ff0b2e68740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81b4010 con 0x55bdcac0c4e0 2024-09-10T08:23:57.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.217+0000 7ff0b2e68740 10 client.0 _ll_register_callbacks cb 0x7ff09c009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:23:57.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.217+0000 7ff0ace00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac1add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.217+0000 7ff0ad800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac1fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.217+0000 7ff0ace00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac1add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33356/0 (socket says 172.21.15.2:33356) 2024-09-10T08:23:57.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.217+0000 7ff0ace00640 1 -- 172.21.15.2:0/2332649244 learned_addr learned my addr 172.21.15.2:0/2332649244 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:23:57.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.218+0000 7ff0ae200640 1 --2- 172.21.15.2:0/2332649244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac1b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.218+0000 7ff0ace00640 1 -- 172.21.15.2:0/2332649244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 msgr2=0x55bdcac1fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:57.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.218+0000 7ff0ace00640 1 --2- 172.21.15.2:0/2332649244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac1fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.218+0000 7ff0ace00640 1 -- 172.21.15.2:0/2332649244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 msgr2=0x55bdcac1b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:23:57.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.218+0000 7ff0ace00640 1 --2- 172.21.15.2:0/2332649244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac1b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.218+0000 7ff0ace00640 1 -- 172.21.15.2:0/2332649244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdcac1b900 con 0x55bdcac0ce00 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.218+0000 7ff0ad800640 1 --2- 172.21.15.2:0/2332649244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac1fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.218+0000 7ff0ace00640 1 --2- 172.21.15.2:0/2332649244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac1add0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff09409bb90 tx=0x7ff09400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851853c9ebeaea75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.218+0000 7ff08f400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.218+0000 7ff08f400640 1 -- 172.21.15.2:0/2332649244 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09409bc00 con 0x55bdcac0ce00 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.218+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2332649244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff09c01dd70 con 0x55bdcac0ce00 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.218+0000 7ff08f400640 1 -- 172.21.15.2:0/2332649244 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7ff09409b050 con 0x55bdcac0ce00 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.219+0000 7ff08f400640 1 -- 172.21.15.2:0/2332649244 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff09409ff00 con 0x55bdcac0ce00 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.219+0000 7ff0b2e68740 10 client.19923 fetch_fsmap learned FSMap version 26 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.219+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2332649244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bdcac91a80 con 0x55bdcac0ce00 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.219+0000 7ff08f400640 1 -- 172.21.15.2:0/2332649244 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff09c01dd70 con 0x55bdcac0ce00 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.219+0000 7ff08f400640 1 -- 172.21.15.2:0/2332649244 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff094018c20 con 0x55bdcac0ce00 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.219+0000 7ff0b2e68740 10 client.19923 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.219+0000 7ff0b2e68740 10 client.19923 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:23:57.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.219+0000 7ff0b2e68740 20 client.19923 populate_metadata read hostname 'smithi002' 2024-09-10T08:23:57.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.219+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2332649244 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bdcac92250 0x55bdcac94670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:23:57.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.219+0000 7ff0b2e68740 4 client.19923 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}] 2024-09-10T08:23:57.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.219+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2332649244 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}) -- 0x55bdcab78ce0 con 0x55bdcac92250 2024-09-10T08:23:57.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.220+0000 7ff0ad800640 1 --2- 172.21.15.2:0/2332649244 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bdcac92250 0x55bdcac94670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:23:57.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.220+0000 7ff0ad800640 1 --2- 172.21.15.2:0/2332649244 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bdcac92250 0x55bdcac94670 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7edf943715b6876 server_cookie=8184fafcfce25488 in_seq=0 out_seq=0 2024-09-10T08:23:57.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.220+0000 7ff08f400640 10 client.19923 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:57.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff08f400640 1 -- 172.21.15.2:0/2332649244 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bdcab78ce0 con 0x55bdcac92250 2024-09-10T08:23:57.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff08f400640 10 client.19923 handle_command_reply: tid=0 2024-09-10T08:23:57.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff0b2e68740 1 client.19923 shutdown 2024-09-10T08:23:57.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2332649244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 msgr2=0x55bdcac1add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:57.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2332649244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac1add0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff09409bb90 tx=0x7ff09400cf60 comp rx=0 tx=0).stop 2024-09-10T08:23:57.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2332649244 shutdown_connections 2024-09-10T08:23:57.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2332649244 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bdcac92250 0x55bdcac94670 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2332649244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdcac176c0 0x55bdcac1b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2332649244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bdcac0ce00 0x55bdcac1add0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff0b2e68740 1 --2- 172.21.15.2:0/2332649244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdcac0c4e0 0x55bdcac1fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2332649244 >> 172.21.15.2:0/2332649244 conn(0x55bdcac08a70 msgr2=0x55bdcac22390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:57.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff08f400640 0 client.19923 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:23:57.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2332649244 shutdown_connections 2024-09-10T08:23:57.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff0b2e68740 1 -- 172.21.15.2:0/2332649244 wait complete. 2024-09-10T08:23:57.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.234+0000 7ff0b2e68740 20 client.19923 trim_cache size 0 max 16384 2024-09-10T08:23:57.237 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:23:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:23:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:23:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 58, 2024-09-10T08:23:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:23:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "4978faaa": { 2024-09-10T08:23:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 58, 2024-09-10T08:23:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:23:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:23:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:23:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:23:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:23:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:23:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 139.8, 2024-09-10T08:23:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:23:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:23:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:23:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:23:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-09-10T08:23:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:23:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:23:57.239 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:23:57.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.235+0000 7ff0b0200640 1 -- 172.21.15.2:0/3246494511 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff0900a2c00 msgr2=0x7ff0900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:57.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.235+0000 7ff0b0200640 1 --2- 172.21.15.2:0/3246494511 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff0900a2c00 0x7ff0900a50c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55bdcac098d0 tx=0x7ff09c005f90 comp rx=0 tx=0).stop 2024-09-10T08:23:57.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.235+0000 7ff0b0200640 1 -- 172.21.15.2:0/3246494511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0a8077040 msgr2=0x7ff0a81a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:23:57.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.235+0000 7ff0b0200640 1 --2- 172.21.15.2:0/3246494511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0a8077040 0x7ff0a81a4c10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff09400e9f0 tx=0x7ff09400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.235+0000 7ff0b0200640 1 -- 172.21.15.2:0/3246494511 shutdown_connections 2024-09-10T08:23:57.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.235+0000 7ff0b0200640 1 --2- 172.21.15.2:0/3246494511 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7ff0900a2c00 0x7ff0900a50c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.235+0000 7ff0b0200640 1 --2- 172.21.15.2:0/3246494511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a8112a30 0x7ff0a819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.235+0000 7ff0b0200640 1 --2- 172.21.15.2:0/3246494511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0a8077040 0x7ff0a81a4c10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.235+0000 7ff0b0200640 1 --2- 172.21.15.2:0/3246494511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0a8076720 0x7ff0a81a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:23:57.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.235+0000 7ff0b0200640 1 -- 172.21.15.2:0/3246494511 >> 172.21.15.2:0/3246494511 conn(0x7ff0a80ffa90 msgr2=0x7ff0a8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:23:57.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.235+0000 7ff0b0200640 1 -- 172.21.15.2:0/3246494511 shutdown_connections 2024-09-10T08:23:57.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:23:57.235+0000 7ff0b0200640 1 -- 172.21.15.2:0/3246494511 wait complete. 2024-09-10T08:23:57.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:57 smithi155 ceph-mon[35405]: pgmap v861: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-10T08:23:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:57 smithi107 ceph-mon[35048]: pgmap v861: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-10T08:23:57.918 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:57 smithi002 ceph-mon[30563]: pgmap v861: 129 pgs: 129 active+clean; 4.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-10T08:23:58.090 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4978faaa', quiesce took 4.0 seconds. Will release after: 15.850000000000001 2024-09-10T08:23:59.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:23:59 smithi155 ceph-mon[35405]: pgmap v862: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-09-10T08:23:59.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:23:59 smithi107 ceph-mon[35048]: pgmap v862: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-09-10T08:24:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:23:59 smithi002 ceph-mon[30563]: pgmap v862: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-09-10T08:24:01.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:01 smithi155 ceph-mon[35405]: pgmap v863: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-10T08:24:01.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:01 smithi107 ceph-mon[35048]: pgmap v863: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-10T08:24:02.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:01 smithi002 ceph-mon[30563]: pgmap v863: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-10T08:24:03.410 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:24:03.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:03 smithi002 ceph-mon[30563]: pgmap v864: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 10 op/s 2024-09-10T08:24:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:03.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:03 smithi155 ceph-mon[35405]: pgmap v864: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 10 op/s 2024-09-10T08:24:03.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:03 smithi107 ceph-mon[35048]: pgmap v864: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 10 op/s 2024-09-10T08:24:04.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.403+0000 7f461f800640 1 -- 172.21.15.2:0/1982419831 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4618106f10 msgr2=0x7f461810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:04.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.403+0000 7f461f800640 1 --2- 172.21.15.2:0/1982419831 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4618106f10 0x7f461810ea90 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4610009ca0 tx=0x7f461002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.403+0000 7f461f800640 1 -- 172.21.15.2:0/1982419831 shutdown_connections 2024-09-10T08:24:04.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.403+0000 7f461f800640 1 --2- 172.21.15.2:0/1982419831 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4618106f10 0x7f461810ea90 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.403+0000 7f461f800640 1 --2- 172.21.15.2:0/1982419831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4618102790 0x7f46181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.403+0000 7f461f800640 1 --2- 172.21.15.2:0/1982419831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4618101de0 0x7f46181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.403+0000 7f461f800640 1 -- 172.21.15.2:0/1982419831 >> 172.21.15.2:0/1982419831 conn(0x7f46180fd8c0 msgr2=0x7f46180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:04.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.403+0000 7f461f800640 1 -- 172.21.15.2:0/1982419831 shutdown_connections 2024-09-10T08:24:04.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.403+0000 7f461f800640 1 -- 172.21.15.2:0/1982419831 wait complete. 2024-09-10T08:24:04.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.404+0000 7f461f800640 1 Processor -- start 2024-09-10T08:24:04.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.405+0000 7f461f800640 1 -- start start 2024-09-10T08:24:04.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.405+0000 7f461f800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4618101de0 0x7f461819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.405+0000 7f461f800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4618102790 0x7f461819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.405+0000 7f461f800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4618106f10 0x7f461819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.405+0000 7f461f800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4618111f70 con 0x7f4618101de0 2024-09-10T08:24:04.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.405+0000 7f461f800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4618111df0 con 0x7f4618102790 2024-09-10T08:24:04.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.405+0000 7f461f800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f46181120f0 con 0x7f4618106f10 2024-09-10T08:24:04.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.405+0000 7f4617e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4618102790 0x7f461819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.405+0000 7f461ce00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4618101de0 0x7f461819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.405+0000 7f461d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4618106f10 0x7f461819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.405+0000 7f4617e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4618102790 0x7f461819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:53510/0 (socket says 172.21.15.2:53510) 2024-09-10T08:24:04.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.405+0000 7f4617e00640 1 -- 172.21.15.2:0/1229488294 learned_addr learned my addr 172.21.15.2:0/1229488294 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:04.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.406+0000 7f4617e00640 1 -- 172.21.15.2:0/1229488294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4618106f10 msgr2=0x7f461819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:04.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.406+0000 7f4617e00640 1 --2- 172.21.15.2:0/1229488294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4618106f10 0x7f461819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.406+0000 7f4617e00640 1 -- 172.21.15.2:0/1229488294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4618101de0 msgr2=0x7f461819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:04.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.406+0000 7f4617e00640 1 --2- 172.21.15.2:0/1229488294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4618101de0 0x7f461819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.406+0000 7f4617e00640 1 -- 172.21.15.2:0/1229488294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461819f5e0 con 0x7f4618102790 2024-09-10T08:24:04.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.406+0000 7f461ce00640 1 --2- 172.21.15.2:0/1229488294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4618101de0 0x7f461819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:04.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.406+0000 7f4617e00640 1 --2- 172.21.15.2:0/1229488294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4618102790 0x7f461819ab70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f460400e9f0 tx=0x7f460400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:04.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.406+0000 7f4615600640 1 -- 172.21.15.2:0/1229488294 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460400cd80 con 0x7f4618102790 2024-09-10T08:24:04.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.407+0000 7f4615600640 1 -- 172.21.15.2:0/1229488294 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4604010910 con 0x7f4618102790 2024-09-10T08:24:04.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.407+0000 7f461f800640 1 -- 172.21.15.2:0/1229488294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461819f8d0 con 0x7f4618102790 2024-09-10T08:24:04.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.407+0000 7f4615600640 1 -- 172.21.15.2:0/1229488294 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604018990 con 0x7f4618102790 2024-09-10T08:24:04.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.407+0000 7f461f800640 1 -- 172.21.15.2:0/1229488294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46181a71b0 con 0x7f4618102790 2024-09-10T08:24:04.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.410+0000 7f4622285740 1 Processor -- start 2024-09-10T08:24:04.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.410+0000 7f4622285740 1 -- start start 2024-09-10T08:24:04.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.410+0000 7f4622285740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e37b90 0x5586d5e37f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.410+0000 7f4622285740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e384b0 0x5586d5e42830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.410+0000 7f4622285740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e42d70 0x5586d5e45160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.410+0000 7f4622285740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5586d5d9e3e0 con 0x5586d5e37b90 2024-09-10T08:24:04.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.410+0000 7f4622285740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5586d5d45d80 con 0x5586d5e42d70 2024-09-10T08:24:04.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.410+0000 7f4622285740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5586d5d9a3a0 con 0x5586d5e384b0 2024-09-10T08:24:04.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.410+0000 7f4615600640 1 -- 172.21.15.2:0/1229488294 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4604010ac0 con 0x7f4618102790 2024-09-10T08:24:04.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.410+0000 7f461d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e37b90 0x5586d5e37f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.410+0000 7f461ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e384b0 0x5586d5e42830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.411+0000 7f461ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e384b0 0x5586d5e42830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59830/0 (socket says 172.21.15.2:59830) 2024-09-10T08:24:04.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.411+0000 7f461ce00640 1 -- 172.21.15.2:0/3539298258 learned_addr learned my addr 172.21.15.2:0/3539298258 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:04.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.411+0000 7f461d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e37b90 0x5586d5e37f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33372/0 (socket says 172.21.15.2:33372) 2024-09-10T08:24:04.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.411+0000 7f4617e00640 1 --2- 172.21.15.2:0/3539298258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e42d70 0x5586d5e45160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.411+0000 7f461ce00640 1 -- 172.21.15.2:0/3539298258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e42d70 msgr2=0x5586d5e45160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:04.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.411+0000 7f461ce00640 1 --2- 172.21.15.2:0/3539298258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e42d70 0x5586d5e45160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.411+0000 7f461ce00640 1 -- 172.21.15.2:0/3539298258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e37b90 msgr2=0x5586d5e37f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:04.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.411+0000 7f461ce00640 1 --2- 172.21.15.2:0/3539298258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e37b90 0x5586d5e37f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.411+0000 7f461ce00640 1 -- 172.21.15.2:0/3539298258 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d5d64ae0 con 0x5586d5e384b0 2024-09-10T08:24:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.411+0000 7f461d800640 1 --2- 172.21.15.2:0/3539298258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e37b90 0x5586d5e37f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.411+0000 7f461ce00640 1 --2- 172.21.15.2:0/3539298258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e384b0 0x5586d5e42830 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f460c0115f0 tx=0x7f460c011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7cbc982e90a192e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.412+0000 7f460a000640 1 -- 172.21.15.2:0/3539298258 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c01e070 con 0x5586d5e384b0 2024-09-10T08:24:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.412+0000 7f460a000640 1 -- 172.21.15.2:0/3539298258 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f460c00fb20 con 0x5586d5e384b0 2024-09-10T08:24:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.412+0000 7f460a000640 1 -- 172.21.15.2:0/3539298258 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c019bf0 con 0x5586d5e384b0 2024-09-10T08:24:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.412+0000 7f4622285740 1 -- 172.21.15.2:0/3539298258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e384b0 msgr2=0x5586d5e42830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.412+0000 7f4622285740 1 --2- 172.21.15.2:0/3539298258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e384b0 0x5586d5e42830 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f460c0115f0 tx=0x7f460c011ab0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.412+0000 7f4615600640 1 --2- 172.21.15.2:0/1229488294 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46000a2c00 0x7f46000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.413+0000 7f4615600640 1 -- 172.21.15.2:0/1229488294 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f46040cf660 con 0x7f4618102790 2024-09-10T08:24:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.413+0000 7f4622285740 1 -- 172.21.15.2:0/3539298258 shutdown_connections 2024-09-10T08:24:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.413+0000 7f4622285740 1 --2- 172.21.15.2:0/3539298258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e42d70 0x5586d5e45160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.413+0000 7f4622285740 1 --2- 172.21.15.2:0/3539298258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e384b0 0x5586d5e42830 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.413+0000 7f4622285740 1 --2- 172.21.15.2:0/3539298258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e37b90 0x5586d5e37f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.413+0000 7f4622285740 1 -- 172.21.15.2:0/3539298258 >> 172.21.15.2:0/3539298258 conn(0x5586d5e34120 msgr2=0x5586d5e34500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.413+0000 7f461d800640 1 --2- 172.21.15.2:0/1229488294 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46000a2c00 0x7f46000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.413+0000 7f4622285740 1 -- 172.21.15.2:0/3539298258 shutdown_connections 2024-09-10T08:24:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.413+0000 7f4622285740 1 -- 172.21.15.2:0/3539298258 wait complete. 2024-09-10T08:24:04.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.414+0000 7f461d800640 1 --2- 172.21.15.2:0/1229488294 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46000a2c00 0x7f46000a50c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4610004af0 tx=0x7f4610004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:04.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.414+0000 7f4622285740 1 -- start start 2024-09-10T08:24:04.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.414+0000 7f4622285740 1 Processor -- start 2024-09-10T08:24:04.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.414+0000 7f4622285740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e3fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.414+0000 7f4622285740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e400f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.414+0000 7f4622285740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 0x5586d5e40630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.414+0000 7f4622285740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5586d5e459e0 con 0x5586d5e42d70 2024-09-10T08:24:04.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.414+0000 7f4622285740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5586d5d590f0 con 0x5586d5e384b0 2024-09-10T08:24:04.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.414+0000 7f4622285740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5586d5e45b60 con 0x5586d5e37b90 2024-09-10T08:24:04.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.414+0000 7f4622285740 10 client.0 _ll_register_callbacks cb 0x5586d5d4a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:04.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.415+0000 7f461d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 0x5586d5e40630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.415+0000 7f461ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e3fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.415+0000 7f4617e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.415+0000 7f461ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e3fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59840/0 (socket says 172.21.15.2:59840) 2024-09-10T08:24:04.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.415+0000 7f461ce00640 1 -- 172.21.15.2:0/577139020 learned_addr learned my addr 172.21.15.2:0/577139020 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:04.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.415+0000 7f461ce00640 1 -- 172.21.15.2:0/577139020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 msgr2=0x5586d5e400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:04.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.415+0000 7f461ce00640 1 --2- 172.21.15.2:0/577139020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e400f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.415+0000 7f461ce00640 1 -- 172.21.15.2:0/577139020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 msgr2=0x5586d5e40630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:04.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.415+0000 7f461ce00640 1 --2- 172.21.15.2:0/577139020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 0x5586d5e40630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.415+0000 7f461ce00640 1 -- 172.21.15.2:0/577139020 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d5b0f3c0 con 0x5586d5e37b90 2024-09-10T08:24:04.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.416+0000 7f461ce00640 1 --2- 172.21.15.2:0/577139020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e3fbb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f460c005e00 tx=0x7f460c00f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be4888cc1efb00fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:04.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.416+0000 7f45fea00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:24:04.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.416+0000 7f45fea00640 1 -- 172.21.15.2:0/577139020 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c01e040 con 0x5586d5e37b90 2024-09-10T08:24:04.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.416+0000 7f4622285740 1 -- 172.21.15.2:0/577139020 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f460c00fb20 con 0x5586d5e37b90 2024-09-10T08:24:04.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.416+0000 7f45fea00640 1 -- 172.21.15.2:0/577139020 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f460c004040 con 0x5586d5e37b90 2024-09-10T08:24:04.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.416+0000 7f45fea00640 1 -- 172.21.15.2:0/577139020 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f460c02f050 con 0x5586d5e37b90 2024-09-10T08:24:04.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.416+0000 7f4622285740 10 client.29881 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:04.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.416+0000 7f4622285740 1 -- 172.21.15.2:0/577139020 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5586d5e516a0 con 0x5586d5e37b90 2024-09-10T08:24:04.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.416+0000 7f45fea00640 1 -- 172.21.15.2:0/577139020 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f460c00fb20 con 0x5586d5e37b90 2024-09-10T08:24:04.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.417+0000 7f45fea00640 1 -- 172.21.15.2:0/577139020 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f460c02a030 con 0x5586d5e37b90 2024-09-10T08:24:04.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.417+0000 7f4622285740 10 client.29881 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:04.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.417+0000 7f4622285740 10 client.29881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:04.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.417+0000 7f4622285740 20 client.29881 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:04.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.417+0000 7f4622285740 1 --2- 172.21.15.2:0/577139020 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586d5e52ab0 0x5586d5e54ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.417+0000 7f4622285740 4 client.29881 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:04.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.417+0000 7f4622285740 1 -- 172.21.15.2:0/577139020 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586d5dab950 con 0x5586d5e52ab0 2024-09-10T08:24:04.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.417+0000 7f4617e00640 1 --2- 172.21.15.2:0/577139020 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586d5e52ab0 0x5586d5e54ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.418+0000 7f4617e00640 1 --2- 172.21.15.2:0/577139020 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586d5e52ab0 0x5586d5e54ea0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e5712d1f3e7c1b server_cookie=70e45b157ca010d in_seq=0 out_seq=0 2024-09-10T08:24:04.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.418+0000 7f45fea00640 10 client.29881 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:04.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.418+0000 7f45fea00640 1 -- 172.21.15.2:0/577139020 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586d5dab950 con 0x5586d5e52ab0 2024-09-10T08:24:04.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.418+0000 7f45fea00640 10 client.29881 handle_command_reply: tid=0 2024-09-10T08:24:04.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.419+0000 7f4622285740 1 client.29881 shutdown 2024-09-10T08:24:04.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.419+0000 7f4622285740 1 -- 172.21.15.2:0/577139020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 msgr2=0x5586d5e3fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:04.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.419+0000 7f4622285740 1 --2- 172.21.15.2:0/577139020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e3fbb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f460c005e00 tx=0x7f460c00f8d0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.419+0000 7f4622285740 1 -- 172.21.15.2:0/577139020 shutdown_connections 2024-09-10T08:24:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.419+0000 7f4622285740 1 --2- 172.21.15.2:0/577139020 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586d5e52ab0 0x5586d5e54ea0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.419+0000 7f4622285740 1 --2- 172.21.15.2:0/577139020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 0x5586d5e40630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.419+0000 7f4622285740 1 --2- 172.21.15.2:0/577139020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.419+0000 7f45fea00640 0 client.29881 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.419+0000 7f4622285740 1 --2- 172.21.15.2:0/577139020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e3fbb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.419+0000 7f4622285740 1 -- 172.21.15.2:0/577139020 >> 172.21.15.2:0/577139020 conn(0x5586d5e34120 msgr2=0x5586d5e39e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.419+0000 7f4622285740 1 -- 172.21.15.2:0/577139020 shutdown_connections 2024-09-10T08:24:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.419+0000 7f4622285740 1 -- 172.21.15.2:0/577139020 wait complete. 2024-09-10T08:24:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.419+0000 7f4622285740 20 client.29881 trim_cache size 0 max 16384 2024-09-10T08:24:04.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.437+0000 7f4622285740 1 Processor -- start 2024-09-10T08:24:04.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.437+0000 7f4622285740 1 -- start start 2024-09-10T08:24:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.437+0000 7f4622285740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e3ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f4622285740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e5a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f4622285740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 0x5586d5e5abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f4622285740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f46181b3980 con 0x5586d5e42d70 2024-09-10T08:24:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f4622285740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46181b1250 con 0x5586d5e384b0 2024-09-10T08:24:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f4622285740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f46181af0a0 con 0x5586d5e37b90 2024-09-10T08:24:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f4617e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e5a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f461d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 0x5586d5e5abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f4617e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e5a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:53542/0 (socket says 172.21.15.2:53542) 2024-09-10T08:24:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f4617e00640 1 -- 172.21.15.2:0/2868085829 learned_addr learned my addr 172.21.15.2:0/2868085829 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f461ce00640 1 --2- 172.21.15.2:0/2868085829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e3ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f4617e00640 1 -- 172.21.15.2:0/2868085829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 msgr2=0x5586d5e3ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f4617e00640 1 --2- 172.21.15.2:0/2868085829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e3ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f4617e00640 1 -- 172.21.15.2:0/2868085829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 msgr2=0x5586d5e5abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f4617e00640 1 --2- 172.21.15.2:0/2868085829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 0x5586d5e5abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.438+0000 7f4617e00640 1 -- 172.21.15.2:0/2868085829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d5e339b0 con 0x5586d5e384b0 2024-09-10T08:24:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.439+0000 7f4617e00640 1 --2- 172.21.15.2:0/2868085829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e5a680 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f46040ccd10 tx=0x7f46040ccbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97e837fd3dc94f24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.439+0000 7f4609600640 1 -- 172.21.15.2:0/2868085829 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460409bb20 con 0x5586d5e384b0 2024-09-10T08:24:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.439+0000 7f4609600640 1 -- 172.21.15.2:0/2868085829 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f460409fe20 con 0x5586d5e384b0 2024-09-10T08:24:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.439+0000 7f4609600640 1 -- 172.21.15.2:0/2868085829 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46040cf2b0 con 0x5586d5e384b0 2024-09-10T08:24:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.439+0000 7f4622285740 1 -- 172.21.15.2:0/2868085829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 msgr2=0x5586d5e5a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.439+0000 7f4622285740 1 --2- 172.21.15.2:0/2868085829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e5a680 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f46040ccd10 tx=0x7f46040ccbe0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.439+0000 7f4622285740 1 -- 172.21.15.2:0/2868085829 shutdown_connections 2024-09-10T08:24:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.439+0000 7f4622285740 1 --2- 172.21.15.2:0/2868085829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 0x5586d5e5abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.439+0000 7f4622285740 1 --2- 172.21.15.2:0/2868085829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e5a680 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.439+0000 7f4622285740 1 --2- 172.21.15.2:0/2868085829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e3ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.439+0000 7f4622285740 1 -- 172.21.15.2:0/2868085829 >> 172.21.15.2:0/2868085829 conn(0x5586d5e34120 msgr2=0x5586d5e39e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:04.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.440+0000 7f4622285740 1 -- 172.21.15.2:0/2868085829 shutdown_connections 2024-09-10T08:24:04.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.440+0000 7f4622285740 1 -- 172.21.15.2:0/2868085829 wait complete. 2024-09-10T08:24:04.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.440+0000 7f4622285740 1 -- start start 2024-09-10T08:24:04.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.440+0000 7f4622285740 1 Processor -- start 2024-09-10T08:24:04.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f4622285740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e4b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f4622285740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e3c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f4622285740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 0x5586d5e3cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f4622285740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f46181af710 con 0x5586d5e42d70 2024-09-10T08:24:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f4622285740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46181b0b80 con 0x5586d5e384b0 2024-09-10T08:24:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f4622285740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f46181afde0 con 0x5586d5e37b90 2024-09-10T08:24:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f4622285740 10 client.0 _ll_register_callbacks cb 0x7f460c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f461ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e4b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f461d800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 0x5586d5e3cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f461ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e4b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59848/0 (socket says 172.21.15.2:59848) 2024-09-10T08:24:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f461ce00640 1 -- 172.21.15.2:0/796549017 learned_addr learned my addr 172.21.15.2:0/796549017 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f4617e00640 1 --2- 172.21.15.2:0/796549017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e3c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f461ce00640 1 -- 172.21.15.2:0/796549017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 msgr2=0x5586d5e3c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f461ce00640 1 --2- 172.21.15.2:0/796549017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e3c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f461ce00640 1 -- 172.21.15.2:0/796549017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 msgr2=0x5586d5e3cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:04.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f461ce00640 1 --2- 172.21.15.2:0/796549017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 0x5586d5e3cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f461ce00640 1 -- 172.21.15.2:0/796549017 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d5e3d120 con 0x5586d5e37b90 2024-09-10T08:24:04.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.441+0000 7f461d800640 1 --2- 172.21.15.2:0/796549017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 0x5586d5e3cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:04.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.442+0000 7f461ce00640 1 --2- 172.21.15.2:0/796549017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e4b290 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f460c005e00 tx=0x7f460c009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=845d5f420abfeab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:04.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.442+0000 7f45fea00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:24:04.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.442+0000 7f45fea00640 1 -- 172.21.15.2:0/796549017 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c01e070 con 0x5586d5e37b90 2024-09-10T08:24:04.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.442+0000 7f4622285740 1 -- 172.21.15.2:0/796549017 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f460409fe20 con 0x5586d5e37b90 2024-09-10T08:24:04.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.442+0000 7f45fea00640 1 -- 172.21.15.2:0/796549017 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f460c0199d0 con 0x5586d5e37b90 2024-09-10T08:24:04.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.442+0000 7f45fea00640 1 -- 172.21.15.2:0/796549017 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f460c0054b0 con 0x5586d5e37b90 2024-09-10T08:24:04.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.442+0000 7f4622285740 10 client.29887 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:04.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.442+0000 7f4622285740 1 -- 172.21.15.2:0/796549017 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5586d5ebd3c0 con 0x5586d5e37b90 2024-09-10T08:24:04.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.443+0000 7f45fea00640 1 -- 172.21.15.2:0/796549017 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f460409fe20 con 0x5586d5e37b90 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.443+0000 7f45fea00640 1 -- 172.21.15.2:0/796549017 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f460c02a040 con 0x5586d5e37b90 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.443+0000 7f4622285740 10 client.29887 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.443+0000 7f4622285740 10 client.29887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.443+0000 7f4622285740 20 client.29887 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.443+0000 7f4622285740 1 --2- 172.21.15.2:0/796549017 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586d5ebdaa0 0x5586d5ebfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.443+0000 7f4622285740 4 client.29887 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.443+0000 7f4622285740 1 -- 172.21.15.2:0/796549017 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586d5da5e60 con 0x5586d5ebdaa0 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.443+0000 7f4617e00640 1 --2- 172.21.15.2:0/796549017 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586d5ebdaa0 0x5586d5ebfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f4617e00640 1 --2- 172.21.15.2:0/796549017 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586d5ebdaa0 0x5586d5ebfec0 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa65898856be6df0 server_cookie=49cd193b4933ce5a in_seq=0 out_seq=0 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f45fea00640 10 client.29887 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f45fea00640 1 -- 172.21.15.2:0/796549017 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5586d5da5e60 con 0x5586d5ebdaa0 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f45fea00640 10 client.29887 handle_command_reply: tid=0 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f4622285740 1 client.29887 shutdown 2024-09-10T08:24:04.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f4622285740 1 -- 172.21.15.2:0/796549017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 msgr2=0x5586d5e4b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:04.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f4622285740 1 --2- 172.21.15.2:0/796549017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e4b290 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f460c005e00 tx=0x7f460c009f60 comp rx=0 tx=0).stop 2024-09-10T08:24:04.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f4622285740 1 -- 172.21.15.2:0/796549017 shutdown_connections 2024-09-10T08:24:04.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f4622285740 1 --2- 172.21.15.2:0/796549017 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5586d5ebdaa0 0x5586d5ebfec0 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f4622285740 1 --2- 172.21.15.2:0/796549017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5586d5e42d70 0x5586d5e3cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f4622285740 1 --2- 172.21.15.2:0/796549017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586d5e384b0 0x5586d5e3c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f4622285740 1 --2- 172.21.15.2:0/796549017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586d5e37b90 0x5586d5e4b290 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f45fea00640 0 client.29887 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:04.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f4622285740 1 -- 172.21.15.2:0/796549017 >> 172.21.15.2:0/796549017 conn(0x5586d5e34120 msgr2=0x5586d5e329a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:04.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f4622285740 1 -- 172.21.15.2:0/796549017 shutdown_connections 2024-09-10T08:24:04.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.444+0000 7f4622285740 1 -- 172.21.15.2:0/796549017 wait complete. 2024-09-10T08:24:04.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.445+0000 7f4622285740 20 client.29887 trim_cache size 0 max 16384 2024-09-10T08:24:04.449 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:24:04.449 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:24:04.449 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:24:04.449 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:24:04.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.445+0000 7f461f800640 1 -- 172.21.15.2:0/1229488294 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46000a2c00 msgr2=0x7f46000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:04.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.445+0000 7f461f800640 1 --2- 172.21.15.2:0/1229488294 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46000a2c00 0x7f46000a50c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4610004af0 tx=0x7f4610004a80 comp rx=0 tx=0).stop 2024-09-10T08:24:04.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.445+0000 7f461f800640 1 -- 172.21.15.2:0/1229488294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4618102790 msgr2=0x7f461819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:04.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.445+0000 7f461f800640 1 --2- 172.21.15.2:0/1229488294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4618102790 0x7f461819ab70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f460400e9f0 tx=0x7f460400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.445+0000 7f461f800640 1 -- 172.21.15.2:0/1229488294 shutdown_connections 2024-09-10T08:24:04.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.445+0000 7f461f800640 1 --2- 172.21.15.2:0/1229488294 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f46000a2c00 0x7f46000a50c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.445+0000 7f461f800640 1 --2- 172.21.15.2:0/1229488294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4618106f10 0x7f461819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.445+0000 7f461f800640 1 --2- 172.21.15.2:0/1229488294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4618102790 0x7f461819ab70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.445+0000 7f461f800640 1 --2- 172.21.15.2:0/1229488294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4618101de0 0x7f461819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:04.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.445+0000 7f461f800640 1 -- 172.21.15.2:0/1229488294 >> 172.21.15.2:0/1229488294 conn(0x7f46180fd8c0 msgr2=0x7f46180ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:04.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.445+0000 7f461f800640 1 -- 172.21.15.2:0/1229488294 shutdown_connections 2024-09-10T08:24:04.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:04.446+0000 7f461f800640 1 -- 172.21.15.2:0/1229488294 wait complete. 2024-09-10T08:24:05.450 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:24:05.451 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:24:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.507+0000 7f93bda00640 1 -- 172.21.15.2:0/4186984962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8103050 msgr2=0x7f93b8103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.507+0000 7f93bda00640 1 --2- 172.21.15.2:0/4186984962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8103050 0x7f93b8103450 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009900 tx=0x7f93a802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.507+0000 7f93bda00640 1 -- 172.21.15.2:0/4186984962 shutdown_connections 2024-09-10T08:24:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.507+0000 7f93bda00640 1 --2- 172.21.15.2:0/4186984962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8101f40 0x7f93b8109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.507+0000 7f93bda00640 1 --2- 172.21.15.2:0/4186984962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8101580 0x7f93b8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.507+0000 7f93bda00640 1 --2- 172.21.15.2:0/4186984962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8103050 0x7f93b8103450 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.507+0000 7f93bda00640 1 -- 172.21.15.2:0/4186984962 >> 172.21.15.2:0/4186984962 conn(0x7f93b80fd0d0 msgr2=0x7f93b80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.508+0000 7f93bda00640 1 -- 172.21.15.2:0/4186984962 shutdown_connections 2024-09-10T08:24:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.508+0000 7f93bda00640 1 -- 172.21.15.2:0/4186984962 wait complete. 2024-09-10T08:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.509+0000 7f93bda00640 1 Processor -- start 2024-09-10T08:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.509+0000 7f93bda00640 1 -- start start 2024-09-10T08:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.509+0000 7f93bda00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8101580 0x7f93b8197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93bda00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8101f40 0x7f93b8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93bda00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8103050 0x7f93b819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93bda00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f93b810d740 con 0x7f93b8101f40 2024-09-10T08:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93bda00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93b810d5c0 con 0x7f93b8101580 2024-09-10T08:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93bda00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f93b810d8c0 con 0x7f93b8103050 2024-09-10T08:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8101580 0x7f93b8197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93b7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8101f40 0x7f93b8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8101580 0x7f93b8197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:53570/0 (socket says 172.21.15.2:53570) 2024-09-10T08:24:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93b7e00640 1 -- 172.21.15.2:0/4184439960 learned_addr learned my addr 172.21.15.2:0/4184439960 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93bd000640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8103050 0x7f93b819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93b7e00640 1 -- 172.21.15.2:0/4184439960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8103050 msgr2=0x7f93b819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93b7e00640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8103050 0x7f93b819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93b7e00640 1 -- 172.21.15.2:0/4184439960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8101f40 msgr2=0x7f93b8198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93b7e00640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8101f40 0x7f93b8198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93b7e00640 1 -- 172.21.15.2:0/4184439960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b819f9f0 con 0x7f93b8101580 2024-09-10T08:24:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.510+0000 7f93b7400640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8101f40 0x7f93b8198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.511+0000 7f93b7e00640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8101580 0x7f93b8197dd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f93a8031440 tx=0x7f93a8032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.511+0000 7f93b4c00640 1 -- 172.21.15.2:0/4184439960 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a803f030 con 0x7f93b8101580 2024-09-10T08:24:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.511+0000 7f93b4c00640 1 -- 172.21.15.2:0/4184439960 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93a8003ce0 con 0x7f93b8101580 2024-09-10T08:24:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.511+0000 7f93bda00640 1 -- 172.21.15.2:0/4184439960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b819fc80 con 0x7f93b8101580 2024-09-10T08:24:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.511+0000 7f93b4c00640 1 -- 172.21.15.2:0/4184439960 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8035070 con 0x7f93b8101580 2024-09-10T08:24:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.511+0000 7f93bda00640 1 -- 172.21.15.2:0/4184439960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b81a0170 con 0x7f93b8101580 2024-09-10T08:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.514+0000 7f93c1fe6740 1 Processor -- start 2024-09-10T08:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.514+0000 7f93c1fe6740 1 -- start start 2024-09-10T08:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.514+0000 7f93c1fe6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d85a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.514+0000 7f93c1fe6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d8aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.514+0000 7f93c1fe6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d8d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.514+0000 7f93c1fe6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e6d5ce6100 con 0x55e6d5d8b040 2024-09-10T08:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.514+0000 7f93c1fe6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e6d5c8e1b0 con 0x55e6d5d85fb0 2024-09-10T08:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.514+0000 7f93c1fe6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e6d5ce20c0 con 0x55e6d5d85670 2024-09-10T08:24:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.515+0000 7f93b4c00640 1 -- 172.21.15.2:0/4184439960 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f93a8042030 con 0x7f93b8101580 2024-09-10T08:24:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.515+0000 7f93b7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d8d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.515+0000 7f93bd000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d85a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.515+0000 7f93b7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d8d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33446/0 (socket says 172.21.15.2:33446) 2024-09-10T08:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.515+0000 7f93b7400640 1 -- 172.21.15.2:0/1554024379 learned_addr learned my addr 172.21.15.2:0/1554024379 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.515+0000 7f93b7e00640 1 --2- 172.21.15.2:0/1554024379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d8aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.515+0000 7f93bd000640 1 -- 172.21.15.2:0/1554024379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 msgr2=0x55e6d5d8aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.515+0000 7f93bd000640 1 --2- 172.21.15.2:0/1554024379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d8aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.515+0000 7f93bd000640 1 -- 172.21.15.2:0/1554024379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 msgr2=0x55e6d5d8d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.515+0000 7f93bd000640 1 --2- 172.21.15.2:0/1554024379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d8d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.515+0000 7f93bd000640 1 -- 172.21.15.2:0/1554024379 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6d59f9210 con 0x55e6d5d85670 2024-09-10T08:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.515+0000 7f93b7400640 1 --2- 172.21.15.2:0/1554024379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d8d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.515+0000 7f93b7e00640 1 --2- 172.21.15.2:0/1554024379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d8aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.516+0000 7f93bd000640 1 --2- 172.21.15.2:0/1554024379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d85a70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009f30 tx=0x7f93b0010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb8e4fcd69fcfda9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.516+0000 7f93ad600640 1 -- 172.21.15.2:0/1554024379 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0016070 con 0x55e6d5d85670 2024-09-10T08:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.516+0000 7f93ad600640 1 -- 172.21.15.2:0/1554024379 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b0004d20 con 0x55e6d5d85670 2024-09-10T08:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.516+0000 7f93ad600640 1 -- 172.21.15.2:0/1554024379 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b00127b0 con 0x55e6d5d85670 2024-09-10T08:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.516+0000 7f93b4c00640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f939c0a2b80 0x7f939c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.517+0000 7f93b4c00640 1 -- 172.21.15.2:0/4184439960 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f93a80ef9f0 con 0x7f93b8101580 2024-09-10T08:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.517+0000 7f93b7e00640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f939c0a2b80 0x7f939c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.517+0000 7f93c1fe6740 1 -- 172.21.15.2:0/1554024379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 msgr2=0x55e6d5d85a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.517+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/1554024379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d85a70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009f30 tx=0x7f93b0010800 comp rx=0 tx=0).stop 2024-09-10T08:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.517+0000 7f93c1fe6740 1 -- 172.21.15.2:0/1554024379 shutdown_connections 2024-09-10T08:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.517+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/1554024379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d8d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.517+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/1554024379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d8aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.517+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/1554024379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d85a70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.517+0000 7f93c1fe6740 1 -- 172.21.15.2:0/1554024379 >> 172.21.15.2:0/1554024379 conn(0x55e6d5d80ce0 msgr2=0x55e6d5d810e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.517+0000 7f93c1fe6740 1 -- 172.21.15.2:0/1554024379 shutdown_connections 2024-09-10T08:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.517+0000 7f93c1fe6740 1 -- 172.21.15.2:0/1554024379 wait complete. 2024-09-10T08:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.517+0000 7f93b7e00640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f939c0a2b80 0x7f939c0a5040 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f93a80bbe40 tx=0x7f93a802a3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.518+0000 7f93c1fe6740 1 -- start start 2024-09-10T08:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.518+0000 7f93c1fe6740 1 Processor -- start 2024-09-10T08:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.518+0000 7f93c1fe6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d8f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.518+0000 7f93c1fe6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d8fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.518+0000 7f93c1fe6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d96d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.518+0000 7f93c1fe6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e6d5d8dfb0 con 0x55e6d5d8b040 2024-09-10T08:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.518+0000 7f93c1fe6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e6d5c6f090 con 0x55e6d5d85fb0 2024-09-10T08:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.518+0000 7f93c1fe6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e6d5d8e130 con 0x55e6d5d85670 2024-09-10T08:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.518+0000 7f93c1fe6740 10 client.0 _ll_register_callbacks cb 0x55e6d5cc3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.519+0000 7f93bd000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d8f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.519+0000 7f93bd000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d8f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59884/0 (socket says 172.21.15.2:59884) 2024-09-10T08:24:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.519+0000 7f93b7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d8fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.519+0000 7f93bd000640 1 -- 172.21.15.2:0/2752156825 learned_addr learned my addr 172.21.15.2:0/2752156825 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.519+0000 7f93bd000640 1 --2- 172.21.15.2:0/2752156825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d96d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.520+0000 7f93b7400640 1 -- 172.21.15.2:0/2752156825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 msgr2=0x55e6d5d8f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.520+0000 7f93b7400640 1 --2- 172.21.15.2:0/2752156825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d8f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.520+0000 7f93b7400640 1 -- 172.21.15.2:0/2752156825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 msgr2=0x55e6d5d96d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.520+0000 7f93b7400640 1 --2- 172.21.15.2:0/2752156825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d96d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.520+0000 7f93b7400640 1 -- 172.21.15.2:0/2752156825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6d5ac5620 con 0x55e6d5d85fb0 2024-09-10T08:24:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.520+0000 7f93bd000640 1 --2- 172.21.15.2:0/2752156825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d96d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.521+0000 7f93b7400640 1 --2- 172.21.15.2:0/2752156825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d8fd30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f93a4012cf0 tx=0x7f93a400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fc60f31890969e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.521+0000 7f93a2000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.521+0000 7f93a2000640 1 -- 172.21.15.2:0/2752156825 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a40120d0 con 0x55e6d5d85fb0 2024-09-10T08:24:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.521+0000 7f93a2000640 1 -- 172.21.15.2:0/2752156825 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f93a4004030 con 0x55e6d5d85fb0 2024-09-10T08:24:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.522+0000 7f93c1fe6740 1 -- 172.21.15.2:0/2752156825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b0004d20 con 0x55e6d5d85fb0 2024-09-10T08:24:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.522+0000 7f93a2000640 1 -- 172.21.15.2:0/2752156825 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93a4025050 con 0x55e6d5d85fb0 2024-09-10T08:24:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.522+0000 7f93c1fe6740 10 client.29969 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.522+0000 7f93c1fe6740 1 -- 172.21.15.2:0/2752156825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e6d5d99ba0 con 0x55e6d5d85fb0 2024-09-10T08:24:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.522+0000 7f93a2000640 1 -- 172.21.15.2:0/2752156825 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b0004d20 con 0x55e6d5d85fb0 2024-09-10T08:24:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.523+0000 7f93a2000640 1 -- 172.21.15.2:0/2752156825 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f93a401c460 con 0x55e6d5d85fb0 2024-09-10T08:24:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.523+0000 7f93c1fe6740 10 client.29969 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.523+0000 7f93c1fe6740 10 client.29969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.523+0000 7f93c1fe6740 20 client.29969 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.523+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/2752156825 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e6d5d9b290 0x55e6d5d9d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.523+0000 7f93c1fe6740 4 client.29969 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.523+0000 7f93c1fe6740 1 -- 172.21.15.2:0/2752156825 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6d5cf3830 con 0x55e6d5d9b290 2024-09-10T08:24:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.523+0000 7f93bd000640 1 --2- 172.21.15.2:0/2752156825 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e6d5d9b290 0x55e6d5d9d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.524+0000 7f93a2000640 10 client.29969 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.524+0000 7f93bd000640 1 --2- 172.21.15.2:0/2752156825 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e6d5d9b290 0x55e6d5d9d680 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddadc99202656e14 server_cookie=be2e1a8010e63aea in_seq=0 out_seq=0 2024-09-10T08:24:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93a2000640 1 -- 172.21.15.2:0/2752156825 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6d5cf3830 con 0x55e6d5d9b290 2024-09-10T08:24:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93a2000640 10 client.29969 handle_command_reply: tid=0 2024-09-10T08:24:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93c1fe6740 1 client.29969 shutdown 2024-09-10T08:24:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93c1fe6740 1 -- 172.21.15.2:0/2752156825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 msgr2=0x55e6d5d8fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/2752156825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d8fd30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f93a4012cf0 tx=0x7f93a400f790 comp rx=0 tx=0).stop 2024-09-10T08:24:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93c1fe6740 1 -- 172.21.15.2:0/2752156825 shutdown_connections 2024-09-10T08:24:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/2752156825 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e6d5d9b290 0x55e6d5d9d680 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/2752156825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d96d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/2752156825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d8fd30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93a2000640 0 client.29969 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/2752156825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d8f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93c1fe6740 1 -- 172.21.15.2:0/2752156825 >> 172.21.15.2:0/2752156825 conn(0x55e6d5d80ce0 msgr2=0x55e6d5d824f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93c1fe6740 1 -- 172.21.15.2:0/2752156825 shutdown_connections 2024-09-10T08:24:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93c1fe6740 1 -- 172.21.15.2:0/2752156825 wait complete. 2024-09-10T08:24:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.525+0000 7f93c1fe6740 20 client.29969 trim_cache size 0 max 16384 2024-09-10T08:24:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.542+0000 7f93c1fe6740 1 Processor -- start 2024-09-10T08:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.542+0000 7f93c1fe6740 1 -- start start 2024-09-10T08:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.542+0000 7f93c1fe6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d810c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.542+0000 7f93c1fe6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d96d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.542+0000 7f93c1fe6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d81600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.542+0000 7f93c1fe6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f93b81aad30 con 0x55e6d5d8b040 2024-09-10T08:24:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.542+0000 7f93c1fe6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93b81a8600 con 0x55e6d5d85fb0 2024-09-10T08:24:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.542+0000 7f93c1fe6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f93b81a6450 con 0x55e6d5d85670 2024-09-10T08:24:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.542+0000 7f93b7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d96d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.542+0000 7f93bd000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d810c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.542+0000 7f93b7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d96d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:53590/0 (socket says 172.21.15.2:53590) 2024-09-10T08:24:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.542+0000 7f93b7400640 1 -- 172.21.15.2:0/3411896902 learned_addr learned my addr 172.21.15.2:0/3411896902 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.543+0000 7f93bd000640 1 --2- 172.21.15.2:0/3411896902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d81600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.543+0000 7f93b7400640 1 -- 172.21.15.2:0/3411896902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 msgr2=0x55e6d5d810c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.543+0000 7f93b7400640 1 --2- 172.21.15.2:0/3411896902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d810c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.543+0000 7f93b7400640 1 -- 172.21.15.2:0/3411896902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 msgr2=0x55e6d5d81600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.543+0000 7f93b7400640 1 --2- 172.21.15.2:0/3411896902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d81600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.543+0000 7f93b7400640 1 -- 172.21.15.2:0/3411896902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6d5d80b00 con 0x55e6d5d85fb0 2024-09-10T08:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.543+0000 7f93bd000640 1 --2- 172.21.15.2:0/3411896902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d81600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.543+0000 7f93b7400640 1 --2- 172.21.15.2:0/3411896902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d96d30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f93a4001eb0 tx=0x7f93a40059b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3ec5278b0872bd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.543+0000 7f93acc00640 1 -- 172.21.15.2:0/3411896902 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a401c520 con 0x55e6d5d85fb0 2024-09-10T08:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.544+0000 7f93acc00640 1 -- 172.21.15.2:0/3411896902 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93a40120d0 con 0x55e6d5d85fb0 2024-09-10T08:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.544+0000 7f93acc00640 1 -- 172.21.15.2:0/3411896902 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4005730 con 0x55e6d5d85fb0 2024-09-10T08:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.544+0000 7f93c1fe6740 1 -- 172.21.15.2:0/3411896902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 msgr2=0x55e6d5d96d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.544+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/3411896902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d96d30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f93a4001eb0 tx=0x7f93a40059b0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.544+0000 7f93c1fe6740 1 -- 172.21.15.2:0/3411896902 shutdown_connections 2024-09-10T08:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.544+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/3411896902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d81600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.544+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/3411896902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d96d30 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.544+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/3411896902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85670 0x55e6d5d810c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.544+0000 7f93c1fe6740 1 -- 172.21.15.2:0/3411896902 >> 172.21.15.2:0/3411896902 conn(0x55e6d5d80ce0 msgr2=0x55e6d5d776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.544+0000 7f93c1fe6740 1 -- 172.21.15.2:0/3411896902 shutdown_connections 2024-09-10T08:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.544+0000 7f93c1fe6740 1 -- 172.21.15.2:0/3411896902 wait complete. 2024-09-10T08:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.545+0000 7f93c1fe6740 1 -- start start 2024-09-10T08:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.545+0000 7f93c1fe6740 1 Processor -- start 2024-09-10T08:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.545+0000 7f93c1fe6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85670 0x55e6d5d93ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.545+0000 7f93c1fe6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d94530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.545+0000 7f93c1fe6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d95540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.545+0000 7f93c1fe6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f93b81a6ac0 con 0x55e6d5d8b040 2024-09-10T08:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.545+0000 7f93c1fe6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93b81a7f30 con 0x55e6d5d85670 2024-09-10T08:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.545+0000 7f93c1fe6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f93b81a7190 con 0x55e6d5d85fb0 2024-09-10T08:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.545+0000 7f93c1fe6740 10 client.0 _ll_register_callbacks cb 0x7f93a4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.546+0000 7f93b7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d94530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.546+0000 7f93bd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85670 0x55e6d5d93ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.546+0000 7f93b7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d94530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:59902/0 (socket says 172.21.15.2:59902) 2024-09-10T08:24:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.546+0000 7f93b7400640 1 -- 172.21.15.2:0/1765596851 learned_addr learned my addr 172.21.15.2:0/1765596851 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.546+0000 7f93bd000640 1 --2- 172.21.15.2:0/1765596851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d95540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.546+0000 7f93b7400640 1 -- 172.21.15.2:0/1765596851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85670 msgr2=0x55e6d5d93ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.546+0000 7f93b7400640 1 --2- 172.21.15.2:0/1765596851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85670 0x55e6d5d93ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.546+0000 7f93b7400640 1 -- 172.21.15.2:0/1765596851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 msgr2=0x55e6d5d95540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.546+0000 7f93b7400640 1 --2- 172.21.15.2:0/1765596851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d95540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.546+0000 7f93b7400640 1 -- 172.21.15.2:0/1765596851 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6d5d95b50 con 0x55e6d5d85fb0 2024-09-10T08:24:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.547+0000 7f93b7400640 1 --2- 172.21.15.2:0/1765596851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d94530 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f93a400fd90 tx=0x7f93a4005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=202c0a80561b2d4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.547+0000 7f93a2000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:24:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.547+0000 7f93a2000640 1 -- 172.21.15.2:0/1765596851 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4012280 con 0x55e6d5d85fb0 2024-09-10T08:24:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.547+0000 7f93c1fe6740 1 -- 172.21.15.2:0/1765596851 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a40120d0 con 0x55e6d5d85fb0 2024-09-10T08:24:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.547+0000 7f93a2000640 1 -- 172.21.15.2:0/1765596851 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f93a4018040 con 0x55e6d5d85fb0 2024-09-10T08:24:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.547+0000 7f93a2000640 1 -- 172.21.15.2:0/1765596851 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93a403b050 con 0x55e6d5d85fb0 2024-09-10T08:24:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.547+0000 7f93c1fe6740 10 client.29905 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.547+0000 7f93c1fe6740 1 -- 172.21.15.2:0/1765596851 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e6d5d96e60 con 0x55e6d5d85fb0 2024-09-10T08:24:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.548+0000 7f93a2000640 1 -- 172.21.15.2:0/1765596851 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93a40120d0 con 0x55e6d5d85fb0 2024-09-10T08:24:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.548+0000 7f93a2000640 1 -- 172.21.15.2:0/1765596851 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f93a4003c70 con 0x55e6d5d85fb0 2024-09-10T08:24:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.548+0000 7f93c1fe6740 10 client.29905 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.548+0000 7f93c1fe6740 10 client.29905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.548+0000 7f93c1fe6740 20 client.29905 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.548+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/1765596851 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e6d5e06420 0x55e6d5e08840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.548+0000 7f93c1fe6740 4 client.29905 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:24:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.548+0000 7f93c1fe6740 1 -- 172.21.15.2:0/1765596851 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6d5cedb80 con 0x55e6d5e06420 2024-09-10T08:24:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.548+0000 7f93bd000640 1 --2- 172.21.15.2:0/1765596851 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e6d5e06420 0x55e6d5e08840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.549+0000 7f93bd000640 1 --2- 172.21.15.2:0/1765596851 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e6d5e06420 0x55e6d5e08840 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d9b973a2d7bcc4 server_cookie=23507aee642bfa51 in_seq=0 out_seq=0 2024-09-10T08:24:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.549+0000 7f93a2000640 10 client.29905 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93a2000640 1 -- 172.21.15.2:0/1765596851 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6d5cedb80 con 0x55e6d5e06420 2024-09-10T08:24:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93a2000640 10 client.29905 handle_command_reply: tid=0 2024-09-10T08:24:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93c1fe6740 1 client.29905 shutdown 2024-09-10T08:24:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93c1fe6740 1 -- 172.21.15.2:0/1765596851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85fb0 msgr2=0x55e6d5d94530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/1765596851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d94530 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f93a400fd90 tx=0x7f93a4005f10 comp rx=0 tx=0).stop 2024-09-10T08:24:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93c1fe6740 1 -- 172.21.15.2:0/1765596851 shutdown_connections 2024-09-10T08:24:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/1765596851 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e6d5e06420 0x55e6d5e08840 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/1765596851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e6d5d8b040 0x55e6d5d95540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/1765596851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6d5d85fb0 0x55e6d5d94530 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93c1fe6740 1 --2- 172.21.15.2:0/1765596851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d5d85670 0x55e6d5d93ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93a2000640 0 client.29905 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93c1fe6740 1 -- 172.21.15.2:0/1765596851 >> 172.21.15.2:0/1765596851 conn(0x55e6d5d80ce0 msgr2=0x55e6d5d7b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93c1fe6740 1 -- 172.21.15.2:0/1765596851 shutdown_connections 2024-09-10T08:24:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93c1fe6740 1 -- 172.21.15.2:0/1765596851 wait complete. 2024-09-10T08:24:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.561+0000 7f93c1fe6740 20 client.29905 trim_cache size 0 max 16384 2024-09-10T08:24:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:24:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:24:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:24:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:24:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.562+0000 7f93bda00640 1 -- 172.21.15.2:0/4184439960 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f939c0a2b80 msgr2=0x7f939c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.562+0000 7f93bda00640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f939c0a2b80 0x7f939c0a5040 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f93a80bbe40 tx=0x7f93a802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.562+0000 7f93bda00640 1 -- 172.21.15.2:0/4184439960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8101580 msgr2=0x7f93b8197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.562+0000 7f93bda00640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8101580 0x7f93b8197dd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f93a8031440 tx=0x7f93a8032ce0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.562+0000 7f93bda00640 1 -- 172.21.15.2:0/4184439960 shutdown_connections 2024-09-10T08:24:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.562+0000 7f93bda00640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f939c0a2b80 0x7f939c0a5040 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.562+0000 7f93bda00640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8103050 0x7f93b819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.562+0000 7f93bda00640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8101f40 0x7f93b8198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.562+0000 7f93bda00640 1 --2- 172.21.15.2:0/4184439960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8101580 0x7f93b8197dd0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.562+0000 7f93bda00640 1 -- 172.21.15.2:0/4184439960 >> 172.21.15.2:0/4184439960 conn(0x7f93b80fd0d0 msgr2=0x7f93b8105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.562+0000 7f93bda00640 1 -- 172.21.15.2:0/4184439960 shutdown_connections 2024-09-10T08:24:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:05.562+0000 7f93bda00640 1 -- 172.21.15.2:0/4184439960 wait complete. 2024-09-10T08:24:05.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:05 smithi002 ceph-mon[30563]: pgmap v865: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-09-10T08:24:05.842 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:05.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:05 smithi155 ceph-mon[35405]: pgmap v865: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-09-10T08:24:05.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:05 smithi107 ceph-mon[35048]: pgmap v865: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-09-10T08:24:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.461+0000 7fe340e00640 1 -- 172.21.15.2:0/253282458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe33c108f60 msgr2=0x7fe33c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.461+0000 7fe340e00640 1 --2- 172.21.15.2:0/253282458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe33c108f60 0x7fe33c110ae0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe334009900 tx=0x7fe33402a410 comp rx=0 tx=0).stop 2024-09-10T08:24:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.461+0000 7fe340e00640 1 -- 172.21.15.2:0/253282458 shutdown_connections 2024-09-10T08:24:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.461+0000 7fe340e00640 1 --2- 172.21.15.2:0/253282458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe33c108f60 0x7fe33c110ae0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.461+0000 7fe340e00640 1 --2- 172.21.15.2:0/253282458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe33c1047e0 0x7fe33c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.461+0000 7fe340e00640 1 --2- 172.21.15.2:0/253282458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c103e30 0x7fe33c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.461+0000 7fe340e00640 1 -- 172.21.15.2:0/253282458 >> 172.21.15.2:0/253282458 conn(0x7fe33c0ff910 msgr2=0x7fe33c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.461+0000 7fe340e00640 1 -- 172.21.15.2:0/253282458 shutdown_connections 2024-09-10T08:24:06.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.461+0000 7fe340e00640 1 -- 172.21.15.2:0/253282458 wait complete. 2024-09-10T08:24:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.462+0000 7fe340e00640 1 Processor -- start 2024-09-10T08:24:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.463+0000 7fe340e00640 1 -- start start 2024-09-10T08:24:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.463+0000 7fe340e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe33c103e30 0x7fe33c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.463+0000 7fe340e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c1047e0 0x7fe33c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.463+0000 7fe340e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe33c108f60 0x7fe33c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.463+0000 7fe340e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c114010 con 0x7fe33c108f60 2024-09-10T08:24:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.463+0000 7fe340e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c113e90 con 0x7fe33c1047e0 2024-09-10T08:24:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.463+0000 7fe340e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c114190 con 0x7fe33c103e30 2024-09-10T08:24:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.463+0000 7fe33be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe33c108f60 0x7fe33c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.463+0000 7fe33b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe33c103e30 0x7fe33c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:06.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.463+0000 7fe33be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe33c108f60 0x7fe33c19ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33474/0 (socket says 172.21.15.2:33474) 2024-09-10T08:24:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.463+0000 7fe33be00640 1 -- 172.21.15.2:0/4241027789 learned_addr learned my addr 172.21.15.2:0/4241027789 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.463+0000 7fe332a00640 1 --2- 172.21.15.2:0/4241027789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c1047e0 0x7fe33c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.464+0000 7fe33b400640 1 -- 172.21.15.2:0/4241027789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c1047e0 msgr2=0x7fe33c19a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.464+0000 7fe33b400640 1 --2- 172.21.15.2:0/4241027789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c1047e0 0x7fe33c19a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.464+0000 7fe33b400640 1 -- 172.21.15.2:0/4241027789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe33c108f60 msgr2=0x7fe33c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.464+0000 7fe33b400640 1 --2- 172.21.15.2:0/4241027789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe33c108f60 0x7fe33c19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:06.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.464+0000 7fe33b400640 1 -- 172.21.15.2:0/4241027789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c19f3c0 con 0x7fe33c103e30 2024-09-10T08:24:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.464+0000 7fe33b400640 1 --2- 172.21.15.2:0/4241027789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe33c103e30 0x7fe33c19a410 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00eb00 tx=0x7fe32c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.464+0000 7fe338c00640 1 -- 172.21.15.2:0/4241027789 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c010430 con 0x7fe33c103e30 2024-09-10T08:24:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.464+0000 7fe340e00640 1 -- 172.21.15.2:0/4241027789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33c19f6b0 con 0x7fe33c103e30 2024-09-10T08:24:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.464+0000 7fe338c00640 1 -- 172.21.15.2:0/4241027789 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe32c010ab0 con 0x7fe33c103e30 2024-09-10T08:24:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.464+0000 7fe338c00640 1 -- 172.21.15.2:0/4241027789 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c017bb0 con 0x7fe33c103e30 2024-09-10T08:24:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.465+0000 7fe340e00640 1 -- 172.21.15.2:0/4241027789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33c1a6f90 con 0x7fe33c103e30 2024-09-10T08:24:06.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.466+0000 7fe340e00640 1 -- 172.21.15.2:0/4241027789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33c105310 con 0x7fe33c103e30 2024-09-10T08:24:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.468+0000 7fe338c00640 1 -- 172.21.15.2:0/4241027789 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe32c010c60 con 0x7fe33c103e30 2024-09-10T08:24:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.469+0000 7fe338c00640 1 --2- 172.21.15.2:0/4241027789 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3200a2b80 0x7fe3200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.469+0000 7fe338c00640 1 -- 172.21.15.2:0/4241027789 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe32c0d0060 con 0x7fe33c103e30 2024-09-10T08:24:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.469+0000 7fe332a00640 1 --2- 172.21.15.2:0/4241027789 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3200a2b80 0x7fe3200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:06.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.470+0000 7fe332a00640 1 --2- 172.21.15.2:0/4241027789 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3200a2b80 0x7fe3200a5040 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe33c101730 tx=0x7fe324005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:06.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.471+0000 7fe338c00640 1 -- 172.21.15.2:0/4241027789 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe32c0541b0 con 0x7fe33c103e30 2024-09-10T08:24:06.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.697+0000 7fe340e00640 1 -- 172.21.15.2:0/4241027789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe33c1a7260 con 0x7fe33c103e30 2024-09-10T08:24:06.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.698+0000 7fe338c00640 1 -- 172.21.15.2:0/4241027789 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fe32c09cb70 con 0x7fe33c103e30 2024-09-10T08:24:06.699 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:24:06.699 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:24:06.699 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:24:06.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.700+0000 7fe340e00640 1 -- 172.21.15.2:0/4241027789 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3200a2b80 msgr2=0x7fe3200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:06.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.700+0000 7fe340e00640 1 --2- 172.21.15.2:0/4241027789 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3200a2b80 0x7fe3200a5040 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe33c101730 tx=0x7fe324005f90 comp rx=0 tx=0).stop 2024-09-10T08:24:06.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.700+0000 7fe340e00640 1 -- 172.21.15.2:0/4241027789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe33c103e30 msgr2=0x7fe33c19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:06.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.700+0000 7fe340e00640 1 --2- 172.21.15.2:0/4241027789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe33c103e30 0x7fe33c19a410 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00eb00 tx=0x7fe32c00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:24:06.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.701+0000 7fe340e00640 1 -- 172.21.15.2:0/4241027789 shutdown_connections 2024-09-10T08:24:06.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.701+0000 7fe340e00640 1 --2- 172.21.15.2:0/4241027789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe33c108f60 0x7fe33c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:06.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.701+0000 7fe340e00640 1 --2- 172.21.15.2:0/4241027789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c1047e0 0x7fe33c19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:06.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.701+0000 7fe340e00640 1 --2- 172.21.15.2:0/4241027789 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe3200a2b80 0x7fe3200a5040 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:06.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.701+0000 7fe340e00640 1 --2- 172.21.15.2:0/4241027789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe33c103e30 0x7fe33c19a410 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:06.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.701+0000 7fe340e00640 1 -- 172.21.15.2:0/4241027789 >> 172.21.15.2:0/4241027789 conn(0x7fe33c0ff910 msgr2=0x7fe33c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:06.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.701+0000 7fe340e00640 1 -- 172.21.15.2:0/4241027789 shutdown_connections 2024-09-10T08:24:06.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:06.701+0000 7fe340e00640 1 -- 172.21.15.2:0/4241027789 wait complete. 2024-09-10T08:24:07.611 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:24:07.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:07 smithi155 ceph-mon[35405]: pgmap v866: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-09-10T08:24:07.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:07 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/4241027789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:07.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:07 smithi107 ceph-mon[35048]: pgmap v866: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-09-10T08:24:07.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:07 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/4241027789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:07.990 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:08.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:07 smithi002 ceph-mon[30563]: pgmap v866: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-09-10T08:24:08.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:07 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/4241027789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:08.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.801+0000 7fcf7b000640 1 -- 172.21.15.2:0/4119633881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf741079c0 msgr2=0x7fcf74107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:08.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.801+0000 7fcf7b000640 1 --2- 172.21.15.2:0/4119633881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf741079c0 0x7fcf74107da0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009860 tx=0x7fcf6802a340 comp rx=0 tx=0).stop 2024-09-10T08:24:08.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.801+0000 7fcf7b000640 1 -- 172.21.15.2:0/4119633881 shutdown_connections 2024-09-10T08:24:08.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.801+0000 7fcf7b000640 1 --2- 172.21.15.2:0/4119633881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf74102360 0x7fcf7410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.801+0000 7fcf7b000640 1 --2- 172.21.15.2:0/4119633881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf741019c0 0x7fcf74101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.801+0000 7fcf7b000640 1 --2- 172.21.15.2:0/4119633881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf741079c0 0x7fcf74107da0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.801+0000 7fcf7b000640 1 -- 172.21.15.2:0/4119633881 >> 172.21.15.2:0/4119633881 conn(0x7fcf740fd740 msgr2=0x7fcf740ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:08.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.801+0000 7fcf7b000640 1 -- 172.21.15.2:0/4119633881 shutdown_connections 2024-09-10T08:24:08.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.801+0000 7fcf7b000640 1 -- 172.21.15.2:0/4119633881 wait complete. 2024-09-10T08:24:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.802+0000 7fcf7b000640 1 Processor -- start 2024-09-10T08:24:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.802+0000 7fcf7b000640 1 -- start start 2024-09-10T08:24:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.803+0000 7fcf7b000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf741019c0 0x7fcf7419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.803+0000 7fcf7b000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf74102360 0x7fcf7419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.803+0000 7fcf7b000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf741079c0 0x7fcf7419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.803+0000 7fcf7b000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74111dc0 con 0x7fcf74102360 2024-09-10T08:24:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.803+0000 7fcf7b000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74111c40 con 0x7fcf741019c0 2024-09-10T08:24:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.803+0000 7fcf7b000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74111f40 con 0x7fcf741079c0 2024-09-10T08:24:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.803+0000 7fcf79200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf74102360 0x7fcf7419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.803+0000 7fcf79200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf74102360 0x7fcf7419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47216/0 (socket says 172.21.15.2:47216) 2024-09-10T08:24:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.803+0000 7fcf79200640 1 -- 172.21.15.2:0/1957036880 learned_addr learned my addr 172.21.15.2:0/1957036880 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.803+0000 7fcf79c00640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf741019c0 0x7fcf7419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.803+0000 7fcf7a600640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf741079c0 0x7fcf7419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.804+0000 7fcf79c00640 1 -- 172.21.15.2:0/1957036880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf741079c0 msgr2=0x7fcf7419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.804+0000 7fcf79c00640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf741079c0 0x7fcf7419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.804+0000 7fcf79c00640 1 -- 172.21.15.2:0/1957036880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf74102360 msgr2=0x7fcf7419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.804+0000 7fcf79c00640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf74102360 0x7fcf7419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.804+0000 7fcf79c00640 1 -- 172.21.15.2:0/1957036880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7419f440 con 0x7fcf741019c0 2024-09-10T08:24:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.804+0000 7fcf79200640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf74102360 0x7fcf7419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.804+0000 7fcf7a600640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf741079c0 0x7fcf7419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:08.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.804+0000 7fcf79c00640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf741019c0 0x7fcf7419a490 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcf6802a860 tx=0x7fcf68033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:08.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.804+0000 7fcf72000640 1 -- 172.21.15.2:0/1957036880 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68037040 con 0x7fcf741019c0 2024-09-10T08:24:08.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.804+0000 7fcf7b000640 1 -- 172.21.15.2:0/1957036880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7419f6d0 con 0x7fcf741019c0 2024-09-10T08:24:08.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.804+0000 7fcf72000640 1 -- 172.21.15.2:0/1957036880 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf68033d50 con 0x7fcf741019c0 2024-09-10T08:24:08.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.804+0000 7fcf72000640 1 -- 172.21.15.2:0/1957036880 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf680039e0 con 0x7fcf741019c0 2024-09-10T08:24:08.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.805+0000 7fcf7b000640 1 -- 172.21.15.2:0/1957036880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf741a7010 con 0x7fcf741019c0 2024-09-10T08:24:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.807+0000 7fcf72000640 1 -- 172.21.15.2:0/1957036880 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcf68042020 con 0x7fcf741019c0 2024-09-10T08:24:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.808+0000 7fcf7f07f740 1 Processor -- start 2024-09-10T08:24:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.808+0000 7fcf7f07f740 1 -- start start 2024-09-10T08:24:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.808+0000 7fcf7f07f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51778e40 0x55fe51779220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.808+0000 7fcf7f07f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51779760 0x55fe51783ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.808+0000 7fcf7f07f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51784020 0x55fe51786410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.808+0000 7fcf7f07f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fe516df3e0 con 0x55fe51784020 2024-09-10T08:24:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.808+0000 7fcf7f07f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fe51686d80 con 0x55fe51779760 2024-09-10T08:24:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.808+0000 7fcf7f07f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fe516db3a0 con 0x55fe51778e40 2024-09-10T08:24:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.809+0000 7fcf79c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51779760 0x55fe51783ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.809+0000 7fcf7a600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51778e40 0x55fe51779220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.809+0000 7fcf7a600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51778e40 0x55fe51779220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:46394/0 (socket says 172.21.15.2:46394) 2024-09-10T08:24:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.809+0000 7fcf7a600640 1 -- 172.21.15.2:0/3285954659 learned_addr learned my addr 172.21.15.2:0/3285954659 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.809+0000 7fcf79200640 1 --2- 172.21.15.2:0/3285954659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51784020 0x55fe51786410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.809+0000 7fcf7a600640 1 -- 172.21.15.2:0/3285954659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51779760 msgr2=0x55fe51783ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:08.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.809+0000 7fcf7a600640 1 --2- 172.21.15.2:0/3285954659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51779760 0x55fe51783ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.809+0000 7fcf7a600640 1 -- 172.21.15.2:0/3285954659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51784020 msgr2=0x55fe51786410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.809+0000 7fcf7a600640 1 --2- 172.21.15.2:0/3285954659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51784020 0x55fe51786410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.809+0000 7fcf7a600640 1 -- 172.21.15.2:0/3285954659 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe516a5ae0 con 0x55fe51778e40 2024-09-10T08:24:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.809+0000 7fcf79200640 1 --2- 172.21.15.2:0/3285954659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51784020 0x55fe51786410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.810+0000 7fcf72000640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf5c0a2b80 0x7fcf5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.810+0000 7fcf7a600640 1 --2- 172.21.15.2:0/3285954659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51778e40 0x55fe51779220 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c011790 tx=0x7fcf6c011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bab96a34daf49f0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.810+0000 7fcf72000640 1 -- 172.21.15.2:0/1957036880 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcf680ee9f0 con 0x7fcf741019c0 2024-09-10T08:24:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.810+0000 7fcf66a00640 1 -- 172.21.15.2:0/3285954659 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c003c30 con 0x55fe51778e40 2024-09-10T08:24:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.810+0000 7fcf66a00640 1 -- 172.21.15.2:0/3285954659 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6c00fe80 con 0x55fe51778e40 2024-09-10T08:24:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.810+0000 7fcf66a00640 1 -- 172.21.15.2:0/3285954659 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0193e0 con 0x55fe51778e40 2024-09-10T08:24:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.810+0000 7fcf79c00640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf5c0a2b80 0x7fcf5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.811+0000 7fcf7f07f740 1 -- 172.21.15.2:0/3285954659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51778e40 msgr2=0x55fe51779220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.811+0000 7fcf79c00640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf5c0a2b80 0x7fcf5c0a5040 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009780 tx=0x7fcf680f4530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.811+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/3285954659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51778e40 0x55fe51779220 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c011790 tx=0x7fcf6c011c50 comp rx=0 tx=0).stop 2024-09-10T08:24:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.811+0000 7fcf7f07f740 1 -- 172.21.15.2:0/3285954659 shutdown_connections 2024-09-10T08:24:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.811+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/3285954659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51784020 0x55fe51786410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.811+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/3285954659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51779760 0x55fe51783ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.811+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/3285954659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51778e40 0x55fe51779220 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.811+0000 7fcf7f07f740 1 -- 172.21.15.2:0/3285954659 >> 172.21.15.2:0/3285954659 conn(0x55fe517753d0 msgr2=0x55fe517757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:08.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.811+0000 7fcf7f07f740 1 -- 172.21.15.2:0/3285954659 shutdown_connections 2024-09-10T08:24:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.811+0000 7fcf7f07f740 1 -- 172.21.15.2:0/3285954659 wait complete. 2024-09-10T08:24:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.812+0000 7fcf7f07f740 1 -- start start 2024-09-10T08:24:08.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.812+0000 7fcf7f07f740 1 Processor -- start 2024-09-10T08:24:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.812+0000 7fcf7f07f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 0x55fe51780f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.812+0000 7fcf7f07f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe51781440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.812+0000 7fcf7f07f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 0x55fe51781980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.812+0000 7fcf7f07f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fe51786bf0 con 0x55fe51779760 2024-09-10T08:24:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.812+0000 7fcf7f07f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fe5169a0f0 con 0x55fe51778e40 2024-09-10T08:24:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.812+0000 7fcf7f07f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fe51786d70 con 0x55fe51784020 2024-09-10T08:24:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.812+0000 7fcf7f07f740 10 client.0 _ll_register_callbacks cb 0x55fe51681bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.813+0000 7fcf79200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe51781440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.813+0000 7fcf79200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe51781440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47228/0 (socket says 172.21.15.2:47228) 2024-09-10T08:24:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.813+0000 7fcf7a600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 0x55fe51781980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.813+0000 7fcf79200640 1 -- 172.21.15.2:0/1165500116 learned_addr learned my addr 172.21.15.2:0/1165500116 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.813+0000 7fcf7a600640 1 --2- 172.21.15.2:0/1165500116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 0x55fe51780f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.813+0000 7fcf79200640 1 -- 172.21.15.2:0/1165500116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 msgr2=0x55fe51781980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.813+0000 7fcf79200640 1 --2- 172.21.15.2:0/1165500116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 0x55fe51781980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.813+0000 7fcf79200640 1 -- 172.21.15.2:0/1165500116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 msgr2=0x55fe51780f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:08.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.813+0000 7fcf79200640 1 --2- 172.21.15.2:0/1165500116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 0x55fe51780f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.813+0000 7fcf79200640 1 -- 172.21.15.2:0/1165500116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe514503c0 con 0x55fe51779760 2024-09-10T08:24:08.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.813+0000 7fcf7a600640 1 --2- 172.21.15.2:0/1165500116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 0x55fe51780f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:08.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.813+0000 7fcf79200640 1 --2- 172.21.15.2:0/1165500116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe51781440 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcf60011d40 tx=0x7fcf6000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d06709bafa609f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:08.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.813+0000 7fcf5b400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:24:08.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.814+0000 7fcf5b400640 1 -- 172.21.15.2:0/1165500116 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6000fe20 con 0x55fe51779760 2024-09-10T08:24:08.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.814+0000 7fcf5b400640 1 -- 172.21.15.2:0/1165500116 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcf6000e5c0 con 0x55fe51779760 2024-09-10T08:24:08.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.814+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1165500116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fe516ec950 con 0x55fe51779760 2024-09-10T08:24:08.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.814+0000 7fcf5b400640 1 -- 172.21.15.2:0/1165500116 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf60027050 con 0x55fe51779760 2024-09-10T08:24:08.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.814+0000 7fcf7f07f740 10 client.19977 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:08.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.814+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1165500116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fe51792c90 con 0x55fe51779760 2024-09-10T08:24:08.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.814+0000 7fcf5b400640 1 -- 172.21.15.2:0/1165500116 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fe516ec950 con 0x55fe51779760 2024-09-10T08:24:08.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.815+0000 7fcf5b400640 1 -- 172.21.15.2:0/1165500116 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcf60022030 con 0x55fe51779760 2024-09-10T08:24:08.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.815+0000 7fcf7f07f740 10 client.19977 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.815+0000 7fcf7f07f740 10 client.19977 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:24:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.815+0000 7fcf7f07f740 20 client.19977 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.815+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/1165500116 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55fe517941a0 0x55fe51796600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.815+0000 7fcf7f07f740 4 client.19977 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.815+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1165500116 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe516e6e60 con 0x55fe517941a0 2024-09-10T08:24:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.815+0000 7fcf7a600640 1 --2- 172.21.15.2:0/1165500116 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55fe517941a0 0x55fe51796600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.816+0000 7fcf7a600640 1 --2- 172.21.15.2:0/1165500116 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55fe517941a0 0x55fe51796600 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=727317357a26d39b server_cookie=647314e76cf7ef0d in_seq=0 out_seq=0 2024-09-10T08:24:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.816+0000 7fcf5b400640 10 client.19977 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:24:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf5b400640 1 -- 172.21.15.2:0/1165500116 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe516e6e60 con 0x55fe517941a0 2024-09-10T08:24:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf5b400640 10 client.19977 handle_command_reply: tid=0 2024-09-10T08:24:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf7f07f740 1 client.19977 shutdown 2024-09-10T08:24:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1165500116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 msgr2=0x55fe51781440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:08.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/1165500116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe51781440 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcf60011d40 tx=0x7fcf6000f790 comp rx=0 tx=0).stop 2024-09-10T08:24:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1165500116 shutdown_connections 2024-09-10T08:24:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/1165500116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 0x55fe51781980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/1165500116 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55fe517941a0 0x55fe51796600 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/1165500116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe51781440 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/1165500116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 0x55fe51780f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1165500116 >> 172.21.15.2:0/1165500116 conn(0x55fe517753d0 msgr2=0x55fe5177b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf5b400640 0 client.19977 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:24:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1165500116 shutdown_connections 2024-09-10T08:24:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1165500116 wait complete. 2024-09-10T08:24:08.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.817+0000 7fcf7f07f740 20 client.19977 trim_cache size 0 max 16384 2024-09-10T08:24:08.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.831+0000 7fcf7f07f740 1 Processor -- start 2024-09-10T08:24:08.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.831+0000 7fcf7f07f740 1 -- start start 2024-09-10T08:24:08.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.831+0000 7fcf7f07f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 0x55fe5177b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.831+0000 7fcf7f07f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe51782e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.831+0000 7fcf7f07f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 0x55fe5179bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.831+0000 7fcf7f07f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b37a0 con 0x55fe51779760 2024-09-10T08:24:08.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.831+0000 7fcf7f07f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b1070 con 0x55fe51778e40 2024-09-10T08:24:08.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.831+0000 7fcf7f07f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741aeec0 con 0x55fe51784020 2024-09-10T08:24:08.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.832+0000 7fcf79200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe51782e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.832+0000 7fcf79200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe51782e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47240/0 (socket says 172.21.15.2:47240) 2024-09-10T08:24:08.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.832+0000 7fcf7a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 0x55fe5177b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.832+0000 7fcf79200640 1 -- 172.21.15.2:0/4107971467 learned_addr learned my addr 172.21.15.2:0/4107971467 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:08.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.832+0000 7fcf7a600640 1 --2- 172.21.15.2:0/4107971467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 0x55fe5179bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.832+0000 7fcf79200640 1 -- 172.21.15.2:0/4107971467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 msgr2=0x55fe5179bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:08.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.832+0000 7fcf79200640 1 --2- 172.21.15.2:0/4107971467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 0x55fe5179bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.832+0000 7fcf79200640 1 -- 172.21.15.2:0/4107971467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 msgr2=0x55fe5177b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:08.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.832+0000 7fcf79200640 1 --2- 172.21.15.2:0/4107971467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 0x55fe5177b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.832+0000 7fcf79200640 1 -- 172.21.15.2:0/4107971467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe51776480 con 0x55fe51779760 2024-09-10T08:24:08.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.832+0000 7fcf79200640 1 --2- 172.21.15.2:0/4107971467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe51782e70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fcf60000c00 tx=0x7fcf60011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac11b0174a3b6956 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:08.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.833+0000 7fcf66000640 1 -- 172.21.15.2:0/4107971467 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6001a3f0 con 0x55fe51779760 2024-09-10T08:24:08.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.833+0000 7fcf66000640 1 -- 172.21.15.2:0/4107971467 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf600055f0 con 0x55fe51779760 2024-09-10T08:24:08.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.833+0000 7fcf66000640 1 -- 172.21.15.2:0/4107971467 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60003b70 con 0x55fe51779760 2024-09-10T08:24:08.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.833+0000 7fcf7f07f740 1 -- 172.21.15.2:0/4107971467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 msgr2=0x55fe51782e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:08.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.833+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/4107971467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe51782e70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fcf60000c00 tx=0x7fcf60011900 comp rx=0 tx=0).stop 2024-09-10T08:24:08.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.833+0000 7fcf7f07f740 1 -- 172.21.15.2:0/4107971467 shutdown_connections 2024-09-10T08:24:08.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.833+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/4107971467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 0x55fe5179bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.833+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/4107971467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe51782e70 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.833+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/4107971467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 0x55fe5177b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.833+0000 7fcf7f07f740 1 -- 172.21.15.2:0/4107971467 >> 172.21.15.2:0/4107971467 conn(0x55fe517753d0 msgr2=0x55fe51782760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.834+0000 7fcf7f07f740 1 -- 172.21.15.2:0/4107971467 shutdown_connections 2024-09-10T08:24:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.834+0000 7fcf7f07f740 1 -- 172.21.15.2:0/4107971467 wait complete. 2024-09-10T08:24:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.834+0000 7fcf7f07f740 1 -- start start 2024-09-10T08:24:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.834+0000 7fcf7f07f740 1 Processor -- start 2024-09-10T08:24:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf7f07f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 0x55fe5178c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf7f07f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe5177da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf7f07f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 0x55fe5177dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf7f07f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741af530 con 0x55fe51779760 2024-09-10T08:24:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf7f07f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b09a0 con 0x55fe51778e40 2024-09-10T08:24:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf7f07f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741afc00 con 0x55fe51784020 2024-09-10T08:24:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf7f07f740 10 client.0 _ll_register_callbacks cb 0x7fcf60009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf79200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe5177da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf79200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe5177da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47248/0 (socket says 172.21.15.2:47248) 2024-09-10T08:24:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf79200640 1 -- 172.21.15.2:0/1244398441 learned_addr learned my addr 172.21.15.2:0/1244398441 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf7a600640 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 0x55fe5178c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf7a600640 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 0x55fe5177dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf79200640 1 -- 172.21.15.2:0/1244398441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 msgr2=0x55fe5177dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf79200640 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 0x55fe5177dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf79200640 1 -- 172.21.15.2:0/1244398441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 msgr2=0x55fe5178c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf79200640 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 0x55fe5178c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf79200640 1 -- 172.21.15.2:0/1244398441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe5177e560 con 0x55fe51779760 2024-09-10T08:24:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.835+0000 7fcf7a600640 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 0x55fe5177dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.836+0000 7fcf79200640 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe5177da60 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fcf600092a0 tx=0x7fcf60011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469b74d6f5bb1a2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:08.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.836+0000 7fcf5b400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:24:08.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.836+0000 7fcf5b400640 1 -- 172.21.15.2:0/1244398441 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60038070 con 0x55fe51779760 2024-09-10T08:24:08.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.836+0000 7fcf5b400640 1 -- 172.21.15.2:0/1244398441 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcf60005320 con 0x55fe51779760 2024-09-10T08:24:08.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.836+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1244398441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf600055f0 con 0x55fe51779760 2024-09-10T08:24:08.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.836+0000 7fcf5b400640 1 -- 172.21.15.2:0/1244398441 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf60040050 con 0x55fe51779760 2024-09-10T08:24:08.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.836+0000 7fcf7f07f740 10 client.19983 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:08.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.836+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1244398441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fe517fec50 con 0x55fe51779760 2024-09-10T08:24:08.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.837+0000 7fcf5b400640 1 -- 172.21.15.2:0/1244398441 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf600055f0 con 0x55fe51779760 2024-09-10T08:24:08.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.837+0000 7fcf5b400640 1 -- 172.21.15.2:0/1244398441 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcf6003b030 con 0x55fe51779760 2024-09-10T08:24:08.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.837+0000 7fcf7f07f740 10 client.19983 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:08.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.837+0000 7fcf7f07f740 10 client.19983 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:24:08.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.837+0000 7fcf7f07f740 20 client.19983 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:08.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.837+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55fe517ff300 0x55fe51801720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:08.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.837+0000 7fcf7f07f740 4 client.19983 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:24:08.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.837+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1244398441 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe51663010 con 0x55fe517ff300 2024-09-10T08:24:08.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.837+0000 7fcf7a600640 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55fe517ff300 0x55fe51801720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.838+0000 7fcf7a600640 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55fe517ff300 0x55fe51801720 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee0c55d91f82a20a server_cookie=66a8fc9e261f4fa in_seq=0 out_seq=0 2024-09-10T08:24:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.838+0000 7fcf5b400640 10 client.19983 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:24:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.838+0000 7fcf5b400640 1 -- 172.21.15.2:0/1244398441 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe51663010 con 0x55fe517ff300 2024-09-10T08:24:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.838+0000 7fcf5b400640 10 client.19983 handle_command_reply: tid=0 2024-09-10T08:24:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.838+0000 7fcf7f07f740 1 client.19983 shutdown 2024-09-10T08:24:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.838+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1244398441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 msgr2=0x55fe5177da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.838+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe5177da60 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fcf600092a0 tx=0x7fcf60011c70 comp rx=0 tx=0).stop 2024-09-10T08:24:08.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.838+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1244398441 shutdown_connections 2024-09-10T08:24:08.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.838+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe51784020 0x55fe5177dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.838+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x55fe517ff300 0x55fe51801720 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.838+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe51779760 0x55fe5177da60 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.838+0000 7fcf7f07f740 1 --2- 172.21.15.2:0/1244398441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe51778e40 0x55fe5178c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.838+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1244398441 >> 172.21.15.2:0/1244398441 conn(0x55fe517753d0 msgr2=0x55fe517816e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:08.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.839+0000 7fcf5b400640 0 client.19983 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:24:08.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.839+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1244398441 shutdown_connections 2024-09-10T08:24:08.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.839+0000 7fcf7f07f740 1 -- 172.21.15.2:0/1244398441 wait complete. 2024-09-10T08:24:08.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.839+0000 7fcf7f07f740 20 client.19983 trim_cache size 0 max 16384 2024-09-10T08:24:08.843 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:24:08.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.839+0000 7fcf7b000640 1 -- 172.21.15.2:0/1957036880 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf5c0a2b80 msgr2=0x7fcf5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:08.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.840+0000 7fcf7b000640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf5c0a2b80 0x7fcf5c0a5040 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009780 tx=0x7fcf680f4530 comp rx=0 tx=0).stop 2024-09-10T08:24:08.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.840+0000 7fcf7b000640 1 -- 172.21.15.2:0/1957036880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf741019c0 msgr2=0x7fcf7419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:08.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.840+0000 7fcf7b000640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf741019c0 0x7fcf7419a490 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcf6802a860 tx=0x7fcf68033b60 comp rx=0 tx=0).stop 2024-09-10T08:24:08.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.840+0000 7fcf7b000640 1 -- 172.21.15.2:0/1957036880 shutdown_connections 2024-09-10T08:24:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.840+0000 7fcf7b000640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf5c0a2b80 0x7fcf5c0a5040 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.840+0000 7fcf7b000640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf741079c0 0x7fcf7419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.840+0000 7fcf7b000640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf74102360 0x7fcf7419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.840+0000 7fcf7b000640 1 --2- 172.21.15.2:0/1957036880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf741019c0 0x7fcf7419a490 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.840+0000 7fcf7b000640 1 -- 172.21.15.2:0/1957036880 >> 172.21.15.2:0/1957036880 conn(0x7fcf740fd740 msgr2=0x7fcf740fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.840+0000 7fcf7b000640 1 -- 172.21.15.2:0/1957036880 shutdown_connections 2024-09-10T08:24:08.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:08.840+0000 7fcf7b000640 1 -- 172.21.15.2:0/1957036880 wait complete. 2024-09-10T08:24:09.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:09 smithi002 ceph-mon[30563]: pgmap v867: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.5 KiB/s wr, 10 op/s 2024-09-10T08:24:09.856 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:24:09.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:09 smithi155 ceph-mon[35405]: pgmap v867: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.5 KiB/s wr, 10 op/s 2024-09-10T08:24:09.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:09 smithi107 ceph-mon[35048]: pgmap v867: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.5 KiB/s wr, 10 op/s 2024-09-10T08:24:10.237 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:10.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.842+0000 7f5023800640 1 -- 172.21.15.2:0/1197023983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f501c109120 msgr2=0x7f501c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:10.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.842+0000 7f5023800640 1 --2- 172.21.15.2:0/1197023983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f501c109120 0x7f501c110ca0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5014009900 tx=0x7f501402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.842+0000 7f5023800640 1 -- 172.21.15.2:0/1197023983 shutdown_connections 2024-09-10T08:24:10.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.842+0000 7f5023800640 1 --2- 172.21.15.2:0/1197023983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f501c109120 0x7f501c110ca0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.842+0000 7f5023800640 1 --2- 172.21.15.2:0/1197023983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f501c1049a0 0x7f501c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.842+0000 7f5023800640 1 --2- 172.21.15.2:0/1197023983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f501c103ff0 0x7f501c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.842+0000 7f5023800640 1 -- 172.21.15.2:0/1197023983 >> 172.21.15.2:0/1197023983 conn(0x7f501c0ffa90 msgr2=0x7f501c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:10.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.842+0000 7f5023800640 1 -- 172.21.15.2:0/1197023983 shutdown_connections 2024-09-10T08:24:10.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.842+0000 7f5023800640 1 -- 172.21.15.2:0/1197023983 wait complete. 2024-09-10T08:24:10.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.843+0000 7f5023800640 1 Processor -- start 2024-09-10T08:24:10.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.843+0000 7f5023800640 1 -- start start 2024-09-10T08:24:10.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5023800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f501c103ff0 0x7f501c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5023800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f501c1049a0 0x7f501c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5023800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f501c109120 0x7f501c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5023800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f501c114150 con 0x7f501c109120 2024-09-10T08:24:10.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5023800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f501c113fd0 con 0x7f501c1049a0 2024-09-10T08:24:10.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5023800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f501c1142d0 con 0x7f501c103ff0 2024-09-10T08:24:10.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5021800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f501c109120 0x7f501c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5021800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f501c109120 0x7f501c19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47276/0 (socket says 172.21.15.2:47276) 2024-09-10T08:24:10.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5021800640 1 -- 172.21.15.2:0/3745461188 learned_addr learned my addr 172.21.15.2:0/3745461188 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:10.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5020e00640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f501c103ff0 0x7f501c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5013e00640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f501c1049a0 0x7f501c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5020e00640 1 -- 172.21.15.2:0/3745461188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f501c1049a0 msgr2=0x7f501c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5020e00640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f501c1049a0 0x7f501c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5020e00640 1 -- 172.21.15.2:0/3745461188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f501c109120 msgr2=0x7f501c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5020e00640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f501c109120 0x7f501c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.844+0000 7f5020e00640 1 -- 172.21.15.2:0/3745461188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c19f560 con 0x7f501c103ff0 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.845+0000 7f5020e00640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f501c103ff0 0x7f501c19a5b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f500c009fd0 tx=0x7f500c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.845+0000 7f501a000640 1 -- 172.21.15.2:0/3745461188 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c004b20 con 0x7f501c103ff0 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.845+0000 7f501a000640 1 -- 172.21.15.2:0/3745461188 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500c004cc0 con 0x7f501c103ff0 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.845+0000 7f5023800640 1 -- 172.21.15.2:0/3745461188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501c19f850 con 0x7f501c103ff0 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.845+0000 7f501a000640 1 -- 172.21.15.2:0/3745461188 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c0107b0 con 0x7f501c103ff0 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.845+0000 7f5023800640 1 -- 172.21.15.2:0/3745461188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501c1a7130 con 0x7f501c103ff0 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.848+0000 7f50263d8740 1 Processor -- start 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.848+0000 7f50263d8740 1 -- start start 2024-09-10T08:24:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.848+0000 7f501a000640 1 -- 172.21.15.2:0/3745461188 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f500c005020 con 0x7f501c103ff0 2024-09-10T08:24:10.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.848+0000 7f50263d8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e4c70 0x560eff9e5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f50263d8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9e5590 0x560eff9ef910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f50263d8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9efe50 0x560eff9f2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f50263d8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560eff94b3e0 con 0x560eff9efe50 2024-09-10T08:24:10.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f50263d8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560eff8f2d80 con 0x560eff9e4c70 2024-09-10T08:24:10.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f50263d8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560eff9473a0 con 0x560eff9e5590 2024-09-10T08:24:10.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f5021800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e4c70 0x560eff9e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f5013e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9efe50 0x560eff9f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f5021800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e4c70 0x560eff9e5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39444/0 (socket says 172.21.15.2:39444) 2024-09-10T08:24:10.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f5021800640 1 -- 172.21.15.2:0/211173302 learned_addr learned my addr 172.21.15.2:0/211173302 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:10.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f5020e00640 1 --2- 172.21.15.2:0/211173302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9e5590 0x560eff9ef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f5013e00640 1 -- 172.21.15.2:0/211173302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9e5590 msgr2=0x560eff9ef910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f5013e00640 1 --2- 172.21.15.2:0/211173302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9e5590 0x560eff9ef910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f5013e00640 1 -- 172.21.15.2:0/211173302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e4c70 msgr2=0x560eff9e5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f5013e00640 1 --2- 172.21.15.2:0/211173302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e4c70 0x560eff9e5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.849+0000 7f5013e00640 1 -- 172.21.15.2:0/211173302 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eff911ae0 con 0x560eff9efe50 2024-09-10T08:24:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.850+0000 7f5013e00640 1 --2- 172.21.15.2:0/211173302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9efe50 0x560eff9f2240 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5004009bf0 tx=0x7f5004011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48cd3e84129ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.850+0000 7f5012000640 1 -- 172.21.15.2:0/211173302 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5004004050 con 0x560eff9efe50 2024-09-10T08:24:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.850+0000 7f5012000640 1 -- 172.21.15.2:0/211173302 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5004005500 con 0x560eff9efe50 2024-09-10T08:24:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.850+0000 7f5012000640 1 -- 172.21.15.2:0/211173302 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5004004d70 con 0x560eff9efe50 2024-09-10T08:24:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.850+0000 7f501a000640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f50000a2b80 0x7f50000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.851+0000 7f501a000640 1 -- 172.21.15.2:0/3745461188 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f500c0cfc10 con 0x7f501c103ff0 2024-09-10T08:24:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.851+0000 7f50263d8740 1 -- 172.21.15.2:0/211173302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9efe50 msgr2=0x560eff9f2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.851+0000 7f50263d8740 1 --2- 172.21.15.2:0/211173302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9efe50 0x560eff9f2240 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5004009bf0 tx=0x7f5004011810 comp rx=0 tx=0).stop 2024-09-10T08:24:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.851+0000 7f5021800640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f50000a2b80 0x7f50000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.851+0000 7f50263d8740 1 -- 172.21.15.2:0/211173302 shutdown_connections 2024-09-10T08:24:10.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.851+0000 7f50263d8740 1 --2- 172.21.15.2:0/211173302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9efe50 0x560eff9f2240 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.851+0000 7f50263d8740 1 --2- 172.21.15.2:0/211173302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9e5590 0x560eff9ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.851+0000 7f50263d8740 1 --2- 172.21.15.2:0/211173302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e4c70 0x560eff9e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.851+0000 7f50263d8740 1 -- 172.21.15.2:0/211173302 >> 172.21.15.2:0/211173302 conn(0x560eff9e1200 msgr2=0x560eff9e15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:10.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.851+0000 7f50263d8740 1 -- 172.21.15.2:0/211173302 shutdown_connections 2024-09-10T08:24:10.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.851+0000 7f50263d8740 1 -- 172.21.15.2:0/211173302 wait complete. 2024-09-10T08:24:10.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.852+0000 7f5021800640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f50000a2b80 0x7f50000a5040 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f50140096c0 tx=0x7f5014005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:10.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.852+0000 7f50263d8740 1 -- start start 2024-09-10T08:24:10.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.852+0000 7f50263d8740 1 Processor -- start 2024-09-10T08:24:10.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.852+0000 7f50263d8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9e4c70 0x560eff9ecc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.852+0000 7f50263d8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 0x560eff9ed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.852+0000 7f50263d8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9efe50 0x560eff9ed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.852+0000 7f50263d8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560eff9f2a70 con 0x560eff9e4c70 2024-09-10T08:24:10.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.852+0000 7f50263d8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560eff9060f0 con 0x560eff9e5590 2024-09-10T08:24:10.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.852+0000 7f50263d8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560eff9f2bf0 con 0x560eff9efe50 2024-09-10T08:24:10.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.853+0000 7f50263d8740 10 client.0 _ll_register_callbacks cb 0x560eff8edbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:10.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.853+0000 7f5020e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9e4c70 0x560eff9ecc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.853+0000 7f5013e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 0x560eff9ed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.853+0000 7f5013e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 0x560eff9ed1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39446/0 (socket says 172.21.15.2:39446) 2024-09-10T08:24:10.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.853+0000 7f5013e00640 1 -- 172.21.15.2:0/779680708 learned_addr learned my addr 172.21.15.2:0/779680708 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:10.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.853+0000 7f5020e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9e4c70 0x560eff9ecc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47298/0 (socket says 172.21.15.2:47298) 2024-09-10T08:24:10.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.853+0000 7f5021800640 1 --2- 172.21.15.2:0/779680708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9efe50 0x560eff9ed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.853+0000 7f5013e00640 1 -- 172.21.15.2:0/779680708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9efe50 msgr2=0x560eff9ed6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:10.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.853+0000 7f5013e00640 1 --2- 172.21.15.2:0/779680708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9efe50 0x560eff9ed6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.853+0000 7f5013e00640 1 -- 172.21.15.2:0/779680708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9e4c70 msgr2=0x560eff9ecc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:10.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.853+0000 7f5013e00640 1 --2- 172.21.15.2:0/779680708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9e4c70 0x560eff9ecc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.853+0000 7f5013e00640 1 -- 172.21.15.2:0/779680708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eff6bc3c0 con 0x560eff9e5590 2024-09-10T08:24:10.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.853+0000 7f5021800640 1 --2- 172.21.15.2:0/779680708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9efe50 0x560eff9ed6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:10.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.854+0000 7f5013e00640 1 --2- 172.21.15.2:0/779680708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 0x560eff9ed1a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5004005770 tx=0x7f5004009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459a86ceb1a7f12f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:10.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.854+0000 7f500aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:10.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.854+0000 7f500aa00640 1 -- 172.21.15.2:0/779680708 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5004029070 con 0x560eff9e5590 2024-09-10T08:24:10.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.854+0000 7f50263d8740 1 -- 172.21.15.2:0/779680708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5004005500 con 0x560eff9e5590 2024-09-10T08:24:10.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.854+0000 7f500aa00640 1 -- 172.21.15.2:0/779680708 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5004004d50 con 0x560eff9e5590 2024-09-10T08:24:10.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.854+0000 7f500aa00640 1 -- 172.21.15.2:0/779680708 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5004035050 con 0x560eff9e5590 2024-09-10T08:24:10.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.854+0000 7f50263d8740 10 client.30011 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:10.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.854+0000 7f50263d8740 1 -- 172.21.15.2:0/779680708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560eff9fea00 con 0x560eff9e5590 2024-09-10T08:24:10.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.854+0000 7f500aa00640 1 -- 172.21.15.2:0/779680708 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5004005500 con 0x560eff9e5590 2024-09-10T08:24:10.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.855+0000 7f500aa00640 1 -- 172.21.15.2:0/779680708 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5004003c90 con 0x560eff9e5590 2024-09-10T08:24:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.855+0000 7f50263d8740 10 client.30011 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.855+0000 7f50263d8740 10 client.30011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:24:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.855+0000 7f50263d8740 20 client.30011 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.855+0000 7f50263d8740 1 --2- 172.21.15.2:0/779680708 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560eff9ffd50 0x560effa02140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.855+0000 7f50263d8740 4 client.30011 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.855+0000 7f50263d8740 1 -- 172.21.15.2:0/779680708 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eff958950 con 0x560eff9ffd50 2024-09-10T08:24:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.855+0000 7f5020e00640 1 --2- 172.21.15.2:0/779680708 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560eff9ffd50 0x560effa02140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.856+0000 7f5020e00640 1 --2- 172.21.15.2:0/779680708 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560eff9ffd50 0x560effa02140 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fed1064f1eaa58ff server_cookie=d4eec1b5ba278b84 in_seq=0 out_seq=0 2024-09-10T08:24:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.856+0000 7f500aa00640 10 client.30011 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:24:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.856+0000 7f500aa00640 1 -- 172.21.15.2:0/779680708 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560eff958950 con 0x560eff9ffd50 2024-09-10T08:24:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.856+0000 7f500aa00640 10 client.30011 handle_command_reply: tid=0 2024-09-10T08:24:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.857+0000 7f50263d8740 1 client.30011 shutdown 2024-09-10T08:24:10.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.857+0000 7f50263d8740 1 -- 172.21.15.2:0/779680708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 msgr2=0x560eff9ed1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:10.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.857+0000 7f50263d8740 1 --2- 172.21.15.2:0/779680708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 0x560eff9ed1a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5004005770 tx=0x7f5004009fd0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.857+0000 7f50263d8740 1 -- 172.21.15.2:0/779680708 shutdown_connections 2024-09-10T08:24:10.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.857+0000 7f50263d8740 1 --2- 172.21.15.2:0/779680708 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560eff9ffd50 0x560effa02140 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.857+0000 7f50263d8740 1 --2- 172.21.15.2:0/779680708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9efe50 0x560eff9ed6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.857+0000 7f50263d8740 1 --2- 172.21.15.2:0/779680708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 0x560eff9ed1a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.857+0000 7f50263d8740 1 --2- 172.21.15.2:0/779680708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9e4c70 0x560eff9ecc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.857+0000 7f500aa00640 0 client.30011 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:24:10.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.857+0000 7f50263d8740 1 -- 172.21.15.2:0/779680708 >> 172.21.15.2:0/779680708 conn(0x560eff9e1200 msgr2=0x560eff9e0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:10.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.857+0000 7f50263d8740 1 -- 172.21.15.2:0/779680708 shutdown_connections 2024-09-10T08:24:10.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.857+0000 7f50263d8740 1 -- 172.21.15.2:0/779680708 wait complete. 2024-09-10T08:24:10.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.857+0000 7f50263d8740 20 client.30011 trim_cache size 0 max 16384 2024-09-10T08:24:10.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.871+0000 7f50263d8740 1 Processor -- start 2024-09-10T08:24:10.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.871+0000 7f50263d8740 1 -- start start 2024-09-10T08:24:10.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.871+0000 7f50263d8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e4c70 0x560eff9ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.871+0000 7f50263d8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9e5590 0x560effa07920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.871+0000 7f50263d8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9efe50 0x560effa07e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.871+0000 7f50263d8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f501c1b38c0 con 0x560eff9efe50 2024-09-10T08:24:10.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.871+0000 7f50263d8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f501c1b1190 con 0x560eff9e4c70 2024-09-10T08:24:10.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.871+0000 7f50263d8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f501c1aefe0 con 0x560eff9e5590 2024-09-10T08:24:10.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.872+0000 7f5021800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9efe50 0x560effa07e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.872+0000 7f5013e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9e5590 0x560effa07920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.872+0000 7f5021800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9efe50 0x560effa07e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47304/0 (socket says 172.21.15.2:47304) 2024-09-10T08:24:10.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.872+0000 7f5021800640 1 -- 172.21.15.2:0/2290035714 learned_addr learned my addr 172.21.15.2:0/2290035714 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:10.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.872+0000 7f5020e00640 1 --2- 172.21.15.2:0/2290035714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e4c70 0x560eff9ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.872+0000 7f5020e00640 1 -- 172.21.15.2:0/2290035714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9e5590 msgr2=0x560effa07920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:10.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.872+0000 7f5020e00640 1 --2- 172.21.15.2:0/2290035714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9e5590 0x560effa07920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.872+0000 7f5020e00640 1 -- 172.21.15.2:0/2290035714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9efe50 msgr2=0x560effa07e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:10.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.872+0000 7f5020e00640 1 --2- 172.21.15.2:0/2290035714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9efe50 0x560effa07e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.872+0000 7f5020e00640 1 -- 172.21.15.2:0/2290035714 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eff9ef370 con 0x560eff9e4c70 2024-09-10T08:24:10.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.872+0000 7f5020e00640 1 --2- 172.21.15.2:0/2290035714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e4c70 0x560eff9ee4c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x560eff9e2060 tx=0x7f500c0cd190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a2ab6d56c0f715 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:10.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.873+0000 7f5011600640 1 -- 172.21.15.2:0/2290035714 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c09bb60 con 0x560eff9e4c70 2024-09-10T08:24:10.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.873+0000 7f5011600640 1 -- 172.21.15.2:0/2290035714 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500c0d4040 con 0x560eff9e4c70 2024-09-10T08:24:10.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.873+0000 7f5011600640 1 -- 172.21.15.2:0/2290035714 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c09b980 con 0x560eff9e4c70 2024-09-10T08:24:10.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.873+0000 7f50263d8740 1 -- 172.21.15.2:0/2290035714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e4c70 msgr2=0x560eff9ee4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:10.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.873+0000 7f50263d8740 1 --2- 172.21.15.2:0/2290035714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e4c70 0x560eff9ee4c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x560eff9e2060 tx=0x7f500c0cd190 comp rx=0 tx=0).stop 2024-09-10T08:24:10.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.873+0000 7f50263d8740 1 -- 172.21.15.2:0/2290035714 shutdown_connections 2024-09-10T08:24:10.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.873+0000 7f50263d8740 1 --2- 172.21.15.2:0/2290035714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9efe50 0x560effa07e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.873+0000 7f50263d8740 1 --2- 172.21.15.2:0/2290035714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9e5590 0x560effa07920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.873+0000 7f50263d8740 1 --2- 172.21.15.2:0/2290035714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e4c70 0x560eff9ee4c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.873+0000 7f50263d8740 1 -- 172.21.15.2:0/2290035714 >> 172.21.15.2:0/2290035714 conn(0x560eff9e1200 msgr2=0x560eff9e0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:10.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.873+0000 7f50263d8740 1 -- 172.21.15.2:0/2290035714 shutdown_connections 2024-09-10T08:24:10.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.873+0000 7f50263d8740 1 -- 172.21.15.2:0/2290035714 wait complete. 2024-09-10T08:24:10.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.874+0000 7f50263d8740 1 -- start start 2024-09-10T08:24:10.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.874+0000 7f50263d8740 1 Processor -- start 2024-09-10T08:24:10.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.874+0000 7f50263d8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9e4c70 0x560eff9f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.874+0000 7f50263d8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 0x560eff9f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.874+0000 7f50263d8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9efe50 0x560eff9f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.874+0000 7f50263d8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f501c1af650 con 0x560eff9e4c70 2024-09-10T08:24:10.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.874+0000 7f50263d8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f501c1b0ac0 con 0x560eff9e5590 2024-09-10T08:24:10.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.874+0000 7f50263d8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f501c1afd20 con 0x560eff9efe50 2024-09-10T08:24:10.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.874+0000 7f50263d8740 10 client.0 _ll_register_callbacks cb 0x7f500c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:10.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.875+0000 7f5020e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9e4c70 0x560eff9f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.875+0000 7f5013e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 0x560eff9f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.875+0000 7f5013e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 0x560eff9f4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39468/0 (socket says 172.21.15.2:39468) 2024-09-10T08:24:10.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.875+0000 7f5013e00640 1 -- 172.21.15.2:0/4255176647 learned_addr learned my addr 172.21.15.2:0/4255176647 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:10.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.875+0000 7f5021800640 1 --2- 172.21.15.2:0/4255176647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9efe50 0x560eff9f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.875+0000 7f5013e00640 1 -- 172.21.15.2:0/4255176647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9efe50 msgr2=0x560eff9f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:10.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.875+0000 7f5013e00640 1 --2- 172.21.15.2:0/4255176647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9efe50 0x560eff9f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.875+0000 7f5013e00640 1 -- 172.21.15.2:0/4255176647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9e4c70 msgr2=0x560eff9f3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:10.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.875+0000 7f5013e00640 1 --2- 172.21.15.2:0/4255176647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9e4c70 0x560eff9f3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.875+0000 7f5013e00640 1 -- 172.21.15.2:0/4255176647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560effa698f0 con 0x560eff9e5590 2024-09-10T08:24:10.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.875+0000 7f5021800640 1 --2- 172.21.15.2:0/4255176647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9efe50 0x560eff9f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:10.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.875+0000 7f5013e00640 1 --2- 172.21.15.2:0/4255176647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 0x560eff9f4090 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5004006010 tx=0x7f50040043e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48631fb2a5e0d1f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:10.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.875+0000 7f500aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:10.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.876+0000 7f500aa00640 1 -- 172.21.15.2:0/4255176647 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500403e060 con 0x560eff9e5590 2024-09-10T08:24:10.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.876+0000 7f50263d8740 1 -- 172.21.15.2:0/4255176647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f500c0d4040 con 0x560eff9e5590 2024-09-10T08:24:10.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.876+0000 7f500aa00640 1 -- 172.21.15.2:0/4255176647 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f5004023b80 con 0x560eff9e5590 2024-09-10T08:24:10.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.876+0000 7f500aa00640 1 -- 172.21.15.2:0/4255176647 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5004045050 con 0x560eff9e5590 2024-09-10T08:24:10.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.876+0000 7f50263d8740 10 client.30017 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:10.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.876+0000 7f50263d8740 1 -- 172.21.15.2:0/4255176647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560effa6ab50 con 0x560eff9e5590 2024-09-10T08:24:10.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.876+0000 7f500aa00640 1 -- 172.21.15.2:0/4255176647 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500c0d4040 con 0x560eff9e5590 2024-09-10T08:24:10.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.876+0000 7f500aa00640 1 -- 172.21.15.2:0/4255176647 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f500402e4d0 con 0x560eff9e5590 2024-09-10T08:24:10.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.877+0000 7f50263d8740 10 client.30017 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:10.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.877+0000 7f50263d8740 10 client.30017 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:24:10.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.877+0000 7f50263d8740 20 client.30017 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:10.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.877+0000 7f50263d8740 1 --2- 172.21.15.2:0/4255176647 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560effa6b1e0 0x560effa6d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:10.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.877+0000 7f50263d8740 4 client.30017 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:24:10.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.877+0000 7f50263d8740 1 -- 172.21.15.2:0/4255176647 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560eff952e60 con 0x560effa6b1e0 2024-09-10T08:24:10.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.877+0000 7f5020e00640 1 --2- 172.21.15.2:0/4255176647 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560effa6b1e0 0x560effa6d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:10.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.877+0000 7f5020e00640 1 --2- 172.21.15.2:0/4255176647 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560effa6b1e0 0x560effa6d600 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d9d8c510e5a5f38 server_cookie=261d79b219cbedd9 in_seq=0 out_seq=0 2024-09-10T08:24:10.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.877+0000 7f500aa00640 10 client.30017 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:24:10.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f500aa00640 1 -- 172.21.15.2:0/4255176647 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560eff952e60 con 0x560effa6b1e0 2024-09-10T08:24:10.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f500aa00640 10 client.30017 handle_command_reply: tid=0 2024-09-10T08:24:10.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f50263d8740 1 client.30017 shutdown 2024-09-10T08:24:10.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f50263d8740 1 -- 172.21.15.2:0/4255176647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 msgr2=0x560eff9f4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:10.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f50263d8740 1 --2- 172.21.15.2:0/4255176647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 0x560eff9f4090 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5004006010 tx=0x7f50040043e0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f50263d8740 1 -- 172.21.15.2:0/4255176647 shutdown_connections 2024-09-10T08:24:10.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f50263d8740 1 --2- 172.21.15.2:0/4255176647 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560effa6b1e0 0x560effa6d600 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f50263d8740 1 --2- 172.21.15.2:0/4255176647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560eff9efe50 0x560eff9f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f50263d8740 1 --2- 172.21.15.2:0/4255176647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560eff9e5590 0x560eff9f4090 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f50263d8740 1 --2- 172.21.15.2:0/4255176647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560eff9e4c70 0x560eff9f3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f500aa00640 0 client.30017 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:24:10.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f50263d8740 1 -- 172.21.15.2:0/4255176647 >> 172.21.15.2:0/4255176647 conn(0x560eff9e1200 msgr2=0x560eff9dfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:10.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f50263d8740 1 -- 172.21.15.2:0/4255176647 shutdown_connections 2024-09-10T08:24:10.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f50263d8740 1 -- 172.21.15.2:0/4255176647 wait complete. 2024-09-10T08:24:10.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.878+0000 7f50263d8740 20 client.30017 trim_cache size 0 max 16384 2024-09-10T08:24:10.889 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:24:10.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.879+0000 7f5023800640 1 -- 172.21.15.2:0/3745461188 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f50000a2b80 msgr2=0x7f50000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:10.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.879+0000 7f5023800640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f50000a2b80 0x7f50000a5040 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f50140096c0 tx=0x7f5014005a20 comp rx=0 tx=0).stop 2024-09-10T08:24:10.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.879+0000 7f5023800640 1 -- 172.21.15.2:0/3745461188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f501c103ff0 msgr2=0x7f501c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:10.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.879+0000 7f5023800640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f501c103ff0 0x7f501c19a5b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f500c009fd0 tx=0x7f500c00ef90 comp rx=0 tx=0).stop 2024-09-10T08:24:10.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.879+0000 7f5023800640 1 -- 172.21.15.2:0/3745461188 shutdown_connections 2024-09-10T08:24:10.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.879+0000 7f5023800640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f501c109120 0x7f501c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.879+0000 7f5023800640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f501c1049a0 0x7f501c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.879+0000 7f5023800640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f50000a2b80 0x7f50000a5040 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.879+0000 7f5023800640 1 --2- 172.21.15.2:0/3745461188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f501c103ff0 0x7f501c19a5b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:10.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.879+0000 7f5023800640 1 -- 172.21.15.2:0/3745461188 >> 172.21.15.2:0/3745461188 conn(0x7f501c0ffa90 msgr2=0x7f501c101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:10.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.879+0000 7f5023800640 1 -- 172.21.15.2:0/3745461188 shutdown_connections 2024-09-10T08:24:10.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:10.879+0000 7f5023800640 1 -- 172.21.15.2:0/3745461188 wait complete. 2024-09-10T08:24:11.836 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:24:11.886 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:11 smithi002 ceph-mon[30563]: pgmap v868: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-10T08:24:11.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:11 smithi155 ceph-mon[35405]: pgmap v868: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-10T08:24:11.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:11 smithi107 ceph-mon[35048]: pgmap v868: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-10T08:24:12.215 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:12.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.794+0000 7f7b95600640 1 -- 172.21.15.2:0/614009617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b90109120 msgr2=0x7f7b90110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:12.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.794+0000 7f7b95600640 1 --2- 172.21.15.2:0/614009617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b90109120 0x7f7b90110ca0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7b88009ca0 tx=0x7f7b8802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.794+0000 7f7b95600640 1 -- 172.21.15.2:0/614009617 shutdown_connections 2024-09-10T08:24:12.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.794+0000 7f7b95600640 1 --2- 172.21.15.2:0/614009617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b90109120 0x7f7b90110ca0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.794+0000 7f7b95600640 1 --2- 172.21.15.2:0/614009617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b901049a0 0x7f7b901089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.794+0000 7f7b95600640 1 --2- 172.21.15.2:0/614009617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b90103ff0 0x7f7b901043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.794+0000 7f7b95600640 1 -- 172.21.15.2:0/614009617 >> 172.21.15.2:0/614009617 conn(0x7f7b900ffa90 msgr2=0x7f7b90101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:12.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.794+0000 7f7b95600640 1 -- 172.21.15.2:0/614009617 shutdown_connections 2024-09-10T08:24:12.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.794+0000 7f7b95600640 1 -- 172.21.15.2:0/614009617 wait complete. 2024-09-10T08:24:12.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.795+0000 7f7b95600640 1 Processor -- start 2024-09-10T08:24:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.795+0000 7f7b95600640 1 -- start start 2024-09-10T08:24:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b95600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b90103ff0 0x7f7b9019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b95600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b901049a0 0x7f7b9019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b95600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b90109120 0x7f7b9019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b95600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90114130 con 0x7f7b901049a0 2024-09-10T08:24:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b95600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90113fb0 con 0x7f7b90103ff0 2024-09-10T08:24:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b95600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b901142b0 con 0x7f7b90109120 2024-09-10T08:24:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b8e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b901049a0 0x7f7b9019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b8e000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b901049a0 0x7f7b9019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47324/0 (socket says 172.21.15.2:47324) 2024-09-10T08:24:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b8e000640 1 -- 172.21.15.2:0/1468204618 learned_addr learned my addr 172.21.15.2:0/1468204618 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b8ea00640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b90103ff0 0x7f7b9019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b8f400640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b90109120 0x7f7b9019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b8e000640 1 -- 172.21.15.2:0/1468204618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b90109120 msgr2=0x7f7b9019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b8e000640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b90109120 0x7f7b9019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b8e000640 1 -- 172.21.15.2:0/1468204618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b90103ff0 msgr2=0x7f7b9019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b8e000640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b90103ff0 0x7f7b9019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.796+0000 7f7b8e000640 1 -- 172.21.15.2:0/1468204618 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9019f540 con 0x7f7b901049a0 2024-09-10T08:24:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.797+0000 7f7b8e000640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b901049a0 0x7f7b9019aad0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00d950 tx=0x7f7b7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.797+0000 7f7b87400640 1 -- 172.21.15.2:0/1468204618 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c017030 con 0x7f7b901049a0 2024-09-10T08:24:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.797+0000 7f7b87400640 1 -- 172.21.15.2:0/1468204618 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b7c004020 con 0x7f7b901049a0 2024-09-10T08:24:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.797+0000 7f7b95600640 1 -- 172.21.15.2:0/1468204618 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9019f830 con 0x7f7b901049a0 2024-09-10T08:24:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.797+0000 7f7b87400640 1 -- 172.21.15.2:0/1468204618 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0047a0 con 0x7f7b901049a0 2024-09-10T08:24:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.797+0000 7f7b95600640 1 -- 172.21.15.2:0/1468204618 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b901a7110 con 0x7f7b901049a0 2024-09-10T08:24:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.800+0000 7f7b98023740 1 Processor -- start 2024-09-10T08:24:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.800+0000 7f7b98023740 1 -- start start 2024-09-10T08:24:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.800+0000 7f7b87400640 1 -- 172.21.15.2:0/1468204618 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7b7c0041d0 con 0x7f7b901049a0 2024-09-10T08:24:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b98023740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab7f6c70 0x55e4ab7f7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b98023740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f7590 0x55e4ab801910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b98023740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab801e50 0x55e4ab804240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b98023740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e4ab75d3e0 con 0x55e4ab7f7590 2024-09-10T08:24:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b98023740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e4ab704d80 con 0x55e4ab801e50 2024-09-10T08:24:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b98023740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e4ab7593a0 con 0x55e4ab7f6c70 2024-09-10T08:24:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b8f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab7f6c70 0x55e4ab7f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f7590 0x55e4ab801910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b8f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab7f6c70 0x55e4ab7f7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:46504/0 (socket says 172.21.15.2:46504) 2024-09-10T08:24:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b8f400640 1 -- 172.21.15.2:0/1682808355 learned_addr learned my addr 172.21.15.2:0/1682808355 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b8e000640 1 --2- 172.21.15.2:0/1682808355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab801e50 0x55e4ab804240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b8f400640 1 -- 172.21.15.2:0/1682808355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab801e50 msgr2=0x55e4ab804240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b8f400640 1 --2- 172.21.15.2:0/1682808355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab801e50 0x55e4ab804240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b8f400640 1 -- 172.21.15.2:0/1682808355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f7590 msgr2=0x55e4ab801910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b8f400640 1 --2- 172.21.15.2:0/1682808355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f7590 0x55e4ab801910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b8f400640 1 -- 172.21.15.2:0/1682808355 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4ab723ae0 con 0x55e4ab7f6c70 2024-09-10T08:24:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.801+0000 7f7b8ea00640 1 --2- 172.21.15.2:0/1682808355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f7590 0x55e4ab801910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.802+0000 7f7b8f400640 1 --2- 172.21.15.2:0/1682808355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab7f6c70 0x55e4ab7f7050 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005cc0 tx=0x7f7b88005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f1db6b6131850f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.802+0000 7f7b7b400640 1 -- 172.21.15.2:0/1682808355 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88031db0 con 0x55e4ab7f6c70 2024-09-10T08:24:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.802+0000 7f7b7b400640 1 -- 172.21.15.2:0/1682808355 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b88036050 con 0x55e4ab7f6c70 2024-09-10T08:24:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.802+0000 7f7b7b400640 1 -- 172.21.15.2:0/1682808355 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b880042a0 con 0x55e4ab7f6c70 2024-09-10T08:24:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.802+0000 7f7b87400640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7b740a2b80 0x7f7b740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.803+0000 7f7b87400640 1 -- 172.21.15.2:0/1468204618 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7b7c0cee30 con 0x7f7b901049a0 2024-09-10T08:24:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.803+0000 7f7b8ea00640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7b740a2b80 0x7f7b740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.803+0000 7f7b98023740 1 -- 172.21.15.2:0/1682808355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab7f6c70 msgr2=0x55e4ab7f7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.803+0000 7f7b98023740 1 --2- 172.21.15.2:0/1682808355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab7f6c70 0x55e4ab7f7050 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005cc0 tx=0x7f7b88005ef0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.803+0000 7f7b98023740 1 -- 172.21.15.2:0/1682808355 shutdown_connections 2024-09-10T08:24:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.803+0000 7f7b98023740 1 --2- 172.21.15.2:0/1682808355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab801e50 0x55e4ab804240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.803+0000 7f7b98023740 1 --2- 172.21.15.2:0/1682808355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f7590 0x55e4ab801910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.803+0000 7f7b98023740 1 --2- 172.21.15.2:0/1682808355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab7f6c70 0x55e4ab7f7050 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.803+0000 7f7b98023740 1 -- 172.21.15.2:0/1682808355 >> 172.21.15.2:0/1682808355 conn(0x55e4ab7f3200 msgr2=0x55e4ab7f35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:12.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.803+0000 7f7b98023740 1 -- 172.21.15.2:0/1682808355 shutdown_connections 2024-09-10T08:24:12.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.803+0000 7f7b98023740 1 -- 172.21.15.2:0/1682808355 wait complete. 2024-09-10T08:24:12.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.803+0000 7f7b8ea00640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7b740a2b80 0x7f7b740a5040 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55e4ab7f4060 tx=0x7f7b80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:12.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.804+0000 7f7b98023740 1 -- start start 2024-09-10T08:24:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.804+0000 7f7b98023740 1 Processor -- start 2024-09-10T08:24:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.805+0000 7f7b98023740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f6c70 0x55e4ab7fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.805+0000 7f7b98023740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab7f7590 0x55e4ab7ff210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.805+0000 7f7b98023740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab801e50 0x55e4ab7ff750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.805+0000 7f7b98023740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e4ab8048a0 con 0x55e4ab801e50 2024-09-10T08:24:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.805+0000 7f7b98023740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e4ab7180f0 con 0x55e4ab7f6c70 2024-09-10T08:24:12.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.805+0000 7f7b98023740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e4ab804a20 con 0x55e4ab7f7590 2024-09-10T08:24:12.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.805+0000 7f7b98023740 10 client.0 _ll_register_callbacks cb 0x55e4ab6ffbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:12.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.805+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f6c70 0x55e4ab7fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.805+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f6c70 0x55e4ab7fecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39516/0 (socket says 172.21.15.2:39516) 2024-09-10T08:24:12.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.805+0000 7f7b8ea00640 1 -- 172.21.15.2:0/2062894938 learned_addr learned my addr 172.21.15.2:0/2062894938 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:12.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.805+0000 7f7b8e000640 1 --2- 172.21.15.2:0/2062894938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab7f7590 0x55e4ab7ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.805+0000 7f7b8f400640 1 --2- 172.21.15.2:0/2062894938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab801e50 0x55e4ab7ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.806+0000 7f7b8ea00640 1 -- 172.21.15.2:0/2062894938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab7f7590 msgr2=0x55e4ab7ff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:12.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.806+0000 7f7b8ea00640 1 --2- 172.21.15.2:0/2062894938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab7f7590 0x55e4ab7ff210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.806+0000 7f7b8ea00640 1 -- 172.21.15.2:0/2062894938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab801e50 msgr2=0x55e4ab7ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:12.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.806+0000 7f7b8ea00640 1 --2- 172.21.15.2:0/2062894938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab801e50 0x55e4ab7ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.806+0000 7f7b8ea00640 1 -- 172.21.15.2:0/2062894938 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4ab4ce3c0 con 0x55e4ab7f6c70 2024-09-10T08:24:12.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.806+0000 7f7b8ea00640 1 --2- 172.21.15.2:0/2062894938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f6c70 0x55e4ab7fecd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7b800045f0 tx=0x7f7b80004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add935e1675eb733 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:12.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.806+0000 7f7b6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:12.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.807+0000 7f7b6fe00640 1 -- 172.21.15.2:0/2062894938 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80009be0 con 0x55e4ab7f6c70 2024-09-10T08:24:12.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.807+0000 7f7b98023740 1 -- 172.21.15.2:0/2062894938 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b88036050 con 0x55e4ab7f6c70 2024-09-10T08:24:12.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.807+0000 7f7b6fe00640 1 -- 172.21.15.2:0/2062894938 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7b8001f640 con 0x55e4ab7f6c70 2024-09-10T08:24:12.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.807+0000 7f7b6fe00640 1 -- 172.21.15.2:0/2062894938 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b8001f840 con 0x55e4ab7f6c70 2024-09-10T08:24:12.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.807+0000 7f7b98023740 10 client.30029 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:12.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.807+0000 7f7b98023740 1 -- 172.21.15.2:0/2062894938 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e4ab810920 con 0x55e4ab7f6c70 2024-09-10T08:24:12.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.807+0000 7f7b6fe00640 1 -- 172.21.15.2:0/2062894938 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b88036050 con 0x55e4ab7f6c70 2024-09-10T08:24:12.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.807+0000 7f7b6fe00640 1 -- 172.21.15.2:0/2062894938 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7b80027070 con 0x55e4ab7f6c70 2024-09-10T08:24:12.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.807+0000 7f7b98023740 10 client.30029 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:12.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.807+0000 7f7b98023740 10 client.30029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:12.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.808+0000 7f7b98023740 20 client.30029 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:12.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.808+0000 7f7b98023740 1 --2- 172.21.15.2:0/2062894938 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4ab811d30 0x55e4ab814140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.808+0000 7f7b98023740 4 client.30029 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:12.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.808+0000 7f7b98023740 1 -- 172.21.15.2:0/2062894938 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4ab76a950 con 0x55e4ab811d30 2024-09-10T08:24:12.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.808+0000 7f7b8e000640 1 --2- 172.21.15.2:0/2062894938 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4ab811d30 0x55e4ab814140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.808+0000 7f7b8e000640 1 --2- 172.21.15.2:0/2062894938 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4ab811d30 0x55e4ab814140 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b26294aa0294bb4f server_cookie=dd58d1836ee64e0c in_seq=0 out_seq=0 2024-09-10T08:24:12.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.808+0000 7f7b6fe00640 10 client.30029 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:12.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.809+0000 7f7b6fe00640 1 -- 172.21.15.2:0/2062894938 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4ab76a950 con 0x55e4ab811d30 2024-09-10T08:24:12.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.809+0000 7f7b6fe00640 10 client.30029 handle_command_reply: tid=0 2024-09-10T08:24:12.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.809+0000 7f7b98023740 1 client.30029 shutdown 2024-09-10T08:24:12.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.809+0000 7f7b98023740 1 -- 172.21.15.2:0/2062894938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f6c70 msgr2=0x55e4ab7fecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:12.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.809+0000 7f7b98023740 1 --2- 172.21.15.2:0/2062894938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f6c70 0x55e4ab7fecd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7b800045f0 tx=0x7f7b80004730 comp rx=0 tx=0).stop 2024-09-10T08:24:12.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.809+0000 7f7b98023740 1 -- 172.21.15.2:0/2062894938 shutdown_connections 2024-09-10T08:24:12.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.809+0000 7f7b98023740 1 --2- 172.21.15.2:0/2062894938 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4ab811d30 0x55e4ab814140 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.809+0000 7f7b98023740 1 --2- 172.21.15.2:0/2062894938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab801e50 0x55e4ab7ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.809+0000 7f7b98023740 1 --2- 172.21.15.2:0/2062894938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab7f7590 0x55e4ab7ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.809+0000 7f7b98023740 1 --2- 172.21.15.2:0/2062894938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f6c70 0x55e4ab7fecd0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.809+0000 7f7b6fe00640 0 client.30029 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:12.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.809+0000 7f7b98023740 1 -- 172.21.15.2:0/2062894938 >> 172.21.15.2:0/2062894938 conn(0x55e4ab7f3200 msgr2=0x55e4ab7f2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:12.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.810+0000 7f7b98023740 1 -- 172.21.15.2:0/2062894938 shutdown_connections 2024-09-10T08:24:12.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.810+0000 7f7b98023740 1 -- 172.21.15.2:0/2062894938 wait complete. 2024-09-10T08:24:12.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.810+0000 7f7b98023740 20 client.30029 trim_cache size 0 max 16384 2024-09-10T08:24:12.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.824+0000 7f7b98023740 1 Processor -- start 2024-09-10T08:24:12.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.824+0000 7f7b98023740 1 -- start start 2024-09-10T08:24:12.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.824+0000 7f7b98023740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f6c70 0x55e4ab819a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.824+0000 7f7b98023740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f7590 0x55e4ab819f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.824+0000 7f7b98023740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab801e50 0x55e4ab7fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.824+0000 7f7b98023740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7b901b38b0 con 0x55e4ab7f6c70 2024-09-10T08:24:12.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.824+0000 7f7b98023740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b901b1180 con 0x55e4ab7f7590 2024-09-10T08:24:12.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.824+0000 7f7b98023740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b901aefd0 con 0x55e4ab801e50 2024-09-10T08:24:12.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.824+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f6c70 0x55e4ab819a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.824+0000 7f7b8e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f7590 0x55e4ab819f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.824+0000 7f7b8e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f7590 0x55e4ab819f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39522/0 (socket says 172.21.15.2:39522) 2024-09-10T08:24:12.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.824+0000 7f7b8e000640 1 -- 172.21.15.2:0/1607093368 learned_addr learned my addr 172.21.15.2:0/1607093368 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:12.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.824+0000 7f7b8f400640 1 --2- 172.21.15.2:0/1607093368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab801e50 0x55e4ab7fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.825+0000 7f7b8e000640 1 -- 172.21.15.2:0/1607093368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab801e50 msgr2=0x55e4ab7fbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:12.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.825+0000 7f7b8e000640 1 --2- 172.21.15.2:0/1607093368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab801e50 0x55e4ab7fbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.825+0000 7f7b8e000640 1 -- 172.21.15.2:0/1607093368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f6c70 msgr2=0x55e4ab819a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:12.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.825+0000 7f7b8e000640 1 --2- 172.21.15.2:0/1607093368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f6c70 0x55e4ab819a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.825+0000 7f7b8e000640 1 -- 172.21.15.2:0/1607093368 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4ab7ee770 con 0x55e4ab7f7590 2024-09-10T08:24:12.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.825+0000 7f7b8f400640 1 --2- 172.21.15.2:0/1607093368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab801e50 0x55e4ab7fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:12.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.825+0000 7f7b8ea00640 1 --2- 172.21.15.2:0/1607093368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f6c70 0x55e4ab819a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:12.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.825+0000 7f7b8e000640 1 --2- 172.21.15.2:0/1607093368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f7590 0x55e4ab819f50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c097c10 tx=0x7f7b7c0cc3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac371f67fd2e184 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:12.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.825+0000 7f7b7aa00640 1 -- 172.21.15.2:0/1607093368 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c09b010 con 0x55e4ab7f7590 2024-09-10T08:24:12.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.825+0000 7f7b7aa00640 1 -- 172.21.15.2:0/1607093368 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b7c09b940 con 0x55e4ab7f7590 2024-09-10T08:24:12.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.825+0000 7f7b7aa00640 1 -- 172.21.15.2:0/1607093368 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c09aba0 con 0x55e4ab7f7590 2024-09-10T08:24:12.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.826+0000 7f7b98023740 1 -- 172.21.15.2:0/1607093368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f7590 msgr2=0x55e4ab819f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:12.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.826+0000 7f7b98023740 1 --2- 172.21.15.2:0/1607093368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f7590 0x55e4ab819f50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c097c10 tx=0x7f7b7c0cc3b0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.826+0000 7f7b98023740 1 -- 172.21.15.2:0/1607093368 shutdown_connections 2024-09-10T08:24:12.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.826+0000 7f7b98023740 1 --2- 172.21.15.2:0/1607093368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab801e50 0x55e4ab7fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.826+0000 7f7b98023740 1 --2- 172.21.15.2:0/1607093368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f7590 0x55e4ab819f50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.826+0000 7f7b98023740 1 --2- 172.21.15.2:0/1607093368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f6c70 0x55e4ab819a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.826+0000 7f7b98023740 1 -- 172.21.15.2:0/1607093368 >> 172.21.15.2:0/1607093368 conn(0x55e4ab7f3200 msgr2=0x55e4ab800d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:12.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.826+0000 7f7b98023740 1 -- 172.21.15.2:0/1607093368 shutdown_connections 2024-09-10T08:24:12.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.826+0000 7f7b98023740 1 -- 172.21.15.2:0/1607093368 wait complete. 2024-09-10T08:24:12.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.827+0000 7f7b98023740 1 -- start start 2024-09-10T08:24:12.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.827+0000 7f7b98023740 1 Processor -- start 2024-09-10T08:24:12.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.827+0000 7f7b98023740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f6c70 0x55e4ab805a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.827+0000 7f7b98023740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f7590 0x55e4ab805f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.827+0000 7f7b98023740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab801e50 0x55e4ab8064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.827+0000 7f7b98023740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7b901af640 con 0x55e4ab7f6c70 2024-09-10T08:24:12.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.827+0000 7f7b98023740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b901b0ab0 con 0x55e4ab7f7590 2024-09-10T08:24:12.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.827+0000 7f7b98023740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b901afd10 con 0x55e4ab801e50 2024-09-10T08:24:12.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.827+0000 7f7b98023740 10 client.0 _ll_register_callbacks cb 0x7f7b80009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:12.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f6c70 0x55e4ab805a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b8f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab801e50 0x55e4ab8064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b8f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab801e50 0x55e4ab8064c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:46522/0 (socket says 172.21.15.2:46522) 2024-09-10T08:24:12.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f6c70 0x55e4ab805a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47348/0 (socket says 172.21.15.2:47348) 2024-09-10T08:24:12.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b8f400640 1 -- 172.21.15.2:0/1698791145 learned_addr learned my addr 172.21.15.2:0/1698791145 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:12.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b8e000640 1 --2- 172.21.15.2:0/1698791145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f7590 0x55e4ab805f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b8f400640 1 -- 172.21.15.2:0/1698791145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f7590 msgr2=0x55e4ab805f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:12.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b8f400640 1 --2- 172.21.15.2:0/1698791145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f7590 0x55e4ab805f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b8f400640 1 -- 172.21.15.2:0/1698791145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f6c70 msgr2=0x55e4ab805a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:12.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b8f400640 1 --2- 172.21.15.2:0/1698791145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f6c70 0x55e4ab805a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b8f400640 1 -- 172.21.15.2:0/1698791145 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4ab87ba10 con 0x55e4ab801e50 2024-09-10T08:24:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b8ea00640 1 --2- 172.21.15.2:0/1698791145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f6c70 0x55e4ab805a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b8f400640 1 --2- 172.21.15.2:0/1698791145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab801e50 0x55e4ab8064c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7b88031960 tx=0x7f7b88031850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a7645cd8be9a8e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.828+0000 7f7b6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:24:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.829+0000 7f7b6fe00640 1 -- 172.21.15.2:0/1698791145 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88051060 con 0x55e4ab801e50 2024-09-10T08:24:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.829+0000 7f7b98023740 1 -- 172.21.15.2:0/1698791145 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b7c09b940 con 0x55e4ab801e50 2024-09-10T08:24:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.829+0000 7f7b6fe00640 1 -- 172.21.15.2:0/1698791145 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7b88003b80 con 0x55e4ab801e50 2024-09-10T08:24:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.829+0000 7f7b6fe00640 1 -- 172.21.15.2:0/1698791145 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b88003d80 con 0x55e4ab801e50 2024-09-10T08:24:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.829+0000 7f7b98023740 10 client.29965 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.829+0000 7f7b98023740 1 -- 172.21.15.2:0/1698791145 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e4ab87ccd0 con 0x55e4ab801e50 2024-09-10T08:24:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.830+0000 7f7b6fe00640 1 -- 172.21.15.2:0/1698791145 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b7c09b940 con 0x55e4ab801e50 2024-09-10T08:24:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.830+0000 7f7b6fe00640 1 -- 172.21.15.2:0/1698791145 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7b88042070 con 0x55e4ab801e50 2024-09-10T08:24:12.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.830+0000 7f7b98023740 10 client.29965 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:12.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.830+0000 7f7b98023740 10 client.29965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:12.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.830+0000 7f7b98023740 20 client.29965 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:12.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.830+0000 7f7b98023740 1 --2- 172.21.15.2:0/1698791145 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4ab87d170 0x55e4ab87f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:12.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.830+0000 7f7b98023740 4 client.29965 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:24:12.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.830+0000 7f7b98023740 1 -- 172.21.15.2:0/1698791145 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4ab764e60 con 0x55e4ab87d170 2024-09-10T08:24:12.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.830+0000 7f7b8ea00640 1 --2- 172.21.15.2:0/1698791145 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4ab87d170 0x55e4ab87f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:12.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b8ea00640 1 --2- 172.21.15.2:0/1698791145 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4ab87d170 0x55e4ab87f590 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98aef84faba71f49 server_cookie=6ce50fbb3144f4ff in_seq=0 out_seq=0 2024-09-10T08:24:12.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b6fe00640 10 client.29965 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:12.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b6fe00640 1 -- 172.21.15.2:0/1698791145 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4ab764e60 con 0x55e4ab87d170 2024-09-10T08:24:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b6fe00640 10 client.29965 handle_command_reply: tid=0 2024-09-10T08:24:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b98023740 1 client.29965 shutdown 2024-09-10T08:24:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b98023740 1 -- 172.21.15.2:0/1698791145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab801e50 msgr2=0x55e4ab8064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b98023740 1 --2- 172.21.15.2:0/1698791145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab801e50 0x55e4ab8064c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7b88031960 tx=0x7f7b88031850 comp rx=0 tx=0).stop 2024-09-10T08:24:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b98023740 1 -- 172.21.15.2:0/1698791145 shutdown_connections 2024-09-10T08:24:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b98023740 1 --2- 172.21.15.2:0/1698791145 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55e4ab87d170 0x55e4ab87f590 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b98023740 1 --2- 172.21.15.2:0/1698791145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4ab801e50 0x55e4ab8064c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b98023740 1 --2- 172.21.15.2:0/1698791145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4ab7f7590 0x55e4ab805f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b6fe00640 0 client.29965 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b98023740 1 --2- 172.21.15.2:0/1698791145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4ab7f6c70 0x55e4ab805a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b98023740 1 -- 172.21.15.2:0/1698791145 >> 172.21.15.2:0/1698791145 conn(0x55e4ab7f3200 msgr2=0x55e4ab800d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:12.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b98023740 1 -- 172.21.15.2:0/1698791145 shutdown_connections 2024-09-10T08:24:12.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.831+0000 7f7b98023740 1 -- 172.21.15.2:0/1698791145 wait complete. 2024-09-10T08:24:12.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.832+0000 7f7b98023740 20 client.29965 trim_cache size 0 max 16384 2024-09-10T08:24:12.847 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:24:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.832+0000 7f7b95600640 1 -- 172.21.15.2:0/1468204618 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7b740a2b80 msgr2=0x7f7b740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.832+0000 7f7b95600640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7b740a2b80 0x7f7b740a5040 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55e4ab7f4060 tx=0x7f7b80005f90 comp rx=0 tx=0).stop 2024-09-10T08:24:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.832+0000 7f7b95600640 1 -- 172.21.15.2:0/1468204618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b901049a0 msgr2=0x7f7b9019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.832+0000 7f7b95600640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b901049a0 0x7f7b9019aad0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00d950 tx=0x7f7b7c00de10 comp rx=0 tx=0).stop 2024-09-10T08:24:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.832+0000 7f7b95600640 1 -- 172.21.15.2:0/1468204618 shutdown_connections 2024-09-10T08:24:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.832+0000 7f7b95600640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7b740a2b80 0x7f7b740a5040 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.832+0000 7f7b95600640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b90109120 0x7f7b9019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.832+0000 7f7b95600640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b901049a0 0x7f7b9019aad0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.832+0000 7f7b95600640 1 --2- 172.21.15.2:0/1468204618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b90103ff0 0x7f7b9019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.832+0000 7f7b95600640 1 -- 172.21.15.2:0/1468204618 >> 172.21.15.2:0/1468204618 conn(0x7f7b900ffa90 msgr2=0x7f7b90101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.832+0000 7f7b95600640 1 -- 172.21.15.2:0/1468204618 shutdown_connections 2024-09-10T08:24:12.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:12.833+0000 7f7b95600640 1 -- 172.21.15.2:0/1468204618 wait complete. 2024-09-10T08:24:13.788 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:13 smithi002 ceph-mon[30563]: pgmap v869: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-10T08:24:13.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:13 smithi155 ceph-mon[35405]: pgmap v869: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-10T08:24:13.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:13 smithi107 ceph-mon[35048]: pgmap v869: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-10T08:24:13.941 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:24:14.322 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:14.790 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:24:14.790 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:24:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.911+0000 7faf9f000640 1 -- 172.21.15.2:0/652377937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf98104010 msgr2=0x7faf981043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.911+0000 7faf9f000640 1 --2- 172.21.15.2:0/652377937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf98104010 0x7faf981043f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7faf88009860 tx=0x7faf8802a340 comp rx=0 tx=0).stop 2024-09-10T08:24:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.911+0000 7faf9f000640 1 -- 172.21.15.2:0/652377937 shutdown_connections 2024-09-10T08:24:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.911+0000 7faf9f000640 1 --2- 172.21.15.2:0/652377937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf98109140 0x7faf98110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.911+0000 7faf9f000640 1 --2- 172.21.15.2:0/652377937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf981049c0 0x7faf98108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.911+0000 7faf9f000640 1 --2- 172.21.15.2:0/652377937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf98104010 0x7faf981043f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.911+0000 7faf9f000640 1 -- 172.21.15.2:0/652377937 >> 172.21.15.2:0/652377937 conn(0x7faf980ffa50 msgr2=0x7faf98101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:14.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.911+0000 7faf9f000640 1 -- 172.21.15.2:0/652377937 shutdown_connections 2024-09-10T08:24:14.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.911+0000 7faf9f000640 1 -- 172.21.15.2:0/652377937 wait complete. 2024-09-10T08:24:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.912+0000 7faf9f000640 1 Processor -- start 2024-09-10T08:24:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.913+0000 7faf9f000640 1 -- start start 2024-09-10T08:24:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.913+0000 7faf9f000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf98104010 0x7faf9819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.913+0000 7faf9f000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf981049c0 0x7faf9819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.913+0000 7faf9f000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf98109140 0x7faf9819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.913+0000 7faf9f000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf98114140 con 0x7faf98104010 2024-09-10T08:24:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.913+0000 7faf9f000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf98113fc0 con 0x7faf98109140 2024-09-10T08:24:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.913+0000 7faf9f000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf981142c0 con 0x7faf981049c0 2024-09-10T08:24:14.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.913+0000 7faf97e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf98104010 0x7faf9819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:14.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.913+0000 7faf9d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf98109140 0x7faf9819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:14.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.913+0000 7faf8fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf981049c0 0x7faf9819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:14.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.913+0000 7faf9d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf98109140 0x7faf9819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39548/0 (socket says 172.21.15.2:39548) 2024-09-10T08:24:14.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.913+0000 7faf9d000640 1 -- 172.21.15.2:0/1554268922 learned_addr learned my addr 172.21.15.2:0/1554268922 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:14.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.914+0000 7faf8fe00640 1 -- 172.21.15.2:0/1554268922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf98109140 msgr2=0x7faf9819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:14.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.914+0000 7faf8fe00640 1 --2- 172.21.15.2:0/1554268922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf98109140 0x7faf9819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:14.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.914+0000 7faf8fe00640 1 -- 172.21.15.2:0/1554268922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf98104010 msgr2=0x7faf9819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:14.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.914+0000 7faf8fe00640 1 --2- 172.21.15.2:0/1554268922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf98104010 0x7faf9819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:14.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.914+0000 7faf8fe00640 1 -- 172.21.15.2:0/1554268922 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9819f5f0 con 0x7faf981049c0 2024-09-10T08:24:14.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.914+0000 7faf8fe00640 1 --2- 172.21.15.2:0/1554268922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf981049c0 0x7faf9819ab80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faf8000e9f0 tx=0x7faf8000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:14.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.915+0000 7faf95600640 1 -- 172.21.15.2:0/1554268922 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8000ccc0 con 0x7faf981049c0 2024-09-10T08:24:14.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.915+0000 7faf9f000640 1 -- 172.21.15.2:0/1554268922 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9819f8e0 con 0x7faf981049c0 2024-09-10T08:24:14.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.915+0000 7faf95600640 1 -- 172.21.15.2:0/1554268922 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf80010910 con 0x7faf981049c0 2024-09-10T08:24:14.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.915+0000 7faf95600640 1 -- 172.21.15.2:0/1554268922 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80018960 con 0x7faf981049c0 2024-09-10T08:24:14.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.915+0000 7faf9f000640 1 -- 172.21.15.2:0/1554268922 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf981a71c0 con 0x7faf981049c0 2024-09-10T08:24:14.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.916+0000 7faf9f000640 1 -- 172.21.15.2:0/1554268922 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf981043f0 con 0x7faf981049c0 2024-09-10T08:24:14.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.918+0000 7faf95600640 1 -- 172.21.15.2:0/1554268922 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7faf80010ac0 con 0x7faf981049c0 2024-09-10T08:24:14.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.920+0000 7faf95600640 1 --2- 172.21.15.2:0/1554268922 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf7c0a2b80 0x7faf7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:14.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.920+0000 7faf95600640 1 -- 172.21.15.2:0/1554268922 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7faf800cfba0 con 0x7faf981049c0 2024-09-10T08:24:14.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.920+0000 7faf97e00640 1 --2- 172.21.15.2:0/1554268922 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf7c0a2b80 0x7faf7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:14.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.921+0000 7faf97e00640 1 --2- 172.21.15.2:0/1554268922 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf7c0a2b80 0x7faf7c0a5040 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7faf8802a930 tx=0x7faf880056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:14.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:14.922+0000 7faf95600640 1 -- 172.21.15.2:0/1554268922 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf8009c060 con 0x7faf981049c0 2024-09-10T08:24:15.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.160+0000 7faf9f000640 1 -- 172.21.15.2:0/1554268922 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf9802d7c0 con 0x7faf981049c0 2024-09-10T08:24:15.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.161+0000 7faf95600640 1 -- 172.21.15.2:0/1554268922 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7faf800a0360 con 0x7faf981049c0 2024-09-10T08:24:15.162 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:24:15.162 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:24:15.162 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:24:15.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.164+0000 7faf9f000640 1 -- 172.21.15.2:0/1554268922 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf7c0a2b80 msgr2=0x7faf7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:15.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.164+0000 7faf9f000640 1 --2- 172.21.15.2:0/1554268922 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf7c0a2b80 0x7faf7c0a5040 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7faf8802a930 tx=0x7faf880056d0 comp rx=0 tx=0).stop 2024-09-10T08:24:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.164+0000 7faf9f000640 1 -- 172.21.15.2:0/1554268922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf981049c0 msgr2=0x7faf9819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.164+0000 7faf9f000640 1 --2- 172.21.15.2:0/1554268922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf981049c0 0x7faf9819ab80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faf8000e9f0 tx=0x7faf8000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:24:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.165+0000 7faf9f000640 1 -- 172.21.15.2:0/1554268922 shutdown_connections 2024-09-10T08:24:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.165+0000 7faf9f000640 1 --2- 172.21.15.2:0/1554268922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf98109140 0x7faf9819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.165+0000 7faf9f000640 1 --2- 172.21.15.2:0/1554268922 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7faf7c0a2b80 0x7faf7c0a5040 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.165+0000 7faf9f000640 1 --2- 172.21.15.2:0/1554268922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf981049c0 0x7faf9819ab80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.165+0000 7faf9f000640 1 --2- 172.21.15.2:0/1554268922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf98104010 0x7faf9819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:15.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.165+0000 7faf9f000640 1 -- 172.21.15.2:0/1554268922 >> 172.21.15.2:0/1554268922 conn(0x7faf980ffa50 msgr2=0x7faf98100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:15.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.165+0000 7faf9f000640 1 -- 172.21.15.2:0/1554268922 shutdown_connections 2024-09-10T08:24:15.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:15.165+0000 7faf9f000640 1 -- 172.21.15.2:0/1554268922 wait complete. 2024-09-10T08:24:15.186 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:15.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:15 smithi002 ceph-mon[30563]: pgmap v870: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-09-10T08:24:15.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:15 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1554268922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:15.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:15 smithi155 ceph-mon[35405]: pgmap v870: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-09-10T08:24:15.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:15 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1554268922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:15.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:15 smithi107 ceph-mon[35048]: pgmap v870: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-09-10T08:24:15.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:15 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1554268922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:16.039 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --set-id 4978faaa --release --await' 2024-09-10T08:24:16.039 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --set-id 4978faaa --release --await 2024-09-10T08:24:16.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.583+0000 7fb763e00640 1 -- 172.21.15.2:0/1550641481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb764108f60 msgr2=0x7fb764110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:16.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.583+0000 7fb763e00640 1 --2- 172.21.15.2:0/1550641481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb764108f60 0x7fb764110ae0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009ca0 tx=0x7fb75c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.583+0000 7fb763e00640 1 -- 172.21.15.2:0/1550641481 shutdown_connections 2024-09-10T08:24:16.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.583+0000 7fb763e00640 1 --2- 172.21.15.2:0/1550641481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb764108f60 0x7fb764110ae0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.583+0000 7fb763e00640 1 --2- 172.21.15.2:0/1550641481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7641047e0 0x7fb764108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.583+0000 7fb763e00640 1 --2- 172.21.15.2:0/1550641481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb764103e30 0x7fb764104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.583+0000 7fb763e00640 1 -- 172.21.15.2:0/1550641481 >> 172.21.15.2:0/1550641481 conn(0x7fb7640ff910 msgr2=0x7fb764101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:16.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.583+0000 7fb763e00640 1 -- 172.21.15.2:0/1550641481 shutdown_connections 2024-09-10T08:24:16.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.583+0000 7fb763e00640 1 -- 172.21.15.2:0/1550641481 wait complete. 2024-09-10T08:24:16.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.584+0000 7fb763e00640 1 Processor -- start 2024-09-10T08:24:16.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.584+0000 7fb763e00640 1 -- start start 2024-09-10T08:24:16.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.585+0000 7fb763e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb764103e30 0x7fb76419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.585+0000 7fb763e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7641047e0 0x7fb76419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.585+0000 7fb763e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb764108f60 0x7fb76419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.585+0000 7fb763e00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb764114010 con 0x7fb764103e30 2024-09-10T08:24:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.585+0000 7fb763e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb764113e90 con 0x7fb764108f60 2024-09-10T08:24:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.585+0000 7fb763e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb764114190 con 0x7fb7641047e0 2024-09-10T08:24:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.585+0000 7fb762000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7641047e0 0x7fb76419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.585+0000 7fb762a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb764103e30 0x7fb76419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.585+0000 7fb762000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7641047e0 0x7fb76419a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:46572/0 (socket says 172.21.15.2:46572) 2024-09-10T08:24:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.585+0000 7fb762000640 1 -- 172.21.15.2:0/3373162605 learned_addr learned my addr 172.21.15.2:0/3373162605 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.585+0000 7fb762a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb764103e30 0x7fb76419a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47386/0 (socket says 172.21.15.2:47386) 2024-09-10T08:24:16.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.585+0000 7fb763400640 1 --2- 172.21.15.2:0/3373162605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb764108f60 0x7fb76419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.586+0000 7fb762000640 1 -- 172.21.15.2:0/3373162605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb764108f60 msgr2=0x7fb76419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:16.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.586+0000 7fb762000640 1 --2- 172.21.15.2:0/3373162605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb764108f60 0x7fb76419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.586+0000 7fb762000640 1 -- 172.21.15.2:0/3373162605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb764103e30 msgr2=0x7fb76419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:16.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.586+0000 7fb762000640 1 --2- 172.21.15.2:0/3373162605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb764103e30 0x7fb76419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.586+0000 7fb762000640 1 -- 172.21.15.2:0/3373162605 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76419f3a0 con 0x7fb7641047e0 2024-09-10T08:24:16.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.586+0000 7fb762000640 1 --2- 172.21.15.2:0/3373162605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7641047e0 0x7fb76419a930 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb75000e9f0 tx=0x7fb75000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:16.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.586+0000 7fb75b400640 1 -- 172.21.15.2:0/3373162605 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75000cd80 con 0x7fb7641047e0 2024-09-10T08:24:16.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.586+0000 7fb75b400640 1 -- 172.21.15.2:0/3373162605 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb750005500 con 0x7fb7641047e0 2024-09-10T08:24:16.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.586+0000 7fb763e00640 1 -- 172.21.15.2:0/3373162605 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76419f690 con 0x7fb7641047e0 2024-09-10T08:24:16.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.586+0000 7fb75b400640 1 -- 172.21.15.2:0/3373162605 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750010690 con 0x7fb7641047e0 2024-09-10T08:24:16.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.586+0000 7fb763e00640 1 -- 172.21.15.2:0/3373162605 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7641a6f70 con 0x7fb7641047e0 2024-09-10T08:24:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.590+0000 7fb76c0c4740 1 Processor -- start 2024-09-10T08:24:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.590+0000 7fb75b400640 1 -- 172.21.15.2:0/3373162605 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb7500046c0 con 0x7fb7641047e0 2024-09-10T08:24:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.590+0000 7fb76c0c4740 1 -- start start 2024-09-10T08:24:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.590+0000 7fb76c0c4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc1978cb0 0x561dc1979090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.590+0000 7fb76c0c4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc19795d0 0x561dc1983950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.590+0000 7fb76c0c4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1983e90 0x561dc1986280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.590+0000 7fb76c0c4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561dc18df290 con 0x561dc19795d0 2024-09-10T08:24:16.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.590+0000 7fb76c0c4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561dc1886b40 con 0x561dc1983e90 2024-09-10T08:24:16.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.590+0000 7fb76c0c4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561dc18db250 con 0x561dc1978cb0 2024-09-10T08:24:16.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.591+0000 7fb762a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc19795d0 0x561dc1983950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.591+0000 7fb762a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc19795d0 0x561dc1983950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47400/0 (socket says 172.21.15.2:47400) 2024-09-10T08:24:16.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.591+0000 7fb763400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc1978cb0 0x561dc1979090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.591+0000 7fb762a00640 1 -- 172.21.15.2:0/13502629 learned_addr learned my addr 172.21.15.2:0/13502629 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:16.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.591+0000 7fb762000640 1 --2- 172.21.15.2:0/13502629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1983e90 0x561dc1986280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.591+0000 7fb762a00640 1 -- 172.21.15.2:0/13502629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc1978cb0 msgr2=0x561dc1979090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:16.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.591+0000 7fb762a00640 1 --2- 172.21.15.2:0/13502629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc1978cb0 0x561dc1979090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.591+0000 7fb762a00640 1 -- 172.21.15.2:0/13502629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1983e90 msgr2=0x561dc1986280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:16.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.591+0000 7fb762a00640 1 --2- 172.21.15.2:0/13502629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1983e90 0x561dc1986280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.591+0000 7fb762a00640 1 -- 172.21.15.2:0/13502629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc16506a0 con 0x561dc19795d0 2024-09-10T08:24:16.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.591+0000 7fb762a00640 1 --2- 172.21.15.2:0/13502629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc19795d0 0x561dc1983950 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb754009cf0 tx=0x7fb754011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b9aa5a01daa338 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:16.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.592+0000 7fb74f400640 1 -- 172.21.15.2:0/13502629 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75401e070 con 0x561dc19795d0 2024-09-10T08:24:16.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.592+0000 7fb74f400640 1 -- 172.21.15.2:0/13502629 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb754019b10 con 0x561dc19795d0 2024-09-10T08:24:16.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.592+0000 7fb74f400640 1 -- 172.21.15.2:0/13502629 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75400ea00 con 0x561dc19795d0 2024-09-10T08:24:16.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.592+0000 7fb75b400640 1 --2- 172.21.15.2:0/3373162605 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7480a2c00 0x7fb7480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.592+0000 7fb75b400640 1 -- 172.21.15.2:0/3373162605 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb7500cf890 con 0x7fb7641047e0 2024-09-10T08:24:16.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.593+0000 7fb763400640 1 --2- 172.21.15.2:0/3373162605 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7480a2c00 0x7fb7480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.593+0000 7fb76c0c4740 1 -- 172.21.15.2:0/13502629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc19795d0 msgr2=0x561dc1983950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:16.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.593+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/13502629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc19795d0 0x561dc1983950 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb754009cf0 tx=0x7fb754011810 comp rx=0 tx=0).stop 2024-09-10T08:24:16.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.593+0000 7fb76c0c4740 1 -- 172.21.15.2:0/13502629 shutdown_connections 2024-09-10T08:24:16.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.593+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/13502629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1983e90 0x561dc1986280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.593+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/13502629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc19795d0 0x561dc1983950 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.593+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/13502629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc1978cb0 0x561dc1979090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.593+0000 7fb76c0c4740 1 -- 172.21.15.2:0/13502629 >> 172.21.15.2:0/13502629 conn(0x561dc1975240 msgr2=0x561dc1975620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:16.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.593+0000 7fb76c0c4740 1 -- 172.21.15.2:0/13502629 shutdown_connections 2024-09-10T08:24:16.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.593+0000 7fb763400640 1 --2- 172.21.15.2:0/3373162605 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7480a2c00 0x7fb7480a50c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb75c02f540 tx=0x7fb75c02f3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:16.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.594+0000 7fb76c0c4740 1 -- 172.21.15.2:0/13502629 wait complete. 2024-09-10T08:24:16.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.594+0000 7fb76c0c4740 1 -- start start 2024-09-10T08:24:16.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.594+0000 7fb76c0c4740 1 Processor -- start 2024-09-10T08:24:16.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb76c0c4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc1983730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb76c0c4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc19809e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb76c0c4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1983e90 0x561dc1980f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb76c0c4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561dc1986b00 con 0x561dc1978cb0 2024-09-10T08:24:16.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb76c0c4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561dc1899fa0 con 0x561dc1983e90 2024-09-10T08:24:16.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb76c0c4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561dc1986c80 con 0x561dc19795d0 2024-09-10T08:24:16.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb76c0c4740 10 client.0 _ll_register_callbacks cb 0x561dc15edba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:16.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb763400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1983e90 0x561dc1980f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb762a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc1983730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb763400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1983e90 0x561dc1980f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39582/0 (socket says 172.21.15.2:39582) 2024-09-10T08:24:16.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb763400640 1 -- 172.21.15.2:0/3249477370 learned_addr learned my addr 172.21.15.2:0/3249477370 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:16.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb762000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc19809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb762000640 1 -- 172.21.15.2:0/3249477370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1983e90 msgr2=0x561dc1980f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:16.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb762000640 1 --2- 172.21.15.2:0/3249477370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1983e90 0x561dc1980f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb762000640 1 -- 172.21.15.2:0/3249477370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 msgr2=0x561dc1983730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:16.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.595+0000 7fb762000640 1 --2- 172.21.15.2:0/3249477370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc1983730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.596+0000 7fb762000640 1 -- 172.21.15.2:0/3249477370 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc1981720 con 0x561dc19795d0 2024-09-10T08:24:16.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.596+0000 7fb763400640 1 --2- 172.21.15.2:0/3249477370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1983e90 0x561dc1980f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:16.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.596+0000 7fb762a00640 1 --2- 172.21.15.2:0/3249477370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc1983730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:16.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.596+0000 7fb762000640 1 --2- 172.21.15.2:0/3249477370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc19809e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x561dc19765c0 tx=0x7fb750004ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17d426cef32f7eee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:16.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.596+0000 7fb743e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:24:16.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.596+0000 7fb743e00640 1 -- 172.21.15.2:0/3249477370 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75009b600 con 0x561dc19795d0 2024-09-10T08:24:16.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.596+0000 7fb743e00640 1 -- 172.21.15.2:0/3249477370 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb7500a4480 con 0x561dc19795d0 2024-09-10T08:24:16.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.596+0000 7fb76c0c4740 1 -- 172.21.15.2:0/3249477370 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb754019b10 con 0x561dc19795d0 2024-09-10T08:24:16.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.596+0000 7fb743e00640 1 -- 172.21.15.2:0/3249477370 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb75009c3a0 con 0x561dc19795d0 2024-09-10T08:24:16.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.596+0000 7fb76c0c4740 10 client.29983 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:16.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.596+0000 7fb76c0c4740 1 -- 172.21.15.2:0/3249477370 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561dc1992980 con 0x561dc19795d0 2024-09-10T08:24:16.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.597+0000 7fb743e00640 1 -- 172.21.15.2:0/3249477370 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb754019b10 con 0x561dc19795d0 2024-09-10T08:24:16.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.597+0000 7fb743e00640 1 -- 172.21.15.2:0/3249477370 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb75009b0f0 con 0x561dc19795d0 2024-09-10T08:24:16.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.597+0000 7fb76c0c4740 10 client.29983 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:16.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.597+0000 7fb76c0c4740 10 client.29983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:16.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.597+0000 7fb76c0c4740 20 client.29983 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:16.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.597+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/3249477370 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561dc1993de0 0x561dc1996250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.597+0000 7fb76c0c4740 4 client.29983 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:16.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.597+0000 7fb76c0c4740 1 -- 172.21.15.2:0/3249477370 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dc18ec800 con 0x561dc1993de0 2024-09-10T08:24:16.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.597+0000 7fb762a00640 1 --2- 172.21.15.2:0/3249477370 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561dc1993de0 0x561dc1996250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.598+0000 7fb762a00640 1 --2- 172.21.15.2:0/3249477370 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561dc1993de0 0x561dc1996250 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40c6fd4d8d2529f2 server_cookie=9ddbf0e12dd97327 in_seq=0 out_seq=0 2024-09-10T08:24:16.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.598+0000 7fb743e00640 10 client.29983 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:16.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb743e00640 1 -- 172.21.15.2:0/3249477370 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561dc18ec800 con 0x561dc1993de0 2024-09-10T08:24:16.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb743e00640 10 client.29983 handle_command_reply: tid=0 2024-09-10T08:24:16.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb76c0c4740 1 client.29983 shutdown 2024-09-10T08:24:16.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb76c0c4740 1 -- 172.21.15.2:0/3249477370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 msgr2=0x561dc19809e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:16.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/3249477370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc19809e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x561dc19765c0 tx=0x7fb750004ae0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb76c0c4740 1 -- 172.21.15.2:0/3249477370 shutdown_connections 2024-09-10T08:24:16.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/3249477370 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561dc1993de0 0x561dc1996250 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/3249477370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1983e90 0x561dc1980f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/3249477370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc19809e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/3249477370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc1983730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb743e00640 0 client.29983 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:16.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb76c0c4740 1 -- 172.21.15.2:0/3249477370 >> 172.21.15.2:0/3249477370 conn(0x561dc1975240 msgr2=0x561dc197af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:16.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb76c0c4740 1 -- 172.21.15.2:0/3249477370 shutdown_connections 2024-09-10T08:24:16.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.599+0000 7fb76c0c4740 1 -- 172.21.15.2:0/3249477370 wait complete. 2024-09-10T08:24:16.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.600+0000 7fb76c0c4740 20 client.29983 trim_cache size 0 max 16384 2024-09-10T08:24:16.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.618+0000 7fb76c0c4740 1 Processor -- start 2024-09-10T08:24:16.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.619+0000 7fb76c0c4740 1 -- start start 2024-09-10T08:24:16.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.619+0000 7fb76c0c4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1975240 0x561dc1974160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.619+0000 7fb76c0c4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc199bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.619+0000 7fb76c0c4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc199c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.619+0000 7fb76c0c4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641b3770 con 0x561dc1978cb0 2024-09-10T08:24:16.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.619+0000 7fb76c0c4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641b1040 con 0x561dc1975240 2024-09-10T08:24:16.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.619+0000 7fb76c0c4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641aee90 con 0x561dc19795d0 2024-09-10T08:24:16.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.619+0000 7fb762000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc199bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.619+0000 7fb762a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1975240 0x561dc1974160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.619+0000 7fb762a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1975240 0x561dc1974160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39584/0 (socket says 172.21.15.2:39584) 2024-09-10T08:24:16.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.619+0000 7fb762a00640 1 -- 172.21.15.2:0/3892659115 learned_addr learned my addr 172.21.15.2:0/3892659115 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:16.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.619+0000 7fb762000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc199bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47414/0 (socket says 172.21.15.2:47414) 2024-09-10T08:24:16.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.619+0000 7fb763400640 1 --2- 172.21.15.2:0/3892659115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc199c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.620+0000 7fb762a00640 1 -- 172.21.15.2:0/3892659115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 msgr2=0x561dc199c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:16.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.620+0000 7fb762a00640 1 --2- 172.21.15.2:0/3892659115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc199c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.620+0000 7fb762a00640 1 -- 172.21.15.2:0/3892659115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 msgr2=0x561dc199bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:16.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.620+0000 7fb762a00640 1 --2- 172.21.15.2:0/3892659115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc199bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.620+0000 7fb762a00640 1 -- 172.21.15.2:0/3892659115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc1979b40 con 0x561dc1975240 2024-09-10T08:24:16.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.620+0000 7fb762a00640 1 --2- 172.21.15.2:0/3892659115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1975240 0x561dc1974160 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb7540058c0 tx=0x7fb754009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf37988bb9248dcd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:16.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.620+0000 7fb74ea00640 1 -- 172.21.15.2:0/3892659115 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75401e040 con 0x561dc1975240 2024-09-10T08:24:16.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.620+0000 7fb74ea00640 1 -- 172.21.15.2:0/3892659115 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb75401f040 con 0x561dc1975240 2024-09-10T08:24:16.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.620+0000 7fb74ea00640 1 -- 172.21.15.2:0/3892659115 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754003cb0 con 0x561dc1975240 2024-09-10T08:24:16.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.621+0000 7fb76c0c4740 1 -- 172.21.15.2:0/3892659115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1975240 msgr2=0x561dc1974160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:16.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.621+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/3892659115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1975240 0x561dc1974160 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb7540058c0 tx=0x7fb754009c80 comp rx=0 tx=0).stop 2024-09-10T08:24:16.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.621+0000 7fb76c0c4740 1 -- 172.21.15.2:0/3892659115 shutdown_connections 2024-09-10T08:24:16.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.621+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/3892659115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc199c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.621+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/3892659115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc199bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.621+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/3892659115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1975240 0x561dc1974160 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.621+0000 7fb76c0c4740 1 -- 172.21.15.2:0/3892659115 >> 172.21.15.2:0/3892659115 conn(0x561dc1981080 msgr2=0x561dc1973a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:16.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.622+0000 7fb76c0c4740 1 -- 172.21.15.2:0/3892659115 shutdown_connections 2024-09-10T08:24:16.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.622+0000 7fb76c0c4740 1 -- 172.21.15.2:0/3892659115 wait complete. 2024-09-10T08:24:16.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.622+0000 7fb76c0c4740 1 -- start start 2024-09-10T08:24:16.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.622+0000 7fb76c0c4740 1 Processor -- start 2024-09-10T08:24:16.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.623+0000 7fb76c0c4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1975240 0x561dc198a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.623+0000 7fb76c0c4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc198a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.623+0000 7fb76c0c4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc19fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.623+0000 7fb76c0c4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641af500 con 0x561dc1978cb0 2024-09-10T08:24:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.623+0000 7fb76c0c4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641b0970 con 0x561dc1975240 2024-09-10T08:24:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.623+0000 7fb762a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1975240 0x561dc198a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.623+0000 7fb762a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1975240 0x561dc198a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:39594/0 (socket says 172.21.15.2:39594) 2024-09-10T08:24:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.623+0000 7fb762a00640 1 -- 172.21.15.2:0/2764987887 learned_addr learned my addr 172.21.15.2:0/2764987887 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.623+0000 7fb76c0c4740 1 -- 172.21.15.2:0/2764987887 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641afbd0 con 0x561dc19795d0 2024-09-10T08:24:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.623+0000 7fb76c0c4740 10 client.0 _ll_register_callbacks cb 0x7fb75000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.623+0000 7fb762000640 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc198a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.623+0000 7fb763400640 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc19fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.624+0000 7fb762000640 1 -- 172.21.15.2:0/2764987887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 msgr2=0x561dc19fdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.624+0000 7fb762000640 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc19fdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.624+0000 7fb762000640 1 -- 172.21.15.2:0/2764987887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1975240 msgr2=0x561dc198a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.624+0000 7fb762000640 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1975240 0x561dc198a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.624+0000 7fb762000640 1 -- 172.21.15.2:0/2764987887 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc19fe040 con 0x561dc1978cb0 2024-09-10T08:24:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.624+0000 7fb763400640 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc19fdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.624+0000 7fb762000640 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc198a810 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb750099f50 tx=0x7fb75000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcc906d970b1c80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.624+0000 7fb743e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:24:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.625+0000 7fb743e00640 1 -- 172.21.15.2:0/2764987887 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7500b8070 con 0x561dc1978cb0 2024-09-10T08:24:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.625+0000 7fb743e00640 1 -- 172.21.15.2:0/2764987887 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fb750097bf0 con 0x561dc1978cb0 2024-09-10T08:24:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.625+0000 7fb76c0c4740 1 -- 172.21.15.2:0/2764987887 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb75401f040 con 0x561dc1978cb0 2024-09-10T08:24:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.625+0000 7fb743e00640 1 -- 172.21.15.2:0/2764987887 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb7500ab050 con 0x561dc1978cb0 2024-09-10T08:24:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.625+0000 7fb76c0c4740 10 client.20043 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.625+0000 7fb76c0c4740 1 -- 172.21.15.2:0/2764987887 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561dc19ff2c0 con 0x561dc1978cb0 2024-09-10T08:24:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.625+0000 7fb743e00640 1 -- 172.21.15.2:0/2764987887 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb75401f040 con 0x561dc1978cb0 2024-09-10T08:24:16.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.626+0000 7fb743e00640 1 -- 172.21.15.2:0/2764987887 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb7500a4b60 con 0x561dc1978cb0 2024-09-10T08:24:16.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.626+0000 7fb76c0c4740 10 client.20043 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:16.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.626+0000 7fb76c0c4740 10 client.20043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:16.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.626+0000 7fb76c0c4740 20 client.20043 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:16.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.626+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561dc1983e90 0x561dc1a01a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:16.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.626+0000 7fb76c0c4740 4 client.20043 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:24:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.626+0000 7fb76c0c4740 1 -- 172.21.15.2:0/2764987887 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561dc18e6d10 con 0x561dc1983e90 2024-09-10T08:24:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.626+0000 7fb762a00640 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561dc1983e90 0x561dc1a01a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.627+0000 7fb762a00640 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561dc1983e90 0x561dc1a01a50 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4825f1c3c5b03d9d server_cookie=e6d32fc04a7ecc67 in_seq=0 out_seq=0 2024-09-10T08:24:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.627+0000 7fb743e00640 10 client.20043 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.628+0000 7fb743e00640 1 -- 172.21.15.2:0/2764987887 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561dc18e6d10 con 0x561dc1983e90 2024-09-10T08:24:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.628+0000 7fb743e00640 10 client.20043 handle_command_reply: tid=0 2024-09-10T08:24:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.628+0000 7fb76c0c4740 1 client.20043 shutdown 2024-09-10T08:24:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.628+0000 7fb76c0c4740 1 -- 172.21.15.2:0/2764987887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 msgr2=0x561dc198a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.628+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc198a810 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb750099f50 tx=0x7fb75000c7d0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.628+0000 7fb76c0c4740 1 -- 172.21.15.2:0/2764987887 shutdown_connections 2024-09-10T08:24:16.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.628+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561dc1983e90 0x561dc1a01a50 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.628+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc19795d0 0x561dc19fdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.628+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561dc1978cb0 0x561dc198a810 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.628+0000 7fb76c0c4740 1 --2- 172.21.15.2:0/2764987887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc1975240 0x561dc198a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.628+0000 7fb743e00640 0 client.20043 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:16.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.628+0000 7fb76c0c4740 1 -- 172.21.15.2:0/2764987887 >> 172.21.15.2:0/2764987887 conn(0x561dc1981080 msgr2=0x561dc1974270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:16.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.629+0000 7fb76c0c4740 1 -- 172.21.15.2:0/2764987887 shutdown_connections 2024-09-10T08:24:16.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.629+0000 7fb76c0c4740 1 -- 172.21.15.2:0/2764987887 wait complete. 2024-09-10T08:24:16.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.629+0000 7fb76c0c4740 20 client.20043 trim_cache size 0 max 16384 2024-09-10T08:24:16.631 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:24:16.631 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:24:16.631 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "a9c13ff2-9ab6-44ca-84ef-d49160dfda9c", 2024-09-10T08:24:16.631 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:24:16.631 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:24:16.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.630+0000 7fb763e00640 1 -- 172.21.15.2:0/3373162605 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7480a2c00 msgr2=0x7fb7480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:16.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.630+0000 7fb763e00640 1 --2- 172.21.15.2:0/3373162605 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7480a2c00 0x7fb7480a50c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb75c02f540 tx=0x7fb75c02f3b0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.630+0000 7fb763e00640 1 -- 172.21.15.2:0/3373162605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7641047e0 msgr2=0x7fb76419a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:16.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.630+0000 7fb763e00640 1 --2- 172.21.15.2:0/3373162605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7641047e0 0x7fb76419a930 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb75000e9f0 tx=0x7fb75000edf0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.630+0000 7fb763e00640 1 -- 172.21.15.2:0/3373162605 shutdown_connections 2024-09-10T08:24:16.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.630+0000 7fb763e00640 1 --2- 172.21.15.2:0/3373162605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb764108f60 0x7fb76419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.630+0000 7fb763e00640 1 --2- 172.21.15.2:0/3373162605 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fb7480a2c00 0x7fb7480a50c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.630+0000 7fb763e00640 1 --2- 172.21.15.2:0/3373162605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7641047e0 0x7fb76419a930 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.630+0000 7fb763e00640 1 --2- 172.21.15.2:0/3373162605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb764103e30 0x7fb76419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:16.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.630+0000 7fb763e00640 1 -- 172.21.15.2:0/3373162605 >> 172.21.15.2:0/3373162605 conn(0x7fb7640ff910 msgr2=0x7fb764100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:16.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.630+0000 7fb763e00640 1 -- 172.21.15.2:0/3373162605 shutdown_connections 2024-09-10T08:24:16.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:16.630+0000 7fb763e00640 1 -- 172.21.15.2:0/3373162605 wait complete. 2024-09-10T08:24:16.871 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:17.858 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:17 smithi002 ceph-mon[30563]: pgmap v871: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:24:17.858 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:17 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:24:17.858 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:17 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:24:17.858 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:17 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:24:17.858 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:17 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:24:17.895 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:24:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:17 smithi155 ceph-mon[35405]: pgmap v871: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:24:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:17 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:24:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:17 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:24:17.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:17 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:24:17.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:17 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:24:17.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:17 smithi107 ceph-mon[35048]: pgmap v871: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T08:24:17.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:17 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:24:17.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:17 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:24:17.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:17 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:24:17.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:17 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:24:18.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.133+0000 7f0256400640 1 -- 172.21.15.2:0/726984044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0250103ff0 msgr2=0x7f02501043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.133+0000 7f0256400640 1 --2- 172.21.15.2:0/726984044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0250103ff0 0x7f02501043d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0240009900 tx=0x7f024002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.133+0000 7f0256400640 1 -- 172.21.15.2:0/726984044 shutdown_connections 2024-09-10T08:24:18.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.133+0000 7f0256400640 1 --2- 172.21.15.2:0/726984044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0250109120 0x7f0250110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.133+0000 7f0256400640 1 --2- 172.21.15.2:0/726984044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02501049a0 0x7f02501089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.133+0000 7f0256400640 1 --2- 172.21.15.2:0/726984044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0250103ff0 0x7f02501043d0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.133+0000 7f0256400640 1 -- 172.21.15.2:0/726984044 >> 172.21.15.2:0/726984044 conn(0x7f02500ffa90 msgr2=0x7f0250101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:18.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.133+0000 7f0256400640 1 -- 172.21.15.2:0/726984044 shutdown_connections 2024-09-10T08:24:18.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.134+0000 7f0256400640 1 -- 172.21.15.2:0/726984044 wait complete. 2024-09-10T08:24:18.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.135+0000 7f0256400640 1 Processor -- start 2024-09-10T08:24:18.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.135+0000 7f0256400640 1 -- start start 2024-09-10T08:24:18.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.135+0000 7f0256400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0250103ff0 0x7f025019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.135+0000 7f0256400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02501049a0 0x7f025019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f0256400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0250109120 0x7f025019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f0256400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0250114120 con 0x7f0250109120 2024-09-10T08:24:18.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f0256400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0250113fa0 con 0x7f0250103ff0 2024-09-10T08:24:18.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f0256400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f02501142a0 con 0x7f02501049a0 2024-09-10T08:24:18.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f024ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02501049a0 0x7f025019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f024ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02501049a0 0x7f025019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51244/0 (socket says 172.21.15.2:51244) 2024-09-10T08:24:18.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f024ea00640 1 -- 172.21.15.2:0/518005257 learned_addr learned my addr 172.21.15.2:0/518005257 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:18.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f024f400640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0250103ff0 0x7f025019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f024fe00640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0250109120 0x7f025019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f024f400640 1 -- 172.21.15.2:0/518005257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02501049a0 msgr2=0x7f025019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f024f400640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02501049a0 0x7f025019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f024f400640 1 -- 172.21.15.2:0/518005257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0250109120 msgr2=0x7f025019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f024f400640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0250109120 0x7f025019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.136+0000 7f024f400640 1 -- 172.21.15.2:0/518005257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025019f5b0 con 0x7f0250103ff0 2024-09-10T08:24:18.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.137+0000 7f024f400640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0250103ff0 0x7f025019a600 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0240005940 tx=0x7f02400040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:18.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.137+0000 7f0247e00640 1 -- 172.21.15.2:0/518005257 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024003f030 con 0x7f0250103ff0 2024-09-10T08:24:18.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.137+0000 7f0256400640 1 -- 172.21.15.2:0/518005257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025019f840 con 0x7f0250103ff0 2024-09-10T08:24:18.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.137+0000 7f0256400640 1 -- 172.21.15.2:0/518005257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02501a7180 con 0x7f0250103ff0 2024-09-10T08:24:18.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.137+0000 7f0247e00640 1 -- 172.21.15.2:0/518005257 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02400044c0 con 0x7f0250103ff0 2024-09-10T08:24:18.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.137+0000 7f0247e00640 1 -- 172.21.15.2:0/518005257 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240049060 con 0x7f0250103ff0 2024-09-10T08:24:18.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.141+0000 7f0258eb8740 1 Processor -- start 2024-09-10T08:24:18.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.141+0000 7f0247e00640 1 -- 172.21.15.2:0/518005257 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0240042030 con 0x7f0250103ff0 2024-09-10T08:24:18.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.141+0000 7f0258eb8740 1 -- start start 2024-09-10T08:24:18.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.141+0000 7f0258eb8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0310 0x55798b2d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.141+0000 7f0258eb8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2d0c30 0x55798b2dafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.141+0000 7f0258eb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2db4f0 0x55798b2dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.141+0000 7f0258eb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55798b236650 con 0x55798b2db4f0 2024-09-10T08:24:18.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.141+0000 7f0258eb8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55798b1ddf10 con 0x55798b2d0310 2024-09-10T08:24:18.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.141+0000 7f0258eb8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55798b232610 con 0x55798b2d0c30 2024-09-10T08:24:18.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.142+0000 7f024f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2d0c30 0x55798b2dafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.142+0000 7f024fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0310 0x55798b2d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.142+0000 7f024fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0310 0x55798b2d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36770/0 (socket says 172.21.15.2:36770) 2024-09-10T08:24:18.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.142+0000 7f024fe00640 1 -- 172.21.15.2:0/1315822861 learned_addr learned my addr 172.21.15.2:0/1315822861 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:18.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.142+0000 7f024ea00640 1 --2- 172.21.15.2:0/1315822861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2db4f0 0x55798b2dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.142+0000 7f024fe00640 1 -- 172.21.15.2:0/1315822861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2d0c30 msgr2=0x55798b2dafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.142+0000 7f024fe00640 1 --2- 172.21.15.2:0/1315822861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2d0c30 0x55798b2dafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.142+0000 7f024fe00640 1 -- 172.21.15.2:0/1315822861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2db4f0 msgr2=0x55798b2dd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.142+0000 7f024fe00640 1 --2- 172.21.15.2:0/1315822861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2db4f0 0x55798b2dd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.142+0000 7f024fe00640 1 -- 172.21.15.2:0/1315822861 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798af45580 con 0x55798b2d0310 2024-09-10T08:24:18.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.142+0000 7f024ea00640 1 --2- 172.21.15.2:0/1315822861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2db4f0 0x55798b2dd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:24:18.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.143+0000 7f024fe00640 1 --2- 172.21.15.2:0/1315822861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0310 0x55798b2d06f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0248009f50 tx=0x7f02480118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d66063ab4ae0fe76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:18.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.143+0000 7f0247e00640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02340a2b80 0x7f02340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.143+0000 7f0244c00640 1 -- 172.21.15.2:0/1315822861 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248004d20 con 0x55798b2d0310 2024-09-10T08:24:18.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.143+0000 7f0244c00640 1 -- 172.21.15.2:0/1315822861 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0248004050 con 0x55798b2d0310 2024-09-10T08:24:18.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.143+0000 7f0247e00640 1 -- 172.21.15.2:0/518005257 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f02400ef430 con 0x7f0250103ff0 2024-09-10T08:24:18.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.143+0000 7f024f400640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02340a2b80 0x7f02340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.143+0000 7f0244c00640 1 -- 172.21.15.2:0/1315822861 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248005280 con 0x55798b2d0310 2024-09-10T08:24:18.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.144+0000 7f0258eb8740 1 -- 172.21.15.2:0/1315822861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0310 msgr2=0x55798b2d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.144+0000 7f0258eb8740 1 --2- 172.21.15.2:0/1315822861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0310 0x55798b2d06f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0248009f50 tx=0x7f02480118d0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.144+0000 7f024f400640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02340a2b80 0x7f02340a5040 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0240009230 tx=0x7f02400f2e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:18.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.144+0000 7f0258eb8740 1 -- 172.21.15.2:0/1315822861 shutdown_connections 2024-09-10T08:24:18.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.144+0000 7f0258eb8740 1 --2- 172.21.15.2:0/1315822861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2db4f0 0x55798b2dd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.144+0000 7f0258eb8740 1 --2- 172.21.15.2:0/1315822861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2d0c30 0x55798b2dafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.144+0000 7f0258eb8740 1 --2- 172.21.15.2:0/1315822861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0310 0x55798b2d06f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.144+0000 7f0258eb8740 1 -- 172.21.15.2:0/1315822861 >> 172.21.15.2:0/1315822861 conn(0x55798b2cc8a0 msgr2=0x55798b2ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:18.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.144+0000 7f0258eb8740 1 -- 172.21.15.2:0/1315822861 shutdown_connections 2024-09-10T08:24:18.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.144+0000 7f0258eb8740 1 -- 172.21.15.2:0/1315822861 wait complete. 2024-09-10T08:24:18.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.145+0000 7f0258eb8740 1 -- start start 2024-09-10T08:24:18.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.145+0000 7f0258eb8740 1 Processor -- start 2024-09-10T08:24:18.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.146+0000 7f0258eb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0310 0x55798b2d84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.146+0000 7f0258eb8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0c30 0x55798b2d89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.146+0000 7f0258eb8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 0x55798b2d8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.146+0000 7f0258eb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55798b2de0c0 con 0x55798b2d0310 2024-09-10T08:24:18.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.146+0000 7f0258eb8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55798b1f1350 con 0x55798b2d0c30 2024-09-10T08:24:18.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.146+0000 7f0258eb8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55798b2de240 con 0x55798b2db4f0 2024-09-10T08:24:18.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.146+0000 7f0258eb8740 10 client.0 _ll_register_callbacks cb 0x55798b214290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:18.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.146+0000 7f024fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0310 0x55798b2d84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.146+0000 7f024fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0310 0x55798b2d84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43524/0 (socket says 172.21.15.2:43524) 2024-09-10T08:24:18.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.146+0000 7f024fe00640 1 -- 172.21.15.2:0/1824021241 learned_addr learned my addr 172.21.15.2:0/1824021241 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:18.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.146+0000 7f024fe00640 1 --2- 172.21.15.2:0/1824021241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 0x55798b2d8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.146+0000 7f024ea00640 1 --2- 172.21.15.2:0/1824021241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0c30 0x55798b2d89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.147+0000 7f024ea00640 1 -- 172.21.15.2:0/1824021241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 msgr2=0x55798b2d8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.147+0000 7f024ea00640 1 --2- 172.21.15.2:0/1824021241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 0x55798b2d8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.147+0000 7f024ea00640 1 -- 172.21.15.2:0/1824021241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0310 msgr2=0x55798b2d84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.147+0000 7f024ea00640 1 --2- 172.21.15.2:0/1824021241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0310 0x55798b2d84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.147+0000 7f024ea00640 1 -- 172.21.15.2:0/1824021241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798b00f960 con 0x55798b2d0c30 2024-09-10T08:24:18.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.147+0000 7f024ea00640 1 --2- 172.21.15.2:0/1824021241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0c30 0x55798b2d89f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f02500fff90 tx=0x7f023c013b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94f366e81c06db64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:18.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.147+0000 7f0239600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:18.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.147+0000 7f0239600640 1 -- 172.21.15.2:0/1824021241 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c0119c0 con 0x55798b2d0c30 2024-09-10T08:24:18.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.147+0000 7f0239600640 1 -- 172.21.15.2:0/1824021241 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f023c011b60 con 0x55798b2d0c30 2024-09-10T08:24:18.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.147+0000 7f0258eb8740 1 -- 172.21.15.2:0/1824021241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0248004050 con 0x55798b2d0c30 2024-09-10T08:24:18.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.148+0000 7f0239600640 1 -- 172.21.15.2:0/1824021241 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f023c011d60 con 0x55798b2d0c30 2024-09-10T08:24:18.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.148+0000 7f0258eb8740 10 client.30071 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:18.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.148+0000 7f0258eb8740 1 -- 172.21.15.2:0/1824021241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55798b2f25b0 con 0x55798b2d0c30 2024-09-10T08:24:18.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.148+0000 7f0239600640 1 -- 172.21.15.2:0/1824021241 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0248004050 con 0x55798b2d0c30 2024-09-10T08:24:18.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.148+0000 7f0239600640 1 -- 172.21.15.2:0/1824021241 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f023c01a070 con 0x55798b2d0c30 2024-09-10T08:24:18.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.148+0000 7f0258eb8740 10 client.30071 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:18.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.148+0000 7f0258eb8740 10 client.30071 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:24:18.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.148+0000 7f0258eb8740 20 client.30071 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:18.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.148+0000 7f0258eb8740 1 --2- 172.21.15.2:0/1824021241 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55798b2f3a70 0x55798b2f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.148+0000 7f0258eb8740 4 client.30071 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:18.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.148+0000 7f0258eb8740 1 -- 172.21.15.2:0/1824021241 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55798b243bc0 con 0x55798b2f3a70 2024-09-10T08:24:18.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.149+0000 7f024fe00640 1 --2- 172.21.15.2:0/1824021241 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55798b2f3a70 0x55798b2f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.149+0000 7f024fe00640 1 --2- 172.21.15.2:0/1824021241 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55798b2f3a70 0x55798b2f5ea0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ab10bcdb039ebb5 server_cookie=31b44ddf21f855be in_seq=0 out_seq=0 2024-09-10T08:24:18.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.149+0000 7f0239600640 10 client.30071 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:18.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.150+0000 7f0239600640 1 -- 172.21.15.2:0/1824021241 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55798b243bc0 con 0x55798b2f3a70 2024-09-10T08:24:18.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.150+0000 7f0239600640 10 client.30071 handle_command_reply: tid=0 2024-09-10T08:24:18.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.150+0000 7f0258eb8740 1 client.30071 shutdown 2024-09-10T08:24:18.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.151+0000 7f0258eb8740 1 -- 172.21.15.2:0/1824021241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0c30 msgr2=0x55798b2d89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.151+0000 7f0258eb8740 1 --2- 172.21.15.2:0/1824021241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0c30 0x55798b2d89f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f02500fff90 tx=0x7f023c013b20 comp rx=0 tx=0).stop 2024-09-10T08:24:18.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.151+0000 7f0258eb8740 1 -- 172.21.15.2:0/1824021241 shutdown_connections 2024-09-10T08:24:18.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.151+0000 7f0258eb8740 1 --2- 172.21.15.2:0/1824021241 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55798b2f3a70 0x55798b2f5ea0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.151+0000 7f0258eb8740 1 --2- 172.21.15.2:0/1824021241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 0x55798b2d8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.151+0000 7f0258eb8740 1 --2- 172.21.15.2:0/1824021241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0c30 0x55798b2d89f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.151+0000 7f0258eb8740 1 --2- 172.21.15.2:0/1824021241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0310 0x55798b2d84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.151+0000 7f0239600640 0 client.30071 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:18.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.151+0000 7f0258eb8740 1 -- 172.21.15.2:0/1824021241 >> 172.21.15.2:0/1824021241 conn(0x55798b2cc8a0 msgr2=0x55798b2cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:18.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.151+0000 7f0258eb8740 1 -- 172.21.15.2:0/1824021241 shutdown_connections 2024-09-10T08:24:18.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.151+0000 7f0258eb8740 1 -- 172.21.15.2:0/1824021241 wait complete. 2024-09-10T08:24:18.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.151+0000 7f0258eb8740 20 client.30071 trim_cache size 0 max 16384 2024-09-10T08:24:18.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.171+0000 7f0258eb8740 1 Processor -- start 2024-09-10T08:24:18.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.171+0000 7f0258eb8740 1 -- start start 2024-09-10T08:24:18.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.171+0000 7f0258eb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0310 0x55798b2fb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.171+0000 7f0258eb8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0c30 0x55798b2fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.171+0000 7f0258eb8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 0x55798b2d5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.171+0000 7f0258eb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f02501af540 con 0x55798b2d0310 2024-09-10T08:24:18.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.171+0000 7f0258eb8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f02501ace10 con 0x55798b2d0c30 2024-09-10T08:24:18.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.171+0000 7f0258eb8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f02501aac60 con 0x55798b2db4f0 2024-09-10T08:24:18.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.171+0000 7f024fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0310 0x55798b2fb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.171+0000 7f024fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0310 0x55798b2fb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43528/0 (socket says 172.21.15.2:43528) 2024-09-10T08:24:18.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.171+0000 7f024fe00640 1 -- 172.21.15.2:0/2640488746 learned_addr learned my addr 172.21.15.2:0/2640488746 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:18.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.172+0000 7f024ea00640 1 --2- 172.21.15.2:0/2640488746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0c30 0x55798b2fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.172+0000 7f024fe00640 1 --2- 172.21.15.2:0/2640488746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 0x55798b2d5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.172+0000 7f024fe00640 1 -- 172.21.15.2:0/2640488746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 msgr2=0x55798b2d5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.172+0000 7f024fe00640 1 --2- 172.21.15.2:0/2640488746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 0x55798b2d5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.172+0000 7f024fe00640 1 -- 172.21.15.2:0/2640488746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0c30 msgr2=0x55798b2fbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.172+0000 7f024fe00640 1 --2- 172.21.15.2:0/2640488746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0c30 0x55798b2fbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.172+0000 7f024fe00640 1 -- 172.21.15.2:0/2640488746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798b2cd530 con 0x55798b2d0310 2024-09-10T08:24:18.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.172+0000 7f024ea00640 1 --2- 172.21.15.2:0/2640488746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0c30 0x55798b2fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:18.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.172+0000 7f024fe00640 1 --2- 172.21.15.2:0/2640488746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0310 0x55798b2fb7b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0248000f80 tx=0x7f0248009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa55d35c480fc51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:18.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.173+0000 7f023be00640 1 -- 172.21.15.2:0/2640488746 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248036070 con 0x55798b2d0310 2024-09-10T08:24:18.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.173+0000 7f023be00640 1 -- 172.21.15.2:0/2640488746 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024802b030 con 0x55798b2d0310 2024-09-10T08:24:18.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.173+0000 7f023be00640 1 -- 172.21.15.2:0/2640488746 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248027cc0 con 0x55798b2d0310 2024-09-10T08:24:18.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.173+0000 7f0258eb8740 1 -- 172.21.15.2:0/2640488746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0310 msgr2=0x55798b2fb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.173+0000 7f0258eb8740 1 --2- 172.21.15.2:0/2640488746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0310 0x55798b2fb7b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0248000f80 tx=0x7f0248009ac0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.173+0000 7f0258eb8740 1 -- 172.21.15.2:0/2640488746 shutdown_connections 2024-09-10T08:24:18.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.174+0000 7f0258eb8740 1 --2- 172.21.15.2:0/2640488746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 0x55798b2d5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.174+0000 7f0258eb8740 1 --2- 172.21.15.2:0/2640488746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0c30 0x55798b2fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.174+0000 7f0258eb8740 1 --2- 172.21.15.2:0/2640488746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0310 0x55798b2fb7b0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.174+0000 7f0258eb8740 1 -- 172.21.15.2:0/2640488746 >> 172.21.15.2:0/2640488746 conn(0x55798b2cc8a0 msgr2=0x55798b2d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:18.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.174+0000 7f0258eb8740 1 -- 172.21.15.2:0/2640488746 shutdown_connections 2024-09-10T08:24:18.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.174+0000 7f0258eb8740 1 -- 172.21.15.2:0/2640488746 wait complete. 2024-09-10T08:24:18.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.174+0000 7f0258eb8740 1 -- start start 2024-09-10T08:24:18.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.174+0000 7f0258eb8740 1 Processor -- start 2024-09-10T08:24:18.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.175+0000 7f0258eb8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0310 0x55798b2ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.175+0000 7f0258eb8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0c30 0x55798b2ec470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.175+0000 7f0258eb8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 0x55798b2e8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.175+0000 7f0258eb8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f02501ab2d0 con 0x55798b2d0c30 2024-09-10T08:24:18.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.175+0000 7f0258eb8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f02501ac740 con 0x55798b2d0310 2024-09-10T08:24:18.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.175+0000 7f0258eb8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f02501ab9a0 con 0x55798b2db4f0 2024-09-10T08:24:18.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.175+0000 7f0258eb8740 10 client.0 _ll_register_callbacks cb 0x7f023c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:18.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.175+0000 7f024ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0c30 0x55798b2ec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.175+0000 7f024fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0310 0x55798b2ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.175+0000 7f024ea00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0c30 0x55798b2ec470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43540/0 (socket says 172.21.15.2:43540) 2024-09-10T08:24:18.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.175+0000 7f024ea00640 1 -- 172.21.15.2:0/252621463 learned_addr learned my addr 172.21.15.2:0/252621463 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:18.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.175+0000 7f024fe00640 1 --2- 172.21.15.2:0/252621463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 0x55798b2e8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.176+0000 7f024fe00640 1 -- 172.21.15.2:0/252621463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 msgr2=0x55798b2e8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.176+0000 7f024fe00640 1 --2- 172.21.15.2:0/252621463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 0x55798b2e8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.176+0000 7f024fe00640 1 -- 172.21.15.2:0/252621463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0c30 msgr2=0x55798b2ec470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.176+0000 7f024fe00640 1 --2- 172.21.15.2:0/252621463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0c30 0x55798b2ec470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.176+0000 7f024fe00640 1 -- 172.21.15.2:0/252621463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798b2e9410 con 0x55798b2d0310 2024-09-10T08:24:18.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.176+0000 7f024ea00640 1 --2- 172.21.15.2:0/252621463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0c30 0x55798b2ec470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:18.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.176+0000 7f024fe00640 1 --2- 172.21.15.2:0/252621463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0310 0x55798b2ebf30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0248014040 tx=0x7f0248032040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3783591642cbc6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:18.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.176+0000 7f0239600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:18.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.176+0000 7f0239600640 1 -- 172.21.15.2:0/252621463 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248036040 con 0x55798b2d0310 2024-09-10T08:24:18.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.176+0000 7f0258eb8740 1 -- 172.21.15.2:0/252621463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f024802b030 con 0x55798b2d0310 2024-09-10T08:24:18.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.176+0000 7f0239600640 1 -- 172.21.15.2:0/252621463 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f0248027640 con 0x55798b2d0310 2024-09-10T08:24:18.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.177+0000 7f0239600640 1 -- 172.21.15.2:0/252621463 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0248045050 con 0x55798b2d0310 2024-09-10T08:24:18.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.177+0000 7f0258eb8740 10 client.30077 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:18.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.177+0000 7f0258eb8740 1 -- 172.21.15.2:0/252621463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55798b35f170 con 0x55798b2d0310 2024-09-10T08:24:18.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.177+0000 7f0239600640 1 -- 172.21.15.2:0/252621463 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024802b030 con 0x55798b2d0310 2024-09-10T08:24:18.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.177+0000 7f0239600640 1 -- 172.21.15.2:0/252621463 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f024803c070 con 0x55798b2d0310 2024-09-10T08:24:18.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.177+0000 7f0258eb8740 10 client.30077 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:18.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.177+0000 7f0258eb8740 10 client.30077 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:24:18.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.177+0000 7f0258eb8740 20 client.30077 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:18.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.177+0000 7f0258eb8740 1 --2- 172.21.15.2:0/252621463 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55798b35f540 0x55798b361930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.177+0000 7f0258eb8740 4 client.30077 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4978faaa", "release": true, "await": true}] 2024-09-10T08:24:18.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.177+0000 7f0258eb8740 1 -- 172.21.15.2:0/252621463 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4978faaa", "release": true, "await": true}) -- 0x55798b23e0d0 con 0x55798b35f540 2024-09-10T08:24:18.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.177+0000 7f024ea00640 1 --2- 172.21.15.2:0/252621463 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55798b35f540 0x55798b361930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.178+0000 7f024ea00640 1 --2- 172.21.15.2:0/252621463 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55798b35f540 0x55798b361930 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5232571d88c95b server_cookie=16d7102d27724ebc in_seq=0 out_seq=0 2024-09-10T08:24:18.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.178+0000 7f0239600640 10 client.30077 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:18.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.180+0000 7f0239600640 1 -- 172.21.15.2:0/252621463 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55798b23e0d0 con 0x55798b35f540 2024-09-10T08:24:18.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.180+0000 7f0239600640 10 client.30077 handle_command_reply: tid=0 2024-09-10T08:24:18.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.180+0000 7f0258eb8740 1 client.30077 shutdown 2024-09-10T08:24:18.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.180+0000 7f0258eb8740 1 -- 172.21.15.2:0/252621463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0310 msgr2=0x55798b2ebf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.180+0000 7f0258eb8740 1 --2- 172.21.15.2:0/252621463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0310 0x55798b2ebf30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0248014040 tx=0x7f0248032040 comp rx=0 tx=0).stop 2024-09-10T08:24:18.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.180+0000 7f0258eb8740 1 -- 172.21.15.2:0/252621463 shutdown_connections 2024-09-10T08:24:18.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.180+0000 7f0258eb8740 1 --2- 172.21.15.2:0/252621463 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55798b35f540 0x55798b361930 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.180+0000 7f0258eb8740 1 --2- 172.21.15.2:0/252621463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798b2db4f0 0x55798b2e8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.180+0000 7f0258eb8740 1 --2- 172.21.15.2:0/252621463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55798b2d0c30 0x55798b2ec470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.180+0000 7f0258eb8740 1 --2- 172.21.15.2:0/252621463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55798b2d0310 0x55798b2ebf30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.180+0000 7f0258eb8740 1 -- 172.21.15.2:0/252621463 >> 172.21.15.2:0/252621463 conn(0x55798b2cc8a0 msgr2=0x55798b2d8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:18.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.180+0000 7f0239600640 0 client.30077 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:18.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.180+0000 7f0258eb8740 1 -- 172.21.15.2:0/252621463 shutdown_connections 2024-09-10T08:24:18.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.181+0000 7f0258eb8740 1 -- 172.21.15.2:0/252621463 wait complete. 2024-09-10T08:24:18.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.181+0000 7f0258eb8740 20 client.30077 trim_cache size 0 max 16384 2024-09-10T08:24:18.183 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:24:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:24:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:24:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 60, 2024-09-10T08:24:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-09-10T08:24:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: "4978faaa": { 2024-09-10T08:24:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 60, 2024-09-10T08:24:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-09-10T08:24:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:24:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:24:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:24:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-09-10T08:24:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 139.8, 2024-09-10T08:24:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-09-10T08:24:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-09-10T08:24:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-09-10T08:24:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-09-10T08:24:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-09-10T08:24:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-09-10T08:24:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:18.185 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:24:18.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.182+0000 7f0256400640 1 -- 172.21.15.2:0/518005257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02340a2b80 msgr2=0x7f02340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.182+0000 7f0256400640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02340a2b80 0x7f02340a5040 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0240009230 tx=0x7f02400f2e70 comp rx=0 tx=0).stop 2024-09-10T08:24:18.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.182+0000 7f0256400640 1 -- 172.21.15.2:0/518005257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0250103ff0 msgr2=0x7f025019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.182+0000 7f0256400640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0250103ff0 0x7f025019a600 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0240005940 tx=0x7f02400040c0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.182+0000 7f0256400640 1 -- 172.21.15.2:0/518005257 shutdown_connections 2024-09-10T08:24:18.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.182+0000 7f0256400640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0250109120 0x7f025019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.182+0000 7f0256400640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f02340a2b80 0x7f02340a5040 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.182+0000 7f0256400640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02501049a0 0x7f025019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.182+0000 7f0256400640 1 --2- 172.21.15.2:0/518005257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0250103ff0 0x7f025019a600 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.182+0000 7f0256400640 1 -- 172.21.15.2:0/518005257 >> 172.21.15.2:0/518005257 conn(0x7f02500ffa90 msgr2=0x7f0250100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:18.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.182+0000 7f0256400640 1 -- 172.21.15.2:0/518005257 shutdown_connections 2024-09-10T08:24:18.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:18.182+0000 7f0256400640 1 -- 172.21.15.2:0/518005257 wait complete. 2024-09-10T08:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.693+0000 7f7a53200640 1 -- 172.21.15.2:0/4285481030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a54103b70 msgr2=0x7f7a54103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.693+0000 7f7a53200640 1 --2- 172.21.15.2:0/4285481030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a54103b70 0x7f7a54103f70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009900 tx=0x7f7a4402a410 comp rx=0 tx=0).stop 2024-09-10T08:24:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.693+0000 7f7a53200640 1 -- 172.21.15.2:0/4285481030 shutdown_connections 2024-09-10T08:24:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.693+0000 7f7a53200640 1 --2- 172.21.15.2:0/4285481030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a54105720 0x7f7a5410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.693+0000 7f7a53200640 1 --2- 172.21.15.2:0/4285481030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a54104d60 0x7f7a541051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.693+0000 7f7a53200640 1 --2- 172.21.15.2:0/4285481030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a54103b70 0x7f7a54103f70 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.693+0000 7f7a53200640 1 -- 172.21.15.2:0/4285481030 >> 172.21.15.2:0/4285481030 conn(0x7f7a540ff2f0 msgr2=0x7f7a54101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.693+0000 7f7a53200640 1 -- 172.21.15.2:0/4285481030 shutdown_connections 2024-09-10T08:24:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.693+0000 7f7a53200640 1 -- 172.21.15.2:0/4285481030 wait complete. 2024-09-10T08:24:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.694+0000 7f7a53200640 1 Processor -- start 2024-09-10T08:24:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.694+0000 7f7a53200640 1 -- start start 2024-09-10T08:24:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a53200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a54103b70 0x7f7a54199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a53200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a54104d60 0x7f7a5419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a53200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a54105720 0x7f7a541a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a53200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410f840 con 0x7f7a54104d60 2024-09-10T08:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a53200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410f6c0 con 0x7f7a54105720 2024-09-10T08:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a53200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410f9c0 con 0x7f7a54103b70 2024-09-10T08:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a51400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a54104d60 0x7f7a5419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a51400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a54104d60 0x7f7a5419a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43558/0 (socket says 172.21.15.2:43558) 2024-09-10T08:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a51400640 1 -- 172.21.15.2:0/262410299 learned_addr learned my addr 172.21.15.2:0/262410299 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a51e00640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a54103b70 0x7f7a54199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a52800640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a54105720 0x7f7a541a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a51400640 1 -- 172.21.15.2:0/262410299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a54103b70 msgr2=0x7f7a54199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a51400640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a54103b70 0x7f7a54199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a51400640 1 -- 172.21.15.2:0/262410299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a54105720 msgr2=0x7f7a541a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a51400640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a54105720 0x7f7a541a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.695+0000 7f7a51400640 1 -- 172.21.15.2:0/262410299 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a541a1ab0 con 0x7f7a54104d60 2024-09-10T08:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.696+0000 7f7a51400640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a54104d60 0x7f7a5419a3d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000d950 tx=0x7f7a4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.696+0000 7f7a4aa00640 1 -- 172.21.15.2:0/262410299 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40017070 con 0x7f7a54104d60 2024-09-10T08:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.696+0000 7f7a4aa00640 1 -- 172.21.15.2:0/262410299 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a40009c30 con 0x7f7a54104d60 2024-09-10T08:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.696+0000 7f7a53200640 1 -- 172.21.15.2:0/262410299 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a541a1da0 con 0x7f7a54104d60 2024-09-10T08:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.696+0000 7f7a4aa00640 1 -- 172.21.15.2:0/262410299 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000f6b0 con 0x7f7a54104d60 2024-09-10T08:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.696+0000 7f7a53200640 1 -- 172.21.15.2:0/262410299 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a541a2270 con 0x7f7a54104d60 2024-09-10T08:24:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.699+0000 7f7a5be57740 1 Processor -- start 2024-09-10T08:24:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.699+0000 7f7a5be57740 1 -- start start 2024-09-10T08:24:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.699+0000 7f7a5be57740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accf260 0x562c8accf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.699+0000 7f7a5be57740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accfba0 0x562c8acd4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.699+0000 7f7a5be57740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8acd4c30 0x562c8acd70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.699+0000 7f7a5be57740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562c8ac34100 con 0x562c8accf260 2024-09-10T08:24:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.699+0000 7f7a5be57740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562c8abdc1b0 con 0x562c8acd4c30 2024-09-10T08:24:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.699+0000 7f7a5be57740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562c8ac300c0 con 0x562c8accfba0 2024-09-10T08:24:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.700+0000 7f7a52800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accf260 0x562c8accf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.700+0000 7f7a52800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accf260 0x562c8accf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43574/0 (socket says 172.21.15.2:43574) 2024-09-10T08:24:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.700+0000 7f7a52800640 1 -- 172.21.15.2:0/3596305841 learned_addr learned my addr 172.21.15.2:0/3596305841 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.700+0000 7f7a51e00640 1 --2- 172.21.15.2:0/3596305841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accfba0 0x562c8acd4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.700+0000 7f7a4aa00640 1 -- 172.21.15.2:0/262410299 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7a400109d0 con 0x7f7a54104d60 2024-09-10T08:24:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.700+0000 7f7a52800640 1 -- 172.21.15.2:0/3596305841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accfba0 msgr2=0x562c8acd4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.700+0000 7f7a52800640 1 --2- 172.21.15.2:0/3596305841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accfba0 0x562c8acd4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.700+0000 7f7a52800640 1 -- 172.21.15.2:0/3596305841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8acd4c30 msgr2=0x562c8acd70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.700+0000 7f7a52800640 1 --2- 172.21.15.2:0/3596305841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8acd4c30 0x562c8acd70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.700+0000 7f7a52800640 1 -- 172.21.15.2:0/3596305841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c8a947210 con 0x562c8accf260 2024-09-10T08:24:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.700+0000 7f7a52800640 1 --2- 172.21.15.2:0/3596305841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accf260 0x562c8accf660 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009fc0 tx=0x7f7a4c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5043886dc27e0d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.701+0000 7f7a3f400640 1 -- 172.21.15.2:0/3596305841 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c003b70 con 0x562c8accf260 2024-09-10T08:24:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.701+0000 7f7a3f400640 1 -- 172.21.15.2:0/3596305841 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a4c003d10 con 0x562c8accf260 2024-09-10T08:24:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.701+0000 7f7a3f400640 1 -- 172.21.15.2:0/3596305841 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c011a40 con 0x562c8accf260 2024-09-10T08:24:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.701+0000 7f7a4aa00640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7a380a2c00 0x7f7a380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.701+0000 7f7a5be57740 1 -- 172.21.15.2:0/3596305841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accf260 msgr2=0x562c8accf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.701+0000 7f7a5be57740 1 --2- 172.21.15.2:0/3596305841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accf260 0x562c8accf660 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009fc0 tx=0x7f7a4c010860 comp rx=0 tx=0).stop 2024-09-10T08:24:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.701+0000 7f7a4aa00640 1 -- 172.21.15.2:0/262410299 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7a400cf070 con 0x7f7a54104d60 2024-09-10T08:24:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.701+0000 7f7a5be57740 1 -- 172.21.15.2:0/3596305841 shutdown_connections 2024-09-10T08:24:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.701+0000 7f7a5be57740 1 --2- 172.21.15.2:0/3596305841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8acd4c30 0x562c8acd70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.701+0000 7f7a5be57740 1 --2- 172.21.15.2:0/3596305841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accfba0 0x562c8acd4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.701+0000 7f7a5be57740 1 --2- 172.21.15.2:0/3596305841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accf260 0x562c8accf660 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.701+0000 7f7a5be57740 1 -- 172.21.15.2:0/3596305841 >> 172.21.15.2:0/3596305841 conn(0x562c8acca8d0 msgr2=0x562c8accacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.702+0000 7f7a51e00640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7a380a2c00 0x7f7a380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.702+0000 7f7a5be57740 1 -- 172.21.15.2:0/3596305841 shutdown_connections 2024-09-10T08:24:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.702+0000 7f7a5be57740 1 -- 172.21.15.2:0/3596305841 wait complete. 2024-09-10T08:24:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.702+0000 7f7a5be57740 1 -- start start 2024-09-10T08:24:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.702+0000 7f7a51e00640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7a380a2c00 0x7f7a380a50c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7a4402aa00 tx=0x7f7a440315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.702+0000 7f7a5be57740 1 Processor -- start 2024-09-10T08:24:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a5be57740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accf260 0x562c8acd9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a5be57740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accfba0 0x562c8acd9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a5be57740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8acd4c30 0x562c8ace09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a5be57740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562c8acd7c00 con 0x562c8accfba0 2024-09-10T08:24:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a5be57740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562c8abbd090 con 0x562c8acd4c30 2024-09-10T08:24:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a5be57740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562c8acd7d80 con 0x562c8accf260 2024-09-10T08:24:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a5be57740 10 client.0 _ll_register_callbacks cb 0x562c8ac11f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a51400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accfba0 0x562c8acd9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a51400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accfba0 0x562c8acd9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43584/0 (socket says 172.21.15.2:43584) 2024-09-10T08:24:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a51400640 1 -- 172.21.15.2:0/835427830 learned_addr learned my addr 172.21.15.2:0/835427830 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a51e00640 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accf260 0x562c8acd9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a52800640 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8acd4c30 0x562c8ace09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a51400640 1 -- 172.21.15.2:0/835427830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accf260 msgr2=0x562c8acd9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a51400640 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accf260 0x562c8acd9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a51400640 1 -- 172.21.15.2:0/835427830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8acd4c30 msgr2=0x562c8ace09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a51400640 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8acd4c30 0x562c8ace09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a51400640 1 -- 172.21.15.2:0/835427830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c8aa13620 con 0x562c8accfba0 2024-09-10T08:24:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.703+0000 7f7a52800640 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8acd4c30 0x562c8ace09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.704+0000 7f7a51400640 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accfba0 0x562c8acd9980 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7a4009b1e0 tx=0x7f7a4009b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5699123a5c9493d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.704+0000 7f7a33e00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:24:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.704+0000 7f7a33e00640 1 -- 172.21.15.2:0/835427830 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40020070 con 0x562c8accfba0 2024-09-10T08:24:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.704+0000 7f7a33e00640 1 -- 172.21.15.2:0/835427830 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7a4009f490 con 0x562c8accfba0 2024-09-10T08:24:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.704+0000 7f7a5be57740 1 -- 172.21.15.2:0/835427830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a4c003d10 con 0x562c8accfba0 2024-09-10T08:24:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.704+0000 7f7a33e00640 1 -- 172.21.15.2:0/835427830 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7a4009f690 con 0x562c8accfba0 2024-09-10T08:24:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.704+0000 7f7a5be57740 10 client.20070 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.704+0000 7f7a5be57740 1 -- 172.21.15.2:0/835427830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c8ace3820 con 0x562c8accfba0 2024-09-10T08:24:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.704+0000 7f7a33e00640 1 -- 172.21.15.2:0/835427830 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a4c003d10 con 0x562c8accfba0 2024-09-10T08:24:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.705+0000 7f7a33e00640 1 -- 172.21.15.2:0/835427830 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7a400d1110 con 0x562c8accfba0 2024-09-10T08:24:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.705+0000 7f7a5be57740 10 client.20070 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.705+0000 7f7a5be57740 10 client.20070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.705+0000 7f7a5be57740 20 client.20070 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.705+0000 7f7a5be57740 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562c8ace4ea0 0x562c8ace7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.705+0000 7f7a5be57740 4 client.20070 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.705+0000 7f7a5be57740 1 -- 172.21.15.2:0/835427830 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c8ac41830 con 0x562c8ace4ea0 2024-09-10T08:24:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.705+0000 7f7a51e00640 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562c8ace4ea0 0x562c8ace7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.706+0000 7f7a51e00640 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562c8ace4ea0 0x562c8ace7290 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cba5032a4e6ad75 server_cookie=bd4fb16c00d65c98 in_seq=0 out_seq=0 2024-09-10T08:24:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.706+0000 7f7a33e00640 10 client.20070 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.706+0000 7f7a33e00640 1 -- 172.21.15.2:0/835427830 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c8ac41830 con 0x562c8ace4ea0 2024-09-10T08:24:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.706+0000 7f7a33e00640 10 client.20070 handle_command_reply: tid=0 2024-09-10T08:24:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.707+0000 7f7a5be57740 1 client.20070 shutdown 2024-09-10T08:24:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.707+0000 7f7a5be57740 1 -- 172.21.15.2:0/835427830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accfba0 msgr2=0x562c8acd9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.707+0000 7f7a5be57740 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accfba0 0x562c8acd9980 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7a4009b1e0 tx=0x7f7a4009b0e0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.707+0000 7f7a5be57740 1 -- 172.21.15.2:0/835427830 shutdown_connections 2024-09-10T08:24:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.707+0000 7f7a5be57740 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562c8ace4ea0 0x562c8ace7290 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.707+0000 7f7a5be57740 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8acd4c30 0x562c8ace09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.707+0000 7f7a5be57740 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8accfba0 0x562c8acd9980 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.707+0000 7f7a5be57740 1 --2- 172.21.15.2:0/835427830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accf260 0x562c8acd9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.707+0000 7f7a33e00640 0 client.20070 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.707+0000 7f7a5be57740 1 -- 172.21.15.2:0/835427830 >> 172.21.15.2:0/835427830 conn(0x562c8acca8d0 msgr2=0x562c8accc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.707+0000 7f7a5be57740 1 -- 172.21.15.2:0/835427830 shutdown_connections 2024-09-10T08:24:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.707+0000 7f7a5be57740 1 -- 172.21.15.2:0/835427830 wait complete. 2024-09-10T08:24:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.707+0000 7f7a5be57740 20 client.20070 trim_cache size 0 max 16384 2024-09-10T08:24:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.722+0000 7f7a5be57740 1 Processor -- start 2024-09-10T08:24:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.722+0000 7f7a5be57740 1 -- start start 2024-09-10T08:24:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.722+0000 7f7a5be57740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accf260 0x562c8acc91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.722+0000 7f7a5be57740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8accfba0 0x562c8ace0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.722+0000 7f7a5be57740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8acd4c30 0x562c8acc9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.722+0000 7f7a5be57740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7a541af250 con 0x562c8acd4c30 2024-09-10T08:24:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.722+0000 7f7a5be57740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a541acb20 con 0x562c8accfba0 2024-09-10T08:24:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.722+0000 7f7a5be57740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a541aa970 con 0x562c8accf260 2024-09-10T08:24:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.723+0000 7f7a51e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accf260 0x562c8acc91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.723+0000 7f7a51e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accf260 0x562c8acc91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51330/0 (socket says 172.21.15.2:51330) 2024-09-10T08:24:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.723+0000 7f7a51e00640 1 -- 172.21.15.2:0/695285664 learned_addr learned my addr 172.21.15.2:0/695285664 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.723+0000 7f7a52800640 1 --2- 172.21.15.2:0/695285664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8acd4c30 0x562c8acc9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.723+0000 7f7a51400640 1 --2- 172.21.15.2:0/695285664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8accfba0 0x562c8ace0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.723+0000 7f7a51e00640 1 -- 172.21.15.2:0/695285664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8accfba0 msgr2=0x562c8ace0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.723+0000 7f7a51e00640 1 --2- 172.21.15.2:0/695285664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8accfba0 0x562c8ace0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.723+0000 7f7a51e00640 1 -- 172.21.15.2:0/695285664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8acd4c30 msgr2=0x562c8acc9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.723+0000 7f7a51e00640 1 --2- 172.21.15.2:0/695285664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8acd4c30 0x562c8acc9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.723+0000 7f7a51e00640 1 -- 172.21.15.2:0/695285664 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c8ace78c0 con 0x562c8accf260 2024-09-10T08:24:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.723+0000 7f7a51e00640 1 --2- 172.21.15.2:0/695285664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accf260 0x562c8acc91f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009900 tx=0x7f7a4403e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b678428e48f99cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.724+0000 7f7a3ea00640 1 -- 172.21.15.2:0/695285664 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44035040 con 0x562c8accf260 2024-09-10T08:24:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.724+0000 7f7a3ea00640 1 -- 172.21.15.2:0/695285664 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a44032da0 con 0x562c8accf260 2024-09-10T08:24:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.724+0000 7f7a3ea00640 1 -- 172.21.15.2:0/695285664 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44004ef0 con 0x562c8accf260 2024-09-10T08:24:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.724+0000 7f7a5be57740 1 -- 172.21.15.2:0/695285664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accf260 msgr2=0x562c8acc91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.724+0000 7f7a5be57740 1 --2- 172.21.15.2:0/695285664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accf260 0x562c8acc91f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009900 tx=0x7f7a4403e000 comp rx=0 tx=0).stop 2024-09-10T08:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.724+0000 7f7a5be57740 1 -- 172.21.15.2:0/695285664 shutdown_connections 2024-09-10T08:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.724+0000 7f7a5be57740 1 --2- 172.21.15.2:0/695285664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8acd4c30 0x562c8acc9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.724+0000 7f7a5be57740 1 --2- 172.21.15.2:0/695285664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8accfba0 0x562c8ace0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.724+0000 7f7a5be57740 1 --2- 172.21.15.2:0/695285664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accf260 0x562c8acc91f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.724+0000 7f7a5be57740 1 -- 172.21.15.2:0/695285664 >> 172.21.15.2:0/695285664 conn(0x562c8acca8d0 msgr2=0x562c8acd9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.725+0000 7f7a5be57740 1 -- 172.21.15.2:0/695285664 shutdown_connections 2024-09-10T08:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.725+0000 7f7a5be57740 1 -- 172.21.15.2:0/695285664 wait complete. 2024-09-10T08:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.725+0000 7f7a5be57740 1 -- start start 2024-09-10T08:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.725+0000 7f7a5be57740 1 Processor -- start 2024-09-10T08:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.725+0000 7f7a5be57740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8accf260 0x562c8acddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.725+0000 7f7a5be57740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accfba0 0x562c8acde190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.725+0000 7f7a5be57740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8acd4c30 0x562c8acdf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a5be57740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7a541aafe0 con 0x562c8acd4c30 2024-09-10T08:24:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a5be57740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a541ac450 con 0x562c8accf260 2024-09-10T08:24:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a5be57740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a541ab6b0 con 0x562c8accfba0 2024-09-10T08:24:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a5be57740 10 client.0 _ll_register_callbacks cb 0x7f7a40097850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a51e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8accf260 0x562c8acddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a51e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8accf260 0x562c8acddc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36844/0 (socket says 172.21.15.2:36844) 2024-09-10T08:24:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a51e00640 1 -- 172.21.15.2:0/1306216369 learned_addr learned my addr 172.21.15.2:0/1306216369 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a52800640 1 --2- 172.21.15.2:0/1306216369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8acd4c30 0x562c8acdf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a51400640 1 --2- 172.21.15.2:0/1306216369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accfba0 0x562c8acde190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a51e00640 1 -- 172.21.15.2:0/1306216369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accfba0 msgr2=0x562c8acde190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a51e00640 1 --2- 172.21.15.2:0/1306216369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accfba0 0x562c8acde190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a51e00640 1 -- 172.21.15.2:0/1306216369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8acd4c30 msgr2=0x562c8acdf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a51e00640 1 --2- 172.21.15.2:0/1306216369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8acd4c30 0x562c8acdf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a51e00640 1 -- 172.21.15.2:0/1306216369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c8acdf6a0 con 0x562c8accf260 2024-09-10T08:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a51e00640 1 --2- 172.21.15.2:0/1306216369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8accf260 0x562c8acddc50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7a440314e0 tx=0x7f7a44004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87f1f2241b01fda server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.726+0000 7f7a33e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a33e00640 1 -- 172.21.15.2:0/1306216369 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44035030 con 0x562c8accf260 2024-09-10T08:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a5be57740 1 -- 172.21.15.2:0/1306216369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a44032da0 con 0x562c8accf260 2024-09-10T08:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a33e00640 1 -- 172.21.15.2:0/1306216369 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7a440326c0 con 0x562c8accf260 2024-09-10T08:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a33e00640 1 -- 172.21.15.2:0/1306216369 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7a4406c050 con 0x562c8accf260 2024-09-10T08:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a5be57740 10 client.30092 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a5be57740 1 -- 172.21.15.2:0/1306216369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c8ace0a70 con 0x562c8accf260 2024-09-10T08:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a33e00640 1 -- 172.21.15.2:0/1306216369 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a44032da0 con 0x562c8accf260 2024-09-10T08:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a33e00640 1 -- 172.21.15.2:0/1306216369 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7a44003bb0 con 0x562c8accf260 2024-09-10T08:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a5be57740 10 client.30092 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a5be57740 10 client.30092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a5be57740 20 client.30092 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a5be57740 1 --2- 172.21.15.2:0/1306216369 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562c8ad4ffc0 0x562c8ad523e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a5be57740 4 client.30092 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.727+0000 7f7a5be57740 1 -- 172.21.15.2:0/1306216369 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c8ac3bb80 con 0x562c8ad4ffc0 2024-09-10T08:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.728+0000 7f7a51400640 1 --2- 172.21.15.2:0/1306216369 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562c8ad4ffc0 0x562c8ad523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.728+0000 7f7a51400640 1 --2- 172.21.15.2:0/1306216369 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562c8ad4ffc0 0x562c8ad523e0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a9c021986667cb server_cookie=e9e01491c5502897 in_seq=0 out_seq=0 2024-09-10T08:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.728+0000 7f7a33e00640 10 client.30092 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a33e00640 1 -- 172.21.15.2:0/1306216369 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c8ac3bb80 con 0x562c8ad4ffc0 2024-09-10T08:24:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a33e00640 10 client.30092 handle_command_reply: tid=0 2024-09-10T08:24:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a5be57740 1 client.30092 shutdown 2024-09-10T08:24:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a5be57740 1 -- 172.21.15.2:0/1306216369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8accf260 msgr2=0x562c8acddc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a5be57740 1 --2- 172.21.15.2:0/1306216369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8accf260 0x562c8acddc50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7a440314e0 tx=0x7f7a44004170 comp rx=0 tx=0).stop 2024-09-10T08:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a5be57740 1 -- 172.21.15.2:0/1306216369 shutdown_connections 2024-09-10T08:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a5be57740 1 --2- 172.21.15.2:0/1306216369 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x562c8ad4ffc0 0x562c8ad523e0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a5be57740 1 --2- 172.21.15.2:0/1306216369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562c8acd4c30 0x562c8acdf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a5be57740 1 --2- 172.21.15.2:0/1306216369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c8accfba0 0x562c8acde190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a5be57740 1 --2- 172.21.15.2:0/1306216369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562c8accf260 0x562c8acddc50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a5be57740 1 -- 172.21.15.2:0/1306216369 >> 172.21.15.2:0/1306216369 conn(0x562c8acca8d0 msgr2=0x562c8accb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a33e00640 0 client.30092 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a5be57740 1 -- 172.21.15.2:0/1306216369 shutdown_connections 2024-09-10T08:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a5be57740 1 -- 172.21.15.2:0/1306216369 wait complete. 2024-09-10T08:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.739+0000 7f7a5be57740 20 client.30092 trim_cache size 0 max 16384 2024-09-10T08:24:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:24:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:24:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:24:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.740+0000 7f7a53200640 1 -- 172.21.15.2:0/262410299 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7a380a2c00 msgr2=0x7f7a380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.740+0000 7f7a53200640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7a380a2c00 0x7f7a380a50c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7a4402aa00 tx=0x7f7a440315d0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.740+0000 7f7a53200640 1 -- 172.21.15.2:0/262410299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a54104d60 msgr2=0x7f7a5419a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.740+0000 7f7a53200640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a54104d60 0x7f7a5419a3d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000d950 tx=0x7f7a4000de10 comp rx=0 tx=0).stop 2024-09-10T08:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.740+0000 7f7a53200640 1 -- 172.21.15.2:0/262410299 shutdown_connections 2024-09-10T08:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.740+0000 7f7a53200640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a54105720 0x7f7a541a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.740+0000 7f7a53200640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a54104d60 0x7f7a5419a3d0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.740+0000 7f7a53200640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7a380a2c00 0x7f7a380a50c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.740+0000 7f7a53200640 1 --2- 172.21.15.2:0/262410299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a54103b70 0x7f7a54199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.740+0000 7f7a53200640 1 -- 172.21.15.2:0/262410299 >> 172.21.15.2:0/262410299 conn(0x7f7a540ff2f0 msgr2=0x7f7a541080e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.740+0000 7f7a53200640 1 -- 172.21.15.2:0/262410299 shutdown_connections 2024-09-10T08:24:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:18.740+0000 7f7a53200640 1 -- 172.21.15.2:0/262410299 wait complete. 2024-09-10T08:24:19.138 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:19.168 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4978faaa', total seconds elapsed: 25.1 2024-09-10T08:24:19.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:19 smithi002 ceph-mon[30563]: pgmap v872: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-10T08:24:19.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:19 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:24:19.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.726+0000 7fa5d0400640 1 -- 172.21.15.2:0/2207757075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8074590 msgr2=0x7fa5c8074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:19.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.726+0000 7fa5d0400640 1 --2- 172.21.15.2:0/2207757075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8074590 0x7fa5c8074970 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009900 tx=0x7fa5bc02a480 comp rx=0 tx=0).stop 2024-09-10T08:24:19.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.726+0000 7fa5d0400640 1 -- 172.21.15.2:0/2207757075 shutdown_connections 2024-09-10T08:24:19.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.726+0000 7fa5d0400640 1 --2- 172.21.15.2:0/2207757075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8112b90 0x7fa5c8114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.726+0000 7fa5d0400640 1 --2- 172.21.15.2:0/2207757075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8074eb0 0x7fa5c8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.726+0000 7fa5d0400640 1 --2- 172.21.15.2:0/2207757075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8074590 0x7fa5c8074970 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.726+0000 7fa5d0400640 1 -- 172.21.15.2:0/2207757075 >> 172.21.15.2:0/2207757075 conn(0x7fa5c80fd8c0 msgr2=0x7fa5c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:19.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.726+0000 7fa5d0400640 1 -- 172.21.15.2:0/2207757075 shutdown_connections 2024-09-10T08:24:19.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.726+0000 7fa5d0400640 1 -- 172.21.15.2:0/2207757075 wait complete. 2024-09-10T08:24:19.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.727+0000 7fa5d0400640 1 Processor -- start 2024-09-10T08:24:19.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.728+0000 7fa5d0400640 1 -- start start 2024-09-10T08:24:19.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.728+0000 7fa5d0400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8074590 0x7fa5c81a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.728+0000 7fa5d0400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8074eb0 0x7fa5c81a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.728+0000 7fa5d0400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8112b90 0x7fa5c819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.728+0000 7fa5d0400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81185a0 con 0x7fa5c8074590 2024-09-10T08:24:19.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.728+0000 7fa5d0400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8118420 con 0x7fa5c8074eb0 2024-09-10T08:24:19.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.728+0000 7fa5d0400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8118720 con 0x7fa5c8112b90 2024-09-10T08:24:19.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.728+0000 7fa5cda00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8074590 0x7fa5c81a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.728+0000 7fa5cda00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8074590 0x7fa5c81a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43612/0 (socket says 172.21.15.2:43612) 2024-09-10T08:24:19.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.728+0000 7fa5cda00640 1 -- 172.21.15.2:0/3162230861 learned_addr learned my addr 172.21.15.2:0/3162230861 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:19.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.728+0000 7fa5cd000640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8074eb0 0x7fa5c81a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.729+0000 7fa5ce400640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8112b90 0x7fa5c819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.729+0000 7fa5cda00640 1 -- 172.21.15.2:0/3162230861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8112b90 msgr2=0x7fa5c819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:19.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.729+0000 7fa5cda00640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8112b90 0x7fa5c819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.729+0000 7fa5cda00640 1 -- 172.21.15.2:0/3162230861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8074eb0 msgr2=0x7fa5c81a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:19.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.729+0000 7fa5cda00640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8074eb0 0x7fa5c81a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.729+0000 7fa5cda00640 1 -- 172.21.15.2:0/3162230861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c819efe0 con 0x7fa5c8074590 2024-09-10T08:24:19.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.729+0000 7fa5cd000640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8074eb0 0x7fa5c81a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:19.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.729+0000 7fa5cda00640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8074590 0x7fa5c81a46f0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc004770 tx=0x7fa5bc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:19.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.729+0000 7fa5c6000640 1 -- 172.21.15.2:0/3162230861 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc03f070 con 0x7fa5c8074590 2024-09-10T08:24:19.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.730+0000 7fa5d0400640 1 -- 172.21.15.2:0/3162230861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c819f270 con 0x7fa5c8074590 2024-09-10T08:24:19.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.730+0000 7fa5c6000640 1 -- 172.21.15.2:0/3162230861 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5bc031ca0 con 0x7fa5c8074590 2024-09-10T08:24:19.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.730+0000 7fa5c6000640 1 -- 172.21.15.2:0/3162230861 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc035030 con 0x7fa5c8074590 2024-09-10T08:24:19.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.730+0000 7fa5d0400640 1 -- 172.21.15.2:0/3162230861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c81ab5c0 con 0x7fa5c8074590 2024-09-10T08:24:19.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.733+0000 7fa5d2eaf740 1 Processor -- start 2024-09-10T08:24:19.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.733+0000 7fa5c6000640 1 -- 172.21.15.2:0/3162230861 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa5bc042070 con 0x7fa5c8074590 2024-09-10T08:24:19.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.733+0000 7fa5d2eaf740 1 -- start start 2024-09-10T08:24:19.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.733+0000 7fa5d2eaf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876db90 0x559f6876df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.733+0000 7fa5d2eaf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876e4b0 0x559f68778830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.733+0000 7fa5d2eaf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f6877b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.733+0000 7fa5d2eaf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559f686d43e0 con 0x559f68778d70 2024-09-10T08:24:19.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.733+0000 7fa5d2eaf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559f6867bd80 con 0x559f6876db90 2024-09-10T08:24:19.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.733+0000 7fa5d2eaf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559f686d03a0 con 0x559f6876e4b0 2024-09-10T08:24:19.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.734+0000 7fa5cd000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f6877b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.734+0000 7fa5ce400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876db90 0x559f6876df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.734+0000 7fa5cd000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f6877b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43616/0 (socket says 172.21.15.2:43616) 2024-09-10T08:24:19.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.734+0000 7fa5cd000640 1 -- 172.21.15.2:0/948102628 learned_addr learned my addr 172.21.15.2:0/948102628 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:19.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.734+0000 7fa5cda00640 1 --2- 172.21.15.2:0/948102628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876e4b0 0x559f68778830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.734+0000 7fa5cd000640 1 -- 172.21.15.2:0/948102628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876e4b0 msgr2=0x559f68778830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:19.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.734+0000 7fa5cd000640 1 --2- 172.21.15.2:0/948102628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876e4b0 0x559f68778830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.734+0000 7fa5cd000640 1 -- 172.21.15.2:0/948102628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876db90 msgr2=0x559f6876df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:19.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.734+0000 7fa5cd000640 1 --2- 172.21.15.2:0/948102628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876db90 0x559f6876df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.734+0000 7fa5cd000640 1 -- 172.21.15.2:0/948102628 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6869aae0 con 0x559f68778d70 2024-09-10T08:24:19.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.734+0000 7fa5cd000640 1 --2- 172.21.15.2:0/948102628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f6877b160 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40114f0 tx=0x7fa5b40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad53bacc72cd6398 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:19.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.735+0000 7fa5baa00640 1 -- 172.21.15.2:0/948102628 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4005020 con 0x559f68778d70 2024-09-10T08:24:19.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.735+0000 7fa5c6000640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa5b00a2b80 0x7fa5b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.735+0000 7fa5baa00640 1 -- 172.21.15.2:0/948102628 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b40047a0 con 0x559f68778d70 2024-09-10T08:24:19.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.735+0000 7fa5baa00640 1 -- 172.21.15.2:0/948102628 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400f9a0 con 0x559f68778d70 2024-09-10T08:24:19.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.735+0000 7fa5c6000640 1 -- 172.21.15.2:0/3162230861 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa5bc0efa50 con 0x7fa5c8074590 2024-09-10T08:24:19.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.735+0000 7fa5ce400640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa5b00a2b80 0x7fa5b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.735+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/948102628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 msgr2=0x559f6877b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:19.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.735+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/948102628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f6877b160 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40114f0 tx=0x7fa5b40119b0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.736+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/948102628 shutdown_connections 2024-09-10T08:24:19.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.736+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/948102628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f6877b160 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.736+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/948102628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876e4b0 0x559f68778830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.736+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/948102628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876db90 0x559f6876df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.736+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/948102628 >> 172.21.15.2:0/948102628 conn(0x559f6876a120 msgr2=0x559f6876a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:19.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.736+0000 7fa5ce400640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa5b00a2b80 0x7fa5b00a5040 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8076600 tx=0x7fa5c0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:19.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.736+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/948102628 shutdown_connections 2024-09-10T08:24:19.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.736+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/948102628 wait complete. 2024-09-10T08:24:19.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.736+0000 7fa5d2eaf740 1 -- start start 2024-09-10T08:24:19.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.737+0000 7fa5d2eaf740 1 Processor -- start 2024-09-10T08:24:19.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.737+0000 7fa5d2eaf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876db90 0x559f68775b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.737+0000 7fa5d2eaf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f687760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.737+0000 7fa5d2eaf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f68776600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.737+0000 7fa5d2eaf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559f6877ba30 con 0x559f68778d70 2024-09-10T08:24:19.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.737+0000 7fa5d2eaf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559f6868f0f0 con 0x559f6876e4b0 2024-09-10T08:24:19.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.737+0000 7fa5d2eaf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559f6877bbb0 con 0x559f6876db90 2024-09-10T08:24:19.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.737+0000 7fa5d2eaf740 10 client.0 _ll_register_callbacks cb 0x559f686809e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:19.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.738+0000 7fa5cda00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876db90 0x559f68775b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.738+0000 7fa5ce400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f68776600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.738+0000 7fa5cd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f687760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.738+0000 7fa5cda00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876db90 0x559f68775b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51382/0 (socket says 172.21.15.2:51382) 2024-09-10T08:24:19.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.738+0000 7fa5cda00640 1 -- 172.21.15.2:0/2561276026 learned_addr learned my addr 172.21.15.2:0/2561276026 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:19.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.738+0000 7fa5ce400640 1 -- 172.21.15.2:0/2561276026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876db90 msgr2=0x559f68775b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:19.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.738+0000 7fa5ce400640 1 --2- 172.21.15.2:0/2561276026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876db90 0x559f68775b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.738+0000 7fa5ce400640 1 -- 172.21.15.2:0/2561276026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 msgr2=0x559f687760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:19.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.738+0000 7fa5ce400640 1 --2- 172.21.15.2:0/2561276026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f687760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.738+0000 7fa5ce400640 1 -- 172.21.15.2:0/2561276026 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f684453c0 con 0x559f68778d70 2024-09-10T08:24:19.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.738+0000 7fa5ce400640 1 --2- 172.21.15.2:0/2561276026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f68776600 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0005fd0 tx=0x7fa5c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1f447a2116e699 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:19.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.738+0000 7fa5af400640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:24:19.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.739+0000 7fa5af400640 1 -- 172.21.15.2:0/2561276026 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c000f810 con 0x559f68778d70 2024-09-10T08:24:19.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.739+0000 7fa5af400640 1 -- 172.21.15.2:0/2561276026 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa5c000f9b0 con 0x559f68778d70 2024-09-10T08:24:19.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.739+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2561276026 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b40047a0 con 0x559f68778d70 2024-09-10T08:24:19.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.739+0000 7fa5af400640 1 -- 172.21.15.2:0/2561276026 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa5c000fbb0 con 0x559f68778d70 2024-09-10T08:24:19.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.739+0000 7fa5d2eaf740 10 client.20088 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:19.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.739+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2561276026 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f68787690 con 0x559f68778d70 2024-09-10T08:24:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.739+0000 7fa5af400640 1 -- 172.21.15.2:0/2561276026 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b40047a0 con 0x559f68778d70 2024-09-10T08:24:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.739+0000 7fa5af400640 1 -- 172.21.15.2:0/2561276026 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa5c0026070 con 0x559f68778d70 2024-09-10T08:24:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.740+0000 7fa5d2eaf740 10 client.20088 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.740+0000 7fa5d2eaf740 10 client.20088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.740+0000 7fa5d2eaf740 20 client.20088 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.740+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/2561276026 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f68788b50 0x559f6878afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.740+0000 7fa5d2eaf740 4 client.20088 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.740+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2561276026 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f686e1950 con 0x559f68788b50 2024-09-10T08:24:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.740+0000 7fa5cda00640 1 --2- 172.21.15.2:0/2561276026 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f68788b50 0x559f6878afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.740+0000 7fa5cda00640 1 --2- 172.21.15.2:0/2561276026 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f68788b50 0x559f6878afc0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=943b1846168339d3 server_cookie=7c0347af86a34eae in_seq=0 out_seq=0 2024-09-10T08:24:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.740+0000 7fa5af400640 10 client.20088 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.741+0000 7fa5af400640 1 -- 172.21.15.2:0/2561276026 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f686e1950 con 0x559f68788b50 2024-09-10T08:24:19.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.741+0000 7fa5af400640 10 client.20088 handle_command_reply: tid=0 2024-09-10T08:24:19.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.741+0000 7fa5d2eaf740 1 client.20088 shutdown 2024-09-10T08:24:19.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.741+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2561276026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 msgr2=0x559f68776600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:19.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.741+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/2561276026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f68776600 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0005fd0 tx=0x7fa5c0004ab0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.741+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2561276026 shutdown_connections 2024-09-10T08:24:19.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.741+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/2561276026 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f68788b50 0x559f6878afc0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.741+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/2561276026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f68776600 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.741+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/2561276026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f687760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.741+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/2561276026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876db90 0x559f68775b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.741+0000 7fa5af400640 0 client.20088 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:19.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.742+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2561276026 >> 172.21.15.2:0/2561276026 conn(0x559f6876a120 msgr2=0x559f6876fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:19.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.742+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2561276026 shutdown_connections 2024-09-10T08:24:19.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.742+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2561276026 wait complete. 2024-09-10T08:24:19.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.742+0000 7fa5d2eaf740 20 client.20088 trim_cache size 0 max 16384 2024-09-10T08:24:19.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.760+0000 7fa5d2eaf740 1 Processor -- start 2024-09-10T08:24:19.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.760+0000 7fa5d2eaf740 1 -- start start 2024-09-10T08:24:19.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5d2eaf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6876db90 0x559f68776250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5d2eaf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f687908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5d2eaf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f68778d70 0x559f68790e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5d2eaf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81b7da0 con 0x559f6876db90 2024-09-10T08:24:19.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5d2eaf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81b5670 con 0x559f6876e4b0 2024-09-10T08:24:19.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5d2eaf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81b34c0 con 0x559f68778d70 2024-09-10T08:24:19.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5cda00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6876db90 0x559f68776250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5cd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f687908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5cd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f687908e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36908/0 (socket says 172.21.15.2:36908) 2024-09-10T08:24:19.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5cd000640 1 -- 172.21.15.2:0/3316757916 learned_addr learned my addr 172.21.15.2:0/3316757916 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:19.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5ce400640 1 --2- 172.21.15.2:0/3316757916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f68778d70 0x559f68790e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5cd000640 1 -- 172.21.15.2:0/3316757916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f68778d70 msgr2=0x559f68790e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:19.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5cd000640 1 --2- 172.21.15.2:0/3316757916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f68778d70 0x559f68790e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5cd000640 1 -- 172.21.15.2:0/3316757916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6876db90 msgr2=0x559f68776250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:19.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.761+0000 7fa5cd000640 1 --2- 172.21.15.2:0/3316757916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6876db90 0x559f68776250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.762+0000 7fa5cd000640 1 -- 172.21.15.2:0/3316757916 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6878b640 con 0x559f6876e4b0 2024-09-10T08:24:19.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.762+0000 7fa5ce400640 1 --2- 172.21.15.2:0/3316757916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f68778d70 0x559f68790e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:19.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.762+0000 7fa5cd000640 1 --2- 172.21.15.2:0/3316757916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f687908e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4011eb0 tx=0x7fa5b4011460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10dd3a8a14e7dcd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:19.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.762+0000 7fa5ba000640 1 -- 172.21.15.2:0/3316757916 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4005190 con 0x559f6876e4b0 2024-09-10T08:24:19.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.762+0000 7fa5ba000640 1 -- 172.21.15.2:0/3316757916 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b400fb90 con 0x559f6876e4b0 2024-09-10T08:24:19.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.762+0000 7fa5ba000640 1 -- 172.21.15.2:0/3316757916 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4003970 con 0x559f6876e4b0 2024-09-10T08:24:19.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.763+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/3316757916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 msgr2=0x559f687908e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:19.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.763+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/3316757916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f687908e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4011eb0 tx=0x7fa5b4011460 comp rx=0 tx=0).stop 2024-09-10T08:24:19.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.763+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/3316757916 shutdown_connections 2024-09-10T08:24:19.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.763+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/3316757916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f68778d70 0x559f68790e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.763+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/3316757916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f687908e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.763+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/3316757916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f6876db90 0x559f68776250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.763+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/3316757916 >> 172.21.15.2:0/3316757916 conn(0x559f6876a120 msgr2=0x559f68775b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:19.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.763+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/3316757916 shutdown_connections 2024-09-10T08:24:19.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.763+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/3316757916 wait complete. 2024-09-10T08:24:19.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.764+0000 7fa5d2eaf740 1 -- start start 2024-09-10T08:24:19.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.764+0000 7fa5d2eaf740 1 Processor -- start 2024-09-10T08:24:19.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.764+0000 7fa5d2eaf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876db90 0x559f68780e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.764+0000 7fa5d2eaf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f68781380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.764+0000 7fa5d2eaf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f6877bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.764+0000 7fa5d2eaf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81b3b30 con 0x559f68778d70 2024-09-10T08:24:19.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.764+0000 7fa5d2eaf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81b4fa0 con 0x559f6876e4b0 2024-09-10T08:24:19.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.764+0000 7fa5d2eaf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c81b4200 con 0x559f6876db90 2024-09-10T08:24:19.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.764+0000 7fa5d2eaf740 10 client.0 _ll_register_callbacks cb 0x7fa5c0003e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:19.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.765+0000 7fa5ce400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f6877bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.765+0000 7fa5cd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f68781380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.765+0000 7fa5ce400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f6877bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43654/0 (socket says 172.21.15.2:43654) 2024-09-10T08:24:19.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.765+0000 7fa5ce400640 1 -- 172.21.15.2:0/2589980670 learned_addr learned my addr 172.21.15.2:0/2589980670 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:19.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.765+0000 7fa5cda00640 1 --2- 172.21.15.2:0/2589980670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876db90 0x559f68780e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.765+0000 7fa5cd000640 1 -- 172.21.15.2:0/2589980670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876db90 msgr2=0x559f68780e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:19.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.765+0000 7fa5cd000640 1 --2- 172.21.15.2:0/2589980670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876db90 0x559f68780e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.765+0000 7fa5cd000640 1 -- 172.21.15.2:0/2589980670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 msgr2=0x559f6877bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:19.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.765+0000 7fa5cd000640 1 --2- 172.21.15.2:0/2589980670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f6877bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.765+0000 7fa5cd000640 1 -- 172.21.15.2:0/2589980670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6877c3b0 con 0x559f6876e4b0 2024-09-10T08:24:19.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.765+0000 7fa5cd000640 1 --2- 172.21.15.2:0/2589980670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f68781380 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x559f6876b470 tx=0x7fa5b4005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e644588b8ac92f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:19.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.765+0000 7fa5af400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:19.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.766+0000 7fa5af400640 1 -- 172.21.15.2:0/2589980670 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4038060 con 0x559f6876e4b0 2024-09-10T08:24:19.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.766+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2589980670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b400fb90 con 0x559f6876e4b0 2024-09-10T08:24:19.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.766+0000 7fa5af400640 1 -- 172.21.15.2:0/2589980670 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fa5b400f830 con 0x559f6876e4b0 2024-09-10T08:24:19.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.766+0000 7fa5af400640 1 -- 172.21.15.2:0/2589980670 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa5b403f050 con 0x559f6876e4b0 2024-09-10T08:24:19.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.766+0000 7fa5d2eaf740 10 client.30110 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:19.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.766+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2589980670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f687f4140 con 0x559f6876e4b0 2024-09-10T08:24:19.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.766+0000 7fa5af400640 1 -- 172.21.15.2:0/2589980670 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b400fb90 con 0x559f6876e4b0 2024-09-10T08:24:19.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.767+0000 7fa5af400640 1 -- 172.21.15.2:0/2589980670 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa5b4033580 con 0x559f6876e4b0 2024-09-10T08:24:19.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.767+0000 7fa5d2eaf740 10 client.30110 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:19.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.767+0000 7fa5d2eaf740 10 client.30110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.767+0000 7fa5d2eaf740 20 client.30110 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.767+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/2589980670 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f687f45f0 0x559f687f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.767+0000 7fa5d2eaf740 4 client.30110 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:24:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.767+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2589980670 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f686dbe60 con 0x559f687f45f0 2024-09-10T08:24:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.767+0000 7fa5cda00640 1 --2- 172.21.15.2:0/2589980670 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f687f45f0 0x559f687f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.767+0000 7fa5cda00640 1 --2- 172.21.15.2:0/2589980670 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f687f45f0 0x559f687f6a10 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a2bd9cd4121dcdf server_cookie=b0e5c7ca37aae108 in_seq=0 out_seq=0 2024-09-10T08:24:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.767+0000 7fa5af400640 10 client.30110 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5af400640 1 -- 172.21.15.2:0/2589980670 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559f686dbe60 con 0x559f687f45f0 2024-09-10T08:24:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5af400640 10 client.30110 handle_command_reply: tid=0 2024-09-10T08:24:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5d2eaf740 1 client.30110 shutdown 2024-09-10T08:24:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2589980670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 msgr2=0x559f68781380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/2589980670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f68781380 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x559f6876b470 tx=0x7fa5b4005990 comp rx=0 tx=0).stop 2024-09-10T08:24:19.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2589980670 shutdown_connections 2024-09-10T08:24:19.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/2589980670 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x559f687f45f0 0x559f687f6a10 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/2589980670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559f68778d70 0x559f6877bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/2589980670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f6876e4b0 0x559f68781380 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5d2eaf740 1 --2- 172.21.15.2:0/2589980670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f6876db90 0x559f68780e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5af400640 0 client.30110 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:19.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2589980670 >> 172.21.15.2:0/2589980670 conn(0x559f6876a120 msgr2=0x559f687691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:19.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2589980670 shutdown_connections 2024-09-10T08:24:19.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5d2eaf740 1 -- 172.21.15.2:0/2589980670 wait complete. 2024-09-10T08:24:19.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.768+0000 7fa5d2eaf740 20 client.30110 trim_cache size 0 max 16384 2024-09-10T08:24:19.788 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:24:19.788 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:24:19.788 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:24:19.788 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:24:19.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.769+0000 7fa5d0400640 1 -- 172.21.15.2:0/3162230861 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa5b00a2b80 msgr2=0x7fa5b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:19.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.769+0000 7fa5d0400640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa5b00a2b80 0x7fa5b00a5040 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8076600 tx=0x7fa5c0009500 comp rx=0 tx=0).stop 2024-09-10T08:24:19.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.769+0000 7fa5d0400640 1 -- 172.21.15.2:0/3162230861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8074590 msgr2=0x7fa5c81a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:19.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.769+0000 7fa5d0400640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8074590 0x7fa5c81a46f0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc004770 tx=0x7fa5bc003c10 comp rx=0 tx=0).stop 2024-09-10T08:24:19.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.769+0000 7fa5d0400640 1 -- 172.21.15.2:0/3162230861 shutdown_connections 2024-09-10T08:24:19.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.769+0000 7fa5d0400640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fa5b00a2b80 0x7fa5b00a5040 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.769+0000 7fa5d0400640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8112b90 0x7fa5c819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.769+0000 7fa5d0400640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c8074eb0 0x7fa5c81a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.769+0000 7fa5d0400640 1 --2- 172.21.15.2:0/3162230861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5c8074590 0x7fa5c81a46f0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:19.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.769+0000 7fa5d0400640 1 -- 172.21.15.2:0/3162230861 >> 172.21.15.2:0/3162230861 conn(0x7fa5c80fd8c0 msgr2=0x7fa5c80fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:19.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.769+0000 7fa5d0400640 1 -- 172.21.15.2:0/3162230861 shutdown_connections 2024-09-10T08:24:19.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:19.769+0000 7fa5d0400640 1 -- 172.21.15.2:0/3162230861 wait complete. 2024-09-10T08:24:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:19 smithi155 ceph-mon[35405]: pgmap v872: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-10T08:24:19.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:19 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:24:19.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:19 smithi107 ceph-mon[35048]: pgmap v872: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-10T08:24:19.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:19 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:24:20.697 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:24:20.698 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:24:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:21.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.662+0000 7f7f7f200640 1 -- 172.21.15.2:0/2762425455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f78103ff0 msgr2=0x7f7f781043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:21.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.662+0000 7f7f7f200640 1 --2- 172.21.15.2:0/2762425455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f78103ff0 0x7f7f781043d0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009900 tx=0x7f7f6802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:24:21.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.662+0000 7f7f7f200640 1 -- 172.21.15.2:0/2762425455 shutdown_connections 2024-09-10T08:24:21.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.662+0000 7f7f7f200640 1 --2- 172.21.15.2:0/2762425455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f78109120 0x7f7f78110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:21.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.662+0000 7f7f7f200640 1 --2- 172.21.15.2:0/2762425455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f781049a0 0x7f7f781089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:21.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.662+0000 7f7f7f200640 1 --2- 172.21.15.2:0/2762425455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f78103ff0 0x7f7f781043d0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:21.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.662+0000 7f7f7f200640 1 -- 172.21.15.2:0/2762425455 >> 172.21.15.2:0/2762425455 conn(0x7f7f780ffa90 msgr2=0x7f7f78101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:21.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.662+0000 7f7f7f200640 1 -- 172.21.15.2:0/2762425455 shutdown_connections 2024-09-10T08:24:21.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.662+0000 7f7f7f200640 1 -- 172.21.15.2:0/2762425455 wait complete. 2024-09-10T08:24:21.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.663+0000 7f7f7f200640 1 Processor -- start 2024-09-10T08:24:21.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.663+0000 7f7f7f200640 1 -- start start 2024-09-10T08:24:21.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f7f200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f78103ff0 0x7f7f7819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f7f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f781049a0 0x7f7f7819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f7f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f78109120 0x7f7f7819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f7f200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7f781141a0 con 0x7f7f78103ff0 2024-09-10T08:24:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f7f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7f78114020 con 0x7f7f78109120 2024-09-10T08:24:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f7f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f78114320 con 0x7f7f781049a0 2024-09-10T08:24:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f77e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f78103ff0 0x7f7f7819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f77e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f78103ff0 0x7f7f7819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43684/0 (socket says 172.21.15.2:43684) 2024-09-10T08:24:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f77e00640 1 -- 172.21.15.2:0/1927951540 learned_addr learned my addr 172.21.15.2:0/1927951540 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f6fe00640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f781049a0 0x7f7f7819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f7d200640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f78109120 0x7f7f7819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f77e00640 1 -- 172.21.15.2:0/1927951540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f781049a0 msgr2=0x7f7f7819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f77e00640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f781049a0 0x7f7f7819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f77e00640 1 -- 172.21.15.2:0/1927951540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f78109120 msgr2=0x7f7f7819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:21.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f77e00640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f78109120 0x7f7f7819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:21.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.664+0000 7f7f77e00640 1 -- 172.21.15.2:0/1927951540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7819f5d0 con 0x7f7f78103ff0 2024-09-10T08:24:21.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.665+0000 7f7f77e00640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f78103ff0 0x7f7f7819a620 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7f68004770 tx=0x7f7f68003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:21.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.665+0000 7f7f75600640 1 -- 172.21.15.2:0/1927951540 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6803f070 con 0x7f7f78103ff0 2024-09-10T08:24:21.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.665+0000 7f7f75600640 1 -- 172.21.15.2:0/1927951540 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f68031ca0 con 0x7f7f78103ff0 2024-09-10T08:24:21.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.665+0000 7f7f7f200640 1 -- 172.21.15.2:0/1927951540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7819f860 con 0x7f7f78103ff0 2024-09-10T08:24:21.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.665+0000 7f7f75600640 1 -- 172.21.15.2:0/1927951540 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68035030 con 0x7f7f78103ff0 2024-09-10T08:24:21.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.665+0000 7f7f7f200640 1 -- 172.21.15.2:0/1927951540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f781a71a0 con 0x7f7f78103ff0 2024-09-10T08:24:21.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.666+0000 7f7f7f200640 1 -- 172.21.15.2:0/1927951540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f781054d0 con 0x7f7f78103ff0 2024-09-10T08:24:21.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.668+0000 7f7f75600640 1 -- 172.21.15.2:0/1927951540 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7f68042070 con 0x7f7f78103ff0 2024-09-10T08:24:21.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.669+0000 7f7f75600640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f5c0a2b80 0x7f7f5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:21.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.670+0000 7f7f75600640 1 -- 172.21.15.2:0/1927951540 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7f680f00c0 con 0x7f7f78103ff0 2024-09-10T08:24:21.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.670+0000 7f7f6fe00640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f5c0a2b80 0x7f7f5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:21.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.670+0000 7f7f6fe00640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f5c0a2b80 0x7f7f5c0a5040 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7f600095f0 tx=0x7f7f60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:21.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.672+0000 7f7f75600640 1 -- 172.21.15.2:0/1927951540 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f680bcc90 con 0x7f7f78103ff0 2024-09-10T08:24:21.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:21 smithi155 ceph-mon[35405]: pgmap v873: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:24:21.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.909+0000 7f7f7f200640 1 -- 172.21.15.2:0/1927951540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f781043d0 con 0x7f7f78103ff0 2024-09-10T08:24:21.910 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:21 smithi002 ceph-mon[30563]: pgmap v873: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:24:21.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.910+0000 7f7f75600640 1 -- 172.21.15.2:0/1927951540 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f7f680bc590 con 0x7f7f78103ff0 2024-09-10T08:24:21.911 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:24:21.911 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:24:21.911 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:24:21.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.913+0000 7f7f7f200640 1 -- 172.21.15.2:0/1927951540 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f5c0a2b80 msgr2=0x7f7f5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:21.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.913+0000 7f7f7f200640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f5c0a2b80 0x7f7f5c0a5040 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7f600095f0 tx=0x7f7f60005f90 comp rx=0 tx=0).stop 2024-09-10T08:24:21.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.913+0000 7f7f7f200640 1 -- 172.21.15.2:0/1927951540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f78103ff0 msgr2=0x7f7f7819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:21.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.913+0000 7f7f7f200640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f78103ff0 0x7f7f7819a620 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7f68004770 tx=0x7f7f68003c10 comp rx=0 tx=0).stop 2024-09-10T08:24:21.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.913+0000 7f7f7f200640 1 -- 172.21.15.2:0/1927951540 shutdown_connections 2024-09-10T08:24:21.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.913+0000 7f7f7f200640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f78109120 0x7f7f7819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:21.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.913+0000 7f7f7f200640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7f5c0a2b80 0x7f7f5c0a5040 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:21.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.913+0000 7f7f7f200640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f781049a0 0x7f7f7819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:21.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.913+0000 7f7f7f200640 1 --2- 172.21.15.2:0/1927951540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f78103ff0 0x7f7f7819a620 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:21.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.913+0000 7f7f7f200640 1 -- 172.21.15.2:0/1927951540 >> 172.21.15.2:0/1927951540 conn(0x7f7f780ffa90 msgr2=0x7f7f78100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:21.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.913+0000 7f7f7f200640 1 -- 172.21.15.2:0/1927951540 shutdown_connections 2024-09-10T08:24:21.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:21.913+0000 7f7f7f200640 1 -- 172.21.15.2:0/1927951540 wait complete. 2024-09-10T08:24:21.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:21 smithi107 ceph-mon[35048]: pgmap v873: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:24:22.815 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:24:22.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:22 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1927951540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:22.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:22 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1927951540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:23.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:22 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1927951540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:23.194 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:23.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.833+0000 7f2a9ac00640 1 -- 172.21.15.2:0/2910670316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a941049c0 msgr2=0x7f2a94108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:23.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.833+0000 7f2a9ac00640 1 --2- 172.21.15.2:0/2910670316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a941049c0 0x7f2a94108a10 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009900 tx=0x7f2a7c02a410 comp rx=0 tx=0).stop 2024-09-10T08:24:23.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.833+0000 7f2a9ac00640 1 -- 172.21.15.2:0/2910670316 shutdown_connections 2024-09-10T08:24:23.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.833+0000 7f2a9ac00640 1 --2- 172.21.15.2:0/2910670316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a94109140 0x7f2a94110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.833+0000 7f2a9ac00640 1 --2- 172.21.15.2:0/2910670316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a941049c0 0x7f2a94108a10 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.833+0000 7f2a9ac00640 1 --2- 172.21.15.2:0/2910670316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a94104010 0x7f2a941043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.833+0000 7f2a9ac00640 1 -- 172.21.15.2:0/2910670316 >> 172.21.15.2:0/2910670316 conn(0x7f2a940ffa50 msgr2=0x7f2a94101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:23.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.833+0000 7f2a9ac00640 1 -- 172.21.15.2:0/2910670316 shutdown_connections 2024-09-10T08:24:23.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.833+0000 7f2a9ac00640 1 -- 172.21.15.2:0/2910670316 wait complete. 2024-09-10T08:24:23.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.834+0000 7f2a9ac00640 1 Processor -- start 2024-09-10T08:24:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.834+0000 7f2a9ac00640 1 -- start start 2024-09-10T08:24:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a9ac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a94104010 0x7f2a9419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a9ac00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a941049c0 0x7f2a9419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a9ac00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a94109140 0x7f2a9419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a9ac00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2a94114140 con 0x7f2a941049c0 2024-09-10T08:24:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a9ac00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2a94113fc0 con 0x7f2a94109140 2024-09-10T08:24:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a9ac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941142c0 con 0x7f2a94104010 2024-09-10T08:24:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a8be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a941049c0 0x7f2a9419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a8be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a941049c0 0x7f2a9419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43702/0 (socket says 172.21.15.2:43702) 2024-09-10T08:24:23.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a8be00640 1 -- 172.21.15.2:0/3512127748 learned_addr learned my addr 172.21.15.2:0/3512127748 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:23.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a93e00640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a94104010 0x7f2a9419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a98c00640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a94109140 0x7f2a9419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a8be00640 1 -- 172.21.15.2:0/3512127748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a94104010 msgr2=0x7f2a9419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:23.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a8be00640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a94104010 0x7f2a9419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a8be00640 1 -- 172.21.15.2:0/3512127748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a94109140 msgr2=0x7f2a9419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:23.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a8be00640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a94109140 0x7f2a9419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.835+0000 7f2a8be00640 1 -- 172.21.15.2:0/3512127748 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9419f660 con 0x7f2a941049c0 2024-09-10T08:24:23.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.836+0000 7f2a8be00640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a941049c0 0x7f2a9419abf0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c006010 tx=0x7f2a7c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:23.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.836+0000 7f2a91600640 1 -- 172.21.15.2:0/3512127748 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c03f030 con 0x7f2a941049c0 2024-09-10T08:24:23.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.836+0000 7f2a9ac00640 1 -- 172.21.15.2:0/3512127748 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9419f8f0 con 0x7f2a941049c0 2024-09-10T08:24:23.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.836+0000 7f2a91600640 1 -- 172.21.15.2:0/3512127748 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a7c004440 con 0x7f2a941049c0 2024-09-10T08:24:23.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.836+0000 7f2a9ac00640 1 -- 172.21.15.2:0/3512127748 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a941a7230 con 0x7f2a941049c0 2024-09-10T08:24:23.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.837+0000 7f2a91600640 1 -- 172.21.15.2:0/3512127748 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c049060 con 0x7f2a941049c0 2024-09-10T08:24:23.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.840+0000 7f2a91600640 1 -- 172.21.15.2:0/3512127748 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2a7c042030 con 0x7f2a941049c0 2024-09-10T08:24:23.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.840+0000 7f2a9d6a4740 1 Processor -- start 2024-09-10T08:24:23.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.840+0000 7f2a9d6a4740 1 -- start start 2024-09-10T08:24:23.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.840+0000 7f2a9d6a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 0x562fe1def050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.840+0000 7f2a9d6a4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1def590 0x562fe1df9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.840+0000 7f2a9d6a4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 0x562fe1dfc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.840+0000 7f2a9d6a4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562fe1d553e0 con 0x562fe1def590 2024-09-10T08:24:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.840+0000 7f2a9d6a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562fe1cfcd80 con 0x562fe1deec70 2024-09-10T08:24:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.840+0000 7f2a9d6a4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562fe1d513a0 con 0x562fe1df9e50 2024-09-10T08:24:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.841+0000 7f2a93e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1def590 0x562fe1df9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.841+0000 7f2a98c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 0x562fe1def050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.841+0000 7f2a98c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 0x562fe1def050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36970/0 (socket says 172.21.15.2:36970) 2024-09-10T08:24:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.841+0000 7f2a98c00640 1 -- 172.21.15.2:0/705179600 learned_addr learned my addr 172.21.15.2:0/705179600 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.841+0000 7f2a8be00640 1 --2- 172.21.15.2:0/705179600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 0x562fe1dfc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.841+0000 7f2a98c00640 1 -- 172.21.15.2:0/705179600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 msgr2=0x562fe1dfc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.841+0000 7f2a98c00640 1 --2- 172.21.15.2:0/705179600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 0x562fe1dfc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.841+0000 7f2a98c00640 1 -- 172.21.15.2:0/705179600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1def590 msgr2=0x562fe1df9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.841+0000 7f2a98c00640 1 --2- 172.21.15.2:0/705179600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1def590 0x562fe1df9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.841+0000 7f2a98c00640 1 -- 172.21.15.2:0/705179600 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe1d1bae0 con 0x562fe1deec70 2024-09-10T08:24:23.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.841+0000 7f2a98c00640 1 --2- 172.21.15.2:0/705179600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 0x562fe1def050 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009f50 tx=0x7f2a8c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4ad3ba0bf9a1a98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:23.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.841+0000 7f2a91600640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2a780a2b80 0x7f2a780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.842+0000 7f2a91600640 1 -- 172.21.15.2:0/3512127748 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2a7c0efb60 con 0x7f2a941049c0 2024-09-10T08:24:23.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.842+0000 7f2a93e00640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2a780a2b80 0x7f2a780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.842+0000 7f2a89200640 1 -- 172.21.15.2:0/705179600 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c004d20 con 0x562fe1deec70 2024-09-10T08:24:23.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.842+0000 7f2a89200640 1 -- 172.21.15.2:0/705179600 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c004050 con 0x562fe1deec70 2024-09-10T08:24:23.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.842+0000 7f2a89200640 1 -- 172.21.15.2:0/705179600 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c005280 con 0x562fe1deec70 2024-09-10T08:24:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.843+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/705179600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 msgr2=0x562fe1def050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.843+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/705179600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 0x562fe1def050 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009f50 tx=0x7f2a8c0118d0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.843+0000 7f2a93e00640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2a780a2b80 0x7f2a780a5040 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x562fe1dec060 tx=0x7f2a84005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.843+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/705179600 shutdown_connections 2024-09-10T08:24:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.843+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/705179600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 0x562fe1dfc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.843+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/705179600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1def590 0x562fe1df9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.843+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/705179600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 0x562fe1def050 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.843+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/705179600 >> 172.21.15.2:0/705179600 conn(0x562fe1deb200 msgr2=0x562fe1deb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.843+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/705179600 shutdown_connections 2024-09-10T08:24:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.843+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/705179600 wait complete. 2024-09-10T08:24:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.844+0000 7f2a9d6a4740 1 -- start start 2024-09-10T08:24:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.844+0000 7f2a9d6a4740 1 Processor -- start 2024-09-10T08:24:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.845+0000 7f2a9d6a4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1deec70 0x562fe1df6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.845+0000 7f2a9d6a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1def590 0x562fe1df7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.845+0000 7f2a9d6a4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1df9e50 0x562fe1df78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.845+0000 7f2a9d6a4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562fe1dfca20 con 0x562fe1df9e50 2024-09-10T08:24:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.845+0000 7f2a9d6a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562fe1d100f0 con 0x562fe1def590 2024-09-10T08:24:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.845+0000 7f2a9d6a4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562fe1dfcba0 con 0x562fe1deec70 2024-09-10T08:24:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.845+0000 7f2a9d6a4740 10 client.0 _ll_register_callbacks cb 0x562fe1cf7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.845+0000 7f2a93e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1deec70 0x562fe1df6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.845+0000 7f2a93e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1deec70 0x562fe1df6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51428/0 (socket says 172.21.15.2:51428) 2024-09-10T08:24:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.845+0000 7f2a93e00640 1 -- 172.21.15.2:0/2759280580 learned_addr learned my addr 172.21.15.2:0/2759280580 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.845+0000 7f2a8be00640 1 --2- 172.21.15.2:0/2759280580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1def590 0x562fe1df7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.845+0000 7f2a98c00640 1 --2- 172.21.15.2:0/2759280580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1df9e50 0x562fe1df78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.846+0000 7f2a8be00640 1 -- 172.21.15.2:0/2759280580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1deec70 msgr2=0x562fe1df6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:23.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.846+0000 7f2a8be00640 1 --2- 172.21.15.2:0/2759280580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1deec70 0x562fe1df6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.846+0000 7f2a8be00640 1 -- 172.21.15.2:0/2759280580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1df9e50 msgr2=0x562fe1df78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:23.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.846+0000 7f2a8be00640 1 --2- 172.21.15.2:0/2759280580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1df9e50 0x562fe1df78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.846+0000 7f2a8be00640 1 -- 172.21.15.2:0/2759280580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe1ac63c0 con 0x562fe1def590 2024-09-10T08:24:23.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.846+0000 7f2a8be00640 1 --2- 172.21.15.2:0/2759280580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1def590 0x562fe1df7370 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x562fe1dec580 tx=0x7f2a7c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6feb8295dde7fad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:23.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.846+0000 7f2a81600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:23.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.846+0000 7f2a81600640 1 -- 172.21.15.2:0/2759280580 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c035070 con 0x562fe1def590 2024-09-10T08:24:23.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.846+0000 7f2a81600640 1 -- 172.21.15.2:0/2759280580 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2a7c0ed320 con 0x562fe1def590 2024-09-10T08:24:23.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.846+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/2759280580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a8c004050 con 0x562fe1def590 2024-09-10T08:24:23.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.846+0000 7f2a81600640 1 -- 172.21.15.2:0/2759280580 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a7c0d6020 con 0x562fe1def590 2024-09-10T08:24:23.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.846+0000 7f2a9d6a4740 10 client.30128 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:23.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.846+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/2759280580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562fe1e11350 con 0x562fe1def590 2024-09-10T08:24:23.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.847+0000 7f2a81600640 1 -- 172.21.15.2:0/2759280580 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c004050 con 0x562fe1def590 2024-09-10T08:24:23.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.847+0000 7f2a81600640 1 -- 172.21.15.2:0/2759280580 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2a7c0b8750 con 0x562fe1def590 2024-09-10T08:24:23.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.847+0000 7f2a9d6a4740 10 client.30128 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:23.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.847+0000 7f2a9d6a4740 10 client.30128 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:24:23.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.847+0000 7f2a9d6a4740 20 client.30128 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:23.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.847+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/2759280580 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562fe1e12800 0x562fe1e14c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.847+0000 7f2a9d6a4740 4 client.30128 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.847+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/2759280580 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fe1d62950 con 0x562fe1e12800 2024-09-10T08:24:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.847+0000 7f2a93e00640 1 --2- 172.21.15.2:0/2759280580 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562fe1e12800 0x562fe1e14c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.848+0000 7f2a93e00640 1 --2- 172.21.15.2:0/2759280580 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562fe1e12800 0x562fe1e14c40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f9c890f1d64aa30 server_cookie=172ebb149d3deadb in_seq=0 out_seq=0 2024-09-10T08:24:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.848+0000 7f2a81600640 10 client.30128 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:24:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.849+0000 7f2a81600640 1 -- 172.21.15.2:0/2759280580 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fe1d62950 con 0x562fe1e12800 2024-09-10T08:24:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.849+0000 7f2a81600640 10 client.30128 handle_command_reply: tid=0 2024-09-10T08:24:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.849+0000 7f2a9d6a4740 1 client.30128 shutdown 2024-09-10T08:24:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.849+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/2759280580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1def590 msgr2=0x562fe1df7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.849+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/2759280580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1def590 0x562fe1df7370 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x562fe1dec580 tx=0x7f2a7c009560 comp rx=0 tx=0).stop 2024-09-10T08:24:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.849+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/2759280580 shutdown_connections 2024-09-10T08:24:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.849+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/2759280580 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562fe1e12800 0x562fe1e14c40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.849+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/2759280580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1df9e50 0x562fe1df78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.849+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/2759280580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1def590 0x562fe1df7370 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.849+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/2759280580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1deec70 0x562fe1df6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.849+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/2759280580 >> 172.21.15.2:0/2759280580 conn(0x562fe1deb200 msgr2=0x562fe1df0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.850+0000 7f2a81600640 0 client.30128 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:24:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.850+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/2759280580 shutdown_connections 2024-09-10T08:24:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.850+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/2759280580 wait complete. 2024-09-10T08:24:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.850+0000 7f2a9d6a4740 20 client.30128 trim_cache size 0 max 16384 2024-09-10T08:24:23.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.864+0000 7f2a9d6a4740 1 Processor -- start 2024-09-10T08:24:23.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.864+0000 7f2a9d6a4740 1 -- start start 2024-09-10T08:24:23.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.864+0000 7f2a9d6a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 0x562fe1df0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.864+0000 7f2a9d6a4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1def590 0x562fe1e1a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.864+0000 7f2a9d6a4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 0x562fe1e1aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.864+0000 7f2a9d6a4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941af410 con 0x562fe1def590 2024-09-10T08:24:23.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.864+0000 7f2a9d6a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941acce0 con 0x562fe1deec70 2024-09-10T08:24:23.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.864+0000 7f2a9d6a4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941aab30 con 0x562fe1df9e50 2024-09-10T08:24:23.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.865+0000 7f2a8be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1def590 0x562fe1e1a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.865+0000 7f2a93e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 0x562fe1df0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.865+0000 7f2a8be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1def590 0x562fe1e1a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43726/0 (socket says 172.21.15.2:43726) 2024-09-10T08:24:23.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.865+0000 7f2a8be00640 1 -- 172.21.15.2:0/785419839 learned_addr learned my addr 172.21.15.2:0/785419839 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:23.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.865+0000 7f2a93e00640 1 --2- 172.21.15.2:0/785419839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 0x562fe1df0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36988/0 (socket says 172.21.15.2:36988) 2024-09-10T08:24:23.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.865+0000 7f2a98c00640 1 --2- 172.21.15.2:0/785419839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 0x562fe1e1aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.865+0000 7f2a93e00640 1 -- 172.21.15.2:0/785419839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 msgr2=0x562fe1e1aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:23.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.865+0000 7f2a93e00640 1 --2- 172.21.15.2:0/785419839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 0x562fe1e1aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.865+0000 7f2a93e00640 1 -- 172.21.15.2:0/785419839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1def590 msgr2=0x562fe1e1a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:23.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.865+0000 7f2a93e00640 1 --2- 172.21.15.2:0/785419839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1def590 0x562fe1e1a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.865+0000 7f2a93e00640 1 -- 172.21.15.2:0/785419839 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe1e0f010 con 0x562fe1deec70 2024-09-10T08:24:23.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.866+0000 7f2a93e00640 1 --2- 172.21.15.2:0/785419839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 0x562fe1df0e20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2a84012cb0 tx=0x7f2a8401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b42c4c31bfce6275 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:23.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.866+0000 7f2a83e00640 1 -- 172.21.15.2:0/785419839 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84004400 con 0x562fe1deec70 2024-09-10T08:24:23.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.866+0000 7f2a83e00640 1 -- 172.21.15.2:0/785419839 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a84005500 con 0x562fe1deec70 2024-09-10T08:24:23.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.866+0000 7f2a83e00640 1 -- 172.21.15.2:0/785419839 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8402b680 con 0x562fe1deec70 2024-09-10T08:24:23.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.867+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/785419839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 msgr2=0x562fe1df0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:23.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.867+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/785419839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 0x562fe1df0e20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2a84012cb0 tx=0x7f2a8401b000 comp rx=0 tx=0).stop 2024-09-10T08:24:23.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.867+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/785419839 shutdown_connections 2024-09-10T08:24:23.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.867+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/785419839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 0x562fe1e1aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.867+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/785419839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1def590 0x562fe1e1a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.867+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/785419839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1deec70 0x562fe1df0e20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.867+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/785419839 >> 172.21.15.2:0/785419839 conn(0x562fe1deb200 msgr2=0x562fe1df0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:23.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.867+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/785419839 shutdown_connections 2024-09-10T08:24:23.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.867+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/785419839 wait complete. 2024-09-10T08:24:23.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.867+0000 7f2a9d6a4740 1 -- start start 2024-09-10T08:24:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.867+0000 7f2a9d6a4740 1 Processor -- start 2024-09-10T08:24:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.868+0000 7f2a9d6a4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1deec70 0x562fe1e065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.868+0000 7f2a9d6a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1def590 0x562fe1e06ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.868+0000 7f2a9d6a4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 0x562fe1e07020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.868+0000 7f2a9d6a4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941ab1a0 con 0x562fe1deec70 2024-09-10T08:24:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.868+0000 7f2a9d6a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941ac610 con 0x562fe1def590 2024-09-10T08:24:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.868+0000 7f2a9d6a4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941ab870 con 0x562fe1df9e50 2024-09-10T08:24:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.868+0000 7f2a9d6a4740 10 client.0 _ll_register_callbacks cb 0x7f2a7c005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.868+0000 7f2a93e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1deec70 0x562fe1e065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.868+0000 7f2a8be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1def590 0x562fe1e06ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.868+0000 7f2a98c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 0x562fe1e07020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.868+0000 7f2a8be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1def590 0x562fe1e06ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:36994/0 (socket says 172.21.15.2:36994) 2024-09-10T08:24:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.868+0000 7f2a8be00640 1 -- 172.21.15.2:0/513571061 learned_addr learned my addr 172.21.15.2:0/513571061 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.869+0000 7f2a98c00640 1 -- 172.21.15.2:0/513571061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1def590 msgr2=0x562fe1e06ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.869+0000 7f2a98c00640 1 --2- 172.21.15.2:0/513571061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1def590 0x562fe1e06ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.869+0000 7f2a98c00640 1 -- 172.21.15.2:0/513571061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1deec70 msgr2=0x562fe1e065a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.869+0000 7f2a98c00640 1 --2- 172.21.15.2:0/513571061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1deec70 0x562fe1e065a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.869+0000 7f2a98c00640 1 -- 172.21.15.2:0/513571061 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe1e7c6e0 con 0x562fe1df9e50 2024-09-10T08:24:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.869+0000 7f2a8be00640 1 --2- 172.21.15.2:0/513571061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1def590 0x562fe1e06ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.869+0000 7f2a98c00640 1 --2- 172.21.15.2:0/513571061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 0x562fe1e07020 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c000f80 tx=0x7f2a8c003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ce58f62d2bd8819 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.869+0000 7f2a81600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:24:23.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.869+0000 7f2a81600640 1 -- 172.21.15.2:0/513571061 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c005350 con 0x562fe1df9e50 2024-09-10T08:24:23.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.869+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/513571061 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a84005500 con 0x562fe1df9e50 2024-09-10T08:24:23.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.870+0000 7f2a81600640 1 -- 172.21.15.2:0/513571061 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f2a8c0054f0 con 0x562fe1df9e50 2024-09-10T08:24:23.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.870+0000 7f2a81600640 1 -- 172.21.15.2:0/513571061 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a8c005730 con 0x562fe1df9e50 2024-09-10T08:24:23.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.870+0000 7f2a9d6a4740 10 client.30067 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:23.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.870+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/513571061 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562fe1e7d980 con 0x562fe1df9e50 2024-09-10T08:24:23.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.870+0000 7f2a81600640 1 -- 172.21.15.2:0/513571061 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a84005500 con 0x562fe1df9e50 2024-09-10T08:24:23.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.870+0000 7f2a81600640 1 -- 172.21.15.2:0/513571061 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2a8c00f510 con 0x562fe1df9e50 2024-09-10T08:24:23.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.870+0000 7f2a9d6a4740 10 client.30067 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:23.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.870+0000 7f2a9d6a4740 10 client.30067 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:24:23.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.870+0000 7f2a9d6a4740 20 client.30067 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:23.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.870+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/513571061 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562fe1e7dec0 0x562fe1e802b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:23.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.870+0000 7f2a9d6a4740 4 client.30067 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:24:23.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.870+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/513571061 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562fe1d5ce60 con 0x562fe1e7dec0 2024-09-10T08:24:23.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.871+0000 7f2a93e00640 1 --2- 172.21.15.2:0/513571061 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562fe1e7dec0 0x562fe1e802b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:23.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.871+0000 7f2a93e00640 1 --2- 172.21.15.2:0/513571061 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562fe1e7dec0 0x562fe1e802b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d3fac1589792b8e server_cookie=b9d60134ba7d4422 in_seq=0 out_seq=0 2024-09-10T08:24:23.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.871+0000 7f2a81600640 10 client.30067 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:24:23.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.872+0000 7f2a81600640 1 -- 172.21.15.2:0/513571061 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562fe1d5ce60 con 0x562fe1e7dec0 2024-09-10T08:24:23.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.872+0000 7f2a81600640 10 client.30067 handle_command_reply: tid=0 2024-09-10T08:24:23.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.872+0000 7f2a9d6a4740 1 client.30067 shutdown 2024-09-10T08:24:23.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.872+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/513571061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 msgr2=0x562fe1e07020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:23.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.872+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/513571061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 0x562fe1e07020 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c000f80 tx=0x7f2a8c003f50 comp rx=0 tx=0).stop 2024-09-10T08:24:23.874 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:23 smithi002 ceph-mon[30563]: pgmap v874: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-10T08:24:23.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.872+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/513571061 shutdown_connections 2024-09-10T08:24:23.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.872+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/513571061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562fe1df9e50 0x562fe1e07020 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.872+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/513571061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562fe1def590 0x562fe1e06ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.872+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/513571061 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x562fe1e7dec0 0x562fe1e802b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.872+0000 7f2a9d6a4740 1 --2- 172.21.15.2:0/513571061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562fe1deec70 0x562fe1e065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.872+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/513571061 >> 172.21.15.2:0/513571061 conn(0x562fe1deb200 msgr2=0x562fe1df7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:23.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.872+0000 7f2a81600640 0 client.30067 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:24:23.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.873+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/513571061 shutdown_connections 2024-09-10T08:24:23.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.873+0000 7f2a9d6a4740 1 -- 172.21.15.2:0/513571061 wait complete. 2024-09-10T08:24:23.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.873+0000 7f2a9d6a4740 20 client.30067 trim_cache size 0 max 16384 2024-09-10T08:24:23.876 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:24:23.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.873+0000 7f2a9ac00640 1 -- 172.21.15.2:0/3512127748 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2a780a2b80 msgr2=0x7f2a780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:23.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.873+0000 7f2a9ac00640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2a780a2b80 0x7f2a780a5040 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x562fe1dec060 tx=0x7f2a84005f90 comp rx=0 tx=0).stop 2024-09-10T08:24:23.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.874+0000 7f2a9ac00640 1 -- 172.21.15.2:0/3512127748 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a941049c0 msgr2=0x7f2a9419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:23.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.874+0000 7f2a9ac00640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a941049c0 0x7f2a9419abf0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c006010 tx=0x7f2a7c004040 comp rx=0 tx=0).stop 2024-09-10T08:24:23.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.874+0000 7f2a9ac00640 1 -- 172.21.15.2:0/3512127748 shutdown_connections 2024-09-10T08:24:23.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.874+0000 7f2a9ac00640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a94109140 0x7f2a9419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.874+0000 7f2a9ac00640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a941049c0 0x7f2a9419abf0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.874+0000 7f2a9ac00640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2a780a2b80 0x7f2a780a5040 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.874+0000 7f2a9ac00640 1 --2- 172.21.15.2:0/3512127748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a94104010 0x7f2a9419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:23.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.874+0000 7f2a9ac00640 1 -- 172.21.15.2:0/3512127748 >> 172.21.15.2:0/3512127748 conn(0x7f2a940ffa50 msgr2=0x7f2a94100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:23.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.874+0000 7f2a9ac00640 1 -- 172.21.15.2:0/3512127748 shutdown_connections 2024-09-10T08:24:23.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:23.874+0000 7f2a9ac00640 1 -- 172.21.15.2:0/3512127748 wait complete. 2024-09-10T08:24:23.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:23 smithi155 ceph-mon[35405]: pgmap v874: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-10T08:24:23.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:23 smithi107 ceph-mon[35048]: pgmap v874: 129 pgs: 129 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-10T08:24:24.627 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:24:25.012 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:25.707 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:25 smithi002 ceph-mon[30563]: pgmap v875: 129 pgs: 129 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2024-09-10T08:24:25.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.721+0000 7f411fe00640 1 -- 172.21.15.2:0/225767002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41181027a0 msgr2=0x7f41181067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:25.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.721+0000 7f411fe00640 1 --2- 172.21.15.2:0/225767002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41181027a0 0x7f41181067f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f4104009900 tx=0x7f410402a400 comp rx=0 tx=0).stop 2024-09-10T08:24:25.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.721+0000 7f411fe00640 1 -- 172.21.15.2:0/225767002 shutdown_connections 2024-09-10T08:24:25.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.721+0000 7f411fe00640 1 --2- 172.21.15.2:0/225767002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4118106f20 0x7f411810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.721+0000 7f411fe00640 1 --2- 172.21.15.2:0/225767002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41181027a0 0x7f41181067f0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.721+0000 7f411fe00640 1 --2- 172.21.15.2:0/225767002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4118101df0 0x7f41181021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.721+0000 7f411fe00640 1 -- 172.21.15.2:0/225767002 >> 172.21.15.2:0/225767002 conn(0x7f41180fd8b0 msgr2=0x7f41180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:25.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.721+0000 7f411fe00640 1 -- 172.21.15.2:0/225767002 shutdown_connections 2024-09-10T08:24:25.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.721+0000 7f411fe00640 1 -- 172.21.15.2:0/225767002 wait complete. 2024-09-10T08:24:25.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.722+0000 7f411fe00640 1 Processor -- start 2024-09-10T08:24:25.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.722+0000 7f411fe00640 1 -- start start 2024-09-10T08:24:25.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.723+0000 7f411fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4118101df0 0x7f41180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.723+0000 7f411fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41181027a0 0x7f4118074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.723+0000 7f411fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118106f20 0x7f4118074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.723+0000 7f411fe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4118111f80 con 0x7f4118106f20 2024-09-10T08:24:25.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.723+0000 7f411fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4118111e00 con 0x7f41181027a0 2024-09-10T08:24:25.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.723+0000 7f411fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4118112100 con 0x7f4118101df0 2024-09-10T08:24:25.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.723+0000 7f411de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118106f20 0x7f4118074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.723+0000 7f411de00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118106f20 0x7f4118074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43748/0 (socket says 172.21.15.2:43748) 2024-09-10T08:24:25.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.723+0000 7f411de00640 1 -- 172.21.15.2:0/3717645297 learned_addr learned my addr 172.21.15.2:0/3717645297 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:25.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.723+0000 7f411d400640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4118101df0 0x7f41180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.723+0000 7f411ca00640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41181027a0 0x7f4118074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.724+0000 7f411de00640 1 -- 172.21.15.2:0/3717645297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4118101df0 msgr2=0x7f41180744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:25.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.724+0000 7f411de00640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4118101df0 0x7f41180744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.724+0000 7f411de00640 1 -- 172.21.15.2:0/3717645297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41181027a0 msgr2=0x7f4118074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:25.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.724+0000 7f411de00640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41181027a0 0x7f4118074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.724+0000 7f411de00640 1 -- 172.21.15.2:0/3717645297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41181a6880 con 0x7f4118106f20 2024-09-10T08:24:25.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.724+0000 7f411de00640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118106f20 0x7f4118074f70 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f411000eb10 tx=0x7f411000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:25.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.724+0000 7f4116000640 1 -- 172.21.15.2:0/3717645297 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110004b20 con 0x7f4118106f20 2024-09-10T08:24:25.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.724+0000 7f411fe00640 1 -- 172.21.15.2:0/3717645297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41181a6b70 con 0x7f4118106f20 2024-09-10T08:24:25.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.724+0000 7f4116000640 1 -- 172.21.15.2:0/3717645297 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4110004cc0 con 0x7f4118106f20 2024-09-10T08:24:25.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.724+0000 7f4116000640 1 -- 172.21.15.2:0/3717645297 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110010760 con 0x7f4118106f20 2024-09-10T08:24:25.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.724+0000 7f411fe00640 1 -- 172.21.15.2:0/3717645297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41181a7080 con 0x7f4118106f20 2024-09-10T08:24:25.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.728+0000 7f4116000640 1 -- 172.21.15.2:0/3717645297 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4110005020 con 0x7f4118106f20 2024-09-10T08:24:25.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.728+0000 7f41227f8740 1 Processor -- start 2024-09-10T08:24:25.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.728+0000 7f41227f8740 1 -- start start 2024-09-10T08:24:25.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.728+0000 7f41227f8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc999c70 0x5559cc99a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.728+0000 7f41227f8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc99a590 0x5559cc9a4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.728+0000 7f41227f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc9a4e50 0x5559cc9a7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.728+0000 7f41227f8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5559cc9003e0 con 0x5559cc999c70 2024-09-10T08:24:25.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.728+0000 7f41227f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5559cc8a7d80 con 0x5559cc9a4e50 2024-09-10T08:24:25.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.728+0000 7f41227f8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5559cc8fc3a0 con 0x5559cc99a590 2024-09-10T08:24:25.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.729+0000 7f411ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc999c70 0x5559cc99a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.729+0000 7f411ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc999c70 0x5559cc99a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43762/0 (socket says 172.21.15.2:43762) 2024-09-10T08:24:25.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.729+0000 7f411ca00640 1 -- 172.21.15.2:0/456714025 learned_addr learned my addr 172.21.15.2:0/456714025 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:25.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.729+0000 7f411d400640 1 --2- 172.21.15.2:0/456714025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc99a590 0x5559cc9a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.729+0000 7f411ca00640 1 --2- 172.21.15.2:0/456714025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc9a4e50 0x5559cc9a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.729+0000 7f411d400640 1 -- 172.21.15.2:0/456714025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc9a4e50 msgr2=0x5559cc9a7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:25.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.729+0000 7f411d400640 1 --2- 172.21.15.2:0/456714025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc9a4e50 0x5559cc9a7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.729+0000 7f411d400640 1 -- 172.21.15.2:0/456714025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc999c70 msgr2=0x5559cc99a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:25.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.729+0000 7f411d400640 1 --2- 172.21.15.2:0/456714025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc999c70 0x5559cc99a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.729+0000 7f411d400640 1 -- 172.21.15.2:0/456714025 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559cc8c6ae0 con 0x5559cc99a590 2024-09-10T08:24:25.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.729+0000 7f411ca00640 1 --2- 172.21.15.2:0/456714025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc999c70 0x5559cc99a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:25.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.730+0000 7f4116000640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41000a2c00 0x7f41000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.730+0000 7f411d400640 1 --2- 172.21.15.2:0/456714025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc99a590 0x5559cc9a4910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f410c00bbf0 tx=0x7f410c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cda4c6f64374c889 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:25.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.730+0000 7f4116000640 1 -- 172.21.15.2:0/3717645297 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f41100cf920 con 0x7f4118106f20 2024-09-10T08:24:25.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.730+0000 7f410a600640 1 -- 172.21.15.2:0/456714025 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c004030 con 0x5559cc99a590 2024-09-10T08:24:25.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.730+0000 7f410a600640 1 -- 172.21.15.2:0/456714025 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f410c0041d0 con 0x5559cc99a590 2024-09-10T08:24:25.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.730+0000 7f410a600640 1 -- 172.21.15.2:0/456714025 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c0116c0 con 0x5559cc99a590 2024-09-10T08:24:25.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.730+0000 7f411ca00640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41000a2c00 0x7f41000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.731+0000 7f41227f8740 1 -- 172.21.15.2:0/456714025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc99a590 msgr2=0x5559cc9a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:25.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.731+0000 7f41227f8740 1 --2- 172.21.15.2:0/456714025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc99a590 0x5559cc9a4910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f410c00bbf0 tx=0x7f410c0107a0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.731+0000 7f411ca00640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41000a2c00 0x7f41000a50c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f41040096c0 tx=0x7f4104000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:25.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.731+0000 7f41227f8740 1 -- 172.21.15.2:0/456714025 shutdown_connections 2024-09-10T08:24:25.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.731+0000 7f41227f8740 1 --2- 172.21.15.2:0/456714025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc9a4e50 0x5559cc9a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.731+0000 7f41227f8740 1 --2- 172.21.15.2:0/456714025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc99a590 0x5559cc9a4910 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.731+0000 7f41227f8740 1 --2- 172.21.15.2:0/456714025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc999c70 0x5559cc99a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.731+0000 7f41227f8740 1 -- 172.21.15.2:0/456714025 >> 172.21.15.2:0/456714025 conn(0x5559cc996200 msgr2=0x5559cc9965e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:25.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.731+0000 7f41227f8740 1 -- 172.21.15.2:0/456714025 shutdown_connections 2024-09-10T08:24:25.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.731+0000 7f41227f8740 1 -- 172.21.15.2:0/456714025 wait complete. 2024-09-10T08:24:25.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.732+0000 7f41227f8740 1 -- start start 2024-09-10T08:24:25.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.732+0000 7f41227f8740 1 Processor -- start 2024-09-10T08:24:25.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.732+0000 7f41227f8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc999c70 0x5559cc9a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.732+0000 7f41227f8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc99a590 0x5559cc9a2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.732+0000 7f41227f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc9a4e50 0x5559cc9a2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.732+0000 7f41227f8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5559cc9a7a20 con 0x5559cc99a590 2024-09-10T08:24:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.732+0000 7f41227f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5559cc8bb0f0 con 0x5559cc9a4e50 2024-09-10T08:24:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.732+0000 7f41227f8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5559cc9a7ba0 con 0x5559cc999c70 2024-09-10T08:24:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.732+0000 7f41227f8740 10 client.0 _ll_register_callbacks cb 0x5559cc8a2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.733+0000 7f411d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc999c70 0x5559cc9a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.733+0000 7f411d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc999c70 0x5559cc9a1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51474/0 (socket says 172.21.15.2:51474) 2024-09-10T08:24:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.733+0000 7f411d400640 1 -- 172.21.15.2:0/1073537427 learned_addr learned my addr 172.21.15.2:0/1073537427 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.733+0000 7f411ca00640 1 --2- 172.21.15.2:0/1073537427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc99a590 0x5559cc9a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.733+0000 7f411de00640 1 --2- 172.21.15.2:0/1073537427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc9a4e50 0x5559cc9a2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.733+0000 7f411d400640 1 -- 172.21.15.2:0/1073537427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc9a4e50 msgr2=0x5559cc9a2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.733+0000 7f411d400640 1 --2- 172.21.15.2:0/1073537427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc9a4e50 0x5559cc9a2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.733+0000 7f411d400640 1 -- 172.21.15.2:0/1073537427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc99a590 msgr2=0x5559cc9a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.733+0000 7f411d400640 1 --2- 172.21.15.2:0/1073537427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc99a590 0x5559cc9a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.733+0000 7f411d400640 1 -- 172.21.15.2:0/1073537427 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559cc6713c0 con 0x5559cc999c70 2024-09-10T08:24:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.734+0000 7f411d400640 1 --2- 172.21.15.2:0/1073537427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc999c70 0x5559cc9a1cd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f410c011010 tx=0x7f410c00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c51640d05b9c86c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.734+0000 7f40fea00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:24:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.734+0000 7f40fea00640 1 -- 172.21.15.2:0/1073537427 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c005040 con 0x5559cc999c70 2024-09-10T08:24:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.734+0000 7f40fea00640 1 -- 172.21.15.2:0/1073537427 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f410c0051e0 con 0x5559cc999c70 2024-09-10T08:24:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.734+0000 7f41227f8740 1 -- 172.21.15.2:0/1073537427 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f410c0041d0 con 0x5559cc999c70 2024-09-10T08:24:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.734+0000 7f40fea00640 1 -- 172.21.15.2:0/1073537427 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f410c005400 con 0x5559cc999c70 2024-09-10T08:24:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.734+0000 7f41227f8740 10 client.30079 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:25.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.734+0000 7f41227f8740 1 -- 172.21.15.2:0/1073537427 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5559cc9b3a20 con 0x5559cc999c70 2024-09-10T08:24:25.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.734+0000 7f40fea00640 1 -- 172.21.15.2:0/1073537427 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f410c0041d0 con 0x5559cc999c70 2024-09-10T08:24:25.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.735+0000 7f40fea00640 1 -- 172.21.15.2:0/1073537427 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f410c004990 con 0x5559cc999c70 2024-09-10T08:24:25.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.735+0000 7f41227f8740 10 client.30079 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:25.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.735+0000 7f41227f8740 10 client.30079 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:24:25.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.735+0000 7f41227f8740 20 client.30079 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:25.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.735+0000 7f41227f8740 1 --2- 172.21.15.2:0/1073537427 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5559cc9b4ee0 0x5559cc9b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.735+0000 7f41227f8740 4 client.30079 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:25.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.735+0000 7f41227f8740 1 -- 172.21.15.2:0/1073537427 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559cc90d950 con 0x5559cc9b4ee0 2024-09-10T08:24:25.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.735+0000 7f411ca00640 1 --2- 172.21.15.2:0/1073537427 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5559cc9b4ee0 0x5559cc9b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.736+0000 7f411ca00640 1 --2- 172.21.15.2:0/1073537427 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5559cc9b4ee0 0x5559cc9b72d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c28fb299c607d46 server_cookie=58883877cc471c19 in_seq=0 out_seq=0 2024-09-10T08:24:25.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.736+0000 7f40fea00640 10 client.30079 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:24:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.736+0000 7f40fea00640 1 -- 172.21.15.2:0/1073537427 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559cc90d950 con 0x5559cc9b4ee0 2024-09-10T08:24:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f40fea00640 10 client.30079 handle_command_reply: tid=0 2024-09-10T08:24:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f41227f8740 1 client.30079 shutdown 2024-09-10T08:24:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f41227f8740 1 -- 172.21.15.2:0/1073537427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc999c70 msgr2=0x5559cc9a1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f41227f8740 1 --2- 172.21.15.2:0/1073537427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc999c70 0x5559cc9a1cd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f410c011010 tx=0x7f410c00bfd0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f41227f8740 1 -- 172.21.15.2:0/1073537427 shutdown_connections 2024-09-10T08:24:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f41227f8740 1 --2- 172.21.15.2:0/1073537427 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5559cc9b4ee0 0x5559cc9b72d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f41227f8740 1 --2- 172.21.15.2:0/1073537427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc9a4e50 0x5559cc9a2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f41227f8740 1 --2- 172.21.15.2:0/1073537427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc99a590 0x5559cc9a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f40fea00640 0 client.30079 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:24:25.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f41227f8740 1 --2- 172.21.15.2:0/1073537427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc999c70 0x5559cc9a1cd0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f41227f8740 1 -- 172.21.15.2:0/1073537427 >> 172.21.15.2:0/1073537427 conn(0x5559cc996200 msgr2=0x5559cc99bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f41227f8740 1 -- 172.21.15.2:0/1073537427 shutdown_connections 2024-09-10T08:24:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f41227f8740 1 -- 172.21.15.2:0/1073537427 wait complete. 2024-09-10T08:24:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.737+0000 7f41227f8740 20 client.30079 trim_cache size 0 max 16384 2024-09-10T08:24:25.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.751+0000 7f41227f8740 1 Processor -- start 2024-09-10T08:24:25.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.751+0000 7f41227f8740 1 -- start start 2024-09-10T08:24:25.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f41227f8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc999c70 0x5559cc9a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f41227f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc99a590 0x5559cc9bc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f41227f8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc9a4e50 0x5559cc9bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f41227f8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f41181b38c0 con 0x5559cc999c70 2024-09-10T08:24:25.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f41227f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f41181b1190 con 0x5559cc99a590 2024-09-10T08:24:25.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f41227f8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41181aefe0 con 0x5559cc9a4e50 2024-09-10T08:24:25.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f411d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc999c70 0x5559cc9a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f411ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc99a590 0x5559cc9bc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f411d400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc999c70 0x5559cc9a3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43776/0 (socket says 172.21.15.2:43776) 2024-09-10T08:24:25.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f411d400640 1 -- 172.21.15.2:0/3995841597 learned_addr learned my addr 172.21.15.2:0/3995841597 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f411de00640 1 --2- 172.21.15.2:0/3995841597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc9a4e50 0x5559cc9bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f411ca00640 1 -- 172.21.15.2:0/3995841597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc9a4e50 msgr2=0x5559cc9bcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f411ca00640 1 --2- 172.21.15.2:0/3995841597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc9a4e50 0x5559cc9bcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f411ca00640 1 -- 172.21.15.2:0/3995841597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc999c70 msgr2=0x5559cc9a3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f411ca00640 1 --2- 172.21.15.2:0/3995841597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc999c70 0x5559cc9a3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.752+0000 7f411ca00640 1 -- 172.21.15.2:0/3995841597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559cc996030 con 0x5559cc99a590 2024-09-10T08:24:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.753+0000 7f411ca00640 1 --2- 172.21.15.2:0/3995841597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc99a590 0x5559cc9bc9e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x5559cc997580 tx=0x7f410403b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d1d98c02fba1fab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.753+0000 7f4109c00640 1 -- 172.21.15.2:0/3995841597 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4104035040 con 0x5559cc99a590 2024-09-10T08:24:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.753+0000 7f4109c00640 1 -- 172.21.15.2:0/3995841597 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f410403b460 con 0x5559cc99a590 2024-09-10T08:24:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.753+0000 7f4109c00640 1 -- 172.21.15.2:0/3995841597 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4104003ea0 con 0x5559cc99a590 2024-09-10T08:24:25.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.753+0000 7f41227f8740 1 -- 172.21.15.2:0/3995841597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc99a590 msgr2=0x5559cc9bc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:25.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.753+0000 7f41227f8740 1 --2- 172.21.15.2:0/3995841597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc99a590 0x5559cc9bc9e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x5559cc997580 tx=0x7f410403b000 comp rx=0 tx=0).stop 2024-09-10T08:24:25.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.754+0000 7f41227f8740 1 -- 172.21.15.2:0/3995841597 shutdown_connections 2024-09-10T08:24:25.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.754+0000 7f41227f8740 1 --2- 172.21.15.2:0/3995841597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc9a4e50 0x5559cc9bcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.754+0000 7f41227f8740 1 --2- 172.21.15.2:0/3995841597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc99a590 0x5559cc9bc9e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.754+0000 7f41227f8740 1 --2- 172.21.15.2:0/3995841597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc999c70 0x5559cc9a3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.754+0000 7f41227f8740 1 -- 172.21.15.2:0/3995841597 >> 172.21.15.2:0/3995841597 conn(0x5559cc996200 msgr2=0x5559cc9a3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:25.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.754+0000 7f41227f8740 1 -- 172.21.15.2:0/3995841597 shutdown_connections 2024-09-10T08:24:25.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.754+0000 7f41227f8740 1 -- 172.21.15.2:0/3995841597 wait complete. 2024-09-10T08:24:25.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.754+0000 7f41227f8740 1 -- start start 2024-09-10T08:24:25.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.754+0000 7f41227f8740 1 Processor -- start 2024-09-10T08:24:25.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f41227f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc999c70 0x5559cc9ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f41227f8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc99a590 0x5559cc99ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f41227f8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc9a4e50 0x5559cc99ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f41227f8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f41181af650 con 0x5559cc99a590 2024-09-10T08:24:25.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f41227f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f41181b0ac0 con 0x5559cc999c70 2024-09-10T08:24:25.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f41227f8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41181afd20 con 0x5559cc9a4e50 2024-09-10T08:24:25.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f41227f8740 10 client.0 _ll_register_callbacks cb 0x7f4104009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:25.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f411d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc999c70 0x5559cc9ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f411d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc999c70 0x5559cc9ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37050/0 (socket says 172.21.15.2:37050) 2024-09-10T08:24:25.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f411ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc99a590 0x5559cc99ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f411d400640 1 -- 172.21.15.2:0/2832702360 learned_addr learned my addr 172.21.15.2:0/2832702360 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:25.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f411de00640 1 --2- 172.21.15.2:0/2832702360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc9a4e50 0x5559cc99ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f411d400640 1 -- 172.21.15.2:0/2832702360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc9a4e50 msgr2=0x5559cc99ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:25.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.755+0000 7f411d400640 1 --2- 172.21.15.2:0/2832702360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc9a4e50 0x5559cc99ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.756+0000 7f411d400640 1 -- 172.21.15.2:0/2832702360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc99a590 msgr2=0x5559cc99ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:25.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.756+0000 7f411d400640 1 --2- 172.21.15.2:0/2832702360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc99a590 0x5559cc99ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.756+0000 7f411d400640 1 -- 172.21.15.2:0/2832702360 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559cc99f780 con 0x5559cc999c70 2024-09-10T08:24:25.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.756+0000 7f411de00640 1 --2- 172.21.15.2:0/2832702360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc9a4e50 0x5559cc99ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:25.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.756+0000 7f411d400640 1 --2- 172.21.15.2:0/2832702360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc999c70 0x5559cc9ad600 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f410c011010 tx=0x7f410c011670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ca774d259641b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:25.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.756+0000 7f40fea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:25.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.756+0000 7f40fea00640 1 -- 172.21.15.2:0/2832702360 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c0051e0 con 0x5559cc999c70 2024-09-10T08:24:25.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.756+0000 7f41227f8740 1 -- 172.21.15.2:0/2832702360 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f410403b460 con 0x5559cc999c70 2024-09-10T08:24:25.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.756+0000 7f40fea00640 1 -- 172.21.15.2:0/2832702360 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f410c004590 con 0x5559cc999c70 2024-09-10T08:24:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.756+0000 7f40fea00640 1 -- 172.21.15.2:0/2832702360 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f410c042050 con 0x5559cc999c70 2024-09-10T08:24:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.756+0000 7f41227f8740 10 client.30152 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.756+0000 7f41227f8740 1 -- 172.21.15.2:0/2832702360 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5559cca1f9b0 con 0x5559cc999c70 2024-09-10T08:24:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.757+0000 7f40fea00640 1 -- 172.21.15.2:0/2832702360 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f410403b460 con 0x5559cc999c70 2024-09-10T08:24:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.757+0000 7f40fea00640 1 -- 172.21.15.2:0/2832702360 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f410c034450 con 0x5559cc999c70 2024-09-10T08:24:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.757+0000 7f41227f8740 10 client.30152 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.757+0000 7f41227f8740 10 client.30152 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:24:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.757+0000 7f41227f8740 20 client.30152 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.757+0000 7f41227f8740 1 --2- 172.21.15.2:0/2832702360 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5559cca20060 0x5559cca22480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.757+0000 7f41227f8740 4 client.30152 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:24:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.757+0000 7f41227f8740 1 -- 172.21.15.2:0/2832702360 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559cc907e60 con 0x5559cca20060 2024-09-10T08:24:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.757+0000 7f411ca00640 1 --2- 172.21.15.2:0/2832702360 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5559cca20060 0x5559cca22480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.758+0000 7f411ca00640 1 --2- 172.21.15.2:0/2832702360 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5559cca20060 0x5559cca22480 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5a84a3b80a5a568 server_cookie=73850c1079a3ad02 in_seq=0 out_seq=0 2024-09-10T08:24:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.758+0000 7f40fea00640 10 client.30152 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:24:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.758+0000 7f40fea00640 1 -- 172.21.15.2:0/2832702360 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559cc907e60 con 0x5559cca20060 2024-09-10T08:24:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.758+0000 7f40fea00640 10 client.30152 handle_command_reply: tid=0 2024-09-10T08:24:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.758+0000 7f41227f8740 1 client.30152 shutdown 2024-09-10T08:24:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.758+0000 7f41227f8740 1 -- 172.21.15.2:0/2832702360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc999c70 msgr2=0x5559cc9ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.758+0000 7f41227f8740 1 --2- 172.21.15.2:0/2832702360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc999c70 0x5559cc9ad600 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f410c011010 tx=0x7f410c011670 comp rx=0 tx=0).stop 2024-09-10T08:24:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.758+0000 7f41227f8740 1 -- 172.21.15.2:0/2832702360 shutdown_connections 2024-09-10T08:24:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.758+0000 7f41227f8740 1 --2- 172.21.15.2:0/2832702360 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x5559cca20060 0x5559cca22480 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.758+0000 7f41227f8740 1 --2- 172.21.15.2:0/2832702360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559cc9a4e50 0x5559cc99ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.758+0000 7f41227f8740 1 --2- 172.21.15.2:0/2832702360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559cc99a590 0x5559cc99ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.758+0000 7f40fea00640 0 client.30152 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:24:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.758+0000 7f41227f8740 1 --2- 172.21.15.2:0/2832702360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5559cc999c70 0x5559cc9ad600 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.759+0000 7f41227f8740 1 -- 172.21.15.2:0/2832702360 >> 172.21.15.2:0/2832702360 conn(0x5559cc996200 msgr2=0x5559cc995380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.759+0000 7f41227f8740 1 -- 172.21.15.2:0/2832702360 shutdown_connections 2024-09-10T08:24:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.759+0000 7f41227f8740 1 -- 172.21.15.2:0/2832702360 wait complete. 2024-09-10T08:24:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.759+0000 7f41227f8740 20 client.30152 trim_cache size 0 max 16384 2024-09-10T08:24:25.765 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:24:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.760+0000 7f411fe00640 1 -- 172.21.15.2:0/3717645297 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41000a2c00 msgr2=0x7f41000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.760+0000 7f411fe00640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41000a2c00 0x7f41000a50c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f41040096c0 tx=0x7f4104000bc0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.760+0000 7f411fe00640 1 -- 172.21.15.2:0/3717645297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118106f20 msgr2=0x7f4118074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.760+0000 7f411fe00640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118106f20 0x7f4118074f70 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f411000eb10 tx=0x7f411000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.760+0000 7f411fe00640 1 -- 172.21.15.2:0/3717645297 shutdown_connections 2024-09-10T08:24:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.760+0000 7f411fe00640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4118106f20 0x7f4118074f70 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.760+0000 7f411fe00640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41181027a0 0x7f4118074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.760+0000 7f411fe00640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41000a2c00 0x7f41000a50c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.760+0000 7f411fe00640 1 --2- 172.21.15.2:0/3717645297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4118101df0 0x7f41180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.760+0000 7f411fe00640 1 -- 172.21.15.2:0/3717645297 >> 172.21.15.2:0/3717645297 conn(0x7f41180fd8b0 msgr2=0x7f41180feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.760+0000 7f411fe00640 1 -- 172.21.15.2:0/3717645297 shutdown_connections 2024-09-10T08:24:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:25.760+0000 7f411fe00640 1 -- 172.21.15.2:0/3717645297 wait complete. 2024-09-10T08:24:25.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:25 smithi155 ceph-mon[35405]: pgmap v875: 129 pgs: 129 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2024-09-10T08:24:25.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:25 smithi107 ceph-mon[35048]: pgmap v875: 129 pgs: 129 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2024-09-10T08:24:26.667 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:24:27.048 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:27.255 INFO:tasks.workunit.client.0.smithi002.stdout:561/600: open fd = 3 2024-09-10T08:24:27.255 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.255 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.255 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.255 INFO:tasks.workunit.client.0.smithi002.stdout:562/600: open fd = 3 2024-09-10T08:24:27.255 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.255 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.255 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.255 INFO:tasks.workunit.client.0.smithi002.stdout:563/600: open fd = 3 2024-09-10T08:24:27.255 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.255 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:564/600: open fd = 3 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:565/600: open fd = 3 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:566/600: open fd = 3 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.256 INFO:tasks.workunit.client.0.smithi002.stdout:567/600: open fd = 3 2024-09-10T08:24:27.257 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.257 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.257 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.257 INFO:tasks.workunit.client.0.smithi002.stdout:568/600: open fd = 3 2024-09-10T08:24:27.257 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.257 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.257 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.257 INFO:tasks.workunit.client.0.smithi002.stdout:569/600: open fd = 3 2024-09-10T08:24:27.257 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.257 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.257 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.257 INFO:tasks.workunit.client.0.smithi002.stdout:570/600: open fd = 3 2024-09-10T08:24:27.258 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.258 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.258 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.258 INFO:tasks.workunit.client.0.smithi002.stdout:571/600: open fd = 3 2024-09-10T08:24:27.258 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.258 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.258 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.258 INFO:tasks.workunit.client.0.smithi002.stdout:572/600: open fd = 3 2024-09-10T08:24:27.258 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.258 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.258 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.258 INFO:tasks.workunit.client.0.smithi002.stdout:573/600: open fd = 3 2024-09-10T08:24:27.258 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:574/600: open fd = 3 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:575/600: open fd = 3 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:576/600: open fd = 3 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.259 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.260 INFO:tasks.workunit.client.0.smithi002.stdout:577/600: open fd = 3 2024-09-10T08:24:27.260 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.260 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.260 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.260 INFO:tasks.workunit.client.0.smithi002.stdout:578/600: open fd = 3 2024-09-10T08:24:27.260 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.260 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.260 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.260 INFO:tasks.workunit.client.0.smithi002.stdout:579/600: open fd = 3 2024-09-10T08:24:27.260 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.260 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.260 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.260 INFO:tasks.workunit.client.0.smithi002.stdout:580/600: open fd = 3 2024-09-10T08:24:27.261 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.261 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.261 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.261 INFO:tasks.workunit.client.0.smithi002.stdout:581/600: open fd = 3 2024-09-10T08:24:27.261 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.261 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.261 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.261 INFO:tasks.workunit.client.0.smithi002.stdout:582/600: open fd = 3 2024-09-10T08:24:27.261 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.261 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.261 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.261 INFO:tasks.workunit.client.0.smithi002.stdout:583/600: open fd = 3 2024-09-10T08:24:27.261 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:584/600: open fd = 3 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:585/600: open fd = 3 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:586/600: open fd = 3 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.262 INFO:tasks.workunit.client.0.smithi002.stdout:587/600: open fd = 3 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:588/600: open fd = 3 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:589/600: open fd = 3 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:590/600: open fd = 3 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.263 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.264 INFO:tasks.workunit.client.0.smithi002.stdout:591/600: open fd = 3 2024-09-10T08:24:27.264 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.264 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.264 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.264 INFO:tasks.workunit.client.0.smithi002.stdout:592/600: open fd = 3 2024-09-10T08:24:27.264 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.264 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.264 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.264 INFO:tasks.workunit.client.0.smithi002.stdout:593/600: open fd = 3 2024-09-10T08:24:27.264 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.264 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.264 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.264 INFO:tasks.workunit.client.0.smithi002.stdout:594/600: open fd = 3 2024-09-10T08:24:27.265 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.265 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.265 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.265 INFO:tasks.workunit.client.0.smithi002.stdout:595/600: open fd = 3 2024-09-10T08:24:27.265 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.268 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.268 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.268 INFO:tasks.workunit.client.0.smithi002.stdout:596/600: open fd = 3 2024-09-10T08:24:27.268 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.268 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.268 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.268 INFO:tasks.workunit.client.0.smithi002.stdout:597/600: open fd = 3 2024-09-10T08:24:27.268 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.269 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.269 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.269 INFO:tasks.workunit.client.0.smithi002.stdout:598/600: open fd = 3 2024-09-10T08:24:27.269 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.269 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.269 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.269 INFO:tasks.workunit.client.0.smithi002.stdout:599/600: open fd = 3 2024-09-10T08:24:27.269 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.269 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.269 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.269 INFO:tasks.workunit.client.0.smithi002.stdout:600/600: open fd = 3 2024-09-10T08:24:27.269 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.269 INFO:tasks.workunit.client.0.smithi002.stdout:write ret = 32 2024-09-10T08:24:27.270 INFO:tasks.workunit.client.0.smithi002.stdout:pread ret = 64 2024-09-10T08:24:27.270 INFO:tasks.workunit.client.0.smithi002.stderr:+ echo OK 2024-09-10T08:24:27.271 INFO:tasks.workunit.client.0.smithi002.stdout:OK 2024-09-10T08:24:27.271 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T08:24:27.271 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T08:24:27.344 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-09-10T08:24:27.344 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T08:24:27.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.725+0000 7fbecfe00640 1 -- 172.21.15.2:0/3931385076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbec81049a0 msgr2=0x7fbec81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:27.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.725+0000 7fbecfe00640 1 --2- 172.21.15.2:0/3931385076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbec81049a0 0x7fbec81089f0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009900 tx=0x7fbeb402a410 comp rx=0 tx=0).stop 2024-09-10T08:24:27.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.725+0000 7fbecfe00640 1 -- 172.21.15.2:0/3931385076 shutdown_connections 2024-09-10T08:24:27.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.725+0000 7fbecfe00640 1 --2- 172.21.15.2:0/3931385076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbec8109120 0x7fbec8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.725+0000 7fbecfe00640 1 --2- 172.21.15.2:0/3931385076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbec81049a0 0x7fbec81089f0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.725+0000 7fbecfe00640 1 --2- 172.21.15.2:0/3931385076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbec8103ff0 0x7fbec81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.725+0000 7fbecfe00640 1 -- 172.21.15.2:0/3931385076 >> 172.21.15.2:0/3931385076 conn(0x7fbec80ffa90 msgr2=0x7fbec8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:27.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.725+0000 7fbecfe00640 1 -- 172.21.15.2:0/3931385076 shutdown_connections 2024-09-10T08:24:27.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.725+0000 7fbecfe00640 1 -- 172.21.15.2:0/3931385076 wait complete. 2024-09-10T08:24:27.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.726+0000 7fbecfe00640 1 Processor -- start 2024-09-10T08:24:27.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.726+0000 7fbecfe00640 1 -- start start 2024-09-10T08:24:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.726+0000 7fbecfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbec8103ff0 0x7fbec819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.726+0000 7fbecfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbec81049a0 0x7fbec819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.726+0000 7fbecfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbec8109120 0x7fbec819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.726+0000 7fbecfe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbec8114120 con 0x7fbec81049a0 2024-09-10T08:24:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.726+0000 7fbecfe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbec8113fa0 con 0x7fbec8103ff0 2024-09-10T08:24:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.726+0000 7fbecfe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81142a0 con 0x7fbec8109120 2024-09-10T08:24:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.727+0000 7fbecca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbec81049a0 0x7fbec819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.727+0000 7fbecca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbec81049a0 0x7fbec819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54092/0 (socket says 172.21.15.2:54092) 2024-09-10T08:24:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.727+0000 7fbecca00640 1 -- 172.21.15.2:0/3772933843 learned_addr learned my addr 172.21.15.2:0/3772933843 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.727+0000 7fbecd400640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbec8103ff0 0x7fbec819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.727+0000 7fbecde00640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbec8109120 0x7fbec819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.727+0000 7fbecca00640 1 -- 172.21.15.2:0/3772933843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbec8109120 msgr2=0x7fbec819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.727+0000 7fbecca00640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbec8109120 0x7fbec819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.727+0000 7fbecca00640 1 -- 172.21.15.2:0/3772933843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbec8103ff0 msgr2=0x7fbec819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.727+0000 7fbecca00640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbec8103ff0 0x7fbec819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.727+0000 7fbecca00640 1 -- 172.21.15.2:0/3772933843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec819f650 con 0x7fbec81049a0 2024-09-10T08:24:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.728+0000 7fbecca00640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbec81049a0 0x7fbec819abe0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4006010 tx=0x7fbeb40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:27.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.728+0000 7fbec6000640 1 -- 172.21.15.2:0/3772933843 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb403f030 con 0x7fbec81049a0 2024-09-10T08:24:27.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.728+0000 7fbec6000640 1 -- 172.21.15.2:0/3772933843 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb4003ce0 con 0x7fbec81049a0 2024-09-10T08:24:27.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.728+0000 7fbec6000640 1 -- 172.21.15.2:0/3772933843 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4035070 con 0x7fbec81049a0 2024-09-10T08:24:27.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.728+0000 7fbecfe00640 1 -- 172.21.15.2:0/3772933843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec819f8e0 con 0x7fbec81049a0 2024-09-10T08:24:27.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.728+0000 7fbecfe00640 1 -- 172.21.15.2:0/3772933843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec819fd70 con 0x7fbec81049a0 2024-09-10T08:24:27.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.732+0000 7fbed27e0740 1 Processor -- start 2024-09-10T08:24:27.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.732+0000 7fbed27e0740 1 -- start start 2024-09-10T08:24:27.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.732+0000 7fbed27e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc80050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.732+0000 7fbed27e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc80590 0x561b2cc8a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.732+0000 7fbed27e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc8ae50 0x561b2cc8d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.732+0000 7fbed27e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561b2cbe63e0 con 0x561b2cc80590 2024-09-10T08:24:27.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.732+0000 7fbed27e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561b2cb8dd80 con 0x561b2cc7fc70 2024-09-10T08:24:27.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.732+0000 7fbed27e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561b2cbe23a0 con 0x561b2cc8ae50 2024-09-10T08:24:27.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.732+0000 7fbecde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc80050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.732+0000 7fbecd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc80590 0x561b2cc8a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.732+0000 7fbecde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc80050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42892/0 (socket says 172.21.15.2:42892) 2024-09-10T08:24:27.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.732+0000 7fbecde00640 1 -- 172.21.15.2:0/3344763935 learned_addr learned my addr 172.21.15.2:0/3344763935 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.733+0000 7fbecd400640 1 -- 172.21.15.2:0/3344763935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc8ae50 msgr2=0x561b2cc8d240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:24:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.733+0000 7fbecd400640 1 --2- 172.21.15.2:0/3344763935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc8ae50 0x561b2cc8d240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.733+0000 7fbecd400640 1 -- 172.21.15.2:0/3344763935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 msgr2=0x561b2cc80050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.733+0000 7fbecd400640 1 --2- 172.21.15.2:0/3344763935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc80050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.733+0000 7fbecd400640 1 -- 172.21.15.2:0/3344763935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2cbacae0 con 0x561b2cc80590 2024-09-10T08:24:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.733+0000 7fbecd400640 1 --2- 172.21.15.2:0/3344763935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc80590 0x561b2cc8a910 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009c20 tx=0x7fbebc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a25500fc6c2c81a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:27.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.733+0000 7fbeba600640 1 -- 172.21.15.2:0/3344763935 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc009dd0 con 0x561b2cc80590 2024-09-10T08:24:27.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.734+0000 7fbeba600640 1 -- 172.21.15.2:0/3344763935 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbebc005020 con 0x561b2cc80590 2024-09-10T08:24:27.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.734+0000 7fbeba600640 1 -- 172.21.15.2:0/3344763935 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc004c40 con 0x561b2cc80590 2024-09-10T08:24:27.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.734+0000 7fbec6000640 1 -- 172.21.15.2:0/3772933843 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbeb4042030 con 0x7fbec81049a0 2024-09-10T08:24:27.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.734+0000 7fbed27e0740 1 -- 172.21.15.2:0/3344763935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc80590 msgr2=0x561b2cc8a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:27.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.734+0000 7fbed27e0740 1 --2- 172.21.15.2:0/3344763935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc80590 0x561b2cc8a910 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009c20 tx=0x7fbebc011810 comp rx=0 tx=0).stop 2024-09-10T08:24:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.734+0000 7fbed27e0740 1 -- 172.21.15.2:0/3344763935 shutdown_connections 2024-09-10T08:24:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.734+0000 7fbed27e0740 1 --2- 172.21.15.2:0/3344763935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc8ae50 0x561b2cc8d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.734+0000 7fbed27e0740 1 --2- 172.21.15.2:0/3344763935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc80590 0x561b2cc8a910 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.734+0000 7fbed27e0740 1 --2- 172.21.15.2:0/3344763935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc80050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.734+0000 7fbed27e0740 1 -- 172.21.15.2:0/3344763935 >> 172.21.15.2:0/3344763935 conn(0x561b2cc7c200 msgr2=0x561b2cc7c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.735+0000 7fbed27e0740 1 -- 172.21.15.2:0/3344763935 shutdown_connections 2024-09-10T08:24:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.735+0000 7fbed27e0740 1 -- 172.21.15.2:0/3344763935 wait complete. 2024-09-10T08:24:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.735+0000 7fbed27e0740 1 -- start start 2024-09-10T08:24:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.735+0000 7fbed27e0740 1 Processor -- start 2024-09-10T08:24:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbec6000640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbeb00a2a90 0x7fbeb00a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbec6000640 1 -- 172.21.15.2:0/3772933843 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbeb40ef9f0 con 0x7fbec81049a0 2024-09-10T08:24:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbed27e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc87d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbecde00640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbeb00a2a90 0x7fbeb00a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbed27e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc80590 0x561b2cc88280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbed27e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc8ae50 0x561b2cc887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbed27e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561b2cc8da20 con 0x561b2cc80590 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbed27e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561b2cba10f0 con 0x561b2cc7fc70 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbed27e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561b2cc8dba0 con 0x561b2cc8ae50 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbed27e0740 10 client.0 _ll_register_callbacks cb 0x561b2cb88bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbecca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc80590 0x561b2cc88280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbecd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc87d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbecca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc80590 0x561b2cc88280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54112/0 (socket says 172.21.15.2:54112) 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.736+0000 7fbecca00640 1 -- 172.21.15.2:0/2563081147 learned_addr learned my addr 172.21.15.2:0/2563081147 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.737+0000 7fbecde00640 1 --2- 172.21.15.2:0/2563081147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc8ae50 0x561b2cc887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.737+0000 7fbecca00640 1 -- 172.21.15.2:0/2563081147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc8ae50 msgr2=0x561b2cc887c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.737+0000 7fbecca00640 1 --2- 172.21.15.2:0/2563081147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc8ae50 0x561b2cc887c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.737+0000 7fbecca00640 1 -- 172.21.15.2:0/2563081147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 msgr2=0x561b2cc87d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.737+0000 7fbecca00640 1 --2- 172.21.15.2:0/2563081147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc87d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.737+0000 7fbecca00640 1 -- 172.21.15.2:0/2563081147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2c9573c0 con 0x561b2cc80590 2024-09-10T08:24:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.737+0000 7fbecd400640 1 --2- 172.21.15.2:0/2563081147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc87d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.737+0000 7fbecde00640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbeb00a2a90 0x7fbeb00a4f50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbec0005fd0 tx=0x7fbec0014b90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.737+0000 7fbecca00640 1 --2- 172.21.15.2:0/2563081147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc80590 0x561b2cc88280 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40c4600 tx=0x7fbeb4003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e6dd86901892dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.737+0000 7fbeaea00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:24:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.738+0000 7fbeaea00640 1 -- 172.21.15.2:0/2563081147 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4049040 con 0x561b2cc80590 2024-09-10T08:24:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.738+0000 7fbeaea00640 1 -- 172.21.15.2:0/2563081147 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbeb40da5e0 con 0x561b2cc80590 2024-09-10T08:24:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.738+0000 7fbed27e0740 1 -- 172.21.15.2:0/2563081147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbebc005020 con 0x561b2cc80590 2024-09-10T08:24:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.738+0000 7fbeaea00640 1 -- 172.21.15.2:0/2563081147 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbeb40bba00 con 0x561b2cc80590 2024-09-10T08:24:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.738+0000 7fbed27e0740 10 client.20148 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.738+0000 7fbed27e0740 1 -- 172.21.15.2:0/2563081147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561b2cc99a70 con 0x561b2cc80590 2024-09-10T08:24:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.738+0000 7fbeaea00640 1 -- 172.21.15.2:0/2563081147 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbebc005020 con 0x561b2cc80590 2024-09-10T08:24:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.738+0000 7fbeaea00640 1 -- 172.21.15.2:0/2563081147 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbeb40da2b0 con 0x561b2cc80590 2024-09-10T08:24:27.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.738+0000 7fbed27e0740 10 client.20148 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:27.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.738+0000 7fbed27e0740 10 client.20148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:27.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.739+0000 7fbed27e0740 20 client.20148 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:27.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.739+0000 7fbed27e0740 1 --2- 172.21.15.2:0/2563081147 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561b2cc9ae10 0x561b2cc9d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.739+0000 7fbed27e0740 4 client.20148 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:27.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.739+0000 7fbed27e0740 1 -- 172.21.15.2:0/2563081147 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b2cbf3950 con 0x561b2cc9ae10 2024-09-10T08:24:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.739+0000 7fbecd400640 1 --2- 172.21.15.2:0/2563081147 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561b2cc9ae10 0x561b2cc9d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.740+0000 7fbecd400640 1 --2- 172.21.15.2:0/2563081147 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561b2cc9ae10 0x561b2cc9d200 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ff14b29953aa49 server_cookie=6ef0f2da5dedbcb4 in_seq=0 out_seq=0 2024-09-10T08:24:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.740+0000 7fbeaea00640 10 client.20148 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.740+0000 7fbeaea00640 1 -- 172.21.15.2:0/2563081147 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b2cbf3950 con 0x561b2cc9ae10 2024-09-10T08:24:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.740+0000 7fbeaea00640 10 client.20148 handle_command_reply: tid=0 2024-09-10T08:24:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.741+0000 7fbed27e0740 1 client.20148 shutdown 2024-09-10T08:24:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.741+0000 7fbed27e0740 1 -- 172.21.15.2:0/2563081147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc80590 msgr2=0x561b2cc88280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.741+0000 7fbed27e0740 1 --2- 172.21.15.2:0/2563081147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc80590 0x561b2cc88280 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40c4600 tx=0x7fbeb4003e90 comp rx=0 tx=0).stop 2024-09-10T08:24:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.741+0000 7fbed27e0740 1 -- 172.21.15.2:0/2563081147 shutdown_connections 2024-09-10T08:24:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.741+0000 7fbed27e0740 1 --2- 172.21.15.2:0/2563081147 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561b2cc9ae10 0x561b2cc9d200 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.741+0000 7fbed27e0740 1 --2- 172.21.15.2:0/2563081147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc8ae50 0x561b2cc887c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.741+0000 7fbed27e0740 1 --2- 172.21.15.2:0/2563081147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc80590 0x561b2cc88280 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.741+0000 7fbed27e0740 1 --2- 172.21.15.2:0/2563081147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc87d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.741+0000 7fbed27e0740 1 -- 172.21.15.2:0/2563081147 >> 172.21.15.2:0/2563081147 conn(0x561b2cc7c200 msgr2=0x561b2cc81f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.741+0000 7fbeaea00640 0 client.20148 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.741+0000 7fbed27e0740 1 -- 172.21.15.2:0/2563081147 shutdown_connections 2024-09-10T08:24:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.741+0000 7fbed27e0740 1 -- 172.21.15.2:0/2563081147 wait complete. 2024-09-10T08:24:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.741+0000 7fbed27e0740 20 client.20148 trim_cache size 0 max 16384 2024-09-10T08:24:27.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.755+0000 7fbed27e0740 1 Processor -- start 2024-09-10T08:24:27.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.755+0000 7fbed27e0740 1 -- start start 2024-09-10T08:24:27.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.755+0000 7fbed27e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc7fc70 0x561b2cc83090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.755+0000 7fbed27e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc80590 0x561b2cc88610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.755+0000 7fbed27e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc8ae50 0x561b2cca2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.755+0000 7fbed27e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81b3890 con 0x561b2cc7fc70 2024-09-10T08:24:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.755+0000 7fbed27e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81b1160 con 0x561b2cc8ae50 2024-09-10T08:24:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.755+0000 7fbed27e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81aefb0 con 0x561b2cc80590 2024-09-10T08:24:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.756+0000 7fbecd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc7fc70 0x561b2cc83090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.756+0000 7fbecd400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc7fc70 0x561b2cc83090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54126/0 (socket says 172.21.15.2:54126) 2024-09-10T08:24:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.756+0000 7fbecd400640 1 -- 172.21.15.2:0/1292168976 learned_addr learned my addr 172.21.15.2:0/1292168976 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.756+0000 7fbecde00640 1 --2- 172.21.15.2:0/1292168976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc8ae50 0x561b2cca2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.756+0000 7fbecca00640 1 --2- 172.21.15.2:0/1292168976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc80590 0x561b2cc88610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.756+0000 7fbecd400640 1 -- 172.21.15.2:0/1292168976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc80590 msgr2=0x561b2cc88610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.756+0000 7fbecd400640 1 --2- 172.21.15.2:0/1292168976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc80590 0x561b2cc88610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.756+0000 7fbecd400640 1 -- 172.21.15.2:0/1292168976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc8ae50 msgr2=0x561b2cca2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.756+0000 7fbecd400640 1 --2- 172.21.15.2:0/1292168976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc8ae50 0x561b2cca2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.756+0000 7fbecd400640 1 -- 172.21.15.2:0/1292168976 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2cc77770 con 0x561b2cc7fc70 2024-09-10T08:24:27.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.756+0000 7fbecde00640 1 --2- 172.21.15.2:0/1292168976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc8ae50 0x561b2cca2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:27.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.757+0000 7fbecd400640 1 --2- 172.21.15.2:0/1292168976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc7fc70 0x561b2cc83090 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0048b0 tx=0x7fbebc0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b38633cf9325f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:27.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.757+0000 7fbeb9c00640 1 -- 172.21.15.2:0/1292168976 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc030070 con 0x561b2cc7fc70 2024-09-10T08:24:27.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.757+0000 7fbeb9c00640 1 -- 172.21.15.2:0/1292168976 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbebc004520 con 0x561b2cc7fc70 2024-09-10T08:24:27.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.757+0000 7fbeb9c00640 1 -- 172.21.15.2:0/1292168976 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc0035f0 con 0x561b2cc7fc70 2024-09-10T08:24:27.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.758+0000 7fbed27e0740 1 -- 172.21.15.2:0/1292168976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc7fc70 msgr2=0x561b2cc83090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:27.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.758+0000 7fbed27e0740 1 --2- 172.21.15.2:0/1292168976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc7fc70 0x561b2cc83090 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0048b0 tx=0x7fbebc0093f0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.758+0000 7fbed27e0740 1 -- 172.21.15.2:0/1292168976 shutdown_connections 2024-09-10T08:24:27.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.758+0000 7fbed27e0740 1 --2- 172.21.15.2:0/1292168976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc8ae50 0x561b2cca2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.758+0000 7fbed27e0740 1 --2- 172.21.15.2:0/1292168976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc80590 0x561b2cc88610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.758+0000 7fbed27e0740 1 --2- 172.21.15.2:0/1292168976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc7fc70 0x561b2cc83090 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.758+0000 7fbed27e0740 1 -- 172.21.15.2:0/1292168976 >> 172.21.15.2:0/1292168976 conn(0x561b2cc7c200 msgr2=0x561b2cc89dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:27.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.758+0000 7fbed27e0740 1 -- 172.21.15.2:0/1292168976 shutdown_connections 2024-09-10T08:24:27.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.758+0000 7fbed27e0740 1 -- 172.21.15.2:0/1292168976 wait complete. 2024-09-10T08:24:27.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.758+0000 7fbed27e0740 1 -- start start 2024-09-10T08:24:27.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.759+0000 7fbed27e0740 1 Processor -- start 2024-09-10T08:24:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.759+0000 7fbed27e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc93600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.759+0000 7fbed27e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc80590 0x561b2cc84ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.759+0000 7fbed27e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc8ae50 0x561b2cc84ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.759+0000 7fbed27e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81af620 con 0x561b2cc8ae50 2024-09-10T08:24:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.759+0000 7fbed27e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81b0a90 con 0x561b2cc7fc70 2024-09-10T08:24:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.759+0000 7fbed27e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81afcf0 con 0x561b2cc80590 2024-09-10T08:24:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.759+0000 7fbed27e0740 10 client.0 _ll_register_callbacks cb 0x7fbeb4005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.759+0000 7fbecd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc93600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.759+0000 7fbecde00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc8ae50 0x561b2cc84ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.759+0000 7fbecca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc80590 0x561b2cc84ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.760+0000 7fbecd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc93600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42912/0 (socket says 172.21.15.2:42912) 2024-09-10T08:24:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.760+0000 7fbecca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc80590 0x561b2cc84ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50656/0 (socket says 172.21.15.2:50656) 2024-09-10T08:24:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.760+0000 7fbecca00640 1 -- 172.21.15.2:0/1948356565 learned_addr learned my addr 172.21.15.2:0/1948356565 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.760+0000 7fbecde00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc8ae50 0x561b2cc84ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54134/0 (socket says 172.21.15.2:54134) 2024-09-10T08:24:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.760+0000 7fbecca00640 1 -- 172.21.15.2:0/1948356565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 msgr2=0x561b2cc93600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.760+0000 7fbecca00640 1 --2- 172.21.15.2:0/1948356565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc93600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.760+0000 7fbecca00640 1 -- 172.21.15.2:0/1948356565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc8ae50 msgr2=0x561b2cc84ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.760+0000 7fbecca00640 1 --2- 172.21.15.2:0/1948356565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc8ae50 0x561b2cc84ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.760+0000 7fbecca00640 1 -- 172.21.15.2:0/1948356565 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2cc857f0 con 0x561b2cc80590 2024-09-10T08:24:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.760+0000 7fbecd400640 1 --2- 172.21.15.2:0/1948356565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc93600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.760+0000 7fbecca00640 1 --2- 172.21.15.2:0/1948356565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc80590 0x561b2cc84ab0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4004650 tx=0x7fbeb40c46a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=485a5863255b106 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.760+0000 7fbeaea00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:24:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.760+0000 7fbeaea00640 1 -- 172.21.15.2:0/1948356565 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4049070 con 0x561b2cc80590 2024-09-10T08:24:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.761+0000 7fbeaea00640 1 -- 172.21.15.2:0/1948356565 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fbeb403bdb0 con 0x561b2cc80590 2024-09-10T08:24:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.761+0000 7fbed27e0740 1 -- 172.21.15.2:0/1948356565 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbebc004520 con 0x561b2cc80590 2024-09-10T08:24:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.761+0000 7fbeaea00640 1 -- 172.21.15.2:0/1948356565 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbeb4081050 con 0x561b2cc80590 2024-09-10T08:24:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.761+0000 7fbed27e0740 10 client.30100 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.761+0000 7fbed27e0740 1 -- 172.21.15.2:0/1948356565 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561b2cd05ae0 con 0x561b2cc80590 2024-09-10T08:24:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.761+0000 7fbeaea00640 1 -- 172.21.15.2:0/1948356565 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbebc004520 con 0x561b2cc80590 2024-09-10T08:24:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.761+0000 7fbeaea00640 1 -- 172.21.15.2:0/1948356565 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbeb40bb310 con 0x561b2cc80590 2024-09-10T08:24:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.761+0000 7fbed27e0740 10 client.30100 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.761+0000 7fbed27e0740 10 client.30100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.762+0000 7fbed27e0740 20 client.30100 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.762+0000 7fbed27e0740 1 --2- 172.21.15.2:0/1948356565 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561b2cd060a0 0x561b2cd084c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.762+0000 7fbed27e0740 4 client.30100 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:24:27.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.762+0000 7fbed27e0740 1 -- 172.21.15.2:0/1948356565 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b2cbede60 con 0x561b2cd060a0 2024-09-10T08:24:27.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.762+0000 7fbecd400640 1 --2- 172.21.15.2:0/1948356565 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561b2cd060a0 0x561b2cd084c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:27.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbecd400640 1 --2- 172.21.15.2:0/1948356565 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561b2cd060a0 0x561b2cd084c0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88233af1e15b3724 server_cookie=fd3941beff359417 in_seq=0 out_seq=0 2024-09-10T08:24:27.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbeaea00640 10 client.30100 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:27.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbeaea00640 1 -- 172.21.15.2:0/1948356565 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b2cbede60 con 0x561b2cd060a0 2024-09-10T08:24:27.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbeaea00640 10 client.30100 handle_command_reply: tid=0 2024-09-10T08:24:27.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbed27e0740 1 client.30100 shutdown 2024-09-10T08:24:27.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbed27e0740 1 -- 172.21.15.2:0/1948356565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc80590 msgr2=0x561b2cc84ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:27.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbed27e0740 1 --2- 172.21.15.2:0/1948356565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc80590 0x561b2cc84ab0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4004650 tx=0x7fbeb40c46a0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbed27e0740 1 -- 172.21.15.2:0/1948356565 shutdown_connections 2024-09-10T08:24:27.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbed27e0740 1 --2- 172.21.15.2:0/1948356565 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x561b2cd060a0 0x561b2cd084c0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbed27e0740 1 --2- 172.21.15.2:0/1948356565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b2cc8ae50 0x561b2cc84ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbed27e0740 1 --2- 172.21.15.2:0/1948356565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b2cc80590 0x561b2cc84ab0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbed27e0740 1 --2- 172.21.15.2:0/1948356565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b2cc7fc70 0x561b2cc93600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbed27e0740 1 -- 172.21.15.2:0/1948356565 >> 172.21.15.2:0/1948356565 conn(0x561b2cc7c200 msgr2=0x561b2cc7ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:27.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbeaea00640 0 client.30100 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:27.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbed27e0740 1 -- 172.21.15.2:0/1948356565 shutdown_connections 2024-09-10T08:24:27.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbed27e0740 1 -- 172.21.15.2:0/1948356565 wait complete. 2024-09-10T08:24:27.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.763+0000 7fbed27e0740 20 client.30100 trim_cache size 0 max 16384 2024-09-10T08:24:27.767 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:24:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.764+0000 7fbecfe00640 1 -- 172.21.15.2:0/3772933843 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbeb00a2a90 msgr2=0x7fbeb00a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.764+0000 7fbecfe00640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbeb00a2a90 0x7fbeb00a4f50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbec0005fd0 tx=0x7fbec0014b90 comp rx=0 tx=0).stop 2024-09-10T08:24:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.764+0000 7fbecfe00640 1 -- 172.21.15.2:0/3772933843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbec81049a0 msgr2=0x7fbec819abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.764+0000 7fbecfe00640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbec81049a0 0x7fbec819abe0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4006010 tx=0x7fbeb40042f0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.764+0000 7fbecfe00640 1 -- 172.21.15.2:0/3772933843 shutdown_connections 2024-09-10T08:24:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.764+0000 7fbecfe00640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fbeb00a2a90 0x7fbeb00a4f50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.764+0000 7fbecfe00640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbec8109120 0x7fbec819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.764+0000 7fbecfe00640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbec81049a0 0x7fbec819abe0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.764+0000 7fbecfe00640 1 --2- 172.21.15.2:0/3772933843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbec8103ff0 0x7fbec819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:27.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.764+0000 7fbecfe00640 1 -- 172.21.15.2:0/3772933843 >> 172.21.15.2:0/3772933843 conn(0x7fbec80ffa90 msgr2=0x7fbec8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:27.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.765+0000 7fbecfe00640 1 -- 172.21.15.2:0/3772933843 shutdown_connections 2024-09-10T08:24:27.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:27.765+0000 7fbecfe00640 1 -- 172.21.15.2:0/3772933843 wait complete. 2024-09-10T08:24:27.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:27 smithi155 ceph-mon[35405]: pgmap v876: 129 pgs: 129 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-09-10T08:24:27.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:27 smithi107 ceph-mon[35048]: pgmap v876: 129 pgs: 129 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-09-10T08:24:28.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:27 smithi002 ceph-mon[30563]: pgmap v876: 129 pgs: 129 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-09-10T08:24:28.484 DEBUG:teuthology.parallel:result is None 2024-09-10T08:24:28.484 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T08:24:28.552 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T08:24:28.552 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-10T08:24:28.563 INFO:tasks.quiescer:joining Quiescers 2024-09-10T08:24:28.563 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-10T08:24:28.563 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:24:28.950 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.708+0000 7f0508c00640 1 -- 172.21.15.2:0/546049734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05041049a0 msgr2=0x7f05041089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.708+0000 7f0508c00640 1 --2- 172.21.15.2:0/546049734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05041049a0 0x7f05041089f0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009900 tx=0x7f04f002a410 comp rx=0 tx=0).stop 2024-09-10T08:24:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.708+0000 7f0508c00640 1 -- 172.21.15.2:0/546049734 shutdown_connections 2024-09-10T08:24:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.708+0000 7f0508c00640 1 --2- 172.21.15.2:0/546049734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0504109120 0x7f0504110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.708+0000 7f0508c00640 1 --2- 172.21.15.2:0/546049734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05041049a0 0x7f05041089f0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.708+0000 7f0508c00640 1 --2- 172.21.15.2:0/546049734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0504103ff0 0x7f05041043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.708+0000 7f0508c00640 1 -- 172.21.15.2:0/546049734 >> 172.21.15.2:0/546049734 conn(0x7f05040ffa90 msgr2=0x7f0504101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:29.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.708+0000 7f0508c00640 1 -- 172.21.15.2:0/546049734 shutdown_connections 2024-09-10T08:24:29.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.709+0000 7f0508c00640 1 -- 172.21.15.2:0/546049734 wait complete. 2024-09-10T08:24:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.709+0000 7f0508c00640 1 Processor -- start 2024-09-10T08:24:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.710+0000 7f0508c00640 1 -- start start 2024-09-10T08:24:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.710+0000 7f0508c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0504103ff0 0x7f050419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.710+0000 7f0508c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05041049a0 0x7f050419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.710+0000 7f0508c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0504109120 0x7f050419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.710+0000 7f0508c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0504114140 con 0x7f05041049a0 2024-09-10T08:24:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.710+0000 7f0508c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0504113fc0 con 0x7f0504103ff0 2024-09-10T08:24:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.710+0000 7f0508c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f05041142c0 con 0x7f0504109120 2024-09-10T08:24:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.710+0000 7f0501400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05041049a0 0x7f050419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.710+0000 7f0501400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05041049a0 0x7f050419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54154/0 (socket says 172.21.15.2:54154) 2024-09-10T08:24:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.710+0000 7f0501400640 1 -- 172.21.15.2:0/844249465 learned_addr learned my addr 172.21.15.2:0/844249465 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.710+0000 7f0501e00640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0504103ff0 0x7f050419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.710+0000 7f0502800640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0504109120 0x7f050419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.711+0000 7f0501e00640 1 -- 172.21.15.2:0/844249465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0504109120 msgr2=0x7f050419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.711+0000 7f0501e00640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0504109120 0x7f050419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.711+0000 7f0501e00640 1 -- 172.21.15.2:0/844249465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05041049a0 msgr2=0x7f050419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.711+0000 7f0501e00640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05041049a0 0x7f050419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.711+0000 7f0501e00640 1 -- 172.21.15.2:0/844249465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050419f5d0 con 0x7f0504103ff0 2024-09-10T08:24:29.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.711+0000 7f0502800640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0504109120 0x7f050419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:29.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.711+0000 7f0501400640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05041049a0 0x7f050419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:29.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.711+0000 7f0501e00640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0504103ff0 0x7f050419a620 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f04f400e9f0 tx=0x7f04f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:29.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.711+0000 7f04faa00640 1 -- 172.21.15.2:0/844249465 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f400ccc0 con 0x7f0504103ff0 2024-09-10T08:24:29.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.711+0000 7f04faa00640 1 -- 172.21.15.2:0/844249465 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f4005020 con 0x7f0504103ff0 2024-09-10T08:24:29.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.711+0000 7f0508c00640 1 -- 172.21.15.2:0/844249465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050419f8c0 con 0x7f0504103ff0 2024-09-10T08:24:29.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.711+0000 7f04faa00640 1 -- 172.21.15.2:0/844249465 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f40049e0 con 0x7f0504103ff0 2024-09-10T08:24:29.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.712+0000 7f0508c00640 1 -- 172.21.15.2:0/844249465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05041a71a0 con 0x7f0504103ff0 2024-09-10T08:24:29.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.713+0000 7f0508c00640 1 -- 172.21.15.2:0/844249465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05041043d0 con 0x7f0504103ff0 2024-09-10T08:24:29.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.714+0000 7f04faa00640 1 -- 172.21.15.2:0/844249465 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f04f4005560 con 0x7f0504103ff0 2024-09-10T08:24:29.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.716+0000 7f04faa00640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04e80a2c00 0x7f04e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:29.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.716+0000 7f04faa00640 1 -- 172.21.15.2:0/844249465 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f04f40cf3f0 con 0x7f0504103ff0 2024-09-10T08:24:29.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.716+0000 7f0501400640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04e80a2c00 0x7f04e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:29.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.717+0000 7f0501400640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04e80a2c00 0x7f04e80a50c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f04f0031550 tx=0x7f04f0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:29.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.718+0000 7f04faa00640 1 -- 172.21.15.2:0/844249465 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04f40051d0 con 0x7f0504103ff0 2024-09-10T08:24:29.730 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:24:29.731 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:24:29.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:29 smithi155 ceph-mon[35405]: pgmap v877: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:24:29.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:29 smithi107 ceph-mon[35048]: pgmap v877: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:24:29.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.938+0000 7f0508c00640 1 -- 172.21.15.2:0/844249465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f050410c3c0 con 0x7f0504103ff0 2024-09-10T08:24:29.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:29 smithi002 ceph-mon[30563]: pgmap v877: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-10T08:24:29.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.939+0000 7f04faa00640 1 -- 172.21.15.2:0/844249465 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f04f409bf00 con 0x7f0504103ff0 2024-09-10T08:24:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:24:29.940 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:24:29.941 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:24:29.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.943+0000 7f0508c00640 1 -- 172.21.15.2:0/844249465 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04e80a2c00 msgr2=0x7f04e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:29.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.943+0000 7f0508c00640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04e80a2c00 0x7f04e80a50c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f04f0031550 tx=0x7f04f0005a20 comp rx=0 tx=0).stop 2024-09-10T08:24:29.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.943+0000 7f0508c00640 1 -- 172.21.15.2:0/844249465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0504103ff0 msgr2=0x7f050419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:29.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.943+0000 7f0508c00640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0504103ff0 0x7f050419a620 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f04f400e9f0 tx=0x7f04f400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:24:29.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.943+0000 7f0508c00640 1 -- 172.21.15.2:0/844249465 shutdown_connections 2024-09-10T08:24:29.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.943+0000 7f0508c00640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f04e80a2c00 0x7f04e80a50c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:29.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.943+0000 7f0508c00640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0504109120 0x7f050419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:29.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.943+0000 7f0508c00640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05041049a0 0x7f050419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:29.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.943+0000 7f0508c00640 1 --2- 172.21.15.2:0/844249465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0504103ff0 0x7f050419a620 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:29.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.943+0000 7f0508c00640 1 -- 172.21.15.2:0/844249465 >> 172.21.15.2:0/844249465 conn(0x7f05040ffa90 msgr2=0x7f0504100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:29.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.944+0000 7f0508c00640 1 -- 172.21.15.2:0/844249465 shutdown_connections 2024-09-10T08:24:29.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:29.944+0000 7f0508c00640 1 -- 172.21.15.2:0/844249465 wait complete. 2024-09-10T08:24:30.736 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24536 quiesce db --cancel --all' 2024-09-10T08:24:30.736 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24536 quiesce db --cancel --all 2024-09-10T08:24:30.765 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:30.905 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:30 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/844249465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:30.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:30 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/844249465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:30.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:30 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/844249465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:31.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.428+0000 7fe3ae000640 1 -- 172.21.15.2:0/2546159078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3a8103ff0 msgr2=0x7fe3a81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.428+0000 7fe3ae000640 1 --2- 172.21.15.2:0/2546159078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3a8103ff0 0x7fe3a81043d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe398009900 tx=0x7fe39802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.428+0000 7fe3ae000640 1 -- 172.21.15.2:0/2546159078 shutdown_connections 2024-09-10T08:24:31.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.428+0000 7fe3ae000640 1 --2- 172.21.15.2:0/2546159078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a8109120 0x7fe3a8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.428+0000 7fe3ae000640 1 --2- 172.21.15.2:0/2546159078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a81049a0 0x7fe3a81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.428+0000 7fe3ae000640 1 --2- 172.21.15.2:0/2546159078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3a8103ff0 0x7fe3a81043d0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.428+0000 7fe3ae000640 1 -- 172.21.15.2:0/2546159078 >> 172.21.15.2:0/2546159078 conn(0x7fe3a80ffa90 msgr2=0x7fe3a8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:31.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.428+0000 7fe3ae000640 1 -- 172.21.15.2:0/2546159078 shutdown_connections 2024-09-10T08:24:31.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.428+0000 7fe3ae000640 1 -- 172.21.15.2:0/2546159078 wait complete. 2024-09-10T08:24:31.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.429+0000 7fe3ae000640 1 Processor -- start 2024-09-10T08:24:31.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.430+0000 7fe3ae000640 1 -- start start 2024-09-10T08:24:31.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.430+0000 7fe3ae000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3a8103ff0 0x7fe3a819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.430+0000 7fe3ae000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a81049a0 0x7fe3a819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.430+0000 7fe3ae000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8109120 0x7fe3a819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.430+0000 7fe3ae000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8114170 con 0x7fe3a8103ff0 2024-09-10T08:24:31.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.430+0000 7fe3ae000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8113ff0 con 0x7fe3a8109120 2024-09-10T08:24:31.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.430+0000 7fe3ae000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81142f0 con 0x7fe3a81049a0 2024-09-10T08:24:31.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.431+0000 7fe3a6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a81049a0 0x7fe3a819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.431+0000 7fe3a7400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3a8103ff0 0x7fe3a819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.431+0000 7fe3a6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a81049a0 0x7fe3a819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50692/0 (socket says 172.21.15.2:50692) 2024-09-10T08:24:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.431+0000 7fe3a6a00640 1 -- 172.21.15.2:0/2009927692 learned_addr learned my addr 172.21.15.2:0/2009927692 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.431+0000 7fe3a7e00640 1 --2- 172.21.15.2:0/2009927692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8109120 0x7fe3a819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.431+0000 7fe3a6a00640 1 -- 172.21.15.2:0/2009927692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8109120 msgr2=0x7fe3a819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.431+0000 7fe3a6a00640 1 --2- 172.21.15.2:0/2009927692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8109120 0x7fe3a819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.431+0000 7fe3a6a00640 1 -- 172.21.15.2:0/2009927692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3a8103ff0 msgr2=0x7fe3a819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.431+0000 7fe3a6a00640 1 --2- 172.21.15.2:0/2009927692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3a8103ff0 0x7fe3a819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.431+0000 7fe3a6a00640 1 -- 172.21.15.2:0/2009927692 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a819f620 con 0x7fe3a81049a0 2024-09-10T08:24:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.432+0000 7fe3a6a00640 1 --2- 172.21.15.2:0/2009927692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a81049a0 0x7fe3a819abb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe39400e9f0 tx=0x7fe39400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:31.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.432+0000 7fe39fe00640 1 -- 172.21.15.2:0/2009927692 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39400cd80 con 0x7fe3a81049a0 2024-09-10T08:24:31.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.432+0000 7fe39fe00640 1 -- 172.21.15.2:0/2009927692 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe394010910 con 0x7fe3a81049a0 2024-09-10T08:24:31.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.432+0000 7fe3ae000640 1 -- 172.21.15.2:0/2009927692 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a819f910 con 0x7fe3a81049a0 2024-09-10T08:24:31.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.432+0000 7fe39fe00640 1 -- 172.21.15.2:0/2009927692 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394018990 con 0x7fe3a81049a0 2024-09-10T08:24:31.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.432+0000 7fe3ae000640 1 -- 172.21.15.2:0/2009927692 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a81a71f0 con 0x7fe3a81049a0 2024-09-10T08:24:31.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.436+0000 7fe3b0c68740 1 Processor -- start 2024-09-10T08:24:31.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.436+0000 7fe39fe00640 1 -- 172.21.15.2:0/2009927692 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe394010ac0 con 0x7fe3a81049a0 2024-09-10T08:24:31.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.436+0000 7fe3b0c68740 1 -- start start 2024-09-10T08:24:31.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.436+0000 7fe3b0c68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4da3ac0 0x5575b4da3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.436+0000 7fe3b0c68740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da43e0 0x5575b4dae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.436+0000 7fe3b0c68740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4daeca0 0x5575b4db1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.436+0000 7fe3b0c68740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5575b4d0a290 con 0x5575b4daeca0 2024-09-10T08:24:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.436+0000 7fe3b0c68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5575b4cb1b40 con 0x5575b4da3ac0 2024-09-10T08:24:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.436+0000 7fe3b0c68740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5575b4d06250 con 0x5575b4da43e0 2024-09-10T08:24:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.437+0000 7fe3a7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da43e0 0x5575b4dae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.437+0000 7fe3a7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da43e0 0x5575b4dae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50700/0 (socket says 172.21.15.2:50700) 2024-09-10T08:24:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.437+0000 7fe3a7400640 1 -- 172.21.15.2:0/3576507137 learned_addr learned my addr 172.21.15.2:0/3576507137 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.437+0000 7fe3a7e00640 1 --2- 172.21.15.2:0/3576507137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4da3ac0 0x5575b4da3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.437+0000 7fe3a6a00640 1 --2- 172.21.15.2:0/3576507137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4daeca0 0x5575b4db1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.437+0000 7fe3a7400640 1 -- 172.21.15.2:0/3576507137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4da3ac0 msgr2=0x5575b4da3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.437+0000 7fe3a7400640 1 --2- 172.21.15.2:0/3576507137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4da3ac0 0x5575b4da3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.437+0000 7fe3a7400640 1 -- 172.21.15.2:0/3576507137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4daeca0 msgr2=0x5575b4db1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.437+0000 7fe3a7400640 1 --2- 172.21.15.2:0/3576507137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4daeca0 0x5575b4db1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.437+0000 7fe3a7400640 1 -- 172.21.15.2:0/3576507137 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b4a7b6a0 con 0x5575b4da43e0 2024-09-10T08:24:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.437+0000 7fe3a7400640 1 --2- 172.21.15.2:0/3576507137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da43e0 0x5575b4dae760 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe398009900 tx=0x7fe398005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6b53bf4ee146c03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.438+0000 7fe393e00640 1 -- 172.21.15.2:0/3576507137 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398035040 con 0x5575b4da43e0 2024-09-10T08:24:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.438+0000 7fe39fe00640 1 --2- 172.21.15.2:0/2009927692 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe38c0a2c00 0x7fe38c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.438+0000 7fe393e00640 1 -- 172.21.15.2:0/3576507137 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe398004400 con 0x5575b4da43e0 2024-09-10T08:24:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.438+0000 7fe393e00640 1 -- 172.21.15.2:0/3576507137 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3980035f0 con 0x5575b4da43e0 2024-09-10T08:24:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.438+0000 7fe39fe00640 1 -- 172.21.15.2:0/2009927692 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe3940cf750 con 0x7fe3a81049a0 2024-09-10T08:24:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.438+0000 7fe3a7e00640 1 --2- 172.21.15.2:0/2009927692 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe38c0a2c00 0x7fe38c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.439+0000 7fe3b0c68740 1 -- 172.21.15.2:0/3576507137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da43e0 msgr2=0x5575b4dae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.439+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/3576507137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da43e0 0x5575b4dae760 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe398009900 tx=0x7fe398005de0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.439+0000 7fe3b0c68740 1 -- 172.21.15.2:0/3576507137 shutdown_connections 2024-09-10T08:24:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.439+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/3576507137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4daeca0 0x5575b4db1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.439+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/3576507137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da43e0 0x5575b4dae760 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.439+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/3576507137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4da3ac0 0x5575b4da3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.439+0000 7fe3b0c68740 1 -- 172.21.15.2:0/3576507137 >> 172.21.15.2:0/3576507137 conn(0x5575b4da0050 msgr2=0x5575b4da0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.439+0000 7fe3a7e00640 1 --2- 172.21.15.2:0/2009927692 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe38c0a2c00 0x7fe38c0a50c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0005fd0 tx=0x7fe3a0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.439+0000 7fe3b0c68740 1 -- 172.21.15.2:0/3576507137 shutdown_connections 2024-09-10T08:24:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.439+0000 7fe3b0c68740 1 -- 172.21.15.2:0/3576507137 wait complete. 2024-09-10T08:24:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.440+0000 7fe3b0c68740 1 -- start start 2024-09-10T08:24:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.440+0000 7fe3b0c68740 1 Processor -- start 2024-09-10T08:24:31.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.440+0000 7fe3b0c68740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4dae550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.440+0000 7fe3b0c68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4da43e0 0x5575b4dab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.440+0000 7fe3b0c68740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4daeca0 0x5575b4dabd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.440+0000 7fe3b0c68740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5575b4db1910 con 0x5575b4daeca0 2024-09-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.440+0000 7fe3b0c68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5575b4cc4fa0 con 0x5575b4da43e0 2024-09-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.440+0000 7fe3b0c68740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5575b4db1a90 con 0x5575b4da3ac0 2024-09-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.441+0000 7fe3b0c68740 10 client.0 _ll_register_callbacks cb 0x5575b4a18ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.441+0000 7fe3a7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4dae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.441+0000 7fe3a7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4daeca0 0x5575b4dabd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.441+0000 7fe3a7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4dae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50710/0 (socket says 172.21.15.2:50710) 2024-09-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.441+0000 7fe3a7400640 1 -- 172.21.15.2:0/636693657 learned_addr learned my addr 172.21.15.2:0/636693657 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.441+0000 7fe3a6a00640 1 --2- 172.21.15.2:0/636693657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4da43e0 0x5575b4dab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.441+0000 7fe3a7400640 1 -- 172.21.15.2:0/636693657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4da43e0 msgr2=0x5575b4dab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.441+0000 7fe3a7400640 1 --2- 172.21.15.2:0/636693657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4da43e0 0x5575b4dab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.441+0000 7fe3a7400640 1 -- 172.21.15.2:0/636693657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4daeca0 msgr2=0x5575b4dabd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.441+0000 7fe3a7400640 1 --2- 172.21.15.2:0/636693657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4daeca0 0x5575b4dabd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.441+0000 7fe3a7400640 1 -- 172.21.15.2:0/636693657 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575b4dac570 con 0x5575b4da3ac0 2024-09-10T08:24:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.442+0000 7fe3a7e00640 1 --2- 172.21.15.2:0/636693657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4daeca0 0x5575b4dabd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.442+0000 7fe3a7400640 1 --2- 172.21.15.2:0/636693657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4dae550 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe3980048f0 tx=0x7fe39802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daff771007c90bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.442+0000 7fe390c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:24:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.442+0000 7fe390c00640 1 -- 172.21.15.2:0/636693657 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398052060 con 0x5575b4da3ac0 2024-09-10T08:24:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.442+0000 7fe3b0c68740 1 -- 172.21.15.2:0/636693657 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe398004400 con 0x5575b4da3ac0 2024-09-10T08:24:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.442+0000 7fe390c00640 1 -- 172.21.15.2:0/636693657 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe398004200 con 0x5575b4da3ac0 2024-09-10T08:24:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.442+0000 7fe390c00640 1 -- 172.21.15.2:0/636693657 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe398035070 con 0x5575b4da3ac0 2024-09-10T08:24:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.443+0000 7fe3b0c68740 10 client.30118 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:31.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.443+0000 7fe3b0c68740 1 -- 172.21.15.2:0/636693657 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5575b4db1910 con 0x5575b4da3ac0 2024-09-10T08:24:31.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.443+0000 7fe390c00640 1 -- 172.21.15.2:0/636693657 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe398004400 con 0x5575b4da3ac0 2024-09-10T08:24:31.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.443+0000 7fe390c00640 1 -- 172.21.15.2:0/636693657 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe398031a00 con 0x5575b4da3ac0 2024-09-10T08:24:31.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.443+0000 7fe3b0c68740 10 client.30118 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:31.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.443+0000 7fe3b0c68740 10 client.30118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:31.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.443+0000 7fe3b0c68740 20 client.30118 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:31.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.443+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/636693657 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575b4dbe950 0x5575b4dc0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.443+0000 7fe3b0c68740 4 client.30118 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.443+0000 7fe3b0c68740 1 -- 172.21.15.2:0/636693657 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575b4d17800 con 0x5575b4dbe950 2024-09-10T08:24:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.444+0000 7fe3a6a00640 1 --2- 172.21.15.2:0/636693657 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575b4dbe950 0x5575b4dc0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.444+0000 7fe3a6a00640 1 --2- 172.21.15.2:0/636693657 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575b4dbe950 0x5575b4dc0d40 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72ac8bcf7184ef server_cookie=5a8aaab1a16fd0ed in_seq=0 out_seq=0 2024-09-10T08:24:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.444+0000 7fe390c00640 10 client.30118 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.445+0000 7fe390c00640 1 -- 172.21.15.2:0/636693657 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575b4d17800 con 0x5575b4dbe950 2024-09-10T08:24:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.445+0000 7fe390c00640 10 client.30118 handle_command_reply: tid=0 2024-09-10T08:24:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.445+0000 7fe3b0c68740 1 client.30118 shutdown 2024-09-10T08:24:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.445+0000 7fe3b0c68740 1 -- 172.21.15.2:0/636693657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 msgr2=0x5575b4dae550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.445+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/636693657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4dae550 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe3980048f0 tx=0x7fe39802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.445+0000 7fe3b0c68740 1 -- 172.21.15.2:0/636693657 shutdown_connections 2024-09-10T08:24:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.445+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/636693657 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575b4dbe950 0x5575b4dc0d40 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.445+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/636693657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4daeca0 0x5575b4dabd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.445+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/636693657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4da43e0 0x5575b4dab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.445+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/636693657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4dae550 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.445+0000 7fe390c00640 0 client.30118 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.445+0000 7fe3b0c68740 1 -- 172.21.15.2:0/636693657 >> 172.21.15.2:0/636693657 conn(0x5575b4da0050 msgr2=0x5575b4d9f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.446+0000 7fe3b0c68740 1 -- 172.21.15.2:0/636693657 shutdown_connections 2024-09-10T08:24:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.446+0000 7fe3b0c68740 1 -- 172.21.15.2:0/636693657 wait complete. 2024-09-10T08:24:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.446+0000 7fe3b0c68740 20 client.30118 trim_cache size 0 max 16384 2024-09-10T08:24:31.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.466+0000 7fe3b0c68740 1 Processor -- start 2024-09-10T08:24:31.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.467+0000 7fe3b0c68740 1 -- start start 2024-09-10T08:24:31.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.467+0000 7fe3b0c68740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4d9f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.467+0000 7fe3b0c68740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4da43e0 0x5575b4db6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.467+0000 7fe3b0c68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4daeca0 0x5575b4db7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.467+0000 7fe3b0c68740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81b3910 con 0x5575b4da43e0 2024-09-10T08:24:31.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.467+0000 7fe3b0c68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81b11e0 con 0x5575b4daeca0 2024-09-10T08:24:31.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.467+0000 7fe3b0c68740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81af030 con 0x5575b4da3ac0 2024-09-10T08:24:31.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.468+0000 7fe3a7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4d9f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.468+0000 7fe3a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4daeca0 0x5575b4db7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.468+0000 7fe3a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4daeca0 0x5575b4db7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42982/0 (socket says 172.21.15.2:42982) 2024-09-10T08:24:31.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.468+0000 7fe3a7e00640 1 -- 172.21.15.2:0/174834497 learned_addr learned my addr 172.21.15.2:0/174834497 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:31.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.468+0000 7fe3a7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4d9f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50714/0 (socket says 172.21.15.2:50714) 2024-09-10T08:24:31.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.468+0000 7fe3a6a00640 1 --2- 172.21.15.2:0/174834497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4da43e0 0x5575b4db6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.468+0000 7fe3a7400640 1 -- 172.21.15.2:0/174834497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4daeca0 msgr2=0x5575b4db7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.468+0000 7fe3a7400640 1 --2- 172.21.15.2:0/174834497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4daeca0 0x5575b4db7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.468+0000 7fe3a7400640 1 -- 172.21.15.2:0/174834497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4da43e0 msgr2=0x5575b4db6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.468+0000 7fe3a7400640 1 --2- 172.21.15.2:0/174834497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4da43e0 0x5575b4db6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.468+0000 7fe3a7400640 1 -- 172.21.15.2:0/174834497 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b4d9e5d0 con 0x5575b4da3ac0 2024-09-10T08:24:31.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.468+0000 7fe3a6a00640 1 --2- 172.21.15.2:0/174834497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4da43e0 0x5575b4db6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:31.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.469+0000 7fe3a7400640 1 --2- 172.21.15.2:0/174834497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4d9f0b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe398031dd0 tx=0x7fe398005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96e86b888e71c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:31.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.469+0000 7fe393400640 1 -- 172.21.15.2:0/174834497 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398052070 con 0x5575b4da3ac0 2024-09-10T08:24:31.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.469+0000 7fe393400640 1 -- 172.21.15.2:0/174834497 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe398031a90 con 0x5575b4da3ac0 2024-09-10T08:24:31.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.469+0000 7fe393400640 1 -- 172.21.15.2:0/174834497 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398005240 con 0x5575b4da3ac0 2024-09-10T08:24:31.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.470+0000 7fe3b0c68740 1 -- 172.21.15.2:0/174834497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 msgr2=0x5575b4d9f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.470+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/174834497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4d9f0b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe398031dd0 tx=0x7fe398005de0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.470+0000 7fe3b0c68740 1 -- 172.21.15.2:0/174834497 shutdown_connections 2024-09-10T08:24:31.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.470+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/174834497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4daeca0 0x5575b4db7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.470+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/174834497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4da43e0 0x5575b4db6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.470+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/174834497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4d9f0b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.470+0000 7fe3b0c68740 1 -- 172.21.15.2:0/174834497 >> 172.21.15.2:0/174834497 conn(0x5575b4da0050 msgr2=0x5575b4d9e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.470+0000 7fe3b0c68740 1 -- 172.21.15.2:0/174834497 shutdown_connections 2024-09-10T08:24:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.470+0000 7fe3b0c68740 1 -- 172.21.15.2:0/174834497 wait complete. 2024-09-10T08:24:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.471+0000 7fe3b0c68740 1 -- start start 2024-09-10T08:24:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.471+0000 7fe3b0c68740 1 Processor -- start 2024-09-10T08:24:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.472+0000 7fe3b0c68740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4db71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.472+0000 7fe3b0c68740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4da43e0 0x5575b4db5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.472+0000 7fe3b0c68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4daeca0 0x5575b4db5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.472+0000 7fe3b0c68740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81af6a0 con 0x5575b4da43e0 2024-09-10T08:24:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.472+0000 7fe3b0c68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81b0b10 con 0x5575b4daeca0 2024-09-10T08:24:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.472+0000 7fe3b0c68740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81afd70 con 0x5575b4da3ac0 2024-09-10T08:24:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.472+0000 7fe3b0c68740 10 client.0 _ll_register_callbacks cb 0x7fe3a00099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.472+0000 7fe3a7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4db71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.472+0000 7fe3a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4daeca0 0x5575b4db5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.472+0000 7fe3a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4daeca0 0x5575b4db5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:42992/0 (socket says 172.21.15.2:42992) 2024-09-10T08:24:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.472+0000 7fe3a7e00640 1 -- 172.21.15.2:0/3990061259 learned_addr learned my addr 172.21.15.2:0/3990061259 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.472+0000 7fe3a6a00640 1 --2- 172.21.15.2:0/3990061259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4da43e0 0x5575b4db5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.473+0000 7fe3a7e00640 1 -- 172.21.15.2:0/3990061259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 msgr2=0x5575b4db71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.473+0000 7fe3a7e00640 1 --2- 172.21.15.2:0/3990061259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4db71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.473+0000 7fe3a7e00640 1 -- 172.21.15.2:0/3990061259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4da43e0 msgr2=0x5575b4db5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.473+0000 7fe3a7e00640 1 --2- 172.21.15.2:0/3990061259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4da43e0 0x5575b4db5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.473+0000 7fe3a7e00640 1 -- 172.21.15.2:0/3990061259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575b4db6050 con 0x5575b4daeca0 2024-09-10T08:24:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.473+0000 7fe3a7e00640 1 --2- 172.21.15.2:0/3990061259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4daeca0 0x5575b4db5ac0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe3a001ead0 tx=0x7fe3a0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f698f1098ba7662 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.473+0000 7fe390c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.473+0000 7fe390c00640 1 -- 172.21.15.2:0/3990061259 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a002f060 con 0x5575b4daeca0 2024-09-10T08:24:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.473+0000 7fe3b0c68740 1 -- 172.21.15.2:0/3990061259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe398031a90 con 0x5575b4daeca0 2024-09-10T08:24:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.473+0000 7fe390c00640 1 -- 172.21.15.2:0/3990061259 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fe3a001d920 con 0x5575b4daeca0 2024-09-10T08:24:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.474+0000 7fe390c00640 1 -- 172.21.15.2:0/3990061259 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3a0036050 con 0x5575b4daeca0 2024-09-10T08:24:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.474+0000 7fe3b0c68740 10 client.30194 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.474+0000 7fe3b0c68740 1 -- 172.21.15.2:0/3990061259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5575b4e2a300 con 0x5575b4daeca0 2024-09-10T08:24:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.474+0000 7fe390c00640 1 -- 172.21.15.2:0/3990061259 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe398031a90 con 0x5575b4daeca0 2024-09-10T08:24:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.474+0000 7fe390c00640 1 -- 172.21.15.2:0/3990061259 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe3a0011440 con 0x5575b4daeca0 2024-09-10T08:24:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.475+0000 7fe3b0c68740 10 client.30194 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.475+0000 7fe3b0c68740 10 client.30194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.475+0000 7fe3b0c68740 20 client.30194 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.475+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/3990061259 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575b4e2aa60 0x5575b4e2ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.475+0000 7fe3b0c68740 4 client.30194 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:24:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.475+0000 7fe3b0c68740 1 -- 172.21.15.2:0/3990061259 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5575b4d11d10 con 0x5575b4e2aa60 2024-09-10T08:24:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.475+0000 7fe3a7400640 1 --2- 172.21.15.2:0/3990061259 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575b4e2aa60 0x5575b4e2ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.476+0000 7fe3a7400640 1 --2- 172.21.15.2:0/3990061259 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575b4e2aa60 0x5575b4e2ce80 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20a7be1446c1d293 server_cookie=b5518f1a25d072b1 in_seq=0 out_seq=0 2024-09-10T08:24:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.476+0000 7fe390c00640 10 client.30194 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.477+0000 7fe390c00640 1 -- 172.21.15.2:0/3990061259 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5575b4d11d10 con 0x5575b4e2aa60 2024-09-10T08:24:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.477+0000 7fe390c00640 10 client.30194 handle_command_reply: tid=0 2024-09-10T08:24:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.477+0000 7fe3b0c68740 1 client.30194 shutdown 2024-09-10T08:24:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.477+0000 7fe3b0c68740 1 -- 172.21.15.2:0/3990061259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4daeca0 msgr2=0x5575b4db5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.477+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/3990061259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4daeca0 0x5575b4db5ac0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe3a001ead0 tx=0x7fe3a0004a40 comp rx=0 tx=0).stop 2024-09-10T08:24:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.477+0000 7fe3b0c68740 1 -- 172.21.15.2:0/3990061259 shutdown_connections 2024-09-10T08:24:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.477+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/3990061259 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x5575b4e2aa60 0x5575b4e2ce80 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.477+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/3990061259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575b4daeca0 0x5575b4db5ac0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.477+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/3990061259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575b4da43e0 0x5575b4db5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.477+0000 7fe3b0c68740 1 --2- 172.21.15.2:0/3990061259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b4da3ac0 0x5575b4db71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.477+0000 7fe390c00640 0 client.30194 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.477+0000 7fe3b0c68740 1 -- 172.21.15.2:0/3990061259 >> 172.21.15.2:0/3990061259 conn(0x5575b4da0050 msgr2=0x5575b4d9f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.478+0000 7fe3b0c68740 1 -- 172.21.15.2:0/3990061259 shutdown_connections 2024-09-10T08:24:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.478+0000 7fe3b0c68740 1 -- 172.21.15.2:0/3990061259 wait complete. 2024-09-10T08:24:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.478+0000 7fe3b0c68740 20 client.30194 trim_cache size 0 max 16384 2024-09-10T08:24:31.479 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:24:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-09-10T08:24:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "fc162a85-92f7-48f6-a3e0-af1e92c199e4", 2024-09-10T08:24:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-09-10T08:24:31.480 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:24:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.479+0000 7fe3ae000640 1 -- 172.21.15.2:0/2009927692 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe38c0a2c00 msgr2=0x7fe38c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.479+0000 7fe3ae000640 1 --2- 172.21.15.2:0/2009927692 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe38c0a2c00 0x7fe38c0a50c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0005fd0 tx=0x7fe3a0009500 comp rx=0 tx=0).stop 2024-09-10T08:24:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.479+0000 7fe3ae000640 1 -- 172.21.15.2:0/2009927692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a81049a0 msgr2=0x7fe3a819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.479+0000 7fe3ae000640 1 --2- 172.21.15.2:0/2009927692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a81049a0 0x7fe3a819abb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe39400e9f0 tx=0x7fe39400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.479+0000 7fe3ae000640 1 -- 172.21.15.2:0/2009927692 shutdown_connections 2024-09-10T08:24:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.479+0000 7fe3ae000640 1 --2- 172.21.15.2:0/2009927692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a8109120 0x7fe3a819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.479+0000 7fe3ae000640 1 --2- 172.21.15.2:0/2009927692 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fe38c0a2c00 0x7fe38c0a50c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.479+0000 7fe3ae000640 1 --2- 172.21.15.2:0/2009927692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a81049a0 0x7fe3a819abb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.479+0000 7fe3ae000640 1 --2- 172.21.15.2:0/2009927692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3a8103ff0 0x7fe3a819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.479+0000 7fe3ae000640 1 -- 172.21.15.2:0/2009927692 >> 172.21.15.2:0/2009927692 conn(0x7fe3a80ffa90 msgr2=0x7fe3a8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.479+0000 7fe3ae000640 1 -- 172.21.15.2:0/2009927692 shutdown_connections 2024-09-10T08:24:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:31.479+0000 7fe3ae000640 1 -- 172.21.15.2:0/2009927692 wait complete. 2024-09-10T08:24:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:31.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:31 smithi002 ceph-mon[30563]: pgmap v878: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:24:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.859+0000 7f413a000640 1 -- 172.21.15.2:0/1300742175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4134105910 msgr2=0x7f413410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.859+0000 7f413a000640 1 --2- 172.21.15.2:0/1300742175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4134105910 0x7f413410c1a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f412c009ca0 tx=0x7f412c02a8b0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.859+0000 7f413a000640 1 -- 172.21.15.2:0/1300742175 shutdown_connections 2024-09-10T08:24:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.859+0000 7f413a000640 1 --2- 172.21.15.2:0/1300742175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4134105910 0x7f413410c1a0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.859+0000 7f413a000640 1 --2- 172.21.15.2:0/1300742175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134104f50 0x7f41341053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.859+0000 7f413a000640 1 --2- 172.21.15.2:0/1300742175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4134103d60 0x7f4134104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.859+0000 7f413a000640 1 -- 172.21.15.2:0/1300742175 >> 172.21.15.2:0/1300742175 conn(0x7f41340ff4e0 msgr2=0x7f4134101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.860+0000 7f413a000640 1 -- 172.21.15.2:0/1300742175 shutdown_connections 2024-09-10T08:24:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.860+0000 7f413a000640 1 -- 172.21.15.2:0/1300742175 wait complete. 2024-09-10T08:24:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.860+0000 7f413a000640 1 Processor -- start 2024-09-10T08:24:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.861+0000 7f413a000640 1 -- start start 2024-09-10T08:24:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.861+0000 7f413a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134103d60 0x7f4134199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.861+0000 7f413a000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4134104f50 0x7f413419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.861+0000 7f413a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4134105910 0x7f41341a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.861+0000 7f413a000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f413410f9e0 con 0x7f4134104f50 2024-09-10T08:24:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.861+0000 7f413a000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f413410f860 con 0x7f4134103d60 2024-09-10T08:24:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.861+0000 7f413a000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f413410fb60 con 0x7f4134105910 2024-09-10T08:24:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.861+0000 7f4133400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134103d60 0x7f4134199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.861+0000 7f4132a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4134104f50 0x7f413419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.861+0000 7f4133400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134103d60 0x7f4134199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:43006/0 (socket says 172.21.15.2:43006) 2024-09-10T08:24:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.861+0000 7f4133400640 1 -- 172.21.15.2:0/3125698290 learned_addr learned my addr 172.21.15.2:0/3125698290 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.862+0000 7f4133e00640 1 --2- 172.21.15.2:0/3125698290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4134105910 0x7f41341a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.862+0000 7f4133400640 1 -- 172.21.15.2:0/3125698290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4134105910 msgr2=0x7f41341a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.862+0000 7f4133400640 1 --2- 172.21.15.2:0/3125698290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4134105910 0x7f41341a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.862+0000 7f4133400640 1 -- 172.21.15.2:0/3125698290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4134104f50 msgr2=0x7f413419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.862+0000 7f4133400640 1 --2- 172.21.15.2:0/3125698290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4134104f50 0x7f413419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.862+0000 7f4133400640 1 -- 172.21.15.2:0/3125698290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41341a1bf0 con 0x7f4134103d60 2024-09-10T08:24:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.862+0000 7f4133400640 1 --2- 172.21.15.2:0/3125698290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134103d60 0x7f4134199fd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f412400d950 tx=0x7f412400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.862+0000 7f412be00640 1 -- 172.21.15.2:0/3125698290 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124017070 con 0x7f4134103d60 2024-09-10T08:24:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.862+0000 7f412be00640 1 -- 172.21.15.2:0/3125698290 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4124009c30 con 0x7f4134103d60 2024-09-10T08:24:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.862+0000 7f413a000640 1 -- 172.21.15.2:0/3125698290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41341a1ee0 con 0x7f4134103d60 2024-09-10T08:24:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.862+0000 7f412be00640 1 -- 172.21.15.2:0/3125698290 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412400f600 con 0x7f4134103d60 2024-09-10T08:24:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.863+0000 7f413a000640 1 -- 172.21.15.2:0/3125698290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41341a2340 con 0x7f4134103d60 2024-09-10T08:24:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.865+0000 7f413d04b740 1 Processor -- start 2024-09-10T08:24:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.866+0000 7f413d04b740 1 -- start start 2024-09-10T08:24:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.866+0000 7f412be00640 1 -- 172.21.15.2:0/3125698290 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f412401a070 con 0x7f4134103d60 2024-09-10T08:24:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.866+0000 7f413d04b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 0x564e574ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.866+0000 7f413d04b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574ce9d0 0x564e574d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.866+0000 7f413d04b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574d3a60 0x564e574d5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.866+0000 7f413d04b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564e57433100 con 0x564e574ce090 2024-09-10T08:24:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.866+0000 7f413d04b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564e573db1b0 con 0x564e574ce9d0 2024-09-10T08:24:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.866+0000 7f413d04b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564e5742f0c0 con 0x564e574d3a60 2024-09-10T08:24:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.866+0000 7f4133e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 0x564e574ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.866+0000 7f4133e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 0x564e574ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54230/0 (socket says 172.21.15.2:54230) 2024-09-10T08:24:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.866+0000 7f4133e00640 1 -- 172.21.15.2:0/974512334 learned_addr learned my addr 172.21.15.2:0/974512334 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.866+0000 7f4133400640 1 --2- 172.21.15.2:0/974512334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574ce9d0 0x564e574d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.866+0000 7f4132a00640 1 --2- 172.21.15.2:0/974512334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574d3a60 0x564e574d5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.867+0000 7f4133e00640 1 -- 172.21.15.2:0/974512334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574d3a60 msgr2=0x564e574d5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.867+0000 7f4133e00640 1 --2- 172.21.15.2:0/974512334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574d3a60 0x564e574d5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.867+0000 7f4133e00640 1 -- 172.21.15.2:0/974512334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574ce9d0 msgr2=0x564e574d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.867+0000 7f4133e00640 1 --2- 172.21.15.2:0/974512334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574ce9d0 0x564e574d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.867+0000 7f4133e00640 1 -- 172.21.15.2:0/974512334 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e57146210 con 0x564e574ce090 2024-09-10T08:24:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.867+0000 7f4133e00640 1 --2- 172.21.15.2:0/974512334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 0x564e574ce490 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f412c009a20 tx=0x7f412c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae04ade6b27d9d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.867+0000 7f412be00640 1 --2- 172.21.15.2:0/3125698290 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41180a2cb0 0x7f41180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.867+0000 7f411fe00640 1 -- 172.21.15.2:0/974512334 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c030a30 con 0x564e574ce090 2024-09-10T08:24:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.867+0000 7f411fe00640 1 -- 172.21.15.2:0/974512334 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412c036040 con 0x564e574ce090 2024-09-10T08:24:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.867+0000 7f411fe00640 1 -- 172.21.15.2:0/974512334 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c0044c0 con 0x564e574ce090 2024-09-10T08:24:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.867+0000 7f412be00640 1 -- 172.21.15.2:0/3125698290 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f41240ceeb0 con 0x7f4134103d60 2024-09-10T08:24:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.868+0000 7f4133400640 1 --2- 172.21.15.2:0/3125698290 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41180a2cb0 0x7f41180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.868+0000 7f413d04b740 1 -- 172.21.15.2:0/974512334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 msgr2=0x564e574ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.868+0000 7f413d04b740 1 --2- 172.21.15.2:0/974512334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 0x564e574ce490 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f412c009a20 tx=0x7f412c004a40 comp rx=0 tx=0).stop 2024-09-10T08:24:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.868+0000 7f413d04b740 1 -- 172.21.15.2:0/974512334 shutdown_connections 2024-09-10T08:24:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.868+0000 7f413d04b740 1 --2- 172.21.15.2:0/974512334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574d3a60 0x564e574d5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.868+0000 7f413d04b740 1 --2- 172.21.15.2:0/974512334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574ce9d0 0x564e574d3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.868+0000 7f413d04b740 1 --2- 172.21.15.2:0/974512334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 0x564e574ce490 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.868+0000 7f413d04b740 1 -- 172.21.15.2:0/974512334 >> 172.21.15.2:0/974512334 conn(0x564e574c9700 msgr2=0x564e574c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.868+0000 7f413d04b740 1 -- 172.21.15.2:0/974512334 shutdown_connections 2024-09-10T08:24:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.868+0000 7f4133400640 1 --2- 172.21.15.2:0/3125698290 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41180a2cb0 0x7f41180a5170 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f412409b250 tx=0x7f41240d28f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.868+0000 7f413d04b740 1 -- 172.21.15.2:0/974512334 wait complete. 2024-09-10T08:24:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.869+0000 7f413d04b740 1 -- start start 2024-09-10T08:24:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.869+0000 7f413d04b740 1 Processor -- start 2024-09-10T08:24:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.869+0000 7f413d04b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce090 0x564e574d8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.869+0000 7f413d04b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce9d0 0x564e574d8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.869+0000 7f413d04b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574d3a60 0x564e574df790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.869+0000 7f413d04b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564e574d69d0 con 0x564e574ce9d0 2024-09-10T08:24:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.869+0000 7f413d04b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564e573bc090 con 0x564e574d3a60 2024-09-10T08:24:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.869+0000 7f413d04b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564e574d6b50 con 0x564e574ce090 2024-09-10T08:24:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f413d04b740 10 client.0 _ll_register_callbacks cb 0x564e57410f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f4132a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce9d0 0x564e574d8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f4132a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce9d0 0x564e574d8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54246/0 (socket says 172.21.15.2:54246) 2024-09-10T08:24:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f4132a00640 1 -- 172.21.15.2:0/3444334402 learned_addr learned my addr 172.21.15.2:0/3444334402 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f4133e00640 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce090 0x564e574d8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f4133e00640 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574d3a60 0x564e574df790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f4132a00640 1 -- 172.21.15.2:0/3444334402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce090 msgr2=0x564e574d8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f4132a00640 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce090 0x564e574d8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f4132a00640 1 -- 172.21.15.2:0/3444334402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574d3a60 msgr2=0x564e574df790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f4132a00640 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574d3a60 0x564e574df790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f4132a00640 1 -- 172.21.15.2:0/3444334402 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e57212620 con 0x564e574ce9d0 2024-09-10T08:24:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f4133e00640 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574d3a60 0x564e574df790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f4132a00640 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce9d0 0x564e574d8750 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4120012720 tx=0x7f4120012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c0587d76e28736 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.870+0000 7f411cc00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:24:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.871+0000 7f411cc00640 1 -- 172.21.15.2:0/3444334402 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412000f920 con 0x564e574ce9d0 2024-09-10T08:24:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.871+0000 7f411cc00640 1 -- 172.21.15.2:0/3444334402 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f412000fac0 con 0x564e574ce9d0 2024-09-10T08:24:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.871+0000 7f413d04b740 1 -- 172.21.15.2:0/3444334402 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f412c036040 con 0x564e574ce9d0 2024-09-10T08:24:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.871+0000 7f411cc00640 1 -- 172.21.15.2:0/3444334402 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f412000fd00 con 0x564e574ce9d0 2024-09-10T08:24:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.871+0000 7f413d04b740 10 client.20190 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.871+0000 7f413d04b740 1 -- 172.21.15.2:0/3444334402 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e574e2650 con 0x564e574ce9d0 2024-09-10T08:24:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.871+0000 7f411cc00640 1 -- 172.21.15.2:0/3444334402 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412c036040 con 0x564e574ce9d0 2024-09-10T08:24:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.871+0000 7f411cc00640 1 -- 172.21.15.2:0/3444334402 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4120010550 con 0x564e574ce9d0 2024-09-10T08:24:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.872+0000 7f413d04b740 10 client.20190 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.872+0000 7f413d04b740 10 client.20190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.872+0000 7f413d04b740 20 client.20190 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.872+0000 7f413d04b740 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564e574e3d70 0x564e574e6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.872+0000 7f413d04b740 4 client.20190 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.872+0000 7f413d04b740 1 -- 172.21.15.2:0/3444334402 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e57440830 con 0x564e574e3d70 2024-09-10T08:24:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.872+0000 7f4133e00640 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564e574e3d70 0x564e574e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.873+0000 7f4133e00640 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564e574e3d70 0x564e574e6180 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16b374d364a85893 server_cookie=15bdd3386bf54fc9 in_seq=0 out_seq=0 2024-09-10T08:24:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.873+0000 7f411cc00640 10 client.20190 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.873+0000 7f411cc00640 1 -- 172.21.15.2:0/3444334402 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e57440830 con 0x564e574e3d70 2024-09-10T08:24:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.873+0000 7f411cc00640 10 client.20190 handle_command_reply: tid=0 2024-09-10T08:24:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.873+0000 7f413d04b740 1 client.20190 shutdown 2024-09-10T08:24:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.874+0000 7f413d04b740 1 -- 172.21.15.2:0/3444334402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce9d0 msgr2=0x564e574d8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.874+0000 7f413d04b740 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce9d0 0x564e574d8750 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4120012720 tx=0x7f4120012be0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.874+0000 7f413d04b740 1 -- 172.21.15.2:0/3444334402 shutdown_connections 2024-09-10T08:24:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.874+0000 7f413d04b740 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564e574e3d70 0x564e574e6180 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.874+0000 7f413d04b740 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574d3a60 0x564e574df790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.874+0000 7f411cc00640 0 client.20190 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.874+0000 7f413d04b740 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce9d0 0x564e574d8750 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.874+0000 7f413d04b740 1 --2- 172.21.15.2:0/3444334402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce090 0x564e574d8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.874+0000 7f413d04b740 1 -- 172.21.15.2:0/3444334402 >> 172.21.15.2:0/3444334402 conn(0x564e574c9700 msgr2=0x564e574caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.874+0000 7f413d04b740 1 -- 172.21.15.2:0/3444334402 shutdown_connections 2024-09-10T08:24:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.874+0000 7f413d04b740 1 -- 172.21.15.2:0/3444334402 wait complete. 2024-09-10T08:24:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.874+0000 7f413d04b740 20 client.20190 trim_cache size 0 max 16384 2024-09-10T08:24:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.889+0000 7f413d04b740 1 Processor -- start 2024-09-10T08:24:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.889+0000 7f413d04b740 1 -- start start 2024-09-10T08:24:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.889+0000 7f413d04b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574c9700 0x564e574d79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.889+0000 7f413d04b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 0x564e574df750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.889+0000 7f413d04b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce9d0 0x564e574caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.889+0000 7f413d04b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f41341af3b0 con 0x564e574ce090 2024-09-10T08:24:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.889+0000 7f413d04b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f41341acc80 con 0x564e574c9700 2024-09-10T08:24:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.889+0000 7f413d04b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41341aaad0 con 0x564e574ce9d0 2024-09-10T08:24:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.889+0000 7f4132a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 0x564e574df750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f4133e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574c9700 0x564e574d79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f4132a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 0x564e574df750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54248/0 (socket says 172.21.15.2:54248) 2024-09-10T08:24:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f4132a00640 1 -- 172.21.15.2:0/2751806696 learned_addr learned my addr 172.21.15.2:0/2751806696 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f4133e00640 1 --2- 172.21.15.2:0/2751806696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce9d0 0x564e574caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f4132a00640 1 -- 172.21.15.2:0/2751806696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce9d0 msgr2=0x564e574caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f4132a00640 1 --2- 172.21.15.2:0/2751806696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce9d0 0x564e574caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f4132a00640 1 -- 172.21.15.2:0/2751806696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574c9700 msgr2=0x564e574d79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f4132a00640 1 --2- 172.21.15.2:0/2751806696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574c9700 0x564e574d79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f4132a00640 1 -- 172.21.15.2:0/2751806696 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e574c9520 con 0x564e574ce090 2024-09-10T08:24:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f4133e00640 1 --2- 172.21.15.2:0/2751806696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce9d0 0x564e574caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f4132a00640 1 --2- 172.21.15.2:0/2751806696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 0x564e574df750 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4120009e20 tx=0x7f41200096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5cd00c054e1e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f411f400640 1 -- 172.21.15.2:0/2751806696 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412000fea0 con 0x564e574ce090 2024-09-10T08:24:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f411f400640 1 -- 172.21.15.2:0/2751806696 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412000f920 con 0x564e574ce090 2024-09-10T08:24:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.890+0000 7f411f400640 1 -- 172.21.15.2:0/2751806696 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41200055a0 con 0x564e574ce090 2024-09-10T08:24:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.891+0000 7f413d04b740 1 -- 172.21.15.2:0/2751806696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 msgr2=0x564e574df750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.891+0000 7f413d04b740 1 --2- 172.21.15.2:0/2751806696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 0x564e574df750 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4120009e20 tx=0x7f41200096c0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.891+0000 7f413d04b740 1 -- 172.21.15.2:0/2751806696 shutdown_connections 2024-09-10T08:24:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.891+0000 7f413d04b740 1 --2- 172.21.15.2:0/2751806696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce9d0 0x564e574caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.891+0000 7f413d04b740 1 --2- 172.21.15.2:0/2751806696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574ce090 0x564e574df750 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.891+0000 7f413d04b740 1 --2- 172.21.15.2:0/2751806696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574c9700 0x564e574d79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.891+0000 7f413d04b740 1 -- 172.21.15.2:0/2751806696 >> 172.21.15.2:0/2751806696 conn(0x564e574d35b0 msgr2=0x564e574c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.891+0000 7f413d04b740 1 -- 172.21.15.2:0/2751806696 shutdown_connections 2024-09-10T08:24:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.891+0000 7f413d04b740 1 -- 172.21.15.2:0/2751806696 wait complete. 2024-09-10T08:24:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.891+0000 7f413d04b740 1 -- start start 2024-09-10T08:24:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.892+0000 7f413d04b740 1 Processor -- start 2024-09-10T08:24:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.892+0000 7f413d04b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574c9700 0x564e574dd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.892+0000 7f413d04b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574ce090 0x564e574dd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.892+0000 7f413d04b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce9d0 0x564e574de5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.892+0000 7f413d04b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f41341ab140 con 0x564e574c9700 2024-09-10T08:24:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.892+0000 7f413d04b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f41341ac5b0 con 0x564e574ce090 2024-09-10T08:24:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.892+0000 7f413d04b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41341ab810 con 0x564e574ce9d0 2024-09-10T08:24:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.892+0000 7f413d04b740 10 client.0 _ll_register_callbacks cb 0x7f4120009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.892+0000 7f4133e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574c9700 0x564e574dd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.892+0000 7f4133e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574c9700 0x564e574dd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54256/0 (socket says 172.21.15.2:54256) 2024-09-10T08:24:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.892+0000 7f4133e00640 1 -- 172.21.15.2:0/1226332013 learned_addr learned my addr 172.21.15.2:0/1226332013 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.892+0000 7f4132a00640 1 --2- 172.21.15.2:0/1226332013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574ce090 0x564e574dd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.893+0000 7f4133e00640 1 --2- 172.21.15.2:0/1226332013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce9d0 0x564e574de5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.893+0000 7f4133e00640 1 -- 172.21.15.2:0/1226332013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce9d0 msgr2=0x564e574de5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.893+0000 7f4133e00640 1 --2- 172.21.15.2:0/1226332013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce9d0 0x564e574de5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.893+0000 7f4133e00640 1 -- 172.21.15.2:0/1226332013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574ce090 msgr2=0x564e574dd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.893+0000 7f4133e00640 1 --2- 172.21.15.2:0/1226332013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574ce090 0x564e574dd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.893+0000 7f4133e00640 1 -- 172.21.15.2:0/1226332013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e574debb0 con 0x564e574c9700 2024-09-10T08:24:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.893+0000 7f4133e00640 1 --2- 172.21.15.2:0/1226332013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574c9700 0x564e574dd050 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f412c000f80 tx=0x7f412c030930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943bea58023f2908 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.893+0000 7f411cc00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:24:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.893+0000 7f411cc00640 1 -- 172.21.15.2:0/1226332013 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c004b40 con 0x564e574c9700 2024-09-10T08:24:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.893+0000 7f413d04b740 1 -- 172.21.15.2:0/1226332013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f412000f920 con 0x564e574c9700 2024-09-10T08:24:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.893+0000 7f411cc00640 1 -- 172.21.15.2:0/1226332013 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f412c004ce0 con 0x564e574c9700 2024-09-10T08:24:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.894+0000 7f411cc00640 1 -- 172.21.15.2:0/1226332013 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f412c003970 con 0x564e574c9700 2024-09-10T08:24:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.894+0000 7f413d04b740 10 client.20196 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.894+0000 7f413d04b740 1 -- 172.21.15.2:0/1226332013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e5754ebd0 con 0x564e574c9700 2024-09-10T08:24:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.894+0000 7f411cc00640 1 -- 172.21.15.2:0/1226332013 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412000f920 con 0x564e574c9700 2024-09-10T08:24:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.894+0000 7f411cc00640 1 -- 172.21.15.2:0/1226332013 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f412c005040 con 0x564e574c9700 2024-09-10T08:24:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.894+0000 7f413d04b740 10 client.20196 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.894+0000 7f413d04b740 10 client.20196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.894+0000 7f413d04b740 20 client.20196 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.894+0000 7f413d04b740 1 --2- 172.21.15.2:0/1226332013 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564e574d3a60 0x564e575510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.894+0000 7f413d04b740 4 client.20196 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:24:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.894+0000 7f413d04b740 1 -- 172.21.15.2:0/1226332013 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e5743ab80 con 0x564e574d3a60 2024-09-10T08:24:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.894+0000 7f4132a00640 1 --2- 172.21.15.2:0/1226332013 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564e574d3a60 0x564e575510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.895+0000 7f4132a00640 1 --2- 172.21.15.2:0/1226332013 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564e574d3a60 0x564e575510e0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cc614d2edeccce server_cookie=a5fee8766ca68d37 in_seq=0 out_seq=0 2024-09-10T08:24:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.895+0000 7f411cc00640 10 client.20196 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:31.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:31 smithi155 ceph-mon[35405]: pgmap v878: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:24:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.909+0000 7f411cc00640 1 -- 172.21.15.2:0/1226332013 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e5743ab80 con 0x564e574d3a60 2024-09-10T08:24:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.909+0000 7f411cc00640 10 client.20196 handle_command_reply: tid=0 2024-09-10T08:24:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.909+0000 7f413d04b740 1 client.20196 shutdown 2024-09-10T08:24:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.909+0000 7f413d04b740 1 -- 172.21.15.2:0/1226332013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574c9700 msgr2=0x564e574dd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.909+0000 7f413d04b740 1 --2- 172.21.15.2:0/1226332013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574c9700 0x564e574dd050 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f412c000f80 tx=0x7f412c030930 comp rx=0 tx=0).stop 2024-09-10T08:24:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.909+0000 7f413d04b740 1 -- 172.21.15.2:0/1226332013 shutdown_connections 2024-09-10T08:24:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.909+0000 7f413d04b740 1 --2- 172.21.15.2:0/1226332013 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x564e574d3a60 0x564e575510e0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.909+0000 7f413d04b740 1 --2- 172.21.15.2:0/1226332013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e574ce9d0 0x564e574de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.909+0000 7f413d04b740 1 --2- 172.21.15.2:0/1226332013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564e574ce090 0x564e574dd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.909+0000 7f413d04b740 1 --2- 172.21.15.2:0/1226332013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564e574c9700 0x564e574dd050 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.909+0000 7f413d04b740 1 -- 172.21.15.2:0/1226332013 >> 172.21.15.2:0/1226332013 conn(0x564e574d35b0 msgr2=0x564e574c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.909+0000 7f411cc00640 0 client.20196 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.910+0000 7f413d04b740 1 -- 172.21.15.2:0/1226332013 shutdown_connections 2024-09-10T08:24:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.910+0000 7f413d04b740 1 -- 172.21.15.2:0/1226332013 wait complete. 2024-09-10T08:24:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.910+0000 7f413d04b740 20 client.20196 trim_cache size 0 max 16384 2024-09-10T08:24:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-09-10T08:24:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "message": "", 2024-09-10T08:24:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-09-10T08:24:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-09-10T08:24:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.910+0000 7f413a000640 1 -- 172.21.15.2:0/3125698290 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41180a2cb0 msgr2=0x7f41180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.910+0000 7f413a000640 1 --2- 172.21.15.2:0/3125698290 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41180a2cb0 0x7f41180a5170 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f412409b250 tx=0x7f41240d28f0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.910+0000 7f413a000640 1 -- 172.21.15.2:0/3125698290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134103d60 msgr2=0x7f4134199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.910+0000 7f413a000640 1 --2- 172.21.15.2:0/3125698290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134103d60 0x7f4134199fd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f412400d950 tx=0x7f412400de10 comp rx=0 tx=0).stop 2024-09-10T08:24:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.911+0000 7f413a000640 1 -- 172.21.15.2:0/3125698290 shutdown_connections 2024-09-10T08:24:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.911+0000 7f413a000640 1 --2- 172.21.15.2:0/3125698290 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f41180a2cb0 0x7f41180a5170 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.911+0000 7f413a000640 1 --2- 172.21.15.2:0/3125698290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4134105910 0x7f41341a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.911+0000 7f413a000640 1 --2- 172.21.15.2:0/3125698290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4134104f50 0x7f413419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.911+0000 7f413a000640 1 --2- 172.21.15.2:0/3125698290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4134103d60 0x7f4134199fd0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.911+0000 7f413a000640 1 -- 172.21.15.2:0/3125698290 >> 172.21.15.2:0/3125698290 conn(0x7f41340ff4e0 msgr2=0x7f41341081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.911+0000 7f413a000640 1 -- 172.21.15.2:0/3125698290 shutdown_connections 2024-09-10T08:24:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-09-10T08:24:31.911+0000 7f413a000640 1 -- 172.21.15.2:0/3125698290 wait complete. 2024-09-10T08:24:31.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:31 smithi107 ceph-mon[35048]: pgmap v878: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:24:32.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:32 smithi002 ceph-mon[30563]: scrub queued for path: / 2024-09-10T08:24:32.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:32 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:24:32.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:32 smithi002 ceph-mon[30563]: scrub summary: active paths [/] 2024-09-10T08:24:32.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:32 smithi002 ceph-mon[30563]: scrub summary: idle+waiting paths [/] 2024-09-10T08:24:32.831 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:24:32.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:32 smithi155 ceph-mon[35405]: scrub queued for path: / 2024-09-10T08:24:32.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:32 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:24:32.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:32 smithi155 ceph-mon[35405]: scrub summary: active paths [/] 2024-09-10T08:24:32.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:32 smithi155 ceph-mon[35405]: scrub summary: idle+waiting paths [/] 2024-09-10T08:24:32.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.404+0000 7fed0d600640 1 -- 172.21.15.2:0/324607488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed08102340 msgr2=0x7fed0810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:32.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.404+0000 7fed0d600640 1 --2- 172.21.15.2:0/324607488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed08102340 0x7fed0810e870 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fed00009cd0 tx=0x7fed0002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.404+0000 7fed0d600640 1 -- 172.21.15.2:0/324607488 shutdown_connections 2024-09-10T08:24:32.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.404+0000 7fed0d600640 1 --2- 172.21.15.2:0/324607488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed08102340 0x7fed0810e870 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.404+0000 7fed0d600640 1 --2- 172.21.15.2:0/324607488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed081019a0 0x7fed08101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.404+0000 7fed0d600640 1 --2- 172.21.15.2:0/324607488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed081079a0 0x7fed08107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.404+0000 7fed0d600640 1 -- 172.21.15.2:0/324607488 >> 172.21.15.2:0/324607488 conn(0x7fed080fd700 msgr2=0x7fed080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:32.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.404+0000 7fed0d600640 1 -- 172.21.15.2:0/324607488 shutdown_connections 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.405+0000 7fed0d600640 1 -- 172.21.15.2:0/324607488 wait complete. 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.406+0000 7fed0d600640 1 Processor -- start 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.406+0000 7fed0d600640 1 -- start start 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.406+0000 7fed0d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed081019a0 0x7fed080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed0d600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed08102340 0x7fed08074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed0d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed081079a0 0x7fed08074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed0d600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fed08111df0 con 0x7fed08102340 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed0d600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed08111c70 con 0x7fed081019a0 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed0d600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed08111f70 con 0x7fed081079a0 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed07400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed08102340 0x7fed08074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed07400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed08102340 0x7fed08074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54284/0 (socket says 172.21.15.2:54284) 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed07400640 1 -- 172.21.15.2:0/2101711914 learned_addr learned my addr 172.21.15.2:0/2101711914 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed07e00640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed081019a0 0x7fed080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed0cc00640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed081079a0 0x7fed08074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed07400640 1 -- 172.21.15.2:0/2101711914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed081079a0 msgr2=0x7fed08074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed07400640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed081079a0 0x7fed08074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed07400640 1 -- 172.21.15.2:0/2101711914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed081019a0 msgr2=0x7fed080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed07400640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed081019a0 0x7fed080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.407+0000 7fed07400640 1 -- 172.21.15.2:0/2101711914 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed081a66d0 con 0x7fed08102340 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.408+0000 7fed07400640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed08102340 0x7fed08074a30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fecf400e9f0 tx=0x7fecf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.408+0000 7fed04c00640 1 -- 172.21.15.2:0/2101711914 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf400cd80 con 0x7fed08102340 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.408+0000 7fed04c00640 1 -- 172.21.15.2:0/2101711914 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4010910 con 0x7fed08102340 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.408+0000 7fed0d600640 1 -- 172.21.15.2:0/2101711914 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed081a69c0 con 0x7fed08102340 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.408+0000 7fed04c00640 1 -- 172.21.15.2:0/2101711914 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4018990 con 0x7fed08102340 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.408+0000 7fed0d600640 1 -- 172.21.15.2:0/2101711914 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed081a6f00 con 0x7fed08102340 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.411+0000 7fed04c00640 1 -- 172.21.15.2:0/2101711914 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fecf4010ac0 con 0x7fed08102340 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.411+0000 7fed11842740 1 Processor -- start 2024-09-10T08:24:32.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.412+0000 7fed11842740 1 -- start start 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.412+0000 7fed11842740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f73c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.412+0000 7fed11842740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 0x560d23f7e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.412+0000 7fed11842740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d23f80e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.412+0000 7fed11842740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560d23eda340 con 0x560d23f73880 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.412+0000 7fed11842740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560d23e81bf0 con 0x560d23f7ea60 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.412+0000 7fed11842740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d23ed6300 con 0x560d23f741a0 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.412+0000 7fed0cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f73c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.412+0000 7fed0cc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f73c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54294/0 (socket says 172.21.15.2:54294) 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.412+0000 7fed0cc00640 1 -- 172.21.15.2:0/2729376696 learned_addr learned my addr 172.21.15.2:0/2729376696 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.412+0000 7fed07e00640 1 --2- 172.21.15.2:0/2729376696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 0x560d23f7e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.412+0000 7fed07400640 1 --2- 172.21.15.2:0/2729376696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d23f80e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.413+0000 7fed0cc00640 1 -- 172.21.15.2:0/2729376696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 msgr2=0x560d23f7e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.413+0000 7fed0cc00640 1 --2- 172.21.15.2:0/2729376696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 0x560d23f7e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.413+0000 7fed0cc00640 1 -- 172.21.15.2:0/2729376696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 msgr2=0x560d23f80e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.413+0000 7fed0cc00640 1 --2- 172.21.15.2:0/2729376696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d23f80e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.413+0000 7fed0cc00640 1 -- 172.21.15.2:0/2729376696 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d23c4b620 con 0x560d23f73880 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.413+0000 7fed07e00640 1 --2- 172.21.15.2:0/2729376696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 0x560d23f7e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.413+0000 7fed07400640 1 --2- 172.21.15.2:0/2729376696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d23f80e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.413+0000 7fed0cc00640 1 --2- 172.21.15.2:0/2729376696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f73c60 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fed00009cd0 tx=0x7fed00009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6abd3f3bb24efb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.413+0000 7fecfd200640 1 -- 172.21.15.2:0/2729376696 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00049070 con 0x560d23f73880 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.414+0000 7fecfd200640 1 -- 172.21.15.2:0/2729376696 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed00036040 con 0x560d23f73880 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.414+0000 7fecfd200640 1 -- 172.21.15.2:0/2729376696 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed000042a0 con 0x560d23f73880 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.414+0000 7fed04c00640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fecec0a2b80 0x7fecec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.414+0000 7fed04c00640 1 -- 172.21.15.2:0/2101711914 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fecf40cf7a0 con 0x7fed08102340 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.414+0000 7fed07e00640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fecec0a2b80 0x7fecec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.414+0000 7fed11842740 1 -- 172.21.15.2:0/2729376696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 msgr2=0x560d23f73c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.414+0000 7fed11842740 1 --2- 172.21.15.2:0/2729376696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f73c60 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fed00009cd0 tx=0x7fed00009a20 comp rx=0 tx=0).stop 2024-09-10T08:24:32.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.414+0000 7fed11842740 1 -- 172.21.15.2:0/2729376696 shutdown_connections 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.414+0000 7fed11842740 1 --2- 172.21.15.2:0/2729376696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d23f80e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.414+0000 7fed11842740 1 --2- 172.21.15.2:0/2729376696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 0x560d23f7e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.414+0000 7fed11842740 1 --2- 172.21.15.2:0/2729376696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f73c60 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.414+0000 7fed11842740 1 -- 172.21.15.2:0/2729376696 >> 172.21.15.2:0/2729376696 conn(0x560d23f6fe10 msgr2=0x560d23f701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.415+0000 7fed11842740 1 -- 172.21.15.2:0/2729376696 shutdown_connections 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.415+0000 7fed11842740 1 -- 172.21.15.2:0/2729376696 wait complete. 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.415+0000 7fed07e00640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fecec0a2b80 0x7fecec0a5040 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x560d23f70c70 tx=0x7fecf800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.415+0000 7fed11842740 1 -- start start 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.415+0000 7fed11842740 1 Processor -- start 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.416+0000 7fed11842740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f7b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.416+0000 7fed11842740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f741a0 0x560d23f7bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.416+0000 7fed11842740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f7ea60 0x560d23f7c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.416+0000 7fed11842740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560d23f81630 con 0x560d23f73880 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.416+0000 7fed11842740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560d23e95060 con 0x560d23f741a0 2024-09-10T08:24:32.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.416+0000 7fed11842740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d23f817b0 con 0x560d23f7ea60 2024-09-10T08:24:32.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.416+0000 7fed11842740 10 client.0 _ll_register_callbacks cb 0x560d23e83af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:32.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.416+0000 7fed07e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f7b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.416+0000 7fed07e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f7b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54300/0 (socket says 172.21.15.2:54300) 2024-09-10T08:24:32.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.416+0000 7fed07e00640 1 -- 172.21.15.2:0/1038218163 learned_addr learned my addr 172.21.15.2:0/1038218163 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:32.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.416+0000 7fed07400640 1 --2- 172.21.15.2:0/1038218163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f741a0 0x560d23f7bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.416+0000 7fed0cc00640 1 --2- 172.21.15.2:0/1038218163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f7ea60 0x560d23f7c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.417+0000 7fed07e00640 1 -- 172.21.15.2:0/1038218163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f7ea60 msgr2=0x560d23f7c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:32.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.417+0000 7fed07e00640 1 --2- 172.21.15.2:0/1038218163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f7ea60 0x560d23f7c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.417+0000 7fed07e00640 1 -- 172.21.15.2:0/1038218163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f741a0 msgr2=0x560d23f7bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:32.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.417+0000 7fed07e00640 1 --2- 172.21.15.2:0/1038218163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f741a0 0x560d23f7bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.417+0000 7fed07e00640 1 -- 172.21.15.2:0/1038218163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d23f8ab80 con 0x560d23f73880 2024-09-10T08:24:32.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.417+0000 7fed07e00640 1 --2- 172.21.15.2:0/1038218163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f7b990 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fecf801f4b0 tx=0x7fecf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87d9f203cc0bd64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:32.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.417+0000 7fecf1600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.417+0000 7fecf1600640 1 -- 172.21.15.2:0/1038218163 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf802c060 con 0x560d23f73880 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.417+0000 7fecf1600640 1 -- 172.21.15.2:0/1038218163 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fecf8017030 con 0x560d23f73880 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.417+0000 7fed11842740 1 -- 172.21.15.2:0/1038218163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed00036040 con 0x560d23f73880 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.418+0000 7fecf1600640 1 -- 172.21.15.2:0/1038218163 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fecf8033050 con 0x560d23f73880 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.418+0000 7fed11842740 10 client.20208 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.418+0000 7fed11842740 1 -- 172.21.15.2:0/1038218163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d23f8d570 con 0x560d23f73880 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.418+0000 7fecf1600640 1 -- 172.21.15.2:0/1038218163 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed00036040 con 0x560d23f73880 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.418+0000 7fecf1600640 1 -- 172.21.15.2:0/1038218163 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fecf80108e0 con 0x560d23f73880 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.418+0000 7fed11842740 10 client.20208 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.418+0000 7fed11842740 10 client.20208 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.418+0000 7fed11842740 20 client.20208 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.418+0000 7fed11842740 1 --2- 172.21.15.2:0/1038218163 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d23f8ea30 0x560d23f90ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.418+0000 7fed11842740 4 client.20208 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.418+0000 7fed11842740 1 -- 172.21.15.2:0/1038218163 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d23ee78b0 con 0x560d23f8ea30 2024-09-10T08:24:32.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.419+0000 7fed07400640 1 --2- 172.21.15.2:0/1038218163 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d23f8ea30 0x560d23f90ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.419+0000 7fed07400640 1 --2- 172.21.15.2:0/1038218163 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d23f8ea30 0x560d23f90ec0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=887039e0de10f3e3 server_cookie=96443ec6dbdc6034 in_seq=0 out_seq=0 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.419+0000 7fecf1600640 10 client.20208 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fecf1600640 1 -- 172.21.15.2:0/1038218163 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d23ee78b0 con 0x560d23f8ea30 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fecf1600640 10 client.20208 handle_command_reply: tid=0 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fed11842740 1 client.20208 shutdown 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fed11842740 1 -- 172.21.15.2:0/1038218163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 msgr2=0x560d23f7b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fed11842740 1 --2- 172.21.15.2:0/1038218163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f7b990 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fecf801f4b0 tx=0x7fecf8004730 comp rx=0 tx=0).stop 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fed11842740 1 -- 172.21.15.2:0/1038218163 shutdown_connections 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fed11842740 1 --2- 172.21.15.2:0/1038218163 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d23f8ea30 0x560d23f90ec0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fed11842740 1 --2- 172.21.15.2:0/1038218163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f7ea60 0x560d23f7c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fed11842740 1 --2- 172.21.15.2:0/1038218163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f741a0 0x560d23f7bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fed11842740 1 --2- 172.21.15.2:0/1038218163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f7b990 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fed11842740 1 -- 172.21.15.2:0/1038218163 >> 172.21.15.2:0/1038218163 conn(0x560d23f6fe10 msgr2=0x560d23f6eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:32.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fecf1600640 0 client.20208 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fed11842740 1 -- 172.21.15.2:0/1038218163 shutdown_connections 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.420+0000 7fed11842740 1 -- 172.21.15.2:0/1038218163 wait complete. 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.421+0000 7fed11842740 20 client.20208 trim_cache size 0 max 16384 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.439+0000 7fed11842740 1 Processor -- start 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.439+0000 7fed11842740 1 -- start start 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.439+0000 7fed11842740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f752d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.439+0000 7fed11842740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 0x560d23f75810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.439+0000 7fed11842740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d23f96790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.439+0000 7fed11842740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fed081b36e0 con 0x560d23f73880 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.439+0000 7fed11842740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed081b0fb0 con 0x560d23f7ea60 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.439+0000 7fed11842740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed081aee00 con 0x560d23f741a0 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.440+0000 7fed07e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.440+0000 7fed07e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f752d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54316/0 (socket says 172.21.15.2:54316) 2024-09-10T08:24:32.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.440+0000 7fed07e00640 1 -- 172.21.15.2:0/13523364 learned_addr learned my addr 172.21.15.2:0/13523364 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.440+0000 7fed0cc00640 1 --2- 172.21.15.2:0/13523364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d23f96790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.440+0000 7fed07400640 1 --2- 172.21.15.2:0/13523364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 0x560d23f75810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.440+0000 7fed0cc00640 1 -- 172.21.15.2:0/13523364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 msgr2=0x560d23f75810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.440+0000 7fed0cc00640 1 --2- 172.21.15.2:0/13523364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 0x560d23f75810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.440+0000 7fed0cc00640 1 -- 172.21.15.2:0/13523364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 msgr2=0x560d23f752d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.440+0000 7fed0cc00640 1 --2- 172.21.15.2:0/13523364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f752d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.440+0000 7fed0cc00640 1 -- 172.21.15.2:0/13523364 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d23f6f6a0 con 0x560d23f7ea60 2024-09-10T08:24:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.440+0000 7fed07e00640 1 --2- 172.21.15.2:0/13523364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f752d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.441+0000 7fed0cc00640 1 --2- 172.21.15.2:0/13523364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d23f96790 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fed0002ae00 tx=0x7fed00003d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d87c9f8d59aef5f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.441+0000 7fecf3e00640 1 -- 172.21.15.2:0/13523364 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00049040 con 0x560d23f7ea60 2024-09-10T08:24:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.441+0000 7fecf3e00640 1 -- 172.21.15.2:0/13523364 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed00036040 con 0x560d23f7ea60 2024-09-10T08:24:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.441+0000 7fecf3e00640 1 -- 172.21.15.2:0/13523364 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00030890 con 0x560d23f7ea60 2024-09-10T08:24:32.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.441+0000 7fed11842740 1 -- 172.21.15.2:0/13523364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 msgr2=0x560d23f96790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.441+0000 7fed11842740 1 --2- 172.21.15.2:0/13523364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d23f96790 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fed0002ae00 tx=0x7fed00003d10 comp rx=0 tx=0).stop 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.442+0000 7fed11842740 1 -- 172.21.15.2:0/13523364 shutdown_connections 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.442+0000 7fed11842740 1 --2- 172.21.15.2:0/13523364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d23f96790 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.442+0000 7fed11842740 1 --2- 172.21.15.2:0/13523364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 0x560d23f75810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.442+0000 7fed11842740 1 --2- 172.21.15.2:0/13523364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f752d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.442+0000 7fed11842740 1 -- 172.21.15.2:0/13523364 >> 172.21.15.2:0/13523364 conn(0x560d23f6fe10 msgr2=0x560d23f76c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.442+0000 7fed11842740 1 -- 172.21.15.2:0/13523364 shutdown_connections 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.442+0000 7fed11842740 1 -- 172.21.15.2:0/13523364 wait complete. 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.442+0000 7fed11842740 1 -- start start 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.442+0000 7fed11842740 1 Processor -- start 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed11842740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f82f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed11842740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 0x560d23f83440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed11842740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d24000f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed11842740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fed081af470 con 0x560d23f73880 2024-09-10T08:24:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed11842740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed081b08e0 con 0x560d23f7ea60 2024-09-10T08:24:32.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed11842740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed081afb40 con 0x560d23f741a0 2024-09-10T08:24:32.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed11842740 10 client.0 _ll_register_callbacks cb 0x7fecf800b980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:32.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed07e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f82f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed07e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f82f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54330/0 (socket says 172.21.15.2:54330) 2024-09-10T08:24:32.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed07e00640 1 -- 172.21.15.2:0/25625523 learned_addr learned my addr 172.21.15.2:0/25625523 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:32.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed0cc00640 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d24000f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed07400640 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 0x560d23f83440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed07e00640 1 -- 172.21.15.2:0/25625523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 msgr2=0x560d23f83440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:32.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed07e00640 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 0x560d23f83440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed07e00640 1 -- 172.21.15.2:0/25625523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 msgr2=0x560d24000f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:32.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed07e00640 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d24000f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed07e00640 1 -- 172.21.15.2:0/25625523 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d24001530 con 0x560d23f73880 2024-09-10T08:24:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.443+0000 7fed0cc00640 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d24000f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.444+0000 7fed07e00640 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f82f00 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fecf8005d00 tx=0x7fecf8011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf68c1198f1b431c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.444+0000 7fecf1600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:24:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.444+0000 7fecf1600640 1 -- 172.21.15.2:0/25625523 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8010840 con 0x560d23f73880 2024-09-10T08:24:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.444+0000 7fecf1600640 1 -- 172.21.15.2:0/25625523 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fecf80109e0 con 0x560d23f73880 2024-09-10T08:24:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.444+0000 7fed11842740 1 -- 172.21.15.2:0/25625523 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed00036040 con 0x560d23f73880 2024-09-10T08:24:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.444+0000 7fecf1600640 1 -- 172.21.15.2:0/25625523 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fecf801fd40 con 0x560d23f73880 2024-09-10T08:24:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.444+0000 7fed11842740 10 client.20214 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.444+0000 7fed11842740 1 -- 172.21.15.2:0/25625523 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d24002810 con 0x560d23f73880 2024-09-10T08:24:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.445+0000 7fecf1600640 1 -- 172.21.15.2:0/25625523 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed00036040 con 0x560d23f73880 2024-09-10T08:24:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.445+0000 7fecf1600640 1 -- 172.21.15.2:0/25625523 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fecf801b750 con 0x560d23f73880 2024-09-10T08:24:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.445+0000 7fed11842740 10 client.20214 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.445+0000 7fed11842740 10 client.20214 resolve_mds: validated gid 24536 aka daemon mds.b 2024-09-10T08:24:32.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.445+0000 7fed11842740 20 client.20214 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:32.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.445+0000 7fed11842740 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d24002f00 0x560d24005320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:32.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.445+0000 7fed11842740 4 client.20214 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-10T08:24:32.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.445+0000 7fed11842740 1 -- 172.21.15.2:0/25625523 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x560d23ee1dc0 con 0x560d24002f00 2024-09-10T08:24:32.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.445+0000 7fed07400640 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d24002f00 0x560d24005320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:32.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fed07400640 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d24002f00 0x560d24005320 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2313d3a74940a81 server_cookie=be559735d1212809 in_seq=0 out_seq=0 2024-09-10T08:24:32.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fecf1600640 10 client.20214 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:32.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fecf1600640 1 -- 172.21.15.2:0/25625523 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x560d23ee1dc0 con 0x560d24002f00 2024-09-10T08:24:32.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fecf1600640 10 client.20214 handle_command_reply: tid=0 2024-09-10T08:24:32.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fed11842740 1 client.20214 shutdown 2024-09-10T08:24:32.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fed11842740 1 -- 172.21.15.2:0/25625523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 msgr2=0x560d23f82f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:32.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fed11842740 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f82f00 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fecf8005d00 tx=0x7fecf8011d40 comp rx=0 tx=0).stop 2024-09-10T08:24:32.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fed11842740 1 -- 172.21.15.2:0/25625523 shutdown_connections 2024-09-10T08:24:32.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fed11842740 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560d24002f00 0x560d24005320 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fed11842740 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560d23f7ea60 0x560d24000f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fed11842740 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d23f741a0 0x560d23f83440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fecf1600640 0 client.20214 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:32.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fed11842740 1 --2- 172.21.15.2:0/25625523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560d23f73880 0x560d23f82f00 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.446+0000 7fed11842740 1 -- 172.21.15.2:0/25625523 >> 172.21.15.2:0/25625523 conn(0x560d23f6fe10 msgr2=0x560d23f752d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:32.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.447+0000 7fed11842740 1 -- 172.21.15.2:0/25625523 shutdown_connections 2024-09-10T08:24:32.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.447+0000 7fed11842740 1 -- 172.21.15.2:0/25625523 wait complete. 2024-09-10T08:24:32.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.447+0000 7fed11842740 20 client.20214 trim_cache size 0 max 16384 2024-09-10T08:24:32.930 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:24:32.930 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-09-10T08:24:32.930 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24536, 2024-09-10T08:24:32.930 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 60, 2024-09-10T08:24:32.930 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": {} 2024-09-10T08:24:32.930 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:24:32.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.447+0000 7fed0d600640 1 -- 172.21.15.2:0/2101711914 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fecec0a2b80 msgr2=0x7fecec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:32.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.447+0000 7fed0d600640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fecec0a2b80 0x7fecec0a5040 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x560d23f70c70 tx=0x7fecf800b410 comp rx=0 tx=0).stop 2024-09-10T08:24:32.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.448+0000 7fed0d600640 1 -- 172.21.15.2:0/2101711914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed08102340 msgr2=0x7fed08074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:32.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.448+0000 7fed0d600640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed08102340 0x7fed08074a30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fecf400e9f0 tx=0x7fecf400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.448+0000 7fed0d600640 1 -- 172.21.15.2:0/2101711914 shutdown_connections 2024-09-10T08:24:32.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.448+0000 7fed0d600640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fecec0a2b80 0x7fecec0a5040 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.448+0000 7fed0d600640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed081079a0 0x7fed08074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.448+0000 7fed0d600640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed08102340 0x7fed08074a30 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.448+0000 7fed0d600640 1 --2- 172.21.15.2:0/2101711914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed081019a0 0x7fed080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:32.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.448+0000 7fed0d600640 1 -- 172.21.15.2:0/2101711914 >> 172.21.15.2:0/2101711914 conn(0x7fed080fd700 msgr2=0x7fed080feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:32.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.448+0000 7fed0d600640 1 -- 172.21.15.2:0/2101711914 shutdown_connections 2024-09-10T08:24:32.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:32.448+0000 7fed0d600640 1 -- 172.21.15.2:0/2101711914 wait complete. 2024-09-10T08:24:32.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:32 smithi107 ceph-mon[35048]: scrub queued for path: / 2024-09-10T08:24:32.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:32 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:24:32.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:32 smithi107 ceph-mon[35048]: scrub summary: active paths [/] 2024-09-10T08:24:32.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:32 smithi107 ceph-mon[35048]: scrub summary: idle+waiting paths [/] 2024-09-10T08:24:33.698 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:33.733 INFO:tasks.quiescer:done joining Quiescers 2024-09-10T08:24:33.734 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-10T08:24:33.744 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi002.front.sepia.ceph.com 2024-09-10T08:24:33.785 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-10T08:24:33.801 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-10T08:24:33.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:33 smithi155 ceph-mon[35405]: pgmap v879: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:24:33.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:33 smithi107 ceph-mon[35048]: pgmap v879: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:24:33.919 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:33 smithi002 ceph-mon[30563]: pgmap v879: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-10T08:24:34.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.338+0000 7f75cfa00640 1 -- 172.21.15.2:0/1507056367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c8101df0 msgr2=0x7f75c81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:34.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.338+0000 7f75cfa00640 1 --2- 172.21.15.2:0/1507056367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c8101df0 0x7f75c81021d0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009900 tx=0x7f75bc02a410 comp rx=0 tx=0).stop 2024-09-10T08:24:34.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.338+0000 7f75cfa00640 1 -- 172.21.15.2:0/1507056367 shutdown_connections 2024-09-10T08:24:34.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.338+0000 7f75cfa00640 1 --2- 172.21.15.2:0/1507056367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75c8106f20 0x7f75c810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.338+0000 7f75cfa00640 1 --2- 172.21.15.2:0/1507056367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75c81027a0 0x7f75c81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.338+0000 7f75cfa00640 1 --2- 172.21.15.2:0/1507056367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c8101df0 0x7f75c81021d0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.338+0000 7f75cfa00640 1 -- 172.21.15.2:0/1507056367 >> 172.21.15.2:0/1507056367 conn(0x7f75c80fd8b0 msgr2=0x7f75c80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:34.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.338+0000 7f75cfa00640 1 -- 172.21.15.2:0/1507056367 shutdown_connections 2024-09-10T08:24:34.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.338+0000 7f75cfa00640 1 -- 172.21.15.2:0/1507056367 wait complete. 2024-09-10T08:24:34.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.339+0000 7f75cfa00640 1 Processor -- start 2024-09-10T08:24:34.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.339+0000 7f75cfa00640 1 -- start start 2024-09-10T08:24:34.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.339+0000 7f75cfa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75c8101df0 0x7f75c819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.339+0000 7f75cfa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c81027a0 0x7f75c819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.339+0000 7f75cfa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75c8106f20 0x7f75c819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.340+0000 7f75cfa00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8111fd0 con 0x7f75c81027a0 2024-09-10T08:24:34.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.340+0000 7f75cfa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8111e50 con 0x7f75c8101df0 2024-09-10T08:24:34.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.340+0000 7f75cfa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8112150 con 0x7f75c8106f20 2024-09-10T08:24:34.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.340+0000 7f75c7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c81027a0 0x7f75c819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.340+0000 7f75c7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c81027a0 0x7f75c819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54360/0 (socket says 172.21.15.2:54360) 2024-09-10T08:24:34.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.340+0000 7f75c7e00640 1 -- 172.21.15.2:0/2045828952 learned_addr learned my addr 172.21.15.2:0/2045828952 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:34.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.340+0000 7f75cd000640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75c8101df0 0x7f75c819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.340+0000 7f75cda00640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75c8106f20 0x7f75c819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.340+0000 7f75c7e00640 1 -- 172.21.15.2:0/2045828952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75c8106f20 msgr2=0x7f75c819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.340+0000 7f75c7e00640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75c8106f20 0x7f75c819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.340+0000 7f75c7e00640 1 -- 172.21.15.2:0/2045828952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75c8101df0 msgr2=0x7f75c819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.340+0000 7f75c7e00640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75c8101df0 0x7f75c819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.340+0000 7f75c7e00640 1 -- 172.21.15.2:0/2045828952 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c819f640 con 0x7f75c81027a0 2024-09-10T08:24:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.341+0000 7f75c7e00640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c81027a0 0x7f75c819abd0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f75b400e9f0 tx=0x7f75b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.341+0000 7f75c5600640 1 -- 172.21.15.2:0/2045828952 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b400cd80 con 0x7f75c81027a0 2024-09-10T08:24:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.341+0000 7f75c5600640 1 -- 172.21.15.2:0/2045828952 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b4005500 con 0x7f75c81027a0 2024-09-10T08:24:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.341+0000 7f75c5600640 1 -- 172.21.15.2:0/2045828952 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4010690 con 0x7f75c81027a0 2024-09-10T08:24:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.341+0000 7f75cfa00640 1 -- 172.21.15.2:0/2045828952 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c819f930 con 0x7f75c81027a0 2024-09-10T08:24:34.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.341+0000 7f75cfa00640 1 -- 172.21.15.2:0/2045828952 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c81a7210 con 0x7f75c81027a0 2024-09-10T08:24:34.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.344+0000 7f75d2445740 1 Processor -- start 2024-09-10T08:24:34.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.345+0000 7f75d2445740 1 -- start start 2024-09-10T08:24:34.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.345+0000 7f75c5600640 1 -- 172.21.15.2:0/2045828952 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f75b40046c0 con 0x7f75c81027a0 2024-09-10T08:24:34.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.345+0000 7f75d2445740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda6b90 0x560b1cda6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.345+0000 7f75d2445740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda74b0 0x560b1cdb1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.345+0000 7f75d2445740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cdb1d70 0x560b1cdb4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.345+0000 7f75d2445740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560b1cd0d3e0 con 0x560b1cda74b0 2024-09-10T08:24:34.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.345+0000 7f75d2445740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560b1ccb4d80 con 0x560b1cdb1d70 2024-09-10T08:24:34.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.345+0000 7f75d2445740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560b1cd093a0 con 0x560b1cda6b90 2024-09-10T08:24:34.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.345+0000 7f75cd000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda74b0 0x560b1cdb1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.345+0000 7f75cd000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda74b0 0x560b1cdb1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54368/0 (socket says 172.21.15.2:54368) 2024-09-10T08:24:34.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.345+0000 7f75cd000640 1 -- 172.21.15.2:0/911761480 learned_addr learned my addr 172.21.15.2:0/911761480 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:34.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.345+0000 7f75cda00640 1 --2- 172.21.15.2:0/911761480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda6b90 0x560b1cda6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.346+0000 7f75c7e00640 1 --2- 172.21.15.2:0/911761480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cdb1d70 0x560b1cdb4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.346+0000 7f75cd000640 1 -- 172.21.15.2:0/911761480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda6b90 msgr2=0x560b1cda6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:34.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.346+0000 7f75cd000640 1 --2- 172.21.15.2:0/911761480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda6b90 0x560b1cda6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.346+0000 7f75cd000640 1 -- 172.21.15.2:0/911761480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cdb1d70 msgr2=0x560b1cdb4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:34.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.346+0000 7f75cd000640 1 --2- 172.21.15.2:0/911761480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cdb1d70 0x560b1cdb4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.346+0000 7f75cd000640 1 -- 172.21.15.2:0/911761480 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b1ccd3ae0 con 0x560b1cda74b0 2024-09-10T08:24:34.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.346+0000 7f75cd000640 1 --2- 172.21.15.2:0/911761480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda74b0 0x560b1cdb1830 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009900 tx=0x7f75bc005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933496a4764c1b97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:34.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.347+0000 7f75ba000640 1 -- 172.21.15.2:0/911761480 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc035040 con 0x560b1cda74b0 2024-09-10T08:24:34.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.347+0000 7f75ba000640 1 -- 172.21.15.2:0/911761480 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75bc004400 con 0x560b1cda74b0 2024-09-10T08:24:34.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.347+0000 7f75ba000640 1 -- 172.21.15.2:0/911761480 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc0035f0 con 0x560b1cda74b0 2024-09-10T08:24:34.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.347+0000 7f75c5600640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75b00a2c00 0x7f75b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.347+0000 7f75c5600640 1 -- 172.21.15.2:0/2045828952 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f75b40cf810 con 0x7f75c81027a0 2024-09-10T08:24:34.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.347+0000 7f75cda00640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75b00a2c00 0x7f75b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.347+0000 7f75d2445740 1 -- 172.21.15.2:0/911761480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda74b0 msgr2=0x560b1cdb1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:34.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.347+0000 7f75d2445740 1 --2- 172.21.15.2:0/911761480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda74b0 0x560b1cdb1830 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009900 tx=0x7f75bc005de0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.348+0000 7f75d2445740 1 -- 172.21.15.2:0/911761480 shutdown_connections 2024-09-10T08:24:34.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.348+0000 7f75d2445740 1 --2- 172.21.15.2:0/911761480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cdb1d70 0x560b1cdb4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.348+0000 7f75d2445740 1 --2- 172.21.15.2:0/911761480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda74b0 0x560b1cdb1830 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.348+0000 7f75d2445740 1 --2- 172.21.15.2:0/911761480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda6b90 0x560b1cda6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.348+0000 7f75d2445740 1 -- 172.21.15.2:0/911761480 >> 172.21.15.2:0/911761480 conn(0x560b1cda3120 msgr2=0x560b1cda3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:34.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.348+0000 7f75d2445740 1 -- 172.21.15.2:0/911761480 shutdown_connections 2024-09-10T08:24:34.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.348+0000 7f75cda00640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75b00a2c00 0x7f75b00a50c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f75c0005fd0 tx=0x7f75c0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:34.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.348+0000 7f75d2445740 1 -- 172.21.15.2:0/911761480 wait complete. 2024-09-10T08:24:34.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.348+0000 7f75d2445740 1 -- start start 2024-09-10T08:24:34.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.349+0000 7f75d2445740 1 Processor -- start 2024-09-10T08:24:34.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.349+0000 7f75d2445740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 0x560b1cdaec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.349+0000 7f75d2445740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda74b0 0x560b1cdaf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.349+0000 7f75d2445740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cdb1d70 0x560b1cdaf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.349+0000 7f75d2445740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560b1cdb49e0 con 0x560b1cda6b90 2024-09-10T08:24:34.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.349+0000 7f75d2445740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560b1ccc80f0 con 0x560b1cdb1d70 2024-09-10T08:24:34.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.349+0000 7f75d2445740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560b1cdb4b60 con 0x560b1cda74b0 2024-09-10T08:24:34.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.349+0000 7f75d2445740 10 client.0 _ll_register_callbacks cb 0x560b1ccb99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:34.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.349+0000 7f75cd000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 0x560b1cdaec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.349+0000 7f75cd000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 0x560b1cdaec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54376/0 (socket says 172.21.15.2:54376) 2024-09-10T08:24:34.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.349+0000 7f75cd000640 1 -- 172.21.15.2:0/1226606344 learned_addr learned my addr 172.21.15.2:0/1226606344 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:34.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.350+0000 7f75cda00640 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cdb1d70 0x560b1cdaf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.350+0000 7f75c7e00640 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda74b0 0x560b1cdaf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.350+0000 7f75cd000640 1 -- 172.21.15.2:0/1226606344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda74b0 msgr2=0x560b1cdaf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:34.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.350+0000 7f75cd000640 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda74b0 0x560b1cdaf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.350+0000 7f75cd000640 1 -- 172.21.15.2:0/1226606344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cdb1d70 msgr2=0x560b1cdaf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:34.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.350+0000 7f75cd000640 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cdb1d70 0x560b1cdaf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.350+0000 7f75cd000640 1 -- 172.21.15.2:0/1226606344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b1ca7e3c0 con 0x560b1cda6b90 2024-09-10T08:24:34.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.350+0000 7f75cda00640 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cdb1d70 0x560b1cdaf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:34.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.350+0000 7f75cd000640 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 0x560b1cdaec30 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f75bc031550 tx=0x7f75bc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918c2642ca19b935 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:34.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.350+0000 7f75aea00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:24:34.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.351+0000 7f75aea00640 1 -- 172.21.15.2:0/1226606344 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc035050 con 0x560b1cda6b90 2024-09-10T08:24:34.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.351+0000 7f75aea00640 1 -- 172.21.15.2:0/1226606344 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f75bc004200 con 0x560b1cda6b90 2024-09-10T08:24:34.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.351+0000 7f75d2445740 1 -- 172.21.15.2:0/1226606344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75bc004400 con 0x560b1cda6b90 2024-09-10T08:24:34.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.351+0000 7f75aea00640 1 -- 172.21.15.2:0/1226606344 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f75bc009520 con 0x560b1cda6b90 2024-09-10T08:24:34.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.351+0000 7f75d2445740 10 client.20226 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:34.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.351+0000 7f75d2445740 1 -- 172.21.15.2:0/1226606344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560b1cdc0660 con 0x560b1cda6b90 2024-09-10T08:24:34.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.351+0000 7f75aea00640 1 -- 172.21.15.2:0/1226606344 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75bc004400 con 0x560b1cda6b90 2024-09-10T08:24:34.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.352+0000 7f75aea00640 1 -- 172.21.15.2:0/1226606344 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f75bc003b50 con 0x560b1cda6b90 2024-09-10T08:24:34.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.352+0000 7f75d2445740 10 client.20226 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:34.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.352+0000 7f75d2445740 10 client.20226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:34.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.352+0000 7f75d2445740 20 client.20226 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:34.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.352+0000 7f75d2445740 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560b1cdc19b0 0x560b1cdc3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.352+0000 7f75d2445740 4 client.20226 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:34.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.352+0000 7f75d2445740 1 -- 172.21.15.2:0/1226606344 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b1cd1a950 con 0x560b1cdc19b0 2024-09-10T08:24:34.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.352+0000 7f75c7e00640 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560b1cdc19b0 0x560b1cdc3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.353+0000 7f75c7e00640 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560b1cdc19b0 0x560b1cdc3da0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2366d043d884954 server_cookie=9612a76a6d1249c4 in_seq=0 out_seq=0 2024-09-10T08:24:34.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.353+0000 7f75aea00640 10 client.20226 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:34.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.353+0000 7f75aea00640 1 -- 172.21.15.2:0/1226606344 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b1cd1a950 con 0x560b1cdc19b0 2024-09-10T08:24:34.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.353+0000 7f75aea00640 10 client.20226 handle_command_reply: tid=0 2024-09-10T08:24:34.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.354+0000 7f75d2445740 1 client.20226 shutdown 2024-09-10T08:24:34.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.354+0000 7f75d2445740 1 -- 172.21.15.2:0/1226606344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 msgr2=0x560b1cdaec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:34.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.354+0000 7f75d2445740 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 0x560b1cdaec30 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f75bc031550 tx=0x7f75bc02a410 comp rx=0 tx=0).stop 2024-09-10T08:24:34.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.354+0000 7f75d2445740 1 -- 172.21.15.2:0/1226606344 shutdown_connections 2024-09-10T08:24:34.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.354+0000 7f75d2445740 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560b1cdc19b0 0x560b1cdc3da0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.354+0000 7f75d2445740 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cdb1d70 0x560b1cdaf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.354+0000 7f75d2445740 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda74b0 0x560b1cdaf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.354+0000 7f75d2445740 1 --2- 172.21.15.2:0/1226606344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 0x560b1cdaec30 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.354+0000 7f75d2445740 1 -- 172.21.15.2:0/1226606344 >> 172.21.15.2:0/1226606344 conn(0x560b1cda3120 msgr2=0x560b1cda8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:34.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.354+0000 7f75aea00640 0 client.20226 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:34.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.354+0000 7f75d2445740 1 -- 172.21.15.2:0/1226606344 shutdown_connections 2024-09-10T08:24:34.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.354+0000 7f75d2445740 1 -- 172.21.15.2:0/1226606344 wait complete. 2024-09-10T08:24:34.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.354+0000 7f75d2445740 20 client.20226 trim_cache size 0 max 16384 2024-09-10T08:24:34.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.373+0000 7f75d2445740 1 Processor -- start 2024-09-10T08:24:34.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.373+0000 7f75d2445740 1 -- start start 2024-09-10T08:24:34.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.373+0000 7f75d2445740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cda6b90 0x560b1cda95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.373+0000 7f75d2445740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda74b0 0x560b1cdabe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.373+0000 7f75d2445740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cdb1d70 0x560b1cdac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.373+0000 7f75d2445740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81b3830 con 0x560b1cdb1d70 2024-09-10T08:24:34.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.373+0000 7f75d2445740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81b1100 con 0x560b1cda6b90 2024-09-10T08:24:34.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.373+0000 7f75d2445740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81aef50 con 0x560b1cda74b0 2024-09-10T08:24:34.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.374+0000 7f75cd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cda6b90 0x560b1cda95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.374+0000 7f75c7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda74b0 0x560b1cdabe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.374+0000 7f75cd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cda6b90 0x560b1cda95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:43160/0 (socket says 172.21.15.2:43160) 2024-09-10T08:24:34.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.374+0000 7f75cd000640 1 -- 172.21.15.2:0/2691615460 learned_addr learned my addr 172.21.15.2:0/2691615460 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:34.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.374+0000 7f75cda00640 1 --2- 172.21.15.2:0/2691615460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cdb1d70 0x560b1cdac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.374+0000 7f75cd000640 1 -- 172.21.15.2:0/2691615460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda74b0 msgr2=0x560b1cdabe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:34.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.374+0000 7f75cd000640 1 --2- 172.21.15.2:0/2691615460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda74b0 0x560b1cdabe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.374+0000 7f75cd000640 1 -- 172.21.15.2:0/2691615460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cdb1d70 msgr2=0x560b1cdac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:34.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.374+0000 7f75cd000640 1 --2- 172.21.15.2:0/2691615460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cdb1d70 0x560b1cdac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.374+0000 7f75cd000640 1 -- 172.21.15.2:0/2691615460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b1cda29b0 con 0x560b1cda6b90 2024-09-10T08:24:34.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.374+0000 7f75c7e00640 1 --2- 172.21.15.2:0/2691615460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda74b0 0x560b1cdabe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:34.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.374+0000 7f75cd000640 1 --2- 172.21.15.2:0/2691615460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cda6b90 0x560b1cda95f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f75bc02ac00 tx=0x7f75bc005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af40c37f4cc58250 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:34.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.375+0000 7f75b9600640 1 -- 172.21.15.2:0/2691615460 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc035040 con 0x560b1cda6b90 2024-09-10T08:24:34.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.375+0000 7f75b9600640 1 -- 172.21.15.2:0/2691615460 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75bc0048d0 con 0x560b1cda6b90 2024-09-10T08:24:34.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.375+0000 7f75b9600640 1 -- 172.21.15.2:0/2691615460 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc0052f0 con 0x560b1cda6b90 2024-09-10T08:24:34.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.375+0000 7f75d2445740 1 -- 172.21.15.2:0/2691615460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cda6b90 msgr2=0x560b1cda95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:34.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.375+0000 7f75d2445740 1 --2- 172.21.15.2:0/2691615460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cda6b90 0x560b1cda95f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f75bc02ac00 tx=0x7f75bc005bd0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.375+0000 7f75d2445740 1 -- 172.21.15.2:0/2691615460 shutdown_connections 2024-09-10T08:24:34.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.375+0000 7f75d2445740 1 --2- 172.21.15.2:0/2691615460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cdb1d70 0x560b1cdac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.375+0000 7f75d2445740 1 --2- 172.21.15.2:0/2691615460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cda74b0 0x560b1cdabe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.375+0000 7f75d2445740 1 --2- 172.21.15.2:0/2691615460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cda6b90 0x560b1cda95f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.375+0000 7f75d2445740 1 -- 172.21.15.2:0/2691615460 >> 172.21.15.2:0/2691615460 conn(0x560b1cda3120 msgr2=0x560b1cdadd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.376+0000 7f75d2445740 1 -- 172.21.15.2:0/2691615460 shutdown_connections 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.376+0000 7f75d2445740 1 -- 172.21.15.2:0/2691615460 wait complete. 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.376+0000 7f75d2445740 1 -- start start 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.376+0000 7f75d2445740 1 Processor -- start 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75d2445740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 0x560b1cdc42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75d2445740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cda74b0 0x560b1cdc4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75d2445740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cdb1d70 0x560b1ce2a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75d2445740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81af5c0 con 0x560b1cda6b90 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75d2445740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81b0a30 con 0x560b1cda74b0 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75d2445740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81afc90 con 0x560b1cdb1d70 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75d2445740 10 client.0 _ll_register_callbacks cb 0x7f75bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75cd000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 0x560b1cdc42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75cd000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 0x560b1cdc42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54390/0 (socket says 172.21.15.2:54390) 2024-09-10T08:24:34.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75cd000640 1 -- 172.21.15.2:0/1265287304 learned_addr learned my addr 172.21.15.2:0/1265287304 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:34.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75c7e00640 1 --2- 172.21.15.2:0/1265287304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cda74b0 0x560b1cdc4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75cda00640 1 --2- 172.21.15.2:0/1265287304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cdb1d70 0x560b1ce2a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75cd000640 1 -- 172.21.15.2:0/1265287304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cdb1d70 msgr2=0x560b1ce2a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:34.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75cd000640 1 --2- 172.21.15.2:0/1265287304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cdb1d70 0x560b1ce2a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75cd000640 1 -- 172.21.15.2:0/1265287304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cda74b0 msgr2=0x560b1cdc4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:34.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75cd000640 1 --2- 172.21.15.2:0/1265287304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cda74b0 0x560b1cdc4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.377+0000 7f75cd000640 1 -- 172.21.15.2:0/1265287304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b1ce2a8d0 con 0x560b1cda6b90 2024-09-10T08:24:34.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.378+0000 7f75cd000640 1 --2- 172.21.15.2:0/1265287304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 0x560b1cdc42d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0096c0 tx=0x7f75bc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8b471539a2fe1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:34.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.378+0000 7f75aea00640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:24:34.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.378+0000 7f75aea00640 1 -- 172.21.15.2:0/1265287304 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc035040 con 0x560b1cda6b90 2024-09-10T08:24:34.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.378+0000 7f75aea00640 1 -- 172.21.15.2:0/1265287304 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f75bc031c90 con 0x560b1cda6b90 2024-09-10T08:24:34.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.378+0000 7f75d2445740 1 -- 172.21.15.2:0/1265287304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75bc0048d0 con 0x560b1cda6b90 2024-09-10T08:24:34.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.378+0000 7f75aea00640 1 -- 172.21.15.2:0/1265287304 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f75bc06c050 con 0x560b1cda6b90 2024-09-10T08:24:34.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.378+0000 7f75d2445740 10 client.20232 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:34.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.378+0000 7f75d2445740 1 -- 172.21.15.2:0/1265287304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560b1ce2bb80 con 0x560b1cda6b90 2024-09-10T08:24:34.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.379+0000 7f75aea00640 1 -- 172.21.15.2:0/1265287304 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75bc0048d0 con 0x560b1cda6b90 2024-09-10T08:24:34.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.379+0000 7f75aea00640 1 -- 172.21.15.2:0/1265287304 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f75bc003b20 con 0x560b1cda6b90 2024-09-10T08:24:34.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.379+0000 7f75d2445740 10 client.20232 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:34.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.379+0000 7f75d2445740 10 client.20232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:34.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.379+0000 7f75d2445740 20 client.20232 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:34.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.379+0000 7f75d2445740 1 --2- 172.21.15.2:0/1265287304 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560b1ce2c000 0x560b1ce2e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:34.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.379+0000 7f75d2445740 4 client.20232 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:24:34.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.379+0000 7f75d2445740 1 -- 172.21.15.2:0/1265287304 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b1cd14e60 con 0x560b1ce2c000 2024-09-10T08:24:34.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.379+0000 7f75c7e00640 1 --2- 172.21.15.2:0/1265287304 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560b1ce2c000 0x560b1ce2e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:34.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.380+0000 7f75c7e00640 1 --2- 172.21.15.2:0/1265287304 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560b1ce2c000 0x560b1ce2e3f0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcf22012046a50cc server_cookie=df552b1d23b2049a in_seq=0 out_seq=0 2024-09-10T08:24:34.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.380+0000 7f75aea00640 10 client.20232 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.380+0000 7f75aea00640 1 -- 172.21.15.2:0/1265287304 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560b1cd14e60 con 0x560b1ce2c000 2024-09-10T08:24:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.380+0000 7f75aea00640 10 client.20232 handle_command_reply: tid=0 2024-09-10T08:24:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.380+0000 7f75d2445740 1 client.20232 shutdown 2024-09-10T08:24:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.380+0000 7f75d2445740 1 -- 172.21.15.2:0/1265287304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 msgr2=0x560b1cdc42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.380+0000 7f75d2445740 1 --2- 172.21.15.2:0/1265287304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 0x560b1cdc42d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0096c0 tx=0x7f75bc0045e0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.380+0000 7f75d2445740 1 -- 172.21.15.2:0/1265287304 shutdown_connections 2024-09-10T08:24:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.380+0000 7f75d2445740 1 --2- 172.21.15.2:0/1265287304 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x560b1ce2c000 0x560b1ce2e3f0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.381+0000 7f75d2445740 1 --2- 172.21.15.2:0/1265287304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b1cdb1d70 0x560b1ce2a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.381+0000 7f75d2445740 1 --2- 172.21.15.2:0/1265287304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b1cda74b0 0x560b1cdc4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.381+0000 7f75d2445740 1 --2- 172.21.15.2:0/1265287304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b1cda6b90 0x560b1cdc42d0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.381+0000 7f75d2445740 1 -- 172.21.15.2:0/1265287304 >> 172.21.15.2:0/1265287304 conn(0x560b1cda3120 msgr2=0x560b1cda19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.381+0000 7f75aea00640 0 client.20232 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.381+0000 7f75d2445740 1 -- 172.21.15.2:0/1265287304 shutdown_connections 2024-09-10T08:24:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.381+0000 7f75d2445740 1 -- 172.21.15.2:0/1265287304 wait complete. 2024-09-10T08:24:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.381+0000 7f75d2445740 20 client.20232 trim_cache size 0 max 16384 2024-09-10T08:24:34.385 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:24:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-09-10T08:24:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-09-10T08:24:34.385 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:24:34.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.381+0000 7f75cfa00640 1 -- 172.21.15.2:0/2045828952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75b00a2c00 msgr2=0x7f75b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:34.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.381+0000 7f75cfa00640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75b00a2c00 0x7f75b00a50c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f75c0005fd0 tx=0x7f75c0009500 comp rx=0 tx=0).stop 2024-09-10T08:24:34.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.381+0000 7f75cfa00640 1 -- 172.21.15.2:0/2045828952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c81027a0 msgr2=0x7f75c819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:34.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.381+0000 7f75cfa00640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c81027a0 0x7f75c819abd0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f75b400e9f0 tx=0x7f75b400edf0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.382+0000 7f75cfa00640 1 -- 172.21.15.2:0/2045828952 shutdown_connections 2024-09-10T08:24:34.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.382+0000 7f75cfa00640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f75b00a2c00 0x7f75b00a50c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.382+0000 7f75cfa00640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75c8106f20 0x7f75c819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.382+0000 7f75cfa00640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c81027a0 0x7f75c819abd0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.382+0000 7f75cfa00640 1 --2- 172.21.15.2:0/2045828952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75c8101df0 0x7f75c819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:34.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.382+0000 7f75cfa00640 1 -- 172.21.15.2:0/2045828952 >> 172.21.15.2:0/2045828952 conn(0x7f75c80fd8b0 msgr2=0x7f75c80ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:34.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.382+0000 7f75cfa00640 1 -- 172.21.15.2:0/2045828952 shutdown_connections 2024-09-10T08:24:34.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:34.382+0000 7f75cfa00640 1 -- 172.21.15.2:0/2045828952 wait complete. 2024-09-10T08:24:34.877 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:34 smithi002 ceph-mon[30563]: scrub summary: idle 2024-09-10T08:24:34.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:34 smithi155 ceph-mon[35405]: scrub summary: idle 2024-09-10T08:24:34.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:34 smithi107 ceph-mon[35048]: scrub summary: idle 2024-09-10T08:24:35.389 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:24:35.389 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:24:35.768 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:35.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:35 smithi002 ceph-mon[30563]: pgmap v880: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:24:35.906 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:35 smithi155 ceph-mon[35405]: pgmap v880: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:24:35.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:35 smithi107 ceph-mon[35048]: pgmap v880: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T08:24:36.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.364+0000 7f316ca00640 1 -- 172.21.15.2:0/1221859594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3168109140 msgr2=0x7f3168110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:36.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.364+0000 7f316ca00640 1 --2- 172.21.15.2:0/1221859594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3168109140 0x7f3168110cc0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3160009ca0 tx=0x7f316002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:24:36.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.364+0000 7f316ca00640 1 -- 172.21.15.2:0/1221859594 shutdown_connections 2024-09-10T08:24:36.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.364+0000 7f316ca00640 1 --2- 172.21.15.2:0/1221859594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3168109140 0x7f3168110cc0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:36.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.364+0000 7f316ca00640 1 --2- 172.21.15.2:0/1221859594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31681049c0 0x7f3168108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:36.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.364+0000 7f316ca00640 1 --2- 172.21.15.2:0/1221859594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3168104010 0x7f31681043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:36.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.364+0000 7f316ca00640 1 -- 172.21.15.2:0/1221859594 >> 172.21.15.2:0/1221859594 conn(0x7f31680ffa50 msgr2=0x7f3168101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:36.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.364+0000 7f316ca00640 1 -- 172.21.15.2:0/1221859594 shutdown_connections 2024-09-10T08:24:36.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.364+0000 7f316ca00640 1 -- 172.21.15.2:0/1221859594 wait complete. 2024-09-10T08:24:36.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.365+0000 7f316ca00640 1 Processor -- start 2024-09-10T08:24:36.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.365+0000 7f316ca00640 1 -- start start 2024-09-10T08:24:36.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.365+0000 7f316ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3168104010 0x7f316819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:36.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.365+0000 7f316ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31681049c0 0x7f316819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:36.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.365+0000 7f316ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3168109140 0x7f316819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:36.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.365+0000 7f316ca00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3168114170 con 0x7f31681049c0 2024-09-10T08:24:36.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.365+0000 7f316ca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3168113ff0 con 0x7f3168109140 2024-09-10T08:24:36.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.365+0000 7f316ca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f31681142f0 con 0x7f3168104010 2024-09-10T08:24:36.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.366+0000 7f315fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31681049c0 0x7f316819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:36.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.366+0000 7f3165e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3168104010 0x7f316819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:36.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.366+0000 7f315fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31681049c0 0x7f316819ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54412/0 (socket says 172.21.15.2:54412) 2024-09-10T08:24:36.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.366+0000 7f315fe00640 1 -- 172.21.15.2:0/1386005858 learned_addr learned my addr 172.21.15.2:0/1386005858 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:36.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.366+0000 7f3166800640 1 --2- 172.21.15.2:0/1386005858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3168109140 0x7f316819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:36.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.366+0000 7f315fe00640 1 -- 172.21.15.2:0/1386005858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3168104010 msgr2=0x7f316819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:36.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.366+0000 7f315fe00640 1 --2- 172.21.15.2:0/1386005858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3168104010 0x7f316819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:36.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.366+0000 7f315fe00640 1 -- 172.21.15.2:0/1386005858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3168109140 msgr2=0x7f316819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:36.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.366+0000 7f315fe00640 1 --2- 172.21.15.2:0/1386005858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3168109140 0x7f316819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:36.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.366+0000 7f315fe00640 1 -- 172.21.15.2:0/1386005858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316819f680 con 0x7f31681049c0 2024-09-10T08:24:36.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.367+0000 7f315fe00640 1 --2- 172.21.15.2:0/1386005858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31681049c0 0x7f316819ac10 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f315000e9f0 tx=0x7f315000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:36.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.367+0000 7f315ea00640 1 -- 172.21.15.2:0/1386005858 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315000cd80 con 0x7f31681049c0 2024-09-10T08:24:36.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.367+0000 7f315ea00640 1 -- 172.21.15.2:0/1386005858 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3150005500 con 0x7f31681049c0 2024-09-10T08:24:36.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.367+0000 7f316ca00640 1 -- 172.21.15.2:0/1386005858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316819f970 con 0x7f31681049c0 2024-09-10T08:24:36.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.367+0000 7f315ea00640 1 -- 172.21.15.2:0/1386005858 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150010690 con 0x7f31681049c0 2024-09-10T08:24:36.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.367+0000 7f316ca00640 1 -- 172.21.15.2:0/1386005858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31681a7250 con 0x7f31681049c0 2024-09-10T08:24:36.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.368+0000 7f316ca00640 1 -- 172.21.15.2:0/1386005858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31681043f0 con 0x7f31681049c0 2024-09-10T08:24:36.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.370+0000 7f315ea00640 1 -- 172.21.15.2:0/1386005858 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f31500046c0 con 0x7f31681049c0 2024-09-10T08:24:36.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.372+0000 7f315ea00640 1 --2- 172.21.15.2:0/1386005858 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f314c0a2b80 0x7f314c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:36.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.372+0000 7f315ea00640 1 -- 172.21.15.2:0/1386005858 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f31500cf830 con 0x7f31681049c0 2024-09-10T08:24:36.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.372+0000 7f3165e00640 1 --2- 172.21.15.2:0/1386005858 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f314c0a2b80 0x7f314c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:36.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.373+0000 7f3165e00640 1 --2- 172.21.15.2:0/1386005858 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f314c0a2b80 0x7f314c0a5040 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3158009bd0 tx=0x7f3158005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:36.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.376+0000 7f315ea00640 1 -- 172.21.15.2:0/1386005858 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f315009c340 con 0x7f31681049c0 2024-09-10T08:24:36.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.595+0000 7f316ca00640 1 -- 172.21.15.2:0/1386005858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f316810bdf0 con 0x7f31681049c0 2024-09-10T08:24:36.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.595+0000 7f315ea00640 1 -- 172.21.15.2:0/1386005858 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f315009bc40 con 0x7f31681049c0 2024-09-10T08:24:36.595 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:24:36.595 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:24:36.595 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:24:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.598+0000 7f316ca00640 1 -- 172.21.15.2:0/1386005858 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f314c0a2b80 msgr2=0x7f314c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.598+0000 7f316ca00640 1 --2- 172.21.15.2:0/1386005858 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f314c0a2b80 0x7f314c0a5040 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3158009bd0 tx=0x7f3158005f90 comp rx=0 tx=0).stop 2024-09-10T08:24:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.598+0000 7f316ca00640 1 -- 172.21.15.2:0/1386005858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31681049c0 msgr2=0x7f316819ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.598+0000 7f316ca00640 1 --2- 172.21.15.2:0/1386005858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31681049c0 0x7f316819ac10 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f315000e9f0 tx=0x7f315000edf0 comp rx=0 tx=0).stop 2024-09-10T08:24:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.598+0000 7f316ca00640 1 -- 172.21.15.2:0/1386005858 shutdown_connections 2024-09-10T08:24:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.598+0000 7f316ca00640 1 --2- 172.21.15.2:0/1386005858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3168109140 0x7f316819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.598+0000 7f316ca00640 1 --2- 172.21.15.2:0/1386005858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31681049c0 0x7f316819ac10 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.598+0000 7f316ca00640 1 --2- 172.21.15.2:0/1386005858 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f314c0a2b80 0x7f314c0a5040 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.598+0000 7f316ca00640 1 --2- 172.21.15.2:0/1386005858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3168104010 0x7f316819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.598+0000 7f316ca00640 1 -- 172.21.15.2:0/1386005858 >> 172.21.15.2:0/1386005858 conn(0x7f31680ffa50 msgr2=0x7f3168100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.598+0000 7f316ca00640 1 -- 172.21.15.2:0/1386005858 shutdown_connections 2024-09-10T08:24:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:36.598+0000 7f316ca00640 1 -- 172.21.15.2:0/1386005858 wait complete. 2024-09-10T08:24:36.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:36 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1386005858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:36.956 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:36 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1386005858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:37.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:36 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1386005858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:24:37.362 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:24:37.744 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:37.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:37 smithi107 ceph-mon[35048]: pgmap v881: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-10T08:24:37.948 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:37 smithi002 ceph-mon[30563]: pgmap v881: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-10T08:24:38.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:37 smithi155 ceph-mon[35405]: pgmap v881: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-10T08:24:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.391+0000 7f8255000640 1 -- 172.21.15.2:0/3802507551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501019c0 msgr2=0x7f8250101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.391+0000 7f8255000640 1 --2- 172.21.15.2:0/3802507551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501019c0 0x7f8250101e20 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8238009930 tx=0x7f823802a420 comp rx=0 tx=0).stop 2024-09-10T08:24:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.391+0000 7f8255000640 1 -- 172.21.15.2:0/3802507551 shutdown_connections 2024-09-10T08:24:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.391+0000 7f8255000640 1 --2- 172.21.15.2:0/3802507551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8250102360 0x7f825010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.391+0000 7f8255000640 1 --2- 172.21.15.2:0/3802507551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501019c0 0x7f8250101e20 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.391+0000 7f8255000640 1 --2- 172.21.15.2:0/3802507551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82501079c0 0x7f8250107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.391+0000 7f8255000640 1 -- 172.21.15.2:0/3802507551 >> 172.21.15.2:0/3802507551 conn(0x7f82500fd6e0 msgr2=0x7f82500ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.392+0000 7f8255000640 1 -- 172.21.15.2:0/3802507551 shutdown_connections 2024-09-10T08:24:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.392+0000 7f8255000640 1 -- 172.21.15.2:0/3802507551 wait complete. 2024-09-10T08:24:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.393+0000 7f8255000640 1 Processor -- start 2024-09-10T08:24:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.393+0000 7f8255000640 1 -- start start 2024-09-10T08:24:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.393+0000 7f8255000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82501019c0 0x7f825019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.393+0000 7f8255000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8250102360 0x7f825019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.393+0000 7f8255000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501079c0 0x7f825019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.393+0000 7f8255000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8250111e10 con 0x7f82501079c0 2024-09-10T08:24:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.393+0000 7f8255000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8250111c90 con 0x7f8250102360 2024-09-10T08:24:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.393+0000 7f8255000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8250111f90 con 0x7f82501019c0 2024-09-10T08:24:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.393+0000 7f824fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501079c0 0x7f825019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.393+0000 7f824fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501079c0 0x7f825019ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39390/0 (socket says 172.21.15.2:39390) 2024-09-10T08:24:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.393+0000 7f824fe00640 1 -- 172.21.15.2:0/697381462 learned_addr learned my addr 172.21.15.2:0/697381462 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:38.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.394+0000 7f824f400640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82501019c0 0x7f825019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.393+0000 7f8247e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8250102360 0x7f825019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.394+0000 7f8247e00640 1 -- 172.21.15.2:0/697381462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82501019c0 msgr2=0x7f825019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:38.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.394+0000 7f8247e00640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82501019c0 0x7f825019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.394+0000 7f8247e00640 1 -- 172.21.15.2:0/697381462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501079c0 msgr2=0x7f825019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.394+0000 7f8247e00640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501079c0 0x7f825019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.394+0000 7f8247e00640 1 -- 172.21.15.2:0/697381462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825019f460 con 0x7f8250102360 2024-09-10T08:24:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.394+0000 7f824fe00640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501079c0 0x7f825019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.394+0000 7f8247e00640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8250102360 0x7f825019a9f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f823802a940 tx=0x7f8238003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.395+0000 7f824cc00640 1 -- 172.21.15.2:0/697381462 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823803f070 con 0x7f8250102360 2024-09-10T08:24:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.395+0000 7f8255000640 1 -- 172.21.15.2:0/697381462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825019f6f0 con 0x7f8250102360 2024-09-10T08:24:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.395+0000 7f824cc00640 1 -- 172.21.15.2:0/697381462 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8238031c90 con 0x7f8250102360 2024-09-10T08:24:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.395+0000 7f824cc00640 1 -- 172.21.15.2:0/697381462 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238035030 con 0x7f8250102360 2024-09-10T08:24:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.395+0000 7f8255000640 1 -- 172.21.15.2:0/697381462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82501a7030 con 0x7f8250102360 2024-09-10T08:24:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.398+0000 7f8258fdd740 1 Processor -- start 2024-09-10T08:24:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.398+0000 7f8258fdd740 1 -- start start 2024-09-10T08:24:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.398+0000 7f824cc00640 1 -- 172.21.15.2:0/697381462 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8238042070 con 0x7f8250102360 2024-09-10T08:24:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.398+0000 7f8258fdd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca643e40 0x557eca644220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.398+0000 7f8258fdd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca644760 0x557eca64eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.398+0000 7f8258fdd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca64f020 0x557eca651410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.398+0000 7f8258fdd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557eca5aa3e0 con 0x557eca64f020 2024-09-10T08:24:38.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.398+0000 7f8258fdd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557eca551d80 con 0x557eca644760 2024-09-10T08:24:38.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.398+0000 7f8258fdd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557eca5a63a0 con 0x557eca643e40 2024-09-10T08:24:38.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.399+0000 7f824fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca643e40 0x557eca644220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.399+0000 7f824fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca643e40 0x557eca644220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50892/0 (socket says 172.21.15.2:50892) 2024-09-10T08:24:38.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.399+0000 7f824fe00640 1 -- 172.21.15.2:0/1971063373 learned_addr learned my addr 172.21.15.2:0/1971063373 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.399+0000 7f8247e00640 1 --2- 172.21.15.2:0/1971063373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca64f020 0x557eca651410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.399+0000 7f824f400640 1 --2- 172.21.15.2:0/1971063373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca644760 0x557eca64eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.399+0000 7f824fe00640 1 -- 172.21.15.2:0/1971063373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca644760 msgr2=0x557eca64eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.399+0000 7f824fe00640 1 --2- 172.21.15.2:0/1971063373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca644760 0x557eca64eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.399+0000 7f824fe00640 1 -- 172.21.15.2:0/1971063373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca64f020 msgr2=0x557eca651410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.399+0000 7f824fe00640 1 --2- 172.21.15.2:0/1971063373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca64f020 0x557eca651410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.399+0000 7f824fe00640 1 -- 172.21.15.2:0/1971063373 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eca570ae0 con 0x557eca643e40 2024-09-10T08:24:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.399+0000 7f824f400640 1 --2- 172.21.15.2:0/1971063373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca644760 0x557eca64eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.399+0000 7f8247e00640 1 --2- 172.21.15.2:0/1971063373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca64f020 0x557eca651410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.399+0000 7f824fe00640 1 --2- 172.21.15.2:0/1971063373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca643e40 0x557eca644220 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8248010890 tx=0x7f8248010d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63f4cf1b2ac05d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.400+0000 7f8244c00640 1 -- 172.21.15.2:0/1971063373 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824800ebf0 con 0x557eca643e40 2024-09-10T08:24:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.400+0000 7f8244c00640 1 -- 172.21.15.2:0/1971063373 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8248015070 con 0x557eca643e40 2024-09-10T08:24:38.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.400+0000 7f8244c00640 1 -- 172.21.15.2:0/1971063373 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82480116a0 con 0x557eca643e40 2024-09-10T08:24:38.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.400+0000 7f824cc00640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82340a2b80 0x7f82340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.400+0000 7f8258fdd740 1 -- 172.21.15.2:0/1971063373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca643e40 msgr2=0x557eca644220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:38.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.400+0000 7f8258fdd740 1 --2- 172.21.15.2:0/1971063373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca643e40 0x557eca644220 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8248010890 tx=0x7f8248010d50 comp rx=0 tx=0).stop 2024-09-10T08:24:38.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.400+0000 7f824cc00640 1 -- 172.21.15.2:0/697381462 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f82380ef9c0 con 0x7f8250102360 2024-09-10T08:24:38.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.401+0000 7f824f400640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82340a2b80 0x7f82340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.401+0000 7f8258fdd740 1 -- 172.21.15.2:0/1971063373 shutdown_connections 2024-09-10T08:24:38.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.401+0000 7f8258fdd740 1 --2- 172.21.15.2:0/1971063373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca64f020 0x557eca651410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.401+0000 7f8258fdd740 1 --2- 172.21.15.2:0/1971063373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca644760 0x557eca64eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.401+0000 7f8258fdd740 1 --2- 172.21.15.2:0/1971063373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca643e40 0x557eca644220 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.401+0000 7f8258fdd740 1 -- 172.21.15.2:0/1971063373 >> 172.21.15.2:0/1971063373 conn(0x557eca6403d0 msgr2=0x557eca6407b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.401+0000 7f8258fdd740 1 -- 172.21.15.2:0/1971063373 shutdown_connections 2024-09-10T08:24:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.401+0000 7f8258fdd740 1 -- 172.21.15.2:0/1971063373 wait complete. 2024-09-10T08:24:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.401+0000 7f8258fdd740 1 -- start start 2024-09-10T08:24:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.401+0000 7f824f400640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82340a2b80 0x7f82340a5040 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x557eca641230 tx=0x7f8240005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f8258fdd740 1 Processor -- start 2024-09-10T08:24:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f8258fdd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca643e40 0x557eca64bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f8258fdd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca644760 0x557eca64c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f8258fdd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca64f020 0x557eca64c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f8258fdd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557eca651bf0 con 0x557eca643e40 2024-09-10T08:24:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f8258fdd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557eca5650f0 con 0x557eca64f020 2024-09-10T08:24:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f8258fdd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557eca651d70 con 0x557eca644760 2024-09-10T08:24:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f8258fdd740 10 client.0 _ll_register_callbacks cb 0x557eca54cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f824f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca643e40 0x557eca64bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f824fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca64f020 0x557eca64c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f824f400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca643e40 0x557eca64bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39414/0 (socket says 172.21.15.2:39414) 2024-09-10T08:24:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f824fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca64f020 0x557eca64c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41386/0 (socket says 172.21.15.2:41386) 2024-09-10T08:24:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f824f400640 1 -- 172.21.15.2:0/3245456202 learned_addr learned my addr 172.21.15.2:0/3245456202 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.402+0000 7f8247e00640 1 --2- 172.21.15.2:0/3245456202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca644760 0x557eca64c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.403+0000 7f824fe00640 1 -- 172.21.15.2:0/3245456202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca644760 msgr2=0x557eca64c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.403+0000 7f824fe00640 1 --2- 172.21.15.2:0/3245456202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca644760 0x557eca64c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.403+0000 7f824fe00640 1 -- 172.21.15.2:0/3245456202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca643e40 msgr2=0x557eca64bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.403+0000 7f824fe00640 1 --2- 172.21.15.2:0/3245456202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca643e40 0x557eca64bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.403+0000 7f824fe00640 1 -- 172.21.15.2:0/3245456202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eca31b3c0 con 0x557eca64f020 2024-09-10T08:24:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.403+0000 7f8247e00640 1 --2- 172.21.15.2:0/3245456202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca644760 0x557eca64c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.403+0000 7f824fe00640 1 --2- 172.21.15.2:0/3245456202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca64f020 0x557eca64c950 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f824800e720 tx=0x7f824800eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa34d5a5dc9d86c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.403+0000 7f823d600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.403+0000 7f823d600640 1 -- 172.21.15.2:0/3245456202 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824801a680 con 0x557eca64f020 2024-09-10T08:24:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.404+0000 7f8258fdd740 1 -- 172.21.15.2:0/3245456202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8248015070 con 0x557eca64f020 2024-09-10T08:24:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.404+0000 7f823d600640 1 -- 172.21.15.2:0/3245456202 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f824801a820 con 0x557eca64f020 2024-09-10T08:24:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.404+0000 7f823d600640 1 -- 172.21.15.2:0/3245456202 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f824802f050 con 0x557eca64f020 2024-09-10T08:24:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.404+0000 7f8258fdd740 10 client.30266 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.404+0000 7f8258fdd740 1 -- 172.21.15.2:0/3245456202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557eca65dc80 con 0x557eca64f020 2024-09-10T08:24:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.404+0000 7f823d600640 1 -- 172.21.15.2:0/3245456202 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8248015070 con 0x557eca64f020 2024-09-10T08:24:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.404+0000 7f823d600640 1 -- 172.21.15.2:0/3245456202 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8248011440 con 0x557eca64f020 2024-09-10T08:24:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.404+0000 7f8258fdd740 10 client.30266 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.404+0000 7f8258fdd740 10 client.30266 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:24:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.404+0000 7f8258fdd740 20 client.30266 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.405+0000 7f8258fdd740 1 --2- 172.21.15.2:0/3245456202 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557eca65f090 0x557eca661480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.405+0000 7f8258fdd740 4 client.30266 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:38.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.405+0000 7f8258fdd740 1 -- 172.21.15.2:0/3245456202 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eca5b7950 con 0x557eca65f090 2024-09-10T08:24:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.405+0000 7f824f400640 1 --2- 172.21.15.2:0/3245456202 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557eca65f090 0x557eca661480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.405+0000 7f824f400640 1 --2- 172.21.15.2:0/3245456202 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557eca65f090 0x557eca661480 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e5a9ecb05751ea6 server_cookie=9b63812055d95fd4 in_seq=0 out_seq=0 2024-09-10T08:24:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.405+0000 7f823d600640 10 client.30266 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:24:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f823d600640 1 -- 172.21.15.2:0/3245456202 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557eca5b7950 con 0x557eca65f090 2024-09-10T08:24:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f823d600640 10 client.30266 handle_command_reply: tid=0 2024-09-10T08:24:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f8258fdd740 1 client.30266 shutdown 2024-09-10T08:24:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f8258fdd740 1 -- 172.21.15.2:0/3245456202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca64f020 msgr2=0x557eca64c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f8258fdd740 1 --2- 172.21.15.2:0/3245456202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca64f020 0x557eca64c950 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f824800e720 tx=0x7f824800eca0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f8258fdd740 1 -- 172.21.15.2:0/3245456202 shutdown_connections 2024-09-10T08:24:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f8258fdd740 1 --2- 172.21.15.2:0/3245456202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca64f020 0x557eca64c950 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f8258fdd740 1 --2- 172.21.15.2:0/3245456202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca644760 0x557eca64c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f8258fdd740 1 --2- 172.21.15.2:0/3245456202 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557eca65f090 0x557eca661480 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f8258fdd740 1 --2- 172.21.15.2:0/3245456202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca643e40 0x557eca64bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f8258fdd740 1 -- 172.21.15.2:0/3245456202 >> 172.21.15.2:0/3245456202 conn(0x557eca6403d0 msgr2=0x557eca6460b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f823d600640 0 client.30266 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:24:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f8258fdd740 1 -- 172.21.15.2:0/3245456202 shutdown_connections 2024-09-10T08:24:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f8258fdd740 1 -- 172.21.15.2:0/3245456202 wait complete. 2024-09-10T08:24:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.407+0000 7f8258fdd740 20 client.30266 trim_cache size 0 max 16384 2024-09-10T08:24:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.421+0000 7f8258fdd740 1 Processor -- start 2024-09-10T08:24:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.421+0000 7f8258fdd740 1 -- start start 2024-09-10T08:24:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f8258fdd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca643e40 0x557eca6468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f8258fdd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca644760 0x557eca6644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f8258fdd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca64f020 0x557eca6649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f8258fdd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f82501b37a0 con 0x557eca64f020 2024-09-10T08:24:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f8258fdd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82501b1070 con 0x557eca644760 2024-09-10T08:24:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f8258fdd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82501aeec0 con 0x557eca643e40 2024-09-10T08:24:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f824fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca64f020 0x557eca6649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f824fe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca64f020 0x557eca6649e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39418/0 (socket says 172.21.15.2:39418) 2024-09-10T08:24:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f824fe00640 1 -- 172.21.15.2:0/1965512065 learned_addr learned my addr 172.21.15.2:0/1965512065 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f824f400640 1 --2- 172.21.15.2:0/1965512065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca643e40 0x557eca6468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f8247e00640 1 --2- 172.21.15.2:0/1965512065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca644760 0x557eca6644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f824fe00640 1 -- 172.21.15.2:0/1965512065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca643e40 msgr2=0x557eca6468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f824fe00640 1 --2- 172.21.15.2:0/1965512065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca643e40 0x557eca6468a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f824fe00640 1 -- 172.21.15.2:0/1965512065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca644760 msgr2=0x557eca6644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f824fe00640 1 --2- 172.21.15.2:0/1965512065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca644760 0x557eca6644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.422+0000 7f824fe00640 1 -- 172.21.15.2:0/1965512065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eca640200 con 0x557eca64f020 2024-09-10T08:24:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.423+0000 7f824fe00640 1 --2- 172.21.15.2:0/1965512065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca64f020 0x557eca6649e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f82480119b0 tx=0x7f82480092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c533bd417ad09434 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.423+0000 7f823fe00640 1 -- 172.21.15.2:0/1965512065 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248011bb0 con 0x557eca64f020 2024-09-10T08:24:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.423+0000 7f823fe00640 1 -- 172.21.15.2:0/1965512065 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8248015040 con 0x557eca64f020 2024-09-10T08:24:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.423+0000 7f823fe00640 1 -- 172.21.15.2:0/1965512065 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824801ad50 con 0x557eca64f020 2024-09-10T08:24:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.424+0000 7f8258fdd740 1 -- 172.21.15.2:0/1965512065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca64f020 msgr2=0x557eca6649e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.424+0000 7f8258fdd740 1 --2- 172.21.15.2:0/1965512065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca64f020 0x557eca6649e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f82480119b0 tx=0x7f82480092a0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.424+0000 7f8258fdd740 1 -- 172.21.15.2:0/1965512065 shutdown_connections 2024-09-10T08:24:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.424+0000 7f8258fdd740 1 --2- 172.21.15.2:0/1965512065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca64f020 0x557eca6649e0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.424+0000 7f8258fdd740 1 --2- 172.21.15.2:0/1965512065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca644760 0x557eca6644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.424+0000 7f8258fdd740 1 --2- 172.21.15.2:0/1965512065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca643e40 0x557eca6468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.424+0000 7f8258fdd740 1 -- 172.21.15.2:0/1965512065 >> 172.21.15.2:0/1965512065 conn(0x557eca6403d0 msgr2=0x557eca64df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:38.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.424+0000 7f8258fdd740 1 -- 172.21.15.2:0/1965512065 shutdown_connections 2024-09-10T08:24:38.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.424+0000 7f8258fdd740 1 -- 172.21.15.2:0/1965512065 wait complete. 2024-09-10T08:24:38.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.424+0000 7f8258fdd740 1 -- start start 2024-09-10T08:24:38.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.424+0000 7f8258fdd740 1 Processor -- start 2024-09-10T08:24:38.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.425+0000 7f8258fdd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca643e40 0x557eca6649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.425+0000 7f8258fdd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca644760 0x557eca661d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.425+0000 7f8258fdd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca64f020 0x557eca662260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.425+0000 7f8258fdd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f82501af530 con 0x557eca644760 2024-09-10T08:24:38.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.425+0000 7f8258fdd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82501b09a0 con 0x557eca643e40 2024-09-10T08:24:38.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.425+0000 7f8258fdd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82501afc00 con 0x557eca64f020 2024-09-10T08:24:38.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.425+0000 7f8258fdd740 10 client.0 _ll_register_callbacks cb 0x7f8248010710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.425+0000 7f8247e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca644760 0x557eca661d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.425+0000 7f8247e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca644760 0x557eca661d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39434/0 (socket says 172.21.15.2:39434) 2024-09-10T08:24:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.425+0000 7f8247e00640 1 -- 172.21.15.2:0/3527597531 learned_addr learned my addr 172.21.15.2:0/3527597531 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.425+0000 7f824f400640 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca643e40 0x557eca6649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.425+0000 7f824fe00640 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca64f020 0x557eca662260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f8247e00640 1 -- 172.21.15.2:0/3527597531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca64f020 msgr2=0x557eca662260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f8247e00640 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca64f020 0x557eca662260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f8247e00640 1 -- 172.21.15.2:0/3527597531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca643e40 msgr2=0x557eca6649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f8247e00640 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca643e40 0x557eca6649d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f8247e00640 1 -- 172.21.15.2:0/3527597531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eca662820 con 0x557eca644760 2024-09-10T08:24:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f824f400640 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca643e40 0x557eca6649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f8247e00640 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca644760 0x557eca661d20 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f82380b88b0 tx=0x7f8238003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97440af28b47a84d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f823d600640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-10T08:24:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f823d600640 1 -- 172.21.15.2:0/3527597531 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238049040 con 0x557eca644760 2024-09-10T08:24:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f823d600640 1 -- 172.21.15.2:0/3527597531 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f82380c4610 con 0x557eca644760 2024-09-10T08:24:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f8258fdd740 1 -- 172.21.15.2:0/3527597531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8248015040 con 0x557eca644760 2024-09-10T08:24:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f823d600640 1 -- 172.21.15.2:0/3527597531 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f82380bb840 con 0x557eca644760 2024-09-10T08:24:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f8258fdd740 10 client.20256 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.426+0000 7f8258fdd740 1 -- 172.21.15.2:0/3527597531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557eca6c9800 con 0x557eca644760 2024-09-10T08:24:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.427+0000 7f823d600640 1 -- 172.21.15.2:0/3527597531 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8248015040 con 0x557eca644760 2024-09-10T08:24:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.427+0000 7f823d600640 1 -- 172.21.15.2:0/3527597531 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f82380bb330 con 0x557eca644760 2024-09-10T08:24:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.427+0000 7f8258fdd740 10 client.20256 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.427+0000 7f8258fdd740 10 client.20256 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T08:24:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.427+0000 7f8258fdd740 20 client.20256 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.427+0000 7f8258fdd740 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557eca6c9f20 0x557eca6cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.427+0000 7f8258fdd740 4 client.20256 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:24:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.427+0000 7f8258fdd740 1 -- 172.21.15.2:0/3527597531 --> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557eca5b1e60 con 0x557eca6c9f20 2024-09-10T08:24:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.427+0000 7f824f400640 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557eca6c9f20 0x557eca6cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.428+0000 7f824f400640 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557eca6c9f20 0x557eca6cc310 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57fca70e3886168b server_cookie=98424272eb2a75f7 in_seq=0 out_seq=0 2024-09-10T08:24:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.428+0000 7f823d600640 10 client.20256 ms_handle_connect on v2:172.21.15.107:6836/3130379099 2024-09-10T08:24:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.428+0000 7f823d600640 1 -- 172.21.15.2:0/3527597531 <== mds.2 v2:172.21.15.107:6836/3130379099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557eca5b1e60 con 0x557eca6c9f20 2024-09-10T08:24:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.428+0000 7f823d600640 10 client.20256 handle_command_reply: tid=0 2024-09-10T08:24:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.428+0000 7f8258fdd740 1 client.20256 shutdown 2024-09-10T08:24:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.428+0000 7f8258fdd740 1 -- 172.21.15.2:0/3527597531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca644760 msgr2=0x557eca661d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.428+0000 7f8258fdd740 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca644760 0x557eca661d20 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f82380b88b0 tx=0x7f8238003f90 comp rx=0 tx=0).stop 2024-09-10T08:24:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f8258fdd740 1 -- 172.21.15.2:0/3527597531 shutdown_connections 2024-09-10T08:24:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f8258fdd740 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557eca64f020 0x557eca662260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f8258fdd740 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.107:6836/3130379099,v1:172.21.15.107:6837/3130379099] conn(0x557eca6c9f20 0x557eca6cc310 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f8258fdd740 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557eca644760 0x557eca661d20 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f8258fdd740 1 --2- 172.21.15.2:0/3527597531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eca643e40 0x557eca6649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f8258fdd740 1 -- 172.21.15.2:0/3527597531 >> 172.21.15.2:0/3527597531 conn(0x557eca6403d0 msgr2=0x557eca659ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f823d600640 0 client.20256 ms_handle_reset on v2:172.21.15.107:6836/3130379099 2024-09-10T08:24:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f8258fdd740 1 -- 172.21.15.2:0/3527597531 shutdown_connections 2024-09-10T08:24:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f8258fdd740 1 -- 172.21.15.2:0/3527597531 wait complete. 2024-09-10T08:24:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f8258fdd740 20 client.20256 trim_cache size 0 max 16384 2024-09-10T08:24:38.432 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:24:38.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f8255000640 1 -- 172.21.15.2:0/697381462 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82340a2b80 msgr2=0x7f82340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:38.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f8255000640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82340a2b80 0x7f82340a5040 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x557eca641230 tx=0x7f8240005f90 comp rx=0 tx=0).stop 2024-09-10T08:24:38.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f8255000640 1 -- 172.21.15.2:0/697381462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8250102360 msgr2=0x7f825019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:38.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.429+0000 7f8255000640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8250102360 0x7f825019a9f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f823802a940 tx=0x7f8238003b90 comp rx=0 tx=0).stop 2024-09-10T08:24:38.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.430+0000 7f8255000640 1 -- 172.21.15.2:0/697381462 shutdown_connections 2024-09-10T08:24:38.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.430+0000 7f8255000640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501079c0 0x7f825019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.430+0000 7f8255000640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8250102360 0x7f825019a9f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.430+0000 7f8255000640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f82340a2b80 0x7f82340a5040 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.430+0000 7f8255000640 1 --2- 172.21.15.2:0/697381462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82501019c0 0x7f825019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:38.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.430+0000 7f8255000640 1 -- 172.21.15.2:0/697381462 >> 172.21.15.2:0/697381462 conn(0x7f82500fd6e0 msgr2=0x7f825010d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:38.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.430+0000 7f8255000640 1 -- 172.21.15.2:0/697381462 shutdown_connections 2024-09-10T08:24:38.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:38.430+0000 7f8255000640 1 -- 172.21.15.2:0/697381462 wait complete. 2024-09-10T08:24:39.274 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:24:39.656 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:39.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:39 smithi107 ceph-mon[35048]: pgmap v882: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-09-10T08:24:39.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:39 smithi002 ceph-mon[30563]: pgmap v882: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-09-10T08:24:40.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:39 smithi155 ceph-mon[35405]: pgmap v882: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-09-10T08:24:40.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.226+0000 7fced1a00640 1 -- 172.21.15.2:0/2059371586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcecc109090 msgr2=0x7fcecc110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:40.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.226+0000 7fced1a00640 1 --2- 172.21.15.2:0/2059371586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcecc109090 0x7fcecc110c10 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009ca0 tx=0x7fcec402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.226+0000 7fced1a00640 1 -- 172.21.15.2:0/2059371586 shutdown_connections 2024-09-10T08:24:40.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.226+0000 7fced1a00640 1 --2- 172.21.15.2:0/2059371586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcecc109090 0x7fcecc110c10 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.226+0000 7fced1a00640 1 --2- 172.21.15.2:0/2059371586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcecc104800 0x7fcecc108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.226+0000 7fced1a00640 1 --2- 172.21.15.2:0/2059371586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcecc103e50 0x7fcecc104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.226+0000 7fced1a00640 1 -- 172.21.15.2:0/2059371586 >> 172.21.15.2:0/2059371586 conn(0x7fcecc0ff8b0 msgr2=0x7fcecc101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:40.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.226+0000 7fced1a00640 1 -- 172.21.15.2:0/2059371586 shutdown_connections 2024-09-10T08:24:40.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.226+0000 7fced1a00640 1 -- 172.21.15.2:0/2059371586 wait complete. 2024-09-10T08:24:40.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.227+0000 7fced1a00640 1 Processor -- start 2024-09-10T08:24:40.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.227+0000 7fced1a00640 1 -- start start 2024-09-10T08:24:40.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.227+0000 7fced1a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcecc103e50 0x7fcecc19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.227+0000 7fced1a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcecc104800 0x7fcecc19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fced1a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcecc109090 0x7fcecc19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fced1a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1140c0 con 0x7fcecc103e50 2024-09-10T08:24:40.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fced1a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc113f40 con 0x7fcecc104800 2024-09-10T08:24:40.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fced1a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc114240 con 0x7fcecc109090 2024-09-10T08:24:40.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fcecbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcecc103e50 0x7fcecc19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fcecbe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcecc103e50 0x7fcecc19a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39458/0 (socket says 172.21.15.2:39458) 2024-09-10T08:24:40.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fcecbe00640 1 -- 172.21.15.2:0/435452476 learned_addr learned my addr 172.21.15.2:0/435452476 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:40.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fcec3e00640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcecc104800 0x7fcecc19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fced1000640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcecc109090 0x7fcecc19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fcecbe00640 1 -- 172.21.15.2:0/435452476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcecc109090 msgr2=0x7fcecc19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:40.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fcecbe00640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcecc109090 0x7fcecc19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fcecbe00640 1 -- 172.21.15.2:0/435452476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcecc104800 msgr2=0x7fcecc19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:40.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fcecbe00640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcecc104800 0x7fcecc19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.228+0000 7fcecbe00640 1 -- 172.21.15.2:0/435452476 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcecc19f530 con 0x7fcecc103e50 2024-09-10T08:24:40.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.229+0000 7fcecbe00640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcecc103e50 0x7fcecc19a580 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00e9f0 tx=0x7fcebc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:40.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.229+0000 7fcec9600640 1 -- 172.21.15.2:0/435452476 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00cd80 con 0x7fcecc103e50 2024-09-10T08:24:40.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.229+0000 7fcec9600640 1 -- 172.21.15.2:0/435452476 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcebc005500 con 0x7fcecc103e50 2024-09-10T08:24:40.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.229+0000 7fcec9600640 1 -- 172.21.15.2:0/435452476 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc010690 con 0x7fcecc103e50 2024-09-10T08:24:40.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.229+0000 7fced1a00640 1 -- 172.21.15.2:0/435452476 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcecc19f820 con 0x7fcecc103e50 2024-09-10T08:24:40.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.229+0000 7fced1a00640 1 -- 172.21.15.2:0/435452476 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcecc1a7100 con 0x7fcecc103e50 2024-09-10T08:24:40.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.232+0000 7fced5c2d740 1 Processor -- start 2024-09-10T08:24:40.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.232+0000 7fcec9600640 1 -- 172.21.15.2:0/435452476 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcebc0046c0 con 0x7fcecc103e50 2024-09-10T08:24:40.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.232+0000 7fced5c2d740 1 -- start start 2024-09-10T08:24:40.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.233+0000 7fced5c2d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 0x560b91ee6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.233+0000 7fced5c2d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee6760 0x560b91ef0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.233+0000 7fced5c2d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ef1020 0x560b91ef3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.233+0000 7fced5c2d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560b91e4c3e0 con 0x560b91ee5e40 2024-09-10T08:24:40.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.233+0000 7fced5c2d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560b91df3d80 con 0x560b91ef1020 2024-09-10T08:24:40.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.233+0000 7fced5c2d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560b91e483a0 con 0x560b91ee6760 2024-09-10T08:24:40.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.233+0000 7fced1000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 0x560b91ee6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.233+0000 7fced1000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 0x560b91ee6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39462/0 (socket says 172.21.15.2:39462) 2024-09-10T08:24:40.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.233+0000 7fced1000640 1 -- 172.21.15.2:0/5357706 learned_addr learned my addr 172.21.15.2:0/5357706 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:40.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.233+0000 7fcecbe00640 1 --2- 172.21.15.2:0/5357706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee6760 0x560b91ef0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.233+0000 7fcec3e00640 1 --2- 172.21.15.2:0/5357706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ef1020 0x560b91ef3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.234+0000 7fced1000640 1 -- 172.21.15.2:0/5357706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee6760 msgr2=0x560b91ef0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:40.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.234+0000 7fced1000640 1 --2- 172.21.15.2:0/5357706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee6760 0x560b91ef0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.234+0000 7fced1000640 1 -- 172.21.15.2:0/5357706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ef1020 msgr2=0x560b91ef3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:40.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.234+0000 7fced1000640 1 --2- 172.21.15.2:0/5357706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ef1020 0x560b91ef3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.234+0000 7fced1000640 1 -- 172.21.15.2:0/5357706 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b91e12ae0 con 0x560b91ee5e40 2024-09-10T08:24:40.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.234+0000 7fcec9600640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fceb00a2b80 0x7fceb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.234+0000 7fcec9600640 1 -- 172.21.15.2:0/435452476 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcebc0cf810 con 0x7fcecc103e50 2024-09-10T08:24:40.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.234+0000 7fced1000640 1 --2- 172.21.15.2:0/5357706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 0x560b91ee6220 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009ca0 tx=0x7fcec4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0e2d469d47399d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:40.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.234+0000 7fcecbe00640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fceb00a2b80 0x7fceb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.234+0000 7fcec1600640 1 -- 172.21.15.2:0/5357706 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4031db0 con 0x560b91ee5e40 2024-09-10T08:24:40.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.234+0000 7fcec1600640 1 -- 172.21.15.2:0/5357706 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec4036040 con 0x560b91ee5e40 2024-09-10T08:24:40.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.234+0000 7fcec1600640 1 -- 172.21.15.2:0/5357706 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4004330 con 0x560b91ee5e40 2024-09-10T08:24:40.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.235+0000 7fced5c2d740 1 -- 172.21.15.2:0/5357706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 msgr2=0x560b91ee6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:40.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.235+0000 7fcecbe00640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fceb00a2b80 0x7fceb00a5040 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcebc09b980 tx=0x7fcebc0d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:40.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.235+0000 7fced5c2d740 1 --2- 172.21.15.2:0/5357706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 0x560b91ee6220 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009ca0 tx=0x7fcec4005f10 comp rx=0 tx=0).stop 2024-09-10T08:24:40.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.235+0000 7fced5c2d740 1 -- 172.21.15.2:0/5357706 shutdown_connections 2024-09-10T08:24:40.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.235+0000 7fced5c2d740 1 --2- 172.21.15.2:0/5357706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ef1020 0x560b91ef3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.235+0000 7fced5c2d740 1 --2- 172.21.15.2:0/5357706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee6760 0x560b91ef0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.235+0000 7fced5c2d740 1 --2- 172.21.15.2:0/5357706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 0x560b91ee6220 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.235+0000 7fced5c2d740 1 -- 172.21.15.2:0/5357706 >> 172.21.15.2:0/5357706 conn(0x560b91ee23d0 msgr2=0x560b91ee27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:40.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.235+0000 7fced5c2d740 1 -- 172.21.15.2:0/5357706 shutdown_connections 2024-09-10T08:24:40.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.235+0000 7fced5c2d740 1 -- 172.21.15.2:0/5357706 wait complete. 2024-09-10T08:24:40.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.236+0000 7fced5c2d740 1 -- start start 2024-09-10T08:24:40.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.236+0000 7fced5c2d740 1 Processor -- start 2024-09-10T08:24:40.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced5c2d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee5e40 0x560b91eedf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced5c2d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee6760 0x560b91eee450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced5c2d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ef1020 0x560b91eee990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced5c2d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560b91ef3a70 con 0x560b91ee6760 2024-09-10T08:24:40.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced5c2d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560b91e070f0 con 0x560b91ef1020 2024-09-10T08:24:40.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced5c2d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560b91ef3bf0 con 0x560b91ee5e40 2024-09-10T08:24:40.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced5c2d740 10 client.0 _ll_register_callbacks cb 0x560b91deebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:40.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee5e40 0x560b91eedf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee5e40 0x560b91eedf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50952/0 (socket says 172.21.15.2:50952) 2024-09-10T08:24:40.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced1000640 1 -- 172.21.15.2:0/2937560851 learned_addr learned my addr 172.21.15.2:0/2937560851 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:40.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced1000640 1 --2- 172.21.15.2:0/2937560851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ef1020 0x560b91eee990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fcec3e00640 1 --2- 172.21.15.2:0/2937560851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee6760 0x560b91eee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced1000640 1 -- 172.21.15.2:0/2937560851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ef1020 msgr2=0x560b91eee990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:40.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced1000640 1 --2- 172.21.15.2:0/2937560851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ef1020 0x560b91eee990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced1000640 1 -- 172.21.15.2:0/2937560851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee6760 msgr2=0x560b91eee450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:40.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced1000640 1 --2- 172.21.15.2:0/2937560851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee6760 0x560b91eee450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.237+0000 7fced1000640 1 -- 172.21.15.2:0/2937560851 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b91bbd3c0 con 0x560b91ee5e40 2024-09-10T08:24:40.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.238+0000 7fced1000640 1 --2- 172.21.15.2:0/2937560851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee5e40 0x560b91eedf10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009620 tx=0x7fcec4031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f66e69abbc2aa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:40.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.238+0000 7fceba000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:24:40.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.238+0000 7fceba000640 1 -- 172.21.15.2:0/2937560851 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4058060 con 0x560b91ee5e40 2024-09-10T08:24:40.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.238+0000 7fced5c2d740 1 -- 172.21.15.2:0/2937560851 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec4036040 con 0x560b91ee5e40 2024-09-10T08:24:40.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.238+0000 7fceba000640 1 -- 172.21.15.2:0/2937560851 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcec4003b80 con 0x560b91ee5e40 2024-09-10T08:24:40.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.238+0000 7fceba000640 1 -- 172.21.15.2:0/2937560851 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcec4003d80 con 0x560b91ee5e40 2024-09-10T08:24:40.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.238+0000 7fced5c2d740 10 client.30214 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:40.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.238+0000 7fced5c2d740 1 -- 172.21.15.2:0/2937560851 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560b91ef3a70 con 0x560b91ee5e40 2024-09-10T08:24:40.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.239+0000 7fceba000640 1 -- 172.21.15.2:0/2937560851 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec4036040 con 0x560b91ee5e40 2024-09-10T08:24:40.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.239+0000 7fceba000640 1 -- 172.21.15.2:0/2937560851 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcec404d030 con 0x560b91ee5e40 2024-09-10T08:24:40.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.239+0000 7fced5c2d740 10 client.30214 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:40.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.239+0000 7fced5c2d740 10 client.30214 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:24:40.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.239+0000 7fced5c2d740 20 client.30214 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:40.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.239+0000 7fced5c2d740 1 --2- 172.21.15.2:0/2937560851 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560b91f00c30 0x560b91f03020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.239+0000 7fced5c2d740 4 client.30214 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:40.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.239+0000 7fced5c2d740 1 -- 172.21.15.2:0/2937560851 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b91e59950 con 0x560b91f00c30 2024-09-10T08:24:40.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.239+0000 7fcec3e00640 1 --2- 172.21.15.2:0/2937560851 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560b91f00c30 0x560b91f03020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.240+0000 7fcec3e00640 1 --2- 172.21.15.2:0/2937560851 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560b91f00c30 0x560b91f03020 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a943b6ee9431a741 server_cookie=38c2fab42ea79333 in_seq=0 out_seq=0 2024-09-10T08:24:40.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.240+0000 7fceba000640 10 client.30214 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:24:40.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.241+0000 7fceba000640 1 -- 172.21.15.2:0/2937560851 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b91e59950 con 0x560b91f00c30 2024-09-10T08:24:40.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.241+0000 7fceba000640 10 client.30214 handle_command_reply: tid=0 2024-09-10T08:24:40.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.241+0000 7fced5c2d740 1 client.30214 shutdown 2024-09-10T08:24:40.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.241+0000 7fced5c2d740 1 -- 172.21.15.2:0/2937560851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee5e40 msgr2=0x560b91eedf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:40.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.241+0000 7fced5c2d740 1 --2- 172.21.15.2:0/2937560851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee5e40 0x560b91eedf10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009620 tx=0x7fcec4031a20 comp rx=0 tx=0).stop 2024-09-10T08:24:40.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.241+0000 7fced5c2d740 1 -- 172.21.15.2:0/2937560851 shutdown_connections 2024-09-10T08:24:40.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.241+0000 7fced5c2d740 1 --2- 172.21.15.2:0/2937560851 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560b91f00c30 0x560b91f03020 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.241+0000 7fced5c2d740 1 --2- 172.21.15.2:0/2937560851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ef1020 0x560b91eee990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.241+0000 7fced5c2d740 1 --2- 172.21.15.2:0/2937560851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee6760 0x560b91eee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.241+0000 7fced5c2d740 1 --2- 172.21.15.2:0/2937560851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee5e40 0x560b91eedf10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.241+0000 7fceba000640 0 client.30214 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:24:40.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.241+0000 7fced5c2d740 1 -- 172.21.15.2:0/2937560851 >> 172.21.15.2:0/2937560851 conn(0x560b91ee23d0 msgr2=0x560b91ee80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:40.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.241+0000 7fced5c2d740 1 -- 172.21.15.2:0/2937560851 shutdown_connections 2024-09-10T08:24:40.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.242+0000 7fced5c2d740 1 -- 172.21.15.2:0/2937560851 wait complete. 2024-09-10T08:24:40.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.242+0000 7fced5c2d740 20 client.30214 trim_cache size 0 max 16384 2024-09-10T08:24:40.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.255+0000 7fced5c2d740 1 Processor -- start 2024-09-10T08:24:40.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.255+0000 7fced5c2d740 1 -- start start 2024-09-10T08:24:40.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.255+0000 7fced5c2d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 0x560b91ee88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.255+0000 7fced5c2d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ee6760 0x560b91f088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fced5c2d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ef1020 0x560b91f08de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fced5c2d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1b36c0 con 0x560b91ee5e40 2024-09-10T08:24:40.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fced5c2d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1b0f90 con 0x560b91ee6760 2024-09-10T08:24:40.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fced5c2d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1aede0 con 0x560b91ef1020 2024-09-10T08:24:40.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fced1000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 0x560b91ee88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fced1000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 0x560b91ee88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39486/0 (socket says 172.21.15.2:39486) 2024-09-10T08:24:40.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fced1000640 1 -- 172.21.15.2:0/2095326538 learned_addr learned my addr 172.21.15.2:0/2095326538 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:40.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fcec3e00640 1 --2- 172.21.15.2:0/2095326538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ee6760 0x560b91f088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fced1000640 1 --2- 172.21.15.2:0/2095326538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ef1020 0x560b91f08de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fced1000640 1 -- 172.21.15.2:0/2095326538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ef1020 msgr2=0x560b91f08de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:40.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fced1000640 1 --2- 172.21.15.2:0/2095326538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ef1020 0x560b91f08de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fced1000640 1 -- 172.21.15.2:0/2095326538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ee6760 msgr2=0x560b91f088a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:40.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fced1000640 1 --2- 172.21.15.2:0/2095326538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ee6760 0x560b91f088a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.256+0000 7fced1000640 1 -- 172.21.15.2:0/2095326538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b91ee3060 con 0x560b91ee5e40 2024-09-10T08:24:40.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.257+0000 7fced1000640 1 --2- 172.21.15.2:0/2095326538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 0x560b91ee88a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fcec4004af0 tx=0x7fcec4004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4967c4f35206a81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:40.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.257+0000 7fcec0c00640 1 -- 172.21.15.2:0/2095326538 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4058030 con 0x560b91ee5e40 2024-09-10T08:24:40.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.257+0000 7fcec0c00640 1 -- 172.21.15.2:0/2095326538 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec403f020 con 0x560b91ee5e40 2024-09-10T08:24:40.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.257+0000 7fcec0c00640 1 -- 172.21.15.2:0/2095326538 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4004b40 con 0x560b91ee5e40 2024-09-10T08:24:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.257+0000 7fced5c2d740 1 -- 172.21.15.2:0/2095326538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 msgr2=0x560b91ee88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.257+0000 7fced5c2d740 1 --2- 172.21.15.2:0/2095326538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 0x560b91ee88a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fcec4004af0 tx=0x7fcec4004a80 comp rx=0 tx=0).stop 2024-09-10T08:24:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.257+0000 7fced5c2d740 1 -- 172.21.15.2:0/2095326538 shutdown_connections 2024-09-10T08:24:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.257+0000 7fced5c2d740 1 --2- 172.21.15.2:0/2095326538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ef1020 0x560b91f08de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.257+0000 7fced5c2d740 1 --2- 172.21.15.2:0/2095326538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ee6760 0x560b91f088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.257+0000 7fced5c2d740 1 --2- 172.21.15.2:0/2095326538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ee5e40 0x560b91ee88a0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.258+0000 7fced5c2d740 1 -- 172.21.15.2:0/2095326538 >> 172.21.15.2:0/2095326538 conn(0x560b91ee23d0 msgr2=0x560b91eeff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.258+0000 7fced5c2d740 1 -- 172.21.15.2:0/2095326538 shutdown_connections 2024-09-10T08:24:40.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.258+0000 7fced5c2d740 1 -- 172.21.15.2:0/2095326538 wait complete. 2024-09-10T08:24:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.258+0000 7fced5c2d740 1 -- start start 2024-09-10T08:24:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.258+0000 7fced5c2d740 1 Processor -- start 2024-09-10T08:24:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.259+0000 7fced5c2d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee5e40 0x560b91ef9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.259+0000 7fced5c2d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ee6760 0x560b91eeab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.259+0000 7fced5c2d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ef1020 0x560b91eeb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.259+0000 7fced5c2d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1af450 con 0x560b91ef1020 2024-09-10T08:24:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.259+0000 7fced5c2d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1b08c0 con 0x560b91ee6760 2024-09-10T08:24:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.259+0000 7fced5c2d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1afb20 con 0x560b91ee5e40 2024-09-10T08:24:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.259+0000 7fced5c2d740 10 client.0 _ll_register_callbacks cb 0x7fceb40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.259+0000 7fcec3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ee6760 0x560b91eeab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.259+0000 7fcec3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ee6760 0x560b91eeab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41460/0 (socket says 172.21.15.2:41460) 2024-09-10T08:24:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.259+0000 7fcec3e00640 1 -- 172.21.15.2:0/677878133 learned_addr learned my addr 172.21.15.2:0/677878133 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:40.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.259+0000 7fced1000640 1 --2- 172.21.15.2:0/677878133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ef1020 0x560b91eeb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.259+0000 7fced1000640 1 --2- 172.21.15.2:0/677878133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee5e40 0x560b91ef9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.260+0000 7fcec3e00640 1 -- 172.21.15.2:0/677878133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee5e40 msgr2=0x560b91ef9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.260+0000 7fcec3e00640 1 --2- 172.21.15.2:0/677878133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee5e40 0x560b91ef9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.260+0000 7fcec3e00640 1 -- 172.21.15.2:0/677878133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ef1020 msgr2=0x560b91eeb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.260+0000 7fcec3e00640 1 --2- 172.21.15.2:0/677878133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ef1020 0x560b91eeb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.260+0000 7fcec3e00640 1 -- 172.21.15.2:0/677878133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b91eeb750 con 0x560b91ee6760 2024-09-10T08:24:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.260+0000 7fcec3e00640 1 --2- 172.21.15.2:0/677878133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ee6760 0x560b91eeab20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fceb4005880 tx=0x7fceb40044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d79faeb50ab673d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:40.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.260+0000 7fceba000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.260+0000 7fceba000640 1 -- 172.21.15.2:0/677878133 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4010430 con 0x560b91ee6760 2024-09-10T08:24:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.260+0000 7fced5c2d740 1 -- 172.21.15.2:0/677878133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec403f020 con 0x560b91ee6760 2024-09-10T08:24:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.260+0000 7fceba000640 1 -- 172.21.15.2:0/677878133 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fceb40105d0 con 0x560b91ee6760 2024-09-10T08:24:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.261+0000 7fceba000640 1 -- 172.21.15.2:0/677878133 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fceb402f050 con 0x560b91ee6760 2024-09-10T08:24:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.261+0000 7fced5c2d740 10 client.30290 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:40.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.261+0000 7fced5c2d740 1 -- 172.21.15.2:0/677878133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560b91f6b790 con 0x560b91ee6760 2024-09-10T08:24:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.261+0000 7fceba000640 1 -- 172.21.15.2:0/677878133 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec403f020 con 0x560b91ee6760 2024-09-10T08:24:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.261+0000 7fceba000640 1 -- 172.21.15.2:0/677878133 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fceb4026070 con 0x560b91ee6760 2024-09-10T08:24:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.261+0000 7fced5c2d740 10 client.30290 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.261+0000 7fced5c2d740 10 client.30290 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T08:24:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.261+0000 7fced5c2d740 20 client.30290 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.261+0000 7fced5c2d740 1 --2- 172.21.15.2:0/677878133 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560b91f6bef0 0x560b91f6e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.261+0000 7fced5c2d740 4 client.30290 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:24:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.261+0000 7fced5c2d740 1 -- 172.21.15.2:0/677878133 --> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b91e53e60 con 0x560b91f6bef0 2024-09-10T08:24:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.262+0000 7fced1000640 1 --2- 172.21.15.2:0/677878133 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560b91f6bef0 0x560b91f6e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.262+0000 7fced1000640 1 --2- 172.21.15.2:0/677878133 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560b91f6bef0 0x560b91f6e310 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63c21d2fab0d6ccd server_cookie=505aa5481cc8f7d5 in_seq=0 out_seq=0 2024-09-10T08:24:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.262+0000 7fceba000640 10 client.30290 ms_handle_connect on v2:172.21.15.155:6838/2871792117 2024-09-10T08:24:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.262+0000 7fceba000640 1 -- 172.21.15.2:0/677878133 <== mds.1 v2:172.21.15.155:6838/2871792117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b91e53e60 con 0x560b91f6bef0 2024-09-10T08:24:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.262+0000 7fceba000640 10 client.30290 handle_command_reply: tid=0 2024-09-10T08:24:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.262+0000 7fced5c2d740 1 client.30290 shutdown 2024-09-10T08:24:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.262+0000 7fced5c2d740 1 -- 172.21.15.2:0/677878133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ee6760 msgr2=0x560b91eeab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.263+0000 7fced5c2d740 1 --2- 172.21.15.2:0/677878133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ee6760 0x560b91eeab20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fceb4005880 tx=0x7fceb40044d0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.263+0000 7fced5c2d740 1 -- 172.21.15.2:0/677878133 shutdown_connections 2024-09-10T08:24:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.263+0000 7fced5c2d740 1 --2- 172.21.15.2:0/677878133 >> [v2:172.21.15.155:6838/2871792117,v1:172.21.15.155:6839/2871792117] conn(0x560b91f6bef0 0x560b91f6e310 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.263+0000 7fced5c2d740 1 --2- 172.21.15.2:0/677878133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b91ef1020 0x560b91eeb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.263+0000 7fced5c2d740 1 --2- 172.21.15.2:0/677878133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560b91ee6760 0x560b91eeab20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.263+0000 7fced5c2d740 1 --2- 172.21.15.2:0/677878133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b91ee5e40 0x560b91ef9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.263+0000 7fceba000640 0 client.30290 ms_handle_reset on v2:172.21.15.155:6838/2871792117 2024-09-10T08:24:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.263+0000 7fced5c2d740 1 -- 172.21.15.2:0/677878133 >> 172.21.15.2:0/677878133 conn(0x560b91ee23d0 msgr2=0x560b91ee0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.263+0000 7fced5c2d740 1 -- 172.21.15.2:0/677878133 shutdown_connections 2024-09-10T08:24:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.263+0000 7fced5c2d740 1 -- 172.21.15.2:0/677878133 wait complete. 2024-09-10T08:24:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.263+0000 7fced5c2d740 20 client.30290 trim_cache size 0 max 16384 2024-09-10T08:24:40.266 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:24:40.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.264+0000 7fced1a00640 1 -- 172.21.15.2:0/435452476 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fceb00a2b80 msgr2=0x7fceb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:40.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.264+0000 7fced1a00640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fceb00a2b80 0x7fceb00a5040 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcebc09b980 tx=0x7fcebc0d5550 comp rx=0 tx=0).stop 2024-09-10T08:24:40.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.264+0000 7fced1a00640 1 -- 172.21.15.2:0/435452476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcecc103e50 msgr2=0x7fcecc19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:40.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.264+0000 7fced1a00640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcecc103e50 0x7fcecc19a580 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00e9f0 tx=0x7fcebc00edf0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.264+0000 7fced1a00640 1 -- 172.21.15.2:0/435452476 shutdown_connections 2024-09-10T08:24:40.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.264+0000 7fced1a00640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fceb00a2b80 0x7fceb00a5040 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.264+0000 7fced1a00640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcecc109090 0x7fcecc19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.264+0000 7fced1a00640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcecc104800 0x7fcecc19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.264+0000 7fced1a00640 1 --2- 172.21.15.2:0/435452476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcecc103e50 0x7fcecc19a580 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:40.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.264+0000 7fced1a00640 1 -- 172.21.15.2:0/435452476 >> 172.21.15.2:0/435452476 conn(0x7fcecc0ff8b0 msgr2=0x7fcecc100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:40.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.264+0000 7fced1a00640 1 -- 172.21.15.2:0/435452476 shutdown_connections 2024-09-10T08:24:40.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:40.264+0000 7fced1a00640 1 -- 172.21.15.2:0/435452476 wait complete. 2024-09-10T08:24:41.048 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:24:41.430 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:41.845 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:41 smithi002 ceph-mon[30563]: pgmap v883: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-09-10T08:24:41.916 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:41 smithi107 ceph-mon[35048]: pgmap v883: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-09-10T08:24:42.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.116+0000 7f9d43600640 1 -- 172.21.15.2:0/1424610239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d3c077040 msgr2=0x7f9d3c1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:42.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.116+0000 7f9d43600640 1 --2- 172.21.15.2:0/1424610239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d3c077040 0x7f9d3c1124f0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009900 tx=0x7f9d2402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.116+0000 7f9d43600640 1 -- 172.21.15.2:0/1424610239 shutdown_connections 2024-09-10T08:24:42.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.116+0000 7f9d43600640 1 --2- 172.21.15.2:0/1424610239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d3c112a30 0x7f9d3c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.116+0000 7f9d43600640 1 --2- 172.21.15.2:0/1424610239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d3c077040 0x7f9d3c1124f0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.116+0000 7f9d43600640 1 --2- 172.21.15.2:0/1424610239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d3c076720 0x7f9d3c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.116+0000 7f9d43600640 1 -- 172.21.15.2:0/1424610239 >> 172.21.15.2:0/1424610239 conn(0x7f9d3c0ffa90 msgr2=0x7f9d3c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:42.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.116+0000 7f9d43600640 1 -- 172.21.15.2:0/1424610239 shutdown_connections 2024-09-10T08:24:42.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.117+0000 7f9d43600640 1 -- 172.21.15.2:0/1424610239 wait complete. 2024-09-10T08:24:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.117+0000 7f9d43600640 1 Processor -- start 2024-09-10T08:24:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.118+0000 7f9d43600640 1 -- start start 2024-09-10T08:24:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.118+0000 7f9d43600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d3c076720 0x7f9d3c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.118+0000 7f9d43600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d3c077040 0x7f9d3c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.118+0000 7f9d43600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d3c112a30 0x7f9d3c1a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.118+0000 7f9d43600640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c1185c0 con 0x7f9d3c112a30 2024-09-10T08:24:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.118+0000 7f9d43600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c118440 con 0x7f9d3c076720 2024-09-10T08:24:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.118+0000 7f9d43600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c118740 con 0x7f9d3c077040 2024-09-10T08:24:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.118+0000 7f9d41600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d3c112a30 0x7f9d3c1a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.118+0000 7f9d40c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d3c076720 0x7f9d3c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.118+0000 7f9d33e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d3c077040 0x7f9d3c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.118+0000 7f9d41600640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d3c112a30 0x7f9d3c1a3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39508/0 (socket says 172.21.15.2:39508) 2024-09-10T08:24:42.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.118+0000 7f9d40c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d3c076720 0x7f9d3c19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41466/0 (socket says 172.21.15.2:41466) 2024-09-10T08:24:42.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.118+0000 7f9d41600640 1 -- 172.21.15.2:0/3906421157 learned_addr learned my addr 172.21.15.2:0/3906421157 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:42.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.119+0000 7f9d40c00640 1 -- 172.21.15.2:0/3906421157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d3c077040 msgr2=0x7f9d3c19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:42.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.119+0000 7f9d40c00640 1 --2- 172.21.15.2:0/3906421157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d3c077040 0x7f9d3c19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.119+0000 7f9d40c00640 1 -- 172.21.15.2:0/3906421157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d3c112a30 msgr2=0x7f9d3c1a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:42.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.119+0000 7f9d40c00640 1 --2- 172.21.15.2:0/3906421157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d3c112a30 0x7f9d3c1a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.119+0000 7f9d40c00640 1 -- 172.21.15.2:0/3906421157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3c1a3bb0 con 0x7f9d3c076720 2024-09-10T08:24:42.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.119+0000 7f9d40c00640 1 --2- 172.21.15.2:0/3906421157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d3c076720 0x7f9d3c19eb80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00e9f0 tx=0x7f9d2c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:42.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.119+0000 7f9d3a000640 1 -- 172.21.15.2:0/3906421157 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00ccc0 con 0x7f9d3c076720 2024-09-10T08:24:42.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.120+0000 7f9d43600640 1 -- 172.21.15.2:0/3906421157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3c1a3ea0 con 0x7f9d3c076720 2024-09-10T08:24:42.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.120+0000 7f9d3a000640 1 -- 172.21.15.2:0/3906421157 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d2c005020 con 0x7f9d3c076720 2024-09-10T08:24:42.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.120+0000 7f9d3a000640 1 -- 172.21.15.2:0/3906421157 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0049e0 con 0x7f9d3c076720 2024-09-10T08:24:42.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.120+0000 7f9d43600640 1 -- 172.21.15.2:0/3906421157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d3c1ab730 con 0x7f9d3c076720 2024-09-10T08:24:42.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.123+0000 7f9d461e2740 1 Processor -- start 2024-09-10T08:24:42.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.123+0000 7f9d3a000640 1 -- 172.21.15.2:0/3906421157 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9d2c005560 con 0x7f9d3c076720 2024-09-10T08:24:42.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.123+0000 7f9d461e2740 1 -- start start 2024-09-10T08:24:42.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d461e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d461e2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4b4590 0x55bb1b4be910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d461e2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4bee50 0x55bb1b4c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d461e2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bb1b41a3e0 con 0x55bb1b4bee50 2024-09-10T08:24:42.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d461e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bb1b3c1d80 con 0x55bb1b4b3c70 2024-09-10T08:24:42.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d461e2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bb1b4163a0 con 0x55bb1b4b4590 2024-09-10T08:24:42.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d33e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4bee50 0x55bb1b4c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d41600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d33e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4bee50 0x55bb1b4c1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39514/0 (socket says 172.21.15.2:39514) 2024-09-10T08:24:42.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d41600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41474/0 (socket says 172.21.15.2:41474) 2024-09-10T08:24:42.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d33e00640 1 -- 172.21.15.2:0/2697413293 learned_addr learned my addr 172.21.15.2:0/2697413293 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:42.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d40c00640 1 --2- 172.21.15.2:0/2697413293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4b4590 0x55bb1b4be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d33e00640 1 -- 172.21.15.2:0/2697413293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4b4590 msgr2=0x55bb1b4be910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:42.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d33e00640 1 --2- 172.21.15.2:0/2697413293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4b4590 0x55bb1b4be910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d33e00640 1 -- 172.21.15.2:0/2697413293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 msgr2=0x55bb1b4b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:42.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d33e00640 1 --2- 172.21.15.2:0/2697413293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.124+0000 7f9d33e00640 1 -- 172.21.15.2:0/2697413293 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb1b3e0ae0 con 0x55bb1b4bee50 2024-09-10T08:24:42.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.125+0000 7f9d33e00640 1 --2- 172.21.15.2:0/2697413293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4bee50 0x55bb1b4c1240 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009900 tx=0x7f9d24005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466bf49e99632026 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:42.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.125+0000 7f9d3a000640 1 --2- 172.21.15.2:0/3906421157 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9d200a2c00 0x7f9d200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.125+0000 7f9d3a000640 1 -- 172.21.15.2:0/3906421157 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9d2c0cf360 con 0x7f9d3c076720 2024-09-10T08:24:42.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.125+0000 7f9d31c00640 1 -- 172.21.15.2:0/2697413293 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24040070 con 0x55bb1b4bee50 2024-09-10T08:24:42.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.125+0000 7f9d41600640 1 --2- 172.21.15.2:0/3906421157 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9d200a2c00 0x7f9d200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.125+0000 7f9d31c00640 1 -- 172.21.15.2:0/2697413293 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d24031870 con 0x55bb1b4bee50 2024-09-10T08:24:42.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.125+0000 7f9d31c00640 1 -- 172.21.15.2:0/2697413293 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24035040 con 0x55bb1b4bee50 2024-09-10T08:24:42.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.126+0000 7f9d461e2740 1 -- 172.21.15.2:0/2697413293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4bee50 msgr2=0x55bb1b4c1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:42.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.126+0000 7f9d41600640 1 --2- 172.21.15.2:0/3906421157 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9d200a2c00 0x7f9d200a50c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9d34005f40 tx=0x7f9d34009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:42.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.126+0000 7f9d461e2740 1 --2- 172.21.15.2:0/2697413293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4bee50 0x55bb1b4c1240 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009900 tx=0x7f9d24005be0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.126+0000 7f9d461e2740 1 -- 172.21.15.2:0/2697413293 shutdown_connections 2024-09-10T08:24:42.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.126+0000 7f9d461e2740 1 --2- 172.21.15.2:0/2697413293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4bee50 0x55bb1b4c1240 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.126+0000 7f9d461e2740 1 --2- 172.21.15.2:0/2697413293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4b4590 0x55bb1b4be910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.126+0000 7f9d461e2740 1 --2- 172.21.15.2:0/2697413293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.126+0000 7f9d461e2740 1 -- 172.21.15.2:0/2697413293 >> 172.21.15.2:0/2697413293 conn(0x55bb1b4b0200 msgr2=0x55bb1b4b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.126+0000 7f9d461e2740 1 -- 172.21.15.2:0/2697413293 shutdown_connections 2024-09-10T08:24:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.126+0000 7f9d461e2740 1 -- 172.21.15.2:0/2697413293 wait complete. 2024-09-10T08:24:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.127+0000 7f9d461e2740 1 -- start start 2024-09-10T08:24:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.127+0000 7f9d461e2740 1 Processor -- start 2024-09-10T08:24:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d461e2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4bbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d461e2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4bc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d461e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4bee50 0x55bb1b4bc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d461e2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bb1b4c1a20 con 0x55bb1b4b4590 2024-09-10T08:24:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d461e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bb1b3d50f0 con 0x55bb1b4bee50 2024-09-10T08:24:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d461e2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bb1b4c1ba0 con 0x55bb1b4b3c70 2024-09-10T08:24:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d461e2740 10 client.0 _ll_register_callbacks cb 0x55bb1b3bcbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:42.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d33e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4bc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d40c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4bbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d40c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4bbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:51012/0 (socket says 172.21.15.2:51012) 2024-09-10T08:24:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d40c00640 1 -- 172.21.15.2:0/1443475085 learned_addr learned my addr 172.21.15.2:0/1443475085 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d41600640 1 --2- 172.21.15.2:0/1443475085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4bee50 0x55bb1b4bc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d40c00640 1 -- 172.21.15.2:0/1443475085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4bee50 msgr2=0x55bb1b4bc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d40c00640 1 --2- 172.21.15.2:0/1443475085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4bee50 0x55bb1b4bc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d40c00640 1 -- 172.21.15.2:0/1443475085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 msgr2=0x55bb1b4bc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d40c00640 1 --2- 172.21.15.2:0/1443475085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4bc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d40c00640 1 -- 172.21.15.2:0/1443475085 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb1b18b3c0 con 0x55bb1b4b3c70 2024-09-10T08:24:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.128+0000 7f9d33e00640 1 --2- 172.21.15.2:0/1443475085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4bc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.129+0000 7f9d40c00640 1 --2- 172.21.15.2:0/1443475085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4bbc70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0a3eb0 tx=0x7f9d2c097ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b47e4554c9fd4fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.129+0000 7f9d2a000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-09-10T08:24:42.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.129+0000 7f9d2a000640 1 -- 172.21.15.2:0/1443475085 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c01c070 con 0x55bb1b4b3c70 2024-09-10T08:24:42.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.129+0000 7f9d461e2740 1 -- 172.21.15.2:0/1443475085 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d24031870 con 0x55bb1b4b3c70 2024-09-10T08:24:42.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.129+0000 7f9d2a000640 1 -- 172.21.15.2:0/1443475085 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9d2c086070 con 0x55bb1b4b3c70 2024-09-10T08:24:42.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.129+0000 7f9d2a000640 1 -- 172.21.15.2:0/1443475085 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d2c0051d0 con 0x55bb1b4b3c70 2024-09-10T08:24:42.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.129+0000 7f9d461e2740 10 client.30232 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:42.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.129+0000 7f9d461e2740 1 -- 172.21.15.2:0/1443475085 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb1b4cd9f0 con 0x55bb1b4b3c70 2024-09-10T08:24:42.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.129+0000 7f9d2a000640 1 -- 172.21.15.2:0/1443475085 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d24031870 con 0x55bb1b4b3c70 2024-09-10T08:24:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.130+0000 7f9d2a000640 1 -- 172.21.15.2:0/1443475085 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9d2c097de0 con 0x55bb1b4b3c70 2024-09-10T08:24:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.130+0000 7f9d461e2740 10 client.30232 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.130+0000 7f9d461e2740 10 client.30232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.130+0000 7f9d461e2740 20 client.30232 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.130+0000 7f9d461e2740 1 --2- 172.21.15.2:0/1443475085 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bb1b4cee00 0x55bb1b4d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.130+0000 7f9d461e2740 4 client.30232 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:24:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.130+0000 7f9d461e2740 1 -- 172.21.15.2:0/1443475085 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb1b427950 con 0x55bb1b4cee00 2024-09-10T08:24:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.130+0000 7f9d33e00640 1 --2- 172.21.15.2:0/1443475085 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bb1b4cee00 0x55bb1b4d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.131+0000 7f9d33e00640 1 --2- 172.21.15.2:0/1443475085 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bb1b4cee00 0x55bb1b4d1210 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd62d54051b6413 server_cookie=b69697ecd748f9fd in_seq=0 out_seq=0 2024-09-10T08:24:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.131+0000 7f9d2a000640 10 client.30232 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d2a000640 1 -- 172.21.15.2:0/1443475085 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb1b427950 con 0x55bb1b4cee00 2024-09-10T08:24:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d2a000640 10 client.30232 handle_command_reply: tid=0 2024-09-10T08:24:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d461e2740 1 client.30232 shutdown 2024-09-10T08:24:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d461e2740 1 -- 172.21.15.2:0/1443475085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4b3c70 msgr2=0x55bb1b4bbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d461e2740 1 --2- 172.21.15.2:0/1443475085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4bbc70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0a3eb0 tx=0x7f9d2c097ad0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d461e2740 1 -- 172.21.15.2:0/1443475085 shutdown_connections 2024-09-10T08:24:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d461e2740 1 --2- 172.21.15.2:0/1443475085 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bb1b4cee00 0x55bb1b4d1210 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d461e2740 1 --2- 172.21.15.2:0/1443475085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4bee50 0x55bb1b4bc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d461e2740 1 --2- 172.21.15.2:0/1443475085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4bc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d461e2740 1 --2- 172.21.15.2:0/1443475085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4bbc70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d2a000640 0 client.30232 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d461e2740 1 -- 172.21.15.2:0/1443475085 >> 172.21.15.2:0/1443475085 conn(0x55bb1b4b0200 msgr2=0x55bb1b4b5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d461e2740 1 -- 172.21.15.2:0/1443475085 shutdown_connections 2024-09-10T08:24:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d461e2740 1 -- 172.21.15.2:0/1443475085 wait complete. 2024-09-10T08:24:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.132+0000 7f9d461e2740 20 client.30232 trim_cache size 0 max 16384 2024-09-10T08:24:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.146+0000 7f9d461e2740 1 Processor -- start 2024-09-10T08:24:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.146+0000 7f9d461e2740 1 -- start start 2024-09-10T08:24:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.146+0000 7f9d461e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.146+0000 7f9d461e2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4d6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.146+0000 7f9d461e2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4bee50 0x55bb1b4bd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.146+0000 7f9d461e2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c1b7d30 con 0x55bb1b4b4590 2024-09-10T08:24:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.146+0000 7f9d461e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c1b5600 con 0x55bb1b4b3c70 2024-09-10T08:24:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.146+0000 7f9d461e2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c1b3450 con 0x55bb1b4bee50 2024-09-10T08:24:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.147+0000 7f9d33e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4d6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.147+0000 7f9d33e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4d6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39532/0 (socket says 172.21.15.2:39532) 2024-09-10T08:24:42.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.147+0000 7f9d33e00640 1 -- 172.21.15.2:0/2990780551 learned_addr learned my addr 172.21.15.2:0/2990780551 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.147+0000 7f9d40c00640 1 --2- 172.21.15.2:0/2990780551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.147+0000 7f9d41600640 1 --2- 172.21.15.2:0/2990780551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4bee50 0x55bb1b4bd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.147+0000 7f9d33e00640 1 -- 172.21.15.2:0/2990780551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4bee50 msgr2=0x55bb1b4bd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:42.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.147+0000 7f9d33e00640 1 --2- 172.21.15.2:0/2990780551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4bee50 0x55bb1b4bd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.147+0000 7f9d33e00640 1 -- 172.21.15.2:0/2990780551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 msgr2=0x55bb1b4d6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:42.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.147+0000 7f9d33e00640 1 --2- 172.21.15.2:0/2990780551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4d6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.147+0000 7f9d33e00640 1 -- 172.21.15.2:0/2990780551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb1b4b0030 con 0x55bb1b4b4590 2024-09-10T08:24:42.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.147+0000 7f9d33e00640 1 --2- 172.21.15.2:0/2990780551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4d6fc0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009900 tx=0x7f9d240054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327fdb328620eb32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:42.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.148+0000 7f9d31200640 1 -- 172.21.15.2:0/2990780551 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24040040 con 0x55bb1b4b4590 2024-09-10T08:24:42.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.148+0000 7f9d31200640 1 -- 172.21.15.2:0/2990780551 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d24009520 con 0x55bb1b4b4590 2024-09-10T08:24:42.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.148+0000 7f9d31200640 1 -- 172.21.15.2:0/2990780551 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24035040 con 0x55bb1b4b4590 2024-09-10T08:24:42.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.148+0000 7f9d461e2740 1 -- 172.21.15.2:0/2990780551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 msgr2=0x55bb1b4d6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:42.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.148+0000 7f9d461e2740 1 --2- 172.21.15.2:0/2990780551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4d6fc0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009900 tx=0x7f9d240054a0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.149+0000 7f9d461e2740 1 -- 172.21.15.2:0/2990780551 shutdown_connections 2024-09-10T08:24:42.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.149+0000 7f9d461e2740 1 --2- 172.21.15.2:0/2990780551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4bee50 0x55bb1b4bd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.149+0000 7f9d461e2740 1 --2- 172.21.15.2:0/2990780551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4d6fc0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.149+0000 7f9d461e2740 1 --2- 172.21.15.2:0/2990780551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4d6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.149+0000 7f9d461e2740 1 -- 172.21.15.2:0/2990780551 >> 172.21.15.2:0/2990780551 conn(0x55bb1b4b0200 msgr2=0x55bb1b4bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:42.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.149+0000 7f9d461e2740 1 -- 172.21.15.2:0/2990780551 shutdown_connections 2024-09-10T08:24:42.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.149+0000 7f9d461e2740 1 -- 172.21.15.2:0/2990780551 wait complete. 2024-09-10T08:24:42.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.149+0000 7f9d461e2740 1 -- start start 2024-09-10T08:24:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.149+0000 7f9d461e2740 1 Processor -- start 2024-09-10T08:24:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d461e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d461e2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4bfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d461e2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4bee50 0x55bb1b4c00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d461e2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c1b3ac0 con 0x55bb1b4b4590 2024-09-10T08:24:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d461e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c1b4f30 con 0x55bb1b4b3c70 2024-09-10T08:24:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d461e2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c1b4190 con 0x55bb1b4bee50 2024-09-10T08:24:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d461e2740 10 client.0 _ll_register_callbacks cb 0x7f9d2c0d2a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:24:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d33e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4bfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d40c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d33e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4bfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39536/0 (socket says 172.21.15.2:39536) 2024-09-10T08:24:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d33e00640 1 -- 172.21.15.2:0/285496385 learned_addr learned my addr 172.21.15.2:0/285496385 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d41600640 1 --2- 172.21.15.2:0/285496385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4bee50 0x55bb1b4c00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d40c00640 1 -- 172.21.15.2:0/285496385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4bee50 msgr2=0x55bb1b4c00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d40c00640 1 --2- 172.21.15.2:0/285496385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4bee50 0x55bb1b4c00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d40c00640 1 -- 172.21.15.2:0/285496385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 msgr2=0x55bb1b4bfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:24:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d40c00640 1 --2- 172.21.15.2:0/285496385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4bfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d40c00640 1 -- 172.21.15.2:0/285496385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb1b4c0750 con 0x55bb1b4b3c70 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.150+0000 7f9d33e00640 1 --2- 172.21.15.2:0/285496385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4bfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.151+0000 7f9d40c00640 1 --2- 172.21.15.2:0/285496385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4c73e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0b1040 tx=0x7f9d2c0862e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f73efdf8b42fa4ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.151+0000 7f9d2a000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.151+0000 7f9d2a000640 1 -- 172.21.15.2:0/285496385 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c01c040 con 0x55bb1b4b3c70 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.151+0000 7f9d461e2740 1 -- 172.21.15.2:0/285496385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d24009520 con 0x55bb1b4b3c70 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.151+0000 7f9d2a000640 1 -- 172.21.15.2:0/285496385 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f9d2c0cf020 con 0x55bb1b4b3c70 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.151+0000 7f9d2a000640 1 -- 172.21.15.2:0/285496385 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d2c09be70 con 0x55bb1b4b3c70 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.151+0000 7f9d461e2740 10 client.30308 fetch_fsmap learned FSMap version 26 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.151+0000 7f9d461e2740 1 -- 172.21.15.2:0/285496385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb1b539a50 con 0x55bb1b4b3c70 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.151+0000 7f9d2a000640 1 -- 172.21.15.2:0/285496385 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d24009520 con 0x55bb1b4b3c70 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.152+0000 7f9d2a000640 1 -- 172.21.15.2:0/285496385 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9d2c09abc0 con 0x55bb1b4b3c70 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.152+0000 7f9d461e2740 10 client.30308 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:24:42.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.152+0000 7f9d461e2740 10 client.30308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:24:42.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.152+0000 7f9d461e2740 20 client.30308 populate_metadata read hostname 'smithi002' 2024-09-10T08:24:42.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.152+0000 7f9d461e2740 1 --2- 172.21.15.2:0/285496385 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bb1b539de0 0x55bb1b53c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:42.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.152+0000 7f9d461e2740 4 client.30308 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:24:42.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.152+0000 7f9d461e2740 1 -- 172.21.15.2:0/285496385 --> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb1b421e60 con 0x55bb1b539de0 2024-09-10T08:24:42.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.152+0000 7f9d33e00640 1 --2- 172.21.15.2:0/285496385 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bb1b539de0 0x55bb1b53c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d33e00640 1 --2- 172.21.15.2:0/285496385 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bb1b539de0 0x55bb1b53c1d0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dea108e6c221507 server_cookie=fafaab145204c501 in_seq=0 out_seq=0 2024-09-10T08:24:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d2a000640 10 client.30308 ms_handle_connect on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d2a000640 1 -- 172.21.15.2:0/285496385 <== mds.0 v2:172.21.15.107:6838/3674188134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb1b421e60 con 0x55bb1b539de0 2024-09-10T08:24:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d2a000640 10 client.30308 handle_command_reply: tid=0 2024-09-10T08:24:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d461e2740 1 client.30308 shutdown 2024-09-10T08:24:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d461e2740 1 -- 172.21.15.2:0/285496385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 msgr2=0x55bb1b4c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d461e2740 1 --2- 172.21.15.2:0/285496385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4c73e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0b1040 tx=0x7f9d2c0862e0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d461e2740 1 -- 172.21.15.2:0/285496385 shutdown_connections 2024-09-10T08:24:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d461e2740 1 --2- 172.21.15.2:0/285496385 >> [v2:172.21.15.107:6838/3674188134,v1:172.21.15.107:6839/3674188134] conn(0x55bb1b539de0 0x55bb1b53c1d0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d461e2740 1 --2- 172.21.15.2:0/285496385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb1b4bee50 0x55bb1b4c00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d461e2740 1 --2- 172.21.15.2:0/285496385 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb1b4b4590 0x55bb1b4bfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d461e2740 1 --2- 172.21.15.2:0/285496385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb1b4b3c70 0x55bb1b4c73e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d2a000640 0 client.30308 ms_handle_reset on v2:172.21.15.107:6838/3674188134 2024-09-10T08:24:42.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.153+0000 7f9d461e2740 1 -- 172.21.15.2:0/285496385 >> 172.21.15.2:0/285496385 conn(0x55bb1b4b0200 msgr2=0x55bb1b4aeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:42.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.154+0000 7f9d461e2740 1 -- 172.21.15.2:0/285496385 shutdown_connections 2024-09-10T08:24:42.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.154+0000 7f9d461e2740 1 -- 172.21.15.2:0/285496385 wait complete. 2024-09-10T08:24:42.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.154+0000 7f9d461e2740 20 client.30308 trim_cache size 0 max 16384 2024-09-10T08:24:42.158 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-09-10T08:24:42.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.154+0000 7f9d43600640 1 -- 172.21.15.2:0/3906421157 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9d200a2c00 msgr2=0x7f9d200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:42.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.154+0000 7f9d43600640 1 --2- 172.21.15.2:0/3906421157 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9d200a2c00 0x7f9d200a50c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9d34005f40 tx=0x7f9d34009500 comp rx=0 tx=0).stop 2024-09-10T08:24:42.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.154+0000 7f9d43600640 1 -- 172.21.15.2:0/3906421157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d3c076720 msgr2=0x7f9d3c19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:42.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.154+0000 7f9d43600640 1 --2- 172.21.15.2:0/3906421157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d3c076720 0x7f9d3c19eb80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00e9f0 tx=0x7f9d2c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.154+0000 7f9d43600640 1 -- 172.21.15.2:0/3906421157 shutdown_connections 2024-09-10T08:24:42.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.154+0000 7f9d43600640 1 --2- 172.21.15.2:0/3906421157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d3c112a30 0x7f9d3c1a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.154+0000 7f9d43600640 1 --2- 172.21.15.2:0/3906421157 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f9d200a2c00 0x7f9d200a50c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.154+0000 7f9d43600640 1 --2- 172.21.15.2:0/3906421157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d3c077040 0x7f9d3c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.154+0000 7f9d43600640 1 --2- 172.21.15.2:0/3906421157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d3c076720 0x7f9d3c19eb80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:42.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.154+0000 7f9d43600640 1 -- 172.21.15.2:0/3906421157 >> 172.21.15.2:0/3906421157 conn(0x7f9d3c0ffa90 msgr2=0x7f9d3c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.155+0000 7f9d43600640 1 -- 172.21.15.2:0/3906421157 shutdown_connections 2024-09-10T08:24:42.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:42.155+0000 7f9d43600640 1 -- 172.21.15.2:0/3906421157 wait complete. 2024-09-10T08:24:42.160 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:41 smithi155 ceph-mon[35405]: pgmap v883: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-09-10T08:24:42.951 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-10T08:24:42.952 INFO:tasks.fwd_scrub:done joining 2024-09-10T08:24:42.952 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T08:24:42.963 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T08:24:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.123+0000 7f2621c00640 1 -- 172.21.15.2:0/1628760087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f261c1058e0 msgr2=0x7f261c10c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.123+0000 7f2621c00640 1 --2- 172.21.15.2:0/1628760087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f261c1058e0 0x7f261c10c170 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f2614009ca0 tx=0x7f261402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.124+0000 7f2621c00640 1 -- 172.21.15.2:0/1628760087 shutdown_connections 2024-09-10T08:24:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.124+0000 7f2621c00640 1 --2- 172.21.15.2:0/1628760087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f261c1058e0 0x7f261c10c170 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.124+0000 7f2621c00640 1 --2- 172.21.15.2:0/1628760087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c104f20 0x7f261c1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.124+0000 7f2621c00640 1 --2- 172.21.15.2:0/1628760087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f261c103d30 0x7f261c104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.124+0000 7f2621c00640 1 -- 172.21.15.2:0/1628760087 >> 172.21.15.2:0/1628760087 conn(0x7f261c0ff4b0 msgr2=0x7f261c1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:43.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.124+0000 7f2621c00640 1 -- 172.21.15.2:0/1628760087 shutdown_connections 2024-09-10T08:24:43.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.124+0000 7f2621c00640 1 -- 172.21.15.2:0/1628760087 wait complete. 2024-09-10T08:24:43.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.125+0000 7f2621c00640 1 Processor -- start 2024-09-10T08:24:43.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.125+0000 7f2621c00640 1 -- start start 2024-09-10T08:24:43.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.125+0000 7f2621c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c103d30 0x7f261c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:43.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.125+0000 7f2621c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f261c104f20 0x7f261c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:43.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.125+0000 7f2621c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f261c1058e0 0x7f261c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:43.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.125+0000 7f2621c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10f9b0 con 0x7f261c104f20 2024-09-10T08:24:43.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.125+0000 7f2621c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10f830 con 0x7f261c103d30 2024-09-10T08:24:43.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.125+0000 7f2621c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10fb30 con 0x7f261c1058e0 2024-09-10T08:24:43.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.125+0000 7f2613e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f261c104f20 0x7f261c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:43.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.125+0000 7f261aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c103d30 0x7f261c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:43.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.125+0000 7f261aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c103d30 0x7f261c19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41538/0 (socket says 172.21.15.2:41538) 2024-09-10T08:24:43.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.126+0000 7f261aa00640 1 -- 172.21.15.2:0/105719297 learned_addr learned my addr 172.21.15.2:0/105719297 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:43.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.126+0000 7f261b400640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f261c1058e0 0x7f261c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:43.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.126+0000 7f261aa00640 1 -- 172.21.15.2:0/105719297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f261c1058e0 msgr2=0x7f261c1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:43.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.126+0000 7f261aa00640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f261c1058e0 0x7f261c1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.126+0000 7f261aa00640 1 -- 172.21.15.2:0/105719297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f261c104f20 msgr2=0x7f261c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:43.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.126+0000 7f261aa00640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f261c104f20 0x7f261c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.126+0000 7f261aa00640 1 -- 172.21.15.2:0/105719297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c1a1c30 con 0x7f261c103d30 2024-09-10T08:24:43.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.126+0000 7f2613e00640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f261c104f20 0x7f261c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:43.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.126+0000 7f261aa00640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c103d30 0x7f261c19a010 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f260c00e9c0 tx=0x7f260c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:43.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.126+0000 7f2613400640 1 -- 172.21.15.2:0/105719297 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c00cd80 con 0x7f261c103d30 2024-09-10T08:24:43.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.126+0000 7f2613400640 1 -- 172.21.15.2:0/105719297 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f260c0035f0 con 0x7f261c103d30 2024-09-10T08:24:43.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.126+0000 7f2621c00640 1 -- 172.21.15.2:0/105719297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261c1a1f20 con 0x7f261c103d30 2024-09-10T08:24:43.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.126+0000 7f2613400640 1 -- 172.21.15.2:0/105719297 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c010690 con 0x7f261c103d30 2024-09-10T08:24:43.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.127+0000 7f2621c00640 1 -- 172.21.15.2:0/105719297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261c1a2380 con 0x7f261c103d30 2024-09-10T08:24:43.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.128+0000 7f2621c00640 1 -- 172.21.15.2:0/105719297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f261c10b0d0 con 0x7f261c103d30 2024-09-10T08:24:43.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.130+0000 7f2613400640 1 -- 172.21.15.2:0/105719297 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f260c0037a0 con 0x7f261c103d30 2024-09-10T08:24:43.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.131+0000 7f2613400640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f26000a2b90 0x7f26000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:43.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.131+0000 7f2613400640 1 -- 172.21.15.2:0/105719297 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f260c0cf7d0 con 0x7f261c103d30 2024-09-10T08:24:43.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.131+0000 7f2613e00640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f26000a2b90 0x7f26000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:43.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.132+0000 7f2613e00640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f26000a2b90 0x7f26000a5050 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f261c19b520 tx=0x7f2604005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:43.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.133+0000 7f2613400640 1 -- 172.21.15.2:0/105719297 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f260c09c2e0 con 0x7f261c103d30 2024-09-10T08:24:43.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.319+0000 7f2621c00640 1 -- 172.21.15.2:0/105719297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f261c02da10 con 0x7f261c103d30 2024-09-10T08:24:43.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.321+0000 7f2613400640 1 -- 172.21.15.2:0/105719297 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24887 (secure 0 0 0) 0x7f260c09bbe0 con 0x7f261c103d30 2024-09-10T08:24:43.321 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:24:43.321 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":95,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:07:05.504273+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:24:43.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.323+0000 7f2621c00640 1 -- 172.21.15.2:0/105719297 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f26000a2b90 msgr2=0x7f26000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:43.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.323+0000 7f2621c00640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f26000a2b90 0x7f26000a5050 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f261c19b520 tx=0x7f2604005f90 comp rx=0 tx=0).stop 2024-09-10T08:24:43.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.323+0000 7f2621c00640 1 -- 172.21.15.2:0/105719297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c103d30 msgr2=0x7f261c19a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:43.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.323+0000 7f2621c00640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c103d30 0x7f261c19a010 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f260c00e9c0 tx=0x7f260c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:24:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.323+0000 7f2621c00640 1 -- 172.21.15.2:0/105719297 shutdown_connections 2024-09-10T08:24:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.323+0000 7f2621c00640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f26000a2b90 0x7f26000a5050 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.323+0000 7f2621c00640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f261c1058e0 0x7f261c1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.323+0000 7f2621c00640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f261c104f20 0x7f261c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.323+0000 7f2621c00640 1 --2- 172.21.15.2:0/105719297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c103d30 0x7f261c19a010 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.323+0000 7f2621c00640 1 -- 172.21.15.2:0/105719297 >> 172.21.15.2:0/105719297 conn(0x7f261c0ff4b0 msgr2=0x7f261c1082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.323+0000 7f2621c00640 1 -- 172.21.15.2:0/105719297 shutdown_connections 2024-09-10T08:24:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.323+0000 7f2621c00640 1 -- 172.21.15.2:0/105719297 wait complete. 2024-09-10T08:24:43.339 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T08:00:26.170464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T08:05:21.042136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T08:05:24.120081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T08:05:34.042103+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T08:24:43.340 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T08:24:43.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.501+0000 7f12c9800640 1 -- 172.21.15.2:0/780046432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12c4103d20 msgr2=0x7f12c4104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:43.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.501+0000 7f12c9800640 1 --2- 172.21.15.2:0/780046432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12c4103d20 0x7f12c4104120 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009ca0 tx=0x7f12b402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.501+0000 7f12c9800640 1 -- 172.21.15.2:0/780046432 shutdown_connections 2024-09-10T08:24:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.501+0000 7f12c9800640 1 --2- 172.21.15.2:0/780046432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12c41058d0 0x7f12c410c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.501+0000 7f12c9800640 1 --2- 172.21.15.2:0/780046432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4104f10 0x7f12c4105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.501+0000 7f12c9800640 1 --2- 172.21.15.2:0/780046432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12c4103d20 0x7f12c4104120 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.501+0000 7f12c9800640 1 -- 172.21.15.2:0/780046432 >> 172.21.15.2:0/780046432 conn(0x7f12c40ff4a0 msgr2=0x7f12c41018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.501+0000 7f12c9800640 1 -- 172.21.15.2:0/780046432 shutdown_connections 2024-09-10T08:24:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.501+0000 7f12c9800640 1 -- 172.21.15.2:0/780046432 wait complete. 2024-09-10T08:24:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.502+0000 7f12c9800640 1 Processor -- start 2024-09-10T08:24:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.502+0000 7f12c9800640 1 -- start start 2024-09-10T08:24:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.502+0000 7f12c9800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4103d20 0x7f12c419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.502+0000 7f12c9800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12c4104f10 0x7f12c419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.502+0000 7f12c9800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12c41058d0 0x7f12c41a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.502+0000 7f12c9800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f12c410f9f0 con 0x7f12c4104f10 2024-09-10T08:24:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.502+0000 7f12c9800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f12c410f870 con 0x7f12c41058d0 2024-09-10T08:24:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.502+0000 7f12c9800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f12c410fb70 con 0x7f12c4103d20 2024-09-10T08:24:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.502+0000 7f12ba000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12c4104f10 0x7f12c419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.502+0000 7f12ba000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12c4104f10 0x7f12c419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39574/0 (socket says 172.21.15.2:39574) 2024-09-10T08:24:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.502+0000 7f12ba000640 1 -- 172.21.15.2:0/1489509554 learned_addr learned my addr 172.21.15.2:0/1489509554 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.502+0000 7f12c2a00640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4103d20 0x7f12c419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.503+0000 7f12c3400640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12c41058d0 0x7f12c41a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.503+0000 7f12ba000640 1 -- 172.21.15.2:0/1489509554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4103d20 msgr2=0x7f12c419a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.503+0000 7f12ba000640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4103d20 0x7f12c419a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.503+0000 7f12ba000640 1 -- 172.21.15.2:0/1489509554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12c41058d0 msgr2=0x7f12c41a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.503+0000 7f12ba000640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12c41058d0 0x7f12c41a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.503+0000 7f12ba000640 1 -- 172.21.15.2:0/1489509554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c41a1d00 con 0x7f12c4104f10 2024-09-10T08:24:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.503+0000 7f12c2a00640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4103d20 0x7f12c419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.503+0000 7f12ba000640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12c4104f10 0x7f12c419a620 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00e9f0 tx=0x7f12ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.503+0000 7f12bbe00640 1 -- 172.21.15.2:0/1489509554 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac00ccc0 con 0x7f12c4104f10 2024-09-10T08:24:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.503+0000 7f12bbe00640 1 -- 172.21.15.2:0/1489509554 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12ac00ce60 con 0x7f12c4104f10 2024-09-10T08:24:43.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.503+0000 7f12bbe00640 1 -- 172.21.15.2:0/1489509554 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac010690 con 0x7f12c4104f10 2024-09-10T08:24:43.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.503+0000 7f12c9800640 1 -- 172.21.15.2:0/1489509554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c41a1ff0 con 0x7f12c4104f10 2024-09-10T08:24:43.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.504+0000 7f12c9800640 1 -- 172.21.15.2:0/1489509554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c41a2460 con 0x7f12c4104f10 2024-09-10T08:24:43.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.505+0000 7f12c9800640 1 -- 172.21.15.2:0/1489509554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c410b180 con 0x7f12c4104f10 2024-09-10T08:24:43.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.507+0000 7f12bbe00640 1 -- 172.21.15.2:0/1489509554 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f12ac0046c0 con 0x7f12c4104f10 2024-09-10T08:24:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.508+0000 7f12bbe00640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f12a80a2b90 0x7f12a80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.508+0000 7f12bbe00640 1 -- 172.21.15.2:0/1489509554 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f12ac0cf860 con 0x7f12c4104f10 2024-09-10T08:24:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.508+0000 7f12c2a00640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f12a80a2b90 0x7f12a80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:43.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.509+0000 7f12c2a00640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f12a80a2b90 0x7f12a80a5050 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f12b4000f80 tx=0x7f12b4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:43.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.512+0000 7f12bbe00640 1 -- 172.21.15.2:0/1489509554 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12ac09c370 con 0x7f12c4104f10 2024-09-10T08:24:43.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.699+0000 7f12c9800640 1 -- 172.21.15.2:0/1489509554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f12c4062e90 con 0x7f12c4104f10 2024-09-10T08:24:43.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.699+0000 7f12bbe00640 1 -- 172.21.15.2:0/1489509554 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f12ac09bc70 con 0x7f12c4104f10 2024-09-10T08:24:43.700 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-09-10T08:24:43.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.701+0000 7f12c9800640 1 -- 172.21.15.2:0/1489509554 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f12a80a2b90 msgr2=0x7f12a80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:43.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.701+0000 7f12c9800640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f12a80a2b90 0x7f12a80a5050 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f12b4000f80 tx=0x7f12b4004a40 comp rx=0 tx=0).stop 2024-09-10T08:24:43.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.701+0000 7f12c9800640 1 -- 172.21.15.2:0/1489509554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12c4104f10 msgr2=0x7f12c419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:43.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.701+0000 7f12c9800640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12c4104f10 0x7f12c419a620 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00e9f0 tx=0x7f12ac00edf0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.701+0000 7f12c9800640 1 -- 172.21.15.2:0/1489509554 shutdown_connections 2024-09-10T08:24:43.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.701+0000 7f12c9800640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12c41058d0 0x7f12c41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.701+0000 7f12c9800640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12c4104f10 0x7f12c419a620 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.701+0000 7f12c9800640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f12a80a2b90 0x7f12a80a5050 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.701+0000 7f12c9800640 1 --2- 172.21.15.2:0/1489509554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c4103d20 0x7f12c419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.701+0000 7f12c9800640 1 -- 172.21.15.2:0/1489509554 >> 172.21.15.2:0/1489509554 conn(0x7f12c40ff4a0 msgr2=0x7f12c4108350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:43.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.702+0000 7f12c9800640 1 -- 172.21.15.2:0/1489509554 shutdown_connections 2024-09-10T08:24:43.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.702+0000 7f12c9800640 1 -- 172.21.15.2:0/1489509554 wait complete. 2024-09-10T08:24:43.718 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T08:24:43.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.879+0000 7f696a800640 1 -- 172.21.15.2:0/2241850907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6964104f10 msgr2=0x7f6964105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:43.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.879+0000 7f696a800640 1 --2- 172.21.15.2:0/2241850907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6964104f10 0x7f6964105390 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f694c009900 tx=0x7f694c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.879+0000 7f696a800640 1 -- 172.21.15.2:0/2241850907 shutdown_connections 2024-09-10T08:24:43.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.879+0000 7f696a800640 1 --2- 172.21.15.2:0/2241850907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69641058d0 0x7f696410c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.879+0000 7f696a800640 1 --2- 172.21.15.2:0/2241850907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6964104f10 0x7f6964105390 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.879+0000 7f696a800640 1 --2- 172.21.15.2:0/2241850907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6964103d20 0x7f6964104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.879+0000 7f696a800640 1 -- 172.21.15.2:0/2241850907 >> 172.21.15.2:0/2241850907 conn(0x7f69640ff4a0 msgr2=0x7f69641018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:43.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.879+0000 7f696a800640 1 -- 172.21.15.2:0/2241850907 shutdown_connections 2024-09-10T08:24:43.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.879+0000 7f696a800640 1 -- 172.21.15.2:0/2241850907 wait complete. 2024-09-10T08:24:43.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.880+0000 7f696a800640 1 Processor -- start 2024-09-10T08:24:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.880+0000 7f696a800640 1 -- start start 2024-09-10T08:24:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f696a800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6964103d20 0x7f696419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f696a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6964104f10 0x7f696419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f696a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69641058d0 0x7f69641a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f696a800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f696410f9f0 con 0x7f6964103d20 2024-09-10T08:24:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f696a800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f696410f870 con 0x7f6964104f10 2024-09-10T08:24:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f696a800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f696410fb70 con 0x7f69641058d0 2024-09-10T08:24:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f6963400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6964103d20 0x7f696419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f6963400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6964103d20 0x7f696419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39604/0 (socket says 172.21.15.2:39604) 2024-09-10T08:24:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f6963400640 1 -- 172.21.15.2:0/3452563614 learned_addr learned my addr 172.21.15.2:0/3452563614 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f695be00640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6964104f10 0x7f696419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f6963e00640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69641058d0 0x7f69641a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f6963400640 1 -- 172.21.15.2:0/3452563614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69641058d0 msgr2=0x7f69641a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f6963400640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69641058d0 0x7f69641a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f6963400640 1 -- 172.21.15.2:0/3452563614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6964104f10 msgr2=0x7f696419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f6963400640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6964104f10 0x7f696419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f6963400640 1 -- 172.21.15.2:0/3452563614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69641a1c90 con 0x7f6964103d20 2024-09-10T08:24:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.881+0000 7f695be00640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6964104f10 0x7f696419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.882+0000 7f6963400640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6964103d20 0x7f696419a070 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f695400e9c0 tx=0x7f695400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.882+0000 7f6960c00640 1 -- 172.21.15.2:0/3452563614 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695400cd80 con 0x7f6964103d20 2024-09-10T08:24:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.882+0000 7f6960c00640 1 -- 172.21.15.2:0/3452563614 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69540035f0 con 0x7f6964103d20 2024-09-10T08:24:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.882+0000 7f6960c00640 1 -- 172.21.15.2:0/3452563614 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954010690 con 0x7f6964103d20 2024-09-10T08:24:43.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.882+0000 7f696a800640 1 -- 172.21.15.2:0/3452563614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69641a1f80 con 0x7f6964103d20 2024-09-10T08:24:43.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.882+0000 7f696a800640 1 -- 172.21.15.2:0/3452563614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69641a23f0 con 0x7f6964103d20 2024-09-10T08:24:43.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.883+0000 7f696a800640 1 -- 172.21.15.2:0/3452563614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f696410b0c0 con 0x7f6964103d20 2024-09-10T08:24:43.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.885+0000 7f6960c00640 1 -- 172.21.15.2:0/3452563614 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f69540037a0 con 0x7f6964103d20 2024-09-10T08:24:43.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.887+0000 7f6960c00640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f69480a2c10 0x7f69480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:43.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.887+0000 7f6960c00640 1 -- 172.21.15.2:0/3452563614 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f69540cf930 con 0x7f6964103d20 2024-09-10T08:24:43.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.888+0000 7f695be00640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f69480a2c10 0x7f69480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:43.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.888+0000 7f695be00640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f69480a2c10 0x7f69480a50d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f694c006010 tx=0x7f694c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:43.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:43.891+0000 7f6960c00640 1 -- 172.21.15.2:0/3452563614 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f695409c440 con 0x7f6964103d20 2024-09-10T08:24:44.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:43 smithi002 ceph-mon[30563]: pgmap v884: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2024-09-10T08:24:44.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:43 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/105719297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:24:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.075+0000 7f696a800640 1 -- 172.21.15.2:0/3452563614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6964062e90 con 0x7f6964103d20 2024-09-10T08:24:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.075+0000 7f6960c00640 1 -- 172.21.15.2:0/3452563614 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f695409bd40 con 0x7f6964103d20 2024-09-10T08:24:44.076 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-09-10T08:24:44.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.077+0000 7f696a800640 1 -- 172.21.15.2:0/3452563614 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f69480a2c10 msgr2=0x7f69480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:44.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.077+0000 7f696a800640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f69480a2c10 0x7f69480a50d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f694c006010 tx=0x7f694c000bc0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.077+0000 7f696a800640 1 -- 172.21.15.2:0/3452563614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6964103d20 msgr2=0x7f696419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:44.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.077+0000 7f696a800640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6964103d20 0x7f696419a070 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f695400e9c0 tx=0x7f695400ee80 comp rx=0 tx=0).stop 2024-09-10T08:24:44.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.078+0000 7f696a800640 1 -- 172.21.15.2:0/3452563614 shutdown_connections 2024-09-10T08:24:44.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.078+0000 7f696a800640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f69480a2c10 0x7f69480a50d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.078+0000 7f696a800640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69641058d0 0x7f69641a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.078+0000 7f696a800640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6964104f10 0x7f696419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.078+0000 7f696a800640 1 --2- 172.21.15.2:0/3452563614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6964103d20 0x7f696419a070 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.078+0000 7f696a800640 1 -- 172.21.15.2:0/3452563614 >> 172.21.15.2:0/3452563614 conn(0x7f69640ff4a0 msgr2=0x7f6964108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:44.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.078+0000 7f696a800640 1 -- 172.21.15.2:0/3452563614 shutdown_connections 2024-09-10T08:24:44.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.078+0000 7f696a800640 1 -- 172.21.15.2:0/3452563614 wait complete. 2024-09-10T08:24:44.096 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T08:24:44.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:43 smithi155 ceph-mon[35405]: pgmap v884: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2024-09-10T08:24:44.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:43 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/105719297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:24:44.168 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:43 smithi107 ceph-mon[35048]: pgmap v884: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2024-09-10T08:24:44.169 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:43 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/105719297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:24:44.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.258+0000 7f8773000640 1 -- 172.21.15.2:0/2658227532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f876c104f10 msgr2=0x7f876c105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:44.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.258+0000 7f8773000640 1 --2- 172.21.15.2:0/2658227532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f876c104f10 0x7f876c105390 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f8758009900 tx=0x7f875802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.258+0000 7f8773000640 1 -- 172.21.15.2:0/2658227532 shutdown_connections 2024-09-10T08:24:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.258+0000 7f8773000640 1 --2- 172.21.15.2:0/2658227532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f876c1058d0 0x7f876c10c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.258+0000 7f8773000640 1 --2- 172.21.15.2:0/2658227532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f876c104f10 0x7f876c105390 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.258+0000 7f8773000640 1 --2- 172.21.15.2:0/2658227532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f876c103d20 0x7f876c104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.258+0000 7f8773000640 1 -- 172.21.15.2:0/2658227532 >> 172.21.15.2:0/2658227532 conn(0x7f876c0ff4a0 msgr2=0x7f876c1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.258+0000 7f8773000640 1 -- 172.21.15.2:0/2658227532 shutdown_connections 2024-09-10T08:24:44.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.258+0000 7f8773000640 1 -- 172.21.15.2:0/2658227532 wait complete. 2024-09-10T08:24:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.259+0000 7f8773000640 1 Processor -- start 2024-09-10T08:24:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.259+0000 7f8773000640 1 -- start start 2024-09-10T08:24:44.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.259+0000 7f8773000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f876c103d20 0x7f876c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:44.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.259+0000 7f8773000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f876c104f10 0x7f876c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:44.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.259+0000 7f8773000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f876c1058d0 0x7f876c1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:44.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.259+0000 7f8773000640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f876c10f9d0 con 0x7f876c103d20 2024-09-10T08:24:44.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.259+0000 7f8773000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f876c10f850 con 0x7f876c1058d0 2024-09-10T08:24:44.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.259+0000 7f8773000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f876c10fb50 con 0x7f876c104f10 2024-09-10T08:24:44.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.260+0000 7f876be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f876c103d20 0x7f876c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:44.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.260+0000 7f876be00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f876c103d20 0x7f876c19a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39624/0 (socket says 172.21.15.2:39624) 2024-09-10T08:24:44.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.260+0000 7f876be00640 1 -- 172.21.15.2:0/967415979 learned_addr learned my addr 172.21.15.2:0/967415979 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.260+0000 7f876b400640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f876c104f10 0x7f876c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.260+0000 7f8771000640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f876c1058d0 0x7f876c1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.260+0000 7f876be00640 1 -- 172.21.15.2:0/967415979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f876c104f10 msgr2=0x7f876c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.260+0000 7f876be00640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f876c104f10 0x7f876c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.260+0000 7f876be00640 1 -- 172.21.15.2:0/967415979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f876c1058d0 msgr2=0x7f876c1a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.260+0000 7f876be00640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f876c1058d0 0x7f876c1a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.260+0000 7f876be00640 1 -- 172.21.15.2:0/967415979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c1a1c80 con 0x7f876c103d20 2024-09-10T08:24:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.260+0000 7f876b400640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f876c104f10 0x7f876c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:24:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.260+0000 7f876be00640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f876c103d20 0x7f876c19a060 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f875c00e9c0 tx=0x7f875c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.261+0000 7f8768c00640 1 -- 172.21.15.2:0/967415979 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c00cd80 con 0x7f876c103d20 2024-09-10T08:24:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.261+0000 7f8768c00640 1 -- 172.21.15.2:0/967415979 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f875c0035f0 con 0x7f876c103d20 2024-09-10T08:24:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.261+0000 7f8773000640 1 -- 172.21.15.2:0/967415979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876c1a1f70 con 0x7f876c103d20 2024-09-10T08:24:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.261+0000 7f8768c00640 1 -- 172.21.15.2:0/967415979 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c010690 con 0x7f876c103d20 2024-09-10T08:24:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.261+0000 7f8773000640 1 -- 172.21.15.2:0/967415979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876c1a23e0 con 0x7f876c103d20 2024-09-10T08:24:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.262+0000 7f8773000640 1 -- 172.21.15.2:0/967415979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f876c10b160 con 0x7f876c103d20 2024-09-10T08:24:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.264+0000 7f8768c00640 1 -- 172.21.15.2:0/967415979 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f875c0037a0 con 0x7f876c103d20 2024-09-10T08:24:44.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.265+0000 7f8768c00640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f87500a2c10 0x7f87500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:44.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.265+0000 7f8768c00640 1 -- 172.21.15.2:0/967415979 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f875c0cf8b0 con 0x7f876c103d20 2024-09-10T08:24:44.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.265+0000 7f876b400640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f87500a2c10 0x7f87500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:44.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.266+0000 7f876b400640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f87500a2c10 0x7f87500a50d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8758006010 tx=0x7f8758000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:44.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.269+0000 7f8768c00640 1 -- 172.21.15.2:0/967415979 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f875c09c3c0 con 0x7f876c103d20 2024-09-10T08:24:44.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.453+0000 7f8773000640 1 -- 172.21.15.2:0/967415979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f876c062ef0 con 0x7f876c103d20 2024-09-10T08:24:44.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.454+0000 7f8768c00640 1 -- 172.21.15.2:0/967415979 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f875c09bcc0 con 0x7f876c103d20 2024-09-10T08:24:44.454 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-09-10T08:24:44.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.456+0000 7f8773000640 1 -- 172.21.15.2:0/967415979 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f87500a2c10 msgr2=0x7f87500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:44.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.456+0000 7f8773000640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f87500a2c10 0x7f87500a50d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8758006010 tx=0x7f8758000bc0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.456+0000 7f8773000640 1 -- 172.21.15.2:0/967415979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f876c103d20 msgr2=0x7f876c19a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:44.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.456+0000 7f8773000640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f876c103d20 0x7f876c19a060 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f875c00e9c0 tx=0x7f875c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:24:44.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.456+0000 7f8773000640 1 -- 172.21.15.2:0/967415979 shutdown_connections 2024-09-10T08:24:44.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.456+0000 7f8773000640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f876c1058d0 0x7f876c1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.456+0000 7f8773000640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f87500a2c10 0x7f87500a50d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.456+0000 7f8773000640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f876c104f10 0x7f876c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.456+0000 7f8773000640 1 --2- 172.21.15.2:0/967415979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f876c103d20 0x7f876c19a060 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.456+0000 7f8773000640 1 -- 172.21.15.2:0/967415979 >> 172.21.15.2:0/967415979 conn(0x7f876c0ff4a0 msgr2=0x7f876c108330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:44.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.456+0000 7f8773000640 1 -- 172.21.15.2:0/967415979 shutdown_connections 2024-09-10T08:24:44.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.456+0000 7f8773000640 1 -- 172.21.15.2:0/967415979 wait complete. 2024-09-10T08:24:44.472 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T08:24:44.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.633+0000 7fecaaa00640 1 -- 172.21.15.2:0/1351310290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca4103c10 msgr2=0x7feca4104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:44.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.633+0000 7fecaaa00640 1 --2- 172.21.15.2:0/1351310290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca4103c10 0x7feca4104010 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fec98009900 tx=0x7fec9802a410 comp rx=0 tx=0).stop 2024-09-10T08:24:44.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.634+0000 7fecaaa00640 1 -- 172.21.15.2:0/1351310290 shutdown_connections 2024-09-10T08:24:44.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.634+0000 7fecaaa00640 1 --2- 172.21.15.2:0/1351310290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca41057c0 0x7feca410c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.634+0000 7fecaaa00640 1 --2- 172.21.15.2:0/1351310290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feca4104e00 0x7feca4105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.634+0000 7fecaaa00640 1 --2- 172.21.15.2:0/1351310290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca4103c10 0x7feca4104010 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.634+0000 7fecaaa00640 1 -- 172.21.15.2:0/1351310290 >> 172.21.15.2:0/1351310290 conn(0x7feca40ff390 msgr2=0x7feca41017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:44.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.634+0000 7fecaaa00640 1 -- 172.21.15.2:0/1351310290 shutdown_connections 2024-09-10T08:24:44.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.634+0000 7fecaaa00640 1 -- 172.21.15.2:0/1351310290 wait complete. 2024-09-10T08:24:44.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.635+0000 7fecaaa00640 1 Processor -- start 2024-09-10T08:24:44.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.635+0000 7fecaaa00640 1 -- start start 2024-09-10T08:24:44.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.635+0000 7fecaaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feca4103c10 0x7feca4199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:44.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.635+0000 7fecaaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca4104e00 0x7feca419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:44.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.635+0000 7fecaaa00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca41057c0 0x7feca41a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:44.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.635+0000 7fecaaa00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7feca410f8c0 con 0x7feca41057c0 2024-09-10T08:24:44.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.635+0000 7fecaaa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feca410f740 con 0x7feca4103c10 2024-09-10T08:24:44.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.635+0000 7fecaaa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feca410fa40 con 0x7feca4104e00 2024-09-10T08:24:44.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.635+0000 7fecaa000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca41057c0 0x7feca41a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:44.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.635+0000 7fecaa000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca41057c0 0x7feca41a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39648/0 (socket says 172.21.15.2:39648) 2024-09-10T08:24:44.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.635+0000 7fecaa000640 1 -- 172.21.15.2:0/1132329164 learned_addr learned my addr 172.21.15.2:0/1132329164 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:44.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.635+0000 7feca9600640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feca4103c10 0x7feca4199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:44.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.635+0000 7feca8c00640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca4104e00 0x7feca419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:44.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.636+0000 7fecaa000640 1 -- 172.21.15.2:0/1132329164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca4104e00 msgr2=0x7feca419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:44.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.636+0000 7fecaa000640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca4104e00 0x7feca419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.636+0000 7fecaa000640 1 -- 172.21.15.2:0/1132329164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feca4103c10 msgr2=0x7feca4199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:44.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.636+0000 7fecaa000640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feca4103c10 0x7feca4199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.636+0000 7fecaa000640 1 -- 172.21.15.2:0/1132329164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca41a1b60 con 0x7feca41057c0 2024-09-10T08:24:44.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.636+0000 7fecaa000640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca41057c0 0x7feca41a14f0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00efd0 tx=0x7fec9c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:44.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.636+0000 7feca2000640 1 -- 172.21.15.2:0/1132329164 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c003b70 con 0x7feca41057c0 2024-09-10T08:24:44.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.636+0000 7feca2000640 1 -- 172.21.15.2:0/1132329164 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec9c003d10 con 0x7feca41057c0 2024-09-10T08:24:44.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.636+0000 7feca2000640 1 -- 172.21.15.2:0/1132329164 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0109f0 con 0x7feca41057c0 2024-09-10T08:24:44.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.636+0000 7fecaaa00640 1 -- 172.21.15.2:0/1132329164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca41a1e50 con 0x7feca41057c0 2024-09-10T08:24:44.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.636+0000 7fecaaa00640 1 -- 172.21.15.2:0/1132329164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca41a22c0 con 0x7feca41057c0 2024-09-10T08:24:44.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.637+0000 7fecaaa00640 1 -- 172.21.15.2:0/1132329164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca410af50 con 0x7feca41057c0 2024-09-10T08:24:44.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.639+0000 7feca2000640 1 -- 172.21.15.2:0/1132329164 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fec9c004020 con 0x7feca41057c0 2024-09-10T08:24:44.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.640+0000 7feca2000640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fec8c0a2b90 0x7fec8c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:44.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.641+0000 7feca2000640 1 -- 172.21.15.2:0/1132329164 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fec9c0cf8b0 con 0x7feca41057c0 2024-09-10T08:24:44.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.641+0000 7feca9600640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fec8c0a2b90 0x7fec8c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:44.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.642+0000 7feca9600640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fec8c0a2b90 0x7fec8c0a5050 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fec9802aa00 tx=0x7fec980315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:44.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.645+0000 7feca2000640 1 -- 172.21.15.2:0/1132329164 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec9c09c3c0 con 0x7feca41057c0 2024-09-10T08:24:44.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:44 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1489509554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:24:44.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:44 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3452563614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:24:44.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:44 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/967415979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:24:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.832+0000 7fecaaa00640 1 -- 172.21.15.2:0/1132329164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7feca4062df0 con 0x7feca41057c0 2024-09-10T08:24:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.832+0000 7feca2000640 1 -- 172.21.15.2:0/1132329164 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7fec9c09bcc0 con 0x7feca41057c0 2024-09-10T08:24:44.833 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-09-10T08:24:44.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.835+0000 7fecaaa00640 1 -- 172.21.15.2:0/1132329164 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fec8c0a2b90 msgr2=0x7fec8c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:44.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.835+0000 7fecaaa00640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fec8c0a2b90 0x7fec8c0a5050 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fec9802aa00 tx=0x7fec980315b0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.835+0000 7fecaaa00640 1 -- 172.21.15.2:0/1132329164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca41057c0 msgr2=0x7feca41a14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:44.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.835+0000 7fecaaa00640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca41057c0 0x7feca41a14f0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00efd0 tx=0x7fec9c00c6a0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.835+0000 7fecaaa00640 1 -- 172.21.15.2:0/1132329164 shutdown_connections 2024-09-10T08:24:44.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.835+0000 7fecaaa00640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca41057c0 0x7feca41a14f0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.835+0000 7fecaaa00640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fec8c0a2b90 0x7fec8c0a5050 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.835+0000 7fecaaa00640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca4104e00 0x7feca419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.835+0000 7fecaaa00640 1 --2- 172.21.15.2:0/1132329164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feca4103c10 0x7feca4199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:44.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.835+0000 7fecaaa00640 1 -- 172.21.15.2:0/1132329164 >> 172.21.15.2:0/1132329164 conn(0x7feca40ff390 msgr2=0x7feca4108120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:44.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.835+0000 7fecaaa00640 1 -- 172.21.15.2:0/1132329164 shutdown_connections 2024-09-10T08:24:44.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:44.835+0000 7fecaaa00640 1 -- 172.21.15.2:0/1132329164 wait complete. 2024-09-10T08:24:44.851 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T08:24:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.008+0000 7f03cfe00640 1 -- 172.21.15.2:0/3919423475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0074bf0 msgr2=0x7f03d0075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.008+0000 7f03cfe00640 1 --2- 172.21.15.2:0/3919423475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0074bf0 0x7f03d0075050 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009900 tx=0x7f03b802a480 comp rx=0 tx=0).stop 2024-09-10T08:24:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.009+0000 7f03cfe00640 1 -- 172.21.15.2:0/3919423475 shutdown_connections 2024-09-10T08:24:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.009+0000 7f03cfe00640 1 --2- 172.21.15.2:0/3919423475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03d0109800 0x7f03d010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.009+0000 7f03cfe00640 1 --2- 172.21.15.2:0/3919423475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0074bf0 0x7f03d0075050 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.009+0000 7f03cfe00640 1 --2- 172.21.15.2:0/3919423475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03d00761d0 0x7f03d0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.009+0000 7f03cfe00640 1 -- 172.21.15.2:0/3919423475 >> 172.21.15.2:0/3919423475 conn(0x7f03d00fd110 msgr2=0x7f03d00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.009+0000 7f03cfe00640 1 -- 172.21.15.2:0/3919423475 shutdown_connections 2024-09-10T08:24:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.009+0000 7f03cfe00640 1 -- 172.21.15.2:0/3919423475 wait complete. 2024-09-10T08:24:45.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03cfe00640 1 Processor -- start 2024-09-10T08:24:45.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03cfe00640 1 -- start start 2024-09-10T08:24:45.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03cfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03d0074bf0 0x7f03d0199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:45.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03cfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03d00761d0 0x7f03d019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:45.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03cfe00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0109800 0x7f03d01a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03cfe00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03d010f830 con 0x7f03d0109800 2024-09-10T08:24:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03cfe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03d010f6b0 con 0x7f03d00761d0 2024-09-10T08:24:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03cfe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03d010f9b0 con 0x7f03d0074bf0 2024-09-10T08:24:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03cf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0109800 0x7f03d01a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03c6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03d00761d0 0x7f03d019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03c6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03d00761d0 0x7f03d019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:41618/0 (socket says 172.21.15.2:41618) 2024-09-10T08:24:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03cf400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0109800 0x7f03d01a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39676/0 (socket says 172.21.15.2:39676) 2024-09-10T08:24:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03c6000640 1 -- 172.21.15.2:0/3502432971 learned_addr learned my addr 172.21.15.2:0/3502432971 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:24:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.010+0000 7f03cea00640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03d0074bf0 0x7f03d0199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.011+0000 7f03c6000640 1 -- 172.21.15.2:0/3502432971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03d0074bf0 msgr2=0x7f03d0199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.011+0000 7f03c6000640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03d0074bf0 0x7f03d0199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.011+0000 7f03c6000640 1 -- 172.21.15.2:0/3502432971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0109800 msgr2=0x7f03d01a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.011+0000 7f03c6000640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0109800 0x7f03d01a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:45.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.011+0000 7f03c6000640 1 -- 172.21.15.2:0/3502432971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d01a1ac0 con 0x7f03d00761d0 2024-09-10T08:24:45.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.011+0000 7f03cf400640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0109800 0x7f03d01a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:45.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.011+0000 7f03cea00640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03d0074bf0 0x7f03d0199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:24:45.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.011+0000 7f03c6000640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03d00761d0 0x7f03d019a3e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f03b8031440 tx=0x7f03b8032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:45.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.011+0000 7f03c7e00640 1 -- 172.21.15.2:0/3502432971 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b803f030 con 0x7f03d00761d0 2024-09-10T08:24:45.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.012+0000 7f03c7e00640 1 -- 172.21.15.2:0/3502432971 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b8003ce0 con 0x7f03d00761d0 2024-09-10T08:24:45.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.012+0000 7f03cfe00640 1 -- 172.21.15.2:0/3502432971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d01a1d50 con 0x7f03d00761d0 2024-09-10T08:24:45.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.012+0000 7f03c7e00640 1 -- 172.21.15.2:0/3502432971 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8035070 con 0x7f03d00761d0 2024-09-10T08:24:45.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.012+0000 7f03cfe00640 1 -- 172.21.15.2:0/3502432971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d01a21e0 con 0x7f03d00761d0 2024-09-10T08:24:45.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.013+0000 7f03cfe00640 1 -- 172.21.15.2:0/3502432971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d01085c0 con 0x7f03d00761d0 2024-09-10T08:24:45.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.015+0000 7f03c7e00640 1 -- 172.21.15.2:0/3502432971 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f03b8042030 con 0x7f03d00761d0 2024-09-10T08:24:45.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.016+0000 7f03c7e00640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f03b40a2b90 0x7f03b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:24:45.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.016+0000 7f03c7e00640 1 -- 172.21.15.2:0/3502432971 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f03b80efc90 con 0x7f03d00761d0 2024-09-10T08:24:45.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.016+0000 7f03cea00640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f03b40a2b90 0x7f03b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:24:45.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.017+0000 7f03cea00640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f03b40a2b90 0x7f03b40a5050 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f03c00095f0 tx=0x7f03c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:24:45.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.018+0000 7f03c7e00640 1 -- 172.21.15.2:0/3502432971 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03b80bc1b0 con 0x7f03d00761d0 2024-09-10T08:24:45.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:44 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1489509554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:24:45.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:44 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3452563614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:24:45.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:44 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/967415979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:24:45.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:44 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1489509554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:24:45.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:44 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3452563614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:24:45.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:44 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/967415979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:24:45.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.249+0000 7f03cfe00640 1 -- 172.21.15.2:0/3502432971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f03d002da10 con 0x7f03d00761d0 2024-09-10T08:24:45.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.259+0000 7f03c7e00640 1 -- 172.21.15.2:0/3502432971 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740362 (secure 0 0 0) 0x7f03b80c04b0 con 0x7f03d00761d0 2024-09-10T08:24:45.269 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:24:45.269 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:24:45.270 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 20, 2024-09-10T08:24:45.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.270 INFO:teuthology.orchestra.run.smithi002.stdout: "active_gid": 14150, 2024-09-10T08:24:45.270 INFO:teuthology.orchestra.run.smithi002.stdout: "active_name": "x", 2024-09-10T08:24:45.270 INFO:teuthology.orchestra.run.smithi002.stdout: "active_addrs": { 2024-09-10T08:24:45.270 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-10T08:24:45.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-10T08:24:45.270 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-10T08:24:45.270 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3751597725 2024-09-10T08:24:45.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-10T08:24:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-10T08:24:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3751597725 2024-09-10T08:24:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-10T08:24:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "active_addr": "172.21.15.2:6801/3751597725", 2024-09-10T08:24:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "active_change": "2024-09-10T07:58:03.548889+0000", 2024-09-10T08:24:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "active_mgr_features": 4540701547738038271, 2024-09-10T08:24:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "available": true, 2024-09-10T08:24:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "standbys": [ 2024-09-10T08:24:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.272 INFO:teuthology.orchestra.run.smithi002.stdout: "gid": 24109, 2024-09-10T08:24:45.272 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "z", 2024-09-10T08:24:45.272 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_features": 4540701547738038271, 2024-09-10T08:24:45.272 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2024-09-10T08:24:45.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.272 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2024-09-10T08:24:45.272 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.272 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.272 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.272 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:45.272 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:45.272 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.272 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.273 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:45.273 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.273 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.273 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.273 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T08:24:45.273 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.273 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.273 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.273 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.273 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.274 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.274 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.274 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.274 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.274 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.274 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.274 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.274 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.274 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.274 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.274 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.275 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.275 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.275 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.275 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.275 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.275 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.275 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.275 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.275 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.276 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.276 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.276 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.276 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.276 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.276 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.276 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.276 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.276 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.276 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.276 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.277 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.277 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.277 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.277 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.277 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.277 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.277 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.277 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.277 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.277 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.277 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.277 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.277 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.278 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.278 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.278 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.278 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.278 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.278 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.278 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.278 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.278 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.278 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.279 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.279 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.279 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.279 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.279 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2024-09-10T08:24:45.279 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2024-09-10T08:24:45.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.279 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.279 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.279 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.280 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.280 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.280 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2024-09-10T08:24:45.280 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.280 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.280 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.280 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2024-09-10T08:24:45.280 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2024-09-10T08:24:45.280 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.280 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.280 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.280 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2024-09-10T08:24:45.281 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.281 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.281 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.281 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2024-09-10T08:24:45.281 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.281 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.281 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.281 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2024-09-10T08:24:45.281 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2024-09-10T08:24:45.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.281 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.282 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.283 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.283 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.283 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.283 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.283 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.283 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2024-09-10T08:24:45.283 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.283 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.283 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.283 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2024-09-10T08:24:45.283 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2024-09-10T08:24:45.283 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.284 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.284 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2024-09-10T08:24:45.284 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.284 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.284 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.284 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2024-09-10T08:24:45.284 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.284 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.284 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.284 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2024-09-10T08:24:45.284 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2024-09-10T08:24:45.285 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.285 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.285 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.285 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.285 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.285 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.285 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2024-09-10T08:24:45.285 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.285 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2024-09-10T08:24:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2024-09-10T08:24:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T08:24:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.287 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.287 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2024-09-10T08:24:45.287 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2024-09-10T08:24:45.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.287 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.287 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.287 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.287 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.287 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.287 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.287 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2024-09-10T08:24:45.287 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.288 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.288 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.288 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.288 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.288 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2024-09-10T08:24:45.288 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.288 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.288 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.288 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2024-09-10T08:24:45.288 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2024-09-10T08:24:45.288 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.289 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.289 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.289 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.289 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.289 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.289 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.289 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T08:24:45.289 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.289 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.289 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.289 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2024-09-10T08:24:45.289 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2024-09-10T08:24:45.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.290 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.290 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.290 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2024-09-10T08:24:45.290 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.290 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.290 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.290 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T08:24:45.290 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T08:24:45.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.290 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.290 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2024-09-10T08:24:45.291 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2024-09-10T08:24:45.291 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:45.291 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.291 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:45.291 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-09-10T08:24:45.291 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-09-10T08:24:45.291 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.291 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T08:24:45.291 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T08:24:45.291 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.291 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.293 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2024-09-10T08:24:45.293 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2024-09-10T08:24:45.293 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.293 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.293 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.293 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T08:24:45.293 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.293 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.293 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.293 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T08:24:45.293 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T08:24:45.293 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.294 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.294 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2024-09-10T08:24:45.294 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2024-09-10T08:24:45.294 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:45.294 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.294 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.294 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2024-09-10T08:24:45.294 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2024-09-10T08:24:45.294 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2024-09-10T08:24:45.294 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.294 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2024-09-10T08:24:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T08:24:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2024-09-10T08:24:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2024-09-10T08:24:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2024-09-10T08:24:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.296 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T08:24:45.296 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T08:24:45.296 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.296 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.296 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2024-09-10T08:24:45.296 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2024-09-10T08:24:45.296 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:45.296 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.296 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.296 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:45.296 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-09-10T08:24:45.296 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.297 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.298 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.298 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.298 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.298 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.298 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.298 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.298 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.298 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.298 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.298 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.298 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.298 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.299 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.299 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.299 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.299 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.299 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.299 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.299 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.299 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.299 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.300 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.301 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.301 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.301 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.301 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.301 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.301 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.301 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.301 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.301 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.301 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.301 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.301 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.302 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.302 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.302 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.302 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.302 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.302 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.302 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.302 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.302 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.302 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.302 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.302 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2024-09-10T08:24:45.303 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2024-09-10T08:24:45.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:45.303 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.303 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:45.303 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.303 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.303 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.303 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T08:24:45.303 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.303 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-09-10T08:24:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-09-10T08:24:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2024-09-10T08:24:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2024-09-10T08:24:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2024-09-10T08:24:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "read", 2024-09-10T08:24:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap", 2024-09-10T08:24:45.305 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap-read" 2024-09-10T08:24:45.305 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.305 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2024-09-10T08:24:45.305 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.305 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.305 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.305 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2024-09-10T08:24:45.305 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2024-09-10T08:24:45.305 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.305 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.305 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.305 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.306 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:45.307 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.307 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.307 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.307 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T08:24:45.307 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.307 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.307 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.307 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2024-09-10T08:24:45.307 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2024-09-10T08:24:45.307 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.307 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.307 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-09-10T08:24:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-09-10T08:24:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T08:24:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T08:24:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2024-09-10T08:24:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2024-09-10T08:24:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.309 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2024-09-10T08:24:45.310 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.310 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.310 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.310 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2024-09-10T08:24:45.310 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2024-09-10T08:24:45.310 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:45.310 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.310 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.310 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-09-10T08:24:45.310 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.310 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.310 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.310 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T08:24:45.311 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.311 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.311 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.311 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2024-09-10T08:24:45.311 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2024-09-10T08:24:45.311 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.311 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.311 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.311 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2024-09-10T08:24:45.311 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.311 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.311 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T08:24:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2024-09-10T08:24:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2024-09-10T08:24:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2024-09-10T08:24:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.313 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.313 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T08:24:45.313 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.313 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.313 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.313 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2024-09-10T08:24:45.313 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2024-09-10T08:24:45.313 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.313 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.313 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.314 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.314 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.314 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.314 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.314 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T08:24:45.314 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T08:24:45.314 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.314 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.314 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2024-09-10T08:24:45.314 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2024-09-10T08:24:45.314 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.315 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.315 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:45.315 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.315 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.315 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.315 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T08:24:45.315 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.315 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.315 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2024-09-10T08:24:45.315 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T08:24:45.316 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:45.316 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.316 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.316 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2024-09-10T08:24:45.316 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.316 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.316 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.316 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T08:24:45.316 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.316 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.316 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.317 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_log_destination": { 2024-09-10T08:24:45.317 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm_log_destination", 2024-09-10T08:24:45.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.317 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.317 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.317 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.317 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.317 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.317 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.317 INFO:teuthology.orchestra.run.smithi002.stdout: "file", 2024-09-10T08:24:45.317 INFO:teuthology.orchestra.run.smithi002.stdout: "file,syslog", 2024-09-10T08:24:45.317 INFO:teuthology.orchestra.run.smithi002.stdout: "syslog" 2024-09-10T08:24:45.317 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T08:24:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2024-09-10T08:24:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2024-09-10T08:24:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T08:24:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2024-09-10T08:24:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2024-09-10T08:24:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.320 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.320 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.320 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T08:24:45.320 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.320 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.320 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.320 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2024-09-10T08:24:45.320 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2024-09-10T08:24:45.320 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.321 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.321 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.321 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.321 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.321 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.321 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T08:24:45.321 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.321 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.321 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2024-09-10T08:24:45.322 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2024-09-10T08:24:45.322 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.322 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.322 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.322 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T08:24:45.322 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.322 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.322 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.322 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-09-10T08:24:45.322 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.322 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.322 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.324 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2024-09-10T08:24:45.324 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2024-09-10T08:24:45.324 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.324 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.324 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.324 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T08:24:45.324 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.324 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.324 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.324 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2024-09-10T08:24:45.324 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.325 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.325 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2024-09-10T08:24:45.325 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2024-09-10T08:24:45.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.325 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.325 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.325 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T08:24:45.325 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.325 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.325 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.325 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-09-10T08:24:45.325 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.326 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.326 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.326 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2024-09-10T08:24:45.326 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2024-09-10T08:24:45.326 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.326 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.326 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T08:24:45.326 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.326 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.326 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2024-09-10T08:24:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2024-09-10T08:24:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2024-09-10T08:24:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T08:24:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.328 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.328 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2024-09-10T08:24:45.328 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.328 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.328 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.328 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2024-09-10T08:24:45.328 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2024-09-10T08:24:45.328 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.328 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.328 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.328 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T08:24:45.329 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.329 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.329 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.329 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2024-09-10T08:24:45.329 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.329 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.329 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.329 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2024-09-10T08:24:45.329 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2024-09-10T08:24:45.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.329 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.330 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.331 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T08:24:45.331 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.331 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.331 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.331 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2024-09-10T08:24:45.331 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.331 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.331 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2024-09-10T08:24:45.331 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2024-09-10T08:24:45.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.331 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.332 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.332 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T08:24:45.332 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.332 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.332 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.332 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2024-09-10T08:24:45.332 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.332 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.332 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.332 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2024-09-10T08:24:45.332 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2024-09-10T08:24:45.333 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.333 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.333 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.333 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T08:24:45.333 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.333 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.333 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.333 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-09-10T08:24:45.333 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.333 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.333 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.333 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2024-09-10T08:24:45.334 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2024-09-10T08:24:45.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.334 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.334 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T08:24:45.334 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.334 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.334 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.334 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2024-09-10T08:24:45.334 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.334 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.334 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2024-09-10T08:24:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2024-09-10T08:24:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T08:24:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-09-10T08:24:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.335 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.336 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2024-09-10T08:24:45.336 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2024-09-10T08:24:45.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.336 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.336 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.336 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T08:24:45.336 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.336 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.336 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.336 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2024-09-10T08:24:45.336 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.336 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.337 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.337 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_samba": { 2024-09-10T08:24:45.337 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_samba", 2024-09-10T08:24:45.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.337 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.337 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T08:24:45.337 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.337 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.337 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.337 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Samba/SMB container image", 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.338 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2024-09-10T08:24:45.339 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.339 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.339 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.339 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2024-09-10T08:24:45.339 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2024-09-10T08:24:45.339 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.339 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.339 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.339 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.339 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.339 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T08:24:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2024-09-10T08:24:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2024-09-10T08:24:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:45.340 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.341 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.341 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.341 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T08:24:45.341 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.341 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.341 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.341 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2024-09-10T08:24:45.341 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T08:24:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2024-09-10T08:24:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T08:24:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2024-09-10T08:24:45.343 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2024-09-10T08:24:45.343 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.343 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.343 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.343 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2024-09-10T08:24:45.343 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.343 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.343 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.343 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T08:24:45.343 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.343 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.343 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.344 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2024-09-10T08:24:45.344 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2024-09-10T08:24:45.344 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.344 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.344 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.344 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2024-09-10T08:24:45.344 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.344 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.344 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.344 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2024-09-10T08:24:45.344 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.344 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.344 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.345 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2024-09-10T08:24:45.345 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2024-09-10T08:24:45.345 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.345 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.345 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.345 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.345 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.345 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.345 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.345 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T08:24:45.345 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.345 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.346 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.346 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2024-09-10T08:24:45.346 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2024-09-10T08:24:45.346 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.346 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.346 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.346 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:45.346 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.346 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.346 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.346 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2024-09-10T08:24:45.346 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: "grafana_dashboards_path": { 2024-09-10T08:24:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "grafana_dashboards_path", 2024-09-10T08:24:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T08:24:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.348 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2024-09-10T08:24:45.349 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.349 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.349 INFO:teuthology.orchestra.run.smithi002.stdout: "hw_monitoring": { 2024-09-10T08:24:45.349 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hw_monitoring", 2024-09-10T08:24:45.349 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.349 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.349 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.350 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.350 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.350 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.350 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.351 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.352 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T08:24:45.352 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.352 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.352 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.352 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.352 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.352 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.352 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.352 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.352 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.352 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.352 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.353 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2024-09-10T08:24:45.354 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2024-09-10T08:24:45.354 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.354 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.354 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.354 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.354 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.354 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.354 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.354 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T08:24:45.354 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.354 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.355 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.355 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.355 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.355 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.355 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.355 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.355 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.355 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.355 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T08:24:45.355 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.356 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.356 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.356 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.356 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.356 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.356 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.356 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.356 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.356 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.356 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.356 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.356 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.357 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.357 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.357 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.357 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.357 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.357 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.357 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.357 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.357 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.357 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.357 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.357 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.358 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.358 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.358 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.358 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.358 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.358 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.358 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.358 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.358 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.358 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.358 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.358 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T08:24:45.360 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T08:24:45.360 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.360 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.360 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.360 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2024-09-10T08:24:45.360 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.360 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.360 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.360 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T08:24:45.360 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.360 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.360 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.361 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.362 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2024-09-10T08:24:45.362 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2024-09-10T08:24:45.362 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.362 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.362 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.362 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.362 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.362 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.362 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T08:24:45.362 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.363 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.363 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2024-09-10T08:24:45.363 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2024-09-10T08:24:45.363 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.363 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.363 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.363 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.363 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.363 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.363 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.364 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2024-09-10T08:24:45.364 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.364 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.364 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.364 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-09-10T08:24:45.364 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-09-10T08:24:45.364 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.364 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.364 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.364 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2024-09-10T08:24:45.364 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.364 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: "oob_default_addr": { 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "oob_default_addr", 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.365 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.366 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "169.254.1.1", 2024-09-10T08:24:45.366 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.366 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.366 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.366 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T08:24:45.366 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.366 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.366 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.366 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2024-09-10T08:24:45.366 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2024-09-10T08:24:45.366 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.366 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.367 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T08:24:45.367 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.367 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.367 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.367 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T08:24:45.367 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.367 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.367 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.367 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2024-09-10T08:24:45.367 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2024-09-10T08:24:45.368 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.368 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.368 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.368 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.368 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.368 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.368 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.368 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T08:24:45.368 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.368 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.368 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.368 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2024-09-10T08:24:45.369 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2024-09-10T08:24:45.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.369 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.369 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.369 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.369 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.369 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.369 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T08:24:45.369 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.369 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.369 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.370 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2024-09-10T08:24:45.370 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2024-09-10T08:24:45.370 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.370 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.370 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.370 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.370 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.370 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.370 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.370 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T08:24:45.370 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.370 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.370 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.371 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2024-09-10T08:24:45.371 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2024-09-10T08:24:45.371 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.371 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.371 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.371 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.371 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.371 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.371 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.371 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T08:24:45.371 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.372 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.372 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2024-09-10T08:24:45.372 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2024-09-10T08:24:45.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.372 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.372 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.372 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.372 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.372 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.372 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.372 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T08:24:45.372 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.373 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.373 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2024-09-10T08:24:45.373 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2024-09-10T08:24:45.373 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.373 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.373 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.373 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2024-09-10T08:24:45.373 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.373 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.373 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.373 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2024-09-10T08:24:45.374 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.374 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.374 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.374 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2024-09-10T08:24:45.374 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2024-09-10T08:24:45.374 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.374 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.374 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.374 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.374 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.374 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.375 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.376 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T08:24:45.376 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.376 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.376 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2024-09-10T08:24:45.376 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2024-09-10T08:24:45.376 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.376 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.376 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.376 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.376 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.376 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T08:24:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2024-09-10T08:24:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2024-09-10T08:24:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.378 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.380 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.380 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.380 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T08:24:45.380 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.380 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.380 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.380 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2024-09-10T08:24:45.380 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.381 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.382 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.382 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.382 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.382 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.382 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.382 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.382 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.382 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.382 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.382 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.382 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.382 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.383 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.383 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.383 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.383 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.383 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.383 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.383 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.383 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.383 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.383 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.383 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.385 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.385 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.385 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.385 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.385 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.385 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.385 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.385 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.385 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.385 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.385 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.386 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.386 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.386 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.386 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.386 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.386 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.386 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2024-09-10T08:24:45.386 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2024-09-10T08:24:45.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.386 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.386 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.387 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2024-09-10T08:24:45.388 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.388 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.388 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.388 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T08:24:45.388 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.388 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.388 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.388 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2024-09-10T08:24:45.388 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.389 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.389 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.389 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T08:24:45.389 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T08:24:45.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.389 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.389 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.389 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.389 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.389 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.389 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.389 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.390 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.391 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.391 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.391 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T08:24:45.391 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T08:24:45.391 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.391 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.391 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.391 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.391 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.391 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.391 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.392 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.392 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.392 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.392 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2024-09-10T08:24:45.392 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T08:24:45.392 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.392 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.392 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.392 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.392 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.392 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.392 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.393 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.394 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.395 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.395 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.395 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.395 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.395 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T08:24:45.395 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T08:24:45.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.395 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.395 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.395 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T08:24:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T08:24:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.397 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.397 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.397 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.397 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.397 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.397 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.397 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.397 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T08:24:45.397 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T08:24:45.397 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.397 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.397 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.398 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.399 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.399 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.399 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.399 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.399 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.399 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.399 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.399 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.399 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T08:24:45.399 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T08:24:45.399 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.399 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.400 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.400 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.400 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.400 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.400 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.400 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.400 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.400 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T08:24:45.400 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T08:24:45.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T08:24:45.401 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.402 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.402 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.402 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.402 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.402 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.402 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.402 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.402 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.402 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.402 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.402 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T08:24:45.402 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T08:24:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T08:24:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-09-10T08:24:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T08:24:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T08:24:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T08:24:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T08:24:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-09-10T08:24:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.408 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.408 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T08:24:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T08:24:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.410 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.410 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.410 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.410 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.410 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T08:24:45.410 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T08:24:45.410 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.410 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T08:24:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T08:24:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T08:24:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T08:24:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T08:24:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T08:24:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.415 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.415 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.415 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.415 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.415 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.415 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T08:24:45.415 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T08:24:45.415 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.415 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T08:24:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T08:24:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T08:24:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T08:24:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T08:24:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T08:24:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T08:24:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T08:24:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T08:24:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T08:24:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T08:24:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T08:24:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T08:24:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T08:24:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T08:24:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T08:24:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T08:24:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T08:24:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T08:24:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T08:24:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T08:24:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T08:24:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T08:24:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2024-09-10T08:24:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T08:24:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T08:24:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2024-09-10T08:24:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T08:24:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T08:24:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T08:24:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T08:24:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-09-10T08:24:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T08:24:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T08:24:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T08:24:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T08:24:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T08:24:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T08:24:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T08:24:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T08:24:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T08:24:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T08:24:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-09-10T08:24:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2024-09-10T08:24:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2024-09-10T08:24:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2024-09-10T08:24:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2024-09-10T08:24:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2024-09-10T08:24:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2024-09-10T08:24:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2024-09-10T08:24:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2024-09-10T08:24:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2024-09-10T08:24:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2024-09-10T08:24:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-09-10T08:24:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-09-10T08:24:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2024-09-10T08:24:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2024-09-10T08:24:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2024-09-10T08:24:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2024-09-10T08:24:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T08:24:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-09-10T08:24:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-09-10T08:24:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-09-10T08:24:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-09-10T08:24:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-09-10T08:24:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2024-09-10T08:24:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-09-10T08:24:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-09-10T08:24:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2024-09-10T08:24:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2024-09-10T08:24:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2024-09-10T08:24:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-09-10T08:24:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-09-10T08:24:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2024-09-10T08:24:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2024-09-10T08:24:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-09-10T08:24:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-09-10T08:24:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-09-10T08:24:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-09-10T08:24:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-09-10T08:24:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2024-09-10T08:24:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2024-09-10T08:24:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-09-10T08:24:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2024-09-10T08:24:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2024-09-10T08:24:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2024-09-10T08:24:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2024-09-10T08:24:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2024-09-10T08:24:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2024-09-10T08:24:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T08:24:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2024-09-10T08:24:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2024-09-10T08:24:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2024-09-10T08:24:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T08:24:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2024-09-10T08:24:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2024-09-10T08:24:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2024-09-10T08:24:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2024-09-10T08:24:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2024-09-10T08:24:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2024-09-10T08:24:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-09-10T08:24:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T08:24:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2024-09-10T08:24:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2024-09-10T08:24:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T08:24:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T08:24:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2024-09-10T08:24:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2024-09-10T08:24:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2024-09-10T08:24:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T08:24:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2024-09-10T08:24:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2024-09-10T08:24:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2024-09-10T08:24:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.490 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.490 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.490 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-09-10T08:24:45.490 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.490 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.490 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.490 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.490 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.490 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.491 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2024-09-10T08:24:45.491 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2024-09-10T08:24:45.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.491 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.491 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2024-09-10T08:24:45.491 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.491 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.491 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.491 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.491 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.492 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.492 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:45.492 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:45.492 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.492 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.492 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.492 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:45.492 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.493 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.493 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.493 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.493 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.493 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.493 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.493 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2024-09-10T08:24:45.494 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2024-09-10T08:24:45.494 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2024-09-10T08:24:45.494 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.494 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2024-09-10T08:24:45.494 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2024-09-10T08:24:45.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.494 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.494 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2024-09-10T08:24:45.494 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.494 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.494 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.495 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T08:24:45.495 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.495 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.495 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.495 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2024-09-10T08:24:45.495 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2024-09-10T08:24:45.495 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.495 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.495 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.495 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2024-09-10T08:24:45.496 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.496 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.496 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.496 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T08:24:45.496 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.496 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.496 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.496 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2024-09-10T08:24:45.496 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2024-09-10T08:24:45.497 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.497 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.497 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.497 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.497 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.497 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.497 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2024-09-10T08:24:45.497 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.497 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.497 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.498 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:45.498 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:45.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.498 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.498 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2024-09-10T08:24:45.498 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2024-09-10T08:24:45.498 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.498 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.498 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T08:24:45.498 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.499 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.499 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.499 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.499 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.499 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.499 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.500 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.500 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.500 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.500 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.500 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.500 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.500 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.500 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.500 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.501 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.501 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.501 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.501 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.501 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.501 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.501 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.501 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.501 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.501 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.502 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.502 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.502 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.502 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.502 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.502 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.502 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.502 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.502 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.503 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.503 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.503 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.503 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.503 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.503 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.503 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.503 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.503 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.503 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.503 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.504 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.504 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.504 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.504 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.504 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.504 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.504 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.505 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.505 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.505 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.505 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.505 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.505 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.505 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.505 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.505 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.506 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2024-09-10T08:24:45.506 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2024-09-10T08:24:45.506 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.506 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.506 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.506 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.506 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.506 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.506 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.506 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2024-09-10T08:24:45.506 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.507 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.507 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.507 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2024-09-10T08:24:45.507 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2024-09-10T08:24:45.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.507 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.507 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.507 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2024-09-10T08:24:45.507 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.507 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.507 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.508 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2024-09-10T08:24:45.508 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.508 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.508 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-09-10T08:24:45.508 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-09-10T08:24:45.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.508 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.508 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2024-09-10T08:24:45.509 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.509 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.509 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.509 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T08:24:45.509 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.509 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.509 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.509 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2024-09-10T08:24:45.509 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2024-09-10T08:24:45.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.510 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.510 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.510 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-09-10T08:24:45.510 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-09-10T08:24:45.510 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2024-09-10T08:24:45.510 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.510 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T08:24:45.510 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.510 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.510 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.511 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2024-09-10T08:24:45.511 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2024-09-10T08:24:45.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.511 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.511 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.511 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.511 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.511 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.511 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2024-09-10T08:24:45.511 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.511 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.512 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.512 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2024-09-10T08:24:45.512 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2024-09-10T08:24:45.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.512 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.512 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2024-09-10T08:24:45.512 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.512 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.513 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.513 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T08:24:45.513 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.513 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.513 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.513 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.513 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.513 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2024-09-10T08:24:45.513 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.514 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.514 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.514 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.514 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.514 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.514 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.514 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.514 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.514 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.514 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.514 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.514 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.515 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.515 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.515 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.515 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.515 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.515 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.515 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.515 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.515 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.515 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.516 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.516 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.516 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.516 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.516 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.516 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.516 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.516 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.516 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.516 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.516 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.517 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.517 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.517 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.517 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.517 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.517 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.517 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.517 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.518 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.518 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.518 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.518 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.518 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.518 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.518 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.518 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.518 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.518 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.519 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.519 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.519 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.519 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.519 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.519 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.519 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.519 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.519 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.520 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.520 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.520 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.520 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.520 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2024-09-10T08:24:45.520 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.520 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.521 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.521 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.521 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.521 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.521 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.521 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.521 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.521 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.521 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.522 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.522 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.522 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.522 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.522 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.522 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.522 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.522 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.522 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.522 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.522 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.523 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.523 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.523 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.523 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.523 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.523 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.523 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.523 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.523 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.523 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.523 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.524 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.524 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.524 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.524 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.524 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.524 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.524 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.524 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.524 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.525 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.525 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.525 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.525 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.525 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.525 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.525 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.525 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.525 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.525 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.526 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.526 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.526 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.526 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.526 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.526 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.526 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.526 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.526 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.526 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.526 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.527 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.527 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.527 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.527 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.527 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2024-09-10T08:24:45.527 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.527 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.527 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.527 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2024-09-10T08:24:45.527 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2024-09-10T08:24:45.528 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.528 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.528 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.528 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2024-09-10T08:24:45.528 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.528 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.528 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.528 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T08:24:45.528 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.528 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.528 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.529 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2024-09-10T08:24:45.529 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2024-09-10T08:24:45.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.529 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.529 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.529 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2024-09-10T08:24:45.529 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.529 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.530 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T08:24:45.530 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.530 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.530 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.530 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.530 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.530 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.530 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.530 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.531 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.531 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.531 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.531 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.531 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.531 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.531 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.531 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.531 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.531 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.532 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.532 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.532 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.532 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.532 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.532 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.532 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.532 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.532 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.532 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.532 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.533 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.533 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.533 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.533 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.533 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.533 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.533 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.533 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.533 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.534 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.534 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.534 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.534 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.534 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.534 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.534 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.534 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.534 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.534 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.534 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.535 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.535 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.535 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.535 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.535 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.535 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.535 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.535 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.535 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.535 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2024-09-10T08:24:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2024-09-10T08:24:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2024-09-10T08:24:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2024-09-10T08:24:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2024-09-10T08:24:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T08:24:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.538 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.539 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.541 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.542 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.544 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2024-09-10T08:24:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2024-09-10T08:24:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2024-09-10T08:24:45.546 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T08:24:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2024-09-10T08:24:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2024-09-10T08:24:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2024-09-10T08:24:45.547 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2024-09-10T08:24:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2024-09-10T08:24:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2024-09-10T08:24:45.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2024-09-10T08:24:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T08:24:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2024-09-10T08:24:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2024-09-10T08:24:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2024-09-10T08:24:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.550 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2024-09-10T08:24:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2024-09-10T08:24:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2024-09-10T08:24:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.551 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T08:24:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T08:24:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2024-09-10T08:24:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.552 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.553 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.554 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.554 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.554 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.554 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.554 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.554 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.554 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.554 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.554 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.554 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.555 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.556 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.557 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.557 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.557 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.557 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.557 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.557 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.557 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.557 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.557 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2024-09-10T08:24:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.560 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.561 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.562 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.562 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.562 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.562 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.562 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.562 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.562 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.562 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.563 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.564 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.565 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.566 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.566 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.566 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.566 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.566 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.566 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.566 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.566 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.566 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.566 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.566 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.567 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2024-09-10T08:24:45.567 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.567 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.567 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.567 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.567 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.568 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.571 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.573 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-09-10T08:24:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2024-09-10T08:24:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2024-09-10T08:24:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.574 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T08:24:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.575 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.576 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.577 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.578 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.579 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.580 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2024-09-10T08:24:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-09-10T08:24:45.581 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-09-10T08:24:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2024-09-10T08:24:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2024-09-10T08:24:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.582 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2024-09-10T08:24:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2024-09-10T08:24:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.583 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.584 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.586 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.587 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.589 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2024-09-10T08:24:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.590 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.591 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.593 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.594 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.596 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2024-09-10T08:24:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.597 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.598 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.599 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.600 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.601 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.602 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.604 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.605 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.606 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2024-09-10T08:24:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2024-09-10T08:24:45.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-09-10T08:24:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2024-09-10T08:24:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2024-09-10T08:24:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T08:24:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.608 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2024-09-10T08:24:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.609 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2024-09-10T08:24:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2024-09-10T08:24:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T08:24:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.610 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-09-10T08:24:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-09-10T08:24:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.611 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.612 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.613 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.615 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.616 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.617 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2024-09-10T08:24:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2024-09-10T08:24:45.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2024-09-10T08:24:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2024-09-10T08:24:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.619 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-09-10T08:24:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.620 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2024-09-10T08:24:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2024-09-10T08:24:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.621 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2024-09-10T08:24:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2024-09-10T08:24:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2024-09-10T08:24:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2024-09-10T08:24:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.623 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T08:24:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T08:24:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.624 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.625 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.626 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.628 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.629 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.630 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2024-09-10T08:24:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2024-09-10T08:24:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.631 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T08:24:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T08:24:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.632 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2024-09-10T08:24:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2024-09-10T08:24:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2024-09-10T08:24:45.633 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2024-09-10T08:24:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.634 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-09-10T08:24:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-09-10T08:24:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-09-10T08:24:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T08:24:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.635 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-09-10T08:24:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-09-10T08:24:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2024-09-10T08:24:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T08:24:45.636 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2024-09-10T08:24:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2024-09-10T08:24:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2024-09-10T08:24:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.637 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-09-10T08:24:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-09-10T08:24:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.638 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2024-09-10T08:24:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2024-09-10T08:24:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2024-09-10T08:24:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-09-10T08:24:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-09-10T08:24:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-09-10T08:24:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-09-10T08:24:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-09-10T08:24:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.640 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-09-10T08:24:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.641 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.642 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.644 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.645 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.646 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.647 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2024-09-10T08:24:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2024-09-10T08:24:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.648 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2024-09-10T08:24:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T08:24:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.649 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2024-09-10T08:24:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2024-09-10T08:24:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.650 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.651 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2024-09-10T08:24:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2024-09-10T08:24:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2024-09-10T08:24:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.652 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.653 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-09-10T08:24:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-09-10T08:24:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.654 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.655 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.656 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.657 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.658 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.659 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.660 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.661 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-09-10T08:24:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-09-10T08:24:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.662 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-09-10T08:24:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-09-10T08:24:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.663 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2024-09-10T08:24:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.664 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.665 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.668 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.669 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.669 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.669 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.669 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.669 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.669 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.669 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.669 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.669 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.669 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.670 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.670 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.670 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.670 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.670 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.670 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.670 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.670 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.671 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.671 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.671 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2024-09-10T08:24:45.671 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.671 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.671 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.671 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.671 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.671 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.672 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.672 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.672 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.672 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.672 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.672 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.672 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.672 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.672 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.672 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.673 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.673 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.673 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.673 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.673 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.673 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.673 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.673 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.674 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.674 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.674 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.674 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.674 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.674 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.674 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.674 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.674 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.674 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.675 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.675 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.675 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.675 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.675 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.675 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.675 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.675 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.675 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.675 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.676 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.676 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.676 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.676 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.676 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.676 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.676 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.676 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.676 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.677 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.677 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.677 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.677 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.677 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.677 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.677 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.677 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.677 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.677 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.678 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2024-09-10T08:24:45.678 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2024-09-10T08:24:45.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.678 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.678 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2024-09-10T08:24:45.678 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.678 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.678 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.678 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T08:24:45.679 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.679 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.679 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.679 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.679 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2024-09-10T08:24:45.679 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.679 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.679 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.680 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.680 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.680 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.680 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.680 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.680 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.680 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.680 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.680 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.680 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.680 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.680 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.681 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.681 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.681 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.681 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.681 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.681 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.681 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.681 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.681 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.681 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.681 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.682 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.682 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.682 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.682 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.682 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.682 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.682 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.682 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.682 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.682 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.683 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.683 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.683 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.683 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.683 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.683 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.683 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.683 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.683 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.683 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.683 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.684 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.684 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.684 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.684 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.684 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.684 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.684 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.684 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.685 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.685 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.685 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.685 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.685 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.685 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.685 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.685 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.685 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2024-09-10T08:24:45.685 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2024-09-10T08:24:45.686 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.686 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.686 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.686 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.686 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.686 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.686 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.686 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.686 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.686 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.686 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.687 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2024-09-10T08:24:45.687 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2024-09-10T08:24:45.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.687 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.687 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.687 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2024-09-10T08:24:45.687 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.687 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.687 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.687 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.687 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.688 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.688 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2024-09-10T08:24:45.688 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2024-09-10T08:24:45.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.688 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.688 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.688 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.688 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.688 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.689 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.689 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.689 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.689 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.689 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2024-09-10T08:24:45.689 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2024-09-10T08:24:45.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.689 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.689 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.690 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.690 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.690 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.690 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.690 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.690 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.690 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.690 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.690 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2024-09-10T08:24:45.691 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2024-09-10T08:24:45.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:45.691 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.691 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.691 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.691 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.691 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.691 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.691 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.692 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.692 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2024-09-10T08:24:45.692 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2024-09-10T08:24:45.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.692 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.692 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.692 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.692 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.692 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.693 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.693 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.693 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.693 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.693 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.693 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2024-09-10T08:24:45.693 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2024-09-10T08:24:45.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.693 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.693 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.694 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.694 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.694 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.694 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.694 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.694 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.694 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.694 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.694 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2024-09-10T08:24:45.694 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2024-09-10T08:24:45.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.695 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.695 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.695 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.695 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.695 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.695 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.695 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.695 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.696 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2024-09-10T08:24:45.696 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2024-09-10T08:24:45.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.696 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.696 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.696 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-09-10T08:24:45.696 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2024-09-10T08:24:45.696 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.696 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.696 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.697 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.697 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.697 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2024-09-10T08:24:45.697 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2024-09-10T08:24:45.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.697 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.697 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.697 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.697 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.698 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.698 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.698 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.698 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.698 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.698 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2024-09-10T08:24:45.698 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2024-09-10T08:24:45.698 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.698 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.699 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.699 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.699 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.699 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.699 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.699 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.699 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.699 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2024-09-10T08:24:45.700 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2024-09-10T08:24:45.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.700 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.700 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.700 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.700 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.700 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.700 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.700 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.701 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.701 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-09-10T08:24:45.701 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.701 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.701 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.701 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2024-09-10T08:24:45.701 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2024-09-10T08:24:45.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.702 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.702 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.702 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.702 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.702 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.702 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T08:24:45.702 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.702 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.702 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.703 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2024-09-10T08:24:45.703 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2024-09-10T08:24:45.703 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.703 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.703 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.703 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.703 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.703 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.703 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.703 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2024-09-10T08:24:45.704 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.704 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.704 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.704 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.704 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.704 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.704 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.704 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.704 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.705 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.705 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.705 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.705 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.705 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.705 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.705 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.705 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.705 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.705 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.706 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.706 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.706 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.706 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.706 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.706 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.706 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.706 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.706 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.706 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.707 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.707 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.707 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.707 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.707 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.707 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.708 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.708 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.708 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.708 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.708 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.708 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.708 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.708 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.708 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.709 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.709 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.709 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.709 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.709 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.709 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.709 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.709 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.709 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.710 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.710 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.710 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.710 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.710 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.710 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.710 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.710 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.710 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.711 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.711 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2024-09-10T08:24:45.711 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.711 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.711 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.711 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.711 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.712 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.712 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.712 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.712 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.712 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.712 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.712 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.712 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.712 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.713 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.713 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.713 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.713 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.713 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.713 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.713 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.713 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.713 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.714 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.714 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.714 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.714 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.714 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.714 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.714 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.714 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.714 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.715 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.715 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.715 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.715 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.715 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.715 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.715 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.715 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.715 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.716 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.716 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.716 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.716 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.716 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.716 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.716 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.716 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.716 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.716 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.717 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.717 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.717 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.717 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.717 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.717 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.717 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.717 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.717 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.718 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.718 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.718 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.718 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2024-09-10T08:24:45.718 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.718 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.718 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.719 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.719 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.719 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.719 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.719 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.719 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.719 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.719 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.719 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.720 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.720 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.720 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.720 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.720 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.720 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.720 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.720 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.720 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.721 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.721 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.721 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.721 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.721 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.721 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.721 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.721 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.721 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.721 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.722 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.722 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.722 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.722 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.722 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.722 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.722 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.722 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.722 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.723 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.723 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.723 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.723 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.723 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.723 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.723 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.723 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.723 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.723 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.723 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.724 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.724 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.724 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.724 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.724 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.724 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.724 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.724 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.725 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.725 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.725 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2024-09-10T08:24:45.725 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.725 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.725 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.725 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2024-09-10T08:24:45.726 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2024-09-10T08:24:45.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.726 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.726 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.726 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T08:24:45.726 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.726 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.726 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.727 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.727 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.727 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.727 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:45.727 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:45.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.727 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.727 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.727 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2024-09-10T08:24:45.727 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.728 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.728 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.728 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.728 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.728 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.728 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.728 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.728 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.728 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.728 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.729 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.729 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.729 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.729 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.729 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.729 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.729 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.729 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.729 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.729 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.729 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.729 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.730 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.730 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.730 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.730 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.730 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.730 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.730 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.730 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.730 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.730 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.731 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.731 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.731 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.731 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.731 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.731 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.731 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.731 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.731 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.731 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.731 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.732 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.732 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.732 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.732 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.732 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.732 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.732 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.732 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.732 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.732 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.732 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.732 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.733 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.733 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.733 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.733 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.733 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.733 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.733 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.733 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.734 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2024-09-10T08:24:45.734 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.734 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.734 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.734 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2024-09-10T08:24:45.734 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2024-09-10T08:24:45.734 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.734 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.734 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.734 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.734 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.735 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.736 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.736 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.736 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T08:24:45.736 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.736 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.736 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.736 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2024-09-10T08:24:45.736 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2024-09-10T08:24:45.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.736 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.736 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.737 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.737 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.737 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.737 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T08:24:45.737 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.737 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.737 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.737 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2024-09-10T08:24:45.737 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2024-09-10T08:24:45.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.737 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.737 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.738 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.740 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.740 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.740 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.740 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.740 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.740 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.740 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.740 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.740 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2024-09-10T08:24:45.740 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2024-09-10T08:24:45.740 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.741 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.741 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.741 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.741 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.741 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.741 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.741 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.741 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.741 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.741 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.741 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2024-09-10T08:24:45.741 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2024-09-10T08:24:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T08:24:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.742 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-09-10T08:24:45.743 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-09-10T08:24:45.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.743 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.743 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.743 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.743 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.743 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.743 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.743 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.743 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.743 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.744 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:45.744 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:45.744 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.744 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.744 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.744 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2024-09-10T08:24:45.744 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2024-09-10T08:24:45.744 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.744 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.744 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.744 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.745 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.745 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2024-09-10T08:24:45.745 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2024-09-10T08:24:45.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.745 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.745 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2024-09-10T08:24:45.745 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.745 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.745 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.745 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.745 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.746 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.747 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.747 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2024-09-10T08:24:45.747 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2024-09-10T08:24:45.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.747 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.747 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.747 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.747 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.747 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.747 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.748 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.748 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.748 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.748 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.748 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.748 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.748 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.748 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.748 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.749 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.749 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.749 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.749 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.749 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.749 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.749 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.749 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.749 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.749 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.749 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.750 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.750 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.750 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.750 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.750 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.750 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.750 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.750 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.750 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.750 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.750 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.750 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.751 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.752 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.752 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.752 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.752 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.752 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.752 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.752 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.752 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.752 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.752 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.752 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.752 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.753 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.753 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.753 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.753 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.753 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.753 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.753 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.753 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.753 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2024-09-10T08:24:45.753 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2024-09-10T08:24:45.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.753 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.754 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.755 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.755 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.755 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.755 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.755 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.755 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.755 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.755 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.755 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2024-09-10T08:24:45.755 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2024-09-10T08:24:45.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.756 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.756 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.756 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T08:24:45.756 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.756 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.756 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.756 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.756 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.756 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.756 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.756 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.756 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.757 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2024-09-10T08:24:45.757 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.757 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.757 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.757 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.757 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.757 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.757 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.757 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.757 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.758 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.758 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.758 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.758 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.758 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.758 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.758 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.758 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.758 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.758 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.758 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.758 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.758 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.759 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.759 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.759 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.759 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.759 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.759 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.759 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.759 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.759 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.760 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.760 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.760 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.760 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.760 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.760 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.760 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.760 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.760 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.761 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.761 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.761 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.761 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.761 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.761 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.761 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.761 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.761 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.761 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.761 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.762 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.762 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.762 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.762 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.762 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.762 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.762 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.762 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.762 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.762 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.762 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.762 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.763 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.763 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-09-10T08:24:45.763 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.763 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.763 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.763 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.763 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.764 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.764 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.764 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.764 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.764 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.764 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.764 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.764 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.764 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.764 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.764 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.764 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.765 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.766 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.766 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.766 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.766 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.766 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.766 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.766 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.766 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.766 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.766 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.767 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.768 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.768 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.768 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.768 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.768 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.768 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.768 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.768 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.768 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.768 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.768 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.769 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2024-09-10T08:24:45.769 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2024-09-10T08:24:45.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.769 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.769 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2024-09-10T08:24:45.769 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.769 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.769 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.769 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T08:24:45.769 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.769 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.769 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.770 INFO:teuthology.orchestra.run.smithi002.stdout: "periodic_async_work": { 2024-09-10T08:24:45.770 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "periodic_async_work", 2024-09-10T08:24:45.770 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.770 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.770 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.770 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.770 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.770 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.770 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.770 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Periodically check for async work", 2024-09-10T08:24:45.770 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.771 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.771 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.771 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2024-09-10T08:24:45.771 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2024-09-10T08:24:45.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.771 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.771 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.771 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:45.771 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.771 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.771 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.771 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T08:24:45.772 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.772 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.772 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.772 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_no_wait": { 2024-09-10T08:24:45.772 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T08:24:45.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.772 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.772 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.772 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.772 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.773 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.773 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T08:24:45.773 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.773 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.773 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2024-09-10T08:24:45.773 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.773 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.773 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.774 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2024-09-10T08:24:45.774 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2024-09-10T08:24:45.774 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:45.774 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.774 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.774 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2024-09-10T08:24:45.774 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.774 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.774 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.774 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.774 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.774 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.775 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.776 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:45.776 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:45.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.776 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.776 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:45.776 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.776 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.776 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.776 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.776 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.776 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.777 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.777 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.777 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.777 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.777 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.777 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.777 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.777 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.777 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.777 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.777 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.777 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.778 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.778 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.778 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.778 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.778 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.778 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.778 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.778 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.778 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.778 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.778 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.779 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.779 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.779 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.779 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.779 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.779 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.779 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.779 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.779 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.779 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.779 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.780 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.780 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.780 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.780 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.780 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.780 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.780 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.780 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.780 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.780 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.780 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.780 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.781 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.781 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.781 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.781 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.781 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.781 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.781 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.781 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.781 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.781 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.781 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.781 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.782 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.783 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.783 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.783 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.783 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.783 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.783 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2024-09-10T08:24:45.783 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2024-09-10T08:24:45.783 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.783 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.783 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2024-09-10T08:24:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2024-09-10T08:24:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2024-09-10T08:24:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T08:24:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "gid": 24113, 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "y", 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_features": 4540701547738038271, 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T08:24:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2024-09-10T08:24:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2024-09-10T08:24:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2024-09-10T08:24:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2024-09-10T08:24:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2024-09-10T08:24:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2024-09-10T08:24:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2024-09-10T08:24:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2024-09-10T08:24:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2024-09-10T08:24:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2024-09-10T08:24:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2024-09-10T08:24:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2024-09-10T08:24:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2024-09-10T08:24:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2024-09-10T08:24:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2024-09-10T08:24:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2024-09-10T08:24:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2024-09-10T08:24:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2024-09-10T08:24:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2024-09-10T08:24:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T08:24:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2024-09-10T08:24:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2024-09-10T08:24:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2024-09-10T08:24:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2024-09-10T08:24:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2024-09-10T08:24:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2024-09-10T08:24:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T08:24:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T08:24:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2024-09-10T08:24:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2024-09-10T08:24:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-09-10T08:24:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-09-10T08:24:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T08:24:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T08:24:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2024-09-10T08:24:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2024-09-10T08:24:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2024-09-10T08:24:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-09-10T08:24:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2024-09-10T08:24:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T08:24:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2024-09-10T08:24:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2024-09-10T08:24:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T08:24:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T08:24:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T08:24:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2024-09-10T08:24:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2024-09-10T08:24:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2024-09-10T08:24:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2024-09-10T08:24:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2024-09-10T08:24:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2024-09-10T08:24:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T08:24:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2024-09-10T08:24:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2024-09-10T08:24:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2024-09-10T08:24:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T08:24:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T08:24:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2024-09-10T08:24:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2024-09-10T08:24:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-09-10T08:24:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-09-10T08:24:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T08:24:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T08:24:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2024-09-10T08:24:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2024-09-10T08:24:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T08:24:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-09-10T08:24:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-09-10T08:24:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2024-09-10T08:24:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2024-09-10T08:24:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2024-09-10T08:24:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "read", 2024-09-10T08:24:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap", 2024-09-10T08:24:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap-read" 2024-09-10T08:24:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2024-09-10T08:24:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2024-09-10T08:24:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2024-09-10T08:24:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T08:24:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T08:24:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2024-09-10T08:24:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2024-09-10T08:24:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-09-10T08:24:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-09-10T08:24:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T08:24:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T08:24:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2024-09-10T08:24:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2024-09-10T08:24:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T08:24:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2024-09-10T08:24:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2024-09-10T08:24:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2024-09-10T08:24:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2024-09-10T08:24:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2024-09-10T08:24:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T08:24:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2024-09-10T08:24:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2024-09-10T08:24:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2024-09-10T08:24:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T08:24:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2024-09-10T08:24:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2024-09-10T08:24:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T08:24:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T08:24:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2024-09-10T08:24:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2024-09-10T08:24:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T08:24:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2024-09-10T08:24:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T08:24:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2024-09-10T08:24:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T08:24:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_log_destination": { 2024-09-10T08:24:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm_log_destination", 2024-09-10T08:24:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "file", 2024-09-10T08:24:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "file,syslog", 2024-09-10T08:24:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "syslog" 2024-09-10T08:24:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T08:24:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2024-09-10T08:24:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2024-09-10T08:24:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T08:24:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2024-09-10T08:24:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2024-09-10T08:24:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T08:24:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2024-09-10T08:24:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2024-09-10T08:24:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T08:24:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2024-09-10T08:24:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2024-09-10T08:24:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T08:24:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2024-09-10T08:24:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2024-09-10T08:24:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2024-09-10T08:24:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T08:24:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2024-09-10T08:24:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2024-09-10T08:24:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2024-09-10T08:24:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T08:24:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2024-09-10T08:24:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2024-09-10T08:24:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2024-09-10T08:24:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T08:24:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2024-09-10T08:24:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2024-09-10T08:24:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2024-09-10T08:24:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T08:24:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2024-09-10T08:24:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2024-09-10T08:24:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2024-09-10T08:24:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T08:24:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2024-09-10T08:24:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2024-09-10T08:24:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2024-09-10T08:24:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T08:24:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2024-09-10T08:24:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2024-09-10T08:24:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2024-09-10T08:24:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T08:24:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-09-10T08:24:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2024-09-10T08:24:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2024-09-10T08:24:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T08:24:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-09-10T08:24:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2024-09-10T08:24:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2024-09-10T08:24:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T08:24:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2024-09-10T08:24:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_samba": { 2024-09-10T08:24:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_samba", 2024-09-10T08:24:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T08:24:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Samba/SMB container image", 2024-09-10T08:24:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2024-09-10T08:24:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2024-09-10T08:24:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T08:24:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2024-09-10T08:24:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2024-09-10T08:24:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2024-09-10T08:24:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T08:24:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2024-09-10T08:24:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2024-09-10T08:24:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T08:24:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2024-09-10T08:24:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T08:24:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2024-09-10T08:24:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T08:24:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2024-09-10T08:24:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2024-09-10T08:24:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2024-09-10T08:24:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T08:24:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2024-09-10T08:24:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2024-09-10T08:24:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2024-09-10T08:24:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2024-09-10T08:24:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2024-09-10T08:24:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2024-09-10T08:24:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T08:24:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2024-09-10T08:24:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2024-09-10T08:24:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "grafana_dashboards_path": { 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "grafana_dashboards_path", 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T08:24:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T08:24:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2024-09-10T08:24:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2024-09-10T08:24:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2024-09-10T08:24:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "hw_monitoring": { 2024-09-10T08:24:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hw_monitoring", 2024-09-10T08:24:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T08:24:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2024-09-10T08:24:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2024-09-10T08:24:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T08:24:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2024-09-10T08:24:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2024-09-10T08:24:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T08:24:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T08:24:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T08:24:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T08:24:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T08:24:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T08:24:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T08:24:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2024-09-10T08:24:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2024-09-10T08:24:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2024-09-10T08:24:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2024-09-10T08:24:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2024-09-10T08:24:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-09-10T08:24:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-09-10T08:24:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "oob_default_addr": { 2024-09-10T08:24:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "oob_default_addr", 2024-09-10T08:24:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "169.254.1.1", 2024-09-10T08:24:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T08:24:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2024-09-10T08:24:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2024-09-10T08:24:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T08:24:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2024-09-10T08:24:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2024-09-10T08:24:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T08:24:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2024-09-10T08:24:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2024-09-10T08:24:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T08:24:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2024-09-10T08:24:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2024-09-10T08:24:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T08:24:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2024-09-10T08:24:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2024-09-10T08:24:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2024-09-10T08:24:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2024-09-10T08:24:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2024-09-10T08:24:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2024-09-10T08:24:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T08:24:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2024-09-10T08:24:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2024-09-10T08:24:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T08:24:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2024-09-10T08:24:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2024-09-10T08:24:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T08:24:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2024-09-10T08:24:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2024-09-10T08:24:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T08:24:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2024-09-10T08:24:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2024-09-10T08:24:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2024-09-10T08:24:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2024-09-10T08:24:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2024-09-10T08:24:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T08:24:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2024-09-10T08:24:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2024-09-10T08:24:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2024-09-10T08:24:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T08:24:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T08:24:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T08:24:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2024-09-10T08:24:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T08:24:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T08:24:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T08:24:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T08:24:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T08:24:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T08:24:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T08:24:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T08:24:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T08:24:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T08:24:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T08:24:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T08:24:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T08:24:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T08:24:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T08:24:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T08:24:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T08:24:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T08:24:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T08:24:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T08:24:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T08:24:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T08:24:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T08:24:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-09-10T08:24:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T08:24:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T08:24:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2024-09-10T08:24:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2024-09-10T08:24:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T08:24:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T08:24:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-09-10T08:24:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T08:24:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T08:24:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T08:24:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T08:24:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T08:24:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T08:24:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T08:24:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T08:24:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T08:24:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T08:24:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T08:24:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T08:24:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T08:24:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T08:24:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T08:24:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T08:24:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T08:24:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T08:24:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T08:24:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T08:24:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T08:24:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T08:24:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T08:24:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T08:24:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T08:24:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T08:24:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T08:24:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T08:24:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T08:24:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T08:24:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T08:24:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T08:24:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T08:24:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T08:24:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T08:24:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2024-09-10T08:24:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T08:24:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T08:24:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2024-09-10T08:24:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T08:24:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T08:24:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T08:24:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T08:24:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-09-10T08:24:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T08:24:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T08:24:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T08:24:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T08:24:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T08:24:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T08:24:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T08:24:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T08:24:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T08:24:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T08:24:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-09-10T08:24:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2024-09-10T08:24:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2024-09-10T08:24:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2024-09-10T08:24:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2024-09-10T08:24:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2024-09-10T08:24:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2024-09-10T08:24:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2024-09-10T08:24:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2024-09-10T08:24:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2024-09-10T08:24:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2024-09-10T08:24:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-09-10T08:24:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-09-10T08:24:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2024-09-10T08:24:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2024-09-10T08:24:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2024-09-10T08:24:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2024-09-10T08:24:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T08:24:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-09-10T08:24:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-09-10T08:24:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-09-10T08:24:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-09-10T08:24:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-09-10T08:24:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2024-09-10T08:24:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-09-10T08:24:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-09-10T08:24:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2024-09-10T08:24:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2024-09-10T08:24:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2024-09-10T08:24:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-09-10T08:24:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-09-10T08:24:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2024-09-10T08:24:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2024-09-10T08:24:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-09-10T08:24:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-09-10T08:24:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-09-10T08:24:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-09-10T08:24:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-09-10T08:24:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2024-09-10T08:24:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2024-09-10T08:24:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-09-10T08:24:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2024-09-10T08:24:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2024-09-10T08:24:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2024-09-10T08:24:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2024-09-10T08:24:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2024-09-10T08:24:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2024-09-10T08:24:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T08:24:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2024-09-10T08:24:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2024-09-10T08:24:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2024-09-10T08:24:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T08:24:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2024-09-10T08:24:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2024-09-10T08:24:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2024-09-10T08:24:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2024-09-10T08:24:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2024-09-10T08:24:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2024-09-10T08:24:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-09-10T08:24:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T08:24:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2024-09-10T08:24:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2024-09-10T08:24:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T08:24:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T08:24:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2024-09-10T08:24:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2024-09-10T08:24:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2024-09-10T08:24:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T08:24:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2024-09-10T08:24:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2024-09-10T08:24:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2024-09-10T08:24:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-09-10T08:24:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2024-09-10T08:24:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2024-09-10T08:24:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2024-09-10T08:24:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2024-09-10T08:24:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2024-09-10T08:24:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2024-09-10T08:24:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2024-09-10T08:24:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2024-09-10T08:24:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2024-09-10T08:24:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T08:24:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2024-09-10T08:24:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2024-09-10T08:24:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2024-09-10T08:24:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T08:24:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2024-09-10T08:24:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2024-09-10T08:24:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2024-09-10T08:24:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2024-09-10T08:24:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2024-09-10T08:24:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T08:24:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2024-09-10T08:24:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2024-09-10T08:24:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2024-09-10T08:24:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2024-09-10T08:24:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2024-09-10T08:24:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2024-09-10T08:24:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2024-09-10T08:24:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-09-10T08:24:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-09-10T08:24:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2024-09-10T08:24:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T08:24:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2024-09-10T08:24:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2024-09-10T08:24:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-09-10T08:24:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-09-10T08:24:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2024-09-10T08:24:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T08:24:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2024-09-10T08:24:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2024-09-10T08:24:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2024-09-10T08:24:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2024-09-10T08:24:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2024-09-10T08:24:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2024-09-10T08:24:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T08:24:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2024-09-10T08:24:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2024-09-10T08:24:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2024-09-10T08:24:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2024-09-10T08:24:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2024-09-10T08:24:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2024-09-10T08:24:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T08:24:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2024-09-10T08:24:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2024-09-10T08:24:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2024-09-10T08:24:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T08:24:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2024-09-10T08:24:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2024-09-10T08:24:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2024-09-10T08:24:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2024-09-10T08:24:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2024-09-10T08:24:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T08:24:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2024-09-10T08:24:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2024-09-10T08:24:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2024-09-10T08:24:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T08:24:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2024-09-10T08:24:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2024-09-10T08:24:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2024-09-10T08:24:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2024-09-10T08:24:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2024-09-10T08:24:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2024-09-10T08:24:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2024-09-10T08:24:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T08:24:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2024-09-10T08:24:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2024-09-10T08:24:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2024-09-10T08:24:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2024-09-10T08:24:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2024-09-10T08:24:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2024-09-10T08:24:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T08:24:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T08:24:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2024-09-10T08:24:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2024-09-10T08:24:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2024-09-10T08:24:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-09-10T08:24:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2024-09-10T08:24:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2024-09-10T08:24:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T08:24:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2024-09-10T08:24:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-09-10T08:24:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-09-10T08:24:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2024-09-10T08:24:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2024-09-10T08:24:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2024-09-10T08:24:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2024-09-10T08:24:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2024-09-10T08:24:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2024-09-10T08:24:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2024-09-10T08:24:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2024-09-10T08:24:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-09-10T08:24:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2024-09-10T08:24:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2024-09-10T08:24:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T08:24:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2024-09-10T08:24:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2024-09-10T08:24:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2024-09-10T08:24:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T08:24:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-09-10T08:24:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-09-10T08:24:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2024-09-10T08:24:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2024-09-10T08:24:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2024-09-10T08:24:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2024-09-10T08:24:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-09-10T08:24:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2024-09-10T08:24:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2024-09-10T08:24:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2024-09-10T08:24:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2024-09-10T08:24:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2024-09-10T08:24:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2024-09-10T08:24:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T08:24:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T08:24:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2024-09-10T08:24:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2024-09-10T08:24:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T08:24:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T08:24:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2024-09-10T08:24:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2024-09-10T08:24:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2024-09-10T08:24:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2024-09-10T08:24:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-09-10T08:24:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-09-10T08:24:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-09-10T08:24:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T08:24:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-09-10T08:24:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-09-10T08:24:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2024-09-10T08:24:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T08:24:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2024-09-10T08:24:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2024-09-10T08:24:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2024-09-10T08:24:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-09-10T08:24:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-09-10T08:24:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2024-09-10T08:24:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2024-09-10T08:24:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2024-09-10T08:24:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-09-10T08:24:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-09-10T08:24:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-09-10T08:24:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-09-10T08:24:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-09-10T08:24:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-09-10T08:24:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2024-09-10T08:24:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2024-09-10T08:24:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2024-09-10T08:24:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T08:24:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2024-09-10T08:24:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2024-09-10T08:24:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2024-09-10T08:24:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2024-09-10T08:24:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2024-09-10T08:24:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-09-10T08:24:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-09-10T08:24:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-09-10T08:24:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-09-10T08:24:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-09-10T08:24:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-09-10T08:24:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2024-09-10T08:24:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2024-09-10T08:24:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2024-09-10T08:24:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2024-09-10T08:24:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2024-09-10T08:24:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T08:24:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2024-09-10T08:24:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2024-09-10T08:24:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2024-09-10T08:24:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2024-09-10T08:24:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2024-09-10T08:24:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2024-09-10T08:24:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2024-09-10T08:24:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2024-09-10T08:24:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2024-09-10T08:24:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2024-09-10T08:24:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2024-09-10T08:24:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2024-09-10T08:24:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2024-09-10T08:24:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2024-09-10T08:24:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2024-09-10T08:24:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2024-09-10T08:24:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2024-09-10T08:24:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2024-09-10T08:24:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2024-09-10T08:24:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2024-09-10T08:24:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-09-10T08:24:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2024-09-10T08:24:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2024-09-10T08:24:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2024-09-10T08:24:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2024-09-10T08:24:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2024-09-10T08:24:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2024-09-10T08:24:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2024-09-10T08:24:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-09-10T08:24:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2024-09-10T08:24:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2024-09-10T08:24:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T08:24:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2024-09-10T08:24:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2024-09-10T08:24:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2024-09-10T08:24:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2024-09-10T08:24:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2024-09-10T08:24:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2024-09-10T08:24:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2024-09-10T08:24:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2024-09-10T08:24:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T08:24:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2024-09-10T08:24:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2024-09-10T08:24:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2024-09-10T08:24:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2024-09-10T08:24:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T08:24:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2024-09-10T08:24:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2024-09-10T08:24:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T08:24:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2024-09-10T08:24:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2024-09-10T08:24:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T08:24:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2024-09-10T08:24:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2024-09-10T08:24:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T08:24:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2024-09-10T08:24:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2024-09-10T08:24:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T08:24:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2024-09-10T08:24:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2024-09-10T08:24:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2024-09-10T08:24:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2024-09-10T08:24:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2024-09-10T08:24:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2024-09-10T08:24:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T08:24:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-09-10T08:24:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-09-10T08:24:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2024-09-10T08:24:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2024-09-10T08:24:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2024-09-10T08:24:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2024-09-10T08:24:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2024-09-10T08:24:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2024-09-10T08:24:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2024-09-10T08:24:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2024-09-10T08:24:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2024-09-10T08:24:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2024-09-10T08:24:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2024-09-10T08:24:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2024-09-10T08:24:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2024-09-10T08:24:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2024-09-10T08:24:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2024-09-10T08:24:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T08:24:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2024-09-10T08:24:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-09-10T08:24:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2024-09-10T08:24:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2024-09-10T08:24:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2024-09-10T08:24:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T08:24:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "periodic_async_work": { 2024-09-10T08:24:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "periodic_async_work", 2024-09-10T08:24:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Periodically check for async work", 2024-09-10T08:24:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2024-09-10T08:24:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2024-09-10T08:24:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T08:24:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_no_wait": { 2024-09-10T08:24:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T08:24:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T08:24:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2024-09-10T08:24:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2024-09-10T08:24:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2024-09-10T08:24:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2024-09-10T08:24:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2024-09-10T08:24:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2024-09-10T08:24:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2024-09-10T08:24:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2024-09-10T08:24:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2024-09-10T08:24:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2024-09-10T08:24:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2024-09-10T08:24:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2024-09-10T08:24:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2024-09-10T08:24:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T08:24:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-10T08:24:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "modules": [ 2024-09-10T08:24:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-09-10T08:24:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "dashboard", 2024-09-10T08:24:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "iostat", 2024-09-10T08:24:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "nfs", 2024-09-10T08:24:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "restful" 2024-09-10T08:24:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2024-09-10T08:24:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2024-09-10T08:24:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T08:24:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2024-09-10T08:24:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2024-09-10T08:24:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2024-09-10T08:24:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2024-09-10T08:24:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2024-09-10T08:24:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2024-09-10T08:24:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2024-09-10T08:24:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2024-09-10T08:24:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2024-09-10T08:24:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2024-09-10T08:24:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2024-09-10T08:24:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2024-09-10T08:24:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2024-09-10T08:24:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2024-09-10T08:24:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2024-09-10T08:24:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2024-09-10T08:24:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2024-09-10T08:24:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2024-09-10T08:24:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2024-09-10T08:24:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2024-09-10T08:24:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2024-09-10T08:24:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2024-09-10T08:24:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T08:24:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2024-09-10T08:24:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2024-09-10T08:24:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2024-09-10T08:24:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2024-09-10T08:24:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2024-09-10T08:24:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2024-09-10T08:24:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T08:24:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2024-09-10T08:24:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2024-09-10T08:24:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2024-09-10T08:24:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T08:24:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T08:24:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2024-09-10T08:24:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2024-09-10T08:24:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-09-10T08:24:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-09-10T08:24:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T08:24:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T08:24:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2024-09-10T08:24:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2024-09-10T08:24:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2024-09-10T08:24:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-09-10T08:24:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2024-09-10T08:24:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T08:24:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2024-09-10T08:24:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2024-09-10T08:24:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T08:24:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T08:24:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T08:24:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2024-09-10T08:24:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2024-09-10T08:24:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2024-09-10T08:24:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2024-09-10T08:24:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2024-09-10T08:24:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2024-09-10T08:24:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T08:24:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2024-09-10T08:24:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2024-09-10T08:24:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2024-09-10T08:24:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T08:24:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T08:24:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2024-09-10T08:24:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2024-09-10T08:24:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-09-10T08:24:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-09-10T08:24:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T08:24:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T08:24:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2024-09-10T08:24:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2024-09-10T08:24:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T08:24:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-09-10T08:24:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-09-10T08:24:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2024-09-10T08:24:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2024-09-10T08:24:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2024-09-10T08:24:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "read", 2024-09-10T08:24:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap", 2024-09-10T08:24:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap-read" 2024-09-10T08:24:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2024-09-10T08:24:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2024-09-10T08:24:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2024-09-10T08:24:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T08:24:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T08:24:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2024-09-10T08:24:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2024-09-10T08:24:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-09-10T08:24:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-09-10T08:24:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T08:24:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T08:24:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2024-09-10T08:24:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2024-09-10T08:24:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T08:24:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2024-09-10T08:24:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2024-09-10T08:24:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2024-09-10T08:24:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-09-10T08:24:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T08:24:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2024-09-10T08:24:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2024-09-10T08:24:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2024-09-10T08:24:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T08:24:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2024-09-10T08:24:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2024-09-10T08:24:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2024-09-10T08:24:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T08:24:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2024-09-10T08:24:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2024-09-10T08:24:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T08:24:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T08:24:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2024-09-10T08:24:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2024-09-10T08:24:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T08:24:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2024-09-10T08:24:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T08:24:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2024-09-10T08:24:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T08:24:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_log_destination": { 2024-09-10T08:24:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm_log_destination", 2024-09-10T08:24:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "file", 2024-09-10T08:24:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "file,syslog", 2024-09-10T08:24:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "syslog" 2024-09-10T08:24:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T08:24:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2024-09-10T08:24:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2024-09-10T08:24:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T08:24:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2024-09-10T08:24:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2024-09-10T08:24:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T08:24:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2024-09-10T08:24:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2024-09-10T08:24:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T08:24:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2024-09-10T08:24:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2024-09-10T08:24:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T08:24:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-09-10T08:24:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2024-09-10T08:24:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2024-09-10T08:24:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T08:24:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2024-09-10T08:24:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2024-09-10T08:24:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2024-09-10T08:24:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T08:24:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2024-09-10T08:24:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2024-09-10T08:24:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2024-09-10T08:24:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T08:24:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-09-10T08:24:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2024-09-10T08:24:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2024-09-10T08:24:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T08:24:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2024-09-10T08:24:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2024-09-10T08:24:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2024-09-10T08:24:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T08:24:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2024-09-10T08:24:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2024-09-10T08:24:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2024-09-10T08:24:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T08:24:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2024-09-10T08:24:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2024-09-10T08:24:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2024-09-10T08:24:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T08:24:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2024-09-10T08:24:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2024-09-10T08:24:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2024-09-10T08:24:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T08:24:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2024-09-10T08:24:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2024-09-10T08:24:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2024-09-10T08:24:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T08:24:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2024-09-10T08:24:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2024-09-10T08:24:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2024-09-10T08:24:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T08:24:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-09-10T08:24:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2024-09-10T08:24:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2024-09-10T08:24:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T08:24:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2024-09-10T08:24:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2024-09-10T08:24:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2024-09-10T08:24:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T08:24:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-09-10T08:24:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2024-09-10T08:24:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2024-09-10T08:24:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T08:24:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2024-09-10T08:24:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_samba": { 2024-09-10T08:24:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_samba", 2024-09-10T08:24:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T08:24:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Samba/SMB container image", 2024-09-10T08:24:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2024-09-10T08:24:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2024-09-10T08:24:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T08:24:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2024-09-10T08:24:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2024-09-10T08:24:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2024-09-10T08:24:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T08:24:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2024-09-10T08:24:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2024-09-10T08:24:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T08:24:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2024-09-10T08:24:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T08:24:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2024-09-10T08:24:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T08:24:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2024-09-10T08:24:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2024-09-10T08:24:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2024-09-10T08:24:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T08:24:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2024-09-10T08:24:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2024-09-10T08:24:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2024-09-10T08:24:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2024-09-10T08:24:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2024-09-10T08:24:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2024-09-10T08:24:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T08:24:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2024-09-10T08:24:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2024-09-10T08:24:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2024-09-10T08:24:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "grafana_dashboards_path": { 2024-09-10T08:24:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "grafana_dashboards_path", 2024-09-10T08:24:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T08:24:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T08:24:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2024-09-10T08:24:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2024-09-10T08:24:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2024-09-10T08:24:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "hw_monitoring": { 2024-09-10T08:24:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hw_monitoring", 2024-09-10T08:24:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T08:24:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2024-09-10T08:24:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2024-09-10T08:24:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T08:24:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2024-09-10T08:24:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2024-09-10T08:24:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T08:24:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T08:24:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T08:24:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T08:24:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T08:24:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T08:24:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T08:24:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2024-09-10T08:24:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T08:24:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2024-09-10T08:24:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2024-09-10T08:24:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2024-09-10T08:24:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2024-09-10T08:24:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2024-09-10T08:24:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T08:24:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2024-09-10T08:24:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2024-09-10T08:24:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2024-09-10T08:24:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-09-10T08:24:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-09-10T08:24:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2024-09-10T08:24:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2024-09-10T08:24:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2024-09-10T08:24:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T08:24:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "oob_default_addr": { 2024-09-10T08:24:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "oob_default_addr", 2024-09-10T08:24:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "169.254.1.1", 2024-09-10T08:24:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T08:24:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2024-09-10T08:24:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2024-09-10T08:24:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T08:24:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T08:24:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2024-09-10T08:24:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2024-09-10T08:24:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T08:24:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2024-09-10T08:24:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2024-09-10T08:24:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T08:24:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2024-09-10T08:24:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2024-09-10T08:24:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T08:24:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2024-09-10T08:24:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2024-09-10T08:24:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T08:24:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2024-09-10T08:24:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2024-09-10T08:24:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T08:24:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2024-09-10T08:24:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2024-09-10T08:24:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2024-09-10T08:24:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2024-09-10T08:24:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2024-09-10T08:24:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2024-09-10T08:24:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T08:24:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2024-09-10T08:24:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2024-09-10T08:24:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T08:24:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2024-09-10T08:24:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2024-09-10T08:24:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T08:24:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2024-09-10T08:24:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2024-09-10T08:24:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T08:24:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2024-09-10T08:24:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2024-09-10T08:24:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T08:24:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2024-09-10T08:24:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2024-09-10T08:24:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T08:24:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2024-09-10T08:24:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2024-09-10T08:24:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2024-09-10T08:24:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2024-09-10T08:24:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T08:24:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2024-09-10T08:24:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2024-09-10T08:24:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2024-09-10T08:24:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T08:24:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2024-09-10T08:24:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T08:24:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T08:24:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T08:24:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T08:24:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T08:24:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T08:24:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2024-09-10T08:24:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T08:24:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T08:24:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T08:24:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T08:24:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T08:24:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T08:24:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T08:24:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T08:24:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T08:24:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T08:24:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T08:24:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T08:24:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T08:24:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T08:24:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T08:24:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T08:24:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T08:24:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T08:24:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T08:24:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T08:24:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T08:24:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T08:24:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T08:24:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-09-10T08:24:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T08:24:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T08:24:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T08:24:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T08:24:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-09-10T08:24:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T08:24:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T08:24:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T08:24:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T08:24:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T08:24:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T08:24:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T08:24:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T08:24:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T08:24:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T08:24:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T08:24:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T08:24:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T08:24:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T08:24:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T08:24:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T08:24:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T08:24:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T08:24:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T08:24:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T08:24:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T08:24:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T08:24:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T08:24:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T08:24:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T08:24:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T08:24:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T08:24:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T08:24:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T08:24:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T08:24:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T08:24:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T08:24:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T08:24:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T08:24:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T08:24:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2024-09-10T08:24:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T08:24:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T08:24:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2024-09-10T08:24:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T08:24:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T08:24:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T08:24:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T08:24:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-09-10T08:24:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T08:24:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T08:24:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T08:24:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T08:24:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T08:24:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T08:24:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T08:24:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T08:24:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T08:24:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T08:24:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-09-10T08:24:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2024-09-10T08:24:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2024-09-10T08:24:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2024-09-10T08:24:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2024-09-10T08:24:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2024-09-10T08:24:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2024-09-10T08:24:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2024-09-10T08:24:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2024-09-10T08:24:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2024-09-10T08:24:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2024-09-10T08:24:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-09-10T08:24:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-09-10T08:24:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2024-09-10T08:24:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2024-09-10T08:24:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2024-09-10T08:24:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2024-09-10T08:24:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T08:24:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-09-10T08:24:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-09-10T08:24:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-09-10T08:24:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-09-10T08:24:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-09-10T08:24:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2024-09-10T08:24:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-09-10T08:24:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-09-10T08:24:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2024-09-10T08:24:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2024-09-10T08:24:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2024-09-10T08:24:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-09-10T08:24:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-09-10T08:24:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2024-09-10T08:24:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2024-09-10T08:24:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-09-10T08:24:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-09-10T08:24:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-09-10T08:24:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-09-10T08:24:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-09-10T08:24:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2024-09-10T08:24:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2024-09-10T08:24:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-09-10T08:24:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2024-09-10T08:24:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2024-09-10T08:24:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2024-09-10T08:24:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2024-09-10T08:24:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2024-09-10T08:24:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2024-09-10T08:24:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T08:24:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2024-09-10T08:24:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2024-09-10T08:24:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2024-09-10T08:24:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T08:24:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2024-09-10T08:24:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2024-09-10T08:24:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2024-09-10T08:24:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2024-09-10T08:24:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2024-09-10T08:24:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2024-09-10T08:24:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-09-10T08:24:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T08:24:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2024-09-10T08:24:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2024-09-10T08:24:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T08:24:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T08:24:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2024-09-10T08:24:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2024-09-10T08:24:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2024-09-10T08:24:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T08:24:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2024-09-10T08:24:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2024-09-10T08:24:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2024-09-10T08:24:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-09-10T08:24:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2024-09-10T08:24:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2024-09-10T08:24:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2024-09-10T08:24:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-09-10T08:24:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2024-09-10T08:24:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2024-09-10T08:24:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2024-09-10T08:24:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2024-09-10T08:24:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2024-09-10T08:24:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2024-09-10T08:24:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T08:24:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2024-09-10T08:24:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2024-09-10T08:24:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2024-09-10T08:24:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T08:24:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2024-09-10T08:24:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2024-09-10T08:24:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2024-09-10T08:24:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2024-09-10T08:24:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2024-09-10T08:24:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T08:24:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2024-09-10T08:24:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2024-09-10T08:24:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2024-09-10T08:24:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2024-09-10T08:24:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2024-09-10T08:24:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2024-09-10T08:24:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2024-09-10T08:24:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-09-10T08:24:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-09-10T08:24:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2024-09-10T08:24:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T08:24:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2024-09-10T08:24:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2024-09-10T08:24:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-09-10T08:24:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-09-10T08:24:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2024-09-10T08:24:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T08:24:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2024-09-10T08:24:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2024-09-10T08:24:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2024-09-10T08:24:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2024-09-10T08:24:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2024-09-10T08:24:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2024-09-10T08:24:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T08:24:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2024-09-10T08:24:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2024-09-10T08:24:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2024-09-10T08:24:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2024-09-10T08:24:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2024-09-10T08:24:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2024-09-10T08:24:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T08:24:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2024-09-10T08:24:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2024-09-10T08:24:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2024-09-10T08:24:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T08:24:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2024-09-10T08:24:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2024-09-10T08:24:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2024-09-10T08:24:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2024-09-10T08:24:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2024-09-10T08:24:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T08:24:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2024-09-10T08:24:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2024-09-10T08:24:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2024-09-10T08:24:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T08:24:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2024-09-10T08:24:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2024-09-10T08:24:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2024-09-10T08:24:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2024-09-10T08:24:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2024-09-10T08:24:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2024-09-10T08:24:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2024-09-10T08:24:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T08:24:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2024-09-10T08:24:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2024-09-10T08:24:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2024-09-10T08:24:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2024-09-10T08:24:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2024-09-10T08:24:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2024-09-10T08:24:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T08:24:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T08:24:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2024-09-10T08:24:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2024-09-10T08:24:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2024-09-10T08:24:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-09-10T08:24:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2024-09-10T08:24:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2024-09-10T08:24:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T08:24:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2024-09-10T08:24:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-09-10T08:24:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-09-10T08:24:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2024-09-10T08:24:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2024-09-10T08:24:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2024-09-10T08:24:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2024-09-10T08:24:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2024-09-10T08:24:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2024-09-10T08:24:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2024-09-10T08:24:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2024-09-10T08:24:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-09-10T08:24:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2024-09-10T08:24:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2024-09-10T08:24:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T08:24:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2024-09-10T08:24:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2024-09-10T08:24:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2024-09-10T08:24:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T08:24:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-09-10T08:24:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-09-10T08:24:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2024-09-10T08:24:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2024-09-10T08:24:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2024-09-10T08:24:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2024-09-10T08:24:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-09-10T08:24:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-09-10T08:24:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-09-10T08:24:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2024-09-10T08:24:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2024-09-10T08:24:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2024-09-10T08:24:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2024-09-10T08:24:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2024-09-10T08:24:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2024-09-10T08:24:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T08:24:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T08:24:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2024-09-10T08:24:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2024-09-10T08:24:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T08:24:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T08:24:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2024-09-10T08:24:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2024-09-10T08:24:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2024-09-10T08:24:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2024-09-10T08:24:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-09-10T08:24:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-09-10T08:24:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-09-10T08:24:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T08:24:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-09-10T08:24:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-09-10T08:24:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2024-09-10T08:24:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T08:24:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2024-09-10T08:24:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2024-09-10T08:24:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2024-09-10T08:24:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-09-10T08:24:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-09-10T08:24:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2024-09-10T08:24:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2024-09-10T08:24:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2024-09-10T08:24:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-09-10T08:24:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-09-10T08:24:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-09-10T08:24:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-09-10T08:24:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-09-10T08:24:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-09-10T08:24:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2024-09-10T08:24:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2024-09-10T08:24:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-09-10T08:24:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2024-09-10T08:24:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T08:24:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2024-09-10T08:24:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2024-09-10T08:24:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2024-09-10T08:24:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2024-09-10T08:24:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2024-09-10T08:24:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-09-10T08:24:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-09-10T08:24:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-09-10T08:24:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-09-10T08:24:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-09-10T08:24:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-09-10T08:24:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2024-09-10T08:24:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2024-09-10T08:24:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2024-09-10T08:24:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2024-09-10T08:24:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2024-09-10T08:24:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T08:24:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2024-09-10T08:24:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2024-09-10T08:24:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2024-09-10T08:24:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2024-09-10T08:24:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2024-09-10T08:24:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2024-09-10T08:24:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2024-09-10T08:24:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2024-09-10T08:24:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2024-09-10T08:24:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2024-09-10T08:24:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2024-09-10T08:24:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2024-09-10T08:24:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-09-10T08:24:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2024-09-10T08:24:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2024-09-10T08:24:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2024-09-10T08:24:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2024-09-10T08:24:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2024-09-10T08:24:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2024-09-10T08:24:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2024-09-10T08:24:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2024-09-10T08:24:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-09-10T08:24:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2024-09-10T08:24:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2024-09-10T08:24:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2024-09-10T08:24:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2024-09-10T08:24:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2024-09-10T08:24:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2024-09-10T08:24:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2024-09-10T08:24:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-09-10T08:24:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2024-09-10T08:24:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2024-09-10T08:24:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T08:24:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2024-09-10T08:24:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2024-09-10T08:24:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2024-09-10T08:24:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2024-09-10T08:24:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2024-09-10T08:24:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2024-09-10T08:24:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2024-09-10T08:24:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2024-09-10T08:24:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T08:24:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2024-09-10T08:24:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2024-09-10T08:24:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2024-09-10T08:24:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2024-09-10T08:24:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T08:24:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2024-09-10T08:24:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2024-09-10T08:24:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T08:24:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2024-09-10T08:24:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2024-09-10T08:24:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T08:24:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2024-09-10T08:24:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2024-09-10T08:24:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T08:24:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2024-09-10T08:24:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2024-09-10T08:24:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T08:24:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2024-09-10T08:24:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2024-09-10T08:24:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2024-09-10T08:24:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2024-09-10T08:24:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2024-09-10T08:24:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2024-09-10T08:24:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T08:24:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-09-10T08:24:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-09-10T08:24:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2024-09-10T08:24:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2024-09-10T08:24:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2024-09-10T08:24:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2024-09-10T08:24:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2024-09-10T08:24:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2024-09-10T08:24:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2024-09-10T08:24:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2024-09-10T08:24:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2024-09-10T08:24:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2024-09-10T08:24:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2024-09-10T08:24:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2024-09-10T08:24:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2024-09-10T08:24:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2024-09-10T08:24:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2024-09-10T08:24:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T08:24:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2024-09-10T08:24:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-09-10T08:24:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2024-09-10T08:24:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2024-09-10T08:24:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2024-09-10T08:24:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T08:24:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "periodic_async_work": { 2024-09-10T08:24:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "periodic_async_work", 2024-09-10T08:24:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Periodically check for async work", 2024-09-10T08:24:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2024-09-10T08:24:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2024-09-10T08:24:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-09-10T08:24:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T08:24:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_no_wait": { 2024-09-10T08:24:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T08:24:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-09-10T08:24:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T08:24:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2024-09-10T08:24:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-09-10T08:24:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-09-10T08:24:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-09-10T08:24:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2024-09-10T08:24:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2024-09-10T08:24:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-09-10T08:24:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2024-09-10T08:24:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2024-09-10T08:24:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2024-09-10T08:24:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-09-10T08:24:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-09-10T08:24:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-09-10T08:24:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-09-10T08:24:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-09-10T08:24:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-09-10T08:24:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-09-10T08:24:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-09-10T08:24:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-09-10T08:24:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-09-10T08:24:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-09-10T08:24:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-09-10T08:24:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-09-10T08:24:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-09-10T08:24:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-09-10T08:24:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-09-10T08:24:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-09-10T08:24:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-09-10T08:24:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-09-10T08:24:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-09-10T08:24:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-09-10T08:24:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-10T08:24:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-09-10T08:24:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2024-09-10T08:24:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2024-09-10T08:24:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-09-10T08:24:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2024-09-10T08:24:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2024-09-10T08:24:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-09-10T08:24:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2024-09-10T08:24:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2024-09-10T08:24:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2024-09-10T08:24:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-09-10T08:24:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-09-10T08:24:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-09-10T08:24:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T08:24:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-09-10T08:24:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-09-10T08:24:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-09-10T08:24:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-09-10T08:24:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-09-10T08:24:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-09-10T08:24:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-09-10T08:24:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "services": { 2024-09-10T08:24:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "dashboard": "https://172.21.15.2:8443/" 2024-09-10T08:24:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "always_on_modules": { 2024-09-10T08:24:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "octopus": [ 2024-09-10T08:24:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-09-10T08:24:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-09-10T08:24:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-09-10T08:24:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-09-10T08:24:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-09-10T08:24:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-09-10T08:24:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-09-10T08:24:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-09-10T08:24:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-09-10T08:24:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-09-10T08:24:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "pacific": [ 2024-09-10T08:24:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-09-10T08:24:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-09-10T08:24:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-09-10T08:24:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-09-10T08:24:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-09-10T08:24:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-09-10T08:24:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-09-10T08:24:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-09-10T08:24:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-09-10T08:24:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-09-10T08:24:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "quincy": [ 2024-09-10T08:24:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-09-10T08:24:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-09-10T08:24:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-09-10T08:24:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-09-10T08:24:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-09-10T08:24:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-09-10T08:24:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-09-10T08:24:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-09-10T08:24:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-09-10T08:24:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-09-10T08:24:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "reef": [ 2024-09-10T08:24:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-09-10T08:24:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-09-10T08:24:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-09-10T08:24:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-09-10T08:24:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-09-10T08:24:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-09-10T08:24:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-09-10T08:24:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-09-10T08:24:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-09-10T08:24:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-09-10T08:24:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-10T08:24:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "squid": [ 2024-09-10T08:24:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-09-10T08:24:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-09-10T08:24:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-09-10T08:24:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-09-10T08:24:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-09-10T08:24:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-09-10T08:24:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-09-10T08:24:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-09-10T08:24:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-09-10T08:24:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-09-10T08:24:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-10T08:24:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "last_failure_osd_epoch": 3, 2024-09-10T08:24:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "active_clients": [ 2024-09-10T08:24:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-09-10T08:24:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-10T08:24:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-10T08:24:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-09-10T08:24:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 145571003 2024-09-10T08:24:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-10T08:24:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "libcephsqlite", 2024-09-10T08:24:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-10T08:24:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-10T08:24:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-09-10T08:24:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4119824153 2024-09-10T08:24:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-10T08:24:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-09-10T08:24:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-10T08:24:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-10T08:24:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-09-10T08:24:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1700134961 2024-09-10T08:24:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-10T08:24:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-09-10T08:24:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-10T08:24:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-10T08:24:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-10T08:24:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-09-10T08:24:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1380984487 2024-09-10T08:24:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-10T08:24:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-10T08:24:46.859 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:24:46.861 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:45 smithi107 ceph-mon[35048]: pgmap v885: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-09-10T08:24:46.861 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:45 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1132329164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:24:46.861 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:45 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3502432971' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T08:24:46.861 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:45 smithi155 ceph-mon[35405]: pgmap v885: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-09-10T08:24:46.861 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:45 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1132329164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:24:46.861 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:45 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3502432971' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T08:24:46.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.265+0000 7f03cfe00640 1 -- 172.21.15.2:0/3502432971 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f03b40a2b90 msgr2=0x7f03b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:46.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.265+0000 7f03cfe00640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f03b40a2b90 0x7f03b40a5050 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f03c00095f0 tx=0x7f03c0005f90 comp rx=0 tx=0).stop 2024-09-10T08:24:46.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.265+0000 7f03cfe00640 1 -- 172.21.15.2:0/3502432971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03d00761d0 msgr2=0x7f03d019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:24:46.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.265+0000 7f03cfe00640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03d00761d0 0x7f03d019a3e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f03b8031440 tx=0x7f03b8032ce0 comp rx=0 tx=0).stop 2024-09-10T08:24:46.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.265+0000 7f03cfe00640 1 -- 172.21.15.2:0/3502432971 shutdown_connections 2024-09-10T08:24:46.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.265+0000 7f03cfe00640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0109800 0x7f03d01a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:46.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.265+0000 7f03cfe00640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03d00761d0 0x7f03d019a3e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:46.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.265+0000 7f03cfe00640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f03b40a2b90 0x7f03b40a5050 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:46.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.265+0000 7f03cfe00640 1 --2- 172.21.15.2:0/3502432971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03d0074bf0 0x7f03d0199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:24:46.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.265+0000 7f03cfe00640 1 -- 172.21.15.2:0/3502432971 >> 172.21.15.2:0/3502432971 conn(0x7f03d00fd110 msgr2=0x7f03d0105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:24:46.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.265+0000 7f03cfe00640 1 -- 172.21.15.2:0/3502432971 shutdown_connections 2024-09-10T08:24:46.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:24:45.265+0000 7f03cfe00640 1 -- 172.21.15.2:0/3502432971 wait complete. 2024-09-10T08:24:46.870 DEBUG:tasks.check_counter:Getting stats from a 2024-09-10T08:24:46.870 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-10T08:24:46.871 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:45 smithi002 ceph-mon[30563]: pgmap v885: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-09-10T08:24:46.871 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:45 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1132329164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:24:46.871 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:45 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3502432971' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T08:24:47.258 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:47.927 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:24:47.927 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:24:47.927 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 323, 2024-09-10T08:24:47.927 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 316, 2024-09-10T08:24:47.927 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 785440, 2024-09-10T08:24:47.927 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 194288, 2024-09-10T08:24:47.927 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-09-10T08:24:47.927 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-09-10T08:24:47.927 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.068393951, 2024-09-10T08:24:47.928 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.033839575, 2024-09-10T08:24:47.928 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.037562702, 2024-09-10T08:24:47.928 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:24:47.928 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:47.928 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 316, 2024-09-10T08:24:47.928 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.029076878, 2024-09-10T08:24:47.928 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000092015 2024-09-10T08:24:47.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:47.928 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:47.928 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:47.928 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:47.928 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:47.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:47.929 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 785440, 2024-09-10T08:24:47.929 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 194288 2024-09-10T08:24:47.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:47.929 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:24:47.929 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 2, 2024-09-10T08:24:47.929 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 240, 2024-09-10T08:24:47.929 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 320, 2024-09-10T08:24:47.929 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 199776, 2024-09-10T08:24:47.929 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-09-10T08:24:47.929 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2024-09-10T08:24:47.929 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.027934475, 2024-09-10T08:24:47.929 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.022393752, 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.001979106, 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 238, 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.016755885, 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000070402 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T08:24:47.930 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 199776 2024-09-10T08:24:47.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:47.931 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:24:47.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2024-09-10T08:24:47.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2024-09-10T08:24:47.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2024-09-10T08:24:47.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2024-09-10T08:24:47.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-09-10T08:24:47.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-09-10T08:24:47.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.000540983, 2024-09-10T08:24:47.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T08:24:47.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000405500, 2024-09-10T08:24:47.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:24:47.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:47.932 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:47.932 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:47.932 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:47.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:47.932 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:47.932 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:47.932 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:47.932 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:47.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:47.932 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:24:47.932 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:24:47.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:47.932 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-09-10T08:24:47.933 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-09-10T08:24:47.933 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-09-10T08:24:47.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:47.933 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-09-10T08:24:47.933 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:24:47.933 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-09-10T08:24:47.933 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:24:47.933 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:24:47.933 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:24:47.933 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:24:47.933 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:24:47.933 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:24:47.933 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:24:47.934 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:24:47.934 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:24:47.934 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:24:47.934 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:24:47.934 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:24:47.934 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:24:47.934 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-09-10T08:24:47.934 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:24:47.934 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-09-10T08:24:47.934 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:24:47.934 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:24:47.934 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:24:47.934 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:24:47.935 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:24:47.935 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:24:47.935 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:24:47.935 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-09-10T08:24:47.935 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:24:47.935 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:24:47.935 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:24:47.935 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-09-10T08:24:47.935 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-09-10T08:24:47.935 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-09-10T08:24:47.935 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:24:47.935 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:24:47.936 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:24:47.936 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:24:47.936 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148721, 2024-09-10T08:24:47.936 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 30, 2024-09-10T08:24:47.936 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:24:47.936 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-09-10T08:24:47.936 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-09-10T08:24:47.936 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-09-10T08:24:47.936 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:24:47.936 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-09-10T08:24:47.936 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:24:47.936 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-09-10T08:24:47.936 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-09-10T08:24:47.937 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-09-10T08:24:47.937 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:24:47.937 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:24:47.937 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-09-10T08:24:47.937 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-09-10T08:24:47.937 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-09-10T08:24:47.937 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-09-10T08:24:47.937 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-09-10T08:24:47.937 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-09-10T08:24:47.937 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-09-10T08:24:47.937 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-09-10T08:24:47.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:47.937 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:24:47.938 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-09-10T08:24:47.938 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-09-10T08:24:47.938 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-09-10T08:24:47.938 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 321, 2024-09-10T08:24:47.938 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 737298, 2024-09-10T08:24:47.938 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:47.938 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 321, 2024-09-10T08:24:47.938 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-09-10T08:24:47.938 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-09-10T08:24:47.938 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 321, 2024-09-10T08:24:47.938 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 737298, 2024-09-10T08:24:47.938 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-09-10T08:24:47.938 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:47.939 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:47.939 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:47.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:47.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:47.939 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:24:47.969 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:47 smithi107 ceph-mon[35048]: pgmap v886: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-10T08:24:48.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:47 smithi002 ceph-mon[30563]: pgmap v886: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-10T08:24:48.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:47 smithi155 ceph-mon[35405]: pgmap v886: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-10T08:24:48.787 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-10T08:24:48.788 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-10T08:24:48.788 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-10T08:24:48.788 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-10T08:24:48.788 DEBUG:tasks.check_counter:Getting stats from d 2024-09-10T08:24:48.788 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-10T08:24:49.165 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:49.807 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:24:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:24:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2024-09-10T08:24:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2024-09-10T08:24:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2024-09-10T08:24:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2024-09-10T08:24:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-09-10T08:24:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-09-10T08:24:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.005547105, 2024-09-10T08:24:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T08:24:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000761685, 2024-09-10T08:24:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:24:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:24:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:24:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:24:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 6, 2024-09-10T08:24:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 246, 2024-09-10T08:24:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 3392, 2024-09-10T08:24:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 200928, 2024-09-10T08:24:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-09-10T08:24:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2024-09-10T08:24:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.032156087, 2024-09-10T08:24:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.023441149, 2024-09-10T08:24:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.004980412, 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 245, 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.023536667, 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000096068 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 200928 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 325, 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 320, 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 784000, 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 197680, 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.064185037, 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.034566047, 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.036235339, 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 319, 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.024637569, 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000077233 2024-09-10T08:24:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 784000, 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 197680 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-09-10T08:24:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:24:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-09-10T08:24:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:24:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:24:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:24:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:24:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:24:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:24:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:24:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:24:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:24:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:24:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:24:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:24:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:24:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-09-10T08:24:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-09-10T08:24:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-09-10T08:24:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:24:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:24:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:24:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:24:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148704, 2024-09-10T08:24:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 29, 2024-09-10T08:24:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:24:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-09-10T08:24:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-09-10T08:24:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-09-10T08:24:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:24:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-09-10T08:24:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:24:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-09-10T08:24:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-09-10T08:24:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-09-10T08:24:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:24:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:24:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-09-10T08:24:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-09-10T08:24:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-09-10T08:24:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-09-10T08:24:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-09-10T08:24:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-09-10T08:24:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-09-10T08:24:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-09-10T08:24:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:24:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-09-10T08:24:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-09-10T08:24:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-09-10T08:24:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 327, 2024-09-10T08:24:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 738078, 2024-09-10T08:24:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 327, 2024-09-10T08:24:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-09-10T08:24:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-09-10T08:24:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 327, 2024-09-10T08:24:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 738078, 2024-09-10T08:24:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-09-10T08:24:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:49.820 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:49.820 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:49.820 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:24:50.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:49 smithi002 ceph-mon[30563]: pgmap v887: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:24:50.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:49 smithi155 ceph-mon[35405]: pgmap v887: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:24:50.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:49 smithi107 ceph-mon[35048]: pgmap v887: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:24:50.521 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-10T08:24:50.521 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-10T08:24:50.522 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-10T08:24:50.522 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-10T08:24:50.522 DEBUG:tasks.check_counter:Getting stats from g 2024-09-10T08:24:50.522 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-10T08:24:50.900 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:51.527 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:24:51.527 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:24:51.527 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2024-09-10T08:24:51.527 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2024-09-10T08:24:51.527 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2024-09-10T08:24:51.527 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2024-09-10T08:24:51.527 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-09-10T08:24:51.527 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-09-10T08:24:51.527 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.003557889, 2024-09-10T08:24:51.527 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T08:24:51.528 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000695209, 2024-09-10T08:24:51.528 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:24:51.528 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:51.528 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:51.528 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:51.528 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:51.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:51.528 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:51.528 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:51.528 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:51.528 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:51.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:51.528 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:24:51.529 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:24:51.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:51.529 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:24:51.529 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 2, 2024-09-10T08:24:51.529 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 241, 2024-09-10T08:24:51.529 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 320, 2024-09-10T08:24:51.529 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 199952, 2024-09-10T08:24:51.529 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-09-10T08:24:51.529 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2024-09-10T08:24:51.529 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.028250414, 2024-09-10T08:24:51.529 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.022592847, 2024-09-10T08:24:51.529 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.001927032, 2024-09-10T08:24:51.529 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 239, 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.017269198, 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000072256 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 199952 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:51.530 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 325, 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 318, 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 785984, 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 195568, 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.064998475, 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.034064508, 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.036943478, 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 318, 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.029462821, 2024-09-10T08:24:51.531 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000092650 2024-09-10T08:24:51.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:51.532 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:51.532 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:51.532 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:51.532 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:51.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:51.532 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 785984, 2024-09-10T08:24:51.532 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 195568 2024-09-10T08:24:51.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:51.532 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-09-10T08:24:51.532 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-09-10T08:24:51.532 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-09-10T08:24:51.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:24:51.533 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-09-10T08:24:51.534 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:24:51.535 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:24:51.535 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:24:51.535 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-09-10T08:24:51.535 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-09-10T08:24:51.535 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-09-10T08:24:51.535 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:24:51.535 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:24:51.535 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:24:51.535 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:24:51.535 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148704, 2024-09-10T08:24:51.535 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 29, 2024-09-10T08:24:51.535 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:24:51.535 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-09-10T08:24:51.536 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-09-10T08:24:51.537 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-09-10T08:24:51.537 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-09-10T08:24:51.537 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-09-10T08:24:51.537 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-09-10T08:24:51.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:51.537 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:24:51.537 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-09-10T08:24:51.537 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-09-10T08:24:51.537 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-09-10T08:24:51.537 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 323, 2024-09-10T08:24:51.537 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 737558, 2024-09-10T08:24:51.537 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:51.537 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 323, 2024-09-10T08:24:51.538 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-09-10T08:24:51.538 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-09-10T08:24:51.538 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 323, 2024-09-10T08:24:51.538 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 737558, 2024-09-10T08:24:51.538 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-09-10T08:24:51.538 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:51.538 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:51.538 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:51.538 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:51.538 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:51.538 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:24:52.017 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:51 smithi002 ceph-mon[30563]: pgmap v888: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T08:24:52.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:51 smithi155 ceph-mon[35405]: pgmap v888: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T08:24:52.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:51 smithi107 ceph-mon[35048]: pgmap v888: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T08:24:52.516 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-10T08:24:52.516 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-10T08:24:52.516 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-10T08:24:52.516 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-10T08:24:52.516 DEBUG:tasks.check_counter:Getting stats from j 2024-09-10T08:24:52.517 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-10T08:24:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:24:53.511 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-10T08:24:53.512 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:24:53.512 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2024-09-10T08:24:53.512 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2024-09-10T08:24:53.512 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2024-09-10T08:24:53.512 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2024-09-10T08:24:53.512 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-09-10T08:24:53.512 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-09-10T08:24:53.512 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.003189930, 2024-09-10T08:24:53.512 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T08:24:53.512 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000530029, 2024-09-10T08:24:53.512 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:24:53.513 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:53.513 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:53.513 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:53.513 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:53.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:53.513 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:53.513 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:53.513 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:53.513 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:53.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:53.513 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:24:53.514 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:24:53.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:53.514 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:24:53.514 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 331, 2024-09-10T08:24:53.514 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 324, 2024-09-10T08:24:53.514 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 787616, 2024-09-10T08:24:53.514 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 199408, 2024-09-10T08:24:53.514 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-09-10T08:24:53.514 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-09-10T08:24:53.515 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.066143184, 2024-09-10T08:24:53.515 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.034686970, 2024-09-10T08:24:53.515 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.037107725, 2024-09-10T08:24:53.515 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:24:53.515 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:53.515 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 324, 2024-09-10T08:24:53.515 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.029634554, 2024-09-10T08:24:53.515 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000091464 2024-09-10T08:24:53.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:53.516 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:53.516 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:53.516 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:53.516 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:53.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:53.516 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 787616, 2024-09-10T08:24:53.516 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 199408 2024-09-10T08:24:53.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:53.516 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:24:53.516 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 2, 2024-09-10T08:24:53.516 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 246, 2024-09-10T08:24:53.517 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 320, 2024-09-10T08:24:53.517 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 200832, 2024-09-10T08:24:53.517 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-09-10T08:24:53.517 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2024-09-10T08:24:53.517 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.028865649, 2024-09-10T08:24:53.517 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.022782548, 2024-09-10T08:24:53.517 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.002356816, 2024-09-10T08:24:53.517 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:24:53.517 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:53.517 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 244, 2024-09-10T08:24:53.518 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.020102286, 2024-09-10T08:24:53.518 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000082386 2024-09-10T08:24:53.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:53.518 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:53.518 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:53.518 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:53.518 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:53.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:53.518 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T08:24:53.518 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 200832 2024-09-10T08:24:53.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-09-10T08:24:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-09-10T08:24:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-09-10T08:24:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-09-10T08:24:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:24:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-09-10T08:24:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:24:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:24:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:24:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:24:53.520 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:24:53.520 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:24:53.520 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:24:53.520 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:24:53.520 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:24:53.520 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:24:53.520 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:24:53.520 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:24:53.520 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:24:53.520 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-09-10T08:24:53.520 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:24:53.520 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-09-10T08:24:53.521 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:24:53.521 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:24:53.521 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:24:53.521 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:24:53.521 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:24:53.521 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:24:53.521 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:24:53.521 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-09-10T08:24:53.521 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:24:53.521 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:24:53.521 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:24:53.521 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148721, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 30, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:24:53.522 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-09-10T08:24:53.523 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:24:53.523 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-09-10T08:24:53.523 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-09-10T08:24:53.523 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-09-10T08:24:53.523 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:24:53.523 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:24:53.523 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-09-10T08:24:53.523 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-09-10T08:24:53.523 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-09-10T08:24:53.523 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-09-10T08:24:53.523 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-09-10T08:24:53.523 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-09-10T08:24:53.523 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-09-10T08:24:53.524 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-09-10T08:24:53.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-10T08:24:53.524 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:24:53.524 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-09-10T08:24:53.524 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-09-10T08:24:53.524 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-09-10T08:24:53.524 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 329, 2024-09-10T08:24:53.524 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 738338, 2024-09-10T08:24:53.524 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:53.524 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 329, 2024-09-10T08:24:53.524 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-09-10T08:24:53.524 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-09-10T08:24:53.524 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 329, 2024-09-10T08:24:53.525 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 738338, 2024-09-10T08:24:53.525 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-09-10T08:24:53.525 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-09-10T08:24:53.525 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-09-10T08:24:53.525 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-09-10T08:24:53.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:53.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-10T08:24:53.525 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-10T08:24:54.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:53 smithi002 ceph-mon[30563]: pgmap v889: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:24:54.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:53 smithi155 ceph-mon[35405]: pgmap v889: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:24:54.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:53 smithi107 ceph-mon[35048]: pgmap v889: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:24:54.331 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-10T08:24:54.331 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-10T08:24:54.331 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-10T08:24:54.331 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-10T08:24:54.332 DEBUG:tasks.check_counter:Getting stats from b 2024-09-10T08:24:54.332 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-10T08:24:54.589 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:24:55.080 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-10T08:24:55.080 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:24:55.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 6755, 2024-09-10T08:24:55.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 8078, 2024-09-10T08:24:55.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1994267, 2024-09-10T08:24:55.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 15550984, 2024-09-10T08:24:55.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 405, 2024-09-10T08:24:55.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 6, 2024-09-10T08:24:55.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.777414510, 2024-09-10T08:24:55.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.233922685, 2024-09-10T08:24:55.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.336614448, 2024-09-10T08:24:55.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.163344641, 2024-09-10T08:24:55.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:55.081 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 8073, 2024-09-10T08:24:55.081 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.810076805, 2024-09-10T08:24:55.081 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000100343 2024-09-10T08:24:55.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:55.081 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3695, 2024-09-10T08:24:55.081 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.001288358, 2024-09-10T08:24:55.081 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000348 2024-09-10T08:24:55.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 96480, 2024-09-10T08:24:55.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 31648 2024-09-10T08:24:55.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.082 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:24:55.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 2274, 2024-09-10T08:24:55.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2868, 2024-09-10T08:24:55.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1418744, 2024-09-10T08:24:55.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 1355361, 2024-09-10T08:24:55.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 31, 2024-09-10T08:24:55.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2024-09-10T08:24:55.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.244109893, 2024-09-10T08:24:55.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.110712029, 2024-09-10T08:24:55.083 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.135700645, 2024-09-10T08:24:55.083 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.016607327, 2024-09-10T08:24:55.083 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:55.083 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2871, 2024-09-10T08:24:55.083 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.953550391, 2024-09-10T08:24:55.083 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000332131 2024-09-10T08:24:55.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.083 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:55.083 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2995, 2024-09-10T08:24:55.083 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000759198, 2024-09-10T08:24:55.083 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000253 2024-09-10T08:24:55.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.083 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 845280, 2024-09-10T08:24:55.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 234592 2024-09-10T08:24:55.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.084 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:24:55.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1924, 2024-09-10T08:24:55.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2778, 2024-09-10T08:24:55.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 615520, 2024-09-10T08:24:55.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 1505563, 2024-09-10T08:24:55.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 14, 2024-09-10T08:24:55.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 6, 2024-09-10T08:24:55.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.188739136, 2024-09-10T08:24:55.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.077802377, 2024-09-10T08:24:55.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.099593161, 2024-09-10T08:24:55.085 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.021075288, 2024-09-10T08:24:55.085 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:55.085 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2776, 2024-09-10T08:24:55.085 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.259833136, 2024-09-10T08:24:55.085 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000093599 2024-09-10T08:24:55.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.085 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:55.085 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3022, 2024-09-10T08:24:55.085 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000953725, 2024-09-10T08:24:55.085 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000315 2024-09-10T08:24:55.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.085 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T08:24:55.085 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 601856 2024-09-10T08:24:55.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.086 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-10T08:24:55.086 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-09-10T08:24:55.086 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-10T08:24:55.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.086 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-09-10T08:24:55.086 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-10T08:24:55.086 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-10T08:24:55.086 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 6775, 2024-09-10T08:24:55.086 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 1.668083931, 2024-09-10T08:24:55.086 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000246211 2024-09-10T08:24:55.086 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:55.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 18, 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000538622, 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000029923 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 676, 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 676, 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 676, 2024-09-10T08:24:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 19.255735506, 2024-09-10T08:24:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.028484815 2024-09-10T08:24:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-09-10T08:24:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-09-10T08:24:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-09-10T08:24:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-09-10T08:24:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 470, 2024-09-10T08:24:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-09-10T08:24:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-09-10T08:24:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 27, 2024-09-10T08:24:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 6, 2024-09-10T08:24:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 21, 2024-09-10T08:24:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-09-10T08:24:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 18, 2024-09-10T08:24:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-09-10T08:24:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 2, 2024-09-10T08:24:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 2, 2024-09-10T08:24:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 2, 2024-09-10T08:24:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 766, 2024-09-10T08:24:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 755, 2024-09-10T08:24:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-09-10T08:24:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-09-10T08:24:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-09-10T08:24:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-09-10T08:24:55.090 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-09-10T08:24:55.090 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-09-10T08:24:55.090 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-09-10T08:24:55.090 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-09-10T08:24:55.090 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-09-10T08:24:55.090 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-09-10T08:24:55.090 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-09-10T08:24:55.090 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-09-10T08:24:55.090 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 382, 2024-09-10T08:24:55.090 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-09-10T08:24:55.090 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 4, 2024-09-10T08:24:55.090 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 234, 2024-09-10T08:24:55.090 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2024-09-10T08:24:55.091 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 382, 2024-09-10T08:24:55.091 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-09-10T08:24:55.091 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T08:24:55.091 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-09-10T08:24:55.091 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 246, 2024-09-10T08:24:55.091 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 28, 2024-09-10T08:24:55.091 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 218, 2024-09-10T08:24:55.091 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 136, 2024-09-10T08:24:55.091 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-09-10T08:24:55.091 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 644, 2024-09-10T08:24:55.091 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 663, 2024-09-10T08:24:55.091 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 599, 2024-09-10T08:24:55.091 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T08:24:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T08:24:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 747, 2024-09-10T08:24:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 709, 2024-09-10T08:24:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 2, 2024-09-10T08:24:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 628 2024-09-10T08:24:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-09-10T08:24:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 11, 2024-09-10T08:24:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-09-10T08:24:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-09-10T08:24:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-09-10T08:24:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 24, 2024-09-10T08:24:55.092 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 3, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 3, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 15, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 206, 2024-09-10T08:24:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 28, 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 2516, 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 2508, 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 2508, 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 8, 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-09-10T08:24:55.094 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 35, 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 76, 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 75, 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 75, 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 1, 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 1, 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 1, 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 12267622, 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 12279313, 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 0, 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2500, 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 5.506171846, 2024-09-10T08:24:55.095 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.002202468 2024-09-10T08:24:55.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.096 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 0 2024-09-10T08:24:55.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.096 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-09-10T08:24:55.096 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 23, 2024-09-10T08:24:55.096 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 54, 2024-09-10T08:24:55.096 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 31, 2024-09-10T08:24:55.096 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 18, 2024-09-10T08:24:55.096 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 20, 2024-09-10T08:24:55.096 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 2, 2024-09-10T08:24:55.096 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 27, 2024-09-10T08:24:55.096 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 29, 2024-09-10T08:24:55.096 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 2, 2024-09-10T08:24:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 2, 2024-09-10T08:24:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 19, 2024-09-10T08:24:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 17, 2024-09-10T08:24:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 56928, 2024-09-10T08:24:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 231712 2024-09-10T08:24:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-09-10T08:24:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 1902, 2024-09-10T08:24:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 1, 2024-09-10T08:24:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 676, 2024-09-10T08:24:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 27, 2024-09-10T08:24:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 2, 2024-09-10T08:24:55.097 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-09-10T08:24:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 18, 2024-09-10T08:24:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.010063998, 2024-09-10T08:24:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000559111 2024-09-10T08:24:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-09-10T08:24:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 28, 2024-09-10T08:24:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.007431428, 2024-09-10T08:24:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000265408 2024-09-10T08:24:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-09-10T08:24:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.098 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-09-10T08:24:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-09-10T08:24:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3, 2024-09-10T08:24:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.001493802, 2024-09-10T08:24:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000497934 2024-09-10T08:24:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-09-10T08:24:55.099 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-09-10T08:24:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-09-10T08:24:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000212594, 2024-09-10T08:24:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000212594 2024-09-10T08:24:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-09-10T08:24:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.100 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.101 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-09-10T08:24:55.101 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.101 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.101 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.101 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-09-10T08:24:55.101 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.101 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.101 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.101 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-09-10T08:24:55.101 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.102 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.102 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.102 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-09-10T08:24:55.102 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 8, 2024-09-10T08:24:55.102 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 16.241359846, 2024-09-10T08:24:55.102 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 2.030169980 2024-09-10T08:24:55.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.102 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-09-10T08:24:55.102 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.102 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.102 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.103 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-09-10T08:24:55.103 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.103 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.103 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.103 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-09-10T08:24:55.103 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-09-10T08:24:55.103 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000205996, 2024-09-10T08:24:55.103 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000205996 2024-09-10T08:24:55.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.103 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-09-10T08:24:55.103 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-09-10T08:24:55.103 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.004160840, 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.002080420 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.002171761, 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001085880 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-09-10T08:24:55.104 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.105 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.105 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.105 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-09-10T08:24:55.105 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-09-10T08:24:55.105 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.011432331, 2024-09-10T08:24:55.105 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.005716165 2024-09-10T08:24:55.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.105 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-09-10T08:24:55.105 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.105 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.105 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.106 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-09-10T08:24:55.106 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.106 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.106 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.106 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-09-10T08:24:55.106 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 601, 2024-09-10T08:24:55.106 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 2.954129961, 2024-09-10T08:24:55.106 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.004915357 2024-09-10T08:24:55.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.106 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-09-10T08:24:55.106 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.106 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.107 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.107 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-09-10T08:24:55.107 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.107 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.107 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.107 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-09-10T08:24:55.107 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.107 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.107 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.108 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-09-10T08:24:55.108 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 9, 2024-09-10T08:24:55.108 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.021446492, 2024-09-10T08:24:55.108 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.002382943 2024-09-10T08:24:55.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.108 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-09-10T08:24:55.108 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.108 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.108 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.108 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-09-10T08:24:55.108 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-09-10T08:24:55.108 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.001626457, 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001626457 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 4, 2024-09-10T08:24:55.109 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 3, 2024-09-10T08:24:55.110 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 1, 2024-09-10T08:24:55.110 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-09-10T08:24:55.110 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 58, 2024-09-10T08:24:55.110 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 58, 2024-09-10T08:24:55.110 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 995, 2024-09-10T08:24:55.110 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-09-10T08:24:55.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.110 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-10T08:24:55.110 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:24:55.110 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-10T08:24:55.111 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:24:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-10T08:24:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:24:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:24:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:24:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:24:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:24:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:24:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:24:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-10T08:24:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:24:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:24:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:24:55.112 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-10T08:24:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-10T08:24:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-10T08:24:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:24:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:24:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:24:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:24:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 295445, 2024-09-10T08:24:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 100, 2024-09-10T08:24:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:24:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-10T08:24:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-10T08:24:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-10T08:24:55.113 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:24:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-10T08:24:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:24:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-10T08:24:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 9144, 2024-09-10T08:24:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 172, 2024-09-10T08:24:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:24:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:24:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-10T08:24:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-10T08:24:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 98194, 2024-09-10T08:24:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 698, 2024-09-10T08:24:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-10T08:24:55.114 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 4955, 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 8569946, 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 33, 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 4955, 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 4955, 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 29.304510349, 2024-09-10T08:24:55.115 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.005914129 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 4955, 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 5948 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 4955, 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 439, 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 4516, 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 450, 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 34, 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 29, 2024-09-10T08:24:55.116 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 2501, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 233, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 518, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 410, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 63, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-09-10T08:24:55.117 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 599, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-09-10T08:24:55.118 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-09-10T08:24:55.119 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-09-10T08:24:55.119 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-09-10T08:24:55.119 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-09-10T08:24:55.119 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-09-10T08:24:55.119 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-09-10T08:24:55.119 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-09-10T08:24:55.119 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-09-10T08:24:55.119 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-09-10T08:24:55.119 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-09-10T08:24:55.119 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 95, 2024-09-10T08:24:55.119 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-09-10T08:24:55.119 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 11, 2024-09-10T08:24:55.119 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 12, 2024-09-10T08:24:55.120 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 12, 2024-09-10T08:24:55.120 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-09-10T08:24:55.120 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-09-10T08:24:55.120 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 1097, 2024-09-10T08:24:55.120 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 0, 2024-09-10T08:24:55.120 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 14 2024-09-10T08:24:55.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.120 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-09-10T08:24:55.120 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 1, 2024-09-10T08:24:55.120 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-09-10T08:24:55.120 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-09-10T08:24:55.120 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 207 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 4, 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 3, 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 4, 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 3, 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.121 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 10949, 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 3046293, 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 10949, 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 10949, 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 3046293, 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.122 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 4955, 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 8570257, 2024-09-10T08:24:55.123 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 4955, 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 8570257, 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-10T08:24:55.124 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:55.125 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-10T08:24:55.125 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 4955, 2024-09-10T08:24:55.125 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 4955, 2024-09-10T08:24:55.125 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 4955, 2024-09-10T08:24:55.125 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 4955, 2024-09-10T08:24:55.125 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:55.125 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.125 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.125 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.125 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:55.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.125 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-09-10T08:24:55.125 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 3, 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 311, 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 3, 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 1, 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 311, 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.126 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x55aa663fe960": { 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 2516, 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 8085009, 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 2516, 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 2500, 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 8085009, 2024-09-10T08:24:55.127 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:55.128 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:55.128 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:55.128 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:55.128 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:55.128 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:55.128 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-10T08:24:55.953 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:55 smithi107 ceph-mon[35048]: pgmap v890: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:24:55.953 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:55 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:24:55.958 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:55 smithi155 ceph-mon[35405]: pgmap v890: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:24:55.959 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:55 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:24:55.979 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-10T08:24:55.979 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-10T08:24:55.979 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=11 2024-09-10T08:24:55.979 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-10T08:24:55.979 DEBUG:tasks.check_counter:Getting stats from e 2024-09-10T08:24:55.980 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-10T08:24:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:55 smithi002 ceph-mon[30563]: pgmap v890: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:24:56.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:55 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:24:56.245 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:24:56.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:56.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:56.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:56.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:56.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:56 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi155 to 176.0M 2024-09-10T08:24:56.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:56 smithi107 ceph-mon[35048]: Unable to set osd_memory_target on smithi155 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T08:24:56.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:56.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:56.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:56.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:56.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:24:56.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:24:56.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:56 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:24:56.790 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-10T08:24:56.791 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:24:56.791 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1120, 2024-09-10T08:24:56.791 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1122, 2024-09-10T08:24:56.791 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 295448, 2024-09-10T08:24:56.791 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 356224, 2024-09-10T08:24:56.791 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 7, 2024-09-10T08:24:56.791 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-09-10T08:24:56.791 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.104415994, 2024-09-10T08:24:56.791 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.043276277, 2024-09-10T08:24:56.791 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.024130313, 2024-09-10T08:24:56.791 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.026195869, 2024-09-10T08:24:56.791 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:56.791 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1120, 2024-09-10T08:24:56.792 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.041656587, 2024-09-10T08:24:56.792 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000037193 2024-09-10T08:24:56.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.792 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:56.792 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.792 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.792 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.792 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:24:56.792 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:24:56.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.792 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:24:56.793 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1336, 2024-09-10T08:24:56.793 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1327, 2024-09-10T08:24:56.793 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1056420, 2024-09-10T08:24:56.793 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 517003, 2024-09-10T08:24:56.793 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-09-10T08:24:56.793 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-09-10T08:24:56.793 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.138833697, 2024-09-10T08:24:56.793 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.050731450, 2024-09-10T08:24:56.793 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.056155095, 2024-09-10T08:24:56.793 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.032170074, 2024-09-10T08:24:56.793 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:56.793 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1327, 2024-09-10T08:24:56.793 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.055133338, 2024-09-10T08:24:56.794 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000041547 2024-09-10T08:24:56.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.794 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:56.794 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.794 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.794 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.794 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 811136, 2024-09-10T08:24:56.794 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 199952 2024-09-10T08:24:56.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.794 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:24:56.794 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1115, 2024-09-10T08:24:56.794 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1358, 2024-09-10T08:24:56.795 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 374288, 2024-09-10T08:24:56.795 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 933518, 2024-09-10T08:24:56.795 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-09-10T08:24:56.795 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-09-10T08:24:56.795 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.145914116, 2024-09-10T08:24:56.795 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.050764262, 2024-09-10T08:24:56.795 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.036044673, 2024-09-10T08:24:56.795 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.043416510, 2024-09-10T08:24:56.795 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:56.795 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1356, 2024-09-10T08:24:56.795 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.082884597, 2024-09-10T08:24:56.795 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000061124 2024-09-10T08:24:56.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.796 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:56.796 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.796 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.796 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.796 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T08:24:56.796 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 579584 2024-09-10T08:24:56.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.796 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-10T08:24:56.796 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-09-10T08:24:56.796 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-10T08:24:56.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.796 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-09-10T08:24:56.797 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-10T08:24:56.797 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-10T08:24:56.797 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 5368, 2024-09-10T08:24:56.797 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.152387591, 2024-09-10T08:24:56.797 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000028388 2024-09-10T08:24:56.797 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:56.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.797 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-09-10T08:24:56.797 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-10T08:24:56.797 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-10T08:24:56.797 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.797 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.797 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.798 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:56.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.798 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-09-10T08:24:56.798 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 0, 2024-09-10T08:24:56.798 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 0, 2024-09-10T08:24:56.798 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-09-10T08:24:56.798 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.798 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.798 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.798 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-09-10T08:24:56.798 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-09-10T08:24:56.798 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-09-10T08:24:56.799 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-09-10T08:24:56.799 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 0, 2024-09-10T08:24:56.799 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-09-10T08:24:56.799 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-09-10T08:24:56.799 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 0, 2024-09-10T08:24:56.799 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 0, 2024-09-10T08:24:56.799 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 0, 2024-09-10T08:24:56.799 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-09-10T08:24:56.799 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 0, 2024-09-10T08:24:56.799 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-09-10T08:24:56.799 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 0, 2024-09-10T08:24:56.799 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 0, 2024-09-10T08:24:56.799 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 1, 2024-09-10T08:24:56.800 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 0, 2024-09-10T08:24:56.800 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 0, 2024-09-10T08:24:56.800 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-09-10T08:24:56.800 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-09-10T08:24:56.800 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-09-10T08:24:56.800 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-09-10T08:24:56.800 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-09-10T08:24:56.800 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-09-10T08:24:56.800 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-09-10T08:24:56.800 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-09-10T08:24:56.800 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-09-10T08:24:56.800 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-09-10T08:24:56.800 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-09-10T08:24:56.801 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-09-10T08:24:56.801 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-09-10T08:24:56.801 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-09-10T08:24:56.801 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 0, 2024-09-10T08:24:56.801 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 0, 2024-09-10T08:24:56.801 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2024-09-10T08:24:56.801 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T08:24:56.801 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-09-10T08:24:56.801 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T08:24:56.801 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-09-10T08:24:56.801 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-09-10T08:24:56.801 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.802 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-09-10T08:24:56.803 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-09-10T08:24:56.803 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-09-10T08:24:56.803 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-09-10T08:24:56.803 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-09-10T08:24:56.803 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-09-10T08:24:56.803 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-09-10T08:24:56.803 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-09-10T08:24:56.803 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-09-10T08:24:56.803 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-09-10T08:24:56.803 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-09-10T08:24:56.803 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-09-10T08:24:56.803 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-09-10T08:24:56.804 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-09-10T08:24:56.804 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-09-10T08:24:56.804 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-09-10T08:24:56.804 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-09-10T08:24:56.804 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-09-10T08:24:56.804 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 0, 2024-09-10T08:24:56.804 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 0, 2024-09-10T08:24:56.804 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T08:24:56.804 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-09-10T08:24:56.804 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-09-10T08:24:56.804 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-09-10T08:24:56.804 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-09-10T08:24:56.804 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-09-10T08:24:56.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.805 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-09-10T08:24:56.805 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-09-10T08:24:56.805 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 0, 2024-09-10T08:24:56.805 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-09-10T08:24:56.805 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-09-10T08:24:56.805 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 5, 2024-09-10T08:24:56.805 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-09-10T08:24:56.805 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-09-10T08:24:56.805 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 0, 2024-09-10T08:24:56.805 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-09-10T08:24:56.805 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-09-10T08:24:56.805 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 2, 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 2, 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 4195737, 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 4195596, 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 5 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.806 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-09-10T08:24:56.807 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 3, 2024-09-10T08:24:56.807 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 3, 2024-09-10T08:24:56.807 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 0, 2024-09-10T08:24:56.807 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 1, 2024-09-10T08:24:56.807 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 1, 2024-09-10T08:24:56.807 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 0, 2024-09-10T08:24:56.807 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 0, 2024-09-10T08:24:56.807 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 0, 2024-09-10T08:24:56.807 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 0, 2024-09-10T08:24:56.807 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 0, 2024-09-10T08:24:56.807 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 0, 2024-09-10T08:24:56.807 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-09-10T08:24:56.807 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 48904, 2024-09-10T08:24:56.808 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 215328 2024-09-10T08:24:56.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.808 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-09-10T08:24:56.808 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 0, 2024-09-10T08:24:56.808 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-09-10T08:24:56.808 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 0, 2024-09-10T08:24:56.808 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 0, 2024-09-10T08:24:56.808 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-09-10T08:24:56.808 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-09-10T08:24:56.808 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.808 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.808 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.809 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-09-10T08:24:56.809 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.809 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.809 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.809 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-09-10T08:24:56.809 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.809 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.809 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.809 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-09-10T08:24:56.809 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.809 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.810 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.810 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-09-10T08:24:56.810 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.810 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.810 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.810 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-09-10T08:24:56.810 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.810 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.810 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.810 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.812 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-09-10T08:24:56.812 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.812 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.812 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.812 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-09-10T08:24:56.812 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.812 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.812 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.812 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-09-10T08:24:56.812 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.813 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.813 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.813 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-09-10T08:24:56.813 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.813 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.813 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.813 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-09-10T08:24:56.813 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.813 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.813 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.814 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-09-10T08:24:56.814 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.814 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.814 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.814 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-09-10T08:24:56.814 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.814 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.814 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.814 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-09-10T08:24:56.814 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.814 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.815 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.815 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-09-10T08:24:56.815 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.815 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.815 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.815 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-09-10T08:24:56.815 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.815 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.815 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.817 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-09-10T08:24:56.817 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.817 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.817 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.817 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-09-10T08:24:56.817 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.817 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.817 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.817 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-09-10T08:24:56.817 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.818 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.818 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.818 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-09-10T08:24:56.818 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.818 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.818 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.818 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-09-10T08:24:56.818 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.818 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.818 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.819 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-09-10T08:24:56.819 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.819 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.819 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.819 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-09-10T08:24:56.819 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.819 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.819 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.819 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-09-10T08:24:56.819 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-09-10T08:24:56.819 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-09-10T08:24:56.820 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.820 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.820 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.820 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:56.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.820 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-09-10T08:24:56.820 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-09-10T08:24:56.820 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 2, 2024-09-10T08:24:56.820 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 1, 2024-09-10T08:24:56.820 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 2, 2024-09-10T08:24:56.820 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-09-10T08:24:56.820 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 0, 2024-09-10T08:24:56.820 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2024-09-10T08:24:56.821 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 3828, 2024-09-10T08:24:56.821 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-09-10T08:24:56.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.821 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-10T08:24:56.821 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:24:56.821 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-10T08:24:56.821 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:24:56.821 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:24:56.821 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:24:56.821 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:24:56.821 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:24:56.821 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:24:56.822 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:24:56.822 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:24:56.822 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:24:56.822 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:24:56.822 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:24:56.822 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:24:56.822 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:24:56.822 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-10T08:24:56.822 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:24:56.822 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-10T08:24:56.822 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:24:56.822 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:24:56.822 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:24:56.823 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:24:56.824 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:24:56.824 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 190907, 2024-09-10T08:24:56.824 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 54, 2024-09-10T08:24:56.824 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:24:56.824 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-10T08:24:56.824 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-10T08:24:56.824 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-10T08:24:56.824 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:24:56.824 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-10T08:24:56.824 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:24:56.824 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-10T08:24:56.824 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 9144, 2024-09-10T08:24:56.824 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 172, 2024-09-10T08:24:56.825 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:24:56.825 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:24:56.825 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-10T08:24:56.825 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-10T08:24:56.825 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 8512, 2024-09-10T08:24:56.825 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 13, 2024-09-10T08:24:56.825 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-10T08:24:56.825 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-10T08:24:56.825 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-10T08:24:56.825 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-10T08:24:56.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.825 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-09-10T08:24:56.825 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-09-10T08:24:56.826 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-09-10T08:24:56.826 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 3230, 2024-09-10T08:24:56.826 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 21, 2024-09-10T08:24:56.826 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 3, 2024-09-10T08:24:56.826 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 3230, 2024-09-10T08:24:56.826 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-09-10T08:24:56.826 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3230, 2024-09-10T08:24:56.826 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 1.669972699, 2024-09-10T08:24:56.826 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000517019 2024-09-10T08:24:56.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.826 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-09-10T08:24:56.826 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-09-10T08:24:56.826 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3230, 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 3231 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 3230, 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 3230, 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 0, 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 2108, 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 0, 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 1120, 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 0, 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 0, 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-09-10T08:24:56.827 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-09-10T08:24:56.828 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-09-10T08:24:56.828 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-09-10T08:24:56.828 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 0, 2024-09-10T08:24:56.828 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-09-10T08:24:56.828 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-09-10T08:24:56.828 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-09-10T08:24:56.828 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 1, 2024-09-10T08:24:56.828 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 0, 2024-09-10T08:24:56.828 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-09-10T08:24:56.828 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-09-10T08:24:56.828 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-09-10T08:24:56.828 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-09-10T08:24:56.828 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-09-10T08:24:56.829 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-09-10T08:24:56.829 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T08:24:56.829 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-09-10T08:24:56.829 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-09-10T08:24:56.829 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-09-10T08:24:56.829 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-09-10T08:24:56.829 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-09-10T08:24:56.829 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-09-10T08:24:56.829 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-09-10T08:24:56.829 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-09-10T08:24:56.829 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-09-10T08:24:56.829 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-09-10T08:24:56.829 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 95, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 2, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 5, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 6, 2024-09-10T08:24:56.830 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 1, 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 0, 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 2, 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 0 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 0, 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-09-10T08:24:56.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.832 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-09-10T08:24:56.832 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-09-10T08:24:56.832 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T08:24:56.832 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-09-10T08:24:56.832 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-09-10T08:24:56.832 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-09-10T08:24:56.832 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-09-10T08:24:56.832 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-09-10T08:24:56.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.832 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:24:56.832 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:56.832 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-10T08:24:56.833 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:56.833 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 3567, 2024-09-10T08:24:56.833 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1384776, 2024-09-10T08:24:56.833 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:56.833 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 3567, 2024-09-10T08:24:56.833 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-10T08:24:56.833 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-10T08:24:56.833 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3567, 2024-09-10T08:24:56.833 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1384776, 2024-09-10T08:24:56.833 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:56.833 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.833 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.833 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3230, 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 1438, 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3230, 2024-09-10T08:24:56.834 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1438, 2024-09-10T08:24:56.835 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:56.835 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.835 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.835 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.835 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:56.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.835 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-09-10T08:24:56.835 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:56.835 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-09-10T08:24:56.835 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:56.835 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-10T08:24:56.835 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-10T08:24:56.835 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3230, 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 3230, 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3230, 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 3230, 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:56.836 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.837 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x563448234b40": { 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-09-10T08:24:56.838 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:56.839 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:56.839 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:56.839 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:56.839 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:56.839 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:56.839 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-10T08:24:57.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:56 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi155 to 176.0M 2024-09-10T08:24:57.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:56 smithi002 ceph-mon[30563]: Unable to set osd_memory_target on smithi155 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T08:24:57.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:24:57.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:24:57.022 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:56 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:24:57.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:56 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi155 to 176.0M 2024-09-10T08:24:57.158 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:56 smithi155 ceph-mon[35405]: Unable to set osd_memory_target on smithi155 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T08:24:57.158 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.158 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.159 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.159 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:24:57.159 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:24:57.159 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:24:57.159 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:56 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' 2024-09-10T08:24:57.609 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-10T08:24:57.609 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-10T08:24:57.609 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-10T08:24:57.609 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-10T08:24:57.609 DEBUG:tasks.check_counter:Getting stats from h 2024-09-10T08:24:57.610 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-10T08:24:57.857 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:24:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:57 smithi107 ceph-mon[35048]: pgmap v891: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T08:24:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:57 smithi107 ceph-mon[35048]: Adjusting osd_memory_target on smithi107 to 175.9M 2024-09-10T08:24:57.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:57 smithi107 ceph-mon[35048]: Unable to set osd_memory_target on smithi107 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-10T08:24:58.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:57 smithi002 ceph-mon[30563]: pgmap v891: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T08:24:58.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:57 smithi002 ceph-mon[30563]: Adjusting osd_memory_target on smithi107 to 175.9M 2024-09-10T08:24:58.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:57 smithi002 ceph-mon[30563]: Unable to set osd_memory_target on smithi107 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-10T08:24:58.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:57 smithi155 ceph-mon[35405]: pgmap v891: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T08:24:58.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:57 smithi155 ceph-mon[35405]: Adjusting osd_memory_target on smithi107 to 175.9M 2024-09-10T08:24:58.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:57 smithi155 ceph-mon[35405]: Unable to set osd_memory_target on smithi107 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-10T08:24:58.377 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-10T08:24:58.377 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:24:58.377 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 32, 2024-09-10T08:24:58.377 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 32, 2024-09-10T08:24:58.377 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 10067, 2024-09-10T08:24:58.378 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 61601, 2024-09-10T08:24:58.378 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 13, 2024-09-10T08:24:58.378 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-09-10T08:24:58.378 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.012047746, 2024-09-10T08:24:58.378 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000497583, 2024-09-10T08:24:58.378 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.005342622, 2024-09-10T08:24:58.378 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000529857, 2024-09-10T08:24:58.379 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:58.379 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 31, 2024-09-10T08:24:58.379 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.020418245, 2024-09-10T08:24:58.379 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000658653 2024-09-10T08:24:58.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.379 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:58.380 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 4, 2024-09-10T08:24:58.380 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000275, 2024-09-10T08:24:58.380 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000068 2024-09-10T08:24:58.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.380 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-10T08:24:58.380 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T08:24:58.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.380 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:24:58.380 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 63, 2024-09-10T08:24:58.381 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 305, 2024-09-10T08:24:58.381 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 11098, 2024-09-10T08:24:58.381 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 1128644, 2024-09-10T08:24:58.381 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 55, 2024-09-10T08:24:58.381 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2024-09-10T08:24:58.381 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.033950251, 2024-09-10T08:24:58.381 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.011848322, 2024-09-10T08:24:58.381 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.017756860, 2024-09-10T08:24:58.381 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000251308, 2024-09-10T08:24:58.381 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:58.381 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 303, 2024-09-10T08:24:58.381 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.025676909, 2024-09-10T08:24:58.382 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000084742 2024-09-10T08:24:58.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.382 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:58.382 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 51, 2024-09-10T08:24:58.382 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000004956, 2024-09-10T08:24:58.382 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000097 2024-09-10T08:24:58.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.382 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-09-10T08:24:58.382 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 580144 2024-09-10T08:24:58.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.382 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:24:58.382 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 2270, 2024-09-10T08:24:58.382 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1648, 2024-09-10T08:24:58.383 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1270657, 2024-09-10T08:24:58.383 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 775404, 2024-09-10T08:24:58.383 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 21, 2024-09-10T08:24:58.383 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2024-09-10T08:24:58.383 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.215796178, 2024-09-10T08:24:58.383 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.099150957, 2024-09-10T08:24:58.383 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.132572424, 2024-09-10T08:24:58.383 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.005674870, 2024-09-10T08:24:58.383 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:24:58.383 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1647, 2024-09-10T08:24:58.383 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.083996671, 2024-09-10T08:24:58.384 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000050999 2024-09-10T08:24:58.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.384 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:24:58.384 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3038, 2024-09-10T08:24:58.384 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000823373, 2024-09-10T08:24:58.384 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000271 2024-09-10T08:24:58.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.384 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 817648, 2024-09-10T08:24:58.384 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 203504 2024-09-10T08:24:58.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.385 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-10T08:24:58.385 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-09-10T08:24:58.385 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-10T08:24:58.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.385 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-09-10T08:24:58.385 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-10T08:24:58.385 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-10T08:24:58.385 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 54, 2024-09-10T08:24:58.385 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.001478188, 2024-09-10T08:24:58.385 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000027373 2024-09-10T08:24:58.385 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:58.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.386 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-09-10T08:24:58.386 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-10T08:24:58.386 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-10T08:24:58.386 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-09-10T08:24:58.386 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000012197, 2024-09-10T08:24:58.386 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000006098 2024-09-10T08:24:58.386 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:58.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.386 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-09-10T08:24:58.386 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 1, 2024-09-10T08:24:58.386 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 1, 2024-09-10T08:24:58.387 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-09-10T08:24:58.387 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-09-10T08:24:58.387 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000125084, 2024-09-10T08:24:58.387 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000125084 2024-09-10T08:24:58.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.387 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-09-10T08:24:58.387 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-09-10T08:24:58.387 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-09-10T08:24:58.387 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-09-10T08:24:58.387 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 11, 2024-09-10T08:24:58.387 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-09-10T08:24:58.387 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-09-10T08:24:58.387 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 16, 2024-09-10T08:24:58.388 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 0, 2024-09-10T08:24:58.388 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 16, 2024-09-10T08:24:58.388 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-09-10T08:24:58.388 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 13, 2024-09-10T08:24:58.388 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 4, 2024-09-10T08:24:58.388 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 1, 2024-09-10T08:24:58.388 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 1, 2024-09-10T08:24:58.388 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 3, 2024-09-10T08:24:58.388 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 41, 2024-09-10T08:24:58.388 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 36, 2024-09-10T08:24:58.388 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-09-10T08:24:58.388 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 5, 2024-09-10T08:24:58.389 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-09-10T08:24:58.389 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-09-10T08:24:58.389 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-09-10T08:24:58.389 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-09-10T08:24:58.389 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-09-10T08:24:58.389 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-09-10T08:24:58.389 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-09-10T08:24:58.389 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-09-10T08:24:58.389 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-09-10T08:24:58.389 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-09-10T08:24:58.389 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-09-10T08:24:58.389 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-09-10T08:24:58.390 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 4, 2024-09-10T08:24:58.390 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 234, 2024-09-10T08:24:58.390 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2024-09-10T08:24:58.390 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T08:24:58.390 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-09-10T08:24:58.390 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T08:24:58.390 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-09-10T08:24:58.390 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-09-10T08:24:58.390 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T08:24:58.390 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T08:24:58.390 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-09-10T08:24:58.390 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-09-10T08:24:58.390 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 1, 2024-09-10T08:24:58.391 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-09-10T08:24:58.391 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T08:24:58.391 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T08:24:58.391 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T08:24:58.391 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 1, 2024-09-10T08:24:58.391 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-09-10T08:24:58.391 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-09-10T08:24:58.391 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-09-10T08:24:58.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.392 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-09-10T08:24:58.392 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-09-10T08:24:58.392 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 17, 2024-09-10T08:24:58.392 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 11, 2024-09-10T08:24:58.392 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 12, 2024-09-10T08:24:58.392 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-09-10T08:24:58.392 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-09-10T08:24:58.392 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-09-10T08:24:58.392 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-09-10T08:24:58.392 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-09-10T08:24:58.392 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-09-10T08:24:58.393 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-09-10T08:24:58.393 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-09-10T08:24:58.393 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-09-10T08:24:58.393 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-09-10T08:24:58.393 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-09-10T08:24:58.393 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-09-10T08:24:58.393 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-09-10T08:24:58.393 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 15, 2024-09-10T08:24:58.393 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 89, 2024-09-10T08:24:58.393 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T08:24:58.393 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-09-10T08:24:58.393 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-09-10T08:24:58.394 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-09-10T08:24:58.394 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-09-10T08:24:58.394 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-09-10T08:24:58.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.394 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-09-10T08:24:58.394 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-09-10T08:24:58.394 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 5, 2024-09-10T08:24:58.394 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-09-10T08:24:58.394 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-09-10T08:24:58.395 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 5, 2024-09-10T08:24:58.395 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-09-10T08:24:58.395 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-09-10T08:24:58.395 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 2, 2024-09-10T08:24:58.395 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-09-10T08:24:58.395 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-09-10T08:24:58.395 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 2, 2024-09-10T08:24:58.395 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 2, 2024-09-10T08:24:58.395 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-09-10T08:24:58.395 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-09-10T08:24:58.395 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-09-10T08:24:58.395 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 4195737, 2024-09-10T08:24:58.395 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 0, 2024-09-10T08:24:58.396 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-09-10T08:24:58.396 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 4, 2024-09-10T08:24:58.396 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.007125151, 2024-09-10T08:24:58.396 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001781287 2024-09-10T08:24:58.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.396 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 0 2024-09-10T08:24:58.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.396 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-09-10T08:24:58.396 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 20, 2024-09-10T08:24:58.396 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 22, 2024-09-10T08:24:58.396 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 2, 2024-09-10T08:24:58.396 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 19, 2024-09-10T08:24:58.397 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 23, 2024-09-10T08:24:58.397 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 4, 2024-09-10T08:24:58.397 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 16, 2024-09-10T08:24:58.397 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 20, 2024-09-10T08:24:58.397 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 4, 2024-09-10T08:24:58.397 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 1, 2024-09-10T08:24:58.397 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 1, 2024-09-10T08:24:58.397 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-09-10T08:24:58.397 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 50876, 2024-09-10T08:24:58.397 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 223520 2024-09-10T08:24:58.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.397 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-09-10T08:24:58.398 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 1, 2024-09-10T08:24:58.398 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-09-10T08:24:58.398 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 1, 2024-09-10T08:24:58.398 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 17, 2024-09-10T08:24:58.398 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-09-10T08:24:58.398 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-09-10T08:24:58.398 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.398 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.398 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.399 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-09-10T08:24:58.399 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.399 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.399 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.399 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-09-10T08:24:58.399 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.399 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.399 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.400 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-09-10T08:24:58.400 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.400 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.400 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.401 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-09-10T08:24:58.401 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.401 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.401 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.402 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-09-10T08:24:58.402 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.402 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.402 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.402 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-09-10T08:24:58.403 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-09-10T08:24:58.403 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000125084, 2024-09-10T08:24:58.403 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000125084 2024-09-10T08:24:58.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.403 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-09-10T08:24:58.403 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.403 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.403 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.403 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-09-10T08:24:58.403 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.403 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.404 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.404 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-09-10T08:24:58.404 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.404 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.405 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.405 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-09-10T08:24:58.405 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.405 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.405 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.406 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-09-10T08:24:58.406 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.406 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.406 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.407 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-09-10T08:24:58.407 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.407 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.407 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.407 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-09-10T08:24:58.407 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.407 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.407 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.407 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-09-10T08:24:58.408 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.408 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.408 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.408 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-09-10T08:24:58.408 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.408 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.408 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.408 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-09-10T08:24:58.408 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.408 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.409 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.409 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-09-10T08:24:58.409 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.409 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.409 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.410 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-09-10T08:24:58.410 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.410 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.412 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.412 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-09-10T08:24:58.412 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.412 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.412 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.413 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-09-10T08:24:58.413 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.413 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.413 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.413 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-09-10T08:24:58.413 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.413 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.413 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.413 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-09-10T08:24:58.413 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-09-10T08:24:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-09-10T08:24:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.415 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-09-10T08:24:58.415 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.415 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.415 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.415 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-09-10T08:24:58.415 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.415 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.415 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.415 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-09-10T08:24:58.415 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.415 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.416 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.416 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-09-10T08:24:58.416 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.416 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.416 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.416 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-09-10T08:24:58.416 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-09-10T08:24:58.416 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-09-10T08:24:58.416 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.416 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.416 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.417 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:58.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.417 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-09-10T08:24:58.417 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-09-10T08:24:58.417 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 2, 2024-09-10T08:24:58.417 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 1, 2024-09-10T08:24:58.417 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 1, 2024-09-10T08:24:58.417 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-09-10T08:24:58.417 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 1, 2024-09-10T08:24:58.417 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2024-09-10T08:24:58.417 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 968, 2024-09-10T08:24:58.417 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-09-10T08:24:58.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.418 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-10T08:24:58.418 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:24:58.418 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-10T08:24:58.418 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:24:58.418 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:24:58.418 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:24:58.418 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:24:58.418 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:24:58.418 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:24:58.418 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:24:58.419 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:24:58.419 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:24:58.419 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:24:58.419 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:24:58.419 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:24:58.419 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:24:58.419 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-10T08:24:58.419 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:24:58.419 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-10T08:24:58.419 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:24:58.419 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:24:58.419 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:24:58.419 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:24:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:24:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:24:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:24:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-10T08:24:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:24:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:24:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:24:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-10T08:24:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-10T08:24:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-10T08:24:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:24:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:24:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:24:58.421 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:24:58.421 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 255013, 2024-09-10T08:24:58.421 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 82, 2024-09-10T08:24:58.421 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:24:58.421 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-10T08:24:58.421 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-10T08:24:58.421 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-10T08:24:58.421 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:24:58.421 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-10T08:24:58.421 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:24:58.421 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-10T08:24:58.421 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 9144, 2024-09-10T08:24:58.421 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 172, 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 81317, 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 327, 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-09-10T08:24:58.422 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-09-10T08:24:58.423 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 42, 2024-09-10T08:24:58.423 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 12036, 2024-09-10T08:24:58.423 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 32, 2024-09-10T08:24:58.423 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 42, 2024-09-10T08:24:58.423 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-09-10T08:24:58.423 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 42, 2024-09-10T08:24:58.423 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.221996262, 2024-09-10T08:24:58.423 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.005285625 2024-09-10T08:24:58.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.423 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-09-10T08:24:58.423 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-09-10T08:24:58.423 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 42, 2024-09-10T08:24:58.423 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 65 2024-09-10T08:24:58.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.424 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 42, 2024-09-10T08:24:58.424 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 0, 2024-09-10T08:24:58.424 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 42, 2024-09-10T08:24:58.424 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-09-10T08:24:58.424 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-09-10T08:24:58.424 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 0, 2024-09-10T08:24:58.424 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 10, 2024-09-10T08:24:58.424 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 0, 2024-09-10T08:24:58.424 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 4, 2024-09-10T08:24:58.424 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 8, 2024-09-10T08:24:58.424 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-09-10T08:24:58.425 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-09-10T08:24:58.425 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-09-10T08:24:58.425 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-09-10T08:24:58.425 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 6, 2024-09-10T08:24:58.425 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-09-10T08:24:58.425 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-09-10T08:24:58.425 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-09-10T08:24:58.425 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 0, 2024-09-10T08:24:58.425 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 20, 2024-09-10T08:24:58.425 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-09-10T08:24:58.425 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-09-10T08:24:58.426 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-09-10T08:24:58.426 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-09-10T08:24:58.426 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-09-10T08:24:58.426 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-09-10T08:24:58.426 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T08:24:58.426 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-09-10T08:24:58.426 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-09-10T08:24:58.426 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-09-10T08:24:58.426 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-09-10T08:24:58.426 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-09-10T08:24:58.426 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-09-10T08:24:58.426 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-09-10T08:24:58.426 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 95, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-09-10T08:24:58.427 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 2, 2024-09-10T08:24:58.428 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 9, 2024-09-10T08:24:58.428 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 13, 2024-09-10T08:24:58.428 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 4, 2024-09-10T08:24:58.428 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-09-10T08:24:58.428 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 16, 2024-09-10T08:24:58.428 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 0, 2024-09-10T08:24:58.428 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 1 2024-09-10T08:24:58.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.428 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-09-10T08:24:58.428 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 1, 2024-09-10T08:24:58.428 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-09-10T08:24:58.428 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-09-10T08:24:58.428 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-09-10T08:24:58.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.429 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-09-10T08:24:58.429 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 97 2024-09-10T08:24:58.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.429 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-09-10T08:24:58.429 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-09-10T08:24:58.429 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T08:24:58.429 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-09-10T08:24:58.429 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-09-10T08:24:58.429 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-09-10T08:24:58.429 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-09-10T08:24:58.429 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 2361, 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1057674, 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 2361, 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 2361, 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1057674, 2024-09-10T08:24:58.430 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:58.431 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.431 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.431 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.431 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:58.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.431 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-09-10T08:24:58.431 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:58.431 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-10T08:24:58.431 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:58.431 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-10T08:24:58.431 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-10T08:24:58.431 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:58.431 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-10T08:24:58.432 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 42, 2024-09-10T08:24:58.432 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 12036, 2024-09-10T08:24:58.432 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 42, 2024-09-10T08:24:58.432 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 12036, 2024-09-10T08:24:58.432 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:58.432 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.432 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.432 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.432 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:58.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.432 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-09-10T08:24:58.432 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:58.432 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-09-10T08:24:58.433 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:58.433 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-10T08:24:58.433 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-10T08:24:58.433 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:58.433 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-10T08:24:58.433 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 42, 2024-09-10T08:24:58.433 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 42, 2024-09-10T08:24:58.433 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 42, 2024-09-10T08:24:58.433 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 42, 2024-09-10T08:24:58.433 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:58.433 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.433 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.433 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.434 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:58.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.434 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-09-10T08:24:58.434 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:58.434 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-10T08:24:58.434 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:58.434 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-10T08:24:58.434 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-10T08:24:58.434 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:58.434 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-10T08:24:58.434 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-10T08:24:58.434 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-10T08:24:58.434 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-09-10T08:24:58.435 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-09-10T08:24:58.435 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:58.435 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.435 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.435 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.435 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:58.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:24:58.435 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x555e04cc6960": { 2024-09-10T08:24:58.435 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:24:58.435 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-10T08:24:58.435 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:24:58.435 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 5, 2024-09-10T08:24:58.435 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1433, 2024-09-10T08:24:58.436 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:24:58.436 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 5, 2024-09-10T08:24:58.436 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-10T08:24:58.436 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-10T08:24:58.436 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 4, 2024-09-10T08:24:58.436 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1433, 2024-09-10T08:24:58.436 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:24:58.436 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:24:58.436 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:24:58.436 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:24:58.436 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:58.436 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:24:58.436 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-10T08:24:59.216 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-10T08:24:59.216 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-10T08:24:59.216 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-10T08:24:59.216 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=17 2024-09-10T08:24:59.216 DEBUG:tasks.check_counter:Getting stats from k 2024-09-10T08:24:59.216 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-10T08:24:59.460 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:24:59.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:24:59 smithi107 ceph-mon[35048]: pgmap v892: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:25:00.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:24:59 smithi002 ceph-mon[30563]: pgmap v892: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:25:00.060 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-10T08:25:00.060 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:25:00.060 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 0, 2024-09-10T08:25:00.060 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2, 2024-09-10T08:25:00.060 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 0, 2024-09-10T08:25:00.061 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 0, 2024-09-10T08:25:00.061 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-09-10T08:25:00.061 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-09-10T08:25:00.061 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.001053248, 2024-09-10T08:25:00.061 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T08:25:00.061 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.000226766, 2024-09-10T08:25:00.061 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:25:00.061 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:00.061 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:25:00.061 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:25:00.061 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:25:00.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:25:00.061 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 2, 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 248, 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 320, 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 201184, 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-09-10T08:25:00.062 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-09-10T08:25:00.063 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.015417749, 2024-09-10T08:25:00.063 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.011621221, 2024-09-10T08:25:00.063 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.001042395, 2024-09-10T08:25:00.063 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:25:00.063 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:00.063 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 246, 2024-09-10T08:25:00.063 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.015025450, 2024-09-10T08:25:00.063 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000061079 2024-09-10T08:25:00.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:25:00.063 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:00.063 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:25:00.063 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:25:00.063 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:25:00.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:25:00.064 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T08:25:00.064 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 201184 2024-09-10T08:25:00.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:25:00.064 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:25:00.064 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 333, 2024-09-10T08:25:00.064 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 326, 2024-09-10T08:25:00.064 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 788160, 2024-09-10T08:25:00.064 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 200688, 2024-09-10T08:25:00.064 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-09-10T08:25:00.064 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-09-10T08:25:00.064 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.052764657, 2024-09-10T08:25:00.064 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.023871444, 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.036606437, 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 326, 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.023357425, 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000071648 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 788160, 2024-09-10T08:25:00.065 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 200688 2024-09-10T08:25:00.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:25:00.066 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-10T08:25:00.066 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-09-10T08:25:00.066 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-10T08:25:00.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:25:00.066 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-10T08:25:00.066 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:25:00.066 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-10T08:25:00.066 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:25:00.066 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:25:00.066 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:25:00.066 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:25:00.066 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:25:00.067 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:25:00.067 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:25:00.067 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:25:00.067 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:25:00.067 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:25:00.067 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:25:00.067 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:25:00.067 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:25:00.067 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-10T08:25:00.067 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:25:00.067 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-10T08:25:00.067 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:25:00.067 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:25:00.068 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:25:00.069 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:25:00.069 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:25:00.069 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 148744, 2024-09-10T08:25:00.069 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 30, 2024-09-10T08:25:00.069 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:25:00.069 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-10T08:25:00.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-10T08:25:00.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-10T08:25:00.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:25:00.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-10T08:25:00.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:25:00.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-10T08:25:00.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:25:00.070 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 331, 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 738598, 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 331, 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 331, 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 738598, 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-10T08:25:00.071 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-10T08:25:00.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:25:00.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-10T08:25:00.072 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-10T08:25:00.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:24:59 smithi155 ceph-mon[35405]: pgmap v892: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:25:00.767 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-10T08:25:00.767 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-10T08:25:00.767 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-10T08:25:00.768 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-10T08:25:00.768 DEBUG:tasks.check_counter:Getting stats from c 2024-09-10T08:25:00.768 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-10T08:25:01.027 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:25:01.526 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-10T08:25:01.526 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:25:01.527 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 3, 2024-09-10T08:25:01.527 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 5, 2024-09-10T08:25:01.527 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 795, 2024-09-10T08:25:01.527 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 960, 2024-09-10T08:25:01.527 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 7, 2024-09-10T08:25:01.527 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 2, 2024-09-10T08:25:01.527 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.002687877, 2024-09-10T08:25:01.528 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.000058692, 2024-09-10T08:25:01.528 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.001532958, 2024-09-10T08:25:01.528 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000066293, 2024-09-10T08:25:01.528 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:01.528 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3, 2024-09-10T08:25:01.528 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.002855035, 2024-09-10T08:25:01.529 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000951678 2024-09-10T08:25:01.529 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.529 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:01.529 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.529 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.530 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.530 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.530 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:25:01.530 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:25:01.530 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.531 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:25:01.531 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2926, 2024-09-10T08:25:01.531 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2919, 2024-09-10T08:25:01.531 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 5751104, 2024-09-10T08:25:01.531 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1023345, 2024-09-10T08:25:01.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2024-09-10T08:25:01.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-09-10T08:25:01.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.310989000, 2024-09-10T08:25:01.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.106020845, 2024-09-10T08:25:01.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.105415764, 2024-09-10T08:25:01.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.083865117, 2024-09-10T08:25:01.533 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:01.533 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2918, 2024-09-10T08:25:01.533 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.118862469, 2024-09-10T08:25:01.536 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000040734 2024-09-10T08:25:01.537 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.537 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:01.537 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.537 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.537 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.538 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.538 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 808224, 2024-09-10T08:25:01.538 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 199664 2024-09-10T08:25:01.538 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.538 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:25:01.538 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1450, 2024-09-10T08:25:01.539 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1691, 2024-09-10T08:25:01.539 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 4257626, 2024-09-10T08:25:01.539 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1040850, 2024-09-10T08:25:01.539 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 7, 2024-09-10T08:25:01.539 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-09-10T08:25:01.539 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.148665906, 2024-09-10T08:25:01.540 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.060025139, 2024-09-10T08:25:01.540 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.035584653, 2024-09-10T08:25:01.540 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.038160065, 2024-09-10T08:25:01.540 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:01.540 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1690, 2024-09-10T08:25:01.540 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.075505056, 2024-09-10T08:25:01.541 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000044677 2024-09-10T08:25:01.541 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.541 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:01.541 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.541 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.542 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.542 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.542 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-10T08:25:01.542 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 581648 2024-09-10T08:25:01.542 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.542 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-09-10T08:25:01.542 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-09-10T08:25:01.543 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-09-10T08:25:01.543 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.543 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-09-10T08:25:01.543 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-09-10T08:25:01.543 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-09-10T08:25:01.543 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6830, 2024-09-10T08:25:01.543 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.185040760, 2024-09-10T08:25:01.544 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000027092 2024-09-10T08:25:01.544 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:01.544 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.544 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-09-10T08:25:01.544 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-09-10T08:25:01.544 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-09-10T08:25:01.545 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.545 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.545 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.545 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:01.545 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.545 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-09-10T08:25:01.546 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2024-09-10T08:25:01.546 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2024-09-10T08:25:01.546 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-09-10T08:25:01.546 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.546 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.547 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.547 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.547 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-09-10T08:25:01.547 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-09-10T08:25:01.547 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-09-10T08:25:01.547 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-09-10T08:25:01.548 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2024-09-10T08:25:01.548 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-09-10T08:25:01.548 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-09-10T08:25:01.548 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 17, 2024-09-10T08:25:01.548 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 5, 2024-09-10T08:25:01.549 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 12, 2024-09-10T08:25:01.549 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-09-10T08:25:01.549 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 9, 2024-09-10T08:25:01.549 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-09-10T08:25:01.549 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-09-10T08:25:01.549 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-09-10T08:25:01.550 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 2, 2024-09-10T08:25:01.550 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2024-09-10T08:25:01.550 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2024-09-10T08:25:01.550 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-09-10T08:25:01.550 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2024-09-10T08:25:01.550 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-09-10T08:25:01.551 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-09-10T08:25:01.551 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-09-10T08:25:01.551 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-09-10T08:25:01.551 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-09-10T08:25:01.551 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-09-10T08:25:01.551 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-09-10T08:25:01.552 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-09-10T08:25:01.552 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-09-10T08:25:01.552 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-09-10T08:25:01.552 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-09-10T08:25:01.552 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-09-10T08:25:01.552 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 4, 2024-09-10T08:25:01.553 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 234, 2024-09-10T08:25:01.553 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2024-09-10T08:25:01.553 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T08:25:01.553 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-09-10T08:25:01.553 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T08:25:01.554 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-09-10T08:25:01.554 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-09-10T08:25:01.554 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T08:25:01.554 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T08:25:01.554 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-09-10T08:25:01.554 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-09-10T08:25:01.555 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T08:25:01.555 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-09-10T08:25:01.555 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T08:25:01.555 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T08:25:01.556 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T08:25:01.556 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2024-09-10T08:25:01.556 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-09-10T08:25:01.556 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-09-10T08:25:01.556 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-09-10T08:25:01.556 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.556 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-09-10T08:25:01.557 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-09-10T08:25:01.557 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-09-10T08:25:01.557 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-09-10T08:25:01.557 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2024-09-10T08:25:01.557 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-09-10T08:25:01.557 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-09-10T08:25:01.557 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-09-10T08:25:01.558 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-09-10T08:25:01.558 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-09-10T08:25:01.558 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-09-10T08:25:01.558 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-09-10T08:25:01.558 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-09-10T08:25:01.558 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-09-10T08:25:01.558 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-09-10T08:25:01.559 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-09-10T08:25:01.559 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-09-10T08:25:01.559 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-09-10T08:25:01.559 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 0, 2024-09-10T08:25:01.559 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 0, 2024-09-10T08:25:01.559 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T08:25:01.559 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-09-10T08:25:01.560 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-09-10T08:25:01.560 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-09-10T08:25:01.560 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-09-10T08:25:01.560 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-09-10T08:25:01.560 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.561 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-09-10T08:25:01.561 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-09-10T08:25:01.561 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2024-09-10T08:25:01.561 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-09-10T08:25:01.561 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-09-10T08:25:01.562 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 2516, 2024-09-10T08:25:01.562 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-09-10T08:25:01.562 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-09-10T08:25:01.562 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2024-09-10T08:25:01.562 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-09-10T08:25:01.562 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-09-10T08:25:01.563 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 2, 2024-09-10T08:25:01.563 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 75, 2024-09-10T08:25:01.563 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-09-10T08:25:01.563 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-09-10T08:25:01.563 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 12267622, 2024-09-10T08:25:01.564 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 12279313, 2024-09-10T08:25:01.564 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 12276816, 2024-09-10T08:25:01.564 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-09-10T08:25:01.564 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.564 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.564 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.565 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.565 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 2516 2024-09-10T08:25:01.565 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.565 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-09-10T08:25:01.565 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 14, 2024-09-10T08:25:01.566 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 17, 2024-09-10T08:25:01.566 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 3, 2024-09-10T08:25:01.566 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 9, 2024-09-10T08:25:01.566 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 11, 2024-09-10T08:25:01.566 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 2, 2024-09-10T08:25:01.567 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 17, 2024-09-10T08:25:01.567 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 19, 2024-09-10T08:25:01.567 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 2, 2024-09-10T08:25:01.567 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-09-10T08:25:01.567 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2024-09-10T08:25:01.567 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2024-09-10T08:25:01.568 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 50552, 2024-09-10T08:25:01.568 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 215328 2024-09-10T08:25:01.568 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.568 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-09-10T08:25:01.568 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2024-09-10T08:25:01.569 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-09-10T08:25:01.569 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2024-09-10T08:25:01.569 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2024-09-10T08:25:01.569 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2024-09-10T08:25:01.569 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-09-10T08:25:01.569 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.570 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.570 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.570 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.570 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-09-10T08:25:01.570 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.570 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.571 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.571 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.571 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-09-10T08:25:01.571 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.571 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.572 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.572 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.572 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-09-10T08:25:01.572 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.572 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.572 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.573 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.573 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-09-10T08:25:01.573 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.573 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.573 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.573 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.574 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-09-10T08:25:01.574 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.574 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.574 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.574 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.574 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-09-10T08:25:01.575 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.575 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.575 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.575 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.575 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-09-10T08:25:01.575 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.575 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.576 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.576 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.576 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-09-10T08:25:01.576 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.576 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.576 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.577 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.577 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-09-10T08:25:01.577 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.577 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.577 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.577 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.577 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-09-10T08:25:01.578 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.578 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.578 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.578 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.578 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-09-10T08:25:01.578 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.580 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.580 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.580 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.580 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-09-10T08:25:01.580 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.580 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.581 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.581 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.581 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-09-10T08:25:01.581 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.581 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.581 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.581 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.582 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-09-10T08:25:01.582 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.582 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.582 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.582 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.582 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-09-10T08:25:01.582 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.583 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.583 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.583 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.583 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-09-10T08:25:01.583 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.583 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.584 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.584 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.584 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-09-10T08:25:01.584 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.584 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.584 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.585 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.585 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-09-10T08:25:01.585 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.585 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.585 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.585 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.586 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-09-10T08:25:01.586 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.586 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.586 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.586 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.587 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-09-10T08:25:01.587 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.587 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.587 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.587 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.587 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-09-10T08:25:01.588 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.588 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.588 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.588 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.588 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-09-10T08:25:01.588 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.588 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.589 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.589 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.589 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-09-10T08:25:01.589 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.589 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.589 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.589 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.589 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-09-10T08:25:01.589 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.589 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-09-10T08:25:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-09-10T08:25:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.590 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-09-10T08:25:01.591 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.591 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.591 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.591 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.591 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-09-10T08:25:01.591 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.591 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.591 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.591 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.591 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-09-10T08:25:01.591 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-09-10T08:25:01.591 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-09-10T08:25:01.591 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.592 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.592 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.592 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:01.592 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.592 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-09-10T08:25:01.592 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2024-09-10T08:25:01.592 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 4, 2024-09-10T08:25:01.592 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 3, 2024-09-10T08:25:01.592 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 2, 2024-09-10T08:25:01.592 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-09-10T08:25:01.592 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2024-09-10T08:25:01.592 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 3835, 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:25:01.593 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:25:01.594 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:25:01.595 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:25:01.595 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-09-10T08:25:01.595 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:25:01.595 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:25:01.595 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:25:01.595 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-09-10T08:25:01.595 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-09-10T08:25:01.595 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-09-10T08:25:01.595 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:25:01.595 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:25:01.595 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:25:01.595 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:25:01.595 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 198242, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 57, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 9144, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 172, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:25:01.596 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-09-10T08:25:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-09-10T08:25:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 54978, 2024-09-10T08:25:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 261, 2024-09-10T08:25:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-09-10T08:25:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-09-10T08:25:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-09-10T08:25:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-09-10T08:25:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-09-10T08:25:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-09-10T08:25:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-09-10T08:25:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 4037, 2024-09-10T08:25:01.597 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 26, 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 4, 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 4037, 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4037, 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.845961296, 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000457260 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4037, 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 4038 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 4037, 2024-09-10T08:25:01.598 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 4037, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 2234, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 1800, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2024-09-10T08:25:01.599 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-09-10T08:25:01.600 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-09-10T08:25:01.600 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-09-10T08:25:01.600 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 2, 2024-09-10T08:25:01.600 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2024-09-10T08:25:01.600 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-09-10T08:25:01.600 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-09-10T08:25:01.600 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-09-10T08:25:01.600 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-09-10T08:25:01.600 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-09-10T08:25:01.600 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-09-10T08:25:01.600 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T08:25:01.600 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-09-10T08:25:01.600 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-09-10T08:25:01.601 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 95, 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 2, 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 6, 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 8, 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 2, 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2024-09-10T08:25:01.602 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-09-10T08:25:01.603 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-09-10T08:25:01.604 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 0, 2024-09-10T08:25:01.604 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-09-10T08:25:01.604 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-09-10T08:25:01.604 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.604 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:25:01.604 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:01.604 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-09-10T08:25:01.604 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:01.604 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 4375, 2024-09-10T08:25:01.604 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 9595379, 2024-09-10T08:25:01.604 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:01.604 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 4375, 2024-09-10T08:25:01.604 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 4375, 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 9595379, 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:01.605 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-09-10T08:25:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-09-10T08:25:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-09-10T08:25:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 4037, 2024-09-10T08:25:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 8085019, 2024-09-10T08:25:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 4037, 2024-09-10T08:25:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 8085019, 2024-09-10T08:25:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 4037, 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 4037, 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 4037, 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 4037, 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:01.607 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.608 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.608 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.608 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:01.608 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.608 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-09-10T08:25:01.608 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:01.608 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-09-10T08:25:01.608 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:01.608 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-09-10T08:25:01.608 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-09-10T08:25:01.608 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:01.608 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-09-10T08:25:01.608 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55763b526b40": { 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:01.609 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-09-10T08:25:01.610 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-09-10T08:25:01.610 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:01.610 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-09-10T08:25:01.610 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-09-10T08:25:01.610 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-09-10T08:25:01.610 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-09-10T08:25:01.610 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-09-10T08:25:01.610 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:01.610 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:01.610 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:01.610 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:01.610 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:01.610 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:01.611 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-10T08:25:01.951 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:01 smithi155 ceph-mon[35405]: pgmap v893: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:25:02.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:01 smithi002 ceph-mon[30563]: pgmap v893: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:25:02.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:01 smithi107 ceph-mon[35048]: pgmap v893: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:25:02.382 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-10T08:25:02.382 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-10T08:25:02.382 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-10T08:25:02.382 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-10T08:25:02.382 DEBUG:tasks.check_counter:Getting stats from f 2024-09-10T08:25:02.383 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-10T08:25:02.644 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:25:03.177 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-10T08:25:03.177 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:25:03.178 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 3270, 2024-09-10T08:25:03.178 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 3270, 2024-09-10T08:25:03.178 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 927818, 2024-09-10T08:25:03.178 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1039043, 2024-09-10T08:25:03.178 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 6, 2024-09-10T08:25:03.178 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 2, 2024-09-10T08:25:03.178 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.293386335, 2024-09-10T08:25:03.178 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.098232757, 2024-09-10T08:25:03.178 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.074999789, 2024-09-10T08:25:03.178 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.097548472, 2024-09-10T08:25:03.179 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:03.179 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3269, 2024-09-10T08:25:03.179 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.135468946, 2024-09-10T08:25:03.179 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000041440 2024-09-10T08:25:03.179 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.179 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:03.179 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.179 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.179 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.179 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.179 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-10T08:25:03.179 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T08:25:03.180 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.180 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:25:03.180 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 4, 2024-09-10T08:25:03.180 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 248, 2024-09-10T08:25:03.180 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 876, 2024-09-10T08:25:03.180 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 582074, 2024-09-10T08:25:03.180 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 5, 2024-09-10T08:25:03.180 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-09-10T08:25:03.180 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.014219133, 2024-09-10T08:25:03.180 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.010461559, 2024-09-10T08:25:03.181 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.001108874, 2024-09-10T08:25:03.181 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000052544, 2024-09-10T08:25:03.181 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:03.181 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 246, 2024-09-10T08:25:03.181 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.017143897, 2024-09-10T08:25:03.181 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000069690 2024-09-10T08:25:03.181 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.181 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:03.181 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.181 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.181 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.182 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.182 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T08:25:03.182 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 581376 2024-09-10T08:25:03.182 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.182 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:25:03.182 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 337, 2024-09-10T08:25:03.182 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 330, 2024-09-10T08:25:03.182 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 808604, 2024-09-10T08:25:03.182 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 200462, 2024-09-10T08:25:03.182 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2024-09-10T08:25:03.183 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 2, 2024-09-10T08:25:03.183 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.051631076, 2024-09-10T08:25:03.183 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.023566051, 2024-09-10T08:25:03.183 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.035730858, 2024-09-10T08:25:03.183 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000023405, 2024-09-10T08:25:03.183 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:03.183 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 329, 2024-09-10T08:25:03.183 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.022062442, 2024-09-10T08:25:03.183 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000067059 2024-09-10T08:25:03.183 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.184 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:03.184 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.184 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.184 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.184 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.184 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 808336, 2024-09-10T08:25:03.184 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 200144 2024-09-10T08:25:03.184 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.184 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-09-10T08:25:03.184 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-09-10T08:25:03.185 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-09-10T08:25:03.185 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.185 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-09-10T08:25:03.185 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-09-10T08:25:03.185 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-09-10T08:25:03.185 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5513, 2024-09-10T08:25:03.185 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.151021756, 2024-09-10T08:25:03.185 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000027393 2024-09-10T08:25:03.185 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:03.185 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.186 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-09-10T08:25:03.186 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-09-10T08:25:03.186 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-09-10T08:25:03.186 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.186 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.186 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.186 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:03.186 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.186 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-09-10T08:25:03.187 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2024-09-10T08:25:03.187 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2024-09-10T08:25:03.187 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-09-10T08:25:03.187 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.187 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.187 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.187 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.187 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-09-10T08:25:03.188 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-09-10T08:25:03.188 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-09-10T08:25:03.188 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-09-10T08:25:03.188 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2024-09-10T08:25:03.188 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-09-10T08:25:03.188 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-09-10T08:25:03.188 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 0, 2024-09-10T08:25:03.188 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 0, 2024-09-10T08:25:03.188 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 0, 2024-09-10T08:25:03.188 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-09-10T08:25:03.189 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 0, 2024-09-10T08:25:03.189 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-09-10T08:25:03.189 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-09-10T08:25:03.189 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-09-10T08:25:03.189 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 1, 2024-09-10T08:25:03.189 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2024-09-10T08:25:03.189 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2024-09-10T08:25:03.189 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-09-10T08:25:03.189 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2024-09-10T08:25:03.189 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-09-10T08:25:03.189 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-09-10T08:25:03.190 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-09-10T08:25:03.190 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-09-10T08:25:03.190 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-09-10T08:25:03.190 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-09-10T08:25:03.190 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-09-10T08:25:03.190 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-09-10T08:25:03.190 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-09-10T08:25:03.190 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-09-10T08:25:03.190 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-09-10T08:25:03.190 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-09-10T08:25:03.190 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 0, 2024-09-10T08:25:03.191 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 0, 2024-09-10T08:25:03.191 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2024-09-10T08:25:03.191 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T08:25:03.191 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-09-10T08:25:03.191 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T08:25:03.191 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-09-10T08:25:03.191 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-09-10T08:25:03.191 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T08:25:03.191 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T08:25:03.191 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-09-10T08:25:03.191 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-09-10T08:25:03.192 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T08:25:03.192 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-09-10T08:25:03.192 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T08:25:03.192 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T08:25:03.192 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T08:25:03.192 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2024-09-10T08:25:03.192 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-09-10T08:25:03.192 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-09-10T08:25:03.192 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-09-10T08:25:03.192 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.192 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-09-10T08:25:03.193 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-09-10T08:25:03.193 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-09-10T08:25:03.193 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-09-10T08:25:03.193 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2024-09-10T08:25:03.193 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-09-10T08:25:03.193 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-09-10T08:25:03.193 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-09-10T08:25:03.193 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-09-10T08:25:03.193 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-09-10T08:25:03.193 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-09-10T08:25:03.193 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-09-10T08:25:03.194 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-09-10T08:25:03.194 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-09-10T08:25:03.194 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-09-10T08:25:03.194 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-09-10T08:25:03.194 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-09-10T08:25:03.194 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-09-10T08:25:03.194 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 0, 2024-09-10T08:25:03.194 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 0, 2024-09-10T08:25:03.194 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T08:25:03.194 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-09-10T08:25:03.194 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-09-10T08:25:03.194 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-09-10T08:25:03.195 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-09-10T08:25:03.195 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-09-10T08:25:03.195 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.195 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-09-10T08:25:03.195 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-09-10T08:25:03.195 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2024-09-10T08:25:03.195 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-09-10T08:25:03.195 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-09-10T08:25:03.195 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 7, 2024-09-10T08:25:03.195 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-09-10T08:25:03.195 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-09-10T08:25:03.195 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2024-09-10T08:25:03.196 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-09-10T08:25:03.196 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-09-10T08:25:03.196 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 2, 2024-09-10T08:25:03.196 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 2, 2024-09-10T08:25:03.196 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-09-10T08:25:03.196 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-09-10T08:25:03.196 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-09-10T08:25:03.196 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 4196271, 2024-09-10T08:25:03.196 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 4196130, 2024-09-10T08:25:03.196 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-09-10T08:25:03.196 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.197 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.197 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.197 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.197 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 7 2024-09-10T08:25:03.197 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.197 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-09-10T08:25:03.197 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 3, 2024-09-10T08:25:03.197 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 3, 2024-09-10T08:25:03.197 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 0, 2024-09-10T08:25:03.197 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 1, 2024-09-10T08:25:03.197 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 1, 2024-09-10T08:25:03.197 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 0, 2024-09-10T08:25:03.198 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 0, 2024-09-10T08:25:03.198 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 0, 2024-09-10T08:25:03.198 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 0, 2024-09-10T08:25:03.198 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-09-10T08:25:03.198 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2024-09-10T08:25:03.198 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2024-09-10T08:25:03.198 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 48936, 2024-09-10T08:25:03.198 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 215328 2024-09-10T08:25:03.198 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.198 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-09-10T08:25:03.198 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2024-09-10T08:25:03.198 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-09-10T08:25:03.199 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2024-09-10T08:25:03.199 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2024-09-10T08:25:03.199 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2024-09-10T08:25:03.199 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-09-10T08:25:03.199 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.199 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.199 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.199 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.199 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-09-10T08:25:03.199 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.199 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.200 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.200 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.200 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-09-10T08:25:03.200 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.200 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.200 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.200 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.200 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-09-10T08:25:03.200 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.200 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.200 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.200 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.201 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-09-10T08:25:03.201 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.201 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.201 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.201 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.201 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-09-10T08:25:03.201 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.201 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.201 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.201 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.201 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-09-10T08:25:03.202 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.202 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.202 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.202 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.202 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-09-10T08:25:03.202 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.202 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.202 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.202 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.202 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-09-10T08:25:03.202 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.203 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.203 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.203 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.203 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-09-10T08:25:03.203 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.203 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.203 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.203 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.203 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-09-10T08:25:03.203 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.203 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.204 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.204 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.204 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-09-10T08:25:03.204 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.205 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.205 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.205 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.205 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-09-10T08:25:03.205 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.208 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.208 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.209 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.209 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-09-10T08:25:03.209 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.209 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.209 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.209 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.209 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-09-10T08:25:03.209 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.209 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.209 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.210 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.210 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-09-10T08:25:03.210 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.210 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.210 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.210 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.210 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-09-10T08:25:03.210 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.210 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.210 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.210 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.211 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-09-10T08:25:03.211 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.211 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.211 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.211 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.211 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-09-10T08:25:03.211 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.211 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.211 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.211 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.211 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-09-10T08:25:03.212 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.212 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.212 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.212 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.212 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-09-10T08:25:03.212 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.212 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.212 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.212 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.212 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-09-10T08:25:03.212 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.213 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.213 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.213 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.213 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-09-10T08:25:03.213 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.213 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.213 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.213 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.213 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-09-10T08:25:03.213 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.213 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.214 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.214 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.214 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-09-10T08:25:03.214 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.214 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.214 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.214 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.214 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-09-10T08:25:03.214 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.214 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.214 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.215 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.215 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-09-10T08:25:03.215 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.215 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.215 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.215 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.215 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-09-10T08:25:03.215 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.215 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.215 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.215 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.216 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-09-10T08:25:03.216 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.216 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.216 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.216 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.216 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-09-10T08:25:03.216 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-09-10T08:25:03.216 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-09-10T08:25:03.216 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.216 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.216 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.217 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:03.217 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.217 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-09-10T08:25:03.217 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2024-09-10T08:25:03.217 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 3, 2024-09-10T08:25:03.217 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 2, 2024-09-10T08:25:03.217 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 2, 2024-09-10T08:25:03.217 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-09-10T08:25:03.217 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2024-09-10T08:25:03.217 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2024-09-10T08:25:03.217 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 3837, 2024-09-10T08:25:03.218 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-09-10T08:25:03.218 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.218 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-09-10T08:25:03.218 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:25:03.218 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-09-10T08:25:03.218 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:25:03.218 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:25:03.218 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:25:03.218 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:25:03.218 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:25:03.219 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:25:03.219 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:25:03.219 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:25:03.219 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:25:03.219 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:25:03.219 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:25:03.219 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:25:03.219 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:25:03.219 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-09-10T08:25:03.219 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:25:03.219 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-09-10T08:25:03.219 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:25:03.220 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:25:03.220 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:25:03.220 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:25:03.220 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:25:03.220 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:25:03.220 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:25:03.220 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-09-10T08:25:03.220 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:25:03.220 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:25:03.220 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:25:03.221 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-09-10T08:25:03.221 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-09-10T08:25:03.221 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-09-10T08:25:03.221 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:25:03.221 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:25:03.221 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:25:03.221 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:25:03.221 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 182127, 2024-09-10T08:25:03.221 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 49, 2024-09-10T08:25:03.221 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:25:03.221 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-09-10T08:25:03.222 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-09-10T08:25:03.222 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-09-10T08:25:03.222 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:25:03.222 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-09-10T08:25:03.222 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:25:03.222 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-09-10T08:25:03.222 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 9144, 2024-09-10T08:25:03.222 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 172, 2024-09-10T08:25:03.222 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:25:03.222 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:25:03.222 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-09-10T08:25:03.222 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-09-10T08:25:03.223 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 8512, 2024-09-10T08:25:03.223 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 13, 2024-09-10T08:25:03.223 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-09-10T08:25:03.223 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-09-10T08:25:03.223 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-09-10T08:25:03.223 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-09-10T08:25:03.223 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.223 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-09-10T08:25:03.223 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-09-10T08:25:03.223 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-09-10T08:25:03.223 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 3269, 2024-09-10T08:25:03.223 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 21, 2024-09-10T08:25:03.224 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 3, 2024-09-10T08:25:03.224 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 3269, 2024-09-10T08:25:03.224 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-09-10T08:25:03.224 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3269, 2024-09-10T08:25:03.224 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.571967044, 2024-09-10T08:25:03.224 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000480870 2024-09-10T08:25:03.224 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.224 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-09-10T08:25:03.224 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-09-10T08:25:03.224 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3269, 2024-09-10T08:25:03.224 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 3270 2024-09-10T08:25:03.225 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.225 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 3269, 2024-09-10T08:25:03.225 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 3269, 2024-09-10T08:25:03.225 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2024-09-10T08:25:03.225 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-09-10T08:25:03.225 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-09-10T08:25:03.225 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 2138, 2024-09-10T08:25:03.225 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2024-09-10T08:25:03.225 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 1129, 2024-09-10T08:25:03.225 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2024-09-10T08:25:03.225 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2024-09-10T08:25:03.225 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-09-10T08:25:03.226 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-09-10T08:25:03.226 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-09-10T08:25:03.226 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-09-10T08:25:03.226 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2024-09-10T08:25:03.226 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-09-10T08:25:03.226 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-09-10T08:25:03.226 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-09-10T08:25:03.226 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 1, 2024-09-10T08:25:03.226 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2024-09-10T08:25:03.226 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-09-10T08:25:03.226 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-09-10T08:25:03.227 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-09-10T08:25:03.227 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-09-10T08:25:03.227 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-09-10T08:25:03.227 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-09-10T08:25:03.227 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T08:25:03.227 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-09-10T08:25:03.227 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-09-10T08:25:03.227 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-09-10T08:25:03.227 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-09-10T08:25:03.227 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-09-10T08:25:03.227 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-09-10T08:25:03.228 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-09-10T08:25:03.228 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-09-10T08:25:03.228 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-09-10T08:25:03.228 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-09-10T08:25:03.228 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-09-10T08:25:03.228 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-09-10T08:25:03.228 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-09-10T08:25:03.228 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-09-10T08:25:03.228 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-09-10T08:25:03.228 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-09-10T08:25:03.228 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-09-10T08:25:03.228 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-09-10T08:25:03.229 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-09-10T08:25:03.229 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 95, 2024-09-10T08:25:03.229 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-09-10T08:25:03.229 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 2, 2024-09-10T08:25:03.229 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 4, 2024-09-10T08:25:03.229 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 5, 2024-09-10T08:25:03.229 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 1, 2024-09-10T08:25:03.229 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-09-10T08:25:03.229 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2024-09-10T08:25:03.229 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2024-09-10T08:25:03.230 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2024-09-10T08:25:03.230 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.230 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-09-10T08:25:03.230 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2024-09-10T08:25:03.230 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-09-10T08:25:03.230 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-09-10T08:25:03.230 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-09-10T08:25:03.230 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.230 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-09-10T08:25:03.230 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-09-10T08:25:03.230 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.231 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-09-10T08:25:03.231 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-09-10T08:25:03.231 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T08:25:03.231 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-09-10T08:25:03.231 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-09-10T08:25:03.231 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 0, 2024-09-10T08:25:03.231 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-09-10T08:25:03.231 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-09-10T08:25:03.231 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.231 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:25:03.231 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:03.231 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-09-10T08:25:03.232 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:03.232 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 3607, 2024-09-10T08:25:03.232 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 1392266, 2024-09-10T08:25:03.232 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:03.232 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 3607, 2024-09-10T08:25:03.232 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-09-10T08:25:03.232 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-09-10T08:25:03.232 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3607, 2024-09-10T08:25:03.232 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1392266, 2024-09-10T08:25:03.232 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:03.232 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.233 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.233 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.233 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:03.233 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.233 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-09-10T08:25:03.233 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:03.233 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-09-10T08:25:03.233 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:03.233 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-09-10T08:25:03.233 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-09-10T08:25:03.233 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:03.234 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-09-10T08:25:03.234 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 3269, 2024-09-10T08:25:03.234 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 1972, 2024-09-10T08:25:03.234 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3269, 2024-09-10T08:25:03.234 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1972, 2024-09-10T08:25:03.234 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:03.234 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.234 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.234 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.234 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:03.234 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.235 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-09-10T08:25:03.235 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:03.235 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-09-10T08:25:03.235 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:03.235 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-09-10T08:25:03.235 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-09-10T08:25:03.235 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:03.235 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-09-10T08:25:03.235 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 3269, 2024-09-10T08:25:03.235 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 3269, 2024-09-10T08:25:03.235 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3269, 2024-09-10T08:25:03.235 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 3269, 2024-09-10T08:25:03.236 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:03.236 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.236 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.236 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.236 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:03.236 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.237 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-09-10T08:25:03.237 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:03.237 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-09-10T08:25:03.237 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:03.237 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-09-10T08:25:03.237 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-09-10T08:25:03.237 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:03.238 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-09-10T08:25:03.238 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-09-10T08:25:03.238 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-09-10T08:25:03.238 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-09-10T08:25:03.238 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-09-10T08:25:03.238 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:03.238 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.238 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.238 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.238 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:03.238 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:03.239 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55f846532b40": { 2024-09-10T08:25:03.239 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:03.239 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-09-10T08:25:03.239 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:03.239 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-09-10T08:25:03.239 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-09-10T08:25:03.239 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:03.239 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-09-10T08:25:03.239 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-09-10T08:25:03.239 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-09-10T08:25:03.239 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-09-10T08:25:03.240 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-09-10T08:25:03.240 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:03.240 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:03.240 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:03.240 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:03.240 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:03.240 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:03.240 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-10T08:25:03.884 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-10T08:25:03.884 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-10T08:25:03.884 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-10T08:25:03.884 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-10T08:25:03.884 DEBUG:tasks.check_counter:Getting stats from i 2024-09-10T08:25:03.884 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-10T08:25:04.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:03 smithi002 ceph-mon[30563]: pgmap v894: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T08:25:04.130 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:25:04.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:03 smithi155 ceph-mon[35405]: pgmap v894: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T08:25:04.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:03 smithi107 ceph-mon[35048]: pgmap v894: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T08:25:04.621 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-10T08:25:04.621 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:25:04.621 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 408, 2024-09-10T08:25:04.621 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 399, 2024-09-10T08:25:04.622 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 832652, 2024-09-10T08:25:04.622 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 289300, 2024-09-10T08:25:04.622 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 16, 2024-09-10T08:25:04.622 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-09-10T08:25:04.622 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.070381069, 2024-09-10T08:25:04.622 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.027860098, 2024-09-10T08:25:04.622 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.042499768, 2024-09-10T08:25:04.622 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000453391, 2024-09-10T08:25:04.623 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:04.623 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 396, 2024-09-10T08:25:04.623 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.031257450, 2024-09-10T08:25:04.623 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000078932 2024-09-10T08:25:04.623 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.623 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:04.623 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 32, 2024-09-10T08:25:04.623 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000014910, 2024-09-10T08:25:04.623 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000465 2024-09-10T08:25:04.624 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.624 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 822848, 2024-09-10T08:25:04.624 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 204944 2024-09-10T08:25:04.624 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.624 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:25:04.624 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1895, 2024-09-10T08:25:04.624 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1518, 2024-09-10T08:25:04.624 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 449409, 2024-09-10T08:25:04.624 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1005326, 2024-09-10T08:25:04.625 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 11, 2024-09-10T08:25:04.625 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-09-10T08:25:04.625 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.144029837, 2024-09-10T08:25:04.625 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.062132271, 2024-09-10T08:25:04.625 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.088350414, 2024-09-10T08:25:04.625 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.006613026, 2024-09-10T08:25:04.625 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:04.625 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1518, 2024-09-10T08:25:04.625 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.074761835, 2024-09-10T08:25:04.625 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000049250 2024-09-10T08:25:04.625 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.626 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:04.626 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3026, 2024-09-10T08:25:04.626 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000631878, 2024-09-10T08:25:04.626 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000208 2024-09-10T08:25:04.626 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.626 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T08:25:04.626 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 582816 2024-09-10T08:25:04.626 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.626 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:25:04.626 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 89, 2024-09-10T08:25:04.627 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 89, 2024-09-10T08:25:04.627 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 19484, 2024-09-10T08:25:04.627 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 701481, 2024-09-10T08:25:04.627 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 65, 2024-09-10T08:25:04.627 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 8, 2024-09-10T08:25:04.627 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.029115686, 2024-09-10T08:25:04.627 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.002558452, 2024-09-10T08:25:04.627 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.024052019, 2024-09-10T08:25:04.627 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000539248, 2024-09-10T08:25:04.627 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:04.628 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 88, 2024-09-10T08:25:04.628 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.023106787, 2024-09-10T08:25:04.628 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000262577 2024-09-10T08:25:04.628 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.628 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:04.628 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 77, 2024-09-10T08:25:04.628 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000008713, 2024-09-10T08:25:04.628 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000113 2024-09-10T08:25:04.628 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.628 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 6448, 2024-09-10T08:25:04.628 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 4976 2024-09-10T08:25:04.628 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.629 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-09-10T08:25:04.629 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-09-10T08:25:04.629 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-09-10T08:25:04.629 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.629 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-09-10T08:25:04.629 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-09-10T08:25:04.629 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-09-10T08:25:04.629 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 53, 2024-09-10T08:25:04.629 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.002068693, 2024-09-10T08:25:04.629 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000039031 2024-09-10T08:25:04.629 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:04.630 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.630 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-09-10T08:25:04.630 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-09-10T08:25:04.630 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-09-10T08:25:04.630 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-09-10T08:25:04.630 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000013937, 2024-09-10T08:25:04.630 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000006968 2024-09-10T08:25:04.630 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:04.630 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.630 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-09-10T08:25:04.630 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 7, 2024-09-10T08:25:04.630 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 4, 2024-09-10T08:25:04.631 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-09-10T08:25:04.631 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4, 2024-09-10T08:25:04.631 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001949424, 2024-09-10T08:25:04.631 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000487356 2024-09-10T08:25:04.631 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.631 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-09-10T08:25:04.631 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 3, 2024-09-10T08:25:04.631 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-09-10T08:25:04.632 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-09-10T08:25:04.632 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 11, 2024-09-10T08:25:04.632 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-09-10T08:25:04.632 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-09-10T08:25:04.632 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 16, 2024-09-10T08:25:04.632 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 0, 2024-09-10T08:25:04.632 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 16, 2024-09-10T08:25:04.632 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-09-10T08:25:04.632 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 13, 2024-09-10T08:25:04.632 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 4, 2024-09-10T08:25:04.633 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-09-10T08:25:04.633 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-09-10T08:25:04.633 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 3, 2024-09-10T08:25:04.633 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 51, 2024-09-10T08:25:04.633 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 43, 2024-09-10T08:25:04.633 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 1, 2024-09-10T08:25:04.633 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 6, 2024-09-10T08:25:04.633 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-09-10T08:25:04.633 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-09-10T08:25:04.633 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-09-10T08:25:04.634 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-09-10T08:25:04.634 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-09-10T08:25:04.634 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-09-10T08:25:04.634 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-09-10T08:25:04.634 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-09-10T08:25:04.634 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-09-10T08:25:04.634 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-09-10T08:25:04.634 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-09-10T08:25:04.634 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-09-10T08:25:04.635 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 4, 2024-09-10T08:25:04.635 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 234, 2024-09-10T08:25:04.635 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2024-09-10T08:25:04.635 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T08:25:04.635 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-09-10T08:25:04.635 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T08:25:04.635 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-09-10T08:25:04.635 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-09-10T08:25:04.635 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T08:25:04.635 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T08:25:04.636 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-09-10T08:25:04.636 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-09-10T08:25:04.636 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 3, 2024-09-10T08:25:04.636 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-09-10T08:25:04.636 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T08:25:04.636 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T08:25:04.636 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T08:25:04.636 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 3, 2024-09-10T08:25:04.636 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-09-10T08:25:04.636 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-09-10T08:25:04.636 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 1 2024-09-10T08:25:04.637 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.637 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-09-10T08:25:04.637 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-09-10T08:25:04.637 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 15, 2024-09-10T08:25:04.637 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 10, 2024-09-10T08:25:04.637 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 12, 2024-09-10T08:25:04.637 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-09-10T08:25:04.637 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-09-10T08:25:04.637 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-09-10T08:25:04.637 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-09-10T08:25:04.637 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-09-10T08:25:04.638 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-09-10T08:25:04.638 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-09-10T08:25:04.638 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-09-10T08:25:04.638 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-09-10T08:25:04.638 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-09-10T08:25:04.638 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-09-10T08:25:04.638 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-09-10T08:25:04.638 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-09-10T08:25:04.638 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 15, 2024-09-10T08:25:04.638 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 89, 2024-09-10T08:25:04.638 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T08:25:04.638 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-09-10T08:25:04.638 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-09-10T08:25:04.639 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-09-10T08:25:04.639 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-09-10T08:25:04.639 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-09-10T08:25:04.639 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.639 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-09-10T08:25:04.639 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-09-10T08:25:04.639 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 7, 2024-09-10T08:25:04.639 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-09-10T08:25:04.639 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-09-10T08:25:04.639 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 7, 2024-09-10T08:25:04.639 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-09-10T08:25:04.639 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-09-10T08:25:04.640 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 2, 2024-09-10T08:25:04.640 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-09-10T08:25:04.640 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-09-10T08:25:04.640 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 2, 2024-09-10T08:25:04.640 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 2, 2024-09-10T08:25:04.640 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-09-10T08:25:04.640 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-09-10T08:25:04.640 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-09-10T08:25:04.640 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 4196271, 2024-09-10T08:25:04.640 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2024-09-10T08:25:04.640 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-09-10T08:25:04.640 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6, 2024-09-10T08:25:04.641 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.010686902, 2024-09-10T08:25:04.641 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001781150 2024-09-10T08:25:04.641 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.641 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2024-09-10T08:25:04.641 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.641 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-09-10T08:25:04.641 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 20, 2024-09-10T08:25:04.641 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 22, 2024-09-10T08:25:04.641 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 2, 2024-09-10T08:25:04.641 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 19, 2024-09-10T08:25:04.642 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 22, 2024-09-10T08:25:04.642 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 3, 2024-09-10T08:25:04.642 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 16, 2024-09-10T08:25:04.642 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 20, 2024-09-10T08:25:04.642 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 4, 2024-09-10T08:25:04.642 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-09-10T08:25:04.642 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 4, 2024-09-10T08:25:04.642 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 4, 2024-09-10T08:25:04.642 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 50492, 2024-09-10T08:25:04.642 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 223520 2024-09-10T08:25:04.643 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.643 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-09-10T08:25:04.643 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 11, 2024-09-10T08:25:04.643 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-09-10T08:25:04.643 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 7, 2024-09-10T08:25:04.643 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 22, 2024-09-10T08:25:04.643 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 1, 2024-09-10T08:25:04.643 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-09-10T08:25:04.643 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.643 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.643 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.643 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.644 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-09-10T08:25:04.644 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-09-10T08:25:04.644 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001511333, 2024-09-10T08:25:04.644 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000755666 2024-09-10T08:25:04.644 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.644 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-09-10T08:25:04.644 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.644 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.644 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.644 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.644 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-09-10T08:25:04.645 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.645 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.645 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.645 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.645 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-09-10T08:25:04.645 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-09-10T08:25:04.645 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000438091, 2024-09-10T08:25:04.645 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000219045 2024-09-10T08:25:04.645 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.645 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-09-10T08:25:04.645 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.646 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.646 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.646 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.646 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-09-10T08:25:04.646 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.646 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.646 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.646 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.646 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-09-10T08:25:04.646 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.646 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.646 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.647 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.647 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-09-10T08:25:04.647 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.647 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.647 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.647 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.647 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-09-10T08:25:04.647 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.647 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.647 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.647 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.648 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-09-10T08:25:04.648 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.648 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.648 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.648 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.648 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-09-10T08:25:04.649 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.649 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.649 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.649 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.649 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-09-10T08:25:04.649 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.649 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.650 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.650 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.650 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-09-10T08:25:04.650 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.650 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.650 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.650 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.650 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-09-10T08:25:04.650 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.650 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.650 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.651 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.651 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-09-10T08:25:04.651 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.651 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.651 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.651 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.651 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-09-10T08:25:04.651 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.651 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.651 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.651 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.651 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-09-10T08:25:04.652 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.652 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.652 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.652 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.652 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-09-10T08:25:04.652 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.652 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.653 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.653 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.653 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-09-10T08:25:04.653 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.653 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.653 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.653 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.653 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-09-10T08:25:04.653 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.653 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.654 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.654 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.654 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-09-10T08:25:04.654 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.654 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.654 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.654 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.654 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-09-10T08:25:04.654 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.654 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.655 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.655 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.655 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-09-10T08:25:04.655 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.655 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.655 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.655 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.655 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-09-10T08:25:04.655 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.655 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.655 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.656 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.656 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-09-10T08:25:04.656 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.656 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.656 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.656 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.656 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-09-10T08:25:04.656 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.656 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.656 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.657 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.657 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-09-10T08:25:04.657 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.657 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.657 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.657 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.657 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-09-10T08:25:04.657 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.657 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.657 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.658 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.658 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-09-10T08:25:04.658 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-09-10T08:25:04.658 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-09-10T08:25:04.658 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.658 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.658 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.658 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:04.658 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.658 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-09-10T08:25:04.658 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2024-09-10T08:25:04.659 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 3, 2024-09-10T08:25:04.659 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 2, 2024-09-10T08:25:04.659 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 1, 2024-09-10T08:25:04.659 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-09-10T08:25:04.659 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 3, 2024-09-10T08:25:04.659 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 1, 2024-09-10T08:25:04.659 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 1004, 2024-09-10T08:25:04.659 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-09-10T08:25:04.659 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.659 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-09-10T08:25:04.659 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:25:04.660 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-09-10T08:25:04.660 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:25:04.660 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:25:04.660 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:25:04.660 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:25:04.660 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:25:04.660 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:25:04.660 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:25:04.660 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:25:04.660 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:25:04.660 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:25:04.661 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:25:04.661 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:25:04.661 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:25:04.661 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-09-10T08:25:04.661 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:25:04.661 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-09-10T08:25:04.661 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:25:04.661 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:25:04.661 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:25:04.661 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:25:04.661 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:25:04.662 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:25:04.662 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:25:04.662 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-09-10T08:25:04.662 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:25:04.662 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:25:04.662 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:25:04.662 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-09-10T08:25:04.662 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-09-10T08:25:04.662 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-09-10T08:25:04.662 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:25:04.662 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:25:04.662 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:25:04.663 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:25:04.663 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 239365, 2024-09-10T08:25:04.663 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 82, 2024-09-10T08:25:04.663 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:25:04.663 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-09-10T08:25:04.663 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-09-10T08:25:04.663 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-09-10T08:25:04.664 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:25:04.664 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-09-10T08:25:04.664 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:25:04.664 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-09-10T08:25:04.664 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 9144, 2024-09-10T08:25:04.664 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 172, 2024-09-10T08:25:04.664 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:25:04.664 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:25:04.664 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-09-10T08:25:04.664 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-09-10T08:25:04.664 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 81077, 2024-09-10T08:25:04.665 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 341, 2024-09-10T08:25:04.665 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-09-10T08:25:04.665 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-09-10T08:25:04.665 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-09-10T08:25:04.665 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-09-10T08:25:04.665 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.665 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-09-10T08:25:04.666 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-09-10T08:25:04.666 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-09-10T08:25:04.666 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 45, 2024-09-10T08:25:04.666 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 12584, 2024-09-10T08:25:04.666 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 28, 2024-09-10T08:25:04.666 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 45, 2024-09-10T08:25:04.666 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-09-10T08:25:04.666 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 45, 2024-09-10T08:25:04.666 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.237994898, 2024-09-10T08:25:04.666 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.005288775 2024-09-10T08:25:04.666 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.667 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-09-10T08:25:04.667 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-09-10T08:25:04.667 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 45, 2024-09-10T08:25:04.667 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 69 2024-09-10T08:25:04.667 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.667 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 45, 2024-09-10T08:25:04.667 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 0, 2024-09-10T08:25:04.667 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 45, 2024-09-10T08:25:04.667 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-09-10T08:25:04.667 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-09-10T08:25:04.667 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 0, 2024-09-10T08:25:04.667 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 10, 2024-09-10T08:25:04.667 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 0, 2024-09-10T08:25:04.668 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 6, 2024-09-10T08:25:04.668 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 8, 2024-09-10T08:25:04.668 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-09-10T08:25:04.668 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-09-10T08:25:04.668 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-09-10T08:25:04.668 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-09-10T08:25:04.668 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 6, 2024-09-10T08:25:04.668 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-09-10T08:25:04.668 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-09-10T08:25:04.668 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-09-10T08:25:04.669 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 0, 2024-09-10T08:25:04.669 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 20, 2024-09-10T08:25:04.669 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-09-10T08:25:04.669 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-09-10T08:25:04.669 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-09-10T08:25:04.669 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-09-10T08:25:04.669 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-09-10T08:25:04.669 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-09-10T08:25:04.669 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T08:25:04.669 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-09-10T08:25:04.669 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-09-10T08:25:04.669 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-09-10T08:25:04.670 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-09-10T08:25:04.670 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-09-10T08:25:04.670 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-09-10T08:25:04.670 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-09-10T08:25:04.670 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-09-10T08:25:04.670 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-09-10T08:25:04.670 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-09-10T08:25:04.670 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-09-10T08:25:04.670 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-09-10T08:25:04.670 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-09-10T08:25:04.670 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-09-10T08:25:04.670 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-09-10T08:25:04.670 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-09-10T08:25:04.671 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-09-10T08:25:04.671 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-09-10T08:25:04.671 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-09-10T08:25:04.671 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 95, 2024-09-10T08:25:04.671 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-09-10T08:25:04.671 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 2, 2024-09-10T08:25:04.671 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 9, 2024-09-10T08:25:04.671 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 15, 2024-09-10T08:25:04.671 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 6, 2024-09-10T08:25:04.671 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-09-10T08:25:04.671 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 17, 2024-09-10T08:25:04.671 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 0, 2024-09-10T08:25:04.672 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 2 2024-09-10T08:25:04.672 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.672 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-09-10T08:25:04.672 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2024-09-10T08:25:04.672 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-09-10T08:25:04.672 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-09-10T08:25:04.672 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-09-10T08:25:04.672 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.672 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-09-10T08:25:04.672 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 97 2024-09-10T08:25:04.672 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.673 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-09-10T08:25:04.673 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-09-10T08:25:04.673 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T08:25:04.673 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-09-10T08:25:04.673 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-09-10T08:25:04.673 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 0, 2024-09-10T08:25:04.673 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-09-10T08:25:04.673 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-09-10T08:25:04.674 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.674 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:25:04.674 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:04.674 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-09-10T08:25:04.674 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:04.674 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 2388, 2024-09-10T08:25:04.674 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 1064415, 2024-09-10T08:25:04.674 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:04.674 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 2388, 2024-09-10T08:25:04.674 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-09-10T08:25:04.674 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-09-10T08:25:04.675 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 2388, 2024-09-10T08:25:04.675 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1064415, 2024-09-10T08:25:04.675 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:04.675 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.675 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.675 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.675 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:04.675 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.675 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-09-10T08:25:04.675 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:04.675 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-09-10T08:25:04.675 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:04.676 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-09-10T08:25:04.676 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-09-10T08:25:04.676 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:04.676 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-09-10T08:25:04.676 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 45, 2024-09-10T08:25:04.676 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 12584, 2024-09-10T08:25:04.676 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 45, 2024-09-10T08:25:04.676 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 12584, 2024-09-10T08:25:04.676 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:04.676 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.676 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.676 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.677 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:04.677 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.677 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-09-10T08:25:04.677 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:04.677 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-09-10T08:25:04.677 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:04.677 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-09-10T08:25:04.677 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-09-10T08:25:04.677 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:04.677 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-09-10T08:25:04.678 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 45, 2024-09-10T08:25:04.678 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 45, 2024-09-10T08:25:04.678 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 45, 2024-09-10T08:25:04.678 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 45, 2024-09-10T08:25:04.678 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:04.678 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.678 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.678 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.678 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:04.678 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.679 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-09-10T08:25:04.679 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:04.679 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-09-10T08:25:04.679 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:04.679 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-09-10T08:25:04.679 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-09-10T08:25:04.679 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:04.679 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-09-10T08:25:04.679 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-09-10T08:25:04.679 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-09-10T08:25:04.679 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-09-10T08:25:04.679 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-09-10T08:25:04.680 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:04.680 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.680 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.680 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.680 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:04.680 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:04.680 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55d621a14960": { 2024-09-10T08:25:04.680 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:04.680 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-09-10T08:25:04.680 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:04.680 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 7, 2024-09-10T08:25:04.680 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 1967, 2024-09-10T08:25:04.681 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:04.681 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 7, 2024-09-10T08:25:04.681 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-09-10T08:25:04.681 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-09-10T08:25:04.681 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 6, 2024-09-10T08:25:04.681 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1967, 2024-09-10T08:25:04.681 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:04.681 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:04.681 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:04.681 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:04.681 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:04.682 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:04.682 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-10T08:25:05.384 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-10T08:25:05.384 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-10T08:25:05.384 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-10T08:25:05.384 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=15 2024-09-10T08:25:05.384 DEBUG:tasks.check_counter:Getting stats from l 2024-09-10T08:25:05.384 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-10T08:25:05.627 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:25:06.019 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:05 smithi155 ceph-mon[35405]: pgmap v895: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:25:06.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:05 smithi002 ceph-mon[30563]: pgmap v895: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:25:06.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:05 smithi107 ceph-mon[35048]: pgmap v895: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:25:06.186 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-10T08:25:06.187 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:25:06.187 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 0, 2024-09-10T08:25:06.187 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2, 2024-09-10T08:25:06.187 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 0, 2024-09-10T08:25:06.187 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 0, 2024-09-10T08:25:06.187 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2024-09-10T08:25:06.187 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 0, 2024-09-10T08:25:06.187 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.001040423, 2024-09-10T08:25:06.187 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T08:25:06.187 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.000211120, 2024-09-10T08:25:06.187 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:25:06.187 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:06.188 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:06.188 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:06.188 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:06.188 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:06.188 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:06.188 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:06.188 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:06.188 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:06.188 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:06.188 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:25:06.188 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:25:06.188 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2, 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 250, 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 320, 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 201536, 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 2, 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.016740452, 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.012539328, 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.001043315, 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 248, 2024-09-10T08:25:06.189 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.019761004, 2024-09-10T08:25:06.190 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000079681 2024-09-10T08:25:06.190 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:06.190 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:06.190 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:06.190 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:06.190 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:06.190 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:06.190 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T08:25:06.190 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 201536 2024-09-10T08:25:06.190 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:06.190 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:25:06.190 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 335, 2024-09-10T08:25:06.190 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 328, 2024-09-10T08:25:06.191 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 788704, 2024-09-10T08:25:06.191 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 201968, 2024-09-10T08:25:06.191 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 2, 2024-09-10T08:25:06.191 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 1, 2024-09-10T08:25:06.191 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.052749318, 2024-09-10T08:25:06.191 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.023453809, 2024-09-10T08:25:06.191 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.036557332, 2024-09-10T08:25:06.191 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:25:06.191 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:25:06.191 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 328, 2024-09-10T08:25:06.191 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.026477775, 2024-09-10T08:25:06.191 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000080724 2024-09-10T08:25:06.191 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:06.192 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:25:06.192 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:06.192 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:06.192 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:06.192 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:06.192 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 788704, 2024-09-10T08:25:06.192 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 201968 2024-09-10T08:25:06.192 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:06.192 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-09-10T08:25:06.192 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 0, 2024-09-10T08:25:06.192 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-09-10T08:25:06.192 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:06.192 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-09-10T08:25:06.193 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:25:06.193 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-09-10T08:25:06.193 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:25:06.193 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:25:06.193 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:25:06.193 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:25:06.193 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:25:06.193 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:25:06.193 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:25:06.193 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:25:06.193 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:25:06.193 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:25:06.193 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:25:06.194 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:25:06.194 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:25:06.194 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-09-10T08:25:06.194 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:25:06.194 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-09-10T08:25:06.194 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:25:06.194 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:25:06.194 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:25:06.194 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:25:06.194 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:25:06.194 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:25:06.194 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:25:06.194 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 148738, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 31, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:25:06.195 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-09-10T08:25:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-09-10T08:25:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-09-10T08:25:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:25:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-09-10T08:25:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:25:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-09-10T08:25:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 0, 2024-09-10T08:25:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 0, 2024-09-10T08:25:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:25:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:25:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-09-10T08:25:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-09-10T08:25:06.196 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 928, 2024-09-10T08:25:06.197 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 2, 2024-09-10T08:25:06.197 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-09-10T08:25:06.197 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-09-10T08:25:06.197 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-09-10T08:25:06.197 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-09-10T08:25:06.197 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-10T08:25:06.197 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:25:06.197 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-09-10T08:25:06.197 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-09-10T08:25:06.197 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-09-10T08:25:06.197 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 333, 2024-09-10T08:25:06.197 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 738858, 2024-09-10T08:25:06.197 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-09-10T08:25:06.198 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 333, 2024-09-10T08:25:06.198 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-09-10T08:25:06.198 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-09-10T08:25:06.198 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 333, 2024-09-10T08:25:06.198 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 738858, 2024-09-10T08:25:06.198 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-09-10T08:25:06.198 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-09-10T08:25:06.198 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-09-10T08:25:06.198 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-09-10T08:25:06.198 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:06.198 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-10T08:25:06.198 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-10T08:25:06.802 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-10T08:25:06.802 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-10T08:25:06.802 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-10T08:25:06.802 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-10T08:25:06.802 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T08:25:06.813 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T08:25:06.813 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T08:25:06.813 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T08:25:06.856 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-10T08:25:06.856 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T08:25:06.936 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T08:25:07.364 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:25:07.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:07 smithi002 ceph-mon[30563]: pgmap v896: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T08:25:07.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.986+0000 7f6bbd200640 1 -- 172.21.15.2:0/556685098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bb8102b90 msgr2=0x7f6bb8102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:07.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.986+0000 7f6bbd200640 1 --2- 172.21.15.2:0/556685098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bb8102b90 0x7f6bb8102f70 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009900 tx=0x7f6ba802a480 comp rx=0 tx=0).stop 2024-09-10T08:25:07.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.986+0000 7f6bbd200640 1 -- 172.21.15.2:0/556685098 shutdown_connections 2024-09-10T08:25:07.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.986+0000 7f6bbd200640 1 --2- 172.21.15.2:0/556685098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb810bfc0 0x7f6bb810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:07.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.986+0000 7f6bbd200640 1 --2- 172.21.15.2:0/556685098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb81034b0 0x7f6bb810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:07.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.986+0000 7f6bbd200640 1 --2- 172.21.15.2:0/556685098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bb8102b90 0x7f6bb8102f70 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:07.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.986+0000 7f6bbd200640 1 -- 172.21.15.2:0/556685098 >> 172.21.15.2:0/556685098 conn(0x7f6bb80fc610 msgr2=0x7f6bb80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:07.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.986+0000 7f6bbd200640 1 -- 172.21.15.2:0/556685098 shutdown_connections 2024-09-10T08:25:07.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.986+0000 7f6bbd200640 1 -- 172.21.15.2:0/556685098 wait complete. 2024-09-10T08:25:07.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.987+0000 7f6bbd200640 1 Processor -- start 2024-09-10T08:25:07.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.987+0000 7f6bbd200640 1 -- start start 2024-09-10T08:25:07.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.988+0000 7f6bbd200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb8102b90 0x7f6bb819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:07.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.988+0000 7f6bbd200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb81034b0 0x7f6bb819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:07.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.988+0000 7f6bbd200640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bb810bfc0 0x7f6bb819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:07.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.988+0000 7f6bbd200640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb8111e10 con 0x7f6bb810bfc0 2024-09-10T08:25:07.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.988+0000 7f6bbd200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb8111c90 con 0x7f6bb8102b90 2024-09-10T08:25:07.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.988+0000 7f6bbd200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb8111f90 con 0x7f6bb81034b0 2024-09-10T08:25:07.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.988+0000 7f6bb7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bb810bfc0 0x7f6bb819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:07.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.988+0000 7f6baea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb81034b0 0x7f6bb819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:07.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.988+0000 7f6baea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb81034b0 0x7f6bb819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:37708/0 (socket says 172.21.15.2:37708) 2024-09-10T08:25:07.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.988+0000 7f6baea00640 1 -- 172.21.15.2:0/2699363626 learned_addr learned my addr 172.21.15.2:0/2699363626 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:25:07.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.988+0000 7f6bb7e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bb810bfc0 0x7f6bb819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49062/0 (socket says 172.21.15.2:49062) 2024-09-10T08:25:07.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.988+0000 7f6bb7400640 1 --2- 172.21.15.2:0/2699363626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb8102b90 0x7f6bb819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:07.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.989+0000 7f6bb7e00640 1 -- 172.21.15.2:0/2699363626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb81034b0 msgr2=0x7f6bb819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:07.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.989+0000 7f6bb7e00640 1 --2- 172.21.15.2:0/2699363626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb81034b0 0x7f6bb819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:07.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.989+0000 7f6bb7e00640 1 -- 172.21.15.2:0/2699363626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb8102b90 msgr2=0x7f6bb819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:07.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.989+0000 7f6bb7e00640 1 --2- 172.21.15.2:0/2699363626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb8102b90 0x7f6bb819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:07.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.989+0000 7f6bb7e00640 1 -- 172.21.15.2:0/2699363626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb819f440 con 0x7f6bb810bfc0 2024-09-10T08:25:07.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.989+0000 7f6bb7e00640 1 --2- 172.21.15.2:0/2699363626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bb810bfc0 0x7f6bb819ed60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009fd0 tx=0x7f6bb000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:07.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.989+0000 7f6bb4c00640 1 -- 172.21.15.2:0/2699363626 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb000ce20 con 0x7f6bb810bfc0 2024-09-10T08:25:07.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.989+0000 7f6bb4c00640 1 -- 172.21.15.2:0/2699363626 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb0005020 con 0x7f6bb810bfc0 2024-09-10T08:25:07.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.989+0000 7f6bbd200640 1 -- 172.21.15.2:0/2699363626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb819f730 con 0x7f6bb810bfc0 2024-09-10T08:25:07.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.989+0000 7f6bb4c00640 1 -- 172.21.15.2:0/2699363626 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0010720 con 0x7f6bb810bfc0 2024-09-10T08:25:07.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.989+0000 7f6bbd200640 1 -- 172.21.15.2:0/2699363626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb81a7010 con 0x7f6bb810bfc0 2024-09-10T08:25:07.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.990+0000 7f6bbd200640 1 -- 172.21.15.2:0/2699363626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb8102f70 con 0x7f6bb810bfc0 2024-09-10T08:25:07.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.992+0000 7f6bb4c00640 1 -- 172.21.15.2:0/2699363626 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6bb0005540 con 0x7f6bb810bfc0 2024-09-10T08:25:07.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.993+0000 7f6bb4c00640 1 --2- 172.21.15.2:0/2699363626 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6b9c0a2c00 0x7f6b9c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:07.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.994+0000 7f6bb4c00640 1 -- 172.21.15.2:0/2699363626 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f6bb00cfa50 con 0x7f6bb810bfc0 2024-09-10T08:25:07.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.994+0000 7f6bb7400640 1 --2- 172.21.15.2:0/2699363626 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6b9c0a2c00 0x7f6b9c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:07.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.995+0000 7f6bb7400640 1 --2- 172.21.15.2:0/2699363626 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6b9c0a2c00 0x7f6b9c0a50c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8000c00 tx=0x7f6ba8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:07.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:07.998+0000 7f6bb4c00640 1 -- 172.21.15.2:0/2699363626 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6bb009c560 con 0x7f6bb810bfc0 2024-09-10T08:25:08.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:07 smithi155 ceph-mon[35405]: pgmap v896: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T08:25:08.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:07 smithi107 ceph-mon[35048]: pgmap v896: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-09-10T08:25:08.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.194+0000 7f6bbd200640 1 -- 172.21.15.2:0/2699363626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6bb81a72f0 con 0x7f6bb810bfc0 2024-09-10T08:25:08.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.194+0000 7f6bb4c00640 1 -- 172.21.15.2:0/2699363626 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v95) ==== 81+0+578 (secure 0 0 0) 0x7f6bb009be60 con 0x7f6bb810bfc0 2024-09-10T08:25:08.195 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3590511604 2024-09-11T07:58:03.548782+0000 2024-09-10T08:25:08.195 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/939674007 2024-09-11T07:58:03.548782+0000 2024-09-10T08:25:08.195 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2447931330 2024-09-11T07:58:03.548782+0000 2024-09-10T08:25:08.195 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/387296843 2024-09-11T07:58:03.548782+0000 2024-09-10T08:25:08.195 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1896108724 2024-09-11T07:57:36.359026+0000 2024-09-10T08:25:08.195 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2121891210 2024-09-11T07:57:36.359026+0000 2024-09-10T08:25:08.195 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/712393349 2024-09-11T07:57:36.359026+0000 2024-09-10T08:25:08.196 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/712393349 2024-09-11T07:57:36.359026+0000 2024-09-10T08:25:08.196 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3590511604 2024-09-11T07:58:03.548782+0000 2024-09-10T08:25:08.196 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1616837180 2024-09-11T07:57:36.359026+0000 2024-09-10T08:25:08.196 INFO:teuthology.orchestra.run.smithi002.stderr:listed 10 entries 2024-09-10T08:25:08.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.197+0000 7f6bbd200640 1 -- 172.21.15.2:0/2699363626 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6b9c0a2c00 msgr2=0x7f6b9c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:08.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.197+0000 7f6bbd200640 1 --2- 172.21.15.2:0/2699363626 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6b9c0a2c00 0x7f6b9c0a50c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8000c00 tx=0x7f6ba8005b50 comp rx=0 tx=0).stop 2024-09-10T08:25:08.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.197+0000 7f6bbd200640 1 -- 172.21.15.2:0/2699363626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bb810bfc0 msgr2=0x7f6bb819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:08.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.197+0000 7f6bbd200640 1 --2- 172.21.15.2:0/2699363626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bb810bfc0 0x7f6bb819ed60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009fd0 tx=0x7f6bb000ef10 comp rx=0 tx=0).stop 2024-09-10T08:25:08.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.197+0000 7f6bbd200640 1 -- 172.21.15.2:0/2699363626 shutdown_connections 2024-09-10T08:25:08.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.197+0000 7f6bbd200640 1 --2- 172.21.15.2:0/2699363626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bb810bfc0 0x7f6bb819ed60 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:08.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.197+0000 7f6bbd200640 1 --2- 172.21.15.2:0/2699363626 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f6b9c0a2c00 0x7f6b9c0a50c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:08.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.197+0000 7f6bbd200640 1 --2- 172.21.15.2:0/2699363626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb81034b0 0x7f6bb819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:08.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.197+0000 7f6bbd200640 1 --2- 172.21.15.2:0/2699363626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bb8102b90 0x7f6bb819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:08.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.197+0000 7f6bbd200640 1 -- 172.21.15.2:0/2699363626 >> 172.21.15.2:0/2699363626 conn(0x7f6bb80fc610 msgr2=0x7f6bb810d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:08.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.197+0000 7f6bbd200640 1 -- 172.21.15.2:0/2699363626 shutdown_connections 2024-09-10T08:25:08.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:08.197+0000 7f6bbd200640 1 -- 172.21.15.2:0/2699363626 wait complete. 2024-09-10T08:25:09.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:08 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2699363626' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:25:09.108 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T08:25:09.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:25:09.109 DEBUG:teuthology.orchestra.run.smithi002:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T08:25:09.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:08 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2699363626' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:25:09.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:08 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2699363626' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:25:09.193 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2024-09-10T08:25:09.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:25:09.193 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T08:25:09.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:25:09.243 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-09-10T08:25:09.279 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T08:25:09.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:25:09.280 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T08:25:09.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:25:09.378 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2024-09-10T08:25:09.692 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T08:25:09.703 INFO:tasks.cephadm:Teardown begin 2024-09-10T08:25:09.703 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T08:25:09.703 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:25:10.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:09 smithi002 ceph-mon[30563]: pgmap v897: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:25:10.082 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:25:10.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:09 smithi155 ceph-mon[35405]: pgmap v897: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:25:10.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:09 smithi107 ceph-mon[35048]: pgmap v897: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:25:10.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.704+0000 7f4966a00640 1 -- 172.21.15.2:0/3675685437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49601049c0 msgr2=0x7f4960108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:10.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.704+0000 7f4966a00640 1 --2- 172.21.15.2:0/3675685437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49601049c0 0x7f4960108a10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4948009900 tx=0x7f494802a400 comp rx=0 tx=0).stop 2024-09-10T08:25:10.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.704+0000 7f4966a00640 1 -- 172.21.15.2:0/3675685437 shutdown_connections 2024-09-10T08:25:10.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.704+0000 7f4966a00640 1 --2- 172.21.15.2:0/3675685437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960109140 0x7f4960110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:10.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.704+0000 7f4966a00640 1 --2- 172.21.15.2:0/3675685437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49601049c0 0x7f4960108a10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:10.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.704+0000 7f4966a00640 1 --2- 172.21.15.2:0/3675685437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4960104010 0x7f49601043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:10.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.704+0000 7f4966a00640 1 -- 172.21.15.2:0/3675685437 >> 172.21.15.2:0/3675685437 conn(0x7f49600ffa50 msgr2=0x7f4960101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:10.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.704+0000 7f4966a00640 1 -- 172.21.15.2:0/3675685437 shutdown_connections 2024-09-10T08:25:10.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.704+0000 7f4966a00640 1 -- 172.21.15.2:0/3675685437 wait complete. 2024-09-10T08:25:10.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.705+0000 7f4966a00640 1 Processor -- start 2024-09-10T08:25:10.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.705+0000 7f4966a00640 1 -- start start 2024-09-10T08:25:10.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.705+0000 7f4966a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4960104010 0x7f496019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:10.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.705+0000 7f4966a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49601049c0 0x7f496019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:10.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.705+0000 7f4966a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960109140 0x7f496019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:10.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.705+0000 7f4966a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4960114160 con 0x7f49601049c0 2024-09-10T08:25:10.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.705+0000 7f4966a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4960113fe0 con 0x7f4960109140 2024-09-10T08:25:10.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.705+0000 7f4966a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f49601142e0 con 0x7f4960104010 2024-09-10T08:25:10.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.706+0000 7f4957e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49601049c0 0x7f496019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:10.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.706+0000 7f495fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4960104010 0x7f496019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:10.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.706+0000 7f495fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4960104010 0x7f496019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:37720/0 (socket says 172.21.15.2:37720) 2024-09-10T08:25:10.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.706+0000 7f495fe00640 1 -- 172.21.15.2:0/1337322369 learned_addr learned my addr 172.21.15.2:0/1337322369 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:25:10.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.706+0000 7f4964a00640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960109140 0x7f496019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:10.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.706+0000 7f495fe00640 1 -- 172.21.15.2:0/1337322369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960109140 msgr2=0x7f496019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:10.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.706+0000 7f495fe00640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960109140 0x7f496019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:10.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.706+0000 7f495fe00640 1 -- 172.21.15.2:0/1337322369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49601049c0 msgr2=0x7f496019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:10.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.706+0000 7f495fe00640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49601049c0 0x7f496019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:10.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.706+0000 7f495fe00640 1 -- 172.21.15.2:0/1337322369 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496019f610 con 0x7f4960104010 2024-09-10T08:25:10.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.706+0000 7f4957e00640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49601049c0 0x7f496019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:25:10.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.706+0000 7f495fe00640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4960104010 0x7f496019a660 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f495000ea60 tx=0x7f495000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:10.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.707+0000 7f495d600640 1 -- 172.21.15.2:0/1337322369 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495000cde0 con 0x7f4960104010 2024-09-10T08:25:10.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.707+0000 7f495d600640 1 -- 172.21.15.2:0/1337322369 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49500035f0 con 0x7f4960104010 2024-09-10T08:25:10.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.707+0000 7f4966a00640 1 -- 172.21.15.2:0/1337322369 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496019f900 con 0x7f4960104010 2024-09-10T08:25:10.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.707+0000 7f495d600640 1 -- 172.21.15.2:0/1337322369 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950010670 con 0x7f4960104010 2024-09-10T08:25:10.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.707+0000 7f4966a00640 1 -- 172.21.15.2:0/1337322369 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49601a71e0 con 0x7f4960104010 2024-09-10T08:25:10.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.708+0000 7f4966a00640 1 -- 172.21.15.2:0/1337322369 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49601043f0 con 0x7f4960104010 2024-09-10T08:25:10.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.710+0000 7f495d600640 1 -- 172.21.15.2:0/1337322369 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f49500037a0 con 0x7f4960104010 2024-09-10T08:25:10.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.711+0000 7f495d600640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f49440a2b80 0x7f49440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:10.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.711+0000 7f495d600640 1 -- 172.21.15.2:0/1337322369 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f49500cf800 con 0x7f4960104010 2024-09-10T08:25:10.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.711+0000 7f4957e00640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f49440a2b80 0x7f49440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:10.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.712+0000 7f4957e00640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f49440a2b80 0x7f49440a5040 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4948009230 tx=0x7f49480058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:10.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.713+0000 7f495d600640 1 -- 172.21.15.2:0/1337322369 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f495009c3a0 con 0x7f4960104010 2024-09-10T08:25:10.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.935+0000 7f4966a00640 1 -- 172.21.15.2:0/1337322369 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f496019b9c0 con 0x7f4960104010 2024-09-10T08:25:10.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.936+0000 7f495d600640 1 -- 172.21.15.2:0/1337322369 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f495009bca0 con 0x7f4960104010 2024-09-10T08:25:10.937 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:25:10.937 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:25:10.937 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:25:10.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.939+0000 7f4966a00640 1 -- 172.21.15.2:0/1337322369 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f49440a2b80 msgr2=0x7f49440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:10.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.939+0000 7f4966a00640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f49440a2b80 0x7f49440a5040 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4948009230 tx=0x7f49480058d0 comp rx=0 tx=0).stop 2024-09-10T08:25:10.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.939+0000 7f4966a00640 1 -- 172.21.15.2:0/1337322369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4960104010 msgr2=0x7f496019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:10.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.939+0000 7f4966a00640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4960104010 0x7f496019a660 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f495000ea60 tx=0x7f495000ef20 comp rx=0 tx=0).stop 2024-09-10T08:25:10.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.939+0000 7f4966a00640 1 -- 172.21.15.2:0/1337322369 shutdown_connections 2024-09-10T08:25:10.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.939+0000 7f4966a00640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960109140 0x7f496019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:10.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.939+0000 7f4966a00640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49601049c0 0x7f496019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:10.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.939+0000 7f4966a00640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f49440a2b80 0x7f49440a5040 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:10.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.939+0000 7f4966a00640 1 --2- 172.21.15.2:0/1337322369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4960104010 0x7f496019a660 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:10.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.939+0000 7f4966a00640 1 -- 172.21.15.2:0/1337322369 >> 172.21.15.2:0/1337322369 conn(0x7f49600ffa50 msgr2=0x7f4960100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:10.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.939+0000 7f4966a00640 1 -- 172.21.15.2:0/1337322369 shutdown_connections 2024-09-10T08:25:10.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:10.939+0000 7f4966a00640 1 -- 172.21.15.2:0/1337322369 wait complete. 2024-09-10T08:25:11.784 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:25:12.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:11 smithi002 ceph-mon[30563]: pgmap v898: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:25:12.021 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:11 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1337322369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:25:12.153 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:11 smithi155 ceph-mon[35405]: pgmap v898: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:25:12.153 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:11 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1337322369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:25:12.164 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:25:12.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:11 smithi107 ceph-mon[35048]: pgmap v898: 129 pgs: 129 active+clean; 4.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:25:12.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:11 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1337322369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:25:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.805+0000 7fcf25400640 1 -- 172.21.15.2:0/465456173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf20106f10 msgr2=0x7fcf2010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.805+0000 7fcf25400640 1 --2- 172.21.15.2:0/465456173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf20106f10 0x7fcf2010ea90 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009ca0 tx=0x7fcf1802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:25:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.805+0000 7fcf25400640 1 -- 172.21.15.2:0/465456173 shutdown_connections 2024-09-10T08:25:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.805+0000 7fcf25400640 1 --2- 172.21.15.2:0/465456173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf20106f10 0x7fcf2010ea90 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.805+0000 7fcf25400640 1 --2- 172.21.15.2:0/465456173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf20102790 0x7fcf201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.805+0000 7fcf25400640 1 --2- 172.21.15.2:0/465456173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf20101de0 0x7fcf201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.805+0000 7fcf25400640 1 -- 172.21.15.2:0/465456173 >> 172.21.15.2:0/465456173 conn(0x7fcf200fd8c0 msgr2=0x7fcf200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.805+0000 7fcf25400640 1 -- 172.21.15.2:0/465456173 shutdown_connections 2024-09-10T08:25:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.805+0000 7fcf25400640 1 -- 172.21.15.2:0/465456173 wait complete. 2024-09-10T08:25:12.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.806+0000 7fcf25400640 1 Processor -- start 2024-09-10T08:25:12.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.806+0000 7fcf25400640 1 -- start start 2024-09-10T08:25:12.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.806+0000 7fcf25400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf20101de0 0x7fcf20198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:12.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.806+0000 7fcf25400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf20102790 0x7fcf20198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:12.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf25400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf20106f10 0x7fcf2019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf25400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcf20111fc0 con 0x7fcf20102790 2024-09-10T08:25:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf25400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcf20111e40 con 0x7fcf20106f10 2024-09-10T08:25:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf25400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcf20112140 con 0x7fcf20101de0 2024-09-10T08:25:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf1e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf20101de0 0x7fcf20198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf15e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf20102790 0x7fcf20198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf1e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf20101de0 0x7fcf20198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:37752/0 (socket says 172.21.15.2:37752) 2024-09-10T08:25:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf1e800640 1 -- 172.21.15.2:0/24221262 learned_addr learned my addr 172.21.15.2:0/24221262 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:25:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf1f200640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf20106f10 0x7fcf2019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf1e800640 1 -- 172.21.15.2:0/24221262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf20106f10 msgr2=0x7fcf2019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf1e800640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf20106f10 0x7fcf2019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf1e800640 1 -- 172.21.15.2:0/24221262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf20102790 msgr2=0x7fcf20198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf1e800640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf20102790 0x7fcf20198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf1e800640 1 -- 172.21.15.2:0/24221262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2019d3b0 con 0x7fcf20101de0 2024-09-10T08:25:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.807+0000 7fcf15e00640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf20102790 0x7fcf20198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:25:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.808+0000 7fcf1e800640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf20101de0 0x7fcf20198400 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000e9f0 tx=0x7fcf1000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.808+0000 7fcf17e00640 1 -- 172.21.15.2:0/24221262 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1000cd80 con 0x7fcf20101de0 2024-09-10T08:25:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.808+0000 7fcf25400640 1 -- 172.21.15.2:0/24221262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2019d6a0 con 0x7fcf20101de0 2024-09-10T08:25:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.808+0000 7fcf17e00640 1 -- 172.21.15.2:0/24221262 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf10010910 con 0x7fcf20101de0 2024-09-10T08:25:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.808+0000 7fcf17e00640 1 -- 172.21.15.2:0/24221262 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf100188f0 con 0x7fcf20101de0 2024-09-10T08:25:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.808+0000 7fcf25400640 1 -- 172.21.15.2:0/24221262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf201a4f80 con 0x7fcf20101de0 2024-09-10T08:25:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.809+0000 7fcf25400640 1 -- 172.21.15.2:0/24221262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf201021c0 con 0x7fcf20101de0 2024-09-10T08:25:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.811+0000 7fcf17e00640 1 -- 172.21.15.2:0/24221262 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcf10010ac0 con 0x7fcf20101de0 2024-09-10T08:25:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.812+0000 7fcf17e00640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf040a2b80 0x7fcf040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.813+0000 7fcf17e00640 1 -- 172.21.15.2:0/24221262 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7fcf100cfd80 con 0x7fcf20101de0 2024-09-10T08:25:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.813+0000 7fcf15e00640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf040a2b80 0x7fcf040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:12.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.813+0000 7fcf15e00640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf040a2b80 0x7fcf040a5040 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcf200fdce0 tx=0x7fcf08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:12.815+0000 7fcf17e00640 1 -- 172.21.15.2:0/24221262 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf1009c240 con 0x7fcf20101de0 2024-09-10T08:25:13.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.035+0000 7fcf25400640 1 -- 172.21.15.2:0/24221262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf2002d7c0 con 0x7fcf20101de0 2024-09-10T08:25:13.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.036+0000 7fcf17e00640 1 -- 172.21.15.2:0/24221262 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fcf100a0540 con 0x7fcf20101de0 2024-09-10T08:25:13.037 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:25:13.037 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"btime":"2024-09-10T08:08:19:853909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1979555440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1979555440},{"type":"v1","addr":"172.21.15.2:6835","nonce":1979555440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2248723056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2248723056},{"type":"v1","addr":"172.21.15.2:6837","nonce":2248723056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/851586467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":851586467},{"type":"v1","addr":"172.21.15.2:6839","nonce":851586467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/91146142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":91146142},{"type":"v1","addr":"172.21.15.2:6841","nonce":91146142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1276815741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1276815741},{"type":"v1","addr":"172.21.15.155:6833","nonce":1276815741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3176720601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3176720601},{"type":"v1","addr":"172.21.15.107:6833","nonce":3176720601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:05:26.574488+0000","modified":"2024-09-10T08:08:19.294639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24499,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/1775771400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1775771400},{"type":"v1","addr":"172.21.15.107:6835","nonce":1775771400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1068995103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1068995103},{"type":"v1","addr":"172.21.15.155:6835","nonce":1068995103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3780222531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3780222531},{"type":"v1","addr":"172.21.15.155:6837","nonce":3780222531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.107:6837/3130379099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3130379099},{"type":"v1","addr":"172.21.15.107:6837","nonce":3130379099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.155:6839/2871792117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2871792117},{"type":"v1","addr":"172.21.15.155:6839","nonce":2871792117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.107:6839/3674188134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3674188134},{"type":"v1","addr":"172.21.15.107:6839","nonce":3674188134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24499,24494]},"id":1}]} 2024-09-10T08:25:13.037 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-09-10T08:25:13.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.038+0000 7fcf25400640 1 -- 172.21.15.2:0/24221262 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf040a2b80 msgr2=0x7fcf040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:13.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.038+0000 7fcf25400640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf040a2b80 0x7fcf040a5040 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcf200fdce0 tx=0x7fcf08005f90 comp rx=0 tx=0).stop 2024-09-10T08:25:13.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.038+0000 7fcf25400640 1 -- 172.21.15.2:0/24221262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf20101de0 msgr2=0x7fcf20198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:13.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.038+0000 7fcf25400640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf20101de0 0x7fcf20198400 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000e9f0 tx=0x7fcf1000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:25:13.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.039+0000 7fcf25400640 1 -- 172.21.15.2:0/24221262 shutdown_connections 2024-09-10T08:25:13.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.039+0000 7fcf25400640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf20106f10 0x7fcf2019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:13.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.039+0000 7fcf25400640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf20102790 0x7fcf20198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:13.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.039+0000 7fcf25400640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fcf040a2b80 0x7fcf040a5040 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:13.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.039+0000 7fcf25400640 1 --2- 172.21.15.2:0/24221262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf20101de0 0x7fcf20198400 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:13.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.039+0000 7fcf25400640 1 -- 172.21.15.2:0/24221262 >> 172.21.15.2:0/24221262 conn(0x7fcf200fd8c0 msgr2=0x7fcf200feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:13.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.039+0000 7fcf25400640 1 -- 172.21.15.2:0/24221262 shutdown_connections 2024-09-10T08:25:13.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:13.039+0000 7fcf25400640 1 -- 172.21.15.2:0/24221262 wait complete. 2024-09-10T08:25:13.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:13 smithi002 ceph-mon[30563]: pgmap v899: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-10T08:25:13.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:13 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/24221262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:25:13.950 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:25:14.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:13 smithi155 ceph-mon[35405]: pgmap v899: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-10T08:25:14.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:13 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/24221262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:25:14.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:13 smithi107 ceph-mon[35048]: pgmap v899: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-10T08:25:14.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:13 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/24221262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:25:14.330 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:25:15.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.996+0000 7f43d1c00640 1 -- 172.21.15.2:0/1108760450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43cc103e30 msgr2=0x7f43cc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.996+0000 7f43d1c00640 1 --2- 172.21.15.2:0/1108760450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43cc103e30 0x7f43cc104210 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009900 tx=0x7f43bc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:25:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.996+0000 7f43d1c00640 1 -- 172.21.15.2:0/1108760450 shutdown_connections 2024-09-10T08:25:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.996+0000 7f43d1c00640 1 --2- 172.21.15.2:0/1108760450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43cc108f60 0x7f43cc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.996+0000 7f43d1c00640 1 --2- 172.21.15.2:0/1108760450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc1047e0 0x7f43cc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.996+0000 7f43d1c00640 1 --2- 172.21.15.2:0/1108760450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43cc103e30 0x7f43cc104210 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.996+0000 7f43d1c00640 1 -- 172.21.15.2:0/1108760450 >> 172.21.15.2:0/1108760450 conn(0x7f43cc0ff910 msgr2=0x7f43cc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.996+0000 7f43d1c00640 1 -- 172.21.15.2:0/1108760450 shutdown_connections 2024-09-10T08:25:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.996+0000 7f43d1c00640 1 -- 172.21.15.2:0/1108760450 wait complete. 2024-09-10T08:25:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.997+0000 7f43d1c00640 1 Processor -- start 2024-09-10T08:25:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.997+0000 7f43d1c00640 1 -- start start 2024-09-10T08:25:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.998+0000 7f43d1c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43cc103e30 0x7f43cc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.998+0000 7f43d1c00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43cc1047e0 0x7f43cc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.998+0000 7f43d1c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc108f60 0x7f43cc19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.998+0000 7f43d1c00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc113f80 con 0x7f43cc1047e0 2024-09-10T08:25:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.998+0000 7f43d1c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc113e00 con 0x7f43cc103e30 2024-09-10T08:25:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.998+0000 7f43d1c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc114100 con 0x7f43cc108f60 2024-09-10T08:25:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.998+0000 7f43cb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43cc1047e0 0x7f43cc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.998+0000 7f43cb400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43cc1047e0 0x7f43cc19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49108/0 (socket says 172.21.15.2:49108) 2024-09-10T08:25:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.998+0000 7f43cb400640 1 -- 172.21.15.2:0/1177141840 learned_addr learned my addr 172.21.15.2:0/1177141840 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:25:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.998+0000 7f43cbe00640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43cc103e30 0x7f43cc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.998+0000 7f43d1200640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc108f60 0x7f43cc19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.999+0000 7f43cb400640 1 -- 172.21.15.2:0/1177141840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc108f60 msgr2=0x7f43cc19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.999+0000 7f43cb400640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc108f60 0x7f43cc19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.999+0000 7f43cb400640 1 -- 172.21.15.2:0/1177141840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43cc103e30 msgr2=0x7f43cc19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.999+0000 7f43cb400640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43cc103e30 0x7f43cc19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.999+0000 7f43cb400640 1 -- 172.21.15.2:0/1177141840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc19f430 con 0x7f43cc1047e0 2024-09-10T08:25:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.999+0000 7f43cb400640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43cc1047e0 0x7f43cc19a9c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f43b800d950 tx=0x7f43b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.999+0000 7f43c8c00640 1 -- 172.21.15.2:0/1177141840 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8017070 con 0x7f43cc1047e0 2024-09-10T08:25:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.999+0000 7f43c8c00640 1 -- 172.21.15.2:0/1177141840 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b8009c30 con 0x7f43cc1047e0 2024-09-10T08:25:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.999+0000 7f43c8c00640 1 -- 172.21.15.2:0/1177141840 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b800f6b0 con 0x7f43cc1047e0 2024-09-10T08:25:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:14.999+0000 7f43d1c00640 1 -- 172.21.15.2:0/1177141840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43cc19f720 con 0x7f43cc1047e0 2024-09-10T08:25:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.000+0000 7f43d1c00640 1 -- 172.21.15.2:0/1177141840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43cc1a7000 con 0x7f43cc1047e0 2024-09-10T08:25:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.001+0000 7f43d1c00640 1 -- 172.21.15.2:0/1177141840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43cc105310 con 0x7f43cc1047e0 2024-09-10T08:25:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.003+0000 7f43c8c00640 1 -- 172.21.15.2:0/1177141840 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f43b80109d0 con 0x7f43cc1047e0 2024-09-10T08:25:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.004+0000 7f43c8c00640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f43b00a2c00 0x7f43b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.005+0000 7f43c8c00640 1 -- 172.21.15.2:0/1177141840 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f43b80cefd0 con 0x7f43cc1047e0 2024-09-10T08:25:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.005+0000 7f43cbe00640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f43b00a2c00 0x7f43b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.006+0000 7f43cbe00640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f43b00a2c00 0x7f43b00a50c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009230 tx=0x7f43bc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:15.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.009+0000 7f43c8c00640 1 -- 172.21.15.2:0/1177141840 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43b8009de0 con 0x7f43cc1047e0 2024-09-10T08:25:15.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.194+0000 7f43d1c00640 1 -- 172.21.15.2:0/1177141840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43cc10c1a0 con 0x7f43cc1047e0 2024-09-10T08:25:15.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.196+0000 7f43c8c00640 1 -- 172.21.15.2:0/1177141840 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24887 (secure 0 0 0) 0x7f43b809b490 con 0x7f43cc1047e0 2024-09-10T08:25:15.197 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-10T08:25:15.197 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":95,"fsid":"26ec64b0-6f4a-11ef-bcea-c7b262605968","created":"2024-09-10T07:57:11.261625+0000","modified":"2024-09-10T08:07:05.504273+0000","last_up_change":"2024-09-10T08:04:30.003447+0000","last_in_change":"2024-09-10T08:04:07.176624+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:00:26.170464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:05:21.042136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:05:24.120081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:05:34.042103+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"41eb8073-0ba0-4e03-a60a-e7b27476ba0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6803","nonce":3825226139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6805","nonce":3825226139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6809","nonce":3825226139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3825226139},{"type":"v1","addr":"172.21.15.2:6807","nonce":3825226139}]},"public_addr":"172.21.15.2:6803/3825226139","cluster_addr":"172.21.15.2:6805/3825226139","heartbeat_back_addr":"172.21.15.2:6809/3825226139","heartbeat_front_addr":"172.21.15.2:6807/3825226139","state":["exists","up"]},{"osd":1,"uuid":"c0f1be1f-dfb5-4fd9-ab1a-0b1831000ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6801","nonce":3710092441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6803","nonce":3710092441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6807","nonce":3710092441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3710092441},{"type":"v1","addr":"172.21.15.107:6805","nonce":3710092441}]},"public_addr":"172.21.15.107:6801/3710092441","cluster_addr":"172.21.15.107:6803/3710092441","heartbeat_back_addr":"172.21.15.107:6807/3710092441","heartbeat_front_addr":"172.21.15.107:6805/3710092441","state":["exists","up"]},{"osd":2,"uuid":"f1c5e964-c621-4441-a87f-17c96c05de10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6801","nonce":158516707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6803","nonce":158516707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6807","nonce":158516707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":158516707},{"type":"v1","addr":"172.21.15.155:6805","nonce":158516707}]},"public_addr":"172.21.15.155:6801/158516707","cluster_addr":"172.21.15.155:6803/158516707","heartbeat_back_addr":"172.21.15.155:6807/158516707","heartbeat_front_addr":"172.21.15.155:6805/158516707","state":["exists","up"]},{"osd":3,"uuid":"400fc872-8c48-40e3-8ecb-b34778d142a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6811","nonce":713998144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6813","nonce":713998144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6817","nonce":713998144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":713998144},{"type":"v1","addr":"172.21.15.2:6815","nonce":713998144}]},"public_addr":"172.21.15.2:6811/713998144","cluster_addr":"172.21.15.2:6813/713998144","heartbeat_back_addr":"172.21.15.2:6817/713998144","heartbeat_front_addr":"172.21.15.2:6815/713998144","state":["exists","up"]},{"osd":4,"uuid":"269f71aa-0f17-4260-ad01-452fdd1b997f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6809","nonce":485793971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6811","nonce":485793971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6815","nonce":485793971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":485793971},{"type":"v1","addr":"172.21.15.107:6813","nonce":485793971}]},"public_addr":"172.21.15.107:6809/485793971","cluster_addr":"172.21.15.107:6811/485793971","heartbeat_back_addr":"172.21.15.107:6815/485793971","heartbeat_front_addr":"172.21.15.107:6813/485793971","state":["exists","up"]},{"osd":5,"uuid":"c7f0af26-b1a3-428b-a75f-f54956e40bf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6809","nonce":2180279362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6811","nonce":2180279362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6815","nonce":2180279362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2180279362},{"type":"v1","addr":"172.21.15.155:6813","nonce":2180279362}]},"public_addr":"172.21.15.155:6809/2180279362","cluster_addr":"172.21.15.155:6811/2180279362","heartbeat_back_addr":"172.21.15.155:6815/2180279362","heartbeat_front_addr":"172.21.15.155:6813/2180279362","state":["exists","up"]},{"osd":6,"uuid":"b0dc96e4-e62c-44ce-9902-7c168af07b9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6819","nonce":854683508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6821","nonce":854683508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6825","nonce":854683508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":854683508},{"type":"v1","addr":"172.21.15.2:6823","nonce":854683508}]},"public_addr":"172.21.15.2:6819/854683508","cluster_addr":"172.21.15.2:6821/854683508","heartbeat_back_addr":"172.21.15.2:6825/854683508","heartbeat_front_addr":"172.21.15.2:6823/854683508","state":["exists","up"]},{"osd":7,"uuid":"8feffc44-c4f3-4e5d-a59f-8ab00b949179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6817","nonce":1479477994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6819","nonce":1479477994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6823","nonce":1479477994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1479477994},{"type":"v1","addr":"172.21.15.107:6821","nonce":1479477994}]},"public_addr":"172.21.15.107:6817/1479477994","cluster_addr":"172.21.15.107:6819/1479477994","heartbeat_back_addr":"172.21.15.107:6823/1479477994","heartbeat_front_addr":"172.21.15.107:6821/1479477994","state":["exists","up"]},{"osd":8,"uuid":"4f9de660-f542-428d-bbdb-a932abc8d4cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6817","nonce":359284864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6819","nonce":359284864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6823","nonce":359284864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":359284864},{"type":"v1","addr":"172.21.15.155:6821","nonce":359284864}]},"public_addr":"172.21.15.155:6817/359284864","cluster_addr":"172.21.15.155:6819/359284864","heartbeat_back_addr":"172.21.15.155:6823/359284864","heartbeat_front_addr":"172.21.15.155:6821/359284864","state":["exists","up"]},{"osd":9,"uuid":"07f3cee5-642d-4887-86ed-46293be9fe24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6827","nonce":758080099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6829","nonce":758080099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6833","nonce":758080099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":758080099},{"type":"v1","addr":"172.21.15.2:6831","nonce":758080099}]},"public_addr":"172.21.15.2:6827/758080099","cluster_addr":"172.21.15.2:6829/758080099","heartbeat_back_addr":"172.21.15.2:6833/758080099","heartbeat_front_addr":"172.21.15.2:6831/758080099","state":["exists","up"]},{"osd":10,"uuid":"2fbd17c0-c632-4ab1-aea8-b6db91498bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6825","nonce":3646489493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6827","nonce":3646489493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6831","nonce":3646489493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3646489493},{"type":"v1","addr":"172.21.15.107:6829","nonce":3646489493}]},"public_addr":"172.21.15.107:6825/3646489493","cluster_addr":"172.21.15.107:6827/3646489493","heartbeat_back_addr":"172.21.15.107:6831/3646489493","heartbeat_front_addr":"172.21.15.107:6829/3646489493","state":["exists","up"]},{"osd":11,"uuid":"e4caf962-e25f-4f3f-b3a7-a4495561ce6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6825","nonce":4052924241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6827","nonce":4052924241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6831","nonce":4052924241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":4052924241},{"type":"v1","addr":"172.21.15.155:6829","nonce":4052924241}]},"public_addr":"172.21.15.155:6825/4052924241","cluster_addr":"172.21.15.155:6827/4052924241","heartbeat_back_addr":"172.21.15.155:6831/4052924241","heartbeat_front_addr":"172.21.15.155:6829/4052924241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:32.131023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T07:59:58.239705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:23.292710+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:00:52.990992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:19.398768+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:01:44.762435+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:15.213860+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:02:42.104142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:07.499612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:03:36.298575+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:03.324334+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:04:28.710025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/939674007":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/2447931330":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/387296843":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1896108724":"2024-09-11T07:57:36.359026+0000","172.21.15.2:0/2121891210":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6801/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/712393349":"2024-09-11T07:57:36.359026+0000","172.21.15.2:6800/3590511604":"2024-09-11T07:58:03.548782+0000","172.21.15.2:0/1616837180":"2024-09-11T07:57:36.359026+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:25:15.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.198+0000 7f43d1c00640 1 -- 172.21.15.2:0/1177141840 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f43b00a2c00 msgr2=0x7f43b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:15.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.198+0000 7f43d1c00640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f43b00a2c00 0x7f43b00a50c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009230 tx=0x7f43bc0058d0 comp rx=0 tx=0).stop 2024-09-10T08:25:15.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.198+0000 7f43d1c00640 1 -- 172.21.15.2:0/1177141840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43cc1047e0 msgr2=0x7f43cc19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:15.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.198+0000 7f43d1c00640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43cc1047e0 0x7f43cc19a9c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f43b800d950 tx=0x7f43b800de10 comp rx=0 tx=0).stop 2024-09-10T08:25:15.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.199+0000 7f43d1c00640 1 -- 172.21.15.2:0/1177141840 shutdown_connections 2024-09-10T08:25:15.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.199+0000 7f43d1c00640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f43b00a2c00 0x7f43b00a50c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:15.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.199+0000 7f43d1c00640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc108f60 0x7f43cc19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:15.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.199+0000 7f43d1c00640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43cc1047e0 0x7f43cc19a9c0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:15.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.199+0000 7f43d1c00640 1 --2- 172.21.15.2:0/1177141840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43cc103e30 0x7f43cc19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:15.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.199+0000 7f43d1c00640 1 -- 172.21.15.2:0/1177141840 >> 172.21.15.2:0/1177141840 conn(0x7f43cc0ff910 msgr2=0x7f43cc100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:15.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.199+0000 7f43d1c00640 1 -- 172.21.15.2:0/1177141840 shutdown_connections 2024-09-10T08:25:15.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:15.199+0000 7f43d1c00640 1 -- 172.21.15.2:0/1177141840 wait complete. 2024-09-10T08:25:15.871 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:15 smithi002 ceph-mon[30563]: pgmap v900: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T08:25:15.871 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:15 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1177141840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:25:15.909 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T08:25:16.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:15 smithi155 ceph-mon[35405]: pgmap v900: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T08:25:16.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:15 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1177141840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:25:16.166 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:15 smithi107 ceph-mon[35048]: pgmap v900: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-10T08:25:16.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:15 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1177141840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:25:16.287 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:25:16.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.872+0000 7f7cb0a00640 1 -- 172.21.15.2:0/4130911535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac1049a0 msgr2=0x7f7cac1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:16.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.872+0000 7f7cb0a00640 1 --2- 172.21.15.2:0/4130911535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac1049a0 0x7f7cac1089f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009900 tx=0x7f7c9402a410 comp rx=0 tx=0).stop 2024-09-10T08:25:16.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.872+0000 7f7cb0a00640 1 -- 172.21.15.2:0/4130911535 shutdown_connections 2024-09-10T08:25:16.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.872+0000 7f7cb0a00640 1 --2- 172.21.15.2:0/4130911535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac109120 0x7f7cac110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:16.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.872+0000 7f7cb0a00640 1 --2- 172.21.15.2:0/4130911535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac1049a0 0x7f7cac1089f0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:16.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.872+0000 7f7cb0a00640 1 --2- 172.21.15.2:0/4130911535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cac103ff0 0x7f7cac1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:16.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.872+0000 7f7cb0a00640 1 -- 172.21.15.2:0/4130911535 >> 172.21.15.2:0/4130911535 conn(0x7f7cac0ffa90 msgr2=0x7f7cac101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:16.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.872+0000 7f7cb0a00640 1 -- 172.21.15.2:0/4130911535 shutdown_connections 2024-09-10T08:25:16.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.872+0000 7f7cb0a00640 1 -- 172.21.15.2:0/4130911535 wait complete. 2024-09-10T08:25:16.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.873+0000 7f7cb0a00640 1 Processor -- start 2024-09-10T08:25:16.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.873+0000 7f7cb0a00640 1 -- start start 2024-09-10T08:25:16.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7cb0a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cac103ff0 0x7f7cac19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:16.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7cb0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac1049a0 0x7f7cac19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:16.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7cb0a00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac109120 0x7f7cac19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:16.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7cb0a00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac114120 con 0x7f7cac109120 2024-09-10T08:25:16.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7cb0a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac113fa0 con 0x7f7cac1049a0 2024-09-10T08:25:16.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7cb0a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac1142a0 con 0x7f7cac103ff0 2024-09-10T08:25:16.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7caa800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac109120 0x7f7cac19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:16.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7caa800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac109120 0x7f7cac19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48604/0 (socket says 172.21.15.2:48604) 2024-09-10T08:25:16.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7caa800640 1 -- 172.21.15.2:0/2782259234 learned_addr learned my addr 172.21.15.2:0/2782259234 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:25:16.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7ca9e00640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cac103ff0 0x7f7cac19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:16.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7ca3e00640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac1049a0 0x7f7cac19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:16.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7caa800640 1 -- 172.21.15.2:0/2782259234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cac103ff0 msgr2=0x7f7cac19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:16.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7caa800640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cac103ff0 0x7f7cac19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:16.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7caa800640 1 -- 172.21.15.2:0/2782259234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac1049a0 msgr2=0x7f7cac19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:16.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7caa800640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac1049a0 0x7f7cac19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:16.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.874+0000 7f7caa800640 1 -- 172.21.15.2:0/2782259234 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac19f650 con 0x7f7cac109120 2024-09-10T08:25:16.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.875+0000 7f7caa800640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac109120 0x7f7cac19ef70 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400eb10 tx=0x7f7ca400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:16.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.875+0000 7f7ca2a00640 1 -- 172.21.15.2:0/2782259234 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4004b20 con 0x7f7cac109120 2024-09-10T08:25:16.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.875+0000 7f7cb0a00640 1 -- 172.21.15.2:0/2782259234 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cac19f940 con 0x7f7cac109120 2024-09-10T08:25:16.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.875+0000 7f7ca2a00640 1 -- 172.21.15.2:0/2782259234 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca4004cc0 con 0x7f7cac109120 2024-09-10T08:25:16.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.875+0000 7f7cb0a00640 1 -- 172.21.15.2:0/2782259234 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cac1a7220 con 0x7f7cac109120 2024-09-10T08:25:16.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.875+0000 7f7ca2a00640 1 -- 172.21.15.2:0/2782259234 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4010700 con 0x7f7cac109120 2024-09-10T08:25:16.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.876+0000 7f7cb0a00640 1 -- 172.21.15.2:0/2782259234 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cac1043d0 con 0x7f7cac109120 2024-09-10T08:25:16.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.879+0000 7f7ca2a00640 1 -- 172.21.15.2:0/2782259234 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7ca4005000 con 0x7f7cac109120 2024-09-10T08:25:16.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.881+0000 7f7ca2a00640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7c900a2b80 0x7f7c900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:16.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.881+0000 7f7ca2a00640 1 -- 172.21.15.2:0/2782259234 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9237+0+0 (secure 0 0 0) 0x7f7ca40cfad0 con 0x7f7cac109120 2024-09-10T08:25:16.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.881+0000 7f7ca9e00640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7c900a2b80 0x7f7c900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:16.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.881+0000 7f7ca9e00640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7c900a2b80 0x7f7c900a5040 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009bd0 tx=0x7f7c9c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:16.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:16.884+0000 7f7ca2a00640 1 -- 172.21.15.2:0/2782259234 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ca409c5e0 con 0x7f7cac109120 2024-09-10T08:25:17.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.107+0000 7f7cb0a00640 1 -- 172.21.15.2:0/2782259234 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7cac10bdd0 con 0x7f7cac109120 2024-09-10T08:25:17.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.110+0000 7f7ca2a00640 1 -- 172.21.15.2:0/2782259234 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f7ca409bee0 con 0x7f7cac109120 2024-09-10T08:25:17.111 INFO:teuthology.orchestra.run.smithi002.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T08:25:17.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.113+0000 7f7cb0a00640 1 -- 172.21.15.2:0/2782259234 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7c900a2b80 msgr2=0x7f7c900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.113+0000 7f7cb0a00640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7c900a2b80 0x7f7c900a5040 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009bd0 tx=0x7f7c9c005f90 comp rx=0 tx=0).stop 2024-09-10T08:25:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.113+0000 7f7cb0a00640 1 -- 172.21.15.2:0/2782259234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac109120 msgr2=0x7f7cac19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.113+0000 7f7cb0a00640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac109120 0x7f7cac19ef70 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400eb10 tx=0x7f7ca400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:25:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.113+0000 7f7cb0a00640 1 -- 172.21.15.2:0/2782259234 shutdown_connections 2024-09-10T08:25:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.113+0000 7f7cb0a00640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cac109120 0x7f7cac19ef70 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.113+0000 7f7cb0a00640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac1049a0 0x7f7cac19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.113+0000 7f7cb0a00640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f7c900a2b80 0x7f7c900a5040 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.113+0000 7f7cb0a00640 1 --2- 172.21.15.2:0/2782259234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cac103ff0 0x7f7cac19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.113+0000 7f7cb0a00640 1 -- 172.21.15.2:0/2782259234 >> 172.21.15.2:0/2782259234 conn(0x7f7cac0ffa90 msgr2=0x7f7cac100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.113+0000 7f7cb0a00640 1 -- 172.21.15.2:0/2782259234 shutdown_connections 2024-09-10T08:25:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:17.113+0000 7f7cb0a00640 1 -- 172.21.15.2:0/2782259234 wait complete. 2024-09-10T08:25:17.407 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-i[65440]: ignoring --setuser ceph since I am not root 2024-09-10T08:25:17.407 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-i[65440]: ignoring --setgroup ceph since I am not root 2024-09-10T08:25:17.407 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-mds[65444]: -- 172.21.15.155:0/2027541129 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a38200b860 con 0x55a381fe9000 2024-09-10T08:25:17.407 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-i[65440]: starting mds.i at 2024-09-10T08:25:17.407 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-c[65139]: ignoring --setuser ceph since I am not root 2024-09-10T08:25:17.407 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-c[65139]: ignoring --setgroup ceph since I am not root 2024-09-10T08:25:17.408 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-mds[65143]: -- 172.21.15.155:0/1241875716 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dbc538bc20 con 0x55dbc5369000 2024-09-10T08:25:17.408 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-c[65139]: starting mds.c at 2024-09-10T08:25:17.408 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-f[64840]: ignoring --setuser ceph since I am not root 2024-09-10T08:25:17.408 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-f[64840]: ignoring --setgroup ceph since I am not root 2024-09-10T08:25:17.409 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-mds[64844]: -- 172.21.15.155:0/3648829029 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c1e3e01860 con 0x55c1e3dde800 2024-09-10T08:25:17.409 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-f[64840]: starting mds.f at 2024-09-10T08:25:17.417 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-e[63891]: ignoring --setuser ceph since I am not root 2024-09-10T08:25:17.417 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-e[63891]: ignoring --setgroup ceph since I am not root 2024-09-10T08:25:17.417 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-mds[63895]: -- 172.21.15.107:0/1639744414 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5638f3a29860 con 0x5638f3a07000 2024-09-10T08:25:17.417 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-e[63891]: starting mds.e at 2024-09-10T08:25:17.418 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-h[64193]: ignoring --setuser ceph since I am not root 2024-09-10T08:25:17.418 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-h[64193]: ignoring --setgroup ceph since I am not root 2024-09-10T08:25:17.418 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-mds[64197]: -- 172.21.15.107:0/2528008687 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5635af0d9860 con 0x5635af0b6800 2024-09-10T08:25:17.418 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-h[64193]: starting mds.h at 2024-09-10T08:25:17.418 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-b[64493]: ignoring --setuser ceph since I am not root 2024-09-10T08:25:17.418 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-b[64493]: ignoring --setgroup ceph since I am not root 2024-09-10T08:25:17.418 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-mds[64497]: -- 172.21.15.107:0/1888996944 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df7c7c1c20 con 0x55df7c79e800 2024-09-10T08:25:17.418 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-b[64493]: starting mds.b at 2024-09-10T08:25:17.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:17 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a[30559]: 2024-09-10T08:25:17.108+0000 7f16e9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T08:25:17.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:17 smithi002 ceph-mon[30563]: pgmap v901: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-10T08:25:17.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:17 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2782259234' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T08:25:17.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:17 smithi002 ceph-mon[30563]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T08:25:17.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:17 smithi002 ceph-mon[30563]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T08:25:17.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:17 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/2782259234' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T08:25:17.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:17 smithi002 ceph-mon[30563]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T08:25:17.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:17 smithi002 ceph-mon[30563]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-10T08:25:17.916 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T08:25:18.156 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-mon[35405]: pgmap v901: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-10T08:25:18.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2782259234' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T08:25:18.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-mon[35405]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T08:25:18.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-mon[35405]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T08:25:18.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/2782259234' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T08:25:18.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-mon[35405]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T08:25:18.157 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:17 smithi155 ceph-mon[35405]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-10T08:25:18.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-mon[35048]: pgmap v901: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-10T08:25:18.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2782259234' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T08:25:18.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-mon[35048]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T08:25:18.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-mon[35048]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T08:25:18.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/2782259234' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T08:25:18.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-mon[35048]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T08:25:18.167 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:17 smithi107 ceph-mon[35048]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-10T08:25:18.298 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:25:18.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.940+0000 7f2141400640 1 -- 172.21.15.2:0/348706300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f213c102360 msgr2=0x7f213c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:18.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.940+0000 7f2141400640 1 --2- 172.21.15.2:0/348706300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f213c102360 0x7f213c10e890 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f2134009930 tx=0x7f213402a420 comp rx=0 tx=0).stop 2024-09-10T08:25:18.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.940+0000 7f2141400640 1 -- 172.21.15.2:0/348706300 shutdown_connections 2024-09-10T08:25:18.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.940+0000 7f2141400640 1 --2- 172.21.15.2:0/348706300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f213c102360 0x7f213c10e890 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:18.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.940+0000 7f2141400640 1 --2- 172.21.15.2:0/348706300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f213c1019c0 0x7f213c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:18.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.940+0000 7f2141400640 1 --2- 172.21.15.2:0/348706300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f213c1079c0 0x7f213c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:18.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.940+0000 7f2141400640 1 -- 172.21.15.2:0/348706300 >> 172.21.15.2:0/348706300 conn(0x7f213c0fd740 msgr2=0x7f213c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:18.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.940+0000 7f2141400640 1 -- 172.21.15.2:0/348706300 shutdown_connections 2024-09-10T08:25:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.940+0000 7f2141400640 1 -- 172.21.15.2:0/348706300 wait complete. 2024-09-10T08:25:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.941+0000 7f2141400640 1 Processor -- start 2024-09-10T08:25:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.941+0000 7f2141400640 1 -- start start 2024-09-10T08:25:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2141400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f213c1019c0 0x7f213c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2141400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f213c102360 0x7f213c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2141400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f213c1079c0 0x7f213c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2141400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f213c111e10 con 0x7f213c1079c0 2024-09-10T08:25:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2141400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f213c111c90 con 0x7f213c102360 2024-09-10T08:25:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2141400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f213c111f90 con 0x7f213c1019c0 2024-09-10T08:25:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2133400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f213c102360 0x7f213c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2133400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f213c102360 0x7f213c19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37040/0 (socket says 172.21.15.2:37040) 2024-09-10T08:25:18.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2133400640 1 -- 172.21.15.2:0/1418365234 learned_addr learned my addr 172.21.15.2:0/1418365234 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:25:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2140a00640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f213c1079c0 0x7f213c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f213be00640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f213c1019c0 0x7f213c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2133400640 1 -- 172.21.15.2:0/1418365234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f213c1019c0 msgr2=0x7f213c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2133400640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f213c1019c0 0x7f213c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2133400640 1 -- 172.21.15.2:0/1418365234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f213c1079c0 msgr2=0x7f213c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2133400640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f213c1079c0 0x7f213c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.942+0000 7f2133400640 1 -- 172.21.15.2:0/1418365234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c19f440 con 0x7f213c102360 2024-09-10T08:25:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.943+0000 7f2133400640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f213c102360 0x7f213c19a9d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f212400ea60 tx=0x7f212400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.943+0000 7f2139600640 1 -- 172.21.15.2:0/1418365234 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212400cde0 con 0x7f213c102360 2024-09-10T08:25:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.943+0000 7f2139600640 1 -- 172.21.15.2:0/1418365234 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21240035f0 con 0x7f213c102360 2024-09-10T08:25:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.943+0000 7f2139600640 1 -- 172.21.15.2:0/1418365234 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124010670 con 0x7f213c102360 2024-09-10T08:25:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.943+0000 7f2141400640 1 -- 172.21.15.2:0/1418365234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213c19f730 con 0x7f213c102360 2024-09-10T08:25:18.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.943+0000 7f2141400640 1 -- 172.21.15.2:0/1418365234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f213c1a7010 con 0x7f213c102360 2024-09-10T08:25:18.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.944+0000 7f2141400640 1 -- 172.21.15.2:0/1418365234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f213c101e20 con 0x7f213c102360 2024-09-10T08:25:18.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.946+0000 7f2139600640 1 -- 172.21.15.2:0/1418365234 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f21240037a0 con 0x7f213c102360 2024-09-10T08:25:18.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.947+0000 7f2139600640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f21200a2c00 0x7f21200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:18.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.947+0000 7f2139600640 1 -- 172.21.15.2:0/1418365234 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9495+0+0 (secure 0 0 0) 0x7f21240cf7c0 con 0x7f213c102360 2024-09-10T08:25:18.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.947+0000 7f213be00640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f21200a2c00 0x7f21200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:18.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.948+0000 7f213be00640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f21200a2c00 0x7f21200a50c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f212c005fd0 tx=0x7f212c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:18.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:18.949+0000 7f2139600640 1 -- 172.21.15.2:0/1418365234 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f212409c1d0 con 0x7f213c102360 2024-09-10T08:25:19.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:19.173+0000 7f2141400640 1 -- 172.21.15.2:0/1418365234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f213c1a73a0 con 0x7f213c102360 2024-09-10T08:25:19.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: pgmap v903: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-10T08:25:19.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.155:6834/2154263130,v1:172.21.15.155:6835/2154263130] up:boot 2024-09-10T08:25:19.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.107:6834/1603666295,v1:172.21.15.107:6835/1603666295] up:boot 2024-09-10T08:25:19.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.107:6837/2116006895,v1:172.21.15.107:6839/2116006895] up:boot 2024-09-10T08:25:19.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.107:6836/4024391658,v1:172.21.15.107:6838/4024391658] up:boot 2024-09-10T08:25:19.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.155:6836/1724210291,v1:172.21.15.155:6838/1724210291] up:boot 2024-09-10T08:25:19.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: mds.? [v2:172.21.15.155:6837/3286757627,v1:172.21.15.155:6839/3286757627] up:boot 2024-09-10T08:25:19.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-10T08:25:19.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:25:19.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:25:19.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:25:19.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:25:19.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:25:19.408 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:19 smithi155 ceph-mon[35405]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:25:19.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: pgmap v903: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-10T08:25:19.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.155:6834/2154263130,v1:172.21.15.155:6835/2154263130] up:boot 2024-09-10T08:25:19.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.107:6834/1603666295,v1:172.21.15.107:6835/1603666295] up:boot 2024-09-10T08:25:19.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.107:6837/2116006895,v1:172.21.15.107:6839/2116006895] up:boot 2024-09-10T08:25:19.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.107:6836/4024391658,v1:172.21.15.107:6838/4024391658] up:boot 2024-09-10T08:25:19.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.155:6836/1724210291,v1:172.21.15.155:6838/1724210291] up:boot 2024-09-10T08:25:19.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: mds.? [v2:172.21.15.155:6837/3286757627,v1:172.21.15.155:6839/3286757627] up:boot 2024-09-10T08:25:19.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-10T08:25:19.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:25:19.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:25:19.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:25:19.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:25:19.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:25:19.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:19 smithi107 ceph-mon[35048]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:25:19.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: pgmap v903: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-10T08:25:19.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.155:6834/2154263130,v1:172.21.15.155:6835/2154263130] up:boot 2024-09-10T08:25:19.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.107:6834/1603666295,v1:172.21.15.107:6835/1603666295] up:boot 2024-09-10T08:25:19.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.107:6837/2116006895,v1:172.21.15.107:6839/2116006895] up:boot 2024-09-10T08:25:19.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.107:6836/4024391658,v1:172.21.15.107:6838/4024391658] up:boot 2024-09-10T08:25:19.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.155:6836/1724210291,v1:172.21.15.155:6838/1724210291] up:boot 2024-09-10T08:25:19.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: mds.? [v2:172.21.15.155:6837/3286757627,v1:172.21.15.155:6839/3286757627] up:boot 2024-09-10T08:25:19.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-10T08:25:19.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:25:19.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:25:19.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:25:19.521 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:25:19.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:25:19.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:19 smithi002 ceph-mon[30563]: from='mgr.14150 172.21.15.2:0/1556114616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:25:20.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:20.126+0000 7f2139600640 1 -- 172.21.15.2:0/1418365234 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f212409bad0 con 0x7f213c102360 2024-09-10T08:25:20.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:20.129+0000 7f2141400640 1 -- 172.21.15.2:0/1418365234 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f21200a2c00 msgr2=0x7f21200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:20.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:20.129+0000 7f2141400640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f21200a2c00 0x7f21200a50c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f212c005fd0 tx=0x7f212c009500 comp rx=0 tx=0).stop 2024-09-10T08:25:20.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:20.129+0000 7f2141400640 1 -- 172.21.15.2:0/1418365234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f213c102360 msgr2=0x7f213c19a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:20.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:20.129+0000 7f2141400640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f213c102360 0x7f213c19a9d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f212400ea60 tx=0x7f212400ef20 comp rx=0 tx=0).stop 2024-09-10T08:25:20.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:20.129+0000 7f2141400640 1 -- 172.21.15.2:0/1418365234 shutdown_connections 2024-09-10T08:25:20.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:20.129+0000 7f2141400640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f213c1079c0 0x7f213c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:20.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:20.129+0000 7f2141400640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f213c102360 0x7f213c19a9d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:20.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:20.129+0000 7f2141400640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f21200a2c00 0x7f21200a50c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:20.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:20.129+0000 7f2141400640 1 --2- 172.21.15.2:0/1418365234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f213c1019c0 0x7f213c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:20.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:20.129+0000 7f2141400640 1 -- 172.21.15.2:0/1418365234 >> 172.21.15.2:0/1418365234 conn(0x7f213c0fd740 msgr2=0x7f213c10d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:20.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:20.129+0000 7f2141400640 1 -- 172.21.15.2:0/1418365234 shutdown_connections 2024-09-10T08:25:20.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:20.129+0000 7f2141400640 1 -- 172.21.15.2:0/1418365234 wait complete. 2024-09-10T08:25:20.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:20 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1418365234' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:25:20.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:20 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:25:20.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:20 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1418365234' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:25:20.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:20 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:25:20.513 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:20 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1418365234' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:25:20.513 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:20 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:25:20.926 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T08:25:21.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:21 smithi002 ceph-mon[30563]: pgmap v904: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-10T08:25:21.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:21 smithi002 ceph-mon[30563]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T08:25:21.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:21 smithi002 ceph-mon[30563]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T08:25:21.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:21 smithi002 ceph-mon[30563]: Cluster is now healthy 2024-09-10T08:25:21.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:21 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:25:21.271 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:21 smithi002 ceph-mon[30563]: fsmap 12 up:standby 2024-09-10T08:25:21.312 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:25:21.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:21 smithi155 ceph-mon[35405]: pgmap v904: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-10T08:25:21.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:21 smithi155 ceph-mon[35405]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T08:25:21.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:21 smithi155 ceph-mon[35405]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T08:25:21.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:21 smithi155 ceph-mon[35405]: Cluster is now healthy 2024-09-10T08:25:21.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:21 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:25:21.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:21 smithi155 ceph-mon[35405]: fsmap 12 up:standby 2024-09-10T08:25:21.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:21 smithi107 ceph-mon[35048]: pgmap v904: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-10T08:25:21.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:21 smithi107 ceph-mon[35048]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T08:25:21.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:21 smithi107 ceph-mon[35048]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T08:25:21.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:21 smithi107 ceph-mon[35048]: Cluster is now healthy 2024-09-10T08:25:21.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:21 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:25:21.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:21 smithi107 ceph-mon[35048]: fsmap 12 up:standby 2024-09-10T08:25:21.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.891+0000 7f2c6c400640 1 -- 172.21.15.2:0/3334927374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c64076720 msgr2=0x7f2c64076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:21.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.891+0000 7f2c6c400640 1 --2- 172.21.15.2:0/3334927374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c64076720 0x7f2c64076b00 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009900 tx=0x7f2c5802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:25:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.891+0000 7f2c6c400640 1 -- 172.21.15.2:0/3334927374 shutdown_connections 2024-09-10T08:25:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.891+0000 7f2c6c400640 1 --2- 172.21.15.2:0/3334927374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c64112a50 0x7f2c64114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.891+0000 7f2c6c400640 1 --2- 172.21.15.2:0/3334927374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c64077040 0x7f2c64112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.891+0000 7f2c6c400640 1 --2- 172.21.15.2:0/3334927374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c64076720 0x7f2c64076b00 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.891+0000 7f2c6c400640 1 -- 172.21.15.2:0/3334927374 >> 172.21.15.2:0/3334927374 conn(0x7f2c640ffa50 msgr2=0x7f2c64101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.891+0000 7f2c6c400640 1 -- 172.21.15.2:0/3334927374 shutdown_connections 2024-09-10T08:25:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.891+0000 7f2c6c400640 1 -- 172.21.15.2:0/3334927374 wait complete. 2024-09-10T08:25:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.892+0000 7f2c6c400640 1 Processor -- start 2024-09-10T08:25:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.892+0000 7f2c6c400640 1 -- start start 2024-09-10T08:25:21.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.893+0000 7f2c6c400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c64076720 0x7f2c641a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.893+0000 7f2c6c400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c64077040 0x7f2c641a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.893+0000 7f2c6c400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c64112a50 0x7f2c6419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.893+0000 7f2c6c400640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c64118630 con 0x7f2c64112a50 2024-09-10T08:25:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.893+0000 7f2c6c400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641184b0 con 0x7f2c64076720 2024-09-10T08:25:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.893+0000 7f2c6c400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641187b0 con 0x7f2c64077040 2024-09-10T08:25:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.893+0000 7f2c6a400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c64112a50 0x7f2c6419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.893+0000 7f2c6a400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c64112a50 0x7f2c6419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48636/0 (socket says 172.21.15.2:48636) 2024-09-10T08:25:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.893+0000 7f2c6a400640 1 -- 172.21.15.2:0/1849074072 learned_addr learned my addr 172.21.15.2:0/1849074072 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:25:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.893+0000 7f2c69000640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c64077040 0x7f2c641a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.893+0000 7f2c69a00640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c64076720 0x7f2c641a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.894+0000 7f2c6a400640 1 -- 172.21.15.2:0/1849074072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c64077040 msgr2=0x7f2c641a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.894+0000 7f2c6a400640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c64077040 0x7f2c641a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.894+0000 7f2c6a400640 1 -- 172.21.15.2:0/1849074072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c64076720 msgr2=0x7f2c641a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.894+0000 7f2c6a400640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c64076720 0x7f2c641a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.894+0000 7f2c6a400640 1 -- 172.21.15.2:0/1849074072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6419f070 con 0x7f2c64112a50 2024-09-10T08:25:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.894+0000 7f2c6a400640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c64112a50 0x7f2c6419e940 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00eb10 tx=0x7f2c5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.894+0000 7f2c62000640 1 -- 172.21.15.2:0/1849074072 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c004b20 con 0x7f2c64112a50 2024-09-10T08:25:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.894+0000 7f2c62000640 1 -- 172.21.15.2:0/1849074072 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c5c004cc0 con 0x7f2c64112a50 2024-09-10T08:25:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.894+0000 7f2c62000640 1 -- 172.21.15.2:0/1849074072 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c010790 con 0x7f2c64112a50 2024-09-10T08:25:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.894+0000 7f2c6c400640 1 -- 172.21.15.2:0/1849074072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6419f360 con 0x7f2c64112a50 2024-09-10T08:25:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.894+0000 7f2c6c400640 1 -- 172.21.15.2:0/1849074072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c641ab650 con 0x7f2c64112a50 2024-09-10T08:25:21.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.895+0000 7f2c6c400640 1 -- 172.21.15.2:0/1849074072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c64076b00 con 0x7f2c64112a50 2024-09-10T08:25:21.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.897+0000 7f2c62000640 1 -- 172.21.15.2:0/1849074072 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2c5c005020 con 0x7f2c64112a50 2024-09-10T08:25:21.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.898+0000 7f2c62000640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c4c0a2c00 0x7f2c4c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:21.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.899+0000 7f2c62000640 1 -- 172.21.15.2:0/1849074072 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9495+0+0 (secure 0 0 0) 0x7f2c5c0cfbe0 con 0x7f2c64112a50 2024-09-10T08:25:21.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.899+0000 7f2c69a00640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c4c0a2c00 0x7f2c4c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:21.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.900+0000 7f2c69a00640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c4c0a2c00 0x7f2c4c0a50c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2c58006010 tx=0x7f2c58000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:21.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:21.903+0000 7f2c62000640 1 -- 172.21.15.2:0/1849074072 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c5c09c5f0 con 0x7f2c64112a50 2024-09-10T08:25:22.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.090+0000 7f2c6c400640 1 -- 172.21.15.2:0/1849074072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2c6402d7c0 con 0x7f2c64112a50 2024-09-10T08:25:22.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.124+0000 7f2c62000640 1 -- 172.21.15.2:0/1849074072 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) ==== 181+0+0 (secure 0 0 0) 0x7f2c5c09bef0 con 0x7f2c64112a50 2024-09-10T08:25:22.123 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' removed 2024-09-10T08:25:22.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.127+0000 7f2c6c400640 1 -- 172.21.15.2:0/1849074072 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c4c0a2c00 msgr2=0x7f2c4c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.127+0000 7f2c6c400640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c4c0a2c00 0x7f2c4c0a50c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2c58006010 tx=0x7f2c58000bc0 comp rx=0 tx=0).stop 2024-09-10T08:25:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.127+0000 7f2c6c400640 1 -- 172.21.15.2:0/1849074072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c64112a50 msgr2=0x7f2c6419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.127+0000 7f2c6c400640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c64112a50 0x7f2c6419e940 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00eb10 tx=0x7f2c5c00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:25:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.128+0000 7f2c6c400640 1 -- 172.21.15.2:0/1849074072 shutdown_connections 2024-09-10T08:25:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.128+0000 7f2c6c400640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c64112a50 0x7f2c6419e940 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.128+0000 7f2c6c400640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7f2c4c0a2c00 0x7f2c4c0a50c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:22.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.128+0000 7f2c6c400640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c64077040 0x7f2c641a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:22.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.128+0000 7f2c6c400640 1 --2- 172.21.15.2:0/1849074072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c64076720 0x7f2c641a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:22.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.128+0000 7f2c6c400640 1 -- 172.21.15.2:0/1849074072 >> 172.21.15.2:0/1849074072 conn(0x7f2c640ffa50 msgr2=0x7f2c64113650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:22.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.128+0000 7f2c6c400640 1 -- 172.21.15.2:0/1849074072 shutdown_connections 2024-09-10T08:25:22.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:22.128+0000 7f2c6c400640 1 -- 172.21.15.2:0/1849074072 wait complete. 2024-09-10T08:25:22.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:22 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1849074072' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:25:22.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:22 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1849074072' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:25:22.513 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:22 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1849074072' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:25:22.950 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T08:25:23.326 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:25:23.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:23 smithi155 ceph-mon[35405]: pgmap v905: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T08:25:23.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:23 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/1849074072' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:25:23.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:23 smithi155 ceph-mon[35405]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T08:25:23.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:23 smithi107 ceph-mon[35048]: pgmap v905: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T08:25:23.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:23 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/1849074072' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:25:23.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:23 smithi107 ceph-mon[35048]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T08:25:23.465 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:23 smithi002 ceph-mon[30563]: pgmap v905: 129 pgs: 129 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T08:25:23.465 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:23 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/1849074072' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:25:23.466 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:23 smithi002 ceph-mon[30563]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T08:25:24.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.011+0000 7fd89bc00640 1 -- 172.21.15.2:0/294641703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894074590 msgr2=0x7fd894074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:24.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.011+0000 7fd89bc00640 1 --2- 172.21.15.2:0/294641703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894074590 0x7fd894074970 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fd888009900 tx=0x7fd88802a480 comp rx=0 tx=0).stop 2024-09-10T08:25:24.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.012+0000 7fd89bc00640 1 -- 172.21.15.2:0/294641703 shutdown_connections 2024-09-10T08:25:24.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.012+0000 7fd89bc00640 1 --2- 172.21.15.2:0/294641703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd894112b90 0x7fd894114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:24.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.012+0000 7fd89bc00640 1 --2- 172.21.15.2:0/294641703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd894074eb0 0x7fd894112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:24.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.012+0000 7fd89bc00640 1 --2- 172.21.15.2:0/294641703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894074590 0x7fd894074970 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:24.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.012+0000 7fd89bc00640 1 -- 172.21.15.2:0/294641703 >> 172.21.15.2:0/294641703 conn(0x7fd8940fd8c0 msgr2=0x7fd8940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:24.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.012+0000 7fd89bc00640 1 -- 172.21.15.2:0/294641703 shutdown_connections 2024-09-10T08:25:24.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.012+0000 7fd89bc00640 1 -- 172.21.15.2:0/294641703 wait complete. 2024-09-10T08:25:24.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.013+0000 7fd89bc00640 1 Processor -- start 2024-09-10T08:25:24.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.013+0000 7fd89bc00640 1 -- start start 2024-09-10T08:25:24.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.013+0000 7fd89bc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd894074590 0x7fd8941a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:24.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.013+0000 7fd89bc00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894074eb0 0x7fd8941a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:24.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.013+0000 7fd89bc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd894112b90 0x7fd89419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:24.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.013+0000 7fd89bc00640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd894118550 con 0x7fd894074eb0 2024-09-10T08:25:24.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.013+0000 7fd89bc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941183d0 con 0x7fd894074590 2024-09-10T08:25:24.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.013+0000 7fd89bc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941186d0 con 0x7fd894112b90 2024-09-10T08:25:24.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.014+0000 7fd899200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd894074590 0x7fd8941a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:24.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.014+0000 7fd893e00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894074eb0 0x7fd8941a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:24.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.014+0000 7fd899200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd894074590 0x7fd8941a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.2:37078/0 (socket says 172.21.15.2:37078) 2024-09-10T08:25:24.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.014+0000 7fd899200640 1 -- 172.21.15.2:0/3041261292 learned_addr learned my addr 172.21.15.2:0/3041261292 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:25:24.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.014+0000 7fd899c00640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd894112b90 0x7fd89419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:24.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.014+0000 7fd899200640 1 -- 172.21.15.2:0/3041261292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd894112b90 msgr2=0x7fd89419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:24.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.014+0000 7fd899200640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd894112b90 0x7fd89419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:24.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.014+0000 7fd899200640 1 -- 172.21.15.2:0/3041261292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894074eb0 msgr2=0x7fd8941a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:24.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.014+0000 7fd899200640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894074eb0 0x7fd8941a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:24.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.014+0000 7fd899200640 1 -- 172.21.15.2:0/3041261292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89419efd0 con 0x7fd894074590 2024-09-10T08:25:24.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.014+0000 7fd893e00640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894074eb0 0x7fd8941a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:25:24.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.014+0000 7fd899200640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd894074590 0x7fd8941a46e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd888006010 tx=0x7fd888004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:24.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.015+0000 7fd891600640 1 -- 172.21.15.2:0/3041261292 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88803f030 con 0x7fd894074590 2024-09-10T08:25:24.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.015+0000 7fd891600640 1 -- 172.21.15.2:0/3041261292 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd888004440 con 0x7fd894074590 2024-09-10T08:25:24.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.015+0000 7fd89bc00640 1 -- 172.21.15.2:0/3041261292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89419f260 con 0x7fd894074590 2024-09-10T08:25:24.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.015+0000 7fd891600640 1 -- 172.21.15.2:0/3041261292 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888035070 con 0x7fd894074590 2024-09-10T08:25:24.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.015+0000 7fd89bc00640 1 -- 172.21.15.2:0/3041261292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8941ab5b0 con 0x7fd894074590 2024-09-10T08:25:24.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.016+0000 7fd89bc00640 1 -- 172.21.15.2:0/3041261292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd894074970 con 0x7fd894074590 2024-09-10T08:25:24.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.018+0000 7fd891600640 1 -- 172.21.15.2:0/3041261292 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd888042030 con 0x7fd894074590 2024-09-10T08:25:24.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.019+0000 7fd891600640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd87c0a2b80 0x7fd87c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:24.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.020+0000 7fd891600640 1 -- 172.21.15.2:0/3041261292 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8941+0+0 (secure 0 0 0) 0x7fd8880ef8e0 con 0x7fd894074590 2024-09-10T08:25:24.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.020+0000 7fd893e00640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd87c0a2b80 0x7fd87c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:24.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.020+0000 7fd893e00640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd87c0a2b80 0x7fd87c0a5040 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd8941000e0 tx=0x7fd880005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:24.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.022+0000 7fd891600640 1 -- 172.21.15.2:0/3041261292 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8880bbed0 con 0x7fd894074590 2024-09-10T08:25:24.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:24.209+0000 7fd89bc00640 1 -- 172.21.15.2:0/3041261292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd894104fd0 con 0x7fd894074590 2024-09-10T08:25:25.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:25.137+0000 7fd891600640 1 -- 172.21.15.2:0/3041261292 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) ==== 169+0+0 (secure 0 0 0) 0x7fd8880c01d0 con 0x7fd894074590 2024-09-10T08:25:25.140 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' removed 2024-09-10T08:25:25.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:25.141+0000 7fd89bc00640 1 -- 172.21.15.2:0/3041261292 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd87c0a2b80 msgr2=0x7fd87c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:25.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:25.141+0000 7fd89bc00640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd87c0a2b80 0x7fd87c0a5040 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd8941000e0 tx=0x7fd880005f90 comp rx=0 tx=0).stop 2024-09-10T08:25:25.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:25.141+0000 7fd89bc00640 1 -- 172.21.15.2:0/3041261292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd894074590 msgr2=0x7fd8941a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:25.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:25.141+0000 7fd89bc00640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd894074590 0x7fd8941a46e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd888006010 tx=0x7fd888004040 comp rx=0 tx=0).stop 2024-09-10T08:25:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:25.142+0000 7fd89bc00640 1 -- 172.21.15.2:0/3041261292 shutdown_connections 2024-09-10T08:25:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:25.142+0000 7fd89bc00640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7fd87c0a2b80 0x7fd87c0a5040 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:25.142+0000 7fd89bc00640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd894112b90 0x7fd89419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:25.142+0000 7fd89bc00640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd894074eb0 0x7fd8941a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:25.142+0000 7fd89bc00640 1 --2- 172.21.15.2:0/3041261292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd894074590 0x7fd8941a46e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:25.142+0000 7fd89bc00640 1 -- 172.21.15.2:0/3041261292 >> 172.21.15.2:0/3041261292 conn(0x7fd8940fd8c0 msgr2=0x7fd8940ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:25.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:25.143+0000 7fd89bc00640 1 -- 172.21.15.2:0/3041261292 shutdown_connections 2024-09-10T08:25:25.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:25.143+0000 7fd89bc00640 1 -- 172.21.15.2:0/3041261292 wait complete. 2024-09-10T08:25:25.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:25 smithi155 ceph-mon[35405]: pgmap v907: 97 pgs: 97 active+clean; 577 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:25:25.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:25 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/3041261292' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:25:25.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:25 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:25:25.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:25 smithi107 ceph-mon[35048]: pgmap v907: 97 pgs: 97 active+clean; 577 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:25:25.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:25 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/3041261292' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:25:25.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:25 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:25:25.436 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:25 smithi002 ceph-mon[30563]: pgmap v907: 97 pgs: 97 active+clean; 577 KiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:25:25.436 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:25 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/3041261292' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:25:25.436 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:25 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:25:26.095 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T08:25:26.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:26 smithi002 ceph-mon[30563]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:25:26.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:26 smithi002 ceph-mon[30563]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T08:25:26.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:26 smithi155 ceph-mon[35405]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:25:26.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:26 smithi155 ceph-mon[35405]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T08:25:26.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:26 smithi107 ceph-mon[35048]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:25:26.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:26 smithi107 ceph-mon[35048]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T08:25:26.491 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:25:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.096+0000 7efd14800640 1 -- 172.21.15.2:0/2886088917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd0c074620 msgr2=0x7efd0c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.096+0000 7efd14800640 1 --2- 172.21.15.2:0/2886088917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd0c074620 0x7efd0c074a00 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7efd00009900 tx=0x7efd0002a480 comp rx=0 tx=0).stop 2024-09-10T08:25:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.096+0000 7efd14800640 1 -- 172.21.15.2:0/2886088917 shutdown_connections 2024-09-10T08:25:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.096+0000 7efd14800640 1 --2- 172.21.15.2:0/2886088917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd0c10e3c0 0x7efd0c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.096+0000 7efd14800640 1 --2- 172.21.15.2:0/2886088917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd0c074f40 0x7efd0c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.096+0000 7efd14800640 1 --2- 172.21.15.2:0/2886088917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd0c074620 0x7efd0c074a00 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.096+0000 7efd14800640 1 -- 172.21.15.2:0/2886088917 >> 172.21.15.2:0/2886088917 conn(0x7efd0c0fd8c0 msgr2=0x7efd0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.096+0000 7efd14800640 1 -- 172.21.15.2:0/2886088917 shutdown_connections 2024-09-10T08:25:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.096+0000 7efd14800640 1 -- 172.21.15.2:0/2886088917 wait complete. 2024-09-10T08:25:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.097+0000 7efd14800640 1 Processor -- start 2024-09-10T08:25:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.097+0000 7efd14800640 1 -- start start 2024-09-10T08:25:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.097+0000 7efd14800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd0c074620 0x7efd0c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.097+0000 7efd14800640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd0c074f40 0x7efd0c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.097+0000 7efd14800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd0c10e3c0 0x7efd0c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.097+0000 7efd14800640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c1141a0 con 0x7efd0c074f40 2024-09-10T08:25:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.097+0000 7efd14800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c114020 con 0x7efd0c10e3c0 2024-09-10T08:25:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.097+0000 7efd14800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c114320 con 0x7efd0c074620 2024-09-10T08:25:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.097+0000 7efd11400640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd0c074f40 0x7efd0c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.097+0000 7efd11e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd0c074620 0x7efd0c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.098+0000 7efd11e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd0c074620 0x7efd0c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.2:50422/0 (socket says 172.21.15.2:50422) 2024-09-10T08:25:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.098+0000 7efd11e00640 1 -- 172.21.15.2:0/981928927 learned_addr learned my addr 172.21.15.2:0/981928927 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-10T08:25:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.098+0000 7efd12800640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd0c10e3c0 0x7efd0c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.098+0000 7efd11400640 1 -- 172.21.15.2:0/981928927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd0c074620 msgr2=0x7efd0c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.098+0000 7efd11400640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd0c074620 0x7efd0c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.098+0000 7efd11400640 1 -- 172.21.15.2:0/981928927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd0c10e3c0 msgr2=0x7efd0c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.098+0000 7efd11400640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd0c10e3c0 0x7efd0c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.098+0000 7efd11400640 1 -- 172.21.15.2:0/981928927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c19f5d0 con 0x7efd0c074f40 2024-09-10T08:25:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.098+0000 7efd12800640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd0c10e3c0 0x7efd0c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:25:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.098+0000 7efd11400640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd0c074f40 0x7efd0c19ab60 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00e9f0 tx=0x7efcfc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.099+0000 7efd06a00640 1 -- 172.21.15.2:0/981928927 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc00ccc0 con 0x7efd0c074f40 2024-09-10T08:25:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.099+0000 7efd06a00640 1 -- 172.21.15.2:0/981928927 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcfc010910 con 0x7efd0c074f40 2024-09-10T08:25:27.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.099+0000 7efd14800640 1 -- 172.21.15.2:0/981928927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0c19f8c0 con 0x7efd0c074f40 2024-09-10T08:25:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.099+0000 7efd06a00640 1 -- 172.21.15.2:0/981928927 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc018960 con 0x7efd0c074f40 2024-09-10T08:25:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.099+0000 7efd14800640 1 -- 172.21.15.2:0/981928927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0c1a71a0 con 0x7efd0c074f40 2024-09-10T08:25:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.100+0000 7efd14800640 1 -- 172.21.15.2:0/981928927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd0c074a00 con 0x7efd0c074f40 2024-09-10T08:25:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.102+0000 7efd06a00640 1 -- 172.21.15.2:0/981928927 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7efcfc010ac0 con 0x7efd0c074f40 2024-09-10T08:25:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.104+0000 7efd06a00640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efcf40a2c00 0x7efcf40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:25:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.104+0000 7efd06a00640 1 -- 172.21.15.2:0/981928927 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8324+0+0 (secure 0 0 0) 0x7efcfc0cf3f0 con 0x7efd0c074f40 2024-09-10T08:25:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.104+0000 7efd11e00640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efcf40a2c00 0x7efcf40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:25:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.105+0000 7efd11e00640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efcf40a2c00 0x7efcf40a50c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7efd000096c0 tx=0x7efd000046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:25:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.108+0000 7efd06a00640 1 -- 172.21.15.2:0/981928927 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcfc09bc40 con 0x7efd0c074f40 2024-09-10T08:25:27.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:27 smithi002 ceph-mon[30563]: pgmap v909: 33 pgs: 33 active+clean; 577 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:25:27.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:27.298+0000 7efd14800640 1 -- 172.21.15.2:0/981928927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7efd0c108fb0 con 0x7efd0c074f40 2024-09-10T08:25:27.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:27 smithi155 ceph-mon[35405]: pgmap v909: 33 pgs: 33 active+clean; 577 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:25:27.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:27 smithi107 ceph-mon[35048]: pgmap v909: 33 pgs: 33 active+clean; 577 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:25:28.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:28.149+0000 7efd06a00640 1 -- 172.21.15.2:0/981928927 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v99) ==== 178+0+0 (secure 0 0 0) 0x7efcfc09ff40 con 0x7efd0c074f40 2024-09-10T08:25:28.149 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data_ec' removed 2024-09-10T08:25:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:28.152+0000 7efd14800640 1 -- 172.21.15.2:0/981928927 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efcf40a2c00 msgr2=0x7efcf40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:28.152+0000 7efd14800640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efcf40a2c00 0x7efcf40a50c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7efd000096c0 tx=0x7efd000046c0 comp rx=0 tx=0).stop 2024-09-10T08:25:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:28.152+0000 7efd14800640 1 -- 172.21.15.2:0/981928927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd0c074f40 msgr2=0x7efd0c19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:25:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:28.152+0000 7efd14800640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd0c074f40 0x7efd0c19ab60 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00e9f0 tx=0x7efcfc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:25:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:28.153+0000 7efd14800640 1 -- 172.21.15.2:0/981928927 shutdown_connections 2024-09-10T08:25:28.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:28.153+0000 7efd14800640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd0c10e3c0 0x7efd0c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:28.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:28.153+0000 7efd14800640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd0c074f40 0x7efd0c19ab60 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:28.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:28.153+0000 7efd14800640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.2:6800/3751597725,v1:172.21.15.2:6801/3751597725] conn(0x7efcf40a2c00 0x7efcf40a50c0 secure :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7efd000096c0 tx=0x7efd000046c0 comp rx=0 tx=0).stop 2024-09-10T08:25:28.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:28.153+0000 7efd14800640 1 --2- 172.21.15.2:0/981928927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd0c074620 0x7efd0c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:25:28.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:28.153+0000 7efd14800640 1 -- 172.21.15.2:0/981928927 >> 172.21.15.2:0/981928927 conn(0x7efd0c0fd8c0 msgr2=0x7efd0c0fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:25:28.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:28.154+0000 7efd14800640 1 -- 172.21.15.2:0/981928927 shutdown_connections 2024-09-10T08:25:28.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-10T08:25:28.155+0000 7efd14800640 1 -- 172.21.15.2:0/981928927 wait complete. 2024-09-10T08:25:28.406 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:28 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/981928927' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:25:28.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:28 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/981928927' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:25:28.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:28 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/981928927' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:25:29.118 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:25:29.180 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:25:29.212 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:25:29.246 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T08:25:29.247 DEBUG:teuthology.orchestra.run.smithi002:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T08:25:29.285 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T08:25:29.302 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T08:25:29.321 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T08:25:29.321 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T08:25:29.321 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.a 2024-09-10T08:25:29.373 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:29 smithi002 ceph-mon[30563]: pgmap v910: 33 pgs: 33 active+clean; 577 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:25:29.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:29 smithi002 ceph-mon[30563]: from='client.? 172.21.15.2:0/981928927' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:25:29.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:29 smithi002 ceph-mon[30563]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T08:25:29.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:29 smithi155 ceph-mon[35405]: pgmap v910: 33 pgs: 33 active+clean; 577 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:25:29.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:29 smithi155 ceph-mon[35405]: from='client.? 172.21.15.2:0/981928927' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:25:29.407 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:29 smithi155 ceph-mon[35405]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T08:25:29.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:29 smithi107 ceph-mon[35048]: pgmap v910: 33 pgs: 33 active+clean; 577 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:25:29.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:29 smithi107 ceph-mon[35048]: from='client.? 172.21.15.2:0/981928927' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:25:29.417 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:29 smithi107 ceph-mon[35048]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T08:25:29.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:29 smithi002 systemd[1]: Stopping Ceph mon.a for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:25:30.222 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:29 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a[30559]: 2024-09-10T08:25:29.792+0000 7f16f0a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:25:30.222 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:29 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a[30559]: 2024-09-10T08:25:29.792+0000 7f16f0a00640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T08:25:30.222 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:29 smithi002 podman[124877]: 2024-09-10 08:25:29.854548121 +0000 UTC m=+0.232059991 container died f3a068f1b9648de30831622ded23e5d2cd06caa8a39b35a11b79cbb97588267c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2) 2024-09-10T08:25:30.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:30 smithi002 podman[124877]: 2024-09-10 08:25:30.370814378 +0000 UTC m=+0.748326246 container cleanup f3a068f1b9648de30831622ded23e5d2cd06caa8a39b35a11b79cbb97588267c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:25:30.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:30 smithi002 bash[124877]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a 2024-09-10T08:25:30.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Sep 10 08:25:30 smithi002 podman[124888]: 2024-09-10 08:25:30.647327655 +0000 UTC m=+0.780975186 container remove f3a068f1b9648de30831622ded23e5d2cd06caa8a39b35a11b79cbb97588267c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:25:30.954 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T08:25:31.042 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:25:31.042 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T08:25:31.042 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T08:25:31.042 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.b 2024-09-10T08:25:31.364 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:31 smithi107 systemd[1]: Stopping Ceph mon.b for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:25:31.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:31 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-b[35044]: 2024-09-10T08:25:31.364+0000 7fd60854a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:25:31.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:31 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-b[35044]: 2024-09-10T08:25:31.364+0000 7fd60854a640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T08:25:31.667 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:31 smithi107 podman[69527]: 2024-09-10 08:25:31.38529855 +0000 UTC m=+0.145515421 container died e8e08e82260ca00f8f86eb37a0c222fef9af7c61472ee21a7c4019d05909f44c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T08:25:32.057 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:31 smithi107 podman[69527]: 2024-09-10 08:25:31.781382673 +0000 UTC m=+0.541599543 container cleanup e8e08e82260ca00f8f86eb37a0c222fef9af7c61472ee21a7c4019d05909f44c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T08:25:32.057 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 10 08:25:31 smithi107 bash[69527]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-b 2024-09-10T08:25:32.183 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T08:25:32.229 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:25:32.230 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T08:25:32.230 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T08:25:32.230 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.c 2024-09-10T08:25:32.553 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:32 smithi155 systemd[1]: Stopping Ceph mon.c for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:25:32.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:32 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-c[35401]: 2024-09-10T08:25:32.553+0000 7f406f26d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:25:32.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:32 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-c[35401]: 2024-09-10T08:25:32.553+0000 7f406f26d640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T08:25:32.907 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:32 smithi155 podman[71169]: 2024-09-10 08:25:32.655963228 +0000 UTC m=+0.229078251 container died 39a0f8f0def57a27a48c921336a08062e5c47cbb611fb105886830eabd773dd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T08:25:33.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:33 smithi155 podman[71169]: 2024-09-10 08:25:33.037592942 +0000 UTC m=+0.610708007 container cleanup 39a0f8f0def57a27a48c921336a08062e5c47cbb611fb105886830eabd773dd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T08:25:33.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Sep 10 08:25:33 smithi155 bash[71169]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mon-c 2024-09-10T08:25:33.464 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T08:25:33.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:25:33.510 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T08:25:33.510 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T08:25:33.510 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.x 2024-09-10T08:25:33.987 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:25:33 smithi002 systemd[1]: Stopping Ceph mgr.x for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:25:34.270 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:25:33 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T08:25:33.985+0000 7f63e4800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:25:34.271 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:25:33 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x[30800]: 2024-09-10T08:25:33.985+0000 7f63e4800640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T08:25:34.271 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:25:34 smithi002 podman[124994]: 2024-09-10 08:25:34.069224267 +0000 UTC m=+0.259102305 container died 3688b0cf814b44ebfc5d4e47d74c7628ee68eb1207efa12ffc495bf5b64c94ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:25:34.711 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:25:34 smithi002 podman[124994]: 2024-09-10 08:25:34.525592063 +0000 UTC m=+0.715470090 container cleanup 3688b0cf814b44ebfc5d4e47d74c7628ee68eb1207efa12ffc495bf5b64c94ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T08:25:34.711 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:25:34 smithi002 bash[124994]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x 2024-09-10T08:25:35.000 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Sep 10 08:25:34 smithi002 podman[125005]: 2024-09-10 08:25:34.710770605 +0000 UTC m=+0.632447870 container remove 3688b0cf814b44ebfc5d4e47d74c7628ee68eb1207efa12ffc495bf5b64c94ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:25:35.020 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T08:25:35.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:25:35.094 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T08:25:35.094 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T08:25:35.095 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.y 2024-09-10T08:25:35.417 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 08:25:35 smithi107 systemd[1]: Stopping Ceph mgr.y for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:25:35.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 08:25:35 smithi107 podman[69649]: 2024-09-10 08:25:35.432208026 +0000 UTC m=+0.141846021 container died eeb53500d4322bee12cf4f83f39123d154e370402e69266d26f4a86fc25c5c43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T08:25:36.096 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 08:25:35 smithi107 podman[69649]: 2024-09-10 08:25:35.861597466 +0000 UTC m=+0.571235461 container cleanup eeb53500d4322bee12cf4f83f39123d154e370402e69266d26f4a86fc25c5c43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T08:25:36.096 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 08:25:35 smithi107 bash[69649]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y 2024-09-10T08:25:36.335 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T08:25:36.357 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 08:25:36 smithi107 podman[69663]: 2024-09-10 08:25:36.096464057 +0000 UTC m=+0.656038652 container remove eeb53500d4322bee12cf4f83f39123d154e370402e69266d26f4a86fc25c5c43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T08:25:36.357 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 08:25:36 smithi107 systemd[1]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-10T08:25:36.357 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 08:25:36 smithi107 systemd[1]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-10T08:25:36.357 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 08:25:36 smithi107 systemd[1]: Stopped Ceph mgr.y for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:25:36.358 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 08:25:36 smithi107 systemd[1]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.y.service: Consumed 6.944s CPU time. 2024-09-10T08:25:36.370 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:25:36.370 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T08:25:36.370 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T08:25:36.371 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.z 2024-09-10T08:25:36.656 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 08:25:36 smithi155 systemd[1]: Stopping Ceph mgr.z for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:25:37.076 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 08:25:36 smithi155 podman[71291]: 2024-09-10 08:25:36.730055736 +0000 UTC m=+0.155152439 container died 30f2ea8a4b4d5339371329e870cf84df1fef45286b7e1452cb9b5d91600943f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-10T08:25:37.406 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 08:25:37 smithi155 podman[71291]: 2024-09-10 08:25:37.159968867 +0000 UTC m=+0.585065569 container cleanup 30f2ea8a4b4d5339371329e870cf84df1fef45286b7e1452cb9b5d91600943f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T08:25:37.406 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 08:25:37 smithi155 bash[71291]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z 2024-09-10T08:25:37.728 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T08:25:37.750 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 08:25:37 smithi155 podman[71303]: 2024-09-10 08:25:37.478152058 +0000 UTC m=+0.743368395 container remove 30f2ea8a4b4d5339371329e870cf84df1fef45286b7e1452cb9b5d91600943f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T08:25:37.750 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 08:25:37 smithi155 systemd[1]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-10T08:25:37.750 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 08:25:37 smithi155 systemd[1]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-10T08:25:37.750 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 08:25:37 smithi155 systemd[1]: Stopped Ceph mgr.z for 26ec64b0-6f4a-11ef-bcea-c7b262605968. 2024-09-10T08:25:37.750 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Sep 10 08:25:37 smithi155 systemd[1]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mgr.z.service: Consumed 7.035s CPU time. 2024-09-10T08:25:37.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:25:37.764 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T08:25:37.764 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T08:25:37.765 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.0 2024-09-10T08:25:38.191 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:37 smithi002 systemd[1]: Stopping Ceph osd.0 for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:25:38.521 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0[41774]: 2024-09-10T08:25:38.190+0000 7f3458800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:25:38.521 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0[41774]: 2024-09-10T08:25:38.190+0000 7f3458800640 -1 osd.0 99 *** Got signal Terminated *** 2024-09-10T08:25:38.521 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0[41774]: 2024-09-10T08:25:38.190+0000 7f3458800640 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:25:44.021 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:43 smithi002 podman[125112]: 2024-09-10 08:25:43.736686626 +0000 UTC m=+5.671991324 container died f7b6a8b63940e294f968ddbf53ae698b3bb2a30c8f282931b2bed23aa710f8f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T08:25:44.631 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:44 smithi002 podman[125112]: 2024-09-10 08:25:44.34666273 +0000 UTC m=+6.281967424 container cleanup f7b6a8b63940e294f968ddbf53ae698b3bb2a30c8f282931b2bed23aa710f8f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:25:44.631 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:44 smithi002 bash[125112]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0 2024-09-10T08:25:45.021 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:44 smithi002 podman[125130]: 2024-09-10 08:25:44.631860729 +0000 UTC m=+0.881422754 container remove f7b6a8b63940e294f968ddbf53ae698b3bb2a30c8f282931b2bed23aa710f8f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:25:45.476 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:45 smithi002 podman[125217]: 2024-09-10 08:25:45.085682023 +0000 UTC m=+0.027153476 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:25:45.476 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:45 smithi002 podman[125217]: 2024-09-10 08:25:45.225385938 +0000 UTC m=+0.166857380 container create 395a2e0fb716bff1973ae60f9bfdd014bedd79978f464eeebb15c1865ded5304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T08:25:45.747 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:45 smithi002 podman[125217]: 2024-09-10 08:25:45.477221851 +0000 UTC m=+0.418693294 container init 395a2e0fb716bff1973ae60f9bfdd014bedd79978f464eeebb15c1865ded5304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T08:25:45.747 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:45 smithi002 podman[125217]: 2024-09-10 08:25:45.484041255 +0000 UTC m=+0.425512698 container start 395a2e0fb716bff1973ae60f9bfdd014bedd79978f464eeebb15c1865ded5304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T08:25:45.747 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:45 smithi002 podman[125217]: 2024-09-10 08:25:45.614923807 +0000 UTC m=+0.556395279 container attach 395a2e0fb716bff1973ae60f9bfdd014bedd79978f464eeebb15c1865ded5304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T08:25:45.747 INFO:journalctl@ceph.osd.0.smithi002.stdout:Sep 10 08:25:45 smithi002 podman[125217]: 2024-09-10 08:25:45.747021345 +0000 UTC m=+0.688492790 container died 395a2e0fb716bff1973ae60f9bfdd014bedd79978f464eeebb15c1865ded5304 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:25:46.594 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T08:25:46.672 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:25:46.672 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T08:25:46.673 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T08:25:46.673 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.1 2024-09-10T08:25:47.006 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:46 smithi107 systemd[1]: Stopping Ceph osd.1 for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:25:47.417 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:47 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1[39509]: 2024-09-10T08:25:47.005+0000 7f397c0ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:25:47.417 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:47 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1[39509]: 2024-09-10T08:25:47.005+0000 7f397c0ad640 -1 osd.1 99 *** Got signal Terminated *** 2024-09-10T08:25:47.417 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:47 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1[39509]: 2024-09-10T08:25:47.005+0000 7f397c0ad640 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:25:52.667 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:52 smithi107 podman[69773]: 2024-09-10 08:25:52.370434464 +0000 UTC m=+5.504034954 container died b807476a103b28e6c8ffa6e41a8aa81bcaacd7cedb6402eb0adbc70505cedfa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T08:25:52.998 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:52 smithi107 podman[69773]: 2024-09-10 08:25:52.764567893 +0000 UTC m=+5.898168382 container cleanup b807476a103b28e6c8ffa6e41a8aa81bcaacd7cedb6402eb0adbc70505cedfa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:25:52.999 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:52 smithi107 bash[69773]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1 2024-09-10T08:25:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:52 smithi107 podman[69788]: 2024-09-10 08:25:52.999438804 +0000 UTC m=+0.622608292 container remove b807476a103b28e6c8ffa6e41a8aa81bcaacd7cedb6402eb0adbc70505cedfa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T08:25:53.577 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:53 smithi107 podman[69877]: 2024-09-10 08:25:53.228206472 +0000 UTC m=+0.015773971 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:25:53.577 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:53 smithi107 podman[69877]: 2024-09-10 08:25:53.34192081 +0000 UTC m=+0.129488302 container create 68dec33b0f5803979a2998e5fca885666d7c07837a3d98d7ce485d0421cd6cec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:25:53.917 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:53 smithi107 podman[69877]: 2024-09-10 08:25:53.577648955 +0000 UTC m=+0.365216451 container init 68dec33b0f5803979a2998e5fca885666d7c07837a3d98d7ce485d0421cd6cec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T08:25:53.917 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:53 smithi107 podman[69877]: 2024-09-10 08:25:53.58119025 +0000 UTC m=+0.368757747 container start 68dec33b0f5803979a2998e5fca885666d7c07837a3d98d7ce485d0421cd6cec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T08:25:53.917 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:53 smithi107 podman[69877]: 2024-09-10 08:25:53.686414451 +0000 UTC m=+0.473981970 container attach 68dec33b0f5803979a2998e5fca885666d7c07837a3d98d7ce485d0421cd6cec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T08:25:53.917 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 08:25:53 smithi107 podman[69877]: 2024-09-10 08:25:53.775401581 +0000 UTC m=+0.562969074 container died 68dec33b0f5803979a2998e5fca885666d7c07837a3d98d7ce485d0421cd6cec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:25:54.458 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T08:25:54.499 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:25:54.499 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T08:25:54.499 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T08:25:54.500 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.2 2024-09-10T08:25:54.797 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:25:54 smithi155 systemd[1]: Stopping Ceph osd.2 for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:25:55.156 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:25:54 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2[40370]: 2024-09-10T08:25:54.795+0000 7fbd7cf00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:25:55.156 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:25:54 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2[40370]: 2024-09-10T08:25:54.795+0000 7fbd7cf00640 -1 osd.2 99 *** Got signal Terminated *** 2024-09-10T08:25:55.157 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:25:54 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2[40370]: 2024-09-10T08:25:54.796+0000 7fbd7cf00640 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:26:00.558 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:26:00 smithi155 podman[71416]: 2024-09-10 08:26:00.238954615 +0000 UTC m=+5.547835553 container died b497a2e232095f8be050f6ad362a8dfb1d4017b4a904840b863f438f993503cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T08:26:00.906 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:26:00 smithi155 podman[71416]: 2024-09-10 08:26:00.655990034 +0000 UTC m=+5.964870966 container cleanup b497a2e232095f8be050f6ad362a8dfb1d4017b4a904840b863f438f993503cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T08:26:00.907 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:26:00 smithi155 bash[71416]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2 2024-09-10T08:26:01.322 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:26:00 smithi155 podman[71443]: 2024-09-10 08:26:00.924156929 +0000 UTC m=+0.686165957 container remove b497a2e232095f8be050f6ad362a8dfb1d4017b4a904840b863f438f993503cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T08:26:01.593 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:26:01 smithi155 podman[71531]: 2024-09-10 08:26:01.225609625 +0000 UTC m=+0.015876376 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:26:01.593 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:26:01 smithi155 podman[71531]: 2024-09-10 08:26:01.33336041 +0000 UTC m=+0.123627196 container create 8b3798152611b9606e6e7d9d3f267490d7f01ea1a056097da45f74951d8ff4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:26:01.594 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:26:01 smithi155 podman[71531]: 2024-09-10 08:26:01.594465939 +0000 UTC m=+0.384732692 container init 8b3798152611b9606e6e7d9d3f267490d7f01ea1a056097da45f74951d8ff4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T08:26:01.907 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:26:01 smithi155 podman[71531]: 2024-09-10 08:26:01.597877819 +0000 UTC m=+0.388144571 container start 8b3798152611b9606e6e7d9d3f267490d7f01ea1a056097da45f74951d8ff4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T08:26:01.907 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:26:01 smithi155 podman[71531]: 2024-09-10 08:26:01.694303157 +0000 UTC m=+0.484569922 container attach 8b3798152611b9606e6e7d9d3f267490d7f01ea1a056097da45f74951d8ff4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T08:26:01.907 INFO:journalctl@ceph.osd.2.smithi155.stdout:Sep 10 08:26:01 smithi155 podman[71531]: 2024-09-10 08:26:01.784435936 +0000 UTC m=+0.574702693 container died 8b3798152611b9606e6e7d9d3f267490d7f01ea1a056097da45f74951d8ff4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T08:26:02.468 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T08:26:02.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:26:02.510 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T08:26:02.510 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T08:26:02.510 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.3 2024-09-10T08:26:02.936 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:02 smithi002 systemd[1]: Stopping Ceph osd.3 for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:26:03.270 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:02 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3[48390]: 2024-09-10T08:26:02.936+0000 7f09aa800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:26:03.270 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:02 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3[48390]: 2024-09-10T08:26:02.936+0000 7f09aa800640 -1 osd.3 99 *** Got signal Terminated *** 2024-09-10T08:26:03.270 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:02 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3[48390]: 2024-09-10T08:26:02.936+0000 7f09aa800640 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:26:04.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:04 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:04.616+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:05.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:04 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:04.982+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:05.770 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3[48390]: 2024-09-10T08:26:05.493+0000 7f09a6200640 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.381849+0000 front 2024-09-10T08:25:40.382096+0000 (oldest deadline 2024-09-10T08:26:05.081837+0000) 2024-09-10T08:26:05.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:05 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:05.560+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:05.906 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:05.652+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:06.167 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:05 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:05.933+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:06.770 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:06 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3[48390]: 2024-09-10T08:26:06.489+0000 7f09a6200640 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.381849+0000 front 2024-09-10T08:25:40.382096+0000 (oldest deadline 2024-09-10T08:26:05.081837+0000) 2024-09-10T08:26:06.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:06 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:06.543+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:06.906 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:06 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:06.615+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:07.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:06 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:06.974+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:07.586 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:07 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:07.332+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:07.770 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:07 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3[48390]: 2024-09-10T08:26:07.498+0000 7f09a6200640 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.381849+0000 front 2024-09-10T08:25:40.382096+0000 (oldest deadline 2024-09-10T08:26:05.081837+0000) 2024-09-10T08:26:07.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:07 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:07.541+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:07.906 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:07 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:07.586+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:07.907 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:07 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:07.777+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:08.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:07 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:07.949+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:08.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:08.378+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:08.657 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:08.610+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:08.771 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:08 smithi002 podman[125319]: 2024-09-10 08:26:08.460413747 +0000 UTC m=+5.654177378 container died 697e3a316d81a10d9eaf2ca50abe8c0ff597591b78fa848d42661c5fcbd9e0fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T08:26:08.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:08 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:08.500+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:09.156 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:08.817+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:09.216 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:08 smithi002 podman[125319]: 2024-09-10 08:26:08.966073698 +0000 UTC m=+6.159837328 container cleanup 697e3a316d81a10d9eaf2ca50abe8c0ff597591b78fa848d42661c5fcbd9e0fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T08:26:09.217 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:08 smithi002 bash[125319]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3 2024-09-10T08:26:09.416 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:08 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:08.967+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:09.520 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:09 smithi002 podman[125335]: 2024-09-10 08:26:09.217310271 +0000 UTC m=+0.744088080 container remove 697e3a316d81a10d9eaf2ca50abe8c0ff597591b78fa848d42661c5fcbd9e0fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T08:26:09.656 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:09 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:09.564+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:09.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:09 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:09.395+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:09.801 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:09 smithi002 podman[125420]: 2024-09-10 08:26:09.692049556 +0000 UTC m=+0.027401579 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:26:09.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:09 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:09.542+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:10.053 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:09 smithi002 podman[125420]: 2024-09-10 08:26:09.801719851 +0000 UTC m=+0.137071914 container create 09fa08cb0a66b679db7e45cefaeae92062e90fb290109a12757d7f6a73846422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T08:26:10.156 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:09 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:09.856+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:10.330 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:10 smithi002 podman[125420]: 2024-09-10 08:26:10.054011812 +0000 UTC m=+0.389363835 container init 09fa08cb0a66b679db7e45cefaeae92062e90fb290109a12757d7f6a73846422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:26:10.330 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:10 smithi002 podman[125420]: 2024-09-10 08:26:10.06085885 +0000 UTC m=+0.396210882 container start 09fa08cb0a66b679db7e45cefaeae92062e90fb290109a12757d7f6a73846422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T08:26:10.330 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:10 smithi002 podman[125420]: 2024-09-10 08:26:10.162097452 +0000 UTC m=+0.497449475 container attach 09fa08cb0a66b679db7e45cefaeae92062e90fb290109a12757d7f6a73846422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T08:26:10.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:09 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:09.922+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:10.656 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:10 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:10.526+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:10.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:10 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:10.395+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:10.740 INFO:journalctl@ceph.osd.3.smithi002.stdout:Sep 10 08:26:10 smithi002 podman[125420]: 2024-09-10 08:26:10.330963589 +0000 UTC m=+0.666315624 container died 09fa08cb0a66b679db7e45cefaeae92062e90fb290109a12757d7f6a73846422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:26:10.740 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:10 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:10.566+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:11.156 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:10 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:10.843+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:11.170 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T08:26:11.247 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:26:11.248 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T08:26:11.248 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T08:26:11.248 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.4 2024-09-10T08:26:11.276 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:10 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:10.932+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:11.549 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:11 smithi107 systemd[1]: Stopping Ceph osd.4 for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:26:11.656 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:11 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:11.564+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:11.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:11 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:11.361+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:11.917 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:11 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4[45998]: 2024-09-10T08:26:11.549+0000 7ff48e0a4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:26:11.917 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:11 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4[45998]: 2024-09-10T08:26:11.549+0000 7ff48e0a4640 -1 osd.4 99 *** Got signal Terminated *** 2024-09-10T08:26:11.917 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:11 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4[45998]: 2024-09-10T08:26:11.549+0000 7ff48e0a4640 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:26:12.020 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:11 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:11.559+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:12.156 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:11 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:11.822+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:12.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:11 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:11.954+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:12.571 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:12 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:12.320+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:12.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:12 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:12.513+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:12.871 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:12 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:12.571+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:13.156 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:12 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:12.871+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:13.276 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:12 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:12.917+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:13.276 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:12 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:12.917+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:13.608 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:13 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:13.338+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:13.609 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:13 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:13.338+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:13.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:13 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:13.276+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:13.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:13 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:13.476+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:13.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:13 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:13.476+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:13.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:13 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:13.317+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:13.872 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:13 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:13.609+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:13.872 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:13 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:13.609+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:14.156 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:13 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:13.872+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:14.157 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:13 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:13.872+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:48.938707+0000 front 2024-09-10T08:25:48.938780+0000 (oldest deadline 2024-09-10T08:26:13.638716+0000) 2024-09-10T08:26:14.166 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:13 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:13.889+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:14.167 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:13 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:13.889+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:14.641 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:14 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:14.386+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:14.642 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:14 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:14.386+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:14.666 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:14 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:14.281+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:14.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:14 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:14.367+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:14.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:14 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:14.435+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:14.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:14 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:14.435+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:14.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:14 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:14.641+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:14.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:14 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:14.641+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:15.167 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:14 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:14.903+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:15.167 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:14 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:14.903+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:15.374 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:14 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:14.912+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:15.374 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:14 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:14.912+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:48.938707+0000 front 2024-09-10T08:25:48.938780+0000 (oldest deadline 2024-09-10T08:26:13.638716+0000) 2024-09-10T08:26:15.641 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:15 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:15.374+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:15.642 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:15 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:15.374+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:15.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:15 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:15.262+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:15.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:15 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:15.383+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:15.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:15 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:15.394+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:15.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:15 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:15.394+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:15.906 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:15 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:15.641+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:15.906 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:15 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:15.641+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:16.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:15 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:15.949+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:16.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:15 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:15.949+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:16.376 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:15 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:15.951+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:16.376 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:15 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:15.951+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:48.938707+0000 front 2024-09-10T08:25:48.938780+0000 (oldest deadline 2024-09-10T08:26:13.638716+0000) 2024-09-10T08:26:16.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:16 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:16.376+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:16.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:16 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:16.376+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:16.666 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:16 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4[45998]: 2024-09-10T08:26:16.283+0000 7ff48a6ba640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.670926+0000 front 2024-09-10T08:25:50.671052+0000 (oldest deadline 2024-09-10T08:26:15.971125+0000) 2024-09-10T08:26:16.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:16 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:16.293+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:16.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:16 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:16.401+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:16.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:16 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:16.418+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:16.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:16 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:16.418+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:16.926 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:16 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:16.658+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:16.926 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:16 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:16.658+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:17.167 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:16 smithi107 podman[69993]: 2024-09-10 08:26:16.901460631 +0000 UTC m=+5.459344345 container died 1a80374265dcacf1885198dc79d3ca7f5cf101a8a0bfcce05020c53612f228bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T08:26:17.167 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:16 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:16.952+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:17.167 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:16 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:16.952+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:17.335 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:16 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:16.926+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:17.335 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:16 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:16.926+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:48.938707+0000 front 2024-09-10T08:25:48.938780+0000 (oldest deadline 2024-09-10T08:26:13.638716+0000) 2024-09-10T08:26:17.566 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:17 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:17.283+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:17.567 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:17 smithi107 podman[69993]: 2024-09-10 08:26:17.332824846 +0000 UTC m=+5.890708559 container cleanup 1a80374265dcacf1885198dc79d3ca7f5cf101a8a0bfcce05020c53612f228bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T08:26:17.567 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:17 smithi107 bash[69993]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4 2024-09-10T08:26:17.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:17.335+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:17.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:17.335+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:17.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:17 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:17.440+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:17.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:17 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:17.440+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:17.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:17 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:17.440+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:17.912 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:17 smithi107 podman[70013]: 2024-09-10 08:26:17.567450367 +0000 UTC m=+0.660733708 container remove 1a80374265dcacf1885198dc79d3ca7f5cf101a8a0bfcce05020c53612f228bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T08:26:17.954 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:17.686+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:17.955 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:17.686+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:18.167 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:17 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:17.912+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:18.167 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:17 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:17.912+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:18.167 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:17 smithi107 podman[70100]: 2024-09-10 08:26:17.899339981 +0000 UTC m=+0.016250548 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:26:18.167 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:18 smithi107 podman[70100]: 2024-09-10 08:26:18.035021469 +0000 UTC m=+0.151932071 container create d053b11ea55355513b853406f3bd58713a740b044d1e45b0a348e91d2b8e51a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:26:18.286 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:17.955+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:18.286 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:17 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:17.955+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:48.938707+0000 front 2024-09-10T08:25:48.938780+0000 (oldest deadline 2024-09-10T08:26:13.638716+0000) 2024-09-10T08:26:18.476 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:18 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:18.240+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:18.476 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:18 smithi107 podman[70100]: 2024-09-10 08:26:18.287287851 +0000 UTC m=+0.404198410 container init d053b11ea55355513b853406f3bd58713a740b044d1e45b0a348e91d2b8e51a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T08:26:18.476 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:18 smithi107 podman[70100]: 2024-09-10 08:26:18.290530768 +0000 UTC m=+0.407441329 container start d053b11ea55355513b853406f3bd58713a740b044d1e45b0a348e91d2b8e51a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T08:26:18.476 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:18 smithi107 podman[70100]: 2024-09-10 08:26:18.387258941 +0000 UTC m=+0.504169501 container attach d053b11ea55355513b853406f3bd58713a740b044d1e45b0a348e91d2b8e51a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T08:26:18.636 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:18 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:18.286+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:18.636 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:18 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:18.286+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:18.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:18 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:18.422+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:18.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:18 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:18.401+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:18.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:18 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:18.401+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:18.780 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 10 08:26:18 smithi107 podman[70100]: 2024-09-10 08:26:18.477095373 +0000 UTC m=+0.594005946 container died d053b11ea55355513b853406f3bd58713a740b044d1e45b0a348e91d2b8e51a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:26:18.906 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:18 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:18.637+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:18.906 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:18 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:18.637+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:19.040 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:18 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:18.939+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:19.040 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:18 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:18.939+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:19.067 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T08:26:19.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:26:19.147 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T08:26:19.147 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T08:26:19.147 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.5 2024-09-10T08:26:19.175 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:18 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:18.913+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:19.175 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:18 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:18.913+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:48.938707+0000 front 2024-09-10T08:25:48.938780+0000 (oldest deadline 2024-09-10T08:26:13.638716+0000) 2024-09-10T08:26:19.318 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:19 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:19.245+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:19.426 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:19 smithi155 systemd[1]: Stopping Ceph osd.5 for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:26:19.426 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:19 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:19.328+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:19.426 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:19 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:19.328+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:19.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:19 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:19.379+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:19.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:19 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:19.444+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:19.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:19 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:19.444+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:19.879 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:19 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:19.625+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:19.880 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:19 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:19.625+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:19.880 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:19 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:19.426+0000 7fd6bf23b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:26:19.880 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:19 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:19.426+0000 7fd6bf23b640 -1 osd.5 99 *** Got signal Terminated *** 2024-09-10T08:26:19.880 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:19 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:19.426+0000 7fd6bf23b640 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:26:19.880 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:19 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:19.880+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:19.880 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:19 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:19.880+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:48.938707+0000 front 2024-09-10T08:25:48.938780+0000 (oldest deadline 2024-09-10T08:26:13.638716+0000) 2024-09-10T08:26:20.242 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:19 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:19.971+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:20.243 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:19 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:19.971+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:20.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:20 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:20.337+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:20.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:20 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:20.337+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:20.666 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:20 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:20.242+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:20.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:20.420+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:20.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:20.395+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:20.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:20.395+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:20.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:20.395+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:21.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:20 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:20.674+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:21.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:20 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:20.674+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:21.157 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:20 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:20.845+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:21.157 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:20 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:20.845+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:48.938707+0000 front 2024-09-10T08:25:48.938780+0000 (oldest deadline 2024-09-10T08:26:13.638716+0000) 2024-09-10T08:26:21.157 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:20 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:20.845+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.939526+0000 front 2024-09-10T08:25:58.939497+0000 (oldest deadline 2024-09-10T08:26:20.639798+0000) 2024-09-10T08:26:21.252 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:20 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:20.956+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:21.252 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:20 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:20.956+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:21.649 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:21 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:21.321+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:21.649 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:21 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:21.321+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:21.666 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:21 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:21.252+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:21.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:21 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:21.252+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:21.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:21 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:21.450+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:21.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:21 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:21.450+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:56.483990+0000 front 2024-09-10T08:25:56.483927+0000 (oldest deadline 2024-09-10T08:26:20.584193+0000) 2024-09-10T08:26:21.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:21 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:21.423+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:21.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:21 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:21.423+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:21.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:21 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:21.423+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:21.906 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:21 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:21.649+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:21.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:21 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:21.649+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:21.907 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:21 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:21.874+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:21.907 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:21 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:21.874+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:48.938707+0000 front 2024-09-10T08:25:48.938780+0000 (oldest deadline 2024-09-10T08:26:13.638716+0000) 2024-09-10T08:26:21.907 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:21 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:21.874+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.939526+0000 front 2024-09-10T08:25:58.939497+0000 (oldest deadline 2024-09-10T08:26:20.639798+0000) 2024-09-10T08:26:22.254 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:21 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:21.957+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:22.254 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:21 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:21.957+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:22.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:22 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:22.350+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:22.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:22 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:22.350+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:22.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:22 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:22.350+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:22.666 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:22 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:22.254+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:22.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:22 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:22.254+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:22.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:22 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:22.464+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:22.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:22 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:22.464+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:56.483990+0000 front 2024-09-10T08:25:56.483927+0000 (oldest deadline 2024-09-10T08:26:20.584193+0000) 2024-09-10T08:26:22.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:22 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:22.407+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:22.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:22 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:22.407+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:22.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:22 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:22.407+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:23.156 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:22 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:22.677+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:23.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:22 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:22.677+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:23.157 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:22 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:22.853+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:23.157 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:22 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:22.853+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:48.938707+0000 front 2024-09-10T08:25:48.938780+0000 (oldest deadline 2024-09-10T08:26:13.638716+0000) 2024-09-10T08:26:23.157 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:22 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:22.853+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.939526+0000 front 2024-09-10T08:25:58.939497+0000 (oldest deadline 2024-09-10T08:26:20.639798+0000) 2024-09-10T08:26:23.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:22 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:22.976+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:23.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:22 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:22.976+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:23.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:23 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:23.220+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:23.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:23 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:23.220+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:23.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:23 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:23.353+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:23.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:23 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:23.353+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:23.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:23 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:23.353+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:23.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:23 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:23.430+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:23.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:23 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:23.430+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:56.483990+0000 front 2024-09-10T08:25:56.483927+0000 (oldest deadline 2024-09-10T08:26:20.584193+0000) 2024-09-10T08:26:23.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:23 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:23.440+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:23.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:23 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:23.440+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:23.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:23 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:23.440+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:24.156 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:23 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:23.821+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:42.636911+0000 front 2024-09-10T08:25:42.636896+0000 (oldest deadline 2024-09-10T08:26:07.336935+0000) 2024-09-10T08:26:24.157 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:23 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:23.821+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:48.938707+0000 front 2024-09-10T08:25:48.938780+0000 (oldest deadline 2024-09-10T08:26:13.638716+0000) 2024-09-10T08:26:24.157 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:23 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5[46853]: 2024-09-10T08:26:23.821+0000 7fd6b7849640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.939526+0000 front 2024-09-10T08:25:58.939497+0000 (oldest deadline 2024-09-10T08:26:20.639798+0000) 2024-09-10T08:26:24.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:23 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:23.683+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:24.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:23 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:23.683+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:24.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:23 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:23.683+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:24.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:24 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:24.211+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:24.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:24 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:24.211+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:24.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:23 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:23.997+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:24.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:23 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:23.997+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:24.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:23 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:23.997+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:24.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:24 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:24.304+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:24.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:24 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:24.304+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:24.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:24 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:24.304+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:24.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:24.470+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:24.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:24.470+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:56.483990+0000 front 2024-09-10T08:25:56.483927+0000 (oldest deadline 2024-09-10T08:26:20.584193+0000) 2024-09-10T08:26:24.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:24.433+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:24.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:24.433+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:24.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:24 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:24.433+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:25.156 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:24 smithi155 podman[71633]: 2024-09-10 08:26:24.886518152 +0000 UTC m=+5.547834786 container died e4200126d24a7ffe0e4929808506089769e41aeca0fc1a606e715255361e42b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T08:26:25.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:24 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:24.711+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:25.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:24 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:24.711+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:25.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:24 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:24.711+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:25.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:25 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:25.232+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:25.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:25 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:25.232+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:25.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:25 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:25.042+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:25.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:25 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:25.042+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:25.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:25 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:25.042+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:25.543 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:25 smithi155 podman[71633]: 2024-09-10 08:26:25.309652954 +0000 UTC m=+5.970969588 container cleanup e4200126d24a7ffe0e4929808506089769e41aeca0fc1a606e715255361e42b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:26:25.544 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:25 smithi155 bash[71633]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5 2024-09-10T08:26:25.544 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:25 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:25.351+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:25.544 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:25 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:25.351+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:25.544 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:25 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:25.351+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:25.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:25 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:25.483+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:25.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:25 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:25.483+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:56.483990+0000 front 2024-09-10T08:25:56.483927+0000 (oldest deadline 2024-09-10T08:26:20.584193+0000) 2024-09-10T08:26:25.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:25 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:25.432+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:25.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:25 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:25.432+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:25.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:25 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:25.432+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:25.906 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:25 smithi155 podman[71648]: 2024-09-10 08:26:25.544483518 +0000 UTC m=+0.651791366 container remove e4200126d24a7ffe0e4929808506089769e41aeca0fc1a606e715255361e42b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T08:26:25.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:25 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:25.747+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:25.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:25 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:25.747+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:25.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:25 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:25.747+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:26.305 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:25 smithi155 podman[71736]: 2024-09-10 08:26:25.865612677 +0000 UTC m=+0.015519360 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:26:26.305 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:25 smithi155 podman[71736]: 2024-09-10 08:26:25.979422211 +0000 UTC m=+0.129328894 container create 91e108d5d5c4a4c0f5b53a52f18bb568d515a58a1ab1e5f4f67e9ab980ad7dcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T08:26:26.305 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:26 smithi155 podman[71736]: 2024-09-10 08:26:26.20633633 +0000 UTC m=+0.356243013 container init 91e108d5d5c4a4c0f5b53a52f18bb568d515a58a1ab1e5f4f67e9ab980ad7dcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T08:26:26.306 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:26 smithi155 podman[71736]: 2024-09-10 08:26:26.209747375 +0000 UTC m=+0.359654059 container start 91e108d5d5c4a4c0f5b53a52f18bb568d515a58a1ab1e5f4f67e9ab980ad7dcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T08:26:26.306 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:26 smithi155 podman[71736]: 2024-09-10 08:26:26.306122342 +0000 UTC m=+0.456029038 container attach 91e108d5d5c4a4c0f5b53a52f18bb568d515a58a1ab1e5f4f67e9ab980ad7dcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T08:26:26.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:26 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:26.252+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:26.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:26 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:26.252+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:26.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:26 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:26.075+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:26.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:26 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:26.075+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:26.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:26 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:26.075+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:26.657 INFO:journalctl@ceph.osd.5.smithi155.stdout:Sep 10 08:26:26 smithi155 podman[71736]: 2024-09-10 08:26:26.400220378 +0000 UTC m=+0.550127062 container died 91e108d5d5c4a4c0f5b53a52f18bb568d515a58a1ab1e5f4f67e9ab980ad7dcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T08:26:26.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:26 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:26.355+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:26.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:26 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:26.355+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:26.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:26 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:26.355+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:26.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:26 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:26.489+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:26.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:26 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:26.489+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:56.483990+0000 front 2024-09-10T08:25:56.483927+0000 (oldest deadline 2024-09-10T08:26:20.584193+0000) 2024-09-10T08:26:26.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:26 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:26.409+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:26.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:26 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:26.409+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:26.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:26 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:26.409+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:27.026 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:26 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:26.779+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:27.027 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:26 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:26.779+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:27.027 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:26 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:26.779+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:27.045 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T08:26:27.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:26:27.124 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T08:26:27.124 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T08:26:27.124 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.6 2024-09-10T08:26:27.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:27 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:27.262+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:27.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:27 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:27.262+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:27.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:27 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:27.086+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:27.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:27 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:27.086+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:27.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:27 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:27.086+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:27.514 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:27 smithi002 systemd[1]: Stopping Ceph osd.6 for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:26:27.514 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:27 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:27.415+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:27.514 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:27 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:27.415+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:27.514 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:27 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:27.415+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:27.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:27 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:27.352+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:27.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:27 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:27.352+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:27.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:27 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:27.352+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:27.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:27 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:27.513+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:27.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:27 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:27.513+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:56.483990+0000 front 2024-09-10T08:25:56.483927+0000 (oldest deadline 2024-09-10T08:26:20.584193+0000) 2024-09-10T08:26:27.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:27 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:27.562+0000 7f703a600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:26:27.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:27 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:27.562+0000 7f703a600640 -1 osd.6 99 *** Got signal Terminated *** 2024-09-10T08:26:27.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:27 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:27.562+0000 7f703a600640 -1 osd.6 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:26:28.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:27 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:27.793+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:28.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:27 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:27.793+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:28.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:27 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:27.793+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:28.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:28 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:28.282+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:28.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:28 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:28.282+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:28.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:28 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:28.057+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:28.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:28 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:28.057+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:28.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:28 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:28.057+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:28.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:28 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:28.353+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:28.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:28 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:28.353+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:28.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:28 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:28.353+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:28.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:28 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:28.468+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:28.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:28 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:28.468+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:56.483990+0000 front 2024-09-10T08:25:56.483927+0000 (oldest deadline 2024-09-10T08:26:20.584193+0000) 2024-09-10T08:26:28.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:28 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:28.447+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:28.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:28 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:28.447+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:28.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:28 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:28.447+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:29.156 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:28 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:28.746+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:29.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:28 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:28.746+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:29.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:28 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:28.746+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:29.290 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:29 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:29.033+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:29.290 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:29 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:29.033+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:29.291 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:29 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:29.033+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:29.291 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:29 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:29.033+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:29.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:29 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:29.401+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:29.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:29 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:29.401+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:29.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:29 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:29.401+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:29.666 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:29 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:29.291+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:29.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:29 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:29.291+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:29.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:29 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:29.291+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.069859+0000 front 2024-09-10T08:26:07.069967+0000 (oldest deadline 2024-09-10T08:26:28.769878+0000) 2024-09-10T08:26:29.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:29 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:29.451+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:29.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:29 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:29.451+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:56.483990+0000 front 2024-09-10T08:25:56.483927+0000 (oldest deadline 2024-09-10T08:26:20.584193+0000) 2024-09-10T08:26:29.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:29 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:29.451+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.485718+0000 front 2024-09-10T08:26:06.485757+0000 (oldest deadline 2024-09-10T08:26:28.785929+0000) 2024-09-10T08:26:29.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:29 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:29.410+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:29.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:29 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:29.410+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:29.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:29 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:29.410+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:30.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:29 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:29.764+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:30.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:29 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:29.764+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:30.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:29 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:29.764+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:30.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:29 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:29.764+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:30.167 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:29 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:29.985+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:30.167 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:29 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:29.985+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:30.167 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:29 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:29.985+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:30.167 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:29 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:29.985+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:30.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:30 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:30.310+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:30.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:30 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:30.311+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:30.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:30 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:30.311+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.069859+0000 front 2024-09-10T08:26:07.069967+0000 (oldest deadline 2024-09-10T08:26:28.769878+0000) 2024-09-10T08:26:30.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:30 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:30.427+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:30.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:30 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:30.427+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:56.483990+0000 front 2024-09-10T08:25:56.483927+0000 (oldest deadline 2024-09-10T08:26:20.584193+0000) 2024-09-10T08:26:30.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:30 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:30.427+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.485718+0000 front 2024-09-10T08:26:06.485757+0000 (oldest deadline 2024-09-10T08:26:28.785929+0000) 2024-09-10T08:26:30.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:30 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:30.434+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:30.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:30 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:30.434+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:30.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:30 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:30.434+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:30.771 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:30 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:30.410+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:30.772 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:30 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:30.410+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:30.772 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:30 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:30.410+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:31.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:30 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:30.771+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:31.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:30 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:30.771+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:31.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:30 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:30.771+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:31.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:30 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:30.771+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:31.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:31 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:31.004+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:31.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:31 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:31.004+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:31.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:31 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:31.004+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:31.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:31 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:31.004+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:31.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:31 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:31.284+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:31.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:31 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:31.284+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:31.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:31 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:31.284+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.069859+0000 front 2024-09-10T08:26:07.069967+0000 (oldest deadline 2024-09-10T08:26:28.769878+0000) 2024-09-10T08:26:31.761 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:31 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:31.421+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:31.761 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:31 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:31.421+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:31.761 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:31 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:31.421+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:31.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:31.439+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:31.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:31.439+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:56.483990+0000 front 2024-09-10T08:25:56.483927+0000 (oldest deadline 2024-09-10T08:26:20.584193+0000) 2024-09-10T08:26:31.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:31.439+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.485718+0000 front 2024-09-10T08:26:06.485757+0000 (oldest deadline 2024-09-10T08:26:28.785929+0000) 2024-09-10T08:26:31.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:31.467+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:31.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:31.467+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:31.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:31.467+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:32.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:31 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:31.761+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:32.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:31 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:31.761+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:32.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:31 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:31.761+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:32.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:31 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:31.761+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:32.277 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:32 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:31.999+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:32.278 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:32 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:31.999+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:32.278 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:32 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:31.999+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:32.278 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:32 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:31.999+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:32.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:32 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:32.278+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:32.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:32 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:32.278+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:32.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:32 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:32.278+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.069859+0000 front 2024-09-10T08:26:07.069967+0000 (oldest deadline 2024-09-10T08:26:28.769878+0000) 2024-09-10T08:26:32.727 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:32 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:32.438+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:32.727 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:32 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:32.438+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:32.727 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:32 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:32.438+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:32.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:32 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:32.409+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.683075+0000 front 2024-09-10T08:25:50.683184+0000 (oldest deadline 2024-09-10T08:26:12.382879+0000) 2024-09-10T08:26:32.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:32 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:32.409+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:56.483990+0000 front 2024-09-10T08:25:56.483927+0000 (oldest deadline 2024-09-10T08:26:20.584193+0000) 2024-09-10T08:26:32.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:32 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6[54899]: 2024-09-10T08:26:32.409+0000 7f7036000640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.485718+0000 front 2024-09-10T08:26:06.485757+0000 (oldest deadline 2024-09-10T08:26:28.785929+0000) 2024-09-10T08:26:32.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:32 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:32.425+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:32.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:32 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:32.425+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:32.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:32 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:32.425+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:33.156 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:32 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:32.727+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:33.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:32 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:32.727+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:33.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:32 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:32.727+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:33.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:32 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:32.727+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:33.295 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:33 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:33.007+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:33.295 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:33 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:33.007+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:33.295 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:33 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:33.007+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:33.295 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:33 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:33.007+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:33.391 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:33 smithi002 podman[125523]: 2024-09-10 08:26:33.133726889 +0000 UTC m=+5.715728142 container died 370556ae7ce86d942f979b667967262ab9735dadad50f1438e469e0c4ba2c4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T08:26:33.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:33 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:33.396+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:33.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:33 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:33.396+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:33.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:33 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:33.396+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:33.666 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:33 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:33.296+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:33.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:33 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:33.296+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:33.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:33 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:33.296+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.069859+0000 front 2024-09-10T08:26:07.069967+0000 (oldest deadline 2024-09-10T08:26:28.769878+0000) 2024-09-10T08:26:33.770 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:33 smithi002 podman[125523]: 2024-09-10 08:26:33.609315012 +0000 UTC m=+6.191316253 container cleanup 370556ae7ce86d942f979b667967262ab9735dadad50f1438e469e0c4ba2c4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2) 2024-09-10T08:26:33.771 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:33 smithi002 bash[125523]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6 2024-09-10T08:26:33.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:33 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:33.403+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:33.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:33 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:33.403+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:33.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:33 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:33.403+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:33.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:33 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:33.403+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:34.156 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:33 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:33.757+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:34.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:33 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:33.757+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:34.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:33 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:33.757+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:34.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:33 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:33.757+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:34.270 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:33 smithi002 podman[125541]: 2024-09-10 08:26:33.828768949 +0000 UTC m=+0.679240051 container remove 370556ae7ce86d942f979b667967262ab9735dadad50f1438e469e0c4ba2c4c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T08:26:34.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:34 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:34.256+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:34.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:34 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:34.256+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:34.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:34 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:34.256+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.069859+0000 front 2024-09-10T08:26:07.069967+0000 (oldest deadline 2024-09-10T08:26:28.769878+0000) 2024-09-10T08:26:34.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:34 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:34.036+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:34.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:34 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:34.036+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:34.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:34 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:34.036+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:34.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:34 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:34.036+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:34.538 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:34 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:34.406+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:34.538 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:34 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:34.406+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:34.538 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:34 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:34.406+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:34.538 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:34 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:34.406+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:34.538 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:34 smithi002 podman[125625]: 2024-09-10 08:26:34.188786965 +0000 UTC m=+0.027670595 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:26:34.539 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:34 smithi002 podman[125625]: 2024-09-10 08:26:34.311512208 +0000 UTC m=+0.150395824 container create bccca54786e8e832d82dcc80920c90031bb1d12deaa91fa1644481dbb8478060 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:26:34.539 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:34 smithi002 podman[125625]: 2024-09-10 08:26:34.538871725 +0000 UTC m=+0.377755355 container init bccca54786e8e832d82dcc80920c90031bb1d12deaa91fa1644481dbb8478060 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:26:34.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:34 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:34.375+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:34.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:34 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:34.375+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:34.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:34 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:34.375+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:34.810 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:34 smithi002 podman[125625]: 2024-09-10 08:26:34.545774912 +0000 UTC m=+0.384658521 container start bccca54786e8e832d82dcc80920c90031bb1d12deaa91fa1644481dbb8478060 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:26:34.810 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:34 smithi002 podman[125625]: 2024-09-10 08:26:34.688677129 +0000 UTC m=+0.527560747 container attach bccca54786e8e832d82dcc80920c90031bb1d12deaa91fa1644481dbb8478060 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T08:26:35.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:34 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:34.740+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:35.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:34 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:34.740+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:35.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:34 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:34.740+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:35.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:34 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:34.740+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:35.207 INFO:journalctl@ceph.osd.6.smithi002.stdout:Sep 10 08:26:34 smithi002 podman[125625]: 2024-09-10 08:26:34.811441263 +0000 UTC m=+0.650324885 container died bccca54786e8e832d82dcc80920c90031bb1d12deaa91fa1644481dbb8478060 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:26:35.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:35 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:35.238+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:35.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:35 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:35.238+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:35.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:35 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:35.238+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.069859+0000 front 2024-09-10T08:26:07.069967+0000 (oldest deadline 2024-09-10T08:26:28.769878+0000) 2024-09-10T08:26:35.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:35 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:35.085+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:35.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:35 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:35.085+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:35.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:35 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:35.085+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:35.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:35 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:35.085+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:35.520 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:35 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:35.358+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:35.520 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:35 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:35.358+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:35.520 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:35 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:35.358+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:35.521 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:35 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:35.358+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:35.643 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T08:26:35.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:35 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:35.398+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:35.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:35 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:35.398+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:35.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:35 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:35.398+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:35.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:26:35.721 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T08:26:35.721 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T08:26:35.721 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.7 2024-09-10T08:26:36.017 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:35 smithi107 systemd[1]: Stopping Ceph osd.7 for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:26:36.156 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:35 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:35.692+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:36.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:35 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:35.692+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:36.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:35 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:35.692+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:36.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:35 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:35.692+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:36.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:36 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:36.113+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:36.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:36 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:36.113+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:36.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:36 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:36.113+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:36.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:36 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:36.113+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:36.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:36 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:36.016+0000 7f05b94a6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:26:36.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:36 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:36.016+0000 7f05b94a6640 -1 osd.7 99 *** Got signal Terminated *** 2024-09-10T08:26:36.418 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:36 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:36.016+0000 7f05b94a6640 -1 osd.7 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:26:36.418 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:36 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:36.253+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:36.418 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:36 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:36.253+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:36.418 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:36 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:36.253+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.069859+0000 front 2024-09-10T08:26:07.069967+0000 (oldest deadline 2024-09-10T08:26:28.769878+0000) 2024-09-10T08:26:36.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:36 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:36.396+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:36.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:36 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:36.396+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:36.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:36 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:36.396+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:36.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:36 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:36.316+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:36.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:36 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:36.316+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:36.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:36 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:36.316+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:36.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:36 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:36.316+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:37.156 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:36 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:36.683+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:37.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:36 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:36.683+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:37.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:36 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:36.683+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:37.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:36 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:36.683+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:37.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:37 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:37.264+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:37.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:37 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:37.264+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:37.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:37 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:37.264+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.069859+0000 front 2024-09-10T08:26:07.069967+0000 (oldest deadline 2024-09-10T08:26:28.769878+0000) 2024-09-10T08:26:37.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:37 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:37.140+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:37.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:37 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:37.140+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:37.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:37 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:37.140+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:37.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:37 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:37.140+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:37.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:37 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:37.346+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:37.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:37 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:37.346+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:37.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:37 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:37.346+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:37.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:37 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:37.338+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:37.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:37 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:37.338+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:37.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:37 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:37.338+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:37.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:37 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:37.338+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:38.156 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:37 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:37.659+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:38.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:37 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:37.659+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:38.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:37 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:37.659+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:38.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:37 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:37.659+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:38.655 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:38 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:38.299+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:38.655 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:38 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:38.299+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:38.655 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:38 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:38.299+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:38.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:38 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:38.252+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:38.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:38 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:38.252+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:38.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:38 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:38.252+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.069859+0000 front 2024-09-10T08:26:07.069967+0000 (oldest deadline 2024-09-10T08:26:28.769878+0000) 2024-09-10T08:26:38.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:38 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:38.252+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.172272+0000 front 2024-09-10T08:26:16.172333+0000 (oldest deadline 2024-09-10T08:26:37.272334+0000) 2024-09-10T08:26:38.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:38 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:38.182+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:38.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:38 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:38.182+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:38.668 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:38 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:38.182+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:38.668 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:38 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:38.182+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:38.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:38.355+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:38.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:38.355+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:38.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:38.355+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:38.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:38 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:38.355+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:38.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:38 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:38.656+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:38.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:38 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:38.656+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:38.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:38 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:38.656+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:38.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:38 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:38.656+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:39.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:39 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:39.301+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:39.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:39 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:39.301+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:39.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:39 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:39.301+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.069859+0000 front 2024-09-10T08:26:07.069967+0000 (oldest deadline 2024-09-10T08:26:28.769878+0000) 2024-09-10T08:26:39.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:39 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:39.301+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.172272+0000 front 2024-09-10T08:26:16.172333+0000 (oldest deadline 2024-09-10T08:26:37.272334+0000) 2024-09-10T08:26:39.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:39 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:39.155+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:39.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:39 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:39.155+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:39.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:39 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:39.155+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:39.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:39 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:39.155+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:39.616 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:39 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:39.309+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:39.617 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:39 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:39.309+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:39.617 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:39 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:39.309+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:39.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:39 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:39.347+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:39.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:39 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:39.347+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:39.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:39 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:39.347+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:39.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:39 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:39.347+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:39.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:39 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:39.347+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:39.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:39 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:39.617+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:39.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:39 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:39.617+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:39.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:39 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:39.617+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:39.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:39 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:39.617+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:40.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:40 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:40.256+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:51.466884+0000 front 2024-09-10T08:25:51.466770+0000 (oldest deadline 2024-09-10T08:26:12.567047+0000) 2024-09-10T08:26:40.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:40 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:40.256+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.269145+0000 front 2024-09-10T08:25:58.269049+0000 (oldest deadline 2024-09-10T08:26:21.168998+0000) 2024-09-10T08:26:40.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:40 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:40.256+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.069859+0000 front 2024-09-10T08:26:07.069967+0000 (oldest deadline 2024-09-10T08:26:28.769878+0000) 2024-09-10T08:26:40.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:40 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7[52619]: 2024-09-10T08:26:40.256+0000 7f05b1ab4640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.172272+0000 front 2024-09-10T08:26:16.172333+0000 (oldest deadline 2024-09-10T08:26:37.272334+0000) 2024-09-10T08:26:40.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:40 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:40.156+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:40.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:40 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:40.156+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:40.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:40 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:40.156+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:40.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:40 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:40.156+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:40.648 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:40 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:40.322+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:40.649 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:40 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:40.322+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:40.649 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:40 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:40.322+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:40.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:40 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:40.334+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:40.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:40 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:40.334+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:40.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:40 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:40.334+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:40.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:40 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:40.334+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:40.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:40 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:40.334+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:40.906 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:40 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:40.648+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:40.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:40 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:40.648+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:40.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:40 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:40.648+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:40.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:40 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:40.648+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:41.417 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:41 smithi107 podman[70214]: 2024-09-10 08:26:41.329773746 +0000 UTC m=+5.411997351 container died c59e4d9ba134aeed0ce1d53b9e4c64a05e3eb7e88c69e637f64d9cdad933c9d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T08:26:41.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:41 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:41.112+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:41.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:41 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:41.112+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:41.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:41 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:41.112+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:41.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:41 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:41.112+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:41.628 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:41 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:41.282+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:41.628 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:41 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:41.282+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:41.628 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:41 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:41.282+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:41.628 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:41 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:41.282+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:41.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:41 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:41.320+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:41.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:41 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:41.320+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:41.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:41 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:41.320+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:41.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:41 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:41.320+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:41.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:41 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:41.320+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:41.906 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:41 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:41.628+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:41.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:41 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:41.628+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:41.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:41 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:41.628+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:41.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:41 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:41.628+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:41.917 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:41 smithi107 podman[70214]: 2024-09-10 08:26:41.716163006 +0000 UTC m=+5.798386611 container cleanup c59e4d9ba134aeed0ce1d53b9e4c64a05e3eb7e88c69e637f64d9cdad933c9d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T08:26:41.917 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:41 smithi107 bash[70214]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7 2024-09-10T08:26:42.330 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:41 smithi107 podman[70229]: 2024-09-10 08:26:41.975683432 +0000 UTC m=+0.640318080 container remove c59e4d9ba134aeed0ce1d53b9e4c64a05e3eb7e88c69e637f64d9cdad933c9d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:26:42.330 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:42 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:42.159+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:42.330 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:42 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:42.159+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:42.330 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:42 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:42.159+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:42.330 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:42 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:42.159+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:42.331 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:42 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:42.159+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:42.611 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:42 smithi107 podman[70320]: 2024-09-10 08:26:42.233560201 +0000 UTC m=+0.015706224 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:26:42.611 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:42 smithi107 podman[70320]: 2024-09-10 08:26:42.351672851 +0000 UTC m=+0.133818876 container create e14428634339fd4688a31c8a397dbcac93ad955c1dae41e2f998037fac548ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T08:26:42.611 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:42 smithi107 podman[70320]: 2024-09-10 08:26:42.55389733 +0000 UTC m=+0.336043355 container init e14428634339fd4688a31c8a397dbcac93ad955c1dae41e2f998037fac548ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2) 2024-09-10T08:26:42.611 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:42 smithi107 podman[70320]: 2024-09-10 08:26:42.557063495 +0000 UTC m=+0.339209519 container start e14428634339fd4688a31c8a397dbcac93ad955c1dae41e2f998037fac548ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2) 2024-09-10T08:26:42.611 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:42 smithi107 podman[70320]: 2024-09-10 08:26:42.612075081 +0000 UTC m=+0.394221116 container attach e14428634339fd4688a31c8a397dbcac93ad955c1dae41e2f998037fac548ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:26:42.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:42 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:42.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:42.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:42 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:42.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:42.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:42 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:42.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:42.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:42 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:42.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:42.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:42.360+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:42.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:42.360+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:42.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:42.360+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:42.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:42.360+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:42.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:42 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:42.360+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:42.916 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 10 08:26:42 smithi107 podman[70320]: 2024-09-10 08:26:42.740763038 +0000 UTC m=+0.522909063 container died e14428634339fd4688a31c8a397dbcac93ad955c1dae41e2f998037fac548ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T08:26:43.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:42 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:42.677+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:43.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:42 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:42.677+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:43.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:42 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:42.677+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:43.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:42 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:42.677+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:43.339 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:43 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:43.163+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:43.339 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:43 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:43.163+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:43.340 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:43 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:43.163+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:43.340 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:43 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:43.163+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:43.340 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:43 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:43.163+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:43.370 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T08:26:43.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:26:43.411 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T08:26:43.411 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T08:26:43.411 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.8 2024-09-10T08:26:43.657 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:43 smithi155 systemd[1]: Stopping Ceph osd.8 for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:26:43.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:43 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:43.378+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:43.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:43 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:43.378+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:43.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:43 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:43.378+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:43.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:43 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:43.378+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:43.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:43 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:43.389+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:43.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:43 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:43.389+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:43.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:43 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:43.389+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:43.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:43 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:43.389+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:43.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:43 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:43.389+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:44.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:43 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:43.694+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:44.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:43 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:43.694+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:44.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:43 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:43.694+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:44.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:43 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:43.694+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:44.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:43 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:43.744+0000 7f2ecb6aa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:26:44.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:43 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:43.744+0000 7f2ecb6aa640 -1 osd.8 99 *** Got signal Terminated *** 2024-09-10T08:26:44.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:43 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:43.744+0000 7f2ecb6aa640 -1 osd.8 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:26:44.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:44 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:44.150+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:44.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:44 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:44.150+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:44.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:44 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:44.150+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:44.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:44 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:44.150+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:44.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:44 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:44.150+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:44.677 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:44 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:44.417+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:44.677 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:44 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:44.417+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:44.677 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:44 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:44.417+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:44.677 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:44 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:44.417+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:44.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:44 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:44.427+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:44.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:44 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:44.427+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:44.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:44 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:44.427+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:44.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:44 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:44.427+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:44.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:44 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:44.427+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:45.156 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:44 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:44.677+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:45.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:44 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:44.677+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:45.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:44 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:44.677+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:45.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:44 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:44.677+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:45.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:45 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:45.126+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:45.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:45 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:45.126+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:45.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:45 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:45.126+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:45.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:45 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:45.126+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:45.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:45 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:45.126+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:45.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:45 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:45.471+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:45.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:45 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:45.471+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:45.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:45 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:45.471+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:45.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:45 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:45.471+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:45.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:45 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:45.471+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:45.906 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:45 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:45.451+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:45.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:45 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:45.451+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:45.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:45 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:45.451+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:45.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:45 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:45.451+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:45.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:45 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:45.662+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:45.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:45 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:45.662+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:45.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:45 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:45.662+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:45.907 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:45 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:45.662+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:46.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:46 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:46.166+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:46.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:46 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:46.166+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:46.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:46 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:46.166+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:46.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:46 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:46.166+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:46.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:46 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:46.166+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:46.694 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:46 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:46.408+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:46.694 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:46 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:46.408+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:46.694 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:46 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:46.408+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:46.694 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:46 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:46.408+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:46.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:46 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:46.507+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:46.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:46 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:46.507+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:46.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:46 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:46.507+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:46.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:46 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:46.507+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:46.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:46 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:46.507+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:47.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:46 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:46.694+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:47.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:46 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:46.694+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:47.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:46 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:46.694+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:47.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:46 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:46.694+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:47.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:47 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:47.141+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:47.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:47 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:47.141+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:47.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:47 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:47.141+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:47.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:47 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:47.141+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:47.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:47 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:47.141+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:47.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:47 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:47.141+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:26:47.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:47 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:47.394+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:47.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:47 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:47.394+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:47.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:47 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:47.394+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:47.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:47 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:47.394+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:48.020 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:47 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:47.547+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:48.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:47 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:47.547+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:48.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:47 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:47.547+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:48.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:47 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:47.547+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:48.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:47 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:47.547+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:48.156 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:47 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:47.701+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:48.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:47 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:47.701+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:48.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:47 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:47.701+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:48.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:47 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:47.701+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:48.157 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:47 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:47.701+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:18.831771+0000 front 2024-09-10T08:26:18.831748+0000 (oldest deadline 2024-09-10T08:26:47.632224+0000) 2024-09-10T08:26:48.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:48 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:48.160+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:48.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:48 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:48.160+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:48.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:48 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:48.160+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:48.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:48 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:48.160+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:48.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:48 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:48.160+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:48.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:48 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:48.160+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:26:48.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:48 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:48.348+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:48.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:48 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:48.348+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:48.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:48 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:48.348+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:48.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:48 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:48.348+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:49.020 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:48 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:48.543+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:49.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:48 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:48.543+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:49.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:48 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:48.543+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:49.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:48 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:48.543+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:49.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:48 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:48.543+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:49.116 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:48 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:48.684+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:39.726567+0000 front 2024-09-10T08:25:39.726484+0000 (oldest deadline 2024-09-10T08:26:04.426615+0000) 2024-09-10T08:26:49.117 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:48 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:48.684+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.728666+0000 front 2024-09-10T08:25:50.728525+0000 (oldest deadline 2024-09-10T08:26:13.028402+0000) 2024-09-10T08:26:49.117 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:48 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:48.684+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.929140+0000 front 2024-09-10T08:25:58.929252+0000 (oldest deadline 2024-09-10T08:26:23.629279+0000) 2024-09-10T08:26:49.117 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:48 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:48.684+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:03.629841+0000 front 2024-09-10T08:26:03.629828+0000 (oldest deadline 2024-09-10T08:26:28.929714+0000) 2024-09-10T08:26:49.117 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:48 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8[53634]: 2024-09-10T08:26:48.684+0000 7f2ec3cb8640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:18.831771+0000 front 2024-09-10T08:26:18.831748+0000 (oldest deadline 2024-09-10T08:26:47.632224+0000) 2024-09-10T08:26:49.397 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:49 smithi155 podman[71838]: 2024-09-10 08:26:49.117593432 +0000 UTC m=+5.503128510 container died 83bf4c2aa07e70ff629fcfa2b2da9d0b2fa6f6778e129f30cb8eeedf1154a98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:26:49.397 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:49 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:49.323+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:49.397 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:49 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:49.323+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:49.397 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:49 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:49.323+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:49.397 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:49 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:49.323+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:49.656 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:49 smithi155 podman[71838]: 2024-09-10 08:26:49.503923232 +0000 UTC m=+5.889458297 container cleanup 83bf4c2aa07e70ff629fcfa2b2da9d0b2fa6f6778e129f30cb8eeedf1154a98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T08:26:49.656 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:49 smithi155 bash[71838]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8 2024-09-10T08:26:49.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:49 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:49.193+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:49.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:49 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:49.193+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:49.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:49 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:49.193+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:49.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:49 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:49.193+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:49.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:49 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:49.193+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:49.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:49 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:49.193+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:26:49.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:49 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:49.499+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:49.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:49 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:49.499+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:49.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:49 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:49.499+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:49.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:49 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:49.499+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:49.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:49 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:49.499+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:49.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:49 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:49.499+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.168308+0000 front 2024-09-10T08:26:24.168200+0000 (oldest deadline 2024-09-10T08:26:49.468078+0000) 2024-09-10T08:26:50.113 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:49 smithi155 podman[71854]: 2024-09-10 08:26:49.763669131 +0000 UTC m=+0.643089177 container remove 83bf4c2aa07e70ff629fcfa2b2da9d0b2fa6f6778e129f30cb8eeedf1154a98b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T08:26:50.374 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:50 smithi155 podman[71944]: 2024-09-10 08:26:50.114400965 +0000 UTC m=+0.113250344 container create ca10d3ec97e372d8b4bdda37086d794657ceb3e6f02ce20487daee6577510262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T08:26:50.374 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:50 smithi155 podman[71944]: 2024-09-10 08:26:50.017228472 +0000 UTC m=+0.016077860 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:26:50.374 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:50 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:50.357+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:50.375 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:50 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:50.357+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:50.375 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:50 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:50.357+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:50.375 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:50 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:50.357+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:50.375 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:50 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:50.357+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:26:50.657 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:50 smithi155 podman[71944]: 2024-09-10 08:26:50.375276952 +0000 UTC m=+0.374126338 container init ca10d3ec97e372d8b4bdda37086d794657ceb3e6f02ce20487daee6577510262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T08:26:50.657 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:50 smithi155 podman[71944]: 2024-09-10 08:26:50.378865446 +0000 UTC m=+0.377714834 container start ca10d3ec97e372d8b4bdda37086d794657ceb3e6f02ce20487daee6577510262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2) 2024-09-10T08:26:50.657 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:50 smithi155 podman[71944]: 2024-09-10 08:26:50.475394155 +0000 UTC m=+0.474243549 container attach ca10d3ec97e372d8b4bdda37086d794657ceb3e6f02ce20487daee6577510262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:26:50.657 INFO:journalctl@ceph.osd.8.smithi155.stdout:Sep 10 08:26:50 smithi155 podman[71944]: 2024-09-10 08:26:50.561300024 +0000 UTC m=+0.560149652 container died ca10d3ec97e372d8b4bdda37086d794657ceb3e6f02ce20487daee6577510262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:26:50.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:50 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:50.184+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:50.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:50 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:50.184+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:50.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:50 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:50.184+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:50.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:50 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:50.184+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:50.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:50 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:50.184+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:50.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:50 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:50.184+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:26:50.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:50 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:50.476+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:50.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:50 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:50.476+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:50.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:50 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:50.476+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:50.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:50 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:50.476+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:50.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:50 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:50.476+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:50.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:50 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:50.476+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.168308+0000 front 2024-09-10T08:26:24.168200+0000 (oldest deadline 2024-09-10T08:26:49.468078+0000) 2024-09-10T08:26:51.171 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T08:26:51.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:26:51.213 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T08:26:51.213 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T08:26:51.213 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.9 2024-09-10T08:26:51.406 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:51 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:51.320+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:51.407 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:51 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:51.320+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:51.407 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:51 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:51.320+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:51.407 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:51 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:51.320+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:51.407 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:51 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:51.320+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:26:51.517 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:51 smithi002 systemd[1]: Stopping Ceph osd.9 for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:26:51.517 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:51 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:51.516+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:51.517 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:51 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:51.516+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:51.517 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:51 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:51.516+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:51.517 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:51 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:51.516+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:51.517 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:51 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:51.516+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:51.518 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:51 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:51.516+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.168308+0000 front 2024-09-10T08:26:24.168200+0000 (oldest deadline 2024-09-10T08:26:49.468078+0000) 2024-09-10T08:26:51.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:51 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:51.189+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:51.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:51 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:51.189+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:51.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:51 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:51.189+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:51.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:51 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:51.189+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:51.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:51 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:51.189+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:51.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:51 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:51.189+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:26:51.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:51 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:51.633+0000 7ff7bb200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:26:51.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:51 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:51.633+0000 7ff7bb200640 -1 osd.9 99 *** Got signal Terminated *** 2024-09-10T08:26:51.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:51 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:51.633+0000 7ff7bb200640 -1 osd.9 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:26:52.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:52 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:52.274+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:52.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:52 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:52.274+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:52.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:52 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:52.274+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:52.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:52 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:52.274+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:52.658 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:52 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:52.274+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:26:52.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:52 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:52.189+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:52.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:52 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:52.189+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:52.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:52 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:52.189+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:52.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:52 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:52.189+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:52.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:52 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:52.189+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:52.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:52 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:52.189+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:26:52.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:52 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:52.515+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:52.770 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:52 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:52.515+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:52.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:52 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:52.515+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:52.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:52 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:52.515+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:52.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:52 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:52.515+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:52.771 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:52 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:52.515+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.168308+0000 front 2024-09-10T08:26:24.168200+0000 (oldest deadline 2024-09-10T08:26:49.468078+0000) 2024-09-10T08:26:53.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:53 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:53.261+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:53.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:53 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:53.261+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:53.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:53 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:53.261+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:53.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:53 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:53.261+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:53.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:53 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:53.261+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:26:53.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:53 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:53.179+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:53.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:53 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:53.179+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:53.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:53 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:53.179+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:53.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:53 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:53.179+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:53.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:53 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:53.179+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:53.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:53 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:53.179+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:26:54.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:53 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:53.520+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:54.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:53 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:53.520+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:54.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:53 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:53.520+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:54.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:53 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:53.520+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:54.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:53 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:53.520+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:54.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:53 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:53.520+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.168308+0000 front 2024-09-10T08:26:24.168200+0000 (oldest deadline 2024-09-10T08:26:49.468078+0000) 2024-09-10T08:26:54.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:54 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:54.296+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:54.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:54 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:54.296+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:54.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:54 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:54.296+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:54.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:54 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:54.296+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:54.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:54 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:54.296+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:26:54.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:54 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:54.206+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:54.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:54 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:54.206+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:54.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:54 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:54.206+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:54.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:54 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:54.206+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:54.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:54 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:54.206+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:54.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:54 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:54.206+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:26:55.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:54 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:54.556+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:55.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:54 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:54.556+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:55.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:54 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:54.556+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:55.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:54 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:54.556+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:55.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:54 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:54.556+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:55.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:54 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:54.556+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.168308+0000 front 2024-09-10T08:26:24.168200+0000 (oldest deadline 2024-09-10T08:26:49.468078+0000) 2024-09-10T08:26:55.558 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:55 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:55.294+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:55.558 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:55 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:55.294+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:55.558 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:55 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:55.294+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:55.558 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:55 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:55.294+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:55.558 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:55 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:55.294+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:26:55.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:55 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:55.226+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:55.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:55 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:55.226+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:55.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:55 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:55.226+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:55.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:55 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:55.226+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:55.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:55 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:55.226+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:55.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:55 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:55.226+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:26:56.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:55 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:55.583+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:56.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:55 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:55.583+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:56.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:55 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:55.583+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:56.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:55 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:55.583+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:56.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:55 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:55.583+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:56.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:55 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:55.583+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.168308+0000 front 2024-09-10T08:26:24.168200+0000 (oldest deadline 2024-09-10T08:26:49.468078+0000) 2024-09-10T08:26:56.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:56 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:56.323+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:56.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:56 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:56.323+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:56.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:56 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:56.323+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:56.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:56 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:56.323+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:56.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:56 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:56.323+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:26:56.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:56 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:56.179+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:56.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:56 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:56.179+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:56.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:56 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:56.179+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:56.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:56 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:56.179+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:56.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:56 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:56.179+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:56.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:56 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:56.179+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:26:56.993 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:56 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:56.568+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:40.559747+0000 front 2024-09-10T08:25:40.559929+0000 (oldest deadline 2024-09-10T08:26:04.659918+0000) 2024-09-10T08:26:56.993 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:56 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:56.568+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.462316+0000 front 2024-09-10T08:25:50.462241+0000 (oldest deadline 2024-09-10T08:26:13.362057+0000) 2024-09-10T08:26:56.993 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:56 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:56.568+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.764249+0000 front 2024-09-10T08:25:59.764390+0000 (oldest deadline 2024-09-10T08:26:20.264004+0000) 2024-09-10T08:26:56.993 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:56 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:56.568+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:06.665511+0000 front 2024-09-10T08:26:06.665760+0000 (oldest deadline 2024-09-10T08:26:32.565651+0000) 2024-09-10T08:26:56.994 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:56 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:56.568+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:13.066805+0000 front 2024-09-10T08:26:13.067359+0000 (oldest deadline 2024-09-10T08:26:38.966481+0000) 2024-09-10T08:26:56.994 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:56 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9[61882]: 2024-09-10T08:26:56.568+0000 7ff7b1c00640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.168308+0000 front 2024-09-10T08:26:24.168200+0000 (oldest deadline 2024-09-10T08:26:49.468078+0000) 2024-09-10T08:26:57.269 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:56 smithi002 podman[125726]: 2024-09-10 08:26:56.992637763 +0000 UTC m=+5.483467184 container died f6dbc8f39405dc33fef53a1af648bf3f295e9c16317ac6de329bb604746930f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T08:26:57.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:57 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:57.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:57.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:57 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:57.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:57.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:57 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:57.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:57.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:57 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:57.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:57.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:57 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:57.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:57.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:57 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:57.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:26:57.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:57 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:57.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:31.137192+0000 front 2024-09-10T08:26:31.137289+0000 (oldest deadline 2024-09-10T08:26:57.036962+0000) 2024-09-10T08:26:57.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:57 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:57.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:57.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:57 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:57.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:57.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:57 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:57.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:57.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:57 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:57.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:57.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:57 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:57.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:26:58.020 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:57 smithi002 podman[125726]: 2024-09-10 08:26:57.627115411 +0000 UTC m=+6.117944807 container cleanup f6dbc8f39405dc33fef53a1af648bf3f295e9c16317ac6de329bb604746930f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T08:26:58.020 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:57 smithi002 bash[125726]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9 2024-09-10T08:26:58.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:57 smithi002 podman[125742]: 2024-09-10 08:26:57.852061773 +0000 UTC m=+0.851053622 container remove f6dbc8f39405dc33fef53a1af648bf3f295e9c16317ac6de329bb604746930f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:26:58.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:58 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:58.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:58.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:58 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:58.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:58.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:58 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:58.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:58.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:58 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:58.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:58.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:58 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:58.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:58.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:58 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:58.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:26:58.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:58 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:58.157+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:31.137192+0000 front 2024-09-10T08:26:31.137289+0000 (oldest deadline 2024-09-10T08:26:57.036962+0000) 2024-09-10T08:26:58.656 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:58 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:58.316+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:58.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:58 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:58.316+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:58.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:58 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:58.316+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:58.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:58 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:58.316+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:58.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:58 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:58.316+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:26:58.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:58 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:58.316+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:26:58.693 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:58 smithi002 podman[125827]: 2024-09-10 08:26:58.313677163 +0000 UTC m=+0.027607636 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:26:58.693 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:58 smithi002 podman[125827]: 2024-09-10 08:26:58.537308074 +0000 UTC m=+0.251238536 container create 210c0efad6fe5d20b2648cc7b10afdce9971a245256abac1a202f0d67e2d03b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T08:26:59.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:58 smithi002 podman[125827]: 2024-09-10 08:26:58.781048029 +0000 UTC m=+0.494978518 container init 210c0efad6fe5d20b2648cc7b10afdce9971a245256abac1a202f0d67e2d03b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:26:59.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:58 smithi002 podman[125827]: 2024-09-10 08:26:58.787918115 +0000 UTC m=+0.501848582 container start 210c0efad6fe5d20b2648cc7b10afdce9971a245256abac1a202f0d67e2d03b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T08:26:59.021 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:58 smithi002 podman[125827]: 2024-09-10 08:26:58.864384416 +0000 UTC m=+0.578314903 container attach 210c0efad6fe5d20b2648cc7b10afdce9971a245256abac1a202f0d67e2d03b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T08:26:59.412 INFO:journalctl@ceph.osd.9.smithi002.stdout:Sep 10 08:26:59 smithi002 podman[125827]: 2024-09-10 08:26:59.053141961 +0000 UTC m=+0.767072425 container died 210c0efad6fe5d20b2648cc7b10afdce9971a245256abac1a202f0d67e2d03b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T08:26:59.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:59 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:59.304+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:26:59.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:59 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:59.304+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:26:59.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:59 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:59.304+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:26:59.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:59 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:59.304+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:26:59.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:59 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:59.304+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:26:59.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:26:59 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:26:59.304+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:26:59.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:59 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:59.206+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:26:59.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:59 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:59.206+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:26:59.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:59 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:59.206+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:26:59.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:59 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:59.206+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:26:59.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:59 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:59.206+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:26:59.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:59 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:59.206+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:26:59.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:26:59 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:26:59.206+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:31.137192+0000 front 2024-09-10T08:26:31.137289+0000 (oldest deadline 2024-09-10T08:26:57.036962+0000) 2024-09-10T08:26:59.869 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T08:26:59.946 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:26:59.947 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T08:26:59.947 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T08:26:59.947 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.10 2024-09-10T08:27:00.274 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:00 smithi107 systemd[1]: Stopping Ceph osd.10 for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:27:00.274 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:00 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:00.220+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:27:00.275 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:00 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:00.220+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:27:00.275 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:00 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:00.220+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:27:00.275 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:00 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:00.220+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:27:00.275 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:00 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:00.220+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:27:00.275 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:00 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:00.220+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:27:00.275 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:00 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:00.220+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:31.137192+0000 front 2024-09-10T08:26:31.137289+0000 (oldest deadline 2024-09-10T08:26:57.036962+0000) 2024-09-10T08:27:00.276 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:00 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:00.274+0000 7fc2d2e98640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:27:00.276 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:00 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:00.274+0000 7fc2d2e98640 -1 osd.10 99 *** Got signal Terminated *** 2024-09-10T08:27:00.276 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:00 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:00.274+0000 7fc2d2e98640 -1 osd.10 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:27:00.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:00 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:00.320+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:27:00.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:00 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:00.320+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:27:00.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:00 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:00.320+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:27:00.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:00 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:00.320+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:27:00.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:00 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:00.320+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:27:00.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:00 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:00.320+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:27:01.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:01 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:01.357+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:27:01.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:01 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:01.357+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:27:01.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:01 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:01.357+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:27:01.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:01 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:01.357+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:27:01.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:01 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:01.357+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:27:01.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:01 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:01.357+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:27:01.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:01 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:01.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:27:01.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:01 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:01.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:27:01.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:01 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:01.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:27:01.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:01 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:01.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:27:01.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:01 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:01.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:27:01.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:01 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:01.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:27:01.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:01 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:01.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:31.137192+0000 front 2024-09-10T08:26:31.137289+0000 (oldest deadline 2024-09-10T08:26:57.036962+0000) 2024-09-10T08:27:02.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:02 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:02.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:27:02.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:02 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:02.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:27:02.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:02 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:02.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:27:02.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:02 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:02.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:27:02.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:02 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:02.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:27:02.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:02 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:02.331+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:27:02.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:02 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:02.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:27:02.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:02 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:02.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:27:02.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:02 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:02.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:27:02.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:02 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:02.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:27:02.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:02 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:02.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:27:02.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:02 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:02.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:27:02.667 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:02 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:02.174+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:31.137192+0000 front 2024-09-10T08:26:31.137289+0000 (oldest deadline 2024-09-10T08:26:57.036962+0000) 2024-09-10T08:27:03.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:03 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:03.147+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:27:03.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:03 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:03.147+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:27:03.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:03 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:03.147+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:27:03.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:03 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:03.147+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:27:03.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:03 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:03.147+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:27:03.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:03 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:03.147+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:27:03.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:03 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:03.147+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:31.137192+0000 front 2024-09-10T08:26:31.137289+0000 (oldest deadline 2024-09-10T08:26:57.036962+0000) 2024-09-10T08:27:03.659 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:03 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:03.317+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:27:03.659 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:03 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:03.317+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:27:03.659 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:03 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:03.317+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:27:03.659 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:03 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:03.317+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:27:03.659 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:03 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:03.317+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:27:03.659 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:03 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:03.317+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:27:04.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:04 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:04.106+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:27:04.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:04 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:04.106+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:27:04.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:04 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:04.106+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:27:04.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:04 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:04.106+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:27:04.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:04 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:04.106+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:27:04.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:04 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:04.106+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:27:04.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:04 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:04.106+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:31.137192+0000 front 2024-09-10T08:26:31.137289+0000 (oldest deadline 2024-09-10T08:26:57.036962+0000) 2024-09-10T08:27:04.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:04 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:04.362+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:27:04.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:04 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:04.362+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:27:04.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:04 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:04.362+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:27:04.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:04 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:04.362+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:27:04.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:04 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:04.362+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:27:04.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:04 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:04.362+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:27:05.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:05 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:05.139+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:41.029125+0000 front 2024-09-10T08:25:41.029086+0000 (oldest deadline 2024-09-10T08:26:04.529098+0000) 2024-09-10T08:27:05.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:05 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:05.139+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:50.330375+0000 front 2024-09-10T08:25:50.330612+0000 (oldest deadline 2024-09-10T08:26:12.630545+0000) 2024-09-10T08:27:05.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:05 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:05.139+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:59.032162+0000 front 2024-09-10T08:25:59.032198+0000 (oldest deadline 2024-09-10T08:26:23.732361+0000) 2024-09-10T08:27:05.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:05 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:05.139+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-09-10T08:26:07.734221+0000 front 2024-09-10T08:26:07.734159+0000 (oldest deadline 2024-09-10T08:26:28.833844+0000) 2024-09-10T08:27:05.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:05 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:05.139+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:16.435226+0000 front 2024-09-10T08:26:16.435249+0000 (oldest deadline 2024-09-10T08:26:41.735428+0000) 2024-09-10T08:27:05.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:05 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:05.139+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:21.736139+0000 front 2024-09-10T08:26:21.736177+0000 (oldest deadline 2024-09-10T08:26:46.435920+0000) 2024-09-10T08:27:05.418 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:05 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10[59159]: 2024-09-10T08:27:05.139+0000 7fc2cb4a6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:31.137192+0000 front 2024-09-10T08:26:31.137289+0000 (oldest deadline 2024-09-10T08:26:57.036962+0000) 2024-09-10T08:27:05.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:05.340+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:27:05.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:05.340+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:27:05.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:05.340+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:27:05.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:05.340+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:27:05.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:05.340+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:27:05.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:05.340+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:27:05.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:05 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:05.340+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-10T08:26:39.289438+0000 front 2024-09-10T08:26:39.289561+0000 (oldest deadline 2024-09-10T08:27:05.189308+0000) 2024-09-10T08:27:06.065 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:05 smithi107 podman[70437]: 2024-09-10 08:27:05.802299204 +0000 UTC m=+5.651468161 container died f0ec50d5c682c1d364d9cfe521aabd13d9290e211e74c9aba287f0e95b588b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T08:27:06.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:06 smithi107 podman[70437]: 2024-09-10 08:27:06.164846839 +0000 UTC m=+6.014015788 container cleanup f0ec50d5c682c1d364d9cfe521aabd13d9290e211e74c9aba287f0e95b588b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T08:27:06.417 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:06 smithi107 bash[70437]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10 2024-09-10T08:27:06.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:06 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:06.377+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:27:06.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:06 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:06.377+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:27:06.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:06 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:06.377+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:27:06.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:06 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:06.377+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:27:06.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:06 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:06.377+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:27:06.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:06 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:06.377+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:27:06.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:06 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:06.377+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-10T08:26:39.289438+0000 front 2024-09-10T08:26:39.289561+0000 (oldest deadline 2024-09-10T08:27:05.189308+0000) 2024-09-10T08:27:06.724 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:06 smithi107 podman[70452]: 2024-09-10 08:27:06.424189749 +0000 UTC m=+0.617996932 container remove f0ec50d5c682c1d364d9cfe521aabd13d9290e211e74c9aba287f0e95b588b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T08:27:07.009 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:06 smithi107 podman[70542]: 2024-09-10 08:27:06.724939865 +0000 UTC m=+0.107189714 container create 34911fc1b2090d93c0edc2c949716f259a5ba67a22632f353f35fe3600ab17e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:27:07.009 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:06 smithi107 podman[70542]: 2024-09-10 08:27:06.633645364 +0000 UTC m=+0.015895223 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:27:07.010 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:06 smithi107 podman[70542]: 2024-09-10 08:27:06.92696609 +0000 UTC m=+0.309215939 container init 34911fc1b2090d93c0edc2c949716f259a5ba67a22632f353f35fe3600ab17e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T08:27:07.010 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:06 smithi107 podman[70542]: 2024-09-10 08:27:06.930163793 +0000 UTC m=+0.312413647 container start 34911fc1b2090d93c0edc2c949716f259a5ba67a22632f353f35fe3600ab17e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:27:07.010 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:07 smithi107 podman[70542]: 2024-09-10 08:27:07.010103813 +0000 UTC m=+0.392353672 container attach 34911fc1b2090d93c0edc2c949716f259a5ba67a22632f353f35fe3600ab17e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T08:27:07.404 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:07 smithi107 podman[70542]: 2024-09-10 08:27:07.1153846 +0000 UTC m=+0.497634449 container died 34911fc1b2090d93c0edc2c949716f259a5ba67a22632f353f35fe3600ab17e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True) 2024-09-10T08:27:07.655 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 10 08:27:07 smithi107 podman[70566]: 2024-09-10 08:27:07.646485313 +0000 UTC m=+0.525193010 container remove 34911fc1b2090d93c0edc2c949716f259a5ba67a22632f353f35fe3600ab17e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:27:07.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:07 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:07.378+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:27:07.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:07 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:07.378+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:27:07.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:07 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:07.378+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:27:07.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:07 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:07.378+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:27:07.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:07 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:07.378+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:27:07.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:07 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:07.378+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:27:07.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:07 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:07.378+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-10T08:26:39.289438+0000 front 2024-09-10T08:26:39.289561+0000 (oldest deadline 2024-09-10T08:27:05.189308+0000) 2024-09-10T08:27:07.661 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T08:27:07.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:27:07.702 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T08:27:07.702 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T08:27:07.702 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.11 2024-09-10T08:27:08.014 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:07 smithi155 systemd[1]: Stopping Ceph osd.11 for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:27:08.392 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:08.014+0000 7f94208db640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:27:08.393 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:08.014+0000 7f94208db640 -1 osd.11 99 *** Got signal Terminated *** 2024-09-10T08:27:08.393 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:08.014+0000 7f94208db640 -1 osd.11 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:27:08.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:08.393+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:27:08.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:08.393+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:27:08.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:08.393+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:27:08.657 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:08.393+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:27:08.658 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:08.393+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:27:08.658 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:08.393+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:27:08.658 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:08 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:08.393+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-10T08:26:39.289438+0000 front 2024-09-10T08:26:39.289561+0000 (oldest deadline 2024-09-10T08:27:05.189308+0000) 2024-09-10T08:27:09.906 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:09 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:09.432+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:27:09.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:09 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:09.432+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:27:09.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:09 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:09.432+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:27:09.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:09 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:09.432+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:27:09.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:09 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:09.432+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:27:09.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:09 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:09.432+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:27:09.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:09 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:09.432+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-10T08:26:39.289438+0000 front 2024-09-10T08:26:39.289561+0000 (oldest deadline 2024-09-10T08:27:05.189308+0000) 2024-09-10T08:27:10.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:10 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:10.480+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:27:10.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:10 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:10.480+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:27:10.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:10 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:10.480+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:27:10.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:10 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:10.480+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:27:10.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:10 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:10.480+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:27:10.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:10 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:10.480+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:27:10.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:10 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:10.480+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-10T08:26:39.289438+0000 front 2024-09-10T08:26:39.289561+0000 (oldest deadline 2024-09-10T08:27:05.189308+0000) 2024-09-10T08:27:10.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:10 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:10.480+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6820 osd.8 since back 2024-09-10T08:26:45.189734+0000 front 2024-09-10T08:26:45.189698+0000 (oldest deadline 2024-09-10T08:27:09.889794+0000) 2024-09-10T08:27:11.906 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:11 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:11.511+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:27:11.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:11 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:11.511+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:27:11.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:11 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:11.511+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:27:11.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:11 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:11.511+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:27:11.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:11 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:11.511+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:27:11.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:11 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:11.511+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:27:11.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:11 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:11.511+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-10T08:26:39.289438+0000 front 2024-09-10T08:26:39.289561+0000 (oldest deadline 2024-09-10T08:27:05.189308+0000) 2024-09-10T08:27:11.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:11 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:11.511+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6820 osd.8 since back 2024-09-10T08:26:45.189734+0000 front 2024-09-10T08:26:45.189698+0000 (oldest deadline 2024-09-10T08:27:09.889794+0000) 2024-09-10T08:27:12.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:12 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:12.463+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-09-10T08:25:38.378755+0000 front 2024-09-10T08:25:38.378778+0000 (oldest deadline 2024-09-10T08:26:06.579453+0000) 2024-09-10T08:27:12.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:12 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:12.463+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-10T08:25:46.580028+0000 front 2024-09-10T08:25:46.580169+0000 (oldest deadline 2024-09-10T08:26:12.479956+0000) 2024-09-10T08:27:12.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:12 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:12.463+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-09-10T08:25:58.682418+0000 front 2024-09-10T08:25:58.682392+0000 (oldest deadline 2024-09-10T08:26:21.582567+0000) 2024-09-10T08:27:12.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:12 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:12.463+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-10T08:26:14.785978+0000 front 2024-09-10T08:26:14.785922+0000 (oldest deadline 2024-09-10T08:26:40.685784+0000) 2024-09-10T08:27:12.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:12 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:12.463+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-09-10T08:26:24.087289+0000 front 2024-09-10T08:26:24.087271+0000 (oldest deadline 2024-09-10T08:26:49.387349+0000) 2024-09-10T08:27:12.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:12 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:12.463+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-09-10T08:26:29.387943+0000 front 2024-09-10T08:26:29.387992+0000 (oldest deadline 2024-09-10T08:26:58.188276+0000) 2024-09-10T08:27:12.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:12 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:12.463+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-10T08:26:39.289438+0000 front 2024-09-10T08:26:39.289561+0000 (oldest deadline 2024-09-10T08:27:05.189308+0000) 2024-09-10T08:27:12.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:12 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11[60209]: 2024-09-10T08:27:12.463+0000 7f941cef1640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.155:6820 osd.8 since back 2024-09-10T08:26:45.189734+0000 front 2024-09-10T08:26:45.189698+0000 (oldest deadline 2024-09-10T08:27:09.889794+0000) 2024-09-10T08:27:13.689 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:13 smithi155 podman[72059]: 2024-09-10 08:27:13.424617925 +0000 UTC m=+5.527179306 container died c9dcc9c9105eff07d4672773e8f5cc2a93cba7c44b40a28d276d96e7b41a47e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T08:27:14.048 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:13 smithi155 podman[72059]: 2024-09-10 08:27:13.789085727 +0000 UTC m=+5.891647110 container cleanup c9dcc9c9105eff07d4672773e8f5cc2a93cba7c44b40a28d276d96e7b41a47e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T08:27:14.048 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:13 smithi155 bash[72059]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11 2024-09-10T08:27:14.352 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:14 smithi155 podman[72074]: 2024-09-10 08:27:14.048582222 +0000 UTC m=+0.620570313 container remove c9dcc9c9105eff07d4672773e8f5cc2a93cba7c44b40a28d276d96e7b41a47e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T08:27:14.651 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:14 smithi155 podman[72164]: 2024-09-10 08:27:14.256187299 +0000 UTC m=+0.015601080 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:27:14.651 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:14 smithi155 podman[72164]: 2024-09-10 08:27:14.357604228 +0000 UTC m=+0.117018010 container create c13ff7d182f7e3c50e932bdedc7a146b2af33caf4dcc635e28763268a2c3b6aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T08:27:14.651 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:14 smithi155 podman[72164]: 2024-09-10 08:27:14.568227222 +0000 UTC m=+0.327641004 container init c13ff7d182f7e3c50e932bdedc7a146b2af33caf4dcc635e28763268a2c3b6aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS) 2024-09-10T08:27:14.651 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:14 smithi155 podman[72164]: 2024-09-10 08:27:14.571488116 +0000 UTC m=+0.330901897 container start c13ff7d182f7e3c50e932bdedc7a146b2af33caf4dcc635e28763268a2c3b6aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:27:14.651 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:14 smithi155 podman[72164]: 2024-09-10 08:27:14.651522331 +0000 UTC m=+0.410936131 container attach c13ff7d182f7e3c50e932bdedc7a146b2af33caf4dcc635e28763268a2c3b6aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:27:14.907 INFO:journalctl@ceph.osd.11.smithi155.stdout:Sep 10 08:27:14 smithi155 podman[72164]: 2024-09-10 08:27:14.768184716 +0000 UTC m=+0.527598507 container died c13ff7d182f7e3c50e932bdedc7a146b2af33caf4dcc635e28763268a2c3b6aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:27:15.337 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T08:27:15.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:27:15.378 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T08:27:15.378 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T08:27:15.378 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.a 2024-09-10T08:27:15.771 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 10 08:27:15 smithi002 systemd[1]: Stopping Ceph mds.a for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:27:16.270 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 10 08:27:15 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-a[68917]: 2024-09-10T08:27:15.854+0000 7fbe93400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:27:16.271 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 10 08:27:15 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-a[68917]: 2024-09-10T08:27:15.854+0000 7fbe93400640 -1 mds.a *** got signal Terminated *** 2024-09-10T08:27:19.395 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 10 08:27:19 smithi002 podman[125929]: 2024-09-10 08:27:19.121241197 +0000 UTC m=+3.456184394 container died aee775cee0bffc88e8395282ee3547ff92b24b44def03b815ca74831a5c99f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T08:27:19.650 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 10 08:27:19 smithi002 podman[125929]: 2024-09-10 08:27:19.634697435 +0000 UTC m=+3.969640630 container cleanup aee775cee0bffc88e8395282ee3547ff92b24b44def03b815ca74831a5c99f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T08:27:19.650 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 10 08:27:19 smithi002 bash[125929]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-a 2024-09-10T08:27:20.162 INFO:journalctl@ceph.mds.a.smithi002.stdout:Sep 10 08:27:19 smithi002 podman[125941]: 2024-09-10 08:27:19.892688692 +0000 UTC m=+0.761308973 container remove aee775cee0bffc88e8395282ee3547ff92b24b44def03b815ca74831a5c99f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T08:27:20.182 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T08:27:20.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:27:20.254 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T08:27:20.254 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T08:27:20.254 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.d 2024-09-10T08:27:20.520 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 10 08:27:20 smithi002 systemd[1]: Stopping Ceph mds.d for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:27:21.021 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 10 08:27:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-d[67753]: 2024-09-10T08:27:20.632+0000 7fe759c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:27:21.021 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 10 08:27:20 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-d[67753]: 2024-09-10T08:27:20.632+0000 7fe759c00640 -1 mds.d *** got signal Terminated *** 2024-09-10T08:27:24.964 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 10 08:27:24 smithi002 podman[126052]: 2024-09-10 08:27:24.643241303 +0000 UTC m=+4.110181227 container died ab11db6cfd54f8e49d332eb25a2e4493ddf0b759ced1b4fa482b615dba4c0000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T08:27:25.270 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 10 08:27:25 smithi002 podman[126052]: 2024-09-10 08:27:25.071195013 +0000 UTC m=+4.538134937 container cleanup ab11db6cfd54f8e49d332eb25a2e4493ddf0b759ced1b4fa482b615dba4c0000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T08:27:25.270 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 10 08:27:25 smithi002 bash[126052]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-d 2024-09-10T08:27:25.598 INFO:journalctl@ceph.mds.d.smithi002.stdout:Sep 10 08:27:25 smithi002 podman[126064]: 2024-09-10 08:27:25.316292349 +0000 UTC m=+0.658879787 container remove ab11db6cfd54f8e49d332eb25a2e4493ddf0b759ced1b4fa482b615dba4c0000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:27:25.625 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T08:27:25.695 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:27:25.696 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T08:27:25.696 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T08:27:25.696 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.g 2024-09-10T08:27:26.021 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 10 08:27:25 smithi002 systemd[1]: Stopping Ceph mds.g for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:27:26.520 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 10 08:27:26 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-g[68510]: 2024-09-10T08:27:26.169+0000 7fa413000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:27:26.521 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 10 08:27:26 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-g[68510]: 2024-09-10T08:27:26.169+0000 7fa413000640 -1 mds.g *** got signal Terminated *** 2024-09-10T08:27:30.516 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 10 08:27:30 smithi002 podman[126173]: 2024-09-10 08:27:30.182772233 +0000 UTC m=+4.205583658 container died 0cf6862e27b09d6c4e336848b745a403c6d715fd99dc5cb3f7b947c6d33c990a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-g, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:27:30.770 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 10 08:27:30 smithi002 podman[126173]: 2024-09-10 08:27:30.632360042 +0000 UTC m=+4.655171466 container cleanup 0cf6862e27b09d6c4e336848b745a403c6d715fd99dc5cb3f7b947c6d33c990a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T08:27:30.771 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 10 08:27:30 smithi002 bash[126173]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-g 2024-09-10T08:27:31.165 INFO:journalctl@ceph.mds.g.smithi002.stdout:Sep 10 08:27:30 smithi002 podman[126186]: 2024-09-10 08:27:30.891973408 +0000 UTC m=+0.694894642 container remove 0cf6862e27b09d6c4e336848b745a403c6d715fd99dc5cb3f7b947c6d33c990a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-g, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T08:27:31.192 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T08:27:31.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:27:31.260 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T08:27:31.260 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T08:27:31.261 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.j 2024-09-10T08:27:31.520 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:27:31 smithi002 systemd[1]: Stopping Ceph mds.j for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:27:32.020 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:27:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-j[67346]: 2024-09-10T08:27:31.655+0000 7f576e400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:27:32.021 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:27:31 smithi002 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-j[67346]: 2024-09-10T08:27:31.655+0000 7f576e400640 -1 mds.j *** got signal Terminated *** 2024-09-10T08:27:33.520 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:27:33 smithi002 podman[126294]: 2024-09-10 08:27:33.270766101 +0000 UTC m=+1.726507178 container died c5db02483c0ca20828e3bef2d03cebfc648df72cd97a2bdef9d32293b2d68444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T08:27:33.862 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:27:33 smithi002 podman[126294]: 2024-09-10 08:27:33.837237408 +0000 UTC m=+2.292978483 container cleanup c5db02483c0ca20828e3bef2d03cebfc648df72cd97a2bdef9d32293b2d68444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-10T08:27:33.862 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:27:33 smithi002 bash[126294]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-j 2024-09-10T08:27:34.300 INFO:journalctl@ceph.mds.j.smithi002.stdout:Sep 10 08:27:34 smithi002 podman[126306]: 2024-09-10 08:27:34.045931681 +0000 UTC m=+0.767327243 container remove c5db02483c0ca20828e3bef2d03cebfc648df72cd97a2bdef9d32293b2d68444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T08:27:34.319 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T08:27:34.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:27:34.387 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T08:27:34.388 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T08:27:34.388 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.b 2024-09-10T08:27:34.667 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:27:34 smithi107 systemd[1]: Stopping Ceph mds.b for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:27:34.933 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:27:34 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-b[64493]: 2024-09-10T08:27:34.698+0000 7fcd4fcc4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:27:34.933 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:27:34 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-b[64493]: 2024-09-10T08:27:34.698+0000 7fcd4fcc4640 -1 mds.b *** got signal Terminated *** 2024-09-10T08:27:37.502 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:27:37 smithi107 podman[70644]: 2024-09-10 08:27:37.178825242 +0000 UTC m=+2.598828778 container died 9f5ccb59747e4ad01c4b6d1bdaa205fa46bf08f2cd53e94cd4d41cd26f6467ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-b, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:27:37.794 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:27:37 smithi107 podman[70644]: 2024-09-10 08:27:37.593852334 +0000 UTC m=+3.013855869 container cleanup 9f5ccb59747e4ad01c4b6d1bdaa205fa46bf08f2cd53e94cd4d41cd26f6467ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T08:27:37.794 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 10 08:27:37 smithi107 bash[70644]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-b 2024-09-10T08:27:38.018 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T08:27:38.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:27:38.053 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T08:27:38.053 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T08:27:38.054 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.e 2024-09-10T08:27:38.355 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:27:38 smithi107 systemd[1]: Stopping Ceph mds.e for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:27:38.667 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:27:38 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-e[63891]: 2024-09-10T08:27:38.355+0000 7fac74da6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:27:38.667 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:27:38 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-e[63891]: 2024-09-10T08:27:38.355+0000 7fac74da6640 -1 mds.e *** got signal Terminated *** 2024-09-10T08:27:41.512 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:27:41 smithi107 podman[70777]: 2024-09-10 08:27:41.182351319 +0000 UTC m=+2.916400613 container died f1a76df045aaee3aefda7aeb6325b004ff8ad57b08e7bfda19b2096fe81a35f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T08:27:41.513 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:27:41 smithi107 podman[70777]: 2024-09-10 08:27:41.51330288 +0000 UTC m=+3.247352174 container cleanup f1a76df045aaee3aefda7aeb6325b004ff8ad57b08e7bfda19b2096fe81a35f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:27:41.917 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:27:41 smithi107 bash[70777]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-e 2024-09-10T08:27:41.917 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 10 08:27:41 smithi107 podman[70790]: 2024-09-10 08:27:41.689689492 +0000 UTC m=+0.504770947 container remove f1a76df045aaee3aefda7aeb6325b004ff8ad57b08e7bfda19b2096fe81a35f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-e, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T08:27:41.940 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T08:27:41.974 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:27:41.974 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T08:27:41.974 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T08:27:41.974 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.h 2024-09-10T08:27:42.292 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:27:42 smithi107 systemd[1]: Stopping Ceph mds.h for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:27:42.667 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:27:42 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-h[64193]: 2024-09-10T08:27:42.292+0000 7f311ec45640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:27:42.667 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:27:42 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-h[64193]: 2024-09-10T08:27:42.292+0000 7f311ec45640 -1 mds.h *** got signal Terminated *** 2024-09-10T08:27:45.450 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:27:45 smithi107 podman[70898]: 2024-09-10 08:27:45.183851683 +0000 UTC m=+2.992541756 container died ebd0d0571fa618e43165d7594aba409190be4d4c2234cc2dde79aca0b98c0574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T08:27:45.800 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:27:45 smithi107 podman[70898]: 2024-09-10 08:27:45.55821904 +0000 UTC m=+3.366909111 container cleanup ebd0d0571fa618e43165d7594aba409190be4d4c2234cc2dde79aca0b98c0574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-h, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T08:27:45.800 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 10 08:27:45 smithi107 bash[70898]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-h 2024-09-10T08:27:45.934 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T08:27:45.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:27:45.968 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T08:27:45.968 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T08:27:45.969 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.k 2024-09-10T08:27:46.417 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:27:46 smithi107 systemd[1]: Stopping Ceph mds.k for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:27:46.417 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:27:46 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-k[63578]: 2024-09-10T08:27:46.293+0000 7f575bd50640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:27:46.417 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:27:46 smithi107 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-k[63578]: 2024-09-10T08:27:46.293+0000 7f575bd50640 -1 mds.k *** got signal Terminated *** 2024-09-10T08:27:50.060 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:27:49 smithi107 podman[71017]: 2024-09-10 08:27:49.789230667 +0000 UTC m=+3.600787098 container died 0740ab8b90aeaa9ba04163831fbe04bbe0a4a45c547d2ff6dd19bf72f901c7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T08:27:50.386 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:27:50 smithi107 podman[71017]: 2024-09-10 08:27:50.144152239 +0000 UTC m=+3.955708670 container cleanup 0740ab8b90aeaa9ba04163831fbe04bbe0a4a45c547d2ff6dd19bf72f901c7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-k, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T08:27:50.386 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 10 08:27:50 smithi107 bash[71017]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-k 2024-09-10T08:27:50.515 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T08:27:50.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:27:50.547 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T08:27:50.548 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T08:27:50.548 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.c 2024-09-10T08:27:50.854 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:27:50 smithi155 systemd[1]: Stopping Ceph mds.c for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:27:51.156 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:27:50 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-c[65139]: 2024-09-10T08:27:50.854+0000 7fd566034640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:27:51.156 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:27:50 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-c[65139]: 2024-09-10T08:27:50.854+0000 7fd566034640 -1 mds.c *** got signal Terminated *** 2024-09-10T08:27:53.454 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:27:53 smithi155 podman[72266]: 2024-09-10 08:27:53.179729539 +0000 UTC m=+2.439186491 container died 4ad1423fd2e4a8578b0c083e331f057e6d08445d671976c966ae210f548d1522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T08:27:53.804 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:27:53 smithi155 podman[72266]: 2024-09-10 08:27:53.570148586 +0000 UTC m=+2.829605538 container cleanup 4ad1423fd2e4a8578b0c083e331f057e6d08445d671976c966ae210f548d1522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:27:53.804 INFO:journalctl@ceph.mds.c.smithi155.stdout:Sep 10 08:27:53 smithi155 bash[72266]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-c 2024-09-10T08:27:53.945 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T08:27:53.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:27:53.981 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T08:27:53.981 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T08:27:53.981 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.f 2024-09-10T08:27:54.407 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:27:54 smithi155 systemd[1]: Stopping Ceph mds.f for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:27:54.407 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:27:54 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-f[64840]: 2024-09-10T08:27:54.299+0000 7f945e1dc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:27:54.407 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:27:54 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-f[64840]: 2024-09-10T08:27:54.299+0000 7f945e1dc640 -1 mds.f *** got signal Terminated *** 2024-09-10T08:27:57.465 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:27:57 smithi155 podman[72400]: 2024-09-10 08:27:57.191487458 +0000 UTC m=+2.993221176 container died 71aa353b688ce3828cf1a49f8998f51917f694419b866baeb4af6ddf6f13b5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-f, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T08:27:57.874 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:27:57 smithi155 podman[72400]: 2024-09-10 08:27:57.565641075 +0000 UTC m=+3.367374785 container cleanup 71aa353b688ce3828cf1a49f8998f51917f694419b866baeb4af6ddf6f13b5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:27:57.874 INFO:journalctl@ceph.mds.f.smithi155.stdout:Sep 10 08:27:57 smithi155 bash[72400]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-f 2024-09-10T08:27:58.048 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T08:27:58.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:27:58.082 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T08:27:58.082 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T08:27:58.082 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.i 2024-09-10T08:27:58.402 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:27:58 smithi155 systemd[1]: Stopping Ceph mds.i for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:27:58.656 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:27:58 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-i[65440]: 2024-09-10T08:27:58.402+0000 7f63cc0c6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:27:58.657 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:27:58 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-i[65440]: 2024-09-10T08:27:58.402+0000 7f63cc0c6640 -1 mds.i *** got signal Terminated *** 2024-09-10T08:28:01.521 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:28:01 smithi155 podman[72521]: 2024-09-10 08:28:01.189522023 +0000 UTC m=+2.889909857 container died af41c8137399c635edb08f7b28270e4bca75fe268f749364b712d821b35574d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:28:01.906 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:28:01 smithi155 podman[72521]: 2024-09-10 08:28:01.660522663 +0000 UTC m=+3.360910495 container cleanup af41c8137399c635edb08f7b28270e4bca75fe268f749364b712d821b35574d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T08:28:01.907 INFO:journalctl@ceph.mds.i.smithi155.stdout:Sep 10 08:28:01 smithi155 bash[72521]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-i 2024-09-10T08:28:02.093 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T08:28:02.126 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:28:02.126 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T08:28:02.127 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T08:28:02.127 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.l 2024-09-10T08:28:02.439 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:28:02 smithi155 systemd[1]: Stopping Ceph mds.l for 26ec64b0-6f4a-11ef-bcea-c7b262605968... 2024-09-10T08:28:02.907 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:28:02 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-l[64525]: 2024-09-10T08:28:02.439+0000 7f68dd29e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:28:02.907 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:28:02 smithi155 ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-l[64525]: 2024-09-10T08:28:02.439+0000 7f68dd29e640 -1 mds.l *** got signal Terminated *** 2024-09-10T08:28:06.714 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:28:06 smithi155 podman[72641]: 2024-09-10 08:28:06.447243675 +0000 UTC m=+4.102331053 container died 23f2a8c101e06a22a8ddfeb18c36275c3d849c119731fac590f68180e57a7963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T08:28:07.157 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:28:06 smithi155 podman[72641]: 2024-09-10 08:28:06.789382943 +0000 UTC m=+4.444470306 container cleanup 23f2a8c101e06a22a8ddfeb18c36275c3d849c119731fac590f68180e57a7963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-l, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:28:07.157 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:28:06 smithi155 bash[72641]: ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-l 2024-09-10T08:28:07.157 INFO:journalctl@ceph.mds.l.smithi155.stdout:Sep 10 08:28:06 smithi155 podman[72653]: 2024-09-10 08:28:06.957015291 +0000 UTC m=+0.498607953 container remove 23f2a8c101e06a22a8ddfeb18c36275c3d849c119731fac590f68180e57a7963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:28:07.211 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26ec64b0-6f4a-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T08:28:07.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:28:07.243 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T08:28:07.244 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T08:28:07.528 INFO:teuthology.orchestra.run.smithi002.stdout:Deleting cluster with fsid: 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T08:28:11.464 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T08:28:11.665 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T08:28:14.125 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T08:28:14.324 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T08:28:16.688 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:28:16.766 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:28:16.801 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:28:16.832 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T08:28:17.469 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898886/remote/smithi002/crash 2024-09-10T08:28:17.977 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/crash -- . 2024-09-10T08:28:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:tar: /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T08:28:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:tar: Error is not recoverable: exiting now 2024-09-10T08:28:18.415 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898886/remote/smithi107/crash 2024-09-10T08:28:18.716 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/crash -- . 2024-09-10T08:28:18.755 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T08:28:18.755 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-09-10T08:28:19.191 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898886/remote/smithi155/crash 2024-09-10T08:28:19.370 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/crash -- . 2024-09-10T08:28:19.418 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T08:28:19.418 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2024-09-10T08:28:19.419 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T08:28:19.420 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T08:28:19.511 INFO:tasks.cephadm:Compressing logs... 2024-09-10T08:28:19.512 DEBUG:teuthology.orchestra.run.smithi002:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T08:28:19.517 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T08:28:19.521 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T08:28:19.547 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T08:28:19.547 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T08:28:19.547 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T08:28:19.549 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T08:28:19.549 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T08:28:19.549 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T08:28:19.550 INFO:teuthology.orchestra.run.smithi155.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T08:28:19.550 INFO:teuthology.orchestra.run.smithi155.stderr:: No such file or directory 2024-09-10T08:28:19.550 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T08:28:19.551 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T08:28:19.551 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T08:28:19.551 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T08:28:19.551 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T08:28:19.553 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-10T08:28:19.554 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T08:28:19.554 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T08:28:19.554 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T08:28:19.560 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T08:28:19.560 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T08:28:19.560 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.cephadm.log: 83.0% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T08:28:19.560 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.log 2024-09-10T08:28:19.560 INFO:teuthology.orchestra.run.smithi107.stderr: 94.5% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T08:28:19.562 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T08:28:19.562 INFO:teuthology.orchestra.run.smithi155.stderr: 82.9% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T08:28:19.562 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T08:28:19.563 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.2.log: --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.log 2024-09-10T08:28:19.563 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-10T08:28:19.563 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.log: 90.4% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T08:28:19.563 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T08:28:19.563 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.l.log: 87.8% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T08:28:19.563 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T08:28:19.563 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.8.log: 94.6% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T08:28:19.563 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-10T08:28:19.564 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mgr.y.log: 91.0% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T08:28:19.564 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mgr.z.log: 91.0% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T08:28:19.566 INFO:teuthology.orchestra.run.smithi107.stderr: 93.2% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T08:28:19.566 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T08:28:19.566 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T08:28:19.566 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T08:28:19.569 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mon.b.log: 87.8% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T08:28:19.570 INFO:teuthology.orchestra.run.smithi002.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-10T08:28:19.570 INFO:teuthology.orchestra.run.smithi002.stderr:: No such file or directory 2024-09-10T08:28:19.570 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T08:28:19.571 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T08:28:19.573 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T08:28:19.573 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T08:28:19.573 INFO:teuthology.orchestra.run.smithi002.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-10T08:28:19.573 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T08:28:19.574 INFO:teuthology.orchestra.run.smithi155.stderr: 93.3% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T08:28:19.574 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T08:28:19.574 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T08:28:19.575 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T08:28:19.575 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T08:28:19.576 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T08:28:19.577 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T08:28:19.577 INFO:teuthology.orchestra.run.smithi002.stderr: 90.2% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T08:28:19.578 INFO:teuthology.orchestra.run.smithi002.stderr: 82.8% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T08:28:19.578 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T08:28:19.579 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.log 2024-09-10T08:28:19.584 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.g.log: /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.6.log: /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T08:28:19.584 INFO:teuthology.orchestra.run.smithi002.stderr: 94.5% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T08:28:19.584 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T08:28:19.587 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T08:28:19.587 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mgr.x.log: 90.9% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T08:28:19.590 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T08:28:19.590 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T08:28:19.591 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T08:28:19.591 INFO:teuthology.orchestra.run.smithi002.stderr: 94.6% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-10T08:28:19.592 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T08:28:19.592 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T08:28:19.598 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T08:28:19.599 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T08:28:19.609 INFO:teuthology.orchestra.run.smithi002.stderr: 94.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T08:28:19.657 INFO:teuthology.orchestra.run.smithi107.stderr: 94.6% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T08:28:19.661 INFO:teuthology.orchestra.run.smithi155.stderr: 94.7% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T08:28:19.677 INFO:teuthology.orchestra.run.smithi155.stderr: 92.9% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T08:28:19.691 INFO:teuthology.orchestra.run.smithi107.stderr: 92.9% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T08:28:19.881 INFO:teuthology.orchestra.run.smithi155.stderr: 94.0% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T08:28:19.963 INFO:teuthology.orchestra.run.smithi002.stderr: 90.6% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T08:28:20.095 INFO:teuthology.orchestra.run.smithi155.stderr: 91.5% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T08:28:20.236 INFO:teuthology.orchestra.run.smithi107.stderr: 91.5% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T08:28:20.500 INFO:teuthology.orchestra.run.smithi002.stderr: 92.5% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T08:28:20.586 INFO:teuthology.orchestra.run.smithi155.stderr: 91.9% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T08:28:20.632 INFO:teuthology.orchestra.run.smithi002.stderr: 92.3% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T08:28:20.734 INFO:teuthology.orchestra.run.smithi002.stderr: 92.2% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T08:28:20.806 INFO:teuthology.orchestra.run.smithi107.stderr: 92.1% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T08:28:20.922 INFO:teuthology.orchestra.run.smithi107.stderr: 92.0% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T08:28:21.048 INFO:teuthology.orchestra.run.smithi002.stderr: 90.3% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T08:28:21.085 INFO:teuthology.orchestra.run.smithi155.stderr: 92.9% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T08:28:21.346 INFO:teuthology.orchestra.run.smithi002.stderr: 93.0% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T08:28:21.349 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-10T08:28:21.349 INFO:teuthology.orchestra.run.smithi002.stderr:real 0m1.803s 2024-09-10T08:28:21.349 INFO:teuthology.orchestra.run.smithi002.stderr:user 0m6.283s 2024-09-10T08:28:21.349 INFO:teuthology.orchestra.run.smithi002.stderr:sys 0m0.607s 2024-09-10T08:28:21.388 INFO:teuthology.orchestra.run.smithi155.stderr: 91.6% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T08:28:21.834 INFO:teuthology.orchestra.run.smithi107.stderr: 93.0% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T08:28:21.879 INFO:teuthology.orchestra.run.smithi107.stderr: 92.3% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T08:28:21.981 INFO:teuthology.orchestra.run.smithi107.stderr: 92.1% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T08:28:21.983 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-09-10T08:28:21.983 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m2.447s 2024-09-10T08:28:21.983 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m10.240s 2024-09-10T08:28:21.983 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m0.370s 2024-09-10T08:28:22.146 INFO:teuthology.orchestra.run.smithi155.stderr: 92.1% -- replaced with /var/log/ceph/26ec64b0-6f4a-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T08:28:22.148 INFO:teuthology.orchestra.run.smithi155.stderr: 2024-09-10T08:28:22.148 INFO:teuthology.orchestra.run.smithi155.stderr:real 0m2.610s 2024-09-10T08:28:22.148 INFO:teuthology.orchestra.run.smithi155.stderr:user 0m7.798s 2024-09-10T08:28:22.149 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m0.315s 2024-09-10T08:28:22.149 INFO:tasks.cephadm:Archiving logs... 2024-09-10T08:28:22.149 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898886/remote/smithi002/log 2024-09-10T08:28:22.150 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T08:28:23.095 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898886/remote/smithi107/log 2024-09-10T08:28:23.096 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T08:28:24.542 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898886/remote/smithi155/log 2024-09-10T08:28:24.543 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T08:28:25.753 INFO:tasks.cephadm:Removing cluster... 2024-09-10T08:28:25.753 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 --force 2024-09-10T08:28:26.036 INFO:teuthology.orchestra.run.smithi002.stdout:Deleting cluster with fsid: 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T08:28:26.477 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 --force 2024-09-10T08:28:26.678 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T08:28:26.950 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 26ec64b0-6f4a-11ef-bcea-c7b262605968 --force 2024-09-10T08:28:27.153 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 26ec64b0-6f4a-11ef-bcea-c7b262605968 2024-09-10T08:28:27.417 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T08:28:27.418 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T08:28:27.458 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T08:28:27.474 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T08:28:27.489 INFO:tasks.cephadm:Teardown complete 2024-09-10T08:28:27.490 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T08:28:27.501 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T08:28:27.501 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T08:28:27.506 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T08:28:27.517 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T08:28:27.633 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T08:28:27.643 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T08:28:27.644 DEBUG:teuthology.orchestra.run.smithi002:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:28:27.648 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:28:27.651 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:28:27.663 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:28:27.665 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:28:27.668 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:28:27.668 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-10T08:28:27.668 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 54 +1333us[+1336us] +/- 35ms 2024-09-10T08:28:27.668 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 +1954us[+1958us] +/- 58ms 2024-09-10T08:28:27.668 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 -5427us[-5423us] +/- 67ms 2024-09-10T08:28:27.668 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:28:27.669 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:28:27.669 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-09-10T08:28:27.669 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 53 +1316us[+1316us] +/- 35ms 2024-09-10T08:28:27.669 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 +1947us[+2145us] +/- 58ms 2024-09-10T08:28:27.669 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -5437us[-5437us] +/- 67ms 2024-09-10T08:28:27.670 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:28:27.680 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:28:27.688 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:28:27.688 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-09-10T08:28:27.688 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 55 +1401us[+1377us] +/- 35ms 2024-09-10T08:28:27.688 INFO:teuthology.orchestra.run.smithi002.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 +2030us[+2008us] +/- 58ms 2024-09-10T08:28:27.688 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 -5366us[-5389us] +/- 67ms 2024-09-10T08:28:27.688 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:28:27.689 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T08:28:27.706 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T08:28:27.706 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T08:28:27.752 DEBUG:teuthology.orchestra.run.smithi002:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T08:28:27.899 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T08:28:27.968 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T08:28:28.035 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:28:28.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:28:28.098 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:28:28.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:28:28.127 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:28:28.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:28:28.155 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T08:28:28.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T08:28:28.292 INFO:teuthology.task.internal:Duration was 2348.826273 seconds 2024-09-10T08:28:28.293 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T08:28:28.381 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T08:28:28.382 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T08:28:28.386 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T08:28:28.388 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T08:28:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:28:28.427 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:28:28.470 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:28:28.955 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T08:28:28.955 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi002.front.sepia.ceph.com 2024-09-10T08:28:28.955 DEBUG:teuthology.orchestra.run.smithi002:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T08:28:29.005 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-09-10T08:28:29.006 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T08:28:29.032 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-09-10T08:28:29.033 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T08:28:29.065 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T08:28:29.065 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T08:28:29.070 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T08:28:29.075 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T08:28:29.132 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T08:28:29.132 DEBUG:teuthology.orchestra.run.smithi002:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T08:28:29.136 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T08:28:29.143 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T08:28:30.871 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T08:28:30.883 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T08:28:30.884 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T08:28:30.940 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T08:28:30.971 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T08:28:31.002 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T08:28:31.015 DEBUG:teuthology.orchestra.run.smithi002:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:28:31.019 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:28:31.021 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:28:31.046 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-09-10T08:28:31.069 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-09-10T08:28:31.086 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = core 2024-09-10T08:28:31.116 DEBUG:teuthology.orchestra.run.smithi002:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:28:31.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:28:31.150 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:28:31.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:28:31.167 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:28:31.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:28:31.184 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T08:28:31.203 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T08:28:31.204 DEBUG:teuthology.misc:Transferring archived files from smithi002:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898886/remote/smithi002 2024-09-10T08:28:31.204 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T08:28:31.330 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898886/remote/smithi107 2024-09-10T08:28:31.330 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T08:28:31.368 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898886/remote/smithi155 2024-09-10T08:28:31.369 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T08:28:31.408 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T08:28:31.409 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T08:28:31.414 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T08:28:31.417 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T08:28:31.459 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T08:28:31.483 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T08:28:31.483 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T08:28:31.534 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T08:28:31.534 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T08:28:31.538 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T08:28:31.541 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T08:28:31.554 INFO:teuthology.orchestra.run.smithi107.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 08:28 /home/ubuntu/cephtest 2024-09-10T08:28:31.557 INFO:teuthology.orchestra.run.smithi155.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 08:28 /home/ubuntu/cephtest 2024-09-10T08:28:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 08:28 /home/ubuntu/cephtest 2024-09-10T08:28:31.571 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T08:28:31.703 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T08:28:31.812 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2348.8262734413147 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-10T08:28:31.813 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T08:28:31.887 INFO:teuthology.run:pass